Merge tag 'armsoc-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc
Pull ARM SoC fixes from Arnd Bergmann:
"This is a first set of bug fixes on top of what was merged for 4.7.
Two patches for lpc32xx address a harmless build warning that was just
introduced, one patch for the mediatek soc driver fixes a warning for
arm64, and the pxa changes are minor cleanups that should have been
part of the original pull requests but that I forgot to apply to the
cleanup-fixes branch earlier"
* tag 'armsoc-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc:
ARM: lpc32xx: fix NR_IRQS confict
ARM: lpc32xx: remove legacy irq controller driver
soc: mtk-pmic-wrap: avoid integer overflow warning
ARM: pxa: Remove CLK_IS_ROOT
ARM: pxa: activate pinctrl for device-tree machines
diff --git a/CREDITS b/CREDITS
index 4312cd0..0f0bf22 100644
--- a/CREDITS
+++ b/CREDITS
@@ -768,6 +768,7 @@
D: Former security contact point (please use vendor-sec@lst.de)
D: ex 2.2 maintainer
D: 2.1.x modular sound
+D: Assigned major/minor numbers maintainer at lanana.org
S: c/o Red Hat UK Ltd
S: Alexandra House
S: Alexandra Terrace
diff --git a/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-savu b/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-savu
index f1e02a9..99fda67 100644
--- a/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-savu
+++ b/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-savu
@@ -3,9 +3,10 @@
Contact: Stefan Achatz <erazor_de@users.sourceforge.net>
Description: The mouse can store 5 profiles which can be switched by the
press of a button. A profile is split into general settings and
- button settings. buttons holds informations about button layout.
- When written, this file lets one write the respective profile
- buttons to the mouse. The data has to be 47 bytes long.
+ button settings. The buttons variable holds information about
+ button layout. When written, this file lets one write the
+ respective profile buttons to the mouse. The data has to be
+ 47 bytes long.
The mouse will reject invalid data.
Which profile to write is determined by the profile number
contained in the data.
@@ -26,8 +27,8 @@
Contact: Stefan Achatz <erazor_de@users.sourceforge.net>
Description: The mouse can store 5 profiles which can be switched by the
press of a button. A profile is split into general settings and
- button settings. profile holds informations like resolution, sensitivity
- and light effects.
+ button settings. A profile holds information like resolution,
+ sensitivity and light effects.
When written, this file lets one write the respective profile
settings back to the mouse. The data has to be 43 bytes long.
The mouse will reject invalid data.
diff --git a/Documentation/ABI/testing/sysfs-block-zram b/Documentation/ABI/testing/sysfs-block-zram
index 2e69e83..4518d30 100644
--- a/Documentation/ABI/testing/sysfs-block-zram
+++ b/Documentation/ABI/testing/sysfs-block-zram
@@ -166,3 +166,12 @@
The mm_stat file is read-only and represents device's mm
statistics (orig_data_size, compr_data_size, etc.) in a format
similar to block layer statistics file format.
+
+What: /sys/block/zram<id>/debug_stat
+Date: July 2016
+Contact: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
+Description:
+ The debug_stat file is read-only and represents various
+ device's debugging info useful for kernel developers. Its
+ format is not documented intentionally and may change
+ anytime without any notice.
diff --git a/Documentation/ABI/testing/sysfs-bus-coresight-devices-etb10 b/Documentation/ABI/testing/sysfs-bus-coresight-devices-etb10
index 4b8d6ec..b5f52608 100644
--- a/Documentation/ABI/testing/sysfs-bus-coresight-devices-etb10
+++ b/Documentation/ABI/testing/sysfs-bus-coresight-devices-etb10
@@ -6,13 +6,6 @@
source for a single sink.
ex: echo 1 > /sys/bus/coresight/devices/20010000.etb/enable_sink
-What: /sys/bus/coresight/devices/<memory_map>.etb/status
-Date: November 2014
-KernelVersion: 3.19
-Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
-Description: (R) List various control and status registers. The specific
- layout and content is driver specific.
-
What: /sys/bus/coresight/devices/<memory_map>.etb/trigger_cntr
Date: November 2014
KernelVersion: 3.19
@@ -22,3 +15,65 @@
following the trigger event. The number of 32-bit words written
into the Trace RAM following the trigger event is equal to the
value stored in this register+1 (from ARM ETB-TRM).
+
+What: /sys/bus/coresight/devices/<memory_map>.etb/mgmt/rdp
+Date: March 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (R) Defines the depth, in words, of the trace RAM in powers of
+ 2. The value is read directly from HW register RDP, 0x004.
+
+What: /sys/bus/coresight/devices/<memory_map>.etb/mgmt/sts
+Date: March 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (R) Shows the value held by the ETB status register. The value
+ is read directly from HW register STS, 0x00C.
+
+What: /sys/bus/coresight/devices/<memory_map>.etb/mgmt/rrp
+Date: March 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (R) Shows the value held by the ETB RAM Read Pointer register
+ that is used to read entries from the Trace RAM over the APB
+ interface. The value is read directly from HW register RRP,
+ 0x014.
+
+What: /sys/bus/coresight/devices/<memory_map>.etb/mgmt/rwp
+Date: March 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (R) Shows the value held by the ETB RAM Write Pointer register
+ that is used to sets the write pointer to write entries from
+ the CoreSight bus into the Trace RAM. The value is read directly
+ from HW register RWP, 0x018.
+
+What: /sys/bus/coresight/devices/<memory_map>.etb/mgmt/trg
+Date: March 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (R) Similar to "trigger_cntr" above except that this value is
+ read directly from HW register TRG, 0x01C.
+
+What: /sys/bus/coresight/devices/<memory_map>.etb/mgmt/ctl
+Date: March 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (R) Shows the value held by the ETB Control register. The value
+ is read directly from HW register CTL, 0x020.
+
+What: /sys/bus/coresight/devices/<memory_map>.etb/mgmt/ffsr
+Date: March 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (R) Shows the value held by the ETB Formatter and Flush Status
+ register. The value is read directly from HW register FFSR,
+ 0x300.
+
+What: /sys/bus/coresight/devices/<memory_map>.etb/mgmt/ffcr
+Date: March 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (R) Shows the value held by the ETB Formatter and Flush Control
+ register. The value is read directly from HW register FFCR,
+ 0x304.
diff --git a/Documentation/ABI/testing/sysfs-bus-coresight-devices-etm4x b/Documentation/ABI/testing/sysfs-bus-coresight-devices-etm4x
index 2355ed8..36258bc 100644
--- a/Documentation/ABI/testing/sysfs-bus-coresight-devices-etm4x
+++ b/Documentation/ABI/testing/sysfs-bus-coresight-devices-etm4x
@@ -359,6 +359,19 @@
Description: (R) Print the content of the Peripheral ID3 Register
(0xFEC). The value is taken directly from the HW.
+What: /sys/bus/coresight/devices/<memory_map>.etm/mgmt/trcconfig
+Date: February 2016
+KernelVersion: 4.07
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (R) Print the content of the trace configuration register
+ (0x010) as currently set by SW.
+
+What: /sys/bus/coresight/devices/<memory_map>.etm/mgmt/trctraceid
+Date: February 2016
+KernelVersion: 4.07
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (R) Print the content of the trace ID register (0x040).
+
What: /sys/bus/coresight/devices/<memory_map>.etm/trcidr/trcidr0
Date: April 2015
KernelVersion: 4.01
diff --git a/Documentation/ABI/testing/sysfs-bus-coresight-devices-stm b/Documentation/ABI/testing/sysfs-bus-coresight-devices-stm
new file mode 100644
index 0000000..1dffabe
--- /dev/null
+++ b/Documentation/ABI/testing/sysfs-bus-coresight-devices-stm
@@ -0,0 +1,53 @@
+What: /sys/bus/coresight/devices/<memory_map>.stm/enable_source
+Date: April 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (RW) Enable/disable tracing on this specific trace macrocell.
+ Enabling the trace macrocell implies it has been configured
+ properly and a sink has been identified for it. The path
+ of coresight components linking the source to the sink is
+ configured and managed automatically by the coresight framework.
+
+What: /sys/bus/coresight/devices/<memory_map>.stm/hwevent_enable
+Date: April 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (RW) Provides access to the HW event enable register, used in
+ conjunction with HW event bank select register.
+
+What: /sys/bus/coresight/devices/<memory_map>.stm/hwevent_select
+Date: April 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (RW) Gives access to the HW event block select register
+ (STMHEBSR) in order to configure up to 256 channels. Used in
+ conjunction with "hwevent_enable" register as described above.
+
+What: /sys/bus/coresight/devices/<memory_map>.stm/port_enable
+Date: April 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (RW) Provides access to the stimulus port enable register
+ (STMSPER). Used in conjunction with "port_select" described
+ below.
+
+What: /sys/bus/coresight/devices/<memory_map>.stm/port_select
+Date: April 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (RW) Used to determine which bank of stimulus port bit in
+ register STMSPER (see above) apply to.
+
+What: /sys/bus/coresight/devices/<memory_map>.stm/status
+Date: April 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (R) List various control and status registers. The specific
+ layout and content is driver specific.
+
+What: /sys/bus/coresight/devices/<memory_map>.stm/traceid
+Date: April 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (RW) Holds the trace ID that will appear in the trace stream
+ coming from this trace entity.
diff --git a/Documentation/ABI/testing/sysfs-bus-coresight-devices-tmc b/Documentation/ABI/testing/sysfs-bus-coresight-devices-tmc
index f38cded5..4fe677e 100644
--- a/Documentation/ABI/testing/sysfs-bus-coresight-devices-tmc
+++ b/Documentation/ABI/testing/sysfs-bus-coresight-devices-tmc
@@ -6,3 +6,80 @@
formatter after a defined number of words have been stored
following the trigger event. Additional interface for this
driver are expected to be added as it matures.
+
+What: /sys/bus/coresight/devices/<memory_map>.tmc/mgmt/rsz
+Date: March 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (R) Defines the size, in 32-bit words, of the local RAM buffer.
+ The value is read directly from HW register RSZ, 0x004.
+
+What: /sys/bus/coresight/devices/<memory_map>.tmc/mgmt/sts
+Date: March 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (R) Shows the value held by the TMC status register. The value
+ is read directly from HW register STS, 0x00C.
+
+What: /sys/bus/coresight/devices/<memory_map>.tmc/mgmt/rrp
+Date: March 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (R) Shows the value held by the TMC RAM Read Pointer register
+ that is used to read entries from the Trace RAM over the APB
+ interface. The value is read directly from HW register RRP,
+ 0x014.
+
+What: /sys/bus/coresight/devices/<memory_map>.tmc/mgmt/rwp
+Date: March 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (R) Shows the value held by the TMC RAM Write Pointer register
+ that is used to sets the write pointer to write entries from
+ the CoreSight bus into the Trace RAM. The value is read directly
+ from HW register RWP, 0x018.
+
+What: /sys/bus/coresight/devices/<memory_map>.tmc/mgmt/trg
+Date: March 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (R) Similar to "trigger_cntr" above except that this value is
+ read directly from HW register TRG, 0x01C.
+
+What: /sys/bus/coresight/devices/<memory_map>.tmc/mgmt/ctl
+Date: March 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (R) Shows the value held by the TMC Control register. The value
+ is read directly from HW register CTL, 0x020.
+
+What: /sys/bus/coresight/devices/<memory_map>.tmc/mgmt/ffsr
+Date: March 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (R) Shows the value held by the TMC Formatter and Flush Status
+ register. The value is read directly from HW register FFSR,
+ 0x300.
+
+What: /sys/bus/coresight/devices/<memory_map>.tmc/mgmt/ffcr
+Date: March 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (R) Shows the value held by the TMC Formatter and Flush Control
+ register. The value is read directly from HW register FFCR,
+ 0x304.
+
+What: /sys/bus/coresight/devices/<memory_map>.tmc/mgmt/mode
+Date: March 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (R) Shows the value held by the TMC Mode register, which
+ indicate the mode the device has been configured to enact. The
+ The value is read directly from the MODE register, 0x028.
+
+What: /sys/bus/coresight/devices/<memory_map>.tmc/mgmt/devid
+Date: March 2016
+KernelVersion: 4.7
+Contact: Mathieu Poirier <mathieu.poirier@linaro.org>
+Description: (R) Indicates the capabilities of the Coresight TMC.
+ The value is read directly from the DEVID register, 0xFC8,
diff --git a/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_24x7 b/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_24x7
index f893337..ec27c6c 100644
--- a/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_24x7
+++ b/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_24x7
@@ -4,7 +4,7 @@
Description:
Provides access to the binary "24x7 catalog" provided by the
hypervisor on POWER7 and 8 systems. This catalog lists events
- avaliable from the powerpc "hv_24x7" pmu. Its format is
+ available from the powerpc "hv_24x7" pmu. Its format is
documented here:
https://raw.githubusercontent.com/jmesmon/catalog-24x7/master/hv-24x7-catalog.h
diff --git a/Documentation/ABI/testing/sysfs-bus-iio b/Documentation/ABI/testing/sysfs-bus-iio
index 3c66248..df44998 100644
--- a/Documentation/ABI/testing/sysfs-bus-iio
+++ b/Documentation/ABI/testing/sysfs-bus-iio
@@ -1233,7 +1233,7 @@
Contact: linux-iio@vger.kernel.org
Description:
Proximity measurement indicating that some
- object is near the sensor, usually be observing
+ object is near the sensor, usually by observing
reflectivity of infrared or ultrasound emitted.
Often these sensors are unit less and as such conversion
to SI units is not possible. Higher proximity measurements
@@ -1255,12 +1255,23 @@
What: /sys/.../iio:deviceX/in_intensityY_raw
What: /sys/.../iio:deviceX/in_intensityY_ir_raw
What: /sys/.../iio:deviceX/in_intensityY_both_raw
+What: /sys/.../iio:deviceX/in_intensityY_uv_raw
KernelVersion: 3.4
Contact: linux-iio@vger.kernel.org
Description:
Unit-less light intensity. Modifiers both and ir indicate
that measurements contains visible and infrared light
- components or just infrared light, respectively.
+ components or just infrared light, respectively. Modifier uv indicates
+ that measurements contain ultraviolet light components.
+
+What: /sys/.../iio:deviceX/in_uvindex_input
+KernelVersion: 4.6
+Contact: linux-iio@vger.kernel.org
+Description:
+ UV light intensity index measuring the human skin's response to
+ different wavelength of sunlight weighted according to the
+ standardised CIE Erythemal Action Spectrum. UV index values range
+ from 0 (low) to >=11 (extreme).
What: /sys/.../iio:deviceX/in_intensity_red_integration_time
What: /sys/.../iio:deviceX/in_intensity_green_integration_time
@@ -1501,3 +1512,56 @@
Description:
Raw (unscaled no offset etc.) pH reading of a substance as a negative
base-10 logarithm of hydrodium ions in a litre of water.
+
+What: /sys/bus/iio/devices/iio:deviceX/mount_matrix
+What: /sys/bus/iio/devices/iio:deviceX/in_mount_matrix
+What: /sys/bus/iio/devices/iio:deviceX/out_mount_matrix
+What: /sys/bus/iio/devices/iio:deviceX/in_anglvel_mount_matrix
+What: /sys/bus/iio/devices/iio:deviceX/in_accel_mount_matrix
+KernelVersion: 4.6
+Contact: linux-iio@vger.kernel.org
+Description:
+ Mounting matrix for IIO sensors. This is a rotation matrix which
+ informs userspace about sensor chip's placement relative to the
+ main hardware it is mounted on.
+ Main hardware placement is defined according to the local
+ reference frame related to the physical quantity the sensor
+ measures.
+ Given that the rotation matrix is defined in a board specific
+ way (platform data and / or device-tree), the main hardware
+ reference frame definition is left to the implementor's choice
+ (see below for a magnetometer example).
+ Applications should apply this rotation matrix to samples so
+ that when main hardware reference frame is aligned onto local
+ reference frame, then sensor chip reference frame is also
+ perfectly aligned with it.
+ Matrix is a 3x3 unitary matrix and typically looks like
+ [0, 1, 0; 1, 0, 0; 0, 0, -1]. Identity matrix
+ [1, 0, 0; 0, 1, 0; 0, 0, 1] means sensor chip and main hardware
+ are perfectly aligned with each other.
+
+ For example, a mounting matrix for a magnetometer sensor informs
+ userspace about sensor chip's ORIENTATION relative to the main
+ hardware.
+ More specifically, main hardware orientation is defined with
+ respect to the LOCAL EARTH GEOMAGNETIC REFERENCE FRAME where :
+ * Y is in the ground plane and positive towards magnetic North ;
+ * X is in the ground plane, perpendicular to the North axis and
+ positive towards the East ;
+ * Z is perpendicular to the ground plane and positive upwards.
+
+ An implementor might consider that for a hand-held device, a
+ 'natural' orientation would be 'front facing camera at the top'.
+ The main hardware reference frame could then be described as :
+ * Y is in the plane of the screen and is positive towards the
+ top of the screen ;
+ * X is in the plane of the screen, perpendicular to Y axis, and
+ positive towards the right hand side of the screen ;
+ * Z is perpendicular to the screen plane and positive out of the
+ screen.
+ Another example for a quadrotor UAV might be :
+ * Y is in the plane of the propellers and positive towards the
+ front-view camera;
+ * X is in the plane of the propellers, perpendicular to Y axis,
+ and positive towards the starboard side of the UAV ;
+ * Z is perpendicular to propellers plane and positive upwards.
diff --git a/Documentation/ABI/testing/sysfs-bus-mcb b/Documentation/ABI/testing/sysfs-bus-mcb
new file mode 100644
index 0000000..77947c5
--- /dev/null
+++ b/Documentation/ABI/testing/sysfs-bus-mcb
@@ -0,0 +1,29 @@
+What: /sys/bus/mcb/devices/mcb:X
+Date: March 2016
+KernelVersion: 4.7
+Contact: Johannes Thumshirn <jth@kernel.org>
+Description: Hardware chip or device hosting the MEN chameleon bus
+
+What: /sys/bus/mcb/devices/mcb:X/revision
+Date: March 2016
+KernelVersion: 4.7
+Contact: Johannes Thumshirn <jth@kernel.org>
+Description: The FPGA's revision number
+
+What: /sys/bus/mcb/devices/mcb:X/minor
+Date: March 2016
+KernelVersion: 4.7
+Contact: Johannes Thumshirn <jth@kernel.org>
+Description: The FPGA's minor number
+
+What: /sys/bus/mcb/devices/mcb:X/model
+Date: March 2016
+KernelVersion: 4.7
+Contact: Johannes Thumshirn <jth@kernel.org>
+Description: The FPGA's model number
+
+What: /sys/bus/mcb/devices/mcb:X/name
+Date: March 2016
+KernelVersion: 4.7
+Contact: Johannes Thumshirn <jth@kernel.org>
+Description: The FPGA's name
diff --git a/Documentation/ABI/testing/sysfs-class-cxl b/Documentation/ABI/testing/sysfs-class-cxl
index 7fd737e..4ba0a2a 100644
--- a/Documentation/ABI/testing/sysfs-class-cxl
+++ b/Documentation/ABI/testing/sysfs-class-cxl
@@ -233,3 +233,11 @@
0 = don't trust, the image may be different (default)
1 = trust that the image will not change.
Users: https://github.com/ibm-capi/libcxl
+
+What: /sys/class/cxl/<card>/psl_timebase_synced
+Date: March 2016
+Contact: linuxppc-dev@lists.ozlabs.org
+Description: read only
+ Returns 1 if the psl timebase register is synchronized
+ with the core timebase register, 0 otherwise.
+Users: https://github.com/ibm-capi/libcxl
diff --git a/Documentation/ABI/testing/sysfs-class-stm b/Documentation/ABI/testing/sysfs-class-stm
index c9aa4f3..77ed3da 100644
--- a/Documentation/ABI/testing/sysfs-class-stm
+++ b/Documentation/ABI/testing/sysfs-class-stm
@@ -12,3 +12,13 @@
Contact: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Description:
Shows the number of channels per master on this STM device.
+
+What: /sys/class/stm/<stm>/hw_override
+Date: March 2016
+KernelVersion: 4.7
+Contact: Alexander Shishkin <alexander.shishkin@linux.intel.com>
+Description:
+ Reads as 0 if master numbers in the STP stream produced by
+ this stm device will match the master numbers assigned by
+ the software or 1 if the stm hardware overrides software
+ assigned masters.
diff --git a/Documentation/ABI/testing/sysfs-driver-hid-picolcd b/Documentation/ABI/testing/sysfs-driver-hid-picolcd
index 08579e7..98fd81a 100644
--- a/Documentation/ABI/testing/sysfs-driver-hid-picolcd
+++ b/Documentation/ABI/testing/sysfs-driver-hid-picolcd
@@ -39,5 +39,5 @@
Note: As device can barely do 2 complete refreshes a second
it only makes sense to adjust this value if only one or two
tiles get changed and it's not appropriate to expect the application
- to flush it's tiny changes explicitely at higher than default rate.
+ to flush its tiny changes explicitly at higher than default rate.
diff --git a/Documentation/ABI/testing/sysfs-firmware-acpi b/Documentation/ABI/testing/sysfs-firmware-acpi
index b4436cca..c7fc72d 100644
--- a/Documentation/ABI/testing/sysfs-firmware-acpi
+++ b/Documentation/ABI/testing/sysfs-firmware-acpi
@@ -169,7 +169,7 @@
to enable/disable/clear ACPI interrupts in user space, which can be
used to debug some ACPI interrupt storm issues.
- Note that only writting to VALID GPE/Fixed Event is allowed,
+ Note that only writing to VALID GPE/Fixed Event is allowed,
i.e. user can only change the status of runtime GPE and
Fixed Event with event handler installed.
diff --git a/Documentation/ABI/testing/sysfs-ibft b/Documentation/ABI/testing/sysfs-ibft
index cac3930..7d6725f 100644
--- a/Documentation/ABI/testing/sysfs-ibft
+++ b/Documentation/ABI/testing/sysfs-ibft
@@ -21,3 +21,13 @@
Description: The /sys/firmware/ibft/ethernetX directory will contain
files that expose the iSCSI Boot Firmware Table NIC data.
Usually this contains the IP address, MAC, and gateway of the NIC.
+
+What: /sys/firmware/ibft/acpi_header
+Date: March 2016
+Contact: David Bond <dbond@suse.com>
+Description: The /sys/firmware/ibft/acpi_header directory will contain files
+ that expose the SIGNATURE, OEM_ID, and OEM_TABLE_ID fields of the
+ acpi table header of the iBFT structure. This will allow for
+ identification of the creator of the table which is useful in
+ determining quirks associated with some adapters when used in
+ hardware vs software iscsi initiator mode.
diff --git a/Documentation/ABI/testing/sysfs-platform-hidma b/Documentation/ABI/testing/sysfs-platform-hidma
new file mode 100644
index 0000000..d364415
--- /dev/null
+++ b/Documentation/ABI/testing/sysfs-platform-hidma
@@ -0,0 +1,9 @@
+What: /sys/devices/platform/hidma-*/chid
+ /sys/devices/platform/QCOM8061:*/chid
+Date: Dec 2015
+KernelVersion: 4.4
+Contact: "Sinan Kaya <okaya@cudeaurora.org>"
+Description:
+ Contains the ID of the channel within the HIDMA instance.
+ It is used to associate a given HIDMA channel with the
+ priority and weight calls in the management interface.
diff --git a/Documentation/ABI/testing/sysfs-platform-usbip-vudc b/Documentation/ABI/testing/sysfs-platform-usbip-vudc
new file mode 100644
index 0000000..81fcfb4
--- /dev/null
+++ b/Documentation/ABI/testing/sysfs-platform-usbip-vudc
@@ -0,0 +1,35 @@
+What: /sys/devices/platform/usbip-vudc.%d/dev_desc
+Date: April 2016
+KernelVersion: 4.6
+Contact: Krzysztof Opasiak <k.opasiak@samsung.com>
+Description:
+ This file allows to read device descriptor of
+ gadget driver which is currently bound to this
+ controller. It is possible to read this file
+ only if gadget driver is bound, otherwise error
+ is returned.
+
+What: /sys/devices/platform/usbip-vudc.%d/usbip_status
+Date: April 2016
+KernelVersion: 4.6
+Contact: Krzysztof Opasiak <k.opasiak@samsung.com>
+Description:
+ Current status of the device.
+ Allowed values:
+ 1 - Device is available and can be exported
+ 2 - Device is currently exported
+ 3 - Fatal error occurred during communication
+ with peer
+
+What: /sys/devices/platform/usbip-vudc.%d/usbip_sockfd
+Date: April 2016
+KernelVersion: 4.6
+Contact: Krzysztof Opasiak <k.opasiak@samsung.com>
+Description:
+ This file allows to export usb device to
+ connection peer. It is done by writing to this
+ file socket fd (as a string for example "8")
+ associated with a connection to remote peer who
+ would like to use this device. It is possible to
+ close the connection by writing -1 instead of
+ socked fd.
diff --git a/Documentation/DocBook/debugobjects.tmpl b/Documentation/DocBook/debugobjects.tmpl
index 24979f6..7e4f34f 100644
--- a/Documentation/DocBook/debugobjects.tmpl
+++ b/Documentation/DocBook/debugobjects.tmpl
@@ -316,8 +316,8 @@
</itemizedlist>
</para>
<para>
- The function returns 1 when the fixup was successful,
- otherwise 0. The return value is used to update the
+ The function returns true when the fixup was successful,
+ otherwise false. The return value is used to update the
statistics.
</para>
<para>
@@ -341,8 +341,8 @@
</itemizedlist>
</para>
<para>
- The function returns 1 when the fixup was successful,
- otherwise 0. The return value is used to update the
+ The function returns true when the fixup was successful,
+ otherwise false. The return value is used to update the
statistics.
</para>
<para>
@@ -359,7 +359,8 @@
statically initialized object or not. In case it is it calls
debug_object_init() and debug_object_activate() to make the
object known to the tracker and marked active. In this case
- the function should return 0 because this is not a real fixup.
+ the function should return false because this is not a real
+ fixup.
</para>
</sect1>
@@ -376,8 +377,8 @@
</itemizedlist>
</para>
<para>
- The function returns 1 when the fixup was successful,
- otherwise 0. The return value is used to update the
+ The function returns true when the fixup was successful,
+ otherwise false. The return value is used to update the
statistics.
</para>
</sect1>
@@ -397,8 +398,8 @@
</itemizedlist>
</para>
<para>
- The function returns 1 when the fixup was successful,
- otherwise 0. The return value is used to update the
+ The function returns true when the fixup was successful,
+ otherwise false. The return value is used to update the
statistics.
</para>
</sect1>
@@ -414,8 +415,8 @@
debug bucket.
</para>
<para>
- The function returns 1 when the fixup was successful,
- otherwise 0. The return value is used to update the
+ The function returns true when the fixup was successful,
+ otherwise false. The return value is used to update the
statistics.
</para>
<para>
@@ -427,7 +428,8 @@
case. The fixup function should check if this is a legitimate
case of a statically initialized object or not. In this case only
debug_object_init() should be called to make the object known to
- the tracker. Then the function should return 0 because this is not
+ the tracker. Then the function should return false because this
+ is not
a real fixup.
</para>
</sect1>
diff --git a/Documentation/DocBook/device-drivers.tmpl b/Documentation/DocBook/device-drivers.tmpl
index 184f3c7..de79efd 100644
--- a/Documentation/DocBook/device-drivers.tmpl
+++ b/Documentation/DocBook/device-drivers.tmpl
@@ -136,6 +136,8 @@
!Iinclude/linux/seqno-fence.h
!Edrivers/dma-buf/reservation.c
!Iinclude/linux/reservation.h
+!Edrivers/dma-buf/sync_file.c
+!Iinclude/linux/sync_file.h
!Edrivers/base/dma-coherent.c
!Edrivers/base/dma-mapping.c
</sect1>
@@ -233,6 +235,7 @@
!Iinclude/media/v4l2-mediabus.h
!Iinclude/media/v4l2-mem2mem.h
!Iinclude/media/v4l2-of.h
+!Iinclude/media/v4l2-rect.h
!Iinclude/media/v4l2-subdev.h
!Iinclude/media/videobuf2-core.h
!Iinclude/media/videobuf2-v4l2.h
diff --git a/Documentation/DocBook/gpu.tmpl b/Documentation/DocBook/gpu.tmpl
index 1692c4d..4a0c599 100644
--- a/Documentation/DocBook/gpu.tmpl
+++ b/Documentation/DocBook/gpu.tmpl
@@ -1617,6 +1617,11 @@
!Iinclude/drm/drm_fb_helper.h
</sect2>
<sect2>
+ <title>Framebuffer CMA Helper Functions Reference</title>
+!Pdrivers/gpu/drm/drm_fb_cma_helper.c framebuffer cma helper functions
+!Edrivers/gpu/drm/drm_fb_cma_helper.c
+ </sect2>
+ <sect2>
<title>Display Port Helper Functions Reference</title>
!Pdrivers/gpu/drm/drm_dp_helper.c dp helpers
!Iinclude/drm/drm_dp_helper.h
@@ -1671,17 +1676,23 @@
!Pdrivers/gpu/drm/drm_crtc.c Tile group
</sect2>
<sect2>
- <title>Bridges</title>
+ <title>Bridges</title>
<sect3>
- <title>Overview</title>
+ <title>Overview</title>
!Pdrivers/gpu/drm/drm_bridge.c overview
</sect3>
<sect3>
- <title>Default bridge callback sequence</title>
+ <title>Default bridge callback sequence</title>
!Pdrivers/gpu/drm/drm_bridge.c bridge callbacks
</sect3>
!Edrivers/gpu/drm/drm_bridge.c
</sect2>
+ <sect2>
+ <title>Panel Helper Reference</title>
+!Iinclude/drm/drm_panel.h
+!Edrivers/gpu/drm/drm_panel.c
+!Pdrivers/gpu/drm/drm_panel.c drm panel
+ </sect2>
</sect1>
<!-- Internals: kms properties -->
@@ -1817,7 +1828,7 @@
</tr>
<tr>
<td rowspan="42" valign="top" >DRM</td>
- <td valign="top" >Generic</td>
+ <td rowspan="2" valign="top" >Generic</td>
<td valign="top" >“rotation”</td>
<td valign="top" >BITMASK</td>
<td valign="top" >{ 0, "rotate-0" },
@@ -1832,6 +1843,13 @@
image along the specified axis prior to rotation</td>
</tr>
<tr>
+ <td valign="top" >“scaling mode”</td>
+ <td valign="top" >ENUM</td>
+ <td valign="top" >{ "None", "Full", "Center", "Full aspect" }</td>
+ <td valign="top" >Connector</td>
+ <td valign="top" >Supported by: amdgpu, gma500, i915, nouveau and radeon.</td>
+ </tr>
+ <tr>
<td rowspan="5" valign="top" >Connector</td>
<td valign="top" >“EDID”</td>
<td valign="top" >BLOB | IMMUTABLE</td>
@@ -2068,21 +2086,12 @@
<td valign="top" >property to suggest an Y offset for a connector</td>
</tr>
<tr>
- <td rowspan="8" valign="top" >Optional</td>
- <td valign="top" >“scaling mode”</td>
- <td valign="top" >ENUM</td>
- <td valign="top" >{ "None", "Full", "Center", "Full aspect" }</td>
- <td valign="top" >Connector</td>
- <td valign="top" >TBD</td>
- </tr>
- <tr>
+ <td rowspan="7" valign="top" >Optional</td>
<td valign="top" >"aspect ratio"</td>
<td valign="top" >ENUM</td>
<td valign="top" >{ "None", "4:3", "16:9" }</td>
<td valign="top" >Connector</td>
- <td valign="top" >DRM property to set aspect ratio from user space app.
- This enum is made generic to allow addition of custom aspect
- ratios.</td>
+ <td valign="top" >TDB</td>
</tr>
<tr>
<td valign="top" >“dirty”</td>
@@ -2153,7 +2162,11 @@
<td valign="top" >ENUM</td>
<td valign="top" >{ "Automatic", "Full", "Limited 16:235" }</td>
<td valign="top" >Connector</td>
- <td valign="top" >TBD</td>
+ <td valign="top" >When this property is set to Limited 16:235
+ and CTM is set, the hardware will be programmed with the
+ result of the multiplication of CTM by the limited range
+ matrix to ensure the pixels normaly in the range 0..1.0 are
+ remapped to the range 16/255..235/255.</td>
</tr>
<tr>
<td valign="top" >“audio”</td>
@@ -3334,7 +3347,7 @@
<title>Video BIOS Table (VBT)</title>
!Pdrivers/gpu/drm/i915/intel_bios.c Video BIOS Table (VBT)
!Idrivers/gpu/drm/i915/intel_bios.c
-!Idrivers/gpu/drm/i915/intel_bios.h
+!Idrivers/gpu/drm/i915/intel_vbt_defs.h
</sect2>
</sect1>
diff --git a/Documentation/DocBook/media/dvb/net.xml b/Documentation/DocBook/media/dvb/net.xml
index d2e44b7..da095ed 100644
--- a/Documentation/DocBook/media/dvb/net.xml
+++ b/Documentation/DocBook/media/dvb/net.xml
@@ -15,7 +15,7 @@
that are present on the transport stream. This is done through
<constant>/dev/dvb/adapter?/net?</constant> device node.
The data will be available via virtual <constant>dvb?_?</constant>
- network interfaces, and will be controled/routed via the standard
+ network interfaces, and will be controlled/routed via the standard
ip tools (like ip, route, netstat, ifconfig, etc).</para>
<para> Data types and and ioctl definitions are defined via
<constant>linux/dvb/net.h</constant> header.</para>
diff --git a/Documentation/DocBook/media/v4l/compat.xml b/Documentation/DocBook/media/v4l/compat.xml
index 5399e89..82fa328 100644
--- a/Documentation/DocBook/media/v4l/compat.xml
+++ b/Documentation/DocBook/media/v4l/compat.xml
@@ -2686,50 +2686,12 @@
<itemizedlist>
<listitem>
- <para>Video Output Overlay (OSD) Interface, <xref
- linkend="osd" />.</para>
- </listitem>
- <listitem>
<para>&VIDIOC-DBG-G-REGISTER; and &VIDIOC-DBG-S-REGISTER;
ioctls.</para>
</listitem>
<listitem>
<para>&VIDIOC-DBG-G-CHIP-INFO; ioctl.</para>
</listitem>
- <listitem>
- <para>&VIDIOC-ENUM-DV-TIMINGS;, &VIDIOC-QUERY-DV-TIMINGS; and
- &VIDIOC-DV-TIMINGS-CAP; ioctls.</para>
- </listitem>
- <listitem>
- <para>Flash API. <xref linkend="flash-controls" /></para>
- </listitem>
- <listitem>
- <para>&VIDIOC-CREATE-BUFS; and &VIDIOC-PREPARE-BUF; ioctls.</para>
- </listitem>
- <listitem>
- <para>Selection API. <xref linkend="selection-api" /></para>
- </listitem>
- <listitem>
- <para>Sub-device selection API: &VIDIOC-SUBDEV-G-SELECTION;
- and &VIDIOC-SUBDEV-S-SELECTION; ioctls.</para>
- </listitem>
- <listitem>
- <para>Support for frequency band enumeration: &VIDIOC-ENUM-FREQ-BANDS; ioctl.</para>
- </listitem>
- <listitem>
- <para>Vendor and device specific media bus pixel formats.
- <xref linkend="v4l2-mbus-vendor-spec-fmts" />.</para>
- </listitem>
- <listitem>
- <para>Importing DMABUF file descriptors as a new IO method described
- in <xref linkend="dmabuf" />.</para>
- </listitem>
- <listitem>
- <para>Exporting DMABUF files using &VIDIOC-EXPBUF; ioctl.</para>
- </listitem>
- <listitem>
- <para>Software Defined Radio (SDR) Interface, <xref linkend="sdr" />.</para>
- </listitem>
</itemizedlist>
</section>
diff --git a/Documentation/DocBook/media/v4l/controls.xml b/Documentation/DocBook/media/v4l/controls.xml
index 361040e..e2e5484 100644
--- a/Documentation/DocBook/media/v4l/controls.xml
+++ b/Documentation/DocBook/media/v4l/controls.xml
@@ -2841,7 +2841,7 @@
overall average bitrate for the stream and keeps it below or equal to the set bitrate. In the first case
the average bitrate for the whole stream will be smaller then the set bitrate. This is caused because the
average is calculated for smaller number of frames, on the other hand enabling this setting will ensure that
-the stream will meet tight bandwidth contraints. Applicable to encoders.
+the stream will meet tight bandwidth constraints. Applicable to encoders.
</entry>
</row>
<row><entry></entry></row>
@@ -4272,13 +4272,6 @@
<section id="flash-controls">
<title>Flash Control Reference</title>
- <note>
- <title>Experimental</title>
-
- <para>This is an <link linkend="experimental">experimental</link>
-interface and may change in the future.</para>
- </note>
-
<para>
The V4L2 flash controls are intended to provide generic access
to flash controller devices. Flash controller devices are
@@ -4743,14 +4736,6 @@
<section id="image-source-controls">
<title>Image Source Control Reference</title>
- <note>
- <title>Experimental</title>
-
- <para>This is an <link
- linkend="experimental">experimental</link> interface and may
- change in the future.</para>
- </note>
-
<para>
The Image Source control class is intended for low-level
control of image source devices such as image sensors. The
@@ -4862,14 +4847,6 @@
<section id="image-process-controls">
<title>Image Process Control Reference</title>
- <note>
- <title>Experimental</title>
-
- <para>This is an <link
- linkend="experimental">experimental</link> interface and may
- change in the future.</para>
- </note>
-
<para>
The Image Process control class is intended for low-level control of
image processing functions. Unlike
@@ -4955,14 +4932,6 @@
<section id="dv-controls">
<title>Digital Video Control Reference</title>
- <note>
- <title>Experimental</title>
-
- <para>This is an <link
- linkend="experimental">experimental</link> interface and may
- change in the future.</para>
- </note>
-
<para>
The Digital Video control class is intended to control receivers
and transmitters for <ulink url="http://en.wikipedia.org/wiki/Vga">VGA</ulink>,
diff --git a/Documentation/DocBook/media/v4l/dev-raw-vbi.xml b/Documentation/DocBook/media/v4l/dev-raw-vbi.xml
index f4b61b6..78599bb 100644
--- a/Documentation/DocBook/media/v4l/dev-raw-vbi.xml
+++ b/Documentation/DocBook/media/v4l/dev-raw-vbi.xml
@@ -85,7 +85,7 @@
results of <constant>VIDIOC_G_FMT</constant>, and call the
&VIDIOC-S-FMT; ioctl with a pointer to this structure. Drivers return
an &EINVAL; only when the given parameters are ambiguous, otherwise
-they modify the parameters according to the hardware capabilites and
+they modify the parameters according to the hardware capabilities and
return the actual parameters. When the driver allocates resources at
this point, it may return an &EBUSY; to indicate the returned
parameters are valid but the required resources are currently not
diff --git a/Documentation/DocBook/media/v4l/dev-sdr.xml b/Documentation/DocBook/media/v4l/dev-sdr.xml
index a659771..6da1157 100644
--- a/Documentation/DocBook/media/v4l/dev-sdr.xml
+++ b/Documentation/DocBook/media/v4l/dev-sdr.xml
@@ -1,11 +1,5 @@
<title>Software Defined Radio Interface (SDR)</title>
- <note>
- <title>Experimental</title>
- <para>This is an <link linkend="experimental"> experimental </link>
- interface and may change in the future.</para>
- </note>
-
<para>
SDR is an abbreviation of Software Defined Radio, the radio device
which uses application software for modulation or demodulation. This interface
diff --git a/Documentation/DocBook/media/v4l/dev-subdev.xml b/Documentation/DocBook/media/v4l/dev-subdev.xml
index 4f0ba58..f4bc27a 100644
--- a/Documentation/DocBook/media/v4l/dev-subdev.xml
+++ b/Documentation/DocBook/media/v4l/dev-subdev.xml
@@ -1,11 +1,5 @@
<title>Sub-device Interface</title>
- <note>
- <title>Experimental</title>
- <para>This is an <link linkend="experimental">experimental</link>
- interface and may change in the future.</para>
- </note>
-
<para>The complex nature of V4L2 devices, where hardware is often made of
several integrated circuits that need to interact with each other in a
controlled way, leads to complex V4L2 drivers. The drivers usually reflect
diff --git a/Documentation/DocBook/media/v4l/io.xml b/Documentation/DocBook/media/v4l/io.xml
index 144158b..e09025d 100644
--- a/Documentation/DocBook/media/v4l/io.xml
+++ b/Documentation/DocBook/media/v4l/io.xml
@@ -475,12 +475,6 @@
<section id="dmabuf">
<title>Streaming I/O (DMA buffer importing)</title>
- <note>
- <title>Experimental</title>
- <para>This is an <link linkend="experimental">experimental</link>
- interface and may change in the future.</para>
- </note>
-
<para>The DMABUF framework provides a generic method for sharing buffers
between multiple devices. Device drivers that support DMABUF can export a DMA
buffer to userspace as a file descriptor (known as the exporter role), import a
diff --git a/Documentation/DocBook/media/v4l/selection-api.xml b/Documentation/DocBook/media/v4l/selection-api.xml
index 28cbded..b764cba 100644
--- a/Documentation/DocBook/media/v4l/selection-api.xml
+++ b/Documentation/DocBook/media/v4l/selection-api.xml
@@ -1,13 +1,6 @@
<section id="selection-api">
- <title>Experimental API for cropping, composing and scaling</title>
-
- <note>
- <title>Experimental</title>
-
- <para>This is an <link linkend="experimental">experimental</link>
-interface and may change in the future.</para>
- </note>
+ <title>API for cropping, composing and scaling</title>
<section>
<title>Introduction</title>
diff --git a/Documentation/DocBook/media/v4l/subdev-formats.xml b/Documentation/DocBook/media/v4l/subdev-formats.xml
index 4e73345..199c84e 100644
--- a/Documentation/DocBook/media/v4l/subdev-formats.xml
+++ b/Documentation/DocBook/media/v4l/subdev-formats.xml
@@ -4002,12 +4002,6 @@
<section id="v4l2-mbus-vendor-spec-fmts">
<title>Vendor and Device Specific Formats</title>
- <note>
- <title>Experimental</title>
- <para>This is an <link linkend="experimental">experimental</link>
-interface and may change in the future.</para>
- </note>
-
<para>This section lists complex data formats that are either vendor or
device specific.
</para>
diff --git a/Documentation/DocBook/media/v4l/vidioc-create-bufs.xml b/Documentation/DocBook/media/v4l/vidioc-create-bufs.xml
index d81fa0d..6528e97 100644
--- a/Documentation/DocBook/media/v4l/vidioc-create-bufs.xml
+++ b/Documentation/DocBook/media/v4l/vidioc-create-bufs.xml
@@ -49,12 +49,6 @@
<refsect1>
<title>Description</title>
- <note>
- <title>Experimental</title>
- <para>This is an <link linkend="experimental"> experimental </link>
- interface and may change in the future.</para>
- </note>
-
<para>This ioctl is used to create buffers for <link linkend="mmap">memory
mapped</link> or <link linkend="userp">user pointer</link> or <link
linkend="dmabuf">DMA buffer</link> I/O. It can be used as an alternative or in
diff --git a/Documentation/DocBook/media/v4l/vidioc-dv-timings-cap.xml b/Documentation/DocBook/media/v4l/vidioc-dv-timings-cap.xml
index a2017bf..ca9ffce 100644
--- a/Documentation/DocBook/media/v4l/vidioc-dv-timings-cap.xml
+++ b/Documentation/DocBook/media/v4l/vidioc-dv-timings-cap.xml
@@ -49,14 +49,9 @@
<refsect1>
<title>Description</title>
- <note>
- <title>Experimental</title>
- <para>This is an <link linkend="experimental"> experimental </link>
- interface and may change in the future.</para>
- </note>
-
- <para>To query the capabilities of the DV receiver/transmitter applications
-can call the <constant>VIDIOC_DV_TIMINGS_CAP</constant> ioctl on a video node
+ <para>To query the capabilities of the DV receiver/transmitter applications initialize the
+<structfield>pad</structfield> field to 0, zero the reserved array of &v4l2-dv-timings-cap;
+and call the <constant>VIDIOC_DV_TIMINGS_CAP</constant> ioctl on a video node
and the driver will fill in the structure. Note that drivers may return
different values after switching the video input or output.</para>
@@ -65,8 +60,8 @@
directly on a subdevice node. The capabilities are specific to inputs (for DV
receivers) or outputs (for DV transmitters), applications must specify the
desired pad number in the &v4l2-dv-timings-cap; <structfield>pad</structfield>
-field. Attempts to query capabilities on a pad that doesn't support them will
-return an &EINVAL;.</para>
+field and zero the <structfield>reserved</structfield> array. Attempts to query
+capabilities on a pad that doesn't support them will return an &EINVAL;.</para>
<table pgwide="1" frame="none" id="v4l2-bt-timings-cap">
<title>struct <structname>v4l2_bt_timings_cap</structname></title>
@@ -145,7 +140,8 @@
<row>
<entry>__u32</entry>
<entry><structfield>reserved</structfield>[2]</entry>
- <entry>Reserved for future extensions. Drivers must set the array to zero.</entry>
+ <entry>Reserved for future extensions. Drivers and applications must
+ set the array to zero.</entry>
</row>
<row>
<entry>union</entry>
diff --git a/Documentation/DocBook/media/v4l/vidioc-enum-dv-timings.xml b/Documentation/DocBook/media/v4l/vidioc-enum-dv-timings.xml
index 6e3cadd..9b3d420 100644
--- a/Documentation/DocBook/media/v4l/vidioc-enum-dv-timings.xml
+++ b/Documentation/DocBook/media/v4l/vidioc-enum-dv-timings.xml
@@ -49,20 +49,15 @@
<refsect1>
<title>Description</title>
- <note>
- <title>Experimental</title>
- <para>This is an <link linkend="experimental"> experimental </link>
- interface and may change in the future.</para>
- </note>
-
<para>While some DV receivers or transmitters support a wide range of timings, others
support only a limited number of timings. With this ioctl applications can enumerate a list
of known supported timings. Call &VIDIOC-DV-TIMINGS-CAP; to check if it also supports other
standards or even custom timings that are not in this list.</para>
<para>To query the available timings, applications initialize the
-<structfield>index</structfield> field and zero the reserved array of &v4l2-enum-dv-timings;
-and call the <constant>VIDIOC_ENUM_DV_TIMINGS</constant> ioctl on a video node with a
+<structfield>index</structfield> field, set the <structfield>pad</structfield> field to 0,
+zero the reserved array of &v4l2-enum-dv-timings; and call the
+<constant>VIDIOC_ENUM_DV_TIMINGS</constant> ioctl on a video node with a
pointer to this structure. Drivers fill the rest of the structure or return an
&EINVAL; when the index is out of bounds. To enumerate all supported DV timings,
applications shall begin at index zero, incrementing by one until the
diff --git a/Documentation/DocBook/media/v4l/vidioc-enum-freq-bands.xml b/Documentation/DocBook/media/v4l/vidioc-enum-freq-bands.xml
index 4e8ea65..a0608ab 100644
--- a/Documentation/DocBook/media/v4l/vidioc-enum-freq-bands.xml
+++ b/Documentation/DocBook/media/v4l/vidioc-enum-freq-bands.xml
@@ -49,12 +49,6 @@
<refsect1>
<title>Description</title>
- <note>
- <title>Experimental</title>
- <para>This is an <link linkend="experimental"> experimental </link>
- interface and may change in the future.</para>
- </note>
-
<para>Enumerates the frequency bands that a tuner or modulator supports.
To do this applications initialize the <structfield>tuner</structfield>,
<structfield>type</structfield> and <structfield>index</structfield> fields,
diff --git a/Documentation/DocBook/media/v4l/vidioc-expbuf.xml b/Documentation/DocBook/media/v4l/vidioc-expbuf.xml
index 0ae0b6a..a6558a6 100644
--- a/Documentation/DocBook/media/v4l/vidioc-expbuf.xml
+++ b/Documentation/DocBook/media/v4l/vidioc-expbuf.xml
@@ -49,12 +49,6 @@
<refsect1>
<title>Description</title>
- <note>
- <title>Experimental</title>
- <para>This is an <link linkend="experimental"> experimental </link>
- interface and may change in the future.</para>
- </note>
-
<para>This ioctl is an extension to the <link linkend="mmap">memory
mapping</link> I/O method, therefore it is available only for
<constant>V4L2_MEMORY_MMAP</constant> buffers. It can be used to export a
diff --git a/Documentation/DocBook/media/v4l/vidioc-g-edid.xml b/Documentation/DocBook/media/v4l/vidioc-g-edid.xml
index 2702536..b7602d3 100644
--- a/Documentation/DocBook/media/v4l/vidioc-g-edid.xml
+++ b/Documentation/DocBook/media/v4l/vidioc-g-edid.xml
@@ -1,6 +1,6 @@
<refentry id="vidioc-g-edid">
<refmeta>
- <refentrytitle>ioctl VIDIOC_G_EDID, VIDIOC_S_EDID</refentrytitle>
+ <refentrytitle>ioctl VIDIOC_G_EDID, VIDIOC_S_EDID, VIDIOC_SUBDEV_G_EDID, VIDIOC_SUBDEV_S_EDID</refentrytitle>
&manvol;
</refmeta>
@@ -71,7 +71,8 @@
<para>To get the EDID data the application has to fill in the <structfield>pad</structfield>,
<structfield>start_block</structfield>, <structfield>blocks</structfield> and <structfield>edid</structfield>
- fields and call <constant>VIDIOC_G_EDID</constant>. The current EDID from block
+ fields, zero the <structfield>reserved</structfield> array and call
+ <constant>VIDIOC_G_EDID</constant>. The current EDID from block
<structfield>start_block</structfield> and of size <structfield>blocks</structfield>
will be placed in the memory <structfield>edid</structfield> points to. The <structfield>edid</structfield>
pointer must point to memory at least <structfield>blocks</structfield> * 128 bytes
@@ -92,8 +93,9 @@
the driver will set <structfield>blocks</structfield> to 0 and it returns 0.</para>
<para>To set the EDID blocks of a receiver the application has to fill in the <structfield>pad</structfield>,
- <structfield>blocks</structfield> and <structfield>edid</structfield> fields and set
- <structfield>start_block</structfield> to 0. It is not possible to set part of an EDID,
+ <structfield>blocks</structfield> and <structfield>edid</structfield> fields, set
+ <structfield>start_block</structfield> to 0 and zero the <structfield>reserved</structfield> array.
+ It is not possible to set part of an EDID,
it is always all or nothing. Setting the EDID data is only valid for receivers as it makes
no sense for a transmitter.</para>
diff --git a/Documentation/DocBook/media/v4l/vidioc-g-selection.xml b/Documentation/DocBook/media/v4l/vidioc-g-selection.xml
index 7865351..997f4e9 100644
--- a/Documentation/DocBook/media/v4l/vidioc-g-selection.xml
+++ b/Documentation/DocBook/media/v4l/vidioc-g-selection.xml
@@ -50,12 +50,6 @@
<refsect1>
<title>Description</title>
- <note>
- <title>Experimental</title>
- <para>This is an <link linkend="experimental"> experimental </link>
- interface and may change in the future.</para>
- </note>
-
<para>The ioctls are used to query and configure selection rectangles.</para>
<para>To query the cropping (composing) rectangle set &v4l2-selection;
@@ -222,7 +216,7 @@
<term><errorcode>ERANGE</errorcode></term>
<listitem>
<para>It is not possible to adjust &v4l2-rect; <structfield>
-r</structfield> rectangle to satisfy all contraints given in the
+r</structfield> rectangle to satisfy all constraints given in the
<structfield>flags</structfield> argument.</para>
</listitem>
</varlistentry>
diff --git a/Documentation/DocBook/media/v4l/vidioc-prepare-buf.xml b/Documentation/DocBook/media/v4l/vidioc-prepare-buf.xml
index fa7ad7e..7bde698 100644
--- a/Documentation/DocBook/media/v4l/vidioc-prepare-buf.xml
+++ b/Documentation/DocBook/media/v4l/vidioc-prepare-buf.xml
@@ -48,12 +48,6 @@
<refsect1>
<title>Description</title>
- <note>
- <title>Experimental</title>
- <para>This is an <link linkend="experimental"> experimental </link>
- interface and may change in the future.</para>
- </note>
-
<para>Applications can optionally call the
<constant>VIDIOC_PREPARE_BUF</constant> ioctl to pass ownership of the buffer
to the driver before actually enqueuing it, using the
diff --git a/Documentation/DocBook/media/v4l/vidioc-query-dv-timings.xml b/Documentation/DocBook/media/v4l/vidioc-query-dv-timings.xml
index 0c93677..d41bf47 100644
--- a/Documentation/DocBook/media/v4l/vidioc-query-dv-timings.xml
+++ b/Documentation/DocBook/media/v4l/vidioc-query-dv-timings.xml
@@ -50,12 +50,6 @@
<refsect1>
<title>Description</title>
- <note>
- <title>Experimental</title>
- <para>This is an <link linkend="experimental"> experimental </link>
- interface and may change in the future.</para>
- </note>
-
<para>The hardware may be able to detect the current DV timings
automatically, similar to sensing the video standard. To do so, applications
call <constant>VIDIOC_QUERY_DV_TIMINGS</constant> with a pointer to a
diff --git a/Documentation/DocBook/media/v4l/vidioc-streamon.xml b/Documentation/DocBook/media/v4l/vidioc-streamon.xml
index df2c63d..89fd7ce 100644
--- a/Documentation/DocBook/media/v4l/vidioc-streamon.xml
+++ b/Documentation/DocBook/media/v4l/vidioc-streamon.xml
@@ -123,6 +123,14 @@
</para>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><errorcode>ENOLINK</errorcode></term>
+ <listitem>
+ <para>The driver implements Media Controller interface and
+ the pipeline link configuration is invalid.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
</refsect1>
</refentry>
diff --git a/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-interval.xml b/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-interval.xml
index cff59f5..9d0251a 100644
--- a/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-interval.xml
+++ b/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-interval.xml
@@ -49,12 +49,6 @@
<refsect1>
<title>Description</title>
- <note>
- <title>Experimental</title>
- <para>This is an <link linkend="experimental">experimental</link>
- interface and may change in the future.</para>
- </note>
-
<para>This ioctl lets applications enumerate available frame intervals on a
given sub-device pad. Frame intervals only makes sense for sub-devices that
can control the frame period on their own. This includes, for instance,
diff --git a/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-size.xml b/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-size.xml
index abd545e..9b91b83 100644
--- a/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-size.xml
+++ b/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-size.xml
@@ -49,12 +49,6 @@
<refsect1>
<title>Description</title>
- <note>
- <title>Experimental</title>
- <para>This is an <link linkend="experimental">experimental</link>
- interface and may change in the future.</para>
- </note>
-
<para>This ioctl allows applications to enumerate all frame sizes
supported by a sub-device on the given pad for the given media bus format.
Supported formats can be retrieved with the &VIDIOC-SUBDEV-ENUM-MBUS-CODE;
diff --git a/Documentation/DocBook/media/v4l/vidioc-subdev-enum-mbus-code.xml b/Documentation/DocBook/media/v4l/vidioc-subdev-enum-mbus-code.xml
index 0bcb278..c67256a 100644
--- a/Documentation/DocBook/media/v4l/vidioc-subdev-enum-mbus-code.xml
+++ b/Documentation/DocBook/media/v4l/vidioc-subdev-enum-mbus-code.xml
@@ -49,12 +49,6 @@
<refsect1>
<title>Description</title>
- <note>
- <title>Experimental</title>
- <para>This is an <link linkend="experimental">experimental</link>
- interface and may change in the future.</para>
- </note>
-
<para>To enumerate media bus formats available at a given sub-device pad
applications initialize the <structfield>pad</structfield>, <structfield>which</structfield>
and <structfield>index</structfield> fields of &v4l2-subdev-mbus-code-enum; and
diff --git a/Documentation/DocBook/media/v4l/vidioc-subdev-g-fmt.xml b/Documentation/DocBook/media/v4l/vidioc-subdev-g-fmt.xml
index a67cde6..781089c 100644
--- a/Documentation/DocBook/media/v4l/vidioc-subdev-g-fmt.xml
+++ b/Documentation/DocBook/media/v4l/vidioc-subdev-g-fmt.xml
@@ -50,12 +50,6 @@
<refsect1>
<title>Description</title>
- <note>
- <title>Experimental</title>
- <para>This is an <link linkend="experimental">experimental</link>
- interface and may change in the future.</para>
- </note>
-
<para>These ioctls are used to negotiate the frame format at specific
subdev pads in the image pipeline.</para>
diff --git a/Documentation/DocBook/media/v4l/vidioc-subdev-g-frame-interval.xml b/Documentation/DocBook/media/v4l/vidioc-subdev-g-frame-interval.xml
index 0bc3ea22..848ec78 100644
--- a/Documentation/DocBook/media/v4l/vidioc-subdev-g-frame-interval.xml
+++ b/Documentation/DocBook/media/v4l/vidioc-subdev-g-frame-interval.xml
@@ -50,12 +50,6 @@
<refsect1>
<title>Description</title>
- <note>
- <title>Experimental</title>
- <para>This is an <link linkend="experimental">experimental</link>
- interface and may change in the future.</para>
- </note>
-
<para>These ioctls are used to get and set the frame interval at specific
subdev pads in the image pipeline. The frame interval only makes sense for
sub-devices that can control the frame period on their own. This includes,
diff --git a/Documentation/DocBook/media/v4l/vidioc-subdev-g-selection.xml b/Documentation/DocBook/media/v4l/vidioc-subdev-g-selection.xml
index c62a736..8346b2e 100644
--- a/Documentation/DocBook/media/v4l/vidioc-subdev-g-selection.xml
+++ b/Documentation/DocBook/media/v4l/vidioc-subdev-g-selection.xml
@@ -49,12 +49,6 @@
<refsect1>
<title>Description</title>
- <note>
- <title>Experimental</title>
- <para>This is an <link linkend="experimental">experimental</link>
- interface and may change in the future.</para>
- </note>
-
<para>The selections are used to configure various image
processing functionality performed by the subdevs which affect the
image size. This currently includes cropping, scaling and
diff --git a/Documentation/IRQ-domain.txt b/Documentation/IRQ-domain.txt
index 8d990bd..82001a2 100644
--- a/Documentation/IRQ-domain.txt
+++ b/Documentation/IRQ-domain.txt
@@ -70,6 +70,7 @@
==== Linear ====
irq_domain_add_linear()
+irq_domain_create_linear()
The linear reverse map maintains a fixed size table indexed by the
hwirq number. When a hwirq is mapped, an irq_desc is allocated for
@@ -81,10 +82,16 @@
allocated for in-use IRQs. The disadvantage is that the table must be
as large as the largest possible hwirq number.
+irq_domain_add_linear() and irq_domain_create_linear() are functionally
+equivalent, except for the first argument is different - the former
+accepts an Open Firmware specific 'struct device_node', while the latter
+accepts a more general abstraction 'struct fwnode_handle'.
+
The majority of drivers should use the linear map.
==== Tree ====
irq_domain_add_tree()
+irq_domain_create_tree()
The irq_domain maintains a radix tree map from hwirq numbers to Linux
IRQs. When an hwirq is mapped, an irq_desc is allocated and the
@@ -95,6 +102,11 @@
hwirq number. The disadvantage is that hwirq to IRQ number lookup is
dependent on how many entries are in the table.
+irq_domain_add_tree() and irq_domain_create_tree() are functionally
+equivalent, except for the first argument is different - the former
+accepts an Open Firmware specific 'struct device_node', while the latter
+accepts a more general abstraction 'struct fwnode_handle'.
+
Very few drivers should need this mapping.
==== No Map ===-
diff --git a/Documentation/Makefile b/Documentation/Makefile
index 1207d79..de955e1 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -1,4 +1,3 @@
-subdir-y := accounting auxdisplay blackfin connector \
+subdir-y := accounting auxdisplay blackfin \
filesystems filesystems ia64 laptops mic misc-devices \
- networking pcmcia prctl ptp timers vDSO video4linux \
- watchdog
+ networking pcmcia prctl ptp timers vDSO watchdog
diff --git a/Documentation/RCU/RTFP.txt b/Documentation/RCU/RTFP.txt
index 370ca00..9bccf16 100644
--- a/Documentation/RCU/RTFP.txt
+++ b/Documentation/RCU/RTFP.txt
@@ -176,13 +176,13 @@
which Mathieu Desnoyers is now maintaining [MathieuDesnoyers2009URCU]
[MathieuDesnoyersPhD]. TINY_RCU [PaulEMcKenney2009BloatWatchRCU] made
its appearance, as did expedited RCU [PaulEMcKenney2009expeditedRCU].
-The problem of resizeable RCU-protected hash tables may now be on a path
+The problem of resizable RCU-protected hash tables may now be on a path
to a solution [JoshTriplett2009RPHash]. A few academic researchers are now
using RCU to solve their parallel problems [HariKannan2009DynamicAnalysisRCU].
2010 produced a simpler preemptible-RCU implementation
based on TREE_RCU [PaulEMcKenney2010SimpleOptRCU], lockdep-RCU
-[PaulEMcKenney2010LockdepRCU], another resizeable RCU-protected hash
+[PaulEMcKenney2010LockdepRCU], another resizable RCU-protected hash
table [HerbertXu2010RCUResizeHash] (this one consuming more memory,
but allowing arbitrary changes in hash function, as required for DoS
avoidance in the networking code), realization of the 2009 RCU-protected
@@ -193,7 +193,7 @@
[LinusTorvalds2011Linux2:6:38:rc1:NPigginVFS], an RCU-protected red-black
tree using software transactional memory to protect concurrent updates
(strange, but true!) [PhilHoward2011RCUTMRBTree], yet another variant of
-RCU-protected resizeable hash tables [Triplett:2011:RPHash], the 3.0 RCU
+RCU-protected resizable hash tables [Triplett:2011:RPHash], the 3.0 RCU
trainwreck [PaulEMcKenney2011RCU3.0trainwreck], and Neil Brown's "Meet the
Lockers" LWN article [NeilBrown2011MeetTheLockers]. Some academic
work looked at debugging uses of RCU [Seyster:2011:RFA:2075416.2075425].
diff --git a/Documentation/adding-syscalls.txt b/Documentation/adding-syscalls.txt
index cc2d4ac..bbb31e0 100644
--- a/Documentation/adding-syscalls.txt
+++ b/Documentation/adding-syscalls.txt
@@ -136,7 +136,7 @@
- xyzzyat(fd, "", ..., AT_EMPTY_PATH) is equivalent to fxyzzy(fd, ...)
(For more details on the rationale of the *at() calls, see the openat(2) man
-page; for an example of AT_EMPTY_PATH, see the statat(2) man page.)
+page; for an example of AT_EMPTY_PATH, see the fstatat(2) man page.)
If your new xyzzy(2) system call involves a parameter describing an offset
within a file, make its type loff_t so that 64-bit offsets can be supported
diff --git a/Documentation/arm/SA1100/Assabet b/Documentation/arm/SA1100/Assabet
index 08b885d..e08a673 100644
--- a/Documentation/arm/SA1100/Assabet
+++ b/Documentation/arm/SA1100/Assabet
@@ -214,7 +214,7 @@
-----------------
All the commands above aren't so useful if they have to be typed in every
-time the Assabet is rebooted. Therefore it's possible to automatize the boot
+time the Assabet is rebooted. Therefore it's possible to automate the boot
process using RedBoot's scripting capability.
For example, I use this to boot Linux with both the kernel and the ramdisk
diff --git a/Documentation/arm64/silicon-errata.txt b/Documentation/arm64/silicon-errata.txt
index ba4b6ac..c6938e5 100644
--- a/Documentation/arm64/silicon-errata.txt
+++ b/Documentation/arm64/silicon-errata.txt
@@ -53,7 +53,9 @@
| ARM | Cortex-A57 | #832075 | ARM64_ERRATUM_832075 |
| ARM | Cortex-A57 | #852523 | N/A |
| ARM | Cortex-A57 | #834220 | ARM64_ERRATUM_834220 |
+| ARM | MMU-500 | #841119,#826419 | N/A |
| | | | |
| Cavium | ThunderX ITS | #22375, #24313 | CAVIUM_ERRATUM_22375 |
| Cavium | ThunderX GICv3 | #23154 | CAVIUM_ERRATUM_23154 |
| Cavium | ThunderX Core | #27456 | CAVIUM_ERRATUM_27456 |
+| Cavium | ThunderX SMMUv2 | #27704 | N/A |
diff --git a/Documentation/block/00-INDEX b/Documentation/block/00-INDEX
index e840b47..e55103a 100644
--- a/Documentation/block/00-INDEX
+++ b/Documentation/block/00-INDEX
@@ -2,6 +2,8 @@
- This file
biodoc.txt
- Notes on the Generic Block Layer Rewrite in Linux 2.5
+biovecs.txt
+ - Immutable biovecs and biovec iterators
capability.txt
- Generic Block Device Capability (/sys/block/<device>/capability)
cfq-iosched.txt
@@ -14,6 +16,8 @@
- Deadline IO scheduler tunables
ioprio.txt
- Block io priorities (in CFQ scheduler)
+pr.txt
+ - Block layer support for Persistent Reservations
null_blk.txt
- Null block for block-layer benchmarking.
queue-sysfs.txt
diff --git a/Documentation/blockdev/zram.txt b/Documentation/blockdev/zram.txt
index 5bda503..13100fb 100644
--- a/Documentation/blockdev/zram.txt
+++ b/Documentation/blockdev/zram.txt
@@ -59,27 +59,16 @@
pre-created. Default: 1.
2) Set max number of compression streams
- Compression backend may use up to max_comp_streams compression streams,
- thus allowing up to max_comp_streams concurrent compression operations.
- By default, compression backend uses single compression stream.
+ Regardless the value passed to this attribute, ZRAM will always
+ allocate multiple compression streams - one per online CPUs - thus
+ allowing several concurrent compression operations. The number of
+ allocated compression streams goes down when some of the CPUs
+ become offline. There is no single-compression-stream mode anymore,
+ unless you are running a UP system or has only 1 CPU online.
- Examples:
- #show max compression streams number
+ To find out how many streams are currently available:
cat /sys/block/zram0/max_comp_streams
- #set max compression streams number to 3
- echo 3 > /sys/block/zram0/max_comp_streams
-
-Note:
-In order to enable compression backend's multi stream support max_comp_streams
-must be initially set to desired concurrency level before ZRAM device
-initialisation. Once the device initialised as a single stream compression
-backend (max_comp_streams equals to 1), you will see error if you try to change
-the value of max_comp_streams because single stream compression backend
-implemented as a special case by lock overhead issue and does not support
-dynamic max_comp_streams. Only multi stream backend supports dynamic
-max_comp_streams adjustment.
-
3) Select compression algorithm
Using comp_algorithm device attribute one can see available and
currently selected (shown in square brackets) compression algorithms,
@@ -183,6 +172,7 @@
pages_compacted RO the number of pages freed during compaction
(available only via zram<id>/mm_stat node)
compact WO trigger memory compaction
+debug_stat RO this file is used for zram debugging purposes
WARNING
=======
diff --git a/Documentation/cgroup-v1/memory.txt b/Documentation/cgroup-v1/memory.txt
index ff71e16..b14abf2 100644
--- a/Documentation/cgroup-v1/memory.txt
+++ b/Documentation/cgroup-v1/memory.txt
@@ -280,17 +280,9 @@
different than user memory, since it can't be swapped out, which makes it
possible to DoS the system by consuming too much of this precious resource.
-Kernel memory won't be accounted at all until limit on a group is set. This
-allows for existing setups to continue working without disruption. The limit
-cannot be set if the cgroup have children, or if there are already tasks in the
-cgroup. Attempting to set the limit under those conditions will return -EBUSY.
-When use_hierarchy == 1 and a group is accounted, its children will
-automatically be accounted regardless of their limit value.
-
-After a group is first limited, it will be kept being accounted until it
-is removed. The memory limitation itself, can of course be removed by writing
--1 to memory.kmem.limit_in_bytes. In this case, kmem will be accounted, but not
-limited.
+Kernel memory accounting is enabled for all memory cgroups by default. But
+it can be disabled system-wide by passing cgroup.memory=nokmem to the kernel
+at boot time. In this case, kernel memory will not be accounted at all.
Kernel memory limits are not imposed for the root cgroup. Usage for the root
cgroup may or may not be accounted. The memory used is accumulated into
diff --git a/Documentation/connector/Makefile b/Documentation/connector/Makefile
deleted file mode 100644
index d98e4df..0000000
--- a/Documentation/connector/Makefile
+++ /dev/null
@@ -1,16 +0,0 @@
-ifneq ($(CONFIG_CONNECTOR),)
-obj-m += cn_test.o
-endif
-
-# List of programs to build
-hostprogs-y := ucon
-
-# Tell kbuild to always build the programs
-always := $(hostprogs-y)
-
-HOSTCFLAGS_ucon.o += -I$(objtree)/usr/include
-
-all: modules
-
-modules clean:
- $(MAKE) -C ../.. SUBDIRS=$(PWD) $@
diff --git a/Documentation/connector/connector.txt b/Documentation/connector/connector.txt
index f6215f9..ab7ca89 100644
--- a/Documentation/connector/connector.txt
+++ b/Documentation/connector/connector.txt
@@ -186,3 +186,11 @@
Some work in netlink area is still being done, so things can be changed in
2.6.15 timeframe, if it will happen, documentation will be updated for that
kernel.
+
+/*****************************************/
+Code samples
+/*****************************************/
+
+Sample code for a connector test module and user space can be found
+in samples/connector/. To build this code, enable CONFIG_CONNECTOR
+and CONFIG_SAMPLES.
diff --git a/Documentation/devices.txt b/Documentation/devices.txt
index 87b4c5e..4035eca 100644
--- a/Documentation/devices.txt
+++ b/Documentation/devices.txt
@@ -1,20 +1,17 @@
- LINUX ALLOCATED DEVICES (2.6+ version)
-
- Maintained by Alan Cox <device@lanana.org>
-
- Last revised: 6th April 2009
+ LINUX ALLOCATED DEVICES (4.x+ version)
This list is the Linux Device List, the official registry of allocated
device numbers and /dev directory nodes for the Linux operating
system.
-The latest version of this list is available from
-http://www.lanana.org/docs/device-list/ or
-ftp://ftp.kernel.org/pub/linux/docs/device-list/. This version may be
-newer than the one distributed with the Linux kernel.
-
-The LaTeX version of this document is no longer maintained.
+The LaTeX version of this document is no longer maintained, nor is
+the document that used to reside at lanana.org. This version in the
+mainline Linux kernel is the master document. Updates shall be sent
+as patches to the kernel maintainers (see the SubmittingPatches document).
+Specifically explore the sections titled "CHAR and MISC DRIVERS", and
+"BLOCK LAYER" in the MAINTAINERS file to find the right maintainers
+to involve for character and block devices.
This document is included by reference into the Filesystem Hierarchy
Standard (FHS). The FHS is available from http://www.pathname.com/fhs/.
@@ -23,60 +20,33 @@
platform only. Allocations marked (68k/Atari) apply to Linux/68k on
the Atari platform only.
-The symbol {2.6} means the allocation is obsolete and scheduled for
-removal once kernel version 2.6 (or equivalent) is released. Some of these
-allocations have already been removed.
-
-This document is in the public domain. The author requests, however,
+This document is in the public domain. The authors requests, however,
that semantically altered versions are not distributed without
-permission of the author, assuming the author can be contacted without
+permission of the authors, assuming the authors can be contacted without
an unreasonable effort.
-In particular, please don't sent patches for this list to Linus, at
-least not without contacting me first.
-
-I do not have any information about these devices beyond what appears
-on this list. Any such information requests will be deleted without
-reply.
-
**** DEVICE DRIVERS AUTHORS PLEASE READ THIS ****
+Linux now has extensive support for dynamic allocation of device numbering
+and can use sysfs and udev (systemd) to handle the naming needs. There are
+still some exceptions in the serial and boot device area. Before asking
+for a device number make sure you actually need one.
+
To have a major number allocated, or a minor number in situations
-where that applies (e.g. busmice), please contact me with the
-appropriate device information. Also, if you have additional
-information regarding any of the devices listed below, or if I have
-made a mistake, I would greatly appreciate a note.
+where that applies (e.g. busmice), please submit a patch and send to
+the authors as indicated above.
-I do, however, make a few requests about the nature of your report.
-This is necessary for me to be able to keep this list up to date and
-correct in a timely manner. First of all, *please* send it to the
-correct address... <device@lanana.org>. I receive hundreds of email
-messages a day, so mail sent to other addresses may very well get lost
-in the avalanche. Please put in a descriptive subject, so I can find
-your mail again should I need to. Too many people send me email
-saying just "device number request" in the subject.
-
-Second, please include a description of the device *in the same format
-as this list*. The reason for this is that it is the only way I have
-found to ensure I have all the requisite information to publish your
+Keep the description of the device *in the same format
+as this list*. The reason for this is that it is the only way we have
+found to ensure we have all the requisite information to publish your
device and avoid conflicts.
-Third, please don't assume that the distributed version of the list is
-up to date. Due to the number of registrations I have to maintain it
-in "batch mode", so there is likely additional registrations that
-haven't been listed yet.
-
-Fourth, remember that Linux now has extensive support for dynamic allocation
-of device numbering and can use sysfs and udev to handle the naming needs.
-There are still some exceptions in the serial and boot device area. Before
-asking for a device number make sure you actually need one.
-
-Finally, sometimes I have to play "namespace police." Please don't be
-offended. I often get submissions for /dev names that would be bound
-to cause conflicts down the road. I am trying to avoid getting in a
+Finally, sometimes we have to play "namespace police." Please don't be
+offended. We often get submissions for /dev names that would be bound
+to cause conflicts down the road. We are trying to avoid getting in a
situation where we would have to suffer an incompatible forward
-change. Therefore, please consult with me *before* you make your
+change. Therefore, please consult with us *before* you make your
device names and numbers in any way public, at least to the point
where it would be at all difficult to get them changed.
@@ -3099,9 +3069,9 @@
129 = /dev/ipath_sma Device used by Subnet Management Agent
130 = /dev/ipath_diag Device used by diagnostics programs
-234-239 UNASSIGNED
-
-240-254 char LOCAL/EXPERIMENTAL USE
+234-254 char RESERVED FOR DYNAMIC ASSIGNMENT
+ Character devices that request a dynamic allocation of major number will
+ take numbers starting from 254 and downward.
240-254 block LOCAL/EXPERIMENTAL USE
Allocated for local/experimental use. For devices not
diff --git a/Documentation/devicetree/bindings/arc/eznps.txt b/Documentation/devicetree/bindings/arc/eznps.txt
new file mode 100644
index 0000000..1aa50c6
--- /dev/null
+++ b/Documentation/devicetree/bindings/arc/eznps.txt
@@ -0,0 +1,7 @@
+EZchip NPS Network Processor Platforms Device Tree Bindings
+---------------------------------------------------------------------------
+
+Appliance main board with NPS400 ASIC.
+
+Required root node properties:
+ - compatible = "ezchip,arc-nps";
diff --git a/Documentation/devicetree/bindings/arm/atmel-at91.txt b/Documentation/devicetree/bindings/arm/atmel-at91.txt
index 1d80046..e1f5ad8 100644
--- a/Documentation/devicetree/bindings/arm/atmel-at91.txt
+++ b/Documentation/devicetree/bindings/arm/atmel-at91.txt
@@ -151,6 +151,65 @@
clocks = <&clk32k>;
};
+SHDWC SAMA5D2-Compatible Shutdown Controller
+
+1) shdwc node
+
+required properties:
+- compatible: should be "atmel,sama5d2-shdwc".
+- reg: should contain registers location and length
+- clocks: phandle to input clock.
+- #address-cells: should be one. The cell is the wake-up input index.
+- #size-cells: should be zero.
+
+optional properties:
+
+- debounce-delay-us: minimum wake-up inputs debouncer period in
+ microseconds. It's usually a board-related property.
+- atmel,wakeup-rtc-timer: boolean to enable Real-Time Clock wake-up.
+
+The node contains child nodes for each wake-up input that the platform uses.
+
+2) input nodes
+
+Wake-up input nodes are usually described in the "board" part of the Device
+Tree. Note also that input 0 is linked to the wake-up pin and is frequently
+used.
+
+Required properties:
+- reg: should contain the wake-up input index [0 - 15].
+
+Optional properties:
+- atmel,wakeup-active-high: boolean, the corresponding wake-up input described
+ by the child, forces the wake-up of the core power supply on a high level.
+ The default is to be active low.
+
+Example:
+
+On the SoC side:
+ shdwc@f8048010 {
+ compatible = "atmel,sama5d2-shdwc";
+ reg = <0xf8048010 0x10>;
+ clocks = <&clk32k>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ atmel,wakeup-rtc-timer;
+ };
+
+On the board side:
+ shdwc@f8048010 {
+ debounce-delay-us = <976>;
+
+ input@0 {
+ reg = <0>;
+ };
+
+ input@1 {
+ reg = <1>;
+ atmel,wakeup-active-high;
+ };
+ };
+
Special Function Registers (SFR)
Special Function Registers (SFR) manage specific aspects of the integrated
diff --git a/Documentation/devicetree/bindings/arm/cci.txt b/Documentation/devicetree/bindings/arm/cci.txt
index a1a5a7e..0f2153e 100644
--- a/Documentation/devicetree/bindings/arm/cci.txt
+++ b/Documentation/devicetree/bindings/arm/cci.txt
@@ -100,7 +100,7 @@
"arm,cci-400-pmu,r0"
"arm,cci-400-pmu,r1"
"arm,cci-400-pmu" - DEPRECATED, permitted only where OS has
- secure acces to CCI registers
+ secure access to CCI registers
"arm,cci-500-pmu,r0"
"arm,cci-550-pmu,r0"
- reg:
diff --git a/Documentation/devicetree/bindings/arm/coresight.txt b/Documentation/devicetree/bindings/arm/coresight.txt
index 62938eb..93147c0c 100644
--- a/Documentation/devicetree/bindings/arm/coresight.txt
+++ b/Documentation/devicetree/bindings/arm/coresight.txt
@@ -19,6 +19,7 @@
- "arm,coresight-etm3x", "arm,primecell";
- "arm,coresight-etm4x", "arm,primecell";
- "qcom,coresight-replicator1x", "arm,primecell";
+ - "arm,coresight-stm", "arm,primecell"; [1]
* reg: physical base address and length of the register
set(s) of the component.
@@ -36,6 +37,14 @@
layout using the generic DT graph presentation found in
"bindings/graph.txt".
+* Additional required properties for System Trace Macrocells (STM):
+ * reg: along with the physical base address and length of the register
+ set as described above, another entry is required to describe the
+ mapping of the extended stimulus port area.
+
+ * reg-names: the only acceptable values are "stm-base" and
+ "stm-stimulus-base", each corresponding to the areas defined in "reg".
+
* Required properties for devices that don't show up on the AMBA bus, such as
non-configurable replicators:
@@ -202,3 +211,22 @@
};
};
};
+
+4. STM
+ stm@20100000 {
+ compatible = "arm,coresight-stm", "arm,primecell";
+ reg = <0 0x20100000 0 0x1000>,
+ <0 0x28000000 0 0x180000>;
+ reg-names = "stm-base", "stm-stimulus-base";
+
+ clocks = <&soc_smc50mhz>;
+ clock-names = "apb_pclk";
+ port {
+ stm_out_port: endpoint {
+ remote-endpoint = <&main_funnel_in_port2>;
+ };
+ };
+ };
+
+[1]. There is currently two version of STM: STM32 and STM500. Both
+have the same HW interface and as such don't need an explicit binding name.
diff --git a/Documentation/devicetree/bindings/arm/l2c2x0.txt b/Documentation/devicetree/bindings/arm/l2c2x0.txt
index fe0398c..c453ab5 100644
--- a/Documentation/devicetree/bindings/arm/l2c2x0.txt
+++ b/Documentation/devicetree/bindings/arm/l2c2x0.txt
@@ -84,6 +84,12 @@
- prefetch-instr : Instruction prefetch. Value: <0> (forcibly disable),
<1> (forcibly enable), property absent (retain settings set by
firmware)
+- arm,dynamic-clock-gating : L2 dynamic clock gating. Value: <0> (forcibly
+ disable), <1> (forcibly enable), property absent (OS specific behavior,
+ preferrably retain firmware settings)
+- arm,standby-mode: L2 standby mode enable. Value <0> (forcibly disable),
+ <1> (forcibly enable), property absent (OS specific behavior,
+ preferrably retain firmware settings)
Example:
diff --git a/Documentation/devicetree/bindings/arm/marvell/ap806-system-controller.txt b/Documentation/devicetree/bindings/arm/marvell/ap806-system-controller.txt
new file mode 100644
index 0000000..8968371
--- /dev/null
+++ b/Documentation/devicetree/bindings/arm/marvell/ap806-system-controller.txt
@@ -0,0 +1,35 @@
+Marvell Armada AP806 System Controller
+======================================
+
+The AP806 is one of the two core HW blocks of the Marvell Armada 7K/8K
+SoCs. It contains a system controller, which provides a number
+registers giving access to numerous features: clocks, pin-muxing and
+many other SoC configuration items. This DT binding allows to describe
+this system controller.
+
+The Device Tree node representing the AP806 system controller provides
+a number of clocks:
+
+ - 0: clock of CPU cluster 0
+ - 1: clock of CPU cluster 1
+ - 2: fixed PLL at 1200 Mhz
+ - 3: MSS clock, derived from the fixed PLL
+
+Required properties:
+
+ - compatible: must be:
+ "marvell,ap806-system-controller", "syscon"
+ - reg: register area of the AP806 system controller
+ - #clock-cells: must be set to 1
+ - clock-output-names: must be defined to:
+ "ap-cpu-cluster-0", "ap-cpu-cluster-1", "ap-fixed", "ap-mss"
+
+Example:
+
+ syscon: system-controller@6f4000 {
+ compatible = "marvell,ap806-system-controller", "syscon";
+ #clock-cells = <1>;
+ clock-output-names = "ap-cpu-cluster-0", "ap-cpu-cluster-1",
+ "ap-fixed", "ap-mss";
+ reg = <0x6f4000 0x1000>;
+ };
diff --git a/Documentation/devicetree/bindings/arm/marvell/cp110-system-controller0.txt b/Documentation/devicetree/bindings/arm/marvell/cp110-system-controller0.txt
new file mode 100644
index 0000000..30c5469
--- /dev/null
+++ b/Documentation/devicetree/bindings/arm/marvell/cp110-system-controller0.txt
@@ -0,0 +1,83 @@
+Marvell Armada CP110 System Controller 0
+========================================
+
+The CP110 is one of the two core HW blocks of the Marvell Armada 7K/8K
+SoCs. It contains two sets of system control registers, System
+Controller 0 and System Controller 1. This Device Tree binding allows
+to describe the first system controller, which provides registers to
+configure various aspects of the SoC.
+
+The Device Tree node representing this System Controller 0 provides a
+number of clocks:
+
+ - a set of core clocks
+ - a set of gatable clocks
+
+Those clocks can be referenced by other Device Tree nodes using two
+cells:
+ - The first cell must be 0 or 1. 0 for the core clocks and 1 for the
+ gatable clocks.
+ - The second cell identifies the particular core clock or gatable
+ clocks.
+
+The following clocks are available:
+ - Core clocks
+ - 0 0 APLL
+ - 0 1 PPv2 core
+ - 0 2 EIP
+ - 0 3 Core
+ - 0 4 NAND core
+ - Gatable clocks
+ - 1 0 Audio
+ - 1 1 Comm Unit
+ - 1 2 NAND
+ - 1 3 PPv2
+ - 1 4 SDIO
+ - 1 5 MG Domain
+ - 1 6 MG Core
+ - 1 7 XOR1
+ - 1 8 XOR0
+ - 1 9 GOP DP
+ - 1 11 PCIe x1 0
+ - 1 12 PCIe x1 1
+ - 1 13 PCIe x4
+ - 1 14 PCIe / XOR
+ - 1 15 SATA
+ - 1 16 SATA USB
+ - 1 17 Main
+ - 1 18 SD/MMC
+ - 1 21 Slow IO (SPI, NOR, BootROM, I2C, UART)
+ - 1 22 USB3H0
+ - 1 23 USB3H1
+ - 1 24 USB3 Device
+ - 1 25 EIP150
+ - 1 26 EIP197
+
+Required properties:
+
+ - compatible: must be:
+ "marvell,cp110-system-controller0", "syscon";
+ - reg: register area of the CP110 system controller 0
+ - #clock-cells: must be set to 2
+ - core-clock-output-names must be set to:
+ "cpm-apll", "cpm-ppv2-core", "cpm-eip", "cpm-core", "cpm-nand-core"
+ - gate-clock-output-names must be set to:
+ "cpm-audio", "cpm-communit", "cpm-nand", "cpm-ppv2", "cpm-sdio",
+ "cpm-mg-domain", "cpm-mg-core", "cpm-xor1", "cpm-xor0", "cpm-gop-dp", "none",
+ "cpm-pcie_x10", "cpm-pcie_x11", "cpm-pcie_x4", "cpm-pcie-xor", "cpm-sata",
+ "cpm-sata-usb", "cpm-main", "cpm-sd-mmc", "none", "none", "cpm-slow-io",
+ "cpm-usb3h0", "cpm-usb3h1", "cpm-usb3dev", "cpm-eip150", "cpm-eip197";
+
+Example:
+
+ cpm_syscon0: system-controller@440000 {
+ compatible = "marvell,cp110-system-controller0", "syscon";
+ reg = <0x440000 0x1000>;
+ #clock-cells = <2>;
+ core-clock-output-names = "cpm-apll", "cpm-ppv2-core", "cpm-eip", "cpm-core", "cpm-nand-core";
+ gate-clock-output-names = "cpm-audio", "cpm-communit", "cpm-nand", "cpm-ppv2", "cpm-sdio",
+ "cpm-mg-domain", "cpm-mg-core", "cpm-xor1", "cpm-xor0", "cpm-gop-dp", "none",
+ "cpm-pcie_x10", "cpm-pcie_x11", "cpm-pcie_x4", "cpm-pcie-xor", "cpm-sata",
+ "cpm-sata-usb", "cpm-main", "cpm-sd-mmc", "none", "none", "cpm-slow-io",
+ "cpm-usb3h0", "cpm-usb3h1", "cpm-usb3dev", "cpm-eip150", "cpm-eip197";
+ };
diff --git a/Documentation/devicetree/bindings/arm/omap/crossbar.txt b/Documentation/devicetree/bindings/arm/omap/crossbar.txt
index a9b28d7..bb5727a 100644
--- a/Documentation/devicetree/bindings/arm/omap/crossbar.txt
+++ b/Documentation/devicetree/bindings/arm/omap/crossbar.txt
@@ -42,7 +42,8 @@
Consumer:
========
See Documentation/devicetree/bindings/interrupt-controller/interrupts.txt and
-Documentation/devicetree/bindings/arm/gic.txt for further details.
+Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt for
+further details.
An interrupt consumer on an SoC using crossbar will use:
interrupts = <GIC_SPI request_number interrupt_level>
diff --git a/Documentation/devicetree/bindings/arm/spear-misc.txt b/Documentation/devicetree/bindings/arm/spear-misc.txt
index cf64982..e404e25 100644
--- a/Documentation/devicetree/bindings/arm/spear-misc.txt
+++ b/Documentation/devicetree/bindings/arm/spear-misc.txt
@@ -6,4 +6,4 @@
misc node required properties:
- compatible Should be "st,spear1340-misc", "syscon".
-- reg: Address range of misc space upto 8K
+- reg: Address range of misc space up to 8K
diff --git a/Documentation/devicetree/bindings/arm/ux500/boards.txt b/Documentation/devicetree/bindings/arm/ux500/boards.txt
index b8737a8..7334c24 100644
--- a/Documentation/devicetree/bindings/arm/ux500/boards.txt
+++ b/Documentation/devicetree/bindings/arm/ux500/boards.txt
@@ -23,7 +23,7 @@
see binding for arm/scu.txt
interrupt-controller:
- see binding for arm/gic.txt
+ see binding for interrupt-controller/arm,gic.txt
timer:
see binding for arm/twd.txt
diff --git a/Documentation/devicetree/bindings/ata/tegra-sata.txt b/Documentation/devicetree/bindings/ata/nvidia,tegra124-ahci.txt
similarity index 100%
rename from Documentation/devicetree/bindings/ata/tegra-sata.txt
rename to Documentation/devicetree/bindings/ata/nvidia,tegra124-ahci.txt
diff --git a/Documentation/devicetree/bindings/bus/ti-gpmc.txt b/Documentation/devicetree/bindings/bus/ti-gpmc.txt
deleted file mode 100644
index 0168370..0000000
--- a/Documentation/devicetree/bindings/bus/ti-gpmc.txt
+++ /dev/null
@@ -1,135 +0,0 @@
-Device tree bindings for OMAP general purpose memory controllers (GPMC)
-
-The actual devices are instantiated from the child nodes of a GPMC node.
-
-Required properties:
-
- - compatible: Should be set to one of the following:
-
- ti,omap2420-gpmc (omap2420)
- ti,omap2430-gpmc (omap2430)
- ti,omap3430-gpmc (omap3430 & omap3630)
- ti,omap4430-gpmc (omap4430 & omap4460 & omap543x)
- ti,am3352-gpmc (am335x devices)
-
- - reg: A resource specifier for the register space
- (see the example below)
- - ti,hwmods: Should be set to "ti,gpmc" until the DT transition is
- completed.
- - #address-cells: Must be set to 2 to allow memory address translation
- - #size-cells: Must be set to 1 to allow CS address passing
- - gpmc,num-cs: The maximum number of chip-select lines that controller
- can support.
- - gpmc,num-waitpins: The maximum number of wait pins that controller can
- support.
- - ranges: Must be set up to reflect the memory layout with four
- integer values for each chip-select line in use:
-
- <cs-number> 0 <physical address of mapping> <size>
-
- Currently, calculated values derived from the contents
- of the per-CS register GPMC_CONFIG7 (as set up by the
- bootloader) are used for the physical address decoding.
- As this will change in the future, filling correct
- values here is a requirement.
-
-Timing properties for child nodes. All are optional and default to 0.
-
- - gpmc,sync-clk-ps: Minimum clock period for synchronous mode, in picoseconds
-
- Chip-select signal timings (in nanoseconds) corresponding to GPMC_CONFIG2:
- - gpmc,cs-on-ns: Assertion time
- - gpmc,cs-rd-off-ns: Read deassertion time
- - gpmc,cs-wr-off-ns: Write deassertion time
-
- ADV signal timings (in nanoseconds) corresponding to GPMC_CONFIG3:
- - gpmc,adv-on-ns: Assertion time
- - gpmc,adv-rd-off-ns: Read deassertion time
- - gpmc,adv-wr-off-ns: Write deassertion time
- - gpmc,adv-aad-mux-on-ns: Assertion time for AAD
- - gpmc,adv-aad-mux-rd-off-ns: Read deassertion time for AAD
- - gpmc,adv-aad-mux-wr-off-ns: Write deassertion time for AAD
-
- WE signals timings (in nanoseconds) corresponding to GPMC_CONFIG4:
- - gpmc,we-on-ns Assertion time
- - gpmc,we-off-ns: Deassertion time
-
- OE signals timings (in nanoseconds) corresponding to GPMC_CONFIG4:
- - gpmc,oe-on-ns: Assertion time
- - gpmc,oe-off-ns: Deassertion time
- - gpmc,oe-aad-mux-on-ns: Assertion time for AAD
- - gpmc,oe-aad-mux-off-ns: Deassertion time for AAD
-
- Access time and cycle time timings (in nanoseconds) corresponding to
- GPMC_CONFIG5:
- - gpmc,page-burst-access-ns: Multiple access word delay
- - gpmc,access-ns: Start-cycle to first data valid delay
- - gpmc,rd-cycle-ns: Total read cycle time
- - gpmc,wr-cycle-ns: Total write cycle time
- - gpmc,bus-turnaround-ns: Turn-around time between successive accesses
- - gpmc,cycle2cycle-delay-ns: Delay between chip-select pulses
- - gpmc,clk-activation-ns: GPMC clock activation time
- - gpmc,wait-monitoring-ns: Start of wait monitoring with regard to valid
- data
-
-Boolean timing parameters. If property is present parameter enabled and
-disabled if omitted:
- - gpmc,adv-extra-delay: ADV signal is delayed by half GPMC clock
- - gpmc,cs-extra-delay: CS signal is delayed by half GPMC clock
- - gpmc,cycle2cycle-diffcsen: Add "cycle2cycle-delay" between successive
- accesses to a different CS
- - gpmc,cycle2cycle-samecsen: Add "cycle2cycle-delay" between successive
- accesses to the same CS
- - gpmc,oe-extra-delay: OE signal is delayed by half GPMC clock
- - gpmc,we-extra-delay: WE signal is delayed by half GPMC clock
- - gpmc,time-para-granularity: Multiply all access times by 2
-
-The following are only applicable to OMAP3+ and AM335x:
- - gpmc,wr-access-ns: In synchronous write mode, for single or
- burst accesses, defines the number of
- GPMC_FCLK cycles from start access time
- to the GPMC_CLK rising edge used by the
- memory device for the first data capture.
- - gpmc,wr-data-mux-bus-ns: In address-data multiplex mode, specifies
- the time when the first data is driven on
- the address-data bus.
-
-GPMC chip-select settings properties for child nodes. All are optional.
-
-- gpmc,burst-length Page/burst length. Must be 4, 8 or 16.
-- gpmc,burst-wrap Enables wrap bursting
-- gpmc,burst-read Enables read page/burst mode
-- gpmc,burst-write Enables write page/burst mode
-- gpmc,device-width Total width of device(s) connected to a GPMC
- chip-select in bytes. The GPMC supports 8-bit
- and 16-bit devices and so this property must be
- 1 or 2.
-- gpmc,mux-add-data Address and data multiplexing configuration.
- Valid values are 1 for address-address-data
- multiplexing mode and 2 for address-data
- multiplexing mode.
-- gpmc,sync-read Enables synchronous read. Defaults to asynchronous
- is this is not set.
-- gpmc,sync-write Enables synchronous writes. Defaults to asynchronous
- is this is not set.
-- gpmc,wait-pin Wait-pin used by client. Must be less than
- "gpmc,num-waitpins".
-- gpmc,wait-on-read Enables wait monitoring on reads.
-- gpmc,wait-on-write Enables wait monitoring on writes.
-
-Example for an AM33xx board:
-
- gpmc: gpmc@50000000 {
- compatible = "ti,am3352-gpmc";
- ti,hwmods = "gpmc";
- reg = <0x50000000 0x2000>;
- interrupts = <100>;
-
- gpmc,num-cs = <8>;
- gpmc,num-waitpins = <2>;
- #address-cells = <2>;
- #size-cells = <1>;
- ranges = <0 0 0x08000000 0x10000000>; /* CS0 @addr 0x8000000, size 0x10000000 */
-
- /* child nodes go here */
- };
diff --git a/Documentation/devicetree/bindings/clock/artpec6.txt b/Documentation/devicetree/bindings/clock/artpec6.txt
new file mode 100644
index 0000000..dff9cdf
--- /dev/null
+++ b/Documentation/devicetree/bindings/clock/artpec6.txt
@@ -0,0 +1,41 @@
+* Clock bindings for Axis ARTPEC-6 chip
+
+The bindings are based on the clock provider binding in
+Documentation/devicetree/bindings/clock/clock-bindings.txt
+
+External clocks:
+----------------
+
+There are two external inputs to the main clock controller which should be
+provided using the common clock bindings.
+- "sys_refclk": External 50 Mhz oscillator (required)
+- "i2s_refclk": Alternate audio reference clock (optional).
+
+Main clock controller
+---------------------
+
+Required properties:
+- #clock-cells: Should be <1>
+ See dt-bindings/clock/axis,artpec6-clkctrl.h for the list of valid identifiers.
+- compatible: Should be "axis,artpec6-clkctrl"
+- reg: Must contain the base address and length of the system controller
+- clocks: Must contain a phandle entry for each clock in clock-names
+- clock-names: Must include the external oscillator ("sys_refclk"). Optional
+ ones are the audio reference clock ("i2s_refclk") and the audio fractional
+ dividers ("frac_clk0" and "frac_clk1").
+
+Examples:
+
+ext_clk: ext_clk {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <50000000>;
+};
+
+clkctrl: clkctrl@f8000000 {
+ #clock-cells = <1>;
+ compatible = "axis,artpec6-clkctrl";
+ reg = <0xf8000000 0x48>;
+ clocks = <&ext_clk>;
+ clock-names = "sys_refclk";
+};
diff --git a/Documentation/devicetree/bindings/clock/axs10x-i2s-pll-clock.txt b/Documentation/devicetree/bindings/clock/axs10x-i2s-pll-clock.txt
new file mode 100644
index 0000000..5ffc8df
--- /dev/null
+++ b/Documentation/devicetree/bindings/clock/axs10x-i2s-pll-clock.txt
@@ -0,0 +1,25 @@
+Binding for the AXS10X I2S PLL clock
+
+This binding uses the common clock binding[1].
+
+[1] Documentation/devicetree/bindings/clock/clock-bindings.txt
+
+Required properties:
+- compatible: shall be "snps,axs10x-i2s-pll-clock"
+- reg : address and length of the I2S PLL register set.
+- clocks: shall be the input parent clock phandle for the PLL.
+- #clock-cells: from common clock binding; Should always be set to 0.
+
+Example:
+ pll_clock: pll_clock {
+ compatible = "fixed-clock";
+ clock-frequency = <27000000>;
+ #clock-cells = <0>;
+ };
+
+ i2s_clock@100a0 {
+ compatible = "snps,axs10x-i2s-pll-clock";
+ reg = <0x100a0 0x10>;
+ clocks = <&pll_clock>;
+ #clock-cells = <0>;
+ };
diff --git a/Documentation/devicetree/bindings/clock/hi3519-crg.txt b/Documentation/devicetree/bindings/clock/hi3519-crg.txt
new file mode 100644
index 0000000..acd1f23
--- /dev/null
+++ b/Documentation/devicetree/bindings/clock/hi3519-crg.txt
@@ -0,0 +1,46 @@
+* Hisilicon Hi3519 Clock and Reset Generator(CRG)
+
+The Hi3519 CRG module provides clock and reset signals to various
+controllers within the SoC.
+
+This binding uses the following bindings:
+ Documentation/devicetree/bindings/clock/clock-bindings.txt
+ Documentation/devicetree/bindings/reset/reset.txt
+
+Required Properties:
+
+- compatible: should be one of the following.
+ - "hisilicon,hi3519-crg" - controller compatible with Hi3519 SoC.
+
+- reg: physical base address of the controller and length of memory mapped
+ region.
+
+- #clock-cells: should be 1.
+
+Each clock is assigned an identifier and client nodes use this identifier
+to specify the clock which they consume.
+
+All these identifier could be found in <dt-bindings/clock/hi3519-clock.h>.
+
+- #reset-cells: should be 2.
+
+A reset signal can be controlled by writing a bit register in the CRG module.
+The reset specifier consists of two cells. The first cell represents the
+register offset relative to the base address. The second cell represents the
+bit index in the register.
+
+Example: CRG nodes
+CRG: clock-reset-controller@12010000 {
+ compatible = "hisilicon,hi3519-crg";
+ reg = <0x12010000 0x10000>;
+ #clock-cells = <1>;
+ #reset-cells = <2>;
+};
+
+Example: consumer nodes
+i2c0: i2c@12110000 {
+ compatible = "hisilicon,hi3519-i2c";
+ reg = <0x12110000 0x1000>;
+ clocks = <&CRG HI3519_I2C0_RST>;
+ resets = <&CRG 0xe4 0>;
+};
diff --git a/Documentation/devicetree/bindings/clock/imx35-clock.txt b/Documentation/devicetree/bindings/clock/imx35-clock.txt
index a703564..f497832 100644
--- a/Documentation/devicetree/bindings/clock/imx35-clock.txt
+++ b/Documentation/devicetree/bindings/clock/imx35-clock.txt
@@ -94,6 +94,7 @@
csi_sel 79
iim_gate 80
gpu2d_gate 81
+ ckli_gate 82
Examples:
diff --git a/Documentation/devicetree/bindings/clock/microchip,pic32.txt b/Documentation/devicetree/bindings/clock/microchip,pic32.txt
new file mode 100644
index 0000000..c93d88f
--- /dev/null
+++ b/Documentation/devicetree/bindings/clock/microchip,pic32.txt
@@ -0,0 +1,39 @@
+Microchip PIC32 Clock Controller Binding
+----------------------------------------
+Microchip clock controller is consists of few oscillators, PLL, multiplexer
+and few divider modules.
+
+This binding uses common clock bindings.
+[1] Documentation/devicetree/bindings/clock/clock-bindings.txt
+
+Required properties:
+- compatible: shall be "microchip,pic32mzda-clk".
+- reg: shall contain base address and length of clock registers.
+- #clock-cells: shall be 1.
+
+Optional properties:
+- microchip,pic32mzda-sosc: shall be added only if platform has
+ secondary oscillator connected.
+
+Example:
+ rootclk: clock-controller@1f801200 {
+ compatible = "microchip,pic32mzda-clk";
+ reg = <0x1f801200 0x200>;
+ #clock-cells = <1>;
+ /* optional */
+ microchip,pic32mzda-sosc;
+ };
+
+
+The clock consumer shall specify the desired clock-output of the clock
+controller (as defined in [2]) by specifying output-id in its "clock"
+phandle cell.
+[2] include/dt-bindings/clock/microchip,pic32-clock.h
+
+For example for UART2:
+uart2: serial@2 {
+ compatible = "microchip,pic32mzda-uart";
+ reg = <>;
+ interrupts = <>;
+ clocks = <&rootclk PB2CLK>;
+};
diff --git a/Documentation/devicetree/bindings/clock/nvidia,tegra124-dfll.txt b/Documentation/devicetree/bindings/clock/nvidia,tegra124-dfll.txt
index ee7e5fd..63f9d82 100644
--- a/Documentation/devicetree/bindings/clock/nvidia,tegra124-dfll.txt
+++ b/Documentation/devicetree/bindings/clock/nvidia,tegra124-dfll.txt
@@ -50,7 +50,7 @@
Example:
-clock@0,70110000 {
+clock@70110000 {
compatible = "nvidia,tegra124-dfll";
reg = <0 0x70110000 0 0x100>, /* DFLL control */
<0 0x70110000 0 0x100>, /* I2C output control */
diff --git a/Documentation/devicetree/bindings/clock/oxnas,stdclk.txt b/Documentation/devicetree/bindings/clock/oxnas,stdclk.txt
new file mode 100644
index 0000000..208cca6
--- /dev/null
+++ b/Documentation/devicetree/bindings/clock/oxnas,stdclk.txt
@@ -0,0 +1,35 @@
+Oxford Semiconductor OXNAS SoC Family Standard Clocks
+================================================
+
+Please also refer to clock-bindings.txt in this directory for common clock
+bindings usage.
+
+Required properties:
+- compatible: Should be "oxsemi,ox810se-stdclk"
+- #clock-cells: 1, see below
+
+Parent node should have the following properties :
+- compatible: Should be "oxsemi,ox810se-sys-ctrl", "syscon", "simple-mfd"
+
+For OX810SE, the clock indices are :
+ - 0: LEON
+ - 1: DMA_SGDMA
+ - 2: CIPHER
+ - 3: SATA
+ - 4: AUDIO
+ - 5: USBMPH
+ - 6: ETHA
+ - 7: PCIA
+ - 8: NAND
+
+example:
+
+sys: sys-ctrl@000000 {
+ compatible = "oxsemi,ox810se-sys-ctrl", "syscon", "simple-mfd";
+ reg = <0x000000 0x100000>;
+
+ stdclk: stdclk {
+ compatible = "oxsemi,ox810se-stdclk";
+ #clock-cells = <1>;
+ };
+};
diff --git a/Documentation/devicetree/bindings/clock/rockchip,rk3188-cru.txt b/Documentation/devicetree/bindings/clock/rockchip,rk3188-cru.txt
index 0c2bf5e..7f36853 100644
--- a/Documentation/devicetree/bindings/clock/rockchip,rk3188-cru.txt
+++ b/Documentation/devicetree/bindings/clock/rockchip,rk3188-cru.txt
@@ -16,7 +16,7 @@
Optional Properties:
- rockchip,grf: phandle to the syscon managing the "general register files"
- If missing pll rates are not changable, due to the missing pll lock status.
+ If missing pll rates are not changeable, due to the missing pll lock status.
Each clock is assigned an identifier and client nodes can use this identifier
to specify the clock which they consume. All available clocks are defined as
diff --git a/Documentation/devicetree/bindings/clock/rockchip,rk3288-cru.txt b/Documentation/devicetree/bindings/clock/rockchip,rk3288-cru.txt
index c9fbb76..8cb47c3 100644
--- a/Documentation/devicetree/bindings/clock/rockchip,rk3288-cru.txt
+++ b/Documentation/devicetree/bindings/clock/rockchip,rk3288-cru.txt
@@ -15,7 +15,7 @@
Optional Properties:
- rockchip,grf: phandle to the syscon managing the "general register files"
- If missing pll rates are not changable, due to the missing pll lock status.
+ If missing pll rates are not changeable, due to the missing pll lock status.
Each clock is assigned an identifier and client nodes can use this identifier
to specify the clock which they consume. All available clocks are defined as
diff --git a/Documentation/devicetree/bindings/clock/rockchip,rk3399-cru.txt b/Documentation/devicetree/bindings/clock/rockchip,rk3399-cru.txt
new file mode 100644
index 0000000..3888dd3
--- /dev/null
+++ b/Documentation/devicetree/bindings/clock/rockchip,rk3399-cru.txt
@@ -0,0 +1,62 @@
+* Rockchip RK3399 Clock and Reset Unit
+
+The RK3399 clock controller generates and supplies clock to various
+controllers within the SoC and also implements a reset controller for SoC
+peripherals.
+
+Required Properties:
+
+- compatible: PMU for CRU should be "rockchip,rk3399-pmucru"
+- compatible: CRU should be "rockchip,rk3399-cru"
+- reg: physical base address of the controller and length of memory mapped
+ region.
+- #clock-cells: should be 1.
+- #reset-cells: should be 1.
+
+Each clock is assigned an identifier and client nodes can use this identifier
+to specify the clock which they consume. All available clocks are defined as
+preprocessor macros in the dt-bindings/clock/rk3399-cru.h headers and can be
+used in device tree sources. Similar macros exist for the reset sources in
+these files.
+
+External clocks:
+
+There are several clocks that are generated outside the SoC. It is expected
+that they are defined using standard clock bindings with following
+clock-output-names:
+ - "xin24m" - crystal input - required,
+ - "xin32k" - rtc clock - optional,
+ - "clkin_gmac" - external GMAC clock - optional,
+ - "clkin_i2s" - external I2S clock - optional,
+ - "pclkin_cif" - external ISP clock - optional,
+ - "clk_usbphy0_480m" - output clock of the pll in the usbphy0
+ - "clk_usbphy1_480m" - output clock of the pll in the usbphy1
+
+Example: Clock controller node:
+
+ pmucru: pmu-clock-controller@ff750000 {
+ compatible = "rockchip,rk3399-pmucru";
+ reg = <0x0 0xff750000 0x0 0x1000>;
+ #clock-cells = <1>;
+ #reset-cells = <1>;
+ };
+
+ cru: clock-controller@ff760000 {
+ compatible = "rockchip,rk3399-cru";
+ reg = <0x0 0xff760000 0x0 0x1000>;
+ #clock-cells = <1>;
+ #reset-cells = <1>;
+ };
+
+Example: UART controller node that consumes the clock generated by the clock
+ controller:
+
+ uart0: serial@ff1a0000 {
+ compatible = "rockchip,rk3399-uart", "snps,dw-apb-uart";
+ reg = <0x0 0xff180000 0x0 0x100>;
+ clocks = <&cru SCLK_UART0>, <&cru PCLK_UART0>;
+ clock-names = "baudclk", "apb_pclk";
+ interrupts = <GIC_SPI 99 IRQ_TYPE_LEVEL_HIGH>;
+ reg-shift = <2>;
+ reg-io-width = <4>;
+ };
diff --git a/Documentation/devicetree/bindings/clock/st/st,clkgen.txt b/Documentation/devicetree/bindings/clock/st/st,clkgen.txt
index 78978f1..b18bf86 100644
--- a/Documentation/devicetree/bindings/clock/st/st,clkgen.txt
+++ b/Documentation/devicetree/bindings/clock/st/st,clkgen.txt
@@ -40,7 +40,7 @@
};
This binding uses the common clock binding[1].
-Each subnode should use the binding discribe in [2]..[7]
+Each subnode should use the binding described in [2]..[7]
[1] Documentation/devicetree/bindings/clock/clock-bindings.txt
[2] Documentation/devicetree/bindings/clock/st,clkgen-divmux.txt
diff --git a/Documentation/devicetree/bindings/clock/sunxi.txt b/Documentation/devicetree/bindings/clock/sunxi.txt
index 834436f..8f7619d 100644
--- a/Documentation/devicetree/bindings/clock/sunxi.txt
+++ b/Documentation/devicetree/bindings/clock/sunxi.txt
@@ -10,6 +10,7 @@
"allwinner,sun4i-a10-pll1-clk" - for the main PLL clock and PLL4
"allwinner,sun6i-a31-pll1-clk" - for the main PLL clock on A31
"allwinner,sun8i-a23-pll1-clk" - for the main PLL clock on A23
+ "allwinner,sun4i-a10-pll3-clk" - for the video PLL clock on A10
"allwinner,sun9i-a80-pll4-clk" - for the peripheral PLLs on A80
"allwinner,sun4i-a10-pll5-clk" - for the PLL5 clock
"allwinner,sun4i-a10-pll6-clk" - for the PLL6 clock
@@ -63,7 +64,9 @@
"allwinner,sun8i-a83t-bus-gates-clk" - for the bus gates on A83T
"allwinner,sun8i-h3-bus-gates-clk" - for the bus gates on H3
"allwinner,sun9i-a80-apbs-gates-clk" - for the APBS gates on A80
+ "allwinner,sun4i-a10-display-clk" - for the display clocks on the A10
"allwinner,sun4i-a10-dram-gates-clk" - for the DRAM gates on A10
+ "allwinner,sun5i-a13-dram-gates-clk" - for the DRAM gates on A13
"allwinner,sun5i-a13-mbus-clk" - for the MBUS clock on A13
"allwinner,sun4i-a10-mmc-clk" - for the MMC clock
"allwinner,sun9i-a80-mmc-clk" - for mmc module clocks on A80
@@ -73,6 +76,8 @@
"allwinner,sun8i-a23-mbus-clk" - for the MBUS clock on A23
"allwinner,sun7i-a20-out-clk" - for the external output clocks
"allwinner,sun7i-a20-gmac-clk" - for the GMAC clock module on A20/A31
+ "allwinner,sun4i-a10-tcon-ch0-clk" - for the TCON channel 0 clock on the A10
+ "allwinner,sun4i-a10-tcon-ch1-clk" - for the TCON channel 1 clock on the A10
"allwinner,sun4i-a10-usb-clk" - for usb gates + resets on A10 / A20
"allwinner,sun5i-a13-usb-clk" - for usb gates + resets on A13
"allwinner,sun6i-a31-usb-clk" - for usb gates + resets on A31
@@ -81,6 +86,7 @@
"allwinner,sun9i-a80-usb-mod-clk" - for usb gates + resets on A80
"allwinner,sun9i-a80-usb-phy-clk" - for usb phy gates + resets on A80
"allwinner,sun4i-a10-ve-clk" - for the Video Engine clock
+ "allwinner,sun6i-a31-display-clk" - for the display clocks
Required properties for all clocks:
- reg : shall be the control register address for the clock.
diff --git a/Documentation/devicetree/bindings/cpufreq/tegra124-cpufreq.txt b/Documentation/devicetree/bindings/cpufreq/nvidia,tegra124-cpufreq.txt
similarity index 100%
rename from Documentation/devicetree/bindings/cpufreq/tegra124-cpufreq.txt
rename to Documentation/devicetree/bindings/cpufreq/nvidia,tegra124-cpufreq.txt
diff --git a/Documentation/devicetree/bindings/display/brcm,bcm-vc4.txt b/Documentation/devicetree/bindings/display/brcm,bcm-vc4.txt
index 9f97df4..a5ea451 100644
--- a/Documentation/devicetree/bindings/display/brcm,bcm-vc4.txt
+++ b/Documentation/devicetree/bindings/display/brcm,bcm-vc4.txt
@@ -35,12 +35,22 @@
as an interrupt/status bit in the HDMI controller
itself). See bindings/pinctrl/brcm,bcm2835-gpio.txt
+Required properties for DPI:
+- compatible: Should be "brcm,bcm2835-dpi"
+- reg: Physical base address and length of the registers
+- clocks: a) core: The core clock the unit runs on
+ b) pixel: The pixel clock that feeds the pixelvalve
+- port: Port node with a single endpoint connecting to the panel
+ device, as defined in [1]
+
Required properties for V3D:
- compatible: Should be "brcm,bcm2835-v3d"
- reg: Physical base address and length of the V3D's registers
- interrupts: The interrupt number
See bindings/interrupt-controller/brcm,bcm2835-armctrl-ic.txt
+[1] Documentation/devicetree/bindings/media/video-interfaces.txt
+
Example:
pixelvalve@7e807000 {
compatible = "brcm,bcm2835-pixelvalve2";
@@ -66,6 +76,22 @@
clock-names = "pixel", "hdmi";
};
+dpi: dpi@7e208000 {
+ compatible = "brcm,bcm2835-dpi";
+ reg = <0x7e208000 0x8c>;
+ clocks = <&clocks BCM2835_CLOCK_VPU>,
+ <&clocks BCM2835_CLOCK_DPI>;
+ clock-names = "core", "pixel";
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port {
+ dpi_out: endpoint@0 {
+ remote-endpoint = <&panel_in>;
+ };
+ };
+};
+
v3d: v3d@7ec00000 {
compatible = "brcm,bcm2835-v3d";
reg = <0x7ec00000 0x1000>;
@@ -75,3 +101,13 @@
vc4: gpu {
compatible = "brcm,bcm2835-vc4";
};
+
+panel: panel {
+ compatible = "ontat,yx700wv03", "simple-panel";
+
+ port {
+ panel_in: endpoint {
+ remote-endpoint = <&dpi_out>;
+ };
+ };
+};
diff --git a/Documentation/devicetree/bindings/display/bridge/analogix_dp.txt b/Documentation/devicetree/bindings/display/bridge/analogix_dp.txt
new file mode 100644
index 0000000..4f2ba8c
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/bridge/analogix_dp.txt
@@ -0,0 +1,52 @@
+Analogix Display Port bridge bindings
+
+Required properties for dp-controller:
+ -compatible:
+ platform specific such as:
+ * "samsung,exynos5-dp"
+ * "rockchip,rk3288-dp"
+ -reg:
+ physical base address of the controller and length
+ of memory mapped region.
+ -interrupts:
+ interrupt combiner values.
+ -clocks:
+ from common clock binding: handle to dp clock.
+ -clock-names:
+ from common clock binding: Shall be "dp".
+ -interrupt-parent:
+ phandle to Interrupt combiner node.
+ -phys:
+ from general PHY binding: the phandle for the PHY device.
+ -phy-names:
+ from general PHY binding: Should be "dp".
+
+Optional properties for dp-controller:
+ -force-hpd:
+ Indicate driver need force hpd when hpd detect failed, this
+ is used for some eDP screen which don't have hpd signal.
+ -hpd-gpios:
+ Hotplug detect GPIO.
+ Indicates which GPIO should be used for hotplug detection
+ -port@[X]: SoC specific port nodes with endpoint definitions as defined
+ in Documentation/devicetree/bindings/media/video-interfaces.txt,
+ please refer to the SoC specific binding document:
+ * Documentation/devicetree/bindings/display/exynos/exynos_dp.txt
+ * Documentation/devicetree/bindings/video/analogix_dp-rockchip.txt
+
+[1]: Documentation/devicetree/bindings/media/video-interfaces.txt
+-------------------------------------------------------------------------------
+
+Example:
+
+ dp-controller {
+ compatible = "samsung,exynos5-dp";
+ reg = <0x145b0000 0x10000>;
+ interrupts = <10 3>;
+ interrupt-parent = <&combiner>;
+ clocks = <&clock 342>;
+ clock-names = "dp";
+
+ phys = <&dp_phy>;
+ phy-names = "dp";
+ };
diff --git a/Documentation/devicetree/bindings/display/exynos/exynos5433-decon.txt b/Documentation/devicetree/bindings/display/exynos/exynos5433-decon.txt
index 377afbf..c9fd7b3 100644
--- a/Documentation/devicetree/bindings/display/exynos/exynos5433-decon.txt
+++ b/Documentation/devicetree/bindings/display/exynos/exynos5433-decon.txt
@@ -5,7 +5,8 @@
buffer to an external LCD interface.
Required properties:
-- compatible: value should be "samsung,exynos5433-decon";
+- compatible: value should be one of:
+ "samsung,exynos5433-decon", "samsung,exynos5433-decon-tv";
- reg: physical base address and length of the DECON registers set.
- interrupts: should contain a list of all DECON IP block interrupts in the
order: VSYNC, LCD_SYSTEM. The interrupt specifier format
@@ -16,7 +17,7 @@
- clocks: must include clock specifiers corresponding to entries in the
clock-names property.
- clock-names: list of clock names sorted in the same order as the clocks
- property. Must contain "aclk_decon", "aclk_smmu_decon0x",
+ property. Must contain "pclk", "aclk_decon", "aclk_smmu_decon0x",
"aclk_xiu_decon0x", "pclk_smmu_decon0x", clk_decon_vclk",
"sclk_decon_eclk"
- ports: contains a port which is connected to mic node. address-cells and
diff --git a/Documentation/devicetree/bindings/display/exynos/exynos_dp.txt b/Documentation/devicetree/bindings/display/exynos/exynos_dp.txt
index fe4a7a2..ade5d8e 100644
--- a/Documentation/devicetree/bindings/display/exynos/exynos_dp.txt
+++ b/Documentation/devicetree/bindings/display/exynos/exynos_dp.txt
@@ -1,20 +1,3 @@
-Device-Tree bindings for Samsung Exynos Embedded DisplayPort Transmitter(eDP)
-
-DisplayPort is industry standard to accommodate the growing board adoption
-of digital display technology within the PC and CE industries.
-It consolidates the internal and external connection methods to reduce device
-complexity and cost. It also supports necessary features for important cross
-industry applications and provides performance scalability to enable the next
-generation of displays that feature higher color depths, refresh rates, and
-display resolutions.
-
-eDP (embedded display port) device is compliant with Embedded DisplayPort
-standard as follows,
-- DisplayPort standard 1.1a for Exynos5250 and Exynos5260.
-- DisplayPort standard 1.3 for Exynos5422s and Exynos5800.
-
-eDP resides between FIMD and panel or FIMD and bridge such as LVDS.
-
The Exynos display port interface should be configured based on
the type of panel connected to it.
@@ -48,26 +31,6 @@
from general PHY binding: the phandle for the PHY device.
-phy-names:
from general PHY binding: Should be "dp".
- -samsung,color-space:
- input video data format.
- COLOR_RGB = 0, COLOR_YCBCR422 = 1, COLOR_YCBCR444 = 2
- -samsung,dynamic-range:
- dynamic range for input video data.
- VESA = 0, CEA = 1
- -samsung,ycbcr-coeff:
- YCbCr co-efficients for input video.
- COLOR_YCBCR601 = 0, COLOR_YCBCR709 = 1
- -samsung,color-depth:
- number of bits per colour component.
- COLOR_6 = 0, COLOR_8 = 1, COLOR_10 = 2, COLOR_12 = 3
- -samsung,link-rate:
- link rate supported by the panel.
- LINK_RATE_1_62GBPS = 0x6, LINK_RATE_2_70GBPS = 0x0A
- -samsung,lane-count:
- number of lanes supported by the panel.
- LANE_COUNT1 = 1, LANE_COUNT2 = 2, LANE_COUNT4 = 4
- - display-timings: timings for the connected panel as described by
- Documentation/devicetree/bindings/display/display-timing.txt
Optional properties for dp-controller:
-interlaced:
@@ -83,17 +46,31 @@
Hotplug detect GPIO.
Indicates which GPIO should be used for hotplug
detection
-Video interfaces:
- Device node can contain video interface port nodes according to [1].
- The following are properties specific to those nodes:
+ -video interfaces: Device node can contain video interface port
+ nodes according to [1].
+ - display-timings: timings for the connected panel as described by
+ Documentation/devicetree/bindings/display/panel/display-timing.txt
- endpoint node connected to bridge or panel node:
- - remote-endpoint: specifies the endpoint in panel or bridge node.
- This node is required in all kinds of exynos dp
- to represent the connection between dp and bridge
- or dp and panel.
+For the below properties, please refer to Analogix DP binding document:
+ * Documentation/devicetree/bindings/display/bridge/analogix_dp.txt
+ -phys (required)
+ -phy-names (required)
+ -hpd-gpios (optional)
+ force-hpd (optional)
-[1]: Documentation/devicetree/bindings/media/video-interfaces.txt
+Deprecated properties for DisplayPort:
+-interlaced: deprecated prop that can parsed from drm_display_mode.
+-vsync-active-high: deprecated prop that can parsed from drm_display_mode.
+-hsync-active-high: deprecated prop that can parsed from drm_display_mode.
+-samsung,ycbcr-coeff: deprecated prop that can parsed from drm_display_mode.
+-samsung,dynamic-range: deprecated prop that can parsed from drm_display_mode.
+-samsung,color-space: deprecated prop that can parsed from drm_display_info.
+-samsung,color-depth: deprecated prop that can parsed from drm_display_info.
+-samsung,link-rate: deprecated prop that can reading from monitor by dpcd method.
+-samsung,lane-count: deprecated prop that can reading from monitor by dpcd method.
+-samsung,hpd-gpio: deprecated name for hpd-gpios.
+
+-------------------------------------------------------------------------------
Example:
@@ -112,13 +89,6 @@
Board Specific portion:
dp-controller {
- samsung,color-space = <0>;
- samsung,dynamic-range = <0>;
- samsung,ycbcr-coeff = <0>;
- samsung,color-depth = <1>;
- samsung,link-rate = <0x0a>;
- samsung,lane-count = <4>;
-
display-timings {
native-mode = <&lcd_timing>;
lcd_timing: 1366x768 {
@@ -135,18 +105,9 @@
};
ports {
- port {
+ port@0 {
dp_out: endpoint {
- remote-endpoint = <&dp_in>;
- };
- };
- };
-
- panel {
- ...
- port {
- dp_in: endpoint {
- remote-endpoint = <&dp_out>;
+ remote-endpoint = <&bridge_in>;
};
};
};
diff --git a/Documentation/devicetree/bindings/display/exynos/exynos_dsim.txt b/Documentation/devicetree/bindings/display/exynos/exynos_dsim.txt
index 22756b3..a782659 100644
--- a/Documentation/devicetree/bindings/display/exynos/exynos_dsim.txt
+++ b/Documentation/devicetree/bindings/display/exynos/exynos_dsim.txt
@@ -41,7 +41,7 @@
endpoint node connected from mic node (reg = 0):
- remote-endpoint: specifies the endpoint in mic node. This node is required
for Exynos5433 mipi dsi. So mic can access to panel node
- thoughout this dsi node.
+ throughout this dsi node.
endpoint node connected to panel node (reg = 1):
- remote-endpoint: specifies the endpoint in panel node. This node is
required in all kinds of exynos mipi dsi to represent
diff --git a/Documentation/devicetree/bindings/display/exynos/exynos_hdmi.txt b/Documentation/devicetree/bindings/display/exynos/exynos_hdmi.txt
index d474f59..a2ec4c1 100644
--- a/Documentation/devicetree/bindings/display/exynos/exynos_hdmi.txt
+++ b/Documentation/devicetree/bindings/display/exynos/exynos_hdmi.txt
@@ -5,6 +5,7 @@
1) "samsung,exynos4210-hdmi"
2) "samsung,exynos4212-hdmi"
3) "samsung,exynos5420-hdmi"
+ 4) "samsung,exynos5433-hdmi"
- reg: physical base address of the hdmi and length of memory mapped
region.
- interrupts: interrupt number to the cpu.
@@ -12,6 +13,11 @@
a) phandle of the gpio controller node.
b) pin number within the gpio controller.
c) optional flags and pull up/down.
+- ddc: phandle to the hdmi ddc node
+- phy: phandle to the hdmi phy node
+- samsung,syscon-phandle: phandle for system controller node for PMU.
+
+Required properties for Exynos 4210, 4212, 5420 and 5433:
- clocks: list of clock IDs from SoC clock driver.
a) hdmi: Gate of HDMI IP bus clock.
b) sclk_hdmi: Gate of HDMI special clock.
@@ -25,9 +31,24 @@
sclk_pixel.
- clock-names: aliases as per driver requirements for above clock IDs:
"hdmi", "sclk_hdmi", "sclk_pixel", "sclk_hdmiphy" and "mout_hdmi".
-- ddc: phandle to the hdmi ddc node
-- phy: phandle to the hdmi phy node
-- samsung,syscon-phandle: phandle for system controller node for PMU.
+
+Required properties for Exynos 5433:
+- clocks: list of clock specifiers according to common clock bindings.
+ a) hdmi_pclk: Gate of HDMI IP APB bus.
+ b) hdmi_i_pclk: Gate of HDMI-PHY IP APB bus.
+ d) i_tmds_clk: Gate of HDMI TMDS clock.
+ e) i_pixel_clk: Gate of HDMI pixel clock.
+ f) i_spdif_clk: Gate of HDMI SPDIF clock.
+ g) oscclk: Oscillator clock, used as parent of following *_user clocks
+ in case HDMI-PHY is not operational.
+ h) tmds_clko: TMDS clock generated by HDMI-PHY.
+ i) tmds_clko_user: MUX used to switch between oscclk and tmds_clko,
+ respectively if HDMI-PHY is off and operational.
+ j) pixel_clko: Pixel clock generated by HDMI-PHY.
+ k) pixel_clko_user: MUX used to switch between oscclk and pixel_clko,
+ respectively if HDMI-PHY is off and operational.
+- clock-names: aliases for above clock specfiers.
+- samsung,sysreg: handle to syscon used to control the system registers.
Example:
diff --git a/Documentation/devicetree/bindings/display/fsl,dcu.txt b/Documentation/devicetree/bindings/display/fsl,dcu.txt
index ebf1be9..ae55cde 100644
--- a/Documentation/devicetree/bindings/display/fsl,dcu.txt
+++ b/Documentation/devicetree/bindings/display/fsl,dcu.txt
@@ -6,17 +6,24 @@
* "fsl,vf610-dcu".
- reg: Address and length of the register set for dcu.
-- clocks: From common clock binding: handle to dcu clock.
-- clock-names: From common clock binding: Shall be "dcu".
+- clocks: Handle to "dcu" and "pix" clock (in the order below)
+ This can be the same clock (e.g. LS1021a)
+ See ../clocks/clock-bindings.txt for details.
+- clock-names: Should be "dcu" and "pix"
+ See ../clocks/clock-bindings.txt for details.
- big-endian Boolean property, LS1021A DCU registers are big-endian.
- fsl,panel: The phandle to panel node.
+Optional properties:
+- fsl,tcon: The phandle to the timing controller node.
+
Examples:
dcu: dcu@2ce0000 {
compatible = "fsl,ls1021a-dcu";
reg = <0x0 0x2ce0000 0x0 0x10000>;
- clocks = <&platform_clk 0>;
- clock-names = "dcu";
+ clocks = <&platform_clk 0>, <&platform_clk 0>;
+ clock-names = "dcu", "pix";
big-endian;
fsl,panel = <&panel>;
+ fsl,tcon = <&tcon>;
};
diff --git a/Documentation/devicetree/bindings/display/fsl,tcon.txt b/Documentation/devicetree/bindings/display/fsl,tcon.txt
new file mode 100644
index 0000000..6fa4ab6
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/fsl,tcon.txt
@@ -0,0 +1,18 @@
+Device Tree bindings for Freescale TCON Driver
+
+Required properties:
+- compatible: Should be one of
+ * "fsl,vf610-tcon".
+
+- reg: Address and length of the register set for tcon.
+- clocks: From common clock binding: handle to tcon ipg clock.
+- clock-names: From common clock binding: Shall be "ipg".
+
+Examples:
+timing-controller@4003d000 {
+ compatible = "fsl,vf610-tcon";
+ reg = <0x4003d000 0x1000>;
+ clocks = <&clks VF610_CLK_TCON0>;
+ clock-names = "ipg";
+ status = "okay";
+};
diff --git a/Documentation/devicetree/bindings/display/hisilicon/dw-dsi.txt b/Documentation/devicetree/bindings/display/hisilicon/dw-dsi.txt
new file mode 100644
index 0000000..d270bfe
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/hisilicon/dw-dsi.txt
@@ -0,0 +1,72 @@
+Device-Tree bindings for DesignWare DSI Host Controller v1.20a driver
+
+A DSI Host Controller resides in the middle of display controller and external
+HDMI converter or panel.
+
+Required properties:
+- compatible: value should be "hisilicon,hi6220-dsi".
+- reg: physical base address and length of dsi controller's registers.
+- clocks: contains APB clock phandle + clock-specifier pair.
+- clock-names: should be "pclk".
+- ports: contains DSI controller input and output sub port.
+ The input port connects to ADE output port with the reg value "0".
+ The output port with the reg value "1", it could connect to panel or
+ any other bridge endpoints.
+ See Documentation/devicetree/bindings/graph.txt for more device graph info.
+
+A example of HiKey board hi6220 SoC and board specific DT entry:
+Example:
+
+SoC specific:
+ dsi: dsi@f4107800 {
+ compatible = "hisilicon,hi6220-dsi";
+ reg = <0x0 0xf4107800 0x0 0x100>;
+ clocks = <&media_ctrl HI6220_DSI_PCLK>;
+ clock-names = "pclk";
+ status = "disabled";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ /* 0 for input port */
+ port@0 {
+ reg = <0>;
+ dsi_in: endpoint {
+ remote-endpoint = <&ade_out>;
+ };
+ };
+ };
+ };
+
+
+Board specific:
+ &dsi {
+ status = "ok";
+
+ ports {
+ /* 1 for output port */
+ port@1 {
+ reg = <1>;
+
+ dsi_out0: endpoint@0 {
+ remote-endpoint = <&adv7533_in>;
+ };
+ };
+ };
+ };
+
+ &i2c2 {
+ ...
+
+ adv7533: adv7533@39 {
+ ...
+
+ port {
+ adv7533_in: endpoint {
+ remote-endpoint = <&dsi_out0>;
+ };
+ };
+ };
+ };
+
diff --git a/Documentation/devicetree/bindings/display/hisilicon/hisi-ade.txt b/Documentation/devicetree/bindings/display/hisilicon/hisi-ade.txt
new file mode 100644
index 0000000..38dc9d6
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/hisilicon/hisi-ade.txt
@@ -0,0 +1,64 @@
+Device-Tree bindings for hisilicon ADE display controller driver
+
+ADE (Advanced Display Engine) is the display controller which grab image
+data from memory, do composition, do post image processing, generate RGB
+timing stream and transfer to DSI.
+
+Required properties:
+- compatible: value should be "hisilicon,hi6220-ade".
+- reg: physical base address and length of the ADE controller's registers.
+- hisilicon,noc-syscon: ADE NOC QoS syscon.
+- resets: The ADE reset controller node.
+- interrupt: the ldi vblank interrupt number used.
+- clocks: a list of phandle + clock-specifier pairs, one for each entry
+ in clock-names.
+- clock-names: should contain:
+ "clk_ade_core" for the ADE core clock.
+ "clk_codec_jpeg" for the media NOC QoS clock, which use the same clock with
+ jpeg codec.
+ "clk_ade_pix" for the ADE pixel clok.
+- assigned-clocks: Should contain "clk_ade_core" and "clk_codec_jpeg" clocks'
+ phandle + clock-specifier pairs.
+- assigned-clock-rates: clock rates, one for each entry in assigned-clocks.
+ The rate of "clk_ade_core" could be "360000000" or "180000000";
+ The rate of "clk_codec_jpeg" could be or less than "1440000000".
+ These rate values could be configured according to performance and power
+ consumption.
+- port: the output port. This contains one endpoint subnode, with its
+ remote-endpoint set to the phandle of the connected DSI input endpoint.
+ See Documentation/devicetree/bindings/graph.txt for more device graph info.
+
+Optional properties:
+- dma-coherent: Present if dma operations are coherent.
+
+
+A example of HiKey board hi6220 SoC specific DT entry:
+Example:
+
+ ade: ade@f4100000 {
+ compatible = "hisilicon,hi6220-ade";
+ reg = <0x0 0xf4100000 0x0 0x7800>;
+ reg-names = "ade_base";
+ hisilicon,noc-syscon = <&medianoc_ade>;
+ resets = <&media_ctrl MEDIA_ADE>;
+ interrupts = <0 115 4>; /* ldi interrupt */
+
+ clocks = <&media_ctrl HI6220_ADE_CORE>,
+ <&media_ctrl HI6220_CODEC_JPEG>,
+ <&media_ctrl HI6220_ADE_PIX_SRC>;
+ /*clock name*/
+ clock-names = "clk_ade_core",
+ "clk_codec_jpeg",
+ "clk_ade_pix";
+
+ assigned-clocks = <&media_ctrl HI6220_ADE_CORE>,
+ <&media_ctrl HI6220_CODEC_JPEG>;
+ assigned-clock-rates = <360000000>, <288000000>;
+ dma-coherent;
+
+ port {
+ ade_out: endpoint {
+ remote-endpoint = <&dsi_in>;
+ };
+ };
+ };
diff --git a/Documentation/devicetree/bindings/display/mediatek/mediatek,disp.txt b/Documentation/devicetree/bindings/display/mediatek/mediatek,disp.txt
new file mode 100644
index 0000000..db6e77e
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/mediatek/mediatek,disp.txt
@@ -0,0 +1,203 @@
+Mediatek display subsystem
+==========================
+
+The Mediatek display subsystem consists of various DISP function blocks in the
+MMSYS register space. The connections between them can be configured by output
+and input selectors in the MMSYS_CONFIG register space. Pixel clock and start
+of frame signal are distributed to the other function blocks by a DISP_MUTEX
+function block.
+
+All DISP device tree nodes must be siblings to the central MMSYS_CONFIG node.
+For a description of the MMSYS_CONFIG binding, see
+Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.txt.
+
+DISP function blocks
+====================
+
+A display stream starts at a source function block that reads pixel data from
+memory and ends with a sink function block that drives pixels on a display
+interface, or writes pixels back to memory. All DISP function blocks have
+their own register space, interrupt, and clock gate. The blocks that can
+access memory additionally have to list the IOMMU and local arbiter they are
+connected to.
+
+For a description of the display interface sink function blocks, see
+Documentation/devicetree/bindings/display/mediatek/mediatek,dsi.txt and
+Documentation/devicetree/bindings/display/mediatek/mediatek,dpi.txt.
+
+Required properties (all function blocks):
+- compatible: "mediatek,<chip>-disp-<function>", one of
+ "mediatek,<chip>-disp-ovl" - overlay (4 layers, blending, csc)
+ "mediatek,<chip>-disp-rdma" - read DMA / line buffer
+ "mediatek,<chip>-disp-wdma" - write DMA
+ "mediatek,<chip>-disp-color" - color processor
+ "mediatek,<chip>-disp-aal" - adaptive ambient light controller
+ "mediatek,<chip>-disp-gamma" - gamma correction
+ "mediatek,<chip>-disp-merge" - merge streams from two RDMA sources
+ "mediatek,<chip>-disp-split" - split stream to two encoders
+ "mediatek,<chip>-disp-ufoe" - data compression engine
+ "mediatek,<chip>-dsi" - DSI controller, see mediatek,dsi.txt
+ "mediatek,<chip>-dpi" - DPI controller, see mediatek,dpi.txt
+ "mediatek,<chip>-disp-mutex" - display mutex
+ "mediatek,<chip>-disp-od" - overdrive
+- reg: Physical base address and length of the function block register space
+- interrupts: The interrupt signal from the function block (required, except for
+ merge and split function blocks).
+- clocks: device clocks
+ See Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+ For most function blocks this is just a single clock input. Only the DSI and
+ DPI controller nodes have multiple clock inputs. These are documented in
+ mediatek,dsi.txt and mediatek,dpi.txt, respectively.
+
+Required properties (DMA function blocks):
+- compatible: Should be one of
+ "mediatek,<chip>-disp-ovl"
+ "mediatek,<chip>-disp-rdma"
+ "mediatek,<chip>-disp-wdma"
+- larb: Should contain a phandle pointing to the local arbiter device as defined
+ in Documentation/devicetree/bindings/soc/mediatek/mediatek,smi-larb.txt
+- iommus: Should point to the respective IOMMU block with master port as
+ argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
+ for details.
+
+Examples:
+
+mmsys: clock-controller@14000000 {
+ compatible = "mediatek,mt8173-mmsys", "syscon";
+ reg = <0 0x14000000 0 0x1000>;
+ power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+ #clock-cells = <1>;
+};
+
+ovl0: ovl@1400c000 {
+ compatible = "mediatek,mt8173-disp-ovl";
+ reg = <0 0x1400c000 0 0x1000>;
+ interrupts = <GIC_SPI 180 IRQ_TYPE_LEVEL_LOW>;
+ power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+ clocks = <&mmsys CLK_MM_DISP_OVL0>;
+ iommus = <&iommu M4U_PORT_DISP_OVL0>;
+ mediatek,larb = <&larb0>;
+};
+
+ovl1: ovl@1400d000 {
+ compatible = "mediatek,mt8173-disp-ovl";
+ reg = <0 0x1400d000 0 0x1000>;
+ interrupts = <GIC_SPI 181 IRQ_TYPE_LEVEL_LOW>;
+ power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+ clocks = <&mmsys CLK_MM_DISP_OVL1>;
+ iommus = <&iommu M4U_PORT_DISP_OVL1>;
+ mediatek,larb = <&larb4>;
+};
+
+rdma0: rdma@1400e000 {
+ compatible = "mediatek,mt8173-disp-rdma";
+ reg = <0 0x1400e000 0 0x1000>;
+ interrupts = <GIC_SPI 182 IRQ_TYPE_LEVEL_LOW>;
+ power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+ clocks = <&mmsys CLK_MM_DISP_RDMA0>;
+ iommus = <&iommu M4U_PORT_DISP_RDMA0>;
+ mediatek,larb = <&larb0>;
+};
+
+rdma1: rdma@1400f000 {
+ compatible = "mediatek,mt8173-disp-rdma";
+ reg = <0 0x1400f000 0 0x1000>;
+ interrupts = <GIC_SPI 183 IRQ_TYPE_LEVEL_LOW>;
+ power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+ clocks = <&mmsys CLK_MM_DISP_RDMA1>;
+ iommus = <&iommu M4U_PORT_DISP_RDMA1>;
+ mediatek,larb = <&larb4>;
+};
+
+rdma2: rdma@14010000 {
+ compatible = "mediatek,mt8173-disp-rdma";
+ reg = <0 0x14010000 0 0x1000>;
+ interrupts = <GIC_SPI 184 IRQ_TYPE_LEVEL_LOW>;
+ power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+ clocks = <&mmsys CLK_MM_DISP_RDMA2>;
+ iommus = <&iommu M4U_PORT_DISP_RDMA2>;
+ mediatek,larb = <&larb4>;
+};
+
+wdma0: wdma@14011000 {
+ compatible = "mediatek,mt8173-disp-wdma";
+ reg = <0 0x14011000 0 0x1000>;
+ interrupts = <GIC_SPI 185 IRQ_TYPE_LEVEL_LOW>;
+ power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+ clocks = <&mmsys CLK_MM_DISP_WDMA0>;
+ iommus = <&iommu M4U_PORT_DISP_WDMA0>;
+ mediatek,larb = <&larb0>;
+};
+
+wdma1: wdma@14012000 {
+ compatible = "mediatek,mt8173-disp-wdma";
+ reg = <0 0x14012000 0 0x1000>;
+ interrupts = <GIC_SPI 186 IRQ_TYPE_LEVEL_LOW>;
+ power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+ clocks = <&mmsys CLK_MM_DISP_WDMA1>;
+ iommus = <&iommu M4U_PORT_DISP_WDMA1>;
+ mediatek,larb = <&larb4>;
+};
+
+color0: color@14013000 {
+ compatible = "mediatek,mt8173-disp-color";
+ reg = <0 0x14013000 0 0x1000>;
+ interrupts = <GIC_SPI 187 IRQ_TYPE_LEVEL_LOW>;
+ power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+ clocks = <&mmsys CLK_MM_DISP_COLOR0>;
+};
+
+color1: color@14014000 {
+ compatible = "mediatek,mt8173-disp-color";
+ reg = <0 0x14014000 0 0x1000>;
+ interrupts = <GIC_SPI 188 IRQ_TYPE_LEVEL_LOW>;
+ power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+ clocks = <&mmsys CLK_MM_DISP_COLOR1>;
+};
+
+aal@14015000 {
+ compatible = "mediatek,mt8173-disp-aal";
+ reg = <0 0x14015000 0 0x1000>;
+ interrupts = <GIC_SPI 189 IRQ_TYPE_LEVEL_LOW>;
+ power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+ clocks = <&mmsys CLK_MM_DISP_AAL>;
+};
+
+gamma@14016000 {
+ compatible = "mediatek,mt8173-disp-gamma";
+ reg = <0 0x14016000 0 0x1000>;
+ interrupts = <GIC_SPI 190 IRQ_TYPE_LEVEL_LOW>;
+ power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+ clocks = <&mmsys CLK_MM_DISP_GAMMA>;
+};
+
+ufoe@1401a000 {
+ compatible = "mediatek,mt8173-disp-ufoe";
+ reg = <0 0x1401a000 0 0x1000>;
+ interrupts = <GIC_SPI 191 IRQ_TYPE_LEVEL_LOW>;
+ power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+ clocks = <&mmsys CLK_MM_DISP_UFOE>;
+};
+
+dsi0: dsi@1401b000 {
+ /* See mediatek,dsi.txt for details */
+};
+
+dpi0: dpi@1401d000 {
+ /* See mediatek,dpi.txt for details */
+};
+
+mutex: mutex@14020000 {
+ compatible = "mediatek,mt8173-disp-mutex";
+ reg = <0 0x14020000 0 0x1000>;
+ interrupts = <GIC_SPI 169 IRQ_TYPE_LEVEL_LOW>;
+ power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+ clocks = <&mmsys CLK_MM_MUTEX_32K>;
+};
+
+od@14023000 {
+ compatible = "mediatek,mt8173-disp-od";
+ reg = <0 0x14023000 0 0x1000>;
+ power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>;
+ clocks = <&mmsys CLK_MM_DISP_OD>;
+};
diff --git a/Documentation/devicetree/bindings/display/mediatek/mediatek,dpi.txt b/Documentation/devicetree/bindings/display/mediatek/mediatek,dpi.txt
new file mode 100644
index 0000000..b6a7e73
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/mediatek/mediatek,dpi.txt
@@ -0,0 +1,35 @@
+Mediatek DPI Device
+===================
+
+The Mediatek DPI function block is a sink of the display subsystem and
+provides 8-bit RGB/YUV444 or 8/10/10-bit YUV422 pixel data on a parallel
+output bus.
+
+Required properties:
+- compatible: "mediatek,<chip>-dpi"
+- reg: Physical base address and length of the controller's registers
+- interrupts: The interrupt signal from the function block.
+- clocks: device clocks
+ See Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- clock-names: must contain "pixel", "engine", and "pll"
+- port: Output port node with endpoint definitions as described in
+ Documentation/devicetree/bindings/graph.txt. This port should be connected
+ to the input port of an attached HDMI or LVDS encoder chip.
+
+Example:
+
+dpi0: dpi@1401d000 {
+ compatible = "mediatek,mt8173-dpi";
+ reg = <0 0x1401d000 0 0x1000>;
+ interrupts = <GIC_SPI 194 IRQ_TYPE_LEVEL_LOW>;
+ clocks = <&mmsys CLK_MM_DPI_PIXEL>,
+ <&mmsys CLK_MM_DPI_ENGINE>,
+ <&apmixedsys CLK_APMIXED_TVDPLL>;
+ clock-names = "pixel", "engine", "pll";
+
+ port {
+ dpi0_out: endpoint {
+ remote-endpoint = <&hdmi0_in>;
+ };
+ };
+};
diff --git a/Documentation/devicetree/bindings/display/mediatek/mediatek,dsi.txt b/Documentation/devicetree/bindings/display/mediatek/mediatek,dsi.txt
new file mode 100644
index 0000000..2b1585a
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/mediatek/mediatek,dsi.txt
@@ -0,0 +1,60 @@
+Mediatek DSI Device
+===================
+
+The Mediatek DSI function block is a sink of the display subsystem and can
+drive up to 4-lane MIPI DSI output. Two DSIs can be synchronized for dual-
+channel output.
+
+Required properties:
+- compatible: "mediatek,<chip>-dsi"
+- reg: Physical base address and length of the controller's registers
+- interrupts: The interrupt signal from the function block.
+- clocks: device clocks
+ See Documentation/devicetree/bindings/clock/clock-bindings.txt for details.
+- clock-names: must contain "engine", "digital", and "hs"
+- phys: phandle link to the MIPI D-PHY controller.
+- phy-names: must contain "dphy"
+- port: Output port node with endpoint definitions as described in
+ Documentation/devicetree/bindings/graph.txt. This port should be connected
+ to the input port of an attached DSI panel or DSI-to-eDP encoder chip.
+
+MIPI TX Configuration Module
+============================
+
+The MIPI TX configuration module controls the MIPI D-PHY.
+
+Required properties:
+- compatible: "mediatek,<chip>-mipi-tx"
+- reg: Physical base address and length of the controller's registers
+- clocks: PLL reference clock
+- clock-output-names: name of the output clock line to the DSI encoder
+- #clock-cells: must be <0>;
+- #phy-cells: must be <0>.
+
+Example:
+
+mipi_tx0: mipi-dphy@10215000 {
+ compatible = "mediatek,mt8173-mipi-tx";
+ reg = <0 0x10215000 0 0x1000>;
+ clocks = <&clk26m>;
+ clock-output-names = "mipi_tx0_pll";
+ #clock-cells = <0>;
+ #phy-cells = <0>;
+};
+
+dsi0: dsi@1401b000 {
+ compatible = "mediatek,mt8173-dsi";
+ reg = <0 0x1401b000 0 0x1000>;
+ interrupts = <GIC_SPI 192 IRQ_TYPE_LEVEL_LOW>;
+ clocks = <&mmsys MM_DSI0_ENGINE>, <&mmsys MM_DSI0_DIGITAL>,
+ <&mipi_tx0>;
+ clock-names = "engine", "digital", "hs";
+ phys = <&mipi_tx0>;
+ phy-names = "dphy";
+
+ port {
+ dsi0_out: endpoint {
+ remote-endpoint = <&panel_in>;
+ };
+ };
+};
diff --git a/Documentation/devicetree/bindings/display/panel/innolux,at070tn92.txt b/Documentation/devicetree/bindings/display/panel/innolux,at070tn92.txt
new file mode 100644
index 0000000..3e10cd7
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/panel/innolux,at070tn92.txt
@@ -0,0 +1,7 @@
+Innolux AT070TN92 7.0" WQVGA TFT LCD panel
+
+Required properties:
+- compatible: should be "innolux,at070tn92"
+
+This binding is compatible with the simple-panel binding, which is specified
+in simple-panel.txt in this directory.
diff --git a/Documentation/devicetree/bindings/display/panel/olimex,lcd-olinuxino-43-ts.txt b/Documentation/devicetree/bindings/display/panel/olimex,lcd-olinuxino-43-ts.txt
new file mode 100644
index 0000000..74540a0
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/panel/olimex,lcd-olinuxino-43-ts.txt
@@ -0,0 +1,7 @@
+Olimex 4.3" TFT LCD panel
+
+Required properties:
+- compatible: should be "olimex,lcd-olinuxino-43-ts"
+
+This binding is compatible with the simple-panel binding, which is specified
+in simple-panel.txt in this directory.
diff --git a/Documentation/devicetree/bindings/display/panel/ontat,yx700wv03.txt b/Documentation/devicetree/bindings/display/panel/ontat,yx700wv03.txt
new file mode 100644
index 0000000..3d8a5e0
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/panel/ontat,yx700wv03.txt
@@ -0,0 +1,7 @@
+On Tat Industrial Company 7" DPI TFT panel.
+
+Required properties:
+- compatible: should be "ontat,yx700wv03"
+
+This binding is compatible with the simple-panel binding, which is specified
+in simple-panel.txt in this directory.
diff --git a/Documentation/devicetree/bindings/display/panel/tpk,f07a-0102.txt b/Documentation/devicetree/bindings/display/panel/tpk,f07a-0102.txt
new file mode 100644
index 0000000..a2613b9
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/panel/tpk,f07a-0102.txt
@@ -0,0 +1,8 @@
+TPK U.S.A. LLC Fusion 7" integrated projected capacitive touch display with,
+800 x 480 (WVGA) LCD panel.
+
+Required properties:
+- compatible: should be "tpk,f07a-0102"
+
+This binding is compatible with the simple-panel binding, which is specified
+in simple-panel.txt in this directory.
diff --git a/Documentation/devicetree/bindings/display/panel/tpk,f10a-0102.txt b/Documentation/devicetree/bindings/display/panel/tpk,f10a-0102.txt
new file mode 100644
index 0000000..b9d0511
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/panel/tpk,f10a-0102.txt
@@ -0,0 +1,8 @@
+TPK U.S.A. LLC Fusion 10.1" integrated projected capacitive touch display with,
+1024 x 600 (WSVGA) LCD panel.
+
+Required properties:
+- compatible: should be "tpk,f10a-0102"
+
+This binding is compatible with the simple-panel binding, which is specified
+in simple-panel.txt in this directory.
diff --git a/Documentation/devicetree/bindings/display/rockchip/analogix_dp-rockchip.txt b/Documentation/devicetree/bindings/display/rockchip/analogix_dp-rockchip.txt
new file mode 100644
index 0000000..e832ff9
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/rockchip/analogix_dp-rockchip.txt
@@ -0,0 +1,92 @@
+Rockchip RK3288 specific extensions to the Analogix Display Port
+================================
+
+Required properties:
+- compatible: "rockchip,rk3288-edp";
+
+- reg: physical base address of the controller and length
+
+- clocks: from common clock binding: handle to dp clock.
+ of memory mapped region.
+
+- clock-names: from common clock binding:
+ Required elements: "dp" "pclk"
+
+- resets: Must contain an entry for each entry in reset-names.
+ See ../reset/reset.txt for details.
+
+- pinctrl-names: Names corresponding to the chip hotplug pinctrl states.
+- pinctrl-0: pin-control mode. should be <&edp_hpd>
+
+- reset-names: Must include the name "dp"
+
+- rockchip,grf: this soc should set GRF regs, so need get grf here.
+
+- ports: there are 2 port nodes with endpoint definitions as defined in
+ Documentation/devicetree/bindings/media/video-interfaces.txt.
+ Port 0: contained 2 endpoints, connecting to the output of vop.
+ Port 1: contained 1 endpoint, connecting to the input of panel.
+
+For the below properties, please refer to Analogix DP binding document:
+ * Documentation/devicetree/bindings/drm/bridge/analogix_dp.txt
+- phys (required)
+- phy-names (required)
+- hpd-gpios (optional)
+- force-hpd (optional)
+-------------------------------------------------------------------------------
+
+Example:
+ dp-controller: dp@ff970000 {
+ compatible = "rockchip,rk3288-dp";
+ reg = <0xff970000 0x4000>;
+ interrupts = <GIC_SPI 98 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&cru SCLK_EDP>, <&cru PCLK_EDP_CTRL>;
+ clock-names = "dp", "pclk";
+ phys = <&dp_phy>;
+ phy-names = "dp";
+
+ rockchip,grf = <&grf>;
+ resets = <&cru 111>;
+ reset-names = "dp";
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&edp_hpd>;
+
+ status = "disabled";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ edp_in: port@0 {
+ reg = <0>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ edp_in_vopb: endpoint@0 {
+ reg = <0>;
+ remote-endpoint = <&vopb_out_edp>;
+ };
+ edp_in_vopl: endpoint@1 {
+ reg = <1>;
+ remote-endpoint = <&vopl_out_edp>;
+ };
+ };
+
+ edp_out: port@1 {
+ reg = <1>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ edp_out_panel: endpoint {
+ reg = <0>;
+ remote-endpoint = <&panel_in_edp>
+ };
+ };
+ };
+ };
+
+ pinctrl {
+ edp {
+ edp_hpd: edp-hpd {
+ rockchip,pins = <7 11 RK_FUNC_2 &pcfg_pull_none>;
+ };
+ };
+ };
diff --git a/Documentation/devicetree/bindings/display/snps,arcpgu.txt b/Documentation/devicetree/bindings/display/snps,arcpgu.txt
new file mode 100644
index 0000000..c5c7dfd
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/snps,arcpgu.txt
@@ -0,0 +1,35 @@
+ARC PGU
+
+This is a display controller found on several development boards produced
+by Synopsys. The ARC PGU is an RGB streamer that reads the data from a
+framebuffer and sends it to a single digital encoder (usually HDMI).
+
+Required properties:
+ - compatible: "snps,arcpgu"
+ - reg: Physical base address and length of the controller's registers.
+ - clocks: A list of phandle + clock-specifier pairs, one for each
+ entry in 'clock-names'.
+ - clock-names: A list of clock names. For ARC PGU it should contain:
+ - "pxlclk" for the clock feeding the output PLL of the controller.
+
+Required sub-nodes:
+ - port: The PGU connection to an encoder chip.
+
+Example:
+
+/ {
+ ...
+
+ pgu@XXXXXXXX {
+ compatible = "snps,arcpgu";
+ reg = <0xXXXXXXXX 0x400>;
+ clocks = <&clock_node>;
+ clock-names = "pxlclk";
+
+ port {
+ pgu_output: endpoint {
+ remote-endpoint = <&hdmi_enc_input>;
+ };
+ };
+ };
+};
diff --git a/Documentation/devicetree/bindings/display/sunxi/sun4i-drm.txt b/Documentation/devicetree/bindings/display/sunxi/sun4i-drm.txt
new file mode 100644
index 0000000..df8f4ae
--- /dev/null
+++ b/Documentation/devicetree/bindings/display/sunxi/sun4i-drm.txt
@@ -0,0 +1,258 @@
+Allwinner A10 Display Pipeline
+==============================
+
+The Allwinner A10 Display pipeline is composed of several components
+that are going to be documented below:
+
+TV Encoder
+----------
+
+The TV Encoder supports the composite and VGA output. It is one end of
+the pipeline.
+
+Required properties:
+ - compatible: value should be "allwinner,sun4i-a10-tv-encoder".
+ - reg: base address and size of memory-mapped region
+ - clocks: the clocks driving the TV encoder
+ - resets: phandle to the reset controller driving the encoder
+
+- ports: A ports node with endpoint definitions as defined in
+ Documentation/devicetree/bindings/media/video-interfaces.txt. The
+ first port should be the input endpoint.
+
+TCON
+----
+
+The TCON acts as a timing controller for RGB, LVDS and TV interfaces.
+
+Required properties:
+ - compatible: value should be "allwinner,sun5i-a13-tcon".
+ - reg: base address and size of memory-mapped region
+ - interrupts: interrupt associated to this IP
+ - clocks: phandles to the clocks feeding the TCON. Three are needed:
+ - 'ahb': the interface clocks
+ - 'tcon-ch0': The clock driving the TCON channel 0
+ - 'tcon-ch1': The clock driving the TCON channel 1
+ - resets: phandles to the reset controllers driving the encoder
+ - "lcd": the reset line for the TCON channel 0
+
+ - clock-names: the clock names mentioned above
+ - reset-names: the reset names mentioned above
+ - clock-output-names: Name of the pixel clock created
+
+- ports: A ports node with endpoint definitions as defined in
+ Documentation/devicetree/bindings/media/video-interfaces.txt. The
+ first port should be the input endpoint, the second one the output
+
+ The output should have two endpoints. The first is the block
+ connected to the TCON channel 0 (usually a panel or a bridge), the
+ second the block connected to the TCON channel 1 (usually the TV
+ encoder)
+
+
+Display Engine Backend
+----------------------
+
+The display engine backend exposes layers and sprites to the
+system.
+
+Required properties:
+ - compatible: value must be one of:
+ * allwinner,sun5i-a13-display-backend
+ - reg: base address and size of the memory-mapped region.
+ - clocks: phandles to the clocks feeding the frontend and backend
+ * ahb: the backend interface clock
+ * mod: the backend module clock
+ * ram: the backend DRAM clock
+ - clock-names: the clock names mentioned above
+ - resets: phandles to the reset controllers driving the backend
+
+- ports: A ports node with endpoint definitions as defined in
+ Documentation/devicetree/bindings/media/video-interfaces.txt. The
+ first port should be the input endpoints, the second one the output
+
+Display Engine Frontend
+-----------------------
+
+The display engine frontend does formats conversion, scaling,
+deinterlacing and color space conversion.
+
+Required properties:
+ - compatible: value must be one of:
+ * allwinner,sun5i-a13-display-frontend
+ - reg: base address and size of the memory-mapped region.
+ - interrupts: interrupt associated to this IP
+ - clocks: phandles to the clocks feeding the frontend and backend
+ * ahb: the backend interface clock
+ * mod: the backend module clock
+ * ram: the backend DRAM clock
+ - clock-names: the clock names mentioned above
+ - resets: phandles to the reset controllers driving the backend
+
+- ports: A ports node with endpoint definitions as defined in
+ Documentation/devicetree/bindings/media/video-interfaces.txt. The
+ first port should be the input endpoints, the second one the outputs
+
+
+Display Engine Pipeline
+-----------------------
+
+The display engine pipeline (and its entry point, since it can be
+either directly the backend or the frontend) is represented as an
+extra node.
+
+Required properties:
+ - compatible: value must be one of:
+ * allwinner,sun5i-a13-display-engine
+
+ - allwinner,pipelines: list of phandle to the display engine
+ frontends available.
+
+Example:
+
+panel: panel {
+ compatible = "olimex,lcd-olinuxino-43-ts";
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ panel_input: endpoint {
+ remote-endpoint = <&tcon0_out_panel>;
+ };
+ };
+};
+
+tve0: tv-encoder@01c0a000 {
+ compatible = "allwinner,sun4i-a10-tv-encoder";
+ reg = <0x01c0a000 0x1000>;
+ clocks = <&ahb_gates 34>;
+ resets = <&tcon_ch0_clk 0>;
+
+ port {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ tve0_in_tcon0: endpoint@0 {
+ reg = <0>;
+ remote-endpoint = <&tcon0_out_tve0>;
+ };
+ };
+};
+
+tcon0: lcd-controller@1c0c000 {
+ compatible = "allwinner,sun5i-a13-tcon";
+ reg = <0x01c0c000 0x1000>;
+ interrupts = <44>;
+ resets = <&tcon_ch0_clk 1>;
+ reset-names = "lcd";
+ clocks = <&ahb_gates 36>,
+ <&tcon_ch0_clk>,
+ <&tcon_ch1_clk>;
+ clock-names = "ahb",
+ "tcon-ch0",
+ "tcon-ch1";
+ clock-output-names = "tcon-pixel-clock";
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ tcon0_in: port@0 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0>;
+
+ tcon0_in_be0: endpoint@0 {
+ reg = <0>;
+ remote-endpoint = <&be0_out_tcon0>;
+ };
+ };
+
+ tcon0_out: port@1 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <1>;
+
+ tcon0_out_panel: endpoint@0 {
+ reg = <0>;
+ remote-endpoint = <&panel_input>;
+ };
+
+ tcon0_out_tve0: endpoint@1 {
+ reg = <1>;
+ remote-endpoint = <&tve0_in_tcon0>;
+ };
+ };
+ };
+};
+
+fe0: display-frontend@1e00000 {
+ compatible = "allwinner,sun5i-a13-display-frontend";
+ reg = <0x01e00000 0x20000>;
+ interrupts = <47>;
+ clocks = <&ahb_gates 46>, <&de_fe_clk>,
+ <&dram_gates 25>;
+ clock-names = "ahb", "mod",
+ "ram";
+ resets = <&de_fe_clk>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ fe0_out: port@1 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <1>;
+
+ fe0_out_be0: endpoint {
+ remote-endpoint = <&be0_in_fe0>;
+ };
+ };
+ };
+};
+
+be0: display-backend@1e60000 {
+ compatible = "allwinner,sun5i-a13-display-backend";
+ reg = <0x01e60000 0x10000>;
+ clocks = <&ahb_gates 44>, <&de_be_clk>,
+ <&dram_gates 26>;
+ clock-names = "ahb", "mod",
+ "ram";
+ resets = <&de_be_clk>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ be0_in: port@0 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0>;
+
+ be0_in_fe0: endpoint@0 {
+ reg = <0>;
+ remote-endpoint = <&fe0_out_be0>;
+ };
+ };
+
+ be0_out: port@1 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <1>;
+
+ be0_out_tcon0: endpoint@0 {
+ reg = <0>;
+ remote-endpoint = <&tcon0_in_be0>;
+ };
+ };
+ };
+};
+
+display-engine {
+ compatible = "allwinner,sun5i-a13-display-engine";
+ allwinner,pipelines = <&fe0>;
+};
diff --git a/Documentation/devicetree/bindings/dma/brcm,bcm2835-dma.txt b/Documentation/devicetree/bindings/dma/brcm,bcm2835-dma.txt
index 1396078..baf9b34 100644
--- a/Documentation/devicetree/bindings/dma/brcm,bcm2835-dma.txt
+++ b/Documentation/devicetree/bindings/dma/brcm,bcm2835-dma.txt
@@ -12,6 +12,10 @@
- reg: Should contain DMA registers location and length.
- interrupts: Should contain the DMA interrupts associated
to the DMA channels in ascending order.
+- interrupt-names: Should contain the names of the interrupt
+ in the form "dmaXX".
+ Use "dma-shared-all" for the common interrupt line
+ that is shared by all dma channels.
- #dma-cells: Must be <1>, the cell in the dmas property of the
client device represents the DREQ number.
- brcm,dma-channel-mask: Bit mask representing the channels
@@ -34,13 +38,35 @@
<1 24>,
<1 25>,
<1 26>,
+ /* dma channel 11-14 share one irq */
<1 27>,
+ <1 27>,
+ <1 27>,
+ <1 27>,
+ /* unused shared irq for all channels */
<1 28>;
+ interrupt-names = "dma0",
+ "dma1",
+ "dma2",
+ "dma3",
+ "dma4",
+ "dma5",
+ "dma6",
+ "dma7",
+ "dma8",
+ "dma9",
+ "dma10",
+ "dma11",
+ "dma12",
+ "dma13",
+ "dma14",
+ "dma-shared-all";
#dma-cells = <1>;
brcm,dma-channel-mask = <0x7f35>;
};
+
DMA clients connected to the BCM2835 DMA controller must use the format
described in the dma.txt file, using a two-cell specifier for each channel.
diff --git a/Documentation/devicetree/bindings/dma/mv-xor.txt b/Documentation/devicetree/bindings/dma/mv-xor.txt
index 276ef81..c075f59 100644
--- a/Documentation/devicetree/bindings/dma/mv-xor.txt
+++ b/Documentation/devicetree/bindings/dma/mv-xor.txt
@@ -1,7 +1,10 @@
* Marvell XOR engines
Required properties:
-- compatible: Should be "marvell,orion-xor" or "marvell,armada-380-xor"
+- compatible: Should be one of the following:
+ - "marvell,orion-xor"
+ - "marvell,armada-380-xor"
+ - "marvell,armada-3700-xor".
- reg: Should contain registers location and length (two sets)
the first set is the low registers, the second set the high
registers for the XOR engine.
diff --git a/Documentation/devicetree/bindings/dma/tegra20-apbdma.txt b/Documentation/devicetree/bindings/dma/nvidia,tegra20-apbdma.txt
similarity index 100%
rename from Documentation/devicetree/bindings/dma/tegra20-apbdma.txt
rename to Documentation/devicetree/bindings/dma/nvidia,tegra20-apbdma.txt
diff --git a/Documentation/devicetree/bindings/dma/nvidia,tegra210-adma.txt b/Documentation/devicetree/bindings/dma/nvidia,tegra210-adma.txt
new file mode 100644
index 0000000..1e1dc8f9
--- /dev/null
+++ b/Documentation/devicetree/bindings/dma/nvidia,tegra210-adma.txt
@@ -0,0 +1,55 @@
+* NVIDIA Tegra Audio DMA (ADMA) controller
+
+The Tegra Audio DMA controller that is used for transferring data
+between system memory and the Audio Processing Engine (APE).
+
+Required properties:
+- compatible: Must be "nvidia,tegra210-adma".
+- reg: Should contain DMA registers location and length. This should be
+ a single entry that includes all of the per-channel registers in one
+ contiguous bank.
+- interrupt-parent: Phandle to the interrupt parent controller.
+- interrupts: Should contain all of the per-channel DMA interrupts in
+ ascending order with respect to the DMA channel index.
+- clocks: Must contain one entry for the ADMA module clock
+ (TEGRA210_CLK_D_AUDIO).
+- clock-names: Must contain the name "d_audio" for the corresponding
+ 'clocks' entry.
+- #dma-cells : Must be 1. The first cell denotes the receive/transmit
+ request number and should be between 1 and the maximum number of
+ requests supported. This value corresponds to the RX/TX_REQUEST_SELECT
+ fields in the ADMA_CHn_CTRL register.
+
+
+Example:
+
+adma: dma@702e2000 {
+ compatible = "nvidia,tegra210-adma";
+ reg = <0x0 0x702e2000 0x0 0x2000>;
+ interrupt-parent = <&tegra_agic>;
+ interrupts = <GIC_SPI 24 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 25 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 26 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 27 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 28 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 29 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 30 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 31 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 32 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 33 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 34 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 35 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 36 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 37 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 38 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 39 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 40 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 41 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 42 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 43 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 44 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 45 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&tegra_car TEGRA210_CLK_D_AUDIO>;
+ clock-names = "d_audio";
+ #dma-cells = <1>;
+};
diff --git a/Documentation/devicetree/bindings/dma/qcom_bam_dma.txt b/Documentation/devicetree/bindings/dma/qcom_bam_dma.txt
index 1c9d48e..9cbf5d9 100644
--- a/Documentation/devicetree/bindings/dma/qcom_bam_dma.txt
+++ b/Documentation/devicetree/bindings/dma/qcom_bam_dma.txt
@@ -13,6 +13,8 @@
- clock-names: must contain "bam_clk" entry
- qcom,ee : indicates the active Execution Environment identifier (0-7) used in
the secure world.
+- qcom,controlled-remotely : optional, indicates that the bam is controlled by
+ remote proccessor i.e. execution environment.
Example:
diff --git a/Documentation/devicetree/bindings/dma/snps-dma.txt b/Documentation/devicetree/bindings/dma/snps-dma.txt
index c261598..0f55832 100644
--- a/Documentation/devicetree/bindings/dma/snps-dma.txt
+++ b/Documentation/devicetree/bindings/dma/snps-dma.txt
@@ -13,6 +13,11 @@
- chan_priority: priority of channels. 0 (default): increase from chan 0->n, 1:
increase from chan n->0
- block_size: Maximum block size supported by the controller
+- data-width: Maximum data width supported by hardware per AHB master
+ (in bytes, power of 2)
+
+
+Deprecated properties:
- data_width: Maximum data width supported by hardware per AHB master
(0 - 8bits, 1 - 16bits, ..., 5 - 256bits)
@@ -38,7 +43,7 @@
chan_allocation_order = <1>;
chan_priority = <1>;
block_size = <0xfff>;
- data_width = <3 3>;
+ data-width = <8 8>;
};
DMA clients connected to the Designware DMA controller must use the format
@@ -47,8 +52,8 @@
1. A phandle pointing to the DMA controller
2. The DMA request line number
-3. Source master for transfers on allocated channel
-4. Destination master for transfers on allocated channel
+3. Memory master for transfers on allocated channel
+4. Peripheral master for transfers on allocated channel
Example:
diff --git a/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt b/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt
index 2291c40..3cf0072 100644
--- a/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt
+++ b/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt
@@ -7,7 +7,7 @@
- compatible: Should be "xlnx,axi-dma-1.00.a"
- #dma-cells: Should be <1>, see "dmas" property below
- reg: Should contain DMA registers location and length.
-- dma-channel child node: Should have atleast one channel and can have upto
+- dma-channel child node: Should have at least one channel and can have up to
two channels per device. This node specifies the properties of each
DMA channel (see child node properties below).
diff --git a/Documentation/devicetree/bindings/dma/xilinx/xilinx_vdma.txt b/Documentation/devicetree/bindings/dma/xilinx/xilinx_vdma.txt
index e4c4d47..a1f2683 100644
--- a/Documentation/devicetree/bindings/dma/xilinx/xilinx_vdma.txt
+++ b/Documentation/devicetree/bindings/dma/xilinx/xilinx_vdma.txt
@@ -3,18 +3,44 @@
as two channels, one is to transmit to the video device and another is
to receive from the video device.
+Xilinx AXI DMA engine, it does transfers between memory and AXI4 stream
+target devices. It can be configured to have one channel or two channels.
+If configured as two channels, one is to transmit to the device and another
+is to receive from the device.
+
+Xilinx AXI CDMA engine, it does transfers between memory-mapped source
+address and a memory-mapped destination address.
+
Required properties:
-- compatible: Should be "xlnx,axi-vdma-1.00.a"
+- compatible: Should be "xlnx,axi-vdma-1.00.a" or "xlnx,axi-dma-1.00.a" or
+ "xlnx,axi-cdma-1.00.a""
- #dma-cells: Should be <1>, see "dmas" property below
- reg: Should contain VDMA registers location and length.
-- xlnx,num-fstores: Should be the number of framebuffers as configured in h/w.
+- xlnx,addrwidth: Should be the vdma addressing size in bits(ex: 32 bits).
+- dma-ranges: Should be as the following <dma_addr cpu_addr max_len>.
- dma-channel child node: Should have at least one channel and can have up to
two channels per device. This node specifies the properties of each
DMA channel (see child node properties below).
+- clocks: Input clock specifier. Refer to common clock bindings.
+- clock-names: List of input clocks
+ For VDMA:
+ Required elements: "s_axi_lite_aclk"
+ Optional elements: "m_axi_mm2s_aclk" "m_axi_s2mm_aclk",
+ "m_axis_mm2s_aclk", "s_axis_s2mm_aclk"
+ For CDMA:
+ Required elements: "s_axi_lite_aclk", "m_axi_aclk"
+ FOR AXIDMA:
+ Required elements: "s_axi_lite_aclk"
+ Optional elements: "m_axi_mm2s_aclk", "m_axi_s2mm_aclk",
+ "m_axi_sg_aclk"
+
+Required properties for VDMA:
+- xlnx,num-fstores: Should be the number of framebuffers as configured in h/w.
Optional properties:
- xlnx,include-sg: Tells configured for Scatter-mode in
the hardware.
+Optional properties for VDMA:
- xlnx,flush-fsync: Tells which channel to Flush on Frame sync.
It takes following values:
{1}, flush both channels
@@ -31,6 +57,7 @@
Optional child node properties:
- xlnx,include-dre: Tells hardware is configured for Data
Realignment Engine.
+Optional child node properties for VDMA:
- xlnx,genlock-mode: Tells Genlock synchronization is
enabled/disabled in hardware.
@@ -41,8 +68,13 @@
compatible = "xlnx,axi-vdma-1.00.a";
#dma_cells = <1>;
reg = < 0x40030000 0x10000 >;
+ dma-ranges = <0x00000000 0x00000000 0x40000000>;
xlnx,num-fstores = <0x8>;
xlnx,flush-fsync = <0x1>;
+ xlnx,addrwidth = <0x20>;
+ clocks = <&clk 0>, <&clk 1>, <&clk 2>, <&clk 3>, <&clk 4>;
+ clock-names = "s_axi_lite_aclk", "m_axi_mm2s_aclk", "m_axi_s2mm_aclk",
+ "m_axis_mm2s_aclk", "s_axis_s2mm_aclk";
dma-channel@40030000 {
compatible = "xlnx,axi-vdma-mm2s-channel";
interrupts = < 0 54 4 >;
diff --git a/Documentation/devicetree/bindings/gpio/ibm,ppc4xx-gpio.txt b/Documentation/devicetree/bindings/gpio/ibm,ppc4xx-gpio.txt
new file mode 100644
index 0000000..d58b395
--- /dev/null
+++ b/Documentation/devicetree/bindings/gpio/ibm,ppc4xx-gpio.txt
@@ -0,0 +1,24 @@
+* IBM/AMCC/APM GPIO Controller for PowerPC 4XX series and compatible SoCs
+
+All GPIOs are pin-shared with other functions. DCRs control whether a
+particular pin that has GPIO capabilities acts as a GPIO or is used for
+another purpose. GPIO outputs are separately programmable to emulate
+an open-drain driver.
+
+Required properties:
+ - compatible: must be "ibm,ppc4xx-gpio"
+ - reg: address and length of the register set for the device
+ - #gpio-cells: must be set to 2. The first cell is the pin number
+ and the second cell is used to specify the gpio polarity:
+ 0 = active high
+ 1 = active low
+ - gpio-controller: marks the device node as a gpio controller.
+
+Example:
+
+GPIO0: gpio@ef600b00 {
+ compatible = "ibm,ppc4xx-gpio";
+ reg = <0xef600b00 0x00000048>;
+ #gpio-cells = <2>;
+ gpio-controller;
+};
diff --git a/Documentation/devicetree/bindings/i2c/i2c-octeon.txt b/Documentation/devicetree/bindings/i2c/i2c-octeon.txt
index dced82e..872d485 100644
--- a/Documentation/devicetree/bindings/i2c/i2c-octeon.txt
+++ b/Documentation/devicetree/bindings/i2c/i2c-octeon.txt
@@ -4,6 +4,12 @@
Compatibility with all cn3XXX, cn5XXX and cn6XXX SOCs.
+ or
+
+ compatible: "cavium,octeon-7890-twsi"
+
+ Compatibility with cn78XX SOCs.
+
- reg: The base address of the TWSI/I2C bus controller register bank.
- #address-cells: Must be <1>.
diff --git a/Documentation/devicetree/bindings/i2c/i2c-rcar.txt b/Documentation/devicetree/bindings/i2c/i2c-rcar.txt
index cf8bfc9..5f0cb50 100644
--- a/Documentation/devicetree/bindings/i2c/i2c-rcar.txt
+++ b/Documentation/devicetree/bindings/i2c/i2c-rcar.txt
@@ -19,6 +19,9 @@
- clock-frequency: desired I2C bus clock frequency in Hz. The absence of this
property indicates the default frequency 100 kHz.
- clocks: clock specifier.
+- dmas: Must contain a list of two references to DMA specifiers, one for
+ transmission, and one for reception.
+- dma-names: Must contain a list of two DMA names, "tx" and "rx".
- i2c-scl-falling-time-ns: see i2c.txt
- i2c-scl-internal-delay-ns: see i2c.txt
diff --git a/Documentation/devicetree/bindings/iio/accel/mma8452.txt b/Documentation/devicetree/bindings/iio/accel/mma8452.txt
index 165937e..45f5c5c 100644
--- a/Documentation/devicetree/bindings/iio/accel/mma8452.txt
+++ b/Documentation/devicetree/bindings/iio/accel/mma8452.txt
@@ -1,4 +1,4 @@
-Freescale MMA8451Q, MMA8452Q, MMA8453Q, MMA8652FC or MMA8653FC
+Freescale MMA8451Q, MMA8452Q, MMA8453Q, MMA8652FC, MMA8653FC or FXLS8471Q
triaxial accelerometer
Required properties:
@@ -9,6 +9,7 @@
* "fsl,mma8453"
* "fsl,mma8652"
* "fsl,mma8653"
+ * "fsl,fxls8471"
- reg: the I2C address of the chip
diff --git a/Documentation/devicetree/bindings/iio/adc/lpc1850-adc.txt b/Documentation/devicetree/bindings/iio/adc/lpc1850-adc.txt
new file mode 100644
index 0000000..0bcae51
--- /dev/null
+++ b/Documentation/devicetree/bindings/iio/adc/lpc1850-adc.txt
@@ -0,0 +1,21 @@
+NXP LPC1850 ADC bindings
+
+Required properties:
+- compatible: Should be "nxp,lpc1850-adc"
+- reg: Offset and length of the register set for the ADC device
+- interrupts: The interrupt number for the ADC device
+- clocks: The root clock of the ADC controller
+- vref-supply: The regulator supply ADC reference voltage
+- resets: phandle to reset controller and line specifier
+
+Example:
+
+adc0: adc@400e3000 {
+ compatible = "nxp,lpc1850-adc";
+ reg = <0x400e3000 0x1000>;
+ interrupts = <17>;
+ clocks = <&ccu1 CLK_APB3_ADC0>;
+ vref-supply = <®_vdda>;
+ resets = <&rgu 40>;
+ status = "disabled";
+};
diff --git a/Documentation/devicetree/bindings/staging/iio/adc/mxs-lradc.txt b/Documentation/devicetree/bindings/iio/adc/mxs-lradc.txt
similarity index 100%
rename from Documentation/devicetree/bindings/staging/iio/adc/mxs-lradc.txt
rename to Documentation/devicetree/bindings/iio/adc/mxs-lradc.txt
diff --git a/Documentation/devicetree/bindings/iio/adc/rockchip-saradc.txt b/Documentation/devicetree/bindings/iio/adc/rockchip-saradc.txt
index a9a5fe1..bf99e2f 100644
--- a/Documentation/devicetree/bindings/iio/adc/rockchip-saradc.txt
+++ b/Documentation/devicetree/bindings/iio/adc/rockchip-saradc.txt
@@ -1,7 +1,11 @@
Rockchip Successive Approximation Register (SAR) A/D Converter bindings
Required properties:
-- compatible: Should be "rockchip,saradc" or "rockchip,rk3066-tsadc"
+- compatible: should be "rockchip,<name>-saradc" or "rockchip,rk3066-tsadc"
+ - "rockchip,saradc": for rk3188, rk3288
+ - "rockchip,rk3066-tsadc": for rk3036
+ - "rockchip,rk3399-saradc": for rk3399
+
- reg: physical base address of the controller and length of memory mapped
region.
- interrupts: The interrupt number to the cpu. The interrupt specifier format
diff --git a/Documentation/devicetree/bindings/iio/dac/ad5592r.txt b/Documentation/devicetree/bindings/iio/dac/ad5592r.txt
new file mode 100644
index 0000000..989f96f
--- /dev/null
+++ b/Documentation/devicetree/bindings/iio/dac/ad5592r.txt
@@ -0,0 +1,155 @@
+Analog Devices AD5592R/AD5593R DAC/ADC device driver
+
+Required properties for the AD5592R:
+ - compatible: Must be "adi,ad5592r"
+ - reg: SPI chip select number for the device
+ - spi-max-frequency: Max SPI frequency to use (< 30000000)
+ - spi-cpol: The AD5592R requires inverse clock polarity (CPOL) mode
+
+Required properties for the AD5593R:
+ - compatible: Must be "adi,ad5593r"
+ - reg: I2C address of the device
+
+Required properties for all supported chips:
+ - #address-cells: Should be 1.
+ - #size-cells: Should be 0.
+ - channel nodes:
+ Each child node represents one channel and has the following
+ Required properties:
+ * reg: Pin on which this channel is connected to.
+ * adi,mode: Mode or function of this channel.
+ Macros specifying the valid values
+ can be found in <dt-bindings/iio/adi,ad5592r.h>.
+
+ The following values are currently supported:
+ * CH_MODE_UNUSED (the pin is unused)
+ * CH_MODE_ADC (the pin is ADC input)
+ * CH_MODE_DAC (the pin is DAC output)
+ * CH_MODE_DAC_AND_ADC (the pin is DAC output
+ but can be monitored by an ADC, since
+ there is no disadvantage this
+ this should be considered as the
+ preferred DAC mode)
+ * CH_MODE_GPIO (the pin is registered
+ with GPIOLIB)
+ Optional properties:
+ * adi,off-state: State of this channel when unused or the
+ device gets removed. Macros specifying the
+ valid values can be found in
+ <dt-bindings/iio/adi,ad5592r.h>.
+
+ * CH_OFFSTATE_PULLDOWN (the pin is pulled down)
+ * CH_OFFSTATE_OUT_LOW (the pin is output low)
+ * CH_OFFSTATE_OUT_HIGH (the pin is output high)
+ * CH_OFFSTATE_OUT_TRISTATE (the pin is
+ tristated output)
+
+
+Optional properties:
+ - vref-supply: Phandle to the external reference voltage supply. This should
+ only be set if there is an external reference voltage connected to the VREF
+ pin. If the property is not set the internal 2.5V reference is used.
+ - reset-gpios : GPIO spec for the RESET pin. If specified, it will be
+ asserted during driver probe.
+ - gpio-controller: Marks the device node as a GPIO controller.
+ - #gpio-cells: Should be 2. The first cell is the GPIO number and the second
+ cell specifies GPIO flags, as defined in <dt-bindings/gpio/gpio.h>.
+
+AD5592R Example:
+
+ #include <dt-bindings/iio/adi,ad5592r.h>
+
+ vref: regulator-vref {
+ compatible = "regulator-fixed";
+ regulator-name = "vref-ad559x";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ };
+
+ ad5592r@0 {
+ #size-cells = <0>;
+ #address-cells = <1>;
+ #gpio-cells = <2>;
+ compatible = "adi,ad5592r";
+ reg = <0>;
+
+ spi-max-frequency = <1000000>;
+ spi-cpol;
+
+ vref-supply = <&vref>; /* optional */
+ reset-gpios = <&gpio0 86 0>; /* optional */
+ gpio-controller;
+
+ channel@0 {
+ reg = <0>;
+ adi,mode = <CH_MODE_DAC>;
+ };
+ channel@1 {
+ reg = <1>;
+ adi,mode = <CH_MODE_ADC>;
+ };
+ channel@2 {
+ reg = <2>;
+ adi,mode = <CH_MODE_DAC_AND_ADC>;
+ };
+ channel@3 {
+ reg = <3>;
+ adi,mode = <CH_MODE_DAC_AND_ADC>;
+ adi,off-state = <CH_OFFSTATE_PULLDOWN>;
+ };
+ channel@4 {
+ reg = <4>;
+ adi,mode = <CH_MODE_UNUSED>;
+ adi,off-state = <CH_OFFSTATE_PULLDOWN>;
+ };
+ channel@5 {
+ reg = <5>;
+ adi,mode = <CH_MODE_GPIO>;
+ adi,off-state = <CH_OFFSTATE_PULLDOWN>;
+ };
+ channel@6 {
+ reg = <6>;
+ adi,mode = <CH_MODE_GPIO>;
+ adi,off-state = <CH_OFFSTATE_PULLDOWN>;
+ };
+ channel@7 {
+ reg = <7>;
+ adi,mode = <CH_MODE_GPIO>;
+ adi,off-state = <CH_OFFSTATE_PULLDOWN>;
+ };
+ };
+
+AD5593R Example:
+
+ #include <dt-bindings/iio/adi,ad5592r.h>
+
+ ad5593r@10 {
+ #size-cells = <0>;
+ #address-cells = <1>;
+ #gpio-cells = <2>;
+ compatible = "adi,ad5593r";
+ reg = <0x10>;
+ gpio-controller;
+
+ channel@0 {
+ reg = <0>;
+ adi,mode = <CH_MODE_DAC>;
+ adi,off-state = <CH_OFFSTATE_PULLDOWN>;
+ };
+ channel@1 {
+ reg = <1>;
+ adi,mode = <CH_MODE_ADC>;
+ adi,off-state = <CH_OFFSTATE_PULLDOWN>;
+ };
+ channel@2 {
+ reg = <2>;
+ adi,mode = <CH_MODE_DAC_AND_ADC>;
+ adi,off-state = <CH_OFFSTATE_PULLDOWN>;
+ };
+ channel@6 {
+ reg = <6>;
+ adi,mode = <CH_MODE_GPIO>;
+ adi,off-state = <CH_OFFSTATE_PULLDOWN>;
+ };
+ };
diff --git a/Documentation/devicetree/bindings/iio/dac/lpc1850-dac.txt b/Documentation/devicetree/bindings/iio/dac/lpc1850-dac.txt
new file mode 100644
index 0000000..7d6647d
--- /dev/null
+++ b/Documentation/devicetree/bindings/iio/dac/lpc1850-dac.txt
@@ -0,0 +1,20 @@
+NXP LPC1850 DAC bindings
+
+Required properties:
+- compatible: Should be "nxp,lpc1850-dac"
+- reg: Offset and length of the register set for the ADC device
+- interrupts: The interrupt number for the ADC device
+- clocks: The root clock of the ADC controller
+- vref-supply: The regulator supply ADC reference voltage
+- resets: phandle to reset controller and line specifier
+
+Example:
+dac: dac@400e1000 {
+ compatible = "nxp,lpc1850-dac";
+ reg = <0x400e1000 0x1000>;
+ interrupts = <0>;
+ clocks = <&ccu1 CLK_APB3_DAC>;
+ vref-supply = <®_vdda>;
+ resets = <&rgu 42>;
+ status = "disabled";
+};
diff --git a/Documentation/devicetree/bindings/iio/imu/inv_mpu6050.txt b/Documentation/devicetree/bindings/iio/imu/inv_mpu6050.txt
index e4d8f1c..a9fc11e 100644
--- a/Documentation/devicetree/bindings/iio/imu/inv_mpu6050.txt
+++ b/Documentation/devicetree/bindings/iio/imu/inv_mpu6050.txt
@@ -8,10 +8,23 @@
- interrupt-parent : should be the phandle for the interrupt controller
- interrupts : interrupt mapping for GPIO IRQ
+Optional properties:
+ - mount-matrix: an optional 3x3 mounting rotation matrix
+
+
Example:
mpu6050@68 {
compatible = "invensense,mpu6050";
reg = <0x68>;
interrupt-parent = <&gpio1>;
interrupts = <18 1>;
+ mount-matrix = "-0.984807753012208", /* x0 */
+ "0", /* y0 */
+ "-0.173648177666930", /* z0 */
+ "0", /* x1 */
+ "-1", /* y1 */
+ "0", /* z1 */
+ "-0.173648177666930", /* x2 */
+ "0", /* y2 */
+ "0.984807753012208"; /* z2 */
};
diff --git a/Documentation/devicetree/bindings/iio/magnetometer/ak8975.txt b/Documentation/devicetree/bindings/iio/magnetometer/ak8975.txt
index 011679f..e1e7dd32 100644
--- a/Documentation/devicetree/bindings/iio/magnetometer/ak8975.txt
+++ b/Documentation/devicetree/bindings/iio/magnetometer/ak8975.txt
@@ -8,6 +8,8 @@
Optional properties:
- gpios : should be device tree identifier of the magnetometer DRDY pin
+ - vdd-supply: an optional regulator that needs to be on to provide VDD
+ - mount-matrix: an optional 3x3 mounting rotation matrix
Example:
@@ -15,4 +17,14 @@
compatible = "asahi-kasei,ak8975";
reg = <0x0c>;
gpios = <&gpj0 7 0>;
+ vdd-supply = <&ldo_3v3_gnss>;
+ mount-matrix = "-0.984807753012208", /* x0 */
+ "0", /* y0 */
+ "-0.173648177666930", /* z0 */
+ "0", /* x1 */
+ "-1", /* y1 */
+ "0", /* z1 */
+ "-0.173648177666930", /* x2 */
+ "0", /* y2 */
+ "0.984807753012208"; /* z2 */
};
diff --git a/Documentation/devicetree/bindings/iio/potentiometer/ds1803.txt b/Documentation/devicetree/bindings/iio/potentiometer/ds1803.txt
new file mode 100644
index 0000000..df77bf5
--- /dev/null
+++ b/Documentation/devicetree/bindings/iio/potentiometer/ds1803.txt
@@ -0,0 +1,21 @@
+* Maxim Integrated DS1803 digital potentiometer driver
+
+The node for this driver must be a child node of a I2C controller, hence
+all mandatory properties for your controller must be specified. See directory:
+
+ Documentation/devicetree/bindings/i2c
+
+for more details.
+
+Required properties:
+ - compatible: Must be one of the following, depending on the
+ model:
+ "maxim,ds1803-010",
+ "maxim,ds1803-050",
+ "maxim,ds1803-100"
+
+Example:
+ds1803: ds1803@1 {
+ reg = <0x28>;
+ compatible = "maxim,ds1803-010";
+};
diff --git a/Documentation/devicetree/bindings/iio/potentiometer/mcp4131.txt b/Documentation/devicetree/bindings/iio/potentiometer/mcp4131.txt
new file mode 100644
index 0000000..3ccba16
--- /dev/null
+++ b/Documentation/devicetree/bindings/iio/potentiometer/mcp4131.txt
@@ -0,0 +1,84 @@
+* Microchip MCP413X/414X/415X/416X/423X/424X/425X/426X Digital Potentiometer
+ driver
+
+The node for this driver must be a child node of a SPI controller, hence
+all mandatory properties described in
+
+ Documentation/devicetree/bindings/spi/spi-bus.txt
+
+must be specified.
+
+Required properties:
+ - compatible: Must be one of the following, depending on the
+ model:
+ "microchip,mcp4131-502"
+ "microchip,mcp4131-103"
+ "microchip,mcp4131-503"
+ "microchip,mcp4131-104"
+ "microchip,mcp4132-502"
+ "microchip,mcp4132-103"
+ "microchip,mcp4132-503"
+ "microchip,mcp4132-104"
+ "microchip,mcp4141-502"
+ "microchip,mcp4141-103"
+ "microchip,mcp4141-503"
+ "microchip,mcp4141-104"
+ "microchip,mcp4142-502"
+ "microchip,mcp4142-103"
+ "microchip,mcp4142-503"
+ "microchip,mcp4142-104"
+ "microchip,mcp4151-502"
+ "microchip,mcp4151-103"
+ "microchip,mcp4151-503"
+ "microchip,mcp4151-104"
+ "microchip,mcp4152-502"
+ "microchip,mcp4152-103"
+ "microchip,mcp4152-503"
+ "microchip,mcp4152-104"
+ "microchip,mcp4161-502"
+ "microchip,mcp4161-103"
+ "microchip,mcp4161-503"
+ "microchip,mcp4161-104"
+ "microchip,mcp4162-502"
+ "microchip,mcp4162-103"
+ "microchip,mcp4162-503"
+ "microchip,mcp4162-104"
+ "microchip,mcp4231-502"
+ "microchip,mcp4231-103"
+ "microchip,mcp4231-503"
+ "microchip,mcp4231-104"
+ "microchip,mcp4232-502"
+ "microchip,mcp4232-103"
+ "microchip,mcp4232-503"
+ "microchip,mcp4232-104"
+ "microchip,mcp4241-502"
+ "microchip,mcp4241-103"
+ "microchip,mcp4241-503"
+ "microchip,mcp4241-104"
+ "microchip,mcp4242-502"
+ "microchip,mcp4242-103"
+ "microchip,mcp4242-503"
+ "microchip,mcp4242-104"
+ "microchip,mcp4251-502"
+ "microchip,mcp4251-103"
+ "microchip,mcp4251-503"
+ "microchip,mcp4251-104"
+ "microchip,mcp4252-502"
+ "microchip,mcp4252-103"
+ "microchip,mcp4252-503"
+ "microchip,mcp4252-104"
+ "microchip,mcp4261-502"
+ "microchip,mcp4261-103"
+ "microchip,mcp4261-503"
+ "microchip,mcp4261-104"
+ "microchip,mcp4262-502"
+ "microchip,mcp4262-103"
+ "microchip,mcp4262-503"
+ "microchip,mcp4262-104"
+
+Example:
+mcp4131: mcp4131@0 {
+ compatible = "mcp4131-502";
+ reg = <0>;
+ spi-max-frequency = <500000>;
+};
diff --git a/Documentation/devicetree/bindings/iio/pressure/hp03.txt b/Documentation/devicetree/bindings/iio/pressure/hp03.txt
new file mode 100644
index 0000000..54e7e70
--- /dev/null
+++ b/Documentation/devicetree/bindings/iio/pressure/hp03.txt
@@ -0,0 +1,17 @@
+HopeRF HP03 digital pressure/temperature sensors
+
+Required properties:
+- compatible: must be "hoperf,hp03"
+- xclr-gpio: must be device tree identifier of the XCLR pin.
+ The XCLR pin is a reset of the ADC in the chip,
+ it must be pulled HI before the conversion and
+ readout of the value from the ADC registers and
+ pulled LO afterward.
+
+Example:
+
+hp03@0x77 {
+ compatible = "hoperf,hp03";
+ reg = <0x77>;
+ xclr-gpio = <&portc 0 0x0>;
+};
diff --git a/Documentation/devicetree/bindings/iio/pressure/ms5611.txt b/Documentation/devicetree/bindings/iio/pressure/ms5611.txt
new file mode 100644
index 0000000..17bca86
--- /dev/null
+++ b/Documentation/devicetree/bindings/iio/pressure/ms5611.txt
@@ -0,0 +1,19 @@
+MEAS ms5611 family pressure sensors
+
+Pressure sensors from MEAS Switzerland with SPI and I2C bus interfaces.
+
+Required properties:
+- compatible: "meas,ms5611" or "meas,ms5607"
+- reg: the I2C address or SPI chip select the device will respond to
+
+Optional properties:
+- vdd-supply: an optional regulator that needs to be on to provide VDD
+ power to the sensor.
+
+Example:
+
+ms5607@77 {
+ compatible = "meas,ms5607";
+ reg = <0x77>;
+ vdd-supply = <&ldo_3v3_gnss>;
+};
diff --git a/Documentation/devicetree/bindings/iio/st-sensors.txt b/Documentation/devicetree/bindings/iio/st-sensors.txt
index d4b87cc..5844cf7 100644
--- a/Documentation/devicetree/bindings/iio/st-sensors.txt
+++ b/Documentation/devicetree/bindings/iio/st-sensors.txt
@@ -16,6 +16,10 @@
- st,drdy-int-pin: the pin on the package that will be used to signal
"data ready" (valid values: 1 or 2). This property is not configurable
on all sensors.
+- drive-open-drain: the interrupt/data ready line will be configured
+ as open drain, which is useful if several sensors share the same
+ interrupt line. (This binding is taken from pinctrl/pinctrl-bindings.txt)
+ This is a boolean property.
Sensors may also have applicable pin control settings, those use the
standard bindings from pinctrl/pinctrl-bindings.txt.
@@ -37,6 +41,7 @@
- st,lsm330-accel
- st,lsm303agr-accel
- st,lis2dh12-accel
+- st,h3lis331dl-accel
Gyroscopes:
- st,l3g4200d-gyro
@@ -46,6 +51,7 @@
- st,l3gd20-gyro
- st,l3g4is-gyro
- st,lsm330-gyro
+- st,lsm9ds0-gyro
Magnetometers:
- st,lsm303agr-magn
diff --git a/Documentation/devicetree/bindings/input/ads7846.txt b/Documentation/devicetree/bindings/input/ads7846.txt
index c6cfe2e..9fc47b0 100644
--- a/Documentation/devicetree/bindings/input/ads7846.txt
+++ b/Documentation/devicetree/bindings/input/ads7846.txt
@@ -29,7 +29,7 @@
ti,vref-delay-usecs vref supply delay in usecs, 0 for
external vref (u16).
ti,vref-mv The VREF voltage, in millivolts (u16).
- Set to 0 to use internal refernce
+ Set to 0 to use internal references
(ADS7846).
ti,keep-vref-on set to keep vref on for differential
measurements as well
diff --git a/Documentation/devicetree/bindings/input/gpio-keys.txt b/Documentation/devicetree/bindings/input/gpio-keys.txt
index 2164123..a949404 100644
--- a/Documentation/devicetree/bindings/input/gpio-keys.txt
+++ b/Documentation/devicetree/bindings/input/gpio-keys.txt
@@ -32,17 +32,17 @@
Example nodes:
- gpio_keys {
+ gpio-keys {
compatible = "gpio-keys";
- #address-cells = <1>;
- #size-cells = <0>;
autorepeat;
- button@21 {
+
+ up {
label = "GPIO Key UP";
linux,code = <103>;
gpios = <&gpio1 0 1>;
};
- button@22 {
+
+ down {
label = "GPIO Key DOWN";
linux,code = <108>;
interrupts = <1 IRQ_TYPE_LEVEL_HIGH 7>;
diff --git a/Documentation/devicetree/bindings/input/touchscreen/brcm,iproc-touchscreen.txt b/Documentation/devicetree/bindings/input/touchscreen/brcm,iproc-touchscreen.txt
index 34e3382..ac5dff41 100644
--- a/Documentation/devicetree/bindings/input/touchscreen/brcm,iproc-touchscreen.txt
+++ b/Documentation/devicetree/bindings/input/touchscreen/brcm,iproc-touchscreen.txt
@@ -2,11 +2,17 @@
Required properties:
- compatible: must be "brcm,iproc-touchscreen"
-- reg: physical base address of the controller and length of memory mapped
- region.
+- ts_syscon: handler of syscon node defining physical base
+ address of the controller and length of memory mapped region.
+ If this property is selected please make sure MFD_SYSCON config
+ is enabled in the defconfig file.
- clocks: The clock provided by the SOC to driver the tsc
-- clock-name: name for the clock
+- clock-names: name for the clock
- interrupts: The touchscreen controller's interrupt
+- address-cells: Specify the number of u32 entries needed in child nodes.
+ Should set to 1.
+- size-cells: Specify number of u32 entries needed to specify child nodes size
+ in reg property. Should set to 1.
Optional properties:
- scanning_period: Time between scans. Each step is 1024 us. Valid 1-256.
@@ -53,13 +59,18 @@
- touchscreen-inverted-x: X axis is inverted (boolean)
- touchscreen-inverted-y: Y axis is inverted (boolean)
-Example:
+Example: An example of touchscreen node
- touchscreen: tsc@0x180A6000 {
+ ts_adc_syscon: ts_adc_syscon@180a6000 {
+ compatible = "brcm,iproc-ts-adc-syscon","syscon";
+ reg = <0x180a6000 0xc30>;
+ };
+
+ touchscreen: touchscreen@180A6000 {
compatible = "brcm,iproc-touchscreen";
#address-cells = <1>;
#size-cells = <1>;
- reg = <0x180A6000 0x40>;
+ ts_syscon = <&ts_adc_syscon>;
clocks = <&adc_clk>;
clock-names = "tsc_clk";
interrupts = <GIC_SPI 164 IRQ_TYPE_LEVEL_HIGH>;
diff --git a/Documentation/devicetree/bindings/input/touchscreen/fsl-mx25-tcq.txt b/Documentation/devicetree/bindings/input/touchscreen/fsl-mx25-tcq.txt
index cdf05f9..abfcab3 100644
--- a/Documentation/devicetree/bindings/input/touchscreen/fsl-mx25-tcq.txt
+++ b/Documentation/devicetree/bindings/input/touchscreen/fsl-mx25-tcq.txt
@@ -15,7 +15,7 @@
- fsl,pen-debounce-ns: Pen debounce time in nanoseconds.
- fsl,pen-threshold: Pen-down threshold for the touchscreen. This is a value
between 1 and 4096. It is the ratio between the internal reference voltage
- and the measured voltage after the plate was precharged. Resistence between
+ and the measured voltage after the plate was precharged. Resistance between
plates and therefore the voltage decreases with pressure so that a smaller
value is equivalent to a higher pressure.
- fsl,settling-time-ns: Settling time in nanoseconds. The settling time is before
diff --git a/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm6345-l1-intc.txt b/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm6345-l1-intc.txt
new file mode 100644
index 0000000..4040905
--- /dev/null
+++ b/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm6345-l1-intc.txt
@@ -0,0 +1,57 @@
+Broadcom BCM6345-style Level 1 interrupt controller
+
+This block is a first level interrupt controller that is typically connected
+directly to one of the HW INT lines on each CPU.
+
+Key elements of the hardware design include:
+
+- 32, 64 or 128 incoming level IRQ lines
+
+- Most onchip peripherals are wired directly to an L1 input
+
+- A separate instance of the register set for each CPU, allowing individual
+ peripheral IRQs to be routed to any CPU
+
+- Contains one or more enable/status word pairs per CPU
+
+- No atomic set/clear operations
+
+- No polarity/level/edge settings
+
+- No FIFO or priority encoder logic; software is expected to read all
+ 2-4 status words to determine which IRQs are pending
+
+Required properties:
+
+- compatible: should be "brcm,bcm<soc>-l1-intc", "brcm,bcm6345-l1-intc"
+- reg: specifies the base physical address and size of the registers;
+ the number of supported IRQs is inferred from the size argument
+- interrupt-controller: identifies the node as an interrupt controller
+- #interrupt-cells: specifies the number of cells needed to encode an interrupt
+ source, should be 1.
+- interrupt-parent: specifies the phandle to the parent interrupt controller(s)
+ this one is cascaded from
+- interrupts: specifies the interrupt line(s) in the interrupt-parent controller
+ node; valid values depend on the type of parent interrupt controller
+
+If multiple reg ranges and interrupt-parent entries are present on an SMP
+system, the driver will allow IRQ SMP affinity to be set up through the
+/proc/irq/ interface. In the simplest possible configuration, only one
+reg range and one interrupt-parent is needed.
+
+The driver operates in native CPU endian by default, there is no support for
+specifying an alternative endianness.
+
+Example:
+
+periph_intc: interrupt-controller@10000000 {
+ compatible = "brcm,bcm63168-l1-intc", "brcm,bcm6345-l1-intc";
+ reg = <0x10000020 0x20>,
+ <0x10000040 0x20>;
+
+ interrupt-controller;
+ #interrupt-cells = <1>;
+
+ interrupt-parent = <&cpu_intc>;
+ interrupts = <2>, <3>;
+};
diff --git a/Documentation/devicetree/bindings/interrupt-controller/ezchip,nps400-ic.txt b/Documentation/devicetree/bindings/interrupt-controller/ezchip,nps400-ic.txt
new file mode 100644
index 0000000..888b2b9
--- /dev/null
+++ b/Documentation/devicetree/bindings/interrupt-controller/ezchip,nps400-ic.txt
@@ -0,0 +1,17 @@
+EZchip NPS Interrupt Controller
+
+Required properties:
+
+- compatible : should be "ezchip,nps400-ic"
+- interrupt-controller : Identifies the node as an interrupt controller
+- #interrupt-cells : Specifies the number of cells needed to encode an
+ interrupt source. The value shall be 1.
+
+
+Example:
+
+intc: interrupt-controller {
+ compatible = "ezchip,nps400-ic";
+ interrupt-controller;
+ #interrupt-cells = <1>;
+};
diff --git a/Documentation/devicetree/bindings/interrupt-controller/mediatek,sysirq.txt b/Documentation/devicetree/bindings/interrupt-controller/mediatek,sysirq.txt
index b8e1674..8cf564d 100644
--- a/Documentation/devicetree/bindings/interrupt-controller/mediatek,sysirq.txt
+++ b/Documentation/devicetree/bindings/interrupt-controller/mediatek,sysirq.txt
@@ -16,8 +16,7 @@
"mediatek,mt6577-sysirq"
"mediatek,mt2701-sysirq"
- interrupt-controller : Identifies the node as an interrupt controller
-- #interrupt-cells : Use the same format as specified by GIC in
- Documentation/devicetree/bindings/arm/gic.txt
+- #interrupt-cells : Use the same format as specified by GIC in arm,gic.txt.
- interrupt-parent: phandle of irq parent for sysirq. The parent must
use the same interrupt-cells format as GIC.
- reg: Physical base address of the intpol registers and length of memory
diff --git a/Documentation/devicetree/bindings/interrupt-controller/nvidia,tegra-ictlr.txt b/Documentation/devicetree/bindings/interrupt-controller/nvidia,tegra20-ictlr.txt
similarity index 100%
rename from Documentation/devicetree/bindings/interrupt-controller/nvidia,tegra-ictlr.txt
rename to Documentation/devicetree/bindings/interrupt-controller/nvidia,tegra20-ictlr.txt
diff --git a/Documentation/devicetree/bindings/interrupt-controller/ti,omap4-wugen-mpu b/Documentation/devicetree/bindings/interrupt-controller/ti,omap4-wugen-mpu
index 43effa0..18d4f40 100644
--- a/Documentation/devicetree/bindings/interrupt-controller/ti,omap4-wugen-mpu
+++ b/Documentation/devicetree/bindings/interrupt-controller/ti,omap4-wugen-mpu
@@ -4,7 +4,7 @@
routes interrupts to the GIC, and also serves as a wakeup source. It
is also referred to as "WUGEN-MPU", hence the name of the binding.
-Reguired properties:
+Required properties:
- compatible : should contain at least "ti,omap4-wugen-mpu" or
"ti,omap5-wugen-mpu"
@@ -20,7 +20,7 @@
- Because this HW ultimately routes interrupts to the GIC, the
interrupt specifier must be that of the GIC.
- Only SPIs can use the WUGEN as an interrupt parent. SGIs and PPIs
- are explicitly forbiden.
+ are explicitly forbidden.
Example:
diff --git a/Documentation/devicetree/bindings/iommu/arm,smmu.txt b/Documentation/devicetree/bindings/iommu/arm,smmu.txt
index 7180745..19fe6f2 100644
--- a/Documentation/devicetree/bindings/iommu/arm,smmu.txt
+++ b/Documentation/devicetree/bindings/iommu/arm,smmu.txt
@@ -16,6 +16,7 @@
"arm,mmu-400"
"arm,mmu-401"
"arm,mmu-500"
+ "cavium,smmu-v2"
depending on the particular implementation and/or the
version of the architecture implemented.
diff --git a/Documentation/devicetree/bindings/media/i2c/adv7180.txt b/Documentation/devicetree/bindings/media/i2c/adv7180.txt
new file mode 100644
index 0000000..0d50115
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/i2c/adv7180.txt
@@ -0,0 +1,29 @@
+* Analog Devices ADV7180 analog video decoder family
+
+The adv7180 family devices are used to capture analog video to different
+digital interfaces like MIPI CSI-2 or parallel video.
+
+Required Properties :
+- compatible : value must be one of
+ "adi,adv7180"
+ "adi,adv7182"
+ "adi,adv7280"
+ "adi,adv7280-m"
+ "adi,adv7281"
+ "adi,adv7281-m"
+ "adi,adv7281-ma"
+ "adi,adv7282"
+ "adi,adv7282-m"
+
+Example:
+
+ i2c0@1c22000 {
+ ...
+ ...
+ adv7180@21 {
+ compatible = "adi,adv7180";
+ reg = <0x21>;
+ };
+ ...
+ };
+
diff --git a/Documentation/devicetree/bindings/media/rcar_vin.txt b/Documentation/devicetree/bindings/media/rcar_vin.txt
index 619193c..6a4e61c 100644
--- a/Documentation/devicetree/bindings/media/rcar_vin.txt
+++ b/Documentation/devicetree/bindings/media/rcar_vin.txt
@@ -5,14 +5,22 @@
family of devices. The current blocks are always slaves and suppot one input
channel which can be either RGB, YUYV or BT656.
- - compatible: Must be one of the following
+ - compatible: Must be one or more of the following
- "renesas,vin-r8a7795" for the R8A7795 device
- "renesas,vin-r8a7794" for the R8A7794 device
- "renesas,vin-r8a7793" for the R8A7793 device
+ - "renesas,vin-r8a7792" for the R8A7792 device
- "renesas,vin-r8a7791" for the R8A7791 device
- "renesas,vin-r8a7790" for the R8A7790 device
- "renesas,vin-r8a7779" for the R8A7779 device
- "renesas,vin-r8a7778" for the R8A7778 device
+ - "renesas,rcar-gen2-vin" for a generic R-Car Gen2 compatible device.
+ - "renesas,rcar-gen3-vin" for a generic R-Car Gen3 compatible device.
+
+ When compatible with the generic version nodes must list the
+ SoC-specific version corresponding to the platform first
+ followed by the generic version.
+
- reg: the register base and size for the device registers
- interrupts: the interrupt for the device
- clocks: Reference to the parent clock
@@ -37,7 +45,7 @@
};
vin0: vin@0xe6ef0000 {
- compatible = "renesas,vin-r8a7790";
+ compatible = "renesas,vin-r8a7790", "renesas,rcar-gen2-vin";
clocks = <&mstp8_clks R8A7790_CLK_VIN0>;
reg = <0 0xe6ef0000 0 0x1000>;
interrupts = <0 188 IRQ_TYPE_LEVEL_HIGH>;
diff --git a/Documentation/devicetree/bindings/media/xilinx/video.txt b/Documentation/devicetree/bindings/media/xilinx/video.txt
index cbd46fa..68ac210 100644
--- a/Documentation/devicetree/bindings/media/xilinx/video.txt
+++ b/Documentation/devicetree/bindings/media/xilinx/video.txt
@@ -20,7 +20,7 @@
- xlnx,video-format: This property represents a video format transmitted on an
AXI bus between video IP cores, using its VF code as defined in "AXI4-Stream
Video IP and System Design Guide" [UG934]. How the format relates to the IP
- core is decribed in the IP core bindings documentation.
+ core is described in the IP core bindings documentation.
- xlnx,video-width: This property qualifies the video format with the sample
width expressed as a number of bits per pixel component. All components must
diff --git a/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra-mc.txt b/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra-mc.txt
deleted file mode 100644
index 3338a28..0000000
--- a/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra-mc.txt
+++ /dev/null
@@ -1,116 +0,0 @@
-NVIDIA Tegra Memory Controller device tree bindings
-===================================================
-
-memory-controller node
-----------------------
-
-Required properties:
-- compatible: Should be "nvidia,tegra<chip>-mc"
-- reg: Physical base address and length of the controller's registers.
-- clocks: Must contain an entry for each entry in clock-names.
- See ../clocks/clock-bindings.txt for details.
-- clock-names: Must include the following entries:
- - mc: the module's clock input
-- interrupts: The interrupt outputs from the controller.
-- #iommu-cells: Should be 1. The single cell of the IOMMU specifier defines
- the SWGROUP of the master.
-
-This device implements an IOMMU that complies with the generic IOMMU binding.
-See ../iommu/iommu.txt for details.
-
-emc-timings subnode
--------------------
-
-The node should contain a "emc-timings" subnode for each supported RAM type (see field RAM_CODE in
-register PMC_STRAPPING_OPT_A).
-
-Required properties for "emc-timings" nodes :
-- nvidia,ram-code : Should contain the value of RAM_CODE this timing set is used for.
-
-timing subnode
---------------
-
-Each "emc-timings" node should contain a subnode for every supported EMC clock rate.
-
-Required properties for timing nodes :
-- clock-frequency : Should contain the memory clock rate in Hz.
-- nvidia,emem-configuration : Values to be written to the EMEM register block. For the Tegra124 SoC
-(see section "15.6.1 MC Registers" in the TRM), these are the registers whose values need to be
-specified, according to the board documentation:
-
- MC_EMEM_ARB_CFG
- MC_EMEM_ARB_OUTSTANDING_REQ
- MC_EMEM_ARB_TIMING_RCD
- MC_EMEM_ARB_TIMING_RP
- MC_EMEM_ARB_TIMING_RC
- MC_EMEM_ARB_TIMING_RAS
- MC_EMEM_ARB_TIMING_FAW
- MC_EMEM_ARB_TIMING_RRD
- MC_EMEM_ARB_TIMING_RAP2PRE
- MC_EMEM_ARB_TIMING_WAP2PRE
- MC_EMEM_ARB_TIMING_R2R
- MC_EMEM_ARB_TIMING_W2W
- MC_EMEM_ARB_TIMING_R2W
- MC_EMEM_ARB_TIMING_W2R
- MC_EMEM_ARB_DA_TURNS
- MC_EMEM_ARB_DA_COVERS
- MC_EMEM_ARB_MISC0
- MC_EMEM_ARB_MISC1
- MC_EMEM_ARB_RING1_THROTTLE
-
-Example SoC include file:
-
-/ {
- mc: memory-controller@0,70019000 {
- compatible = "nvidia,tegra124-mc";
- reg = <0x0 0x70019000 0x0 0x1000>;
- clocks = <&tegra_car TEGRA124_CLK_MC>;
- clock-names = "mc";
-
- interrupts = <GIC_SPI 77 IRQ_TYPE_LEVEL_HIGH>;
-
- #iommu-cells = <1>;
- };
-
- sdhci@0,700b0000 {
- compatible = "nvidia,tegra124-sdhci";
- ...
- iommus = <&mc TEGRA_SWGROUP_SDMMC1A>;
- };
-};
-
-Example board file:
-
-/ {
- memory-controller@0,70019000 {
- emc-timings-3 {
- nvidia,ram-code = <3>;
-
- timing-12750000 {
- clock-frequency = <12750000>;
-
- nvidia,emem-configuration = <
- 0x40040001 /* MC_EMEM_ARB_CFG */
- 0x8000000a /* MC_EMEM_ARB_OUTSTANDING_REQ */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
- 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
- 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
- 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
- 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
- 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
- 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
- 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
- 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
- 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
- 0x000a0402 /* MC_EMEM_ARB_DA_COVERS */
- 0x77e30303 /* MC_EMEM_ARB_MISC0 */
- 0x70000f03 /* MC_EMEM_ARB_MISC1 */
- 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
- >;
- };
- };
- };
-};
diff --git a/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra124-emc.txt b/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra124-emc.txt
new file mode 100644
index 0000000..ba0bc3f
--- /dev/null
+++ b/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra124-emc.txt
@@ -0,0 +1,374 @@
+NVIDIA Tegra124 SoC EMC (external memory controller)
+====================================================
+
+Required properties :
+- compatible : Should be "nvidia,tegra124-emc".
+- reg : physical base address and length of the controller's registers.
+- nvidia,memory-controller : phandle of the MC driver.
+
+The node should contain a "emc-timings" subnode for each supported RAM type
+(see field RAM_CODE in register PMC_STRAPPING_OPT_A), with its unit address
+being its RAM_CODE.
+
+Required properties for "emc-timings" nodes :
+- nvidia,ram-code : Should contain the value of RAM_CODE this timing set is
+used for.
+
+Each "emc-timings" node should contain a "timing" subnode for every supported
+EMC clock rate. The "timing" subnodes should have the clock rate in Hz as
+their unit address.
+
+Required properties for "timing" nodes :
+- clock-frequency : Should contain the memory clock rate in Hz.
+- The following properties contain EMC timing characterization values
+(specified in the board documentation) :
+ - nvidia,emc-auto-cal-config : EMC_AUTO_CAL_CONFIG
+ - nvidia,emc-auto-cal-config2 : EMC_AUTO_CAL_CONFIG2
+ - nvidia,emc-auto-cal-config3 : EMC_AUTO_CAL_CONFIG3
+ - nvidia,emc-auto-cal-interval : EMC_AUTO_CAL_INTERVAL
+ - nvidia,emc-bgbias-ctl0 : EMC_BGBIAS_CTL0
+ - nvidia,emc-cfg : EMC_CFG
+ - nvidia,emc-cfg-2 : EMC_CFG_2
+ - nvidia,emc-ctt-term-ctrl : EMC_CTT_TERM_CTRL
+ - nvidia,emc-mode-1 : Mode Register 1
+ - nvidia,emc-mode-2 : Mode Register 2
+ - nvidia,emc-mode-4 : Mode Register 4
+ - nvidia,emc-mode-reset : Mode Register 0
+ - nvidia,emc-mrs-wait-cnt : EMC_MRS_WAIT_CNT
+ - nvidia,emc-sel-dpd-ctrl : EMC_SEL_DPD_CTRL
+ - nvidia,emc-xm2dqspadctrl2 : EMC_XM2DQSPADCTRL2
+ - nvidia,emc-zcal-cnt-long : EMC_ZCAL_WAIT_CNT after clock change
+ - nvidia,emc-zcal-interval : EMC_ZCAL_INTERVAL
+- nvidia,emc-configuration : EMC timing characterization data. These are the
+registers (see section "15.6.2 EMC Registers" in the TRM) whose values need to
+be specified, according to the board documentation:
+
+ EMC_RC
+ EMC_RFC
+ EMC_RFC_SLR
+ EMC_RAS
+ EMC_RP
+ EMC_R2W
+ EMC_W2R
+ EMC_R2P
+ EMC_W2P
+ EMC_RD_RCD
+ EMC_WR_RCD
+ EMC_RRD
+ EMC_REXT
+ EMC_WEXT
+ EMC_WDV
+ EMC_WDV_MASK
+ EMC_QUSE
+ EMC_QUSE_WIDTH
+ EMC_IBDLY
+ EMC_EINPUT
+ EMC_EINPUT_DURATION
+ EMC_PUTERM_EXTRA
+ EMC_PUTERM_WIDTH
+ EMC_PUTERM_ADJ
+ EMC_CDB_CNTL_1
+ EMC_CDB_CNTL_2
+ EMC_CDB_CNTL_3
+ EMC_QRST
+ EMC_QSAFE
+ EMC_RDV
+ EMC_RDV_MASK
+ EMC_REFRESH
+ EMC_BURST_REFRESH_NUM
+ EMC_PRE_REFRESH_REQ_CNT
+ EMC_PDEX2WR
+ EMC_PDEX2RD
+ EMC_PCHG2PDEN
+ EMC_ACT2PDEN
+ EMC_AR2PDEN
+ EMC_RW2PDEN
+ EMC_TXSR
+ EMC_TXSRDLL
+ EMC_TCKE
+ EMC_TCKESR
+ EMC_TPD
+ EMC_TFAW
+ EMC_TRPAB
+ EMC_TCLKSTABLE
+ EMC_TCLKSTOP
+ EMC_TREFBW
+ EMC_FBIO_CFG6
+ EMC_ODT_WRITE
+ EMC_ODT_READ
+ EMC_FBIO_CFG5
+ EMC_CFG_DIG_DLL
+ EMC_CFG_DIG_DLL_PERIOD
+ EMC_DLL_XFORM_DQS0
+ EMC_DLL_XFORM_DQS1
+ EMC_DLL_XFORM_DQS2
+ EMC_DLL_XFORM_DQS3
+ EMC_DLL_XFORM_DQS4
+ EMC_DLL_XFORM_DQS5
+ EMC_DLL_XFORM_DQS6
+ EMC_DLL_XFORM_DQS7
+ EMC_DLL_XFORM_DQS8
+ EMC_DLL_XFORM_DQS9
+ EMC_DLL_XFORM_DQS10
+ EMC_DLL_XFORM_DQS11
+ EMC_DLL_XFORM_DQS12
+ EMC_DLL_XFORM_DQS13
+ EMC_DLL_XFORM_DQS14
+ EMC_DLL_XFORM_DQS15
+ EMC_DLL_XFORM_QUSE0
+ EMC_DLL_XFORM_QUSE1
+ EMC_DLL_XFORM_QUSE2
+ EMC_DLL_XFORM_QUSE3
+ EMC_DLL_XFORM_QUSE4
+ EMC_DLL_XFORM_QUSE5
+ EMC_DLL_XFORM_QUSE6
+ EMC_DLL_XFORM_QUSE7
+ EMC_DLL_XFORM_ADDR0
+ EMC_DLL_XFORM_ADDR1
+ EMC_DLL_XFORM_ADDR2
+ EMC_DLL_XFORM_ADDR3
+ EMC_DLL_XFORM_ADDR4
+ EMC_DLL_XFORM_ADDR5
+ EMC_DLL_XFORM_QUSE8
+ EMC_DLL_XFORM_QUSE9
+ EMC_DLL_XFORM_QUSE10
+ EMC_DLL_XFORM_QUSE11
+ EMC_DLL_XFORM_QUSE12
+ EMC_DLL_XFORM_QUSE13
+ EMC_DLL_XFORM_QUSE14
+ EMC_DLL_XFORM_QUSE15
+ EMC_DLI_TRIM_TXDQS0
+ EMC_DLI_TRIM_TXDQS1
+ EMC_DLI_TRIM_TXDQS2
+ EMC_DLI_TRIM_TXDQS3
+ EMC_DLI_TRIM_TXDQS4
+ EMC_DLI_TRIM_TXDQS5
+ EMC_DLI_TRIM_TXDQS6
+ EMC_DLI_TRIM_TXDQS7
+ EMC_DLI_TRIM_TXDQS8
+ EMC_DLI_TRIM_TXDQS9
+ EMC_DLI_TRIM_TXDQS10
+ EMC_DLI_TRIM_TXDQS11
+ EMC_DLI_TRIM_TXDQS12
+ EMC_DLI_TRIM_TXDQS13
+ EMC_DLI_TRIM_TXDQS14
+ EMC_DLI_TRIM_TXDQS15
+ EMC_DLL_XFORM_DQ0
+ EMC_DLL_XFORM_DQ1
+ EMC_DLL_XFORM_DQ2
+ EMC_DLL_XFORM_DQ3
+ EMC_DLL_XFORM_DQ4
+ EMC_DLL_XFORM_DQ5
+ EMC_DLL_XFORM_DQ6
+ EMC_DLL_XFORM_DQ7
+ EMC_XM2CMDPADCTRL
+ EMC_XM2CMDPADCTRL4
+ EMC_XM2CMDPADCTRL5
+ EMC_XM2DQPADCTRL2
+ EMC_XM2DQPADCTRL3
+ EMC_XM2CLKPADCTRL
+ EMC_XM2CLKPADCTRL2
+ EMC_XM2COMPPADCTRL
+ EMC_XM2VTTGENPADCTRL
+ EMC_XM2VTTGENPADCTRL2
+ EMC_XM2VTTGENPADCTRL3
+ EMC_XM2DQSPADCTRL3
+ EMC_XM2DQSPADCTRL4
+ EMC_XM2DQSPADCTRL5
+ EMC_XM2DQSPADCTRL6
+ EMC_DSR_VTTGEN_DRV
+ EMC_TXDSRVTTGEN
+ EMC_FBIO_SPARE
+ EMC_ZCAL_WAIT_CNT
+ EMC_MRS_WAIT_CNT2
+ EMC_CTT
+ EMC_CTT_DURATION
+ EMC_CFG_PIPE
+ EMC_DYN_SELF_REF_CONTROL
+ EMC_QPOP
+
+Example SoC include file:
+
+/ {
+ emc@7001b000 {
+ compatible = "nvidia,tegra124-emc";
+ reg = <0x0 0x7001b000 0x0 0x1000>;
+
+ nvidia,memory-controller = <&mc>;
+ };
+};
+
+Example board file:
+
+/ {
+ emc@7001b000 {
+ emc-timings-3 {
+ nvidia,ram-code = <3>;
+
+ timing-12750000 {
+ clock-frequency = <12750000>;
+
+ nvidia,emc-zcal-cnt-long = <0x00000042>;
+ nvidia,emc-auto-cal-interval = <0x001fffff>;
+ nvidia,emc-ctt-term-ctrl = <0x00000802>;
+ nvidia,emc-cfg = <0x73240000>;
+ nvidia,emc-cfg-2 = <0x000008c5>;
+ nvidia,emc-sel-dpd-ctrl = <0x00040128>;
+ nvidia,emc-bgbias-ctl0 = <0x00000008>;
+ nvidia,emc-auto-cal-config = <0xa1430000>;
+ nvidia,emc-auto-cal-config2 = <0x00000000>;
+ nvidia,emc-auto-cal-config3 = <0x00000000>;
+ nvidia,emc-mode-reset = <0x80001221>;
+ nvidia,emc-mode-1 = <0x80100003>;
+ nvidia,emc-mode-2 = <0x80200008>;
+ nvidia,emc-mode-4 = <0x00000000>;
+
+ nvidia,emc-configuration = <
+ 0x00000000 /* EMC_RC */
+ 0x00000003 /* EMC_RFC */
+ 0x00000000 /* EMC_RFC_SLR */
+ 0x00000000 /* EMC_RAS */
+ 0x00000000 /* EMC_RP */
+ 0x00000004 /* EMC_R2W */
+ 0x0000000a /* EMC_W2R */
+ 0x00000003 /* EMC_R2P */
+ 0x0000000b /* EMC_W2P */
+ 0x00000000 /* EMC_RD_RCD */
+ 0x00000000 /* EMC_WR_RCD */
+ 0x00000003 /* EMC_RRD */
+ 0x00000003 /* EMC_REXT */
+ 0x00000000 /* EMC_WEXT */
+ 0x00000006 /* EMC_WDV */
+ 0x00000006 /* EMC_WDV_MASK */
+ 0x00000006 /* EMC_QUSE */
+ 0x00000002 /* EMC_QUSE_WIDTH */
+ 0x00000000 /* EMC_IBDLY */
+ 0x00000005 /* EMC_EINPUT */
+ 0x00000005 /* EMC_EINPUT_DURATION */
+ 0x00010000 /* EMC_PUTERM_EXTRA */
+ 0x00000003 /* EMC_PUTERM_WIDTH */
+ 0x00000000 /* EMC_PUTERM_ADJ */
+ 0x00000000 /* EMC_CDB_CNTL_1 */
+ 0x00000000 /* EMC_CDB_CNTL_2 */
+ 0x00000000 /* EMC_CDB_CNTL_3 */
+ 0x00000004 /* EMC_QRST */
+ 0x0000000c /* EMC_QSAFE */
+ 0x0000000d /* EMC_RDV */
+ 0x0000000f /* EMC_RDV_MASK */
+ 0x00000060 /* EMC_REFRESH */
+ 0x00000000 /* EMC_BURST_REFRESH_NUM */
+ 0x00000018 /* EMC_PRE_REFRESH_REQ_CNT */
+ 0x00000002 /* EMC_PDEX2WR */
+ 0x00000002 /* EMC_PDEX2RD */
+ 0x00000001 /* EMC_PCHG2PDEN */
+ 0x00000000 /* EMC_ACT2PDEN */
+ 0x00000007 /* EMC_AR2PDEN */
+ 0x0000000f /* EMC_RW2PDEN */
+ 0x00000005 /* EMC_TXSR */
+ 0x00000005 /* EMC_TXSRDLL */
+ 0x00000004 /* EMC_TCKE */
+ 0x00000005 /* EMC_TCKESR */
+ 0x00000004 /* EMC_TPD */
+ 0x00000000 /* EMC_TFAW */
+ 0x00000000 /* EMC_TRPAB */
+ 0x00000005 /* EMC_TCLKSTABLE */
+ 0x00000005 /* EMC_TCLKSTOP */
+ 0x00000064 /* EMC_TREFBW */
+ 0x00000000 /* EMC_FBIO_CFG6 */
+ 0x00000000 /* EMC_ODT_WRITE */
+ 0x00000000 /* EMC_ODT_READ */
+ 0x106aa298 /* EMC_FBIO_CFG5 */
+ 0x002c00a0 /* EMC_CFG_DIG_DLL */
+ 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
+ 0x00064000 /* EMC_DLL_XFORM_DQS0 */
+ 0x00064000 /* EMC_DLL_XFORM_DQS1 */
+ 0x00064000 /* EMC_DLL_XFORM_DQS2 */
+ 0x00064000 /* EMC_DLL_XFORM_DQS3 */
+ 0x00064000 /* EMC_DLL_XFORM_DQS4 */
+ 0x00064000 /* EMC_DLL_XFORM_DQS5 */
+ 0x00064000 /* EMC_DLL_XFORM_DQS6 */
+ 0x00064000 /* EMC_DLL_XFORM_DQS7 */
+ 0x00064000 /* EMC_DLL_XFORM_DQS8 */
+ 0x00064000 /* EMC_DLL_XFORM_DQS9 */
+ 0x00064000 /* EMC_DLL_XFORM_DQS10 */
+ 0x00064000 /* EMC_DLL_XFORM_DQS11 */
+ 0x00064000 /* EMC_DLL_XFORM_DQS12 */
+ 0x00064000 /* EMC_DLL_XFORM_DQS13 */
+ 0x00064000 /* EMC_DLL_XFORM_DQS14 */
+ 0x00064000 /* EMC_DLL_XFORM_DQS15 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
+ 0x00000000 /* EMC_DLL_XFORM_ADDR0 */
+ 0x00000000 /* EMC_DLL_XFORM_ADDR1 */
+ 0x00000000 /* EMC_DLL_XFORM_ADDR2 */
+ 0x00000000 /* EMC_DLL_XFORM_ADDR3 */
+ 0x00000000 /* EMC_DLL_XFORM_ADDR4 */
+ 0x00000000 /* EMC_DLL_XFORM_ADDR5 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE8 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE9 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE10 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE11 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE12 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE13 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE14 */
+ 0x00000000 /* EMC_DLL_XFORM_QUSE15 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS8 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS9 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS10 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS11 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS12 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS13 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS14 */
+ 0x00000000 /* EMC_DLI_TRIM_TXDQS15 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ0 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ1 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ2 */
+ 0x000fc000 /* EMC_DLL_XFORM_DQ3 */
+ 0x0000fc00 /* EMC_DLL_XFORM_DQ4 */
+ 0x0000fc00 /* EMC_DLL_XFORM_DQ5 */
+ 0x0000fc00 /* EMC_DLL_XFORM_DQ6 */
+ 0x0000fc00 /* EMC_DLL_XFORM_DQ7 */
+ 0x10000280 /* EMC_XM2CMDPADCTRL */
+ 0x00000000 /* EMC_XM2CMDPADCTRL4 */
+ 0x00111111 /* EMC_XM2CMDPADCTRL5 */
+ 0x00000000 /* EMC_XM2DQPADCTRL2 */
+ 0x00000000 /* EMC_XM2DQPADCTRL3 */
+ 0x77ffc081 /* EMC_XM2CLKPADCTRL */
+ 0x00000e0e /* EMC_XM2CLKPADCTRL2 */
+ 0x81f1f108 /* EMC_XM2COMPPADCTRL */
+ 0x07070004 /* EMC_XM2VTTGENPADCTRL */
+ 0x0000003f /* EMC_XM2VTTGENPADCTRL2 */
+ 0x016eeeee /* EMC_XM2VTTGENPADCTRL3 */
+ 0x51451400 /* EMC_XM2DQSPADCTRL3 */
+ 0x00514514 /* EMC_XM2DQSPADCTRL4 */
+ 0x00514514 /* EMC_XM2DQSPADCTRL5 */
+ 0x51451400 /* EMC_XM2DQSPADCTRL6 */
+ 0x0000003f /* EMC_DSR_VTTGEN_DRV */
+ 0x00000007 /* EMC_TXDSRVTTGEN */
+ 0x00000000 /* EMC_FBIO_SPARE */
+ 0x00000042 /* EMC_ZCAL_WAIT_CNT */
+ 0x000e000e /* EMC_MRS_WAIT_CNT2 */
+ 0x00000000 /* EMC_CTT */
+ 0x00000003 /* EMC_CTT_DURATION */
+ 0x0000f2f3 /* EMC_CFG_PIPE */
+ 0x800001c5 /* EMC_DYN_SELF_REF_CONTROL */
+ 0x0000000a /* EMC_QPOP */
+ >;
+ };
+ };
+ };
+};
diff --git a/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra30-mc.txt b/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra30-mc.txt
new file mode 100644
index 0000000..8dbe470
--- /dev/null
+++ b/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra30-mc.txt
@@ -0,0 +1,116 @@
+NVIDIA Tegra Memory Controller device tree bindings
+===================================================
+
+memory-controller node
+----------------------
+
+Required properties:
+- compatible: Should be "nvidia,tegra<chip>-mc"
+- reg: Physical base address and length of the controller's registers.
+- clocks: Must contain an entry for each entry in clock-names.
+ See ../clocks/clock-bindings.txt for details.
+- clock-names: Must include the following entries:
+ - mc: the module's clock input
+- interrupts: The interrupt outputs from the controller.
+- #iommu-cells: Should be 1. The single cell of the IOMMU specifier defines
+ the SWGROUP of the master.
+
+This device implements an IOMMU that complies with the generic IOMMU binding.
+See ../iommu/iommu.txt for details.
+
+emc-timings subnode
+-------------------
+
+The node should contain a "emc-timings" subnode for each supported RAM type (see field RAM_CODE in
+register PMC_STRAPPING_OPT_A).
+
+Required properties for "emc-timings" nodes :
+- nvidia,ram-code : Should contain the value of RAM_CODE this timing set is used for.
+
+timing subnode
+--------------
+
+Each "emc-timings" node should contain a subnode for every supported EMC clock rate.
+
+Required properties for timing nodes :
+- clock-frequency : Should contain the memory clock rate in Hz.
+- nvidia,emem-configuration : Values to be written to the EMEM register block. For the Tegra124 SoC
+(see section "15.6.1 MC Registers" in the TRM), these are the registers whose values need to be
+specified, according to the board documentation:
+
+ MC_EMEM_ARB_CFG
+ MC_EMEM_ARB_OUTSTANDING_REQ
+ MC_EMEM_ARB_TIMING_RCD
+ MC_EMEM_ARB_TIMING_RP
+ MC_EMEM_ARB_TIMING_RC
+ MC_EMEM_ARB_TIMING_RAS
+ MC_EMEM_ARB_TIMING_FAW
+ MC_EMEM_ARB_TIMING_RRD
+ MC_EMEM_ARB_TIMING_RAP2PRE
+ MC_EMEM_ARB_TIMING_WAP2PRE
+ MC_EMEM_ARB_TIMING_R2R
+ MC_EMEM_ARB_TIMING_W2W
+ MC_EMEM_ARB_TIMING_R2W
+ MC_EMEM_ARB_TIMING_W2R
+ MC_EMEM_ARB_DA_TURNS
+ MC_EMEM_ARB_DA_COVERS
+ MC_EMEM_ARB_MISC0
+ MC_EMEM_ARB_MISC1
+ MC_EMEM_ARB_RING1_THROTTLE
+
+Example SoC include file:
+
+/ {
+ mc: memory-controller@70019000 {
+ compatible = "nvidia,tegra124-mc";
+ reg = <0x0 0x70019000 0x0 0x1000>;
+ clocks = <&tegra_car TEGRA124_CLK_MC>;
+ clock-names = "mc";
+
+ interrupts = <GIC_SPI 77 IRQ_TYPE_LEVEL_HIGH>;
+
+ #iommu-cells = <1>;
+ };
+
+ sdhci@700b0000 {
+ compatible = "nvidia,tegra124-sdhci";
+ ...
+ iommus = <&mc TEGRA_SWGROUP_SDMMC1A>;
+ };
+};
+
+Example board file:
+
+/ {
+ memory-controller@70019000 {
+ emc-timings-3 {
+ nvidia,ram-code = <3>;
+
+ timing-12750000 {
+ clock-frequency = <12750000>;
+
+ nvidia,emem-configuration = <
+ 0x40040001 /* MC_EMEM_ARB_CFG */
+ 0x8000000a /* MC_EMEM_ARB_OUTSTANDING_REQ */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RCD */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RP */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RC */
+ 0x00000000 /* MC_EMEM_ARB_TIMING_RAS */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_FAW */
+ 0x00000001 /* MC_EMEM_ARB_TIMING_RRD */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_RAP2PRE */
+ 0x00000008 /* MC_EMEM_ARB_TIMING_WAP2PRE */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2R */
+ 0x00000002 /* MC_EMEM_ARB_TIMING_W2W */
+ 0x00000003 /* MC_EMEM_ARB_TIMING_R2W */
+ 0x00000006 /* MC_EMEM_ARB_TIMING_W2R */
+ 0x06030203 /* MC_EMEM_ARB_DA_TURNS */
+ 0x000a0402 /* MC_EMEM_ARB_DA_COVERS */
+ 0x77e30303 /* MC_EMEM_ARB_MISC0 */
+ 0x70000f03 /* MC_EMEM_ARB_MISC1 */
+ 0x001f0000 /* MC_EMEM_ARB_RING1_THROTTLE */
+ >;
+ };
+ };
+ };
+};
diff --git a/Documentation/devicetree/bindings/memory-controllers/omap-gpmc.txt b/Documentation/devicetree/bindings/memory-controllers/omap-gpmc.txt
new file mode 100644
index 0000000..21055e2
--- /dev/null
+++ b/Documentation/devicetree/bindings/memory-controllers/omap-gpmc.txt
@@ -0,0 +1,152 @@
+Device tree bindings for OMAP general purpose memory controllers (GPMC)
+
+The actual devices are instantiated from the child nodes of a GPMC node.
+
+Required properties:
+
+ - compatible: Should be set to one of the following:
+
+ ti,omap2420-gpmc (omap2420)
+ ti,omap2430-gpmc (omap2430)
+ ti,omap3430-gpmc (omap3430 & omap3630)
+ ti,omap4430-gpmc (omap4430 & omap4460 & omap543x)
+ ti,am3352-gpmc (am335x devices)
+
+ - reg: A resource specifier for the register space
+ (see the example below)
+ - ti,hwmods: Should be set to "ti,gpmc" until the DT transition is
+ completed.
+ - #address-cells: Must be set to 2 to allow memory address translation
+ - #size-cells: Must be set to 1 to allow CS address passing
+ - gpmc,num-cs: The maximum number of chip-select lines that controller
+ can support.
+ - gpmc,num-waitpins: The maximum number of wait pins that controller can
+ support.
+ - ranges: Must be set up to reflect the memory layout with four
+ integer values for each chip-select line in use:
+
+ <cs-number> 0 <physical address of mapping> <size>
+
+ Currently, calculated values derived from the contents
+ of the per-CS register GPMC_CONFIG7 (as set up by the
+ bootloader) are used for the physical address decoding.
+ As this will change in the future, filling correct
+ values here is a requirement.
+ - interrupt-controller: The GPMC driver implements and interrupt controller for
+ the NAND events "fifoevent" and "termcount" plus the
+ rising/falling edges on the GPMC_WAIT pins.
+ The interrupt number mapping is as follows
+ 0 - NAND_fifoevent
+ 1 - NAND_termcount
+ 2 - GPMC_WAIT0 pin edge
+ 3 - GPMC_WAIT1 pin edge, and so on.
+ - interrupt-cells: Must be set to 2
+ - gpio-controller: The GPMC driver implements a GPIO controller for the
+ GPMC WAIT pins that can be used as general purpose inputs.
+ 0 maps to GPMC_WAIT0 pin.
+ - gpio-cells: Must be set to 2
+
+Timing properties for child nodes. All are optional and default to 0.
+
+ - gpmc,sync-clk-ps: Minimum clock period for synchronous mode, in picoseconds
+
+ Chip-select signal timings (in nanoseconds) corresponding to GPMC_CONFIG2:
+ - gpmc,cs-on-ns: Assertion time
+ - gpmc,cs-rd-off-ns: Read deassertion time
+ - gpmc,cs-wr-off-ns: Write deassertion time
+
+ ADV signal timings (in nanoseconds) corresponding to GPMC_CONFIG3:
+ - gpmc,adv-on-ns: Assertion time
+ - gpmc,adv-rd-off-ns: Read deassertion time
+ - gpmc,adv-wr-off-ns: Write deassertion time
+ - gpmc,adv-aad-mux-on-ns: Assertion time for AAD
+ - gpmc,adv-aad-mux-rd-off-ns: Read deassertion time for AAD
+ - gpmc,adv-aad-mux-wr-off-ns: Write deassertion time for AAD
+
+ WE signals timings (in nanoseconds) corresponding to GPMC_CONFIG4:
+ - gpmc,we-on-ns Assertion time
+ - gpmc,we-off-ns: Deassertion time
+
+ OE signals timings (in nanoseconds) corresponding to GPMC_CONFIG4:
+ - gpmc,oe-on-ns: Assertion time
+ - gpmc,oe-off-ns: Deassertion time
+ - gpmc,oe-aad-mux-on-ns: Assertion time for AAD
+ - gpmc,oe-aad-mux-off-ns: Deassertion time for AAD
+
+ Access time and cycle time timings (in nanoseconds) corresponding to
+ GPMC_CONFIG5:
+ - gpmc,page-burst-access-ns: Multiple access word delay
+ - gpmc,access-ns: Start-cycle to first data valid delay
+ - gpmc,rd-cycle-ns: Total read cycle time
+ - gpmc,wr-cycle-ns: Total write cycle time
+ - gpmc,bus-turnaround-ns: Turn-around time between successive accesses
+ - gpmc,cycle2cycle-delay-ns: Delay between chip-select pulses
+ - gpmc,clk-activation-ns: GPMC clock activation time
+ - gpmc,wait-monitoring-ns: Start of wait monitoring with regard to valid
+ data
+
+Boolean timing parameters. If property is present parameter enabled and
+disabled if omitted:
+ - gpmc,adv-extra-delay: ADV signal is delayed by half GPMC clock
+ - gpmc,cs-extra-delay: CS signal is delayed by half GPMC clock
+ - gpmc,cycle2cycle-diffcsen: Add "cycle2cycle-delay" between successive
+ accesses to a different CS
+ - gpmc,cycle2cycle-samecsen: Add "cycle2cycle-delay" between successive
+ accesses to the same CS
+ - gpmc,oe-extra-delay: OE signal is delayed by half GPMC clock
+ - gpmc,we-extra-delay: WE signal is delayed by half GPMC clock
+ - gpmc,time-para-granularity: Multiply all access times by 2
+
+The following are only applicable to OMAP3+ and AM335x:
+ - gpmc,wr-access-ns: In synchronous write mode, for single or
+ burst accesses, defines the number of
+ GPMC_FCLK cycles from start access time
+ to the GPMC_CLK rising edge used by the
+ memory device for the first data capture.
+ - gpmc,wr-data-mux-bus-ns: In address-data multiplex mode, specifies
+ the time when the first data is driven on
+ the address-data bus.
+
+GPMC chip-select settings properties for child nodes. All are optional.
+
+- gpmc,burst-length Page/burst length. Must be 4, 8 or 16.
+- gpmc,burst-wrap Enables wrap bursting
+- gpmc,burst-read Enables read page/burst mode
+- gpmc,burst-write Enables write page/burst mode
+- gpmc,device-width Total width of device(s) connected to a GPMC
+ chip-select in bytes. The GPMC supports 8-bit
+ and 16-bit devices and so this property must be
+ 1 or 2.
+- gpmc,mux-add-data Address and data multiplexing configuration.
+ Valid values are 1 for address-address-data
+ multiplexing mode and 2 for address-data
+ multiplexing mode.
+- gpmc,sync-read Enables synchronous read. Defaults to asynchronous
+ is this is not set.
+- gpmc,sync-write Enables synchronous writes. Defaults to asynchronous
+ is this is not set.
+- gpmc,wait-pin Wait-pin used by client. Must be less than
+ "gpmc,num-waitpins".
+- gpmc,wait-on-read Enables wait monitoring on reads.
+- gpmc,wait-on-write Enables wait monitoring on writes.
+
+Example for an AM33xx board:
+
+ gpmc: gpmc@50000000 {
+ compatible = "ti,am3352-gpmc";
+ ti,hwmods = "gpmc";
+ reg = <0x50000000 0x2000>;
+ interrupts = <100>;
+
+ gpmc,num-cs = <8>;
+ gpmc,num-waitpins = <2>;
+ #address-cells = <2>;
+ #size-cells = <1>;
+ ranges = <0 0 0x08000000 0x10000000>; /* CS0 @addr 0x8000000, size 0x10000000 */
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ gpio-controller;
+ #gpio-cells = <2>;
+
+ /* child nodes go here */
+ };
diff --git a/Documentation/devicetree/bindings/memory-controllers/tegra-emc.txt b/Documentation/devicetree/bindings/memory-controllers/tegra-emc.txt
deleted file mode 100644
index b59c625d..0000000
--- a/Documentation/devicetree/bindings/memory-controllers/tegra-emc.txt
+++ /dev/null
@@ -1,374 +0,0 @@
-NVIDIA Tegra124 SoC EMC (external memory controller)
-====================================================
-
-Required properties :
-- compatible : Should be "nvidia,tegra124-emc".
-- reg : physical base address and length of the controller's registers.
-- nvidia,memory-controller : phandle of the MC driver.
-
-The node should contain a "emc-timings" subnode for each supported RAM type
-(see field RAM_CODE in register PMC_STRAPPING_OPT_A), with its unit address
-being its RAM_CODE.
-
-Required properties for "emc-timings" nodes :
-- nvidia,ram-code : Should contain the value of RAM_CODE this timing set is
-used for.
-
-Each "emc-timings" node should contain a "timing" subnode for every supported
-EMC clock rate. The "timing" subnodes should have the clock rate in Hz as
-their unit address.
-
-Required properties for "timing" nodes :
-- clock-frequency : Should contain the memory clock rate in Hz.
-- The following properties contain EMC timing characterization values
-(specified in the board documentation) :
- - nvidia,emc-auto-cal-config : EMC_AUTO_CAL_CONFIG
- - nvidia,emc-auto-cal-config2 : EMC_AUTO_CAL_CONFIG2
- - nvidia,emc-auto-cal-config3 : EMC_AUTO_CAL_CONFIG3
- - nvidia,emc-auto-cal-interval : EMC_AUTO_CAL_INTERVAL
- - nvidia,emc-bgbias-ctl0 : EMC_BGBIAS_CTL0
- - nvidia,emc-cfg : EMC_CFG
- - nvidia,emc-cfg-2 : EMC_CFG_2
- - nvidia,emc-ctt-term-ctrl : EMC_CTT_TERM_CTRL
- - nvidia,emc-mode-1 : Mode Register 1
- - nvidia,emc-mode-2 : Mode Register 2
- - nvidia,emc-mode-4 : Mode Register 4
- - nvidia,emc-mode-reset : Mode Register 0
- - nvidia,emc-mrs-wait-cnt : EMC_MRS_WAIT_CNT
- - nvidia,emc-sel-dpd-ctrl : EMC_SEL_DPD_CTRL
- - nvidia,emc-xm2dqspadctrl2 : EMC_XM2DQSPADCTRL2
- - nvidia,emc-zcal-cnt-long : EMC_ZCAL_WAIT_CNT after clock change
- - nvidia,emc-zcal-interval : EMC_ZCAL_INTERVAL
-- nvidia,emc-configuration : EMC timing characterization data. These are the
-registers (see section "15.6.2 EMC Registers" in the TRM) whose values need to
-be specified, according to the board documentation:
-
- EMC_RC
- EMC_RFC
- EMC_RFC_SLR
- EMC_RAS
- EMC_RP
- EMC_R2W
- EMC_W2R
- EMC_R2P
- EMC_W2P
- EMC_RD_RCD
- EMC_WR_RCD
- EMC_RRD
- EMC_REXT
- EMC_WEXT
- EMC_WDV
- EMC_WDV_MASK
- EMC_QUSE
- EMC_QUSE_WIDTH
- EMC_IBDLY
- EMC_EINPUT
- EMC_EINPUT_DURATION
- EMC_PUTERM_EXTRA
- EMC_PUTERM_WIDTH
- EMC_PUTERM_ADJ
- EMC_CDB_CNTL_1
- EMC_CDB_CNTL_2
- EMC_CDB_CNTL_3
- EMC_QRST
- EMC_QSAFE
- EMC_RDV
- EMC_RDV_MASK
- EMC_REFRESH
- EMC_BURST_REFRESH_NUM
- EMC_PRE_REFRESH_REQ_CNT
- EMC_PDEX2WR
- EMC_PDEX2RD
- EMC_PCHG2PDEN
- EMC_ACT2PDEN
- EMC_AR2PDEN
- EMC_RW2PDEN
- EMC_TXSR
- EMC_TXSRDLL
- EMC_TCKE
- EMC_TCKESR
- EMC_TPD
- EMC_TFAW
- EMC_TRPAB
- EMC_TCLKSTABLE
- EMC_TCLKSTOP
- EMC_TREFBW
- EMC_FBIO_CFG6
- EMC_ODT_WRITE
- EMC_ODT_READ
- EMC_FBIO_CFG5
- EMC_CFG_DIG_DLL
- EMC_CFG_DIG_DLL_PERIOD
- EMC_DLL_XFORM_DQS0
- EMC_DLL_XFORM_DQS1
- EMC_DLL_XFORM_DQS2
- EMC_DLL_XFORM_DQS3
- EMC_DLL_XFORM_DQS4
- EMC_DLL_XFORM_DQS5
- EMC_DLL_XFORM_DQS6
- EMC_DLL_XFORM_DQS7
- EMC_DLL_XFORM_DQS8
- EMC_DLL_XFORM_DQS9
- EMC_DLL_XFORM_DQS10
- EMC_DLL_XFORM_DQS11
- EMC_DLL_XFORM_DQS12
- EMC_DLL_XFORM_DQS13
- EMC_DLL_XFORM_DQS14
- EMC_DLL_XFORM_DQS15
- EMC_DLL_XFORM_QUSE0
- EMC_DLL_XFORM_QUSE1
- EMC_DLL_XFORM_QUSE2
- EMC_DLL_XFORM_QUSE3
- EMC_DLL_XFORM_QUSE4
- EMC_DLL_XFORM_QUSE5
- EMC_DLL_XFORM_QUSE6
- EMC_DLL_XFORM_QUSE7
- EMC_DLL_XFORM_ADDR0
- EMC_DLL_XFORM_ADDR1
- EMC_DLL_XFORM_ADDR2
- EMC_DLL_XFORM_ADDR3
- EMC_DLL_XFORM_ADDR4
- EMC_DLL_XFORM_ADDR5
- EMC_DLL_XFORM_QUSE8
- EMC_DLL_XFORM_QUSE9
- EMC_DLL_XFORM_QUSE10
- EMC_DLL_XFORM_QUSE11
- EMC_DLL_XFORM_QUSE12
- EMC_DLL_XFORM_QUSE13
- EMC_DLL_XFORM_QUSE14
- EMC_DLL_XFORM_QUSE15
- EMC_DLI_TRIM_TXDQS0
- EMC_DLI_TRIM_TXDQS1
- EMC_DLI_TRIM_TXDQS2
- EMC_DLI_TRIM_TXDQS3
- EMC_DLI_TRIM_TXDQS4
- EMC_DLI_TRIM_TXDQS5
- EMC_DLI_TRIM_TXDQS6
- EMC_DLI_TRIM_TXDQS7
- EMC_DLI_TRIM_TXDQS8
- EMC_DLI_TRIM_TXDQS9
- EMC_DLI_TRIM_TXDQS10
- EMC_DLI_TRIM_TXDQS11
- EMC_DLI_TRIM_TXDQS12
- EMC_DLI_TRIM_TXDQS13
- EMC_DLI_TRIM_TXDQS14
- EMC_DLI_TRIM_TXDQS15
- EMC_DLL_XFORM_DQ0
- EMC_DLL_XFORM_DQ1
- EMC_DLL_XFORM_DQ2
- EMC_DLL_XFORM_DQ3
- EMC_DLL_XFORM_DQ4
- EMC_DLL_XFORM_DQ5
- EMC_DLL_XFORM_DQ6
- EMC_DLL_XFORM_DQ7
- EMC_XM2CMDPADCTRL
- EMC_XM2CMDPADCTRL4
- EMC_XM2CMDPADCTRL5
- EMC_XM2DQPADCTRL2
- EMC_XM2DQPADCTRL3
- EMC_XM2CLKPADCTRL
- EMC_XM2CLKPADCTRL2
- EMC_XM2COMPPADCTRL
- EMC_XM2VTTGENPADCTRL
- EMC_XM2VTTGENPADCTRL2
- EMC_XM2VTTGENPADCTRL3
- EMC_XM2DQSPADCTRL3
- EMC_XM2DQSPADCTRL4
- EMC_XM2DQSPADCTRL5
- EMC_XM2DQSPADCTRL6
- EMC_DSR_VTTGEN_DRV
- EMC_TXDSRVTTGEN
- EMC_FBIO_SPARE
- EMC_ZCAL_WAIT_CNT
- EMC_MRS_WAIT_CNT2
- EMC_CTT
- EMC_CTT_DURATION
- EMC_CFG_PIPE
- EMC_DYN_SELF_REF_CONTROL
- EMC_QPOP
-
-Example SoC include file:
-
-/ {
- emc@0,7001b000 {
- compatible = "nvidia,tegra124-emc";
- reg = <0x0 0x7001b000 0x0 0x1000>;
-
- nvidia,memory-controller = <&mc>;
- };
-};
-
-Example board file:
-
-/ {
- emc@0,7001b000 {
- emc-timings-3 {
- nvidia,ram-code = <3>;
-
- timing-12750000 {
- clock-frequency = <12750000>;
-
- nvidia,emc-zcal-cnt-long = <0x00000042>;
- nvidia,emc-auto-cal-interval = <0x001fffff>;
- nvidia,emc-ctt-term-ctrl = <0x00000802>;
- nvidia,emc-cfg = <0x73240000>;
- nvidia,emc-cfg-2 = <0x000008c5>;
- nvidia,emc-sel-dpd-ctrl = <0x00040128>;
- nvidia,emc-bgbias-ctl0 = <0x00000008>;
- nvidia,emc-auto-cal-config = <0xa1430000>;
- nvidia,emc-auto-cal-config2 = <0x00000000>;
- nvidia,emc-auto-cal-config3 = <0x00000000>;
- nvidia,emc-mode-reset = <0x80001221>;
- nvidia,emc-mode-1 = <0x80100003>;
- nvidia,emc-mode-2 = <0x80200008>;
- nvidia,emc-mode-4 = <0x00000000>;
-
- nvidia,emc-configuration = <
- 0x00000000 /* EMC_RC */
- 0x00000003 /* EMC_RFC */
- 0x00000000 /* EMC_RFC_SLR */
- 0x00000000 /* EMC_RAS */
- 0x00000000 /* EMC_RP */
- 0x00000004 /* EMC_R2W */
- 0x0000000a /* EMC_W2R */
- 0x00000003 /* EMC_R2P */
- 0x0000000b /* EMC_W2P */
- 0x00000000 /* EMC_RD_RCD */
- 0x00000000 /* EMC_WR_RCD */
- 0x00000003 /* EMC_RRD */
- 0x00000003 /* EMC_REXT */
- 0x00000000 /* EMC_WEXT */
- 0x00000006 /* EMC_WDV */
- 0x00000006 /* EMC_WDV_MASK */
- 0x00000006 /* EMC_QUSE */
- 0x00000002 /* EMC_QUSE_WIDTH */
- 0x00000000 /* EMC_IBDLY */
- 0x00000005 /* EMC_EINPUT */
- 0x00000005 /* EMC_EINPUT_DURATION */
- 0x00010000 /* EMC_PUTERM_EXTRA */
- 0x00000003 /* EMC_PUTERM_WIDTH */
- 0x00000000 /* EMC_PUTERM_ADJ */
- 0x00000000 /* EMC_CDB_CNTL_1 */
- 0x00000000 /* EMC_CDB_CNTL_2 */
- 0x00000000 /* EMC_CDB_CNTL_3 */
- 0x00000004 /* EMC_QRST */
- 0x0000000c /* EMC_QSAFE */
- 0x0000000d /* EMC_RDV */
- 0x0000000f /* EMC_RDV_MASK */
- 0x00000060 /* EMC_REFRESH */
- 0x00000000 /* EMC_BURST_REFRESH_NUM */
- 0x00000018 /* EMC_PRE_REFRESH_REQ_CNT */
- 0x00000002 /* EMC_PDEX2WR */
- 0x00000002 /* EMC_PDEX2RD */
- 0x00000001 /* EMC_PCHG2PDEN */
- 0x00000000 /* EMC_ACT2PDEN */
- 0x00000007 /* EMC_AR2PDEN */
- 0x0000000f /* EMC_RW2PDEN */
- 0x00000005 /* EMC_TXSR */
- 0x00000005 /* EMC_TXSRDLL */
- 0x00000004 /* EMC_TCKE */
- 0x00000005 /* EMC_TCKESR */
- 0x00000004 /* EMC_TPD */
- 0x00000000 /* EMC_TFAW */
- 0x00000000 /* EMC_TRPAB */
- 0x00000005 /* EMC_TCLKSTABLE */
- 0x00000005 /* EMC_TCLKSTOP */
- 0x00000064 /* EMC_TREFBW */
- 0x00000000 /* EMC_FBIO_CFG6 */
- 0x00000000 /* EMC_ODT_WRITE */
- 0x00000000 /* EMC_ODT_READ */
- 0x106aa298 /* EMC_FBIO_CFG5 */
- 0x002c00a0 /* EMC_CFG_DIG_DLL */
- 0x00008000 /* EMC_CFG_DIG_DLL_PERIOD */
- 0x00064000 /* EMC_DLL_XFORM_DQS0 */
- 0x00064000 /* EMC_DLL_XFORM_DQS1 */
- 0x00064000 /* EMC_DLL_XFORM_DQS2 */
- 0x00064000 /* EMC_DLL_XFORM_DQS3 */
- 0x00064000 /* EMC_DLL_XFORM_DQS4 */
- 0x00064000 /* EMC_DLL_XFORM_DQS5 */
- 0x00064000 /* EMC_DLL_XFORM_DQS6 */
- 0x00064000 /* EMC_DLL_XFORM_DQS7 */
- 0x00064000 /* EMC_DLL_XFORM_DQS8 */
- 0x00064000 /* EMC_DLL_XFORM_DQS9 */
- 0x00064000 /* EMC_DLL_XFORM_DQS10 */
- 0x00064000 /* EMC_DLL_XFORM_DQS11 */
- 0x00064000 /* EMC_DLL_XFORM_DQS12 */
- 0x00064000 /* EMC_DLL_XFORM_DQS13 */
- 0x00064000 /* EMC_DLL_XFORM_DQS14 */
- 0x00064000 /* EMC_DLL_XFORM_DQS15 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE0 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE1 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE2 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE3 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE4 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE5 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE6 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE7 */
- 0x00000000 /* EMC_DLL_XFORM_ADDR0 */
- 0x00000000 /* EMC_DLL_XFORM_ADDR1 */
- 0x00000000 /* EMC_DLL_XFORM_ADDR2 */
- 0x00000000 /* EMC_DLL_XFORM_ADDR3 */
- 0x00000000 /* EMC_DLL_XFORM_ADDR4 */
- 0x00000000 /* EMC_DLL_XFORM_ADDR5 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE8 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE9 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE10 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE11 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE12 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE13 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE14 */
- 0x00000000 /* EMC_DLL_XFORM_QUSE15 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS0 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS1 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS2 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS3 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS4 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS5 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS6 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS7 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS8 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS9 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS10 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS11 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS12 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS13 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS14 */
- 0x00000000 /* EMC_DLI_TRIM_TXDQS15 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ0 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ1 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ2 */
- 0x000fc000 /* EMC_DLL_XFORM_DQ3 */
- 0x0000fc00 /* EMC_DLL_XFORM_DQ4 */
- 0x0000fc00 /* EMC_DLL_XFORM_DQ5 */
- 0x0000fc00 /* EMC_DLL_XFORM_DQ6 */
- 0x0000fc00 /* EMC_DLL_XFORM_DQ7 */
- 0x10000280 /* EMC_XM2CMDPADCTRL */
- 0x00000000 /* EMC_XM2CMDPADCTRL4 */
- 0x00111111 /* EMC_XM2CMDPADCTRL5 */
- 0x00000000 /* EMC_XM2DQPADCTRL2 */
- 0x00000000 /* EMC_XM2DQPADCTRL3 */
- 0x77ffc081 /* EMC_XM2CLKPADCTRL */
- 0x00000e0e /* EMC_XM2CLKPADCTRL2 */
- 0x81f1f108 /* EMC_XM2COMPPADCTRL */
- 0x07070004 /* EMC_XM2VTTGENPADCTRL */
- 0x0000003f /* EMC_XM2VTTGENPADCTRL2 */
- 0x016eeeee /* EMC_XM2VTTGENPADCTRL3 */
- 0x51451400 /* EMC_XM2DQSPADCTRL3 */
- 0x00514514 /* EMC_XM2DQSPADCTRL4 */
- 0x00514514 /* EMC_XM2DQSPADCTRL5 */
- 0x51451400 /* EMC_XM2DQSPADCTRL6 */
- 0x0000003f /* EMC_DSR_VTTGEN_DRV */
- 0x00000007 /* EMC_TXDSRVTTGEN */
- 0x00000000 /* EMC_FBIO_SPARE */
- 0x00000042 /* EMC_ZCAL_WAIT_CNT */
- 0x000e000e /* EMC_MRS_WAIT_CNT2 */
- 0x00000000 /* EMC_CTT */
- 0x00000003 /* EMC_CTT_DURATION */
- 0x0000f2f3 /* EMC_CFG_PIPE */
- 0x800001c5 /* EMC_DYN_SELF_REF_CONTROL */
- 0x0000000a /* EMC_QPOP */
- >;
- };
- };
- };
-};
diff --git a/Documentation/devicetree/bindings/mfd/arizona.txt b/Documentation/devicetree/bindings/mfd/arizona.txt
index 9b30011..a6e2ea41 100644
--- a/Documentation/devicetree/bindings/mfd/arizona.txt
+++ b/Documentation/devicetree/bindings/mfd/arizona.txt
@@ -1,6 +1,6 @@
Cirrus Logic/Wolfson Microelectronics Arizona class audio SoCs
-These devices are audio SoCs with extensive digital capabilites and a range
+These devices are audio SoCs with extensive digital capabilities and a range
of analogue I/O.
Required properties:
diff --git a/Documentation/devicetree/bindings/mfd/axp20x.txt b/Documentation/devicetree/bindings/mfd/axp20x.txt
index fd39fa5..d20b103 100644
--- a/Documentation/devicetree/bindings/mfd/axp20x.txt
+++ b/Documentation/devicetree/bindings/mfd/axp20x.txt
@@ -6,10 +6,11 @@
axp209 (X-Powers)
axp221 (X-Powers)
axp223 (X-Powers)
+axp809 (X-Powers)
Required properties:
- compatible: "x-powers,axp152", "x-powers,axp202", "x-powers,axp209",
- "x-powers,axp221", "x-powers,axp223"
+ "x-powers,axp221", "x-powers,axp223", "x-powers,axp809"
- reg: The I2C slave address or RSB hardware address for the AXP chip
- interrupt-parent: The parent interrupt controller
- interrupts: SoC NMI / GPIO interrupt connected to the PMIC's IRQ pin
@@ -18,7 +19,9 @@
Optional properties:
- x-powers,dcdc-freq: defines the work frequency of DC-DC in KHz
- (range: 750-1875). Default: 1.5MHz
+ AXP152/20X: range: 750-1875, Default: 1.5 MHz
+ AXP22X/80X: range: 1800-4050, Default: 3 MHz
+
- <input>-supply: a phandle to the regulator supply node. May be omitted if
inputs are unregulated, such as using the IPSOUT output
from the PMIC.
@@ -77,6 +80,30 @@
LDO_IO1 : LDO : ips-supply : GPIO 1
RTC_LDO : LDO : ips-supply : always on
+AXP809 regulators, type, and corresponding input supply names:
+
+Regulator Type Supply Name Notes
+--------- ---- ----------- -----
+DCDC1 : DC-DC buck : vin1-supply
+DCDC2 : DC-DC buck : vin2-supply
+DCDC3 : DC-DC buck : vin3-supply
+DCDC4 : DC-DC buck : vin4-supply
+DCDC5 : DC-DC buck : vin5-supply
+DC1SW : On/Off Switch : : DCDC1 secondary output
+DC5LDO : LDO : : input from DCDC5
+ALDO1 : LDO : aldoin-supply : shared supply
+ALDO2 : LDO : aldoin-supply : shared supply
+ALDO3 : LDO : aldoin-supply : shared supply
+DLDO1 : LDO : dldoin-supply : shared supply
+DLDO2 : LDO : dldoin-supply : shared supply
+ELDO1 : LDO : eldoin-supply : shared supply
+ELDO2 : LDO : eldoin-supply : shared supply
+ELDO3 : LDO : eldoin-supply : shared supply
+LDO_IO0 : LDO : ips-supply : GPIO 0
+LDO_IO1 : LDO : ips-supply : GPIO 1
+RTC_LDO : LDO : ips-supply : always on
+SW : On/Off Switch : swin-supply
+
Example:
axp209: pmic@34 {
diff --git a/Documentation/devicetree/bindings/mfd/hisilicon,hi655x.txt b/Documentation/devicetree/bindings/mfd/hisilicon,hi655x.txt
new file mode 100644
index 0000000..0548569
--- /dev/null
+++ b/Documentation/devicetree/bindings/mfd/hisilicon,hi655x.txt
@@ -0,0 +1,27 @@
+Hisilicon Hi655x Power Management Integrated Circuit (PMIC)
+
+The hardware layout for access PMIC Hi655x from AP SoC Hi6220.
+Between PMIC Hi655x and Hi6220, the physical signal channel is SSI.
+We can use memory-mapped I/O to communicate.
+
++----------------+ +-------------+
+| | | |
+| Hi6220 | SSI bus | Hi655x |
+| |-------------| |
+| |(REGMAP_MMIO)| |
++----------------+ +-------------+
+
+Required properties:
+- compatible: Should be "hisilicon,hi655x-pmic".
+- reg: Base address of PMIC on Hi6220 SoC.
+- interrupt-controller: Hi655x has internal IRQs (has own IRQ domain).
+- pmic-gpios: The GPIO used by PMIC IRQ.
+
+Example:
+ pmic: pmic@f8000000 {
+ compatible = "hisilicon,hi655x-pmic";
+ reg = <0x0 0xf8000000 0x0 0x1000>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ pmic-gpios = <&gpio1 2 GPIO_ACTIVE_HIGH>;
+ }
diff --git a/Documentation/devicetree/bindings/mfd/max77620.txt b/Documentation/devicetree/bindings/mfd/max77620.txt
new file mode 100644
index 0000000..2ad44f7
--- /dev/null
+++ b/Documentation/devicetree/bindings/mfd/max77620.txt
@@ -0,0 +1,143 @@
+MAX77620 Power management IC from Maxim Semiconductor.
+
+Required properties:
+-------------------
+- compatible: Must be one of
+ "maxim,max77620"
+ "maxim,max20024".
+- reg: I2C device address.
+
+Optional properties:
+-------------------
+- interrupts: The interrupt on the parent the controller is
+ connected to.
+- interrupt-controller: Marks the device node as an interrupt controller.
+- #interrupt-cells: is <2> and their usage is compliant to the 2 cells
+ variant of <../interrupt-controller/interrupts.txt>
+ IRQ numbers for different interrupt source of MAX77620
+ are defined at dt-bindings/mfd/max77620.h.
+
+Optional subnodes and their properties:
+=======================================
+
+Flexible power sequence configurations:
+--------------------------------------
+The Flexible Power Sequencer (FPS) allows each regulator to power up under
+hardware or software control. Additionally, each regulator can power on
+independently or among a group of other regulators with an adjustable power-up
+and power-down delays (sequencing). GPIO1, GPIO2, and GPIO3 can be programmed
+to be part of a sequence allowing external regulators to be sequenced along
+with internal regulators. 32KHz clock can be programmed to be part of a
+sequence.
+
+The flexible sequencing structure consists of two hardware enable inputs
+(EN0, EN1), and 3 master sequencing timers called FPS0, FPS1 and FPS2.
+Each master sequencing timer is programmable through its configuration
+register to have a hardware enable source (EN1 or EN2) or a software enable
+source (SW). When enabled/disabled, the master sequencing timer generates
+eight sequencing events on different time periods called slots. The time
+period between each event is programmable within the configuration register.
+Each regulator, GPIO1, GPIO2, GPIO3, and 32KHz clock has a flexible power
+sequence slave register which allows its enable source to be specified as
+a flexible power sequencer timer or a software bit. When a FPS source of
+regulators, GPIOs and clocks specifies the enable source to be a flexible
+power sequencer, the power up and power down delays can be specified in
+the regulators, GPIOs and clocks flexible power sequencer configuration
+registers.
+
+When FPS event cleared (set to LOW), regulators, GPIOs and 32KHz
+clock are set into following state at the sequencing event that
+corresponds to its flexible sequencer configuration register.
+ Sleep state: In this state, regulators, GPIOs
+ and 32KHz clock get disabled at
+ the sequencing event.
+ Global Low Power Mode (GLPM): In this state, regulators are set in
+ low power mode at the sequencing event.
+
+The configuration parameters of FPS is provided through sub-node "fps"
+and their child for FPS specific. The child node name for FPS are "fps0",
+"fps1", and "fps2" for FPS0, FPS1 and FPS2 respectively.
+
+The FPS configurations like FPS source, power up and power down slots for
+regulators, GPIOs and 32kHz clocks are provided in their respective
+configuration nodes which is explained in respective sub-system DT
+binding document.
+
+There is need for different FPS configuration parameters based on system
+state like when system state changed from active to suspend or active to
+power off (shutdown).
+
+Optional properties:
+-------------------
+-maxim,fps-event-source: u32, FPS event source like external
+ hardware input to PMIC i.e. EN0, EN1 or
+ software (SW).
+ The macros are defined on
+ dt-bindings/mfd/max77620.h
+ for different control source.
+ - MAX77620_FPS_EVENT_SRC_EN0
+ for hardware input pin EN0.
+ - MAX77620_FPS_EVENT_SRC_EN1
+ for hardware input pin EN1.
+ - MAX77620_FPS_EVENT_SRC_SW
+ for software control.
+
+-maxim,shutdown-fps-time-period-us: u32, FPS time period in microseconds
+ when system enters in to shutdown
+ state.
+
+-maxim,suspend-fps-time-period-us: u32, FPS time period in microseconds
+ when system enters in to suspend state.
+
+-maxim,device-state-on-disabled-event: u32, describe the PMIC state when FPS
+ event cleared (set to LOW) whether it
+ should go to sleep state or low-power
+ state. Following are valid values:
+ - MAX77620_FPS_INACTIVE_STATE_SLEEP
+ to set the PMIC state to sleep.
+ - MAX77620_FPS_INACTIVE_STATE_LOW_POWER
+ to set the PMIC state to low
+ power.
+ Absence of this property or other value
+ will not change device state when FPS
+ event get cleared.
+
+Here supported time periods by device in microseconds are as follows:
+MAX77620 supports 40, 80, 160, 320, 640, 1280, 2560 and 5120 microseconds.
+MAX20024 supports 20, 40, 80, 160, 320, 640, 1280 and 2540 microseconds.
+
+For DT binding details of different sub modules like GPIO, pincontrol,
+regulator, power, please refer respective device-tree binding document
+under their respective sub-system directories.
+
+Example:
+--------
+#include <dt-bindings/mfd/max77620.h>
+
+max77620@3c {
+ compatible = "maxim,max77620";
+ reg = <0x3c>;
+
+ interrupt-parent = <&intc>;
+ interrupts = <0 86 IRQ_TYPE_NONE>;
+
+ interrupt-controller;
+ #interrupt-cells = <2>;
+
+ fps {
+ fps0 {
+ maxim,shutdown-fps-time-period-us = <1280>;
+ maxim,fps-event-source = <MAX77620_FPS_EVENT_SRC_EN1>;
+ };
+
+ fps1 {
+ maxim,shutdown-fps-time-period-us = <1280>;
+ maxim,fps-event-source = <MAX77620_FPS_EVENT_SRC_EN0>;
+ };
+
+ fps2 {
+ maxim,shutdown-fps-time-period-us = <1280>;
+ maxim,fps-event-source = <MAX77620_FPS_EVENT_SRC_SW>;
+ };
+ };
+};
diff --git a/Documentation/devicetree/bindings/mfd/qcom-rpm.txt b/Documentation/devicetree/bindings/mfd/qcom-rpm.txt
index 5e97a95..b98b291 100644
--- a/Documentation/devicetree/bindings/mfd/qcom-rpm.txt
+++ b/Documentation/devicetree/bindings/mfd/qcom-rpm.txt
@@ -178,7 +178,7 @@
- qcom,force-mode:
Usage: optional (default if no other qcom,force-mode is specified)
Value type: <u32>
- Defintion: indicates that the regulator should be forced to a
+ Definition: indicates that the regulator should be forced to a
particular mode, valid values are:
QCOM_RPM_FORCE_MODE_NONE - do not force any mode
QCOM_RPM_FORCE_MODE_LPM - force into low power mode
@@ -204,7 +204,7 @@
- qcom,force-mode:
Usage: optional
Value type: <u32>
- Defintion: indicates that the regulator should not be forced to any
+ Definition: indicates that the regulator should not be forced to any
particular mode, valid values are:
QCOM_RPM_FORCE_MODE_NONE - do not force any mode
QCOM_RPM_FORCE_MODE_LPM - force into low power mode
diff --git a/Documentation/devicetree/bindings/mips/brcm/soc.txt b/Documentation/devicetree/bindings/mips/brcm/soc.txt
index 7bab90c..4a7e030 100644
--- a/Documentation/devicetree/bindings/mips/brcm/soc.txt
+++ b/Documentation/devicetree/bindings/mips/brcm/soc.txt
@@ -4,7 +4,8 @@
- compatible: "brcm,bcm3384", "brcm,bcm33843"
"brcm,bcm3384-viper", "brcm,bcm33843-viper"
- "brcm,bcm6328", "brcm,bcm6368",
+ "brcm,bcm6328", "brcm,bcm6358", "brcm,bcm6368",
+ "brcm,bcm63168", "brcm,bcm63268",
"brcm,bcm7125", "brcm,bcm7346", "brcm,bcm7358", "brcm,bcm7360",
"brcm,bcm7362", "brcm,bcm7420", "brcm,bcm7425"
diff --git a/Documentation/devicetree/bindings/mips/cavium/ciu3.txt b/Documentation/devicetree/bindings/mips/cavium/ciu3.txt
new file mode 100644
index 0000000..616862a
--- /dev/null
+++ b/Documentation/devicetree/bindings/mips/cavium/ciu3.txt
@@ -0,0 +1,27 @@
+* Central Interrupt Unit v3
+
+Properties:
+- compatible: "cavium,octeon-7890-ciu3"
+
+ Compatibility with 78XX and 73XX SOCs.
+
+- interrupt-controller: This is an interrupt controller.
+
+- reg: The base address of the CIU's register bank.
+
+- #interrupt-cells: Must be <2>. The first cell is source number.
+ The second cell indicates the triggering semantics, and may have a
+ value of either 4 for level semantics, or 1 for edge semantics.
+
+Example:
+ interrupt-controller@1010000000000 {
+ compatible = "cavium,octeon-7890-ciu3";
+ interrupt-controller;
+ /* Interrupts are specified by two parts:
+ * 1) Source number (20 significant bits)
+ * 2) Trigger type: (4 == level, 1 == edge)
+ */
+ #address-cells = <0>;
+ #interrupt-cells = <2>;
+ reg = <0x10100 0x00000000 0x0 0xb0000000>;
+ };
diff --git a/Documentation/devicetree/bindings/mmc/mmc-pwrseq-emmc.txt b/Documentation/devicetree/bindings/mmc/mmc-pwrseq-emmc.txt
index 0cb827b..3d965d5 100644
--- a/Documentation/devicetree/bindings/mmc/mmc-pwrseq-emmc.txt
+++ b/Documentation/devicetree/bindings/mmc/mmc-pwrseq-emmc.txt
@@ -1,7 +1,7 @@
* The simple eMMC hardware reset provider
The purpose of this driver is to perform standard eMMC hw reset
-procedure, as descibed by Jedec 4.4 specification. This procedure is
+procedure, as described by Jedec 4.4 specification. This procedure is
performed just after MMC core enabled power to the given mmc host (to
fix possible issues if bootloader has left eMMC card in initialized or
unknown state), and before performing complete system reboot (also in
diff --git a/Documentation/devicetree/bindings/mtd/brcm,brcmnand.txt b/Documentation/devicetree/bindings/mtd/brcm,brcmnand.txt
index c2546ce..7066597 100644
--- a/Documentation/devicetree/bindings/mtd/brcm,brcmnand.txt
+++ b/Documentation/devicetree/bindings/mtd/brcm,brcmnand.txt
@@ -24,6 +24,7 @@
brcm,brcmnand-v5.0
brcm,brcmnand-v6.0
brcm,brcmnand-v6.1
+ brcm,brcmnand-v6.2
brcm,brcmnand-v7.0
brcm,brcmnand-v7.1
brcm,brcmnand
@@ -52,7 +53,7 @@
v7.0. Use this property to describe the rare
earlier versions of this core that include WP
- -- Additonal SoC-specific NAND controller properties --
+ -- Additional SoC-specific NAND controller properties --
The NAND controller is integrated differently on the variety of SoCs on which it
is found. Part of this integration involves providing status and enable bits
diff --git a/Documentation/devicetree/bindings/mtd/gpmc-nand.txt b/Documentation/devicetree/bindings/mtd/gpmc-nand.txt
index fb733c4..3ee7e20 100644
--- a/Documentation/devicetree/bindings/mtd/gpmc-nand.txt
+++ b/Documentation/devicetree/bindings/mtd/gpmc-nand.txt
@@ -13,7 +13,11 @@
Required properties:
- - reg: The CS line the peripheral is connected to
+ - compatible: "ti,omap2-nand"
+ - reg: range id (CS number), base offset and length of the
+ NAND I/O space
+ - interrupt-parent: must point to gpmc node
+ - interrupts: Two interrupt specifiers, one for fifoevent, one for termcount.
Optional properties:
@@ -44,6 +48,7 @@
locating ECC errors for BCHx algorithms. SoC devices which have
ELM hardware engines should specify this device node in .dtsi
Using ELM for ECC error correction frees some CPU cycles.
+ - rb-gpios: GPIO specifier for the ready/busy# pin.
For inline partition table parsing (optional):
@@ -55,20 +60,26 @@
gpmc: gpmc@50000000 {
compatible = "ti,am3352-gpmc";
ti,hwmods = "gpmc";
- reg = <0x50000000 0x1000000>;
+ reg = <0x50000000 0x36c>;
interrupts = <100>;
gpmc,num-cs = <8>;
gpmc,num-waitpins = <2>;
#address-cells = <2>;
#size-cells = <1>;
- ranges = <0 0 0x08000000 0x2000>; /* CS0: NAND */
+ ranges = <0 0 0x08000000 0x1000000>; /* CS0 space, 16MB */
elm_id = <&elm>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
nand@0,0 {
- reg = <0 0 0>; /* CS0, offset 0 */
+ compatible = "ti,omap2-nand";
+ reg = <0 0 4>; /* CS0, offset 0, NAND I/O window 4 */
+ interrupt-parent = <&gpmc>;
+ interrupts = <0 IRQ_TYPE_NONE>, <1 IRQ_TYPE NONE>;
nand-bus-width = <16>;
ti,nand-ecc-opt = "bch8";
ti,nand-xfer-type = "polled";
+ rb-gpios = <&gpmc 0 GPIO_ACTIVE_HIGH>; /* gpmc_wait0 */
gpmc,sync-clk-ps = <0>;
gpmc,cs-on-ns = <0>;
diff --git a/Documentation/devicetree/bindings/mtd/nand.txt b/Documentation/devicetree/bindings/mtd/nand.txt
index b53f92e..68342ea 100644
--- a/Documentation/devicetree/bindings/mtd/nand.txt
+++ b/Documentation/devicetree/bindings/mtd/nand.txt
@@ -1,8 +1,31 @@
-* MTD generic binding
+* NAND chip and NAND controller generic binding
+
+NAND controller/NAND chip representation:
+
+The NAND controller should be represented with its own DT node, and all
+NAND chips attached to this controller should be defined as children nodes
+of the NAND controller. This representation should be enforced even for
+simple controllers supporting only one chip.
+
+Mandatory NAND controller properties:
+- #address-cells: depends on your controller. Should at least be 1 to
+ encode the CS line id.
+- #size-cells: depends on your controller. Put zero unless you need a
+ mapping between CS lines and dedicated memory regions
+
+Optional NAND controller properties
+- ranges: only needed if you need to define a mapping between CS lines and
+ memory regions
+
+Optional NAND chip properties:
- nand-ecc-mode : String, operation mode of the NAND ecc mode.
- Supported values are: "none", "soft", "hw", "hw_syndrome", "hw_oob_first",
- "soft_bch".
+ Supported values are: "none", "soft", "hw", "hw_syndrome",
+ "hw_oob_first".
+ Deprecated values:
+ "soft_bch": use "soft" and nand-ecc-algo instead
+- nand-ecc-algo: string, algorithm of NAND ECC.
+ Supported values are: "hamming", "bch".
- nand-bus-width : 8 or 16 bus width if not present 8
- nand-on-flash-bbt: boolean to enable on flash bbt option if not present false
@@ -19,3 +42,19 @@
The interpretation of these parameters is implementation-defined, so not all
implementations must support all possible combinations. However, implementations
are encouraged to further specify the value(s) they support.
+
+Example:
+
+ nand-controller {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ /* controller specific properties */
+
+ nand@0 {
+ reg = <0>;
+ nand-ecc-mode = "soft_bch";
+
+ /* controller specific properties */
+ };
+ };
diff --git a/Documentation/devicetree/bindings/net/hisilicon-hns-nic.txt b/Documentation/devicetree/bindings/net/hisilicon-hns-nic.txt
index b9ff4ba..f0421ee 100644
--- a/Documentation/devicetree/bindings/net/hisilicon-hns-nic.txt
+++ b/Documentation/devicetree/bindings/net/hisilicon-hns-nic.txt
@@ -8,7 +8,7 @@
specifies a reference to the associating hardware driver node.
see Documentation/devicetree/bindings/net/hisilicon-hns-dsaf.txt
- port-id: is the index of port provided by DSAF (the accelerator). DSAF can
- connect to 8 PHYs. Port 0 to 1 are both used for adminstration purpose. They
+ connect to 8 PHYs. Port 0 to 1 are both used for administration purpose. They
are called debug ports.
The remaining 6 PHYs are taken according to the mode of DSAF.
diff --git a/Documentation/devicetree/bindings/net/phy.txt b/Documentation/devicetree/bindings/net/phy.txt
index c00a9a8..bc1c3c8 100644
--- a/Documentation/devicetree/bindings/net/phy.txt
+++ b/Documentation/devicetree/bindings/net/phy.txt
@@ -35,8 +35,6 @@
- broken-turn-around: If set, indicates the PHY device does not correctly
release the turn around line low at the end of a MDIO transaction.
-- reset-gpios: Reference to a GPIO used to reset the phy.
-
Example:
ethernet-phy@0 {
@@ -44,5 +42,4 @@
interrupt-parent = <40000>;
interrupts = <35 1>;
reg = <0>;
- reset-gpios = <&gpio1 17 GPIO_ACTIVE_LOW>;
};
diff --git a/Documentation/devicetree/bindings/net/stmmac.txt b/Documentation/devicetree/bindings/net/stmmac.txt
index 4d302db..95816c5 100644
--- a/Documentation/devicetree/bindings/net/stmmac.txt
+++ b/Documentation/devicetree/bindings/net/stmmac.txt
@@ -51,8 +51,8 @@
AXI register inside the DMA module:
- snps,lpi_en: enable Low Power Interface
- snps,xit_frm: unlock on WoL
- - snps,wr_osr_lmt: max write oustanding req. limit
- - snps,rd_osr_lmt: max read oustanding req. limit
+ - snps,wr_osr_lmt: max write outstanding req. limit
+ - snps,rd_osr_lmt: max read outstanding req. limit
- snps,kbbe: do not cross 1KiB boundary.
- snps,axi_all: align address
- snps,blen: this is a vector of supported burst length.
diff --git a/Documentation/devicetree/bindings/net/ti,dp83867.txt b/Documentation/devicetree/bindings/net/ti,dp83867.txt
index 58d935b..5d21141 100644
--- a/Documentation/devicetree/bindings/net/ti,dp83867.txt
+++ b/Documentation/devicetree/bindings/net/ti,dp83867.txt
@@ -2,7 +2,7 @@
Required properties:
- reg - The ID number for the phy, usually a small integer
- - ti,rx-internal-delay - RGMII Recieve Clock Delay - see dt-bindings/net/ti-dp83867.h
+ - ti,rx-internal-delay - RGMII Receive Clock Delay - see dt-bindings/net/ti-dp83867.h
for applicable values
- ti,tx-internal-delay - RGMII Transmit Clock Delay - see dt-bindings/net/ti-dp83867.h
for applicable values
diff --git a/Documentation/devicetree/bindings/opp/opp.txt b/Documentation/devicetree/bindings/opp/opp.txt
index 601256f..ee91cbd 100644
--- a/Documentation/devicetree/bindings/opp/opp.txt
+++ b/Documentation/devicetree/bindings/opp/opp.txt
@@ -45,7 +45,7 @@
phandle to a OPP table in their DT node. The OPP core will use this phandle to
find the operating points for the device.
-If required, this can be extended for SoC vendor specfic bindings. Such bindings
+If required, this can be extended for SoC vendor specific bindings. Such bindings
should be documented as Documentation/devicetree/bindings/power/<vendor>-opp.txt
and should have a compatible description like: "operating-points-v2-<vendor>".
diff --git a/Documentation/devicetree/bindings/pci/designware-pcie.txt b/Documentation/devicetree/bindings/pci/designware-pcie.txt
index 64f2fff..6c5322c 100644
--- a/Documentation/devicetree/bindings/pci/designware-pcie.txt
+++ b/Documentation/devicetree/bindings/pci/designware-pcie.txt
@@ -31,7 +31,7 @@
Example configuration:
- pcie: pcie@0xdffff000 {
+ pcie: pcie@dffff000 {
compatible = "snps,dw-pcie";
reg = <0xdffff000 0x1000>, /* Controller registers */
<0xd0000000 0x2000>; /* PCI config space */
diff --git a/Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.txt b/Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.txt
index 3be80c6..83aeb1f 100644
--- a/Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.txt
+++ b/Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.txt
@@ -4,8 +4,8 @@
and thus inherits all the common properties defined in designware-pcie.txt.
Required properties:
-- compatible: "fsl,imx6q-pcie"
-- reg: base addresse and length of the pcie controller
+- compatible: "fsl,imx6q-pcie", "fsl,imx6sx-pcie", "fsl,imx6qp-pcie"
+- reg: base address and length of the PCIe controller
- interrupts: A list of interrupt outputs of the controller. Must contain an
entry for each entry in the interrupt-names property.
- interrupt-names: Must include the following entries:
@@ -19,6 +19,20 @@
- fsl,tx-deemph-gen2-6db: Gen2 (6db) De-emphasis value. Default: 20
- fsl,tx-swing-full: Gen2 TX SWING FULL value. Default: 127
- fsl,tx-swing-low: TX launch amplitude swing_low value. Default: 127
+- fsl,max-link-speed: Specify PCI gen for link capability. Must be '2' for
+ gen2, otherwise will default to gen1. Note that the IMX6 LVDS clock outputs
+ do not meet gen2 jitter requirements and thus for gen2 capability a gen2
+ compliant clock generator should be used and configured.
+- reset-gpio: Should specify the GPIO for controlling the PCI bus device reset
+ signal. It's not polarity aware and defaults to active-low reset sequence
+ (L=reset state, H=operation state).
+- reset-gpio-active-high: If present then the reset sequence using the GPIO
+ specified in the "reset-gpio" property is reversed (H=reset state,
+ L=operation state).
+
+Additional required properties for imx6sx-pcie:
+- clock names: Must include the following additional entries:
+ - "pcie_inbound_axi"
Example:
diff --git a/Documentation/devicetree/bindings/pci/hisilicon-pcie.txt b/Documentation/devicetree/bindings/pci/hisilicon-pcie.txt
index b721bea..59c2f47 100644
--- a/Documentation/devicetree/bindings/pci/hisilicon-pcie.txt
+++ b/Documentation/devicetree/bindings/pci/hisilicon-pcie.txt
@@ -34,11 +34,11 @@
ranges = <0x82000000 0 0x00000000 0x220 0x00000000 0 0x10000000>;
num-lanes = <8>;
port-id = <1>;
- #interrupts-cells = <1>;
- interrupts-map-mask = <0xf800 0 0 7>;
- interrupts-map = <0x0 0 0 1 &mbigen_pcie 1 10
- 0x0 0 0 2 &mbigen_pcie 2 11
- 0x0 0 0 3 &mbigen_pcie 3 12
- 0x0 0 0 4 &mbigen_pcie 4 13>;
+ #interrupt-cells = <1>;
+ interrupt-map-mask = <0xf800 0 0 7>;
+ interrupt-map = <0x0 0 0 1 &mbigen_pcie 1 10
+ 0x0 0 0 2 &mbigen_pcie 2 11
+ 0x0 0 0 3 &mbigen_pcie 3 12
+ 0x0 0 0 4 &mbigen_pcie 4 13>;
status = "ok";
};
diff --git a/Documentation/devicetree/bindings/pci/pci-armada8k.txt b/Documentation/devicetree/bindings/pci/pci-armada8k.txt
new file mode 100644
index 0000000..598533a
--- /dev/null
+++ b/Documentation/devicetree/bindings/pci/pci-armada8k.txt
@@ -0,0 +1,38 @@
+* Marvell Armada 7K/8K PCIe interface
+
+This PCIe host controller is based on the Synopsis Designware PCIe IP
+and thus inherits all the common properties defined in designware-pcie.txt.
+
+Required properties:
+- compatible: "marvell,armada8k-pcie"
+- reg: must contain two register regions
+ - the control register region
+ - the config space region
+- reg-names:
+ - "ctrl" for the control register region
+ - "config" for the config space region
+- interrupts: Interrupt specifier for the PCIe controler
+- clocks: reference to the PCIe controller clock
+
+Example:
+
+ pcie@f2600000 {
+ compatible = "marvell,armada8k-pcie", "snps,dw-pcie";
+ reg = <0 0xf2600000 0 0x10000>, <0 0xf6f00000 0 0x80000>;
+ reg-names = "ctrl", "config";
+ #address-cells = <3>;
+ #size-cells = <2>;
+ #interrupt-cells = <1>;
+ device_type = "pci";
+ dma-coherent;
+
+ bus-range = <0 0xff>;
+ ranges = <0x81000000 0 0xf9000000 0 0xf9000000 0 0x10000 /* downstream I/O */
+ 0x82000000 0 0xf6000000 0 0xf6000000 0 0xf00000>; /* non-prefetchable memory */
+ interrupt-map-mask = <0 0 0 0>;
+ interrupt-map = <0 0 0 0 &gic 0 GIC_SPI 32 IRQ_TYPE_LEVEL_HIGH>;
+ interrupts = <GIC_SPI 32 IRQ_TYPE_LEVEL_HIGH>;
+ num-lanes = <1>;
+ clocks = <&cpm_syscon0 1 13>;
+ status = "disabled";
+ };
diff --git a/Documentation/devicetree/bindings/pci/pci-keystone.txt b/Documentation/devicetree/bindings/pci/pci-keystone.txt
index 54eae29..d08a4d5 100644
--- a/Documentation/devicetree/bindings/pci/pci-keystone.txt
+++ b/Documentation/devicetree/bindings/pci/pci-keystone.txt
@@ -56,6 +56,7 @@
phy-names: name of the Generic Keystine SerDes phy for PCI
- If boot loader already does PCI link establishment, then phys and
phy-names shouldn't be present.
+ interrupts: platform interrupt for error interrupts.
Designware DT Properties not applicable for Keystone PCI
diff --git a/Documentation/devicetree/bindings/phy/bcm-ns-usb2-phy.txt b/Documentation/devicetree/bindings/phy/bcm-ns-usb2-phy.txt
new file mode 100644
index 0000000..a7aee9e
--- /dev/null
+++ b/Documentation/devicetree/bindings/phy/bcm-ns-usb2-phy.txt
@@ -0,0 +1,21 @@
+Driver for Broadcom Northstar USB 2.0 PHY
+
+Required properties:
+- compatible: brcm,ns-usb2-phy
+- reg: iomem address range of DMU (Device Management Unit)
+- reg-names: "dmu", the only needed & supported reg right now
+- clocks: USB PHY reference clock
+- clock-names: "phy-ref-clk", the only needed & supported clock right now
+
+To initialize USB 2.0 PHY driver needs to setup PLL correctly. To do this it
+requires passing phandle to the USB PHY reference clock.
+
+Example:
+ usb2-phy {
+ compatible = "brcm,ns-usb2-phy";
+ reg = <0x1800c000 0x1000>;
+ reg-names = "dmu";
+ #phy-cells = <0>;
+ clocks = <&genpll BCM_NSP_GENPLL_USB_PHY_REF_CLK>;
+ clock-names = "phy-ref-clk";
+ };
diff --git a/Documentation/devicetree/bindings/phy/brcm,brcmstb-sata-phy.txt b/Documentation/devicetree/bindings/phy/brcm,brcmstb-sata-phy.txt
deleted file mode 100644
index d87ab7c..0000000
--- a/Documentation/devicetree/bindings/phy/brcm,brcmstb-sata-phy.txt
+++ /dev/null
@@ -1,41 +0,0 @@
-* Broadcom SATA3 PHY for STB
-
-Required properties:
-- compatible: should be one or more of
- "brcm,bcm7425-sata-phy"
- "brcm,bcm7445-sata-phy"
- "brcm,phy-sata3"
-- address-cells: should be 1
-- size-cells: should be 0
-- reg: register range for the PHY PCB interface
-- reg-names: should be "phy"
-
-Sub-nodes:
- Each port's PHY should be represented as a sub-node.
-
-Sub-nodes required properties:
-- reg: the PHY number
-- phy-cells: generic PHY binding; must be 0
-Optional:
-- brcm,enable-ssc: use spread spectrum clocking (SSC) on this port
-
-
-Example:
-
- sata-phy@f0458100 {
- compatible = "brcm,bcm7445-sata-phy", "brcm,phy-sata3";
- reg = <0xf0458100 0x1e00>, <0xf045804c 0x10>;
- reg-names = "phy";
- #address-cells = <1>;
- #size-cells = <0>;
-
- sata-phy@0 {
- reg = <0>;
- #phy-cells = <0>;
- };
-
- sata-phy@1 {
- reg = <1>;
- #phy-cells = <0>;
- };
- };
diff --git a/Documentation/devicetree/bindings/phy/brcm-sata-phy.txt b/Documentation/devicetree/bindings/phy/brcm-sata-phy.txt
new file mode 100644
index 0000000..d023120
--- /dev/null
+++ b/Documentation/devicetree/bindings/phy/brcm-sata-phy.txt
@@ -0,0 +1,44 @@
+* Broadcom SATA3 PHY
+
+Required properties:
+- compatible: should be one or more of
+ "brcm,bcm7425-sata-phy"
+ "brcm,bcm7445-sata-phy"
+ "brcm,iproc-ns2-sata-phy"
+ "brcm,phy-sata3"
+- address-cells: should be 1
+- size-cells: should be 0
+- reg: register ranges for the PHY PCB interface
+- reg-names: should be "phy" and "phy-ctrl"
+ The "phy-ctrl" registers are only required for
+ "brcm,iproc-ns2-sata-phy".
+
+Sub-nodes:
+ Each port's PHY should be represented as a sub-node.
+
+Sub-nodes required properties:
+- reg: the PHY number
+- phy-cells: generic PHY binding; must be 0
+
+Sub-nodes optional properties:
+- brcm,enable-ssc: use spread spectrum clocking (SSC) on this port
+ This property is not applicable for "brcm,iproc-ns2-sata-phy".
+
+Example:
+ sata-phy@f0458100 {
+ compatible = "brcm,bcm7445-sata-phy", "brcm,phy-sata3";
+ reg = <0xf0458100 0x1e00>, <0xf045804c 0x10>;
+ reg-names = "phy";
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ sata-phy@0 {
+ reg = <0>;
+ #phy-cells = <0>;
+ };
+
+ sata-phy@1 {
+ reg = <1>;
+ #phy-cells = <0>;
+ };
+ };
diff --git a/Documentation/devicetree/bindings/phy/phy-mt65xx-usb.txt b/Documentation/devicetree/bindings/phy/phy-mt65xx-usb.txt
index 00100cf..33a2b1e 100644
--- a/Documentation/devicetree/bindings/phy/phy-mt65xx-usb.txt
+++ b/Documentation/devicetree/bindings/phy/phy-mt65xx-usb.txt
@@ -4,7 +4,9 @@
This binding describes a usb3.0 phy for mt65xx platforms of Medaitek SoC.
Required properties (controller (parent) node):
- - compatible : should be "mediatek,mt8173-u3phy"
+ - compatible : should be one of
+ "mediatek,mt2701-u3phy"
+ "mediatek,mt8173-u3phy"
- reg : offset and length of register for phy, exclude port's
register.
- clocks : a list of phandle + clock-specifier pairs, one for each
diff --git a/Documentation/devicetree/bindings/phy/phy-stih41x-usb.txt b/Documentation/devicetree/bindings/phy/phy-stih41x-usb.txt
index 00944a0..744b480 100644
--- a/Documentation/devicetree/bindings/phy/phy-stih41x-usb.txt
+++ b/Documentation/devicetree/bindings/phy/phy-stih41x-usb.txt
@@ -17,7 +17,7 @@
usb2_phy: usb2phy@0 {
compatible = "st,stih416-usb-phy";
- #phy-cell = <0>;
+ #phy-cells = <0>;
st,syscfg = <&syscfg_rear>;
clocks = <&clk_sysin>;
clock-names = "osc_phy";
diff --git a/Documentation/devicetree/bindings/phy/rcar-gen2-phy.txt b/Documentation/devicetree/bindings/phy/rcar-gen2-phy.txt
index d564ba4..91da947 100644
--- a/Documentation/devicetree/bindings/phy/rcar-gen2-phy.txt
+++ b/Documentation/devicetree/bindings/phy/rcar-gen2-phy.txt
@@ -7,6 +7,12 @@
- compatible: "renesas,usb-phy-r8a7790" if the device is a part of R8A7790 SoC.
"renesas,usb-phy-r8a7791" if the device is a part of R8A7791 SoC.
"renesas,usb-phy-r8a7794" if the device is a part of R8A7794 SoC.
+ "renesas,rcar-gen2-usb-phy" for a generic R-Car Gen2 compatible device.
+
+ When compatible with the generic version, nodes must list the
+ SoC-specific version corresponding to the platform first
+ followed by the generic version.
+
- reg: offset and length of the register block.
- #address-cells: number of address cells for the USB channel subnodes, must
be <1>.
@@ -34,7 +40,7 @@
Example (Lager board):
usb-phy@e6590100 {
- compatible = "renesas,usb-phy-r8a7790";
+ compatible = "renesas,usb-phy-r8a7790", "renesas,rcar-gen2-usb-phy";
reg = <0 0xe6590100 0 0x100>;
#address-cells = <1>;
#size-cells = <0>;
diff --git a/Documentation/devicetree/bindings/phy/rcar-gen3-phy-usb2.txt b/Documentation/devicetree/bindings/phy/rcar-gen3-phy-usb2.txt
index eaf7e9b..2281d6c 100644
--- a/Documentation/devicetree/bindings/phy/rcar-gen3-phy-usb2.txt
+++ b/Documentation/devicetree/bindings/phy/rcar-gen3-phy-usb2.txt
@@ -6,6 +6,12 @@
Required properties:
- compatible: "renesas,usb2-phy-r8a7795" if the device is a part of an R8A7795
SoC.
+ "renesas,rcar-gen3-usb2-phy" for a generic R-Car Gen3 compatible device.
+
+ When compatible with the generic version, nodes must list the
+ SoC-specific version corresponding to the platform first
+ followed by the generic version.
+
- reg: offset and length of the partial USB 2.0 Host register block.
- clocks: clock phandle and specifier pair(s).
- #phy-cells: see phy-bindings.txt in the same directory, must be <0>.
@@ -15,18 +21,20 @@
combined, the device tree node should set interrupt properties to use the
channel as USB OTG:
- interrupts: interrupt specifier for the PHY.
+- vbus-supply: Phandle to a regulator that provides power to the VBUS. This
+ regulator will be managed during the PHY power on/off sequence.
Example (R-Car H3):
usb-phy@ee080200 {
- compatible = "renesas,usb2-phy-r8a7795";
+ compatible = "renesas,usb2-phy-r8a7795", "renesas,rcar-gen3-usb2-phy";
reg = <0 0xee080200 0 0x700>;
interrupts = <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp7_clks R8A7795_CLK_EHCI0>;
};
usb-phy@ee0a0200 {
- compatible = "renesas,usb2-phy-r8a7795";
+ compatible = "renesas,usb2-phy-r8a7795", "renesas,rcar-gen3-usb2-phy";
reg = <0 0xee0a0200 0 0x700>;
clocks = <&mstp7_clks R8A7795_CLK_EHCI0>;
};
diff --git a/Documentation/devicetree/bindings/phy/samsung-phy.txt b/Documentation/devicetree/bindings/phy/samsung-phy.txt
index 0289d3b..9872ba8 100644
--- a/Documentation/devicetree/bindings/phy/samsung-phy.txt
+++ b/Documentation/devicetree/bindings/phy/samsung-phy.txt
@@ -2,9 +2,20 @@
-------------------------------------------------
Required properties:
-- compatible : should be "samsung,s5pv210-mipi-video-phy";
+- compatible : should be one of the listed compatibles:
+ - "samsung,s5pv210-mipi-video-phy"
+ - "samsung,exynos5420-mipi-video-phy"
+ - "samsung,exynos5433-mipi-video-phy"
- #phy-cells : from the generic phy bindings, must be 1;
-- syscon - phandle to the PMU system controller;
+
+In case of s5pv210 and exynos5420 compatible PHYs:
+- syscon - phandle to the PMU system controller
+
+In case of exynos5433 compatible PHY:
+ - samsung,pmu-syscon - phandle to the PMU system controller
+ - samsung,disp-sysreg - phandle to the DISP system registers controller
+ - samsung,cam0-sysreg - phandle to the CAM0 system registers controller
+ - samsung,cam1-sysreg - phandle to the CAM1 system registers controller
For "samsung,s5pv210-mipi-video-phy" compatible PHYs the second cell in
the PHY specifier identifies the PHY and its meaning is as follows:
@@ -12,6 +23,9 @@
1 - MIPI DSIM 0,
2 - MIPI CSIS 1,
3 - MIPI DSIM 1.
+"samsung,exynos5420-mipi-video-phy" and "samsung,exynos5433-mipi-video-phy"
+supports additional fifth PHY:
+ 4 - MIPI CSIS 2.
Samsung EXYNOS SoC series Display Port PHY
-------------------------------------------------
diff --git a/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-xusb-padctl.txt b/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-xusb-padctl.txt
index 8a6223d..4048f43a 100644
--- a/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-xusb-padctl.txt
+++ b/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-xusb-padctl.txt
@@ -85,7 +85,7 @@
SoC file extract:
-----------------
- padctl@0,7009f000 {
+ padctl@7009f000 {
compatible = "nvidia,tegra124-xusb-padctl";
reg = <0x0 0x7009f000 0x0 0x1000>;
resets = <&tegra_car 142>;
@@ -97,7 +97,7 @@
Board file extract:
-------------------
- pcie-controller@0,01003000 {
+ pcie-controller@01003000 {
...
phys = <&padctl 0>;
@@ -108,7 +108,7 @@
...
- padctl: padctl@0,7009f000 {
+ padctl: padctl@7009f000 {
pinctrl-0 = <&padctl_default>;
pinctrl-names = "default";
diff --git a/Documentation/devicetree/bindings/pinctrl/qcom,pmic-gpio.txt b/Documentation/devicetree/bindings/pinctrl/qcom,pmic-gpio.txt
index a90c812..a54c39e 100644
--- a/Documentation/devicetree/bindings/pinctrl/qcom,pmic-gpio.txt
+++ b/Documentation/devicetree/bindings/pinctrl/qcom,pmic-gpio.txt
@@ -122,7 +122,7 @@
2: 1.5uA (PMIC_GPIO_PULL_UP_1P5)
3: 31.5uA (PMIC_GPIO_PULL_UP_31P5)
4: 1.5uA + 30uA boost (PMIC_GPIO_PULL_UP_1P5_30)
- If this property is ommited 30uA strength will be used if
+ If this property is omitted 30uA strength will be used if
pull up is selected
- bias-high-impedance:
diff --git a/Documentation/devicetree/bindings/pinctrl/renesas,pfc-pinctrl.txt b/Documentation/devicetree/bindings/pinctrl/renesas,pfc-pinctrl.txt
index ffadb7a..74e6ec0 100644
--- a/Documentation/devicetree/bindings/pinctrl/renesas,pfc-pinctrl.txt
+++ b/Documentation/devicetree/bindings/pinctrl/renesas,pfc-pinctrl.txt
@@ -72,8 +72,8 @@
The pin configuration parameters use the generic pinconf bindings defined in
pinctrl-bindings.txt in this directory. The supported parameters are
-bias-disable, bias-pull-up, bias-pull-down and power-source. For pins that
-have a configurable I/O voltage, the power-source value should be the
+bias-disable, bias-pull-up, bias-pull-down, drive strength and power-source. For
+pins that have a configurable I/O voltage, the power-source value should be the
nominal I/O voltage in millivolts.
diff --git a/Documentation/devicetree/bindings/power/qcom,coincell-charger.txt b/Documentation/devicetree/bindings/power/qcom,coincell-charger.txt
index 0e6d875..7478992 100644
--- a/Documentation/devicetree/bindings/power/qcom,coincell-charger.txt
+++ b/Documentation/devicetree/bindings/power/qcom,coincell-charger.txt
@@ -29,7 +29,7 @@
- qcom,charger-disable:
Usage: optional
Value type: <boolean>
- Definition: definining this property disables charging
+ Definition: defining this property disables charging
This charger is a sub-node of one of the 8941 PMIC blocks, and is specified
as a child node in DTS of that node. See ../mfd/qcom,spmi-pmic.txt and
diff --git a/Documentation/devicetree/bindings/regulator/palmas-pmic.txt b/Documentation/devicetree/bindings/regulator/palmas-pmic.txt
index 725393c..9987281 100644
--- a/Documentation/devicetree/bindings/regulator/palmas-pmic.txt
+++ b/Documentation/devicetree/bindings/regulator/palmas-pmic.txt
@@ -1,5 +1,12 @@
* palmas regulator IP block devicetree bindings
+The tps659038 for the AM57x class have OTP spins that
+have different part numbers but the same functionality. There
+is not a need to add the OTP spins to the palmas driver. The
+spin devices should use the tps659038 as it's compatible value.
+This is the list of those devices:
+tps659037
+
Required properties:
- compatible : Should be from the list
ti,twl6035-pmic
@@ -8,6 +15,7 @@
ti,tps65913-pmic
ti,tps65914-pmic
ti,tps65917-pmic
+ ti,tps659038-pmic
and also the generic series names
ti,palmas-pmic
- interrupt-parent : The parent interrupt controller which is palmas.
diff --git a/Documentation/devicetree/bindings/rtc/maxim-ds1302.txt b/Documentation/devicetree/bindings/rtc/maxim-ds1302.txt
new file mode 100644
index 0000000..ba470c5
--- /dev/null
+++ b/Documentation/devicetree/bindings/rtc/maxim-ds1302.txt
@@ -0,0 +1,46 @@
+* Maxim/Dallas Semiconductor DS-1302 RTC
+
+Simple device which could be used to store date/time between reboots.
+
+The device uses the standard MicroWire half-duplex transfer timing.
+Master output is set on low clock and sensed by the RTC on the rising
+edge. Master input is set by the RTC on the trailing edge and is sensed
+by the master on low clock.
+
+Required properties:
+
+- compatible : Should be "maxim,ds1302"
+
+Required SPI properties:
+
+- reg : Should be address of the device chip select within
+ the controller.
+
+- spi-max-frequency : DS-1302 has 500 kHz if powered at 2.2V,
+ and 2MHz if powered at 5V.
+
+- spi-3wire : The device has a shared signal IN/OUT line.
+
+- spi-lsb-first : DS-1302 requires least significant bit first
+ transfers.
+
+- spi-cs-high: DS-1302 has active high chip select line. This is
+ required unless inverted in hardware.
+
+Example:
+
+spi@901c {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ compatible = "icpdas,lp8841-spi-rtc";
+ reg = <0x901c 0x1>;
+
+ rtc@0 {
+ compatible = "maxim,ds1302";
+ reg = <0>;
+ spi-max-frequency = <500000>;
+ spi-3wire;
+ spi-lsb-first;
+ spi-cs-high;
+ };
+};
diff --git a/Documentation/devicetree/bindings/rtc/rtc-palmas.txt b/Documentation/devicetree/bindings/rtc/rtc-palmas.txt
index adbccc0..eb1c7fd 100644
--- a/Documentation/devicetree/bindings/rtc/rtc-palmas.txt
+++ b/Documentation/devicetree/bindings/rtc/rtc-palmas.txt
@@ -15,9 +15,9 @@
battery is chargeable or not. If charging battery then driver can
enable the charging.
- ti,backup-battery-charge-high-current: Enable high current charging in
- backup battery. Device supports the < 100mA and > 100mA charging.
- The high current will be > 100mA. Absence of this property will
- charge battery to lower current i.e. < 100mA.
+ backup battery. Device supports the < 100uA and > 100uA charging.
+ The high current will be > 100uA. Absence of this property will
+ charge battery to lower current i.e. < 100uA.
Example:
palmas: tps65913@58 {
diff --git a/Documentation/devicetree/bindings/rtc/sa1100-rtc.txt b/Documentation/devicetree/bindings/rtc/sa1100-rtc.txt
index 0cda19a..968ac82 100644
--- a/Documentation/devicetree/bindings/rtc/sa1100-rtc.txt
+++ b/Documentation/devicetree/bindings/rtc/sa1100-rtc.txt
@@ -13,5 +13,5 @@
compatible = "mrvl,mmp-rtc";
reg = <0xd4010000 0x1000>;
interrupts = <5>, <6>;
- interrupt-name = "rtc 1Hz", "rtc alarm";
+ interrupt-names = "rtc 1Hz", "rtc alarm";
};
diff --git a/Documentation/devicetree/bindings/serial/arm,mps2-uart.txt b/Documentation/devicetree/bindings/serial/arm,mps2-uart.txt
new file mode 100644
index 0000000..128cc6a
--- /dev/null
+++ b/Documentation/devicetree/bindings/serial/arm,mps2-uart.txt
@@ -0,0 +1,19 @@
+ARM MPS2 UART
+
+Required properties:
+- compatible : Should be "arm,mps2-uart"
+- reg : Address and length of the register set
+- interrupts : Reference to the UART RX, TX and overrun interrupts
+
+Required clocking property:
+- clocks : The input clock of the UART
+
+
+Examples:
+
+uart0: serial@40004000 {
+ compatible = "arm,mps2-uart";
+ reg = <0x40004000 0x1000>;
+ interrupts = <0 1 12>;
+ clocks = <&sysclk>;
+};
diff --git a/Documentation/devicetree/bindings/serial/fsl-imx-uart.txt b/Documentation/devicetree/bindings/serial/fsl-imx-uart.txt
index ed94c21..1e82802 100644
--- a/Documentation/devicetree/bindings/serial/fsl-imx-uart.txt
+++ b/Documentation/devicetree/bindings/serial/fsl-imx-uart.txt
@@ -6,7 +6,7 @@
- interrupts : Should contain uart interrupt
Optional properties:
-- fsl,uart-has-rtscts : Indicate the uart has rts and cts
+- uart-has-rtscts : Indicate the uart has rts and cts
- fsl,irda-mode : Indicate the uart supports irda mode
- fsl,dte-mode : Indicate the uart works in DTE mode. The uart works
in DCE mode by default.
@@ -24,6 +24,6 @@
compatible = "fsl,imx51-uart", "fsl,imx21-uart";
reg = <0x73fbc000 0x4000>;
interrupts = <31>;
- fsl,uart-has-rtscts;
+ uart-has-rtscts;
fsl,dte-mode;
};
diff --git a/Documentation/devicetree/bindings/serial/fsl-mxs-auart.txt b/Documentation/devicetree/bindings/serial/fsl-mxs-auart.txt
index 7c408c8..5c96d41 100644
--- a/Documentation/devicetree/bindings/serial/fsl-mxs-auart.txt
+++ b/Documentation/devicetree/bindings/serial/fsl-mxs-auart.txt
@@ -1,8 +1,10 @@
* Freescale MXS Application UART (AUART)
-Required properties:
-- compatible : Should be "fsl,<soc>-auart". The supported SoCs include
- imx23 and imx28.
+Required properties for all SoCs:
+- compatible : Should be one of fallowing variants:
+ "fsl,imx23-auart" - Freescale i.MX23
+ "fsl,imx28-auart" - Freescale i.MX28
+ "alphascale,asm9260-auart" - Alphascale ASM9260
- reg : Address and length of the register set for the device
- interrupts : Should contain the auart interrupt numbers
- dmas: DMA specifier, consisting of a phandle to DMA controller node
@@ -10,8 +12,14 @@
Refer to dma.txt and fsl-mxs-dma.txt for details.
- dma-names: "rx" for RX channel, "tx" for TX channel.
+Required properties for "alphascale,asm9260-auart":
+- clocks : the clocks feeding the watchdog timer. See clock-bindings.txt
+- clock-names : should be set to
+ "mod" - source for tick counter.
+ "ahb" - ahb gate.
+
Optional properties:
-- fsl,uart-has-rtscts : Indicate the UART has RTS and CTS lines
+- uart-has-rtscts : Indicate the UART has RTS and CTS lines
for hardware flow control,
it also means you enable the DMA support for this UART.
- {rts,cts,dtr,dsr,rng,dcd}-gpios: specify a GPIO for RTS/CTS/DTR/DSR/RI/DCD
diff --git a/Documentation/devicetree/bindings/serial/microchip,pic32-uart.txt b/Documentation/devicetree/bindings/serial/microchip,pic32-uart.txt
new file mode 100644
index 0000000..65b38bf
--- /dev/null
+++ b/Documentation/devicetree/bindings/serial/microchip,pic32-uart.txt
@@ -0,0 +1,29 @@
+* Microchip Universal Asynchronous Receiver Transmitter (UART)
+
+Required properties:
+- compatible: Should be "microchip,pic32mzda-uart"
+- reg: Should contain registers location and length
+- interrupts: Should contain interrupt
+- clocks: Phandle to the clock.
+ See: Documentation/devicetree/bindings/clock/clock-bindings.txt
+- pinctrl-names: A pinctrl state names "default" must be defined.
+- pinctrl-0: Phandle referencing pin configuration of the UART peripheral.
+ See: Documentation/devicetree/bindings/pinctrl/pinctrl-binding.txt
+
+Optional properties:
+- cts-gpios: CTS pin for UART
+
+Example:
+ uart1: serial@1f822000 {
+ compatible = "microchip,pic32mzda-uart";
+ reg = <0x1f822000 0x50>;
+ interrupts = <112 IRQ_TYPE_LEVEL_HIGH>,
+ <113 IRQ_TYPE_LEVEL_HIGH>,
+ <114 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&PBCLK2>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_uart1
+ &pinctrl_uart1_cts
+ &pinctrl_uart1_rts>;
+ cts-gpios = <&gpio1 15 0>;
+ };
diff --git a/Documentation/devicetree/bindings/tty/serial/mvebu-uart.txt b/Documentation/devicetree/bindings/serial/mvebu-uart.txt
similarity index 100%
rename from Documentation/devicetree/bindings/tty/serial/mvebu-uart.txt
rename to Documentation/devicetree/bindings/serial/mvebu-uart.txt
diff --git a/Documentation/devicetree/bindings/serial/serial.txt b/Documentation/devicetree/bindings/serial/serial.txt
new file mode 100644
index 0000000..fd970f7
--- /dev/null
+++ b/Documentation/devicetree/bindings/serial/serial.txt
@@ -0,0 +1,57 @@
+Generic Serial DT Bindings
+
+This document lists a set of generic properties for describing UARTs in a
+device tree. Whether these properties apply to a particular device depends on
+the DT bindings for the actual device.
+
+Optional properties:
+ - cts-gpios: Must contain a GPIO specifier, referring to the GPIO pin to be
+ used as the UART's CTS line.
+ - dcd-gpios: Must contain a GPIO specifier, referring to the GPIO pin to be
+ used as the UART's DCD line.
+ - dsr-gpios: Must contain a GPIO specifier, referring to the GPIO pin to be
+ used as the UART's DSR line.
+ - dtr-gpios: Must contain a GPIO specifier, referring to the GPIO pin to be
+ used as the UART's DTR line.
+ - rng-gpios: Must contain a GPIO specifier, referring to the GPIO pin to be
+ used as the UART's RNG line.
+ - rts-gpios: Must contain a GPIO specifier, referring to the GPIO pin to be
+ used as the UART's RTS line.
+
+ - uart-has-rtscts: The presence of this property indicates that the
+ UART has dedicated lines for RTS/CTS hardware flow control, and that
+ they are available for use (wired and enabled by pinmux configuration).
+ This depends on both the UART hardware and the board wiring.
+ Note that this property is mutually-exclusive with "cts-gpios" and
+ "rts-gpios" above.
+
+
+Examples:
+
+ uart1: serial@48022000 {
+ compatible = "ti,am3352-uart", "ti,omap3-uart";
+ ti,hwmods = "uart2";
+ clock-frequency = <48000000>;
+ reg = <0x48022000 0x2000>;
+ interrupts = <73>;
+ dmas = <&edma 28 0>, <&edma 29 0>;
+ dma-names = "tx", "rx";
+ dtr-gpios = <&gpio2 22 GPIO_ACTIVE_LOW>;
+ dsr-gpios = <&gpio2 23 GPIO_ACTIVE_LOW>;
+ dcd-gpios = <&gpio2 24 GPIO_ACTIVE_LOW>;
+ rng-gpios = <&gpio2 25 GPIO_ACTIVE_LOW>;
+ cts-gpios = <&gpio0 12 GPIO_ACTIVE_LOW>;
+ rts-gpios = <&gpio0 13 GPIO_ACTIVE_LOW>;
+ status = "okay";
+ };
+
+ scifa4: serial@e6c80000 {
+ compatible = "renesas,scifa-sh73a0", "renesas,scifa";
+ reg = <0xe6c80000 0x100>;
+ interrupts = <GIC_SPI 78 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&mstp2_clks SH73A0_CLK_SCIFA4>;
+ clock-names = "fck";
+ power-domains = <&pd_a3sp>;
+ uart-has-rtscts;
+ status = "okay";
+ };
diff --git a/Documentation/devicetree/bindings/serial/sirf-uart.txt b/Documentation/devicetree/bindings/serial/sirf-uart.txt
index 67e2a0a..1e48bbb 100644
--- a/Documentation/devicetree/bindings/serial/sirf-uart.txt
+++ b/Documentation/devicetree/bindings/serial/sirf-uart.txt
@@ -9,9 +9,9 @@
- clocks : Should contain uart clock number
Optional properties:
-- sirf,uart-has-rtscts: we have hardware flow controller pins in hardware
-- rts-gpios: RTS pin for USP-based UART if sirf,uart-has-rtscts is true
-- cts-gpios: CTS pin for USP-based UART if sirf,uart-has-rtscts is true
+- uart-has-rtscts: we have hardware flow controller pins in hardware
+- rts-gpios: RTS pin for USP-based UART if uart-has-rtscts is true
+- cts-gpios: CTS pin for USP-based UART if uart-has-rtscts is true
Example:
@@ -28,7 +28,7 @@
usp@b0090000 {
compatible = "sirf,prima2-usp-uart";
- sirf,uart-has-rtscts;
+ uart-has-rtscts;
rts-gpios = <&gpio 15 0>;
cts-gpios = <&gpio 46 0>;
};
diff --git a/Documentation/devicetree/bindings/soc/ti/keystone-navigator-qmss.txt b/Documentation/devicetree/bindings/soc/ti/keystone-navigator-qmss.txt
index d1ce21a..64c66a5 100644
--- a/Documentation/devicetree/bindings/soc/ti/keystone-navigator-qmss.txt
+++ b/Documentation/devicetree/bindings/soc/ti/keystone-navigator-qmss.txt
@@ -42,7 +42,7 @@
- queue-pools : child node classifying the queue ranges into pools.
Queue ranges are grouped into 3 type of pools:
- qpend : pool of qpend(interruptible) queues
- - general-purpose : pool of general queues, primarly used
+ - general-purpose : pool of general queues, primarily used
as free descriptor queues or the
transmit DMA queues.
- accumulator : pool of queues on PDSP accumulator channel
@@ -50,7 +50,7 @@
-- qrange : number of queues to use per queue range, specified as
<"base queue #" "# of queues">.
-- interrupts : Optional property to specify the interrupt mapping
- for interruptible queues. The driver additionaly sets
+ for interruptible queues. The driver additionally sets
the interrupt affinity hint based on the cpu mask.
-- qalloc-by-id : Optional property to specify that the queues in this
range can only be allocated by queue id.
@@ -80,7 +80,7 @@
latency : time to delay the interrupt, specified
in microseconds.
-- multi-queue : Optional property to specify that the channel has to
- monitor upto 32 queues starting at the base queue #.
+ monitor up to 32 queues starting at the base queue #.
- descriptor-regions : child node describing the memory regions for keystone
navigator packet DMA descriptors. The memory for
descriptors will be allocated by the driver.
diff --git a/Documentation/devicetree/bindings/sound/davinci-mcbsp.txt b/Documentation/devicetree/bindings/sound/davinci-mcbsp.txt
new file mode 100644
index 0000000..55b53e1
--- /dev/null
+++ b/Documentation/devicetree/bindings/sound/davinci-mcbsp.txt
@@ -0,0 +1,51 @@
+Texas Instruments DaVinci McBSP module
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This binding describes the "Multi-channel Buffered Serial Port" (McBSP)
+audio interface found in some TI DaVinci processors like the OMAP-L138 or AM180x.
+
+
+Required properties:
+~~~~~~~~~~~~~~~~~~~~
+- compatible :
+ "ti,da850-mcbsp" : for DA850, AM180x and OPAM-L138 platforms
+
+- reg : physical base address and length of the controller memory mapped
+ region(s).
+- reg-names : Should contain:
+ * "mpu" for the main registers (required).
+ * "dat" for the data FIFO (optional).
+
+- dmas: three element list of DMA controller phandles, DMA request line and
+ TC channel ordered triplets.
+- dma-names: identifier string for each DMA request line in the dmas property.
+ These strings correspond 1:1 with the ordered pairs in dmas. The dma
+ identifiers must be "rx" and "tx".
+
+Optional properties:
+~~~~~~~~~~~~~~~~~~~~
+- interrupts : Interrupt numbers for McBSP
+- interrupt-names : Known interrupt names are "rx" and "tx"
+
+- pinctrl-0: Should specify pin control group used for this controller.
+- pinctrl-names: Should contain only one value - "default", for more details
+ please refer to pinctrl-bindings.txt
+
+Example (AM1808):
+~~~~~~~~~~~~~~~~~
+
+mcbsp0: mcbsp@1d10000 {
+ compatible = "ti,da850-mcbsp";
+ pinctrl-names = "default";
+ pinctrl-0 = <&mcbsp0_pins>;
+
+ reg = <0x00110000 0x1000>,
+ <0x00310000 0x1000>;
+ reg-names = "mpu", "dat";
+ interrupts = <97 98>;
+ interrupts-names = "rx", "tx";
+ dmas = <&edma0 3 1
+ &edma0 2 1>;
+ dma-names = "tx", "rx";
+ status = "okay";
+};
diff --git a/Documentation/devicetree/bindings/sound/fsl-sai.txt b/Documentation/devicetree/bindings/sound/fsl-sai.txt
index 044e5d7..740b467 100644
--- a/Documentation/devicetree/bindings/sound/fsl-sai.txt
+++ b/Documentation/devicetree/bindings/sound/fsl-sai.txt
@@ -7,8 +7,8 @@
Required properties:
- - compatible : Compatible list, contains "fsl,vf610-sai" or
- "fsl,imx6sx-sai".
+ - compatible : Compatible list, contains "fsl,vf610-sai",
+ "fsl,imx6sx-sai" or "fsl,imx6ul-sai"
- reg : Offset and length of the register set for the device.
@@ -48,6 +48,11 @@
receive data by following their own bit clocks and
frame sync clocks separately.
+Optional properties (for mx6ul):
+
+ - fsl,sai-mclk-direction-output: This is a boolean property. If present,
+ indicates that SAI will output the SAI MCLK clock.
+
Note:
- If both fsl,sai-asynchronous and fsl,sai-synchronous-rx are absent, the
default synchronous mode (sync Rx with Tx) will be used, which means both
diff --git a/Documentation/devicetree/bindings/sound/nvidia,tegra30-hda.txt b/Documentation/devicetree/bindings/sound/nvidia,tegra30-hda.txt
index 275c6ea..44d2745 100644
--- a/Documentation/devicetree/bindings/sound/nvidia,tegra30-hda.txt
+++ b/Documentation/devicetree/bindings/sound/nvidia,tegra30-hda.txt
@@ -15,7 +15,7 @@
Example:
-hda@0,70030000 {
+hda@70030000 {
compatible = "nvidia,tegra124-hda", "nvidia,tegra30-hda";
reg = <0x0 0x70030000 0x0 0x10000>;
interrupts = <GIC_SPI 81 IRQ_TYPE_LEVEL_HIGH>;
diff --git a/Documentation/devicetree/bindings/sound/pcm5102a.txt b/Documentation/devicetree/bindings/sound/pcm5102a.txt
new file mode 100644
index 0000000..c63ab0b6
--- /dev/null
+++ b/Documentation/devicetree/bindings/sound/pcm5102a.txt
@@ -0,0 +1,13 @@
+PCM5102a audio CODECs
+
+These devices does not use I2C or SPI.
+
+Required properties:
+
+ - compatible : set as "ti,pcm5102a"
+
+Examples:
+
+ pcm5102a: pcm5102a {
+ compatible = "ti,pcm5102a";
+ };
diff --git a/Documentation/devicetree/bindings/spi/microchip,spi-pic32.txt b/Documentation/devicetree/bindings/spi/microchip,spi-pic32.txt
new file mode 100644
index 0000000..79de379f
--- /dev/null
+++ b/Documentation/devicetree/bindings/spi/microchip,spi-pic32.txt
@@ -0,0 +1,34 @@
+Microchip PIC32 SPI Master controller
+
+Required properties:
+- compatible: Should be "microchip,pic32mzda-spi".
+- reg: Address and length of register space for the device.
+- interrupts: Should contain all three spi interrupts in sequence
+ of <fault-irq>, <receive-irq>, <transmit-irq>.
+- interrupt-names: Should be "fault", "rx", "tx" in order.
+- clocks: Phandle of the clock generating SPI clock on the bus.
+- clock-names: Should be "mck0".
+- cs-gpios: Specifies the gpio pins to be used for chipselects.
+ See: Documentation/devicetree/bindings/spi/spi-bus.txt
+
+Optional properties:
+- dmas: Two or more DMA channel specifiers following the convention outlined
+ in Documentation/devicetree/bindings/dma/dma.txt
+- dma-names: Names for the dma channels. There must be at least one channel
+ named "spi-tx" for transmit and named "spi-rx" for receive.
+
+Example:
+
+spi1: spi@1f821000 {
+ compatible = "microchip,pic32mzda-spi";
+ reg = <0x1f821000 0x200>;
+ interrupts = <109 IRQ_TYPE_LEVEL_HIGH>,
+ <110 IRQ_TYPE_LEVEL_HIGH>,
+ <111 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "fault", "rx", "tx";
+ clocks = <&PBCLK2>;
+ clock-names = "mck0";
+ cs-gpios = <&gpio3 4 GPIO_ACTIVE_LOW>;
+ dmas = <&dma 134>, <&dma 135>;
+ dma-names = "spi-rx", "spi-tx";
+};
diff --git a/Documentation/devicetree/bindings/spi/spi-fsl-dspi.txt b/Documentation/devicetree/bindings/spi/spi-fsl-dspi.txt
index 1ad0fe3..ff5893d 100644
--- a/Documentation/devicetree/bindings/spi/spi-fsl-dspi.txt
+++ b/Documentation/devicetree/bindings/spi/spi-fsl-dspi.txt
@@ -16,8 +16,7 @@
Optional property:
- big-endian: If present the dspi device's registers are implemented
- in big endian mode, otherwise in native mode(same with CPU), for more
- detail please see: Documentation/devicetree/bindings/regmap/regmap.txt.
+ in big endian mode.
Optional SPI slave node properties:
- fsl,spi-cs-sck-delay: a delay in nanoseconds between activating chip
diff --git a/Documentation/devicetree/bindings/spi/sqi-pic32.txt b/Documentation/devicetree/bindings/spi/sqi-pic32.txt
new file mode 100644
index 0000000..c82d021
--- /dev/null
+++ b/Documentation/devicetree/bindings/spi/sqi-pic32.txt
@@ -0,0 +1,18 @@
+Microchip PIC32 Quad SPI controller
+-----------------------------------
+Required properties:
+- compatible: Should be "microchip,pic32mzda-sqi".
+- reg: Address and length of SQI controller register space.
+- interrupts: Should contain SQI interrupt.
+- clocks: Should contain phandle of two clocks in sequence, one that drives
+ clock on SPI bus and other that drives SQI controller.
+- clock-names: Should be "spi_ck" and "reg_ck" in order.
+
+Example:
+ sqi1: spi@1f8e2000 {
+ compatible = "microchip,pic32mzda-sqi";
+ reg = <0x1f8e2000 0x200>;
+ clocks = <&rootclk REF2CLK>, <&rootclk PB5CLK>;
+ clock-names = "spi_ck", "reg_ck";
+ interrupts = <169 IRQ_TYPE_LEVEL_HIGH>;
+ };
diff --git a/Documentation/devicetree/bindings/sram/sram.txt b/Documentation/devicetree/bindings/sram/sram.txt
index 227e3a3..add48f0 100644
--- a/Documentation/devicetree/bindings/sram/sram.txt
+++ b/Documentation/devicetree/bindings/sram/sram.txt
@@ -51,7 +51,7 @@
compatible = "mmio-sram";
reg = <0x5c000000 0x40000>; /* 256 KiB SRAM at address 0x5c000000 */
- #adress-cells = <1>;
+ #address-cells = <1>;
#size-cells = <1>;
ranges = <0 0x5c000000 0x40000>;
diff --git a/Documentation/devicetree/bindings/thermal/nvidia,tegra124-soctherm.txt b/Documentation/devicetree/bindings/thermal/nvidia,tegra124-soctherm.txt
new file mode 100644
index 0000000..6908d3a
--- /dev/null
+++ b/Documentation/devicetree/bindings/thermal/nvidia,tegra124-soctherm.txt
@@ -0,0 +1,55 @@
+Tegra124 SOCTHERM thermal management system
+
+The SOCTHERM IP block contains thermal sensors, support for polled
+or interrupt-based thermal monitoring, CPU and GPU throttling based
+on temperature trip points, and handling external overcurrent
+notifications. It is also used to manage emergency shutdown in an
+overheating situation.
+
+Required properties :
+- compatible : For Tegra124, must contain "nvidia,tegra124-soctherm".
+ For Tegra132, must contain "nvidia,tegra132-soctherm".
+ For Tegra210, must contain "nvidia,tegra210-soctherm".
+- reg : Should contain 1 entry:
+ - SOCTHERM register set
+- interrupts : Defines the interrupt used by SOCTHERM
+- clocks : Must contain an entry for each entry in clock-names.
+ See ../clocks/clock-bindings.txt for details.
+- clock-names : Must include the following entries:
+ - tsensor
+ - soctherm
+- resets : Must contain an entry for each entry in reset-names.
+ See ../reset/reset.txt for details.
+- reset-names : Must include the following entries:
+ - soctherm
+- #thermal-sensor-cells : Should be 1. See ./thermal.txt for a description
+ of this property. See <dt-bindings/thermal/tegra124-soctherm.h> for a
+ list of valid values when referring to thermal sensors.
+
+
+Example :
+
+ soctherm@700e2000 {
+ compatible = "nvidia,tegra124-soctherm";
+ reg = <0x0 0x700e2000 0x0 0x1000>;
+ interrupts = <GIC_SPI 48 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&tegra_car TEGRA124_CLK_TSENSOR>,
+ <&tegra_car TEGRA124_CLK_SOC_THERM>;
+ clock-names = "tsensor", "soctherm";
+ resets = <&tegra_car 78>;
+ reset-names = "soctherm";
+
+ #thermal-sensor-cells = <1>;
+ };
+
+Example: referring to thermal sensors :
+
+ thermal-zones {
+ cpu {
+ polling-delay-passive = <1000>;
+ polling-delay = <1000>;
+
+ thermal-sensors =
+ <&soctherm TEGRA124_SOCTHERM_SENSOR_CPU>;
+ };
+ };
diff --git a/Documentation/devicetree/bindings/thermal/tegra-soctherm.txt b/Documentation/devicetree/bindings/thermal/tegra-soctherm.txt
deleted file mode 100644
index 6b68cd1..0000000
--- a/Documentation/devicetree/bindings/thermal/tegra-soctherm.txt
+++ /dev/null
@@ -1,55 +0,0 @@
-Tegra124 SOCTHERM thermal management system
-
-The SOCTHERM IP block contains thermal sensors, support for polled
-or interrupt-based thermal monitoring, CPU and GPU throttling based
-on temperature trip points, and handling external overcurrent
-notifications. It is also used to manage emergency shutdown in an
-overheating situation.
-
-Required properties :
-- compatible : For Tegra124, must contain "nvidia,tegra124-soctherm".
- For Tegra132, must contain "nvidia,tegra132-soctherm".
- For Tegra210, must contain "nvidia,tegra210-soctherm".
-- reg : Should contain 1 entry:
- - SOCTHERM register set
-- interrupts : Defines the interrupt used by SOCTHERM
-- clocks : Must contain an entry for each entry in clock-names.
- See ../clocks/clock-bindings.txt for details.
-- clock-names : Must include the following entries:
- - tsensor
- - soctherm
-- resets : Must contain an entry for each entry in reset-names.
- See ../reset/reset.txt for details.
-- reset-names : Must include the following entries:
- - soctherm
-- #thermal-sensor-cells : Should be 1. See ./thermal.txt for a description
- of this property. See <dt-bindings/thermal/tegra124-soctherm.h> for a
- list of valid values when referring to thermal sensors.
-
-
-Example :
-
- soctherm@0,700e2000 {
- compatible = "nvidia,tegra124-soctherm";
- reg = <0x0 0x700e2000 0x0 0x1000>;
- interrupts = <GIC_SPI 48 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&tegra_car TEGRA124_CLK_TSENSOR>,
- <&tegra_car TEGRA124_CLK_SOC_THERM>;
- clock-names = "tsensor", "soctherm";
- resets = <&tegra_car 78>;
- reset-names = "soctherm";
-
- #thermal-sensor-cells = <1>;
- };
-
-Example: referring to thermal sensors :
-
- thermal-zones {
- cpu {
- polling-delay-passive = <1000>;
- polling-delay = <1000>;
-
- thermal-sensors =
- <&soctherm TEGRA124_SOCTHERM_SENSOR_CPU>;
- };
- };
diff --git a/Documentation/devicetree/bindings/timer/ezchip,nps400-timer.txt b/Documentation/devicetree/bindings/timer/ezchip,nps400-timer.txt
new file mode 100644
index 0000000..c8c03d7
--- /dev/null
+++ b/Documentation/devicetree/bindings/timer/ezchip,nps400-timer.txt
@@ -0,0 +1,15 @@
+NPS Network Processor
+
+Required properties:
+
+- compatible : should be "ezchip,nps400-timer"
+
+Clocks required for compatible = "ezchip,nps400-timer":
+- clocks : Must contain a single entry describing the clock input
+
+Example:
+
+timer {
+ compatible = "ezchip,nps400-timer";
+ clocks = <&sysclk>;
+};
diff --git a/Documentation/devicetree/bindings/timer/snps,arc-timer.txt b/Documentation/devicetree/bindings/timer/snps,arc-timer.txt
new file mode 100644
index 0000000..4ef0246
--- /dev/null
+++ b/Documentation/devicetree/bindings/timer/snps,arc-timer.txt
@@ -0,0 +1,31 @@
+Synopsys ARC Local Timer with Interrupt Capabilities
+- Found on all ARC CPUs (ARC700/ARCHS)
+- Can be optionally programmed to interrupt on Limit
+- Two idential copies TIMER0 and TIMER1 exist in ARC cores and historically
+ TIMER0 used as clockevent provider (true for all ARC cores)
+ TIMER1 used for clocksource (mandatory for ARC700, optional for ARC HS)
+
+Required properties:
+
+- compatible : should be "snps,arc-timer"
+- interrupts : single Interrupt going into parent intc
+ (16 for ARCHS cores, 3 for ARC700 cores)
+- clocks : phandle to the source clock
+
+Optional properties:
+
+- interrupt-parent : phandle to parent intc
+
+Example:
+
+ timer0 {
+ compatible = "snps,arc-timer";
+ interrupts = <3>;
+ interrupt-parent = <&core_intc>;
+ clocks = <&core_clk>;
+ };
+
+ timer1 {
+ compatible = "snps,arc-timer";
+ clocks = <&core_clk>;
+ };
diff --git a/Documentation/devicetree/bindings/timer/snps,archs-gfrc.txt b/Documentation/devicetree/bindings/timer/snps,archs-gfrc.txt
new file mode 100644
index 0000000..b6cd1b3
--- /dev/null
+++ b/Documentation/devicetree/bindings/timer/snps,archs-gfrc.txt
@@ -0,0 +1,14 @@
+Synopsys ARC Free Running 64-bit Global Timer for ARC HS CPUs
+- clocksource provider for SMP SoC
+
+Required properties:
+
+- compatible : should be "snps,archs-gfrc"
+- clocks : phandle to the source clock
+
+Example:
+
+ gfrc {
+ compatible = "snps,archs-gfrc";
+ clocks = <&core_clk>;
+ };
diff --git a/Documentation/devicetree/bindings/timer/snps,archs-rtc.txt b/Documentation/devicetree/bindings/timer/snps,archs-rtc.txt
new file mode 100644
index 0000000..47bd7a7
--- /dev/null
+++ b/Documentation/devicetree/bindings/timer/snps,archs-rtc.txt
@@ -0,0 +1,14 @@
+Synopsys ARC Free Running 64-bit Local Timer for ARC HS CPUs
+- clocksource provider for UP SoC
+
+Required properties:
+
+- compatible : should be "snps,archs-rtc"
+- clocks : phandle to the source clock
+
+Example:
+
+ rtc {
+ compatible = "snps,arc-rtc";
+ clocks = <&core_clk>;
+ };
diff --git a/Documentation/devicetree/bindings/usb/dwc3.txt b/Documentation/devicetree/bindings/usb/dwc3.txt
index fb2ad0a..7d7ce08 100644
--- a/Documentation/devicetree/bindings/usb/dwc3.txt
+++ b/Documentation/devicetree/bindings/usb/dwc3.txt
@@ -14,7 +14,6 @@
the second element is expected to be a handle to the USB3/SS PHY
- phys: from the *Generic PHY* bindings
- phy-names: from the *Generic PHY* bindings
- - tx-fifo-resize: determines if the FIFO *has* to be reallocated.
- snps,usb3_lpm_capable: determines if platform is USB3 LPM capable
- snps,disable_scramble_quirk: true when SW should disable data scrambling.
Only really useful for FPGA builds.
@@ -38,6 +37,8 @@
- snps,dis_u2_susphy_quirk: when set core will disable USB2 suspend phy.
- snps,dis_enblslpm_quirk: when set clears the enblslpm in GUSB2PHYCFG,
disabling the suspend signal to the PHY.
+ - snps,dis_rxdet_inp3_quirk: when set core will disable receiver detection
+ in PHY P3 power state.
- snps,is-utmi-l1-suspend: true when DWC3 asserts output signal
utmi_l1_suspend_n, false when asserts utmi_sleep_n
- snps,hird-threshold: HIRD threshold
@@ -47,6 +48,8 @@
register for post-silicon frame length adjustment when the
fladj_30mhz_sdbnd signal is invalid or incorrect.
+ - <DEPRECATED> tx-fifo-resize: determines if the FIFO *has* to be reallocated.
+
This is usually a subnode to DWC3 glue to which it is connected.
dwc3@4a030000 {
@@ -54,5 +57,4 @@
reg = <0x4a030000 0xcfff>;
interrupts = <0 92 4>
usb-phy = <&usb2_phy>, <&usb3,phy>;
- tx-fifo-resize;
};
diff --git a/Documentation/devicetree/bindings/usb/qcom,dwc3.txt b/Documentation/devicetree/bindings/usb/qcom,dwc3.txt
index ca164e7..39acb08 100644
--- a/Documentation/devicetree/bindings/usb/qcom,dwc3.txt
+++ b/Documentation/devicetree/bindings/usb/qcom,dwc3.txt
@@ -59,7 +59,6 @@
interrupts = <0 205 0x4>;
phys = <&hs_phy>, <&ss_phy>;
phy-names = "usb2-phy", "usb3-phy";
- tx-fifo-resize;
dr_mode = "host";
};
};
diff --git a/Documentation/devicetree/bindings/vendor-prefixes.txt b/Documentation/devicetree/bindings/vendor-prefixes.txt
index 3af48e8..a7440bc 100644
--- a/Documentation/devicetree/bindings/vendor-prefixes.txt
+++ b/Documentation/devicetree/bindings/vendor-prefixes.txt
@@ -23,11 +23,13 @@
ampire Ampire Co., Ltd.
ams AMS AG
amstaos AMS-Taos Inc.
+analogix Analogix Semiconductor, Inc.
apm Applied Micro Circuits Corporation (APM)
aptina Aptina Imaging
arasan Arasan Chip Systems
arm ARM Ltd.
armadeus ARMadeus Systems SARL
+arrow Arrow Electronics
artesyn Artesyn Embedded Technologies Inc.
asahi-kasei Asahi Kasei Corp.
aspeed ASPEED Technology Inc.
@@ -60,6 +62,7 @@
compulab CompuLab Ltd.
cortina Cortina Systems, Inc.
cosmic Cosmic Circuits
+creative Creative Technology Ltd
crystalfontz Crystalfontz America, Inc.
cubietech Cubietech, Ltd.
cypress Cypress Semiconductor Corporation
@@ -72,11 +75,14 @@
dlg Dialog Semiconductor
dlink D-Link Corporation
dmo Data Modul AG
+dptechnics DPTechnics
+dragino Dragino Technology Co., Limited
ea Embedded Artists AB
ebv EBV Elektronik
edt Emerging Display Technologies
eeti eGalax_eMPIA Technology Inc
elan Elan Microelectronic Corp.
+embest Shenzhen Embest Technology Co., Ltd.
emmicro EM Microelectronic
energymicro Silicon Laboratories (formerly Energy Micro AS)
epcos EPCOS AG
@@ -88,6 +94,7 @@
everest Everest Semiconductor Co. Ltd.
everspin Everspin Technologies, Inc.
excito Excito
+ezchip EZchip Semiconductor
fcs Fairchild Semiconductor
firefly Firefly
focaltech FocalTech Systems Co.,Ltd
@@ -121,6 +128,7 @@
ifi Ingenieurburo Fur Ic-Technologie (I/F/I)
iom Iomega Corporation
img Imagination Technologies Ltd.
+inforce Inforce Computing
ingenic Ingenic Semiconductor
innolux Innolux Corporation
intel Intel Corporation
@@ -144,6 +152,7 @@
lltc Linear Technology Corporation
marvell Marvell Technology Group Ltd.
maxim Maxim Integrated Products
+meas Measurement Specialties
mediatek MediaTek Inc.
melexis Melexis N.V.
merrii Merrii Technology Co., Ltd.
@@ -175,7 +184,9 @@
nxp NXP Semiconductors
okaya Okaya Electric America, Inc.
olimex OLIMEX Ltd.
+onion Onion Corporation
onnn ON Semiconductor Corp.
+ontat On Tat Industrial Company
opencores OpenCores.org
option Option NV
ortustech Ortus Technology Co., Ltd.
@@ -252,6 +263,7 @@
toshiba Toshiba Corporation
toumaz Toumaz
tplink TP-LINK Technologies Co., Ltd.
+tpk TPK U.S.A. LLC
tronfy Tronfy
tronsmart Tronsmart
truly Truly Semiconductors Limited
diff --git a/Documentation/devicetree/bindings/video/bridge/anx7814.txt b/Documentation/devicetree/bindings/video/bridge/anx7814.txt
new file mode 100644
index 0000000..b2a22c2
--- /dev/null
+++ b/Documentation/devicetree/bindings/video/bridge/anx7814.txt
@@ -0,0 +1,40 @@
+Analogix ANX7814 SlimPort (Full-HD Transmitter)
+-----------------------------------------------
+
+The ANX7814 is an ultra-low power Full-HD (1080p60) SlimPort transmitter
+designed for portable devices.
+
+Required properties:
+
+ - compatible : "analogix,anx7814"
+ - reg : I2C address of the device
+ - interrupt-parent : Should be the phandle of the interrupt controller
+ that services interrupts for this device
+ - interrupts : Should contain the INTP interrupt
+ - hpd-gpios : Which GPIO to use for hpd
+ - pd-gpios : Which GPIO to use for power down
+ - reset-gpios : Which GPIO to use for reset
+
+Optional properties:
+
+ - dvdd10-supply : Regulator for 1.0V digital core power.
+ - Video port for HDMI input, using the DT bindings defined in [1].
+
+[1]: Documentation/devicetree/bindings/media/video-interfaces.txt
+
+Example:
+
+ anx7814: anx7814@38 {
+ compatible = "analogix,anx7814";
+ reg = <0x38>;
+ interrupt-parent = <&gpio0>;
+ interrupts = <99 IRQ_TYPE_LEVEL_LOW>; /* INTP */
+ hpd-gpios = <&pio 36 GPIO_ACTIVE_HIGH>;
+ pd-gpios = <&pio 33 GPIO_ACTIVE_HIGH>;
+ reset-gpios = <&pio 98 GPIO_ACTIVE_HIGH>;
+ port {
+ anx7814_in: endpoint {
+ remote-endpoint = <&hdmi0_out>;
+ };
+ };
+ };
diff --git a/Documentation/devicetree/bindings/watchdog/microchip,pic32-dmt.txt b/Documentation/devicetree/bindings/watchdog/microchip,pic32-dmt.txt
new file mode 100644
index 0000000..852f694
--- /dev/null
+++ b/Documentation/devicetree/bindings/watchdog/microchip,pic32-dmt.txt
@@ -0,0 +1,19 @@
+* Microchip PIC32 Deadman Timer
+
+The deadman timer is used to reset the processor in the event of a software
+malfunction. It is a free-running instruction fetch timer, which is clocked
+whenever an instruction fetch occurs until a count match occurs.
+
+Required properties:
+- compatible: must be "microchip,pic32mzda-dmt".
+- reg: physical base address of the controller and length of memory mapped
+ region.
+- clocks: phandle of parent clock (should be &PBCLK7).
+
+Example:
+
+ watchdog@1f800a00 {
+ compatible = "microchip,pic32mzda-dmt";
+ reg = <0x1f800a00 0x80>;
+ clocks = <&PBCLK7>;
+ };
diff --git a/Documentation/devicetree/bindings/watchdog/microchip,pic32-wdt.txt b/Documentation/devicetree/bindings/watchdog/microchip,pic32-wdt.txt
new file mode 100644
index 0000000..d140103
--- /dev/null
+++ b/Documentation/devicetree/bindings/watchdog/microchip,pic32-wdt.txt
@@ -0,0 +1,18 @@
+* Microchip PIC32 Watchdog Timer
+
+When enabled, the watchdog peripheral can be used to reset the device if the
+WDT is not cleared periodically in software.
+
+Required properties:
+- compatible: must be "microchip,pic32mzda-wdt".
+- reg: physical base address of the controller and length of memory mapped
+ region.
+- clocks: phandle of source clk. should be <&LPRC> clk.
+
+Example:
+
+ watchdog@1f800800 {
+ compatible = "microchip,pic32mzda-wdt";
+ reg = <0x1f800800 0x200>;
+ clocks = <&LPRC>;
+ };
diff --git a/Documentation/driver-model/devres.txt b/Documentation/driver-model/devres.txt
index 73b98df..c63eea0 100644
--- a/Documentation/driver-model/devres.txt
+++ b/Documentation/driver-model/devres.txt
@@ -236,6 +236,7 @@
CLOCK
devm_clk_get()
devm_clk_put()
+ devm_clk_hw_register()
DMA
dmam_alloc_coherent()
@@ -267,6 +268,13 @@
devm_iio_kfifo_free()
devm_iio_trigger_alloc()
devm_iio_trigger_free()
+ devm_iio_channel_get()
+ devm_iio_channel_release()
+ devm_iio_channel_get_all()
+ devm_iio_channel_release_all()
+
+INPUT
+ devm_input_allocate_device()
IO region
devm_release_mem_region()
@@ -317,6 +325,9 @@
devm_kvasprintf()
devm_kzalloc()
+MFD
+ devm_mfd_add_devices()
+
PCI
pcim_enable_device() : after success, all PCI ops become managed
pcim_pin_device() : keep PCI device enabled after release
@@ -328,6 +339,8 @@
PINCTRL
devm_pinctrl_get()
devm_pinctrl_put()
+ devm_pinctrl_register()
+ devm_pinctrl_unregister()
PWM
devm_pwm_get()
diff --git a/Documentation/fb/udlfb.txt b/Documentation/fb/udlfb.txt
index 57d2f29..c985cb6 100644
--- a/Documentation/fb/udlfb.txt
+++ b/Documentation/fb/udlfb.txt
@@ -9,7 +9,7 @@
USB wire. That hardware framebuffer is able to drive the VGA, DVI, or HDMI
monitor with no CPU involvement until a pixel has to change.
-The CPU or other local resource does all the rendering; optinally compares the
+The CPU or other local resource does all the rendering; optionally compares the
result with a local shadow of the remote hardware framebuffer to identify
the minimal set of pixels that have changed; and compresses and sends those
pixels line-by-line via USB bulk transfers.
@@ -66,10 +66,10 @@
At that point, a /dev/fb? interface will be present for user-mode applications
to open and begin writing to the framebuffer of the DisplayLink device using
standard fbdev calls. Note that if mmap() is used, by default the user mode
-application must send down damage notifcations to trigger repaints of the
+application must send down damage notifications to trigger repaints of the
changed regions. Alternatively, udlfb can be recompiled with experimental
defio support enabled, to support a page-fault based detection mechanism
-that can work without explicit notifcation.
+that can work without explicit notification.
The most common client of udlfb is xf86-video-displaylink or a modified
xf86-video-fbdev X server. These servers have no real DisplayLink specific
diff --git a/Documentation/features/perf/perf-regs/arch-support.txt b/Documentation/features/perf/perf-regs/arch-support.txt
index e2b4a78..f179b1f 100644
--- a/Documentation/features/perf/perf-regs/arch-support.txt
+++ b/Documentation/features/perf/perf-regs/arch-support.txt
@@ -27,7 +27,7 @@
| nios2: | TODO |
| openrisc: | TODO |
| parisc: | TODO |
- | powerpc: | TODO |
+ | powerpc: | ok |
| s390: | TODO |
| score: | TODO |
| sh: | TODO |
diff --git a/Documentation/features/perf/perf-stackdump/arch-support.txt b/Documentation/features/perf/perf-stackdump/arch-support.txt
index 3dc24b0..85777c5 100644
--- a/Documentation/features/perf/perf-stackdump/arch-support.txt
+++ b/Documentation/features/perf/perf-stackdump/arch-support.txt
@@ -27,7 +27,7 @@
| nios2: | TODO |
| openrisc: | TODO |
| parisc: | TODO |
- | powerpc: | TODO |
+ | powerpc: | ok |
| s390: | TODO |
| score: | TODO |
| sh: | TODO |
diff --git a/Documentation/filesystems/cifs/README b/Documentation/filesystems/cifs/README
index 2d5622f..a5478840 100644
--- a/Documentation/filesystems/cifs/README
+++ b/Documentation/filesystems/cifs/README
@@ -272,7 +272,7 @@
same domain (e.g. running winbind or nss_ldap) and
the server supports the Unix Extensions then the uid
and gid can be retrieved from the server (and uid
- and gid would not have to be specifed on the mount.
+ and gid would not have to be specified on the mount.
For servers which do not support the CIFS Unix
extensions, the default uid (and gid) returned on lookup
of existing files will be the uid (gid) of the person
diff --git a/Documentation/filesystems/nilfs2.txt b/Documentation/filesystems/nilfs2.txt
index 41c3d33..5b21ef7 100644
--- a/Documentation/filesystems/nilfs2.txt
+++ b/Documentation/filesystems/nilfs2.txt
@@ -268,3 +268,8 @@
( regular file, directory, or symlink )
For detail on the format of each file, please see include/linux/nilfs2_fs.h.
+
+There are no patents or other intellectual property that we protect
+with regard to the design of NILFS2. It is allowed to replicate the
+design in hopes that other operating systems could share (mount, read,
+write, etc.) data stored in this format.
diff --git a/Documentation/filesystems/pohmelfs/design_notes.txt b/Documentation/filesystems/pohmelfs/design_notes.txt
index 8aef9133..106d17f 100644
--- a/Documentation/filesystems/pohmelfs/design_notes.txt
+++ b/Documentation/filesystems/pohmelfs/design_notes.txt
@@ -29,7 +29,7 @@
* Read request (data read, directory listing, lookup requests) balancing between multiple servers.
* Write requests are replicated to multiple servers and completed only when all of them are acked.
* Ability to add and/or remove servers from the working set at run-time.
- * Strong authentification and possible data encryption in network channel.
+ * Strong authentication and possible data encryption in network channel.
* Extended attributes support.
POHMELFS is based on transactions, which are potentially long-standing objects that live
diff --git a/Documentation/filesystems/proc.txt b/Documentation/filesystems/proc.txt
index 7f5607a..e8d0075 100644
--- a/Documentation/filesystems/proc.txt
+++ b/Documentation/filesystems/proc.txt
@@ -225,6 +225,7 @@
TracerPid PID of process tracing this process (0 if not)
Uid Real, effective, saved set, and file system UIDs
Gid Real, effective, saved set, and file system GIDs
+ Umask file mode creation mask
FDSize number of file descriptor slots currently allocated
Groups supplementary group list
NStgid descendant namespace thread group ID hierarchy
diff --git a/Documentation/filesystems/qnx6.txt b/Documentation/filesystems/qnx6.txt
index 4086797..4f3d6a8 100644
--- a/Documentation/filesystems/qnx6.txt
+++ b/Documentation/filesystems/qnx6.txt
@@ -16,7 +16,7 @@
concepts of blocks, inodes and directories.
On QNX it is possible to create little endian and big endian qnx6 filesystems.
This feature makes it possible to create and use a different endianness fs
-for the target (QNX is used on quite a range of embedded systems) plattform
+for the target (QNX is used on quite a range of embedded systems) platform
running on a different endianness.
The Linux driver handles endianness transparently. (LE and BE)
diff --git a/Documentation/firmware_class/README b/Documentation/firmware_class/README
index 71f8685..cafdca8b 100644
--- a/Documentation/firmware_class/README
+++ b/Documentation/firmware_class/README
@@ -20,7 +20,7 @@
1), kernel(driver):
- calls request_firmware(&fw_entry, $FIRMWARE, device)
- - kernel searchs the fimware image with name $FIRMWARE directly
+ - kernel searches the firmware image with name $FIRMWARE directly
in the below search path of root filesystem:
User customized search path by module parameter 'path'[1]
"/lib/firmware/updates/" UTS_RELEASE,
diff --git a/Documentation/gdb-kernel-debugging.txt b/Documentation/gdb-kernel-debugging.txt
index 7050ce8..4ab7d43 100644
--- a/Documentation/gdb-kernel-debugging.txt
+++ b/Documentation/gdb-kernel-debugging.txt
@@ -139,6 +139,27 @@
start_comm = "swapper/2\000\000\000\000\000\000"
}
+ o Dig into a radix tree data structure, such as the IRQ descriptors:
+ (gdb) print (struct irq_desc)$lx_radix_tree_lookup(irq_desc_tree, 18)
+ $6 = {
+ irq_common_data = {
+ state_use_accessors = 67584,
+ handler_data = 0x0 <__vectors_start>,
+ msi_desc = 0x0 <__vectors_start>,
+ affinity = {{
+ bits = {65535}
+ }}
+ },
+ irq_data = {
+ mask = 0,
+ irq = 18,
+ hwirq = 27,
+ common = 0xee803d80,
+ chip = 0xc0eb0854 <gic_data>,
+ domain = 0xee808000,
+ parent_data = 0x0 <__vectors_start>,
+ chip_data = 0xc0eb0854 <gic_data>
+ } <... trimmed ...>
List of commands and functions
------------------------------
diff --git a/Documentation/hwmon/abituguru b/Documentation/hwmon/abituguru
index 915f320..f1d4fe4 100644
--- a/Documentation/hwmon/abituguru
+++ b/Documentation/hwmon/abituguru
@@ -25,7 +25,7 @@
1) For revisions 2 and 3 uGuru's the driver can autodetect the
sensortype (Volt or Temp) for bank1 sensors, for revision 1 uGuru's
this doesnot always work. For these uGuru's the autodection can
- be overriden with the bank1_types module param. For all 3 known
+ be overridden with the bank1_types module param. For all 3 known
revison 1 motherboards the correct use of this param is:
bank1_types=1,1,0,0,0,0,0,2,0,0,0,0,2,0,0,1
You may also need to specify the fan_sensors option for these boards
diff --git a/Documentation/i2c/i2c-topology b/Documentation/i2c/i2c-topology
new file mode 100644
index 0000000..e0aefee
--- /dev/null
+++ b/Documentation/i2c/i2c-topology
@@ -0,0 +1,370 @@
+I2C topology
+============
+
+There are a couple of reasons for building more complex i2c topologies
+than a straight-forward i2c bus with one adapter and one or more devices.
+
+1. A mux may be needed on the bus to prevent address collisions.
+
+2. The bus may be accessible from some external bus master, and arbitration
+ may be needed to determine if it is ok to access the bus.
+
+3. A device (particularly RF tuners) may want to avoid the digital noise
+ from the i2c bus, at least most of the time, and sits behind a gate
+ that has to be operated before the device can be accessed.
+
+Etc
+
+These constructs are represented as i2c adapter trees by Linux, where
+each adapter has a parent adapter (except the root adapter) and zero or
+more child adapters. The root adapter is the actual adapter that issues
+i2c transfers, and all adapters with a parent are part of an "i2c-mux"
+object (quoted, since it can also be an arbitrator or a gate).
+
+Depending of the particular mux driver, something happens when there is
+an i2c transfer on one of its child adapters. The mux driver can
+obviously operate a mux, but it can also do arbitration with an external
+bus master or open a gate. The mux driver has two operations for this,
+select and deselect. select is called before the transfer and (the
+optional) deselect is called after the transfer.
+
+
+Locking
+=======
+
+There are two variants of locking available to i2c muxes, they can be
+mux-locked or parent-locked muxes. As is evident from below, it can be
+useful to know if a mux is mux-locked or if it is parent-locked. The
+following list was correct at the time of writing:
+
+In drivers/i2c/muxes/
+i2c-arb-gpio-challenge Parent-locked
+i2c-mux-gpio Normally parent-locked, mux-locked iff
+ all involved gpio pins are controlled by the
+ same i2c root adapter that they mux.
+i2c-mux-pca9541 Parent-locked
+i2c-mux-pca954x Parent-locked
+i2c-mux-pinctrl Normally parent-locked, mux-locked iff
+ all involved pinctrl devices are controlled
+ by the same i2c root adapter that they mux.
+i2c-mux-reg Parent-locked
+
+In drivers/iio/
+imu/inv_mpu6050/ Mux-locked
+
+In drivers/media/
+dvb-frontends/m88ds3103 Parent-locked
+dvb-frontends/rtl2830 Parent-locked
+dvb-frontends/rtl2832 Mux-locked
+dvb-frontends/si2168 Mux-locked
+usb/cx231xx/ Parent-locked
+
+
+Mux-locked muxes
+----------------
+
+Mux-locked muxes does not lock the entire parent adapter during the
+full select-transfer-deselect transaction, only the muxes on the parent
+adapter are locked. Mux-locked muxes are mostly interesting if the
+select and/or deselect operations must use i2c transfers to complete
+their tasks. Since the parent adapter is not fully locked during the
+full transaction, unrelated i2c transfers may interleave the different
+stages of the transaction. This has the benefit that the mux driver
+may be easier and cleaner to implement, but it has some caveats.
+
+ML1. If you build a topology with a mux-locked mux being the parent
+ of a parent-locked mux, this might break the expectation from the
+ parent-locked mux that the root adapter is locked during the
+ transaction.
+
+ML2. It is not safe to build arbitrary topologies with two (or more)
+ mux-locked muxes that are not siblings, when there are address
+ collisions between the devices on the child adapters of these
+ non-sibling muxes.
+
+ I.e. the select-transfer-deselect transaction targeting e.g. device
+ address 0x42 behind mux-one may be interleaved with a similar
+ operation targeting device address 0x42 behind mux-two. The
+ intension with such a topology would in this hypothetical example
+ be that mux-one and mux-two should not be selected simultaneously,
+ but mux-locked muxes do not guarantee that in all topologies.
+
+ML3. A mux-locked mux cannot be used by a driver for auto-closing
+ gates/muxes, i.e. something that closes automatically after a given
+ number (one, in most cases) of i2c transfers. Unrelated i2c transfers
+ may creep in and close prematurely.
+
+ML4. If any non-i2c operation in the mux driver changes the i2c mux state,
+ the driver has to lock the root adapter during that operation.
+ Otherwise garbage may appear on the bus as seen from devices
+ behind the mux, when an unrelated i2c transfer is in flight during
+ the non-i2c mux-changing operation.
+
+
+Mux-locked Example
+------------------
+
+ .----------. .--------.
+ .--------. | mux- |-----| dev D1 |
+ | root |--+--| locked | '--------'
+ '--------' | | mux M1 |--. .--------.
+ | '----------' '--| dev D2 |
+ | .--------. '--------'
+ '--| dev D3 |
+ '--------'
+
+When there is an access to D1, this happens:
+
+ 1. Someone issues an i2c-transfer to D1.
+ 2. M1 locks muxes on its parent (the root adapter in this case).
+ 3. M1 calls ->select to ready the mux.
+ 4. M1 (presumably) does some i2c-transfers as part of its select.
+ These transfers are normal i2c-transfers that locks the parent
+ adapter.
+ 5. M1 feeds the i2c-transfer from step 1 to its parent adapter as a
+ normal i2c-transfer that locks the parent adapter.
+ 6. M1 calls ->deselect, if it has one.
+ 7. Same rules as in step 4, but for ->deselect.
+ 8. M1 unlocks muxes on its parent.
+
+This means that accesses to D2 are lockout out for the full duration
+of the entire operation. But accesses to D3 are possibly interleaved
+at any point.
+
+
+Parent-locked muxes
+-------------------
+
+Parent-locked muxes lock the parent adapter during the full select-
+transfer-deselect transaction. The implication is that the mux driver
+has to ensure that any and all i2c transfers through that parent
+adapter during the transaction are unlocked i2c transfers (using e.g.
+__i2c_transfer), or a deadlock will follow. There are a couple of
+caveats.
+
+PL1. If you build a topology with a parent-locked mux being the child
+ of another mux, this might break a possible assumption from the
+ child mux that the root adapter is unused between its select op
+ and the actual transfer (e.g. if the child mux is auto-closing
+ and the parent mux issus i2c-transfers as part of its select).
+ This is especially the case if the parent mux is mux-locked, but
+ it may also happen if the parent mux is parent-locked.
+
+PL2. If select/deselect calls out to other subsystems such as gpio,
+ pinctrl, regmap or iio, it is essential that any i2c transfers
+ caused by these subsystems are unlocked. This can be convoluted to
+ accomplish, maybe even impossible if an acceptably clean solution
+ is sought.
+
+
+Parent-locked Example
+---------------------
+
+ .----------. .--------.
+ .--------. | parent- |-----| dev D1 |
+ | root |--+--| locked | '--------'
+ '--------' | | mux M1 |--. .--------.
+ | '----------' '--| dev D2 |
+ | .--------. '--------'
+ '--| dev D3 |
+ '--------'
+
+When there is an access to D1, this happens:
+
+ 1. Someone issues an i2c-transfer to D1.
+ 2. M1 locks muxes on its parent (the root adapter in this case).
+ 3. M1 locks its parent adapter.
+ 4. M1 calls ->select to ready the mux.
+ 5. If M1 does any i2c-transfers (on this root adapter) as part of
+ its select, those transfers must be unlocked i2c-transfers so
+ that they do not deadlock the root adapter.
+ 6. M1 feeds the i2c-transfer from step 1 to the root adapter as an
+ unlocked i2c-transfer, so that it does not deadlock the parent
+ adapter.
+ 7. M1 calls ->deselect, if it has one.
+ 8. Same rules as in step 5, but for ->deselect.
+ 9. M1 unlocks its parent adapter.
+10. M1 unlocks muxes on its parent.
+
+
+This means that accesses to both D2 and D3 are locked out for the full
+duration of the entire operation.
+
+
+Complex Examples
+================
+
+Parent-locked mux as parent of parent-locked mux
+------------------------------------------------
+
+This is a useful topology, but it can be bad.
+
+ .----------. .----------. .--------.
+ .--------. | parent- |-----| parent- |-----| dev D1 |
+ | root |--+--| locked | | locked | '--------'
+ '--------' | | mux M1 |--. | mux M2 |--. .--------.
+ | '----------' | '----------' '--| dev D2 |
+ | .--------. | .--------. '--------'
+ '--| dev D4 | '--| dev D3 |
+ '--------' '--------'
+
+When any device is accessed, all other devices are locked out for
+the full duration of the operation (both muxes lock their parent,
+and specifically when M2 requests its parent to lock, M1 passes
+the buck to the root adapter).
+
+This topology is bad if M2 is an auto-closing mux and M1->select
+issues any unlocked i2c transfers on the root adapter that may leak
+through and be seen by the M2 adapter, thus closing M2 prematurely.
+
+
+Mux-locked mux as parent of mux-locked mux
+------------------------------------------
+
+This is a good topology.
+
+ .----------. .----------. .--------.
+ .--------. | mux- |-----| mux- |-----| dev D1 |
+ | root |--+--| locked | | locked | '--------'
+ '--------' | | mux M1 |--. | mux M2 |--. .--------.
+ | '----------' | '----------' '--| dev D2 |
+ | .--------. | .--------. '--------'
+ '--| dev D4 | '--| dev D3 |
+ '--------' '--------'
+
+When device D1 is accessed, accesses to D2 are locked out for the
+full duration of the operation (muxes on the top child adapter of M1
+are locked). But accesses to D3 and D4 are possibly interleaved at
+any point. Accesses to D3 locks out D1 and D2, but accesses to D4
+are still possibly interleaved.
+
+
+Mux-locked mux as parent of parent-locked mux
+---------------------------------------------
+
+This is probably a bad topology.
+
+ .----------. .----------. .--------.
+ .--------. | mux- |-----| parent- |-----| dev D1 |
+ | root |--+--| locked | | locked | '--------'
+ '--------' | | mux M1 |--. | mux M2 |--. .--------.
+ | '----------' | '----------' '--| dev D2 |
+ | .--------. | .--------. '--------'
+ '--| dev D4 | '--| dev D3 |
+ '--------' '--------'
+
+When device D1 is accessed, accesses to D2 and D3 are locked out
+for the full duration of the operation (M1 locks child muxes on the
+root adapter). But accesses to D4 are possibly interleaved at any
+point.
+
+This kind of topology is generally not suitable and should probably
+be avoided. The reason is that M2 probably assumes that there will
+be no i2c transfers during its calls to ->select and ->deselect, and
+if there are, any such transfers might appear on the slave side of M2
+as partial i2c transfers, i.e. garbage or worse. This might cause
+device lockups and/or other problems.
+
+The topology is especially troublesome if M2 is an auto-closing
+mux. In that case, any interleaved accesses to D4 might close M2
+prematurely, as might any i2c-transfers part of M1->select.
+
+But if M2 is not making the above stated assumption, and if M2 is not
+auto-closing, the topology is fine.
+
+
+Parent-locked mux as parent of mux-locked mux
+---------------------------------------------
+
+This is a good topology.
+
+ .----------. .----------. .--------.
+ .--------. | parent- |-----| mux- |-----| dev D1 |
+ | root |--+--| locked | | locked | '--------'
+ '--------' | | mux M1 |--. | mux M2 |--. .--------.
+ | '----------' | '----------' '--| dev D2 |
+ | .--------. | .--------. '--------'
+ '--| dev D4 | '--| dev D3 |
+ '--------' '--------'
+
+When D1 is accessed, accesses to D2 are locked out for the full
+duration of the operation (muxes on the top child adapter of M1
+are locked). Accesses to D3 and D4 are possibly interleaved at
+any point, just as is expected for mux-locked muxes.
+
+When D3 or D4 are accessed, everything else is locked out. For D3
+accesses, M1 locks the root adapter. For D4 accesses, the root
+adapter is locked directly.
+
+
+Two mux-locked sibling muxes
+----------------------------
+
+This is a good topology.
+
+ .--------.
+ .----------. .--| dev D1 |
+ | mux- |--' '--------'
+ .--| locked | .--------.
+ | | mux M1 |-----| dev D2 |
+ | '----------' '--------'
+ | .----------. .--------.
+ .--------. | | mux- |-----| dev D3 |
+ | root |--+--| locked | '--------'
+ '--------' | | mux M2 |--. .--------.
+ | '----------' '--| dev D4 |
+ | .--------. '--------'
+ '--| dev D5 |
+ '--------'
+
+When D1 is accessed, accesses to D2, D3 and D4 are locked out. But
+accesses to D5 may be interleaved at any time.
+
+
+Two parent-locked sibling muxes
+-------------------------------
+
+This is a good topology.
+
+ .--------.
+ .----------. .--| dev D1 |
+ | parent- |--' '--------'
+ .--| locked | .--------.
+ | | mux M1 |-----| dev D2 |
+ | '----------' '--------'
+ | .----------. .--------.
+ .--------. | | parent- |-----| dev D3 |
+ | root |--+--| locked | '--------'
+ '--------' | | mux M2 |--. .--------.
+ | '----------' '--| dev D4 |
+ | .--------. '--------'
+ '--| dev D5 |
+ '--------'
+
+When any device is accessed, accesses to all other devices are locked
+out.
+
+
+Mux-locked and parent-locked sibling muxes
+------------------------------------------
+
+This is a good topology.
+
+ .--------.
+ .----------. .--| dev D1 |
+ | mux- |--' '--------'
+ .--| locked | .--------.
+ | | mux M1 |-----| dev D2 |
+ | '----------' '--------'
+ | .----------. .--------.
+ .--------. | | parent- |-----| dev D3 |
+ | root |--+--| locked | '--------'
+ '--------' | | mux M2 |--. .--------.
+ | '----------' '--| dev D4 |
+ | .--------. '--------'
+ '--| dev D5 |
+ '--------'
+
+When D1 or D2 are accessed, accesses to D3 and D4 are locked out while
+accesses to D5 may interleave. When D3 or D4 are accessed, accesses to
+all other devices are locked out.
diff --git a/Documentation/infiniband/ipoib.txt b/Documentation/infiniband/ipoib.txt
index f2cfe26..47c1dd9 100644
--- a/Documentation/infiniband/ipoib.txt
+++ b/Documentation/infiniband/ipoib.txt
@@ -25,7 +25,7 @@
main interface for a subinterface is in "parent."
Child interface create/delete can also be done using IPoIB's
- rtnl_link_ops, where childs created using either way behave the same.
+ rtnl_link_ops, where children created using either way behave the same.
Datagram vs Connected modes
diff --git a/Documentation/isa.txt b/Documentation/isa.txt
new file mode 100644
index 0000000..f232c26
--- /dev/null
+++ b/Documentation/isa.txt
@@ -0,0 +1,121 @@
+ISA Drivers
+-----------
+
+The following text is adapted from the commit message of the initial
+commit of the ISA bus driver authored by Rene Herman.
+
+During the recent "isa drivers using platform devices" discussion it was
+pointed out that (ALSA) ISA drivers ran into the problem of not having
+the option to fail driver load (device registration rather) upon not
+finding their hardware due to a probe() error not being passed up
+through the driver model. In the course of that, I suggested a separate
+ISA bus might be best; Russell King agreed and suggested this bus could
+use the .match() method for the actual device discovery.
+
+The attached does this. For this old non (generically) discoverable ISA
+hardware only the driver itself can do discovery so as a difference with
+the platform_bus, this isa_bus also distributes match() up to the
+driver.
+
+As another difference: these devices only exist in the driver model due
+to the driver creating them because it might want to drive them, meaning
+that all device creation has been made internal as well.
+
+The usage model this provides is nice, and has been acked from the ALSA
+side by Takashi Iwai and Jaroslav Kysela. The ALSA driver module_init's
+now (for oldisa-only drivers) become:
+
+static int __init alsa_card_foo_init(void)
+{
+ return isa_register_driver(&snd_foo_isa_driver, SNDRV_CARDS);
+}
+
+static void __exit alsa_card_foo_exit(void)
+{
+ isa_unregister_driver(&snd_foo_isa_driver);
+}
+
+Quite like the other bus models therefore. This removes a lot of
+duplicated init code from the ALSA ISA drivers.
+
+The passed in isa_driver struct is the regular driver struct embedding a
+struct device_driver, the normal probe/remove/shutdown/suspend/resume
+callbacks, and as indicated that .match callback.
+
+The "SNDRV_CARDS" you see being passed in is a "unsigned int ndev"
+parameter, indicating how many devices to create and call our methods
+with.
+
+The platform_driver callbacks are called with a platform_device param;
+the isa_driver callbacks are being called with a "struct device *dev,
+unsigned int id" pair directly -- with the device creation completely
+internal to the bus it's much cleaner to not leak isa_dev's by passing
+them in at all. The id is the only thing we ever want other then the
+struct device * anyways, and it makes for nicer code in the callbacks as
+well.
+
+With this additional .match() callback ISA drivers have all options. If
+ALSA would want to keep the old non-load behaviour, it could stick all
+of the old .probe in .match, which would only keep them registered after
+everything was found to be present and accounted for. If it wanted the
+behaviour of always loading as it inadvertently did for a bit after the
+changeover to platform devices, it could just not provide a .match() and
+do everything in .probe() as before.
+
+If it, as Takashi Iwai already suggested earlier as a way of following
+the model from saner buses more closely, wants to load when a later bind
+could conceivably succeed, it could use .match() for the prerequisites
+(such as checking the user wants the card enabled and that port/irq/dma
+values have been passed in) and .probe() for everything else. This is
+the nicest model.
+
+To the code...
+
+This exports only two functions; isa_{,un}register_driver().
+
+isa_register_driver() register's the struct device_driver, and then
+loops over the passed in ndev creating devices and registering them.
+This causes the bus match method to be called for them, which is:
+
+int isa_bus_match(struct device *dev, struct device_driver *driver)
+{
+ struct isa_driver *isa_driver = to_isa_driver(driver);
+
+ if (dev->platform_data == isa_driver) {
+ if (!isa_driver->match ||
+ isa_driver->match(dev, to_isa_dev(dev)->id))
+ return 1;
+ dev->platform_data = NULL;
+ }
+ return 0;
+}
+
+The first thing this does is check if this device is in fact one of this
+driver's devices by seeing if the device's platform_data pointer is set
+to this driver. Platform devices compare strings, but we don't need to
+do that with everything being internal, so isa_register_driver() abuses
+dev->platform_data as a isa_driver pointer which we can then check here.
+I believe platform_data is available for this, but if rather not, moving
+the isa_driver pointer to the private struct isa_dev is ofcourse fine as
+well.
+
+Then, if the the driver did not provide a .match, it matches. If it did,
+the driver match() method is called to determine a match.
+
+If it did _not_ match, dev->platform_data is reset to indicate this to
+isa_register_driver which can then unregister the device again.
+
+If during all this, there's any error, or no devices matched at all
+everything is backed out again and the error, or -ENODEV, is returned.
+
+isa_unregister_driver() just unregisters the matched devices and the
+driver itself.
+
+module_isa_driver is a helper macro for ISA drivers which do not do
+anything special in module init/exit. This eliminates a lot of
+boilerplate code. Each module may only use this macro once, and calling
+it replaces module_init and module_exit.
+
+max_num_isa_dev is a macro to determine the maximum possible number of
+ISA devices which may be registered in the I/O port address space given
+the address extent of the ISA devices.
diff --git a/Documentation/ja_JP/HOWTO b/Documentation/ja_JP/HOWTO
index 52ef02b..581c14b 100644
--- a/Documentation/ja_JP/HOWTO
+++ b/Documentation/ja_JP/HOWTO
@@ -290,12 +290,6 @@
- このプロセスはカーネルが 「準備ができた」と考えられるまで継続しま
す。このプロセスはだいたい 6週間継続します。
- - 各リリースでの既知の後戻り問題(regression: このリリースの中で新規
- に作り込まれた問題を指す) はその都度 Linux-kernel メーリングリスト
- に投稿されます。ゴールとしては、カーネルが 「準備ができた」と宣言
- する前にこのリストの長さをゼロに減らすことですが、現実には、数個の
- 後戻り問題がリリース時にたびたび残ってしまいます。
-
Andrew Morton が Linux-kernel メーリングリストにカーネルリリースについ
て書いたことをここで言っておくことは価値があります-
「カーネルがいつリリースされるかは誰も知りません。なぜなら、これは現
diff --git a/Documentation/kdump/gdbmacros.txt b/Documentation/kdump/gdbmacros.txt
index 9b9b454..35f6a98 100644
--- a/Documentation/kdump/gdbmacros.txt
+++ b/Documentation/kdump/gdbmacros.txt
@@ -15,15 +15,16 @@
define bttnobp
set $tasks_off=((size_t)&((struct task_struct *)0)->tasks)
- set $pid_off=((size_t)&((struct task_struct *)0)->pids[1].pid_list.next)
+ set $pid_off=((size_t)&((struct task_struct *)0)->thread_group.next)
set $init_t=&init_task
set $next_t=(((char *)($init_t->tasks).next) - $tasks_off)
+ set var $stacksize = sizeof(union thread_union)
while ($next_t != $init_t)
set $next_t=(struct task_struct *)$next_t
printf "\npid %d; comm %s:\n", $next_t.pid, $next_t.comm
printf "===================\n"
- set var $stackp = $next_t.thread.esp
- set var $stack_top = ($stackp & ~4095) + 4096
+ set var $stackp = $next_t.thread.sp
+ set var $stack_top = ($stackp & ~($stacksize - 1)) + $stacksize
while ($stackp < $stack_top)
if (*($stackp) > _stext && *($stackp) < _sinittext)
@@ -31,13 +32,13 @@
end
set $stackp += 4
end
- set $next_th=(((char *)$next_t->pids[1].pid_list.next) - $pid_off)
+ set $next_th=(((char *)$next_t->thread_group.next) - $pid_off)
while ($next_th != $next_t)
set $next_th=(struct task_struct *)$next_th
printf "\npid %d; comm %s:\n", $next_t.pid, $next_t.comm
printf "===================\n"
- set var $stackp = $next_t.thread.esp
- set var $stack_top = ($stackp & ~4095) + 4096
+ set var $stackp = $next_t.thread.sp
+ set var $stack_top = ($stackp & ~($stacksize - 1)) + stacksize
while ($stackp < $stack_top)
if (*($stackp) > _stext && *($stackp) < _sinittext)
@@ -45,7 +46,7 @@
end
set $stackp += 4
end
- set $next_th=(((char *)$next_th->pids[1].pid_list.next) - $pid_off)
+ set $next_th=(((char *)$next_th->thread_group.next) - $pid_off)
end
set $next_t=(char *)($next_t->tasks.next) - $tasks_off
end
@@ -54,42 +55,44 @@
dump all thread stack traces on a kernel compiled with !CONFIG_FRAME_POINTER
end
+define btthreadstack
+ set var $pid_task = $arg0
+
+ printf "\npid %d; comm %s:\n", $pid_task.pid, $pid_task.comm
+ printf "task struct: "
+ print $pid_task
+ printf "===================\n"
+ set var $stackp = $pid_task.thread.sp
+ set var $stacksize = sizeof(union thread_union)
+ set var $stack_top = ($stackp & ~($stacksize - 1)) + $stacksize
+ set var $stack_bot = ($stackp & ~($stacksize - 1))
+
+ set $stackp = *((unsigned long *) $stackp)
+ while (($stackp < $stack_top) && ($stackp > $stack_bot))
+ set var $addr = *(((unsigned long *) $stackp) + 1)
+ info symbol $addr
+ set $stackp = *((unsigned long *) $stackp)
+ end
+end
+document btthreadstack
+ dump a thread stack using the given task structure pointer
+end
+
+
define btt
set $tasks_off=((size_t)&((struct task_struct *)0)->tasks)
- set $pid_off=((size_t)&((struct task_struct *)0)->pids[1].pid_list.next)
+ set $pid_off=((size_t)&((struct task_struct *)0)->thread_group.next)
set $init_t=&init_task
set $next_t=(((char *)($init_t->tasks).next) - $tasks_off)
while ($next_t != $init_t)
set $next_t=(struct task_struct *)$next_t
- printf "\npid %d; comm %s:\n", $next_t.pid, $next_t.comm
- printf "===================\n"
- set var $stackp = $next_t.thread.esp
- set var $stack_top = ($stackp & ~4095) + 4096
- set var $stack_bot = ($stackp & ~4095)
+ btthreadstack $next_t
- set $stackp = *($stackp)
- while (($stackp < $stack_top) && ($stackp > $stack_bot))
- set var $addr = *($stackp + 4)
- info symbol $addr
- set $stackp = *($stackp)
- end
-
- set $next_th=(((char *)$next_t->pids[1].pid_list.next) - $pid_off)
+ set $next_th=(((char *)$next_t->thread_group.next) - $pid_off)
while ($next_th != $next_t)
set $next_th=(struct task_struct *)$next_th
- printf "\npid %d; comm %s:\n", $next_t.pid, $next_t.comm
- printf "===================\n"
- set var $stackp = $next_t.thread.esp
- set var $stack_top = ($stackp & ~4095) + 4096
- set var $stack_bot = ($stackp & ~4095)
-
- set $stackp = *($stackp)
- while (($stackp < $stack_top) && ($stackp > $stack_bot))
- set var $addr = *($stackp + 4)
- info symbol $addr
- set $stackp = *($stackp)
- end
- set $next_th=(((char *)$next_th->pids[1].pid_list.next) - $pid_off)
+ btthreadstack $next_th
+ set $next_th=(((char *)$next_th->thread_group.next) - $pid_off)
end
set $next_t=(char *)($next_t->tasks.next) - $tasks_off
end
@@ -101,7 +104,7 @@
define btpid
set var $pid = $arg0
set $tasks_off=((size_t)&((struct task_struct *)0)->tasks)
- set $pid_off=((size_t)&((struct task_struct *)0)->pids[1].pid_list.next)
+ set $pid_off=((size_t)&((struct task_struct *)0)->thread_group.next)
set $init_t=&init_task
set $next_t=(((char *)($init_t->tasks).next) - $tasks_off)
set var $pid_task = 0
@@ -113,29 +116,18 @@
set $pid_task = $next_t
end
- set $next_th=(((char *)$next_t->pids[1].pid_list.next) - $pid_off)
+ set $next_th=(((char *)$next_t->thread_group.next) - $pid_off)
while ($next_th != $next_t)
set $next_th=(struct task_struct *)$next_th
if ($next_th.pid == $pid)
set $pid_task = $next_th
end
- set $next_th=(((char *)$next_th->pids[1].pid_list.next) - $pid_off)
+ set $next_th=(((char *)$next_th->thread_group.next) - $pid_off)
end
set $next_t=(char *)($next_t->tasks.next) - $tasks_off
end
- printf "\npid %d; comm %s:\n", $pid_task.pid, $pid_task.comm
- printf "===================\n"
- set var $stackp = $pid_task.thread.esp
- set var $stack_top = ($stackp & ~4095) + 4096
- set var $stack_bot = ($stackp & ~4095)
-
- set $stackp = *($stackp)
- while (($stackp < $stack_top) && ($stackp > $stack_bot))
- set var $addr = *($stackp + 4)
- info symbol $addr
- set $stackp = *($stackp)
- end
+ btthreadstack $pid_task
end
document btpid
backtrace of pid
@@ -145,7 +137,7 @@
define trapinfo
set var $pid = $arg0
set $tasks_off=((size_t)&((struct task_struct *)0)->tasks)
- set $pid_off=((size_t)&((struct task_struct *)0)->pids[1].pid_list.next)
+ set $pid_off=((size_t)&((struct task_struct *)0)->thread_group.next)
set $init_t=&init_task
set $next_t=(((char *)($init_t->tasks).next) - $tasks_off)
set var $pid_task = 0
@@ -157,13 +149,13 @@
set $pid_task = $next_t
end
- set $next_th=(((char *)$next_t->pids[1].pid_list.next) - $pid_off)
+ set $next_th=(((char *)$next_t->thread_group.next) - $pid_off)
while ($next_th != $next_t)
set $next_th=(struct task_struct *)$next_th
if ($next_th.pid == $pid)
set $pid_task = $next_th
end
- set $next_th=(((char *)$next_th->pids[1].pid_list.next) - $pid_off)
+ set $next_th=(((char *)$next_th->thread_group.next) - $pid_off)
end
set $next_t=(char *)($next_t->tasks.next) - $tasks_off
end
diff --git a/Documentation/kdump/kdump.txt b/Documentation/kdump/kdump.txt
index bc4bd5a..88ff63d 100644
--- a/Documentation/kdump/kdump.txt
+++ b/Documentation/kdump/kdump.txt
@@ -263,12 +263,6 @@
crashkernel=<range1>:<size1>[,<range2>:<size2>,...][@offset]
range=start-[end]
-Please note, on arm, the offset is required.
- crashkernel=<range1>:<size1>[,<range2>:<size2>,...]@offset
- range=start-[end]
-
- 'start' is inclusive and 'end' is exclusive.
-
For example:
crashkernel=512M-2G:64M,2G-:128M
@@ -307,10 +301,9 @@
on the memory consumption of the kdump system. In general this is not
dependent on the memory size of the production system.
- On arm, use "crashkernel=Y@X". Note that the start address of the kernel
- will be aligned to 128MiB (0x08000000), so if the start address is not then
- any space below the alignment point may be overwritten by the dump-capture kernel,
- which means it is possible that the vmcore is not that precise as expected.
+ On arm, the use of "crashkernel=Y@X" is no longer necessary; the
+ kernel will automatically locate the crash kernel image within the
+ first 512MB of RAM if X is not given.
Load the Dump-capture Kernel
diff --git a/Documentation/kernel-docs.txt b/Documentation/kernel-docs.txt
index fe217c1..1dafc52 100644
--- a/Documentation/kernel-docs.txt
+++ b/Documentation/kernel-docs.txt
@@ -194,15 +194,15 @@
simple---most of the complexity (other than talking to the
hardware) involves managing network packets in memory".
- * Title: "Writing Linux Device Drivers"
+ * Title: "Linux Kernel Hackers' Guide"
Author: Michael K. Johnson.
- URL: http://users.evitech.fi/~tk/rtos/writing_linux_device_d.html
- Keywords: files, VFS, file operations, kernel interface, character
- vs block devices, I/O access, hardware interrupts, DMA, access to
- user memory, memory allocation, timers.
- Description: Introductory 50-minutes (sic) tutorial on writing
- device drivers. 12 pages written by the same author of the "Kernel
- Hackers' Guide" which give a very good overview of the topic.
+ URL: http://www.tldp.org/LDP/khg/HyperNews/get/khg.html
+ Keywords: device drivers, files, VFS, kernel interface, character vs
+ block devices, hardware interrupts, scsi, DMA, access to user memory,
+ memory allocation, timers.
+ Description: A guide designed to help you get up to speed on the
+ concepts that are not intuitevly obvious, and to document the internal
+ structures of Linux.
* Title: "The Venus kernel interface"
Author: Peter J. Braam.
@@ -250,7 +250,7 @@
* Title: "Analysis of the Ext2fs structure"
Author: Louis-Dominique Dubeau.
- URL: http://www.nondot.org/sabre/os/files/FileSystems/ext2fs/
+ URL: http://teaching.csse.uwa.edu.au/units/CITS2002/fs-ext2/
Keywords: ext2, filesystem, ext2fs.
Description: Description of ext2's blocks, directories, inodes,
bitmaps, invariants...
@@ -266,14 +266,14 @@
* Title: "Kernel API changes from 2.0 to 2.2"
Author: Richard Gooch.
- URL:
- http://www.linuxhq.com/guides/LKMPG/node28.html
+ URL: http://www.safe-mbox.com/~rgooch/linux/docs/porting-to-2.2.html
Keywords: 2.2, changes.
Description: Kernel functions/structures/variables which changed
from 2.0.x to 2.2.x.
* Title: "Kernel API changes from 2.2 to 2.4"
Author: Richard Gooch.
+ URL: http://www.safe-mbox.com/~rgooch/linux/docs/porting-to-2.4.html
Keywords: 2.4, changes.
Description: Kernel functions/structures/variables which changed
from 2.2.x to 2.4.x.
@@ -609,6 +609,13 @@
Pages: 432.
ISBN: 0-201-63338-8
+ * Title: "Linux Kernel Development, 3rd Edition"
+ Author: Robert Love
+ Publisher: Addison-Wesley.
+ Date: July, 2010
+ Pages: 440
+ ISBN: 978-0672329463
+
MISCELLANEOUS:
* Name: linux/Documentation
diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt
index 7944031..82b42c9 100644
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -1054,6 +1054,12 @@
the driver will use only 32-bit accessors to read/write
the device registers.
+ meson,<addr>
+ Start an early, polled-mode console on a meson serial
+ port at the specified address. The serial port must
+ already be setup and configured. Options are not yet
+ supported.
+
msm_serial,<addr>
Start an early, polled-mode console on an msm serial
port at the specified address. The serial port
@@ -1787,6 +1793,13 @@
PCI device 00:14.0 write the parameter as:
ivrs_hpet[0]=00:14.0
+ ivrs_acpihid [HW,X86_64]
+ Provide an override to the ACPI-HID:UID<->DEVICE-ID
+ mapping provided in the IVRS ACPI table. For
+ example, to map UART-HID:UID AMD0020:0 to
+ PCI device 00:14.5 write the parameter as:
+ ivrs_acpihid[00:14.5]=AMD0020:0
+
js= [HW,JOY] Analog joystick
See Documentation/input/joystick.txt.
@@ -2161,6 +2174,14 @@
[KNL,SH] Allow user to override the default size for
per-device physically contiguous DMA buffers.
+ memhp_default_state=online/offline
+ [KNL] Set the initial state for the memory hotplug
+ onlining policy. If not specified, the default value is
+ set according to the
+ CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE kernel config
+ option.
+ See Documentation/memory-hotplug.txt.
+
memmap=exactmap [KNL,X86] Enable setting of an exact
E820 memory map, as specified by the user.
Such memmap=exactmap lines can be constructed based on
@@ -2944,11 +2965,6 @@
for broken drivers that don't call it.
skip_isa_align [X86] do not align io start addr, so can
handle more pci cards
- firmware [ARM] Do not re-enumerate the bus but instead
- just use the configuration from the
- bootloader. This is currently used on
- IXP2000 systems where the bus has to be
- configured a certain way for adjunct CPUs.
noearly [X86] Don't do any early type 1 scanning.
This might help on some broken boards which
machine check when some devices' config space
diff --git a/Documentation/ko_KR/HOWTO b/Documentation/ko_KR/HOWTO
index 5a81b39..9a3e659 100644
--- a/Documentation/ko_KR/HOWTO
+++ b/Documentation/ko_KR/HOWTO
@@ -236,9 +236,9 @@
- 새로운 커널이 배포되자마자 2주의 시간이 주어진다. 이 기간동은
메인테이너들은 큰 diff들을 Linus에게 제출할 수 있다. 대개 이 패치들은
몇 주 동안 -next 커널내에 이미 있었던 것들이다. 큰 변경들을 제출하는 데
- 선호되는 방법은 git(커널의 소스 관리 툴, 더 많은 정보들은 http://git.or.cz/
- 에서 참조할 수 있다)를 사용하는 것이지만 순수한 패치파일의 형식으로 보내는
- 것도 무관하다.
+ 선호되는 방법은 git(커널의 소스 관리 툴, 더 많은 정보들은
+ http://git-scm.com/ 에서 참조할 수 있다)를 사용하는 것이지만 순수한
+ 패치파일의 형식으로 보내는 것도 무관하다.
- 2주 후에 -rc1 커널이 배포되며 지금부터는 전체 커널의 안정성에 영향을
미칠수 있는 새로운 기능들을 포함하지 않는 패치들만이 추가될 수 있다.
완전히 새로운 드라이버(혹은 파일시스템)는 -rc1 이후에만 받아들여진다는
@@ -253,8 +253,6 @@
것이다.
- 이러한 프로세스는 커널이 "준비(ready)"되었다고 여겨질때까지 계속된다.
프로세스는 대체로 6주간 지속된다.
- - 각 -rc 배포에 있는 알려진 회귀의 목록들은 다음 URI에 남겨진다.
- http://kernelnewbies.org/known_regressions
커널 배포에 있어서 언급할만한 가치가 있는 리눅스 커널 메일링 리스트의
Andrew Morton의 글이 있다.
diff --git a/Documentation/laptops/toshiba_haps.txt b/Documentation/laptops/toshiba_haps.txt
index 11dbcfd..0c1d88d 100644
--- a/Documentation/laptops/toshiba_haps.txt
+++ b/Documentation/laptops/toshiba_haps.txt
@@ -19,7 +19,7 @@
--------------
This driver provides support for the accelerometer found in various Toshiba
-laptops, being called "Toshiba HDD Protection - Shock Sensor" officialy,
+laptops, being called "Toshiba HDD Protection - Shock Sensor" officially,
and detects laptops automatically with this device.
On Windows, Toshiba provided software monitors this device and provides
automatic HDD protection (head unload) on sudden moves or harsh vibrations,
diff --git a/Documentation/lzo.txt b/Documentation/lzo.txt
index ea45dd3..285c54f 100644
--- a/Documentation/lzo.txt
+++ b/Documentation/lzo.txt
@@ -69,9 +69,9 @@
IMPORTANT NOTE : in the code some length checks are missing because certain
instructions are called under the assumption that a certain number of bytes
- follow because it has already been garanteed before parsing the instructions.
+ follow because it has already been guaranteed before parsing the instructions.
They just have to "refill" this credit if they consume extra bytes. This is
- an implementation design choice independant on the algorithm or encoding.
+ an implementation design choice independent on the algorithm or encoding.
Byte sequences
diff --git a/Documentation/md-cluster.txt b/Documentation/md-cluster.txt
index c100c71..3888327 100644
--- a/Documentation/md-cluster.txt
+++ b/Documentation/md-cluster.txt
@@ -316,3 +316,9 @@
nodes are using the raid which is achieved by lock all bitmap
locks within the cluster, and also those locks are unlocked
accordingly.
+
+7. Unsupported features
+
+There are somethings which are not supported by cluster MD yet.
+
+- update size and change array_sectors.
diff --git a/Documentation/memory-hotplug.txt b/Documentation/memory-hotplug.txt
index 443f4b4..0d7cb95 100644
--- a/Documentation/memory-hotplug.txt
+++ b/Documentation/memory-hotplug.txt
@@ -261,10 +261,11 @@
% cat /sys/devices/system/memory/auto_online_blocks
-The default is "offline" which means the newly added memory is not in a
-ready-to-use state and you have to "online" the newly added memory blocks
-manually. Automatic onlining can be requested by writing "online" to
-"auto_online_blocks" file:
+The default depends on the CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE kernel config
+option. If it is disabled the default is "offline" which means the newly added
+memory is not in a ready-to-use state and you have to "online" the newly added
+memory blocks manually. Automatic onlining can be requested by writing "online"
+to "auto_online_blocks" file:
% echo online > /sys/devices/system/memory/auto_online_blocks
diff --git a/Documentation/mmc/00-INDEX b/Documentation/mmc/00-INDEX
index a9ba672..4623bc0 100644
--- a/Documentation/mmc/00-INDEX
+++ b/Documentation/mmc/00-INDEX
@@ -6,3 +6,5 @@
- info on SD and MMC device partitions
mmc-async-req.txt
- info on mmc asynchronous requests
+mmc-tools.txt
+ - info on mmc-utils tools
diff --git a/Documentation/mmc/mmc-tools.txt b/Documentation/mmc/mmc-tools.txt
new file mode 100644
index 0000000..735509c
--- /dev/null
+++ b/Documentation/mmc/mmc-tools.txt
@@ -0,0 +1,34 @@
+MMC tools introduction
+======================
+
+There is one MMC test tools called mmc-utils, which is maintained by Chris Ball,
+you can find it at the below public git repository:
+http://git.kernel.org/cgit/linux/kernel/git/cjb/mmc-utils.git/
+
+Functions
+=========
+
+The mmc-utils tools can do the following:
+ - Print and parse extcsd data.
+ - Determine the eMMC writeprotect status.
+ - Set the eMMC writeprotect status.
+ - Set the eMMC data sector size to 4KB by disabling emulation.
+ - Create general purpose partition.
+ - Enable the enhanced user area.
+ - Enable write reliability per partition.
+ - Print the response to STATUS_SEND (CMD13).
+ - Enable the boot partition.
+ - Set Boot Bus Conditions.
+ - Enable the eMMC BKOPS feature.
+ - Permanently enable the eMMC H/W Reset feature.
+ - Permanently disable the eMMC H/W Reset feature.
+ - Send Sanitize command.
+ - Program authentication key for the device.
+ - Counter value for the rpmb device will be read to stdout.
+ - Read from rpmb device to output.
+ - Write to rpmb device from data file.
+ - Enable the eMMC cache feature.
+ - Disable the eMMC cache feature.
+ - Print and parse CID data.
+ - Print and parse CSD data.
+ - Print and parse SCR data.
diff --git a/Documentation/networking/can.txt b/Documentation/networking/can.txt
index 6ab619f..d58ff84 100644
--- a/Documentation/networking/can.txt
+++ b/Documentation/networking/can.txt
@@ -1256,7 +1256,7 @@
7. SocketCAN resources
-----------------------
- The Linux CAN / SocketCAN project ressources (project site / mailing list)
+ The Linux CAN / SocketCAN project resources (project site / mailing list)
are referenced in the MAINTAINERS file in the Linux source tree.
Search for CAN NETWORK [LAYERS|DRIVERS].
diff --git a/Documentation/networking/filter.txt b/Documentation/networking/filter.txt
index b9a4edf..683ada5 100644
--- a/Documentation/networking/filter.txt
+++ b/Documentation/networking/filter.txt
@@ -230,7 +230,7 @@
mul 0, 4 A * <x>
div 0, 4 A / <x>
mod 0, 4 A % <x>
- neg 0, 4 !A
+ neg !A
and 0, 4 A & <x>
or 0, 4 A | <x>
xor 0, 4 A ^ <x>
diff --git a/Documentation/powerpc/eeh-pci-error-recovery.txt b/Documentation/powerpc/eeh-pci-error-recovery.txt
index 9d4e33df..6781892 100644
--- a/Documentation/powerpc/eeh-pci-error-recovery.txt
+++ b/Documentation/powerpc/eeh-pci-error-recovery.txt
@@ -12,7 +12,7 @@
The IBM POWER-based pSeries and iSeries computers include PCI bus
controller chips that have extended capabilities for detecting and
reporting a large variety of PCI bus error conditions. These features
-go under the name of "EEH", for "Extended Error Handling". The EEH
+go under the name of "EEH", for "Enhanced Error Handling". The EEH
hardware features allow PCI bus errors to be cleared and a PCI
card to be "rebooted", without also having to reboot the operating
system.
diff --git a/Documentation/pps/pps.txt b/Documentation/pps/pps.txt
index 7cb7264..50022b3 100644
--- a/Documentation/pps/pps.txt
+++ b/Documentation/pps/pps.txt
@@ -98,7 +98,7 @@
};
and then calling the function pps_register_source() in your
-intialization routine as follows:
+initialization routine as follows:
source = pps_register_source(&pps_ktimer_info,
PPS_CAPTUREASSERT | PPS_OFFSETASSERT);
diff --git a/Documentation/robust-futexes.txt b/Documentation/robust-futexes.txt
index af6fce2..61c22d6 100644
--- a/Documentation/robust-futexes.txt
+++ b/Documentation/robust-futexes.txt
@@ -126,9 +126,9 @@
- no VM changes are needed - 'struct address_space' is left alone.
- - no registration of individual locks is needed: robust mutexes dont
+ - no registration of individual locks is needed: robust mutexes don't
need any extra per-lock syscalls. Robust mutexes thus become a very
- lightweight primitive - so they dont force the application designer
+ lightweight primitive - so they don't force the application designer
to do a hard choice between performance and robustness - robust
mutexes are just as fast.
@@ -202,7 +202,7 @@
Testing, architecture support
-----------------------------
-i've tested the new syscalls on x86 and x86_64, and have made sure the
+I've tested the new syscalls on x86 and x86_64, and have made sure the
parsing of the userspace list is robust [ ;-) ] even if the list is
deliberately corrupted.
diff --git a/Documentation/rpmsg.txt b/Documentation/rpmsg.txt
index f7edc3a..a95e36a 100644
--- a/Documentation/rpmsg.txt
+++ b/Documentation/rpmsg.txt
@@ -249,24 +249,12 @@
static struct rpmsg_driver rpmsg_sample_client = {
.drv.name = KBUILD_MODNAME,
- .drv.owner = THIS_MODULE,
.id_table = rpmsg_driver_sample_id_table,
.probe = rpmsg_sample_probe,
.callback = rpmsg_sample_cb,
.remove = rpmsg_sample_remove,
};
-
-static int __init init(void)
-{
- return register_rpmsg_driver(&rpmsg_sample_client);
-}
-module_init(init);
-
-static void __exit fini(void)
-{
- unregister_rpmsg_driver(&rpmsg_sample_client);
-}
-module_exit(fini);
+module_rpmsg_driver(rpmsg_sample_client);
Note: a similar sample which can be built and loaded can be found
in samples/rpmsg/.
diff --git a/Documentation/scsi/ChangeLog.megaraid_sas b/Documentation/scsi/ChangeLog.megaraid_sas
index 18b5709..00ffdf18 100644
--- a/Documentation/scsi/ChangeLog.megaraid_sas
+++ b/Documentation/scsi/ChangeLog.megaraid_sas
@@ -63,7 +63,7 @@
Current Version : 06.506.00.00-rc1
Old Version : 06.504.01.00-rc1
1. Add 4k FastPath DIF support.
- 2. Dont load DevHandle unless FastPath enabled.
+ 2. Don't load DevHandle unless FastPath enabled.
3. Version and Changelog update.
-------------------------------------------------------------------------------
Release Date : Mon. Oct 1, 2012 17:00:00 PST 2012 -
@@ -105,7 +105,7 @@
1. Fix reglockFlags for degraded raid5/6 for MR 9360/9380.
2. Mask off flags in ioctl path to prevent memory scribble with older
MegaCLI versions.
- 3. Remove poll_mode_io module paramater, sysfs node, and associated code.
+ 3. Remove poll_mode_io module parameter, sysfs node, and associated code.
-------------------------------------------------------------------------------
Release Date : Wed. Oct 5, 2011 17:00:00 PST 2010 -
(emaild-id:megaraidlinux@lsi.com)
@@ -199,7 +199,7 @@
1. Add the Online Controller Reset (OCR) to the Driver.
OCR is the new feature for megaraid_sas driver which
will allow the fw to do the chip reset which will not
- affact the OS behavious.
+ affect the OS behavior.
To add the OCR support, driver need to do:
a). reset the controller chips -- Xscale and Gen2 which
@@ -233,7 +233,7 @@
failed state. Driver will kill adapter if can't bring back FW after the
this three times reset.
4. Add the input parameter max_sectors to 1MB support to our GEN2 controller.
- customer can use the input paramenter max_sectors to add 1MB support to GEN2
+ customer can use the input parameter max_sectors to add 1MB support to GEN2
controller.
1 Release Date : Thur. Oct 29, 2009 09:12:45 PST 2009 -
@@ -582,11 +582,11 @@
1 Release Date : Wed Feb 03 14:31:44 PST 2006 - Sumant Patro <Sumant.Patro@lsil.com>
2 Current Version : 00.00.02.04
-3 Older Version : 00.00.02.04
+3 Older Version : 00.00.02.04
-i. Remove superflous instance_lock
+i. Remove superfluous instance_lock
- gets rid of the otherwise superflous instance_lock and avoids an unsave
+ gets rid of the otherwise superfluous instance_lock and avoids an unsafe
unsynchronized access in the error handler.
- Christoph Hellwig <hch@lst.de>
@@ -594,43 +594,43 @@
1 Release Date : Wed Feb 03 14:31:44 PST 2006 - Sumant Patro <Sumant.Patro@lsil.com>
2 Current Version : 00.00.02.04
-3 Older Version : 00.00.02.04
+3 Older Version : 00.00.02.04
i. Support for 1078 type (ppc IOP) controller, device id : 0x60 added.
- During initialization, depending on the device id, the template members
- are initialized with function pointers specific to the ppc or
- xscale controllers.
+ During initialization, depending on the device id, the template members
+ are initialized with function pointers specific to the ppc or
+ xscale controllers.
-Sumant Patro <Sumant.Patro@lsil.com>
-1 Release Date : Fri Feb 03 14:16:25 PST 2006 - Sumant Patro
+1 Release Date : Fri Feb 03 14:16:25 PST 2006 - Sumant Patro
<Sumant.Patro@lsil.com>
2 Current Version : 00.00.02.04
-3 Older Version : 00.00.02.02
-i. Register 16 byte CDB capability with scsi midlayer
+3 Older Version : 00.00.02.02
+i. Register 16 byte CDB capability with scsi midlayer
- "This patch properly registers the 16 byte command length capability of the
- megaraid_sas controlled hardware with the scsi midlayer. All megaraid_sas
+ "This patch properly registers the 16 byte command length capability of the
+ megaraid_sas controlled hardware with the scsi midlayer. All megaraid_sas
hardware supports 16 byte CDB's."
- -Joshua Giles <joshua_giles@dell.com>
+ -Joshua Giles <joshua_giles@dell.com>
1 Release Date : Mon Jan 23 14:09:01 PST 2006 - Sumant Patro <Sumant.Patro@lsil.com>
2 Current Version : 00.00.02.02
-3 Older Version : 00.00.02.01
+3 Older Version : 00.00.02.01
-i. New template defined to represent each family of controllers (identified by processor used).
- The template will have defintions that will be initialised to appropritae values for a specific family of controllers. The template definition has four function pointers. During driver initialisation the function pointers will be set based on the controller family type. This change is done to support new controllers that has different processors and thus different register set.
+i. New template defined to represent each family of controllers (identified by processor used).
+ The template will have definitions that will be initialised to appropriate values for a specific family of controllers. The template definition has four function pointers. During driver initialisation the function pointers will be set based on the controller family type. This change is done to support new controllers that has different processors and thus different register set.
-Sumant Patro <Sumant.Patro@lsil.com>
1 Release Date : Mon Dec 19 14:36:26 PST 2005 - Sumant Patro <Sumant.Patro@lsil.com>
-2 Current Version : 00.00.02.00-rc4
-3 Older Version : 00.00.02.01
+2 Current Version : 00.00.02.00-rc4
+3 Older Version : 00.00.02.01
-i. Code reorganized to remove code duplication in megasas_build_cmd.
+i. Code reorganized to remove code duplication in megasas_build_cmd.
- "There's a lot of duplicate code megasas_build_cmd. Move that out of the different codepathes and merge the reminder of megasas_build_cmd into megasas_queue_command"
+ "There's a lot of duplicate code megasas_build_cmd. Move that out of the different codepaths and merge the reminder of megasas_build_cmd into megasas_queue_command"
- Christoph Hellwig <hch@lst.de>
diff --git a/Documentation/scsi/bfa.txt b/Documentation/scsi/bfa.txt
index f2d6e9d..3cc4d80 100644
--- a/Documentation/scsi/bfa.txt
+++ b/Documentation/scsi/bfa.txt
@@ -50,7 +50,7 @@
http://www.brocade.com/services-support/drivers-downloads/adapters/Linux.page
-and then click following respective util pacakge link
+and then click following respective util package link
Version Link
diff --git a/Documentation/scsi/g_NCR5380.txt b/Documentation/scsi/g_NCR5380.txt
index 3b80f56..fd88015 100644
--- a/Documentation/scsi/g_NCR5380.txt
+++ b/Documentation/scsi/g_NCR5380.txt
@@ -23,11 +23,10 @@
If the default configuration does not work for you, you can use the kernel
command lines (eg using the lilo append command):
- ncr5380=port,irq,dma
- ncr53c400=port,irq
-or
- ncr5380=base,irq,dma
- ncr53c400=base,irq
+ ncr5380=addr,irq
+ ncr53c400=addr,irq
+ ncr53c400a=addr,irq
+ dtc3181e=addr,irq
The driver does not probe for any addresses or ports other than those in
the OVERRIDE or given to the kernel as above.
@@ -36,19 +35,17 @@
/proc/scsi/g_NCR5380/x where x is the scsi card number as detected at boot
time. More info to come in the future.
-When NCR53c400 support is compiled in, BIOS parameters will be returned by
-the driver (the raw 5380 driver does not and I don't plan to fiddle with
-it!).
-
This driver works as a module.
When included as a module, parameters can be passed on the insmod/modprobe
command line:
ncr_irq=xx the interrupt
ncr_addr=xx the port or base address (for port or memory
mapped, resp.)
- ncr_dma=xx the DMA
ncr_5380=1 to set up for a NCR5380 board
ncr_53c400=1 to set up for a NCR53C400 board
+ ncr_53c400a=1 to set up for a NCR53C400A board
+ dtc_3181e=1 to set up for a Domex Technology Corp 3181E board
+ hp_c2502=1 to set up for a Hewlett Packard C2502 board
e.g.
modprobe g_NCR5380 ncr_irq=5 ncr_addr=0x350 ncr_5380=1
for a port mapped NCR5380 board or
diff --git a/Documentation/scsi/scsi-parameters.txt b/Documentation/scsi/scsi-parameters.txt
index 2bfd6f6..1241ac1 100644
--- a/Documentation/scsi/scsi-parameters.txt
+++ b/Documentation/scsi/scsi-parameters.txt
@@ -27,13 +27,15 @@
aic79xx= [HW,SCSI]
See Documentation/scsi/aic79xx.txt.
- atascsi= [HW,SCSI] Atari SCSI
+ atascsi= [HW,SCSI]
+ See drivers/scsi/atari_scsi.c.
BusLogic= [HW,SCSI]
See drivers/scsi/BusLogic.c, comment before function
BusLogic_ParseDriverOptions().
dtc3181e= [HW,SCSI]
+ See Documentation/scsi/g_NCR5380.txt.
eata= [HW,SCSI]
@@ -51,8 +53,8 @@
ips= [HW,SCSI] Adaptec / IBM ServeRAID controller
See header of drivers/scsi/ips.c.
- mac5380= [HW,SCSI] Format:
- <can_queue>,<cmd_per_lun>,<sg_tablesize>,<hostid>,<use_tags>
+ mac5380= [HW,SCSI]
+ See drivers/scsi/mac_scsi.c.
max_luns= [SCSI] Maximum number of LUNs to probe.
Should be between 1 and 2^32-1.
@@ -65,10 +67,13 @@
See header of drivers/scsi/NCR_D700.c.
ncr5380= [HW,SCSI]
+ See Documentation/scsi/g_NCR5380.txt.
ncr53c400= [HW,SCSI]
+ See Documentation/scsi/g_NCR5380.txt.
ncr53c400a= [HW,SCSI]
+ See Documentation/scsi/g_NCR5380.txt.
ncr53c406a= [HW,SCSI]
diff --git a/Documentation/security/LoadPin.txt b/Documentation/security/LoadPin.txt
new file mode 100644
index 0000000..e11877f
--- /dev/null
+++ b/Documentation/security/LoadPin.txt
@@ -0,0 +1,17 @@
+LoadPin is a Linux Security Module that ensures all kernel-loaded files
+(modules, firmware, etc) all originate from the same filesystem, with
+the expectation that such a filesystem is backed by a read-only device
+such as dm-verity or CDROM. This allows systems that have a verified
+and/or unchangeable filesystem to enforce module and firmware loading
+restrictions without needing to sign the files individually.
+
+The LSM is selectable at build-time with CONFIG_SECURITY_LOADPIN, and
+can be controlled at boot-time with the kernel command line option
+"loadpin.enabled". By default, it is enabled, but can be disabled at
+boot ("loadpin.enabled=0").
+
+LoadPin starts pinning when it sees the first file loaded. If the
+block device backing the filesystem is not read-only, a sysctl is
+created to toggle pinning: /proc/sys/kernel/loadpin/enabled. (Having
+a mutable filesystem means pinning is mutable too, but having the
+sysctl allows for easy testing on systems with a mutable filesystem.)
diff --git a/Documentation/security/keys.txt b/Documentation/security/keys.txt
index 8c18387..20d0571 100644
--- a/Documentation/security/keys.txt
+++ b/Documentation/security/keys.txt
@@ -823,6 +823,36 @@
A process must have search permission on the key for this function to be
successful.
+ (*) Compute a Diffie-Hellman shared secret or public key
+
+ long keyctl(KEYCTL_DH_COMPUTE, struct keyctl_dh_params *params,
+ char *buffer, size_t buflen);
+
+ The params struct contains serial numbers for three keys:
+
+ - The prime, p, known to both parties
+ - The local private key
+ - The base integer, which is either a shared generator or the
+ remote public key
+
+ The value computed is:
+
+ result = base ^ private (mod prime)
+
+ If the base is the shared generator, the result is the local
+ public key. If the base is the remote public key, the result is
+ the shared secret.
+
+ The buffer length must be at least the length of the prime, or zero.
+
+ If the buffer length is nonzero, the length of the result is
+ returned when it is successfully calculated and copied in to the
+ buffer. When the buffer length is zero, the minimum required
+ buffer length is returned.
+
+ This function will return error EOPNOTSUPP if the key type is not
+ supported, error ENOKEY if the key could not be found, or error
+ EACCES if the key is not readable by the caller.
===============
KERNEL SERVICES
@@ -999,6 +1029,10 @@
struct key *keyring_alloc(const char *description, uid_t uid, gid_t gid,
const struct cred *cred,
key_perm_t perm,
+ int (*restrict_link)(struct key *,
+ const struct key_type *,
+ unsigned long,
+ const union key_payload *),
unsigned long flags,
struct key *dest);
@@ -1010,6 +1044,24 @@
KEY_ALLOC_NOT_IN_QUOTA in flags if the keyring shouldn't be accounted
towards the user's quota). Error ENOMEM can also be returned.
+ If restrict_link not NULL, it should point to a function that will be
+ called each time an attempt is made to link a key into the new keyring.
+ This function is called to check whether a key may be added into the keying
+ or not. Callers of key_create_or_update() within the kernel can pass
+ KEY_ALLOC_BYPASS_RESTRICTION to suppress the check. An example of using
+ this is to manage rings of cryptographic keys that are set up when the
+ kernel boots where userspace is also permitted to add keys - provided they
+ can be verified by a key the kernel already has.
+
+ When called, the restriction function will be passed the keyring being
+ added to, the key flags value and the type and payload of the key being
+ added. Note that when a new key is being created, this is called between
+ payload preparsing and actual key creation. The function should return 0
+ to allow the link or an error to reject it.
+
+ A convenience function, restrict_link_reject, exists to always return
+ -EPERM to in this case.
+
(*) To check the validity of a key, this function can be called:
diff --git a/Documentation/security/self-protection.txt b/Documentation/security/self-protection.txt
new file mode 100644
index 0000000..babd637
--- /dev/null
+++ b/Documentation/security/self-protection.txt
@@ -0,0 +1,261 @@
+# Kernel Self-Protection
+
+Kernel self-protection is the design and implementation of systems and
+structures within the Linux kernel to protect against security flaws in
+the kernel itself. This covers a wide range of issues, including removing
+entire classes of bugs, blocking security flaw exploitation methods,
+and actively detecting attack attempts. Not all topics are explored in
+this document, but it should serve as a reasonable starting point and
+answer any frequently asked questions. (Patches welcome, of course!)
+
+In the worst-case scenario, we assume an unprivileged local attacker
+has arbitrary read and write access to the kernel's memory. In many
+cases, bugs being exploited will not provide this level of access,
+but with systems in place that defend against the worst case we'll
+cover the more limited cases as well. A higher bar, and one that should
+still be kept in mind, is protecting the kernel against a _privileged_
+local attacker, since the root user has access to a vastly increased
+attack surface. (Especially when they have the ability to load arbitrary
+kernel modules.)
+
+The goals for successful self-protection systems would be that they
+are effective, on by default, require no opt-in by developers, have no
+performance impact, do not impede kernel debugging, and have tests. It
+is uncommon that all these goals can be met, but it is worth explicitly
+mentioning them, since these aspects need to be explored, dealt with,
+and/or accepted.
+
+
+## Attack Surface Reduction
+
+The most fundamental defense against security exploits is to reduce the
+areas of the kernel that can be used to redirect execution. This ranges
+from limiting the exposed APIs available to userspace, making in-kernel
+APIs hard to use incorrectly, minimizing the areas of writable kernel
+memory, etc.
+
+### Strict kernel memory permissions
+
+When all of kernel memory is writable, it becomes trivial for attacks
+to redirect execution flow. To reduce the availability of these targets
+the kernel needs to protect its memory with a tight set of permissions.
+
+#### Executable code and read-only data must not be writable
+
+Any areas of the kernel with executable memory must not be writable.
+While this obviously includes the kernel text itself, we must consider
+all additional places too: kernel modules, JIT memory, etc. (There are
+temporary exceptions to this rule to support things like instruction
+alternatives, breakpoints, kprobes, etc. If these must exist in a
+kernel, they are implemented in a way where the memory is temporarily
+made writable during the update, and then returned to the original
+permissions.)
+
+In support of this are (the poorly named) CONFIG_DEBUG_RODATA and
+CONFIG_DEBUG_SET_MODULE_RONX, which seek to make sure that code is not
+writable, data is not executable, and read-only data is neither writable
+nor executable.
+
+#### Function pointers and sensitive variables must not be writable
+
+Vast areas of kernel memory contain function pointers that are looked
+up by the kernel and used to continue execution (e.g. descriptor/vector
+tables, file/network/etc operation structures, etc). The number of these
+variables must be reduced to an absolute minimum.
+
+Many such variables can be made read-only by setting them "const"
+so that they live in the .rodata section instead of the .data section
+of the kernel, gaining the protection of the kernel's strict memory
+permissions as described above.
+
+For variables that are initialized once at __init time, these can
+be marked with the (new and under development) __ro_after_init
+attribute.
+
+What remains are variables that are updated rarely (e.g. GDT). These
+will need another infrastructure (similar to the temporary exceptions
+made to kernel code mentioned above) that allow them to spend the rest
+of their lifetime read-only. (For example, when being updated, only the
+CPU thread performing the update would be given uninterruptible write
+access to the memory.)
+
+#### Segregation of kernel memory from userspace memory
+
+The kernel must never execute userspace memory. The kernel must also never
+access userspace memory without explicit expectation to do so. These
+rules can be enforced either by support of hardware-based restrictions
+(x86's SMEP/SMAP, ARM's PXN/PAN) or via emulation (ARM's Memory Domains).
+By blocking userspace memory in this way, execution and data parsing
+cannot be passed to trivially-controlled userspace memory, forcing
+attacks to operate entirely in kernel memory.
+
+### Reduced access to syscalls
+
+One trivial way to eliminate many syscalls for 64-bit systems is building
+without CONFIG_COMPAT. However, this is rarely a feasible scenario.
+
+The "seccomp" system provides an opt-in feature made available to
+userspace, which provides a way to reduce the number of kernel entry
+points available to a running process. This limits the breadth of kernel
+code that can be reached, possibly reducing the availability of a given
+bug to an attack.
+
+An area of improvement would be creating viable ways to keep access to
+things like compat, user namespaces, BPF creation, and perf limited only
+to trusted processes. This would keep the scope of kernel entry points
+restricted to the more regular set of normally available to unprivileged
+userspace.
+
+### Restricting access to kernel modules
+
+The kernel should never allow an unprivileged user the ability to
+load specific kernel modules, since that would provide a facility to
+unexpectedly extend the available attack surface. (The on-demand loading
+of modules via their predefined subsystems, e.g. MODULE_ALIAS_*, is
+considered "expected" here, though additional consideration should be
+given even to these.) For example, loading a filesystem module via an
+unprivileged socket API is nonsense: only the root or physically local
+user should trigger filesystem module loading. (And even this can be up
+for debate in some scenarios.)
+
+To protect against even privileged users, systems may need to either
+disable module loading entirely (e.g. monolithic kernel builds or
+modules_disabled sysctl), or provide signed modules (e.g.
+CONFIG_MODULE_SIG_FORCE, or dm-crypt with LoadPin), to keep from having
+root load arbitrary kernel code via the module loader interface.
+
+
+## Memory integrity
+
+There are many memory structures in the kernel that are regularly abused
+to gain execution control during an attack, By far the most commonly
+understood is that of the stack buffer overflow in which the return
+address stored on the stack is overwritten. Many other examples of this
+kind of attack exist, and protections exist to defend against them.
+
+### Stack buffer overflow
+
+The classic stack buffer overflow involves writing past the expected end
+of a variable stored on the stack, ultimately writing a controlled value
+to the stack frame's stored return address. The most widely used defense
+is the presence of a stack canary between the stack variables and the
+return address (CONFIG_CC_STACKPROTECTOR), which is verified just before
+the function returns. Other defenses include things like shadow stacks.
+
+### Stack depth overflow
+
+A less well understood attack is using a bug that triggers the
+kernel to consume stack memory with deep function calls or large stack
+allocations. With this attack it is possible to write beyond the end of
+the kernel's preallocated stack space and into sensitive structures. Two
+important changes need to be made for better protections: moving the
+sensitive thread_info structure elsewhere, and adding a faulting memory
+hole at the bottom of the stack to catch these overflows.
+
+### Heap memory integrity
+
+The structures used to track heap free lists can be sanity-checked during
+allocation and freeing to make sure they aren't being used to manipulate
+other memory areas.
+
+### Counter integrity
+
+Many places in the kernel use atomic counters to track object references
+or perform similar lifetime management. When these counters can be made
+to wrap (over or under) this traditionally exposes a use-after-free
+flaw. By trapping atomic wrapping, this class of bug vanishes.
+
+### Size calculation overflow detection
+
+Similar to counter overflow, integer overflows (usually size calculations)
+need to be detected at runtime to kill this class of bug, which
+traditionally leads to being able to write past the end of kernel buffers.
+
+
+## Statistical defenses
+
+While many protections can be considered deterministic (e.g. read-only
+memory cannot be written to), some protections provide only statistical
+defense, in that an attack must gather enough information about a
+running system to overcome the defense. While not perfect, these do
+provide meaningful defenses.
+
+### Canaries, blinding, and other secrets
+
+It should be noted that things like the stack canary discussed earlier
+are technically statistical defenses, since they rely on a (leakable)
+secret value.
+
+Blinding literal values for things like JITs, where the executable
+contents may be partially under the control of userspace, need a similar
+secret value.
+
+It is critical that the secret values used must be separate (e.g.
+different canary per stack) and high entropy (e.g. is the RNG actually
+working?) in order to maximize their success.
+
+### Kernel Address Space Layout Randomization (KASLR)
+
+Since the location of kernel memory is almost always instrumental in
+mounting a successful attack, making the location non-deterministic
+raises the difficulty of an exploit. (Note that this in turn makes
+the value of leaks higher, since they may be used to discover desired
+memory locations.)
+
+#### Text and module base
+
+By relocating the physical and virtual base address of the kernel at
+boot-time (CONFIG_RANDOMIZE_BASE), attacks needing kernel code will be
+frustrated. Additionally, offsetting the module loading base address
+means that even systems that load the same set of modules in the same
+order every boot will not share a common base address with the rest of
+the kernel text.
+
+#### Stack base
+
+If the base address of the kernel stack is not the same between processes,
+or even not the same between syscalls, targets on or beyond the stack
+become more difficult to locate.
+
+#### Dynamic memory base
+
+Much of the kernel's dynamic memory (e.g. kmalloc, vmalloc, etc) ends up
+being relatively deterministic in layout due to the order of early-boot
+initializations. If the base address of these areas is not the same
+between boots, targeting them is frustrated, requiring a leak specific
+to the region.
+
+
+## Preventing Leaks
+
+Since the locations of sensitive structures are the primary target for
+attacks, it is important to defend against leaks of both kernel memory
+addresses and kernel memory contents (since they may contain kernel
+addresses or other sensitive things like canary values).
+
+### Unique identifiers
+
+Kernel memory addresses must never be used as identifiers exposed to
+userspace. Instead, use an atomic counter, an idr, or similar unique
+identifier.
+
+### Memory initialization
+
+Memory copied to userspace must always be fully initialized. If not
+explicitly memset(), this will require changes to the compiler to make
+sure structure holes are cleared.
+
+### Memory poisoning
+
+When releasing memory, it is best to poison the contents (clear stack on
+syscall return, wipe heap memory on a free), to avoid reuse attacks that
+rely on the old contents of memory. This frustrates many uninitialized
+variable attacks, stack info leaks, heap info leaks, and use-after-free
+attacks.
+
+### Destination tracking
+
+To help kill classes of bugs that result in kernel addresses being
+written to userspace, the destination of writes needs to be tracked. If
+the buffer is destined for userspace (e.g. seq_file backed /proc files),
+it should automatically censor sensitive values.
diff --git a/Documentation/serial/driver b/Documentation/serial/driver
index 379468e..da193e0 100644
--- a/Documentation/serial/driver
+++ b/Documentation/serial/driver
@@ -28,7 +28,7 @@
the correct port structure (via uart_get_console) and decoding command line
arguments (uart_parse_options).
-There is also a helper function (uart_write_console) which performs a
+There is also a helper function (uart_console_write) which performs a
character by character write, translating newlines to CRLF sequences.
Driver writers are recommended to use this function rather than implementing
their own version.
@@ -41,27 +41,23 @@
necessary locking using port->lock. There are some exceptions (which
are described in the uart_ops listing below.)
-There are three locks. A per-port spinlock, a per-port tmpbuf semaphore,
-and an overall semaphore.
+There are two locks. A per-port spinlock, and an overall semaphore.
From the core driver perspective, the port->lock locks the following
data:
port->mctrl
port->icount
- info->xmit.head (circ->head)
- info->xmit.tail (circ->tail)
+ port->state->xmit.head (circ_buf->head)
+ port->state->xmit.tail (circ_buf->tail)
The low level driver is free to use this lock to provide any additional
locking.
-The core driver uses the info->tmpbuf_sem lock to prevent multi-threaded
-access to the info->tmpbuf bouncebuffer used for port writes.
-
The port_sem semaphore is used to protect against ports being added/
removed or reconfigured at inappropriate times. Since v2.6.27, this
semaphore has been the 'mutex' member of the tty_port struct, and
-commonly referred to as the port mutex (or port->mutex).
+commonly referred to as the port mutex.
uart_ops
@@ -135,6 +131,24 @@
Interrupts: locally disabled.
This call must not sleep
+ throttle(port)
+ Notify the serial driver that input buffers for the line discipline are
+ close to full, and it should somehow signal that no more characters
+ should be sent to the serial port.
+ This will be called only if hardware assisted flow control is enabled.
+
+ Locking: serialized with .unthrottle() and termios modification by the
+ tty layer.
+
+ unthrottle(port)
+ Notify the serial driver that characters can now be sent to the serial
+ port without fear of overrunning the input buffers of the line
+ disciplines.
+ This will be called only if hardware assisted flow control is enabled.
+
+ Locking: serialized with .throttle() and termios modification by the
+ tty layer.
+
send_xchar(port,ch)
Transmit a high priority character, even if the port is stopped.
This is used to implement XON/XOFF flow control and tcflow(). If
@@ -172,9 +186,7 @@
should be terminated when another call is made with a zero
ctl.
- Locking: none.
- Interrupts: caller dependent.
- This call must not sleep
+ Locking: caller holds tty_port->mutex
startup(port)
Grab any interrupt resources and initialise any low level driver
@@ -192,7 +204,7 @@
RTS nor DTR; this will have already been done via a separate
call to set_mctrl.
- Drivers must not access port->info once this call has completed.
+ Drivers must not access port->state once this call has completed.
This method will only be called when there are no more users of
this port.
@@ -204,7 +216,7 @@
Flush any write buffers, reset any DMA state and stop any
ongoing DMA transfers.
- This will be called whenever the port->info->xmit circular
+ This will be called whenever the port->state->xmit circular
buffer is cleared.
Locking: port->lock taken.
@@ -250,10 +262,15 @@
Other flags may be used (eg, xon/xoff characters) if your
hardware supports hardware "soft" flow control.
- Locking: caller holds port->mutex
+ Locking: caller holds tty_port->mutex
Interrupts: caller dependent.
This call must not sleep
+ set_ldisc(port,termios)
+ Notifier for discipline change. See Documentation/serial/tty.txt.
+
+ Locking: caller holds tty_port->mutex
+
pm(port,state,oldstate)
Perform any power management related activities on the specified
port. State indicates the new state (defined by
@@ -371,7 +388,7 @@
Interrupts: n/a
uart_get_divisor(port,baud)
- Return the divsor (baud_base / baud) for the specified baud
+ Return the divisor (baud_base / baud) for the specified baud
rate, appropriately rounded.
If 38400 baud and custom divisor is selected, return the
@@ -449,11 +466,12 @@
mctrl_gpio_free(dev, gpios):
This will free the requested gpios in mctrl_gpio_init().
- As devm_* function are used, there's generally no need to call
+ As devm_* functions are used, there's generally no need to call
this function.
mctrl_gpio_to_gpiod(gpios, gidx)
- This returns the gpio structure associated to the modem line index.
+ This returns the gpio_desc structure associated to the modem line
+ index.
mctrl_gpio_set(gpios, mctrl):
This will sets the gpios according to the mctrl state.
diff --git a/Documentation/serial/tty.txt b/Documentation/serial/tty.txt
index 798cba8..b487809 100644
--- a/Documentation/serial/tty.txt
+++ b/Documentation/serial/tty.txt
@@ -210,9 +210,6 @@
TTY_OTHER_CLOSED Device is a pty and the other side has closed.
-TTY_OTHER_DONE Device is a pty and the other side has closed and
- all pending input processing has been completed.
-
TTY_NO_WRITE_SPLIT Prevent driver from splitting up writes into
smaller chunks.
diff --git a/Documentation/sound/alsa/HD-Audio.txt b/Documentation/sound/alsa/HD-Audio.txt
index e7193aa..d4510eb 100644
--- a/Documentation/sound/alsa/HD-Audio.txt
+++ b/Documentation/sound/alsa/HD-Audio.txt
@@ -655,17 +655,6 @@
and next kernels are found in for-linus and for-next branches,
respectively.
-If you are using the latest Linus tree, it'd be better to pull the
-above GIT tree onto it. If you are using the older kernels, an easy
-way to try the latest ALSA code is to build from the snapshot
-tarball. There are daily tarballs and the latest snapshot tarball.
-All can be built just like normal alsa-driver release packages, that
-is, installed via the usual spells: configure, make and make
-install(-modules). See INSTALL in the package. The snapshot tarballs
-are found at:
-
-- ftp://ftp.suse.com/pub/people/tiwai/snapshot/
-
Sending a Bug Report
~~~~~~~~~~~~~~~~~~~~
@@ -699,7 +688,12 @@
alsa-info
~~~~~~~~~
The script `alsa-info.sh` is a very useful tool to gather the audio
-device information. You can fetch the latest version from:
+device information. It's included in alsa-utils package. The latest
+version can be found on git repository:
+
+- git://git.alsa-project.org/alsa-utils.git
+
+The script can be fetched directly from the following URL, too:
- http://www.alsa-project.org/alsa-info.sh
@@ -836,15 +830,11 @@
(mixer) elements, set/get the control element value, simulate the PCM
operation, the jack plugging simulation, etc.
-The package is found in:
-
-- ftp://ftp.suse.com/pub/people/tiwai/misc/
-
-A git repository is available:
+The program is found in the git repository below:
- git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/hda-emu.git
-See README file in the tarball for more details about hda-emu
+See README file in the repository for more details about hda-emu
program.
diff --git a/Documentation/sound/alsa/compress_offload.txt b/Documentation/sound/alsa/compress_offload.txt
index 630c492..8ba556a 100644
--- a/Documentation/sound/alsa/compress_offload.txt
+++ b/Documentation/sound/alsa/compress_offload.txt
@@ -149,7 +149,7 @@
================
When playing thru an album, the decoders have the ability to skip the encoder
delay and padding and directly move from one track content to another. The end
-user can perceive this as gapless playback as we dont have silence while
+user can perceive this as gapless playback as we don't have silence while
switching from one track to another
Also, there might be low-intensity noises due to encoding. Perfect gapless is
@@ -184,7 +184,7 @@
- Fill data of the first track
- Trigger start
- User-space finished sending all,
-- Indicaite next track data by sending set_next_track
+- Indicate next track data by sending set_next_track
- Set metadata of the next track
- then call partial_drain to flush most of buffer in DSP
- Fill data of the next track
diff --git a/Documentation/sound/alsa/soc/dapm.txt b/Documentation/sound/alsa/soc/dapm.txt
index 6faab48..c45bd79 100644
--- a/Documentation/sound/alsa/soc/dapm.txt
+++ b/Documentation/sound/alsa/soc/dapm.txt
@@ -132,7 +132,7 @@
SND_SOC_DAPM_MIXER("Output Mixer", WM8731_PWR, 4, 1, wm8731_output_mixer_controls,
ARRAY_SIZE(wm8731_output_mixer_controls)),
-If you dont want the mixer elements prefixed with the name of the mixer widget,
+If you don't want the mixer elements prefixed with the name of the mixer widget,
you can use SND_SOC_DAPM_MIXER_NAMED_CTL instead. the parameters are the same
as for SND_SOC_DAPM_MIXER.
diff --git a/Documentation/sound/alsa/soc/overview.txt b/Documentation/sound/alsa/soc/overview.txt
index ff88f52..f3f28b7 100644
--- a/Documentation/sound/alsa/soc/overview.txt
+++ b/Documentation/sound/alsa/soc/overview.txt
@@ -63,7 +63,7 @@
and any audio DSP drivers for that platform.
* Machine class driver: The machine driver class acts as the glue that
- decribes and binds the other component drivers together to form an ALSA
+ describes and binds the other component drivers together to form an ALSA
"sound card device". It handles any machine specific controls and
machine level audio events (e.g. turning on an amp at start of playback).
diff --git a/Documentation/sound/alsa/timestamping.txt b/Documentation/sound/alsa/timestamping.txt
index 0b191a2..1b6473f 100644
--- a/Documentation/sound/alsa/timestamping.txt
+++ b/Documentation/sound/alsa/timestamping.txt
@@ -129,7 +129,7 @@
interpolation of the results
In some hardware-specific configuration, the system timestamp is
-latched by a low-level audio subsytem, and the information provided
+latched by a low-level audio subsystem, and the information provided
back to the driver. Due to potential delays in the communication with
the hardware, there is a risk of misalignment with the avail and delay
information. To make sure applications are not confused, a
diff --git a/Documentation/sync_file.txt b/Documentation/sync_file.txt
new file mode 100644
index 0000000..eaf8297
--- /dev/null
+++ b/Documentation/sync_file.txt
@@ -0,0 +1,69 @@
+ Sync File API Guide
+ ~~~~~~~~~~~~~~~~~~~
+
+ Gustavo Padovan
+ <gustavo at padovan dot org>
+
+This document serves as a guide for device drivers writers on what the
+sync_file API is, and how drivers can support it. Sync file is the carrier of
+the fences(struct fence) that needs to synchronized between drivers or across
+process boundaries.
+
+The sync_file API is meant to be used to send and receive fence information
+to/from userspace. It enables userspace to do explicit fencing, where instead
+of attaching a fence to the buffer a producer driver (such as a GPU or V4L
+driver) sends the fence related to the buffer to userspace via a sync_file.
+
+The sync_file then can be sent to the consumer (DRM driver for example), that
+will not use the buffer for anything before the fence(s) signals, i.e., the
+driver that issued the fence is not using/processing the buffer anymore, so it
+signals that the buffer is ready to use. And vice-versa for the consumer ->
+producer part of the cycle.
+
+Sync files allows userspace awareness on buffer sharing synchronization between
+drivers.
+
+Sync file was originally added in the Android kernel but current Linux Desktop
+can benefit a lot from it.
+
+in-fences and out-fences
+------------------------
+
+Sync files can go either to or from userspace. When a sync_file is sent from
+the driver to userspace we call the fences it contains 'out-fences'. They are
+related to a buffer that the driver is processing or is going to process, so
+the driver an create out-fence to be able to notify, through fence_signal(),
+when it has finished using (or processing) that buffer. Out-fences are fences
+that the driver creates.
+
+On the other hand if the driver receives fence(s) through a sync_file from
+userspace we call these fence(s) 'in-fences'. Receiveing in-fences means that
+we need to wait for the fence(s) to signal before using any buffer related to
+the in-fences.
+
+Creating Sync Files
+-------------------
+
+When a driver needs to send an out-fence userspace it creates a sync_file.
+
+Interface:
+ struct sync_file *sync_file_create(struct fence *fence);
+
+The caller pass the out-fence and gets back the sync_file. That is just the
+first step, next it needs to install an fd on sync_file->file. So it gets an
+fd:
+
+ fd = get_unused_fd_flags(O_CLOEXEC);
+
+and installs it on sync_file->file:
+
+ fd_install(fd, sync_file->file);
+
+The sync_file fd now can be sent to userspace.
+
+If the creation process fail, or the sync_file needs to be released by any
+other reason fput(sync_file->file) should be used.
+
+References:
+[1] struct sync_file in include/linux/sync_file.h
+[2] All interfaces mentioned above defined in include/linux/sync_file.h
diff --git a/Documentation/sysctl/vm.txt b/Documentation/sysctl/vm.txt
index 34a5fec..720355c 100644
--- a/Documentation/sysctl/vm.txt
+++ b/Documentation/sysctl/vm.txt
@@ -57,6 +57,7 @@
- panic_on_oom
- percpu_pagelist_fraction
- stat_interval
+- stat_refresh
- swappiness
- user_reserve_kbytes
- vfs_cache_pressure
@@ -755,6 +756,19 @@
==============================================================
+stat_refresh
+
+Any read or write (by root only) flushes all the per-cpu vm statistics
+into their global totals, for more accurate reports when testing
+e.g. cat /proc/sys/vm/stat_refresh /proc/meminfo
+
+As a side-effect, it also checks for negative totals (elsewhere reported
+as 0) and "fails" with EINVAL if any are found, with a warning in dmesg.
+(At time of writing, a few stats are known sometimes to be found negative,
+with no ill effects: errors and warnings on these stats are suppressed.)
+
+==============================================================
+
swappiness
This control is used to define how aggressive the kernel will swap
diff --git a/Documentation/sysrq.txt b/Documentation/sysrq.txt
index 13f5619..3a3b30a 100644
--- a/Documentation/sysrq.txt
+++ b/Documentation/sysrq.txt
@@ -212,7 +212,7 @@
overwritten since you registered it.
The Magic SysRQ system works by registering key operations against a key op
-lookup table, which is defined in 'drivers/char/sysrq.c'. This key table has
+lookup table, which is defined in 'drivers/tty/sysrq.c'. This key table has
a number of operations registered into it at compile time, but is mutable,
and 2 functions are exported for interface to it:
register_sysrq_key and unregister_sysrq_key.
diff --git a/Documentation/timers/hrtimers.txt b/Documentation/timers/hrtimers.txt
index ce31f65..588d857 100644
--- a/Documentation/timers/hrtimers.txt
+++ b/Documentation/timers/hrtimers.txt
@@ -28,9 +28,9 @@
- the unpredictable [O(N)] overhead of cascading leads to delays which
necessitate a more complex handling of high resolution timers, which
- in turn decreases robustness. Such a design still led to rather large
+ in turn decreases robustness. Such a design still leads to rather large
timing inaccuracies. Cascading is a fundamental property of the timer
- wheel concept, it cannot be 'designed out' without unevitably
+ wheel concept, it cannot be 'designed out' without inevitably
degrading other portions of the timers.c code in an unacceptable way.
- the implementation of the current posix-timer subsystem on top of
@@ -119,7 +119,7 @@
hrtimer functions now have clearer behavior and clearer names - such as
hrtimer_try_to_cancel() and hrtimer_cancel() [which are roughly
equivalent to del_timer() and del_timer_sync()] - so there's no direct
-1:1 mapping between them on the algorithmical level, and thus no real
+1:1 mapping between them on the algorithmic level, and thus no real
potential for code sharing either.
Basic data types: every time value, absolute or relative, is in a
diff --git a/Documentation/trace/coresight.txt b/Documentation/trace/coresight.txt
index 0a5c329..a33c88c 100644
--- a/Documentation/trace/coresight.txt
+++ b/Documentation/trace/coresight.txt
@@ -190,8 +190,8 @@
Last but not least, "struct module *owner" is expected to be set to reflect
the information carried in "THIS_MODULE".
-How to use
-----------
+How to use the tracer modules
+-----------------------------
Before trace collection can start, a coresight sink needs to be identify.
There is no limit on the amount of sinks (nor sources) that can be enabled at
@@ -297,3 +297,36 @@
Instruction 13570831 0x8026B584 E28DD00C false ADD sp,sp,#0xc
Instruction 0 0x8026B588 E8BD8000 true LDM sp!,{pc}
Timestamp Timestamp: 17107041535
+
+How to use the STM module
+-------------------------
+
+Using the System Trace Macrocell module is the same as the tracers - the only
+difference is that clients are driving the trace capture rather
+than the program flow through the code.
+
+As with any other CoreSight component, specifics about the STM tracer can be
+found in sysfs with more information on each entry being found in [1]:
+
+root@genericarmv8:~# ls /sys/bus/coresight/devices/20100000.stm
+enable_source hwevent_select port_enable subsystem uevent
+hwevent_enable mgmt port_select traceid
+root@genericarmv8:~#
+
+Like any other source a sink needs to be identified and the STM enabled before
+being used:
+
+root@genericarmv8:~# echo 1 > /sys/bus/coresight/devices/20010000.etf/enable_sink
+root@genericarmv8:~# echo 1 > /sys/bus/coresight/devices/20100000.stm/enable_source
+
+From there user space applications can request and use channels using the devfs
+interface provided for that purpose by the generic STM API:
+
+root@genericarmv8:~# ls -l /dev/20100000.stm
+crw------- 1 root root 10, 61 Jan 3 18:11 /dev/20100000.stm
+root@genericarmv8:~#
+
+Details on how to use the generic STM API can be found here [2].
+
+[1]. Documentation/ABI/testing/sysfs-bus-coresight-devices-stm
+[2]. Documentation/trace/stm.txt
diff --git a/Documentation/trace/events.txt b/Documentation/trace/events.txt
index c010be8..08d74d7 100644
--- a/Documentation/trace/events.txt
+++ b/Documentation/trace/events.txt
@@ -512,3 +512,1558 @@
Note that there can be only one traceon or traceoff trigger per
triggering event.
+
+- hist
+
+ This command aggregates event hits into a hash table keyed on one or
+ more trace event format fields (or stacktrace) and a set of running
+ totals derived from one or more trace event format fields and/or
+ event counts (hitcount).
+
+ The format of a hist trigger is as follows:
+
+ hist:keys=<field1[,field2,...]>[:values=<field1[,field2,...]>]
+ [:sort=<field1[,field2,...]>][:size=#entries][:pause][:continue]
+ [:clear][:name=histname1] [if <filter>]
+
+ When a matching event is hit, an entry is added to a hash table
+ using the key(s) and value(s) named. Keys and values correspond to
+ fields in the event's format description. Values must correspond to
+ numeric fields - on an event hit, the value(s) will be added to a
+ sum kept for that field. The special string 'hitcount' can be used
+ in place of an explicit value field - this is simply a count of
+ event hits. If 'values' isn't specified, an implicit 'hitcount'
+ value will be automatically created and used as the only value.
+ Keys can be any field, or the special string 'stacktrace', which
+ will use the event's kernel stacktrace as the key. The keywords
+ 'keys' or 'key' can be used to specify keys, and the keywords
+ 'values', 'vals', or 'val' can be used to specify values. Compound
+ keys consisting of up to two fields can be specified by the 'keys'
+ keyword. Hashing a compound key produces a unique entry in the
+ table for each unique combination of component keys, and can be
+ useful for providing more fine-grained summaries of event data.
+ Additionally, sort keys consisting of up to two fields can be
+ specified by the 'sort' keyword. If more than one field is
+ specified, the result will be a 'sort within a sort': the first key
+ is taken to be the primary sort key and the second the secondary
+ key. If a hist trigger is given a name using the 'name' parameter,
+ its histogram data will be shared with other triggers of the same
+ name, and trigger hits will update this common data. Only triggers
+ with 'compatible' fields can be combined in this way; triggers are
+ 'compatible' if the fields named in the trigger share the same
+ number and type of fields and those fields also have the same names.
+ Note that any two events always share the compatible 'hitcount' and
+ 'stacktrace' fields and can therefore be combined using those
+ fields, however pointless that may be.
+
+ 'hist' triggers add a 'hist' file to each event's subdirectory.
+ Reading the 'hist' file for the event will dump the hash table in
+ its entirety to stdout. If there are multiple hist triggers
+ attached to an event, there will be a table for each trigger in the
+ output. The table displayed for a named trigger will be the same as
+ any other instance having the same name. Each printed hash table
+ entry is a simple list of the keys and values comprising the entry;
+ keys are printed first and are delineated by curly braces, and are
+ followed by the set of value fields for the entry. By default,
+ numeric fields are displayed as base-10 integers. This can be
+ modified by appending any of the following modifiers to the field
+ name:
+
+ .hex display a number as a hex value
+ .sym display an address as a symbol
+ .sym-offset display an address as a symbol and offset
+ .syscall display a syscall id as a system call name
+ .execname display a common_pid as a program name
+
+ Note that in general the semantics of a given field aren't
+ interpreted when applying a modifier to it, but there are some
+ restrictions to be aware of in this regard:
+
+ - only the 'hex' modifier can be used for values (because values
+ are essentially sums, and the other modifiers don't make sense
+ in that context).
+ - the 'execname' modifier can only be used on a 'common_pid'. The
+ reason for this is that the execname is simply the 'comm' value
+ saved for the 'current' process when an event was triggered,
+ which is the same as the common_pid value saved by the event
+ tracing code. Trying to apply that comm value to other pid
+ values wouldn't be correct, and typically events that care save
+ pid-specific comm fields in the event itself.
+
+ A typical usage scenario would be the following to enable a hist
+ trigger, read its current contents, and then turn it off:
+
+ # echo 'hist:keys=skbaddr.hex:vals=len' > \
+ /sys/kernel/debug/tracing/events/net/netif_rx/trigger
+
+ # cat /sys/kernel/debug/tracing/events/net/netif_rx/hist
+
+ # echo '!hist:keys=skbaddr.hex:vals=len' > \
+ /sys/kernel/debug/tracing/events/net/netif_rx/trigger
+
+ The trigger file itself can be read to show the details of the
+ currently attached hist trigger. This information is also displayed
+ at the top of the 'hist' file when read.
+
+ By default, the size of the hash table is 2048 entries. The 'size'
+ parameter can be used to specify more or fewer than that. The units
+ are in terms of hashtable entries - if a run uses more entries than
+ specified, the results will show the number of 'drops', the number
+ of hits that were ignored. The size should be a power of 2 between
+ 128 and 131072 (any non- power-of-2 number specified will be rounded
+ up).
+
+ The 'sort' parameter can be used to specify a value field to sort
+ on. The default if unspecified is 'hitcount' and the default sort
+ order is 'ascending'. To sort in the opposite direction, append
+ .descending' to the sort key.
+
+ The 'pause' parameter can be used to pause an existing hist trigger
+ or to start a hist trigger but not log any events until told to do
+ so. 'continue' or 'cont' can be used to start or restart a paused
+ hist trigger.
+
+ The 'clear' parameter will clear the contents of a running hist
+ trigger and leave its current paused/active state.
+
+ Note that the 'pause', 'cont', and 'clear' parameters should be
+ applied using 'append' shell operator ('>>') if applied to an
+ existing trigger, rather than via the '>' operator, which will cause
+ the trigger to be removed through truncation.
+
+- enable_hist/disable_hist
+
+ The enable_hist and disable_hist triggers can be used to have one
+ event conditionally start and stop another event's already-attached
+ hist trigger. Any number of enable_hist and disable_hist triggers
+ can be attached to a given event, allowing that event to kick off
+ and stop aggregations on a host of other events.
+
+ The format is very similar to the enable/disable_event triggers:
+
+ enable_hist:<system>:<event>[:count]
+ disable_hist:<system>:<event>[:count]
+
+ Instead of enabling or disabling the tracing of the target event
+ into the trace buffer as the enable/disable_event triggers do, the
+ enable/disable_hist triggers enable or disable the aggregation of
+ the target event into a hash table.
+
+ A typical usage scenario for the enable_hist/disable_hist triggers
+ would be to first set up a paused hist trigger on some event,
+ followed by an enable_hist/disable_hist pair that turns the hist
+ aggregation on and off when conditions of interest are hit:
+
+ # echo 'hist:keys=skbaddr.hex:vals=len:pause' > \
+ /sys/kernel/debug/tracing/events/net/netif_receive_skb/trigger
+
+ # echo 'enable_hist:net:netif_receive_skb if filename==/usr/bin/wget' > \
+ /sys/kernel/debug/tracing/events/sched/sched_process_exec/trigger
+
+ # echo 'disable_hist:net:netif_receive_skb if comm==wget' > \
+ /sys/kernel/debug/tracing/events/sched/sched_process_exit/trigger
+
+ The above sets up an initially paused hist trigger which is unpaused
+ and starts aggregating events when a given program is executed, and
+ which stops aggregating when the process exits and the hist trigger
+ is paused again.
+
+ The examples below provide a more concrete illustration of the
+ concepts and typical usage patterns discussed above.
+
+
+6.2 'hist' trigger examples
+---------------------------
+
+ The first set of examples creates aggregations using the kmalloc
+ event. The fields that can be used for the hist trigger are listed
+ in the kmalloc event's format file:
+
+ # cat /sys/kernel/debug/tracing/events/kmem/kmalloc/format
+ name: kmalloc
+ ID: 374
+ format:
+ field:unsigned short common_type; offset:0; size:2; signed:0;
+ field:unsigned char common_flags; offset:2; size:1; signed:0;
+ field:unsigned char common_preempt_count; offset:3; size:1; signed:0;
+ field:int common_pid; offset:4; size:4; signed:1;
+
+ field:unsigned long call_site; offset:8; size:8; signed:0;
+ field:const void * ptr; offset:16; size:8; signed:0;
+ field:size_t bytes_req; offset:24; size:8; signed:0;
+ field:size_t bytes_alloc; offset:32; size:8; signed:0;
+ field:gfp_t gfp_flags; offset:40; size:4; signed:0;
+
+ We'll start by creating a hist trigger that generates a simple table
+ that lists the total number of bytes requested for each function in
+ the kernel that made one or more calls to kmalloc:
+
+ # echo 'hist:key=call_site:val=bytes_req' > \
+ /sys/kernel/debug/tracing/events/kmem/kmalloc/trigger
+
+ This tells the tracing system to create a 'hist' trigger using the
+ call_site field of the kmalloc event as the key for the table, which
+ just means that each unique call_site address will have an entry
+ created for it in the table. The 'val=bytes_req' parameter tells
+ the hist trigger that for each unique entry (call_site) in the
+ table, it should keep a running total of the number of bytes
+ requested by that call_site.
+
+ We'll let it run for awhile and then dump the contents of the 'hist'
+ file in the kmalloc event's subdirectory (for readability, a number
+ of entries have been omitted):
+
+ # cat /sys/kernel/debug/tracing/events/kmem/kmalloc/hist
+ # trigger info: hist:keys=call_site:vals=bytes_req:sort=hitcount:size=2048 [active]
+
+ { call_site: 18446744072106379007 } hitcount: 1 bytes_req: 176
+ { call_site: 18446744071579557049 } hitcount: 1 bytes_req: 1024
+ { call_site: 18446744071580608289 } hitcount: 1 bytes_req: 16384
+ { call_site: 18446744071581827654 } hitcount: 1 bytes_req: 24
+ { call_site: 18446744071580700980 } hitcount: 1 bytes_req: 8
+ { call_site: 18446744071579359876 } hitcount: 1 bytes_req: 152
+ { call_site: 18446744071580795365 } hitcount: 3 bytes_req: 144
+ { call_site: 18446744071581303129 } hitcount: 3 bytes_req: 144
+ { call_site: 18446744071580713234 } hitcount: 4 bytes_req: 2560
+ { call_site: 18446744071580933750 } hitcount: 4 bytes_req: 736
+ .
+ .
+ .
+ { call_site: 18446744072106047046 } hitcount: 69 bytes_req: 5576
+ { call_site: 18446744071582116407 } hitcount: 73 bytes_req: 2336
+ { call_site: 18446744072106054684 } hitcount: 136 bytes_req: 140504
+ { call_site: 18446744072106224230 } hitcount: 136 bytes_req: 19584
+ { call_site: 18446744072106078074 } hitcount: 153 bytes_req: 2448
+ { call_site: 18446744072106062406 } hitcount: 153 bytes_req: 36720
+ { call_site: 18446744071582507929 } hitcount: 153 bytes_req: 37088
+ { call_site: 18446744072102520590 } hitcount: 273 bytes_req: 10920
+ { call_site: 18446744071582143559 } hitcount: 358 bytes_req: 716
+ { call_site: 18446744072106465852 } hitcount: 417 bytes_req: 56712
+ { call_site: 18446744072102523378 } hitcount: 485 bytes_req: 27160
+ { call_site: 18446744072099568646 } hitcount: 1676 bytes_req: 33520
+
+ Totals:
+ Hits: 4610
+ Entries: 45
+ Dropped: 0
+
+ The output displays a line for each entry, beginning with the key
+ specified in the trigger, followed by the value(s) also specified in
+ the trigger. At the beginning of the output is a line that displays
+ the trigger info, which can also be displayed by reading the
+ 'trigger' file:
+
+ # cat /sys/kernel/debug/tracing/events/kmem/kmalloc/trigger
+ hist:keys=call_site:vals=bytes_req:sort=hitcount:size=2048 [active]
+
+ At the end of the output are a few lines that display the overall
+ totals for the run. The 'Hits' field shows the total number of
+ times the event trigger was hit, the 'Entries' field shows the total
+ number of used entries in the hash table, and the 'Dropped' field
+ shows the number of hits that were dropped because the number of
+ used entries for the run exceeded the maximum number of entries
+ allowed for the table (normally 0, but if not a hint that you may
+ want to increase the size of the table using the 'size' parameter).
+
+ Notice in the above output that there's an extra field, 'hitcount',
+ which wasn't specified in the trigger. Also notice that in the
+ trigger info output, there's a parameter, 'sort=hitcount', which
+ wasn't specified in the trigger either. The reason for that is that
+ every trigger implicitly keeps a count of the total number of hits
+ attributed to a given entry, called the 'hitcount'. That hitcount
+ information is explicitly displayed in the output, and in the
+ absence of a user-specified sort parameter, is used as the default
+ sort field.
+
+ The value 'hitcount' can be used in place of an explicit value in
+ the 'values' parameter if you don't really need to have any
+ particular field summed and are mainly interested in hit
+ frequencies.
+
+ To turn the hist trigger off, simply call up the trigger in the
+ command history and re-execute it with a '!' prepended:
+
+ # echo '!hist:key=call_site:val=bytes_req' > \
+ /sys/kernel/debug/tracing/events/kmem/kmalloc/trigger
+
+ Finally, notice that the call_site as displayed in the output above
+ isn't really very useful. It's an address, but normally addresses
+ are displayed in hex. To have a numeric field displayed as a hex
+ value, simply append '.hex' to the field name in the trigger:
+
+ # echo 'hist:key=call_site.hex:val=bytes_req' > \
+ /sys/kernel/debug/tracing/events/kmem/kmalloc/trigger
+
+ # cat /sys/kernel/debug/tracing/events/kmem/kmalloc/hist
+ # trigger info: hist:keys=call_site.hex:vals=bytes_req:sort=hitcount:size=2048 [active]
+
+ { call_site: ffffffffa026b291 } hitcount: 1 bytes_req: 433
+ { call_site: ffffffffa07186ff } hitcount: 1 bytes_req: 176
+ { call_site: ffffffff811ae721 } hitcount: 1 bytes_req: 16384
+ { call_site: ffffffff811c5134 } hitcount: 1 bytes_req: 8
+ { call_site: ffffffffa04a9ebb } hitcount: 1 bytes_req: 511
+ { call_site: ffffffff8122e0a6 } hitcount: 1 bytes_req: 12
+ { call_site: ffffffff8107da84 } hitcount: 1 bytes_req: 152
+ { call_site: ffffffff812d8246 } hitcount: 1 bytes_req: 24
+ { call_site: ffffffff811dc1e5 } hitcount: 3 bytes_req: 144
+ { call_site: ffffffffa02515e8 } hitcount: 3 bytes_req: 648
+ { call_site: ffffffff81258159 } hitcount: 3 bytes_req: 144
+ { call_site: ffffffff811c80f4 } hitcount: 4 bytes_req: 544
+ .
+ .
+ .
+ { call_site: ffffffffa06c7646 } hitcount: 106 bytes_req: 8024
+ { call_site: ffffffffa06cb246 } hitcount: 132 bytes_req: 31680
+ { call_site: ffffffffa06cef7a } hitcount: 132 bytes_req: 2112
+ { call_site: ffffffff8137e399 } hitcount: 132 bytes_req: 23232
+ { call_site: ffffffffa06c941c } hitcount: 185 bytes_req: 171360
+ { call_site: ffffffffa06f2a66 } hitcount: 185 bytes_req: 26640
+ { call_site: ffffffffa036a70e } hitcount: 265 bytes_req: 10600
+ { call_site: ffffffff81325447 } hitcount: 292 bytes_req: 584
+ { call_site: ffffffffa072da3c } hitcount: 446 bytes_req: 60656
+ { call_site: ffffffffa036b1f2 } hitcount: 526 bytes_req: 29456
+ { call_site: ffffffffa0099c06 } hitcount: 1780 bytes_req: 35600
+
+ Totals:
+ Hits: 4775
+ Entries: 46
+ Dropped: 0
+
+ Even that's only marginally more useful - while hex values do look
+ more like addresses, what users are typically more interested in
+ when looking at text addresses are the corresponding symbols
+ instead. To have an address displayed as symbolic value instead,
+ simply append '.sym' or '.sym-offset' to the field name in the
+ trigger:
+
+ # echo 'hist:key=call_site.sym:val=bytes_req' > \
+ /sys/kernel/debug/tracing/events/kmem/kmalloc/trigger
+
+ # cat /sys/kernel/debug/tracing/events/kmem/kmalloc/hist
+ # trigger info: hist:keys=call_site.sym:vals=bytes_req:sort=hitcount:size=2048 [active]
+
+ { call_site: [ffffffff810adcb9] syslog_print_all } hitcount: 1 bytes_req: 1024
+ { call_site: [ffffffff8154bc62] usb_control_msg } hitcount: 1 bytes_req: 8
+ { call_site: [ffffffffa00bf6fe] hidraw_send_report [hid] } hitcount: 1 bytes_req: 7
+ { call_site: [ffffffff8154acbe] usb_alloc_urb } hitcount: 1 bytes_req: 192
+ { call_site: [ffffffffa00bf1ca] hidraw_report_event [hid] } hitcount: 1 bytes_req: 7
+ { call_site: [ffffffff811e3a25] __seq_open_private } hitcount: 1 bytes_req: 40
+ { call_site: [ffffffff8109524a] alloc_fair_sched_group } hitcount: 2 bytes_req: 128
+ { call_site: [ffffffff811febd5] fsnotify_alloc_group } hitcount: 2 bytes_req: 528
+ { call_site: [ffffffff81440f58] __tty_buffer_request_room } hitcount: 2 bytes_req: 2624
+ { call_site: [ffffffff81200ba6] inotify_new_group } hitcount: 2 bytes_req: 96
+ { call_site: [ffffffffa05e19af] ieee80211_start_tx_ba_session [mac80211] } hitcount: 2 bytes_req: 464
+ { call_site: [ffffffff81672406] tcp_get_metrics } hitcount: 2 bytes_req: 304
+ { call_site: [ffffffff81097ec2] alloc_rt_sched_group } hitcount: 2 bytes_req: 128
+ { call_site: [ffffffff81089b05] sched_create_group } hitcount: 2 bytes_req: 1424
+ .
+ .
+ .
+ { call_site: [ffffffffa04a580c] intel_crtc_page_flip [i915] } hitcount: 1185 bytes_req: 123240
+ { call_site: [ffffffffa0287592] drm_mode_page_flip_ioctl [drm] } hitcount: 1185 bytes_req: 104280
+ { call_site: [ffffffffa04c4a3c] intel_plane_duplicate_state [i915] } hitcount: 1402 bytes_req: 190672
+ { call_site: [ffffffff812891ca] ext4_find_extent } hitcount: 1518 bytes_req: 146208
+ { call_site: [ffffffffa029070e] drm_vma_node_allow [drm] } hitcount: 1746 bytes_req: 69840
+ { call_site: [ffffffffa045e7c4] i915_gem_do_execbuffer.isra.23 [i915] } hitcount: 2021 bytes_req: 792312
+ { call_site: [ffffffffa02911f2] drm_modeset_lock_crtc [drm] } hitcount: 2592 bytes_req: 145152
+ { call_site: [ffffffffa0489a66] intel_ring_begin [i915] } hitcount: 2629 bytes_req: 378576
+ { call_site: [ffffffffa046041c] i915_gem_execbuffer2 [i915] } hitcount: 2629 bytes_req: 3783248
+ { call_site: [ffffffff81325607] apparmor_file_alloc_security } hitcount: 5192 bytes_req: 10384
+ { call_site: [ffffffffa00b7c06] hid_report_raw_event [hid] } hitcount: 5529 bytes_req: 110584
+ { call_site: [ffffffff8131ebf7] aa_alloc_task_context } hitcount: 21943 bytes_req: 702176
+ { call_site: [ffffffff8125847d] ext4_htree_store_dirent } hitcount: 55759 bytes_req: 5074265
+
+ Totals:
+ Hits: 109928
+ Entries: 71
+ Dropped: 0
+
+ Because the default sort key above is 'hitcount', the above shows a
+ the list of call_sites by increasing hitcount, so that at the bottom
+ we see the functions that made the most kmalloc calls during the
+ run. If instead we we wanted to see the top kmalloc callers in
+ terms of the number of bytes requested rather than the number of
+ calls, and we wanted the top caller to appear at the top, we can use
+ the 'sort' parameter, along with the 'descending' modifier:
+
+ # echo 'hist:key=call_site.sym:val=bytes_req:sort=bytes_req.descending' > \
+ /sys/kernel/debug/tracing/events/kmem/kmalloc/trigger
+
+ # cat /sys/kernel/debug/tracing/events/kmem/kmalloc/hist
+ # trigger info: hist:keys=call_site.sym:vals=bytes_req:sort=bytes_req.descending:size=2048 [active]
+
+ { call_site: [ffffffffa046041c] i915_gem_execbuffer2 [i915] } hitcount: 2186 bytes_req: 3397464
+ { call_site: [ffffffffa045e7c4] i915_gem_do_execbuffer.isra.23 [i915] } hitcount: 1790 bytes_req: 712176
+ { call_site: [ffffffff8125847d] ext4_htree_store_dirent } hitcount: 8132 bytes_req: 513135
+ { call_site: [ffffffff811e2a1b] seq_buf_alloc } hitcount: 106 bytes_req: 440128
+ { call_site: [ffffffffa0489a66] intel_ring_begin [i915] } hitcount: 2186 bytes_req: 314784
+ { call_site: [ffffffff812891ca] ext4_find_extent } hitcount: 2174 bytes_req: 208992
+ { call_site: [ffffffff811ae8e1] __kmalloc } hitcount: 8 bytes_req: 131072
+ { call_site: [ffffffffa04c4a3c] intel_plane_duplicate_state [i915] } hitcount: 859 bytes_req: 116824
+ { call_site: [ffffffffa02911f2] drm_modeset_lock_crtc [drm] } hitcount: 1834 bytes_req: 102704
+ { call_site: [ffffffffa04a580c] intel_crtc_page_flip [i915] } hitcount: 972 bytes_req: 101088
+ { call_site: [ffffffffa0287592] drm_mode_page_flip_ioctl [drm] } hitcount: 972 bytes_req: 85536
+ { call_site: [ffffffffa00b7c06] hid_report_raw_event [hid] } hitcount: 3333 bytes_req: 66664
+ { call_site: [ffffffff8137e559] sg_kmalloc } hitcount: 209 bytes_req: 61632
+ .
+ .
+ .
+ { call_site: [ffffffff81095225] alloc_fair_sched_group } hitcount: 2 bytes_req: 128
+ { call_site: [ffffffff81097ec2] alloc_rt_sched_group } hitcount: 2 bytes_req: 128
+ { call_site: [ffffffff812d8406] copy_semundo } hitcount: 2 bytes_req: 48
+ { call_site: [ffffffff81200ba6] inotify_new_group } hitcount: 1 bytes_req: 48
+ { call_site: [ffffffffa027121a] drm_getmagic [drm] } hitcount: 1 bytes_req: 48
+ { call_site: [ffffffff811e3a25] __seq_open_private } hitcount: 1 bytes_req: 40
+ { call_site: [ffffffff811c52f4] bprm_change_interp } hitcount: 2 bytes_req: 16
+ { call_site: [ffffffff8154bc62] usb_control_msg } hitcount: 1 bytes_req: 8
+ { call_site: [ffffffffa00bf1ca] hidraw_report_event [hid] } hitcount: 1 bytes_req: 7
+ { call_site: [ffffffffa00bf6fe] hidraw_send_report [hid] } hitcount: 1 bytes_req: 7
+
+ Totals:
+ Hits: 32133
+ Entries: 81
+ Dropped: 0
+
+ To display the offset and size information in addition to the symbol
+ name, just use 'sym-offset' instead:
+
+ # echo 'hist:key=call_site.sym-offset:val=bytes_req:sort=bytes_req.descending' > \
+ /sys/kernel/debug/tracing/events/kmem/kmalloc/trigger
+
+ # cat /sys/kernel/debug/tracing/events/kmem/kmalloc/hist
+ # trigger info: hist:keys=call_site.sym-offset:vals=bytes_req:sort=bytes_req.descending:size=2048 [active]
+
+ { call_site: [ffffffffa046041c] i915_gem_execbuffer2+0x6c/0x2c0 [i915] } hitcount: 4569 bytes_req: 3163720
+ { call_site: [ffffffffa0489a66] intel_ring_begin+0xc6/0x1f0 [i915] } hitcount: 4569 bytes_req: 657936
+ { call_site: [ffffffffa045e7c4] i915_gem_do_execbuffer.isra.23+0x694/0x1020 [i915] } hitcount: 1519 bytes_req: 472936
+ { call_site: [ffffffffa045e646] i915_gem_do_execbuffer.isra.23+0x516/0x1020 [i915] } hitcount: 3050 bytes_req: 211832
+ { call_site: [ffffffff811e2a1b] seq_buf_alloc+0x1b/0x50 } hitcount: 34 bytes_req: 148384
+ { call_site: [ffffffffa04a580c] intel_crtc_page_flip+0xbc/0x870 [i915] } hitcount: 1385 bytes_req: 144040
+ { call_site: [ffffffff811ae8e1] __kmalloc+0x191/0x1b0 } hitcount: 8 bytes_req: 131072
+ { call_site: [ffffffffa0287592] drm_mode_page_flip_ioctl+0x282/0x360 [drm] } hitcount: 1385 bytes_req: 121880
+ { call_site: [ffffffffa02911f2] drm_modeset_lock_crtc+0x32/0x100 [drm] } hitcount: 1848 bytes_req: 103488
+ { call_site: [ffffffffa04c4a3c] intel_plane_duplicate_state+0x2c/0xa0 [i915] } hitcount: 461 bytes_req: 62696
+ { call_site: [ffffffffa029070e] drm_vma_node_allow+0x2e/0xd0 [drm] } hitcount: 1541 bytes_req: 61640
+ { call_site: [ffffffff815f8d7b] sk_prot_alloc+0xcb/0x1b0 } hitcount: 57 bytes_req: 57456
+ .
+ .
+ .
+ { call_site: [ffffffff8109524a] alloc_fair_sched_group+0x5a/0x1a0 } hitcount: 2 bytes_req: 128
+ { call_site: [ffffffffa027b921] drm_vm_open_locked+0x31/0xa0 [drm] } hitcount: 3 bytes_req: 96
+ { call_site: [ffffffff8122e266] proc_self_follow_link+0x76/0xb0 } hitcount: 8 bytes_req: 96
+ { call_site: [ffffffff81213e80] load_elf_binary+0x240/0x1650 } hitcount: 3 bytes_req: 84
+ { call_site: [ffffffff8154bc62] usb_control_msg+0x42/0x110 } hitcount: 1 bytes_req: 8
+ { call_site: [ffffffffa00bf6fe] hidraw_send_report+0x7e/0x1a0 [hid] } hitcount: 1 bytes_req: 7
+ { call_site: [ffffffffa00bf1ca] hidraw_report_event+0x8a/0x120 [hid] } hitcount: 1 bytes_req: 7
+
+ Totals:
+ Hits: 26098
+ Entries: 64
+ Dropped: 0
+
+ We can also add multiple fields to the 'values' parameter. For
+ example, we might want to see the total number of bytes allocated
+ alongside bytes requested, and display the result sorted by bytes
+ allocated in a descending order:
+
+ # echo 'hist:keys=call_site.sym:values=bytes_req,bytes_alloc:sort=bytes_alloc.descending' > \
+ /sys/kernel/debug/tracing/events/kmem/kmalloc/trigger
+
+ # cat /sys/kernel/debug/tracing/events/kmem/kmalloc/hist
+ # trigger info: hist:keys=call_site.sym:vals=bytes_req,bytes_alloc:sort=bytes_alloc.descending:size=2048 [active]
+
+ { call_site: [ffffffffa046041c] i915_gem_execbuffer2 [i915] } hitcount: 7403 bytes_req: 4084360 bytes_alloc: 5958016
+ { call_site: [ffffffff811e2a1b] seq_buf_alloc } hitcount: 541 bytes_req: 2213968 bytes_alloc: 2228224
+ { call_site: [ffffffffa0489a66] intel_ring_begin [i915] } hitcount: 7404 bytes_req: 1066176 bytes_alloc: 1421568
+ { call_site: [ffffffffa045e7c4] i915_gem_do_execbuffer.isra.23 [i915] } hitcount: 1565 bytes_req: 557368 bytes_alloc: 1037760
+ { call_site: [ffffffff8125847d] ext4_htree_store_dirent } hitcount: 9557 bytes_req: 595778 bytes_alloc: 695744
+ { call_site: [ffffffffa045e646] i915_gem_do_execbuffer.isra.23 [i915] } hitcount: 5839 bytes_req: 430680 bytes_alloc: 470400
+ { call_site: [ffffffffa04c4a3c] intel_plane_duplicate_state [i915] } hitcount: 2388 bytes_req: 324768 bytes_alloc: 458496
+ { call_site: [ffffffffa02911f2] drm_modeset_lock_crtc [drm] } hitcount: 3911 bytes_req: 219016 bytes_alloc: 250304
+ { call_site: [ffffffff815f8d7b] sk_prot_alloc } hitcount: 235 bytes_req: 236880 bytes_alloc: 240640
+ { call_site: [ffffffff8137e559] sg_kmalloc } hitcount: 557 bytes_req: 169024 bytes_alloc: 221760
+ { call_site: [ffffffffa00b7c06] hid_report_raw_event [hid] } hitcount: 9378 bytes_req: 187548 bytes_alloc: 206312
+ { call_site: [ffffffffa04a580c] intel_crtc_page_flip [i915] } hitcount: 1519 bytes_req: 157976 bytes_alloc: 194432
+ .
+ .
+ .
+ { call_site: [ffffffff8109bd3b] sched_autogroup_create_attach } hitcount: 2 bytes_req: 144 bytes_alloc: 192
+ { call_site: [ffffffff81097ee8] alloc_rt_sched_group } hitcount: 2 bytes_req: 128 bytes_alloc: 128
+ { call_site: [ffffffff8109524a] alloc_fair_sched_group } hitcount: 2 bytes_req: 128 bytes_alloc: 128
+ { call_site: [ffffffff81095225] alloc_fair_sched_group } hitcount: 2 bytes_req: 128 bytes_alloc: 128
+ { call_site: [ffffffff81097ec2] alloc_rt_sched_group } hitcount: 2 bytes_req: 128 bytes_alloc: 128
+ { call_site: [ffffffff81213e80] load_elf_binary } hitcount: 3 bytes_req: 84 bytes_alloc: 96
+ { call_site: [ffffffff81079a2e] kthread_create_on_node } hitcount: 1 bytes_req: 56 bytes_alloc: 64
+ { call_site: [ffffffffa00bf6fe] hidraw_send_report [hid] } hitcount: 1 bytes_req: 7 bytes_alloc: 8
+ { call_site: [ffffffff8154bc62] usb_control_msg } hitcount: 1 bytes_req: 8 bytes_alloc: 8
+ { call_site: [ffffffffa00bf1ca] hidraw_report_event [hid] } hitcount: 1 bytes_req: 7 bytes_alloc: 8
+
+ Totals:
+ Hits: 66598
+ Entries: 65
+ Dropped: 0
+
+ Finally, to finish off our kmalloc example, instead of simply having
+ the hist trigger display symbolic call_sites, we can have the hist
+ trigger additionally display the complete set of kernel stack traces
+ that led to each call_site. To do that, we simply use the special
+ value 'stacktrace' for the key parameter:
+
+ # echo 'hist:keys=stacktrace:values=bytes_req,bytes_alloc:sort=bytes_alloc' > \
+ /sys/kernel/debug/tracing/events/kmem/kmalloc/trigger
+
+ The above trigger will use the kernel stack trace in effect when an
+ event is triggered as the key for the hash table. This allows the
+ enumeration of every kernel callpath that led up to a particular
+ event, along with a running total of any of the event fields for
+ that event. Here we tally bytes requested and bytes allocated for
+ every callpath in the system that led up to a kmalloc (in this case
+ every callpath to a kmalloc for a kernel compile):
+
+ # cat /sys/kernel/debug/tracing/events/kmem/kmalloc/hist
+ # trigger info: hist:keys=stacktrace:vals=bytes_req,bytes_alloc:sort=bytes_alloc:size=2048 [active]
+
+ { stacktrace:
+ __kmalloc_track_caller+0x10b/0x1a0
+ kmemdup+0x20/0x50
+ hidraw_report_event+0x8a/0x120 [hid]
+ hid_report_raw_event+0x3ea/0x440 [hid]
+ hid_input_report+0x112/0x190 [hid]
+ hid_irq_in+0xc2/0x260 [usbhid]
+ __usb_hcd_giveback_urb+0x72/0x120
+ usb_giveback_urb_bh+0x9e/0xe0
+ tasklet_hi_action+0xf8/0x100
+ __do_softirq+0x114/0x2c0
+ irq_exit+0xa5/0xb0
+ do_IRQ+0x5a/0xf0
+ ret_from_intr+0x0/0x30
+ cpuidle_enter+0x17/0x20
+ cpu_startup_entry+0x315/0x3e0
+ rest_init+0x7c/0x80
+ } hitcount: 3 bytes_req: 21 bytes_alloc: 24
+ { stacktrace:
+ __kmalloc_track_caller+0x10b/0x1a0
+ kmemdup+0x20/0x50
+ hidraw_report_event+0x8a/0x120 [hid]
+ hid_report_raw_event+0x3ea/0x440 [hid]
+ hid_input_report+0x112/0x190 [hid]
+ hid_irq_in+0xc2/0x260 [usbhid]
+ __usb_hcd_giveback_urb+0x72/0x120
+ usb_giveback_urb_bh+0x9e/0xe0
+ tasklet_hi_action+0xf8/0x100
+ __do_softirq+0x114/0x2c0
+ irq_exit+0xa5/0xb0
+ do_IRQ+0x5a/0xf0
+ ret_from_intr+0x0/0x30
+ } hitcount: 3 bytes_req: 21 bytes_alloc: 24
+ { stacktrace:
+ kmem_cache_alloc_trace+0xeb/0x150
+ aa_alloc_task_context+0x27/0x40
+ apparmor_cred_prepare+0x1f/0x50
+ security_prepare_creds+0x16/0x20
+ prepare_creds+0xdf/0x1a0
+ SyS_capset+0xb5/0x200
+ system_call_fastpath+0x12/0x6a
+ } hitcount: 1 bytes_req: 32 bytes_alloc: 32
+ .
+ .
+ .
+ { stacktrace:
+ __kmalloc+0x11b/0x1b0
+ i915_gem_execbuffer2+0x6c/0x2c0 [i915]
+ drm_ioctl+0x349/0x670 [drm]
+ do_vfs_ioctl+0x2f0/0x4f0
+ SyS_ioctl+0x81/0xa0
+ system_call_fastpath+0x12/0x6a
+ } hitcount: 17726 bytes_req: 13944120 bytes_alloc: 19593808
+ { stacktrace:
+ __kmalloc+0x11b/0x1b0
+ load_elf_phdrs+0x76/0xa0
+ load_elf_binary+0x102/0x1650
+ search_binary_handler+0x97/0x1d0
+ do_execveat_common.isra.34+0x551/0x6e0
+ SyS_execve+0x3a/0x50
+ return_from_execve+0x0/0x23
+ } hitcount: 33348 bytes_req: 17152128 bytes_alloc: 20226048
+ { stacktrace:
+ kmem_cache_alloc_trace+0xeb/0x150
+ apparmor_file_alloc_security+0x27/0x40
+ security_file_alloc+0x16/0x20
+ get_empty_filp+0x93/0x1c0
+ path_openat+0x31/0x5f0
+ do_filp_open+0x3a/0x90
+ do_sys_open+0x128/0x220
+ SyS_open+0x1e/0x20
+ system_call_fastpath+0x12/0x6a
+ } hitcount: 4766422 bytes_req: 9532844 bytes_alloc: 38131376
+ { stacktrace:
+ __kmalloc+0x11b/0x1b0
+ seq_buf_alloc+0x1b/0x50
+ seq_read+0x2cc/0x370
+ proc_reg_read+0x3d/0x80
+ __vfs_read+0x28/0xe0
+ vfs_read+0x86/0x140
+ SyS_read+0x46/0xb0
+ system_call_fastpath+0x12/0x6a
+ } hitcount: 19133 bytes_req: 78368768 bytes_alloc: 78368768
+
+ Totals:
+ Hits: 6085872
+ Entries: 253
+ Dropped: 0
+
+ If you key a hist trigger on common_pid, in order for example to
+ gather and display sorted totals for each process, you can use the
+ special .execname modifier to display the executable names for the
+ processes in the table rather than raw pids. The example below
+ keeps a per-process sum of total bytes read:
+
+ # echo 'hist:key=common_pid.execname:val=count:sort=count.descending' > \
+ /sys/kernel/debug/tracing/events/syscalls/sys_enter_read/trigger
+
+ # cat /sys/kernel/debug/tracing/events/syscalls/sys_enter_read/hist
+ # trigger info: hist:keys=common_pid.execname:vals=count:sort=count.descending:size=2048 [active]
+
+ { common_pid: gnome-terminal [ 3196] } hitcount: 280 count: 1093512
+ { common_pid: Xorg [ 1309] } hitcount: 525 count: 256640
+ { common_pid: compiz [ 2889] } hitcount: 59 count: 254400
+ { common_pid: bash [ 8710] } hitcount: 3 count: 66369
+ { common_pid: dbus-daemon-lau [ 8703] } hitcount: 49 count: 47739
+ { common_pid: irqbalance [ 1252] } hitcount: 27 count: 27648
+ { common_pid: 01ifupdown [ 8705] } hitcount: 3 count: 17216
+ { common_pid: dbus-daemon [ 772] } hitcount: 10 count: 12396
+ { common_pid: Socket Thread [ 8342] } hitcount: 11 count: 11264
+ { common_pid: nm-dhcp-client. [ 8701] } hitcount: 6 count: 7424
+ { common_pid: gmain [ 1315] } hitcount: 18 count: 6336
+ .
+ .
+ .
+ { common_pid: postgres [ 1892] } hitcount: 2 count: 32
+ { common_pid: postgres [ 1891] } hitcount: 2 count: 32
+ { common_pid: gmain [ 8704] } hitcount: 2 count: 32
+ { common_pid: upstart-dbus-br [ 2740] } hitcount: 21 count: 21
+ { common_pid: nm-dispatcher.a [ 8696] } hitcount: 1 count: 16
+ { common_pid: indicator-datet [ 2904] } hitcount: 1 count: 16
+ { common_pid: gdbus [ 2998] } hitcount: 1 count: 16
+ { common_pid: rtkit-daemon [ 2052] } hitcount: 1 count: 8
+ { common_pid: init [ 1] } hitcount: 2 count: 2
+
+ Totals:
+ Hits: 2116
+ Entries: 51
+ Dropped: 0
+
+ Similarly, if you key a hist trigger on syscall id, for example to
+ gather and display a list of systemwide syscall hits, you can use
+ the special .syscall modifier to display the syscall names rather
+ than raw ids. The example below keeps a running total of syscall
+ counts for the system during the run:
+
+ # echo 'hist:key=id.syscall:val=hitcount' > \
+ /sys/kernel/debug/tracing/events/raw_syscalls/sys_enter/trigger
+
+ # cat /sys/kernel/debug/tracing/events/raw_syscalls/sys_enter/hist
+ # trigger info: hist:keys=id.syscall:vals=hitcount:sort=hitcount:size=2048 [active]
+
+ { id: sys_fsync [ 74] } hitcount: 1
+ { id: sys_newuname [ 63] } hitcount: 1
+ { id: sys_prctl [157] } hitcount: 1
+ { id: sys_statfs [137] } hitcount: 1
+ { id: sys_symlink [ 88] } hitcount: 1
+ { id: sys_sendmmsg [307] } hitcount: 1
+ { id: sys_semctl [ 66] } hitcount: 1
+ { id: sys_readlink [ 89] } hitcount: 3
+ { id: sys_bind [ 49] } hitcount: 3
+ { id: sys_getsockname [ 51] } hitcount: 3
+ { id: sys_unlink [ 87] } hitcount: 3
+ { id: sys_rename [ 82] } hitcount: 4
+ { id: unknown_syscall [ 58] } hitcount: 4
+ { id: sys_connect [ 42] } hitcount: 4
+ { id: sys_getpid [ 39] } hitcount: 4
+ .
+ .
+ .
+ { id: sys_rt_sigprocmask [ 14] } hitcount: 952
+ { id: sys_futex [202] } hitcount: 1534
+ { id: sys_write [ 1] } hitcount: 2689
+ { id: sys_setitimer [ 38] } hitcount: 2797
+ { id: sys_read [ 0] } hitcount: 3202
+ { id: sys_select [ 23] } hitcount: 3773
+ { id: sys_writev [ 20] } hitcount: 4531
+ { id: sys_poll [ 7] } hitcount: 8314
+ { id: sys_recvmsg [ 47] } hitcount: 13738
+ { id: sys_ioctl [ 16] } hitcount: 21843
+
+ Totals:
+ Hits: 67612
+ Entries: 72
+ Dropped: 0
+
+ The syscall counts above provide a rough overall picture of system
+ call activity on the system; we can see for example that the most
+ popular system call on this system was the 'sys_ioctl' system call.
+
+ We can use 'compound' keys to refine that number and provide some
+ further insight as to which processes exactly contribute to the
+ overall ioctl count.
+
+ The command below keeps a hitcount for every unique combination of
+ system call id and pid - the end result is essentially a table
+ that keeps a per-pid sum of system call hits. The results are
+ sorted using the system call id as the primary key, and the
+ hitcount sum as the secondary key:
+
+ # echo 'hist:key=id.syscall,common_pid.execname:val=hitcount:sort=id,hitcount' > \
+ /sys/kernel/debug/tracing/events/raw_syscalls/sys_enter/trigger
+
+ # cat /sys/kernel/debug/tracing/events/raw_syscalls/sys_enter/hist
+ # trigger info: hist:keys=id.syscall,common_pid.execname:vals=hitcount:sort=id.syscall,hitcount:size=2048 [active]
+
+ { id: sys_read [ 0], common_pid: rtkit-daemon [ 1877] } hitcount: 1
+ { id: sys_read [ 0], common_pid: gdbus [ 2976] } hitcount: 1
+ { id: sys_read [ 0], common_pid: console-kit-dae [ 3400] } hitcount: 1
+ { id: sys_read [ 0], common_pid: postgres [ 1865] } hitcount: 1
+ { id: sys_read [ 0], common_pid: deja-dup-monito [ 3543] } hitcount: 2
+ { id: sys_read [ 0], common_pid: NetworkManager [ 890] } hitcount: 2
+ { id: sys_read [ 0], common_pid: evolution-calen [ 3048] } hitcount: 2
+ { id: sys_read [ 0], common_pid: postgres [ 1864] } hitcount: 2
+ { id: sys_read [ 0], common_pid: nm-applet [ 3022] } hitcount: 2
+ { id: sys_read [ 0], common_pid: whoopsie [ 1212] } hitcount: 2
+ .
+ .
+ .
+ { id: sys_ioctl [ 16], common_pid: bash [ 8479] } hitcount: 1
+ { id: sys_ioctl [ 16], common_pid: bash [ 3472] } hitcount: 12
+ { id: sys_ioctl [ 16], common_pid: gnome-terminal [ 3199] } hitcount: 16
+ { id: sys_ioctl [ 16], common_pid: Xorg [ 1267] } hitcount: 1808
+ { id: sys_ioctl [ 16], common_pid: compiz [ 2994] } hitcount: 5580
+ .
+ .
+ .
+ { id: sys_waitid [247], common_pid: upstart-dbus-br [ 2690] } hitcount: 3
+ { id: sys_waitid [247], common_pid: upstart-dbus-br [ 2688] } hitcount: 16
+ { id: sys_inotify_add_watch [254], common_pid: gmain [ 975] } hitcount: 2
+ { id: sys_inotify_add_watch [254], common_pid: gmain [ 3204] } hitcount: 4
+ { id: sys_inotify_add_watch [254], common_pid: gmain [ 2888] } hitcount: 4
+ { id: sys_inotify_add_watch [254], common_pid: gmain [ 3003] } hitcount: 4
+ { id: sys_inotify_add_watch [254], common_pid: gmain [ 2873] } hitcount: 4
+ { id: sys_inotify_add_watch [254], common_pid: gmain [ 3196] } hitcount: 6
+ { id: sys_openat [257], common_pid: java [ 2623] } hitcount: 2
+ { id: sys_eventfd2 [290], common_pid: ibus-ui-gtk3 [ 2760] } hitcount: 4
+ { id: sys_eventfd2 [290], common_pid: compiz [ 2994] } hitcount: 6
+
+ Totals:
+ Hits: 31536
+ Entries: 323
+ Dropped: 0
+
+ The above list does give us a breakdown of the ioctl syscall by
+ pid, but it also gives us quite a bit more than that, which we
+ don't really care about at the moment. Since we know the syscall
+ id for sys_ioctl (16, displayed next to the sys_ioctl name), we
+ can use that to filter out all the other syscalls:
+
+ # echo 'hist:key=id.syscall,common_pid.execname:val=hitcount:sort=id,hitcount if id == 16' > \
+ /sys/kernel/debug/tracing/events/raw_syscalls/sys_enter/trigger
+
+ # cat /sys/kernel/debug/tracing/events/raw_syscalls/sys_enter/hist
+ # trigger info: hist:keys=id.syscall,common_pid.execname:vals=hitcount:sort=id.syscall,hitcount:size=2048 if id == 16 [active]
+
+ { id: sys_ioctl [ 16], common_pid: gmain [ 2769] } hitcount: 1
+ { id: sys_ioctl [ 16], common_pid: evolution-addre [ 8571] } hitcount: 1
+ { id: sys_ioctl [ 16], common_pid: gmain [ 3003] } hitcount: 1
+ { id: sys_ioctl [ 16], common_pid: gmain [ 2781] } hitcount: 1
+ { id: sys_ioctl [ 16], common_pid: gmain [ 2829] } hitcount: 1
+ { id: sys_ioctl [ 16], common_pid: bash [ 8726] } hitcount: 1
+ { id: sys_ioctl [ 16], common_pid: bash [ 8508] } hitcount: 1
+ { id: sys_ioctl [ 16], common_pid: gmain [ 2970] } hitcount: 1
+ { id: sys_ioctl [ 16], common_pid: gmain [ 2768] } hitcount: 1
+ .
+ .
+ .
+ { id: sys_ioctl [ 16], common_pid: pool [ 8559] } hitcount: 45
+ { id: sys_ioctl [ 16], common_pid: pool [ 8555] } hitcount: 48
+ { id: sys_ioctl [ 16], common_pid: pool [ 8551] } hitcount: 48
+ { id: sys_ioctl [ 16], common_pid: avahi-daemon [ 896] } hitcount: 66
+ { id: sys_ioctl [ 16], common_pid: Xorg [ 1267] } hitcount: 26674
+ { id: sys_ioctl [ 16], common_pid: compiz [ 2994] } hitcount: 73443
+
+ Totals:
+ Hits: 101162
+ Entries: 103
+ Dropped: 0
+
+ The above output shows that 'compiz' and 'Xorg' are far and away
+ the heaviest ioctl callers (which might lead to questions about
+ whether they really need to be making all those calls and to
+ possible avenues for further investigation.)
+
+ The compound key examples used a key and a sum value (hitcount) to
+ sort the output, but we can just as easily use two keys instead.
+ Here's an example where we use a compound key composed of the the
+ common_pid and size event fields. Sorting with pid as the primary
+ key and 'size' as the secondary key allows us to display an
+ ordered summary of the recvfrom sizes, with counts, received by
+ each process:
+
+ # echo 'hist:key=common_pid.execname,size:val=hitcount:sort=common_pid,size' > \
+ /sys/kernel/debug/tracing/events/syscalls/sys_enter_recvfrom/trigger
+
+ # cat /sys/kernel/debug/tracing/events/syscalls/sys_enter_recvfrom/hist
+ # trigger info: hist:keys=common_pid.execname,size:vals=hitcount:sort=common_pid.execname,size:size=2048 [active]
+
+ { common_pid: smbd [ 784], size: 4 } hitcount: 1
+ { common_pid: dnsmasq [ 1412], size: 4096 } hitcount: 672
+ { common_pid: postgres [ 1796], size: 1000 } hitcount: 6
+ { common_pid: postgres [ 1867], size: 1000 } hitcount: 10
+ { common_pid: bamfdaemon [ 2787], size: 28 } hitcount: 2
+ { common_pid: bamfdaemon [ 2787], size: 14360 } hitcount: 1
+ { common_pid: compiz [ 2994], size: 8 } hitcount: 1
+ { common_pid: compiz [ 2994], size: 20 } hitcount: 11
+ { common_pid: gnome-terminal [ 3199], size: 4 } hitcount: 2
+ { common_pid: firefox [ 8817], size: 4 } hitcount: 1
+ { common_pid: firefox [ 8817], size: 8 } hitcount: 5
+ { common_pid: firefox [ 8817], size: 588 } hitcount: 2
+ { common_pid: firefox [ 8817], size: 628 } hitcount: 1
+ { common_pid: firefox [ 8817], size: 6944 } hitcount: 1
+ { common_pid: firefox [ 8817], size: 408880 } hitcount: 2
+ { common_pid: firefox [ 8822], size: 8 } hitcount: 2
+ { common_pid: firefox [ 8822], size: 160 } hitcount: 2
+ { common_pid: firefox [ 8822], size: 320 } hitcount: 2
+ { common_pid: firefox [ 8822], size: 352 } hitcount: 1
+ .
+ .
+ .
+ { common_pid: pool [ 8923], size: 1960 } hitcount: 10
+ { common_pid: pool [ 8923], size: 2048 } hitcount: 10
+ { common_pid: pool [ 8924], size: 1960 } hitcount: 10
+ { common_pid: pool [ 8924], size: 2048 } hitcount: 10
+ { common_pid: pool [ 8928], size: 1964 } hitcount: 4
+ { common_pid: pool [ 8928], size: 1965 } hitcount: 2
+ { common_pid: pool [ 8928], size: 2048 } hitcount: 6
+ { common_pid: pool [ 8929], size: 1982 } hitcount: 1
+ { common_pid: pool [ 8929], size: 2048 } hitcount: 1
+
+ Totals:
+ Hits: 2016
+ Entries: 224
+ Dropped: 0
+
+ The above example also illustrates the fact that although a compound
+ key is treated as a single entity for hashing purposes, the sub-keys
+ it's composed of can be accessed independently.
+
+ The next example uses a string field as the hash key and
+ demonstrates how you can manually pause and continue a hist trigger.
+ In this example, we'll aggregate fork counts and don't expect a
+ large number of entries in the hash table, so we'll drop it to a
+ much smaller number, say 256:
+
+ # echo 'hist:key=child_comm:val=hitcount:size=256' > \
+ /sys/kernel/debug/tracing/events/sched/sched_process_fork/trigger
+
+ # cat /sys/kernel/debug/tracing/events/sched/sched_process_fork/hist
+ # trigger info: hist:keys=child_comm:vals=hitcount:sort=hitcount:size=256 [active]
+
+ { child_comm: dconf worker } hitcount: 1
+ { child_comm: ibus-daemon } hitcount: 1
+ { child_comm: whoopsie } hitcount: 1
+ { child_comm: smbd } hitcount: 1
+ { child_comm: gdbus } hitcount: 1
+ { child_comm: kthreadd } hitcount: 1
+ { child_comm: dconf worker } hitcount: 1
+ { child_comm: evolution-alarm } hitcount: 2
+ { child_comm: Socket Thread } hitcount: 2
+ { child_comm: postgres } hitcount: 2
+ { child_comm: bash } hitcount: 3
+ { child_comm: compiz } hitcount: 3
+ { child_comm: evolution-sourc } hitcount: 4
+ { child_comm: dhclient } hitcount: 4
+ { child_comm: pool } hitcount: 5
+ { child_comm: nm-dispatcher.a } hitcount: 8
+ { child_comm: firefox } hitcount: 8
+ { child_comm: dbus-daemon } hitcount: 8
+ { child_comm: glib-pacrunner } hitcount: 10
+ { child_comm: evolution } hitcount: 23
+
+ Totals:
+ Hits: 89
+ Entries: 20
+ Dropped: 0
+
+ If we want to pause the hist trigger, we can simply append :pause to
+ the command that started the trigger. Notice that the trigger info
+ displays as [paused]:
+
+ # echo 'hist:key=child_comm:val=hitcount:size=256:pause' >> \
+ /sys/kernel/debug/tracing/events/sched/sched_process_fork/trigger
+
+ # cat /sys/kernel/debug/tracing/events/sched/sched_process_fork/hist
+ # trigger info: hist:keys=child_comm:vals=hitcount:sort=hitcount:size=256 [paused]
+
+ { child_comm: dconf worker } hitcount: 1
+ { child_comm: kthreadd } hitcount: 1
+ { child_comm: dconf worker } hitcount: 1
+ { child_comm: gdbus } hitcount: 1
+ { child_comm: ibus-daemon } hitcount: 1
+ { child_comm: Socket Thread } hitcount: 2
+ { child_comm: evolution-alarm } hitcount: 2
+ { child_comm: smbd } hitcount: 2
+ { child_comm: bash } hitcount: 3
+ { child_comm: whoopsie } hitcount: 3
+ { child_comm: compiz } hitcount: 3
+ { child_comm: evolution-sourc } hitcount: 4
+ { child_comm: pool } hitcount: 5
+ { child_comm: postgres } hitcount: 6
+ { child_comm: firefox } hitcount: 8
+ { child_comm: dhclient } hitcount: 10
+ { child_comm: emacs } hitcount: 12
+ { child_comm: dbus-daemon } hitcount: 20
+ { child_comm: nm-dispatcher.a } hitcount: 20
+ { child_comm: evolution } hitcount: 35
+ { child_comm: glib-pacrunner } hitcount: 59
+
+ Totals:
+ Hits: 199
+ Entries: 21
+ Dropped: 0
+
+ To manually continue having the trigger aggregate events, append
+ :cont instead. Notice that the trigger info displays as [active]
+ again, and the data has changed:
+
+ # echo 'hist:key=child_comm:val=hitcount:size=256:cont' >> \
+ /sys/kernel/debug/tracing/events/sched/sched_process_fork/trigger
+
+ # cat /sys/kernel/debug/tracing/events/sched/sched_process_fork/hist
+ # trigger info: hist:keys=child_comm:vals=hitcount:sort=hitcount:size=256 [active]
+
+ { child_comm: dconf worker } hitcount: 1
+ { child_comm: dconf worker } hitcount: 1
+ { child_comm: kthreadd } hitcount: 1
+ { child_comm: gdbus } hitcount: 1
+ { child_comm: ibus-daemon } hitcount: 1
+ { child_comm: Socket Thread } hitcount: 2
+ { child_comm: evolution-alarm } hitcount: 2
+ { child_comm: smbd } hitcount: 2
+ { child_comm: whoopsie } hitcount: 3
+ { child_comm: compiz } hitcount: 3
+ { child_comm: evolution-sourc } hitcount: 4
+ { child_comm: bash } hitcount: 5
+ { child_comm: pool } hitcount: 5
+ { child_comm: postgres } hitcount: 6
+ { child_comm: firefox } hitcount: 8
+ { child_comm: dhclient } hitcount: 11
+ { child_comm: emacs } hitcount: 12
+ { child_comm: dbus-daemon } hitcount: 22
+ { child_comm: nm-dispatcher.a } hitcount: 22
+ { child_comm: evolution } hitcount: 35
+ { child_comm: glib-pacrunner } hitcount: 59
+
+ Totals:
+ Hits: 206
+ Entries: 21
+ Dropped: 0
+
+ The previous example showed how to start and stop a hist trigger by
+ appending 'pause' and 'continue' to the hist trigger command. A
+ hist trigger can also be started in a paused state by initially
+ starting the trigger with ':pause' appended. This allows you to
+ start the trigger only when you're ready to start collecting data
+ and not before. For example, you could start the trigger in a
+ paused state, then unpause it and do something you want to measure,
+ then pause the trigger again when done.
+
+ Of course, doing this manually can be difficult and error-prone, but
+ it is possible to automatically start and stop a hist trigger based
+ on some condition, via the enable_hist and disable_hist triggers.
+
+ For example, suppose we wanted to take a look at the relative
+ weights in terms of skb length for each callpath that leads to a
+ netif_receieve_skb event when downloading a decent-sized file using
+ wget.
+
+ First we set up an initially paused stacktrace trigger on the
+ netif_receive_skb event:
+
+ # echo 'hist:key=stacktrace:vals=len:pause' > \
+ /sys/kernel/debug/tracing/events/net/netif_receive_skb/trigger
+
+ Next, we set up an 'enable_hist' trigger on the sched_process_exec
+ event, with an 'if filename==/usr/bin/wget' filter. The effect of
+ this new trigger is that it will 'unpause' the hist trigger we just
+ set up on netif_receive_skb if and only if it sees a
+ sched_process_exec event with a filename of '/usr/bin/wget'. When
+ that happens, all netif_receive_skb events are aggregated into a
+ hash table keyed on stacktrace:
+
+ # echo 'enable_hist:net:netif_receive_skb if filename==/usr/bin/wget' > \
+ /sys/kernel/debug/tracing/events/sched/sched_process_exec/trigger
+
+ The aggregation continues until the netif_receive_skb is paused
+ again, which is what the following disable_hist event does by
+ creating a similar setup on the sched_process_exit event, using the
+ filter 'comm==wget':
+
+ # echo 'disable_hist:net:netif_receive_skb if comm==wget' > \
+ /sys/kernel/debug/tracing/events/sched/sched_process_exit/trigger
+
+ Whenever a process exits and the comm field of the disable_hist
+ trigger filter matches 'comm==wget', the netif_receive_skb hist
+ trigger is disabled.
+
+ The overall effect is that netif_receive_skb events are aggregated
+ into the hash table for only the duration of the wget. Executing a
+ wget command and then listing the 'hist' file will display the
+ output generated by the wget command:
+
+ $ wget https://www.kernel.org/pub/linux/kernel/v3.x/patch-3.19.xz
+
+ # cat /sys/kernel/debug/tracing/events/net/netif_receive_skb/hist
+ # trigger info: hist:keys=stacktrace:vals=len:sort=hitcount:size=2048 [paused]
+
+ { stacktrace:
+ __netif_receive_skb_core+0x46d/0x990
+ __netif_receive_skb+0x18/0x60
+ netif_receive_skb_internal+0x23/0x90
+ napi_gro_receive+0xc8/0x100
+ ieee80211_deliver_skb+0xd6/0x270 [mac80211]
+ ieee80211_rx_handlers+0xccf/0x22f0 [mac80211]
+ ieee80211_prepare_and_rx_handle+0x4e7/0xc40 [mac80211]
+ ieee80211_rx+0x31d/0x900 [mac80211]
+ iwlagn_rx_reply_rx+0x3db/0x6f0 [iwldvm]
+ iwl_rx_dispatch+0x8e/0xf0 [iwldvm]
+ iwl_pcie_irq_handler+0xe3c/0x12f0 [iwlwifi]
+ irq_thread_fn+0x20/0x50
+ irq_thread+0x11f/0x150
+ kthread+0xd2/0xf0
+ ret_from_fork+0x42/0x70
+ } hitcount: 85 len: 28884
+ { stacktrace:
+ __netif_receive_skb_core+0x46d/0x990
+ __netif_receive_skb+0x18/0x60
+ netif_receive_skb_internal+0x23/0x90
+ napi_gro_complete+0xa4/0xe0
+ dev_gro_receive+0x23a/0x360
+ napi_gro_receive+0x30/0x100
+ ieee80211_deliver_skb+0xd6/0x270 [mac80211]
+ ieee80211_rx_handlers+0xccf/0x22f0 [mac80211]
+ ieee80211_prepare_and_rx_handle+0x4e7/0xc40 [mac80211]
+ ieee80211_rx+0x31d/0x900 [mac80211]
+ iwlagn_rx_reply_rx+0x3db/0x6f0 [iwldvm]
+ iwl_rx_dispatch+0x8e/0xf0 [iwldvm]
+ iwl_pcie_irq_handler+0xe3c/0x12f0 [iwlwifi]
+ irq_thread_fn+0x20/0x50
+ irq_thread+0x11f/0x150
+ kthread+0xd2/0xf0
+ } hitcount: 98 len: 664329
+ { stacktrace:
+ __netif_receive_skb_core+0x46d/0x990
+ __netif_receive_skb+0x18/0x60
+ process_backlog+0xa8/0x150
+ net_rx_action+0x15d/0x340
+ __do_softirq+0x114/0x2c0
+ do_softirq_own_stack+0x1c/0x30
+ do_softirq+0x65/0x70
+ __local_bh_enable_ip+0xb5/0xc0
+ ip_finish_output+0x1f4/0x840
+ ip_output+0x6b/0xc0
+ ip_local_out_sk+0x31/0x40
+ ip_send_skb+0x1a/0x50
+ udp_send_skb+0x173/0x2a0
+ udp_sendmsg+0x2bf/0x9f0
+ inet_sendmsg+0x64/0xa0
+ sock_sendmsg+0x3d/0x50
+ } hitcount: 115 len: 13030
+ { stacktrace:
+ __netif_receive_skb_core+0x46d/0x990
+ __netif_receive_skb+0x18/0x60
+ netif_receive_skb_internal+0x23/0x90
+ napi_gro_complete+0xa4/0xe0
+ napi_gro_flush+0x6d/0x90
+ iwl_pcie_irq_handler+0x92a/0x12f0 [iwlwifi]
+ irq_thread_fn+0x20/0x50
+ irq_thread+0x11f/0x150
+ kthread+0xd2/0xf0
+ ret_from_fork+0x42/0x70
+ } hitcount: 934 len: 5512212
+
+ Totals:
+ Hits: 1232
+ Entries: 4
+ Dropped: 0
+
+ The above shows all the netif_receive_skb callpaths and their total
+ lengths for the duration of the wget command.
+
+ The 'clear' hist trigger param can be used to clear the hash table.
+ Suppose we wanted to try another run of the previous example but
+ this time also wanted to see the complete list of events that went
+ into the histogram. In order to avoid having to set everything up
+ again, we can just clear the histogram first:
+
+ # echo 'hist:key=stacktrace:vals=len:clear' >> \
+ /sys/kernel/debug/tracing/events/net/netif_receive_skb/trigger
+
+ Just to verify that it is in fact cleared, here's what we now see in
+ the hist file:
+
+ # cat /sys/kernel/debug/tracing/events/net/netif_receive_skb/hist
+ # trigger info: hist:keys=stacktrace:vals=len:sort=hitcount:size=2048 [paused]
+
+ Totals:
+ Hits: 0
+ Entries: 0
+ Dropped: 0
+
+ Since we want to see the detailed list of every netif_receive_skb
+ event occurring during the new run, which are in fact the same
+ events being aggregated into the hash table, we add some additional
+ 'enable_event' events to the triggering sched_process_exec and
+ sched_process_exit events as such:
+
+ # echo 'enable_event:net:netif_receive_skb if filename==/usr/bin/wget' > \
+ /sys/kernel/debug/tracing/events/sched/sched_process_exec/trigger
+
+ # echo 'disable_event:net:netif_receive_skb if comm==wget' > \
+ /sys/kernel/debug/tracing/events/sched/sched_process_exit/trigger
+
+ If you read the trigger files for the sched_process_exec and
+ sched_process_exit triggers, you should see two triggers for each:
+ one enabling/disabling the hist aggregation and the other
+ enabling/disabling the logging of events:
+
+ # cat /sys/kernel/debug/tracing/events/sched/sched_process_exec/trigger
+ enable_event:net:netif_receive_skb:unlimited if filename==/usr/bin/wget
+ enable_hist:net:netif_receive_skb:unlimited if filename==/usr/bin/wget
+
+ # cat /sys/kernel/debug/tracing/events/sched/sched_process_exit/trigger
+ enable_event:net:netif_receive_skb:unlimited if comm==wget
+ disable_hist:net:netif_receive_skb:unlimited if comm==wget
+
+ In other words, whenever either of the sched_process_exec or
+ sched_process_exit events is hit and matches 'wget', it enables or
+ disables both the histogram and the event log, and what you end up
+ with is a hash table and set of events just covering the specified
+ duration. Run the wget command again:
+
+ $ wget https://www.kernel.org/pub/linux/kernel/v3.x/patch-3.19.xz
+
+ Displaying the 'hist' file should show something similar to what you
+ saw in the last run, but this time you should also see the
+ individual events in the trace file:
+
+ # cat /sys/kernel/debug/tracing/trace
+
+ # tracer: nop
+ #
+ # entries-in-buffer/entries-written: 183/1426 #P:4
+ #
+ # _-----=> irqs-off
+ # / _----=> need-resched
+ # | / _---=> hardirq/softirq
+ # || / _--=> preempt-depth
+ # ||| / delay
+ # TASK-PID CPU# |||| TIMESTAMP FUNCTION
+ # | | | |||| | |
+ wget-15108 [000] ..s1 31769.606929: netif_receive_skb: dev=lo skbaddr=ffff88009c353100 len=60
+ wget-15108 [000] ..s1 31769.606999: netif_receive_skb: dev=lo skbaddr=ffff88009c353200 len=60
+ dnsmasq-1382 [000] ..s1 31769.677652: netif_receive_skb: dev=lo skbaddr=ffff88009c352b00 len=130
+ dnsmasq-1382 [000] ..s1 31769.685917: netif_receive_skb: dev=lo skbaddr=ffff88009c352200 len=138
+ ##### CPU 2 buffer started ####
+ irq/29-iwlwifi-559 [002] ..s. 31772.031529: netif_receive_skb: dev=wlan0 skbaddr=ffff88009d433d00 len=2948
+ irq/29-iwlwifi-559 [002] ..s. 31772.031572: netif_receive_skb: dev=wlan0 skbaddr=ffff88009d432200 len=1500
+ irq/29-iwlwifi-559 [002] ..s. 31772.032196: netif_receive_skb: dev=wlan0 skbaddr=ffff88009d433100 len=2948
+ irq/29-iwlwifi-559 [002] ..s. 31772.032761: netif_receive_skb: dev=wlan0 skbaddr=ffff88009d433000 len=2948
+ irq/29-iwlwifi-559 [002] ..s. 31772.033220: netif_receive_skb: dev=wlan0 skbaddr=ffff88009d432e00 len=1500
+ .
+ .
+ .
+
+ The following example demonstrates how multiple hist triggers can be
+ attached to a given event. This capability can be useful for
+ creating a set of different summaries derived from the same set of
+ events, or for comparing the effects of different filters, among
+ other things.
+
+ # echo 'hist:keys=skbaddr.hex:vals=len if len < 0' >> \
+ /sys/kernel/debug/tracing/events/net/netif_receive_skb/trigger
+ # echo 'hist:keys=skbaddr.hex:vals=len if len > 4096' >> \
+ /sys/kernel/debug/tracing/events/net/netif_receive_skb/trigger
+ # echo 'hist:keys=skbaddr.hex:vals=len if len == 256' >> \
+ /sys/kernel/debug/tracing/events/net/netif_receive_skb/trigger
+ # echo 'hist:keys=skbaddr.hex:vals=len' >> \
+ /sys/kernel/debug/tracing/events/net/netif_receive_skb/trigger
+ # echo 'hist:keys=len:vals=common_preempt_count' >> \
+ /sys/kernel/debug/tracing/events/net/netif_receive_skb/trigger
+
+ The above set of commands create four triggers differing only in
+ their filters, along with a completely different though fairly
+ nonsensical trigger. Note that in order to append multiple hist
+ triggers to the same file, you should use the '>>' operator to
+ append them ('>' will also add the new hist trigger, but will remove
+ any existing hist triggers beforehand).
+
+ Displaying the contents of the 'hist' file for the event shows the
+ contents of all five histograms:
+
+ # cat /sys/kernel/debug/tracing/events/net/netif_receive_skb/hist
+
+ # event histogram
+ #
+ # trigger info: hist:keys=len:vals=hitcount,common_preempt_count:sort=hitcount:size=2048 [active]
+ #
+
+ { len: 176 } hitcount: 1 common_preempt_count: 0
+ { len: 223 } hitcount: 1 common_preempt_count: 0
+ { len: 4854 } hitcount: 1 common_preempt_count: 0
+ { len: 395 } hitcount: 1 common_preempt_count: 0
+ { len: 177 } hitcount: 1 common_preempt_count: 0
+ { len: 446 } hitcount: 1 common_preempt_count: 0
+ { len: 1601 } hitcount: 1 common_preempt_count: 0
+ .
+ .
+ .
+ { len: 1280 } hitcount: 66 common_preempt_count: 0
+ { len: 116 } hitcount: 81 common_preempt_count: 40
+ { len: 708 } hitcount: 112 common_preempt_count: 0
+ { len: 46 } hitcount: 221 common_preempt_count: 0
+ { len: 1264 } hitcount: 458 common_preempt_count: 0
+
+ Totals:
+ Hits: 1428
+ Entries: 147
+ Dropped: 0
+
+
+ # event histogram
+ #
+ # trigger info: hist:keys=skbaddr.hex:vals=hitcount,len:sort=hitcount:size=2048 [active]
+ #
+
+ { skbaddr: ffff8800baee5e00 } hitcount: 1 len: 130
+ { skbaddr: ffff88005f3d5600 } hitcount: 1 len: 1280
+ { skbaddr: ffff88005f3d4900 } hitcount: 1 len: 1280
+ { skbaddr: ffff88009fed6300 } hitcount: 1 len: 115
+ { skbaddr: ffff88009fe0ad00 } hitcount: 1 len: 115
+ { skbaddr: ffff88008cdb1900 } hitcount: 1 len: 46
+ { skbaddr: ffff880064b5ef00 } hitcount: 1 len: 118
+ { skbaddr: ffff880044e3c700 } hitcount: 1 len: 60
+ { skbaddr: ffff880100065900 } hitcount: 1 len: 46
+ { skbaddr: ffff8800d46bd500 } hitcount: 1 len: 116
+ { skbaddr: ffff88005f3d5f00 } hitcount: 1 len: 1280
+ { skbaddr: ffff880100064700 } hitcount: 1 len: 365
+ { skbaddr: ffff8800badb6f00 } hitcount: 1 len: 60
+ .
+ .
+ .
+ { skbaddr: ffff88009fe0be00 } hitcount: 27 len: 24677
+ { skbaddr: ffff88009fe0a400 } hitcount: 27 len: 23052
+ { skbaddr: ffff88009fe0b700 } hitcount: 31 len: 25589
+ { skbaddr: ffff88009fe0b600 } hitcount: 32 len: 27326
+ { skbaddr: ffff88006a462800 } hitcount: 68 len: 71678
+ { skbaddr: ffff88006a463700 } hitcount: 70 len: 72678
+ { skbaddr: ffff88006a462b00 } hitcount: 71 len: 77589
+ { skbaddr: ffff88006a463600 } hitcount: 73 len: 71307
+ { skbaddr: ffff88006a462200 } hitcount: 81 len: 81032
+
+ Totals:
+ Hits: 1451
+ Entries: 318
+ Dropped: 0
+
+
+ # event histogram
+ #
+ # trigger info: hist:keys=skbaddr.hex:vals=hitcount,len:sort=hitcount:size=2048 if len == 256 [active]
+ #
+
+
+ Totals:
+ Hits: 0
+ Entries: 0
+ Dropped: 0
+
+
+ # event histogram
+ #
+ # trigger info: hist:keys=skbaddr.hex:vals=hitcount,len:sort=hitcount:size=2048 if len > 4096 [active]
+ #
+
+ { skbaddr: ffff88009fd2c300 } hitcount: 1 len: 7212
+ { skbaddr: ffff8800d2bcce00 } hitcount: 1 len: 7212
+ { skbaddr: ffff8800d2bcd700 } hitcount: 1 len: 7212
+ { skbaddr: ffff8800d2bcda00 } hitcount: 1 len: 21492
+ { skbaddr: ffff8800ae2e2d00 } hitcount: 1 len: 7212
+ { skbaddr: ffff8800d2bcdb00 } hitcount: 1 len: 7212
+ { skbaddr: ffff88006a4df500 } hitcount: 1 len: 4854
+ { skbaddr: ffff88008ce47b00 } hitcount: 1 len: 18636
+ { skbaddr: ffff8800ae2e2200 } hitcount: 1 len: 12924
+ { skbaddr: ffff88005f3e1000 } hitcount: 1 len: 4356
+ { skbaddr: ffff8800d2bcdc00 } hitcount: 2 len: 24420
+ { skbaddr: ffff8800d2bcc200 } hitcount: 2 len: 12996
+
+ Totals:
+ Hits: 14
+ Entries: 12
+ Dropped: 0
+
+
+ # event histogram
+ #
+ # trigger info: hist:keys=skbaddr.hex:vals=hitcount,len:sort=hitcount:size=2048 if len < 0 [active]
+ #
+
+
+ Totals:
+ Hits: 0
+ Entries: 0
+ Dropped: 0
+
+ Named triggers can be used to have triggers share a common set of
+ histogram data. This capability is mostly useful for combining the
+ output of events generated by tracepoints contained inside inline
+ functions, but names can be used in a hist trigger on any event.
+ For example, these two triggers when hit will update the same 'len'
+ field in the shared 'foo' histogram data:
+
+ # echo 'hist:name=foo:keys=skbaddr.hex:vals=len' > \
+ /sys/kernel/debug/tracing/events/net/netif_receive_skb/trigger
+ # echo 'hist:name=foo:keys=skbaddr.hex:vals=len' > \
+ /sys/kernel/debug/tracing/events/net/netif_rx/trigger
+
+ You can see that they're updating common histogram data by reading
+ each event's hist files at the same time:
+
+ # cat /sys/kernel/debug/tracing/events/net/netif_receive_skb/hist;
+ cat /sys/kernel/debug/tracing/events/net/netif_rx/hist
+
+ # event histogram
+ #
+ # trigger info: hist:name=foo:keys=skbaddr.hex:vals=hitcount,len:sort=hitcount:size=2048 [active]
+ #
+
+ { skbaddr: ffff88000ad53500 } hitcount: 1 len: 46
+ { skbaddr: ffff8800af5a1500 } hitcount: 1 len: 76
+ { skbaddr: ffff8800d62a1900 } hitcount: 1 len: 46
+ { skbaddr: ffff8800d2bccb00 } hitcount: 1 len: 468
+ { skbaddr: ffff8800d3c69900 } hitcount: 1 len: 46
+ { skbaddr: ffff88009ff09100 } hitcount: 1 len: 52
+ { skbaddr: ffff88010f13ab00 } hitcount: 1 len: 168
+ { skbaddr: ffff88006a54f400 } hitcount: 1 len: 46
+ { skbaddr: ffff8800d2bcc500 } hitcount: 1 len: 260
+ { skbaddr: ffff880064505000 } hitcount: 1 len: 46
+ { skbaddr: ffff8800baf24e00 } hitcount: 1 len: 32
+ { skbaddr: ffff88009fe0ad00 } hitcount: 1 len: 46
+ { skbaddr: ffff8800d3edff00 } hitcount: 1 len: 44
+ { skbaddr: ffff88009fe0b400 } hitcount: 1 len: 168
+ { skbaddr: ffff8800a1c55a00 } hitcount: 1 len: 40
+ { skbaddr: ffff8800d2bcd100 } hitcount: 1 len: 40
+ { skbaddr: ffff880064505f00 } hitcount: 1 len: 174
+ { skbaddr: ffff8800a8bff200 } hitcount: 1 len: 160
+ { skbaddr: ffff880044e3cc00 } hitcount: 1 len: 76
+ { skbaddr: ffff8800a8bfe700 } hitcount: 1 len: 46
+ { skbaddr: ffff8800d2bcdc00 } hitcount: 1 len: 32
+ { skbaddr: ffff8800a1f64800 } hitcount: 1 len: 46
+ { skbaddr: ffff8800d2bcde00 } hitcount: 1 len: 988
+ { skbaddr: ffff88006a5dea00 } hitcount: 1 len: 46
+ { skbaddr: ffff88002e37a200 } hitcount: 1 len: 44
+ { skbaddr: ffff8800a1f32c00 } hitcount: 2 len: 676
+ { skbaddr: ffff88000ad52600 } hitcount: 2 len: 107
+ { skbaddr: ffff8800a1f91e00 } hitcount: 2 len: 92
+ { skbaddr: ffff8800af5a0200 } hitcount: 2 len: 142
+ { skbaddr: ffff8800d2bcc600 } hitcount: 2 len: 220
+ { skbaddr: ffff8800ba36f500 } hitcount: 2 len: 92
+ { skbaddr: ffff8800d021f800 } hitcount: 2 len: 92
+ { skbaddr: ffff8800a1f33600 } hitcount: 2 len: 675
+ { skbaddr: ffff8800a8bfff00 } hitcount: 3 len: 138
+ { skbaddr: ffff8800d62a1300 } hitcount: 3 len: 138
+ { skbaddr: ffff88002e37a100 } hitcount: 4 len: 184
+ { skbaddr: ffff880064504400 } hitcount: 4 len: 184
+ { skbaddr: ffff8800a8bfec00 } hitcount: 4 len: 184
+ { skbaddr: ffff88000ad53700 } hitcount: 5 len: 230
+ { skbaddr: ffff8800d2bcdb00 } hitcount: 5 len: 196
+ { skbaddr: ffff8800a1f90000 } hitcount: 6 len: 276
+ { skbaddr: ffff88006a54f900 } hitcount: 6 len: 276
+
+ Totals:
+ Hits: 81
+ Entries: 42
+ Dropped: 0
+ # event histogram
+ #
+ # trigger info: hist:name=foo:keys=skbaddr.hex:vals=hitcount,len:sort=hitcount:size=2048 [active]
+ #
+
+ { skbaddr: ffff88000ad53500 } hitcount: 1 len: 46
+ { skbaddr: ffff8800af5a1500 } hitcount: 1 len: 76
+ { skbaddr: ffff8800d62a1900 } hitcount: 1 len: 46
+ { skbaddr: ffff8800d2bccb00 } hitcount: 1 len: 468
+ { skbaddr: ffff8800d3c69900 } hitcount: 1 len: 46
+ { skbaddr: ffff88009ff09100 } hitcount: 1 len: 52
+ { skbaddr: ffff88010f13ab00 } hitcount: 1 len: 168
+ { skbaddr: ffff88006a54f400 } hitcount: 1 len: 46
+ { skbaddr: ffff8800d2bcc500 } hitcount: 1 len: 260
+ { skbaddr: ffff880064505000 } hitcount: 1 len: 46
+ { skbaddr: ffff8800baf24e00 } hitcount: 1 len: 32
+ { skbaddr: ffff88009fe0ad00 } hitcount: 1 len: 46
+ { skbaddr: ffff8800d3edff00 } hitcount: 1 len: 44
+ { skbaddr: ffff88009fe0b400 } hitcount: 1 len: 168
+ { skbaddr: ffff8800a1c55a00 } hitcount: 1 len: 40
+ { skbaddr: ffff8800d2bcd100 } hitcount: 1 len: 40
+ { skbaddr: ffff880064505f00 } hitcount: 1 len: 174
+ { skbaddr: ffff8800a8bff200 } hitcount: 1 len: 160
+ { skbaddr: ffff880044e3cc00 } hitcount: 1 len: 76
+ { skbaddr: ffff8800a8bfe700 } hitcount: 1 len: 46
+ { skbaddr: ffff8800d2bcdc00 } hitcount: 1 len: 32
+ { skbaddr: ffff8800a1f64800 } hitcount: 1 len: 46
+ { skbaddr: ffff8800d2bcde00 } hitcount: 1 len: 988
+ { skbaddr: ffff88006a5dea00 } hitcount: 1 len: 46
+ { skbaddr: ffff88002e37a200 } hitcount: 1 len: 44
+ { skbaddr: ffff8800a1f32c00 } hitcount: 2 len: 676
+ { skbaddr: ffff88000ad52600 } hitcount: 2 len: 107
+ { skbaddr: ffff8800a1f91e00 } hitcount: 2 len: 92
+ { skbaddr: ffff8800af5a0200 } hitcount: 2 len: 142
+ { skbaddr: ffff8800d2bcc600 } hitcount: 2 len: 220
+ { skbaddr: ffff8800ba36f500 } hitcount: 2 len: 92
+ { skbaddr: ffff8800d021f800 } hitcount: 2 len: 92
+ { skbaddr: ffff8800a1f33600 } hitcount: 2 len: 675
+ { skbaddr: ffff8800a8bfff00 } hitcount: 3 len: 138
+ { skbaddr: ffff8800d62a1300 } hitcount: 3 len: 138
+ { skbaddr: ffff88002e37a100 } hitcount: 4 len: 184
+ { skbaddr: ffff880064504400 } hitcount: 4 len: 184
+ { skbaddr: ffff8800a8bfec00 } hitcount: 4 len: 184
+ { skbaddr: ffff88000ad53700 } hitcount: 5 len: 230
+ { skbaddr: ffff8800d2bcdb00 } hitcount: 5 len: 196
+ { skbaddr: ffff8800a1f90000 } hitcount: 6 len: 276
+ { skbaddr: ffff88006a54f900 } hitcount: 6 len: 276
+
+ Totals:
+ Hits: 81
+ Entries: 42
+ Dropped: 0
+
+ And here's an example that shows how to combine histogram data from
+ any two events even if they don't share any 'compatible' fields
+ other than 'hitcount' and 'stacktrace'. These commands create a
+ couple of triggers named 'bar' using those fields:
+
+ # echo 'hist:name=bar:key=stacktrace:val=hitcount' > \
+ /sys/kernel/debug/tracing/events/sched/sched_process_fork/trigger
+ # echo 'hist:name=bar:key=stacktrace:val=hitcount' > \
+ /sys/kernel/debug/tracing/events/net/netif_rx/trigger
+
+ And displaying the output of either shows some interesting if
+ somewhat confusing output:
+
+ # cat /sys/kernel/debug/tracing/events/sched/sched_process_fork/hist
+ # cat /sys/kernel/debug/tracing/events/net/netif_rx/hist
+
+ # event histogram
+ #
+ # trigger info: hist:name=bar:keys=stacktrace:vals=hitcount:sort=hitcount:size=2048 [active]
+ #
+
+ { stacktrace:
+ _do_fork+0x18e/0x330
+ kernel_thread+0x29/0x30
+ kthreadd+0x154/0x1b0
+ ret_from_fork+0x3f/0x70
+ } hitcount: 1
+ { stacktrace:
+ netif_rx_internal+0xb2/0xd0
+ netif_rx_ni+0x20/0x70
+ dev_loopback_xmit+0xaa/0xd0
+ ip_mc_output+0x126/0x240
+ ip_local_out_sk+0x31/0x40
+ igmp_send_report+0x1e9/0x230
+ igmp_timer_expire+0xe9/0x120
+ call_timer_fn+0x39/0xf0
+ run_timer_softirq+0x1e1/0x290
+ __do_softirq+0xfd/0x290
+ irq_exit+0x98/0xb0
+ smp_apic_timer_interrupt+0x4a/0x60
+ apic_timer_interrupt+0x6d/0x80
+ cpuidle_enter+0x17/0x20
+ call_cpuidle+0x3b/0x60
+ cpu_startup_entry+0x22d/0x310
+ } hitcount: 1
+ { stacktrace:
+ netif_rx_internal+0xb2/0xd0
+ netif_rx_ni+0x20/0x70
+ dev_loopback_xmit+0xaa/0xd0
+ ip_mc_output+0x17f/0x240
+ ip_local_out_sk+0x31/0x40
+ ip_send_skb+0x1a/0x50
+ udp_send_skb+0x13e/0x270
+ udp_sendmsg+0x2bf/0x980
+ inet_sendmsg+0x67/0xa0
+ sock_sendmsg+0x38/0x50
+ SYSC_sendto+0xef/0x170
+ SyS_sendto+0xe/0x10
+ entry_SYSCALL_64_fastpath+0x12/0x6a
+ } hitcount: 2
+ { stacktrace:
+ netif_rx_internal+0xb2/0xd0
+ netif_rx+0x1c/0x60
+ loopback_xmit+0x6c/0xb0
+ dev_hard_start_xmit+0x219/0x3a0
+ __dev_queue_xmit+0x415/0x4f0
+ dev_queue_xmit_sk+0x13/0x20
+ ip_finish_output2+0x237/0x340
+ ip_finish_output+0x113/0x1d0
+ ip_output+0x66/0xc0
+ ip_local_out_sk+0x31/0x40
+ ip_send_skb+0x1a/0x50
+ udp_send_skb+0x16d/0x270
+ udp_sendmsg+0x2bf/0x980
+ inet_sendmsg+0x67/0xa0
+ sock_sendmsg+0x38/0x50
+ ___sys_sendmsg+0x14e/0x270
+ } hitcount: 76
+ { stacktrace:
+ netif_rx_internal+0xb2/0xd0
+ netif_rx+0x1c/0x60
+ loopback_xmit+0x6c/0xb0
+ dev_hard_start_xmit+0x219/0x3a0
+ __dev_queue_xmit+0x415/0x4f0
+ dev_queue_xmit_sk+0x13/0x20
+ ip_finish_output2+0x237/0x340
+ ip_finish_output+0x113/0x1d0
+ ip_output+0x66/0xc0
+ ip_local_out_sk+0x31/0x40
+ ip_send_skb+0x1a/0x50
+ udp_send_skb+0x16d/0x270
+ udp_sendmsg+0x2bf/0x980
+ inet_sendmsg+0x67/0xa0
+ sock_sendmsg+0x38/0x50
+ ___sys_sendmsg+0x269/0x270
+ } hitcount: 77
+ { stacktrace:
+ netif_rx_internal+0xb2/0xd0
+ netif_rx+0x1c/0x60
+ loopback_xmit+0x6c/0xb0
+ dev_hard_start_xmit+0x219/0x3a0
+ __dev_queue_xmit+0x415/0x4f0
+ dev_queue_xmit_sk+0x13/0x20
+ ip_finish_output2+0x237/0x340
+ ip_finish_output+0x113/0x1d0
+ ip_output+0x66/0xc0
+ ip_local_out_sk+0x31/0x40
+ ip_send_skb+0x1a/0x50
+ udp_send_skb+0x16d/0x270
+ udp_sendmsg+0x2bf/0x980
+ inet_sendmsg+0x67/0xa0
+ sock_sendmsg+0x38/0x50
+ SYSC_sendto+0xef/0x170
+ } hitcount: 88
+ { stacktrace:
+ _do_fork+0x18e/0x330
+ SyS_clone+0x19/0x20
+ entry_SYSCALL_64_fastpath+0x12/0x6a
+ } hitcount: 244
+
+ Totals:
+ Hits: 489
+ Entries: 7
+ Dropped: 0
diff --git a/Documentation/trace/ftrace.txt b/Documentation/trace/ftrace.txt
index 9857606..a6b3705 100644
--- a/Documentation/trace/ftrace.txt
+++ b/Documentation/trace/ftrace.txt
@@ -210,6 +210,11 @@
Note, sched_switch and sched_wake_up will also trace events
listed in this file.
+ To have the PIDs of children of tasks with their PID in this file
+ added on fork, enable the "event-fork" option. That option will also
+ cause the PIDs of tasks to be removed from this file when the task
+ exits.
+
set_graph_function:
Set a "trigger" function where tracing should start
@@ -725,16 +730,14 @@
nohex
nobin
noblock
-nostacktrace
trace_printk
-noftrace_preempt
nobranch
annotate
nouserstacktrace
nosym-userobj
noprintk-msg-only
context-info
-latency-format
+nolatency-format
sleep-time
graph-time
record-cmd
@@ -742,7 +745,10 @@
nodisable_on_free
irq-info
markers
+noevent-fork
function-trace
+nodisplay-graph
+nostacktrace
To disable one of the options, echo in the option prepended with
"no".
@@ -796,11 +802,6 @@
block - When set, reading trace_pipe will not block when polled.
- stacktrace - This is one of the options that changes the trace
- itself. When a trace is recorded, so is the stack
- of functions. This allows for back traces of
- trace sites.
-
trace_printk - Can disable trace_printk() from writing into the buffer.
branch - Enable branch tracing with the tracer.
@@ -897,6 +898,10 @@
When disabled, the trace_marker will error with EINVAL
on write.
+ event-fork - When set, tasks with PIDs listed in set_event_pid will have
+ the PIDs of their children added to set_event_pid when those
+ tasks fork. Also, when tasks with PIDs in set_event_pid exit,
+ their PIDs will be removed from the file.
function-trace - The latency tracers will enable function tracing
if this option is enabled (default it is). When
@@ -904,8 +909,17 @@
functions. This keeps the overhead of the tracer down
when performing latency tests.
- Note: Some tracers have their own options. They only appear
- when the tracer is active.
+ display-graph - When set, the latency tracers (irqsoff, wakeup, etc) will
+ use function graph tracing instead of function tracing.
+
+ stacktrace - This is one of the options that changes the trace
+ itself. When a trace is recorded, so is the stack
+ of functions. This allows for back traces of
+ trace sites.
+
+ Note: Some tracers have their own options. They only appear in this
+ file when the tracer is active. They always appear in the
+ options directory.
diff --git a/Documentation/usb/chipidea.txt b/Documentation/usb/chipidea.txt
index 678741b..edf7cdf 100644
--- a/Documentation/usb/chipidea.txt
+++ b/Documentation/usb/chipidea.txt
@@ -3,14 +3,17 @@
To show how to demo OTG HNP and SRP functions via sys input files
with 2 Freescale i.MX6Q sabre SD boards.
-1.1 How to enable OTG FSM in menuconfig
+1.1 How to enable OTG FSM
---------------------------------------
-Select CONFIG_USB_OTG_FSM, rebuild kernel Image and modules.
-If you want to check some internal variables for otg fsm,
-mount debugfs, there are 2 files which can show otg fsm
-variables and some controller registers value:
+1.1.1 Select CONFIG_USB_OTG_FSM in menuconfig, rebuild kernel
+Image and modules. If you want to check some internal
+variables for otg fsm, mount debugfs, there are 2 files
+which can show otg fsm variables and some controller registers value:
cat /sys/kernel/debug/ci_hdrc.0/otg
cat /sys/kernel/debug/ci_hdrc.0/registers
+1.1.2 Add below entries in your dts file for your controller node
+ otg-rev = <0x0200>;
+ adp-disable;
1.2 Test operations
-------------------
diff --git a/Documentation/video4linux/CARDLIST.cx23885 b/Documentation/video4linux/CARDLIST.cx23885
index 44a4cfb..85a8fdc 100644
--- a/Documentation/video4linux/CARDLIST.cx23885
+++ b/Documentation/video4linux/CARDLIST.cx23885
@@ -52,3 +52,5 @@
51 -> DVBSky T982 [4254:0982]
52 -> Hauppauge WinTV-HVR5525 [0070:f038]
53 -> Hauppauge WinTV Starburst [0070:c12a]
+ 54 -> ViewCast 260e [1576:0260]
+ 55 -> ViewCast 460e [1576:0460]
diff --git a/Documentation/video4linux/CARDLIST.em28xx b/Documentation/video4linux/CARDLIST.em28xx
index 6720999..6784220 100644
--- a/Documentation/video4linux/CARDLIST.em28xx
+++ b/Documentation/video4linux/CARDLIST.em28xx
@@ -76,9 +76,9 @@
75 -> Dikom DK300 (em2882)
76 -> KWorld PlusTV 340U or UB435-Q (ATSC) (em2870) [1b80:a340]
77 -> EM2874 Leadership ISDBT (em2874)
- 78 -> PCTV nanoStick T2 290e (em28174)
+ 78 -> PCTV nanoStick T2 290e (em28174) [2013:024f]
79 -> Terratec Cinergy H5 (em2884) [eb1a:2885,0ccd:10a2,0ccd:10ad,0ccd:10b6]
- 80 -> PCTV DVB-S2 Stick (460e) (em28174)
+ 80 -> PCTV DVB-S2 Stick (460e) (em28174) [2013:024c]
81 -> Hauppauge WinTV HVR 930C (em2884) [2040:1605]
82 -> Terratec Cinergy HTC Stick (em2884) [0ccd:00b2]
83 -> Honestech Vidbox NW03 (em2860) [eb1a:5006]
@@ -90,9 +90,11 @@
89 -> Delock 61959 (em2874) [1b80:e1cc]
90 -> KWorld USB ATSC TV Stick UB435-Q V2 (em2874) [1b80:e346]
91 -> SpeedLink Vicious And Devine Laplace webcam (em2765) [1ae7:9003,1ae7:9004]
- 92 -> PCTV DVB-S2 Stick (461e) (em28178)
+ 92 -> PCTV DVB-S2 Stick (461e) (em28178) [2013:0258]
93 -> KWorld USB ATSC TV Stick UB435-Q V3 (em2874) [1b80:e34c]
- 94 -> PCTV tripleStick (292e) (em28178)
+ 94 -> PCTV tripleStick (292e) (em28178) [2013:025f,2040:0264]
95 -> Leadtek VC100 (em2861) [0413:6f07]
- 96 -> Terratec Cinergy T2 Stick HD (em28178)
+ 96 -> Terratec Cinergy T2 Stick HD (em28178) [eb1a:8179]
97 -> Elgato EyeTV Hybrid 2008 INT (em2884) [0fd9:0018]
+ 98 -> PLEX PX-BCUD (em28178) [3275:0085]
+ 99 -> Hauppauge WinTV-dualHD DVB (em28174) [2040:0265]
diff --git a/Documentation/video4linux/README.cx88 b/Documentation/video4linux/README.cx88
index 35fae23..b09ce36 100644
--- a/Documentation/video4linux/README.cx88
+++ b/Documentation/video4linux/README.cx88
@@ -50,7 +50,7 @@
cx88-cards.c. If that worked, mail me your changes as unified
diff ("diff -u").
(3) Or you can mail me the config information. I need at least the
- following informations to add the card:
+ following information to add the card:
* the PCI Subsystem ID ("0070:3400" from the line above,
"lspci -v" output is fine too).
diff --git a/Documentation/video4linux/bttv/Sound-FAQ b/Documentation/video4linux/bttv/Sound-FAQ
index d3f1d77..646a47d 100644
--- a/Documentation/video4linux/bttv/Sound-FAQ
+++ b/Documentation/video4linux/bttv/Sound-FAQ
@@ -55,7 +55,7 @@
to connect the mux chip.
As mentioned above, there is a array which holds the required
-informations for each known board. You basically have to create a new
+information for each known board. You basically have to create a new
line for your board. The important fields are these two:
struct tvcard
diff --git a/Documentation/video4linux/v4l2-framework.txt b/Documentation/video4linux/v4l2-framework.txt
index fa41608..cbefc79 100644
--- a/Documentation/video4linux/v4l2-framework.txt
+++ b/Documentation/video4linux/v4l2-framework.txt
@@ -35,7 +35,7 @@
common code into utility functions shared by all drivers.
A good example to look at as a reference is the v4l2-pci-skeleton.c
-source that is available in this directory. It is a skeleton driver for
+source that is available in samples/v4l/. It is a skeleton driver for
a PCI capture card, and demonstrates how to use the V4L2 driver
framework. It can be used as a template for real PCI video capture driver.
diff --git a/Documentation/video4linux/v4l2-pci-skeleton.c b/Documentation/video4linux/v4l2-pci-skeleton.c
deleted file mode 100644
index 79af0c0..0000000
--- a/Documentation/video4linux/v4l2-pci-skeleton.c
+++ /dev/null
@@ -1,923 +0,0 @@
-/*
- * This is a V4L2 PCI Skeleton Driver. It gives an initial skeleton source
- * for use with other PCI drivers.
- *
- * This skeleton PCI driver assumes that the card has an S-Video connector as
- * input 0 and an HDMI connector as input 1.
- *
- * Copyright 2014 Cisco Systems, Inc. and/or its affiliates. All rights reserved.
- *
- * This program is free software; you may redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; version 2 of the License.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
- * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
- * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
- * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
- * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
- * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
- * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
- * SOFTWARE.
- */
-
-#include <linux/types.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-#include <linux/init.h>
-#include <linux/kmod.h>
-#include <linux/mutex.h>
-#include <linux/pci.h>
-#include <linux/interrupt.h>
-#include <linux/videodev2.h>
-#include <linux/v4l2-dv-timings.h>
-#include <media/v4l2-device.h>
-#include <media/v4l2-dev.h>
-#include <media/v4l2-ioctl.h>
-#include <media/v4l2-dv-timings.h>
-#include <media/v4l2-ctrls.h>
-#include <media/v4l2-event.h>
-#include <media/videobuf2-v4l2.h>
-#include <media/videobuf2-dma-contig.h>
-
-MODULE_DESCRIPTION("V4L2 PCI Skeleton Driver");
-MODULE_AUTHOR("Hans Verkuil");
-MODULE_LICENSE("GPL v2");
-
-/**
- * struct skeleton - All internal data for one instance of device
- * @pdev: PCI device
- * @v4l2_dev: top-level v4l2 device struct
- * @vdev: video node structure
- * @ctrl_handler: control handler structure
- * @lock: ioctl serialization mutex
- * @std: current SDTV standard
- * @timings: current HDTV timings
- * @format: current pix format
- * @input: current video input (0 = SDTV, 1 = HDTV)
- * @queue: vb2 video capture queue
- * @alloc_ctx: vb2 contiguous DMA context
- * @qlock: spinlock controlling access to buf_list and sequence
- * @buf_list: list of buffers queued for DMA
- * @sequence: frame sequence counter
- */
-struct skeleton {
- struct pci_dev *pdev;
- struct v4l2_device v4l2_dev;
- struct video_device vdev;
- struct v4l2_ctrl_handler ctrl_handler;
- struct mutex lock;
- v4l2_std_id std;
- struct v4l2_dv_timings timings;
- struct v4l2_pix_format format;
- unsigned input;
-
- struct vb2_queue queue;
- struct vb2_alloc_ctx *alloc_ctx;
-
- spinlock_t qlock;
- struct list_head buf_list;
- unsigned field;
- unsigned sequence;
-};
-
-struct skel_buffer {
- struct vb2_buffer vb;
- struct list_head list;
-};
-
-static inline struct skel_buffer *to_skel_buffer(struct vb2_buffer *vb2)
-{
- return container_of(vb2, struct skel_buffer, vb);
-}
-
-static const struct pci_device_id skeleton_pci_tbl[] = {
- /* { PCI_DEVICE(PCI_VENDOR_ID_, PCI_DEVICE_ID_) }, */
- { 0, }
-};
-MODULE_DEVICE_TABLE(pci, skeleton_pci_tbl);
-
-/*
- * HDTV: this structure has the capabilities of the HDTV receiver.
- * It is used to constrain the huge list of possible formats based
- * upon the hardware capabilities.
- */
-static const struct v4l2_dv_timings_cap skel_timings_cap = {
- .type = V4L2_DV_BT_656_1120,
- /* keep this initialization for compatibility with GCC < 4.4.6 */
- .reserved = { 0 },
- V4L2_INIT_BT_TIMINGS(
- 720, 1920, /* min/max width */
- 480, 1080, /* min/max height */
- 27000000, 74250000, /* min/max pixelclock*/
- V4L2_DV_BT_STD_CEA861, /* Supported standards */
- /* capabilities */
- V4L2_DV_BT_CAP_INTERLACED | V4L2_DV_BT_CAP_PROGRESSIVE
- )
-};
-
-/*
- * Supported SDTV standards. This does the same job as skel_timings_cap, but
- * for standard TV formats.
- */
-#define SKEL_TVNORMS V4L2_STD_ALL
-
-/*
- * Interrupt handler: typically interrupts happen after a new frame has been
- * captured. It is the job of the handler to remove the new frame from the
- * internal list and give it back to the vb2 framework, updating the sequence
- * counter, field and timestamp at the same time.
- */
-static irqreturn_t skeleton_irq(int irq, void *dev_id)
-{
-#ifdef TODO
- struct skeleton *skel = dev_id;
-
- /* handle interrupt */
-
- /* Once a new frame has been captured, mark it as done like this: */
- if (captured_new_frame) {
- ...
- spin_lock(&skel->qlock);
- list_del(&new_buf->list);
- spin_unlock(&skel->qlock);
- v4l2_get_timestamp(&new_buf->vb.v4l2_buf.timestamp);
- new_buf->vb.v4l2_buf.sequence = skel->sequence++;
- new_buf->vb.v4l2_buf.field = skel->field;
- if (skel->format.field == V4L2_FIELD_ALTERNATE) {
- if (skel->field == V4L2_FIELD_BOTTOM)
- skel->field = V4L2_FIELD_TOP;
- else if (skel->field == V4L2_FIELD_TOP)
- skel->field = V4L2_FIELD_BOTTOM;
- }
- vb2_buffer_done(&new_buf->vb, VB2_BUF_STATE_DONE);
- }
-#endif
- return IRQ_HANDLED;
-}
-
-/*
- * Setup the constraints of the queue: besides setting the number of planes
- * per buffer and the size and allocation context of each plane, it also
- * checks if sufficient buffers have been allocated. Usually 3 is a good
- * minimum number: many DMA engines need a minimum of 2 buffers in the
- * queue and you need to have another available for userspace processing.
- */
-static int queue_setup(struct vb2_queue *vq,
- unsigned int *nbuffers, unsigned int *nplanes,
- unsigned int sizes[], void *alloc_ctxs[])
-{
- struct skeleton *skel = vb2_get_drv_priv(vq);
-
- skel->field = skel->format.field;
- if (skel->field == V4L2_FIELD_ALTERNATE) {
- /*
- * You cannot use read() with FIELD_ALTERNATE since the field
- * information (TOP/BOTTOM) cannot be passed back to the user.
- */
- if (vb2_fileio_is_active(vq))
- return -EINVAL;
- skel->field = V4L2_FIELD_TOP;
- }
-
- if (vq->num_buffers + *nbuffers < 3)
- *nbuffers = 3 - vq->num_buffers;
- alloc_ctxs[0] = skel->alloc_ctx;
-
- if (*nplanes)
- return sizes[0] < skel->format.sizeimage ? -EINVAL : 0;
- *nplanes = 1;
- sizes[0] = skel->format.sizeimage;
- return 0;
-}
-
-/*
- * Prepare the buffer for queueing to the DMA engine: check and set the
- * payload size.
- */
-static int buffer_prepare(struct vb2_buffer *vb)
-{
- struct skeleton *skel = vb2_get_drv_priv(vb->vb2_queue);
- unsigned long size = skel->format.sizeimage;
-
- if (vb2_plane_size(vb, 0) < size) {
- dev_err(&skel->pdev->dev, "buffer too small (%lu < %lu)\n",
- vb2_plane_size(vb, 0), size);
- return -EINVAL;
- }
-
- vb2_set_plane_payload(vb, 0, size);
- return 0;
-}
-
-/*
- * Queue this buffer to the DMA engine.
- */
-static void buffer_queue(struct vb2_buffer *vb)
-{
- struct skeleton *skel = vb2_get_drv_priv(vb->vb2_queue);
- struct skel_buffer *buf = to_skel_buffer(vb);
- unsigned long flags;
-
- spin_lock_irqsave(&skel->qlock, flags);
- list_add_tail(&buf->list, &skel->buf_list);
-
- /* TODO: Update any DMA pointers if necessary */
-
- spin_unlock_irqrestore(&skel->qlock, flags);
-}
-
-static void return_all_buffers(struct skeleton *skel,
- enum vb2_buffer_state state)
-{
- struct skel_buffer *buf, *node;
- unsigned long flags;
-
- spin_lock_irqsave(&skel->qlock, flags);
- list_for_each_entry_safe(buf, node, &skel->buf_list, list) {
- vb2_buffer_done(&buf->vb, state);
- list_del(&buf->list);
- }
- spin_unlock_irqrestore(&skel->qlock, flags);
-}
-
-/*
- * Start streaming. First check if the minimum number of buffers have been
- * queued. If not, then return -ENOBUFS and the vb2 framework will call
- * this function again the next time a buffer has been queued until enough
- * buffers are available to actually start the DMA engine.
- */
-static int start_streaming(struct vb2_queue *vq, unsigned int count)
-{
- struct skeleton *skel = vb2_get_drv_priv(vq);
- int ret = 0;
-
- skel->sequence = 0;
-
- /* TODO: start DMA */
-
- if (ret) {
- /*
- * In case of an error, return all active buffers to the
- * QUEUED state
- */
- return_all_buffers(skel, VB2_BUF_STATE_QUEUED);
- }
- return ret;
-}
-
-/*
- * Stop the DMA engine. Any remaining buffers in the DMA queue are dequeued
- * and passed on to the vb2 framework marked as STATE_ERROR.
- */
-static void stop_streaming(struct vb2_queue *vq)
-{
- struct skeleton *skel = vb2_get_drv_priv(vq);
-
- /* TODO: stop DMA */
-
- /* Release all active buffers */
- return_all_buffers(skel, VB2_BUF_STATE_ERROR);
-}
-
-/*
- * The vb2 queue ops. Note that since q->lock is set we can use the standard
- * vb2_ops_wait_prepare/finish helper functions. If q->lock would be NULL,
- * then this driver would have to provide these ops.
- */
-static struct vb2_ops skel_qops = {
- .queue_setup = queue_setup,
- .buf_prepare = buffer_prepare,
- .buf_queue = buffer_queue,
- .start_streaming = start_streaming,
- .stop_streaming = stop_streaming,
- .wait_prepare = vb2_ops_wait_prepare,
- .wait_finish = vb2_ops_wait_finish,
-};
-
-/*
- * Required ioctl querycap. Note that the version field is prefilled with
- * the version of the kernel.
- */
-static int skeleton_querycap(struct file *file, void *priv,
- struct v4l2_capability *cap)
-{
- struct skeleton *skel = video_drvdata(file);
-
- strlcpy(cap->driver, KBUILD_MODNAME, sizeof(cap->driver));
- strlcpy(cap->card, "V4L2 PCI Skeleton", sizeof(cap->card));
- snprintf(cap->bus_info, sizeof(cap->bus_info), "PCI:%s",
- pci_name(skel->pdev));
- cap->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_READWRITE |
- V4L2_CAP_STREAMING;
- cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
- return 0;
-}
-
-/*
- * Helper function to check and correct struct v4l2_pix_format. It's used
- * not only in VIDIOC_TRY/S_FMT, but also elsewhere if changes to the SDTV
- * standard, HDTV timings or the video input would require updating the
- * current format.
- */
-static void skeleton_fill_pix_format(struct skeleton *skel,
- struct v4l2_pix_format *pix)
-{
- pix->pixelformat = V4L2_PIX_FMT_YUYV;
- if (skel->input == 0) {
- /* S-Video input */
- pix->width = 720;
- pix->height = (skel->std & V4L2_STD_525_60) ? 480 : 576;
- pix->field = V4L2_FIELD_INTERLACED;
- pix->colorspace = V4L2_COLORSPACE_SMPTE170M;
- } else {
- /* HDMI input */
- pix->width = skel->timings.bt.width;
- pix->height = skel->timings.bt.height;
- if (skel->timings.bt.interlaced) {
- pix->field = V4L2_FIELD_ALTERNATE;
- pix->height /= 2;
- } else {
- pix->field = V4L2_FIELD_NONE;
- }
- pix->colorspace = V4L2_COLORSPACE_REC709;
- }
-
- /*
- * The YUYV format is four bytes for every two pixels, so bytesperline
- * is width * 2.
- */
- pix->bytesperline = pix->width * 2;
- pix->sizeimage = pix->bytesperline * pix->height;
- pix->priv = 0;
-}
-
-static int skeleton_try_fmt_vid_cap(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct skeleton *skel = video_drvdata(file);
- struct v4l2_pix_format *pix = &f->fmt.pix;
-
- /*
- * Due to historical reasons providing try_fmt with an unsupported
- * pixelformat will return -EINVAL for video receivers. Webcam drivers,
- * however, will silently correct the pixelformat. Some video capture
- * applications rely on this behavior...
- */
- if (pix->pixelformat != V4L2_PIX_FMT_YUYV)
- return -EINVAL;
- skeleton_fill_pix_format(skel, pix);
- return 0;
-}
-
-static int skeleton_s_fmt_vid_cap(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct skeleton *skel = video_drvdata(file);
- int ret;
-
- ret = skeleton_try_fmt_vid_cap(file, priv, f);
- if (ret)
- return ret;
-
- /*
- * It is not allowed to change the format while buffers for use with
- * streaming have already been allocated.
- */
- if (vb2_is_busy(&skel->queue))
- return -EBUSY;
-
- /* TODO: change format */
- skel->format = f->fmt.pix;
- return 0;
-}
-
-static int skeleton_g_fmt_vid_cap(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct skeleton *skel = video_drvdata(file);
-
- f->fmt.pix = skel->format;
- return 0;
-}
-
-static int skeleton_enum_fmt_vid_cap(struct file *file, void *priv,
- struct v4l2_fmtdesc *f)
-{
- if (f->index != 0)
- return -EINVAL;
-
- f->pixelformat = V4L2_PIX_FMT_YUYV;
- return 0;
-}
-
-static int skeleton_s_std(struct file *file, void *priv, v4l2_std_id std)
-{
- struct skeleton *skel = video_drvdata(file);
-
- /* S_STD is not supported on the HDMI input */
- if (skel->input)
- return -ENODATA;
-
- /*
- * No change, so just return. Some applications call S_STD again after
- * the buffers for streaming have been set up, so we have to allow for
- * this behavior.
- */
- if (std == skel->std)
- return 0;
-
- /*
- * Changing the standard implies a format change, which is not allowed
- * while buffers for use with streaming have already been allocated.
- */
- if (vb2_is_busy(&skel->queue))
- return -EBUSY;
-
- /* TODO: handle changing std */
-
- skel->std = std;
-
- /* Update the internal format */
- skeleton_fill_pix_format(skel, &skel->format);
- return 0;
-}
-
-static int skeleton_g_std(struct file *file, void *priv, v4l2_std_id *std)
-{
- struct skeleton *skel = video_drvdata(file);
-
- /* G_STD is not supported on the HDMI input */
- if (skel->input)
- return -ENODATA;
-
- *std = skel->std;
- return 0;
-}
-
-/*
- * Query the current standard as seen by the hardware. This function shall
- * never actually change the standard, it just detects and reports.
- * The framework will initially set *std to tvnorms (i.e. the set of
- * supported standards by this input), and this function should just AND
- * this value. If there is no signal, then *std should be set to 0.
- */
-static int skeleton_querystd(struct file *file, void *priv, v4l2_std_id *std)
-{
- struct skeleton *skel = video_drvdata(file);
-
- /* QUERY_STD is not supported on the HDMI input */
- if (skel->input)
- return -ENODATA;
-
-#ifdef TODO
- /*
- * Query currently seen standard. Initial value of *std is
- * V4L2_STD_ALL. This function should look something like this:
- */
- get_signal_info();
- if (no_signal) {
- *std = 0;
- return 0;
- }
- /* Use signal information to reduce the number of possible standards */
- if (signal_has_525_lines)
- *std &= V4L2_STD_525_60;
- else
- *std &= V4L2_STD_625_50;
-#endif
- return 0;
-}
-
-static int skeleton_s_dv_timings(struct file *file, void *_fh,
- struct v4l2_dv_timings *timings)
-{
- struct skeleton *skel = video_drvdata(file);
-
- /* S_DV_TIMINGS is not supported on the S-Video input */
- if (skel->input == 0)
- return -ENODATA;
-
- /* Quick sanity check */
- if (!v4l2_valid_dv_timings(timings, &skel_timings_cap, NULL, NULL))
- return -EINVAL;
-
- /* Check if the timings are part of the CEA-861 timings. */
- if (!v4l2_find_dv_timings_cap(timings, &skel_timings_cap,
- 0, NULL, NULL))
- return -EINVAL;
-
- /* Return 0 if the new timings are the same as the current timings. */
- if (v4l2_match_dv_timings(timings, &skel->timings, 0, false))
- return 0;
-
- /*
- * Changing the timings implies a format change, which is not allowed
- * while buffers for use with streaming have already been allocated.
- */
- if (vb2_is_busy(&skel->queue))
- return -EBUSY;
-
- /* TODO: Configure new timings */
-
- /* Save timings */
- skel->timings = *timings;
-
- /* Update the internal format */
- skeleton_fill_pix_format(skel, &skel->format);
- return 0;
-}
-
-static int skeleton_g_dv_timings(struct file *file, void *_fh,
- struct v4l2_dv_timings *timings)
-{
- struct skeleton *skel = video_drvdata(file);
-
- /* G_DV_TIMINGS is not supported on the S-Video input */
- if (skel->input == 0)
- return -ENODATA;
-
- *timings = skel->timings;
- return 0;
-}
-
-static int skeleton_enum_dv_timings(struct file *file, void *_fh,
- struct v4l2_enum_dv_timings *timings)
-{
- struct skeleton *skel = video_drvdata(file);
-
- /* ENUM_DV_TIMINGS is not supported on the S-Video input */
- if (skel->input == 0)
- return -ENODATA;
-
- return v4l2_enum_dv_timings_cap(timings, &skel_timings_cap,
- NULL, NULL);
-}
-
-/*
- * Query the current timings as seen by the hardware. This function shall
- * never actually change the timings, it just detects and reports.
- * If no signal is detected, then return -ENOLINK. If the hardware cannot
- * lock to the signal, then return -ENOLCK. If the signal is out of range
- * of the capabilities of the system (e.g., it is possible that the receiver
- * can lock but that the DMA engine it is connected to cannot handle
- * pixelclocks above a certain frequency), then -ERANGE is returned.
- */
-static int skeleton_query_dv_timings(struct file *file, void *_fh,
- struct v4l2_dv_timings *timings)
-{
- struct skeleton *skel = video_drvdata(file);
-
- /* QUERY_DV_TIMINGS is not supported on the S-Video input */
- if (skel->input == 0)
- return -ENODATA;
-
-#ifdef TODO
- /*
- * Query currently seen timings. This function should look
- * something like this:
- */
- detect_timings();
- if (no_signal)
- return -ENOLINK;
- if (cannot_lock_to_signal)
- return -ENOLCK;
- if (signal_out_of_range_of_capabilities)
- return -ERANGE;
-
- /* Useful for debugging */
- v4l2_print_dv_timings(skel->v4l2_dev.name, "query_dv_timings:",
- timings, true);
-#endif
- return 0;
-}
-
-static int skeleton_dv_timings_cap(struct file *file, void *fh,
- struct v4l2_dv_timings_cap *cap)
-{
- struct skeleton *skel = video_drvdata(file);
-
- /* DV_TIMINGS_CAP is not supported on the S-Video input */
- if (skel->input == 0)
- return -ENODATA;
- *cap = skel_timings_cap;
- return 0;
-}
-
-static int skeleton_enum_input(struct file *file, void *priv,
- struct v4l2_input *i)
-{
- if (i->index > 1)
- return -EINVAL;
-
- i->type = V4L2_INPUT_TYPE_CAMERA;
- if (i->index == 0) {
- i->std = SKEL_TVNORMS;
- strlcpy(i->name, "S-Video", sizeof(i->name));
- i->capabilities = V4L2_IN_CAP_STD;
- } else {
- i->std = 0;
- strlcpy(i->name, "HDMI", sizeof(i->name));
- i->capabilities = V4L2_IN_CAP_DV_TIMINGS;
- }
- return 0;
-}
-
-static int skeleton_s_input(struct file *file, void *priv, unsigned int i)
-{
- struct skeleton *skel = video_drvdata(file);
-
- if (i > 1)
- return -EINVAL;
-
- /*
- * Changing the input implies a format change, which is not allowed
- * while buffers for use with streaming have already been allocated.
- */
- if (vb2_is_busy(&skel->queue))
- return -EBUSY;
-
- skel->input = i;
- /*
- * Update tvnorms. The tvnorms value is used by the core to implement
- * VIDIOC_ENUMSTD so it has to be correct. If tvnorms == 0, then
- * ENUMSTD will return -ENODATA.
- */
- skel->vdev.tvnorms = i ? 0 : SKEL_TVNORMS;
-
- /* Update the internal format */
- skeleton_fill_pix_format(skel, &skel->format);
- return 0;
-}
-
-static int skeleton_g_input(struct file *file, void *priv, unsigned int *i)
-{
- struct skeleton *skel = video_drvdata(file);
-
- *i = skel->input;
- return 0;
-}
-
-/* The control handler. */
-static int skeleton_s_ctrl(struct v4l2_ctrl *ctrl)
-{
- /*struct skeleton *skel =
- container_of(ctrl->handler, struct skeleton, ctrl_handler);*/
-
- switch (ctrl->id) {
- case V4L2_CID_BRIGHTNESS:
- /* TODO: set brightness to ctrl->val */
- break;
- case V4L2_CID_CONTRAST:
- /* TODO: set contrast to ctrl->val */
- break;
- case V4L2_CID_SATURATION:
- /* TODO: set saturation to ctrl->val */
- break;
- case V4L2_CID_HUE:
- /* TODO: set hue to ctrl->val */
- break;
- default:
- return -EINVAL;
- }
- return 0;
-}
-
-/* ------------------------------------------------------------------
- File operations for the device
- ------------------------------------------------------------------*/
-
-static const struct v4l2_ctrl_ops skel_ctrl_ops = {
- .s_ctrl = skeleton_s_ctrl,
-};
-
-/*
- * The set of all supported ioctls. Note that all the streaming ioctls
- * use the vb2 helper functions that take care of all the locking and
- * that also do ownership tracking (i.e. only the filehandle that requested
- * the buffers can call the streaming ioctls, all other filehandles will
- * receive -EBUSY if they attempt to call the same streaming ioctls).
- *
- * The last three ioctls also use standard helper functions: these implement
- * standard behavior for drivers with controls.
- */
-static const struct v4l2_ioctl_ops skel_ioctl_ops = {
- .vidioc_querycap = skeleton_querycap,
- .vidioc_try_fmt_vid_cap = skeleton_try_fmt_vid_cap,
- .vidioc_s_fmt_vid_cap = skeleton_s_fmt_vid_cap,
- .vidioc_g_fmt_vid_cap = skeleton_g_fmt_vid_cap,
- .vidioc_enum_fmt_vid_cap = skeleton_enum_fmt_vid_cap,
-
- .vidioc_g_std = skeleton_g_std,
- .vidioc_s_std = skeleton_s_std,
- .vidioc_querystd = skeleton_querystd,
-
- .vidioc_s_dv_timings = skeleton_s_dv_timings,
- .vidioc_g_dv_timings = skeleton_g_dv_timings,
- .vidioc_enum_dv_timings = skeleton_enum_dv_timings,
- .vidioc_query_dv_timings = skeleton_query_dv_timings,
- .vidioc_dv_timings_cap = skeleton_dv_timings_cap,
-
- .vidioc_enum_input = skeleton_enum_input,
- .vidioc_g_input = skeleton_g_input,
- .vidioc_s_input = skeleton_s_input,
-
- .vidioc_reqbufs = vb2_ioctl_reqbufs,
- .vidioc_create_bufs = vb2_ioctl_create_bufs,
- .vidioc_querybuf = vb2_ioctl_querybuf,
- .vidioc_qbuf = vb2_ioctl_qbuf,
- .vidioc_dqbuf = vb2_ioctl_dqbuf,
- .vidioc_expbuf = vb2_ioctl_expbuf,
- .vidioc_streamon = vb2_ioctl_streamon,
- .vidioc_streamoff = vb2_ioctl_streamoff,
-
- .vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
- .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
-};
-
-/*
- * The set of file operations. Note that all these ops are standard core
- * helper functions.
- */
-static const struct v4l2_file_operations skel_fops = {
- .owner = THIS_MODULE,
- .open = v4l2_fh_open,
- .release = vb2_fop_release,
- .unlocked_ioctl = video_ioctl2,
- .read = vb2_fop_read,
- .mmap = vb2_fop_mmap,
- .poll = vb2_fop_poll,
-};
-
-/*
- * The initial setup of this device instance. Note that the initial state of
- * the driver should be complete. So the initial format, standard, timings
- * and video input should all be initialized to some reasonable value.
- */
-static int skeleton_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
-{
- /* The initial timings are chosen to be 720p60. */
- static const struct v4l2_dv_timings timings_def =
- V4L2_DV_BT_CEA_1280X720P60;
- struct skeleton *skel;
- struct video_device *vdev;
- struct v4l2_ctrl_handler *hdl;
- struct vb2_queue *q;
- int ret;
-
- /* Enable PCI */
- ret = pci_enable_device(pdev);
- if (ret)
- return ret;
- ret = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
- if (ret) {
- dev_err(&pdev->dev, "no suitable DMA available.\n");
- goto disable_pci;
- }
-
- /* Allocate a new instance */
- skel = devm_kzalloc(&pdev->dev, sizeof(struct skeleton), GFP_KERNEL);
- if (!skel)
- return -ENOMEM;
-
- /* Allocate the interrupt */
- ret = devm_request_irq(&pdev->dev, pdev->irq,
- skeleton_irq, 0, KBUILD_MODNAME, skel);
- if (ret) {
- dev_err(&pdev->dev, "request_irq failed\n");
- goto disable_pci;
- }
- skel->pdev = pdev;
-
- /* Fill in the initial format-related settings */
- skel->timings = timings_def;
- skel->std = V4L2_STD_625_50;
- skeleton_fill_pix_format(skel, &skel->format);
-
- /* Initialize the top-level structure */
- ret = v4l2_device_register(&pdev->dev, &skel->v4l2_dev);
- if (ret)
- goto disable_pci;
-
- mutex_init(&skel->lock);
-
- /* Add the controls */
- hdl = &skel->ctrl_handler;
- v4l2_ctrl_handler_init(hdl, 4);
- v4l2_ctrl_new_std(hdl, &skel_ctrl_ops,
- V4L2_CID_BRIGHTNESS, 0, 255, 1, 127);
- v4l2_ctrl_new_std(hdl, &skel_ctrl_ops,
- V4L2_CID_CONTRAST, 0, 255, 1, 16);
- v4l2_ctrl_new_std(hdl, &skel_ctrl_ops,
- V4L2_CID_SATURATION, 0, 255, 1, 127);
- v4l2_ctrl_new_std(hdl, &skel_ctrl_ops,
- V4L2_CID_HUE, -128, 127, 1, 0);
- if (hdl->error) {
- ret = hdl->error;
- goto free_hdl;
- }
- skel->v4l2_dev.ctrl_handler = hdl;
-
- /* Initialize the vb2 queue */
- q = &skel->queue;
- q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
- q->io_modes = VB2_MMAP | VB2_DMABUF | VB2_READ;
- q->drv_priv = skel;
- q->buf_struct_size = sizeof(struct skel_buffer);
- q->ops = &skel_qops;
- q->mem_ops = &vb2_dma_contig_memops;
- q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
- /*
- * Assume that this DMA engine needs to have at least two buffers
- * available before it can be started. The start_streaming() op
- * won't be called until at least this many buffers are queued up.
- */
- q->min_buffers_needed = 2;
- /*
- * The serialization lock for the streaming ioctls. This is the same
- * as the main serialization lock, but if some of the non-streaming
- * ioctls could take a long time to execute, then you might want to
- * have a different lock here to prevent VIDIOC_DQBUF from being
- * blocked while waiting for another action to finish. This is
- * generally not needed for PCI devices, but USB devices usually do
- * want a separate lock here.
- */
- q->lock = &skel->lock;
- /*
- * Since this driver can only do 32-bit DMA we must make sure that
- * the vb2 core will allocate the buffers in 32-bit DMA memory.
- */
- q->gfp_flags = GFP_DMA32;
- ret = vb2_queue_init(q);
- if (ret)
- goto free_hdl;
-
- skel->alloc_ctx = vb2_dma_contig_init_ctx(&pdev->dev);
- if (IS_ERR(skel->alloc_ctx)) {
- dev_err(&pdev->dev, "Can't allocate buffer context");
- ret = PTR_ERR(skel->alloc_ctx);
- goto free_hdl;
- }
- INIT_LIST_HEAD(&skel->buf_list);
- spin_lock_init(&skel->qlock);
-
- /* Initialize the video_device structure */
- vdev = &skel->vdev;
- strlcpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name));
- /*
- * There is nothing to clean up, so release is set to an empty release
- * function. The release callback must be non-NULL.
- */
- vdev->release = video_device_release_empty;
- vdev->fops = &skel_fops,
- vdev->ioctl_ops = &skel_ioctl_ops,
- /*
- * The main serialization lock. All ioctls are serialized by this
- * lock. Exception: if q->lock is set, then the streaming ioctls
- * are serialized by that separate lock.
- */
- vdev->lock = &skel->lock;
- vdev->queue = q;
- vdev->v4l2_dev = &skel->v4l2_dev;
- /* Supported SDTV standards, if any */
- vdev->tvnorms = SKEL_TVNORMS;
- video_set_drvdata(vdev, skel);
-
- ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
- if (ret)
- goto free_ctx;
-
- dev_info(&pdev->dev, "V4L2 PCI Skeleton Driver loaded\n");
- return 0;
-
-free_ctx:
- vb2_dma_contig_cleanup_ctx(skel->alloc_ctx);
-free_hdl:
- v4l2_ctrl_handler_free(&skel->ctrl_handler);
- v4l2_device_unregister(&skel->v4l2_dev);
-disable_pci:
- pci_disable_device(pdev);
- return ret;
-}
-
-static void skeleton_remove(struct pci_dev *pdev)
-{
- struct v4l2_device *v4l2_dev = pci_get_drvdata(pdev);
- struct skeleton *skel = container_of(v4l2_dev, struct skeleton, v4l2_dev);
-
- video_unregister_device(&skel->vdev);
- v4l2_ctrl_handler_free(&skel->ctrl_handler);
- vb2_dma_contig_cleanup_ctx(skel->alloc_ctx);
- v4l2_device_unregister(&skel->v4l2_dev);
- pci_disable_device(skel->pdev);
-}
-
-static struct pci_driver skeleton_driver = {
- .name = KBUILD_MODNAME,
- .probe = skeleton_probe,
- .remove = skeleton_remove,
- .id_table = skeleton_pci_tbl,
-};
-
-module_pci_driver(skeleton_driver);
diff --git a/Documentation/video4linux/vivid.txt b/Documentation/video4linux/vivid.txt
index e35d376..8da5d2a 100644
--- a/Documentation/video4linux/vivid.txt
+++ b/Documentation/video4linux/vivid.txt
@@ -294,7 +294,7 @@
These inputs support all combinations of the field setting. Special care has
been taken to faithfully reproduce how fields are handled for the different
-TV standards. This is particularly noticable when generating a horizontally
+TV standards. This is particularly noticeable when generating a horizontally
moving image so the temporal effect of using interlaced formats becomes clearly
visible. For 50 Hz standards the top field is the oldest and the bottom field
is the newest in time. For 60 Hz standards that is reversed: the bottom field
@@ -313,7 +313,7 @@
The pixel aspect ratio will depend on the TV standard. The video aspect ratio
can be selected through the 'Standard Aspect Ratio' Vivid control.
Choices are '4x3', '16x9' which will give letterboxed widescreen video and
-'16x9 Anomorphic' which will give full screen squashed anamorphic widescreen
+'16x9 Anamorphic' which will give full screen squashed anamorphic widescreen
video that will need to be scaled accordingly.
The TV 'tuner' supports a frequency range of 44-958 MHz. Channels are available
@@ -862,7 +862,7 @@
RDS Stereo:
RDS Artificial Head:
RDS Compressed:
-RDS Dymanic PTY:
+RDS Dynamic PTY:
RDS Traffic Announcement:
RDS Traffic Program:
RDS Music: these are all controls that set the RDS data that is transmitted by
diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 4d0542c..a4482cc 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -199,8 +199,8 @@
Parameters: vcpu id (apic id on x86)
Returns: vcpu fd on success, -1 on error
-This API adds a vcpu to a virtual machine. The vcpu id is a small integer
-in the range [0, max_vcpus).
+This API adds a vcpu to a virtual machine. No more than max_vcpus may be added.
+The vcpu id is an integer in the range [0, max_vcpu_id).
The recommended max_vcpus value can be retrieved using the KVM_CAP_NR_VCPUS of
the KVM_CHECK_EXTENSION ioctl() at run-time.
@@ -212,6 +212,12 @@
If the KVM_CAP_MAX_VCPUS does not exist, you should assume that max_vcpus is
same as the value returned from KVM_CAP_NR_VCPUS.
+The maximum possible value for max_vcpu_id can be retrieved using the
+KVM_CAP_MAX_VCPU_ID of the KVM_CHECK_EXTENSION ioctl() at run-time.
+
+If the KVM_CAP_MAX_VCPU_ID does not exist, you should assume that max_vcpu_id
+is the same as the value returned from KVM_CAP_MAX_VCPUS.
+
On powerpc using book3s_hv mode, the vcpus are mapped onto virtual
threads in one or more virtual CPU cores. (This is because the
hardware requires all the hardware threads in a CPU core to be in the
@@ -3788,6 +3794,14 @@
Fails if VCPU has already been created, or if the irqchip is already in the
kernel (i.e. KVM_CREATE_IRQCHIP has already been called).
+7.6 KVM_CAP_S390_RI
+
+Architectures: s390
+Parameters: none
+
+Allows use of runtime-instrumentation introduced with zEC12 processor.
+Will return -EINVAL if the machine does not support runtime-instrumentation.
+Will return -EBUSY if a VCPU has already been created.
8. Other capabilities.
----------------------
diff --git a/Documentation/virtual/kvm/devices/s390_flic.txt b/Documentation/virtual/kvm/devices/s390_flic.txt
index e3e314c..6b0e115 100644
--- a/Documentation/virtual/kvm/devices/s390_flic.txt
+++ b/Documentation/virtual/kvm/devices/s390_flic.txt
@@ -11,6 +11,7 @@
- add interrupts (KVM_DEV_FLIC_ENQUEUE)
- inspect currently pending interrupts (KVM_FLIC_GET_ALL_IRQS)
- purge all pending floating interrupts (KVM_DEV_FLIC_CLEAR_IRQS)
+- purge one pending floating I/O interrupt (KVM_DEV_FLIC_CLEAR_IO_IRQ)
- enable/disable for the guest transparent async page faults
- register and modify adapter interrupt sources (KVM_DEV_FLIC_ADAPTER_*)
@@ -40,6 +41,11 @@
Simply deletes all elements from the list of currently pending floating
interrupts. No interrupts are injected into the guest.
+ KVM_DEV_FLIC_CLEAR_IO_IRQ
+ Deletes one (if any) I/O interrupt for a subchannel identified by the
+ subsystem identification word passed via the buffer specified by
+ attr->addr (address) and attr->attr (length).
+
KVM_DEV_FLIC_APF_ENABLE
Enables async page faults for the guest. So in case of a major page fault
the host is allowed to handle this async and continues the guest.
@@ -68,7 +74,7 @@
KVM_DEV_FLIC_ADAPTER_MODIFY
Modifies attributes of an existing I/O adapter interrupt source. Takes
- a kvm_s390_io_adapter_req specifiying the adapter and the operation:
+ a kvm_s390_io_adapter_req specifying the adapter and the operation:
struct kvm_s390_io_adapter_req {
__u32 id;
@@ -94,3 +100,9 @@
KVM_S390_IO_ADAPTER_UNMAP
release a userspace page for the translated address specified in addr
from the list of mappings
+
+Note: The KVM_SET_DEVICE_ATTR/KVM_GET_DEVICE_ATTR device ioctls executed on
+FLIC with an unknown group or attribute gives the error code EINVAL (instead of
+ENXIO, as specified in the API documentation). It is not possible to conclude
+that a FLIC operation is unavailable based on the error code resulting from a
+usage attempt.
diff --git a/Documentation/vm/hugetlbpage.txt b/Documentation/vm/hugetlbpage.txt
index 54dd9b9..59cbc80 100644
--- a/Documentation/vm/hugetlbpage.txt
+++ b/Documentation/vm/hugetlbpage.txt
@@ -220,7 +220,7 @@
node list of "all" with numactl --interleave or --membind [-m] to achieve
interleaving over all nodes in the system or cpuset.
-4) Any task mempolicy specifed--e.g., using numactl--will be constrained by
+4) Any task mempolicy specified--e.g., using numactl--will be constrained by
the resource limits of any cpuset in which the task runs. Thus, there will
be no way for a task with non-default policy running in a cpuset with a
subset of the system nodes to allocate huge pages outside the cpuset
@@ -275,10 +275,10 @@
options sets the owner and group of the root of the file system. By default
the uid and gid of the current process are taken. The mode option sets the
mode of root of file system to value & 01777. This value is given in octal.
-By default the value 0755 is picked. If the paltform supports multiple huge
+By default the value 0755 is picked. If the platform supports multiple huge
page sizes, the pagesize option can be used to specify the huge page size and
associated pool. pagesize is specified in bytes. If pagesize is not specified
-the paltform's default huge page size and associated pool will be used. The
+the platform's default huge page size and associated pool will be used. The
size option sets the maximum value of memory (huge pages) allowed for that
filesystem (/mnt/huge). The size option can be specified in bytes, or as a
percentage of the specified huge page pool (nr_hugepages). The size is
diff --git a/Documentation/vm/pagemap.txt b/Documentation/vm/pagemap.txt
index 0e1e555..eafcefa 100644
--- a/Documentation/vm/pagemap.txt
+++ b/Documentation/vm/pagemap.txt
@@ -62,7 +62,7 @@
14. SWAPBACKED
15. COMPOUND_HEAD
16. COMPOUND_TAIL
- 16. HUGE
+ 17. HUGE
18. UNEVICTABLE
19. HWPOISON
20. NOPAGE
diff --git a/Documentation/vm/transhuge.txt b/Documentation/vm/transhuge.txt
index d9cb65c..7c871d6 100644
--- a/Documentation/vm/transhuge.txt
+++ b/Documentation/vm/transhuge.txt
@@ -340,7 +340,7 @@
== Graceful fallback ==
-Code walking pagetables but unware about huge pmds can simply call
+Code walking pagetables but unaware about huge pmds can simply call
split_huge_pmd(vma, pmd, addr) where the pmd is the one returned by
pmd_offset. It's trivial to make the code transparent hugepage aware
by just grepping for "pmd_offset" and adding split_huge_pmd where
@@ -394,9 +394,9 @@
Refcounting on THP is mostly consistent with refcounting on other compound
pages:
- - get_page()/put_page() and GUP operate in head page's ->_count.
+ - get_page()/put_page() and GUP operate in head page's ->_refcount.
- - ->_count in tail pages is always zero: get_page_unless_zero() never
+ - ->_refcount in tail pages is always zero: get_page_unless_zero() never
succeed on tail pages.
- map/unmap of the pages with PTE entry increment/decrement ->_mapcount
@@ -414,7 +414,7 @@
map/unmap of the whole compound page.
We set PG_double_map when a PMD of the page got split for the first time,
-but still have PMD mapping. The addtional references go away with last
+but still have PMD mapping. The additional references go away with last
compound_mapcount.
split_huge_page internally has to distribute the refcounts in the head
@@ -426,16 +426,16 @@
sum of mapcount of all sub-pages plus one (split_huge_page caller must
have reference for head page).
-split_huge_page uses migration entries to stabilize page->_count and
+split_huge_page uses migration entries to stabilize page->_refcount and
page->_mapcount.
We safe against physical memory scanners too: the only legitimate way
scanner can get reference to a page is get_page_unless_zero().
-All tail pages has zero ->_count until atomic_add(). It prevent scanner
-from geting reference to tail page up to the point. After the atomic_add()
-we don't care about ->_count value. We already known how many references
-with should uncharge from head page.
+All tail pages have zero ->_refcount until atomic_add(). This prevents the
+scanner from getting a reference to the tail page up to that point. After the
+atomic_add() we don't care about the ->_refcount value. We already known how
+many references should be uncharged from the head page.
For head page get_page_unless_zero() will succeed and we don't mind. It's
clear where reference should go after split: it will stay on head page.
diff --git a/Documentation/vm/z3fold.txt b/Documentation/vm/z3fold.txt
new file mode 100644
index 0000000..38e4dac
--- /dev/null
+++ b/Documentation/vm/z3fold.txt
@@ -0,0 +1,26 @@
+z3fold
+------
+
+z3fold is a special purpose allocator for storing compressed pages.
+It is designed to store up to three compressed pages per physical page.
+It is a zbud derivative which allows for higher compression
+ratio keeping the simplicity and determinism of its predecessor.
+
+The main differences between z3fold and zbud are:
+* unlike zbud, z3fold allows for up to PAGE_SIZE allocations
+* z3fold can hold up to 3 compressed pages in its page
+* z3fold doesn't export any API itself and is thus intended to be used
+ via the zpool API.
+
+To keep the determinism and simplicity, z3fold, just like zbud, always
+stores an integral number of compressed pages per page, but it can store
+up to 3 pages unlike zbud which can store at most 2. Therefore the
+compression ratio goes to around 2.7x while zbud's one is around 1.7x.
+
+Unlike zbud (but like zsmalloc for that matter) z3fold_alloc() does not
+return a dereferenceable pointer. Instead, it returns an unsigned long
+handle which encodes actual location of the allocated object.
+
+Keeping effective compression ratio close to zsmalloc's, z3fold doesn't
+depend on MMU enabled and provides more predictable reclaim behavior
+which makes it a better fit for small and response-critical systems.
diff --git a/Documentation/w1/slaves/w1_therm b/Documentation/w1/slaves/w1_therm
index 13411fe..d1f93af 100644
--- a/Documentation/w1/slaves/w1_therm
+++ b/Documentation/w1/slaves/w1_therm
@@ -33,7 +33,15 @@
powered it would be possible to convert all the devices at the same
time and then go back to read individual sensors. That isn't
currently supported. The driver also doesn't support reduced
-precision (which would also reduce the conversion time).
+precision (which would also reduce the conversion time) when reading values.
+
+Writing a value between 9 and 12 to the sysfs w1_slave file will change the
+precision of the sensor for the next readings. This value is in (volatile)
+SRAM, so it is reset when the sensor gets power-cycled.
+
+To store the current precision configuration into EEPROM, the value 0
+has to be written to the sysfs w1_slave file. Since the EEPROM has a limited
+amount of writes (>50k), this command should be used wisely.
The module parameter strong_pullup can be set to 0 to disable the
strong pullup, 1 to enable autodetection or 2 to force strong pullup.
diff --git a/Documentation/x86/intel_mpx.txt b/Documentation/x86/intel_mpx.txt
index 818518a..1a5a121 100644
--- a/Documentation/x86/intel_mpx.txt
+++ b/Documentation/x86/intel_mpx.txt
@@ -136,7 +136,7 @@
If we were to preallocate them for the 128TB of user virtual address
space, we would need to reserve 512TB+2GB, which is larger than the
entire virtual address space today. This means they can not be reserved
- ahead of time. Also, a single process's pre-popualated bounds directory
+ ahead of time. Also, a single process's pre-populated bounds directory
consumes 2GB of virtual *AND* physical memory. IOW, it's completely
infeasible to prepopulate bounds directories.
@@ -151,7 +151,7 @@
these calls.
Q: Could a bounds fault be handed to userspace and the tables allocated
- there in a signal handler intead of in the kernel?
+ there in a signal handler instead of in the kernel?
A: mmap() is not on the list of safe async handler functions and even
if mmap() would work it still requires locking or nasty tricks to
keep track of the allocation state there.
diff --git a/Documentation/xillybus.txt b/Documentation/xillybus.txt
index 81d111b..1660145 100644
--- a/Documentation/xillybus.txt
+++ b/Documentation/xillybus.txt
@@ -215,7 +215,7 @@
choice is a non-zero value, to match standard UNIX behavior.
* synchronous: A non-zero value means that the pipe is synchronous. See
- Syncronization above.
+ Synchronization above.
* bufsize: Each DMA buffer's size. Always a power of two.
diff --git a/Documentation/zh_CN/HOWTO b/Documentation/zh_CN/HOWTO
index 54ea24f..f0613b9 100644
--- a/Documentation/zh_CN/HOWTO
+++ b/Documentation/zh_CN/HOWTO
@@ -207,7 +207,7 @@
- 每当一个新版本的内核被发布，为期两周的集成窗口将被打开。在这段时间里
维护者可以向Linus提交大段的修改，通常这些修改已经被放到-mm内核中几个
星期了。提交大量修改的首选方式是使用git工具（内核的代码版本管理工具
- ，更多的信息可以在http://git.or.cz/获取），不过使用普通补丁也是可以
+ ，更多的信息可以在http://git-scm.com/获取），不过使用普通补丁也是可以
的。
- 两个星期以后-rc1版本内核发布。之后只有不包含可能影响整个内核稳定性的
新功能的补丁才可能被接受。请注意一个全新的驱动程序（或者文件系统）有
@@ -218,8 +218,6 @@
时，一个新的-rc版本就会被发布。计划是每周都发布新的-rc版本。
- 这个过程一直持续下去直到内核被认为达到足够稳定的状态，持续时间大概是
6个星期。
- - 以下地址跟踪了在每个-rc发布中发现的退步列表：
- http://kernelnewbies.org/known_regressions
关于内核发布，值得一提的是Andrew Morton在linux-kernel邮件列表中如是说：
“没有人知道新内核何时会被发布，因为发布是根据已知bug的情况来决定
diff --git a/Documentation/zh_CN/arm64/booting.txt b/Documentation/zh_CN/arm64/booting.txt
index 1145bf8..c1dd968 100644
--- a/Documentation/zh_CN/arm64/booting.txt
+++ b/Documentation/zh_CN/arm64/booting.txt
@@ -8,7 +8,7 @@
M: Will Deacon <will.deacon@arm.com>
zh_CN: Fu Wei <wefu@redhat.com>
-C: 1926e54f115725a9248d0c4c65c22acaf94de4c4
+C: 55f058e7574c3615dea4615573a19bdb258696c6
---------------------------------------------------------------------
Documentation/arm64/booting.txt 的中文翻译
@@ -20,7 +20,7 @@
中文版维护者： 傅炜 Fu Wei <wefu@redhat.com>
中文版翻译者： 傅炜 Fu Wei <wefu@redhat.com>
中文版校译者： 傅炜 Fu Wei <wefu@redhat.com>
-本文翻译提交时的 Git 检出点为： 1926e54f115725a9248d0c4c65c22acaf94de4c4
+本文翻译提交时的 Git 检出点为： 55f058e7574c3615dea4615573a19bdb258696c6
以下为正文
---------------------------------------------------------------------
@@ -125,18 +125,22 @@
1 - 4K
2 - 16K
3 - 64K
- 位 3-63: 保留。
+ 位 3: 内核物理位置
+ 0 - 2MB 对齐基址应尽量靠近内存起始处，因为
+ 其基址以下的内存无法通过线性映射访问
+ 1 - 2MB 对齐基址可以在物理内存的任意位置
+ 位 4-63: 保留。
- 当 image_size 为零时，引导装载程序应试图在内核映像末尾之后尽可能
多地保留空闲内存供内核直接使用。对内存空间的需求量因所选定的内核
特性而异, 并无实际限制。
-内核映像必须被放置在靠近可用系统内存起始的 2MB 对齐为基址的
-text_offset 字节处，并从该处被调用。当前，对 Linux 来说在此基址以下的
-内存是无法使用的，因此强烈建议将系统内存的起始作为这个基址。2MB 对齐
-基址和内核映像起始地址之间的区域对于内核来说没有特殊意义，且可能被
-用于其他目的。
+内核映像必须被放置在任意一个可用系统内存 2MB 对齐基址的 text_offset
+字节处，并从该处被调用。2MB 对齐基址和内核映像起始地址之间的区域对于
+内核来说没有特殊意义，且可能被用于其他目的。
从映像起始地址算起，最少必须准备 image_size 字节的空闲内存供内核使用。
+注： v4.6 之前的版本无法使用内核映像物理偏移以下的内存，所以当时建议
+将映像尽量放置在靠近系统内存起始的地方。
任何提供给内核的内存（甚至在映像起始地址之前），若未从内核中标记为保留
(如在设备树（dtb）的 memreserve 区域），都将被认为对内核是可用。
diff --git a/Kbuild b/Kbuild
index f55cefd..3d0ae15 100644
--- a/Kbuild
+++ b/Kbuild
@@ -5,6 +5,7 @@
# 2) Generate timeconst.h
# 3) Generate asm-offsets.h (may need bounds.h and timeconst.h)
# 4) Check for missing system calls
+# 5) Generate constants.py (may need bounds.h)
# Default sed regexp - multiline due to syntax constraints
define sed-y
@@ -96,5 +97,14 @@
missing-syscalls: scripts/checksyscalls.sh $(offsets-file) FORCE
$(call cmd,syscalls)
+#####
+# 5) Generate constants for Python GDB integration
+#
+
+extra-$(CONFIG_GDB_SCRIPTS) += build_constants_py
+
+build_constants_py: $(obj)/$(timeconst-file) $(obj)/$(bounds-file)
+ @$(MAKE) $(build)=scripts/gdb/linux $@
+
# Keep these three files during make clean
no-clean-files := $(bounds-file) $(offsets-file) $(timeconst-file)
diff --git a/MAINTAINERS b/MAINTAINERS
index 804bc4f..3302006 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -175,7 +175,6 @@
8250/16?50 (AND CLONE UARTS) SERIAL DRIVER
M: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
L: linux-serial@vger.kernel.org
-W: http://serial.sourceforge.net
S: Maintained
T: git git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty.git
F: drivers/tty/serial/8250*
@@ -777,6 +776,15 @@
F: drivers/android/
F: drivers/staging/android/
+ANDROID ION DRIVER
+M: Laura Abbott <labbott@redhat.com>
+M: Sumit Semwal <sumit.semwal@linaro.org>
+L: devel@driverdev.osuosl.org
+S: Supported
+F: drivers/staging/android/ion
+F: drivers/staging/android/uapi/ion.h
+F: drivers/staging/android/uapi/ion_test.h
+
AOA (Apple Onboard Audio) ALSA DRIVER
M: Johannes Berg <johannes@sipsolutions.net>
L: linuxppc-dev@lists.ozlabs.org
@@ -848,6 +856,12 @@
F: drivers/net/arcnet/
F: include/uapi/linux/if_arcnet.h
+ARC PGU DRM DRIVER
+M: Alexey Brodkin <abrodkin@synopsys.com>
+S: Supported
+F: drivers/gpu/drm/arc/
+F: Documentation/devicetree/bindings/display/snps,arcpgu.txt
+
ARM HDLCD DRM DRIVER
M: Liviu Dudau <liviu.dudau@arm.com>
S: Supported
@@ -977,7 +991,7 @@
L: linux-arm-kernel@axis.com
F: arch/arm/mach-artpec
F: arch/arm/boot/dts/artpec6*
-F: drivers/clk/clk-artpec6.c
+F: drivers/clk/axis
ARM/ASPEED MACHINE SUPPORT
M: Joel Stanley <joel@jms.id.au>
@@ -1937,6 +1951,16 @@
S: Maintained
F: drivers/platform/x86/asus-wireless.c
+ASYMMETRIC KEYS
+M: David Howells <dhowells@redhat.com>
+L: keyrings@vger.kernel.org
+S: Maintained
+F: Documentation/crypto/asymmetric-keys.txt
+F: include/linux/verification.h
+F: include/crypto/public_key.h
+F: include/crypto/pkcs7.h
+F: crypto/asymmetric_keys/
+
ASYNCHRONOUS TRANSFERS/TRANSFORMS (IOAT) API
R: Dan Williams <dan.j.williams@intel.com>
W: http://sourceforge.net/projects/xscaleiop
@@ -2048,6 +2072,11 @@
S: Supported
F: drivers/tty/serial/atmel_serial.c
+ATMEL AT91 SAMA5D2-Compatible Shutdown Controller
+M: Nicolas Ferre <nicolas.ferre@atmel.com>
+S: Supported
+F: drivers/power/reset/at91-sama5d2_shdwc.c
+
ATMEL SAMA5D2 ADC DRIVER
M: Ludovic Desroches <ludovic.desroches@atmel.com>
L: linux-iio@vger.kernel.org
@@ -3824,9 +3853,25 @@
S: Maintained
F: drivers/gpu/drm/
F: drivers/gpu/vga/
+F: Documentation/DocBook/gpu.*
F: include/drm/
F: include/uapi/drm/
+DRM DRIVER FOR AST SERVER GRAPHICS CHIPS
+M: Dave Airlie <airlied@redhat.com>
+S: Odd Fixes
+F: drivers/gpu/drm/ast/
+
+DRM DRIVER FOR BOCHS VIRTUAL GPU
+M: Gerd Hoffmann <kraxel@redhat.com>
+S: Odd Fixes
+F: drivers/gpu/drm/bochs/
+
+DRM DRIVER FOR QEMU'S CIRRUS DEVICE
+M: Dave Airlie <airlied@redhat.com>
+S: Odd Fixes
+F: drivers/gpu/drm/cirrus/
+
RADEON and AMDGPU DRM DRIVERS
M: Alex Deucher <alexander.deucher@amd.com>
M: Christian König <christian.koenig@amd.com>
@@ -3834,9 +3879,9 @@
T: git git://people.freedesktop.org/~agd5f/linux
S: Supported
F: drivers/gpu/drm/radeon/
-F: include/uapi/drm/radeon*
+F: include/uapi/drm/radeon_drm.h
F: drivers/gpu/drm/amd/
-F: include/uapi/drm/amdgpu*
+F: include/uapi/drm/amdgpu_drm.h
DRM PANEL DRIVERS
M: Thierry Reding <thierry.reding@gmail.com>
@@ -3859,7 +3904,7 @@
S: Supported
F: drivers/gpu/drm/i915/
F: include/drm/i915*
-F: include/uapi/drm/i915*
+F: include/uapi/drm/i915_drm.h
DRM DRIVERS FOR ATMEL HLCDC
M: Boris Brezillon <boris.brezillon@free-electrons.com>
@@ -3868,6 +3913,13 @@
F: drivers/gpu/drm/atmel-hlcdc/
F: Documentation/devicetree/bindings/drm/atmel/
+DRM DRIVERS FOR ALLWINNER A10
+M: Maxime Ripard <maxime.ripard@free-electrons.com>
+L: dri-devel@lists.freedesktop.org
+S: Supported
+F: drivers/gpu/drm/sun4i/
+F: Documentation/devicetree/bindings/display/sunxi/sun4i-drm.txt
+
DRM DRIVERS FOR EXYNOS
M: Inki Dae <inki.dae@samsung.com>
M: Joonyoung Shim <jy0922.shim@samsung.com>
@@ -3877,8 +3929,8 @@
T: git git://git.kernel.org/pub/scm/linux/kernel/git/daeinki/drm-exynos.git
S: Supported
F: drivers/gpu/drm/exynos/
-F: include/drm/exynos*
-F: include/uapi/drm/exynos*
+F: include/uapi/drm/exynos_drm.h
+F: Documentation/devicetree/bindings/display/exynos/
DRM DRIVERS FOR FREESCALE DCU
M: Stefan Agner <stefan@agner.ch>
@@ -3887,6 +3939,7 @@
S: Supported
F: drivers/gpu/drm/fsl-dcu/
F: Documentation/devicetree/bindings/display/fsl,dcu.txt
+F: Documentation/devicetree/bindings/display/fsl,tcon.txt
F: Documentation/devicetree/bindings/display/panel/nec,nl4827hc19_05b.txt
DRM DRIVERS FOR FREESCALE IMX
@@ -3902,12 +3955,45 @@
L: dri-devel@lists.freedesktop.org
T: git git://github.com/patjak/drm-gma500
S: Maintained
-F: drivers/gpu/drm/gma500
-F: include/drm/gma500*
+F: drivers/gpu/drm/gma500/
+
+DRM DRIVERS FOR HISILICON
+M: Xinliang Liu <z.liuxinliang@hisilicon.com>
+R: Xinwei Kong <kong.kongxinwei@hisilicon.com>
+R: Chen Feng <puck.chen@hisilicon.com>
+L: dri-devel@lists.freedesktop.org
+T: git git://github.com/xin3liang/linux.git
+S: Maintained
+F: drivers/gpu/drm/hisilicon/
+F: Documentation/devicetree/bindings/display/hisilicon/
+
+DRM DRIVER FOR INTEL I810 VIDEO CARDS
+S: Orphan / Obsolete
+F: drivers/gpu/drm/i810/
+F: include/uapi/drm/i810_drm.h
+
+DRM DRIVER FOR MSM ADRENO GPU
+M: Rob Clark <robdclark@gmail.com>
+L: linux-arm-msm@vger.kernel.org
+L: dri-devel@lists.freedesktop.org
+L: freedreno@lists.freedesktop.org
+T: git git://people.freedesktop.org/~robclark/linux
+S: Maintained
+F: drivers/gpu/drm/msm/
+F: include/uapi/drm/msm_drm.h
+F: Documentation/devicetree/bindings/display/msm/
+
+DRM DRIVER FOR NVIDIA GEFORCE/QUADRO GPUS
+M: Ben Skeggs <bskeggs@redhat.com>
+L: dri-devel@lists.freedesktop.org
+L: nouveau@lists.freedesktop.org
+T: git git://github.com/skeggsb/linux
+S: Supported
+F: drivers/gpu/drm/nouveau/
+F: include/uapi/drm/nouveau_drm.h
DRM DRIVERS FOR NVIDIA TEGRA
M: Thierry Reding <thierry.reding@gmail.com>
-M: Terje Bergström <tbergstrom@nvidia.com>
L: dri-devel@lists.freedesktop.org
L: linux-tegra@vger.kernel.org
T: git git://anongit.freedesktop.org/tegra/linux.git
@@ -3918,22 +4004,54 @@
F: include/uapi/drm/tegra_drm.h
F: Documentation/devicetree/bindings/display/tegra/nvidia,tegra20-host1x.txt
+DRM DRIVER FOR MATROX G200/G400 GRAPHICS CARDS
+S: Orphan / Obsolete
+F: drivers/gpu/drm/mga/
+F: include/uapi/drm/mga_drm.h
+
+DRM DRIVER FOR MGA G200 SERVER GRAPHICS CHIPS
+M: Dave Airlie <airlied@redhat.com>
+S: Odd Fixes
+F: drivers/gpu/drm/mgag200/
+
+DRM DRIVER FOR RAGE 128 VIDEO CARDS
+S: Orphan / Obsolete
+F: drivers/gpu/drm/r128/
+F: include/uapi/drm/r128_drm.h
+
DRM DRIVERS FOR RENESAS
M: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
L: dri-devel@lists.freedesktop.org
L: linux-renesas-soc@vger.kernel.org
-T: git git://people.freedesktop.org/~airlied/linux
+T: git git://linuxtv.org/pinchartl/fbdev
S: Supported
F: drivers/gpu/drm/rcar-du/
F: drivers/gpu/drm/shmobile/
F: include/linux/platform_data/shmob_drm.h
+F: Documentation/devicetree/bindings/display/renesas,du.txt
+
+DRM DRIVER FOR QXL VIRTUAL GPU
+M: Dave Airlie <airlied@redhat.com>
+S: Odd Fixes
+F: drivers/gpu/drm/qxl/
+F: include/uapi/drm/qxl_drm.h
DRM DRIVERS FOR ROCKCHIP
M: Mark Yao <mark.yao@rock-chips.com>
L: dri-devel@lists.freedesktop.org
S: Maintained
F: drivers/gpu/drm/rockchip/
-F: Documentation/devicetree/bindings/display/rockchip*
+F: Documentation/devicetree/bindings/display/rockchip/
+
+DRM DRIVER FOR SAVAGE VIDEO CARDS
+S: Orphan / Obsolete
+F: drivers/gpu/drm/savage/
+F: include/uapi/drm/savage_drm.h
+
+DRM DRIVER FOR SIS VIDEO CARDS
+S: Orphan / Obsolete
+F: drivers/gpu/drm/sis/
+F: include/uapi/drm/sis_drm.h
DRM DRIVERS FOR STI
M: Benjamin Gaignard <benjamin.gaignard@linaro.org>
@@ -3944,14 +4062,43 @@
F: drivers/gpu/drm/sti
F: Documentation/devicetree/bindings/display/st,stih4xx.txt
+DRM DRIVER FOR TDFX VIDEO CARDS
+S: Orphan / Obsolete
+F: drivers/gpu/drm/tdfx/
+
+DRM DRIVER FOR USB DISPLAYLINK VIDEO ADAPTERS
+M: Dave Airlie <airlied@redhat.com>
+S: Odd Fixes
+F: drivers/gpu/drm/udl/
+
DRM DRIVERS FOR VIVANTE GPU IP
M: Lucas Stach <l.stach@pengutronix.de>
R: Russell King <linux+etnaviv@armlinux.org.uk>
R: Christian Gmeiner <christian.gmeiner@gmail.com>
L: dri-devel@lists.freedesktop.org
S: Maintained
-F: drivers/gpu/drm/etnaviv
-F: Documentation/devicetree/bindings/display/etnaviv
+F: drivers/gpu/drm/etnaviv/
+F: include/uapi/drm/etnaviv_drm.h
+F: Documentation/devicetree/bindings/display/etnaviv/
+
+DRM DRIVER FOR VMWARE VIRTUAL GPU
+M: "VMware Graphics" <linux-graphics-maintainer@vmware.com>
+M: Sinclair Yeh <syeh@vmware.com>
+M: Thomas Hellstrom <thellstrom@vmware.com>
+L: dri-devel@lists.freedesktop.org
+T: git git://people.freedesktop.org/~syeh/repos_linux
+T: git git://people.freedesktop.org/~thomash/linux
+S: Supported
+F: drivers/gpu/drm/vmwgfx/
+F: include/uapi/drm/vmwgfx_drm.h
+
+DRM DRIVERS FOR VC4
+M: Eric Anholt <eric@anholt.net>
+T: git git://github.com/anholt/linux
+S: Supported
+F: drivers/gpu/drm/vc4/
+F: include/uapi/drm/vc4_drm.h
+F: Documentation/devicetree/bindings/display/brcm,bcm-vc4.txt
DSBR100 USB FM RADIO DRIVER
M: Alexey Klimov <klimov.linux@gmail.com>
@@ -4444,6 +4591,12 @@
F: drivers/video/fbdev/exynos/exynos_mipi*
F: include/video/exynos_mipi*
+EZchip NPS platform support
+M: Noam Camus <noamc@ezchip.com>
+S: Supported
+F: arch/arc/plat-eznps
+F: arch/arc/boot/dts/eznps.dts
+
F71805F HARDWARE MONITORING DRIVER
M: Jean Delvare <jdelvare@suse.com>
L: linux-hwmon@vger.kernel.org
@@ -4726,6 +4879,7 @@
M: Timur Tabi <timur@tabi.org>
M: Nicolin Chen <nicoleotsuka@gmail.com>
M: Xiubo Li <Xiubo.Lee@gmail.com>
+R: Fabio Estevam <fabio.estevam@nxp.com>
L: alsa-devel@alsa-project.org (moderated for non-subscribers)
L: linuxppc-dev@lists.ozlabs.org
S: Maintained
@@ -4735,6 +4889,7 @@
FREESCALE QORIQ MANAGEMENT COMPLEX DRIVER
M: "J. German Rivera" <German.Rivera@freescale.com>
+M: Stuart Yoder <stuart.yoder@nxp.com>
L: linux-kernel@vger.kernel.org
S: Maintained
F: drivers/staging/fsl-mc/
@@ -4772,7 +4927,7 @@
F2FS FILE SYSTEM
M: Jaegeuk Kim <jaegeuk@kernel.org>
M: Changman Lee <cm224.lee@samsung.com>
-R: Chao Yu <chao2.yu@samsung.com>
+R: Chao Yu <yuchao0@huawei.com>
L: linux-f2fs-devel@lists.sourceforge.net
W: http://en.wikipedia.org/wiki/F2FS
T: git git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs.git
@@ -4838,6 +4993,7 @@
GDB KERNEL DEBUGGING HELPER SCRIPTS
M: Jan Kiszka <jan.kiszka@siemens.com>
+M: Kieran Bingham <kieran@bingham.xyz>
S: Supported
F: scripts/gdb/
@@ -5341,6 +5497,7 @@
M: Peter Rosin <peda@axentia.se>
L: linux-i2c@vger.kernel.org
S: Maintained
+F: Documentation/i2c/i2c-topology
F: Documentation/i2c/muxes/
F: Documentation/devicetree/bindings/i2c/i2c-mux*
F: drivers/i2c/i2c-mux.c
@@ -5605,7 +5762,7 @@
M: Jonathan Cameron <jic23@kernel.org>
R: Hartmut Knaack <knaack.h@gmx.de>
R: Lars-Peter Clausen <lars@metafoo.de>
-R: Peter Meerwald <pmeerw@pmeerw.net>
+R: Peter Meerwald-Stadler <pmeerw@pmeerw.net>
L: linux-iio@vger.kernel.org
S: Maintained
F: drivers/iio/
@@ -5889,6 +6046,7 @@
INTEL WIRELESS WIFI LINK (iwlwifi)
M: Johannes Berg <johannes.berg@intel.com>
M: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
+M: Luca Coelho <luciano.coelho@intel.com>
M: Intel Linux Wireless <linuxwifi@intel.com>
L: linux-wireless@vger.kernel.org
W: http://intellinuxwireless.org
@@ -6054,6 +6212,13 @@
F: kernel/irq/irqdomain.c
F: kernel/irq/msi.c
+ISA
+M: William Breathitt Gray <vilhelm.gray@gmail.com>
+S: Maintained
+F: Documentation/isa.txt
+F: drivers/base/isa.c
+F: include/linux/isa.h
+
ISAPNP
M: Jaroslav Kysela <perex@perex.cz>
S: Maintained
@@ -6234,7 +6399,7 @@
F: arch/*/include/asm/kasan.h
F: arch/*/mm/kasan_init*
F: Documentation/kasan.txt
-F: include/linux/kasan.h
+F: include/linux/kasan*.h
F: lib/test_kasan.c
F: mm/kasan/
F: scripts/Makefile.kasan
@@ -6403,6 +6568,8 @@
F: Documentation/security/keys.txt
F: include/linux/key.h
F: include/linux/key-type.h
+F: include/linux/keyctl.h
+F: include/uapi/linux/keyctl.h
F: include/keys/
F: security/keys/
@@ -6485,7 +6652,7 @@
F: include/net/l3mdev.h
LANTIQ MIPS ARCHITECTURE
-M: John Crispin <blogic@openwrt.org>
+M: John Crispin <john@phrozen.org>
L: linux-mips@linux-mips.org
S: Maintained
F: arch/mips/lantiq
@@ -6667,6 +6834,19 @@
S: Supported
F: Documentation/powerpc/
F: arch/powerpc/
+F: drivers/char/tpm/tpm_ibmvtpm*
+F: drivers/crypto/nx/
+F: drivers/crypto/vmx/
+F: drivers/net/ethernet/ibm/ibmveth.*
+F: drivers/net/ethernet/ibm/ibmvnic.*
+F: drivers/pci/hotplug/rpa*
+F: drivers/scsi/ibmvscsi/
+N: opal
+N: /pmac
+N: powermac
+N: powernv
+N: [^a-z0-9]ps3
+N: pseries
LINUX FOR POWER MACINTOSH
M: Benjamin Herrenschmidt <benh@kernel.crashing.org>
@@ -6962,6 +7142,8 @@
M: Russell King <rmk+kernel@armlinux.org.uk>
S: Maintained
F: drivers/gpu/drm/armada/
+F: include/uapi/drm/armada_drm.h
+F: Documentation/devicetree/bindings/display/armada/
MARVELL 88E6352 DSA support
M: Guenter Roeck <linux@roeck-us.net>
@@ -7326,6 +7508,15 @@
F: Documentation/mips/
F: arch/mips/
+MIPS/LOONGSON1 ARCHITECTURE
+M: Keguang Zhang <keguang.zhang@gmail.com>
+L: linux-mips@linux-mips.org
+S: Maintained
+F: arch/mips/loongson32/
+F: arch/mips/include/asm/mach-loongson32/
+F: drivers/*/*loongson1*
+F: drivers/*/*/*loongson1*
+
MIROSOUND PCM20 FM RADIO RECEIVER DRIVER
M: Hans Verkuil <hverkuil@xs4all.nl>
L: linux-media@vger.kernel.org
@@ -7593,10 +7784,10 @@
L: linux-scsi@vger.kernel.org
S: Maintained
F: Documentation/scsi/g_NCR5380.txt
+F: Documentation/scsi/dtc3x80.txt
F: drivers/scsi/NCR5380.*
F: drivers/scsi/arm/cumana_1.c
F: drivers/scsi/arm/oak.c
-F: drivers/scsi/atari_NCR5380.c
F: drivers/scsi/atari_scsi.*
F: drivers/scsi/dmx3191d.c
F: drivers/scsi/dtc.*
@@ -7848,6 +8039,7 @@
M: Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>
L: linux-nilfs@vger.kernel.org
W: http://nilfs.sourceforge.net/
+W: http://nilfs.osdn.jp/
T: git git://github.com/konis/nilfs2.git
S: Supported
F: Documentation/filesystems/nilfs2.txt
@@ -8228,7 +8420,6 @@
OPENRISC ARCHITECTURE
M: Jonas Bonn <jonas@southpole.se>
W: http://openrisc.net
-L: linux@lists.openrisc.net (moderated for non-subscribers)
S: Maintained
T: git git://openrisc.net/~jonas/linux
F: arch/openrisc/
@@ -8349,7 +8540,6 @@
PANASONIC MN10300/AM33/AM34 PORT
M: David Howells <dhowells@redhat.com>
-M: Koichi Yasutake <yasutake.koichi@jp.panasonic.com>
L: linux-am33-list@redhat.com (moderated for non-subscribers)
W: ftp://ftp.redhat.com/pub/redhat/gnupro/AM33/
S: Maintained
@@ -8783,7 +8973,6 @@
PIN CONTROLLER - ST SPEAR
M: Viresh Kumar <vireshk@kernel.org>
-L: spear-devel@list.st.com
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
W: http://www.st.com/spear
S: Maintained
@@ -9245,7 +9434,7 @@
F: drivers/video/fbdev/aty/aty128fb.c
RALINK MIPS ARCHITECTURE
-M: John Crispin <blogic@openwrt.org>
+M: John Crispin <john@phrozen.org>
L: linux-mips@linux-mips.org
S: Maintained
F: arch/mips/ralink
@@ -9808,6 +9997,7 @@
SYSTEM TRACE MODULE CLASS
M: Alexander Shishkin <alexander.shishkin@linux.intel.com>
S: Maintained
+T: git git://git.kernel.org/pub/scm/linux/kernel/git/ash/stm.git
F: Documentation/trace/stm.txt
F: drivers/hwtracing/stm/
F: include/linux/stm.h
@@ -9987,7 +10177,6 @@
SECURE DIGITAL HOST CONTROLLER INTERFACE (SDHCI) ST SPEAR DRIVER
M: Viresh Kumar <vireshk@kernel.org>
-L: spear-devel@list.st.com
L: linux-mmc@vger.kernel.org
S: Maintained
F: drivers/mmc/host/sdhci-spear.c
@@ -10025,6 +10214,12 @@
S: Supported
F: security/apparmor/
+LOADPIN SECURITY MODULE
+M: Kees Cook <keescook@chromium.org>
+T: git git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git lsm/loadpin
+S: Supported
+F: security/loadpin/
+
YAMA SECURITY MODULE
M: Kees Cook <keescook@chromium.org>
T: git git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git yama/tip
@@ -10544,7 +10739,6 @@
SPEAR PLATFORM SUPPORT
M: Viresh Kumar <vireshk@kernel.org>
M: Shiraz Hashim <shiraz.linux.kernel@gmail.com>
-L: spear-devel@list.st.com
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
W: http://www.st.com/spear
S: Maintained
@@ -10553,7 +10747,6 @@
SPEAR CLOCK FRAMEWORK SUPPORT
M: Viresh Kumar <vireshk@kernel.org>
-L: spear-devel@list.st.com
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
W: http://www.st.com/spear
S: Maintained
@@ -10996,10 +11189,11 @@
S: Supported
F: drivers/clk/tegra/
-TEGRA DMA DRIVER
+TEGRA DMA DRIVERS
M: Laxman Dewangan <ldewangan@nvidia.com>
+M: Jon Hunter <jonathanh@nvidia.com>
S: Supported
-F: drivers/dma/tegra20-apb-dma.c
+F: drivers/dma/tegra*
TEGRA I2C DRIVER
M: Laxman Dewangan <ldewangan@nvidia.com>
@@ -11338,6 +11532,14 @@
S: Odd Fixes
F: drivers/media/pci/tw68/
+TW686X VIDEO4LINUX DRIVER
+M: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
+L: linux-media@vger.kernel.org
+T: git git://linuxtv.org/media_tree.git
+W: http://linuxtv.org
+S: Maintained
+F: drivers/media/pci/tw686x/
+
TPM DEVICE DRIVER
M: Peter Huewe <peterhuewe@gmx.de>
M: Marcel Selhorst <tpmdd@selhorst.net>
@@ -11456,7 +11658,8 @@
F: fs/ubifs/
UCLINUX (M68KNOMMU AND COLDFIRE)
-M: Greg Ungerer <gerg@uclinux.org>
+M: Greg Ungerer <gerg@linux-m68k.org>
+W: http://www.linux-m68k.org/
W: http://www.uclinux.org/
L: linux-m68k@lists.linux-m68k.org
L: uclinux-dev@uclinux.org (subscribers-only)
@@ -12246,6 +12449,12 @@
F: kernel/workqueue.c
F: Documentation/workqueue.txt
+X-POWERS MULTIFUNCTION PMIC DEVICE DRIVERS
+M: Chen-Yu Tsai <wens@csie.org>
+L: linux-kernel@vger.kernel.org
+S: Maintained
+N: axp[128]
+
X.25 NETWORK LAYER
M: Andrew Hendry <andrew.hendry@gmail.com>
L: linux-x25@vger.kernel.org
diff --git a/README b/README
index afc4f0d..e8c8a6d 100644
--- a/README
+++ b/README
@@ -2,7 +2,7 @@
These are the release notes for Linux version 4. Read them carefully,
as they tell you what this is all about, explain how to install the
-kernel, and what to do if something goes wrong.
+kernel, and what to do if something goes wrong.
WHAT IS LINUX?
@@ -16,7 +16,7 @@
and multistack networking including IPv4 and IPv6.
It is distributed under the GNU General Public License - see the
- accompanying COPYING file for more details.
+ accompanying COPYING file for more details.
ON WHAT HARDWARE DOES IT RUN?
@@ -44,7 +44,7 @@
system: there are much better sources available.
- There are various README files in the Documentation/ subdirectory:
- these typically contain kernel-specific installation notes for some
+ these typically contain kernel-specific installation notes for some
drivers for example. See Documentation/00-INDEX for a list of what
is contained in each file. Please read the Changes file, as it
contains information about the problems, which may result by upgrading
@@ -276,7 +276,7 @@
To have the build system also tell the reason for the rebuild of each
target, use "V=2". The default is "V=0".
- - Keep a backup kernel handy in case something goes wrong. This is
+ - Keep a backup kernel handy in case something goes wrong. This is
especially true for the development releases, since each new release
contains new code which has not been debugged. Make sure you keep a
backup of the modules corresponding to that kernel, as well. If you
@@ -290,7 +290,7 @@
- In order to boot your new kernel, you'll need to copy the kernel
image (e.g. .../linux/arch/i386/boot/bzImage after compilation)
- to the place where your regular bootable kernel is found.
+ to the place where your regular bootable kernel is found.
- Booting a kernel directly from a floppy without the assistance of a
bootloader such as LILO, is no longer supported.
@@ -303,10 +303,10 @@
to update the loading map! If you don't, you won't be able to boot
the new kernel image.
- Reinstalling LILO is usually a matter of running /sbin/lilo.
+ Reinstalling LILO is usually a matter of running /sbin/lilo.
You may wish to edit /etc/lilo.conf to specify an entry for your
old kernel image (say, /vmlinux.old) in case the new one does not
- work. See the LILO docs for more information.
+ work. See the LILO docs for more information.
After reinstalling LILO, you should be all set. Shutdown the system,
reboot, and enjoy!
@@ -314,9 +314,9 @@
If you ever need to change the default root device, video mode,
ramdisk size, etc. in the kernel image, use the 'rdev' program (or
alternatively the LILO boot options when appropriate). No need to
- recompile the kernel to change these parameters.
+ recompile the kernel to change these parameters.
- - Reboot with the new kernel and enjoy.
+ - Reboot with the new kernel and enjoy.
IF SOMETHING GOES WRONG:
@@ -383,7 +383,7 @@
is followed by a function with a higher address you will find the one
you want. In fact, it may be a good idea to include a bit of
"context" in your problem report, giving a few lines around the
- interesting one.
+ interesting one.
If you for some reason cannot do the above (you have a pre-compiled
kernel image or similar), telling me as much about your setup as
diff --git a/arch/Kconfig b/arch/Kconfig
index 81869a5..b16e74e 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -187,7 +187,11 @@
config HAVE_KPROBES_ON_FTRACE
bool
+config HAVE_NMI
+ bool
+
config HAVE_NMI_WATCHDOG
+ depends on HAVE_NMI
bool
#
# An arch should select this if it provides all these things:
@@ -517,6 +521,11 @@
- ARCH_MMAP_RND_BITS_MIN
- ARCH_MMAP_RND_BITS_MAX
+config HAVE_EXIT_THREAD
+ bool
+ help
+ An architecture implements exit_thread.
+
config ARCH_MMAP_RND_BITS_MIN
int
@@ -638,4 +647,7 @@
config ARCH_NO_COHERENT_DMA_MMAP
bool
+config CPU_NO_EFFICIENT_FFS
+ def_bool n
+
source "kernel/gcov/Kconfig"
diff --git a/arch/alpha/Kconfig b/arch/alpha/Kconfig
index fe99f89..7f312d8 100644
--- a/arch/alpha/Kconfig
+++ b/arch/alpha/Kconfig
@@ -26,6 +26,7 @@
select MODULES_USE_ELF_RELA
select ODD_RT_SIGACTION
select OLD_SIGSUSPEND
+ select CPU_NO_EFFICIENT_FFS if !ALPHA_EV67
help
The Alpha is a 64-bit general-purpose processor designed and
marketed by the Digital Equipment Corporation of blessed memory,
diff --git a/arch/alpha/kernel/pci-sysfs.c b/arch/alpha/kernel/pci-sysfs.c
index 99e8d47..92c0d46 100644
--- a/arch/alpha/kernel/pci-sysfs.c
+++ b/arch/alpha/kernel/pci-sysfs.c
@@ -77,10 +77,10 @@
if (i >= PCI_ROM_RESOURCE)
return -ENODEV;
- if (!__pci_mmap_fits(pdev, i, vma, sparse))
+ if (res->flags & IORESOURCE_MEM && iomem_is_exclusive(res->start))
return -EINVAL;
- if (iomem_is_exclusive(res->start))
+ if (!__pci_mmap_fits(pdev, i, vma, sparse))
return -EINVAL;
pcibios_resource_to_bus(pdev->bus, &bar, res);
diff --git a/arch/alpha/kernel/process.c b/arch/alpha/kernel/process.c
index 84d1326..b483156 100644
--- a/arch/alpha/kernel/process.c
+++ b/arch/alpha/kernel/process.c
@@ -210,14 +210,6 @@
}
EXPORT_SYMBOL(start_thread);
-/*
- * Free current thread data structures etc..
- */
-void
-exit_thread(void)
-{
-}
-
void
flush_thread(void)
{
diff --git a/arch/arc/Kconfig b/arch/arc/Kconfig
index a876743..0dcbacf 100644
--- a/arch/arc/Kconfig
+++ b/arch/arc/Kconfig
@@ -10,8 +10,9 @@
def_bool y
select ARCH_SUPPORTS_ATOMIC_RMW if ARC_HAS_LLSC
select BUILDTIME_EXTABLE_SORT
- select COMMON_CLK
+ select CLKSRC_OF
select CLONE_BACKWARDS
+ select COMMON_CLK
select GENERIC_ATOMIC64
select GENERIC_CLOCKEVENTS
select GENERIC_FIND_FIRST_BIT
@@ -30,6 +31,7 @@
select HAVE_MOD_ARCH_SPECIFIC if ARC_DW2_UNWIND
select HAVE_OPROFILE
select HAVE_PERF_EVENTS
+ select HANDLE_DOMAIN_IRQ
select IRQ_DOMAIN
select MODULES_USE_ELF_RELA
select NO_BOOTMEM
@@ -95,6 +97,7 @@
source "arch/arc/plat-tb10x/Kconfig"
source "arch/arc/plat-axs10x/Kconfig"
#New platform adds here
+source "arch/arc/plat-eznps/Kconfig"
endmenu
@@ -104,6 +107,7 @@
config ISA_ARCOMPACT
bool "ARCompact ISA"
+ select CPU_NO_EFFICIENT_FFS
help
The original ARC ISA of ARC600/700 cores
@@ -490,6 +494,17 @@
config ARC_PLAT_NEEDS_PHYS_TO_DMA
bool
+config ARC_KVADDR_SIZE
+ int "Kernel Virtaul Address Space size (MB)"
+ range 0 512
+ default "256"
+ help
+ The kernel address space is carved out of 256MB of translated address
+ space for catering to vmalloc, modules, pkmap, fixmap. This however may
+ not suffice vmalloc requirements of a 4K CPU EZChip system. So allow
+ this to be stretched to 512 MB (by extending into the reserved
+ kernel-user gutter)
+
config ARC_CURR_IN_REG
bool "Dedicate Register r25 for current_task pointer"
default y
diff --git a/arch/arc/Makefile b/arch/arc/Makefile
index def69e3..02fabef 100644
--- a/arch/arc/Makefile
+++ b/arch/arc/Makefile
@@ -115,6 +115,11 @@
core-$(CONFIG_ARC_PLAT_SIM) += arch/arc/plat-sim/
core-$(CONFIG_ARC_PLAT_TB10X) += arch/arc/plat-tb10x/
core-$(CONFIG_ARC_PLAT_AXS10X) += arch/arc/plat-axs10x/
+core-$(CONFIG_ARC_PLAT_EZNPS) += arch/arc/plat-eznps/
+
+ifdef CONFIG_ARC_PLAT_EZNPS
+KBUILD_CPPFLAGS += -I$(srctree)/arch/arc/plat-eznps/include
+endif
drivers-$(CONFIG_OPROFILE) += arch/arc/oprofile/
diff --git a/arch/arc/boot/dts/abilis_tb10x.dtsi b/arch/arc/boot/dts/abilis_tb10x.dtsi
index cfb5052..de53f5c 100644
--- a/arch/arc/boot/dts/abilis_tb10x.dtsi
+++ b/arch/arc/boot/dts/abilis_tb10x.dtsi
@@ -35,6 +35,20 @@
};
};
+ /* TIMER0 with interrupt for clockevent */
+ timer0 {
+ compatible = "snps,arc-timer";
+ interrupts = <3>;
+ interrupt-parent = <&intc>;
+ clocks = <&cpu_clk>;
+ };
+
+ /* TIMER1 for free running clocksource */
+ timer1 {
+ compatible = "snps,arc-timer";
+ clocks = <&cpu_clk>;
+ };
+
soc100 {
#address-cells = <1>;
#size-cells = <1>;
@@ -112,7 +126,7 @@
chan_allocation_order = <0>;
chan_priority = <1>;
block_size = <0x7ff>;
- data_width = <2>;
+ data-width = <4>;
clocks = <&ahb_clk>;
clock-names = "hclk";
};
diff --git a/arch/arc/boot/dts/axc001.dtsi b/arch/arc/boot/dts/axc001.dtsi
index 420dcfd..3e02f15 100644
--- a/arch/arc/boot/dts/axc001.dtsi
+++ b/arch/arc/boot/dts/axc001.dtsi
@@ -11,6 +11,8 @@
* Note that this file only supports the 770D CPU
*/
+/include/ "skeleton.dtsi"
+
/ {
compatible = "snps,arc";
clock-frequency = <750000000>; /* 750 MHZ */
@@ -24,7 +26,13 @@
ranges = <0x00000000 0xf0000000 0x10000000>;
- cpu_intc: arc700-intc@cpu {
+ core_clk: core_clk {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <750000000>;
+ };
+
+ core_intc: arc700-intc@cpu {
compatible = "snps,arc700-intc";
interrupt-controller;
#interrupt-cells = <1>;
@@ -48,7 +56,7 @@
reg = <0>;
interrupt-controller;
#interrupt-cells = <2>;
- interrupt-parent = <&cpu_intc>;
+ interrupt-parent = <&core_intc>;
interrupts = <15>;
};
};
@@ -86,15 +94,33 @@
compatible = "snps,dw-apb-ictl";
reg = < 0xe0012000 0x200 >;
interrupt-controller;
- interrupt-parent = <&cpu_intc>;
+ interrupt-parent = <&core_intc>;
interrupts = < 7 >;
};
memory {
#address-cells = <1>;
#size-cells = <1>;
- ranges = <0x00000000 0x80000000 0x40000000>;
+ ranges = <0x00000000 0x80000000 0x20000000>;
device_type = "memory";
- reg = <0x80000000 0x20000000>; /* 512MiB */
+ reg = <0x80000000 0x1b000000>; /* (512 - 32) MiB */
+ };
+
+ reserved-memory {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+ /*
+ * We just move frame buffer area to the very end of
+ * available DDR. And even though in case of ARC770 there's
+ * no strict requirement for a frame-buffer to be in any
+ * particular location it allows us to use the same
+ * base board's DT node for ARC PGU as for ARc HS38.
+ */
+ frame_buffer: frame_buffer@9e000000 {
+ compatible = "shared-dma-pool";
+ reg = <0x9e000000 0x2000000>;
+ no-map;
+ };
};
};
diff --git a/arch/arc/boot/dts/axc003.dtsi b/arch/arc/boot/dts/axc003.dtsi
index f90fadf..378e455 100644
--- a/arch/arc/boot/dts/axc003.dtsi
+++ b/arch/arc/boot/dts/axc003.dtsi
@@ -10,6 +10,8 @@
* Device tree for AXC003 CPU card: HS38x UP configuration
*/
+/include/ "skeleton_hs.dtsi"
+
/ {
compatible = "snps,arc";
clock-frequency = <90000000>;
@@ -23,7 +25,13 @@
ranges = <0x00000000 0xf0000000 0x10000000>;
- cpu_intc: archs-intc@cpu {
+ core_clk: core_clk {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <90000000>;
+ };
+
+ core_intc: archs-intc@cpu {
compatible = "snps,archs-intc";
interrupt-controller;
#interrupt-cells = <1>;
@@ -47,7 +55,7 @@
reg = <0>;
interrupt-controller;
#interrupt-cells = <2>;
- interrupt-parent = <&cpu_intc>;
+ interrupt-parent = <&core_intc>;
interrupts = <25>;
};
};
@@ -66,7 +74,7 @@
arcpct0: pct {
compatible = "snps,archs-pct";
#interrupt-cells = <1>;
- interrupt-parent = <&cpu_intc>;
+ interrupt-parent = <&core_intc>;
interrupts = <20>;
};
};
@@ -89,7 +97,7 @@
compatible = "snps,dw-apb-ictl";
reg = < 0xe0012000 0x200 >;
interrupt-controller;
- interrupt-parent = <&cpu_intc>;
+ interrupt-parent = <&core_intc>;
interrupts = < 24 >;
};
@@ -100,4 +108,18 @@
device_type = "memory";
reg = <0x80000000 0x20000000>; /* 512MiB */
};
+
+ reserved-memory {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+ /*
+ * Move frame buffer out of IOC aperture (0x8z-0xAz).
+ */
+ frame_buffer: frame_buffer@be000000 {
+ compatible = "shared-dma-pool";
+ reg = <0xbe000000 0x2000000>;
+ no-map;
+ };
+ };
};
diff --git a/arch/arc/boot/dts/axc003_idu.dtsi b/arch/arc/boot/dts/axc003_idu.dtsi
index 06a9f29..64c94b2 100644
--- a/arch/arc/boot/dts/axc003_idu.dtsi
+++ b/arch/arc/boot/dts/axc003_idu.dtsi
@@ -10,6 +10,8 @@
* Device tree for AXC003 CPU card: HS38x2 (Dual Core) with IDU intc
*/
+/include/ "skeleton_hs_idu.dtsi"
+
/ {
compatible = "snps,arc";
clock-frequency = <90000000>;
@@ -23,7 +25,13 @@
ranges = <0x00000000 0xf0000000 0x10000000>;
- cpu_intc: archs-intc@cpu {
+ core_clk: core_clk {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <100000000>;
+ };
+
+ core_intc: archs-intc@cpu {
compatible = "snps,archs-intc";
interrupt-controller;
#interrupt-cells = <1>;
@@ -32,7 +40,7 @@
idu_intc: idu-interrupt-controller {
compatible = "snps,archs-idu-intc";
interrupt-controller;
- interrupt-parent = <&cpu_intc>;
+ interrupt-parent = <&core_intc>;
/*
* <hwirq distribution>
@@ -89,7 +97,7 @@
arcpct0: pct {
compatible = "snps,archs-pct";
#interrupt-cells = <1>;
- interrupt-parent = <&cpu_intc>;
+ interrupt-parent = <&core_intc>;
interrupts = <20>;
};
};
@@ -123,4 +131,18 @@
device_type = "memory";
reg = <0x80000000 0x20000000>; /* 512MiB */
};
+
+ reserved-memory {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+ /*
+ * Move frame buffer out of IOC aperture (0x8z-0xAz).
+ */
+ frame_buffer: frame_buffer@be000000 {
+ compatible = "shared-dma-pool";
+ reg = <0xbe000000 0x2000000>;
+ no-map;
+ };
+ };
};
diff --git a/arch/arc/boot/dts/axs10x_mb.dtsi b/arch/arc/boot/dts/axs10x_mb.dtsi
index 44a578c..d6c1bbc 100644
--- a/arch/arc/boot/dts/axs10x_mb.dtsi
+++ b/arch/arc/boot/dts/axs10x_mb.dtsi
@@ -16,7 +16,20 @@
ranges = <0x00000000 0xe0000000 0x10000000>;
interrupt-parent = <&mb_intc>;
+ i2sclk: i2sclk@100a0 {
+ compatible = "snps,axs10x-i2s-pll-clock";
+ reg = <0x100a0 0x10>;
+ clocks = <&i2spll_clk>;
+ #clock-cells = <0>;
+ };
+
clocks {
+ i2spll_clk: i2spll_clk {
+ compatible = "fixed-clock";
+ clock-frequency = <27000000>;
+ #clock-cells = <0>;
+ };
+
i2cclk: i2cclk {
compatible = "fixed-clock";
clock-frequency = <50000000>;
@@ -34,6 +47,12 @@
clock-frequency = <50000000>;
#clock-cells = <0>;
};
+
+ pguclk: pguclk {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <74440000>;
+ };
};
ethernet@0x18000 {
@@ -147,6 +166,37 @@
clocks = <&i2cclk>;
interrupts = <16>;
+ adv7511:adv7511@39{
+ compatible="adi,adv7511";
+ reg = <0x39>;
+ interrupts = <23>;
+ adi,input-depth = <8>;
+ adi,input-colorspace = "rgb";
+ adi,input-clock = "1x";
+ adi,clock-delay = <0x03>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ /* RGB/YUV input */
+ port@0 {
+ reg = <0>;
+ adv7511_input:endpoint {
+ remote-endpoint = <&pgu_output>;
+ };
+ };
+
+ /* HDMI output */
+ port@1 {
+ reg = <1>;
+ adv7511_output: endpoint {
+ remote-endpoint = <&hdmi_connector_in>;
+ };
+ };
+ };
+ };
+
eeprom@0x54{
compatible = "24c01";
reg = <0x54>;
@@ -160,6 +210,16 @@
};
};
+ hdmi0: connector {
+ compatible = "hdmi-connector";
+ type = "a";
+ port {
+ hdmi_connector_in: endpoint {
+ remote-endpoint = <&adv7511_output>;
+ };
+ };
+ };
+
gpio0:gpio@13000 {
compatible = "snps,dw-apb-gpio";
reg = <0x13000 0x1000>;
@@ -221,5 +281,19 @@
reg = <2>;
};
};
+
+ pgu@17000 {
+ compatible = "snps,arcpgu";
+ reg = <0x17000 0x400>;
+ encoder-slave = <&adv7511>;
+ clocks = <&pguclk>;
+ clock-names = "pxlclk";
+ memory-region = <&frame_buffer>;
+ port {
+ pgu_output: endpoint {
+ remote-endpoint = <&adv7511_input>;
+ };
+ };
+ };
};
};
diff --git a/arch/arc/boot/dts/eznps.dts b/arch/arc/boot/dts/eznps.dts
new file mode 100644
index 0000000..b89f6c3
--- /dev/null
+++ b/arch/arc/boot/dts/eznps.dts
@@ -0,0 +1,96 @@
+/*
+ * Copyright(c) 2015 EZchip Technologies.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ */
+
+/dts-v1/;
+
+/ {
+ compatible = "ezchip,arc-nps";
+ clock-frequency = <83333333>; /* 83.333333 MHZ */
+ #address-cells = <1>;
+ #size-cells = <1>;
+ interrupt-parent = <&intc>;
+ present-cpus = "0-1,16-17";
+ possible-cpus = "0-4095";
+
+ aliases {
+ ethernet0 = &gmac0;
+ };
+
+ chosen {
+ bootargs = "earlycon=uart8250,mmio32be,0xf7209000,115200n8 console=ttyS0,115200n8";
+ };
+
+ memory {
+ device_type = "memory";
+ reg = <0x80000000 0x20000000>; /* 512M */
+ };
+
+ clocks {
+ sysclk: sysclk {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <83333333>;
+ };
+ };
+
+ soc {
+ compatible = "simple-bus";
+ #address-cells = <1>;
+ #size-cells = <1>;
+
+ /* child and parent address space 1:1 mapped */
+ ranges;
+
+ intc: interrupt-controller {
+ compatible = "ezchip,nps400-ic";
+ interrupt-controller;
+ #interrupt-cells = <1>;
+ };
+
+ timer0: timer_clkevt {
+ compatible = "snps,arc-timer";
+ interrupts = <3>;
+ clocks = <&sysclk>;
+ };
+
+ timer1: timer_clksrc {
+ compatible = "ezchip,nps400-timer";
+ clocks = <&sysclk>;
+ clock-names="sysclk";
+ };
+
+ uart@f7209000 {
+ compatible = "snps,dw-apb-uart";
+ device_type = "serial";
+ reg = <0xf7209000 0x100>;
+ interrupts = <6>;
+ clocks = <&sysclk>;
+ clock-names="baudclk";
+ baud = <115200>;
+ reg-shift = <2>;
+ reg-io-width = <4>;
+ native-endian;
+ };
+
+ gmac0: ethernet@f7470000 {
+ compatible = "ezchip,nps-mgt-enet";
+ reg = <0xf7470000 0x1940>;
+ interrupts = <7>;
+ /* Filled in by U-Boot */
+ mac-address = [ 00 C0 00 F0 04 03 ];
+ };
+ };
+};
diff --git a/arch/arc/boot/dts/nsim_700.dts b/arch/arc/boot/dts/nsim_700.dts
index 105a001..5d5e373 100644
--- a/arch/arc/boot/dts/nsim_700.dts
+++ b/arch/arc/boot/dts/nsim_700.dts
@@ -14,7 +14,7 @@
clock-frequency = <80000000>; /* 80 MHZ */
#address-cells = <1>;
#size-cells = <1>;
- interrupt-parent = <&intc>;
+ interrupt-parent = <&core_intc>;
chosen {
bootargs = "earlycon=arc_uart,mmio32,0xc0fc1000,115200n8 console=ttyARC0,115200n8";
@@ -32,7 +32,13 @@
/* child and parent address space 1:1 mapped */
ranges;
- intc: interrupt-controller {
+ core_clk: core_clk {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <80000000>;
+ };
+
+ core_intc: interrupt-controller {
compatible = "snps,arc700-intc";
interrupt-controller;
#interrupt-cells = <1>;
diff --git a/arch/arc/boot/dts/nsim_hs.dts b/arch/arc/boot/dts/nsim_hs.dts
index f46633e..bf05fe5 100644
--- a/arch/arc/boot/dts/nsim_hs.dts
+++ b/arch/arc/boot/dts/nsim_hs.dts
@@ -7,7 +7,7 @@
*/
/dts-v1/;
-/include/ "skeleton.dtsi"
+/include/ "skeleton_hs.dtsi"
/ {
compatible = "snps,nsim_hs";
@@ -39,6 +39,12 @@
bus addr, parent bus addr, size */
ranges = <0x80000000 0x0 0x80000000 0x80000000>;
+ core_clk: core_clk {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <80000000>;
+ };
+
core_intc: core-interrupt-controller {
compatible = "snps,archs-intc";
interrupt-controller;
diff --git a/arch/arc/boot/dts/nsim_hs_idu.dts b/arch/arc/boot/dts/nsim_hs_idu.dts
index 46ab319..99eabe1 100644
--- a/arch/arc/boot/dts/nsim_hs_idu.dts
+++ b/arch/arc/boot/dts/nsim_hs_idu.dts
@@ -7,7 +7,7 @@
*/
/dts-v1/;
-/include/ "skeleton.dtsi"
+/include/ "skeleton_hs_idu.dtsi"
/ {
compatible = "snps,nsim_hs";
@@ -29,6 +29,12 @@
/* child and parent address space 1:1 mapped */
ranges;
+ core_clk: core_clk {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <80000000>;
+ };
+
core_intc: core-interrupt-controller {
compatible = "snps,archs-intc";
interrupt-controller;
diff --git a/arch/arc/boot/dts/nsimosci.dts b/arch/arc/boot/dts/nsimosci.dts
index d94b4ce..b5b060a 100644
--- a/arch/arc/boot/dts/nsimosci.dts
+++ b/arch/arc/boot/dts/nsimosci.dts
@@ -14,7 +14,7 @@
clock-frequency = <20000000>; /* 20 MHZ */
#address-cells = <1>;
#size-cells = <1>;
- interrupt-parent = <&intc>;
+ interrupt-parent = <&core_intc>;
chosen {
/* this is for console on PGU */
@@ -35,7 +35,13 @@
/* child and parent address space 1:1 mapped */
ranges;
- intc: interrupt-controller {
+ core_clk: core_clk {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <20000000>;
+ };
+
+ core_intc: interrupt-controller {
compatible = "snps,arc700-intc";
interrupt-controller;
#interrupt-cells = <1>;
diff --git a/arch/arc/boot/dts/nsimosci_hs.dts b/arch/arc/boot/dts/nsimosci_hs.dts
index 034a313..325e730 100644
--- a/arch/arc/boot/dts/nsimosci_hs.dts
+++ b/arch/arc/boot/dts/nsimosci_hs.dts
@@ -7,7 +7,7 @@
*/
/dts-v1/;
-/include/ "skeleton.dtsi"
+/include/ "skeleton_hs.dtsi"
/ {
compatible = "snps,nsimosci_hs";
@@ -35,6 +35,12 @@
/* child and parent address space 1:1 mapped */
ranges;
+ core_clk: core_clk {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <20000000>;
+ };
+
core_intc: core-interrupt-controller {
compatible = "snps,archs-intc";
interrupt-controller;
diff --git a/arch/arc/boot/dts/nsimosci_hs_idu.dts b/arch/arc/boot/dts/nsimosci_hs_idu.dts
index 8a1297e..ee03d71 100644
--- a/arch/arc/boot/dts/nsimosci_hs_idu.dts
+++ b/arch/arc/boot/dts/nsimosci_hs_idu.dts
@@ -7,7 +7,7 @@
*/
/dts-v1/;
-/include/ "skeleton.dtsi"
+/include/ "skeleton_hs_idu.dtsi"
/ {
compatible = "snps,nsimosci_hs";
@@ -33,6 +33,12 @@
/* child and parent address space 1:1 mapped */
ranges;
+ core_clk: core_clk {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <5000000>;
+ };
+
core_intc: core-interrupt-controller {
compatible = "snps,archs-intc";
interrupt-controller;
diff --git a/arch/arc/boot/dts/skeleton.dtsi b/arch/arc/boot/dts/skeleton.dtsi
index 296d371..3a10cc6 100644
--- a/arch/arc/boot/dts/skeleton.dtsi
+++ b/arch/arc/boot/dts/skeleton.dtsi
@@ -30,6 +30,20 @@
};
};
+ /* TIMER0 with interrupt for clockevent */
+ timer0 {
+ compatible = "snps,arc-timer";
+ interrupts = <3>;
+ interrupt-parent = <&core_intc>;
+ clocks = <&core_clk>;
+ };
+
+ /* TIMER1 for free running clocksource */
+ timer1 {
+ compatible = "snps,arc-timer";
+ clocks = <&core_clk>;
+ };
+
memory {
device_type = "memory";
reg = <0x80000000 0x10000000>; /* 256M */
diff --git a/arch/arc/boot/dts/skeleton_hs.dtsi b/arch/arc/boot/dts/skeleton_hs.dtsi
new file mode 100644
index 0000000..71fd308
--- /dev/null
+++ b/arch/arc/boot/dts/skeleton_hs.dtsi
@@ -0,0 +1,52 @@
+/*
+ * Copyright (C) 2016 Synopsys, Inc. (www.synopsys.com)
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+/ {
+ compatible = "snps,arc";
+ clock-frequency = <80000000>; /* 80 MHZ */
+ #address-cells = <1>;
+ #size-cells = <1>;
+ chosen { };
+ aliases { };
+
+ cpus {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ cpu@0 {
+ device_type = "cpu";
+ compatible = "snps,archs38";
+ reg = <0>;
+ };
+ };
+
+ /* TIMER0 with interrupt for clockevent */
+ timer0 {
+ compatible = "snps,arc-timer";
+ interrupts = <16>;
+ interrupt-parent = <&core_intc>;
+ clocks = <&core_clk>;
+ };
+
+ /* 64-bit Local RTC: preferred clocksource for UP */
+ rtc {
+ compatible = "snps,archs-timer-rtc";
+ clocks = <&core_clk>;
+ };
+
+ /* TIMER1 for free running clocksource: Fallback if rtc not found */
+ timer1 {
+ compatible = "snps,arc-timer";
+ clocks = <&core_clk>;
+ };
+
+ memory {
+ device_type = "memory";
+ reg = <0x80000000 0x10000000>; /* 256M */
+ };
+};
diff --git a/arch/arc/boot/dts/skeleton_hs_idu.dtsi b/arch/arc/boot/dts/skeleton_hs_idu.dtsi
new file mode 100644
index 0000000..d1cb25a
--- /dev/null
+++ b/arch/arc/boot/dts/skeleton_hs_idu.dtsi
@@ -0,0 +1,46 @@
+/*
+ * Copyright (C) 2016 Synopsys, Inc. (www.synopsys.com)
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+/ {
+ compatible = "snps,arc";
+ clock-frequency = <80000000>; /* 80 MHZ */
+ #address-cells = <1>;
+ #size-cells = <1>;
+ chosen { };
+ aliases { };
+
+ cpus {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ cpu@0 {
+ device_type = "cpu";
+ compatible = "snps,archs38xN";
+ reg = <0>;
+ };
+ };
+
+ /* TIMER0 with interrupt for clockevent */
+ timer0 {
+ compatible = "snps,arc-timer";
+ interrupts = <16>;
+ interrupt-parent = <&core_intc>;
+ clocks = <&core_clk>;
+ };
+
+ /* 64-bit Global Free Running Counter */
+ gfrc {
+ compatible = "snps,archs-timer-gfrc";
+ clocks = <&core_clk>;
+ };
+
+ memory {
+ device_type = "memory";
+ reg = <0x80000000 0x10000000>; /* 256M */
+ };
+};
diff --git a/arch/arc/boot/dts/vdk_axc003.dtsi b/arch/arc/boot/dts/vdk_axc003.dtsi
index 84226bd..ad4ee43 100644
--- a/arch/arc/boot/dts/vdk_axc003.dtsi
+++ b/arch/arc/boot/dts/vdk_axc003.dtsi
@@ -10,6 +10,8 @@
* Device tree for AXC003 CPU card: HS38x UP configuration (VDK version)
*/
+/include/ "skeleton_hs.dtsi"
+
/ {
compatible = "snps,arc";
clock-frequency = <50000000>;
@@ -23,7 +25,13 @@
ranges = <0x00000000 0xf0000000 0x10000000>;
- cpu_intc: archs-intc@cpu {
+ core_clk: core_clk {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <50000000>;
+ };
+
+ core_intc: archs-intc@cpu {
compatible = "snps,archs-intc";
interrupt-controller;
#interrupt-cells = <1>;
@@ -33,7 +41,7 @@
compatible = "snps,dw-apb-uart";
reg = <0x5000 0x100>;
clock-frequency = <2403200>;
- interrupt-parent = <&cpu_intc>;
+ interrupt-parent = <&core_intc>;
interrupts = <19>;
baud = <115200>;
reg-shift = <2>;
@@ -47,7 +55,7 @@
compatible = "snps,dw-apb-ictl";
reg = < 0xe0012000 0x200 >;
interrupt-controller;
- interrupt-parent = <&cpu_intc>;
+ interrupt-parent = <&core_intc>;
interrupts = < 18 >;
};
diff --git a/arch/arc/boot/dts/vdk_axc003_idu.dtsi b/arch/arc/boot/dts/vdk_axc003_idu.dtsi
index 31f0fb5..a3cb626 100644
--- a/arch/arc/boot/dts/vdk_axc003_idu.dtsi
+++ b/arch/arc/boot/dts/vdk_axc003_idu.dtsi
@@ -11,6 +11,8 @@
* HS38x2 (Dual Core) with IDU intc (VDK version)
*/
+/include/ "skeleton_hs_idu.dtsi"
+
/ {
compatible = "snps,arc";
clock-frequency = <50000000>;
@@ -24,7 +26,13 @@
ranges = <0x00000000 0xf0000000 0x10000000>;
- cpu_intc: archs-intc@cpu {
+ core_clk: core_clk {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <50000000>;
+ };
+
+ core_intc: archs-intc@cpu {
compatible = "snps,archs-intc";
interrupt-controller;
#interrupt-cells = <1>;
@@ -33,7 +41,7 @@
idu_intc: idu-interrupt-controller {
compatible = "snps,archs-idu-intc";
interrupt-controller;
- interrupt-parent = <&cpu_intc>;
+ interrupt-parent = <&core_intc>;
/*
* <hwirq distribution>
diff --git a/arch/arc/configs/nps_defconfig b/arch/arc/configs/nps_defconfig
new file mode 100644
index 0000000..ede625c
--- /dev/null
+++ b/arch/arc/configs/nps_defconfig
@@ -0,0 +1,84 @@
+# CONFIG_LOCALVERSION_AUTO is not set
+# CONFIG_SWAP is not set
+CONFIG_SYSVIPC=y
+CONFIG_NO_HZ_IDLE=y
+CONFIG_HIGH_RES_TIMERS=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_BLK_DEV_INITRD=y
+CONFIG_SYSCTL_SYSCALL=y
+# CONFIG_EPOLL is not set
+# CONFIG_SIGNALFD is not set
+# CONFIG_TIMERFD is not set
+# CONFIG_EVENTFD is not set
+# CONFIG_AIO is not set
+CONFIG_EMBEDDED=y
+CONFIG_PERF_EVENTS=y
+# CONFIG_COMPAT_BRK is not set
+CONFIG_KPROBES=y
+CONFIG_MODULES=y
+CONFIG_MODULE_FORCE_LOAD=y
+CONFIG_MODULE_UNLOAD=y
+# CONFIG_BLK_DEV_BSG is not set
+# CONFIG_IOSCHED_DEADLINE is not set
+# CONFIG_IOSCHED_CFQ is not set
+CONFIG_ARC_PLAT_EZNPS=y
+CONFIG_SMP=y
+CONFIG_NR_CPUS=4096
+CONFIG_ARC_CACHE_LINE_SHIFT=5
+# CONFIG_ARC_CACHE_PAGES is not set
+# CONFIG_ARC_HAS_LLSC is not set
+CONFIG_ARC_KVADDR_SIZE=402
+CONFIG_ARC_EMUL_UNALIGNED=y
+CONFIG_ARC_UBOOT_SUPPORT=y
+CONFIG_PREEMPT=y
+CONFIG_NET=y
+CONFIG_UNIX=y
+CONFIG_INET=y
+CONFIG_IP_PNP=y
+# CONFIG_INET_XFRM_MODE_TRANSPORT is not set
+# CONFIG_INET_XFRM_MODE_TUNNEL is not set
+# CONFIG_INET_XFRM_MODE_BEET is not set
+# CONFIG_INET_LRO is not set
+# CONFIG_INET_DIAG is not set
+# CONFIG_IPV6 is not set
+# CONFIG_WIRELESS is not set
+CONFIG_DEVTMPFS=y
+CONFIG_DEVTMPFS_MOUNT=y
+# CONFIG_PREVENT_FIRMWARE_BUILD is not set
+CONFIG_BLK_DEV_RAM=y
+CONFIG_BLK_DEV_RAM_COUNT=1
+CONFIG_BLK_DEV_RAM_SIZE=2048
+CONFIG_NETDEVICES=y
+CONFIG_NETCONSOLE=y
+# CONFIG_NET_VENDOR_BROADCOM is not set
+# CONFIG_NET_VENDOR_MICREL is not set
+# CONFIG_NET_VENDOR_STMICRO is not set
+# CONFIG_WLAN is not set
+# CONFIG_INPUT_MOUSEDEV is not set
+# CONFIG_INPUT_KEYBOARD is not set
+# CONFIG_INPUT_MOUSE is not set
+# CONFIG_SERIO is not set
+# CONFIG_LEGACY_PTYS is not set
+# CONFIG_DEVKMEM is not set
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y
+CONFIG_SERIAL_8250_NR_UARTS=1
+CONFIG_SERIAL_8250_RUNTIME_UARTS=1
+CONFIG_SERIAL_8250_DW=y
+CONFIG_SERIAL_OF_PLATFORM=y
+# CONFIG_HW_RANDOM is not set
+# CONFIG_HWMON is not set
+# CONFIG_USB_SUPPORT is not set
+# CONFIG_DNOTIFY is not set
+CONFIG_PROC_KCORE=y
+CONFIG_TMPFS=y
+# CONFIG_MISC_FILESYSTEMS is not set
+CONFIG_NFS_FS=y
+CONFIG_ROOT_NFS=y
+CONFIG_DEBUG_INFO=y
+# CONFIG_ENABLE_WARN_DEPRECATED is not set
+# CONFIG_ENABLE_MUST_CHECK is not set
+CONFIG_MAGIC_SYSRQ=y
+CONFIG_DEBUG_MEMORY_INIT=y
+CONFIG_ENABLE_DEFAULT_TRACERS=y
diff --git a/arch/arc/include/asm/atomic.h b/arch/arc/include/asm/atomic.h
index 7730d30..5f3dcbb 100644
--- a/arch/arc/include/asm/atomic.h
+++ b/arch/arc/include/asm/atomic.h
@@ -17,6 +17,8 @@
#include <asm/barrier.h>
#include <asm/smp.h>
+#ifndef CONFIG_ARC_PLAT_EZNPS
+
#define atomic_read(v) READ_ONCE((v)->counter)
#ifdef CONFIG_ARC_HAS_LLSC
@@ -180,13 +182,88 @@
ATOMIC_OP(or, |=, or)
ATOMIC_OP(xor, ^=, xor)
-#undef ATOMIC_OPS
-#undef ATOMIC_OP_RETURN
-#undef ATOMIC_OP
#undef SCOND_FAIL_RETRY_VAR_DEF
#undef SCOND_FAIL_RETRY_ASM
#undef SCOND_FAIL_RETRY_VARS
+#else /* CONFIG_ARC_PLAT_EZNPS */
+
+static inline int atomic_read(const atomic_t *v)
+{
+ int temp;
+
+ __asm__ __volatile__(
+ " ld.di %0, [%1]"
+ : "=r"(temp)
+ : "r"(&v->counter)
+ : "memory");
+ return temp;
+}
+
+static inline void atomic_set(atomic_t *v, int i)
+{
+ __asm__ __volatile__(
+ " st.di %0,[%1]"
+ :
+ : "r"(i), "r"(&v->counter)
+ : "memory");
+}
+
+#define ATOMIC_OP(op, c_op, asm_op) \
+static inline void atomic_##op(int i, atomic_t *v) \
+{ \
+ __asm__ __volatile__( \
+ " mov r2, %0\n" \
+ " mov r3, %1\n" \
+ " .word %2\n" \
+ : \
+ : "r"(i), "r"(&v->counter), "i"(asm_op) \
+ : "r2", "r3", "memory"); \
+} \
+
+#define ATOMIC_OP_RETURN(op, c_op, asm_op) \
+static inline int atomic_##op##_return(int i, atomic_t *v) \
+{ \
+ unsigned int temp = i; \
+ \
+ /* Explicit full memory barrier needed before/after */ \
+ smp_mb(); \
+ \
+ __asm__ __volatile__( \
+ " mov r2, %0\n" \
+ " mov r3, %1\n" \
+ " .word %2\n" \
+ " mov %0, r2" \
+ : "+r"(temp) \
+ : "r"(&v->counter), "i"(asm_op) \
+ : "r2", "r3", "memory"); \
+ \
+ smp_mb(); \
+ \
+ temp c_op i; \
+ \
+ return temp; \
+}
+
+#define ATOMIC_OPS(op, c_op, asm_op) \
+ ATOMIC_OP(op, c_op, asm_op) \
+ ATOMIC_OP_RETURN(op, c_op, asm_op)
+
+ATOMIC_OPS(add, +=, CTOP_INST_AADD_DI_R2_R2_R3)
+#define atomic_sub(i, v) atomic_add(-(i), (v))
+#define atomic_sub_return(i, v) atomic_add_return(-(i), (v))
+
+ATOMIC_OP(and, &=, CTOP_INST_AAND_DI_R2_R2_R3)
+#define atomic_andnot(mask, v) atomic_and(~(mask), (v))
+ATOMIC_OP(or, |=, CTOP_INST_AOR_DI_R2_R2_R3)
+ATOMIC_OP(xor, ^=, CTOP_INST_AXOR_DI_R2_R2_R3)
+
+#endif /* CONFIG_ARC_PLAT_EZNPS */
+
+#undef ATOMIC_OPS
+#undef ATOMIC_OP_RETURN
+#undef ATOMIC_OP
+
/**
* __atomic_add_unless - add unless the number is a given value
* @v: pointer of type atomic_t
diff --git a/arch/arc/include/asm/barrier.h b/arch/arc/include/asm/barrier.h
index a720998..b1e3274 100644
--- a/arch/arc/include/asm/barrier.h
+++ b/arch/arc/include/asm/barrier.h
@@ -30,9 +30,7 @@
#define rmb() asm volatile("dmb 1\n" : : : "memory")
#define wmb() asm volatile("dmb 2\n" : : : "memory")
-#endif
-
-#ifdef CONFIG_ISA_ARCOMPACT
+#elif !defined(CONFIG_ARC_PLAT_EZNPS) /* CONFIG_ISA_ARCOMPACT */
/*
* ARCompact based cores (ARC700) only have SYNC instruction which is super
@@ -41,6 +39,14 @@
*/
#define mb() asm volatile("sync\n" : : : "memory")
+
+#else /* CONFIG_ARC_PLAT_EZNPS */
+
+#include <plat/ctop.h>
+
+#define mb() asm volatile (".word %0" : : "i"(CTOP_INST_SCHD_RW) : "memory")
+#define rmb() asm volatile (".word %0" : : "i"(CTOP_INST_SCHD_RD) : "memory")
+
#endif
#include <asm-generic/barrier.h>
diff --git a/arch/arc/include/asm/bitops.h b/arch/arc/include/asm/bitops.h
index 0352fb8..8da87fe 100644
--- a/arch/arc/include/asm/bitops.h
+++ b/arch/arc/include/asm/bitops.h
@@ -22,7 +22,7 @@
#include <asm/smp.h>
#endif
-#if defined(CONFIG_ARC_HAS_LLSC)
+#ifdef CONFIG_ARC_HAS_LLSC
/*
* Hardware assisted Atomic-R-M-W
@@ -88,7 +88,7 @@
return (old & (1 << nr)) != 0; \
}
-#else /* !CONFIG_ARC_HAS_LLSC */
+#elif !defined(CONFIG_ARC_PLAT_EZNPS)
/*
* Non hardware assisted Atomic-R-M-W
@@ -139,7 +139,55 @@
return (old & (1UL << (nr & 0x1f))) != 0; \
}
-#endif /* CONFIG_ARC_HAS_LLSC */
+#else /* CONFIG_ARC_PLAT_EZNPS */
+
+#define BIT_OP(op, c_op, asm_op) \
+static inline void op##_bit(unsigned long nr, volatile unsigned long *m)\
+{ \
+ m += nr >> 5; \
+ \
+ nr = (1UL << (nr & 0x1f)); \
+ if (asm_op == CTOP_INST_AAND_DI_R2_R2_R3) \
+ nr = ~nr; \
+ \
+ __asm__ __volatile__( \
+ " mov r2, %0\n" \
+ " mov r3, %1\n" \
+ " .word %2\n" \
+ : \
+ : "r"(nr), "r"(m), "i"(asm_op) \
+ : "r2", "r3", "memory"); \
+}
+
+#define TEST_N_BIT_OP(op, c_op, asm_op) \
+static inline int test_and_##op##_bit(unsigned long nr, volatile unsigned long *m)\
+{ \
+ unsigned long old; \
+ \
+ m += nr >> 5; \
+ \
+ nr = old = (1UL << (nr & 0x1f)); \
+ if (asm_op == CTOP_INST_AAND_DI_R2_R2_R3) \
+ old = ~old; \
+ \
+ /* Explicit full memory barrier needed before/after */ \
+ smp_mb(); \
+ \
+ __asm__ __volatile__( \
+ " mov r2, %0\n" \
+ " mov r3, %1\n" \
+ " .word %2\n" \
+ " mov %0, r2" \
+ : "+r"(old) \
+ : "r"(m), "i"(asm_op) \
+ : "r2", "r3", "memory"); \
+ \
+ smp_mb(); \
+ \
+ return (old & nr) != 0; \
+}
+
+#endif /* CONFIG_ARC_PLAT_EZNPS */
/***************************************
* Non atomic variants
@@ -181,9 +229,15 @@
/* __test_and_set_bit(), __test_and_clear_bit(), __test_and_change_bit() */\
__TEST_N_BIT_OP(op, c_op, asm_op)
+#ifndef CONFIG_ARC_PLAT_EZNPS
BIT_OPS(set, |, bset)
BIT_OPS(clear, & ~, bclr)
BIT_OPS(change, ^, bxor)
+#else
+BIT_OPS(set, |, CTOP_INST_AOR_DI_R2_R2_R3)
+BIT_OPS(clear, & ~, CTOP_INST_AAND_DI_R2_R2_R3)
+BIT_OPS(change, ^, CTOP_INST_AXOR_DI_R2_R2_R3)
+#endif
/*
* This routine doesn't need to be atomic.
diff --git a/arch/arc/include/asm/clk.h b/arch/arc/include/asm/clk.h
deleted file mode 100644
index bf9d29f5..0000000
--- a/arch/arc/include/asm/clk.h
+++ /dev/null
@@ -1,22 +0,0 @@
-/*
- * Copyright (C) 2012 Synopsys, Inc. (www.synopsys.com)
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-
-#ifndef _ASM_ARC_CLK_H
-#define _ASM_ARC_CLK_H
-
-/* Although we can't really hide core_freq, the accessor is still better way */
-extern unsigned long core_freq;
-
-static inline unsigned long arc_get_core_freq(void)
-{
- return core_freq;
-}
-
-extern int arc_set_core_freq(unsigned long);
-
-#endif
diff --git a/arch/arc/include/asm/cmpxchg.h b/arch/arc/include/asm/cmpxchg.h
index a444be6..d819de1 100644
--- a/arch/arc/include/asm/cmpxchg.h
+++ b/arch/arc/include/asm/cmpxchg.h
@@ -44,7 +44,7 @@
return prev;
}
-#else
+#elif !defined(CONFIG_ARC_PLAT_EZNPS)
static inline unsigned long
__cmpxchg(volatile void *ptr, unsigned long expected, unsigned long new)
@@ -64,23 +64,48 @@
return prev;
}
+#else /* CONFIG_ARC_PLAT_EZNPS */
+
+static inline unsigned long
+__cmpxchg(volatile void *ptr, unsigned long expected, unsigned long new)
+{
+ /*
+ * Explicit full memory barrier needed before/after
+ */
+ smp_mb();
+
+ write_aux_reg(CTOP_AUX_GPA1, expected);
+
+ __asm__ __volatile__(
+ " mov r2, %0\n"
+ " mov r3, %1\n"
+ " .word %2\n"
+ " mov %0, r2"
+ : "+r"(new)
+ : "r"(ptr), "i"(CTOP_INST_EXC_DI_R2_R2_R3)
+ : "r2", "r3", "memory");
+
+ smp_mb();
+
+ return new;
+}
+
#endif /* CONFIG_ARC_HAS_LLSC */
#define cmpxchg(ptr, o, n) ((typeof(*(ptr)))__cmpxchg((ptr), \
(unsigned long)(o), (unsigned long)(n)))
/*
- * Since not supported natively, ARC cmpxchg() uses atomic_ops_lock (UP/SMP)
- * just to gaurantee semantics.
- * atomic_cmpxchg() needs to use the same locks as it's other atomic siblings
- * which also happens to be atomic_ops_lock.
- *
- * Thus despite semantically being different, implementation of atomic_cmpxchg()
- * is same as cmpxchg().
+ * atomic_cmpxchg is same as cmpxchg
+ * LLSC: only different in data-type, semantics are exactly same
+ * !LLSC: cmpxchg() has to use an external lock atomic_ops_lock to guarantee
+ * semantics, and this lock also happens to be used by atomic_*()
*/
#define atomic_cmpxchg(v, o, n) ((int)cmpxchg(&((v)->counter), (o), (n)))
+#ifndef CONFIG_ARC_PLAT_EZNPS
+
/*
* xchg (reg with memory) based on "Native atomic" EX insn
*/
@@ -143,6 +168,41 @@
#endif
+#else /* CONFIG_ARC_PLAT_EZNPS */
+
+static inline unsigned long __xchg(unsigned long val, volatile void *ptr,
+ int size)
+{
+ extern unsigned long __xchg_bad_pointer(void);
+
+ switch (size) {
+ case 4:
+ /*
+ * Explicit full memory barrier needed before/after
+ */
+ smp_mb();
+
+ __asm__ __volatile__(
+ " mov r2, %0\n"
+ " mov r3, %1\n"
+ " .word %2\n"
+ " mov %0, r2\n"
+ : "+r"(val)
+ : "r"(ptr), "i"(CTOP_INST_XEX_DI_R2_R2_R3)
+ : "r2", "r3", "memory");
+
+ smp_mb();
+
+ return val;
+ }
+ return __xchg_bad_pointer();
+}
+
+#define xchg(ptr, with) ((typeof(*(ptr)))__xchg((unsigned long)(with), (ptr), \
+ sizeof(*(ptr))))
+
+#endif /* CONFIG_ARC_PLAT_EZNPS */
+
/*
* "atomic" variant of xchg()
* REQ: It needs to follow the same serialization rules as other atomic_xxx()
diff --git a/arch/arc/include/asm/entry-compact.h b/arch/arc/include/asm/entry-compact.h
index 1d8f57c..e0e1faf0 100644
--- a/arch/arc/include/asm/entry-compact.h
+++ b/arch/arc/include/asm/entry-compact.h
@@ -36,6 +36,10 @@
#include <asm/irqflags-compact.h>
#include <asm/thread_info.h> /* For THREAD_SIZE */
+#ifdef CONFIG_ARC_PLAT_EZNPS
+#include <plat/ctop.h>
+#endif
+
/*--------------------------------------------------------------
* Switch to Kernel Mode stack if SP points to User Mode stack
*
@@ -296,11 +300,13 @@
bic \reg, sp, (THREAD_SIZE - 1)
.endm
+#ifndef CONFIG_ARC_PLAT_EZNPS
/* Get CPU-ID of this core */
.macro GET_CPU_ID reg
lr \reg, [identity]
lsr \reg, \reg, 8
bmsk \reg, \reg, 7
.endm
+#endif
#endif /* __ASM_ARC_ENTRY_COMPACT_H */
diff --git a/arch/arc/include/asm/hugepage.h b/arch/arc/include/asm/hugepage.h
index 7afe3356..317ff77 100644
--- a/arch/arc/include/asm/hugepage.h
+++ b/arch/arc/include/asm/hugepage.h
@@ -61,8 +61,6 @@
extern void update_mmu_cache_pmd(struct vm_area_struct *vma, unsigned long addr,
pmd_t *pmd);
-#define has_transparent_hugepage() 1
-
/* Generic variants assume pgtable_t is struct page *, hence need for these */
#define __HAVE_ARCH_PGTABLE_DEPOSIT
extern void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,
diff --git a/arch/arc/include/asm/irq.h b/arch/arc/include/asm/irq.h
index 49014f0..c0fa0d2 100644
--- a/arch/arc/include/asm/irq.h
+++ b/arch/arc/include/asm/irq.h
@@ -13,21 +13,14 @@
#define NR_IRQS 128 /* allow some CPU external IRQ handling */
/* Platform Independent IRQs */
-#ifdef CONFIG_ISA_ARCOMPACT
-#define TIMER0_IRQ 3
-#define TIMER1_IRQ 4
-#else
-#define TIMER0_IRQ 16
-#define TIMER1_IRQ 17
+#ifdef CONFIG_ISA_ARCV2
+#define IPI_IRQ 19
+#define SOFTIRQ_IRQ 21
#endif
#include <linux/interrupt.h>
#include <asm-generic/irq.h>
extern void arc_init_IRQ(void);
-void arc_local_timer_setup(void);
-void arc_request_percpu_irq(int irq, int cpu,
- irqreturn_t (*isr)(int irq, void *dev),
- const char *irq_nm, void *percpu_dev);
#endif
diff --git a/arch/arc/include/asm/page.h b/arch/arc/include/asm/page.h
index 0d53854..296c342 100644
--- a/arch/arc/include/asm/page.h
+++ b/arch/arc/include/asm/page.h
@@ -31,7 +31,11 @@
* These are used to make use of C type-checking..
*/
typedef struct {
+#ifdef CONFIG_ARC_HAS_PAE40
+ unsigned long long pte;
+#else
unsigned long pte;
+#endif
} pte_t;
typedef struct {
unsigned long pgd;
diff --git a/arch/arc/include/asm/pgtable.h b/arch/arc/include/asm/pgtable.h
index 10d4b8b..034bbdc 100644
--- a/arch/arc/include/asm/pgtable.h
+++ b/arch/arc/include/asm/pgtable.h
@@ -217,7 +217,7 @@
#define BITS_FOR_PTE (PGDIR_SHIFT - PAGE_SHIFT)
#define BITS_FOR_PGD (32 - PGDIR_SHIFT)
-#define PGDIR_SIZE (1UL << PGDIR_SHIFT) /* vaddr span, not PDG sz */
+#define PGDIR_SIZE _BITUL(PGDIR_SHIFT) /* vaddr span, not PDG sz */
#define PGDIR_MASK (~(PGDIR_SIZE-1))
#define PTRS_PER_PTE _BITUL(BITS_FOR_PTE)
diff --git a/arch/arc/include/asm/processor.h b/arch/arc/include/asm/processor.h
index 1d694c1..f9048994 100644
--- a/arch/arc/include/asm/processor.h
+++ b/arch/arc/include/asm/processor.h
@@ -57,9 +57,19 @@
* A lot of busy-wait loops in SMP are based off of non-volatile data otherwise
* get optimised away by gcc
*/
-#define cpu_relax() __asm__ __volatile__ ("" : : : "memory")
+#ifndef CONFIG_EZNPS_MTM_EXT
-#define cpu_relax_lowlatency() cpu_relax()
+#define cpu_relax() barrier()
+#define cpu_relax_lowlatency() cpu_relax()
+
+#else
+
+#define cpu_relax() \
+ __asm__ __volatile__ (".word %0" : : "i"(CTOP_INST_SCHD_RW) : "memory")
+
+#define cpu_relax_lowlatency() barrier()
+
+#endif
#define copy_segments(tsk, mm) do { } while (0)
#define release_segments(mm) do { } while (0)
@@ -97,7 +107,7 @@
#endif /* !__ASSEMBLY__ */
/*
- * System Memory Map on ARC
+ * Default System Memory Map on ARC
*
* ---------------------------- (lower 2G, Translated) -------------------------
* 0x0000_0000 0x5FFF_FFFF (user vaddr: TASK_SIZE)
@@ -109,20 +119,37 @@
* 0xC000_0000 0xFFFF_FFFF (peripheral uncached space)
* -----------------------------------------------------------------------------
*/
-#define VMALLOC_START 0x70000000
-/*
- * 1 PGDIR_SIZE each for fixmap/pkmap, 2 PGDIR_SIZE gutter
- * See asm/highmem.h for details
- */
-#define VMALLOC_SIZE (PAGE_OFFSET - VMALLOC_START - PGDIR_SIZE * 4)
+#define TASK_SIZE 0x60000000
+
+#define VMALLOC_START (PAGE_OFFSET - (CONFIG_ARC_KVADDR_SIZE << 20))
+
+/* 1 PGDIR_SIZE each for fixmap/pkmap, 2 PGDIR_SIZE gutter (see asm/highmem.h) */
+#define VMALLOC_SIZE ((CONFIG_ARC_KVADDR_SIZE << 20) - PGDIR_SIZE * 4)
+
#define VMALLOC_END (VMALLOC_START + VMALLOC_SIZE)
-#define USER_KERNEL_GUTTER 0x10000000
+#define USER_KERNEL_GUTTER (VMALLOC_START - TASK_SIZE)
-#define TASK_SIZE (VMALLOC_START - USER_KERNEL_GUTTER)
-
+#ifdef CONFIG_ARC_PLAT_EZNPS
+/* NPS architecture defines special window of 129M in user address space for
+ * special memory areas, when accessing this window the MMU do not use TLB.
+ * Instead MMU direct the access to:
+ * 0x57f00000:0x57ffffff -- 1M of closely coupled memory (aka CMEM)
+ * 0x58000000:0x5fffffff -- 16 huge pages, 8M each, with fixed map (aka FMTs)
+ *
+ * CMEM - is the fastest memory we got and its size is 16K.
+ * FMT - is used to map either to internal/external memory.
+ * Internal memory is the second fast memory and its size is 16M
+ * External memory is the biggest memory (16G) and also the slowest.
+ *
+ * STACK_TOP need to be PMD align (21bit) that is why we supply 0x57e00000.
+ */
+#define STACK_TOP 0x57e00000
+#else
#define STACK_TOP TASK_SIZE
+#endif
+
#define STACK_TOP_MAX STACK_TOP
/* This decides where the kernel will search for a free chunk of vm
diff --git a/arch/arc/include/asm/setup.h b/arch/arc/include/asm/setup.h
index 3078466..48b37c6 100644
--- a/arch/arc/include/asm/setup.h
+++ b/arch/arc/include/asm/setup.h
@@ -12,7 +12,11 @@
#include <linux/types.h>
#include <uapi/asm/setup.h>
+#ifdef CONFIG_ARC_PLAT_EZNPS
+#define COMMAND_LINE_SIZE 2048
+#else
#define COMMAND_LINE_SIZE 256
+#endif
/*
* Data structure to map a ID to string
diff --git a/arch/arc/include/asm/spinlock.h b/arch/arc/include/asm/spinlock.h
index db8c59d..800e7c4 100644
--- a/arch/arc/include/asm/spinlock.h
+++ b/arch/arc/include/asm/spinlock.h
@@ -610,7 +610,9 @@
static inline int arch_read_trylock(arch_rwlock_t *rw)
{
int ret = 0;
+ unsigned long flags;
+ local_irq_save(flags);
arch_spin_lock(&(rw->lock_mutex));
/*
@@ -623,6 +625,7 @@
}
arch_spin_unlock(&(rw->lock_mutex));
+ local_irq_restore(flags);
smp_mb();
return ret;
@@ -632,7 +635,9 @@
static inline int arch_write_trylock(arch_rwlock_t *rw)
{
int ret = 0;
+ unsigned long flags;
+ local_irq_save(flags);
arch_spin_lock(&(rw->lock_mutex));
/*
@@ -646,6 +651,7 @@
ret = 1;
}
arch_spin_unlock(&(rw->lock_mutex));
+ local_irq_restore(flags);
return ret;
}
@@ -664,16 +670,24 @@
static inline void arch_read_unlock(arch_rwlock_t *rw)
{
+ unsigned long flags;
+
+ local_irq_save(flags);
arch_spin_lock(&(rw->lock_mutex));
rw->counter++;
arch_spin_unlock(&(rw->lock_mutex));
+ local_irq_restore(flags);
}
static inline void arch_write_unlock(arch_rwlock_t *rw)
{
+ unsigned long flags;
+
+ local_irq_save(flags);
arch_spin_lock(&(rw->lock_mutex));
rw->counter = __ARCH_RW_LOCK_UNLOCKED__;
arch_spin_unlock(&(rw->lock_mutex));
+ local_irq_restore(flags);
}
#endif
diff --git a/arch/arc/include/uapi/asm/byteorder.h b/arch/arc/include/uapi/asm/byteorder.h
index 9da71d4..ea5ca444 100644
--- a/arch/arc/include/uapi/asm/byteorder.h
+++ b/arch/arc/include/uapi/asm/byteorder.h
@@ -9,7 +9,7 @@
#ifndef __ASM_ARC_BYTEORDER_H
#define __ASM_ARC_BYTEORDER_H
-#ifdef CONFIG_CPU_BIG_ENDIAN
+#ifdef __BIG_ENDIAN__
#include <linux/byteorder/big_endian.h>
#else
#include <linux/byteorder/little_endian.h>
diff --git a/arch/arc/include/uapi/asm/unistd.h b/arch/arc/include/uapi/asm/unistd.h
index 39e58d1..41fa2ec 100644
--- a/arch/arc/include/uapi/asm/unistd.h
+++ b/arch/arc/include/uapi/asm/unistd.h
@@ -15,6 +15,7 @@
#if !defined(_UAPI_ASM_ARC_UNISTD_H) || defined(__SYSCALL)
#define _UAPI_ASM_ARC_UNISTD_H
+#define __ARCH_WANT_RENAMEAT
#define __ARCH_WANT_SYS_EXECVE
#define __ARCH_WANT_SYS_CLONE
#define __ARCH_WANT_SYS_VFORK
diff --git a/arch/arc/kernel/Makefile b/arch/arc/kernel/Makefile
index 1bc2036..cfcdedf 100644
--- a/arch/arc/kernel/Makefile
+++ b/arch/arc/kernel/Makefile
@@ -9,7 +9,7 @@
CFLAGS_ptrace.o += -DUTS_MACHINE='"$(UTS_MACHINE)"'
obj-y := arcksyms.o setup.o irq.o time.o reset.o ptrace.o process.o devtree.o
-obj-y += signal.o traps.o sys.o troubleshoot.o stacktrace.o disasm.o clk.o
+obj-y += signal.o traps.o sys.o troubleshoot.o stacktrace.o disasm.o
obj-$(CONFIG_ISA_ARCOMPACT) += entry-compact.o intc-compact.o
obj-$(CONFIG_ISA_ARCV2) += entry-arcv2.o intc-arcv2.o
obj-$(CONFIG_PCI) += pcibios.o
diff --git a/arch/arc/kernel/clk.c b/arch/arc/kernel/clk.c
deleted file mode 100644
index 10c7b0b..0000000
--- a/arch/arc/kernel/clk.c
+++ /dev/null
@@ -1,21 +0,0 @@
-/*
- * Copyright (C) 2012 Synopsys, Inc. (www.synopsys.com)
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-
-#include <asm/clk.h>
-
-unsigned long core_freq = 80000000;
-
-/*
- * As of now we default to device-tree provided clock
- * In future we can determine this in early boot
- */
-int arc_set_core_freq(unsigned long freq)
-{
- core_freq = freq;
- return 0;
-}
diff --git a/arch/arc/kernel/ctx_sw.c b/arch/arc/kernel/ctx_sw.c
index 5d446df..6f4cb0d 100644
--- a/arch/arc/kernel/ctx_sw.c
+++ b/arch/arc/kernel/ctx_sw.c
@@ -16,6 +16,9 @@
#include <asm/asm-offsets.h>
#include <linux/sched.h>
+#ifdef CONFIG_ARC_PLAT_EZNPS
+#include <plat/ctop.h>
+#endif
#define KSP_WORD_OFF ((TASK_THREAD + THREAD_KSP) / 4)
@@ -67,9 +70,16 @@
#ifndef CONFIG_SMP
"st %2, [@_current_task] \n\t"
#else
+#ifdef CONFIG_ARC_PLAT_EZNPS
+ "lr r24, [%4] \n\t"
+#ifndef CONFIG_EZNPS_MTM_EXT
+ "lsr r24, r24, 4 \n\t"
+#endif
+#else
"lr r24, [identity] \n\t"
"lsr r24, r24, 8 \n\t"
"bmsk r24, r24, 7 \n\t"
+#endif
"add2 r24, @_current_task, r24 \n\t"
"st %2, [r24] \n\t"
#endif
@@ -107,6 +117,9 @@
: "=r"(tmp)
: "n"(KSP_WORD_OFF), "r"(next), "r"(prev)
+#ifdef CONFIG_ARC_PLAT_EZNPS
+ , "i"(CTOP_AUX_LOGIC_GLOBAL_ID)
+#endif
: "blink"
);
diff --git a/arch/arc/kernel/devtree.c b/arch/arc/kernel/devtree.c
index 7e844fd..f1e07c2 100644
--- a/arch/arc/kernel/devtree.c
+++ b/arch/arc/kernel/devtree.c
@@ -14,7 +14,6 @@
#include <linux/memblock.h>
#include <linux/of.h>
#include <linux/of_fdt.h>
-#include <asm/clk.h>
#include <asm/mach_desc.h>
#ifdef CONFIG_SERIAL_EARLYCON
@@ -28,14 +27,12 @@
static void __init arc_set_early_base_baud(unsigned long dt_root)
{
- unsigned int core_clk = arc_get_core_freq();
-
if (of_flat_dt_is_compatible(dt_root, "abilis,arc-tb10x"))
- arc_base_baud = core_clk/3;
+ arc_base_baud = 166666666; /* Fixed 166.6MHz clk (TB10x) */
else if (of_flat_dt_is_compatible(dt_root, "snps,arc-sdp"))
arc_base_baud = 33333333; /* Fixed 33MHz clk (AXS10x) */
else
- arc_base_baud = core_clk;
+ arc_base_baud = 50000000; /* Fixed default 50MHz */
}
#else
#define arc_set_early_base_baud(dt_root)
@@ -65,8 +62,6 @@
{
const struct machine_desc *mdesc;
unsigned long dt_root;
- const void *clk;
- int len;
if (!early_init_dt_scan(dt))
return NULL;
@@ -76,10 +71,6 @@
machine_halt();
dt_root = of_get_flat_dt_root();
- clk = of_get_flat_dt_prop(dt_root, "clock-frequency", &len);
- if (clk)
- arc_set_core_freq(of_read_ulong(clk, len/4));
-
arc_set_early_base_baud(dt_root);
return mdesc;
diff --git a/arch/arc/kernel/intc-arcv2.c b/arch/arc/kernel/intc-arcv2.c
index 9425263..6c24faf 100644
--- a/arch/arc/kernel/intc-arcv2.c
+++ b/arch/arc/kernel/intc-arcv2.c
@@ -137,23 +137,30 @@
.map = arcv2_irq_map,
};
-static struct irq_domain *root_domain;
static int __init
init_onchip_IRQ(struct device_node *intc, struct device_node *parent)
{
+ struct irq_domain *root_domain;
+
if (parent)
panic("DeviceTree incore intc not a root irq controller\n");
- root_domain = irq_domain_add_legacy(intc, NR_CPU_IRQS, 0, 0,
- &arcv2_irq_ops, NULL);
-
+ root_domain = irq_domain_add_linear(intc, NR_CPU_IRQS, &arcv2_irq_ops, NULL);
if (!root_domain)
panic("root irq domain not avail\n");
- /* with this we don't need to export root_domain */
+ /*
+ * Needed for primary domain lookup to succeed
+ * This is a primary irqchip, and can never have a parent
+ */
irq_set_default_host(root_domain);
+#ifdef CONFIG_SMP
+ irq_create_mapping(root_domain, IPI_IRQ);
+#endif
+ irq_create_mapping(root_domain, SOFTIRQ_IRQ);
+
return 0;
}
diff --git a/arch/arc/kernel/intc-compact.c b/arch/arc/kernel/intc-compact.c
index 224d1c3..c5cceca 100644
--- a/arch/arc/kernel/intc-compact.c
+++ b/arch/arc/kernel/intc-compact.c
@@ -14,6 +14,8 @@
#include <linux/irqchip.h>
#include <asm/irq.h>
+#define TIMER0_IRQ 3 /* Fixed by ISA */
+
/*
* Early Hardware specific Interrupt setup
* -Platform independent, needed for each CPU (not foldable into init_IRQ)
@@ -79,8 +81,9 @@
static int arc_intc_domain_map(struct irq_domain *d, unsigned int irq,
irq_hw_number_t hw)
{
- switch (irq) {
+ switch (hw) {
case TIMER0_IRQ:
+ irq_set_percpu_devid(irq);
irq_set_chip_and_handler(irq, &onchip_intc, handle_percpu_irq);
break;
default:
@@ -94,21 +97,23 @@
.map = arc_intc_domain_map,
};
-static struct irq_domain *root_domain;
-
static int __init
init_onchip_IRQ(struct device_node *intc, struct device_node *parent)
{
+ struct irq_domain *root_domain;
+
if (parent)
panic("DeviceTree incore intc not a root irq controller\n");
- root_domain = irq_domain_add_legacy(intc, NR_CPU_IRQS, 0, 0,
+ root_domain = irq_domain_add_linear(intc, NR_CPU_IRQS,
&arc_intc_domain_ops, NULL);
-
if (!root_domain)
panic("root irq domain not avail\n");
- /* with this we don't need to export root_domain */
+ /*
+ * Needed for primary domain lookup to succeed
+ * This is a primary irqchip, and can never have a parent
+ */
irq_set_default_host(root_domain);
return 0;
diff --git a/arch/arc/kernel/irq.c b/arch/arc/kernel/irq.c
index ba17f85..538b36a 100644
--- a/arch/arc/kernel/irq.c
+++ b/arch/arc/kernel/irq.c
@@ -41,53 +41,7 @@
* "C" Entry point for any ARC ISR, called from low level vector handler
* @irq is the vector number read from ICAUSE reg of on-chip intc
*/
-void arch_do_IRQ(unsigned int irq, struct pt_regs *regs)
+void arch_do_IRQ(unsigned int hwirq, struct pt_regs *regs)
{
- struct pt_regs *old_regs = set_irq_regs(regs);
-
- irq_enter();
- generic_handle_irq(irq);
- irq_exit();
- set_irq_regs(old_regs);
-}
-
-/*
- * API called for requesting percpu interrupts - called by each CPU
- * - For boot CPU, actually request the IRQ with genirq core + enables
- * - For subsequent callers only enable called locally
- *
- * Relies on being called by boot cpu first (i.e. request called ahead) of
- * any enable as expected by genirq. Hence Suitable only for TIMER, IPI
- * which are guaranteed to be setup on boot core first.
- * Late probed peripherals such as perf can't use this as there no guarantee
- * of being called on boot CPU first.
- */
-
-void arc_request_percpu_irq(int irq, int cpu,
- irqreturn_t (*isr)(int irq, void *dev),
- const char *irq_nm,
- void *percpu_dev)
-{
- /* Boot cpu calls request, all call enable */
- if (!cpu) {
- int rc;
-
-#ifdef CONFIG_ISA_ARCOMPACT
- /*
- * A subsequent request_percpu_irq() fails if percpu_devid is
- * not set. That in turns sets NOAUTOEN, meaning each core needs
- * to call enable_percpu_irq()
- *
- * For ARCv2, this is done in irq map function since we know
- * which irqs are strictly per cpu
- */
- irq_set_percpu_devid(irq);
-#endif
-
- rc = request_percpu_irq(irq, isr, irq_nm, percpu_dev);
- if (rc)
- panic("Percpu IRQ request failed for %d\n", irq);
- }
-
- enable_percpu_irq(irq, 0);
+ handle_domain_irq(NULL, hwirq, regs);
}
diff --git a/arch/arc/kernel/mcip.c b/arch/arc/kernel/mcip.c
index c41c364..72f9179 100644
--- a/arch/arc/kernel/mcip.c
+++ b/arch/arc/kernel/mcip.c
@@ -15,9 +15,6 @@
#include <asm/mcip.h>
#include <asm/setup.h>
-#define IPI_IRQ 19
-#define SOFTIRQ_IRQ 21
-
static char smp_cpuinfo_buf[128];
static int idu_detected;
@@ -116,15 +113,13 @@
IS_AVAIL1(mp.dbg, "DEBUG "),
IS_AVAIL1(mp.gfrc, "GFRC"));
+ cpuinfo_arc700[0].extn.gfrc = mp.gfrc;
idu_detected = mp.idu;
if (mp.dbg) {
__mcip_cmd_data(CMD_DEBUG_SET_SELECT, 0, 0xf);
__mcip_cmd_data(CMD_DEBUG_SET_MASK, 0xf, 0xf);
}
-
- if (IS_ENABLED(CONFIG_ARC_HAS_GFRC) && !mp.gfrc)
- panic("kernel trying to use non-existent GFRC\n");
}
struct plat_smp_ops plat_smp_ops = {
diff --git a/arch/arc/kernel/process.c b/arch/arc/kernel/process.c
index a3f750e..b5db9e7 100644
--- a/arch/arc/kernel/process.c
+++ b/arch/arc/kernel/process.c
@@ -183,13 +183,6 @@
{
}
-/*
- * Free any architecture-specific thread data structures, etc.
- */
-void exit_thread(void)
-{
-}
-
int dump_fpu(struct pt_regs *regs, elf_fpregset_t *fpu)
{
return 0;
diff --git a/arch/arc/kernel/setup.c b/arch/arc/kernel/setup.c
index 151acf0..f63b8bf 100644
--- a/arch/arc/kernel/setup.c
+++ b/arch/arc/kernel/setup.c
@@ -13,7 +13,6 @@
#include <linux/console.h>
#include <linux/module.h>
#include <linux/cpu.h>
-#include <linux/clk-provider.h>
#include <linux/of_fdt.h>
#include <linux/of_platform.h>
#include <linux/cache.h>
@@ -24,7 +23,6 @@
#include <asm/page.h>
#include <asm/irq.h>
#include <asm/unwind.h>
-#include <asm/clk.h>
#include <asm/mach_desc.h>
#include <asm/smp.h>
@@ -220,10 +218,6 @@
if (tbl->info.id == 0)
n += scnprintf(buf + n, len - n, "UNKNOWN ARC Processor\n");
- n += scnprintf(buf + n, len - n, "CPU speed\t: %u.%02u Mhz\n",
- (unsigned int)(arc_get_core_freq() / 1000000),
- (unsigned int)(arc_get_core_freq() / 10000) % 100);
-
n += scnprintf(buf + n, len - n, "Timers\t\t: %s%s%s%s\nISA Extn\t: ",
IS_AVAIL1(cpu->extn.timer0, "Timer0 "),
IS_AVAIL1(cpu->extn.timer1, "Timer1 "),
@@ -314,9 +308,6 @@
if (!cpu->extn.timer1)
panic("Timer1 is not present!\n");
- if (IS_ENABLED(CONFIG_ARC_HAS_RTC) && !cpu->extn.rtc)
- panic("RTC is not present\n");
-
#ifdef CONFIG_ARC_HAS_DCCM
/*
* DCCM can be arbit placed in hardware.
@@ -444,7 +435,6 @@
static int __init customize_machine(void)
{
- of_clk_init(NULL);
/*
* Traverses flattened DeviceTree - registering platform devices
* (if any) complete with their resources
@@ -477,6 +467,8 @@
{
char *str;
int cpu_id = ptr_to_cpu(v);
+ struct device_node *core_clk = of_find_node_by_name(NULL, "core_clk");
+ u32 freq = 0;
if (!cpu_online(cpu_id)) {
seq_printf(m, "processor [%d]\t: Offline\n", cpu_id);
@@ -489,6 +481,11 @@
seq_printf(m, arc_cpu_mumbojumbo(cpu_id, str, PAGE_SIZE));
+ of_property_read_u32(core_clk, "clock-frequency", &freq);
+ if (freq)
+ seq_printf(m, "CPU speed\t: %u.%02u Mhz\n",
+ freq / 1000000, (freq / 10000) % 100);
+
seq_printf(m, "Bogo MIPS\t: %lu.%02lu\n",
loops_per_jiffy / (500000 / HZ),
(loops_per_jiffy / (5000 / HZ)) % 100);
diff --git a/arch/arc/kernel/smp.c b/arch/arc/kernel/smp.c
index 4cb3add..f183cc6 100644
--- a/arch/arc/kernel/smp.c
+++ b/arch/arc/kernel/smp.c
@@ -126,11 +126,6 @@
current->active_mm = mm;
cpumask_set_cpu(cpu, mm_cpumask(mm));
- notify_cpu_starting(cpu);
- set_cpu_online(cpu, true);
-
- pr_info("## CPU%u LIVE ##: Executing Code...\n", cpu);
-
/* Some SMP H/w setup - for each cpu */
if (plat_smp_ops.init_per_cpu)
plat_smp_ops.init_per_cpu(cpu);
@@ -138,7 +133,10 @@
if (machine_desc->init_per_cpu)
machine_desc->init_per_cpu(cpu);
- arc_local_timer_setup();
+ notify_cpu_starting(cpu);
+ set_cpu_online(cpu, true);
+
+ pr_info("## CPU%u LIVE ##: Executing Code...\n", cpu);
local_irq_enable();
preempt_disable();
@@ -346,6 +344,10 @@
/*
* API called by platform code to hookup arch-common ISR to their IPI IRQ
+ *
+ * Note: If IPI is provided by platform (vs. say ARC MCIP), their intc setup/map
+ * function needs to call call irq_set_percpu_devid() for IPI IRQ, otherwise
+ * request_percpu_irq() below will fail
*/
static DEFINE_PER_CPU(int, ipi_dev);
@@ -353,7 +355,16 @@
{
int *dev = per_cpu_ptr(&ipi_dev, cpu);
- arc_request_percpu_irq(irq, cpu, do_IPI, "IPI Interrupt", dev);
+ /* Boot cpu calls request, all call enable */
+ if (!cpu) {
+ int rc;
+
+ rc = request_percpu_irq(irq, do_IPI, "IPI Interrupt", dev);
+ if (rc)
+ panic("Percpu IRQ request failed for %d\n", irq);
+ }
+
+ enable_percpu_irq(irq, 0);
return 0;
}
diff --git a/arch/arc/kernel/time.c b/arch/arc/kernel/time.c
index 7d9a736..4549ab2 100644
--- a/arch/arc/kernel/time.c
+++ b/arch/arc/kernel/time.c
@@ -29,21 +29,16 @@
* which however is currently broken
*/
-#include <linux/spinlock.h>
#include <linux/interrupt.h>
-#include <linux/module.h>
-#include <linux/sched.h>
-#include <linux/kernel.h>
-#include <linux/time.h>
-#include <linux/init.h>
-#include <linux/timex.h>
-#include <linux/profile.h>
+#include <linux/clk.h>
+#include <linux/clk-provider.h>
#include <linux/clocksource.h>
#include <linux/clockchips.h>
+#include <linux/cpu.h>
+#include <linux/of.h>
+#include <linux/of_irq.h>
#include <asm/irq.h>
#include <asm/arcregs.h>
-#include <asm/clk.h>
-#include <asm/mach_desc.h>
#include <asm/mcip.h>
@@ -60,16 +55,35 @@
#define ARC_TIMER_MAX 0xFFFFFFFF
+static unsigned long arc_timer_freq;
+
+static int noinline arc_get_timer_clk(struct device_node *node)
+{
+ struct clk *clk;
+ int ret;
+
+ clk = of_clk_get(node, 0);
+ if (IS_ERR(clk)) {
+ pr_err("timer missing clk");
+ return PTR_ERR(clk);
+ }
+
+ ret = clk_prepare_enable(clk);
+ if (ret) {
+ pr_err("Couldn't enable parent clk\n");
+ return ret;
+ }
+
+ arc_timer_freq = clk_get_rate(clk);
+
+ return 0;
+}
+
/********** Clock Source Device *********/
#ifdef CONFIG_ARC_HAS_GFRC
-static int arc_counter_setup(void)
-{
- return 1;
-}
-
-static cycle_t arc_counter_read(struct clocksource *cs)
+static cycle_t arc_read_gfrc(struct clocksource *cs)
{
unsigned long flags;
union {
@@ -94,15 +108,31 @@
return stamp.full;
}
-static struct clocksource arc_counter = {
+static struct clocksource arc_counter_gfrc = {
.name = "ARConnect GFRC",
.rating = 400,
- .read = arc_counter_read,
+ .read = arc_read_gfrc,
.mask = CLOCKSOURCE_MASK(64),
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
};
-#else
+static void __init arc_cs_setup_gfrc(struct device_node *node)
+{
+ int exists = cpuinfo_arc700[0].extn.gfrc;
+ int ret;
+
+ if (WARN(!exists, "Global-64-bit-Ctr clocksource not detected"))
+ return;
+
+ ret = arc_get_timer_clk(node);
+ if (ret)
+ return;
+
+ clocksource_register_hz(&arc_counter_gfrc, arc_timer_freq);
+}
+CLOCKSOURCE_OF_DECLARE(arc_gfrc, "snps,archs-timer-gfrc", arc_cs_setup_gfrc);
+
+#endif
#ifdef CONFIG_ARC_HAS_RTC
@@ -110,15 +140,7 @@
#define AUX_RTC_LOW 0x104
#define AUX_RTC_HIGH 0x105
-int arc_counter_setup(void)
-{
- write_aux_reg(AUX_RTC_CTRL, 1);
-
- /* Not usable in SMP */
- return !IS_ENABLED(CONFIG_SMP);
-}
-
-static cycle_t arc_counter_read(struct clocksource *cs)
+static cycle_t arc_read_rtc(struct clocksource *cs)
{
unsigned long status;
union {
@@ -142,47 +164,78 @@
return stamp.full;
}
-static struct clocksource arc_counter = {
+static struct clocksource arc_counter_rtc = {
.name = "ARCv2 RTC",
.rating = 350,
- .read = arc_counter_read,
+ .read = arc_read_rtc,
.mask = CLOCKSOURCE_MASK(64),
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
};
-#else /* !CONFIG_ARC_HAS_RTC */
+static void __init arc_cs_setup_rtc(struct device_node *node)
+{
+ int exists = cpuinfo_arc700[smp_processor_id()].extn.rtc;
+ int ret;
+
+ if (WARN(!exists, "Local-64-bit-Ctr clocksource not detected"))
+ return;
+
+ /* Local to CPU hence not usable in SMP */
+ if (WARN(IS_ENABLED(CONFIG_SMP), "Local-64-bit-Ctr not usable in SMP"))
+ return;
+
+ ret = arc_get_timer_clk(node);
+ if (ret)
+ return;
+
+ write_aux_reg(AUX_RTC_CTRL, 1);
+
+ clocksource_register_hz(&arc_counter_rtc, arc_timer_freq);
+}
+CLOCKSOURCE_OF_DECLARE(arc_rtc, "snps,archs-timer-rtc", arc_cs_setup_rtc);
+
+#endif
/*
- * set 32bit TIMER1 to keep counting monotonically and wraparound
+ * 32bit TIMER1 to keep counting monotonically and wraparound
*/
-int arc_counter_setup(void)
-{
- write_aux_reg(ARC_REG_TIMER1_LIMIT, ARC_TIMER_MAX);
- write_aux_reg(ARC_REG_TIMER1_CNT, 0);
- write_aux_reg(ARC_REG_TIMER1_CTRL, TIMER_CTRL_NH);
- /* Not usable in SMP */
- return !IS_ENABLED(CONFIG_SMP);
-}
-
-static cycle_t arc_counter_read(struct clocksource *cs)
+static cycle_t arc_read_timer1(struct clocksource *cs)
{
return (cycle_t) read_aux_reg(ARC_REG_TIMER1_CNT);
}
-static struct clocksource arc_counter = {
+static struct clocksource arc_counter_timer1 = {
.name = "ARC Timer1",
.rating = 300,
- .read = arc_counter_read,
+ .read = arc_read_timer1,
.mask = CLOCKSOURCE_MASK(32),
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
};
-#endif
-#endif
+static void __init arc_cs_setup_timer1(struct device_node *node)
+{
+ int ret;
+
+ /* Local to CPU hence not usable in SMP */
+ if (IS_ENABLED(CONFIG_SMP))
+ return;
+
+ ret = arc_get_timer_clk(node);
+ if (ret)
+ return;
+
+ write_aux_reg(ARC_REG_TIMER1_LIMIT, ARC_TIMER_MAX);
+ write_aux_reg(ARC_REG_TIMER1_CNT, 0);
+ write_aux_reg(ARC_REG_TIMER1_CTRL, TIMER_CTRL_NH);
+
+ clocksource_register_hz(&arc_counter_timer1, arc_timer_freq);
+}
/********** Clock Event Device *********/
+static int arc_timer_irq;
+
/*
* Arm the timer to interrupt after @cycles
* The distinction for oneshot/periodic is done in arc_event_timer_ack() below
@@ -209,7 +262,7 @@
* At X Hz, 1 sec = 1000ms -> X cycles;
* 10ms -> X / 100 cycles
*/
- arc_timer_event_setup(arc_get_core_freq() / HZ);
+ arc_timer_event_setup(arc_timer_freq / HZ);
return 0;
}
@@ -218,7 +271,6 @@
.features = CLOCK_EVT_FEAT_ONESHOT |
CLOCK_EVT_FEAT_PERIODIC,
.rating = 300,
- .irq = TIMER0_IRQ, /* hardwired, no need for resources */
.set_next_event = arc_clkevent_set_next_event,
.set_state_periodic = arc_clkevent_set_periodic,
};
@@ -244,45 +296,81 @@
return IRQ_HANDLED;
}
-/*
- * Setup the local event timer for @cpu
- */
-void arc_local_timer_setup()
+static int arc_timer_cpu_notify(struct notifier_block *self,
+ unsigned long action, void *hcpu)
{
struct clock_event_device *evt = this_cpu_ptr(&arc_clockevent_device);
- int cpu = smp_processor_id();
- evt->cpumask = cpumask_of(cpu);
- clockevents_config_and_register(evt, arc_get_core_freq(),
+ evt->cpumask = cpumask_of(smp_processor_id());
+
+ switch (action & ~CPU_TASKS_FROZEN) {
+ case CPU_STARTING:
+ clockevents_config_and_register(evt, arc_timer_freq,
+ 0, ULONG_MAX);
+ enable_percpu_irq(arc_timer_irq, 0);
+ break;
+ case CPU_DYING:
+ disable_percpu_irq(arc_timer_irq);
+ break;
+ }
+
+ return NOTIFY_OK;
+}
+
+static struct notifier_block arc_timer_cpu_nb = {
+ .notifier_call = arc_timer_cpu_notify,
+};
+
+/*
+ * clockevent setup for boot CPU
+ */
+static void __init arc_clockevent_setup(struct device_node *node)
+{
+ struct clock_event_device *evt = this_cpu_ptr(&arc_clockevent_device);
+ int ret;
+
+ register_cpu_notifier(&arc_timer_cpu_nb);
+
+ arc_timer_irq = irq_of_parse_and_map(node, 0);
+ if (arc_timer_irq <= 0)
+ panic("clockevent: missing irq");
+
+ ret = arc_get_timer_clk(node);
+ if (ret)
+ panic("clockevent: missing clk");
+
+ evt->irq = arc_timer_irq;
+ evt->cpumask = cpumask_of(smp_processor_id());
+ clockevents_config_and_register(evt, arc_timer_freq,
0, ARC_TIMER_MAX);
- /* setup the per-cpu timer IRQ handler - for all cpus */
- arc_request_percpu_irq(TIMER0_IRQ, cpu, timer_irq_handler,
- "Timer0 (per-cpu-tick)", evt);
+ /* Needs apriori irq_set_percpu_devid() done in intc map function */
+ ret = request_percpu_irq(arc_timer_irq, timer_irq_handler,
+ "Timer0 (per-cpu-tick)", evt);
+ if (ret)
+ panic("clockevent: unable to request irq\n");
+
+ enable_percpu_irq(arc_timer_irq, 0);
}
+static void __init arc_of_timer_init(struct device_node *np)
+{
+ static int init_count = 0;
+
+ if (!init_count) {
+ init_count = 1;
+ arc_clockevent_setup(np);
+ } else {
+ arc_cs_setup_timer1(np);
+ }
+}
+CLOCKSOURCE_OF_DECLARE(arc_clkevt, "snps,arc-timer", arc_of_timer_init);
+
/*
* Called from start_kernel() - boot CPU only
- *
- * -Sets up h/w timers as applicable on boot cpu
- * -Also sets up any global state needed for timer subsystem:
- * - for "counting" timer, registers a clocksource, usable across CPUs
- * (provided that underlying counter h/w is synchronized across cores)
- * - for "event" timer, sets up TIMER0 IRQ (as that is platform agnostic)
*/
void __init time_init(void)
{
- /*
- * sets up the timekeeping free-flowing counter which also returns
- * whether the counter is usable as clocksource
- */
- if (arc_counter_setup())
- /*
- * CLK upto 4.29 GHz can be safely represented in 32 bits
- * because Max 32 bit number is 4,294,967,295
- */
- clocksource_register_hz(&arc_counter, arc_get_core_freq());
-
- /* sets up the periodic event timer */
- arc_local_timer_setup();
+ of_clk_init(NULL);
+ clocksource_probe();
}
diff --git a/arch/arc/mm/tlb.c b/arch/arc/mm/tlb.c
index 7046c12..ec868a9 100644
--- a/arch/arc/mm/tlb.c
+++ b/arch/arc/mm/tlb.c
@@ -814,6 +814,17 @@
printk(arc_mmu_mumbojumbo(0, str, sizeof(str)));
+ /*
+ * Can't be done in processor.h due to header include depenedencies
+ */
+ BUILD_BUG_ON(!IS_ALIGNED((CONFIG_ARC_KVADDR_SIZE << 20), PMD_SIZE));
+
+ /*
+ * stack top size sanity check,
+ * Can't be done in processor.h due to header include depenedencies
+ */
+ BUILD_BUG_ON(!IS_ALIGNED(STACK_TOP, PMD_SIZE));
+
/* For efficiency sake, kernel is compile time built for a MMU ver
* This must match the hardware it is running on.
* Linux built for MMU V2, if run on MMU V1 will break down because V1
diff --git a/arch/arc/plat-axs10x/axs10x.c b/arch/arc/plat-axs10x/axs10x.c
index 1b0f0f4..8654870 100644
--- a/arch/arc/plat-axs10x/axs10x.c
+++ b/arch/arc/plat-axs10x/axs10x.c
@@ -14,10 +14,11 @@
*
*/
+#include <linux/of_fdt.h>
#include <linux/of_platform.h>
+#include <linux/libfdt.h>
#include <asm/asm-offsets.h>
-#include <asm/clk.h>
#include <asm/io.h>
#include <asm/mach_desc.h>
#include <asm/mcip.h>
@@ -389,6 +390,13 @@
static void __init axs103_early_init(void)
{
+ int offset = fdt_path_offset(initial_boot_params, "/cpu_card/core_clk");
+ const struct fdt_property *prop = fdt_get_property(initial_boot_params,
+ offset,
+ "clock-frequency",
+ NULL);
+ u32 freq = be32_to_cpu(*(u32*)(prop->data)) / 1000000, orig = freq;
+
/*
* AXS103 configurations for SMP/QUAD configurations share device tree
* which defaults to 90 MHz. However recent failures of Quad config
@@ -401,12 +409,10 @@
#ifdef CONFIG_ARC_MCIP
unsigned int num_cores = (read_aux_reg(ARC_REG_MCIP_BCR) >> 16) & 0x3F;
if (num_cores > 2)
- arc_set_core_freq(50 * 1000000);
- else if (num_cores == 2)
- arc_set_core_freq(75 * 1000000);
+ freq = 50;
#endif
- switch (arc_get_core_freq()/1000000) {
+ switch (freq) {
case 33:
axs103_set_freq(1, 1, 1);
break;
@@ -431,11 +437,18 @@
* DT "clock-frequency" might not match with board value.
* Hence update it to match the board value.
*/
- arc_set_core_freq(axs103_get_freq() * 1000000);
+ freq = axs103_get_freq();
break;
}
- pr_info("Freq is %dMHz\n", axs103_get_freq());
+ pr_info("Freq is %dMHz\n", freq);
+
+ /* Patching .dtb in-place with new core clock value */
+ if (freq != orig ) {
+ freq = cpu_to_be32(freq * 1000000);
+ fdt_setprop_inplace(initial_boot_params, offset,
+ "clock-frequency", &freq, sizeof(freq));
+ }
/* Memory maps already config in pre-bootloader */
diff --git a/arch/arc/plat-eznps/Kconfig b/arch/arc/plat-eznps/Kconfig
new file mode 100644
index 0000000..1d175cc
--- /dev/null
+++ b/arch/arc/plat-eznps/Kconfig
@@ -0,0 +1,35 @@
+#
+# For a description of the syntax of this configuration file,
+# see Documentation/kbuild/kconfig-language.txt.
+#
+
+menuconfig ARC_PLAT_EZNPS
+ bool "\"EZchip\" ARC dev platform"
+ select ARC_HAS_COH_CACHES if SMP
+ select CPU_BIG_ENDIAN
+ select CLKSRC_NPS
+ select EZNPS_GIC
+ select EZCHIP_NPS_MANAGEMENT_ENET if ETHERNET
+ help
+ Support for EZchip development platforms,
+ based on ARC700 cores.
+ We handle few flavours:
+ - Hardware Emulator AKA HE which is FPGA based chasis
+ - Simulator based on MetaWare nSIM
+ - NPS400 chip based on ASIC
+
+config EZNPS_MTM_EXT
+ bool "ARC-EZchip MTM Extensions"
+ select CPUMASK_OFFSTACK
+ depends on ARC_PLAT_EZNPS && SMP
+ default y
+ help
+ Here we add new hierarchy for CPUs topology.
+ We got:
+ Core
+ Thread
+ At the new thread level each CPU represent one HW thread.
+ At highest hierarchy each core contain 16 threads,
+ any of them seem like CPU from Linux point of view.
+ All threads within same core share the execution unit of the
+ core and HW scheduler round robin between them.
diff --git a/arch/arc/plat-eznps/Makefile b/arch/arc/plat-eznps/Makefile
new file mode 100644
index 0000000..21091b1
--- /dev/null
+++ b/arch/arc/plat-eznps/Makefile
@@ -0,0 +1,7 @@
+#
+# Makefile for the linux kernel.
+#
+
+obj-y := entry.o platform.o
+obj-$(CONFIG_SMP) += smp.o
+obj-$(CONFIG_EZNPS_MTM_EXT) += mtm.o
diff --git a/arch/arc/plat-eznps/entry.S b/arch/arc/plat-eznps/entry.S
new file mode 100644
index 0000000..328261c
--- /dev/null
+++ b/arch/arc/plat-eznps/entry.S
@@ -0,0 +1,70 @@
+/*******************************************************************************
+
+ EZNPS CPU startup Code
+ Copyright(c) 2012 EZchip Technologies.
+
+ This program is free software; you can redistribute it and/or modify it
+ under the terms and conditions of the GNU General Public License,
+ version 2, as published by the Free Software Foundation.
+
+ This program is distributed in the hope it will be useful, but WITHOUT
+ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ more details.
+
+ The full GNU General Public License is included in this distribution in
+ the file called "COPYING".
+
+*******************************************************************************/
+#include <linux/linkage.h>
+#include <asm/entry.h>
+#include <asm/cache.h>
+#include <plat/ctop.h>
+
+ .cpu A7
+
+ .section .init.text, "ax",@progbits
+ .align 1024 ; HW requierment for restart first PC
+
+ENTRY(res_service)
+#ifdef CONFIG_EZNPS_MTM_EXT
+ ; There is no work for HW thread id != 0
+ lr r3, [CTOP_AUX_THREAD_ID]
+ cmp r3, 0
+ jne stext
+#endif
+
+#ifdef CONFIG_ARC_HAS_DCACHE
+ ; With no cache coherency mechanism D$ need to be used very carefully.
+ ; Address space:
+ ; 0G-2G: We disable CONFIG_ARC_CACHE_PAGES.
+ ; 2G-3G: We disable D$ by setting this bit.
+ ; 3G-4G: D$ is disabled by architecture.
+ ; FMT are huge pages for user application reside at 0-2G.
+ ; Only FMT left as one who can use D$ where each such page got
+ ; disable/enable bit for cachability.
+ ; Programmer will use FMT pages for private data so cache coherency
+ ; would not be a problem.
+ ; First thing we invalidate D$
+ sr 1, [ARC_REG_DC_IVDC]
+ sr HW_COMPLY_KRN_NOT_D_CACHED, [CTOP_AUX_HW_COMPLY]
+#endif
+
+#ifdef CONFIG_SMP
+ ; We set logical cpuid to be used by GET_CPUID
+ ; We do not use physical cpuid since we want ids to be continious when
+ ; it comes to cpus on the same quad cluster.
+ ; This is useful for applications that used shared resources of a quad
+ ; cluster such SRAMS.
+ lr r3, [CTOP_AUX_CORE_ID]
+ sr r3, [CTOP_AUX_LOGIC_CORE_ID]
+ lr r3, [CTOP_AUX_CLUSTER_ID]
+ ; Set logical is acheived by swap of 2 middle bits of cluster id (4 bit)
+ ; r3 is used since we use short instruction and we need q-class reg
+ .short CTOP_INST_MOV2B_FLIP_R3_B1_B2_INST
+ .word CTOP_INST_MOV2B_FLIP_R3_B1_B2_LIMM
+ sr r3, [CTOP_AUX_LOGIC_CLUSTER_ID]
+#endif
+
+ j stext
+END(res_service)
diff --git a/arch/arc/plat-eznps/include/plat/ctop.h b/arch/arc/plat-eznps/include/plat/ctop.h
new file mode 100644
index 0000000..9d6718c
--- /dev/null
+++ b/arch/arc/plat-eznps/include/plat/ctop.h
@@ -0,0 +1,209 @@
+/*
+ * Copyright(c) 2015 EZchip Technologies.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ */
+
+#ifndef _PLAT_EZNPS_CTOP_H
+#define _PLAT_EZNPS_CTOP_H
+
+#ifndef CONFIG_ARC_PLAT_EZNPS
+#error "Incorrect ctop.h include"
+#endif
+
+#include <soc/nps/common.h>
+
+/* core auxiliary registers */
+#ifdef __ASSEMBLY__
+#define CTOP_AUX_BASE (-0x800)
+#else
+#define CTOP_AUX_BASE 0xFFFFF800
+#endif
+
+#define CTOP_AUX_GLOBAL_ID (CTOP_AUX_BASE + 0x000)
+#define CTOP_AUX_CLUSTER_ID (CTOP_AUX_BASE + 0x004)
+#define CTOP_AUX_CORE_ID (CTOP_AUX_BASE + 0x008)
+#define CTOP_AUX_THREAD_ID (CTOP_AUX_BASE + 0x00C)
+#define CTOP_AUX_LOGIC_GLOBAL_ID (CTOP_AUX_BASE + 0x010)
+#define CTOP_AUX_LOGIC_CLUSTER_ID (CTOP_AUX_BASE + 0x014)
+#define CTOP_AUX_LOGIC_CORE_ID (CTOP_AUX_BASE + 0x018)
+#define CTOP_AUX_MT_CTRL (CTOP_AUX_BASE + 0x020)
+#define CTOP_AUX_HW_COMPLY (CTOP_AUX_BASE + 0x024)
+#define CTOP_AUX_LPC (CTOP_AUX_BASE + 0x030)
+#define CTOP_AUX_EFLAGS (CTOP_AUX_BASE + 0x080)
+#define CTOP_AUX_IACK (CTOP_AUX_BASE + 0x088)
+#define CTOP_AUX_GPA1 (CTOP_AUX_BASE + 0x08C)
+#define CTOP_AUX_UDMC (CTOP_AUX_BASE + 0x300)
+
+/* EZchip core instructions */
+#define CTOP_INST_HWSCHD_OFF_R3 0x3B6F00BF
+#define CTOP_INST_HWSCHD_OFF_R4 0x3C6F00BF
+#define CTOP_INST_HWSCHD_RESTORE_R3 0x3E6F70C3
+#define CTOP_INST_HWSCHD_RESTORE_R4 0x3E6F7103
+#define CTOP_INST_SCHD_RW 0x3E6F7004
+#define CTOP_INST_SCHD_RD 0x3E6F7084
+#define CTOP_INST_ASRI_0_R3 0x3B56003E
+#define CTOP_INST_XEX_DI_R2_R2_R3 0x4A664C00
+#define CTOP_INST_EXC_DI_R2_R2_R3 0x4A664C01
+#define CTOP_INST_AADD_DI_R2_R2_R3 0x4A664C02
+#define CTOP_INST_AAND_DI_R2_R2_R3 0x4A664C04
+#define CTOP_INST_AOR_DI_R2_R2_R3 0x4A664C05
+#define CTOP_INST_AXOR_DI_R2_R2_R3 0x4A664C06
+
+/* Do not use D$ for address in 2G-3G */
+#define HW_COMPLY_KRN_NOT_D_CACHED _BITUL(28)
+
+#define NPS_MSU_EN_CFG 0x80
+#define NPS_CRG_BLKID 0x480
+#define NPS_CRG_SYNC_BIT _BITUL(0)
+#define NPS_GIM_BLKID 0x5C0
+
+/* GIM registers and fields*/
+#define NPS_GIM_UART_LINE _BITUL(7)
+#define NPS_GIM_DBG_LAN_EAST_TX_DONE_LINE _BITUL(10)
+#define NPS_GIM_DBG_LAN_EAST_RX_RDY_LINE _BITUL(11)
+#define NPS_GIM_DBG_LAN_WEST_TX_DONE_LINE _BITUL(25)
+#define NPS_GIM_DBG_LAN_WEST_RX_RDY_LINE _BITUL(26)
+
+#ifndef __ASSEMBLY__
+/* Functional registers definition */
+struct nps_host_reg_mtm_cfg {
+ union {
+ struct {
+ u32 gen:1, gdis:1, clk_gate_dis:1, asb:1,
+ __reserved:9, nat:3, ten:16;
+ };
+ u32 value;
+ };
+};
+
+struct nps_host_reg_mtm_cpu_cfg {
+ union {
+ struct {
+ u32 csa:22, dmsid:6, __reserved:3, cs:1;
+ };
+ u32 value;
+ };
+};
+
+struct nps_host_reg_thr_init {
+ union {
+ struct {
+ u32 str:1, __reserved:27, thr_id:4;
+ };
+ u32 value;
+ };
+};
+
+struct nps_host_reg_thr_init_sts {
+ union {
+ struct {
+ u32 bsy:1, err:1, __reserved:26, thr_id:4;
+ };
+ u32 value;
+ };
+};
+
+struct nps_host_reg_msu_en_cfg {
+ union {
+ struct {
+ u32 __reserved1:11,
+ rtc_en:1, ipc_en:1, gim_1_en:1,
+ gim_0_en:1, ipi_en:1, buff_e_rls_bmuw:1,
+ buff_e_alc_bmuw:1, buff_i_rls_bmuw:1, buff_i_alc_bmuw:1,
+ buff_e_rls_bmue:1, buff_e_alc_bmue:1, buff_i_rls_bmue:1,
+ buff_i_alc_bmue:1, __reserved2:1, buff_e_pre_en:1,
+ buff_i_pre_en:1, pmuw_ja_en:1, pmue_ja_en:1,
+ pmuw_nj_en:1, pmue_nj_en:1, msu_en:1;
+ };
+ u32 value;
+ };
+};
+
+struct nps_host_reg_gim_p_int_dst {
+ union {
+ struct {
+ u32 int_out_en:1, __reserved1:4,
+ is:1, intm:2, __reserved2:4,
+ nid:4, __reserved3:4, cid:4,
+ __reserved4:4, tid:4;
+ };
+ u32 value;
+ };
+};
+
+/* AUX registers definition */
+struct nps_host_reg_aux_udmc {
+ union {
+ struct {
+ u32 dcp:1, cme:1, __reserved:19, nat:3,
+ __reserved2:5, dcas:3;
+ };
+ u32 value;
+ };
+};
+
+struct nps_host_reg_aux_mt_ctrl {
+ union {
+ struct {
+ u32 mten:1, hsen:1, scd:1, sten:1,
+ st_cnt:8, __reserved:8,
+ hs_cnt:8, __reserved1:4;
+ };
+ u32 value;
+ };
+};
+
+struct nps_host_reg_aux_hw_comply {
+ union {
+ struct {
+ u32 me:1, le:1, te:1, knc:1, __reserved:28;
+ };
+ u32 value;
+ };
+};
+
+struct nps_host_reg_aux_lpc {
+ union {
+ struct {
+ u32 mep:1, __reserved:31;
+ };
+ u32 value;
+ };
+};
+
+/* CRG registers */
+#define REG_GEN_PURP_0 nps_host_reg_non_cl(NPS_CRG_BLKID, 0x1BF)
+
+/* GIM registers */
+#define REG_GIM_P_INT_EN_0 nps_host_reg_non_cl(NPS_GIM_BLKID, 0x100)
+#define REG_GIM_P_INT_POL_0 nps_host_reg_non_cl(NPS_GIM_BLKID, 0x110)
+#define REG_GIM_P_INT_SENS_0 nps_host_reg_non_cl(NPS_GIM_BLKID, 0x114)
+#define REG_GIM_P_INT_BLK_0 nps_host_reg_non_cl(NPS_GIM_BLKID, 0x118)
+#define REG_GIM_P_INT_DST_10 nps_host_reg_non_cl(NPS_GIM_BLKID, 0x13A)
+#define REG_GIM_P_INT_DST_11 nps_host_reg_non_cl(NPS_GIM_BLKID, 0x13B)
+#define REG_GIM_P_INT_DST_25 nps_host_reg_non_cl(NPS_GIM_BLKID, 0x149)
+#define REG_GIM_P_INT_DST_26 nps_host_reg_non_cl(NPS_GIM_BLKID, 0x14A)
+
+#else
+
+.macro GET_CPU_ID reg
+ lr \reg, [CTOP_AUX_LOGIC_GLOBAL_ID]
+#ifndef CONFIG_EZNPS_MTM_EXT
+ lsr \reg, \reg, 4
+#endif
+.endm
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* _PLAT_EZNPS_CTOP_H */
diff --git a/arch/arc/plat-eznps/include/plat/mtm.h b/arch/arc/plat-eznps/include/plat/mtm.h
new file mode 100644
index 0000000..29b91b5
--- /dev/null
+++ b/arch/arc/plat-eznps/include/plat/mtm.h
@@ -0,0 +1,60 @@
+/*
+ * Copyright(c) 2015 EZchip Technologies.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ */
+
+#ifndef _PLAT_EZNPS_MTM_H
+#define _PLAT_EZNPS_MTM_H
+
+#include <plat/ctop.h>
+
+static inline void *nps_mtm_reg_addr(u32 cpu, u32 reg)
+{
+ struct global_id gid;
+ u32 core, blkid;
+
+ gid.value = cpu;
+ core = gid.core;
+ blkid = (((core & 0x0C) << 2) | (core & 0x03));
+
+ return nps_host_reg(cpu, blkid, reg);
+}
+
+#ifdef CONFIG_EZNPS_MTM_EXT
+#define NPS_CPU_TO_THREAD_NUM(cpu) \
+ ({ struct global_id gid; gid.value = cpu; gid.thread; })
+
+/* MTM registers */
+#define MTM_CFG(cpu) nps_mtm_reg_addr(cpu, 0x81)
+#define MTM_THR_INIT(cpu) nps_mtm_reg_addr(cpu, 0x92)
+#define MTM_THR_INIT_STS(cpu) nps_mtm_reg_addr(cpu, 0x93)
+
+#define get_thread(map) map.thread
+#define eznps_max_cpus 4096
+#define eznps_cpus_per_cluster 256
+
+void mtm_enable_core(unsigned int cpu);
+int mtm_enable_thread(int cpu);
+#else /* !CONFIG_EZNPS_MTM_EXT */
+
+#define get_thread(map) 0
+#define eznps_max_cpus 256
+#define eznps_cpus_per_cluster 16
+#define mtm_enable_core(cpu)
+#define mtm_enable_thread(cpu) 1
+#define NPS_CPU_TO_THREAD_NUM(cpu) 0
+
+#endif /* CONFIG_EZNPS_MTM_EXT */
+
+#endif /* _PLAT_EZNPS_MTM_H */
diff --git a/arch/arc/plat-eznps/include/plat/smp.h b/arch/arc/plat-eznps/include/plat/smp.h
new file mode 100644
index 0000000..06b59bd
--- /dev/null
+++ b/arch/arc/plat-eznps/include/plat/smp.h
@@ -0,0 +1,26 @@
+/*
+ * Copyright(c) 2015 EZchip Technologies.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ */
+
+#ifndef __PLAT_EZNPS_SMP_H
+#define __PLAT_EZNPS_SMP_H
+
+#ifdef CONFIG_SMP
+
+extern void res_service(void);
+
+#endif /* CONFIG_SMP */
+
+#endif
diff --git a/arch/arc/plat-eznps/mtm.c b/arch/arc/plat-eznps/mtm.c
new file mode 100644
index 0000000..aaaaffd
--- /dev/null
+++ b/arch/arc/plat-eznps/mtm.c
@@ -0,0 +1,133 @@
+/*
+ * Copyright(c) 2015 EZchip Technologies.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ */
+
+#include <linux/smp.h>
+#include <linux/io.h>
+#include <linux/log2.h>
+#include <asm/arcregs.h>
+#include <plat/mtm.h>
+#include <plat/smp.h>
+
+#define MT_CTRL_HS_CNT 0xFF
+#define MT_CTRL_ST_CNT 0xF
+#define NPS_NUM_HW_THREADS 0x10
+
+static void mtm_init_nat(int cpu)
+{
+ struct nps_host_reg_mtm_cfg mtm_cfg;
+ struct nps_host_reg_aux_udmc udmc;
+ int log_nat, nat = 0, i, t;
+
+ /* Iterate core threads and update nat */
+ for (i = 0, t = cpu; i < NPS_NUM_HW_THREADS; i++, t++)
+ nat += test_bit(t, cpumask_bits(cpu_possible_mask));
+
+ log_nat = ilog2(nat);
+
+ udmc.value = read_aux_reg(CTOP_AUX_UDMC);
+ udmc.nat = log_nat;
+ write_aux_reg(CTOP_AUX_UDMC, udmc.value);
+
+ mtm_cfg.value = ioread32be(MTM_CFG(cpu));
+ mtm_cfg.nat = log_nat;
+ iowrite32be(mtm_cfg.value, MTM_CFG(cpu));
+}
+
+static void mtm_init_thread(int cpu)
+{
+ int i, tries = 5;
+ struct nps_host_reg_thr_init thr_init;
+ struct nps_host_reg_thr_init_sts thr_init_sts;
+
+ /* Set thread init register */
+ thr_init.value = 0;
+ iowrite32be(thr_init.value, MTM_THR_INIT(cpu));
+ thr_init.thr_id = NPS_CPU_TO_THREAD_NUM(cpu);
+ thr_init.str = 1;
+ iowrite32be(thr_init.value, MTM_THR_INIT(cpu));
+
+ /* Poll till thread init is done */
+ for (i = 0; i < tries; i++) {
+ thr_init_sts.value = ioread32be(MTM_THR_INIT_STS(cpu));
+ if (thr_init_sts.thr_id == thr_init.thr_id) {
+ if (thr_init_sts.bsy)
+ continue;
+ else if (thr_init_sts.err)
+ pr_warn("Failed to thread init cpu %u\n", cpu);
+ break;
+ }
+
+ pr_warn("Wrong thread id in thread init for cpu %u\n", cpu);
+ break;
+ }
+
+ if (i == tries)
+ pr_warn("Got thread init timeout for cpu %u\n", cpu);
+}
+
+int mtm_enable_thread(int cpu)
+{
+ struct nps_host_reg_mtm_cfg mtm_cfg;
+
+ if (NPS_CPU_TO_THREAD_NUM(cpu) == 0)
+ return 1;
+
+ /* Enable thread in mtm */
+ mtm_cfg.value = ioread32be(MTM_CFG(cpu));
+ mtm_cfg.ten |= (1 << (NPS_CPU_TO_THREAD_NUM(cpu)));
+ iowrite32be(mtm_cfg.value, MTM_CFG(cpu));
+
+ return 0;
+}
+
+void mtm_enable_core(unsigned int cpu)
+{
+ int i;
+ struct nps_host_reg_aux_mt_ctrl mt_ctrl;
+ struct nps_host_reg_mtm_cfg mtm_cfg;
+
+ if (NPS_CPU_TO_THREAD_NUM(cpu) != 0)
+ return;
+
+ /* Initialize Number of Active Threads */
+ mtm_init_nat(cpu);
+
+ /* Initialize mtm_cfg */
+ mtm_cfg.value = ioread32be(MTM_CFG(cpu));
+ mtm_cfg.ten = 1;
+ iowrite32be(mtm_cfg.value, MTM_CFG(cpu));
+
+ /* Initialize all other threads in core */
+ for (i = 1; i < NPS_NUM_HW_THREADS; i++)
+ mtm_init_thread(cpu + i);
+
+
+ /* Enable HW schedule, stall counter, mtm */
+ mt_ctrl.value = 0;
+ mt_ctrl.hsen = 1;
+ mt_ctrl.hs_cnt = MT_CTRL_HS_CNT;
+ mt_ctrl.sten = 1;
+ mt_ctrl.st_cnt = MT_CTRL_ST_CNT;
+ mt_ctrl.mten = 1;
+ write_aux_reg(CTOP_AUX_MT_CTRL, mt_ctrl.value);
+
+ /*
+ * HW scheduling mechanism will start working
+ * Only after call to instruction "schd.rw".
+ * cpu_relax() calls "schd.rw" instruction.
+ */
+ cpu_relax();
+}
diff --git a/arch/arc/plat-eznps/platform.c b/arch/arc/plat-eznps/platform.c
new file mode 100644
index 0000000..7ad6d2b
--- /dev/null
+++ b/arch/arc/plat-eznps/platform.c
@@ -0,0 +1,102 @@
+/*
+ * Copyright(c) 2015 EZchip Technologies.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ */
+
+#include <linux/init.h>
+#include <linux/io.h>
+#include <asm/mach_desc.h>
+#include <plat/mtm.h>
+
+static void __init eznps_configure_msu(void)
+{
+ int cpu;
+ struct nps_host_reg_msu_en_cfg msu_en_cfg = {.value = 0};
+
+ msu_en_cfg.msu_en = 1;
+ msu_en_cfg.ipi_en = 1;
+ msu_en_cfg.gim_0_en = 1;
+ msu_en_cfg.gim_1_en = 1;
+
+ /* enable IPI and GIM messages on all clusters */
+ for (cpu = 0 ; cpu < eznps_max_cpus; cpu += eznps_cpus_per_cluster)
+ iowrite32be(msu_en_cfg.value,
+ nps_host_reg(cpu, NPS_MSU_BLKID, NPS_MSU_EN_CFG));
+}
+
+static void __init eznps_configure_gim(void)
+{
+ u32 reg_value;
+ u32 gim_int_lines;
+ struct nps_host_reg_gim_p_int_dst gim_p_int_dst = {.value = 0};
+
+ gim_int_lines = NPS_GIM_UART_LINE;
+ gim_int_lines |= NPS_GIM_DBG_LAN_EAST_TX_DONE_LINE;
+ gim_int_lines |= NPS_GIM_DBG_LAN_EAST_RX_RDY_LINE;
+ gim_int_lines |= NPS_GIM_DBG_LAN_WEST_TX_DONE_LINE;
+ gim_int_lines |= NPS_GIM_DBG_LAN_WEST_RX_RDY_LINE;
+
+ /*
+ * IRQ polarity
+ * low or high level
+ * negative or positive edge
+ */
+ reg_value = ioread32be(REG_GIM_P_INT_POL_0);
+ reg_value &= ~gim_int_lines;
+ iowrite32be(reg_value, REG_GIM_P_INT_POL_0);
+
+ /* IRQ type level or edge */
+ reg_value = ioread32be(REG_GIM_P_INT_SENS_0);
+ reg_value |= NPS_GIM_DBG_LAN_EAST_TX_DONE_LINE;
+ reg_value |= NPS_GIM_DBG_LAN_WEST_TX_DONE_LINE;
+ iowrite32be(reg_value, REG_GIM_P_INT_SENS_0);
+
+ /*
+ * GIM interrupt select type for
+ * dbg_lan TX and RX interrupts
+ * should be type 1
+ * type 0 = IRQ line 6
+ * type 1 = IRQ line 7
+ */
+ gim_p_int_dst.is = 1;
+ iowrite32be(gim_p_int_dst.value, REG_GIM_P_INT_DST_10);
+ iowrite32be(gim_p_int_dst.value, REG_GIM_P_INT_DST_11);
+ iowrite32be(gim_p_int_dst.value, REG_GIM_P_INT_DST_25);
+ iowrite32be(gim_p_int_dst.value, REG_GIM_P_INT_DST_26);
+
+ /*
+ * CTOP IRQ lines should be defined
+ * as blocking in GIM
+ */
+ iowrite32be(gim_int_lines, REG_GIM_P_INT_BLK_0);
+
+ /* enable CTOP IRQ lines in GIM */
+ iowrite32be(gim_int_lines, REG_GIM_P_INT_EN_0);
+}
+
+static void __init eznps_early_init(void)
+{
+ eznps_configure_msu();
+ eznps_configure_gim();
+}
+
+static const char *eznps_compat[] __initconst = {
+ "ezchip,arc-nps",
+ NULL,
+};
+
+MACHINE_START(NPS, "nps")
+ .dt_compat = eznps_compat,
+ .init_early = eznps_early_init,
+MACHINE_END
diff --git a/arch/arc/plat-eznps/smp.c b/arch/arc/plat-eznps/smp.c
new file mode 100644
index 0000000..5e901f8
--- /dev/null
+++ b/arch/arc/plat-eznps/smp.c
@@ -0,0 +1,155 @@
+/*
+ * Copyright(c) 2015 EZchip Technologies.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ */
+
+#include <linux/smp.h>
+#include <linux/of_fdt.h>
+#include <linux/io.h>
+#include <linux/irqdomain.h>
+#include <asm/irq.h>
+#include <plat/ctop.h>
+#include <plat/smp.h>
+#include <plat/mtm.h>
+
+#define NPS_DEFAULT_MSID 0x34
+#define NPS_MTM_CPU_CFG 0x90
+
+static char smp_cpuinfo_buf[128] = {"Extn [EZNPS-SMP]\t: On\n"};
+
+/* Get cpu map from device tree */
+static int __init eznps_get_map(const char *name, struct cpumask *cpumask)
+{
+ unsigned long dt_root = of_get_flat_dt_root();
+ const char *buf;
+
+ buf = of_get_flat_dt_prop(dt_root, name, NULL);
+ if (!buf)
+ return 1;
+
+ cpulist_parse(buf, cpumask);
+
+ return 0;
+}
+
+/* Update board cpu maps */
+static void __init eznps_init_cpumasks(void)
+{
+ struct cpumask cpumask;
+
+ if (eznps_get_map("present-cpus", &cpumask)) {
+ pr_err("Failed to get present-cpus from dtb");
+ return;
+ }
+ init_cpu_present(&cpumask);
+
+ if (eznps_get_map("possible-cpus", &cpumask)) {
+ pr_err("Failed to get possible-cpus from dtb");
+ return;
+ }
+ init_cpu_possible(&cpumask);
+}
+
+static void eznps_init_core(unsigned int cpu)
+{
+ u32 sync_value;
+ struct nps_host_reg_aux_hw_comply hw_comply;
+ struct nps_host_reg_aux_lpc lpc;
+
+ if (NPS_CPU_TO_THREAD_NUM(cpu) != 0)
+ return;
+
+ hw_comply.value = read_aux_reg(CTOP_AUX_HW_COMPLY);
+ hw_comply.me = 1;
+ hw_comply.le = 1;
+ hw_comply.te = 1;
+ write_aux_reg(CTOP_AUX_HW_COMPLY, hw_comply.value);
+
+ /* Enable MMU clock */
+ lpc.mep = 1;
+ write_aux_reg(CTOP_AUX_LPC, lpc.value);
+
+ /* Boot CPU only */
+ if (!cpu) {
+ /* Write to general purpose register in CRG */
+ sync_value = ioread32be(REG_GEN_PURP_0);
+ sync_value |= NPS_CRG_SYNC_BIT;
+ iowrite32be(sync_value, REG_GEN_PURP_0);
+ }
+}
+
+/*
+ * Master kick starting another CPU
+ */
+static void __init eznps_smp_wakeup_cpu(int cpu, unsigned long pc)
+{
+ struct nps_host_reg_mtm_cpu_cfg cpu_cfg;
+
+ if (mtm_enable_thread(cpu) == 0)
+ return;
+
+ /* set PC, dmsid, and start CPU */
+ cpu_cfg.value = (u32)res_service;
+ cpu_cfg.dmsid = NPS_DEFAULT_MSID;
+ cpu_cfg.cs = 1;
+ iowrite32be(cpu_cfg.value, nps_mtm_reg_addr(cpu, NPS_MTM_CPU_CFG));
+}
+
+static void eznps_ipi_send(int cpu)
+{
+ struct global_id gid;
+ struct {
+ union {
+ struct {
+ u32 num:8, cluster:8, core:8, thread:8;
+ };
+ u32 value;
+ };
+ } ipi;
+
+ gid.value = cpu;
+ ipi.thread = get_thread(gid);
+ ipi.core = gid.core;
+ ipi.cluster = nps_cluster_logic_to_phys(gid.cluster);
+ ipi.num = NPS_IPI_IRQ;
+
+ __asm__ __volatile__(
+ " mov r3, %0\n"
+ " .word %1\n"
+ :
+ : "r"(ipi.value), "i"(CTOP_INST_ASRI_0_R3)
+ : "r3");
+}
+
+static void eznps_init_per_cpu(int cpu)
+{
+ smp_ipi_irq_setup(cpu, NPS_IPI_IRQ);
+
+ eznps_init_core(cpu);
+ mtm_enable_core(cpu);
+}
+
+static void eznps_ipi_clear(int irq)
+{
+ write_aux_reg(CTOP_AUX_IACK, 1 << irq);
+}
+
+struct plat_smp_ops plat_smp_ops = {
+ .info = smp_cpuinfo_buf,
+ .init_early_smp = eznps_init_cpumasks,
+ .cpu_kick = eznps_smp_wakeup_cpu,
+ .ipi_send = eznps_ipi_send,
+ .init_per_cpu = eznps_init_per_cpu,
+ .ipi_clear = eznps_ipi_clear,
+};
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index b99d25b..90542db 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -50,6 +50,7 @@
select HAVE_DMA_CONTIGUOUS if MMU
select HAVE_DYNAMIC_FTRACE if (!XIP_KERNEL) && !CPU_ENDIAN_BE32 && MMU
select HAVE_EFFICIENT_UNALIGNED_ACCESS if (CPU_V6 || CPU_V6K || CPU_V7) && MMU
+ select HAVE_EXIT_THREAD
select HAVE_FTRACE_MCOUNT_RECORD if (!XIP_KERNEL)
select HAVE_FUNCTION_GRAPH_TRACER if (!THUMB2_KERNEL)
select HAVE_FUNCTION_TRACER if (!XIP_KERNEL)
@@ -66,6 +67,7 @@
select HAVE_KRETPROBES if (HAVE_KPROBES)
select HAVE_MEMBLOCK
select HAVE_MOD_ARCH_SPECIFIC
+ select HAVE_NMI
select HAVE_OPROFILE if (HAVE_PERF_EVENTS)
select HAVE_OPTPROBES if !THUMB2_KERNEL
select HAVE_PERF_EVENTS
diff --git a/arch/arm/boot/Makefile b/arch/arm/boot/Makefile
index 48fab15..446705a 100644
--- a/arch/arm/boot/Makefile
+++ b/arch/arm/boot/Makefile
@@ -88,7 +88,7 @@
$(call if_changed,objcopy)
@$(kecho) ' Kernel: $@ is ready'
-PHONY += initrd
+PHONY += initrd install zinstall uinstall
initrd:
@test "$(INITRD_PHYS)" != "" || \
(echo This machine does not support INITRD; exit -1)
diff --git a/arch/arm/boot/dts/Makefile b/arch/arm/boot/dts/Makefile
index 0f89d87..06b6c2d 100644
--- a/arch/arm/boot/dts/Makefile
+++ b/arch/arm/boot/dts/Makefile
@@ -399,6 +399,7 @@
imx6ul-tx6ul-mainboard.dtb
dtb-$(CONFIG_SOC_IMX7D) += \
imx7d-cl-som-imx7.dtb \
+ imx7d-nitrogen7.dtb \
imx7d-sbc-imx7.dtb \
imx7d-sdb.dtb
dtb-$(CONFIG_SOC_LS1021A) += \
diff --git a/arch/arm/boot/dts/bcm-cygnus.dtsi b/arch/arm/boot/dts/bcm-cygnus.dtsi
index 3878793..b42fe55 100644
--- a/arch/arm/boot/dts/bcm-cygnus.dtsi
+++ b/arch/arm/boot/dts/bcm-cygnus.dtsi
@@ -351,9 +351,16 @@
<&pinctrl 142 10 1>;
};
- touchscreen: tsc@180a6000 {
+ ts_adc_syscon: ts_adc_syscon@180a6000 {
+ compatible = "brcm,iproc-ts-adc-syscon", "syscon";
+ reg = <0x180a6000 0xc30>;
+ };
+
+ touchscreen: touchscreen@180a6000 {
compatible = "brcm,iproc-touchscreen";
- reg = <0x180a6000 0x40>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ts_syscon = <&ts_adc_syscon>;
clocks = <&asiu_clks BCM_CYGNUS_ASIU_ADC_CLK>;
clock-names = "tsc_clk";
interrupts = <GIC_SPI 164 IRQ_TYPE_LEVEL_HIGH>;
diff --git a/arch/arm/boot/dts/bcm283x.dtsi b/arch/arm/boot/dts/bcm283x.dtsi
index 31cc2f2..10b27b9 100644
--- a/arch/arm/boot/dts/bcm283x.dtsi
+++ b/arch/arm/boot/dts/bcm283x.dtsi
@@ -48,9 +48,29 @@
<1 24>,
<1 25>,
<1 26>,
+ /* dma channel 11-14 share one irq */
<1 27>,
+ <1 27>,
+ <1 27>,
+ <1 27>,
+ /* unused shared irq for all channels */
<1 28>;
-
+ interrupt-names = "dma0",
+ "dma1",
+ "dma2",
+ "dma3",
+ "dma4",
+ "dma5",
+ "dma6",
+ "dma7",
+ "dma8",
+ "dma9",
+ "dma10",
+ "dma11",
+ "dma12",
+ "dma13",
+ "dma14",
+ "dma-shared-all";
#dma-cells = <1>;
brcm,dma-channel-mask = <0x7f35>;
};
diff --git a/arch/arm/boot/dts/exynos3250-monk.dts b/arch/arm/boot/dts/exynos3250-monk.dts
index 267f81a..8c89062 100644
--- a/arch/arm/boot/dts/exynos3250-monk.dts
+++ b/arch/arm/boot/dts/exynos3250-monk.dts
@@ -14,6 +14,7 @@
/dts-v1/;
#include "exynos3250.dtsi"
+#include "exynos4412-ppmu-common.dtsi"
#include <dt-bindings/input/input.h>
#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/clock/samsung,s2mps11.h>
@@ -156,6 +157,12 @@
};
};
+&bus_dmc {
+ devfreq-events = <&ppmu_dmc0_3>, <&ppmu_dmc1_3>;
+ vdd-supply = <&buck1_reg>;
+ status = "okay";
+};
+
&cpu0 {
cpu0-supply = <&buck2_reg>;
};
@@ -458,46 +465,6 @@
status = "okay";
};
-&ppmu_dmc0 {
- status = "okay";
-
- events {
- ppmu_dmc0_3: ppmu-event3-dmc0 {
- event-name = "ppmu-event3-dmc0";
- };
- };
-};
-
-&ppmu_dmc1 {
- status = "okay";
-
- events {
- ppmu_dmc1_3: ppmu-event3-dmc1 {
- event-name = "ppmu-event3-dmc1";
- };
- };
-};
-
-&ppmu_leftbus {
- status = "okay";
-
- events {
- ppmu_leftbus_3: ppmu-event3-leftbus {
- event-name = "ppmu-event3-leftbus";
- };
- };
-};
-
-&ppmu_rightbus {
- status = "okay";
-
- events {
- ppmu_rightbus_3: ppmu-event3-rightbus {
- event-name = "ppmu-event3-rightbus";
- };
- };
-};
-
&xusbxti {
clock-frequency = <24000000>;
};
diff --git a/arch/arm/boot/dts/exynos3250-rinato.dts b/arch/arm/boot/dts/exynos3250-rinato.dts
index 31eb09b..e4228195 100644
--- a/arch/arm/boot/dts/exynos3250-rinato.dts
+++ b/arch/arm/boot/dts/exynos3250-rinato.dts
@@ -14,6 +14,7 @@
/dts-v1/;
#include "exynos3250.dtsi"
+#include "exynos4412-ppmu-common.dtsi"
#include <dt-bindings/input/input.h>
#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/clock/samsung,s2mps11.h>
@@ -147,6 +148,53 @@
};
};
+&bus_dmc {
+ devfreq-events = <&ppmu_dmc0_3>, <&ppmu_dmc1_3>;
+ vdd-supply = <&buck1_reg>;
+ status = "okay";
+};
+
+&bus_leftbus {
+ devfreq-events = <&ppmu_leftbus_3>, <&ppmu_rightbus_3>;
+ vdd-supply = <&buck3_reg>;
+ status = "okay";
+};
+
+&bus_rightbus {
+ devfreq = <&bus_leftbus>;
+ status = "okay";
+};
+
+&bus_lcd0 {
+ devfreq = <&bus_leftbus>;
+ status = "okay";
+};
+
+&bus_fsys {
+ devfreq = <&bus_leftbus>;
+ status = "okay";
+};
+
+&bus_mcuisp {
+ devfreq = <&bus_leftbus>;
+ status = "okay";
+};
+
+&bus_isp {
+ devfreq = <&bus_leftbus>;
+ status = "okay";
+};
+
+&bus_peril {
+ devfreq = <&bus_leftbus>;
+ status = "okay";
+};
+
+&bus_mfc {
+ devfreq = <&bus_leftbus>;
+ status = "okay";
+};
+
&cpu0 {
cpu0-supply = <&buck2_reg>;
};
@@ -635,46 +683,6 @@
status = "okay";
};
-&ppmu_dmc0 {
- status = "okay";
-
- events {
- ppmu_dmc0_3: ppmu-event3-dmc0 {
- event-name = "ppmu-event3-dmc0";
- };
- };
-};
-
-&ppmu_dmc1 {
- status = "okay";
-
- events {
- ppmu_dmc1_3: ppmu-event3-dmc1 {
- event-name = "ppmu-event3-dmc1";
- };
- };
-};
-
-&ppmu_leftbus {
- status = "okay";
-
- events {
- ppmu_leftbus_3: ppmu-event3-leftbus {
- event-name = "ppmu-event3-leftbus";
- };
- };
-};
-
-&ppmu_rightbus {
- status = "okay";
-
- events {
- ppmu_rightbus_3: ppmu-event3-rightbus {
- event-name = "ppmu-event3-rightbus";
- };
- };
-};
-
&xusbxti {
clock-frequency = <24000000>;
};
diff --git a/arch/arm/boot/dts/exynos3250.dtsi b/arch/arm/boot/dts/exynos3250.dtsi
index 094782b..62f3dcd 100644
--- a/arch/arm/boot/dts/exynos3250.dtsi
+++ b/arch/arm/boot/dts/exynos3250.dtsi
@@ -713,6 +713,187 @@
clock-names = "ppmu";
status = "disabled";
};
+
+ bus_dmc: bus_dmc {
+ compatible = "samsung,exynos-bus";
+ clocks = <&cmu_dmc CLK_DIV_DMC>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_dmc_opp_table>;
+ status = "disabled";
+ };
+
+ bus_dmc_opp_table: opp_table1 {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ opp@50000000 {
+ opp-hz = /bits/ 64 <50000000>;
+ opp-microvolt = <800000>;
+ };
+ opp@100000000 {
+ opp-hz = /bits/ 64 <100000000>;
+ opp-microvolt = <800000>;
+ };
+ opp@134000000 {
+ opp-hz = /bits/ 64 <134000000>;
+ opp-microvolt = <800000>;
+ };
+ opp@200000000 {
+ opp-hz = /bits/ 64 <200000000>;
+ opp-microvolt = <825000>;
+ };
+ opp@400000000 {
+ opp-hz = /bits/ 64 <400000000>;
+ opp-microvolt = <875000>;
+ };
+ };
+
+ bus_leftbus: bus_leftbus {
+ compatible = "samsung,exynos-bus";
+ clocks = <&cmu CLK_DIV_GDL>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_leftbus_opp_table>;
+ status = "disabled";
+ };
+
+ bus_rightbus: bus_rightbus {
+ compatible = "samsung,exynos-bus";
+ clocks = <&cmu CLK_DIV_GDR>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_leftbus_opp_table>;
+ status = "disabled";
+ };
+
+ bus_lcd0: bus_lcd0 {
+ compatible = "samsung,exynos-bus";
+ clocks = <&cmu CLK_DIV_ACLK_160>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_leftbus_opp_table>;
+ status = "disabled";
+ };
+
+ bus_fsys: bus_fsys {
+ compatible = "samsung,exynos-bus";
+ clocks = <&cmu CLK_DIV_ACLK_200>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_leftbus_opp_table>;
+ status = "disabled";
+ };
+
+ bus_mcuisp: bus_mcuisp {
+ compatible = "samsung,exynos-bus";
+ clocks = <&cmu CLK_DIV_ACLK_400_MCUISP>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_mcuisp_opp_table>;
+ status = "disabled";
+ };
+
+ bus_isp: bus_isp {
+ compatible = "samsung,exynos-bus";
+ clocks = <&cmu CLK_DIV_ACLK_266>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_isp_opp_table>;
+ status = "disabled";
+ };
+
+ bus_peril: bus_peril {
+ compatible = "samsung,exynos-bus";
+ clocks = <&cmu CLK_DIV_ACLK_100>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_peril_opp_table>;
+ status = "disabled";
+ };
+
+ bus_mfc: bus_mfc {
+ compatible = "samsung,exynos-bus";
+ clocks = <&cmu CLK_SCLK_MFC>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_leftbus_opp_table>;
+ status = "disabled";
+ };
+
+ bus_leftbus_opp_table: opp_table2 {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ opp@50000000 {
+ opp-hz = /bits/ 64 <50000000>;
+ opp-microvolt = <900000>;
+ };
+ opp@80000000 {
+ opp-hz = /bits/ 64 <80000000>;
+ opp-microvolt = <900000>;
+ };
+ opp@100000000 {
+ opp-hz = /bits/ 64 <100000000>;
+ opp-microvolt = <1000000>;
+ };
+ opp@134000000 {
+ opp-hz = /bits/ 64 <134000000>;
+ opp-microvolt = <1000000>;
+ };
+ opp@200000000 {
+ opp-hz = /bits/ 64 <200000000>;
+ opp-microvolt = <1000000>;
+ };
+ };
+
+ bus_mcuisp_opp_table: opp_table3 {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ opp@50000000 {
+ opp-hz = /bits/ 64 <50000000>;
+ };
+ opp@80000000 {
+ opp-hz = /bits/ 64 <80000000>;
+ };
+ opp@100000000 {
+ opp-hz = /bits/ 64 <100000000>;
+ };
+ opp@200000000 {
+ opp-hz = /bits/ 64 <200000000>;
+ };
+ opp@400000000 {
+ opp-hz = /bits/ 64 <400000000>;
+ };
+ };
+
+ bus_isp_opp_table: opp_table4 {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ opp@50000000 {
+ opp-hz = /bits/ 64 <50000000>;
+ };
+ opp@80000000 {
+ opp-hz = /bits/ 64 <80000000>;
+ };
+ opp@100000000 {
+ opp-hz = /bits/ 64 <100000000>;
+ };
+ opp@200000000 {
+ opp-hz = /bits/ 64 <200000000>;
+ };
+ opp@300000000 {
+ opp-hz = /bits/ 64 <300000000>;
+ };
+ };
+
+ bus_peril_opp_table: opp_table5 {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ opp@50000000 {
+ opp-hz = /bits/ 64 <50000000>;
+ };
+ opp@80000000 {
+ opp-hz = /bits/ 64 <80000000>;
+ };
+ opp@100000000 {
+ opp-hz = /bits/ 64 <100000000>;
+ };
+ };
};
};
diff --git a/arch/arm/boot/dts/exynos4210.dtsi b/arch/arm/boot/dts/exynos4210.dtsi
index c1cb8df..2d9b029 100644
--- a/arch/arm/boot/dts/exynos4210.dtsi
+++ b/arch/arm/boot/dts/exynos4210.dtsi
@@ -257,6 +257,165 @@
power-domains = <&pd_lcd1>;
#iommu-cells = <0>;
};
+
+ bus_dmc: bus_dmc {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DIV_DMC>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_dmc_opp_table>;
+ status = "disabled";
+ };
+
+ bus_acp: bus_acp {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DIV_ACP>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_acp_opp_table>;
+ status = "disabled";
+ };
+
+ bus_peri: bus_peri {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_ACLK100>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_peri_opp_table>;
+ status = "disabled";
+ };
+
+ bus_fsys: bus_fsys {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_ACLK133>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_fsys_opp_table>;
+ status = "disabled";
+ };
+
+ bus_display: bus_display {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_ACLK160>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_display_opp_table>;
+ status = "disabled";
+ };
+
+ bus_lcd0: bus_lcd0 {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_ACLK200>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_leftbus_opp_table>;
+ status = "disabled";
+ };
+
+ bus_leftbus: bus_leftbus {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DIV_GDL>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_leftbus_opp_table>;
+ status = "disabled";
+ };
+
+ bus_rightbus: bus_rightbus {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DIV_GDR>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_leftbus_opp_table>;
+ status = "disabled";
+ };
+
+ bus_mfc: bus_mfc {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_SCLK_MFC>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_leftbus_opp_table>;
+ status = "disabled";
+ };
+
+ bus_dmc_opp_table: opp_table1 {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ opp@134000000 {
+ opp-hz = /bits/ 64 <134000000>;
+ opp-microvolt = <1025000>;
+ };
+ opp@267000000 {
+ opp-hz = /bits/ 64 <267000000>;
+ opp-microvolt = <1050000>;
+ };
+ opp@400000000 {
+ opp-hz = /bits/ 64 <400000000>;
+ opp-microvolt = <1150000>;
+ };
+ };
+
+ bus_acp_opp_table: opp_table2 {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ opp@134000000 {
+ opp-hz = /bits/ 64 <134000000>;
+ };
+ opp@160000000 {
+ opp-hz = /bits/ 64 <160000000>;
+ };
+ opp@200000000 {
+ opp-hz = /bits/ 64 <200000000>;
+ };
+ };
+
+ bus_peri_opp_table: opp_table3 {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ opp@5000000 {
+ opp-hz = /bits/ 64 <5000000>;
+ };
+ opp@100000000 {
+ opp-hz = /bits/ 64 <100000000>;
+ };
+ };
+
+ bus_fsys_opp_table: opp_table4 {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ opp@10000000 {
+ opp-hz = /bits/ 64 <10000000>;
+ };
+ opp@134000000 {
+ opp-hz = /bits/ 64 <134000000>;
+ };
+ };
+
+ bus_display_opp_table: opp_table5 {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ opp@100000000 {
+ opp-hz = /bits/ 64 <100000000>;
+ };
+ opp@134000000 {
+ opp-hz = /bits/ 64 <134000000>;
+ };
+ opp@160000000 {
+ opp-hz = /bits/ 64 <160000000>;
+ };
+ };
+
+ bus_leftbus_opp_table: opp_table6 {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ opp@100000000 {
+ opp-hz = /bits/ 64 <100000000>;
+ };
+ opp@160000000 {
+ opp-hz = /bits/ 64 <160000000>;
+ };
+ opp@200000000 {
+ opp-hz = /bits/ 64 <200000000>;
+ };
+ };
};
&gic {
diff --git a/arch/arm/boot/dts/exynos4412-odroid-common.dtsi b/arch/arm/boot/dts/exynos4412-odroid-common.dtsi
index cab0f07..ec7619a 100644
--- a/arch/arm/boot/dts/exynos4412-odroid-common.dtsi
+++ b/arch/arm/boot/dts/exynos4412-odroid-common.dtsi
@@ -11,6 +11,7 @@
#include <dt-bindings/input/input.h>
#include <dt-bindings/clock/maxim,max77686.h>
#include "exynos4412.dtsi"
+#include "exynos4412-ppmu-common.dtsi"
#include <dt-bindings/gpio/gpio.h>
/ {
@@ -108,6 +109,53 @@
};
};
+&bus_dmc {
+ devfreq-events = <&ppmu_dmc0_3>, <&ppmu_dmc1_3>;
+ vdd-supply = <&buck1_reg>;
+ status = "okay";
+};
+
+&bus_acp {
+ devfreq = <&bus_dmc>;
+ status = "okay";
+};
+
+&bus_c2c {
+ devfreq = <&bus_dmc>;
+ status = "okay";
+};
+
+&bus_leftbus {
+ devfreq-events = <&ppmu_leftbus_3>, <&ppmu_rightbus_3>;
+ vdd-supply = <&buck3_reg>;
+ status = "okay";
+};
+
+&bus_rightbus {
+ devfreq = <&bus_leftbus>;
+ status = "okay";
+};
+
+&bus_display {
+ devfreq = <&bus_leftbus>;
+ status = "okay";
+};
+
+&bus_fsys {
+ devfreq = <&bus_leftbus>;
+ status = "okay";
+};
+
+&bus_peri {
+ devfreq = <&bus_leftbus>;
+ status = "okay";
+};
+
+&bus_mfc {
+ devfreq = <&bus_leftbus>;
+ status = "okay";
+};
+
&cpu0 {
cpu0-supply = <&buck2_reg>;
};
@@ -359,8 +407,8 @@
buck1_reg: BUCK1 {
regulator-name = "vdd_mif";
- regulator-min-microvolt = <1000000>;
- regulator-max-microvolt = <1000000>;
+ regulator-min-microvolt = <900000>;
+ regulator-max-microvolt = <1100000>;
regulator-always-on;
regulator-boot-on;
};
@@ -375,8 +423,8 @@
buck3_reg: BUCK3 {
regulator-name = "vdd_int";
- regulator-min-microvolt = <1000000>;
- regulator-max-microvolt = <1000000>;
+ regulator-min-microvolt = <900000>;
+ regulator-max-microvolt = <1050000>;
regulator-always-on;
regulator-boot-on;
};
diff --git a/arch/arm/boot/dts/exynos4412-ppmu-common.dtsi b/arch/arm/boot/dts/exynos4412-ppmu-common.dtsi
new file mode 100644
index 0000000..16e4b77
--- /dev/null
+++ b/arch/arm/boot/dts/exynos4412-ppmu-common.dtsi
@@ -0,0 +1,50 @@
+/*
+ * Device tree sources for Exynos4412 PPMU common device tree
+ *
+ * Copyright (C) 2015 Samsung Electronics
+ * Author: Chanwoo Choi <cw00.choi@samsung.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+&ppmu_dmc0 {
+ status = "okay";
+
+ events {
+ ppmu_dmc0_3: ppmu-event3-dmc0 {
+ event-name = "ppmu-event3-dmc0";
+ };
+ };
+};
+
+&ppmu_dmc1 {
+ status = "okay";
+
+ events {
+ ppmu_dmc1_3: ppmu-event3-dmc1 {
+ event-name = "ppmu-event3-dmc1";
+ };
+ };
+};
+
+&ppmu_leftbus {
+ status = "okay";
+
+ events {
+ ppmu_leftbus_3: ppmu-event3-leftbus {
+ event-name = "ppmu-event3-leftbus";
+ };
+ };
+};
+
+&ppmu_rightbus {
+ status = "okay";
+
+ events {
+ ppmu_rightbus_3: ppmu-event3-rightbus {
+ event-name = "ppmu-event3-rightbus";
+ };
+ };
+};
diff --git a/arch/arm/boot/dts/exynos4412-trats2.dts b/arch/arm/boot/dts/exynos4412-trats2.dts
index 5d1eaea..9336fd4 100644
--- a/arch/arm/boot/dts/exynos4412-trats2.dts
+++ b/arch/arm/boot/dts/exynos4412-trats2.dts
@@ -14,6 +14,7 @@
/dts-v1/;
#include "exynos4412.dtsi"
+#include "exynos4412-ppmu-common.dtsi"
#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/interrupt-controller/irq.h>
#include <dt-bindings/clock/maxim,max77686.h>
@@ -288,6 +289,53 @@
status = "okay";
};
+&bus_dmc {
+ devfreq-events = <&ppmu_dmc0_3>, <&ppmu_dmc1_3>;
+ vdd-supply = <&buck1_reg>;
+ status = "okay";
+};
+
+&bus_acp {
+ devfreq = <&bus_dmc>;
+ status = "okay";
+};
+
+&bus_c2c {
+ devfreq = <&bus_dmc>;
+ status = "okay";
+};
+
+&bus_leftbus {
+ devfreq-events = <&ppmu_leftbus_3>, <&ppmu_rightbus_3>;
+ vdd-supply = <&buck3_reg>;
+ status = "okay";
+};
+
+&bus_rightbus {
+ devfreq = <&bus_leftbus>;
+ status = "okay";
+};
+
+&bus_display {
+ devfreq = <&bus_leftbus>;
+ status = "okay";
+};
+
+&bus_fsys {
+ devfreq = <&bus_leftbus>;
+ status = "okay";
+};
+
+&bus_peri {
+ devfreq = <&bus_leftbus>;
+ status = "okay";
+};
+
+&bus_mfc {
+ devfreq = <&bus_leftbus>;
+ status = "okay";
+};
+
&cpu0 {
cpu0-supply = <&buck2_reg>;
};
@@ -871,46 +919,6 @@
assigned-clock-parents = <&clock CLK_XUSBXTI>;
};
-&ppmu_dmc0 {
- status = "okay";
-
- events {
- ppmu_dmc0_3: ppmu-event3-dmc0 {
- event-name = "ppmu-event3-dmc0";
- };
- };
-};
-
-&ppmu_dmc1 {
- status = "okay";
-
- events {
- ppmu_dmc1_3: ppmu-event3-dmc1 {
- event-name = "ppmu-event3-dmc1";
- };
- };
-};
-
-&ppmu_leftbus {
- status = "okay";
-
- events {
- ppmu_leftbus_3: ppmu-event3-leftbus {
- event-name = "ppmu-event3-leftbus";
- };
- };
-};
-
-&ppmu_rightbus {
- status = "okay";
-
- events {
- ppmu_rightbus_3: ppmu-event3-rightbus {
- event-name = "ppmu-event3-rightbus";
- };
- };
-};
-
&pinctrl_0 {
pinctrl-names = "default";
pinctrl-0 = <&sleep0>;
diff --git a/arch/arm/boot/dts/exynos4x12.dtsi b/arch/arm/boot/dts/exynos4x12.dtsi
index b7490ea0..c452499 100644
--- a/arch/arm/boot/dts/exynos4x12.dtsi
+++ b/arch/arm/boot/dts/exynos4x12.dtsi
@@ -281,6 +281,180 @@
clocks = <&clock CLK_SMMU_LITE1>, <&clock CLK_FIMC_LITE1>;
#iommu-cells = <0>;
};
+
+ bus_dmc: bus_dmc {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DIV_DMC>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_dmc_opp_table>;
+ status = "disabled";
+ };
+
+ bus_acp: bus_acp {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DIV_ACP>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_acp_opp_table>;
+ status = "disabled";
+ };
+
+ bus_c2c: bus_c2c {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DIV_C2C>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_dmc_opp_table>;
+ status = "disabled";
+ };
+
+ bus_dmc_opp_table: opp_table1 {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ opp@100000000 {
+ opp-hz = /bits/ 64 <100000000>;
+ opp-microvolt = <900000>;
+ };
+ opp@134000000 {
+ opp-hz = /bits/ 64 <134000000>;
+ opp-microvolt = <900000>;
+ };
+ opp@160000000 {
+ opp-hz = /bits/ 64 <160000000>;
+ opp-microvolt = <900000>;
+ };
+ opp@267000000 {
+ opp-hz = /bits/ 64 <267000000>;
+ opp-microvolt = <950000>;
+ };
+ opp@400000000 {
+ opp-hz = /bits/ 64 <400000000>;
+ opp-microvolt = <1050000>;
+ };
+ };
+
+ bus_acp_opp_table: opp_table2 {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ opp@100000000 {
+ opp-hz = /bits/ 64 <100000000>;
+ };
+ opp@134000000 {
+ opp-hz = /bits/ 64 <134000000>;
+ };
+ opp@160000000 {
+ opp-hz = /bits/ 64 <160000000>;
+ };
+ opp@267000000 {
+ opp-hz = /bits/ 64 <267000000>;
+ };
+ };
+
+ bus_leftbus: bus_leftbus {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DIV_GDL>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_leftbus_opp_table>;
+ status = "disabled";
+ };
+
+ bus_rightbus: bus_rightbus {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DIV_GDR>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_leftbus_opp_table>;
+ status = "disabled";
+ };
+
+ bus_display: bus_display {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_ACLK160>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_display_opp_table>;
+ status = "disabled";
+ };
+
+ bus_fsys: bus_fsys {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_ACLK133>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_fsys_opp_table>;
+ status = "disabled";
+ };
+
+ bus_peri: bus_peri {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_ACLK100>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_peri_opp_table>;
+ status = "disabled";
+ };
+
+ bus_mfc: bus_mfc {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_SCLK_MFC>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_leftbus_opp_table>;
+ status = "disabled";
+ };
+
+ bus_leftbus_opp_table: opp_table3 {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ opp@100000000 {
+ opp-hz = /bits/ 64 <100000000>;
+ opp-microvolt = <900000>;
+ };
+ opp@134000000 {
+ opp-hz = /bits/ 64 <134000000>;
+ opp-microvolt = <925000>;
+ };
+ opp@160000000 {
+ opp-hz = /bits/ 64 <160000000>;
+ opp-microvolt = <950000>;
+ };
+ opp@200000000 {
+ opp-hz = /bits/ 64 <200000000>;
+ opp-microvolt = <1000000>;
+ };
+ };
+
+ bus_display_opp_table: opp_table4 {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ opp@160000000 {
+ opp-hz = /bits/ 64 <160000000>;
+ };
+ opp@200000000 {
+ opp-hz = /bits/ 64 <200000000>;
+ };
+ };
+
+ bus_fsys_opp_table: opp_table5 {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ opp@100000000 {
+ opp-hz = /bits/ 64 <100000000>;
+ };
+ opp@134000000 {
+ opp-hz = /bits/ 64 <134000000>;
+ };
+ };
+
+ bus_peri_opp_table: opp_table6 {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ opp@50000000 {
+ opp-hz = /bits/ 64 <50000000>;
+ };
+ opp@100000000 {
+ opp-hz = /bits/ 64 <100000000>;
+ };
+ };
};
&combiner {
diff --git a/arch/arm/boot/dts/exynos5250-arndale.dts b/arch/arm/boot/dts/exynos5250-arndale.dts
index 1e25152..85dad29 100644
--- a/arch/arm/boot/dts/exynos5250-arndale.dts
+++ b/arch/arm/boot/dts/exynos5250-arndale.dts
@@ -124,8 +124,6 @@
&dp {
status = "okay";
samsung,color-space = <0>;
- samsung,dynamic-range = <0>;
- samsung,ycbcr-coeff = <0>;
samsung,color-depth = <1>;
samsung,link-rate = <0x0a>;
samsung,lane-count = <4>;
diff --git a/arch/arm/boot/dts/exynos5250-smdk5250.dts b/arch/arm/boot/dts/exynos5250-smdk5250.dts
index 0e2eb3f..b7992b1 100644
--- a/arch/arm/boot/dts/exynos5250-smdk5250.dts
+++ b/arch/arm/boot/dts/exynos5250-smdk5250.dts
@@ -80,8 +80,6 @@
&dp {
samsung,color-space = <0>;
- samsung,dynamic-range = <0>;
- samsung,ycbcr-coeff = <0>;
samsung,color-depth = <1>;
samsung,link-rate = <0x0a>;
samsung,lane-count = <4>;
diff --git a/arch/arm/boot/dts/exynos5250-snow-common.dtsi b/arch/arm/boot/dts/exynos5250-snow-common.dtsi
index c9889b1..ddfe1f5 100644
--- a/arch/arm/boot/dts/exynos5250-snow-common.dtsi
+++ b/arch/arm/boot/dts/exynos5250-snow-common.dtsi
@@ -236,12 +236,10 @@
pinctrl-names = "default";
pinctrl-0 = <&dp_hpd>;
samsung,color-space = <0>;
- samsung,dynamic-range = <0>;
- samsung,ycbcr-coeff = <0>;
samsung,color-depth = <1>;
samsung,link-rate = <0x0a>;
samsung,lane-count = <2>;
- samsung,hpd-gpio = <&gpx0 7 GPIO_ACTIVE_HIGH>;
+ hpd-gpios = <&gpx0 7 GPIO_ACTIVE_HIGH>;
ports {
port0 {
diff --git a/arch/arm/boot/dts/exynos5250-spring.dts b/arch/arm/boot/dts/exynos5250-spring.dts
index 273d662..ac291f5 100644
--- a/arch/arm/boot/dts/exynos5250-spring.dts
+++ b/arch/arm/boot/dts/exynos5250-spring.dts
@@ -74,12 +74,10 @@
pinctrl-names = "default";
pinctrl-0 = <&dp_hpd_gpio>;
samsung,color-space = <0>;
- samsung,dynamic-range = <0>;
- samsung,ycbcr-coeff = <0>;
samsung,color-depth = <1>;
samsung,link-rate = <0x0a>;
samsung,lane-count = <1>;
- samsung,hpd-gpio = <&gpc3 0 GPIO_ACTIVE_HIGH>;
+ hpd-gpios = <&gpc3 0 GPIO_ACTIVE_HIGH>;
};
&ehci {
diff --git a/arch/arm/boot/dts/exynos5420-peach-pit.dts b/arch/arm/boot/dts/exynos5420-peach-pit.dts
index 8811e17..f9d2e4f 100644
--- a/arch/arm/boot/dts/exynos5420-peach-pit.dts
+++ b/arch/arm/boot/dts/exynos5420-peach-pit.dts
@@ -157,12 +157,10 @@
pinctrl-names = "default";
pinctrl-0 = <&dp_hpd_gpio>;
samsung,color-space = <0>;
- samsung,dynamic-range = <0>;
- samsung,ycbcr-coeff = <0>;
samsung,color-depth = <1>;
samsung,link-rate = <0x06>;
samsung,lane-count = <2>;
- samsung,hpd-gpio = <&gpx2 6 GPIO_ACTIVE_HIGH>;
+ hpd-gpios = <&gpx2 6 GPIO_ACTIVE_HIGH>;
ports {
port0 {
diff --git a/arch/arm/boot/dts/exynos5420-smdk5420.dts b/arch/arm/boot/dts/exynos5420-smdk5420.dts
index 9b77940..2e748d1 100644
--- a/arch/arm/boot/dts/exynos5420-smdk5420.dts
+++ b/arch/arm/boot/dts/exynos5420-smdk5420.dts
@@ -102,8 +102,6 @@
pinctrl-names = "default";
pinctrl-0 = <&dp_hpd>;
samsung,color-space = <0>;
- samsung,dynamic-range = <0>;
- samsung,ycbcr-coeff = <0>;
samsung,color-depth = <1>;
samsung,link-rate = <0x0a>;
samsung,lane-count = <4>;
diff --git a/arch/arm/boot/dts/exynos5420.dtsi b/arch/arm/boot/dts/exynos5420.dtsi
index 4c85234..c6e05eb 100644
--- a/arch/arm/boot/dts/exynos5420.dtsi
+++ b/arch/arm/boot/dts/exynos5420.dtsi
@@ -294,6 +294,42 @@
};
};
+ nocp_mem0_0: nocp@10CA1000 {
+ compatible = "samsung,exynos5420-nocp";
+ reg = <0x10CA1000 0x200>;
+ status = "disabled";
+ };
+
+ nocp_mem0_1: nocp@10CA1400 {
+ compatible = "samsung,exynos5420-nocp";
+ reg = <0x10CA1400 0x200>;
+ status = "disabled";
+ };
+
+ nocp_mem1_0: nocp@10CA1800 {
+ compatible = "samsung,exynos5420-nocp";
+ reg = <0x10CA1800 0x200>;
+ status = "disabled";
+ };
+
+ nocp_mem1_1: nocp@10CA1C00 {
+ compatible = "samsung,exynos5420-nocp";
+ reg = <0x10CA1C00 0x200>;
+ status = "disabled";
+ };
+
+ nocp_g3d_0: nocp@11A51000 {
+ compatible = "samsung,exynos5420-nocp";
+ reg = <0x11A51000 0x200>;
+ status = "disabled";
+ };
+
+ nocp_g3d_1: nocp@11A51400 {
+ compatible = "samsung,exynos5420-nocp";
+ reg = <0x11A51400 0x200>;
+ status = "disabled";
+ };
+
gsc_pd: power-domain@10044000 {
compatible = "samsung,exynos4210-pd";
reg = <0x10044000 0x20>;
@@ -1188,6 +1224,377 @@
power-domains = <&disp_pd>;
#iommu-cells = <0>;
};
+
+ bus_wcore: bus_wcore {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DOUT_ACLK400_WCORE>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_wcore_opp_table>;
+ status = "disabled";
+ };
+
+ bus_noc: bus_noc {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DOUT_ACLK100_NOC>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_noc_opp_table>;
+ status = "disabled";
+ };
+
+ bus_fsys_apb: bus_fsys_apb {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DOUT_PCLK200_FSYS>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_fsys_apb_opp_table>;
+ status = "disabled";
+ };
+
+ bus_fsys: bus_fsys {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DOUT_ACLK200_FSYS>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_fsys_apb_opp_table>;
+ status = "disabled";
+ };
+
+ bus_fsys2: bus_fsys2 {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DOUT_ACLK200_FSYS2>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_fsys2_opp_table>;
+ status = "disabled";
+ };
+
+ bus_mfc: bus_mfc {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DOUT_ACLK333>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_mfc_opp_table>;
+ status = "disabled";
+ };
+
+ bus_gen: bus_gen {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DOUT_ACLK266>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_gen_opp_table>;
+ status = "disabled";
+ };
+
+ bus_peri: bus_peri {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DOUT_ACLK66>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_peri_opp_table>;
+ status = "disabled";
+ };
+
+ bus_g2d: bus_g2d {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DOUT_ACLK333_G2D>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_g2d_opp_table>;
+ status = "disabled";
+ };
+
+ bus_g2d_acp: bus_g2d_acp {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DOUT_ACLK266_G2D>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_g2d_acp_opp_table>;
+ status = "disabled";
+ };
+
+ bus_jpeg: bus_jpeg {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DOUT_ACLK300_JPEG>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_jpeg_opp_table>;
+ status = "disabled";
+ };
+
+ bus_jpeg_apb: bus_jpeg_apb {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DOUT_ACLK166>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_jpeg_apb_opp_table>;
+ status = "disabled";
+ };
+
+ bus_disp1_fimd: bus_disp1_fimd {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DOUT_ACLK300_DISP1>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_disp1_fimd_opp_table>;
+ status = "disabled";
+ };
+
+ bus_disp1: bus_disp1 {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DOUT_ACLK400_DISP1>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_disp1_opp_table>;
+ status = "disabled";
+ };
+
+ bus_gscl_scaler: bus_gscl_scaler {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DOUT_ACLK300_GSCL>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_gscl_opp_table>;
+ status = "disabled";
+ };
+
+ bus_mscl: bus_mscl {
+ compatible = "samsung,exynos-bus";
+ clocks = <&clock CLK_DOUT_ACLK400_MSCL>;
+ clock-names = "bus";
+ operating-points-v2 = <&bus_mscl_opp_table>;
+ status = "disabled";
+ };
+
+ bus_wcore_opp_table: opp_table2 {
+ compatible = "operating-points-v2";
+
+ opp00 {
+ opp-hz = /bits/ 64 <84000000>;
+ opp-microvolt = <925000>;
+ };
+ opp01 {
+ opp-hz = /bits/ 64 <111000000>;
+ opp-microvolt = <950000>;
+ };
+ opp02 {
+ opp-hz = /bits/ 64 <222000000>;
+ opp-microvolt = <950000>;
+ };
+ opp03 {
+ opp-hz = /bits/ 64 <333000000>;
+ opp-microvolt = <950000>;
+ };
+ opp04 {
+ opp-hz = /bits/ 64 <400000000>;
+ opp-microvolt = <987500>;
+ };
+ };
+
+ bus_noc_opp_table: opp_table3 {
+ compatible = "operating-points-v2";
+
+ opp00 {
+ opp-hz = /bits/ 64 <67000000>;
+ };
+ opp01 {
+ opp-hz = /bits/ 64 <75000000>;
+ };
+ opp02 {
+ opp-hz = /bits/ 64 <86000000>;
+ };
+ opp03 {
+ opp-hz = /bits/ 64 <100000000>;
+ };
+ };
+
+ bus_fsys_apb_opp_table: opp_table4 {
+ compatible = "operating-points-v2";
+ opp-shared;
+
+ opp00 {
+ opp-hz = /bits/ 64 <100000000>;
+ };
+ opp01 {
+ opp-hz = /bits/ 64 <200000000>;
+ };
+ };
+
+ bus_fsys2_opp_table: opp_table5 {
+ compatible = "operating-points-v2";
+
+ opp00 {
+ opp-hz = /bits/ 64 <75000000>;
+ };
+ opp01 {
+ opp-hz = /bits/ 64 <100000000>;
+ };
+ opp02 {
+ opp-hz = /bits/ 64 <150000000>;
+ };
+ };
+
+ bus_mfc_opp_table: opp_table6 {
+ compatible = "operating-points-v2";
+
+ opp00 {
+ opp-hz = /bits/ 64 <96000000>;
+ };
+ opp01 {
+ opp-hz = /bits/ 64 <111000000>;
+ };
+ opp02 {
+ opp-hz = /bits/ 64 <167000000>;
+ };
+ opp03 {
+ opp-hz = /bits/ 64 <222000000>;
+ };
+ opp04 {
+ opp-hz = /bits/ 64 <333000000>;
+ };
+ };
+
+ bus_gen_opp_table: opp_table7 {
+ compatible = "operating-points-v2";
+
+ opp00 {
+ opp-hz = /bits/ 64 <89000000>;
+ };
+ opp01 {
+ opp-hz = /bits/ 64 <133000000>;
+ };
+ opp02 {
+ opp-hz = /bits/ 64 <178000000>;
+ };
+ opp03 {
+ opp-hz = /bits/ 64 <267000000>;
+ };
+ };
+
+ bus_peri_opp_table: opp_table8 {
+ compatible = "operating-points-v2";
+
+ opp00 {
+ opp-hz = /bits/ 64 <67000000>;
+ };
+ };
+
+ bus_g2d_opp_table: opp_table9 {
+ compatible = "operating-points-v2";
+
+ opp00 {
+ opp-hz = /bits/ 64 <84000000>;
+ };
+ opp01 {
+ opp-hz = /bits/ 64 <167000000>;
+ };
+ opp02 {
+ opp-hz = /bits/ 64 <222000000>;
+ };
+ opp03 {
+ opp-hz = /bits/ 64 <300000000>;
+ };
+ opp04 {
+ opp-hz = /bits/ 64 <333000000>;
+ };
+ };
+
+ bus_g2d_acp_opp_table: opp_table10 {
+ compatible = "operating-points-v2";
+
+ opp00 {
+ opp-hz = /bits/ 64 <67000000>;
+ };
+ opp01 {
+ opp-hz = /bits/ 64 <133000000>;
+ };
+ opp02 {
+ opp-hz = /bits/ 64 <178000000>;
+ };
+ opp03 {
+ opp-hz = /bits/ 64 <267000000>;
+ };
+ };
+
+ bus_jpeg_opp_table: opp_table11 {
+ compatible = "operating-points-v2";
+
+ opp00 {
+ opp-hz = /bits/ 64 <75000000>;
+ };
+ opp01 {
+ opp-hz = /bits/ 64 <150000000>;
+ };
+ opp02 {
+ opp-hz = /bits/ 64 <200000000>;
+ };
+ opp03 {
+ opp-hz = /bits/ 64 <300000000>;
+ };
+ };
+
+ bus_jpeg_apb_opp_table: opp_table12 {
+ compatible = "operating-points-v2";
+
+ opp00 {
+ opp-hz = /bits/ 64 <84000000>;
+ };
+ opp01 {
+ opp-hz = /bits/ 64 <111000000>;
+ };
+ opp02 {
+ opp-hz = /bits/ 64 <134000000>;
+ };
+ opp03 {
+ opp-hz = /bits/ 64 <167000000>;
+ };
+ };
+
+ bus_disp1_fimd_opp_table: opp_table13 {
+ compatible = "operating-points-v2";
+
+ opp00 {
+ opp-hz = /bits/ 64 <120000000>;
+ };
+ opp01 {
+ opp-hz = /bits/ 64 <200000000>;
+ };
+ };
+
+ bus_disp1_opp_table: opp_table14 {
+ compatible = "operating-points-v2";
+
+ opp00 {
+ opp-hz = /bits/ 64 <120000000>;
+ };
+ opp01 {
+ opp-hz = /bits/ 64 <200000000>;
+ };
+ opp02 {
+ opp-hz = /bits/ 64 <300000000>;
+ };
+ };
+
+ bus_gscl_opp_table: opp_table15 {
+ compatible = "operating-points-v2";
+
+ opp00 {
+ opp-hz = /bits/ 64 <150000000>;
+ };
+ opp01 {
+ opp-hz = /bits/ 64 <200000000>;
+ };
+ opp02 {
+ opp-hz = /bits/ 64 <300000000>;
+ };
+ };
+
+ bus_mscl_opp_table: opp_table16 {
+ compatible = "operating-points-v2";
+
+ opp00 {
+ opp-hz = /bits/ 64 <84000000>;
+ };
+ opp01 {
+ opp-hz = /bits/ 64 <167000000>;
+ };
+ opp02 {
+ opp-hz = /bits/ 64 <222000000>;
+ };
+ opp03 {
+ opp-hz = /bits/ 64 <333000000>;
+ };
+ opp04 {
+ opp-hz = /bits/ 64 <400000000>;
+ };
+ };
};
&dp {
diff --git a/arch/arm/boot/dts/exynos5422-odroidxu3-common.dtsi b/arch/arm/boot/dts/exynos5422-odroidxu3-common.dtsi
index 20fa761..2a4e10b 100644
--- a/arch/arm/boot/dts/exynos5422-odroidxu3-common.dtsi
+++ b/arch/arm/boot/dts/exynos5422-odroidxu3-common.dtsi
@@ -56,6 +56,89 @@
};
};
+&bus_wcore {
+ devfreq-events = <&nocp_mem0_0>, <&nocp_mem0_1>,
+ <&nocp_mem1_0>, <&nocp_mem1_1>;
+ vdd-supply = <&buck3_reg>;
+ exynos,saturation-ratio = <100>;
+ status = "okay";
+};
+
+&bus_noc {
+ devfreq = <&bus_wcore>;
+ status = "okay";
+};
+
+&bus_fsys_apb {
+ devfreq = <&bus_wcore>;
+ status = "okay";
+};
+
+&bus_fsys {
+ devfreq = <&bus_wcore>;
+ status = "okay";
+};
+
+&bus_fsys2 {
+ devfreq = <&bus_wcore>;
+ status = "okay";
+};
+
+&bus_mfc {
+ devfreq = <&bus_wcore>;
+ status = "okay";
+};
+
+&bus_gen {
+ devfreq = <&bus_wcore>;
+ status = "okay";
+};
+
+&bus_peri {
+ devfreq = <&bus_wcore>;
+ status = "okay";
+};
+
+&bus_g2d {
+ devfreq = <&bus_wcore>;
+ status = "okay";
+};
+
+&bus_g2d_acp {
+ devfreq = <&bus_wcore>;
+ status = "okay";
+};
+
+&bus_jpeg {
+ devfreq = <&bus_wcore>;
+ status = "okay";
+};
+
+&bus_jpeg_apb {
+ devfreq = <&bus_wcore>;
+ status = "okay";
+};
+
+&bus_disp1_fimd {
+ devfreq = <&bus_wcore>;
+ status = "okay";
+};
+
+&bus_disp1 {
+ devfreq = <&bus_wcore>;
+ status = "okay";
+};
+
+&bus_gscl_scaler {
+ devfreq = <&bus_wcore>;
+ status = "okay";
+};
+
+&bus_mscl {
+ devfreq = <&bus_wcore>;
+ status = "okay";
+};
+
&clock_audss {
assigned-clocks = <&clock_audss EXYNOS_MOUT_AUDSS>,
<&clock_audss EXYNOS_MOUT_I2S>,
@@ -361,6 +444,22 @@
vqmmc-supply = <&ldo13_reg>;
};
+&nocp_mem0_0 {
+ status = "okay";
+};
+
+&nocp_mem0_1 {
+ status = "okay";
+};
+
+&nocp_mem1_0 {
+ status = "okay";
+};
+
+&nocp_mem1_1 {
+ status = "okay";
+};
+
&pinctrl_0 {
hdmi_hpd_irq: hdmi-hpd-irq {
samsung,pins = "gpx3-7";
diff --git a/arch/arm/boot/dts/exynos5800-peach-pi.dts b/arch/arm/boot/dts/exynos5800-peach-pi.dts
index f959925..62ceb89 100644
--- a/arch/arm/boot/dts/exynos5800-peach-pi.dts
+++ b/arch/arm/boot/dts/exynos5800-peach-pi.dts
@@ -157,8 +157,6 @@
pinctrl-names = "default";
pinctrl-0 = <&dp_hpd_gpio>;
samsung,color-space = <0>;
- samsung,dynamic-range = <0>;
- samsung,ycbcr-coeff = <0>;
samsung,color-depth = <1>;
samsung,link-rate = <0x0a>;
samsung,lane-count = <2>;
diff --git a/arch/arm/boot/dts/imx6q-apalis-ixora.dts b/arch/arm/boot/dts/imx6q-apalis-ixora.dts
index 8e3e146..8e67ca2 100644
--- a/arch/arm/boot/dts/imx6q-apalis-ixora.dts
+++ b/arch/arm/boot/dts/imx6q-apalis-ixora.dts
@@ -219,8 +219,9 @@
};
&pcie {
- /* active-low meaning opposite of regular PERST# active-low polarity */
- reset-gpio = <&gpio1 28 GPIO_ACTIVE_LOW>;
+ /* active-high meaning opposite of regular PERST# active-low polarity */
+ reset-gpio = <&gpio1 28 GPIO_ACTIVE_HIGH>;
+ reset-gpio-active-high;
status = "okay";
};
diff --git a/arch/arm/boot/dts/imx6qp.dtsi b/arch/arm/boot/dts/imx6qp.dtsi
index 1ada714..886dbf2 100644
--- a/arch/arm/boot/dts/imx6qp.dtsi
+++ b/arch/arm/boot/dts/imx6qp.dtsi
@@ -82,5 +82,8 @@
"ldb_di0", "ldb_di1", "prg";
};
+ pcie: pcie@0x01000000 {
+ compatible = "fsl,imx6qp-pcie", "snps,dw-pcie";
+ };
};
};
diff --git a/arch/arm/boot/dts/imx7d-nitrogen7.dts b/arch/arm/boot/dts/imx7d-nitrogen7.dts
new file mode 100644
index 0000000..1ce9780
--- /dev/null
+++ b/arch/arm/boot/dts/imx7d-nitrogen7.dts
@@ -0,0 +1,745 @@
+/*
+ * Copyright 2016 Boundary Devices, Inc.
+ *
+ * This file is dual-licensed: you can use it either under the terms
+ * of the GPL or the X11 license, at your option. Note that this dual
+ * licensing only applies to this file, and not this project as a
+ * whole.
+ *
+ * a) This file is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of the
+ * License, or (at your option) any later version.
+ *
+ * This file is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * Or, alternatively,
+ *
+ * b) Permission is hereby granted, free of charge, to any person
+ * obtaining a copy of this software and associated documentation
+ * files (the "Software"), to deal in the Software without
+ * restriction, including without limitation the rights to use,
+ * copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following
+ * conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
+ * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+ * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+ * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+
+/dts-v1/;
+
+#include <dt-bindings/input/input.h>
+#include "imx7d.dtsi"
+
+/ {
+ model = "Boundary Devices i.MX7 Nitrogen7 Board";
+ compatible = "boundary,imx7d-nitrogen7", "fsl,imx7d";
+
+ aliases {
+ fb_lcd = &lcdif;
+ t_lcd = &t_lcd;
+ };
+
+ memory {
+ reg = <0x80000000 0x40000000>;
+ };
+
+ backlight-j9 {
+ compatible = "gpio-backlight";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_backlight_j9>;
+ gpios = <&gpio1 7 GPIO_ACTIVE_HIGH>;
+ default-on;
+ };
+
+ backlight-j20 {
+ compatible = "pwm-backlight";
+ pwms = <&pwm1 0 5000000>;
+ brightness-levels = <0 4 8 16 32 64 128 255>;
+ default-brightness-level = <6>;
+ status = "okay";
+ };
+
+ reg_usb_otg1_vbus: regulator-usb-otg1-vbus {
+ compatible = "regulator-fixed";
+ regulator-name = "usb_otg1_vbus";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ gpio = <&gpio1 5 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ };
+
+ reg_usb_otg2_vbus: regulator-usb-otg2-vbus {
+ compatible = "regulator-fixed";
+ regulator-name = "usb_otg2_vbus";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ gpio = <&gpio4 7 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ };
+
+ reg_can2_3v3: regulator-can2-3v3 {
+ compatible = "regulator-fixed";
+ regulator-name = "can2-3v3";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ gpio = <&gpio2 14 GPIO_ACTIVE_LOW>;
+ };
+
+ reg_vref_1v8: regulator-vref-1v8 {
+ compatible = "regulator-fixed";
+ regulator-name = "vref-1v8";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ };
+
+ reg_vref_3v3: regulator-vref-3v3 {
+ compatible = "regulator-fixed";
+ regulator-name = "vref-3v3";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ };
+
+ reg_wlan: regulator-wlan {
+ compatible = "regulator-fixed";
+ regulator-min-microvolt = <3300000>;
+ regulator-max-microvolt = <3300000>;
+ clocks = <&clks IMX7D_CLKO2_ROOT_DIV>;
+ clock-names = "slow";
+ regulator-name = "reg_wlan";
+ startup-delay-us = <70000>;
+ gpio = <&gpio4 21 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ };
+};
+
+&adc1 {
+ vref-supply = <®_vref_1v8>;
+ status = "okay";
+};
+
+&adc2 {
+ vref-supply = <®_vref_1v8>;
+ status = "okay";
+};
+
+&clks {
+ assigned-clocks = <&clks IMX7D_CLKO2_ROOT_SRC>,
+ <&clks IMX7D_CLKO2_ROOT_DIV>;
+ assigned-clock-parents = <&clks IMX7D_CKIL>;
+ assigned-clock-rates = <0>, <32768>;
+};
+
+&cpu0 {
+ arm-supply = <&sw1a_reg>;
+};
+
+&fec1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_enet1>;
+ assigned-clocks = <&clks IMX7D_ENET1_TIME_ROOT_SRC>,
+ <&clks IMX7D_ENET1_TIME_ROOT_CLK>;
+ assigned-clock-parents = <&clks IMX7D_PLL_ENET_MAIN_100M_CLK>;
+ assigned-clock-rates = <0>, <100000000>;
+ phy-mode = "rgmii";
+ phy-handle = <ðphy0>;
+ fsl,magic-packet;
+ status = "okay";
+
+ mdio {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ ethphy0: ethernet-phy@4 {
+ reg = <4>;
+ };
+ };
+};
+
+&flexcan2 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_flexcan2>;
+ xceiver-supply = <®_can2_3v3>;
+ status = "okay";
+};
+
+&i2c1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_i2c1>;
+ status = "okay";
+
+ pmic: pfuze3000@08 {
+ compatible = "fsl,pfuze3000";
+ reg = <0x08>;
+
+ regulators {
+ sw1a_reg: sw1a {
+ regulator-min-microvolt = <700000>;
+ regulator-max-microvolt = <1475000>;
+ regulator-boot-on;
+ regulator-always-on;
+ regulator-ramp-delay = <6250>;
+ };
+
+ /* use sw1c_reg to align with pfuze100/pfuze200 */
+ sw1c_reg: sw1b {
+ regulator-min-microvolt = <700000>;
+ regulator-max-microvolt = <1475000>;
+ regulator-boot-on;
+ regulator-always-on;
+ regulator-ramp-delay = <6250>;
+ };
+
+ sw2_reg: sw2 {
+ regulator-min-microvolt = <1500000>;
+ regulator-max-microvolt = <1850000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ sw3a_reg: sw3 {
+ regulator-min-microvolt = <900000>;
+ regulator-max-microvolt = <1650000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ swbst_reg: swbst {
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5150000>;
+ };
+
+ snvs_reg: vsnvs {
+ regulator-min-microvolt = <1000000>;
+ regulator-max-microvolt = <3000000>;
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ vref_reg: vrefddr {
+ regulator-boot-on;
+ regulator-always-on;
+ };
+
+ vgen1_reg: vldo1 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ };
+
+ vgen2_reg: vldo2 {
+ regulator-min-microvolt = <800000>;
+ regulator-max-microvolt = <1550000>;
+ regulator-always-on;
+ };
+
+ vgen3_reg: vccsd {
+ regulator-min-microvolt = <2850000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ };
+
+ vgen4_reg: v33 {
+ regulator-min-microvolt = <2850000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ };
+
+ vgen5_reg: vldo3 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ };
+
+ vgen6_reg: vldo4 {
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <3300000>;
+ regulator-always-on;
+ };
+ };
+ };
+};
+
+&i2c2 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_i2c2>;
+ status = "okay";
+
+ rtc@68 {
+ compatible = "rv4162";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_i2c2_rv4162>;
+ reg = <0x68>;
+ interrupts-extended = <&gpio2 15 IRQ_TYPE_LEVEL_LOW>;
+ };
+};
+
+&i2c3 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_i2c3>;
+ status = "okay";
+
+ touch@48 {
+ compatible = "ti,tsc2004";
+ reg = <0x48>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_i2c3_tsc2004>;
+ interrupts-extended = <&gpio3 4 IRQ_TYPE_EDGE_FALLING>;
+ wakeup-gpios = <&gpio3 4 GPIO_ACTIVE_LOW>;
+ };
+};
+
+&i2c4 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_i2c4>;
+ status = "okay";
+
+ codec: wm8960@1a {
+ compatible = "wlf,wm8960";
+ reg = <0x1a>;
+ clocks = <&clks IMX7D_AUDIO_MCLK_ROOT_CLK>;
+ clock-names = "mclk";
+ wlf,shared-lrclk;
+ };
+};
+
+&lcdif {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_lcdif_dat
+ &pinctrl_lcdif_ctrl>;
+ lcd-supply = <®_vref_3v3>;
+ display = <&display0>;
+ status = "okay";
+
+ display0: lcd-display {
+ bits-per-pixel = <16>;
+ bus-width = <18>;
+
+ display-timings {
+ native-mode = <&t_lcd>;
+ t_lcd: t_lcd_default {
+ /* default to Okaya display */
+ clock-frequency = <30000000>;
+ hactive = <800>;
+ vactive = <480>;
+ hfront-porch = <40>;
+ hback-porch = <40>;
+ hsync-len = <48>;
+ vback-porch = <29>;
+ vfront-porch = <13>;
+ vsync-len = <3>;
+ hsync-active = <0>;
+ vsync-active = <0>;
+ de-active = <1>;
+ pixelclk-active = <0>;
+ };
+ };
+ };
+};
+
+&pwm1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_pwm1>;
+ status = "okay";
+};
+
+&pwm2 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_pwm2>;
+ status = "okay";
+};
+
+&uart1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_uart1>;
+ assigned-clocks = <&clks IMX7D_UART1_ROOT_SRC>;
+ assigned-clock-parents = <&clks IMX7D_OSC_24M_CLK>;
+ status = "okay";
+};
+
+&uart2 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_uart2>;
+ assigned-clocks = <&clks IMX7D_UART2_ROOT_SRC>;
+ assigned-clock-parents = <&clks IMX7D_OSC_24M_CLK>;
+ status = "okay";
+};
+
+&uart3 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_uart3>;
+ assigned-clocks = <&clks IMX7D_UART3_ROOT_SRC>;
+ assigned-clock-parents = <&clks IMX7D_OSC_24M_CLK>;
+ status = "okay";
+};
+
+&uart6 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_uart6>;
+ assigned-clocks = <&clks IMX7D_UART6_ROOT_SRC>;
+ assigned-clock-parents = <&clks IMX7D_PLL_SYS_MAIN_240M_CLK>;
+ fsl,uart-has-rtscts;
+ status = "okay";
+};
+
+&usbotg1 {
+ vbus-supply = <®_usb_otg1_vbus>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_usbotg1>;
+ status = "okay";
+};
+
+&usbotg2 {
+ vbus-supply = <®_usb_otg2_vbus>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_usbotg2>;
+ dr_mode = "host";
+ status = "okay";
+};
+
+&usdhc1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_usdhc1>;
+ cd-gpios = <&gpio5 0 GPIO_ACTIVE_LOW>;
+ vmmc-supply = <&vgen3_reg>;
+ bus-width = <4>;
+ fsl,tuning-step = <2>;
+ wakeup-source;
+ keep-power-in-suspend;
+ status = "okay";
+};
+
+&usdhc2 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_usdhc2>;
+ bus-width = <4>;
+ non-removable;
+ vmmc-supply = <®_wlan>;
+ cap-power-off-card;
+ keep-power-in-suspend;
+ status = "okay";
+
+ wlcore: wlcore@2 {
+ compatible = "ti,wl1271";
+ reg = <2>;
+ interrupt-parent = <&gpio4>;
+ interrupts = <20 IRQ_TYPE_LEVEL_HIGH>;
+ ref-clock-frequency = <38400000>;
+ };
+};
+
+&usdhc3 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_usdhc3>;
+ assigned-clocks = <&clks IMX7D_USDHC3_ROOT_CLK>;
+ assigned-clock-rates = <400000000>;
+ bus-width = <8>;
+ fsl,tuning-step = <2>;
+ non-removable;
+ status = "okay";
+};
+
+&wdog1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_wdog1>;
+ status = "okay";
+};
+
+&iomuxc {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_hog_1 &pinctrl_j2>;
+
+ pinctrl_hog_1: hoggrp-1 {
+ fsl,pins = <
+ MX7D_PAD_SD3_RESET_B__GPIO6_IO11 0x5d
+ MX7D_PAD_GPIO1_IO13__GPIO1_IO13 0x7d
+ MX7D_PAD_ECSPI2_MISO__GPIO4_IO22 0x7d
+ >;
+ };
+
+ pinctrl_enet1: enet1grp {
+ fsl,pins = <
+ MX7D_PAD_GPIO1_IO10__ENET1_MDIO 0x3
+ MX7D_PAD_GPIO1_IO11__ENET1_MDC 0x3
+ MX7D_PAD_GPIO1_IO12__CCM_ENET_REF_CLK1 0x3
+ MX7D_PAD_ENET1_RGMII_TXC__ENET1_RGMII_TXC 0x71
+ MX7D_PAD_ENET1_RGMII_TD0__ENET1_RGMII_TD0 0x71
+ MX7D_PAD_ENET1_RGMII_TD1__ENET1_RGMII_TD1 0x71
+ MX7D_PAD_ENET1_RGMII_TD2__ENET1_RGMII_TD2 0x71
+ MX7D_PAD_ENET1_RGMII_TD3__ENET1_RGMII_TD3 0x71
+ MX7D_PAD_ENET1_RGMII_TX_CTL__ENET1_RGMII_TX_CTL 0x71
+ MX7D_PAD_ENET1_RGMII_RXC__ENET1_RGMII_RXC 0x71
+ MX7D_PAD_ENET1_RGMII_RD0__ENET1_RGMII_RD0 0x11
+ MX7D_PAD_ENET1_RGMII_RD1__ENET1_RGMII_RD1 0x11
+ MX7D_PAD_ENET1_RGMII_RD2__ENET1_RGMII_RD2 0x11
+ MX7D_PAD_ENET1_RGMII_RD3__ENET1_RGMII_RD3 0x71
+ MX7D_PAD_ENET1_RGMII_RX_CTL__ENET1_RGMII_RX_CTL 0x11
+ MX7D_PAD_SD3_STROBE__GPIO6_IO10 0x75
+ >;
+ };
+
+ pinctrl_flexcan2: flexcan2grp {
+ fsl,pins = <
+ MX7D_PAD_GPIO1_IO14__FLEXCAN2_RX 0x7d
+ MX7D_PAD_GPIO1_IO15__FLEXCAN2_TX 0x7d
+ MX7D_PAD_EPDC_DATA14__GPIO2_IO14 0x7d
+ >;
+ };
+
+ pinctrl_i2c1: i2c1grp {
+ fsl,pins = <
+ MX7D_PAD_I2C1_SDA__I2C1_SDA 0x4000007f
+ MX7D_PAD_I2C1_SCL__I2C1_SCL 0x4000007f
+ >;
+ };
+
+ pinctrl_i2c2: i2c2grp {
+ fsl,pins = <
+ MX7D_PAD_I2C2_SDA__I2C2_SDA 0x4000007f
+ MX7D_PAD_I2C2_SCL__I2C2_SCL 0x4000007f
+ >;
+ };
+
+ pinctrl_i2c2_rv4162: i2c2-rv4162grp {
+ fsl,pins = <
+ MX7D_PAD_EPDC_DATA15__GPIO2_IO15 0x7d
+ >;
+ };
+
+ pinctrl_i2c3: i2c3grp {
+ fsl,pins = <
+ MX7D_PAD_I2C3_SDA__I2C3_SDA 0x4000007f
+ MX7D_PAD_I2C3_SCL__I2C3_SCL 0x4000007f
+ >;
+ };
+
+ pinctrl_i2c3_tsc2004: i2c3tsc2004grp {
+ fsl,pins = <
+ MX7D_PAD_LCD_RESET__GPIO3_IO4 0x79
+ MX7D_PAD_SD2_WP__GPIO5_IO10 0x7d
+ >;
+ };
+
+ pinctrl_i2c4: i2c4grp {
+ fsl,pins = <
+ MX7D_PAD_I2C4_SDA__I2C4_SDA 0x4000007f
+ MX7D_PAD_I2C4_SCL__I2C4_SCL 0x4000007f
+ >;
+ };
+
+ pinctrl_j2: j2grp {
+ fsl,pins = <
+ MX7D_PAD_SAI1_TX_DATA__GPIO6_IO15 0x7d
+ MX7D_PAD_EPDC_BDR0__GPIO2_IO28 0x7d
+ MX7D_PAD_SAI1_RX_DATA__GPIO6_IO12 0x7d
+ MX7D_PAD_EPDC_BDR1__GPIO2_IO29 0x7d
+ MX7D_PAD_SD1_WP__GPIO5_IO1 0x7d
+ MX7D_PAD_EPDC_SDSHR__GPIO2_IO19 0x7d
+ MX7D_PAD_SD1_RESET_B__GPIO5_IO2 0x7d
+ MX7D_PAD_SD2_RESET_B__GPIO5_IO11 0x7d
+ MX7D_PAD_EPDC_DATA07__GPIO2_IO7 0x7d
+ MX7D_PAD_EPDC_DATA08__GPIO2_IO8 0x7d
+ MX7D_PAD_EPDC_DATA09__GPIO2_IO9 0x7d
+ MX7D_PAD_EPDC_DATA10__GPIO2_IO10 0x7d
+ MX7D_PAD_EPDC_DATA11__GPIO2_IO11 0x7d
+ MX7D_PAD_EPDC_DATA12__GPIO2_IO12 0x7d
+ MX7D_PAD_SAI1_TX_SYNC__GPIO6_IO14 0x7d
+ MX7D_PAD_EPDC_DATA13__GPIO2_IO13 0x7d
+ MX7D_PAD_SAI1_TX_BCLK__GPIO6_IO13 0x7d
+ MX7D_PAD_SD2_CD_B__GPIO5_IO9 0x7d
+ MX7D_PAD_EPDC_GDCLK__GPIO2_IO24 0x7d
+ MX7D_PAD_SAI2_RX_DATA__GPIO6_IO21 0x7d
+ MX7D_PAD_EPDC_GDOE__GPIO2_IO25 0x7d
+ MX7D_PAD_EPDC_GDRL__GPIO2_IO26 0x7d
+ MX7D_PAD_SAI2_TX_DATA__GPIO6_IO22 0x7d
+ MX7D_PAD_EPDC_SDCE0__GPIO2_IO20 0x7d
+ MX7D_PAD_SAI2_TX_BCLK__GPIO6_IO20 0x7d
+ MX7D_PAD_EPDC_SDCE1__GPIO2_IO21 0x7d
+ MX7D_PAD_SAI2_TX_SYNC__GPIO6_IO19 0x7d
+ MX7D_PAD_EPDC_SDCE2__GPIO2_IO22 0x7d
+ MX7D_PAD_EPDC_SDCE3__GPIO2_IO23 0x7d
+ MX7D_PAD_EPDC_GDSP__GPIO2_IO27 0x7d
+ MX7D_PAD_EPDC_SDCLK__GPIO2_IO16 0x7d
+ MX7D_PAD_EPDC_SDLE__GPIO2_IO17 0x7d
+ MX7D_PAD_EPDC_SDOE__GPIO2_IO18 0x7d
+ MX7D_PAD_EPDC_PWR_COM__GPIO2_IO30 0x7d
+ MX7D_PAD_EPDC_PWR_STAT__GPIO2_IO31 0x7d
+ >;
+ };
+
+ pinctrl_lcdif_dat: lcdifdatgrp {
+ fsl,pins = <
+ MX7D_PAD_LCD_DATA00__LCD_DATA0 0x79
+ MX7D_PAD_LCD_DATA01__LCD_DATA1 0x79
+ MX7D_PAD_LCD_DATA02__LCD_DATA2 0x79
+ MX7D_PAD_LCD_DATA03__LCD_DATA3 0x79
+ MX7D_PAD_LCD_DATA04__LCD_DATA4 0x79
+ MX7D_PAD_LCD_DATA05__LCD_DATA5 0x79
+ MX7D_PAD_LCD_DATA06__LCD_DATA6 0x79
+ MX7D_PAD_LCD_DATA07__LCD_DATA7 0x79
+ MX7D_PAD_LCD_DATA08__LCD_DATA8 0x79
+ MX7D_PAD_LCD_DATA09__LCD_DATA9 0x79
+ MX7D_PAD_LCD_DATA10__LCD_DATA10 0x79
+ MX7D_PAD_LCD_DATA11__LCD_DATA11 0x79
+ MX7D_PAD_LCD_DATA12__LCD_DATA12 0x79
+ MX7D_PAD_LCD_DATA13__LCD_DATA13 0x79
+ MX7D_PAD_LCD_DATA14__LCD_DATA14 0x79
+ MX7D_PAD_LCD_DATA15__LCD_DATA15 0x79
+ MX7D_PAD_LCD_DATA16__LCD_DATA16 0x79
+ MX7D_PAD_LCD_DATA17__LCD_DATA17 0x79
+ MX7D_PAD_LCD_DATA18__LCD_DATA18 0x79
+ MX7D_PAD_LCD_DATA19__LCD_DATA19 0x79
+ MX7D_PAD_LCD_DATA20__LCD_DATA20 0x79
+ MX7D_PAD_LCD_DATA21__LCD_DATA21 0x79
+ MX7D_PAD_LCD_DATA22__LCD_DATA22 0x79
+ MX7D_PAD_LCD_DATA23__LCD_DATA23 0x79
+ >;
+ };
+
+ pinctrl_lcdif_ctrl: lcdifctrlgrp {
+ fsl,pins = <
+ MX7D_PAD_LCD_CLK__LCD_CLK 0x79
+ MX7D_PAD_LCD_ENABLE__LCD_ENABLE 0x79
+ MX7D_PAD_LCD_VSYNC__LCD_VSYNC 0x79
+ MX7D_PAD_LCD_HSYNC__LCD_HSYNC 0x79
+ >;
+ };
+
+ pinctrl_pwm2: pwm2grp {
+ fsl,pins = <
+ MX7D_PAD_GPIO1_IO09__PWM2_OUT 0x7d
+ >;
+ };
+
+ pinctrl_uart1: uart1grp {
+ fsl,pins = <
+ MX7D_PAD_UART1_TX_DATA__UART1_DCE_TX 0x79
+ MX7D_PAD_UART1_RX_DATA__UART1_DCE_RX 0x79
+ >;
+ };
+
+ pinctrl_uart2: uart2grp {
+ fsl,pins = <
+ MX7D_PAD_UART2_TX_DATA__UART2_DCE_TX 0x79
+ MX7D_PAD_UART2_RX_DATA__UART2_DCE_RX 0x79
+ >;
+ };
+
+ pinctrl_uart3: uart3grp {
+ fsl,pins = <
+ MX7D_PAD_UART3_TX_DATA__UART3_DCE_TX 0x79
+ MX7D_PAD_UART3_RX_DATA__UART3_DCE_RX 0x79
+ MX7D_PAD_EPDC_DATA04__GPIO2_IO4 0x7d
+ >;
+ };
+
+ pinctrl_uart6: uart6grp {
+ fsl,pins = <
+ MX7D_PAD_ECSPI1_MOSI__UART6_DCE_TX 0x79
+ MX7D_PAD_ECSPI1_SCLK__UART6_DCE_RX 0x79
+ MX7D_PAD_ECSPI1_SS0__UART6_DCE_CTS 0x79
+ MX7D_PAD_ECSPI1_MISO__UART6_DCE_RTS 0x79
+ >;
+ };
+
+ pinctrl_usbotg2: usbotg2grp {
+ fsl,pins = <
+ MX7D_PAD_UART3_RTS_B__USB_OTG2_OC 0x7d
+ MX7D_PAD_UART3_CTS_B__GPIO4_IO7 0x14
+ >;
+ };
+
+ pinctrl_usdhc1: usdhc1grp {
+ fsl,pins = <
+ MX7D_PAD_SD1_CMD__SD1_CMD 0x59
+ MX7D_PAD_SD1_CLK__SD1_CLK 0x19
+ MX7D_PAD_SD1_DATA0__SD1_DATA0 0x59
+ MX7D_PAD_SD1_DATA1__SD1_DATA1 0x59
+ MX7D_PAD_SD1_DATA2__SD1_DATA2 0x59
+ MX7D_PAD_SD1_DATA3__SD1_DATA3 0x59
+ MX7D_PAD_GPIO1_IO08__SD1_VSELECT 0x75
+ MX7D_PAD_SD1_CD_B__GPIO5_IO0 0x75
+ >;
+ };
+
+ pinctrl_usdhc2: usdhc2grp {
+ fsl,pins = <
+ MX7D_PAD_SD2_CMD__SD2_CMD 0x59
+ MX7D_PAD_SD2_CLK__SD2_CLK 0x19
+ MX7D_PAD_SD2_DATA0__SD2_DATA0 0x59
+ MX7D_PAD_SD2_DATA1__SD2_DATA1 0x59
+ MX7D_PAD_SD2_DATA2__SD2_DATA2 0x59
+ MX7D_PAD_SD2_DATA3__SD2_DATA3 0x59
+ MX7D_PAD_ECSPI2_SCLK__GPIO4_IO20 0x59
+ MX7D_PAD_ECSPI2_MOSI__GPIO4_IO21 0x59
+ >;
+ };
+
+ pinctrl_usdhc3: usdhc3grp {
+ fsl,pins = <
+ MX7D_PAD_SD3_CMD__SD3_CMD 0x59
+ MX7D_PAD_SD3_CLK__SD3_CLK 0x19
+ MX7D_PAD_SD3_DATA0__SD3_DATA0 0x59
+ MX7D_PAD_SD3_DATA1__SD3_DATA1 0x59
+ MX7D_PAD_SD3_DATA2__SD3_DATA2 0x59
+ MX7D_PAD_SD3_DATA3__SD3_DATA3 0x59
+ MX7D_PAD_SD3_DATA4__SD3_DATA4 0x59
+ MX7D_PAD_SD3_DATA5__SD3_DATA5 0x59
+ MX7D_PAD_SD3_DATA6__SD3_DATA6 0x59
+ MX7D_PAD_SD3_DATA7__SD3_DATA7 0x59
+ >;
+ };
+};
+
+&iomuxc_lpsr {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_hog_2>;
+
+ pinctrl_hog_2: hoggrp-2 {
+ fsl,pins = <
+ MX7D_PAD_GPIO1_IO02__GPIO1_IO2 0x7d
+ MX7D_PAD_GPIO1_IO03__CCM_CLKO2 0x7d
+ >;
+ };
+
+ pinctrl_backlight_j9: backlightj9grp {
+ fsl,pins = <
+ MX7D_PAD_GPIO1_IO07__GPIO1_IO7 0x7d
+ >;
+ };
+
+ pinctrl_pwm1: pwm1grp {
+ fsl,pins = <
+ MX7D_PAD_GPIO1_IO01__PWM1_OUT 0x7d
+ >;
+ };
+
+ pinctrl_usbotg1: usbotg1grp {
+ fsl,pins = <
+ MX7D_PAD_GPIO1_IO04__USB_OTG1_OC 0x7d
+ MX7D_PAD_GPIO1_IO05__GPIO1_IO5 0x14
+ >;
+ };
+
+ pinctrl_wdog1: wdog1grp {
+ fsl,pins = <
+ MX7D_PAD_GPIO1_IO00__WDOD1_WDOG_B 0x75
+ >;
+ };
+};
diff --git a/arch/arm/boot/dts/imx7d.dtsi b/arch/arm/boot/dts/imx7d.dtsi
index b5a50e0..6b3faa2 100644
--- a/arch/arm/boot/dts/imx7d.dtsi
+++ b/arch/arm/boot/dts/imx7d.dtsi
@@ -651,6 +651,17 @@
#pwm-cells = <2>;
status = "disabled";
};
+
+ lcdif: lcdif@30730000 {
+ compatible = "fsl,imx7d-lcdif", "fsl,imx28-lcdif";
+ reg = <0x30730000 0x10000>;
+ interrupts = <GIC_SPI 5 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&clks IMX7D_LCDIF_PIXEL_ROOT_CLK>,
+ <&clks IMX7D_CLK_DUMMY>,
+ <&clks IMX7D_CLK_DUMMY>;
+ clock-names = "pix", "axi", "disp_axi";
+ status = "disabled";
+ };
};
aips3: aips-bus@30800000 {
@@ -693,6 +704,26 @@
status = "disabled";
};
+ flexcan1: can@30a00000 {
+ compatible = "fsl,imx7d-flexcan", "fsl,imx6q-flexcan";
+ reg = <0x30a00000 0x10000>;
+ interrupts = <GIC_SPI 110 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&clks IMX7D_CLK_DUMMY>,
+ <&clks IMX7D_CAN1_ROOT_CLK>;
+ clock-names = "ipg", "per";
+ status = "disabled";
+ };
+
+ flexcan2: can@30a10000 {
+ compatible = "fsl,imx7d-flexcan", "fsl,imx6q-flexcan";
+ reg = <0x30a10000 0x10000>;
+ interrupts = <GIC_SPI 111 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&clks IMX7D_CLK_DUMMY>,
+ <&clks IMX7D_CAN2_ROOT_CLK>;
+ clock-names = "ipg", "per";
+ status = "disabled";
+ };
+
i2c1: i2c@30a20000 {
#address-cells = <1>;
#size-cells = <0>;
diff --git a/arch/arm/boot/dts/r8a7779.dtsi b/arch/arm/boot/dts/r8a7779.dtsi
index 0c82097..b9bbcce 100644
--- a/arch/arm/boot/dts/r8a7779.dtsi
+++ b/arch/arm/boot/dts/r8a7779.dtsi
@@ -14,6 +14,7 @@
#include <dt-bindings/clock/r8a7779-clock.h>
#include <dt-bindings/interrupt-controller/arm-gic.h>
#include <dt-bindings/interrupt-controller/irq.h>
+#include <dt-bindings/power/r8a7779-sysc.h>
/ {
compatible = "renesas,r8a7779";
@@ -34,18 +35,21 @@
compatible = "arm,cortex-a9";
reg = <1>;
clock-frequency = <1000000000>;
+ power-domains = <&sysc R8A7779_PD_ARM1>;
};
cpu@2 {
device_type = "cpu";
compatible = "arm,cortex-a9";
reg = <2>;
clock-frequency = <1000000000>;
+ power-domains = <&sysc R8A7779_PD_ARM2>;
};
cpu@3 {
device_type = "cpu";
compatible = "arm,cortex-a9";
reg = <3>;
clock-frequency = <1000000000>;
+ power-domains = <&sysc R8A7779_PD_ARM3>;
};
};
@@ -173,7 +177,7 @@
reg = <0xffc70000 0x1000>;
interrupts = <GIC_SPI 79 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp0_clks R8A7779_CLK_I2C0>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7779_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -184,7 +188,7 @@
reg = <0xffc71000 0x1000>;
interrupts = <GIC_SPI 82 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp0_clks R8A7779_CLK_I2C1>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7779_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -195,7 +199,7 @@
reg = <0xffc72000 0x1000>;
interrupts = <GIC_SPI 80 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp0_clks R8A7779_CLK_I2C2>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7779_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -206,7 +210,7 @@
reg = <0xffc73000 0x1000>;
interrupts = <GIC_SPI 81 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp0_clks R8A7779_CLK_I2C3>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7779_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -218,7 +222,7 @@
clocks = <&mstp0_clks R8A7779_CLK_SCIF0>,
<&cpg_clocks R8A7779_CLK_S1>, <&scif_clk>;
clock-names = "fck", "brg_int", "scif_clk";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7779_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -230,7 +234,7 @@
clocks = <&mstp0_clks R8A7779_CLK_SCIF1>,
<&cpg_clocks R8A7779_CLK_S1>, <&scif_clk>;
clock-names = "fck", "brg_int", "scif_clk";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7779_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -242,7 +246,7 @@
clocks = <&mstp0_clks R8A7779_CLK_SCIF2>,
<&cpg_clocks R8A7779_CLK_S1>, <&scif_clk>;
clock-names = "fck", "brg_int", "scif_clk";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7779_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -254,7 +258,7 @@
clocks = <&mstp0_clks R8A7779_CLK_SCIF3>,
<&cpg_clocks R8A7779_CLK_S1>, <&scif_clk>;
clock-names = "fck", "brg_int", "scif_clk";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7779_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -266,7 +270,7 @@
clocks = <&mstp0_clks R8A7779_CLK_SCIF4>,
<&cpg_clocks R8A7779_CLK_S1>, <&scif_clk>;
clock-names = "fck", "brg_int", "scif_clk";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7779_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -278,7 +282,7 @@
clocks = <&mstp0_clks R8A7779_CLK_SCIF5>,
<&cpg_clocks R8A7779_CLK_S1>, <&scif_clk>;
clock-names = "fck", "brg_int", "scif_clk";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7779_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -300,7 +304,7 @@
<GIC_SPI 34 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp0_clks R8A7779_CLK_TMU0>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7779_PD_ALWAYS_ON>;
#renesas,channels = <3>;
@@ -315,7 +319,7 @@
<GIC_SPI 38 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp0_clks R8A7779_CLK_TMU1>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7779_PD_ALWAYS_ON>;
#renesas,channels = <3>;
@@ -330,7 +334,7 @@
<GIC_SPI 42 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp0_clks R8A7779_CLK_TMU2>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7779_PD_ALWAYS_ON>;
#renesas,channels = <3>;
@@ -342,7 +346,7 @@
reg = <0xfc600000 0x2000>;
interrupts = <GIC_SPI 100 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp1_clks R8A7779_CLK_SATA>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7779_PD_ALWAYS_ON>;
};
sdhi0: sd@ffe4c000 {
@@ -350,7 +354,7 @@
reg = <0xffe4c000 0x100>;
interrupts = <GIC_SPI 104 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp3_clks R8A7779_CLK_SDHI0>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7779_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -359,7 +363,7 @@
reg = <0xffe4d000 0x100>;
interrupts = <GIC_SPI 105 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp3_clks R8A7779_CLK_SDHI1>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7779_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -368,7 +372,7 @@
reg = <0xffe4e000 0x100>;
interrupts = <GIC_SPI 107 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp3_clks R8A7779_CLK_SDHI2>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7779_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -377,7 +381,7 @@
reg = <0xffe4f000 0x100>;
interrupts = <GIC_SPI 106 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp3_clks R8A7779_CLK_SDHI3>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7779_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -388,7 +392,7 @@
#address-cells = <1>;
#size-cells = <0>;
clocks = <&mstp0_clks R8A7779_CLK_HSPI>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7779_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -399,7 +403,7 @@
#address-cells = <1>;
#size-cells = <0>;
clocks = <&mstp0_clks R8A7779_CLK_HSPI>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7779_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -410,7 +414,7 @@
#address-cells = <1>;
#size-cells = <0>;
clocks = <&mstp0_clks R8A7779_CLK_HSPI>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7779_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -419,7 +423,7 @@
reg = <0 0xfff80000 0 0x40000>;
interrupts = <GIC_SPI 31 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp1_clks R8A7779_CLK_DU>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7779_PD_ALWAYS_ON>;
status = "disabled";
ports {
@@ -585,4 +589,10 @@
"mmc1", "mmc0";
};
};
+
+ sysc: system-controller@ffd85000 {
+ compatible = "renesas,r8a7779-sysc";
+ reg = <0xffd85000 0x0200>;
+ #power-domain-cells = <1>;
+ };
};
diff --git a/arch/arm/boot/dts/r8a7790.dtsi b/arch/arm/boot/dts/r8a7790.dtsi
index 935064f..83cf23c 100644
--- a/arch/arm/boot/dts/r8a7790.dtsi
+++ b/arch/arm/boot/dts/r8a7790.dtsi
@@ -13,6 +13,7 @@
#include <dt-bindings/clock/r8a7790-clock.h>
#include <dt-bindings/interrupt-controller/arm-gic.h>
#include <dt-bindings/interrupt-controller/irq.h>
+#include <dt-bindings/power/r8a7790-sysc.h>
/ {
compatible = "renesas,r8a7790";
@@ -52,6 +53,7 @@
voltage-tolerance = <1>; /* 1% */
clocks = <&cpg_clocks R8A7790_CLK_Z>;
clock-latency = <300000>; /* 300 us */
+ power-domains = <&sysc R8A7790_PD_CA15_CPU0>;
next-level-cache = <&L2_CA15>;
/* kHz - uV - OPPs unknown yet */
@@ -68,6 +70,7 @@
compatible = "arm,cortex-a15";
reg = <1>;
clock-frequency = <1300000000>;
+ power-domains = <&sysc R8A7790_PD_CA15_CPU1>;
next-level-cache = <&L2_CA15>;
};
@@ -76,6 +79,7 @@
compatible = "arm,cortex-a15";
reg = <2>;
clock-frequency = <1300000000>;
+ power-domains = <&sysc R8A7790_PD_CA15_CPU2>;
next-level-cache = <&L2_CA15>;
};
@@ -84,6 +88,7 @@
compatible = "arm,cortex-a15";
reg = <3>;
clock-frequency = <1300000000>;
+ power-domains = <&sysc R8A7790_PD_CA15_CPU3>;
next-level-cache = <&L2_CA15>;
};
@@ -92,6 +97,7 @@
compatible = "arm,cortex-a7";
reg = <0x100>;
clock-frequency = <780000000>;
+ power-domains = <&sysc R8A7790_PD_CA7_CPU0>;
next-level-cache = <&L2_CA7>;
};
@@ -100,6 +106,7 @@
compatible = "arm,cortex-a7";
reg = <0x101>;
clock-frequency = <780000000>;
+ power-domains = <&sysc R8A7790_PD_CA7_CPU1>;
next-level-cache = <&L2_CA7>;
};
@@ -108,6 +115,7 @@
compatible = "arm,cortex-a7";
reg = <0x102>;
clock-frequency = <780000000>;
+ power-domains = <&sysc R8A7790_PD_CA7_CPU2>;
next-level-cache = <&L2_CA7>;
};
@@ -116,6 +124,7 @@
compatible = "arm,cortex-a7";
reg = <0x103>;
clock-frequency = <780000000>;
+ power-domains = <&sysc R8A7790_PD_CA7_CPU3>;
next-level-cache = <&L2_CA7>;
};
};
@@ -141,12 +150,14 @@
L2_CA15: cache-controller@0 {
compatible = "cache";
+ power-domains = <&sysc R8A7790_PD_CA15_SCU>;
cache-unified;
cache-level = <2>;
};
L2_CA7: cache-controller@1 {
compatible = "cache";
+ power-domains = <&sysc R8A7790_PD_CA7_SCU>;
cache-unified;
cache-level = <2>;
};
@@ -173,7 +184,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7790_CLK_GPIO0>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
};
gpio1: gpio@e6051000 {
@@ -186,7 +197,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7790_CLK_GPIO1>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
};
gpio2: gpio@e6052000 {
@@ -199,7 +210,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7790_CLK_GPIO2>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
};
gpio3: gpio@e6053000 {
@@ -212,7 +223,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7790_CLK_GPIO3>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
};
gpio4: gpio@e6054000 {
@@ -225,7 +236,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7790_CLK_GPIO4>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
};
gpio5: gpio@e6055000 {
@@ -238,7 +249,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7790_CLK_GPIO5>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
};
thermal: thermal@e61f0000 {
@@ -248,7 +259,7 @@
reg = <0 0xe61f0000 0 0x14>, <0 0xe61f0100 0 0x38>;
interrupts = <GIC_SPI 69 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp5_clks R8A7790_CLK_THERMAL>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
#thermal-sensor-cells = <0>;
};
@@ -267,7 +278,7 @@
<GIC_SPI 143 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp1_clks R8A7790_CLK_CMT0>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
renesas,channels-mask = <0x60>;
@@ -287,7 +298,7 @@
<GIC_SPI 127 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp3_clks R8A7790_CLK_CMT1>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
renesas,channels-mask = <0xff>;
@@ -304,7 +315,7 @@
<GIC_SPI 2 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 3 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp4_clks R8A7790_CLK_IRQC>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
};
dmac0: dma-controller@e6700000 {
@@ -333,7 +344,7 @@
"ch12", "ch13", "ch14";
clocks = <&mstp2_clks R8A7790_CLK_SYS_DMAC0>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <15>;
};
@@ -364,7 +375,7 @@
"ch12", "ch13", "ch14";
clocks = <&mstp2_clks R8A7790_CLK_SYS_DMAC1>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <15>;
};
@@ -393,7 +404,7 @@
"ch12";
clocks = <&mstp5_clks R8A7790_CLK_AUDIO_DMAC0>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <13>;
};
@@ -422,7 +433,7 @@
"ch12";
clocks = <&mstp5_clks R8A7790_CLK_AUDIO_DMAC1>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <13>;
};
@@ -434,7 +445,7 @@
GIC_SPI 109 IRQ_TYPE_LEVEL_HIGH>;
interrupt-names = "ch0", "ch1";
clocks = <&mstp3_clks R8A7790_CLK_USBDMAC0>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <2>;
};
@@ -446,7 +457,7 @@
GIC_SPI 110 IRQ_TYPE_LEVEL_HIGH>;
interrupt-names = "ch0", "ch1";
clocks = <&mstp3_clks R8A7790_CLK_USBDMAC1>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <2>;
};
@@ -458,7 +469,7 @@
reg = <0 0xe6508000 0 0x40>;
interrupts = <GIC_SPI 287 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp9_clks R8A7790_CLK_I2C0>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <110>;
status = "disabled";
};
@@ -470,7 +481,7 @@
reg = <0 0xe6518000 0 0x40>;
interrupts = <GIC_SPI 288 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp9_clks R8A7790_CLK_I2C1>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <6>;
status = "disabled";
};
@@ -482,7 +493,7 @@
reg = <0 0xe6530000 0 0x40>;
interrupts = <GIC_SPI 286 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp9_clks R8A7790_CLK_I2C2>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <6>;
status = "disabled";
};
@@ -494,7 +505,7 @@
reg = <0 0xe6540000 0 0x40>;
interrupts = <GIC_SPI 290 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp9_clks R8A7790_CLK_I2C3>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <110>;
status = "disabled";
};
@@ -508,7 +519,7 @@
clocks = <&mstp3_clks R8A7790_CLK_IIC0>;
dmas = <&dmac0 0x61>, <&dmac0 0x62>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -521,7 +532,7 @@
clocks = <&mstp3_clks R8A7790_CLK_IIC1>;
dmas = <&dmac0 0x65>, <&dmac0 0x66>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -534,7 +545,7 @@
clocks = <&mstp3_clks R8A7790_CLK_IIC2>;
dmas = <&dmac0 0x69>, <&dmac0 0x6a>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -547,7 +558,7 @@
clocks = <&mstp9_clks R8A7790_CLK_IICDVFS>;
dmas = <&dmac0 0x77>, <&dmac0 0x78>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -558,7 +569,7 @@
clocks = <&mstp3_clks R8A7790_CLK_MMCIF0>;
dmas = <&dmac0 0xd1>, <&dmac0 0xd2>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
reg-io-width = <4>;
status = "disabled";
max-frequency = <97500000>;
@@ -571,7 +582,7 @@
clocks = <&mstp3_clks R8A7790_CLK_MMCIF1>;
dmas = <&dmac0 0xe1>, <&dmac0 0xe2>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
reg-io-width = <4>;
status = "disabled";
max-frequency = <97500000>;
@@ -590,7 +601,7 @@
dmas = <&dmac1 0xcd>, <&dmac1 0xce>;
dma-names = "tx", "rx";
max-frequency = <195000000>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -602,7 +613,7 @@
dmas = <&dmac1 0xc9>, <&dmac1 0xca>;
dma-names = "tx", "rx";
max-frequency = <195000000>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -614,7 +625,7 @@
dmas = <&dmac1 0xc1>, <&dmac1 0xc2>;
dma-names = "tx", "rx";
max-frequency = <97500000>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -626,7 +637,7 @@
dmas = <&dmac1 0xd3>, <&dmac1 0xd4>;
dma-names = "tx", "rx";
max-frequency = <97500000>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -639,7 +650,7 @@
clock-names = "fck";
dmas = <&dmac0 0x21>, <&dmac0 0x22>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -652,7 +663,7 @@
clock-names = "fck";
dmas = <&dmac0 0x25>, <&dmac0 0x26>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -665,7 +676,7 @@
clock-names = "fck";
dmas = <&dmac0 0x27>, <&dmac0 0x28>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -678,7 +689,7 @@
clock-names = "fck";
dmas = <&dmac0 0x3d>, <&dmac0 0x3e>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -691,7 +702,7 @@
clock-names = "fck";
dmas = <&dmac0 0x19>, <&dmac0 0x1a>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -704,7 +715,7 @@
clock-names = "fck";
dmas = <&dmac0 0x1d>, <&dmac0 0x1e>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -718,7 +729,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x29>, <&dmac0 0x2a>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -732,7 +743,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x2d>, <&dmac0 0x2e>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -746,7 +757,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x2b>, <&dmac0 0x2c>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -760,7 +771,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x39>, <&dmac0 0x3a>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -774,7 +785,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x4d>, <&dmac0 0x4e>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -783,7 +794,7 @@
reg = <0 0xee700000 0 0x400>;
interrupts = <GIC_SPI 162 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp8_clks R8A7790_CLK_ETHER>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
phy-mode = "rmii";
#address-cells = <1>;
#size-cells = <0>;
@@ -796,7 +807,7 @@
reg = <0 0xe6800000 0 0x800>, <0 0xee0e8000 0 0x4000>;
interrupts = <GIC_SPI 163 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp8_clks R8A7790_CLK_ETHERAVB>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
#address-cells = <1>;
#size-cells = <0>;
status = "disabled";
@@ -807,7 +818,7 @@
reg = <0 0xee300000 0 0x2000>;
interrupts = <GIC_SPI 105 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp8_clks R8A7790_CLK_SATA0>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -816,7 +827,7 @@
reg = <0 0xee500000 0 0x2000>;
interrupts = <GIC_SPI 106 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp8_clks R8A7790_CLK_SATA1>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -828,7 +839,7 @@
dmas = <&usb_dmac0 0>, <&usb_dmac0 1>,
<&usb_dmac1 0>, <&usb_dmac1 1>;
dma-names = "ch0", "ch1", "ch2", "ch3";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
renesas,buswait = <4>;
phys = <&usb0 1>;
phy-names = "usb";
@@ -842,7 +853,7 @@
#size-cells = <0>;
clocks = <&mstp7_clks R8A7790_CLK_HSUSB>;
clock-names = "usbhs";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
usb0: usb-channel@0 {
@@ -860,7 +871,7 @@
reg = <0 0xe6ef0000 0 0x1000>;
interrupts = <GIC_SPI 188 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp8_clks R8A7790_CLK_VIN0>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -869,7 +880,7 @@
reg = <0 0xe6ef1000 0 0x1000>;
interrupts = <GIC_SPI 189 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp8_clks R8A7790_CLK_VIN1>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -878,7 +889,7 @@
reg = <0 0xe6ef2000 0 0x1000>;
interrupts = <GIC_SPI 190 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp8_clks R8A7790_CLK_VIN2>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -887,7 +898,7 @@
reg = <0 0xe6ef3000 0 0x1000>;
interrupts = <GIC_SPI 191 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp8_clks R8A7790_CLK_VIN3>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -896,7 +907,7 @@
reg = <0 0xfe920000 0 0x8000>;
interrupts = <GIC_SPI 266 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp1_clks R8A7790_CLK_VSP1_R>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
renesas,has-sru;
renesas,#rpf = <5>;
@@ -909,7 +920,7 @@
reg = <0 0xfe928000 0 0x8000>;
interrupts = <GIC_SPI 267 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp1_clks R8A7790_CLK_VSP1_S>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
renesas,has-lut;
renesas,has-sru;
@@ -923,7 +934,7 @@
reg = <0 0xfe930000 0 0x8000>;
interrupts = <GIC_SPI 246 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp1_clks R8A7790_CLK_VSP1_DU0>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
renesas,has-lif;
renesas,has-lut;
@@ -937,7 +948,7 @@
reg = <0 0xfe938000 0 0x8000>;
interrupts = <GIC_SPI 247 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp1_clks R8A7790_CLK_VSP1_DU1>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
renesas,has-lif;
renesas,has-lut;
@@ -992,7 +1003,7 @@
clocks = <&mstp9_clks R8A7790_CLK_RCAN0>,
<&cpg_clocks R8A7790_CLK_RCAN>, <&can_clk>;
clock-names = "clkp1", "clkp2", "can_clk";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -1003,7 +1014,7 @@
clocks = <&mstp9_clks R8A7790_CLK_RCAN1>,
<&cpg_clocks R8A7790_CLK_RCAN>, <&can_clk>;
clock-names = "clkp1", "clkp2", "can_clk";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -1012,7 +1023,7 @@
reg = <0 0xfe980000 0 0x10300>;
interrupts = <GIC_SPI 272 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp1_clks R8A7790_CLK_JPU>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
};
clocks {
@@ -1447,6 +1458,12 @@
};
};
+ sysc: system-controller@e6180000 {
+ compatible = "renesas,r8a7790-sysc";
+ reg = <0 0xe6180000 0 0x0200>;
+ #power-domain-cells = <1>;
+ };
+
qspi: spi@e6b10000 {
compatible = "renesas,qspi-r8a7790", "renesas,qspi";
reg = <0 0xe6b10000 0 0x2c>;
@@ -1454,7 +1471,7 @@
clocks = <&mstp9_clks R8A7790_CLK_QSPI_MOD>;
dmas = <&dmac0 0x17>, <&dmac0 0x18>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
num-cs = <1>;
#address-cells = <1>;
#size-cells = <0>;
@@ -1468,7 +1485,7 @@
clocks = <&mstp0_clks R8A7790_CLK_MSIOF0>;
dmas = <&dmac0 0x51>, <&dmac0 0x52>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
#address-cells = <1>;
#size-cells = <0>;
status = "disabled";
@@ -1481,7 +1498,7 @@
clocks = <&mstp2_clks R8A7790_CLK_MSIOF1>;
dmas = <&dmac0 0x55>, <&dmac0 0x56>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
#address-cells = <1>;
#size-cells = <0>;
status = "disabled";
@@ -1494,7 +1511,7 @@
clocks = <&mstp2_clks R8A7790_CLK_MSIOF2>;
dmas = <&dmac0 0x41>, <&dmac0 0x42>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
#address-cells = <1>;
#size-cells = <0>;
status = "disabled";
@@ -1507,7 +1524,7 @@
clocks = <&mstp2_clks R8A7790_CLK_MSIOF3>;
dmas = <&dmac0 0x45>, <&dmac0 0x46>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
#address-cells = <1>;
#size-cells = <0>;
status = "disabled";
@@ -1518,7 +1535,7 @@
reg = <0 0xee000000 0 0xc00>;
interrupts = <GIC_SPI 101 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp3_clks R8A7790_CLK_SSUSB>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
phys = <&usb2 1>;
phy-names = "usb";
status = "disabled";
@@ -1531,7 +1548,7 @@
<0 0xee080000 0 0x1100>;
interrupts = <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp7_clks R8A7790_CLK_EHCI>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
bus-range = <0 0>;
@@ -1566,7 +1583,7 @@
<0 0xee0a0000 0 0x1100>;
interrupts = <GIC_SPI 112 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp7_clks R8A7790_CLK_EHCI>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
bus-range = <1 1>;
@@ -1584,7 +1601,7 @@
compatible = "renesas,pci-r8a7790", "renesas,pci-rcar-gen2";
device_type = "pci";
clocks = <&mstp7_clks R8A7790_CLK_EHCI>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
reg = <0 0xee0d0000 0 0xc00>,
<0 0xee0c0000 0 0x1100>;
interrupts = <GIC_SPI 113 IRQ_TYPE_LEVEL_HIGH>;
@@ -1637,7 +1654,7 @@
interrupt-map = <0 0 0 0 &gic GIC_SPI 116 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp3_clks R8A7790_CLK_PCIEC>, <&pcie_bus_clk>;
clock-names = "pcie", "pcie_bus";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -1680,7 +1697,7 @@
"mix.0", "mix.1",
"dvc.0", "dvc.1",
"clk_a", "clk_b", "clk_c", "clk_i";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7790_PD_ALWAYS_ON>;
status = "disabled";
diff --git a/arch/arm/boot/dts/r8a7791.dtsi b/arch/arm/boot/dts/r8a7791.dtsi
index 565c270..db67e34 100644
--- a/arch/arm/boot/dts/r8a7791.dtsi
+++ b/arch/arm/boot/dts/r8a7791.dtsi
@@ -13,6 +13,7 @@
#include <dt-bindings/clock/r8a7791-clock.h>
#include <dt-bindings/interrupt-controller/arm-gic.h>
#include <dt-bindings/interrupt-controller/irq.h>
+#include <dt-bindings/power/r8a7791-sysc.h>
/ {
compatible = "renesas,r8a7791";
@@ -51,6 +52,7 @@
voltage-tolerance = <1>; /* 1% */
clocks = <&cpg_clocks R8A7791_CLK_Z>;
clock-latency = <300000>; /* 300 us */
+ power-domains = <&sysc R8A7791_PD_CA15_CPU0>;
next-level-cache = <&L2_CA15>;
/* kHz - uV - OPPs unknown yet */
@@ -67,6 +69,7 @@
compatible = "arm,cortex-a15";
reg = <1>;
clock-frequency = <1500000000>;
+ power-domains = <&sysc R8A7791_PD_CA15_CPU1>;
next-level-cache = <&L2_CA15>;
};
};
@@ -92,6 +95,7 @@
L2_CA15: cache-controller@0 {
compatible = "cache";
+ power-domains = <&sysc R8A7791_PD_CA15_SCU>;
cache-unified;
cache-level = <2>;
};
@@ -118,7 +122,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7791_CLK_GPIO0>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
};
gpio1: gpio@e6051000 {
@@ -131,7 +135,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7791_CLK_GPIO1>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
};
gpio2: gpio@e6052000 {
@@ -144,7 +148,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7791_CLK_GPIO2>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
};
gpio3: gpio@e6053000 {
@@ -157,7 +161,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7791_CLK_GPIO3>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
};
gpio4: gpio@e6054000 {
@@ -170,7 +174,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7791_CLK_GPIO4>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
};
gpio5: gpio@e6055000 {
@@ -183,7 +187,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7791_CLK_GPIO5>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
};
gpio6: gpio@e6055400 {
@@ -196,7 +200,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7791_CLK_GPIO6>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
};
gpio7: gpio@e6055800 {
@@ -209,7 +213,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7791_CLK_GPIO7>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
};
thermal: thermal@e61f0000 {
@@ -219,7 +223,7 @@
reg = <0 0xe61f0000 0 0x14>, <0 0xe61f0100 0 0x38>;
interrupts = <GIC_SPI 69 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp5_clks R8A7791_CLK_THERMAL>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
#thermal-sensor-cells = <0>;
};
@@ -238,7 +242,7 @@
<GIC_SPI 143 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp1_clks R8A7791_CLK_CMT0>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
renesas,channels-mask = <0x60>;
@@ -258,7 +262,7 @@
<GIC_SPI 127 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp3_clks R8A7791_CLK_CMT1>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
renesas,channels-mask = <0xff>;
@@ -281,7 +285,7 @@
<GIC_SPI 16 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 17 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp4_clks R8A7791_CLK_IRQC>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
};
dmac0: dma-controller@e6700000 {
@@ -310,7 +314,7 @@
"ch12", "ch13", "ch14";
clocks = <&mstp2_clks R8A7791_CLK_SYS_DMAC0>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <15>;
};
@@ -341,7 +345,7 @@
"ch12", "ch13", "ch14";
clocks = <&mstp2_clks R8A7791_CLK_SYS_DMAC1>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <15>;
};
@@ -370,7 +374,7 @@
"ch12";
clocks = <&mstp5_clks R8A7791_CLK_AUDIO_DMAC0>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <13>;
};
@@ -399,7 +403,7 @@
"ch12";
clocks = <&mstp5_clks R8A7791_CLK_AUDIO_DMAC1>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <13>;
};
@@ -411,7 +415,7 @@
GIC_SPI 109 IRQ_TYPE_LEVEL_HIGH>;
interrupt-names = "ch0", "ch1";
clocks = <&mstp3_clks R8A7791_CLK_USBDMAC0>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <2>;
};
@@ -423,7 +427,7 @@
GIC_SPI 110 IRQ_TYPE_LEVEL_HIGH>;
interrupt-names = "ch0", "ch1";
clocks = <&mstp3_clks R8A7791_CLK_USBDMAC1>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <2>;
};
@@ -436,7 +440,7 @@
reg = <0 0xe6508000 0 0x40>;
interrupts = <GIC_SPI 287 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp9_clks R8A7791_CLK_I2C0>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <6>;
status = "disabled";
};
@@ -448,7 +452,7 @@
reg = <0 0xe6518000 0 0x40>;
interrupts = <GIC_SPI 288 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp9_clks R8A7791_CLK_I2C1>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <6>;
status = "disabled";
};
@@ -460,7 +464,7 @@
reg = <0 0xe6530000 0 0x40>;
interrupts = <GIC_SPI 286 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp9_clks R8A7791_CLK_I2C2>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <6>;
status = "disabled";
};
@@ -472,7 +476,7 @@
reg = <0 0xe6540000 0 0x40>;
interrupts = <GIC_SPI 290 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp9_clks R8A7791_CLK_I2C3>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <6>;
status = "disabled";
};
@@ -484,7 +488,7 @@
reg = <0 0xe6520000 0 0x40>;
interrupts = <GIC_SPI 19 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp9_clks R8A7791_CLK_I2C4>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <6>;
status = "disabled";
};
@@ -497,7 +501,7 @@
reg = <0 0xe6528000 0 0x40>;
interrupts = <GIC_SPI 20 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp9_clks R8A7791_CLK_I2C5>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <110>;
status = "disabled";
};
@@ -512,7 +516,7 @@
clocks = <&mstp9_clks R8A7791_CLK_IICDVFS>;
dmas = <&dmac0 0x77>, <&dmac0 0x78>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -525,7 +529,7 @@
clocks = <&mstp3_clks R8A7791_CLK_IIC0>;
dmas = <&dmac0 0x61>, <&dmac0 0x62>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -538,7 +542,7 @@
clocks = <&mstp3_clks R8A7791_CLK_IIC1>;
dmas = <&dmac0 0x65>, <&dmac0 0x66>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -554,7 +558,7 @@
clocks = <&mstp3_clks R8A7791_CLK_MMCIF0>;
dmas = <&dmac0 0xd1>, <&dmac0 0xd2>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
reg-io-width = <4>;
status = "disabled";
max-frequency = <97500000>;
@@ -567,7 +571,7 @@
clocks = <&mstp3_clks R8A7791_CLK_SDHI0>;
dmas = <&dmac1 0xcd>, <&dmac1 0xce>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -578,7 +582,7 @@
clocks = <&mstp3_clks R8A7791_CLK_SDHI1>;
dmas = <&dmac1 0xc1>, <&dmac1 0xc2>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -589,7 +593,7 @@
clocks = <&mstp3_clks R8A7791_CLK_SDHI2>;
dmas = <&dmac1 0xd3>, <&dmac1 0xd4>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -602,7 +606,7 @@
clock-names = "fck";
dmas = <&dmac0 0x21>, <&dmac0 0x22>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -615,7 +619,7 @@
clock-names = "fck";
dmas = <&dmac0 0x25>, <&dmac0 0x26>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -628,7 +632,7 @@
clock-names = "fck";
dmas = <&dmac0 0x27>, <&dmac0 0x28>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -641,7 +645,7 @@
clock-names = "fck";
dmas = <&dmac0 0x1b>, <&dmac0 0x1c>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -654,7 +658,7 @@
clock-names = "fck";
dmas = <&dmac0 0x1f>, <&dmac0 0x20>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -667,7 +671,7 @@
clock-names = "fck";
dmas = <&dmac0 0x23>, <&dmac0 0x24>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -680,7 +684,7 @@
clock-names = "fck";
dmas = <&dmac0 0x3d>, <&dmac0 0x3e>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -693,7 +697,7 @@
clock-names = "fck";
dmas = <&dmac0 0x19>, <&dmac0 0x1a>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -706,7 +710,7 @@
clock-names = "fck";
dmas = <&dmac0 0x1d>, <&dmac0 0x1e>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -720,7 +724,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x29>, <&dmac0 0x2a>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -734,7 +738,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x2d>, <&dmac0 0x2e>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -748,7 +752,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x2b>, <&dmac0 0x2c>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -762,7 +766,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x2f>, <&dmac0 0x30>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -776,7 +780,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0xfb>, <&dmac0 0xfc>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -790,7 +794,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0xfd>, <&dmac0 0xfe>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -804,7 +808,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x39>, <&dmac0 0x3a>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -818,7 +822,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x4d>, <&dmac0 0x4e>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -832,7 +836,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x3b>, <&dmac0 0x3c>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -841,7 +845,7 @@
reg = <0 0xee700000 0 0x400>;
interrupts = <GIC_SPI 162 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp8_clks R8A7791_CLK_ETHER>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
phy-mode = "rmii";
#address-cells = <1>;
#size-cells = <0>;
@@ -854,7 +858,7 @@
reg = <0 0xe6800000 0 0x800>, <0 0xee0e8000 0 0x4000>;
interrupts = <GIC_SPI 163 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp8_clks R8A7791_CLK_ETHERAVB>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
#address-cells = <1>;
#size-cells = <0>;
status = "disabled";
@@ -865,7 +869,7 @@
reg = <0 0xee300000 0 0x2000>;
interrupts = <GIC_SPI 105 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp8_clks R8A7791_CLK_SATA0>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -874,7 +878,7 @@
reg = <0 0xee500000 0 0x2000>;
interrupts = <GIC_SPI 106 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp8_clks R8A7791_CLK_SATA1>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -886,7 +890,7 @@
dmas = <&usb_dmac0 0>, <&usb_dmac0 1>,
<&usb_dmac1 0>, <&usb_dmac1 1>;
dma-names = "ch0", "ch1", "ch2", "ch3";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
renesas,buswait = <4>;
phys = <&usb0 1>;
phy-names = "usb";
@@ -900,7 +904,7 @@
#size-cells = <0>;
clocks = <&mstp7_clks R8A7791_CLK_HSUSB>;
clock-names = "usbhs";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
usb0: usb-channel@0 {
@@ -918,7 +922,7 @@
reg = <0 0xe6ef0000 0 0x1000>;
interrupts = <GIC_SPI 188 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp8_clks R8A7791_CLK_VIN0>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -927,7 +931,7 @@
reg = <0 0xe6ef1000 0 0x1000>;
interrupts = <GIC_SPI 189 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp8_clks R8A7791_CLK_VIN1>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -936,7 +940,7 @@
reg = <0 0xe6ef2000 0 0x1000>;
interrupts = <GIC_SPI 190 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp8_clks R8A7791_CLK_VIN2>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -945,7 +949,7 @@
reg = <0 0xfe928000 0 0x8000>;
interrupts = <GIC_SPI 267 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp1_clks R8A7791_CLK_VSP1_S>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
renesas,has-lut;
renesas,has-sru;
@@ -959,7 +963,7 @@
reg = <0 0xfe930000 0 0x8000>;
interrupts = <GIC_SPI 246 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp1_clks R8A7791_CLK_VSP1_DU0>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
renesas,has-lif;
renesas,has-lut;
@@ -973,7 +977,7 @@
reg = <0 0xfe938000 0 0x8000>;
interrupts = <GIC_SPI 247 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp1_clks R8A7791_CLK_VSP1_DU1>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
renesas,has-lif;
renesas,has-lut;
@@ -1019,7 +1023,7 @@
clocks = <&mstp9_clks R8A7791_CLK_RCAN0>,
<&cpg_clocks R8A7791_CLK_RCAN>, <&can_clk>;
clock-names = "clkp1", "clkp2", "can_clk";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -1030,7 +1034,7 @@
clocks = <&mstp9_clks R8A7791_CLK_RCAN1>,
<&cpg_clocks R8A7791_CLK_RCAN>, <&can_clk>;
clock-names = "clkp1", "clkp2", "can_clk";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -1039,7 +1043,7 @@
reg = <0 0xfe980000 0 0x10300>;
interrupts = <GIC_SPI 272 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp1_clks R8A7791_CLK_JPU>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
};
clocks {
@@ -1463,6 +1467,12 @@
};
};
+ sysc: system-controller@e6180000 {
+ compatible = "renesas,r8a7791-sysc";
+ reg = <0 0xe6180000 0 0x0200>;
+ #power-domain-cells = <1>;
+ };
+
qspi: spi@e6b10000 {
compatible = "renesas,qspi-r8a7791", "renesas,qspi";
reg = <0 0xe6b10000 0 0x2c>;
@@ -1470,7 +1480,7 @@
clocks = <&mstp9_clks R8A7791_CLK_QSPI_MOD>;
dmas = <&dmac0 0x17>, <&dmac0 0x18>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
num-cs = <1>;
#address-cells = <1>;
#size-cells = <0>;
@@ -1484,7 +1494,7 @@
clocks = <&mstp0_clks R8A7791_CLK_MSIOF0>;
dmas = <&dmac0 0x51>, <&dmac0 0x52>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
#address-cells = <1>;
#size-cells = <0>;
status = "disabled";
@@ -1497,7 +1507,7 @@
clocks = <&mstp2_clks R8A7791_CLK_MSIOF1>;
dmas = <&dmac0 0x55>, <&dmac0 0x56>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
#address-cells = <1>;
#size-cells = <0>;
status = "disabled";
@@ -1510,7 +1520,7 @@
clocks = <&mstp2_clks R8A7791_CLK_MSIOF2>;
dmas = <&dmac0 0x41>, <&dmac0 0x42>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
#address-cells = <1>;
#size-cells = <0>;
status = "disabled";
@@ -1521,7 +1531,7 @@
reg = <0 0xee000000 0 0xc00>;
interrupts = <GIC_SPI 101 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp3_clks R8A7791_CLK_SSUSB>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
phys = <&usb2 1>;
phy-names = "usb";
status = "disabled";
@@ -1534,7 +1544,7 @@
<0 0xee080000 0 0x1100>;
interrupts = <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp7_clks R8A7791_CLK_EHCI>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
bus-range = <0 0>;
@@ -1569,7 +1579,7 @@
<0 0xee0c0000 0 0x1100>;
interrupts = <GIC_SPI 113 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp7_clks R8A7791_CLK_EHCI>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
bus-range = <1 1>;
@@ -1619,7 +1629,7 @@
interrupt-map = <0 0 0 0 &gic GIC_SPI 116 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp3_clks R8A7791_CLK_PCIEC>, <&pcie_bus_clk>;
clock-names = "pcie", "pcie_bus";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -1722,7 +1732,7 @@
"mix.0", "mix.1",
"dvc.0", "dvc.1",
"clk_a", "clk_b", "clk_c", "clk_i";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7791_PD_ALWAYS_ON>;
status = "disabled";
diff --git a/arch/arm/boot/dts/r8a7793.dtsi b/arch/arm/boot/dts/r8a7793.dtsi
index cf6dc2a..1dd6d20 100644
--- a/arch/arm/boot/dts/r8a7793.dtsi
+++ b/arch/arm/boot/dts/r8a7793.dtsi
@@ -11,6 +11,7 @@
#include <dt-bindings/clock/r8a7793-clock.h>
#include <dt-bindings/interrupt-controller/arm-gic.h>
#include <dt-bindings/interrupt-controller/irq.h>
+#include <dt-bindings/power/r8a7793-sysc.h>
/ {
compatible = "renesas,r8a7793";
@@ -43,6 +44,7 @@
voltage-tolerance = <1>; /* 1% */
clocks = <&cpg_clocks R8A7793_CLK_Z>;
clock-latency = <300000>; /* 300 us */
+ power-domains = <&sysc R8A7793_PD_CA15_CPU0>;
/* kHz - uV - OPPs unknown yet */
operating-points = <1500000 1000000>,
@@ -76,6 +78,7 @@
L2_CA15: cache-controller@0 {
compatible = "cache";
+ power-domains = <&sysc R8A7793_PD_CA15_SCU>;
cache-unified;
cache-level = <2>;
};
@@ -102,7 +105,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7793_CLK_GPIO0>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
};
gpio1: gpio@e6051000 {
@@ -115,7 +118,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7793_CLK_GPIO1>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
};
gpio2: gpio@e6052000 {
@@ -128,7 +131,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7793_CLK_GPIO2>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
};
gpio3: gpio@e6053000 {
@@ -141,7 +144,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7793_CLK_GPIO3>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
};
gpio4: gpio@e6054000 {
@@ -154,7 +157,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7793_CLK_GPIO4>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
};
gpio5: gpio@e6055000 {
@@ -167,7 +170,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7793_CLK_GPIO5>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
};
gpio6: gpio@e6055400 {
@@ -180,7 +183,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7793_CLK_GPIO6>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
};
gpio7: gpio@e6055800 {
@@ -193,7 +196,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7793_CLK_GPIO7>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
};
thermal: thermal@e61f0000 {
@@ -203,7 +206,7 @@
reg = <0 0xe61f0000 0 0x14>, <0 0xe61f0100 0 0x38>;
interrupts = <GIC_SPI 69 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp5_clks R8A7793_CLK_THERMAL>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
#thermal-sensor-cells = <0>;
};
@@ -222,7 +225,7 @@
<GIC_SPI 143 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp1_clks R8A7793_CLK_CMT0>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
renesas,channels-mask = <0x60>;
@@ -242,7 +245,7 @@
<GIC_SPI 127 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp3_clks R8A7793_CLK_CMT1>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
renesas,channels-mask = <0xff>;
@@ -265,7 +268,7 @@
<GIC_SPI 16 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 17 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp4_clks R8A7793_CLK_IRQC>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
};
dmac0: dma-controller@e6700000 {
@@ -294,7 +297,7 @@
"ch12", "ch13", "ch14";
clocks = <&mstp2_clks R8A7793_CLK_SYS_DMAC0>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <15>;
};
@@ -325,7 +328,7 @@
"ch12", "ch13", "ch14";
clocks = <&mstp2_clks R8A7793_CLK_SYS_DMAC1>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <15>;
};
@@ -354,7 +357,7 @@
"ch12";
clocks = <&mstp5_clks R8A7793_CLK_AUDIO_DMAC0>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <13>;
};
@@ -383,7 +386,7 @@
"ch12";
clocks = <&mstp5_clks R8A7793_CLK_AUDIO_DMAC1>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <13>;
};
@@ -396,7 +399,7 @@
reg = <0 0xe6508000 0 0x40>;
interrupts = <GIC_SPI 287 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp9_clks R8A7793_CLK_I2C0>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <6>;
status = "disabled";
};
@@ -408,7 +411,7 @@
reg = <0 0xe6518000 0 0x40>;
interrupts = <GIC_SPI 288 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp9_clks R8A7793_CLK_I2C1>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <6>;
status = "disabled";
};
@@ -420,7 +423,7 @@
reg = <0 0xe6530000 0 0x40>;
interrupts = <GIC_SPI 286 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp9_clks R8A7793_CLK_I2C2>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <6>;
status = "disabled";
};
@@ -432,7 +435,7 @@
reg = <0 0xe6540000 0 0x40>;
interrupts = <GIC_SPI 290 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp9_clks R8A7793_CLK_I2C3>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <6>;
status = "disabled";
};
@@ -444,7 +447,7 @@
reg = <0 0xe6520000 0 0x40>;
interrupts = <GIC_SPI 19 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp9_clks R8A7793_CLK_I2C4>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <6>;
status = "disabled";
};
@@ -457,7 +460,7 @@
reg = <0 0xe6528000 0 0x40>;
interrupts = <GIC_SPI 20 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp9_clks R8A7793_CLK_I2C5>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <110>;
status = "disabled";
};
@@ -472,7 +475,7 @@
clocks = <&mstp9_clks R8A7793_CLK_IICDVFS>;
dmas = <&dmac0 0x77>, <&dmac0 0x78>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -485,7 +488,7 @@
clocks = <&mstp3_clks R8A7793_CLK_IIC0>;
dmas = <&dmac0 0x61>, <&dmac0 0x62>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -498,7 +501,7 @@
clocks = <&mstp3_clks R8A7793_CLK_IIC1>;
dmas = <&dmac0 0x65>, <&dmac0 0x66>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -514,7 +517,7 @@
clocks = <&mstp3_clks R8A7793_CLK_SDHI0>;
dmas = <&dmac0 0xcd>, <&dmac0 0xce>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -525,7 +528,7 @@
clocks = <&mstp3_clks R8A7793_CLK_SDHI1>;
dmas = <&dmac0 0xc1>, <&dmac0 0xc2>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -536,7 +539,7 @@
clocks = <&mstp3_clks R8A7793_CLK_SDHI2>;
dmas = <&dmac0 0xd3>, <&dmac0 0xd4>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -549,7 +552,7 @@
clock-names = "fck";
dmas = <&dmac0 0x21>, <&dmac0 0x22>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -562,7 +565,7 @@
clock-names = "fck";
dmas = <&dmac0 0x25>, <&dmac0 0x26>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -575,7 +578,7 @@
clock-names = "fck";
dmas = <&dmac0 0x27>, <&dmac0 0x28>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -588,7 +591,7 @@
clock-names = "fck";
dmas = <&dmac0 0x1b>, <&dmac0 0x1c>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -601,7 +604,7 @@
clock-names = "fck";
dmas = <&dmac0 0x1f>, <&dmac0 0x20>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -614,7 +617,7 @@
clock-names = "fck";
dmas = <&dmac0 0x23>, <&dmac0 0x24>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -627,7 +630,7 @@
clock-names = "fck";
dmas = <&dmac0 0x3d>, <&dmac0 0x3e>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -640,7 +643,7 @@
clock-names = "fck";
dmas = <&dmac0 0x19>, <&dmac0 0x1a>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -653,7 +656,7 @@
clock-names = "fck";
dmas = <&dmac0 0x1d>, <&dmac0 0x1e>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -667,7 +670,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x29>, <&dmac0 0x2a>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -681,7 +684,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x2d>, <&dmac0 0x2e>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -695,7 +698,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x2b>, <&dmac0 0x2c>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -709,7 +712,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x2f>, <&dmac0 0x30>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -723,7 +726,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0xfb>, <&dmac0 0xfc>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -737,7 +740,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0xfd>, <&dmac0 0xfe>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -751,7 +754,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x39>, <&dmac0 0x3a>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -765,7 +768,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x4d>, <&dmac0 0x4e>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -779,7 +782,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x3b>, <&dmac0 0x3c>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -788,7 +791,7 @@
reg = <0 0xee700000 0 0x400>;
interrupts = <GIC_SPI 162 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp8_clks R8A7793_CLK_ETHER>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
phy-mode = "rmii";
#address-cells = <1>;
#size-cells = <0>;
@@ -802,7 +805,7 @@
clocks = <&mstp9_clks R8A7793_CLK_QSPI_MOD>;
dmas = <&dmac0 0x17>, <&dmac0 0x18>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
num-cs = <1>;
#address-cells = <1>;
#size-cells = <0>;
@@ -846,7 +849,7 @@
clocks = <&mstp9_clks R8A7793_CLK_RCAN0>,
<&cpg_clocks R8A7793_CLK_RCAN>, <&can_clk>;
clock-names = "clkp1", "clkp2", "can_clk";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -857,7 +860,7 @@
clocks = <&mstp9_clks R8A7793_CLK_RCAN1>,
<&cpg_clocks R8A7793_CLK_RCAN>, <&can_clk>;
clock-names = "clkp1", "clkp2", "can_clk";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -1221,6 +1224,12 @@
};
};
+ sysc: system-controller@e6180000 {
+ compatible = "renesas,r8a7793-sysc";
+ reg = <0 0xe6180000 0 0x0200>;
+ #power-domain-cells = <1>;
+ };
+
ipmmu_sy0: mmu@e6280000 {
compatible = "renesas,ipmmu-r8a7793", "renesas,ipmmu-vmsa";
reg = <0 0xe6280000 0 0x1000>;
@@ -1316,7 +1325,7 @@
"src.4", "src.3", "src.2", "src.1", "src.0",
"dvc.0", "dvc.1",
"clk_a", "clk_b", "clk_c", "clk_i";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7793_PD_ALWAYS_ON>;
status = "disabled";
diff --git a/arch/arm/boot/dts/r8a7794.dtsi b/arch/arm/boot/dts/r8a7794.dtsi
index e45b23f..f334a3a 100644
--- a/arch/arm/boot/dts/r8a7794.dtsi
+++ b/arch/arm/boot/dts/r8a7794.dtsi
@@ -12,6 +12,7 @@
#include <dt-bindings/clock/r8a7794-clock.h>
#include <dt-bindings/interrupt-controller/arm-gic.h>
#include <dt-bindings/interrupt-controller/irq.h>
+#include <dt-bindings/power/r8a7794-sysc.h>
/ {
compatible = "renesas,r8a7794";
@@ -42,6 +43,7 @@
compatible = "arm,cortex-a7";
reg = <0>;
clock-frequency = <1000000000>;
+ power-domains = <&sysc R8A7794_PD_CA7_CPU0>;
next-level-cache = <&L2_CA7>;
};
@@ -50,12 +52,14 @@
compatible = "arm,cortex-a7";
reg = <1>;
clock-frequency = <1000000000>;
+ power-domains = <&sysc R8A7794_PD_CA7_CPU1>;
next-level-cache = <&L2_CA7>;
};
};
L2_CA7: cache-controller@1 {
compatible = "cache";
+ power-domains = <&sysc R8A7794_PD_CA7_SCU>;
cache-unified;
cache-level = <2>;
};
@@ -82,7 +86,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7794_CLK_GPIO0>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
};
gpio1: gpio@e6051000 {
@@ -95,7 +99,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7794_CLK_GPIO1>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
};
gpio2: gpio@e6052000 {
@@ -108,7 +112,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7794_CLK_GPIO2>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
};
gpio3: gpio@e6053000 {
@@ -121,7 +125,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7794_CLK_GPIO3>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
};
gpio4: gpio@e6054000 {
@@ -134,7 +138,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7794_CLK_GPIO4>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
};
gpio5: gpio@e6055000 {
@@ -147,7 +151,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7794_CLK_GPIO5>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
};
gpio6: gpio@e6055400 {
@@ -160,7 +164,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&mstp9_clks R8A7794_CLK_GPIO6>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
};
cmt0: timer@ffca0000 {
@@ -170,7 +174,7 @@
<GIC_SPI 143 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp1_clks R8A7794_CLK_CMT0>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
renesas,channels-mask = <0x60>;
@@ -190,7 +194,7 @@
<GIC_SPI 127 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp3_clks R8A7794_CLK_CMT1>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
renesas,channels-mask = <0xff>;
@@ -221,7 +225,7 @@
<GIC_SPI 16 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 17 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp4_clks R8A7794_CLK_IRQC>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
};
pfc: pin-controller@e6060000 {
@@ -255,7 +259,7 @@
"ch12", "ch13", "ch14";
clocks = <&mstp2_clks R8A7794_CLK_SYS_DMAC0>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <15>;
};
@@ -286,7 +290,7 @@
"ch12", "ch13", "ch14";
clocks = <&mstp2_clks R8A7794_CLK_SYS_DMAC1>;
clock-names = "fck";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <15>;
};
@@ -300,7 +304,7 @@
clock-names = "fck";
dmas = <&dmac0 0x21>, <&dmac0 0x22>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -313,7 +317,7 @@
clock-names = "fck";
dmas = <&dmac0 0x25>, <&dmac0 0x26>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -326,7 +330,7 @@
clock-names = "fck";
dmas = <&dmac0 0x27>, <&dmac0 0x28>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -339,7 +343,7 @@
clock-names = "fck";
dmas = <&dmac0 0x1b>, <&dmac0 0x1c>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -352,7 +356,7 @@
clock-names = "fck";
dmas = <&dmac0 0x1f>, <&dmac0 0x20>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -365,7 +369,7 @@
clock-names = "fck";
dmas = <&dmac0 0x23>, <&dmac0 0x24>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -378,7 +382,7 @@
clock-names = "fck";
dmas = <&dmac0 0x3d>, <&dmac0 0x3e>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -391,7 +395,7 @@
clock-names = "fck";
dmas = <&dmac0 0x19>, <&dmac0 0x1a>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -404,7 +408,7 @@
clock-names = "fck";
dmas = <&dmac0 0x1d>, <&dmac0 0x1e>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -418,7 +422,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x29>, <&dmac0 0x2a>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -432,7 +436,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x2d>, <&dmac0 0x2e>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -446,7 +450,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x2b>, <&dmac0 0x2c>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -460,7 +464,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x2f>, <&dmac0 0x30>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -474,7 +478,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0xfb>, <&dmac0 0xfc>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -488,7 +492,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0xfd>, <&dmac0 0xfe>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -502,7 +506,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x39>, <&dmac0 0x3a>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -516,7 +520,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x4d>, <&dmac0 0x4e>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -530,7 +534,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x3b>, <&dmac0 0x3c>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -539,7 +543,7 @@
reg = <0 0xee700000 0 0x400>;
interrupts = <GIC_SPI 162 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp8_clks R8A7794_CLK_ETHER>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
phy-mode = "rmii";
#address-cells = <1>;
#size-cells = <0>;
@@ -552,7 +556,7 @@
reg = <0 0xe6800000 0 0x800>, <0 0xee0e8000 0 0x4000>;
interrupts = <GIC_SPI 163 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp8_clks R8A7794_CLK_ETHERAVB>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
#address-cells = <1>;
#size-cells = <0>;
status = "disabled";
@@ -564,7 +568,7 @@
reg = <0 0xe6508000 0 0x40>;
interrupts = <GIC_SPI 287 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp9_clks R8A7794_CLK_I2C0>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
#address-cells = <1>;
#size-cells = <0>;
i2c-scl-internal-delay-ns = <6>;
@@ -576,7 +580,7 @@
reg = <0 0xe6518000 0 0x40>;
interrupts = <GIC_SPI 288 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp9_clks R8A7794_CLK_I2C1>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
#address-cells = <1>;
#size-cells = <0>;
i2c-scl-internal-delay-ns = <6>;
@@ -588,7 +592,7 @@
reg = <0 0xe6530000 0 0x40>;
interrupts = <GIC_SPI 286 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp9_clks R8A7794_CLK_I2C2>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
#address-cells = <1>;
#size-cells = <0>;
i2c-scl-internal-delay-ns = <6>;
@@ -600,7 +604,7 @@
reg = <0 0xe6540000 0 0x40>;
interrupts = <GIC_SPI 290 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp9_clks R8A7794_CLK_I2C3>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
#address-cells = <1>;
#size-cells = <0>;
i2c-scl-internal-delay-ns = <6>;
@@ -612,7 +616,7 @@
reg = <0 0xe6520000 0 0x40>;
interrupts = <GIC_SPI 19 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp9_clks R8A7794_CLK_I2C4>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
#address-cells = <1>;
#size-cells = <0>;
i2c-scl-internal-delay-ns = <6>;
@@ -624,7 +628,7 @@
reg = <0 0xe6528000 0 0x40>;
interrupts = <GIC_SPI 20 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp9_clks R8A7794_CLK_I2C5>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
#address-cells = <1>;
#size-cells = <0>;
i2c-scl-internal-delay-ns = <6>;
@@ -638,7 +642,7 @@
clocks = <&mstp3_clks R8A7794_CLK_IIC0>;
dmas = <&dmac0 0x61>, <&dmac0 0x62>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
#address-cells = <1>;
#size-cells = <0>;
status = "disabled";
@@ -651,7 +655,7 @@
clocks = <&mstp3_clks R8A7794_CLK_IIC1>;
dmas = <&dmac0 0x65>, <&dmac0 0x66>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
#address-cells = <1>;
#size-cells = <0>;
status = "disabled";
@@ -664,7 +668,7 @@
clocks = <&mstp3_clks R8A7794_CLK_MMCIF0>;
dmas = <&dmac0 0xd1>, <&dmac0 0xd2>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
reg-io-width = <4>;
status = "disabled";
};
@@ -674,7 +678,7 @@
reg = <0 0xee100000 0 0x200>;
interrupts = <GIC_SPI 165 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp3_clks R8A7794_CLK_SDHI0>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -683,7 +687,7 @@
reg = <0 0xee140000 0 0x100>;
interrupts = <GIC_SPI 167 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp3_clks R8A7794_CLK_SDHI1>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -692,7 +696,7 @@
reg = <0 0xee160000 0 0x100>;
interrupts = <GIC_SPI 168 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp3_clks R8A7794_CLK_SDHI2>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -703,7 +707,7 @@
clocks = <&mstp9_clks R8A7794_CLK_QSPI_MOD>;
dmas = <&dmac0 0x17>, <&dmac0 0x18>;
dma-names = "tx", "rx";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
num-cs = <1>;
#address-cells = <1>;
#size-cells = <0>;
@@ -715,7 +719,7 @@
reg = <0 0xe6ef0000 0 0x1000>;
interrupts = <GIC_SPI 188 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp8_clks R8A7794_CLK_VIN0>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -724,7 +728,7 @@
reg = <0 0xe6ef1000 0 0x1000>;
interrupts = <GIC_SPI 189 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp8_clks R8A7794_CLK_VIN1>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -735,7 +739,7 @@
<0 0xee080000 0 0x1100>;
interrupts = <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp7_clks R8A7794_CLK_EHCI>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
bus-range = <0 0>;
@@ -770,7 +774,7 @@
<0 0xee0c0000 0 0x1100>;
interrupts = <GIC_SPI 113 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp7_clks R8A7794_CLK_EHCI>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
bus-range = <1 1>;
@@ -803,7 +807,7 @@
reg = <0 0xe6590000 0 0x100>;
interrupts = <GIC_SPI 107 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&mstp7_clks R8A7794_CLK_HSUSB>;
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
renesas,buswait = <4>;
phys = <&usb0 1>;
phy-names = "usb";
@@ -817,7 +821,7 @@
#size-cells = <0>;
clocks = <&mstp7_clks R8A7794_CLK_HSUSB>;
clock-names = "usbhs";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
usb0: usb-channel@0 {
@@ -865,7 +869,7 @@
clocks = <&mstp9_clks R8A7794_CLK_RCAN0>,
<&cpg_clocks R8A7794_CLK_RCAN>, <&can_clk>;
clock-names = "clkp1", "clkp2", "can_clk";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -876,7 +880,7 @@
clocks = <&mstp9_clks R8A7794_CLK_RCAN1>,
<&cpg_clocks R8A7794_CLK_RCAN>, <&can_clk>;
clock-names = "clkp1", "clkp2", "can_clk";
- power-domains = <&cpg_clocks>;
+ power-domains = <&sysc R8A7794_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -1213,6 +1217,12 @@
};
};
+ sysc: system-controller@e6180000 {
+ compatible = "renesas,r8a7794-sysc";
+ reg = <0 0xe6180000 0 0x0200>;
+ #power-domain-cells = <1>;
+ };
+
ipmmu_sy0: mmu@e6280000 {
compatible = "renesas,ipmmu-r8a7794", "renesas,ipmmu-vmsa";
reg = <0 0xe6280000 0 0x1000>;
diff --git a/arch/arm/boot/dts/spear13xx.dtsi b/arch/arm/boot/dts/spear13xx.dtsi
index 14594ce..449acf0 100644
--- a/arch/arm/boot/dts/spear13xx.dtsi
+++ b/arch/arm/boot/dts/spear13xx.dtsi
@@ -117,7 +117,7 @@
chan_priority = <1>;
block_size = <0xfff>;
dma-masters = <2>;
- data_width = <3 3>;
+ data-width = <8 8>;
};
dma@eb000000 {
@@ -133,7 +133,7 @@
chan_allocation_order = <1>;
chan_priority = <1>;
block_size = <0xfff>;
- data_width = <3 3>;
+ data-width = <8 8>;
};
fsmc: flash@b0000000 {
diff --git a/arch/arm/boot/dts/tegra124-jetson-tk1.dts b/arch/arm/boot/dts/tegra124-jetson-tk1.dts
index a99f07a..941f362 100644
--- a/arch/arm/boot/dts/tegra124-jetson-tk1.dts
+++ b/arch/arm/boot/dts/tegra124-jetson-tk1.dts
@@ -38,11 +38,17 @@
vddio-pex-ctl-supply = <&vdd_3v3_lp0>;
avdd-pll-erefe-supply = <&avdd_1v05_run>;
+ /* Mini PCIe */
pci@1,0 {
+ phys = <&{/padctl@0,7009f000/pads/pcie/lanes/pcie-4}>;
+ phy-names = "pcie-0";
status = "okay";
};
+ /* Gigabit Ethernet */
pci@2,0 {
+ phys = <&{/padctl@0,7009f000/pads/pcie/lanes/pcie-2}>;
+ phy-names = "pcie-0";
status = "okay";
};
};
@@ -1677,6 +1683,9 @@
sata@0,70020000 {
status = "okay";
+ phys = <&{/padctl@0,7009f000/pads/sata/lanes/sata-0}>;
+ phy-names = "sata-0";
+
hvdd-supply = <&vdd_3v3_lp0>;
vddio-supply = <&vdd_1v05_run>;
avdd-supply = <&vdd_1v05_run>;
@@ -1689,28 +1698,107 @@
status = "okay";
};
- padctl@0,7009f000 {
- pinctrl-0 = <&padctl_default>;
- pinctrl-names = "default";
+ usb@0,70090000 {
+ phys = <&{/padctl@0,7009f000/pads/usb2/lanes/usb2-0}>, /* Micro A/B */
+ <&{/padctl@0,7009f000/pads/usb2/lanes/usb2-1}>, /* Mini PCIe */
+ <&{/padctl@0,7009f000/pads/usb2/lanes/usb2-2}>, /* USB3 */
+ <&{/padctl@0,7009f000/pads/pcie/lanes/pcie-0}>; /* USB3 */
+ phy-names = "usb2-0", "usb2-1", "usb2-2", "usb3-0";
- padctl_default: pinmux {
- usb3 {
- nvidia,lanes = "pcie-0", "pcie-1";
- nvidia,function = "usb3";
- nvidia,iddq = <0>;
+ avddio-pex-supply = <&vdd_1v05_run>;
+ dvddio-pex-supply = <&vdd_1v05_run>;
+ avdd-usb-supply = <&vdd_3v3_lp0>;
+ avdd-pll-utmip-supply = <&vddio_1v8>;
+ avdd-pll-erefe-supply = <&avdd_1v05_run>;
+ avdd-usb-ss-pll-supply = <&vdd_1v05_run>;
+ hvdd-usb-ss-supply = <&vdd_3v3_lp0>;
+ hvdd-usb-ss-pll-e-supply = <&vdd_3v3_lp0>;
+
+ status = "okay";
+ };
+
+ padctl@0,7009f000 {
+ status = "okay";
+
+ pads {
+ usb2 {
+ status = "okay";
+
+ lanes {
+ usb2-0 {
+ nvidia,function = "xusb";
+ status = "okay";
+ };
+
+ usb2-1 {
+ nvidia,function = "xusb";
+ status = "okay";
+ };
+
+ usb2-2 {
+ nvidia,function = "xusb";
+ status = "okay";
+ };
+ };
};
pcie {
- nvidia,lanes = "pcie-2", "pcie-3",
- "pcie-4";
- nvidia,function = "pcie";
- nvidia,iddq = <0>;
+ status = "okay";
+
+ lanes {
+ pcie-0 {
+ nvidia,function = "usb3-ss";
+ status = "okay";
+ };
+
+ pcie-2 {
+ nvidia,function = "pcie";
+ status = "okay";
+ };
+
+ pcie-4 {
+ nvidia,function = "pcie";
+ status = "okay";
+ };
+ };
};
sata {
- nvidia,lanes = "sata-0";
- nvidia,function = "sata";
- nvidia,iddq = <0>;
+ status = "okay";
+
+ lanes {
+ sata-0 {
+ nvidia,function = "sata";
+ status = "okay";
+ };
+ };
+ };
+ };
+
+ ports {
+ /* Micro A/B */
+ usb2-0 {
+ status = "okay";
+ mode = "otg";
+ };
+
+ /* Mini PCIe */
+ usb2-1 {
+ status = "okay";
+ mode = "host";
+ };
+
+ /* USB3 */
+ usb2-2 {
+ status = "okay";
+ mode = "host";
+
+ vbus-supply = <&vdd_usb3_vbus>;
+ };
+
+ usb3-0 {
+ nvidia,usb2-companion = <2>;
+ status = "okay";
};
};
};
diff --git a/arch/arm/boot/dts/tegra124-nyan.dtsi b/arch/arm/boot/dts/tegra124-nyan.dtsi
index 5f1fc14..0710a60 100644
--- a/arch/arm/boot/dts/tegra124-nyan.dtsi
+++ b/arch/arm/boot/dts/tegra124-nyan.dtsi
@@ -224,7 +224,7 @@
regulator-always-on;
};
- ldo0 {
+ avdd_1v05_run: ldo0 {
regulator-name = "+1.05V_RUN_AVDD";
regulator-min-microvolt = <1050000>;
regulator-max-microvolt = <1050000>;
@@ -368,6 +368,99 @@
status = "okay";
};
+ usb@0,70090000 {
+ phys = <&{/padctl@0,7009f000/pads/usb2/lanes/usb2-0}>, /* 1st USB A */
+ <&{/padctl@0,7009f000/pads/usb2/lanes/usb2-1}>, /* Internal USB */
+ <&{/padctl@0,7009f000/pads/usb2/lanes/usb2-2}>, /* 2nd USB A */
+ <&{/padctl@0,7009f000/pads/pcie/lanes/pcie-0}>, /* 1st USB A */
+ <&{/padctl@0,7009f000/pads/pcie/lanes/pcie-1}>; /* 2nd USB A */
+ phy-names = "usb2-0", "usb2-1", "usb2-2", "usb3-0", "usb3-1";
+
+ avddio-pex-supply = <&vdd_1v05_run>;
+ dvddio-pex-supply = <&vdd_1v05_run>;
+ avdd-usb-supply = <&vdd_3v3_lp0>;
+ avdd-pll-utmip-supply = <&vddio_1v8>;
+ avdd-pll-erefe-supply = <&avdd_1v05_run>;
+ avdd-usb-ss-pll-supply = <&vdd_1v05_run>;
+ hvdd-usb-ss-supply = <&vdd_3v3_lp0>;
+ hvdd-usb-ss-pll-e-supply = <&vdd_3v3_lp0>;
+
+ status = "okay";
+ };
+
+ padctl@0,7009f000 {
+ status = "okay";
+
+ pads {
+ usb2 {
+ status = "okay";
+
+ lanes {
+ usb2-0 {
+ nvidia,function = "xusb";
+ status = "okay";
+ };
+
+ usb2-1 {
+ nvidia,function = "xusb";
+ status = "okay";
+ };
+
+ usb2-2 {
+ nvidia,function = "xusb";
+ status = "okay";
+ };
+ };
+ };
+
+ pcie {
+ status = "okay";
+
+ lanes {
+ pcie-0 {
+ nvidia,function = "usb3-ss";
+ status = "okay";
+ };
+
+ pcie-1 {
+ nvidia,function = "usb3-ss";
+ status = "okay";
+ };
+ };
+ };
+ };
+
+ ports {
+ usb2-0 {
+ vbus-supply = <&vdd_usb1_vbus>;
+ status = "okay";
+ mode = "otg";
+ };
+
+ usb2-1 {
+ vbus-supply = <&vdd_run_cam>;
+ status = "okay";
+ mode = "host";
+ };
+
+ usb2-2 {
+ vbus-supply = <&vdd_usb3_vbus>;
+ status = "okay";
+ mode = "host";
+ };
+
+ usb3-0 {
+ nvidia,usb2-companion = <0>;
+ status = "okay";
+ };
+
+ usb3-1 {
+ nvidia,usb2-companion = <1>;
+ status = "okay";
+ };
+ };
+ };
+
sdhci0_pwrseq: sdhci0_pwrseq {
compatible = "mmc-pwrseq-simple";
@@ -414,33 +507,6 @@
};
};
- usb@0,7d000000 { /* Rear external USB port. */
- status = "okay";
- };
-
- usb-phy@0,7d000000 {
- status = "okay";
- vbus-supply = <&vdd_usb1_vbus>;
- };
-
- usb@0,7d004000 { /* Internal webcam. */
- status = "okay";
- };
-
- usb-phy@0,7d004000 {
- status = "okay";
- vbus-supply = <&vdd_run_cam>;
- };
-
- usb@0,7d008000 { /* Left external USB port. */
- status = "okay";
- };
-
- usb-phy@0,7d008000 {
- status = "okay";
- vbus-supply = <&vdd_usb3_vbus>;
- };
-
backlight: backlight {
compatible = "pwm-backlight";
diff --git a/arch/arm/boot/dts/tegra124-venice2.dts b/arch/arm/boot/dts/tegra124-venice2.dts
index 0318258..973446d 100644
--- a/arch/arm/boot/dts/tegra124-venice2.dts
+++ b/arch/arm/boot/dts/tegra124-venice2.dts
@@ -757,7 +757,7 @@
regulator-always-on;
};
- ldo0 {
+ avdd_1v05_run: ldo0 {
regulator-name = "+1.05V_RUN_AVDD";
regulator-min-microvolt = <1050000>;
regulator-max-microvolt = <1050000>;
@@ -899,6 +899,105 @@
status = "okay";
};
+ usb@0,70090000 {
+ phys = <&{/padctl@0,7009f000/pads/usb2/lanes/usb2-0}>, /* 1st USB A */
+ <&{/padctl@0,7009f000/pads/usb2/lanes/usb2-1}>, /* Internal USB */
+ <&{/padctl@0,7009f000/pads/usb2/lanes/usb2-2}>, /* 2nd USB A */
+ <&{/padctl@0,7009f000/pads/pcie/lanes/pcie-0}>, /* 1st USB A */
+ <&{/padctl@0,7009f000/pads/pcie/lanes/pcie-1}>; /* 2nd USB A */
+ phy-names = "usb2-0", "usb2-1", "usb2-2", "usb3-0", "usb3-1";
+
+ avddio-pex-supply = <&vdd_1v05_run>;
+ dvddio-pex-supply = <&vdd_1v05_run>;
+ avdd-usb-supply = <&vdd_3v3_lp0>;
+ avdd-pll-utmip-supply = <&vddio_1v8>;
+ avdd-pll-erefe-supply = <&avdd_1v05_run>;
+ avdd-usb-ss-pll-supply = <&vdd_1v05_run>;
+ hvdd-usb-ss-supply = <&vdd_3v3_lp0>;
+ hvdd-usb-ss-pll-e-supply = <&vdd_3v3_lp0>;
+
+ status = "okay";
+ };
+
+ padctl@0,7009f000 {
+ pads {
+ usb2 {
+ status = "okay";
+
+ lanes {
+ usb2-0 {
+ nvidia,function = "xusb";
+ status = "okay";
+ };
+
+ usb2-1 {
+ nvidia,function = "xusb";
+ status = "okay";
+ };
+
+ usb2-2 {
+ nvidia,function = "xusb";
+ status = "okay";
+ };
+ };
+ };
+
+ pcie {
+ status = "okay";
+
+ lanes {
+ pcie-0 {
+ nvidia,function = "usb3-ss";
+ status = "okay";
+ };
+
+ pcie-1 {
+ nvidia,function = "usb3-ss";
+ status = "okay";
+ };
+
+ pcie-1 {
+ nvidia,function = "usb3-ss";
+ status = "okay";
+ };
+ };
+ };
+ };
+
+ ports {
+ usb2-0 {
+ status = "okay";
+ mode = "otg";
+
+ vbus-supply = <&vdd_usb1_vbus>;
+ };
+
+ usb2-1 {
+ status = "okay";
+ mode = "host";
+
+ vbus-supply = <&vdd_run_cam>;
+ };
+
+ usb2-2 {
+ status = "okay";
+ mode = "host";
+
+ vbus-supply = <&vdd_usb3_vbus>;
+ };
+
+ usb3-0 {
+ nvidia,usb2-companion = <0>;
+ status = "okay";
+ };
+
+ usb3-1 {
+ nvidia,usb2-companion = <2>;
+ status = "okay";
+ };
+ };
+ };
+
sdhci@0,700b0400 {
cd-gpios = <&gpio TEGRA_GPIO(V, 2) GPIO_ACTIVE_HIGH>;
power-gpios = <&gpio TEGRA_GPIO(R, 0) GPIO_ACTIVE_HIGH>;
diff --git a/arch/arm/boot/dts/tegra124.dtsi b/arch/arm/boot/dts/tegra124.dtsi
index e4eac1f..ea48118 100644
--- a/arch/arm/boot/dts/tegra124.dtsi
+++ b/arch/arm/boot/dts/tegra124.dtsi
@@ -2,7 +2,6 @@
#include <dt-bindings/gpio/tegra-gpio.h>
#include <dt-bindings/memory/tegra124-mc.h>
#include <dt-bindings/pinctrl/pinctrl-tegra.h>
-#include <dt-bindings/pinctrl/pinctrl-tegra-xusb.h>
#include <dt-bindings/interrupt-controller/arm-gic.h>
#include <dt-bindings/reset/tegra124-car.h>
#include <dt-bindings/thermal/tegra124-soctherm.h>
@@ -51,9 +50,6 @@
reset-names = "pex", "afi", "pcie_x";
status = "disabled";
- phys = <&padctl TEGRA_XUSB_PADCTL_PCIE>;
- phy-names = "pcie";
-
pci@1,0 {
device_type = "pci";
assigned-addresses = <0x82000800 0 0x01000000 0 0x1000>;
@@ -622,8 +618,6 @@
<&tegra_car 123>,
<&tegra_car 129>;
reset-names = "sata", "sata-oob", "sata-cold";
- phys = <&padctl TEGRA_XUSB_PADCTL_SATA>;
- phy-names = "sata-phy";
status = "disabled";
};
@@ -642,13 +636,172 @@
status = "disabled";
};
+ usb@0,70090000 {
+ compatible = "nvidia,tegra124-xusb";
+ reg = <0x0 0x70090000 0x0 0x8000>,
+ <0x0 0x70098000 0x0 0x1000>,
+ <0x0 0x70099000 0x0 0x1000>;
+ reg-names = "hcd", "fpci", "ipfs";
+
+ interrupts = <GIC_SPI 39 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 40 IRQ_TYPE_LEVEL_HIGH>;
+
+ clocks = <&tegra_car TEGRA124_CLK_XUSB_HOST>,
+ <&tegra_car TEGRA124_CLK_XUSB_HOST_SRC>,
+ <&tegra_car TEGRA124_CLK_XUSB_FALCON_SRC>,
+ <&tegra_car TEGRA124_CLK_XUSB_SS>,
+ <&tegra_car TEGRA124_CLK_XUSB_SS_DIV2>,
+ <&tegra_car TEGRA124_CLK_XUSB_SS_SRC>,
+ <&tegra_car TEGRA124_CLK_XUSB_HS_SRC>,
+ <&tegra_car TEGRA124_CLK_XUSB_FS_SRC>,
+ <&tegra_car TEGRA124_CLK_PLL_U_480M>,
+ <&tegra_car TEGRA124_CLK_CLK_M>,
+ <&tegra_car TEGRA124_CLK_PLL_E>;
+ clock-names = "xusb_host", "xusb_host_src",
+ "xusb_falcon_src", "xusb_ss",
+ "xusb_ss_div2", "xusb_ss_src",
+ "xusb_hs_src", "xusb_fs_src",
+ "pll_u_480m", "clk_m", "pll_e";
+ resets = <&tegra_car 89>, <&tegra_car 156>,
+ <&tegra_car 143>;
+ reset-names = "xusb_host", "xusb_ss", "xusb_src";
+
+ nvidia,xusb-padctl = <&padctl>;
+
+ status = "disabled";
+ };
+
padctl: padctl@0,7009f000 {
compatible = "nvidia,tegra124-xusb-padctl";
reg = <0x0 0x7009f000 0x0 0x1000>;
resets = <&tegra_car 142>;
reset-names = "padctl";
- #phy-cells = <1>;
+ pads {
+ usb2 {
+ status = "disabled";
+
+ lanes {
+ usb2-0 {
+ status = "disabled";
+ #phy-cells = <0>;
+ };
+
+ usb2-1 {
+ status = "disabled";
+ #phy-cells = <0>;
+ };
+
+ usb2-2 {
+ status = "disabled";
+ #phy-cells = <0>;
+ };
+ };
+ };
+
+ ulpi {
+ status = "disabled";
+
+ lanes {
+ ulpi-0 {
+ status = "disabled";
+ #phy-cells = <0>;
+ };
+ };
+ };
+
+ hsic {
+ status = "disabled";
+
+ lanes {
+ hsic-0 {
+ status = "disabled";
+ #phy-cells = <0>;
+ };
+
+ hsic-1 {
+ status = "disabled";
+ #phy-cells = <0>;
+ };
+ };
+ };
+
+ pcie {
+ status = "disabled";
+
+ lanes {
+ pcie-0 {
+ status = "disabled";
+ #phy-cells = <0>;
+ };
+
+ pcie-1 {
+ status = "disabled";
+ #phy-cells = <0>;
+ };
+
+ pcie-2 {
+ status = "disabled";
+ #phy-cells = <0>;
+ };
+
+ pcie-3 {
+ status = "disabled";
+ #phy-cells = <0>;
+ };
+
+ pcie-4 {
+ status = "disabled";
+ #phy-cells = <0>;
+ };
+ };
+ };
+
+ sata {
+ status = "disabled";
+
+ lanes {
+ sata-0 {
+ status = "disabled";
+ #phy-cells = <0>;
+ };
+ };
+ };
+ };
+
+ ports {
+ usb2-0 {
+ status = "disabled";
+ };
+
+ usb2-1 {
+ status = "disabled";
+ };
+
+ usb2-2 {
+ status = "disabled";
+ };
+
+ ulpi-0 {
+ status = "disabled";
+ };
+
+ hsic-0 {
+ status = "disabled";
+ };
+
+ hsic-1 {
+ status = "disabled";
+ };
+
+ usb3-0 {
+ status = "disabled";
+ };
+
+ usb3-1 {
+ status = "disabled";
+ };
+ };
};
sdhci@0,700b0000 {
diff --git a/arch/arm/boot/dts/vf-colibri-eval-v3.dtsi b/arch/arm/boot/dts/vf-colibri-eval-v3.dtsi
index 4d8b7f6..a8a8e43 100644
--- a/arch/arm/boot/dts/vf-colibri-eval-v3.dtsi
+++ b/arch/arm/boot/dts/vf-colibri-eval-v3.dtsi
@@ -50,6 +50,11 @@
clock-frequency = <16000000>;
};
+ panel: panel {
+ compatible = "edt,et057090dhu";
+ backlight = <&bl>;
+ };
+
reg_3v3: regulator-3v3 {
compatible = "regulator-fixed";
regulator-name = "3.3V";
@@ -83,6 +88,13 @@
status = "okay";
};
+&dcu0 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_dcu0_1>;
+ fsl,panel = <&panel>;
+ status = "okay";
+};
+
&dspi1 {
status = "okay";
@@ -134,6 +146,10 @@
vin-supply = <®_3v3>;
};
+&tcon0 {
+ status = "okay";
+};
+
&uart0 {
status = "okay";
};
diff --git a/arch/arm/boot/dts/vf-colibri.dtsi b/arch/arm/boot/dts/vf-colibri.dtsi
index 226a86f..b741709 100644
--- a/arch/arm/boot/dts/vf-colibri.dtsi
+++ b/arch/arm/boot/dts/vf-colibri.dtsi
@@ -222,6 +222,39 @@
>;
};
+ pinctrl_dcu0_1: dcu0grp_1 {
+ fsl,pins = <
+ VF610_PAD_PTE0__DCU0_HSYNC 0x1902
+ VF610_PAD_PTE1__DCU0_VSYNC 0x1902
+ VF610_PAD_PTE2__DCU0_PCLK 0x1902
+ VF610_PAD_PTE4__DCU0_DE 0x1902
+ VF610_PAD_PTE5__DCU0_R0 0x1902
+ VF610_PAD_PTE6__DCU0_R1 0x1902
+ VF610_PAD_PTE7__DCU0_R2 0x1902
+ VF610_PAD_PTE8__DCU0_R3 0x1902
+ VF610_PAD_PTE9__DCU0_R4 0x1902
+ VF610_PAD_PTE10__DCU0_R5 0x1902
+ VF610_PAD_PTE11__DCU0_R6 0x1902
+ VF610_PAD_PTE12__DCU0_R7 0x1902
+ VF610_PAD_PTE13__DCU0_G0 0x1902
+ VF610_PAD_PTE14__DCU0_G1 0x1902
+ VF610_PAD_PTE15__DCU0_G2 0x1902
+ VF610_PAD_PTE16__DCU0_G3 0x1902
+ VF610_PAD_PTE17__DCU0_G4 0x1902
+ VF610_PAD_PTE18__DCU0_G5 0x1902
+ VF610_PAD_PTE19__DCU0_G6 0x1902
+ VF610_PAD_PTE20__DCU0_G7 0x1902
+ VF610_PAD_PTE21__DCU0_B0 0x1902
+ VF610_PAD_PTE22__DCU0_B1 0x1902
+ VF610_PAD_PTE23__DCU0_B2 0x1902
+ VF610_PAD_PTE24__DCU0_B3 0x1902
+ VF610_PAD_PTE25__DCU0_B4 0x1902
+ VF610_PAD_PTE26__DCU0_B5 0x1902
+ VF610_PAD_PTE27__DCU0_B6 0x1902
+ VF610_PAD_PTE28__DCU0_B7 0x1902
+ >;
+ };
+
pinctrl_dspi1: dspi1grp {
fsl,pins = <
VF610_PAD_PTD5__DSPI1_CS0 0x33e2
diff --git a/arch/arm/boot/dts/vfxxx.dtsi b/arch/arm/boot/dts/vfxxx.dtsi
index 04ef54d..2c13ec6 100644
--- a/arch/arm/boot/dts/vfxxx.dtsi
+++ b/arch/arm/boot/dts/vfxxx.dtsi
@@ -311,6 +311,14 @@
<20000000>;
};
+ tcon0: timing-controller@4003d000 {
+ compatible = "fsl,vf610-tcon";
+ reg = <0x4003d000 0x1000>;
+ clocks = <&clks VF610_CLK_TCON0>;
+ clock-names = "ipg";
+ status = "disabled";
+ };
+
wdoga5: wdog@4003e000 {
compatible = "fsl,vf610-wdt", "fsl,imx21-wdt";
reg = <0x4003e000 0x1000>;
@@ -416,6 +424,17 @@
status = "disabled";
};
+ dcu0: dcu@40058000 {
+ compatible = "fsl,vf610-dcu";
+ reg = <0x40058000 0x1200>;
+ interrupts = <30 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&clks VF610_CLK_DCU0>,
+ <&clks VF610_CLK_DCU0_DIV>;
+ clock-names = "dcu", "pix";
+ fsl,tcon = <&tcon0>;
+ status = "disabled";
+ };
+
i2c0: i2c@40066000 {
#address-cells = <1>;
#size-cells = <0>;
diff --git a/arch/arm/configs/bcm_defconfig b/arch/arm/configs/bcm_defconfig
index 7117662..909049a2 100644
--- a/arch/arm/configs/bcm_defconfig
+++ b/arch/arm/configs/bcm_defconfig
@@ -12,7 +12,6 @@
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CGROUP_CPUACCT=y
-CONFIG_RESOURCE_COUNTERS=y
CONFIG_CGROUP_SCHED=y
CONFIG_BLK_CGROUP=y
CONFIG_NAMESPACES=y
diff --git a/arch/arm/configs/zx_defconfig b/arch/arm/configs/zx_defconfig
index ab683fb..d6253a4 100644
--- a/arch/arm/configs/zx_defconfig
+++ b/arch/arm/configs/zx_defconfig
@@ -7,7 +7,6 @@
CONFIG_CGROUP_DEBUG=y
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_CPUACCT=y
-CONFIG_RESOURCE_COUNTERS=y
CONFIG_CGROUP_SCHED=y
CONFIG_RT_GROUP_SCHED=y
CONFIG_NAMESPACES=y
diff --git a/arch/arm/include/asm/dma-mapping.h b/arch/arm/include/asm/dma-mapping.h
index 6ad1ced..a83570f 100644
--- a/arch/arm/include/asm/dma-mapping.h
+++ b/arch/arm/include/asm/dma-mapping.h
@@ -118,7 +118,7 @@
#define arch_setup_dma_ops arch_setup_dma_ops
extern void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
- struct iommu_ops *iommu, bool coherent);
+ const struct iommu_ops *iommu, bool coherent);
#define arch_teardown_dma_ops arch_teardown_dma_ops
extern void arch_teardown_dma_ops(struct device *dev);
@@ -162,8 +162,6 @@
static inline void dma_mark_clean(void *addr, size_t size) { }
-extern int arm_dma_set_mask(struct device *dev, u64 dma_mask);
-
/**
* arm_dma_alloc - allocate consistent memory for DMA
* @dev: valid struct device pointer, or NULL for ISA and EISA-like devices
diff --git a/arch/arm/include/asm/io.h b/arch/arm/include/asm/io.h
index 4859820..781ef5f 100644
--- a/arch/arm/include/asm/io.h
+++ b/arch/arm/include/asm/io.h
@@ -392,9 +392,18 @@
#define ioremap ioremap
#define ioremap_nocache ioremap
+/*
+ * Do not use ioremap_cache for mapping memory. Use memremap instead.
+ */
void __iomem *ioremap_cache(resource_size_t res_cookie, size_t size);
#define ioremap_cache ioremap_cache
+/*
+ * Do not use ioremap_cached in new code. Provided for the benefit of
+ * the pxa2xx-flash MTD driver only.
+ */
+void __iomem *ioremap_cached(resource_size_t res_cookie, size_t size);
+
void __iomem *ioremap_wc(resource_size_t res_cookie, size_t size);
#define ioremap_wc ioremap_wc
#define ioremap_wt ioremap_wc
@@ -402,6 +411,9 @@
void iounmap(volatile void __iomem *iomem_cookie);
#define iounmap iounmap
+void *arch_memremap_wb(phys_addr_t phys_addr, size_t size);
+#define arch_memremap_wb arch_memremap_wb
+
/*
* io{read,write}{16,32}be() macros
*/
diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
index 738d5ee..0df6b1f 100644
--- a/arch/arm/include/asm/kvm_host.h
+++ b/arch/arm/include/asm/kvm_host.h
@@ -187,6 +187,7 @@
struct kvm_vcpu_stat {
u32 halt_successful_poll;
u32 halt_attempted_poll;
+ u32 halt_poll_invalid;
u32 halt_wakeup;
u32 hvc_exit_stat;
u64 wfe_exit_stat;
@@ -290,6 +291,7 @@
static inline void kvm_arch_sync_events(struct kvm *kvm) {}
static inline void kvm_arch_vcpu_uninit(struct kvm_vcpu *vcpu) {}
static inline void kvm_arch_sched_in(struct kvm_vcpu *vcpu, int cpu) {}
+static inline void kvm_arch_vcpu_block_finish(struct kvm_vcpu *vcpu) {}
static inline void kvm_arm_init_debug(void) {}
static inline void kvm_arm_setup_debug(struct kvm_vcpu *vcpu) {}
diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
index f17a8d4..f9a6506 100644
--- a/arch/arm/include/asm/kvm_mmu.h
+++ b/arch/arm/include/asm/kvm_mmu.h
@@ -47,6 +47,7 @@
#include <linux/highmem.h>
#include <asm/cacheflush.h>
#include <asm/pgalloc.h>
+#include <asm/stage2_pgtable.h>
int create_hyp_mappings(void *from, void *to);
int create_hyp_io_mappings(void *from, void *to, phys_addr_t);
@@ -106,14 +107,16 @@
clean_pte_table(pte);
}
-static inline void kvm_set_s2pte_writable(pte_t *pte)
+static inline pte_t kvm_s2pte_mkwrite(pte_t pte)
{
- pte_val(*pte) |= L_PTE_S2_RDWR;
+ pte_val(pte) |= L_PTE_S2_RDWR;
+ return pte;
}
-static inline void kvm_set_s2pmd_writable(pmd_t *pmd)
+static inline pmd_t kvm_s2pmd_mkwrite(pmd_t pmd)
{
- pmd_val(*pmd) |= L_PMD_S2_RDWR;
+ pmd_val(pmd) |= L_PMD_S2_RDWR;
+ return pmd;
}
static inline void kvm_set_s2pte_readonly(pte_t *pte)
@@ -136,22 +139,6 @@
return (pmd_val(*pmd) & L_PMD_S2_RDWR) == L_PMD_S2_RDONLY;
}
-
-/* Open coded p*d_addr_end that can deal with 64bit addresses */
-#define kvm_pgd_addr_end(addr, end) \
-({ u64 __boundary = ((addr) + PGDIR_SIZE) & PGDIR_MASK; \
- (__boundary - 1 < (end) - 1)? __boundary: (end); \
-})
-
-#define kvm_pud_addr_end(addr,end) (end)
-
-#define kvm_pmd_addr_end(addr, end) \
-({ u64 __boundary = ((addr) + PMD_SIZE) & PMD_MASK; \
- (__boundary - 1 < (end) - 1)? __boundary: (end); \
-})
-
-#define kvm_pgd_index(addr) pgd_index(addr)
-
static inline bool kvm_page_empty(void *ptr)
{
struct page *ptr_page = virt_to_page(ptr);
@@ -160,19 +147,11 @@
#define kvm_pte_table_empty(kvm, ptep) kvm_page_empty(ptep)
#define kvm_pmd_table_empty(kvm, pmdp) kvm_page_empty(pmdp)
-#define kvm_pud_table_empty(kvm, pudp) (0)
+#define kvm_pud_table_empty(kvm, pudp) false
-#define KVM_PREALLOC_LEVEL 0
-
-static inline void *kvm_get_hwpgd(struct kvm *kvm)
-{
- return kvm->arch.pgd;
-}
-
-static inline unsigned int kvm_get_hwpgd_size(void)
-{
- return PTRS_PER_S2_PGD * sizeof(pgd_t);
-}
+#define hyp_pte_table_empty(ptep) kvm_page_empty(ptep)
+#define hyp_pmd_table_empty(pmdp) kvm_page_empty(pmdp)
+#define hyp_pud_table_empty(pudp) false
struct kvm;
diff --git a/arch/arm/include/asm/memory.h b/arch/arm/include/asm/memory.h
index 9427fd6..31c07a2 100644
--- a/arch/arm/include/asm/memory.h
+++ b/arch/arm/include/asm/memory.h
@@ -288,19 +288,43 @@
#define __va(x) ((void *)__phys_to_virt((phys_addr_t)(x)))
#define pfn_to_kaddr(pfn) __va((phys_addr_t)(pfn) << PAGE_SHIFT)
-extern unsigned long (*arch_virt_to_idmap)(unsigned long x);
+extern long long arch_phys_to_idmap_offset;
/*
- * These are for systems that have a hardware interconnect supported alias of
- * physical memory for idmap purposes. Most cases should leave these
+ * These are for systems that have a hardware interconnect supported alias
+ * of physical memory for idmap purposes. Most cases should leave these
* untouched. Note: this can only return addresses less than 4GiB.
*/
+static inline bool arm_has_idmap_alias(void)
+{
+ return IS_ENABLED(CONFIG_MMU) && arch_phys_to_idmap_offset != 0;
+}
+
+#define IDMAP_INVALID_ADDR ((u32)~0)
+
+static inline unsigned long phys_to_idmap(phys_addr_t addr)
+{
+ if (IS_ENABLED(CONFIG_MMU) && arch_phys_to_idmap_offset) {
+ addr += arch_phys_to_idmap_offset;
+ if (addr > (u32)~0)
+ addr = IDMAP_INVALID_ADDR;
+ }
+ return addr;
+}
+
+static inline phys_addr_t idmap_to_phys(unsigned long idmap)
+{
+ phys_addr_t addr = idmap;
+
+ if (IS_ENABLED(CONFIG_MMU) && arch_phys_to_idmap_offset)
+ addr -= arch_phys_to_idmap_offset;
+
+ return addr;
+}
+
static inline unsigned long __virt_to_idmap(unsigned long x)
{
- if (IS_ENABLED(CONFIG_MMU) && arch_virt_to_idmap)
- return arch_virt_to_idmap(x);
- else
- return __virt_to_phys(x);
+ return phys_to_idmap(__virt_to_phys(x));
}
#define virt_to_idmap(x) __virt_to_idmap((unsigned long)(x))
diff --git a/arch/arm/include/asm/pgtable-3level.h b/arch/arm/include/asm/pgtable-3level.h
index dc46398..fa70db7 100644
--- a/arch/arm/include/asm/pgtable-3level.h
+++ b/arch/arm/include/asm/pgtable-3level.h
@@ -281,11 +281,6 @@
flush_pmd_entry(pmdp);
}
-static inline int has_transparent_hugepage(void)
-{
- return 1;
-}
-
#endif /* __ASSEMBLY__ */
#endif /* _ASM_PGTABLE_3LEVEL_H */
diff --git a/arch/arm/include/asm/stage2_pgtable.h b/arch/arm/include/asm/stage2_pgtable.h
new file mode 100644
index 0000000..460d616
--- /dev/null
+++ b/arch/arm/include/asm/stage2_pgtable.h
@@ -0,0 +1,61 @@
+/*
+ * Copyright (C) 2016 - ARM Ltd
+ *
+ * stage2 page table helpers
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __ARM_S2_PGTABLE_H_
+#define __ARM_S2_PGTABLE_H_
+
+#define stage2_pgd_none(pgd) pgd_none(pgd)
+#define stage2_pgd_clear(pgd) pgd_clear(pgd)
+#define stage2_pgd_present(pgd) pgd_present(pgd)
+#define stage2_pgd_populate(pgd, pud) pgd_populate(NULL, pgd, pud)
+#define stage2_pud_offset(pgd, address) pud_offset(pgd, address)
+#define stage2_pud_free(pud) pud_free(NULL, pud)
+
+#define stage2_pud_none(pud) pud_none(pud)
+#define stage2_pud_clear(pud) pud_clear(pud)
+#define stage2_pud_present(pud) pud_present(pud)
+#define stage2_pud_populate(pud, pmd) pud_populate(NULL, pud, pmd)
+#define stage2_pmd_offset(pud, address) pmd_offset(pud, address)
+#define stage2_pmd_free(pmd) pmd_free(NULL, pmd)
+
+#define stage2_pud_huge(pud) pud_huge(pud)
+
+/* Open coded p*d_addr_end that can deal with 64bit addresses */
+static inline phys_addr_t stage2_pgd_addr_end(phys_addr_t addr, phys_addr_t end)
+{
+ phys_addr_t boundary = (addr + PGDIR_SIZE) & PGDIR_MASK;
+
+ return (boundary - 1 < end - 1) ? boundary : end;
+}
+
+#define stage2_pud_addr_end(addr, end) (end)
+
+static inline phys_addr_t stage2_pmd_addr_end(phys_addr_t addr, phys_addr_t end)
+{
+ phys_addr_t boundary = (addr + PMD_SIZE) & PMD_MASK;
+
+ return (boundary - 1 < end - 1) ? boundary : end;
+}
+
+#define stage2_pgd_index(addr) pgd_index(addr)
+
+#define stage2_pte_table_empty(ptep) kvm_page_empty(ptep)
+#define stage2_pmd_table_empty(pmdp) kvm_page_empty(pmdp)
+#define stage2_pud_table_empty(pudp) false
+
+#endif /* __ARM_S2_PGTABLE_H_ */
diff --git a/arch/arm/kernel/bios32.c b/arch/arm/kernel/bios32.c
index 066f7f9..05e61a2 100644
--- a/arch/arm/kernel/bios32.c
+++ b/arch/arm/kernel/bios32.c
@@ -550,9 +550,6 @@
if (!strcmp(str, "debug")) {
debug_pci = 1;
return NULL;
- } else if (!strcmp(str, "firmware")) {
- pci_add_flags(PCI_PROBE_ONLY);
- return NULL;
}
return str;
}
diff --git a/arch/arm/kernel/process.c b/arch/arm/kernel/process.c
index 4adfb46..4a803c5 100644
--- a/arch/arm/kernel/process.c
+++ b/arch/arm/kernel/process.c
@@ -193,9 +193,9 @@
/*
* Free current thread data structures etc..
*/
-void exit_thread(void)
+void exit_thread(struct task_struct *tsk)
{
- thread_notify(THREAD_NOTIFY_EXIT, current_thread_info());
+ thread_notify(THREAD_NOTIFY_EXIT, task_thread_info(tsk));
}
void flush_thread(void)
@@ -420,7 +420,8 @@
npages = 1; /* for sigpage */
npages += vdso_total_pages;
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
hint = sigpage_addr(mm, npages);
addr = get_unmapped_area(NULL, hint, npages << PAGE_SHIFT, 0, 0);
if (IS_ERR_VALUE(addr)) {
diff --git a/arch/arm/kernel/reboot.c b/arch/arm/kernel/reboot.c
index 71a2ff9..3fa867a 100644
--- a/arch/arm/kernel/reboot.c
+++ b/arch/arm/kernel/reboot.c
@@ -104,8 +104,6 @@
{
local_irq_disable();
smp_send_stop();
-
- local_irq_disable();
while (1);
}
@@ -150,6 +148,5 @@
/* Whoops - the platform was unable to reboot. Tell the user! */
printk("Reboot failed -- System halted\n");
- local_irq_disable();
while (1);
}
diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
index 7d4e285..7b53500 100644
--- a/arch/arm/kernel/setup.c
+++ b/arch/arm/kernel/setup.c
@@ -941,6 +941,12 @@
late_initcall(init_machine_late);
#ifdef CONFIG_KEXEC
+/*
+ * The crash region must be aligned to 128MB to avoid
+ * zImage relocating below the reserved region.
+ */
+#define CRASH_ALIGN (128 << 20)
+
static inline unsigned long long get_total_mem(void)
{
unsigned long total;
@@ -968,6 +974,26 @@
if (ret)
return;
+ if (crash_base <= 0) {
+ unsigned long long crash_max = idmap_to_phys((u32)~0);
+ crash_base = memblock_find_in_range(CRASH_ALIGN, crash_max,
+ crash_size, CRASH_ALIGN);
+ if (!crash_base) {
+ pr_err("crashkernel reservation failed - No suitable area found.\n");
+ return;
+ }
+ } else {
+ unsigned long long start;
+
+ start = memblock_find_in_range(crash_base,
+ crash_base + crash_size,
+ crash_size, SECTION_SIZE);
+ if (start != crash_base) {
+ pr_err("crashkernel reservation failed - memory is in use.\n");
+ return;
+ }
+ }
+
ret = memblock_reserve(crash_base, crash_size);
if (ret < 0) {
pr_warn("crashkernel reservation failed - memory is in use (0x%lx)\n",
diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
index baee702..df90bc5 100644
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -644,9 +644,11 @@
break;
case IPI_CPU_BACKTRACE:
+ printk_nmi_enter();
irq_enter();
nmi_cpu_backtrace(regs);
irq_exit();
+ printk_nmi_exit();
break;
default:
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index 9ef013d..237d5d8 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -444,7 +444,7 @@
kvm_next_vmid &= (1 << kvm_vmid_bits) - 1;
/* update vttbr to be used with the new vmid */
- pgd_phys = virt_to_phys(kvm_get_hwpgd(kvm));
+ pgd_phys = virt_to_phys(kvm->arch.pgd);
BUG_ON(pgd_phys & ~VTTBR_BADDR_MASK);
vmid = ((u64)(kvm->arch.vmid) << VTTBR_VMID_SHIFT) & VTTBR_VMID_MASK(kvm_vmid_bits);
kvm->arch.vttbr = pgd_phys | vmid;
diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
index be30212..45c43ae 100644
--- a/arch/arm/kvm/mmu.c
+++ b/arch/arm/kvm/mmu.c
@@ -43,11 +43,9 @@
static unsigned long hyp_idmap_end;
static phys_addr_t hyp_idmap_vector;
+#define S2_PGD_SIZE (PTRS_PER_S2_PGD * sizeof(pgd_t))
#define hyp_pgd_order get_order(PTRS_PER_PGD * sizeof(pgd_t))
-#define kvm_pmd_huge(_x) (pmd_huge(_x) || pmd_trans_huge(_x))
-#define kvm_pud_huge(_x) pud_huge(_x)
-
#define KVM_S2PTE_FLAG_IS_IOMAP (1UL << 0)
#define KVM_S2_FLAG_LOGGING_ACTIVE (1UL << 1)
@@ -69,14 +67,7 @@
static void kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa)
{
- /*
- * This function also gets called when dealing with HYP page
- * tables. As HYP doesn't have an associated struct kvm (and
- * the HYP page tables are fairly static), we don't do
- * anything there.
- */
- if (kvm)
- kvm_call_hyp(__kvm_tlb_flush_vmid_ipa, kvm, ipa);
+ kvm_call_hyp(__kvm_tlb_flush_vmid_ipa, kvm, ipa);
}
/*
@@ -115,7 +106,7 @@
*/
static void stage2_dissolve_pmd(struct kvm *kvm, phys_addr_t addr, pmd_t *pmd)
{
- if (!kvm_pmd_huge(*pmd))
+ if (!pmd_thp_or_huge(*pmd))
return;
pmd_clear(pmd);
@@ -155,29 +146,29 @@
return p;
}
-static void clear_pgd_entry(struct kvm *kvm, pgd_t *pgd, phys_addr_t addr)
+static void clear_stage2_pgd_entry(struct kvm *kvm, pgd_t *pgd, phys_addr_t addr)
{
- pud_t *pud_table __maybe_unused = pud_offset(pgd, 0);
- pgd_clear(pgd);
+ pud_t *pud_table __maybe_unused = stage2_pud_offset(pgd, 0UL);
+ stage2_pgd_clear(pgd);
kvm_tlb_flush_vmid_ipa(kvm, addr);
- pud_free(NULL, pud_table);
+ stage2_pud_free(pud_table);
put_page(virt_to_page(pgd));
}
-static void clear_pud_entry(struct kvm *kvm, pud_t *pud, phys_addr_t addr)
+static void clear_stage2_pud_entry(struct kvm *kvm, pud_t *pud, phys_addr_t addr)
{
- pmd_t *pmd_table = pmd_offset(pud, 0);
- VM_BUG_ON(pud_huge(*pud));
- pud_clear(pud);
+ pmd_t *pmd_table __maybe_unused = stage2_pmd_offset(pud, 0);
+ VM_BUG_ON(stage2_pud_huge(*pud));
+ stage2_pud_clear(pud);
kvm_tlb_flush_vmid_ipa(kvm, addr);
- pmd_free(NULL, pmd_table);
+ stage2_pmd_free(pmd_table);
put_page(virt_to_page(pud));
}
-static void clear_pmd_entry(struct kvm *kvm, pmd_t *pmd, phys_addr_t addr)
+static void clear_stage2_pmd_entry(struct kvm *kvm, pmd_t *pmd, phys_addr_t addr)
{
pte_t *pte_table = pte_offset_kernel(pmd, 0);
- VM_BUG_ON(kvm_pmd_huge(*pmd));
+ VM_BUG_ON(pmd_thp_or_huge(*pmd));
pmd_clear(pmd);
kvm_tlb_flush_vmid_ipa(kvm, addr);
pte_free_kernel(NULL, pte_table);
@@ -204,7 +195,7 @@
* the corresponding TLBs, we call kvm_flush_dcache_p*() to make sure
* the IO subsystem will never hit in the cache.
*/
-static void unmap_ptes(struct kvm *kvm, pmd_t *pmd,
+static void unmap_stage2_ptes(struct kvm *kvm, pmd_t *pmd,
phys_addr_t addr, phys_addr_t end)
{
phys_addr_t start_addr = addr;
@@ -226,21 +217,21 @@
}
} while (pte++, addr += PAGE_SIZE, addr != end);
- if (kvm_pte_table_empty(kvm, start_pte))
- clear_pmd_entry(kvm, pmd, start_addr);
+ if (stage2_pte_table_empty(start_pte))
+ clear_stage2_pmd_entry(kvm, pmd, start_addr);
}
-static void unmap_pmds(struct kvm *kvm, pud_t *pud,
+static void unmap_stage2_pmds(struct kvm *kvm, pud_t *pud,
phys_addr_t addr, phys_addr_t end)
{
phys_addr_t next, start_addr = addr;
pmd_t *pmd, *start_pmd;
- start_pmd = pmd = pmd_offset(pud, addr);
+ start_pmd = pmd = stage2_pmd_offset(pud, addr);
do {
- next = kvm_pmd_addr_end(addr, end);
+ next = stage2_pmd_addr_end(addr, end);
if (!pmd_none(*pmd)) {
- if (kvm_pmd_huge(*pmd)) {
+ if (pmd_thp_or_huge(*pmd)) {
pmd_t old_pmd = *pmd;
pmd_clear(pmd);
@@ -250,57 +241,64 @@
put_page(virt_to_page(pmd));
} else {
- unmap_ptes(kvm, pmd, addr, next);
+ unmap_stage2_ptes(kvm, pmd, addr, next);
}
}
} while (pmd++, addr = next, addr != end);
- if (kvm_pmd_table_empty(kvm, start_pmd))
- clear_pud_entry(kvm, pud, start_addr);
+ if (stage2_pmd_table_empty(start_pmd))
+ clear_stage2_pud_entry(kvm, pud, start_addr);
}
-static void unmap_puds(struct kvm *kvm, pgd_t *pgd,
+static void unmap_stage2_puds(struct kvm *kvm, pgd_t *pgd,
phys_addr_t addr, phys_addr_t end)
{
phys_addr_t next, start_addr = addr;
pud_t *pud, *start_pud;
- start_pud = pud = pud_offset(pgd, addr);
+ start_pud = pud = stage2_pud_offset(pgd, addr);
do {
- next = kvm_pud_addr_end(addr, end);
- if (!pud_none(*pud)) {
- if (pud_huge(*pud)) {
+ next = stage2_pud_addr_end(addr, end);
+ if (!stage2_pud_none(*pud)) {
+ if (stage2_pud_huge(*pud)) {
pud_t old_pud = *pud;
- pud_clear(pud);
+ stage2_pud_clear(pud);
kvm_tlb_flush_vmid_ipa(kvm, addr);
-
kvm_flush_dcache_pud(old_pud);
-
put_page(virt_to_page(pud));
} else {
- unmap_pmds(kvm, pud, addr, next);
+ unmap_stage2_pmds(kvm, pud, addr, next);
}
}
} while (pud++, addr = next, addr != end);
- if (kvm_pud_table_empty(kvm, start_pud))
- clear_pgd_entry(kvm, pgd, start_addr);
+ if (stage2_pud_table_empty(start_pud))
+ clear_stage2_pgd_entry(kvm, pgd, start_addr);
}
-
-static void unmap_range(struct kvm *kvm, pgd_t *pgdp,
- phys_addr_t start, u64 size)
+/**
+ * unmap_stage2_range -- Clear stage2 page table entries to unmap a range
+ * @kvm: The VM pointer
+ * @start: The intermediate physical base address of the range to unmap
+ * @size: The size of the area to unmap
+ *
+ * Clear a range of stage-2 mappings, lowering the various ref-counts. Must
+ * be called while holding mmu_lock (unless for freeing the stage2 pgd before
+ * destroying the VM), otherwise another faulting VCPU may come in and mess
+ * with things behind our backs.
+ */
+static void unmap_stage2_range(struct kvm *kvm, phys_addr_t start, u64 size)
{
pgd_t *pgd;
phys_addr_t addr = start, end = start + size;
phys_addr_t next;
- pgd = pgdp + kvm_pgd_index(addr);
+ pgd = kvm->arch.pgd + stage2_pgd_index(addr);
do {
- next = kvm_pgd_addr_end(addr, end);
- if (!pgd_none(*pgd))
- unmap_puds(kvm, pgd, addr, next);
+ next = stage2_pgd_addr_end(addr, end);
+ if (!stage2_pgd_none(*pgd))
+ unmap_stage2_puds(kvm, pgd, addr, next);
} while (pgd++, addr = next, addr != end);
}
@@ -322,11 +320,11 @@
pmd_t *pmd;
phys_addr_t next;
- pmd = pmd_offset(pud, addr);
+ pmd = stage2_pmd_offset(pud, addr);
do {
- next = kvm_pmd_addr_end(addr, end);
+ next = stage2_pmd_addr_end(addr, end);
if (!pmd_none(*pmd)) {
- if (kvm_pmd_huge(*pmd))
+ if (pmd_thp_or_huge(*pmd))
kvm_flush_dcache_pmd(*pmd);
else
stage2_flush_ptes(kvm, pmd, addr, next);
@@ -340,11 +338,11 @@
pud_t *pud;
phys_addr_t next;
- pud = pud_offset(pgd, addr);
+ pud = stage2_pud_offset(pgd, addr);
do {
- next = kvm_pud_addr_end(addr, end);
- if (!pud_none(*pud)) {
- if (pud_huge(*pud))
+ next = stage2_pud_addr_end(addr, end);
+ if (!stage2_pud_none(*pud)) {
+ if (stage2_pud_huge(*pud))
kvm_flush_dcache_pud(*pud);
else
stage2_flush_pmds(kvm, pud, addr, next);
@@ -360,9 +358,9 @@
phys_addr_t next;
pgd_t *pgd;
- pgd = kvm->arch.pgd + kvm_pgd_index(addr);
+ pgd = kvm->arch.pgd + stage2_pgd_index(addr);
do {
- next = kvm_pgd_addr_end(addr, end);
+ next = stage2_pgd_addr_end(addr, end);
stage2_flush_puds(kvm, pgd, addr, next);
} while (pgd++, addr = next, addr != end);
}
@@ -391,6 +389,100 @@
srcu_read_unlock(&kvm->srcu, idx);
}
+static void clear_hyp_pgd_entry(pgd_t *pgd)
+{
+ pud_t *pud_table __maybe_unused = pud_offset(pgd, 0UL);
+ pgd_clear(pgd);
+ pud_free(NULL, pud_table);
+ put_page(virt_to_page(pgd));
+}
+
+static void clear_hyp_pud_entry(pud_t *pud)
+{
+ pmd_t *pmd_table __maybe_unused = pmd_offset(pud, 0);
+ VM_BUG_ON(pud_huge(*pud));
+ pud_clear(pud);
+ pmd_free(NULL, pmd_table);
+ put_page(virt_to_page(pud));
+}
+
+static void clear_hyp_pmd_entry(pmd_t *pmd)
+{
+ pte_t *pte_table = pte_offset_kernel(pmd, 0);
+ VM_BUG_ON(pmd_thp_or_huge(*pmd));
+ pmd_clear(pmd);
+ pte_free_kernel(NULL, pte_table);
+ put_page(virt_to_page(pmd));
+}
+
+static void unmap_hyp_ptes(pmd_t *pmd, phys_addr_t addr, phys_addr_t end)
+{
+ pte_t *pte, *start_pte;
+
+ start_pte = pte = pte_offset_kernel(pmd, addr);
+ do {
+ if (!pte_none(*pte)) {
+ kvm_set_pte(pte, __pte(0));
+ put_page(virt_to_page(pte));
+ }
+ } while (pte++, addr += PAGE_SIZE, addr != end);
+
+ if (hyp_pte_table_empty(start_pte))
+ clear_hyp_pmd_entry(pmd);
+}
+
+static void unmap_hyp_pmds(pud_t *pud, phys_addr_t addr, phys_addr_t end)
+{
+ phys_addr_t next;
+ pmd_t *pmd, *start_pmd;
+
+ start_pmd = pmd = pmd_offset(pud, addr);
+ do {
+ next = pmd_addr_end(addr, end);
+ /* Hyp doesn't use huge pmds */
+ if (!pmd_none(*pmd))
+ unmap_hyp_ptes(pmd, addr, next);
+ } while (pmd++, addr = next, addr != end);
+
+ if (hyp_pmd_table_empty(start_pmd))
+ clear_hyp_pud_entry(pud);
+}
+
+static void unmap_hyp_puds(pgd_t *pgd, phys_addr_t addr, phys_addr_t end)
+{
+ phys_addr_t next;
+ pud_t *pud, *start_pud;
+
+ start_pud = pud = pud_offset(pgd, addr);
+ do {
+ next = pud_addr_end(addr, end);
+ /* Hyp doesn't use huge puds */
+ if (!pud_none(*pud))
+ unmap_hyp_pmds(pud, addr, next);
+ } while (pud++, addr = next, addr != end);
+
+ if (hyp_pud_table_empty(start_pud))
+ clear_hyp_pgd_entry(pgd);
+}
+
+static void unmap_hyp_range(pgd_t *pgdp, phys_addr_t start, u64 size)
+{
+ pgd_t *pgd;
+ phys_addr_t addr = start, end = start + size;
+ phys_addr_t next;
+
+ /*
+ * We don't unmap anything from HYP, except at the hyp tear down.
+ * Hence, we don't have to invalidate the TLBs here.
+ */
+ pgd = pgdp + pgd_index(addr);
+ do {
+ next = pgd_addr_end(addr, end);
+ if (!pgd_none(*pgd))
+ unmap_hyp_puds(pgd, addr, next);
+ } while (pgd++, addr = next, addr != end);
+}
+
/**
* free_boot_hyp_pgd - free HYP boot page tables
*
@@ -401,14 +493,14 @@
mutex_lock(&kvm_hyp_pgd_mutex);
if (boot_hyp_pgd) {
- unmap_range(NULL, boot_hyp_pgd, hyp_idmap_start, PAGE_SIZE);
- unmap_range(NULL, boot_hyp_pgd, TRAMPOLINE_VA, PAGE_SIZE);
+ unmap_hyp_range(boot_hyp_pgd, hyp_idmap_start, PAGE_SIZE);
+ unmap_hyp_range(boot_hyp_pgd, TRAMPOLINE_VA, PAGE_SIZE);
free_pages((unsigned long)boot_hyp_pgd, hyp_pgd_order);
boot_hyp_pgd = NULL;
}
if (hyp_pgd)
- unmap_range(NULL, hyp_pgd, TRAMPOLINE_VA, PAGE_SIZE);
+ unmap_hyp_range(hyp_pgd, TRAMPOLINE_VA, PAGE_SIZE);
mutex_unlock(&kvm_hyp_pgd_mutex);
}
@@ -433,9 +525,9 @@
if (hyp_pgd) {
for (addr = PAGE_OFFSET; virt_addr_valid(addr); addr += PGDIR_SIZE)
- unmap_range(NULL, hyp_pgd, KERN_TO_HYP(addr), PGDIR_SIZE);
+ unmap_hyp_range(hyp_pgd, KERN_TO_HYP(addr), PGDIR_SIZE);
for (addr = VMALLOC_START; is_vmalloc_addr((void*)addr); addr += PGDIR_SIZE)
- unmap_range(NULL, hyp_pgd, KERN_TO_HYP(addr), PGDIR_SIZE);
+ unmap_hyp_range(hyp_pgd, KERN_TO_HYP(addr), PGDIR_SIZE);
free_pages((unsigned long)hyp_pgd, hyp_pgd_order);
hyp_pgd = NULL;
@@ -645,20 +737,6 @@
__phys_to_pfn(phys_addr), PAGE_HYP_DEVICE);
}
-/* Free the HW pgd, one page at a time */
-static void kvm_free_hwpgd(void *hwpgd)
-{
- free_pages_exact(hwpgd, kvm_get_hwpgd_size());
-}
-
-/* Allocate the HW PGD, making sure that each page gets its own refcount */
-static void *kvm_alloc_hwpgd(void)
-{
- unsigned int size = kvm_get_hwpgd_size();
-
- return alloc_pages_exact(size, GFP_KERNEL | __GFP_ZERO);
-}
-
/**
* kvm_alloc_stage2_pgd - allocate level-1 table for stage-2 translation.
* @kvm: The KVM struct pointer for the VM.
@@ -673,81 +751,22 @@
int kvm_alloc_stage2_pgd(struct kvm *kvm)
{
pgd_t *pgd;
- void *hwpgd;
if (kvm->arch.pgd != NULL) {
kvm_err("kvm_arch already initialized?\n");
return -EINVAL;
}
- hwpgd = kvm_alloc_hwpgd();
- if (!hwpgd)
+ /* Allocate the HW PGD, making sure that each page gets its own refcount */
+ pgd = alloc_pages_exact(S2_PGD_SIZE, GFP_KERNEL | __GFP_ZERO);
+ if (!pgd)
return -ENOMEM;
- /* When the kernel uses more levels of page tables than the
- * guest, we allocate a fake PGD and pre-populate it to point
- * to the next-level page table, which will be the real
- * initial page table pointed to by the VTTBR.
- *
- * When KVM_PREALLOC_LEVEL==2, we allocate a single page for
- * the PMD and the kernel will use folded pud.
- * When KVM_PREALLOC_LEVEL==1, we allocate 2 consecutive PUD
- * pages.
- */
- if (KVM_PREALLOC_LEVEL > 0) {
- int i;
-
- /*
- * Allocate fake pgd for the page table manipulation macros to
- * work. This is not used by the hardware and we have no
- * alignment requirement for this allocation.
- */
- pgd = kmalloc(PTRS_PER_S2_PGD * sizeof(pgd_t),
- GFP_KERNEL | __GFP_ZERO);
-
- if (!pgd) {
- kvm_free_hwpgd(hwpgd);
- return -ENOMEM;
- }
-
- /* Plug the HW PGD into the fake one. */
- for (i = 0; i < PTRS_PER_S2_PGD; i++) {
- if (KVM_PREALLOC_LEVEL == 1)
- pgd_populate(NULL, pgd + i,
- (pud_t *)hwpgd + i * PTRS_PER_PUD);
- else if (KVM_PREALLOC_LEVEL == 2)
- pud_populate(NULL, pud_offset(pgd, 0) + i,
- (pmd_t *)hwpgd + i * PTRS_PER_PMD);
- }
- } else {
- /*
- * Allocate actual first-level Stage-2 page table used by the
- * hardware for Stage-2 page table walks.
- */
- pgd = (pgd_t *)hwpgd;
- }
-
kvm_clean_pgd(pgd);
kvm->arch.pgd = pgd;
return 0;
}
-/**
- * unmap_stage2_range -- Clear stage2 page table entries to unmap a range
- * @kvm: The VM pointer
- * @start: The intermediate physical base address of the range to unmap
- * @size: The size of the area to unmap
- *
- * Clear a range of stage-2 mappings, lowering the various ref-counts. Must
- * be called while holding mmu_lock (unless for freeing the stage2 pgd before
- * destroying the VM), otherwise another faulting VCPU may come in and mess
- * with things behind our backs.
- */
-static void unmap_stage2_range(struct kvm *kvm, phys_addr_t start, u64 size)
-{
- unmap_range(kvm, kvm->arch.pgd, start, size);
-}
-
static void stage2_unmap_memslot(struct kvm *kvm,
struct kvm_memory_slot *memslot)
{
@@ -830,10 +849,8 @@
return;
unmap_stage2_range(kvm, 0, KVM_PHYS_SIZE);
- kvm_free_hwpgd(kvm_get_hwpgd(kvm));
- if (KVM_PREALLOC_LEVEL > 0)
- kfree(kvm->arch.pgd);
-
+ /* Free the HW pgd, one page at a time */
+ free_pages_exact(kvm->arch.pgd, S2_PGD_SIZE);
kvm->arch.pgd = NULL;
}
@@ -843,16 +860,16 @@
pgd_t *pgd;
pud_t *pud;
- pgd = kvm->arch.pgd + kvm_pgd_index(addr);
- if (WARN_ON(pgd_none(*pgd))) {
+ pgd = kvm->arch.pgd + stage2_pgd_index(addr);
+ if (WARN_ON(stage2_pgd_none(*pgd))) {
if (!cache)
return NULL;
pud = mmu_memory_cache_alloc(cache);
- pgd_populate(NULL, pgd, pud);
+ stage2_pgd_populate(pgd, pud);
get_page(virt_to_page(pgd));
}
- return pud_offset(pgd, addr);
+ return stage2_pud_offset(pgd, addr);
}
static pmd_t *stage2_get_pmd(struct kvm *kvm, struct kvm_mmu_memory_cache *cache,
@@ -862,15 +879,15 @@
pmd_t *pmd;
pud = stage2_get_pud(kvm, cache, addr);
- if (pud_none(*pud)) {
+ if (stage2_pud_none(*pud)) {
if (!cache)
return NULL;
pmd = mmu_memory_cache_alloc(cache);
- pud_populate(NULL, pud, pmd);
+ stage2_pud_populate(pud, pmd);
get_page(virt_to_page(pud));
}
- return pmd_offset(pud, addr);
+ return stage2_pmd_offset(pud, addr);
}
static int stage2_set_pmd_huge(struct kvm *kvm, struct kvm_mmu_memory_cache
@@ -893,11 +910,14 @@
VM_BUG_ON(pmd_present(*pmd) && pmd_pfn(*pmd) != pmd_pfn(*new_pmd));
old_pmd = *pmd;
- kvm_set_pmd(pmd, *new_pmd);
- if (pmd_present(old_pmd))
+ if (pmd_present(old_pmd)) {
+ pmd_clear(pmd);
kvm_tlb_flush_vmid_ipa(kvm, addr);
- else
+ } else {
get_page(virt_to_page(pmd));
+ }
+
+ kvm_set_pmd(pmd, *new_pmd);
return 0;
}
@@ -946,15 +966,38 @@
/* Create 2nd stage page table mapping - Level 3 */
old_pte = *pte;
- kvm_set_pte(pte, *new_pte);
- if (pte_present(old_pte))
+ if (pte_present(old_pte)) {
+ kvm_set_pte(pte, __pte(0));
kvm_tlb_flush_vmid_ipa(kvm, addr);
- else
+ } else {
get_page(virt_to_page(pte));
+ }
+ kvm_set_pte(pte, *new_pte);
return 0;
}
+#ifndef __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG
+static int stage2_ptep_test_and_clear_young(pte_t *pte)
+{
+ if (pte_young(*pte)) {
+ *pte = pte_mkold(*pte);
+ return 1;
+ }
+ return 0;
+}
+#else
+static int stage2_ptep_test_and_clear_young(pte_t *pte)
+{
+ return __ptep_test_and_clear_young(pte);
+}
+#endif
+
+static int stage2_pmdp_test_and_clear_young(pmd_t *pmd)
+{
+ return stage2_ptep_test_and_clear_young((pte_t *)pmd);
+}
+
/**
* kvm_phys_addr_ioremap - map a device range to guest IPA
*
@@ -978,7 +1021,7 @@
pte_t pte = pfn_pte(pfn, PAGE_S2_DEVICE);
if (writable)
- kvm_set_s2pte_writable(&pte);
+ pte = kvm_s2pte_mkwrite(pte);
ret = mmu_topup_memory_cache(&cache, KVM_MMU_CACHE_MIN_PAGES,
KVM_NR_MEM_OBJS);
@@ -1078,12 +1121,12 @@
pmd_t *pmd;
phys_addr_t next;
- pmd = pmd_offset(pud, addr);
+ pmd = stage2_pmd_offset(pud, addr);
do {
- next = kvm_pmd_addr_end(addr, end);
+ next = stage2_pmd_addr_end(addr, end);
if (!pmd_none(*pmd)) {
- if (kvm_pmd_huge(*pmd)) {
+ if (pmd_thp_or_huge(*pmd)) {
if (!kvm_s2pmd_readonly(pmd))
kvm_set_s2pmd_readonly(pmd);
} else {
@@ -1106,12 +1149,12 @@
pud_t *pud;
phys_addr_t next;
- pud = pud_offset(pgd, addr);
+ pud = stage2_pud_offset(pgd, addr);
do {
- next = kvm_pud_addr_end(addr, end);
- if (!pud_none(*pud)) {
+ next = stage2_pud_addr_end(addr, end);
+ if (!stage2_pud_none(*pud)) {
/* TODO:PUD not supported, revisit later if supported */
- BUG_ON(kvm_pud_huge(*pud));
+ BUG_ON(stage2_pud_huge(*pud));
stage2_wp_pmds(pud, addr, next);
}
} while (pud++, addr = next, addr != end);
@@ -1128,7 +1171,7 @@
pgd_t *pgd;
phys_addr_t next;
- pgd = kvm->arch.pgd + kvm_pgd_index(addr);
+ pgd = kvm->arch.pgd + stage2_pgd_index(addr);
do {
/*
* Release kvm_mmu_lock periodically if the memory region is
@@ -1140,8 +1183,8 @@
if (need_resched() || spin_needbreak(&kvm->mmu_lock))
cond_resched_lock(&kvm->mmu_lock);
- next = kvm_pgd_addr_end(addr, end);
- if (pgd_present(*pgd))
+ next = stage2_pgd_addr_end(addr, end);
+ if (stage2_pgd_present(*pgd))
stage2_wp_puds(pgd, addr, next);
} while (pgd++, addr = next, addr != end);
}
@@ -1320,7 +1363,7 @@
pmd_t new_pmd = pfn_pmd(pfn, mem_type);
new_pmd = pmd_mkhuge(new_pmd);
if (writable) {
- kvm_set_s2pmd_writable(&new_pmd);
+ new_pmd = kvm_s2pmd_mkwrite(new_pmd);
kvm_set_pfn_dirty(pfn);
}
coherent_cache_guest_page(vcpu, pfn, PMD_SIZE, fault_ipa_uncached);
@@ -1329,7 +1372,7 @@
pte_t new_pte = pfn_pte(pfn, mem_type);
if (writable) {
- kvm_set_s2pte_writable(&new_pte);
+ new_pte = kvm_s2pte_mkwrite(new_pte);
kvm_set_pfn_dirty(pfn);
mark_page_dirty(kvm, gfn);
}
@@ -1348,6 +1391,8 @@
* Resolve the access fault by making the page young again.
* Note that because the faulting entry is guaranteed not to be
* cached in the TLB, we don't need to invalidate anything.
+ * Only the HW Access Flag updates are supported for Stage 2 (no DBM),
+ * so there is no need for atomic (pte|pmd)_mkyoung operations.
*/
static void handle_access_fault(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa)
{
@@ -1364,7 +1409,7 @@
if (!pmd || pmd_none(*pmd)) /* Nothing there */
goto out;
- if (kvm_pmd_huge(*pmd)) { /* THP, HugeTLB */
+ if (pmd_thp_or_huge(*pmd)) { /* THP, HugeTLB */
*pmd = pmd_mkyoung(*pmd);
pfn = pmd_pfn(*pmd);
pfn_valid = true;
@@ -1588,25 +1633,14 @@
if (!pmd || pmd_none(*pmd)) /* Nothing there */
return 0;
- if (kvm_pmd_huge(*pmd)) { /* THP, HugeTLB */
- if (pmd_young(*pmd)) {
- *pmd = pmd_mkold(*pmd);
- return 1;
- }
-
- return 0;
- }
+ if (pmd_thp_or_huge(*pmd)) /* THP, HugeTLB */
+ return stage2_pmdp_test_and_clear_young(pmd);
pte = pte_offset_kernel(pmd, gpa);
if (pte_none(*pte))
return 0;
- if (pte_young(*pte)) {
- *pte = pte_mkold(*pte); /* Just a page... */
- return 1;
- }
-
- return 0;
+ return stage2_ptep_test_and_clear_young(pte);
}
static int kvm_test_age_hva_handler(struct kvm *kvm, gpa_t gpa, void *data)
@@ -1618,7 +1652,7 @@
if (!pmd || pmd_none(*pmd)) /* Nothing there */
return 0;
- if (kvm_pmd_huge(*pmd)) /* THP, HugeTLB */
+ if (pmd_thp_or_huge(*pmd)) /* THP, HugeTLB */
return pmd_young(*pmd);
pte = pte_offset_kernel(pmd, gpa);
diff --git a/arch/arm/mach-keystone/keystone.c b/arch/arm/mach-keystone/keystone.c
index e6b9cb1..a33a296 100644
--- a/arch/arm/mach-keystone/keystone.c
+++ b/arch/arm/mach-keystone/keystone.c
@@ -63,11 +63,6 @@
of_platform_populate(NULL, of_default_bus_match_table, NULL, NULL);
}
-static unsigned long keystone_virt_to_idmap(unsigned long x)
-{
- return (phys_addr_t)(x) - CONFIG_PAGE_OFFSET + KEYSTONE_LOW_PHYS_START;
-}
-
static long long __init keystone_pv_fixup(void)
{
long long offset;
@@ -91,7 +86,7 @@
offset = KEYSTONE_HIGH_PHYS_START - KEYSTONE_LOW_PHYS_START;
/* Populate the arch idmap hook */
- arch_virt_to_idmap = keystone_virt_to_idmap;
+ arch_phys_to_idmap_offset = -offset;
return offset;
}
diff --git a/arch/arm/mach-omap2/gpmc-nand.c b/arch/arm/mach-omap2/gpmc-nand.c
index 72918c4..f6ac027 100644
--- a/arch/arm/mach-omap2/gpmc-nand.c
+++ b/arch/arm/mach-omap2/gpmc-nand.c
@@ -97,10 +97,7 @@
gpmc_nand_res[2].start = gpmc_get_client_irq(GPMC_IRQ_COUNT_EVENT);
memset(&s, 0, sizeof(struct gpmc_settings));
- if (gpmc_nand_data->of_node)
- gpmc_read_settings_dt(gpmc_nand_data->of_node, &s);
- else
- gpmc_set_legacy(gpmc_nand_data, &s);
+ gpmc_set_legacy(gpmc_nand_data, &s);
s.device_nand = true;
@@ -121,8 +118,6 @@
if (err < 0)
goto out_free_cs;
- gpmc_update_nand_reg(&gpmc_nand_data->reg, gpmc_nand_data->cs);
-
if (!gpmc_hwecc_bch_capable(gpmc_nand_data->ecc_opt)) {
pr_err("omap2-nand: Unsupported NAND ECC scheme selected\n");
err = -EINVAL;
diff --git a/arch/arm/mach-pxa/spitz.c b/arch/arm/mach-pxa/spitz.c
index d9578bc..bd7cd8b 100644
--- a/arch/arm/mach-pxa/spitz.c
+++ b/arch/arm/mach-pxa/spitz.c
@@ -763,14 +763,49 @@
.pattern = scan_ff_pattern
};
-static struct nand_ecclayout akita_oobinfo = {
- .oobfree = { {0x08, 0x09} },
- .eccbytes = 24,
- .eccpos = {
- 0x05, 0x01, 0x02, 0x03, 0x06, 0x07, 0x15, 0x11,
- 0x12, 0x13, 0x16, 0x17, 0x25, 0x21, 0x22, 0x23,
- 0x26, 0x27, 0x35, 0x31, 0x32, 0x33, 0x36, 0x37,
- },
+static int akita_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section > 12)
+ return -ERANGE;
+
+ switch (section % 3) {
+ case 0:
+ oobregion->offset = 5;
+ oobregion->length = 1;
+ break;
+
+ case 1:
+ oobregion->offset = 1;
+ oobregion->length = 3;
+ break;
+
+ case 2:
+ oobregion->offset = 6;
+ oobregion->length = 2;
+ break;
+ }
+
+ oobregion->offset += (section / 3) * 0x10;
+
+ return 0;
+}
+
+static int akita_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section)
+ return -ERANGE;
+
+ oobregion->offset = 8;
+ oobregion->length = 9;
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops akita_ooblayout_ops = {
+ .ecc = akita_ooblayout_ecc,
+ .free = akita_ooblayout_free,
};
static struct sharpsl_nand_platform_data spitz_nand_pdata = {
@@ -804,11 +839,11 @@
} else if (machine_is_akita()) {
spitz_nand_partitions[1].size = 58 * 1024 * 1024;
spitz_nand_bbt.len = 1;
- spitz_nand_pdata.ecc_layout = &akita_oobinfo;
+ spitz_nand_pdata.ecc_layout = &akita_ooblayout_ops;
} else if (machine_is_borzoi()) {
spitz_nand_partitions[1].size = 32 * 1024 * 1024;
spitz_nand_bbt.len = 1;
- spitz_nand_pdata.ecc_layout = &akita_oobinfo;
+ spitz_nand_pdata.ecc_layout = &akita_ooblayout_ops;
}
platform_device_register(&spitz_nand_device);
diff --git a/arch/arm/mm/Kconfig b/arch/arm/mm/Kconfig
index 5534766..cb569b6 100644
--- a/arch/arm/mm/Kconfig
+++ b/arch/arm/mm/Kconfig
@@ -421,18 +421,21 @@
select CPU_USE_DOMAINS if MMU
select NEED_KUSER_HELPERS
select TLS_REG_EMUL if SMP || !MMU
+ select CPU_NO_EFFICIENT_FFS
config CPU_32v4
bool
select CPU_USE_DOMAINS if MMU
select NEED_KUSER_HELPERS
select TLS_REG_EMUL if SMP || !MMU
+ select CPU_NO_EFFICIENT_FFS
config CPU_32v4T
bool
select CPU_USE_DOMAINS if MMU
select NEED_KUSER_HELPERS
select TLS_REG_EMUL if SMP || !MMU
+ select CPU_NO_EFFICIENT_FFS
config CPU_32v5
bool
diff --git a/arch/arm/mm/cache-l2x0.c b/arch/arm/mm/cache-l2x0.c
index 9f9d542..c61996c 100644
--- a/arch/arm/mm/cache-l2x0.c
+++ b/arch/arm/mm/cache-l2x0.c
@@ -647,11 +647,6 @@
aux &= ~(L310_AUX_CTRL_FULL_LINE_ZERO | L310_AUX_CTRL_EARLY_BRESP);
}
- /* r3p0 or later has power control register */
- if (rev >= L310_CACHE_ID_RTL_R3P0)
- l2x0_saved_regs.pwr_ctrl = L310_DYNAMIC_CLK_GATING_EN |
- L310_STNDBY_MODE_EN;
-
/*
* Always enable non-secure access to the lockdown registers -
* we write to them as part of the L2C enable sequence so they
@@ -1141,6 +1136,7 @@
u32 filter[2] = { 0, 0 };
u32 assoc;
u32 prefetch;
+ u32 power;
u32 val;
int ret;
@@ -1271,6 +1267,26 @@
}
l2x0_saved_regs.prefetch_ctrl = prefetch;
+
+ power = l2x0_saved_regs.pwr_ctrl |
+ L310_DYNAMIC_CLK_GATING_EN | L310_STNDBY_MODE_EN;
+
+ ret = of_property_read_u32(np, "arm,dynamic-clock-gating", &val);
+ if (!ret) {
+ if (!val)
+ power &= ~L310_DYNAMIC_CLK_GATING_EN;
+ } else if (ret != -EINVAL) {
+ pr_err("L2C-310 OF dynamic-clock-gating property value is missing or invalid\n");
+ }
+ ret = of_property_read_u32(np, "arm,standby-mode", &val);
+ if (!ret) {
+ if (!val)
+ power &= ~L310_STNDBY_MODE_EN;
+ } else if (ret != -EINVAL) {
+ pr_err("L2C-310 OF standby-mode property value is missing or invalid\n");
+ }
+
+ l2x0_saved_regs.pwr_ctrl = power;
}
static const struct l2c_init_data of_l2c310_data __initconst = {
diff --git a/arch/arm/mm/cache-uniphier.c b/arch/arm/mm/cache-uniphier.c
index a6fa7b7..c8e2f49 100644
--- a/arch/arm/mm/cache-uniphier.c
+++ b/arch/arm/mm/cache-uniphier.c
@@ -96,6 +96,7 @@
void __iomem *ctrl_base;
void __iomem *rev_base;
void __iomem *op_base;
+ void __iomem *way_ctrl_base;
u32 way_present_mask;
u32 way_locked_mask;
u32 nsets;
@@ -256,10 +257,13 @@
struct uniphier_cache_data *data,
u32 way_mask)
{
+ unsigned int cpu;
+
data->way_locked_mask = way_mask & data->way_present_mask;
- writel_relaxed(~data->way_locked_mask & data->way_present_mask,
- data->ctrl_base + UNIPHIER_SSCLPDAWCR);
+ for_each_possible_cpu(cpu)
+ writel_relaxed(~data->way_locked_mask & data->way_present_mask,
+ data->way_ctrl_base + 4 * cpu);
}
static void uniphier_cache_maint_range(unsigned long start, unsigned long end,
@@ -459,6 +463,8 @@
goto err;
}
+ data->way_ctrl_base = data->ctrl_base + 0xc00;
+
if (*cache_level == 2) {
u32 revision = readl(data->rev_base + UNIPHIER_SSCID);
/*
@@ -467,6 +473,22 @@
*/
if (revision <= 0x16)
data->range_op_max_size = (u32)1 << 22;
+
+ /*
+ * Unfortunatly, the offset address of active way control base
+ * varies from SoC to SoC.
+ */
+ switch (revision) {
+ case 0x11: /* sLD3 */
+ data->way_ctrl_base = data->ctrl_base + 0x870;
+ break;
+ case 0x12: /* LD4 */
+ case 0x16: /* sld8 */
+ data->way_ctrl_base = data->ctrl_base + 0x840;
+ break;
+ default:
+ break;
+ }
}
data->range_op_max_size -= data->line_size;
diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c
index c941e93..ff7ed56 100644
--- a/arch/arm/mm/dma-mapping.c
+++ b/arch/arm/mm/dma-mapping.c
@@ -190,7 +190,6 @@
.sync_single_for_device = arm_dma_sync_single_for_device,
.sync_sg_for_cpu = arm_dma_sync_sg_for_cpu,
.sync_sg_for_device = arm_dma_sync_sg_for_device,
- .set_dma_mask = arm_dma_set_mask,
};
EXPORT_SYMBOL(arm_dma_ops);
@@ -209,7 +208,6 @@
.get_sgtable = arm_dma_get_sgtable,
.map_page = arm_coherent_dma_map_page,
.map_sg = arm_dma_map_sg,
- .set_dma_mask = arm_dma_set_mask,
};
EXPORT_SYMBOL(arm_coherent_dma_ops);
@@ -1143,16 +1141,6 @@
}
EXPORT_SYMBOL(dma_supported);
-int arm_dma_set_mask(struct device *dev, u64 dma_mask)
-{
- if (!dev->dma_mask || !dma_supported(dev, dma_mask))
- return -EIO;
-
- *dev->dma_mask = dma_mask;
-
- return 0;
-}
-
#define PREALLOC_DMA_DEBUG_ENTRIES 4096
static int __init dma_debug_do_init(void)
@@ -2006,8 +1994,6 @@
.unmap_sg = arm_iommu_unmap_sg,
.sync_sg_for_cpu = arm_iommu_sync_sg_for_cpu,
.sync_sg_for_device = arm_iommu_sync_sg_for_device,
-
- .set_dma_mask = arm_dma_set_mask,
};
struct dma_map_ops iommu_coherent_ops = {
@@ -2021,8 +2007,6 @@
.map_sg = arm_coherent_iommu_map_sg,
.unmap_sg = arm_coherent_iommu_unmap_sg,
-
- .set_dma_mask = arm_dma_set_mask,
};
/**
@@ -2215,7 +2199,7 @@
}
static bool arm_setup_iommu_dma_ops(struct device *dev, u64 dma_base, u64 size,
- struct iommu_ops *iommu)
+ const struct iommu_ops *iommu)
{
struct dma_iommu_mapping *mapping;
@@ -2253,7 +2237,7 @@
#else
static bool arm_setup_iommu_dma_ops(struct device *dev, u64 dma_base, u64 size,
- struct iommu_ops *iommu)
+ const struct iommu_ops *iommu)
{
return false;
}
@@ -2270,7 +2254,7 @@
}
void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
- struct iommu_ops *iommu, bool coherent)
+ const struct iommu_ops *iommu, bool coherent)
{
struct dma_map_ops *dma_ops;
diff --git a/arch/arm/mm/idmap.c b/arch/arm/mm/idmap.c
index bd274a0..c1a48f8 100644
--- a/arch/arm/mm/idmap.c
+++ b/arch/arm/mm/idmap.c
@@ -15,7 +15,7 @@
* page tables.
*/
pgd_t *idmap_pgd;
-unsigned long (*arch_virt_to_idmap)(unsigned long x);
+long long arch_phys_to_idmap_offset;
#ifdef CONFIG_ARM_LPAE
static void idmap_add_pmd(pud_t *pud, unsigned long addr, unsigned long end,
diff --git a/arch/arm/mm/ioremap.c b/arch/arm/mm/ioremap.c
index 66a978d..ff0eed2 100644
--- a/arch/arm/mm/ioremap.c
+++ b/arch/arm/mm/ioremap.c
@@ -297,9 +297,10 @@
}
/*
- * Don't allow RAM to be mapped - this causes problems with ARMv6+
+ * Don't allow RAM to be mapped with mismatched attributes - this
+ * causes problems with ARMv6+
*/
- if (WARN_ON(pfn_valid(pfn)))
+ if (WARN_ON(pfn_valid(pfn) && mtype != MT_MEMORY_RW))
return NULL;
area = get_vm_area_caller(size, VM_IOREMAP, caller);
@@ -380,11 +381,15 @@
EXPORT_SYMBOL(ioremap);
void __iomem *ioremap_cache(resource_size_t res_cookie, size_t size)
+ __alias(ioremap_cached);
+
+void __iomem *ioremap_cached(resource_size_t res_cookie, size_t size)
{
return arch_ioremap_caller(res_cookie, size, MT_DEVICE_CACHED,
__builtin_return_address(0));
}
EXPORT_SYMBOL(ioremap_cache);
+EXPORT_SYMBOL(ioremap_cached);
void __iomem *ioremap_wc(resource_size_t res_cookie, size_t size)
{
@@ -414,6 +419,13 @@
__builtin_return_address(0));
}
+void *arch_memremap_wb(phys_addr_t phys_addr, size_t size)
+{
+ return (__force void *)arch_ioremap_caller(phys_addr, size,
+ MT_MEMORY_RW,
+ __builtin_return_address(0));
+}
+
void __iounmap(volatile void __iomem *io_addr)
{
void *addr = (void *)(PAGE_MASK & (unsigned long)io_addr);
diff --git a/arch/arm/mm/nommu.c b/arch/arm/mm/nommu.c
index d5805e4..2740967 100644
--- a/arch/arm/mm/nommu.c
+++ b/arch/arm/mm/nommu.c
@@ -368,11 +368,15 @@
EXPORT_SYMBOL(ioremap);
void __iomem *ioremap_cache(resource_size_t res_cookie, size_t size)
+ __alias(ioremap_cached);
+
+void __iomem *ioremap_cached(resource_size_t res_cookie, size_t size)
{
return __arm_ioremap_caller(res_cookie, size, MT_DEVICE_CACHED,
__builtin_return_address(0));
}
EXPORT_SYMBOL(ioremap_cache);
+EXPORT_SYMBOL(ioremap_cached);
void __iomem *ioremap_wc(resource_size_t res_cookie, size_t size)
{
@@ -381,6 +385,11 @@
}
EXPORT_SYMBOL(ioremap_wc);
+void *arch_memremap_wb(phys_addr_t phys_addr, size_t size)
+{
+ return (void *)phys_addr;
+}
+
void __iounmap(volatile void __iomem *addr)
{
}
diff --git a/arch/arm/tools/Makefile b/arch/arm/tools/Makefile
index 32d05c8..6e4cd18 100644
--- a/arch/arm/tools/Makefile
+++ b/arch/arm/tools/Makefile
@@ -4,7 +4,10 @@
# Copyright (C) 2001 Russell King
#
-include/generated/mach-types.h: $(src)/gen-mach-types $(src)/mach-types
- @$(kecho) ' Generating $@'
- @mkdir -p $(dir $@)
- $(Q)$(AWK) -f $^ > $@ || { rm -f $@; /bin/false; }
+quiet_cmd_gen_mach = GEN $@
+ cmd_gen_mach = mkdir -p $(dir $@) && \
+ $(AWK) -f $(filter-out $(PHONY),$^) > $@ || \
+ { rm -f $@; /bin/false; }
+
+include/generated/mach-types.h: $(src)/gen-mach-types $(src)/mach-types FORCE
+ $(call if_changed,gen_mach)
diff --git a/arch/arm/vfp/vfpmodule.c b/arch/arm/vfp/vfpmodule.c
index 2a61e4b..73085d3 100644
--- a/arch/arm/vfp/vfpmodule.c
+++ b/arch/arm/vfp/vfpmodule.c
@@ -156,10 +156,6 @@
* - we could be preempted if tree preempt rcu is enabled, so
* it is unsafe to use thread->cpu.
* THREAD_NOTIFY_EXIT
- * - the thread (v) will be running on the local CPU, so
- * v === current_thread_info()
- * - thread->cpu is the local CPU number at the time it is accessed,
- * but may change at any time.
* - we could be preempted if tree preempt rcu is enabled, so
* it is unsafe to use thread->cpu.
*/
diff --git a/arch/arm64/boot/dts/renesas/r8a7795.dtsi b/arch/arm64/boot/dts/renesas/r8a7795.dtsi
index 7cb2d72..3285a92 100644
--- a/arch/arm64/boot/dts/renesas/r8a7795.dtsi
+++ b/arch/arm64/boot/dts/renesas/r8a7795.dtsi
@@ -10,6 +10,7 @@
#include <dt-bindings/clock/r8a7795-cpg-mssr.h>
#include <dt-bindings/interrupt-controller/arm-gic.h>
+#include <dt-bindings/power/r8a7795-sysc.h>
/ {
compatible = "renesas,r8a7795";
@@ -39,6 +40,7 @@
compatible = "arm,cortex-a57", "arm,armv8";
reg = <0x0>;
device_type = "cpu";
+ power-domains = <&sysc R8A7795_PD_CA57_CPU0>;
next-level-cache = <&L2_CA57>;
enable-method = "psci";
};
@@ -47,6 +49,7 @@
compatible = "arm,cortex-a57","arm,armv8";
reg = <0x1>;
device_type = "cpu";
+ power-domains = <&sysc R8A7795_PD_CA57_CPU1>;
next-level-cache = <&L2_CA57>;
enable-method = "psci";
};
@@ -54,6 +57,7 @@
compatible = "arm,cortex-a57","arm,armv8";
reg = <0x2>;
device_type = "cpu";
+ power-domains = <&sysc R8A7795_PD_CA57_CPU2>;
next-level-cache = <&L2_CA57>;
enable-method = "psci";
};
@@ -61,6 +65,7 @@
compatible = "arm,cortex-a57","arm,armv8";
reg = <0x3>;
device_type = "cpu";
+ power-domains = <&sysc R8A7795_PD_CA57_CPU3>;
next-level-cache = <&L2_CA57>;
enable-method = "psci";
};
@@ -68,12 +73,14 @@
L2_CA57: cache-controller@0 {
compatible = "cache";
+ power-domains = <&sysc R8A7795_PD_CA57_SCU>;
cache-unified;
cache-level = <2>;
};
L2_CA53: cache-controller@1 {
compatible = "cache";
+ power-domains = <&sysc R8A7795_PD_CA53_SCU>;
cache-unified;
cache-level = <2>;
};
@@ -168,7 +175,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&cpg CPG_MOD 912>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
};
gpio1: gpio@e6051000 {
@@ -182,7 +189,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&cpg CPG_MOD 911>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
};
gpio2: gpio@e6052000 {
@@ -196,7 +203,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&cpg CPG_MOD 910>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
};
gpio3: gpio@e6053000 {
@@ -210,7 +217,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&cpg CPG_MOD 909>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
};
gpio4: gpio@e6054000 {
@@ -224,7 +231,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&cpg CPG_MOD 908>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
};
gpio5: gpio@e6055000 {
@@ -238,7 +245,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&cpg CPG_MOD 907>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
};
gpio6: gpio@e6055400 {
@@ -252,7 +259,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&cpg CPG_MOD 906>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
};
gpio7: gpio@e6055800 {
@@ -266,7 +273,7 @@
#interrupt-cells = <2>;
interrupt-controller;
clocks = <&cpg CPG_MOD 905>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
};
pmu_a57 {
@@ -302,6 +309,12 @@
#power-domain-cells = <0>;
};
+ sysc: system-controller@e6180000 {
+ compatible = "renesas,r8a7795-sysc";
+ reg = <0 0xe6180000 0 0x0400>;
+ #power-domain-cells = <1>;
+ };
+
audma0: dma-controller@ec700000 {
compatible = "renesas,rcar-dmac";
reg = <0 0xec700000 0 0x10000>;
@@ -329,7 +342,7 @@
"ch12", "ch13", "ch14", "ch15";
clocks = <&cpg CPG_MOD 502>;
clock-names = "fck";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <16>;
};
@@ -361,7 +374,7 @@
"ch12", "ch13", "ch14", "ch15";
clocks = <&cpg CPG_MOD 501>;
clock-names = "fck";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <16>;
};
@@ -383,7 +396,7 @@
GIC_SPI 18 IRQ_TYPE_LEVEL_HIGH
GIC_SPI 161 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cpg CPG_MOD 407>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
};
dmac0: dma-controller@e6700000 {
@@ -414,7 +427,7 @@
"ch12", "ch13", "ch14", "ch15";
clocks = <&cpg CPG_MOD 219>;
clock-names = "fck";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <16>;
};
@@ -447,7 +460,7 @@
"ch12", "ch13", "ch14", "ch15";
clocks = <&cpg CPG_MOD 218>;
clock-names = "fck";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <16>;
};
@@ -480,7 +493,7 @@
"ch12", "ch13", "ch14", "ch15";
clocks = <&cpg CPG_MOD 217>;
clock-names = "fck";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <16>;
};
@@ -522,7 +535,7 @@
"ch20", "ch21", "ch22", "ch23",
"ch24";
clocks = <&cpg CPG_MOD 812>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
phy-mode = "rgmii-id";
#address-cells = <1>;
#size-cells = <0>;
@@ -539,7 +552,7 @@
clock-names = "clkp1", "clkp2", "can_clk";
assigned-clocks = <&cpg CPG_CORE R8A7795_CLK_CANFD>;
assigned-clock-rates = <40000000>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -554,7 +567,7 @@
clock-names = "clkp1", "clkp2", "can_clk";
assigned-clocks = <&cpg CPG_CORE R8A7795_CLK_CANFD>;
assigned-clock-rates = <40000000>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -570,7 +583,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac1 0x31>, <&dmac1 0x30>;
dma-names = "tx", "rx";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -586,7 +599,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac1 0x33>, <&dmac1 0x32>;
dma-names = "tx", "rx";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -602,7 +615,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac1 0x35>, <&dmac1 0x34>;
dma-names = "tx", "rx";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -618,7 +631,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x37>, <&dmac0 0x36>;
dma-names = "tx", "rx";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -634,7 +647,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x39>, <&dmac0 0x38>;
dma-names = "tx", "rx";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -649,7 +662,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac1 0x51>, <&dmac1 0x50>;
dma-names = "tx", "rx";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -664,7 +677,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac1 0x53>, <&dmac1 0x52>;
dma-names = "tx", "rx";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -679,7 +692,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac1 0x13>, <&dmac1 0x12>;
dma-names = "tx", "rx";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -694,7 +707,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x57>, <&dmac0 0x56>;
dma-names = "tx", "rx";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -709,7 +722,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac0 0x59>, <&dmac0 0x58>;
dma-names = "tx", "rx";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -724,7 +737,7 @@
clock-names = "fck", "brg_int", "scif_clk";
dmas = <&dmac1 0x5b>, <&dmac1 0x5a>;
dma-names = "tx", "rx";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -735,7 +748,7 @@
reg = <0 0xe6500000 0 0x40>;
interrupts = <GIC_SPI 287 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cpg CPG_MOD 931>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <110>;
status = "disabled";
};
@@ -747,7 +760,7 @@
reg = <0 0xe6508000 0 0x40>;
interrupts = <GIC_SPI 288 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cpg CPG_MOD 930>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <6>;
status = "disabled";
};
@@ -759,7 +772,7 @@
reg = <0 0xe6510000 0 0x40>;
interrupts = <GIC_SPI 286 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cpg CPG_MOD 929>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <6>;
status = "disabled";
};
@@ -771,7 +784,7 @@
reg = <0 0xe66d0000 0 0x40>;
interrupts = <GIC_SPI 290 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cpg CPG_MOD 928>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <110>;
status = "disabled";
};
@@ -783,7 +796,7 @@
reg = <0 0xe66d8000 0 0x40>;
interrupts = <GIC_SPI 19 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cpg CPG_MOD 927>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <110>;
status = "disabled";
};
@@ -795,7 +808,7 @@
reg = <0 0xe66e0000 0 0x40>;
interrupts = <GIC_SPI 20 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cpg CPG_MOD 919>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <110>;
status = "disabled";
};
@@ -807,7 +820,7 @@
reg = <0 0xe66e8000 0 0x40>;
interrupts = <GIC_SPI 21 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cpg CPG_MOD 918>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
i2c-scl-internal-delay-ns = <6>;
status = "disabled";
};
@@ -857,7 +870,7 @@
"src.1", "src.0",
"dvc.0", "dvc.1",
"clk_a", "clk_b", "clk_c", "clk_i";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
rcar_sound,dvc {
@@ -991,7 +1004,7 @@
reg = <0 0xee000000 0 0xc00>;
interrupts = <GIC_SPI 102 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cpg CPG_MOD 328>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -1000,7 +1013,7 @@
reg = <0 0xee040000 0 0xc00>;
interrupts = <GIC_SPI 98 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cpg CPG_MOD 327>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -1012,7 +1025,7 @@
GIC_SPI 109 IRQ_TYPE_LEVEL_HIGH>;
interrupt-names = "ch0", "ch1";
clocks = <&cpg CPG_MOD 330>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <2>;
};
@@ -1025,7 +1038,7 @@
GIC_SPI 110 IRQ_TYPE_LEVEL_HIGH>;
interrupt-names = "ch0", "ch1";
clocks = <&cpg CPG_MOD 331>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
#dma-cells = <1>;
dma-channels = <2>;
};
@@ -1035,7 +1048,7 @@
reg = <0 0xee100000 0 0x2000>;
interrupts = <GIC_SPI 165 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cpg CPG_MOD 314>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -1044,7 +1057,7 @@
reg = <0 0xee120000 0 0x2000>;
interrupts = <GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cpg CPG_MOD 313>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -1053,7 +1066,7 @@
reg = <0 0xee140000 0 0x2000>;
interrupts = <GIC_SPI 167 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cpg CPG_MOD 312>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
cap-mmc-highspeed;
status = "disabled";
};
@@ -1063,7 +1076,7 @@
reg = <0 0xee160000 0 0x2000>;
interrupts = <GIC_SPI 168 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cpg CPG_MOD 311>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
cap-mmc-highspeed;
status = "disabled";
};
@@ -1073,7 +1086,7 @@
reg = <0 0xee080200 0 0x700>;
interrupts = <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cpg CPG_MOD 703>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
#phy-cells = <0>;
status = "disabled";
};
@@ -1082,7 +1095,7 @@
compatible = "renesas,usb2-phy-r8a7795";
reg = <0 0xee0a0200 0 0x700>;
clocks = <&cpg CPG_MOD 702>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
#phy-cells = <0>;
status = "disabled";
};
@@ -1091,7 +1104,7 @@
compatible = "renesas,usb2-phy-r8a7795";
reg = <0 0xee0c0200 0 0x700>;
clocks = <&cpg CPG_MOD 701>;
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
#phy-cells = <0>;
status = "disabled";
};
@@ -1103,7 +1116,7 @@
clocks = <&cpg CPG_MOD 703>;
phys = <&usb2_phy0>;
phy-names = "usb";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -1114,7 +1127,7 @@
clocks = <&cpg CPG_MOD 702>;
phys = <&usb2_phy1>;
phy-names = "usb";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -1125,7 +1138,7 @@
clocks = <&cpg CPG_MOD 701>;
phys = <&usb2_phy2>;
phy-names = "usb";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -1136,7 +1149,7 @@
clocks = <&cpg CPG_MOD 703>;
phys = <&usb2_phy0>;
phy-names = "usb";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -1147,7 +1160,7 @@
clocks = <&cpg CPG_MOD 702>;
phys = <&usb2_phy1>;
phy-names = "usb";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -1158,7 +1171,7 @@
clocks = <&cpg CPG_MOD 701>;
phys = <&usb2_phy2>;
phy-names = "usb";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
pciec0: pcie@fe000000 {
@@ -1182,7 +1195,7 @@
interrupt-map = <0 0 0 0 &gic GIC_SPI 116 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cpg CPG_MOD 319>, <&pcie_bus_clk>;
clock-names = "pcie", "pcie_bus";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
@@ -1207,7 +1220,7 @@
interrupt-map = <0 0 0 0 &gic GIC_SPI 148 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cpg CPG_MOD 318>, <&pcie_bus_clk>;
clock-names = "pcie", "pcie_bus";
- power-domains = <&cpg>;
+ power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
status = "disabled";
};
};
diff --git a/arch/arm64/include/asm/dma-mapping.h b/arch/arm64/include/asm/dma-mapping.h
index ba437f0..7dbea6c 100644
--- a/arch/arm64/include/asm/dma-mapping.h
+++ b/arch/arm64/include/asm/dma-mapping.h
@@ -48,7 +48,7 @@
}
void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
- struct iommu_ops *iommu, bool coherent);
+ const struct iommu_ops *iommu, bool coherent);
#define arch_setup_dma_ops arch_setup_dma_ops
#ifdef CONFIG_IOMMU_DMA
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 1b3dc9df..2cdb6b5 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -85,32 +85,37 @@
#define HCR_HOST_VHE_FLAGS (HCR_RW | HCR_TGE | HCR_E2H)
/* TCR_EL2 Registers bits */
-#define TCR_EL2_RES1 ((1 << 31) | (1 << 23))
-#define TCR_EL2_TBI (1 << 20)
-#define TCR_EL2_PS (7 << 16)
-#define TCR_EL2_PS_40B (2 << 16)
-#define TCR_EL2_TG0 (1 << 14)
-#define TCR_EL2_SH0 (3 << 12)
-#define TCR_EL2_ORGN0 (3 << 10)
-#define TCR_EL2_IRGN0 (3 << 8)
-#define TCR_EL2_T0SZ 0x3f
-#define TCR_EL2_MASK (TCR_EL2_TG0 | TCR_EL2_SH0 | \
- TCR_EL2_ORGN0 | TCR_EL2_IRGN0 | TCR_EL2_T0SZ)
+#define TCR_EL2_RES1 ((1 << 31) | (1 << 23))
+#define TCR_EL2_TBI (1 << 20)
+#define TCR_EL2_PS_SHIFT 16
+#define TCR_EL2_PS_MASK (7 << TCR_EL2_PS_SHIFT)
+#define TCR_EL2_PS_40B (2 << TCR_EL2_PS_SHIFT)
+#define TCR_EL2_TG0_MASK TCR_TG0_MASK
+#define TCR_EL2_SH0_MASK TCR_SH0_MASK
+#define TCR_EL2_ORGN0_MASK TCR_ORGN0_MASK
+#define TCR_EL2_IRGN0_MASK TCR_IRGN0_MASK
+#define TCR_EL2_T0SZ_MASK 0x3f
+#define TCR_EL2_MASK (TCR_EL2_TG0_MASK | TCR_EL2_SH0_MASK | \
+ TCR_EL2_ORGN0_MASK | TCR_EL2_IRGN0_MASK | TCR_EL2_T0SZ_MASK)
/* VTCR_EL2 Registers bits */
#define VTCR_EL2_RES1 (1 << 31)
-#define VTCR_EL2_PS_MASK (7 << 16)
-#define VTCR_EL2_TG0_MASK (1 << 14)
-#define VTCR_EL2_TG0_4K (0 << 14)
-#define VTCR_EL2_TG0_64K (1 << 14)
-#define VTCR_EL2_SH0_MASK (3 << 12)
-#define VTCR_EL2_SH0_INNER (3 << 12)
-#define VTCR_EL2_ORGN0_MASK (3 << 10)
-#define VTCR_EL2_ORGN0_WBWA (1 << 10)
-#define VTCR_EL2_IRGN0_MASK (3 << 8)
-#define VTCR_EL2_IRGN0_WBWA (1 << 8)
-#define VTCR_EL2_SL0_MASK (3 << 6)
-#define VTCR_EL2_SL0_LVL1 (1 << 6)
+#define VTCR_EL2_HD (1 << 22)
+#define VTCR_EL2_HA (1 << 21)
+#define VTCR_EL2_PS_MASK TCR_EL2_PS_MASK
+#define VTCR_EL2_TG0_MASK TCR_TG0_MASK
+#define VTCR_EL2_TG0_4K TCR_TG0_4K
+#define VTCR_EL2_TG0_16K TCR_TG0_16K
+#define VTCR_EL2_TG0_64K TCR_TG0_64K
+#define VTCR_EL2_SH0_MASK TCR_SH0_MASK
+#define VTCR_EL2_SH0_INNER TCR_SH0_INNER
+#define VTCR_EL2_ORGN0_MASK TCR_ORGN0_MASK
+#define VTCR_EL2_ORGN0_WBWA TCR_ORGN0_WBWA
+#define VTCR_EL2_IRGN0_MASK TCR_IRGN0_MASK
+#define VTCR_EL2_IRGN0_WBWA TCR_IRGN0_WBWA
+#define VTCR_EL2_SL0_SHIFT 6
+#define VTCR_EL2_SL0_MASK (3 << VTCR_EL2_SL0_SHIFT)
+#define VTCR_EL2_SL0_LVL1 (1 << VTCR_EL2_SL0_SHIFT)
#define VTCR_EL2_T0SZ_MASK 0x3f
#define VTCR_EL2_T0SZ_40B 24
#define VTCR_EL2_VS_SHIFT 19
@@ -126,35 +131,45 @@
* (see hyp-init.S).
*
* Note that when using 4K pages, we concatenate two first level page tables
- * together.
+ * together. With 16K pages, we concatenate 16 first level page tables.
*
* The magic numbers used for VTTBR_X in this patch can be found in Tables
* D4-23 and D4-25 in ARM DDI 0487A.b.
*/
+
+#define VTCR_EL2_T0SZ_IPA VTCR_EL2_T0SZ_40B
+#define VTCR_EL2_COMMON_BITS (VTCR_EL2_SH0_INNER | VTCR_EL2_ORGN0_WBWA | \
+ VTCR_EL2_IRGN0_WBWA | VTCR_EL2_RES1)
+
#ifdef CONFIG_ARM64_64K_PAGES
/*
* Stage2 translation configuration:
- * 40bits input (T0SZ = 24)
* 64kB pages (TG0 = 1)
* 2 level page tables (SL = 1)
*/
-#define VTCR_EL2_FLAGS (VTCR_EL2_TG0_64K | VTCR_EL2_SH0_INNER | \
- VTCR_EL2_ORGN0_WBWA | VTCR_EL2_IRGN0_WBWA | \
- VTCR_EL2_SL0_LVL1 | VTCR_EL2_RES1)
-#define VTTBR_X (38 - VTCR_EL2_T0SZ_40B)
-#else
+#define VTCR_EL2_TGRAN_FLAGS (VTCR_EL2_TG0_64K | VTCR_EL2_SL0_LVL1)
+#define VTTBR_X_TGRAN_MAGIC 38
+#elif defined(CONFIG_ARM64_16K_PAGES)
/*
* Stage2 translation configuration:
- * 40bits input (T0SZ = 24)
+ * 16kB pages (TG0 = 2)
+ * 2 level page tables (SL = 1)
+ */
+#define VTCR_EL2_TGRAN_FLAGS (VTCR_EL2_TG0_16K | VTCR_EL2_SL0_LVL1)
+#define VTTBR_X_TGRAN_MAGIC 42
+#else /* 4K */
+/*
+ * Stage2 translation configuration:
* 4kB pages (TG0 = 0)
* 3 level page tables (SL = 1)
*/
-#define VTCR_EL2_FLAGS (VTCR_EL2_TG0_4K | VTCR_EL2_SH0_INNER | \
- VTCR_EL2_ORGN0_WBWA | VTCR_EL2_IRGN0_WBWA | \
- VTCR_EL2_SL0_LVL1 | VTCR_EL2_RES1)
-#define VTTBR_X (37 - VTCR_EL2_T0SZ_40B)
+#define VTCR_EL2_TGRAN_FLAGS (VTCR_EL2_TG0_4K | VTCR_EL2_SL0_LVL1)
+#define VTTBR_X_TGRAN_MAGIC 37
#endif
+#define VTCR_EL2_FLAGS (VTCR_EL2_COMMON_BITS | VTCR_EL2_TGRAN_FLAGS)
+#define VTTBR_X (VTTBR_X_TGRAN_MAGIC - VTCR_EL2_T0SZ_IPA)
+
#define VTTBR_BADDR_SHIFT (VTTBR_X - 1)
#define VTTBR_BADDR_MASK (((UL(1) << (PHYS_MASK_SHIFT - VTTBR_X)) - 1) << VTTBR_BADDR_SHIFT)
#define VTTBR_VMID_SHIFT (UL(48))
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 90a8d23..e63d23b 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -295,6 +295,7 @@
struct kvm_vcpu_stat {
u32 halt_successful_poll;
u32 halt_attempted_poll;
+ u32 halt_poll_invalid;
u32 halt_wakeup;
u32 hvc_exit_stat;
u64 wfe_exit_stat;
@@ -369,6 +370,7 @@
static inline void kvm_arch_sync_events(struct kvm *kvm) {}
static inline void kvm_arch_vcpu_uninit(struct kvm_vcpu *vcpu) {}
static inline void kvm_arch_sched_in(struct kvm_vcpu *vcpu, int cpu) {}
+static inline void kvm_arch_vcpu_block_finish(struct kvm_vcpu *vcpu) {}
void kvm_arm_init_debug(void);
void kvm_arm_setup_debug(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index e8d39d4..f05ac27 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -45,18 +45,6 @@
*/
#define TRAMPOLINE_VA (HYP_PAGE_OFFSET_MASK & PAGE_MASK)
-/*
- * KVM_MMU_CACHE_MIN_PAGES is the number of stage2 page table translation
- * levels in addition to the PGD and potentially the PUD which are
- * pre-allocated (we pre-allocate the fake PGD and the PUD when the Stage-2
- * tables use one level of tables less than the kernel.
- */
-#ifdef CONFIG_ARM64_64K_PAGES
-#define KVM_MMU_CACHE_MIN_PAGES 1
-#else
-#define KVM_MMU_CACHE_MIN_PAGES 2
-#endif
-
#ifdef __ASSEMBLY__
#include <asm/alternative.h>
@@ -91,6 +79,8 @@
#define KVM_PHYS_SIZE (1UL << KVM_PHYS_SHIFT)
#define KVM_PHYS_MASK (KVM_PHYS_SIZE - 1UL)
+#include <asm/stage2_pgtable.h>
+
int create_hyp_mappings(void *from, void *to);
int create_hyp_io_mappings(void *from, void *to, phys_addr_t);
void free_boot_hyp_pgd(void);
@@ -122,19 +112,32 @@
static inline void kvm_clean_pte(pte_t *pte) {}
static inline void kvm_clean_pte_entry(pte_t *pte) {}
-static inline void kvm_set_s2pte_writable(pte_t *pte)
+static inline pte_t kvm_s2pte_mkwrite(pte_t pte)
{
- pte_val(*pte) |= PTE_S2_RDWR;
+ pte_val(pte) |= PTE_S2_RDWR;
+ return pte;
}
-static inline void kvm_set_s2pmd_writable(pmd_t *pmd)
+static inline pmd_t kvm_s2pmd_mkwrite(pmd_t pmd)
{
- pmd_val(*pmd) |= PMD_S2_RDWR;
+ pmd_val(pmd) |= PMD_S2_RDWR;
+ return pmd;
}
static inline void kvm_set_s2pte_readonly(pte_t *pte)
{
- pte_val(*pte) = (pte_val(*pte) & ~PTE_S2_RDWR) | PTE_S2_RDONLY;
+ pteval_t pteval;
+ unsigned long tmp;
+
+ asm volatile("// kvm_set_s2pte_readonly\n"
+ " prfm pstl1strm, %2\n"
+ "1: ldxr %0, %2\n"
+ " and %0, %0, %3 // clear PTE_S2_RDWR\n"
+ " orr %0, %0, %4 // set PTE_S2_RDONLY\n"
+ " stxr %w1, %0, %2\n"
+ " cbnz %w1, 1b\n"
+ : "=&r" (pteval), "=&r" (tmp), "+Q" (pte_val(*pte))
+ : "L" (~PTE_S2_RDWR), "L" (PTE_S2_RDONLY));
}
static inline bool kvm_s2pte_readonly(pte_t *pte)
@@ -144,69 +147,12 @@
static inline void kvm_set_s2pmd_readonly(pmd_t *pmd)
{
- pmd_val(*pmd) = (pmd_val(*pmd) & ~PMD_S2_RDWR) | PMD_S2_RDONLY;
+ kvm_set_s2pte_readonly((pte_t *)pmd);
}
static inline bool kvm_s2pmd_readonly(pmd_t *pmd)
{
- return (pmd_val(*pmd) & PMD_S2_RDWR) == PMD_S2_RDONLY;
-}
-
-
-#define kvm_pgd_addr_end(addr, end) pgd_addr_end(addr, end)
-#define kvm_pud_addr_end(addr, end) pud_addr_end(addr, end)
-#define kvm_pmd_addr_end(addr, end) pmd_addr_end(addr, end)
-
-/*
- * In the case where PGDIR_SHIFT is larger than KVM_PHYS_SHIFT, we can address
- * the entire IPA input range with a single pgd entry, and we would only need
- * one pgd entry. Note that in this case, the pgd is actually not used by
- * the MMU for Stage-2 translations, but is merely a fake pgd used as a data
- * structure for the kernel pgtable macros to work.
- */
-#if PGDIR_SHIFT > KVM_PHYS_SHIFT
-#define PTRS_PER_S2_PGD_SHIFT 0
-#else
-#define PTRS_PER_S2_PGD_SHIFT (KVM_PHYS_SHIFT - PGDIR_SHIFT)
-#endif
-#define PTRS_PER_S2_PGD (1 << PTRS_PER_S2_PGD_SHIFT)
-
-#define kvm_pgd_index(addr) (((addr) >> PGDIR_SHIFT) & (PTRS_PER_S2_PGD - 1))
-
-/*
- * If we are concatenating first level stage-2 page tables, we would have less
- * than or equal to 16 pointers in the fake PGD, because that's what the
- * architecture allows. In this case, (4 - CONFIG_PGTABLE_LEVELS)
- * represents the first level for the host, and we add 1 to go to the next
- * level (which uses contatenation) for the stage-2 tables.
- */
-#if PTRS_PER_S2_PGD <= 16
-#define KVM_PREALLOC_LEVEL (4 - CONFIG_PGTABLE_LEVELS + 1)
-#else
-#define KVM_PREALLOC_LEVEL (0)
-#endif
-
-static inline void *kvm_get_hwpgd(struct kvm *kvm)
-{
- pgd_t *pgd = kvm->arch.pgd;
- pud_t *pud;
-
- if (KVM_PREALLOC_LEVEL == 0)
- return pgd;
-
- pud = pud_offset(pgd, 0);
- if (KVM_PREALLOC_LEVEL == 1)
- return pud;
-
- BUG_ON(KVM_PREALLOC_LEVEL != 2);
- return pmd_offset(pud, 0);
-}
-
-static inline unsigned int kvm_get_hwpgd_size(void)
-{
- if (KVM_PREALLOC_LEVEL > 0)
- return PTRS_PER_S2_PGD * PAGE_SIZE;
- return PTRS_PER_S2_PGD * sizeof(pgd_t);
+ return kvm_s2pte_readonly((pte_t *)pmd);
}
static inline bool kvm_page_empty(void *ptr)
@@ -215,23 +161,20 @@
return page_count(ptr_page) == 1;
}
-#define kvm_pte_table_empty(kvm, ptep) kvm_page_empty(ptep)
+#define hyp_pte_table_empty(ptep) kvm_page_empty(ptep)
#ifdef __PAGETABLE_PMD_FOLDED
-#define kvm_pmd_table_empty(kvm, pmdp) (0)
+#define hyp_pmd_table_empty(pmdp) (0)
#else
-#define kvm_pmd_table_empty(kvm, pmdp) \
- (kvm_page_empty(pmdp) && (!(kvm) || KVM_PREALLOC_LEVEL < 2))
+#define hyp_pmd_table_empty(pmdp) kvm_page_empty(pmdp)
#endif
#ifdef __PAGETABLE_PUD_FOLDED
-#define kvm_pud_table_empty(kvm, pudp) (0)
+#define hyp_pud_table_empty(pudp) (0)
#else
-#define kvm_pud_table_empty(kvm, pudp) \
- (kvm_page_empty(pudp) && (!(kvm) || KVM_PREALLOC_LEVEL < 1))
+#define hyp_pud_table_empty(pudp) kvm_page_empty(pudp)
#endif
-
struct kvm;
#define kvm_flush_dcache_to_poc(a,l) __flush_dcache_area((a), (l))
diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h
index 9786f77..2813748 100644
--- a/arch/arm64/include/asm/pgtable-hwdef.h
+++ b/arch/arm64/include/asm/pgtable-hwdef.h
@@ -207,23 +207,69 @@
#define TCR_T1SZ(x) ((UL(64) - (x)) << TCR_T1SZ_OFFSET)
#define TCR_TxSZ(x) (TCR_T0SZ(x) | TCR_T1SZ(x))
#define TCR_TxSZ_WIDTH 6
-#define TCR_IRGN_NC ((UL(0) << 8) | (UL(0) << 24))
-#define TCR_IRGN_WBWA ((UL(1) << 8) | (UL(1) << 24))
-#define TCR_IRGN_WT ((UL(2) << 8) | (UL(2) << 24))
-#define TCR_IRGN_WBnWA ((UL(3) << 8) | (UL(3) << 24))
-#define TCR_IRGN_MASK ((UL(3) << 8) | (UL(3) << 24))
-#define TCR_ORGN_NC ((UL(0) << 10) | (UL(0) << 26))
-#define TCR_ORGN_WBWA ((UL(1) << 10) | (UL(1) << 26))
-#define TCR_ORGN_WT ((UL(2) << 10) | (UL(2) << 26))
-#define TCR_ORGN_WBnWA ((UL(3) << 10) | (UL(3) << 26))
-#define TCR_ORGN_MASK ((UL(3) << 10) | (UL(3) << 26))
-#define TCR_SHARED ((UL(3) << 12) | (UL(3) << 28))
-#define TCR_TG0_4K (UL(0) << 14)
-#define TCR_TG0_64K (UL(1) << 14)
-#define TCR_TG0_16K (UL(2) << 14)
-#define TCR_TG1_16K (UL(1) << 30)
-#define TCR_TG1_4K (UL(2) << 30)
-#define TCR_TG1_64K (UL(3) << 30)
+
+#define TCR_IRGN0_SHIFT 8
+#define TCR_IRGN0_MASK (UL(3) << TCR_IRGN0_SHIFT)
+#define TCR_IRGN0_NC (UL(0) << TCR_IRGN0_SHIFT)
+#define TCR_IRGN0_WBWA (UL(1) << TCR_IRGN0_SHIFT)
+#define TCR_IRGN0_WT (UL(2) << TCR_IRGN0_SHIFT)
+#define TCR_IRGN0_WBnWA (UL(3) << TCR_IRGN0_SHIFT)
+
+#define TCR_IRGN1_SHIFT 24
+#define TCR_IRGN1_MASK (UL(3) << TCR_IRGN1_SHIFT)
+#define TCR_IRGN1_NC (UL(0) << TCR_IRGN1_SHIFT)
+#define TCR_IRGN1_WBWA (UL(1) << TCR_IRGN1_SHIFT)
+#define TCR_IRGN1_WT (UL(2) << TCR_IRGN1_SHIFT)
+#define TCR_IRGN1_WBnWA (UL(3) << TCR_IRGN1_SHIFT)
+
+#define TCR_IRGN_NC (TCR_IRGN0_NC | TCR_IRGN1_NC)
+#define TCR_IRGN_WBWA (TCR_IRGN0_WBWA | TCR_IRGN1_WBWA)
+#define TCR_IRGN_WT (TCR_IRGN0_WT | TCR_IRGN1_WT)
+#define TCR_IRGN_WBnWA (TCR_IRGN0_WBnWA | TCR_IRGN1_WBnWA)
+#define TCR_IRGN_MASK (TCR_IRGN0_MASK | TCR_IRGN1_MASK)
+
+
+#define TCR_ORGN0_SHIFT 10
+#define TCR_ORGN0_MASK (UL(3) << TCR_ORGN0_SHIFT)
+#define TCR_ORGN0_NC (UL(0) << TCR_ORGN0_SHIFT)
+#define TCR_ORGN0_WBWA (UL(1) << TCR_ORGN0_SHIFT)
+#define TCR_ORGN0_WT (UL(2) << TCR_ORGN0_SHIFT)
+#define TCR_ORGN0_WBnWA (UL(3) << TCR_ORGN0_SHIFT)
+
+#define TCR_ORGN1_SHIFT 26
+#define TCR_ORGN1_MASK (UL(3) << TCR_ORGN1_SHIFT)
+#define TCR_ORGN1_NC (UL(0) << TCR_ORGN1_SHIFT)
+#define TCR_ORGN1_WBWA (UL(1) << TCR_ORGN1_SHIFT)
+#define TCR_ORGN1_WT (UL(2) << TCR_ORGN1_SHIFT)
+#define TCR_ORGN1_WBnWA (UL(3) << TCR_ORGN1_SHIFT)
+
+#define TCR_ORGN_NC (TCR_ORGN0_NC | TCR_ORGN1_NC)
+#define TCR_ORGN_WBWA (TCR_ORGN0_WBWA | TCR_ORGN1_WBWA)
+#define TCR_ORGN_WT (TCR_ORGN0_WT | TCR_ORGN1_WT)
+#define TCR_ORGN_WBnWA (TCR_ORGN0_WBnWA | TCR_ORGN1_WBnWA)
+#define TCR_ORGN_MASK (TCR_ORGN0_MASK | TCR_ORGN1_MASK)
+
+#define TCR_SH0_SHIFT 12
+#define TCR_SH0_MASK (UL(3) << TCR_SH0_SHIFT)
+#define TCR_SH0_INNER (UL(3) << TCR_SH0_SHIFT)
+
+#define TCR_SH1_SHIFT 28
+#define TCR_SH1_MASK (UL(3) << TCR_SH1_SHIFT)
+#define TCR_SH1_INNER (UL(3) << TCR_SH1_SHIFT)
+#define TCR_SHARED (TCR_SH0_INNER | TCR_SH1_INNER)
+
+#define TCR_TG0_SHIFT 14
+#define TCR_TG0_MASK (UL(3) << TCR_TG0_SHIFT)
+#define TCR_TG0_4K (UL(0) << TCR_TG0_SHIFT)
+#define TCR_TG0_64K (UL(1) << TCR_TG0_SHIFT)
+#define TCR_TG0_16K (UL(2) << TCR_TG0_SHIFT)
+
+#define TCR_TG1_SHIFT 30
+#define TCR_TG1_MASK (UL(3) << TCR_TG1_SHIFT)
+#define TCR_TG1_16K (UL(1) << TCR_TG1_SHIFT)
+#define TCR_TG1_4K (UL(2) << TCR_TG1_SHIFT)
+#define TCR_TG1_64K (UL(3) << TCR_TG1_SHIFT)
+
#define TCR_ASID16 (UL(1) << 36)
#define TCR_TBI0 (UL(1) << 37)
#define TCR_HA (UL(1) << 39)
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index 2da46ae..46472a9 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -300,6 +300,8 @@
#define pmd_mkyoung(pmd) pte_pmd(pte_mkyoung(pmd_pte(pmd)))
#define pmd_mknotpresent(pmd) (__pmd(pmd_val(pmd) & ~PMD_SECT_VALID))
+#define pmd_thp_or_huge(pmd) (pmd_huge(pmd) || pmd_trans_huge(pmd))
+
#define __HAVE_ARCH_PMD_WRITE
#define pmd_write(pmd) pte_write(pmd_pte(pmd))
@@ -314,11 +316,6 @@
#define set_pmd_at(mm, addr, pmdp, pmd) set_pte_at(mm, addr, (pte_t *)pmdp, pmd_pte(pmd))
-static inline int has_transparent_hugepage(void)
-{
- return 1;
-}
-
#define __pgprot_modify(prot,mask,bits) \
__pgprot((pgprot_val(prot) & ~(mask)) | (bits))
@@ -554,14 +551,12 @@
* Atomic pte/pmd modifications.
*/
#define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG
-static inline int ptep_test_and_clear_young(struct vm_area_struct *vma,
- unsigned long address,
- pte_t *ptep)
+static inline int __ptep_test_and_clear_young(pte_t *ptep)
{
pteval_t pteval;
unsigned int tmp, res;
- asm volatile("// ptep_test_and_clear_young\n"
+ asm volatile("// __ptep_test_and_clear_young\n"
" prfm pstl1strm, %2\n"
"1: ldxr %0, %2\n"
" ubfx %w3, %w0, %5, #1 // extract PTE_AF (young)\n"
@@ -574,6 +569,13 @@
return res;
}
+static inline int ptep_test_and_clear_young(struct vm_area_struct *vma,
+ unsigned long address,
+ pte_t *ptep)
+{
+ return __ptep_test_and_clear_young(ptep);
+}
+
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
#define __HAVE_ARCH_PMDP_TEST_AND_CLEAR_YOUNG
static inline int pmdp_test_and_clear_young(struct vm_area_struct *vma,
diff --git a/arch/arm64/include/asm/stage2_pgtable-nopmd.h b/arch/arm64/include/asm/stage2_pgtable-nopmd.h
new file mode 100644
index 0000000..2656a0f
--- /dev/null
+++ b/arch/arm64/include/asm/stage2_pgtable-nopmd.h
@@ -0,0 +1,42 @@
+/*
+ * Copyright (C) 2016 - ARM Ltd
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __ARM64_S2_PGTABLE_NOPMD_H_
+#define __ARM64_S2_PGTABLE_NOPMD_H_
+
+#include <asm/stage2_pgtable-nopud.h>
+
+#define __S2_PGTABLE_PMD_FOLDED
+
+#define S2_PMD_SHIFT S2_PUD_SHIFT
+#define S2_PTRS_PER_PMD 1
+#define S2_PMD_SIZE (1UL << S2_PMD_SHIFT)
+#define S2_PMD_MASK (~(S2_PMD_SIZE-1))
+
+#define stage2_pud_none(pud) (0)
+#define stage2_pud_present(pud) (1)
+#define stage2_pud_clear(pud) do { } while (0)
+#define stage2_pud_populate(pud, pmd) do { } while (0)
+#define stage2_pmd_offset(pud, address) ((pmd_t *)(pud))
+
+#define stage2_pmd_free(pmd) do { } while (0)
+
+#define stage2_pmd_addr_end(addr, end) (end)
+
+#define stage2_pud_huge(pud) (0)
+#define stage2_pmd_table_empty(pmdp) (0)
+
+#endif
diff --git a/arch/arm64/include/asm/stage2_pgtable-nopud.h b/arch/arm64/include/asm/stage2_pgtable-nopud.h
new file mode 100644
index 0000000..5ee87b5
--- /dev/null
+++ b/arch/arm64/include/asm/stage2_pgtable-nopud.h
@@ -0,0 +1,39 @@
+/*
+ * Copyright (C) 2016 - ARM Ltd
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __ARM64_S2_PGTABLE_NOPUD_H_
+#define __ARM64_S2_PGTABLE_NOPUD_H_
+
+#define __S2_PGTABLE_PUD_FOLDED
+
+#define S2_PUD_SHIFT S2_PGDIR_SHIFT
+#define S2_PTRS_PER_PUD 1
+#define S2_PUD_SIZE (_AC(1, UL) << S2_PUD_SHIFT)
+#define S2_PUD_MASK (~(S2_PUD_SIZE-1))
+
+#define stage2_pgd_none(pgd) (0)
+#define stage2_pgd_present(pgd) (1)
+#define stage2_pgd_clear(pgd) do { } while (0)
+#define stage2_pgd_populate(pgd, pud) do { } while (0)
+
+#define stage2_pud_offset(pgd, address) ((pud_t *)(pgd))
+
+#define stage2_pud_free(x) do { } while (0)
+
+#define stage2_pud_addr_end(addr, end) (end)
+#define stage2_pud_table_empty(pmdp) (0)
+
+#endif
diff --git a/arch/arm64/include/asm/stage2_pgtable.h b/arch/arm64/include/asm/stage2_pgtable.h
new file mode 100644
index 0000000..8b68099
--- /dev/null
+++ b/arch/arm64/include/asm/stage2_pgtable.h
@@ -0,0 +1,142 @@
+/*
+ * Copyright (C) 2016 - ARM Ltd
+ *
+ * stage2 page table helpers
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __ARM64_S2_PGTABLE_H_
+#define __ARM64_S2_PGTABLE_H_
+
+#include <asm/pgtable.h>
+
+/*
+ * The hardware supports concatenation of up to 16 tables at stage2 entry level
+ * and we use the feature whenever possible.
+ *
+ * Now, the minimum number of bits resolved at any level is (PAGE_SHIFT - 3).
+ * On arm64, the smallest PAGE_SIZE supported is 4k, which means
+ * (PAGE_SHIFT - 3) > 4 holds for all page sizes.
+ * This implies, the total number of page table levels at stage2 expected
+ * by the hardware is actually the number of levels required for (KVM_PHYS_SHIFT - 4)
+ * in normal translations(e.g, stage1), since we cannot have another level in
+ * the range (KVM_PHYS_SHIFT, KVM_PHYS_SHIFT - 4).
+ */
+#define STAGE2_PGTABLE_LEVELS ARM64_HW_PGTABLE_LEVELS(KVM_PHYS_SHIFT - 4)
+
+/*
+ * With all the supported VA_BITs and 40bit guest IPA, the following condition
+ * is always true:
+ *
+ * STAGE2_PGTABLE_LEVELS <= CONFIG_PGTABLE_LEVELS
+ *
+ * We base our stage-2 page table walker helpers on this assumption and
+ * fall back to using the host version of the helper wherever possible.
+ * i.e, if a particular level is not folded (e.g, PUD) at stage2, we fall back
+ * to using the host version, since it is guaranteed it is not folded at host.
+ *
+ * If the condition breaks in the future, we can rearrange the host level
+ * definitions and reuse them for stage2. Till then...
+ */
+#if STAGE2_PGTABLE_LEVELS > CONFIG_PGTABLE_LEVELS
+#error "Unsupported combination of guest IPA and host VA_BITS."
+#endif
+
+/* S2_PGDIR_SHIFT is the size mapped by top-level stage2 entry */
+#define S2_PGDIR_SHIFT ARM64_HW_PGTABLE_LEVEL_SHIFT(4 - STAGE2_PGTABLE_LEVELS)
+#define S2_PGDIR_SIZE (_AC(1, UL) << S2_PGDIR_SHIFT)
+#define S2_PGDIR_MASK (~(S2_PGDIR_SIZE - 1))
+
+/*
+ * The number of PTRS across all concatenated stage2 tables given by the
+ * number of bits resolved at the initial level.
+ */
+#define PTRS_PER_S2_PGD (1 << (KVM_PHYS_SHIFT - S2_PGDIR_SHIFT))
+
+/*
+ * KVM_MMU_CACHE_MIN_PAGES is the number of stage2 page table translation
+ * levels in addition to the PGD.
+ */
+#define KVM_MMU_CACHE_MIN_PAGES (STAGE2_PGTABLE_LEVELS - 1)
+
+
+#if STAGE2_PGTABLE_LEVELS > 3
+
+#define S2_PUD_SHIFT ARM64_HW_PGTABLE_LEVEL_SHIFT(1)
+#define S2_PUD_SIZE (_AC(1, UL) << S2_PUD_SHIFT)
+#define S2_PUD_MASK (~(S2_PUD_SIZE - 1))
+
+#define stage2_pgd_none(pgd) pgd_none(pgd)
+#define stage2_pgd_clear(pgd) pgd_clear(pgd)
+#define stage2_pgd_present(pgd) pgd_present(pgd)
+#define stage2_pgd_populate(pgd, pud) pgd_populate(NULL, pgd, pud)
+#define stage2_pud_offset(pgd, address) pud_offset(pgd, address)
+#define stage2_pud_free(pud) pud_free(NULL, pud)
+
+#define stage2_pud_table_empty(pudp) kvm_page_empty(pudp)
+
+static inline phys_addr_t stage2_pud_addr_end(phys_addr_t addr, phys_addr_t end)
+{
+ phys_addr_t boundary = (addr + S2_PUD_SIZE) & S2_PUD_MASK;
+
+ return (boundary - 1 < end - 1) ? boundary : end;
+}
+
+#endif /* STAGE2_PGTABLE_LEVELS > 3 */
+
+
+#if STAGE2_PGTABLE_LEVELS > 2
+
+#define S2_PMD_SHIFT ARM64_HW_PGTABLE_LEVEL_SHIFT(2)
+#define S2_PMD_SIZE (_AC(1, UL) << S2_PMD_SHIFT)
+#define S2_PMD_MASK (~(S2_PMD_SIZE - 1))
+
+#define stage2_pud_none(pud) pud_none(pud)
+#define stage2_pud_clear(pud) pud_clear(pud)
+#define stage2_pud_present(pud) pud_present(pud)
+#define stage2_pud_populate(pud, pmd) pud_populate(NULL, pud, pmd)
+#define stage2_pmd_offset(pud, address) pmd_offset(pud, address)
+#define stage2_pmd_free(pmd) pmd_free(NULL, pmd)
+
+#define stage2_pud_huge(pud) pud_huge(pud)
+#define stage2_pmd_table_empty(pmdp) kvm_page_empty(pmdp)
+
+static inline phys_addr_t stage2_pmd_addr_end(phys_addr_t addr, phys_addr_t end)
+{
+ phys_addr_t boundary = (addr + S2_PMD_SIZE) & S2_PMD_MASK;
+
+ return (boundary - 1 < end - 1) ? boundary : end;
+}
+
+#endif /* STAGE2_PGTABLE_LEVELS > 2 */
+
+#define stage2_pte_table_empty(ptep) kvm_page_empty(ptep)
+
+#if STAGE2_PGTABLE_LEVELS == 2
+#include <asm/stage2_pgtable-nopmd.h>
+#elif STAGE2_PGTABLE_LEVELS == 3
+#include <asm/stage2_pgtable-nopud.h>
+#endif
+
+
+#define stage2_pgd_index(addr) (((addr) >> S2_PGDIR_SHIFT) & (PTRS_PER_S2_PGD - 1))
+
+static inline phys_addr_t stage2_pgd_addr_end(phys_addr_t addr, phys_addr_t end)
+{
+ phys_addr_t boundary = (addr + S2_PGDIR_SIZE) & S2_PGDIR_MASK;
+
+ return (boundary - 1 < end - 1) ? boundary : end;
+}
+
+#endif /* __ARM64_S2_PGTABLE_H_ */
diff --git a/arch/arm64/include/uapi/asm/unistd.h b/arch/arm64/include/uapi/asm/unistd.h
index 1caadc2..043d17a2 100644
--- a/arch/arm64/include/uapi/asm/unistd.h
+++ b/arch/arm64/include/uapi/asm/unistd.h
@@ -13,4 +13,7 @@
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
+
+#define __ARCH_WANT_RENAMEAT
+
#include <asm-generic/unistd.h>
diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
index 48eea68..6cd2612 100644
--- a/arch/arm64/kernel/process.c
+++ b/arch/arm64/kernel/process.c
@@ -200,13 +200,6 @@
__show_regs(regs);
}
-/*
- * Free current thread data structures etc..
- */
-void exit_thread(void)
-{
-}
-
static void tls_thread_flush(void)
{
asm ("msr tpidr_el0, xzr");
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 64fc030..9fefb00 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -95,7 +95,8 @@
};
void *ret;
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
current->mm->context.vdso = (void *)addr;
/* Map vectors page at the high address. */
@@ -163,7 +164,8 @@
/* Be sure to map the data page */
vdso_mapping_len = vdso_text_len + PAGE_SIZE;
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
if (IS_ERR_VALUE(vdso_base)) {
ret = ERR_PTR(vdso_base);
diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index de7450d..aa2e34e 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -22,7 +22,6 @@
config KVM
bool "Kernel-based Virtual Machine (KVM) support"
depends on OF
- depends on !ARM64_16K_PAGES
select MMU_NOTIFIER
select PREEMPT_NOTIFIERS
select ANON_INODES
diff --git a/arch/arm64/kvm/hyp/s2-setup.c b/arch/arm64/kvm/hyp/s2-setup.c
index bcbe761..b81f409 100644
--- a/arch/arm64/kvm/hyp/s2-setup.c
+++ b/arch/arm64/kvm/hyp/s2-setup.c
@@ -66,6 +66,14 @@
val |= 64 - (parange > 40 ? 40 : parange);
/*
+ * Check the availability of Hardware Access Flag / Dirty Bit
+ * Management in ID_AA64MMFR1_EL1 and enable the feature in VTCR_EL2.
+ */
+ tmp = (read_sysreg(id_aa64mmfr1_el1) >> ID_AA64MMFR1_HADBS_SHIFT) & 0xf;
+ if (IS_ENABLED(CONFIG_ARM64_HW_AFDBM) && tmp)
+ val |= VTCR_EL2_HA;
+
+ /*
* Read the VMIDBits bits from ID_AA64MMFR1_EL1 and set the VS
* bit in VTCR_EL2.
*/
diff --git a/arch/arm64/mm/dma-mapping.c b/arch/arm64/mm/dma-mapping.c
index fd8b942..c566ec8 100644
--- a/arch/arm64/mm/dma-mapping.c
+++ b/arch/arm64/mm/dma-mapping.c
@@ -562,8 +562,8 @@
struct page **pages;
pgprot_t prot = __get_dma_pgprot(attrs, PAGE_KERNEL, coherent);
- pages = iommu_dma_alloc(dev, iosize, gfp, ioprot, handle,
- flush_page);
+ pages = iommu_dma_alloc(dev, iosize, gfp, attrs, ioprot,
+ handle, flush_page);
if (!pages)
return NULL;
@@ -947,13 +947,13 @@
#else
static void __iommu_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
- struct iommu_ops *iommu)
+ const struct iommu_ops *iommu)
{ }
#endif /* CONFIG_IOMMU_DMA */
void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
- struct iommu_ops *iommu, bool coherent)
+ const struct iommu_ops *iommu, bool coherent)
{
if (!dev->archdata.dma_ops)
dev->archdata.dma_ops = &swiotlb_dma_ops;
diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c
index 589fd28..aa8aee7 100644
--- a/arch/arm64/mm/hugetlbpage.c
+++ b/arch/arm64/mm/hugetlbpage.c
@@ -307,6 +307,7 @@
} else if (ps == PUD_SIZE) {
hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
} else {
+ hugetlb_bad_size();
pr_err("hugepagesz: Unsupported page size %lu K\n", ps >> 10);
return 0;
}
diff --git a/arch/avr32/Kconfig b/arch/avr32/Kconfig
index 18b8877..7e75d45 100644
--- a/arch/avr32/Kconfig
+++ b/arch/avr32/Kconfig
@@ -4,6 +4,7 @@
# that we usually don't need on AVR32.
select EXPERT
select HAVE_CLK
+ select HAVE_EXIT_THREAD
select HAVE_OPROFILE
select HAVE_KPROBES
select VIRT_TO_BUS
@@ -17,6 +18,7 @@
select GENERIC_CLOCKEVENTS
select HAVE_MOD_ARCH_SPECIFIC
select MODULES_USE_ELF_RELA
+ select HAVE_NMI
help
AVR32 is a high-performance 32-bit RISC microprocessor core,
designed for cost-sensitive embedded applications, with particular
diff --git a/arch/avr32/kernel/process.c b/arch/avr32/kernel/process.c
index 42a53e74..68e5b9d 100644
--- a/arch/avr32/kernel/process.c
+++ b/arch/avr32/kernel/process.c
@@ -62,9 +62,9 @@
/*
* Free current thread data structures etc
*/
-void exit_thread(void)
+void exit_thread(struct task_struct *tsk)
{
- ocd_disable(current);
+ ocd_disable(tsk);
}
void flush_thread(void)
diff --git a/arch/avr32/mach-at32ap/at32ap700x.c b/arch/avr32/mach-at32ap/at32ap700x.c
index bf445aa..00d6dcc 100644
--- a/arch/avr32/mach-at32ap/at32ap700x.c
+++ b/arch/avr32/mach-at32ap/at32ap700x.c
@@ -1365,8 +1365,8 @@
slave->dma_dev = &dw_dmac0_device.dev;
slave->src_id = 0;
slave->dst_id = 1;
- slave->src_master = 1;
- slave->dst_master = 0;
+ slave->m_master = 1;
+ slave->p_master = 0;
data->dma_slave = slave;
data->dma_filter = at32_mci_dma_filter;
@@ -2061,16 +2061,16 @@
if (flags & AC97C_CAPTURE) {
rx_dws->dma_dev = &dw_dmac0_device.dev;
rx_dws->src_id = 3;
- rx_dws->src_master = 0;
- rx_dws->dst_master = 1;
+ rx_dws->m_master = 0;
+ rx_dws->p_master = 1;
}
/* Check if DMA slave interface for playback should be configured. */
if (flags & AC97C_PLAYBACK) {
tx_dws->dma_dev = &dw_dmac0_device.dev;
tx_dws->dst_id = 4;
- tx_dws->src_master = 0;
- tx_dws->dst_master = 1;
+ tx_dws->m_master = 0;
+ tx_dws->p_master = 1;
}
if (platform_device_add_data(pdev, data,
@@ -2141,8 +2141,8 @@
dws->dma_dev = &dw_dmac0_device.dev;
dws->dst_id = 2;
- dws->src_master = 0;
- dws->dst_master = 1;
+ dws->m_master = 0;
+ dws->p_master = 1;
if (platform_device_add_data(pdev, data,
sizeof(struct atmel_abdac_pdata)))
diff --git a/arch/blackfin/Kconfig b/arch/blackfin/Kconfig
index a63c122..28c63fe 100644
--- a/arch/blackfin/Kconfig
+++ b/arch/blackfin/Kconfig
@@ -40,6 +40,7 @@
select HAVE_MOD_ARCH_SPECIFIC
select MODULES_USE_ELF_RELA
select HAVE_DEBUG_STACKOVERFLOW
+ select HAVE_NMI
config GENERIC_CSUM
def_bool y
diff --git a/arch/blackfin/include/asm/processor.h b/arch/blackfin/include/asm/processor.h
index 7acd466..0c265ab 100644
--- a/arch/blackfin/include/asm/processor.h
+++ b/arch/blackfin/include/asm/processor.h
@@ -76,13 +76,6 @@
}
/*
- * Free current thread data structures etc..
- */
-static inline void exit_thread(void)
-{
-}
-
-/*
* Return saved PC of a blocked thread.
*/
#define thread_saved_pc(tsk) (tsk->thread.pc)
diff --git a/arch/c6x/include/uapi/asm/unistd.h b/arch/c6x/include/uapi/asm/unistd.h
index e7d09a6..12d73d9 100644
--- a/arch/c6x/include/uapi/asm/unistd.h
+++ b/arch/c6x/include/uapi/asm/unistd.h
@@ -14,6 +14,7 @@
* more details.
*/
+#define __ARCH_WANT_RENAMEAT
#define __ARCH_WANT_SYS_CLONE
/* Use the standard ABI for syscalls. */
diff --git a/arch/c6x/kernel/process.c b/arch/c6x/kernel/process.c
index 3ae9f5a..0ee7686 100644
--- a/arch/c6x/kernel/process.c
+++ b/arch/c6x/kernel/process.c
@@ -82,10 +82,6 @@
{
}
-void exit_thread(void)
-{
-}
-
/*
* Do necessary setup to start up a newly executed thread.
*/
diff --git a/arch/cris/Kconfig b/arch/cris/Kconfig
index 99bda1b..deba266 100644
--- a/arch/cris/Kconfig
+++ b/arch/cris/Kconfig
@@ -59,6 +59,7 @@
select GENERIC_IOMAP
select MODULES_USE_ELF_RELA
select CLONE_BACKWARDS2
+ select HAVE_EXIT_THREAD if ETRAX_ARCH_V32
select OLD_SIGSUSPEND
select OLD_SIGACTION
select GPIOLIB
@@ -69,6 +70,7 @@
select GENERIC_CLOCKEVENTS if ETRAX_ARCH_V32
select GENERIC_SCHED_CLOCK if ETRAX_ARCH_V32
select HAVE_DEBUG_BUGVERBOSE if ETRAX_ARCH_V32
+ select HAVE_NMI
config HZ
int
diff --git a/arch/cris/arch-v10/kernel/process.c b/arch/cris/arch-v10/kernel/process.c
index 02b7834..96e5afe 100644
--- a/arch/cris/arch-v10/kernel/process.c
+++ b/arch/cris/arch-v10/kernel/process.c
@@ -35,15 +35,6 @@
local_irq_enable();
}
-/*
- * Free current thread data structures etc..
- */
-
-void exit_thread(void)
-{
- /* Nothing needs to be done. */
-}
-
/* if the watchdog is enabled, we can simply disable interrupts and go
* into an eternal loop, and the watchdog will reset the CPU after 0.1s
* if on the other hand the watchdog wasn't enabled, we just enable it and wait
diff --git a/arch/cris/arch-v32/drivers/mach-a3/nandflash.c b/arch/cris/arch-v32/drivers/mach-a3/nandflash.c
index 5aa3f51..3f646c7 100644
--- a/arch/cris/arch-v32/drivers/mach-a3/nandflash.c
+++ b/arch/cris/arch-v32/drivers/mach-a3/nandflash.c
@@ -157,6 +157,7 @@
/* 20 us command delay time */
this->chip_delay = 20;
this->ecc.mode = NAND_ECC_SOFT;
+ this->ecc.algo = NAND_ECC_HAMMING;
/* Enable the following for a flash based bad block table */
/* this->bbt_options = NAND_BBT_USE_FLASH; */
diff --git a/arch/cris/arch-v32/drivers/mach-fs/nandflash.c b/arch/cris/arch-v32/drivers/mach-fs/nandflash.c
index a7c17b0..a745405 100644
--- a/arch/cris/arch-v32/drivers/mach-fs/nandflash.c
+++ b/arch/cris/arch-v32/drivers/mach-fs/nandflash.c
@@ -148,6 +148,7 @@
/* 20 us command delay time */
this->chip_delay = 20;
this->ecc.mode = NAND_ECC_SOFT;
+ this->ecc.algo = NAND_ECC_HAMMING;
/* Enable the following for a flash based bad block table */
/* this->bbt_options = NAND_BBT_USE_FLASH; */
diff --git a/arch/cris/arch-v32/kernel/process.c b/arch/cris/arch-v32/kernel/process.c
index c7ce784..4d1afa9 100644
--- a/arch/cris/arch-v32/kernel/process.c
+++ b/arch/cris/arch-v32/kernel/process.c
@@ -33,9 +33,9 @@
*/
extern void deconfigure_bp(long pid);
-void exit_thread(void)
+void exit_thread(struct task_struct *tsk)
{
- deconfigure_bp(current->pid);
+ deconfigure_bp(tsk->pid);
}
/*
diff --git a/arch/frv/include/asm/processor.h b/arch/frv/include/asm/processor.h
index ae8d423..73f0a79 100644
--- a/arch/frv/include/asm/processor.h
+++ b/arch/frv/include/asm/processor.h
@@ -97,13 +97,6 @@
#define forget_segments() do { } while (0)
/*
- * Free current thread data structures etc..
- */
-static inline void exit_thread(void)
-{
-}
-
-/*
* Return saved PC of a blocked thread.
*/
extern unsigned long thread_saved_pc(struct task_struct *tsk);
diff --git a/arch/h8300/Kconfig b/arch/h8300/Kconfig
index 986ea84..aa232de 100644
--- a/arch/h8300/Kconfig
+++ b/arch/h8300/Kconfig
@@ -20,6 +20,7 @@
select HAVE_KERNEL_GZIP
select HAVE_KERNEL_LZO
select HAVE_ARCH_KGDB
+ select CPU_NO_EFFICIENT_FFS
config RWSEM_GENERIC_SPINLOCK
def_bool y
diff --git a/arch/h8300/include/asm/processor.h b/arch/h8300/include/asm/processor.h
index 54e3fd8..111df73 100644
--- a/arch/h8300/include/asm/processor.h
+++ b/arch/h8300/include/asm/processor.h
@@ -111,13 +111,6 @@
}
/*
- * Free current thread data structures etc..
- */
-static inline void exit_thread(void)
-{
-}
-
-/*
* Return saved PC of a blocked thread.
*/
unsigned long thread_saved_pc(struct task_struct *tsk);
diff --git a/arch/h8300/include/uapi/asm/unistd.h b/arch/h8300/include/uapi/asm/unistd.h
index 7a2eb69..7dd20ef 100644
--- a/arch/h8300/include/uapi/asm/unistd.h
+++ b/arch/h8300/include/uapi/asm/unistd.h
@@ -1,3 +1,5 @@
#define __ARCH_NOMMU
+#define __ARCH_WANT_RENAMEAT
+
#include <asm-generic/unistd.h>
diff --git a/arch/hexagon/include/uapi/asm/unistd.h b/arch/hexagon/include/uapi/asm/unistd.h
index ffee405d..2151760 100644
--- a/arch/hexagon/include/uapi/asm/unistd.h
+++ b/arch/hexagon/include/uapi/asm/unistd.h
@@ -27,6 +27,7 @@
*/
#define sys_mmap2 sys_mmap_pgoff
+#define __ARCH_WANT_RENAMEAT
#define __ARCH_WANT_SYS_EXECVE
#define __ARCH_WANT_SYS_CLONE
#define __ARCH_WANT_SYS_VFORK
diff --git a/arch/hexagon/kernel/process.c b/arch/hexagon/kernel/process.c
index a9ebd47..d9edfd3 100644
--- a/arch/hexagon/kernel/process.c
+++ b/arch/hexagon/kernel/process.c
@@ -137,13 +137,6 @@
}
/*
- * Free any architecture-specific thread data structures, etc.
- */
-void exit_thread(void)
-{
-}
-
-/*
* Some archs flush debug and FPU info here
*/
void flush_thread(void)
diff --git a/arch/hexagon/kernel/vdso.c b/arch/hexagon/kernel/vdso.c
index 0bf5a87..3ea96841 100644
--- a/arch/hexagon/kernel/vdso.c
+++ b/arch/hexagon/kernel/vdso.c
@@ -65,7 +65,8 @@
unsigned long vdso_base;
struct mm_struct *mm = current->mm;
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
/* Try to get it loaded right near ld.so/glibc. */
vdso_base = STACK_TOP;
diff --git a/arch/ia64/Kconfig b/arch/ia64/Kconfig
index b534eba..f80758c 100644
--- a/arch/ia64/Kconfig
+++ b/arch/ia64/Kconfig
@@ -18,6 +18,7 @@
select ACPI_SYSTEM_POWER_STATES_SUPPORT if ACPI
select ARCH_MIGHT_HAVE_ACPI_PDC if ACPI
select HAVE_UNSTABLE_SCHED_CLOCK
+ select HAVE_EXIT_THREAD
select HAVE_IDE
select HAVE_OPROFILE
select HAVE_KPROBES
diff --git a/arch/ia64/hp/sim/simserial.c b/arch/ia64/hp/sim/simserial.c
index e70cade..21fd50d 100644
--- a/arch/ia64/hp/sim/simserial.c
+++ b/arch/ia64/hp/sim/simserial.c
@@ -300,7 +300,7 @@
if ((cmd != TIOCGSERIAL) && (cmd != TIOCSSERIAL) &&
(cmd != TIOCSERCONFIG) && (cmd != TIOCSERGSTRUCT) &&
(cmd != TIOCMIWAIT)) {
- if (tty->flags & (1 << TTY_IO_ERROR))
+ if (tty_io_error(tty))
return -EIO;
}
diff --git a/arch/ia64/kernel/mca.c b/arch/ia64/kernel/mca.c
index 2889412..07a4e32 100644
--- a/arch/ia64/kernel/mca.c
+++ b/arch/ia64/kernel/mca.c
@@ -1904,13 +1904,10 @@
unsigned long action,
void *hcpu)
{
- int hotcpu = (unsigned long) hcpu;
-
switch (action) {
case CPU_ONLINE:
case CPU_ONLINE_FROZEN:
- smp_call_function_single(hotcpu, ia64_mca_cmc_vector_adjust,
- NULL, 0);
+ ia64_mca_cmc_vector_adjust(NULL);
break;
}
return NOTIFY_OK;
diff --git a/arch/ia64/kernel/perfmon.c b/arch/ia64/kernel/perfmon.c
index 9cd607b..2436ad5 100644
--- a/arch/ia64/kernel/perfmon.c
+++ b/arch/ia64/kernel/perfmon.c
@@ -4542,8 +4542,8 @@
/*
- * called only from exit_thread(): task == current
- * we come here only if current has a context attached (loaded or masked)
+ * called only from exit_thread()
+ * we come here only if the task has a context attached (loaded or masked)
*/
void
pfm_exit_thread(struct task_struct *task)
diff --git a/arch/ia64/kernel/process.c b/arch/ia64/kernel/process.c
index b515149..aae6c4d 100644
--- a/arch/ia64/kernel/process.c
+++ b/arch/ia64/kernel/process.c
@@ -570,22 +570,22 @@
}
/*
- * Clean up state associated with current thread. This is called when
+ * Clean up state associated with a thread. This is called when
* the thread calls exit().
*/
void
-exit_thread (void)
+exit_thread (struct task_struct *tsk)
{
- ia64_drop_fpu(current);
+ ia64_drop_fpu(tsk);
#ifdef CONFIG_PERFMON
/* if needed, stop monitoring and flush state to perfmon context */
- if (current->thread.pfm_context)
- pfm_exit_thread(current);
+ if (tsk->thread.pfm_context)
+ pfm_exit_thread(tsk);
/* free debug register resources */
- if (current->thread.flags & IA64_THREAD_DBG_VALID)
- pfm_release_debug_registers(current);
+ if (tsk->thread.flags & IA64_THREAD_DBG_VALID)
+ pfm_release_debug_registers(tsk);
#endif
}
diff --git a/arch/ia64/kernel/traps.c b/arch/ia64/kernel/traps.c
index 6f7d4a4..77edd68 100644
--- a/arch/ia64/kernel/traps.c
+++ b/arch/ia64/kernel/traps.c
@@ -548,6 +548,7 @@
return;
}
switch (vector) {
+ default:
case 29:
siginfo.si_code = TRAP_HWBKPT;
#ifdef CONFIG_ITANIUM
diff --git a/arch/ia64/kernel/unaligned.c b/arch/ia64/kernel/unaligned.c
index e7ae608..7f0d316 100644
--- a/arch/ia64/kernel/unaligned.c
+++ b/arch/ia64/kernel/unaligned.c
@@ -1378,6 +1378,7 @@
* extract the instruction from the bundle given the slot number
*/
switch (ipsr->ri) {
+ default:
case 0: u.l = (bundle[0] >> 5); break;
case 1: u.l = (bundle[0] >> 46) | (bundle[1] << 18); break;
case 2: u.l = (bundle[1] >> 23); break;
diff --git a/arch/ia64/sn/kernel/io_acpi_init.c b/arch/ia64/sn/kernel/io_acpi_init.c
index 231234c..c31fe63 100644
--- a/arch/ia64/sn/kernel/io_acpi_init.c
+++ b/arch/ia64/sn/kernel/io_acpi_init.c
@@ -426,7 +426,6 @@
void
sn_acpi_slot_fixup(struct pci_dev *dev)
{
- void __iomem *addr;
struct pcidev_info *pcidev_info = NULL;
struct sn_irq_info *sn_irq_info = NULL;
struct resource *res;
diff --git a/arch/ia64/sn/kernel/io_init.c b/arch/ia64/sn/kernel/io_init.c
index c15a41e..d63809a 100644
--- a/arch/ia64/sn/kernel/io_init.c
+++ b/arch/ia64/sn/kernel/io_init.c
@@ -151,7 +151,7 @@
{
int idx;
struct resource *res;
- unsigned long addr, size;
+ unsigned long size;
struct pcidev_info *pcidev_info;
struct sn_irq_info *sn_irq_info;
int status;
@@ -186,7 +186,7 @@
continue;
res->start = pcidev_info->pdi_pio_mapped_addr[idx];
- res->end = addr + size;
+ res->end = res->start + size;
/*
* if it's already in the device structure, remove it before
diff --git a/arch/ia64/sn/kernel/sn2/sn2_smp.c b/arch/ia64/sn/kernel/sn2/sn2_smp.c
index f9c8d9f..c98dc96 100644
--- a/arch/ia64/sn/kernel/sn2/sn2_smp.c
+++ b/arch/ia64/sn/kernel/sn2/sn2_smp.c
@@ -54,7 +54,7 @@
volatile unsigned long *, unsigned long,
volatile unsigned long *, unsigned long);
void
-sn2_ptc_deadlock_recovery(short *, short, short, int,
+sn2_ptc_deadlock_recovery(nodemask_t, short, short, int,
volatile unsigned long *, unsigned long,
volatile unsigned long *, unsigned long);
@@ -169,7 +169,7 @@
int use_cpu_ptcga;
volatile unsigned long *ptc0, *ptc1;
unsigned long itc, itc2, flags, data0 = 0, data1 = 0, rr_value, old_rr = 0;
- short nasids[MAX_NUMNODES], nix;
+ short nix;
nodemask_t nodes_flushed;
int active, max_active, deadlock, flush_opt = sn2_flush_opt;
@@ -218,9 +218,7 @@
}
itc = ia64_get_itc();
- nix = 0;
- for_each_node_mask(cnode, nodes_flushed)
- nasids[nix++] = cnodeid_to_nasid(cnode);
+ nix = nodes_weight(nodes_flushed);
rr_value = (mm->context << 3) | REGION_NUMBER(start);
@@ -270,8 +268,10 @@
data0 = (data0 & ~SH2_PTC_ADDR_MASK) | (start & SH2_PTC_ADDR_MASK);
deadlock = 0;
active = 0;
- for (ibegin = 0, i = 0; i < nix; i++) {
- nasid = nasids[i];
+ ibegin = 0;
+ i = 0;
+ for_each_node_mask(cnode, nodes_flushed) {
+ nasid = cnodeid_to_nasid(cnode);
if (use_cpu_ptcga && unlikely(nasid == mynasid)) {
ia64_ptcga(start, nbits << 2);
ia64_srlz_i();
@@ -286,13 +286,14 @@
if ((deadlock = wait_piowc())) {
if (flush_opt == 1)
goto done;
- sn2_ptc_deadlock_recovery(nasids, ibegin, i, mynasid, ptc0, data0, ptc1, data1);
+ sn2_ptc_deadlock_recovery(nodes_flushed, ibegin, i, mynasid, ptc0, data0, ptc1, data1);
if (reset_max_active_on_deadlock())
max_active = 1;
}
active = 0;
ibegin = i + 1;
}
+ i++;
}
start += (1UL << nbits);
} while (start < end);
@@ -327,11 +328,12 @@
*/
void
-sn2_ptc_deadlock_recovery(short *nasids, short ib, short ie, int mynasid,
+sn2_ptc_deadlock_recovery(nodemask_t nodes, short ib, short ie, int mynasid,
volatile unsigned long *ptc0, unsigned long data0,
volatile unsigned long *ptc1, unsigned long data1)
{
short nasid, i;
+ int cnode;
unsigned long *piows, zeroval, n;
__this_cpu_inc(ptcstats.deadlocks);
@@ -339,17 +341,26 @@
piows = (unsigned long *) pda->pio_write_status_addr;
zeroval = pda->pio_write_status_val;
+ i = 0;
+ for_each_node_mask(cnode, nodes) {
+ if (i < ib)
+ goto next;
- for (i=ib; i <= ie; i++) {
- nasid = nasids[i];
+ if (i > ie)
+ break;
+
+ nasid = cnodeid_to_nasid(cnode);
if (local_node_uses_ptc_ga(is_shub1()) && nasid == mynasid)
- continue;
+ goto next;
+
ptc0 = CHANGE_NASID(nasid, ptc0);
if (ptc1)
ptc1 = CHANGE_NASID(nasid, ptc1);
n = sn2_ptc_deadlock_recovery_core(ptc0, data0, ptc1, data1, piows, zeroval);
__this_cpu_add(ptcstats.deadlocks2, n);
+next:
+ i++;
}
}
diff --git a/arch/m32r/Kconfig b/arch/m32r/Kconfig
index c82b292..3cc8498 100644
--- a/arch/m32r/Kconfig
+++ b/arch/m32r/Kconfig
@@ -17,6 +17,7 @@
select ARCH_USES_GETTIMEOFFSET
select MODULES_USE_ELF_RELA
select HAVE_DEBUG_STACKOVERFLOW
+ select CPU_NO_EFFICIENT_FFS
config SBUS
bool
diff --git a/arch/m32r/kernel/process.c b/arch/m32r/kernel/process.c
index e69221d..a88b1f0 100644
--- a/arch/m32r/kernel/process.c
+++ b/arch/m32r/kernel/process.c
@@ -101,15 +101,6 @@
#endif
}
-/*
- * Free current thread data structures etc..
- */
-void exit_thread(void)
-{
- /* Nothing to do. */
- DPRINTK("pid = %d\n", current->pid);
-}
-
void flush_thread(void)
{
DPRINTK("pid = %d\n", current->pid);
diff --git a/arch/m32r/kernel/smp.c b/arch/m32r/kernel/smp.c
index 62d6961..564052e 100644
--- a/arch/m32r/kernel/smp.c
+++ b/arch/m32r/kernel/smp.c
@@ -164,6 +164,7 @@
spin_unlock(&flushcache_lock);
preempt_enable();
}
+EXPORT_SYMBOL(smp_flush_cache_all);
void smp_flush_cache_all_interrupt(void)
{
diff --git a/arch/m68k/Kconfig.cpu b/arch/m68k/Kconfig.cpu
index c1beb5a..8ace920 100644
--- a/arch/m68k/Kconfig.cpu
+++ b/arch/m68k/Kconfig.cpu
@@ -40,6 +40,7 @@
select CPU_HAS_NO_MULDIV64
select CPU_HAS_NO_UNALIGNED
select GENERIC_CSUM
+ select CPU_NO_EFFICIENT_FFS
help
The Freescale (was Motorola) 68000 CPU is the first generation of
the well known M68K family of processors. The CPU core as well as
@@ -51,6 +52,7 @@
bool
select CPU_HAS_NO_BITFIELDS
select CPU_HAS_NO_UNALIGNED
+ select CPU_NO_EFFICIENT_FFS
help
The Freescale (was then Motorola) CPU32 is a CPU core that is
based on the 68020 processor. For the most part it is used in
@@ -130,6 +132,7 @@
depends on !MMU
select COLDFIRE_SW_A7
select HAVE_MBAR
+ select CPU_NO_EFFICIENT_FFS
help
Motorola ColdFire 5206 processor support.
@@ -138,6 +141,7 @@
depends on !MMU
select COLDFIRE_SW_A7
select HAVE_MBAR
+ select CPU_NO_EFFICIENT_FFS
help
Motorola ColdFire 5206e processor support.
@@ -163,6 +167,7 @@
depends on !MMU
select COLDFIRE_SW_A7
select HAVE_MBAR
+ select CPU_NO_EFFICIENT_FFS
help
Motorola ColdFire 5249 processor support.
@@ -171,6 +176,7 @@
depends on !MMU
select COLDFIRE_SW_A7
select HAVE_MBAR
+ select CPU_NO_EFFICIENT_FFS
help
Freescale (Motorola) Coldfire 5251/5253 processor support.
@@ -189,6 +195,7 @@
depends on !MMU
select COLDFIRE_SW_A7
select HAVE_MBAR
+ select CPU_NO_EFFICIENT_FFS
help
Motorola ColdFire 5272 processor support.
@@ -217,6 +224,7 @@
select COLDFIRE_SW_A7
select HAVE_CACHE_CB
select HAVE_MBAR
+ select CPU_NO_EFFICIENT_FFS
help
Motorola ColdFire 5307 processor support.
@@ -242,6 +250,7 @@
select COLDFIRE_SW_A7
select HAVE_CACHE_CB
select HAVE_MBAR
+ select CPU_NO_EFFICIENT_FFS
help
Motorola ColdFire 5407 processor support.
@@ -251,6 +260,7 @@
select MMU_COLDFIRE if MMU
select HAVE_CACHE_CB
select HAVE_MBAR
+ select CPU_NO_EFFICIENT_FFS
help
Freescale ColdFire 5470/5471/5472/5473/5474/5475 processor support.
@@ -260,6 +270,7 @@
select M54xx
select HAVE_CACHE_CB
select HAVE_MBAR
+ select CPU_NO_EFFICIENT_FFS
help
Freescale ColdFire 5480/5481/5482/5483/5484/5485 processor support.
diff --git a/arch/m68k/include/asm/processor.h b/arch/m68k/include/asm/processor.h
index 20dda1d..a6ce2ec 100644
--- a/arch/m68k/include/asm/processor.h
+++ b/arch/m68k/include/asm/processor.h
@@ -153,13 +153,6 @@
{
}
-/*
- * Free current thread data structures etc..
- */
-static inline void exit_thread(void)
-{
-}
-
extern unsigned long thread_saved_pc(struct task_struct *tsk);
unsigned long get_wchan(struct task_struct *p);
diff --git a/arch/metag/Kconfig b/arch/metag/Kconfig
index a0fa88d..5b7a45d 100644
--- a/arch/metag/Kconfig
+++ b/arch/metag/Kconfig
@@ -11,6 +11,7 @@
select HAVE_DEBUG_KMEMLEAK
select HAVE_DEBUG_STACKOVERFLOW
select HAVE_DYNAMIC_FTRACE
+ select HAVE_EXIT_THREAD
select HAVE_FTRACE_MCOUNT_RECORD
select HAVE_FUNCTION_TRACER
select HAVE_KERNEL_BZIP2
@@ -29,6 +30,7 @@
select OF
select OF_EARLY_FLATTREE
select SPARSE_IRQ
+ select CPU_NO_EFFICIENT_FFS
config STACKTRACE_SUPPORT
def_bool y
diff --git a/arch/metag/include/asm/processor.h b/arch/metag/include/asm/processor.h
index 0838ca6..a0333eb 100644
--- a/arch/metag/include/asm/processor.h
+++ b/arch/metag/include/asm/processor.h
@@ -134,8 +134,6 @@
#define copy_segments(tsk, mm) do { } while (0)
#define release_segments(mm) do { } while (0)
-extern void exit_thread(void);
-
/*
* Return saved PC of a blocked thread.
*/
diff --git a/arch/metag/include/uapi/asm/unistd.h b/arch/metag/include/uapi/asm/unistd.h
index b80b8e8..459b6ec 100644
--- a/arch/metag/include/uapi/asm/unistd.h
+++ b/arch/metag/include/uapi/asm/unistd.h
@@ -7,6 +7,8 @@
* (at your option) any later version.
*/
+#define __ARCH_WANT_RENAMEAT
+
/* Use the standard ABI for syscalls. */
#include <asm-generic/unistd.h>
diff --git a/arch/metag/kernel/process.c b/arch/metag/kernel/process.c
index 7f54618..3506279 100644
--- a/arch/metag/kernel/process.c
+++ b/arch/metag/kernel/process.c
@@ -345,10 +345,10 @@
/*
* Free current thread data structures etc.
*/
-void exit_thread(void)
+void exit_thread(struct task_struct *tsk)
{
- clear_fpu(¤t->thread);
- clear_dsp(¤t->thread);
+ clear_fpu(&tsk->thread);
+ clear_dsp(&tsk->thread);
}
/* TODO: figure out how to unwind the kernel stack here to figure out
diff --git a/arch/metag/mm/hugetlbpage.c b/arch/metag/mm/hugetlbpage.c
index b38700ae..db1b7da 100644
--- a/arch/metag/mm/hugetlbpage.c
+++ b/arch/metag/mm/hugetlbpage.c
@@ -239,6 +239,7 @@
if (ps == (1 << HPAGE_SHIFT)) {
hugetlb_add_hstate(HPAGE_SHIFT - PAGE_SHIFT);
} else {
+ hugetlb_bad_size();
pr_err("hugepagesz: Unsupported page size %lu M\n",
ps >> 20);
return 0;
diff --git a/arch/microblaze/Kconfig b/arch/microblaze/Kconfig
index 3d793b5..f17c3a4 100644
--- a/arch/microblaze/Kconfig
+++ b/arch/microblaze/Kconfig
@@ -32,6 +32,7 @@
select OF_EARLY_FLATTREE
select TRACING_SUPPORT
select VIRT_TO_BUS
+ select CPU_NO_EFFICIENT_FFS
config SWAP
def_bool n
diff --git a/arch/microblaze/include/asm/processor.h b/arch/microblaze/include/asm/processor.h
index 497a988..c38d0dd 100644
--- a/arch/microblaze/include/asm/processor.h
+++ b/arch/microblaze/include/asm/processor.h
@@ -70,11 +70,6 @@
{
}
-/* Free all resources held by a thread. */
-static inline void exit_thread(void)
-{
-}
-
extern unsigned long thread_saved_pc(struct task_struct *t);
extern unsigned long get_wchan(struct task_struct *p);
@@ -127,11 +122,6 @@
{
}
-/* Free current thread data structures etc. */
-static inline void exit_thread(void)
-{
-}
-
/* Return saved (kernel) PC of a blocked thread. */
# define thread_saved_pc(tsk) \
((tsk)->thread.regs ? (tsk)->thread.regs->r15 : 0)
diff --git a/arch/microblaze/include/asm/unistd.h b/arch/microblaze/include/asm/unistd.h
index 76ed17b..805ae5d 100644
--- a/arch/microblaze/include/asm/unistd.h
+++ b/arch/microblaze/include/asm/unistd.h
@@ -38,6 +38,6 @@
#endif /* __ASSEMBLY__ */
-#define __NR_syscalls 389
+#define __NR_syscalls 392
#endif /* _ASM_MICROBLAZE_UNISTD_H */
diff --git a/arch/microblaze/include/uapi/asm/unistd.h b/arch/microblaze/include/uapi/asm/unistd.h
index 32850c7..a8bd3fa 100644
--- a/arch/microblaze/include/uapi/asm/unistd.h
+++ b/arch/microblaze/include/uapi/asm/unistd.h
@@ -404,5 +404,8 @@
#define __NR_memfd_create 386
#define __NR_bpf 387
#define __NR_execveat 388
+#define __NR_userfaultfd 389
+#define __NR_membarrier 390
+#define __NR_mlock2 391
#endif /* _UAPI_ASM_MICROBLAZE_UNISTD_H */
diff --git a/arch/microblaze/kernel/syscall_table.S b/arch/microblaze/kernel/syscall_table.S
index 29c8568..6b3dd99 100644
--- a/arch/microblaze/kernel/syscall_table.S
+++ b/arch/microblaze/kernel/syscall_table.S
@@ -389,3 +389,6 @@
.long sys_memfd_create
.long sys_bpf
.long sys_execveat
+ .long sys_userfaultfd
+ .long sys_membarrier /* 390 */
+ .long sys_mlock2
diff --git a/arch/microblaze/pci/pci-common.c b/arch/microblaze/pci/pci-common.c
index 35654be..14cba60 100644
--- a/arch/microblaze/pci/pci-common.c
+++ b/arch/microblaze/pci/pci-common.c
@@ -48,6 +48,8 @@
resource_size_t isa_mem_base;
unsigned long isa_io_base;
+EXPORT_SYMBOL(isa_io_base);
+
static int pci_bus_count;
struct pci_controller *pcibios_alloc_controller(struct device_node *dev)
diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index d2ac117..4693884 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -48,6 +48,7 @@
select GENERIC_SCHED_CLOCK if !CAVIUM_OCTEON_SOC
select GENERIC_CMOS_UPDATE
select HAVE_MOD_ARCH_SPECIFIC
+ select HAVE_NMI
select VIRT_TO_BUS
select MODULES_USE_ELF_REL if MODULES
select MODULES_USE_ELF_RELA if MODULES && 64BIT
@@ -62,6 +63,7 @@
select HAVE_IRQ_TIME_ACCOUNTING
select GENERIC_TIME_VSYSCALL
select ARCH_CLOCKSOURCE_DATA
+ select HANDLE_DOMAIN_IRQ
menu "Machine selection"
@@ -137,7 +139,7 @@
select SYS_SUPPORTS_32BIT_KERNEL
select SYS_SUPPORTS_BIG_ENDIAN
select SYS_SUPPORTS_MIPS16
- select SYS_SUPPORTS_ZBOOT
+ select SYS_SUPPORTS_ZBOOT_UART_PROM
select USE_OF
help
Support for the Atheros AR71XX/AR724X/AR913X SoCs.
@@ -194,6 +196,7 @@
select GPIOLIB
select LEDS_GPIO_REGISTER
select BCM47XX_NVRAM
+ select BCM47XX_SPROM
help
Support for BCM47XX based boards
@@ -471,6 +474,7 @@
select SYS_SUPPORTS_MULTITHREADING
select SYS_SUPPORTS_SMARTMIPS
select SYS_SUPPORTS_ZBOOT
+ select SYS_SUPPORTS_RELOCATABLE
select USE_OF
select ZONE_DMA32 if 64BIT
select BUILTIN_DTB
@@ -505,6 +509,7 @@
select MIPS_MSC
select SYS_HAS_CPU_MIPS32_R1
select SYS_HAS_CPU_MIPS32_R2
+ select SYS_HAS_CPU_MIPS32_R6
select SYS_HAS_CPU_MIPS64_R1
select SYS_HAS_EARLY_PRINTK
select SYS_SUPPORTS_32BIT_KERNEL
@@ -514,6 +519,7 @@
select SYS_SUPPORTS_SMARTMIPS
select SYS_SUPPORTS_MICROMIPS
select SYS_SUPPORTS_MIPS16
+ select SYS_SUPPORTS_RELOCATABLE
select USB_EHCI_BIG_ENDIAN_DESC
select USB_EHCI_BIG_ENDIAN_MMIO
select USE_OF
@@ -1153,6 +1159,13 @@
config HOLES_IN_ZONE
bool
+config SYS_SUPPORTS_RELOCATABLE
+ bool
+ help
+ Selected if the platform supports relocating the kernel.
+ The platform must provide plat_get_fdt() if it selects CONFIG_USE_OF
+ to allow access to command line and entropy sources.
+
#
# Endianness selection. Sufficiently obscure so many users don't know what to
# answer,so we try hard to limit the available choices. Also the use of a
@@ -1340,11 +1353,30 @@
select CPU_SUPPORTS_HUGEPAGES
select WEAK_ORDERING
select WEAK_REORDERING_BEYOND_LLSC
+ select MIPS_PGD_C0_CONTEXT
select GPIOLIB
help
The Loongson 3 processor implements the MIPS64R2 instruction
set with many extensions.
+config LOONGSON3_ENHANCEMENT
+ bool "New Loongson 3 CPU Enhancements"
+ default n
+ select CPU_MIPSR2
+ select CPU_HAS_PREFETCH
+ depends on CPU_LOONGSON3
+ help
+ New Loongson 3 CPU (since Loongson-3A R2, as opposed to Loongson-3A
+ R1, Loongson-3B R1 and Loongson-3B R2) has many enhancements, such as
+ FTLB, L1-VCache, EI/DI/Wait/Prefetch instruction, DSP/DSPv2 ASE, User
+ Local register, Read-Inhibit/Execute-Inhibit, SFB (Store Fill Buffer),
+ Fast TLB refill support, etc.
+
+ This option enable those enhancements which are not probed at run
+ time. If you want a generic kernel to run on all Loongson 3 machines,
+ please say 'N' here. If you want a high-performance kernel to run on
+ new Loongson 3 machines only, please say 'Y' here.
+
config CPU_LOONGSON2E
bool "Loongson 2E"
depends on SYS_HAS_CPU_LOONGSON2E
@@ -1373,6 +1405,8 @@
bool "Loongson 1B"
depends on SYS_HAS_CPU_LOONGSON1B
select CPU_LOONGSON1
+ select ARCH_WANT_OPTIONAL_GPIOLIB
+ select LEDS_GPIO_REGISTER
help
The Loongson 1B is a 32-bit SoC, which implements the MIPS32
release 2 instruction set.
@@ -1671,6 +1705,7 @@
select CPU_HAS_PREFETCH
select CPU_MIPSR2
select CPU_SUPPORTS_HUGEPAGES
+ select MIPS_ASID_BITS_VARIABLE
help
Netlogic Microsystems XLP processors.
endchoice
@@ -1796,6 +1831,7 @@
select MIPS_L1_CACHE_SHIFT_6
select SYS_SUPPORTS_SMP
select SYS_SUPPORTS_HOTPLUG_CPU
+ select CPU_HAS_RIXI
config CPU_BMIPS5000
bool
@@ -1803,10 +1839,12 @@
select MIPS_L1_CACHE_SHIFT_7
select SYS_SUPPORTS_SMP
select SYS_SUPPORTS_HOTPLUG_CPU
+ select CPU_HAS_RIXI
config SYS_HAS_CPU_LOONGSON3
bool
select CPU_SUPPORTS_CPUFREQ
+ select CPU_HAS_RIXI
config SYS_HAS_CPU_LOONGSON2E
bool
@@ -1959,11 +1997,15 @@
config CPU_MIPSR2
bool
default y if CPU_MIPS32_R2 || CPU_MIPS64_R2 || CPU_CAVIUM_OCTEON
+ select CPU_HAS_RIXI
select MIPS_SPRAM
config CPU_MIPSR6
bool
default y if CPU_MIPS32_R6 || CPU_MIPS64_R6
+ select CPU_HAS_RIXI
+ select HAVE_ARCH_BITREVERSE
+ select MIPS_ASID_BITS_VARIABLE
select MIPS_SPRAM
config EVA
@@ -1997,7 +2039,7 @@
#
config HARDWARE_WATCHPOINTS
bool
- default y if CPU_MIPSR1 || CPU_MIPSR2
+ default y if CPU_MIPSR1 || CPU_MIPSR2 || CPU_MIPSR6
menu "Kernel type"
@@ -2040,6 +2082,16 @@
emulation when determining guest CPU Frequency. Instead, the guest's
timer frequency is specified directly.
+config MIPS_VA_BITS_48
+ bool "48 bits virtual memory"
+ depends on 64BIT
+ help
+ Support a maximum at least 48 bits of application virtual memory.
+ Default is 40 bits or less, depending on the CPU.
+ This option result in a small memory overhead for page tables.
+ This option is only supported with 16k and 64k page sizes.
+ If unsure, say N.
+
choice
prompt "Kernel page size"
default PAGE_SIZE_4KB
@@ -2047,6 +2099,7 @@
config PAGE_SIZE_4KB
bool "4kB"
depends on !CPU_LOONGSON2 && !CPU_LOONGSON3
+ depends on !MIPS_VA_BITS_48
help
This option select the standard 4kB Linux page size. On some
R3000-family processors this is the only available page size. Using
@@ -2056,6 +2109,7 @@
config PAGE_SIZE_8KB
bool "8kB"
depends on CPU_R8000 || CPU_CAVIUM_OCTEON
+ depends on !MIPS_VA_BITS_48
help
Using 8kB page size will result in higher performance kernel at
the price of higher memory consumption. This option is available
@@ -2074,6 +2128,7 @@
config PAGE_SIZE_32KB
bool "32kB"
depends on CPU_CAVIUM_OCTEON
+ depends on !MIPS_VA_BITS_48
help
Using 32kB page size will result in higher performance kernel at
the price of higher memory consumption. This option is available
@@ -2278,7 +2333,7 @@
config MIPS_CPS
bool "MIPS Coherent Processing System support"
- depends on SYS_SUPPORTS_MIPS_CPS && !CPU_MIPSR6
+ depends on SYS_SUPPORTS_MIPS_CPS
select MIPS_CM
select MIPS_CPC
select MIPS_CPS_PM if HOTPLUG_CPU
@@ -2369,6 +2424,9 @@
config XKS01
bool
+config CPU_HAS_RIXI
+ bool
+
#
# Vectored interrupt mode is an R2 feature
#
@@ -2399,6 +2457,21 @@
config CPU_R4400_WORKAROUNDS
bool
+config MIPS_ASID_SHIFT
+ int
+ default 6 if CPU_R3000 || CPU_TX39XX
+ default 4 if CPU_R8000
+ default 0
+
+config MIPS_ASID_BITS
+ int
+ default 0 if MIPS_ASID_BITS_VARIABLE
+ default 6 if CPU_R3000 || CPU_TX39XX
+ default 8
+
+config MIPS_ASID_BITS_VARIABLE
+ bool
+
#
# - Highmem only makes sense for the 32-bit kernel.
# - The current highmem code will only work properly on physically indexed
@@ -2468,6 +2541,61 @@
config SYS_SUPPORTS_NUMA
bool
+config RELOCATABLE
+ bool "Relocatable kernel"
+ depends on SYS_SUPPORTS_RELOCATABLE && (CPU_MIPS32_R2 || CPU_MIPS64_R2 || CPU_MIPS32_R6 || CPU_MIPS64_R6)
+ help
+ This builds a kernel image that retains relocation information
+ so it can be loaded someplace besides the default 1MB.
+ The relocations make the kernel binary about 15% larger,
+ but are discarded at runtime
+
+config RELOCATION_TABLE_SIZE
+ hex "Relocation table size"
+ depends on RELOCATABLE
+ range 0x0 0x01000000
+ default "0x00100000"
+ ---help---
+ A table of relocation data will be appended to the kernel binary
+ and parsed at boot to fix up the relocated kernel.
+
+ This option allows the amount of space reserved for the table to be
+ adjusted, although the default of 1Mb should be ok in most cases.
+
+ The build will fail and a valid size suggested if this is too small.
+
+ If unsure, leave at the default value.
+
+config RANDOMIZE_BASE
+ bool "Randomize the address of the kernel image"
+ depends on RELOCATABLE
+ ---help---
+ Randomizes the physical and virtual address at which the
+ kernel image is loaded, as a security feature that
+ deters exploit attempts relying on knowledge of the location
+ of kernel internals.
+
+ Entropy is generated using any coprocessor 0 registers available.
+
+ The kernel will be offset by up to RANDOMIZE_BASE_MAX_OFFSET.
+
+ If unsure, say N.
+
+config RANDOMIZE_BASE_MAX_OFFSET
+ hex "Maximum kASLR offset" if EXPERT
+ depends on RANDOMIZE_BASE
+ range 0x0 0x40000000 if EVA || 64BIT
+ range 0x0 0x08000000
+ default "0x01000000"
+ ---help---
+ When kASLR is active, this provides the maximum offset that will
+ be applied to the kernel image. It should be set according to the
+ amount of physical RAM available in the target system minus
+ PHYSICAL_START and must be a power of 2.
+
+ This is limited by the size of KSEG0, 256Mb on 32-bit or 1Gb with
+ EVA or 64-bit. The default is 16Mb.
+
config NODES_SHIFT
int
default "6"
@@ -2475,7 +2603,7 @@
config HW_PERF_EVENTS
bool "Enable hardware performance counter support for perf events"
- depends on PERF_EVENTS && OPROFILE=n && (CPU_MIPS32 || CPU_MIPS64 || CPU_R10000 || CPU_SB1 || CPU_CAVIUM_OCTEON || CPU_XLP || CPU_LOONGSON3)
+ depends on PERF_EVENTS && !OPROFILE && (CPU_MIPS32 || CPU_MIPS64 || CPU_R10000 || CPU_SB1 || CPU_CAVIUM_OCTEON || CPU_XLP || CPU_LOONGSON3)
default y
help
Enable hardware performance counter support for perf events. If
@@ -2808,6 +2936,10 @@
config MIPS_CMDLINE_FROM_BOOTLOADER
bool "Bootloader kernel arguments if available"
+
+ config MIPS_CMDLINE_BUILTIN_EXTEND
+ depends on CMDLINE_BOOL
+ bool "Extend builtin kernel arguments with bootloader arguments"
endchoice
endmenu
@@ -2985,6 +3117,7 @@
config BINFMT_ELF32
bool
default y if MIPS32_O32 || MIPS32_N32
+ select ELFCORE
endmenu
diff --git a/arch/mips/Makefile b/arch/mips/Makefile
index e78d60d..efd7a9d 100644
--- a/arch/mips/Makefile
+++ b/arch/mips/Makefile
@@ -12,6 +12,9 @@
# for "archclean" cleaning up for this architecture.
#
+archscripts: scripts_basic
+ $(Q)$(MAKE) $(build)=arch/mips/boot/tools relocs
+
KBUILD_DEFCONFIG := ip22_defconfig
#
@@ -93,6 +96,10 @@
KBUILD_AFLAGS_MODULE += -mlong-calls
KBUILD_CFLAGS_MODULE += -mlong-calls
+ifeq ($(CONFIG_RELOCATABLE),y)
+LDFLAGS_vmlinux += --emit-relocs
+endif
+
#
# pass -msoft-float to GAS if it supports it. However on newer binutils
# (specifically newer than 2.24.51.20140728) we then also need to explicitly
@@ -193,6 +200,8 @@
toolchain-msa := $(call cc-option-yn,$(mips-cflags) -mhard-float -mfp64 -Wa$(comma)-mmsa)
cflags-$(toolchain-msa) += -DTOOLCHAIN_SUPPORTS_MSA
endif
+toolchain-virt := $(call cc-option-yn,$(mips-cflags) -mvirt)
+cflags-$(toolchain-virt) += -DTOOLCHAIN_SUPPORTS_VIRT
cflags-$(CONFIG_MIPS_COMPACT_BRANCHES_NEVER) += -mcompact-branches=never
cflags-$(CONFIG_MIPS_COMPACT_BRANCHES_OPTIMAL) += -mcompact-branches=optimal
@@ -310,6 +319,10 @@
$(bootvars-y) $@
endif
+CMD_RELOCS = arch/mips/boot/tools/relocs
+quiet_cmd_relocs = RELOCS $<
+ cmd_relocs = $(CMD_RELOCS) $<
+
#
# Some machines like the Indy need 32-bit ELF binaries for booting purposes.
# Other need ECOFF, so we build a 32-bit ELF binary for them which we then
@@ -318,6 +331,11 @@
quiet_cmd_32 = OBJCOPY $@
cmd_32 = $(OBJCOPY) -O $(32bit-bfd) $(OBJCOPYFLAGS) $< $@
vmlinux.32: vmlinux
+ifeq ($(CONFIG_RELOCATABLE)$(CONFIG_64BIT),yy)
+# Currently, objcopy fails to handle the relocations in the elf64
+# So the relocs tool must be run here to remove them first
+ $(call cmd,relocs)
+endif
$(call cmd,32)
#
@@ -333,6 +351,9 @@
# boot
$(boot-y): $(vmlinux-32) FORCE
+ifeq ($(CONFIG_RELOCATABLE)$(CONFIG_32BIT),yy)
+ $(call cmd,relocs)
+endif
$(Q)$(MAKE) $(build)=arch/mips/boot VMLINUX=$(vmlinux-32) \
$(bootvars-y) arch/mips/boot/$@
@@ -385,6 +406,7 @@
archclean:
$(Q)$(MAKE) $(clean)=arch/mips/boot
$(Q)$(MAKE) $(clean)=arch/mips/boot/compressed
+ $(Q)$(MAKE) $(clean)=arch/mips/boot/tools
$(Q)$(MAKE) $(clean)=arch/mips/lasat
define archhelp
diff --git a/arch/mips/alchemy/common/clock.c b/arch/mips/alchemy/common/clock.c
index bd34f40..7ba7ea0 100644
--- a/arch/mips/alchemy/common/clock.c
+++ b/arch/mips/alchemy/common/clock.c
@@ -1043,8 +1043,7 @@
/* Root of the Alchemy clock tree: external 12MHz crystal osc */
c = clk_register_fixed_rate(NULL, ALCHEMY_ROOT_CLK, NULL,
- CLK_IS_ROOT,
- ALCHEMY_ROOTCLK_RATE);
+ 0, ALCHEMY_ROOTCLK_RATE);
ERRCK(c)
/* CPU core clock */
diff --git a/arch/mips/ath79/Kconfig b/arch/mips/ath79/Kconfig
index 13c04cf..dfc6020 100644
--- a/arch/mips/ath79/Kconfig
+++ b/arch/mips/ath79/Kconfig
@@ -71,18 +71,6 @@
Say 'Y' here if you want your kernel to support the
Ubiquiti Networks XM (rev 1.0) board.
-choice
- prompt "Build a DTB in the kernel"
- optional
- help
- Select a devicetree that should be built into the kernel.
-
- config DTB_TL_WR1043ND_V1
- bool "TL-WR1043ND Version 1"
- select BUILTIN_DTB
- select SOC_AR913X
-endchoice
-
endmenu
config SOC_AR71XX
diff --git a/arch/mips/ath79/clock.c b/arch/mips/ath79/clock.c
index 618dfd7..2e73784 100644
--- a/arch/mips/ath79/clock.c
+++ b/arch/mips/ath79/clock.c
@@ -18,17 +18,21 @@
#include <linux/clk.h>
#include <linux/clkdev.h>
#include <linux/clk-provider.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <dt-bindings/clock/ath79-clk.h>
#include <asm/div64.h>
#include <asm/mach-ath79/ath79.h>
#include <asm/mach-ath79/ar71xx_regs.h>
#include "common.h"
+#include "machtypes.h"
#define AR71XX_BASE_FREQ 40000000
#define AR724X_BASE_FREQ 40000000
-static struct clk *clks[3];
+static struct clk *clks[ATH79_CLK_END];
static struct clk_onecell_data clk_data = {
.clks = clks,
.clk_num = ARRAY_SIZE(clks),
@@ -40,7 +44,7 @@
struct clk *clk;
int err;
- clk = clk_register_fixed_rate(NULL, id, NULL, CLK_IS_ROOT, rate);
+ clk = clk_register_fixed_rate(NULL, id, NULL, 0, rate);
if (!clk)
panic("failed to allocate %s clock structure", id);
@@ -78,59 +82,123 @@
ahb_rate = cpu_rate / div;
ath79_add_sys_clkdev("ref", ref_rate);
- clks[0] = ath79_add_sys_clkdev("cpu", cpu_rate);
- clks[1] = ath79_add_sys_clkdev("ddr", ddr_rate);
- clks[2] = ath79_add_sys_clkdev("ahb", ahb_rate);
+ clks[ATH79_CLK_CPU] = ath79_add_sys_clkdev("cpu", cpu_rate);
+ clks[ATH79_CLK_DDR] = ath79_add_sys_clkdev("ddr", ddr_rate);
+ clks[ATH79_CLK_AHB] = ath79_add_sys_clkdev("ahb", ahb_rate);
clk_add_alias("wdt", NULL, "ahb", NULL);
clk_add_alias("uart", NULL, "ahb", NULL);
}
+static struct clk * __init ath79_reg_ffclk(const char *name,
+ const char *parent_name, unsigned int mult, unsigned int div)
+{
+ struct clk *clk;
+
+ clk = clk_register_fixed_factor(NULL, name, parent_name, 0, mult, div);
+ if (!clk)
+ panic("failed to allocate %s clock structure", name);
+
+ return clk;
+}
+
+static void __init ar724x_clk_init(struct clk *ref_clk, void __iomem *pll_base)
+{
+ u32 pll;
+ u32 mult, div, ddr_div, ahb_div;
+
+ pll = __raw_readl(pll_base + AR724X_PLL_REG_CPU_CONFIG);
+
+ mult = ((pll >> AR724X_PLL_FB_SHIFT) & AR724X_PLL_FB_MASK);
+ div = ((pll >> AR724X_PLL_REF_DIV_SHIFT) & AR724X_PLL_REF_DIV_MASK) * 2;
+
+ ddr_div = ((pll >> AR724X_DDR_DIV_SHIFT) & AR724X_DDR_DIV_MASK) + 1;
+ ahb_div = (((pll >> AR724X_AHB_DIV_SHIFT) & AR724X_AHB_DIV_MASK) + 1) * 2;
+
+ clks[ATH79_CLK_CPU] = ath79_reg_ffclk("cpu", "ref", mult, div);
+ clks[ATH79_CLK_DDR] = ath79_reg_ffclk("ddr", "ref", mult, div * ddr_div);
+ clks[ATH79_CLK_AHB] = ath79_reg_ffclk("ahb", "ref", mult, div * ahb_div);
+}
+
static void __init ar724x_clocks_init(void)
{
- unsigned long ref_rate;
- unsigned long cpu_rate;
- unsigned long ddr_rate;
- unsigned long ahb_rate;
- u32 pll;
- u32 freq;
- u32 div;
+ struct clk *ref_clk;
- ref_rate = AR724X_BASE_FREQ;
- pll = ath79_pll_rr(AR724X_PLL_REG_CPU_CONFIG);
+ ref_clk = ath79_add_sys_clkdev("ref", AR724X_BASE_FREQ);
- div = ((pll >> AR724X_PLL_FB_SHIFT) & AR724X_PLL_FB_MASK);
- freq = div * ref_rate;
+ ar724x_clk_init(ref_clk, ath79_pll_base);
- div = ((pll >> AR724X_PLL_REF_DIV_SHIFT) & AR724X_PLL_REF_DIV_MASK) * 2;
- freq /= div;
-
- cpu_rate = freq;
-
- div = ((pll >> AR724X_DDR_DIV_SHIFT) & AR724X_DDR_DIV_MASK) + 1;
- ddr_rate = freq / div;
-
- div = (((pll >> AR724X_AHB_DIV_SHIFT) & AR724X_AHB_DIV_MASK) + 1) * 2;
- ahb_rate = cpu_rate / div;
-
- ath79_add_sys_clkdev("ref", ref_rate);
- clks[0] = ath79_add_sys_clkdev("cpu", cpu_rate);
- clks[1] = ath79_add_sys_clkdev("ddr", ddr_rate);
- clks[2] = ath79_add_sys_clkdev("ahb", ahb_rate);
+ /* just make happy plat_time_init() from arch/mips/ath79/setup.c */
+ clk_register_clkdev(clks[ATH79_CLK_CPU], "cpu", NULL);
+ clk_register_clkdev(clks[ATH79_CLK_DDR], "ddr", NULL);
+ clk_register_clkdev(clks[ATH79_CLK_AHB], "ahb", NULL);
clk_add_alias("wdt", NULL, "ahb", NULL);
clk_add_alias("uart", NULL, "ahb", NULL);
}
+static void __init ar9330_clk_init(struct clk *ref_clk, void __iomem *pll_base)
+{
+ u32 clock_ctrl;
+ u32 ref_div;
+ u32 ninit_mul;
+ u32 out_div;
+
+ u32 cpu_div;
+ u32 ddr_div;
+ u32 ahb_div;
+
+ clock_ctrl = __raw_readl(pll_base + AR933X_PLL_CLOCK_CTRL_REG);
+ if (clock_ctrl & AR933X_PLL_CLOCK_CTRL_BYPASS) {
+ ref_div = 1;
+ ninit_mul = 1;
+ out_div = 1;
+
+ cpu_div = 1;
+ ddr_div = 1;
+ ahb_div = 1;
+ } else {
+ u32 cpu_config;
+ u32 t;
+
+ cpu_config = __raw_readl(pll_base + AR933X_PLL_CPU_CONFIG_REG);
+
+ t = (cpu_config >> AR933X_PLL_CPU_CONFIG_REFDIV_SHIFT) &
+ AR933X_PLL_CPU_CONFIG_REFDIV_MASK;
+ ref_div = t;
+
+ ninit_mul = (cpu_config >> AR933X_PLL_CPU_CONFIG_NINT_SHIFT) &
+ AR933X_PLL_CPU_CONFIG_NINT_MASK;
+
+ t = (cpu_config >> AR933X_PLL_CPU_CONFIG_OUTDIV_SHIFT) &
+ AR933X_PLL_CPU_CONFIG_OUTDIV_MASK;
+ if (t == 0)
+ t = 1;
+
+ out_div = (1 << t);
+
+ cpu_div = ((clock_ctrl >> AR933X_PLL_CLOCK_CTRL_CPU_DIV_SHIFT) &
+ AR933X_PLL_CLOCK_CTRL_CPU_DIV_MASK) + 1;
+
+ ddr_div = ((clock_ctrl >> AR933X_PLL_CLOCK_CTRL_DDR_DIV_SHIFT) &
+ AR933X_PLL_CLOCK_CTRL_DDR_DIV_MASK) + 1;
+
+ ahb_div = ((clock_ctrl >> AR933X_PLL_CLOCK_CTRL_AHB_DIV_SHIFT) &
+ AR933X_PLL_CLOCK_CTRL_AHB_DIV_MASK) + 1;
+ }
+
+ clks[ATH79_CLK_CPU] = ath79_reg_ffclk("cpu", "ref",
+ ninit_mul, ref_div * out_div * cpu_div);
+ clks[ATH79_CLK_DDR] = ath79_reg_ffclk("ddr", "ref",
+ ninit_mul, ref_div * out_div * ddr_div);
+ clks[ATH79_CLK_AHB] = ath79_reg_ffclk("ahb", "ref",
+ ninit_mul, ref_div * out_div * ahb_div);
+}
+
static void __init ar933x_clocks_init(void)
{
+ struct clk *ref_clk;
unsigned long ref_rate;
- unsigned long cpu_rate;
- unsigned long ddr_rate;
- unsigned long ahb_rate;
- u32 clock_ctrl;
- u32 cpu_config;
- u32 freq;
u32 t;
t = ath79_reset_rr(AR933X_RESET_REG_BOOTSTRAP);
@@ -139,46 +207,14 @@
else
ref_rate = (25 * 1000 * 1000);
- clock_ctrl = ath79_pll_rr(AR933X_PLL_CLOCK_CTRL_REG);
- if (clock_ctrl & AR933X_PLL_CLOCK_CTRL_BYPASS) {
- cpu_rate = ref_rate;
- ahb_rate = ref_rate;
- ddr_rate = ref_rate;
- } else {
- cpu_config = ath79_pll_rr(AR933X_PLL_CPU_CONFIG_REG);
+ ref_clk = ath79_add_sys_clkdev("ref", ref_rate);
- t = (cpu_config >> AR933X_PLL_CPU_CONFIG_REFDIV_SHIFT) &
- AR933X_PLL_CPU_CONFIG_REFDIV_MASK;
- freq = ref_rate / t;
+ ar9330_clk_init(ref_clk, ath79_pll_base);
- t = (cpu_config >> AR933X_PLL_CPU_CONFIG_NINT_SHIFT) &
- AR933X_PLL_CPU_CONFIG_NINT_MASK;
- freq *= t;
-
- t = (cpu_config >> AR933X_PLL_CPU_CONFIG_OUTDIV_SHIFT) &
- AR933X_PLL_CPU_CONFIG_OUTDIV_MASK;
- if (t == 0)
- t = 1;
-
- freq >>= t;
-
- t = ((clock_ctrl >> AR933X_PLL_CLOCK_CTRL_CPU_DIV_SHIFT) &
- AR933X_PLL_CLOCK_CTRL_CPU_DIV_MASK) + 1;
- cpu_rate = freq / t;
-
- t = ((clock_ctrl >> AR933X_PLL_CLOCK_CTRL_DDR_DIV_SHIFT) &
- AR933X_PLL_CLOCK_CTRL_DDR_DIV_MASK) + 1;
- ddr_rate = freq / t;
-
- t = ((clock_ctrl >> AR933X_PLL_CLOCK_CTRL_AHB_DIV_SHIFT) &
- AR933X_PLL_CLOCK_CTRL_AHB_DIV_MASK) + 1;
- ahb_rate = freq / t;
- }
-
- ath79_add_sys_clkdev("ref", ref_rate);
- clks[0] = ath79_add_sys_clkdev("cpu", cpu_rate);
- clks[1] = ath79_add_sys_clkdev("ddr", ddr_rate);
- clks[2] = ath79_add_sys_clkdev("ahb", ahb_rate);
+ /* just make happy plat_time_init() from arch/mips/ath79/setup.c */
+ clk_register_clkdev(clks[ATH79_CLK_CPU], "cpu", NULL);
+ clk_register_clkdev(clks[ATH79_CLK_DDR], "ddr", NULL);
+ clk_register_clkdev(clks[ATH79_CLK_AHB], "ahb", NULL);
clk_add_alias("wdt", NULL, "ahb", NULL);
clk_add_alias("uart", NULL, "ref", NULL);
@@ -310,9 +346,9 @@
ahb_rate = cpu_pll / (postdiv + 1);
ath79_add_sys_clkdev("ref", ref_rate);
- clks[0] = ath79_add_sys_clkdev("cpu", cpu_rate);
- clks[1] = ath79_add_sys_clkdev("ddr", ddr_rate);
- clks[2] = ath79_add_sys_clkdev("ahb", ahb_rate);
+ clks[ATH79_CLK_CPU] = ath79_add_sys_clkdev("cpu", cpu_rate);
+ clks[ATH79_CLK_DDR] = ath79_add_sys_clkdev("ddr", ddr_rate);
+ clks[ATH79_CLK_AHB] = ath79_add_sys_clkdev("ahb", ahb_rate);
clk_add_alias("wdt", NULL, "ref", NULL);
clk_add_alias("uart", NULL, "ref", NULL);
@@ -397,9 +433,9 @@
ahb_rate = cpu_pll / (postdiv + 1);
ath79_add_sys_clkdev("ref", ref_rate);
- clks[0] = ath79_add_sys_clkdev("cpu", cpu_rate);
- clks[1] = ath79_add_sys_clkdev("ddr", ddr_rate);
- clks[2] = ath79_add_sys_clkdev("ahb", ahb_rate);
+ clks[ATH79_CLK_CPU] = ath79_add_sys_clkdev("cpu", cpu_rate);
+ clks[ATH79_CLK_DDR] = ath79_add_sys_clkdev("ddr", ddr_rate);
+ clks[ATH79_CLK_AHB] = ath79_add_sys_clkdev("ahb", ahb_rate);
clk_add_alias("wdt", NULL, "ref", NULL);
clk_add_alias("uart", NULL, "ref", NULL);
@@ -419,8 +455,6 @@
qca955x_clocks_init();
else
BUG();
-
- of_clk_init(NULL);
}
unsigned long __init
@@ -447,8 +481,49 @@
CLK_OF_DECLARE(ar7100, "qca,ar7100-pll", ath79_clocks_init_dt);
CLK_OF_DECLARE(ar7240, "qca,ar7240-pll", ath79_clocks_init_dt);
-CLK_OF_DECLARE(ar9130, "qca,ar9130-pll", ath79_clocks_init_dt);
-CLK_OF_DECLARE(ar9330, "qca,ar9330-pll", ath79_clocks_init_dt);
CLK_OF_DECLARE(ar9340, "qca,ar9340-pll", ath79_clocks_init_dt);
CLK_OF_DECLARE(ar9550, "qca,qca9550-pll", ath79_clocks_init_dt);
+
+static void __init ath79_clocks_init_dt_ng(struct device_node *np)
+{
+ struct clk *ref_clk;
+ void __iomem *pll_base;
+ const char *dnfn = of_node_full_name(np);
+
+ ref_clk = of_clk_get(np, 0);
+ if (IS_ERR(ref_clk)) {
+ pr_err("%s: of_clk_get failed\n", dnfn);
+ goto err;
+ }
+
+ pll_base = of_iomap(np, 0);
+ if (!pll_base) {
+ pr_err("%s: can't map pll registers\n", dnfn);
+ goto err_clk;
+ }
+
+ if (of_device_is_compatible(np, "qca,ar9130-pll"))
+ ar724x_clk_init(ref_clk, pll_base);
+ else if (of_device_is_compatible(np, "qca,ar9330-pll"))
+ ar9330_clk_init(ref_clk, pll_base);
+ else {
+ pr_err("%s: could not find any appropriate clk_init()\n", dnfn);
+ goto err_clk;
+ }
+
+ if (of_clk_add_provider(np, of_clk_src_onecell_get, &clk_data)) {
+ pr_err("%s: could not register clk provider\n", dnfn);
+ goto err_clk;
+ }
+
+ return;
+
+err_clk:
+ clk_put(ref_clk);
+
+err:
+ return;
+}
+CLK_OF_DECLARE(ar9130_clk, "qca,ar9130-pll", ath79_clocks_init_dt_ng);
+CLK_OF_DECLARE(ar9330_clk, "qca,ar9330-pll", ath79_clocks_init_dt_ng);
#endif
diff --git a/arch/mips/ath79/common.c b/arch/mips/ath79/common.c
index 3cedd1f..d071a3a 100644
--- a/arch/mips/ath79/common.c
+++ b/arch/mips/ath79/common.c
@@ -46,12 +46,12 @@
{
ath79_ddr_base = ioremap_nocache(AR71XX_DDR_CTRL_BASE,
AR71XX_DDR_CTRL_SIZE);
- if (soc_is_ar71xx() || soc_is_ar934x()) {
- ath79_ddr_wb_flush_base = ath79_ddr_base + 0x9c;
- ath79_ddr_pci_win_base = ath79_ddr_base + 0x7c;
- } else {
+ if (soc_is_ar913x() || soc_is_ar724x() || soc_is_ar933x()) {
ath79_ddr_wb_flush_base = ath79_ddr_base + 0x7c;
ath79_ddr_pci_win_base = 0;
+ } else {
+ ath79_ddr_wb_flush_base = ath79_ddr_base + 0x9c;
+ ath79_ddr_pci_win_base = ath79_ddr_base + 0x7c;
}
}
EXPORT_SYMBOL_GPL(ath79_ddr_ctrl_init);
@@ -76,14 +76,14 @@
{
BUG_ON(!ath79_ddr_pci_win_base);
- __raw_writel(AR71XX_PCI_WIN0_OFFS, ath79_ddr_pci_win_base + 0);
- __raw_writel(AR71XX_PCI_WIN1_OFFS, ath79_ddr_pci_win_base + 1);
- __raw_writel(AR71XX_PCI_WIN2_OFFS, ath79_ddr_pci_win_base + 2);
- __raw_writel(AR71XX_PCI_WIN3_OFFS, ath79_ddr_pci_win_base + 3);
- __raw_writel(AR71XX_PCI_WIN4_OFFS, ath79_ddr_pci_win_base + 4);
- __raw_writel(AR71XX_PCI_WIN5_OFFS, ath79_ddr_pci_win_base + 5);
- __raw_writel(AR71XX_PCI_WIN6_OFFS, ath79_ddr_pci_win_base + 6);
- __raw_writel(AR71XX_PCI_WIN7_OFFS, ath79_ddr_pci_win_base + 7);
+ __raw_writel(AR71XX_PCI_WIN0_OFFS, ath79_ddr_pci_win_base + 0x0);
+ __raw_writel(AR71XX_PCI_WIN1_OFFS, ath79_ddr_pci_win_base + 0x4);
+ __raw_writel(AR71XX_PCI_WIN2_OFFS, ath79_ddr_pci_win_base + 0x8);
+ __raw_writel(AR71XX_PCI_WIN3_OFFS, ath79_ddr_pci_win_base + 0xc);
+ __raw_writel(AR71XX_PCI_WIN4_OFFS, ath79_ddr_pci_win_base + 0x10);
+ __raw_writel(AR71XX_PCI_WIN5_OFFS, ath79_ddr_pci_win_base + 0x14);
+ __raw_writel(AR71XX_PCI_WIN6_OFFS, ath79_ddr_pci_win_base + 0x18);
+ __raw_writel(AR71XX_PCI_WIN7_OFFS, ath79_ddr_pci_win_base + 0x1c);
}
EXPORT_SYMBOL_GPL(ath79_ddr_set_pci_windows);
diff --git a/arch/mips/ath79/early_printk.c b/arch/mips/ath79/early_printk.c
index b955faf..d1adc59 100644
--- a/arch/mips/ath79/early_printk.c
+++ b/arch/mips/ath79/early_printk.c
@@ -31,13 +31,15 @@
} while (1);
}
+#define BOTH_EMPTY (UART_LSR_TEMT | UART_LSR_THRE)
+
static void prom_putchar_ar71xx(unsigned char ch)
{
void __iomem *base = (void __iomem *)(KSEG1ADDR(AR71XX_UART_BASE));
- prom_putchar_wait(base + UART_LSR * 4, UART_LSR_THRE, UART_LSR_THRE);
+ prom_putchar_wait(base + UART_LSR * 4, BOTH_EMPTY, BOTH_EMPTY);
__raw_writel(ch, base + UART_TX * 4);
- prom_putchar_wait(base + UART_LSR * 4, UART_LSR_THRE, UART_LSR_THRE);
+ prom_putchar_wait(base + UART_LSR * 4, BOTH_EMPTY, BOTH_EMPTY);
}
static void prom_putchar_ar933x(unsigned char ch)
diff --git a/arch/mips/ath79/setup.c b/arch/mips/ath79/setup.c
index be451ee4a..7adab18 100644
--- a/arch/mips/ath79/setup.c
+++ b/arch/mips/ath79/setup.c
@@ -17,6 +17,7 @@
#include <linux/bootmem.h>
#include <linux/err.h>
#include <linux/clk.h>
+#include <linux/clk-provider.h>
#include <linux/of_platform.h>
#include <linux/of_fdt.h>
@@ -203,26 +204,57 @@
fdt_start = fw_getenvl("fdt_start");
if (fdt_start)
__dt_setup_arch((void *)KSEG0ADDR(fdt_start));
-#ifdef CONFIG_BUILTIN_DTB
- else
- __dt_setup_arch(__dtb_start);
-#endif
+ else if (fw_arg0 == -2)
+ __dt_setup_arch((void *)KSEG0ADDR(fw_arg1));
- ath79_reset_base = ioremap_nocache(AR71XX_RESET_BASE,
- AR71XX_RESET_SIZE);
- ath79_pll_base = ioremap_nocache(AR71XX_PLL_BASE,
- AR71XX_PLL_SIZE);
- ath79_detect_sys_type();
- ath79_ddr_ctrl_init();
+ if (mips_machtype != ATH79_MACH_GENERIC_OF) {
+ ath79_reset_base = ioremap_nocache(AR71XX_RESET_BASE,
+ AR71XX_RESET_SIZE);
+ ath79_pll_base = ioremap_nocache(AR71XX_PLL_BASE,
+ AR71XX_PLL_SIZE);
+ ath79_detect_sys_type();
+ ath79_ddr_ctrl_init();
- if (mips_machtype != ATH79_MACH_GENERIC_OF)
detect_memory_region(0, ATH79_MEM_SIZE_MIN, ATH79_MEM_SIZE_MAX);
- _machine_restart = ath79_restart;
+ /* OF machines should use the reset driver */
+ _machine_restart = ath79_restart;
+ }
+
_machine_halt = ath79_halt;
pm_power_off = ath79_halt;
}
+static void __init ath79_of_plat_time_init(void)
+{
+ struct device_node *np;
+ struct clk *clk;
+ unsigned long cpu_clk_rate;
+
+ of_clk_init(NULL);
+
+ np = of_get_cpu_node(0, NULL);
+ if (!np) {
+ pr_err("Failed to get CPU node\n");
+ return;
+ }
+
+ clk = of_clk_get(np, 0);
+ if (IS_ERR(clk)) {
+ pr_err("Failed to get CPU clock: %ld\n", PTR_ERR(clk));
+ return;
+ }
+
+ cpu_clk_rate = clk_get_rate(clk);
+
+ pr_info("CPU clock: %lu.%03lu MHz\n",
+ cpu_clk_rate / 1000000, (cpu_clk_rate / 1000) % 1000);
+
+ mips_hpt_frequency = cpu_clk_rate / 2;
+
+ clk_put(clk);
+}
+
void __init plat_time_init(void)
{
unsigned long cpu_clk_rate;
@@ -230,6 +262,11 @@
unsigned long ddr_clk_rate;
unsigned long ref_clk_rate;
+ if (IS_ENABLED(CONFIG_OF) && mips_machtype == ATH79_MACH_GENERIC_OF) {
+ ath79_of_plat_time_init();
+ return;
+ }
+
ath79_clocks_init();
cpu_clk_rate = ath79_get_sys_clk_rate("cpu");
diff --git a/arch/mips/bcm47xx/Makefile b/arch/mips/bcm47xx/Makefile
index 66bea4e..6d86150 100644
--- a/arch/mips/bcm47xx/Makefile
+++ b/arch/mips/bcm47xx/Makefile
@@ -3,5 +3,5 @@
# under Linux.
#
-obj-y += irq.o prom.o serial.o setup.o time.o sprom.o
+obj-y += irq.o prom.o serial.o setup.o time.o
obj-y += board.o buttons.o leds.o workarounds.o
diff --git a/arch/mips/bcm47xx/bcm47xx_private.h b/arch/mips/bcm47xx/bcm47xx_private.h
index 41796be..0367ac7 100644
--- a/arch/mips/bcm47xx/bcm47xx_private.h
+++ b/arch/mips/bcm47xx/bcm47xx_private.h
@@ -10,9 +10,6 @@
/* prom.c */
void __init bcm47xx_prom_highmem_init(void);
-/* sprom.c */
-void bcm47xx_sprom_register_fallbacks(void);
-
/* buttons.c */
int __init bcm47xx_buttons_register(void);
diff --git a/arch/mips/bcm47xx/setup.c b/arch/mips/bcm47xx/setup.c
index c807e32..6054d49 100644
--- a/arch/mips/bcm47xx/setup.c
+++ b/arch/mips/bcm47xx/setup.c
@@ -28,6 +28,7 @@
#include "bcm47xx_private.h"
+#include <linux/bcm47xx_sprom.h>
#include <linux/export.h>
#include <linux/types.h>
#include <linux/ethtool.h>
@@ -151,7 +152,6 @@
pr_info("Using bcma bus\n");
#ifdef CONFIG_BCM47XX_BCMA
bcm47xx_bus_type = BCM47XX_BUS_TYPE_BCMA;
- bcm47xx_sprom_register_fallbacks();
bcm47xx_register_bcma();
bcm47xx_set_system_type(bcm47xx_bus.bcma.bus.chipinfo.id);
#ifdef CONFIG_HIGHMEM
diff --git a/arch/mips/bcm47xx/sprom.c b/arch/mips/bcm47xx/sprom.c
deleted file mode 100644
index ca7ad13..0000000
--- a/arch/mips/bcm47xx/sprom.c
+++ /dev/null
@@ -1,724 +0,0 @@
-/*
- * Copyright (C) 2004 Florian Schirmer <jolt@tuxbox.org>
- * Copyright (C) 2006 Felix Fietkau <nbd@openwrt.org>
- * Copyright (C) 2006 Michael Buesch <m@bues.ch>
- * Copyright (C) 2010 Waldemar Brodkorb <wbx@openadk.org>
- * Copyright (C) 2010-2012 Hauke Mehrtens <hauke@hauke-m.de>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- *
- * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED
- * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN
- * NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
- * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
- * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
- * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
- * ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
- * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 675 Mass Ave, Cambridge, MA 02139, USA.
- */
-
-#include <bcm47xx.h>
-#include <linux/if_ether.h>
-#include <linux/etherdevice.h>
-
-static void create_key(const char *prefix, const char *postfix,
- const char *name, char *buf, int len)
-{
- if (prefix && postfix)
- snprintf(buf, len, "%s%s%s", prefix, name, postfix);
- else if (prefix)
- snprintf(buf, len, "%s%s", prefix, name);
- else if (postfix)
- snprintf(buf, len, "%s%s", name, postfix);
- else
- snprintf(buf, len, "%s", name);
-}
-
-static int get_nvram_var(const char *prefix, const char *postfix,
- const char *name, char *buf, int len, bool fallback)
-{
- char key[40];
- int err;
-
- create_key(prefix, postfix, name, key, sizeof(key));
-
- err = bcm47xx_nvram_getenv(key, buf, len);
- if (fallback && err == -ENOENT && prefix) {
- create_key(NULL, postfix, name, key, sizeof(key));
- err = bcm47xx_nvram_getenv(key, buf, len);
- }
- return err;
-}
-
-#define NVRAM_READ_VAL(type) \
-static void nvram_read_ ## type(const char *prefix, \
- const char *postfix, const char *name, \
- type *val, type allset, bool fallback) \
-{ \
- char buf[100]; \
- int err; \
- type var; \
- \
- err = get_nvram_var(prefix, postfix, name, buf, sizeof(buf), \
- fallback); \
- if (err < 0) \
- return; \
- err = kstrto ## type(strim(buf), 0, &var); \
- if (err) { \
- pr_warn("can not parse nvram name %s%s%s with value %s got %i\n", \
- prefix, name, postfix, buf, err); \
- return; \
- } \
- if (allset && var == allset) \
- return; \
- *val = var; \
-}
-
-NVRAM_READ_VAL(u8)
-NVRAM_READ_VAL(s8)
-NVRAM_READ_VAL(u16)
-NVRAM_READ_VAL(u32)
-
-#undef NVRAM_READ_VAL
-
-static void nvram_read_u32_2(const char *prefix, const char *name,
- u16 *val_lo, u16 *val_hi, bool fallback)
-{
- char buf[100];
- int err;
- u32 val;
-
- err = get_nvram_var(prefix, NULL, name, buf, sizeof(buf), fallback);
- if (err < 0)
- return;
- err = kstrtou32(strim(buf), 0, &val);
- if (err) {
- pr_warn("can not parse nvram name %s%s with value %s got %i\n",
- prefix, name, buf, err);
- return;
- }
- *val_lo = (val & 0x0000FFFFU);
- *val_hi = (val & 0xFFFF0000U) >> 16;
-}
-
-static void nvram_read_leddc(const char *prefix, const char *name,
- u8 *leddc_on_time, u8 *leddc_off_time,
- bool fallback)
-{
- char buf[100];
- int err;
- u32 val;
-
- err = get_nvram_var(prefix, NULL, name, buf, sizeof(buf), fallback);
- if (err < 0)
- return;
- err = kstrtou32(strim(buf), 0, &val);
- if (err) {
- pr_warn("can not parse nvram name %s%s with value %s got %i\n",
- prefix, name, buf, err);
- return;
- }
-
- if (val == 0xffff || val == 0xffffffff)
- return;
-
- *leddc_on_time = val & 0xff;
- *leddc_off_time = (val >> 16) & 0xff;
-}
-
-static void bcm47xx_nvram_parse_macaddr(char *buf, u8 macaddr[6])
-{
- if (strchr(buf, ':'))
- sscanf(buf, "%hhx:%hhx:%hhx:%hhx:%hhx:%hhx", &macaddr[0],
- &macaddr[1], &macaddr[2], &macaddr[3], &macaddr[4],
- &macaddr[5]);
- else if (strchr(buf, '-'))
- sscanf(buf, "%hhx-%hhx-%hhx-%hhx-%hhx-%hhx", &macaddr[0],
- &macaddr[1], &macaddr[2], &macaddr[3], &macaddr[4],
- &macaddr[5]);
- else
- pr_warn("Can not parse mac address: %s\n", buf);
-}
-
-static void nvram_read_macaddr(const char *prefix, const char *name,
- u8 val[6], bool fallback)
-{
- char buf[100];
- int err;
-
- err = get_nvram_var(prefix, NULL, name, buf, sizeof(buf), fallback);
- if (err < 0)
- return;
-
- bcm47xx_nvram_parse_macaddr(buf, val);
-}
-
-static void nvram_read_alpha2(const char *prefix, const char *name,
- char val[2], bool fallback)
-{
- char buf[10];
- int err;
-
- err = get_nvram_var(prefix, NULL, name, buf, sizeof(buf), fallback);
- if (err < 0)
- return;
- if (buf[0] == '0')
- return;
- if (strlen(buf) > 2) {
- pr_warn("alpha2 is too long %s\n", buf);
- return;
- }
- memcpy(val, buf, 2);
-}
-
-/* This is one-function-only macro, it uses local "sprom" variable! */
-#define ENTRY(_revmask, _type, _prefix, _name, _val, _allset, _fallback) \
- if (_revmask & BIT(sprom->revision)) \
- nvram_read_ ## _type(_prefix, NULL, _name, &sprom->_val, \
- _allset, _fallback)
-/*
- * Special version of filling function that can be safely called for any SPROM
- * revision. For every NVRAM to SPROM mapping it contains bitmask of revisions
- * for which the mapping is valid.
- * It obviously requires some hexadecimal/bitmasks knowledge, but allows
- * writing cleaner code (easy revisions handling).
- * Note that while SPROM revision 0 was never used, we still keep BIT(0)
- * reserved for it, just to keep numbering sane.
- */
-static void bcm47xx_sprom_fill_auto(struct ssb_sprom *sprom,
- const char *prefix, bool fallback)
-{
- const char *pre = prefix;
- bool fb = fallback;
-
- /* Broadcom extracts it for rev 8+ but it was found on 2 and 4 too */
- ENTRY(0xfffffffe, u16, pre, "devid", dev_id, 0, fallback);
-
- ENTRY(0xfffffffe, u16, pre, "boardrev", board_rev, 0, true);
- ENTRY(0xfffffffe, u32, pre, "boardflags", boardflags, 0, fb);
- ENTRY(0xfffffff0, u32, pre, "boardflags2", boardflags2, 0, fb);
- ENTRY(0xfffff800, u32, pre, "boardflags3", boardflags3, 0, fb);
- ENTRY(0x00000002, u16, pre, "boardflags", boardflags_lo, 0, fb);
- ENTRY(0xfffffffc, u16, pre, "boardtype", board_type, 0, true);
- ENTRY(0xfffffffe, u16, pre, "boardnum", board_num, 0, fb);
- ENTRY(0x00000002, u8, pre, "cc", country_code, 0, fb);
- ENTRY(0xfffffff8, u8, pre, "regrev", regrev, 0, fb);
-
- ENTRY(0xfffffffe, u8, pre, "ledbh0", gpio0, 0xff, fb);
- ENTRY(0xfffffffe, u8, pre, "ledbh1", gpio1, 0xff, fb);
- ENTRY(0xfffffffe, u8, pre, "ledbh2", gpio2, 0xff, fb);
- ENTRY(0xfffffffe, u8, pre, "ledbh3", gpio3, 0xff, fb);
-
- ENTRY(0x0000070e, u16, pre, "pa0b0", pa0b0, 0, fb);
- ENTRY(0x0000070e, u16, pre, "pa0b1", pa0b1, 0, fb);
- ENTRY(0x0000070e, u16, pre, "pa0b2", pa0b2, 0, fb);
- ENTRY(0x0000070e, u8, pre, "pa0itssit", itssi_bg, 0, fb);
- ENTRY(0x0000070e, u8, pre, "pa0maxpwr", maxpwr_bg, 0, fb);
-
- ENTRY(0x0000070c, u8, pre, "opo", opo, 0, fb);
- ENTRY(0xfffffffe, u8, pre, "aa2g", ant_available_bg, 0, fb);
- ENTRY(0xfffffffe, u8, pre, "aa5g", ant_available_a, 0, fb);
- ENTRY(0x000007fe, s8, pre, "ag0", antenna_gain.a0, 0, fb);
- ENTRY(0x000007fe, s8, pre, "ag1", antenna_gain.a1, 0, fb);
- ENTRY(0x000007f0, s8, pre, "ag2", antenna_gain.a2, 0, fb);
- ENTRY(0x000007f0, s8, pre, "ag3", antenna_gain.a3, 0, fb);
-
- ENTRY(0x0000070e, u16, pre, "pa1b0", pa1b0, 0, fb);
- ENTRY(0x0000070e, u16, pre, "pa1b1", pa1b1, 0, fb);
- ENTRY(0x0000070e, u16, pre, "pa1b2", pa1b2, 0, fb);
- ENTRY(0x0000070c, u16, pre, "pa1lob0", pa1lob0, 0, fb);
- ENTRY(0x0000070c, u16, pre, "pa1lob1", pa1lob1, 0, fb);
- ENTRY(0x0000070c, u16, pre, "pa1lob2", pa1lob2, 0, fb);
- ENTRY(0x0000070c, u16, pre, "pa1hib0", pa1hib0, 0, fb);
- ENTRY(0x0000070c, u16, pre, "pa1hib1", pa1hib1, 0, fb);
- ENTRY(0x0000070c, u16, pre, "pa1hib2", pa1hib2, 0, fb);
- ENTRY(0x0000070e, u8, pre, "pa1itssit", itssi_a, 0, fb);
- ENTRY(0x0000070e, u8, pre, "pa1maxpwr", maxpwr_a, 0, fb);
- ENTRY(0x0000070c, u8, pre, "pa1lomaxpwr", maxpwr_al, 0, fb);
- ENTRY(0x0000070c, u8, pre, "pa1himaxpwr", maxpwr_ah, 0, fb);
-
- ENTRY(0x00000708, u8, pre, "bxa2g", bxa2g, 0, fb);
- ENTRY(0x00000708, u8, pre, "rssisav2g", rssisav2g, 0, fb);
- ENTRY(0x00000708, u8, pre, "rssismc2g", rssismc2g, 0, fb);
- ENTRY(0x00000708, u8, pre, "rssismf2g", rssismf2g, 0, fb);
- ENTRY(0x00000708, u8, pre, "bxa5g", bxa5g, 0, fb);
- ENTRY(0x00000708, u8, pre, "rssisav5g", rssisav5g, 0, fb);
- ENTRY(0x00000708, u8, pre, "rssismc5g", rssismc5g, 0, fb);
- ENTRY(0x00000708, u8, pre, "rssismf5g", rssismf5g, 0, fb);
- ENTRY(0x00000708, u8, pre, "tri2g", tri2g, 0, fb);
- ENTRY(0x00000708, u8, pre, "tri5g", tri5g, 0, fb);
- ENTRY(0x00000708, u8, pre, "tri5gl", tri5gl, 0, fb);
- ENTRY(0x00000708, u8, pre, "tri5gh", tri5gh, 0, fb);
- ENTRY(0x00000708, s8, pre, "rxpo2g", rxpo2g, 0, fb);
- ENTRY(0x00000708, s8, pre, "rxpo5g", rxpo5g, 0, fb);
- ENTRY(0xfffffff0, u8, pre, "txchain", txchain, 0xf, fb);
- ENTRY(0xfffffff0, u8, pre, "rxchain", rxchain, 0xf, fb);
- ENTRY(0xfffffff0, u8, pre, "antswitch", antswitch, 0xff, fb);
- ENTRY(0x00000700, u8, pre, "tssipos2g", fem.ghz2.tssipos, 0, fb);
- ENTRY(0x00000700, u8, pre, "extpagain2g", fem.ghz2.extpa_gain, 0, fb);
- ENTRY(0x00000700, u8, pre, "pdetrange2g", fem.ghz2.pdet_range, 0, fb);
- ENTRY(0x00000700, u8, pre, "triso2g", fem.ghz2.tr_iso, 0, fb);
- ENTRY(0x00000700, u8, pre, "antswctl2g", fem.ghz2.antswlut, 0, fb);
- ENTRY(0x00000700, u8, pre, "tssipos5g", fem.ghz5.tssipos, 0, fb);
- ENTRY(0x00000700, u8, pre, "extpagain5g", fem.ghz5.extpa_gain, 0, fb);
- ENTRY(0x00000700, u8, pre, "pdetrange5g", fem.ghz5.pdet_range, 0, fb);
- ENTRY(0x00000700, u8, pre, "triso5g", fem.ghz5.tr_iso, 0, fb);
- ENTRY(0x00000700, u8, pre, "antswctl5g", fem.ghz5.antswlut, 0, fb);
- ENTRY(0x000000f0, u8, pre, "txpid2ga0", txpid2g[0], 0, fb);
- ENTRY(0x000000f0, u8, pre, "txpid2ga1", txpid2g[1], 0, fb);
- ENTRY(0x000000f0, u8, pre, "txpid2ga2", txpid2g[2], 0, fb);
- ENTRY(0x000000f0, u8, pre, "txpid2ga3", txpid2g[3], 0, fb);
- ENTRY(0x000000f0, u8, pre, "txpid5ga0", txpid5g[0], 0, fb);
- ENTRY(0x000000f0, u8, pre, "txpid5ga1", txpid5g[1], 0, fb);
- ENTRY(0x000000f0, u8, pre, "txpid5ga2", txpid5g[2], 0, fb);
- ENTRY(0x000000f0, u8, pre, "txpid5ga3", txpid5g[3], 0, fb);
- ENTRY(0x000000f0, u8, pre, "txpid5gla0", txpid5gl[0], 0, fb);
- ENTRY(0x000000f0, u8, pre, "txpid5gla1", txpid5gl[1], 0, fb);
- ENTRY(0x000000f0, u8, pre, "txpid5gla2", txpid5gl[2], 0, fb);
- ENTRY(0x000000f0, u8, pre, "txpid5gla3", txpid5gl[3], 0, fb);
- ENTRY(0x000000f0, u8, pre, "txpid5gha0", txpid5gh[0], 0, fb);
- ENTRY(0x000000f0, u8, pre, "txpid5gha1", txpid5gh[1], 0, fb);
- ENTRY(0x000000f0, u8, pre, "txpid5gha2", txpid5gh[2], 0, fb);
- ENTRY(0x000000f0, u8, pre, "txpid5gha3", txpid5gh[3], 0, fb);
-
- ENTRY(0xffffff00, u8, pre, "tempthresh", tempthresh, 0, fb);
- ENTRY(0xffffff00, u8, pre, "tempoffset", tempoffset, 0, fb);
- ENTRY(0xffffff00, u16, pre, "rawtempsense", rawtempsense, 0, fb);
- ENTRY(0xffffff00, u8, pre, "measpower", measpower, 0, fb);
- ENTRY(0xffffff00, u8, pre, "tempsense_slope", tempsense_slope, 0, fb);
- ENTRY(0xffffff00, u8, pre, "tempcorrx", tempcorrx, 0, fb);
- ENTRY(0xffffff00, u8, pre, "tempsense_option", tempsense_option, 0, fb);
- ENTRY(0x00000700, u8, pre, "freqoffset_corr", freqoffset_corr, 0, fb);
- ENTRY(0x00000700, u8, pre, "iqcal_swp_dis", iqcal_swp_dis, 0, fb);
- ENTRY(0x00000700, u8, pre, "hw_iqcal_en", hw_iqcal_en, 0, fb);
- ENTRY(0x00000700, u8, pre, "elna2g", elna2g, 0, fb);
- ENTRY(0x00000700, u8, pre, "elna5g", elna5g, 0, fb);
- ENTRY(0xffffff00, u8, pre, "phycal_tempdelta", phycal_tempdelta, 0, fb);
- ENTRY(0xffffff00, u8, pre, "temps_period", temps_period, 0, fb);
- ENTRY(0xffffff00, u8, pre, "temps_hysteresis", temps_hysteresis, 0, fb);
- ENTRY(0xffffff00, u8, pre, "measpower1", measpower1, 0, fb);
- ENTRY(0xffffff00, u8, pre, "measpower2", measpower2, 0, fb);
-
- ENTRY(0x000001f0, u16, pre, "cck2gpo", cck2gpo, 0, fb);
- ENTRY(0x000001f0, u32, pre, "ofdm2gpo", ofdm2gpo, 0, fb);
- ENTRY(0x000001f0, u32, pre, "ofdm5gpo", ofdm5gpo, 0, fb);
- ENTRY(0x000001f0, u32, pre, "ofdm5glpo", ofdm5glpo, 0, fb);
- ENTRY(0x000001f0, u32, pre, "ofdm5ghpo", ofdm5ghpo, 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs2gpo0", mcs2gpo[0], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs2gpo1", mcs2gpo[1], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs2gpo2", mcs2gpo[2], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs2gpo3", mcs2gpo[3], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs2gpo4", mcs2gpo[4], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs2gpo5", mcs2gpo[5], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs2gpo6", mcs2gpo[6], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs2gpo7", mcs2gpo[7], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5gpo0", mcs5gpo[0], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5gpo1", mcs5gpo[1], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5gpo2", mcs5gpo[2], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5gpo3", mcs5gpo[3], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5gpo4", mcs5gpo[4], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5gpo5", mcs5gpo[5], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5gpo6", mcs5gpo[6], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5gpo7", mcs5gpo[7], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5glpo0", mcs5glpo[0], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5glpo1", mcs5glpo[1], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5glpo2", mcs5glpo[2], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5glpo3", mcs5glpo[3], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5glpo4", mcs5glpo[4], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5glpo5", mcs5glpo[5], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5glpo6", mcs5glpo[6], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5glpo7", mcs5glpo[7], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5ghpo0", mcs5ghpo[0], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5ghpo1", mcs5ghpo[1], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5ghpo2", mcs5ghpo[2], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5ghpo3", mcs5ghpo[3], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5ghpo4", mcs5ghpo[4], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5ghpo5", mcs5ghpo[5], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5ghpo6", mcs5ghpo[6], 0, fb);
- ENTRY(0x000001f0, u16, pre, "mcs5ghpo7", mcs5ghpo[7], 0, fb);
- ENTRY(0x000001f0, u16, pre, "cddpo", cddpo, 0, fb);
- ENTRY(0x000001f0, u16, pre, "stbcpo", stbcpo, 0, fb);
- ENTRY(0x000001f0, u16, pre, "bw40po", bw40po, 0, fb);
- ENTRY(0x000001f0, u16, pre, "bwduppo", bwduppo, 0, fb);
-
- ENTRY(0xfffffe00, u16, pre, "cckbw202gpo", cckbw202gpo, 0, fb);
- ENTRY(0xfffffe00, u16, pre, "cckbw20ul2gpo", cckbw20ul2gpo, 0, fb);
- ENTRY(0x00000600, u32, pre, "legofdmbw202gpo", legofdmbw202gpo, 0, fb);
- ENTRY(0x00000600, u32, pre, "legofdmbw20ul2gpo", legofdmbw20ul2gpo, 0, fb);
- ENTRY(0x00000600, u32, pre, "legofdmbw205glpo", legofdmbw205glpo, 0, fb);
- ENTRY(0x00000600, u32, pre, "legofdmbw20ul5glpo", legofdmbw20ul5glpo, 0, fb);
- ENTRY(0x00000600, u32, pre, "legofdmbw205gmpo", legofdmbw205gmpo, 0, fb);
- ENTRY(0x00000600, u32, pre, "legofdmbw20ul5gmpo", legofdmbw20ul5gmpo, 0, fb);
- ENTRY(0x00000600, u32, pre, "legofdmbw205ghpo", legofdmbw205ghpo, 0, fb);
- ENTRY(0x00000600, u32, pre, "legofdmbw20ul5ghpo", legofdmbw20ul5ghpo, 0, fb);
- ENTRY(0xfffffe00, u32, pre, "mcsbw202gpo", mcsbw202gpo, 0, fb);
- ENTRY(0x00000600, u32, pre, "mcsbw20ul2gpo", mcsbw20ul2gpo, 0, fb);
- ENTRY(0xfffffe00, u32, pre, "mcsbw402gpo", mcsbw402gpo, 0, fb);
- ENTRY(0xfffffe00, u32, pre, "mcsbw205glpo", mcsbw205glpo, 0, fb);
- ENTRY(0x00000600, u32, pre, "mcsbw20ul5glpo", mcsbw20ul5glpo, 0, fb);
- ENTRY(0xfffffe00, u32, pre, "mcsbw405glpo", mcsbw405glpo, 0, fb);
- ENTRY(0xfffffe00, u32, pre, "mcsbw205gmpo", mcsbw205gmpo, 0, fb);
- ENTRY(0x00000600, u32, pre, "mcsbw20ul5gmpo", mcsbw20ul5gmpo, 0, fb);
- ENTRY(0xfffffe00, u32, pre, "mcsbw405gmpo", mcsbw405gmpo, 0, fb);
- ENTRY(0xfffffe00, u32, pre, "mcsbw205ghpo", mcsbw205ghpo, 0, fb);
- ENTRY(0x00000600, u32, pre, "mcsbw20ul5ghpo", mcsbw20ul5ghpo, 0, fb);
- ENTRY(0xfffffe00, u32, pre, "mcsbw405ghpo", mcsbw405ghpo, 0, fb);
- ENTRY(0x00000600, u16, pre, "mcs32po", mcs32po, 0, fb);
- ENTRY(0x00000600, u16, pre, "legofdm40duppo", legofdm40duppo, 0, fb);
- ENTRY(0x00000700, u8, pre, "pcieingress_war", pcieingress_war, 0, fb);
-
- /* TODO: rev 11 support */
- ENTRY(0x00000700, u8, pre, "rxgainerr2ga0", rxgainerr2ga[0], 0, fb);
- ENTRY(0x00000700, u8, pre, "rxgainerr2ga1", rxgainerr2ga[1], 0, fb);
- ENTRY(0x00000700, u8, pre, "rxgainerr2ga2", rxgainerr2ga[2], 0, fb);
- ENTRY(0x00000700, u8, pre, "rxgainerr5gla0", rxgainerr5gla[0], 0, fb);
- ENTRY(0x00000700, u8, pre, "rxgainerr5gla1", rxgainerr5gla[1], 0, fb);
- ENTRY(0x00000700, u8, pre, "rxgainerr5gla2", rxgainerr5gla[2], 0, fb);
- ENTRY(0x00000700, u8, pre, "rxgainerr5gma0", rxgainerr5gma[0], 0, fb);
- ENTRY(0x00000700, u8, pre, "rxgainerr5gma1", rxgainerr5gma[1], 0, fb);
- ENTRY(0x00000700, u8, pre, "rxgainerr5gma2", rxgainerr5gma[2], 0, fb);
- ENTRY(0x00000700, u8, pre, "rxgainerr5gha0", rxgainerr5gha[0], 0, fb);
- ENTRY(0x00000700, u8, pre, "rxgainerr5gha1", rxgainerr5gha[1], 0, fb);
- ENTRY(0x00000700, u8, pre, "rxgainerr5gha2", rxgainerr5gha[2], 0, fb);
- ENTRY(0x00000700, u8, pre, "rxgainerr5gua0", rxgainerr5gua[0], 0, fb);
- ENTRY(0x00000700, u8, pre, "rxgainerr5gua1", rxgainerr5gua[1], 0, fb);
- ENTRY(0x00000700, u8, pre, "rxgainerr5gua2", rxgainerr5gua[2], 0, fb);
-
- ENTRY(0xfffffe00, u8, pre, "sar2g", sar2g, 0, fb);
- ENTRY(0xfffffe00, u8, pre, "sar5g", sar5g, 0, fb);
-
- /* TODO: rev 11 support */
- ENTRY(0x00000700, u8, pre, "noiselvl2ga0", noiselvl2ga[0], 0, fb);
- ENTRY(0x00000700, u8, pre, "noiselvl2ga1", noiselvl2ga[1], 0, fb);
- ENTRY(0x00000700, u8, pre, "noiselvl2ga2", noiselvl2ga[2], 0, fb);
- ENTRY(0x00000700, u8, pre, "noiselvl5gla0", noiselvl5gla[0], 0, fb);
- ENTRY(0x00000700, u8, pre, "noiselvl5gla1", noiselvl5gla[1], 0, fb);
- ENTRY(0x00000700, u8, pre, "noiselvl5gla2", noiselvl5gla[2], 0, fb);
- ENTRY(0x00000700, u8, pre, "noiselvl5gma0", noiselvl5gma[0], 0, fb);
- ENTRY(0x00000700, u8, pre, "noiselvl5gma1", noiselvl5gma[1], 0, fb);
- ENTRY(0x00000700, u8, pre, "noiselvl5gma2", noiselvl5gma[2], 0, fb);
- ENTRY(0x00000700, u8, pre, "noiselvl5gha0", noiselvl5gha[0], 0, fb);
- ENTRY(0x00000700, u8, pre, "noiselvl5gha1", noiselvl5gha[1], 0, fb);
- ENTRY(0x00000700, u8, pre, "noiselvl5gha2", noiselvl5gha[2], 0, fb);
- ENTRY(0x00000700, u8, pre, "noiselvl5gua0", noiselvl5gua[0], 0, fb);
- ENTRY(0x00000700, u8, pre, "noiselvl5gua1", noiselvl5gua[1], 0, fb);
- ENTRY(0x00000700, u8, pre, "noiselvl5gua2", noiselvl5gua[2], 0, fb);
-}
-#undef ENTRY /* It's specififc, uses local variable, don't use it (again). */
-
-static void bcm47xx_fill_sprom_path_r4589(struct ssb_sprom *sprom,
- const char *prefix, bool fallback)
-{
- char postfix[2];
- int i;
-
- for (i = 0; i < ARRAY_SIZE(sprom->core_pwr_info); i++) {
- struct ssb_sprom_core_pwr_info *pwr_info;
-
- pwr_info = &sprom->core_pwr_info[i];
-
- snprintf(postfix, sizeof(postfix), "%i", i);
- nvram_read_u8(prefix, postfix, "maxp2ga",
- &pwr_info->maxpwr_2g, 0, fallback);
- nvram_read_u8(prefix, postfix, "itt2ga",
- &pwr_info->itssi_2g, 0, fallback);
- nvram_read_u8(prefix, postfix, "itt5ga",
- &pwr_info->itssi_5g, 0, fallback);
- nvram_read_u16(prefix, postfix, "pa2gw0a",
- &pwr_info->pa_2g[0], 0, fallback);
- nvram_read_u16(prefix, postfix, "pa2gw1a",
- &pwr_info->pa_2g[1], 0, fallback);
- nvram_read_u16(prefix, postfix, "pa2gw2a",
- &pwr_info->pa_2g[2], 0, fallback);
- nvram_read_u8(prefix, postfix, "maxp5ga",
- &pwr_info->maxpwr_5g, 0, fallback);
- nvram_read_u8(prefix, postfix, "maxp5gha",
- &pwr_info->maxpwr_5gh, 0, fallback);
- nvram_read_u8(prefix, postfix, "maxp5gla",
- &pwr_info->maxpwr_5gl, 0, fallback);
- nvram_read_u16(prefix, postfix, "pa5gw0a",
- &pwr_info->pa_5g[0], 0, fallback);
- nvram_read_u16(prefix, postfix, "pa5gw1a",
- &pwr_info->pa_5g[1], 0, fallback);
- nvram_read_u16(prefix, postfix, "pa5gw2a",
- &pwr_info->pa_5g[2], 0, fallback);
- nvram_read_u16(prefix, postfix, "pa5glw0a",
- &pwr_info->pa_5gl[0], 0, fallback);
- nvram_read_u16(prefix, postfix, "pa5glw1a",
- &pwr_info->pa_5gl[1], 0, fallback);
- nvram_read_u16(prefix, postfix, "pa5glw2a",
- &pwr_info->pa_5gl[2], 0, fallback);
- nvram_read_u16(prefix, postfix, "pa5ghw0a",
- &pwr_info->pa_5gh[0], 0, fallback);
- nvram_read_u16(prefix, postfix, "pa5ghw1a",
- &pwr_info->pa_5gh[1], 0, fallback);
- nvram_read_u16(prefix, postfix, "pa5ghw2a",
- &pwr_info->pa_5gh[2], 0, fallback);
- }
-}
-
-static void bcm47xx_fill_sprom_path_r45(struct ssb_sprom *sprom,
- const char *prefix, bool fallback)
-{
- char postfix[2];
- int i;
-
- for (i = 0; i < ARRAY_SIZE(sprom->core_pwr_info); i++) {
- struct ssb_sprom_core_pwr_info *pwr_info;
-
- pwr_info = &sprom->core_pwr_info[i];
-
- snprintf(postfix, sizeof(postfix), "%i", i);
- nvram_read_u16(prefix, postfix, "pa2gw3a",
- &pwr_info->pa_2g[3], 0, fallback);
- nvram_read_u16(prefix, postfix, "pa5gw3a",
- &pwr_info->pa_5g[3], 0, fallback);
- nvram_read_u16(prefix, postfix, "pa5glw3a",
- &pwr_info->pa_5gl[3], 0, fallback);
- nvram_read_u16(prefix, postfix, "pa5ghw3a",
- &pwr_info->pa_5gh[3], 0, fallback);
- }
-}
-
-static bool bcm47xx_is_valid_mac(u8 *mac)
-{
- return mac && !(mac[0] == 0x00 && mac[1] == 0x90 && mac[2] == 0x4c);
-}
-
-static int bcm47xx_increase_mac_addr(u8 *mac, u8 num)
-{
- u8 *oui = mac + ETH_ALEN/2 - 1;
- u8 *p = mac + ETH_ALEN - 1;
-
- do {
- (*p) += num;
- if (*p > num)
- break;
- p--;
- num = 1;
- } while (p != oui);
-
- if (p == oui) {
- pr_err("unable to fetch mac address\n");
- return -ENOENT;
- }
- return 0;
-}
-
-static int mac_addr_used = 2;
-
-static void bcm47xx_fill_sprom_ethernet(struct ssb_sprom *sprom,
- const char *prefix, bool fallback)
-{
- bool fb = fallback;
-
- nvram_read_macaddr(prefix, "et0macaddr", sprom->et0mac, fallback);
- nvram_read_u8(prefix, NULL, "et0mdcport", &sprom->et0mdcport, 0,
- fallback);
- nvram_read_u8(prefix, NULL, "et0phyaddr", &sprom->et0phyaddr, 0,
- fallback);
-
- nvram_read_macaddr(prefix, "et1macaddr", sprom->et1mac, fallback);
- nvram_read_u8(prefix, NULL, "et1mdcport", &sprom->et1mdcport, 0,
- fallback);
- nvram_read_u8(prefix, NULL, "et1phyaddr", &sprom->et1phyaddr, 0,
- fallback);
-
- nvram_read_macaddr(prefix, "et2macaddr", sprom->et2mac, fb);
- nvram_read_u8(prefix, NULL, "et2mdcport", &sprom->et2mdcport, 0, fb);
- nvram_read_u8(prefix, NULL, "et2phyaddr", &sprom->et2phyaddr, 0, fb);
-
- nvram_read_macaddr(prefix, "macaddr", sprom->il0mac, fallback);
- nvram_read_macaddr(prefix, "il0macaddr", sprom->il0mac, fallback);
-
- /* The address prefix 00:90:4C is used by Broadcom in their initial
- * configuration. When a mac address with the prefix 00:90:4C is used
- * all devices from the same series are sharing the same mac address.
- * To prevent mac address collisions we replace them with a mac address
- * based on the base address.
- */
- if (!bcm47xx_is_valid_mac(sprom->il0mac)) {
- u8 mac[6];
-
- nvram_read_macaddr(NULL, "et0macaddr", mac, false);
- if (bcm47xx_is_valid_mac(mac)) {
- int err = bcm47xx_increase_mac_addr(mac, mac_addr_used);
-
- if (!err) {
- ether_addr_copy(sprom->il0mac, mac);
- mac_addr_used++;
- }
- }
- }
-}
-
-static void bcm47xx_fill_board_data(struct ssb_sprom *sprom, const char *prefix,
- bool fallback)
-{
- nvram_read_u32_2(prefix, "boardflags", &sprom->boardflags_lo,
- &sprom->boardflags_hi, fallback);
- nvram_read_u32_2(prefix, "boardflags2", &sprom->boardflags2_lo,
- &sprom->boardflags2_hi, fallback);
-}
-
-void bcm47xx_fill_sprom(struct ssb_sprom *sprom, const char *prefix,
- bool fallback)
-{
- bcm47xx_fill_sprom_ethernet(sprom, prefix, fallback);
- bcm47xx_fill_board_data(sprom, prefix, fallback);
-
- nvram_read_u8(prefix, NULL, "sromrev", &sprom->revision, 0, fallback);
-
- /* Entries requiring custom functions */
- nvram_read_alpha2(prefix, "ccode", sprom->alpha2, fallback);
- if (sprom->revision >= 3)
- nvram_read_leddc(prefix, "leddc", &sprom->leddc_on_time,
- &sprom->leddc_off_time, fallback);
-
- switch (sprom->revision) {
- case 4:
- case 5:
- bcm47xx_fill_sprom_path_r4589(sprom, prefix, fallback);
- bcm47xx_fill_sprom_path_r45(sprom, prefix, fallback);
- break;
- case 8:
- case 9:
- bcm47xx_fill_sprom_path_r4589(sprom, prefix, fallback);
- break;
- }
-
- bcm47xx_sprom_fill_auto(sprom, prefix, fallback);
-}
-
-#if defined(CONFIG_BCM47XX_SSB)
-static int bcm47xx_get_sprom_ssb(struct ssb_bus *bus, struct ssb_sprom *out)
-{
- char prefix[10];
-
- switch (bus->bustype) {
- case SSB_BUSTYPE_SSB:
- bcm47xx_fill_sprom(out, NULL, false);
- return 0;
- case SSB_BUSTYPE_PCI:
- memset(out, 0, sizeof(struct ssb_sprom));
- snprintf(prefix, sizeof(prefix), "pci/%u/%u/",
- bus->host_pci->bus->number + 1,
- PCI_SLOT(bus->host_pci->devfn));
- bcm47xx_fill_sprom(out, prefix, false);
- return 0;
- default:
- pr_warn("Unable to fill SPROM for given bustype.\n");
- return -EINVAL;
- }
-}
-#endif
-
-#if defined(CONFIG_BCM47XX_BCMA)
-/*
- * Having many NVRAM entries for PCI devices led to repeating prefixes like
- * pci/1/1/ all the time and wasting flash space. So at some point Broadcom
- * decided to introduce prefixes like 0: 1: 2: etc.
- * If we find e.g. devpath0=pci/2/1 or devpath0=pci/2/1/ we should use 0:
- * instead of pci/2/1/.
- */
-static void bcm47xx_sprom_apply_prefix_alias(char *prefix, size_t prefix_size)
-{
- size_t prefix_len = strlen(prefix);
- size_t short_len = prefix_len - 1;
- char nvram_var[10];
- char buf[20];
- int i;
-
- /* Passed prefix has to end with a slash */
- if (prefix_len <= 0 || prefix[prefix_len - 1] != '/')
- return;
-
- for (i = 0; i < 3; i++) {
- if (snprintf(nvram_var, sizeof(nvram_var), "devpath%d", i) <= 0)
- continue;
- if (bcm47xx_nvram_getenv(nvram_var, buf, sizeof(buf)) < 0)
- continue;
- if (!strcmp(buf, prefix) ||
- (short_len && strlen(buf) == short_len && !strncmp(buf, prefix, short_len))) {
- snprintf(prefix, prefix_size, "%d:", i);
- return;
- }
- }
-}
-
-static int bcm47xx_get_sprom_bcma(struct bcma_bus *bus, struct ssb_sprom *out)
-{
- struct bcma_boardinfo *binfo = &bus->boardinfo;
- struct bcma_device *core;
- char buf[10];
- char *prefix;
- bool fallback = false;
-
- switch (bus->hosttype) {
- case BCMA_HOSTTYPE_PCI:
- memset(out, 0, sizeof(struct ssb_sprom));
- /* On BCM47XX all PCI buses share the same domain */
- if (config_enabled(CONFIG_BCM47XX))
- snprintf(buf, sizeof(buf), "pci/%u/%u/",
- bus->host_pci->bus->number + 1,
- PCI_SLOT(bus->host_pci->devfn));
- else
- snprintf(buf, sizeof(buf), "pci/%u/%u/",
- pci_domain_nr(bus->host_pci->bus) + 1,
- bus->host_pci->bus->number);
- bcm47xx_sprom_apply_prefix_alias(buf, sizeof(buf));
- prefix = buf;
- break;
- case BCMA_HOSTTYPE_SOC:
- memset(out, 0, sizeof(struct ssb_sprom));
- core = bcma_find_core(bus, BCMA_CORE_80211);
- if (core) {
- snprintf(buf, sizeof(buf), "sb/%u/",
- core->core_index);
- prefix = buf;
- fallback = true;
- } else {
- prefix = NULL;
- }
- break;
- default:
- pr_warn("Unable to fill SPROM for given bustype.\n");
- return -EINVAL;
- }
-
- nvram_read_u16(prefix, NULL, "boardvendor", &binfo->vendor, 0, true);
- if (!binfo->vendor)
- binfo->vendor = SSB_BOARDVENDOR_BCM;
- nvram_read_u16(prefix, NULL, "boardtype", &binfo->type, 0, true);
-
- bcm47xx_fill_sprom(out, prefix, fallback);
-
- return 0;
-}
-#endif
-
-/*
- * On bcm47xx we need to register SPROM fallback handler very early, so we can't
- * use anything like platform device / driver for this.
- */
-void bcm47xx_sprom_register_fallbacks(void)
-{
-#if defined(CONFIG_BCM47XX_SSB)
- if (ssb_arch_register_fallback_sprom(&bcm47xx_get_sprom_ssb))
- pr_warn("Failed to register ssb SPROM handler\n");
-#endif
-
-#if defined(CONFIG_BCM47XX_BCMA)
- if (bcma_arch_register_fallback_sprom(&bcm47xx_get_sprom_bcma))
- pr_warn("Failed to register bcma SPROM handler\n");
-#endif
-}
diff --git a/arch/mips/bmips/Kconfig b/arch/mips/bmips/Kconfig
index e2c4fd6..264328d 100644
--- a/arch/mips/bmips/Kconfig
+++ b/arch/mips/bmips/Kconfig
@@ -21,6 +21,10 @@
bool "BCM93384WVG Viper CPU (EXPERIMENTAL)"
select BUILTIN_DTB
+config DT_BCM96358NB4SER
+ bool "BCM96358NB4SER"
+ select BUILTIN_DTB
+
config DT_BCM96368MVWG
bool "BCM96368MVWG"
select BUILTIN_DTB
diff --git a/arch/mips/bmips/setup.c b/arch/mips/bmips/setup.c
index 3553528..f146d12 100644
--- a/arch/mips/bmips/setup.c
+++ b/arch/mips/bmips/setup.c
@@ -95,6 +95,15 @@
bcm63xx_fixup_cpu1();
}
+static void bcm6358_quirks(void)
+{
+ /*
+ * BCM6358 needs special handling for its shared TLB, so
+ * disable SMP for now
+ */
+ bmips_smp_enabled = 0;
+}
+
static void bcm6368_quirks(void)
{
bcm63xx_fixup_cpu1();
@@ -104,13 +113,16 @@
{ "brcm,bcm3384-viper", &bcm3384_viper_quirks },
{ "brcm,bcm33843-viper", &bcm3384_viper_quirks },
{ "brcm,bcm6328", &bcm6328_quirks },
+ { "brcm,bcm6358", &bcm6358_quirks },
{ "brcm,bcm6368", &bcm6368_quirks },
{ "brcm,bcm63168", &bcm6368_quirks },
+ { "brcm,bcm63268", &bcm6368_quirks },
{ },
};
void __init prom_init(void)
{
+ bmips_cpu_setup();
register_bmips_smp_ops();
}
diff --git a/arch/mips/boot/compressed/Makefile b/arch/mips/boot/compressed/Makefile
index 309d2ad..90aca95 100644
--- a/arch/mips/boot/compressed/Makefile
+++ b/arch/mips/boot/compressed/Makefile
@@ -37,8 +37,13 @@
vmlinuzobjs-$(CONFIG_SYS_SUPPORTS_ZBOOT_UART16550) += $(obj)/uart-16550.o
vmlinuzobjs-$(CONFIG_SYS_SUPPORTS_ZBOOT_UART_PROM) += $(obj)/uart-prom.o
vmlinuzobjs-$(CONFIG_MIPS_ALCHEMY) += $(obj)/uart-alchemy.o
+vmlinuzobjs-$(CONFIG_ATH79) += $(obj)/uart-ath79.o
endif
+extra-y += uart-ath79.c
+$(obj)/uart-ath79.c: $(srctree)/arch/mips/ath79/early_printk.c
+ $(call cmd,shipped)
+
vmlinuzobjs-$(CONFIG_KERNEL_XZ) += $(obj)/ashldi3.o $(obj)/bswapsi.o
extra-y += ashldi3.c bswapsi.c
diff --git a/arch/mips/boot/dts/brcm/Makefile b/arch/mips/boot/dts/brcm/Makefile
index eabeb60..fda9d38 100644
--- a/arch/mips/boot/dts/brcm/Makefile
+++ b/arch/mips/boot/dts/brcm/Makefile
@@ -1,5 +1,6 @@
dtb-$(CONFIG_DT_BCM93384WVG) += bcm93384wvg.dtb
dtb-$(CONFIG_DT_BCM93384WVG_VIPER) += bcm93384wvg_viper.dtb
+dtb-$(CONFIG_DT_BCM96358NB4SER) += bcm96358nb4ser.dtb
dtb-$(CONFIG_DT_BCM96368MVWG) += bcm96368mvwg.dtb
dtb-$(CONFIG_DT_BCM9EJTAGPRB) += bcm9ejtagprb.dtb
dtb-$(CONFIG_DT_BCM97125CBMB) += bcm97125cbmb.dtb
@@ -14,6 +15,7 @@
dtb-$(CONFIG_DT_NONE) += \
bcm93384wvg.dtb \
bcm93384wvg_viper.dtb \
+ bcm96358nb4ser.dtb \
bcm96368mvwg.dtb \
bcm9ejtagprb.dtb \
bcm97125cbmb.dtb \
diff --git a/arch/mips/boot/dts/brcm/bcm6328.dtsi b/arch/mips/boot/dts/brcm/bcm6328.dtsi
index 9d19236..5633b9d 100644
--- a/arch/mips/boot/dts/brcm/bcm6328.dtsi
+++ b/arch/mips/boot/dts/brcm/bcm6328.dtsi
@@ -23,7 +23,7 @@
};
clocks {
- periph_clk: periph_clk {
+ periph_clk: periph-clk {
compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <50000000>;
@@ -31,11 +31,11 @@
};
aliases {
- leds0 = &leds0;
- uart0 = &uart0;
+ serial0 = &uart0;
+ serial1 = &uart1;
};
- cpu_intc: cpu_intc {
+ cpu_intc: interrupt-controller {
#address-cells = <0>;
compatible = "mti,cpu-interrupt-controller";
@@ -50,16 +50,16 @@
compatible = "simple-bus";
ranges;
- periph_intc: periph_intc@10000020 {
- compatible = "brcm,bcm3380-l2-intc";
- reg = <0x10000024 0x4 0x1000002c 0x4>,
- <0x10000020 0x4 0x10000028 0x4>;
+ periph_intc: interrupt-controller@10000020 {
+ compatible = "brcm,bcm6345-l1-intc";
+ reg = <0x10000020 0x10>,
+ <0x10000030 0x10>;
interrupt-controller;
#interrupt-cells = <1>;
interrupt-parent = <&cpu_intc>;
- interrupts = <2>;
+ interrupts = <2>, <3>;
};
uart0: serial@10000100 {
@@ -71,13 +71,22 @@
status = "disabled";
};
- timer: timer@10000040 {
+ uart1: serial@10000120 {
+ compatible = "brcm,bcm6345-uart";
+ reg = <0x10000120 0x18>;
+ interrupt-parent = <&periph_intc>;
+ interrupts = <39>;
+ clocks = <&periph_clk>;
+ status = "disabled";
+ };
+
+ timer: syscon@10000040 {
compatible = "syscon";
reg = <0x10000040 0x2c>;
native-endian;
};
- reboot {
+ reboot: syscon-reboot@10000068 {
compatible = "syscon-reboot";
regmap = <&timer>;
offset = <0x28>;
@@ -91,5 +100,24 @@
reg = <0x10000800 0x24>;
status = "disabled";
};
+
+ ehci: usb@10002500 {
+ compatible = "brcm,bcm6328-ehci", "generic-ehci";
+ reg = <0x10002500 0x100>;
+ big-endian;
+ interrupt-parent = <&periph_intc>;
+ interrupts = <42>;
+ status = "disabled";
+ };
+
+ ohci: usb@10002600 {
+ compatible = "brcm,bcm6328-ohci", "generic-ohci";
+ reg = <0x10002600 0x100>;
+ big-endian;
+ no-big-frame-no;
+ interrupt-parent = <&periph_intc>;
+ interrupts = <41>;
+ status = "disabled";
+ };
};
};
diff --git a/arch/mips/boot/dts/brcm/bcm6358.dtsi b/arch/mips/boot/dts/brcm/bcm6358.dtsi
new file mode 100644
index 0000000..f9d8d392
--- /dev/null
+++ b/arch/mips/boot/dts/brcm/bcm6358.dtsi
@@ -0,0 +1,130 @@
+/ {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "brcm,bcm6358";
+
+ cpus {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ mips-hpt-frequency = <150000000>;
+
+ cpu@0 {
+ compatible = "brcm,bmips4350";
+ device_type = "cpu";
+ reg = <0>;
+ };
+
+ cpu@1 {
+ compatible = "brcm,bmips4350";
+ device_type = "cpu";
+ reg = <1>;
+ };
+ };
+
+ clocks {
+ periph_clk: periph-clk {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <50000000>;
+ };
+ };
+
+ aliases {
+ serial0 = &uart0;
+ serial1 = &uart1;
+ };
+
+ cpu_intc: interrupt-controller {
+ #address-cells = <0>;
+ compatible = "mti,cpu-interrupt-controller";
+
+ interrupt-controller;
+ #interrupt-cells = <1>;
+ };
+
+ ubus {
+ #address-cells = <1>;
+ #size-cells = <1>;
+
+ compatible = "simple-bus";
+ ranges;
+
+ periph_cntl: syscon@fffe0000 {
+ compatible = "syscon";
+ reg = <0xfffe0000 0xc>;
+ native-endian;
+ };
+
+ reboot: syscon-reboot@fffe0008 {
+ compatible = "syscon-reboot";
+ regmap = <&periph_cntl>;
+ offset = <0x8>;
+ mask = <0x1>;
+ };
+
+ periph_intc: interrupt-controller@fffe000c {
+ compatible = "brcm,bcm6345-l1-intc";
+ reg = <0xfffe000c 0x8>,
+ <0xfffe0038 0x8>;
+
+ interrupt-controller;
+ #interrupt-cells = <1>;
+
+ interrupt-parent = <&cpu_intc>;
+ interrupts = <2>, <3>;
+ };
+
+ leds0: led-controller@fffe00d0 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ compatible = "brcm,bcm6358-leds";
+ reg = <0xfffe00d0 0x8>;
+
+ status = "disabled";
+ };
+
+ uart0: serial@fffe0100 {
+ compatible = "brcm,bcm6345-uart";
+ reg = <0xfffe0100 0x18>;
+
+ interrupt-parent = <&periph_intc>;
+ interrupts = <2>;
+
+ clocks = <&periph_clk>;
+
+ status = "disabled";
+ };
+
+ uart1: serial@fffe0120 {
+ compatible = "brcm,bcm6345-uart";
+ reg = <0xfffe0120 0x18>;
+
+ interrupt-parent = <&periph_intc>;
+ interrupts = <3>;
+
+ clocks = <&periph_clk>;
+
+ status = "disabled";
+ };
+
+ ehci: usb@fffe1300 {
+ compatible = "brcm,bcm6358-ehci", "generic-ehci";
+ reg = <0xfffe1300 0x100>;
+ big-endian;
+ interrupt-parent = <&periph_intc>;
+ interrupts = <10>;
+ status = "disabled";
+ };
+
+ ohci: usb@fffe1400 {
+ compatible = "brcm,bcm6358-ohci", "generic-ohci";
+ reg = <0xfffe1400 0x100>;
+ big-endian;
+ no-big-frame-no;
+ interrupt-parent = <&periph_intc>;
+ interrupts = <5>;
+ status = "disabled";
+ };
+ };
+};
diff --git a/arch/mips/boot/dts/brcm/bcm6368.dtsi b/arch/mips/boot/dts/brcm/bcm6368.dtsi
index 1f6b9b5..d0e3a70 100644
--- a/arch/mips/boot/dts/brcm/bcm6368.dtsi
+++ b/arch/mips/boot/dts/brcm/bcm6368.dtsi
@@ -20,11 +20,10 @@
device_type = "cpu";
reg = <1>;
};
-
};
clocks {
- periph_clk: periph_clk {
+ periph_clk: periph-clk {
compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <50000000>;
@@ -32,11 +31,11 @@
};
aliases {
- leds0 = &leds0;
- uart0 = &uart0;
+ serial0 = &uart0;
+ serial1 = &uart1;
};
- cpu_intc: cpu_intc {
+ cpu_intc: interrupt-controller {
#address-cells = <0>;
compatible = "mti,cpu-interrupt-controller";
@@ -64,16 +63,16 @@
mask = <0x1>;
};
- periph_intc: periph_intc@10000020 {
- compatible = "brcm,bcm3380-l2-intc";
- reg = <0x10000024 0x4 0x1000002c 0x4>,
- <0x10000020 0x4 0x10000028 0x4>;
+ periph_intc: interrupt-controller@10000020 {
+ compatible = "brcm,bcm6345-l1-intc";
+ reg = <0x10000020 0x10>,
+ <0x10000030 0x10>;
interrupt-controller;
#interrupt-cells = <1>;
interrupt-parent = <&cpu_intc>;
- interrupts = <2>;
+ interrupts = <2>, <3>;
};
leds0: led-controller@100000d0 {
@@ -93,7 +92,16 @@
status = "disabled";
};
- ehci0: usb@10001500 {
+ uart1: serial@10000120 {
+ compatible = "brcm,bcm6345-uart";
+ reg = <0x10000120 0x18>;
+ interrupt-parent = <&periph_intc>;
+ interrupts = <3>;
+ clocks = <&periph_clk>;
+ status = "disabled";
+ };
+
+ ehci: usb@10001500 {
compatible = "brcm,bcm6368-ehci", "generic-ehci";
reg = <0x10001500 0x100>;
big-endian;
@@ -102,7 +110,7 @@
status = "disabled";
};
- ohci0: usb@10001600 {
+ ohci: usb@10001600 {
compatible = "brcm,bcm6368-ohci", "generic-ohci";
reg = <0x10001600 0x100>;
big-endian;
diff --git a/arch/mips/boot/dts/brcm/bcm7125.dtsi b/arch/mips/boot/dts/brcm/bcm7125.dtsi
index 3ae1605..550e1d9 100644
--- a/arch/mips/boot/dts/brcm/bcm7125.dtsi
+++ b/arch/mips/boot/dts/brcm/bcm7125.dtsi
@@ -85,14 +85,15 @@
compatible = "brcm,bcm7120-l2-intc";
reg = <0x406780 0x8>;
- brcm,int-map-mask = <0x44>;
+ brcm,int-map-mask = <0x44>, <0xf000000>;
brcm,int-fwd-mask = <0x70000>;
interrupt-controller;
#interrupt-cells = <1>;
interrupt-parent = <&periph_intc>;
- interrupts = <18>;
+ interrupts = <18>, <19>;
+ interrupt-names = "upg_main", "upg_bsc";
};
sun_top_ctrl: syscon@404000 {
@@ -118,6 +119,70 @@
status = "disabled";
};
+ uart1: serial@406b40 {
+ compatible = "ns16550a";
+ reg = <0x406b40 0x20>;
+ reg-io-width = <0x4>;
+ reg-shift = <0x2>;
+ native-endian;
+ interrupt-parent = <&periph_intc>;
+ interrupts = <64>;
+ clocks = <&uart_clk>;
+ status = "disabled";
+ };
+
+ uart2: serial@406b80 {
+ compatible = "ns16550a";
+ reg = <0x406b80 0x20>;
+ reg-io-width = <0x4>;
+ reg-shift = <0x2>;
+ native-endian;
+ interrupt-parent = <&periph_intc>;
+ interrupts = <65>;
+ clocks = <&uart_clk>;
+ status = "disabled";
+ };
+
+ bsca: i2c@406200 {
+ clock-frequency = <390000>;
+ compatible = "brcm,brcmstb-i2c";
+ interrupt-parent = <&upg_irq0_intc>;
+ reg = <0x406200 0x58>;
+ interrupts = <24>;
+ interrupt-names = "upg_bsca";
+ status = "disabled";
+ };
+
+ bscb: i2c@406280 {
+ clock-frequency = <390000>;
+ compatible = "brcm,brcmstb-i2c";
+ interrupt-parent = <&upg_irq0_intc>;
+ reg = <0x406280 0x58>;
+ interrupts = <25>;
+ interrupt-names = "upg_bscb";
+ status = "disabled";
+ };
+
+ bscc: i2c@406300 {
+ clock-frequency = <390000>;
+ compatible = "brcm,brcmstb-i2c";
+ interrupt-parent = <&upg_irq0_intc>;
+ reg = <0x406300 0x58>;
+ interrupts = <26>;
+ interrupt-names = "upg_bscc";
+ status = "disabled";
+ };
+
+ bscd: i2c@406380 {
+ clock-frequency = <390000>;
+ compatible = "brcm,brcmstb-i2c";
+ interrupt-parent = <&upg_irq0_intc>;
+ reg = <0x406380 0x58>;
+ interrupts = <27>;
+ interrupt-names = "upg_bscd";
+ status = "disabled";
+ };
+
ehci0: usb@488300 {
compatible = "brcm,bcm7125-ehci", "generic-ehci";
reg = <0x488300 0x100>;
diff --git a/arch/mips/boot/dts/brcm/bcm7346.dtsi b/arch/mips/boot/dts/brcm/bcm7346.dtsi
index be79919..ec95906 100644
--- a/arch/mips/boot/dts/brcm/bcm7346.dtsi
+++ b/arch/mips/boot/dts/brcm/bcm7346.dtsi
@@ -24,8 +24,6 @@
aliases {
uart0 = &uart0;
- uart1 = &uart1;
- uart2 = &uart2;
};
cpu_intc: cpu_intc {
@@ -323,8 +321,6 @@
interrupts = <40>;
#address-cells = <1>;
#size-cells = <0>;
- brcm,broken-ncq;
- brcm,broken-phy;
status = "disabled";
sata0: sata-port@0 {
@@ -338,7 +334,7 @@
};
};
- sata_phy: sata-phy@1800000 {
+ sata_phy: sata-phy@180100 {
compatible = "brcm,bcm7425-sata-phy", "brcm,phy-sata3";
reg = <0x180100 0x0eff>;
reg-names = "phy";
diff --git a/arch/mips/boot/dts/brcm/bcm7358.dtsi b/arch/mips/boot/dts/brcm/bcm7358.dtsi
index 060805b..ca57fb5 100644
--- a/arch/mips/boot/dts/brcm/bcm7358.dtsi
+++ b/arch/mips/boot/dts/brcm/bcm7358.dtsi
@@ -18,8 +18,6 @@
aliases {
uart0 = &uart0;
- uart1 = &uart1;
- uart2 = &uart2;
};
cpu_intc: cpu_intc {
diff --git a/arch/mips/boot/dts/brcm/bcm7360.dtsi b/arch/mips/boot/dts/brcm/bcm7360.dtsi
index bcdb09b..1c0c3d4 100644
--- a/arch/mips/boot/dts/brcm/bcm7360.dtsi
+++ b/arch/mips/boot/dts/brcm/bcm7360.dtsi
@@ -18,8 +18,6 @@
aliases {
uart0 = &uart0;
- uart1 = &uart1;
- uart2 = &uart2;
};
cpu_intc: cpu_intc {
@@ -241,5 +239,45 @@
interrupts = <66>;
status = "disabled";
};
+
+ sata: sata@181000 {
+ compatible = "brcm,bcm7425-ahci", "brcm,sata3-ahci";
+ reg-names = "ahci", "top-ctrl";
+ reg = <0x181000 0xa9c>, <0x180020 0x1c>;
+ interrupt-parent = <&periph_intc>;
+ interrupts = <86>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+
+ sata0: sata-port@0 {
+ reg = <0>;
+ phys = <&sata_phy0>;
+ };
+
+ sata1: sata-port@1 {
+ reg = <1>;
+ phys = <&sata_phy1>;
+ };
+ };
+
+ sata_phy: sata-phy@180100 {
+ compatible = "brcm,bcm7425-sata-phy", "brcm,phy-sata3";
+ reg = <0x180100 0x0eff>;
+ reg-names = "phy";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+
+ sata_phy0: sata-phy@0 {
+ reg = <0>;
+ #phy-cells = <0>;
+ };
+
+ sata_phy1: sata-phy@1 {
+ reg = <1>;
+ #phy-cells = <0>;
+ };
+ };
};
};
diff --git a/arch/mips/boot/dts/brcm/bcm7362.dtsi b/arch/mips/boot/dts/brcm/bcm7362.dtsi
index d3b1b76..6b4713a 100644
--- a/arch/mips/boot/dts/brcm/bcm7362.dtsi
+++ b/arch/mips/boot/dts/brcm/bcm7362.dtsi
@@ -24,8 +24,6 @@
aliases {
uart0 = &uart0;
- uart1 = &uart1;
- uart2 = &uart2;
};
cpu_intc: cpu_intc {
@@ -246,8 +244,6 @@
interrupts = <86>;
#address-cells = <1>;
#size-cells = <0>;
- brcm,broken-ncq;
- brcm,broken-phy;
status = "disabled";
sata0: sata-port@0 {
@@ -261,7 +257,7 @@
};
};
- sata_phy: sata-phy@1800000 {
+ sata_phy: sata-phy@180100 {
compatible = "brcm,bcm7425-sata-phy", "brcm,phy-sata3";
reg = <0x180100 0x0eff>;
reg-names = "phy";
diff --git a/arch/mips/boot/dts/brcm/bcm7420.dtsi b/arch/mips/boot/dts/brcm/bcm7420.dtsi
index 3302a1b..0586bf6 100644
--- a/arch/mips/boot/dts/brcm/bcm7420.dtsi
+++ b/arch/mips/boot/dts/brcm/bcm7420.dtsi
@@ -86,14 +86,15 @@
compatible = "brcm,bcm7120-l2-intc";
reg = <0x406780 0x8>;
- brcm,int-map-mask = <0x44>;
+ brcm,int-map-mask = <0x44>, <0x1f000000>;
brcm,int-fwd-mask = <0x70000>;
interrupt-controller;
#interrupt-cells = <1>;
interrupt-parent = <&periph_intc>;
- interrupts = <18>;
+ interrupts = <18>, <19>;
+ interrupt-names = "upg_main", "upg_bsc";
};
sun_top_ctrl: syscon@404000 {
@@ -118,6 +119,78 @@
status = "disabled";
};
+ uart1: serial@406b40 {
+ compatible = "ns16550a";
+ reg = <0x406b40 0x20>;
+ reg-io-width = <0x4>;
+ reg-shift = <0x2>;
+ interrupt-parent = <&periph_intc>;
+ interrupts = <64>;
+ clocks = <&uart_clk>;
+ status = "disabled";
+ };
+
+ uart2: serial@406b80 {
+ compatible = "ns16550a";
+ reg = <0x406b80 0x20>;
+ reg-io-width = <0x4>;
+ reg-shift = <0x2>;
+ interrupt-parent = <&periph_intc>;
+ interrupts = <65>;
+ clocks = <&uart_clk>;
+ status = "disabled";
+ };
+
+ bsca: i2c@406200 {
+ clock-frequency = <390000>;
+ compatible = "brcm,brcmstb-i2c";
+ interrupt-parent = <&upg_irq0_intc>;
+ reg = <0x406200 0x58>;
+ interrupts = <24>;
+ interrupt-names = "upg_bsca";
+ status = "disabled";
+ };
+
+ bscb: i2c@406280 {
+ clock-frequency = <390000>;
+ compatible = "brcm,brcmstb-i2c";
+ interrupt-parent = <&upg_irq0_intc>;
+ reg = <0x406280 0x58>;
+ interrupts = <25>;
+ interrupt-names = "upg_bscb";
+ status = "disabled";
+ };
+
+ bscc: i2c@406300 {
+ clock-frequency = <390000>;
+ compatible = "brcm,brcmstb-i2c";
+ interrupt-parent = <&upg_irq0_intc>;
+ reg = <0x406300 0x58>;
+ interrupts = <26>;
+ interrupt-names = "upg_bscc";
+ status = "disabled";
+ };
+
+ bscd: i2c@406380 {
+ clock-frequency = <390000>;
+ compatible = "brcm,brcmstb-i2c";
+ interrupt-parent = <&upg_irq0_intc>;
+ reg = <0x406380 0x58>;
+ interrupts = <27>;
+ interrupt-names = "upg_bscd";
+ status = "disabled";
+ };
+
+ bsce: i2c@406800 {
+ clock-frequency = <390000>;
+ compatible = "brcm,brcmstb-i2c";
+ interrupt-parent = <&upg_irq0_intc>;
+ reg = <0x406800 0x58>;
+ interrupts = <28>;
+ interrupt-names = "upg_bsce";
+ status = "disabled";
+ };
+
enet0: ethernet@468000 {
phy-mode = "internal";
phy-handle = <&phy1>;
diff --git a/arch/mips/boot/dts/brcm/bcm7425.dtsi b/arch/mips/boot/dts/brcm/bcm7425.dtsi
index 15b27aa..c1c15ed 100644
--- a/arch/mips/boot/dts/brcm/bcm7425.dtsi
+++ b/arch/mips/boot/dts/brcm/bcm7425.dtsi
@@ -87,14 +87,32 @@
compatible = "brcm,bcm7120-l2-intc";
reg = <0x406780 0x8>;
- brcm,int-map-mask = <0x44>;
+ brcm,int-map-mask = <0x44>, <0x7000000>;
brcm,int-fwd-mask = <0x70000>;
interrupt-controller;
#interrupt-cells = <1>;
interrupt-parent = <&periph_intc>;
- interrupts = <55>;
+ interrupts = <55>, <53>;
+ interrupt-names = "upg_main", "upg_bsc";
+ };
+
+ upg_aon_irq0_intc: upg_aon_irq0_intc@409480 {
+ compatible = "brcm,bcm7120-l2-intc";
+ reg = <0x409480 0x8>;
+
+ brcm,int-map-mask = <0x40>, <0x18000000>, <0x100000>;
+ brcm,int-fwd-mask = <0>;
+ brcm,irq-can-wake;
+
+ interrupt-controller;
+ #interrupt-cells = <1>;
+
+ interrupt-parent = <&periph_intc>;
+ interrupts = <56>, <54>, <59>;
+ interrupt-names = "upg_main_aon", "upg_bsc_aon",
+ "upg_spi";
};
sun_top_ctrl: syscon@404000 {
@@ -119,6 +137,78 @@
status = "disabled";
};
+ uart1: serial@406b40 {
+ compatible = "ns16550a";
+ reg = <0x406b40 0x20>;
+ reg-io-width = <0x4>;
+ reg-shift = <0x2>;
+ interrupt-parent = <&periph_intc>;
+ interrupts = <62>;
+ clocks = <&uart_clk>;
+ status = "disabled";
+ };
+
+ uart2: serial@406b80 {
+ compatible = "ns16550a";
+ reg = <0x406b80 0x20>;
+ reg-io-width = <0x4>;
+ reg-shift = <0x2>;
+ interrupt-parent = <&periph_intc>;
+ interrupts = <63>;
+ clocks = <&uart_clk>;
+ status = "disabled";
+ };
+
+ bsca: i2c@409180 {
+ clock-frequency = <390000>;
+ compatible = "brcm,brcmstb-i2c";
+ interrupt-parent = <&upg_aon_irq0_intc>;
+ reg = <0x409180 0x58>;
+ interrupts = <27>;
+ interrupt-names = "upg_bsca";
+ status = "disabled";
+ };
+
+ bscb: i2c@409400 {
+ clock-frequency = <390000>;
+ compatible = "brcm,brcmstb-i2c";
+ interrupt-parent = <&upg_aon_irq0_intc>;
+ reg = <0x409400 0x58>;
+ interrupts = <28>;
+ interrupt-names = "upg_bscb";
+ status = "disabled";
+ };
+
+ bscc: i2c@406200 {
+ clock-frequency = <390000>;
+ compatible = "brcm,brcmstb-i2c";
+ interrupt-parent = <&upg_irq0_intc>;
+ reg = <0x406200 0x58>;
+ interrupts = <24>;
+ interrupt-names = "upg_bscc";
+ status = "disabled";
+ };
+
+ bscd: i2c@406280 {
+ clock-frequency = <390000>;
+ compatible = "brcm,brcmstb-i2c";
+ interrupt-parent = <&upg_irq0_intc>;
+ reg = <0x406280 0x58>;
+ interrupts = <25>;
+ interrupt-names = "upg_bscd";
+ status = "disabled";
+ };
+
+ bsce: i2c@406300 {
+ clock-frequency = <390000>;
+ compatible = "brcm,brcmstb-i2c";
+ interrupt-parent = <&upg_irq0_intc>;
+ reg = <0x406300 0x58>;
+ interrupts = <26>;
+ interrupt-names = "upg_bsce";
+ status = "disabled";
+ };
+
enet0: ethernet@b80000 {
phy-mode = "internal";
phy-handle = <&phy1>;
@@ -227,11 +317,9 @@
reg-names = "ahci", "top-ctrl";
reg = <0x181000 0xa9c>, <0x180020 0x1c>;
interrupt-parent = <&periph_intc>;
- interrupts = <40>;
+ interrupts = <41>;
#address-cells = <1>;
#size-cells = <0>;
- brcm,broken-ncq;
- brcm,broken-phy;
status = "disabled";
sata0: sata-port@0 {
@@ -245,7 +333,7 @@
};
};
- sata_phy: sata-phy@1800000 {
+ sata_phy: sata-phy@180100 {
compatible = "brcm,bcm7425-sata-phy", "brcm,phy-sata3";
reg = <0x180100 0x0eff>;
reg-names = "phy";
diff --git a/arch/mips/boot/dts/brcm/bcm7435.dtsi b/arch/mips/boot/dts/brcm/bcm7435.dtsi
index 56035e5..a874d3a 100644
--- a/arch/mips/boot/dts/brcm/bcm7435.dtsi
+++ b/arch/mips/boot/dts/brcm/bcm7435.dtsi
@@ -7,7 +7,7 @@
#address-cells = <1>;
#size-cells = <0>;
- mips-hpt-frequency = <163125000>;
+ mips-hpt-frequency = <175625000>;
cpu@0 {
compatible = "brcm,bmips5200";
@@ -63,13 +63,14 @@
periph_intc: periph_intc@41b500 {
compatible = "brcm,bcm7038-l1-intc";
- reg = <0x41b500 0x40>, <0x41b600 0x40>;
+ reg = <0x41b500 0x40>, <0x41b600 0x40>,
+ <0x41b700 0x40>, <0x41b800 0x40>;
interrupt-controller;
#interrupt-cells = <1>;
interrupt-parent = <&cpu_intc>;
- interrupts = <2>, <3>;
+ interrupts = <2>, <3>, <2>, <3>;
};
sun_l2_intc: sun_l2_intc@403000 {
@@ -101,14 +102,32 @@
compatible = "brcm,bcm7120-l2-intc";
reg = <0x406780 0x8>;
- brcm,int-map-mask = <0x44>;
+ brcm,int-map-mask = <0x44>, <0x7000000>;
brcm,int-fwd-mask = <0x70000>;
interrupt-controller;
#interrupt-cells = <1>;
interrupt-parent = <&periph_intc>;
- interrupts = <60>;
+ interrupts = <60>, <58>;
+ interrupt-names = "upg_main", "upg_bsc";
+ };
+
+ upg_aon_irq0_intc: upg_aon_irq0_intc@409480 {
+ compatible = "brcm,bcm7120-l2-intc";
+ reg = <0x409480 0x8>;
+
+ brcm,int-map-mask = <0x40>, <0x18000000>, <0x100000>;
+ brcm,int-fwd-mask = <0>;
+ brcm,irq-can-wake;
+
+ interrupt-controller;
+ #interrupt-cells = <1>;
+
+ interrupt-parent = <&periph_intc>;
+ interrupts = <61>, <59>, <64>;
+ interrupt-names = "upg_main_aon", "upg_bsc_aon",
+ "upg_spi";
};
sun_top_ctrl: syscon@404000 {
@@ -133,6 +152,78 @@
status = "disabled";
};
+ uart1: serial@406b40 {
+ compatible = "ns16550a";
+ reg = <0x406b40 0x20>;
+ reg-io-width = <0x4>;
+ reg-shift = <0x2>;
+ interrupt-parent = <&periph_intc>;
+ interrupts = <67>;
+ clocks = <&uart_clk>;
+ status = "disabled";
+ };
+
+ uart2: serial@406b80 {
+ compatible = "ns16550a";
+ reg = <0x406b80 0x20>;
+ reg-io-width = <0x4>;
+ reg-shift = <0x2>;
+ interrupt-parent = <&periph_intc>;
+ interrupts = <68>;
+ clocks = <&uart_clk>;
+ status = "disabled";
+ };
+
+ bsca: i2c@406300 {
+ clock-frequency = <390000>;
+ compatible = "brcm,brcmstb-i2c";
+ interrupt-parent = <&upg_irq0_intc>;
+ reg = <0x406300 0x58>;
+ interrupts = <26>;
+ interrupt-names = "upg_bsca";
+ status = "disabled";
+ };
+
+ bscb: i2c@409400 {
+ clock-frequency = <390000>;
+ compatible = "brcm,brcmstb-i2c";
+ interrupt-parent = <&upg_aon_irq0_intc>;
+ reg = <0x409400 0x58>;
+ interrupts = <28>;
+ interrupt-names = "upg_bscb";
+ status = "disabled";
+ };
+
+ bscc: i2c@406200 {
+ clock-frequency = <390000>;
+ compatible = "brcm,brcmstb-i2c";
+ interrupt-parent = <&upg_irq0_intc>;
+ reg = <0x406200 0x58>;
+ interrupts = <24>;
+ interrupt-names = "upg_bscc";
+ status = "disabled";
+ };
+
+ bscd: i2c@406280 {
+ clock-frequency = <390000>;
+ compatible = "brcm,brcmstb-i2c";
+ interrupt-parent = <&upg_irq0_intc>;
+ reg = <0x406280 0x58>;
+ interrupts = <25>;
+ interrupt-names = "upg_bscd";
+ status = "disabled";
+ };
+
+ bsce: i2c@409180 {
+ clock-frequency = <390000>;
+ compatible = "brcm,brcmstb-i2c";
+ interrupt-parent = <&upg_aon_irq0_intc>;
+ reg = <0x409180 0x58>;
+ interrupts = <27>;
+ interrupt-names = "upg_bsce";
+ status = "disabled";
+ };
+
enet0: ethernet@b80000 {
phy-mode = "internal";
phy-handle = <&phy1>;
@@ -235,5 +326,45 @@
interrupts = <78>;
status = "disabled";
};
+
+ sata: sata@181000 {
+ compatible = "brcm,bcm7425-ahci", "brcm,sata3-ahci";
+ reg-names = "ahci", "top-ctrl";
+ reg = <0x181000 0xa9c>, <0x180020 0x1c>;
+ interrupt-parent = <&periph_intc>;
+ interrupts = <45>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+
+ sata0: sata-port@0 {
+ reg = <0>;
+ phys = <&sata_phy0>;
+ };
+
+ sata1: sata-port@1 {
+ reg = <1>;
+ phys = <&sata_phy1>;
+ };
+ };
+
+ sata_phy: sata-phy@180100 {
+ compatible = "brcm,bcm7425-sata-phy", "brcm,phy-sata3";
+ reg = <0x180100 0x0eff>;
+ reg-names = "phy";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ status = "disabled";
+
+ sata_phy0: sata-phy@0 {
+ reg = <0>;
+ #phy-cells = <0>;
+ };
+
+ sata_phy1: sata-phy@1 {
+ reg = <1>;
+ #phy-cells = <0>;
+ };
+ };
};
};
diff --git a/arch/mips/boot/dts/brcm/bcm96358nb4ser.dts b/arch/mips/boot/dts/brcm/bcm96358nb4ser.dts
new file mode 100644
index 0000000..f412117
--- /dev/null
+++ b/arch/mips/boot/dts/brcm/bcm96358nb4ser.dts
@@ -0,0 +1,46 @@
+/dts-v1/;
+
+/include/ "bcm6358.dtsi"
+
+/ {
+ compatible = "sfr,nb4-ser", "brcm,bcm6358";
+ model = "SFR Neufbox 4 (Sercomm)";
+
+ memory@0 {
+ device_type = "memory";
+ reg = <0x00000000 0x02000000>;
+ };
+
+ chosen {
+ stdout-path = &uart0;
+ };
+};
+
+&leds0 {
+ status = "ok";
+
+ led@0 {
+ reg = <0>;
+ active-low;
+ label = "nb4-ser:white:alarm";
+ };
+ led@2 {
+ reg = <2>;
+ active-low;
+ label = "nb4-ser:white:tv";
+ };
+ led@3 {
+ reg = <3>;
+ active-low;
+ label = "nb4-ser:white:tel";
+ };
+ led@4 {
+ reg = <4>;
+ active-low;
+ label = "nb4-ser:white:adsl";
+ };
+};
+
+&uart0 {
+ status = "okay";
+};
diff --git a/arch/mips/boot/dts/brcm/bcm96368mvwg.dts b/arch/mips/boot/dts/brcm/bcm96368mvwg.dts
index 0e890c2..8c71c68 100644
--- a/arch/mips/boot/dts/brcm/bcm96368mvwg.dts
+++ b/arch/mips/boot/dts/brcm/bcm96368mvwg.dts
@@ -22,10 +22,10 @@
};
/* FIXME: need to set up USB_CTRL registers first */
-&ehci0 {
+&ehci {
status = "disabled";
};
-&ohci0 {
+&ohci {
status = "disabled";
};
diff --git a/arch/mips/boot/dts/brcm/bcm97125cbmb.dts b/arch/mips/boot/dts/brcm/bcm97125cbmb.dts
index e046b11..f2449d1 100644
--- a/arch/mips/boot/dts/brcm/bcm97125cbmb.dts
+++ b/arch/mips/boot/dts/brcm/bcm97125cbmb.dts
@@ -21,6 +21,30 @@
status = "okay";
};
+&uart1 {
+ status = "okay";
+};
+
+&uart2 {
+ status = "okay";
+};
+
+&bsca {
+ status = "okay";
+};
+
+&bscb {
+ status = "okay";
+};
+
+&bscc {
+ status = "okay";
+};
+
+&bscd {
+ status = "okay";
+};
+
/* FIXME: USB is wonky; disable it for now */
&ehci0 {
status = "disabled";
diff --git a/arch/mips/boot/dts/brcm/bcm97360svmb.dts b/arch/mips/boot/dts/brcm/bcm97360svmb.dts
index d48462e..73124be 100644
--- a/arch/mips/boot/dts/brcm/bcm97360svmb.dts
+++ b/arch/mips/boot/dts/brcm/bcm97360svmb.dts
@@ -56,3 +56,11 @@
&ohci0 {
status = "okay";
};
+
+&sata {
+ status = "okay";
+};
+
+&sata_phy {
+ status = "okay";
+};
diff --git a/arch/mips/boot/dts/brcm/bcm97420c.dts b/arch/mips/boot/dts/brcm/bcm97420c.dts
index 67fe1f3..600d57a 100644
--- a/arch/mips/boot/dts/brcm/bcm97420c.dts
+++ b/arch/mips/boot/dts/brcm/bcm97420c.dts
@@ -23,6 +23,34 @@
status = "okay";
};
+&uart1 {
+ status = "okay";
+};
+
+&uart2 {
+ status = "okay";
+};
+
+&bsca {
+ status = "okay";
+};
+
+&bscb {
+ status = "okay";
+};
+
+&bscc {
+ status = "okay";
+};
+
+&bscd {
+ status = "okay";
+};
+
+&bsce {
+ status = "okay";
+};
+
/* FIXME: MAC driver comes up but cannot attach to PHY */
&enet0 {
status = "disabled";
diff --git a/arch/mips/boot/dts/brcm/bcm97425svmb.dts b/arch/mips/boot/dts/brcm/bcm97425svmb.dts
index 689c68a..119c714 100644
--- a/arch/mips/boot/dts/brcm/bcm97425svmb.dts
+++ b/arch/mips/boot/dts/brcm/bcm97425svmb.dts
@@ -23,6 +23,34 @@
status = "okay";
};
+&uart1 {
+ status = "okay";
+};
+
+&uart2 {
+ status = "okay";
+};
+
+&bsca {
+ status = "okay";
+};
+
+&bscb {
+ status = "okay";
+};
+
+&bscc {
+ status = "okay";
+};
+
+&bscd {
+ status = "okay";
+};
+
+&bsce {
+ status = "okay";
+};
+
&enet0 {
status = "okay";
};
diff --git a/arch/mips/boot/dts/brcm/bcm97435svmb.dts b/arch/mips/boot/dts/brcm/bcm97435svmb.dts
index 1df0881..43e3ba2 100644
--- a/arch/mips/boot/dts/brcm/bcm97435svmb.dts
+++ b/arch/mips/boot/dts/brcm/bcm97435svmb.dts
@@ -14,7 +14,7 @@
};
chosen {
- bootargs = "console=ttyS0,115200 maxcpus=1";
+ bootargs = "console=ttyS0,115200";
stdout-path = &uart0;
};
};
@@ -23,6 +23,34 @@
status = "okay";
};
+&uart1 {
+ status = "okay";
+};
+
+&uart2 {
+ status = "okay";
+};
+
+&bsca {
+ status = "okay";
+};
+
+&bscb {
+ status = "okay";
+};
+
+&bscc {
+ status = "okay";
+};
+
+&bscd {
+ status = "okay";
+};
+
+&bsce {
+ status = "okay";
+};
+
&enet0 {
status = "okay";
};
@@ -58,3 +86,11 @@
&ohci3 {
status = "okay";
};
+
+&sata {
+ status = "okay";
+};
+
+&sata_phy {
+ status = "okay";
+};
diff --git a/arch/mips/boot/dts/cavium-octeon/dlink_dsr-1000n.dts b/arch/mips/boot/dts/cavium-octeon/dlink_dsr-1000n.dts
new file mode 100644
index 0000000..d6bc994
--- /dev/null
+++ b/arch/mips/boot/dts/cavium-octeon/dlink_dsr-1000n.dts
@@ -0,0 +1,78 @@
+/*
+ * Device tree source for D-Link DSR-1000N.
+ *
+ * Written by: Aaro Koskinen <aaro.koskinen@iki.fi>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+/include/ "octeon_3xxx.dtsi"
+
+/ {
+ model = "dlink,dsr-1000n";
+
+ soc@0 {
+ smi0: mdio@1180000001800 {
+ phy8: ethernet-phy@8 {
+ reg = <8>;
+ compatible = "ethernet-phy-ieee802.3-c22";
+ };
+ };
+
+ pip: pip@11800a0000000 {
+ interface@0 {
+ ethernet@0 {
+ fixed-link {
+ speed = <1000>;
+ full-duplex;
+ };
+ };
+ ethernet@1 {
+ fixed-link {
+ speed = <1000>;
+ full-duplex;
+ };
+ };
+ ethernet@2 {
+ phy-handle = <&phy8>;
+ };
+ };
+ };
+
+ twsi0: i2c@1180000001000 {
+ rtc@68 {
+ compatible = "dallas,ds1337";
+ reg = <0x68>;
+ };
+ };
+
+ uart0: serial@1180000000800 {
+ clock-frequency = <500000000>;
+ };
+
+ usbn: usbn@1180068000000 {
+ refclk-frequency = <12000000>;
+ refclk-type = "crystal";
+ };
+ };
+
+ leds {
+ compatible = "gpio-leds";
+
+ usb1 {
+ label = "usb1";
+ gpios = <&gpio 9 1>; /* Active low */
+ };
+
+ usb2 {
+ label = "usb2";
+ gpios = <&gpio 10 1>; /* Active low */
+ };
+ };
+
+ aliases {
+ pip = &pip;
+ };
+};
diff --git a/arch/mips/boot/dts/cavium-octeon/octeon_3xxx.dts b/arch/mips/boot/dts/cavium-octeon/octeon_3xxx.dts
index 9c48e05..de61f02 100644
--- a/arch/mips/boot/dts/cavium-octeon/octeon_3xxx.dts
+++ b/arch/mips/boot/dts/cavium-octeon/octeon_3xxx.dts
@@ -1,4 +1,3 @@
-/dts-v1/;
/*
* OCTEON 3XXX, 5XXX, 63XX device tree skeleton.
*
@@ -6,56 +5,12 @@
* use. Because of this, it contains a super-set of the available
* devices and properties.
*/
+
+/include/ "octeon_3xxx.dtsi"
+
/ {
- compatible = "cavium,octeon-3860";
- #address-cells = <2>;
- #size-cells = <2>;
- interrupt-parent = <&ciu>;
-
soc@0 {
- compatible = "simple-bus";
- #address-cells = <2>;
- #size-cells = <2>;
- ranges; /* Direct mapping */
-
- ciu: interrupt-controller@1070000000000 {
- compatible = "cavium,octeon-3860-ciu";
- interrupt-controller;
- /* Interrupts are specified by two parts:
- * 1) Controller register (0 or 1)
- * 2) Bit within the register (0..63)
- */
- #interrupt-cells = <2>;
- reg = <0x10700 0x00000000 0x0 0x7000>;
- };
-
- gpio: gpio-controller@1070000000800 {
- #gpio-cells = <2>;
- compatible = "cavium,octeon-3860-gpio";
- reg = <0x10700 0x00000800 0x0 0x100>;
- gpio-controller;
- /* Interrupts are specified by two parts:
- * 1) GPIO pin number (0..15)
- * 2) Triggering (1 - edge rising
- * 2 - edge falling
- * 4 - level active high
- * 8 - level active low)
- */
- interrupt-controller;
- #interrupt-cells = <2>;
- /* The GPIO pin connect to 16 consecutive CUI bits */
- interrupts = <0 16>, <0 17>, <0 18>, <0 19>,
- <0 20>, <0 21>, <0 22>, <0 23>,
- <0 24>, <0 25>, <0 26>, <0 27>,
- <0 28>, <0 29>, <0 30>, <0 31>;
- };
-
smi0: mdio@1180000001800 {
- compatible = "cavium,octeon-3860-mdio";
- #address-cells = <1>;
- #size-cells = <0>;
- reg = <0x11800 0x00001800 0x0 0x40>;
-
phy0: ethernet-phy@0 {
compatible = "marvell,88e1118";
marvell,reg-init =
@@ -220,35 +175,16 @@
};
pip: pip@11800a0000000 {
- compatible = "cavium,octeon-3860-pip";
- #address-cells = <1>;
- #size-cells = <0>;
- reg = <0x11800 0xa0000000 0x0 0x2000>;
-
interface@0 {
- compatible = "cavium,octeon-3860-pip-interface";
- #address-cells = <1>;
- #size-cells = <0>;
- reg = <0>; /* interface */
-
ethernet@0 {
- compatible = "cavium,octeon-3860-pip-port";
- reg = <0x0>; /* Port */
- local-mac-address = [ 00 00 00 00 00 00 ];
phy-handle = <&phy2>;
cavium,alt-phy-handle = <&phy100>;
};
ethernet@1 {
- compatible = "cavium,octeon-3860-pip-port";
- reg = <0x1>; /* Port */
- local-mac-address = [ 00 00 00 00 00 00 ];
phy-handle = <&phy3>;
cavium,alt-phy-handle = <&phy101>;
};
ethernet@2 {
- compatible = "cavium,octeon-3860-pip-port";
- reg = <0x2>; /* Port */
- local-mac-address = [ 00 00 00 00 00 00 ];
phy-handle = <&phy4>;
cavium,alt-phy-handle = <&phy102>;
};
@@ -322,11 +258,6 @@
};
interface@1 {
- compatible = "cavium,octeon-3860-pip-interface";
- #address-cells = <1>;
- #size-cells = <0>;
- reg = <1>; /* interface */
-
ethernet@0 {
compatible = "cavium,octeon-3860-pip-port";
reg = <0x0>; /* Port */
@@ -355,13 +286,6 @@
};
twsi0: i2c@1180000001000 {
- #address-cells = <1>;
- #size-cells = <0>;
- compatible = "cavium,octeon-3860-twsi";
- reg = <0x11800 0x00001000 0x0 0x200>;
- interrupts = <0 45>;
- clock-frequency = <100000>;
-
rtc@68 {
compatible = "dallas,ds1337";
reg = <0x68>;
@@ -381,15 +305,6 @@
clock-frequency = <100000>;
};
- uart0: serial@1180000000800 {
- compatible = "cavium,octeon-3860-uart","ns16550";
- reg = <0x11800 0x00000800 0x0 0x400>;
- clock-frequency = <0>;
- current-speed = <115200>;
- reg-shift = <3>;
- interrupts = <0 34>;
- };
-
uart1: serial@1180000000c00 {
compatible = "cavium,octeon-3860-uart","ns16550";
reg = <0x11800 0x00000c00 0x0 0x400>;
@@ -409,98 +324,6 @@
};
bootbus: bootbus@1180000000000 {
- compatible = "cavium,octeon-3860-bootbus";
- reg = <0x11800 0x00000000 0x0 0x200>;
- /* The chip select number and offset */
- #address-cells = <2>;
- /* The size of the chip select region */
- #size-cells = <1>;
- ranges = <0 0 0x0 0x1f400000 0xc00000>,
- <1 0 0x10000 0x30000000 0>,
- <2 0 0x10000 0x40000000 0>,
- <3 0 0x10000 0x50000000 0>,
- <4 0 0x0 0x1d020000 0x10000>,
- <5 0 0x0 0x1d040000 0x10000>,
- <6 0 0x0 0x1d050000 0x10000>,
- <7 0 0x10000 0x90000000 0>;
-
- cavium,cs-config@0 {
- compatible = "cavium,octeon-3860-bootbus-config";
- cavium,cs-index = <0>;
- cavium,t-adr = <20>;
- cavium,t-ce = <60>;
- cavium,t-oe = <60>;
- cavium,t-we = <45>;
- cavium,t-rd-hld = <35>;
- cavium,t-wr-hld = <45>;
- cavium,t-pause = <0>;
- cavium,t-wait = <0>;
- cavium,t-page = <35>;
- cavium,t-rd-dly = <0>;
-
- cavium,pages = <0>;
- cavium,bus-width = <8>;
- };
- cavium,cs-config@4 {
- compatible = "cavium,octeon-3860-bootbus-config";
- cavium,cs-index = <4>;
- cavium,t-adr = <320>;
- cavium,t-ce = <320>;
- cavium,t-oe = <320>;
- cavium,t-we = <320>;
- cavium,t-rd-hld = <320>;
- cavium,t-wr-hld = <320>;
- cavium,t-pause = <320>;
- cavium,t-wait = <320>;
- cavium,t-page = <320>;
- cavium,t-rd-dly = <0>;
-
- cavium,pages = <0>;
- cavium,bus-width = <8>;
- };
- cavium,cs-config@5 {
- compatible = "cavium,octeon-3860-bootbus-config";
- cavium,cs-index = <5>;
- cavium,t-adr = <5>;
- cavium,t-ce = <300>;
- cavium,t-oe = <125>;
- cavium,t-we = <150>;
- cavium,t-rd-hld = <100>;
- cavium,t-wr-hld = <30>;
- cavium,t-pause = <0>;
- cavium,t-wait = <30>;
- cavium,t-page = <320>;
- cavium,t-rd-dly = <0>;
-
- cavium,pages = <0>;
- cavium,bus-width = <16>;
- };
- cavium,cs-config@6 {
- compatible = "cavium,octeon-3860-bootbus-config";
- cavium,cs-index = <6>;
- cavium,t-adr = <5>;
- cavium,t-ce = <300>;
- cavium,t-oe = <270>;
- cavium,t-we = <150>;
- cavium,t-rd-hld = <100>;
- cavium,t-wr-hld = <70>;
- cavium,t-pause = <0>;
- cavium,t-wait = <0>;
- cavium,t-page = <320>;
- cavium,t-rd-dly = <0>;
-
- cavium,pages = <0>;
- cavium,wait-mode;
- cavium,bus-width = <16>;
- };
-
- flash0: nor@0,0 {
- compatible = "cfi-flash";
- reg = <0 0 0x800000>;
- #address-cells = <1>;
- #size-cells = <1>;
- };
-
led0: led-display@4,0 {
compatible = "avago,hdsp-253x";
reg = <4 0x20 0x20>, <4 0 0x20>;
@@ -515,17 +338,6 @@
};
};
- dma0: dma-engine@1180000000100 {
- compatible = "cavium,octeon-5750-bootbus-dma";
- reg = <0x11800 0x00000100 0x0 0x8>;
- interrupts = <0 63>;
- };
- dma1: dma-engine@1180000000108 {
- compatible = "cavium,octeon-5750-bootbus-dma";
- reg = <0x11800 0x00000108 0x0 0x8>;
- interrupts = <0 63>;
- };
-
uctl: uctl@118006f000000 {
compatible = "cavium,octeon-6335-uctl";
reg = <0x11800 0x6f000000 0x0 0x100>;
@@ -552,21 +364,10 @@
};
usbn: usbn@1180068000000 {
- compatible = "cavium,octeon-5750-usbn";
- reg = <0x11800 0x68000000 0x0 0x1000>;
- ranges; /* Direct mapping */
- #address-cells = <2>;
- #size-cells = <2>;
/* 12MHz, 24MHz and 48MHz allowed */
refclk-frequency = <12000000>;
/* Either "crystal" or "external" */
refclk-type = "crystal";
-
- usbc@16f0010000000 {
- compatible = "cavium,octeon-5750-usbc";
- reg = <0x16f00 0x10000000 0x0 0x80000>;
- interrupts = <0 56>;
- };
};
};
diff --git a/arch/mips/boot/dts/cavium-octeon/octeon_3xxx.dtsi b/arch/mips/boot/dts/cavium-octeon/octeon_3xxx.dtsi
new file mode 100644
index 0000000..5302148
--- /dev/null
+++ b/arch/mips/boot/dts/cavium-octeon/octeon_3xxx.dtsi
@@ -0,0 +1,231 @@
+/* OCTEON 3XXX DTS common parts. */
+
+/dts-v1/;
+
+/ {
+ compatible = "cavium,octeon-3860";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ interrupt-parent = <&ciu>;
+
+ soc@0 {
+ compatible = "simple-bus";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges; /* Direct mapping */
+
+ ciu: interrupt-controller@1070000000000 {
+ compatible = "cavium,octeon-3860-ciu";
+ interrupt-controller;
+ /* Interrupts are specified by two parts:
+ * 1) Controller register (0 or 1)
+ * 2) Bit within the register (0..63)
+ */
+ #interrupt-cells = <2>;
+ reg = <0x10700 0x00000000 0x0 0x7000>;
+ };
+
+ gpio: gpio-controller@1070000000800 {
+ #gpio-cells = <2>;
+ compatible = "cavium,octeon-3860-gpio";
+ reg = <0x10700 0x00000800 0x0 0x100>;
+ gpio-controller;
+ /* Interrupts are specified by two parts:
+ * 1) GPIO pin number (0..15)
+ * 2) Triggering (1 - edge rising
+ * 2 - edge falling
+ * 4 - level active high
+ * 8 - level active low)
+ */
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ /* The GPIO pin connect to 16 consecutive CUI bits */
+ interrupts = <0 16>, <0 17>, <0 18>, <0 19>,
+ <0 20>, <0 21>, <0 22>, <0 23>,
+ <0 24>, <0 25>, <0 26>, <0 27>,
+ <0 28>, <0 29>, <0 30>, <0 31>;
+ };
+
+ smi0: mdio@1180000001800 {
+ compatible = "cavium,octeon-3860-mdio";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x11800 0x00001800 0x0 0x40>;
+ };
+
+ pip: pip@11800a0000000 {
+ compatible = "cavium,octeon-3860-pip";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0x11800 0xa0000000 0x0 0x2000>;
+
+ interface@0 {
+ compatible = "cavium,octeon-3860-pip-interface";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <0>; /* interface */
+
+ ethernet@0 {
+ compatible = "cavium,octeon-3860-pip-port";
+ reg = <0x0>; /* Port */
+ local-mac-address = [ 00 00 00 00 00 00 ];
+ };
+ ethernet@1 {
+ compatible = "cavium,octeon-3860-pip-port";
+ reg = <0x1>; /* Port */
+ local-mac-address = [ 00 00 00 00 00 00 ];
+ };
+ ethernet@2 {
+ compatible = "cavium,octeon-3860-pip-port";
+ reg = <0x2>; /* Port */
+ local-mac-address = [ 00 00 00 00 00 00 ];
+ };
+ };
+
+ interface@1 {
+ compatible = "cavium,octeon-3860-pip-interface";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ reg = <1>; /* interface */
+ };
+ };
+
+ twsi0: i2c@1180000001000 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ compatible = "cavium,octeon-3860-twsi";
+ reg = <0x11800 0x00001000 0x0 0x200>;
+ interrupts = <0 45>;
+ clock-frequency = <100000>;
+ };
+
+ uart0: serial@1180000000800 {
+ compatible = "cavium,octeon-3860-uart","ns16550";
+ reg = <0x11800 0x00000800 0x0 0x400>;
+ clock-frequency = <0>;
+ current-speed = <115200>;
+ reg-shift = <3>;
+ interrupts = <0 34>;
+ };
+
+ bootbus: bootbus@1180000000000 {
+ compatible = "cavium,octeon-3860-bootbus";
+ reg = <0x11800 0x00000000 0x0 0x200>;
+ /* The chip select number and offset */
+ #address-cells = <2>;
+ /* The size of the chip select region */
+ #size-cells = <1>;
+ ranges = <0 0 0x0 0x1f400000 0xc00000>,
+ <1 0 0x10000 0x30000000 0>,
+ <2 0 0x10000 0x40000000 0>,
+ <3 0 0x10000 0x50000000 0>,
+ <4 0 0x0 0x1d020000 0x10000>,
+ <5 0 0x0 0x1d040000 0x10000>,
+ <6 0 0x0 0x1d050000 0x10000>,
+ <7 0 0x10000 0x90000000 0>;
+
+ cavium,cs-config@0 {
+ compatible = "cavium,octeon-3860-bootbus-config";
+ cavium,cs-index = <0>;
+ cavium,t-adr = <20>;
+ cavium,t-ce = <60>;
+ cavium,t-oe = <60>;
+ cavium,t-we = <45>;
+ cavium,t-rd-hld = <35>;
+ cavium,t-wr-hld = <45>;
+ cavium,t-pause = <0>;
+ cavium,t-wait = <0>;
+ cavium,t-page = <35>;
+ cavium,t-rd-dly = <0>;
+
+ cavium,pages = <0>;
+ cavium,bus-width = <8>;
+ };
+ cavium,cs-config@4 {
+ compatible = "cavium,octeon-3860-bootbus-config";
+ cavium,cs-index = <4>;
+ cavium,t-adr = <320>;
+ cavium,t-ce = <320>;
+ cavium,t-oe = <320>;
+ cavium,t-we = <320>;
+ cavium,t-rd-hld = <320>;
+ cavium,t-wr-hld = <320>;
+ cavium,t-pause = <320>;
+ cavium,t-wait = <320>;
+ cavium,t-page = <320>;
+ cavium,t-rd-dly = <0>;
+
+ cavium,pages = <0>;
+ cavium,bus-width = <8>;
+ };
+ cavium,cs-config@5 {
+ compatible = "cavium,octeon-3860-bootbus-config";
+ cavium,cs-index = <5>;
+ cavium,t-adr = <5>;
+ cavium,t-ce = <300>;
+ cavium,t-oe = <125>;
+ cavium,t-we = <150>;
+ cavium,t-rd-hld = <100>;
+ cavium,t-wr-hld = <30>;
+ cavium,t-pause = <0>;
+ cavium,t-wait = <30>;
+ cavium,t-page = <320>;
+ cavium,t-rd-dly = <0>;
+
+ cavium,pages = <0>;
+ cavium,bus-width = <16>;
+ };
+ cavium,cs-config@6 {
+ compatible = "cavium,octeon-3860-bootbus-config";
+ cavium,cs-index = <6>;
+ cavium,t-adr = <5>;
+ cavium,t-ce = <300>;
+ cavium,t-oe = <270>;
+ cavium,t-we = <150>;
+ cavium,t-rd-hld = <100>;
+ cavium,t-wr-hld = <70>;
+ cavium,t-pause = <0>;
+ cavium,t-wait = <0>;
+ cavium,t-page = <320>;
+ cavium,t-rd-dly = <0>;
+
+ cavium,pages = <0>;
+ cavium,wait-mode;
+ cavium,bus-width = <16>;
+ };
+
+ flash0: nor@0,0 {
+ compatible = "cfi-flash";
+ reg = <0 0 0x800000>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ };
+ };
+
+ dma0: dma-engine@1180000000100 {
+ compatible = "cavium,octeon-5750-bootbus-dma";
+ reg = <0x11800 0x00000100 0x0 0x8>;
+ interrupts = <0 63>;
+ };
+
+ dma1: dma-engine@1180000000108 {
+ compatible = "cavium,octeon-5750-bootbus-dma";
+ reg = <0x11800 0x00000108 0x0 0x8>;
+ interrupts = <0 63>;
+ };
+
+ usbn: usbn@1180068000000 {
+ compatible = "cavium,octeon-5750-usbn";
+ reg = <0x11800 0x68000000 0x0 0x1000>;
+ ranges; /* Direct mapping */
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ usbc@16f0010000000 {
+ compatible = "cavium,octeon-5750-usbc";
+ reg = <0x16f00 0x10000000 0x0 0x80000>;
+ interrupts = <0 56>;
+ };
+ };
+ };
+};
diff --git a/arch/mips/boot/dts/cavium-octeon/ubnt_e100.dts b/arch/mips/boot/dts/cavium-octeon/ubnt_e100.dts
new file mode 100644
index 0000000..243e5dc
--- /dev/null
+++ b/arch/mips/boot/dts/cavium-octeon/ubnt_e100.dts
@@ -0,0 +1,59 @@
+/*
+ * Device tree source for EdgeRouter Lite.
+ *
+ * Written by: Aaro Koskinen <aaro.koskinen@iki.fi>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+/include/ "octeon_3xxx.dtsi"
+
+/ {
+ model = "ubnt,e100";
+
+ soc@0 {
+ smi0: mdio@1180000001800 {
+ phy5: ethernet-phy@5 {
+ reg = <5>;
+ compatible = "ethernet-phy-ieee802.3-c22";
+ };
+ phy6: ethernet-phy@6 {
+ reg = <6>;
+ compatible = "ethernet-phy-ieee802.3-c22";
+ };
+ phy7: ethernet-phy@7 {
+ reg = <7>;
+ compatible = "ethernet-phy-ieee802.3-c22";
+ };
+ };
+
+ pip: pip@11800a0000000 {
+ interface@0 {
+ ethernet@0 {
+ phy-handle = <&phy7>;
+ };
+ ethernet@1 {
+ phy-handle = <&phy6>;
+ };
+ ethernet@2 {
+ phy-handle = <&phy5>;
+ };
+ };
+ };
+
+ uart0: serial@1180000000800 {
+ clock-frequency = <500000000>;
+ };
+
+ usbn: usbn@1180068000000 {
+ refclk-frequency = <12000000>;
+ refclk-type = "crystal";
+ };
+ };
+
+ aliases {
+ pip = &pip;
+ };
+};
diff --git a/arch/mips/boot/dts/ingenic/jz4740.dtsi b/arch/mips/boot/dts/ingenic/jz4740.dtsi
index 8b2437c..4a9c8f2 100644
--- a/arch/mips/boot/dts/ingenic/jz4740.dtsi
+++ b/arch/mips/boot/dts/ingenic/jz4740.dtsi
@@ -65,4 +65,18 @@
clocks = <&ext>, <&cgu JZ4740_CLK_UART1>;
clock-names = "baud", "module";
};
+
+ uhc: uhc@13030000 {
+ compatible = "ingenic,jz4740-ohci", "generic-ohci";
+ reg = <0x13030000 0x1000>;
+
+ clocks = <&cgu JZ4740_CLK_UHC>;
+ assigned-clocks = <&cgu JZ4740_CLK_UHC>;
+ assigned-clock-rates = <48000000>;
+
+ interrupt-parent = <&intc>;
+ interrupts = <3>;
+
+ status = "disabled";
+ };
};
diff --git a/arch/mips/boot/dts/lantiq/easy50712.dts b/arch/mips/boot/dts/lantiq/easy50712.dts
index 143b8a3..b599625 100644
--- a/arch/mips/boot/dts/lantiq/easy50712.dts
+++ b/arch/mips/boot/dts/lantiq/easy50712.dts
@@ -52,7 +52,7 @@
};
gpio: pinmux@E100B10 {
- compatible = "lantiq,pinctrl-xway";
+ compatible = "lantiq,danube-pinctrl";
pinctrl-names = "default";
pinctrl-0 = <&state_default>;
diff --git a/arch/mips/boot/dts/pic32/pic32mzda-clk.dtsi b/arch/mips/boot/dts/pic32/pic32mzda-clk.dtsi
deleted file mode 100644
index ef13350..0000000
--- a/arch/mips/boot/dts/pic32/pic32mzda-clk.dtsi
+++ /dev/null
@@ -1,236 +0,0 @@
-/*
- * Device Tree Source for PIC32MZDA clock data
- *
- * Purna Chandra Mandal <purna.mandal@microchip.com>
- * Copyright (C) 2015 Microchip Technology Inc. All rights reserved.
- *
- * Licensed under GPLv2 or later.
- */
-
-/* all fixed rate clocks */
-
-/ {
- POSC:posc_clk { /* On-chip primary oscillator */
- #clock-cells = <0>;
- compatible = "fixed-clock";
- clock-frequency = <24000000>;
- };
-
- FRC:frc_clk { /* internal FRC oscillator */
- #clock-cells = <0>;
- compatible = "fixed-clock";
- clock-frequency = <8000000>;
- };
-
- BFRC:bfrc_clk { /* internal backup FRC oscillator */
- #clock-cells = <0>;
- compatible = "fixed-clock";
- clock-frequency = <8000000>;
- };
-
- LPRC:lprc_clk { /* internal low-power FRC oscillator */
- #clock-cells = <0>;
- compatible = "fixed-clock";
- clock-frequency = <32000>;
- };
-
- /* UPLL provides clock to USBCORE */
- UPLL:usb_phy_clk {
- #clock-cells = <0>;
- compatible = "fixed-clock";
- clock-frequency = <24000000>;
- clock-output-names = "usbphy_clk";
- };
-
- TxCKI:txcki_clk { /* external clock input on TxCLKI pin */
- #clock-cells = <0>;
- compatible = "fixed-clock";
- clock-frequency = <4000000>;
- status = "disabled";
- };
-
- /* external clock input on REFCLKIx pin */
- REFIx:refix_clk {
- #clock-cells = <0>;
- compatible = "fixed-clock";
- clock-frequency = <24000000>;
- status = "disabled";
- };
-
- /* PIC32 specific clks */
- pic32_clktree {
- #address-cells = <1>;
- #size-cells = <1>;
- reg = <0x1f801200 0x200>;
- compatible = "microchip,pic32mzda-clk";
- ranges = <0 0x1f801200 0x200>;
-
- /* secondary oscillator; external input on SOSCI pin */
- SOSC:sosc_clk@0 {
- #clock-cells = <0>;
- compatible = "microchip,pic32mzda-sosc";
- clock-frequency = <32768>;
- reg = <0x000 0x10>, /* enable reg */
- <0x1d0 0x10>; /* status reg */
- microchip,bit-mask = <0x02>; /* enable mask */
- microchip,status-bit-mask = <0x10>; /* status-mask*/
- };
-
- FRCDIV:frcdiv_clk {
- #clock-cells = <0>;
- compatible = "microchip,pic32mzda-frcdivclk";
- clocks = <&FRC>;
- clock-output-names = "frcdiv_clk";
- };
-
- /* System PLL clock */
- SYSPLL:spll_clk@020 {
- #clock-cells = <0>;
- compatible = "microchip,pic32mzda-syspll";
- reg = <0x020 0x10>, /* SPLL register */
- <0x1d0 0x10>; /* CLKSTAT register */
- clocks = <&POSC>, <&FRC>;
- clock-output-names = "sys_pll";
- microchip,status-bit-mask = <0x80>; /* SPLLRDY */
- };
-
- /* system clock; mux with postdiv & slew */
- SYSCLK:sys_clk@1c0 {
- #clock-cells = <0>;
- compatible = "microchip,pic32mzda-sysclk-v2";
- reg = <0x1c0 0x04>; /* SLEWCON */
- clocks = <&FRCDIV>, <&SYSPLL>, <&POSC>, <&SOSC>,
- <&LPRC>, <&FRCDIV>;
- microchip,clock-indices = <0>, <1>, <2>, <4>,
- <5>, <7>;
- clock-output-names = "sys_clk";
- };
-
- /* Peripheral bus1 clock */
- PBCLK1:pb1_clk@140 {
- reg = <0x140 0x10>;
- #clock-cells = <0>;
- compatible = "microchip,pic32mzda-pbclk";
- clocks = <&SYSCLK>;
- clock-output-names = "pb1_clk";
- /* used by system modules, not gateable */
- microchip,ignore-unused;
- };
-
- /* Peripheral bus2 clock */
- PBCLK2:pb2_clk@150 {
- reg = <0x150 0x10>;
- #clock-cells = <0>;
- compatible = "microchip,pic32mzda-pbclk";
- clocks = <&SYSCLK>;
- clock-output-names = "pb2_clk";
- /* avoid gating even if unused */
- microchip,ignore-unused;
- };
-
- /* Peripheral bus3 clock */
- PBCLK3:pb3_clk@160 {
- reg = <0x160 0x10>;
- #clock-cells = <0>;
- compatible = "microchip,pic32mzda-pbclk";
- clocks = <&SYSCLK>;
- clock-output-names = "pb3_clk";
- };
-
- /* Peripheral bus4 clock(I/O ports, GPIO) */
- PBCLK4:pb4_clk@170 {
- reg = <0x170 0x10>;
- #clock-cells = <0>;
- compatible = "microchip,pic32mzda-pbclk";
- clocks = <&SYSCLK>;
- clock-output-names = "pb4_clk";
- };
-
- /* Peripheral bus clock */
- PBCLK5:pb5_clk@180 {
- reg = <0x180 0x10>;
- #clock-cells = <0>;
- compatible = "microchip,pic32mzda-pbclk";
- clocks = <&SYSCLK>;
- clock-output-names = "pb5_clk";
- };
-
- /* Peripheral Bus6 clock; */
- PBCLK6:pb6_clk@190 {
- reg = <0x190 0x10>;
- compatible = "microchip,pic32mzda-pbclk";
- clocks = <&SYSCLK>;
- #clock-cells = <0>;
- };
-
- /* Peripheral bus7 clock */
- PBCLK7:pb7_clk@1a0 {
- reg = <0x1a0 0x10>;
- #clock-cells = <0>;
- compatible = "microchip,pic32mzda-pbclk";
- /* CPU is driven by this clock; so named */
- clock-output-names = "cpu_clk";
- clocks = <&SYSCLK>;
- };
-
- /* Reference Oscillator clock for SPI/I2S */
- REFCLKO1:refo1_clk@80 {
- reg = <0x080 0x20>;
- #clock-cells = <0>;
- compatible = "microchip,pic32mzda-refoclk";
- clocks = <&SYSCLK>, <&PBCLK1>, <&POSC>, <&FRC>, <&LPRC>,
- <&SOSC>, <&SYSPLL>, <&REFIx>, <&BFRC>;
- microchip,clock-indices = <0>, <1>, <2>, <3>, <4>,
- <5>, <7>, <8>, <9>;
- clock-output-names = "refo1_clk";
- };
-
- /* Reference Oscillator clock for SQI */
- REFCLKO2:refo2_clk@a0 {
- reg = <0x0a0 0x20>;
- #clock-cells = <0>;
- compatible = "microchip,pic32mzda-refoclk";
- clocks = <&SYSCLK>, <&PBCLK1>, <&POSC>, <&FRC>, <&LPRC>,
- <&SOSC>, <&SYSPLL>, <&REFIx>, <&BFRC>;
- microchip,clock-indices = <0>, <1>, <2>, <3>, <4>,
- <5>, <7>, <8>, <9>;
- clock-output-names = "refo2_clk";
- };
-
- /* Reference Oscillator clock, ADC */
- REFCLKO3:refo3_clk@c0 {
- reg = <0x0c0 0x20>;
- compatible = "microchip,pic32mzda-refoclk";
- clocks = <&SYSCLK>, <&PBCLK1>, <&POSC>, <&FRC>, <&LPRC>,
- <&SOSC>, <&SYSPLL>, <&REFIx>, <&BFRC>;
- microchip,clock-indices = <0>, <1>, <2>, <3>, <4>,
- <5>, <7>, <8>, <9>;
- #clock-cells = <0>;
- clock-output-names = "refo3_clk";
- };
-
- /* Reference Oscillator clock */
- REFCLKO4:refo4_clk@e0 {
- reg = <0x0e0 0x20>;
- compatible = "microchip,pic32mzda-refoclk";
- clocks = <&SYSCLK>, <&PBCLK1>, <&POSC>, <&FRC>, <&LPRC>,
- <&SOSC>, <&SYSPLL>, <&REFIx>, <&BFRC>;
- microchip,clock-indices = <0>, <1>, <2>, <3>, <4>,
- <5>, <7>, <8>, <9>;
- #clock-cells = <0>;
- clock-output-names = "refo4_clk";
- };
-
- /* Reference Oscillator clock, LCD */
- REFCLKO5:refo5_clk@100 {
- reg = <0x100 0x20>;
- compatible = "microchip,pic32mzda-refoclk";
- clocks = <&SYSCLK>,<&PBCLK1>,<&POSC>,<&FRC>,<&LPRC>,
- <&SOSC>,<&SYSPLL>,<&REFIx>,<&BFRC>;
- microchip,clock-indices = <0>, <1>, <2>, <3>, <4>,
- <5>, <7>, <8>, <9>;
- #clock-cells = <0>;
- clock-output-names = "refo5_clk";
- };
- };
-};
diff --git a/arch/mips/boot/dts/pic32/pic32mzda.dtsi b/arch/mips/boot/dts/pic32/pic32mzda.dtsi
index ad9e3318..5353a63 100644
--- a/arch/mips/boot/dts/pic32/pic32mzda.dtsi
+++ b/arch/mips/boot/dts/pic32/pic32mzda.dtsi
@@ -6,11 +6,9 @@
* published by the Free Software Foundation.
*
*/
-
+#include <dt-bindings/clock/microchip,pic32-clock.h>
#include <dt-bindings/interrupt-controller/irq.h>
-#include "pic32mzda-clk.dtsi"
-
/ {
#address-cells = <1>;
#size-cells = <1>;
@@ -50,6 +48,29 @@
interrupts = <0 IRQ_TYPE_EDGE_RISING>;
};
+ /* external clock input on TxCLKI pin */
+ txcki: txcki_clk {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <4000000>;
+ status = "disabled";
+ };
+
+ /* external input on REFCLKIx pin */
+ refix: refix_clk {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <24000000>;
+ status = "disabled";
+ };
+
+ rootclk: clock-controller@1f801200 {
+ compatible = "microchip,pic32mzda-clk";
+ reg = <0x1f801200 0x200>;
+ #clock-cells = <1>;
+ microchip,pic32mzda-sosc;
+ };
+
evic: interrupt-controller@1f810000 {
compatible = "microchip,pic32mzda-evic";
interrupt-controller;
@@ -63,7 +84,7 @@
#size-cells = <1>;
compatible = "microchip,pic32mzda-pinctrl";
reg = <0x1f801400 0x400>;
- clocks = <&PBCLK1>;
+ clocks = <&rootclk PB1CLK>;
};
/* PORTA */
@@ -75,7 +96,7 @@
gpio-controller;
interrupt-controller;
#interrupt-cells = <2>;
- clocks = <&PBCLK4>;
+ clocks = <&rootclk PB4CLK>;
microchip,gpio-bank = <0>;
gpio-ranges = <&pic32_pinctrl 0 0 16>;
};
@@ -89,7 +110,7 @@
gpio-controller;
interrupt-controller;
#interrupt-cells = <2>;
- clocks = <&PBCLK4>;
+ clocks = <&rootclk PB4CLK>;
microchip,gpio-bank = <1>;
gpio-ranges = <&pic32_pinctrl 0 16 16>;
};
@@ -103,7 +124,7 @@
gpio-controller;
interrupt-controller;
#interrupt-cells = <2>;
- clocks = <&PBCLK4>;
+ clocks = <&rootclk PB4CLK>;
microchip,gpio-bank = <2>;
gpio-ranges = <&pic32_pinctrl 0 32 16>;
};
@@ -117,7 +138,7 @@
gpio-controller;
interrupt-controller;
#interrupt-cells = <2>;
- clocks = <&PBCLK4>;
+ clocks = <&rootclk PB4CLK>;
microchip,gpio-bank = <3>;
gpio-ranges = <&pic32_pinctrl 0 48 16>;
};
@@ -131,7 +152,7 @@
gpio-controller;
interrupt-controller;
#interrupt-cells = <2>;
- clocks = <&PBCLK4>;
+ clocks = <&rootclk PB4CLK>;
microchip,gpio-bank = <4>;
gpio-ranges = <&pic32_pinctrl 0 64 16>;
};
@@ -145,7 +166,7 @@
gpio-controller;
interrupt-controller;
#interrupt-cells = <2>;
- clocks = <&PBCLK4>;
+ clocks = <&rootclk PB4CLK>;
microchip,gpio-bank = <5>;
gpio-ranges = <&pic32_pinctrl 0 80 16>;
};
@@ -159,7 +180,7 @@
gpio-controller;
interrupt-controller;
#interrupt-cells = <2>;
- clocks = <&PBCLK4>;
+ clocks = <&rootclk PB4CLK>;
microchip,gpio-bank = <6>;
gpio-ranges = <&pic32_pinctrl 0 96 16>;
};
@@ -173,7 +194,7 @@
gpio-controller;
interrupt-controller;
#interrupt-cells = <2>;
- clocks = <&PBCLK4>;
+ clocks = <&rootclk PB4CLK>;
microchip,gpio-bank = <7>;
gpio-ranges = <&pic32_pinctrl 0 112 16>;
};
@@ -189,7 +210,7 @@
gpio-controller;
interrupt-controller;
#interrupt-cells = <2>;
- clocks = <&PBCLK4>;
+ clocks = <&rootclk PB4CLK>;
microchip,gpio-bank = <8>;
gpio-ranges = <&pic32_pinctrl 0 128 16>;
};
@@ -203,7 +224,7 @@
gpio-controller;
interrupt-controller;
#interrupt-cells = <2>;
- clocks = <&PBCLK4>;
+ clocks = <&rootclk PB4CLK>;
microchip,gpio-bank = <9>;
gpio-ranges = <&pic32_pinctrl 0 144 16>;
};
@@ -212,7 +233,7 @@
compatible = "microchip,pic32mzda-sdhci";
reg = <0x1f8ec000 0x100>;
interrupts = <191 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&REFCLKO4>, <&PBCLK5>;
+ clocks = <&rootclk REF4CLK>, <&rootclk PB5CLK>;
clock-names = "base_clk", "sys_clk";
bus-width = <4>;
cap-sd-highspeed;
@@ -225,7 +246,7 @@
interrupts = <112 IRQ_TYPE_LEVEL_HIGH>,
<113 IRQ_TYPE_LEVEL_HIGH>,
<114 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&PBCLK2>;
+ clocks = <&rootclk PB2CLK>;
status = "disabled";
};
@@ -235,7 +256,7 @@
interrupts = <145 IRQ_TYPE_LEVEL_HIGH>,
<146 IRQ_TYPE_LEVEL_HIGH>,
<147 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&PBCLK2>;
+ clocks = <&rootclk PB2CLK>;
status = "disabled";
};
@@ -245,7 +266,7 @@
interrupts = <157 IRQ_TYPE_LEVEL_HIGH>,
<158 IRQ_TYPE_LEVEL_HIGH>,
<159 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&PBCLK2>;
+ clocks = <&rootclk PB2CLK>;
status = "disabled";
};
@@ -255,7 +276,7 @@
interrupts = <170 IRQ_TYPE_LEVEL_HIGH>,
<171 IRQ_TYPE_LEVEL_HIGH>,
<172 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&PBCLK2>;
+ clocks = <&rootclk PB2CLK>;
status = "disabled";
};
@@ -265,7 +286,7 @@
interrupts = <179 IRQ_TYPE_LEVEL_HIGH>,
<180 IRQ_TYPE_LEVEL_HIGH>,
<181 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&PBCLK2>;
+ clocks = <&rootclk PB2CLK>;
status = "disabled";
};
@@ -275,7 +296,7 @@
interrupts = <188 IRQ_TYPE_LEVEL_HIGH>,
<189 IRQ_TYPE_LEVEL_HIGH>,
<190 IRQ_TYPE_LEVEL_HIGH>;
- clocks = <&PBCLK2>;
+ clocks = <&rootclk PB2CLK>;
status = "disabled";
};
};
diff --git a/arch/mips/boot/dts/pic32/pic32mzda_sk.dts b/arch/mips/boot/dts/pic32/pic32mzda_sk.dts
index 5d434a5..fc74010 100644
--- a/arch/mips/boot/dts/pic32/pic32mzda_sk.dts
+++ b/arch/mips/boot/dts/pic32/pic32mzda_sk.dts
@@ -95,8 +95,9 @@
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_sdhc1>;
status = "okay";
- assigned-clocks = <&REFCLKO2>,<&REFCLKO4>,<&REFCLKO5>;
- assigned-clock-rates = <50000000>,<25000000>,<40000000>;
+ assigned-clocks = <&rootclk REF2CLK>, <&rootclk REF4CLK>,
+ <&rootclk REF5CLK>;
+ assigned-clock-rates = <50000000>, <25000000>, <40000000>;
};
&pic32_pinctrl {
diff --git a/arch/mips/boot/dts/qca/Makefile b/arch/mips/boot/dts/qca/Makefile
index 2d61455d..63a9ddf 100644
--- a/arch/mips/boot/dts/qca/Makefile
+++ b/arch/mips/boot/dts/qca/Makefile
@@ -1,8 +1,9 @@
# All DTBs
dtb-$(CONFIG_ATH79) += ar9132_tl_wr1043nd_v1.dtb
-
-# Select a DTB to build in the kernel
-obj-$(CONFIG_DTB_TL_WR1043ND_V1) += ar9132_tl_wr1043nd_v1.dtb.o
+dtb-$(CONFIG_ATH79) += ar9331_dpt_module.dtb
+dtb-$(CONFIG_ATH79) += ar9331_dragino_ms14.dtb
+dtb-$(CONFIG_ATH79) += ar9331_omega.dtb
+dtb-$(CONFIG_ATH79) += ar9331_tl_mr3020.dtb
# Force kbuild to make empty built-in.o if necessary
obj- += dummy.o
diff --git a/arch/mips/boot/dts/qca/ar9132.dtsi b/arch/mips/boot/dts/qca/ar9132.dtsi
index 3c2ed9e..302f0a8 100644
--- a/arch/mips/boot/dts/qca/ar9132.dtsi
+++ b/arch/mips/boot/dts/qca/ar9132.dtsi
@@ -1,3 +1,5 @@
+#include <dt-bindings/clock/ath79-clk.h>
+
/ {
compatible = "qca,ar9132";
@@ -11,6 +13,7 @@
cpu@0 {
device_type = "cpu";
compatible = "mips,mips24Kc";
+ clocks = <&pll ATH79_CLK_CPU>;
reg = <0>;
};
};
@@ -52,12 +55,12 @@
#qca,ddr-wb-channel-cells = <1>;
};
- uart@18020000 {
+ uart: uart@18020000 {
compatible = "ns8250";
reg = <0x18020000 0x20>;
interrupts = <3>;
- clocks = <&pll 2>;
+ clocks = <&pll ATH79_CLK_AHB>;
clock-names = "uart";
reg-io-width = <4>;
@@ -94,13 +97,13 @@
clock-output-names = "cpu", "ddr", "ahb";
};
- wdt@18060008 {
+ wdt: wdt@18060008 {
compatible = "qca,ar7130-wdt";
reg = <0x18060008 0x8>;
interrupts = <4>;
- clocks = <&pll 2>;
+ clocks = <&pll ATH79_CLK_AHB>;
clock-names = "wdt";
};
@@ -125,7 +128,7 @@
};
};
- usb@1b000100 {
+ usb: usb@1b000100 {
compatible = "qca,ar7100-ehci", "generic-ehci";
reg = <0x1b000100 0x100>;
@@ -140,11 +143,11 @@
status = "disabled";
};
- spi@1f000000 {
+ spi: spi@1f000000 {
compatible = "qca,ar9132-spi", "qca,ar7100-spi";
reg = <0x1f000000 0x10>;
- clocks = <&pll 2>;
+ clocks = <&pll ATH79_CLK_AHB>;
clock-names = "ahb";
status = "disabled";
diff --git a/arch/mips/boot/dts/qca/ar9132_tl_wr1043nd_v1.dts b/arch/mips/boot/dts/qca/ar9132_tl_wr1043nd_v1.dts
index 4f1540e5f..3c3b7ce 100644
--- a/arch/mips/boot/dts/qca/ar9132_tl_wr1043nd_v1.dts
+++ b/arch/mips/boot/dts/qca/ar9132_tl_wr1043nd_v1.dts
@@ -9,10 +9,6 @@
compatible = "tplink,tl-wr1043nd-v1", "qca,ar9132";
model = "TP-Link TL-WR1043ND Version 1";
- alias {
- serial0 = "/ahb/apb/uart@18020000";
- };
-
memory@0 {
device_type = "memory";
reg = <0x0 0x2000000>;
@@ -24,55 +20,6 @@
clock-frequency = <40000000>;
};
- ahb {
- apb {
- uart@18020000 {
- status = "okay";
- };
-
- pll-controller@18050000 {
- clocks = <&extosc>;
- };
- };
-
- usb@1b000100 {
- status = "okay";
- };
-
- spi@1f000000 {
- status = "okay";
- num-cs = <1>;
-
- flash@0 {
- #address-cells = <1>;
- #size-cells = <1>;
- compatible = "s25sl064a";
- reg = <0>;
- spi-max-frequency = <25000000>;
-
- partition@0 {
- label = "u-boot";
- reg = <0x000000 0x020000>;
- };
-
- partition@1 {
- label = "firmware";
- reg = <0x020000 0x7D0000>;
- };
-
- partition@2 {
- label = "art";
- reg = <0x7F0000 0x010000>;
- read-only;
- };
- };
- };
- };
-
- usb-phy {
- status = "okay";
- };
-
gpio-keys {
compatible = "gpio-keys-polled";
#address-cells = <1>;
@@ -118,3 +65,48 @@
};
};
};
+
+&uart {
+ status = "okay";
+};
+
+&pll {
+ clocks = <&extosc>;
+};
+
+&usb {
+ status = "okay";
+};
+
+&usb_phy {
+ status = "okay";
+};
+
+&spi {
+ status = "okay";
+ num-cs = <1>;
+
+ flash@0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "s25sl064a";
+ reg = <0>;
+ spi-max-frequency = <25000000>;
+
+ partition@0 {
+ label = "u-boot";
+ reg = <0x000000 0x020000>;
+ };
+
+ partition@1 {
+ label = "firmware";
+ reg = <0x020000 0x7D0000>;
+ };
+
+ partition@2 {
+ label = "art";
+ reg = <0x7F0000 0x010000>;
+ read-only;
+ };
+ };
+};
diff --git a/arch/mips/boot/dts/qca/ar9331.dtsi b/arch/mips/boot/dts/qca/ar9331.dtsi
new file mode 100644
index 0000000..cf47ed4
--- /dev/null
+++ b/arch/mips/boot/dts/qca/ar9331.dtsi
@@ -0,0 +1,155 @@
+#include <dt-bindings/clock/ath79-clk.h>
+
+/ {
+ compatible = "qca,ar9331";
+
+ #address-cells = <1>;
+ #size-cells = <1>;
+
+ cpus {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ cpu@0 {
+ device_type = "cpu";
+ compatible = "mips,mips24Kc";
+ clocks = <&pll ATH79_CLK_CPU>;
+ reg = <0>;
+ };
+ };
+
+ cpuintc: interrupt-controller {
+ compatible = "qca,ar7100-cpu-intc";
+
+ interrupt-controller;
+ #interrupt-cells = <1>;
+
+ qca,ddr-wb-channel-interrupts = <2>, <3>;
+ qca,ddr-wb-channels = <&ddr_ctrl 3>, <&ddr_ctrl 2>;
+ };
+
+ ref: ref {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ };
+
+ ahb {
+ compatible = "simple-bus";
+ ranges;
+
+ #address-cells = <1>;
+ #size-cells = <1>;
+
+ interrupt-parent = <&cpuintc>;
+
+ apb {
+ compatible = "simple-bus";
+ ranges;
+
+ #address-cells = <1>;
+ #size-cells = <1>;
+
+ interrupt-parent = <&miscintc>;
+
+ ddr_ctrl: memory-controller@18000000 {
+ compatible = "qca,ar7240-ddr-controller";
+ reg = <0x18000000 0x100>;
+
+ #qca,ddr-wb-channel-cells = <1>;
+ };
+
+ uart: uart@18020000 {
+ compatible = "qca,ar9330-uart";
+ reg = <0x18020000 0x14>;
+
+ interrupts = <3>;
+
+ clocks = <&ref>;
+ clock-names = "uart";
+
+ status = "disabled";
+ };
+
+ gpio: gpio@18040000 {
+ compatible = "qca,ar7100-gpio";
+ reg = <0x18040000 0x34>;
+ interrupts = <2>;
+
+ ngpios = <30>;
+
+ gpio-controller;
+ #gpio-cells = <2>;
+
+ interrupt-controller;
+ #interrupt-cells = <2>;
+
+ status = "disabled";
+ };
+
+ pll: pll-controller@18050000 {
+ compatible = "qca,ar9330-pll";
+ reg = <0x18050000 0x100>;
+
+ clocks = <&ref>;
+ clock-names = "ref";
+
+ #clock-cells = <1>;
+ };
+
+ miscintc: interrupt-controller@18060010 {
+ compatible = "qca,ar7240-misc-intc";
+ reg = <0x18060010 0x4>;
+
+ interrupt-parent = <&cpuintc>;
+ interrupts = <6>;
+
+ interrupt-controller;
+ #interrupt-cells = <1>;
+ };
+
+ rst: reset-controller@1806001c {
+ compatible = "qca,ar7100-reset";
+ reg = <0x1806001c 0x4>;
+
+ #reset-cells = <1>;
+ };
+ };
+
+ usb: usb@1b000100 {
+ compatible = "chipidea,usb2";
+ reg = <0x1b000000 0x200>;
+
+ interrupts = <3>;
+ resets = <&rst 5>;
+
+ phy-names = "usb-phy";
+ phys = <&usb_phy>;
+
+ status = "disabled";
+ };
+
+ spi: spi@1f000000 {
+ compatible = "qca,ar7100-spi";
+ reg = <0x1f000000 0x10>;
+
+ clocks = <&pll ATH79_CLK_AHB>;
+ clock-names = "ahb";
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ status = "disabled";
+ };
+ };
+
+ usb_phy: usb-phy {
+ compatible = "qca,ar7100-usb-phy";
+
+ reset-names = "usb-phy", "usb-suspend-override";
+ resets = <&rst 4>, <&rst 3>;
+
+ #phy-cells = <0>;
+
+ status = "disabled";
+ };
+};
diff --git a/arch/mips/boot/dts/qca/ar9331_dpt_module.dts b/arch/mips/boot/dts/qca/ar9331_dpt_module.dts
new file mode 100644
index 0000000..98e7450
--- /dev/null
+++ b/arch/mips/boot/dts/qca/ar9331_dpt_module.dts
@@ -0,0 +1,78 @@
+/dts-v1/;
+
+#include <dt-bindings/gpio/gpio.h>
+#include <dt-bindings/input/input.h>
+
+#include "ar9331.dtsi"
+
+/ {
+ model = "DPTechnics DPT-Module";
+ compatible = "dptechnics,dpt-module";
+
+ aliases {
+ serial0 = &uart;
+ };
+
+ memory@0 {
+ device_type = "memory";
+ reg = <0x0 0x4000000>;
+ };
+
+ leds {
+ compatible = "gpio-leds";
+
+ system {
+ label = "dpt-module:green:system";
+ gpios = <&gpio 27 GPIO_ACTIVE_LOW>;
+ default-state = "off";
+ };
+ };
+
+ gpio-keys-polled {
+ compatible = "gpio-keys-polled";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ poll-interval = <100>;
+
+ button@0 {
+ label = "reset";
+ linux,code = <KEY_RESTART>;
+ gpios = <&gpio 11 GPIO_ACTIVE_LOW>;
+ };
+ };
+};
+
+&ref {
+ clock-frequency = <25000000>;
+};
+
+&uart {
+ status = "okay";
+};
+
+&gpio {
+ status = "okay";
+};
+
+&usb {
+ dr_mode = "host";
+ status = "okay";
+};
+
+&usb_phy {
+ status = "okay";
+};
+
+&spi {
+ num-chipselects = <1>;
+ status = "okay";
+
+ /* Winbond 25Q128FVSG SPI flash */
+ spiflash: w25q128@0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "winbond,w25q128", "jedec,spi-nor";
+ spi-max-frequency = <104000000>;
+ reg = <0>;
+ };
+};
diff --git a/arch/mips/boot/dts/qca/ar9331_dragino_ms14.dts b/arch/mips/boot/dts/qca/ar9331_dragino_ms14.dts
new file mode 100644
index 0000000..56f8320
--- /dev/null
+++ b/arch/mips/boot/dts/qca/ar9331_dragino_ms14.dts
@@ -0,0 +1,102 @@
+/dts-v1/;
+
+#include <dt-bindings/gpio/gpio.h>
+#include <dt-bindings/input/input.h>
+
+#include "ar9331.dtsi"
+
+/ {
+ model = "Dragino MS14 (Dragino 2)";
+ compatible = "dragino,ms14";
+
+ aliases {
+ serial0 = &uart;
+ };
+
+ memory@0 {
+ device_type = "memory";
+ reg = <0x0 0x4000000>;
+ };
+
+ leds {
+ compatible = "gpio-leds";
+
+ wlan {
+ label = "dragino2:red:wlan";
+ gpios = <&gpio 0 GPIO_ACTIVE_HIGH>;
+ default-state = "off";
+ };
+
+ lan {
+ label = "dragino2:red:lan";
+ gpios = <&gpio 13 GPIO_ACTIVE_LOW>;
+ default-state = "off";
+ };
+
+ wan {
+ label = "dragino2:red:wan";
+ gpios = <&gpio 17 GPIO_ACTIVE_LOW>;
+ default-state = "off";
+ };
+
+ system {
+ label = "dragino2:red:system";
+ gpios = <&gpio 28 GPIO_ACTIVE_HIGH>;
+ default-state = "off";
+ };
+ };
+
+ gpio-keys-polled {
+ compatible = "gpio-keys-polled";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ poll-interval = <100>;
+
+ button@0 {
+ label = "jumpstart";
+ linux,code = <KEY_WPS_BUTTON>;
+ gpios = <&gpio 11 GPIO_ACTIVE_LOW>;
+ };
+
+ button@1 {
+ label = "reset";
+ linux,code = <KEY_RESTART>;
+ gpios = <&gpio 12 GPIO_ACTIVE_LOW>;
+ };
+ };
+};
+
+&ref {
+ clock-frequency = <25000000>;
+};
+
+&uart {
+ status = "okay";
+};
+
+&gpio {
+ status = "okay";
+};
+
+&usb {
+ dr_mode = "host";
+ status = "okay";
+};
+
+&usb_phy {
+ status = "okay";
+};
+
+&spi {
+ num-chipselects = <1>;
+ status = "okay";
+
+ /* Winbond 25Q128BVFG SPI flash */
+ spiflash: w25q128@0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "winbond,w25q128", "jedec,spi-nor";
+ spi-max-frequency = <104000000>;
+ reg = <0>;
+ };
+};
diff --git a/arch/mips/boot/dts/qca/ar9331_omega.dts b/arch/mips/boot/dts/qca/ar9331_omega.dts
new file mode 100644
index 0000000..b2be3b0
--- /dev/null
+++ b/arch/mips/boot/dts/qca/ar9331_omega.dts
@@ -0,0 +1,78 @@
+/dts-v1/;
+
+#include <dt-bindings/gpio/gpio.h>
+#include <dt-bindings/input/input.h>
+
+#include "ar9331.dtsi"
+
+/ {
+ model = "Onion Omega";
+ compatible = "onion,omega";
+
+ aliases {
+ serial0 = &uart;
+ };
+
+ memory@0 {
+ device_type = "memory";
+ reg = <0x0 0x4000000>;
+ };
+
+ leds {
+ compatible = "gpio-leds";
+
+ system {
+ label = "onion:amber:system";
+ gpios = <&gpio 27 GPIO_ACTIVE_LOW>;
+ default-state = "off";
+ };
+ };
+
+ gpio-keys-polled {
+ compatible = "gpio-keys-polled";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ poll-interval = <100>;
+
+ button@0 {
+ label = "reset";
+ linux,code = <KEY_RESTART>;
+ gpios = <&gpio 11 GPIO_ACTIVE_HIGH>;
+ };
+ };
+};
+
+&ref {
+ clock-frequency = <25000000>;
+};
+
+&uart {
+ status = "okay";
+};
+
+&gpio {
+ status = "okay";
+};
+
+&usb {
+ dr_mode = "host";
+ status = "okay";
+};
+
+&usb_phy {
+ status = "okay";
+};
+
+&spi {
+ num-chipselects = <1>;
+ status = "okay";
+
+ /* Winbond 25Q128FVSG SPI flash */
+ spiflash: w25q128@0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "winbond,w25q128", "jedec,spi-nor";
+ spi-max-frequency = <104000000>;
+ reg = <0>;
+ };
+};
diff --git a/arch/mips/boot/dts/qca/ar9331_tl_mr3020.dts b/arch/mips/boot/dts/qca/ar9331_tl_mr3020.dts
new file mode 100644
index 0000000..919cf3b
--- /dev/null
+++ b/arch/mips/boot/dts/qca/ar9331_tl_mr3020.dts
@@ -0,0 +1,118 @@
+/dts-v1/;
+
+#include <dt-bindings/gpio/gpio.h>
+#include <dt-bindings/input/input.h>
+
+#include "ar9331.dtsi"
+
+/ {
+ model = "TP-Link TL-MR3020";
+ compatible = "tplink,tl-mr3020";
+
+ aliases {
+ serial0 = &uart;
+ };
+
+ memory@0 {
+ device_type = "memory";
+ reg = <0x0 0x2000000>;
+ };
+
+ leds {
+ compatible = "gpio-leds";
+
+ wlan {
+ label = "tp-link:green:wlan";
+ gpios = <&gpio 0 GPIO_ACTIVE_HIGH>;
+ default-state = "off";
+ };
+
+ lan {
+ label = "tp-link:green:lan";
+ gpios = <&gpio 17 GPIO_ACTIVE_LOW>;
+ default-state = "off";
+ };
+
+ wps {
+ label = "tp-link:green:wps";
+ gpios = <&gpio 26 GPIO_ACTIVE_LOW>;
+ default-state = "off";
+ };
+
+ led3g {
+ label = "tp-link:green:3g";
+ gpios = <&gpio 27 GPIO_ACTIVE_LOW>;
+ default-state = "off";
+ };
+ };
+
+ gpio-keys-polled {
+ compatible = "gpio-keys-polled";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ poll-interval = <100>;
+
+ button@0 {
+ label = "wps";
+ linux,code = <KEY_WPS_BUTTON>;
+ gpios = <&gpio 11 GPIO_ACTIVE_HIGH>;
+ };
+
+ button@1 {
+ label = "sw1";
+ linux,code = <BTN_0>;
+ gpios = <&gpio 18 GPIO_ACTIVE_HIGH>;
+ };
+
+ button@2 {
+ label = "sw2";
+ linux,code = <BTN_1>;
+ gpios = <&gpio 20 GPIO_ACTIVE_HIGH>;
+ };
+ };
+
+ reg_usb_vbus: reg_usb_vbus {
+ compatible = "regulator-fixed";
+ regulator-name = "usb_vbus";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ gpio = <&gpio 8 GPIO_ACTIVE_HIGH>;
+ enable-active-high;
+ };
+};
+
+&ref {
+ clock-frequency = <25000000>;
+};
+
+&uart {
+ status = "okay";
+};
+
+&gpio {
+ status = "okay";
+};
+
+&usb {
+ dr_mode = "host";
+ vbus-supply = <®_usb_vbus>;
+ status = "okay";
+};
+
+&usb_phy {
+ status = "okay";
+};
+
+&spi {
+ num-chipselects = <1>;
+ status = "okay";
+
+ /* Spansion S25FL032PIF SPI flash */
+ spiflash: s25sl032p@0 {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ compatible = "spansion,s25sl032p", "jedec,spi-nor";
+ spi-max-frequency = <104000000>;
+ reg = <0>;
+ };
+};
diff --git a/arch/mips/boot/tools/.gitignore b/arch/mips/boot/tools/.gitignore
new file mode 100644
index 0000000..be0ed06
--- /dev/null
+++ b/arch/mips/boot/tools/.gitignore
@@ -0,0 +1 @@
+relocs
diff --git a/arch/mips/boot/tools/Makefile b/arch/mips/boot/tools/Makefile
new file mode 100644
index 0000000..d232a68
--- /dev/null
+++ b/arch/mips/boot/tools/Makefile
@@ -0,0 +1,8 @@
+
+hostprogs-y += relocs
+relocs-objs += relocs_32.o
+relocs-objs += relocs_64.o
+relocs-objs += relocs_main.o
+PHONY += relocs
+relocs: $(obj)/relocs
+ @:
diff --git a/arch/mips/boot/tools/relocs.c b/arch/mips/boot/tools/relocs.c
new file mode 100644
index 0000000..b9cbf78
--- /dev/null
+++ b/arch/mips/boot/tools/relocs.c
@@ -0,0 +1,680 @@
+/* This is included from relocs_32/64.c */
+
+#define ElfW(type) _ElfW(ELF_BITS, type)
+#define _ElfW(bits, type) __ElfW(bits, type)
+#define __ElfW(bits, type) Elf##bits##_##type
+
+#define Elf_Addr ElfW(Addr)
+#define Elf_Ehdr ElfW(Ehdr)
+#define Elf_Phdr ElfW(Phdr)
+#define Elf_Shdr ElfW(Shdr)
+#define Elf_Sym ElfW(Sym)
+
+static Elf_Ehdr ehdr;
+
+struct relocs {
+ uint32_t *offset;
+ unsigned long count;
+ unsigned long size;
+};
+
+static struct relocs relocs;
+
+struct section {
+ Elf_Shdr shdr;
+ struct section *link;
+ Elf_Sym *symtab;
+ Elf_Rel *reltab;
+ char *strtab;
+ long shdr_offset;
+};
+static struct section *secs;
+
+static const char * const regex_sym_kernel = {
+/* Symbols matching these regex's should never be relocated */
+ "^(__crc_)",
+};
+
+static regex_t sym_regex_c;
+
+static int regex_skip_reloc(const char *sym_name)
+{
+ return !regexec(&sym_regex_c, sym_name, 0, NULL, 0);
+}
+
+static void regex_init(void)
+{
+ char errbuf[128];
+ int err;
+
+ err = regcomp(&sym_regex_c, regex_sym_kernel,
+ REG_EXTENDED|REG_NOSUB);
+
+ if (err) {
+ regerror(err, &sym_regex_c, errbuf, sizeof(errbuf));
+ die("%s", errbuf);
+ }
+}
+
+static const char *rel_type(unsigned type)
+{
+ static const char * const type_name[] = {
+#define REL_TYPE(X)[X] = #X
+ REL_TYPE(R_MIPS_NONE),
+ REL_TYPE(R_MIPS_16),
+ REL_TYPE(R_MIPS_32),
+ REL_TYPE(R_MIPS_REL32),
+ REL_TYPE(R_MIPS_26),
+ REL_TYPE(R_MIPS_HI16),
+ REL_TYPE(R_MIPS_LO16),
+ REL_TYPE(R_MIPS_GPREL16),
+ REL_TYPE(R_MIPS_LITERAL),
+ REL_TYPE(R_MIPS_GOT16),
+ REL_TYPE(R_MIPS_PC16),
+ REL_TYPE(R_MIPS_CALL16),
+ REL_TYPE(R_MIPS_GPREL32),
+ REL_TYPE(R_MIPS_64),
+ REL_TYPE(R_MIPS_HIGHER),
+ REL_TYPE(R_MIPS_HIGHEST),
+ REL_TYPE(R_MIPS_PC21_S2),
+ REL_TYPE(R_MIPS_PC26_S2),
+#undef REL_TYPE
+ };
+ const char *name = "unknown type rel type name";
+
+ if (type < ARRAY_SIZE(type_name) && type_name[type])
+ name = type_name[type];
+ return name;
+}
+
+static const char *sec_name(unsigned shndx)
+{
+ const char *sec_strtab;
+ const char *name;
+
+ sec_strtab = secs[ehdr.e_shstrndx].strtab;
+ if (shndx < ehdr.e_shnum)
+ name = sec_strtab + secs[shndx].shdr.sh_name;
+ else if (shndx == SHN_ABS)
+ name = "ABSOLUTE";
+ else if (shndx == SHN_COMMON)
+ name = "COMMON";
+ else
+ name = "<noname>";
+ return name;
+}
+
+static struct section *sec_lookup(const char *secname)
+{
+ int i;
+
+ for (i = 0; i < ehdr.e_shnum; i++)
+ if (strcmp(secname, sec_name(i)) == 0)
+ return &secs[i];
+
+ return NULL;
+}
+
+static const char *sym_name(const char *sym_strtab, Elf_Sym *sym)
+{
+ const char *name;
+
+ if (sym->st_name)
+ name = sym_strtab + sym->st_name;
+ else
+ name = sec_name(sym->st_shndx);
+ return name;
+}
+
+#if BYTE_ORDER == LITTLE_ENDIAN
+#define le16_to_cpu(val) (val)
+#define le32_to_cpu(val) (val)
+#define le64_to_cpu(val) (val)
+#define be16_to_cpu(val) bswap_16(val)
+#define be32_to_cpu(val) bswap_32(val)
+#define be64_to_cpu(val) bswap_64(val)
+
+#define cpu_to_le16(val) (val)
+#define cpu_to_le32(val) (val)
+#define cpu_to_le64(val) (val)
+#define cpu_to_be16(val) bswap_16(val)
+#define cpu_to_be32(val) bswap_32(val)
+#define cpu_to_be64(val) bswap_64(val)
+#endif
+#if BYTE_ORDER == BIG_ENDIAN
+#define le16_to_cpu(val) bswap_16(val)
+#define le32_to_cpu(val) bswap_32(val)
+#define le64_to_cpu(val) bswap_64(val)
+#define be16_to_cpu(val) (val)
+#define be32_to_cpu(val) (val)
+#define be64_to_cpu(val) (val)
+
+#define cpu_to_le16(val) bswap_16(val)
+#define cpu_to_le32(val) bswap_32(val)
+#define cpu_to_le64(val) bswap_64(val)
+#define cpu_to_be16(val) (val)
+#define cpu_to_be32(val) (val)
+#define cpu_to_be64(val) (val)
+#endif
+
+static uint16_t elf16_to_cpu(uint16_t val)
+{
+ if (ehdr.e_ident[EI_DATA] == ELFDATA2LSB)
+ return le16_to_cpu(val);
+ else
+ return be16_to_cpu(val);
+}
+
+static uint32_t elf32_to_cpu(uint32_t val)
+{
+ if (ehdr.e_ident[EI_DATA] == ELFDATA2LSB)
+ return le32_to_cpu(val);
+ else
+ return be32_to_cpu(val);
+}
+
+static uint32_t cpu_to_elf32(uint32_t val)
+{
+ if (ehdr.e_ident[EI_DATA] == ELFDATA2LSB)
+ return cpu_to_le32(val);
+ else
+ return cpu_to_be32(val);
+}
+
+#define elf_half_to_cpu(x) elf16_to_cpu(x)
+#define elf_word_to_cpu(x) elf32_to_cpu(x)
+
+#if ELF_BITS == 64
+static uint64_t elf64_to_cpu(uint64_t val)
+{
+ if (ehdr.e_ident[EI_DATA] == ELFDATA2LSB)
+ return le64_to_cpu(val);
+ else
+ return be64_to_cpu(val);
+}
+#define elf_addr_to_cpu(x) elf64_to_cpu(x)
+#define elf_off_to_cpu(x) elf64_to_cpu(x)
+#define elf_xword_to_cpu(x) elf64_to_cpu(x)
+#else
+#define elf_addr_to_cpu(x) elf32_to_cpu(x)
+#define elf_off_to_cpu(x) elf32_to_cpu(x)
+#define elf_xword_to_cpu(x) elf32_to_cpu(x)
+#endif
+
+static void read_ehdr(FILE *fp)
+{
+ if (fread(&ehdr, sizeof(ehdr), 1, fp) != 1)
+ die("Cannot read ELF header: %s\n", strerror(errno));
+
+ if (memcmp(ehdr.e_ident, ELFMAG, SELFMAG) != 0)
+ die("No ELF magic\n");
+
+ if (ehdr.e_ident[EI_CLASS] != ELF_CLASS)
+ die("Not a %d bit executable\n", ELF_BITS);
+
+ if ((ehdr.e_ident[EI_DATA] != ELFDATA2LSB) &&
+ (ehdr.e_ident[EI_DATA] != ELFDATA2MSB))
+ die("Unknown ELF Endianness\n");
+
+ if (ehdr.e_ident[EI_VERSION] != EV_CURRENT)
+ die("Unknown ELF version\n");
+
+ /* Convert the fields to native endian */
+ ehdr.e_type = elf_half_to_cpu(ehdr.e_type);
+ ehdr.e_machine = elf_half_to_cpu(ehdr.e_machine);
+ ehdr.e_version = elf_word_to_cpu(ehdr.e_version);
+ ehdr.e_entry = elf_addr_to_cpu(ehdr.e_entry);
+ ehdr.e_phoff = elf_off_to_cpu(ehdr.e_phoff);
+ ehdr.e_shoff = elf_off_to_cpu(ehdr.e_shoff);
+ ehdr.e_flags = elf_word_to_cpu(ehdr.e_flags);
+ ehdr.e_ehsize = elf_half_to_cpu(ehdr.e_ehsize);
+ ehdr.e_phentsize = elf_half_to_cpu(ehdr.e_phentsize);
+ ehdr.e_phnum = elf_half_to_cpu(ehdr.e_phnum);
+ ehdr.e_shentsize = elf_half_to_cpu(ehdr.e_shentsize);
+ ehdr.e_shnum = elf_half_to_cpu(ehdr.e_shnum);
+ ehdr.e_shstrndx = elf_half_to_cpu(ehdr.e_shstrndx);
+
+ if ((ehdr.e_type != ET_EXEC) && (ehdr.e_type != ET_DYN))
+ die("Unsupported ELF header type\n");
+
+ if (ehdr.e_machine != ELF_MACHINE)
+ die("Not for %s\n", ELF_MACHINE_NAME);
+
+ if (ehdr.e_version != EV_CURRENT)
+ die("Unknown ELF version\n");
+
+ if (ehdr.e_ehsize != sizeof(Elf_Ehdr))
+ die("Bad Elf header size\n");
+
+ if (ehdr.e_phentsize != sizeof(Elf_Phdr))
+ die("Bad program header entry\n");
+
+ if (ehdr.e_shentsize != sizeof(Elf_Shdr))
+ die("Bad section header entry\n");
+
+ if (ehdr.e_shstrndx >= ehdr.e_shnum)
+ die("String table index out of bounds\n");
+}
+
+static void read_shdrs(FILE *fp)
+{
+ int i;
+ Elf_Shdr shdr;
+
+ secs = calloc(ehdr.e_shnum, sizeof(struct section));
+ if (!secs)
+ die("Unable to allocate %d section headers\n", ehdr.e_shnum);
+
+ if (fseek(fp, ehdr.e_shoff, SEEK_SET) < 0)
+ die("Seek to %d failed: %s\n", ehdr.e_shoff, strerror(errno));
+
+ for (i = 0; i < ehdr.e_shnum; i++) {
+ struct section *sec = &secs[i];
+
+ sec->shdr_offset = ftell(fp);
+ if (fread(&shdr, sizeof(shdr), 1, fp) != 1)
+ die("Cannot read ELF section headers %d/%d: %s\n",
+ i, ehdr.e_shnum, strerror(errno));
+ sec->shdr.sh_name = elf_word_to_cpu(shdr.sh_name);
+ sec->shdr.sh_type = elf_word_to_cpu(shdr.sh_type);
+ sec->shdr.sh_flags = elf_xword_to_cpu(shdr.sh_flags);
+ sec->shdr.sh_addr = elf_addr_to_cpu(shdr.sh_addr);
+ sec->shdr.sh_offset = elf_off_to_cpu(shdr.sh_offset);
+ sec->shdr.sh_size = elf_xword_to_cpu(shdr.sh_size);
+ sec->shdr.sh_link = elf_word_to_cpu(shdr.sh_link);
+ sec->shdr.sh_info = elf_word_to_cpu(shdr.sh_info);
+ sec->shdr.sh_addralign = elf_xword_to_cpu(shdr.sh_addralign);
+ sec->shdr.sh_entsize = elf_xword_to_cpu(shdr.sh_entsize);
+ if (sec->shdr.sh_link < ehdr.e_shnum)
+ sec->link = &secs[sec->shdr.sh_link];
+ }
+}
+
+static void read_strtabs(FILE *fp)
+{
+ int i;
+
+ for (i = 0; i < ehdr.e_shnum; i++) {
+ struct section *sec = &secs[i];
+
+ if (sec->shdr.sh_type != SHT_STRTAB)
+ continue;
+
+ sec->strtab = malloc(sec->shdr.sh_size);
+ if (!sec->strtab)
+ die("malloc of %d bytes for strtab failed\n",
+ sec->shdr.sh_size);
+
+ if (fseek(fp, sec->shdr.sh_offset, SEEK_SET) < 0)
+ die("Seek to %d failed: %s\n",
+ sec->shdr.sh_offset, strerror(errno));
+
+ if (fread(sec->strtab, 1, sec->shdr.sh_size, fp) !=
+ sec->shdr.sh_size)
+ die("Cannot read symbol table: %s\n", strerror(errno));
+ }
+}
+
+static void read_symtabs(FILE *fp)
+{
+ int i, j;
+
+ for (i = 0; i < ehdr.e_shnum; i++) {
+ struct section *sec = &secs[i];
+ if (sec->shdr.sh_type != SHT_SYMTAB)
+ continue;
+
+ sec->symtab = malloc(sec->shdr.sh_size);
+ if (!sec->symtab)
+ die("malloc of %d bytes for symtab failed\n",
+ sec->shdr.sh_size);
+
+ if (fseek(fp, sec->shdr.sh_offset, SEEK_SET) < 0)
+ die("Seek to %d failed: %s\n",
+ sec->shdr.sh_offset, strerror(errno));
+
+ if (fread(sec->symtab, 1, sec->shdr.sh_size, fp) !=
+ sec->shdr.sh_size)
+ die("Cannot read symbol table: %s\n", strerror(errno));
+
+ for (j = 0; j < sec->shdr.sh_size/sizeof(Elf_Sym); j++) {
+ Elf_Sym *sym = &sec->symtab[j];
+
+ sym->st_name = elf_word_to_cpu(sym->st_name);
+ sym->st_value = elf_addr_to_cpu(sym->st_value);
+ sym->st_size = elf_xword_to_cpu(sym->st_size);
+ sym->st_shndx = elf_half_to_cpu(sym->st_shndx);
+ }
+ }
+}
+
+static void read_relocs(FILE *fp)
+{
+ static unsigned long base = 0;
+ int i, j;
+
+ if (!base) {
+ struct section *sec = sec_lookup(".text");
+
+ if (!sec)
+ die("Could not find .text section\n");
+
+ base = sec->shdr.sh_addr;
+ }
+
+ for (i = 0; i < ehdr.e_shnum; i++) {
+ struct section *sec = &secs[i];
+
+ if (sec->shdr.sh_type != SHT_REL_TYPE)
+ continue;
+
+ sec->reltab = malloc(sec->shdr.sh_size);
+ if (!sec->reltab)
+ die("malloc of %d bytes for relocs failed\n",
+ sec->shdr.sh_size);
+
+ if (fseek(fp, sec->shdr.sh_offset, SEEK_SET) < 0)
+ die("Seek to %d failed: %s\n",
+ sec->shdr.sh_offset, strerror(errno));
+
+ if (fread(sec->reltab, 1, sec->shdr.sh_size, fp) !=
+ sec->shdr.sh_size)
+ die("Cannot read symbol table: %s\n", strerror(errno));
+
+ for (j = 0; j < sec->shdr.sh_size/sizeof(Elf_Rel); j++) {
+ Elf_Rel *rel = &sec->reltab[j];
+
+ rel->r_offset = elf_addr_to_cpu(rel->r_offset);
+ /* Set offset into kernel image */
+ rel->r_offset -= base;
+#if (ELF_BITS == 32)
+ rel->r_info = elf_xword_to_cpu(rel->r_info);
+#else
+ /* Convert MIPS64 RELA format - only the symbol
+ * index needs converting to native endianness
+ */
+ rel->r_info = rel->r_info;
+ ELF_R_SYM(rel->r_info) = elf32_to_cpu(ELF_R_SYM(rel->r_info));
+#endif
+#if (SHT_REL_TYPE == SHT_RELA)
+ rel->r_addend = elf_xword_to_cpu(rel->r_addend);
+#endif
+ }
+ }
+}
+
+static void remove_relocs(FILE *fp)
+{
+ int i;
+ Elf_Shdr shdr;
+
+ for (i = 0; i < ehdr.e_shnum; i++) {
+ struct section *sec = &secs[i];
+
+ if (sec->shdr.sh_type != SHT_REL_TYPE)
+ continue;
+
+ if (fseek(fp, sec->shdr_offset, SEEK_SET) < 0)
+ die("Seek to %d failed: %s\n",
+ sec->shdr_offset, strerror(errno));
+
+ if (fread(&shdr, sizeof(shdr), 1, fp) != 1)
+ die("Cannot read ELF section headers %d/%d: %s\n",
+ i, ehdr.e_shnum, strerror(errno));
+
+ /* Set relocation section size to 0, effectively removing it.
+ * This is necessary due to lack of support for relocations
+ * in objcopy when creating 32bit elf from 64bit elf.
+ */
+ shdr.sh_size = 0;
+
+ if (fseek(fp, sec->shdr_offset, SEEK_SET) < 0)
+ die("Seek to %d failed: %s\n",
+ sec->shdr_offset, strerror(errno));
+
+ if (fwrite(&shdr, sizeof(shdr), 1, fp) != 1)
+ die("Cannot write ELF section headers %d/%d: %s\n",
+ i, ehdr.e_shnum, strerror(errno));
+ }
+}
+
+static void add_reloc(struct relocs *r, uint32_t offset, unsigned type)
+{
+ /* Relocation representation in binary table:
+ * |76543210|76543210|76543210|76543210|
+ * | Type | offset from _text >> 2 |
+ */
+ offset >>= 2;
+ if (offset > 0x00FFFFFF)
+ die("Kernel image exceeds maximum size for relocation!\n");
+
+ offset = (offset & 0x00FFFFFF) | ((type & 0xFF) << 24);
+
+ if (r->count == r->size) {
+ unsigned long newsize = r->size + 50000;
+ void *mem = realloc(r->offset, newsize * sizeof(r->offset[0]));
+
+ if (!mem)
+ die("realloc failed\n");
+
+ r->offset = mem;
+ r->size = newsize;
+ }
+ r->offset[r->count++] = offset;
+}
+
+static void walk_relocs(int (*process)(struct section *sec, Elf_Rel *rel,
+ Elf_Sym *sym, const char *symname))
+{
+ int i;
+
+ /* Walk through the relocations */
+ for (i = 0; i < ehdr.e_shnum; i++) {
+ char *sym_strtab;
+ Elf_Sym *sh_symtab;
+ struct section *sec_applies, *sec_symtab;
+ int j;
+ struct section *sec = &secs[i];
+
+ if (sec->shdr.sh_type != SHT_REL_TYPE)
+ continue;
+
+ sec_symtab = sec->link;
+ sec_applies = &secs[sec->shdr.sh_info];
+ if (!(sec_applies->shdr.sh_flags & SHF_ALLOC))
+ continue;
+
+ sh_symtab = sec_symtab->symtab;
+ sym_strtab = sec_symtab->link->strtab;
+ for (j = 0; j < sec->shdr.sh_size/sizeof(Elf_Rel); j++) {
+ Elf_Rel *rel = &sec->reltab[j];
+ Elf_Sym *sym = &sh_symtab[ELF_R_SYM(rel->r_info)];
+ const char *symname = sym_name(sym_strtab, sym);
+
+ process(sec, rel, sym, symname);
+ }
+ }
+}
+
+static int do_reloc(struct section *sec, Elf_Rel *rel, Elf_Sym *sym,
+ const char *symname)
+{
+ unsigned r_type = ELF_R_TYPE(rel->r_info);
+ unsigned bind = ELF_ST_BIND(sym->st_info);
+
+ if ((bind == STB_WEAK) && (sym->st_value == 0)) {
+ /* Don't relocate weak symbols without a target */
+ return 0;
+ }
+
+ if (regex_skip_reloc(symname))
+ return 0;
+
+ switch (r_type) {
+ case R_MIPS_NONE:
+ case R_MIPS_REL32:
+ case R_MIPS_PC16:
+ case R_MIPS_PC21_S2:
+ case R_MIPS_PC26_S2:
+ /*
+ * NONE can be ignored and PC relative relocations don't
+ * need to be adjusted.
+ */
+ case R_MIPS_HIGHEST:
+ case R_MIPS_HIGHER:
+ /* We support relocating within the same 4Gb segment only,
+ * thus leaving the top 32bits unchanged
+ */
+ case R_MIPS_LO16:
+ /* We support relocating by 64k jumps only
+ * thus leaving the bottom 16bits unchanged
+ */
+ break;
+
+ case R_MIPS_64:
+ case R_MIPS_32:
+ case R_MIPS_26:
+ case R_MIPS_HI16:
+ add_reloc(&relocs, rel->r_offset, r_type);
+ break;
+
+ default:
+ die("Unsupported relocation type: %s (%d)\n",
+ rel_type(r_type), r_type);
+ break;
+ }
+
+ return 0;
+}
+
+static int write_reloc_as_bin(uint32_t v, FILE *f)
+{
+ unsigned char buf[4];
+
+ v = cpu_to_elf32(v);
+
+ memcpy(buf, &v, sizeof(uint32_t));
+ return fwrite(buf, 1, 4, f);
+}
+
+static int write_reloc_as_text(uint32_t v, FILE *f)
+{
+ int res;
+
+ res = fprintf(f, "\t.long 0x%08"PRIx32"\n", v);
+ if (res < 0)
+ return res;
+ else
+ return sizeof(uint32_t);
+}
+
+static void emit_relocs(int as_text, int as_bin, FILE *outf)
+{
+ int i;
+ int (*write_reloc)(uint32_t, FILE *) = write_reloc_as_bin;
+ int size = 0;
+ int size_reserved;
+ struct section *sec_reloc;
+
+ sec_reloc = sec_lookup(".data.reloc");
+ if (!sec_reloc)
+ die("Could not find relocation section\n");
+
+ size_reserved = sec_reloc->shdr.sh_size;
+
+ /* Collect up the relocations */
+ walk_relocs(do_reloc);
+
+ /* Print the relocations */
+ if (as_text) {
+ /* Print the relocations in a form suitable that
+ * gas will like.
+ */
+ printf(".section \".data.reloc\",\"a\"\n");
+ printf(".balign 4\n");
+ /* Output text to stdout */
+ write_reloc = write_reloc_as_text;
+ outf = stdout;
+ } else if (as_bin) {
+ /* Output raw binary to stdout */
+ outf = stdout;
+ } else {
+ /* Seek to offset of the relocation section.
+ * Each relocation is then written into the
+ * vmlinux kernel image.
+ */
+ if (fseek(outf, sec_reloc->shdr.sh_offset, SEEK_SET) < 0) {
+ die("Seek to %d failed: %s\n",
+ sec_reloc->shdr.sh_offset, strerror(errno));
+ }
+ }
+
+ for (i = 0; i < relocs.count; i++)
+ size += write_reloc(relocs.offset[i], outf);
+
+ /* Print a stop, but only if we've actually written some relocs */
+ if (size)
+ size += write_reloc(0, outf);
+
+ if (size > size_reserved)
+ /* Die, but suggest a value for CONFIG_RELOCATION_TABLE_SIZE
+ * which will fix this problem and allow a bit of headroom
+ * if more kernel features are enabled
+ */
+ die("Relocations overflow available space!\n" \
+ "Please adjust CONFIG_RELOCATION_TABLE_SIZE " \
+ "to at least 0x%08x\n", (size + 0x1000) & ~0xFFF);
+}
+
+/*
+ * As an aid to debugging problems with different linkers
+ * print summary information about the relocs.
+ * Since different linkers tend to emit the sections in
+ * different orders we use the section names in the output.
+ */
+static int do_reloc_info(struct section *sec, Elf_Rel *rel, ElfW(Sym) *sym,
+ const char *symname)
+{
+ printf("%16s 0x%08x %16s %40s %16s\n",
+ sec_name(sec->shdr.sh_info),
+ (unsigned int)rel->r_offset,
+ rel_type(ELF_R_TYPE(rel->r_info)),
+ symname,
+ sec_name(sym->st_shndx));
+ return 0;
+}
+
+static void print_reloc_info(void)
+{
+ printf("%16s %10s %16s %40s %16s\n",
+ "reloc section",
+ "offset",
+ "reloc type",
+ "symbol",
+ "symbol section");
+ walk_relocs(do_reloc_info);
+}
+
+#if ELF_BITS == 64
+# define process process_64
+#else
+# define process process_32
+#endif
+
+void process(FILE *fp, int as_text, int as_bin,
+ int show_reloc_info, int keep_relocs)
+{
+ regex_init();
+ read_ehdr(fp);
+ read_shdrs(fp);
+ read_strtabs(fp);
+ read_symtabs(fp);
+ read_relocs(fp);
+ if (show_reloc_info) {
+ print_reloc_info();
+ return;
+ }
+ emit_relocs(as_text, as_bin, fp);
+ if (!keep_relocs)
+ remove_relocs(fp);
+}
diff --git a/arch/mips/boot/tools/relocs.h b/arch/mips/boot/tools/relocs.h
new file mode 100644
index 0000000..3cf676f
--- /dev/null
+++ b/arch/mips/boot/tools/relocs.h
@@ -0,0 +1,45 @@
+#ifndef RELOCS_H
+#define RELOCS_H
+
+#include <stdio.h>
+#include <stdarg.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <inttypes.h>
+#include <string.h>
+#include <errno.h>
+#include <unistd.h>
+#include <elf.h>
+#include <byteswap.h>
+#define USE_BSD
+#include <endian.h>
+#include <regex.h>
+
+void die(char *fmt, ...);
+
+/*
+ * Introduced for MIPSr6
+ */
+#ifndef R_MIPS_PC21_S2
+#define R_MIPS_PC21_S2 60
+#endif
+
+#ifndef R_MIPS_PC26_S2
+#define R_MIPS_PC26_S2 61
+#endif
+
+#define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0]))
+
+enum symtype {
+ S_ABS,
+ S_REL,
+ S_SEG,
+ S_LIN,
+ S_NSYMTYPES
+};
+
+void process_32(FILE *fp, int as_text, int as_bin,
+ int show_reloc_info, int keep_relocs);
+void process_64(FILE *fp, int as_text, int as_bin,
+ int show_reloc_info, int keep_relocs);
+#endif /* RELOCS_H */
diff --git a/arch/mips/boot/tools/relocs_32.c b/arch/mips/boot/tools/relocs_32.c
new file mode 100644
index 0000000..915bdc0
--- /dev/null
+++ b/arch/mips/boot/tools/relocs_32.c
@@ -0,0 +1,17 @@
+#include "relocs.h"
+
+#define ELF_BITS 32
+
+#define ELF_MACHINE EM_MIPS
+#define ELF_MACHINE_NAME "MIPS"
+#define SHT_REL_TYPE SHT_REL
+#define Elf_Rel ElfW(Rel)
+
+#define ELF_CLASS ELFCLASS32
+#define ELF_R_SYM(val) ELF32_R_SYM(val)
+#define ELF_R_TYPE(val) ELF32_R_TYPE(val)
+#define ELF_ST_TYPE(o) ELF32_ST_TYPE(o)
+#define ELF_ST_BIND(o) ELF32_ST_BIND(o)
+#define ELF_ST_VISIBILITY(o) ELF32_ST_VISIBILITY(o)
+
+#include "relocs.c"
diff --git a/arch/mips/boot/tools/relocs_64.c b/arch/mips/boot/tools/relocs_64.c
new file mode 100644
index 0000000..b671b5e
--- /dev/null
+++ b/arch/mips/boot/tools/relocs_64.c
@@ -0,0 +1,27 @@
+#include "relocs.h"
+
+#define ELF_BITS 64
+
+#define ELF_MACHINE EM_MIPS
+#define ELF_MACHINE_NAME "MIPS64"
+#define SHT_REL_TYPE SHT_RELA
+#define Elf_Rel Elf64_Rela
+
+typedef uint8_t Elf64_Byte;
+
+typedef struct {
+ Elf64_Word r_sym; /* Symbol index. */
+ Elf64_Byte r_ssym; /* Special symbol. */
+ Elf64_Byte r_type3; /* Third relocation. */
+ Elf64_Byte r_type2; /* Second relocation. */
+ Elf64_Byte r_type; /* First relocation. */
+} Elf64_Mips_Rela;
+
+#define ELF_CLASS ELFCLASS64
+#define ELF_R_SYM(val) (((Elf64_Mips_Rela *)(&val))->r_sym)
+#define ELF_R_TYPE(val) (((Elf64_Mips_Rela *)(&val))->r_type)
+#define ELF_ST_TYPE(o) ELF64_ST_TYPE(o)
+#define ELF_ST_BIND(o) ELF64_ST_BIND(o)
+#define ELF_ST_VISIBILITY(o) ELF64_ST_VISIBILITY(o)
+
+#include "relocs.c"
diff --git a/arch/mips/boot/tools/relocs_main.c b/arch/mips/boot/tools/relocs_main.c
new file mode 100644
index 0000000..d8fe234
--- /dev/null
+++ b/arch/mips/boot/tools/relocs_main.c
@@ -0,0 +1,84 @@
+
+#include <stdio.h>
+#include <stdint.h>
+#include <stdarg.h>
+#include <stdlib.h>
+#include <string.h>
+#include <errno.h>
+#include <endian.h>
+#include <elf.h>
+
+#include "relocs.h"
+
+void die(char *fmt, ...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ vfprintf(stderr, fmt, ap);
+ va_end(ap);
+ exit(1);
+}
+
+static void usage(void)
+{
+ die("relocs [--reloc-info|--text|--bin|--keep] vmlinux\n");
+}
+
+int main(int argc, char **argv)
+{
+ int show_reloc_info, as_text, as_bin, keep_relocs;
+ const char *fname;
+ FILE *fp;
+ int i;
+ unsigned char e_ident[EI_NIDENT];
+
+ show_reloc_info = 0;
+ as_text = 0;
+ as_bin = 0;
+ keep_relocs = 0;
+ fname = NULL;
+ for (i = 1; i < argc; i++) {
+ char *arg = argv[i];
+
+ if (*arg == '-') {
+ if (strcmp(arg, "--reloc-info") == 0) {
+ show_reloc_info = 1;
+ continue;
+ }
+ if (strcmp(arg, "--text") == 0) {
+ as_text = 1;
+ continue;
+ }
+ if (strcmp(arg, "--bin") == 0) {
+ as_bin = 1;
+ continue;
+ }
+ if (strcmp(arg, "--keep") == 0) {
+ keep_relocs = 1;
+ continue;
+ }
+ } else if (!fname) {
+ fname = arg;
+ continue;
+ }
+ usage();
+ }
+ if (!fname)
+ usage();
+
+ fp = fopen(fname, "r+");
+ if (!fp)
+ die("Cannot open %s: %s\n", fname, strerror(errno));
+
+ if (fread(&e_ident, 1, EI_NIDENT, fp) != EI_NIDENT)
+ die("Cannot read %s: %s", fname, strerror(errno));
+
+ rewind(fp);
+ if (e_ident[EI_CLASS] == ELFCLASS64)
+ process_64(fp, as_text, as_bin, show_reloc_info, keep_relocs);
+ else
+ process_32(fp, as_text, as_bin, show_reloc_info, keep_relocs);
+ fclose(fp);
+ return 0;
+}
diff --git a/arch/mips/cavium-octeon/csrc-octeon.c b/arch/mips/cavium-octeon/csrc-octeon.c
index 1882e64..23c2344 100644
--- a/arch/mips/cavium-octeon/csrc-octeon.c
+++ b/arch/mips/cavium-octeon/csrc-octeon.c
@@ -19,6 +19,7 @@
#include <asm/octeon/cvmx-ipd-defs.h>
#include <asm/octeon/cvmx-mio-defs.h>
#include <asm/octeon/cvmx-rst-defs.h>
+#include <asm/octeon/cvmx-fpa-defs.h>
static u64 f;
static u64 rdiv;
@@ -65,9 +66,13 @@
*/
void octeon_init_cvmcount(void)
{
+ u64 clk_reg;
unsigned long flags;
unsigned loops = 2;
+ clk_reg = octeon_has_feature(OCTEON_FEATURE_FPA3) ?
+ CVMX_FPA_CLK_COUNT : CVMX_IPD_CLK_COUNT;
+
/* Clobber loops so GCC will not unroll the following while loop. */
asm("" : "+r" (loops));
@@ -77,18 +82,18 @@
* which should give more deterministic timing.
*/
while (loops--) {
- u64 ipd_clk_count = cvmx_read_csr(CVMX_IPD_CLK_COUNT);
+ u64 clk_count = cvmx_read_csr(clk_reg);
if (rdiv != 0) {
- ipd_clk_count *= rdiv;
+ clk_count *= rdiv;
if (f != 0) {
asm("dmultu\t%[cnt],%[f]\n\t"
"mfhi\t%[cnt]"
- : [cnt] "+r" (ipd_clk_count)
+ : [cnt] "+r" (clk_count)
: [f] "r" (f)
: "hi", "lo");
}
}
- write_c0_cvmcount(ipd_clk_count);
+ write_c0_cvmcount(clk_count);
}
local_irq_restore(flags);
}
diff --git a/arch/mips/cavium-octeon/executive/cvmx-helper.c b/arch/mips/cavium-octeon/executive/cvmx-helper.c
index 376701f..ff26d02 100644
--- a/arch/mips/cavium-octeon/executive/cvmx-helper.c
+++ b/arch/mips/cavium-octeon/executive/cvmx-helper.c
@@ -87,6 +87,8 @@
return 9;
if (OCTEON_IS_MODEL(OCTEON_CN56XX) || OCTEON_IS_MODEL(OCTEON_CN52XX))
return 4;
+ if (OCTEON_IS_MODEL(OCTEON_CN7XXX))
+ return 5;
else
return 3;
}
@@ -260,6 +262,41 @@
}
/**
+ * @INTERNAL
+ * Return interface mode for CN7XXX.
+ */
+static cvmx_helper_interface_mode_t __cvmx_get_mode_cn7xxx(int interface)
+{
+ union cvmx_gmxx_inf_mode mode;
+
+ mode.u64 = cvmx_read_csr(CVMX_GMXX_INF_MODE(interface));
+
+ switch (interface) {
+ case 0:
+ case 1:
+ switch (mode.cn68xx.mode) {
+ case 0:
+ return CVMX_HELPER_INTERFACE_MODE_DISABLED;
+ case 1:
+ case 2:
+ return CVMX_HELPER_INTERFACE_MODE_SGMII;
+ case 3:
+ return CVMX_HELPER_INTERFACE_MODE_XAUI;
+ default:
+ return CVMX_HELPER_INTERFACE_MODE_SGMII;
+ }
+ case 2:
+ return CVMX_HELPER_INTERFACE_MODE_NPI;
+ case 3:
+ return CVMX_HELPER_INTERFACE_MODE_LOOP;
+ case 4:
+ return CVMX_HELPER_INTERFACE_MODE_RGMII;
+ default:
+ return CVMX_HELPER_INTERFACE_MODE_DISABLED;
+ }
+}
+
+/**
* Get the operating mode of an interface. Depending on the Octeon
* chip and configuration, this function returns an enumeration
* of the type of packet I/O supported by an interface.
@@ -278,6 +315,12 @@
return CVMX_HELPER_INTERFACE_MODE_DISABLED;
/*
+ * OCTEON III models
+ */
+ if (OCTEON_IS_MODEL(OCTEON_CN7XXX))
+ return __cvmx_get_mode_cn7xxx(interface);
+
+ /*
* Octeon II models
*/
if (OCTEON_IS_MODEL(OCTEON_CN6XXX) || OCTEON_IS_MODEL(OCTEON_CNF71XX))
diff --git a/arch/mips/cavium-octeon/executive/cvmx-sysinfo.c b/arch/mips/cavium-octeon/executive/cvmx-sysinfo.c
index 3d17fac..cc1b1d2 100644
--- a/arch/mips/cavium-octeon/executive/cvmx-sysinfo.c
+++ b/arch/mips/cavium-octeon/executive/cvmx-sysinfo.c
@@ -32,86 +32,22 @@
#include <linux/module.h>
#include <asm/octeon/cvmx.h>
-#include <asm/octeon/cvmx-spinlock.h>
#include <asm/octeon/cvmx-sysinfo.h>
-/**
+/*
* This structure defines the private state maintained by sysinfo module.
- *
*/
-static struct {
- struct cvmx_sysinfo sysinfo; /* system information */
- cvmx_spinlock_t lock; /* mutex spinlock */
-
-} state = {
- .lock = CVMX_SPINLOCK_UNLOCKED_INITIALIZER
-};
-
+static struct cvmx_sysinfo sysinfo; /* system information */
/*
- * Global variables that define the min/max of the memory region set
- * up for 32 bit userspace access.
- */
-uint64_t linux_mem32_min;
-uint64_t linux_mem32_max;
-uint64_t linux_mem32_wired;
-uint64_t linux_mem32_offset;
-
-/**
- * This function returns the application information as obtained
+ * Returns the application information as obtained
* by the bootloader. This provides the core mask of the cores
* running the same application image, as well as the physical
* memory regions available to the core.
- *
- * Returns Pointer to the boot information structure
- *
*/
struct cvmx_sysinfo *cvmx_sysinfo_get(void)
{
- return &(state.sysinfo);
+ return &sysinfo;
}
EXPORT_SYMBOL(cvmx_sysinfo_get);
-/**
- * This function is used in non-simple executive environments (such as
- * Linux kernel, u-boot, etc.) to configure the minimal fields that
- * are required to use simple executive files directly.
- *
- * Locking (if required) must be handled outside of this
- * function
- *
- * @phy_mem_desc_ptr:
- * Pointer to global physical memory descriptor
- * (bootmem descriptor) @board_type: Octeon board
- * type enumeration
- *
- * @board_rev_major:
- * Board major revision
- * @board_rev_minor:
- * Board minor revision
- * @cpu_clock_hz:
- * CPU clock freqency in hertz
- *
- * Returns 0: Failure
- * 1: success
- */
-int cvmx_sysinfo_minimal_initialize(void *phy_mem_desc_ptr,
- uint16_t board_type,
- uint8_t board_rev_major,
- uint8_t board_rev_minor,
- uint32_t cpu_clock_hz)
-{
-
- /* The sysinfo structure was already initialized */
- if (state.sysinfo.board_type)
- return 0;
-
- memset(&(state.sysinfo), 0x0, sizeof(state.sysinfo));
- state.sysinfo.phy_mem_desc_ptr = phy_mem_desc_ptr;
- state.sysinfo.board_type = board_type;
- state.sysinfo.board_rev_major = board_rev_major;
- state.sysinfo.board_rev_minor = board_rev_minor;
- state.sysinfo.cpu_clock_hz = cpu_clock_hz;
-
- return 1;
-}
diff --git a/arch/mips/cavium-octeon/executive/octeon-model.c b/arch/mips/cavium-octeon/executive/octeon-model.c
index b2104bd..d08a2bc 100644
--- a/arch/mips/cavium-octeon/executive/octeon-model.c
+++ b/arch/mips/cavium-octeon/executive/octeon-model.c
@@ -71,11 +71,11 @@
uint32_t fuse_data = 0;
fus3.u64 = 0;
- if (!OCTEON_IS_MODEL(OCTEON_CN6XXX))
+ if (OCTEON_IS_MODEL(OCTEON_CN3XXX) || OCTEON_IS_MODEL(OCTEON_CN5XXX))
fus3.u64 = cvmx_read_csr(CVMX_L2D_FUS3);
fus_dat2.u64 = cvmx_read_csr(CVMX_MIO_FUS_DAT2);
fus_dat3.u64 = cvmx_read_csr(CVMX_MIO_FUS_DAT3);
- num_cores = cvmx_pop(cvmx_read_csr(CVMX_CIU_FUSE));
+ num_cores = cvmx_octeon_num_cores();
/* Make sure the non existent devices look disabled */
switch ((chip_id >> 8) & 0xff) {
@@ -121,6 +121,15 @@
* later.
*/
switch (num_cores) {
+ case 48:
+ core_model = "90";
+ break;
+ case 44:
+ core_model = "88";
+ break;
+ case 40:
+ core_model = "85";
+ break;
case 32:
core_model = "80";
break;
@@ -297,7 +306,7 @@
if (fus_dat3.s.nozip)
suffix = "SCP";
- if (fus_dat3.s.bar2_en)
+ if (fus_dat3.cn56xx.bar2_en)
suffix = "NSPB2";
}
if (fus3.cn56xx.crip_1024k)
@@ -369,6 +378,73 @@
else
suffix = "AAP";
break;
+ case 0x94: /* CNF71XX */
+ family = "F71";
+ if (fus_dat3.cnf71xx.nozip)
+ suffix = "SCP";
+ else
+ suffix = "AAP";
+ break;
+ case 0x95: /* CN78XX */
+ if (num_cores == 6) /* Other core counts match generic */
+ core_model = "35";
+ if (OCTEON_IS_MODEL(OCTEON_CN76XX))
+ family = "76";
+ else
+ family = "78";
+ if (fus_dat3.cn78xx.l2c_crip == 2)
+ family = "77";
+ if (fus_dat3.cn78xx.nozip
+ && fus_dat3.cn78xx.nodfa_dte
+ && fus_dat3.cn78xx.nohna_dte) {
+ if (fus_dat3.cn78xx.nozip &&
+ !fus_dat2.cn78xx.raid_en &&
+ fus_dat3.cn78xx.nohna_dte) {
+ suffix = "CP";
+ } else {
+ suffix = "SCP";
+ }
+ } else if (fus_dat2.cn78xx.raid_en == 0)
+ suffix = "HCP";
+ else
+ suffix = "AAP";
+ break;
+ case 0x96: /* CN70XX */
+ family = "70";
+ if (cvmx_read_csr(CVMX_MIO_FUS_PDF) & (0x1ULL << 32))
+ family = "71";
+ if (fus_dat2.cn70xx.nocrypto)
+ suffix = "CP";
+ else if (fus_dat3.cn70xx.nodfa_dte)
+ suffix = "SCP";
+ else
+ suffix = "AAP";
+ break;
+ case 0x97: /* CN73XX */
+ if (num_cores == 6) /* Other core counts match generic */
+ core_model = "35";
+ family = "73";
+ if (fus_dat3.cn73xx.l2c_crip == 2)
+ family = "72";
+ if (fus_dat3.cn73xx.nozip
+ && fus_dat3.cn73xx.nodfa_dte
+ && fus_dat3.cn73xx.nohna_dte) {
+ if (!fus_dat2.cn73xx.raid_en)
+ suffix = "CP";
+ else
+ suffix = "SCP";
+ } else
+ suffix = "AAP";
+ break;
+ case 0x98: /* CN75XX */
+ family = "F75";
+ if (fus_dat3.cn78xx.nozip
+ && fus_dat3.cn78xx.nodfa_dte
+ && fus_dat3.cn78xx.nohna_dte)
+ suffix = "SCP";
+ else
+ suffix = "AAP";
+ break;
default:
family = "XX";
core_model = "XX";
diff --git a/arch/mips/cavium-octeon/octeon-irq.c b/arch/mips/cavium-octeon/octeon-irq.c
index 4f9eb05..368eb49 100644
--- a/arch/mips/cavium-octeon/octeon-irq.c
+++ b/arch/mips/cavium-octeon/octeon-irq.c
@@ -3,7 +3,7 @@
* License. See the file "COPYING" in the main directory of this archive
* for more details.
*
- * Copyright (C) 2004-2014 Cavium, Inc.
+ * Copyright (C) 2004-2016 Cavium, Inc.
*/
#include <linux/of_address.h>
@@ -19,16 +19,53 @@
#include <asm/octeon/octeon.h>
#include <asm/octeon/cvmx-ciu2-defs.h>
+#include <asm/octeon/cvmx-ciu3-defs.h>
static DEFINE_PER_CPU(unsigned long, octeon_irq_ciu0_en_mirror);
static DEFINE_PER_CPU(unsigned long, octeon_irq_ciu1_en_mirror);
static DEFINE_PER_CPU(raw_spinlock_t, octeon_irq_ciu_spinlock);
+static DEFINE_PER_CPU(unsigned int, octeon_irq_ciu3_idt_ip2);
+
+static DEFINE_PER_CPU(unsigned int, octeon_irq_ciu3_idt_ip3);
+static DEFINE_PER_CPU(struct octeon_ciu3_info *, octeon_ciu3_info);
+#define CIU3_MBOX_PER_CORE 10
+
+/*
+ * The 8 most significant bits of the intsn identify the interrupt major block.
+ * Each major block might use its own interrupt domain. Thus 256 domains are
+ * needed.
+ */
+#define MAX_CIU3_DOMAINS 256
+
+typedef irq_hw_number_t (*octeon_ciu3_intsn2hw_t)(struct irq_domain *, unsigned int);
+
+/* Information for each ciu3 in the system */
+struct octeon_ciu3_info {
+ u64 ciu3_addr;
+ int node;
+ struct irq_domain *domain[MAX_CIU3_DOMAINS];
+ octeon_ciu3_intsn2hw_t intsn2hw[MAX_CIU3_DOMAINS];
+};
+
+/* Each ciu3 in the system uses its own data (one ciu3 per node) */
+static struct octeon_ciu3_info *octeon_ciu3_info_per_node[4];
struct octeon_irq_ciu_domain_data {
int num_sum; /* number of sum registers (2 or 3). */
};
-static __read_mostly u8 octeon_irq_ciu_to_irq[8][64];
+/* Register offsets from ciu3_addr */
+#define CIU3_CONST 0x220
+#define CIU3_IDT_CTL(_idt) ((_idt) * 8 + 0x110000)
+#define CIU3_IDT_PP(_idt, _idx) ((_idt) * 32 + (_idx) * 8 + 0x120000)
+#define CIU3_IDT_IO(_idt) ((_idt) * 8 + 0x130000)
+#define CIU3_DEST_PP_INT(_pp_ip) ((_pp_ip) * 8 + 0x200000)
+#define CIU3_DEST_IO_INT(_io) ((_io) * 8 + 0x210000)
+#define CIU3_ISC_CTL(_intsn) ((_intsn) * 8 + 0x80000000)
+#define CIU3_ISC_W1C(_intsn) ((_intsn) * 8 + 0x90000000)
+#define CIU3_ISC_W1S(_intsn) ((_intsn) * 8 + 0xa0000000)
+
+static __read_mostly int octeon_irq_ciu_to_irq[8][64];
struct octeon_ciu_chip_data {
union {
@@ -39,10 +76,11 @@
struct { /* only used for ciu/ciu2 */
u8 line;
u8 bit;
- u8 gpio_line;
};
};
+ int gpio_line;
int current_cpu; /* Next CPU expected to take this irq */
+ int ciu_node; /* NUMA node number of the CIU */
};
struct octeon_core_chip_data {
@@ -626,6 +664,18 @@
}
}
+static int octeon_irq_ciu_set_type(struct irq_data *data, unsigned int t)
+{
+ irqd_set_trigger_type(data, t);
+
+ if (t & IRQ_TYPE_EDGE_BOTH)
+ irq_set_handler_locked(data, handle_edge_irq);
+ else
+ irq_set_handler_locked(data, handle_level_irq);
+
+ return IRQ_SET_MASK_OK;
+}
+
static void octeon_irq_gpio_setup(struct irq_data *data)
{
union cvmx_gpio_bit_cfgx cfg;
@@ -663,7 +713,7 @@
irqd_set_trigger_type(data, t);
octeon_irq_gpio_setup(data);
- if (irqd_get_trigger_type(data) & IRQ_TYPE_EDGE_BOTH)
+ if (t & IRQ_TYPE_EDGE_BOTH)
irq_set_handler_locked(data, handle_edge_irq);
else
irq_set_handler_locked(data, handle_level_irq);
@@ -863,6 +913,16 @@
}
#endif
+static unsigned int edge_startup(struct irq_data *data)
+{
+ /* ack any pending edge-irq at startup, so there is
+ * an _edge_ to fire on when the event reappears.
+ */
+ data->chip->irq_ack(data);
+ data->chip->irq_enable(data);
+ return 0;
+}
+
/*
* Newer octeon chips have support for lockless CIU operation.
*/
@@ -1158,16 +1218,6 @@
static struct irq_chip *octeon_irq_ciu_chip_edge;
static struct irq_chip *octeon_irq_gpio_chip;
-static bool octeon_irq_virq_in_range(unsigned int virq)
-{
- /* We cannot let it overflow the mapping array. */
- if (virq < (1ul << 8 * sizeof(octeon_irq_ciu_to_irq[0][0])))
- return true;
-
- WARN_ONCE(true, "virq out of range %u.\n", virq);
- return false;
-}
-
static int octeon_irq_ciu_map(struct irq_domain *d,
unsigned int virq, irq_hw_number_t hw)
{
@@ -1176,13 +1226,6 @@
unsigned int bit = hw & 63;
struct octeon_irq_ciu_domain_data *dd = d->host_data;
- if (!octeon_irq_virq_in_range(virq))
- return -EINVAL;
-
- /* Don't map irq if it is reserved for GPIO. */
- if (line == 0 && bit >= 16 && bit <32)
- return 0;
-
if (line >= dd->num_sum || octeon_irq_ciu_to_irq[line][bit] != 0)
return -EINVAL;
@@ -1215,9 +1258,6 @@
unsigned int line, bit;
int r;
- if (!octeon_irq_virq_in_range(virq))
- return -EINVAL;
-
line = (hw + gpiod->base_hwirq) >> 6;
bit = (hw + gpiod->base_hwirq) & 63;
if (line > ARRAY_SIZE(octeon_irq_ciu_to_irq) ||
@@ -1899,9 +1939,6 @@
unsigned int line = hw >> 6;
unsigned int bit = hw & 63;
- if (!octeon_irq_virq_in_range(virq))
- return -EINVAL;
-
/*
* Don't map irq if it is reserved for GPIO.
* (Line 7 are the GPIO lines.)
@@ -2294,10 +2331,598 @@
return 0;
}
+int octeon_irq_ciu3_xlat(struct irq_domain *d,
+ struct device_node *node,
+ const u32 *intspec,
+ unsigned int intsize,
+ unsigned long *out_hwirq,
+ unsigned int *out_type)
+{
+ struct octeon_ciu3_info *ciu3_info = d->host_data;
+ unsigned int hwirq, type, intsn_major;
+ union cvmx_ciu3_iscx_ctl isc;
+
+ if (intsize < 2)
+ return -EINVAL;
+ hwirq = intspec[0];
+ type = intspec[1];
+
+ if (hwirq >= (1 << 20))
+ return -EINVAL;
+
+ intsn_major = hwirq >> 12;
+ switch (intsn_major) {
+ case 0x04: /* Software handled separately. */
+ return -EINVAL;
+ default:
+ break;
+ }
+
+ isc.u64 = cvmx_read_csr(ciu3_info->ciu3_addr + CIU3_ISC_CTL(hwirq));
+ if (!isc.s.imp)
+ return -EINVAL;
+
+ switch (type) {
+ case 4: /* official value for level triggering. */
+ *out_type = IRQ_TYPE_LEVEL_HIGH;
+ break;
+ case 0: /* unofficial value, but we might as well let it work. */
+ case 1: /* official value for edge triggering. */
+ *out_type = IRQ_TYPE_EDGE_RISING;
+ break;
+ default: /* Nothing else is acceptable. */
+ return -EINVAL;
+ }
+
+ *out_hwirq = hwirq;
+
+ return 0;
+}
+
+void octeon_irq_ciu3_enable(struct irq_data *data)
+{
+ int cpu;
+ union cvmx_ciu3_iscx_ctl isc_ctl;
+ union cvmx_ciu3_iscx_w1c isc_w1c;
+ u64 isc_ctl_addr;
+
+ struct octeon_ciu_chip_data *cd;
+
+ cpu = next_cpu_for_irq(data);
+
+ cd = irq_data_get_irq_chip_data(data);
+
+ isc_w1c.u64 = 0;
+ isc_w1c.s.en = 1;
+ cvmx_write_csr(cd->ciu3_addr + CIU3_ISC_W1C(cd->intsn), isc_w1c.u64);
+
+ isc_ctl_addr = cd->ciu3_addr + CIU3_ISC_CTL(cd->intsn);
+ isc_ctl.u64 = 0;
+ isc_ctl.s.en = 1;
+ isc_ctl.s.idt = per_cpu(octeon_irq_ciu3_idt_ip2, cpu);
+ cvmx_write_csr(isc_ctl_addr, isc_ctl.u64);
+ cvmx_read_csr(isc_ctl_addr);
+}
+
+void octeon_irq_ciu3_disable(struct irq_data *data)
+{
+ u64 isc_ctl_addr;
+ union cvmx_ciu3_iscx_w1c isc_w1c;
+
+ struct octeon_ciu_chip_data *cd;
+
+ cd = irq_data_get_irq_chip_data(data);
+
+ isc_w1c.u64 = 0;
+ isc_w1c.s.en = 1;
+
+ isc_ctl_addr = cd->ciu3_addr + CIU3_ISC_CTL(cd->intsn);
+ cvmx_write_csr(cd->ciu3_addr + CIU3_ISC_W1C(cd->intsn), isc_w1c.u64);
+ cvmx_write_csr(isc_ctl_addr, 0);
+ cvmx_read_csr(isc_ctl_addr);
+}
+
+void octeon_irq_ciu3_ack(struct irq_data *data)
+{
+ u64 isc_w1c_addr;
+ union cvmx_ciu3_iscx_w1c isc_w1c;
+ struct octeon_ciu_chip_data *cd;
+ u32 trigger_type = irqd_get_trigger_type(data);
+
+ /*
+ * We use a single irq_chip, so we have to do nothing to ack a
+ * level interrupt.
+ */
+ if (!(trigger_type & IRQ_TYPE_EDGE_BOTH))
+ return;
+
+ cd = irq_data_get_irq_chip_data(data);
+
+ isc_w1c.u64 = 0;
+ isc_w1c.s.raw = 1;
+
+ isc_w1c_addr = cd->ciu3_addr + CIU3_ISC_W1C(cd->intsn);
+ cvmx_write_csr(isc_w1c_addr, isc_w1c.u64);
+ cvmx_read_csr(isc_w1c_addr);
+}
+
+void octeon_irq_ciu3_mask(struct irq_data *data)
+{
+ union cvmx_ciu3_iscx_w1c isc_w1c;
+ u64 isc_w1c_addr;
+ struct octeon_ciu_chip_data *cd;
+
+ cd = irq_data_get_irq_chip_data(data);
+
+ isc_w1c.u64 = 0;
+ isc_w1c.s.en = 1;
+
+ isc_w1c_addr = cd->ciu3_addr + CIU3_ISC_W1C(cd->intsn);
+ cvmx_write_csr(isc_w1c_addr, isc_w1c.u64);
+ cvmx_read_csr(isc_w1c_addr);
+}
+
+void octeon_irq_ciu3_mask_ack(struct irq_data *data)
+{
+ union cvmx_ciu3_iscx_w1c isc_w1c;
+ u64 isc_w1c_addr;
+ struct octeon_ciu_chip_data *cd;
+ u32 trigger_type = irqd_get_trigger_type(data);
+
+ cd = irq_data_get_irq_chip_data(data);
+
+ isc_w1c.u64 = 0;
+ isc_w1c.s.en = 1;
+
+ /*
+ * We use a single irq_chip, so only ack an edge (!level)
+ * interrupt.
+ */
+ if (trigger_type & IRQ_TYPE_EDGE_BOTH)
+ isc_w1c.s.raw = 1;
+
+ isc_w1c_addr = cd->ciu3_addr + CIU3_ISC_W1C(cd->intsn);
+ cvmx_write_csr(isc_w1c_addr, isc_w1c.u64);
+ cvmx_read_csr(isc_w1c_addr);
+}
+
+#ifdef CONFIG_SMP
+int octeon_irq_ciu3_set_affinity(struct irq_data *data,
+ const struct cpumask *dest, bool force)
+{
+ union cvmx_ciu3_iscx_ctl isc_ctl;
+ union cvmx_ciu3_iscx_w1c isc_w1c;
+ u64 isc_ctl_addr;
+ int cpu;
+ bool enable_one = !irqd_irq_disabled(data) && !irqd_irq_masked(data);
+ struct octeon_ciu_chip_data *cd = irq_data_get_irq_chip_data(data);
+
+ if (!cpumask_subset(dest, cpumask_of_node(cd->ciu_node)))
+ return -EINVAL;
+
+ if (!enable_one)
+ return IRQ_SET_MASK_OK;
+
+ cd = irq_data_get_irq_chip_data(data);
+ cpu = cpumask_first(dest);
+ if (cpu >= nr_cpu_ids)
+ cpu = smp_processor_id();
+ cd->current_cpu = cpu;
+
+ isc_w1c.u64 = 0;
+ isc_w1c.s.en = 1;
+ cvmx_write_csr(cd->ciu3_addr + CIU3_ISC_W1C(cd->intsn), isc_w1c.u64);
+
+ isc_ctl_addr = cd->ciu3_addr + CIU3_ISC_CTL(cd->intsn);
+ isc_ctl.u64 = 0;
+ isc_ctl.s.en = 1;
+ isc_ctl.s.idt = per_cpu(octeon_irq_ciu3_idt_ip2, cpu);
+ cvmx_write_csr(isc_ctl_addr, isc_ctl.u64);
+ cvmx_read_csr(isc_ctl_addr);
+
+ return IRQ_SET_MASK_OK;
+}
+#endif
+
+static struct irq_chip octeon_irq_chip_ciu3 = {
+ .name = "CIU3",
+ .irq_startup = edge_startup,
+ .irq_enable = octeon_irq_ciu3_enable,
+ .irq_disable = octeon_irq_ciu3_disable,
+ .irq_ack = octeon_irq_ciu3_ack,
+ .irq_mask = octeon_irq_ciu3_mask,
+ .irq_mask_ack = octeon_irq_ciu3_mask_ack,
+ .irq_unmask = octeon_irq_ciu3_enable,
+ .irq_set_type = octeon_irq_ciu_set_type,
+#ifdef CONFIG_SMP
+ .irq_set_affinity = octeon_irq_ciu3_set_affinity,
+ .irq_cpu_offline = octeon_irq_cpu_offline_ciu,
+#endif
+};
+
+int octeon_irq_ciu3_mapx(struct irq_domain *d, unsigned int virq,
+ irq_hw_number_t hw, struct irq_chip *chip)
+{
+ struct octeon_ciu3_info *ciu3_info = d->host_data;
+ struct octeon_ciu_chip_data *cd = kzalloc_node(sizeof(*cd), GFP_KERNEL,
+ ciu3_info->node);
+ if (!cd)
+ return -ENOMEM;
+ cd->intsn = hw;
+ cd->current_cpu = -1;
+ cd->ciu3_addr = ciu3_info->ciu3_addr;
+ cd->ciu_node = ciu3_info->node;
+ irq_set_chip_and_handler(virq, chip, handle_edge_irq);
+ irq_set_chip_data(virq, cd);
+
+ return 0;
+}
+
+static int octeon_irq_ciu3_map(struct irq_domain *d,
+ unsigned int virq, irq_hw_number_t hw)
+{
+ return octeon_irq_ciu3_mapx(d, virq, hw, &octeon_irq_chip_ciu3);
+}
+
+static struct irq_domain_ops octeon_dflt_domain_ciu3_ops = {
+ .map = octeon_irq_ciu3_map,
+ .unmap = octeon_irq_free_cd,
+ .xlate = octeon_irq_ciu3_xlat,
+};
+
+static void octeon_irq_ciu3_ip2(void)
+{
+ union cvmx_ciu3_destx_pp_int dest_pp_int;
+ struct octeon_ciu3_info *ciu3_info;
+ u64 ciu3_addr;
+
+ ciu3_info = __this_cpu_read(octeon_ciu3_info);
+ ciu3_addr = ciu3_info->ciu3_addr;
+
+ dest_pp_int.u64 = cvmx_read_csr(ciu3_addr + CIU3_DEST_PP_INT(3 * cvmx_get_local_core_num()));
+
+ if (likely(dest_pp_int.s.intr)) {
+ irq_hw_number_t intsn = dest_pp_int.s.intsn;
+ irq_hw_number_t hw;
+ struct irq_domain *domain;
+ /* Get the domain to use from the major block */
+ int block = intsn >> 12;
+ int ret;
+
+ domain = ciu3_info->domain[block];
+ if (ciu3_info->intsn2hw[block])
+ hw = ciu3_info->intsn2hw[block](domain, intsn);
+ else
+ hw = intsn;
+
+ ret = handle_domain_irq(domain, hw, NULL);
+ if (ret < 0) {
+ union cvmx_ciu3_iscx_w1c isc_w1c;
+ u64 isc_w1c_addr = ciu3_addr + CIU3_ISC_W1C(intsn);
+
+ isc_w1c.u64 = 0;
+ isc_w1c.s.en = 1;
+ cvmx_write_csr(isc_w1c_addr, isc_w1c.u64);
+ cvmx_read_csr(isc_w1c_addr);
+ spurious_interrupt();
+ }
+ } else {
+ spurious_interrupt();
+ }
+}
+
+/*
+ * 10 mbox per core starting from zero.
+ * Base mbox is core * 10
+ */
+static unsigned int octeon_irq_ciu3_base_mbox_intsn(int core)
+{
+ /* SW (mbox) are 0x04 in bits 12..19 */
+ return 0x04000 + CIU3_MBOX_PER_CORE * core;
+}
+
+static unsigned int octeon_irq_ciu3_mbox_intsn_for_core(int core, unsigned int mbox)
+{
+ return octeon_irq_ciu3_base_mbox_intsn(core) + mbox;
+}
+
+static unsigned int octeon_irq_ciu3_mbox_intsn_for_cpu(int cpu, unsigned int mbox)
+{
+ int local_core = octeon_coreid_for_cpu(cpu) & 0x3f;
+
+ return octeon_irq_ciu3_mbox_intsn_for_core(local_core, mbox);
+}
+
+static void octeon_irq_ciu3_mbox(void)
+{
+ union cvmx_ciu3_destx_pp_int dest_pp_int;
+ struct octeon_ciu3_info *ciu3_info;
+ u64 ciu3_addr;
+ int core = cvmx_get_local_core_num();
+
+ ciu3_info = __this_cpu_read(octeon_ciu3_info);
+ ciu3_addr = ciu3_info->ciu3_addr;
+
+ dest_pp_int.u64 = cvmx_read_csr(ciu3_addr + CIU3_DEST_PP_INT(1 + 3 * core));
+
+ if (likely(dest_pp_int.s.intr)) {
+ irq_hw_number_t intsn = dest_pp_int.s.intsn;
+ int mbox = intsn - octeon_irq_ciu3_base_mbox_intsn(core);
+
+ if (likely(mbox >= 0 && mbox < CIU3_MBOX_PER_CORE)) {
+ do_IRQ(mbox + OCTEON_IRQ_MBOX0);
+ } else {
+ union cvmx_ciu3_iscx_w1c isc_w1c;
+ u64 isc_w1c_addr = ciu3_addr + CIU3_ISC_W1C(intsn);
+
+ isc_w1c.u64 = 0;
+ isc_w1c.s.en = 1;
+ cvmx_write_csr(isc_w1c_addr, isc_w1c.u64);
+ cvmx_read_csr(isc_w1c_addr);
+ spurious_interrupt();
+ }
+ } else {
+ spurious_interrupt();
+ }
+}
+
+void octeon_ciu3_mbox_send(int cpu, unsigned int mbox)
+{
+ struct octeon_ciu3_info *ciu3_info;
+ unsigned int intsn;
+ union cvmx_ciu3_iscx_w1s isc_w1s;
+ u64 isc_w1s_addr;
+
+ if (WARN_ON_ONCE(mbox >= CIU3_MBOX_PER_CORE))
+ return;
+
+ intsn = octeon_irq_ciu3_mbox_intsn_for_cpu(cpu, mbox);
+ ciu3_info = per_cpu(octeon_ciu3_info, cpu);
+ isc_w1s_addr = ciu3_info->ciu3_addr + CIU3_ISC_W1S(intsn);
+
+ isc_w1s.u64 = 0;
+ isc_w1s.s.raw = 1;
+
+ cvmx_write_csr(isc_w1s_addr, isc_w1s.u64);
+ cvmx_read_csr(isc_w1s_addr);
+}
+
+static void octeon_irq_ciu3_mbox_set_enable(struct irq_data *data, int cpu, bool en)
+{
+ struct octeon_ciu3_info *ciu3_info;
+ unsigned int intsn;
+ u64 isc_ctl_addr, isc_w1c_addr;
+ union cvmx_ciu3_iscx_ctl isc_ctl;
+ unsigned int mbox = data->irq - OCTEON_IRQ_MBOX0;
+
+ intsn = octeon_irq_ciu3_mbox_intsn_for_cpu(cpu, mbox);
+ ciu3_info = per_cpu(octeon_ciu3_info, cpu);
+ isc_w1c_addr = ciu3_info->ciu3_addr + CIU3_ISC_W1C(intsn);
+ isc_ctl_addr = ciu3_info->ciu3_addr + CIU3_ISC_CTL(intsn);
+
+ isc_ctl.u64 = 0;
+ isc_ctl.s.en = 1;
+
+ cvmx_write_csr(isc_w1c_addr, isc_ctl.u64);
+ cvmx_write_csr(isc_ctl_addr, 0);
+ if (en) {
+ unsigned int idt = per_cpu(octeon_irq_ciu3_idt_ip3, cpu);
+
+ isc_ctl.u64 = 0;
+ isc_ctl.s.en = 1;
+ isc_ctl.s.idt = idt;
+ cvmx_write_csr(isc_ctl_addr, isc_ctl.u64);
+ }
+ cvmx_read_csr(isc_ctl_addr);
+}
+
+static void octeon_irq_ciu3_mbox_enable(struct irq_data *data)
+{
+ int cpu;
+ unsigned int mbox = data->irq - OCTEON_IRQ_MBOX0;
+
+ WARN_ON(mbox >= CIU3_MBOX_PER_CORE);
+
+ for_each_online_cpu(cpu)
+ octeon_irq_ciu3_mbox_set_enable(data, cpu, true);
+}
+
+static void octeon_irq_ciu3_mbox_disable(struct irq_data *data)
+{
+ int cpu;
+ unsigned int mbox = data->irq - OCTEON_IRQ_MBOX0;
+
+ WARN_ON(mbox >= CIU3_MBOX_PER_CORE);
+
+ for_each_online_cpu(cpu)
+ octeon_irq_ciu3_mbox_set_enable(data, cpu, false);
+}
+
+static void octeon_irq_ciu3_mbox_ack(struct irq_data *data)
+{
+ struct octeon_ciu3_info *ciu3_info;
+ unsigned int intsn;
+ u64 isc_w1c_addr;
+ union cvmx_ciu3_iscx_w1c isc_w1c;
+ unsigned int mbox = data->irq - OCTEON_IRQ_MBOX0;
+
+ intsn = octeon_irq_ciu3_mbox_intsn_for_core(cvmx_get_local_core_num(), mbox);
+
+ isc_w1c.u64 = 0;
+ isc_w1c.s.raw = 1;
+
+ ciu3_info = __this_cpu_read(octeon_ciu3_info);
+ isc_w1c_addr = ciu3_info->ciu3_addr + CIU3_ISC_W1C(intsn);
+ cvmx_write_csr(isc_w1c_addr, isc_w1c.u64);
+ cvmx_read_csr(isc_w1c_addr);
+}
+
+static void octeon_irq_ciu3_mbox_cpu_online(struct irq_data *data)
+{
+ octeon_irq_ciu3_mbox_set_enable(data, smp_processor_id(), true);
+}
+
+static void octeon_irq_ciu3_mbox_cpu_offline(struct irq_data *data)
+{
+ octeon_irq_ciu3_mbox_set_enable(data, smp_processor_id(), false);
+}
+
+static int octeon_irq_ciu3_alloc_resources(struct octeon_ciu3_info *ciu3_info)
+{
+ u64 b = ciu3_info->ciu3_addr;
+ int idt_ip2, idt_ip3, idt_ip4;
+ int unused_idt2;
+ int core = cvmx_get_local_core_num();
+ int i;
+
+ __this_cpu_write(octeon_ciu3_info, ciu3_info);
+
+ /*
+ * 4 idt per core starting from 1 because zero is reserved.
+ * Base idt per core is 4 * core + 1
+ */
+ idt_ip2 = core * 4 + 1;
+ idt_ip3 = core * 4 + 2;
+ idt_ip4 = core * 4 + 3;
+ unused_idt2 = core * 4 + 4;
+ __this_cpu_write(octeon_irq_ciu3_idt_ip2, idt_ip2);
+ __this_cpu_write(octeon_irq_ciu3_idt_ip3, idt_ip3);
+
+ /* ip2 interrupts for this CPU */
+ cvmx_write_csr(b + CIU3_IDT_CTL(idt_ip2), 0);
+ cvmx_write_csr(b + CIU3_IDT_PP(idt_ip2, 0), 1ull << core);
+ cvmx_write_csr(b + CIU3_IDT_IO(idt_ip2), 0);
+
+ /* ip3 interrupts for this CPU */
+ cvmx_write_csr(b + CIU3_IDT_CTL(idt_ip3), 1);
+ cvmx_write_csr(b + CIU3_IDT_PP(idt_ip3, 0), 1ull << core);
+ cvmx_write_csr(b + CIU3_IDT_IO(idt_ip3), 0);
+
+ /* ip4 interrupts for this CPU */
+ cvmx_write_csr(b + CIU3_IDT_CTL(idt_ip4), 2);
+ cvmx_write_csr(b + CIU3_IDT_PP(idt_ip4, 0), 0);
+ cvmx_write_csr(b + CIU3_IDT_IO(idt_ip4), 0);
+
+ cvmx_write_csr(b + CIU3_IDT_CTL(unused_idt2), 0);
+ cvmx_write_csr(b + CIU3_IDT_PP(unused_idt2, 0), 0);
+ cvmx_write_csr(b + CIU3_IDT_IO(unused_idt2), 0);
+
+ for (i = 0; i < CIU3_MBOX_PER_CORE; i++) {
+ unsigned int intsn = octeon_irq_ciu3_mbox_intsn_for_core(core, i);
+
+ cvmx_write_csr(b + CIU3_ISC_W1C(intsn), 2);
+ cvmx_write_csr(b + CIU3_ISC_CTL(intsn), 0);
+ }
+
+ return 0;
+}
+
+static void octeon_irq_setup_secondary_ciu3(void)
+{
+ struct octeon_ciu3_info *ciu3_info;
+
+ ciu3_info = octeon_ciu3_info_per_node[cvmx_get_node_num()];
+ octeon_irq_ciu3_alloc_resources(ciu3_info);
+ irq_cpu_online();
+
+ /* Enable the CIU lines */
+ set_c0_status(STATUSF_IP3 | STATUSF_IP2);
+ if (octeon_irq_use_ip4)
+ set_c0_status(STATUSF_IP4);
+ else
+ clear_c0_status(STATUSF_IP4);
+}
+
+static struct irq_chip octeon_irq_chip_ciu3_mbox = {
+ .name = "CIU3-M",
+ .irq_enable = octeon_irq_ciu3_mbox_enable,
+ .irq_disable = octeon_irq_ciu3_mbox_disable,
+ .irq_ack = octeon_irq_ciu3_mbox_ack,
+
+ .irq_cpu_online = octeon_irq_ciu3_mbox_cpu_online,
+ .irq_cpu_offline = octeon_irq_ciu3_mbox_cpu_offline,
+ .flags = IRQCHIP_ONOFFLINE_ENABLED,
+};
+
+static int __init octeon_irq_init_ciu3(struct device_node *ciu_node,
+ struct device_node *parent)
+{
+ int i;
+ int node;
+ struct irq_domain *domain;
+ struct octeon_ciu3_info *ciu3_info;
+ const __be32 *zero_addr;
+ u64 base_addr;
+ union cvmx_ciu3_const consts;
+
+ node = 0; /* of_node_to_nid(ciu_node); */
+ ciu3_info = kzalloc_node(sizeof(*ciu3_info), GFP_KERNEL, node);
+
+ if (!ciu3_info)
+ return -ENOMEM;
+
+ zero_addr = of_get_address(ciu_node, 0, NULL, NULL);
+ if (WARN_ON(!zero_addr))
+ return -EINVAL;
+
+ base_addr = of_translate_address(ciu_node, zero_addr);
+ base_addr = (u64)phys_to_virt(base_addr);
+
+ ciu3_info->ciu3_addr = base_addr;
+ ciu3_info->node = node;
+
+ consts.u64 = cvmx_read_csr(base_addr + CIU3_CONST);
+
+ octeon_irq_setup_secondary = octeon_irq_setup_secondary_ciu3;
+
+ octeon_irq_ip2 = octeon_irq_ciu3_ip2;
+ octeon_irq_ip3 = octeon_irq_ciu3_mbox;
+ octeon_irq_ip4 = octeon_irq_ip4_mask;
+
+ if (node == cvmx_get_node_num()) {
+ /* Mips internal */
+ octeon_irq_init_core();
+
+ /* Only do per CPU things if it is the CIU of the boot node. */
+ i = irq_alloc_descs_from(OCTEON_IRQ_MBOX0, 8, node);
+ WARN_ON(i < 0);
+
+ for (i = 0; i < 8; i++)
+ irq_set_chip_and_handler(i + OCTEON_IRQ_MBOX0,
+ &octeon_irq_chip_ciu3_mbox, handle_percpu_irq);
+ }
+
+ /*
+ * Initialize all domains to use the default domain. Specific major
+ * blocks will overwrite the default domain as needed.
+ */
+ domain = irq_domain_add_tree(ciu_node, &octeon_dflt_domain_ciu3_ops,
+ ciu3_info);
+ for (i = 0; i < MAX_CIU3_DOMAINS; i++)
+ ciu3_info->domain[i] = domain;
+
+ octeon_ciu3_info_per_node[node] = ciu3_info;
+
+ if (node == cvmx_get_node_num()) {
+ /* Only do per CPU things if it is the CIU of the boot node. */
+ octeon_irq_ciu3_alloc_resources(ciu3_info);
+ if (node == 0)
+ irq_set_default_host(domain);
+
+ octeon_irq_use_ip4 = false;
+ /* Enable the CIU lines */
+ set_c0_status(STATUSF_IP2 | STATUSF_IP3);
+ clear_c0_status(STATUSF_IP4);
+ }
+
+ return 0;
+}
+
static struct of_device_id ciu_types[] __initdata = {
{.compatible = "cavium,octeon-3860-ciu", .data = octeon_irq_init_ciu},
{.compatible = "cavium,octeon-3860-gpio", .data = octeon_irq_init_gpio},
{.compatible = "cavium,octeon-6880-ciu2", .data = octeon_irq_init_ciu2},
+ {.compatible = "cavium,octeon-7890-ciu3", .data = octeon_irq_init_ciu3},
{.compatible = "cavium,octeon-7130-cib", .data = octeon_irq_init_cib},
{}
};
diff --git a/arch/mips/cavium-octeon/octeon-platform.c b/arch/mips/cavium-octeon/octeon-platform.c
index d113c8d..7aeafed 100644
--- a/arch/mips/cavium-octeon/octeon-platform.c
+++ b/arch/mips/cavium-octeon/octeon-platform.c
@@ -13,6 +13,7 @@
#include <linux/i2c.h>
#include <linux/usb.h>
#include <linux/dma-mapping.h>
+#include <linux/etherdevice.h>
#include <linux/module.h>
#include <linux/mutex.h>
#include <linux/slab.h>
@@ -525,10 +526,17 @@
static void __init octeon_fdt_set_mac_addr(int n, u64 *pmac)
{
+ const u8 *old_mac;
+ int old_len;
u8 new_mac[6];
u64 mac = *pmac;
int r;
+ old_mac = fdt_getprop(initial_boot_params, n, "local-mac-address",
+ &old_len);
+ if (!old_mac || old_len != 6 || is_valid_ether_addr(old_mac))
+ return;
+
new_mac[0] = (mac >> 40) & 0xff;
new_mac[1] = (mac >> 32) & 0xff;
new_mac[2] = (mac >> 24) & 0xff;
@@ -560,7 +568,7 @@
fdt_nop_node(initial_boot_params, node);
}
-static void __init octeon_fdt_pip_port(int iface, int i, int p, int max, u64 *pmac)
+static void __init octeon_fdt_pip_port(int iface, int i, int p, int max)
{
char name_buffer[20];
int eth;
@@ -583,10 +591,9 @@
phy_addr = cvmx_helper_board_get_mii_address(ipd_port);
octeon_fdt_set_phy(eth, phy_addr);
- octeon_fdt_set_mac_addr(eth, pmac);
}
-static void __init octeon_fdt_pip_iface(int pip, int idx, u64 *pmac)
+static void __init octeon_fdt_pip_iface(int pip, int idx)
{
char name_buffer[20];
int iface;
@@ -602,7 +609,73 @@
count = cvmx_helper_ports_on_interface(idx);
for (p = 0; p < 16; p++)
- octeon_fdt_pip_port(iface, idx, p, count - 1, pmac);
+ octeon_fdt_pip_port(iface, idx, p, count - 1);
+}
+
+void __init octeon_fill_mac_addresses(void)
+{
+ const char *alias_prop;
+ char name_buffer[20];
+ u64 mac_addr_base;
+ int aliases;
+ int pip;
+ int i;
+
+ aliases = fdt_path_offset(initial_boot_params, "/aliases");
+ if (aliases < 0)
+ return;
+
+ mac_addr_base =
+ ((octeon_bootinfo->mac_addr_base[0] & 0xffull)) << 40 |
+ ((octeon_bootinfo->mac_addr_base[1] & 0xffull)) << 32 |
+ ((octeon_bootinfo->mac_addr_base[2] & 0xffull)) << 24 |
+ ((octeon_bootinfo->mac_addr_base[3] & 0xffull)) << 16 |
+ ((octeon_bootinfo->mac_addr_base[4] & 0xffull)) << 8 |
+ (octeon_bootinfo->mac_addr_base[5] & 0xffull);
+
+ for (i = 0; i < 2; i++) {
+ int mgmt;
+
+ snprintf(name_buffer, sizeof(name_buffer), "mix%d", i);
+ alias_prop = fdt_getprop(initial_boot_params, aliases,
+ name_buffer, NULL);
+ if (!alias_prop)
+ continue;
+ mgmt = fdt_path_offset(initial_boot_params, alias_prop);
+ if (mgmt < 0)
+ continue;
+ octeon_fdt_set_mac_addr(mgmt, &mac_addr_base);
+ }
+
+ alias_prop = fdt_getprop(initial_boot_params, aliases, "pip", NULL);
+ if (!alias_prop)
+ return;
+
+ pip = fdt_path_offset(initial_boot_params, alias_prop);
+ if (pip < 0)
+ return;
+
+ for (i = 0; i <= 4; i++) {
+ int iface;
+ int p;
+
+ snprintf(name_buffer, sizeof(name_buffer), "interface@%d", i);
+ iface = fdt_subnode_offset(initial_boot_params, pip,
+ name_buffer);
+ if (iface < 0)
+ continue;
+ for (p = 0; p < 16; p++) {
+ int eth;
+
+ snprintf(name_buffer, sizeof(name_buffer),
+ "ethernet@%x", p);
+ eth = fdt_subnode_offset(initial_boot_params, iface,
+ name_buffer);
+ if (eth < 0)
+ continue;
+ octeon_fdt_set_mac_addr(eth, &mac_addr_base);
+ }
+ }
}
int __init octeon_prune_device_tree(void)
@@ -612,7 +685,6 @@
const char *alias_prop;
char name_buffer[20];
int aliases;
- u64 mac_addr_base;
if (fdt_check_header(initial_boot_params))
panic("Corrupt Device Tree.");
@@ -623,15 +695,6 @@
return -EINVAL;
}
-
- mac_addr_base =
- ((octeon_bootinfo->mac_addr_base[0] & 0xffull)) << 40 |
- ((octeon_bootinfo->mac_addr_base[1] & 0xffull)) << 32 |
- ((octeon_bootinfo->mac_addr_base[2] & 0xffull)) << 24 |
- ((octeon_bootinfo->mac_addr_base[3] & 0xffull)) << 16 |
- ((octeon_bootinfo->mac_addr_base[4] & 0xffull)) << 8 |
- (octeon_bootinfo->mac_addr_base[5] & 0xffull);
-
if (OCTEON_IS_MODEL(OCTEON_CN52XX) || OCTEON_IS_MODEL(OCTEON_CN63XX))
max_port = 2;
else if (OCTEON_IS_MODEL(OCTEON_CN56XX) || OCTEON_IS_MODEL(OCTEON_CN68XX))
@@ -660,7 +723,6 @@
} else {
int phy_addr = cvmx_helper_board_get_mii_address(CVMX_HELPER_BOARD_MGMT_IPD_PORT + i);
octeon_fdt_set_phy(mgmt, phy_addr);
- octeon_fdt_set_mac_addr(mgmt, &mac_addr_base);
}
}
}
@@ -670,7 +732,7 @@
int pip = fdt_path_offset(initial_boot_params, pip_path);
if (pip >= 0)
for (i = 0; i <= 4; i++)
- octeon_fdt_pip_iface(pip, i, &mac_addr_base);
+ octeon_fdt_pip_iface(pip, i);
}
/* I2C */
diff --git a/arch/mips/cavium-octeon/setup.c b/arch/mips/cavium-octeon/setup.c
index cd7101f..64f852b 100644
--- a/arch/mips/cavium-octeon/setup.c
+++ b/arch/mips/cavium-octeon/setup.c
@@ -43,8 +43,6 @@
#include <asm/octeon/cvmx-mio-defs.h>
#include <asm/octeon/cvmx-rst-defs.h>
-extern struct plat_smp_ops octeon_smp_ops;
-
#ifdef CONFIG_PCI
extern void pci_console_init(const char *arg);
#endif
@@ -466,15 +464,25 @@
static char __read_mostly octeon_system_type[80];
-static int __init init_octeon_system_type(void)
+static void __init init_octeon_system_type(void)
{
- snprintf(octeon_system_type, sizeof(octeon_system_type), "%s (%s)",
- cvmx_board_type_to_string(octeon_bootinfo->board_type),
- octeon_model_get_string(read_c0_prid()));
+ char const *board_type;
- return 0;
+ board_type = cvmx_board_type_to_string(octeon_bootinfo->board_type);
+ if (board_type == NULL) {
+ struct device_node *root;
+ int ret;
+
+ root = of_find_node_by_path("/");
+ ret = of_property_read_string(root, "model", &board_type);
+ of_node_put(root);
+ if (ret)
+ board_type = "Unsupported Board";
+ }
+
+ snprintf(octeon_system_type, sizeof(octeon_system_type), "%s (%s)",
+ board_type, octeon_model_get_string(read_c0_prid()));
}
-early_initcall(init_octeon_system_type);
/**
* Return a string representing the system type
@@ -492,8 +500,6 @@
void octeon_user_io_init(void)
{
union octeon_cvmemctl cvmmemctl;
- union cvmx_iob_fau_timeout fau_timeout;
- union cvmx_pow_nw_tim nm_tim;
/* Get the current settings for CP0_CVMMEMCTL_REG */
cvmmemctl.u64 = read_c0_cvmmemctl();
@@ -595,17 +601,27 @@
CONFIG_CAVIUM_OCTEON_CVMSEG_SIZE,
CONFIG_CAVIUM_OCTEON_CVMSEG_SIZE * 128);
- /* Set a default for the hardware timeouts */
- fau_timeout.u64 = 0;
- fau_timeout.s.tout_val = 0xfff;
- /* Disable tagwait FAU timeout */
- fau_timeout.s.tout_enb = 0;
- cvmx_write_csr(CVMX_IOB_FAU_TIMEOUT, fau_timeout.u64);
+ if (octeon_has_feature(OCTEON_FEATURE_FAU)) {
+ union cvmx_iob_fau_timeout fau_timeout;
- nm_tim.u64 = 0;
- /* 4096 cycles */
- nm_tim.s.nw_tim = 3;
- cvmx_write_csr(CVMX_POW_NW_TIM, nm_tim.u64);
+ /* Set a default for the hardware timeouts */
+ fau_timeout.u64 = 0;
+ fau_timeout.s.tout_val = 0xfff;
+ /* Disable tagwait FAU timeout */
+ fau_timeout.s.tout_enb = 0;
+ cvmx_write_csr(CVMX_IOB_FAU_TIMEOUT, fau_timeout.u64);
+ }
+
+ if ((!OCTEON_IS_MODEL(OCTEON_CN68XX) &&
+ !OCTEON_IS_MODEL(OCTEON_CN7XXX)) ||
+ OCTEON_IS_MODEL(OCTEON_CN70XX)) {
+ union cvmx_pow_nw_tim nm_tim;
+
+ nm_tim.u64 = 0;
+ /* 4096 cycles */
+ nm_tim.s.nw_tim = 3;
+ cvmx_write_csr(CVMX_POW_NW_TIM, nm_tim.u64);
+ }
write_octeon_c0_icacheerr(0);
write_c0_derraddr1(0);
@@ -637,9 +653,22 @@
sysinfo = cvmx_sysinfo_get();
memset(sysinfo, 0, sizeof(*sysinfo));
sysinfo->system_dram_size = octeon_bootinfo->dram_size << 20;
- sysinfo->phy_mem_desc_ptr =
- cvmx_phys_to_ptr(octeon_bootinfo->phy_mem_desc_addr);
- sysinfo->core_mask = octeon_bootinfo->core_mask;
+ sysinfo->phy_mem_desc_addr = (u64)phys_to_virt(octeon_bootinfo->phy_mem_desc_addr);
+
+ if ((octeon_bootinfo->major_version > 1) ||
+ (octeon_bootinfo->major_version == 1 &&
+ octeon_bootinfo->minor_version >= 4))
+ cvmx_coremask_copy(&sysinfo->core_mask,
+ &octeon_bootinfo->ext_core_mask);
+ else
+ cvmx_coremask_set64(&sysinfo->core_mask,
+ octeon_bootinfo->core_mask);
+
+ /* Some broken u-boot pass garbage in upper bits, clear them out */
+ if (!OCTEON_IS_MODEL(OCTEON_CN78XX))
+ for (i = 512; i < 1024; i++)
+ cvmx_coremask_clear_core(&sysinfo->core_mask, i);
+
sysinfo->exception_base_addr = octeon_bootinfo->exception_base_addr;
sysinfo->cpu_clock_hz = octeon_bootinfo->eclock_hz;
sysinfo->dram_data_rate_hz = octeon_bootinfo->dclock_hz * 2;
@@ -867,7 +896,7 @@
#endif
octeon_user_io_init();
- register_smp_ops(&octeon_smp_ops);
+ octeon_setup_smp();
}
/* Exclude a single page from the regions obtained in plat_mem_setup. */
@@ -1079,6 +1108,7 @@
}
}
+void __init octeon_fill_mac_addresses(void);
int octeon_prune_device_tree(void);
extern const char __appended_dtb;
@@ -1088,11 +1118,13 @@
{
const void *fdt;
bool do_prune;
+ bool fill_mac;
#ifdef CONFIG_MIPS_ELF_APPENDED_DTB
if (!fdt_check_header(&__appended_dtb)) {
fdt = &__appended_dtb;
do_prune = false;
+ fill_mac = true;
pr_info("Using appended Device Tree.\n");
} else
#endif
@@ -1101,13 +1133,16 @@
if (fdt_check_header(fdt))
panic("Corrupt Device Tree passed to kernel.");
do_prune = false;
+ fill_mac = false;
pr_info("Using passed Device Tree.\n");
} else if (OCTEON_IS_MODEL(OCTEON_CN68XX)) {
fdt = &__dtb_octeon_68xx_begin;
do_prune = true;
+ fill_mac = true;
} else {
fdt = &__dtb_octeon_3xxx_begin;
do_prune = true;
+ fill_mac = true;
}
initial_boot_params = (void *)fdt;
@@ -1116,7 +1151,10 @@
octeon_prune_device_tree();
pr_info("Using internal Device Tree.\n");
}
+ if (fill_mac)
+ octeon_fill_mac_addresses();
unflatten_and_copy_device_tree();
+ init_octeon_system_type();
}
static int __initdata disable_octeon_edac_p;
diff --git a/arch/mips/cavium-octeon/smp.c b/arch/mips/cavium-octeon/smp.c
index 42412ba..dff88aa 100644
--- a/arch/mips/cavium-octeon/smp.c
+++ b/arch/mips/cavium-octeon/smp.c
@@ -30,25 +30,55 @@
EXPORT_SYMBOL(octeon_bootloader_entry_addr);
#endif
+static void octeon_icache_flush(void)
+{
+ asm volatile ("synci 0($0)\n");
+}
+
+static void (*octeon_message_functions[8])(void) = {
+ scheduler_ipi,
+ generic_smp_call_function_interrupt,
+ octeon_icache_flush,
+};
+
static irqreturn_t mailbox_interrupt(int irq, void *dev_id)
{
- const int coreid = cvmx_get_core_num();
- uint64_t action;
+ u64 mbox_clrx = CVMX_CIU_MBOX_CLRX(cvmx_get_core_num());
+ u64 action;
+ int i;
- /* Load the mailbox register to figure out what we're supposed to do */
- action = cvmx_read_csr(CVMX_CIU_MBOX_CLRX(coreid)) & 0xffff;
+ /*
+ * Make sure the function array initialization remains
+ * correct.
+ */
+ BUILD_BUG_ON(SMP_RESCHEDULE_YOURSELF != (1 << 0));
+ BUILD_BUG_ON(SMP_CALL_FUNCTION != (1 << 1));
+ BUILD_BUG_ON(SMP_ICACHE_FLUSH != (1 << 2));
+
+ /*
+ * Load the mailbox register to figure out what we're supposed
+ * to do.
+ */
+ action = cvmx_read_csr(mbox_clrx);
+
+ if (OCTEON_IS_MODEL(OCTEON_CN68XX))
+ action &= 0xff;
+ else
+ action &= 0xffff;
/* Clear the mailbox to clear the interrupt */
- cvmx_write_csr(CVMX_CIU_MBOX_CLRX(coreid), action);
+ cvmx_write_csr(mbox_clrx, action);
- if (action & SMP_CALL_FUNCTION)
- generic_smp_call_function_interrupt();
- if (action & SMP_RESCHEDULE_YOURSELF)
- scheduler_ipi();
+ for (i = 0; i < ARRAY_SIZE(octeon_message_functions) && action;) {
+ if (action & 1) {
+ void (*fn)(void) = octeon_message_functions[i];
- /* Check if we've been told to flush the icache */
- if (action & SMP_ICACHE_FLUSH)
- asm volatile ("synci 0($0)\n");
+ if (fn)
+ fn();
+ }
+ action >>= 1;
+ i++;
+ }
return IRQ_HANDLED;
}
@@ -97,13 +127,15 @@
#endif
}
-static void octeon_smp_setup(void)
+static void __init octeon_smp_setup(void)
{
const int coreid = cvmx_get_core_num();
int cpus;
int id;
- int core_mask = octeon_get_boot_coremask();
+ struct cvmx_sysinfo *sysinfo = cvmx_sysinfo_get();
+
#ifdef CONFIG_HOTPLUG_CPU
+ int core_mask = octeon_get_boot_coremask();
unsigned int num_cores = cvmx_octeon_num_cores();
#endif
@@ -119,7 +151,7 @@
/* The present CPUs get the lowest CPU numbers. */
cpus = 1;
for (id = 0; id < NR_CPUS; id++) {
- if ((id != coreid) && (core_mask & (1 << id))) {
+ if ((id != coreid) && cvmx_coremask_is_core_set(&sysinfo->core_mask, id)) {
set_cpu_possible(cpus, true);
set_cpu_present(cpus, true);
__cpu_number_map[id] = cpus;
@@ -196,7 +228,7 @@
* Callout to firmware before smp_init
*
*/
-void octeon_prepare_cpus(unsigned int max_cpus)
+static void __init octeon_prepare_cpus(unsigned int max_cpus)
{
/*
* Only the low order mailbox bits are used for IPIs, leave
@@ -242,7 +274,7 @@
cpumask_clear_cpu(cpu, &cpu_callin_map);
octeon_fixup_irqs();
- flush_cache_all();
+ __flush_cache_all();
local_flush_tlb_all();
return 0;
@@ -388,3 +420,92 @@
.cpu_die = octeon_cpu_die,
#endif
};
+
+static irqreturn_t octeon_78xx_reched_interrupt(int irq, void *dev_id)
+{
+ scheduler_ipi();
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t octeon_78xx_call_function_interrupt(int irq, void *dev_id)
+{
+ generic_smp_call_function_interrupt();
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t octeon_78xx_icache_flush_interrupt(int irq, void *dev_id)
+{
+ octeon_icache_flush();
+ return IRQ_HANDLED;
+}
+
+/*
+ * Callout to firmware before smp_init
+ */
+static void octeon_78xx_prepare_cpus(unsigned int max_cpus)
+{
+ if (request_irq(OCTEON_IRQ_MBOX0 + 0,
+ octeon_78xx_reched_interrupt,
+ IRQF_PERCPU | IRQF_NO_THREAD, "Scheduler",
+ octeon_78xx_reched_interrupt)) {
+ panic("Cannot request_irq for SchedulerIPI");
+ }
+ if (request_irq(OCTEON_IRQ_MBOX0 + 1,
+ octeon_78xx_call_function_interrupt,
+ IRQF_PERCPU | IRQF_NO_THREAD, "SMP-Call",
+ octeon_78xx_call_function_interrupt)) {
+ panic("Cannot request_irq for SMP-Call");
+ }
+ if (request_irq(OCTEON_IRQ_MBOX0 + 2,
+ octeon_78xx_icache_flush_interrupt,
+ IRQF_PERCPU | IRQF_NO_THREAD, "ICache-Flush",
+ octeon_78xx_icache_flush_interrupt)) {
+ panic("Cannot request_irq for ICache-Flush");
+ }
+}
+
+static void octeon_78xx_send_ipi_single(int cpu, unsigned int action)
+{
+ int i;
+
+ for (i = 0; i < 8; i++) {
+ if (action & 1)
+ octeon_ciu3_mbox_send(cpu, i);
+ action >>= 1;
+ }
+}
+
+static void octeon_78xx_send_ipi_mask(const struct cpumask *mask,
+ unsigned int action)
+{
+ unsigned int cpu;
+
+ for_each_cpu(cpu, mask)
+ octeon_78xx_send_ipi_single(cpu, action);
+}
+
+static struct plat_smp_ops octeon_78xx_smp_ops = {
+ .send_ipi_single = octeon_78xx_send_ipi_single,
+ .send_ipi_mask = octeon_78xx_send_ipi_mask,
+ .init_secondary = octeon_init_secondary,
+ .smp_finish = octeon_smp_finish,
+ .boot_secondary = octeon_boot_secondary,
+ .smp_setup = octeon_smp_setup,
+ .prepare_cpus = octeon_78xx_prepare_cpus,
+#ifdef CONFIG_HOTPLUG_CPU
+ .cpu_disable = octeon_cpu_disable,
+ .cpu_die = octeon_cpu_die,
+#endif
+};
+
+void __init octeon_setup_smp(void)
+{
+ struct plat_smp_ops *ops;
+
+ if (octeon_has_feature(OCTEON_FEATURE_CIU3))
+ ops = &octeon_78xx_smp_ops;
+ else
+ ops = &octeon_smp_ops;
+
+ register_smp_ops(ops);
+}
diff --git a/arch/mips/configs/bcm47xx_defconfig b/arch/mips/configs/bcm47xx_defconfig
index 0db4eb3..fad8e96 100644
--- a/arch/mips/configs/bcm47xx_defconfig
+++ b/arch/mips/configs/bcm47xx_defconfig
@@ -23,7 +23,6 @@
CONFIG_IP_MROUTE_MULTIPLE_TABLES=y
CONFIG_SYN_COOKIES=y
CONFIG_TCP_CONG_ADVANCED=y
-CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_MULTIPLE_TABLES=y
CONFIG_IPV6_SUBTREES=y
CONFIG_IPV6_MROUTE=y
diff --git a/arch/mips/configs/bcm63xx_defconfig b/arch/mips/configs/bcm63xx_defconfig
index 3fec264..5599a9f 100644
--- a/arch/mips/configs/bcm63xx_defconfig
+++ b/arch/mips/configs/bcm63xx_defconfig
@@ -44,6 +44,7 @@
# CONFIG_STANDALONE is not set
# CONFIG_PREVENT_FIRMWARE_BUILD is not set
CONFIG_MTD=y
+CONFIG_MTD_BCM63XX_PARTS=y
CONFIG_MTD_CFI=y
CONFIG_MTD_CFI_INTELEXT=y
CONFIG_MTD_CFI_AMDSTD=y
diff --git a/arch/mips/configs/bigsur_defconfig b/arch/mips/configs/bigsur_defconfig
index e070dac..d20b09d 100644
--- a/arch/mips/configs/bigsur_defconfig
+++ b/arch/mips/configs/bigsur_defconfig
@@ -62,7 +62,6 @@
CONFIG_INET_XFRM_MODE_TUNNEL=m
# CONFIG_INET_LRO is not set
CONFIG_TCP_MD5SIG=y
-CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_IPV6_ROUTE_INFO=y
CONFIG_IPV6_OPTIMISTIC_DAD=y
diff --git a/arch/mips/configs/bmips_be_defconfig b/arch/mips/configs/bmips_be_defconfig
index 24dcb90..acf7785 100644
--- a/arch/mips/configs/bmips_be_defconfig
+++ b/arch/mips/configs/bmips_be_defconfig
@@ -36,6 +36,7 @@
CONFIG_PRINTK_TIME=y
CONFIG_BRCMSTB_GISB_ARB=y
CONFIG_MTD=y
+CONFIG_MTD_BCM63XX_PARTS=y
CONFIG_MTD_CFI=y
CONFIG_MTD_CFI_INTELEXT=y
CONFIG_MTD_CFI_AMDSTD=y
diff --git a/arch/mips/configs/cavium_octeon_defconfig b/arch/mips/configs/cavium_octeon_defconfig
index e57058d..dcac308 100644
--- a/arch/mips/configs/cavium_octeon_defconfig
+++ b/arch/mips/configs/cavium_octeon_defconfig
@@ -119,14 +119,16 @@
CONFIG_SPI_OCTEON=y
# CONFIG_HWMON is not set
CONFIG_WATCHDOG=y
-# CONFIG_USB_SUPPORT is not set
-CONFIG_USB_EHCI_BIG_ENDIAN_MMIO=y
-CONFIG_USB_OHCI_BIG_ENDIAN_MMIO=y
-CONFIG_USB_OHCI_LITTLE_ENDIAN=y
+CONFIG_USB=m
+CONFIG_USB_EHCI_HCD=m
+CONFIG_USB_EHCI_HCD_PLATFORM=m
+CONFIG_USB_OHCI_HCD=m
+CONFIG_USB_OHCI_HCD_PLATFORM=m
CONFIG_RTC_CLASS=y
CONFIG_RTC_DRV_DS1307=y
CONFIG_STAGING=y
CONFIG_OCTEON_ETHERNET=y
+CONFIG_OCTEON_USB=m
# CONFIG_IOMMU_SUPPORT is not set
CONFIG_EXT4_FS=y
CONFIG_EXT4_FS_POSIX_ACL=y
@@ -152,6 +154,9 @@
CONFIG_SECURITY_NETWORK=y
CONFIG_CRYPTO_CBC=y
CONFIG_CRYPTO_HMAC=y
-CONFIG_CRYPTO_MD5=y
+CONFIG_CRYPTO_MD5_OCTEON=y
+CONFIG_CRYPTO_SHA1_OCTEON=m
+CONFIG_CRYPTO_SHA256_OCTEON=m
+CONFIG_CRYPTO_SHA512_OCTEON=m
CONFIG_CRYPTO_DES=y
# CONFIG_CRYPTO_ANSI_CPRNG is not set
diff --git a/arch/mips/configs/db1xxx_defconfig b/arch/mips/configs/db1xxx_defconfig
index 3bdb72a..f0c8971 100644
--- a/arch/mips/configs/db1xxx_defconfig
+++ b/arch/mips/configs/db1xxx_defconfig
@@ -18,7 +18,6 @@
CONFIG_CGROUP_DEVICE=y
CONFIG_CPUSETS=y
CONFIG_CGROUP_CPUACCT=y
-CONFIG_RESOURCE_COUNTERS=y
CONFIG_MEMCG=y
CONFIG_MEMCG_SWAP=y
CONFIG_MEMCG_KMEM=y
diff --git a/arch/mips/configs/decstation_defconfig b/arch/mips/configs/decstation_defconfig
index ebc011c..2b6cb41 100644
--- a/arch/mips/configs/decstation_defconfig
+++ b/arch/mips/configs/decstation_defconfig
@@ -30,7 +30,6 @@
CONFIG_INET_XFRM_MODE_TUNNEL=m
CONFIG_INET_XFRM_MODE_BEET=m
CONFIG_TCP_MD5SIG=y
-CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_IPV6_ROUTE_INFO=y
CONFIG_INET6_AH=m
diff --git a/arch/mips/configs/ip22_defconfig b/arch/mips/configs/ip22_defconfig
index 6ba9ce9..5d83ff7 100644
--- a/arch/mips/configs/ip22_defconfig
+++ b/arch/mips/configs/ip22_defconfig
@@ -48,7 +48,6 @@
CONFIG_INET_XFRM_MODE_BEET=m
# CONFIG_INET_LRO is not set
CONFIG_TCP_MD5SIG=y
-CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_IPV6_ROUTE_INFO=y
CONFIG_IPV6_OPTIMISTIC_DAD=y
diff --git a/arch/mips/configs/ip27_defconfig b/arch/mips/configs/ip27_defconfig
index 77e9f50..2b74aee 100644
--- a/arch/mips/configs/ip27_defconfig
+++ b/arch/mips/configs/ip27_defconfig
@@ -43,7 +43,6 @@
CONFIG_INET_XFRM_MODE_BEET=m
CONFIG_TCP_MD5SIG=y
CONFIG_IPV6=y
-CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_IPV6_ROUTE_INFO=y
CONFIG_IPV6_OPTIMISTIC_DAD=y
diff --git a/arch/mips/configs/jazz_defconfig b/arch/mips/configs/jazz_defconfig
index a5e85e1..3019fce 100644
--- a/arch/mips/configs/jazz_defconfig
+++ b/arch/mips/configs/jazz_defconfig
@@ -34,7 +34,6 @@
CONFIG_INET_XFRM_MODE_TRANSPORT=m
CONFIG_INET_XFRM_MODE_TUNNEL=m
CONFIG_TCP_MD5SIG=y
-CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_IPV6_ROUTE_INFO=y
CONFIG_INET6_AH=m
diff --git a/arch/mips/configs/lemote2f_defconfig b/arch/mips/configs/lemote2f_defconfig
index d1f198b..5da76e0 100644
--- a/arch/mips/configs/lemote2f_defconfig
+++ b/arch/mips/configs/lemote2f_defconfig
@@ -71,7 +71,6 @@
CONFIG_TCP_CONG_BIC=y
CONFIG_DEFAULT_BIC=y
CONFIG_TCP_MD5SIG=y
-CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_IPV6_TUNNEL=m
CONFIG_IPV6_MULTIPLE_TABLES=y
diff --git a/arch/mips/configs/loongson1b_defconfig b/arch/mips/configs/loongson1b_defconfig
new file mode 100644
index 0000000..c442f27
--- /dev/null
+++ b/arch/mips/configs/loongson1b_defconfig
@@ -0,0 +1,125 @@
+CONFIG_MACH_LOONGSON32=y
+CONFIG_PREEMPT=y
+# CONFIG_SECCOMP is not set
+# CONFIG_LOCALVERSION_AUTO is not set
+CONFIG_KERNEL_XZ=y
+CONFIG_SYSVIPC=y
+CONFIG_HIGH_RES_TIMERS=y
+CONFIG_BSD_PROCESS_ACCT=y
+CONFIG_BSD_PROCESS_ACCT_V3=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_LOG_BUF_SHIFT=16
+CONFIG_NAMESPACES=y
+CONFIG_CC_OPTIMIZE_FOR_SIZE=y
+CONFIG_EXPERT=y
+CONFIG_PERF_EVENTS=y
+# CONFIG_COMPAT_BRK is not set
+CONFIG_MODULES=y
+CONFIG_MODULE_UNLOAD=y
+CONFIG_MODVERSIONS=y
+# CONFIG_LBDAF is not set
+# CONFIG_BLK_DEV_BSG is not set
+# CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set
+# CONFIG_SUSPEND is not set
+CONFIG_NET=y
+CONFIG_PACKET=y
+CONFIG_UNIX=y
+CONFIG_INET=y
+CONFIG_IP_PNP=y
+CONFIG_IP_PNP_DHCP=y
+CONFIG_SYN_COOKIES=y
+# CONFIG_INET_XFRM_MODE_TRANSPORT is not set
+# CONFIG_INET_XFRM_MODE_TUNNEL is not set
+# CONFIG_INET_XFRM_MODE_BEET is not set
+# CONFIG_INET_DIAG is not set
+# CONFIG_IPV6 is not set
+# CONFIG_WIRELESS is not set
+CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
+CONFIG_DEVTMPFS=y
+CONFIG_DEVTMPFS_MOUNT=y
+# CONFIG_STANDALONE is not set
+CONFIG_MTD=y
+CONFIG_MTD_CMDLINE_PARTS=y
+CONFIG_MTD_BLOCK=y
+CONFIG_MTD_NAND=y
+CONFIG_MTD_NAND_LOONGSON1=y
+CONFIG_MTD_UBI=y
+CONFIG_BLK_DEV_LOOP=y
+CONFIG_SCSI=m
+# CONFIG_SCSI_PROC_FS is not set
+CONFIG_BLK_DEV_SD=m
+# CONFIG_SCSI_LOWLEVEL is not set
+CONFIG_NETDEVICES=y
+# CONFIG_NET_VENDOR_BROADCOM is not set
+# CONFIG_NET_VENDOR_INTEL is not set
+# CONFIG_NET_VENDOR_MARVELL is not set
+# CONFIG_NET_VENDOR_MICREL is not set
+# CONFIG_NET_VENDOR_NATSEMI is not set
+# CONFIG_NET_VENDOR_SEEQ is not set
+# CONFIG_NET_VENDOR_SMSC is not set
+CONFIG_STMMAC_ETH=y
+# CONFIG_NET_VENDOR_WIZNET is not set
+# CONFIG_WLAN is not set
+CONFIG_INPUT_EVDEV=y
+# CONFIG_INPUT_KEYBOARD is not set
+# CONFIG_INPUT_MOUSE is not set
+# CONFIG_SERIO is not set
+CONFIG_VT_HW_CONSOLE_BINDING=y
+CONFIG_LEGACY_PTY_COUNT=8
+# CONFIG_DEVKMEM is not set
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y
+# CONFIG_HW_RANDOM is not set
+CONFIG_GPIOLIB=y
+CONFIG_GPIO_LOONGSON1=y
+# CONFIG_HWMON is not set
+# CONFIG_VGA_CONSOLE is not set
+CONFIG_HID_GENERIC=m
+CONFIG_USB_HID=m
+CONFIG_USB=y
+CONFIG_USB_ANNOUNCE_NEW_DEVICES=y
+CONFIG_USB_EHCI_HCD=y
+# CONFIG_USB_EHCI_TT_NEWSCHED is not set
+CONFIG_USB_EHCI_HCD_PLATFORM=y
+CONFIG_USB_STORAGE=m
+CONFIG_USB_SERIAL=m
+CONFIG_USB_SERIAL_PL2303=m
+CONFIG_NEW_LEDS=y
+CONFIG_LEDS_CLASS=y
+CONFIG_LEDS_GPIO=y
+CONFIG_LEDS_TRIGGERS=y
+CONFIG_LEDS_TRIGGER_HEARTBEAT=y
+CONFIG_RTC_CLASS=y
+CONFIG_RTC_DRV_LOONGSON1=y
+# CONFIG_IOMMU_SUPPORT is not set
+CONFIG_EXT2_FS=y
+CONFIG_EXT2_FS_XATTR=y
+CONFIG_EXT2_FS_POSIX_ACL=y
+CONFIG_EXT2_FS_SECURITY=y
+CONFIG_EXT3_FS=y
+CONFIG_EXT3_FS_POSIX_ACL=y
+CONFIG_EXT3_FS_SECURITY=y
+# CONFIG_DNOTIFY is not set
+CONFIG_VFAT_FS=y
+CONFIG_PROC_KCORE=y
+CONFIG_TMPFS=y
+CONFIG_TMPFS_POSIX_ACL=y
+CONFIG_UBIFS_FS=y
+CONFIG_UBIFS_FS_ADVANCED_COMPR=y
+CONFIG_UBIFS_ATIME_SUPPORT=y
+CONFIG_NFS_FS=y
+CONFIG_ROOT_NFS=y
+CONFIG_NLS_CODEPAGE_437=m
+CONFIG_NLS_ISO8859_1=m
+CONFIG_DYNAMIC_DEBUG=y
+# CONFIG_ENABLE_WARN_DEPRECATED is not set
+# CONFIG_ENABLE_MUST_CHECK is not set
+CONFIG_DEBUG_FS=y
+CONFIG_MAGIC_SYSRQ=y
+# CONFIG_SCHED_DEBUG is not set
+# CONFIG_DEBUG_PREEMPT is not set
+# CONFIG_FTRACE is not set
+# CONFIG_EARLY_PRINTK is not set
+# CONFIG_CRYPTO_ECHAINIV is not set
+# CONFIG_CRYPTO_HW is not set
diff --git a/arch/mips/configs/loongson3_defconfig b/arch/mips/configs/loongson3_defconfig
index f8bf915c..7f95c4b 100644
--- a/arch/mips/configs/loongson3_defconfig
+++ b/arch/mips/configs/loongson3_defconfig
@@ -25,7 +25,6 @@
CONFIG_TASK_IO_ACCOUNTING=y
CONFIG_LOG_BUF_SHIFT=14
CONFIG_CPUSETS=y
-CONFIG_RESOURCE_COUNTERS=y
CONFIG_MEMCG=y
CONFIG_MEMCG_SWAP=y
CONFIG_BLK_CGROUP=y
diff --git a/arch/mips/configs/ls1b_defconfig b/arch/mips/configs/ls1b_defconfig
deleted file mode 100644
index 1b2cc1f..0000000
--- a/arch/mips/configs/ls1b_defconfig
+++ /dev/null
@@ -1,110 +0,0 @@
-CONFIG_MACH_LOONGSON32=y
-CONFIG_PREEMPT=y
-# CONFIG_SECCOMP is not set
-CONFIG_EXPERIMENTAL=y
-# CONFIG_LOCALVERSION_AUTO is not set
-CONFIG_SYSVIPC=y
-CONFIG_BSD_PROCESS_ACCT=y
-CONFIG_BSD_PROCESS_ACCT_V3=y
-CONFIG_HIGH_RES_TIMERS=y
-CONFIG_IKCONFIG=y
-CONFIG_IKCONFIG_PROC=y
-CONFIG_LOG_BUF_SHIFT=16
-CONFIG_NAMESPACES=y
-CONFIG_BLK_DEV_INITRD=y
-CONFIG_RD_BZIP2=y
-CONFIG_RD_LZMA=y
-CONFIG_EXPERT=y
-CONFIG_PERF_EVENTS=y
-# CONFIG_COMPAT_BRK is not set
-CONFIG_MODULES=y
-CONFIG_MODULE_UNLOAD=y
-CONFIG_MODVERSIONS=y
-# CONFIG_LBDAF is not set
-# CONFIG_BLK_DEV_BSG is not set
-# CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set
-# CONFIG_SUSPEND is not set
-CONFIG_NET=y
-CONFIG_PACKET=y
-CONFIG_UNIX=y
-CONFIG_INET=y
-CONFIG_IP_PNP=y
-CONFIG_IP_PNP_DHCP=y
-CONFIG_SYN_COOKIES=y
-# CONFIG_INET_XFRM_MODE_TRANSPORT is not set
-# CONFIG_INET_XFRM_MODE_TUNNEL is not set
-# CONFIG_INET_XFRM_MODE_BEET is not set
-# CONFIG_INET_DIAG is not set
-# CONFIG_IPV6 is not set
-# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
-CONFIG_DEVTMPFS=y
-CONFIG_DEVTMPFS_MOUNT=y
-# CONFIG_STANDALONE is not set
-CONFIG_BLK_DEV_LOOP=y
-CONFIG_SCSI=m
-# CONFIG_SCSI_PROC_FS is not set
-CONFIG_BLK_DEV_SD=m
-# CONFIG_SCSI_LOWLEVEL is not set
-CONFIG_NETDEVICES=y
-# CONFIG_NET_VENDOR_BROADCOM is not set
-# CONFIG_NET_VENDOR_CHELSIO is not set
-# CONFIG_NET_VENDOR_INTEL is not set
-# CONFIG_NET_VENDOR_MARVELL is not set
-# CONFIG_NET_VENDOR_MICREL is not set
-# CONFIG_NET_VENDOR_NATSEMI is not set
-# CONFIG_NET_VENDOR_SEEQ is not set
-# CONFIG_NET_VENDOR_SMSC is not set
-CONFIG_STMMAC_ETH=y
-CONFIG_STMMAC_DA=y
-# CONFIG_NET_VENDOR_WIZNET is not set
-# CONFIG_WLAN is not set
-CONFIG_INPUT_EVDEV=y
-# CONFIG_INPUT_KEYBOARD is not set
-# CONFIG_INPUT_MOUSE is not set
-# CONFIG_SERIO is not set
-CONFIG_VT_HW_CONSOLE_BINDING=y
-CONFIG_LEGACY_PTY_COUNT=8
-# CONFIG_DEVKMEM is not set
-CONFIG_SERIAL_8250=y
-CONFIG_SERIAL_8250_CONSOLE=y
-# CONFIG_HW_RANDOM is not set
-# CONFIG_HWMON is not set
-# CONFIG_VGA_CONSOLE is not set
-CONFIG_USB_HID=m
-CONFIG_HID_GENERIC=m
-CONFIG_USB=y
-CONFIG_USB_ANNOUNCE_NEW_DEVICES=y
-CONFIG_USB_EHCI_HCD=y
-CONFIG_USB_EHCI_HCD_PLATFORM=y
-# CONFIG_USB_EHCI_TT_NEWSCHED is not set
-CONFIG_USB_STORAGE=m
-CONFIG_USB_SERIAL=m
-CONFIG_USB_SERIAL_PL2303=m
-CONFIG_RTC_CLASS=y
-CONFIG_RTC_DRV_LOONGSON1=y
-# CONFIG_IOMMU_SUPPORT is not set
-CONFIG_EXT2_FS=y
-CONFIG_EXT2_FS_XATTR=y
-CONFIG_EXT2_FS_POSIX_ACL=y
-CONFIG_EXT2_FS_SECURITY=y
-CONFIG_EXT3_FS=y
-CONFIG_EXT3_FS_POSIX_ACL=y
-CONFIG_EXT3_FS_SECURITY=y
-# CONFIG_DNOTIFY is not set
-CONFIG_VFAT_FS=y
-CONFIG_PROC_KCORE=y
-CONFIG_TMPFS=y
-CONFIG_TMPFS_POSIX_ACL=y
-# CONFIG_MISC_FILESYSTEMS is not set
-CONFIG_NFS_FS=y
-CONFIG_ROOT_NFS=y
-CONFIG_NLS_CODEPAGE_437=m
-CONFIG_NLS_ISO8859_1=m
-# CONFIG_ENABLE_WARN_DEPRECATED is not set
-# CONFIG_ENABLE_MUST_CHECK is not set
-CONFIG_MAGIC_SYSRQ=y
-# CONFIG_SCHED_DEBUG is not set
-# CONFIG_DEBUG_PREEMPT is not set
-# CONFIG_FTRACE is not set
-# CONFIG_EARLY_PRINTK is not set
diff --git a/arch/mips/configs/mtx1_defconfig b/arch/mips/configs/mtx1_defconfig
index 9b6926d..f3f6005 100644
--- a/arch/mips/configs/mtx1_defconfig
+++ b/arch/mips/configs/mtx1_defconfig
@@ -51,7 +51,6 @@
CONFIG_INET_XFRM_MODE_TRANSPORT=m
CONFIG_INET_XFRM_MODE_TUNNEL=m
CONFIG_INET_XFRM_MODE_BEET=m
-CONFIG_IPV6_PRIVACY=y
CONFIG_INET6_AH=m
CONFIG_INET6_ESP=m
CONFIG_INET6_IPCOMP=m
diff --git a/arch/mips/configs/nlm_xlp_defconfig b/arch/mips/configs/nlm_xlp_defconfig
index b3d1d37f..b496c25 100644
--- a/arch/mips/configs/nlm_xlp_defconfig
+++ b/arch/mips/configs/nlm_xlp_defconfig
@@ -95,7 +95,6 @@
CONFIG_TCP_CONG_ILLINOIS=m
CONFIG_TCP_MD5SIG=y
CONFIG_IPV6=y
-CONFIG_IPV6_PRIVACY=y
CONFIG_INET6_AH=m
CONFIG_INET6_ESP=m
CONFIG_INET6_IPCOMP=m
diff --git a/arch/mips/configs/nlm_xlr_defconfig b/arch/mips/configs/nlm_xlr_defconfig
index 3d8016d..8e99ad8 100644
--- a/arch/mips/configs/nlm_xlr_defconfig
+++ b/arch/mips/configs/nlm_xlr_defconfig
@@ -75,7 +75,6 @@
CONFIG_TCP_CONG_ILLINOIS=m
CONFIG_TCP_MD5SIG=y
CONFIG_IPV6=y
-CONFIG_IPV6_PRIVACY=y
CONFIG_INET6_AH=m
CONFIG_INET6_ESP=m
CONFIG_INET6_IPCOMP=m
diff --git a/arch/mips/configs/rm200_defconfig b/arch/mips/configs/rm200_defconfig
index 82db4e3..c2b4e3f 100644
--- a/arch/mips/configs/rm200_defconfig
+++ b/arch/mips/configs/rm200_defconfig
@@ -37,7 +37,6 @@
CONFIG_INET_XFRM_MODE_TUNNEL=m
CONFIG_INET_XFRM_MODE_BEET=m
CONFIG_TCP_MD5SIG=y
-CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_IPV6_ROUTE_INFO=y
CONFIG_INET6_AH=m
diff --git a/arch/mips/dec/setup.c b/arch/mips/dec/setup.c
index a0b8943..1c3bf9f 100644
--- a/arch/mips/dec/setup.c
+++ b/arch/mips/dec/setup.c
@@ -60,6 +60,7 @@
int dec_tc_bus;
DEFINE_SPINLOCK(ioasic_ssr_lock);
+EXPORT_SYMBOL(ioasic_ssr_lock);
volatile u32 *ioasic_base;
diff --git a/arch/mips/include/asm/Kbuild b/arch/mips/include/asm/Kbuild
index c7fe4d0..9740066 100644
--- a/arch/mips/include/asm/Kbuild
+++ b/arch/mips/include/asm/Kbuild
@@ -1,5 +1,6 @@
# MIPS headers
generic-(CONFIG_GENERIC_CSUM) += checksum.h
+generic-y += clkdev.h
generic-y += cputime.h
generic-y += current.h
generic-y += dma-contiguous.h
diff --git a/arch/mips/include/asm/asmmacro.h b/arch/mips/include/asm/asmmacro.h
index 867f924..6741673 100644
--- a/arch/mips/include/asm/asmmacro.h
+++ b/arch/mips/include/asm/asmmacro.h
@@ -235,6 +235,7 @@
.macro ld_b wd, off, base
.set push
.set mips32r2
+ .set fp=64
.set msa
ld.b $w\wd, \off(\base)
.set pop
@@ -243,6 +244,7 @@
.macro ld_h wd, off, base
.set push
.set mips32r2
+ .set fp=64
.set msa
ld.h $w\wd, \off(\base)
.set pop
@@ -251,6 +253,7 @@
.macro ld_w wd, off, base
.set push
.set mips32r2
+ .set fp=64
.set msa
ld.w $w\wd, \off(\base)
.set pop
@@ -268,6 +271,7 @@
.macro st_b wd, off, base
.set push
.set mips32r2
+ .set fp=64
.set msa
st.b $w\wd, \off(\base)
.set pop
@@ -276,6 +280,7 @@
.macro st_h wd, off, base
.set push
.set mips32r2
+ .set fp=64
.set msa
st.h $w\wd, \off(\base)
.set pop
@@ -284,6 +289,7 @@
.macro st_w wd, off, base
.set push
.set mips32r2
+ .set fp=64
.set msa
st.w $w\wd, \off(\base)
.set pop
@@ -298,21 +304,21 @@
.set pop
.endm
- .macro copy_u_w ws, n
+ .macro copy_s_w ws, n
.set push
.set mips32r2
.set fp=64
.set msa
- copy_u.w $1, $w\ws[\n]
+ copy_s.w $1, $w\ws[\n]
.set pop
.endm
- .macro copy_u_d ws, n
+ .macro copy_s_d ws, n
.set push
.set mips64r2
.set fp=64
.set msa
- copy_u.d $1, $w\ws[\n]
+ copy_s.d $1, $w\ws[\n]
.set pop
.endm
@@ -346,8 +352,8 @@
#define STH_MSA_INSN 0x5800081f
#define STW_MSA_INSN 0x5800082f
#define STD_MSA_INSN 0x5800083f
-#define COPY_UW_MSA_INSN 0x58f00056
-#define COPY_UD_MSA_INSN 0x58f80056
+#define COPY_SW_MSA_INSN 0x58b00056
+#define COPY_SD_MSA_INSN 0x58b80056
#define INSERT_W_MSA_INSN 0x59300816
#define INSERT_D_MSA_INSN 0x59380816
#else
@@ -361,8 +367,8 @@
#define STH_MSA_INSN 0x78000825
#define STW_MSA_INSN 0x78000826
#define STD_MSA_INSN 0x78000827
-#define COPY_UW_MSA_INSN 0x78f00059
-#define COPY_UD_MSA_INSN 0x78f80059
+#define COPY_SW_MSA_INSN 0x78b00059
+#define COPY_SD_MSA_INSN 0x78b80059
#define INSERT_W_MSA_INSN 0x79300819
#define INSERT_D_MSA_INSN 0x79380819
#endif
@@ -393,7 +399,7 @@
.set push
.set noat
SET_HARDFLOAT
- addu $1, \base, \off
+ PTR_ADDU $1, \base, \off
.word LDB_MSA_INSN | (\wd << 6)
.set pop
.endm
@@ -402,7 +408,7 @@
.set push
.set noat
SET_HARDFLOAT
- addu $1, \base, \off
+ PTR_ADDU $1, \base, \off
.word LDH_MSA_INSN | (\wd << 6)
.set pop
.endm
@@ -411,7 +417,7 @@
.set push
.set noat
SET_HARDFLOAT
- addu $1, \base, \off
+ PTR_ADDU $1, \base, \off
.word LDW_MSA_INSN | (\wd << 6)
.set pop
.endm
@@ -420,7 +426,7 @@
.set push
.set noat
SET_HARDFLOAT
- addu $1, \base, \off
+ PTR_ADDU $1, \base, \off
.word LDD_MSA_INSN | (\wd << 6)
.set pop
.endm
@@ -429,7 +435,7 @@
.set push
.set noat
SET_HARDFLOAT
- addu $1, \base, \off
+ PTR_ADDU $1, \base, \off
.word STB_MSA_INSN | (\wd << 6)
.set pop
.endm
@@ -438,7 +444,7 @@
.set push
.set noat
SET_HARDFLOAT
- addu $1, \base, \off
+ PTR_ADDU $1, \base, \off
.word STH_MSA_INSN | (\wd << 6)
.set pop
.endm
@@ -447,7 +453,7 @@
.set push
.set noat
SET_HARDFLOAT
- addu $1, \base, \off
+ PTR_ADDU $1, \base, \off
.word STW_MSA_INSN | (\wd << 6)
.set pop
.endm
@@ -456,26 +462,26 @@
.set push
.set noat
SET_HARDFLOAT
- addu $1, \base, \off
+ PTR_ADDU $1, \base, \off
.word STD_MSA_INSN | (\wd << 6)
.set pop
.endm
- .macro copy_u_w ws, n
+ .macro copy_s_w ws, n
.set push
.set noat
SET_HARDFLOAT
.insn
- .word COPY_UW_MSA_INSN | (\n << 16) | (\ws << 11)
+ .word COPY_SW_MSA_INSN | (\n << 16) | (\ws << 11)
.set pop
.endm
- .macro copy_u_d ws, n
+ .macro copy_s_d ws, n
.set push
.set noat
SET_HARDFLOAT
.insn
- .word COPY_UD_MSA_INSN | (\n << 16) | (\ws << 11)
+ .word COPY_SD_MSA_INSN | (\n << 16) | (\ws << 11)
.set pop
.endm
@@ -496,41 +502,52 @@
.endm
#endif
+#ifdef TOOLCHAIN_SUPPORTS_MSA
+#define FPR_BASE_OFFS THREAD_FPR0
+#define FPR_BASE $1
+#else
+#define FPR_BASE_OFFS 0
+#define FPR_BASE \thread
+#endif
+
.macro msa_save_all thread
- st_d 0, THREAD_FPR0, \thread
- st_d 1, THREAD_FPR1, \thread
- st_d 2, THREAD_FPR2, \thread
- st_d 3, THREAD_FPR3, \thread
- st_d 4, THREAD_FPR4, \thread
- st_d 5, THREAD_FPR5, \thread
- st_d 6, THREAD_FPR6, \thread
- st_d 7, THREAD_FPR7, \thread
- st_d 8, THREAD_FPR8, \thread
- st_d 9, THREAD_FPR9, \thread
- st_d 10, THREAD_FPR10, \thread
- st_d 11, THREAD_FPR11, \thread
- st_d 12, THREAD_FPR12, \thread
- st_d 13, THREAD_FPR13, \thread
- st_d 14, THREAD_FPR14, \thread
- st_d 15, THREAD_FPR15, \thread
- st_d 16, THREAD_FPR16, \thread
- st_d 17, THREAD_FPR17, \thread
- st_d 18, THREAD_FPR18, \thread
- st_d 19, THREAD_FPR19, \thread
- st_d 20, THREAD_FPR20, \thread
- st_d 21, THREAD_FPR21, \thread
- st_d 22, THREAD_FPR22, \thread
- st_d 23, THREAD_FPR23, \thread
- st_d 24, THREAD_FPR24, \thread
- st_d 25, THREAD_FPR25, \thread
- st_d 26, THREAD_FPR26, \thread
- st_d 27, THREAD_FPR27, \thread
- st_d 28, THREAD_FPR28, \thread
- st_d 29, THREAD_FPR29, \thread
- st_d 30, THREAD_FPR30, \thread
- st_d 31, THREAD_FPR31, \thread
.set push
.set noat
+#ifdef TOOLCHAIN_SUPPORTS_MSA
+ PTR_ADDU FPR_BASE, \thread, FPR_BASE_OFFS
+#endif
+ st_d 0, THREAD_FPR0 - FPR_BASE_OFFS, FPR_BASE
+ st_d 1, THREAD_FPR1 - FPR_BASE_OFFS, FPR_BASE
+ st_d 2, THREAD_FPR2 - FPR_BASE_OFFS, FPR_BASE
+ st_d 3, THREAD_FPR3 - FPR_BASE_OFFS, FPR_BASE
+ st_d 4, THREAD_FPR4 - FPR_BASE_OFFS, FPR_BASE
+ st_d 5, THREAD_FPR5 - FPR_BASE_OFFS, FPR_BASE
+ st_d 6, THREAD_FPR6 - FPR_BASE_OFFS, FPR_BASE
+ st_d 7, THREAD_FPR7 - FPR_BASE_OFFS, FPR_BASE
+ st_d 8, THREAD_FPR8 - FPR_BASE_OFFS, FPR_BASE
+ st_d 9, THREAD_FPR9 - FPR_BASE_OFFS, FPR_BASE
+ st_d 10, THREAD_FPR10 - FPR_BASE_OFFS, FPR_BASE
+ st_d 11, THREAD_FPR11 - FPR_BASE_OFFS, FPR_BASE
+ st_d 12, THREAD_FPR12 - FPR_BASE_OFFS, FPR_BASE
+ st_d 13, THREAD_FPR13 - FPR_BASE_OFFS, FPR_BASE
+ st_d 14, THREAD_FPR14 - FPR_BASE_OFFS, FPR_BASE
+ st_d 15, THREAD_FPR15 - FPR_BASE_OFFS, FPR_BASE
+ st_d 16, THREAD_FPR16 - FPR_BASE_OFFS, FPR_BASE
+ st_d 17, THREAD_FPR17 - FPR_BASE_OFFS, FPR_BASE
+ st_d 18, THREAD_FPR18 - FPR_BASE_OFFS, FPR_BASE
+ st_d 19, THREAD_FPR19 - FPR_BASE_OFFS, FPR_BASE
+ st_d 20, THREAD_FPR20 - FPR_BASE_OFFS, FPR_BASE
+ st_d 21, THREAD_FPR21 - FPR_BASE_OFFS, FPR_BASE
+ st_d 22, THREAD_FPR22 - FPR_BASE_OFFS, FPR_BASE
+ st_d 23, THREAD_FPR23 - FPR_BASE_OFFS, FPR_BASE
+ st_d 24, THREAD_FPR24 - FPR_BASE_OFFS, FPR_BASE
+ st_d 25, THREAD_FPR25 - FPR_BASE_OFFS, FPR_BASE
+ st_d 26, THREAD_FPR26 - FPR_BASE_OFFS, FPR_BASE
+ st_d 27, THREAD_FPR27 - FPR_BASE_OFFS, FPR_BASE
+ st_d 28, THREAD_FPR28 - FPR_BASE_OFFS, FPR_BASE
+ st_d 29, THREAD_FPR29 - FPR_BASE_OFFS, FPR_BASE
+ st_d 30, THREAD_FPR30 - FPR_BASE_OFFS, FPR_BASE
+ st_d 31, THREAD_FPR31 - FPR_BASE_OFFS, FPR_BASE
SET_HARDFLOAT
_cfcmsa $1, MSA_CSR
sw $1, THREAD_MSA_CSR(\thread)
@@ -543,41 +560,47 @@
SET_HARDFLOAT
lw $1, THREAD_MSA_CSR(\thread)
_ctcmsa MSA_CSR, $1
- .set pop
- ld_d 0, THREAD_FPR0, \thread
- ld_d 1, THREAD_FPR1, \thread
- ld_d 2, THREAD_FPR2, \thread
- ld_d 3, THREAD_FPR3, \thread
- ld_d 4, THREAD_FPR4, \thread
- ld_d 5, THREAD_FPR5, \thread
- ld_d 6, THREAD_FPR6, \thread
- ld_d 7, THREAD_FPR7, \thread
- ld_d 8, THREAD_FPR8, \thread
- ld_d 9, THREAD_FPR9, \thread
- ld_d 10, THREAD_FPR10, \thread
- ld_d 11, THREAD_FPR11, \thread
- ld_d 12, THREAD_FPR12, \thread
- ld_d 13, THREAD_FPR13, \thread
- ld_d 14, THREAD_FPR14, \thread
- ld_d 15, THREAD_FPR15, \thread
- ld_d 16, THREAD_FPR16, \thread
- ld_d 17, THREAD_FPR17, \thread
- ld_d 18, THREAD_FPR18, \thread
- ld_d 19, THREAD_FPR19, \thread
- ld_d 20, THREAD_FPR20, \thread
- ld_d 21, THREAD_FPR21, \thread
- ld_d 22, THREAD_FPR22, \thread
- ld_d 23, THREAD_FPR23, \thread
- ld_d 24, THREAD_FPR24, \thread
- ld_d 25, THREAD_FPR25, \thread
- ld_d 26, THREAD_FPR26, \thread
- ld_d 27, THREAD_FPR27, \thread
- ld_d 28, THREAD_FPR28, \thread
- ld_d 29, THREAD_FPR29, \thread
- ld_d 30, THREAD_FPR30, \thread
- ld_d 31, THREAD_FPR31, \thread
+#ifdef TOOLCHAIN_SUPPORTS_MSA
+ PTR_ADDU FPR_BASE, \thread, FPR_BASE_OFFS
+#endif
+ ld_d 0, THREAD_FPR0 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 1, THREAD_FPR1 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 2, THREAD_FPR2 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 3, THREAD_FPR3 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 4, THREAD_FPR4 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 5, THREAD_FPR5 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 6, THREAD_FPR6 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 7, THREAD_FPR7 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 8, THREAD_FPR8 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 9, THREAD_FPR9 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 10, THREAD_FPR10 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 11, THREAD_FPR11 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 12, THREAD_FPR12 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 13, THREAD_FPR13 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 14, THREAD_FPR14 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 15, THREAD_FPR15 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 16, THREAD_FPR16 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 17, THREAD_FPR17 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 18, THREAD_FPR18 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 19, THREAD_FPR19 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 20, THREAD_FPR20 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 21, THREAD_FPR21 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 22, THREAD_FPR22 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 23, THREAD_FPR23 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 24, THREAD_FPR24 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 25, THREAD_FPR25 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 26, THREAD_FPR26 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 27, THREAD_FPR27 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 28, THREAD_FPR28 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 29, THREAD_FPR29 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 30, THREAD_FPR30 - FPR_BASE_OFFS, FPR_BASE
+ ld_d 31, THREAD_FPR31 - FPR_BASE_OFFS, FPR_BASE
+ .set pop
.endm
+#undef FPR_BASE_OFFS
+#undef FPR_BASE
+
.macro msa_init_upper wd
#ifdef CONFIG_64BIT
insert_d \wd, 1
diff --git a/arch/mips/include/asm/bitops.h b/arch/mips/include/asm/bitops.h
index ce9666c..fa57cef 100644
--- a/arch/mips/include/asm/bitops.h
+++ b/arch/mips/include/asm/bitops.h
@@ -19,25 +19,10 @@
#include <asm/byteorder.h> /* sigh ... */
#include <asm/compiler.h>
#include <asm/cpu-features.h>
+#include <asm/llsc.h>
#include <asm/sgidefs.h>
#include <asm/war.h>
-#if _MIPS_SZLONG == 32
-#define SZLONG_LOG 5
-#define SZLONG_MASK 31UL
-#define __LL "ll "
-#define __SC "sc "
-#define __INS "ins "
-#define __EXT "ext "
-#elif _MIPS_SZLONG == 64
-#define SZLONG_LOG 6
-#define SZLONG_MASK 63UL
-#define __LL "lld "
-#define __SC "scd "
-#define __INS "dins "
-#define __EXT "dext "
-#endif
-
/*
* These are the "slower" versions of the functions and are in bitops.c.
* These functions call raw_local_irq_{save,restore}().
diff --git a/arch/mips/include/asm/bitrev.h b/arch/mips/include/asm/bitrev.h
new file mode 100644
index 0000000..bc739a4
--- /dev/null
+++ b/arch/mips/include/asm/bitrev.h
@@ -0,0 +1,30 @@
+#ifndef __MIPS_ASM_BITREV_H__
+#define __MIPS_ASM_BITREV_H__
+
+#include <linux/swab.h>
+
+static __always_inline __attribute_const__ u32 __arch_bitrev32(u32 x)
+{
+ u32 ret;
+
+ asm("bitswap %0, %1" : "=r"(ret) : "r"(__swab32(x)));
+ return ret;
+}
+
+static __always_inline __attribute_const__ u16 __arch_bitrev16(u16 x)
+{
+ u16 ret;
+
+ asm("bitswap %0, %1" : "=r"(ret) : "r"(__swab16(x)));
+ return ret;
+}
+
+static __always_inline __attribute_const__ u8 __arch_bitrev8(u8 x)
+{
+ u8 ret;
+
+ asm("bitswap %0, %1" : "=r"(ret) : "r"(x));
+ return ret;
+}
+
+#endif /* __MIPS_ASM_BITREV_H__ */
diff --git a/arch/mips/include/asm/bmips.h b/arch/mips/include/asm/bmips.h
index 6d25ad3..a92aee7 100644
--- a/arch/mips/include/asm/bmips.h
+++ b/arch/mips/include/asm/bmips.h
@@ -88,6 +88,7 @@
extern void bmips_ebase_setup(void);
extern asmlinkage void plat_wired_tlb_setup(void);
+extern void bmips_cpu_setup(void);
static inline unsigned long bmips_read_zscm_reg(unsigned int offset)
{
diff --git a/arch/mips/include/asm/bootinfo.h b/arch/mips/include/asm/bootinfo.h
index b603804..9f67033 100644
--- a/arch/mips/include/asm/bootinfo.h
+++ b/arch/mips/include/asm/bootinfo.h
@@ -144,4 +144,22 @@
#endif /* CONFIG_SWIOTLB */
+#ifdef CONFIG_USE_OF
+/**
+ * plat_get_fdt() - Return a pointer to the platform's device tree blob
+ *
+ * This function provides a platform independent API to get a pointer to the
+ * flattened device tree blob. The interface between bootloader and kernel
+ * is not consistent across platforms so it is necessary to provide this
+ * API such that common startup code can locate the FDT.
+ *
+ * This is used by the KASLR code to get command line arguments and random
+ * seed from the device tree. Any platform wishing to use KASLR should
+ * provide this API and select SYS_SUPPORTS_RELOCATABLE.
+ *
+ * Return: Pointer to the flattened device tree blob.
+ */
+extern void *plat_get_fdt(void);
+#endif /* CONFIG_USE_OF */
+
#endif /* _ASM_BOOTINFO_H */
diff --git a/arch/mips/include/asm/cacheflush.h b/arch/mips/include/asm/cacheflush.h
index 723229f..34ed22e 100644
--- a/arch/mips/include/asm/cacheflush.h
+++ b/arch/mips/include/asm/cacheflush.h
@@ -51,7 +51,6 @@
unsigned long start, unsigned long end);
extern void (*flush_cache_page)(struct vm_area_struct *vma, unsigned long page, unsigned long pfn);
extern void __flush_dcache_page(struct page *page);
-extern void __flush_icache_page(struct vm_area_struct *vma, struct page *page);
#define ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE 1
static inline void flush_dcache_page(struct page *page)
@@ -77,11 +76,6 @@
static inline void flush_icache_page(struct vm_area_struct *vma,
struct page *page)
{
- if (!cpu_has_ic_fills_f_dc && (vma->vm_flags & VM_EXEC) &&
- Page_dcache_dirty(page)) {
- __flush_icache_page(vma, page);
- ClearPageDcacheDirty(page);
- }
}
extern void (*flush_icache_range)(unsigned long start, unsigned long end);
@@ -132,6 +126,7 @@
static inline void flush_kernel_dcache_page(struct page *page)
{
BUG_ON(cpu_has_dc_aliases && PageHighMem(page));
+ flush_dcache_page(page);
}
/*
diff --git a/arch/mips/include/asm/cacheops.h b/arch/mips/include/asm/cacheops.h
index c3212ff..8031fbc 100644
--- a/arch/mips/include/asm/cacheops.h
+++ b/arch/mips/include/asm/cacheops.h
@@ -21,6 +21,7 @@
#define Cache_I 0x00
#define Cache_D 0x01
#define Cache_T 0x02
+#define Cache_V 0x02 /* Loongson-3 */
#define Cache_S 0x03
#define Index_Writeback_Inv 0x00
@@ -107,4 +108,9 @@
*/
#define Hit_Invalidate_I_Loongson2 (Cache_I | 0x00)
+/*
+ * Loongson3-specific cacheops
+ */
+#define Index_Writeback_Inv_V (Cache_V | Index_Writeback_Inv)
+
#endif /* __ASM_CACHEOPS_H */
diff --git a/arch/mips/include/asm/clkdev.h b/arch/mips/include/asm/clkdev.h
deleted file mode 100644
index 1b3ad7b..0000000
--- a/arch/mips/include/asm/clkdev.h
+++ /dev/null
@@ -1,27 +0,0 @@
-/*
- * based on arch/arm/include/asm/clkdev.h
- *
- * Copyright (C) 2008 Russell King.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * Helper for the clk API to assist looking up a struct clk.
- */
-#ifndef __ASM_CLKDEV_H
-#define __ASM_CLKDEV_H
-
-#include <linux/slab.h>
-
-#ifndef CONFIG_COMMON_CLK
-#define __clk_get(clk) ({ 1; })
-#define __clk_put(clk) do { } while (0)
-#endif
-
-static inline struct clk_lookup_alloc *__clkdev_alloc(size_t size)
-{
- return kzalloc(size, GFP_KERNEL);
-}
-
-#endif
diff --git a/arch/mips/include/asm/cpu-features.h b/arch/mips/include/asm/cpu-features.h
index eeec8c8..e961c8a 100644
--- a/arch/mips/include/asm/cpu-features.h
+++ b/arch/mips/include/asm/cpu-features.h
@@ -35,6 +35,9 @@
#ifndef cpu_has_htw
#define cpu_has_htw (cpu_data[0].options & MIPS_CPU_HTW)
#endif
+#ifndef cpu_has_ldpte
+#define cpu_has_ldpte (cpu_data[0].options & MIPS_CPU_LDPTE)
+#endif
#ifndef cpu_has_rixiex
#define cpu_has_rixiex (cpu_data[0].options & MIPS_CPU_RIXIEX)
#endif
@@ -117,6 +120,21 @@
#ifndef kernel_uses_llsc
#define kernel_uses_llsc cpu_has_llsc
#endif
+#ifndef cpu_has_guestctl0ext
+#define cpu_has_guestctl0ext (cpu_data[0].options & MIPS_CPU_GUESTCTL0EXT)
+#endif
+#ifndef cpu_has_guestctl1
+#define cpu_has_guestctl1 (cpu_data[0].options & MIPS_CPU_GUESTCTL1)
+#endif
+#ifndef cpu_has_guestctl2
+#define cpu_has_guestctl2 (cpu_data[0].options & MIPS_CPU_GUESTCTL2)
+#endif
+#ifndef cpu_has_guestid
+#define cpu_has_guestid (cpu_data[0].options & MIPS_CPU_GUESTID)
+#endif
+#ifndef cpu_has_drg
+#define cpu_has_drg (cpu_data[0].options & MIPS_CPU_DRG)
+#endif
#ifndef cpu_has_mips16
#define cpu_has_mips16 (cpu_data[0].ases & MIPS_ASE_MIPS16)
#endif
@@ -142,8 +160,14 @@
# endif
#endif
+#ifndef cpu_has_lpa
+#define cpu_has_lpa (cpu_data[0].options & MIPS_CPU_LPA)
+#endif
+#ifndef cpu_has_mvh
+#define cpu_has_mvh (cpu_data[0].options & MIPS_CPU_MVH)
+#endif
#ifndef cpu_has_xpa
-#define cpu_has_xpa (cpu_data[0].options & MIPS_CPU_XPA)
+#define cpu_has_xpa (cpu_has_lpa && cpu_has_mvh)
#endif
#ifndef cpu_has_vtag_icache
#define cpu_has_vtag_icache (cpu_data[0].icache.flags & MIPS_CACHE_VTAG)
@@ -180,6 +204,16 @@
#endif
#endif
+/* __builtin_constant_p(cpu_has_mips_r) && cpu_has_mips_r */
+#if !((defined(cpu_has_mips32r1) && cpu_has_mips32r1) || \
+ (defined(cpu_has_mips32r2) && cpu_has_mips32r2) || \
+ (defined(cpu_has_mips32r6) && cpu_has_mips32r6) || \
+ (defined(cpu_has_mips64r1) && cpu_has_mips64r1) || \
+ (defined(cpu_has_mips64r2) && cpu_has_mips64r2) || \
+ (defined(cpu_has_mips64r6) && cpu_has_mips64r6))
+#define CPU_NO_EFFICIENT_FFS 1
+#endif
+
#ifndef cpu_has_mips_1
# define cpu_has_mips_1 (!cpu_has_mips_r6)
#endif
@@ -307,10 +341,18 @@
#define cpu_has_dsp2 (cpu_data[0].ases & MIPS_ASE_DSP2P)
#endif
+#ifndef cpu_has_dsp3
+#define cpu_has_dsp3 (cpu_data[0].ases & MIPS_ASE_DSP3)
+#endif
+
#ifndef cpu_has_mipsmt
#define cpu_has_mipsmt (cpu_data[0].ases & MIPS_ASE_MIPSMT)
#endif
+#ifndef cpu_has_vp
+#define cpu_has_vp (cpu_data[0].options & MIPS_CPU_VP)
+#endif
+
#ifndef cpu_has_userlocal
#define cpu_has_userlocal (cpu_data[0].options & MIPS_CPU_ULRI)
#endif
@@ -421,4 +463,107 @@
#define cpu_has_nan_2008 (cpu_data[0].options & MIPS_CPU_NAN_2008)
#endif
+#ifndef cpu_has_ebase_wg
+# define cpu_has_ebase_wg (cpu_data[0].options & MIPS_CPU_EBASE_WG)
+#endif
+
+#ifndef cpu_has_badinstr
+# define cpu_has_badinstr (cpu_data[0].options & MIPS_CPU_BADINSTR)
+#endif
+
+#ifndef cpu_has_badinstrp
+# define cpu_has_badinstrp (cpu_data[0].options & MIPS_CPU_BADINSTRP)
+#endif
+
+#ifndef cpu_has_contextconfig
+# define cpu_has_contextconfig (cpu_data[0].options & MIPS_CPU_CTXTC)
+#endif
+
+#ifndef cpu_has_perf
+# define cpu_has_perf (cpu_data[0].options & MIPS_CPU_PERF)
+#endif
+
+/*
+ * Guest capabilities
+ */
+#ifndef cpu_guest_has_conf1
+#define cpu_guest_has_conf1 (cpu_data[0].guest.conf & (1 << 1))
+#endif
+#ifndef cpu_guest_has_conf2
+#define cpu_guest_has_conf2 (cpu_data[0].guest.conf & (1 << 2))
+#endif
+#ifndef cpu_guest_has_conf3
+#define cpu_guest_has_conf3 (cpu_data[0].guest.conf & (1 << 3))
+#endif
+#ifndef cpu_guest_has_conf4
+#define cpu_guest_has_conf4 (cpu_data[0].guest.conf & (1 << 4))
+#endif
+#ifndef cpu_guest_has_conf5
+#define cpu_guest_has_conf5 (cpu_data[0].guest.conf & (1 << 5))
+#endif
+#ifndef cpu_guest_has_conf6
+#define cpu_guest_has_conf6 (cpu_data[0].guest.conf & (1 << 6))
+#endif
+#ifndef cpu_guest_has_conf7
+#define cpu_guest_has_conf7 (cpu_data[0].guest.conf & (1 << 7))
+#endif
+#ifndef cpu_guest_has_fpu
+#define cpu_guest_has_fpu (cpu_data[0].guest.options & MIPS_CPU_FPU)
+#endif
+#ifndef cpu_guest_has_watch
+#define cpu_guest_has_watch (cpu_data[0].guest.options & MIPS_CPU_WATCH)
+#endif
+#ifndef cpu_guest_has_contextconfig
+#define cpu_guest_has_contextconfig (cpu_data[0].guest.options & MIPS_CPU_CTXTC)
+#endif
+#ifndef cpu_guest_has_segments
+#define cpu_guest_has_segments (cpu_data[0].guest.options & MIPS_CPU_SEGMENTS)
+#endif
+#ifndef cpu_guest_has_badinstr
+#define cpu_guest_has_badinstr (cpu_data[0].guest.options & MIPS_CPU_BADINSTR)
+#endif
+#ifndef cpu_guest_has_badinstrp
+#define cpu_guest_has_badinstrp (cpu_data[0].guest.options & MIPS_CPU_BADINSTRP)
+#endif
+#ifndef cpu_guest_has_htw
+#define cpu_guest_has_htw (cpu_data[0].guest.options & MIPS_CPU_HTW)
+#endif
+#ifndef cpu_guest_has_msa
+#define cpu_guest_has_msa (cpu_data[0].guest.ases & MIPS_ASE_MSA)
+#endif
+#ifndef cpu_guest_has_kscr
+#define cpu_guest_has_kscr(n) (cpu_data[0].guest.kscratch_mask & (1u << (n)))
+#endif
+#ifndef cpu_guest_has_rw_llb
+#define cpu_guest_has_rw_llb (cpu_has_mips_r6 || (cpu_data[0].guest.options & MIPS_CPU_RW_LLB))
+#endif
+#ifndef cpu_guest_has_perf
+#define cpu_guest_has_perf (cpu_data[0].guest.options & MIPS_CPU_PERF)
+#endif
+#ifndef cpu_guest_has_maar
+#define cpu_guest_has_maar (cpu_data[0].guest.options & MIPS_CPU_MAAR)
+#endif
+
+/*
+ * Guest dynamic capabilities
+ */
+#ifndef cpu_guest_has_dyn_fpu
+#define cpu_guest_has_dyn_fpu (cpu_data[0].guest.options_dyn & MIPS_CPU_FPU)
+#endif
+#ifndef cpu_guest_has_dyn_watch
+#define cpu_guest_has_dyn_watch (cpu_data[0].guest.options_dyn & MIPS_CPU_WATCH)
+#endif
+#ifndef cpu_guest_has_dyn_contextconfig
+#define cpu_guest_has_dyn_contextconfig (cpu_data[0].guest.options_dyn & MIPS_CPU_CTXTC)
+#endif
+#ifndef cpu_guest_has_dyn_perf
+#define cpu_guest_has_dyn_perf (cpu_data[0].guest.options_dyn & MIPS_CPU_PERF)
+#endif
+#ifndef cpu_guest_has_dyn_msa
+#define cpu_guest_has_dyn_msa (cpu_data[0].guest.ases_dyn & MIPS_ASE_MSA)
+#endif
+#ifndef cpu_guest_has_dyn_maar
+#define cpu_guest_has_dyn_maar (cpu_data[0].guest.options_dyn & MIPS_CPU_MAAR)
+#endif
+
#endif /* __ASM_CPU_FEATURES_H */
diff --git a/arch/mips/include/asm/cpu-info.h b/arch/mips/include/asm/cpu-info.h
index af12c1f..edbe273 100644
--- a/arch/mips/include/asm/cpu-info.h
+++ b/arch/mips/include/asm/cpu-info.h
@@ -28,6 +28,15 @@
unsigned char flags; /* Flags describing cache properties */
};
+struct guest_info {
+ unsigned long ases;
+ unsigned long ases_dyn;
+ unsigned long long options;
+ unsigned long long options_dyn;
+ u8 conf;
+ u8 kscratch_mask;
+};
+
/*
* Flag definitions
*/
@@ -40,6 +49,9 @@
struct cpuinfo_mips {
unsigned long asid_cache;
+#ifdef CONFIG_MIPS_ASID_BITS_VARIABLE
+ unsigned long asid_mask;
+#endif
/*
* Capability and feature descriptor structure for MIPS CPU
@@ -60,6 +72,7 @@
int tlbsizeftlbways;
struct cache_desc icache; /* Primary I-cache */
struct cache_desc dcache; /* Primary D or combined I/D cache */
+ struct cache_desc vcache; /* Victim cache, between pcache and scache */
struct cache_desc scache; /* Secondary cache */
struct cache_desc tcache; /* Tertiary/split secondary cache */
int srsets; /* Shadow register sets */
@@ -68,7 +81,7 @@
#ifdef CONFIG_64BIT
int vmbits; /* Virtual memory size in bits */
#endif
-#ifdef CONFIG_MIPS_MT_SMP
+#if defined(CONFIG_MIPS_MT_SMP) || defined(CONFIG_CPU_MIPSR6)
/*
* There is not necessarily a 1:1 mapping of VPE num to CPU number
* in particular on multi-core systems.
@@ -91,6 +104,11 @@
* htw_start/htw_stop calls
*/
unsigned int htw_seq;
+
+ /* VZ & Guest features */
+ struct guest_info guest;
+ unsigned int gtoffset_mask;
+ unsigned int guestid_mask;
} __attribute__((aligned(SMP_CACHE_BYTES)));
extern struct cpuinfo_mips cpu_data[];
@@ -125,10 +143,31 @@
unsigned long n;
};
-#ifdef CONFIG_MIPS_MT_SMP
+#if defined(CONFIG_MIPS_MT_SMP) || defined(CONFIG_CPU_MIPSR6)
# define cpu_vpe_id(cpuinfo) ((cpuinfo)->vpe_id)
#else
# define cpu_vpe_id(cpuinfo) ({ (void)cpuinfo; 0; })
#endif
+static inline unsigned long cpu_asid_inc(void)
+{
+ return 1 << CONFIG_MIPS_ASID_SHIFT;
+}
+
+static inline unsigned long cpu_asid_mask(struct cpuinfo_mips *cpuinfo)
+{
+#ifdef CONFIG_MIPS_ASID_BITS_VARIABLE
+ return cpuinfo->asid_mask;
+#endif
+ return ((1 << CONFIG_MIPS_ASID_BITS) - 1) << CONFIG_MIPS_ASID_SHIFT;
+}
+
+static inline void set_cpu_asid_mask(struct cpuinfo_mips *cpuinfo,
+ unsigned long asid_mask)
+{
+#ifdef CONFIG_MIPS_ASID_BITS_VARIABLE
+ cpuinfo->asid_mask = asid_mask;
+#endif
+}
+
#endif /* __ASM_CPU_INFO_H */
diff --git a/arch/mips/include/asm/cpu-type.h b/arch/mips/include/asm/cpu-type.h
index abee2bf..fbe1881 100644
--- a/arch/mips/include/asm/cpu-type.h
+++ b/arch/mips/include/asm/cpu-type.h
@@ -77,8 +77,13 @@
*/
#endif
+#ifdef CONFIG_SYS_HAS_CPU_MIPS32_R6
+ case CPU_M6250:
+#endif
+
#ifdef CONFIG_SYS_HAS_CPU_MIPS64_R6
case CPU_I6400:
+ case CPU_P6600:
#endif
#ifdef CONFIG_SYS_HAS_CPU_R3000
diff --git a/arch/mips/include/asm/cpu.h b/arch/mips/include/asm/cpu.h
index a97ca97..f672df8 100644
--- a/arch/mips/include/asm/cpu.h
+++ b/arch/mips/include/asm/cpu.h
@@ -42,6 +42,7 @@
#define PRID_COMP_LEXRA 0x0b0000
#define PRID_COMP_NETLOGIC 0x0c0000
#define PRID_COMP_CAVIUM 0x0d0000
+#define PRID_COMP_LOONGSON 0x140000
#define PRID_COMP_INGENIC_D0 0xd00000 /* JZ4740, JZ4750 */
#define PRID_COMP_INGENIC_D1 0xd10000 /* JZ4770, JZ4775 */
#define PRID_COMP_INGENIC_E1 0xe10000 /* JZ4780 */
@@ -118,9 +119,11 @@
#define PRID_IMP_INTERAPTIV_MP 0xa100
#define PRID_IMP_PROAPTIV_UP 0xa200
#define PRID_IMP_PROAPTIV_MP 0xa300
+#define PRID_IMP_P6600 0xa400
#define PRID_IMP_M5150 0xa700
#define PRID_IMP_P5600 0xa800
#define PRID_IMP_I6400 0xa900
+#define PRID_IMP_M6250 0xab00
/*
* These are the PRID's for when 23:16 == PRID_COMP_SIBYTE
@@ -169,6 +172,8 @@
#define PRID_IMP_CAVIUM_CNF71XX 0x9400
#define PRID_IMP_CAVIUM_CN78XX 0x9500
#define PRID_IMP_CAVIUM_CN70XX 0x9600
+#define PRID_IMP_CAVIUM_CN73XX 0x9700
+#define PRID_IMP_CAVIUM_CNF75XX 0x9800
/*
* These are the PRID's for when 23:16 == PRID_COMP_INGENIC_*
@@ -237,9 +242,10 @@
#define PRID_REV_LOONGSON1B 0x0020
#define PRID_REV_LOONGSON2E 0x0002
#define PRID_REV_LOONGSON2F 0x0003
-#define PRID_REV_LOONGSON3A 0x0005
+#define PRID_REV_LOONGSON3A_R1 0x0005
#define PRID_REV_LOONGSON3B_R1 0x0006
#define PRID_REV_LOONGSON3B_R2 0x0007
+#define PRID_REV_LOONGSON3A_R2 0x0008
/*
* Older processors used to encode processor version and revision in two
@@ -307,8 +313,8 @@
CPU_4KC, CPU_4KEC, CPU_4KSC, CPU_24K, CPU_34K, CPU_1004K, CPU_74K,
CPU_ALCHEMY, CPU_PR4450, CPU_BMIPS32, CPU_BMIPS3300, CPU_BMIPS4350,
CPU_BMIPS4380, CPU_BMIPS5000, CPU_JZRISC, CPU_LOONGSON1, CPU_M14KC,
- CPU_M14KEC, CPU_INTERAPTIV, CPU_P5600, CPU_PROAPTIV, CPU_1074K, CPU_M5150,
- CPU_I6400,
+ CPU_M14KEC, CPU_INTERAPTIV, CPU_P5600, CPU_PROAPTIV, CPU_1074K,
+ CPU_M5150, CPU_I6400, CPU_P6600, CPU_M6250,
/*
* MIPS64 class processors
@@ -346,48 +352,68 @@
MIPS_CPU_ISA_M64R6)
/*
+ * Private version of BIT_ULL() to escape include file recursion hell.
+ * We soon will have to switch to another mechanism that will work with
+ * more than 64 bits anyway.
+ */
+#define MBIT_ULL(bit) (1ULL << (bit))
+
+/*
* CPU Option encodings
*/
-#define MIPS_CPU_TLB 0x00000001ull /* CPU has TLB */
-#define MIPS_CPU_4KEX 0x00000002ull /* "R4K" exception model */
-#define MIPS_CPU_3K_CACHE 0x00000004ull /* R3000-style caches */
-#define MIPS_CPU_4K_CACHE 0x00000008ull /* R4000-style caches */
-#define MIPS_CPU_TX39_CACHE 0x00000010ull /* TX3900-style caches */
-#define MIPS_CPU_FPU 0x00000020ull /* CPU has FPU */
-#define MIPS_CPU_32FPR 0x00000040ull /* 32 dbl. prec. FP registers */
-#define MIPS_CPU_COUNTER 0x00000080ull /* Cycle count/compare */
-#define MIPS_CPU_WATCH 0x00000100ull /* watchpoint registers */
-#define MIPS_CPU_DIVEC 0x00000200ull /* dedicated interrupt vector */
-#define MIPS_CPU_VCE 0x00000400ull /* virt. coherence conflict possible */
-#define MIPS_CPU_CACHE_CDEX_P 0x00000800ull /* Create_Dirty_Exclusive CACHE op */
-#define MIPS_CPU_CACHE_CDEX_S 0x00001000ull /* ... same for seconary cache ... */
-#define MIPS_CPU_MCHECK 0x00002000ull /* Machine check exception */
-#define MIPS_CPU_EJTAG 0x00004000ull /* EJTAG exception */
-#define MIPS_CPU_NOFPUEX 0x00008000ull /* no FPU exception */
-#define MIPS_CPU_LLSC 0x00010000ull /* CPU has ll/sc instructions */
-#define MIPS_CPU_INCLUSIVE_CACHES 0x00020000ull /* P-cache subset enforced */
-#define MIPS_CPU_PREFETCH 0x00040000ull /* CPU has usable prefetch */
-#define MIPS_CPU_VINT 0x00080000ull /* CPU supports MIPSR2 vectored interrupts */
-#define MIPS_CPU_VEIC 0x00100000ull /* CPU supports MIPSR2 external interrupt controller mode */
-#define MIPS_CPU_ULRI 0x00200000ull /* CPU has ULRI feature */
-#define MIPS_CPU_PCI 0x00400000ull /* CPU has Perf Ctr Int indicator */
-#define MIPS_CPU_RIXI 0x00800000ull /* CPU has TLB Read/eXec Inhibit */
-#define MIPS_CPU_MICROMIPS 0x01000000ull /* CPU has microMIPS capability */
-#define MIPS_CPU_TLBINV 0x02000000ull /* CPU supports TLBINV/F */
-#define MIPS_CPU_SEGMENTS 0x04000000ull /* CPU supports Segmentation Control registers */
-#define MIPS_CPU_EVA 0x80000000ull /* CPU supports Enhanced Virtual Addressing */
-#define MIPS_CPU_HTW 0x100000000ull /* CPU support Hardware Page Table Walker */
-#define MIPS_CPU_RIXIEX 0x200000000ull /* CPU has unique exception codes for {Read, Execute}-Inhibit exceptions */
-#define MIPS_CPU_MAAR 0x400000000ull /* MAAR(I) registers are present */
-#define MIPS_CPU_FRE 0x800000000ull /* FRE & UFE bits implemented */
-#define MIPS_CPU_RW_LLB 0x1000000000ull /* LLADDR/LLB writes are allowed */
-#define MIPS_CPU_XPA 0x2000000000ull /* CPU supports Extended Physical Addressing */
-#define MIPS_CPU_CDMM 0x4000000000ull /* CPU has Common Device Memory Map */
-#define MIPS_CPU_BP_GHIST 0x8000000000ull /* R12K+ Branch Prediction Global History */
-#define MIPS_CPU_SP 0x10000000000ull /* Small (1KB) page support */
-#define MIPS_CPU_FTLB 0x20000000000ull /* CPU has Fixed-page-size TLB */
-#define MIPS_CPU_NAN_LEGACY 0x40000000000ull /* Legacy NaN implemented */
-#define MIPS_CPU_NAN_2008 0x80000000000ull /* 2008 NaN implemented */
+#define MIPS_CPU_TLB MBIT_ULL( 0) /* CPU has TLB */
+#define MIPS_CPU_4KEX MBIT_ULL( 1) /* "R4K" exception model */
+#define MIPS_CPU_3K_CACHE MBIT_ULL( 2) /* R3000-style caches */
+#define MIPS_CPU_4K_CACHE MBIT_ULL( 3) /* R4000-style caches */
+#define MIPS_CPU_TX39_CACHE MBIT_ULL( 4) /* TX3900-style caches */
+#define MIPS_CPU_FPU MBIT_ULL( 5) /* CPU has FPU */
+#define MIPS_CPU_32FPR MBIT_ULL( 6) /* 32 dbl. prec. FP registers */
+#define MIPS_CPU_COUNTER MBIT_ULL( 7) /* Cycle count/compare */
+#define MIPS_CPU_WATCH MBIT_ULL( 8) /* watchpoint registers */
+#define MIPS_CPU_DIVEC MBIT_ULL( 9) /* dedicated interrupt vector */
+#define MIPS_CPU_VCE MBIT_ULL(10) /* virt. coherence conflict possible */
+#define MIPS_CPU_CACHE_CDEX_P MBIT_ULL(11) /* Create_Dirty_Exclusive CACHE op */
+#define MIPS_CPU_CACHE_CDEX_S MBIT_ULL(12) /* ... same for seconary cache ... */
+#define MIPS_CPU_MCHECK MBIT_ULL(13) /* Machine check exception */
+#define MIPS_CPU_EJTAG MBIT_ULL(14) /* EJTAG exception */
+#define MIPS_CPU_NOFPUEX MBIT_ULL(15) /* no FPU exception */
+#define MIPS_CPU_LLSC MBIT_ULL(16) /* CPU has ll/sc instructions */
+#define MIPS_CPU_INCLUSIVE_CACHES MBIT_ULL(17) /* P-cache subset enforced */
+#define MIPS_CPU_PREFETCH MBIT_ULL(18) /* CPU has usable prefetch */
+#define MIPS_CPU_VINT MBIT_ULL(19) /* CPU supports MIPSR2 vectored interrupts */
+#define MIPS_CPU_VEIC MBIT_ULL(20) /* CPU supports MIPSR2 external interrupt controller mode */
+#define MIPS_CPU_ULRI MBIT_ULL(21) /* CPU has ULRI feature */
+#define MIPS_CPU_PCI MBIT_ULL(22) /* CPU has Perf Ctr Int indicator */
+#define MIPS_CPU_RIXI MBIT_ULL(23) /* CPU has TLB Read/eXec Inhibit */
+#define MIPS_CPU_MICROMIPS MBIT_ULL(24) /* CPU has microMIPS capability */
+#define MIPS_CPU_TLBINV MBIT_ULL(25) /* CPU supports TLBINV/F */
+#define MIPS_CPU_SEGMENTS MBIT_ULL(26) /* CPU supports Segmentation Control registers */
+#define MIPS_CPU_EVA MBIT_ULL(27) /* CPU supports Enhanced Virtual Addressing */
+#define MIPS_CPU_HTW MBIT_ULL(28) /* CPU support Hardware Page Table Walker */
+#define MIPS_CPU_RIXIEX MBIT_ULL(29) /* CPU has unique exception codes for {Read, Execute}-Inhibit exceptions */
+#define MIPS_CPU_MAAR MBIT_ULL(30) /* MAAR(I) registers are present */
+#define MIPS_CPU_FRE MBIT_ULL(31) /* FRE & UFE bits implemented */
+#define MIPS_CPU_RW_LLB MBIT_ULL(32) /* LLADDR/LLB writes are allowed */
+#define MIPS_CPU_LPA MBIT_ULL(33) /* CPU supports Large Physical Addressing */
+#define MIPS_CPU_CDMM MBIT_ULL(34) /* CPU has Common Device Memory Map */
+#define MIPS_CPU_BP_GHIST MBIT_ULL(35) /* R12K+ Branch Prediction Global History */
+#define MIPS_CPU_SP MBIT_ULL(36) /* Small (1KB) page support */
+#define MIPS_CPU_FTLB MBIT_ULL(37) /* CPU has Fixed-page-size TLB */
+#define MIPS_CPU_NAN_LEGACY MBIT_ULL(38) /* Legacy NaN implemented */
+#define MIPS_CPU_NAN_2008 MBIT_ULL(39) /* 2008 NaN implemented */
+#define MIPS_CPU_VP MBIT_ULL(40) /* MIPSr6 Virtual Processors (multi-threading) */
+#define MIPS_CPU_LDPTE MBIT_ULL(41) /* CPU has ldpte/lddir instructions */
+#define MIPS_CPU_MVH MBIT_ULL(42) /* CPU supports MFHC0/MTHC0 */
+#define MIPS_CPU_EBASE_WG MBIT_ULL(43) /* CPU has EBase.WG */
+#define MIPS_CPU_BADINSTR MBIT_ULL(44) /* CPU has BadInstr register */
+#define MIPS_CPU_BADINSTRP MBIT_ULL(45) /* CPU has BadInstrP register */
+#define MIPS_CPU_CTXTC MBIT_ULL(46) /* CPU has [X]ConfigContext registers */
+#define MIPS_CPU_PERF MBIT_ULL(47) /* CPU has MIPS performance counters */
+#define MIPS_CPU_GUESTCTL0EXT MBIT_ULL(48) /* CPU has VZ GuestCtl0Ext register */
+#define MIPS_CPU_GUESTCTL1 MBIT_ULL(49) /* CPU has VZ GuestCtl1 register */
+#define MIPS_CPU_GUESTCTL2 MBIT_ULL(50) /* CPU has VZ GuestCtl2 register */
+#define MIPS_CPU_GUESTID MBIT_ULL(51) /* CPU uses VZ ASE GuestID feature */
+#define MIPS_CPU_DRG MBIT_ULL(52) /* CPU has VZ Direct Root to Guest (DRG) */
/*
* CPU ASE encodings
@@ -401,5 +427,6 @@
#define MIPS_ASE_DSP2P 0x00000040 /* Signal Processing ASE Rev 2 */
#define MIPS_ASE_VZ 0x00000080 /* Virtualization ASE */
#define MIPS_ASE_MSA 0x00000100 /* MIPS SIMD Architecture */
+#define MIPS_ASE_DSP3 0x00000200 /* Signal Processing ASE Rev 3*/
#endif /* _ASM_CPU_H */
diff --git a/arch/mips/include/asm/elf.h b/arch/mips/include/asm/elf.h
index e090fc3..f5f4571 100644
--- a/arch/mips/include/asm/elf.h
+++ b/arch/mips/include/asm/elf.h
@@ -111,6 +111,11 @@
#define R_MIPS_CALLHI16 30
#define R_MIPS_CALLLO16 31
/*
+ * Introduced for MIPSr6.
+ */
+#define R_MIPS_PC21_S2 60
+#define R_MIPS_PC26_S2 61
+/*
* This range is reserved for vendor specific relocations.
*/
#define R_MIPS_LOVENDOR 100
@@ -170,16 +175,14 @@
#define SHF_MIPS_NAMES 0x02000000
#define SHF_MIPS_NODUPES 0x01000000
-#ifndef ELF_ARCH
-/* ELF register definitions */
-#define ELF_NGREG 45
-#define ELF_NFPREG 33
-
-typedef unsigned long elf_greg_t;
-typedef elf_greg_t elf_gregset_t[ELF_NGREG];
-
-typedef double elf_fpreg_t;
-typedef elf_fpreg_t elf_fpregset_t[ELF_NFPREG];
+#define MIPS_ABI_FP_ANY 0 /* FP ABI doesn't matter */
+#define MIPS_ABI_FP_DOUBLE 1 /* -mdouble-float */
+#define MIPS_ABI_FP_SINGLE 2 /* -msingle-float */
+#define MIPS_ABI_FP_SOFT 3 /* -msoft-float */
+#define MIPS_ABI_FP_OLD_64 4 /* -mips32r2 -mfp64 */
+#define MIPS_ABI_FP_XX 5 /* -mfpxx */
+#define MIPS_ABI_FP_64 6 /* -mips32r2 -mfp64 */
+#define MIPS_ABI_FP_64A 7 /* -mips32r2 -mfp64 -mno-odd-spreg */
struct mips_elf_abiflags_v0 {
uint16_t version; /* Version of flags structure */
@@ -196,51 +199,22 @@
uint32_t flags2;
};
-#define MIPS_ABI_FP_ANY 0 /* FP ABI doesn't matter */
-#define MIPS_ABI_FP_DOUBLE 1 /* -mdouble-float */
-#define MIPS_ABI_FP_SINGLE 2 /* -msingle-float */
-#define MIPS_ABI_FP_SOFT 3 /* -msoft-float */
-#define MIPS_ABI_FP_OLD_64 4 /* -mips32r2 -mfp64 */
-#define MIPS_ABI_FP_XX 5 /* -mfpxx */
-#define MIPS_ABI_FP_64 6 /* -mips32r2 -mfp64 */
-#define MIPS_ABI_FP_64A 7 /* -mips32r2 -mfp64 -mno-odd-spreg */
+#ifndef ELF_ARCH
+/* ELF register definitions */
+#define ELF_NGREG 45
+#define ELF_NFPREG 33
+
+typedef unsigned long elf_greg_t;
+typedef elf_greg_t elf_gregset_t[ELF_NGREG];
+
+typedef double elf_fpreg_t;
+typedef elf_fpreg_t elf_fpregset_t[ELF_NFPREG];
#ifdef CONFIG_32BIT
-
-/*
- * In order to be sure that we don't attempt to execute an O32 binary which
- * requires 64 bit FP (FR=1) on a system which does not support it we refuse
- * to execute any binary which has bits specified by the following macro set
- * in its ELF header flags.
- */
-#ifdef CONFIG_MIPS_O32_FP64_SUPPORT
-# define __MIPS_O32_FP64_MUST_BE_ZERO 0
-#else
-# define __MIPS_O32_FP64_MUST_BE_ZERO EF_MIPS_FP64
-#endif
-
/*
* This is used to ensure we don't load something for the wrong architecture.
*/
-#define elf_check_arch(hdr) \
-({ \
- int __res = 1; \
- struct elfhdr *__h = (hdr); \
- \
- if (!mips_elf_check_machine(__h)) \
- __res = 0; \
- if (__h->e_ident[EI_CLASS] != ELFCLASS32) \
- __res = 0; \
- if ((__h->e_flags & EF_MIPS_ABI2) != 0) \
- __res = 0; \
- if (((__h->e_flags & EF_MIPS_ABI) != 0) && \
- ((__h->e_flags & EF_MIPS_ABI) != EF_MIPS_ABI_O32)) \
- __res = 0; \
- if (__h->e_flags & __MIPS_O32_FP64_MUST_BE_ZERO) \
- __res = 0; \
- \
- __res; \
-})
+#define elf_check_arch elfo32_check_arch
/*
* These are used to set parameters in the core dumps.
@@ -253,18 +227,7 @@
/*
* This is used to ensure we don't load something for the wrong architecture.
*/
-#define elf_check_arch(hdr) \
-({ \
- int __res = 1; \
- struct elfhdr *__h = (hdr); \
- \
- if (!mips_elf_check_machine(__h)) \
- __res = 0; \
- if (__h->e_ident[EI_CLASS] != ELFCLASS64) \
- __res = 0; \
- \
- __res; \
-})
+#define elf_check_arch elfn64_check_arch
/*
* These are used to set parameters in the core dumps.
@@ -285,11 +248,81 @@
#endif /* !defined(ELF_ARCH) */
+/*
+ * In order to be sure that we don't attempt to execute an O32 binary which
+ * requires 64 bit FP (FR=1) on a system which does not support it we refuse
+ * to execute any binary which has bits specified by the following macro set
+ * in its ELF header flags.
+ */
+#ifdef CONFIG_MIPS_O32_FP64_SUPPORT
+# define __MIPS_O32_FP64_MUST_BE_ZERO 0
+#else
+# define __MIPS_O32_FP64_MUST_BE_ZERO EF_MIPS_FP64
+#endif
+
#define mips_elf_check_machine(x) ((x)->e_machine == EM_MIPS)
#define vmcore_elf32_check_arch mips_elf_check_machine
#define vmcore_elf64_check_arch mips_elf_check_machine
+/*
+ * Return non-zero if HDR identifies an o32 ELF binary.
+ */
+#define elfo32_check_arch(hdr) \
+({ \
+ int __res = 1; \
+ struct elfhdr *__h = (hdr); \
+ \
+ if (!mips_elf_check_machine(__h)) \
+ __res = 0; \
+ if (__h->e_ident[EI_CLASS] != ELFCLASS32) \
+ __res = 0; \
+ if ((__h->e_flags & EF_MIPS_ABI2) != 0) \
+ __res = 0; \
+ if (((__h->e_flags & EF_MIPS_ABI) != 0) && \
+ ((__h->e_flags & EF_MIPS_ABI) != EF_MIPS_ABI_O32)) \
+ __res = 0; \
+ if (__h->e_flags & __MIPS_O32_FP64_MUST_BE_ZERO) \
+ __res = 0; \
+ \
+ __res; \
+})
+
+/*
+ * Return non-zero if HDR identifies an n64 ELF binary.
+ */
+#define elfn64_check_arch(hdr) \
+({ \
+ int __res = 1; \
+ struct elfhdr *__h = (hdr); \
+ \
+ if (!mips_elf_check_machine(__h)) \
+ __res = 0; \
+ if (__h->e_ident[EI_CLASS] != ELFCLASS64) \
+ __res = 0; \
+ \
+ __res; \
+})
+
+/*
+ * Return non-zero if HDR identifies an n32 ELF binary.
+ */
+#define elfn32_check_arch(hdr) \
+({ \
+ int __res = 1; \
+ struct elfhdr *__h = (hdr); \
+ \
+ if (!mips_elf_check_machine(__h)) \
+ __res = 0; \
+ if (__h->e_ident[EI_CLASS] != ELFCLASS32) \
+ __res = 0; \
+ if (((__h->e_flags & EF_MIPS_ABI2) == 0) || \
+ ((__h->e_flags & EF_MIPS_ABI) != 0)) \
+ __res = 0; \
+ \
+ __res; \
+})
+
struct mips_abi;
extern struct mips_abi mips_abi;
@@ -300,17 +333,16 @@
#define SET_PERSONALITY2(ex, state) \
do { \
- if (personality(current->personality) != PER_LINUX) \
- set_personality(PER_LINUX); \
- \
clear_thread_flag(TIF_HYBRID_FPREGS); \
set_thread_flag(TIF_32BIT_FPREGS); \
\
- mips_set_personality_fp(state); \
- \
current->thread.abi = &mips_abi; \
\
+ mips_set_personality_fp(state); \
mips_set_personality_nan(state); \
+ \
+ if (personality(current->personality) != PER_LINUX) \
+ set_personality(PER_LINUX); \
} while (0)
#endif /* CONFIG_32BIT */
@@ -321,6 +353,7 @@
#define __SET_PERSONALITY32_N32() \
do { \
set_thread_flag(TIF_32BIT_ADDR); \
+ \
current->thread.abi = &mips_abi_n32; \
} while (0)
#else
@@ -336,9 +369,9 @@
clear_thread_flag(TIF_HYBRID_FPREGS); \
set_thread_flag(TIF_32BIT_FPREGS); \
\
- mips_set_personality_fp(state); \
- \
current->thread.abi = &mips_abi_32; \
+ \
+ mips_set_personality_fp(state); \
} while (0)
#else
#define __SET_PERSONALITY32_O32(ex, state) \
diff --git a/arch/mips/include/asm/hazards.h b/arch/mips/include/asm/hazards.h
index 7b99efd..dbb1eb6 100644
--- a/arch/mips/include/asm/hazards.h
+++ b/arch/mips/include/asm/hazards.h
@@ -22,7 +22,8 @@
/*
* TLB hazards
*/
-#if defined(CONFIG_CPU_MIPSR2) || defined(CONFIG_CPU_MIPSR6) && !defined(CONFIG_CPU_CAVIUM_OCTEON)
+#if (defined(CONFIG_CPU_MIPSR2) || defined(CONFIG_CPU_MIPSR6)) && \
+ !defined(CONFIG_CPU_CAVIUM_OCTEON) && !defined(CONFIG_LOONGSON3_ENHANCEMENT)
/*
* MIPSR2 defines ehb for hazard avoidance
@@ -155,8 +156,8 @@
} while (0)
#elif defined(CONFIG_MIPS_ALCHEMY) || defined(CONFIG_CPU_CAVIUM_OCTEON) || \
- defined(CONFIG_CPU_LOONGSON2) || defined(CONFIG_CPU_R10000) || \
- defined(CONFIG_CPU_R5500) || defined(CONFIG_CPU_XLR)
+ defined(CONFIG_CPU_LOONGSON2) || defined(CONFIG_LOONGSON3_ENHANCEMENT) || \
+ defined(CONFIG_CPU_R10000) || defined(CONFIG_CPU_R5500) || defined(CONFIG_CPU_XLR)
/*
* R10000 rocks - all hazards handled in hardware, so this becomes a nobrainer.
diff --git a/arch/mips/include/asm/highmem.h b/arch/mips/include/asm/highmem.h
index 01880b3..64f2500 100644
--- a/arch/mips/include/asm/highmem.h
+++ b/arch/mips/include/asm/highmem.h
@@ -19,8 +19,10 @@
#ifdef __KERNEL__
+#include <linux/bug.h>
#include <linux/interrupt.h>
#include <linux/uaccess.h>
+#include <asm/cpu-features.h>
#include <asm/kmap_types.h>
/* undef for production */
@@ -50,7 +52,7 @@
extern void __kunmap_atomic(void *kvaddr);
extern void *kmap_atomic_pfn(unsigned long pfn);
-#define flush_cache_kmaps() flush_cache_all()
+#define flush_cache_kmaps() BUG_ON(cpu_has_dc_aliases)
extern void kmap_init(void);
diff --git a/arch/mips/include/asm/io.h b/arch/mips/include/asm/io.h
index 2b4dc7a..ecabc00 100644
--- a/arch/mips/include/asm/io.h
+++ b/arch/mips/include/asm/io.h
@@ -304,10 +304,10 @@
#undef __IS_KSEG1
}
-#ifdef CONFIG_CPU_CAVIUM_OCTEON
-#define war_octeon_io_reorder_wmb() wmb()
+#if defined(CONFIG_CPU_CAVIUM_OCTEON) || defined(CONFIG_LOONGSON3_ENHANCEMENT)
+#define war_io_reorder_wmb() wmb()
#else
-#define war_octeon_io_reorder_wmb() do { } while (0)
+#define war_io_reorder_wmb() do { } while (0)
#endif
#define __BUILD_MEMORY_SINGLE(pfx, bwlq, type, irq) \
@@ -318,7 +318,7 @@
volatile type *__mem; \
type __val; \
\
- war_octeon_io_reorder_wmb(); \
+ war_io_reorder_wmb(); \
\
__mem = (void *)__swizzle_addr_##bwlq((unsigned long)(mem)); \
\
@@ -387,7 +387,7 @@
volatile type *__addr; \
type __val; \
\
- war_octeon_io_reorder_wmb(); \
+ war_io_reorder_wmb(); \
\
__addr = (void *)__swizzle_addr_##bwlq(mips_io_port_base + port); \
\
diff --git a/arch/mips/include/asm/irq_regs.h b/arch/mips/include/asm/irq_regs.h
index 33bd2a0..8c48d6d 100644
--- a/arch/mips/include/asm/irq_regs.h
+++ b/arch/mips/include/asm/irq_regs.h
@@ -18,4 +18,14 @@
return current_thread_info()->regs;
}
+static inline struct pt_regs *set_irq_regs(struct pt_regs *new_regs)
+{
+ struct pt_regs *old_regs;
+
+ old_regs = get_irq_regs();
+ current_thread_info()->regs = new_regs;
+
+ return old_regs;
+}
+
#endif /* __ASM_IRQ_REGS_H */
diff --git a/arch/mips/include/asm/irqflags.h b/arch/mips/include/asm/irqflags.h
index 65c351e..9d3610b 100644
--- a/arch/mips/include/asm/irqflags.h
+++ b/arch/mips/include/asm/irqflags.h
@@ -41,7 +41,12 @@
" .set push \n"
" .set reorder \n"
" .set noat \n"
+#if defined(CONFIG_CPU_LOONGSON3)
+ " mfc0 %[flags], $12 \n"
+ " di \n"
+#else
" di %[flags] \n"
+#endif
" andi %[flags], 1 \n"
" " __stringify(__irq_disable_hazard) " \n"
" .set pop \n"
diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index f6b1279..6733ac5 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -122,6 +122,7 @@
u32 flush_dcache_exits;
u32 halt_successful_poll;
u32 halt_attempted_poll;
+ u32 halt_poll_invalid;
u32 halt_wakeup;
};
@@ -311,17 +312,18 @@
#define MIPS3_PG_FRAME 0x3fffffc0
#define VPN2_MASK 0xffffe000
+#define KVM_ENTRYHI_ASID MIPS_ENTRYHI_ASID
#define TLB_IS_GLOBAL(x) (((x).tlb_lo0 & MIPS3_PG_G) && \
((x).tlb_lo1 & MIPS3_PG_G))
#define TLB_VPN2(x) ((x).tlb_hi & VPN2_MASK)
-#define TLB_ASID(x) ((x).tlb_hi & ASID_MASK)
+#define TLB_ASID(x) ((x).tlb_hi & KVM_ENTRYHI_ASID)
#define TLB_IS_VALID(x, va) (((va) & (1 << PAGE_SHIFT)) \
? ((x).tlb_lo1 & MIPS3_PG_V) \
: ((x).tlb_lo0 & MIPS3_PG_V))
#define TLB_HI_VPN2_HIT(x, y) ((TLB_VPN2(x) & ~(x).tlb_mask) == \
((y) & VPN2_MASK & ~(x).tlb_mask))
#define TLB_HI_ASID_HIT(x, y) (TLB_IS_GLOBAL(x) || \
- TLB_ASID(x) == ((y) & ASID_MASK))
+ TLB_ASID(x) == ((y) & KVM_ENTRYHI_ASID))
struct kvm_mips_tlb {
long tlb_mask;
@@ -747,7 +749,7 @@
uint32_t kvm_mips_read_count(struct kvm_vcpu *vcpu);
void kvm_mips_write_count(struct kvm_vcpu *vcpu, uint32_t count);
-void kvm_mips_write_compare(struct kvm_vcpu *vcpu, uint32_t compare);
+void kvm_mips_write_compare(struct kvm_vcpu *vcpu, uint32_t compare, bool ack);
void kvm_mips_init_count(struct kvm_vcpu *vcpu);
int kvm_mips_set_count_ctl(struct kvm_vcpu *vcpu, s64 count_ctl);
int kvm_mips_set_count_resume(struct kvm_vcpu *vcpu, s64 count_resume);
@@ -812,5 +814,6 @@
static inline void kvm_arch_sched_in(struct kvm_vcpu *vcpu, int cpu) {}
static inline void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu) {}
static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {}
+static inline void kvm_arch_vcpu_block_finish(struct kvm_vcpu *vcpu) {}
#endif /* __MIPS_KVM_HOST_H__ */
diff --git a/arch/mips/include/asm/llsc.h b/arch/mips/include/asm/llsc.h
new file mode 100644
index 0000000..c6d17d1
--- /dev/null
+++ b/arch/mips/include/asm/llsc.h
@@ -0,0 +1,28 @@
+/*
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License. See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * Macros for 32/64-bit neutral inline assembler
+ */
+
+#ifndef __ASM_LLSC_H
+#define __ASM_LLSC_H
+
+#if _MIPS_SZLONG == 32
+#define SZLONG_LOG 5
+#define SZLONG_MASK 31UL
+#define __LL "ll "
+#define __SC "sc "
+#define __INS "ins "
+#define __EXT "ext "
+#elif _MIPS_SZLONG == 64
+#define SZLONG_LOG 6
+#define SZLONG_MASK 63UL
+#define __LL "lld "
+#define __SC "scd "
+#define __INS "dins "
+#define __EXT "dext "
+#endif
+
+#endif /* __ASM_LLSC_H */
diff --git a/arch/mips/include/asm/mach-bmips/cpu-feature-overrides.h b/arch/mips/include/asm/mach-bmips/cpu-feature-overrides.h
new file mode 100644
index 0000000..fa0583e
--- /dev/null
+++ b/arch/mips/include/asm/mach-bmips/cpu-feature-overrides.h
@@ -0,0 +1,14 @@
+#ifndef __ASM_MACH_BMIPS_CPU_FEATURE_OVERRIDES_H
+#define __ASM_MACH_BMIPS_CPU_FEATURE_OVERRIDES_H
+
+/* Invariants across all BMIPS processors */
+#define cpu_has_vtag_icache 0
+#define cpu_icache_snoops_remote_store 1
+
+/* Processor ISA compatibility is MIPS32R1 */
+#define cpu_has_mips32r1 1
+#define cpu_has_mips32r2 0
+#define cpu_has_mips64r1 0
+#define cpu_has_mips64r2 0
+
+#endif /* __ASM_MACH_BMIPS_CPU_FEATURE_OVERRIDES_H */
diff --git a/arch/mips/include/asm/mach-bmips/ioremap.h b/arch/mips/include/asm/mach-bmips/ioremap.h
new file mode 100644
index 0000000..29c7a7b
--- /dev/null
+++ b/arch/mips/include/asm/mach-bmips/ioremap.h
@@ -0,0 +1,33 @@
+#ifndef __ASM_MACH_BMIPS_IOREMAP_H
+#define __ASM_MACH_BMIPS_IOREMAP_H
+
+#include <linux/types.h>
+
+static inline phys_addr_t fixup_bigphys_addr(phys_addr_t phys_addr, phys_addr_t size)
+{
+ return phys_addr;
+}
+
+static inline int is_bmips_internal_registers(phys_addr_t offset)
+{
+ if (offset >= 0xfff80000)
+ return 1;
+
+ return 0;
+}
+
+static inline void __iomem *plat_ioremap(phys_addr_t offset, unsigned long size,
+ unsigned long flags)
+{
+ if (is_bmips_internal_registers(offset))
+ return (void __iomem *)offset;
+
+ return NULL;
+}
+
+static inline int plat_iounmap(const volatile void __iomem *addr)
+{
+ return is_bmips_internal_registers((unsigned long)addr);
+}
+
+#endif /* __ASM_MACH_BMIPS_IOREMAP_H */
diff --git a/arch/mips/include/asm/mach-jz4740/jz4740_nand.h b/arch/mips/include/asm/mach-jz4740/jz4740_nand.h
index 398733e..7f7b0fc 100644
--- a/arch/mips/include/asm/mach-jz4740/jz4740_nand.h
+++ b/arch/mips/include/asm/mach-jz4740/jz4740_nand.h
@@ -27,7 +27,7 @@
unsigned char banks[JZ_NAND_NUM_BANKS];
- void (*ident_callback)(struct platform_device *, struct nand_chip *,
+ void (*ident_callback)(struct platform_device *, struct mtd_info *,
struct mtd_partition **, int *num_partitions);
};
diff --git a/arch/mips/include/asm/mach-jz4740/platform.h b/arch/mips/include/asm/mach-jz4740/platform.h
index 32cfbe6..073b8bf 100644
--- a/arch/mips/include/asm/mach-jz4740/platform.h
+++ b/arch/mips/include/asm/mach-jz4740/platform.h
@@ -19,7 +19,6 @@
#include <linux/platform_device.h>
-extern struct platform_device jz4740_usb_ohci_device;
extern struct platform_device jz4740_udc_device;
extern struct platform_device jz4740_udc_xceiv_device;
extern struct platform_device jz4740_mmc_device;
diff --git a/arch/mips/include/asm/mach-lantiq/falcon/lantiq_soc.h b/arch/mips/include/asm/mach-lantiq/falcon/lantiq_soc.h
index 98d6a2f..7023883 100644
--- a/arch/mips/include/asm/mach-lantiq/falcon/lantiq_soc.h
+++ b/arch/mips/include/asm/mach-lantiq/falcon/lantiq_soc.h
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2010 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2010 John Crispin <john@phrozen.org>
*/
#ifndef _LTQ_FALCON_H__
diff --git a/arch/mips/include/asm/mach-lantiq/lantiq.h b/arch/mips/include/asm/mach-lantiq/lantiq.h
index 4e5ae65..8064d7a 100644
--- a/arch/mips/include/asm/mach-lantiq/lantiq.h
+++ b/arch/mips/include/asm/mach-lantiq/lantiq.h
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2010 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2010 John Crispin <john@phrozen.org>
*/
#ifndef _LANTIQ_H__
#define _LANTIQ_H__
diff --git a/arch/mips/include/asm/mach-lantiq/lantiq_platform.h b/arch/mips/include/asm/mach-lantiq/lantiq_platform.h
index e23bf7c..17d2fdc 100644
--- a/arch/mips/include/asm/mach-lantiq/lantiq_platform.h
+++ b/arch/mips/include/asm/mach-lantiq/lantiq_platform.h
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2010 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2010 John Crispin <john@phrozen.org>
*/
#ifndef _LANTIQ_PLATFORM_H__
diff --git a/arch/mips/include/asm/mach-lantiq/xway/irq.h b/arch/mips/include/asm/mach-lantiq/xway/irq.h
index a1471d2..83e5f03 100644
--- a/arch/mips/include/asm/mach-lantiq/xway/irq.h
+++ b/arch/mips/include/asm/mach-lantiq/xway/irq.h
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2010 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2010 John Crispin <john@phrozen.org>
*/
#ifndef __LANTIQ_IRQ_H
diff --git a/arch/mips/include/asm/mach-lantiq/xway/lantiq_irq.h b/arch/mips/include/asm/mach-lantiq/xway/lantiq_irq.h
index 5eadfe5..1410763 100644
--- a/arch/mips/include/asm/mach-lantiq/xway/lantiq_irq.h
+++ b/arch/mips/include/asm/mach-lantiq/xway/lantiq_irq.h
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2010 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2010 John Crispin <john@phrozen.org>
*/
#ifndef _LANTIQ_XWAY_IRQ_H__
diff --git a/arch/mips/include/asm/mach-lantiq/xway/lantiq_soc.h b/arch/mips/include/asm/mach-lantiq/xway/lantiq_soc.h
index dd6005b..f873107 100644
--- a/arch/mips/include/asm/mach-lantiq/xway/lantiq_soc.h
+++ b/arch/mips/include/asm/mach-lantiq/xway/lantiq_soc.h
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2010 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2010 John Crispin <john@phrozen.org>
*/
#ifndef _LTQ_XWAY_H__
diff --git a/arch/mips/include/asm/mach-lantiq/xway/xway_dma.h b/arch/mips/include/asm/mach-lantiq/xway/xway_dma.h
index 5f8693d..4901833 100644
--- a/arch/mips/include/asm/mach-lantiq/xway/xway_dma.h
+++ b/arch/mips/include/asm/mach-lantiq/xway/xway_dma.h
@@ -12,7 +12,7 @@
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA.
*
- * Copyright (C) 2011 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2011 John Crispin <john@phrozen.org>
*/
#ifndef LTQ_DMA_H__
diff --git a/arch/mips/include/asm/mach-loongson32/cpufreq.h b/arch/mips/include/asm/mach-loongson32/cpufreq.h
index 6843fa1..2f1ecb0 100644
--- a/arch/mips/include/asm/mach-loongson32/cpufreq.h
+++ b/arch/mips/include/asm/mach-loongson32/cpufreq.h
@@ -9,7 +9,6 @@
* option) any later version.
*/
-
#ifndef __ASM_MACH_LOONGSON32_CPUFREQ_H
#define __ASM_MACH_LOONGSON32_CPUFREQ_H
diff --git a/arch/mips/include/asm/mach-loongson32/dma.h b/arch/mips/include/asm/mach-loongson32/dma.h
new file mode 100644
index 0000000..ad1dec7
--- /dev/null
+++ b/arch/mips/include/asm/mach-loongson32/dma.h
@@ -0,0 +1,25 @@
+/*
+ * Copyright (c) 2015 Zhang, Keguang <keguang.zhang@gmail.com>
+ *
+ * Loongson 1 NAND platform support.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#ifndef __ASM_MACH_LOONGSON32_DMA_H
+#define __ASM_MACH_LOONGSON32_DMA_H
+
+#define LS1X_DMA_CHANNEL0 0
+#define LS1X_DMA_CHANNEL1 1
+#define LS1X_DMA_CHANNEL2 2
+
+struct plat_ls1x_dma {
+ int nr_channels;
+};
+
+extern struct plat_ls1x_dma ls1b_dma_pdata;
+
+#endif /* __ASM_MACH_LOONGSON32_DMA_H */
diff --git a/arch/mips/include/asm/mach-loongson32/irq.h b/arch/mips/include/asm/mach-loongson32/irq.h
index 0d35b99..c1c7441 100644
--- a/arch/mips/include/asm/mach-loongson32/irq.h
+++ b/arch/mips/include/asm/mach-loongson32/irq.h
@@ -9,7 +9,6 @@
* option) any later version.
*/
-
#ifndef __ASM_MACH_LOONGSON32_IRQ_H
#define __ASM_MACH_LOONGSON32_IRQ_H
diff --git a/arch/mips/include/asm/mach-loongson32/loongson1.h b/arch/mips/include/asm/mach-loongson32/loongson1.h
index 12aa129..978f6df 100644
--- a/arch/mips/include/asm/mach-loongson32/loongson1.h
+++ b/arch/mips/include/asm/mach-loongson32/loongson1.h
@@ -9,7 +9,6 @@
* option) any later version.
*/
-
#ifndef __ASM_MACH_LOONGSON32_LOONGSON1_H
#define __ASM_MACH_LOONGSON32_LOONGSON1_H
@@ -18,6 +17,9 @@
/* Loongson 1 Register Bases */
#define LS1X_MUX_BASE 0x1fd00420
#define LS1X_INTC_BASE 0x1fd01040
+#define LS1X_GPIO0_BASE 0x1fd010c0
+#define LS1X_GPIO1_BASE 0x1fd010c4
+#define LS1X_DMAC_BASE 0x1fd01160
#define LS1X_EHCI_BASE 0x1fe00000
#define LS1X_OHCI_BASE 0x1fe08000
#define LS1X_GMAC0_BASE 0x1fe10000
diff --git a/arch/mips/include/asm/mach-loongson32/nand.h b/arch/mips/include/asm/mach-loongson32/nand.h
new file mode 100644
index 0000000..e274912
--- /dev/null
+++ b/arch/mips/include/asm/mach-loongson32/nand.h
@@ -0,0 +1,30 @@
+/*
+ * Copyright (c) 2015 Zhang, Keguang <keguang.zhang@gmail.com>
+ *
+ * Loongson 1 NAND platform support.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#ifndef __ASM_MACH_LOONGSON32_NAND_H
+#define __ASM_MACH_LOONGSON32_NAND_H
+
+#include <linux/dmaengine.h>
+#include <linux/mtd/partitions.h>
+
+struct plat_ls1x_nand {
+ struct mtd_partition *parts;
+ unsigned int nr_parts;
+
+ int hold_cycle;
+ int wait_cycle;
+};
+
+extern struct plat_ls1x_nand ls1b_nand_pdata;
+
+bool ls1x_dma_filter_fn(struct dma_chan *chan, void *param);
+
+#endif /* __ASM_MACH_LOONGSON32_NAND_H */
diff --git a/arch/mips/include/asm/mach-loongson32/platform.h b/arch/mips/include/asm/mach-loongson32/platform.h
index c32f03f..672531a 100644
--- a/arch/mips/include/asm/mach-loongson32/platform.h
+++ b/arch/mips/include/asm/mach-loongson32/platform.h
@@ -7,20 +7,28 @@
* option) any later version.
*/
-
#ifndef __ASM_MACH_LOONGSON32_PLATFORM_H
#define __ASM_MACH_LOONGSON32_PLATFORM_H
#include <linux/platform_device.h>
+#include <dma.h>
+#include <nand.h>
+
extern struct platform_device ls1x_uart_pdev;
extern struct platform_device ls1x_cpufreq_pdev;
+extern struct platform_device ls1x_dma_pdev;
extern struct platform_device ls1x_eth0_pdev;
extern struct platform_device ls1x_eth1_pdev;
extern struct platform_device ls1x_ehci_pdev;
+extern struct platform_device ls1x_gpio0_pdev;
+extern struct platform_device ls1x_gpio1_pdev;
+extern struct platform_device ls1x_nand_pdev;
extern struct platform_device ls1x_rtc_pdev;
-extern void __init ls1x_clk_init(void);
-extern void __init ls1x_serial_setup(struct platform_device *pdev);
+void __init ls1x_clk_init(void);
+void __init ls1x_dma_set_platdata(struct plat_ls1x_dma *pdata);
+void __init ls1x_nand_set_platdata(struct plat_ls1x_nand *pdata);
+void __init ls1x_serial_set_uartclk(struct platform_device *pdev);
#endif /* __ASM_MACH_LOONGSON32_PLATFORM_H */
diff --git a/arch/mips/include/asm/mach-loongson32/regs-clk.h b/arch/mips/include/asm/mach-loongson32/regs-clk.h
index 1f5a715..4d56fc3 100644
--- a/arch/mips/include/asm/mach-loongson32/regs-clk.h
+++ b/arch/mips/include/asm/mach-loongson32/regs-clk.h
@@ -19,18 +19,18 @@
#define LS1X_CLK_PLL_DIV LS1X_CLK_REG(0x4)
/* Clock PLL Divisor Register Bits */
-#define DIV_DC_EN (0x1 << 31)
-#define DIV_DC_RST (0x1 << 30)
-#define DIV_CPU_EN (0x1 << 25)
-#define DIV_CPU_RST (0x1 << 24)
-#define DIV_DDR_EN (0x1 << 19)
-#define DIV_DDR_RST (0x1 << 18)
-#define RST_DC_EN (0x1 << 5)
-#define RST_DC (0x1 << 4)
-#define RST_DDR_EN (0x1 << 3)
-#define RST_DDR (0x1 << 2)
-#define RST_CPU_EN (0x1 << 1)
-#define RST_CPU 0x1
+#define DIV_DC_EN BIT(31)
+#define DIV_DC_RST BIT(30)
+#define DIV_CPU_EN BIT(25)
+#define DIV_CPU_RST BIT(24)
+#define DIV_DDR_EN BIT(19)
+#define DIV_DDR_RST BIT(18)
+#define RST_DC_EN BIT(5)
+#define RST_DC BIT(4)
+#define RST_DDR_EN BIT(3)
+#define RST_DDR BIT(2)
+#define RST_CPU_EN BIT(1)
+#define RST_CPU BIT(0)
#define DIV_DC_SHIFT 26
#define DIV_CPU_SHIFT 20
diff --git a/arch/mips/include/asm/mach-loongson32/regs-mux.h b/arch/mips/include/asm/mach-loongson32/regs-mux.h
index 8302d92..7c394f9 100644
--- a/arch/mips/include/asm/mach-loongson32/regs-mux.h
+++ b/arch/mips/include/asm/mach-loongson32/regs-mux.h
@@ -19,49 +19,49 @@
#define LS1X_MUX_CTRL1 LS1X_MUX_REG(0x4)
/* MUX CTRL0 Register Bits */
-#define UART0_USE_PWM23 (0x1 << 28)
-#define UART0_USE_PWM01 (0x1 << 27)
-#define UART1_USE_LCD0_5_6_11 (0x1 << 26)
-#define I2C2_USE_CAN1 (0x1 << 25)
-#define I2C1_USE_CAN0 (0x1 << 24)
-#define NAND3_USE_UART5 (0x1 << 23)
-#define NAND3_USE_UART4 (0x1 << 22)
-#define NAND3_USE_UART1_DAT (0x1 << 21)
-#define NAND3_USE_UART1_CTS (0x1 << 20)
-#define NAND3_USE_PWM23 (0x1 << 19)
-#define NAND3_USE_PWM01 (0x1 << 18)
-#define NAND2_USE_UART5 (0x1 << 17)
-#define NAND2_USE_UART4 (0x1 << 16)
-#define NAND2_USE_UART1_DAT (0x1 << 15)
-#define NAND2_USE_UART1_CTS (0x1 << 14)
-#define NAND2_USE_PWM23 (0x1 << 13)
-#define NAND2_USE_PWM01 (0x1 << 12)
-#define NAND1_USE_UART5 (0x1 << 11)
-#define NAND1_USE_UART4 (0x1 << 10)
-#define NAND1_USE_UART1_DAT (0x1 << 9)
-#define NAND1_USE_UART1_CTS (0x1 << 8)
-#define NAND1_USE_PWM23 (0x1 << 7)
-#define NAND1_USE_PWM01 (0x1 << 6)
-#define GMAC1_USE_UART1 (0x1 << 4)
-#define GMAC1_USE_UART0 (0x1 << 3)
-#define LCD_USE_UART0_DAT (0x1 << 2)
-#define LCD_USE_UART15 (0x1 << 1)
-#define LCD_USE_UART0 0x1
+#define UART0_USE_PWM23 BIT(28)
+#define UART0_USE_PWM01 BIT(27)
+#define UART1_USE_LCD0_5_6_11 BIT(26)
+#define I2C2_USE_CAN1 BIT(25)
+#define I2C1_USE_CAN0 BIT(24)
+#define NAND3_USE_UART5 BIT(23)
+#define NAND3_USE_UART4 BIT(22)
+#define NAND3_USE_UART1_DAT BIT(21)
+#define NAND3_USE_UART1_CTS BIT(20)
+#define NAND3_USE_PWM23 BIT(19)
+#define NAND3_USE_PWM01 BIT(18)
+#define NAND2_USE_UART5 BIT(17)
+#define NAND2_USE_UART4 BIT(16)
+#define NAND2_USE_UART1_DAT BIT(15)
+#define NAND2_USE_UART1_CTS BIT(14)
+#define NAND2_USE_PWM23 BIT(13)
+#define NAND2_USE_PWM01 BIT(12)
+#define NAND1_USE_UART5 BIT(11)
+#define NAND1_USE_UART4 BIT(10)
+#define NAND1_USE_UART1_DAT BIT(9)
+#define NAND1_USE_UART1_CTS BIT(8)
+#define NAND1_USE_PWM23 BIT(7)
+#define NAND1_USE_PWM01 BIT(6)
+#define GMAC1_USE_UART1 BIT(4)
+#define GMAC1_USE_UART0 BIT(3)
+#define LCD_USE_UART0_DAT BIT(2)
+#define LCD_USE_UART15 BIT(1)
+#define LCD_USE_UART0 BIT(0)
/* MUX CTRL1 Register Bits */
-#define USB_RESET (0x1 << 31)
-#define SPI1_CS_USE_PWM01 (0x1 << 24)
-#define SPI1_USE_CAN (0x1 << 23)
-#define DISABLE_DDR_CONFSPACE (0x1 << 20)
-#define DDR32TO16EN (0x1 << 16)
-#define GMAC1_SHUT (0x1 << 13)
-#define GMAC0_SHUT (0x1 << 12)
-#define USB_SHUT (0x1 << 11)
-#define UART1_3_USE_CAN1 (0x1 << 5)
-#define UART1_2_USE_CAN0 (0x1 << 4)
-#define GMAC1_USE_TXCLK (0x1 << 3)
-#define GMAC0_USE_TXCLK (0x1 << 2)
-#define GMAC1_USE_PWM23 (0x1 << 1)
-#define GMAC0_USE_PWM01 0x1
+#define USB_RESET BIT(31)
+#define SPI1_CS_USE_PWM01 BIT(24)
+#define SPI1_USE_CAN BIT(23)
+#define DISABLE_DDR_CONFSPACE BIT(20)
+#define DDR32TO16EN BIT(16)
+#define GMAC1_SHUT BIT(13)
+#define GMAC0_SHUT BIT(12)
+#define USB_SHUT BIT(11)
+#define UART1_3_USE_CAN1 BIT(5)
+#define UART1_2_USE_CAN0 BIT(4)
+#define GMAC1_USE_TXCLK BIT(3)
+#define GMAC0_USE_TXCLK BIT(2)
+#define GMAC1_USE_PWM23 BIT(1)
+#define GMAC0_USE_PWM01 BIT(0)
#endif /* __ASM_MACH_LOONGSON32_REGS_MUX_H */
diff --git a/arch/mips/include/asm/mach-loongson32/regs-pwm.h b/arch/mips/include/asm/mach-loongson32/regs-pwm.h
index 69f174e..4119600 100644
--- a/arch/mips/include/asm/mach-loongson32/regs-pwm.h
+++ b/arch/mips/include/asm/mach-loongson32/regs-pwm.h
@@ -19,11 +19,11 @@
#define PWM_CTRL 0xc
/* PWM Control Register Bits */
-#define CNT_RST (0x1 << 7)
-#define INT_SR (0x1 << 6)
-#define INT_EN (0x1 << 5)
-#define PWM_SINGLE (0x1 << 4)
-#define PWM_OE (0x1 << 3)
-#define CNT_EN 0x1
+#define CNT_RST BIT(7)
+#define INT_SR BIT(6)
+#define INT_EN BIT(5)
+#define PWM_SINGLE BIT(4)
+#define PWM_OE BIT(3)
+#define CNT_EN BIT(0)
#endif /* __ASM_MACH_LOONGSON32_REGS_PWM_H */
diff --git a/arch/mips/include/asm/mach-loongson64/cpu-feature-overrides.h b/arch/mips/include/asm/mach-loongson64/cpu-feature-overrides.h
index 98963c2..89328a3d 100644
--- a/arch/mips/include/asm/mach-loongson64/cpu-feature-overrides.h
+++ b/arch/mips/include/asm/mach-loongson64/cpu-feature-overrides.h
@@ -16,11 +16,6 @@
#ifndef __ASM_MACH_LOONGSON64_CPU_FEATURE_OVERRIDES_H
#define __ASM_MACH_LOONGSON64_CPU_FEATURE_OVERRIDES_H
-#define cpu_dcache_line_size() 32
-#define cpu_icache_line_size() 32
-#define cpu_scache_line_size() 32
-
-
#define cpu_has_32fpr 1
#define cpu_has_3k_cache 0
#define cpu_has_4k_cache 1
@@ -31,24 +26,17 @@
#define cpu_has_counter 1
#define cpu_has_dc_aliases (PAGE_SIZE < 0x4000)
#define cpu_has_divec 0
-#define cpu_has_dsp 0
-#define cpu_has_dsp2 0
#define cpu_has_ejtag 0
-#define cpu_has_ic_fills_f_dc 0
#define cpu_has_inclusive_pcaches 1
#define cpu_has_llsc 1
#define cpu_has_mcheck 0
#define cpu_has_mdmx 0
#define cpu_has_mips16 0
-#define cpu_has_mips32r2 0
#define cpu_has_mips3d 0
-#define cpu_has_mips64r2 0
#define cpu_has_mipsmt 0
-#define cpu_has_prefetch 0
#define cpu_has_smartmips 0
#define cpu_has_tlb 1
#define cpu_has_tx39_cache 0
-#define cpu_has_userlocal 0
#define cpu_has_vce 0
#define cpu_has_veic 0
#define cpu_has_vint 0
@@ -56,6 +44,10 @@
#define cpu_has_watch 1
#define cpu_has_local_ebase 0
-#define cpu_has_wsbh IS_ENABLED(CONFIG_CPU_LOONGSON3)
+#ifdef CONFIG_CPU_LOONGSON3
+#define cpu_has_wsbh 1
+#define cpu_has_ic_fills_f_dc 1
+#define cpu_hwrena_impl_bits 0xc0000000
+#endif
#endif /* __ASM_MACH_LOONGSON64_CPU_FEATURE_OVERRIDES_H */
diff --git a/arch/mips/include/asm/mach-loongson64/kernel-entry-init.h b/arch/mips/include/asm/mach-loongson64/kernel-entry-init.h
index 3f2f84f..8393bc54 100644
--- a/arch/mips/include/asm/mach-loongson64/kernel-entry-init.h
+++ b/arch/mips/include/asm/mach-loongson64/kernel-entry-init.h
@@ -23,8 +23,15 @@
or t0, (0x1 << 7)
mtc0 t0, $16, 3
/* Set ELPA on LOONGSON3 pagegrain */
- li t0, (0x1 << 29)
+ mfc0 t0, $5, 1
+ or t0, (0x1 << 29)
mtc0 t0, $5, 1
+#ifdef CONFIG_LOONGSON3_ENHANCEMENT
+ /* Enable STFill Buffer */
+ mfc0 t0, $16, 6
+ or t0, 0x100
+ mtc0 t0, $16, 6
+#endif
_ehb
.set pop
#endif
@@ -42,8 +49,15 @@
or t0, (0x1 << 7)
mtc0 t0, $16, 3
/* Set ELPA on LOONGSON3 pagegrain */
- li t0, (0x1 << 29)
+ mfc0 t0, $5, 1
+ or t0, (0x1 << 29)
mtc0 t0, $5, 1
+#ifdef CONFIG_LOONGSON3_ENHANCEMENT
+ /* Enable STFill Buffer */
+ mfc0 t0, $16, 6
+ or t0, 0x100
+ mtc0 t0, $16, 6
+#endif
_ehb
.set pop
#endif
diff --git a/arch/mips/include/asm/mach-ralink/mt7620.h b/arch/mips/include/asm/mach-ralink/mt7620.h
index 455d406..a73350b 100644
--- a/arch/mips/include/asm/mach-ralink/mt7620.h
+++ b/arch/mips/include/asm/mach-ralink/mt7620.h
@@ -7,7 +7,7 @@
*
* Copyright (C) 2008-2011 Gabor Juhos <juhosg@openwrt.org>
* Copyright (C) 2008 Imre Kaloz <kaloz@openwrt.org>
- * Copyright (C) 2013 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2013 John Crispin <john@phrozen.org>
*/
#ifndef _MT7620_REGS_H_
@@ -72,6 +72,7 @@
#define SYSCFG0_DRAM_TYPE_SDRAM 0
#define SYSCFG0_DRAM_TYPE_DDR1 1
#define SYSCFG0_DRAM_TYPE_DDR2 2
+#define SYSCFG0_DRAM_TYPE_UNKNOWN 3
#define SYSCFG0_DRAM_TYPE_DDR2_MT7628 0
#define SYSCFG0_DRAM_TYPE_DDR1_MT7628 1
diff --git a/arch/mips/include/asm/mach-ralink/mt7621.h b/arch/mips/include/asm/mach-ralink/mt7621.h
index 610b61e..a672e06 100644
--- a/arch/mips/include/asm/mach-ralink/mt7621.h
+++ b/arch/mips/include/asm/mach-ralink/mt7621.h
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2015 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2015 John Crispin <john@phrozen.org>
*/
#ifndef _MT7621_REGS_H_
diff --git a/arch/mips/include/asm/mach-ralink/pinmux.h b/arch/mips/include/asm/mach-ralink/pinmux.h
index be106cb..ba8ac33 100644
--- a/arch/mips/include/asm/mach-ralink/pinmux.h
+++ b/arch/mips/include/asm/mach-ralink/pinmux.h
@@ -3,7 +3,7 @@
* it under the terms of the GNU General Public License version 2 as
* publishhed by the Free Software Foundation.
*
- * Copyright (C) 2012 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2012 John Crispin <john@phrozen.org>
*/
#ifndef _RT288X_PINMUX_H__
diff --git a/arch/mips/include/asm/mach-ralink/ralink_regs.h b/arch/mips/include/asm/mach-ralink/ralink_regs.h
index 4c9fba6..9df1a53 100644
--- a/arch/mips/include/asm/mach-ralink/ralink_regs.h
+++ b/arch/mips/include/asm/mach-ralink/ralink_regs.h
@@ -1,7 +1,7 @@
/*
* Ralink SoC register definitions
*
- * Copyright (C) 2013 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2013 John Crispin <john@phrozen.org>
* Copyright (C) 2008-2010 Gabor Juhos <juhosg@openwrt.org>
* Copyright (C) 2008 Imre Kaloz <kaloz@openwrt.org>
*
diff --git a/arch/mips/include/asm/mach-ralink/rt288x.h b/arch/mips/include/asm/mach-ralink/rt288x.h
index 03ad716..25ae104 100644
--- a/arch/mips/include/asm/mach-ralink/rt288x.h
+++ b/arch/mips/include/asm/mach-ralink/rt288x.h
@@ -7,7 +7,7 @@
*
* Copyright (C) 2008-2011 Gabor Juhos <juhosg@openwrt.org>
* Copyright (C) 2008 Imre Kaloz <kaloz@openwrt.org>
- * Copyright (C) 2013 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2013 John Crispin <john@phrozen.org>
*/
#ifndef _RT288X_REGS_H_
diff --git a/arch/mips/include/asm/mach-ralink/rt305x.h b/arch/mips/include/asm/mach-ralink/rt305x.h
index 2eea793..ac2d65c 100644
--- a/arch/mips/include/asm/mach-ralink/rt305x.h
+++ b/arch/mips/include/asm/mach-ralink/rt305x.h
@@ -7,7 +7,7 @@
*
* Copyright (C) 2008-2011 Gabor Juhos <juhosg@openwrt.org>
* Copyright (C) 2008 Imre Kaloz <kaloz@openwrt.org>
- * Copyright (C) 2013 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2013 John Crispin <john@phrozen.org>
*/
#ifndef _RT305X_REGS_H_
diff --git a/arch/mips/include/asm/mips-cm.h b/arch/mips/include/asm/mips-cm.h
index d463539..9411a4c 100644
--- a/arch/mips/include/asm/mips-cm.h
+++ b/arch/mips/include/asm/mips-cm.h
@@ -208,6 +208,7 @@
BUILD_CM_RW(sys_config2, MIPS_CM_GCB_OFS + 0x150)
BUILD_CM_RW(l2_pft_control, MIPS_CM_GCB_OFS + 0x300)
BUILD_CM_RW(l2_pft_control_b, MIPS_CM_GCB_OFS + 0x308)
+BUILD_CM_RW(bev_base, MIPS_CM_GCB_OFS + 0x680)
/* Core Local & Core Other register accessor functions */
BUILD_CM_Cx_RW(reset_release, 0x00)
@@ -290,8 +291,8 @@
#define CM_GCR_GIC_BASE_GICEN_MSK (_ULCAST_(0x1) << 0)
/* GCR_CPC_BASE register fields */
-#define CM_GCR_CPC_BASE_CPCBASE_SHF 17
-#define CM_GCR_CPC_BASE_CPCBASE_MSK (_ULCAST_(0x7fff) << 17)
+#define CM_GCR_CPC_BASE_CPCBASE_SHF 15
+#define CM_GCR_CPC_BASE_CPCBASE_MSK (_ULCAST_(0x1ffff) << 15)
#define CM_GCR_CPC_BASE_CPCEN_SHF 0
#define CM_GCR_CPC_BASE_CPCEN_MSK (_ULCAST_(0x1) << 0)
@@ -461,7 +462,10 @@
if (mips_cm_revision() >= CM_REV_CM3)
return read_gcr_sys_config2() & CM_GCR_SYS_CONFIG2_MAXVPW_MSK;
- return smp_num_siblings;
+ if (config_enabled(CONFIG_SMP))
+ return smp_num_siblings;
+
+ return 1;
}
/**
@@ -505,7 +509,7 @@
#else /* !CONFIG_MIPS_CM */
-static inline void mips_cm_lock_other(unsigned int core) { }
+static inline void mips_cm_lock_other(unsigned int core, unsigned int vp) { }
static inline void mips_cm_unlock_other(void) { }
#endif /* !CONFIG_MIPS_CM */
diff --git a/arch/mips/include/asm/mips-cpc.h b/arch/mips/include/asm/mips-cpc.h
index e090352..8c519f9 100644
--- a/arch/mips/include/asm/mips-cpc.h
+++ b/arch/mips/include/asm/mips-cpc.h
@@ -106,6 +106,9 @@
BUILD_CPC_Cx_RW(cmd, 0x00)
BUILD_CPC_Cx_RW(stat_conf, 0x08)
BUILD_CPC_Cx_RW(other, 0x10)
+BUILD_CPC_Cx_RW(vp_stop, 0x20)
+BUILD_CPC_Cx_RW(vp_run, 0x28)
+BUILD_CPC_Cx_RW(vp_running, 0x30)
/* CPC_Cx_CMD register fields */
#define CPC_Cx_CMD_SHF 0
diff --git a/arch/mips/include/asm/mipsregs.h b/arch/mips/include/asm/mipsregs.h
index 3ad19ad..25d0157 100644
--- a/arch/mips/include/asm/mipsregs.h
+++ b/arch/mips/include/asm/mipsregs.h
@@ -55,8 +55,14 @@
#define CP0_BADINSTR $8, 1
#define CP0_COUNT $9
#define CP0_ENTRYHI $10
+#define CP0_GUESTCTL1 $10, 4
+#define CP0_GUESTCTL2 $10, 5
+#define CP0_GUESTCTL3 $10, 6
#define CP0_COMPARE $11
+#define CP0_GUESTCTL0EXT $11, 4
#define CP0_STATUS $12
+#define CP0_GUESTCTL0 $12, 6
+#define CP0_GTOFFSET $12, 7
#define CP0_CAUSE $13
#define CP0_EPC $14
#define CP0_PRID $15
@@ -229,6 +235,8 @@
/* MIPS32/64 EntryHI bit definitions */
#define MIPS_ENTRYHI_EHINV (_ULCAST_(1) << 10)
+#define MIPS_ENTRYHI_ASIDX (_ULCAST_(0x3) << 8)
+#define MIPS_ENTRYHI_ASID (_ULCAST_(0xff) << 0)
/*
* R4x00 interrupt enable / cause bits
@@ -390,6 +398,8 @@
#define CAUSEF_IP7 (_ULCAST_(1) << 15)
#define CAUSEB_FDCI 21
#define CAUSEF_FDCI (_ULCAST_(1) << 21)
+#define CAUSEB_WP 22
+#define CAUSEF_WP (_ULCAST_(1) << 22)
#define CAUSEB_IV 23
#define CAUSEF_IV (_ULCAST_(1) << 23)
#define CAUSEB_PCI 26
@@ -611,7 +621,8 @@
#define MIPS_CONF4_MMUEXTDEF_MMUSIZEEXT (_ULCAST_(1) << 14)
#define MIPS_CONF4_MMUEXTDEF_FTLBSIZEEXT (_ULCAST_(2) << 14)
#define MIPS_CONF4_MMUEXTDEF_VTLBSIZEEXT (_ULCAST_(3) << 14)
-#define MIPS_CONF4_KSCREXIST (_ULCAST_(255) << 16)
+#define MIPS_CONF4_KSCREXIST_SHIFT (16)
+#define MIPS_CONF4_KSCREXIST (_ULCAST_(255) << MIPS_CONF4_KSCREXIST_SHIFT)
#define MIPS_CONF4_VTLBSIZEEXT_SHIFT (24)
#define MIPS_CONF4_VTLBSIZEEXT (_ULCAST_(15) << MIPS_CONF4_VTLBSIZEEXT_SHIFT)
#define MIPS_CONF4_AE (_ULCAST_(1) << 28)
@@ -623,6 +634,7 @@
#define MIPS_CONF5_MRP (_ULCAST_(1) << 3)
#define MIPS_CONF5_LLB (_ULCAST_(1) << 4)
#define MIPS_CONF5_MVH (_ULCAST_(1) << 5)
+#define MIPS_CONF5_VP (_ULCAST_(1) << 7)
#define MIPS_CONF5_FRE (_ULCAST_(1) << 8)
#define MIPS_CONF5_UFE (_ULCAST_(1) << 9)
#define MIPS_CONF5_MSAEN (_ULCAST_(1) << 27)
@@ -633,6 +645,8 @@
#define MIPS_CONF6_SYND (_ULCAST_(1) << 13)
/* proAptiv FTLB on/off bit */
#define MIPS_CONF6_FTLBEN (_ULCAST_(1) << 15)
+/* Loongson-3 FTLB on/off bit */
+#define MIPS_CONF6_FTLBDIS (_ULCAST_(1) << 22)
/* FTLB probability bits */
#define MIPS_CONF6_FTLBP_SHIFT (16)
@@ -645,12 +659,38 @@
/* FTLB probability bits for R6 */
#define MIPS_CONF7_FTLBP_SHIFT (18)
+/* WatchLo* register definitions */
+#define MIPS_WATCHLO_IRW (_ULCAST_(0x7) << 0)
+
+/* WatchHi* register definitions */
+#define MIPS_WATCHHI_M (_ULCAST_(1) << 31)
+#define MIPS_WATCHHI_G (_ULCAST_(1) << 30)
+#define MIPS_WATCHHI_WM (_ULCAST_(0x3) << 28)
+#define MIPS_WATCHHI_WM_R_RVA (_ULCAST_(0) << 28)
+#define MIPS_WATCHHI_WM_R_GPA (_ULCAST_(1) << 28)
+#define MIPS_WATCHHI_WM_G_GVA (_ULCAST_(2) << 28)
+#define MIPS_WATCHHI_EAS (_ULCAST_(0x3) << 24)
+#define MIPS_WATCHHI_ASID (_ULCAST_(0xff) << 16)
+#define MIPS_WATCHHI_MASK (_ULCAST_(0x1ff) << 3)
+#define MIPS_WATCHHI_I (_ULCAST_(1) << 2)
+#define MIPS_WATCHHI_R (_ULCAST_(1) << 1)
+#define MIPS_WATCHHI_W (_ULCAST_(1) << 0)
+#define MIPS_WATCHHI_IRW (_ULCAST_(0x7) << 0)
+
/* MAAR bit definitions */
#define MIPS_MAAR_ADDR ((BIT_ULL(BITS_PER_LONG - 12) - 1) << 12)
#define MIPS_MAAR_ADDR_SHIFT 12
#define MIPS_MAAR_S (_ULCAST_(1) << 1)
#define MIPS_MAAR_V (_ULCAST_(1) << 0)
+/* EBase bit definitions */
+#define MIPS_EBASE_CPUNUM_SHIFT 0
+#define MIPS_EBASE_CPUNUM (_ULCAST_(0x3ff) << 0)
+#define MIPS_EBASE_WG_SHIFT 11
+#define MIPS_EBASE_WG (_ULCAST_(1) << 11)
+#define MIPS_EBASE_BASE_SHIFT 12
+#define MIPS_EBASE_BASE (~_ULCAST_((1 << MIPS_EBASE_BASE_SHIFT) - 1))
+
/* CMGCRBase bit definitions */
#define MIPS_CMGCRB_BASE 11
#define MIPS_CMGCRF_BASE (~_ULCAST_((1 << MIPS_CMGCRB_BASE) - 1))
@@ -706,6 +746,94 @@
#define MIPS_PWCTL_PSN_SHIFT 0
#define MIPS_PWCTL_PSN_MASK 0x0000003f
+/* GuestCtl0 fields */
+#define MIPS_GCTL0_GM_SHIFT 31
+#define MIPS_GCTL0_GM (_ULCAST_(1) << MIPS_GCTL0_GM_SHIFT)
+#define MIPS_GCTL0_RI_SHIFT 30
+#define MIPS_GCTL0_RI (_ULCAST_(1) << MIPS_GCTL0_RI_SHIFT)
+#define MIPS_GCTL0_MC_SHIFT 29
+#define MIPS_GCTL0_MC (_ULCAST_(1) << MIPS_GCTL0_MC_SHIFT)
+#define MIPS_GCTL0_CP0_SHIFT 28
+#define MIPS_GCTL0_CP0 (_ULCAST_(1) << MIPS_GCTL0_CP0_SHIFT)
+#define MIPS_GCTL0_AT_SHIFT 26
+#define MIPS_GCTL0_AT (_ULCAST_(0x3) << MIPS_GCTL0_AT_SHIFT)
+#define MIPS_GCTL0_GT_SHIFT 25
+#define MIPS_GCTL0_GT (_ULCAST_(1) << MIPS_GCTL0_GT_SHIFT)
+#define MIPS_GCTL0_CG_SHIFT 24
+#define MIPS_GCTL0_CG (_ULCAST_(1) << MIPS_GCTL0_CG_SHIFT)
+#define MIPS_GCTL0_CF_SHIFT 23
+#define MIPS_GCTL0_CF (_ULCAST_(1) << MIPS_GCTL0_CF_SHIFT)
+#define MIPS_GCTL0_G1_SHIFT 22
+#define MIPS_GCTL0_G1 (_ULCAST_(1) << MIPS_GCTL0_G1_SHIFT)
+#define MIPS_GCTL0_G0E_SHIFT 19
+#define MIPS_GCTL0_G0E (_ULCAST_(1) << MIPS_GCTL0_G0E_SHIFT)
+#define MIPS_GCTL0_PT_SHIFT 18
+#define MIPS_GCTL0_PT (_ULCAST_(1) << MIPS_GCTL0_PT_SHIFT)
+#define MIPS_GCTL0_RAD_SHIFT 9
+#define MIPS_GCTL0_RAD (_ULCAST_(1) << MIPS_GCTL0_RAD_SHIFT)
+#define MIPS_GCTL0_DRG_SHIFT 8
+#define MIPS_GCTL0_DRG (_ULCAST_(1) << MIPS_GCTL0_DRG_SHIFT)
+#define MIPS_GCTL0_G2_SHIFT 7
+#define MIPS_GCTL0_G2 (_ULCAST_(1) << MIPS_GCTL0_G2_SHIFT)
+#define MIPS_GCTL0_GEXC_SHIFT 2
+#define MIPS_GCTL0_GEXC (_ULCAST_(0x1f) << MIPS_GCTL0_GEXC_SHIFT)
+#define MIPS_GCTL0_SFC2_SHIFT 1
+#define MIPS_GCTL0_SFC2 (_ULCAST_(1) << MIPS_GCTL0_SFC2_SHIFT)
+#define MIPS_GCTL0_SFC1_SHIFT 0
+#define MIPS_GCTL0_SFC1 (_ULCAST_(1) << MIPS_GCTL0_SFC1_SHIFT)
+
+/* GuestCtl0.AT Guest address translation control */
+#define MIPS_GCTL0_AT_ROOT 1 /* Guest MMU under Root control */
+#define MIPS_GCTL0_AT_GUEST 3 /* Guest MMU under Guest control */
+
+/* GuestCtl0.GExcCode Hypervisor exception cause codes */
+#define MIPS_GCTL0_GEXC_GPSI 0 /* Guest Privileged Sensitive Instruction */
+#define MIPS_GCTL0_GEXC_GSFC 1 /* Guest Software Field Change */
+#define MIPS_GCTL0_GEXC_HC 2 /* Hypercall */
+#define MIPS_GCTL0_GEXC_GRR 3 /* Guest Reserved Instruction Redirect */
+#define MIPS_GCTL0_GEXC_GVA 8 /* Guest Virtual Address available */
+#define MIPS_GCTL0_GEXC_GHFC 9 /* Guest Hardware Field Change */
+#define MIPS_GCTL0_GEXC_GPA 10 /* Guest Physical Address available */
+
+/* GuestCtl0Ext fields */
+#define MIPS_GCTL0EXT_RPW_SHIFT 8
+#define MIPS_GCTL0EXT_RPW (_ULCAST_(0x3) << MIPS_GCTL0EXT_RPW_SHIFT)
+#define MIPS_GCTL0EXT_NCC_SHIFT 6
+#define MIPS_GCTL0EXT_NCC (_ULCAST_(0x3) << MIPS_GCTL0EXT_NCC_SHIFT)
+#define MIPS_GCTL0EXT_CGI_SHIFT 4
+#define MIPS_GCTL0EXT_CGI (_ULCAST_(1) << MIPS_GCTL0EXT_CGI_SHIFT)
+#define MIPS_GCTL0EXT_FCD_SHIFT 3
+#define MIPS_GCTL0EXT_FCD (_ULCAST_(1) << MIPS_GCTL0EXT_FCD_SHIFT)
+#define MIPS_GCTL0EXT_OG_SHIFT 2
+#define MIPS_GCTL0EXT_OG (_ULCAST_(1) << MIPS_GCTL0EXT_OG_SHIFT)
+#define MIPS_GCTL0EXT_BG_SHIFT 1
+#define MIPS_GCTL0EXT_BG (_ULCAST_(1) << MIPS_GCTL0EXT_BG_SHIFT)
+#define MIPS_GCTL0EXT_MG_SHIFT 0
+#define MIPS_GCTL0EXT_MG (_ULCAST_(1) << MIPS_GCTL0EXT_MG_SHIFT)
+
+/* GuestCtl0Ext.RPW Root page walk configuration */
+#define MIPS_GCTL0EXT_RPW_BOTH 0 /* Root PW for GPA->RPA and RVA->RPA */
+#define MIPS_GCTL0EXT_RPW_GPA 2 /* Root PW for GPA->RPA */
+#define MIPS_GCTL0EXT_RPW_RVA 3 /* Root PW for RVA->RPA */
+
+/* GuestCtl0Ext.NCC Nested cache coherency attributes */
+#define MIPS_GCTL0EXT_NCC_IND 0 /* Guest CCA independent of Root CCA */
+#define MIPS_GCTL0EXT_NCC_MOD 1 /* Guest CCA modified by Root CCA */
+
+/* GuestCtl1 fields */
+#define MIPS_GCTL1_ID_SHIFT 0
+#define MIPS_GCTL1_ID_WIDTH 8
+#define MIPS_GCTL1_ID (_ULCAST_(0xff) << MIPS_GCTL1_ID_SHIFT)
+#define MIPS_GCTL1_RID_SHIFT 16
+#define MIPS_GCTL1_RID_WIDTH 8
+#define MIPS_GCTL1_RID (_ULCAST_(0xff) << MIPS_GCTL1_RID_SHIFT)
+#define MIPS_GCTL1_EID_SHIFT 24
+#define MIPS_GCTL1_EID_WIDTH 8
+#define MIPS_GCTL1_EID (_ULCAST_(0xff) << MIPS_GCTL1_EID_SHIFT)
+
+/* GuestID reserved for root context */
+#define MIPS_GCTL1_ROOT_GUESTID 0
+
/* CDMMBase register bit definitions */
#define MIPS_CDMMBASE_SIZE_SHIFT 0
#define MIPS_CDMMBASE_SIZE (_ULCAST_(511) << MIPS_CDMMBASE_SIZE_SHIFT)
@@ -757,6 +885,15 @@
/* Disable Branch Return Cache */
#define R10K_DIAG_D_BRC (_ULCAST_(1) << 22)
+/* Flush ITLB */
+#define LOONGSON_DIAG_ITLB (_ULCAST_(1) << 2)
+/* Flush DTLB */
+#define LOONGSON_DIAG_DTLB (_ULCAST_(1) << 3)
+/* Flush VTLB */
+#define LOONGSON_DIAG_VTLB (_ULCAST_(1) << 12)
+/* Flush FTLB */
+#define LOONGSON_DIAG_FTLB (_ULCAST_(1) << 13)
+
/*
* Coprocessor 1 (FPU) register names
*/
@@ -1186,9 +1323,15 @@
#define read_c0_context() __read_ulong_c0_register($4, 0)
#define write_c0_context(val) __write_ulong_c0_register($4, 0, val)
+#define read_c0_contextconfig() __read_32bit_c0_register($4, 1)
+#define write_c0_contextconfig(val) __write_32bit_c0_register($4, 1, val)
+
#define read_c0_userlocal() __read_ulong_c0_register($4, 2)
#define write_c0_userlocal(val) __write_ulong_c0_register($4, 2, val)
+#define read_c0_xcontextconfig() __read_ulong_c0_register($4, 3)
+#define write_c0_xcontextconfig(val) __write_ulong_c0_register($4, 3, val)
+
#define read_c0_pagemask() __read_32bit_c0_register($5, 0)
#define write_c0_pagemask(val) __write_32bit_c0_register($5, 0, val)
@@ -1206,6 +1349,9 @@
#define read_c0_badvaddr() __read_ulong_c0_register($8, 0)
#define write_c0_badvaddr(val) __write_ulong_c0_register($8, 0, val)
+#define read_c0_badinstr() __read_32bit_c0_register($8, 1)
+#define read_c0_badinstrp() __read_32bit_c0_register($8, 2)
+
#define read_c0_count() __read_32bit_c0_register($9, 0)
#define write_c0_count(val) __write_32bit_c0_register($9, 0, val)
@@ -1218,9 +1364,21 @@
#define read_c0_entryhi() __read_ulong_c0_register($10, 0)
#define write_c0_entryhi(val) __write_ulong_c0_register($10, 0, val)
+#define read_c0_guestctl1() __read_32bit_c0_register($10, 4)
+#define write_c0_guestctl1(val) __write_32bit_c0_register($10, 4, val)
+
+#define read_c0_guestctl2() __read_32bit_c0_register($10, 5)
+#define write_c0_guestctl2(val) __write_32bit_c0_register($10, 5, val)
+
+#define read_c0_guestctl3() __read_32bit_c0_register($10, 6)
+#define write_c0_guestctl3(val) __write_32bit_c0_register($10, 6, val)
+
#define read_c0_compare() __read_32bit_c0_register($11, 0)
#define write_c0_compare(val) __write_32bit_c0_register($11, 0, val)
+#define read_c0_guestctl0ext() __read_32bit_c0_register($11, 4)
+#define write_c0_guestctl0ext(val) __write_32bit_c0_register($11, 4, val)
+
#define read_c0_compare2() __read_32bit_c0_register($11, 6) /* pnx8550 */
#define write_c0_compare2(val) __write_32bit_c0_register($11, 6, val)
@@ -1231,6 +1389,12 @@
#define write_c0_status(val) __write_32bit_c0_register($12, 0, val)
+#define read_c0_guestctl0() __read_32bit_c0_register($12, 6)
+#define write_c0_guestctl0(val) __write_32bit_c0_register($12, 6, val)
+
+#define read_c0_gtoffset() __read_32bit_c0_register($12, 7)
+#define write_c0_gtoffset(val) __write_32bit_c0_register($12, 7, val)
+
#define read_c0_cause() __read_32bit_c0_register($13, 0)
#define write_c0_cause(val) __write_32bit_c0_register($13, 0, val)
@@ -1416,6 +1580,9 @@
#define read_c0_ebase() __read_32bit_c0_register($15, 1)
#define write_c0_ebase(val) __write_32bit_c0_register($15, 1, val)
+#define read_c0_ebase_64() __read_64bit_c0_register($15, 1)
+#define write_c0_ebase_64(val) __write_64bit_c0_register($15, 1, val)
+
#define read_c0_cdmmbase() __read_ulong_c0_register($15, 2)
#define write_c0_cdmmbase(val) __write_ulong_c0_register($15, 2, val)
@@ -1442,6 +1609,12 @@
#define read_c0_pwctl() __read_32bit_c0_register($6, 6)
#define write_c0_pwctl(val) __write_32bit_c0_register($6, 6, val)
+#define read_c0_pgd() __read_64bit_c0_register($9, 7)
+#define write_c0_pgd(val) __write_64bit_c0_register($9, 7, val)
+
+#define read_c0_kpgd() __read_64bit_c0_register($31, 7)
+#define write_c0_kpgd(val) __write_64bit_c0_register($31, 7, val)
+
/* Cavium OCTEON (cnMIPS) */
#define read_c0_cvmcount() __read_ulong_c0_register($9, 6)
#define write_c0_cvmcount(val) __write_ulong_c0_register($9, 6, val)
@@ -1507,6 +1680,317 @@
#define write_c0_brcm_sleepcount(val) __write_32bit_c0_register($22, 7, val)
/*
+ * Macros to access the guest system control coprocessor
+ */
+
+#ifdef TOOLCHAIN_SUPPORTS_VIRT
+
+#define __read_32bit_gc0_register(source, sel) \
+({ int __res; \
+ __asm__ __volatile__( \
+ ".set\tpush\n\t" \
+ ".set\tmips32r2\n\t" \
+ ".set\tvirt\n\t" \
+ "mfgc0\t%0, $%1, %2\n\t" \
+ ".set\tpop" \
+ : "=r" (__res) \
+ : "i" (source), "i" (sel)); \
+ __res; \
+})
+
+#define __read_64bit_gc0_register(source, sel) \
+({ unsigned long long __res; \
+ __asm__ __volatile__( \
+ ".set\tpush\n\t" \
+ ".set\tmips64r2\n\t" \
+ ".set\tvirt\n\t" \
+ "dmfgc0\t%0, $%1, %2\n\t" \
+ ".set\tpop" \
+ : "=r" (__res) \
+ : "i" (source), "i" (sel)); \
+ __res; \
+})
+
+#define __write_32bit_gc0_register(register, sel, value) \
+do { \
+ __asm__ __volatile__( \
+ ".set\tpush\n\t" \
+ ".set\tmips32r2\n\t" \
+ ".set\tvirt\n\t" \
+ "mtgc0\t%z0, $%1, %2\n\t" \
+ ".set\tpop" \
+ : : "Jr" ((unsigned int)(value)), \
+ "i" (register), "i" (sel)); \
+} while (0)
+
+#define __write_64bit_gc0_register(register, sel, value) \
+do { \
+ __asm__ __volatile__( \
+ ".set\tpush\n\t" \
+ ".set\tmips64r2\n\t" \
+ ".set\tvirt\n\t" \
+ "dmtgc0\t%z0, $%1, %2\n\t" \
+ ".set\tpop" \
+ : : "Jr" (value), \
+ "i" (register), "i" (sel)); \
+} while (0)
+
+#else /* TOOLCHAIN_SUPPORTS_VIRT */
+
+#define __read_32bit_gc0_register(source, sel) \
+({ int __res; \
+ __asm__ __volatile__( \
+ ".set\tpush\n\t" \
+ ".set\tnoat\n\t" \
+ "# mfgc0\t$1, $%1, %2\n\t" \
+ ".word\t(0x40610000 | %1 << 11 | %2)\n\t" \
+ "move\t%0, $1\n\t" \
+ ".set\tpop" \
+ : "=r" (__res) \
+ : "i" (source), "i" (sel)); \
+ __res; \
+})
+
+#define __read_64bit_gc0_register(source, sel) \
+({ unsigned long long __res; \
+ __asm__ __volatile__( \
+ ".set\tpush\n\t" \
+ ".set\tnoat\n\t" \
+ "# dmfgc0\t$1, $%1, %2\n\t" \
+ ".word\t(0x40610100 | %1 << 11 | %2)\n\t" \
+ "move\t%0, $1\n\t" \
+ ".set\tpop" \
+ : "=r" (__res) \
+ : "i" (source), "i" (sel)); \
+ __res; \
+})
+
+#define __write_32bit_gc0_register(register, sel, value) \
+do { \
+ __asm__ __volatile__( \
+ ".set\tpush\n\t" \
+ ".set\tnoat\n\t" \
+ "move\t$1, %0\n\t" \
+ "# mtgc0\t$1, $%1, %2\n\t" \
+ ".word\t(0x40610200 | %1 << 11 | %2)\n\t" \
+ ".set\tpop" \
+ : : "Jr" ((unsigned int)(value)), \
+ "i" (register), "i" (sel)); \
+} while (0)
+
+#define __write_64bit_gc0_register(register, sel, value) \
+do { \
+ __asm__ __volatile__( \
+ ".set\tpush\n\t" \
+ ".set\tnoat\n\t" \
+ "move\t$1, %0\n\t" \
+ "# dmtgc0\t$1, $%1, %2\n\t" \
+ ".word\t(0x40610300 | %1 << 11 | %2)\n\t" \
+ ".set\tpop" \
+ : : "Jr" (value), \
+ "i" (register), "i" (sel)); \
+} while (0)
+
+#endif /* !TOOLCHAIN_SUPPORTS_VIRT */
+
+#define __read_ulong_gc0_register(reg, sel) \
+ ((sizeof(unsigned long) == 4) ? \
+ (unsigned long) __read_32bit_gc0_register(reg, sel) : \
+ (unsigned long) __read_64bit_gc0_register(reg, sel))
+
+#define __write_ulong_gc0_register(reg, sel, val) \
+do { \
+ if (sizeof(unsigned long) == 4) \
+ __write_32bit_gc0_register(reg, sel, val); \
+ else \
+ __write_64bit_gc0_register(reg, sel, val); \
+} while (0)
+
+#define read_gc0_index() __read_32bit_gc0_register(0, 0)
+#define write_gc0_index(val) __write_32bit_gc0_register(0, 0, val)
+
+#define read_gc0_entrylo0() __read_ulong_gc0_register(2, 0)
+#define write_gc0_entrylo0(val) __write_ulong_gc0_register(2, 0, val)
+
+#define read_gc0_entrylo1() __read_ulong_gc0_register(3, 0)
+#define write_gc0_entrylo1(val) __write_ulong_gc0_register(3, 0, val)
+
+#define read_gc0_context() __read_ulong_gc0_register(4, 0)
+#define write_gc0_context(val) __write_ulong_gc0_register(4, 0, val)
+
+#define read_gc0_contextconfig() __read_32bit_gc0_register(4, 1)
+#define write_gc0_contextconfig(val) __write_32bit_gc0_register(4, 1, val)
+
+#define read_gc0_userlocal() __read_ulong_gc0_register(4, 2)
+#define write_gc0_userlocal(val) __write_ulong_gc0_register(4, 2, val)
+
+#define read_gc0_xcontextconfig() __read_ulong_gc0_register(4, 3)
+#define write_gc0_xcontextconfig(val) __write_ulong_gc0_register(4, 3, val)
+
+#define read_gc0_pagemask() __read_32bit_gc0_register(5, 0)
+#define write_gc0_pagemask(val) __write_32bit_gc0_register(5, 0, val)
+
+#define read_gc0_pagegrain() __read_32bit_gc0_register(5, 1)
+#define write_gc0_pagegrain(val) __write_32bit_gc0_register(5, 1, val)
+
+#define read_gc0_segctl0() __read_ulong_gc0_register(5, 2)
+#define write_gc0_segctl0(val) __write_ulong_gc0_register(5, 2, val)
+
+#define read_gc0_segctl1() __read_ulong_gc0_register(5, 3)
+#define write_gc0_segctl1(val) __write_ulong_gc0_register(5, 3, val)
+
+#define read_gc0_segctl2() __read_ulong_gc0_register(5, 4)
+#define write_gc0_segctl2(val) __write_ulong_gc0_register(5, 4, val)
+
+#define read_gc0_pwbase() __read_ulong_gc0_register(5, 5)
+#define write_gc0_pwbase(val) __write_ulong_gc0_register(5, 5, val)
+
+#define read_gc0_pwfield() __read_ulong_gc0_register(5, 6)
+#define write_gc0_pwfield(val) __write_ulong_gc0_register(5, 6, val)
+
+#define read_gc0_pwsize() __read_ulong_gc0_register(5, 7)
+#define write_gc0_pwsize(val) __write_ulong_gc0_register(5, 7, val)
+
+#define read_gc0_wired() __read_32bit_gc0_register(6, 0)
+#define write_gc0_wired(val) __write_32bit_gc0_register(6, 0, val)
+
+#define read_gc0_pwctl() __read_32bit_gc0_register(6, 6)
+#define write_gc0_pwctl(val) __write_32bit_gc0_register(6, 6, val)
+
+#define read_gc0_hwrena() __read_32bit_gc0_register(7, 0)
+#define write_gc0_hwrena(val) __write_32bit_gc0_register(7, 0, val)
+
+#define read_gc0_badvaddr() __read_ulong_gc0_register(8, 0)
+#define write_gc0_badvaddr(val) __write_ulong_gc0_register(8, 0, val)
+
+#define read_gc0_badinstr() __read_32bit_gc0_register(8, 1)
+#define write_gc0_badinstr(val) __write_32bit_gc0_register(8, 1, val)
+
+#define read_gc0_badinstrp() __read_32bit_gc0_register(8, 2)
+#define write_gc0_badinstrp(val) __write_32bit_gc0_register(8, 2, val)
+
+#define read_gc0_count() __read_32bit_gc0_register(9, 0)
+
+#define read_gc0_entryhi() __read_ulong_gc0_register(10, 0)
+#define write_gc0_entryhi(val) __write_ulong_gc0_register(10, 0, val)
+
+#define read_gc0_compare() __read_32bit_gc0_register(11, 0)
+#define write_gc0_compare(val) __write_32bit_gc0_register(11, 0, val)
+
+#define read_gc0_status() __read_32bit_gc0_register(12, 0)
+#define write_gc0_status(val) __write_32bit_gc0_register(12, 0, val)
+
+#define read_gc0_intctl() __read_32bit_gc0_register(12, 1)
+#define write_gc0_intctl(val) __write_32bit_gc0_register(12, 1, val)
+
+#define read_gc0_cause() __read_32bit_gc0_register(13, 0)
+#define write_gc0_cause(val) __write_32bit_gc0_register(13, 0, val)
+
+#define read_gc0_epc() __read_ulong_gc0_register(14, 0)
+#define write_gc0_epc(val) __write_ulong_gc0_register(14, 0, val)
+
+#define read_gc0_ebase() __read_32bit_gc0_register(15, 1)
+#define write_gc0_ebase(val) __write_32bit_gc0_register(15, 1, val)
+
+#define read_gc0_ebase_64() __read_64bit_gc0_register(15, 1)
+#define write_gc0_ebase_64(val) __write_64bit_gc0_register(15, 1, val)
+
+#define read_gc0_config() __read_32bit_gc0_register(16, 0)
+#define read_gc0_config1() __read_32bit_gc0_register(16, 1)
+#define read_gc0_config2() __read_32bit_gc0_register(16, 2)
+#define read_gc0_config3() __read_32bit_gc0_register(16, 3)
+#define read_gc0_config4() __read_32bit_gc0_register(16, 4)
+#define read_gc0_config5() __read_32bit_gc0_register(16, 5)
+#define read_gc0_config6() __read_32bit_gc0_register(16, 6)
+#define read_gc0_config7() __read_32bit_gc0_register(16, 7)
+#define write_gc0_config(val) __write_32bit_gc0_register(16, 0, val)
+#define write_gc0_config1(val) __write_32bit_gc0_register(16, 1, val)
+#define write_gc0_config2(val) __write_32bit_gc0_register(16, 2, val)
+#define write_gc0_config3(val) __write_32bit_gc0_register(16, 3, val)
+#define write_gc0_config4(val) __write_32bit_gc0_register(16, 4, val)
+#define write_gc0_config5(val) __write_32bit_gc0_register(16, 5, val)
+#define write_gc0_config6(val) __write_32bit_gc0_register(16, 6, val)
+#define write_gc0_config7(val) __write_32bit_gc0_register(16, 7, val)
+
+#define read_gc0_watchlo0() __read_ulong_gc0_register(18, 0)
+#define read_gc0_watchlo1() __read_ulong_gc0_register(18, 1)
+#define read_gc0_watchlo2() __read_ulong_gc0_register(18, 2)
+#define read_gc0_watchlo3() __read_ulong_gc0_register(18, 3)
+#define read_gc0_watchlo4() __read_ulong_gc0_register(18, 4)
+#define read_gc0_watchlo5() __read_ulong_gc0_register(18, 5)
+#define read_gc0_watchlo6() __read_ulong_gc0_register(18, 6)
+#define read_gc0_watchlo7() __read_ulong_gc0_register(18, 7)
+#define write_gc0_watchlo0(val) __write_ulong_gc0_register(18, 0, val)
+#define write_gc0_watchlo1(val) __write_ulong_gc0_register(18, 1, val)
+#define write_gc0_watchlo2(val) __write_ulong_gc0_register(18, 2, val)
+#define write_gc0_watchlo3(val) __write_ulong_gc0_register(18, 3, val)
+#define write_gc0_watchlo4(val) __write_ulong_gc0_register(18, 4, val)
+#define write_gc0_watchlo5(val) __write_ulong_gc0_register(18, 5, val)
+#define write_gc0_watchlo6(val) __write_ulong_gc0_register(18, 6, val)
+#define write_gc0_watchlo7(val) __write_ulong_gc0_register(18, 7, val)
+
+#define read_gc0_watchhi0() __read_32bit_gc0_register(19, 0)
+#define read_gc0_watchhi1() __read_32bit_gc0_register(19, 1)
+#define read_gc0_watchhi2() __read_32bit_gc0_register(19, 2)
+#define read_gc0_watchhi3() __read_32bit_gc0_register(19, 3)
+#define read_gc0_watchhi4() __read_32bit_gc0_register(19, 4)
+#define read_gc0_watchhi5() __read_32bit_gc0_register(19, 5)
+#define read_gc0_watchhi6() __read_32bit_gc0_register(19, 6)
+#define read_gc0_watchhi7() __read_32bit_gc0_register(19, 7)
+#define write_gc0_watchhi0(val) __write_32bit_gc0_register(19, 0, val)
+#define write_gc0_watchhi1(val) __write_32bit_gc0_register(19, 1, val)
+#define write_gc0_watchhi2(val) __write_32bit_gc0_register(19, 2, val)
+#define write_gc0_watchhi3(val) __write_32bit_gc0_register(19, 3, val)
+#define write_gc0_watchhi4(val) __write_32bit_gc0_register(19, 4, val)
+#define write_gc0_watchhi5(val) __write_32bit_gc0_register(19, 5, val)
+#define write_gc0_watchhi6(val) __write_32bit_gc0_register(19, 6, val)
+#define write_gc0_watchhi7(val) __write_32bit_gc0_register(19, 7, val)
+
+#define read_gc0_xcontext() __read_ulong_gc0_register(20, 0)
+#define write_gc0_xcontext(val) __write_ulong_gc0_register(20, 0, val)
+
+#define read_gc0_perfctrl0() __read_32bit_gc0_register(25, 0)
+#define write_gc0_perfctrl0(val) __write_32bit_gc0_register(25, 0, val)
+#define read_gc0_perfcntr0() __read_32bit_gc0_register(25, 1)
+#define write_gc0_perfcntr0(val) __write_32bit_gc0_register(25, 1, val)
+#define read_gc0_perfcntr0_64() __read_64bit_gc0_register(25, 1)
+#define write_gc0_perfcntr0_64(val) __write_64bit_gc0_register(25, 1, val)
+#define read_gc0_perfctrl1() __read_32bit_gc0_register(25, 2)
+#define write_gc0_perfctrl1(val) __write_32bit_gc0_register(25, 2, val)
+#define read_gc0_perfcntr1() __read_32bit_gc0_register(25, 3)
+#define write_gc0_perfcntr1(val) __write_32bit_gc0_register(25, 3, val)
+#define read_gc0_perfcntr1_64() __read_64bit_gc0_register(25, 3)
+#define write_gc0_perfcntr1_64(val) __write_64bit_gc0_register(25, 3, val)
+#define read_gc0_perfctrl2() __read_32bit_gc0_register(25, 4)
+#define write_gc0_perfctrl2(val) __write_32bit_gc0_register(25, 4, val)
+#define read_gc0_perfcntr2() __read_32bit_gc0_register(25, 5)
+#define write_gc0_perfcntr2(val) __write_32bit_gc0_register(25, 5, val)
+#define read_gc0_perfcntr2_64() __read_64bit_gc0_register(25, 5)
+#define write_gc0_perfcntr2_64(val) __write_64bit_gc0_register(25, 5, val)
+#define read_gc0_perfctrl3() __read_32bit_gc0_register(25, 6)
+#define write_gc0_perfctrl3(val) __write_32bit_gc0_register(25, 6, val)
+#define read_gc0_perfcntr3() __read_32bit_gc0_register(25, 7)
+#define write_gc0_perfcntr3(val) __write_32bit_gc0_register(25, 7, val)
+#define read_gc0_perfcntr3_64() __read_64bit_gc0_register(25, 7)
+#define write_gc0_perfcntr3_64(val) __write_64bit_gc0_register(25, 7, val)
+
+#define read_gc0_errorepc() __read_ulong_gc0_register(30, 0)
+#define write_gc0_errorepc(val) __write_ulong_gc0_register(30, 0, val)
+
+#define read_gc0_kscratch1() __read_ulong_gc0_register(31, 2)
+#define read_gc0_kscratch2() __read_ulong_gc0_register(31, 3)
+#define read_gc0_kscratch3() __read_ulong_gc0_register(31, 4)
+#define read_gc0_kscratch4() __read_ulong_gc0_register(31, 5)
+#define read_gc0_kscratch5() __read_ulong_gc0_register(31, 6)
+#define read_gc0_kscratch6() __read_ulong_gc0_register(31, 7)
+#define write_gc0_kscratch1(val) __write_ulong_gc0_register(31, 2, val)
+#define write_gc0_kscratch2(val) __write_ulong_gc0_register(31, 3, val)
+#define write_gc0_kscratch3(val) __write_ulong_gc0_register(31, 4, val)
+#define write_gc0_kscratch4(val) __write_ulong_gc0_register(31, 5, val)
+#define write_gc0_kscratch5(val) __write_ulong_gc0_register(31, 6, val)
+#define write_gc0_kscratch6(val) __write_ulong_gc0_register(31, 7, val)
+
+/*
* Macros to access the floating point coprocessor control registers
*/
#define _read_32bit_cp1_register(source, gas_hardfloat) \
@@ -2001,47 +2485,159 @@
".set reorder");
}
+#ifdef TOOLCHAIN_SUPPORTS_VIRT
+
/*
- * Manipulate bits in a c0 register.
+ * Guest TLB operations.
+ *
+ * It is responsibility of the caller to take care of any TLB hazards.
*/
-#define __BUILD_SET_C0(name) \
+static inline void guest_tlb_probe(void)
+{
+ __asm__ __volatile__(
+ ".set push\n\t"
+ ".set noreorder\n\t"
+ ".set virt\n\t"
+ "tlbgp\n\t"
+ ".set pop");
+}
+
+static inline void guest_tlb_read(void)
+{
+ __asm__ __volatile__(
+ ".set push\n\t"
+ ".set noreorder\n\t"
+ ".set virt\n\t"
+ "tlbgr\n\t"
+ ".set pop");
+}
+
+static inline void guest_tlb_write_indexed(void)
+{
+ __asm__ __volatile__(
+ ".set push\n\t"
+ ".set noreorder\n\t"
+ ".set virt\n\t"
+ "tlbgwi\n\t"
+ ".set pop");
+}
+
+static inline void guest_tlb_write_random(void)
+{
+ __asm__ __volatile__(
+ ".set push\n\t"
+ ".set noreorder\n\t"
+ ".set virt\n\t"
+ "tlbgwr\n\t"
+ ".set pop");
+}
+
+/*
+ * Guest TLB Invalidate Flush
+ */
+static inline void guest_tlbinvf(void)
+{
+ __asm__ __volatile__(
+ ".set push\n\t"
+ ".set noreorder\n\t"
+ ".set virt\n\t"
+ "tlbginvf\n\t"
+ ".set pop");
+}
+
+#else /* TOOLCHAIN_SUPPORTS_VIRT */
+
+/*
+ * Guest TLB operations.
+ *
+ * It is responsibility of the caller to take care of any TLB hazards.
+ */
+static inline void guest_tlb_probe(void)
+{
+ __asm__ __volatile__(
+ "# tlbgp\n\t"
+ ".word 0x42000010");
+}
+
+static inline void guest_tlb_read(void)
+{
+ __asm__ __volatile__(
+ "# tlbgr\n\t"
+ ".word 0x42000009");
+}
+
+static inline void guest_tlb_write_indexed(void)
+{
+ __asm__ __volatile__(
+ "# tlbgwi\n\t"
+ ".word 0x4200000a");
+}
+
+static inline void guest_tlb_write_random(void)
+{
+ __asm__ __volatile__(
+ "# tlbgwr\n\t"
+ ".word 0x4200000e");
+}
+
+/*
+ * Guest TLB Invalidate Flush
+ */
+static inline void guest_tlbinvf(void)
+{
+ __asm__ __volatile__(
+ "# tlbginvf\n\t"
+ ".word 0x4200000c");
+}
+
+#endif /* !TOOLCHAIN_SUPPORTS_VIRT */
+
+/*
+ * Manipulate bits in a register.
+ */
+#define __BUILD_SET_COMMON(name) \
static inline unsigned int \
-set_c0_##name(unsigned int set) \
+set_##name(unsigned int set) \
{ \
unsigned int res, new; \
\
- res = read_c0_##name(); \
+ res = read_##name(); \
new = res | set; \
- write_c0_##name(new); \
+ write_##name(new); \
\
return res; \
} \
\
static inline unsigned int \
-clear_c0_##name(unsigned int clear) \
+clear_##name(unsigned int clear) \
{ \
unsigned int res, new; \
\
- res = read_c0_##name(); \
+ res = read_##name(); \
new = res & ~clear; \
- write_c0_##name(new); \
+ write_##name(new); \
\
return res; \
} \
\
static inline unsigned int \
-change_c0_##name(unsigned int change, unsigned int val) \
+change_##name(unsigned int change, unsigned int val) \
{ \
unsigned int res, new; \
\
- res = read_c0_##name(); \
+ res = read_##name(); \
new = res & ~change; \
new |= (val & change); \
- write_c0_##name(new); \
+ write_##name(new); \
\
return res; \
}
+/*
+ * Manipulate bits in a c0 register.
+ */
+#define __BUILD_SET_C0(name) __BUILD_SET_COMMON(c0_##name)
+
__BUILD_SET_C0(status)
__BUILD_SET_C0(cause)
__BUILD_SET_C0(config)
@@ -2050,6 +2646,11 @@
__BUILD_SET_C0(intctl)
__BUILD_SET_C0(srsmap)
__BUILD_SET_C0(pagegrain)
+__BUILD_SET_C0(guestctl0)
+__BUILD_SET_C0(guestctl0ext)
+__BUILD_SET_C0(guestctl1)
+__BUILD_SET_C0(guestctl2)
+__BUILD_SET_C0(guestctl3)
__BUILD_SET_C0(brcm_config_0)
__BUILD_SET_C0(brcm_bus_pll)
__BUILD_SET_C0(brcm_reset)
@@ -2059,12 +2660,21 @@
__BUILD_SET_C0(brcm_mode)
/*
+ * Manipulate bits in a guest c0 register.
+ */
+#define __BUILD_SET_GC0(name) __BUILD_SET_COMMON(gc0_##name)
+
+__BUILD_SET_GC0(status)
+__BUILD_SET_GC0(cause)
+__BUILD_SET_GC0(ebase)
+
+/*
* Return low 10 bits of ebase.
* Note that under KVM (MIPSVZ) this returns vcpu id.
*/
static inline unsigned int get_ebase_cpunum(void)
{
- return read_c0_ebase() & 0x3ff;
+ return read_c0_ebase() & MIPS_EBASE_CPUNUM;
}
#endif /* !__ASSEMBLY__ */
diff --git a/arch/mips/include/asm/mmu_context.h b/arch/mips/include/asm/mmu_context.h
index 45914b5..fc57e13 100644
--- a/arch/mips/include/asm/mmu_context.h
+++ b/arch/mips/include/asm/mmu_context.h
@@ -65,37 +65,32 @@
back_to_back_c0_hazard(); \
TLBMISS_HANDLER_SETUP_PGD(swapper_pg_dir)
#endif /* CONFIG_MIPS_PGD_C0_CONTEXT*/
-#if defined(CONFIG_CPU_R3000) || defined(CONFIG_CPU_TX39XX)
-
-#define ASID_INC 0x40
-#define ASID_MASK 0xfc0
-
-#elif defined(CONFIG_CPU_R8000)
-
-#define ASID_INC 0x10
-#define ASID_MASK 0xff0
-
-#else /* FIXME: not correct for R6000 */
-
-#define ASID_INC 0x1
-#define ASID_MASK 0xff
-
-#endif
-
-#define cpu_context(cpu, mm) ((mm)->context.asid[cpu])
-#define cpu_asid(cpu, mm) (cpu_context((cpu), (mm)) & ASID_MASK)
-#define asid_cache(cpu) (cpu_data[cpu].asid_cache)
-
-static inline void enter_lazy_tlb(struct mm_struct *mm, struct task_struct *tsk)
-{
-}
/*
* All unused by hardware upper bits will be considered
* as a software asid extension.
*/
-#define ASID_VERSION_MASK ((unsigned long)~(ASID_MASK|(ASID_MASK-1)))
-#define ASID_FIRST_VERSION ((unsigned long)(~ASID_VERSION_MASK) + 1)
+static unsigned long asid_version_mask(unsigned int cpu)
+{
+ unsigned long asid_mask = cpu_asid_mask(&cpu_data[cpu]);
+
+ return ~(asid_mask | (asid_mask - 1));
+}
+
+static unsigned long asid_first_version(unsigned int cpu)
+{
+ return ~asid_version_mask(cpu) + 1;
+}
+
+#define cpu_context(cpu, mm) ((mm)->context.asid[cpu])
+#define asid_cache(cpu) (cpu_data[cpu].asid_cache)
+#define cpu_asid(cpu, mm) \
+ (cpu_context((cpu), (mm)) & cpu_asid_mask(&cpu_data[cpu]))
+
+static inline void enter_lazy_tlb(struct mm_struct *mm, struct task_struct *tsk)
+{
+}
+
/* Normal, classic MIPS get_new_mmu_context */
static inline void
@@ -104,7 +99,7 @@
extern void kvm_local_flush_tlb_all(void);
unsigned long asid = asid_cache(cpu);
- if (! ((asid += ASID_INC) & ASID_MASK) ) {
+ if (!((asid += cpu_asid_inc()) & cpu_asid_mask(&cpu_data[cpu]))) {
if (cpu_has_vtag_icache)
flush_icache_all();
#ifdef CONFIG_KVM
@@ -113,7 +108,7 @@
local_flush_tlb_all(); /* start new asid cycle */
#endif
if (!asid) /* fix version if needed */
- asid = ASID_FIRST_VERSION;
+ asid = asid_first_version(cpu);
}
cpu_context(cpu, mm) = asid_cache(cpu) = asid;
@@ -145,7 +140,7 @@
htw_stop();
/* Check if our ASID is of an older version and thus invalid */
- if ((cpu_context(cpu, next) ^ asid_cache(cpu)) & ASID_VERSION_MASK)
+ if ((cpu_context(cpu, next) ^ asid_cache(cpu)) & asid_version_mask(cpu))
get_new_mmu_context(next, cpu);
write_c0_entryhi(cpu_asid(cpu, next));
TLBMISS_HANDLER_SETUP_PGD(next->pgd);
diff --git a/arch/mips/include/asm/msa.h b/arch/mips/include/asm/msa.h
index bbb85fe..6e4effa 100644
--- a/arch/mips/include/asm/msa.h
+++ b/arch/mips/include/asm/msa.h
@@ -147,6 +147,19 @@
_restore_msa(t);
}
+static inline void init_msa_upper(void)
+{
+ /*
+ * Check cpu_has_msa only if it's a constant. This will allow the
+ * compiler to optimise out code for CPUs without MSA without adding
+ * an extra redundant check for CPUs with MSA.
+ */
+ if (__builtin_constant_p(cpu_has_msa) && !cpu_has_msa)
+ return;
+
+ _init_msa_upper();
+}
+
#ifdef TOOLCHAIN_SUPPORTS_MSA
#define __BUILD_MSA_CTL_REG(name, cs) \
diff --git a/arch/mips/include/asm/octeon/cvmx-bootinfo.h b/arch/mips/include/asm/octeon/cvmx-bootinfo.h
index d92cf59..6278776 100644
--- a/arch/mips/include/asm/octeon/cvmx-bootinfo.h
+++ b/arch/mips/include/asm/octeon/cvmx-bootinfo.h
@@ -32,6 +32,8 @@
#ifndef __CVMX_BOOTINFO_H__
#define __CVMX_BOOTINFO_H__
+#include "cvmx-coremask.h"
+
/*
* Current major and minor versions of the CVMX bootinfo block that is
* passed from the bootloader to the application. This is versioned
@@ -39,7 +41,7 @@
* versions.
*/
#define CVMX_BOOTINFO_MAJ_VER 1
-#define CVMX_BOOTINFO_MIN_VER 3
+#define CVMX_BOOTINFO_MIN_VER 4
#if (CVMX_BOOTINFO_MAJ_VER == 1)
#define CVMX_BOOTINFO_OCTEON_SERIAL_LEN 20
@@ -124,6 +126,13 @@
*/
uint64_t fdt_addr;
#endif
+#if (CVMX_BOOTINFO_MIN_VER >= 4)
+ /*
+ * Coremask used for processors with more than 32 cores
+ * or with OCI. This replaces core_mask.
+ */
+ struct cvmx_coremask ext_core_mask;
+#endif
#else /* __BIG_ENDIAN */
/*
* Little-Endian: When the CPU mode is switched to
@@ -177,6 +186,9 @@
#if (CVMX_BOOTINFO_MIN_VER >= 3)
uint64_t fdt_addr;
#endif
+#if (CVMX_BOOTINFO_MIN_VER >= 4)
+ struct cvmx_coremask ext_core_mask;
+#endif
#endif
};
@@ -388,7 +400,7 @@
ENUM_BRD_TYPE_CASE(CVMX_BOARD_TYPE_KONTRON_S1901)
ENUM_BRD_TYPE_CASE(CVMX_BOARD_TYPE_CUST_PRIVATE_MAX)
}
- return "Unsupported Board";
+ return NULL;
}
#define ENUM_CHIP_TYPE_CASE(x) \
diff --git a/arch/mips/include/asm/octeon/cvmx-ciu3-defs.h b/arch/mips/include/asm/octeon/cvmx-ciu3-defs.h
new file mode 100644
index 0000000..547f778
--- /dev/null
+++ b/arch/mips/include/asm/octeon/cvmx-ciu3-defs.h
@@ -0,0 +1,353 @@
+/*
+ * Copyright (c) 2003-2016 Cavium Inc.
+ *
+ * This file is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, Version 2, as
+ * published by the Free Software Foundation.
+ *
+ * This file is distributed in the hope that it will be useful, but
+ * AS-IS and WITHOUT ANY WARRANTY; without even the implied warranty
+ * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE, TITLE, or
+ * NONINFRINGEMENT. See the GNU General Public License for more
+ * details.
+ *
+ */
+
+#ifndef __CVMX_CIU3_DEFS_H__
+#define __CVMX_CIU3_DEFS_H__
+
+#define CVMX_CIU3_FUSE CVMX_ADD_IO_SEG(0x00010100000001A0ull)
+#define CVMX_CIU3_BIST CVMX_ADD_IO_SEG(0x00010100000001C0ull)
+#define CVMX_CIU3_CONST CVMX_ADD_IO_SEG(0x0001010000000220ull)
+#define CVMX_CIU3_CTL CVMX_ADD_IO_SEG(0x00010100000000E0ull)
+#define CVMX_CIU3_DESTX_IO_INT(offset) (CVMX_ADD_IO_SEG(0x0001010000210000ull) + ((offset) & 7) * 8)
+#define CVMX_CIU3_DESTX_PP_INT(offset) (CVMX_ADD_IO_SEG(0x0001010000200000ull) + ((offset) & 255) * 8)
+#define CVMX_CIU3_GSTOP CVMX_ADD_IO_SEG(0x0001010000000140ull)
+#define CVMX_CIU3_IDTX_CTL(offset) (CVMX_ADD_IO_SEG(0x0001010000110000ull) + ((offset) & 255) * 8)
+#define CVMX_CIU3_IDTX_IO(offset) (CVMX_ADD_IO_SEG(0x0001010000130000ull) + ((offset) & 255) * 8)
+#define CVMX_CIU3_IDTX_PPX(offset, block_id) (CVMX_ADD_IO_SEG(0x0001010000120000ull) + ((block_id) & 255) * 0x20ull)
+#define CVMX_CIU3_INTR_RAM_ECC_CTL CVMX_ADD_IO_SEG(0x0001010000000260ull)
+#define CVMX_CIU3_INTR_RAM_ECC_ST CVMX_ADD_IO_SEG(0x0001010000000280ull)
+#define CVMX_CIU3_INTR_READY CVMX_ADD_IO_SEG(0x00010100000002A0ull)
+#define CVMX_CIU3_INTR_SLOWDOWN CVMX_ADD_IO_SEG(0x0001010000000240ull)
+#define CVMX_CIU3_ISCX_CTL(offset) (CVMX_ADD_IO_SEG(0x0001010080000000ull) + ((offset) & 1048575) * 8)
+#define CVMX_CIU3_ISCX_W1C(offset) (CVMX_ADD_IO_SEG(0x0001010090000000ull) + ((offset) & 1048575) * 8)
+#define CVMX_CIU3_ISCX_W1S(offset) (CVMX_ADD_IO_SEG(0x00010100A0000000ull) + ((offset) & 1048575) * 8)
+#define CVMX_CIU3_NMI CVMX_ADD_IO_SEG(0x0001010000000160ull)
+#define CVMX_CIU3_SISCX(offset) (CVMX_ADD_IO_SEG(0x0001010000220000ull) + ((offset) & 255) * 8)
+#define CVMX_CIU3_TIMX(offset) (CVMX_ADD_IO_SEG(0x0001010000010000ull) + ((offset) & 15) * 8)
+
+union cvmx_ciu3_bist {
+ uint64_t u64;
+ struct cvmx_ciu3_bist_s {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t reserved_9_63 : 55;
+ uint64_t bist : 9;
+#else
+ uint64_t bist : 9;
+ uint64_t reserved_9_63 : 55;
+#endif
+ } s;
+};
+
+union cvmx_ciu3_const {
+ uint64_t u64;
+ struct cvmx_ciu3_const_s {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t dests_io : 16;
+ uint64_t pintsn : 16;
+ uint64_t dests_pp : 16;
+ uint64_t idt : 16;
+#else
+ uint64_t idt : 16;
+ uint64_t dests_pp : 16;
+ uint64_t pintsn : 16;
+ uint64_t dests_io : 16;
+#endif
+ } s;
+};
+
+union cvmx_ciu3_ctl {
+ uint64_t u64;
+ struct cvmx_ciu3_ctl_s {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t reserved_5_63 : 59;
+ uint64_t mcd_sel : 2;
+ uint64_t iscmem_le : 1;
+ uint64_t seq_dis : 1;
+ uint64_t cclk_dis : 1;
+#else
+ uint64_t cclk_dis : 1;
+ uint64_t seq_dis : 1;
+ uint64_t iscmem_le : 1;
+ uint64_t mcd_sel : 2;
+ uint64_t reserved_5_63 : 59;
+#endif
+ } s;
+};
+
+union cvmx_ciu3_destx_io_int {
+ uint64_t u64;
+ struct cvmx_ciu3_destx_io_int_s {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t reserved_52_63 : 12;
+ uint64_t intsn : 20;
+ uint64_t reserved_10_31 : 22;
+ uint64_t intidt : 8;
+ uint64_t newint : 1;
+ uint64_t intr : 1;
+#else
+ uint64_t intr : 1;
+ uint64_t newint : 1;
+ uint64_t intidt : 8;
+ uint64_t reserved_10_31 : 22;
+ uint64_t intsn : 20;
+ uint64_t reserved_52_63 : 12;
+#endif
+ } s;
+};
+
+union cvmx_ciu3_destx_pp_int {
+ uint64_t u64;
+ struct cvmx_ciu3_destx_pp_int_s {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t reserved_52_63 : 12;
+ uint64_t intsn : 20;
+ uint64_t reserved_10_31 : 22;
+ uint64_t intidt : 8;
+ uint64_t newint : 1;
+ uint64_t intr : 1;
+#else
+ uint64_t intr : 1;
+ uint64_t newint : 1;
+ uint64_t intidt : 8;
+ uint64_t reserved_10_31 : 22;
+ uint64_t intsn : 20;
+ uint64_t reserved_52_63 : 12;
+#endif
+ } s;
+};
+
+union cvmx_ciu3_gstop {
+ uint64_t u64;
+ struct cvmx_ciu3_gstop_s {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t reserved_1_63 : 63;
+ uint64_t gstop : 1;
+#else
+ uint64_t gstop : 1;
+ uint64_t reserved_1_63 : 63;
+#endif
+ } s;
+};
+
+union cvmx_ciu3_idtx_ctl {
+ uint64_t u64;
+ struct cvmx_ciu3_idtx_ctl_s {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t reserved_52_63 : 12;
+ uint64_t intsn : 20;
+ uint64_t reserved_4_31 : 28;
+ uint64_t intr : 1;
+ uint64_t newint : 1;
+ uint64_t ip_num : 2;
+#else
+ uint64_t ip_num : 2;
+ uint64_t newint : 1;
+ uint64_t intr : 1;
+ uint64_t reserved_4_31 : 28;
+ uint64_t intsn : 20;
+ uint64_t reserved_52_63 : 12;
+#endif
+ } s;
+};
+
+union cvmx_ciu3_idtx_io {
+ uint64_t u64;
+ struct cvmx_ciu3_idtx_io_s {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t reserved_5_63 : 59;
+ uint64_t io : 5;
+#else
+ uint64_t io : 5;
+ uint64_t reserved_5_63 : 59;
+#endif
+ } s;
+};
+
+union cvmx_ciu3_idtx_ppx {
+ uint64_t u64;
+ struct cvmx_ciu3_idtx_ppx_s {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t reserved_48_63 : 16;
+ uint64_t pp : 48;
+#else
+ uint64_t pp : 48;
+ uint64_t reserved_48_63 : 16;
+#endif
+ } s;
+};
+
+union cvmx_ciu3_intr_ram_ecc_ctl {
+ uint64_t u64;
+ struct cvmx_ciu3_intr_ram_ecc_ctl_s {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t reserved_3_63 : 61;
+ uint64_t flip_synd : 2;
+ uint64_t ecc_ena : 1;
+#else
+ uint64_t ecc_ena : 1;
+ uint64_t flip_synd : 2;
+ uint64_t reserved_3_63 : 61;
+#endif
+ } s;
+};
+
+union cvmx_ciu3_intr_ram_ecc_st {
+ uint64_t u64;
+ struct cvmx_ciu3_intr_ram_ecc_st_s {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t reserved_52_63 : 12;
+ uint64_t addr : 20;
+ uint64_t reserved_6_31 : 26;
+ uint64_t sisc_dbe : 1;
+ uint64_t sisc_sbe : 1;
+ uint64_t idt_dbe : 1;
+ uint64_t idt_sbe : 1;
+ uint64_t isc_dbe : 1;
+ uint64_t isc_sbe : 1;
+#else
+ uint64_t isc_sbe : 1;
+ uint64_t isc_dbe : 1;
+ uint64_t idt_sbe : 1;
+ uint64_t idt_dbe : 1;
+ uint64_t sisc_sbe : 1;
+ uint64_t sisc_dbe : 1;
+ uint64_t reserved_6_31 : 26;
+ uint64_t addr : 20;
+ uint64_t reserved_52_63 : 12;
+#endif
+ } s;
+};
+
+union cvmx_ciu3_intr_ready {
+ uint64_t u64;
+ struct cvmx_ciu3_intr_ready_s {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t reserved_46_63 : 18;
+ uint64_t index : 14;
+ uint64_t reserved_1_31 : 31;
+ uint64_t ready : 1;
+#else
+ uint64_t ready : 1;
+ uint64_t reserved_1_31 : 31;
+ uint64_t index : 14;
+ uint64_t reserved_46_63 : 18;
+#endif
+ } s;
+};
+
+union cvmx_ciu3_intr_slowdown {
+ uint64_t u64;
+ struct cvmx_ciu3_intr_slowdown_s {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t reserved_3_63 : 61;
+ uint64_t ctl : 3;
+#else
+ uint64_t ctl : 3;
+ uint64_t reserved_3_63 : 61;
+#endif
+ } s;
+};
+
+union cvmx_ciu3_iscx_ctl {
+ uint64_t u64;
+ struct cvmx_ciu3_iscx_ctl_s {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t reserved_24_63 : 40;
+ uint64_t idt : 8;
+ uint64_t imp : 1;
+ uint64_t reserved_2_14 : 13;
+ uint64_t en : 1;
+ uint64_t raw : 1;
+#else
+ uint64_t raw : 1;
+ uint64_t en : 1;
+ uint64_t reserved_2_14 : 13;
+ uint64_t imp : 1;
+ uint64_t idt : 8;
+ uint64_t reserved_24_63 : 40;
+#endif
+ } s;
+};
+
+union cvmx_ciu3_iscx_w1c {
+ uint64_t u64;
+ struct cvmx_ciu3_iscx_w1c_s {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t reserved_2_63 : 62;
+ uint64_t en : 1;
+ uint64_t raw : 1;
+#else
+ uint64_t raw : 1;
+ uint64_t en : 1;
+ uint64_t reserved_2_63 : 62;
+#endif
+ } s;
+};
+
+union cvmx_ciu3_iscx_w1s {
+ uint64_t u64;
+ struct cvmx_ciu3_iscx_w1s_s {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t reserved_2_63 : 62;
+ uint64_t en : 1;
+ uint64_t raw : 1;
+#else
+ uint64_t raw : 1;
+ uint64_t en : 1;
+ uint64_t reserved_2_63 : 62;
+#endif
+ } s;
+};
+
+union cvmx_ciu3_nmi {
+ uint64_t u64;
+ struct cvmx_ciu3_nmi_s {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t reserved_48_63 : 16;
+ uint64_t nmi : 48;
+#else
+ uint64_t nmi : 48;
+ uint64_t reserved_48_63 : 16;
+#endif
+ } s;
+};
+
+union cvmx_ciu3_siscx {
+ uint64_t u64;
+ struct cvmx_ciu3_siscx_s {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t en : 64;
+#else
+ uint64_t en : 64;
+#endif
+ } s;
+};
+
+union cvmx_ciu3_timx {
+ uint64_t u64;
+ struct cvmx_ciu3_timx_s {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t reserved_37_63 : 27;
+ uint64_t one_shot : 1;
+ uint64_t len : 36;
+#else
+ uint64_t len : 36;
+ uint64_t one_shot : 1;
+ uint64_t reserved_37_63 : 27;
+#endif
+ } s;
+};
+
+#endif
diff --git a/arch/mips/include/asm/octeon/cvmx-coremask.h b/arch/mips/include/asm/octeon/cvmx-coremask.h
new file mode 100644
index 0000000..097dc09
--- /dev/null
+++ b/arch/mips/include/asm/octeon/cvmx-coremask.h
@@ -0,0 +1,89 @@
+/*
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License. See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * Copyright (c) 2016 Cavium Inc. (support@cavium.com).
+ *
+ */
+
+/*
+ * Module to support operations on bitmap of cores. Coremask can be used to
+ * select a specific core, a group of cores, or all available cores, for
+ * initialization and differentiation of roles within a single shared binary
+ * executable image.
+ *
+ * The core numbers used in this file are the same value as what is found in
+ * the COP0_EBASE register and the rdhwr 0 instruction.
+ *
+ * For the CN78XX and other multi-node environments the core numbers are not
+ * contiguous. The core numbers for the CN78XX are as follows:
+ *
+ * Node 0: Cores 0 - 47
+ * Node 1: Cores 128 - 175
+ * Node 2: Cores 256 - 303
+ * Node 3: Cores 384 - 431
+ *
+ */
+
+#ifndef __CVMX_COREMASK_H__
+#define __CVMX_COREMASK_H__
+
+#define CVMX_MIPS_MAX_CORES 1024
+/* bits per holder */
+#define CVMX_COREMASK_ELTSZ 64
+
+/* cvmx_coremask_t's size in u64 */
+#define CVMX_COREMASK_BMPSZ (CVMX_MIPS_MAX_CORES / CVMX_COREMASK_ELTSZ)
+
+
+/* cvmx_coremask_t */
+struct cvmx_coremask {
+ u64 coremask_bitmap[CVMX_COREMASK_BMPSZ];
+};
+
+/*
+ * Is ``core'' set in the coremask?
+ */
+static inline bool cvmx_coremask_is_core_set(const struct cvmx_coremask *pcm,
+ int core)
+{
+ int n, i;
+
+ n = core % CVMX_COREMASK_ELTSZ;
+ i = core / CVMX_COREMASK_ELTSZ;
+
+ return (pcm->coremask_bitmap[i] & ((u64)1 << n)) != 0;
+}
+
+/*
+ * Make a copy of a coremask
+ */
+static inline void cvmx_coremask_copy(struct cvmx_coremask *dest,
+ const struct cvmx_coremask *src)
+{
+ memcpy(dest, src, sizeof(*dest));
+}
+
+/*
+ * Set the lower 64-bit of the coremask.
+ */
+static inline void cvmx_coremask_set64(struct cvmx_coremask *pcm,
+ uint64_t coremask_64)
+{
+ pcm->coremask_bitmap[0] = coremask_64;
+}
+
+/*
+ * Clear ``core'' from the coremask.
+ */
+static inline void cvmx_coremask_clear_core(struct cvmx_coremask *pcm, int core)
+{
+ int n, i;
+
+ n = core % CVMX_COREMASK_ELTSZ;
+ i = core / CVMX_COREMASK_ELTSZ;
+ pcm->coremask_bitmap[i] &= ~(1ull << n);
+}
+
+#endif /* __CVMX_COREMASK_H__ */
diff --git a/arch/mips/include/asm/octeon/cvmx-fpa-defs.h b/arch/mips/include/asm/octeon/cvmx-fpa-defs.h
index 1d79e3c..887ff8e 100644
--- a/arch/mips/include/asm/octeon/cvmx-fpa-defs.h
+++ b/arch/mips/include/asm/octeon/cvmx-fpa-defs.h
@@ -66,6 +66,7 @@
#define CVMX_FPA_WART_CTL (CVMX_ADD_IO_SEG(0x00011800280000D8ull))
#define CVMX_FPA_WART_STATUS (CVMX_ADD_IO_SEG(0x00011800280000E0ull))
#define CVMX_FPA_WQE_THRESHOLD (CVMX_ADD_IO_SEG(0x0001180028000468ull))
+#define CVMX_FPA_CLK_COUNT (CVMX_ADD_IO_SEG(0x00012800000000F0ull))
union cvmx_fpa_addr_range_error {
uint64_t u64;
diff --git a/arch/mips/include/asm/octeon/cvmx-mio-defs.h b/arch/mips/include/asm/octeon/cvmx-mio-defs.h
index bb0ae33..5196c04 100644
--- a/arch/mips/include/asm/octeon/cvmx-mio-defs.h
+++ b/arch/mips/include/asm/octeon/cvmx-mio-defs.h
@@ -1481,7 +1481,9 @@
uint64_t u64;
struct cvmx_mio_fus_dat2_s {
#ifdef __BIG_ENDIAN_BITFIELD
- uint64_t reserved_48_63:16;
+ uint64_t reserved_59_63:5;
+ uint64_t run_platform:3;
+ uint64_t gbl_pwr_throttle:8;
uint64_t fus118:1;
uint64_t rom_info:10;
uint64_t power_limit:2;
@@ -1513,7 +1515,9 @@
uint64_t power_limit:2;
uint64_t rom_info:10;
uint64_t fus118:1;
- uint64_t reserved_48_63:16;
+ uint64_t gbl_pwr_throttle:8;
+ uint64_t run_platform:3;
+ uint64_t reserved_59_63:5;
#endif
} s;
struct cvmx_mio_fus_dat2_cn30xx {
@@ -1837,50 +1841,192 @@
#endif
} cn68xx;
struct cvmx_mio_fus_dat2_cn68xx cn68xxp1;
+ struct cvmx_mio_fus_dat2_cn70xx {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t reserved_48_63:16;
+ uint64_t fus118:1;
+ uint64_t rom_info:10;
+ uint64_t power_limit:2;
+ uint64_t dorm_crypto:1;
+ uint64_t fus318:1;
+ uint64_t raid_en:1;
+ uint64_t reserved_31_29:3;
+ uint64_t nodfa_cp2:1;
+ uint64_t nomul:1;
+ uint64_t nocrypto:1;
+ uint64_t reserved_25_24:2;
+ uint64_t chip_id:8;
+ uint64_t reserved_15_0:16;
+#else
+ uint64_t reserved_15_0:16;
+ uint64_t chip_id:8;
+ uint64_t reserved_25_24:2;
+ uint64_t nocrypto:1;
+ uint64_t nomul:1;
+ uint64_t nodfa_cp2:1;
+ uint64_t reserved_31_29:3;
+ uint64_t raid_en:1;
+ uint64_t fus318:1;
+ uint64_t dorm_crypto:1;
+ uint64_t power_limit:2;
+ uint64_t rom_info:10;
+ uint64_t fus118:1;
+ uint64_t reserved_48_63:16;
+#endif
+ } cn70xx;
+ struct cvmx_mio_fus_dat2_cn70xx cn70xxp1;
+ struct cvmx_mio_fus_dat2_cn73xx {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t reserved_59_63:5;
+ uint64_t run_platform:3;
+ uint64_t gbl_pwr_throttle:8;
+ uint64_t fus118:1;
+ uint64_t rom_info:10;
+ uint64_t power_limit:2;
+ uint64_t dorm_crypto:1;
+ uint64_t fus318:1;
+ uint64_t raid_en:1;
+ uint64_t reserved_31_29:3;
+ uint64_t nodfa_cp2:1;
+ uint64_t nomul:1;
+ uint64_t nocrypto:1;
+ uint64_t reserved_25_24:2;
+ uint64_t chip_id:8;
+ uint64_t reserved_15_0:16;
+#else
+ uint64_t reserved_15_0:16;
+ uint64_t chip_id:8;
+ uint64_t reserved_25_24:2;
+ uint64_t nocrypto:1;
+ uint64_t nomul:1;
+ uint64_t nodfa_cp2:1;
+ uint64_t reserved_31_29:3;
+ uint64_t raid_en:1;
+ uint64_t fus318:1;
+ uint64_t dorm_crypto:1;
+ uint64_t power_limit:2;
+ uint64_t rom_info:10;
+ uint64_t fus118:1;
+ uint64_t gbl_pwr_throttle:8;
+ uint64_t run_platform:3;
+ uint64_t reserved_59_63:5;
+#endif
+ } cn73xx;
+ struct cvmx_mio_fus_dat2_cn78xx {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t reserved_59_63:5;
+ uint64_t run_platform:3;
+ uint64_t reserved_48_55:8;
+ uint64_t fus118:1;
+ uint64_t rom_info:10;
+ uint64_t power_limit:2;
+ uint64_t dorm_crypto:1;
+ uint64_t fus318:1;
+ uint64_t raid_en:1;
+ uint64_t reserved_31_29:3;
+ uint64_t nodfa_cp2:1;
+ uint64_t nomul:1;
+ uint64_t nocrypto:1;
+ uint64_t reserved_25_24:2;
+ uint64_t chip_id:8;
+ uint64_t reserved_0_15:16;
+#else
+ uint64_t reserved_0_15:16;
+ uint64_t chip_id:8;
+ uint64_t reserved_25_24:2;
+ uint64_t nocrypto:1;
+ uint64_t nomul:1;
+ uint64_t nodfa_cp2:1;
+ uint64_t reserved_31_29:3;
+ uint64_t raid_en:1;
+ uint64_t fus318:1;
+ uint64_t dorm_crypto:1;
+ uint64_t power_limit:2;
+ uint64_t rom_info:10;
+ uint64_t fus118:1;
+ uint64_t reserved_48_55:8;
+ uint64_t run_platform:3;
+ uint64_t reserved_59_63:5;
+#endif
+ } cn78xx;
+ struct cvmx_mio_fus_dat2_cn78xxp2 {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t reserved_59_63:5;
+ uint64_t run_platform:3;
+ uint64_t gbl_pwr_throttle:8;
+ uint64_t fus118:1;
+ uint64_t rom_info:10;
+ uint64_t power_limit:2;
+ uint64_t dorm_crypto:1;
+ uint64_t fus318:1;
+ uint64_t raid_en:1;
+ uint64_t reserved_31_29:3;
+ uint64_t nodfa_cp2:1;
+ uint64_t nomul:1;
+ uint64_t nocrypto:1;
+ uint64_t reserved_25_24:2;
+ uint64_t chip_id:8;
+ uint64_t reserved_0_15:16;
+#else
+ uint64_t reserved_0_15:16;
+ uint64_t chip_id:8;
+ uint64_t reserved_25_24:2;
+ uint64_t nocrypto:1;
+ uint64_t nomul:1;
+ uint64_t nodfa_cp2:1;
+ uint64_t reserved_31_29:3;
+ uint64_t raid_en:1;
+ uint64_t fus318:1;
+ uint64_t dorm_crypto:1;
+ uint64_t power_limit:2;
+ uint64_t rom_info:10;
+ uint64_t fus118:1;
+ uint64_t gbl_pwr_throttle:8;
+ uint64_t run_platform:3;
+ uint64_t reserved_59_63:5;
+#endif
+ } cn78xxp2;
struct cvmx_mio_fus_dat2_cn61xx cnf71xx;
+ struct cvmx_mio_fus_dat2_cn73xx cnf75xx;
};
union cvmx_mio_fus_dat3 {
uint64_t u64;
struct cvmx_mio_fus_dat3_s {
#ifdef __BIG_ENDIAN_BITFIELD
- uint64_t reserved_58_63:6;
+ uint64_t ema0:6;
uint64_t pll_ctl:10;
uint64_t dfa_info_dte:3;
uint64_t dfa_info_clm:4;
- uint64_t reserved_40_40:1;
- uint64_t ema:2;
+ uint64_t pll_alt_matrix:1;
+ uint64_t reserved_38_39:2;
uint64_t efus_lck_rsv:1;
uint64_t efus_lck_man:1;
uint64_t pll_half_dis:1;
uint64_t l2c_crip:3;
- uint64_t pll_div4:1;
- uint64_t reserved_29_30:2;
- uint64_t bar2_en:1;
+ uint64_t reserved_28_31:4;
uint64_t efus_lck:1;
uint64_t efus_ign:1;
uint64_t nozip:1;
uint64_t nodfa_dte:1;
- uint64_t icache:24;
+ uint64_t reserved_0_23:24;
#else
- uint64_t icache:24;
+ uint64_t reserved_0_23:24;
uint64_t nodfa_dte:1;
uint64_t nozip:1;
uint64_t efus_ign:1;
uint64_t efus_lck:1;
- uint64_t bar2_en:1;
- uint64_t reserved_29_30:2;
- uint64_t pll_div4:1;
+ uint64_t reserved_28_31:4;
uint64_t l2c_crip:3;
uint64_t pll_half_dis:1;
uint64_t efus_lck_man:1;
uint64_t efus_lck_rsv:1;
- uint64_t ema:2;
- uint64_t reserved_40_40:1;
+ uint64_t reserved_38_39:2;
+ uint64_t pll_alt_matrix:1;
uint64_t dfa_info_clm:4;
uint64_t dfa_info_dte:3;
uint64_t pll_ctl:10;
- uint64_t reserved_58_63:6;
+ uint64_t ema0:6;
#endif
} s;
struct cvmx_mio_fus_dat3_cn30xx {
@@ -2022,7 +2168,239 @@
struct cvmx_mio_fus_dat3_cn61xx cn66xx;
struct cvmx_mio_fus_dat3_cn61xx cn68xx;
struct cvmx_mio_fus_dat3_cn61xx cn68xxp1;
+ struct cvmx_mio_fus_dat3_cn70xx {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t ema0:6;
+ uint64_t pll_ctl:10;
+ uint64_t dfa_info_dte:3;
+ uint64_t dfa_info_clm:4;
+ uint64_t pll_alt_matrix:1;
+ uint64_t pll_bwadj_denom:2;
+ uint64_t efus_lck_rsv:1;
+ uint64_t efus_lck_man:1;
+ uint64_t pll_half_dis:1;
+ uint64_t l2c_crip:3;
+ uint64_t use_int_refclk:1;
+ uint64_t zip_info:2;
+ uint64_t bar2_sz_conf:1;
+ uint64_t efus_lck:1;
+ uint64_t efus_ign:1;
+ uint64_t nozip:1;
+ uint64_t nodfa_dte:1;
+ uint64_t ema1:6;
+ uint64_t reserved_0_17:18;
+#else
+ uint64_t reserved_0_17:18;
+ uint64_t ema1:6;
+ uint64_t nodfa_dte:1;
+ uint64_t nozip:1;
+ uint64_t efus_ign:1;
+ uint64_t efus_lck:1;
+ uint64_t bar2_sz_conf:1;
+ uint64_t zip_info:2;
+ uint64_t use_int_refclk:1;
+ uint64_t l2c_crip:3;
+ uint64_t pll_half_dis:1;
+ uint64_t efus_lck_man:1;
+ uint64_t efus_lck_rsv:1;
+ uint64_t pll_bwadj_denom:2;
+ uint64_t pll_alt_matrix:1;
+ uint64_t dfa_info_clm:4;
+ uint64_t dfa_info_dte:3;
+ uint64_t pll_ctl:10;
+ uint64_t ema0:6;
+#endif
+ } cn70xx;
+ struct cvmx_mio_fus_dat3_cn70xxp1 {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t ema0:6;
+ uint64_t pll_ctl:10;
+ uint64_t dfa_info_dte:3;
+ uint64_t dfa_info_clm:4;
+ uint64_t reserved_38_40:3;
+ uint64_t efus_lck_rsv:1;
+ uint64_t efus_lck_man:1;
+ uint64_t pll_half_dis:1;
+ uint64_t l2c_crip:3;
+ uint64_t reserved_31_31:1;
+ uint64_t zip_info:2;
+ uint64_t bar2_sz_conf:1;
+ uint64_t efus_lck:1;
+ uint64_t efus_ign:1;
+ uint64_t nozip:1;
+ uint64_t nodfa_dte:1;
+ uint64_t ema1:6;
+ uint64_t reserved_0_17:18;
+#else
+ uint64_t reserved_0_17:18;
+ uint64_t ema1:6;
+ uint64_t nodfa_dte:1;
+ uint64_t nozip:1;
+ uint64_t efus_ign:1;
+ uint64_t efus_lck:1;
+ uint64_t bar2_sz_conf:1;
+ uint64_t zip_info:2;
+ uint64_t reserved_31_31:1;
+ uint64_t l2c_crip:3;
+ uint64_t pll_half_dis:1;
+ uint64_t efus_lck_man:1;
+ uint64_t efus_lck_rsv:1;
+ uint64_t reserved_38_40:3;
+ uint64_t dfa_info_clm:4;
+ uint64_t dfa_info_dte:3;
+ uint64_t pll_ctl:10;
+ uint64_t ema0:6;
+#endif
+ } cn70xxp1;
+ struct cvmx_mio_fus_dat3_cn73xx {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t ema0:6;
+ uint64_t pll_ctl:10;
+ uint64_t dfa_info_dte:3;
+ uint64_t dfa_info_clm:4;
+ uint64_t pll_alt_matrix:1;
+ uint64_t pll_bwadj_denom:2;
+ uint64_t efus_lck_rsv:1;
+ uint64_t efus_lck_man:1;
+ uint64_t pll_half_dis:1;
+ uint64_t l2c_crip:3;
+ uint64_t use_int_refclk:1;
+ uint64_t zip_info:2;
+ uint64_t bar2_sz_conf:1;
+ uint64_t efus_lck:1;
+ uint64_t efus_ign:1;
+ uint64_t nozip:1;
+ uint64_t nodfa_dte:1;
+ uint64_t ema1:6;
+ uint64_t nohna_dte:1;
+ uint64_t hna_info_dte:3;
+ uint64_t hna_info_clm:4;
+ uint64_t reserved_9_9:1;
+ uint64_t core_pll_mul:5;
+ uint64_t pnr_pll_mul:4;
+#else
+ uint64_t pnr_pll_mul:4;
+ uint64_t core_pll_mul:5;
+ uint64_t reserved_9_9:1;
+ uint64_t hna_info_clm:4;
+ uint64_t hna_info_dte:3;
+ uint64_t nohna_dte:1;
+ uint64_t ema1:6;
+ uint64_t nodfa_dte:1;
+ uint64_t nozip:1;
+ uint64_t efus_ign:1;
+ uint64_t efus_lck:1;
+ uint64_t bar2_sz_conf:1;
+ uint64_t zip_info:2;
+ uint64_t use_int_refclk:1;
+ uint64_t l2c_crip:3;
+ uint64_t pll_half_dis:1;
+ uint64_t efus_lck_man:1;
+ uint64_t efus_lck_rsv:1;
+ uint64_t pll_bwadj_denom:2;
+ uint64_t pll_alt_matrix:1;
+ uint64_t dfa_info_clm:4;
+ uint64_t dfa_info_dte:3;
+ uint64_t pll_ctl:10;
+ uint64_t ema0:6;
+#endif
+ } cn73xx;
+ struct cvmx_mio_fus_dat3_cn78xx {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t ema0:6;
+ uint64_t pll_ctl:10;
+ uint64_t dfa_info_dte:3;
+ uint64_t dfa_info_clm:4;
+ uint64_t reserved_38_40:3;
+ uint64_t efus_lck_rsv:1;
+ uint64_t efus_lck_man:1;
+ uint64_t pll_half_dis:1;
+ uint64_t l2c_crip:3;
+ uint64_t reserved_31_31:1;
+ uint64_t zip_info:2;
+ uint64_t bar2_sz_conf:1;
+ uint64_t efus_lck:1;
+ uint64_t efus_ign:1;
+ uint64_t nozip:1;
+ uint64_t nodfa_dte:1;
+ uint64_t ema1:6;
+ uint64_t nohna_dte:1;
+ uint64_t hna_info_dte:3;
+ uint64_t hna_info_clm:4;
+ uint64_t reserved_0_9:10;
+#else
+ uint64_t reserved_0_9:10;
+ uint64_t hna_info_clm:4;
+ uint64_t hna_info_dte:3;
+ uint64_t nohna_dte:1;
+ uint64_t ema1:6;
+ uint64_t nodfa_dte:1;
+ uint64_t nozip:1;
+ uint64_t efus_ign:1;
+ uint64_t efus_lck:1;
+ uint64_t bar2_sz_conf:1;
+ uint64_t zip_info:2;
+ uint64_t reserved_31_31:1;
+ uint64_t l2c_crip:3;
+ uint64_t pll_half_dis:1;
+ uint64_t efus_lck_man:1;
+ uint64_t efus_lck_rsv:1;
+ uint64_t reserved_38_40:3;
+ uint64_t dfa_info_clm:4;
+ uint64_t dfa_info_dte:3;
+ uint64_t pll_ctl:10;
+ uint64_t ema0:6;
+#endif
+ } cn78xx;
+ struct cvmx_mio_fus_dat3_cn73xx cn78xxp2;
struct cvmx_mio_fus_dat3_cn61xx cnf71xx;
+ struct cvmx_mio_fus_dat3_cnf75xx {
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint64_t ema0:6;
+ uint64_t pll_ctl:10;
+ uint64_t dfa_info_dte:3;
+ uint64_t dfa_info_clm:4;
+ uint64_t pll_alt_matrix:1;
+ uint64_t pll_bwadj_denom:2;
+ uint64_t efus_lck_rsv:1;
+ uint64_t efus_lck_man:1;
+ uint64_t pll_half_dis:1;
+ uint64_t l2c_crip:3;
+ uint64_t use_int_refclk:1;
+ uint64_t zip_info:2;
+ uint64_t bar2_sz_conf:1;
+ uint64_t efus_lck:1;
+ uint64_t efus_ign:1;
+ uint64_t nozip:1;
+ uint64_t nodfa_dte:1;
+ uint64_t ema1:6;
+ uint64_t reserved_9_17:9;
+ uint64_t core_pll_mul:5;
+ uint64_t pnr_pll_mul:4;
+#else
+ uint64_t pnr_pll_mul:4;
+ uint64_t core_pll_mul:5;
+ uint64_t reserved_9_17:9;
+ uint64_t ema1:6;
+ uint64_t nodfa_dte:1;
+ uint64_t nozip:1;
+ uint64_t efus_ign:1;
+ uint64_t efus_lck:1;
+ uint64_t bar2_sz_conf:1;
+ uint64_t zip_info:2;
+ uint64_t use_int_refclk:1;
+ uint64_t l2c_crip:3;
+ uint64_t pll_half_dis:1;
+ uint64_t efus_lck_man:1;
+ uint64_t efus_lck_rsv:1;
+ uint64_t pll_bwadj_denom:2;
+ uint64_t pll_alt_matrix:1;
+ uint64_t dfa_info_clm:4;
+ uint64_t dfa_info_dte:3;
+ uint64_t pll_ctl:10;
+ uint64_t ema0:6;
+#endif
+ } cnf75xx;
};
union cvmx_mio_fus_ema {
diff --git a/arch/mips/include/asm/octeon/cvmx-sysinfo.h b/arch/mips/include/asm/octeon/cvmx-sysinfo.h
index 2131197..c6c3ee3 100644
--- a/arch/mips/include/asm/octeon/cvmx-sysinfo.h
+++ b/arch/mips/include/asm/octeon/cvmx-sysinfo.h
@@ -4,7 +4,7 @@
* Contact: support@caviumnetworks.com
* This file is part of the OCTEON SDK
*
- * Copyright (c) 2003-2008 Cavium Networks
+ * Copyright (c) 2003-2016 Cavium, Inc.
*
* This file is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License, Version 2, as
@@ -32,6 +32,8 @@
#ifndef __CVMX_SYSINFO_H__
#define __CVMX_SYSINFO_H__
+#include "cvmx-coremask.h"
+
#define OCTEON_SERIAL_LEN 20
/**
* Structure describing application specific information.
@@ -50,8 +52,7 @@
uint64_t system_dram_size;
/* ptr to memory descriptor block */
- void *phy_mem_desc_ptr;
-
+ uint64_t phy_mem_desc_addr;
/* Application image specific variables */
/* stack top address (virtual) */
@@ -63,7 +64,7 @@
/* heap size in bytes */
uint32_t heap_size;
/* coremask defining cores running application */
- uint32_t core_mask;
+ struct cvmx_coremask core_mask;
/* Deprecated, use cvmx_coremask_first_core() to select init core */
uint32_t init_core;
@@ -121,32 +122,4 @@
extern struct cvmx_sysinfo *cvmx_sysinfo_get(void);
-/**
- * This function is used in non-simple executive environments (such as
- * Linux kernel, u-boot, etc.) to configure the minimal fields that
- * are required to use simple executive files directly.
- *
- * Locking (if required) must be handled outside of this
- * function
- *
- * @phy_mem_desc_ptr: Pointer to global physical memory descriptor
- * (bootmem descriptor) @board_type: Octeon board
- * type enumeration
- *
- * @board_rev_major:
- * Board major revision
- * @board_rev_minor:
- * Board minor revision
- * @cpu_clock_hz:
- * CPU clock freqency in hertz
- *
- * Returns 0: Failure
- * 1: success
- */
-extern int cvmx_sysinfo_minimal_initialize(void *phy_mem_desc_ptr,
- uint16_t board_type,
- uint8_t board_rev_major,
- uint8_t board_rev_minor,
- uint32_t cpu_clock_hz);
-
#endif /* __CVMX_SYSINFO_H__ */
diff --git a/arch/mips/include/asm/octeon/cvmx.h b/arch/mips/include/asm/octeon/cvmx.h
index 3e982e0..2530e87 100644
--- a/arch/mips/include/asm/octeon/cvmx.h
+++ b/arch/mips/include/asm/octeon/cvmx.h
@@ -57,6 +57,7 @@
#include <asm/octeon/cvmx-sysinfo.h>
#include <asm/octeon/cvmx-ciu-defs.h>
+#include <asm/octeon/cvmx-ciu3-defs.h>
#include <asm/octeon/cvmx-gpio-defs.h>
#include <asm/octeon/cvmx-iob-defs.h>
#include <asm/octeon/cvmx-ipd-defs.h>
@@ -341,6 +342,21 @@
return core_num;
}
+/* Maximum # of bits to define core in node */
+#define CVMX_NODE_NO_SHIFT 7
+#define CVMX_NODE_MASK 0x3
+static inline unsigned int cvmx_get_node_num(void)
+{
+ unsigned int core_num = cvmx_get_core_num();
+
+ return (core_num >> CVMX_NODE_NO_SHIFT) & CVMX_NODE_MASK;
+}
+
+static inline unsigned int cvmx_get_local_core_num(void)
+{
+ return cvmx_get_core_num() & ((1 << CVMX_NODE_NO_SHIFT) - 1);
+}
+
/**
* Returns the number of bits set in the provided value.
* Simple wrapper for POP instruction.
@@ -448,8 +464,15 @@
/* Return the number of cores available in the chip */
static inline uint32_t cvmx_octeon_num_cores(void)
{
- uint32_t ciu_fuse = (uint32_t) cvmx_read_csr(CVMX_CIU_FUSE) & 0xffff;
- return cvmx_pop(ciu_fuse);
+ u64 ciu_fuse_reg;
+ u64 ciu_fuse;
+
+ if (OCTEON_IS_OCTEON3() && !OCTEON_IS_MODEL(OCTEON_CN70XX))
+ ciu_fuse_reg = CVMX_CIU3_FUSE;
+ else
+ ciu_fuse_reg = CVMX_CIU_FUSE;
+ ciu_fuse = cvmx_read_csr(ciu_fuse_reg);
+ return cvmx_dpop(ciu_fuse);
}
#endif /* __CVMX_H__ */
diff --git a/arch/mips/include/asm/octeon/octeon-feature.h b/arch/mips/include/asm/octeon/octeon-feature.h
index 3ed10a8..a19ca3b 100644
--- a/arch/mips/include/asm/octeon/octeon-feature.h
+++ b/arch/mips/include/asm/octeon/octeon-feature.h
@@ -81,6 +81,10 @@
OCTEON_FEATURE_HFA,
OCTEON_FEATURE_DFM,
OCTEON_FEATURE_CIU2,
+ OCTEON_FEATURE_CIU3,
+ /* Octeon has FPA first seen on 78XX */
+ OCTEON_FEATURE_FPA3,
+ OCTEON_FEATURE_FAU,
OCTEON_MAX_FEATURE
};
@@ -110,7 +114,7 @@
* Returns Non zero if the feature exists. Zero if the feature does not
* exist.
*/
-static inline int octeon_has_feature(enum octeon_feature feature)
+static inline bool octeon_has_feature(enum octeon_feature feature)
{
switch (feature) {
case OCTEON_FEATURE_SAAD:
@@ -122,7 +126,7 @@
fus_2.u64 = cvmx_read_csr(CVMX_MIO_FUS_DAT2);
return !fus_2.s.nocrypto && !fus_2.s.nomul && fus_2.s.dorm_crypto;
} else {
- return 0;
+ return false;
}
case OCTEON_FEATURE_PCIE:
@@ -190,11 +194,20 @@
case OCTEON_FEATURE_CIU2:
return OCTEON_IS_MODEL(OCTEON_CN68XX);
+ case OCTEON_FEATURE_CIU3:
+ case OCTEON_FEATURE_FPA3:
+ return OCTEON_IS_MODEL(OCTEON_CN78XX)
+ || OCTEON_IS_MODEL(OCTEON_CNF75XX)
+ || OCTEON_IS_MODEL(OCTEON_CN73XX);
+ case OCTEON_FEATURE_FAU:
+ return !(OCTEON_IS_MODEL(OCTEON_CN78XX)
+ || OCTEON_IS_MODEL(OCTEON_CNF75XX)
+ || OCTEON_IS_MODEL(OCTEON_CN73XX));
default:
break;
}
- return 0;
+ return false;
}
#endif /* __OCTEON_FEATURE_H__ */
diff --git a/arch/mips/include/asm/octeon/octeon-model.h b/arch/mips/include/asm/octeon/octeon-model.h
index 92b377e..6c68517 100644
--- a/arch/mips/include/asm/octeon/octeon-model.h
+++ b/arch/mips/include/asm/octeon/octeon-model.h
@@ -74,7 +74,12 @@
* CN7XXX models with new revision encoding
*/
+#define OCTEON_CNF75XX_PASS1_0 0x000d9800
+#define OCTEON_CNF75XX (OCTEON_CNF75XX_PASS1_0 | OM_IGNORE_REVISION)
+#define OCTEON_CNF75XX_PASS1_X (OCTEON_CNF75XX_PASS1_0 | OM_IGNORE_MINOR_REVISION)
+
#define OCTEON_CN73XX_PASS1_0 0x000d9700
+#define OCTEON_CN73XX_PASS1_1 0x000d9701
#define OCTEON_CN73XX (OCTEON_CN73XX_PASS1_0 | OM_IGNORE_REVISION)
#define OCTEON_CN73XX_PASS1_X (OCTEON_CN73XX_PASS1_0 | \
OM_IGNORE_MINOR_REVISION)
diff --git a/arch/mips/include/asm/octeon/octeon.h b/arch/mips/include/asm/octeon/octeon.h
index de9f74e..07c0516 100644
--- a/arch/mips/include/asm/octeon/octeon.h
+++ b/arch/mips/include/asm/octeon/octeon.h
@@ -299,6 +299,31 @@
cvmx_read64_uint32(address ^ 4);
}
+#ifdef CONFIG_SMP
+void octeon_setup_smp(void);
+#else
+static inline void octeon_setup_smp(void) {}
+#endif
+
+struct irq_domain;
+struct device_node;
+struct irq_data;
+struct irq_chip;
+void octeon_ciu3_mbox_send(int cpu, unsigned int mbox);
+int octeon_irq_ciu3_xlat(struct irq_domain *d,
+ struct device_node *node,
+ const u32 *intspec,
+ unsigned int intsize,
+ unsigned long *out_hwirq,
+ unsigned int *out_type);
+void octeon_irq_ciu3_enable(struct irq_data *data);
+void octeon_irq_ciu3_disable(struct irq_data *data);
+void octeon_irq_ciu3_ack(struct irq_data *data);
+void octeon_irq_ciu3_mask(struct irq_data *data);
+void octeon_irq_ciu3_mask_ack(struct irq_data *data);
+int octeon_irq_ciu3_mapx(struct irq_domain *d, unsigned int virq,
+ irq_hw_number_t hw, struct irq_chip *chip);
+
/* Octeon multiplier save/restore routines from octeon_switch.S */
void octeon_mult_save(void);
void octeon_mult_restore(void);
diff --git a/arch/mips/include/asm/pci.h b/arch/mips/include/asm/pci.h
index 8c16fb7..86b239d 100644
--- a/arch/mips/include/asm/pci.h
+++ b/arch/mips/include/asm/pci.h
@@ -43,8 +43,6 @@
and XFree86. Eventually will be removed. */
unsigned int need_domain_info;
- int iommu;
-
/* Optional access methods for reading/writing the bus number
of the PCI controller */
int (*get_busno)(void);
@@ -106,11 +104,11 @@
struct pci_dev;
/*
- * The PCI address space does equal the physical memory address space. The
- * networking and block device layers use this boolean for bounce buffer
- * decisions. This is set if any hose does not have an IOMMU.
+ * The PCI address space does equal the physical memory address space.
+ * The networking and block device layers use this boolean for bounce
+ * buffer decisions.
*/
-extern unsigned int PCI_DMA_BUS_IS_PHYS;
+#define PCI_DMA_BUS_IS_PHYS (1)
#ifdef CONFIG_PCI_DOMAINS
#define pci_domain_nr(bus) ((struct pci_controller *)(bus)->sysdata)->index
diff --git a/arch/mips/include/asm/pgtable-32.h b/arch/mips/include/asm/pgtable-32.h
index 832e216..d21f3da 100644
--- a/arch/mips/include/asm/pgtable-32.h
+++ b/arch/mips/include/asm/pgtable-32.h
@@ -103,8 +103,8 @@
pmd_val(*pmdp) = ((unsigned long) invalid_pte_table);
}
-#if defined(CONFIG_PHYS_ADDR_T_64BIT) && defined(CONFIG_CPU_MIPS32)
-#define pte_page(x) pfn_to_page(pte_pfn(x))
+#if defined(CONFIG_XPA)
+
#define pte_pfn(x) (((unsigned long)((x).pte_high >> _PFN_SHIFT)) | (unsigned long)((x).pte_low << _PAGE_PRESENT_SHIFT))
static inline pte_t
pfn_pte(unsigned long pfn, pgprot_t prot)
@@ -118,9 +118,21 @@
return pte;
}
-#else
+#elif defined(CONFIG_PHYS_ADDR_T_64BIT) && defined(CONFIG_CPU_MIPS32)
-#define pte_page(x) pfn_to_page(pte_pfn(x))
+#define pte_pfn(x) ((unsigned long)((x).pte_high >> 6))
+
+static inline pte_t pfn_pte(unsigned long pfn, pgprot_t prot)
+{
+ pte_t pte;
+
+ pte.pte_high = (pfn << 6) | (pgprot_val(prot) & 0x3f);
+ pte.pte_low = pgprot_val(prot);
+
+ return pte;
+}
+
+#else
#ifdef CONFIG_CPU_VR41XX
#define pte_pfn(x) ((unsigned long)((x).pte >> (PAGE_SHIFT + 2)))
@@ -131,6 +143,8 @@
#endif
#endif /* defined(CONFIG_PHYS_ADDR_T_64BIT) && defined(CONFIG_CPU_MIPS32) */
+#define pte_page(x) pfn_to_page(pte_pfn(x))
+
#define __pgd_offset(address) pgd_index(address)
#define __pud_offset(address) (((address) >> PUD_SHIFT) & (PTRS_PER_PUD-1))
#define __pmd_offset(address) (((address) >> PMD_SHIFT) & (PTRS_PER_PMD-1))
@@ -166,7 +180,7 @@
#else
-#if defined(CONFIG_PHYS_ADDR_T_64BIT) && defined(CONFIG_CPU_MIPS32)
+#if defined(CONFIG_XPA)
/* Swap entries must have VALID and GLOBAL bits cleared. */
#define __swp_type(x) (((x).val >> 4) & 0x1f)
@@ -175,6 +189,15 @@
#define __pte_to_swp_entry(pte) ((swp_entry_t) { (pte).pte_high })
#define __swp_entry_to_pte(x) ((pte_t) { 0, (x).val })
+#elif defined(CONFIG_PHYS_ADDR_T_64BIT) && defined(CONFIG_CPU_MIPS32)
+
+/* Swap entries must have VALID and GLOBAL bits cleared. */
+#define __swp_type(x) (((x).val >> 2) & 0x1f)
+#define __swp_offset(x) ((x).val >> 7)
+#define __swp_entry(type, offset) ((swp_entry_t) { ((type) << 2) | ((offset) << 7) })
+#define __pte_to_swp_entry(pte) ((swp_entry_t) { (pte).pte_high })
+#define __swp_entry_to_pte(x) ((pte_t) { 0, (x).val })
+
#else
/*
* Constraints:
diff --git a/arch/mips/include/asm/pgtable-64.h b/arch/mips/include/asm/pgtable-64.h
index cf661a2..514cbc0 100644
--- a/arch/mips/include/asm/pgtable-64.h
+++ b/arch/mips/include/asm/pgtable-64.h
@@ -17,7 +17,7 @@
#include <asm/cachectl.h>
#include <asm/fixmap.h>
-#ifdef CONFIG_PAGE_SIZE_64KB
+#if defined(CONFIG_PAGE_SIZE_64KB) && !defined(CONFIG_MIPS_VA_BITS_48)
#include <asm-generic/pgtable-nopmd.h>
#else
#include <asm-generic/pgtable-nopud.h>
@@ -90,7 +90,11 @@
#define PTE_ORDER 0
#endif
#ifdef CONFIG_PAGE_SIZE_16KB
-#define PGD_ORDER 0
+#ifdef CONFIG_MIPS_VA_BITS_48
+#define PGD_ORDER 1
+#else
+#define PGD_ORDER 0
+#endif
#define PUD_ORDER aieeee_attempt_to_allocate_pud
#define PMD_ORDER 0
#define PTE_ORDER 0
@@ -104,7 +108,11 @@
#ifdef CONFIG_PAGE_SIZE_64KB
#define PGD_ORDER 0
#define PUD_ORDER aieeee_attempt_to_allocate_pud
+#ifdef CONFIG_MIPS_VA_BITS_48
+#define PMD_ORDER 0
+#else
#define PMD_ORDER aieeee_attempt_to_allocate_pmd
+#endif
#define PTE_ORDER 0
#endif
@@ -114,11 +122,7 @@
#endif
#define PTRS_PER_PTE ((PAGE_SIZE << PTE_ORDER) / sizeof(pte_t))
-#if PGDIR_SIZE >= TASK_SIZE64
-#define USER_PTRS_PER_PGD (1)
-#else
-#define USER_PTRS_PER_PGD (TASK_SIZE64 / PGDIR_SIZE)
-#endif
+#define USER_PTRS_PER_PGD ((TASK_SIZE64 / PGDIR_SIZE)?(TASK_SIZE64 / PGDIR_SIZE):1)
#define FIRST_USER_ADDRESS 0UL
/*
diff --git a/arch/mips/include/asm/pgtable-bits.h b/arch/mips/include/asm/pgtable-bits.h
index 97b3138..f88a48c 100644
--- a/arch/mips/include/asm/pgtable-bits.h
+++ b/arch/mips/include/asm/pgtable-bits.h
@@ -32,149 +32,132 @@
* unpredictable things. The code (when it is written) to deal with
* this problem will be in the update_mmu_cache() code for the r4k.
*/
-#if defined(CONFIG_PHYS_ADDR_T_64BIT) && defined(CONFIG_CPU_MIPS32)
+#if defined(CONFIG_XPA)
/*
- * The following bits are implemented by the TLB hardware
+ * Page table bit offsets used for 64 bit physical addressing on
+ * MIPS32r5 with XPA.
*/
-#define _PAGE_NO_EXEC_SHIFT 0
-#define _PAGE_NO_EXEC (1 << _PAGE_NO_EXEC_SHIFT)
-#define _PAGE_NO_READ_SHIFT (_PAGE_NO_EXEC_SHIFT + 1)
-#define _PAGE_NO_READ (1 << _PAGE_NO_READ_SHIFT)
-#define _PAGE_GLOBAL_SHIFT (_PAGE_NO_READ_SHIFT + 1)
-#define _PAGE_GLOBAL (1 << _PAGE_GLOBAL_SHIFT)
-#define _PAGE_VALID_SHIFT (_PAGE_GLOBAL_SHIFT + 1)
-#define _PAGE_VALID (1 << _PAGE_VALID_SHIFT)
-#define _PAGE_DIRTY_SHIFT (_PAGE_VALID_SHIFT + 1)
-#define _PAGE_DIRTY (1 << _PAGE_DIRTY_SHIFT)
-#define _CACHE_SHIFT (_PAGE_DIRTY_SHIFT + 1)
-#define _CACHE_MASK (7 << _CACHE_SHIFT)
+enum pgtable_bits {
+ /* Used by TLB hardware (placed in EntryLo*) */
+ _PAGE_NO_EXEC_SHIFT,
+ _PAGE_NO_READ_SHIFT,
+ _PAGE_GLOBAL_SHIFT,
+ _PAGE_VALID_SHIFT,
+ _PAGE_DIRTY_SHIFT,
+ _CACHE_SHIFT,
-/*
- * The following bits are implemented in software
- */
-#define _PAGE_PRESENT_SHIFT (24)
-#define _PAGE_PRESENT (1 << _PAGE_PRESENT_SHIFT)
-#define _PAGE_READ_SHIFT (_PAGE_PRESENT_SHIFT + 1)
-#define _PAGE_READ (1 << _PAGE_READ_SHIFT)
-#define _PAGE_WRITE_SHIFT (_PAGE_READ_SHIFT + 1)
-#define _PAGE_WRITE (1 << _PAGE_WRITE_SHIFT)
-#define _PAGE_ACCESSED_SHIFT (_PAGE_WRITE_SHIFT + 1)
-#define _PAGE_ACCESSED (1 << _PAGE_ACCESSED_SHIFT)
-#define _PAGE_MODIFIED_SHIFT (_PAGE_ACCESSED_SHIFT + 1)
-#define _PAGE_MODIFIED (1 << _PAGE_MODIFIED_SHIFT)
-
-#define _PFN_SHIFT (PAGE_SHIFT - 12 + _CACHE_SHIFT + 3)
+ /* Used only by software (masked out before writing EntryLo*) */
+ _PAGE_PRESENT_SHIFT = 24,
+ _PAGE_WRITE_SHIFT,
+ _PAGE_ACCESSED_SHIFT,
+ _PAGE_MODIFIED_SHIFT,
+};
/*
* Bits for extended EntryLo0/EntryLo1 registers
*/
#define _PFNX_MASK 0xffffff
+#elif defined(CONFIG_PHYS_ADDR_T_64BIT) && defined(CONFIG_CPU_MIPS32)
+
+/*
+ * Page table bit offsets used for 36 bit physical addressing on MIPS32,
+ * for example with Alchemy or Netlogic XLP/XLR.
+ */
+enum pgtable_bits {
+ /* Used by TLB hardware (placed in EntryLo*) */
+ _PAGE_GLOBAL_SHIFT,
+ _PAGE_VALID_SHIFT,
+ _PAGE_DIRTY_SHIFT,
+ _CACHE_SHIFT,
+
+ /* Used only by software (masked out before writing EntryLo*) */
+ _PAGE_PRESENT_SHIFT = _CACHE_SHIFT + 3,
+ _PAGE_NO_READ_SHIFT,
+ _PAGE_WRITE_SHIFT,
+ _PAGE_ACCESSED_SHIFT,
+ _PAGE_MODIFIED_SHIFT,
+};
+
#elif defined(CONFIG_CPU_R3000) || defined(CONFIG_CPU_TX39XX)
-/*
- * The following bits are implemented in software
- */
-#define _PAGE_PRESENT_SHIFT (0)
-#define _PAGE_PRESENT (1 << _PAGE_PRESENT_SHIFT)
-#define _PAGE_READ_SHIFT (_PAGE_PRESENT_SHIFT + 1)
-#define _PAGE_READ (1 << _PAGE_READ_SHIFT)
-#define _PAGE_WRITE_SHIFT (_PAGE_READ_SHIFT + 1)
-#define _PAGE_WRITE (1 << _PAGE_WRITE_SHIFT)
-#define _PAGE_ACCESSED_SHIFT (_PAGE_WRITE_SHIFT + 1)
-#define _PAGE_ACCESSED (1 << _PAGE_ACCESSED_SHIFT)
-#define _PAGE_MODIFIED_SHIFT (_PAGE_ACCESSED_SHIFT + 1)
-#define _PAGE_MODIFIED (1 << _PAGE_MODIFIED_SHIFT)
+/* Page table bits used for r3k systems */
+enum pgtable_bits {
+ /* Used only by software (writes to EntryLo ignored) */
+ _PAGE_PRESENT_SHIFT,
+ _PAGE_NO_READ_SHIFT,
+ _PAGE_WRITE_SHIFT,
+ _PAGE_ACCESSED_SHIFT,
+ _PAGE_MODIFIED_SHIFT,
-/*
- * The following bits are implemented by the TLB hardware
- */
-#define _PAGE_GLOBAL_SHIFT (_PAGE_MODIFIED_SHIFT + 4)
-#define _PAGE_GLOBAL (1 << _PAGE_GLOBAL_SHIFT)
-#define _PAGE_VALID_SHIFT (_PAGE_GLOBAL_SHIFT + 1)
-#define _PAGE_VALID (1 << _PAGE_VALID_SHIFT)
-#define _PAGE_DIRTY_SHIFT (_PAGE_VALID_SHIFT + 1)
-#define _PAGE_DIRTY (1 << _PAGE_DIRTY_SHIFT)
-#define _CACHE_UNCACHED_SHIFT (_PAGE_DIRTY_SHIFT + 1)
-#define _CACHE_UNCACHED (1 << _CACHE_UNCACHED_SHIFT)
-#define _CACHE_MASK _CACHE_UNCACHED
-
-#define _PFN_SHIFT PAGE_SHIFT
+ /* Used by TLB hardware (placed in EntryLo) */
+ _PAGE_GLOBAL_SHIFT = 8,
+ _PAGE_VALID_SHIFT,
+ _PAGE_DIRTY_SHIFT,
+ _CACHE_UNCACHED_SHIFT,
+};
#else
-/*
- * Below are the "Normal" R4K cases
- */
-/*
- * The following bits are implemented in software
- */
-#define _PAGE_PRESENT_SHIFT 0
-#define _PAGE_PRESENT (1 << _PAGE_PRESENT_SHIFT)
-/* R2 or later cores check for RI/XI support to determine _PAGE_READ */
-#if defined(CONFIG_CPU_MIPSR2) || defined(CONFIG_CPU_MIPSR6)
-#define _PAGE_WRITE_SHIFT (_PAGE_PRESENT_SHIFT + 1)
-#define _PAGE_WRITE (1 << _PAGE_WRITE_SHIFT)
-#else
-#define _PAGE_READ_SHIFT (_PAGE_PRESENT_SHIFT + 1)
-#define _PAGE_READ (1 << _PAGE_READ_SHIFT)
-#define _PAGE_WRITE_SHIFT (_PAGE_READ_SHIFT + 1)
-#define _PAGE_WRITE (1 << _PAGE_WRITE_SHIFT)
+/* Page table bits used for r4k systems */
+enum pgtable_bits {
+ /* Used only by software (masked out before writing EntryLo*) */
+ _PAGE_PRESENT_SHIFT,
+#if !defined(CONFIG_CPU_HAS_RIXI)
+ _PAGE_NO_READ_SHIFT,
#endif
-#define _PAGE_ACCESSED_SHIFT (_PAGE_WRITE_SHIFT + 1)
-#define _PAGE_ACCESSED (1 << _PAGE_ACCESSED_SHIFT)
-#define _PAGE_MODIFIED_SHIFT (_PAGE_ACCESSED_SHIFT + 1)
-#define _PAGE_MODIFIED (1 << _PAGE_MODIFIED_SHIFT)
-
+ _PAGE_WRITE_SHIFT,
+ _PAGE_ACCESSED_SHIFT,
+ _PAGE_MODIFIED_SHIFT,
#if defined(CONFIG_64BIT) && defined(CONFIG_MIPS_HUGE_TLB_SUPPORT)
-/* Huge TLB page */
-#define _PAGE_HUGE_SHIFT (_PAGE_MODIFIED_SHIFT + 1)
-#define _PAGE_HUGE (1 << _PAGE_HUGE_SHIFT)
-#endif /* CONFIG_64BIT && CONFIG_MIPS_HUGE_TLB_SUPPORT */
-
-#if defined(CONFIG_CPU_MIPSR2) || defined(CONFIG_CPU_MIPSR6)
-/* XI - page cannot be executed */
-#ifdef _PAGE_HUGE_SHIFT
-#define _PAGE_NO_EXEC_SHIFT (_PAGE_HUGE_SHIFT + 1)
-#else
-#define _PAGE_NO_EXEC_SHIFT (_PAGE_MODIFIED_SHIFT + 1)
+ _PAGE_HUGE_SHIFT,
#endif
-#define _PAGE_NO_EXEC (cpu_has_rixi ? (1 << _PAGE_NO_EXEC_SHIFT) : 0)
-/* RI - page cannot be read */
-#define _PAGE_READ_SHIFT (_PAGE_NO_EXEC_SHIFT + 1)
-#define _PAGE_READ (cpu_has_rixi ? 0 : (1 << _PAGE_READ_SHIFT))
-#define _PAGE_NO_READ_SHIFT _PAGE_READ_SHIFT
-#define _PAGE_NO_READ (cpu_has_rixi ? (1 << _PAGE_READ_SHIFT) : 0)
-#endif /* defined(CONFIG_CPU_MIPSR2) || defined(CONFIG_CPU_MIPSR6) */
-
-#if defined(_PAGE_NO_READ_SHIFT)
-#define _PAGE_GLOBAL_SHIFT (_PAGE_NO_READ_SHIFT + 1)
-#elif defined(_PAGE_HUGE_SHIFT)
-#define _PAGE_GLOBAL_SHIFT (_PAGE_HUGE_SHIFT + 1)
-#else
-#define _PAGE_GLOBAL_SHIFT (_PAGE_MODIFIED_SHIFT + 1)
+ /* Used by TLB hardware (placed in EntryLo*) */
+#if defined(CONFIG_CPU_HAS_RIXI)
+ _PAGE_NO_EXEC_SHIFT,
+ _PAGE_NO_READ_SHIFT,
#endif
-#define _PAGE_GLOBAL (1 << _PAGE_GLOBAL_SHIFT)
-
-#define _PAGE_VALID_SHIFT (_PAGE_GLOBAL_SHIFT + 1)
-#define _PAGE_VALID (1 << _PAGE_VALID_SHIFT)
-#define _PAGE_DIRTY_SHIFT (_PAGE_VALID_SHIFT + 1)
-#define _PAGE_DIRTY (1 << _PAGE_DIRTY_SHIFT)
-#define _CACHE_SHIFT (_PAGE_DIRTY_SHIFT + 1)
-#define _CACHE_MASK (7 << _CACHE_SHIFT)
-
-#define _PFN_SHIFT (PAGE_SHIFT - 12 + _CACHE_SHIFT + 3)
+ _PAGE_GLOBAL_SHIFT,
+ _PAGE_VALID_SHIFT,
+ _PAGE_DIRTY_SHIFT,
+ _CACHE_SHIFT,
+};
#endif /* defined(CONFIG_PHYS_ADDR_T_64BIT && defined(CONFIG_CPU_MIPS32) */
+/* Used only by software */
+#define _PAGE_PRESENT (1 << _PAGE_PRESENT_SHIFT)
+#define _PAGE_WRITE (1 << _PAGE_WRITE_SHIFT)
+#define _PAGE_ACCESSED (1 << _PAGE_ACCESSED_SHIFT)
+#define _PAGE_MODIFIED (1 << _PAGE_MODIFIED_SHIFT)
+#if defined(CONFIG_64BIT) && defined(CONFIG_MIPS_HUGE_TLB_SUPPORT)
+# define _PAGE_HUGE (1 << _PAGE_HUGE_SHIFT)
+#endif
+
+/* Used by TLB hardware (placed in EntryLo*) */
+#if defined(CONFIG_XPA)
+# define _PAGE_NO_EXEC (1 << _PAGE_NO_EXEC_SHIFT)
+#elif defined(CONFIG_CPU_HAS_RIXI)
+# define _PAGE_NO_EXEC (cpu_has_rixi ? (1 << _PAGE_NO_EXEC_SHIFT) : 0)
+#endif
+#define _PAGE_NO_READ (1 << _PAGE_NO_READ_SHIFT)
+#define _PAGE_GLOBAL (1 << _PAGE_GLOBAL_SHIFT)
+#define _PAGE_VALID (1 << _PAGE_VALID_SHIFT)
+#define _PAGE_DIRTY (1 << _PAGE_DIRTY_SHIFT)
+#if defined(CONFIG_CPU_R3000) || defined(CONFIG_CPU_TX39XX)
+# define _CACHE_UNCACHED (1 << _CACHE_UNCACHED_SHIFT)
+# define _CACHE_MASK _CACHE_UNCACHED
+# define _PFN_SHIFT PAGE_SHIFT
+#else
+# define _CACHE_MASK (7 << _CACHE_SHIFT)
+# define _PFN_SHIFT (PAGE_SHIFT - 12 + _CACHE_SHIFT + 3)
+#endif
+
#ifndef _PAGE_NO_EXEC
#define _PAGE_NO_EXEC 0
#endif
-#ifndef _PAGE_NO_READ
-#define _PAGE_NO_READ 0
-#endif
#define _PAGE_SILENT_READ _PAGE_VALID
#define _PAGE_SILENT_WRITE _PAGE_DIRTY
@@ -191,14 +174,13 @@
*/
-#ifndef __ASSEMBLY__
/*
* pte_to_entrylo converts a page table entry (PTE) into a Mips
* entrylo0/1 value.
*/
static inline uint64_t pte_to_entrylo(unsigned long pte_val)
{
-#if defined(CONFIG_CPU_MIPSR2) || defined(CONFIG_CPU_MIPSR6)
+#ifdef CONFIG_CPU_HAS_RIXI
if (cpu_has_rixi) {
int sa;
#ifdef CONFIG_32BIT
@@ -218,7 +200,6 @@
return pte_val >> _PAGE_GLOBAL_SHIFT;
}
-#endif
/*
* Cache attributes
@@ -274,7 +255,7 @@
#define _CACHE_UNCACHED_ACCELERATED (7<<_CACHE_SHIFT)
#endif
-#define __READABLE (_PAGE_SILENT_READ | _PAGE_READ | _PAGE_ACCESSED)
+#define __READABLE (_PAGE_SILENT_READ | _PAGE_ACCESSED)
#define __WRITEABLE (_PAGE_SILENT_WRITE | _PAGE_WRITE | _PAGE_MODIFIED)
#define _PAGE_CHG_MASK (_PAGE_ACCESSED | _PAGE_MODIFIED | \
diff --git a/arch/mips/include/asm/pgtable.h b/arch/mips/include/asm/pgtable.h
index 9a4fe01..a6b611f 100644
--- a/arch/mips/include/asm/pgtable.h
+++ b/arch/mips/include/asm/pgtable.h
@@ -23,18 +23,19 @@
struct mm_struct;
struct vm_area_struct;
-#define PAGE_NONE __pgprot(_PAGE_PRESENT | _CACHE_CACHABLE_NONCOHERENT)
-#define PAGE_SHARED __pgprot(_PAGE_PRESENT | _PAGE_WRITE | _PAGE_READ | \
+#define PAGE_NONE __pgprot(_PAGE_PRESENT | _PAGE_NO_READ | \
+ _CACHE_CACHABLE_NONCOHERENT)
+#define PAGE_SHARED __pgprot(_PAGE_PRESENT | _PAGE_WRITE | \
_page_cachable_default)
-#define PAGE_COPY __pgprot(_PAGE_PRESENT | _PAGE_READ | _PAGE_NO_EXEC | \
+#define PAGE_COPY __pgprot(_PAGE_PRESENT | _PAGE_NO_EXEC | \
_page_cachable_default)
-#define PAGE_READONLY __pgprot(_PAGE_PRESENT | _PAGE_READ | \
+#define PAGE_READONLY __pgprot(_PAGE_PRESENT | \
_page_cachable_default)
#define PAGE_KERNEL __pgprot(_PAGE_PRESENT | __READABLE | __WRITEABLE | \
_PAGE_GLOBAL | _page_cachable_default)
#define PAGE_KERNEL_NC __pgprot(_PAGE_PRESENT | __READABLE | __WRITEABLE | \
_PAGE_GLOBAL | _CACHE_CACHABLE_NONCOHERENT)
-#define PAGE_USERIO __pgprot(_PAGE_PRESENT | _PAGE_READ | _PAGE_WRITE | \
+#define PAGE_USERIO __pgprot(_PAGE_PRESENT | _PAGE_WRITE | \
_page_cachable_default)
#define PAGE_KERNEL_UNCACHED __pgprot(_PAGE_PRESENT | __READABLE | \
__WRITEABLE | _PAGE_GLOBAL | _CACHE_UNCACHED)
@@ -127,10 +128,19 @@
} \
} while(0)
+static inline void set_pte_at(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, pte_t pteval);
+
#if defined(CONFIG_PHYS_ADDR_T_64BIT) && defined(CONFIG_CPU_MIPS32)
-#define pte_none(pte) (!(((pte).pte_high) & ~_PAGE_GLOBAL))
+#ifdef CONFIG_XPA
+# define pte_none(pte) (!(((pte).pte_high) & ~_PAGE_GLOBAL))
+#else
+# define pte_none(pte) (!(((pte).pte_low | (pte).pte_high) & ~_PAGE_GLOBAL))
+#endif
+
#define pte_present(pte) ((pte).pte_low & _PAGE_PRESENT)
+#define pte_no_exec(pte) ((pte).pte_low & _PAGE_NO_EXEC)
static inline void set_pte(pte_t *ptep, pte_t pte)
{
@@ -138,17 +148,23 @@
smp_wmb();
ptep->pte_low = pte.pte_low;
+#ifdef CONFIG_XPA
if (pte.pte_high & _PAGE_GLOBAL) {
+#else
+ if (pte.pte_low & _PAGE_GLOBAL) {
+#endif
pte_t *buddy = ptep_buddy(ptep);
/*
* Make sure the buddy is global too (if it's !none,
* it better already be global)
*/
- if (pte_none(*buddy))
+ if (pte_none(*buddy)) {
+ if (!config_enabled(CONFIG_XPA))
+ buddy->pte_low |= _PAGE_GLOBAL;
buddy->pte_high |= _PAGE_GLOBAL;
+ }
}
}
-#define set_pte_at(mm, addr, ptep, pteval) set_pte(ptep, pteval)
static inline void pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep)
{
@@ -156,8 +172,13 @@
htw_stop();
/* Preserve global status for the pair */
- if (ptep_buddy(ptep)->pte_high & _PAGE_GLOBAL)
- null.pte_high = _PAGE_GLOBAL;
+ if (config_enabled(CONFIG_XPA)) {
+ if (ptep_buddy(ptep)->pte_high & _PAGE_GLOBAL)
+ null.pte_high = _PAGE_GLOBAL;
+ } else {
+ if (ptep_buddy(ptep)->pte_low & _PAGE_GLOBAL)
+ null.pte_low = null.pte_high = _PAGE_GLOBAL;
+ }
set_pte_at(mm, addr, ptep, null);
htw_start();
@@ -166,6 +187,7 @@
#define pte_none(pte) (!(pte_val(pte) & ~_PAGE_GLOBAL))
#define pte_present(pte) (pte_val(pte) & _PAGE_PRESENT)
+#define pte_no_exec(pte) (pte_val(pte) & _PAGE_NO_EXEC)
/*
* Certain architectures need to do special things when pte's
@@ -187,30 +209,42 @@
* For SMP, multiple CPUs can race, so we need to do
* this atomically.
*/
-#ifdef CONFIG_64BIT
-#define LL_INSN "lld"
-#define SC_INSN "scd"
-#else /* CONFIG_32BIT */
-#define LL_INSN "ll"
-#define SC_INSN "sc"
-#endif
unsigned long page_global = _PAGE_GLOBAL;
unsigned long tmp;
- __asm__ __volatile__ (
- " .set push\n"
- " .set noreorder\n"
- "1: " LL_INSN " %[tmp], %[buddy]\n"
- " bnez %[tmp], 2f\n"
- " or %[tmp], %[tmp], %[global]\n"
- " " SC_INSN " %[tmp], %[buddy]\n"
- " beqz %[tmp], 1b\n"
- " nop\n"
- "2:\n"
- " .set pop"
- : [buddy] "+m" (buddy->pte),
- [tmp] "=&r" (tmp)
+ if (kernel_uses_llsc && R10000_LLSC_WAR) {
+ __asm__ __volatile__ (
+ " .set arch=r4000 \n"
+ " .set push \n"
+ " .set noreorder \n"
+ "1:" __LL "%[tmp], %[buddy] \n"
+ " bnez %[tmp], 2f \n"
+ " or %[tmp], %[tmp], %[global] \n"
+ __SC "%[tmp], %[buddy] \n"
+ " beqzl %[tmp], 1b \n"
+ " nop \n"
+ "2: \n"
+ " .set pop \n"
+ " .set mips0 \n"
+ : [buddy] "+m" (buddy->pte), [tmp] "=&r" (tmp)
: [global] "r" (page_global));
+ } else if (kernel_uses_llsc) {
+ __asm__ __volatile__ (
+ " .set "MIPS_ISA_ARCH_LEVEL" \n"
+ " .set push \n"
+ " .set noreorder \n"
+ "1:" __LL "%[tmp], %[buddy] \n"
+ " bnez %[tmp], 2f \n"
+ " or %[tmp], %[tmp], %[global] \n"
+ __SC "%[tmp], %[buddy] \n"
+ " beqz %[tmp], 1b \n"
+ " nop \n"
+ "2: \n"
+ " .set pop \n"
+ " .set mips0 \n"
+ : [buddy] "+m" (buddy->pte), [tmp] "=&r" (tmp)
+ : [global] "r" (page_global));
+ }
#else /* !CONFIG_SMP */
if (pte_none(*buddy))
pte_val(*buddy) = pte_val(*buddy) | _PAGE_GLOBAL;
@@ -218,7 +252,6 @@
}
#endif
}
-#define set_pte_at(mm, addr, ptep, pteval) set_pte(ptep, pteval)
static inline void pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep)
{
@@ -234,6 +267,22 @@
}
#endif
+static inline void set_pte_at(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, pte_t pteval)
+{
+ extern void __update_cache(unsigned long address, pte_t pte);
+
+ if (!pte_present(pteval))
+ goto cache_sync_done;
+
+ if (pte_present(*ptep) && (pte_pfn(*ptep) == pte_pfn(pteval)))
+ goto cache_sync_done;
+
+ __update_cache(addr, pteval);
+cache_sync_done:
+ set_pte(ptep, pteval);
+}
+
/*
* (pmds are folded into puds so this doesn't get actually called,
* but the define is needed for a generic inline function.)
@@ -270,6 +319,8 @@
static inline pte_t pte_wrprotect(pte_t pte)
{
pte.pte_low &= ~_PAGE_WRITE;
+ if (!config_enabled(CONFIG_XPA))
+ pte.pte_low &= ~_PAGE_SILENT_WRITE;
pte.pte_high &= ~_PAGE_SILENT_WRITE;
return pte;
}
@@ -277,6 +328,8 @@
static inline pte_t pte_mkclean(pte_t pte)
{
pte.pte_low &= ~_PAGE_MODIFIED;
+ if (!config_enabled(CONFIG_XPA))
+ pte.pte_low &= ~_PAGE_SILENT_WRITE;
pte.pte_high &= ~_PAGE_SILENT_WRITE;
return pte;
}
@@ -284,6 +337,8 @@
static inline pte_t pte_mkold(pte_t pte)
{
pte.pte_low &= ~_PAGE_ACCESSED;
+ if (!config_enabled(CONFIG_XPA))
+ pte.pte_low &= ~_PAGE_SILENT_READ;
pte.pte_high &= ~_PAGE_SILENT_READ;
return pte;
}
@@ -291,24 +346,33 @@
static inline pte_t pte_mkwrite(pte_t pte)
{
pte.pte_low |= _PAGE_WRITE;
- if (pte.pte_low & _PAGE_MODIFIED)
+ if (pte.pte_low & _PAGE_MODIFIED) {
+ if (!config_enabled(CONFIG_XPA))
+ pte.pte_low |= _PAGE_SILENT_WRITE;
pte.pte_high |= _PAGE_SILENT_WRITE;
+ }
return pte;
}
static inline pte_t pte_mkdirty(pte_t pte)
{
pte.pte_low |= _PAGE_MODIFIED;
- if (pte.pte_low & _PAGE_WRITE)
+ if (pte.pte_low & _PAGE_WRITE) {
+ if (!config_enabled(CONFIG_XPA))
+ pte.pte_low |= _PAGE_SILENT_WRITE;
pte.pte_high |= _PAGE_SILENT_WRITE;
+ }
return pte;
}
static inline pte_t pte_mkyoung(pte_t pte)
{
pte.pte_low |= _PAGE_ACCESSED;
- if (pte.pte_low & _PAGE_READ)
+ if (!(pte.pte_low & _PAGE_NO_READ)) {
+ if (!config_enabled(CONFIG_XPA))
+ pte.pte_low |= _PAGE_SILENT_READ;
pte.pte_high |= _PAGE_SILENT_READ;
+ }
return pte;
}
#else
@@ -353,13 +417,8 @@
static inline pte_t pte_mkyoung(pte_t pte)
{
pte_val(pte) |= _PAGE_ACCESSED;
-#if defined(CONFIG_CPU_MIPSR2) || defined(CONFIG_CPU_MIPSR6)
if (!(pte_val(pte) & _PAGE_NO_READ))
pte_val(pte) |= _PAGE_SILENT_READ;
- else
-#endif
- if (pte_val(pte) & _PAGE_READ)
- pte_val(pte) |= _PAGE_SILENT_READ;
return pte;
}
@@ -411,7 +470,7 @@
*/
#define mk_pte(page, pgprot) pfn_pte(page_to_pfn(page), (pgprot))
-#if defined(CONFIG_PHYS_ADDR_T_64BIT) && defined(CONFIG_CPU_MIPS32)
+#if defined(CONFIG_XPA)
static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
{
pte.pte_low &= (_PAGE_MODIFIED | _PAGE_ACCESSED | _PFNX_MASK);
@@ -420,6 +479,15 @@
pte.pte_high |= pgprot_val(newprot) & ~_PFN_MASK;
return pte;
}
+#elif defined(CONFIG_PHYS_ADDR_T_64BIT) && defined(CONFIG_CPU_MIPS32)
+static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
+{
+ pte.pte_low &= _PAGE_CHG_MASK;
+ pte.pte_high &= (_PFN_MASK | _CACHE_MASK);
+ pte.pte_low |= pgprot_val(newprot);
+ pte.pte_high |= pgprot_val(newprot) & ~(_PFN_MASK | _CACHE_MASK);
+ return pte;
+}
#else
static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
{
@@ -430,15 +498,12 @@
extern void __update_tlb(struct vm_area_struct *vma, unsigned long address,
pte_t pte);
-extern void __update_cache(struct vm_area_struct *vma, unsigned long address,
- pte_t pte);
static inline void update_mmu_cache(struct vm_area_struct *vma,
unsigned long address, pte_t *ptep)
{
pte_t pte = *ptep;
__update_tlb(vma, address, pte);
- __update_cache(vma, address, pte);
}
static inline void update_mmu_cache_pmd(struct vm_area_struct *vma,
@@ -468,6 +533,7 @@
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+#define has_transparent_hugepage has_transparent_hugepage
extern int has_transparent_hugepage(void);
static inline int pmd_trans_huge(pmd_t pmd)
@@ -542,13 +608,8 @@
{
pmd_val(pmd) |= _PAGE_ACCESSED;
-#if defined(CONFIG_CPU_MIPSR2) || defined(CONFIG_CPU_MIPSR6)
if (!(pmd_val(pmd) & _PAGE_NO_READ))
pmd_val(pmd) |= _PAGE_SILENT_READ;
- else
-#endif
- if (pmd_val(pmd) & _PAGE_READ)
- pmd_val(pmd) |= _PAGE_SILENT_READ;
return pmd;
}
diff --git a/arch/mips/include/asm/processor.h b/arch/mips/include/asm/processor.h
index 041153f..7e78b62 100644
--- a/arch/mips/include/asm/processor.h
+++ b/arch/mips/include/asm/processor.h
@@ -63,7 +63,11 @@
* 8192EB ...
*/
#define TASK_SIZE32 0x7fff8000UL
-#define TASK_SIZE64 0x10000000000UL
+#ifdef CONFIG_MIPS_VA_BITS_48
+#define TASK_SIZE64 (0x1UL << ((cpu_data[0].vmbits>48)?48:cpu_data[0].vmbits))
+#else
+#define TASK_SIZE64 0x10000000000UL
+#endif
#define TASK_SIZE (test_thread_flag(TIF_32BIT_ADDR) ? TASK_SIZE32 : TASK_SIZE64)
#define STACK_TOP_MAX TASK_SIZE64
@@ -355,6 +359,10 @@
*/
extern void start_thread(struct pt_regs * regs, unsigned long pc, unsigned long sp);
+static inline void flush_thread(void)
+{
+}
+
unsigned long get_wchan(struct task_struct *p);
#define __KSTK_TOS(tsk) ((unsigned long)task_stack_page(tsk) + \
diff --git a/arch/mips/include/asm/seccomp.h b/arch/mips/include/asm/seccomp.h
index 1d8a2e2..684fb3a 100644
--- a/arch/mips/include/asm/seccomp.h
+++ b/arch/mips/include/asm/seccomp.h
@@ -2,27 +2,32 @@
#include <linux/unistd.h>
-/*
- * Kludge alert:
- *
- * The generic seccomp code currently allows only a single compat ABI. Until
- * this is fixed we priorize O32 as the compat ABI over N32.
- */
-#ifdef CONFIG_MIPS32_O32
+#ifdef CONFIG_COMPAT
+static inline const int *get_compat_mode1_syscalls(void)
+{
+ static const int syscalls_O32[] = {
+ __NR_O32_Linux + 3, __NR_O32_Linux + 4,
+ __NR_O32_Linux + 1, __NR_O32_Linux + 193,
+ 0, /* null terminated */
+ };
+ static const int syscalls_N32[] = {
+ __NR_N32_Linux + 0, __NR_N32_Linux + 1,
+ __NR_N32_Linux + 58, __NR_N32_Linux + 211,
+ 0, /* null terminated */
+ };
-#define __NR_seccomp_read_32 4003
-#define __NR_seccomp_write_32 4004
-#define __NR_seccomp_exit_32 4001
-#define __NR_seccomp_sigreturn_32 4193 /* rt_sigreturn */
+ if (config_enabled(CONFIG_MIPS32_O32) && test_thread_flag(TIF_32BIT_REGS))
+ return syscalls_O32;
-#elif defined(CONFIG_MIPS32_N32)
+ if (config_enabled(CONFIG_MIPS32_N32))
+ return syscalls_N32;
-#define __NR_seccomp_read_32 6000
-#define __NR_seccomp_write_32 6001
-#define __NR_seccomp_exit_32 6058
-#define __NR_seccomp_sigreturn_32 6211 /* rt_sigreturn */
+ BUG();
+}
-#endif /* CONFIG_MIPS32_O32 */
+#define get_compat_mode1_syscalls get_compat_mode1_syscalls
+
+#endif /* CONFIG_COMPAT */
#include <asm-generic/seccomp.h>
diff --git a/arch/mips/include/asm/sibyte/bcm1480_regs.h b/arch/mips/include/asm/sibyte/bcm1480_regs.h
index ec0dacf..32a8483 100644
--- a/arch/mips/include/asm/sibyte/bcm1480_regs.h
+++ b/arch/mips/include/asm/sibyte/bcm1480_regs.h
@@ -415,8 +415,8 @@
(cpu)*BCM1480_IMR_ALIAS_MAILBOX_SPACING)
#define A_BCM1480_IMR_ALIAS_MAILBOX_REGISTER(cpu, reg) (A_BCM1480_IMR_ALIAS_MAILBOX(cpu)+(reg))
-#define R_BCM1480_IMR_ALIAS_MAILBOX_0 0x0000 /* 0x0x0 */
-#define R_BCM1480_IMR_ALIAS_MAILBOX_0_SET 0x0008 /* 0x0x8 */
+#define R_BCM1480_IMR_ALIAS_MAILBOX_0 0x0000
+#define R_BCM1480_IMR_ALIAS_MAILBOX_0_SET 0x0008
/*
* these macros work together to build the address of a mailbox
diff --git a/arch/mips/include/asm/signal.h b/arch/mips/include/asm/signal.h
index 003e273..2292373 100644
--- a/arch/mips/include/asm/signal.h
+++ b/arch/mips/include/asm/signal.h
@@ -11,11 +11,17 @@
#include <uapi/asm/signal.h>
+#ifdef CONFIG_MIPS32_COMPAT
+extern struct mips_abi mips_abi_32;
-#ifdef CONFIG_TRAD_SIGNALS
-#define sig_uses_siginfo(ka) ((ka)->sa.sa_flags & SA_SIGINFO)
+#define sig_uses_siginfo(ka, abi) \
+ ((abi != &mips_abi_32) ? 1 : \
+ ((ka)->sa.sa_flags & SA_SIGINFO))
#else
-#define sig_uses_siginfo(ka) (1)
+#define sig_uses_siginfo(ka, abi) \
+ (config_enabled(CONFIG_64BIT) ? 1 : \
+ (config_enabled(CONFIG_TRAD_SIGNALS) ? \
+ ((ka)->sa.sa_flags & SA_SIGINFO) : 1) )
#endif
#include <asm/sigcontext.h>
diff --git a/arch/mips/include/asm/smp-cps.h b/arch/mips/include/asm/smp-cps.h
index 326c16e..2ae1f61 100644
--- a/arch/mips/include/asm/smp-cps.h
+++ b/arch/mips/include/asm/smp-cps.h
@@ -29,7 +29,7 @@
extern void mips_cps_core_entry(void);
extern void mips_cps_core_init(void);
-extern struct vpe_boot_config *mips_cps_boot_vpes(void);
+extern void mips_cps_boot_vpes(struct core_boot_config *cfg, unsigned vpe);
extern void mips_cps_pm_save(void);
extern void mips_cps_pm_restore(void);
diff --git a/arch/mips/include/asm/switch_to.h b/arch/mips/include/asm/switch_to.h
index 28b5d84a..ebb5c0f 100644
--- a/arch/mips/include/asm/switch_to.h
+++ b/arch/mips/include/asm/switch_to.h
@@ -105,7 +105,7 @@
__clear_software_ll_bit(); \
if (cpu_has_userlocal) \
write_c0_userlocal(task_thread_info(next)->tp_value); \
- __restore_watch(); \
+ __restore_watch(next); \
(last) = resume(prev, next, task_thread_info(next)); \
} while (0)
diff --git a/arch/mips/include/asm/uasm.h b/arch/mips/include/asm/uasm.h
index fc1cdd2..b6ecfee 100644
--- a/arch/mips/include/asm/uasm.h
+++ b/arch/mips/include/asm/uasm.h
@@ -171,7 +171,8 @@
Ip_u3u1u2(_xor);
Ip_u2u1u3(_xori);
Ip_u2u1(_yield);
-
+Ip_u1u2(_ldpte);
+Ip_u2u1u3(_lddir);
/* Handle labels. */
struct uasm_label {
diff --git a/arch/mips/include/asm/watch.h b/arch/mips/include/asm/watch.h
index 20126ec..6ffe3ea 100644
--- a/arch/mips/include/asm/watch.h
+++ b/arch/mips/include/asm/watch.h
@@ -12,21 +12,21 @@
#include <asm/mipsregs.h>
-void mips_install_watch_registers(void);
+void mips_install_watch_registers(struct task_struct *t);
void mips_read_watch_registers(void);
void mips_clear_watch_registers(void);
void mips_probe_watch_registers(struct cpuinfo_mips *c);
#ifdef CONFIG_HARDWARE_WATCHPOINTS
-#define __restore_watch() do { \
+#define __restore_watch(task) do { \
if (unlikely(test_bit(TIF_LOAD_WATCH, \
- ¤t_thread_info()->flags))) { \
- mips_install_watch_registers(); \
+ &task_thread_info(task)->flags))) { \
+ mips_install_watch_registers(task); \
} \
} while (0)
#else
-#define __restore_watch() do {} while (0)
+#define __restore_watch(task) do {} while (0)
#endif
#endif /* _ASM_WATCH_H */
diff --git a/arch/mips/include/uapi/asm/inst.h b/arch/mips/include/uapi/asm/inst.h
index ddea53e..8051f9a 100644
--- a/arch/mips/include/uapi/asm/inst.h
+++ b/arch/mips/include/uapi/asm/inst.h
@@ -167,6 +167,7 @@
fceill_op = 0x0a, ffloorl_op = 0x0b,
fround_op = 0x0c, ftrunc_op = 0x0d,
fceil_op = 0x0e, ffloor_op = 0x0f,
+ fsel_op = 0x10,
fmovc_op = 0x11, fmovz_op = 0x12,
fmovn_op = 0x13, fseleqz_op = 0x14,
frecip_op = 0x15, frsqrt_op = 0x16,
@@ -204,6 +205,16 @@
};
/*
+ * func field for page table walker (Loongson-3).
+ */
+enum ptw_func {
+ lwdir_op = 0x00,
+ lwpte_op = 0x01,
+ lddir_op = 0x02,
+ ldpte_op = 0x03,
+};
+
+/*
* func field for special3 lx opcodes (Cavium Octeon).
*/
enum lx_func {
diff --git a/arch/mips/include/uapi/asm/siginfo.h b/arch/mips/include/uapi/asm/siginfo.h
index cc49dc2..8069cf7 100644
--- a/arch/mips/include/uapi/asm/siginfo.h
+++ b/arch/mips/include/uapi/asm/siginfo.h
@@ -28,7 +28,7 @@
#define __ARCH_SIGSYS
-#include <uapi/asm-generic/siginfo.h>
+#include <asm-generic/siginfo.h>
/* We can't use generic siginfo_t, because our si_code and si_errno are swapped */
typedef struct siginfo {
@@ -42,13 +42,13 @@
/* kill() */
struct {
- pid_t _pid; /* sender's pid */
+ __kernel_pid_t _pid; /* sender's pid */
__ARCH_SI_UID_T _uid; /* sender's uid */
} _kill;
/* POSIX.1b timers */
struct {
- timer_t _tid; /* timer id */
+ __kernel_timer_t _tid; /* timer id */
int _overrun; /* overrun count */
char _pad[sizeof( __ARCH_SI_UID_T) - sizeof(int)];
sigval_t _sigval; /* same as below */
@@ -57,26 +57,26 @@
/* POSIX.1b signals */
struct {
- pid_t _pid; /* sender's pid */
+ __kernel_pid_t _pid; /* sender's pid */
__ARCH_SI_UID_T _uid; /* sender's uid */
sigval_t _sigval;
} _rt;
/* SIGCHLD */
struct {
- pid_t _pid; /* which child */
+ __kernel_pid_t _pid; /* which child */
__ARCH_SI_UID_T _uid; /* sender's uid */
int _status; /* exit code */
- clock_t _utime;
- clock_t _stime;
+ __kernel_clock_t _utime;
+ __kernel_clock_t _stime;
} _sigchld;
/* IRIX SIGCHLD */
struct {
- pid_t _pid; /* which child */
- clock_t _utime;
+ __kernel_pid_t _pid; /* which child */
+ __kernel_clock_t _utime;
int _status; /* exit code */
- clock_t _stime;
+ __kernel_clock_t _stime;
} _irix_sigchld;
/* SIGILL, SIGFPE, SIGSEGV, SIGBUS */
@@ -123,6 +123,4 @@
#define SI_TIMER __SI_CODE(__SI_TIMER, -3) /* sent by timer expiration */
#define SI_MESGQ __SI_CODE(__SI_MESGQ, -4) /* sent by real time mesq state change */
-#include <asm-generic/siginfo.h>
-
#endif /* _UAPI_ASM_SIGINFO_H */
diff --git a/arch/mips/jz4740/board-qi_lb60.c b/arch/mips/jz4740/board-qi_lb60.c
index 934b15b..258fd03 100644
--- a/arch/mips/jz4740/board-qi_lb60.c
+++ b/arch/mips/jz4740/board-qi_lb60.c
@@ -39,8 +39,6 @@
#include "clock.h"
-static bool is_avt2;
-
/* GPIOs */
#define QI_LB60_GPIO_SD_CD JZ_GPIO_PORTD(0)
#define QI_LB60_GPIO_SD_VCC_EN_N JZ_GPIO_PORTD(2)
@@ -50,20 +48,6 @@
#define QI_LB60_GPIO_KEYIN8 JZ_GPIO_PORTD(26)
/* NAND */
-static struct nand_ecclayout qi_lb60_ecclayout_1gb = {
- .eccbytes = 36,
- .eccpos = {
- 6, 7, 8, 9, 10, 11, 12, 13,
- 14, 15, 16, 17, 18, 19, 20, 21,
- 22, 23, 24, 25, 26, 27, 28, 29,
- 30, 31, 32, 33, 34, 35, 36, 37,
- 38, 39, 40, 41
- },
- .oobfree = {
- { .offset = 2, .length = 4 },
- { .offset = 42, .length = 22 }
- },
-};
/* Early prototypes of the QI LB60 had only 1GB of NAND.
* In order to support these devices as well the partition and ecc layout is
@@ -86,25 +70,6 @@
},
};
-static struct nand_ecclayout qi_lb60_ecclayout_2gb = {
- .eccbytes = 72,
- .eccpos = {
- 12, 13, 14, 15, 16, 17, 18, 19,
- 20, 21, 22, 23, 24, 25, 26, 27,
- 28, 29, 30, 31, 32, 33, 34, 35,
- 36, 37, 38, 39, 40, 41, 42, 43,
- 44, 45, 46, 47, 48, 49, 50, 51,
- 52, 53, 54, 55, 56, 57, 58, 59,
- 60, 61, 62, 63, 64, 65, 66, 67,
- 68, 69, 70, 71, 72, 73, 74, 75,
- 76, 77, 78, 79, 80, 81, 82, 83
- },
- .oobfree = {
- { .offset = 2, .length = 10 },
- { .offset = 84, .length = 44 },
- },
-};
-
static struct mtd_partition qi_lb60_partitions_2gb[] = {
{
.name = "NAND BOOT partition",
@@ -123,19 +88,67 @@
},
};
+static int qi_lb60_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section)
+ return -ERANGE;
+
+ oobregion->length = 36;
+ oobregion->offset = 6;
+
+ if (mtd->oobsize == 128) {
+ oobregion->length *= 2;
+ oobregion->offset *= 2;
+ }
+
+ return 0;
+}
+
+static int qi_lb60_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ int eccbytes = 36, eccoff = 6;
+
+ if (section > 1)
+ return -ERANGE;
+
+ if (mtd->oobsize == 128) {
+ eccbytes *= 2;
+ eccoff *= 2;
+ }
+
+ if (!section) {
+ oobregion->offset = 2;
+ oobregion->length = eccoff - 2;
+ } else {
+ oobregion->offset = eccoff + eccbytes;
+ oobregion->length = mtd->oobsize - oobregion->offset;
+ }
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops qi_lb60_ooblayout_ops = {
+ .ecc = qi_lb60_ooblayout_ecc,
+ .free = qi_lb60_ooblayout_free,
+};
+
static void qi_lb60_nand_ident(struct platform_device *pdev,
- struct nand_chip *chip, struct mtd_partition **partitions,
+ struct mtd_info *mtd, struct mtd_partition **partitions,
int *num_partitions)
{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+
if (chip->page_shift == 12) {
- chip->ecc.layout = &qi_lb60_ecclayout_2gb;
*partitions = qi_lb60_partitions_2gb;
*num_partitions = ARRAY_SIZE(qi_lb60_partitions_2gb);
} else {
- chip->ecc.layout = &qi_lb60_ecclayout_1gb;
*partitions = qi_lb60_partitions_1gb;
*num_partitions = ARRAY_SIZE(qi_lb60_partitions_1gb);
}
+
+ mtd_set_ooblayout(mtd, &qi_lb60_ooblayout_ops);
}
static struct jz_nand_platform_data qi_lb60_nand_pdata = {
@@ -367,43 +380,12 @@
.power_active_low = 1,
};
-/* OHCI */
-static struct regulator_consumer_supply avt2_usb_regulator_consumer =
- REGULATOR_SUPPLY("vbus", "jz4740-ohci");
-
-static struct regulator_init_data avt2_usb_regulator_init_data = {
- .num_consumer_supplies = 1,
- .consumer_supplies = &avt2_usb_regulator_consumer,
- .constraints = {
- .name = "USB power",
- .min_uV = 5000000,
- .max_uV = 5000000,
- .valid_modes_mask = REGULATOR_MODE_NORMAL,
- .valid_ops_mask = REGULATOR_CHANGE_STATUS,
- },
-};
-
-static struct fixed_voltage_config avt2_usb_regulator_data = {
- .supply_name = "USB power",
- .microvolts = 5000000,
- .gpio = JZ_GPIO_PORTB(17),
- .init_data = &avt2_usb_regulator_init_data,
-};
-
-static struct platform_device avt2_usb_regulator_device = {
- .name = "reg-fixed-voltage",
- .id = -1,
- .dev = {
- .platform_data = &avt2_usb_regulator_data,
- }
-};
-
+/* beeper */
static struct pwm_lookup qi_lb60_pwm_lookup[] = {
PWM_LOOKUP("jz4740-pwm", 4, "pwm-beeper", NULL, 0,
PWM_POLARITY_NORMAL),
};
-/* beeper */
static struct platform_device qi_lb60_pwm_beeper = {
.name = "pwm-beeper",
.id = -1,
@@ -487,11 +469,6 @@
spi_register_board_info(qi_lb60_spi_board_info,
ARRAY_SIZE(qi_lb60_spi_board_info));
- if (is_avt2) {
- platform_device_register(&avt2_usb_regulator_device);
- platform_device_register(&jz4740_usb_ohci_device);
- }
-
pwm_add_table(qi_lb60_pwm_lookup, ARRAY_SIZE(qi_lb60_pwm_lookup));
return platform_add_devices(jz_platform_devices,
@@ -499,19 +476,9 @@
}
-static __init int board_avt2(char *str)
-{
- qi_lb60_mmc_pdata.card_detect_active_low = 1;
- is_avt2 = true;
-
- return 1;
-}
-__setup("avt2", board_avt2);
-
static int __init qi_lb60_board_setup(void)
{
- printk(KERN_INFO "Qi Hardware JZ4740 QI %s setup\n",
- is_avt2 ? "AVT2" : "LB60");
+ printk(KERN_INFO "Qi Hardware JZ4740 QI LB60 setup\n");
board_gpio_setup();
diff --git a/arch/mips/jz4740/platform.c b/arch/mips/jz4740/platform.c
index e8a463b..2f1dab3 100644
--- a/arch/mips/jz4740/platform.c
+++ b/arch/mips/jz4740/platform.c
@@ -32,31 +32,6 @@
#include "clock.h"
-/* OHCI controller */
-static struct resource jz4740_usb_ohci_resources[] = {
- {
- .start = JZ4740_UHC_BASE_ADDR,
- .end = JZ4740_UHC_BASE_ADDR + 0x1000 - 1,
- .flags = IORESOURCE_MEM,
- },
- {
- .start = JZ4740_IRQ_UHC,
- .end = JZ4740_IRQ_UHC,
- .flags = IORESOURCE_IRQ,
- },
-};
-
-struct platform_device jz4740_usb_ohci_device = {
- .name = "jz4740-ohci",
- .id = -1,
- .dev = {
- .dma_mask = &jz4740_usb_ohci_device.dev.coherent_dma_mask,
- .coherent_dma_mask = DMA_BIT_MASK(32),
- },
- .num_resources = ARRAY_SIZE(jz4740_usb_ohci_resources),
- .resource = jz4740_usb_ohci_resources,
-};
-
/* USB Device Controller */
struct platform_device jz4740_udc_xceiv_device = {
.name = "usb_phy_generic",
diff --git a/arch/mips/kernel/Makefile b/arch/mips/kernel/Makefile
index b0988fd..e6053d0 100644
--- a/arch/mips/kernel/Makefile
+++ b/arch/mips/kernel/Makefile
@@ -44,7 +44,7 @@
obj-$(CONFIG_SMP) += smp.o
obj-$(CONFIG_SMP_UP) += smp-up.o
-obj-$(CONFIG_CPU_BMIPS) += smp-bmips.o bmips_vec.o
+obj-$(CONFIG_CPU_BMIPS) += smp-bmips.o bmips_vec.o bmips_5xxx_init.o
obj-$(CONFIG_MIPS_MT) += mips-mt.o
obj-$(CONFIG_MIPS_MT_FPAFF) += mips-mt-fpaff.o
@@ -83,6 +83,8 @@
obj-$(CONFIG_GPIO_TXX9) += gpio_txx9.o
+obj-$(CONFIG_RELOCATABLE) += relocate.o
+
obj-$(CONFIG_KEXEC) += machine_kexec.o relocate_kernel.o crash.o
obj-$(CONFIG_CRASH_DUMP) += crash_dump.o
obj-$(CONFIG_EARLY_PRINTK) += early_printk.o
diff --git a/arch/mips/kernel/asm-offsets.c b/arch/mips/kernel/asm-offsets.c
index 154e203..1ea973b 100644
--- a/arch/mips/kernel/asm-offsets.c
+++ b/arch/mips/kernel/asm-offsets.c
@@ -14,6 +14,7 @@
#include <linux/mm.h>
#include <linux/kbuild.h>
#include <linux/suspend.h>
+#include <asm/cpu-info.h>
#include <asm/pm.h>
#include <asm/ptrace.h>
#include <asm/processor.h>
@@ -338,6 +339,15 @@
}
#endif
+void output_cpuinfo_defines(void)
+{
+ COMMENT(" MIPS cpuinfo offsets. ");
+ DEFINE(CPUINFO_SIZE, sizeof(struct cpuinfo_mips));
+#ifdef CONFIG_MIPS_ASID_BITS_VARIABLE
+ OFFSET(CPUINFO_ASID_MASK, cpuinfo_mips, asid_mask);
+#endif
+}
+
void output_kvm_defines(void)
{
COMMENT(" KVM/MIPS Specfic offsets. ");
diff --git a/arch/mips/kernel/binfmt_elfn32.c b/arch/mips/kernel/binfmt_elfn32.c
index 1b992c6..58ad63d 100644
--- a/arch/mips/kernel/binfmt_elfn32.c
+++ b/arch/mips/kernel/binfmt_elfn32.c
@@ -30,21 +30,7 @@
/*
* This is used to ensure we don't load something for the wrong architecture.
*/
-#define elf_check_arch(hdr) \
-({ \
- int __res = 1; \
- struct elfhdr *__h = (hdr); \
- \
- if (!mips_elf_check_machine(__h)) \
- __res = 0; \
- if (__h->e_ident[EI_CLASS] != ELFCLASS32) \
- __res = 0; \
- if (((__h->e_flags & EF_MIPS_ABI2) == 0) || \
- ((__h->e_flags & EF_MIPS_ABI) != 0)) \
- __res = 0; \
- \
- __res; \
-})
+#define elf_check_arch elfn32_check_arch
#define TASK32_SIZE 0x7fff8000UL
#undef ELF_ET_DYN_BASE
diff --git a/arch/mips/kernel/binfmt_elfo32.c b/arch/mips/kernel/binfmt_elfo32.c
index abd3aff..49fb881 100644
--- a/arch/mips/kernel/binfmt_elfo32.c
+++ b/arch/mips/kernel/binfmt_elfo32.c
@@ -28,39 +28,9 @@
typedef elf_fpreg_t elf_fpregset_t[ELF_NFPREG];
/*
- * In order to be sure that we don't attempt to execute an O32 binary which
- * requires 64 bit FP (FR=1) on a system which does not support it we refuse
- * to execute any binary which has bits specified by the following macro set
- * in its ELF header flags.
- */
-#ifdef CONFIG_MIPS_O32_FP64_SUPPORT
-# define __MIPS_O32_FP64_MUST_BE_ZERO 0
-#else
-# define __MIPS_O32_FP64_MUST_BE_ZERO EF_MIPS_FP64
-#endif
-
-/*
* This is used to ensure we don't load something for the wrong architecture.
*/
-#define elf_check_arch(hdr) \
-({ \
- int __res = 1; \
- struct elfhdr *__h = (hdr); \
- \
- if (!mips_elf_check_machine(__h)) \
- __res = 0; \
- if (__h->e_ident[EI_CLASS] != ELFCLASS32) \
- __res = 0; \
- if ((__h->e_flags & EF_MIPS_ABI2) != 0) \
- __res = 0; \
- if (((__h->e_flags & EF_MIPS_ABI) != 0) && \
- ((__h->e_flags & EF_MIPS_ABI) != EF_MIPS_ABI_O32)) \
- __res = 0; \
- if (__h->e_flags & __MIPS_O32_FP64_MUST_BE_ZERO) \
- __res = 0; \
- \
- __res; \
-})
+#define elf_check_arch elfo32_check_arch
#ifdef CONFIG_KVM_GUEST
#define TASK32_SIZE 0x3fff8000UL
diff --git a/arch/mips/kernel/bmips_5xxx_init.S b/arch/mips/kernel/bmips_5xxx_init.S
new file mode 100644
index 0000000..adaa82e
--- /dev/null
+++ b/arch/mips/kernel/bmips_5xxx_init.S
@@ -0,0 +1,753 @@
+
+/*
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License. See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * Copyright (C) 2011-2012 by Broadcom Corporation
+ *
+ * Init for bmips 5000.
+ * Used to init second core in dual core 5000's.
+ */
+
+#include <linux/init.h>
+
+#include <asm/asm.h>
+#include <asm/asmmacro.h>
+#include <asm/cacheops.h>
+#include <asm/regdef.h>
+#include <asm/mipsregs.h>
+#include <asm/stackframe.h>
+#include <asm/addrspace.h>
+#include <asm/hazards.h>
+#include <asm/bmips.h>
+
+#ifdef CONFIG_CPU_BMIPS5000
+
+
+#define cacheop(kva, size, linesize, op) \
+ .set noreorder ; \
+ addu t1, kva, size ; \
+ subu t2, linesize, 1 ; \
+ not t2 ; \
+ and t0, kva, t2 ; \
+ addiu t1, t1, -1 ; \
+ and t1, t2 ; \
+9: cache op, 0(t0) ; \
+ bne t0, t1, 9b ; \
+ addu t0, linesize ; \
+ .set reorder ;
+
+
+
+#define IS_SHIFT 22
+#define IL_SHIFT 19
+#define IA_SHIFT 16
+#define DS_SHIFT 13
+#define DL_SHIFT 10
+#define DA_SHIFT 7
+#define IS_MASK 7
+#define IL_MASK 7
+#define IA_MASK 7
+#define DS_MASK 7
+#define DL_MASK 7
+#define DA_MASK 7
+#define ICE_MASK 0x80000000
+#define DCE_MASK 0x40000000
+
+#define CP0_BRCM_CONFIG0 $22, 0
+#define CP0_BRCM_MODE $22, 1
+#define CP0_CONFIG_K0_MASK 7
+
+#define CP0_ICACHE_TAG_LO $28
+#define CP0_ICACHE_DATA_LO $28, 1
+#define CP0_DCACHE_TAG_LO $28, 2
+#define CP0_D_SEC_CACHE_DATA_LO $28, 3
+#define CP0_ICACHE_TAG_HI $29
+#define CP0_ICACHE_DATA_HI $29, 1
+#define CP0_DCACHE_TAG_HI $29, 2
+
+#define CP0_BRCM_MODE_Luc_MASK (1 << 11)
+#define CP0_BRCM_CONFIG0_CWF_MASK (1 << 20)
+#define CP0_BRCM_CONFIG0_TSE_MASK (1 << 19)
+#define CP0_BRCM_MODE_SET_MASK (1 << 7)
+#define CP0_BRCM_MODE_ClkRATIO_MASK (7 << 4)
+#define CP0_BRCM_MODE_BrPRED_MASK (3 << 24)
+#define CP0_BRCM_MODE_BrPRED_SHIFT 24
+#define CP0_BRCM_MODE_BrHIST_MASK (0x1f << 20)
+#define CP0_BRCM_MODE_BrHIST_SHIFT 20
+
+/* ZSC L2 Cache Register Access Register Definitions */
+#define BRCM_ZSC_ALL_REGS_SELECT 0x7 << 24
+
+#define BRCM_ZSC_CONFIG_REG 0 << 3
+#define BRCM_ZSC_REQ_BUFFER_REG 2 << 3
+#define BRCM_ZSC_RBUS_ADDR_MAPPING_REG0 4 << 3
+#define BRCM_ZSC_RBUS_ADDR_MAPPING_REG1 6 << 3
+#define BRCM_ZSC_RBUS_ADDR_MAPPING_REG2 8 << 3
+
+#define BRCM_ZSC_SCB0_ADDR_MAPPING_REG0 0xa << 3
+#define BRCM_ZSC_SCB0_ADDR_MAPPING_REG1 0xc << 3
+
+#define BRCM_ZSC_SCB1_ADDR_MAPPING_REG0 0xe << 3
+#define BRCM_ZSC_SCB1_ADDR_MAPPING_REG1 0x10 << 3
+
+#define BRCM_ZSC_CONFIG_LMB1En 1 << (15)
+#define BRCM_ZSC_CONFIG_LMB0En 1 << (14)
+
+/* branch predition values */
+
+#define BRCM_BrPRED_ALL_TAKEN (0x0)
+#define BRCM_BrPRED_ALL_NOT_TAKEN (0x1)
+#define BRCM_BrPRED_BHT_ENABLE (0x2)
+#define BRCM_BrPRED_PREDICT_BACKWARD (0x3)
+
+
+
+.align 2
+/*
+ * Function: size_i_cache
+ * Arguments: None
+ * Returns: v0 = i cache size, v1 = I cache line size
+ * Description: compute the I-cache size and I-cache line size
+ * Trashes: v0, v1, a0, t0
+ *
+ * pseudo code:
+ *
+ */
+
+LEAF(size_i_cache)
+ .set noreorder
+
+ mfc0 a0, CP0_CONFIG, 1
+ move t0, a0
+
+ /*
+ * Determine sets per way: IS
+ *
+ * This field contains the number of sets (i.e., indices) per way of
+ * the instruction cache:
+ * i) 0x0: 64, ii) 0x1: 128, iii) 0x2: 256, iv) 0x3: 512, v) 0x4: 1k
+ * vi) 0x5 - 0x7: Reserved.
+ */
+
+ srl a0, a0, IS_SHIFT
+ and a0, a0, IS_MASK
+
+ /* sets per way = (64<<IS) */
+
+ li v0, 0x40
+ sllv v0, v0, a0
+
+ /*
+ * Determine line size
+ *
+ * This field contains the line size of the instruction cache:
+ * i) 0x0: No I-cache present, i) 0x3: 16 bytes, ii) 0x4: 32 bytes, iii)
+ * 0x5: 64 bytes, iv) the rest: Reserved.
+ */
+
+ move a0, t0
+
+ srl a0, a0, IL_SHIFT
+ and a0, a0, IL_MASK
+
+ beqz a0, no_i_cache
+ nop
+
+ /* line size = 2 ^ (IL+1) */
+
+ addi a0, a0, 1
+ li v1, 1
+ sll v1, v1, a0
+
+ /* v0 now have sets per way, multiply it by line size now
+ * that will give the set size
+ */
+
+ sll v0, v0, a0
+
+ /*
+ * Determine set associativity
+ *
+ * This field contains the set associativity of the instruction cache.
+ * i) 0x0: Direct mapped, ii) 0x1: 2-way, iii) 0x2: 3-way, iv) 0x3:
+ * 4-way, v) 0x4 - 0x7: Reserved.
+ */
+
+ move a0, t0
+
+ srl a0, a0, IA_SHIFT
+ and a0, a0, IA_MASK
+ addi a0, a0, 0x1
+
+ /* v0 has the set size, multiply it by
+ * set associativiy, to get the cache size
+ */
+
+ multu v0, a0 /*multu is interlocked, so no need to insert nops */
+ mflo v0
+ b 1f
+ nop
+
+no_i_cache:
+ move v0, zero
+ move v1, zero
+1:
+ jr ra
+ nop
+ .set reorder
+
+END(size_i_cache)
+
+/*
+ * Function: size_d_cache
+ * Arguments: None
+ * Returns: v0 = d cache size, v1 = d cache line size
+ * Description: compute the D-cache size and D-cache line size.
+ * Trashes: v0, v1, a0, t0
+ *
+ */
+
+LEAF(size_d_cache)
+ .set noreorder
+
+ mfc0 a0, CP0_CONFIG, 1
+ move t0, a0
+
+ /*
+ * Determine sets per way: IS
+ *
+ * This field contains the number of sets (i.e., indices) per way of
+ * the instruction cache:
+ * i) 0x0: 64, ii) 0x1: 128, iii) 0x2: 256, iv) 0x3: 512, v) 0x4: 1k
+ * vi) 0x5 - 0x7: Reserved.
+ */
+
+ srl a0, a0, DS_SHIFT
+ and a0, a0, DS_MASK
+
+ /* sets per way = (64<<IS) */
+
+ li v0, 0x40
+ sllv v0, v0, a0
+
+ /*
+ * Determine line size
+ *
+ * This field contains the line size of the instruction cache:
+ * i) 0x0: No I-cache present, i) 0x3: 16 bytes, ii) 0x4: 32 bytes, iii)
+ * 0x5: 64 bytes, iv) the rest: Reserved.
+ */
+ move a0, t0
+
+ srl a0, a0, DL_SHIFT
+ and a0, a0, DL_MASK
+
+ beqz a0, no_d_cache
+ nop
+
+ /* line size = 2 ^ (IL+1) */
+
+ addi a0, a0, 1
+ li v1, 1
+ sll v1, v1, a0
+
+ /* v0 now have sets per way, multiply it by line size now
+ * that will give the set size
+ */
+
+ sll v0, v0, a0
+
+ /* determine set associativity
+ *
+ * This field contains the set associativity of the instruction cache.
+ * i) 0x0: Direct mapped, ii) 0x1: 2-way, iii) 0x2: 3-way, iv) 0x3:
+ * 4-way, v) 0x4 - 0x7: Reserved.
+ */
+
+ move a0, t0
+
+ srl a0, a0, DA_SHIFT
+ and a0, a0, DA_MASK
+ addi a0, a0, 0x1
+
+ /* v0 has the set size, multiply it by
+ * set associativiy, to get the cache size
+ */
+
+ multu v0, a0 /*multu is interlocked, so no need to insert nops */
+ mflo v0
+
+ b 1f
+ nop
+
+no_d_cache:
+ move v0, zero
+ move v1, zero
+1:
+ jr ra
+ nop
+ .set reorder
+
+END(size_d_cache)
+
+
+/*
+ * Function: enable_ID
+ * Arguments: None
+ * Returns: None
+ * Description: Enable I and D caches, initialize I and D-caches, also set
+ * hardware delay for d-cache (TP0).
+ * Trashes: t0
+ *
+ */
+ .global enable_ID
+ .ent enable_ID
+ .set noreorder
+enable_ID:
+ mfc0 t0, CP0_BRCM_CONFIG0
+ or t0, t0, (ICE_MASK | DCE_MASK)
+ mtc0 t0, CP0_BRCM_CONFIG0
+ jr ra
+ nop
+
+ .end enable_ID
+ .set reorder
+
+
+/*
+ * Function: l1_init
+ * Arguments: None
+ * Returns: None
+ * Description: Enable I and D caches, and initialize I and D-caches
+ * Trashes: a0, v0, v1, t0, t1, t2, t8
+ *
+ */
+ .globl l1_init
+ .ent l1_init
+ .set noreorder
+l1_init:
+
+ /* save return address */
+ move t8, ra
+
+
+ /* initialize I and D cache Data and Tag registers. */
+ mtc0 zero, CP0_ICACHE_TAG_LO
+ mtc0 zero, CP0_ICACHE_TAG_HI
+ mtc0 zero, CP0_ICACHE_DATA_LO
+ mtc0 zero, CP0_ICACHE_DATA_HI
+ mtc0 zero, CP0_DCACHE_TAG_LO
+ mtc0 zero, CP0_DCACHE_TAG_HI
+
+ /* Enable Caches before Clearing. If the caches are disabled
+ * then the cache operations to clear the cache will be ignored
+ */
+
+ jal enable_ID
+ nop
+
+ jal size_i_cache /* v0 = i-cache size, v1 = i-cache line size */
+ nop
+
+ /* run uncached in kseg 1 */
+ la k0, 1f
+ lui k1, 0x2000
+ or k0, k1, k0
+ jr k0
+ nop
+1:
+
+ /*
+ * set K0 cache mode
+ */
+
+ mfc0 t0, CP0_CONFIG
+ and t0, t0, ~CP0_CONFIG_K0_MASK
+ or t0, t0, 3 /* Write Back mode */
+ mtc0 t0, CP0_CONFIG
+
+ /*
+ * Initialize instruction cache.
+ */
+
+ li a0, KSEG0
+ cacheop(a0, v0, v1, Index_Store_Tag_I)
+
+ /*
+ * Now we can run from I-$, kseg 0
+ */
+ la k0, 1f
+ lui k1, 0x2000
+ or k0, k1, k0
+ xor k0, k1, k0
+ jr k0
+ nop
+1:
+ /*
+ * Initialize data cache.
+ */
+
+ jal size_d_cache /* v0 = d-cache size, v1 = d-cache line size */
+ nop
+
+
+ li a0, KSEG0
+ cacheop(a0, v0, v1, Index_Store_Tag_D)
+
+ jr t8
+ nop
+
+ .end l1_init
+ .set reorder
+
+
+/*
+ * Function: set_other_config
+ * Arguments: none
+ * Returns: None
+ * Description: initialize other remainder configuration to defaults.
+ * Trashes: t0, t1
+ *
+ * pseudo code:
+ *
+ */
+LEAF(set_other_config)
+ .set noreorder
+
+ /* enable Bus error for I-fetch */
+ mfc0 t0, CP0_CACHEERR, 0
+ li t1, 0x4
+ or t0, t1
+ mtc0 t0, CP0_CACHEERR, 0
+
+ /* enable Bus error for Load */
+ mfc0 t0, CP0_CACHEERR, 1
+ li t1, 0x4
+ or t0, t1
+ mtc0 t0, CP0_CACHEERR, 1
+
+ /* enable Bus Error for Store */
+ mfc0 t0, CP0_CACHEERR, 2
+ li t1, 0x4
+ or t0, t1
+ mtc0 t0, CP0_CACHEERR, 2
+
+ jr ra
+ nop
+ .set reorder
+END(set_other_config)
+
+/*
+ * Function: set_branch_pred
+ * Arguments: none
+ * Returns: None
+ * Description:
+ * Trashes: t0, t1
+ *
+ * pseudo code:
+ *
+ */
+
+LEAF(set_branch_pred)
+ .set noreorder
+ mfc0 t0, CP0_BRCM_MODE
+ li t1, ~(CP0_BRCM_MODE_BrPRED_MASK | CP0_BRCM_MODE_BrHIST_MASK )
+ and t0, t0, t1
+
+ /* enable Branch prediction */
+ li t1, BRCM_BrPRED_BHT_ENABLE
+ sll t1, CP0_BRCM_MODE_BrPRED_SHIFT
+ or t0, t0, t1
+
+ /* set history count to 8 */
+ li t1, 8
+ sll t1, CP0_BRCM_MODE_BrHIST_SHIFT
+ or t0, t0, t1
+
+ mtc0 t0, CP0_BRCM_MODE
+ jr ra
+ nop
+ .set reorder
+END(set_branch_pred)
+
+
+/*
+ * Function: set_luc
+ * Arguments: set link uncached.
+ * Returns: None
+ * Description:
+ * Trashes: t0, t1
+ *
+ */
+LEAF(set_luc)
+ .set noreorder
+ mfc0 t0, CP0_BRCM_MODE
+ li t1, ~(CP0_BRCM_MODE_Luc_MASK)
+ and t0, t0, t1
+
+ /* set Luc */
+ ori t0, t0, CP0_BRCM_MODE_Luc_MASK
+
+ mtc0 t0, CP0_BRCM_MODE
+ jr ra
+ nop
+ .set reorder
+END(set_luc)
+
+/*
+ * Function: set_cwf_tse
+ * Arguments: set CWF and TSE bits
+ * Returns: None
+ * Description:
+ * Trashes: t0, t1
+ *
+ */
+LEAF(set_cwf_tse)
+ .set noreorder
+ mfc0 t0, CP0_BRCM_CONFIG0
+ li t1, (CP0_BRCM_CONFIG0_CWF_MASK | CP0_BRCM_CONFIG0_TSE_MASK)
+ or t0, t0, t1
+
+ mtc0 t0, CP0_BRCM_CONFIG0
+ jr ra
+ nop
+ .set reorder
+END(set_cwf_tse)
+
+/*
+ * Function: set_clock_ratio
+ * Arguments: set clock ratio specified by a0
+ * Returns: None
+ * Description:
+ * Trashes: v0, v1, a0, a1
+ *
+ * pseudo code:
+ *
+ */
+LEAF(set_clock_ratio)
+ .set noreorder
+
+ mfc0 t0, CP0_BRCM_MODE
+ li t1, ~(CP0_BRCM_MODE_SET_MASK | CP0_BRCM_MODE_ClkRATIO_MASK)
+ and t0, t0, t1
+ li t1, CP0_BRCM_MODE_SET_MASK
+ or t0, t0, t1
+ or t0, t0, a0
+ mtc0 t0, CP0_BRCM_MODE
+ jr ra
+ nop
+ .set reorder
+END(set_clock_ratio)
+/*
+ * Function: set_zephyr
+ * Arguments: None
+ * Returns: None
+ * Description: Set any zephyr bits
+ * Trashes: t0 & t1
+ *
+ */
+LEAF(set_zephyr)
+ .set noreorder
+
+ /* enable read/write of CP0 #22 sel. 8 */
+ li t0, 0x5a455048
+ .word 0x4088b00f /* mtc0 t0, $22, 15 */
+
+ .word 0x4008b008 /* mfc0 t0, $22, 8 */
+ li t1, 0x09008000 /* turn off pref, jtb */
+ or t0, t0, t1
+ .word 0x4088b008 /* mtc0 t0, $22, 8 */
+ sync
+
+ /* disable read/write of CP0 #22 sel 8 */
+ li t0, 0x0
+ .word 0x4088b00f /* mtc0 t0, $22, 15 */
+
+
+ jr ra
+ nop
+ .set reorder
+
+END(set_zephyr)
+
+
+/*
+ * Function: set_llmb
+ * Arguments: a0=0 disable llmb, a0=1 enables llmb
+ * Returns: None
+ * Description:
+ * Trashes: t0, t1, t2
+ *
+ * pseudo code:
+ *
+ */
+LEAF(set_llmb)
+ .set noreorder
+
+ li t2, 0x90000000 | BRCM_ZSC_ALL_REGS_SELECT | BRCM_ZSC_CONFIG_REG
+ sync
+ cache 0x7, 0x0(t2)
+ sync
+ mfc0 t0, CP0_D_SEC_CACHE_DATA_LO
+ li t1, ~(BRCM_ZSC_CONFIG_LMB1En | BRCM_ZSC_CONFIG_LMB0En)
+ and t0, t0, t1
+
+ beqz a0, svlmb
+ nop
+
+enable_lmb:
+ li t1, (BRCM_ZSC_CONFIG_LMB1En | BRCM_ZSC_CONFIG_LMB0En)
+ or t0, t0, t1
+
+svlmb:
+ mtc0 t0, CP0_D_SEC_CACHE_DATA_LO
+ sync
+ cache 0xb, 0x0(t2)
+ sync
+
+ jr ra
+ nop
+ .set reorder
+
+END(set_llmb)
+/*
+ * Function: core_init
+ * Arguments: none
+ * Returns: None
+ * Description: initialize core related configuration
+ * Trashes: v0,v1,a0,a1,t8
+ *
+ * pseudo code:
+ *
+ */
+ .globl core_init
+ .ent core_init
+ .set noreorder
+core_init:
+ move t8, ra
+
+ /* set Zephyr bits. */
+ bal set_zephyr
+ nop
+
+#if ENABLE_FPU==1
+ /* initialize the Floating point unit (both TPs) */
+ bal init_fpu
+ nop
+#endif
+
+ /* set low latency memory bus */
+ li a0, 1
+ bal set_llmb
+ nop
+
+ /* set branch prediction (TP0 only) */
+ bal set_branch_pred
+ nop
+
+ /* set link uncached */
+ bal set_luc
+ nop
+
+ /* set CWF and TSE */
+ bal set_cwf_tse
+ nop
+
+ /*
+ *set clock ratio by setting 1 to 'set'
+ * and 0 to ClkRatio, (TP0 only)
+ */
+ li a0, 0
+ bal set_clock_ratio
+ nop
+
+ /* set other configuration to defaults */
+ bal set_other_config
+ nop
+
+ move ra, t8
+ jr ra
+ nop
+
+ .set reorder
+ .end core_init
+
+/*
+ * Function: clear_jump_target_buffer
+ * Arguments: None
+ * Returns: None
+ * Description:
+ * Trashes: t0, t1, t2
+ *
+ */
+#define RESET_CALL_RETURN_STACK_THIS_THREAD (0x06<<16)
+#define RESET_JUMP_TARGET_BUFFER_THIS_THREAD (0x04<<16)
+#define JTB_CS_CNTL_MASK (0xFF<<16)
+
+ .globl clear_jump_target_buffer
+ .ent clear_jump_target_buffer
+ .set noreorder
+clear_jump_target_buffer:
+
+ mfc0 t0, $22, 2
+ nop
+ nop
+
+ li t1, ~JTB_CS_CNTL_MASK
+ and t0, t0, t1
+ li t2, RESET_CALL_RETURN_STACK_THIS_THREAD
+ or t0, t0, t2
+ mtc0 t0, $22, 2
+ nop
+ nop
+
+ and t0, t0, t1
+ li t2, RESET_JUMP_TARGET_BUFFER_THIS_THREAD
+ or t0, t0, t2
+ mtc0 t0, $22, 2
+ nop
+ nop
+ jr ra
+ nop
+
+ .end clear_jump_target_buffer
+ .set reorder
+/*
+ * Function: bmips_cache_init
+ * Arguments: None
+ * Returns: None
+ * Description: Enable I and D caches, and initialize I and D-caches
+ * Trashes: v0, v1, t0, t1, t2, t5, t7, t8
+ *
+ */
+ .globl bmips_5xxx_init
+ .ent bmips_5xxx_init
+ .set noreorder
+bmips_5xxx_init:
+
+ /* save return address and A0 */
+ move t7, ra
+ move t5, a0
+
+ jal l1_init
+ nop
+
+ jal core_init
+ nop
+
+ jal clear_jump_target_buffer
+ nop
+
+ mtc0 zero, CP0_CAUSE
+
+ move a0, t5
+ jr t7
+ nop
+
+ .end bmips_5xxx_init
+ .set reorder
+
+
+#endif
diff --git a/arch/mips/kernel/bmips_vec.S b/arch/mips/kernel/bmips_vec.S
index 8649507..921a5fa 100644
--- a/arch/mips/kernel/bmips_vec.S
+++ b/arch/mips/kernel/bmips_vec.S
@@ -88,12 +88,13 @@
li k1, (1 << 19)
mfc0 k0, CP0_STATUS
and k0, k1
- beqz k0, bmips_smp_entry
+ beqz k0, soft_reset
#if defined(CONFIG_CPU_BMIPS5000)
mfc0 k0, CP0_PRID
li k1, PRID_IMP_BMIPS5000
- andi k0, 0xff00
+ /* mask with PRID_IMP_BMIPS5000 to cover both variants */
+ andi k0, PRID_IMP_BMIPS5000
bne k0, k1, 1f
/* if we're not on core 0, this must be the SMP boot signal */
@@ -125,13 +126,48 @@
.set arch=r4000
eret
+#ifdef CONFIG_SMP
+soft_reset:
+
+#if defined(CONFIG_CPU_BMIPS5000)
+ mfc0 k0, CP0_PRID
+ andi k0, 0xff00
+ li k1, PRID_IMP_BMIPS5200
+ bne k0, k1, bmips_smp_entry
+
+ /* if running on TP 1, jump to bmips_smp_entry */
+ mfc0 k0, $22
+ li k1, (1 << 24)
+ and k1, k0
+ bnez k1, bmips_smp_entry
+ nop
+
+ /*
+ * running on TP0, can not be core 0 (the boot core).
+ * Check for soft reset. Indicates a warm boot
+ */
+ mfc0 k0, $12
+ li k1, (1 << 20)
+ and k0, k1
+ beqz k0, bmips_smp_entry
+
+ /*
+ * Warm boot.
+ * Cache init is only done on TP0
+ */
+ la k0, bmips_5xxx_init
+ jalr k0
+ nop
+
+ b bmips_smp_entry
+ nop
+#endif
+
/***********************************************************************
* CPU1 reset vector (used for the initial boot only)
* This is still part of bmips_reset_nmi_vec().
***********************************************************************/
-#ifdef CONFIG_SMP
-
bmips_smp_entry:
/* set up CP0 STATUS; enable FPU */
@@ -166,10 +202,12 @@
2:
#endif /* CONFIG_CPU_BMIPS4350 || CONFIG_CPU_BMIPS4380 */
#if defined(CONFIG_CPU_BMIPS5000)
- /* set exception vector base */
+ /* mask with PRID_IMP_BMIPS5000 to cover both variants */
li k1, PRID_IMP_BMIPS5000
+ andi k0, PRID_IMP_BMIPS5000
bne k0, k1, 3f
+ /* set exception vector base */
la k0, ebase
lw k0, 0(k0)
mtc0 k0, $15, 1
@@ -263,6 +301,8 @@
#endif /* CONFIG_CPU_BMIPS4380 */
#if defined(CONFIG_CPU_BMIPS5000)
li t1, PRID_IMP_BMIPS5000
+ /* mask with PRID_IMP_BMIPS5000 to cover both variants */
+ andi t2, PRID_IMP_BMIPS5000
bne t2, t1, 2f
mfc0 t0, $22, 5
diff --git a/arch/mips/kernel/branch.c b/arch/mips/kernel/branch.c
index d8f9b35..ceca6cc 100644
--- a/arch/mips/kernel/branch.c
+++ b/arch/mips/kernel/branch.c
@@ -688,21 +688,9 @@
}
lose_fpu(1); /* Save FPU state for the emulator. */
reg = insn.i_format.rt;
- bit = 0;
- switch (insn.i_format.rs) {
- case bc1eqz_op:
- /* Test bit 0 */
- if (get_fpr32(¤t->thread.fpu.fpr[reg], 0)
- & 0x1)
- bit = 1;
- break;
- case bc1nez_op:
- /* Test bit 0 */
- if (!(get_fpr32(¤t->thread.fpu.fpr[reg], 0)
- & 0x1))
- bit = 1;
- break;
- }
+ bit = get_fpr32(¤t->thread.fpu.fpr[reg], 0) & 0x1;
+ if (insn.i_format.rs == bc1eqz_op)
+ bit = !bit;
own_fpu(1);
if (bit)
epc = epc + 4 +
diff --git a/arch/mips/kernel/cevt-r4k.c b/arch/mips/kernel/cevt-r4k.c
index 8dfe6a6..e4c21bb 100644
--- a/arch/mips/kernel/cevt-r4k.c
+++ b/arch/mips/kernel/cevt-r4k.c
@@ -28,6 +28,83 @@
return res;
}
+/**
+ * calculate_min_delta() - Calculate a good minimum delta for mips_next_event().
+ *
+ * Running under virtualisation can introduce overhead into mips_next_event() in
+ * the form of hypervisor emulation of CP0_Count/CP0_Compare registers,
+ * potentially with an unnatural frequency, which makes a fixed min_delta_ns
+ * value inappropriate as it may be too small.
+ *
+ * It can also introduce occasional latency from the guest being descheduled.
+ *
+ * This function calculates a good minimum delta based roughly on the 75th
+ * percentile of the time taken to do the mips_next_event() sequence, in order
+ * to handle potentially higher overhead while also eliminating outliers due to
+ * unpredictable hypervisor latency (which can be handled by retries).
+ *
+ * Return: An appropriate minimum delta for the clock event device.
+ */
+static unsigned int calculate_min_delta(void)
+{
+ unsigned int cnt, i, j, k, l;
+ unsigned int buf1[4], buf2[3];
+ unsigned int min_delta;
+
+ /*
+ * Calculate the median of 5 75th percentiles of 5 samples of how long
+ * it takes to set CP0_Compare = CP0_Count + delta.
+ */
+ for (i = 0; i < 5; ++i) {
+ for (j = 0; j < 5; ++j) {
+ /*
+ * This is like the code in mips_next_event(), and
+ * directly measures the borderline "safe" delta.
+ */
+ cnt = read_c0_count();
+ write_c0_compare(cnt);
+ cnt = read_c0_count() - cnt;
+
+ /* Sorted insert into buf1 */
+ for (k = 0; k < j; ++k) {
+ if (cnt < buf1[k]) {
+ l = min_t(unsigned int,
+ j, ARRAY_SIZE(buf1) - 1);
+ for (; l > k; --l)
+ buf1[l] = buf1[l - 1];
+ break;
+ }
+ }
+ if (k < ARRAY_SIZE(buf1))
+ buf1[k] = cnt;
+ }
+
+ /* Sorted insert of 75th percentile into buf2 */
+ for (k = 0; k < i; ++k) {
+ if (buf1[ARRAY_SIZE(buf1) - 1] < buf2[k]) {
+ l = min_t(unsigned int,
+ i, ARRAY_SIZE(buf2) - 1);
+ for (; l > k; --l)
+ buf2[l] = buf2[l - 1];
+ break;
+ }
+ }
+ if (k < ARRAY_SIZE(buf2))
+ buf2[k] = buf1[ARRAY_SIZE(buf1) - 1];
+ }
+
+ /* Use 2 * median of 75th percentiles */
+ min_delta = buf2[ARRAY_SIZE(buf2) - 1] * 2;
+
+ /* Don't go too low */
+ if (min_delta < 0x300)
+ min_delta = 0x300;
+
+ pr_debug("%s: median 75th percentile=%#x, min_delta=%#x\n",
+ __func__, buf2[ARRAY_SIZE(buf2) - 1], min_delta);
+ return min_delta;
+}
+
DEFINE_PER_CPU(struct clock_event_device, mips_clockevent_device);
int cp0_timer_irq_installed;
@@ -177,7 +254,7 @@
{
unsigned int cpu = smp_processor_id();
struct clock_event_device *cd;
- unsigned int irq;
+ unsigned int irq, min_delta;
if (!cpu_has_counter || !mips_hpt_frequency)
return -ENXIO;
@@ -203,7 +280,8 @@
/* Calculate the min / max delta */
cd->max_delta_ns = clockevent_delta2ns(0x7fffffff, cd);
- cd->min_delta_ns = clockevent_delta2ns(0x300, cd);
+ min_delta = calculate_min_delta();
+ cd->min_delta_ns = clockevent_delta2ns(min_delta, cd);
cd->rating = 300;
cd->irq = irq;
diff --git a/arch/mips/kernel/cps-vec.S b/arch/mips/kernel/cps-vec.S
index ac81edd..51b98dc 100644
--- a/arch/mips/kernel/cps-vec.S
+++ b/arch/mips/kernel/cps-vec.S
@@ -18,9 +18,12 @@
#include <asm/mipsmtregs.h>
#include <asm/pm.h>
+#define GCR_CPC_BASE_OFS 0x0088
#define GCR_CL_COHERENCE_OFS 0x2008
#define GCR_CL_ID_OFS 0x2028
+#define CPC_CL_VC_RUN_OFS 0x2028
+
.extern mips_cm_base
.set noreorder
@@ -60,6 +63,37 @@
nop
.endm
+ /*
+ * Set dest to non-zero if the core supports MIPSr6 multithreading
+ * (ie. VPs), else zero. If MIPSr6 multithreading is not supported then
+ * branch to nomt.
+ */
+ .macro has_vp dest, nomt
+ mfc0 \dest, CP0_CONFIG, 1
+ bgez \dest, \nomt
+ mfc0 \dest, CP0_CONFIG, 2
+ bgez \dest, \nomt
+ mfc0 \dest, CP0_CONFIG, 3
+ bgez \dest, \nomt
+ mfc0 \dest, CP0_CONFIG, 4
+ bgez \dest, \nomt
+ mfc0 \dest, CP0_CONFIG, 5
+ andi \dest, \dest, MIPS_CONF5_VP
+ beqz \dest, \nomt
+ nop
+ .endm
+
+ /* Calculate an uncached address for the CM GCRs */
+ .macro cmgcrb dest
+ .set push
+ .set noat
+ MFC0 $1, CP0_CMGCRBASE
+ PTR_SLL $1, $1, 4
+ PTR_LI \dest, UNCAC_BASE
+ PTR_ADDU \dest, \dest, $1
+ .set pop
+ .endm
+
.section .text.cps-vec
.balign 0x1000
@@ -90,120 +124,64 @@
li t0, ST0_CU1 | ST0_CU0 | ST0_BEV | STATUS_BITDEPS
mtc0 t0, CP0_STATUS
- /*
- * Clear the bits used to index the caches. Note that the architecture
- * dictates that writing to any of TagLo or TagHi selects 0 or 2 should
- * be valid for all MIPS32 CPUs, even those for which said writes are
- * unnecessary.
- */
- mtc0 zero, CP0_TAGLO, 0
- mtc0 zero, CP0_TAGHI, 0
- mtc0 zero, CP0_TAGLO, 2
- mtc0 zero, CP0_TAGHI, 2
- ehb
-
- /* Primary cache configuration is indicated by Config1 */
- mfc0 v0, CP0_CONFIG, 1
-
- /* Detect I-cache line size */
- _EXT t0, v0, MIPS_CONF1_IL_SHF, MIPS_CONF1_IL_SZ
- beqz t0, icache_done
- li t1, 2
- sllv t0, t1, t0
-
- /* Detect I-cache size */
- _EXT t1, v0, MIPS_CONF1_IS_SHF, MIPS_CONF1_IS_SZ
- xori t2, t1, 0x7
- beqz t2, 1f
- li t3, 32
- addiu t1, t1, 1
- sllv t1, t3, t1
-1: /* At this point t1 == I-cache sets per way */
- _EXT t2, v0, MIPS_CONF1_IA_SHF, MIPS_CONF1_IA_SZ
- addiu t2, t2, 1
- mul t1, t1, t0
- mul t1, t1, t2
-
- li a0, CKSEG0
- PTR_ADD a1, a0, t1
-1: cache Index_Store_Tag_I, 0(a0)
- PTR_ADD a0, a0, t0
- bne a0, a1, 1b
+ /* Skip cache & coherence setup if we're already coherent */
+ cmgcrb v1
+ lw s7, GCR_CL_COHERENCE_OFS(v1)
+ bnez s7, 1f
nop
-icache_done:
- /* Detect D-cache line size */
- _EXT t0, v0, MIPS_CONF1_DL_SHF, MIPS_CONF1_DL_SZ
- beqz t0, dcache_done
- li t1, 2
- sllv t0, t1, t0
-
- /* Detect D-cache size */
- _EXT t1, v0, MIPS_CONF1_DS_SHF, MIPS_CONF1_DS_SZ
- xori t2, t1, 0x7
- beqz t2, 1f
- li t3, 32
- addiu t1, t1, 1
- sllv t1, t3, t1
-1: /* At this point t1 == D-cache sets per way */
- _EXT t2, v0, MIPS_CONF1_DA_SHF, MIPS_CONF1_DA_SZ
- addiu t2, t2, 1
- mul t1, t1, t0
- mul t1, t1, t2
-
- li a0, CKSEG0
- PTR_ADDU a1, a0, t1
- PTR_SUBU a1, a1, t0
-1: cache Index_Store_Tag_D, 0(a0)
- bne a0, a1, 1b
- PTR_ADD a0, a0, t0
-dcache_done:
-
- /* Set Kseg0 CCA to that in s0 */
- mfc0 t0, CP0_CONFIG
- ori t0, 0x7
- xori t0, 0x7
- or t0, t0, s0
- mtc0 t0, CP0_CONFIG
- ehb
-
- /* Calculate an uncached address for the CM GCRs */
- MFC0 v1, CP0_CMGCRBASE
- PTR_SLL v1, v1, 4
- PTR_LI t0, UNCAC_BASE
- PTR_ADDU v1, v1, t0
+ /* Initialize the L1 caches */
+ jal mips_cps_cache_init
+ nop
/* Enter the coherent domain */
li t0, 0xff
sw t0, GCR_CL_COHERENCE_OFS(v1)
ehb
+ /* Set Kseg0 CCA to that in s0 */
+1: mfc0 t0, CP0_CONFIG
+ ori t0, 0x7
+ xori t0, 0x7
+ or t0, t0, s0
+ mtc0 t0, CP0_CONFIG
+ ehb
+
/* Jump to kseg0 */
PTR_LA t0, 1f
jr t0
nop
/*
- * We're up, cached & coherent. Perform any further required core-level
- * initialisation.
+ * We're up, cached & coherent. Perform any EVA initialization necessary
+ * before we access memory.
*/
-1: jal mips_cps_core_init
+1: eva_init
+
+ /* Retrieve boot configuration pointers */
+ jal mips_cps_get_bootcfg
nop
- /* Do any EVA initialization if necessary */
- eva_init
+ /* Skip core-level init if we started up coherent */
+ bnez s7, 1f
+ nop
+
+ /* Perform any further required core-level initialisation */
+ jal mips_cps_core_init
+ nop
/*
* Boot any other VPEs within this core that should be online, and
* deactivate this VPE if it should be offline.
*/
+ move a1, t9
jal mips_cps_boot_vpes
- nop
+ move a0, v0
/* Off we go! */
- PTR_L t1, VPEBOOTCFG_PC(v0)
- PTR_L gp, VPEBOOTCFG_GP(v0)
- PTR_L sp, VPEBOOTCFG_SP(v0)
+1: PTR_L t1, VPEBOOTCFG_PC(v1)
+ PTR_L gp, VPEBOOTCFG_GP(v1)
+ PTR_L sp, VPEBOOTCFG_SP(v1)
jr t1
nop
END(mips_cps_core_entry)
@@ -245,7 +223,6 @@
.org 0x480
LEAF(excep_ejtag)
- DUMP_EXCEP("EJTAG")
PTR_LA k0, ejtag_debug_handler
jr k0
nop
@@ -323,22 +300,35 @@
nop
END(mips_cps_core_init)
-LEAF(mips_cps_boot_vpes)
- /* Retrieve CM base address */
- PTR_LA t0, mips_cm_base
- PTR_L t0, 0(t0)
-
+/**
+ * mips_cps_get_bootcfg() - retrieve boot configuration pointers
+ *
+ * Returns: pointer to struct core_boot_config in v0, pointer to
+ * struct vpe_boot_config in v1, VPE ID in t9
+ */
+LEAF(mips_cps_get_bootcfg)
/* Calculate a pointer to this cores struct core_boot_config */
+ cmgcrb t0
lw t0, GCR_CL_ID_OFS(t0)
li t1, COREBOOTCFG_SIZE
mul t0, t0, t1
PTR_LA t1, mips_cps_core_bootcfg
PTR_L t1, 0(t1)
- PTR_ADDU t0, t0, t1
+ PTR_ADDU v0, t0, t1
/* Calculate this VPEs ID. If the core doesn't support MT use 0 */
li t9, 0
-#ifdef CONFIG_MIPS_MT_SMP
+#if defined(CONFIG_CPU_MIPSR6)
+ has_vp ta2, 1f
+
+ /*
+ * Assume non-contiguous numbering. Perhaps some day we'll need
+ * to handle contiguous VP numbering, but no such systems yet
+ * exist.
+ */
+ mfc0 t9, $3, 1
+ andi t9, t9, 0xff
+#elif defined(CONFIG_MIPS_MT_SMP)
has_mt ta2, 1f
/* Find the number of VPEs present in the core */
@@ -362,22 +352,43 @@
1: /* Calculate a pointer to this VPEs struct vpe_boot_config */
li t1, VPEBOOTCFG_SIZE
- mul v0, t9, t1
- PTR_L ta3, COREBOOTCFG_VPECONFIG(t0)
- PTR_ADDU v0, v0, ta3
+ mul v1, t9, t1
+ PTR_L ta3, COREBOOTCFG_VPECONFIG(v0)
+ PTR_ADDU v1, v1, ta3
-#ifdef CONFIG_MIPS_MT_SMP
-
- /* If the core doesn't support MT then return */
- bnez ta2, 1f
- nop
jr ra
nop
+ END(mips_cps_get_bootcfg)
+
+LEAF(mips_cps_boot_vpes)
+ PTR_L ta2, COREBOOTCFG_VPEMASK(a0)
+ PTR_L ta3, COREBOOTCFG_VPECONFIG(a0)
+
+#if defined(CONFIG_CPU_MIPSR6)
+
+ has_vp t0, 5f
+
+ /* Find base address of CPC */
+ cmgcrb t3
+ PTR_L t1, GCR_CPC_BASE_OFS(t3)
+ PTR_LI t2, ~0x7fff
+ and t1, t1, t2
+ PTR_LI t2, UNCAC_BASE
+ PTR_ADD t1, t1, t2
+
+ /* Set VC_RUN to the VPE mask */
+ PTR_S ta2, CPC_CL_VC_RUN_OFS(t1)
+ ehb
+
+#elif defined(CONFIG_MIPS_MT)
.set push
.set mt
-1: /* Enter VPE configuration state */
+ /* If the core doesn't support MT then return */
+ has_mt t0, 5f
+
+ /* Enter VPE configuration state */
dvpe
PTR_LA t1, 1f
jr.hb t1
@@ -388,7 +399,6 @@
ehb
/* Loop through each VPE */
- PTR_L ta2, COREBOOTCFG_VPEMASK(t0)
move t8, ta2
li ta1, 0
@@ -465,7 +475,7 @@
/* Check whether this VPE is meant to be running */
li t0, 1
- sll t0, t0, t9
+ sll t0, t0, a1
and t0, t0, t8
bnez t0, 2f
nop
@@ -482,10 +492,84 @@
#endif /* CONFIG_MIPS_MT_SMP */
/* Return */
- jr ra
+5: jr ra
nop
END(mips_cps_boot_vpes)
+LEAF(mips_cps_cache_init)
+ /*
+ * Clear the bits used to index the caches. Note that the architecture
+ * dictates that writing to any of TagLo or TagHi selects 0 or 2 should
+ * be valid for all MIPS32 CPUs, even those for which said writes are
+ * unnecessary.
+ */
+ mtc0 zero, CP0_TAGLO, 0
+ mtc0 zero, CP0_TAGHI, 0
+ mtc0 zero, CP0_TAGLO, 2
+ mtc0 zero, CP0_TAGHI, 2
+ ehb
+
+ /* Primary cache configuration is indicated by Config1 */
+ mfc0 v0, CP0_CONFIG, 1
+
+ /* Detect I-cache line size */
+ _EXT t0, v0, MIPS_CONF1_IL_SHF, MIPS_CONF1_IL_SZ
+ beqz t0, icache_done
+ li t1, 2
+ sllv t0, t1, t0
+
+ /* Detect I-cache size */
+ _EXT t1, v0, MIPS_CONF1_IS_SHF, MIPS_CONF1_IS_SZ
+ xori t2, t1, 0x7
+ beqz t2, 1f
+ li t3, 32
+ addiu t1, t1, 1
+ sllv t1, t3, t1
+1: /* At this point t1 == I-cache sets per way */
+ _EXT t2, v0, MIPS_CONF1_IA_SHF, MIPS_CONF1_IA_SZ
+ addiu t2, t2, 1
+ mul t1, t1, t0
+ mul t1, t1, t2
+
+ li a0, CKSEG0
+ PTR_ADD a1, a0, t1
+1: cache Index_Store_Tag_I, 0(a0)
+ PTR_ADD a0, a0, t0
+ bne a0, a1, 1b
+ nop
+icache_done:
+
+ /* Detect D-cache line size */
+ _EXT t0, v0, MIPS_CONF1_DL_SHF, MIPS_CONF1_DL_SZ
+ beqz t0, dcache_done
+ li t1, 2
+ sllv t0, t1, t0
+
+ /* Detect D-cache size */
+ _EXT t1, v0, MIPS_CONF1_DS_SHF, MIPS_CONF1_DS_SZ
+ xori t2, t1, 0x7
+ beqz t2, 1f
+ li t3, 32
+ addiu t1, t1, 1
+ sllv t1, t3, t1
+1: /* At this point t1 == D-cache sets per way */
+ _EXT t2, v0, MIPS_CONF1_DA_SHF, MIPS_CONF1_DA_SZ
+ addiu t2, t2, 1
+ mul t1, t1, t0
+ mul t1, t1, t2
+
+ li a0, CKSEG0
+ PTR_ADDU a1, a0, t1
+ PTR_SUBU a1, a1, t0
+1: cache Index_Store_Tag_D, 0(a0)
+ bne a0, a1, 1b
+ PTR_ADD a0, a0, t0
+dcache_done:
+
+ jr ra
+ nop
+ END(mips_cps_cache_init)
+
#if defined(CONFIG_MIPS_CPS_PM) && defined(CONFIG_CPU_PM)
/* Calculate a pointer to this CPUs struct mips_static_suspend_state */
diff --git a/arch/mips/kernel/cpu-probe.c b/arch/mips/kernel/cpu-probe.c
index b725b71..5ac5c3e 100644
--- a/arch/mips/kernel/cpu-probe.c
+++ b/arch/mips/kernel/cpu-probe.c
@@ -539,6 +539,7 @@
switch (c->cputype) {
case CPU_PROAPTIV:
case CPU_P5600:
+ case CPU_P6600:
/* proAptiv & related cores use Config6 to enable the FTLB */
config = read_c0_config6();
/* Clear the old probability value */
@@ -561,6 +562,19 @@
write_c0_config7(config | (calculate_ftlb_probability(c)
<< MIPS_CONF7_FTLBP_SHIFT));
break;
+ case CPU_LOONGSON3:
+ /* Flush ITLB, DTLB, VTLB and FTLB */
+ write_c0_diag(LOONGSON_DIAG_ITLB | LOONGSON_DIAG_DTLB |
+ LOONGSON_DIAG_VTLB | LOONGSON_DIAG_FTLB);
+ /* Loongson-3 cores use Config6 to enable the FTLB */
+ config = read_c0_config6();
+ if (enable)
+ /* Enable FTLB */
+ write_c0_config6(config & ~MIPS_CONF6_FTLBDIS);
+ else
+ /* Disable FTLB */
+ write_c0_config6(config | MIPS_CONF6_FTLBDIS);
+ break;
default:
return 1;
}
@@ -634,6 +648,8 @@
if (config1 & MIPS_CONF1_MD)
c->ases |= MIPS_ASE_MDMX;
+ if (config1 & MIPS_CONF1_PC)
+ c->options |= MIPS_CPU_PERF;
if (config1 & MIPS_CONF1_WR)
c->options |= MIPS_CPU_WATCH;
if (config1 & MIPS_CONF1_CA)
@@ -673,18 +689,25 @@
if (config3 & MIPS_CONF3_SM) {
c->ases |= MIPS_ASE_SMARTMIPS;
- c->options |= MIPS_CPU_RIXI;
+ c->options |= MIPS_CPU_RIXI | MIPS_CPU_CTXTC;
}
if (config3 & MIPS_CONF3_RXI)
c->options |= MIPS_CPU_RIXI;
+ if (config3 & MIPS_CONF3_CTXTC)
+ c->options |= MIPS_CPU_CTXTC;
if (config3 & MIPS_CONF3_DSP)
c->ases |= MIPS_ASE_DSP;
- if (config3 & MIPS_CONF3_DSP2P)
+ if (config3 & MIPS_CONF3_DSP2P) {
c->ases |= MIPS_ASE_DSP2P;
+ if (cpu_has_mips_r6)
+ c->ases |= MIPS_ASE_DSP3;
+ }
if (config3 & MIPS_CONF3_VINT)
c->options |= MIPS_CPU_VINT;
if (config3 & MIPS_CONF3_VEIC)
c->options |= MIPS_CPU_VEIC;
+ if (config3 & MIPS_CONF3_LPA)
+ c->options |= MIPS_CPU_LPA;
if (config3 & MIPS_CONF3_MT)
c->ases |= MIPS_ASE_MIPSMT;
if (config3 & MIPS_CONF3_ULRI)
@@ -695,6 +718,10 @@
c->ases |= MIPS_ASE_VZ;
if (config3 & MIPS_CONF3_SC)
c->options |= MIPS_CPU_SEGMENTS;
+ if (config3 & MIPS_CONF3_BI)
+ c->options |= MIPS_CPU_BADINSTR;
+ if (config3 & MIPS_CONF3_BP)
+ c->options |= MIPS_CPU_BADINSTRP;
if (config3 & MIPS_CONF3_MSA)
c->ases |= MIPS_ASE_MSA;
if (config3 & MIPS_CONF3_PW) {
@@ -715,6 +742,7 @@
unsigned int newcf4;
unsigned int mmuextdef;
unsigned int ftlb_page = MIPS_CONF4_FTLBPAGESIZE;
+ unsigned long asid_mask;
config4 = read_c0_config4();
@@ -773,7 +801,20 @@
}
}
- c->kscratch_mask = (config4 >> 16) & 0xff;
+ c->kscratch_mask = (config4 & MIPS_CONF4_KSCREXIST)
+ >> MIPS_CONF4_KSCREXIST_SHIFT;
+
+ asid_mask = MIPS_ENTRYHI_ASID;
+ if (config4 & MIPS_CONF4_AE)
+ asid_mask |= MIPS_ENTRYHI_ASIDX;
+ set_cpu_asid_mask(c, asid_mask);
+
+ /*
+ * Warn if the computed ASID mask doesn't match the mask the kernel
+ * is built for. This may indicate either a serious problem or an
+ * easy optimisation opportunity, but either way should be addressed.
+ */
+ WARN_ON(asid_mask != cpu_asid_mask(c));
return config4 & MIPS_CONF_M;
}
@@ -796,6 +837,8 @@
if (config5 & MIPS_CONF5_MVH)
c->options |= MIPS_CPU_XPA;
#endif
+ if (cpu_has_mips_r6 && (config5 & MIPS_CONF5_VP))
+ c->options |= MIPS_CPU_VP;
return config5 & MIPS_CONF_M;
}
@@ -826,17 +869,43 @@
if (ok)
ok = decode_config5(c);
- mips_probe_watch_registers(c);
+ /* Probe the EBase.WG bit */
+ if (cpu_has_mips_r2_r6) {
+ u64 ebase;
+ unsigned int status;
- if (cpu_has_rixi) {
- /* Enable the RIXI exceptions */
- set_c0_pagegrain(PG_IEC);
- back_to_back_c0_hazard();
- /* Verify the IEC bit is set */
- if (read_c0_pagegrain() & PG_IEC)
- c->options |= MIPS_CPU_RIXIEX;
+ /* {read,write}_c0_ebase_64() may be UNDEFINED prior to r6 */
+ ebase = cpu_has_mips64r6 ? read_c0_ebase_64()
+ : (s32)read_c0_ebase();
+ if (ebase & MIPS_EBASE_WG) {
+ /* WG bit already set, we can avoid the clumsy probe */
+ c->options |= MIPS_CPU_EBASE_WG;
+ } else {
+ /* Its UNDEFINED to change EBase while BEV=0 */
+ status = read_c0_status();
+ write_c0_status(status | ST0_BEV);
+ irq_enable_hazard();
+ /*
+ * On pre-r6 cores, this may well clobber the upper bits
+ * of EBase. This is hard to avoid without potentially
+ * hitting UNDEFINED dm*c0 behaviour if EBase is 32-bit.
+ */
+ if (cpu_has_mips64r6)
+ write_c0_ebase_64(ebase | MIPS_EBASE_WG);
+ else
+ write_c0_ebase(ebase | MIPS_EBASE_WG);
+ back_to_back_c0_hazard();
+ /* Restore BEV */
+ write_c0_status(status);
+ if (read_c0_ebase() & MIPS_EBASE_WG) {
+ c->options |= MIPS_CPU_EBASE_WG;
+ write_c0_ebase(ebase);
+ }
+ }
}
+ mips_probe_watch_registers(c);
+
#ifndef CONFIG_MIPS_CPS
if (cpu_has_mips_r2_r6) {
c->core = get_ebase_cpunum();
@@ -846,6 +915,235 @@
#endif
}
+/*
+ * Probe for certain guest capabilities by writing config bits and reading back.
+ * Finally write back the original value.
+ */
+#define probe_gc0_config(name, maxconf, bits) \
+do { \
+ unsigned int tmp; \
+ tmp = read_gc0_##name(); \
+ write_gc0_##name(tmp | (bits)); \
+ back_to_back_c0_hazard(); \
+ maxconf = read_gc0_##name(); \
+ write_gc0_##name(tmp); \
+} while (0)
+
+/*
+ * Probe for dynamic guest capabilities by changing certain config bits and
+ * reading back to see if they change. Finally write back the original value.
+ */
+#define probe_gc0_config_dyn(name, maxconf, dynconf, bits) \
+do { \
+ maxconf = read_gc0_##name(); \
+ write_gc0_##name(maxconf ^ (bits)); \
+ back_to_back_c0_hazard(); \
+ dynconf = maxconf ^ read_gc0_##name(); \
+ write_gc0_##name(maxconf); \
+ maxconf |= dynconf; \
+} while (0)
+
+static inline unsigned int decode_guest_config0(struct cpuinfo_mips *c)
+{
+ unsigned int config0;
+
+ probe_gc0_config(config, config0, MIPS_CONF_M);
+
+ if (config0 & MIPS_CONF_M)
+ c->guest.conf |= BIT(1);
+ return config0 & MIPS_CONF_M;
+}
+
+static inline unsigned int decode_guest_config1(struct cpuinfo_mips *c)
+{
+ unsigned int config1, config1_dyn;
+
+ probe_gc0_config_dyn(config1, config1, config1_dyn,
+ MIPS_CONF_M | MIPS_CONF1_PC | MIPS_CONF1_WR |
+ MIPS_CONF1_FP);
+
+ if (config1 & MIPS_CONF1_FP)
+ c->guest.options |= MIPS_CPU_FPU;
+ if (config1_dyn & MIPS_CONF1_FP)
+ c->guest.options_dyn |= MIPS_CPU_FPU;
+
+ if (config1 & MIPS_CONF1_WR)
+ c->guest.options |= MIPS_CPU_WATCH;
+ if (config1_dyn & MIPS_CONF1_WR)
+ c->guest.options_dyn |= MIPS_CPU_WATCH;
+
+ if (config1 & MIPS_CONF1_PC)
+ c->guest.options |= MIPS_CPU_PERF;
+ if (config1_dyn & MIPS_CONF1_PC)
+ c->guest.options_dyn |= MIPS_CPU_PERF;
+
+ if (config1 & MIPS_CONF_M)
+ c->guest.conf |= BIT(2);
+ return config1 & MIPS_CONF_M;
+}
+
+static inline unsigned int decode_guest_config2(struct cpuinfo_mips *c)
+{
+ unsigned int config2;
+
+ probe_gc0_config(config2, config2, MIPS_CONF_M);
+
+ if (config2 & MIPS_CONF_M)
+ c->guest.conf |= BIT(3);
+ return config2 & MIPS_CONF_M;
+}
+
+static inline unsigned int decode_guest_config3(struct cpuinfo_mips *c)
+{
+ unsigned int config3, config3_dyn;
+
+ probe_gc0_config_dyn(config3, config3, config3_dyn,
+ MIPS_CONF_M | MIPS_CONF3_MSA | MIPS_CONF3_CTXTC);
+
+ if (config3 & MIPS_CONF3_CTXTC)
+ c->guest.options |= MIPS_CPU_CTXTC;
+ if (config3_dyn & MIPS_CONF3_CTXTC)
+ c->guest.options_dyn |= MIPS_CPU_CTXTC;
+
+ if (config3 & MIPS_CONF3_PW)
+ c->guest.options |= MIPS_CPU_HTW;
+
+ if (config3 & MIPS_CONF3_SC)
+ c->guest.options |= MIPS_CPU_SEGMENTS;
+
+ if (config3 & MIPS_CONF3_BI)
+ c->guest.options |= MIPS_CPU_BADINSTR;
+ if (config3 & MIPS_CONF3_BP)
+ c->guest.options |= MIPS_CPU_BADINSTRP;
+
+ if (config3 & MIPS_CONF3_MSA)
+ c->guest.ases |= MIPS_ASE_MSA;
+ if (config3_dyn & MIPS_CONF3_MSA)
+ c->guest.ases_dyn |= MIPS_ASE_MSA;
+
+ if (config3 & MIPS_CONF_M)
+ c->guest.conf |= BIT(4);
+ return config3 & MIPS_CONF_M;
+}
+
+static inline unsigned int decode_guest_config4(struct cpuinfo_mips *c)
+{
+ unsigned int config4;
+
+ probe_gc0_config(config4, config4,
+ MIPS_CONF_M | MIPS_CONF4_KSCREXIST);
+
+ c->guest.kscratch_mask = (config4 & MIPS_CONF4_KSCREXIST)
+ >> MIPS_CONF4_KSCREXIST_SHIFT;
+
+ if (config4 & MIPS_CONF_M)
+ c->guest.conf |= BIT(5);
+ return config4 & MIPS_CONF_M;
+}
+
+static inline unsigned int decode_guest_config5(struct cpuinfo_mips *c)
+{
+ unsigned int config5, config5_dyn;
+
+ probe_gc0_config_dyn(config5, config5, config5_dyn,
+ MIPS_CONF_M | MIPS_CONF5_MRP);
+
+ if (config5 & MIPS_CONF5_MRP)
+ c->guest.options |= MIPS_CPU_MAAR;
+ if (config5_dyn & MIPS_CONF5_MRP)
+ c->guest.options_dyn |= MIPS_CPU_MAAR;
+
+ if (config5 & MIPS_CONF5_LLB)
+ c->guest.options |= MIPS_CPU_RW_LLB;
+
+ if (config5 & MIPS_CONF_M)
+ c->guest.conf |= BIT(6);
+ return config5 & MIPS_CONF_M;
+}
+
+static inline void decode_guest_configs(struct cpuinfo_mips *c)
+{
+ unsigned int ok;
+
+ ok = decode_guest_config0(c);
+ if (ok)
+ ok = decode_guest_config1(c);
+ if (ok)
+ ok = decode_guest_config2(c);
+ if (ok)
+ ok = decode_guest_config3(c);
+ if (ok)
+ ok = decode_guest_config4(c);
+ if (ok)
+ decode_guest_config5(c);
+}
+
+static inline void cpu_probe_guestctl0(struct cpuinfo_mips *c)
+{
+ unsigned int guestctl0, temp;
+
+ guestctl0 = read_c0_guestctl0();
+
+ if (guestctl0 & MIPS_GCTL0_G0E)
+ c->options |= MIPS_CPU_GUESTCTL0EXT;
+ if (guestctl0 & MIPS_GCTL0_G1)
+ c->options |= MIPS_CPU_GUESTCTL1;
+ if (guestctl0 & MIPS_GCTL0_G2)
+ c->options |= MIPS_CPU_GUESTCTL2;
+ if (!(guestctl0 & MIPS_GCTL0_RAD)) {
+ c->options |= MIPS_CPU_GUESTID;
+
+ /*
+ * Probe for Direct Root to Guest (DRG). Set GuestCtl1.RID = 0
+ * first, otherwise all data accesses will be fully virtualised
+ * as if they were performed by guest mode.
+ */
+ write_c0_guestctl1(0);
+ tlbw_use_hazard();
+
+ write_c0_guestctl0(guestctl0 | MIPS_GCTL0_DRG);
+ back_to_back_c0_hazard();
+ temp = read_c0_guestctl0();
+
+ if (temp & MIPS_GCTL0_DRG) {
+ write_c0_guestctl0(guestctl0);
+ c->options |= MIPS_CPU_DRG;
+ }
+ }
+}
+
+static inline void cpu_probe_guestctl1(struct cpuinfo_mips *c)
+{
+ if (cpu_has_guestid) {
+ /* determine the number of bits of GuestID available */
+ write_c0_guestctl1(MIPS_GCTL1_ID);
+ back_to_back_c0_hazard();
+ c->guestid_mask = (read_c0_guestctl1() & MIPS_GCTL1_ID)
+ >> MIPS_GCTL1_ID_SHIFT;
+ write_c0_guestctl1(0);
+ }
+}
+
+static inline void cpu_probe_gtoffset(struct cpuinfo_mips *c)
+{
+ /* determine the number of bits of GTOffset available */
+ write_c0_gtoffset(0xffffffff);
+ back_to_back_c0_hazard();
+ c->gtoffset_mask = read_c0_gtoffset();
+ write_c0_gtoffset(0);
+}
+
+static inline void cpu_probe_vz(struct cpuinfo_mips *c)
+{
+ cpu_probe_guestctl0(c);
+ if (cpu_has_guestctl1)
+ cpu_probe_guestctl1(c);
+
+ cpu_probe_gtoffset(c);
+
+ decode_guest_configs(c);
+}
+
#define R4K_OPTS (MIPS_CPU_TLB | MIPS_CPU_4KEX | MIPS_CPU_4K_CACHE \
| MIPS_CPU_COUNTER)
@@ -1172,7 +1470,7 @@
set_isa(c, MIPS_CPU_ISA_III);
c->fpu_msk31 |= FPU_CSR_CONDX;
break;
- case PRID_REV_LOONGSON3A:
+ case PRID_REV_LOONGSON3A_R1:
c->cputype = CPU_LOONGSON3;
__cpu_name[cpu] = "ICT Loongson-3";
set_elf_platform(cpu, "loongson3a");
@@ -1314,6 +1612,10 @@
c->cputype = CPU_P5600;
__cpu_name[cpu] = "MIPS P5600";
break;
+ case PRID_IMP_P6600:
+ c->cputype = CPU_P6600;
+ __cpu_name[cpu] = "MIPS P6600";
+ break;
case PRID_IMP_I6400:
c->cputype = CPU_I6400;
__cpu_name[cpu] = "MIPS I6400";
@@ -1322,6 +1624,10 @@
c->cputype = CPU_M5150;
__cpu_name[cpu] = "MIPS M5150";
break;
+ case PRID_IMP_M6250:
+ c->cputype = CPU_M6250;
+ __cpu_name[cpu] = "MIPS M6250";
+ break;
}
decode_configs(c);
@@ -1435,6 +1741,7 @@
c->cputype = CPU_BMIPS4380;
__cpu_name[cpu] = "Broadcom BMIPS4380";
set_elf_platform(cpu, "bmips4380");
+ c->options |= MIPS_CPU_RIXI;
} else {
c->cputype = CPU_BMIPS4350;
__cpu_name[cpu] = "Broadcom BMIPS4350";
@@ -1445,9 +1752,12 @@
case PRID_IMP_BMIPS5000:
case PRID_IMP_BMIPS5200:
c->cputype = CPU_BMIPS5000;
- __cpu_name[cpu] = "Broadcom BMIPS5000";
+ if ((c->processor_id & PRID_IMP_MASK) == PRID_IMP_BMIPS5200)
+ __cpu_name[cpu] = "Broadcom BMIPS5200";
+ else
+ __cpu_name[cpu] = "Broadcom BMIPS5000";
set_elf_platform(cpu, "bmips5000");
- c->options |= MIPS_CPU_ULRI;
+ c->options |= MIPS_CPU_ULRI | MIPS_CPU_RIXI;
break;
}
}
@@ -1481,6 +1791,8 @@
set_elf_platform(cpu, "octeon2");
break;
case PRID_IMP_CAVIUM_CN70XX:
+ case PRID_IMP_CAVIUM_CN73XX:
+ case PRID_IMP_CAVIUM_CNF75XX:
case PRID_IMP_CAVIUM_CN78XX:
c->cputype = CPU_CAVIUM_OCTEON3;
__cpu_name[cpu] = "Cavium Octeon III";
@@ -1493,6 +1805,29 @@
}
}
+static inline void cpu_probe_loongson(struct cpuinfo_mips *c, unsigned int cpu)
+{
+ switch (c->processor_id & PRID_IMP_MASK) {
+ case PRID_IMP_LOONGSON_64: /* Loongson-2/3 */
+ switch (c->processor_id & PRID_REV_MASK) {
+ case PRID_REV_LOONGSON3A_R2:
+ c->cputype = CPU_LOONGSON3;
+ __cpu_name[cpu] = "ICT Loongson-3";
+ set_elf_platform(cpu, "loongson3a");
+ set_isa(c, MIPS_CPU_ISA_M64R2);
+ break;
+ }
+
+ decode_configs(c);
+ c->options |= MIPS_CPU_TLBINV | MIPS_CPU_LDPTE;
+ c->writecombine = _CACHE_UNCACHED_ACCELERATED;
+ break;
+ default:
+ panic("Unknown Loongson Processor ID!");
+ break;
+ }
+}
+
static inline void cpu_probe_ingenic(struct cpuinfo_mips *c, unsigned int cpu)
{
decode_configs(c);
@@ -1640,6 +1975,9 @@
case PRID_COMP_CAVIUM:
cpu_probe_cavium(c, cpu);
break;
+ case PRID_COMP_LOONGSON:
+ cpu_probe_loongson(c, cpu);
+ break;
case PRID_COMP_INGENIC_D0:
case PRID_COMP_INGENIC_D1:
case PRID_COMP_INGENIC_E1:
@@ -1660,6 +1998,15 @@
*/
BUG_ON(current_cpu_type() != c->cputype);
+ if (cpu_has_rixi) {
+ /* Enable the RIXI exceptions */
+ set_c0_pagegrain(PG_IEC);
+ back_to_back_c0_hazard();
+ /* Verify the IEC bit is set */
+ if (read_c0_pagegrain() & PG_IEC)
+ c->options |= MIPS_CPU_RIXIEX;
+ }
+
if (mips_fpu_disabled)
c->options &= ~MIPS_CPU_FPU;
@@ -1699,6 +2046,9 @@
elf_hwcap |= HWCAP_MIPS_MSA;
}
+ if (cpu_has_vz)
+ cpu_probe_vz(c);
+
cpu_probe_vmbits(c);
#ifdef CONFIG_64BIT
diff --git a/arch/mips/kernel/crash.c b/arch/mips/kernel/crash.c
index d434d5d..610f0f3 100644
--- a/arch/mips/kernel/crash.c
+++ b/arch/mips/kernel/crash.c
@@ -14,12 +14,22 @@
static cpumask_t cpus_in_crash = CPU_MASK_NONE;
#ifdef CONFIG_SMP
-static void crash_shutdown_secondary(void *ignore)
+static void crash_shutdown_secondary(void *passed_regs)
{
- struct pt_regs *regs;
+ struct pt_regs *regs = passed_regs;
int cpu = smp_processor_id();
- regs = task_pt_regs(current);
+ /*
+ * If we are passed registers, use those. Otherwise get the
+ * regs from the last interrupt, which should be correct, as
+ * we are in an interrupt. But if the regs are not there,
+ * pull them from the top of the stack. They are probably
+ * wrong, but we need something to keep from crashing again.
+ */
+ if (!regs)
+ regs = get_irq_regs();
+ if (!regs)
+ regs = task_pt_regs(current);
if (!cpu_online(cpu))
return;
diff --git a/arch/mips/kernel/genex.S b/arch/mips/kernel/genex.S
index baa7b6f..17326a9 100644
--- a/arch/mips/kernel/genex.S
+++ b/arch/mips/kernel/genex.S
@@ -130,7 +130,7 @@
/* end of rollback region (the region size must be power of two) */
1:
jr ra
- nop
+ nop
.set pop
END(__r4k_wait)
@@ -172,7 +172,7 @@
mfc0 k0, CP0_EPC
.set noreorder
j k0
- rfe
+ rfe
#else
and k0, ST0_IE
bnez k0, 1f
@@ -189,7 +189,7 @@
LONG_L s0, TI_REGS($28)
LONG_S sp, TI_REGS($28)
PTR_LA ra, ret_from_irq
- PTR_LA v0, plat_irq_dispatch
+ PTR_LA v0, plat_irq_dispatch
jr v0
#ifdef CONFIG_CPU_MICROMIPS
nop
@@ -292,7 +292,7 @@
MFC0 k0, CP0_DESAVE
.set mips32
deret
- .set pop
+ .set pop
END(ejtag_debug_handler)
/*
@@ -329,10 +329,10 @@
* Clear BEV - required for page fault exception handler to work
*/
mfc0 k0, CP0_STATUS
- ori k0, k0, ST0_EXL
+ ori k0, k0, ST0_EXL
li k1, ~(ST0_BEV | ST0_ERL)
- and k0, k0, k1
- mtc0 k0, CP0_STATUS
+ and k0, k0, k1
+ mtc0 k0, CP0_STATUS
_ehb
SAVE_ALL
move a0, sp
@@ -396,7 +396,7 @@
.macro __BUILD_count exception
LONG_L t0,exception_count_\exception
- LONG_ADDIU t0, 1
+ LONG_ADDIU t0, 1
LONG_S t0,exception_count_\exception
.comm exception_count\exception, 8, 8
.endm
@@ -455,10 +455,10 @@
.set noreorder
/* check if TLB contains a entry for EPC */
MFC0 k1, CP0_ENTRYHI
- andi k1, 0xff /* ASID_MASK */
+ andi k1, MIPS_ENTRYHI_ASID | MIPS_ENTRYHI_ASIDX
MFC0 k0, CP0_EPC
- PTR_SRL k0, _PAGE_SHIFT + 1
- PTR_SLL k0, _PAGE_SHIFT + 1
+ PTR_SRL k0, _PAGE_SHIFT + 1
+ PTR_SLL k0, _PAGE_SHIFT + 1
or k1, k0
MTC0 k1, CP0_ENTRYHI
mtc0_tlbw_hazard
@@ -478,27 +478,27 @@
/* microMIPS: 0x007d6b3c: rdhwr v1,$29 */
MFC0 k1, CP0_EPC
#if defined(CONFIG_CPU_MICROMIPS) || defined(CONFIG_CPU_MIPS32_R2) || defined(CONFIG_CPU_MIPS64_R2)
- and k0, k1, 1
- beqz k0, 1f
- xor k1, k0
- lhu k0, (k1)
- lhu k1, 2(k1)
- ins k1, k0, 16, 16
- lui k0, 0x007d
- b docheck
- ori k0, 0x6b3c
+ and k0, k1, 1
+ beqz k0, 1f
+ xor k1, k0
+ lhu k0, (k1)
+ lhu k1, 2(k1)
+ ins k1, k0, 16, 16
+ lui k0, 0x007d
+ b docheck
+ ori k0, 0x6b3c
1:
- lui k0, 0x7c03
- lw k1, (k1)
- ori k0, 0xe83b
+ lui k0, 0x7c03
+ lw k1, (k1)
+ ori k0, 0xe83b
#else
- andi k0, k1, 1
- bnez k0, handle_ri
- lui k0, 0x7c03
- lw k1, (k1)
- ori k0, 0xe83b
+ andi k0, k1, 1
+ bnez k0, handle_ri
+ lui k0, 0x7c03
+ lw k1, (k1)
+ ori k0, 0xe83b
#endif
- .set reorder
+ .set reorder
docheck:
bne k0, k1, handle_ri /* if not ours */
diff --git a/arch/mips/kernel/head.S b/arch/mips/kernel/head.S
index 4e4cc5b..56e8fed 100644
--- a/arch/mips/kernel/head.S
+++ b/arch/mips/kernel/head.S
@@ -21,7 +21,6 @@
#include <asm/asmmacro.h>
#include <asm/irqflags.h>
#include <asm/regdef.h>
-#include <asm/pgtable-bits.h>
#include <asm/mipsregs.h>
#include <asm/stackframe.h>
@@ -132,7 +131,27 @@
set_saved_sp sp, t0, t1
PTR_SUBU sp, 4 * SZREG # init stack pointer
+#ifdef CONFIG_RELOCATABLE
+ /* Copy kernel and apply the relocations */
+ jal relocate_kernel
+
+ /* Repoint the sp into the new kernel image */
+ PTR_LI sp, _THREAD_SIZE - 32 - PT_SIZE
+ PTR_ADDU sp, $28
+ set_saved_sp sp, t0, t1
+ PTR_SUBU sp, 4 * SZREG # init stack pointer
+
+ /*
+ * relocate_kernel returns the entry point either
+ * in the relocated kernel or the original if for
+ * some reason relocation failed - jump there now
+ * with instruction hazard barrier because of the
+ * newly sync'd icache.
+ */
+ jr.hb v0
+#else
j start_kernel
+#endif
END(kernel_entry)
#ifdef CONFIG_SMP
diff --git a/arch/mips/kernel/idle.c b/arch/mips/kernel/idle.c
index 46794d6..60ab4c4 100644
--- a/arch/mips/kernel/idle.c
+++ b/arch/mips/kernel/idle.c
@@ -181,6 +181,11 @@
case CPU_XLP:
cpu_wait = r4k_wait;
break;
+ case CPU_LOONGSON3:
+ if ((c->processor_id & PRID_REV_MASK) >= PRID_REV_LOONGSON3A_R2)
+ cpu_wait = r4k_wait;
+ break;
+
case CPU_BMIPS5000:
cpu_wait = r4k_wait_irqoff;
break;
diff --git a/arch/mips/kernel/mips-r2-to-r6-emul.c b/arch/mips/kernel/mips-r2-to-r6-emul.c
index 3fff89a..625ee77 100644
--- a/arch/mips/kernel/mips-r2-to-r6-emul.c
+++ b/arch/mips/kernel/mips-r2-to-r6-emul.c
@@ -28,6 +28,7 @@
#include <asm/inst.h>
#include <asm/mips-r2-to-r6-emul.h>
#include <asm/local.h>
+#include <asm/mipsregs.h>
#include <asm/ptrace.h>
#include <asm/uaccess.h>
@@ -1251,10 +1252,10 @@
" j 10b\n"
" .previous\n"
" .section __ex_table,\"a\"\n"
- " .word 1b,8b\n"
- " .word 2b,8b\n"
- " .word 3b,8b\n"
- " .word 4b,8b\n"
+ STR(PTR) " 1b,8b\n"
+ STR(PTR) " 2b,8b\n"
+ STR(PTR) " 3b,8b\n"
+ STR(PTR) " 4b,8b\n"
" .previous\n"
" .set pop\n"
: "+&r"(rt), "=&r"(rs),
@@ -1326,10 +1327,10 @@
" j 10b\n"
" .previous\n"
" .section __ex_table,\"a\"\n"
- " .word 1b,8b\n"
- " .word 2b,8b\n"
- " .word 3b,8b\n"
- " .word 4b,8b\n"
+ STR(PTR) " 1b,8b\n"
+ STR(PTR) " 2b,8b\n"
+ STR(PTR) " 3b,8b\n"
+ STR(PTR) " 4b,8b\n"
" .previous\n"
" .set pop\n"
: "+&r"(rt), "=&r"(rs),
@@ -1397,10 +1398,10 @@
" j 9b\n"
" .previous\n"
" .section __ex_table,\"a\"\n"
- " .word 1b,8b\n"
- " .word 2b,8b\n"
- " .word 3b,8b\n"
- " .word 4b,8b\n"
+ STR(PTR) " 1b,8b\n"
+ STR(PTR) " 2b,8b\n"
+ STR(PTR) " 3b,8b\n"
+ STR(PTR) " 4b,8b\n"
" .previous\n"
" .set pop\n"
: "+&r"(rt), "=&r"(rs),
@@ -1467,10 +1468,10 @@
" j 9b\n"
" .previous\n"
" .section __ex_table,\"a\"\n"
- " .word 1b,8b\n"
- " .word 2b,8b\n"
- " .word 3b,8b\n"
- " .word 4b,8b\n"
+ STR(PTR) " 1b,8b\n"
+ STR(PTR) " 2b,8b\n"
+ STR(PTR) " 3b,8b\n"
+ STR(PTR) " 4b,8b\n"
" .previous\n"
" .set pop\n"
: "+&r"(rt), "=&r"(rs),
@@ -1582,14 +1583,14 @@
" j 9b\n"
" .previous\n"
" .section __ex_table,\"a\"\n"
- " .word 1b,8b\n"
- " .word 2b,8b\n"
- " .word 3b,8b\n"
- " .word 4b,8b\n"
- " .word 5b,8b\n"
- " .word 6b,8b\n"
- " .word 7b,8b\n"
- " .word 0b,8b\n"
+ STR(PTR) " 1b,8b\n"
+ STR(PTR) " 2b,8b\n"
+ STR(PTR) " 3b,8b\n"
+ STR(PTR) " 4b,8b\n"
+ STR(PTR) " 5b,8b\n"
+ STR(PTR) " 6b,8b\n"
+ STR(PTR) " 7b,8b\n"
+ STR(PTR) " 0b,8b\n"
" .previous\n"
" .set pop\n"
: "+&r"(rt), "=&r"(rs),
@@ -1701,14 +1702,14 @@
" j 9b\n"
" .previous\n"
" .section __ex_table,\"a\"\n"
- " .word 1b,8b\n"
- " .word 2b,8b\n"
- " .word 3b,8b\n"
- " .word 4b,8b\n"
- " .word 5b,8b\n"
- " .word 6b,8b\n"
- " .word 7b,8b\n"
- " .word 0b,8b\n"
+ STR(PTR) " 1b,8b\n"
+ STR(PTR) " 2b,8b\n"
+ STR(PTR) " 3b,8b\n"
+ STR(PTR) " 4b,8b\n"
+ STR(PTR) " 5b,8b\n"
+ STR(PTR) " 6b,8b\n"
+ STR(PTR) " 7b,8b\n"
+ STR(PTR) " 0b,8b\n"
" .previous\n"
" .set pop\n"
: "+&r"(rt), "=&r"(rs),
@@ -1820,14 +1821,14 @@
" j 9b\n"
" .previous\n"
" .section __ex_table,\"a\"\n"
- " .word 1b,8b\n"
- " .word 2b,8b\n"
- " .word 3b,8b\n"
- " .word 4b,8b\n"
- " .word 5b,8b\n"
- " .word 6b,8b\n"
- " .word 7b,8b\n"
- " .word 0b,8b\n"
+ STR(PTR) " 1b,8b\n"
+ STR(PTR) " 2b,8b\n"
+ STR(PTR) " 3b,8b\n"
+ STR(PTR) " 4b,8b\n"
+ STR(PTR) " 5b,8b\n"
+ STR(PTR) " 6b,8b\n"
+ STR(PTR) " 7b,8b\n"
+ STR(PTR) " 0b,8b\n"
" .previous\n"
" .set pop\n"
: "+&r"(rt), "=&r"(rs),
@@ -1938,14 +1939,14 @@
" j 9b\n"
" .previous\n"
" .section __ex_table,\"a\"\n"
- " .word 1b,8b\n"
- " .word 2b,8b\n"
- " .word 3b,8b\n"
- " .word 4b,8b\n"
- " .word 5b,8b\n"
- " .word 6b,8b\n"
- " .word 7b,8b\n"
- " .word 0b,8b\n"
+ STR(PTR) " 1b,8b\n"
+ STR(PTR) " 2b,8b\n"
+ STR(PTR) " 3b,8b\n"
+ STR(PTR) " 4b,8b\n"
+ STR(PTR) " 5b,8b\n"
+ STR(PTR) " 6b,8b\n"
+ STR(PTR) " 7b,8b\n"
+ STR(PTR) " 0b,8b\n"
" .previous\n"
" .set pop\n"
: "+&r"(rt), "=&r"(rs),
@@ -2000,7 +2001,7 @@
"j 2b\n"
".previous\n"
".section __ex_table,\"a\"\n"
- ".word 1b, 3b\n"
+ STR(PTR) " 1b,3b\n"
".previous\n"
: "=&r"(res), "+&r"(err)
: "r"(vaddr), "i"(SIGSEGV)
@@ -2058,7 +2059,7 @@
"j 2b\n"
".previous\n"
".section __ex_table,\"a\"\n"
- ".word 1b, 3b\n"
+ STR(PTR) " 1b,3b\n"
".previous\n"
: "+&r"(res), "+&r"(err)
: "r"(vaddr), "i"(SIGSEGV));
@@ -2119,7 +2120,7 @@
"j 2b\n"
".previous\n"
".section __ex_table,\"a\"\n"
- ".word 1b, 3b\n"
+ STR(PTR) " 1b,3b\n"
".previous\n"
: "=&r"(res), "+&r"(err)
: "r"(vaddr), "i"(SIGSEGV)
@@ -2182,7 +2183,7 @@
"j 2b\n"
".previous\n"
".section __ex_table,\"a\"\n"
- ".word 1b, 3b\n"
+ STR(PTR) " 1b,3b\n"
".previous\n"
: "+&r"(res), "+&r"(err)
: "r"(vaddr), "i"(SIGSEGV));
diff --git a/arch/mips/kernel/module-rela.c b/arch/mips/kernel/module-rela.c
index 9083d63..7811688 100644
--- a/arch/mips/kernel/module-rela.c
+++ b/arch/mips/kernel/module-rela.c
@@ -16,6 +16,7 @@
* Copyright (C) 2001 Rusty Russell.
* Copyright (C) 2003, 2004 Ralf Baechle (ralf@linux-mips.org)
* Copyright (C) 2005 Thiemo Seufer
+ * Copyright (C) 2015 Imagination Technologies Ltd.
*/
#include <linux/elf.h>
@@ -35,15 +36,13 @@
static int apply_r_mips_26_rela(struct module *me, u32 *location, Elf_Addr v)
{
if (v % 4) {
- pr_err("module %s: dangerous R_MIPS_26 RELArelocation\n",
+ pr_err("module %s: dangerous R_MIPS_26 RELA relocation\n",
me->name);
return -ENOEXEC;
}
if ((v & 0xf0000000) != (((unsigned long)location + 4) & 0xf0000000)) {
- printk(KERN_ERR
- "module %s: relocation overflow\n",
- me->name);
+ pr_err("module %s: relocation overflow\n", me->name);
return -ENOEXEC;
}
@@ -67,6 +66,48 @@
return 0;
}
+static int apply_r_mips_pc_rela(struct module *me, u32 *location, Elf_Addr v,
+ unsigned bits)
+{
+ unsigned long mask = GENMASK(bits - 1, 0);
+ unsigned long se_bits;
+ long offset;
+
+ if (v % 4) {
+ pr_err("module %s: dangerous R_MIPS_PC%u RELA relocation\n",
+ me->name, bits);
+ return -ENOEXEC;
+ }
+
+ offset = ((long)v - (long)location) >> 2;
+
+ /* check the sign bit onwards are identical - ie. we didn't overflow */
+ se_bits = (offset & BIT(bits - 1)) ? ~0ul : 0;
+ if ((offset & ~mask) != (se_bits & ~mask)) {
+ pr_err("module %s: relocation overflow\n", me->name);
+ return -ENOEXEC;
+ }
+
+ *location = (*location & ~mask) | (offset & mask);
+
+ return 0;
+}
+
+static int apply_r_mips_pc16_rela(struct module *me, u32 *location, Elf_Addr v)
+{
+ return apply_r_mips_pc_rela(me, location, v, 16);
+}
+
+static int apply_r_mips_pc21_rela(struct module *me, u32 *location, Elf_Addr v)
+{
+ return apply_r_mips_pc_rela(me, location, v, 21);
+}
+
+static int apply_r_mips_pc26_rela(struct module *me, u32 *location, Elf_Addr v)
+{
+ return apply_r_mips_pc_rela(me, location, v, 26);
+}
+
static int apply_r_mips_64_rela(struct module *me, u32 *location, Elf_Addr v)
{
*(Elf_Addr *)location = v;
@@ -99,9 +140,12 @@
[R_MIPS_26] = apply_r_mips_26_rela,
[R_MIPS_HI16] = apply_r_mips_hi16_rela,
[R_MIPS_LO16] = apply_r_mips_lo16_rela,
+ [R_MIPS_PC16] = apply_r_mips_pc16_rela,
[R_MIPS_64] = apply_r_mips_64_rela,
[R_MIPS_HIGHER] = apply_r_mips_higher_rela,
- [R_MIPS_HIGHEST] = apply_r_mips_highest_rela
+ [R_MIPS_HIGHEST] = apply_r_mips_highest_rela,
+ [R_MIPS_PC21_S2] = apply_r_mips_pc21_rela,
+ [R_MIPS_PC26_S2] = apply_r_mips_pc26_rela,
};
int apply_relocate_add(Elf_Shdr *sechdrs, const char *strtab,
@@ -126,11 +170,11 @@
/* This is the symbol it is referring to */
sym = (Elf_Sym *)sechdrs[symindex].sh_addr
+ ELF_MIPS_R_SYM(rel[i]);
- if (IS_ERR_VALUE(sym->st_value)) {
+ if (sym->st_value >= -MAX_ERRNO) {
/* Ignore unresolved weak symbol */
if (ELF_ST_BIND(sym->st_info) == STB_WEAK)
continue;
- printk(KERN_WARNING "%s: Unknown symbol %s\n",
+ pr_warn("%s: Unknown symbol %s\n",
me->name, strtab + sym->st_name);
return -ENOENT;
}
diff --git a/arch/mips/kernel/module.c b/arch/mips/kernel/module.c
index f9b2936..79850e3 100644
--- a/arch/mips/kernel/module.c
+++ b/arch/mips/kernel/module.c
@@ -73,8 +73,7 @@
}
if ((v & 0xf0000000) != (((unsigned long)location + 4) & 0xf0000000)) {
- printk(KERN_ERR
- "module %s: relocation overflow\n",
+ pr_err("module %s: relocation overflow\n",
me->name);
return -ENOEXEC;
}
@@ -183,13 +182,62 @@
return -ENOEXEC;
}
+static int apply_r_mips_pc_rel(struct module *me, u32 *location, Elf_Addr v,
+ unsigned bits)
+{
+ unsigned long mask = GENMASK(bits - 1, 0);
+ unsigned long se_bits;
+ long offset;
+
+ if (v % 4) {
+ pr_err("module %s: dangerous R_MIPS_PC%u REL relocation\n",
+ me->name, bits);
+ return -ENOEXEC;
+ }
+
+ /* retrieve & sign extend implicit addend */
+ offset = *location & mask;
+ offset |= (offset & BIT(bits - 1)) ? ~mask : 0;
+
+ offset += ((long)v - (long)location) >> 2;
+
+ /* check the sign bit onwards are identical - ie. we didn't overflow */
+ se_bits = (offset & BIT(bits - 1)) ? ~0ul : 0;
+ if ((offset & ~mask) != (se_bits & ~mask)) {
+ pr_err("module %s: relocation overflow\n", me->name);
+ return -ENOEXEC;
+ }
+
+ *location = (*location & ~mask) | (offset & mask);
+
+ return 0;
+}
+
+static int apply_r_mips_pc16_rel(struct module *me, u32 *location, Elf_Addr v)
+{
+ return apply_r_mips_pc_rel(me, location, v, 16);
+}
+
+static int apply_r_mips_pc21_rel(struct module *me, u32 *location, Elf_Addr v)
+{
+ return apply_r_mips_pc_rel(me, location, v, 21);
+}
+
+static int apply_r_mips_pc26_rel(struct module *me, u32 *location, Elf_Addr v)
+{
+ return apply_r_mips_pc_rel(me, location, v, 26);
+}
+
static int (*reloc_handlers_rel[]) (struct module *me, u32 *location,
Elf_Addr v) = {
[R_MIPS_NONE] = apply_r_mips_none,
[R_MIPS_32] = apply_r_mips_32_rel,
[R_MIPS_26] = apply_r_mips_26_rel,
[R_MIPS_HI16] = apply_r_mips_hi16_rel,
- [R_MIPS_LO16] = apply_r_mips_lo16_rel
+ [R_MIPS_LO16] = apply_r_mips_lo16_rel,
+ [R_MIPS_PC16] = apply_r_mips_pc16_rel,
+ [R_MIPS_PC21_S2] = apply_r_mips_pc21_rel,
+ [R_MIPS_PC26_S2] = apply_r_mips_pc26_rel,
};
int apply_relocate(Elf_Shdr *sechdrs, const char *strtab,
@@ -215,12 +263,12 @@
/* This is the symbol it is referring to */
sym = (Elf_Sym *)sechdrs[symindex].sh_addr
+ ELF_MIPS_R_SYM(rel[i]);
- if (IS_ERR_VALUE(sym->st_value)) {
+ if (sym->st_value >= -MAX_ERRNO) {
/* Ignore unresolved weak symbol */
if (ELF_ST_BIND(sym->st_info) == STB_WEAK)
continue;
- printk(KERN_WARNING "%s: Unknown symbol %s\n",
- me->name, strtab + sym->st_name);
+ pr_warn("%s: Unknown symbol %s\n",
+ me->name, strtab + sym->st_name);
return -ENOENT;
}
diff --git a/arch/mips/kernel/perf_event_mipsxx.c b/arch/mips/kernel/perf_event_mipsxx.c
index 9bc1191..d3ba9f4 100644
--- a/arch/mips/kernel/perf_event_mipsxx.c
+++ b/arch/mips/kernel/perf_event_mipsxx.c
@@ -101,8 +101,6 @@
static struct mips_pmu mipspmu;
-#define M_CONFIG1_PC (1 << 4)
-
#define M_PERFCTL_EXL (1 << 0)
#define M_PERFCTL_KERNEL (1 << 1)
#define M_PERFCTL_SUPERVISOR (1 << 2)
@@ -754,7 +752,7 @@
static int __n_counters(void)
{
- if (!(read_c0_config1() & M_CONFIG1_PC))
+ if (!cpu_has_perf)
return 0;
if (!(read_c0_perfctrl0() & M_PERFCTL_MORE))
return 1;
@@ -825,6 +823,16 @@
[PERF_COUNT_HW_BRANCH_MISSES] = { 0x27, CNTR_ODD, T },
};
+static const struct mips_perf_event i6400_event_map[PERF_COUNT_HW_MAX] = {
+ [PERF_COUNT_HW_CPU_CYCLES] = { 0x00, CNTR_EVEN | CNTR_ODD },
+ [PERF_COUNT_HW_INSTRUCTIONS] = { 0x01, CNTR_EVEN | CNTR_ODD },
+ /* These only count dcache, not icache */
+ [PERF_COUNT_HW_CACHE_REFERENCES] = { 0x45, CNTR_EVEN | CNTR_ODD },
+ [PERF_COUNT_HW_CACHE_MISSES] = { 0x48, CNTR_EVEN | CNTR_ODD },
+ [PERF_COUNT_HW_BRANCH_INSTRUCTIONS] = { 0x15, CNTR_EVEN | CNTR_ODD },
+ [PERF_COUNT_HW_BRANCH_MISSES] = { 0x16, CNTR_EVEN | CNTR_ODD },
+};
+
static const struct mips_perf_event loongson3_event_map[PERF_COUNT_HW_MAX] = {
[PERF_COUNT_HW_CPU_CYCLES] = { 0x00, CNTR_EVEN },
[PERF_COUNT_HW_INSTRUCTIONS] = { 0x00, CNTR_ODD },
@@ -1015,6 +1023,46 @@
},
};
+static const struct mips_perf_event i6400_cache_map
+ [PERF_COUNT_HW_CACHE_MAX]
+ [PERF_COUNT_HW_CACHE_OP_MAX]
+ [PERF_COUNT_HW_CACHE_RESULT_MAX] = {
+[C(L1D)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = { 0x46, CNTR_EVEN | CNTR_ODD },
+ [C(RESULT_MISS)] = { 0x49, CNTR_EVEN | CNTR_ODD },
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = { 0x47, CNTR_EVEN | CNTR_ODD },
+ [C(RESULT_MISS)] = { 0x4a, CNTR_EVEN | CNTR_ODD },
+ },
+},
+[C(L1I)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = { 0x84, CNTR_EVEN | CNTR_ODD },
+ [C(RESULT_MISS)] = { 0x85, CNTR_EVEN | CNTR_ODD },
+ },
+},
+[C(DTLB)] = {
+ /* Can't distinguish read & write */
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = { 0x40, CNTR_EVEN | CNTR_ODD },
+ [C(RESULT_MISS)] = { 0x41, CNTR_EVEN | CNTR_ODD },
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = { 0x40, CNTR_EVEN | CNTR_ODD },
+ [C(RESULT_MISS)] = { 0x41, CNTR_EVEN | CNTR_ODD },
+ },
+},
+[C(BPU)] = {
+ /* Conditional branches / mispredicted */
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = { 0x15, CNTR_EVEN | CNTR_ODD },
+ [C(RESULT_MISS)] = { 0x16, CNTR_EVEN | CNTR_ODD },
+ },
+},
+};
+
static const struct mips_perf_event loongson3_cache_map
[PERF_COUNT_HW_CACHE_MAX]
[PERF_COUNT_HW_CACHE_OP_MAX]
@@ -1556,6 +1604,7 @@
#endif
break;
case CPU_P5600:
+ case CPU_P6600:
case CPU_I6400:
/* 8-bit event numbers */
raw_id = config & 0x1ff;
@@ -1718,11 +1767,16 @@
mipspmu.general_event_map = &mipsxxcore_event_map2;
mipspmu.cache_event_map = &mipsxxcore_cache_map2;
break;
- case CPU_I6400:
- mipspmu.name = "mips/I6400";
+ case CPU_P6600:
+ mipspmu.name = "mips/P6600";
mipspmu.general_event_map = &mipsxxcore_event_map2;
mipspmu.cache_event_map = &mipsxxcore_cache_map2;
break;
+ case CPU_I6400:
+ mipspmu.name = "mips/I6400";
+ mipspmu.general_event_map = &i6400_event_map;
+ mipspmu.cache_event_map = &i6400_cache_map;
+ break;
case CPU_1004K:
mipspmu.name = "mips/1004K";
mipspmu.general_event_map = &mipsxxcore_event_map;
diff --git a/arch/mips/kernel/pm-cps.c b/arch/mips/kernel/pm-cps.c
index fa3f9eb..adda3ff 100644
--- a/arch/mips/kernel/pm-cps.c
+++ b/arch/mips/kernel/pm-cps.c
@@ -224,11 +224,18 @@
uasm_build_label(pl, *pp, lbl);
/* Generate the cache ops */
- for (i = 0; i < unroll_lines; i++)
- uasm_i_cache(pp, op, i * cache->linesz, t0);
+ for (i = 0; i < unroll_lines; i++) {
+ if (cpu_has_mips_r6) {
+ uasm_i_cache(pp, op, 0, t0);
+ uasm_i_addiu(pp, t0, t0, cache->linesz);
+ } else {
+ uasm_i_cache(pp, op, i * cache->linesz, t0);
+ }
+ }
- /* Update the base address */
- uasm_i_addiu(pp, t0, t0, unroll_lines * cache->linesz);
+ if (!cpu_has_mips_r6)
+ /* Update the base address */
+ uasm_i_addiu(pp, t0, t0, unroll_lines * cache->linesz);
/* Loop if we haven't reached the end address yet */
uasm_il_bne(pp, pr, t0, t1, lbl);
diff --git a/arch/mips/kernel/pm.c b/arch/mips/kernel/pm.c
index fefdf39..dc81489 100644
--- a/arch/mips/kernel/pm.c
+++ b/arch/mips/kernel/pm.c
@@ -56,7 +56,7 @@
write_c0_userlocal(current_thread_info()->tp_value);
/* Restore watch registers */
- __restore_watch();
+ __restore_watch(current);
}
/**
diff --git a/arch/mips/kernel/proc.c b/arch/mips/kernel/proc.c
index 298b2b7..97dc01b 100644
--- a/arch/mips/kernel/proc.c
+++ b/arch/mips/kernel/proc.c
@@ -114,6 +114,7 @@
if (cpu_has_smartmips) seq_printf(m, "%s", " smartmips");
if (cpu_has_dsp) seq_printf(m, "%s", " dsp");
if (cpu_has_dsp2) seq_printf(m, "%s", " dsp2");
+ if (cpu_has_dsp3) seq_printf(m, "%s", " dsp3");
if (cpu_has_mipsmt) seq_printf(m, "%s", " mt");
if (cpu_has_mmips) seq_printf(m, "%s", " micromips");
if (cpu_has_vz) seq_printf(m, "%s", " vz");
diff --git a/arch/mips/kernel/process.c b/arch/mips/kernel/process.c
index 92880ce..411c971e 100644
--- a/arch/mips/kernel/process.c
+++ b/arch/mips/kernel/process.c
@@ -73,14 +73,6 @@
regs->regs[29] = sp;
}
-void exit_thread(void)
-{
-}
-
-void flush_thread(void)
-{
-}
-
int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
{
/*
@@ -455,7 +447,7 @@
*sp + sizeof(*regs) <= stack_page + THREAD_SIZE - 32) {
regs = (struct pt_regs *)*sp;
pc = regs->cp0_epc;
- if (__kernel_text_address(pc)) {
+ if (!user_mode(regs) && __kernel_text_address(pc)) {
*sp = regs->regs[29];
*ra = regs->regs[31];
return pc;
@@ -580,11 +572,19 @@
return value;
}
+static void prepare_for_fp_mode_switch(void *info)
+{
+ struct mm_struct *mm = info;
+
+ if (current->mm == mm)
+ lose_fpu(1);
+}
+
int mips_set_process_fp_mode(struct task_struct *task, unsigned int value)
{
const unsigned int known_bits = PR_FP_MODE_FR | PR_FP_MODE_FRE;
- unsigned long switch_count;
struct task_struct *t;
+ int max_users;
/* Check the value is valid */
if (value & ~known_bits)
@@ -601,6 +601,9 @@
if (!(value & PR_FP_MODE_FR) && cpu_has_fpu && cpu_has_mips_r6)
return -EOPNOTSUPP;
+ /* Proceed with the mode switch */
+ preempt_disable();
+
/* Save FP & vector context, then disable FPU & MSA */
if (task->signal == current->signal)
lose_fpu(1);
@@ -610,31 +613,17 @@
smp_mb__after_atomic();
/*
- * If there are multiple online CPUs then wait until all threads whose
- * FP mode is about to change have been context switched. This approach
- * allows us to only worry about whether an FP mode switch is in
- * progress when FP is first used in a tasks time slice. Pretty much all
- * of the mode switch overhead can thus be confined to cases where mode
- * switches are actually occurring. That is, to here. However for the
- * thread performing the mode switch it may take a while...
+ * If there are multiple online CPUs then force any which are running
+ * threads in this process to lose their FPU context, which they can't
+ * regain until fp_mode_switching is cleared later.
*/
if (num_online_cpus() > 1) {
- spin_lock_irq(&task->sighand->siglock);
+ /* No need to send an IPI for the local CPU */
+ max_users = (task->mm == current->mm) ? 1 : 0;
- for_each_thread(task, t) {
- if (t == current)
- continue;
-
- switch_count = t->nvcsw + t->nivcsw;
-
- do {
- spin_unlock_irq(&task->sighand->siglock);
- cond_resched();
- spin_lock_irq(&task->sighand->siglock);
- } while ((t->nvcsw + t->nivcsw) == switch_count);
- }
-
- spin_unlock_irq(&task->sighand->siglock);
+ if (atomic_read(¤t->mm->mm_users) > max_users)
+ smp_call_function(prepare_for_fp_mode_switch,
+ (void *)current->mm, 1);
}
/*
@@ -659,6 +648,7 @@
/* Allow threads to use FP again */
atomic_set(&task->mm->context.fp_mode_switching, 0);
+ preempt_enable();
return 0;
}
diff --git a/arch/mips/kernel/ptrace.c b/arch/mips/kernel/ptrace.c
index a5279b2..0dcf691 100644
--- a/arch/mips/kernel/ptrace.c
+++ b/arch/mips/kernel/ptrace.c
@@ -57,8 +57,7 @@
/* Begin with data registers set to all 1s... */
memset(&target->thread.fpu.fpr, ~0, sizeof(target->thread.fpu.fpr));
- /* ...and FCSR zeroed */
- target->thread.fpu.fcr31 = 0;
+ /* FCSR has been preset by `mips_set_personality_nan'. */
/*
* Record that the target has "used" math, such that the context
@@ -80,6 +79,22 @@
}
/*
+ * Poke at FCSR according to its mask. Don't set the cause bits as
+ * this is currently not handled correctly in FP context restoration
+ * and will cause an oops if a corresponding enable bit is set.
+ */
+static void ptrace_setfcr31(struct task_struct *child, u32 value)
+{
+ u32 fcr31;
+ u32 mask;
+
+ value &= ~FPU_CSR_ALL_X;
+ fcr31 = child->thread.fpu.fcr31;
+ mask = boot_cpu_data.fpu_msk31;
+ child->thread.fpu.fcr31 = (value & ~mask) | (fcr31 & mask);
+}
+
+/*
* Read a general register set. We always use the 64-bit format, even
* for 32-bit kernels and for 32-bit processes on a 64-bit kernel.
* Registers are sign extended to fill the available space.
@@ -159,9 +174,7 @@
{
union fpureg *fregs;
u64 fpr_val;
- u32 fcr31;
u32 value;
- u32 mask;
int i;
if (!access_ok(VERIFY_READ, data, 33 * 8))
@@ -176,9 +189,7 @@
}
__get_user(value, data + 64);
- fcr31 = child->thread.fpu.fcr31;
- mask = boot_cpu_data.fpu_msk31;
- child->thread.fpu.fcr31 = (value & ~mask) | (fcr31 & mask);
+ ptrace_setfcr31(child, value);
/* FIR may not be written. */
@@ -210,7 +221,8 @@
for (i = 0; i < boot_cpu_data.watch_reg_use_cnt; i++) {
__put_user(child->thread.watch.mips3264.watchlo[i],
&addr->WATCH_STYLE.watchlo[i]);
- __put_user(child->thread.watch.mips3264.watchhi[i] & 0xfff,
+ __put_user(child->thread.watch.mips3264.watchhi[i] &
+ (MIPS_WATCHHI_MASK | MIPS_WATCHHI_IRW),
&addr->WATCH_STYLE.watchhi[i]);
__put_user(boot_cpu_data.watch_reg_masks[i],
&addr->WATCH_STYLE.watch_masks[i]);
@@ -252,12 +264,12 @@
}
#endif
__get_user(ht[i], &addr->WATCH_STYLE.watchhi[i]);
- if (ht[i] & ~0xff8)
+ if (ht[i] & ~MIPS_WATCHHI_MASK)
return -EINVAL;
}
/* Install them. */
for (i = 0; i < boot_cpu_data.watch_reg_use_cnt; i++) {
- if (lt[i] & 7)
+ if (lt[i] & MIPS_WATCHLO_IRW)
watch_active = 1;
child->thread.watch.mips3264.watchlo[i] = lt[i];
/* Set the G bit. */
@@ -805,7 +817,7 @@
break;
#endif
case FPC_CSR:
- child->thread.fpu.fcr31 = data & ~FPU_CSR_ALL_X;
+ ptrace_setfcr31(child, data);
break;
case DSP_BASE ... DSP_BASE + 5: {
dspreg_t *dregs;
diff --git a/arch/mips/kernel/r4k_fpu.S b/arch/mips/kernel/r4k_fpu.S
index 17732f8..56d86b0 100644
--- a/arch/mips/kernel/r4k_fpu.S
+++ b/arch/mips/kernel/r4k_fpu.S
@@ -244,17 +244,17 @@
.set push
.set noat
#ifdef CONFIG_64BIT
- copy_u_d \wr, 1
+ copy_s_d \wr, 1
EX sd $1, \off(\base)
#elif defined(CONFIG_CPU_LITTLE_ENDIAN)
- copy_u_w \wr, 2
+ copy_s_w \wr, 2
EX sw $1, \off(\base)
- copy_u_w \wr, 3
+ copy_s_w \wr, 3
EX sw $1, (\off+4)(\base)
#else /* CONFIG_CPU_BIG_ENDIAN */
- copy_u_w \wr, 2
+ copy_s_w \wr, 2
EX sw $1, (\off+4)(\base)
- copy_u_w \wr, 3
+ copy_s_w \wr, 3
EX sw $1, \off(\base)
#endif
.set pop
diff --git a/arch/mips/kernel/r4k_switch.S b/arch/mips/kernel/r4k_switch.S
index 92cd051..2f0a3b2 100644
--- a/arch/mips/kernel/r4k_switch.S
+++ b/arch/mips/kernel/r4k_switch.S
@@ -15,7 +15,6 @@
#include <asm/fpregdef.h>
#include <asm/mipsregs.h>
#include <asm/asm-offsets.h>
-#include <asm/pgtable-bits.h>
#include <asm/regdef.h>
#include <asm/stackframe.h>
#include <asm/thread_info.h>
diff --git a/arch/mips/kernel/relocate.c b/arch/mips/kernel/relocate.c
new file mode 100644
index 0000000..ca1cc30
--- /dev/null
+++ b/arch/mips/kernel/relocate.c
@@ -0,0 +1,386 @@
+/*
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License. See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * Support for Kernel relocation at boot time
+ *
+ * Copyright (C) 2015, Imagination Technologies Ltd.
+ * Authors: Matt Redfearn (matt.redfearn@imgtec.com)
+ */
+#include <asm/bootinfo.h>
+#include <asm/cacheflush.h>
+#include <asm/fw/fw.h>
+#include <asm/sections.h>
+#include <asm/setup.h>
+#include <asm/timex.h>
+#include <linux/elf.h>
+#include <linux/kernel.h>
+#include <linux/libfdt.h>
+#include <linux/of_fdt.h>
+#include <linux/sched.h>
+#include <linux/start_kernel.h>
+#include <linux/string.h>
+#include <linux/printk.h>
+
+#define RELOCATED(x) ((void *)((long)x + offset))
+
+extern u32 _relocation_start[]; /* End kernel image / start relocation table */
+extern u32 _relocation_end[]; /* End relocation table */
+
+extern long __start___ex_table; /* Start exception table */
+extern long __stop___ex_table; /* End exception table */
+
+static inline u32 __init get_synci_step(void)
+{
+ u32 res;
+
+ __asm__("rdhwr %0, $1" : "=r" (res));
+
+ return res;
+}
+
+static void __init sync_icache(void *kbase, unsigned long kernel_length)
+{
+ void *kend = kbase + kernel_length;
+ u32 step = get_synci_step();
+
+ do {
+ __asm__ __volatile__(
+ "synci 0(%0)"
+ : /* no output */
+ : "r" (kbase));
+
+ kbase += step;
+ } while (kbase < kend);
+
+ /* Completion barrier */
+ __sync();
+}
+
+static int __init apply_r_mips_64_rel(u32 *loc_orig, u32 *loc_new, long offset)
+{
+ *(u64 *)loc_new += offset;
+
+ return 0;
+}
+
+static int __init apply_r_mips_32_rel(u32 *loc_orig, u32 *loc_new, long offset)
+{
+ *loc_new += offset;
+
+ return 0;
+}
+
+static int __init apply_r_mips_26_rel(u32 *loc_orig, u32 *loc_new, long offset)
+{
+ unsigned long target_addr = (*loc_orig) & 0x03ffffff;
+
+ if (offset % 4) {
+ pr_err("Dangerous R_MIPS_26 REL relocation\n");
+ return -ENOEXEC;
+ }
+
+ /* Original target address */
+ target_addr <<= 2;
+ target_addr += (unsigned long)loc_orig & ~0x03ffffff;
+
+ /* Get the new target address */
+ target_addr += offset;
+
+ if ((target_addr & 0xf0000000) != ((unsigned long)loc_new & 0xf0000000)) {
+ pr_err("R_MIPS_26 REL relocation overflow\n");
+ return -ENOEXEC;
+ }
+
+ target_addr -= (unsigned long)loc_new & ~0x03ffffff;
+ target_addr >>= 2;
+
+ *loc_new = (*loc_new & ~0x03ffffff) | (target_addr & 0x03ffffff);
+
+ return 0;
+}
+
+
+static int __init apply_r_mips_hi16_rel(u32 *loc_orig, u32 *loc_new, long offset)
+{
+ unsigned long insn = *loc_orig;
+ unsigned long target = (insn & 0xffff) << 16; /* high 16bits of target */
+
+ target += offset;
+
+ *loc_new = (insn & ~0xffff) | ((target >> 16) & 0xffff);
+ return 0;
+}
+
+static int (*reloc_handlers_rel[]) (u32 *, u32 *, long) __initdata = {
+ [R_MIPS_64] = apply_r_mips_64_rel,
+ [R_MIPS_32] = apply_r_mips_32_rel,
+ [R_MIPS_26] = apply_r_mips_26_rel,
+ [R_MIPS_HI16] = apply_r_mips_hi16_rel,
+};
+
+int __init do_relocations(void *kbase_old, void *kbase_new, long offset)
+{
+ u32 *r;
+ u32 *loc_orig;
+ u32 *loc_new;
+ int type;
+ int res;
+
+ for (r = _relocation_start; r < _relocation_end; r++) {
+ /* Sentinel for last relocation */
+ if (*r == 0)
+ break;
+
+ type = (*r >> 24) & 0xff;
+ loc_orig = (void *)(kbase_old + ((*r & 0x00ffffff) << 2));
+ loc_new = RELOCATED(loc_orig);
+
+ if (reloc_handlers_rel[type] == NULL) {
+ /* Unsupported relocation */
+ pr_err("Unhandled relocation type %d at 0x%pK\n",
+ type, loc_orig);
+ return -ENOEXEC;
+ }
+
+ res = reloc_handlers_rel[type](loc_orig, loc_new, offset);
+ if (res)
+ return res;
+ }
+
+ return 0;
+}
+
+/*
+ * The exception table is filled in by the relocs tool after vmlinux is linked.
+ * It must be relocated separately since there will not be any relocation
+ * information for it filled in by the linker.
+ */
+static int __init relocate_exception_table(long offset)
+{
+ unsigned long *etable_start, *etable_end, *e;
+
+ etable_start = RELOCATED(&__start___ex_table);
+ etable_end = RELOCATED(&__stop___ex_table);
+
+ for (e = etable_start; e < etable_end; e++)
+ *e += offset;
+
+ return 0;
+}
+
+#ifdef CONFIG_RANDOMIZE_BASE
+
+static inline __init unsigned long rotate_xor(unsigned long hash,
+ const void *area, size_t size)
+{
+ size_t i;
+ unsigned long *ptr = (unsigned long *)area;
+
+ for (i = 0; i < size / sizeof(hash); i++) {
+ /* Rotate by odd number of bits and XOR. */
+ hash = (hash << ((sizeof(hash) * 8) - 7)) | (hash >> 7);
+ hash ^= ptr[i];
+ }
+
+ return hash;
+}
+
+static inline __init unsigned long get_random_boot(void)
+{
+ unsigned long entropy = random_get_entropy();
+ unsigned long hash = 0;
+
+ /* Attempt to create a simple but unpredictable starting entropy. */
+ hash = rotate_xor(hash, linux_banner, strlen(linux_banner));
+
+ /* Add in any runtime entropy we can get */
+ hash = rotate_xor(hash, &entropy, sizeof(entropy));
+
+#if defined(CONFIG_USE_OF)
+ /* Get any additional entropy passed in device tree */
+ {
+ int node, len;
+ u64 *prop;
+
+ node = fdt_path_offset(initial_boot_params, "/chosen");
+ if (node >= 0) {
+ prop = fdt_getprop_w(initial_boot_params, node,
+ "kaslr-seed", &len);
+ if (prop && (len == sizeof(u64)))
+ hash = rotate_xor(hash, prop, sizeof(*prop));
+ }
+ }
+#endif /* CONFIG_USE_OF */
+
+ return hash;
+}
+
+static inline __init bool kaslr_disabled(void)
+{
+ char *str;
+
+#if defined(CONFIG_CMDLINE_BOOL)
+ const char *builtin_cmdline = CONFIG_CMDLINE;
+
+ str = strstr(builtin_cmdline, "nokaslr");
+ if (str == builtin_cmdline ||
+ (str > builtin_cmdline && *(str - 1) == ' '))
+ return true;
+#endif
+ str = strstr(arcs_cmdline, "nokaslr");
+ if (str == arcs_cmdline || (str > arcs_cmdline && *(str - 1) == ' '))
+ return true;
+
+ return false;
+}
+
+static inline void __init *determine_relocation_address(void)
+{
+ /* Choose a new address for the kernel */
+ unsigned long kernel_length;
+ void *dest = &_text;
+ unsigned long offset;
+
+ if (kaslr_disabled())
+ return dest;
+
+ kernel_length = (long)_end - (long)(&_text);
+
+ offset = get_random_boot() << 16;
+ offset &= (CONFIG_RANDOMIZE_BASE_MAX_OFFSET - 1);
+ if (offset < kernel_length)
+ offset += ALIGN(kernel_length, 0xffff);
+
+ return RELOCATED(dest);
+}
+
+#else
+
+static inline void __init *determine_relocation_address(void)
+{
+ /*
+ * Choose a new address for the kernel
+ * For now we'll hard code the destination
+ */
+ return (void *)0xffffffff81000000;
+}
+
+#endif
+
+static inline int __init relocation_addr_valid(void *loc_new)
+{
+ if ((unsigned long)loc_new & 0x0000ffff) {
+ /* Inappropriately aligned new location */
+ return 0;
+ }
+ if ((unsigned long)loc_new < (unsigned long)&_end) {
+ /* New location overlaps original kernel */
+ return 0;
+ }
+ return 1;
+}
+
+void *__init relocate_kernel(void)
+{
+ void *loc_new;
+ unsigned long kernel_length;
+ unsigned long bss_length;
+ long offset = 0;
+ int res = 1;
+ /* Default to original kernel entry point */
+ void *kernel_entry = start_kernel;
+
+ /* Get the command line */
+ fw_init_cmdline();
+#if defined(CONFIG_USE_OF)
+ /* Deal with the device tree */
+ early_init_dt_scan(plat_get_fdt());
+ if (boot_command_line[0]) {
+ /* Boot command line was passed in device tree */
+ strlcpy(arcs_cmdline, boot_command_line, COMMAND_LINE_SIZE);
+ }
+#endif /* CONFIG_USE_OF */
+
+ kernel_length = (long)(&_relocation_start) - (long)(&_text);
+ bss_length = (long)&__bss_stop - (long)&__bss_start;
+
+ loc_new = determine_relocation_address();
+
+ /* Sanity check relocation address */
+ if (relocation_addr_valid(loc_new))
+ offset = (unsigned long)loc_new - (unsigned long)(&_text);
+
+ /* Reset the command line now so we don't end up with a duplicate */
+ arcs_cmdline[0] = '\0';
+
+ if (offset) {
+ /* Copy the kernel to it's new location */
+ memcpy(loc_new, &_text, kernel_length);
+
+ /* Perform relocations on the new kernel */
+ res = do_relocations(&_text, loc_new, offset);
+ if (res < 0)
+ goto out;
+
+ /* Sync the caches ready for execution of new kernel */
+ sync_icache(loc_new, kernel_length);
+
+ res = relocate_exception_table(offset);
+ if (res < 0)
+ goto out;
+
+ /*
+ * The original .bss has already been cleared, and
+ * some variables such as command line parameters
+ * stored to it so make a copy in the new location.
+ */
+ memcpy(RELOCATED(&__bss_start), &__bss_start, bss_length);
+
+ /* The current thread is now within the relocated image */
+ __current_thread_info = RELOCATED(&init_thread_union);
+
+ /* Return the new kernel's entry point */
+ kernel_entry = RELOCATED(start_kernel);
+ }
+out:
+ return kernel_entry;
+}
+
+/*
+ * Show relocation information on panic.
+ */
+void show_kernel_relocation(const char *level)
+{
+ unsigned long offset;
+
+ offset = __pa_symbol(_text) - __pa_symbol(VMLINUX_LOAD_ADDRESS);
+
+ if (IS_ENABLED(CONFIG_RELOCATABLE) && offset > 0) {
+ printk(level);
+ pr_cont("Kernel relocated by 0x%pK\n", (void *)offset);
+ pr_cont(" .text @ 0x%pK\n", _text);
+ pr_cont(" .data @ 0x%pK\n", _sdata);
+ pr_cont(" .bss @ 0x%pK\n", __bss_start);
+ }
+}
+
+static int kernel_location_notifier_fn(struct notifier_block *self,
+ unsigned long v, void *p)
+{
+ show_kernel_relocation(KERN_EMERG);
+ return NOTIFY_DONE;
+}
+
+static struct notifier_block kernel_location_notifier = {
+ .notifier_call = kernel_location_notifier_fn
+};
+
+static int __init register_kernel_offset_dumper(void)
+{
+ atomic_notifier_chain_register(&panic_notifier_list,
+ &kernel_location_notifier);
+ return 0;
+}
+__initcall(register_kernel_offset_dumper);
diff --git a/arch/mips/kernel/scall32-o32.S b/arch/mips/kernel/scall32-o32.S
index d01fe53..c8e43e0 100644
--- a/arch/mips/kernel/scall32-o32.S
+++ b/arch/mips/kernel/scall32-o32.S
@@ -35,7 +35,6 @@
lw t1, PT_EPC(sp) # skip syscall on return
- subu v0, v0, __NR_O32_Linux # check syscall number
addiu t1, 4 # skip to next instruction
sw t1, PT_EPC(sp)
@@ -89,6 +88,7 @@
and t0, t1
bnez t0, syscall_trace_entry # -> yes
syscall_common:
+ subu v0, v0, __NR_O32_Linux # check syscall number
sltiu t0, v0, __NR_O32_Linux_syscalls + 1
beqz t0, illegal_syscall
@@ -118,24 +118,23 @@
syscall_trace_entry:
SAVE_STATIC
- move s0, v0
move a0, sp
/*
* syscall number is in v0 unless we called syscall(__NR_###)
* where the real syscall number is in a0
*/
- addiu a1, v0, __NR_O32_Linux
- bnez v0, 1f /* __NR_syscall at offset 0 */
+ move a1, v0
+ subu t2, v0, __NR_O32_Linux
+ bnez t2, 1f /* __NR_syscall at offset 0 */
lw a1, PT_R4(sp)
1: jal syscall_trace_enter
bltz v0, 1f # seccomp failed? Skip syscall
- move v0, s0 # restore syscall
-
RESTORE_STATIC
+ lw v0, PT_R2(sp) # Restore syscall (maybe modified)
lw a0, PT_R4(sp) # Restore argument registers
lw a1, PT_R5(sp)
lw a2, PT_R6(sp)
diff --git a/arch/mips/kernel/scall64-64.S b/arch/mips/kernel/scall64-64.S
index 6b73ecc..e6ede12 100644
--- a/arch/mips/kernel/scall64-64.S
+++ b/arch/mips/kernel/scall64-64.S
@@ -82,15 +82,14 @@
syscall_trace_entry:
SAVE_STATIC
- move s0, v0
move a0, sp
move a1, v0
jal syscall_trace_enter
bltz v0, 1f # seccomp failed? Skip syscall
- move v0, s0
RESTORE_STATIC
+ ld v0, PT_R2(sp) # Restore syscall (maybe modified)
ld a0, PT_R4(sp) # Restore argument registers
ld a1, PT_R5(sp)
ld a2, PT_R6(sp)
diff --git a/arch/mips/kernel/scall64-n32.S b/arch/mips/kernel/scall64-n32.S
index 71f99d5..9c0b387 100644
--- a/arch/mips/kernel/scall64-n32.S
+++ b/arch/mips/kernel/scall64-n32.S
@@ -42,9 +42,6 @@
#endif
beqz t0, not_n32_scall
- dsll t0, v0, 3 # offset into table
- ld t2, (sysn32_call_table - (__NR_N32_Linux * 8))(t0)
-
sd a3, PT_R26(sp) # save a3 for syscall restarting
li t1, _TIF_WORK_SYSCALL_ENTRY
@@ -53,6 +50,9 @@
bnez t0, n32_syscall_trace_entry
syscall_common:
+ dsll t0, v0, 3 # offset into table
+ ld t2, (sysn32_call_table - (__NR_N32_Linux * 8))(t0)
+
jalr t2 # Do The Real Thing (TM)
li t0, -EMAXERRNO - 1 # error?
@@ -71,21 +71,25 @@
n32_syscall_trace_entry:
SAVE_STATIC
- move s0, t2
move a0, sp
move a1, v0
jal syscall_trace_enter
bltz v0, 1f # seccomp failed? Skip syscall
- move t2, s0
RESTORE_STATIC
+ ld v0, PT_R2(sp) # Restore syscall (maybe modified)
ld a0, PT_R4(sp) # Restore argument registers
ld a1, PT_R5(sp)
ld a2, PT_R6(sp)
ld a3, PT_R7(sp)
ld a4, PT_R8(sp)
ld a5, PT_R9(sp)
+
+ dsubu t2, v0, __NR_N32_Linux # check (new) syscall number
+ sltiu t0, t2, __NR_N32_Linux_syscalls + 1
+ beqz t0, not_n32_scall
+
j syscall_common
1: j syscall_exit
diff --git a/arch/mips/kernel/scall64-o32.S b/arch/mips/kernel/scall64-o32.S
index 91b43ee..f4f28b1 100644
--- a/arch/mips/kernel/scall64-o32.S
+++ b/arch/mips/kernel/scall64-o32.S
@@ -52,9 +52,6 @@
sll a2, a2, 0
sll a3, a3, 0
- dsll t0, v0, 3 # offset into table
- ld t2, (sys32_call_table - (__NR_O32_Linux * 8))(t0)
-
sd a3, PT_R26(sp) # save a3 for syscall restarting
/*
@@ -88,6 +85,9 @@
bnez t0, trace_a_syscall
syscall_common:
+ dsll t0, v0, 3 # offset into table
+ ld t2, (sys32_call_table - (__NR_O32_Linux * 8))(t0)
+
jalr t2 # Do The Real Thing (TM)
li t0, -EMAXERRNO - 1 # error?
@@ -112,7 +112,6 @@
sd a6, PT_R10(sp)
sd a7, PT_R11(sp) # For indirect syscalls
- move s0, t2 # Save syscall pointer
move a0, sp
/*
* absolute syscall number is in v0 unless we called syscall(__NR_###)
@@ -133,8 +132,8 @@
bltz v0, 1f # seccomp failed? Skip syscall
- move t2, s0
RESTORE_STATIC
+ ld v0, PT_R2(sp) # Restore syscall (maybe modified)
ld a0, PT_R4(sp) # Restore argument registers
ld a1, PT_R5(sp)
ld a2, PT_R6(sp)
@@ -143,6 +142,11 @@
ld a5, PT_R9(sp)
ld a6, PT_R10(sp)
ld a7, PT_R11(sp) # For indirect syscalls
+
+ dsubu t0, v0, __NR_O32_Linux # check (new) syscall number
+ sltiu t0, t0, __NR_O32_Linux_syscalls + 1
+ beqz t0, not_o32_scall
+
j syscall_common
1: j syscall_exit
diff --git a/arch/mips/kernel/setup.c b/arch/mips/kernel/setup.c
index 4f60734..ef408a0 100644
--- a/arch/mips/kernel/setup.c
+++ b/arch/mips/kernel/setup.c
@@ -26,6 +26,7 @@
#include <linux/sizes.h>
#include <linux/device.h>
#include <linux/dma-contiguous.h>
+#include <linux/decompress/generic.h>
#include <asm/addrspace.h>
#include <asm/bootinfo.h>
@@ -52,13 +53,6 @@
#endif
/*
- * Despite it's name this variable is even if we don't have PCI
- */
-unsigned int PCI_DMA_BUS_IS_PHYS;
-
-EXPORT_SYMBOL(PCI_DMA_BUS_IS_PHYS);
-
-/*
* Setup information
*
* These are initialized so they are in the .data section
@@ -250,6 +244,35 @@
return 0;
}
+/* In some conditions (e.g. big endian bootloader with a little endian
+ kernel), the initrd might appear byte swapped. Try to detect this and
+ byte swap it if needed. */
+static void __init maybe_bswap_initrd(void)
+{
+#if defined(CONFIG_CPU_CAVIUM_OCTEON)
+ u64 buf;
+
+ /* Check for CPIO signature */
+ if (!memcmp((void *)initrd_start, "070701", 6))
+ return;
+
+ /* Check for compressed initrd */
+ if (decompress_method((unsigned char *)initrd_start, 8, NULL))
+ return;
+
+ /* Try again with a byte swapped header */
+ buf = swab64p((u64 *)initrd_start);
+ if (!memcmp(&buf, "070701", 6) ||
+ decompress_method((unsigned char *)(&buf), 8, NULL)) {
+ unsigned long i;
+
+ pr_info("Byteswapped initrd detected\n");
+ for (i = initrd_start; i < ALIGN(initrd_end, 8); i += 8)
+ swab64s((u64 *)i);
+ }
+#endif
+}
+
static void __init finalize_initrd(void)
{
unsigned long size = initrd_end - initrd_start;
@@ -263,6 +286,8 @@
goto disable;
}
+ maybe_bswap_initrd();
+
reserve_bootmem(__pa(initrd_start), size, BOOTMEM_DEFAULT);
initrd_below_start_ok = 1;
@@ -469,6 +494,29 @@
*/
reserve_bootmem(PFN_PHYS(mapstart), bootmap_size, BOOTMEM_DEFAULT);
+#ifdef CONFIG_RELOCATABLE
+ /*
+ * The kernel reserves all memory below its _end symbol as bootmem,
+ * but the kernel may now be at a much higher address. The memory
+ * between the original and new locations may be returned to the system.
+ */
+ if (__pa_symbol(_text) > __pa_symbol(VMLINUX_LOAD_ADDRESS)) {
+ unsigned long offset;
+ extern void show_kernel_relocation(const char *level);
+
+ offset = __pa_symbol(_text) - __pa_symbol(VMLINUX_LOAD_ADDRESS);
+ free_bootmem(__pa_symbol(VMLINUX_LOAD_ADDRESS), offset);
+
+#if defined(CONFIG_DEBUG_KERNEL) && defined(CONFIG_DEBUG_INFO)
+ /*
+ * This information is necessary when debugging the kernel
+ * But is a security vulnerability otherwise!
+ */
+ show_kernel_relocation(KERN_INFO);
+#endif
+ }
+#endif
+
/*
* Reserve initrd memory if needed.
*/
@@ -624,6 +672,8 @@
#define USE_PROM_CMDLINE IS_ENABLED(CONFIG_MIPS_CMDLINE_FROM_BOOTLOADER)
#define USE_DTB_CMDLINE IS_ENABLED(CONFIG_MIPS_CMDLINE_FROM_DTB)
#define EXTEND_WITH_PROM IS_ENABLED(CONFIG_MIPS_CMDLINE_DTB_EXTEND)
+#define BUILTIN_EXTEND_WITH_PROM \
+ IS_ENABLED(CONFIG_MIPS_CMDLINE_BUILTIN_EXTEND)
static void __init arch_mem_init(char **cmdline_p)
{
@@ -657,15 +707,23 @@
strlcpy(boot_command_line, arcs_cmdline, COMMAND_LINE_SIZE);
if (EXTEND_WITH_PROM && arcs_cmdline[0]) {
- strlcat(boot_command_line, " ", COMMAND_LINE_SIZE);
+ if (boot_command_line[0])
+ strlcat(boot_command_line, " ", COMMAND_LINE_SIZE);
strlcat(boot_command_line, arcs_cmdline, COMMAND_LINE_SIZE);
}
#if defined(CONFIG_CMDLINE_BOOL)
if (builtin_cmdline[0]) {
- strlcat(boot_command_line, " ", COMMAND_LINE_SIZE);
+ if (boot_command_line[0])
+ strlcat(boot_command_line, " ", COMMAND_LINE_SIZE);
strlcat(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
}
+
+ if (BUILTIN_EXTEND_WITH_PROM && arcs_cmdline[0]) {
+ if (boot_command_line[0])
+ strlcat(boot_command_line, " ", COMMAND_LINE_SIZE);
+ strlcat(boot_command_line, arcs_cmdline, COMMAND_LINE_SIZE);
+ }
#endif
#endif
strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
@@ -706,6 +764,9 @@
for_each_memblock(reserved, reg)
if (reg->size != 0)
reserve_bootmem(reg->base, reg->size, BOOTMEM_DEFAULT);
+
+ reserve_bootmem_region(__pa_symbol(&__nosave_begin),
+ __pa_symbol(&__nosave_end)); /* Reserve for hibernation */
}
static void __init resource_init(void)
diff --git a/arch/mips/kernel/signal.c b/arch/mips/kernel/signal.c
index bf792e2..ab04229 100644
--- a/arch/mips/kernel/signal.c
+++ b/arch/mips/kernel/signal.c
@@ -195,6 +195,9 @@
unsigned int csr;
int i, err;
+ if (!config_enabled(CONFIG_CPU_HAS_MSA))
+ return SIGSYS;
+
if (size != sizeof(*msa))
return -EINVAL;
@@ -398,8 +401,8 @@
}
fp_done:
- if (used & USED_EXTCONTEXT)
- err |= restore_extcontext(sc_to_extcontext(sc));
+ if (!err && (used & USED_EXTCONTEXT))
+ err = restore_extcontext(sc_to_extcontext(sc));
return err ?: sig;
}
@@ -798,7 +801,7 @@
regs->regs[0] = 0; /* Don't deal with this again. */
}
- if (sig_uses_siginfo(&ksig->ka))
+ if (sig_uses_siginfo(&ksig->ka, abi))
ret = abi->setup_rt_frame(vdso + abi->vdso->off_rt_sigreturn,
ksig, regs, oldset);
else
diff --git a/arch/mips/kernel/signal32.c b/arch/mips/kernel/signal32.c
index 4909639..78c8349 100644
--- a/arch/mips/kernel/signal32.c
+++ b/arch/mips/kernel/signal32.c
@@ -227,6 +227,12 @@
err |= __put_user(from->si_uid, &to->si_uid);
err |= __put_user(from->si_int, &to->si_int);
break;
+ case __SI_SYS >> 16:
+ err |= __copy_to_user(&to->si_call_addr, &from->si_call_addr,
+ sizeof(compat_uptr_t));
+ err |= __put_user(from->si_syscall, &to->si_syscall);
+ err |= __put_user(from->si_arch, &to->si_arch);
+ break;
}
}
return err;
diff --git a/arch/mips/kernel/smp-bmips.c b/arch/mips/kernel/smp-bmips.c
index 78cf8c2..e02addc 100644
--- a/arch/mips/kernel/smp-bmips.c
+++ b/arch/mips/kernel/smp-bmips.c
@@ -243,6 +243,7 @@
break;
case CPU_BMIPS5000:
write_c0_brcm_action(ACTION_CLR_IPI(smp_processor_id(), 0));
+ current_cpu_data.core = (read_c0_brcm_config() >> 25) & 3;
break;
}
}
@@ -565,3 +566,90 @@
* once the wired entries are present.
*/
}
+
+void __init bmips_cpu_setup(void)
+{
+ void __iomem __maybe_unused *cbr = BMIPS_GET_CBR();
+ u32 __maybe_unused cfg;
+
+ switch (current_cpu_type()) {
+ case CPU_BMIPS3300:
+ /* Set BIU to async mode */
+ set_c0_brcm_bus_pll(BIT(22));
+ __sync();
+
+ /* put the BIU back in sync mode */
+ clear_c0_brcm_bus_pll(BIT(22));
+
+ /* clear BHTD to enable branch history table */
+ clear_c0_brcm_reset(BIT(16));
+
+ /* Flush and enable RAC */
+ cfg = __raw_readl(cbr + BMIPS_RAC_CONFIG);
+ __raw_writel(cfg | 0x100, BMIPS_RAC_CONFIG);
+ __raw_readl(cbr + BMIPS_RAC_CONFIG);
+
+ cfg = __raw_readl(cbr + BMIPS_RAC_CONFIG);
+ __raw_writel(cfg | 0xf, BMIPS_RAC_CONFIG);
+ __raw_readl(cbr + BMIPS_RAC_CONFIG);
+
+ cfg = __raw_readl(cbr + BMIPS_RAC_ADDRESS_RANGE);
+ __raw_writel(cfg | 0x0fff0000, cbr + BMIPS_RAC_ADDRESS_RANGE);
+ __raw_readl(cbr + BMIPS_RAC_ADDRESS_RANGE);
+ break;
+
+ case CPU_BMIPS4380:
+ /* CBG workaround for early BMIPS4380 CPUs */
+ switch (read_c0_prid()) {
+ case 0x2a040:
+ case 0x2a042:
+ case 0x2a044:
+ case 0x2a060:
+ cfg = __raw_readl(cbr + BMIPS_L2_CONFIG);
+ __raw_writel(cfg & ~0x07000000, cbr + BMIPS_L2_CONFIG);
+ __raw_readl(cbr + BMIPS_L2_CONFIG);
+ }
+
+ /* clear BHTD to enable branch history table */
+ clear_c0_brcm_config_0(BIT(21));
+
+ /* XI/ROTR enable */
+ set_c0_brcm_config_0(BIT(23));
+ set_c0_brcm_cmt_ctrl(BIT(15));
+ break;
+
+ case CPU_BMIPS5000:
+ /* enable RDHWR, BRDHWR */
+ set_c0_brcm_config(BIT(17) | BIT(21));
+
+ /* Disable JTB */
+ __asm__ __volatile__(
+ " .set noreorder\n"
+ " li $8, 0x5a455048\n"
+ " .word 0x4088b00f\n" /* mtc0 t0, $22, 15 */
+ " .word 0x4008b008\n" /* mfc0 t0, $22, 8 */
+ " li $9, 0x00008000\n"
+ " or $8, $8, $9\n"
+ " .word 0x4088b008\n" /* mtc0 t0, $22, 8 */
+ " sync\n"
+ " li $8, 0x0\n"
+ " .word 0x4088b00f\n" /* mtc0 t0, $22, 15 */
+ " .set reorder\n"
+ : : : "$8", "$9");
+
+ /* XI enable */
+ set_c0_brcm_config(BIT(27));
+
+ /* enable MIPS32R2 ROR instruction for XI TLB handlers */
+ __asm__ __volatile__(
+ " li $8, 0x5a455048\n"
+ " .word 0x4088b00f\n" /* mtc0 $8, $22, 15 */
+ " nop; nop; nop\n"
+ " .word 0x4008b008\n" /* mfc0 $8, $22, 8 */
+ " lui $9, 0x0100\n"
+ " or $8, $9\n"
+ " .word 0x4088b008\n" /* mtc0 $8, $22, 8 */
+ : : : "$8", "$9");
+ break;
+ }
+}
diff --git a/arch/mips/kernel/smp-cps.c b/arch/mips/kernel/smp-cps.c
index 253e140..1061bd2 100644
--- a/arch/mips/kernel/smp-cps.c
+++ b/arch/mips/kernel/smp-cps.c
@@ -27,15 +27,27 @@
#include <asm/time.h>
#include <asm/uasm.h>
+static bool threads_disabled;
static DECLARE_BITMAP(core_power, NR_CPUS);
struct core_boot_config *mips_cps_core_bootcfg;
+static int __init setup_nothreads(char *s)
+{
+ threads_disabled = true;
+ return 0;
+}
+early_param("nothreads", setup_nothreads);
+
static unsigned core_vpe_count(unsigned core)
{
unsigned cfg;
- if (!config_enabled(CONFIG_MIPS_MT_SMP) || !cpu_has_mipsmt)
+ if (threads_disabled)
+ return 1;
+
+ if ((!config_enabled(CONFIG_MIPS_MT_SMP) || !cpu_has_mipsmt)
+ && (!config_enabled(CONFIG_CPU_MIPSR6) || !cpu_has_vp))
return 1;
mips_cm_lock_other(core, 0);
@@ -47,11 +59,12 @@
static void __init cps_smp_setup(void)
{
unsigned int ncores, nvpes, core_vpes;
+ unsigned long core_entry;
int c, v;
/* Detect & record VPE topology */
ncores = mips_cm_numcores();
- pr_info("VPE topology ");
+ pr_info("%s topology ", cpu_has_mips_r6 ? "VP" : "VPE");
for (c = nvpes = 0; c < ncores; c++) {
core_vpes = core_vpe_count(c);
pr_cont("%c%u", c ? ',' : '{', core_vpes);
@@ -62,7 +75,7 @@
for (v = 0; v < min_t(int, core_vpes, NR_CPUS - nvpes); v++) {
cpu_data[nvpes + v].core = c;
-#ifdef CONFIG_MIPS_MT_SMP
+#if defined(CONFIG_MIPS_MT_SMP) || defined(CONFIG_CPU_MIPSR6)
cpu_data[nvpes + v].vpe_id = v;
#endif
}
@@ -91,6 +104,11 @@
/* Make core 0 coherent with everything */
write_gcr_cl_coherence(0xff);
+ if (mips_cm_revision() >= CM_REV_CM3) {
+ core_entry = CKSEG1ADDR((unsigned long)mips_cps_core_entry);
+ write_gcr_bev_base(core_entry);
+ }
+
#ifdef CONFIG_MIPS_MT_FPAFF
/* If we have an FPU, enroll ourselves in the FPU-full mask */
if (cpu_has_fpu)
@@ -213,6 +231,18 @@
if (mips_cpc_present()) {
/* Reset the core */
mips_cpc_lock_other(core);
+
+ if (mips_cm_revision() >= CM_REV_CM3) {
+ /* Run VP0 following the reset */
+ write_cpc_co_vp_run(0x1);
+
+ /*
+ * Ensure that the VP_RUN register is written before the
+ * core leaves reset.
+ */
+ wmb();
+ }
+
write_cpc_co_cmd(CPC_Cx_CMD_RESET);
timeout = 100;
@@ -250,7 +280,10 @@
static void remote_vpe_boot(void *dummy)
{
- mips_cps_boot_vpes();
+ unsigned core = current_cpu_data.core;
+ struct core_boot_config *core_cfg = &mips_cps_core_bootcfg[core];
+
+ mips_cps_boot_vpes(core_cfg, cpu_vpe_id(¤t_cpu_data));
}
static void cps_boot_secondary(int cpu, struct task_struct *idle)
@@ -259,6 +292,7 @@
unsigned vpe_id = cpu_vpe_id(&cpu_data[cpu]);
struct core_boot_config *core_cfg = &mips_cps_core_bootcfg[core];
struct vpe_boot_config *vpe_cfg = &core_cfg->vpe_config[vpe_id];
+ unsigned long core_entry;
unsigned int remote;
int err;
@@ -276,6 +310,13 @@
goto out;
}
+ if (cpu_has_vp) {
+ mips_cm_lock_other(core, vpe_id);
+ core_entry = CKSEG1ADDR((unsigned long)mips_cps_core_entry);
+ write_gcr_co_reset_base(core_entry);
+ mips_cm_unlock_other();
+ }
+
if (core != current_cpu_data.core) {
/* Boot a VPE on another powered up core */
for (remote = 0; remote < NR_CPUS; remote++) {
@@ -293,10 +334,10 @@
goto out;
}
- BUG_ON(!cpu_has_mipsmt);
+ BUG_ON(!cpu_has_mipsmt && !cpu_has_vp);
/* Boot a VPE on this core */
- mips_cps_boot_vpes();
+ mips_cps_boot_vpes(core_cfg, vpe_id);
out:
preempt_enable();
}
@@ -307,6 +348,17 @@
if (cpu_has_mipsmt)
dmt();
+ if (mips_cm_revision() >= CM_REV_CM3) {
+ unsigned ident = gic_read_local_vp_id();
+
+ /*
+ * Ensure that our calculation of the VP ID matches up with
+ * what the GIC reports, otherwise we'll have configured
+ * interrupts incorrectly.
+ */
+ BUG_ON(ident != mips_cm_vp_id(smp_processor_id()));
+ }
+
change_c0_status(ST0_IM, STATUSF_IP2 | STATUSF_IP3 | STATUSF_IP4 |
STATUSF_IP5 | STATUSF_IP6 | STATUSF_IP7);
}
diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index 27cb638..f9d01e9 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -243,18 +243,6 @@
struct irq_domain *ipidomain;
struct device_node *node;
- /*
- * In some cases like qemu-malta, it is desired to try SMP with
- * a single core. Qemu-malta has no GIC, so an attempt to set any IPIs
- * would cause a BUG_ON() to be triggered since there's no ipidomain.
- *
- * Since for a single core system IPIs aren't required really, skip the
- * initialisation which should generally keep any such configurations
- * happy and only fail hard when trying to truely run SMP.
- */
- if (cpumask_weight(cpu_possible_mask) == 1)
- return 0;
-
node = of_irq_find_parent(of_root);
ipidomain = irq_find_matching_host(node, DOMAIN_BUS_IPI);
@@ -266,7 +254,17 @@
if (node && !ipidomain)
ipidomain = irq_find_matching_host(NULL, DOMAIN_BUS_IPI);
- BUG_ON(!ipidomain);
+ /*
+ * There are systems which only use IPI domains some of the time,
+ * depending upon configuration we don't know until runtime. An
+ * example is Malta where we may compile in support for GIC & the
+ * MT ASE, but run on a system which has multiple VPEs in a single
+ * core and doesn't include a GIC. Until all IPI implementations
+ * have been converted to use IPI domains the best we can do here
+ * is to return & hope some other code sets up the IPIs.
+ */
+ if (!ipidomain)
+ return 0;
call_virq = irq_reserve_ipi(ipidomain, cpu_possible_mask);
BUG_ON(!call_virq);
diff --git a/arch/mips/kernel/spram.c b/arch/mips/kernel/spram.c
index 8489c88..d6e6cf7 100644
--- a/arch/mips/kernel/spram.c
+++ b/arch/mips/kernel/spram.c
@@ -210,6 +210,7 @@
case CPU_P5600:
case CPU_QEMU_GENERIC:
case CPU_I6400:
+ case CPU_P6600:
config0 = read_c0_config();
/* FIXME: addresses are Malta specific */
if (config0 & (1<<24)) {
diff --git a/arch/mips/kernel/traps.c b/arch/mips/kernel/traps.c
index ae0c89d..4a1712b 100644
--- a/arch/mips/kernel/traps.c
+++ b/arch/mips/kernel/traps.c
@@ -145,7 +145,7 @@
if (!task)
task = current;
- if (raw_show_trace || !__kernel_text_address(pc)) {
+ if (raw_show_trace || user_mode(regs) || !__kernel_text_address(pc)) {
show_raw_backtrace(sp);
return;
}
@@ -399,11 +399,8 @@
if (in_interrupt())
panic("Fatal exception in interrupt");
- if (panic_on_oops) {
- printk(KERN_EMERG "Fatal exception: panic in 5 seconds");
- ssleep(5);
+ if (panic_on_oops)
panic("Fatal exception");
- }
if (regs && kexec_should_crash(current))
crash_kexec(regs);
@@ -1249,7 +1246,7 @@
err = init_fpu();
if (msa && !err) {
enable_msa();
- _init_msa_upper();
+ init_msa_upper();
set_thread_flag(TIF_USEDMSA);
set_thread_flag(TIF_MSA_CTX_LIVE);
}
@@ -1312,7 +1309,7 @@
*/
prior_msa = test_and_set_thread_flag(TIF_MSA_CTX_LIVE);
if (!prior_msa && was_fpu_owner) {
- _init_msa_upper();
+ init_msa_upper();
goto out;
}
@@ -1329,7 +1326,7 @@
* of each vector register such that it cannot see data left
* behind by another task.
*/
- _init_msa_upper();
+ init_msa_upper();
} else {
/* We need to restore the vector context. */
restore_msa(current);
@@ -1356,7 +1353,6 @@
unsigned long fcr31;
unsigned int cpid;
int status, err;
- unsigned long __maybe_unused flags;
int sig;
prev_state = exception_enter();
@@ -1501,16 +1497,13 @@
{
siginfo_t info = { .si_signo = SIGTRAP, .si_code = TRAP_HWBKPT };
enum ctx_state prev_state;
- u32 cause;
prev_state = exception_enter();
/*
* Clear WP (bit 22) bit of cause register so we don't loop
* forever.
*/
- cause = read_c0_cause();
- cause &= ~(1 << 22);
- write_c0_cause(cause);
+ clear_c0_cause(CAUSEF_WP);
/*
* If the current thread has the watch registers loaded, save
@@ -1647,6 +1640,7 @@
case CPU_P5600:
case CPU_QEMU_GENERIC:
case CPU_I6400:
+ case CPU_P6600:
{
#define ERRCTL_PE 0x80000000
#define ERRCTL_L2P 0x00800000
@@ -1777,7 +1771,8 @@
/* For the moment, report the problem and hang. */
if ((cpu_has_mips_r2_r6) &&
- ((current_cpu_data.processor_id & 0xff0000) == PRID_COMP_MIPS)) {
+ (((current_cpu_data.processor_id & 0xff0000) == PRID_COMP_MIPS) ||
+ ((current_cpu_data.processor_id & 0xff0000) == PRID_COMP_LOONGSON))) {
pr_err("FTLB error exception, cp0_ecc=0x%08x:\n",
read_c0_ecc());
pr_err("cp0_errorepc == %0*lx\n", field, read_c0_errorepc());
@@ -2119,6 +2114,13 @@
* o read IntCtl.IPFDC to determine the fast debug channel interrupt
*/
if (cpu_has_mips_r2_r6) {
+ /*
+ * We shouldn't trust a secondary core has a sane EBASE register
+ * so use the one calculated by the boot CPU.
+ */
+ if (!is_boot_cpu)
+ write_c0_ebase(ebase);
+
cp0_compare_irq_shift = CAUSEB_TI - CAUSEB_IP;
cp0_compare_irq = (read_c0_intctl() >> INTCTLB_IPTI) & 7;
cp0_perfcount_irq = (read_c0_intctl() >> INTCTLB_IPPCI) & 7;
@@ -2134,7 +2136,7 @@
}
if (!cpu_data[cpu].asid_cache)
- cpu_data[cpu].asid_cache = ASID_FIRST_VERSION;
+ cpu_data[cpu].asid_cache = asid_first_version(cpu);
atomic_inc(&init_mm.mm_count);
current->active_mm = &init_mm;
diff --git a/arch/mips/kernel/unaligned.c b/arch/mips/kernel/unaligned.c
index 5c62065..28b3af7 100644
--- a/arch/mips/kernel/unaligned.c
+++ b/arch/mips/kernel/unaligned.c
@@ -1191,6 +1191,7 @@
case ldc1_op:
case swc1_op:
case sdc1_op:
+ case cop1x_op:
die_if_kernel("Unaligned FP access in kernel code", regs);
BUG_ON(!used_math());
diff --git a/arch/mips/kernel/vdso.c b/arch/mips/kernel/vdso.c
index 975e997..54e1663 100644
--- a/arch/mips/kernel/vdso.c
+++ b/arch/mips/kernel/vdso.c
@@ -104,7 +104,8 @@
struct resource gic_res;
int ret;
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
/*
* Determine total area size. This includes the VDSO data itself, the
diff --git a/arch/mips/kernel/vmlinux.lds.S b/arch/mips/kernel/vmlinux.lds.S
index 54d653e..a82c178 100644
--- a/arch/mips/kernel/vmlinux.lds.S
+++ b/arch/mips/kernel/vmlinux.lds.S
@@ -136,6 +136,27 @@
#ifdef CONFIG_SMP
PERCPU_SECTION(1 << CONFIG_MIPS_L1_CACHE_SHIFT)
#endif
+
+#ifdef CONFIG_RELOCATABLE
+ . = ALIGN(4);
+
+ .data.reloc : {
+ _relocation_start = .;
+ /*
+ * Space for relocation table
+ * This needs to be filled so that the
+ * relocs tool can overwrite the content.
+ * An invalid value is left at the start of the
+ * section to abort relocation if the table
+ * has not been filled in.
+ */
+ LONG(0xFFFFFFFF);
+ FILL(0);
+ . += CONFIG_RELOCATION_TABLE_SIZE - 4;
+ _relocation_end = .;
+ }
+#endif
+
#ifdef CONFIG_MIPS_RAW_APPENDED_DTB
__appended_dtb = .;
/* leave space for appended DTB */
diff --git a/arch/mips/kernel/watch.c b/arch/mips/kernel/watch.c
index 2a03abb..19fcab7 100644
--- a/arch/mips/kernel/watch.c
+++ b/arch/mips/kernel/watch.c
@@ -15,10 +15,9 @@
* Install the watch registers for the current thread. A maximum of
* four registers are installed although the machine may have more.
*/
-void mips_install_watch_registers(void)
+void mips_install_watch_registers(struct task_struct *t)
{
- struct mips3264_watch_reg_state *watches =
- ¤t->thread.watch.mips3264;
+ struct mips3264_watch_reg_state *watches = &t->thread.watch.mips3264;
switch (current_cpu_data.watch_reg_use_cnt) {
default:
BUG();
@@ -26,16 +25,20 @@
write_c0_watchlo3(watches->watchlo[3]);
/* Write 1 to the I, R, and W bits to clear them, and
1 to G so all ASIDs are trapped. */
- write_c0_watchhi3(0x40000007 | watches->watchhi[3]);
+ write_c0_watchhi3(MIPS_WATCHHI_G | MIPS_WATCHHI_IRW |
+ watches->watchhi[3]);
case 3:
write_c0_watchlo2(watches->watchlo[2]);
- write_c0_watchhi2(0x40000007 | watches->watchhi[2]);
+ write_c0_watchhi2(MIPS_WATCHHI_G | MIPS_WATCHHI_IRW |
+ watches->watchhi[2]);
case 2:
write_c0_watchlo1(watches->watchlo[1]);
- write_c0_watchhi1(0x40000007 | watches->watchhi[1]);
+ write_c0_watchhi1(MIPS_WATCHHI_G | MIPS_WATCHHI_IRW |
+ watches->watchhi[1]);
case 1:
write_c0_watchlo0(watches->watchlo[0]);
- write_c0_watchhi0(0x40000007 | watches->watchhi[0]);
+ write_c0_watchhi0(MIPS_WATCHHI_G | MIPS_WATCHHI_IRW |
+ watches->watchhi[0]);
}
}
@@ -52,22 +55,26 @@
default:
BUG();
case 4:
- watches->watchhi[3] = (read_c0_watchhi3() & 0x0fff);
+ watches->watchhi[3] = (read_c0_watchhi3() &
+ (MIPS_WATCHHI_MASK | MIPS_WATCHHI_IRW));
case 3:
- watches->watchhi[2] = (read_c0_watchhi2() & 0x0fff);
+ watches->watchhi[2] = (read_c0_watchhi2() &
+ (MIPS_WATCHHI_MASK | MIPS_WATCHHI_IRW));
case 2:
- watches->watchhi[1] = (read_c0_watchhi1() & 0x0fff);
+ watches->watchhi[1] = (read_c0_watchhi1() &
+ (MIPS_WATCHHI_MASK | MIPS_WATCHHI_IRW));
case 1:
- watches->watchhi[0] = (read_c0_watchhi0() & 0x0fff);
+ watches->watchhi[0] = (read_c0_watchhi0() &
+ (MIPS_WATCHHI_MASK | MIPS_WATCHHI_IRW));
}
if (current_cpu_data.watch_reg_use_cnt == 1 &&
- (watches->watchhi[0] & 7) == 0) {
+ (watches->watchhi[0] & MIPS_WATCHHI_IRW) == 0) {
/* Pathological case of release 1 architecture that
* doesn't set the condition bits. We assume that
* since we got here, the watch condition was met and
* signal that the conditions requested in watchlo
* were met. */
- watches->watchhi[0] |= (watches->watchlo[0] & 7);
+ watches->watchhi[0] |= (watches->watchlo[0] & MIPS_WATCHHI_IRW);
}
}
@@ -110,86 +117,86 @@
* Check which of the I,R and W bits are supported, then
* disable the register.
*/
- write_c0_watchlo0(7);
+ write_c0_watchlo0(MIPS_WATCHLO_IRW);
back_to_back_c0_hazard();
t = read_c0_watchlo0();
write_c0_watchlo0(0);
- c->watch_reg_masks[0] = t & 7;
+ c->watch_reg_masks[0] = t & MIPS_WATCHLO_IRW;
/* Write the mask bits and read them back to determine which
* can be used. */
c->watch_reg_count = 1;
c->watch_reg_use_cnt = 1;
t = read_c0_watchhi0();
- write_c0_watchhi0(t | 0xff8);
+ write_c0_watchhi0(t | MIPS_WATCHHI_MASK);
back_to_back_c0_hazard();
t = read_c0_watchhi0();
- c->watch_reg_masks[0] |= (t & 0xff8);
- if ((t & 0x80000000) == 0)
+ c->watch_reg_masks[0] |= (t & MIPS_WATCHHI_MASK);
+ if ((t & MIPS_WATCHHI_M) == 0)
return;
- write_c0_watchlo1(7);
+ write_c0_watchlo1(MIPS_WATCHLO_IRW);
back_to_back_c0_hazard();
t = read_c0_watchlo1();
write_c0_watchlo1(0);
- c->watch_reg_masks[1] = t & 7;
+ c->watch_reg_masks[1] = t & MIPS_WATCHLO_IRW;
c->watch_reg_count = 2;
c->watch_reg_use_cnt = 2;
t = read_c0_watchhi1();
- write_c0_watchhi1(t | 0xff8);
+ write_c0_watchhi1(t | MIPS_WATCHHI_MASK);
back_to_back_c0_hazard();
t = read_c0_watchhi1();
- c->watch_reg_masks[1] |= (t & 0xff8);
- if ((t & 0x80000000) == 0)
+ c->watch_reg_masks[1] |= (t & MIPS_WATCHHI_MASK);
+ if ((t & MIPS_WATCHHI_M) == 0)
return;
- write_c0_watchlo2(7);
+ write_c0_watchlo2(MIPS_WATCHLO_IRW);
back_to_back_c0_hazard();
t = read_c0_watchlo2();
write_c0_watchlo2(0);
- c->watch_reg_masks[2] = t & 7;
+ c->watch_reg_masks[2] = t & MIPS_WATCHLO_IRW;
c->watch_reg_count = 3;
c->watch_reg_use_cnt = 3;
t = read_c0_watchhi2();
- write_c0_watchhi2(t | 0xff8);
+ write_c0_watchhi2(t | MIPS_WATCHHI_MASK);
back_to_back_c0_hazard();
t = read_c0_watchhi2();
- c->watch_reg_masks[2] |= (t & 0xff8);
- if ((t & 0x80000000) == 0)
+ c->watch_reg_masks[2] |= (t & MIPS_WATCHHI_MASK);
+ if ((t & MIPS_WATCHHI_M) == 0)
return;
- write_c0_watchlo3(7);
+ write_c0_watchlo3(MIPS_WATCHLO_IRW);
back_to_back_c0_hazard();
t = read_c0_watchlo3();
write_c0_watchlo3(0);
- c->watch_reg_masks[3] = t & 7;
+ c->watch_reg_masks[3] = t & MIPS_WATCHLO_IRW;
c->watch_reg_count = 4;
c->watch_reg_use_cnt = 4;
t = read_c0_watchhi3();
- write_c0_watchhi3(t | 0xff8);
+ write_c0_watchhi3(t | MIPS_WATCHHI_MASK);
back_to_back_c0_hazard();
t = read_c0_watchhi3();
- c->watch_reg_masks[3] |= (t & 0xff8);
- if ((t & 0x80000000) == 0)
+ c->watch_reg_masks[3] |= (t & MIPS_WATCHHI_MASK);
+ if ((t & MIPS_WATCHHI_M) == 0)
return;
/* We use at most 4, but probe and report up to 8. */
c->watch_reg_count = 5;
t = read_c0_watchhi4();
- if ((t & 0x80000000) == 0)
+ if ((t & MIPS_WATCHHI_M) == 0)
return;
c->watch_reg_count = 6;
t = read_c0_watchhi5();
- if ((t & 0x80000000) == 0)
+ if ((t & MIPS_WATCHHI_M) == 0)
return;
c->watch_reg_count = 7;
t = read_c0_watchhi6();
- if ((t & 0x80000000) == 0)
+ if ((t & MIPS_WATCHHI_M) == 0)
return;
c->watch_reg_count = 8;
diff --git a/arch/mips/kvm/emulate.c b/arch/mips/kvm/emulate.c
index b37954c..396df6e 100644
--- a/arch/mips/kvm/emulate.c
+++ b/arch/mips/kvm/emulate.c
@@ -302,12 +302,31 @@
*/
static uint32_t kvm_mips_read_count_running(struct kvm_vcpu *vcpu, ktime_t now)
{
- ktime_t expires;
+ struct mips_coproc *cop0 = vcpu->arch.cop0;
+ ktime_t expires, threshold;
+ uint32_t count, compare;
int running;
- /* Is the hrtimer pending? */
+ /* Calculate the biased and scaled guest CP0_Count */
+ count = vcpu->arch.count_bias + kvm_mips_ktime_to_count(vcpu, now);
+ compare = kvm_read_c0_guest_compare(cop0);
+
+ /*
+ * Find whether CP0_Count has reached the closest timer interrupt. If
+ * not, we shouldn't inject it.
+ */
+ if ((int32_t)(count - compare) < 0)
+ return count;
+
+ /*
+ * The CP0_Count we're going to return has already reached the closest
+ * timer interrupt. Quickly check if it really is a new interrupt by
+ * looking at whether the interval until the hrtimer expiry time is
+ * less than 1/4 of the timer period.
+ */
expires = hrtimer_get_expires(&vcpu->arch.comparecount_timer);
- if (ktime_compare(now, expires) >= 0) {
+ threshold = ktime_add_ns(now, vcpu->arch.count_period / 4);
+ if (ktime_before(expires, threshold)) {
/*
* Cancel it while we handle it so there's no chance of
* interference with the timeout handler.
@@ -329,8 +348,7 @@
}
}
- /* Return the biased and scaled guest CP0_Count */
- return vcpu->arch.count_bias + kvm_mips_ktime_to_count(vcpu, now);
+ return count;
}
/**
@@ -420,32 +438,6 @@
}
/**
- * kvm_mips_update_hrtimer() - Update next expiry time of hrtimer.
- * @vcpu: Virtual CPU.
- *
- * Recalculates and updates the expiry time of the hrtimer. This can be used
- * after timer parameters have been altered which do not depend on the time that
- * the change occurs (in those cases kvm_mips_freeze_hrtimer() and
- * kvm_mips_resume_hrtimer() are used directly).
- *
- * It is guaranteed that no timer interrupts will be lost in the process.
- *
- * Assumes !kvm_mips_count_disabled(@vcpu) (guest CP0_Count timer is running).
- */
-static void kvm_mips_update_hrtimer(struct kvm_vcpu *vcpu)
-{
- ktime_t now;
- uint32_t count;
-
- /*
- * freeze_hrtimer takes care of a timer interrupts <= count, and
- * resume_hrtimer the hrtimer takes care of a timer interrupts > count.
- */
- now = kvm_mips_freeze_hrtimer(vcpu, &count);
- kvm_mips_resume_hrtimer(vcpu, now, count);
-}
-
-/**
* kvm_mips_write_count() - Modify the count and update timer.
* @vcpu: Virtual CPU.
* @count: Guest CP0_Count value to set.
@@ -540,23 +532,42 @@
* kvm_mips_write_compare() - Modify compare and update timer.
* @vcpu: Virtual CPU.
* @compare: New CP0_Compare value.
+ * @ack: Whether to acknowledge timer interrupt.
*
* Update CP0_Compare to a new value and update the timeout.
+ * If @ack, atomically acknowledge any pending timer interrupt, otherwise ensure
+ * any pending timer interrupt is preserved.
*/
-void kvm_mips_write_compare(struct kvm_vcpu *vcpu, uint32_t compare)
+void kvm_mips_write_compare(struct kvm_vcpu *vcpu, uint32_t compare, bool ack)
{
struct mips_coproc *cop0 = vcpu->arch.cop0;
+ int dc;
+ u32 old_compare = kvm_read_c0_guest_compare(cop0);
+ ktime_t now;
+ uint32_t count;
/* if unchanged, must just be an ack */
- if (kvm_read_c0_guest_compare(cop0) == compare)
+ if (old_compare == compare) {
+ if (!ack)
+ return;
+ kvm_mips_callbacks->dequeue_timer_int(vcpu);
+ kvm_write_c0_guest_compare(cop0, compare);
return;
+ }
- /* Update compare */
+ /* freeze_hrtimer() takes care of timer interrupts <= count */
+ dc = kvm_mips_count_disabled(vcpu);
+ if (!dc)
+ now = kvm_mips_freeze_hrtimer(vcpu, &count);
+
+ if (ack)
+ kvm_mips_callbacks->dequeue_timer_int(vcpu);
+
kvm_write_c0_guest_compare(cop0, compare);
- /* Update timeout if count enabled */
- if (!kvm_mips_count_disabled(vcpu))
- kvm_mips_update_hrtimer(vcpu);
+ /* resume_hrtimer() takes care of timer interrupts > count */
+ if (!dc)
+ kvm_mips_resume_hrtimer(vcpu, now, count);
}
/**
@@ -1068,15 +1079,15 @@
kvm_read_c0_guest_ebase(cop0));
} else if (rd == MIPS_CP0_TLB_HI && sel == 0) {
uint32_t nasid =
- vcpu->arch.gprs[rt] & ASID_MASK;
+ vcpu->arch.gprs[rt] & KVM_ENTRYHI_ASID;
if ((KSEGX(vcpu->arch.gprs[rt]) != CKSEG0) &&
((kvm_read_c0_guest_entryhi(cop0) &
- ASID_MASK) != nasid)) {
+ KVM_ENTRYHI_ASID) != nasid)) {
kvm_debug("MTCz, change ASID from %#lx to %#lx\n",
kvm_read_c0_guest_entryhi(cop0)
- & ASID_MASK,
+ & KVM_ENTRYHI_ASID,
vcpu->arch.gprs[rt]
- & ASID_MASK);
+ & KVM_ENTRYHI_ASID);
/* Blow away the shadow host TLBs */
kvm_mips_flush_host_tlb(1);
@@ -1095,9 +1106,9 @@
/* If we are writing to COMPARE */
/* Clear pending timer interrupt, if any */
- kvm_mips_callbacks->dequeue_timer_int(vcpu);
kvm_mips_write_compare(vcpu,
- vcpu->arch.gprs[rt]);
+ vcpu->arch.gprs[rt],
+ true);
} else if ((rd == MIPS_CP0_STATUS) && (sel == 0)) {
unsigned int old_val, val, change;
@@ -1620,7 +1631,7 @@
*/
index = kvm_mips_guest_tlb_lookup(vcpu, (va & VPN2_MASK) |
(kvm_read_c0_guest_entryhi
- (cop0) & ASID_MASK));
+ (cop0) & KVM_ENTRYHI_ASID));
if (index < 0) {
vcpu->arch.host_cp0_entryhi = (va & VPN2_MASK);
@@ -1786,7 +1797,7 @@
struct mips_coproc *cop0 = vcpu->arch.cop0;
struct kvm_vcpu_arch *arch = &vcpu->arch;
unsigned long entryhi = (vcpu->arch. host_cp0_badvaddr & VPN2_MASK) |
- (kvm_read_c0_guest_entryhi(cop0) & ASID_MASK);
+ (kvm_read_c0_guest_entryhi(cop0) & KVM_ENTRYHI_ASID);
if ((kvm_read_c0_guest_status(cop0) & ST0_EXL) == 0) {
/* save old pc */
@@ -1833,7 +1844,7 @@
struct kvm_vcpu_arch *arch = &vcpu->arch;
unsigned long entryhi =
(vcpu->arch.host_cp0_badvaddr & VPN2_MASK) |
- (kvm_read_c0_guest_entryhi(cop0) & ASID_MASK);
+ (kvm_read_c0_guest_entryhi(cop0) & KVM_ENTRYHI_ASID);
if ((kvm_read_c0_guest_status(cop0) & ST0_EXL) == 0) {
/* save old pc */
@@ -1878,7 +1889,7 @@
struct mips_coproc *cop0 = vcpu->arch.cop0;
struct kvm_vcpu_arch *arch = &vcpu->arch;
unsigned long entryhi = (vcpu->arch.host_cp0_badvaddr & VPN2_MASK) |
- (kvm_read_c0_guest_entryhi(cop0) & ASID_MASK);
+ (kvm_read_c0_guest_entryhi(cop0) & KVM_ENTRYHI_ASID);
if ((kvm_read_c0_guest_status(cop0) & ST0_EXL) == 0) {
/* save old pc */
@@ -1922,7 +1933,7 @@
struct mips_coproc *cop0 = vcpu->arch.cop0;
struct kvm_vcpu_arch *arch = &vcpu->arch;
unsigned long entryhi = (vcpu->arch.host_cp0_badvaddr & VPN2_MASK) |
- (kvm_read_c0_guest_entryhi(cop0) & ASID_MASK);
+ (kvm_read_c0_guest_entryhi(cop0) & KVM_ENTRYHI_ASID);
if ((kvm_read_c0_guest_status(cop0) & ST0_EXL) == 0) {
/* save old pc */
@@ -1967,7 +1978,7 @@
#ifdef DEBUG
struct mips_coproc *cop0 = vcpu->arch.cop0;
unsigned long entryhi = (vcpu->arch.host_cp0_badvaddr & VPN2_MASK) |
- (kvm_read_c0_guest_entryhi(cop0) & ASID_MASK);
+ (kvm_read_c0_guest_entryhi(cop0) & KVM_ENTRYHI_ASID);
int index;
/* If address not in the guest TLB, then we are in trouble */
@@ -1994,7 +2005,7 @@
{
struct mips_coproc *cop0 = vcpu->arch.cop0;
unsigned long entryhi = (vcpu->arch.host_cp0_badvaddr & VPN2_MASK) |
- (kvm_read_c0_guest_entryhi(cop0) & ASID_MASK);
+ (kvm_read_c0_guest_entryhi(cop0) & KVM_ENTRYHI_ASID);
struct kvm_vcpu_arch *arch = &vcpu->arch;
if ((kvm_read_c0_guest_status(cop0) & ST0_EXL) == 0) {
@@ -2569,7 +2580,8 @@
*/
index = kvm_mips_guest_tlb_lookup(vcpu,
(va & VPN2_MASK) |
- (kvm_read_c0_guest_entryhi(vcpu->arch.cop0) & ASID_MASK));
+ (kvm_read_c0_guest_entryhi(vcpu->arch.cop0) &
+ KVM_ENTRYHI_ASID));
if (index < 0) {
if (exccode == EXCCODE_TLBL) {
er = kvm_mips_emulate_tlbmiss_ld(cause, opc, run, vcpu);
diff --git a/arch/mips/kvm/locore.S b/arch/mips/kvm/locore.S
index 81687ab..3ef0300 100644
--- a/arch/mips/kvm/locore.S
+++ b/arch/mips/kvm/locore.S
@@ -32,7 +32,6 @@
EXPORT(x);
/* Overload, Danger Will Robinson!! */
-#define PT_HOST_ASID PT_BVADDR
#define PT_HOST_USERLOCAL PT_EPC
#define CP0_DDATA_LO $28,3
@@ -49,45 +48,18 @@
* a1: vcpu
*/
.set noreorder
- .set noat
FEXPORT(__kvm_mips_vcpu_run)
/* k0/k1 not being used in host kernel context */
INT_ADDIU k1, sp, -PT_SIZE
- LONG_S $0, PT_R0(k1)
- LONG_S $1, PT_R1(k1)
- LONG_S $2, PT_R2(k1)
- LONG_S $3, PT_R3(k1)
-
- LONG_S $4, PT_R4(k1)
- LONG_S $5, PT_R5(k1)
- LONG_S $6, PT_R6(k1)
- LONG_S $7, PT_R7(k1)
-
- LONG_S $8, PT_R8(k1)
- LONG_S $9, PT_R9(k1)
- LONG_S $10, PT_R10(k1)
- LONG_S $11, PT_R11(k1)
- LONG_S $12, PT_R12(k1)
- LONG_S $13, PT_R13(k1)
- LONG_S $14, PT_R14(k1)
- LONG_S $15, PT_R15(k1)
LONG_S $16, PT_R16(k1)
LONG_S $17, PT_R17(k1)
-
LONG_S $18, PT_R18(k1)
LONG_S $19, PT_R19(k1)
LONG_S $20, PT_R20(k1)
LONG_S $21, PT_R21(k1)
LONG_S $22, PT_R22(k1)
LONG_S $23, PT_R23(k1)
- LONG_S $24, PT_R24(k1)
- LONG_S $25, PT_R25(k1)
-
- /*
- * XXXKYMA k0/k1 not saved, not being used if we got here through
- * an ioctl()
- */
LONG_S $28, PT_R28(k1)
LONG_S $29, PT_R29(k1)
@@ -104,11 +76,6 @@
mfc0 v0, CP0_STATUS
LONG_S v0, PT_STATUS(k1)
- /* Save host ASID, shove it into the BVADDR location */
- mfc0 v1, CP0_ENTRYHI
- andi v1, 0xff
- LONG_S v1, PT_HOST_ASID(k1)
-
/* Save DDATA_LO, will be used to store pointer to vcpu */
mfc0 v1, CP0_DDATA_LO
LONG_S v1, PT_HOST_USERLOCAL(k1)
@@ -170,13 +137,21 @@
INT_SLL t2, t2, 2 /* x4 */
REG_ADDU t3, t1, t2
LONG_L k0, (t3)
- andi k0, k0, 0xff
+#ifdef CONFIG_MIPS_ASID_BITS_VARIABLE
+ li t3, CPUINFO_SIZE/4
+ mul t2, t2, t3 /* x sizeof(struct cpuinfo_mips)/4 */
+ LONG_L t2, (cpu_data + CPUINFO_ASID_MASK)(t2)
+ and k0, k0, t2
+#else
+ andi k0, k0, MIPS_ENTRYHI_ASID
+#endif
mtc0 k0, CP0_ENTRYHI
ehb
/* Disable RDHWR access */
mtc0 zero, CP0_HWRENA
+ .set noat
/* Now load up the Guest Context from VCPU */
LONG_L $1, VCPU_R1(k1)
LONG_L $2, VCPU_R2(k1)
@@ -288,6 +263,8 @@
LONG_S $30, VCPU_R30(k1)
LONG_S $31, VCPU_R31(k1)
+ .set at
+
/* We need to save hi/lo and restore them on the way out */
mfhi t0
LONG_S t0, VCPU_HI(k1)
@@ -339,9 +316,7 @@
/* load up the host EBASE */
mfc0 v0, CP0_STATUS
- .set at
or k0, v0, ST0_BEV
- .set noat
mtc0 k0, CP0_STATUS
ehb
@@ -353,7 +328,6 @@
* If FPU is enabled, save FCR31 and clear it so that later ctc1's don't
* trigger FPE for pending exceptions.
*/
- .set at
and v1, v0, ST0_CU1
beqz v1, 1f
nop
@@ -363,7 +337,6 @@
sw t0, VCPU_FCR31(k1)
ctc1 zero,fcr31
.set pop
- .set noat
1:
#ifdef CONFIG_CPU_HAS_MSA
@@ -386,10 +359,8 @@
#endif
/* Now that the new EBASE has been loaded, unset BEV and KSU_USER */
- .set at
and v0, v0, ~(ST0_EXL | KSU_USER | ST0_IE)
or v0, v0, ST0_CU0
- .set noat
mtc0 v0, CP0_STATUS
ehb
@@ -456,18 +427,14 @@
/* Switch EBASE back to the one used by KVM */
mfc0 v1, CP0_STATUS
- .set at
or k0, v1, ST0_BEV
- .set noat
mtc0 k0, CP0_STATUS
ehb
mtc0 t0, CP0_EBASE
/* Setup status register for running guest in UM */
- .set at
or v1, v1, (ST0_EXL | KSU_USER | ST0_IE)
and v1, v1, ~(ST0_CU0 | ST0_MX)
- .set noat
mtc0 v1, CP0_STATUS
ehb
@@ -489,13 +456,21 @@
INT_SLL t2, t2, 2 /* x4 */
REG_ADDU t3, t1, t2
LONG_L k0, (t3)
- andi k0, k0, 0xff
+#ifdef CONFIG_MIPS_ASID_BITS_VARIABLE
+ li t3, CPUINFO_SIZE/4
+ mul t2, t2, t3 /* x sizeof(struct cpuinfo_mips)/4 */
+ LONG_L t2, (cpu_data + CPUINFO_ASID_MASK)(t2)
+ and k0, k0, t2
+#else
+ andi k0, k0, MIPS_ENTRYHI_ASID
+#endif
mtc0 k0, CP0_ENTRYHI
ehb
/* Disable RDHWR access */
mtc0 zero, CP0_HWRENA
+ .set noat
/* load the guest context from VCPU and return */
LONG_L $0, VCPU_R0(k1)
LONG_L $1, VCPU_R1(k1)
@@ -541,6 +516,7 @@
LONG_L k1, VCPU_R27(k1)
eret
+ .set at
__kvm_mips_return_to_host:
/* EBASE is already pointing to Linux */
@@ -551,16 +527,6 @@
LONG_L k0, PT_HOST_USERLOCAL(k1)
mtc0 k0, CP0_DDATA_LO
- /* Restore host ASID */
- LONG_L k0, PT_HOST_ASID(sp)
- andi k0, 0xff
- mtc0 k0,CP0_ENTRYHI
- ehb
-
- /* Load context saved on the host stack */
- LONG_L $0, PT_R0(k1)
- LONG_L $1, PT_R1(k1)
-
/*
* r2/v0 is the return code, shift it down by 2 (arithmetic)
* to recover the err code
@@ -568,19 +534,7 @@
INT_SRA k0, v0, 2
move $2, k0
- LONG_L $3, PT_R3(k1)
- LONG_L $4, PT_R4(k1)
- LONG_L $5, PT_R5(k1)
- LONG_L $6, PT_R6(k1)
- LONG_L $7, PT_R7(k1)
- LONG_L $8, PT_R8(k1)
- LONG_L $9, PT_R9(k1)
- LONG_L $10, PT_R10(k1)
- LONG_L $11, PT_R11(k1)
- LONG_L $12, PT_R12(k1)
- LONG_L $13, PT_R13(k1)
- LONG_L $14, PT_R14(k1)
- LONG_L $15, PT_R15(k1)
+ /* Load context saved on the host stack */
LONG_L $16, PT_R16(k1)
LONG_L $17, PT_R17(k1)
LONG_L $18, PT_R18(k1)
@@ -589,10 +543,6 @@
LONG_L $21, PT_R21(k1)
LONG_L $22, PT_R22(k1)
LONG_L $23, PT_R23(k1)
- LONG_L $24, PT_R24(k1)
- LONG_L $25, PT_R25(k1)
-
- /* Host k0/k1 were not saved */
LONG_L $28, PT_R28(k1)
LONG_L $29, PT_R29(k1)
diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
index 70ef1a4..dc052fb 100644
--- a/arch/mips/kvm/mips.c
+++ b/arch/mips/kvm/mips.c
@@ -56,6 +56,7 @@
{ "flush_dcache", VCPU_STAT(flush_dcache_exits), KVM_STAT_VCPU },
{ "halt_successful_poll", VCPU_STAT(halt_successful_poll), KVM_STAT_VCPU },
{ "halt_attempted_poll", VCPU_STAT(halt_attempted_poll), KVM_STAT_VCPU },
+ { "halt_poll_invalid", VCPU_STAT(halt_poll_invalid), KVM_STAT_VCPU },
{ "halt_wakeup", VCPU_STAT(halt_wakeup), KVM_STAT_VCPU },
{NULL}
};
@@ -1079,7 +1080,8 @@
r = KVM_COALESCED_MMIO_PAGE_OFFSET;
break;
case KVM_CAP_MIPS_FPU:
- r = !!cpu_has_fpu;
+ /* We don't handle systems with inconsistent cpu_has_fpu */
+ r = !!raw_cpu_has_fpu;
break;
case KVM_CAP_MIPS_MSA:
/*
@@ -1555,8 +1557,10 @@
/* Disable MSA & FPU */
disable_msa();
- if (vcpu->arch.fpu_inuse & KVM_MIPS_FPU_FPU)
+ if (vcpu->arch.fpu_inuse & KVM_MIPS_FPU_FPU) {
clear_c0_status(ST0_CU1 | ST0_FR);
+ disable_fpu_hazard();
+ }
vcpu->arch.fpu_inuse &= ~(KVM_MIPS_FPU_FPU | KVM_MIPS_FPU_MSA);
} else if (vcpu->arch.fpu_inuse & KVM_MIPS_FPU_FPU) {
set_c0_status(ST0_CU1);
@@ -1567,6 +1571,7 @@
/* Disable FPU */
clear_c0_status(ST0_CU1 | ST0_FR);
+ disable_fpu_hazard();
}
preempt_enable();
}
diff --git a/arch/mips/kvm/tlb.c b/arch/mips/kvm/tlb.c
index e0e1d0a..ed021ae 100644
--- a/arch/mips/kvm/tlb.c
+++ b/arch/mips/kvm/tlb.c
@@ -49,12 +49,18 @@
uint32_t kvm_mips_get_kernel_asid(struct kvm_vcpu *vcpu)
{
- return vcpu->arch.guest_kernel_asid[smp_processor_id()] & ASID_MASK;
+ int cpu = smp_processor_id();
+
+ return vcpu->arch.guest_kernel_asid[cpu] &
+ cpu_asid_mask(&cpu_data[cpu]);
}
uint32_t kvm_mips_get_user_asid(struct kvm_vcpu *vcpu)
{
- return vcpu->arch.guest_user_asid[smp_processor_id()] & ASID_MASK;
+ int cpu = smp_processor_id();
+
+ return vcpu->arch.guest_user_asid[cpu] &
+ cpu_asid_mask(&cpu_data[cpu]);
}
inline uint32_t kvm_mips_get_commpage_asid(struct kvm_vcpu *vcpu)
@@ -78,7 +84,8 @@
old_pagemask = read_c0_pagemask();
kvm_info("HOST TLBs:\n");
- kvm_info("ASID: %#lx\n", read_c0_entryhi() & ASID_MASK);
+ kvm_info("ASID: %#lx\n", read_c0_entryhi() &
+ cpu_asid_mask(¤t_cpu_data));
for (i = 0; i < current_cpu_data.tlbsize; i++) {
write_c0_index(i);
@@ -268,6 +275,7 @@
int even;
struct kvm *kvm = vcpu->kvm;
const int flush_dcache_mask = 0;
+ int ret;
if (KVM_GUEST_KSEGX(badvaddr) != KVM_GUEST_KSEG0) {
kvm_err("%s: Invalid BadVaddr: %#lx\n", __func__, badvaddr);
@@ -299,14 +307,18 @@
pfn1 = kvm->arch.guest_pmap[gfn];
}
- entryhi = (vaddr | kvm_mips_get_kernel_asid(vcpu));
entrylo0 = mips3_paddr_to_tlbpfn(pfn0 << PAGE_SHIFT) | (0x3 << 3) |
(1 << 2) | (0x1 << 1);
entrylo1 = mips3_paddr_to_tlbpfn(pfn1 << PAGE_SHIFT) | (0x3 << 3) |
(1 << 2) | (0x1 << 1);
- return kvm_mips_host_tlb_write(vcpu, entryhi, entrylo0, entrylo1,
- flush_dcache_mask);
+ preempt_disable();
+ entryhi = (vaddr | kvm_mips_get_kernel_asid(vcpu));
+ ret = kvm_mips_host_tlb_write(vcpu, entryhi, entrylo0, entrylo1,
+ flush_dcache_mask);
+ preempt_enable();
+
+ return ret;
}
EXPORT_SYMBOL_GPL(kvm_mips_handle_kseg0_tlb_fault);
@@ -361,6 +373,7 @@
unsigned long entryhi = 0, entrylo0 = 0, entrylo1 = 0;
struct kvm *kvm = vcpu->kvm;
kvm_pfn_t pfn0, pfn1;
+ int ret;
if ((tlb->tlb_hi & VPN2_MASK) == 0) {
pfn0 = 0;
@@ -387,9 +400,6 @@
*hpa1 = pfn1 << PAGE_SHIFT;
/* Get attributes from the Guest TLB */
- entryhi = (tlb->tlb_hi & VPN2_MASK) | (KVM_GUEST_KERNEL_MODE(vcpu) ?
- kvm_mips_get_kernel_asid(vcpu) :
- kvm_mips_get_user_asid(vcpu));
entrylo0 = mips3_paddr_to_tlbpfn(pfn0 << PAGE_SHIFT) | (0x3 << 3) |
(tlb->tlb_lo0 & MIPS3_PG_D) | (tlb->tlb_lo0 & MIPS3_PG_V);
entrylo1 = mips3_paddr_to_tlbpfn(pfn1 << PAGE_SHIFT) | (0x3 << 3) |
@@ -398,8 +408,15 @@
kvm_debug("@ %#lx tlb_lo0: 0x%08lx tlb_lo1: 0x%08lx\n", vcpu->arch.pc,
tlb->tlb_lo0, tlb->tlb_lo1);
- return kvm_mips_host_tlb_write(vcpu, entryhi, entrylo0, entrylo1,
- tlb->tlb_mask);
+ preempt_disable();
+ entryhi = (tlb->tlb_hi & VPN2_MASK) | (KVM_GUEST_KERNEL_MODE(vcpu) ?
+ kvm_mips_get_kernel_asid(vcpu) :
+ kvm_mips_get_user_asid(vcpu));
+ ret = kvm_mips_host_tlb_write(vcpu, entryhi, entrylo0, entrylo1,
+ tlb->tlb_mask);
+ preempt_enable();
+
+ return ret;
}
EXPORT_SYMBOL_GPL(kvm_mips_handle_mapped_seg_tlb_fault);
@@ -564,15 +581,15 @@
{
unsigned long asid = asid_cache(cpu);
- asid += ASID_INC;
- if (!(asid & ASID_MASK)) {
+ asid += cpu_asid_inc();
+ if (!(asid & cpu_asid_mask(&cpu_data[cpu]))) {
if (cpu_has_vtag_icache)
flush_icache_all();
kvm_local_flush_tlb_all(); /* start new asid cycle */
if (!asid) /* fix version if needed */
- asid = ASID_FIRST_VERSION;
+ asid = asid_first_version(cpu);
}
cpu_context(cpu, mm) = asid_cache(cpu) = asid;
@@ -627,6 +644,7 @@
/* Restore ASID once we are scheduled back after preemption */
void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
{
+ unsigned long asid_mask = cpu_asid_mask(&cpu_data[cpu]);
unsigned long flags;
int newasid = 0;
@@ -637,7 +655,7 @@
local_irq_save(flags);
if ((vcpu->arch.guest_kernel_asid[cpu] ^ asid_cache(cpu)) &
- ASID_VERSION_MASK) {
+ asid_version_mask(cpu)) {
kvm_get_new_mmu_context(&vcpu->arch.guest_kernel_mm, cpu, vcpu);
vcpu->arch.guest_kernel_asid[cpu] =
vcpu->arch.guest_kernel_mm.context.asid[cpu];
@@ -672,7 +690,7 @@
*/
if (current->flags & PF_VCPU) {
write_c0_entryhi(vcpu->arch.
- preempt_entryhi & ASID_MASK);
+ preempt_entryhi & asid_mask);
ehb();
}
} else {
@@ -687,11 +705,11 @@
if (KVM_GUEST_KERNEL_MODE(vcpu))
write_c0_entryhi(vcpu->arch.
guest_kernel_asid[cpu] &
- ASID_MASK);
+ asid_mask);
else
write_c0_entryhi(vcpu->arch.
guest_user_asid[cpu] &
- ASID_MASK);
+ asid_mask);
ehb();
}
}
@@ -721,7 +739,7 @@
kvm_mips_callbacks->vcpu_get_regs(vcpu);
if (((cpu_context(cpu, current->mm) ^ asid_cache(cpu)) &
- ASID_VERSION_MASK)) {
+ asid_version_mask(cpu))) {
kvm_debug("%s: Dropping MMU Context: %#lx\n", __func__,
cpu_context(cpu, current->mm));
drop_mmu_context(current->mm, cpu);
@@ -748,7 +766,8 @@
inst = *(opc);
} else {
vpn2 = (unsigned long) opc & VPN2_MASK;
- asid = kvm_read_c0_guest_entryhi(cop0) & ASID_MASK;
+ asid = kvm_read_c0_guest_entryhi(cop0) &
+ KVM_ENTRYHI_ASID;
index = kvm_mips_guest_tlb_lookup(vcpu, vpn2 | asid);
if (index < 0) {
kvm_err("%s: get_user_failed for %p, vcpu: %p, ASID: %#lx\n",
diff --git a/arch/mips/kvm/trap_emul.c b/arch/mips/kvm/trap_emul.c
index c4038d2..6ba0faf 100644
--- a/arch/mips/kvm/trap_emul.c
+++ b/arch/mips/kvm/trap_emul.c
@@ -505,7 +505,8 @@
kvm_write_c0_guest_intctl(cop0, 0xFC000000);
/* Put in vcpu id as CPUNum into Ebase Reg to handle SMP Guests */
- kvm_write_c0_guest_ebase(cop0, KVM_GUEST_KSEG0 | (vcpu_id & 0xFF));
+ kvm_write_c0_guest_ebase(cop0, KVM_GUEST_KSEG0 |
+ (vcpu_id & MIPS_EBASE_CPUNUM));
return 0;
}
@@ -546,7 +547,7 @@
kvm_mips_write_count(vcpu, v);
break;
case KVM_REG_MIPS_CP0_COMPARE:
- kvm_mips_write_compare(vcpu, v);
+ kvm_mips_write_compare(vcpu, v, false);
break;
case KVM_REG_MIPS_CP0_CAUSE:
/*
diff --git a/arch/mips/lantiq/Kconfig b/arch/mips/lantiq/Kconfig
index e10d333..177769d 100644
--- a/arch/mips/lantiq/Kconfig
+++ b/arch/mips/lantiq/Kconfig
@@ -25,7 +25,17 @@
endchoice
choice
- prompt "Devicetree"
+ prompt "Built-in device tree"
+ help
+ Legacy bootloaders do not pass a DTB pointer to the kernel, so
+ if a "wrapper" is not being used, the kernel will need to include
+ a device tree that matches the target board.
+
+ The builtin DTB will only be used if the firmware does not supply
+ a valid DTB.
+
+config LANTIQ_DT_NONE
+ bool "None"
config DT_EASY50712
bool "Easy50712"
diff --git a/arch/mips/lantiq/Makefile b/arch/mips/lantiq/Makefile
index 690257ab..2718652 100644
--- a/arch/mips/lantiq/Makefile
+++ b/arch/mips/lantiq/Makefile
@@ -1,4 +1,4 @@
-# Copyright (C) 2010 John Crispin <blogic@openwrt.org>
+# Copyright (C) 2010 John Crispin <john@phrozen.org>
#
# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License version 2 as published
diff --git a/arch/mips/lantiq/clk.c b/arch/mips/lantiq/clk.c
index a0706fd..149f051 100644
--- a/arch/mips/lantiq/clk.c
+++ b/arch/mips/lantiq/clk.c
@@ -4,7 +4,7 @@
* by the Free Software Foundation.
*
* Copyright (C) 2010 Thomas Langer <thomas.langer@lantiq.com>
- * Copyright (C) 2010 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2010 John Crispin <john@phrozen.org>
*/
#include <linux/io.h>
#include <linux/export.h>
diff --git a/arch/mips/lantiq/clk.h b/arch/mips/lantiq/clk.h
index 7376ce8..e806e04 100644
--- a/arch/mips/lantiq/clk.h
+++ b/arch/mips/lantiq/clk.h
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2010 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2010 John Crispin <john@phrozen.org>
*/
#ifndef _LTQ_CLK_H__
diff --git a/arch/mips/lantiq/early_printk.c b/arch/mips/lantiq/early_printk.c
index 9b28d09..44bccae 100644
--- a/arch/mips/lantiq/early_printk.c
+++ b/arch/mips/lantiq/early_printk.c
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2010 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2010 John Crispin <john@phrozen.org>
*/
#include <linux/cpu.h>
diff --git a/arch/mips/lantiq/falcon/prom.c b/arch/mips/lantiq/falcon/prom.c
index aa94979..75315c0 100644
--- a/arch/mips/lantiq/falcon/prom.c
+++ b/arch/mips/lantiq/falcon/prom.c
@@ -4,7 +4,7 @@
* by the Free Software Foundation.
*
* Copyright (C) 2012 Thomas Langer <thomas.langer@lantiq.com>
- * Copyright (C) 2012 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2012 John Crispin <john@phrozen.org>
*/
#include <linux/kernel.h>
diff --git a/arch/mips/lantiq/falcon/reset.c b/arch/mips/lantiq/falcon/reset.c
index 5682482..7a535d7 100644
--- a/arch/mips/lantiq/falcon/reset.c
+++ b/arch/mips/lantiq/falcon/reset.c
@@ -4,7 +4,7 @@
* by the Free Software Foundation.
*
* Copyright (C) 2012 Thomas Langer <thomas.langer@lantiq.com>
- * Copyright (C) 2012 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2012 John Crispin <john@phrozen.org>
*/
#include <linux/init.h>
diff --git a/arch/mips/lantiq/falcon/sysctrl.c b/arch/mips/lantiq/falcon/sysctrl.c
index 7edcd49..2a1b302 100644
--- a/arch/mips/lantiq/falcon/sysctrl.c
+++ b/arch/mips/lantiq/falcon/sysctrl.c
@@ -4,7 +4,7 @@
* by the Free Software Foundation.
*
* Copyright (C) 2011 Thomas Langer <thomas.langer@lantiq.com>
- * Copyright (C) 2011 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2011 John Crispin <john@phrozen.org>
*/
#include <linux/ioport.h>
diff --git a/arch/mips/lantiq/irq.c b/arch/mips/lantiq/irq.c
index 2e7f60c..ff17669e 100644
--- a/arch/mips/lantiq/irq.c
+++ b/arch/mips/lantiq/irq.c
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2010 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2010 John Crispin <john@phrozen.org>
* Copyright (C) 2010 Thomas Langer <thomas.langer@lantiq.com>
*/
diff --git a/arch/mips/lantiq/prom.c b/arch/mips/lantiq/prom.c
index 297bcaa..5f693ac7 100644
--- a/arch/mips/lantiq/prom.c
+++ b/arch/mips/lantiq/prom.c
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2010 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2010 John Crispin <john@phrozen.org>
*/
#include <linux/export.h>
@@ -65,6 +65,8 @@
void __init plat_mem_setup(void)
{
+ void *dtb;
+
ioport_resource.start = IOPORT_RESOURCE_START;
ioport_resource.end = IOPORT_RESOURCE_END;
iomem_resource.start = IOMEM_RESOURCE_START;
@@ -72,11 +74,18 @@
set_io_port_base((unsigned long) KSEG1);
+ if (fw_arg0 == -2) /* UHI interface */
+ dtb = (void *)fw_arg1;
+ else if (__dtb_start != __dtb_end)
+ dtb = (void *)__dtb_start;
+ else
+ panic("no dtb found");
+
/*
- * Load the builtin devicetree. This causes the chosen node to be
+ * Load the devicetree. This causes the chosen node to be
* parsed resulting in our memory appearing
*/
- __dt_setup_arch(__dtb_start);
+ __dt_setup_arch(dtb);
}
void __init device_tree_init(void)
diff --git a/arch/mips/lantiq/prom.h b/arch/mips/lantiq/prom.h
index bfd2d58..4b6576c 100644
--- a/arch/mips/lantiq/prom.h
+++ b/arch/mips/lantiq/prom.h
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2010 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2010 John Crispin <john@phrozen.org>
*/
#ifndef _LTQ_PROM_H__
diff --git a/arch/mips/lantiq/xway/clk.c b/arch/mips/lantiq/xway/clk.c
index 07f6d5b..41fc30d 100644
--- a/arch/mips/lantiq/xway/clk.c
+++ b/arch/mips/lantiq/xway/clk.c
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2010 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2010 John Crispin <john@phrozen.org>
* Copyright (C) 2013-2015 Lantiq Beteiligungs-GmbH & Co.KG
*/
diff --git a/arch/mips/lantiq/xway/dcdc.c b/arch/mips/lantiq/xway/dcdc.c
index ae8e930..08f7aba 100644
--- a/arch/mips/lantiq/xway/dcdc.c
+++ b/arch/mips/lantiq/xway/dcdc.c
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2012 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2012 John Crispin <john@phrozen.org>
* Copyright (C) 2010 Sameer Ahmad, Lantiq GmbH
*/
diff --git a/arch/mips/lantiq/xway/dma.c b/arch/mips/lantiq/xway/dma.c
index 34a116e..cef8117 100644
--- a/arch/mips/lantiq/xway/dma.c
+++ b/arch/mips/lantiq/xway/dma.c
@@ -12,7 +12,7 @@
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA.
*
- * Copyright (C) 2011 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2011 John Crispin <john@phrozen.org>
*/
#include <linux/init.h>
diff --git a/arch/mips/lantiq/xway/gptu.c b/arch/mips/lantiq/xway/gptu.c
index f1492b2..0f1bbea 100644
--- a/arch/mips/lantiq/xway/gptu.c
+++ b/arch/mips/lantiq/xway/gptu.c
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2012 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2012 John Crispin <john@phrozen.org>
* Copyright (C) 2012 Lantiq GmbH
*/
diff --git a/arch/mips/lantiq/xway/prom.c b/arch/mips/lantiq/xway/prom.c
index 8f6e02f..9475b25 100644
--- a/arch/mips/lantiq/xway/prom.c
+++ b/arch/mips/lantiq/xway/prom.c
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2010 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2010 John Crispin <john@phrozen.org>
* Copyright (C) 2013-2015 Lantiq Beteiligungs-GmbH & Co.KG
*/
diff --git a/arch/mips/lantiq/xway/reset.c b/arch/mips/lantiq/xway/reset.c
index bc29bb3..83fd65d 100644
--- a/arch/mips/lantiq/xway/reset.c
+++ b/arch/mips/lantiq/xway/reset.c
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2010 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2010 John Crispin <john@phrozen.org>
* Copyright (C) 2013-2015 Lantiq Beteiligungs-GmbH & Co.KG
*/
@@ -258,7 +258,7 @@
return ltq_deassert_device(rcdev, id);
}
-static struct reset_control_ops reset_ops = {
+static const struct reset_control_ops reset_ops = {
.reset = ltq_reset_device,
.assert = ltq_assert_device,
.deassert = ltq_deassert_device,
diff --git a/arch/mips/lantiq/xway/sysctrl.c b/arch/mips/lantiq/xway/sysctrl.c
index 80554e8..236193b 100644
--- a/arch/mips/lantiq/xway/sysctrl.c
+++ b/arch/mips/lantiq/xway/sysctrl.c
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2011-2012 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2011-2012 John Crispin <john@phrozen.org>
* Copyright (C) 2013-2015 Lantiq Beteiligungs-GmbH & Co.KG
*/
diff --git a/arch/mips/lantiq/xway/vmmc.c b/arch/mips/lantiq/xway/vmmc.c
index d001bc3..4625495 100644
--- a/arch/mips/lantiq/xway/vmmc.c
+++ b/arch/mips/lantiq/xway/vmmc.c
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2012 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2012 John Crispin <john@phrozen.org>
*/
#include <linux/module.h>
diff --git a/arch/mips/lantiq/xway/xrx200_phy_fw.c b/arch/mips/lantiq/xway/xrx200_phy_fw.c
index 199094a..71e518c 100644
--- a/arch/mips/lantiq/xway/xrx200_phy_fw.c
+++ b/arch/mips/lantiq/xway/xrx200_phy_fw.c
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2012 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2012 John Crispin <john@phrozen.org>
*/
#include <linux/delay.h>
@@ -112,6 +112,6 @@
module_platform_driver(xway_phy_driver);
-MODULE_AUTHOR("John Crispin <blogic@openwrt.org>");
+MODULE_AUTHOR("John Crispin <john@phrozen.org>");
MODULE_DESCRIPTION("Lantiq XRX200 PHY Firmware Loader");
MODULE_LICENSE("GPL");
diff --git a/arch/mips/lib/dump_tlb.c b/arch/mips/lib/dump_tlb.c
index 92a3731..0f80b93 100644
--- a/arch/mips/lib/dump_tlb.c
+++ b/arch/mips/lib/dump_tlb.c
@@ -19,6 +19,8 @@
pr_info("Index : %0x\n", read_c0_index());
pr_info("PageMask : %0x\n", read_c0_pagemask());
+ if (cpu_has_guestid)
+ pr_info("GuestCtl1: %0x\n", read_c0_guestctl1());
pr_info("EntryHi : %0*lx\n", field, read_c0_entryhi());
pr_info("EntryLo0 : %0*lx\n", field, read_c0_entrylo0());
pr_info("EntryLo1 : %0*lx\n", field, read_c0_entrylo1());
@@ -72,7 +74,10 @@
{
unsigned long s_entryhi, entryhi, asid;
unsigned long long entrylo0, entrylo1, pa;
- unsigned int s_index, s_pagemask, pagemask, c0, c1, i;
+ unsigned int s_index, s_pagemask, s_guestctl1 = 0;
+ unsigned int pagemask, guestctl1 = 0, c0, c1, i;
+ unsigned long asidmask = cpu_asid_mask(¤t_cpu_data);
+ int asidwidth = DIV_ROUND_UP(ilog2(asidmask) + 1, 4);
#ifdef CONFIG_32BIT
bool xpa = cpu_has_xpa && (read_c0_pagegrain() & PG_ELPA);
int pwidth = xpa ? 11 : 8;
@@ -86,7 +91,9 @@
s_pagemask = read_c0_pagemask();
s_entryhi = read_c0_entryhi();
s_index = read_c0_index();
- asid = s_entryhi & 0xff;
+ asid = s_entryhi & asidmask;
+ if (cpu_has_guestid)
+ s_guestctl1 = read_c0_guestctl1();
for (i = first; i <= last; i++) {
write_c0_index(i);
@@ -97,6 +104,8 @@
entryhi = read_c0_entryhi();
entrylo0 = read_c0_entrylo0();
entrylo1 = read_c0_entrylo1();
+ if (cpu_has_guestid)
+ guestctl1 = read_c0_guestctl1();
/* EHINV bit marks entire entry as invalid */
if (cpu_has_tlbinv && entryhi & MIPS_ENTRYHI_EHINV)
@@ -115,7 +124,7 @@
* due to duplicate TLB entry.
*/
if (!((entrylo0 | entrylo1) & ENTRYLO_G) &&
- (entryhi & 0xff) != asid)
+ (entryhi & asidmask) != asid)
continue;
/*
@@ -126,15 +135,19 @@
c0 = (entrylo0 & ENTRYLO_C) >> ENTRYLO_C_SHIFT;
c1 = (entrylo1 & ENTRYLO_C) >> ENTRYLO_C_SHIFT;
- printk("va=%0*lx asid=%02lx\n",
+ printk("va=%0*lx asid=%0*lx",
vwidth, (entryhi & ~0x1fffUL),
- entryhi & 0xff);
+ asidwidth, entryhi & asidmask);
+ if (cpu_has_guestid)
+ printk(" gid=%02lx",
+ (guestctl1 & MIPS_GCTL1_RID)
+ >> MIPS_GCTL1_RID_SHIFT);
/* RI/XI are in awkward places, so mask them off separately */
pa = entrylo0 & ~(MIPS_ENTRYLO_RI | MIPS_ENTRYLO_XI);
if (xpa)
pa |= (unsigned long long)readx_c0_entrylo0() << 30;
pa = (pa << 6) & PAGE_MASK;
- printk("\t[");
+ printk("\n\t[");
if (cpu_has_rixi)
printk("ri=%d xi=%d ",
(entrylo0 & MIPS_ENTRYLO_RI) ? 1 : 0,
@@ -164,6 +177,8 @@
write_c0_entryhi(s_entryhi);
write_c0_index(s_index);
write_c0_pagemask(s_pagemask);
+ if (cpu_has_guestid)
+ write_c0_guestctl1(s_guestctl1);
}
void dump_tlb_all(void)
diff --git a/arch/mips/lib/memset.S b/arch/mips/lib/memset.S
index 8f0019a..18a1ccd 100644
--- a/arch/mips/lib/memset.S
+++ b/arch/mips/lib/memset.S
@@ -228,10 +228,12 @@
.hidden __memset
.endif
+#ifdef CONFIG_CPU_MIPSR6
.Lbyte_fixup\@:
PTR_SUBU a2, $0, t0
jr ra
PTR_ADDIU a2, 1
+#endif /* CONFIG_CPU_MIPSR6 */
.Lfirst_fixup\@:
jr ra
diff --git a/arch/mips/lib/r3k_dump_tlb.c b/arch/mips/lib/r3k_dump_tlb.c
index cfcbb52..744f4a7 100644
--- a/arch/mips/lib/r3k_dump_tlb.c
+++ b/arch/mips/lib/r3k_dump_tlb.c
@@ -29,9 +29,10 @@
{
int i;
unsigned int asid;
- unsigned long entryhi, entrylo0;
+ unsigned long entryhi, entrylo0, asid_mask;
- asid = read_c0_entryhi() & ASID_MASK;
+ asid_mask = cpu_asid_mask(¤t_cpu_data);
+ asid = read_c0_entryhi() & asid_mask;
for (i = first; i <= last; i++) {
write_c0_index(i<<8);
@@ -46,7 +47,7 @@
/* Unused entries have a virtual address of KSEG0. */
if ((entryhi & PAGE_MASK) != KSEG0 &&
(entrylo0 & R3K_ENTRYLO_G ||
- (entryhi & ASID_MASK) == asid)) {
+ (entryhi & asid_mask) == asid)) {
/*
* Only print entries in use
*/
@@ -55,7 +56,7 @@
printk("va=%08lx asid=%08lx"
" [pa=%06lx n=%d d=%d v=%d g=%d]",
entryhi & PAGE_MASK,
- entryhi & ASID_MASK,
+ entryhi & asid_mask,
entrylo0 & PAGE_MASK,
(entrylo0 & R3K_ENTRYLO_N) ? 1 : 0,
(entrylo0 & R3K_ENTRYLO_D) ? 1 : 0,
diff --git a/arch/mips/loongson32/common/platform.c b/arch/mips/loongson32/common/platform.c
index ddf1d4c..f2c714d 100644
--- a/arch/mips/loongson32/common/platform.c
+++ b/arch/mips/loongson32/common/platform.c
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 2011 Zhang, Keguang <keguang.zhang@gmail.com>
+ * Copyright (c) 2011-2016 Zhang, Keguang <keguang.zhang@gmail.com>
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
@@ -10,14 +10,17 @@
#include <linux/clk.h>
#include <linux/dma-mapping.h>
#include <linux/err.h>
+#include <linux/mtd/partitions.h>
+#include <linux/sizes.h>
#include <linux/phy.h>
#include <linux/serial_8250.h>
#include <linux/stmmac.h>
#include <linux/usb/ehci_pdriver.h>
-#include <asm-generic/sizes.h>
-#include <cpufreq.h>
#include <loongson1.h>
+#include <cpufreq.h>
+#include <dma.h>
+#include <nand.h>
/* 8250/16550 compatible UART */
#define LS1X_UART(_id) \
@@ -45,7 +48,7 @@
},
};
-void __init ls1x_serial_setup(struct platform_device *pdev)
+void __init ls1x_serial_set_uartclk(struct platform_device *pdev)
{
struct clk *clk;
struct plat_serial8250_port *p;
@@ -77,6 +80,42 @@
},
};
+/* DMA */
+static struct resource ls1x_dma_resources[] = {
+ [0] = {
+ .start = LS1X_DMAC_BASE,
+ .end = LS1X_DMAC_BASE + SZ_4 - 1,
+ .flags = IORESOURCE_MEM,
+ },
+ [1] = {
+ .start = LS1X_DMA0_IRQ,
+ .end = LS1X_DMA0_IRQ,
+ .flags = IORESOURCE_IRQ,
+ },
+ [2] = {
+ .start = LS1X_DMA1_IRQ,
+ .end = LS1X_DMA1_IRQ,
+ .flags = IORESOURCE_IRQ,
+ },
+ [3] = {
+ .start = LS1X_DMA2_IRQ,
+ .end = LS1X_DMA2_IRQ,
+ .flags = IORESOURCE_IRQ,
+ },
+};
+
+struct platform_device ls1x_dma_pdev = {
+ .name = "ls1x-dma",
+ .id = -1,
+ .num_resources = ARRAY_SIZE(ls1x_dma_resources),
+ .resource = ls1x_dma_resources,
+};
+
+void __init ls1x_dma_set_platdata(struct plat_ls1x_dma *pdata)
+{
+ ls1x_dma_pdev.dev.platform_data = pdata;
+}
+
/* Synopsys Ethernet GMAC */
static struct stmmac_mdio_bus_data ls1x_mdio_bus_data = {
.phy_mask = 0,
@@ -198,6 +237,64 @@
},
};
+/* GPIO */
+static struct resource ls1x_gpio0_resources[] = {
+ [0] = {
+ .start = LS1X_GPIO0_BASE,
+ .end = LS1X_GPIO0_BASE + SZ_4 - 1,
+ .flags = IORESOURCE_MEM,
+ },
+};
+
+struct platform_device ls1x_gpio0_pdev = {
+ .name = "ls1x-gpio",
+ .id = 0,
+ .num_resources = ARRAY_SIZE(ls1x_gpio0_resources),
+ .resource = ls1x_gpio0_resources,
+};
+
+static struct resource ls1x_gpio1_resources[] = {
+ [0] = {
+ .start = LS1X_GPIO1_BASE,
+ .end = LS1X_GPIO1_BASE + SZ_4 - 1,
+ .flags = IORESOURCE_MEM,
+ },
+};
+
+struct platform_device ls1x_gpio1_pdev = {
+ .name = "ls1x-gpio",
+ .id = 1,
+ .num_resources = ARRAY_SIZE(ls1x_gpio1_resources),
+ .resource = ls1x_gpio1_resources,
+};
+
+/* NAND Flash */
+static struct resource ls1x_nand_resources[] = {
+ [0] = {
+ .start = LS1X_NAND_BASE,
+ .end = LS1X_NAND_BASE + SZ_32 - 1,
+ .flags = IORESOURCE_MEM,
+ },
+ [1] = {
+ /* DMA channel 0 is dedicated to NAND */
+ .start = LS1X_DMA_CHANNEL0,
+ .end = LS1X_DMA_CHANNEL0,
+ .flags = IORESOURCE_DMA,
+ },
+};
+
+struct platform_device ls1x_nand_pdev = {
+ .name = "ls1x-nand",
+ .id = -1,
+ .num_resources = ARRAY_SIZE(ls1x_nand_resources),
+ .resource = ls1x_nand_resources,
+};
+
+void __init ls1x_nand_set_platdata(struct plat_ls1x_nand *pdata)
+{
+ ls1x_nand_pdev.dev.platform_data = pdata;
+}
+
/* USB EHCI */
static u64 ls1x_ehci_dmamask = DMA_BIT_MASK(32);
diff --git a/arch/mips/loongson32/common/reset.c b/arch/mips/loongson32/common/reset.c
index c41e4ca..8a1d9cc 100644
--- a/arch/mips/loongson32/common/reset.c
+++ b/arch/mips/loongson32/common/reset.c
@@ -9,12 +9,13 @@
#include <linux/io.h>
#include <linux/pm.h>
+#include <linux/sizes.h>
#include <asm/idle.h>
#include <asm/reboot.h>
#include <loongson1.h>
-static void __iomem *wdt_base;
+static void __iomem *wdt_reg_base;
static void ls1x_halt(void)
{
@@ -26,9 +27,9 @@
static void ls1x_restart(char *command)
{
- __raw_writel(0x1, wdt_base + WDT_EN);
- __raw_writel(0x1, wdt_base + WDT_TIMER);
- __raw_writel(0x1, wdt_base + WDT_SET);
+ __raw_writel(0x1, wdt_reg_base + WDT_EN);
+ __raw_writel(0x1, wdt_reg_base + WDT_TIMER);
+ __raw_writel(0x1, wdt_reg_base + WDT_SET);
ls1x_halt();
}
@@ -40,8 +41,8 @@
static int __init ls1x_reboot_setup(void)
{
- wdt_base = ioremap_nocache(LS1X_WDT_BASE, 0x0f);
- if (!wdt_base)
+ wdt_reg_base = ioremap_nocache(LS1X_WDT_BASE, (SZ_4 + SZ_8));
+ if (!wdt_reg_base)
panic("Failed to remap watchdog registers");
_machine_restart = ls1x_restart;
diff --git a/arch/mips/loongson32/common/time.c b/arch/mips/loongson32/common/time.c
index 0996b02..ff224f0 100644
--- a/arch/mips/loongson32/common/time.c
+++ b/arch/mips/loongson32/common/time.c
@@ -9,6 +9,7 @@
#include <linux/clk.h>
#include <linux/interrupt.h>
+#include <linux/sizes.h>
#include <asm/time.h>
#include <loongson1.h>
@@ -35,25 +36,25 @@
DEFINE_RAW_SPINLOCK(ls1x_timer_lock);
-static void __iomem *timer_base;
+static void __iomem *timer_reg_base;
static uint32_t ls1x_jiffies_per_tick;
static inline void ls1x_pwmtimer_set_period(uint32_t period)
{
- __raw_writel(period, timer_base + PWM_HRC);
- __raw_writel(period, timer_base + PWM_LRC);
+ __raw_writel(period, timer_reg_base + PWM_HRC);
+ __raw_writel(period, timer_reg_base + PWM_LRC);
}
static inline void ls1x_pwmtimer_restart(void)
{
- __raw_writel(0x0, timer_base + PWM_CNT);
- __raw_writel(INT_EN | CNT_EN, timer_base + PWM_CTRL);
+ __raw_writel(0x0, timer_reg_base + PWM_CNT);
+ __raw_writel(INT_EN | CNT_EN, timer_reg_base + PWM_CTRL);
}
void __init ls1x_pwmtimer_init(void)
{
- timer_base = ioremap(LS1X_TIMER_BASE, 0xf);
- if (!timer_base)
+ timer_reg_base = ioremap_nocache(LS1X_TIMER_BASE, SZ_16);
+ if (!timer_reg_base)
panic("Failed to remap timer registers");
ls1x_jiffies_per_tick = DIV_ROUND_CLOSEST(mips_hpt_frequency, HZ);
@@ -86,7 +87,7 @@
*/
jifs = jiffies;
/* read the count */
- count = __raw_readl(timer_base + PWM_CNT);
+ count = __raw_readl(timer_reg_base + PWM_CNT);
/*
* It's possible for count to appear to go the wrong way for this
@@ -131,7 +132,7 @@
raw_spin_lock(&ls1x_timer_lock);
ls1x_pwmtimer_set_period(ls1x_jiffies_per_tick);
ls1x_pwmtimer_restart();
- __raw_writel(INT_EN | CNT_EN, timer_base + PWM_CTRL);
+ __raw_writel(INT_EN | CNT_EN, timer_reg_base + PWM_CTRL);
raw_spin_unlock(&ls1x_timer_lock);
return 0;
@@ -140,7 +141,7 @@
static int ls1x_clockevent_tick_resume(struct clock_event_device *cd)
{
raw_spin_lock(&ls1x_timer_lock);
- __raw_writel(INT_EN | CNT_EN, timer_base + PWM_CTRL);
+ __raw_writel(INT_EN | CNT_EN, timer_reg_base + PWM_CTRL);
raw_spin_unlock(&ls1x_timer_lock);
return 0;
@@ -149,8 +150,8 @@
static int ls1x_clockevent_set_state_shutdown(struct clock_event_device *cd)
{
raw_spin_lock(&ls1x_timer_lock);
- __raw_writel(__raw_readl(timer_base + PWM_CTRL) & ~CNT_EN,
- timer_base + PWM_CTRL);
+ __raw_writel(__raw_readl(timer_reg_base + PWM_CTRL) & ~CNT_EN,
+ timer_reg_base + PWM_CTRL);
raw_spin_unlock(&ls1x_timer_lock);
return 0;
@@ -220,7 +221,7 @@
#ifdef CONFIG_CEVT_CSRC_LS1X
/* setup LS1X PWM timer */
- clk = clk_get(NULL, "ls1x_pwmtimer");
+ clk = clk_get(NULL, "ls1x-pwmtimer");
if (IS_ERR(clk))
panic("unable to get timer clock, err=%ld", PTR_ERR(clk));
diff --git a/arch/mips/loongson32/ls1b/board.c b/arch/mips/loongson32/ls1b/board.c
index 58daeea..38a1d40 100644
--- a/arch/mips/loongson32/ls1b/board.c
+++ b/arch/mips/loongson32/ls1b/board.c
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 2011 Zhang, Keguang <keguang.zhang@gmail.com>
+ * Copyright (c) 2011-2016 Zhang, Keguang <keguang.zhang@gmail.com>
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
@@ -7,26 +7,83 @@
* option) any later version.
*/
+#include <linux/leds.h>
+#include <linux/mtd/partitions.h>
+#include <linux/sizes.h>
+
+#include <loongson1.h>
+#include <dma.h>
+#include <nand.h>
#include <platform.h>
+struct plat_ls1x_dma ls1x_dma_pdata = {
+ .nr_channels = 3,
+};
+
+static struct mtd_partition ls1x_nand_parts[] = {
+ {
+ .name = "kernel",
+ .offset = 0,
+ .size = SZ_16M,
+ },
+ {
+ .name = "rootfs",
+ .offset = MTDPART_OFS_APPEND,
+ .size = MTDPART_SIZ_FULL,
+ },
+};
+
+struct plat_ls1x_nand ls1x_nand_pdata = {
+ .parts = ls1x_nand_parts,
+ .nr_parts = ARRAY_SIZE(ls1x_nand_parts),
+ .hold_cycle = 0x2,
+ .wait_cycle = 0xc,
+};
+
+static const struct gpio_led ls1x_gpio_leds[] __initconst = {
+ {
+ .name = "LED9",
+ .default_trigger = "heartbeat",
+ .gpio = 38,
+ .active_low = 1,
+ .default_state = LEDS_GPIO_DEFSTATE_OFF,
+ }, {
+ .name = "LED6",
+ .default_trigger = "nand-disk",
+ .gpio = 39,
+ .active_low = 1,
+ .default_state = LEDS_GPIO_DEFSTATE_OFF,
+ },
+};
+
+static const struct gpio_led_platform_data ls1x_led_pdata __initconst = {
+ .num_leds = ARRAY_SIZE(ls1x_gpio_leds),
+ .leds = ls1x_gpio_leds,
+};
+
static struct platform_device *ls1b_platform_devices[] __initdata = {
&ls1x_uart_pdev,
&ls1x_cpufreq_pdev,
+ &ls1x_dma_pdev,
&ls1x_eth0_pdev,
&ls1x_eth1_pdev,
&ls1x_ehci_pdev,
+ &ls1x_gpio0_pdev,
+ &ls1x_gpio1_pdev,
+ &ls1x_nand_pdev,
&ls1x_rtc_pdev,
};
static int __init ls1b_platform_init(void)
{
- int err;
+ ls1x_serial_set_uartclk(&ls1x_uart_pdev);
+ ls1x_dma_set_platdata(&ls1x_dma_pdata);
+ ls1x_nand_set_platdata(&ls1x_nand_pdata);
- ls1x_serial_setup(&ls1x_uart_pdev);
+ gpio_led_register_device(-1, &ls1x_led_pdata);
- err = platform_add_devices(ls1b_platform_devices,
+ return platform_add_devices(ls1b_platform_devices,
ARRAY_SIZE(ls1b_platform_devices));
- return err;
}
arch_initcall(ls1b_platform_init);
diff --git a/arch/mips/loongson64/Platform b/arch/mips/loongson64/Platform
index 85d8089..0fce460 100644
--- a/arch/mips/loongson64/Platform
+++ b/arch/mips/loongson64/Platform
@@ -31,7 +31,7 @@
# can't easily be used safely within the kbuild framework.
#
ifeq ($(call cc-ifversion, -ge, 0409, y), y)
- ifeq ($(call ld-ifversion, -ge, 22500000, y), y)
+ ifeq ($(call ld-ifversion, -ge, 225000000, y), y)
cflags-$(CONFIG_CPU_LOONGSON3) += \
$(call cc-option,-march=loongson3a -U_MIPS_ISA -D_MIPS_ISA=_MIPS_ISA_MIPS64)
else
diff --git a/arch/mips/loongson64/common/env.c b/arch/mips/loongson64/common/env.c
index d6d07ad..57d590a 100644
--- a/arch/mips/loongson64/common/env.c
+++ b/arch/mips/loongson64/common/env.c
@@ -105,6 +105,10 @@
loongson_chiptemp[1] = 0x900010001fe0019c;
loongson_chiptemp[2] = 0x900020001fe0019c;
loongson_chiptemp[3] = 0x900030001fe0019c;
+ loongson_freqctrl[0] = 0x900000001fe001d0;
+ loongson_freqctrl[1] = 0x900010001fe001d0;
+ loongson_freqctrl[2] = 0x900020001fe001d0;
+ loongson_freqctrl[3] = 0x900030001fe001d0;
loongson_sysconf.ht_control_base = 0x90000EFDFB000000;
loongson_sysconf.workarounds = WORKAROUND_CPUFREQ;
} else if (ecpu->cputype == Loongson_3B) {
@@ -187,7 +191,8 @@
case PRID_REV_LOONGSON2F:
cpu_clock_freq = 797000000;
break;
- case PRID_REV_LOONGSON3A:
+ case PRID_REV_LOONGSON3A_R1:
+ case PRID_REV_LOONGSON3A_R2:
cpu_clock_freq = 900000000;
break;
case PRID_REV_LOONGSON3B_R1:
diff --git a/arch/mips/loongson64/loongson-3/Makefile b/arch/mips/loongson64/loongson-3/Makefile
index 622fead..44bc148 100644
--- a/arch/mips/loongson64/loongson-3/Makefile
+++ b/arch/mips/loongson64/loongson-3/Makefile
@@ -1,7 +1,7 @@
#
# Makefile for Loongson-3 family machines
#
-obj-y += irq.o cop2-ex.o platform.o
+obj-y += irq.o cop2-ex.o platform.o acpi_init.o
obj-$(CONFIG_SMP) += smp.o
diff --git a/drivers/platform/mips/acpi_init.c b/arch/mips/loongson64/loongson-3/acpi_init.c
similarity index 100%
rename from drivers/platform/mips/acpi_init.c
rename to arch/mips/loongson64/loongson-3/acpi_init.c
diff --git a/arch/mips/loongson64/loongson-3/irq.c b/arch/mips/loongson64/loongson-3/irq.c
index 0f75b6b..8e76490 100644
--- a/arch/mips/loongson64/loongson-3/irq.c
+++ b/arch/mips/loongson64/loongson-3/irq.c
@@ -24,19 +24,21 @@
}
}
+#define UNUSED_IPS (CAUSEF_IP5 | CAUSEF_IP4 | CAUSEF_IP1 | CAUSEF_IP0)
+
void mach_irq_dispatch(unsigned int pending)
{
if (pending & CAUSEF_IP7)
do_IRQ(LOONGSON_TIMER_IRQ);
#if defined(CONFIG_SMP)
- else if (pending & CAUSEF_IP6)
+ if (pending & CAUSEF_IP6)
loongson3_ipi_interrupt(NULL);
#endif
- else if (pending & CAUSEF_IP3)
+ if (pending & CAUSEF_IP3)
ht_irqdispatch();
- else if (pending & CAUSEF_IP2)
+ if (pending & CAUSEF_IP2)
do_IRQ(LOONGSON_UART_IRQ);
- else {
+ if (pending & UNUSED_IPS) {
pr_err("%s : spurious interrupt\n", __func__);
spurious_interrupt();
}
diff --git a/arch/mips/loongson64/loongson-3/numa.c b/arch/mips/loongson64/loongson-3/numa.c
index 6f9e010..282c5a8 100644
--- a/arch/mips/loongson64/loongson-3/numa.c
+++ b/arch/mips/loongson64/loongson-3/numa.c
@@ -213,10 +213,10 @@
BOOTMEM_DEFAULT);
if (node == 0 && node_end_pfn(0) >= (0xffffffff >> PAGE_SHIFT)) {
- /* Reserve 0xff800000~0xffffffff for RS780E integrated GPU */
+ /* Reserve 0xfe000000~0xffffffff for RS780E integrated GPU */
reserve_bootmem_node(NODE_DATA(node),
- (node_addrspace_offset | 0xff800000),
- 8 << 20, BOOTMEM_DEFAULT);
+ (node_addrspace_offset | 0xfe000000),
+ 32 << 20, BOOTMEM_DEFAULT);
}
sparse_memory_present_with_active_regions(node);
diff --git a/arch/mips/loongson64/loongson-3/smp.c b/arch/mips/loongson64/loongson-3/smp.c
index 509832a9..e59759a 100644
--- a/arch/mips/loongson64/loongson-3/smp.c
+++ b/arch/mips/loongson64/loongson-3/smp.c
@@ -421,7 +421,6 @@
local_irq_save(flags);
fixup_irqs();
local_irq_restore(flags);
- flush_cache_all();
local_flush_tlb_all();
return 0;
@@ -440,7 +439,7 @@
* flush all L1 entries at first. Then, another core (usually Core 0) can
* safely disable the clock of the target core. loongson3_play_dead() is
* called via CKSEG1 (uncached and unmmaped) */
-static void loongson3a_play_dead(int *state_addr)
+static void loongson3a_r1_play_dead(int *state_addr)
{
register int val;
register long cpuid, core, node, count;
@@ -502,6 +501,89 @@
: "a1");
}
+static void loongson3a_r2_play_dead(int *state_addr)
+{
+ register int val;
+ register long cpuid, core, node, count;
+ register void *addr, *base, *initfunc;
+
+ __asm__ __volatile__(
+ " .set push \n"
+ " .set noreorder \n"
+ " li %[addr], 0x80000000 \n" /* KSEG0 */
+ "1: cache 0, 0(%[addr]) \n" /* flush L1 ICache */
+ " cache 0, 1(%[addr]) \n"
+ " cache 0, 2(%[addr]) \n"
+ " cache 0, 3(%[addr]) \n"
+ " cache 1, 0(%[addr]) \n" /* flush L1 DCache */
+ " cache 1, 1(%[addr]) \n"
+ " cache 1, 2(%[addr]) \n"
+ " cache 1, 3(%[addr]) \n"
+ " addiu %[sets], %[sets], -1 \n"
+ " bnez %[sets], 1b \n"
+ " addiu %[addr], %[addr], 0x40 \n"
+ " li %[addr], 0x80000000 \n" /* KSEG0 */
+ "2: cache 2, 0(%[addr]) \n" /* flush L1 VCache */
+ " cache 2, 1(%[addr]) \n"
+ " cache 2, 2(%[addr]) \n"
+ " cache 2, 3(%[addr]) \n"
+ " cache 2, 4(%[addr]) \n"
+ " cache 2, 5(%[addr]) \n"
+ " cache 2, 6(%[addr]) \n"
+ " cache 2, 7(%[addr]) \n"
+ " cache 2, 8(%[addr]) \n"
+ " cache 2, 9(%[addr]) \n"
+ " cache 2, 10(%[addr]) \n"
+ " cache 2, 11(%[addr]) \n"
+ " cache 2, 12(%[addr]) \n"
+ " cache 2, 13(%[addr]) \n"
+ " cache 2, 14(%[addr]) \n"
+ " cache 2, 15(%[addr]) \n"
+ " addiu %[vsets], %[vsets], -1 \n"
+ " bnez %[vsets], 2b \n"
+ " addiu %[addr], %[addr], 0x40 \n"
+ " li %[val], 0x7 \n" /* *state_addr = CPU_DEAD; */
+ " sw %[val], (%[state_addr]) \n"
+ " sync \n"
+ " cache 21, (%[state_addr]) \n" /* flush entry of *state_addr */
+ " .set pop \n"
+ : [addr] "=&r" (addr), [val] "=&r" (val)
+ : [state_addr] "r" (state_addr),
+ [sets] "r" (cpu_data[smp_processor_id()].dcache.sets),
+ [vsets] "r" (cpu_data[smp_processor_id()].vcache.sets));
+
+ __asm__ __volatile__(
+ " .set push \n"
+ " .set noreorder \n"
+ " .set mips64 \n"
+ " mfc0 %[cpuid], $15, 1 \n"
+ " andi %[cpuid], 0x3ff \n"
+ " dli %[base], 0x900000003ff01000 \n"
+ " andi %[core], %[cpuid], 0x3 \n"
+ " sll %[core], 8 \n" /* get core id */
+ " or %[base], %[base], %[core] \n"
+ " andi %[node], %[cpuid], 0xc \n"
+ " dsll %[node], 42 \n" /* get node id */
+ " or %[base], %[base], %[node] \n"
+ "1: li %[count], 0x100 \n" /* wait for init loop */
+ "2: bnez %[count], 2b \n" /* limit mailbox access */
+ " addiu %[count], -1 \n"
+ " ld %[initfunc], 0x20(%[base]) \n" /* get PC via mailbox */
+ " beqz %[initfunc], 1b \n"
+ " nop \n"
+ " ld $sp, 0x28(%[base]) \n" /* get SP via mailbox */
+ " ld $gp, 0x30(%[base]) \n" /* get GP via mailbox */
+ " ld $a1, 0x38(%[base]) \n"
+ " jr %[initfunc] \n" /* jump to initial PC */
+ " nop \n"
+ " .set pop \n"
+ : [core] "=&r" (core), [node] "=&r" (node),
+ [base] "=&r" (base), [cpuid] "=&r" (cpuid),
+ [count] "=&r" (count), [initfunc] "=&r" (initfunc)
+ : /* No Input */
+ : "a1");
+}
+
static void loongson3b_play_dead(int *state_addr)
{
register int val;
@@ -573,13 +655,18 @@
void (*play_dead_at_ckseg1)(int *);
idle_task_exit();
- switch (loongson_sysconf.cputype) {
- case Loongson_3A:
+ switch (read_c0_prid() & PRID_REV_MASK) {
+ case PRID_REV_LOONGSON3A_R1:
default:
play_dead_at_ckseg1 =
- (void *)CKSEG1ADDR((unsigned long)loongson3a_play_dead);
+ (void *)CKSEG1ADDR((unsigned long)loongson3a_r1_play_dead);
break;
- case Loongson_3B:
+ case PRID_REV_LOONGSON3A_R2:
+ play_dead_at_ckseg1 =
+ (void *)CKSEG1ADDR((unsigned long)loongson3a_r2_play_dead);
+ break;
+ case PRID_REV_LOONGSON3B_R1:
+ case PRID_REV_LOONGSON3B_R2:
play_dead_at_ckseg1 =
(void *)CKSEG1ADDR((unsigned long)loongson3b_play_dead);
break;
@@ -594,9 +681,9 @@
uint64_t core_id = cpu_data[cpu].core;
uint64_t package_id = cpu_data[cpu].package;
- if (loongson_sysconf.cputype == Loongson_3A) {
+ if ((read_c0_prid() & PRID_REV_MASK) == PRID_REV_LOONGSON3A_R1) {
LOONGSON_CHIPCFG(package_id) &= ~(1 << (12 + core_id));
- } else if (loongson_sysconf.cputype == Loongson_3B) {
+ } else {
if (!(loongson_sysconf.workarounds & WORKAROUND_CPUHOTPLUG))
LOONGSON_FREQCTRL(package_id) &= ~(1 << (core_id * 4 + 3));
}
@@ -607,9 +694,9 @@
uint64_t core_id = cpu_data[cpu].core;
uint64_t package_id = cpu_data[cpu].package;
- if (loongson_sysconf.cputype == Loongson_3A) {
+ if ((read_c0_prid() & PRID_REV_MASK) == PRID_REV_LOONGSON3A_R1) {
LOONGSON_CHIPCFG(package_id) |= 1 << (12 + core_id);
- } else if (loongson_sysconf.cputype == Loongson_3B) {
+ } else {
if (!(loongson_sysconf.workarounds & WORKAROUND_CPUHOTPLUG))
LOONGSON_FREQCTRL(package_id) |= 1 << (core_id * 4 + 3);
}
diff --git a/arch/mips/math-emu/Makefile b/arch/mips/math-emu/Makefile
index a19641d..e9bbc2a 100644
--- a/arch/mips/math-emu/Makefile
+++ b/arch/mips/math-emu/Makefile
@@ -4,9 +4,9 @@
obj-y += cp1emu.o ieee754dp.o ieee754sp.o ieee754.o \
dp_div.o dp_mul.o dp_sub.o dp_add.o dp_fsp.o dp_cmp.o dp_simple.o \
- dp_tint.o dp_fint.o dp_maddf.o dp_msubf.o dp_2008class.o dp_fmin.o dp_fmax.o \
+ dp_tint.o dp_fint.o dp_maddf.o dp_2008class.o dp_fmin.o dp_fmax.o \
sp_div.o sp_mul.o sp_sub.o sp_add.o sp_fdp.o sp_cmp.o sp_simple.o \
- sp_tint.o sp_fint.o sp_maddf.o sp_msubf.o sp_2008class.o sp_fmin.o sp_fmax.o \
+ sp_tint.o sp_fint.o sp_maddf.o sp_2008class.o sp_fmin.o sp_fmax.o \
dsemul.o
lib-y += ieee754d.o \
diff --git a/arch/mips/math-emu/cp1emu.c b/arch/mips/math-emu/cp1emu.c
index cdfd44f..d96e912b 100644
--- a/arch/mips/math-emu/cp1emu.c
+++ b/arch/mips/math-emu/cp1emu.c
@@ -445,9 +445,11 @@
case spec_op:
switch (insn.r_format.func) {
case jalr_op:
- regs->regs[insn.r_format.rd] =
- regs->cp0_epc + dec_insn.pc_inc +
- dec_insn.next_pc_inc;
+ if (insn.r_format.rd != 0) {
+ regs->regs[insn.r_format.rd] =
+ regs->cp0_epc + dec_insn.pc_inc +
+ dec_insn.next_pc_inc;
+ }
/* Fall through */
case jr_op:
/* For R6, JR already emulated in jalr_op */
@@ -973,9 +975,10 @@
struct mm_decoded_insn dec_insn, void *__user *fault_addr)
{
unsigned long contpc = xcp->cp0_epc + dec_insn.pc_inc;
- unsigned int cond, cbit;
+ unsigned int cond, cbit, bit0;
mips_instruction ir;
int likely, pc_inc;
+ union fpureg *fpr;
u32 __user *wva;
u64 __user *dva;
u32 wval;
@@ -1187,14 +1190,14 @@
return SIGILL;
cond = likely = 0;
+ fpr = ¤t->thread.fpu.fpr[MIPSInst_RT(ir)];
+ bit0 = get_fpr32(fpr, 0) & 0x1;
switch (MIPSInst_RS(ir)) {
case bc1eqz_op:
- if (get_fpr32(¤t->thread.fpu.fpr[MIPSInst_RT(ir)], 0) & 0x1)
- cond = 1;
+ cond = bit0 == 0;
break;
case bc1nez_op:
- if (!(get_fpr32(¤t->thread.fpu.fpr[MIPSInst_RT(ir)], 0) & 0x1))
- cond = 1;
+ cond = bit0 != 0;
break;
}
goto branch_common;
@@ -1674,7 +1677,7 @@
union ieee754sp(*b) (union ieee754sp, union ieee754sp);
union ieee754sp(*u) (union ieee754sp);
} handler;
- union ieee754sp fs, ft;
+ union ieee754sp fd, fs, ft;
switch (MIPSInst_FUNC(ir)) {
/* binary ops */
@@ -1945,6 +1948,17 @@
rfmt = w_fmt;
goto copcsr;
+ case fsel_op:
+ if (!cpu_has_mips_r6)
+ return SIGILL;
+
+ SPFROMREG(fd, MIPSInst_FD(ir));
+ if (fd.bits & 0x1)
+ SPFROMREG(rv.s, MIPSInst_FT(ir));
+ else
+ SPFROMREG(rv.s, MIPSInst_FS(ir));
+ break;
+
case fcvtl_op:
if (!cpu_has_mips_3_4_5_64_r2_r6)
return SIGILL;
@@ -1993,7 +2007,7 @@
}
case d_fmt: {
- union ieee754dp fs, ft;
+ union ieee754dp fd, fs, ft;
union {
union ieee754dp(*b) (union ieee754dp, union ieee754dp);
union ieee754dp(*u) (union ieee754dp);
@@ -2243,6 +2257,17 @@
rfmt = w_fmt;
goto copcsr;
+ case fsel_op:
+ if (!cpu_has_mips_r6)
+ return SIGILL;
+
+ DPFROMREG(fd, MIPSInst_FD(ir));
+ if (fd.bits & 0x1)
+ DPFROMREG(rv.d, MIPSInst_FT(ir));
+ else
+ DPFROMREG(rv.d, MIPSInst_FS(ir));
+ break;
+
case fcvtl_op:
if (!cpu_has_mips_3_4_5_64_r2_r6)
return SIGILL;
diff --git a/arch/mips/math-emu/dp_maddf.c b/arch/mips/math-emu/dp_maddf.c
index 119eda9..4a2d03c7 100644
--- a/arch/mips/math-emu/dp_maddf.c
+++ b/arch/mips/math-emu/dp_maddf.c
@@ -14,8 +14,12 @@
#include "ieee754dp.h"
-union ieee754dp ieee754dp_maddf(union ieee754dp z, union ieee754dp x,
- union ieee754dp y)
+enum maddf_flags {
+ maddf_negate_product = 1 << 0,
+};
+
+static union ieee754dp _dp_maddf(union ieee754dp z, union ieee754dp x,
+ union ieee754dp y, enum maddf_flags flags)
{
int re;
int rs;
@@ -32,16 +36,15 @@
COMPXDP;
COMPYDP;
-
- u64 zm; int ze; int zs __maybe_unused; int zc;
+ COMPZDP;
EXPLODEXDP;
EXPLODEYDP;
- EXPLODEDP(z, zc, zs, ze, zm)
+ EXPLODEZDP;
FLUSHXDP;
FLUSHYDP;
- FLUSHDP(z, zc, zs, ze, zm);
+ FLUSHZDP;
ieee754_clearcx();
@@ -50,7 +53,7 @@
ieee754_setcx(IEEE754_INVALID_OPERATION);
return ieee754dp_nanxcpt(z);
case IEEE754_CLASS_DNORM:
- DPDNORMx(zm, ze);
+ DPDNORMZ;
/* QNAN is handled separately below */
}
@@ -154,13 +157,15 @@
re = xe + ye;
rs = xs ^ ys;
+ if (flags & maddf_negate_product)
+ rs ^= 1;
/* shunt to top of word */
xm <<= 64 - (DP_FBITS + 1);
ym <<= 64 - (DP_FBITS + 1);
/*
- * Multiply 32 bits xm, ym to give high 32 bits rm with stickness.
+ * Multiply 64 bits xm, ym to give high 64 bits rm with stickness.
*/
/* 32 * 32 => 64 */
@@ -198,7 +203,7 @@
if ((s64) rm < 0) {
rm = (rm >> (64 - (DP_FBITS + 1 + 3))) |
((rm << (DP_FBITS + 1 + 3)) != 0);
- re++;
+ re++;
} else {
rm = (rm >> (64 - (DP_FBITS + 1 + 3 + 1))) |
((rm << (DP_FBITS + 1 + 3 + 1)) != 0);
@@ -263,3 +268,15 @@
return ieee754dp_format(zs, ze, zm);
}
+
+union ieee754dp ieee754dp_maddf(union ieee754dp z, union ieee754dp x,
+ union ieee754dp y)
+{
+ return _dp_maddf(z, x, y, 0);
+}
+
+union ieee754dp ieee754dp_msubf(union ieee754dp z, union ieee754dp x,
+ union ieee754dp y)
+{
+ return _dp_maddf(z, x, y, maddf_negate_product);
+}
diff --git a/arch/mips/math-emu/dp_msubf.c b/arch/mips/math-emu/dp_msubf.c
deleted file mode 100644
index 1224126..0000000
--- a/arch/mips/math-emu/dp_msubf.c
+++ /dev/null
@@ -1,269 +0,0 @@
-/*
- * IEEE754 floating point arithmetic
- * double precision: MSUB.f (Fused Multiply Subtract)
- * MSUBF.fmt: FPR[fd] = FPR[fd] - (FPR[fs] x FPR[ft])
- *
- * MIPS floating point support
- * Copyright (C) 2015 Imagination Technologies, Ltd.
- * Author: Markos Chandras <markos.chandras@imgtec.com>
- *
- * This program is free software; you can distribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; version 2 of the License.
- */
-
-#include "ieee754dp.h"
-
-union ieee754dp ieee754dp_msubf(union ieee754dp z, union ieee754dp x,
- union ieee754dp y)
-{
- int re;
- int rs;
- u64 rm;
- unsigned lxm;
- unsigned hxm;
- unsigned lym;
- unsigned hym;
- u64 lrm;
- u64 hrm;
- u64 t;
- u64 at;
- int s;
-
- COMPXDP;
- COMPYDP;
-
- u64 zm; int ze; int zs __maybe_unused; int zc;
-
- EXPLODEXDP;
- EXPLODEYDP;
- EXPLODEDP(z, zc, zs, ze, zm)
-
- FLUSHXDP;
- FLUSHYDP;
- FLUSHDP(z, zc, zs, ze, zm);
-
- ieee754_clearcx();
-
- switch (zc) {
- case IEEE754_CLASS_SNAN:
- ieee754_setcx(IEEE754_INVALID_OPERATION);
- return ieee754dp_nanxcpt(z);
- case IEEE754_CLASS_DNORM:
- DPDNORMx(zm, ze);
- /* QNAN is handled separately below */
- }
-
- switch (CLPAIR(xc, yc)) {
- case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_SNAN):
- case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_SNAN):
- case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_SNAN):
- case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_SNAN):
- case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_SNAN):
- return ieee754dp_nanxcpt(y);
-
- case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_SNAN):
- case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_QNAN):
- case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_ZERO):
- case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_NORM):
- case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_DNORM):
- case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_INF):
- return ieee754dp_nanxcpt(x);
-
- case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_QNAN):
- case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_QNAN):
- case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_QNAN):
- case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_QNAN):
- return y;
-
- case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_QNAN):
- case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_ZERO):
- case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_NORM):
- case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_DNORM):
- case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_INF):
- return x;
-
-
- /*
- * Infinity handling
- */
- case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_ZERO):
- case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_INF):
- if (zc == IEEE754_CLASS_QNAN)
- return z;
- ieee754_setcx(IEEE754_INVALID_OPERATION);
- return ieee754dp_indef();
-
- case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_INF):
- case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_INF):
- case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_NORM):
- case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_DNORM):
- case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_INF):
- if (zc == IEEE754_CLASS_QNAN)
- return z;
- return ieee754dp_inf(xs ^ ys);
-
- case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_ZERO):
- case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_NORM):
- case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_DNORM):
- case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_ZERO):
- case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_ZERO):
- if (zc == IEEE754_CLASS_INF)
- return ieee754dp_inf(zs);
- /* Multiplication is 0 so just return z */
- return z;
-
- case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_DNORM):
- DPDNORMX;
-
- case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_DNORM):
- if (zc == IEEE754_CLASS_QNAN)
- return z;
- else if (zc == IEEE754_CLASS_INF)
- return ieee754dp_inf(zs);
- DPDNORMY;
- break;
-
- case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_NORM):
- if (zc == IEEE754_CLASS_QNAN)
- return z;
- else if (zc == IEEE754_CLASS_INF)
- return ieee754dp_inf(zs);
- DPDNORMX;
- break;
-
- case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_NORM):
- if (zc == IEEE754_CLASS_QNAN)
- return z;
- else if (zc == IEEE754_CLASS_INF)
- return ieee754dp_inf(zs);
- /* fall through to real computations */
- }
-
- /* Finally get to do some computation */
-
- /*
- * Do the multiplication bit first
- *
- * rm = xm * ym, re = xe + ye basically
- *
- * At this point xm and ym should have been normalized.
- */
- assert(xm & DP_HIDDEN_BIT);
- assert(ym & DP_HIDDEN_BIT);
-
- re = xe + ye;
- rs = xs ^ ys;
-
- /* shunt to top of word */
- xm <<= 64 - (DP_FBITS + 1);
- ym <<= 64 - (DP_FBITS + 1);
-
- /*
- * Multiply 32 bits xm, ym to give high 32 bits rm with stickness.
- */
-
- /* 32 * 32 => 64 */
-#define DPXMULT(x, y) ((u64)(x) * (u64)y)
-
- lxm = xm;
- hxm = xm >> 32;
- lym = ym;
- hym = ym >> 32;
-
- lrm = DPXMULT(lxm, lym);
- hrm = DPXMULT(hxm, hym);
-
- t = DPXMULT(lxm, hym);
-
- at = lrm + (t << 32);
- hrm += at < lrm;
- lrm = at;
-
- hrm = hrm + (t >> 32);
-
- t = DPXMULT(hxm, lym);
-
- at = lrm + (t << 32);
- hrm += at < lrm;
- lrm = at;
-
- hrm = hrm + (t >> 32);
-
- rm = hrm | (lrm != 0);
-
- /*
- * Sticky shift down to normal rounding precision.
- */
- if ((s64) rm < 0) {
- rm = (rm >> (64 - (DP_FBITS + 1 + 3))) |
- ((rm << (DP_FBITS + 1 + 3)) != 0);
- re++;
- } else {
- rm = (rm >> (64 - (DP_FBITS + 1 + 3 + 1))) |
- ((rm << (DP_FBITS + 1 + 3 + 1)) != 0);
- }
- assert(rm & (DP_HIDDEN_BIT << 3));
-
- /* And now the subtraction */
-
- /* flip sign of r and handle as add */
- rs ^= 1;
-
- assert(zm & DP_HIDDEN_BIT);
-
- /*
- * Provide guard,round and stick bit space.
- */
- zm <<= 3;
-
- if (ze > re) {
- /*
- * Have to shift y fraction right to align.
- */
- s = ze - re;
- rm = XDPSRS(rm, s);
- re += s;
- } else if (re > ze) {
- /*
- * Have to shift x fraction right to align.
- */
- s = re - ze;
- zm = XDPSRS(zm, s);
- ze += s;
- }
- assert(ze == re);
- assert(ze <= DP_EMAX);
-
- if (zs == rs) {
- /*
- * Generate 28 bit result of adding two 27 bit numbers
- * leaving result in xm, xs and xe.
- */
- zm = zm + rm;
-
- if (zm >> (DP_FBITS + 1 + 3)) { /* carry out */
- zm = XDPSRS1(zm);
- ze++;
- }
- } else {
- if (zm >= rm) {
- zm = zm - rm;
- } else {
- zm = rm - zm;
- zs = rs;
- }
- if (zm == 0)
- return ieee754dp_zero(ieee754_csr.rm == FPU_CSR_RD);
-
- /*
- * Normalize to rounding precision.
- */
- while ((zm >> (DP_FBITS + 3)) == 0) {
- zm <<= 1;
- ze--;
- }
- }
-
- return ieee754dp_format(zs, ze, zm);
-}
diff --git a/arch/mips/math-emu/dp_mul.c b/arch/mips/math-emu/dp_mul.c
index d0901f0..87d0b44 100644
--- a/arch/mips/math-emu/dp_mul.c
+++ b/arch/mips/math-emu/dp_mul.c
@@ -125,7 +125,7 @@
ym <<= 64 - (DP_FBITS + 1);
/*
- * Multiply 32 bits xm, ym to give high 32 bits rm with stickness.
+ * Multiply 64 bits xm, ym to give high 64 bits rm with stickness.
*/
/* 32 * 32 => 64 */
@@ -163,7 +163,7 @@
if ((s64) rm < 0) {
rm = (rm >> (64 - (DP_FBITS + 1 + 3))) |
((rm << (DP_FBITS + 1 + 3)) != 0);
- re++;
+ re++;
} else {
rm = (rm >> (64 - (DP_FBITS + 1 + 3 + 1))) |
((rm << (DP_FBITS + 1 + 3 + 1)) != 0);
diff --git a/arch/mips/math-emu/dsemul.c b/arch/mips/math-emu/dsemul.c
index 46b964d..d4ceacd 100644
--- a/arch/mips/math-emu/dsemul.c
+++ b/arch/mips/math-emu/dsemul.c
@@ -60,7 +60,7 @@
unsigned int rs;
s32 v;
- rs = (((insn.mm_a_format.rs + 0x1e) & 0xf) + 2);
+ rs = (((insn.mm_a_format.rs + 0xe) & 0xf) + 2);
v = regs->cp0_epc & ~3;
v += insn.mm_a_format.simmediate << 2;
regs->regs[rs] = (long)v;
diff --git a/arch/mips/math-emu/ieee754dp.c b/arch/mips/math-emu/ieee754dp.c
index 47d26c8..465a034 100644
--- a/arch/mips/math-emu/ieee754dp.c
+++ b/arch/mips/math-emu/ieee754dp.c
@@ -54,10 +54,13 @@
assert(ieee754dp_issnan(r));
ieee754_setcx(IEEE754_INVALID_OPERATION);
- if (ieee754_csr.nan2008)
+ if (ieee754_csr.nan2008) {
DPMANT(r) |= DP_MBIT(DP_FBITS - 1);
- else
- r = ieee754dp_indef();
+ } else {
+ DPMANT(r) &= ~DP_MBIT(DP_FBITS - 1);
+ if (!ieee754dp_isnan(r))
+ DPMANT(r) |= DP_MBIT(DP_FBITS - 2);
+ }
return r;
}
diff --git a/arch/mips/math-emu/ieee754dp.h b/arch/mips/math-emu/ieee754dp.h
index e2babd9..9ba0230 100644
--- a/arch/mips/math-emu/ieee754dp.h
+++ b/arch/mips/math-emu/ieee754dp.h
@@ -60,6 +60,7 @@
while ((m >> DP_FBITS) == 0) { m <<= 1; e--; }
#define DPDNORMX DPDNORMx(xm, xe)
#define DPDNORMY DPDNORMx(ym, ye)
+#define DPDNORMZ DPDNORMx(zm, ze)
static inline union ieee754dp builddp(int s, int bx, u64 m)
{
diff --git a/arch/mips/math-emu/ieee754int.h b/arch/mips/math-emu/ieee754int.h
index ed7bb27..8bc2f69 100644
--- a/arch/mips/math-emu/ieee754int.h
+++ b/arch/mips/math-emu/ieee754int.h
@@ -55,6 +55,9 @@
#define COMPYSP \
unsigned ym; int ye; int ys; int yc
+#define COMPZSP \
+ unsigned zm; int ze; int zs; int zc
+
#define EXPLODESP(v, vc, vs, ve, vm) \
{ \
vs = SPSIGN(v); \
@@ -81,6 +84,7 @@
}
#define EXPLODEXSP EXPLODESP(x, xc, xs, xe, xm)
#define EXPLODEYSP EXPLODESP(y, yc, ys, ye, ym)
+#define EXPLODEZSP EXPLODESP(z, zc, zs, ze, zm)
#define COMPXDP \
@@ -89,6 +93,9 @@
#define COMPYDP \
u64 ym; int ye; int ys; int yc
+#define COMPZDP \
+ u64 zm; int ze; int zs; int zc
+
#define EXPLODEDP(v, vc, vs, ve, vm) \
{ \
vm = DPMANT(v); \
@@ -115,6 +122,7 @@
}
#define EXPLODEXDP EXPLODEDP(x, xc, xs, xe, xm)
#define EXPLODEYDP EXPLODEDP(y, yc, ys, ye, ym)
+#define EXPLODEZDP EXPLODEDP(z, zc, zs, ze, zm)
#define FLUSHDP(v, vc, vs, ve, vm) \
if (vc==IEEE754_CLASS_DNORM) { \
@@ -140,7 +148,9 @@
#define FLUSHXDP FLUSHDP(x, xc, xs, xe, xm)
#define FLUSHYDP FLUSHDP(y, yc, ys, ye, ym)
+#define FLUSHZDP FLUSHDP(z, zc, zs, ze, zm)
#define FLUSHXSP FLUSHSP(x, xc, xs, xe, xm)
#define FLUSHYSP FLUSHSP(y, yc, ys, ye, ym)
+#define FLUSHZSP FLUSHSP(z, zc, zs, ze, zm)
#endif /* __IEEE754INT_H */
diff --git a/arch/mips/math-emu/ieee754sp.c b/arch/mips/math-emu/ieee754sp.c
index e0b2c45..260e6896 100644
--- a/arch/mips/math-emu/ieee754sp.c
+++ b/arch/mips/math-emu/ieee754sp.c
@@ -54,10 +54,13 @@
assert(ieee754sp_issnan(r));
ieee754_setcx(IEEE754_INVALID_OPERATION);
- if (ieee754_csr.nan2008)
+ if (ieee754_csr.nan2008) {
SPMANT(r) |= SP_MBIT(SP_FBITS - 1);
- else
- r = ieee754sp_indef();
+ } else {
+ SPMANT(r) &= ~SP_MBIT(SP_FBITS - 1);
+ if (!ieee754sp_isnan(r))
+ SPMANT(r) |= SP_MBIT(SP_FBITS - 2);
+ }
return r;
}
@@ -138,7 +141,8 @@
} else {
/* sticky right shift es bits
*/
- SPXSRSXn(es);
+ xm = XSPSRS(xm, es);
+ xe += es;
assert((xm & (SP_HIDDEN_BIT << 3)) == 0);
assert(xe == SP_EMIN);
}
diff --git a/arch/mips/math-emu/ieee754sp.h b/arch/mips/math-emu/ieee754sp.h
index 374a3f0..8476067 100644
--- a/arch/mips/math-emu/ieee754sp.h
+++ b/arch/mips/math-emu/ieee754sp.h
@@ -46,25 +46,24 @@
}
/* 3bit extended single precision sticky right shift */
-#define SPXSRSXn(rs) \
- (xe += rs, \
- xm = (rs > (SP_FBITS+3))?1:((xm) >> (rs)) | ((xm) << (32-(rs)) != 0))
+#define XSPSRS(v, rs) \
+ ((rs > (SP_FBITS+3))?1:((v) >> (rs)) | ((v) << (32-(rs)) != 0))
+
+#define XSPSRS1(m) \
+ ((m >> 1) | (m & 1))
#define SPXSRSX1() \
- (xe++, (xm = (xm >> 1) | (xm & 1)))
-
-#define SPXSRSYn(rs) \
- (ye+=rs, \
- ym = (rs > (SP_FBITS+3))?1:((ym) >> (rs)) | ((ym) << (32-(rs)) != 0))
+ (xe++, (xm = XSPSRS1(xm)))
#define SPXSRSY1() \
- (ye++, (ym = (ym >> 1) | (ym & 1)))
+ (ye++, (ym = XSPSRS1(ym)))
/* convert denormal to normalized with extended exponent */
#define SPDNORMx(m,e) \
while ((m >> SP_FBITS) == 0) { m <<= 1; e--; }
#define SPDNORMX SPDNORMx(xm, xe)
#define SPDNORMY SPDNORMx(ym, ye)
+#define SPDNORMZ SPDNORMx(zm, ze)
static inline union ieee754sp buildsp(int s, int bx, unsigned m)
{
diff --git a/arch/mips/math-emu/sp_add.c b/arch/mips/math-emu/sp_add.c
index f1c87b0..c55c0c0 100644
--- a/arch/mips/math-emu/sp_add.c
+++ b/arch/mips/math-emu/sp_add.c
@@ -132,13 +132,15 @@
* Have to shift y fraction right to align.
*/
s = xe - ye;
- SPXSRSYn(s);
+ ym = XSPSRS(ym, s);
+ ye += s;
} else if (ye > xe) {
/*
* Have to shift x fraction right to align.
*/
s = ye - xe;
- SPXSRSXn(s);
+ xm = XSPSRS(xm, s);
+ xe += s;
}
assert(xe == ye);
assert(xe <= SP_EMAX);
diff --git a/arch/mips/math-emu/sp_maddf.c b/arch/mips/math-emu/sp_maddf.c
index dd1dd83..a8cd8b4 100644
--- a/arch/mips/math-emu/sp_maddf.c
+++ b/arch/mips/math-emu/sp_maddf.c
@@ -14,8 +14,12 @@
#include "ieee754sp.h"
-union ieee754sp ieee754sp_maddf(union ieee754sp z, union ieee754sp x,
- union ieee754sp y)
+enum maddf_flags {
+ maddf_negate_product = 1 << 0,
+};
+
+static union ieee754sp _sp_maddf(union ieee754sp z, union ieee754sp x,
+ union ieee754sp y, enum maddf_flags flags)
{
int re;
int rs;
@@ -32,15 +36,15 @@
COMPXSP;
COMPYSP;
- u32 zm; int ze; int zs __maybe_unused; int zc;
+ COMPZSP;
EXPLODEXSP;
EXPLODEYSP;
- EXPLODESP(z, zc, zs, ze, zm)
+ EXPLODEZSP;
FLUSHXSP;
FLUSHYSP;
- FLUSHSP(z, zc, zs, ze, zm);
+ FLUSHZSP;
ieee754_clearcx();
@@ -49,7 +53,7 @@
ieee754_setcx(IEEE754_INVALID_OPERATION);
return ieee754sp_nanxcpt(z);
case IEEE754_CLASS_DNORM:
- SPDNORMx(zm, ze);
+ SPDNORMZ;
/* QNAN is handled separately below */
}
@@ -154,6 +158,8 @@
re = xe + ye;
rs = xs ^ ys;
+ if (flags & maddf_negate_product)
+ rs ^= 1;
/* shunt to top of word */
xm <<= 32 - (SP_FBITS + 1);
@@ -208,16 +214,18 @@
if (ze > re) {
/*
- * Have to shift y fraction right to align.
+ * Have to shift r fraction right to align.
*/
s = ze - re;
- SPXSRSYn(s);
+ rm = XSPSRS(rm, s);
+ re += s;
} else if (re > ze) {
/*
- * Have to shift x fraction right to align.
+ * Have to shift z fraction right to align.
*/
s = re - ze;
- SPXSRSYn(s);
+ zm = XSPSRS(zm, s);
+ ze += s;
}
assert(ze == re);
assert(ze <= SP_EMAX);
@@ -230,7 +238,8 @@
zm = zm + rm;
if (zm >> (SP_FBITS + 1 + 3)) { /* carry out */
- SPXSRSX1();
+ zm = XSPSRS1(zm);
+ ze++;
}
} else {
if (zm >= rm) {
@@ -253,3 +262,15 @@
}
return ieee754sp_format(zs, ze, zm);
}
+
+union ieee754sp ieee754sp_maddf(union ieee754sp z, union ieee754sp x,
+ union ieee754sp y)
+{
+ return _sp_maddf(z, x, y, 0);
+}
+
+union ieee754sp ieee754sp_msubf(union ieee754sp z, union ieee754sp x,
+ union ieee754sp y)
+{
+ return _sp_maddf(z, x, y, maddf_negate_product);
+}
diff --git a/arch/mips/math-emu/sp_msubf.c b/arch/mips/math-emu/sp_msubf.c
deleted file mode 100644
index 81c38b980..0000000
--- a/arch/mips/math-emu/sp_msubf.c
+++ /dev/null
@@ -1,258 +0,0 @@
-/*
- * IEEE754 floating point arithmetic
- * single precision: MSUB.f (Fused Multiply Subtract)
- * MSUBF.fmt: FPR[fd] = FPR[fd] - (FPR[fs] x FPR[ft])
- *
- * MIPS floating point support
- * Copyright (C) 2015 Imagination Technologies, Ltd.
- * Author: Markos Chandras <markos.chandras@imgtec.com>
- *
- * This program is free software; you can distribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; version 2 of the License.
- */
-
-#include "ieee754sp.h"
-
-union ieee754sp ieee754sp_msubf(union ieee754sp z, union ieee754sp x,
- union ieee754sp y)
-{
- int re;
- int rs;
- unsigned rm;
- unsigned short lxm;
- unsigned short hxm;
- unsigned short lym;
- unsigned short hym;
- unsigned lrm;
- unsigned hrm;
- unsigned t;
- unsigned at;
- int s;
-
- COMPXSP;
- COMPYSP;
- u32 zm; int ze; int zs __maybe_unused; int zc;
-
- EXPLODEXSP;
- EXPLODEYSP;
- EXPLODESP(z, zc, zs, ze, zm)
-
- FLUSHXSP;
- FLUSHYSP;
- FLUSHSP(z, zc, zs, ze, zm);
-
- ieee754_clearcx();
-
- switch (zc) {
- case IEEE754_CLASS_SNAN:
- ieee754_setcx(IEEE754_INVALID_OPERATION);
- return ieee754sp_nanxcpt(z);
- case IEEE754_CLASS_DNORM:
- SPDNORMx(zm, ze);
- /* QNAN is handled separately below */
- }
-
- switch (CLPAIR(xc, yc)) {
- case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_SNAN):
- case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_SNAN):
- case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_SNAN):
- case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_SNAN):
- case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_SNAN):
- return ieee754sp_nanxcpt(y);
-
- case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_SNAN):
- case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_QNAN):
- case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_ZERO):
- case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_NORM):
- case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_DNORM):
- case CLPAIR(IEEE754_CLASS_SNAN, IEEE754_CLASS_INF):
- return ieee754sp_nanxcpt(x);
-
- case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_QNAN):
- case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_QNAN):
- case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_QNAN):
- case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_QNAN):
- return y;
-
- case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_QNAN):
- case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_ZERO):
- case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_NORM):
- case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_DNORM):
- case CLPAIR(IEEE754_CLASS_QNAN, IEEE754_CLASS_INF):
- return x;
-
- /*
- * Infinity handling
- */
- case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_ZERO):
- case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_INF):
- if (zc == IEEE754_CLASS_QNAN)
- return z;
- ieee754_setcx(IEEE754_INVALID_OPERATION);
- return ieee754sp_indef();
-
- case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_INF):
- case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_INF):
- case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_NORM):
- case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_DNORM):
- case CLPAIR(IEEE754_CLASS_INF, IEEE754_CLASS_INF):
- if (zc == IEEE754_CLASS_QNAN)
- return z;
- return ieee754sp_inf(xs ^ ys);
-
- case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_ZERO):
- case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_NORM):
- case CLPAIR(IEEE754_CLASS_ZERO, IEEE754_CLASS_DNORM):
- case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_ZERO):
- case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_ZERO):
- if (zc == IEEE754_CLASS_INF)
- return ieee754sp_inf(zs);
- /* Multiplication is 0 so just return z */
- return z;
-
- case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_DNORM):
- SPDNORMX;
-
- case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_DNORM):
- if (zc == IEEE754_CLASS_QNAN)
- return z;
- else if (zc == IEEE754_CLASS_INF)
- return ieee754sp_inf(zs);
- SPDNORMY;
- break;
-
- case CLPAIR(IEEE754_CLASS_DNORM, IEEE754_CLASS_NORM):
- if (zc == IEEE754_CLASS_QNAN)
- return z;
- else if (zc == IEEE754_CLASS_INF)
- return ieee754sp_inf(zs);
- SPDNORMX;
- break;
-
- case CLPAIR(IEEE754_CLASS_NORM, IEEE754_CLASS_NORM):
- if (zc == IEEE754_CLASS_QNAN)
- return z;
- else if (zc == IEEE754_CLASS_INF)
- return ieee754sp_inf(zs);
- /* fall through to real compuation */
- }
-
- /* Finally get to do some computation */
-
- /*
- * Do the multiplication bit first
- *
- * rm = xm * ym, re = xe + ye basically
- *
- * At this point xm and ym should have been normalized.
- */
-
- /* rm = xm * ym, re = xe+ye basically */
- assert(xm & SP_HIDDEN_BIT);
- assert(ym & SP_HIDDEN_BIT);
-
- re = xe + ye;
- rs = xs ^ ys;
-
- /* shunt to top of word */
- xm <<= 32 - (SP_FBITS + 1);
- ym <<= 32 - (SP_FBITS + 1);
-
- /*
- * Multiply 32 bits xm, ym to give high 32 bits rm with stickness.
- */
- lxm = xm & 0xffff;
- hxm = xm >> 16;
- lym = ym & 0xffff;
- hym = ym >> 16;
-
- lrm = lxm * lym; /* 16 * 16 => 32 */
- hrm = hxm * hym; /* 16 * 16 => 32 */
-
- t = lxm * hym; /* 16 * 16 => 32 */
- at = lrm + (t << 16);
- hrm += at < lrm;
- lrm = at;
- hrm = hrm + (t >> 16);
-
- t = hxm * lym; /* 16 * 16 => 32 */
- at = lrm + (t << 16);
- hrm += at < lrm;
- lrm = at;
- hrm = hrm + (t >> 16);
-
- rm = hrm | (lrm != 0);
-
- /*
- * Sticky shift down to normal rounding precision.
- */
- if ((int) rm < 0) {
- rm = (rm >> (32 - (SP_FBITS + 1 + 3))) |
- ((rm << (SP_FBITS + 1 + 3)) != 0);
- re++;
- } else {
- rm = (rm >> (32 - (SP_FBITS + 1 + 3 + 1))) |
- ((rm << (SP_FBITS + 1 + 3 + 1)) != 0);
- }
- assert(rm & (SP_HIDDEN_BIT << 3));
-
- /* And now the subtraction */
-
- /* Flip sign of r and handle as add */
- rs ^= 1;
-
- assert(zm & SP_HIDDEN_BIT);
-
- /*
- * Provide guard,round and stick bit space.
- */
- zm <<= 3;
-
- if (ze > re) {
- /*
- * Have to shift y fraction right to align.
- */
- s = ze - re;
- SPXSRSYn(s);
- } else if (re > ze) {
- /*
- * Have to shift x fraction right to align.
- */
- s = re - ze;
- SPXSRSYn(s);
- }
- assert(ze == re);
- assert(ze <= SP_EMAX);
-
- if (zs == rs) {
- /*
- * Generate 28 bit result of adding two 27 bit numbers
- * leaving result in zm, zs and ze.
- */
- zm = zm + rm;
-
- if (zm >> (SP_FBITS + 1 + 3)) { /* carry out */
- SPXSRSX1(); /* shift preserving sticky */
- }
- } else {
- if (zm >= rm) {
- zm = zm - rm;
- } else {
- zm = rm - zm;
- zs = rs;
- }
- if (zm == 0)
- return ieee754sp_zero(ieee754_csr.rm == FPU_CSR_RD);
-
- /*
- * Normalize in extended single precision
- */
- while ((zm >> (SP_MBITS + 3)) == 0) {
- zm <<= 1;
- ze--;
- }
-
- }
- return ieee754sp_format(zs, ze, zm);
-}
diff --git a/arch/mips/math-emu/sp_sub.c b/arch/mips/math-emu/sp_sub.c
index ec5f937..dc998ed 100644
--- a/arch/mips/math-emu/sp_sub.c
+++ b/arch/mips/math-emu/sp_sub.c
@@ -134,13 +134,15 @@
* have to shift y fraction right to align
*/
s = xe - ye;
- SPXSRSYn(s);
+ ym = XSPSRS(ym, s);
+ ye += s;
} else if (ye > xe) {
/*
* have to shift x fraction right to align
*/
s = ye - xe;
- SPXSRSXn(s);
+ xm = XSPSRS(xm, s);
+ xe += s;
}
assert(xe == ye);
assert(xe <= SP_EMAX);
diff --git a/arch/mips/mm/c-r4k.c b/arch/mips/mm/c-r4k.c
index caac3d7..ef7f925 100644
--- a/arch/mips/mm/c-r4k.c
+++ b/arch/mips/mm/c-r4k.c
@@ -77,6 +77,7 @@
*/
static unsigned long icache_size __read_mostly;
static unsigned long dcache_size __read_mostly;
+static unsigned long vcache_size __read_mostly;
static unsigned long scache_size __read_mostly;
/*
@@ -447,6 +448,11 @@
r4k_blast_scache();
break;
+ case CPU_BMIPS5000:
+ r4k_blast_scache();
+ __sync();
+ break;
+
default:
r4k_blast_dcache();
r4k_blast_icache();
@@ -492,7 +498,14 @@
if (!(has_valid_asid(vma->vm_mm)))
return;
- r4k_blast_dcache();
+ /*
+ * If dcache can alias, we must blast it since mapping is changing.
+ * If executable, we must ensure any dirty lines are written back far
+ * enough to be visible to icache.
+ */
+ if (cpu_has_dc_aliases || (exec && !cpu_has_ic_fills_f_dc))
+ r4k_blast_dcache();
+ /* If executable, blast stale lines from icache */
if (exec)
r4k_blast_icache();
}
@@ -502,7 +515,7 @@
{
int exec = vma->vm_flags & VM_EXEC;
- if (cpu_has_dc_aliases || (exec && !cpu_has_ic_fills_f_dc))
+ if (cpu_has_dc_aliases || exec)
r4k_on_each_cpu(local_r4k_flush_cache_range, vma);
}
@@ -1148,6 +1161,8 @@
c->dcache.ways *
c->dcache.linesz;
c->dcache.waybit = 0;
+ if ((prid & PRID_REV_MASK) >= PRID_REV_LOONGSON3A_R2)
+ c->options |= MIPS_CPU_PREFETCH;
break;
case CPU_CAVIUM_OCTEON3:
@@ -1278,6 +1293,8 @@
case CPU_M5150:
case CPU_QEMU_GENERIC:
case CPU_I6400:
+ case CPU_P6600:
+ case CPU_M6250:
if (!(read_c0_config7() & MIPS_CONF7_IAR) &&
(c->icache.waysize > PAGE_SIZE))
c->icache.flags |= MIPS_CACHE_ALIASES;
@@ -1304,9 +1321,16 @@
break;
case CPU_ALCHEMY:
+ case CPU_I6400:
c->icache.flags |= MIPS_CACHE_IC_F_DC;
break;
+ case CPU_BMIPS5000:
+ c->icache.flags |= MIPS_CACHE_IC_F_DC;
+ /* Cache aliases are handled in hardware; allow HIGHMEM */
+ c->dcache.flags &= ~MIPS_CACHE_ALIASES;
+ break;
+
case CPU_LOONGSON2:
/*
* LOONGSON2 has 4 way icache, but when using indexed cache op,
@@ -1328,6 +1352,31 @@
c->dcache.linesz);
}
+static void probe_vcache(void)
+{
+ struct cpuinfo_mips *c = ¤t_cpu_data;
+ unsigned int config2, lsize;
+
+ if (current_cpu_type() != CPU_LOONGSON3)
+ return;
+
+ config2 = read_c0_config2();
+ if ((lsize = ((config2 >> 20) & 15)))
+ c->vcache.linesz = 2 << lsize;
+ else
+ c->vcache.linesz = lsize;
+
+ c->vcache.sets = 64 << ((config2 >> 24) & 15);
+ c->vcache.ways = 1 + ((config2 >> 16) & 15);
+
+ vcache_size = c->vcache.sets * c->vcache.ways * c->vcache.linesz;
+
+ c->vcache.waybit = 0;
+
+ pr_info("Unified victim cache %ldkB %s, linesize %d bytes.\n",
+ vcache_size >> 10, way_string[c->vcache.ways], c->vcache.linesz);
+}
+
/*
* If you even _breathe_ on this function, look at the gcc output and make sure
* it does not pop things on and off the stack for the cache sizing loop that
@@ -1650,6 +1699,7 @@
struct cpuinfo_mips *c = ¤t_cpu_data;
probe_pcache();
+ probe_vcache();
setup_scache();
r4k_blast_dcache_page_setup();
@@ -1671,7 +1721,7 @@
* This code supports virtually indexed processors and will be
* unnecessarily inefficient on physically indexed processors.
*/
- if (c->dcache.linesz)
+ if (c->dcache.linesz && cpu_has_dc_aliases)
shm_align_mask = max_t( unsigned long,
c->dcache.sets * c->dcache.linesz - 1,
PAGE_SIZE - 1);
@@ -1744,12 +1794,24 @@
flush_icache_range = (void *)b5k_instruction_hazard;
local_flush_icache_range = (void *)b5k_instruction_hazard;
- /* Cache aliases are handled in hardware; allow HIGHMEM */
- current_cpu_data.dcache.flags &= ~MIPS_CACHE_ALIASES;
/* Optimization: an L2 flush implicitly flushes the L1 */
current_cpu_data.options |= MIPS_CPU_INCLUSIVE_CACHES;
break;
+ case CPU_LOONGSON3:
+ /* Loongson-3 maintains cache coherency by hardware */
+ __flush_cache_all = cache_noop;
+ __flush_cache_vmap = cache_noop;
+ __flush_cache_vunmap = cache_noop;
+ __flush_kernel_vmap_range = (void *)cache_noop;
+ flush_cache_mm = (void *)cache_noop;
+ flush_cache_page = (void *)cache_noop;
+ flush_cache_range = (void *)cache_noop;
+ flush_cache_sigtramp = (void *)cache_noop;
+ flush_icache_all = (void *)cache_noop;
+ flush_data_cache_page = (void *)cache_noop;
+ local_flush_data_cache_page = (void *)cache_noop;
+ break;
}
}
diff --git a/arch/mips/mm/cache.c b/arch/mips/mm/cache.c
index 3f159ca..bf04c6c 100644
--- a/arch/mips/mm/cache.c
+++ b/arch/mips/mm/cache.c
@@ -16,6 +16,7 @@
#include <linux/mm.h>
#include <asm/cacheflush.h>
+#include <asm/highmem.h>
#include <asm/processor.h>
#include <asm/cpu.h>
#include <asm/cpu-features.h>
@@ -83,8 +84,6 @@
struct address_space *mapping = page_mapping(page);
unsigned long addr;
- if (PageHighMem(page))
- return;
if (mapping && !mapping_mapped(mapping)) {
SetPageDcacheDirty(page);
return;
@@ -95,8 +94,15 @@
* case is for exec env/arg pages and those are %99 certainly going to
* get faulted into the tlb (and thus flushed) anyways.
*/
- addr = (unsigned long) page_address(page);
+ if (PageHighMem(page))
+ addr = (unsigned long)kmap_atomic(page);
+ else
+ addr = (unsigned long)page_address(page);
+
flush_data_cache_page(addr);
+
+ if (PageHighMem(page))
+ __kunmap_atomic((void *)addr);
}
EXPORT_SYMBOL(__flush_dcache_page);
@@ -119,33 +125,28 @@
EXPORT_SYMBOL(__flush_anon_page);
-void __flush_icache_page(struct vm_area_struct *vma, struct page *page)
-{
- unsigned long addr;
-
- if (PageHighMem(page))
- return;
-
- addr = (unsigned long) page_address(page);
- flush_data_cache_page(addr);
-}
-EXPORT_SYMBOL_GPL(__flush_icache_page);
-
-void __update_cache(struct vm_area_struct *vma, unsigned long address,
- pte_t pte)
+void __update_cache(unsigned long address, pte_t pte)
{
struct page *page;
unsigned long pfn, addr;
- int exec = (vma->vm_flags & VM_EXEC) && !cpu_has_ic_fills_f_dc;
+ int exec = !pte_no_exec(pte) && !cpu_has_ic_fills_f_dc;
pfn = pte_pfn(pte);
if (unlikely(!pfn_valid(pfn)))
return;
page = pfn_to_page(pfn);
- if (page_mapping(page) && Page_dcache_dirty(page)) {
- addr = (unsigned long) page_address(page);
+ if (Page_dcache_dirty(page)) {
+ if (PageHighMem(page))
+ addr = (unsigned long)kmap_atomic(page);
+ else
+ addr = (unsigned long)page_address(page);
+
if (exec || pages_do_alias(addr, address & PAGE_MASK))
flush_data_cache_page(addr);
+
+ if (PageHighMem(page))
+ __kunmap_atomic((void *)addr);
+
ClearPageDcacheDirty(page);
}
}
diff --git a/arch/mips/mm/dma-default.c b/arch/mips/mm/dma-default.c
index 730d394..cb557d2 100644
--- a/arch/mips/mm/dma-default.c
+++ b/arch/mips/mm/dma-default.c
@@ -88,19 +88,20 @@
else
#endif
#if defined(CONFIG_ZONE_DMA32) && defined(CONFIG_ZONE_DMA)
- if (dev->coherent_dma_mask < DMA_BIT_MASK(32))
+ if (dev == NULL || dev->coherent_dma_mask < DMA_BIT_MASK(32))
dma_flag = __GFP_DMA;
else if (dev->coherent_dma_mask < DMA_BIT_MASK(64))
dma_flag = __GFP_DMA32;
else
#endif
#if defined(CONFIG_ZONE_DMA32) && !defined(CONFIG_ZONE_DMA)
- if (dev->coherent_dma_mask < DMA_BIT_MASK(64))
+ if (dev == NULL || dev->coherent_dma_mask < DMA_BIT_MASK(64))
dma_flag = __GFP_DMA32;
else
#endif
#if defined(CONFIG_ZONE_DMA) && !defined(CONFIG_ZONE_DMA32)
- if (dev->coherent_dma_mask < DMA_BIT_MASK(sizeof(phys_addr_t) * 8))
+ if (dev == NULL ||
+ dev->coherent_dma_mask < DMA_BIT_MASK(sizeof(phys_addr_t) * 8))
dma_flag = __GFP_DMA;
else
#endif
diff --git a/arch/mips/mm/init.c b/arch/mips/mm/init.c
index 7e5fa09..9b58eb5 100644
--- a/arch/mips/mm/init.c
+++ b/arch/mips/mm/init.c
@@ -98,8 +98,10 @@
idx += in_interrupt() ? FIX_N_COLOURS : 0;
vaddr = __fix_to_virt(FIX_CMAP_END - idx);
pte = mk_pte(page, prot);
-#if defined(CONFIG_PHYS_ADDR_T_64BIT) && defined(CONFIG_CPU_MIPS32)
+#if defined(CONFIG_XPA)
entrylo = pte_to_entrylo(pte.pte_high);
+#elif defined(CONFIG_PHYS_ADDR_T_64BIT) && defined(CONFIG_CPU_MIPS32)
+ entrylo = pte.pte_high;
#else
entrylo = pte_to_entrylo(pte_val(pte));
#endif
@@ -110,9 +112,11 @@
write_c0_entrylo0(entrylo);
write_c0_entrylo1(entrylo);
#ifdef CONFIG_XPA
- entrylo = (pte.pte_low & _PFNX_MASK);
- writex_c0_entrylo0(entrylo);
- writex_c0_entrylo1(entrylo);
+ if (cpu_has_xpa) {
+ entrylo = (pte.pte_low & _PFNX_MASK);
+ writex_c0_entrylo0(entrylo);
+ writex_c0_entrylo1(entrylo);
+ }
#endif
tlbidx = read_c0_wired();
write_c0_wired(tlbidx + 1);
@@ -196,7 +200,7 @@
if (cpu_has_dc_aliases)
SetPageDcacheDirty(page);
}
- if ((vma->vm_flags & VM_EXEC) && !cpu_has_ic_fills_f_dc)
+ if (vma->vm_flags & VM_EXEC)
flush_cache_page(vma, vaddr, page_to_pfn(page));
}
diff --git a/arch/mips/mm/page.c b/arch/mips/mm/page.c
index 885d73f..c41953c 100644
--- a/arch/mips/mm/page.c
+++ b/arch/mips/mm/page.c
@@ -188,6 +188,15 @@
}
break;
+ case CPU_LOONGSON3:
+ /* Loongson-3 only support the Pref_Load/Pref_Store. */
+ pref_bias_clear_store = 128;
+ pref_bias_copy_load = 128;
+ pref_bias_copy_store = 128;
+ pref_src_mode = Pref_Load;
+ pref_dst_mode = Pref_Store;
+ break;
+
default:
pref_bias_clear_store = 128;
pref_bias_copy_load = 256;
diff --git a/arch/mips/mm/sc-mips.c b/arch/mips/mm/sc-mips.c
index 91dec32..286a4d5 100644
--- a/arch/mips/mm/sc-mips.c
+++ b/arch/mips/mm/sc-mips.c
@@ -141,6 +141,7 @@
case CPU_P5600:
case CPU_BMIPS5000:
case CPU_QEMU_GENERIC:
+ case CPU_P6600:
if (config2 & (1 << 12))
return 0;
}
diff --git a/arch/mips/mm/tlb-r3k.c b/arch/mips/mm/tlb-r3k.c
index b4f366f..1290b99 100644
--- a/arch/mips/mm/tlb-r3k.c
+++ b/arch/mips/mm/tlb-r3k.c
@@ -43,7 +43,7 @@
{
unsigned long old_ctx;
- old_ctx = read_c0_entryhi() & ASID_MASK;
+ old_ctx = read_c0_entryhi() & cpu_asid_mask(¤t_cpu_data);
write_c0_entrylo0(0);
while (entry < current_cpu_data.tlbsize) {
write_c0_index(entry << 8);
@@ -81,6 +81,7 @@
void local_flush_tlb_range(struct vm_area_struct *vma, unsigned long start,
unsigned long end)
{
+ unsigned long asid_mask = cpu_asid_mask(¤t_cpu_data);
struct mm_struct *mm = vma->vm_mm;
int cpu = smp_processor_id();
@@ -89,13 +90,13 @@
#ifdef DEBUG_TLB
printk("[tlbrange<%lu,0x%08lx,0x%08lx>]",
- cpu_context(cpu, mm) & ASID_MASK, start, end);
+ cpu_context(cpu, mm) & asid_mask, start, end);
#endif
local_irq_save(flags);
size = (end - start + (PAGE_SIZE - 1)) >> PAGE_SHIFT;
if (size <= current_cpu_data.tlbsize) {
- int oldpid = read_c0_entryhi() & ASID_MASK;
- int newpid = cpu_context(cpu, mm) & ASID_MASK;
+ int oldpid = read_c0_entryhi() & asid_mask;
+ int newpid = cpu_context(cpu, mm) & asid_mask;
start &= PAGE_MASK;
end += PAGE_SIZE - 1;
@@ -159,6 +160,7 @@
void local_flush_tlb_page(struct vm_area_struct *vma, unsigned long page)
{
+ unsigned long asid_mask = cpu_asid_mask(¤t_cpu_data);
int cpu = smp_processor_id();
if (cpu_context(cpu, vma->vm_mm) != 0) {
@@ -168,10 +170,10 @@
#ifdef DEBUG_TLB
printk("[tlbpage<%lu,0x%08lx>]", cpu_context(cpu, vma->vm_mm), page);
#endif
- newpid = cpu_context(cpu, vma->vm_mm) & ASID_MASK;
+ newpid = cpu_context(cpu, vma->vm_mm) & asid_mask;
page &= PAGE_MASK;
local_irq_save(flags);
- oldpid = read_c0_entryhi() & ASID_MASK;
+ oldpid = read_c0_entryhi() & asid_mask;
write_c0_entryhi(page | newpid);
BARRIER;
tlb_probe();
@@ -190,6 +192,7 @@
void __update_tlb(struct vm_area_struct *vma, unsigned long address, pte_t pte)
{
+ unsigned long asid_mask = cpu_asid_mask(¤t_cpu_data);
unsigned long flags;
int idx, pid;
@@ -199,10 +202,10 @@
if (current->active_mm != vma->vm_mm)
return;
- pid = read_c0_entryhi() & ASID_MASK;
+ pid = read_c0_entryhi() & asid_mask;
#ifdef DEBUG_TLB
- if ((pid != (cpu_context(cpu, vma->vm_mm) & ASID_MASK)) || (cpu_context(cpu, vma->vm_mm) == 0)) {
+ if ((pid != (cpu_context(cpu, vma->vm_mm) & asid_mask)) || (cpu_context(cpu, vma->vm_mm) == 0)) {
printk("update_mmu_cache: Wheee, bogus tlbpid mmpid=%lu tlbpid=%d\n",
(cpu_context(cpu, vma->vm_mm)), pid);
}
@@ -228,6 +231,7 @@
void add_wired_entry(unsigned long entrylo0, unsigned long entrylo1,
unsigned long entryhi, unsigned long pagemask)
{
+ unsigned long asid_mask = cpu_asid_mask(¤t_cpu_data);
unsigned long flags;
unsigned long old_ctx;
static unsigned long wired = 0;
@@ -243,7 +247,7 @@
local_irq_save(flags);
/* Save old context and create impossible VPN2 value */
- old_ctx = read_c0_entryhi() & ASID_MASK;
+ old_ctx = read_c0_entryhi() & asid_mask;
old_pagemask = read_c0_pagemask();
w = read_c0_wired();
write_c0_wired(w + 1);
@@ -266,7 +270,7 @@
#endif
local_irq_save(flags);
- old_ctx = read_c0_entryhi() & ASID_MASK;
+ old_ctx = read_c0_entryhi() & asid_mask;
write_c0_entrylo0(entrylo0);
write_c0_entryhi(entryhi);
write_c0_index(wired);
diff --git a/arch/mips/mm/tlb-r4k.c b/arch/mips/mm/tlb-r4k.c
index c17d762..e8b335c 100644
--- a/arch/mips/mm/tlb-r4k.c
+++ b/arch/mips/mm/tlb-r4k.c
@@ -28,25 +28,28 @@
extern void build_tlb_refill_handler(void);
/*
- * LOONGSON2/3 has a 4 entry itlb which is a subset of dtlb,
- * unfortunately, itlb is not totally transparent to software.
+ * LOONGSON-2 has a 4 entry itlb which is a subset of jtlb, LOONGSON-3 has
+ * a 4 entry itlb and a 4 entry dtlb which are subsets of jtlb. Unfortunately,
+ * itlb/dtlb are not totally transparent to software.
*/
-static inline void flush_itlb(void)
+static inline void flush_micro_tlb(void)
{
switch (current_cpu_type()) {
case CPU_LOONGSON2:
+ write_c0_diag(LOONGSON_DIAG_ITLB);
+ break;
case CPU_LOONGSON3:
- write_c0_diag(4);
+ write_c0_diag(LOONGSON_DIAG_ITLB | LOONGSON_DIAG_DTLB);
break;
default:
break;
}
}
-static inline void flush_itlb_vm(struct vm_area_struct *vma)
+static inline void flush_micro_tlb_vm(struct vm_area_struct *vma)
{
if (vma->vm_flags & VM_EXEC)
- flush_itlb();
+ flush_micro_tlb();
}
void local_flush_tlb_all(void)
@@ -93,7 +96,7 @@
tlbw_use_hazard();
write_c0_entryhi(old_ctx);
htw_start();
- flush_itlb();
+ flush_micro_tlb();
local_irq_restore(flags);
}
EXPORT_SYMBOL(local_flush_tlb_all);
@@ -159,7 +162,7 @@
} else {
drop_mmu_context(mm, cpu);
}
- flush_itlb();
+ flush_micro_tlb();
local_irq_restore(flags);
}
}
@@ -205,7 +208,7 @@
} else {
local_flush_tlb_all();
}
- flush_itlb();
+ flush_micro_tlb();
local_irq_restore(flags);
}
@@ -240,7 +243,7 @@
finish:
write_c0_entryhi(oldpid);
htw_start();
- flush_itlb_vm(vma);
+ flush_micro_tlb_vm(vma);
local_irq_restore(flags);
}
}
@@ -274,7 +277,7 @@
}
write_c0_entryhi(oldpid);
htw_start();
- flush_itlb();
+ flush_micro_tlb();
local_irq_restore(flags);
}
@@ -301,7 +304,7 @@
local_irq_save(flags);
htw_stop();
- pid = read_c0_entryhi() & ASID_MASK;
+ pid = read_c0_entryhi() & cpu_asid_mask(¤t_cpu_data);
address &= (PAGE_MASK << 1);
write_c0_entryhi(address | pid);
pgdp = pgd_offset(vma->vm_mm, address);
@@ -336,10 +339,12 @@
#if defined(CONFIG_PHYS_ADDR_T_64BIT) && defined(CONFIG_CPU_MIPS32)
#ifdef CONFIG_XPA
write_c0_entrylo0(pte_to_entrylo(ptep->pte_high));
- writex_c0_entrylo0(ptep->pte_low & _PFNX_MASK);
+ if (cpu_has_xpa)
+ writex_c0_entrylo0(ptep->pte_low & _PFNX_MASK);
ptep++;
write_c0_entrylo1(pte_to_entrylo(ptep->pte_high));
- writex_c0_entrylo1(ptep->pte_low & _PFNX_MASK);
+ if (cpu_has_xpa)
+ writex_c0_entrylo1(ptep->pte_low & _PFNX_MASK);
#else
write_c0_entrylo0(ptep->pte_high);
ptep++;
@@ -357,7 +362,7 @@
}
tlbw_use_hazard();
htw_start();
- flush_itlb_vm(vma);
+ flush_micro_tlb_vm(vma);
local_irq_restore(flags);
}
@@ -400,19 +405,20 @@
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-int __init has_transparent_hugepage(void)
+int has_transparent_hugepage(void)
{
- unsigned int mask;
- unsigned long flags;
+ static unsigned int mask = -1;
- local_irq_save(flags);
- write_c0_pagemask(PM_HUGE_MASK);
- back_to_back_c0_hazard();
- mask = read_c0_pagemask();
- write_c0_pagemask(PM_DEFAULT_MASK);
+ if (mask == -1) { /* first call comes during __init */
+ unsigned long flags;
- local_irq_restore(flags);
-
+ local_irq_save(flags);
+ write_c0_pagemask(PM_HUGE_MASK);
+ back_to_back_c0_hazard();
+ mask = read_c0_pagemask();
+ write_c0_pagemask(PM_DEFAULT_MASK);
+ local_irq_restore(flags);
+ }
return mask == PM_HUGE_MASK;
}
diff --git a/arch/mips/mm/tlb-r8k.c b/arch/mips/mm/tlb-r8k.c
index 138a2ec..e86e2e5 100644
--- a/arch/mips/mm/tlb-r8k.c
+++ b/arch/mips/mm/tlb-r8k.c
@@ -194,7 +194,7 @@
if (current->active_mm != vma->vm_mm)
return;
- pid = read_c0_entryhi() & ASID_MASK;
+ pid = read_c0_entryhi() & cpu_asid_mask(¤t_cpu_data);
local_irq_save(flags);
address &= PAGE_MASK;
diff --git a/arch/mips/mm/tlbex.c b/arch/mips/mm/tlbex.c
index 84c6e3f..274da90 100644
--- a/arch/mips/mm/tlbex.c
+++ b/arch/mips/mm/tlbex.c
@@ -234,20 +234,16 @@
pr_debug("\n");
pr_define("_PAGE_PRESENT_SHIFT %d\n", _PAGE_PRESENT_SHIFT);
- pr_define("_PAGE_READ_SHIFT %d\n", _PAGE_READ_SHIFT);
+ pr_define("_PAGE_NO_READ_SHIFT %d\n", _PAGE_NO_READ_SHIFT);
pr_define("_PAGE_WRITE_SHIFT %d\n", _PAGE_WRITE_SHIFT);
pr_define("_PAGE_ACCESSED_SHIFT %d\n", _PAGE_ACCESSED_SHIFT);
pr_define("_PAGE_MODIFIED_SHIFT %d\n", _PAGE_MODIFIED_SHIFT);
#ifdef CONFIG_MIPS_HUGE_TLB_SUPPORT
pr_define("_PAGE_HUGE_SHIFT %d\n", _PAGE_HUGE_SHIFT);
#endif
-#if defined(CONFIG_CPU_MIPSR2) || defined(CONFIG_CPU_MIPSR6)
- if (cpu_has_rixi) {
#ifdef _PAGE_NO_EXEC_SHIFT
+ if (cpu_has_rixi)
pr_define("_PAGE_NO_EXEC_SHIFT %d\n", _PAGE_NO_EXEC_SHIFT);
- pr_define("_PAGE_NO_READ_SHIFT %d\n", _PAGE_NO_READ_SHIFT);
-#endif
- }
#endif
pr_define("_PAGE_GLOBAL_SHIFT %d\n", _PAGE_GLOBAL_SHIFT);
pr_define("_PAGE_VALID_SHIFT %d\n", _PAGE_VALID_SHIFT);
@@ -284,7 +280,12 @@
#define C0_ENTRYLO1 3, 0
#define C0_CONTEXT 4, 0
#define C0_PAGEMASK 5, 0
+#define C0_PWBASE 5, 5
+#define C0_PWFIELD 5, 6
+#define C0_PWSIZE 5, 7
+#define C0_PWCTL 6, 6
#define C0_BADVADDR 8, 0
+#define C0_PGD 9, 7
#define C0_ENTRYHI 10, 0
#define C0_EPC 14, 0
#define C0_XCONTEXT 20, 0
@@ -630,6 +631,11 @@
static __maybe_unused void build_convert_pte_to_entrylo(u32 **p,
unsigned int reg)
{
+ if (_PAGE_GLOBAL_SHIFT == 0) {
+ /* pte_t is already in EntryLo format */
+ return;
+ }
+
if (cpu_has_rixi && _PAGE_NO_EXEC) {
if (fill_includes_sw_bits) {
UASM_i_ROTR(p, reg, reg, ilog2(_PAGE_GLOBAL));
@@ -808,7 +814,10 @@
if (pgd_reg != -1) {
/* pgd is in pgd_reg */
- UASM_i_MFC0(p, ptr, c0_kscratch(), pgd_reg);
+ if (cpu_has_ldpte)
+ UASM_i_MFC0(p, ptr, C0_PWBASE);
+ else
+ UASM_i_MFC0(p, ptr, c0_kscratch(), pgd_reg);
} else {
#if defined(CONFIG_MIPS_PGD_C0_CONTEXT)
/*
@@ -1007,39 +1016,40 @@
static void build_update_entries(u32 **p, unsigned int tmp, unsigned int ptep)
{
- /*
- * 64bit address support (36bit on a 32bit CPU) in a 32bit
- * Kernel is a special case. Only a few CPUs use it.
- */
- if (config_enabled(CONFIG_PHYS_ADDR_T_64BIT) && !cpu_has_64bits) {
- int pte_off_even = sizeof(pte_t) / 2;
- int pte_off_odd = pte_off_even + sizeof(pte_t);
-#ifdef CONFIG_XPA
- const int scratch = 1; /* Our extra working register */
+ int pte_off_even = 0;
+ int pte_off_odd = sizeof(pte_t);
- uasm_i_addu(p, scratch, 0, ptep);
+#if defined(CONFIG_CPU_MIPS32) && defined(CONFIG_PHYS_ADDR_T_64BIT)
+ /* The low 32 bits of EntryLo is stored in pte_high */
+ pte_off_even += offsetof(pte_t, pte_high);
+ pte_off_odd += offsetof(pte_t, pte_high);
#endif
+
+ if (config_enabled(CONFIG_XPA)) {
uasm_i_lw(p, tmp, pte_off_even, ptep); /* even pte */
- uasm_i_lw(p, ptep, pte_off_odd, ptep); /* odd pte */
UASM_i_ROTR(p, tmp, tmp, ilog2(_PAGE_GLOBAL));
- UASM_i_ROTR(p, ptep, ptep, ilog2(_PAGE_GLOBAL));
UASM_i_MTC0(p, tmp, C0_ENTRYLO0);
- UASM_i_MTC0(p, ptep, C0_ENTRYLO1);
-#ifdef CONFIG_XPA
- uasm_i_lw(p, tmp, 0, scratch);
- uasm_i_lw(p, ptep, sizeof(pte_t), scratch);
- uasm_i_lui(p, scratch, 0xff);
- uasm_i_ori(p, scratch, scratch, 0xffff);
- uasm_i_and(p, tmp, scratch, tmp);
- uasm_i_and(p, ptep, scratch, ptep);
- uasm_i_mthc0(p, tmp, C0_ENTRYLO0);
- uasm_i_mthc0(p, ptep, C0_ENTRYLO1);
-#endif
+
+ if (cpu_has_xpa && !mips_xpa_disabled) {
+ uasm_i_lw(p, tmp, 0, ptep);
+ uasm_i_ext(p, tmp, tmp, 0, 24);
+ uasm_i_mthc0(p, tmp, C0_ENTRYLO0);
+ }
+
+ uasm_i_lw(p, tmp, pte_off_odd, ptep); /* odd pte */
+ UASM_i_ROTR(p, tmp, tmp, ilog2(_PAGE_GLOBAL));
+ UASM_i_MTC0(p, tmp, C0_ENTRYLO1);
+
+ if (cpu_has_xpa && !mips_xpa_disabled) {
+ uasm_i_lw(p, tmp, sizeof(pte_t), ptep);
+ uasm_i_ext(p, tmp, tmp, 0, 24);
+ uasm_i_mthc0(p, tmp, C0_ENTRYLO1);
+ }
return;
}
- UASM_i_LW(p, tmp, 0, ptep); /* get even pte */
- UASM_i_LW(p, ptep, sizeof(pte_t), ptep); /* get odd pte */
+ UASM_i_LW(p, tmp, pte_off_even, ptep); /* get even pte */
+ UASM_i_LW(p, ptep, pte_off_odd, ptep); /* get odd pte */
if (r45k_bvahwbug())
build_tlb_probe_entry(p);
build_convert_pte_to_entrylo(p, tmp);
@@ -1421,6 +1431,108 @@
dump_handler("r4000_tlb_refill", (u32 *)ebase, 64);
}
+static void setup_pw(void)
+{
+ unsigned long pgd_i, pgd_w;
+#ifndef __PAGETABLE_PMD_FOLDED
+ unsigned long pmd_i, pmd_w;
+#endif
+ unsigned long pt_i, pt_w;
+ unsigned long pte_i, pte_w;
+#ifdef CONFIG_MIPS_HUGE_TLB_SUPPORT
+ unsigned long psn;
+
+ psn = ilog2(_PAGE_HUGE); /* bit used to indicate huge page */
+#endif
+ pgd_i = PGDIR_SHIFT; /* 1st level PGD */
+#ifndef __PAGETABLE_PMD_FOLDED
+ pgd_w = PGDIR_SHIFT - PMD_SHIFT + PGD_ORDER;
+
+ pmd_i = PMD_SHIFT; /* 2nd level PMD */
+ pmd_w = PMD_SHIFT - PAGE_SHIFT;
+#else
+ pgd_w = PGDIR_SHIFT - PAGE_SHIFT + PGD_ORDER;
+#endif
+
+ pt_i = PAGE_SHIFT; /* 3rd level PTE */
+ pt_w = PAGE_SHIFT - 3;
+
+ pte_i = ilog2(_PAGE_GLOBAL);
+ pte_w = 0;
+
+#ifndef __PAGETABLE_PMD_FOLDED
+ write_c0_pwfield(pgd_i << 24 | pmd_i << 12 | pt_i << 6 | pte_i);
+ write_c0_pwsize(1 << 30 | pgd_w << 24 | pmd_w << 12 | pt_w << 6 | pte_w);
+#else
+ write_c0_pwfield(pgd_i << 24 | pt_i << 6 | pte_i);
+ write_c0_pwsize(1 << 30 | pgd_w << 24 | pt_w << 6 | pte_w);
+#endif
+
+#ifdef CONFIG_MIPS_HUGE_TLB_SUPPORT
+ write_c0_pwctl(1 << 6 | psn);
+#endif
+ write_c0_kpgd(swapper_pg_dir);
+ kscratch_used_mask |= (1 << 7); /* KScratch6 is used for KPGD */
+}
+
+static void build_loongson3_tlb_refill_handler(void)
+{
+ u32 *p = tlb_handler;
+ struct uasm_label *l = labels;
+ struct uasm_reloc *r = relocs;
+
+ memset(labels, 0, sizeof(labels));
+ memset(relocs, 0, sizeof(relocs));
+ memset(tlb_handler, 0, sizeof(tlb_handler));
+
+ if (check_for_high_segbits) {
+ uasm_i_dmfc0(&p, K0, C0_BADVADDR);
+ uasm_i_dsrl_safe(&p, K1, K0, PGDIR_SHIFT + PGD_ORDER + PAGE_SHIFT - 3);
+ uasm_il_beqz(&p, &r, K1, label_vmalloc);
+ uasm_i_nop(&p);
+
+ uasm_il_bgez(&p, &r, K0, label_large_segbits_fault);
+ uasm_i_nop(&p);
+ uasm_l_vmalloc(&l, p);
+ }
+
+ uasm_i_dmfc0(&p, K1, C0_PGD);
+
+ uasm_i_lddir(&p, K0, K1, 3); /* global page dir */
+#ifndef __PAGETABLE_PMD_FOLDED
+ uasm_i_lddir(&p, K1, K0, 1); /* middle page dir */
+#endif
+ uasm_i_ldpte(&p, K1, 0); /* even */
+ uasm_i_ldpte(&p, K1, 1); /* odd */
+ uasm_i_tlbwr(&p);
+
+ /* restore page mask */
+ if (PM_DEFAULT_MASK >> 16) {
+ uasm_i_lui(&p, K0, PM_DEFAULT_MASK >> 16);
+ uasm_i_ori(&p, K0, K0, PM_DEFAULT_MASK & 0xffff);
+ uasm_i_mtc0(&p, K0, C0_PAGEMASK);
+ } else if (PM_DEFAULT_MASK) {
+ uasm_i_ori(&p, K0, 0, PM_DEFAULT_MASK);
+ uasm_i_mtc0(&p, K0, C0_PAGEMASK);
+ } else {
+ uasm_i_mtc0(&p, 0, C0_PAGEMASK);
+ }
+
+ uasm_i_eret(&p);
+
+ if (check_for_high_segbits) {
+ uasm_l_large_segbits_fault(&l, p);
+ UASM_i_LA(&p, K1, (unsigned long)tlb_do_page_fault_0);
+ uasm_i_jr(&p, K1);
+ uasm_i_nop(&p);
+ }
+
+ uasm_resolve_relocs(relocs, labels);
+ memcpy((void *)(ebase + 0x80), tlb_handler, 0x80);
+ local_flush_icache_range(ebase + 0x80, ebase + 0x100);
+ dump_handler("loongson3_tlb_refill", (u32 *)(ebase + 0x80), 32);
+}
+
extern u32 handle_tlbl[], handle_tlbl_end[];
extern u32 handle_tlbs[], handle_tlbs_end[];
extern u32 handle_tlbm[], handle_tlbm_end[];
@@ -1468,7 +1580,10 @@
} else {
/* PGD in c0_KScratch */
uasm_i_jr(&p, 31);
- UASM_i_MTC0(&p, a0, c0_kscratch(), pgd_reg);
+ if (cpu_has_ldpte)
+ UASM_i_MTC0(&p, a0, C0_PWBASE);
+ else
+ UASM_i_MTC0(&p, a0, c0_kscratch(), pgd_reg);
}
#else
#ifdef CONFIG_SMP
@@ -1523,19 +1638,19 @@
static void
iPTE_SW(u32 **p, struct uasm_reloc **r, unsigned int pte, unsigned int ptr,
- unsigned int mode)
+ unsigned int mode, unsigned int scratch)
{
-#ifdef CONFIG_PHYS_ADDR_T_64BIT
unsigned int hwmode = mode & (_PAGE_VALID | _PAGE_DIRTY);
+ unsigned int swmode = mode & ~hwmode;
- if (!cpu_has_64bits) {
- const int scratch = 1; /* Our extra working register */
-
- uasm_i_lui(p, scratch, (mode >> 16));
+ if (config_enabled(CONFIG_XPA) && !cpu_has_64bits) {
+ uasm_i_lui(p, scratch, swmode >> 16);
uasm_i_or(p, pte, pte, scratch);
- } else
-#endif
- uasm_i_ori(p, pte, pte, mode);
+ BUG_ON(swmode & 0xffff);
+ } else {
+ uasm_i_ori(p, pte, pte, mode);
+ }
+
#ifdef CONFIG_SMP
# ifdef CONFIG_PHYS_ADDR_T_64BIT
if (cpu_has_64bits)
@@ -1554,6 +1669,7 @@
/* no uasm_i_nop needed */
uasm_i_ll(p, pte, sizeof(pte_t) / 2, ptr);
uasm_i_ori(p, pte, pte, hwmode);
+ BUG_ON(hwmode & ~0xffff);
uasm_i_sc(p, pte, sizeof(pte_t) / 2, ptr);
uasm_il_beqz(p, r, pte, label_smp_pgtable_change);
/* no uasm_i_nop needed */
@@ -1575,6 +1691,7 @@
if (!cpu_has_64bits) {
uasm_i_lw(p, pte, sizeof(pte_t) / 2, ptr);
uasm_i_ori(p, pte, pte, hwmode);
+ BUG_ON(hwmode & ~0xffff);
uasm_i_sw(p, pte, sizeof(pte_t) / 2, ptr);
uasm_i_lw(p, pte, 0, ptr);
}
@@ -1615,9 +1732,8 @@
cur = t;
}
uasm_i_andi(p, t, cur,
- (_PAGE_PRESENT | _PAGE_READ) >> _PAGE_PRESENT_SHIFT);
- uasm_i_xori(p, t, t,
- (_PAGE_PRESENT | _PAGE_READ) >> _PAGE_PRESENT_SHIFT);
+ (_PAGE_PRESENT | _PAGE_NO_READ) >> _PAGE_PRESENT_SHIFT);
+ uasm_i_xori(p, t, t, _PAGE_PRESENT >> _PAGE_PRESENT_SHIFT);
uasm_il_bnez(p, r, t, lid);
if (pte == t)
/* You lose the SMP race :-(*/
@@ -1628,11 +1744,11 @@
/* Make PTE valid, store result in PTR. */
static void
build_make_valid(u32 **p, struct uasm_reloc **r, unsigned int pte,
- unsigned int ptr)
+ unsigned int ptr, unsigned int scratch)
{
unsigned int mode = _PAGE_VALID | _PAGE_ACCESSED;
- iPTE_SW(p, r, pte, ptr, mode);
+ iPTE_SW(p, r, pte, ptr, mode, scratch);
}
/*
@@ -1668,12 +1784,12 @@
*/
static void
build_make_write(u32 **p, struct uasm_reloc **r, unsigned int pte,
- unsigned int ptr)
+ unsigned int ptr, unsigned int scratch)
{
unsigned int mode = (_PAGE_ACCESSED | _PAGE_MODIFIED | _PAGE_VALID
| _PAGE_DIRTY);
- iPTE_SW(p, r, pte, ptr, mode);
+ iPTE_SW(p, r, pte, ptr, mode, scratch);
}
/*
@@ -1778,7 +1894,7 @@
build_r3000_tlbchange_handler_head(&p, K0, K1);
build_pte_present(&p, &r, K0, K1, -1, label_nopage_tlbl);
uasm_i_nop(&p); /* load delay */
- build_make_valid(&p, &r, K0, K1);
+ build_make_valid(&p, &r, K0, K1, -1);
build_r3000_tlb_reload_write(&p, &l, &r, K0, K1);
uasm_l_nopage_tlbl(&l, p);
@@ -1809,7 +1925,7 @@
build_r3000_tlbchange_handler_head(&p, K0, K1);
build_pte_writable(&p, &r, K0, K1, -1, label_nopage_tlbs);
uasm_i_nop(&p); /* load delay */
- build_make_write(&p, &r, K0, K1);
+ build_make_write(&p, &r, K0, K1, -1);
build_r3000_tlb_reload_write(&p, &l, &r, K0, K1);
uasm_l_nopage_tlbs(&l, p);
@@ -1840,7 +1956,7 @@
build_r3000_tlbchange_handler_head(&p, K0, K1);
build_pte_modifiable(&p, &r, K0, K1, -1, label_nopage_tlbm);
uasm_i_nop(&p); /* load delay */
- build_make_write(&p, &r, K0, K1);
+ build_make_write(&p, &r, K0, K1, -1);
build_r3000_pte_reload_tlbwi(&p, K0, K1);
uasm_l_nopage_tlbm(&l, p);
@@ -2008,7 +2124,7 @@
}
uasm_l_tlbl_goaround1(&l, p);
}
- build_make_valid(&p, &r, wr.r1, wr.r2);
+ build_make_valid(&p, &r, wr.r1, wr.r2, wr.r3);
build_r4000_tlbchange_handler_tail(&p, &l, &r, wr.r1, wr.r2);
#ifdef CONFIG_MIPS_HUGE_TLB_SUPPORT
@@ -2122,7 +2238,7 @@
build_pte_writable(&p, &r, wr.r1, wr.r2, wr.r3, label_nopage_tlbs);
if (m4kc_tlbp_war())
build_tlb_probe_entry(&p);
- build_make_write(&p, &r, wr.r1, wr.r2);
+ build_make_write(&p, &r, wr.r1, wr.r2, wr.r3);
build_r4000_tlbchange_handler_tail(&p, &l, &r, wr.r1, wr.r2);
#ifdef CONFIG_MIPS_HUGE_TLB_SUPPORT
@@ -2178,7 +2294,7 @@
if (m4kc_tlbp_war())
build_tlb_probe_entry(&p);
/* Present and writable bits set, set accessed and dirty bits. */
- build_make_write(&p, &r, wr.r1, wr.r2);
+ build_make_write(&p, &r, wr.r1, wr.r2, wr.r3);
build_r4000_tlbchange_handler_tail(&p, &l, &r, wr.r1, wr.r2);
#ifdef CONFIG_MIPS_HUGE_TLB_SUPPORT
@@ -2311,9 +2427,7 @@
if (CONFIG_PGTABLE_LEVELS >= 3)
pwsize |= ilog2(PTRS_PER_PMD) << MIPS_PWSIZE_MDW_SHIFT;
- /* If XPA has been enabled, PTEs are 64-bit in size. */
- if (config_enabled(CONFIG_64BITS) || (read_c0_pagegrain() & PG_ELPA))
- pwsize |= 1;
+ pwsize |= ilog2(sizeof(pte_t)/4) << MIPS_PWSIZE_PTEW_SHIFT;
write_c0_pwsize(pwsize);
@@ -2394,6 +2508,9 @@
*/
static int run_once = 0;
+ if (config_enabled(CONFIG_XPA) && !cpu_has_rixi)
+ panic("Kernels supporting XPA currently require CPUs with RIXI");
+
output_pgtable_bits_defines();
check_pabits();
@@ -2437,13 +2554,18 @@
break;
default:
+ if (cpu_has_ldpte)
+ setup_pw();
+
if (!run_once) {
scratch_reg = allocate_kscratch();
build_setup_pgd();
build_r4000_tlb_load_handler();
build_r4000_tlb_store_handler();
build_r4000_tlb_modify_handler();
- if (!cpu_has_local_ebase)
+ if (cpu_has_ldpte)
+ build_loongson3_tlb_refill_handler();
+ else if (!cpu_has_local_ebase)
build_r4000_tlb_refill_handler();
flush_tlb_handlers();
run_once++;
diff --git a/arch/mips/mm/uasm-mips.c b/arch/mips/mm/uasm-mips.c
index b4a83789..9c2220a 100644
--- a/arch/mips/mm/uasm-mips.c
+++ b/arch/mips/mm/uasm-mips.c
@@ -153,6 +153,8 @@
{ insn_xori, M(xori_op, 0, 0, 0, 0, 0), RS | RT | UIMM },
{ insn_xor, M(spec_op, 0, 0, 0, 0, xor_op), RS | RT | RD },
{ insn_yield, M(spec3_op, 0, 0, 0, 0, yield_op), RS | RD },
+ { insn_ldpte, M(lwc2_op, 0, 0, 0, ldpte_op, mult_op), RS | RD },
+ { insn_lddir, M(lwc2_op, 0, 0, 0, lddir_op, mult_op), RS | RT | RD },
{ insn_invalid, 0, 0 }
};
diff --git a/arch/mips/mm/uasm.c b/arch/mips/mm/uasm.c
index 319051c..ad718de 100644
--- a/arch/mips/mm/uasm.c
+++ b/arch/mips/mm/uasm.c
@@ -60,6 +60,7 @@
insn_sltiu, insn_sltu, insn_sra, insn_srl, insn_srlv, insn_subu,
insn_sw, insn_sync, insn_syscall, insn_tlbp, insn_tlbr, insn_tlbwi,
insn_tlbwr, insn_wait, insn_wsbh, insn_xor, insn_xori, insn_yield,
+ insn_lddir, insn_ldpte,
};
struct insn {
@@ -335,6 +336,8 @@
I_u1u2s3(_bbit1);
I_u3u1u2(_lwx)
I_u3u1u2(_ldx)
+I_u1u2(_ldpte)
+I_u2u1u3(_lddir)
#ifdef CONFIG_CPU_CAVIUM_OCTEON
#include <asm/octeon/octeon.h>
diff --git a/arch/mips/mti-malta/malta-setup.c b/arch/mips/mti-malta/malta-setup.c
index 4740c82..33d5ff5 100644
--- a/arch/mips/mti-malta/malta-setup.c
+++ b/arch/mips/mti-malta/malta-setup.c
@@ -248,10 +248,15 @@
#endif
}
+void __init *plat_get_fdt(void)
+{
+ return (void *)__dtb_start;
+}
+
void __init plat_mem_setup(void)
{
unsigned int i;
- void *fdt = __dtb_start;
+ void *fdt = plat_get_fdt();
fdt = malta_dt_shim(fdt);
__dt_setup_arch(fdt);
diff --git a/arch/mips/mti-malta/malta-time.c b/arch/mips/mti-malta/malta-time.c
index b7bf721..7407da0 100644
--- a/arch/mips/mti-malta/malta-time.c
+++ b/arch/mips/mti-malta/malta-time.c
@@ -21,6 +21,7 @@
#include <linux/i8253.h>
#include <linux/init.h>
#include <linux/kernel_stat.h>
+#include <linux/math64.h>
#include <linux/sched.h>
#include <linux/spinlock.h>
#include <linux/interrupt.h>
@@ -72,6 +73,8 @@
{
unsigned long flags;
unsigned int count, start;
+ unsigned char secs1, secs2, ctrl;
+ int secs;
cycle_t giccount = 0, gicstart = 0;
#if defined(CONFIG_KVM_GUEST) && CONFIG_KVM_GUEST_TIMER_FREQ
@@ -81,32 +84,51 @@
local_irq_save(flags);
- /* Start counter exactly on falling edge of update flag. */
- while (CMOS_READ(RTC_REG_A) & RTC_UIP);
- while (!(CMOS_READ(RTC_REG_A) & RTC_UIP));
-
- /* Initialize counters. */
- start = read_c0_count();
- if (gic_present) {
+ if (gic_present)
gic_start_count();
- gicstart = gic_read_count();
- }
- /* Read counter exactly on falling edge of update flag. */
+ /*
+ * Read counters exactly on rising edge of update flag.
+ * This helps get an accurate reading under virtualisation.
+ */
while (CMOS_READ(RTC_REG_A) & RTC_UIP);
while (!(CMOS_READ(RTC_REG_A) & RTC_UIP));
+ start = read_c0_count();
+ if (gic_present)
+ gicstart = gic_read_count();
+ /* Wait for falling edge before reading RTC. */
+ while (CMOS_READ(RTC_REG_A) & RTC_UIP);
+ secs1 = CMOS_READ(RTC_SECONDS);
+
+ /* Read counters again exactly on rising edge of update flag. */
+ while (!(CMOS_READ(RTC_REG_A) & RTC_UIP));
count = read_c0_count();
if (gic_present)
giccount = gic_read_count();
+ /* Wait for falling edge before reading RTC again. */
+ while (CMOS_READ(RTC_REG_A) & RTC_UIP);
+ secs2 = CMOS_READ(RTC_SECONDS);
+
+ ctrl = CMOS_READ(RTC_CONTROL);
+
local_irq_restore(flags);
+ if (!(ctrl & RTC_DM_BINARY) || RTC_ALWAYS_BCD) {
+ secs1 = bcd2bin(secs1);
+ secs2 = bcd2bin(secs2);
+ }
+ secs = secs2 - secs1;
+ if (secs < 1)
+ secs += 60;
+
count -= start;
+ count /= secs;
mips_hpt_frequency = count;
if (gic_present) {
- giccount -= gicstart;
+ giccount = div_u64(giccount - gicstart, secs);
gic_frequency = giccount;
}
}
diff --git a/arch/mips/mti-sead3/sead3-setup.c b/arch/mips/mti-sead3/sead3-setup.c
index e43f480..9f2f9b2 100644
--- a/arch/mips/mti-sead3/sead3-setup.c
+++ b/arch/mips/mti-sead3/sead3-setup.c
@@ -83,6 +83,11 @@
}
}
+void __init *plat_get_fdt(void)
+{
+ return (void *)__dtb_start;
+}
+
void __init plat_mem_setup(void)
{
/* allow command line/bootloader env to override memory size in DT */
diff --git a/arch/mips/netlogic/common/reset.S b/arch/mips/netlogic/common/reset.S
index edbab9b..c474981 100644
--- a/arch/mips/netlogic/common/reset.S
+++ b/arch/mips/netlogic/common/reset.S
@@ -50,7 +50,6 @@
#include <asm/netlogic/xlp-hal/sys.h>
#include <asm/netlogic/xlp-hal/cpucontrol.h>
-#define CP0_EBASE $15
#define SYS_CPU_COHERENT_BASE CKSEG1ADDR(XLP_DEFAULT_IO_BASE) + \
XLP_IO_SYS_OFFSET(0) + XLP_IO_PCI_HDRSZ + \
SYS_CPU_NONCOHERENT_MODE * 4
@@ -92,7 +91,7 @@
* registers. On XLPII CPUs, usual cache instructions work.
*/
.macro xlp_flush_l1_dcache
- mfc0 t0, CP0_EBASE, 0
+ mfc0 t0, CP0_PRID
andi t0, t0, PRID_IMP_MASK
slt t1, t0, 0x1200
beqz t1, 15f
@@ -171,7 +170,7 @@
nop
1: /* Entry point on core wakeup */
- mfc0 t0, CP0_EBASE, 0 /* processor ID */
+ mfc0 t0, CP0_PRID /* processor ID */
andi t0, PRID_IMP_MASK
li t1, 0x1500 /* XLP 9xx */
beq t0, t1, 2f /* does not need to set coherent */
@@ -182,8 +181,8 @@
nop
/* set bit in SYS coherent register for the core */
- mfc0 t0, CP0_EBASE, 1
- mfc0 t1, CP0_EBASE, 1
+ mfc0 t0, CP0_EBASE
+ mfc0 t1, CP0_EBASE
srl t1, 5
andi t1, 0x3 /* t1 <- node */
li t2, 0x40000
@@ -232,7 +231,7 @@
* NOTE: All GPR contents are lost after the mtcr above!
*/
- mfc0 v0, CP0_EBASE, 1
+ mfc0 v0, CP0_EBASE
andi v0, 0x3ff /* v0 <- node/core */
/*
diff --git a/arch/mips/netlogic/common/smpboot.S b/arch/mips/netlogic/common/smpboot.S
index 805355b..f0cc4c9 100644
--- a/arch/mips/netlogic/common/smpboot.S
+++ b/arch/mips/netlogic/common/smpboot.S
@@ -48,8 +48,6 @@
#include <asm/netlogic/xlp-hal/sys.h>
#include <asm/netlogic/xlp-hal/cpucontrol.h>
-#define CP0_EBASE $15
-
.set noreorder
.set noat
.set arch=xlr /* for mfcr/mtcr, XLR is sufficient */
@@ -86,7 +84,7 @@
PTR_L gp, 0(t1)
/* a0 has the processor id */
- mfc0 a0, CP0_EBASE, 1
+ mfc0 a0, CP0_EBASE
andi a0, 0x3ff /* a0 <- node/core */
PTR_LA t0, nlm_early_init_secondary
jalr t0
diff --git a/arch/mips/netlogic/xlp/nlm_hal.c b/arch/mips/netlogic/xlp/nlm_hal.c
index 80ec929..25ee694 100644
--- a/arch/mips/netlogic/xlp/nlm_hal.c
+++ b/arch/mips/netlogic/xlp/nlm_hal.c
@@ -58,7 +58,7 @@
nodep->coremask = 1; /* node 0, boot cpu */
nodep->sysbase = nlm_get_sys_regbase(node);
nodep->picbase = nlm_get_pic_regbase(node);
- nodep->ebase = read_c0_ebase() & (~((1 << 12) - 1));
+ nodep->ebase = read_c0_ebase() & MIPS_EBASE_BASE;
if (cpu_is_xlp9xx())
nodep->socbus = xlp9xx_get_socbus(node);
else
diff --git a/arch/mips/netlogic/xlr/setup.c b/arch/mips/netlogic/xlr/setup.c
index d118b9a..72ceddc 100644
--- a/arch/mips/netlogic/xlr/setup.c
+++ b/arch/mips/netlogic/xlr/setup.c
@@ -168,7 +168,7 @@
nodep = nlm_current_node();
nodep->picbase = nlm_mmio_base(NETLOGIC_IO_PIC_OFFSET);
- nodep->ebase = read_c0_ebase() & (~((1 << 12) - 1));
+ nodep->ebase = read_c0_ebase() & MIPS_EBASE_BASE;
spin_lock_init(&nodep->piclock);
}
diff --git a/arch/mips/oprofile/common.c b/arch/mips/oprofile/common.c
index 3c9ec3d..2f33992 100644
--- a/arch/mips/oprofile/common.c
+++ b/arch/mips/oprofile/common.c
@@ -77,7 +77,7 @@
struct op_mips_model *lmodel = NULL;
int res;
- switch (current_cpu_type()) {
+ switch (boot_cpu_type()) {
case CPU_5KC:
case CPU_M14KC:
case CPU_M14KEC:
diff --git a/arch/mips/oprofile/op_model_mipsxx.c b/arch/mips/oprofile/op_model_mipsxx.c
index 8f988a6..45cb274 100644
--- a/arch/mips/oprofile/op_model_mipsxx.c
+++ b/arch/mips/oprofile/op_model_mipsxx.c
@@ -269,11 +269,9 @@
return handled;
}
-#define M_CONFIG1_PC (1 << 4)
-
static inline int __n_counters(void)
{
- if (!(read_c0_config1() & M_CONFIG1_PC))
+ if (!cpu_has_perf)
return 0;
if (!(read_c0_perfctrl0() & M_PERFCTL_MORE))
return 1;
diff --git a/arch/mips/pci/fixup-lantiq.c b/arch/mips/pci/fixup-lantiq.c
index c2ce41e..2b5427d 100644
--- a/arch/mips/pci/fixup-lantiq.c
+++ b/arch/mips/pci/fixup-lantiq.c
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2012 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2012 John Crispin <john@phrozen.org>
*/
#include <linux/of_irq.h>
diff --git a/arch/mips/pci/ops-lantiq.c b/arch/mips/pci/ops-lantiq.c
index e5738ee..f51e108 100644
--- a/arch/mips/pci/ops-lantiq.c
+++ b/arch/mips/pci/ops-lantiq.c
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2010 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2010 John Crispin <john@phrozen.org>
*/
#include <linux/types.h>
diff --git a/arch/mips/pci/pci-alchemy.c b/arch/mips/pci/pci-alchemy.c
index 2895263..c8994c1 100644
--- a/arch/mips/pci/pci-alchemy.c
+++ b/arch/mips/pci/pci-alchemy.c
@@ -76,7 +76,7 @@
unsigned long old_ctx;
/* Save old context and create impossible VPN2 value */
- old_ctx = read_c0_entryhi() & 0xff;
+ old_ctx = read_c0_entryhi() & MIPS_ENTRYHI_ASID;
old_pagemask = read_c0_pagemask();
write_c0_index(entry);
write_c0_pagemask(pagemask);
diff --git a/arch/mips/pci/pci-ip32.c b/arch/mips/pci/pci-ip32.c
index b1e061f..7ae89d0 100644
--- a/arch/mips/pci/pci-ip32.c
+++ b/arch/mips/pci/pci-ip32.c
@@ -116,7 +116,6 @@
.pci_ops = &mace_pci_ops,
.mem_resource = &mace_pci_mem_resource,
.io_resource = &mace_pci_io_resource,
- .iommu = 0,
.mem_offset = MACE_PCI_MEM_OFFSET,
.io_offset = 0,
.io_map_base = CKSEG1ADDR(MACEPCI_LOW_IO),
diff --git a/arch/mips/pci/pci-lantiq.c b/arch/mips/pci/pci-lantiq.c
index 6a15dbd..b9deab1 100644
--- a/arch/mips/pci/pci-lantiq.c
+++ b/arch/mips/pci/pci-lantiq.c
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2010 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2010 John Crispin <john@phrozen.org>
*/
#include <linux/types.h>
diff --git a/arch/mips/pci/pci-lantiq.h b/arch/mips/pci/pci-lantiq.h
index 66bf6cd..0cc7125 100644
--- a/arch/mips/pci/pci-lantiq.h
+++ b/arch/mips/pci/pci-lantiq.h
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2010 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2010 John Crispin <john@phrozen.org>
*/
#ifndef _LTQ_PCI_H__
diff --git a/arch/mips/pci/pci-mt7620.c b/arch/mips/pci/pci-mt7620.c
index 1ae932c2..6ce8162 100644
--- a/arch/mips/pci/pci-mt7620.c
+++ b/arch/mips/pci/pci-mt7620.c
@@ -2,7 +2,7 @@
* Ralink MT7620A SoC PCI support
*
* Copyright (C) 2007-2013 Bruce Chang (Mediatek)
- * Copyright (C) 2013-2016 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2013-2016 John Crispin <john@phrozen.org>
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License version 2 as published
diff --git a/arch/mips/pci/pci-rt2880.c b/arch/mips/pci/pci-rt2880.c
index a245cad..f2a1050 100644
--- a/arch/mips/pci/pci-rt2880.c
+++ b/arch/mips/pci/pci-rt2880.c
@@ -1,7 +1,7 @@
/*
* Ralink RT288x SoC PCI register definitions
*
- * Copyright (C) 2009 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2009 John Crispin <john@phrozen.org>
* Copyright (C) 2009 Gabor Juhos <juhosg@openwrt.org>
*
* Parts of this file are based on Ralink's 2.6.21 BSP
diff --git a/arch/mips/pci/pci.c b/arch/mips/pci/pci.c
index b8a0bf5..f1b11f0 100644
--- a/arch/mips/pci/pci.c
+++ b/arch/mips/pci/pci.c
@@ -83,9 +83,6 @@
LIST_HEAD(resources);
struct pci_bus *bus;
- if (!hose->iommu)
- PCI_DMA_BUS_IS_PHYS = 1;
-
if (hose->get_busno && pci_has_flag(PCI_PROBE_ONLY))
next_busno = (*hose->get_busno)();
diff --git a/arch/mips/pic32/pic32mzda/time.c b/arch/mips/pic32/pic32mzda/time.c
index ca6a62b..62a0a78 100644
--- a/arch/mips/pic32/pic32mzda/time.c
+++ b/arch/mips/pic32/pic32mzda/time.c
@@ -11,13 +11,12 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
* for more details.
*/
-#include <linux/clk.h>
#include <linux/clk-provider.h>
#include <linux/clocksource.h>
#include <linux/init.h>
+#include <linux/irqdomain.h>
#include <linux/of.h>
#include <linux/of_irq.h>
-#include <linux/irqdomain.h>
#include <asm/time.h>
@@ -58,16 +57,12 @@
void __init plat_time_init(void)
{
- struct clk *clk;
+ unsigned long rate = pic32_get_pbclk(7);
of_clk_init(NULL);
- clk = clk_get_sys("cpu_clk", NULL);
- if (IS_ERR(clk))
- panic("unable to get CPU clock, err=%ld", PTR_ERR(clk));
- clk_prepare_enable(clk);
- pr_info("CPU Clock: %ldMHz\n", clk_get_rate(clk) / 1000000);
- mips_hpt_frequency = clk_get_rate(clk) / 2;
+ pr_info("CPU Clock: %ldMHz\n", rate / 1000000);
+ mips_hpt_frequency = rate / 2;
clocksource_probe();
}
diff --git a/arch/mips/pistachio/init.c b/arch/mips/pistachio/init.c
index 96ba2cc..956c92e 100644
--- a/arch/mips/pistachio/init.c
+++ b/arch/mips/pistachio/init.c
@@ -2,6 +2,7 @@
* Pistachio platform setup
*
* Copyright (C) 2014 Google, Inc.
+ * Copyright (C) 2016 Imagination Technologies
*
* This program is free software; you can redistribute it and/or modify it
* under the terms and conditions of the GNU General Public License,
@@ -9,6 +10,7 @@
*/
#include <linux/init.h>
+#include <linux/io.h>
#include <linux/kernel.h>
#include <linux/of_address.h>
#include <linux/of_fdt.h>
@@ -24,9 +26,38 @@
#include <asm/smp-ops.h>
#include <asm/traps.h>
+/*
+ * Core revision register decoding
+ * Bits 23 to 20: Major rev
+ * Bits 15 to 8: Minor rev
+ * Bits 7 to 0: Maintenance rev
+ */
+#define PISTACHIO_CORE_REV_REG 0xB81483D0
+#define PISTACHIO_CORE_REV_A1 0x00100006
+#define PISTACHIO_CORE_REV_B0 0x00100106
+
const char *get_system_type(void)
{
- return "IMG Pistachio SoC";
+ u32 core_rev;
+ const char *sys_type;
+
+ core_rev = __raw_readl((const void *)PISTACHIO_CORE_REV_REG);
+
+ switch (core_rev) {
+ case PISTACHIO_CORE_REV_B0:
+ sys_type = "IMG Pistachio SoC (B0)";
+ break;
+
+ case PISTACHIO_CORE_REV_A1:
+ sys_type = "IMG Pistachio SoC (A1)";
+ break;
+
+ default:
+ sys_type = "IMG Pistachio SoC";
+ break;
+ }
+
+ return sys_type;
}
static void __init plat_setup_iocoherency(void)
@@ -109,6 +140,8 @@
mips_cm_probe();
mips_cpc_probe();
register_cps_smp_ops();
+
+ pr_info("SoC Type: %s\n", get_system_type());
}
void __init prom_free_prom_memory(void)
diff --git a/arch/mips/pmcs-msp71xx/msp_setup.c b/arch/mips/pmcs-msp71xx/msp_setup.c
index 9d293b3..a63b736 100644
--- a/arch/mips/pmcs-msp71xx/msp_setup.c
+++ b/arch/mips/pmcs-msp71xx/msp_setup.c
@@ -118,7 +118,7 @@
/* No chip-specific reset code, just jump to the ROM reset vector */
set_c0_status(ST0_BEV | ST0_ERL);
change_c0_config(CONF_CM_CMASK, CONF_CM_UNCACHED);
- flush_cache_all();
+ __flush_cache_all();
write_c0_wired(0);
__asm__ __volatile__("jr\t%0"::"r"(0xbfc00000));
diff --git a/arch/mips/pnx833x/common/setup.c b/arch/mips/pnx833x/common/setup.c
index 99b4d94..8a7443b 100644
--- a/arch/mips/pnx833x/common/setup.c
+++ b/arch/mips/pnx833x/common/setup.c
@@ -38,9 +38,6 @@
int __init plat_mem_setup(void)
{
- /* fake pci bus to avoid bounce buffers */
- PCI_DMA_BUS_IS_PHYS = 1;
-
/* set mips clock to 320MHz */
#if defined(CONFIG_SOC_PNX8335)
PNX8335_WRITEFIELD(0x17, CLOCK_PLL_CPU_CTL, FREQ);
diff --git a/arch/mips/ralink/Makefile b/arch/mips/ralink/Makefile
index 0d1795a..fe34715 100644
--- a/arch/mips/ralink/Makefile
+++ b/arch/mips/ralink/Makefile
@@ -4,7 +4,7 @@
# Makefile for the Ralink common stuff
#
# Copyright (C) 2009-2011 Gabor Juhos <juhosg@openwrt.org>
-# Copyright (C) 2013 John Crispin <blogic@openwrt.org>
+# Copyright (C) 2013 John Crispin <john@phrozen.org>
obj-y := prom.o of.o reset.o
diff --git a/arch/mips/ralink/bootrom.c b/arch/mips/ralink/bootrom.c
index 5403468..e1fa597 100644
--- a/arch/mips/ralink/bootrom.c
+++ b/arch/mips/ralink/bootrom.c
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2013 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2013 John Crispin <john@phrozen.org>
*/
#include <linux/debugfs.h>
diff --git a/arch/mips/ralink/cevt-rt3352.c b/arch/mips/ralink/cevt-rt3352.c
index e46f91f..3ad0b07 100644
--- a/arch/mips/ralink/cevt-rt3352.c
+++ b/arch/mips/ralink/cevt-rt3352.c
@@ -3,7 +3,7 @@
* License. See the file "COPYING" in the main directory of this archive
* for more details.
*
- * Copyright (C) 2013 by John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2013 by John Crispin <john@phrozen.org>
*/
#include <linux/clockchips.h>
diff --git a/arch/mips/ralink/clk.c b/arch/mips/ralink/clk.c
index 25c4a61..ebaa7cc 100644
--- a/arch/mips/ralink/clk.c
+++ b/arch/mips/ralink/clk.c
@@ -4,7 +4,7 @@
* by the Free Software Foundation.
*
* Copyright (C) 2011 Gabor Juhos <juhosg@openwrt.org>
- * Copyright (C) 2013 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2013 John Crispin <john@phrozen.org>
*/
#include <linux/kernel.h>
diff --git a/arch/mips/ralink/common.h b/arch/mips/ralink/common.h
index 8e7d8e6..b8245d0 100644
--- a/arch/mips/ralink/common.h
+++ b/arch/mips/ralink/common.h
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2013 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2013 John Crispin <john@phrozen.org>
*/
#ifndef _RALINK_COMMON_H__
diff --git a/arch/mips/ralink/ill_acc.c b/arch/mips/ralink/ill_acc.c
index e10d10b..765d5ba 100644
--- a/arch/mips/ralink/ill_acc.c
+++ b/arch/mips/ralink/ill_acc.c
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2013 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2013 John Crispin <john@phrozen.org>
*/
#include <linux/interrupt.h>
diff --git a/arch/mips/ralink/irq-gic.c b/arch/mips/ralink/irq-gic.c
index 50d6c55..2058280 100644
--- a/arch/mips/ralink/irq-gic.c
+++ b/arch/mips/ralink/irq-gic.c
@@ -4,7 +4,7 @@
* by the Free Software Foundation.
*
* Copyright (C) 2015 Nikolay Martynov <mar.kolya@gmail.com>
- * Copyright (C) 2015 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2015 John Crispin <john@phrozen.org>
*/
#include <linux/init.h>
diff --git a/arch/mips/ralink/irq.c b/arch/mips/ralink/irq.c
index 4cf77f3..4911c14 100644
--- a/arch/mips/ralink/irq.c
+++ b/arch/mips/ralink/irq.c
@@ -4,7 +4,7 @@
* by the Free Software Foundation.
*
* Copyright (C) 2009 Gabor Juhos <juhosg@openwrt.org>
- * Copyright (C) 2013 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2013 John Crispin <john@phrozen.org>
*/
#include <linux/io.h>
diff --git a/arch/mips/ralink/mt7620.c b/arch/mips/ralink/mt7620.c
index 0d3d1a9..88b82fe 100644
--- a/arch/mips/ralink/mt7620.c
+++ b/arch/mips/ralink/mt7620.c
@@ -7,7 +7,7 @@
*
* Copyright (C) 2008-2011 Gabor Juhos <juhosg@openwrt.org>
* Copyright (C) 2008 Imre Kaloz <kaloz@openwrt.org>
- * Copyright (C) 2013 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2013 John Crispin <john@phrozen.org>
*/
#include <linux/kernel.h>
@@ -581,11 +581,14 @@
(rev & CHIP_REV_ECO_MASK));
cfg0 = __raw_readl(sysc + SYSC_REG_SYSTEM_CONFIG0);
- if (is_mt76x8())
+ if (is_mt76x8()) {
dram_type = cfg0 & DRAM_TYPE_MT7628_MASK;
- else
+ } else {
dram_type = (cfg0 >> SYSCFG0_DRAM_TYPE_SHIFT) &
SYSCFG0_DRAM_TYPE_MASK;
+ if (dram_type == SYSCFG0_DRAM_TYPE_UNKNOWN)
+ dram_type = SYSCFG0_DRAM_TYPE_SDRAM;
+ }
soc_info->mem_base = MT7620_DRAM_BASE;
if (is_mt76x8())
diff --git a/arch/mips/ralink/mt7621.c b/arch/mips/ralink/mt7621.c
index e9b9fa3..a45bbbe 100644
--- a/arch/mips/ralink/mt7621.c
+++ b/arch/mips/ralink/mt7621.c
@@ -4,7 +4,7 @@
* by the Free Software Foundation.
*
* Copyright (C) 2015 Nikolay Martynov <mar.kolya@gmail.com>
- * Copyright (C) 2015 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2015 John Crispin <john@phrozen.org>
*/
#include <linux/kernel.h>
diff --git a/arch/mips/ralink/of.c b/arch/mips/ralink/of.c
index f9eda5d..0aa67a2 100644
--- a/arch/mips/ralink/of.c
+++ b/arch/mips/ralink/of.c
@@ -5,7 +5,7 @@
*
* Copyright (C) 2008 Imre Kaloz <kaloz@openwrt.org>
* Copyright (C) 2008-2009 Gabor Juhos <juhosg@openwrt.org>
- * Copyright (C) 2013 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2013 John Crispin <john@phrozen.org>
*/
#include <linux/io.h>
diff --git a/arch/mips/ralink/prom.c b/arch/mips/ralink/prom.c
index 39a9142f..5a73c5e 100644
--- a/arch/mips/ralink/prom.c
+++ b/arch/mips/ralink/prom.c
@@ -5,7 +5,7 @@
*
* Copyright (C) 2009 Gabor Juhos <juhosg@openwrt.org>
* Copyright (C) 2010 Joonas Lahtinen <joonas.lahtinen@gmail.com>
- * Copyright (C) 2013 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2013 John Crispin <john@phrozen.org>
*/
#include <linux/string.h>
diff --git a/arch/mips/ralink/reset.c b/arch/mips/ralink/reset.c
index ee117c4..64543d6 100644
--- a/arch/mips/ralink/reset.c
+++ b/arch/mips/ralink/reset.c
@@ -5,7 +5,7 @@
*
* Copyright (C) 2008-2009 Gabor Juhos <juhosg@openwrt.org>
* Copyright (C) 2008 Imre Kaloz <kaloz@openwrt.org>
- * Copyright (C) 2013 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2013 John Crispin <john@phrozen.org>
*/
#include <linux/pm.h>
@@ -61,7 +61,7 @@
return ralink_deassert_device(rcdev, id);
}
-static struct reset_control_ops reset_ops = {
+static const struct reset_control_ops reset_ops = {
.reset = ralink_reset_device,
.assert = ralink_assert_device,
.deassert = ralink_deassert_device,
diff --git a/arch/mips/ralink/rt288x.c b/arch/mips/ralink/rt288x.c
index 3c84166..285796e 100644
--- a/arch/mips/ralink/rt288x.c
+++ b/arch/mips/ralink/rt288x.c
@@ -7,7 +7,7 @@
*
* Copyright (C) 2008-2011 Gabor Juhos <juhosg@openwrt.org>
* Copyright (C) 2008 Imre Kaloz <kaloz@openwrt.org>
- * Copyright (C) 2013 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2013 John Crispin <john@phrozen.org>
*/
#include <linux/kernel.h>
diff --git a/arch/mips/ralink/rt305x.c b/arch/mips/ralink/rt305x.c
index d7c4ba4..c8a28c4b 100644
--- a/arch/mips/ralink/rt305x.c
+++ b/arch/mips/ralink/rt305x.c
@@ -7,7 +7,7 @@
*
* Copyright (C) 2008-2011 Gabor Juhos <juhosg@openwrt.org>
* Copyright (C) 2008 Imre Kaloz <kaloz@openwrt.org>
- * Copyright (C) 2013 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2013 John Crispin <john@phrozen.org>
*/
#include <linux/kernel.h>
diff --git a/arch/mips/ralink/rt3883.c b/arch/mips/ralink/rt3883.c
index fafec94..4cef916 100644
--- a/arch/mips/ralink/rt3883.c
+++ b/arch/mips/ralink/rt3883.c
@@ -7,7 +7,7 @@
*
* Copyright (C) 2008 Imre Kaloz <kaloz@openwrt.org>
* Copyright (C) 2008-2011 Gabor Juhos <juhosg@openwrt.org>
- * Copyright (C) 2013 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2013 John Crispin <john@phrozen.org>
*/
#include <linux/kernel.h>
diff --git a/arch/mips/ralink/timer-gic.c b/arch/mips/ralink/timer-gic.c
index 5b4f186..069771d 100644
--- a/arch/mips/ralink/timer-gic.c
+++ b/arch/mips/ralink/timer-gic.c
@@ -4,7 +4,7 @@
* by the Free Software Foundation.
*
* Copyright (C) 2015 Nikolay Martynov <mar.kolya@gmail.com>
- * Copyright (C) 2015 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2015 John Crispin <john@phrozen.org>
*/
#include <linux/init.h>
diff --git a/arch/mips/ralink/timer.c b/arch/mips/ralink/timer.c
index 82c72a1..b0343ff 100644
--- a/arch/mips/ralink/timer.c
+++ b/arch/mips/ralink/timer.c
@@ -3,7 +3,7 @@
* under the terms of the GNU General Public License version 2 as published
* by the Free Software Foundation.
*
- * Copyright (C) 2013 John Crispin <blogic@openwrt.org>
+ * Copyright (C) 2013 John Crispin <john@phrozen.org>
*/
#include <linux/module.h>
@@ -180,5 +180,5 @@
module_platform_driver(rt_timer_driver);
MODULE_DESCRIPTION("Ralink RT2880 timer");
-MODULE_AUTHOR("John Crispin <blogic@openwrt.org");
+MODULE_AUTHOR("John Crispin <john@phrozen.org");
MODULE_LICENSE("GPL");
diff --git a/arch/mips/sibyte/Kconfig b/arch/mips/sibyte/Kconfig
index cb9a095..707b884 100644
--- a/arch/mips/sibyte/Kconfig
+++ b/arch/mips/sibyte/Kconfig
@@ -143,7 +143,8 @@
config SIBYTE_BUS_WATCHER
bool "Support for Bus Watcher statistics"
depends on SIBYTE_SB1xxx_SOC && \
- (SIBYTE_BCM112X || SIBYTE_SB1250)
+ (SIBYTE_BCM112X || SIBYTE_SB1250 || \
+ SIBYTE_BCM1x55 || SIBYTE_BCM1x80)
help
Handle and keep statistics on the bus error interrupts (COR_ECC,
BAD_ECC, IO_BUS).
diff --git a/arch/mips/vdso/Makefile b/arch/mips/vdso/Makefile
index ee3617c..b369509 100644
--- a/arch/mips/vdso/Makefile
+++ b/arch/mips/vdso/Makefile
@@ -50,13 +50,17 @@
cmd_vdsold = $(CC) $(c_flags) $(VDSO_LDFLAGS) \
-Wl,-T $(filter %.lds,$^) $(filter %.o,$^) -o $@
+# Strip rule for the raw .so files
+$(obj)/%.so.raw: OBJCOPYFLAGS := -S
+$(obj)/%.so.raw: $(obj)/%.so.dbg.raw FORCE
+ $(call if_changed,objcopy)
+
hostprogs-y := genvdso
quiet_cmd_genvdso = GENVDSO $@
define cmd_genvdso
- cp $< $(<:%.dbg=%) && \
- $(OBJCOPY) -S $< $(<:%.dbg=%) && \
- $(obj)/genvdso $< $(<:%.dbg=%) $@ $(VDSO_NAME)
+ $(foreach file,$(filter %.raw,$^),cp $(file) $(file:%.raw=%) &&) \
+ $(obj)/genvdso $(<:%.raw=%) $(<:%.dbg.raw=%) $@ $(VDSO_NAME)
endef
#
@@ -66,7 +70,10 @@
native-abi := $(filter -mabi=%,$(KBUILD_CFLAGS))
targets += $(obj-vdso-y)
-targets += vdso.lds vdso.so.dbg vdso.so vdso-image.c
+targets += vdso.lds
+targets += vdso.so.dbg.raw vdso.so.raw
+targets += vdso.so.dbg vdso.so
+targets += vdso-image.c
obj-vdso := $(obj-vdso-y:%.o=$(obj)/%.o)
@@ -75,10 +82,11 @@
$(obj)/vdso.lds: KBUILD_CPPFLAGS := $(native-abi)
-$(obj)/vdso.so.dbg: $(obj)/vdso.lds $(obj-vdso) FORCE
+$(obj)/vdso.so.dbg.raw: $(obj)/vdso.lds $(obj-vdso) FORCE
$(call if_changed,vdsold)
-$(obj)/vdso-image.c: $(obj)/vdso.so.dbg $(obj)/genvdso FORCE
+$(obj)/vdso-image.c: $(obj)/vdso.so.dbg.raw $(obj)/vdso.so.raw \
+ $(obj)/genvdso FORCE
$(call if_changed,genvdso)
obj-y += vdso-image.o
@@ -89,7 +97,10 @@
# Define these outside the ifdef to ensure they are picked up by clean.
targets += $(obj-vdso-y:%.o=%-o32.o)
-targets += vdso-o32.lds vdso-o32.so.dbg vdso-o32.so vdso-o32-image.c
+targets += vdso-o32.lds
+targets += vdso-o32.so.dbg.raw vdso-o32.so.raw
+targets += vdso-o32.so.dbg vdso-o32.so
+targets += vdso-o32-image.c
ifdef CONFIG_MIPS32_O32
@@ -109,11 +120,12 @@
$(obj)/vdso-o32.lds: $(src)/vdso.lds.S FORCE
$(call if_changed_dep,cpp_lds_S)
-$(obj)/vdso-o32.so.dbg: $(obj)/vdso-o32.lds $(obj-vdso-o32) FORCE
+$(obj)/vdso-o32.so.dbg.raw: $(obj)/vdso-o32.lds $(obj-vdso-o32) FORCE
$(call if_changed,vdsold)
$(obj)/vdso-o32-image.c: VDSO_NAME := o32
-$(obj)/vdso-o32-image.c: $(obj)/vdso-o32.so.dbg $(obj)/genvdso FORCE
+$(obj)/vdso-o32-image.c: $(obj)/vdso-o32.so.dbg.raw $(obj)/vdso-o32.so.raw \
+ $(obj)/genvdso FORCE
$(call if_changed,genvdso)
obj-y += vdso-o32-image.o
@@ -125,7 +137,10 @@
#
targets += $(obj-vdso-y:%.o=%-n32.o)
-targets += vdso-n32.lds vdso-n32.so.dbg vdso-n32.so vdso-n32-image.c
+targets += vdso-n32.lds
+targets += vdso-n32.so.dbg.raw vdso-n32.so.raw
+targets += vdso-n32.so.dbg vdso-n32.so
+targets += vdso-n32-image.c
ifdef CONFIG_MIPS32_N32
@@ -145,11 +160,12 @@
$(obj)/vdso-n32.lds: $(src)/vdso.lds.S FORCE
$(call if_changed_dep,cpp_lds_S)
-$(obj)/vdso-n32.so.dbg: $(obj)/vdso-n32.lds $(obj-vdso-n32) FORCE
+$(obj)/vdso-n32.so.dbg.raw: $(obj)/vdso-n32.lds $(obj-vdso-n32) FORCE
$(call if_changed,vdsold)
$(obj)/vdso-n32-image.c: VDSO_NAME := n32
-$(obj)/vdso-n32-image.c: $(obj)/vdso-n32.so.dbg $(obj)/genvdso FORCE
+$(obj)/vdso-n32-image.c: $(obj)/vdso-n32.so.dbg.raw $(obj)/vdso-n32.so.raw \
+ $(obj)/genvdso FORCE
$(call if_changed,genvdso)
obj-y += vdso-n32-image.o
diff --git a/arch/mips/vr41xx/common/pmu.c b/arch/mips/vr41xx/common/pmu.c
index d7f7558..39a0db3 100644
--- a/arch/mips/vr41xx/common/pmu.c
+++ b/arch/mips/vr41xx/common/pmu.c
@@ -73,7 +73,7 @@
default:
set_c0_status(ST0_BEV | ST0_ERL);
change_c0_config(CONF_CM_CMASK, CONF_CM_UNCACHED);
- flush_cache_all();
+ __flush_cache_all();
write_c0_wired(0);
__asm__("jr %0"::"r"(0xbfc00000));
break;
diff --git a/arch/mn10300/Kconfig b/arch/mn10300/Kconfig
index 06ddb55..9627e81 100644
--- a/arch/mn10300/Kconfig
+++ b/arch/mn10300/Kconfig
@@ -1,5 +1,6 @@
config MN10300
def_bool y
+ select HAVE_EXIT_THREAD
select HAVE_OPROFILE
select HAVE_UID16
select GENERIC_IRQ_SHOW
diff --git a/arch/mn10300/configs/asb2364_defconfig b/arch/mn10300/configs/asb2364_defconfig
index fbb96ae..cd0a6cb 100644
--- a/arch/mn10300/configs/asb2364_defconfig
+++ b/arch/mn10300/configs/asb2364_defconfig
@@ -11,7 +11,6 @@
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CGROUP_CPUACCT=y
-CONFIG_RESOURCE_COUNTERS=y
CONFIG_RELAY=y
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
CONFIG_EXPERT=y
diff --git a/arch/mn10300/include/asm/fpu.h b/arch/mn10300/include/asm/fpu.h
index 738ff72..a47e995 100644
--- a/arch/mn10300/include/asm/fpu.h
+++ b/arch/mn10300/include/asm/fpu.h
@@ -76,11 +76,9 @@
preempt_enable();
}
-static inline void exit_fpu(void)
+static inline void exit_fpu(struct task_struct *tsk)
{
#ifdef CONFIG_LAZY_SAVE_FPU
- struct task_struct *tsk = current;
-
preempt_disable();
if (fpu_state_owner == tsk)
fpu_state_owner = NULL;
@@ -123,7 +121,7 @@
static inline void fpu_save(struct fpu_state_struct *s) {}
static inline void fpu_kill_state(struct task_struct *tsk) {}
static inline void unlazy_fpu(struct task_struct *tsk) {}
-static inline void exit_fpu(void) {}
+static inline void exit_fpu(struct task_struct *tsk) {}
static inline void flush_fpu(void) {}
static inline int fpu_setup_sigcontext(struct fpucontext *buf) { return 0; }
static inline int fpu_restore_sigcontext(struct fpucontext *buf) { return 0; }
diff --git a/arch/mn10300/kernel/process.c b/arch/mn10300/kernel/process.c
index 3707da5..cbede4e 100644
--- a/arch/mn10300/kernel/process.c
+++ b/arch/mn10300/kernel/process.c
@@ -103,9 +103,9 @@
/*
* free current thread data structures etc..
*/
-void exit_thread(void)
+void exit_thread(struct task_struct *tsk)
{
- exit_fpu();
+ exit_fpu(tsk);
}
void flush_thread(void)
diff --git a/arch/nios2/Kconfig b/arch/nios2/Kconfig
index 87ca653..51a56c8 100644
--- a/arch/nios2/Kconfig
+++ b/arch/nios2/Kconfig
@@ -15,6 +15,7 @@
select SOC_BUS
select SPARSE_IRQ
select USB_ARCH_HAS_HCD if USB_SUPPORT
+ select CPU_NO_EFFICIENT_FFS
config GENERIC_CSUM
def_bool y
diff --git a/arch/nios2/Makefile b/arch/nios2/Makefile
index 2328f82..e74afc1 100644
--- a/arch/nios2/Makefile
+++ b/arch/nios2/Makefile
@@ -20,7 +20,7 @@
export MMU
-LIBGCC := $(shell $(CC) $(KBUILD_CFLAGS) -print-libgcc-file-name)
+LIBGCC := $(shell $(CC) $(KBUILD_CFLAGS) $(KCFLAGS) -print-libgcc-file-name)
KBUILD_CFLAGS += -pipe -D__linux__ -D__ELF__
KBUILD_CFLAGS += $(if $(CONFIG_NIOS2_HW_MUL_SUPPORT),-mhw-mul,-mno-hw-mul)
@@ -53,7 +53,7 @@
archclean:
$(Q)$(MAKE) $(clean)=$(nios2-boot)
-%.dtb:
+%.dtb: | scripts
$(Q)$(MAKE) $(build)=$(nios2-boot) $(nios2-boot)/$@
dtbs:
diff --git a/arch/nios2/include/asm/processor.h b/arch/nios2/include/asm/processor.h
index c2ba45c..1c953f0 100644
--- a/arch/nios2/include/asm/processor.h
+++ b/arch/nios2/include/asm/processor.h
@@ -75,11 +75,6 @@
{
}
-/* Free current thread data structures etc.. */
-static inline void exit_thread(void)
-{
-}
-
/* Return saved PC of a blocked thread. */
#define thread_saved_pc(tsk) ((tsk)->thread.kregs->ea)
diff --git a/arch/nios2/include/uapi/asm/unistd.h b/arch/nios2/include/uapi/asm/unistd.h
index c4bf795..51a32c7 100644
--- a/arch/nios2/include/uapi/asm/unistd.h
+++ b/arch/nios2/include/uapi/asm/unistd.h
@@ -17,6 +17,8 @@
#define sys_mmap2 sys_mmap_pgoff
+#define __ARCH_WANT_RENAMEAT
+
/* Use the standard ABI for syscalls */
#include <asm-generic/unistd.h>
diff --git a/arch/openrisc/Kconfig b/arch/openrisc/Kconfig
index e118c02..142cb05 100644
--- a/arch/openrisc/Kconfig
+++ b/arch/openrisc/Kconfig
@@ -25,6 +25,7 @@
select MODULES_USE_ELF_RELA
select HAVE_DEBUG_STACKOVERFLOW
select OR1K_PIC
+ select CPU_NO_EFFICIENT_FFS if !OPENRISC_HAVE_INST_FF1
config MMU
def_bool y
diff --git a/arch/openrisc/include/asm/processor.h b/arch/openrisc/include/asm/processor.h
index 4d235e3..70334c9 100644
--- a/arch/openrisc/include/asm/processor.h
+++ b/arch/openrisc/include/asm/processor.h
@@ -85,15 +85,6 @@
unsigned long get_wchan(struct task_struct *p);
/*
- * Free current thread data structures etc..
- */
-
-extern inline void exit_thread(void)
-{
- /* Nothing needs to be done. */
-}
-
-/*
* Return saved PC of a blocked thread. For now, this is the "user" PC
*/
extern unsigned long thread_saved_pc(struct task_struct *t);
diff --git a/arch/openrisc/include/uapi/asm/unistd.h b/arch/openrisc/include/uapi/asm/unistd.h
index ce40b71..471905b 100644
--- a/arch/openrisc/include/uapi/asm/unistd.h
+++ b/arch/openrisc/include/uapi/asm/unistd.h
@@ -20,6 +20,7 @@
#define sys_mmap2 sys_mmap_pgoff
+#define __ARCH_WANT_RENAMEAT
#define __ARCH_WANT_SYS_FORK
#define __ARCH_WANT_SYS_CLONE
diff --git a/arch/parisc/Kconfig b/arch/parisc/Kconfig
index 88cfaa8..3d498a6 100644
--- a/arch/parisc/Kconfig
+++ b/arch/parisc/Kconfig
@@ -32,6 +32,7 @@
select HAVE_ARCH_AUDITSYSCALL
select HAVE_ARCH_SECCOMP_FILTER
select ARCH_NO_COHERENT_DMA_MMAP
+ select CPU_NO_EFFICIENT_FFS
help
The PA-RISC microprocessor is designed by Hewlett-Packard and used
diff --git a/arch/parisc/kernel/process.c b/arch/parisc/kernel/process.c
index 809905a..4063943 100644
--- a/arch/parisc/kernel/process.c
+++ b/arch/parisc/kernel/process.c
@@ -144,13 +144,6 @@
void (*pm_power_off)(void) = machine_power_off;
EXPORT_SYMBOL(pm_power_off);
-/*
- * Free current thread data structures etc..
- */
-void exit_thread(void)
-{
-}
-
void flush_thread(void)
{
/* Only needs to handle fpu stuff or perf monitors.
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index a18a0dc..01f7464 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -116,6 +116,8 @@
select GENERIC_ATOMIC64 if PPC32
select ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE
select HAVE_PERF_EVENTS
+ select HAVE_PERF_REGS
+ select HAVE_PERF_USER_STACK_DUMP
select HAVE_REGS_AND_STACK_ACCESS_API
select HAVE_HW_BREAKPOINT if PERF_EVENTS && PPC_BOOK3S_64
select ARCH_WANT_IPC_PARSE_VERSION
@@ -153,6 +155,7 @@
select NO_BOOTMEM
select HAVE_GENERIC_RCU_GUP
select HAVE_PERF_EVENTS_NMI if PPC64
+ select HAVE_NMI if PERF_EVENTS
select EDAC_SUPPORT
select EDAC_ATOMIC_SCRUB
select ARCH_HAS_DMA_SET_COHERENT_MASK
@@ -606,9 +609,9 @@
config FORCE_MAX_ZONEORDER
int "Maximum zone order"
- range 9 64 if PPC64 && PPC_64K_PAGES
+ range 8 9 if PPC64 && PPC_64K_PAGES
default "9" if PPC64 && PPC_64K_PAGES
- range 13 64 if PPC64 && !PPC_64K_PAGES
+ range 9 13 if PPC64 && !PPC_64K_PAGES
default "13" if PPC64 && !PPC_64K_PAGES
range 9 64 if PPC32 && PPC_16K_PAGES
default "9" if PPC32 && PPC_16K_PAGES
@@ -795,7 +798,6 @@
config FSL_LBC
bool "Freescale Local Bus support"
- depends on FSL_SOC
help
Enables reporting of errors from the Freescale local bus
controller. Also contains some common code used by
diff --git a/arch/powerpc/Kconfig.debug b/arch/powerpc/Kconfig.debug
index 638f9ce..d3fcf7e 100644
--- a/arch/powerpc/Kconfig.debug
+++ b/arch/powerpc/Kconfig.debug
@@ -19,14 +19,6 @@
depends on !PPC_DISABLE_WERROR
default y
-config STRICT_MM_TYPECHECKS
- bool "Do extra type checking on mm types"
- default n
- help
- This option turns on extra type checking for some mm related types.
-
- If you don't know what this means, say N.
-
config PRINT_STACK_DEPTH
int "Stack depth to print" if DEBUG_KERNEL
default 64
diff --git a/arch/powerpc/boot/Makefile b/arch/powerpc/boot/Makefile
index 6116510..8fe78a3 100644
--- a/arch/powerpc/boot/Makefile
+++ b/arch/powerpc/boot/Makefile
@@ -362,9 +362,6 @@
$(obj)/cuImage.%: vmlinux $(obj)/%.dtb $(wrapperbits)
$(call if_changed,wrap,cuboot-$*,,$(obj)/$*.dtb)
-$(obj)/cuImage.%: vmlinux $(obj)/fsl/%.dtb $(wrapperbits)
- $(call if_changed,wrap,cuboot-$*,,$(obj)/fsl/$*.dtb)
-
$(obj)/simpleImage.initrd.%: vmlinux $(obj)/%.dtb $(wrapperbits)
$(call if_changed,wrap,simpleboot-$*,,$(obj)/$*.dtb,$(obj)/ramdisk.image.gz)
@@ -381,6 +378,9 @@
$(obj)/%.dtb: $(src)/dts/%.dts FORCE
$(call if_changed_dep,dtc)
+$(obj)/%.dtb: $(src)/dts/fsl/%.dts FORCE
+ $(call if_changed_dep,dtc)
+
# If there isn't a platform selected then just strip the vmlinux.
ifeq (,$(image-y))
image-y := vmlinux.strip
diff --git a/arch/powerpc/boot/dts/canyonlands.dts b/arch/powerpc/boot/dts/canyonlands.dts
index 3dc75de..549c24c 100644
--- a/arch/powerpc/boot/dts/canyonlands.dts
+++ b/arch/powerpc/boot/dts/canyonlands.dts
@@ -190,12 +190,21 @@
/* DMA */ 0x2 &UIC0 0xc 0x4>;
};
+ AHBDMA: dma@bffd0800 {
+ compatible = "snps,dma-spear1340";
+ reg = <4 0xbffd0800 0x400>;
+ interrupt-parent = <&UIC3>;
+ interrupts = <0x5 0x4>;
+ #dma-cells = <3>;
+ };
+
SATA0: sata@bffd1000 {
compatible = "amcc,sata-460ex";
- reg = <4 0xbffd1000 0x800 4 0xbffd0800 0x400>;
+ reg = <4 0xbffd1000 0x800>;
interrupt-parent = <&UIC3>;
- interrupts = <0x0 0x4 /* SATA */
- 0x5 0x4>; /* AHBDMA */
+ interrupts = <0x0 0x4>;
+ dmas = <&AHBDMA 0 1 0>;
+ dma-names = "sata-dma";
};
POB0: opb {
diff --git a/arch/powerpc/boot/dts/fsl/gef_ppc9a.dts b/arch/powerpc/boot/dts/fsl/gef_ppc9a.dts
index 0424fc2..c88d4ef 100644
--- a/arch/powerpc/boot/dts/fsl/gef_ppc9a.dts
+++ b/arch/powerpc/boot/dts/fsl/gef_ppc9a.dts
@@ -211,6 +211,10 @@
0x0 0x00400000>;
};
};
+
+ pci1: pcie@fef09000 {
+ status = "disabled";
+ };
};
/include/ "mpc8641si-post.dtsi"
diff --git a/arch/powerpc/boot/dts/fsl/gef_sbc310.dts b/arch/powerpc/boot/dts/fsl/gef_sbc310.dts
index 84b3d38..8385157 100644
--- a/arch/powerpc/boot/dts/fsl/gef_sbc310.dts
+++ b/arch/powerpc/boot/dts/fsl/gef_sbc310.dts
@@ -24,10 +24,6 @@
model = "GEF_SBC310";
compatible = "gef,sbc310";
- aliases {
- pci1 = &pci1;
- };
-
memory {
device_type = "memory";
reg = <0x0 0x40000000>; // set by uboot
@@ -223,29 +219,11 @@
};
pci1: pcie@fef09000 {
- compatible = "fsl,mpc8641-pcie";
- device_type = "pci";
- #size-cells = <2>;
- #address-cells = <3>;
reg = <0xfef09000 0x1000>;
- bus-range = <0x0 0xff>;
ranges = <0x02000000 0x0 0xc0000000 0xc0000000 0x0 0x20000000
0x01000000 0x0 0x00000000 0xfe400000 0x0 0x00400000>;
- clock-frequency = <100000000>;
- interrupts = <0x19 0x2 0 0>;
- interrupt-map-mask = <0xf800 0x0 0x0 0x7>;
- interrupt-map = <
- 0x0000 0x0 0x0 0x1 &mpic 0x4 0x2
- 0x0000 0x0 0x0 0x2 &mpic 0x5 0x2
- 0x0000 0x0 0x0 0x3 &mpic 0x6 0x2
- 0x0000 0x0 0x0 0x4 &mpic 0x7 0x2
- >;
pcie@0 {
- reg = <0 0 0 0 0>;
- #size-cells = <2>;
- #address-cells = <3>;
- device_type = "pci";
ranges = <0x02000000 0x0 0xc0000000
0x02000000 0x0 0xc0000000
0x0 0x20000000
diff --git a/arch/powerpc/boot/dts/fsl/gef_sbc610.dts b/arch/powerpc/boot/dts/fsl/gef_sbc610.dts
index 974446a..ff423ab 100644
--- a/arch/powerpc/boot/dts/fsl/gef_sbc610.dts
+++ b/arch/powerpc/boot/dts/fsl/gef_sbc610.dts
@@ -209,6 +209,10 @@
0x0 0x00400000>;
};
};
+
+ pci1: pcie@fef09000 {
+ status = "disabled";
+ };
};
/include/ "mpc8641si-post.dtsi"
diff --git a/arch/powerpc/boot/dts/fsl/mpc8641_hpcn.dts b/arch/powerpc/boot/dts/fsl/mpc8641_hpcn.dts
index 554001f..11bea3e 100644
--- a/arch/powerpc/boot/dts/fsl/mpc8641_hpcn.dts
+++ b/arch/powerpc/boot/dts/fsl/mpc8641_hpcn.dts
@@ -15,10 +15,6 @@
model = "MPC8641HPCN";
compatible = "fsl,mpc8641hpcn";
- aliases {
- pci1 = &pci1;
- };
-
memory {
device_type = "memory";
reg = <0x00000000 0x40000000>; // 1G at 0x0
@@ -359,29 +355,11 @@
};
pci1: pcie@ffe09000 {
- compatible = "fsl,mpc8641-pcie";
- device_type = "pci";
- #size-cells = <2>;
- #address-cells = <3>;
reg = <0xffe09000 0x1000>;
- bus-range = <0 0xff>;
ranges = <0x02000000 0x0 0xa0000000 0xa0000000 0x0 0x20000000
0x01000000 0x0 0x00000000 0xffc10000 0x0 0x00010000>;
- clock-frequency = <100000000>;
- interrupts = <25 2 0 0>;
- interrupt-map-mask = <0xf800 0 0 7>;
- interrupt-map = <
- /* IDSEL 0x0 */
- 0x0000 0 0 1 &mpic 4 1
- 0x0000 0 0 2 &mpic 5 1
- 0x0000 0 0 3 &mpic 6 1
- 0x0000 0 0 4 &mpic 7 1
- >;
+
pcie@0 {
- reg = <0 0 0 0 0>;
- #size-cells = <2>;
- #address-cells = <3>;
- device_type = "pci";
ranges = <0x02000000 0x0 0xa0000000
0x02000000 0x0 0xa0000000
0x0 0x20000000
diff --git a/arch/powerpc/boot/dts/fsl/mpc8641_hpcn_36b.dts b/arch/powerpc/boot/dts/fsl/mpc8641_hpcn_36b.dts
index fec5867..7ff6204 100644
--- a/arch/powerpc/boot/dts/fsl/mpc8641_hpcn_36b.dts
+++ b/arch/powerpc/boot/dts/fsl/mpc8641_hpcn_36b.dts
@@ -17,10 +17,6 @@
#address-cells = <2>;
#size-cells = <2>;
- aliases {
- pci1 = &pci1;
- };
-
memory {
device_type = "memory";
reg = <0x0 0x00000000 0x0 0x40000000>; // 1G at 0x0
@@ -326,29 +322,11 @@
};
pci1: pcie@fffe09000 {
- compatible = "fsl,mpc8641-pcie";
- device_type = "pci";
- #size-cells = <2>;
- #address-cells = <3>;
reg = <0x0f 0xffe09000 0x0 0x1000>;
- bus-range = <0x0 0xff>;
ranges = <0x02000000 0x0 0xe0000000 0x0c 0x20000000 0x0 0x20000000
0x01000000 0x0 0x00000000 0x0f 0xffc10000 0x0 0x00010000>;
- clock-frequency = <100000000>;
- interrupts = <25 2 0 0>;
- interrupt-map-mask = <0xf800 0 0 7>;
- interrupt-map = <
- /* IDSEL 0x0 */
- 0x0000 0 0 1 &mpic 4 1
- 0x0000 0 0 2 &mpic 5 1
- 0x0000 0 0 3 &mpic 6 1
- 0x0000 0 0 4 &mpic 7 1
- >;
+
pcie@0 {
- reg = <0 0 0 0 0>;
- #size-cells = <2>;
- #address-cells = <3>;
- device_type = "pci";
ranges = <0x02000000 0x0 0xe0000000
0x02000000 0x0 0xe0000000
0x0 0x20000000
diff --git a/arch/powerpc/boot/dts/fsl/mpc8641si-post.dtsi b/arch/powerpc/boot/dts/fsl/mpc8641si-post.dtsi
index 70889d8..eeb7c65 100644
--- a/arch/powerpc/boot/dts/fsl/mpc8641si-post.dtsi
+++ b/arch/powerpc/boot/dts/fsl/mpc8641si-post.dtsi
@@ -102,19 +102,46 @@
bus-range = <0x0 0xff>;
clock-frequency = <100000000>;
interrupts = <24 2 0 0>;
- interrupt-map-mask = <0xf800 0x0 0x0 0x7>;
-
- interrupt-map = <
- 0x0000 0x0 0x0 0x1 &mpic 0x0 0x1
- 0x0000 0x0 0x0 0x2 &mpic 0x1 0x1
- 0x0000 0x0 0x0 0x3 &mpic 0x2 0x1
- 0x0000 0x0 0x0 0x4 &mpic 0x3 0x1
- >;
pcie@0 {
reg = <0 0 0 0 0>;
+ #interrupt-cells = <1>;
#size-cells = <2>;
#address-cells = <3>;
device_type = "pci";
+ interrupts = <24 2 0 0>;
+ interrupt-map-mask = <0xf800 0x0 0x0 0x7>;
+ interrupt-map = <
+ 0x0000 0x0 0x0 0x1 &mpic 0x0 0x1 0x0 0x0
+ 0x0000 0x0 0x0 0x2 &mpic 0x1 0x1 0x0 0x0
+ 0x0000 0x0 0x0 0x3 &mpic 0x2 0x1 0x0 0x0
+ 0x0000 0x0 0x0 0x4 &mpic 0x3 0x1 0x0 0x0
+ >;
+ };
+};
+
+&pci1 {
+ compatible = "fsl,mpc8641-pcie";
+ device_type = "pci";
+ #size-cells = <2>;
+ #address-cells = <3>;
+ bus-range = <0x0 0xff>;
+ clock-frequency = <100000000>;
+ interrupts = <25 2 0 0>;
+
+ pcie@0 {
+ reg = <0 0 0 0 0>;
+ #interrupt-cells = <1>;
+ #size-cells = <2>;
+ #address-cells = <3>;
+ device_type = "pci";
+ interrupts = <25 2 0 0>;
+ interrupt-map-mask = <0xf800 0x0 0x0 0x7>;
+ interrupt-map = <
+ 0x0000 0x0 0x0 0x1 &mpic 0x4 0x1 0x0 0x0
+ 0x0000 0x0 0x0 0x2 &mpic 0x5 0x1 0x0 0x0
+ 0x0000 0x0 0x0 0x3 &mpic 0x6 0x1 0x0 0x0
+ 0x0000 0x0 0x0 0x4 &mpic 0x7 0x1 0x0 0x0
+ >;
};
};
diff --git a/arch/powerpc/boot/dts/fsl/mpc8641si-pre.dtsi b/arch/powerpc/boot/dts/fsl/mpc8641si-pre.dtsi
index 9e03328..7c6db6f 100644
--- a/arch/powerpc/boot/dts/fsl/mpc8641si-pre.dtsi
+++ b/arch/powerpc/boot/dts/fsl/mpc8641si-pre.dtsi
@@ -25,6 +25,7 @@
serial0 = &serial0;
serial1 = &serial1;
pci0 = &pci0;
+ pci1 = &pci1;
};
cpus {
diff --git a/arch/powerpc/boot/dts/fsl/sbc8641d.dts b/arch/powerpc/boot/dts/fsl/sbc8641d.dts
index 0a9733c..75870a1 100644
--- a/arch/powerpc/boot/dts/fsl/sbc8641d.dts
+++ b/arch/powerpc/boot/dts/fsl/sbc8641d.dts
@@ -19,10 +19,6 @@
model = "SBC8641D";
compatible = "wind,sbc8641";
- aliases {
- pci1 = &pci1;
- };
-
memory {
device_type = "memory";
reg = <0x00000000 0x20000000>; // 512M at 0x0
@@ -165,30 +161,11 @@
};
pci1: pcie@f8009000 {
- compatible = "fsl,mpc8641-pcie";
- device_type = "pci";
- #size-cells = <2>;
- #address-cells = <3>;
reg = <0xf8009000 0x1000>;
- bus-range = <0 0xff>;
ranges = <0x02000000 0x0 0xa0000000 0xa0000000 0x0 0x20000000
0x01000000 0x0 0x00000000 0xe3000000 0x0 0x00100000>;
- clock-frequency = <100000000>;
- interrupts = <25 2 0 0>;
- interrupt-map-mask = <0xf800 0 0 7>;
- interrupt-map = <
- /* IDSEL 0x0 */
- 0x0000 0 0 1 &mpic 4 1
- 0x0000 0 0 2 &mpic 5 1
- 0x0000 0 0 3 &mpic 6 1
- 0x0000 0 0 4 &mpic 7 1
- >;
pcie@0 {
- reg = <0 0 0 0 0>;
- #size-cells = <2>;
- #address-cells = <3>;
- device_type = "pci";
ranges = <0x02000000 0x0 0xa0000000
0x02000000 0x0 0xa0000000
0x0 0x20000000
diff --git a/arch/powerpc/boot/dts/fsl/t1023si-post.dtsi b/arch/powerpc/boot/dts/fsl/t1023si-post.dtsi
index 99e421d..6e0b489 100644
--- a/arch/powerpc/boot/dts/fsl/t1023si-post.dtsi
+++ b/arch/powerpc/boot/dts/fsl/t1023si-post.dtsi
@@ -263,7 +263,7 @@
};
rcpm: global-utilities@e2000 {
- compatible = "fsl,t1023-rcpm", "fsl,qoriq-rcpm-2.0";
+ compatible = "fsl,t1023-rcpm", "fsl,qoriq-rcpm-2.1";
reg = <0xe2000 0x1000>;
};
diff --git a/arch/powerpc/boot/dts/fsl/t1040si-post.dtsi b/arch/powerpc/boot/dts/fsl/t1040si-post.dtsi
index e0f4da5..507649ec 100644
--- a/arch/powerpc/boot/dts/fsl/t1040si-post.dtsi
+++ b/arch/powerpc/boot/dts/fsl/t1040si-post.dtsi
@@ -472,7 +472,7 @@
};
rcpm: global-utilities@e2000 {
- compatible = "fsl,t1040-rcpm", "fsl,qoriq-rcpm-2.0";
+ compatible = "fsl,t1040-rcpm", "fsl,qoriq-rcpm-2.1";
reg = <0xe2000 0x1000>;
};
diff --git a/arch/powerpc/boot/dts/fsl/t104xrdb.dtsi b/arch/powerpc/boot/dts/fsl/t104xrdb.dtsi
index 72691ef..7c4afdb 100644
--- a/arch/powerpc/boot/dts/fsl/t104xrdb.dtsi
+++ b/arch/powerpc/boot/dts/fsl/t104xrdb.dtsi
@@ -109,7 +109,7 @@
flash@0 {
#address-cells = <1>;
#size-cells = <1>;
- compatible = "micron,n25q512a", "jedec,spi-nor";
+ compatible = "micron,n25q512ax3", "jedec,spi-nor";
reg = <0>;
spi-max-frequency = <10000000>; /* input clock */
};
diff --git a/arch/powerpc/boot/dts/fsl/t208xrdb.dtsi b/arch/powerpc/boot/dts/fsl/t208xrdb.dtsi
index dc93268..ff87e67 100644
--- a/arch/powerpc/boot/dts/fsl/t208xrdb.dtsi
+++ b/arch/powerpc/boot/dts/fsl/t208xrdb.dtsi
@@ -113,7 +113,7 @@
flash@0 {
#address-cells = <1>;
#size-cells = <1>;
- compatible = "micron,n25q512a", "jedec,spi-nor";
+ compatible = "micron,n25q512ax3", "jedec,spi-nor";
reg = <0>;
spi-max-frequency = <10000000>; /* input clock */
};
diff --git a/arch/powerpc/include/asm/book3s/32/hash.h b/arch/powerpc/include/asm/book3s/32/hash.h
index 264b754..880db13 100644
--- a/arch/powerpc/include/asm/book3s/32/hash.h
+++ b/arch/powerpc/include/asm/book3s/32/hash.h
@@ -39,8 +39,5 @@
#define _PMD_PRESENT_MASK (PAGE_MASK)
#define _PMD_BAD (~PAGE_MASK)
-/* Hash table based platforms need atomic updates of the linux PTE */
-#define PTE_ATOMIC_UPDATES 1
-
#endif /* __KERNEL__ */
#endif /* _ASM_POWERPC_BOOK3S_32_HASH_H */
diff --git a/arch/powerpc/include/asm/book3s/32/mmu-hash.h b/arch/powerpc/include/asm/book3s/32/mmu-hash.h
index 16f513e..b82e063 100644
--- a/arch/powerpc/include/asm/book3s/32/mmu-hash.h
+++ b/arch/powerpc/include/asm/book3s/32/mmu-hash.h
@@ -1,5 +1,5 @@
-#ifndef _ASM_POWERPC_MMU_HASH32_H_
-#define _ASM_POWERPC_MMU_HASH32_H_
+#ifndef _ASM_POWERPC_BOOK3S_32_MMU_HASH_H_
+#define _ASM_POWERPC_BOOK3S_32_MMU_HASH_H_
/*
* 32-bit hash table MMU support
*/
@@ -90,4 +90,4 @@
#define mmu_virtual_psize MMU_PAGE_4K
#define mmu_linear_psize MMU_PAGE_256M
-#endif /* _ASM_POWERPC_MMU_HASH32_H_ */
+#endif /* _ASM_POWERPC_BOOK3S_32_MMU_HASH_H_ */
diff --git a/arch/powerpc/include/asm/book3s/32/pgalloc.h b/arch/powerpc/include/asm/book3s/32/pgalloc.h
new file mode 100644
index 0000000..a235019
--- /dev/null
+++ b/arch/powerpc/include/asm/book3s/32/pgalloc.h
@@ -0,0 +1,109 @@
+#ifndef _ASM_POWERPC_BOOK3S_32_PGALLOC_H
+#define _ASM_POWERPC_BOOK3S_32_PGALLOC_H
+
+#include <linux/threads.h>
+
+/* For 32-bit, all levels of page tables are just drawn from get_free_page() */
+#define MAX_PGTABLE_INDEX_SIZE 0
+
+extern void __bad_pte(pmd_t *pmd);
+
+extern pgd_t *pgd_alloc(struct mm_struct *mm);
+extern void pgd_free(struct mm_struct *mm, pgd_t *pgd);
+
+/*
+ * We don't have any real pmd's, and this code never triggers because
+ * the pgd will always be present..
+ */
+/* #define pmd_alloc_one(mm,address) ({ BUG(); ((pmd_t *)2); }) */
+#define pmd_free(mm, x) do { } while (0)
+#define __pmd_free_tlb(tlb,x,a) do { } while (0)
+/* #define pgd_populate(mm, pmd, pte) BUG() */
+
+#ifndef CONFIG_BOOKE
+
+static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmdp,
+ pte_t *pte)
+{
+ *pmdp = __pmd(__pa(pte) | _PMD_PRESENT);
+}
+
+static inline void pmd_populate(struct mm_struct *mm, pmd_t *pmdp,
+ pgtable_t pte_page)
+{
+ *pmdp = __pmd((page_to_pfn(pte_page) << PAGE_SHIFT) | _PMD_PRESENT);
+}
+
+#define pmd_pgtable(pmd) pmd_page(pmd)
+#else
+
+static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmdp,
+ pte_t *pte)
+{
+ *pmdp = __pmd((unsigned long)pte | _PMD_PRESENT);
+}
+
+static inline void pmd_populate(struct mm_struct *mm, pmd_t *pmdp,
+ pgtable_t pte_page)
+{
+ *pmdp = __pmd((unsigned long)lowmem_page_address(pte_page) | _PMD_PRESENT);
+}
+
+#define pmd_pgtable(pmd) pmd_page(pmd)
+#endif
+
+extern pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long addr);
+extern pgtable_t pte_alloc_one(struct mm_struct *mm, unsigned long addr);
+
+static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte)
+{
+ free_page((unsigned long)pte);
+}
+
+static inline void pte_free(struct mm_struct *mm, pgtable_t ptepage)
+{
+ pgtable_page_dtor(ptepage);
+ __free_page(ptepage);
+}
+
+static inline void pgtable_free(void *table, unsigned index_size)
+{
+ BUG_ON(index_size); /* 32-bit doesn't use this */
+ free_page((unsigned long)table);
+}
+
+#define check_pgt_cache() do { } while (0)
+
+#ifdef CONFIG_SMP
+static inline void pgtable_free_tlb(struct mmu_gather *tlb,
+ void *table, int shift)
+{
+ unsigned long pgf = (unsigned long)table;
+ BUG_ON(shift > MAX_PGTABLE_INDEX_SIZE);
+ pgf |= shift;
+ tlb_remove_table(tlb, (void *)pgf);
+}
+
+static inline void __tlb_remove_table(void *_table)
+{
+ void *table = (void *)((unsigned long)_table & ~MAX_PGTABLE_INDEX_SIZE);
+ unsigned shift = (unsigned long)_table & MAX_PGTABLE_INDEX_SIZE;
+
+ pgtable_free(table, shift);
+}
+#else
+static inline void pgtable_free_tlb(struct mmu_gather *tlb,
+ void *table, int shift)
+{
+ pgtable_free(table, shift);
+}
+#endif
+
+static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t table,
+ unsigned long address)
+{
+ tlb_flush_pgtable(tlb, address);
+ pgtable_page_dtor(table);
+ pgtable_free_tlb(tlb, page_address(table), 0);
+}
+#endif /* _ASM_POWERPC_BOOK3S_32_PGALLOC_H */
diff --git a/arch/powerpc/include/asm/book3s/64/hash-4k.h b/arch/powerpc/include/asm/book3s/64/hash-4k.h
index 5f08a08..1af837c 100644
--- a/arch/powerpc/include/asm/book3s/64/hash-4k.h
+++ b/arch/powerpc/include/asm/book3s/64/hash-4k.h
@@ -5,58 +5,31 @@
* for each page table entry. The PMD and PGD level use a 32b record for
* each entry by assuming that each entry is page aligned.
*/
-#define PTE_INDEX_SIZE 9
-#define PMD_INDEX_SIZE 7
-#define PUD_INDEX_SIZE 9
-#define PGD_INDEX_SIZE 9
+#define H_PTE_INDEX_SIZE 9
+#define H_PMD_INDEX_SIZE 7
+#define H_PUD_INDEX_SIZE 9
+#define H_PGD_INDEX_SIZE 9
#ifndef __ASSEMBLY__
-#define PTE_TABLE_SIZE (sizeof(pte_t) << PTE_INDEX_SIZE)
-#define PMD_TABLE_SIZE (sizeof(pmd_t) << PMD_INDEX_SIZE)
-#define PUD_TABLE_SIZE (sizeof(pud_t) << PUD_INDEX_SIZE)
-#define PGD_TABLE_SIZE (sizeof(pgd_t) << PGD_INDEX_SIZE)
-#endif /* __ASSEMBLY__ */
-
-#define PTRS_PER_PTE (1 << PTE_INDEX_SIZE)
-#define PTRS_PER_PMD (1 << PMD_INDEX_SIZE)
-#define PTRS_PER_PUD (1 << PUD_INDEX_SIZE)
-#define PTRS_PER_PGD (1 << PGD_INDEX_SIZE)
-
-/* PMD_SHIFT determines what a second-level page table entry can map */
-#define PMD_SHIFT (PAGE_SHIFT + PTE_INDEX_SIZE)
-#define PMD_SIZE (1UL << PMD_SHIFT)
-#define PMD_MASK (~(PMD_SIZE-1))
+#define H_PTE_TABLE_SIZE (sizeof(pte_t) << H_PTE_INDEX_SIZE)
+#define H_PMD_TABLE_SIZE (sizeof(pmd_t) << H_PMD_INDEX_SIZE)
+#define H_PUD_TABLE_SIZE (sizeof(pud_t) << H_PUD_INDEX_SIZE)
+#define H_PGD_TABLE_SIZE (sizeof(pgd_t) << H_PGD_INDEX_SIZE)
/* With 4k base page size, hugepage PTEs go at the PMD level */
#define MIN_HUGEPTE_SHIFT PMD_SHIFT
-/* PUD_SHIFT determines what a third-level page table entry can map */
-#define PUD_SHIFT (PMD_SHIFT + PMD_INDEX_SIZE)
-#define PUD_SIZE (1UL << PUD_SHIFT)
-#define PUD_MASK (~(PUD_SIZE-1))
-
-/* PGDIR_SHIFT determines what a fourth-level page table entry can map */
-#define PGDIR_SHIFT (PUD_SHIFT + PUD_INDEX_SIZE)
-#define PGDIR_SIZE (1UL << PGDIR_SHIFT)
-#define PGDIR_MASK (~(PGDIR_SIZE-1))
-
-/* Bits to mask out from a PMD to get to the PTE page */
-#define PMD_MASKED_BITS 0
-/* Bits to mask out from a PUD to get to the PMD page */
-#define PUD_MASKED_BITS 0
-/* Bits to mask out from a PGD to get to the PUD page */
-#define PGD_MASKED_BITS 0
-
/* PTE flags to conserve for HPTE identification */
-#define _PAGE_HPTEFLAGS (_PAGE_BUSY | _PAGE_HASHPTE | \
- _PAGE_F_SECOND | _PAGE_F_GIX)
-
-/* shift to put page number into pte */
-#define PTE_RPN_SHIFT (12)
-#define PTE_RPN_SIZE (45) /* gives 57-bit real addresses */
-
-#define _PAGE_4K_PFN 0
-#ifndef __ASSEMBLY__
+#define _PAGE_HPTEFLAGS (H_PAGE_BUSY | H_PAGE_HASHPTE | \
+ H_PAGE_F_SECOND | H_PAGE_F_GIX)
+/*
+ * Not supported by 4k linux page size
+ */
+#define H_PAGE_4K_PFN 0x0
+#define H_PAGE_THP_HUGE 0x0
+#define H_PAGE_COMBO 0x0
+#define H_PTE_FRAG_NR 0
+#define H_PTE_FRAG_SIZE_SHIFT 0
/*
* On all 4K setups, remap_4k_pfn() equates to remap_pfn_range()
*/
@@ -64,26 +37,7 @@
remap_pfn_range((vma), (addr), (pfn), PAGE_SIZE, (prot))
#ifdef CONFIG_HUGETLB_PAGE
-/*
- * For 4k page size, we support explicit hugepage via hugepd
- */
-static inline int pmd_huge(pmd_t pmd)
-{
- return 0;
-}
-
-static inline int pud_huge(pud_t pud)
-{
- return 0;
-}
-
-static inline int pgd_huge(pgd_t pgd)
-{
- return 0;
-}
-#define pgd_huge pgd_huge
-
-static inline int hugepd_ok(hugepd_t hpd)
+static inline int hash__hugepd_ok(hugepd_t hpd)
{
/*
* if it is not a pte and have hugepd shift mask
@@ -94,7 +48,65 @@
return true;
return false;
}
-#define is_hugepd(hpd) (hugepd_ok(hpd))
+#endif
+
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+
+static inline char *get_hpte_slot_array(pmd_t *pmdp)
+{
+ BUG();
+ return NULL;
+}
+
+static inline unsigned int hpte_valid(unsigned char *hpte_slot_array, int index)
+{
+ BUG();
+ return 0;
+}
+
+static inline unsigned int hpte_hash_index(unsigned char *hpte_slot_array,
+ int index)
+{
+ BUG();
+ return 0;
+}
+
+static inline void mark_hpte_slot_valid(unsigned char *hpte_slot_array,
+ unsigned int index, unsigned int hidx)
+{
+ BUG();
+}
+
+static inline int hash__pmd_trans_huge(pmd_t pmd)
+{
+ return 0;
+}
+
+static inline int hash__pmd_same(pmd_t pmd_a, pmd_t pmd_b)
+{
+ BUG();
+ return 0;
+}
+
+static inline pmd_t hash__pmd_mkhuge(pmd_t pmd)
+{
+ BUG();
+ return pmd;
+}
+
+extern unsigned long hash__pmd_hugepage_update(struct mm_struct *mm,
+ unsigned long addr, pmd_t *pmdp,
+ unsigned long clr, unsigned long set);
+extern pmd_t hash__pmdp_collapse_flush(struct vm_area_struct *vma,
+ unsigned long address, pmd_t *pmdp);
+extern void hash__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,
+ pgtable_t pgtable);
+extern pgtable_t hash__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp);
+extern void hash__pmdp_huge_split_prepare(struct vm_area_struct *vma,
+ unsigned long address, pmd_t *pmdp);
+extern pmd_t hash__pmdp_huge_get_and_clear(struct mm_struct *mm,
+ unsigned long addr, pmd_t *pmdp);
+extern int hash__has_transparent_hugepage(void);
#endif
#endif /* !__ASSEMBLY__ */
diff --git a/arch/powerpc/include/asm/book3s/64/hash-64k.h b/arch/powerpc/include/asm/book3s/64/hash-64k.h
index 0a7956a..5aae4f5 100644
--- a/arch/powerpc/include/asm/book3s/64/hash-64k.h
+++ b/arch/powerpc/include/asm/book3s/64/hash-64k.h
@@ -1,73 +1,44 @@
#ifndef _ASM_POWERPC_BOOK3S_64_HASH_64K_H
#define _ASM_POWERPC_BOOK3S_64_HASH_64K_H
-#define PTE_INDEX_SIZE 8
-#define PMD_INDEX_SIZE 5
-#define PUD_INDEX_SIZE 5
-#define PGD_INDEX_SIZE 12
-
-#define PTRS_PER_PTE (1 << PTE_INDEX_SIZE)
-#define PTRS_PER_PMD (1 << PMD_INDEX_SIZE)
-#define PTRS_PER_PUD (1 << PUD_INDEX_SIZE)
-#define PTRS_PER_PGD (1 << PGD_INDEX_SIZE)
+#define H_PTE_INDEX_SIZE 8
+#define H_PMD_INDEX_SIZE 5
+#define H_PUD_INDEX_SIZE 5
+#define H_PGD_INDEX_SIZE 12
/* With 4k base page size, hugepage PTEs go at the PMD level */
#define MIN_HUGEPTE_SHIFT PAGE_SHIFT
-/* PMD_SHIFT determines what a second-level page table entry can map */
-#define PMD_SHIFT (PAGE_SHIFT + PTE_INDEX_SIZE)
-#define PMD_SIZE (1UL << PMD_SHIFT)
-#define PMD_MASK (~(PMD_SIZE-1))
-
-/* PUD_SHIFT determines what a third-level page table entry can map */
-#define PUD_SHIFT (PMD_SHIFT + PMD_INDEX_SIZE)
-#define PUD_SIZE (1UL << PUD_SHIFT)
-#define PUD_MASK (~(PUD_SIZE-1))
-
-/* PGDIR_SHIFT determines what a fourth-level page table entry can map */
-#define PGDIR_SHIFT (PUD_SHIFT + PUD_INDEX_SIZE)
-#define PGDIR_SIZE (1UL << PGDIR_SHIFT)
-#define PGDIR_MASK (~(PGDIR_SIZE-1))
-
-#define _PAGE_COMBO 0x00001000 /* this is a combo 4k page */
-#define _PAGE_4K_PFN 0x00002000 /* PFN is for a single 4k page */
+#define H_PAGE_COMBO 0x00001000 /* this is a combo 4k page */
+#define H_PAGE_4K_PFN 0x00002000 /* PFN is for a single 4k page */
/*
- * Used to track subpage group valid if _PAGE_COMBO is set
- * This overloads _PAGE_F_GIX and _PAGE_F_SECOND
+ * We need to differentiate between explicit huge page and THP huge
+ * page, since THP huge page also need to track real subpage details
*/
-#define _PAGE_COMBO_VALID (_PAGE_F_GIX | _PAGE_F_SECOND)
+#define H_PAGE_THP_HUGE H_PAGE_4K_PFN
+
+/*
+ * Used to track subpage group valid if H_PAGE_COMBO is set
+ * This overloads H_PAGE_F_GIX and H_PAGE_F_SECOND
+ */
+#define H_PAGE_COMBO_VALID (H_PAGE_F_GIX | H_PAGE_F_SECOND)
/* PTE flags to conserve for HPTE identification */
-#define _PAGE_HPTEFLAGS (_PAGE_BUSY | _PAGE_F_SECOND | \
- _PAGE_F_GIX | _PAGE_HASHPTE | _PAGE_COMBO)
-
-/* Shift to put page number into pte.
- *
- * That gives us a max RPN of 41 bits, which means a max of 57 bits
- * of addressable physical space, or 53 bits for the special 4k PFNs.
- */
-#define PTE_RPN_SHIFT (16)
-#define PTE_RPN_SIZE (41)
-
+#define _PAGE_HPTEFLAGS (H_PAGE_BUSY | H_PAGE_F_SECOND | \
+ H_PAGE_F_GIX | H_PAGE_HASHPTE | H_PAGE_COMBO)
/*
* we support 16 fragments per PTE page of 64K size.
*/
-#define PTE_FRAG_NR 16
+#define H_PTE_FRAG_NR 16
/*
* We use a 2K PTE page fragment and another 2K for storing
* real_pte_t hash index
*/
-#define PTE_FRAG_SIZE_SHIFT 12
+#define H_PTE_FRAG_SIZE_SHIFT 12
#define PTE_FRAG_SIZE (1UL << PTE_FRAG_SIZE_SHIFT)
-/* Bits to mask out from a PMD to get to the PTE page */
-#define PMD_MASKED_BITS 0xc0000000000000ffUL
-/* Bits to mask out from a PUD to get to the PMD page */
-#define PUD_MASKED_BITS 0xc0000000000000ffUL
-/* Bits to mask out from a PGD to get to the PUD page */
-#define PGD_MASKED_BITS 0xc0000000000000ffUL
-
#ifndef __ASSEMBLY__
+#include <asm/errno.h>
/*
* With 64K pages on hash table, we have a special PTE format that
@@ -83,9 +54,9 @@
rpte.pte = pte;
rpte.hidx = 0;
- if (pte_val(pte) & _PAGE_COMBO) {
+ if (pte_val(pte) & H_PAGE_COMBO) {
/*
- * Make sure we order the hidx load against the _PAGE_COMBO
+ * Make sure we order the hidx load against the H_PAGE_COMBO
* check. The store side ordering is done in __hash_page_4K
*/
smp_rmb();
@@ -97,9 +68,9 @@
static inline unsigned long __rpte_to_hidx(real_pte_t rpte, unsigned long index)
{
- if ((pte_val(rpte.pte) & _PAGE_COMBO))
+ if ((pte_val(rpte.pte) & H_PAGE_COMBO))
return (rpte.hidx >> (index<<2)) & 0xf;
- return (pte_val(rpte.pte) >> _PAGE_F_GIX_SHIFT) & 0xf;
+ return (pte_val(rpte.pte) >> H_PAGE_F_GIX_SHIFT) & 0xf;
}
#define __rpte_to_pte(r) ((r).pte)
@@ -122,79 +93,32 @@
#define pte_iterate_hashed_end() } while(0); } } while(0)
#define pte_pagesize_index(mm, addr, pte) \
- (((pte) & _PAGE_COMBO)? MMU_PAGE_4K: MMU_PAGE_64K)
+ (((pte) & H_PAGE_COMBO)? MMU_PAGE_4K: MMU_PAGE_64K)
-#define remap_4k_pfn(vma, addr, pfn, prot) \
- (WARN_ON(((pfn) >= (1UL << PTE_RPN_SIZE))) ? -EINVAL : \
- remap_pfn_range((vma), (addr), (pfn), PAGE_SIZE, \
- __pgprot(pgprot_val((prot)) | _PAGE_4K_PFN)))
+extern int remap_pfn_range(struct vm_area_struct *, unsigned long addr,
+ unsigned long pfn, unsigned long size, pgprot_t);
+static inline int hash__remap_4k_pfn(struct vm_area_struct *vma, unsigned long addr,
+ unsigned long pfn, pgprot_t prot)
+{
+ if (pfn > (PTE_RPN_MASK >> PAGE_SHIFT)) {
+ WARN(1, "remap_4k_pfn called with wrong pfn value\n");
+ return -EINVAL;
+ }
+ return remap_pfn_range(vma, addr, pfn, PAGE_SIZE,
+ __pgprot(pgprot_val(prot) | H_PAGE_4K_PFN));
+}
-#define PTE_TABLE_SIZE PTE_FRAG_SIZE
+#define H_PTE_TABLE_SIZE PTE_FRAG_SIZE
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-#define PMD_TABLE_SIZE ((sizeof(pmd_t) << PMD_INDEX_SIZE) + (sizeof(unsigned long) << PMD_INDEX_SIZE))
+#define H_PMD_TABLE_SIZE ((sizeof(pmd_t) << PMD_INDEX_SIZE) + \
+ (sizeof(unsigned long) << PMD_INDEX_SIZE))
#else
-#define PMD_TABLE_SIZE (sizeof(pmd_t) << PMD_INDEX_SIZE)
+#define H_PMD_TABLE_SIZE (sizeof(pmd_t) << PMD_INDEX_SIZE)
#endif
-#define PUD_TABLE_SIZE (sizeof(pud_t) << PUD_INDEX_SIZE)
-#define PGD_TABLE_SIZE (sizeof(pgd_t) << PGD_INDEX_SIZE)
-
-#ifdef CONFIG_HUGETLB_PAGE
-/*
- * We have PGD_INDEX_SIZ = 12 and PTE_INDEX_SIZE = 8, so that we can have
- * 16GB hugepage pte in PGD and 16MB hugepage pte at PMD;
- *
- * Defined in such a way that we can optimize away code block at build time
- * if CONFIG_HUGETLB_PAGE=n.
- */
-static inline int pmd_huge(pmd_t pmd)
-{
- /*
- * leaf pte for huge page
- */
- return !!(pmd_val(pmd) & _PAGE_PTE);
-}
-
-static inline int pud_huge(pud_t pud)
-{
- /*
- * leaf pte for huge page
- */
- return !!(pud_val(pud) & _PAGE_PTE);
-}
-
-static inline int pgd_huge(pgd_t pgd)
-{
- /*
- * leaf pte for huge page
- */
- return !!(pgd_val(pgd) & _PAGE_PTE);
-}
-#define pgd_huge pgd_huge
-
-#ifdef CONFIG_DEBUG_VM
-extern int hugepd_ok(hugepd_t hpd);
-#define is_hugepd(hpd) (hugepd_ok(hpd))
-#else
-/*
- * With 64k page size, we have hugepage ptes in the pgd and pmd entries. We don't
- * need to setup hugepage directory for them. Our pte and page directory format
- * enable us to have this enabled.
- */
-static inline int hugepd_ok(hugepd_t hpd)
-{
- return 0;
-}
-#define is_hugepd(pdep) 0
-#endif /* CONFIG_DEBUG_VM */
-
-#endif /* CONFIG_HUGETLB_PAGE */
+#define H_PUD_TABLE_SIZE (sizeof(pud_t) << PUD_INDEX_SIZE)
+#define H_PGD_TABLE_SIZE (sizeof(pgd_t) << PGD_INDEX_SIZE)
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-extern unsigned long pmd_hugepage_update(struct mm_struct *mm,
- unsigned long addr,
- pmd_t *pmdp,
- unsigned long clr,
- unsigned long set);
static inline char *get_hpte_slot_array(pmd_t *pmdp)
{
/*
@@ -253,50 +177,35 @@
* that for explicit huge pages.
*
*/
-static inline int pmd_trans_huge(pmd_t pmd)
+static inline int hash__pmd_trans_huge(pmd_t pmd)
{
- return !!((pmd_val(pmd) & (_PAGE_PTE | _PAGE_THP_HUGE)) ==
- (_PAGE_PTE | _PAGE_THP_HUGE));
+ return !!((pmd_val(pmd) & (_PAGE_PTE | H_PAGE_THP_HUGE)) ==
+ (_PAGE_PTE | H_PAGE_THP_HUGE));
}
-static inline int pmd_large(pmd_t pmd)
+static inline int hash__pmd_same(pmd_t pmd_a, pmd_t pmd_b)
{
- return !!(pmd_val(pmd) & _PAGE_PTE);
+ return (((pmd_raw(pmd_a) ^ pmd_raw(pmd_b)) & ~cpu_to_be64(_PAGE_HPTEFLAGS)) == 0);
}
-static inline pmd_t pmd_mknotpresent(pmd_t pmd)
+static inline pmd_t hash__pmd_mkhuge(pmd_t pmd)
{
- return __pmd(pmd_val(pmd) & ~_PAGE_PRESENT);
+ return __pmd(pmd_val(pmd) | (_PAGE_PTE | H_PAGE_THP_HUGE));
}
-#define __HAVE_ARCH_PMD_SAME
-static inline int pmd_same(pmd_t pmd_a, pmd_t pmd_b)
-{
- return (((pmd_val(pmd_a) ^ pmd_val(pmd_b)) & ~_PAGE_HPTEFLAGS) == 0);
-}
-
-static inline int __pmdp_test_and_clear_young(struct mm_struct *mm,
- unsigned long addr, pmd_t *pmdp)
-{
- unsigned long old;
-
- if ((pmd_val(*pmdp) & (_PAGE_ACCESSED | _PAGE_HASHPTE)) == 0)
- return 0;
- old = pmd_hugepage_update(mm, addr, pmdp, _PAGE_ACCESSED, 0);
- return ((old & _PAGE_ACCESSED) != 0);
-}
-
-#define __HAVE_ARCH_PMDP_SET_WRPROTECT
-static inline void pmdp_set_wrprotect(struct mm_struct *mm, unsigned long addr,
- pmd_t *pmdp)
-{
-
- if ((pmd_val(*pmdp) & _PAGE_RW) == 0)
- return;
-
- pmd_hugepage_update(mm, addr, pmdp, _PAGE_RW, 0);
-}
-
+extern unsigned long hash__pmd_hugepage_update(struct mm_struct *mm,
+ unsigned long addr, pmd_t *pmdp,
+ unsigned long clr, unsigned long set);
+extern pmd_t hash__pmdp_collapse_flush(struct vm_area_struct *vma,
+ unsigned long address, pmd_t *pmdp);
+extern void hash__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,
+ pgtable_t pgtable);
+extern pgtable_t hash__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp);
+extern void hash__pmdp_huge_split_prepare(struct vm_area_struct *vma,
+ unsigned long address, pmd_t *pmdp);
+extern pmd_t hash__pmdp_huge_get_and_clear(struct mm_struct *mm,
+ unsigned long addr, pmd_t *pmdp);
+extern int hash__has_transparent_hugepage(void);
#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
#endif /* __ASSEMBLY__ */
diff --git a/arch/powerpc/include/asm/book3s/64/hash.h b/arch/powerpc/include/asm/book3s/64/hash.h
index d0ee6fc..f61cad3 100644
--- a/arch/powerpc/include/asm/book3s/64/hash.h
+++ b/arch/powerpc/include/asm/book3s/64/hash.h
@@ -13,48 +13,12 @@
* We could create separate kernel read-only if we used the 3 PP bits
* combinations that newer processors provide but we currently don't.
*/
-#define _PAGE_BIT_SWAP_TYPE 0
-
-#define _PAGE_EXEC 0x00001 /* execute permission */
-#define _PAGE_RW 0x00002 /* read & write access allowed */
-#define _PAGE_READ 0x00004 /* read access allowed */
-#define _PAGE_USER 0x00008 /* page may be accessed by userspace */
-#define _PAGE_GUARDED 0x00010 /* G: guarded (side-effect) page */
-/* M (memory coherence) is always set in the HPTE, so we don't need it here */
-#define _PAGE_COHERENT 0x0
-#define _PAGE_NO_CACHE 0x00020 /* I: cache inhibit */
-#define _PAGE_WRITETHRU 0x00040 /* W: cache write-through */
-#define _PAGE_DIRTY 0x00080 /* C: page changed */
-#define _PAGE_ACCESSED 0x00100 /* R: page referenced */
-#define _PAGE_SPECIAL 0x00400 /* software: special page */
-#define _PAGE_BUSY 0x00800 /* software: PTE & hash are busy */
-
-#ifdef CONFIG_MEM_SOFT_DIRTY
-#define _PAGE_SOFT_DIRTY 0x200 /* software: software dirty tracking */
-#else
-#define _PAGE_SOFT_DIRTY 0x000
-#endif
-
-#define _PAGE_F_GIX_SHIFT 57
-#define _PAGE_F_GIX (7ul << 57) /* HPTE index within HPTEG */
-#define _PAGE_F_SECOND (1ul << 60) /* HPTE is in 2ndary HPTEG */
-#define _PAGE_HASHPTE (1ul << 61) /* PTE has associated HPTE */
-#define _PAGE_PTE (1ul << 62) /* distinguishes PTEs from pointers */
-#define _PAGE_PRESENT (1ul << 63) /* pte contains a translation */
-
-/*
- * We need to differentiate between explicit huge page and THP huge
- * page, since THP huge page also need to track real subpage details
- */
-#define _PAGE_THP_HUGE _PAGE_4K_PFN
-
-/*
- * set of bits not changed in pmd_modify.
- */
-#define _HPAGE_CHG_MASK (PTE_RPN_MASK | _PAGE_HPTEFLAGS | _PAGE_DIRTY | \
- _PAGE_ACCESSED | _PAGE_THP_HUGE | _PAGE_PTE | \
- _PAGE_SOFT_DIRTY)
-
+#define H_PAGE_BUSY 0x00800 /* software: PTE & hash are busy */
+#define H_PTE_NONE_MASK _PAGE_HPTEFLAGS
+#define H_PAGE_F_GIX_SHIFT 57
+#define H_PAGE_F_GIX (7ul << 57) /* HPTE index within HPTEG */
+#define H_PAGE_F_SECOND (1ul << 60) /* HPTE is in 2ndary HPTEG */
+#define H_PAGE_HASHPTE (1ul << 61) /* PTE has associated HPTE */
#ifdef CONFIG_PPC_64K_PAGES
#include <asm/book3s/64/hash-64k.h>
@@ -65,29 +29,33 @@
/*
* Size of EA range mapped by our pagetables.
*/
-#define PGTABLE_EADDR_SIZE (PTE_INDEX_SIZE + PMD_INDEX_SIZE + \
- PUD_INDEX_SIZE + PGD_INDEX_SIZE + PAGE_SHIFT)
-#define PGTABLE_RANGE (ASM_CONST(1) << PGTABLE_EADDR_SIZE)
+#define H_PGTABLE_EADDR_SIZE (H_PTE_INDEX_SIZE + H_PMD_INDEX_SIZE + \
+ H_PUD_INDEX_SIZE + H_PGD_INDEX_SIZE + PAGE_SHIFT)
+#define H_PGTABLE_RANGE (ASM_CONST(1) << H_PGTABLE_EADDR_SIZE)
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-#define PMD_CACHE_INDEX (PMD_INDEX_SIZE + 1)
+/*
+ * only with hash we need to use the second half of pmd page table
+ * to store pointer to deposited pgtable_t
+ */
+#define H_PMD_CACHE_INDEX (H_PMD_INDEX_SIZE + 1)
#else
-#define PMD_CACHE_INDEX PMD_INDEX_SIZE
+#define H_PMD_CACHE_INDEX H_PMD_INDEX_SIZE
#endif
/*
* Define the address range of the kernel non-linear virtual area
*/
-#define KERN_VIRT_START ASM_CONST(0xD000000000000000)
-#define KERN_VIRT_SIZE ASM_CONST(0x0000100000000000)
+#define H_KERN_VIRT_START ASM_CONST(0xD000000000000000)
+#define H_KERN_VIRT_SIZE ASM_CONST(0x0000100000000000)
/*
* The vmalloc space starts at the beginning of that region, and
* occupies half of it on hash CPUs and a quarter of it on Book3E
* (we keep a quarter for the virtual memmap)
*/
-#define VMALLOC_START KERN_VIRT_START
-#define VMALLOC_SIZE (KERN_VIRT_SIZE >> 1)
-#define VMALLOC_END (VMALLOC_START + VMALLOC_SIZE)
+#define H_VMALLOC_START H_KERN_VIRT_START
+#define H_VMALLOC_SIZE (H_KERN_VIRT_SIZE >> 1)
+#define H_VMALLOC_END (H_VMALLOC_START + H_VMALLOC_SIZE)
/*
* Region IDs
@@ -96,7 +64,7 @@
#define REGION_MASK (0xfUL << REGION_SHIFT)
#define REGION_ID(ea) (((unsigned long)(ea)) >> REGION_SHIFT)
-#define VMALLOC_REGION_ID (REGION_ID(VMALLOC_START))
+#define VMALLOC_REGION_ID (REGION_ID(H_VMALLOC_START))
#define KERNEL_REGION_ID (REGION_ID(PAGE_OFFSET))
#define VMEMMAP_REGION_ID (0xfUL) /* Server only */
#define USER_REGION_ID (0UL)
@@ -105,381 +73,97 @@
* Defines the address of the vmemap area, in its own region on
* hash table CPUs.
*/
-#define VMEMMAP_BASE (VMEMMAP_REGION_ID << REGION_SHIFT)
+#define H_VMEMMAP_BASE (VMEMMAP_REGION_ID << REGION_SHIFT)
#ifdef CONFIG_PPC_MM_SLICES
#define HAVE_ARCH_UNMAPPED_AREA
#define HAVE_ARCH_UNMAPPED_AREA_TOPDOWN
#endif /* CONFIG_PPC_MM_SLICES */
-/* No separate kernel read-only */
-#define _PAGE_KERNEL_RW (_PAGE_RW | _PAGE_DIRTY) /* user access blocked by key */
-#define _PAGE_KERNEL_RO _PAGE_KERNEL_RW
-#define _PAGE_KERNEL_RWX (_PAGE_DIRTY | _PAGE_RW | _PAGE_EXEC)
-
-/* Strong Access Ordering */
-#define _PAGE_SAO (_PAGE_WRITETHRU | _PAGE_NO_CACHE | _PAGE_COHERENT)
-
-/* No page size encoding in the linux PTE */
-#define _PAGE_PSIZE 0
/* PTEIDX nibble */
#define _PTEIDX_SECONDARY 0x8
#define _PTEIDX_GROUP_IX 0x7
-/* Hash table based platforms need atomic updates of the linux PTE */
-#define PTE_ATOMIC_UPDATES 1
-#define _PTE_NONE_MASK _PAGE_HPTEFLAGS
-/*
- * The mask convered by the RPN must be a ULL on 32-bit platforms with
- * 64-bit PTEs
- */
-#define PTE_RPN_MASK (((1UL << PTE_RPN_SIZE) - 1) << PTE_RPN_SHIFT)
-/*
- * _PAGE_CHG_MASK masks of bits that are to be preserved across
- * pgprot changes
- */
-#define _PAGE_CHG_MASK (PTE_RPN_MASK | _PAGE_HPTEFLAGS | _PAGE_DIRTY | \
- _PAGE_ACCESSED | _PAGE_SPECIAL | _PAGE_PTE | \
- _PAGE_SOFT_DIRTY)
-/*
- * Mask of bits returned by pte_pgprot()
- */
-#define PAGE_PROT_BITS (_PAGE_GUARDED | _PAGE_COHERENT | _PAGE_NO_CACHE | \
- _PAGE_WRITETHRU | _PAGE_4K_PFN | \
- _PAGE_USER | _PAGE_ACCESSED | \
- _PAGE_RW | _PAGE_DIRTY | _PAGE_EXEC | \
- _PAGE_SOFT_DIRTY)
-/*
- * We define 2 sets of base prot bits, one for basic pages (ie,
- * cacheable kernel and user pages) and one for non cacheable
- * pages. We always set _PAGE_COHERENT when SMP is enabled or
- * the processor might need it for DMA coherency.
- */
-#define _PAGE_BASE_NC (_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_PSIZE)
-#define _PAGE_BASE (_PAGE_BASE_NC | _PAGE_COHERENT)
-
-/* Permission masks used to generate the __P and __S table,
- *
- * Note:__pgprot is defined in arch/powerpc/include/asm/page.h
- *
- * Write permissions imply read permissions for now (we could make write-only
- * pages on BookE but we don't bother for now). Execute permission control is
- * possible on platforms that define _PAGE_EXEC
- *
- * Note due to the way vm flags are laid out, the bits are XWR
- */
-#define PAGE_NONE __pgprot(_PAGE_BASE)
-#define PAGE_SHARED __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_RW)
-#define PAGE_SHARED_X __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_RW | \
- _PAGE_EXEC)
-#define PAGE_COPY __pgprot(_PAGE_BASE | _PAGE_USER )
-#define PAGE_COPY_X __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_EXEC)
-#define PAGE_READONLY __pgprot(_PAGE_BASE | _PAGE_USER )
-#define PAGE_READONLY_X __pgprot(_PAGE_BASE | _PAGE_USER | _PAGE_EXEC)
-
-#define __P000 PAGE_NONE
-#define __P001 PAGE_READONLY
-#define __P010 PAGE_COPY
-#define __P011 PAGE_COPY
-#define __P100 PAGE_READONLY_X
-#define __P101 PAGE_READONLY_X
-#define __P110 PAGE_COPY_X
-#define __P111 PAGE_COPY_X
-
-#define __S000 PAGE_NONE
-#define __S001 PAGE_READONLY
-#define __S010 PAGE_SHARED
-#define __S011 PAGE_SHARED
-#define __S100 PAGE_READONLY_X
-#define __S101 PAGE_READONLY_X
-#define __S110 PAGE_SHARED_X
-#define __S111 PAGE_SHARED_X
-
-/* Permission masks used for kernel mappings */
-#define PAGE_KERNEL __pgprot(_PAGE_BASE | _PAGE_KERNEL_RW)
-#define PAGE_KERNEL_NC __pgprot(_PAGE_BASE_NC | _PAGE_KERNEL_RW | \
- _PAGE_NO_CACHE)
-#define PAGE_KERNEL_NCG __pgprot(_PAGE_BASE_NC | _PAGE_KERNEL_RW | \
- _PAGE_NO_CACHE | _PAGE_GUARDED)
-#define PAGE_KERNEL_X __pgprot(_PAGE_BASE | _PAGE_KERNEL_RWX)
-#define PAGE_KERNEL_RO __pgprot(_PAGE_BASE | _PAGE_KERNEL_RO)
-#define PAGE_KERNEL_ROX __pgprot(_PAGE_BASE | _PAGE_KERNEL_ROX)
-
-/* Protection used for kernel text. We want the debuggers to be able to
- * set breakpoints anywhere, so don't write protect the kernel text
- * on platforms where such control is possible.
- */
-#if defined(CONFIG_KGDB) || defined(CONFIG_XMON) || defined(CONFIG_BDI_SWITCH) ||\
- defined(CONFIG_KPROBES) || defined(CONFIG_DYNAMIC_FTRACE)
-#define PAGE_KERNEL_TEXT PAGE_KERNEL_X
-#else
-#define PAGE_KERNEL_TEXT PAGE_KERNEL_ROX
-#endif
-
-/* Make modules code happy. We don't set RO yet */
-#define PAGE_KERNEL_EXEC PAGE_KERNEL_X
-#define PAGE_AGP (PAGE_KERNEL_NC)
-
-#define PMD_BAD_BITS (PTE_TABLE_SIZE-1)
-#define PUD_BAD_BITS (PMD_TABLE_SIZE-1)
+#define H_PMD_BAD_BITS (PTE_TABLE_SIZE-1)
+#define H_PUD_BAD_BITS (PMD_TABLE_SIZE-1)
#ifndef __ASSEMBLY__
-#define pmd_bad(pmd) (pmd_val(pmd) & PMD_BAD_BITS)
-#define pmd_page_vaddr(pmd) __va(pmd_val(pmd) & ~PMD_MASKED_BITS)
-
-#define pud_bad(pud) (pud_val(pud) & PUD_BAD_BITS)
-#define pud_page_vaddr(pud) __va(pud_val(pud) & ~PUD_MASKED_BITS)
-
-/* Pointers in the page table tree are physical addresses */
-#define __pgtable_ptr_val(ptr) __pa(ptr)
-
-#define pgd_index(address) (((address) >> (PGDIR_SHIFT)) & (PTRS_PER_PGD - 1))
-#define pud_index(address) (((address) >> (PUD_SHIFT)) & (PTRS_PER_PUD - 1))
-#define pmd_index(address) (((address) >> (PMD_SHIFT)) & (PTRS_PER_PMD - 1))
-#define pte_index(address) (((address) >> (PAGE_SHIFT)) & (PTRS_PER_PTE - 1))
+#define hash__pmd_bad(pmd) (pmd_val(pmd) & H_PMD_BAD_BITS)
+#define hash__pud_bad(pud) (pud_val(pud) & H_PUD_BAD_BITS)
+static inline int hash__pgd_bad(pgd_t pgd)
+{
+ return (pgd_val(pgd) == 0);
+}
extern void hpte_need_flush(struct mm_struct *mm, unsigned long addr,
pte_t *ptep, unsigned long pte, int huge);
extern unsigned long htab_convert_pte_flags(unsigned long pteflags);
/* Atomic PTE updates */
-static inline unsigned long pte_update(struct mm_struct *mm,
- unsigned long addr,
- pte_t *ptep, unsigned long clr,
- unsigned long set,
- int huge)
+static inline unsigned long hash__pte_update(struct mm_struct *mm,
+ unsigned long addr,
+ pte_t *ptep, unsigned long clr,
+ unsigned long set,
+ int huge)
{
- unsigned long old, tmp;
+ __be64 old_be, tmp_be;
+ unsigned long old;
__asm__ __volatile__(
"1: ldarx %0,0,%3 # pte_update\n\
- andi. %1,%0,%6\n\
+ and. %1,%0,%6\n\
bne- 1b \n\
andc %1,%0,%4 \n\
or %1,%1,%7\n\
stdcx. %1,0,%3 \n\
bne- 1b"
- : "=&r" (old), "=&r" (tmp), "=m" (*ptep)
- : "r" (ptep), "r" (clr), "m" (*ptep), "i" (_PAGE_BUSY), "r" (set)
+ : "=&r" (old_be), "=&r" (tmp_be), "=m" (*ptep)
+ : "r" (ptep), "r" (cpu_to_be64(clr)), "m" (*ptep),
+ "r" (cpu_to_be64(H_PAGE_BUSY)), "r" (cpu_to_be64(set))
: "cc" );
/* huge pages use the old page table lock */
if (!huge)
assert_pte_locked(mm, addr);
- if (old & _PAGE_HASHPTE)
+ old = be64_to_cpu(old_be);
+ if (old & H_PAGE_HASHPTE)
hpte_need_flush(mm, addr, ptep, old, huge);
return old;
}
-static inline int __ptep_test_and_clear_young(struct mm_struct *mm,
- unsigned long addr, pte_t *ptep)
-{
- unsigned long old;
-
- if ((pte_val(*ptep) & (_PAGE_ACCESSED | _PAGE_HASHPTE)) == 0)
- return 0;
- old = pte_update(mm, addr, ptep, _PAGE_ACCESSED, 0, 0);
- return (old & _PAGE_ACCESSED) != 0;
-}
-#define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG
-#define ptep_test_and_clear_young(__vma, __addr, __ptep) \
-({ \
- int __r; \
- __r = __ptep_test_and_clear_young((__vma)->vm_mm, __addr, __ptep); \
- __r; \
-})
-
-#define __HAVE_ARCH_PTEP_SET_WRPROTECT
-static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep)
-{
-
- if ((pte_val(*ptep) & _PAGE_RW) == 0)
- return;
-
- pte_update(mm, addr, ptep, _PAGE_RW, 0, 0);
-}
-
-static inline void huge_ptep_set_wrprotect(struct mm_struct *mm,
- unsigned long addr, pte_t *ptep)
-{
- if ((pte_val(*ptep) & _PAGE_RW) == 0)
- return;
-
- pte_update(mm, addr, ptep, _PAGE_RW, 0, 1);
-}
-
-/*
- * We currently remove entries from the hashtable regardless of whether
- * the entry was young or dirty. The generic routines only flush if the
- * entry was young or dirty which is not good enough.
- *
- * We should be more intelligent about this but for the moment we override
- * these functions and force a tlb flush unconditionally
- */
-#define __HAVE_ARCH_PTEP_CLEAR_YOUNG_FLUSH
-#define ptep_clear_flush_young(__vma, __address, __ptep) \
-({ \
- int __young = __ptep_test_and_clear_young((__vma)->vm_mm, __address, \
- __ptep); \
- __young; \
-})
-
-#define __HAVE_ARCH_PTEP_GET_AND_CLEAR
-static inline pte_t ptep_get_and_clear(struct mm_struct *mm,
- unsigned long addr, pte_t *ptep)
-{
- unsigned long old = pte_update(mm, addr, ptep, ~0UL, 0, 0);
- return __pte(old);
-}
-
-static inline void pte_clear(struct mm_struct *mm, unsigned long addr,
- pte_t * ptep)
-{
- pte_update(mm, addr, ptep, ~0UL, 0, 0);
-}
-
-
/* Set the dirty and/or accessed bits atomically in a linux PTE, this
* function doesn't need to flush the hash entry
*/
-static inline void __ptep_set_access_flags(pte_t *ptep, pte_t entry)
+static inline void hash__ptep_set_access_flags(pte_t *ptep, pte_t entry)
{
- unsigned long bits = pte_val(entry) &
- (_PAGE_DIRTY | _PAGE_ACCESSED | _PAGE_RW | _PAGE_EXEC |
- _PAGE_SOFT_DIRTY);
+ __be64 old, tmp, val, mask;
- unsigned long old, tmp;
+ mask = cpu_to_be64(_PAGE_DIRTY | _PAGE_ACCESSED | _PAGE_READ | _PAGE_WRITE |
+ _PAGE_EXEC | _PAGE_SOFT_DIRTY);
+
+ val = pte_raw(entry) & mask;
__asm__ __volatile__(
"1: ldarx %0,0,%4\n\
- andi. %1,%0,%6\n\
+ and. %1,%0,%6\n\
bne- 1b \n\
or %0,%3,%0\n\
stdcx. %0,0,%4\n\
bne- 1b"
:"=&r" (old), "=&r" (tmp), "=m" (*ptep)
- :"r" (bits), "r" (ptep), "m" (*ptep), "i" (_PAGE_BUSY)
+ :"r" (val), "r" (ptep), "m" (*ptep), "r" (cpu_to_be64(H_PAGE_BUSY))
:"cc");
}
-static inline int pgd_bad(pgd_t pgd)
+static inline int hash__pte_same(pte_t pte_a, pte_t pte_b)
{
- return (pgd_val(pgd) == 0);
+ return (((pte_raw(pte_a) ^ pte_raw(pte_b)) & ~cpu_to_be64(_PAGE_HPTEFLAGS)) == 0);
}
-#define __HAVE_ARCH_PTE_SAME
-#define pte_same(A,B) (((pte_val(A) ^ pte_val(B)) & ~_PAGE_HPTEFLAGS) == 0)
-static inline unsigned long pgd_page_vaddr(pgd_t pgd)
+static inline int hash__pte_none(pte_t pte)
{
- return (unsigned long)__va(pgd_val(pgd) & ~PGD_MASKED_BITS);
-}
-
-
-/* Generic accessors to PTE bits */
-static inline int pte_write(pte_t pte) { return !!(pte_val(pte) & _PAGE_RW);}
-static inline int pte_dirty(pte_t pte) { return !!(pte_val(pte) & _PAGE_DIRTY); }
-static inline int pte_young(pte_t pte) { return !!(pte_val(pte) & _PAGE_ACCESSED); }
-static inline int pte_special(pte_t pte) { return !!(pte_val(pte) & _PAGE_SPECIAL); }
-static inline int pte_none(pte_t pte) { return (pte_val(pte) & ~_PTE_NONE_MASK) == 0; }
-static inline pgprot_t pte_pgprot(pte_t pte) { return __pgprot(pte_val(pte) & PAGE_PROT_BITS); }
-
-#ifdef CONFIG_HAVE_ARCH_SOFT_DIRTY
-static inline bool pte_soft_dirty(pte_t pte)
-{
- return !!(pte_val(pte) & _PAGE_SOFT_DIRTY);
-}
-static inline pte_t pte_mksoft_dirty(pte_t pte)
-{
- return __pte(pte_val(pte) | _PAGE_SOFT_DIRTY);
-}
-
-static inline pte_t pte_clear_soft_dirty(pte_t pte)
-{
- return __pte(pte_val(pte) & ~_PAGE_SOFT_DIRTY);
-}
-#endif /* CONFIG_HAVE_ARCH_SOFT_DIRTY */
-
-#ifdef CONFIG_NUMA_BALANCING
-/*
- * These work without NUMA balancing but the kernel does not care. See the
- * comment in include/asm-generic/pgtable.h . On powerpc, this will only
- * work for user pages and always return true for kernel pages.
- */
-static inline int pte_protnone(pte_t pte)
-{
- return (pte_val(pte) &
- (_PAGE_PRESENT | _PAGE_USER)) == _PAGE_PRESENT;
-}
-#endif /* CONFIG_NUMA_BALANCING */
-
-static inline int pte_present(pte_t pte)
-{
- return !!(pte_val(pte) & _PAGE_PRESENT);
-}
-
-/* Conversion functions: convert a page and protection to a page entry,
- * and a page entry and page directory to the page they refer to.
- *
- * Even if PTEs can be unsigned long long, a PFN is always an unsigned
- * long for now.
- */
-static inline pte_t pfn_pte(unsigned long pfn, pgprot_t pgprot)
-{
- return __pte((((pte_basic_t)(pfn) << PTE_RPN_SHIFT) & PTE_RPN_MASK) |
- pgprot_val(pgprot));
-}
-
-static inline unsigned long pte_pfn(pte_t pte)
-{
- return (pte_val(pte) & PTE_RPN_MASK) >> PTE_RPN_SHIFT;
-}
-
-/* Generic modifiers for PTE bits */
-static inline pte_t pte_wrprotect(pte_t pte)
-{
- return __pte(pte_val(pte) & ~_PAGE_RW);
-}
-
-static inline pte_t pte_mkclean(pte_t pte)
-{
- return __pte(pte_val(pte) & ~_PAGE_DIRTY);
-}
-
-static inline pte_t pte_mkold(pte_t pte)
-{
- return __pte(pte_val(pte) & ~_PAGE_ACCESSED);
-}
-
-static inline pte_t pte_mkwrite(pte_t pte)
-{
- return __pte(pte_val(pte) | _PAGE_RW);
-}
-
-static inline pte_t pte_mkdirty(pte_t pte)
-{
- return __pte(pte_val(pte) | _PAGE_DIRTY | _PAGE_SOFT_DIRTY);
-}
-
-static inline pte_t pte_mkyoung(pte_t pte)
-{
- return __pte(pte_val(pte) | _PAGE_ACCESSED);
-}
-
-static inline pte_t pte_mkspecial(pte_t pte)
-{
- return __pte(pte_val(pte) | _PAGE_SPECIAL);
-}
-
-static inline pte_t pte_mkhuge(pte_t pte)
-{
- return pte;
-}
-
-static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
-{
- return __pte((pte_val(pte) & _PAGE_CHG_MASK) | pgprot_val(newprot));
+ return (pte_val(pte) & ~H_PTE_NONE_MASK) == 0;
}
/* This low level function performs the actual PTE insertion
@@ -487,8 +171,8 @@
* an horrible mess that I'm not going to try to clean up now but
* I'm keeping it in one place rather than spread around
*/
-static inline void __set_pte_at(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, pte_t pte, int percpu)
+static inline void hash__set_pte_at(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, pte_t pte, int percpu)
{
/*
* Anything else just stores the PTE normally. That covers all 64-bit
@@ -497,53 +181,6 @@
*ptep = pte;
}
-/*
- * Macro to mark a page protection value as "uncacheable".
- */
-
-#define _PAGE_CACHE_CTL (_PAGE_COHERENT | _PAGE_GUARDED | _PAGE_NO_CACHE | \
- _PAGE_WRITETHRU)
-
-#define pgprot_noncached pgprot_noncached
-static inline pgprot_t pgprot_noncached(pgprot_t prot)
-{
- return __pgprot((pgprot_val(prot) & ~_PAGE_CACHE_CTL) |
- _PAGE_NO_CACHE | _PAGE_GUARDED);
-}
-
-#define pgprot_noncached_wc pgprot_noncached_wc
-static inline pgprot_t pgprot_noncached_wc(pgprot_t prot)
-{
- return __pgprot((pgprot_val(prot) & ~_PAGE_CACHE_CTL) |
- _PAGE_NO_CACHE);
-}
-
-#define pgprot_cached pgprot_cached
-static inline pgprot_t pgprot_cached(pgprot_t prot)
-{
- return __pgprot((pgprot_val(prot) & ~_PAGE_CACHE_CTL) |
- _PAGE_COHERENT);
-}
-
-#define pgprot_cached_wthru pgprot_cached_wthru
-static inline pgprot_t pgprot_cached_wthru(pgprot_t prot)
-{
- return __pgprot((pgprot_val(prot) & ~_PAGE_CACHE_CTL) |
- _PAGE_COHERENT | _PAGE_WRITETHRU);
-}
-
-#define pgprot_cached_noncoherent pgprot_cached_noncoherent
-static inline pgprot_t pgprot_cached_noncoherent(pgprot_t prot)
-{
- return __pgprot(pgprot_val(prot) & ~_PAGE_CACHE_CTL);
-}
-
-#define pgprot_writecombine pgprot_writecombine
-static inline pgprot_t pgprot_writecombine(pgprot_t prot)
-{
- return pgprot_noncached_wc(prot);
-}
-
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
extern void hpte_do_hugepage_flush(struct mm_struct *mm, unsigned long addr,
pmd_t *pmdp, unsigned long old_pmd);
@@ -556,6 +193,14 @@
}
#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
+
+extern int hash__map_kernel_page(unsigned long ea, unsigned long pa,
+ unsigned long flags);
+extern int __meminit hash__vmemmap_create_mapping(unsigned long start,
+ unsigned long page_size,
+ unsigned long phys);
+extern void hash__vmemmap_remove_mapping(unsigned long start,
+ unsigned long page_size);
#endif /* !__ASSEMBLY__ */
#endif /* __KERNEL__ */
#endif /* _ASM_POWERPC_BOOK3S_64_HASH_H */
diff --git a/arch/powerpc/include/asm/book3s/64/hugetlb-radix.h b/arch/powerpc/include/asm/book3s/64/hugetlb-radix.h
new file mode 100644
index 0000000..60f47649
--- /dev/null
+++ b/arch/powerpc/include/asm/book3s/64/hugetlb-radix.h
@@ -0,0 +1,14 @@
+#ifndef _ASM_POWERPC_BOOK3S_64_HUGETLB_RADIX_H
+#define _ASM_POWERPC_BOOK3S_64_HUGETLB_RADIX_H
+/*
+ * For radix we want generic code to handle hugetlb. But then if we want
+ * both hash and radix to be enabled together we need to workaround the
+ * limitations.
+ */
+void radix__flush_hugetlb_page(struct vm_area_struct *vma, unsigned long vmaddr);
+void radix__local_flush_hugetlb_page(struct vm_area_struct *vma, unsigned long vmaddr);
+extern unsigned long
+radix__hugetlb_get_unmapped_area(struct file *file, unsigned long addr,
+ unsigned long len, unsigned long pgoff,
+ unsigned long flags);
+#endif
diff --git a/arch/powerpc/include/asm/book3s/64/mmu-hash.h b/arch/powerpc/include/asm/book3s/64/mmu-hash.h
index 0cea480..290157e 100644
--- a/arch/powerpc/include/asm/book3s/64/mmu-hash.h
+++ b/arch/powerpc/include/asm/book3s/64/mmu-hash.h
@@ -1,5 +1,5 @@
-#ifndef _ASM_POWERPC_MMU_HASH64_H_
-#define _ASM_POWERPC_MMU_HASH64_H_
+#ifndef _ASM_POWERPC_BOOK3S_64_MMU_HASH_H_
+#define _ASM_POWERPC_BOOK3S_64_MMU_HASH_H_
/*
* PowerPC64 memory management structures
*
@@ -78,6 +78,10 @@
#define HPTE_V_SECONDARY ASM_CONST(0x0000000000000002)
#define HPTE_V_VALID ASM_CONST(0x0000000000000001)
+/*
+ * ISA 3.0 have a different HPTE format.
+ */
+#define HPTE_R_3_0_SSIZE_SHIFT 58
#define HPTE_R_PP0 ASM_CONST(0x8000000000000000)
#define HPTE_R_TS ASM_CONST(0x4000000000000000)
#define HPTE_R_KEY_HI ASM_CONST(0x3000000000000000)
@@ -115,6 +119,7 @@
#define POWER7_TLB_SETS 128 /* # sets in POWER7 TLB */
#define POWER8_TLB_SETS 512 /* # sets in POWER8 TLB */
#define POWER9_TLB_SETS_HASH 256 /* # sets in POWER9 TLB Hash mode */
+#define POWER9_TLB_SETS_RADIX 128 /* # sets in POWER9 TLB Radix mode */
#ifndef __ASSEMBLY__
@@ -127,24 +132,6 @@
extern unsigned long htab_size_bytes;
extern unsigned long htab_hash_mask;
-/*
- * Page size definition
- *
- * shift : is the "PAGE_SHIFT" value for that page size
- * sllp : is a bit mask with the value of SLB L || LP to be or'ed
- * directly to a slbmte "vsid" value
- * penc : is the HPTE encoding mask for the "LP" field:
- *
- */
-struct mmu_psize_def
-{
- unsigned int shift; /* number of bits */
- int penc[MMU_PAGE_COUNT]; /* HPTE encoding */
- unsigned int tlbiel; /* tlbiel supported for that page size */
- unsigned long avpnm; /* bits to mask out in AVPN in the HPTE */
- unsigned long sllp; /* SLB L||LP (exact mask to use in slbmte) */
-};
-extern struct mmu_psize_def mmu_psize_defs[MMU_PAGE_COUNT];
static inline int shift_to_mmu_psize(unsigned int shift)
{
@@ -210,11 +197,6 @@
/*
* The current system page and segment sizes
*/
-extern int mmu_linear_psize;
-extern int mmu_virtual_psize;
-extern int mmu_vmalloc_psize;
-extern int mmu_vmemmap_psize;
-extern int mmu_io_psize;
extern int mmu_kernel_ssize;
extern int mmu_highuser_ssize;
extern u16 mmu_slb_size;
@@ -247,7 +229,8 @@
*/
v = (vpn >> (23 - VPN_SHIFT)) & ~(mmu_psize_defs[psize].avpnm);
v <<= HPTE_V_AVPN_SHIFT;
- v |= ((unsigned long) ssize) << HPTE_V_SSIZE_SHIFT;
+ if (!cpu_has_feature(CPU_FTR_ARCH_300))
+ v |= ((unsigned long) ssize) << HPTE_V_SSIZE_SHIFT;
return v;
}
@@ -271,8 +254,12 @@
* aligned for the requested page size
*/
static inline unsigned long hpte_encode_r(unsigned long pa, int base_psize,
- int actual_psize)
+ int actual_psize, int ssize)
{
+
+ if (cpu_has_feature(CPU_FTR_ARCH_300))
+ pa |= ((unsigned long) ssize) << HPTE_R_3_0_SSIZE_SHIFT;
+
/* A 4K page needs no special encoding */
if (actual_psize == MMU_PAGE_4K)
return pa & HPTE_R_RPN;
@@ -476,7 +463,7 @@
add rt,rt,rx
/* 4 bits per slice and we have one slice per 1TB */
-#define SLICE_ARRAY_SIZE (PGTABLE_RANGE >> 41)
+#define SLICE_ARRAY_SIZE (H_PGTABLE_RANGE >> 41)
#ifndef __ASSEMBLY__
@@ -512,38 +499,6 @@
static inline void subpage_prot_init_new_context(struct mm_struct *mm) { }
#endif /* CONFIG_PPC_SUBPAGE_PROT */
-typedef unsigned long mm_context_id_t;
-struct spinlock;
-
-typedef struct {
- mm_context_id_t id;
- u16 user_psize; /* page size index */
-
-#ifdef CONFIG_PPC_MM_SLICES
- u64 low_slices_psize; /* SLB page size encodings */
- unsigned char high_slices_psize[SLICE_ARRAY_SIZE];
-#else
- u16 sllp; /* SLB page size encoding */
-#endif
- unsigned long vdso_base;
-#ifdef CONFIG_PPC_SUBPAGE_PROT
- struct subpage_prot_table spt;
-#endif /* CONFIG_PPC_SUBPAGE_PROT */
-#ifdef CONFIG_PPC_ICSWX
- struct spinlock *cop_lockp; /* guard acop and cop_pid */
- unsigned long acop; /* mask of enabled coprocessor types */
- unsigned int cop_pid; /* pid value used with coprocessors */
-#endif /* CONFIG_PPC_ICSWX */
-#ifdef CONFIG_PPC_64K_PAGES
- /* for 4K PTE fragment support */
- void *pte_frag;
-#endif
-#ifdef CONFIG_SPAPR_TCE_IOMMU
- struct list_head iommu_group_mem_list;
-#endif
-} mm_context_t;
-
-
#if 0
/*
* The code below is equivalent to this function for arguments
@@ -579,7 +534,7 @@
/*
* Bad address. We return VSID 0 for that
*/
- if ((ea & ~REGION_MASK) >= PGTABLE_RANGE)
+ if ((ea & ~REGION_MASK) >= H_PGTABLE_RANGE)
return 0;
if (ssize == MMU_SEGSIZE_256M)
@@ -613,4 +568,4 @@
#endif /* __ASSEMBLY__ */
-#endif /* _ASM_POWERPC_MMU_HASH64_H_ */
+#endif /* _ASM_POWERPC_BOOK3S_64_MMU_HASH_H_ */
diff --git a/arch/powerpc/include/asm/book3s/64/mmu.h b/arch/powerpc/include/asm/book3s/64/mmu.h
new file mode 100644
index 0000000..5854263
--- /dev/null
+++ b/arch/powerpc/include/asm/book3s/64/mmu.h
@@ -0,0 +1,137 @@
+#ifndef _ASM_POWERPC_BOOK3S_64_MMU_H_
+#define _ASM_POWERPC_BOOK3S_64_MMU_H_
+
+#ifndef __ASSEMBLY__
+/*
+ * Page size definition
+ *
+ * shift : is the "PAGE_SHIFT" value for that page size
+ * sllp : is a bit mask with the value of SLB L || LP to be or'ed
+ * directly to a slbmte "vsid" value
+ * penc : is the HPTE encoding mask for the "LP" field:
+ *
+ */
+struct mmu_psize_def {
+ unsigned int shift; /* number of bits */
+ int penc[MMU_PAGE_COUNT]; /* HPTE encoding */
+ unsigned int tlbiel; /* tlbiel supported for that page size */
+ unsigned long avpnm; /* bits to mask out in AVPN in the HPTE */
+ union {
+ unsigned long sllp; /* SLB L||LP (exact mask to use in slbmte) */
+ unsigned long ap; /* Ap encoding used by PowerISA 3.0 */
+ };
+};
+extern struct mmu_psize_def mmu_psize_defs[MMU_PAGE_COUNT];
+
+#define radix_enabled() mmu_has_feature(MMU_FTR_RADIX)
+
+#endif /* __ASSEMBLY__ */
+
+/* 64-bit classic hash table MMU */
+#include <asm/book3s/64/mmu-hash.h>
+
+#ifndef __ASSEMBLY__
+/*
+ * ISA 3.0 partiton and process table entry format
+ */
+struct prtb_entry {
+ __be64 prtb0;
+ __be64 prtb1;
+};
+extern struct prtb_entry *process_tb;
+
+struct patb_entry {
+ __be64 patb0;
+ __be64 patb1;
+};
+extern struct patb_entry *partition_tb;
+
+#define PATB_HR (1UL << 63)
+#define PATB_GR (1UL << 63)
+#define RPDB_MASK 0x0ffffffffffff00fUL
+#define RPDB_SHIFT (1UL << 8)
+/*
+ * Limit process table to PAGE_SIZE table. This
+ * also limit the max pid we can support.
+ * MAX_USER_CONTEXT * 16 bytes of space.
+ */
+#define PRTB_SIZE_SHIFT (CONTEXT_BITS + 4)
+/*
+ * Power9 currently only support 64K partition table size.
+ */
+#define PATB_SIZE_SHIFT 16
+
+typedef unsigned long mm_context_id_t;
+struct spinlock;
+
+typedef struct {
+ mm_context_id_t id;
+ u16 user_psize; /* page size index */
+
+#ifdef CONFIG_PPC_MM_SLICES
+ u64 low_slices_psize; /* SLB page size encodings */
+ unsigned char high_slices_psize[SLICE_ARRAY_SIZE];
+#else
+ u16 sllp; /* SLB page size encoding */
+#endif
+ unsigned long vdso_base;
+#ifdef CONFIG_PPC_SUBPAGE_PROT
+ struct subpage_prot_table spt;
+#endif /* CONFIG_PPC_SUBPAGE_PROT */
+#ifdef CONFIG_PPC_ICSWX
+ struct spinlock *cop_lockp; /* guard acop and cop_pid */
+ unsigned long acop; /* mask of enabled coprocessor types */
+ unsigned int cop_pid; /* pid value used with coprocessors */
+#endif /* CONFIG_PPC_ICSWX */
+#ifdef CONFIG_PPC_64K_PAGES
+ /* for 4K PTE fragment support */
+ void *pte_frag;
+#endif
+#ifdef CONFIG_SPAPR_TCE_IOMMU
+ struct list_head iommu_group_mem_list;
+#endif
+} mm_context_t;
+
+/*
+ * The current system page and segment sizes
+ */
+extern int mmu_linear_psize;
+extern int mmu_virtual_psize;
+extern int mmu_vmalloc_psize;
+extern int mmu_vmemmap_psize;
+extern int mmu_io_psize;
+
+/* MMU initialization */
+extern void radix_init_native(void);
+extern void hash__early_init_mmu(void);
+extern void radix__early_init_mmu(void);
+static inline void early_init_mmu(void)
+{
+ if (radix_enabled())
+ return radix__early_init_mmu();
+ return hash__early_init_mmu();
+}
+extern void hash__early_init_mmu_secondary(void);
+extern void radix__early_init_mmu_secondary(void);
+static inline void early_init_mmu_secondary(void)
+{
+ if (radix_enabled())
+ return radix__early_init_mmu_secondary();
+ return hash__early_init_mmu_secondary();
+}
+
+extern void hash__setup_initial_memory_limit(phys_addr_t first_memblock_base,
+ phys_addr_t first_memblock_size);
+extern void radix__setup_initial_memory_limit(phys_addr_t first_memblock_base,
+ phys_addr_t first_memblock_size);
+static inline void setup_initial_memory_limit(phys_addr_t first_memblock_base,
+ phys_addr_t first_memblock_size)
+{
+ if (radix_enabled())
+ return radix__setup_initial_memory_limit(first_memblock_base,
+ first_memblock_size);
+ return hash__setup_initial_memory_limit(first_memblock_base,
+ first_memblock_size);
+}
+#endif /* __ASSEMBLY__ */
+#endif /* _ASM_POWERPC_BOOK3S_64_MMU_H_ */
diff --git a/arch/powerpc/include/asm/book3s/64/pgalloc.h b/arch/powerpc/include/asm/book3s/64/pgalloc.h
new file mode 100644
index 0000000..488279e
--- /dev/null
+++ b/arch/powerpc/include/asm/book3s/64/pgalloc.h
@@ -0,0 +1,207 @@
+#ifndef _ASM_POWERPC_BOOK3S_64_PGALLOC_H
+#define _ASM_POWERPC_BOOK3S_64_PGALLOC_H
+/*
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <linux/slab.h>
+#include <linux/cpumask.h>
+#include <linux/percpu.h>
+
+struct vmemmap_backing {
+ struct vmemmap_backing *list;
+ unsigned long phys;
+ unsigned long virt_addr;
+};
+extern struct vmemmap_backing *vmemmap_list;
+
+/*
+ * Functions that deal with pagetables that could be at any level of
+ * the table need to be passed an "index_size" so they know how to
+ * handle allocation. For PTE pages (which are linked to a struct
+ * page for now, and drawn from the main get_free_pages() pool), the
+ * allocation size will be (2^index_size * sizeof(pointer)) and
+ * allocations are drawn from the kmem_cache in PGT_CACHE(index_size).
+ *
+ * The maximum index size needs to be big enough to allow any
+ * pagetable sizes we need, but small enough to fit in the low bits of
+ * any page table pointer. In other words all pagetables, even tiny
+ * ones, must be aligned to allow at least enough low 0 bits to
+ * contain this value. This value is also used as a mask, so it must
+ * be one less than a power of two.
+ */
+#define MAX_PGTABLE_INDEX_SIZE 0xf
+
+extern struct kmem_cache *pgtable_cache[];
+#define PGT_CACHE(shift) ({ \
+ BUG_ON(!(shift)); \
+ pgtable_cache[(shift) - 1]; \
+ })
+
+#define PGALLOC_GFP GFP_KERNEL | __GFP_NOTRACK | __GFP_REPEAT | __GFP_ZERO
+
+extern pte_t *pte_fragment_alloc(struct mm_struct *, unsigned long, int);
+extern void pte_fragment_free(unsigned long *, int);
+extern void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int shift);
+#ifdef CONFIG_SMP
+extern void __tlb_remove_table(void *_table);
+#endif
+
+static inline pgd_t *radix__pgd_alloc(struct mm_struct *mm)
+{
+#ifdef CONFIG_PPC_64K_PAGES
+ return (pgd_t *)__get_free_page(PGALLOC_GFP);
+#else
+ struct page *page;
+ page = alloc_pages(PGALLOC_GFP, 4);
+ if (!page)
+ return NULL;
+ return (pgd_t *) page_address(page);
+#endif
+}
+
+static inline void radix__pgd_free(struct mm_struct *mm, pgd_t *pgd)
+{
+#ifdef CONFIG_PPC_64K_PAGES
+ free_page((unsigned long)pgd);
+#else
+ free_pages((unsigned long)pgd, 4);
+#endif
+}
+
+static inline pgd_t *pgd_alloc(struct mm_struct *mm)
+{
+ if (radix_enabled())
+ return radix__pgd_alloc(mm);
+ return kmem_cache_alloc(PGT_CACHE(PGD_INDEX_SIZE), GFP_KERNEL);
+}
+
+static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd)
+{
+ if (radix_enabled())
+ return radix__pgd_free(mm, pgd);
+ kmem_cache_free(PGT_CACHE(PGD_INDEX_SIZE), pgd);
+}
+
+static inline void pgd_populate(struct mm_struct *mm, pgd_t *pgd, pud_t *pud)
+{
+ pgd_set(pgd, __pgtable_ptr_val(pud) | PGD_VAL_BITS);
+}
+
+static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr)
+{
+ return kmem_cache_alloc(PGT_CACHE(PUD_INDEX_SIZE),
+ GFP_KERNEL|__GFP_REPEAT);
+}
+
+static inline void pud_free(struct mm_struct *mm, pud_t *pud)
+{
+ kmem_cache_free(PGT_CACHE(PUD_INDEX_SIZE), pud);
+}
+
+static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd)
+{
+ pud_set(pud, __pgtable_ptr_val(pmd) | PUD_VAL_BITS);
+}
+
+static inline void __pud_free_tlb(struct mmu_gather *tlb, pud_t *pud,
+ unsigned long address)
+{
+ pgtable_free_tlb(tlb, pud, PUD_INDEX_SIZE);
+}
+
+static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long addr)
+{
+ return kmem_cache_alloc(PGT_CACHE(PMD_CACHE_INDEX),
+ GFP_KERNEL|__GFP_REPEAT);
+}
+
+static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd)
+{
+ kmem_cache_free(PGT_CACHE(PMD_CACHE_INDEX), pmd);
+}
+
+static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd,
+ unsigned long address)
+{
+ return pgtable_free_tlb(tlb, pmd, PMD_CACHE_INDEX);
+}
+
+static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd,
+ pte_t *pte)
+{
+ pmd_set(pmd, __pgtable_ptr_val(pte) | PMD_VAL_BITS);
+}
+
+static inline void pmd_populate(struct mm_struct *mm, pmd_t *pmd,
+ pgtable_t pte_page)
+{
+ pmd_set(pmd, __pgtable_ptr_val(pte_page) | PMD_VAL_BITS);
+}
+
+static inline pgtable_t pmd_pgtable(pmd_t pmd)
+{
+ return (pgtable_t)pmd_page_vaddr(pmd);
+}
+
+#ifdef CONFIG_PPC_4K_PAGES
+static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm,
+ unsigned long address)
+{
+ return (pte_t *)__get_free_page(GFP_KERNEL | __GFP_REPEAT | __GFP_ZERO);
+}
+
+static inline pgtable_t pte_alloc_one(struct mm_struct *mm,
+ unsigned long address)
+{
+ struct page *page;
+ pte_t *pte;
+
+ pte = pte_alloc_one_kernel(mm, address);
+ if (!pte)
+ return NULL;
+ page = virt_to_page(pte);
+ if (!pgtable_page_ctor(page)) {
+ __free_page(page);
+ return NULL;
+ }
+ return pte;
+}
+#else /* if CONFIG_PPC_64K_PAGES */
+
+static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm,
+ unsigned long address)
+{
+ return (pte_t *)pte_fragment_alloc(mm, address, 1);
+}
+
+static inline pgtable_t pte_alloc_one(struct mm_struct *mm,
+ unsigned long address)
+{
+ return (pgtable_t)pte_fragment_alloc(mm, address, 0);
+}
+#endif
+
+static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte)
+{
+ pte_fragment_free((unsigned long *)pte, 1);
+}
+
+static inline void pte_free(struct mm_struct *mm, pgtable_t ptepage)
+{
+ pte_fragment_free((unsigned long *)ptepage, 0);
+}
+
+static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t table,
+ unsigned long address)
+{
+ tlb_flush_pgtable(tlb, address);
+ pgtable_free_tlb(tlb, table, 0);
+}
+
+#define check_pgt_cache() do { } while (0)
+
+#endif /* _ASM_POWERPC_BOOK3S_64_PGALLOC_H */
diff --git a/arch/powerpc/include/asm/book3s/64/pgtable-4k.h b/arch/powerpc/include/asm/book3s/64/pgtable-4k.h
new file mode 100644
index 0000000..71e9abc
--- /dev/null
+++ b/arch/powerpc/include/asm/book3s/64/pgtable-4k.h
@@ -0,0 +1,53 @@
+#ifndef _ASM_POWERPC_BOOK3S_64_PGTABLE_4K_H
+#define _ASM_POWERPC_BOOK3S_64_PGTABLE_4K_H
+/*
+ * hash 4k can't share hugetlb and also doesn't support THP
+ */
+#ifndef __ASSEMBLY__
+#ifdef CONFIG_HUGETLB_PAGE
+static inline int pmd_huge(pmd_t pmd)
+{
+ /*
+ * leaf pte for huge page
+ */
+ if (radix_enabled())
+ return !!(pmd_val(pmd) & _PAGE_PTE);
+ return 0;
+}
+
+static inline int pud_huge(pud_t pud)
+{
+ /*
+ * leaf pte for huge page
+ */
+ if (radix_enabled())
+ return !!(pud_val(pud) & _PAGE_PTE);
+ return 0;
+}
+
+static inline int pgd_huge(pgd_t pgd)
+{
+ /*
+ * leaf pte for huge page
+ */
+ if (radix_enabled())
+ return !!(pgd_val(pgd) & _PAGE_PTE);
+ return 0;
+}
+#define pgd_huge pgd_huge
+/*
+ * With radix , we have hugepage ptes in the pud and pmd entries. We don't
+ * need to setup hugepage directory for them. Our pte and page directory format
+ * enable us to have this enabled.
+ */
+static inline int hugepd_ok(hugepd_t hpd)
+{
+ if (radix_enabled())
+ return 0;
+ return hash__hugepd_ok(hpd);
+}
+#define is_hugepd(hpd) (hugepd_ok(hpd))
+#endif /* CONFIG_HUGETLB_PAGE */
+#endif /* __ASSEMBLY__ */
+
+#endif /*_ASM_POWERPC_BOOK3S_64_PGTABLE_4K_H */
diff --git a/arch/powerpc/include/asm/book3s/64/pgtable-64k.h b/arch/powerpc/include/asm/book3s/64/pgtable-64k.h
new file mode 100644
index 0000000..cb2d0a5
--- /dev/null
+++ b/arch/powerpc/include/asm/book3s/64/pgtable-64k.h
@@ -0,0 +1,64 @@
+#ifndef _ASM_POWERPC_BOOK3S_64_PGTABLE_64K_H
+#define _ASM_POWERPC_BOOK3S_64_PGTABLE_64K_H
+
+#ifndef __ASSEMBLY__
+#ifdef CONFIG_HUGETLB_PAGE
+/*
+ * We have PGD_INDEX_SIZ = 12 and PTE_INDEX_SIZE = 8, so that we can have
+ * 16GB hugepage pte in PGD and 16MB hugepage pte at PMD;
+ *
+ * Defined in such a way that we can optimize away code block at build time
+ * if CONFIG_HUGETLB_PAGE=n.
+ */
+static inline int pmd_huge(pmd_t pmd)
+{
+ /*
+ * leaf pte for huge page
+ */
+ return !!(pmd_val(pmd) & _PAGE_PTE);
+}
+
+static inline int pud_huge(pud_t pud)
+{
+ /*
+ * leaf pte for huge page
+ */
+ return !!(pud_val(pud) & _PAGE_PTE);
+}
+
+static inline int pgd_huge(pgd_t pgd)
+{
+ /*
+ * leaf pte for huge page
+ */
+ return !!(pgd_val(pgd) & _PAGE_PTE);
+}
+#define pgd_huge pgd_huge
+
+#ifdef CONFIG_DEBUG_VM
+extern int hugepd_ok(hugepd_t hpd);
+#define is_hugepd(hpd) (hugepd_ok(hpd))
+#else
+/*
+ * With 64k page size, we have hugepage ptes in the pgd and pmd entries. We don't
+ * need to setup hugepage directory for them. Our pte and page directory format
+ * enable us to have this enabled.
+ */
+static inline int hugepd_ok(hugepd_t hpd)
+{
+ return 0;
+}
+#define is_hugepd(pdep) 0
+#endif /* CONFIG_DEBUG_VM */
+
+#endif /* CONFIG_HUGETLB_PAGE */
+
+static inline int remap_4k_pfn(struct vm_area_struct *vma, unsigned long addr,
+ unsigned long pfn, pgprot_t prot)
+{
+ if (radix_enabled())
+ BUG();
+ return hash__remap_4k_pfn(vma, addr, pfn, prot);
+}
+#endif /* __ASSEMBLY__ */
+#endif /*_ASM_POWERPC_BOOK3S_64_PGTABLE_64K_H */
diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h
index 77d3ce0..88a5eca 100644
--- a/arch/powerpc/include/asm/book3s/64/pgtable.h
+++ b/arch/powerpc/include/asm/book3s/64/pgtable.h
@@ -1,13 +1,247 @@
#ifndef _ASM_POWERPC_BOOK3S_64_PGTABLE_H_
#define _ASM_POWERPC_BOOK3S_64_PGTABLE_H_
+
/*
- * This file contains the functions and defines necessary to modify and use
- * the ppc64 hashed page table.
+ * Common bits between hash and Radix page table
*/
+#define _PAGE_BIT_SWAP_TYPE 0
+
+#define _PAGE_EXEC 0x00001 /* execute permission */
+#define _PAGE_WRITE 0x00002 /* write access allowed */
+#define _PAGE_READ 0x00004 /* read access allowed */
+#define _PAGE_RW (_PAGE_READ | _PAGE_WRITE)
+#define _PAGE_RWX (_PAGE_READ | _PAGE_WRITE | _PAGE_EXEC)
+#define _PAGE_PRIVILEGED 0x00008 /* kernel access only */
+#define _PAGE_SAO 0x00010 /* Strong access order */
+#define _PAGE_NON_IDEMPOTENT 0x00020 /* non idempotent memory */
+#define _PAGE_TOLERANT 0x00030 /* tolerant memory, cache inhibited */
+#define _PAGE_DIRTY 0x00080 /* C: page changed */
+#define _PAGE_ACCESSED 0x00100 /* R: page referenced */
+/*
+ * Software bits
+ */
+#define _RPAGE_SW0 0x2000000000000000UL
+#define _RPAGE_SW1 0x00800
+#define _RPAGE_SW2 0x00400
+#define _RPAGE_SW3 0x00200
+#ifdef CONFIG_MEM_SOFT_DIRTY
+#define _PAGE_SOFT_DIRTY _RPAGE_SW3 /* software: software dirty tracking */
+#else
+#define _PAGE_SOFT_DIRTY 0x00000
+#endif
+#define _PAGE_SPECIAL _RPAGE_SW2 /* software: special page */
+
+
+#define _PAGE_PTE (1ul << 62) /* distinguishes PTEs from pointers */
+#define _PAGE_PRESENT (1ul << 63) /* pte contains a translation */
+/*
+ * Drivers request for cache inhibited pte mapping using _PAGE_NO_CACHE
+ * Instead of fixing all of them, add an alternate define which
+ * maps CI pte mapping.
+ */
+#define _PAGE_NO_CACHE _PAGE_TOLERANT
+/*
+ * We support 57 bit real address in pte. Clear everything above 57, and
+ * every thing below PAGE_SHIFT;
+ */
+#define PTE_RPN_MASK (((1UL << 57) - 1) & (PAGE_MASK))
+/*
+ * set of bits not changed in pmd_modify. Even though we have hash specific bits
+ * in here, on radix we expect them to be zero.
+ */
+#define _HPAGE_CHG_MASK (PTE_RPN_MASK | _PAGE_HPTEFLAGS | _PAGE_DIRTY | \
+ _PAGE_ACCESSED | H_PAGE_THP_HUGE | _PAGE_PTE | \
+ _PAGE_SOFT_DIRTY)
+/*
+ * user access blocked by key
+ */
+#define _PAGE_KERNEL_RW (_PAGE_PRIVILEGED | _PAGE_RW | _PAGE_DIRTY)
+#define _PAGE_KERNEL_RO (_PAGE_PRIVILEGED | _PAGE_READ)
+#define _PAGE_KERNEL_RWX (_PAGE_PRIVILEGED | _PAGE_DIRTY | \
+ _PAGE_RW | _PAGE_EXEC)
+/*
+ * No page size encoding in the linux PTE
+ */
+#define _PAGE_PSIZE 0
+/*
+ * _PAGE_CHG_MASK masks of bits that are to be preserved across
+ * pgprot changes
+ */
+#define _PAGE_CHG_MASK (PTE_RPN_MASK | _PAGE_HPTEFLAGS | _PAGE_DIRTY | \
+ _PAGE_ACCESSED | _PAGE_SPECIAL | _PAGE_PTE | \
+ _PAGE_SOFT_DIRTY)
+/*
+ * Mask of bits returned by pte_pgprot()
+ */
+#define PAGE_PROT_BITS (_PAGE_SAO | _PAGE_NON_IDEMPOTENT | _PAGE_TOLERANT | \
+ H_PAGE_4K_PFN | _PAGE_PRIVILEGED | _PAGE_ACCESSED | \
+ _PAGE_READ | _PAGE_WRITE | _PAGE_DIRTY | _PAGE_EXEC | \
+ _PAGE_SOFT_DIRTY)
+/*
+ * We define 2 sets of base prot bits, one for basic pages (ie,
+ * cacheable kernel and user pages) and one for non cacheable
+ * pages. We always set _PAGE_COHERENT when SMP is enabled or
+ * the processor might need it for DMA coherency.
+ */
+#define _PAGE_BASE_NC (_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_PSIZE)
+#define _PAGE_BASE (_PAGE_BASE_NC)
+
+/* Permission masks used to generate the __P and __S table,
+ *
+ * Note:__pgprot is defined in arch/powerpc/include/asm/page.h
+ *
+ * Write permissions imply read permissions for now (we could make write-only
+ * pages on BookE but we don't bother for now). Execute permission control is
+ * possible on platforms that define _PAGE_EXEC
+ *
+ * Note due to the way vm flags are laid out, the bits are XWR
+ */
+#define PAGE_NONE __pgprot(_PAGE_BASE | _PAGE_PRIVILEGED)
+#define PAGE_SHARED __pgprot(_PAGE_BASE | _PAGE_RW)
+#define PAGE_SHARED_X __pgprot(_PAGE_BASE | _PAGE_RW | _PAGE_EXEC)
+#define PAGE_COPY __pgprot(_PAGE_BASE | _PAGE_READ)
+#define PAGE_COPY_X __pgprot(_PAGE_BASE | _PAGE_READ | _PAGE_EXEC)
+#define PAGE_READONLY __pgprot(_PAGE_BASE | _PAGE_READ)
+#define PAGE_READONLY_X __pgprot(_PAGE_BASE | _PAGE_READ | _PAGE_EXEC)
+
+#define __P000 PAGE_NONE
+#define __P001 PAGE_READONLY
+#define __P010 PAGE_COPY
+#define __P011 PAGE_COPY
+#define __P100 PAGE_READONLY_X
+#define __P101 PAGE_READONLY_X
+#define __P110 PAGE_COPY_X
+#define __P111 PAGE_COPY_X
+
+#define __S000 PAGE_NONE
+#define __S001 PAGE_READONLY
+#define __S010 PAGE_SHARED
+#define __S011 PAGE_SHARED
+#define __S100 PAGE_READONLY_X
+#define __S101 PAGE_READONLY_X
+#define __S110 PAGE_SHARED_X
+#define __S111 PAGE_SHARED_X
+
+/* Permission masks used for kernel mappings */
+#define PAGE_KERNEL __pgprot(_PAGE_BASE | _PAGE_KERNEL_RW)
+#define PAGE_KERNEL_NC __pgprot(_PAGE_BASE_NC | _PAGE_KERNEL_RW | \
+ _PAGE_TOLERANT)
+#define PAGE_KERNEL_NCG __pgprot(_PAGE_BASE_NC | _PAGE_KERNEL_RW | \
+ _PAGE_NON_IDEMPOTENT)
+#define PAGE_KERNEL_X __pgprot(_PAGE_BASE | _PAGE_KERNEL_RWX)
+#define PAGE_KERNEL_RO __pgprot(_PAGE_BASE | _PAGE_KERNEL_RO)
+#define PAGE_KERNEL_ROX __pgprot(_PAGE_BASE | _PAGE_KERNEL_ROX)
+
+/*
+ * Protection used for kernel text. We want the debuggers to be able to
+ * set breakpoints anywhere, so don't write protect the kernel text
+ * on platforms where such control is possible.
+ */
+#if defined(CONFIG_KGDB) || defined(CONFIG_XMON) || defined(CONFIG_BDI_SWITCH) || \
+ defined(CONFIG_KPROBES) || defined(CONFIG_DYNAMIC_FTRACE)
+#define PAGE_KERNEL_TEXT PAGE_KERNEL_X
+#else
+#define PAGE_KERNEL_TEXT PAGE_KERNEL_ROX
+#endif
+
+/* Make modules code happy. We don't set RO yet */
+#define PAGE_KERNEL_EXEC PAGE_KERNEL_X
+#define PAGE_AGP (PAGE_KERNEL_NC)
+
+#ifndef __ASSEMBLY__
+/*
+ * page table defines
+ */
+extern unsigned long __pte_index_size;
+extern unsigned long __pmd_index_size;
+extern unsigned long __pud_index_size;
+extern unsigned long __pgd_index_size;
+extern unsigned long __pmd_cache_index;
+#define PTE_INDEX_SIZE __pte_index_size
+#define PMD_INDEX_SIZE __pmd_index_size
+#define PUD_INDEX_SIZE __pud_index_size
+#define PGD_INDEX_SIZE __pgd_index_size
+#define PMD_CACHE_INDEX __pmd_cache_index
+/*
+ * Because of use of pte fragments and THP, size of page table
+ * are not always derived out of index size above.
+ */
+extern unsigned long __pte_table_size;
+extern unsigned long __pmd_table_size;
+extern unsigned long __pud_table_size;
+extern unsigned long __pgd_table_size;
+#define PTE_TABLE_SIZE __pte_table_size
+#define PMD_TABLE_SIZE __pmd_table_size
+#define PUD_TABLE_SIZE __pud_table_size
+#define PGD_TABLE_SIZE __pgd_table_size
+
+extern unsigned long __pmd_val_bits;
+extern unsigned long __pud_val_bits;
+extern unsigned long __pgd_val_bits;
+#define PMD_VAL_BITS __pmd_val_bits
+#define PUD_VAL_BITS __pud_val_bits
+#define PGD_VAL_BITS __pgd_val_bits
+
+extern unsigned long __pte_frag_nr;
+#define PTE_FRAG_NR __pte_frag_nr
+extern unsigned long __pte_frag_size_shift;
+#define PTE_FRAG_SIZE_SHIFT __pte_frag_size_shift
+#define PTE_FRAG_SIZE (1UL << PTE_FRAG_SIZE_SHIFT)
+/*
+ * Pgtable size used by swapper, init in asm code
+ */
+#define MAX_PGD_TABLE_SIZE (sizeof(pgd_t) << RADIX_PGD_INDEX_SIZE)
+
+#define PTRS_PER_PTE (1 << PTE_INDEX_SIZE)
+#define PTRS_PER_PMD (1 << PMD_INDEX_SIZE)
+#define PTRS_PER_PUD (1 << PUD_INDEX_SIZE)
+#define PTRS_PER_PGD (1 << PGD_INDEX_SIZE)
+
+/* PMD_SHIFT determines what a second-level page table entry can map */
+#define PMD_SHIFT (PAGE_SHIFT + PTE_INDEX_SIZE)
+#define PMD_SIZE (1UL << PMD_SHIFT)
+#define PMD_MASK (~(PMD_SIZE-1))
+
+/* PUD_SHIFT determines what a third-level page table entry can map */
+#define PUD_SHIFT (PMD_SHIFT + PMD_INDEX_SIZE)
+#define PUD_SIZE (1UL << PUD_SHIFT)
+#define PUD_MASK (~(PUD_SIZE-1))
+
+/* PGDIR_SHIFT determines what a fourth-level page table entry can map */
+#define PGDIR_SHIFT (PUD_SHIFT + PUD_INDEX_SIZE)
+#define PGDIR_SIZE (1UL << PGDIR_SHIFT)
+#define PGDIR_MASK (~(PGDIR_SIZE-1))
+
+/* Bits to mask out from a PMD to get to the PTE page */
+#define PMD_MASKED_BITS 0xc0000000000000ffUL
+/* Bits to mask out from a PUD to get to the PMD page */
+#define PUD_MASKED_BITS 0xc0000000000000ffUL
+/* Bits to mask out from a PGD to get to the PUD page */
+#define PGD_MASKED_BITS 0xc0000000000000ffUL
+
+extern unsigned long __vmalloc_start;
+extern unsigned long __vmalloc_end;
+#define VMALLOC_START __vmalloc_start
+#define VMALLOC_END __vmalloc_end
+
+extern unsigned long __kernel_virt_start;
+extern unsigned long __kernel_virt_size;
+#define KERN_VIRT_START __kernel_virt_start
+#define KERN_VIRT_SIZE __kernel_virt_size
+extern struct page *vmemmap;
+extern unsigned long ioremap_bot;
+#endif /* __ASSEMBLY__ */
#include <asm/book3s/64/hash.h>
-#include <asm/barrier.h>
+#include <asm/book3s/64/radix.h>
+#ifdef CONFIG_PPC_64K_PAGES
+#include <asm/book3s/64/pgtable-64k.h>
+#else
+#include <asm/book3s/64/pgtable-4k.h>
+#endif
+
+#include <asm/barrier.h>
/*
* The second half of the kernel virtual space is used for IO mappings,
* it's itself carved into the PIO region (ISA and PHB IO space) and
@@ -26,8 +260,6 @@
#define IOREMAP_BASE (PHB_IO_END)
#define IOREMAP_END (KERN_VIRT_START + KERN_VIRT_SIZE)
-#define vmemmap ((struct page *)VMEMMAP_BASE)
-
/* Advertise special mapping type for AGP */
#define HAVE_PAGE_AGP
@@ -45,7 +277,7 @@
#define __real_pte(e,p) ((real_pte_t){(e)})
#define __rpte_to_pte(r) ((r).pte)
-#define __rpte_to_hidx(r,index) (pte_val(__rpte_to_pte(r)) >>_PAGE_F_GIX_SHIFT)
+#define __rpte_to_hidx(r,index) (pte_val(__rpte_to_pte(r)) >> H_PAGE_F_GIX_SHIFT)
#define pte_iterate_hashed_subpages(rpte, psize, va, index, shift) \
do { \
@@ -62,6 +294,327 @@
#endif /* __real_pte */
+static inline unsigned long pte_update(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, unsigned long clr,
+ unsigned long set, int huge)
+{
+ if (radix_enabled())
+ return radix__pte_update(mm, addr, ptep, clr, set, huge);
+ return hash__pte_update(mm, addr, ptep, clr, set, huge);
+}
+/*
+ * For hash even if we have _PAGE_ACCESSED = 0, we do a pte_update.
+ * We currently remove entries from the hashtable regardless of whether
+ * the entry was young or dirty.
+ *
+ * We should be more intelligent about this but for the moment we override
+ * these functions and force a tlb flush unconditionally
+ * For radix: H_PAGE_HASHPTE should be zero. Hence we can use the same
+ * function for both hash and radix.
+ */
+static inline int __ptep_test_and_clear_young(struct mm_struct *mm,
+ unsigned long addr, pte_t *ptep)
+{
+ unsigned long old;
+
+ if ((pte_val(*ptep) & (_PAGE_ACCESSED | H_PAGE_HASHPTE)) == 0)
+ return 0;
+ old = pte_update(mm, addr, ptep, _PAGE_ACCESSED, 0, 0);
+ return (old & _PAGE_ACCESSED) != 0;
+}
+
+#define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG
+#define ptep_test_and_clear_young(__vma, __addr, __ptep) \
+({ \
+ int __r; \
+ __r = __ptep_test_and_clear_young((__vma)->vm_mm, __addr, __ptep); \
+ __r; \
+})
+
+#define __HAVE_ARCH_PTEP_SET_WRPROTECT
+static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep)
+{
+
+ if ((pte_val(*ptep) & _PAGE_WRITE) == 0)
+ return;
+
+ pte_update(mm, addr, ptep, _PAGE_WRITE, 0, 0);
+}
+
+static inline void huge_ptep_set_wrprotect(struct mm_struct *mm,
+ unsigned long addr, pte_t *ptep)
+{
+ if ((pte_val(*ptep) & _PAGE_WRITE) == 0)
+ return;
+
+ pte_update(mm, addr, ptep, _PAGE_WRITE, 0, 1);
+}
+
+#define __HAVE_ARCH_PTEP_GET_AND_CLEAR
+static inline pte_t ptep_get_and_clear(struct mm_struct *mm,
+ unsigned long addr, pte_t *ptep)
+{
+ unsigned long old = pte_update(mm, addr, ptep, ~0UL, 0, 0);
+ return __pte(old);
+}
+
+static inline void pte_clear(struct mm_struct *mm, unsigned long addr,
+ pte_t * ptep)
+{
+ pte_update(mm, addr, ptep, ~0UL, 0, 0);
+}
+static inline int pte_write(pte_t pte) { return !!(pte_val(pte) & _PAGE_WRITE);}
+static inline int pte_dirty(pte_t pte) { return !!(pte_val(pte) & _PAGE_DIRTY); }
+static inline int pte_young(pte_t pte) { return !!(pte_val(pte) & _PAGE_ACCESSED); }
+static inline int pte_special(pte_t pte) { return !!(pte_val(pte) & _PAGE_SPECIAL); }
+static inline pgprot_t pte_pgprot(pte_t pte) { return __pgprot(pte_val(pte) & PAGE_PROT_BITS); }
+
+#ifdef CONFIG_HAVE_ARCH_SOFT_DIRTY
+static inline bool pte_soft_dirty(pte_t pte)
+{
+ return !!(pte_val(pte) & _PAGE_SOFT_DIRTY);
+}
+static inline pte_t pte_mksoft_dirty(pte_t pte)
+{
+ return __pte(pte_val(pte) | _PAGE_SOFT_DIRTY);
+}
+
+static inline pte_t pte_clear_soft_dirty(pte_t pte)
+{
+ return __pte(pte_val(pte) & ~_PAGE_SOFT_DIRTY);
+}
+#endif /* CONFIG_HAVE_ARCH_SOFT_DIRTY */
+
+#ifdef CONFIG_NUMA_BALANCING
+/*
+ * These work without NUMA balancing but the kernel does not care. See the
+ * comment in include/asm-generic/pgtable.h . On powerpc, this will only
+ * work for user pages and always return true for kernel pages.
+ */
+static inline int pte_protnone(pte_t pte)
+{
+ return (pte_val(pte) & (_PAGE_PRESENT | _PAGE_PRIVILEGED)) ==
+ (_PAGE_PRESENT | _PAGE_PRIVILEGED);
+}
+#endif /* CONFIG_NUMA_BALANCING */
+
+static inline int pte_present(pte_t pte)
+{
+ return !!(pte_val(pte) & _PAGE_PRESENT);
+}
+/*
+ * Conversion functions: convert a page and protection to a page entry,
+ * and a page entry and page directory to the page they refer to.
+ *
+ * Even if PTEs can be unsigned long long, a PFN is always an unsigned
+ * long for now.
+ */
+static inline pte_t pfn_pte(unsigned long pfn, pgprot_t pgprot)
+{
+ return __pte((((pte_basic_t)(pfn) << PAGE_SHIFT) & PTE_RPN_MASK) |
+ pgprot_val(pgprot));
+}
+
+static inline unsigned long pte_pfn(pte_t pte)
+{
+ return (pte_val(pte) & PTE_RPN_MASK) >> PAGE_SHIFT;
+}
+
+/* Generic modifiers for PTE bits */
+static inline pte_t pte_wrprotect(pte_t pte)
+{
+ return __pte(pte_val(pte) & ~_PAGE_WRITE);
+}
+
+static inline pte_t pte_mkclean(pte_t pte)
+{
+ return __pte(pte_val(pte) & ~_PAGE_DIRTY);
+}
+
+static inline pte_t pte_mkold(pte_t pte)
+{
+ return __pte(pte_val(pte) & ~_PAGE_ACCESSED);
+}
+
+static inline pte_t pte_mkwrite(pte_t pte)
+{
+ /*
+ * write implies read, hence set both
+ */
+ return __pte(pte_val(pte) | _PAGE_RW);
+}
+
+static inline pte_t pte_mkdirty(pte_t pte)
+{
+ return __pte(pte_val(pte) | _PAGE_DIRTY | _PAGE_SOFT_DIRTY);
+}
+
+static inline pte_t pte_mkyoung(pte_t pte)
+{
+ return __pte(pte_val(pte) | _PAGE_ACCESSED);
+}
+
+static inline pte_t pte_mkspecial(pte_t pte)
+{
+ return __pte(pte_val(pte) | _PAGE_SPECIAL);
+}
+
+static inline pte_t pte_mkhuge(pte_t pte)
+{
+ return pte;
+}
+
+static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
+{
+ /* FIXME!! check whether this need to be a conditional */
+ return __pte((pte_val(pte) & _PAGE_CHG_MASK) | pgprot_val(newprot));
+}
+
+static inline bool pte_user(pte_t pte)
+{
+ return !(pte_val(pte) & _PAGE_PRIVILEGED);
+}
+
+/* Encode and de-code a swap entry */
+#define MAX_SWAPFILES_CHECK() do { \
+ BUILD_BUG_ON(MAX_SWAPFILES_SHIFT > SWP_TYPE_BITS); \
+ /* \
+ * Don't have overlapping bits with _PAGE_HPTEFLAGS \
+ * We filter HPTEFLAGS on set_pte. \
+ */ \
+ BUILD_BUG_ON(_PAGE_HPTEFLAGS & (0x1f << _PAGE_BIT_SWAP_TYPE)); \
+ BUILD_BUG_ON(_PAGE_HPTEFLAGS & _PAGE_SWP_SOFT_DIRTY); \
+ } while (0)
+/*
+ * on pte we don't need handle RADIX_TREE_EXCEPTIONAL_SHIFT;
+ */
+#define SWP_TYPE_BITS 5
+#define __swp_type(x) (((x).val >> _PAGE_BIT_SWAP_TYPE) \
+ & ((1UL << SWP_TYPE_BITS) - 1))
+#define __swp_offset(x) (((x).val & PTE_RPN_MASK) >> PAGE_SHIFT)
+#define __swp_entry(type, offset) ((swp_entry_t) { \
+ ((type) << _PAGE_BIT_SWAP_TYPE) \
+ | (((offset) << PAGE_SHIFT) & PTE_RPN_MASK)})
+/*
+ * swp_entry_t must be independent of pte bits. We build a swp_entry_t from
+ * swap type and offset we get from swap and convert that to pte to find a
+ * matching pte in linux page table.
+ * Clear bits not found in swap entries here.
+ */
+#define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val((pte)) & ~_PAGE_PTE })
+#define __swp_entry_to_pte(x) __pte((x).val | _PAGE_PTE)
+
+#ifdef CONFIG_MEM_SOFT_DIRTY
+#define _PAGE_SWP_SOFT_DIRTY (1UL << (SWP_TYPE_BITS + _PAGE_BIT_SWAP_TYPE))
+#else
+#define _PAGE_SWP_SOFT_DIRTY 0UL
+#endif /* CONFIG_MEM_SOFT_DIRTY */
+
+#ifdef CONFIG_HAVE_ARCH_SOFT_DIRTY
+static inline pte_t pte_swp_mksoft_dirty(pte_t pte)
+{
+ return __pte(pte_val(pte) | _PAGE_SWP_SOFT_DIRTY);
+}
+static inline bool pte_swp_soft_dirty(pte_t pte)
+{
+ return !!(pte_val(pte) & _PAGE_SWP_SOFT_DIRTY);
+}
+static inline pte_t pte_swp_clear_soft_dirty(pte_t pte)
+{
+ return __pte(pte_val(pte) & ~_PAGE_SWP_SOFT_DIRTY);
+}
+#endif /* CONFIG_HAVE_ARCH_SOFT_DIRTY */
+
+static inline bool check_pte_access(unsigned long access, unsigned long ptev)
+{
+ /*
+ * This check for _PAGE_RWX and _PAGE_PRESENT bits
+ */
+ if (access & ~ptev)
+ return false;
+ /*
+ * This check for access to privilege space
+ */
+ if ((access & _PAGE_PRIVILEGED) != (ptev & _PAGE_PRIVILEGED))
+ return false;
+
+ return true;
+}
+/*
+ * Generic functions with hash/radix callbacks
+ */
+
+static inline void __ptep_set_access_flags(pte_t *ptep, pte_t entry)
+{
+ if (radix_enabled())
+ return radix__ptep_set_access_flags(ptep, entry);
+ return hash__ptep_set_access_flags(ptep, entry);
+}
+
+#define __HAVE_ARCH_PTE_SAME
+static inline int pte_same(pte_t pte_a, pte_t pte_b)
+{
+ if (radix_enabled())
+ return radix__pte_same(pte_a, pte_b);
+ return hash__pte_same(pte_a, pte_b);
+}
+
+static inline int pte_none(pte_t pte)
+{
+ if (radix_enabled())
+ return radix__pte_none(pte);
+ return hash__pte_none(pte);
+}
+
+static inline void __set_pte_at(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, pte_t pte, int percpu)
+{
+ if (radix_enabled())
+ return radix__set_pte_at(mm, addr, ptep, pte, percpu);
+ return hash__set_pte_at(mm, addr, ptep, pte, percpu);
+}
+
+#define _PAGE_CACHE_CTL (_PAGE_NON_IDEMPOTENT | _PAGE_TOLERANT)
+
+#define pgprot_noncached pgprot_noncached
+static inline pgprot_t pgprot_noncached(pgprot_t prot)
+{
+ return __pgprot((pgprot_val(prot) & ~_PAGE_CACHE_CTL) |
+ _PAGE_NON_IDEMPOTENT);
+}
+
+#define pgprot_noncached_wc pgprot_noncached_wc
+static inline pgprot_t pgprot_noncached_wc(pgprot_t prot)
+{
+ return __pgprot((pgprot_val(prot) & ~_PAGE_CACHE_CTL) |
+ _PAGE_TOLERANT);
+}
+
+#define pgprot_cached pgprot_cached
+static inline pgprot_t pgprot_cached(pgprot_t prot)
+{
+ return __pgprot((pgprot_val(prot) & ~_PAGE_CACHE_CTL));
+}
+
+#define pgprot_writecombine pgprot_writecombine
+static inline pgprot_t pgprot_writecombine(pgprot_t prot)
+{
+ return pgprot_noncached_wc(prot);
+}
+/*
+ * check a pte mapping have cache inhibited property
+ */
+static inline bool pte_ci(pte_t pte)
+{
+ unsigned long pte_v = pte_val(pte);
+
+ if (((pte_v & _PAGE_CACHE_CTL) == _PAGE_TOLERANT) ||
+ ((pte_v & _PAGE_CACHE_CTL) == _PAGE_NON_IDEMPOTENT))
+ return true;
+ return false;
+}
+
static inline void pmd_set(pmd_t *pmdp, unsigned long val)
{
*pmdp = __pmd(val);
@@ -75,6 +628,13 @@
#define pmd_none(pmd) (!pmd_val(pmd))
#define pmd_present(pmd) (!pmd_none(pmd))
+static inline int pmd_bad(pmd_t pmd)
+{
+ if (radix_enabled())
+ return radix__pmd_bad(pmd);
+ return hash__pmd_bad(pmd);
+}
+
static inline void pud_set(pud_t *pudp, unsigned long val)
{
*pudp = __pud(val);
@@ -100,6 +660,15 @@
return __pud(pte_val(pte));
}
#define pud_write(pud) pte_write(pud_pte(pud))
+
+static inline int pud_bad(pud_t pud)
+{
+ if (radix_enabled())
+ return radix__pud_bad(pud);
+ return hash__pud_bad(pud);
+}
+
+
#define pgd_write(pgd) pte_write(pgd_pte(pgd))
static inline void pgd_set(pgd_t *pgdp, unsigned long val)
{
@@ -124,8 +693,27 @@
return __pgd(pte_val(pte));
}
+static inline int pgd_bad(pgd_t pgd)
+{
+ if (radix_enabled())
+ return radix__pgd_bad(pgd);
+ return hash__pgd_bad(pgd);
+}
+
extern struct page *pgd_page(pgd_t pgd);
+/* Pointers in the page table tree are physical addresses */
+#define __pgtable_ptr_val(ptr) __pa(ptr)
+
+#define pmd_page_vaddr(pmd) __va(pmd_val(pmd) & ~PMD_MASKED_BITS)
+#define pud_page_vaddr(pud) __va(pud_val(pud) & ~PUD_MASKED_BITS)
+#define pgd_page_vaddr(pgd) __va(pgd_val(pgd) & ~PGD_MASKED_BITS)
+
+#define pgd_index(address) (((address) >> (PGDIR_SHIFT)) & (PTRS_PER_PGD - 1))
+#define pud_index(address) (((address) >> (PUD_SHIFT)) & (PTRS_PER_PUD - 1))
+#define pmd_index(address) (((address) >> (PMD_SHIFT)) & (PTRS_PER_PMD - 1))
+#define pte_index(address) (((address) >> (PAGE_SHIFT)) & (PTRS_PER_PTE - 1))
+
/*
* Find an entry in a page-table-directory. We combine the address region
* (the high order N bits) and the pgd portion of the address.
@@ -156,73 +744,42 @@
#define pgd_ERROR(e) \
pr_err("%s:%d: bad pgd %08lx.\n", __FILE__, __LINE__, pgd_val(e))
-/* Encode and de-code a swap entry */
-#define MAX_SWAPFILES_CHECK() do { \
- BUILD_BUG_ON(MAX_SWAPFILES_SHIFT > SWP_TYPE_BITS); \
- /* \
- * Don't have overlapping bits with _PAGE_HPTEFLAGS \
- * We filter HPTEFLAGS on set_pte. \
- */ \
- BUILD_BUG_ON(_PAGE_HPTEFLAGS & (0x1f << _PAGE_BIT_SWAP_TYPE)); \
- BUILD_BUG_ON(_PAGE_HPTEFLAGS & _PAGE_SWP_SOFT_DIRTY); \
- } while (0)
-/*
- * on pte we don't need handle RADIX_TREE_EXCEPTIONAL_SHIFT;
- */
-#define SWP_TYPE_BITS 5
-#define __swp_type(x) (((x).val >> _PAGE_BIT_SWAP_TYPE) \
- & ((1UL << SWP_TYPE_BITS) - 1))
-#define __swp_offset(x) (((x).val & PTE_RPN_MASK) >> PTE_RPN_SHIFT)
-#define __swp_entry(type, offset) ((swp_entry_t) { \
- ((type) << _PAGE_BIT_SWAP_TYPE) \
- | (((offset) << PTE_RPN_SHIFT) & PTE_RPN_MASK)})
-/*
- * swp_entry_t must be independent of pte bits. We build a swp_entry_t from
- * swap type and offset we get from swap and convert that to pte to find a
- * matching pte in linux page table.
- * Clear bits not found in swap entries here.
- */
-#define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val((pte)) & ~_PAGE_PTE })
-#define __swp_entry_to_pte(x) __pte((x).val | _PAGE_PTE)
-
-#ifdef CONFIG_MEM_SOFT_DIRTY
-#define _PAGE_SWP_SOFT_DIRTY (1UL << (SWP_TYPE_BITS + _PAGE_BIT_SWAP_TYPE))
-#else
-#define _PAGE_SWP_SOFT_DIRTY 0UL
-#endif /* CONFIG_MEM_SOFT_DIRTY */
-
-#ifdef CONFIG_HAVE_ARCH_SOFT_DIRTY
-static inline pte_t pte_swp_mksoft_dirty(pte_t pte)
-{
- return __pte(pte_val(pte) | _PAGE_SWP_SOFT_DIRTY);
-}
-static inline bool pte_swp_soft_dirty(pte_t pte)
-{
- return !!(pte_val(pte) & _PAGE_SWP_SOFT_DIRTY);
-}
-static inline pte_t pte_swp_clear_soft_dirty(pte_t pte)
-{
- return __pte(pte_val(pte) & ~_PAGE_SWP_SOFT_DIRTY);
-}
-#endif /* CONFIG_HAVE_ARCH_SOFT_DIRTY */
-
void pgtable_cache_add(unsigned shift, void (*ctor)(void *));
void pgtable_cache_init(void);
+static inline int map_kernel_page(unsigned long ea, unsigned long pa,
+ unsigned long flags)
+{
+ if (radix_enabled()) {
+#if defined(CONFIG_PPC_RADIX_MMU) && defined(DEBUG_VM)
+ unsigned long page_size = 1 << mmu_psize_defs[mmu_io_psize].shift;
+ WARN((page_size != PAGE_SIZE), "I/O page size != PAGE_SIZE");
+#endif
+ return radix__map_kernel_page(ea, pa, __pgprot(flags), PAGE_SIZE);
+ }
+ return hash__map_kernel_page(ea, pa, flags);
+}
+
+static inline int __meminit vmemmap_create_mapping(unsigned long start,
+ unsigned long page_size,
+ unsigned long phys)
+{
+ if (radix_enabled())
+ return radix__vmemmap_create_mapping(start, page_size, phys);
+ return hash__vmemmap_create_mapping(start, page_size, phys);
+}
+
+#ifdef CONFIG_MEMORY_HOTPLUG
+static inline void vmemmap_remove_mapping(unsigned long start,
+ unsigned long page_size)
+{
+ if (radix_enabled())
+ return radix__vmemmap_remove_mapping(start, page_size);
+ return hash__vmemmap_remove_mapping(start, page_size);
+}
+#endif
struct page *realmode_pfn_to_page(unsigned long pfn);
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-extern pmd_t pfn_pmd(unsigned long pfn, pgprot_t pgprot);
-extern pmd_t mk_pmd(struct page *page, pgprot_t pgprot);
-extern pmd_t pmd_modify(pmd_t pmd, pgprot_t newprot);
-extern void set_pmd_at(struct mm_struct *mm, unsigned long addr,
- pmd_t *pmdp, pmd_t pmd);
-extern void update_mmu_cache_pmd(struct vm_area_struct *vma, unsigned long addr,
- pmd_t *pmd);
-extern int has_transparent_hugepage(void);
-#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
-
-
static inline pte_t pmd_pte(pmd_t pmd)
{
return __pte(pmd_val(pmd));
@@ -237,7 +794,6 @@
{
return (pte_t *)pmd;
}
-
#define pmd_pfn(pmd) pte_pfn(pmd_pte(pmd))
#define pmd_dirty(pmd) pte_dirty(pmd_pte(pmd))
#define pmd_young(pmd) pte_young(pmd_pte(pmd))
@@ -264,9 +820,87 @@
#define __HAVE_ARCH_PMD_WRITE
#define pmd_write(pmd) pte_write(pmd_pte(pmd))
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+extern pmd_t pfn_pmd(unsigned long pfn, pgprot_t pgprot);
+extern pmd_t mk_pmd(struct page *page, pgprot_t pgprot);
+extern pmd_t pmd_modify(pmd_t pmd, pgprot_t newprot);
+extern void set_pmd_at(struct mm_struct *mm, unsigned long addr,
+ pmd_t *pmdp, pmd_t pmd);
+extern void update_mmu_cache_pmd(struct vm_area_struct *vma, unsigned long addr,
+ pmd_t *pmd);
+extern int hash__has_transparent_hugepage(void);
+static inline int has_transparent_hugepage(void)
+{
+ if (radix_enabled())
+ return radix__has_transparent_hugepage();
+ return hash__has_transparent_hugepage();
+}
+#define has_transparent_hugepage has_transparent_hugepage
+
+static inline unsigned long
+pmd_hugepage_update(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp,
+ unsigned long clr, unsigned long set)
+{
+ if (radix_enabled())
+ return radix__pmd_hugepage_update(mm, addr, pmdp, clr, set);
+ return hash__pmd_hugepage_update(mm, addr, pmdp, clr, set);
+}
+
+static inline int pmd_large(pmd_t pmd)
+{
+ return !!(pmd_val(pmd) & _PAGE_PTE);
+}
+
+static inline pmd_t pmd_mknotpresent(pmd_t pmd)
+{
+ return __pmd(pmd_val(pmd) & ~_PAGE_PRESENT);
+}
+/*
+ * For radix we should always find H_PAGE_HASHPTE zero. Hence
+ * the below will work for radix too
+ */
+static inline int __pmdp_test_and_clear_young(struct mm_struct *mm,
+ unsigned long addr, pmd_t *pmdp)
+{
+ unsigned long old;
+
+ if ((pmd_val(*pmdp) & (_PAGE_ACCESSED | H_PAGE_HASHPTE)) == 0)
+ return 0;
+ old = pmd_hugepage_update(mm, addr, pmdp, _PAGE_ACCESSED, 0);
+ return ((old & _PAGE_ACCESSED) != 0);
+}
+
+#define __HAVE_ARCH_PMDP_SET_WRPROTECT
+static inline void pmdp_set_wrprotect(struct mm_struct *mm, unsigned long addr,
+ pmd_t *pmdp)
+{
+
+ if ((pmd_val(*pmdp) & _PAGE_WRITE) == 0)
+ return;
+
+ pmd_hugepage_update(mm, addr, pmdp, _PAGE_WRITE, 0);
+}
+
+static inline int pmd_trans_huge(pmd_t pmd)
+{
+ if (radix_enabled())
+ return radix__pmd_trans_huge(pmd);
+ return hash__pmd_trans_huge(pmd);
+}
+
+#define __HAVE_ARCH_PMD_SAME
+static inline int pmd_same(pmd_t pmd_a, pmd_t pmd_b)
+{
+ if (radix_enabled())
+ return radix__pmd_same(pmd_a, pmd_b);
+ return hash__pmd_same(pmd_a, pmd_b);
+}
+
static inline pmd_t pmd_mkhuge(pmd_t pmd)
{
- return __pmd(pmd_val(pmd) | (_PAGE_PTE | _PAGE_THP_HUGE));
+ if (radix_enabled())
+ return radix__pmd_mkhuge(pmd);
+ return hash__pmd_mkhuge(pmd);
}
#define __HAVE_ARCH_PMDP_SET_ACCESS_FLAGS
@@ -277,37 +911,63 @@
#define __HAVE_ARCH_PMDP_TEST_AND_CLEAR_YOUNG
extern int pmdp_test_and_clear_young(struct vm_area_struct *vma,
unsigned long address, pmd_t *pmdp);
-#define __HAVE_ARCH_PMDP_CLEAR_YOUNG_FLUSH
-extern int pmdp_clear_flush_young(struct vm_area_struct *vma,
- unsigned long address, pmd_t *pmdp);
#define __HAVE_ARCH_PMDP_HUGE_GET_AND_CLEAR
-extern pmd_t pmdp_huge_get_and_clear(struct mm_struct *mm,
- unsigned long addr, pmd_t *pmdp);
+static inline pmd_t pmdp_huge_get_and_clear(struct mm_struct *mm,
+ unsigned long addr, pmd_t *pmdp)
+{
+ if (radix_enabled())
+ return radix__pmdp_huge_get_and_clear(mm, addr, pmdp);
+ return hash__pmdp_huge_get_and_clear(mm, addr, pmdp);
+}
-extern pmd_t pmdp_collapse_flush(struct vm_area_struct *vma,
- unsigned long address, pmd_t *pmdp);
+static inline pmd_t pmdp_collapse_flush(struct vm_area_struct *vma,
+ unsigned long address, pmd_t *pmdp)
+{
+ if (radix_enabled())
+ return radix__pmdp_collapse_flush(vma, address, pmdp);
+ return hash__pmdp_collapse_flush(vma, address, pmdp);
+}
#define pmdp_collapse_flush pmdp_collapse_flush
#define __HAVE_ARCH_PGTABLE_DEPOSIT
-extern void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,
- pgtable_t pgtable);
+static inline void pgtable_trans_huge_deposit(struct mm_struct *mm,
+ pmd_t *pmdp, pgtable_t pgtable)
+{
+ if (radix_enabled())
+ return radix__pgtable_trans_huge_deposit(mm, pmdp, pgtable);
+ return hash__pgtable_trans_huge_deposit(mm, pmdp, pgtable);
+}
+
#define __HAVE_ARCH_PGTABLE_WITHDRAW
-extern pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp);
+static inline pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm,
+ pmd_t *pmdp)
+{
+ if (radix_enabled())
+ return radix__pgtable_trans_huge_withdraw(mm, pmdp);
+ return hash__pgtable_trans_huge_withdraw(mm, pmdp);
+}
#define __HAVE_ARCH_PMDP_INVALIDATE
extern void pmdp_invalidate(struct vm_area_struct *vma, unsigned long address,
pmd_t *pmdp);
#define __HAVE_ARCH_PMDP_HUGE_SPLIT_PREPARE
-extern void pmdp_huge_split_prepare(struct vm_area_struct *vma,
- unsigned long address, pmd_t *pmdp);
+static inline void pmdp_huge_split_prepare(struct vm_area_struct *vma,
+ unsigned long address, pmd_t *pmdp)
+{
+ if (radix_enabled())
+ return radix__pmdp_huge_split_prepare(vma, address, pmdp);
+ return hash__pmdp_huge_split_prepare(vma, address, pmdp);
+}
#define pmd_move_must_withdraw pmd_move_must_withdraw
struct spinlock;
static inline int pmd_move_must_withdraw(struct spinlock *new_pmd_ptl,
struct spinlock *old_pmd_ptl)
{
+ if (radix_enabled())
+ return false;
/*
* Archs like ppc64 use pgtable to store per pmd
* specific information. So when we switch the pmd,
@@ -315,5 +975,6 @@
*/
return true;
}
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
#endif /* __ASSEMBLY__ */
#endif /* _ASM_POWERPC_BOOK3S_64_PGTABLE_H_ */
diff --git a/arch/powerpc/include/asm/book3s/64/radix-4k.h b/arch/powerpc/include/asm/book3s/64/radix-4k.h
new file mode 100644
index 0000000..7c3b1fe
--- /dev/null
+++ b/arch/powerpc/include/asm/book3s/64/radix-4k.h
@@ -0,0 +1,12 @@
+#ifndef _ASM_POWERPC_PGTABLE_RADIX_4K_H
+#define _ASM_POWERPC_PGTABLE_RADIX_4K_H
+
+/*
+ * For 4K page size supported index is 13/9/9/9
+ */
+#define RADIX_PTE_INDEX_SIZE 9 /* 2MB huge page */
+#define RADIX_PMD_INDEX_SIZE 9 /* 1G huge page */
+#define RADIX_PUD_INDEX_SIZE 9
+#define RADIX_PGD_INDEX_SIZE 13
+
+#endif /* _ASM_POWERPC_PGTABLE_RADIX_4K_H */
diff --git a/arch/powerpc/include/asm/book3s/64/radix-64k.h b/arch/powerpc/include/asm/book3s/64/radix-64k.h
new file mode 100644
index 0000000..82dc355
--- /dev/null
+++ b/arch/powerpc/include/asm/book3s/64/radix-64k.h
@@ -0,0 +1,12 @@
+#ifndef _ASM_POWERPC_PGTABLE_RADIX_64K_H
+#define _ASM_POWERPC_PGTABLE_RADIX_64K_H
+
+/*
+ * For 64K page size supported index is 13/9/9/5
+ */
+#define RADIX_PTE_INDEX_SIZE 5 /* 2MB huge page */
+#define RADIX_PMD_INDEX_SIZE 9 /* 1G huge page */
+#define RADIX_PUD_INDEX_SIZE 9
+#define RADIX_PGD_INDEX_SIZE 13
+
+#endif /* _ASM_POWERPC_PGTABLE_RADIX_64K_H */
diff --git a/arch/powerpc/include/asm/book3s/64/radix.h b/arch/powerpc/include/asm/book3s/64/radix.h
new file mode 100644
index 0000000..937d4e2
--- /dev/null
+++ b/arch/powerpc/include/asm/book3s/64/radix.h
@@ -0,0 +1,232 @@
+#ifndef _ASM_POWERPC_PGTABLE_RADIX_H
+#define _ASM_POWERPC_PGTABLE_RADIX_H
+
+#ifndef __ASSEMBLY__
+#include <asm/cmpxchg.h>
+#endif
+
+#ifdef CONFIG_PPC_64K_PAGES
+#include <asm/book3s/64/radix-64k.h>
+#else
+#include <asm/book3s/64/radix-4k.h>
+#endif
+
+/* An empty PTE can still have a R or C writeback */
+#define RADIX_PTE_NONE_MASK (_PAGE_DIRTY | _PAGE_ACCESSED)
+
+/* Bits to set in a RPMD/RPUD/RPGD */
+#define RADIX_PMD_VAL_BITS (0x8000000000000000UL | RADIX_PTE_INDEX_SIZE)
+#define RADIX_PUD_VAL_BITS (0x8000000000000000UL | RADIX_PMD_INDEX_SIZE)
+#define RADIX_PGD_VAL_BITS (0x8000000000000000UL | RADIX_PUD_INDEX_SIZE)
+
+/* Don't have anything in the reserved bits and leaf bits */
+#define RADIX_PMD_BAD_BITS 0x60000000000000e0UL
+#define RADIX_PUD_BAD_BITS 0x60000000000000e0UL
+#define RADIX_PGD_BAD_BITS 0x60000000000000e0UL
+
+/*
+ * Size of EA range mapped by our pagetables.
+ */
+#define RADIX_PGTABLE_EADDR_SIZE (RADIX_PTE_INDEX_SIZE + RADIX_PMD_INDEX_SIZE + \
+ RADIX_PUD_INDEX_SIZE + RADIX_PGD_INDEX_SIZE + PAGE_SHIFT)
+#define RADIX_PGTABLE_RANGE (ASM_CONST(1) << RADIX_PGTABLE_EADDR_SIZE)
+
+/*
+ * We support 52 bit address space, Use top bit for kernel
+ * virtual mapping. Also make sure kernel fit in the top
+ * quadrant.
+ *
+ * +------------------+
+ * +------------------+ Kernel virtual map (0xc008000000000000)
+ * | |
+ * | |
+ * | |
+ * 0b11......+------------------+ Kernel linear map (0xc....)
+ * | |
+ * | 2 quadrant |
+ * | |
+ * 0b10......+------------------+
+ * | |
+ * | 1 quadrant |
+ * | |
+ * 0b01......+------------------+
+ * | |
+ * | 0 quadrant |
+ * | |
+ * 0b00......+------------------+
+ *
+ *
+ * 3rd quadrant expanded:
+ * +------------------------------+
+ * | |
+ * | |
+ * | |
+ * +------------------------------+ Kernel IO map end (0xc010000000000000)
+ * | |
+ * | |
+ * | 1/2 of virtual map |
+ * | |
+ * | |
+ * +------------------------------+ Kernel IO map start
+ * | |
+ * | 1/4 of virtual map |
+ * | |
+ * +------------------------------+ Kernel vmemap start
+ * | |
+ * | 1/4 of virtual map |
+ * | |
+ * +------------------------------+ Kernel virt start (0xc008000000000000)
+ * | |
+ * | |
+ * | |
+ * +------------------------------+ Kernel linear (0xc.....)
+ */
+
+#define RADIX_KERN_VIRT_START ASM_CONST(0xc008000000000000)
+#define RADIX_KERN_VIRT_SIZE ASM_CONST(0x0008000000000000)
+
+/*
+ * The vmalloc space starts at the beginning of that region, and
+ * occupies a quarter of it on radix config.
+ * (we keep a quarter for the virtual memmap)
+ */
+#define RADIX_VMALLOC_START RADIX_KERN_VIRT_START
+#define RADIX_VMALLOC_SIZE (RADIX_KERN_VIRT_SIZE >> 2)
+#define RADIX_VMALLOC_END (RADIX_VMALLOC_START + RADIX_VMALLOC_SIZE)
+/*
+ * Defines the address of the vmemap area, in its own region on
+ * hash table CPUs.
+ */
+#define RADIX_VMEMMAP_BASE (RADIX_VMALLOC_END)
+
+#ifndef __ASSEMBLY__
+#define RADIX_PTE_TABLE_SIZE (sizeof(pte_t) << RADIX_PTE_INDEX_SIZE)
+#define RADIX_PMD_TABLE_SIZE (sizeof(pmd_t) << RADIX_PMD_INDEX_SIZE)
+#define RADIX_PUD_TABLE_SIZE (sizeof(pud_t) << RADIX_PUD_INDEX_SIZE)
+#define RADIX_PGD_TABLE_SIZE (sizeof(pgd_t) << RADIX_PGD_INDEX_SIZE)
+
+static inline unsigned long radix__pte_update(struct mm_struct *mm,
+ unsigned long addr,
+ pte_t *ptep, unsigned long clr,
+ unsigned long set,
+ int huge)
+{
+ pte_t pte;
+ unsigned long old_pte, new_pte;
+
+ do {
+ pte = READ_ONCE(*ptep);
+ old_pte = pte_val(pte);
+ new_pte = (old_pte | set) & ~clr;
+
+ } while (!pte_xchg(ptep, __pte(old_pte), __pte(new_pte)));
+
+ /* We already do a sync in cmpxchg, is ptesync needed ?*/
+ asm volatile("ptesync" : : : "memory");
+ /* huge pages use the old page table lock */
+ if (!huge)
+ assert_pte_locked(mm, addr);
+
+ return old_pte;
+}
+
+/*
+ * Set the dirty and/or accessed bits atomically in a linux PTE, this
+ * function doesn't need to invalidate tlb.
+ */
+static inline void radix__ptep_set_access_flags(pte_t *ptep, pte_t entry)
+{
+ pte_t pte;
+ unsigned long old_pte, new_pte;
+ unsigned long set = pte_val(entry) & (_PAGE_DIRTY | _PAGE_ACCESSED |
+ _PAGE_RW | _PAGE_EXEC);
+ do {
+ pte = READ_ONCE(*ptep);
+ old_pte = pte_val(pte);
+ new_pte = old_pte | set;
+
+ } while (!pte_xchg(ptep, __pte(old_pte), __pte(new_pte)));
+
+ /* We already do a sync in cmpxchg, is ptesync needed ?*/
+ asm volatile("ptesync" : : : "memory");
+}
+
+static inline int radix__pte_same(pte_t pte_a, pte_t pte_b)
+{
+ return ((pte_raw(pte_a) ^ pte_raw(pte_b)) == 0);
+}
+
+static inline int radix__pte_none(pte_t pte)
+{
+ return (pte_val(pte) & ~RADIX_PTE_NONE_MASK) == 0;
+}
+
+static inline void radix__set_pte_at(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, pte_t pte, int percpu)
+{
+ *ptep = pte;
+ asm volatile("ptesync" : : : "memory");
+}
+
+static inline int radix__pmd_bad(pmd_t pmd)
+{
+ return !!(pmd_val(pmd) & RADIX_PMD_BAD_BITS);
+}
+
+static inline int radix__pmd_same(pmd_t pmd_a, pmd_t pmd_b)
+{
+ return ((pmd_raw(pmd_a) ^ pmd_raw(pmd_b)) == 0);
+}
+
+static inline int radix__pud_bad(pud_t pud)
+{
+ return !!(pud_val(pud) & RADIX_PUD_BAD_BITS);
+}
+
+
+static inline int radix__pgd_bad(pgd_t pgd)
+{
+ return !!(pgd_val(pgd) & RADIX_PGD_BAD_BITS);
+}
+
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+
+static inline int radix__pmd_trans_huge(pmd_t pmd)
+{
+ return !!(pmd_val(pmd) & _PAGE_PTE);
+}
+
+static inline pmd_t radix__pmd_mkhuge(pmd_t pmd)
+{
+ return __pmd(pmd_val(pmd) | _PAGE_PTE);
+}
+static inline void radix__pmdp_huge_split_prepare(struct vm_area_struct *vma,
+ unsigned long address, pmd_t *pmdp)
+{
+ /* Nothing to do for radix. */
+ return;
+}
+
+extern unsigned long radix__pmd_hugepage_update(struct mm_struct *mm, unsigned long addr,
+ pmd_t *pmdp, unsigned long clr,
+ unsigned long set);
+extern pmd_t radix__pmdp_collapse_flush(struct vm_area_struct *vma,
+ unsigned long address, pmd_t *pmdp);
+extern void radix__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,
+ pgtable_t pgtable);
+extern pgtable_t radix__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp);
+extern pmd_t radix__pmdp_huge_get_and_clear(struct mm_struct *mm,
+ unsigned long addr, pmd_t *pmdp);
+extern int radix__has_transparent_hugepage(void);
+#endif
+
+extern int __meminit radix__vmemmap_create_mapping(unsigned long start,
+ unsigned long page_size,
+ unsigned long phys);
+extern void radix__vmemmap_remove_mapping(unsigned long start,
+ unsigned long page_size);
+
+extern int radix__map_kernel_page(unsigned long ea, unsigned long pa,
+ pgprot_t flags, unsigned int psz);
+#endif /* __ASSEMBLY__ */
+#endif
diff --git a/arch/powerpc/include/asm/book3s/64/tlbflush-hash.h b/arch/powerpc/include/asm/book3s/64/tlbflush-hash.h
index 1b753f9..f12ddf5 100644
--- a/arch/powerpc/include/asm/book3s/64/tlbflush-hash.h
+++ b/arch/powerpc/include/asm/book3s/64/tlbflush-hash.h
@@ -1,8 +1,6 @@
#ifndef _ASM_POWERPC_BOOK3S_64_TLBFLUSH_HASH_H
#define _ASM_POWERPC_BOOK3S_64_TLBFLUSH_HASH_H
-#define MMU_NO_CONTEXT 0
-
/*
* TLB flushing for 64-bit hash-MMU CPUs
*/
@@ -29,14 +27,21 @@
static inline void arch_enter_lazy_mmu_mode(void)
{
- struct ppc64_tlb_batch *batch = this_cpu_ptr(&ppc64_tlb_batch);
+ struct ppc64_tlb_batch *batch;
+ if (radix_enabled())
+ return;
+ batch = this_cpu_ptr(&ppc64_tlb_batch);
batch->active = 1;
}
static inline void arch_leave_lazy_mmu_mode(void)
{
- struct ppc64_tlb_batch *batch = this_cpu_ptr(&ppc64_tlb_batch);
+ struct ppc64_tlb_batch *batch;
+
+ if (radix_enabled())
+ return;
+ batch = this_cpu_ptr(&ppc64_tlb_batch);
if (batch->index)
__flush_tlb_pending(batch);
@@ -52,40 +57,42 @@
extern void flush_hash_hugepage(unsigned long vsid, unsigned long addr,
pmd_t *pmdp, unsigned int psize, int ssize,
unsigned long flags);
-
-static inline void local_flush_tlb_mm(struct mm_struct *mm)
+static inline void hash__local_flush_tlb_mm(struct mm_struct *mm)
{
}
-static inline void flush_tlb_mm(struct mm_struct *mm)
+static inline void hash__flush_tlb_mm(struct mm_struct *mm)
{
}
-static inline void local_flush_tlb_page(struct vm_area_struct *vma,
- unsigned long vmaddr)
+static inline void hash__local_flush_tlb_page(struct vm_area_struct *vma,
+ unsigned long vmaddr)
{
}
-static inline void flush_tlb_page(struct vm_area_struct *vma,
- unsigned long vmaddr)
+static inline void hash__flush_tlb_page(struct vm_area_struct *vma,
+ unsigned long vmaddr)
{
}
-static inline void flush_tlb_page_nohash(struct vm_area_struct *vma,
- unsigned long vmaddr)
+static inline void hash__flush_tlb_page_nohash(struct vm_area_struct *vma,
+ unsigned long vmaddr)
{
}
-static inline void flush_tlb_range(struct vm_area_struct *vma,
- unsigned long start, unsigned long end)
+static inline void hash__flush_tlb_range(struct vm_area_struct *vma,
+ unsigned long start, unsigned long end)
{
}
-static inline void flush_tlb_kernel_range(unsigned long start,
- unsigned long end)
+static inline void hash__flush_tlb_kernel_range(unsigned long start,
+ unsigned long end)
{
}
+
+struct mmu_gather;
+extern void hash__tlb_flush(struct mmu_gather *tlb);
/* Private function for use by PCI IO mapping code */
extern void __flush_hash_table_range(struct mm_struct *mm, unsigned long start,
unsigned long end);
diff --git a/arch/powerpc/include/asm/book3s/64/tlbflush-radix.h b/arch/powerpc/include/asm/book3s/64/tlbflush-radix.h
new file mode 100644
index 0000000..13ef388
--- /dev/null
+++ b/arch/powerpc/include/asm/book3s/64/tlbflush-radix.h
@@ -0,0 +1,33 @@
+#ifndef _ASM_POWERPC_TLBFLUSH_RADIX_H
+#define _ASM_POWERPC_TLBFLUSH_RADIX_H
+
+struct vm_area_struct;
+struct mm_struct;
+struct mmu_gather;
+
+static inline int mmu_get_ap(int psize)
+{
+ return mmu_psize_defs[psize].ap;
+}
+
+extern void radix__flush_tlb_range(struct vm_area_struct *vma, unsigned long start,
+ unsigned long end);
+extern void radix__flush_tlb_kernel_range(unsigned long start, unsigned long end);
+
+extern void radix__local_flush_tlb_mm(struct mm_struct *mm);
+extern void radix__local_flush_tlb_page(struct vm_area_struct *vma, unsigned long vmaddr);
+extern void radix___local_flush_tlb_page(struct mm_struct *mm, unsigned long vmaddr,
+ unsigned long ap, int nid);
+extern void radix__tlb_flush(struct mmu_gather *tlb);
+#ifdef CONFIG_SMP
+extern void radix__flush_tlb_mm(struct mm_struct *mm);
+extern void radix__flush_tlb_page(struct vm_area_struct *vma, unsigned long vmaddr);
+extern void radix___flush_tlb_page(struct mm_struct *mm, unsigned long vmaddr,
+ unsigned long ap, int nid);
+#else
+#define radix__flush_tlb_mm(mm) radix__local_flush_tlb_mm(mm)
+#define radix__flush_tlb_page(vma,addr) radix__local_flush_tlb_page(vma,addr)
+#define radix___flush_tlb_page(mm,addr,p,i) radix___local_flush_tlb_page(mm,addr,p,i)
+#endif
+
+#endif
diff --git a/arch/powerpc/include/asm/book3s/64/tlbflush.h b/arch/powerpc/include/asm/book3s/64/tlbflush.h
new file mode 100644
index 0000000..d98424a
--- /dev/null
+++ b/arch/powerpc/include/asm/book3s/64/tlbflush.h
@@ -0,0 +1,76 @@
+#ifndef _ASM_POWERPC_BOOK3S_64_TLBFLUSH_H
+#define _ASM_POWERPC_BOOK3S_64_TLBFLUSH_H
+
+#define MMU_NO_CONTEXT ~0UL
+
+
+#include <asm/book3s/64/tlbflush-hash.h>
+#include <asm/book3s/64/tlbflush-radix.h>
+
+static inline void flush_tlb_range(struct vm_area_struct *vma,
+ unsigned long start, unsigned long end)
+{
+ if (radix_enabled())
+ return radix__flush_tlb_range(vma, start, end);
+ return hash__flush_tlb_range(vma, start, end);
+}
+
+static inline void flush_tlb_kernel_range(unsigned long start,
+ unsigned long end)
+{
+ if (radix_enabled())
+ return radix__flush_tlb_kernel_range(start, end);
+ return hash__flush_tlb_kernel_range(start, end);
+}
+
+static inline void local_flush_tlb_mm(struct mm_struct *mm)
+{
+ if (radix_enabled())
+ return radix__local_flush_tlb_mm(mm);
+ return hash__local_flush_tlb_mm(mm);
+}
+
+static inline void local_flush_tlb_page(struct vm_area_struct *vma,
+ unsigned long vmaddr)
+{
+ if (radix_enabled())
+ return radix__local_flush_tlb_page(vma, vmaddr);
+ return hash__local_flush_tlb_page(vma, vmaddr);
+}
+
+static inline void flush_tlb_page_nohash(struct vm_area_struct *vma,
+ unsigned long vmaddr)
+{
+ if (radix_enabled())
+ return radix__flush_tlb_page(vma, vmaddr);
+ return hash__flush_tlb_page_nohash(vma, vmaddr);
+}
+
+static inline void tlb_flush(struct mmu_gather *tlb)
+{
+ if (radix_enabled())
+ return radix__tlb_flush(tlb);
+ return hash__tlb_flush(tlb);
+}
+
+#ifdef CONFIG_SMP
+static inline void flush_tlb_mm(struct mm_struct *mm)
+{
+ if (radix_enabled())
+ return radix__flush_tlb_mm(mm);
+ return hash__flush_tlb_mm(mm);
+}
+
+static inline void flush_tlb_page(struct vm_area_struct *vma,
+ unsigned long vmaddr)
+{
+ if (radix_enabled())
+ return radix__flush_tlb_page(vma, vmaddr);
+ return hash__flush_tlb_page(vma, vmaddr);
+}
+#else
+#define flush_tlb_mm(mm) local_flush_tlb_mm(mm)
+#define flush_tlb_page(vma, addr) local_flush_tlb_page(vma, addr)
+#endif /* CONFIG_SMP */
+
+#endif /* _ASM_POWERPC_BOOK3S_64_TLBFLUSH_H */
diff --git a/arch/powerpc/include/asm/book3s/pgalloc.h b/arch/powerpc/include/asm/book3s/pgalloc.h
new file mode 100644
index 0000000..54f591e
--- /dev/null
+++ b/arch/powerpc/include/asm/book3s/pgalloc.h
@@ -0,0 +1,19 @@
+#ifndef _ASM_POWERPC_BOOK3S_PGALLOC_H
+#define _ASM_POWERPC_BOOK3S_PGALLOC_H
+
+#include <linux/mm.h>
+
+extern void tlb_remove_table(struct mmu_gather *tlb, void *table);
+static inline void tlb_flush_pgtable(struct mmu_gather *tlb,
+ unsigned long address)
+{
+
+}
+
+#ifdef CONFIG_PPC64
+#include <asm/book3s/64/pgalloc.h>
+#else
+#include <asm/book3s/32/pgalloc.h>
+#endif
+
+#endif /* _ASM_POWERPC_BOOK3S_PGALLOC_H */
diff --git a/arch/powerpc/include/asm/hugetlb.h b/arch/powerpc/include/asm/hugetlb.h
index 42814f0..e2d9f49 100644
--- a/arch/powerpc/include/asm/hugetlb.h
+++ b/arch/powerpc/include/asm/hugetlb.h
@@ -8,6 +8,8 @@
extern struct kmem_cache *hugepte_cache;
#ifdef CONFIG_PPC_BOOK3S_64
+
+#include <asm/book3s/64/hugetlb-radix.h>
/*
* This should work for other subarchs too. But right now we use the
* new format only for 64bit book3s
@@ -31,7 +33,19 @@
{
return mmu_psize_to_shift(hugepd_mmu_psize(hpd));
}
+static inline void flush_hugetlb_page(struct vm_area_struct *vma,
+ unsigned long vmaddr)
+{
+ if (radix_enabled())
+ return radix__flush_hugetlb_page(vma, vmaddr);
+}
+static inline void __local_flush_hugetlb_page(struct vm_area_struct *vma,
+ unsigned long vmaddr)
+{
+ if (radix_enabled())
+ return radix__local_flush_hugetlb_page(vma, vmaddr);
+}
#else
static inline pte_t *hugepd_page(hugepd_t hpd)
diff --git a/arch/powerpc/include/asm/kvm_book3s_64.h b/arch/powerpc/include/asm/kvm_book3s_64.h
index 7529aab..1f4497f 100644
--- a/arch/powerpc/include/asm/kvm_book3s_64.h
+++ b/arch/powerpc/include/asm/kvm_book3s_64.h
@@ -276,19 +276,24 @@
return ptel;
}
-static inline int hpte_cache_flags_ok(unsigned long ptel, unsigned long io_type)
+static inline bool hpte_cache_flags_ok(unsigned long hptel, bool is_ci)
{
- unsigned int wimg = ptel & HPTE_R_WIMG;
+ unsigned int wimg = hptel & HPTE_R_WIMG;
/* Handle SAO */
if (wimg == (HPTE_R_W | HPTE_R_I | HPTE_R_M) &&
cpu_has_feature(CPU_FTR_ARCH_206))
wimg = HPTE_R_M;
- if (!io_type)
+ if (!is_ci)
return wimg == HPTE_R_M;
-
- return (wimg & (HPTE_R_W | HPTE_R_I)) == io_type;
+ /*
+ * if host is mapped cache inhibited, make sure hptel also have
+ * cache inhibited.
+ */
+ if (wimg & HPTE_R_W) /* FIXME!! is this ok for all guest. ? */
+ return false;
+ return !!(wimg & HPTE_R_I);
}
/*
@@ -305,9 +310,9 @@
*/
old_pte = READ_ONCE(*ptep);
/*
- * wait until _PAGE_BUSY is clear then set it atomically
+ * wait until H_PAGE_BUSY is clear then set it atomically
*/
- if (unlikely(pte_val(old_pte) & _PAGE_BUSY)) {
+ if (unlikely(pte_val(old_pte) & H_PAGE_BUSY)) {
cpu_relax();
continue;
}
@@ -319,27 +324,12 @@
if (writing && pte_write(old_pte))
new_pte = pte_mkdirty(new_pte);
- if (pte_val(old_pte) == __cmpxchg_u64((unsigned long *)ptep,
- pte_val(old_pte),
- pte_val(new_pte))) {
+ if (pte_xchg(ptep, old_pte, new_pte))
break;
- }
}
return new_pte;
}
-
-/* Return HPTE cache control bits corresponding to Linux pte bits */
-static inline unsigned long hpte_cache_bits(unsigned long pte_val)
-{
-#if _PAGE_NO_CACHE == HPTE_R_I && _PAGE_WRITETHRU == HPTE_R_W
- return pte_val & (HPTE_R_W | HPTE_R_I);
-#else
- return ((pte_val & _PAGE_NO_CACHE) ? HPTE_R_I : 0) +
- ((pte_val & _PAGE_WRITETHRU) ? HPTE_R_W : 0);
-#endif
-}
-
static inline bool hpte_read_permission(unsigned long pp, unsigned long key)
{
if (key)
diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h
index d7b3431..ec35af3 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -40,6 +40,9 @@
#define KVM_MAX_VCORES NR_CPUS
#define KVM_USER_MEM_SLOTS 512
+#include <asm/cputhreads.h>
+#define KVM_MAX_VCPU_ID (threads_per_subcore * KVM_MAX_VCORES)
+
#ifdef CONFIG_KVM_MMIO
#define KVM_COALESCED_MMIO_PAGE_OFFSET 1
#endif
@@ -113,6 +116,7 @@
u32 ext_intr_exits;
u32 halt_successful_poll;
u32 halt_attempted_poll;
+ u32 halt_poll_invalid;
u32 halt_wakeup;
u32 dbell_exits;
u32 gdbell_exits;
@@ -724,5 +728,6 @@
static inline void kvm_arch_exit(void) {}
static inline void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu) {}
static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {}
+static inline void kvm_arch_vcpu_block_finish(struct kvm_vcpu *vcpu) {}
#endif /* __POWERPC_KVM_HOST_H__ */
diff --git a/arch/powerpc/include/asm/machdep.h b/arch/powerpc/include/asm/machdep.h
index fd22442..6bdcd0d 100644
--- a/arch/powerpc/include/asm/machdep.h
+++ b/arch/powerpc/include/asm/machdep.h
@@ -256,6 +256,7 @@
#ifdef CONFIG_ARCH_RANDOM
int (*get_random_seed)(unsigned long *v);
#endif
+ int (*update_partition_table)(u64);
};
extern void e500_idle(void);
diff --git a/arch/powerpc/include/asm/mmu.h b/arch/powerpc/include/asm/mmu.h
index 8ca1c98..e53ebeb 100644
--- a/arch/powerpc/include/asm/mmu.h
+++ b/arch/powerpc/include/asm/mmu.h
@@ -88,6 +88,11 @@
*/
#define MMU_FTR_1T_SEGMENT ASM_CONST(0x40000000)
+/*
+ * Radix page table available
+ */
+#define MMU_FTR_RADIX ASM_CONST(0x80000000)
+
/* MMU feature bit sets for various CPUs */
#define MMU_FTRS_DEFAULT_HPTE_ARCH_V2 \
MMU_FTR_HPTE_TABLE | MMU_FTR_PPCAS_ARCH_V2
@@ -110,9 +115,25 @@
DECLARE_PER_CPU(int, next_tlbcam_idx);
#endif
+enum {
+ MMU_FTRS_POSSIBLE = MMU_FTR_HPTE_TABLE | MMU_FTR_TYPE_8xx |
+ MMU_FTR_TYPE_40x | MMU_FTR_TYPE_44x | MMU_FTR_TYPE_FSL_E |
+ MMU_FTR_TYPE_47x | MMU_FTR_USE_HIGH_BATS | MMU_FTR_BIG_PHYS |
+ MMU_FTR_USE_TLBIVAX_BCAST | MMU_FTR_USE_TLBILX |
+ MMU_FTR_LOCK_BCAST_INVAL | MMU_FTR_NEED_DTLB_SW_LRU |
+ MMU_FTR_USE_TLBRSRV | MMU_FTR_USE_PAIRED_MAS |
+ MMU_FTR_NO_SLBIE_B | MMU_FTR_16M_PAGE | MMU_FTR_TLBIEL |
+ MMU_FTR_LOCKLESS_TLBIE | MMU_FTR_CI_LARGE_PAGE |
+ MMU_FTR_1T_SEGMENT |
+#ifdef CONFIG_PPC_RADIX_MMU
+ MMU_FTR_RADIX |
+#endif
+ 0,
+};
+
static inline int mmu_has_feature(unsigned long feature)
{
- return (cur_cpu_spec->mmu_features & feature);
+ return (MMU_FTRS_POSSIBLE & cur_cpu_spec->mmu_features & feature);
}
static inline void mmu_clear_feature(unsigned long feature)
@@ -122,13 +143,6 @@
extern unsigned int __start___mmu_ftr_fixup, __stop___mmu_ftr_fixup;
-/* MMU initialization */
-extern void early_init_mmu(void);
-extern void early_init_mmu_secondary(void);
-
-extern void setup_initial_memory_limit(phys_addr_t first_memblock_base,
- phys_addr_t first_memblock_size);
-
#ifdef CONFIG_PPC64
/* This is our real memory area size on ppc64 server, on embedded, we
* make it match the size our of bolted TLB area
@@ -181,10 +195,20 @@
#define MMU_PAGE_COUNT 15
-#if defined(CONFIG_PPC_STD_MMU_64)
-/* 64-bit classic hash table MMU */
-#include <asm/book3s/64/mmu-hash.h>
-#elif defined(CONFIG_PPC_STD_MMU_32)
+#ifdef CONFIG_PPC_BOOK3S_64
+#include <asm/book3s/64/mmu.h>
+#else /* CONFIG_PPC_BOOK3S_64 */
+
+#ifndef __ASSEMBLY__
+/* MMU initialization */
+extern void early_init_mmu(void);
+extern void early_init_mmu_secondary(void);
+extern void setup_initial_memory_limit(phys_addr_t first_memblock_base,
+ phys_addr_t first_memblock_size);
+#endif /* __ASSEMBLY__ */
+#endif
+
+#if defined(CONFIG_PPC_STD_MMU_32)
/* 32-bit classic hash table MMU */
#include <asm/book3s/32/mmu-hash.h>
#elif defined(CONFIG_40x)
@@ -201,6 +225,9 @@
# include <asm/mmu-8xx.h>
#endif
+#ifndef radix_enabled
+#define radix_enabled() (0)
+#endif
#endif /* __KERNEL__ */
#endif /* _ASM_POWERPC_MMU_H_ */
diff --git a/arch/powerpc/include/asm/mmu_context.h b/arch/powerpc/include/asm/mmu_context.h
index 4eaab40..9d2cd0c 100644
--- a/arch/powerpc/include/asm/mmu_context.h
+++ b/arch/powerpc/include/asm/mmu_context.h
@@ -33,16 +33,27 @@
extern long mm_iommu_mapped_inc(struct mm_iommu_table_group_mem_t *mem);
extern void mm_iommu_mapped_dec(struct mm_iommu_table_group_mem_t *mem);
#endif
-
-extern void switch_mmu_context(struct mm_struct *prev, struct mm_struct *next);
extern void switch_slb(struct task_struct *tsk, struct mm_struct *mm);
extern void set_context(unsigned long id, pgd_t *pgd);
#ifdef CONFIG_PPC_BOOK3S_64
+extern void radix__switch_mmu_context(struct mm_struct *prev,
+ struct mm_struct *next);
+static inline void switch_mmu_context(struct mm_struct *prev,
+ struct mm_struct *next,
+ struct task_struct *tsk)
+{
+ if (radix_enabled())
+ return radix__switch_mmu_context(prev, next);
+ return switch_slb(tsk, next);
+}
+
extern int __init_new_context(void);
extern void __destroy_context(int context_id);
static inline void mmu_context_init(void) { }
#else
+extern void switch_mmu_context(struct mm_struct *prev, struct mm_struct *next,
+ struct task_struct *tsk);
extern unsigned long __init_new_context(void);
extern void __destroy_context(unsigned long context_id);
extern void mmu_context_init(void);
@@ -88,17 +99,11 @@
if (cpu_has_feature(CPU_FTR_ALTIVEC))
asm volatile ("dssall");
#endif /* CONFIG_ALTIVEC */
-
- /* The actual HW switching method differs between the various
- * sub architectures.
+ /*
+ * The actual HW switching method differs between the various
+ * sub architectures. Out of line for now
*/
-#ifdef CONFIG_PPC_STD_MMU_64
- switch_slb(tsk, next);
-#else
- /* Out of line for now */
- switch_mmu_context(prev, next);
-#endif
-
+ switch_mmu_context(prev, next, tsk);
}
#define deactivate_mm(tsk,mm) do { } while (0)
diff --git a/arch/powerpc/include/asm/pgalloc-32.h b/arch/powerpc/include/asm/nohash/32/pgalloc.h
similarity index 100%
rename from arch/powerpc/include/asm/pgalloc-32.h
rename to arch/powerpc/include/asm/nohash/32/pgalloc.h
diff --git a/arch/powerpc/include/asm/nohash/64/pgalloc.h b/arch/powerpc/include/asm/nohash/64/pgalloc.h
new file mode 100644
index 0000000..0c12a3b
--- /dev/null
+++ b/arch/powerpc/include/asm/nohash/64/pgalloc.h
@@ -0,0 +1,212 @@
+#ifndef _ASM_POWERPC_PGALLOC_64_H
+#define _ASM_POWERPC_PGALLOC_64_H
+/*
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <linux/slab.h>
+#include <linux/cpumask.h>
+#include <linux/percpu.h>
+
+struct vmemmap_backing {
+ struct vmemmap_backing *list;
+ unsigned long phys;
+ unsigned long virt_addr;
+};
+extern struct vmemmap_backing *vmemmap_list;
+
+/*
+ * Functions that deal with pagetables that could be at any level of
+ * the table need to be passed an "index_size" so they know how to
+ * handle allocation. For PTE pages (which are linked to a struct
+ * page for now, and drawn from the main get_free_pages() pool), the
+ * allocation size will be (2^index_size * sizeof(pointer)) and
+ * allocations are drawn from the kmem_cache in PGT_CACHE(index_size).
+ *
+ * The maximum index size needs to be big enough to allow any
+ * pagetable sizes we need, but small enough to fit in the low bits of
+ * any page table pointer. In other words all pagetables, even tiny
+ * ones, must be aligned to allow at least enough low 0 bits to
+ * contain this value. This value is also used as a mask, so it must
+ * be one less than a power of two.
+ */
+#define MAX_PGTABLE_INDEX_SIZE 0xf
+
+extern struct kmem_cache *pgtable_cache[];
+#define PGT_CACHE(shift) ({ \
+ BUG_ON(!(shift)); \
+ pgtable_cache[(shift) - 1]; \
+ })
+
+static inline pgd_t *pgd_alloc(struct mm_struct *mm)
+{
+ return kmem_cache_alloc(PGT_CACHE(PGD_INDEX_SIZE), GFP_KERNEL);
+}
+
+static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd)
+{
+ kmem_cache_free(PGT_CACHE(PGD_INDEX_SIZE), pgd);
+}
+
+#ifndef CONFIG_PPC_64K_PAGES
+
+#define pgd_populate(MM, PGD, PUD) pgd_set(PGD, (unsigned long)PUD)
+
+static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr)
+{
+ return kmem_cache_alloc(PGT_CACHE(PUD_INDEX_SIZE),
+ GFP_KERNEL|__GFP_REPEAT);
+}
+
+static inline void pud_free(struct mm_struct *mm, pud_t *pud)
+{
+ kmem_cache_free(PGT_CACHE(PUD_INDEX_SIZE), pud);
+}
+
+static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd)
+{
+ pud_set(pud, (unsigned long)pmd);
+}
+
+static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd,
+ pte_t *pte)
+{
+ pmd_set(pmd, (unsigned long)pte);
+}
+
+static inline void pmd_populate(struct mm_struct *mm, pmd_t *pmd,
+ pgtable_t pte_page)
+{
+ pmd_set(pmd, (unsigned long)page_address(pte_page));
+}
+
+#define pmd_pgtable(pmd) pmd_page(pmd)
+
+static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm,
+ unsigned long address)
+{
+ return (pte_t *)__get_free_page(GFP_KERNEL | __GFP_REPEAT | __GFP_ZERO);
+}
+
+static inline pgtable_t pte_alloc_one(struct mm_struct *mm,
+ unsigned long address)
+{
+ struct page *page;
+ pte_t *pte;
+
+ pte = pte_alloc_one_kernel(mm, address);
+ if (!pte)
+ return NULL;
+ page = virt_to_page(pte);
+ if (!pgtable_page_ctor(page)) {
+ __free_page(page);
+ return NULL;
+ }
+ return page;
+}
+
+static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte)
+{
+ free_page((unsigned long)pte);
+}
+
+static inline void pte_free(struct mm_struct *mm, pgtable_t ptepage)
+{
+ pgtable_page_dtor(ptepage);
+ __free_page(ptepage);
+}
+
+extern void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int shift);
+#ifdef CONFIG_SMP
+extern void __tlb_remove_table(void *_table);
+#endif
+static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t table,
+ unsigned long address)
+{
+ tlb_flush_pgtable(tlb, address);
+ pgtable_free_tlb(tlb, page_address(table), 0);
+}
+
+#else /* if CONFIG_PPC_64K_PAGES */
+
+extern pte_t *pte_fragment_alloc(struct mm_struct *, unsigned long, int);
+extern void pte_fragment_free(unsigned long *, int);
+extern void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int shift);
+#ifdef CONFIG_SMP
+extern void __tlb_remove_table(void *_table);
+#endif
+
+#define pud_populate(mm, pud, pmd) pud_set(pud, (unsigned long)pmd)
+
+static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd,
+ pte_t *pte)
+{
+ pmd_set(pmd, (unsigned long)pte);
+}
+
+static inline void pmd_populate(struct mm_struct *mm, pmd_t *pmd,
+ pgtable_t pte_page)
+{
+ pmd_set(pmd, (unsigned long)pte_page);
+}
+
+static inline pgtable_t pmd_pgtable(pmd_t pmd)
+{
+ return (pgtable_t)(pmd_val(pmd) & ~PMD_MASKED_BITS);
+}
+
+static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm,
+ unsigned long address)
+{
+ return (pte_t *)pte_fragment_alloc(mm, address, 1);
+}
+
+static inline pgtable_t pte_alloc_one(struct mm_struct *mm,
+ unsigned long address)
+{
+ return (pgtable_t)pte_fragment_alloc(mm, address, 0);
+}
+
+static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte)
+{
+ pte_fragment_fre((unsigned long *)pte, 1);
+}
+
+static inline void pte_free(struct mm_struct *mm, pgtable_t ptepage)
+{
+ pte_fragment_free((unsigned long *)ptepage, 0);
+}
+
+static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t table,
+ unsigned long address)
+{
+ tlb_flush_pgtable(tlb, address);
+ pgtable_free_tlb(tlb, table, 0);
+}
+#endif /* CONFIG_PPC_64K_PAGES */
+
+static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long addr)
+{
+ return kmem_cache_alloc(PGT_CACHE(PMD_CACHE_INDEX),
+ GFP_KERNEL|__GFP_REPEAT);
+}
+
+static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd)
+{
+ kmem_cache_free(PGT_CACHE(PMD_CACHE_INDEX), pmd);
+}
+
+#define __pmd_free_tlb(tlb, pmd, addr) \
+ pgtable_free_tlb(tlb, pmd, PMD_CACHE_INDEX)
+#ifndef CONFIG_PPC_64K_PAGES
+#define __pud_free_tlb(tlb, pud, addr) \
+ pgtable_free_tlb(tlb, pud, PUD_INDEX_SIZE)
+
+#endif /* CONFIG_PPC_64K_PAGES */
+
+#define check_pgt_cache() do { } while (0)
+
+#endif /* _ASM_POWERPC_PGALLOC_64_H */
diff --git a/arch/powerpc/include/asm/nohash/64/pgtable.h b/arch/powerpc/include/asm/nohash/64/pgtable.h
index 10debb9..d4d808c 100644
--- a/arch/powerpc/include/asm/nohash/64/pgtable.h
+++ b/arch/powerpc/include/asm/nohash/64/pgtable.h
@@ -108,9 +108,6 @@
#ifndef __ASSEMBLY__
/* pte_clear moved to later in this file */
-/* Pointers in the page table tree are virtual addresses */
-#define __pgtable_ptr_val(ptr) ((unsigned long)(ptr))
-
#define PMD_BAD_BITS (PTE_TABLE_SIZE-1)
#define PUD_BAD_BITS (PMD_TABLE_SIZE-1)
@@ -362,6 +359,13 @@
void pgtable_cache_add(unsigned shift, void (*ctor)(void *));
void pgtable_cache_init(void);
+extern int map_kernel_page(unsigned long ea, unsigned long pa,
+ unsigned long flags);
+extern int __meminit vmemmap_create_mapping(unsigned long start,
+ unsigned long page_size,
+ unsigned long phys);
+extern void vmemmap_remove_mapping(unsigned long start,
+ unsigned long page_size);
#endif /* __ASSEMBLY__ */
#endif /* _ASM_POWERPC_NOHASH_64_PGTABLE_H */
diff --git a/arch/powerpc/include/asm/nohash/pgalloc.h b/arch/powerpc/include/asm/nohash/pgalloc.h
new file mode 100644
index 0000000..b39ec95
--- /dev/null
+++ b/arch/powerpc/include/asm/nohash/pgalloc.h
@@ -0,0 +1,23 @@
+#ifndef _ASM_POWERPC_NOHASH_PGALLOC_H
+#define _ASM_POWERPC_NOHASH_PGALLOC_H
+
+#include <linux/mm.h>
+
+extern void tlb_remove_table(struct mmu_gather *tlb, void *table);
+#ifdef CONFIG_PPC64
+extern void tlb_flush_pgtable(struct mmu_gather *tlb, unsigned long address);
+#else
+/* 44x etc which is BOOKE not BOOK3E */
+static inline void tlb_flush_pgtable(struct mmu_gather *tlb,
+ unsigned long address)
+{
+
+}
+#endif /* !CONFIG_PPC_BOOK3E */
+
+#ifdef CONFIG_PPC64
+#include <asm/nohash/64/pgalloc.h>
+#else
+#include <asm/nohash/32/pgalloc.h>
+#endif
+#endif /* _ASM_POWERPC_NOHASH_PGALLOC_H */
diff --git a/arch/powerpc/include/asm/opal-api.h b/arch/powerpc/include/asm/opal-api.h
index f8faaae..9bb8ddf 100644
--- a/arch/powerpc/include/asm/opal-api.h
+++ b/arch/powerpc/include/asm/opal-api.h
@@ -368,16 +368,16 @@
};
enum opal_msg_type {
- OPAL_MSG_ASYNC_COMP = 0, /* params[0] = token, params[1] = rc,
+ OPAL_MSG_ASYNC_COMP = 0, /* params[0] = token, params[1] = rc,
* additional params function-specific
*/
- OPAL_MSG_MEM_ERR,
- OPAL_MSG_EPOW,
- OPAL_MSG_SHUTDOWN, /* params[0] = 1 reboot, 0 shutdown */
- OPAL_MSG_HMI_EVT,
- OPAL_MSG_DPO,
- OPAL_MSG_PRD,
- OPAL_MSG_OCC,
+ OPAL_MSG_MEM_ERR = 1,
+ OPAL_MSG_EPOW = 2,
+ OPAL_MSG_SHUTDOWN = 3, /* params[0] = 1 reboot, 0 shutdown */
+ OPAL_MSG_HMI_EVT = 4,
+ OPAL_MSG_DPO = 5,
+ OPAL_MSG_PRD = 6,
+ OPAL_MSG_OCC = 7,
OPAL_MSG_TYPE_MAX,
};
diff --git a/arch/powerpc/include/asm/page.h b/arch/powerpc/include/asm/page.h
index ab3d897..51db3a3 100644
--- a/arch/powerpc/include/asm/page.h
+++ b/arch/powerpc/include/asm/page.h
@@ -288,7 +288,11 @@
#ifndef __ASSEMBLY__
+#ifdef CONFIG_PPC_BOOK3S_64
+#include <asm/pgtable-be-types.h>
+#else
#include <asm/pgtable-types.h>
+#endif
typedef struct { signed long pd; } hugepd_t;
@@ -312,12 +316,20 @@
#endif
struct vm_area_struct;
-
+#ifdef CONFIG_PPC_BOOK3S_64
+/*
+ * For BOOK3s 64 with 4k and 64K linux page size
+ * we want to use pointers, because the page table
+ * actually store pfn
+ */
+typedef pte_t *pgtable_t;
+#else
#if defined(CONFIG_PPC_64K_PAGES) && defined(CONFIG_PPC64)
typedef pte_t *pgtable_t;
#else
typedef struct page *pgtable_t;
#endif
+#endif
#include <asm-generic/memory_model.h>
#endif /* __ASSEMBLY__ */
diff --git a/arch/powerpc/include/asm/page_64.h b/arch/powerpc/include/asm/page_64.h
index d908a46..dd5f071 100644
--- a/arch/powerpc/include/asm/page_64.h
+++ b/arch/powerpc/include/asm/page_64.h
@@ -93,7 +93,7 @@
#define SLICE_LOW_TOP (0x100000000ul)
#define SLICE_NUM_LOW (SLICE_LOW_TOP >> SLICE_LOW_SHIFT)
-#define SLICE_NUM_HIGH (PGTABLE_RANGE >> SLICE_HIGH_SHIFT)
+#define SLICE_NUM_HIGH (H_PGTABLE_RANGE >> SLICE_HIGH_SHIFT)
#define GET_LOW_SLICE_INDEX(addr) ((addr) >> SLICE_LOW_SHIFT)
#define GET_HIGH_SLICE_INDEX(addr) ((addr) >> SLICE_HIGH_SHIFT)
@@ -128,8 +128,6 @@
extern void slice_set_range_psize(struct mm_struct *mm, unsigned long start,
unsigned long len, unsigned int psize);
-#define slice_mm_new_context(mm) ((mm)->context.id == MMU_NO_CONTEXT)
-
#endif /* __ASSEMBLY__ */
#else
#define slice_init()
@@ -151,7 +149,6 @@
#define slice_set_range_psize(mm, start, len, psize) \
slice_set_user_psize((mm), (psize))
-#define slice_mm_new_context(mm) 1
#endif /* CONFIG_PPC_MM_SLICES */
#ifdef CONFIG_HUGETLB_PAGE
diff --git a/arch/powerpc/include/asm/pci-bridge.h b/arch/powerpc/include/asm/pci-bridge.h
index f5056e3..467c0b0 100644
--- a/arch/powerpc/include/asm/pci-bridge.h
+++ b/arch/powerpc/include/asm/pci-bridge.h
@@ -17,33 +17,34 @@
* PCI controller operations
*/
struct pci_controller_ops {
- void (*dma_dev_setup)(struct pci_dev *dev);
+ void (*dma_dev_setup)(struct pci_dev *pdev);
void (*dma_bus_setup)(struct pci_bus *bus);
- int (*probe_mode)(struct pci_bus *);
+ int (*probe_mode)(struct pci_bus *bus);
/* Called when pci_enable_device() is called. Returns true to
* allow assignment/enabling of the device. */
- bool (*enable_device_hook)(struct pci_dev *);
+ bool (*enable_device_hook)(struct pci_dev *pdev);
- void (*disable_device)(struct pci_dev *);
+ void (*disable_device)(struct pci_dev *pdev);
- void (*release_device)(struct pci_dev *);
+ void (*release_device)(struct pci_dev *pdev);
/* Called during PCI resource reassignment */
- resource_size_t (*window_alignment)(struct pci_bus *, unsigned long type);
- void (*reset_secondary_bus)(struct pci_dev *dev);
+ resource_size_t (*window_alignment)(struct pci_bus *bus,
+ unsigned long type);
+ void (*reset_secondary_bus)(struct pci_dev *pdev);
#ifdef CONFIG_PCI_MSI
- int (*setup_msi_irqs)(struct pci_dev *dev,
+ int (*setup_msi_irqs)(struct pci_dev *pdev,
int nvec, int type);
- void (*teardown_msi_irqs)(struct pci_dev *dev);
+ void (*teardown_msi_irqs)(struct pci_dev *pdev);
#endif
- int (*dma_set_mask)(struct pci_dev *dev, u64 dma_mask);
- u64 (*dma_get_required_mask)(struct pci_dev *dev);
+ int (*dma_set_mask)(struct pci_dev *pdev, u64 dma_mask);
+ u64 (*dma_get_required_mask)(struct pci_dev *pdev);
- void (*shutdown)(struct pci_controller *);
+ void (*shutdown)(struct pci_controller *hose);
};
/*
@@ -208,14 +209,14 @@
#ifdef CONFIG_EEH
struct eeh_dev *edev; /* eeh device */
#endif
-#define IODA_INVALID_PE (-1)
+#define IODA_INVALID_PE 0xFFFFFFFF
#ifdef CONFIG_PPC_POWERNV
- int pe_number;
+ unsigned int pe_number;
int vf_index; /* VF index in the PF */
#ifdef CONFIG_PCI_IOV
u16 vfs_expanded; /* number of VFs IOV BAR expanded */
u16 num_vfs; /* number of VFs enabled*/
- int *pe_num_map; /* PE# for the first VF PE or array */
+ unsigned int *pe_num_map; /* PE# for the first VF PE or array */
bool m64_single_mode; /* Use M64 BAR in Single Mode */
#define IODA_INVALID_M64 (-1)
int (*m64_map)[PCI_SRIOV_NUM_BARS];
@@ -234,7 +235,9 @@
extern struct pci_dn *pci_get_pdn(struct pci_dev *pdev);
extern struct pci_dn *add_dev_pci_data(struct pci_dev *pdev);
extern void remove_dev_pci_data(struct pci_dev *pdev);
-extern void *update_dn_pci_info(struct device_node *dn, void *data);
+extern struct pci_dn *pci_add_device_node_info(struct pci_controller *hose,
+ struct device_node *dn);
+extern void pci_remove_device_node_info(struct device_node *dn);
static inline int pci_device_from_OF_node(struct device_node *np,
u8 *bus, u8 *devfn)
@@ -256,13 +259,13 @@
#endif
/** Find the bus corresponding to the indicated device node */
-extern struct pci_bus *pcibios_find_pci_bus(struct device_node *dn);
+extern struct pci_bus *pci_find_bus_by_node(struct device_node *dn);
/** Remove all of the PCI devices under this bus */
-extern void pcibios_remove_pci_devices(struct pci_bus *bus);
+extern void pci_hp_remove_devices(struct pci_bus *bus);
/** Discover new pci devices under this bus, and add them */
-extern void pcibios_add_pci_devices(struct pci_bus *bus);
+extern void pci_hp_add_devices(struct pci_bus *bus);
extern void isa_bridge_find_early(struct pci_controller *hose);
diff --git a/arch/powerpc/include/asm/pgalloc-64.h b/arch/powerpc/include/asm/pgalloc-64.h
deleted file mode 100644
index 8d5fc3a..0000000
--- a/arch/powerpc/include/asm/pgalloc-64.h
+++ /dev/null
@@ -1,266 +0,0 @@
-#ifndef _ASM_POWERPC_PGALLOC_64_H
-#define _ASM_POWERPC_PGALLOC_64_H
-/*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- */
-
-#include <linux/slab.h>
-#include <linux/cpumask.h>
-#include <linux/percpu.h>
-
-struct vmemmap_backing {
- struct vmemmap_backing *list;
- unsigned long phys;
- unsigned long virt_addr;
-};
-extern struct vmemmap_backing *vmemmap_list;
-
-/*
- * Functions that deal with pagetables that could be at any level of
- * the table need to be passed an "index_size" so they know how to
- * handle allocation. For PTE pages (which are linked to a struct
- * page for now, and drawn from the main get_free_pages() pool), the
- * allocation size will be (2^index_size * sizeof(pointer)) and
- * allocations are drawn from the kmem_cache in PGT_CACHE(index_size).
- *
- * The maximum index size needs to be big enough to allow any
- * pagetable sizes we need, but small enough to fit in the low bits of
- * any page table pointer. In other words all pagetables, even tiny
- * ones, must be aligned to allow at least enough low 0 bits to
- * contain this value. This value is also used as a mask, so it must
- * be one less than a power of two.
- */
-#define MAX_PGTABLE_INDEX_SIZE 0xf
-
-extern struct kmem_cache *pgtable_cache[];
-#define PGT_CACHE(shift) ({ \
- BUG_ON(!(shift)); \
- pgtable_cache[(shift) - 1]; \
- })
-
-static inline pgd_t *pgd_alloc(struct mm_struct *mm)
-{
- return kmem_cache_alloc(PGT_CACHE(PGD_INDEX_SIZE), GFP_KERNEL);
-}
-
-static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd)
-{
- kmem_cache_free(PGT_CACHE(PGD_INDEX_SIZE), pgd);
-}
-
-#ifndef CONFIG_PPC_64K_PAGES
-
-#define pgd_populate(MM, PGD, PUD) pgd_set(PGD, __pgtable_ptr_val(PUD))
-
-static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr)
-{
- return kmem_cache_alloc(PGT_CACHE(PUD_INDEX_SIZE),
- GFP_KERNEL|__GFP_REPEAT);
-}
-
-static inline void pud_free(struct mm_struct *mm, pud_t *pud)
-{
- kmem_cache_free(PGT_CACHE(PUD_INDEX_SIZE), pud);
-}
-
-static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd)
-{
- pud_set(pud, __pgtable_ptr_val(pmd));
-}
-
-static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd,
- pte_t *pte)
-{
- pmd_set(pmd, __pgtable_ptr_val(pte));
-}
-
-static inline void pmd_populate(struct mm_struct *mm, pmd_t *pmd,
- pgtable_t pte_page)
-{
- pmd_set(pmd, __pgtable_ptr_val(page_address(pte_page)));
-}
-
-#define pmd_pgtable(pmd) pmd_page(pmd)
-
-static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm,
- unsigned long address)
-{
- return (pte_t *)__get_free_page(GFP_KERNEL | __GFP_REPEAT | __GFP_ZERO);
-}
-
-static inline pgtable_t pte_alloc_one(struct mm_struct *mm,
- unsigned long address)
-{
- struct page *page;
- pte_t *pte;
-
- pte = pte_alloc_one_kernel(mm, address);
- if (!pte)
- return NULL;
- page = virt_to_page(pte);
- if (!pgtable_page_ctor(page)) {
- __free_page(page);
- return NULL;
- }
- return page;
-}
-
-static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte)
-{
- free_page((unsigned long)pte);
-}
-
-static inline void pte_free(struct mm_struct *mm, pgtable_t ptepage)
-{
- pgtable_page_dtor(ptepage);
- __free_page(ptepage);
-}
-
-static inline void pgtable_free(void *table, unsigned index_size)
-{
- if (!index_size)
- free_page((unsigned long)table);
- else {
- BUG_ON(index_size > MAX_PGTABLE_INDEX_SIZE);
- kmem_cache_free(PGT_CACHE(index_size), table);
- }
-}
-
-#ifdef CONFIG_SMP
-static inline void pgtable_free_tlb(struct mmu_gather *tlb,
- void *table, int shift)
-{
- unsigned long pgf = (unsigned long)table;
- BUG_ON(shift > MAX_PGTABLE_INDEX_SIZE);
- pgf |= shift;
- tlb_remove_table(tlb, (void *)pgf);
-}
-
-static inline void __tlb_remove_table(void *_table)
-{
- void *table = (void *)((unsigned long)_table & ~MAX_PGTABLE_INDEX_SIZE);
- unsigned shift = (unsigned long)_table & MAX_PGTABLE_INDEX_SIZE;
-
- pgtable_free(table, shift);
-}
-#else /* !CONFIG_SMP */
-static inline void pgtable_free_tlb(struct mmu_gather *tlb,
- void *table, int shift)
-{
- pgtable_free(table, shift);
-}
-#endif /* CONFIG_SMP */
-
-static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t table,
- unsigned long address)
-{
- tlb_flush_pgtable(tlb, address);
- pgtable_page_dtor(table);
- pgtable_free_tlb(tlb, page_address(table), 0);
-}
-
-#else /* if CONFIG_PPC_64K_PAGES */
-
-extern pte_t *page_table_alloc(struct mm_struct *, unsigned long, int);
-extern void page_table_free(struct mm_struct *, unsigned long *, int);
-extern void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int shift);
-#ifdef CONFIG_SMP
-extern void __tlb_remove_table(void *_table);
-#endif
-
-#ifndef __PAGETABLE_PUD_FOLDED
-/* book3s 64 is 4 level page table */
-static inline void pgd_populate(struct mm_struct *mm, pgd_t *pgd, pud_t *pud)
-{
- pgd_set(pgd, __pgtable_ptr_val(pud));
-}
-
-static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long addr)
-{
- return kmem_cache_alloc(PGT_CACHE(PUD_INDEX_SIZE),
- GFP_KERNEL|__GFP_REPEAT);
-}
-
-static inline void pud_free(struct mm_struct *mm, pud_t *pud)
-{
- kmem_cache_free(PGT_CACHE(PUD_INDEX_SIZE), pud);
-}
-#endif
-
-static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd)
-{
- pud_set(pud, __pgtable_ptr_val(pmd));
-}
-
-static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd,
- pte_t *pte)
-{
- pmd_set(pmd, __pgtable_ptr_val(pte));
-}
-
-static inline void pmd_populate(struct mm_struct *mm, pmd_t *pmd,
- pgtable_t pte_page)
-{
- pmd_set(pmd, __pgtable_ptr_val(pte_page));
-}
-
-static inline pgtable_t pmd_pgtable(pmd_t pmd)
-{
- return (pgtable_t)pmd_page_vaddr(pmd);
-}
-
-static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm,
- unsigned long address)
-{
- return (pte_t *)page_table_alloc(mm, address, 1);
-}
-
-static inline pgtable_t pte_alloc_one(struct mm_struct *mm,
- unsigned long address)
-{
- return (pgtable_t)page_table_alloc(mm, address, 0);
-}
-
-static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte)
-{
- page_table_free(mm, (unsigned long *)pte, 1);
-}
-
-static inline void pte_free(struct mm_struct *mm, pgtable_t ptepage)
-{
- page_table_free(mm, (unsigned long *)ptepage, 0);
-}
-
-static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t table,
- unsigned long address)
-{
- tlb_flush_pgtable(tlb, address);
- pgtable_free_tlb(tlb, table, 0);
-}
-#endif /* CONFIG_PPC_64K_PAGES */
-
-static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long addr)
-{
- return kmem_cache_alloc(PGT_CACHE(PMD_CACHE_INDEX),
- GFP_KERNEL|__GFP_REPEAT);
-}
-
-static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd)
-{
- kmem_cache_free(PGT_CACHE(PMD_CACHE_INDEX), pmd);
-}
-
-#define __pmd_free_tlb(tlb, pmd, addr) \
- pgtable_free_tlb(tlb, pmd, PMD_CACHE_INDEX)
-#ifndef __PAGETABLE_PUD_FOLDED
-#define __pud_free_tlb(tlb, pud, addr) \
- pgtable_free_tlb(tlb, pud, PUD_INDEX_SIZE)
-
-#endif /* __PAGETABLE_PUD_FOLDED */
-
-#define check_pgt_cache() do { } while (0)
-
-#endif /* _ASM_POWERPC_PGALLOC_64_H */
diff --git a/arch/powerpc/include/asm/pgalloc.h b/arch/powerpc/include/asm/pgalloc.h
index fc3ee06..0413457 100644
--- a/arch/powerpc/include/asm/pgalloc.h
+++ b/arch/powerpc/include/asm/pgalloc.h
@@ -1,25 +1,12 @@
#ifndef _ASM_POWERPC_PGALLOC_H
#define _ASM_POWERPC_PGALLOC_H
-#ifdef __KERNEL__
#include <linux/mm.h>
-#ifdef CONFIG_PPC_BOOK3E
-extern void tlb_flush_pgtable(struct mmu_gather *tlb, unsigned long address);
-#else /* CONFIG_PPC_BOOK3E */
-static inline void tlb_flush_pgtable(struct mmu_gather *tlb,
- unsigned long address)
-{
-}
-#endif /* !CONFIG_PPC_BOOK3E */
-
-extern void tlb_remove_table(struct mmu_gather *tlb, void *table);
-
-#ifdef CONFIG_PPC64
-#include <asm/pgalloc-64.h>
+#ifdef CONFIG_PPC_BOOK3S
+#include <asm/book3s/pgalloc.h>
#else
-#include <asm/pgalloc-32.h>
+#include <asm/nohash/pgalloc.h>
#endif
-#endif /* __KERNEL__ */
#endif /* _ASM_POWERPC_PGALLOC_H */
diff --git a/arch/powerpc/include/asm/pgtable-be-types.h b/arch/powerpc/include/asm/pgtable-be-types.h
new file mode 100644
index 0000000..e2bf2086
--- /dev/null
+++ b/arch/powerpc/include/asm/pgtable-be-types.h
@@ -0,0 +1,92 @@
+#ifndef _ASM_POWERPC_PGTABLE_BE_TYPES_H
+#define _ASM_POWERPC_PGTABLE_BE_TYPES_H
+
+#include <asm/cmpxchg.h>
+
+/* PTE level */
+typedef struct { __be64 pte; } pte_t;
+#define __pte(x) ((pte_t) { cpu_to_be64(x) })
+static inline unsigned long pte_val(pte_t x)
+{
+ return be64_to_cpu(x.pte);
+}
+
+static inline __be64 pte_raw(pte_t x)
+{
+ return x.pte;
+}
+
+/* PMD level */
+#ifdef CONFIG_PPC64
+typedef struct { __be64 pmd; } pmd_t;
+#define __pmd(x) ((pmd_t) { cpu_to_be64(x) })
+static inline unsigned long pmd_val(pmd_t x)
+{
+ return be64_to_cpu(x.pmd);
+}
+
+static inline __be64 pmd_raw(pmd_t x)
+{
+ return x.pmd;
+}
+
+/*
+ * 64 bit hash always use 4 level table. Everybody else use 4 level
+ * only for 4K page size.
+ */
+#if defined(CONFIG_PPC_BOOK3S_64) || !defined(CONFIG_PPC_64K_PAGES)
+typedef struct { __be64 pud; } pud_t;
+#define __pud(x) ((pud_t) { cpu_to_be64(x) })
+static inline unsigned long pud_val(pud_t x)
+{
+ return be64_to_cpu(x.pud);
+}
+#endif /* CONFIG_PPC_BOOK3S_64 || !CONFIG_PPC_64K_PAGES */
+#endif /* CONFIG_PPC64 */
+
+/* PGD level */
+typedef struct { __be64 pgd; } pgd_t;
+#define __pgd(x) ((pgd_t) { cpu_to_be64(x) })
+static inline unsigned long pgd_val(pgd_t x)
+{
+ return be64_to_cpu(x.pgd);
+}
+
+/* Page protection bits */
+typedef struct { unsigned long pgprot; } pgprot_t;
+#define pgprot_val(x) ((x).pgprot)
+#define __pgprot(x) ((pgprot_t) { (x) })
+
+/*
+ * With hash config 64k pages additionally define a bigger "real PTE" type that
+ * gathers the "second half" part of the PTE for pseudo 64k pages
+ */
+#if defined(CONFIG_PPC_64K_PAGES) && defined(CONFIG_PPC_STD_MMU_64)
+typedef struct { pte_t pte; unsigned long hidx; } real_pte_t;
+#else
+typedef struct { pte_t pte; } real_pte_t;
+#endif
+
+static inline bool pte_xchg(pte_t *ptep, pte_t old, pte_t new)
+{
+ unsigned long *p = (unsigned long *)ptep;
+ __be64 prev;
+
+ prev = (__force __be64)__cmpxchg_u64(p, (__force unsigned long)pte_raw(old),
+ (__force unsigned long)pte_raw(new));
+
+ return pte_raw(old) == prev;
+}
+
+static inline bool pmd_xchg(pmd_t *pmdp, pmd_t old, pmd_t new)
+{
+ unsigned long *p = (unsigned long *)pmdp;
+ __be64 prev;
+
+ prev = (__force __be64)__cmpxchg_u64(p, (__force unsigned long)pmd_raw(old),
+ (__force unsigned long)pmd_raw(new));
+
+ return pmd_raw(old) == prev;
+}
+
+#endif /* _ASM_POWERPC_PGTABLE_BE_TYPES_H */
diff --git a/arch/powerpc/include/asm/pgtable-types.h b/arch/powerpc/include/asm/pgtable-types.h
index 43140f8..e7f4f3e 100644
--- a/arch/powerpc/include/asm/pgtable-types.h
+++ b/arch/powerpc/include/asm/pgtable-types.h
@@ -1,9 +1,6 @@
#ifndef _ASM_POWERPC_PGTABLE_TYPES_H
#define _ASM_POWERPC_PGTABLE_TYPES_H
-#ifdef CONFIG_STRICT_MM_TYPECHECKS
-/* These are used to make use of C type-checking. */
-
/* PTE level */
typedef struct { pte_basic_t pte; } pte_t;
#define __pte(x) ((pte_t) { (x) })
@@ -48,49 +45,6 @@
#define pgprot_val(x) ((x).pgprot)
#define __pgprot(x) ((pgprot_t) { (x) })
-#else
-
-/*
- * .. while these make it easier on the compiler
- */
-
-typedef pte_basic_t pte_t;
-#define __pte(x) (x)
-static inline pte_basic_t pte_val(pte_t pte)
-{
- return pte;
-}
-
-#ifdef CONFIG_PPC64
-typedef unsigned long pmd_t;
-#define __pmd(x) (x)
-static inline unsigned long pmd_val(pmd_t pmd)
-{
- return pmd;
-}
-
-#if defined(CONFIG_PPC_BOOK3S_64) || !defined(CONFIG_PPC_64K_PAGES)
-typedef unsigned long pud_t;
-#define __pud(x) (x)
-static inline unsigned long pud_val(pud_t pud)
-{
- return pud;
-}
-#endif /* CONFIG_PPC_BOOK3S_64 || !CONFIG_PPC_64K_PAGES */
-#endif /* CONFIG_PPC64 */
-
-typedef unsigned long pgd_t;
-#define __pgd(x) (x)
-static inline unsigned long pgd_val(pgd_t pgd)
-{
- return pgd;
-}
-
-typedef unsigned long pgprot_t;
-#define pgprot_val(x) (x)
-#define __pgprot(x) (x)
-
-#endif /* CONFIG_STRICT_MM_TYPECHECKS */
/*
* With hash config 64k pages additionally define a bigger "real PTE" type that
* gathers the "second half" part of the PTE for pseudo 64k pages
@@ -100,4 +54,16 @@
#else
typedef struct { pte_t pte; } real_pte_t;
#endif
+
+#ifdef CONFIG_PPC_STD_MMU_64
+#include <asm/cmpxchg.h>
+
+static inline bool pte_xchg(pte_t *ptep, pte_t old, pte_t new)
+{
+ unsigned long *p = (unsigned long *)ptep;
+
+ return pte_val(old) == __cmpxchg_u64(p, pte_val(old), pte_val(new));
+}
+#endif
+
#endif /* _ASM_POWERPC_PGTABLE_TYPES_H */
diff --git a/arch/powerpc/include/asm/pgtable.h b/arch/powerpc/include/asm/pgtable.h
index 47897a3..ee09e99 100644
--- a/arch/powerpc/include/asm/pgtable.h
+++ b/arch/powerpc/include/asm/pgtable.h
@@ -65,7 +65,6 @@
struct page **pages, int *nr);
#ifndef CONFIG_TRANSPARENT_HUGEPAGE
#define pmd_large(pmd) 0
-#define has_transparent_hugepage() 0
#endif
pte_t *__find_linux_pte_or_hugepte(pgd_t *pgdir, unsigned long ea,
bool *is_thp, unsigned *shift);
diff --git a/arch/powerpc/include/asm/ppc-opcode.h b/arch/powerpc/include/asm/ppc-opcode.h
index 7ab04fc..1d035c1 100644
--- a/arch/powerpc/include/asm/ppc-opcode.h
+++ b/arch/powerpc/include/asm/ppc-opcode.h
@@ -131,6 +131,7 @@
/* sorted alphabetically */
#define PPC_INST_BHRBE 0x7c00025c
#define PPC_INST_CLRBHRB 0x7c00035c
+#define PPC_INST_CP_ABORT 0x7c00068c
#define PPC_INST_DCBA 0x7c0005ec
#define PPC_INST_DCBA_MASK 0xfc0007fe
#define PPC_INST_DCBAL 0x7c2005ec
@@ -285,6 +286,7 @@
#endif
/* Deal with instructions that older assemblers aren't aware of */
+#define PPC_CP_ABORT stringify_in_c(.long PPC_INST_CP_ABORT)
#define PPC_DCBAL(a, b) stringify_in_c(.long PPC_INST_DCBAL | \
__PPC_RA(a) | __PPC_RB(b))
#define PPC_DCBZL(a, b) stringify_in_c(.long PPC_INST_DCBZL | \
diff --git a/arch/powerpc/include/asm/ppc-pci.h b/arch/powerpc/include/asm/ppc-pci.h
index ca0c5bf..8753e4e 100644
--- a/arch/powerpc/include/asm/ppc-pci.h
+++ b/arch/powerpc/include/asm/ppc-pci.h
@@ -33,9 +33,9 @@
struct device_node;
struct pci_dn;
-typedef void *(*traverse_func)(struct device_node *me, void *data);
-void *traverse_pci_devices(struct device_node *start, traverse_func pre,
- void *data);
+void *pci_traverse_device_nodes(struct device_node *start,
+ void *(*fn)(struct device_node *, void *),
+ void *data);
void *traverse_pci_dn(struct pci_dn *root,
void *(*fn)(struct pci_dn *, void *),
void *data);
diff --git a/arch/powerpc/include/asm/ppc_asm.h b/arch/powerpc/include/asm/ppc_asm.h
index 499d9f8..2b31632 100644
--- a/arch/powerpc/include/asm/ppc_asm.h
+++ b/arch/powerpc/include/asm/ppc_asm.h
@@ -427,7 +427,10 @@
li r4,1024; \
mtctr r4; \
lis r4,KERNELBASE@h; \
+ .machine push; \
+ .machine "power4"; \
0: tlbie r4; \
+ .machine pop; \
addi r4,r4,0x1000; \
bdnz 0b
#endif
diff --git a/arch/powerpc/include/asm/pte-common.h b/arch/powerpc/include/asm/pte-common.h
index 1ec67b0..2eeaf80 100644
--- a/arch/powerpc/include/asm/pte-common.h
+++ b/arch/powerpc/include/asm/pte-common.h
@@ -76,6 +76,16 @@
*/
#ifndef __ASSEMBLY__
extern unsigned long bad_call_to_PMD_PAGE_SIZE(void);
+
+/*
+ * Don't just check for any non zero bits in __PAGE_USER, since for book3e
+ * and PTE_64BIT, PAGE_KERNEL_X contains _PAGE_BAP_SR which is also in
+ * _PAGE_USER. Need to explicitly match _PAGE_BAP_UR bit in that case too.
+ */
+static inline bool pte_user(pte_t pte)
+{
+ return (pte_val(pte) & _PAGE_USER) == _PAGE_USER;
+}
#endif /* __ASSEMBLY__ */
/* Location of the PFN in the PTE. Most 32-bit platforms use the same
@@ -184,13 +194,6 @@
/* Make modules code happy. We don't set RO yet */
#define PAGE_KERNEL_EXEC PAGE_KERNEL_X
-/*
- * Don't just check for any non zero bits in __PAGE_USER, since for book3e
- * and PTE_64BIT, PAGE_KERNEL_X contains _PAGE_BAP_SR which is also in
- * _PAGE_USER. Need to explicitly match _PAGE_BAP_UR bit in that case too.
- */
-#define pte_user(val) ((val & _PAGE_USER) == _PAGE_USER)
-
/* Advertise special mapping type for AGP */
#define PAGE_AGP (PAGE_KERNEL_NC)
#define HAVE_PAGE_AGP
@@ -198,3 +201,12 @@
/* Advertise support for _PAGE_SPECIAL */
#define __HAVE_ARCH_PTE_SPECIAL
+#ifndef _PAGE_READ
+/* if not defined, we should not find _PAGE_WRITE too */
+#define _PAGE_READ 0
+#define _PAGE_WRITE _PAGE_RW
+#endif
+
+#ifndef H_PAGE_4K_PFN
+#define H_PAGE_4K_PFN 0
+#endif
diff --git a/arch/powerpc/include/asm/reg.h b/arch/powerpc/include/asm/reg.h
index f5f4c66..c1e82e9 100644
--- a/arch/powerpc/include/asm/reg.h
+++ b/arch/powerpc/include/asm/reg.h
@@ -347,6 +347,7 @@
#define LPCR_LPES_SH 2
#define LPCR_RMI 0x00000002 /* real mode is cache inhibit */
#define LPCR_HDICE 0x00000001 /* Hyp Decr enable (HV,PR,EE) */
+#define LPCR_UPRT 0x00400000 /* Use Process Table (ISA 3) */
#ifndef SPRN_LPID
#define SPRN_LPID 0x13F /* Logical Partition Identifier */
#endif
@@ -587,6 +588,7 @@
#define SPRN_PIR 0x3FF /* Processor Identification Register */
#endif
#define SPRN_TIR 0x1BE /* Thread Identification Register */
+#define SPRN_PTCR 0x1D0 /* Partition table control Register */
#define SPRN_PSPB 0x09F /* Problem State Priority Boost reg */
#define SPRN_PTEHI 0x3D5 /* 981 7450 PTE HI word (S/W TLB load) */
#define SPRN_PTELO 0x3D6 /* 982 7450 PTE LO word (S/W TLB load) */
@@ -1182,6 +1184,7 @@
#define PVR_970GX 0x0045
#define PVR_POWER7p 0x004A
#define PVR_POWER8E 0x004B
+#define PVR_POWER8NVL 0x004C
#define PVR_POWER8 0x004D
#define PVR_BE 0x0070
#define PVR_PA6T 0x0090
diff --git a/arch/powerpc/include/asm/tlbflush.h b/arch/powerpc/include/asm/tlbflush.h
index 9f77f85..1b38eea 100644
--- a/arch/powerpc/include/asm/tlbflush.h
+++ b/arch/powerpc/include/asm/tlbflush.h
@@ -58,6 +58,7 @@
#elif defined(CONFIG_PPC_STD_MMU_32)
+#define MMU_NO_CONTEXT (0)
/*
* TLB flushing for "classic" hash-MMU 32-bit CPUs, 6xx, 7xx, 7xxx
*/
@@ -78,7 +79,7 @@
}
#elif defined(CONFIG_PPC_STD_MMU_64)
-#include <asm/book3s/64/tlbflush-hash.h>
+#include <asm/book3s/64/tlbflush.h>
#else
#error Unsupported MMU type
#endif
diff --git a/arch/powerpc/include/uapi/asm/perf_regs.h b/arch/powerpc/include/uapi/asm/perf_regs.h
new file mode 100644
index 0000000..6a93209
--- /dev/null
+++ b/arch/powerpc/include/uapi/asm/perf_regs.h
@@ -0,0 +1,50 @@
+#ifndef _UAPI_ASM_POWERPC_PERF_REGS_H
+#define _UAPI_ASM_POWERPC_PERF_REGS_H
+
+enum perf_event_powerpc_regs {
+ PERF_REG_POWERPC_R0,
+ PERF_REG_POWERPC_R1,
+ PERF_REG_POWERPC_R2,
+ PERF_REG_POWERPC_R3,
+ PERF_REG_POWERPC_R4,
+ PERF_REG_POWERPC_R5,
+ PERF_REG_POWERPC_R6,
+ PERF_REG_POWERPC_R7,
+ PERF_REG_POWERPC_R8,
+ PERF_REG_POWERPC_R9,
+ PERF_REG_POWERPC_R10,
+ PERF_REG_POWERPC_R11,
+ PERF_REG_POWERPC_R12,
+ PERF_REG_POWERPC_R13,
+ PERF_REG_POWERPC_R14,
+ PERF_REG_POWERPC_R15,
+ PERF_REG_POWERPC_R16,
+ PERF_REG_POWERPC_R17,
+ PERF_REG_POWERPC_R18,
+ PERF_REG_POWERPC_R19,
+ PERF_REG_POWERPC_R20,
+ PERF_REG_POWERPC_R21,
+ PERF_REG_POWERPC_R22,
+ PERF_REG_POWERPC_R23,
+ PERF_REG_POWERPC_R24,
+ PERF_REG_POWERPC_R25,
+ PERF_REG_POWERPC_R26,
+ PERF_REG_POWERPC_R27,
+ PERF_REG_POWERPC_R28,
+ PERF_REG_POWERPC_R29,
+ PERF_REG_POWERPC_R30,
+ PERF_REG_POWERPC_R31,
+ PERF_REG_POWERPC_NIP,
+ PERF_REG_POWERPC_MSR,
+ PERF_REG_POWERPC_ORIG_R3,
+ PERF_REG_POWERPC_CTR,
+ PERF_REG_POWERPC_LINK,
+ PERF_REG_POWERPC_XER,
+ PERF_REG_POWERPC_CCR,
+ PERF_REG_POWERPC_SOFTE,
+ PERF_REG_POWERPC_TRAP,
+ PERF_REG_POWERPC_DAR,
+ PERF_REG_POWERPC_DSISR,
+ PERF_REG_POWERPC_MAX,
+};
+#endif /* _UAPI_ASM_POWERPC_PERF_REGS_H */
diff --git a/arch/powerpc/kernel/asm-offsets.c b/arch/powerpc/kernel/asm-offsets.c
index c9370d4..9ea0955 100644
--- a/arch/powerpc/kernel/asm-offsets.c
+++ b/arch/powerpc/kernel/asm-offsets.c
@@ -438,7 +438,11 @@
DEFINE(BUG_ENTRY_SIZE, sizeof(struct bug_entry));
#endif
+#ifdef MAX_PGD_TABLE_SIZE
+ DEFINE(PGD_TABLE_SIZE, MAX_PGD_TABLE_SIZE);
+#else
DEFINE(PGD_TABLE_SIZE, PGD_TABLE_SIZE);
+#endif
DEFINE(PTE_SIZE, sizeof(pte_t));
#ifdef CONFIG_KVM
diff --git a/arch/powerpc/kernel/btext.c b/arch/powerpc/kernel/btext.c
index 41c011c..8275858 100644
--- a/arch/powerpc/kernel/btext.c
+++ b/arch/powerpc/kernel/btext.c
@@ -162,7 +162,7 @@
offset = ((unsigned long) dispDeviceBase) - base;
size = dispDeviceRowBytes * dispDeviceRect[3] + offset
+ dispDeviceRect[0];
- vbase = __ioremap(base, size, _PAGE_NO_CACHE);
+ vbase = __ioremap(base, size, pgprot_val(pgprot_noncached_wc(__pgprot(0))));
if (vbase == 0)
return;
logicalDisplayBase = vbase + offset;
diff --git a/arch/powerpc/kernel/cputable.c b/arch/powerpc/kernel/cputable.c
index 6c662b8..eeeacf6 100644
--- a/arch/powerpc/kernel/cputable.c
+++ b/arch/powerpc/kernel/cputable.c
@@ -63,7 +63,6 @@
extern void __setup_cpu_ppc970(unsigned long offset, struct cpu_spec* spec);
extern void __setup_cpu_ppc970MP(unsigned long offset, struct cpu_spec* spec);
extern void __setup_cpu_pa6t(unsigned long offset, struct cpu_spec* spec);
-extern void __setup_cpu_a2(unsigned long offset, struct cpu_spec* spec);
extern void __restore_cpu_pa6t(void);
extern void __restore_cpu_ppc970(void);
extern void __setup_cpu_power7(unsigned long offset, struct cpu_spec* spec);
@@ -72,7 +71,6 @@
extern void __restore_cpu_power8(void);
extern void __setup_cpu_power9(unsigned long offset, struct cpu_spec* spec);
extern void __restore_cpu_power9(void);
-extern void __restore_cpu_a2(void);
extern void __flush_tlb_power7(unsigned int action);
extern void __flush_tlb_power8(unsigned int action);
extern void __flush_tlb_power9(unsigned int action);
diff --git a/arch/powerpc/kernel/eeh.c b/arch/powerpc/kernel/eeh.c
index 6544017..c9bc78e 100644
--- a/arch/powerpc/kernel/eeh.c
+++ b/arch/powerpc/kernel/eeh.c
@@ -48,7 +48,7 @@
/** Overview:
- * EEH, or "Extended Error Handling" is a PCI bridge technology for
+ * EEH, or "Enhanced Error Handling" is a PCI bridge technology for
* dealing with PCI bus errors that can't be dealt with within the
* usual PCI framework, except by check-stopping the CPU. Systems
* that are designed for high-availability/reliability cannot afford
@@ -1068,7 +1068,7 @@
struct pci_controller *phb;
struct eeh_dev *edev = pdn_to_eeh_dev(pdn);
- if (!edev || !eeh_enabled())
+ if (!edev)
return;
if (!eeh_has_flag(EEH_PROBE_MODE_DEVTREE))
@@ -1336,14 +1336,11 @@
id->subdevice != pdev->subsystem_device)
continue;
- goto reset;
+ return eeh_pe_reset_and_recover(pe);
}
}
return eeh_unfreeze_pe(pe, true);
-
-reset:
- return eeh_pe_reset_and_recover(pe);
}
/**
diff --git a/arch/powerpc/kernel/eeh_driver.c b/arch/powerpc/kernel/eeh_driver.c
index fb6207d..2714a3b 100644
--- a/arch/powerpc/kernel/eeh_driver.c
+++ b/arch/powerpc/kernel/eeh_driver.c
@@ -171,6 +171,16 @@
if (!edev)
return NULL;
+ /*
+ * We cannot access the config space on some adapters.
+ * Otherwise, it will cause fenced PHB. We don't save
+ * the content in their config space and will restore
+ * from the initial config space saved when the EEH
+ * device is created.
+ */
+ if (edev->pe && (edev->pe->state & EEH_PE_CFG_RESTRICTED))
+ return NULL;
+
pdev = eeh_dev_to_pci_dev(edev);
if (!pdev)
return NULL;
@@ -312,6 +322,19 @@
if (!edev)
return NULL;
+ /*
+ * The content in the config space isn't saved because
+ * the blocked config space on some adapters. We have
+ * to restore the initial saved config space when the
+ * EEH device is created.
+ */
+ if (edev->pe && (edev->pe->state & EEH_PE_CFG_RESTRICTED)) {
+ if (list_is_last(&edev->list, &edev->pe->edevs))
+ eeh_pe_restore_bars(edev->pe);
+
+ return NULL;
+ }
+
pdev = eeh_dev_to_pci_dev(edev);
if (!pdev)
return NULL;
@@ -552,7 +575,7 @@
int eeh_pe_reset_and_recover(struct eeh_pe *pe)
{
- int result, ret;
+ int ret;
/* Bail if the PE is being recovered */
if (pe->state & EEH_PE_RECOVERING)
@@ -564,9 +587,6 @@
/* Save states */
eeh_pe_dev_traverse(pe, eeh_dev_save_state, NULL);
- /* Report error */
- eeh_pe_dev_traverse(pe, eeh_report_error, &result);
-
/* Issue reset */
ret = eeh_reset_pe(pe);
if (ret) {
@@ -581,15 +601,9 @@
return ret;
}
- /* Notify completion of reset */
- eeh_pe_dev_traverse(pe, eeh_report_reset, &result);
-
/* Restore device state */
eeh_pe_dev_traverse(pe, eeh_dev_restore_state, NULL);
- /* Resume */
- eeh_pe_dev_traverse(pe, eeh_report_resume, NULL);
-
/* Clear recovery mode */
eeh_pe_state_clear(pe, EEH_PE_RECOVERING);
@@ -621,7 +635,7 @@
* We don't remove the corresponding PE instances because
* we need the information afterwords. The attached EEH
* devices are expected to be attached soon when calling
- * into pcibios_add_pci_devices().
+ * into pci_hp_add_devices().
*/
eeh_pe_state_mark(pe, EEH_PE_KEEP);
if (bus) {
@@ -630,7 +644,7 @@
} else {
eeh_pe_state_clear(pe, EEH_PE_PRI_BUS);
pci_lock_rescan_remove();
- pcibios_remove_pci_devices(bus);
+ pci_hp_remove_devices(bus);
pci_unlock_rescan_remove();
}
} else if (frozen_bus) {
@@ -681,7 +695,7 @@
if (pe->type & EEH_PE_VF)
eeh_add_virt_device(edev, NULL);
else
- pcibios_add_pci_devices(bus);
+ pci_hp_add_devices(bus);
} else if (frozen_bus && rmv_data->removed) {
pr_info("EEH: Sleep 5s ahead of partial hotplug\n");
ssleep(5);
@@ -691,7 +705,7 @@
if (pe->type & EEH_PE_VF)
eeh_add_virt_device(edev, NULL);
else
- pcibios_add_pci_devices(frozen_bus);
+ pci_hp_add_devices(frozen_bus);
}
eeh_pe_state_clear(pe, EEH_PE_KEEP);
@@ -896,7 +910,7 @@
eeh_pe_dev_mode_mark(pe, EEH_DEV_REMOVED);
pci_lock_rescan_remove();
- pcibios_remove_pci_devices(frozen_bus);
+ pci_hp_remove_devices(frozen_bus);
pci_unlock_rescan_remove();
}
}
@@ -981,7 +995,7 @@
bus = eeh_pe_bus_get(phb_pe);
eeh_pe_dev_traverse(pe,
eeh_report_failure, NULL);
- pcibios_remove_pci_devices(bus);
+ pci_hp_remove_devices(bus);
}
pci_unlock_rescan_remove();
}
diff --git a/arch/powerpc/kernel/eeh_event.c b/arch/powerpc/kernel/eeh_event.c
index 4eefb6e..82e7327 100644
--- a/arch/powerpc/kernel/eeh_event.c
+++ b/arch/powerpc/kernel/eeh_event.c
@@ -36,7 +36,7 @@
static DEFINE_SPINLOCK(eeh_eventlist_lock);
static struct semaphore eeh_eventlist_sem;
-LIST_HEAD(eeh_eventlist);
+static LIST_HEAD(eeh_eventlist);
/**
* eeh_event_handler - Dispatch EEH events.
diff --git a/arch/powerpc/kernel/eeh_pe.c b/arch/powerpc/kernel/eeh_pe.c
index eea48d8..f0520da 100644
--- a/arch/powerpc/kernel/eeh_pe.c
+++ b/arch/powerpc/kernel/eeh_pe.c
@@ -249,7 +249,7 @@
} else {
if (edev->pe_config_addr &&
(edev->pe_config_addr == pe->addr))
- return pe;
+ return pe;
}
/* Try BDF address */
diff --git a/arch/powerpc/kernel/entry_64.S b/arch/powerpc/kernel/entry_64.S
index 39a79c8..73e461a 100644
--- a/arch/powerpc/kernel/entry_64.S
+++ b/arch/powerpc/kernel/entry_64.S
@@ -37,6 +37,7 @@
#include <asm/hw_irq.h>
#include <asm/context_tracking.h>
#include <asm/tm.h>
+#include <asm/ppc-opcode.h>
/*
* System calls.
@@ -509,6 +510,14 @@
ldarx r6,0,r1
END_FTR_SECTION_IFSET(CPU_FTR_STCX_CHECKS_ADDRESS)
+BEGIN_FTR_SECTION
+/*
+ * A cp_abort (copy paste abort) here ensures that when context switching, a
+ * copy from one process can't leak into the paste of another.
+ */
+ PPC_CP_ABORT
+END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300)
+
#ifdef CONFIG_PPC_BOOK3S
/* Cancel all explict user streams as they will have no use after context
* switch and will stop the HW from creating streams itself
@@ -520,7 +529,10 @@
std r6,PACACURRENT(r13) /* Set new 'current' */
ld r8,KSP(r4) /* new stack pointer */
-#ifdef CONFIG_PPC_BOOK3S
+#ifdef CONFIG_PPC_STD_MMU_64
+BEGIN_MMU_FTR_SECTION
+ b 2f
+END_MMU_FTR_SECTION_IFSET(MMU_FTR_RADIX)
BEGIN_FTR_SECTION
clrrdi r6,r8,28 /* get its ESID */
clrrdi r9,r1,28 /* get current sp ESID */
@@ -566,7 +578,7 @@
slbmte r7,r0
isync
2:
-#endif /* !CONFIG_PPC_BOOK3S */
+#endif /* CONFIG_PPC_STD_MMU_64 */
CURRENT_THREAD_INFO(r7, r8) /* base of new stack */
/* Note: this uses SWITCH_FRAME_SIZE rather than INT_FRAME_SIZE
diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index 7716ceb..4c94406 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -189,7 +189,7 @@
#endif /* CONFIG_PPC_P7_NAP */
EXCEPTION_PROLOG_0(PACA_EXMC)
BEGIN_FTR_SECTION
- b machine_check_pSeries_early
+ b machine_check_powernv_early
FTR_SECTION_ELSE
b machine_check_pSeries_0
ALT_FTR_SECTION_END_IFSET(CPU_FTR_HVMODE)
@@ -209,11 +209,6 @@
EXCEPTION_PROLOG_1(PACA_EXSLB, KVMTEST, 0x380)
std r3,PACA_EXSLB+EX_R3(r13)
mfspr r3,SPRN_DAR
-#ifdef __DISABLED__
- /* Keep that around for when we re-implement dynamic VSIDs */
- cmpdi r3,0
- bge slb_miss_user_pseries
-#endif /* __DISABLED__ */
mfspr r12,SPRN_SRR1
#ifndef CONFIG_RELOCATABLE
b slb_miss_realmode
@@ -240,11 +235,6 @@
EXCEPTION_PROLOG_1(PACA_EXSLB, KVMTEST, 0x480)
std r3,PACA_EXSLB+EX_R3(r13)
mfspr r3,SPRN_SRR0 /* SRR0 is faulting address */
-#ifdef __DISABLED__
- /* Keep that around for when we re-implement dynamic VSIDs */
- cmpdi r3,0
- bge slb_miss_user_pseries
-#endif /* __DISABLED__ */
mfspr r12,SPRN_SRR1
#ifndef CONFIG_RELOCATABLE
b slb_miss_realmode
@@ -443,7 +433,7 @@
.align 7
/* moved from 0x200 */
-machine_check_pSeries_early:
+machine_check_powernv_early:
BEGIN_FTR_SECTION
EXCEPTION_PROLOG_1(PACA_EXMC, NOTEST, 0x200)
/*
@@ -709,34 +699,6 @@
#endif /* CONFIG_PPC_PSERIES */
-#ifdef __DISABLED__
-/*
- * This is used for when the SLB miss handler has to go virtual,
- * which doesn't happen for now anymore but will once we re-implement
- * dynamic VSIDs for shared page tables
- */
-slb_miss_user_pseries:
- std r10,PACA_EXGEN+EX_R10(r13)
- std r11,PACA_EXGEN+EX_R11(r13)
- std r12,PACA_EXGEN+EX_R12(r13)
- GET_SCRATCH0(r10)
- ld r11,PACA_EXSLB+EX_R9(r13)
- ld r12,PACA_EXSLB+EX_R3(r13)
- std r10,PACA_EXGEN+EX_R13(r13)
- std r11,PACA_EXGEN+EX_R9(r13)
- std r12,PACA_EXGEN+EX_R3(r13)
- clrrdi r12,r13,32
- mfmsr r10
- mfspr r11,SRR0 /* save SRR0 */
- ori r12,r12,slb_miss_user_common@l /* virt addr of handler */
- ori r10,r10,MSR_IR|MSR_DR|MSR_RI
- mtspr SRR0,r12
- mfspr r12,SRR1 /* and SRR1 */
- mtspr SRR1,r10
- rfid
- b . /* prevent spec. execution */
-#endif /* __DISABLED__ */
-
#ifdef CONFIG_KVM_BOOK3S_64_HANDLER
kvmppc_skip_interrupt:
/*
@@ -764,11 +726,10 @@
#endif
/*
- * Code from here down to __end_handlers is invoked from the
- * exception prologs above. Because the prologs assemble the
- * addresses of these handlers using the LOAD_HANDLER macro,
- * which uses an ori instruction, these handlers must be in
- * the first 64k of the kernel image.
+ * Ensure that any handlers that get invoked from the exception prologs
+ * above are below the first 64KB (0x10000) of the kernel image because
+ * the prologs assemble the addresses of these handlers using the
+ * LOAD_HANDLER macro, which uses an ori instruction.
*/
/*** Common interrupt handlers ***/
@@ -953,11 +914,6 @@
#endif
STD_RELON_EXCEPTION_PSERIES(0x5700, 0x1700, altivec_assist)
- /* Other future vectors */
- .align 7
- .globl __end_interrupts
-__end_interrupts:
-
.align 7
system_call_entry:
b system_call_common
@@ -983,7 +939,13 @@
ld r3,PACA_EXGEN+EX_DAR(r13)
lwz r4,PACA_EXGEN+EX_DSISR(r13)
li r5,0x300
+ std r3,_DAR(r1)
+ std r4,_DSISR(r1)
+BEGIN_MMU_FTR_SECTION
b do_hash_page /* Try to handle as hpte fault */
+MMU_FTR_SECTION_ELSE
+ b handle_page_fault
+ALT_MMU_FTR_SECTION_END_IFCLR(MMU_FTR_RADIX)
.align 7
.globl h_data_storage_common
@@ -1008,74 +970,16 @@
ld r3,_NIP(r1)
andis. r4,r12,0x5820
li r5,0x400
+ std r3,_DAR(r1)
+ std r4,_DSISR(r1)
+BEGIN_MMU_FTR_SECTION
b do_hash_page /* Try to handle as hpte fault */
+MMU_FTR_SECTION_ELSE
+ b handle_page_fault
+ALT_MMU_FTR_SECTION_END_IFCLR(MMU_FTR_RADIX)
STD_EXCEPTION_COMMON(0xe20, h_instr_storage, unknown_exception)
-/*
- * Here is the common SLB miss user that is used when going to virtual
- * mode for SLB misses, that is currently not used
- */
-#ifdef __DISABLED__
- .align 7
- .globl slb_miss_user_common
-slb_miss_user_common:
- mflr r10
- std r3,PACA_EXGEN+EX_DAR(r13)
- stw r9,PACA_EXGEN+EX_CCR(r13)
- std r10,PACA_EXGEN+EX_LR(r13)
- std r11,PACA_EXGEN+EX_SRR0(r13)
- bl slb_allocate_user
-
- ld r10,PACA_EXGEN+EX_LR(r13)
- ld r3,PACA_EXGEN+EX_R3(r13)
- lwz r9,PACA_EXGEN+EX_CCR(r13)
- ld r11,PACA_EXGEN+EX_SRR0(r13)
- mtlr r10
- beq- slb_miss_fault
-
- andi. r10,r12,MSR_RI /* check for unrecoverable exception */
- beq- unrecov_user_slb
- mfmsr r10
-
-.machine push
-.machine "power4"
- mtcrf 0x80,r9
-.machine pop
-
- clrrdi r10,r10,2 /* clear RI before setting SRR0/1 */
- mtmsrd r10,1
-
- mtspr SRR0,r11
- mtspr SRR1,r12
-
- ld r9,PACA_EXGEN+EX_R9(r13)
- ld r10,PACA_EXGEN+EX_R10(r13)
- ld r11,PACA_EXGEN+EX_R11(r13)
- ld r12,PACA_EXGEN+EX_R12(r13)
- ld r13,PACA_EXGEN+EX_R13(r13)
- rfid
- b .
-
-slb_miss_fault:
- EXCEPTION_PROLOG_COMMON(0x380, PACA_EXGEN)
- ld r4,PACA_EXGEN+EX_DAR(r13)
- li r5,0
- std r4,_DAR(r1)
- std r5,_DSISR(r1)
- b handle_page_fault
-
-unrecov_user_slb:
- EXCEPTION_PROLOG_COMMON(0x4200, PACA_EXGEN)
- RECONCILE_IRQ_STATE(r10, r11)
- bl save_nvgprs
-1: addi r3,r1,STACK_FRAME_OVERHEAD
- bl unrecoverable_exception
- b 1b
-
-#endif /* __DISABLED__ */
-
-
/*
* Machine check is different because we use a different
* save area: PACA_EXMC instead of PACA_EXGEN.
@@ -1230,10 +1134,6 @@
STD_EXCEPTION_COMMON(0xf60, facility_unavailable, facility_unavailable_exception)
STD_EXCEPTION_COMMON(0xf80, hv_facility_unavailable, facility_unavailable_exception)
- .align 7
- .globl __end_handlers
-__end_handlers:
-
/* Equivalents to the above handlers for relocation-on interrupt vectors */
STD_RELON_EXCEPTION_HV_OOL(0xe40, emulation_assist)
MASKABLE_RELON_EXCEPTION_HV_OOL(0xe80, h_doorbell)
@@ -1244,6 +1144,17 @@
STD_RELON_EXCEPTION_PSERIES_OOL(0xf60, facility_unavailable)
STD_RELON_EXCEPTION_HV_OOL(0xf80, hv_facility_unavailable)
+ /*
+ * The __end_interrupts marker must be past the out-of-line (OOL)
+ * handlers, so that they are copied to real address 0x100 when running
+ * a relocatable kernel. This ensures they can be reached from the short
+ * trampoline handlers (like 0x4f00, 0x4f20, etc.) which branch
+ * directly, without using LOAD_HANDLER().
+ */
+ .align 7
+ .globl __end_interrupts
+__end_interrupts:
+
#if defined(CONFIG_PPC_PSERIES) || defined(CONFIG_PPC_POWERNV)
/*
* Data area reserved for FWNMI option.
@@ -1476,8 +1387,11 @@
stw r9,PACA_EXSLB+EX_CCR(r13) /* save CR in exc. frame */
std r10,PACA_EXSLB+EX_LR(r13) /* save LR */
+#ifdef CONFIG_PPC_STD_MMU_64
+BEGIN_MMU_FTR_SECTION
bl slb_allocate_realmode
-
+END_MMU_FTR_SECTION_IFCLR(MMU_FTR_RADIX)
+#endif
/* All done -- return from exception. */
ld r10,PACA_EXSLB+EX_LR(r13)
@@ -1485,7 +1399,9 @@
lwz r9,PACA_EXSLB+EX_CCR(r13) /* get saved CR */
mtlr r10
-
+BEGIN_MMU_FTR_SECTION
+ b 2f
+END_MMU_FTR_SECTION_IFSET(MMU_FTR_RADIX)
andi. r10,r12,MSR_RI /* check for unrecoverable exception */
beq- 2f
@@ -1536,9 +1452,7 @@
*/
.align 7
do_hash_page:
- std r3,_DAR(r1)
- std r4,_DSISR(r1)
-
+#ifdef CONFIG_PPC_STD_MMU_64
andis. r0,r4,0xa410 /* weird error? */
bne- handle_page_fault /* if not, try to insert a HPTE */
andis. r0,r4,DSISR_DABRMATCH@h
@@ -1566,6 +1480,7 @@
/* Error */
blt- 13f
+#endif /* CONFIG_PPC_STD_MMU_64 */
/* Here we have a page fault that hash_page can't handle. */
handle_page_fault:
@@ -1592,6 +1507,7 @@
12: b ret_from_except_lite
+#ifdef CONFIG_PPC_STD_MMU_64
/* We have a page fault that hash_page could handle but HV refused
* the PTE insertion
*/
@@ -1601,6 +1517,7 @@
ld r4,_DAR(r1)
bl low_hash_fault
b ret_from_except
+#endif
/*
* We come here as a result of a DSI at a point where we don't want
diff --git a/arch/powerpc/kernel/ftrace.c b/arch/powerpc/kernel/ftrace.c
index 9dac18da..1123a4d 100644
--- a/arch/powerpc/kernel/ftrace.c
+++ b/arch/powerpc/kernel/ftrace.c
@@ -607,3 +607,13 @@
return sys_call_table[nr*2];
}
#endif /* CONFIG_FTRACE_SYSCALLS && CONFIG_PPC64 */
+
+#if defined(CONFIG_PPC64) && (!defined(_CALL_ELF) || _CALL_ELF != 2)
+char *arch_ftrace_match_adjust(char *str, const char *search)
+{
+ if (str[0] == '.' && search[0] != '.')
+ return str + 1;
+ else
+ return str;
+}
+#endif /* defined(CONFIG_PPC64) && (!defined(_CALL_ELF) || _CALL_ELF != 2) */
diff --git a/arch/powerpc/kernel/head_64.S b/arch/powerpc/kernel/head_64.S
index 4286775..2d14774 100644
--- a/arch/powerpc/kernel/head_64.S
+++ b/arch/powerpc/kernel/head_64.S
@@ -973,13 +973,16 @@
* This stuff goes at the beginning of the bss, which is page-aligned.
*/
.section ".bss"
-
- .align PAGE_SHIFT
-
- .globl empty_zero_page
-empty_zero_page:
- .space PAGE_SIZE
+/*
+ * pgd dir should be aligned to PGD_TABLE_SIZE which is 64K.
+ * We will need to find a better way to fix this
+ */
+ .align 16
.globl swapper_pg_dir
swapper_pg_dir:
.space PGD_TABLE_SIZE
+
+ .globl empty_zero_page
+empty_zero_page:
+ .space PAGE_SIZE
diff --git a/arch/powerpc/kernel/ibmebus.c b/arch/powerpc/kernel/ibmebus.c
index ac86c53..a89f4f7 100644
--- a/arch/powerpc/kernel/ibmebus.c
+++ b/arch/powerpc/kernel/ibmebus.c
@@ -408,7 +408,7 @@
return len+1;
}
-struct device_attribute ibmebus_bus_device_attrs[] = {
+static struct device_attribute ibmebus_bus_device_attrs[] = {
__ATTR_RO(devspec),
__ATTR_RO(name),
__ATTR_RO(modalias),
diff --git a/arch/powerpc/kernel/isa-bridge.c b/arch/powerpc/kernel/isa-bridge.c
index 0f19970..ae13161 100644
--- a/arch/powerpc/kernel/isa-bridge.c
+++ b/arch/powerpc/kernel/isa-bridge.c
@@ -109,14 +109,14 @@
size = 0x10000;
__ioremap_at(phb_io_base_phys, (void *)ISA_IO_BASE,
- size, _PAGE_NO_CACHE|_PAGE_GUARDED);
+ size, pgprot_val(pgprot_noncached(__pgprot(0))));
return;
inval_range:
printk(KERN_ERR "no ISA IO ranges or unexpected isa range, "
"mapping 64k\n");
__ioremap_at(phb_io_base_phys, (void *)ISA_IO_BASE,
- 0x10000, _PAGE_NO_CACHE|_PAGE_GUARDED);
+ 0x10000, pgprot_val(pgprot_noncached(__pgprot(0))));
}
diff --git a/arch/powerpc/kernel/machine_kexec.c b/arch/powerpc/kernel/machine_kexec.c
index 015ae55..2694d07 100644
--- a/arch/powerpc/kernel/machine_kexec.c
+++ b/arch/powerpc/kernel/machine_kexec.c
@@ -228,17 +228,12 @@
static void __init export_crashk_values(struct device_node *node)
{
- struct property *prop;
-
/* There might be existing crash kernel properties, but we can't
* be sure what's in them, so remove them. */
- prop = of_find_property(node, "linux,crashkernel-base", NULL);
- if (prop)
- of_remove_property(node, prop);
-
- prop = of_find_property(node, "linux,crashkernel-size", NULL);
- if (prop)
- of_remove_property(node, prop);
+ of_remove_property(node, of_find_property(node,
+ "linux,crashkernel-base", NULL));
+ of_remove_property(node, of_find_property(node,
+ "linux,crashkernel-size", NULL));
if (crashk_res.start != 0) {
crashk_base = cpu_to_be_ulong(crashk_res.start),
@@ -258,16 +253,13 @@
static int __init kexec_setup(void)
{
struct device_node *node;
- struct property *prop;
node = of_find_node_by_path("/chosen");
if (!node)
return -ENOENT;
/* remove any stale properties so ours can be found */
- prop = of_find_property(node, kernel_end_prop.name, NULL);
- if (prop)
- of_remove_property(node, prop);
+ of_remove_property(node, of_find_property(node, kernel_end_prop.name, NULL));
/* information needed by userspace when using default_machine_kexec */
kernel_end = cpu_to_be_ulong(__pa(_end));
diff --git a/arch/powerpc/kernel/machine_kexec_64.c b/arch/powerpc/kernel/machine_kexec_64.c
index 0fbd75d..b8c202d 100644
--- a/arch/powerpc/kernel/machine_kexec_64.c
+++ b/arch/powerpc/kernel/machine_kexec_64.c
@@ -76,6 +76,7 @@
* end of the blocked region (begin >= high). Use the
* boolean identity !(a || b) === (!a && !b).
*/
+#ifdef CONFIG_PPC_STD_MMU_64
if (htab_address) {
low = __pa(htab_address);
high = low + htab_size_bytes;
@@ -88,6 +89,7 @@
return -ETXTBSY;
}
}
+#endif /* CONFIG_PPC_STD_MMU_64 */
/* We also should not overwrite the tce tables */
for_each_node_by_type(node, "pci") {
@@ -381,7 +383,7 @@
/* NOTREACHED */
}
-#ifndef CONFIG_PPC_BOOK3E
+#ifdef CONFIG_PPC_STD_MMU_64
/* Values we need to export to the second kernel via the device tree. */
static unsigned long htab_base;
static unsigned long htab_size;
@@ -401,7 +403,6 @@
static int __init export_htab_values(void)
{
struct device_node *node;
- struct property *prop;
/* On machines with no htab htab_address is NULL */
if (!htab_address)
@@ -412,12 +413,8 @@
return -ENODEV;
/* remove any stale propertys so ours can be found */
- prop = of_find_property(node, htab_base_prop.name, NULL);
- if (prop)
- of_remove_property(node, prop);
- prop = of_find_property(node, htab_size_prop.name, NULL);
- if (prop)
- of_remove_property(node, prop);
+ of_remove_property(node, of_find_property(node, htab_base_prop.name, NULL));
+ of_remove_property(node, of_find_property(node, htab_size_prop.name, NULL));
htab_base = cpu_to_be64(__pa(htab_address));
of_add_property(node, &htab_base_prop);
@@ -428,4 +425,4 @@
return 0;
}
late_initcall(export_htab_values);
-#endif /* !CONFIG_PPC_BOOK3E */
+#endif /* CONFIG_PPC_STD_MMU_64 */
diff --git a/arch/powerpc/kernel/mce.c b/arch/powerpc/kernel/mce.c
index 671fd51..ef267fd 100644
--- a/arch/powerpc/kernel/mce.c
+++ b/arch/powerpc/kernel/mce.c
@@ -37,7 +37,7 @@
static DEFINE_PER_CPU(struct machine_check_event[MAX_MC_EVT], mce_event_queue);
static void machine_check_process_queued_event(struct irq_work *work);
-struct irq_work mce_event_process_work = {
+static struct irq_work mce_event_process_work = {
.func = machine_check_process_queued_event,
};
diff --git a/arch/powerpc/kernel/mce_power.c b/arch/powerpc/kernel/mce_power.c
index ee62b19..7353991 100644
--- a/arch/powerpc/kernel/mce_power.c
+++ b/arch/powerpc/kernel/mce_power.c
@@ -72,11 +72,15 @@
void __flush_tlb_power9(unsigned int action)
{
+ if (radix_enabled())
+ flush_tlb_206(POWER9_TLB_SETS_RADIX, action);
+
flush_tlb_206(POWER9_TLB_SETS_HASH, action);
}
/* flush SLBs and reload */
+#ifdef CONFIG_PPC_STD_MMU_64
static void flush_and_reload_slb(void)
{
struct slb_shadow *slb;
@@ -110,6 +114,7 @@
asm volatile("slbmte %0,%1" : : "r" (rs), "r" (rb));
}
}
+#endif
static long mce_handle_derror(uint64_t dsisr, uint64_t slb_error_bits)
{
@@ -120,6 +125,7 @@
* reset the error bits whenever we handle them so that at the end
* we can check whether we handled all of them or not.
* */
+#ifdef CONFIG_PPC_STD_MMU_64
if (dsisr & slb_error_bits) {
flush_and_reload_slb();
/* reset error bits */
@@ -131,6 +137,7 @@
/* reset error bits */
dsisr &= ~P7_DSISR_MC_TLB_MULTIHIT_MFTLB;
}
+#endif
/* Any other errors we don't understand? */
if (dsisr & 0xffffffffUL)
handled = 0;
@@ -150,6 +157,7 @@
switch (P7_SRR1_MC_IFETCH(srr1)) {
case 0:
break;
+#ifdef CONFIG_PPC_STD_MMU_64
case P7_SRR1_MC_IFETCH_SLB_PARITY:
case P7_SRR1_MC_IFETCH_SLB_MULTIHIT:
/* flush and reload SLBs for SLB errors. */
@@ -162,6 +170,7 @@
handled = 1;
}
break;
+#endif
default:
break;
}
@@ -175,10 +184,12 @@
handled = mce_handle_common_ierror(srr1);
+#ifdef CONFIG_PPC_STD_MMU_64
if (P7_SRR1_MC_IFETCH(srr1) == P7_SRR1_MC_IFETCH_SLB_BOTH) {
flush_and_reload_slb();
handled = 1;
}
+#endif
return handled;
}
@@ -321,10 +332,12 @@
handled = mce_handle_common_ierror(srr1);
+#ifdef CONFIG_PPC_STD_MMU_64
if (P7_SRR1_MC_IFETCH(srr1) == P8_SRR1_MC_IFETCH_ERAT_MULTIHIT) {
flush_and_reload_slb();
handled = 1;
}
+#endif
return handled;
}
diff --git a/arch/powerpc/kernel/misc_32.S b/arch/powerpc/kernel/misc_32.S
index bf5160f..285ca8c 100644
--- a/arch/powerpc/kernel/misc_32.S
+++ b/arch/powerpc/kernel/misc_32.S
@@ -599,12 +599,6 @@
mr r4,r10
blr
-_GLOBAL(abs)
- srawi r4,r3,31
- xor r3,r3,r4
- sub r3,r3,r4
- blr
-
#ifdef CONFIG_SMP
_GLOBAL(start_secondary_resume)
/* Reset stack */
diff --git a/arch/powerpc/kernel/nvram_64.c b/arch/powerpc/kernel/nvram_64.c
index 0cab9e8..856f9a7 100644
--- a/arch/powerpc/kernel/nvram_64.c
+++ b/arch/powerpc/kernel/nvram_64.c
@@ -15,8 +15,6 @@
* parsing code.
*/
-#include <linux/module.h>
-
#include <linux/types.h>
#include <linux/errno.h>
#include <linux/fs.h>
@@ -1231,12 +1229,4 @@
return rc;
}
-
-static void __exit nvram_cleanup(void)
-{
- misc_deregister( &nvram_dev );
-}
-
-module_init(nvram_init);
-module_exit(nvram_cleanup);
-MODULE_LICENSE("GPL");
+device_initcall(nvram_init);
diff --git a/arch/powerpc/kernel/pci-hotplug.c b/arch/powerpc/kernel/pci-hotplug.c
index 59c4361..2d71269 100644
--- a/arch/powerpc/kernel/pci-hotplug.c
+++ b/arch/powerpc/kernel/pci-hotplug.c
@@ -21,6 +21,35 @@
#include <asm/firmware.h>
#include <asm/eeh.h>
+static struct pci_bus *find_bus_among_children(struct pci_bus *bus,
+ struct device_node *dn)
+{
+ struct pci_bus *child = NULL;
+ struct pci_bus *tmp;
+
+ if (pci_bus_to_OF_node(bus) == dn)
+ return bus;
+
+ list_for_each_entry(tmp, &bus->children, node) {
+ child = find_bus_among_children(tmp, dn);
+ if (child)
+ break;
+ }
+
+ return child;
+}
+
+struct pci_bus *pci_find_bus_by_node(struct device_node *dn)
+{
+ struct pci_dn *pdn = PCI_DN(dn);
+
+ if (!pdn || !pdn->phb || !pdn->phb->bus)
+ return NULL;
+
+ return find_bus_among_children(pdn->phb->bus, dn);
+}
+EXPORT_SYMBOL_GPL(pci_find_bus_by_node);
+
/**
* pcibios_release_device - release PCI device
* @dev: PCI device
@@ -38,20 +67,20 @@
}
/**
- * pcibios_remove_pci_devices - remove all devices under this bus
+ * pci_hp_remove_devices - remove all devices under this bus
* @bus: the indicated PCI bus
*
* Remove all of the PCI devices under this bus both from the
* linux pci device tree, and from the powerpc EEH address cache.
*/
-void pcibios_remove_pci_devices(struct pci_bus *bus)
+void pci_hp_remove_devices(struct pci_bus *bus)
{
struct pci_dev *dev, *tmp;
struct pci_bus *child_bus;
/* First go down child busses */
list_for_each_entry(child_bus, &bus->children, node)
- pcibios_remove_pci_devices(child_bus);
+ pci_hp_remove_devices(child_bus);
pr_debug("PCI: Removing devices on bus %04x:%02x\n",
pci_domain_nr(bus), bus->number);
@@ -60,11 +89,10 @@
pci_stop_and_remove_bus_device(dev);
}
}
-
-EXPORT_SYMBOL_GPL(pcibios_remove_pci_devices);
+EXPORT_SYMBOL_GPL(pci_hp_remove_devices);
/**
- * pcibios_add_pci_devices - adds new pci devices to bus
+ * pci_hp_add_devices - adds new pci devices to bus
* @bus: the indicated PCI bus
*
* This routine will find and fixup new pci devices under
@@ -74,7 +102,7 @@
* is how this routine differs from other, similar pcibios
* routines.)
*/
-void pcibios_add_pci_devices(struct pci_bus * bus)
+void pci_hp_add_devices(struct pci_bus *bus)
{
int slotno, mode, pass, max;
struct pci_dev *dev;
@@ -92,7 +120,8 @@
if (mode == PCI_PROBE_DEVTREE) {
/* use ofdt-based probe */
of_rescan_bus(dn, bus);
- } else if (mode == PCI_PROBE_NORMAL) {
+ } else if (mode == PCI_PROBE_NORMAL &&
+ dn->child && PCI_DN(dn->child)) {
/*
* Use legacy probe. In the partial hotplug case, we
* probably have grandchildren devices unplugged. So
@@ -114,4 +143,4 @@
}
pcibios_finish_adding_to_bus(bus);
}
-EXPORT_SYMBOL_GPL(pcibios_add_pci_devices);
+EXPORT_SYMBOL_GPL(pci_hp_add_devices);
diff --git a/arch/powerpc/kernel/pci_64.c b/arch/powerpc/kernel/pci_64.c
index 60bb187..3759df5 100644
--- a/arch/powerpc/kernel/pci_64.c
+++ b/arch/powerpc/kernel/pci_64.c
@@ -38,7 +38,7 @@
* ISA drivers use hard coded offsets. If no ISA bus exists nothing
* is mapped on the first 64K of IO space
*/
-unsigned long pci_io_base = ISA_IO_BASE;
+unsigned long pci_io_base;
EXPORT_SYMBOL(pci_io_base);
static int __init pcibios_init(void)
@@ -47,6 +47,7 @@
printk(KERN_INFO "PCI: Probing PCI hardware\n");
+ pci_io_base = ISA_IO_BASE;
/* For now, override phys_mem_access_prot. If we need it,g
* later, we may move that initialization to each ppc_md
*/
@@ -159,7 +160,7 @@
/* Establish the mapping */
if (__ioremap_at(phys_page, area->addr, size_page,
- _PAGE_NO_CACHE | _PAGE_GUARDED) == NULL)
+ pgprot_val(pgprot_noncached(__pgprot(0)))) == NULL)
return -ENOMEM;
/* Fixup hose IO resource */
diff --git a/arch/powerpc/kernel/pci_dn.c b/arch/powerpc/kernel/pci_dn.c
index 38102cb..ecdccce 100644
--- a/arch/powerpc/kernel/pci_dn.c
+++ b/arch/powerpc/kernel/pci_dn.c
@@ -282,13 +282,9 @@
#endif /* CONFIG_PCI_IOV */
}
-/*
- * Traverse_func that inits the PCI fields of the device node.
- * NOTE: this *must* be done before read/write config to the device.
- */
-void *update_dn_pci_info(struct device_node *dn, void *data)
+struct pci_dn *pci_add_device_node_info(struct pci_controller *hose,
+ struct device_node *dn)
{
- struct pci_controller *phb = data;
const __be32 *type = of_get_property(dn, "ibm,pci-config-space-type", NULL);
const __be32 *regs;
struct device_node *parent;
@@ -299,7 +295,7 @@
return NULL;
dn->data = pdn;
pdn->node = dn;
- pdn->phb = phb;
+ pdn->phb = hose;
#ifdef CONFIG_PPC_POWERNV
pdn->pe_number = IODA_INVALID_PE;
#endif
@@ -331,8 +327,32 @@
if (pdn->parent)
list_add_tail(&pdn->list, &pdn->parent->child_list);
- return NULL;
+ return pdn;
}
+EXPORT_SYMBOL_GPL(pci_add_device_node_info);
+
+void pci_remove_device_node_info(struct device_node *dn)
+{
+ struct pci_dn *pdn = dn ? PCI_DN(dn) : NULL;
+#ifdef CONFIG_EEH
+ struct eeh_dev *edev = pdn_to_eeh_dev(pdn);
+
+ if (edev)
+ edev->pdn = NULL;
+#endif
+
+ if (!pdn)
+ return;
+
+ WARN_ON(!list_empty(&pdn->child_list));
+ list_del(&pdn->list);
+ if (pdn->parent)
+ of_node_put(pdn->parent->node);
+
+ dn->data = NULL;
+ kfree(pdn);
+}
+EXPORT_SYMBOL_GPL(pci_remove_device_node_info);
/*
* Traverse a device tree stopping each PCI device in the tree.
@@ -352,8 +372,9 @@
* one of these nodes we also assume its siblings are non-pci for
* performance.
*/
-void *traverse_pci_devices(struct device_node *start, traverse_func pre,
- void *data)
+void *pci_traverse_device_nodes(struct device_node *start,
+ void *(*fn)(struct device_node *, void *),
+ void *data)
{
struct device_node *dn, *nextdn;
void *ret;
@@ -368,8 +389,11 @@
if (classp)
class = of_read_number(classp, 1);
- if (pre && ((ret = pre(dn, data)) != NULL))
- return ret;
+ if (fn) {
+ ret = fn(dn, data);
+ if (ret)
+ return ret;
+ }
/* If we are a PCI bridge, go down */
if (dn->child && ((class >> 8) == PCI_CLASS_BRIDGE_PCI ||
@@ -391,6 +415,7 @@
}
return NULL;
}
+EXPORT_SYMBOL_GPL(pci_traverse_device_nodes);
static struct pci_dn *pci_dn_next_one(struct pci_dn *root,
struct pci_dn *pdn)
@@ -432,6 +457,18 @@
return NULL;
}
+static void *add_pdn(struct device_node *dn, void *data)
+{
+ struct pci_controller *hose = data;
+ struct pci_dn *pdn;
+
+ pdn = pci_add_device_node_info(hose, dn);
+ if (!pdn)
+ return ERR_PTR(-ENOMEM);
+
+ return NULL;
+}
+
/**
* pci_devs_phb_init_dynamic - setup pci devices under this PHB
* phb: pci-to-host bridge (top-level bridge connecting to cpu)
@@ -446,8 +483,7 @@
struct pci_dn *pdn;
/* PHB nodes themselves must not match */
- update_dn_pci_info(dn, phb);
- pdn = dn->data;
+ pdn = pci_add_device_node_info(phb, dn);
if (pdn) {
pdn->devfn = pdn->busno = -1;
pdn->vendor_id = pdn->device_id = pdn->class_code = 0;
@@ -456,7 +492,7 @@
}
/* Update dn->phb ptrs for new phb and children devices */
- traverse_pci_devices(dn, update_dn_pci_info, phb);
+ pci_traverse_device_nodes(dn, add_pdn, phb);
}
/**
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index 2a9280b..e2f12cb 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -38,6 +38,7 @@
#include <linux/random.h>
#include <linux/hw_breakpoint.h>
#include <linux/uaccess.h>
+#include <linux/elf-randomize.h>
#include <asm/pgtable.h>
#include <asm/io.h>
@@ -55,6 +56,7 @@
#include <asm/firmware.h>
#endif
#include <asm/code-patching.h>
+#include <asm/exec.h>
#include <asm/livepatch.h>
#include <linux/kprobes.h>
@@ -1077,7 +1079,7 @@
}
#endif /* CONFIG_PPC64 */
-#ifdef CONFIG_PPC_BOOK3S_64
+#ifdef CONFIG_PPC_STD_MMU_64
batch = this_cpu_ptr(&ppc64_tlb_batch);
if (batch->active) {
current_thread_info()->local_flags |= _TLF_LAZY_MMU;
@@ -1085,7 +1087,7 @@
__flush_tlb_pending(batch);
batch->active = 0;
}
-#endif /* CONFIG_PPC_BOOK3S_64 */
+#endif /* CONFIG_PPC_STD_MMU_64 */
#ifdef CONFIG_PPC_ADV_DEBUG_REGS
switch_booke_debug_regs(&new->thread.debug);
@@ -1131,7 +1133,7 @@
last = _switch(old_thread, new_thread);
-#ifdef CONFIG_PPC_BOOK3S_64
+#ifdef CONFIG_PPC_STD_MMU_64
if (current_thread_info()->local_flags & _TLF_LAZY_MMU) {
current_thread_info()->local_flags &= ~_TLF_LAZY_MMU;
batch = this_cpu_ptr(&ppc64_tlb_batch);
@@ -1140,8 +1142,7 @@
if (current_thread_info()->task->thread.regs)
restore_math(current_thread_info()->task->thread.regs);
-
-#endif /* CONFIG_PPC_BOOK3S_64 */
+#endif /* CONFIG_PPC_STD_MMU_64 */
return last;
}
@@ -1328,10 +1329,6 @@
show_instructions(regs);
}
-void exit_thread(void)
-{
-}
-
void flush_thread(void)
{
#ifdef CONFIG_HAVE_HW_BREAKPOINT
@@ -1376,6 +1373,9 @@
unsigned long sp_vsid;
unsigned long llp = mmu_psize_defs[mmu_linear_psize].sllp;
+ if (radix_enabled())
+ return;
+
if (mmu_has_feature(MMU_FTR_1T_SEGMENT))
sp_vsid = get_kernel_vsid(sp, MMU_SEGSIZE_1T)
<< SLB_VSID_SHIFT_1T;
@@ -1924,7 +1924,8 @@
* the heap, we can put it above 1TB so it is backed by a 1TB
* segment. Otherwise the heap will be in the bottom 1TB
* which always uses 256MB segments and this may result in a
- * performance penalty.
+ * performance penalty. We don't need to worry about radix. For
+ * radix, mmu_highuser_ssize remains unchanged from 256MB.
*/
if (!is_32bit_task() && (mmu_highuser_ssize == MMU_SEGSIZE_1T))
base = max_t(unsigned long, mm->brk, 1UL << SID_SHIFT_1T);
diff --git a/arch/powerpc/kernel/prom.c b/arch/powerpc/kernel/prom.c
index a15fe1d..946e34ff 100644
--- a/arch/powerpc/kernel/prom.c
+++ b/arch/powerpc/kernel/prom.c
@@ -34,6 +34,7 @@
#include <linux/of.h>
#include <linux/of_fdt.h>
#include <linux/libfdt.h>
+#include <linux/cpu.h>
#include <asm/prom.h>
#include <asm/rtas.h>
@@ -167,6 +168,7 @@
*/
{CPU_FTR_TM_COMP, 0, 0,
PPC_FEATURE2_HTM_COMP|PPC_FEATURE2_HTM_NOSC_COMP, 22, 0, 0},
+ {0, MMU_FTR_RADIX, 0, 0, 40, 0, 0},
};
static void __init scan_features(unsigned long node, const unsigned char *ftrs,
diff --git a/arch/powerpc/kernel/rtasd.c b/arch/powerpc/kernel/rtasd.c
index aa610ce..c638e24 100644
--- a/arch/powerpc/kernel/rtasd.c
+++ b/arch/powerpc/kernel/rtasd.c
@@ -442,7 +442,7 @@
}
static void rtas_event_scan(struct work_struct *w);
-DECLARE_DELAYED_WORK(event_scan_work, rtas_event_scan);
+static DECLARE_DELAYED_WORK(event_scan_work, rtas_event_scan);
/*
* Delay should be at least one second since some machines have problems if
diff --git a/arch/powerpc/kernel/setup-common.c b/arch/powerpc/kernel/setup-common.c
index 44c8d03..8ca79b7 100644
--- a/arch/powerpc/kernel/setup-common.c
+++ b/arch/powerpc/kernel/setup-common.c
@@ -128,9 +128,7 @@
machine_shutdown();
if (ppc_md.restart)
ppc_md.restart(cmd);
-#ifdef CONFIG_SMP
smp_send_stop();
-#endif
printk(KERN_EMERG "System Halted, OK to turn off power\n");
local_irq_disable();
while (1) ;
@@ -141,9 +139,7 @@
machine_shutdown();
if (pm_power_off)
pm_power_off();
-#ifdef CONFIG_SMP
smp_send_stop();
-#endif
printk(KERN_EMERG "System Halted, OK to turn off power\n");
local_irq_disable();
while (1) ;
@@ -159,9 +155,7 @@
machine_shutdown();
if (ppc_md.halt)
ppc_md.halt();
-#ifdef CONFIG_SMP
smp_send_stop();
-#endif
printk(KERN_EMERG "System Halted, OK to turn off power\n");
local_irq_disable();
while (1) ;
diff --git a/arch/powerpc/kernel/swsusp.c b/arch/powerpc/kernel/swsusp.c
index 6669b17..6ae9bd5 100644
--- a/arch/powerpc/kernel/swsusp.c
+++ b/arch/powerpc/kernel/swsusp.c
@@ -31,6 +31,6 @@
void restore_processor_state(void)
{
#ifdef CONFIG_PPC32
- switch_mmu_context(current->active_mm, current->active_mm);
+ switch_mmu_context(current->active_mm, current->active_mm, NULL);
#endif
}
diff --git a/arch/powerpc/kernel/time.c b/arch/powerpc/kernel/time.c
index 81b0900..3ed9a5a 100644
--- a/arch/powerpc/kernel/time.c
+++ b/arch/powerpc/kernel/time.c
@@ -55,6 +55,7 @@
#include <linux/delay.h>
#include <linux/irq_work.h>
#include <linux/clk-provider.h>
+#include <linux/suspend.h>
#include <asm/trace.h>
#include <asm/io.h>
diff --git a/arch/powerpc/kernel/vdso.c b/arch/powerpc/kernel/vdso.c
index def1b8b..6767605 100644
--- a/arch/powerpc/kernel/vdso.c
+++ b/arch/powerpc/kernel/vdso.c
@@ -195,7 +195,8 @@
* and end up putting it elsewhere.
* Add enough to the size so that the result can be aligned.
*/
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
vdso_base = get_unmapped_area(NULL, vdso_base,
(vdso_pages << PAGE_SHIFT) +
((VDSO_ALIGNMENT - 1) & PAGE_MASK),
diff --git a/arch/powerpc/kernel/vio.c b/arch/powerpc/kernel/vio.c
index 5f8dcda..8d7358f 100644
--- a/arch/powerpc/kernel/vio.c
+++ b/arch/powerpc/kernel/vio.c
@@ -87,7 +87,7 @@
* @curr: bytes currently allocated
* @high: high water mark for IO data usage
*/
-struct vio_cmo {
+static struct vio_cmo {
spinlock_t lock;
struct delayed_work balance_q;
struct list_head device_list;
@@ -615,7 +615,7 @@
return dma_iommu_ops.get_required_mask(dev);
}
-struct dma_map_ops vio_dma_mapping_ops = {
+static struct dma_map_ops vio_dma_mapping_ops = {
.alloc = vio_dma_iommu_alloc_coherent,
.free = vio_dma_iommu_free_coherent,
.mmap = dma_direct_mmap_coherent,
diff --git a/arch/powerpc/kvm/book3s.c b/arch/powerpc/kvm/book3s.c
index b34220d..47018fc 100644
--- a/arch/powerpc/kvm/book3s.c
+++ b/arch/powerpc/kvm/book3s.c
@@ -54,6 +54,7 @@
{ "queue_intr", VCPU_STAT(queue_intr) },
{ "halt_successful_poll", VCPU_STAT(halt_successful_poll), },
{ "halt_attempted_poll", VCPU_STAT(halt_attempted_poll), },
+ { "halt_poll_invalid", VCPU_STAT(halt_poll_invalid) },
{ "halt_wakeup", VCPU_STAT(halt_wakeup) },
{ "pf_storage", VCPU_STAT(pf_storage) },
{ "sp_storage", VCPU_STAT(sp_storage) },
diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c
index c7b78d8..05f09ae 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c
@@ -447,7 +447,7 @@
struct revmap_entry *rev;
struct page *page, *pages[1];
long index, ret, npages;
- unsigned long is_io;
+ bool is_ci;
unsigned int writing, write_ok;
struct vm_area_struct *vma;
unsigned long rcbits;
@@ -503,7 +503,7 @@
smp_rmb();
ret = -EFAULT;
- is_io = 0;
+ is_ci = false;
pfn = 0;
page = NULL;
pte_size = PAGE_SIZE;
@@ -521,7 +521,7 @@
pfn = vma->vm_pgoff +
((hva - vma->vm_start) >> PAGE_SHIFT);
pte_size = psize;
- is_io = hpte_cache_bits(pgprot_val(vma->vm_page_prot));
+ is_ci = pte_ci(__pte((pgprot_val(vma->vm_page_prot))));
write_ok = vma->vm_flags & VM_WRITE;
}
up_read(¤t->mm->mmap_sem);
@@ -558,10 +558,9 @@
goto out_put;
/* Check WIMG vs. the actual page we're accessing */
- if (!hpte_cache_flags_ok(r, is_io)) {
- if (is_io)
+ if (!hpte_cache_flags_ok(r, is_ci)) {
+ if (is_ci)
goto out_put;
-
/*
* Allow guest to map emulated device memory as
* uncacheable, but actually make it cacheable.
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index 84fb4fc..e20beae 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -27,6 +27,7 @@
#include <linux/export.h>
#include <linux/fs.h>
#include <linux/anon_inodes.h>
+#include <linux/cpu.h>
#include <linux/cpumask.h>
#include <linux/spinlock.h>
#include <linux/page-flags.h>
@@ -3271,6 +3272,12 @@
if (!cpu_has_feature(CPU_FTR_HVMODE) ||
!cpu_has_feature(CPU_FTR_ARCH_206))
return -EIO;
+ /*
+ * Disable KVM for Power9, untill the required bits merged.
+ */
+ if (cpu_has_feature(CPU_FTR_ARCH_300))
+ return -EIO;
+
return 0;
}
diff --git a/arch/powerpc/kvm/book3s_hv_rm_mmu.c b/arch/powerpc/kvm/book3s_hv_rm_mmu.c
index 4cb8db05..99b4e9d 100644
--- a/arch/powerpc/kvm/book3s_hv_rm_mmu.c
+++ b/arch/powerpc/kvm/book3s_hv_rm_mmu.c
@@ -175,7 +175,7 @@
unsigned long g_ptel;
struct kvm_memory_slot *memslot;
unsigned hpage_shift;
- unsigned long is_io;
+ bool is_ci;
unsigned long *rmap;
pte_t *ptep;
unsigned int writing;
@@ -199,7 +199,7 @@
gfn = gpa >> PAGE_SHIFT;
memslot = __gfn_to_memslot(kvm_memslots_raw(kvm), gfn);
pa = 0;
- is_io = ~0ul;
+ is_ci = false;
rmap = NULL;
if (!(memslot && !(memslot->flags & KVM_MEMSLOT_INVALID))) {
/* Emulated MMIO - mark this with key=31 */
@@ -250,7 +250,7 @@
if (writing && !pte_write(pte))
/* make the actual HPTE be read-only */
ptel = hpte_make_readonly(ptel);
- is_io = hpte_cache_bits(pte_val(pte));
+ is_ci = pte_ci(pte);
pa = pte_pfn(pte) << PAGE_SHIFT;
pa |= hva & (host_pte_size - 1);
pa |= gpa & ~PAGE_MASK;
@@ -267,9 +267,9 @@
else
pteh |= HPTE_V_ABSENT;
- /* Check WIMG */
- if (is_io != ~0ul && !hpte_cache_flags_ok(ptel, is_io)) {
- if (is_io)
+ /*If we had host pte mapping then Check WIMG */
+ if (ptep && !hpte_cache_flags_ok(ptel, is_ci)) {
+ if (is_ci)
return H_PARAMETER;
/*
* Allow guest to map emulated device memory as
diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c
index 95bceca..8e4f64f 100644
--- a/arch/powerpc/kvm/book3s_pr.c
+++ b/arch/powerpc/kvm/book3s_pr.c
@@ -882,6 +882,24 @@
}
#endif
+static void kvmppc_setup_debug(struct kvm_vcpu *vcpu)
+{
+ if (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP) {
+ u64 msr = kvmppc_get_msr(vcpu);
+
+ kvmppc_set_msr(vcpu, msr | MSR_SE);
+ }
+}
+
+static void kvmppc_clear_debug(struct kvm_vcpu *vcpu)
+{
+ if (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP) {
+ u64 msr = kvmppc_get_msr(vcpu);
+
+ kvmppc_set_msr(vcpu, msr & ~MSR_SE);
+ }
+}
+
int kvmppc_handle_exit_pr(struct kvm_run *run, struct kvm_vcpu *vcpu,
unsigned int exit_nr)
{
@@ -1207,10 +1225,18 @@
break;
#endif
case BOOK3S_INTERRUPT_MACHINE_CHECK:
- case BOOK3S_INTERRUPT_TRACE:
kvmppc_book3s_queue_irqprio(vcpu, exit_nr);
r = RESUME_GUEST;
break;
+ case BOOK3S_INTERRUPT_TRACE:
+ if (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP) {
+ run->exit_reason = KVM_EXIT_DEBUG;
+ r = RESUME_HOST;
+ } else {
+ kvmppc_book3s_queue_irqprio(vcpu, exit_nr);
+ r = RESUME_GUEST;
+ }
+ break;
default:
{
ulong shadow_srr1 = vcpu->arch.shadow_srr1;
@@ -1479,6 +1505,8 @@
goto out;
}
+ kvmppc_setup_debug(vcpu);
+
/*
* Interrupts could be timers for the guest which we have to inject
* again, so let's postpone them until we're in the guest and if we
@@ -1501,6 +1529,8 @@
ret = __kvmppc_vcpu_run(kvm_run, vcpu);
+ kvmppc_clear_debug(vcpu);
+
/* No need for kvm_guest_exit. It's done in handle_exit.
We also get here with interrupts enabled. */
@@ -1683,7 +1713,11 @@
static int kvmppc_core_check_processor_compat_pr(void)
{
- /* we are always compatible */
+ /*
+ * Disable KVM for Power9 untill the required bits merged.
+ */
+ if (cpu_has_feature(CPU_FTR_ARCH_300))
+ return -EIO;
return 0;
}
diff --git a/arch/powerpc/kvm/book3s_xics.c b/arch/powerpc/kvm/book3s_xics.c
index 46871d5..a75ba38 100644
--- a/arch/powerpc/kvm/book3s_xics.c
+++ b/arch/powerpc/kvm/book3s_xics.c
@@ -92,7 +92,7 @@
* we are the only setter, thus concurrent access is undefined
* to begin with.
*/
- if (level == 1 || level == KVM_INTERRUPT_SET_LEVEL)
+ if ((level == 1 && state->lsi) || level == KVM_INTERRUPT_SET_LEVEL)
state->asserted = 1;
else if (level == 0 || level == KVM_INTERRUPT_UNSET) {
state->asserted = 0;
@@ -280,7 +280,7 @@
if (!success)
goto bail;
- XICS_DBG("UPD [%04x] - C:%02x M:%02x PP: %02x PI:%06x R:%d O:%d\n",
+ XICS_DBG("UPD [%04lx] - C:%02x M:%02x PP: %02x PI:%06x R:%d O:%d\n",
icp->server_num,
old.cppr, old.mfrr, old.pending_pri, old.xisr,
old.need_resend, old.out_ee);
@@ -336,7 +336,7 @@
union kvmppc_icp_state old_state, new_state;
bool success;
- XICS_DBG("try deliver %#x(P:%#x) to server %#x\n", irq, priority,
+ XICS_DBG("try deliver %#x(P:%#x) to server %#lx\n", irq, priority,
icp->server_num);
do {
@@ -1174,9 +1174,11 @@
prio = irqp->saved_priority;
}
val |= prio << KVM_XICS_PRIORITY_SHIFT;
- if (irqp->asserted)
- val |= KVM_XICS_LEVEL_SENSITIVE | KVM_XICS_PENDING;
- else if (irqp->masked_pending || irqp->resend)
+ if (irqp->lsi) {
+ val |= KVM_XICS_LEVEL_SENSITIVE;
+ if (irqp->asserted)
+ val |= KVM_XICS_PENDING;
+ } else if (irqp->masked_pending || irqp->resend)
val |= KVM_XICS_PENDING;
ret = 0;
}
@@ -1228,9 +1230,13 @@
irqp->priority = prio;
irqp->resend = 0;
irqp->masked_pending = 0;
+ irqp->lsi = 0;
irqp->asserted = 0;
- if ((val & KVM_XICS_PENDING) && (val & KVM_XICS_LEVEL_SENSITIVE))
- irqp->asserted = 1;
+ if (val & KVM_XICS_LEVEL_SENSITIVE) {
+ irqp->lsi = 1;
+ if (val & KVM_XICS_PENDING)
+ irqp->asserted = 1;
+ }
irqp->exists = 1;
arch_spin_unlock(&ics->lock);
local_irq_restore(flags);
@@ -1249,11 +1255,10 @@
return ics_deliver_irq(xics, irq, level);
}
-int kvm_set_msi(struct kvm_kernel_irq_routing_entry *irq_entry, struct kvm *kvm,
- int irq_source_id, int level, bool line_status)
+int kvm_arch_set_irq_inatomic(struct kvm_kernel_irq_routing_entry *irq_entry,
+ struct kvm *kvm, int irq_source_id,
+ int level, bool line_status)
{
- if (!level)
- return -1;
return kvm_set_irq(kvm, irq_source_id, irq_entry->gsi,
level, line_status);
}
diff --git a/arch/powerpc/kvm/book3s_xics.h b/arch/powerpc/kvm/book3s_xics.h
index 56ea44f..a46b954 100644
--- a/arch/powerpc/kvm/book3s_xics.h
+++ b/arch/powerpc/kvm/book3s_xics.h
@@ -39,6 +39,7 @@
u8 saved_priority;
u8 resend;
u8 masked_pending;
+ u8 lsi; /* level-sensitive interrupt */
u8 asserted; /* Only for LSI */
u8 exists;
};
diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c
index 4d66f44..4afae69 100644
--- a/arch/powerpc/kvm/booke.c
+++ b/arch/powerpc/kvm/booke.c
@@ -64,6 +64,7 @@
{ "ext_intr", VCPU_STAT(ext_intr_exits) },
{ "halt_successful_poll", VCPU_STAT(halt_successful_poll) },
{ "halt_attempted_poll", VCPU_STAT(halt_attempted_poll) },
+ { "halt_poll_invalid", VCPU_STAT(halt_poll_invalid) },
{ "halt_wakeup", VCPU_STAT(halt_wakeup) },
{ "doorbell", VCPU_STAT(dbell_exits) },
{ "guest doorbell", VCPU_STAT(gdbell_exits) },
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index 6a68730..02416fe 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -800,9 +800,9 @@
}
}
-int kvmppc_handle_load(struct kvm_run *run, struct kvm_vcpu *vcpu,
- unsigned int rt, unsigned int bytes,
- int is_default_endian)
+static int __kvmppc_handle_load(struct kvm_run *run, struct kvm_vcpu *vcpu,
+ unsigned int rt, unsigned int bytes,
+ int is_default_endian, int sign_extend)
{
int idx, ret;
bool host_swabbed;
@@ -827,7 +827,7 @@
vcpu->arch.mmio_host_swabbed = host_swabbed;
vcpu->mmio_needed = 1;
vcpu->mmio_is_write = 0;
- vcpu->arch.mmio_sign_extend = 0;
+ vcpu->arch.mmio_sign_extend = sign_extend;
idx = srcu_read_lock(&vcpu->kvm->srcu);
@@ -844,6 +844,13 @@
return EMULATE_DO_MMIO;
}
+
+int kvmppc_handle_load(struct kvm_run *run, struct kvm_vcpu *vcpu,
+ unsigned int rt, unsigned int bytes,
+ int is_default_endian)
+{
+ return __kvmppc_handle_load(run, vcpu, rt, bytes, is_default_endian, 0);
+}
EXPORT_SYMBOL_GPL(kvmppc_handle_load);
/* Same as above, but sign extends */
@@ -851,12 +858,7 @@
unsigned int rt, unsigned int bytes,
int is_default_endian)
{
- int r;
-
- vcpu->arch.mmio_sign_extend = 1;
- r = kvmppc_handle_load(run, vcpu, rt, bytes, is_default_endian);
-
- return r;
+ return __kvmppc_handle_load(run, vcpu, rt, bytes, is_default_endian, 1);
}
int kvmppc_handle_store(struct kvm_run *run, struct kvm_vcpu *vcpu,
diff --git a/arch/powerpc/lib/copy_32.S b/arch/powerpc/lib/copy_32.S
index c44df2d..99f37f2 100644
--- a/arch/powerpc/lib/copy_32.S
+++ b/arch/powerpc/lib/copy_32.S
@@ -217,7 +217,7 @@
bdnz 40b
65: blr
-_GLOBAL(generic_memcpy)
+generic_memcpy:
srwi. r7,r5,3
addi r6,r3,-4
addi r4,r4,-4
diff --git a/arch/powerpc/lib/sstep.c b/arch/powerpc/lib/sstep.c
index dc885b3..3362299 100644
--- a/arch/powerpc/lib/sstep.c
+++ b/arch/powerpc/lib/sstep.c
@@ -925,6 +925,7 @@
}
}
#endif
+ break; /* illegal instruction */
case 31:
switch ((instr >> 1) & 0x3ff) {
@@ -1818,9 +1819,11 @@
case 4:
__get_user_asmx(val, op.ea, err, "lwarx");
break;
+#ifdef __powerpc64__
case 8:
__get_user_asmx(val, op.ea, err, "ldarx");
break;
+#endif
default:
return 0;
}
@@ -1841,9 +1844,11 @@
case 4:
__put_user_asmx(op.val, op.ea, err, "stwcx.", cr);
break;
+#ifdef __powerpc64__
case 8:
__put_user_asmx(op.val, op.ea, err, "stdcx.", cr);
break;
+#endif
default:
return 0;
}
diff --git a/arch/powerpc/lib/xor_vmx.c b/arch/powerpc/lib/xor_vmx.c
index 07f49f1..f9de69a 100644
--- a/arch/powerpc/lib/xor_vmx.c
+++ b/arch/powerpc/lib/xor_vmx.c
@@ -17,7 +17,17 @@
*
* Author: Anton Blanchard <anton@au.ibm.com>
*/
+
+/*
+ * Sparse (as at v0.5.0) gets very, very confused by this file.
+ * Make it a bit simpler for it.
+ */
+#if !defined(__CHECKER__)
#include <altivec.h>
+#else
+#define vec_xor(a, b) a ^ b
+#define vector __attribute__((vector_size(16)))
+#endif
#include <linux/preempt.h>
#include <linux/export.h>
diff --git a/arch/powerpc/mm/Makefile b/arch/powerpc/mm/Makefile
index adfee3f..f2cea6d 100644
--- a/arch/powerpc/mm/Makefile
+++ b/arch/powerpc/mm/Makefile
@@ -13,10 +13,11 @@
tlb_nohash_low.o
obj-$(CONFIG_PPC_BOOK3E) += tlb_low_$(CONFIG_WORD_SIZE)e.o
hash64-$(CONFIG_PPC_NATIVE) := hash_native_64.o
-obj-$(CONFIG_PPC_STD_MMU_64) += hash_utils_64.o slb_low.o slb.o $(hash64-y)
-obj-$(CONFIG_PPC_STD_MMU_32) += ppc_mmu_32.o hash_low_32.o
-obj-$(CONFIG_PPC_STD_MMU) += tlb_hash$(CONFIG_WORD_SIZE).o \
- mmu_context_hash$(CONFIG_WORD_SIZE).o
+obj-$(CONFIG_PPC_BOOK3E_64) += pgtable-book3e.o
+obj-$(CONFIG_PPC_STD_MMU_64) += pgtable-hash64.o hash_utils_64.o slb_low.o slb.o $(hash64-y) mmu_context_book3s64.o pgtable-book3s64.o
+obj-$(CONFIG_PPC_RADIX_MMU) += pgtable-radix.o tlb-radix.o
+obj-$(CONFIG_PPC_STD_MMU_32) += ppc_mmu_32.o hash_low_32.o mmu_context_hash32.o
+obj-$(CONFIG_PPC_STD_MMU) += tlb_hash$(CONFIG_WORD_SIZE).o
ifeq ($(CONFIG_PPC_STD_MMU_64),y)
obj-$(CONFIG_PPC_4K_PAGES) += hash64_4k.o
obj-$(CONFIG_PPC_64K_PAGES) += hash64_64k.o
@@ -33,6 +34,7 @@
obj-y += hugetlbpage.o
ifeq ($(CONFIG_HUGETLB_PAGE),y)
obj-$(CONFIG_PPC_STD_MMU_64) += hugetlbpage-hash64.o
+obj-$(CONFIG_PPC_RADIX_MMU) += hugetlbpage-radix.o
obj-$(CONFIG_PPC_BOOK3E_MMU) += hugetlbpage-book3e.o
endif
obj-$(CONFIG_TRANSPARENT_HUGEPAGE) += hugepage-hash64.o
diff --git a/arch/powerpc/mm/fsl_booke_mmu.c b/arch/powerpc/mm/fsl_booke_mmu.c
index a1b2713..139dec4 100644
--- a/arch/powerpc/mm/fsl_booke_mmu.c
+++ b/arch/powerpc/mm/fsl_booke_mmu.c
@@ -135,7 +135,7 @@
TLBCAM[index].MAS7 = (u64)phys >> 32;
/* Below is unlikely -- only for large user pages or similar */
- if (pte_user(flags)) {
+ if (pte_user(__pte(flags))) {
TLBCAM[index].MAS3 |= MAS3_UX | MAS3_UR;
TLBCAM[index].MAS3 |= ((flags & _PAGE_RW) ? MAS3_UW : 0);
}
diff --git a/arch/powerpc/mm/hash64_4k.c b/arch/powerpc/mm/hash64_4k.c
index 47d1b26..6333b27 100644
--- a/arch/powerpc/mm/hash64_4k.c
+++ b/arch/powerpc/mm/hash64_4k.c
@@ -34,21 +34,21 @@
old_pte = pte_val(pte);
/* If PTE busy, retry the access */
- if (unlikely(old_pte & _PAGE_BUSY))
+ if (unlikely(old_pte & H_PAGE_BUSY))
return 0;
/* If PTE permissions don't match, take page fault */
- if (unlikely(access & ~old_pte))
+ if (unlikely(!check_pte_access(access, old_pte)))
return 1;
/*
* Try to lock the PTE, add ACCESSED and DIRTY if it was
* a write access. Since this is 4K insert of 64K page size
- * also add _PAGE_COMBO
+ * also add H_PAGE_COMBO
*/
- new_pte = old_pte | _PAGE_BUSY | _PAGE_ACCESSED;
- if (access & _PAGE_RW)
+ new_pte = old_pte | H_PAGE_BUSY | _PAGE_ACCESSED;
+ if (access & _PAGE_WRITE)
new_pte |= _PAGE_DIRTY;
- } while (old_pte != __cmpxchg_u64((unsigned long *)ptep,
- old_pte, new_pte));
+ } while (!pte_xchg(ptep, __pte(old_pte), __pte(new_pte)));
+
/*
* PP bits. _PAGE_USER is already PP bit 0x2, so we only
* need to add in 0x1 if it's a read-only user page
@@ -60,22 +60,22 @@
rflags = hash_page_do_lazy_icache(rflags, __pte(old_pte), trap);
vpn = hpt_vpn(ea, vsid, ssize);
- if (unlikely(old_pte & _PAGE_HASHPTE)) {
+ if (unlikely(old_pte & H_PAGE_HASHPTE)) {
/*
* There MIGHT be an HPTE for this pte
*/
hash = hpt_hash(vpn, shift, ssize);
- if (old_pte & _PAGE_F_SECOND)
+ if (old_pte & H_PAGE_F_SECOND)
hash = ~hash;
slot = (hash & htab_hash_mask) * HPTES_PER_GROUP;
- slot += (old_pte & _PAGE_F_GIX) >> _PAGE_F_GIX_SHIFT;
+ slot += (old_pte & H_PAGE_F_GIX) >> H_PAGE_F_GIX_SHIFT;
if (ppc_md.hpte_updatepp(slot, rflags, vpn, MMU_PAGE_4K,
MMU_PAGE_4K, ssize, flags) == -1)
old_pte &= ~_PAGE_HPTEFLAGS;
}
- if (likely(!(old_pte & _PAGE_HASHPTE))) {
+ if (likely(!(old_pte & H_PAGE_HASHPTE))) {
pa = pte_pfn(__pte(old_pte)) << PAGE_SHIFT;
hash = hpt_hash(vpn, shift, ssize);
@@ -115,9 +115,10 @@
MMU_PAGE_4K, MMU_PAGE_4K, old_pte);
return -1;
}
- new_pte = (new_pte & ~_PAGE_HPTEFLAGS) | _PAGE_HASHPTE;
- new_pte |= (slot << _PAGE_F_GIX_SHIFT) & (_PAGE_F_SECOND | _PAGE_F_GIX);
+ new_pte = (new_pte & ~_PAGE_HPTEFLAGS) | H_PAGE_HASHPTE;
+ new_pte |= (slot << H_PAGE_F_GIX_SHIFT) &
+ (H_PAGE_F_SECOND | H_PAGE_F_GIX);
}
- *ptep = __pte(new_pte & ~_PAGE_BUSY);
+ *ptep = __pte(new_pte & ~H_PAGE_BUSY);
return 0;
}
diff --git a/arch/powerpc/mm/hash64_64k.c b/arch/powerpc/mm/hash64_64k.c
index b2d659c..16644e1 100644
--- a/arch/powerpc/mm/hash64_64k.c
+++ b/arch/powerpc/mm/hash64_64k.c
@@ -23,7 +23,7 @@
unsigned long g_idx;
unsigned long ptev = pte_val(rpte.pte);
- g_idx = (ptev & _PAGE_COMBO_VALID) >> _PAGE_F_GIX_SHIFT;
+ g_idx = (ptev & H_PAGE_COMBO_VALID) >> H_PAGE_F_GIX_SHIFT;
index = index >> 2;
if (g_idx & (0x1 << index))
return true;
@@ -37,12 +37,12 @@
{
unsigned long g_idx;
- if (!(ptev & _PAGE_COMBO))
+ if (!(ptev & H_PAGE_COMBO))
return ptev;
index = index >> 2;
g_idx = 0x1 << index;
- return ptev | (g_idx << _PAGE_F_GIX_SHIFT);
+ return ptev | (g_idx << H_PAGE_F_GIX_SHIFT);
}
int __hash_page_4K(unsigned long ea, unsigned long access, unsigned long vsid,
@@ -66,21 +66,21 @@
old_pte = pte_val(pte);
/* If PTE busy, retry the access */
- if (unlikely(old_pte & _PAGE_BUSY))
+ if (unlikely(old_pte & H_PAGE_BUSY))
return 0;
/* If PTE permissions don't match, take page fault */
- if (unlikely(access & ~old_pte))
+ if (unlikely(!check_pte_access(access, old_pte)))
return 1;
/*
* Try to lock the PTE, add ACCESSED and DIRTY if it was
* a write access. Since this is 4K insert of 64K page size
- * also add _PAGE_COMBO
+ * also add H_PAGE_COMBO
*/
- new_pte = old_pte | _PAGE_BUSY | _PAGE_ACCESSED | _PAGE_COMBO;
- if (access & _PAGE_RW)
+ new_pte = old_pte | H_PAGE_BUSY | _PAGE_ACCESSED | H_PAGE_COMBO;
+ if (access & _PAGE_WRITE)
new_pte |= _PAGE_DIRTY;
- } while (old_pte != __cmpxchg_u64((unsigned long *)ptep,
- old_pte, new_pte));
+ } while (!pte_xchg(ptep, __pte(old_pte), __pte(new_pte)));
+
/*
* Handle the subpage protection bits
*/
@@ -103,21 +103,21 @@
/*
*None of the sub 4k page is hashed
*/
- if (!(old_pte & _PAGE_HASHPTE))
+ if (!(old_pte & H_PAGE_HASHPTE))
goto htab_insert_hpte;
/*
* Check if the pte was already inserted into the hash table
* as a 64k HW page, and invalidate the 64k HPTE if so.
*/
- if (!(old_pte & _PAGE_COMBO)) {
+ if (!(old_pte & H_PAGE_COMBO)) {
flush_hash_page(vpn, rpte, MMU_PAGE_64K, ssize, flags);
/*
* clear the old slot details from the old and new pte.
* On hash insert failure we use old pte value and we don't
* want slot information there if we have a insert failure.
*/
- old_pte &= ~(_PAGE_HASHPTE | _PAGE_F_GIX | _PAGE_F_SECOND);
- new_pte &= ~(_PAGE_HASHPTE | _PAGE_F_GIX | _PAGE_F_SECOND);
+ old_pte &= ~(H_PAGE_HASHPTE | H_PAGE_F_GIX | H_PAGE_F_SECOND);
+ new_pte &= ~(H_PAGE_HASHPTE | H_PAGE_F_GIX | H_PAGE_F_SECOND);
goto htab_insert_hpte;
}
/*
@@ -143,15 +143,15 @@
if (ret == -1)
goto htab_insert_hpte;
- *ptep = __pte(new_pte & ~_PAGE_BUSY);
+ *ptep = __pte(new_pte & ~H_PAGE_BUSY);
return 0;
}
htab_insert_hpte:
/*
- * handle _PAGE_4K_PFN case
+ * handle H_PAGE_4K_PFN case
*/
- if (old_pte & _PAGE_4K_PFN) {
+ if (old_pte & H_PAGE_4K_PFN) {
/*
* All the sub 4k page have the same
* physical address.
@@ -199,20 +199,20 @@
}
/*
* Insert slot number & secondary bit in PTE second half,
- * clear _PAGE_BUSY and set appropriate HPTE slot bit
- * Since we have _PAGE_BUSY set on ptep, we can be sure
+ * clear H_PAGE_BUSY and set appropriate HPTE slot bit
+ * Since we have H_PAGE_BUSY set on ptep, we can be sure
* nobody is undating hidx.
*/
hidxp = (unsigned long *)(ptep + PTRS_PER_PTE);
rpte.hidx &= ~(0xfUL << (subpg_index << 2));
*hidxp = rpte.hidx | (slot << (subpg_index << 2));
new_pte = mark_subptegroup_valid(new_pte, subpg_index);
- new_pte |= _PAGE_HASHPTE;
+ new_pte |= H_PAGE_HASHPTE;
/*
* check __real_pte for details on matching smp_rmb()
*/
smp_wmb();
- *ptep = __pte(new_pte & ~_PAGE_BUSY);
+ *ptep = __pte(new_pte & ~H_PAGE_BUSY);
return 0;
}
@@ -220,7 +220,6 @@
unsigned long vsid, pte_t *ptep, unsigned long trap,
unsigned long flags, int ssize)
{
-
unsigned long hpte_group;
unsigned long rflags, pa;
unsigned long old_pte, new_pte;
@@ -235,27 +234,26 @@
old_pte = pte_val(pte);
/* If PTE busy, retry the access */
- if (unlikely(old_pte & _PAGE_BUSY))
+ if (unlikely(old_pte & H_PAGE_BUSY))
return 0;
/* If PTE permissions don't match, take page fault */
- if (unlikely(access & ~old_pte))
+ if (unlikely(!check_pte_access(access, old_pte)))
return 1;
/*
* Check if PTE has the cache-inhibit bit set
* If so, bail out and refault as a 4k page
*/
if (!mmu_has_feature(MMU_FTR_CI_LARGE_PAGE) &&
- unlikely(old_pte & _PAGE_NO_CACHE))
+ unlikely(pte_ci(pte)))
return 0;
/*
* Try to lock the PTE, add ACCESSED and DIRTY if it was
* a write access.
*/
- new_pte = old_pte | _PAGE_BUSY | _PAGE_ACCESSED;
- if (access & _PAGE_RW)
+ new_pte = old_pte | H_PAGE_BUSY | _PAGE_ACCESSED;
+ if (access & _PAGE_WRITE)
new_pte |= _PAGE_DIRTY;
- } while (old_pte != __cmpxchg_u64((unsigned long *)ptep,
- old_pte, new_pte));
+ } while (!pte_xchg(ptep, __pte(old_pte), __pte(new_pte)));
rflags = htab_convert_pte_flags(new_pte);
@@ -264,22 +262,22 @@
rflags = hash_page_do_lazy_icache(rflags, __pte(old_pte), trap);
vpn = hpt_vpn(ea, vsid, ssize);
- if (unlikely(old_pte & _PAGE_HASHPTE)) {
+ if (unlikely(old_pte & H_PAGE_HASHPTE)) {
/*
* There MIGHT be an HPTE for this pte
*/
hash = hpt_hash(vpn, shift, ssize);
- if (old_pte & _PAGE_F_SECOND)
+ if (old_pte & H_PAGE_F_SECOND)
hash = ~hash;
slot = (hash & htab_hash_mask) * HPTES_PER_GROUP;
- slot += (old_pte & _PAGE_F_GIX) >> _PAGE_F_GIX_SHIFT;
+ slot += (old_pte & H_PAGE_F_GIX) >> H_PAGE_F_GIX_SHIFT;
if (ppc_md.hpte_updatepp(slot, rflags, vpn, MMU_PAGE_64K,
MMU_PAGE_64K, ssize, flags) == -1)
old_pte &= ~_PAGE_HPTEFLAGS;
}
- if (likely(!(old_pte & _PAGE_HASHPTE))) {
+ if (likely(!(old_pte & H_PAGE_HASHPTE))) {
pa = pte_pfn(__pte(old_pte)) << PAGE_SHIFT;
hash = hpt_hash(vpn, shift, ssize);
@@ -319,9 +317,10 @@
MMU_PAGE_64K, MMU_PAGE_64K, old_pte);
return -1;
}
- new_pte = (new_pte & ~_PAGE_HPTEFLAGS) | _PAGE_HASHPTE;
- new_pte |= (slot << _PAGE_F_GIX_SHIFT) & (_PAGE_F_SECOND | _PAGE_F_GIX);
+ new_pte = (new_pte & ~_PAGE_HPTEFLAGS) | H_PAGE_HASHPTE;
+ new_pte |= (slot << H_PAGE_F_GIX_SHIFT) &
+ (H_PAGE_F_SECOND | H_PAGE_F_GIX);
}
- *ptep = __pte(new_pte & ~_PAGE_BUSY);
+ *ptep = __pte(new_pte & ~H_PAGE_BUSY);
return 0;
}
diff --git a/arch/powerpc/mm/hash_native_64.c b/arch/powerpc/mm/hash_native_64.c
index 8eaac81..d873f65 100644
--- a/arch/powerpc/mm/hash_native_64.c
+++ b/arch/powerpc/mm/hash_native_64.c
@@ -221,7 +221,7 @@
return -1;
hpte_v = hpte_encode_v(vpn, psize, apsize, ssize) | vflags | HPTE_V_VALID;
- hpte_r = hpte_encode_r(pa, psize, apsize) | rflags;
+ hpte_r = hpte_encode_r(pa, psize, apsize, ssize) | rflags;
if (!(vflags & HPTE_V_BOLTED)) {
DBG_LOW(" i=%x hpte_v=%016lx, hpte_r=%016lx\n",
@@ -719,6 +719,12 @@
local_irq_restore(flags);
}
+static int native_update_partition_table(u64 patb1)
+{
+ partition_tb->patb1 = cpu_to_be64(patb1);
+ return 0;
+}
+
void __init hpte_init_native(void)
{
ppc_md.hpte_invalidate = native_hpte_invalidate;
@@ -729,4 +735,7 @@
ppc_md.hpte_clear_all = native_hpte_clear;
ppc_md.flush_hash_range = native_flush_hash_range;
ppc_md.hugepage_invalidate = native_hugepage_invalidate;
+
+ if (cpu_has_feature(CPU_FTR_ARCH_300))
+ ppc_md.update_partition_table = native_update_partition_table;
}
diff --git a/arch/powerpc/mm/hash_utils_64.c b/arch/powerpc/mm/hash_utils_64.c
index 7635b1c..5926896 100644
--- a/arch/powerpc/mm/hash_utils_64.c
+++ b/arch/powerpc/mm/hash_utils_64.c
@@ -167,16 +167,22 @@
if ((pteflags & _PAGE_EXEC) == 0)
rflags |= HPTE_R_N;
/*
- * PP bits:
+ * PPP bits:
* Linux uses slb key 0 for kernel and 1 for user.
- * kernel areas are mapped with PP=00
- * and there is no kernel RO (_PAGE_KERNEL_RO).
- * User area is mapped with PP=0x2 for read/write
- * or PP=0x3 for read-only (including writeable but clean pages).
+ * kernel RW areas are mapped with PPP=0b000
+ * User area is mapped with PPP=0b010 for read/write
+ * or PPP=0b011 for read-only (including writeable but clean pages).
*/
- if (pteflags & _PAGE_USER) {
- rflags |= 0x2;
- if (!((pteflags & _PAGE_RW) && (pteflags & _PAGE_DIRTY)))
+ if (pteflags & _PAGE_PRIVILEGED) {
+ /*
+ * Kernel read only mapped with ppp bits 0b110
+ */
+ if (!(pteflags & _PAGE_WRITE))
+ rflags |= (HPTE_R_PP0 | 0x2);
+ } else {
+ if (pteflags & _PAGE_RWX)
+ rflags |= 0x2;
+ if (!((pteflags & _PAGE_WRITE) && (pteflags & _PAGE_DIRTY)))
rflags |= 0x1;
}
/*
@@ -186,12 +192,13 @@
/*
* Add in WIG bits
*/
- if (pteflags & _PAGE_WRITETHRU)
- rflags |= HPTE_R_W;
- if (pteflags & _PAGE_NO_CACHE)
+
+ if ((pteflags & _PAGE_CACHE_CTL) == _PAGE_TOLERANT)
rflags |= HPTE_R_I;
- if (pteflags & _PAGE_GUARDED)
- rflags |= HPTE_R_G;
+ if ((pteflags & _PAGE_CACHE_CTL ) == _PAGE_NON_IDEMPOTENT)
+ rflags |= (HPTE_R_I | HPTE_R_G);
+ if ((pteflags & _PAGE_CACHE_CTL) == _PAGE_SAO)
+ rflags |= (HPTE_R_I | HPTE_R_W);
return rflags;
}
@@ -669,6 +676,41 @@
}
#endif /* CONFIG_MEMORY_HOTPLUG */
+static void __init hash_init_partition_table(phys_addr_t hash_table,
+ unsigned long pteg_count)
+{
+ unsigned long ps_field;
+ unsigned long htab_size;
+ unsigned long patb_size = 1UL << PATB_SIZE_SHIFT;
+
+ /*
+ * slb llp encoding for the page size used in VPM real mode.
+ * We can ignore that for lpid 0
+ */
+ ps_field = 0;
+ htab_size = __ilog2(pteg_count) - 11;
+
+ BUILD_BUG_ON_MSG((PATB_SIZE_SHIFT > 24), "Partition table size too large.");
+ partition_tb = __va(memblock_alloc_base(patb_size, patb_size,
+ MEMBLOCK_ALLOC_ANYWHERE));
+
+ /* Initialize the Partition Table with no entries */
+ memset((void *)partition_tb, 0, patb_size);
+ partition_tb->patb0 = cpu_to_be64(ps_field | hash_table | htab_size);
+ /*
+ * FIXME!! This should be done via update_partition table
+ * For now UPRT is 0 for us.
+ */
+ partition_tb->patb1 = 0;
+ DBG("Partition table %p\n", partition_tb);
+ /*
+ * update partition table control register,
+ * 64 K size.
+ */
+ mtspr(SPRN_PTCR, __pa(partition_tb) | (PATB_SIZE_SHIFT - 12));
+
+}
+
static void __init htab_initialize(void)
{
unsigned long table;
@@ -737,8 +779,11 @@
/* Initialize the HPT with no entries */
memset((void *)table, 0, htab_size_bytes);
- /* Set SDR1 */
- mtspr(SPRN_SDR1, _SDR1);
+ if (!cpu_has_feature(CPU_FTR_ARCH_300))
+ /* Set SDR1 */
+ mtspr(SPRN_SDR1, _SDR1);
+ else
+ hash_init_partition_table(table, pteg_count);
}
prot = pgprot_val(PAGE_KERNEL);
@@ -823,8 +868,38 @@
#undef KB
#undef MB
-void __init early_init_mmu(void)
+void __init hash__early_init_mmu(void)
{
+ /*
+ * initialize page table size
+ */
+ __pte_frag_nr = H_PTE_FRAG_NR;
+ __pte_frag_size_shift = H_PTE_FRAG_SIZE_SHIFT;
+
+ __pte_index_size = H_PTE_INDEX_SIZE;
+ __pmd_index_size = H_PMD_INDEX_SIZE;
+ __pud_index_size = H_PUD_INDEX_SIZE;
+ __pgd_index_size = H_PGD_INDEX_SIZE;
+ __pmd_cache_index = H_PMD_CACHE_INDEX;
+ __pte_table_size = H_PTE_TABLE_SIZE;
+ __pmd_table_size = H_PMD_TABLE_SIZE;
+ __pud_table_size = H_PUD_TABLE_SIZE;
+ __pgd_table_size = H_PGD_TABLE_SIZE;
+ /*
+ * 4k use hugepd format, so for hash set then to
+ * zero
+ */
+ __pmd_val_bits = 0;
+ __pud_val_bits = 0;
+ __pgd_val_bits = 0;
+
+ __kernel_virt_start = H_KERN_VIRT_START;
+ __kernel_virt_size = H_KERN_VIRT_SIZE;
+ __vmalloc_start = H_VMALLOC_START;
+ __vmalloc_end = H_VMALLOC_END;
+ vmemmap = (struct page *)H_VMEMMAP_BASE;
+ ioremap_bot = IOREMAP_BASE;
+
/* Initialize the MMU Hash table and create the linear mapping
* of memory. Has to be done before SLB initialization as this is
* currently where the page size encoding is obtained.
@@ -836,12 +911,16 @@
}
#ifdef CONFIG_SMP
-void early_init_mmu_secondary(void)
+void hash__early_init_mmu_secondary(void)
{
/* Initialize hash table for that CPU */
- if (!firmware_has_feature(FW_FEATURE_LPAR))
- mtspr(SPRN_SDR1, _SDR1);
-
+ if (!firmware_has_feature(FW_FEATURE_LPAR)) {
+ if (!cpu_has_feature(CPU_FTR_ARCH_300))
+ mtspr(SPRN_SDR1, _SDR1);
+ else
+ mtspr(SPRN_PTCR,
+ __pa(partition_tb) | (PATB_SIZE_SHIFT - 12));
+ }
/* Initialize SLB */
slb_initialize();
}
@@ -920,7 +999,7 @@
* Userspace sets the subpage permissions using the subpage_prot system call.
*
* Result is 0: full permissions, _PAGE_RW: read-only,
- * _PAGE_USER or _PAGE_USER|_PAGE_RW: no access.
+ * _PAGE_RWX: no access.
*/
static int subpage_protection(struct mm_struct *mm, unsigned long ea)
{
@@ -946,8 +1025,13 @@
/* extract 2-bit bitfield for this 4k subpage */
spp >>= 30 - 2 * ((ea >> 12) & 0xf);
- /* turn 0,1,2,3 into combination of _PAGE_USER and _PAGE_RW */
- spp = ((spp & 2) ? _PAGE_USER : 0) | ((spp & 1) ? _PAGE_RW : 0);
+ /*
+ * 0 -> full premission
+ * 1 -> Read only
+ * 2 -> no access.
+ * We return the flag that need to be cleared.
+ */
+ spp = ((spp & 2) ? _PAGE_RWX : 0) | ((spp & 1) ? _PAGE_WRITE : 0);
return spp;
}
@@ -1084,7 +1168,7 @@
/* Pre-check access permissions (will be re-checked atomically
* in __hash_page_XX but this pre-check is a fast path
*/
- if (access & ~pte_val(*ptep)) {
+ if (!check_pte_access(access, pte_val(*ptep))) {
DBG_LOW(" no access !\n");
rc = 1;
goto bail;
@@ -1122,8 +1206,8 @@
#endif
/* Do actual hashing */
#ifdef CONFIG_PPC_64K_PAGES
- /* If _PAGE_4K_PFN is set, make sure this is a 4k segment */
- if ((pte_val(*ptep) & _PAGE_4K_PFN) && psize == MMU_PAGE_64K) {
+ /* If H_PAGE_4K_PFN is set, make sure this is a 4k segment */
+ if ((pte_val(*ptep) & H_PAGE_4K_PFN) && psize == MMU_PAGE_64K) {
demote_segment_4k(mm, ea);
psize = MMU_PAGE_4K;
}
@@ -1131,8 +1215,7 @@
/* If this PTE is non-cacheable and we have restrictions on
* using non cacheable large pages, then we switch to 4k
*/
- if (mmu_ci_restrictions && psize == MMU_PAGE_64K &&
- (pte_val(*ptep) & _PAGE_NO_CACHE)) {
+ if (mmu_ci_restrictions && psize == MMU_PAGE_64K && pte_ci(*ptep)) {
if (user_region) {
demote_segment_4k(mm, ea);
psize = MMU_PAGE_4K;
@@ -1209,7 +1292,7 @@
int __hash_page(unsigned long ea, unsigned long msr, unsigned long trap,
unsigned long dsisr)
{
- unsigned long access = _PAGE_PRESENT;
+ unsigned long access = _PAGE_PRESENT | _PAGE_READ;
unsigned long flags = 0;
struct mm_struct *mm = current->mm;
@@ -1220,14 +1303,18 @@
flags |= HPTE_NOHPTE_UPDATE;
if (dsisr & DSISR_ISSTORE)
- access |= _PAGE_RW;
+ access |= _PAGE_WRITE;
/*
- * We need to set the _PAGE_USER bit if MSR_PR is set or if we are
- * accessing a userspace segment (even from the kernel). We assume
- * kernel addresses always have the high bit set.
+ * We set _PAGE_PRIVILEGED only when
+ * kernel mode access kernel space.
+ *
+ * _PAGE_PRIVILEGED is NOT set
+ * 1) when kernel mode access user space
+ * 2) user space access kernel space.
*/
+ access |= _PAGE_PRIVILEGED;
if ((msr & MSR_PR) || (REGION_ID(ea) == USER_REGION_ID))
- access |= _PAGE_USER;
+ access &= ~_PAGE_PRIVILEGED;
if (trap == 0x400)
access |= _PAGE_EXEC;
@@ -1235,6 +1322,30 @@
return hash_page_mm(mm, ea, access, trap, flags);
}
+#ifdef CONFIG_PPC_MM_SLICES
+static bool should_hash_preload(struct mm_struct *mm, unsigned long ea)
+{
+ int psize = get_slice_psize(mm, ea);
+
+ /* We only prefault standard pages for now */
+ if (unlikely(psize != mm->context.user_psize))
+ return false;
+
+ /*
+ * Don't prefault if subpage protection is enabled for the EA.
+ */
+ if (unlikely((psize == MMU_PAGE_4K) && subpage_protection(mm, ea)))
+ return false;
+
+ return true;
+}
+#else
+static bool should_hash_preload(struct mm_struct *mm, unsigned long ea)
+{
+ return true;
+}
+#endif
+
void hash_preload(struct mm_struct *mm, unsigned long ea,
unsigned long access, unsigned long trap)
{
@@ -1247,11 +1358,8 @@
BUG_ON(REGION_ID(ea) != USER_REGION_ID);
-#ifdef CONFIG_PPC_MM_SLICES
- /* We only prefault standard pages for now */
- if (unlikely(get_slice_psize(mm, ea) != mm->context.user_psize))
+ if (!should_hash_preload(mm, ea))
return;
-#endif
DBG_LOW("hash_preload(mm=%p, mm->pgdir=%p, ea=%016lx, access=%lx,"
" trap=%lx\n", mm, mm->pgd, ea, access, trap);
@@ -1282,13 +1390,13 @@
WARN_ON(hugepage_shift);
#ifdef CONFIG_PPC_64K_PAGES
- /* If either _PAGE_4K_PFN or _PAGE_NO_CACHE is set (and we are on
+ /* If either H_PAGE_4K_PFN or cache inhibited is set (and we are on
* a 64K kernel), then we don't preload, hash_page() will take
* care of it once we actually try to access the page.
* That way we don't have to duplicate all of the logic for segment
* page size demotion here
*/
- if (pte_val(*ptep) & (_PAGE_4K_PFN | _PAGE_NO_CACHE))
+ if ((pte_val(*ptep) & H_PAGE_4K_PFN) || pte_ci(*ptep))
goto out_exit;
#endif /* CONFIG_PPC_64K_PAGES */
@@ -1570,7 +1678,7 @@
}
#endif /* CONFIG_DEBUG_PAGEALLOC */
-void setup_initial_memory_limit(phys_addr_t first_memblock_base,
+void hash__setup_initial_memory_limit(phys_addr_t first_memblock_base,
phys_addr_t first_memblock_size)
{
/* We don't currently support the first MEMBLOCK not mapping 0
diff --git a/arch/powerpc/mm/hugepage-hash64.c b/arch/powerpc/mm/hugepage-hash64.c
index eb2accd..ba3fc22 100644
--- a/arch/powerpc/mm/hugepage-hash64.c
+++ b/arch/powerpc/mm/hugepage-hash64.c
@@ -37,20 +37,20 @@
old_pmd = pmd_val(pmd);
/* If PMD busy, retry the access */
- if (unlikely(old_pmd & _PAGE_BUSY))
+ if (unlikely(old_pmd & H_PAGE_BUSY))
return 0;
/* If PMD permissions don't match, take page fault */
- if (unlikely(access & ~old_pmd))
+ if (unlikely(!check_pte_access(access, old_pmd)))
return 1;
/*
* Try to lock the PTE, add ACCESSED and DIRTY if it was
* a write access
*/
- new_pmd = old_pmd | _PAGE_BUSY | _PAGE_ACCESSED;
- if (access & _PAGE_RW)
+ new_pmd = old_pmd | H_PAGE_BUSY | _PAGE_ACCESSED;
+ if (access & _PAGE_WRITE)
new_pmd |= _PAGE_DIRTY;
- } while (old_pmd != __cmpxchg_u64((unsigned long *)pmdp,
- old_pmd, new_pmd));
+ } while (!pmd_xchg(pmdp, __pmd(old_pmd), __pmd(new_pmd)));
+
rflags = htab_convert_pte_flags(new_pmd);
#if 0
@@ -78,7 +78,7 @@
* base page size. This is because demote_segment won't flush
* hash page table entries.
*/
- if ((old_pmd & _PAGE_HASHPTE) && !(old_pmd & _PAGE_COMBO)) {
+ if ((old_pmd & H_PAGE_HASHPTE) && !(old_pmd & H_PAGE_COMBO)) {
flush_hash_hugepage(vsid, ea, pmdp, MMU_PAGE_64K,
ssize, flags);
/*
@@ -125,7 +125,7 @@
hash = hpt_hash(vpn, shift, ssize);
/* insert new entry */
pa = pmd_pfn(__pmd(old_pmd)) << PAGE_SHIFT;
- new_pmd |= _PAGE_HASHPTE;
+ new_pmd |= H_PAGE_HASHPTE;
repeat:
hpte_group = ((hash & htab_hash_mask) * HPTES_PER_GROUP) & ~0x7UL;
@@ -169,17 +169,17 @@
mark_hpte_slot_valid(hpte_slot_array, index, slot);
}
/*
- * Mark the pte with _PAGE_COMBO, if we are trying to hash it with
+ * Mark the pte with H_PAGE_COMBO, if we are trying to hash it with
* base page size 4k.
*/
if (psize == MMU_PAGE_4K)
- new_pmd |= _PAGE_COMBO;
+ new_pmd |= H_PAGE_COMBO;
/*
* The hpte valid is stored in the pgtable whose address is in the
* second half of the PMD. Order this against clearing of the busy bit in
* huge pmd.
*/
smp_wmb();
- *pmdp = __pmd(new_pmd & ~_PAGE_BUSY);
+ *pmdp = __pmd(new_pmd & ~H_PAGE_BUSY);
return 0;
}
diff --git a/arch/powerpc/mm/hugetlbpage-hash64.c b/arch/powerpc/mm/hugetlbpage-hash64.c
index 8555fce..3058560 100644
--- a/arch/powerpc/mm/hugetlbpage-hash64.c
+++ b/arch/powerpc/mm/hugetlbpage-hash64.c
@@ -47,18 +47,19 @@
do {
old_pte = pte_val(*ptep);
/* If PTE busy, retry the access */
- if (unlikely(old_pte & _PAGE_BUSY))
+ if (unlikely(old_pte & H_PAGE_BUSY))
return 0;
/* If PTE permissions don't match, take page fault */
- if (unlikely(access & ~old_pte))
+ if (unlikely(!check_pte_access(access, old_pte)))
return 1;
+
/* Try to lock the PTE, add ACCESSED and DIRTY if it was
* a write access */
- new_pte = old_pte | _PAGE_BUSY | _PAGE_ACCESSED;
- if (access & _PAGE_RW)
+ new_pte = old_pte | H_PAGE_BUSY | _PAGE_ACCESSED;
+ if (access & _PAGE_WRITE)
new_pte |= _PAGE_DIRTY;
- } while(old_pte != __cmpxchg_u64((unsigned long *)ptep,
- old_pte, new_pte));
+ } while(!pte_xchg(ptep, __pte(old_pte), __pte(new_pte)));
+
rflags = htab_convert_pte_flags(new_pte);
sz = ((1UL) << shift);
@@ -68,28 +69,28 @@
rflags = hash_page_do_lazy_icache(rflags, __pte(old_pte), trap);
/* Check if pte already has an hpte (case 2) */
- if (unlikely(old_pte & _PAGE_HASHPTE)) {
+ if (unlikely(old_pte & H_PAGE_HASHPTE)) {
/* There MIGHT be an HPTE for this pte */
unsigned long hash, slot;
hash = hpt_hash(vpn, shift, ssize);
- if (old_pte & _PAGE_F_SECOND)
+ if (old_pte & H_PAGE_F_SECOND)
hash = ~hash;
slot = (hash & htab_hash_mask) * HPTES_PER_GROUP;
- slot += (old_pte & _PAGE_F_GIX) >> _PAGE_F_GIX_SHIFT;
+ slot += (old_pte & H_PAGE_F_GIX) >> H_PAGE_F_GIX_SHIFT;
if (ppc_md.hpte_updatepp(slot, rflags, vpn, mmu_psize,
mmu_psize, ssize, flags) == -1)
old_pte &= ~_PAGE_HPTEFLAGS;
}
- if (likely(!(old_pte & _PAGE_HASHPTE))) {
+ if (likely(!(old_pte & H_PAGE_HASHPTE))) {
unsigned long hash = hpt_hash(vpn, shift, ssize);
pa = pte_pfn(__pte(old_pte)) << PAGE_SHIFT;
/* clear HPTE slot informations in new PTE */
- new_pte = (new_pte & ~_PAGE_HPTEFLAGS) | _PAGE_HASHPTE;
+ new_pte = (new_pte & ~_PAGE_HPTEFLAGS) | H_PAGE_HASHPTE;
slot = hpte_insert_repeating(hash, vpn, pa, rflags, 0,
mmu_psize, ssize);
@@ -105,14 +106,14 @@
return -1;
}
- new_pte |= (slot << _PAGE_F_GIX_SHIFT) &
- (_PAGE_F_SECOND | _PAGE_F_GIX);
+ new_pte |= (slot << H_PAGE_F_GIX_SHIFT) &
+ (H_PAGE_F_SECOND | H_PAGE_F_GIX);
}
/*
* No need to use ldarx/stdcx here
*/
- *ptep = __pte(new_pte & ~_PAGE_BUSY);
+ *ptep = __pte(new_pte & ~H_PAGE_BUSY);
return 0;
}
diff --git a/arch/powerpc/mm/hugetlbpage-radix.c b/arch/powerpc/mm/hugetlbpage-radix.c
new file mode 100644
index 0000000..1e11559
--- /dev/null
+++ b/arch/powerpc/mm/hugetlbpage-radix.c
@@ -0,0 +1,87 @@
+#include <linux/mm.h>
+#include <linux/hugetlb.h>
+#include <asm/pgtable.h>
+#include <asm/pgalloc.h>
+#include <asm/cacheflush.h>
+#include <asm/machdep.h>
+#include <asm/mman.h>
+
+void radix__flush_hugetlb_page(struct vm_area_struct *vma, unsigned long vmaddr)
+{
+ unsigned long ap, shift;
+ struct hstate *hstate = hstate_file(vma->vm_file);
+
+ shift = huge_page_shift(hstate);
+ if (shift == mmu_psize_defs[MMU_PAGE_2M].shift)
+ ap = mmu_get_ap(MMU_PAGE_2M);
+ else if (shift == mmu_psize_defs[MMU_PAGE_1G].shift)
+ ap = mmu_get_ap(MMU_PAGE_1G);
+ else {
+ WARN(1, "Wrong huge page shift\n");
+ return ;
+ }
+ radix___flush_tlb_page(vma->vm_mm, vmaddr, ap, 0);
+}
+
+void radix__local_flush_hugetlb_page(struct vm_area_struct *vma, unsigned long vmaddr)
+{
+ unsigned long ap, shift;
+ struct hstate *hstate = hstate_file(vma->vm_file);
+
+ shift = huge_page_shift(hstate);
+ if (shift == mmu_psize_defs[MMU_PAGE_2M].shift)
+ ap = mmu_get_ap(MMU_PAGE_2M);
+ else if (shift == mmu_psize_defs[MMU_PAGE_1G].shift)
+ ap = mmu_get_ap(MMU_PAGE_1G);
+ else {
+ WARN(1, "Wrong huge page shift\n");
+ return ;
+ }
+ radix___local_flush_tlb_page(vma->vm_mm, vmaddr, ap, 0);
+}
+
+/*
+ * A vairant of hugetlb_get_unmapped_area doing topdown search
+ * FIXME!! should we do as x86 does or non hugetlb area does ?
+ * ie, use topdown or not based on mmap_is_legacy check ?
+ */
+unsigned long
+radix__hugetlb_get_unmapped_area(struct file *file, unsigned long addr,
+ unsigned long len, unsigned long pgoff,
+ unsigned long flags)
+{
+ struct mm_struct *mm = current->mm;
+ struct vm_area_struct *vma;
+ struct hstate *h = hstate_file(file);
+ struct vm_unmapped_area_info info;
+
+ if (len & ~huge_page_mask(h))
+ return -EINVAL;
+ if (len > TASK_SIZE)
+ return -ENOMEM;
+
+ if (flags & MAP_FIXED) {
+ if (prepare_hugepage_range(file, addr, len))
+ return -EINVAL;
+ return addr;
+ }
+
+ if (addr) {
+ addr = ALIGN(addr, huge_page_size(h));
+ vma = find_vma(mm, addr);
+ if (TASK_SIZE - len >= addr &&
+ (!vma || addr + len <= vma->vm_start))
+ return addr;
+ }
+ /*
+ * We are always doing an topdown search here. Slice code
+ * does that too.
+ */
+ info.flags = VM_UNMAPPED_AREA_TOPDOWN;
+ info.length = len;
+ info.low_limit = PAGE_SIZE;
+ info.high_limit = current->mm->mmap_base;
+ info.align_mask = PAGE_MASK & ~huge_page_mask(h);
+ info.align_offset = 0;
+ return vm_unmapped_area(&info);
+}
diff --git a/arch/powerpc/mm/hugetlbpage.c b/arch/powerpc/mm/hugetlbpage.c
index d991b9e..5aac1a3 100644
--- a/arch/powerpc/mm/hugetlbpage.c
+++ b/arch/powerpc/mm/hugetlbpage.c
@@ -711,6 +711,9 @@
struct hstate *hstate = hstate_file(file);
int mmu_psize = shift_to_mmu_psize(huge_page_shift(hstate));
+ if (radix_enabled())
+ return radix__hugetlb_get_unmapped_area(file, addr, len,
+ pgoff, flags);
return slice_get_unmapped_area(addr, len, flags, mmu_psize, 1);
}
#endif
@@ -719,14 +722,14 @@
{
#ifdef CONFIG_PPC_MM_SLICES
unsigned int psize = get_slice_psize(vma->vm_mm, vma->vm_start);
-
- return 1UL << mmu_psize_to_shift(psize);
-#else
+ /* With radix we don't use slice, so derive it from vma*/
+ if (!radix_enabled())
+ return 1UL << mmu_psize_to_shift(psize);
+#endif
if (!is_vm_hugetlb_page(vma))
return PAGE_SIZE;
return huge_page_size(hstate_vma(vma));
-#endif
}
static inline bool is_power_of_4(unsigned long x)
@@ -772,8 +775,10 @@
size = memparse(str, &str);
- if (add_huge_page_size(size) != 0)
- printk(KERN_WARNING "Invalid huge page size specified(%llu)\n", size);
+ if (add_huge_page_size(size) != 0) {
+ hugetlb_bad_size();
+ pr_err("Invalid huge page size specified(%llu)\n", size);
+ }
return 1;
}
@@ -823,7 +828,7 @@
{
int psize;
- if (!mmu_has_feature(MMU_FTR_16M_PAGE))
+ if (!radix_enabled() && !mmu_has_feature(MMU_FTR_16M_PAGE))
return -ENODEV;
for (psize = 0; psize < MMU_PAGE_COUNT; ++psize) {
@@ -863,6 +868,9 @@
HPAGE_SHIFT = mmu_psize_defs[MMU_PAGE_16M].shift;
else if (mmu_psize_defs[MMU_PAGE_1M].shift)
HPAGE_SHIFT = mmu_psize_defs[MMU_PAGE_1M].shift;
+ else if (mmu_psize_defs[MMU_PAGE_2M].shift)
+ HPAGE_SHIFT = mmu_psize_defs[MMU_PAGE_2M].shift;
+
return 0;
}
@@ -1003,9 +1011,9 @@
end = pte_end;
pte = READ_ONCE(*ptep);
- mask = _PAGE_PRESENT | _PAGE_USER;
+ mask = _PAGE_PRESENT | _PAGE_READ;
if (write)
- mask |= _PAGE_RW;
+ mask |= _PAGE_WRITE;
if ((pte_val(pte) & mask) != mask)
return 0;
diff --git a/arch/powerpc/mm/init_64.c b/arch/powerpc/mm/init_64.c
index ba65566..33709bd 100644
--- a/arch/powerpc/mm/init_64.c
+++ b/arch/powerpc/mm/init_64.c
@@ -66,11 +66,11 @@
#include "mmu_decl.h"
#ifdef CONFIG_PPC_STD_MMU_64
-#if PGTABLE_RANGE > USER_VSID_RANGE
+#if H_PGTABLE_RANGE > USER_VSID_RANGE
#warning Limited user VSID range means pagetable space is wasted
#endif
-#if (TASK_SIZE_USER64 < PGTABLE_RANGE) && (TASK_SIZE_USER64 < USER_VSID_RANGE)
+#if (TASK_SIZE_USER64 < H_PGTABLE_RANGE) && (TASK_SIZE_USER64 < USER_VSID_RANGE)
#warning TASK_SIZE is smaller than it needs to be.
#endif
#endif /* CONFIG_PPC_STD_MMU_64 */
@@ -189,75 +189,6 @@
return 0;
}
-/* On hash-based CPUs, the vmemmap is bolted in the hash table.
- *
- * On Book3E CPUs, the vmemmap is currently mapped in the top half of
- * the vmalloc space using normal page tables, though the size of
- * pages encoded in the PTEs can be different
- */
-
-#ifdef CONFIG_PPC_BOOK3E
-static int __meminit vmemmap_create_mapping(unsigned long start,
- unsigned long page_size,
- unsigned long phys)
-{
- /* Create a PTE encoding without page size */
- unsigned long i, flags = _PAGE_PRESENT | _PAGE_ACCESSED |
- _PAGE_KERNEL_RW;
-
- /* PTEs only contain page size encodings up to 32M */
- BUG_ON(mmu_psize_defs[mmu_vmemmap_psize].enc > 0xf);
-
- /* Encode the size in the PTE */
- flags |= mmu_psize_defs[mmu_vmemmap_psize].enc << 8;
-
- /* For each PTE for that area, map things. Note that we don't
- * increment phys because all PTEs are of the large size and
- * thus must have the low bits clear
- */
- for (i = 0; i < page_size; i += PAGE_SIZE)
- BUG_ON(map_kernel_page(start + i, phys, flags));
-
- return 0;
-}
-
-#ifdef CONFIG_MEMORY_HOTPLUG
-static void vmemmap_remove_mapping(unsigned long start,
- unsigned long page_size)
-{
-}
-#endif
-#else /* CONFIG_PPC_BOOK3E */
-static int __meminit vmemmap_create_mapping(unsigned long start,
- unsigned long page_size,
- unsigned long phys)
-{
- int rc = htab_bolt_mapping(start, start + page_size, phys,
- pgprot_val(PAGE_KERNEL),
- mmu_vmemmap_psize, mmu_kernel_ssize);
- if (rc < 0) {
- int rc2 = htab_remove_mapping(start, start + page_size,
- mmu_vmemmap_psize,
- mmu_kernel_ssize);
- BUG_ON(rc2 && (rc2 != -ENOENT));
- }
- return rc;
-}
-
-#ifdef CONFIG_MEMORY_HOTPLUG
-static void vmemmap_remove_mapping(unsigned long start,
- unsigned long page_size)
-{
- int rc = htab_remove_mapping(start, start + page_size,
- mmu_vmemmap_psize,
- mmu_kernel_ssize);
- BUG_ON((rc < 0) && (rc != -ENOENT));
- WARN_ON(rc == -ENOENT);
-}
-#endif
-
-#endif /* CONFIG_PPC_BOOK3E */
-
struct vmemmap_backing *vmemmap_list;
static struct vmemmap_backing *next;
static int num_left;
diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index ac79dbd..2fd57fa 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -68,12 +68,15 @@
EXPORT_SYMBOL(kmap_pte);
pgprot_t kmap_prot;
EXPORT_SYMBOL(kmap_prot);
+#define TOP_ZONE ZONE_HIGHMEM
static inline pte_t *virt_to_kpte(unsigned long vaddr)
{
return pte_offset_kernel(pmd_offset(pud_offset(pgd_offset_k(vaddr),
vaddr), vaddr), vaddr);
}
+#else
+#define TOP_ZONE ZONE_NORMAL
#endif
int page_is_ram(unsigned long pfn)
@@ -267,14 +270,9 @@
*/
int dma_pfn_limit_to_zone(u64 pfn_limit)
{
- enum zone_type top_zone = ZONE_NORMAL;
int i;
-#ifdef CONFIG_HIGHMEM
- top_zone = ZONE_HIGHMEM;
-#endif
-
- for (i = top_zone; i >= 0; i--) {
+ for (i = TOP_ZONE; i >= 0; i--) {
if (max_zone_pfns[i] <= pfn_limit)
return i;
}
@@ -289,7 +287,6 @@
{
unsigned long long total_ram = memblock_phys_mem_size();
phys_addr_t top_of_ram = memblock_end_of_DRAM();
- enum zone_type top_zone;
#ifdef CONFIG_PPC32
unsigned long v = __fix_to_virt(__end_of_fixed_addresses - 1);
@@ -313,13 +310,9 @@
(long int)((top_of_ram - total_ram) >> 20));
#ifdef CONFIG_HIGHMEM
- top_zone = ZONE_HIGHMEM;
limit_zone_pfn(ZONE_NORMAL, lowmem_end_addr >> PAGE_SHIFT);
-#else
- top_zone = ZONE_NORMAL;
#endif
-
- limit_zone_pfn(top_zone, top_of_ram >> PAGE_SHIFT);
+ limit_zone_pfn(TOP_ZONE, top_of_ram >> PAGE_SHIFT);
zone_limits_final = true;
free_area_init_nodes(max_zone_pfns);
@@ -498,7 +491,10 @@
* We don't need to worry about _PAGE_PRESENT here because we are
* called with either mm->page_table_lock held or ptl lock held
*/
- unsigned long access = 0, trap;
+ unsigned long access, trap;
+
+ if (radix_enabled())
+ return;
/* We only want HPTEs for linux PTEs that have _PAGE_ACCESSED set */
if (!pte_young(*ptep) || address >= TASK_SIZE)
@@ -511,13 +507,19 @@
*
* We also avoid filling the hash if not coming from a fault
*/
- if (current->thread.regs == NULL)
+
+ trap = current->thread.regs ? TRAP(current->thread.regs) : 0UL;
+ switch (trap) {
+ case 0x300:
+ access = 0UL;
+ break;
+ case 0x400:
+ access = _PAGE_EXEC;
+ break;
+ default:
return;
- trap = TRAP(current->thread.regs);
- if (trap == 0x400)
- access |= _PAGE_EXEC;
- else if (trap != 0x300)
- return;
+ }
+
hash_preload(vma->vm_mm, address, access, trap);
#endif /* CONFIG_PPC_STD_MMU */
#if (defined(CONFIG_PPC_BOOK3E_64) || defined(CONFIG_PPC_FSL_BOOK3E)) \
diff --git a/arch/powerpc/mm/mmap.c b/arch/powerpc/mm/mmap.c
index 4087705..2f1e443 100644
--- a/arch/powerpc/mm/mmap.c
+++ b/arch/powerpc/mm/mmap.c
@@ -26,6 +26,9 @@
#include <linux/mm.h>
#include <linux/random.h>
#include <linux/sched.h>
+#include <linux/elf-randomize.h>
+#include <linux/security.h>
+#include <linux/mman.h>
/*
* Top of mmap area (just below the process stack).
@@ -78,6 +81,111 @@
return PAGE_ALIGN(TASK_SIZE - gap - rnd);
}
+#ifdef CONFIG_PPC_RADIX_MMU
+/*
+ * Same function as generic code used only for radix, because we don't need to overload
+ * the generic one. But we will have to duplicate, because hash select
+ * HAVE_ARCH_UNMAPPED_AREA
+ */
+static unsigned long
+radix__arch_get_unmapped_area(struct file *filp, unsigned long addr,
+ unsigned long len, unsigned long pgoff,
+ unsigned long flags)
+{
+ struct mm_struct *mm = current->mm;
+ struct vm_area_struct *vma;
+ struct vm_unmapped_area_info info;
+
+ if (len > TASK_SIZE - mmap_min_addr)
+ return -ENOMEM;
+
+ if (flags & MAP_FIXED)
+ return addr;
+
+ if (addr) {
+ addr = PAGE_ALIGN(addr);
+ vma = find_vma(mm, addr);
+ if (TASK_SIZE - len >= addr && addr >= mmap_min_addr &&
+ (!vma || addr + len <= vma->vm_start))
+ return addr;
+ }
+
+ info.flags = 0;
+ info.length = len;
+ info.low_limit = mm->mmap_base;
+ info.high_limit = TASK_SIZE;
+ info.align_mask = 0;
+ return vm_unmapped_area(&info);
+}
+
+static unsigned long
+radix__arch_get_unmapped_area_topdown(struct file *filp,
+ const unsigned long addr0,
+ const unsigned long len,
+ const unsigned long pgoff,
+ const unsigned long flags)
+{
+ struct vm_area_struct *vma;
+ struct mm_struct *mm = current->mm;
+ unsigned long addr = addr0;
+ struct vm_unmapped_area_info info;
+
+ /* requested length too big for entire address space */
+ if (len > TASK_SIZE - mmap_min_addr)
+ return -ENOMEM;
+
+ if (flags & MAP_FIXED)
+ return addr;
+
+ /* requesting a specific address */
+ if (addr) {
+ addr = PAGE_ALIGN(addr);
+ vma = find_vma(mm, addr);
+ if (TASK_SIZE - len >= addr && addr >= mmap_min_addr &&
+ (!vma || addr + len <= vma->vm_start))
+ return addr;
+ }
+
+ info.flags = VM_UNMAPPED_AREA_TOPDOWN;
+ info.length = len;
+ info.low_limit = max(PAGE_SIZE, mmap_min_addr);
+ info.high_limit = mm->mmap_base;
+ info.align_mask = 0;
+ addr = vm_unmapped_area(&info);
+
+ /*
+ * A failed mmap() very likely causes application failure,
+ * so fall back to the bottom-up function here. This scenario
+ * can happen with large stack limits and large mmap()
+ * allocations.
+ */
+ if (addr & ~PAGE_MASK) {
+ VM_BUG_ON(addr != -ENOMEM);
+ info.flags = 0;
+ info.low_limit = TASK_UNMAPPED_BASE;
+ info.high_limit = TASK_SIZE;
+ addr = vm_unmapped_area(&info);
+ }
+
+ return addr;
+}
+
+static void radix__arch_pick_mmap_layout(struct mm_struct *mm,
+ unsigned long random_factor)
+{
+ if (mmap_is_legacy()) {
+ mm->mmap_base = TASK_UNMAPPED_BASE;
+ mm->get_unmapped_area = radix__arch_get_unmapped_area;
+ } else {
+ mm->mmap_base = mmap_base(random_factor);
+ mm->get_unmapped_area = radix__arch_get_unmapped_area_topdown;
+ }
+}
+#else
+/* dummy */
+extern void radix__arch_pick_mmap_layout(struct mm_struct *mm,
+ unsigned long random_factor);
+#endif
/*
* This function, called very early during the creation of a new
* process VM image, sets up which VM layout function to use:
@@ -89,6 +197,8 @@
if (current->flags & PF_RANDOMIZE)
random_factor = arch_mmap_rnd();
+ if (radix_enabled())
+ return radix__arch_pick_mmap_layout(mm, random_factor);
/*
* Fall back to the standard layout if the personality
* bit is set, or if the expected stack growth is unlimited:
diff --git a/arch/powerpc/mm/mmu_context_book3s64.c b/arch/powerpc/mm/mmu_context_book3s64.c
new file mode 100644
index 0000000..227b2a6
--- /dev/null
+++ b/arch/powerpc/mm/mmu_context_book3s64.c
@@ -0,0 +1,187 @@
+/*
+ * MMU context allocation for 64-bit kernels.
+ *
+ * Copyright (C) 2004 Anton Blanchard, IBM Corp. <anton@samba.org>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ *
+ */
+
+#include <linux/sched.h>
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/string.h>
+#include <linux/types.h>
+#include <linux/mm.h>
+#include <linux/spinlock.h>
+#include <linux/idr.h>
+#include <linux/export.h>
+#include <linux/gfp.h>
+#include <linux/slab.h>
+
+#include <asm/mmu_context.h>
+#include <asm/pgalloc.h>
+
+#include "icswx.h"
+
+static DEFINE_SPINLOCK(mmu_context_lock);
+static DEFINE_IDA(mmu_context_ida);
+
+int __init_new_context(void)
+{
+ int index;
+ int err;
+
+again:
+ if (!ida_pre_get(&mmu_context_ida, GFP_KERNEL))
+ return -ENOMEM;
+
+ spin_lock(&mmu_context_lock);
+ err = ida_get_new_above(&mmu_context_ida, 1, &index);
+ spin_unlock(&mmu_context_lock);
+
+ if (err == -EAGAIN)
+ goto again;
+ else if (err)
+ return err;
+
+ if (index > MAX_USER_CONTEXT) {
+ spin_lock(&mmu_context_lock);
+ ida_remove(&mmu_context_ida, index);
+ spin_unlock(&mmu_context_lock);
+ return -ENOMEM;
+ }
+
+ return index;
+}
+EXPORT_SYMBOL_GPL(__init_new_context);
+static int radix__init_new_context(struct mm_struct *mm, int index)
+{
+ unsigned long rts_field;
+
+ /*
+ * set the process table entry,
+ */
+ rts_field = 3ull << PPC_BITLSHIFT(2);
+ process_tb[index].prtb0 = cpu_to_be64(rts_field | __pa(mm->pgd) | RADIX_PGD_INDEX_SIZE);
+ return 0;
+}
+
+int init_new_context(struct task_struct *tsk, struct mm_struct *mm)
+{
+ int index;
+
+ index = __init_new_context();
+ if (index < 0)
+ return index;
+
+ if (radix_enabled()) {
+ radix__init_new_context(mm, index);
+ } else {
+
+ /* The old code would re-promote on fork, we don't do that
+ * when using slices as it could cause problem promoting slices
+ * that have been forced down to 4K
+ *
+ * For book3s we have MMU_NO_CONTEXT set to be ~0. Hence check
+ * explicitly against context.id == 0. This ensures that we
+ * properly initialize context slice details for newly allocated
+ * mm's (which will have id == 0) and don't alter context slice
+ * inherited via fork (which will have id != 0).
+ *
+ * We should not be calling init_new_context() on init_mm. Hence a
+ * check against 0 is ok.
+ */
+ if (mm->context.id == 0)
+ slice_set_user_psize(mm, mmu_virtual_psize);
+ subpage_prot_init_new_context(mm);
+ }
+ mm->context.id = index;
+#ifdef CONFIG_PPC_ICSWX
+ mm->context.cop_lockp = kmalloc(sizeof(spinlock_t), GFP_KERNEL);
+ if (!mm->context.cop_lockp) {
+ __destroy_context(index);
+ subpage_prot_free(mm);
+ mm->context.id = MMU_NO_CONTEXT;
+ return -ENOMEM;
+ }
+ spin_lock_init(mm->context.cop_lockp);
+#endif /* CONFIG_PPC_ICSWX */
+
+#ifdef CONFIG_PPC_64K_PAGES
+ mm->context.pte_frag = NULL;
+#endif
+#ifdef CONFIG_SPAPR_TCE_IOMMU
+ mm_iommu_init(&mm->context);
+#endif
+ return 0;
+}
+
+void __destroy_context(int context_id)
+{
+ spin_lock(&mmu_context_lock);
+ ida_remove(&mmu_context_ida, context_id);
+ spin_unlock(&mmu_context_lock);
+}
+EXPORT_SYMBOL_GPL(__destroy_context);
+
+#ifdef CONFIG_PPC_64K_PAGES
+static void destroy_pagetable_page(struct mm_struct *mm)
+{
+ int count;
+ void *pte_frag;
+ struct page *page;
+
+ pte_frag = mm->context.pte_frag;
+ if (!pte_frag)
+ return;
+
+ page = virt_to_page(pte_frag);
+ /* drop all the pending references */
+ count = ((unsigned long)pte_frag & ~PAGE_MASK) >> PTE_FRAG_SIZE_SHIFT;
+ /* We allow PTE_FRAG_NR fragments from a PTE page */
+ if (page_ref_sub_and_test(page, PTE_FRAG_NR - count)) {
+ pgtable_page_dtor(page);
+ free_hot_cold_page(page, 0);
+ }
+}
+
+#else
+static inline void destroy_pagetable_page(struct mm_struct *mm)
+{
+ return;
+}
+#endif
+
+
+void destroy_context(struct mm_struct *mm)
+{
+#ifdef CONFIG_SPAPR_TCE_IOMMU
+ mm_iommu_cleanup(&mm->context);
+#endif
+
+#ifdef CONFIG_PPC_ICSWX
+ drop_cop(mm->context.acop, mm);
+ kfree(mm->context.cop_lockp);
+ mm->context.cop_lockp = NULL;
+#endif /* CONFIG_PPC_ICSWX */
+
+ if (radix_enabled())
+ process_tb[mm->context.id].prtb1 = 0;
+ else
+ subpage_prot_free(mm);
+ destroy_pagetable_page(mm);
+ __destroy_context(mm->context.id);
+ mm->context.id = MMU_NO_CONTEXT;
+}
+
+#ifdef CONFIG_PPC_RADIX_MMU
+void radix__switch_mmu_context(struct mm_struct *prev, struct mm_struct *next)
+{
+ mtspr(SPRN_PID, next->context.id);
+ asm volatile("isync": : :"memory");
+}
+#endif
diff --git a/arch/powerpc/mm/mmu_context_hash64.c b/arch/powerpc/mm/mmu_context_hash64.c
deleted file mode 100644
index 9ca6fe1..0000000
--- a/arch/powerpc/mm/mmu_context_hash64.c
+++ /dev/null
@@ -1,151 +0,0 @@
-/*
- * MMU context allocation for 64-bit kernels.
- *
- * Copyright (C) 2004 Anton Blanchard, IBM Corp. <anton@samba.org>
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
- */
-
-#include <linux/sched.h>
-#include <linux/kernel.h>
-#include <linux/errno.h>
-#include <linux/string.h>
-#include <linux/types.h>
-#include <linux/mm.h>
-#include <linux/spinlock.h>
-#include <linux/idr.h>
-#include <linux/export.h>
-#include <linux/gfp.h>
-#include <linux/slab.h>
-
-#include <asm/mmu_context.h>
-#include <asm/pgalloc.h>
-
-#include "icswx.h"
-
-static DEFINE_SPINLOCK(mmu_context_lock);
-static DEFINE_IDA(mmu_context_ida);
-
-int __init_new_context(void)
-{
- int index;
- int err;
-
-again:
- if (!ida_pre_get(&mmu_context_ida, GFP_KERNEL))
- return -ENOMEM;
-
- spin_lock(&mmu_context_lock);
- err = ida_get_new_above(&mmu_context_ida, 1, &index);
- spin_unlock(&mmu_context_lock);
-
- if (err == -EAGAIN)
- goto again;
- else if (err)
- return err;
-
- if (index > MAX_USER_CONTEXT) {
- spin_lock(&mmu_context_lock);
- ida_remove(&mmu_context_ida, index);
- spin_unlock(&mmu_context_lock);
- return -ENOMEM;
- }
-
- return index;
-}
-EXPORT_SYMBOL_GPL(__init_new_context);
-
-int init_new_context(struct task_struct *tsk, struct mm_struct *mm)
-{
- int index;
-
- index = __init_new_context();
- if (index < 0)
- return index;
-
- /* The old code would re-promote on fork, we don't do that
- * when using slices as it could cause problem promoting slices
- * that have been forced down to 4K
- */
- if (slice_mm_new_context(mm))
- slice_set_user_psize(mm, mmu_virtual_psize);
- subpage_prot_init_new_context(mm);
- mm->context.id = index;
-#ifdef CONFIG_PPC_ICSWX
- mm->context.cop_lockp = kmalloc(sizeof(spinlock_t), GFP_KERNEL);
- if (!mm->context.cop_lockp) {
- __destroy_context(index);
- subpage_prot_free(mm);
- mm->context.id = MMU_NO_CONTEXT;
- return -ENOMEM;
- }
- spin_lock_init(mm->context.cop_lockp);
-#endif /* CONFIG_PPC_ICSWX */
-
-#ifdef CONFIG_PPC_64K_PAGES
- mm->context.pte_frag = NULL;
-#endif
-#ifdef CONFIG_SPAPR_TCE_IOMMU
- mm_iommu_init(&mm->context);
-#endif
- return 0;
-}
-
-void __destroy_context(int context_id)
-{
- spin_lock(&mmu_context_lock);
- ida_remove(&mmu_context_ida, context_id);
- spin_unlock(&mmu_context_lock);
-}
-EXPORT_SYMBOL_GPL(__destroy_context);
-
-#ifdef CONFIG_PPC_64K_PAGES
-static void destroy_pagetable_page(struct mm_struct *mm)
-{
- int count;
- void *pte_frag;
- struct page *page;
-
- pte_frag = mm->context.pte_frag;
- if (!pte_frag)
- return;
-
- page = virt_to_page(pte_frag);
- /* drop all the pending references */
- count = ((unsigned long)pte_frag & ~PAGE_MASK) >> PTE_FRAG_SIZE_SHIFT;
- /* We allow PTE_FRAG_NR fragments from a PTE page */
- if (page_ref_sub_and_test(page, PTE_FRAG_NR - count)) {
- pgtable_page_dtor(page);
- free_hot_cold_page(page, 0);
- }
-}
-
-#else
-static inline void destroy_pagetable_page(struct mm_struct *mm)
-{
- return;
-}
-#endif
-
-
-void destroy_context(struct mm_struct *mm)
-{
-#ifdef CONFIG_SPAPR_TCE_IOMMU
- mm_iommu_cleanup(&mm->context);
-#endif
-
-#ifdef CONFIG_PPC_ICSWX
- drop_cop(mm->context.acop, mm);
- kfree(mm->context.cop_lockp);
- mm->context.cop_lockp = NULL;
-#endif /* CONFIG_PPC_ICSWX */
-
- destroy_pagetable_page(mm);
- __destroy_context(mm->context.id);
- subpage_prot_free(mm);
- mm->context.id = MMU_NO_CONTEXT;
-}
diff --git a/arch/powerpc/mm/mmu_context_nohash.c b/arch/powerpc/mm/mmu_context_nohash.c
index 986afbc..7d95bc4 100644
--- a/arch/powerpc/mm/mmu_context_nohash.c
+++ b/arch/powerpc/mm/mmu_context_nohash.c
@@ -226,7 +226,8 @@
static void context_check_map(void) { }
#endif
-void switch_mmu_context(struct mm_struct *prev, struct mm_struct *next)
+void switch_mmu_context(struct mm_struct *prev, struct mm_struct *next,
+ struct task_struct *tsk)
{
unsigned int i, id, cpu = smp_processor_id();
unsigned long *map;
@@ -334,8 +335,7 @@
mm->context.active = 0;
#ifdef CONFIG_PPC_MM_SLICES
- if (slice_mm_new_context(mm))
- slice_set_user_psize(mm, mmu_virtual_psize);
+ slice_set_user_psize(mm, mmu_virtual_psize);
#endif
return 0;
diff --git a/arch/powerpc/mm/mmu_decl.h b/arch/powerpc/mm/mmu_decl.h
index bfb7c0b..6af6532 100644
--- a/arch/powerpc/mm/mmu_decl.h
+++ b/arch/powerpc/mm/mmu_decl.h
@@ -108,11 +108,6 @@
#endif /* CONFIG_PPC32 */
-#ifdef CONFIG_PPC64
-extern int map_kernel_page(unsigned long ea, unsigned long pa,
- unsigned long flags);
-#endif /* CONFIG_PPC64 */
-
extern unsigned long ioremap_bot;
extern unsigned long __max_low_memory;
extern phys_addr_t __initial_memory_limit_addr;
diff --git a/arch/powerpc/mm/pgtable-book3e.c b/arch/powerpc/mm/pgtable-book3e.c
new file mode 100644
index 0000000..a229893
--- /dev/null
+++ b/arch/powerpc/mm/pgtable-book3e.c
@@ -0,0 +1,122 @@
+/*
+ * Copyright 2005, Paul Mackerras, IBM Corporation.
+ * Copyright 2009, Benjamin Herrenschmidt, IBM Corporation.
+ * Copyright 2015-2016, Aneesh Kumar K.V, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <linux/sched.h>
+#include <linux/memblock.h>
+#include <asm/pgalloc.h>
+#include <asm/tlb.h>
+#include <asm/dma.h>
+
+#include "mmu_decl.h"
+
+#ifdef CONFIG_SPARSEMEM_VMEMMAP
+/*
+ * On Book3E CPUs, the vmemmap is currently mapped in the top half of
+ * the vmalloc space using normal page tables, though the size of
+ * pages encoded in the PTEs can be different
+ */
+int __meminit vmemmap_create_mapping(unsigned long start,
+ unsigned long page_size,
+ unsigned long phys)
+{
+ /* Create a PTE encoding without page size */
+ unsigned long i, flags = _PAGE_PRESENT | _PAGE_ACCESSED |
+ _PAGE_KERNEL_RW;
+
+ /* PTEs only contain page size encodings up to 32M */
+ BUG_ON(mmu_psize_defs[mmu_vmemmap_psize].enc > 0xf);
+
+ /* Encode the size in the PTE */
+ flags |= mmu_psize_defs[mmu_vmemmap_psize].enc << 8;
+
+ /* For each PTE for that area, map things. Note that we don't
+ * increment phys because all PTEs are of the large size and
+ * thus must have the low bits clear
+ */
+ for (i = 0; i < page_size; i += PAGE_SIZE)
+ BUG_ON(map_kernel_page(start + i, phys, flags));
+
+ return 0;
+}
+
+#ifdef CONFIG_MEMORY_HOTPLUG
+void vmemmap_remove_mapping(unsigned long start,
+ unsigned long page_size)
+{
+}
+#endif
+#endif /* CONFIG_SPARSEMEM_VMEMMAP */
+
+static __ref void *early_alloc_pgtable(unsigned long size)
+{
+ void *pt;
+
+ pt = __va(memblock_alloc_base(size, size, __pa(MAX_DMA_ADDRESS)));
+ memset(pt, 0, size);
+
+ return pt;
+}
+
+/*
+ * map_kernel_page currently only called by __ioremap
+ * map_kernel_page adds an entry to the ioremap page table
+ * and adds an entry to the HPT, possibly bolting it
+ */
+int map_kernel_page(unsigned long ea, unsigned long pa, unsigned long flags)
+{
+ pgd_t *pgdp;
+ pud_t *pudp;
+ pmd_t *pmdp;
+ pte_t *ptep;
+
+ BUILD_BUG_ON(TASK_SIZE_USER64 > PGTABLE_RANGE);
+ if (slab_is_available()) {
+ pgdp = pgd_offset_k(ea);
+ pudp = pud_alloc(&init_mm, pgdp, ea);
+ if (!pudp)
+ return -ENOMEM;
+ pmdp = pmd_alloc(&init_mm, pudp, ea);
+ if (!pmdp)
+ return -ENOMEM;
+ ptep = pte_alloc_kernel(pmdp, ea);
+ if (!ptep)
+ return -ENOMEM;
+ set_pte_at(&init_mm, ea, ptep, pfn_pte(pa >> PAGE_SHIFT,
+ __pgprot(flags)));
+ } else {
+ pgdp = pgd_offset_k(ea);
+#ifndef __PAGETABLE_PUD_FOLDED
+ if (pgd_none(*pgdp)) {
+ pudp = early_alloc_pgtable(PUD_TABLE_SIZE);
+ BUG_ON(pudp == NULL);
+ pgd_populate(&init_mm, pgdp, pudp);
+ }
+#endif /* !__PAGETABLE_PUD_FOLDED */
+ pudp = pud_offset(pgdp, ea);
+ if (pud_none(*pudp)) {
+ pmdp = early_alloc_pgtable(PMD_TABLE_SIZE);
+ BUG_ON(pmdp == NULL);
+ pud_populate(&init_mm, pudp, pmdp);
+ }
+ pmdp = pmd_offset(pudp, ea);
+ if (!pmd_present(*pmdp)) {
+ ptep = early_alloc_pgtable(PAGE_SIZE);
+ BUG_ON(ptep == NULL);
+ pmd_populate_kernel(&init_mm, pmdp, ptep);
+ }
+ ptep = pte_offset_kernel(pmdp, ea);
+ set_pte_at(&init_mm, ea, ptep, pfn_pte(pa >> PAGE_SHIFT,
+ __pgprot(flags)));
+ }
+
+ smp_wmb();
+ return 0;
+}
diff --git a/arch/powerpc/mm/pgtable-book3s64.c b/arch/powerpc/mm/pgtable-book3s64.c
new file mode 100644
index 0000000..eb44511
--- /dev/null
+++ b/arch/powerpc/mm/pgtable-book3s64.c
@@ -0,0 +1,118 @@
+/*
+ * Copyright 2015-2016, Aneesh Kumar K.V, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <linux/sched.h>
+#include <asm/pgalloc.h>
+#include <asm/tlb.h>
+
+#include "mmu_decl.h"
+#include <trace/events/thp.h>
+
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+/*
+ * This is called when relaxing access to a hugepage. It's also called in the page
+ * fault path when we don't hit any of the major fault cases, ie, a minor
+ * update of _PAGE_ACCESSED, _PAGE_DIRTY, etc... The generic code will have
+ * handled those two for us, we additionally deal with missing execute
+ * permission here on some processors
+ */
+int pmdp_set_access_flags(struct vm_area_struct *vma, unsigned long address,
+ pmd_t *pmdp, pmd_t entry, int dirty)
+{
+ int changed;
+#ifdef CONFIG_DEBUG_VM
+ WARN_ON(!pmd_trans_huge(*pmdp));
+ assert_spin_locked(&vma->vm_mm->page_table_lock);
+#endif
+ changed = !pmd_same(*(pmdp), entry);
+ if (changed) {
+ __ptep_set_access_flags(pmdp_ptep(pmdp), pmd_pte(entry));
+ /*
+ * Since we are not supporting SW TLB systems, we don't
+ * have any thing similar to flush_tlb_page_nohash()
+ */
+ }
+ return changed;
+}
+
+int pmdp_test_and_clear_young(struct vm_area_struct *vma,
+ unsigned long address, pmd_t *pmdp)
+{
+ return __pmdp_test_and_clear_young(vma->vm_mm, address, pmdp);
+}
+/*
+ * set a new huge pmd. We should not be called for updating
+ * an existing pmd entry. That should go via pmd_hugepage_update.
+ */
+void set_pmd_at(struct mm_struct *mm, unsigned long addr,
+ pmd_t *pmdp, pmd_t pmd)
+{
+#ifdef CONFIG_DEBUG_VM
+ WARN_ON(pte_present(pmd_pte(*pmdp)) && !pte_protnone(pmd_pte(*pmdp)));
+ assert_spin_locked(&mm->page_table_lock);
+ WARN_ON(!pmd_trans_huge(pmd));
+#endif
+ trace_hugepage_set_pmd(addr, pmd_val(pmd));
+ return set_pte_at(mm, addr, pmdp_ptep(pmdp), pmd_pte(pmd));
+}
+/*
+ * We use this to invalidate a pmdp entry before switching from a
+ * hugepte to regular pmd entry.
+ */
+void pmdp_invalidate(struct vm_area_struct *vma, unsigned long address,
+ pmd_t *pmdp)
+{
+ pmd_hugepage_update(vma->vm_mm, address, pmdp, _PAGE_PRESENT, 0);
+ flush_tlb_range(vma, address, address + HPAGE_PMD_SIZE);
+ /*
+ * This ensures that generic code that rely on IRQ disabling
+ * to prevent a parallel THP split work as expected.
+ */
+ kick_all_cpus_sync();
+}
+
+static pmd_t pmd_set_protbits(pmd_t pmd, pgprot_t pgprot)
+{
+ return __pmd(pmd_val(pmd) | pgprot_val(pgprot));
+}
+
+pmd_t pfn_pmd(unsigned long pfn, pgprot_t pgprot)
+{
+ unsigned long pmdv;
+
+ pmdv = (pfn << PAGE_SHIFT) & PTE_RPN_MASK;
+ return pmd_set_protbits(__pmd(pmdv), pgprot);
+}
+
+pmd_t mk_pmd(struct page *page, pgprot_t pgprot)
+{
+ return pfn_pmd(page_to_pfn(page), pgprot);
+}
+
+pmd_t pmd_modify(pmd_t pmd, pgprot_t newprot)
+{
+ unsigned long pmdv;
+
+ pmdv = pmd_val(pmd);
+ pmdv &= _HPAGE_CHG_MASK;
+ return pmd_set_protbits(__pmd(pmdv), newprot);
+}
+
+/*
+ * This is called at the end of handling a user page fault, when the
+ * fault has been handled by updating a HUGE PMD entry in the linux page tables.
+ * We use it to preload an HPTE into the hash table corresponding to
+ * the updated linux HUGE PMD entry.
+ */
+void update_mmu_cache_pmd(struct vm_area_struct *vma, unsigned long addr,
+ pmd_t *pmd)
+{
+ return;
+}
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
diff --git a/arch/powerpc/mm/pgtable-hash64.c b/arch/powerpc/mm/pgtable-hash64.c
new file mode 100644
index 0000000..c23e286
--- /dev/null
+++ b/arch/powerpc/mm/pgtable-hash64.c
@@ -0,0 +1,342 @@
+/*
+ * Copyright 2005, Paul Mackerras, IBM Corporation.
+ * Copyright 2009, Benjamin Herrenschmidt, IBM Corporation.
+ * Copyright 2015-2016, Aneesh Kumar K.V, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <linux/sched.h>
+#include <asm/pgalloc.h>
+#include <asm/tlb.h>
+
+#include "mmu_decl.h"
+
+#define CREATE_TRACE_POINTS
+#include <trace/events/thp.h>
+
+#ifdef CONFIG_SPARSEMEM_VMEMMAP
+/*
+ * On hash-based CPUs, the vmemmap is bolted in the hash table.
+ *
+ */
+int __meminit hash__vmemmap_create_mapping(unsigned long start,
+ unsigned long page_size,
+ unsigned long phys)
+{
+ int rc = htab_bolt_mapping(start, start + page_size, phys,
+ pgprot_val(PAGE_KERNEL),
+ mmu_vmemmap_psize, mmu_kernel_ssize);
+ if (rc < 0) {
+ int rc2 = htab_remove_mapping(start, start + page_size,
+ mmu_vmemmap_psize,
+ mmu_kernel_ssize);
+ BUG_ON(rc2 && (rc2 != -ENOENT));
+ }
+ return rc;
+}
+
+#ifdef CONFIG_MEMORY_HOTPLUG
+void hash__vmemmap_remove_mapping(unsigned long start,
+ unsigned long page_size)
+{
+ int rc = htab_remove_mapping(start, start + page_size,
+ mmu_vmemmap_psize,
+ mmu_kernel_ssize);
+ BUG_ON((rc < 0) && (rc != -ENOENT));
+ WARN_ON(rc == -ENOENT);
+}
+#endif
+#endif /* CONFIG_SPARSEMEM_VMEMMAP */
+
+/*
+ * map_kernel_page currently only called by __ioremap
+ * map_kernel_page adds an entry to the ioremap page table
+ * and adds an entry to the HPT, possibly bolting it
+ */
+int hash__map_kernel_page(unsigned long ea, unsigned long pa, unsigned long flags)
+{
+ pgd_t *pgdp;
+ pud_t *pudp;
+ pmd_t *pmdp;
+ pte_t *ptep;
+
+ BUILD_BUG_ON(TASK_SIZE_USER64 > H_PGTABLE_RANGE);
+ if (slab_is_available()) {
+ pgdp = pgd_offset_k(ea);
+ pudp = pud_alloc(&init_mm, pgdp, ea);
+ if (!pudp)
+ return -ENOMEM;
+ pmdp = pmd_alloc(&init_mm, pudp, ea);
+ if (!pmdp)
+ return -ENOMEM;
+ ptep = pte_alloc_kernel(pmdp, ea);
+ if (!ptep)
+ return -ENOMEM;
+ set_pte_at(&init_mm, ea, ptep, pfn_pte(pa >> PAGE_SHIFT,
+ __pgprot(flags)));
+ } else {
+ /*
+ * If the mm subsystem is not fully up, we cannot create a
+ * linux page table entry for this mapping. Simply bolt an
+ * entry in the hardware page table.
+ *
+ */
+ if (htab_bolt_mapping(ea, ea + PAGE_SIZE, pa, flags,
+ mmu_io_psize, mmu_kernel_ssize)) {
+ printk(KERN_ERR "Failed to do bolted mapping IO "
+ "memory at %016lx !\n", pa);
+ return -ENOMEM;
+ }
+ }
+
+ smp_wmb();
+ return 0;
+}
+
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+
+unsigned long hash__pmd_hugepage_update(struct mm_struct *mm, unsigned long addr,
+ pmd_t *pmdp, unsigned long clr,
+ unsigned long set)
+{
+ __be64 old_be, tmp;
+ unsigned long old;
+
+#ifdef CONFIG_DEBUG_VM
+ WARN_ON(!pmd_trans_huge(*pmdp));
+ assert_spin_locked(&mm->page_table_lock);
+#endif
+
+ __asm__ __volatile__(
+ "1: ldarx %0,0,%3\n\
+ and. %1,%0,%6\n\
+ bne- 1b \n\
+ andc %1,%0,%4 \n\
+ or %1,%1,%7\n\
+ stdcx. %1,0,%3 \n\
+ bne- 1b"
+ : "=&r" (old_be), "=&r" (tmp), "=m" (*pmdp)
+ : "r" (pmdp), "r" (cpu_to_be64(clr)), "m" (*pmdp),
+ "r" (cpu_to_be64(H_PAGE_BUSY)), "r" (cpu_to_be64(set))
+ : "cc" );
+
+ old = be64_to_cpu(old_be);
+
+ trace_hugepage_update(addr, old, clr, set);
+ if (old & H_PAGE_HASHPTE)
+ hpte_do_hugepage_flush(mm, addr, pmdp, old);
+ return old;
+}
+
+pmd_t hash__pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long address,
+ pmd_t *pmdp)
+{
+ pmd_t pmd;
+
+ VM_BUG_ON(address & ~HPAGE_PMD_MASK);
+ VM_BUG_ON(pmd_trans_huge(*pmdp));
+
+ pmd = *pmdp;
+ pmd_clear(pmdp);
+ /*
+ * Wait for all pending hash_page to finish. This is needed
+ * in case of subpage collapse. When we collapse normal pages
+ * to hugepage, we first clear the pmd, then invalidate all
+ * the PTE entries. The assumption here is that any low level
+ * page fault will see a none pmd and take the slow path that
+ * will wait on mmap_sem. But we could very well be in a
+ * hash_page with local ptep pointer value. Such a hash page
+ * can result in adding new HPTE entries for normal subpages.
+ * That means we could be modifying the page content as we
+ * copy them to a huge page. So wait for parallel hash_page
+ * to finish before invalidating HPTE entries. We can do this
+ * by sending an IPI to all the cpus and executing a dummy
+ * function there.
+ */
+ kick_all_cpus_sync();
+ /*
+ * Now invalidate the hpte entries in the range
+ * covered by pmd. This make sure we take a
+ * fault and will find the pmd as none, which will
+ * result in a major fault which takes mmap_sem and
+ * hence wait for collapse to complete. Without this
+ * the __collapse_huge_page_copy can result in copying
+ * the old content.
+ */
+ flush_tlb_pmd_range(vma->vm_mm, &pmd, address);
+ return pmd;
+}
+
+/*
+ * We want to put the pgtable in pmd and use pgtable for tracking
+ * the base page size hptes
+ */
+void hash__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,
+ pgtable_t pgtable)
+{
+ pgtable_t *pgtable_slot;
+ assert_spin_locked(&mm->page_table_lock);
+ /*
+ * we store the pgtable in the second half of PMD
+ */
+ pgtable_slot = (pgtable_t *)pmdp + PTRS_PER_PMD;
+ *pgtable_slot = pgtable;
+ /*
+ * expose the deposited pgtable to other cpus.
+ * before we set the hugepage PTE at pmd level
+ * hash fault code looks at the deposted pgtable
+ * to store hash index values.
+ */
+ smp_wmb();
+}
+
+pgtable_t hash__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp)
+{
+ pgtable_t pgtable;
+ pgtable_t *pgtable_slot;
+
+ assert_spin_locked(&mm->page_table_lock);
+ pgtable_slot = (pgtable_t *)pmdp + PTRS_PER_PMD;
+ pgtable = *pgtable_slot;
+ /*
+ * Once we withdraw, mark the entry NULL.
+ */
+ *pgtable_slot = NULL;
+ /*
+ * We store HPTE information in the deposited PTE fragment.
+ * zero out the content on withdraw.
+ */
+ memset(pgtable, 0, PTE_FRAG_SIZE);
+ return pgtable;
+}
+
+void hash__pmdp_huge_split_prepare(struct vm_area_struct *vma,
+ unsigned long address, pmd_t *pmdp)
+{
+ VM_BUG_ON(address & ~HPAGE_PMD_MASK);
+ VM_BUG_ON(REGION_ID(address) != USER_REGION_ID);
+
+ /*
+ * We can't mark the pmd none here, because that will cause a race
+ * against exit_mmap. We need to continue mark pmd TRANS HUGE, while
+ * we spilt, but at the same time we wan't rest of the ppc64 code
+ * not to insert hash pte on this, because we will be modifying
+ * the deposited pgtable in the caller of this function. Hence
+ * clear the _PAGE_USER so that we move the fault handling to
+ * higher level function and that will serialize against ptl.
+ * We need to flush existing hash pte entries here even though,
+ * the translation is still valid, because we will withdraw
+ * pgtable_t after this.
+ */
+ pmd_hugepage_update(vma->vm_mm, address, pmdp, 0, _PAGE_PRIVILEGED);
+}
+
+/*
+ * A linux hugepage PMD was changed and the corresponding hash table entries
+ * neesd to be flushed.
+ */
+void hpte_do_hugepage_flush(struct mm_struct *mm, unsigned long addr,
+ pmd_t *pmdp, unsigned long old_pmd)
+{
+ int ssize;
+ unsigned int psize;
+ unsigned long vsid;
+ unsigned long flags = 0;
+ const struct cpumask *tmp;
+
+ /* get the base page size,vsid and segment size */
+#ifdef CONFIG_DEBUG_VM
+ psize = get_slice_psize(mm, addr);
+ BUG_ON(psize == MMU_PAGE_16M);
+#endif
+ if (old_pmd & H_PAGE_COMBO)
+ psize = MMU_PAGE_4K;
+ else
+ psize = MMU_PAGE_64K;
+
+ if (!is_kernel_addr(addr)) {
+ ssize = user_segment_size(addr);
+ vsid = get_vsid(mm->context.id, addr, ssize);
+ WARN_ON(vsid == 0);
+ } else {
+ vsid = get_kernel_vsid(addr, mmu_kernel_ssize);
+ ssize = mmu_kernel_ssize;
+ }
+
+ tmp = cpumask_of(smp_processor_id());
+ if (cpumask_equal(mm_cpumask(mm), tmp))
+ flags |= HPTE_LOCAL_UPDATE;
+
+ return flush_hash_hugepage(vsid, addr, pmdp, psize, ssize, flags);
+}
+
+pmd_t hash__pmdp_huge_get_and_clear(struct mm_struct *mm,
+ unsigned long addr, pmd_t *pmdp)
+{
+ pmd_t old_pmd;
+ pgtable_t pgtable;
+ unsigned long old;
+ pgtable_t *pgtable_slot;
+
+ old = pmd_hugepage_update(mm, addr, pmdp, ~0UL, 0);
+ old_pmd = __pmd(old);
+ /*
+ * We have pmd == none and we are holding page_table_lock.
+ * So we can safely go and clear the pgtable hash
+ * index info.
+ */
+ pgtable_slot = (pgtable_t *)pmdp + PTRS_PER_PMD;
+ pgtable = *pgtable_slot;
+ /*
+ * Let's zero out old valid and hash index details
+ * hash fault look at them.
+ */
+ memset(pgtable, 0, PTE_FRAG_SIZE);
+ /*
+ * Serialize against find_linux_pte_or_hugepte which does lock-less
+ * lookup in page tables with local interrupts disabled. For huge pages
+ * it casts pmd_t to pte_t. Since format of pte_t is different from
+ * pmd_t we want to prevent transit from pmd pointing to page table
+ * to pmd pointing to huge page (and back) while interrupts are disabled.
+ * We clear pmd to possibly replace it with page table pointer in
+ * different code paths. So make sure we wait for the parallel
+ * find_linux_pte_or_hugepage to finish.
+ */
+ kick_all_cpus_sync();
+ return old_pmd;
+}
+
+int hash__has_transparent_hugepage(void)
+{
+
+ if (!mmu_has_feature(MMU_FTR_16M_PAGE))
+ return 0;
+ /*
+ * We support THP only if PMD_SIZE is 16MB.
+ */
+ if (mmu_psize_defs[MMU_PAGE_16M].shift != PMD_SHIFT)
+ return 0;
+ /*
+ * We need to make sure that we support 16MB hugepage in a segement
+ * with base page size 64K or 4K. We only enable THP with a PAGE_SIZE
+ * of 64K.
+ */
+ /*
+ * If we have 64K HPTE, we will be using that by default
+ */
+ if (mmu_psize_defs[MMU_PAGE_64K].shift &&
+ (mmu_psize_defs[MMU_PAGE_64K].penc[MMU_PAGE_16M] == -1))
+ return 0;
+ /*
+ * Ok we only have 4K HPTE
+ */
+ if (mmu_psize_defs[MMU_PAGE_4K].penc[MMU_PAGE_16M] == -1)
+ return 0;
+
+ return 1;
+}
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
diff --git a/arch/powerpc/mm/pgtable-radix.c b/arch/powerpc/mm/pgtable-radix.c
new file mode 100644
index 0000000..18b2c11
--- /dev/null
+++ b/arch/powerpc/mm/pgtable-radix.c
@@ -0,0 +1,526 @@
+/*
+ * Page table handling routines for radix page table.
+ *
+ * Copyright 2015-2016, Aneesh Kumar K.V, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+#include <linux/sched.h>
+#include <linux/memblock.h>
+#include <linux/of_fdt.h>
+
+#include <asm/pgtable.h>
+#include <asm/pgalloc.h>
+#include <asm/dma.h>
+#include <asm/machdep.h>
+#include <asm/mmu.h>
+#include <asm/firmware.h>
+
+#include <trace/events/thp.h>
+
+static int native_update_partition_table(u64 patb1)
+{
+ partition_tb->patb1 = cpu_to_be64(patb1);
+ return 0;
+}
+
+static __ref void *early_alloc_pgtable(unsigned long size)
+{
+ void *pt;
+
+ pt = __va(memblock_alloc_base(size, size, MEMBLOCK_ALLOC_ANYWHERE));
+ memset(pt, 0, size);
+
+ return pt;
+}
+
+int radix__map_kernel_page(unsigned long ea, unsigned long pa,
+ pgprot_t flags,
+ unsigned int map_page_size)
+{
+ pgd_t *pgdp;
+ pud_t *pudp;
+ pmd_t *pmdp;
+ pte_t *ptep;
+ /*
+ * Make sure task size is correct as per the max adddr
+ */
+ BUILD_BUG_ON(TASK_SIZE_USER64 > RADIX_PGTABLE_RANGE);
+ if (slab_is_available()) {
+ pgdp = pgd_offset_k(ea);
+ pudp = pud_alloc(&init_mm, pgdp, ea);
+ if (!pudp)
+ return -ENOMEM;
+ if (map_page_size == PUD_SIZE) {
+ ptep = (pte_t *)pudp;
+ goto set_the_pte;
+ }
+ pmdp = pmd_alloc(&init_mm, pudp, ea);
+ if (!pmdp)
+ return -ENOMEM;
+ if (map_page_size == PMD_SIZE) {
+ ptep = (pte_t *)pudp;
+ goto set_the_pte;
+ }
+ ptep = pte_alloc_kernel(pmdp, ea);
+ if (!ptep)
+ return -ENOMEM;
+ } else {
+ pgdp = pgd_offset_k(ea);
+ if (pgd_none(*pgdp)) {
+ pudp = early_alloc_pgtable(PUD_TABLE_SIZE);
+ BUG_ON(pudp == NULL);
+ pgd_populate(&init_mm, pgdp, pudp);
+ }
+ pudp = pud_offset(pgdp, ea);
+ if (map_page_size == PUD_SIZE) {
+ ptep = (pte_t *)pudp;
+ goto set_the_pte;
+ }
+ if (pud_none(*pudp)) {
+ pmdp = early_alloc_pgtable(PMD_TABLE_SIZE);
+ BUG_ON(pmdp == NULL);
+ pud_populate(&init_mm, pudp, pmdp);
+ }
+ pmdp = pmd_offset(pudp, ea);
+ if (map_page_size == PMD_SIZE) {
+ ptep = (pte_t *)pudp;
+ goto set_the_pte;
+ }
+ if (!pmd_present(*pmdp)) {
+ ptep = early_alloc_pgtable(PAGE_SIZE);
+ BUG_ON(ptep == NULL);
+ pmd_populate_kernel(&init_mm, pmdp, ptep);
+ }
+ ptep = pte_offset_kernel(pmdp, ea);
+ }
+
+set_the_pte:
+ set_pte_at(&init_mm, ea, ptep, pfn_pte(pa >> PAGE_SHIFT, flags));
+ smp_wmb();
+ return 0;
+}
+
+static void __init radix_init_pgtable(void)
+{
+ int loop_count;
+ u64 base, end, start_addr;
+ unsigned long rts_field;
+ struct memblock_region *reg;
+ unsigned long linear_page_size;
+
+ /* We don't support slb for radix */
+ mmu_slb_size = 0;
+ /*
+ * Create the linear mapping, using standard page size for now
+ */
+ loop_count = 0;
+ for_each_memblock(memory, reg) {
+
+ start_addr = reg->base;
+
+redo:
+ if (loop_count < 1 && mmu_psize_defs[MMU_PAGE_1G].shift)
+ linear_page_size = PUD_SIZE;
+ else if (loop_count < 2 && mmu_psize_defs[MMU_PAGE_2M].shift)
+ linear_page_size = PMD_SIZE;
+ else
+ linear_page_size = PAGE_SIZE;
+
+ base = _ALIGN_UP(start_addr, linear_page_size);
+ end = _ALIGN_DOWN(reg->base + reg->size, linear_page_size);
+
+ pr_info("Mapping range 0x%lx - 0x%lx with 0x%lx\n",
+ (unsigned long)base, (unsigned long)end,
+ linear_page_size);
+
+ while (base < end) {
+ radix__map_kernel_page((unsigned long)__va(base),
+ base, PAGE_KERNEL_X,
+ linear_page_size);
+ base += linear_page_size;
+ }
+ /*
+ * map the rest using lower page size
+ */
+ if (end < reg->base + reg->size) {
+ start_addr = end;
+ loop_count++;
+ goto redo;
+ }
+ }
+ /*
+ * Allocate Partition table and process table for the
+ * host.
+ */
+ BUILD_BUG_ON_MSG((PRTB_SIZE_SHIFT > 23), "Process table size too large.");
+ process_tb = early_alloc_pgtable(1UL << PRTB_SIZE_SHIFT);
+ /*
+ * Fill in the process table.
+ * we support 52 bits, hence 52-28 = 24, 11000
+ */
+ rts_field = 3ull << PPC_BITLSHIFT(2);
+ process_tb->prtb0 = cpu_to_be64(rts_field | __pa(init_mm.pgd) | RADIX_PGD_INDEX_SIZE);
+ /*
+ * Fill in the partition table. We are suppose to use effective address
+ * of process table here. But our linear mapping also enable us to use
+ * physical address here.
+ */
+ ppc_md.update_partition_table(__pa(process_tb) | (PRTB_SIZE_SHIFT - 12) | PATB_GR);
+ pr_info("Process table %p and radix root for kernel: %p\n", process_tb, init_mm.pgd);
+}
+
+static void __init radix_init_partition_table(void)
+{
+ unsigned long rts_field;
+ /*
+ * we support 52 bits, hence 52-28 = 24, 11000
+ */
+ rts_field = 3ull << PPC_BITLSHIFT(2);
+
+ BUILD_BUG_ON_MSG((PATB_SIZE_SHIFT > 24), "Partition table size too large.");
+ partition_tb = early_alloc_pgtable(1UL << PATB_SIZE_SHIFT);
+ partition_tb->patb0 = cpu_to_be64(rts_field | __pa(init_mm.pgd) |
+ RADIX_PGD_INDEX_SIZE | PATB_HR);
+ printk("Partition table %p\n", partition_tb);
+
+ memblock_set_current_limit(MEMBLOCK_ALLOC_ANYWHERE);
+ /*
+ * update partition table control register,
+ * 64 K size.
+ */
+ mtspr(SPRN_PTCR, __pa(partition_tb) | (PATB_SIZE_SHIFT - 12));
+}
+
+void __init radix_init_native(void)
+{
+ ppc_md.update_partition_table = native_update_partition_table;
+}
+
+static int __init get_idx_from_shift(unsigned int shift)
+{
+ int idx = -1;
+
+ switch (shift) {
+ case 0xc:
+ idx = MMU_PAGE_4K;
+ break;
+ case 0x10:
+ idx = MMU_PAGE_64K;
+ break;
+ case 0x15:
+ idx = MMU_PAGE_2M;
+ break;
+ case 0x1e:
+ idx = MMU_PAGE_1G;
+ break;
+ }
+ return idx;
+}
+
+static int __init radix_dt_scan_page_sizes(unsigned long node,
+ const char *uname, int depth,
+ void *data)
+{
+ int size = 0;
+ int shift, idx;
+ unsigned int ap;
+ const __be32 *prop;
+ const char *type = of_get_flat_dt_prop(node, "device_type", NULL);
+
+ /* We are scanning "cpu" nodes only */
+ if (type == NULL || strcmp(type, "cpu") != 0)
+ return 0;
+
+ prop = of_get_flat_dt_prop(node, "ibm,processor-radix-AP-encodings", &size);
+ if (!prop)
+ return 0;
+
+ pr_info("Page sizes from device-tree:\n");
+ for (; size >= 4; size -= 4, ++prop) {
+
+ struct mmu_psize_def *def;
+
+ /* top 3 bit is AP encoding */
+ shift = be32_to_cpu(prop[0]) & ~(0xe << 28);
+ ap = be32_to_cpu(prop[0]) >> 29;
+ pr_info("Page size sift = %d AP=0x%x\n", shift, ap);
+
+ idx = get_idx_from_shift(shift);
+ if (idx < 0)
+ continue;
+
+ def = &mmu_psize_defs[idx];
+ def->shift = shift;
+ def->ap = ap;
+ }
+
+ /* needed ? */
+ cur_cpu_spec->mmu_features &= ~MMU_FTR_NO_SLBIE_B;
+ return 1;
+}
+
+static void __init radix_init_page_sizes(void)
+{
+ int rc;
+
+ /*
+ * Try to find the available page sizes in the device-tree
+ */
+ rc = of_scan_flat_dt(radix_dt_scan_page_sizes, NULL);
+ if (rc != 0) /* Found */
+ goto found;
+ /*
+ * let's assume we have page 4k and 64k support
+ */
+ mmu_psize_defs[MMU_PAGE_4K].shift = 12;
+ mmu_psize_defs[MMU_PAGE_4K].ap = 0x0;
+
+ mmu_psize_defs[MMU_PAGE_64K].shift = 16;
+ mmu_psize_defs[MMU_PAGE_64K].ap = 0x5;
+found:
+#ifdef CONFIG_SPARSEMEM_VMEMMAP
+ if (mmu_psize_defs[MMU_PAGE_2M].shift) {
+ /*
+ * map vmemmap using 2M if available
+ */
+ mmu_vmemmap_psize = MMU_PAGE_2M;
+ }
+#endif /* CONFIG_SPARSEMEM_VMEMMAP */
+ return;
+}
+
+void __init radix__early_init_mmu(void)
+{
+ unsigned long lpcr;
+ /*
+ * setup LPCR UPRT based on mmu_features
+ */
+ lpcr = mfspr(SPRN_LPCR);
+ mtspr(SPRN_LPCR, lpcr | LPCR_UPRT);
+
+#ifdef CONFIG_PPC_64K_PAGES
+ /* PAGE_SIZE mappings */
+ mmu_virtual_psize = MMU_PAGE_64K;
+#else
+ mmu_virtual_psize = MMU_PAGE_4K;
+#endif
+
+#ifdef CONFIG_SPARSEMEM_VMEMMAP
+ /* vmemmap mapping */
+ mmu_vmemmap_psize = mmu_virtual_psize;
+#endif
+ /*
+ * initialize page table size
+ */
+ __pte_index_size = RADIX_PTE_INDEX_SIZE;
+ __pmd_index_size = RADIX_PMD_INDEX_SIZE;
+ __pud_index_size = RADIX_PUD_INDEX_SIZE;
+ __pgd_index_size = RADIX_PGD_INDEX_SIZE;
+ __pmd_cache_index = RADIX_PMD_INDEX_SIZE;
+ __pte_table_size = RADIX_PTE_TABLE_SIZE;
+ __pmd_table_size = RADIX_PMD_TABLE_SIZE;
+ __pud_table_size = RADIX_PUD_TABLE_SIZE;
+ __pgd_table_size = RADIX_PGD_TABLE_SIZE;
+
+ __pmd_val_bits = RADIX_PMD_VAL_BITS;
+ __pud_val_bits = RADIX_PUD_VAL_BITS;
+ __pgd_val_bits = RADIX_PGD_VAL_BITS;
+
+ __kernel_virt_start = RADIX_KERN_VIRT_START;
+ __kernel_virt_size = RADIX_KERN_VIRT_SIZE;
+ __vmalloc_start = RADIX_VMALLOC_START;
+ __vmalloc_end = RADIX_VMALLOC_END;
+ vmemmap = (struct page *)RADIX_VMEMMAP_BASE;
+ ioremap_bot = IOREMAP_BASE;
+ /*
+ * For now radix also use the same frag size
+ */
+ __pte_frag_nr = H_PTE_FRAG_NR;
+ __pte_frag_size_shift = H_PTE_FRAG_SIZE_SHIFT;
+
+ radix_init_page_sizes();
+ if (!firmware_has_feature(FW_FEATURE_LPAR))
+ radix_init_partition_table();
+
+ radix_init_pgtable();
+}
+
+void radix__early_init_mmu_secondary(void)
+{
+ unsigned long lpcr;
+ /*
+ * setup LPCR UPRT based on mmu_features
+ */
+ lpcr = mfspr(SPRN_LPCR);
+ mtspr(SPRN_LPCR, lpcr | LPCR_UPRT);
+ /*
+ * update partition table control register, 64 K size.
+ */
+ if (!firmware_has_feature(FW_FEATURE_LPAR))
+ mtspr(SPRN_PTCR,
+ __pa(partition_tb) | (PATB_SIZE_SHIFT - 12));
+}
+
+void radix__setup_initial_memory_limit(phys_addr_t first_memblock_base,
+ phys_addr_t first_memblock_size)
+{
+ /* We don't currently support the first MEMBLOCK not mapping 0
+ * physical on those processors
+ */
+ BUG_ON(first_memblock_base != 0);
+ /*
+ * We limit the allocation that depend on ppc64_rma_size
+ * to first_memblock_size. We also clamp it to 1GB to
+ * avoid some funky things such as RTAS bugs.
+ *
+ * On radix config we really don't have a limitation
+ * on real mode access. But keeping it as above works
+ * well enough.
+ */
+ ppc64_rma_size = min_t(u64, first_memblock_size, 0x40000000);
+ /*
+ * Finally limit subsequent allocations. We really don't want
+ * to limit the memblock allocations to rma_size. FIXME!! should
+ * we even limit at all ?
+ */
+ memblock_set_current_limit(first_memblock_base + first_memblock_size);
+}
+
+#ifdef CONFIG_SPARSEMEM_VMEMMAP
+int __meminit radix__vmemmap_create_mapping(unsigned long start,
+ unsigned long page_size,
+ unsigned long phys)
+{
+ /* Create a PTE encoding */
+ unsigned long flags = _PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_KERNEL_RW;
+
+ BUG_ON(radix__map_kernel_page(start, phys, __pgprot(flags), page_size));
+ return 0;
+}
+
+#ifdef CONFIG_MEMORY_HOTPLUG
+void radix__vmemmap_remove_mapping(unsigned long start, unsigned long page_size)
+{
+ /* FIXME!! intel does more. We should free page tables mapping vmemmap ? */
+}
+#endif
+#endif
+
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+
+unsigned long radix__pmd_hugepage_update(struct mm_struct *mm, unsigned long addr,
+ pmd_t *pmdp, unsigned long clr,
+ unsigned long set)
+{
+ unsigned long old;
+
+#ifdef CONFIG_DEBUG_VM
+ WARN_ON(!radix__pmd_trans_huge(*pmdp));
+ assert_spin_locked(&mm->page_table_lock);
+#endif
+
+ old = radix__pte_update(mm, addr, (pte_t *)pmdp, clr, set, 1);
+ trace_hugepage_update(addr, old, clr, set);
+
+ return old;
+}
+
+pmd_t radix__pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long address,
+ pmd_t *pmdp)
+
+{
+ pmd_t pmd;
+
+ VM_BUG_ON(address & ~HPAGE_PMD_MASK);
+ VM_BUG_ON(radix__pmd_trans_huge(*pmdp));
+ /*
+ * khugepaged calls this for normal pmd
+ */
+ pmd = *pmdp;
+ pmd_clear(pmdp);
+ /*FIXME!! Verify whether we need this kick below */
+ kick_all_cpus_sync();
+ flush_tlb_range(vma, address, address + HPAGE_PMD_SIZE);
+ return pmd;
+}
+
+/*
+ * For us pgtable_t is pte_t *. Inorder to save the deposisted
+ * page table, we consider the allocated page table as a list
+ * head. On withdraw we need to make sure we zero out the used
+ * list_head memory area.
+ */
+void radix__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,
+ pgtable_t pgtable)
+{
+ struct list_head *lh = (struct list_head *) pgtable;
+
+ assert_spin_locked(pmd_lockptr(mm, pmdp));
+
+ /* FIFO */
+ if (!pmd_huge_pte(mm, pmdp))
+ INIT_LIST_HEAD(lh);
+ else
+ list_add(lh, (struct list_head *) pmd_huge_pte(mm, pmdp));
+ pmd_huge_pte(mm, pmdp) = pgtable;
+}
+
+pgtable_t radix__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp)
+{
+ pte_t *ptep;
+ pgtable_t pgtable;
+ struct list_head *lh;
+
+ assert_spin_locked(pmd_lockptr(mm, pmdp));
+
+ /* FIFO */
+ pgtable = pmd_huge_pte(mm, pmdp);
+ lh = (struct list_head *) pgtable;
+ if (list_empty(lh))
+ pmd_huge_pte(mm, pmdp) = NULL;
+ else {
+ pmd_huge_pte(mm, pmdp) = (pgtable_t) lh->next;
+ list_del(lh);
+ }
+ ptep = (pte_t *) pgtable;
+ *ptep = __pte(0);
+ ptep++;
+ *ptep = __pte(0);
+ return pgtable;
+}
+
+
+pmd_t radix__pmdp_huge_get_and_clear(struct mm_struct *mm,
+ unsigned long addr, pmd_t *pmdp)
+{
+ pmd_t old_pmd;
+ unsigned long old;
+
+ old = radix__pmd_hugepage_update(mm, addr, pmdp, ~0UL, 0);
+ old_pmd = __pmd(old);
+ /*
+ * Serialize against find_linux_pte_or_hugepte which does lock-less
+ * lookup in page tables with local interrupts disabled. For huge pages
+ * it casts pmd_t to pte_t. Since format of pte_t is different from
+ * pmd_t we want to prevent transit from pmd pointing to page table
+ * to pmd pointing to huge page (and back) while interrupts are disabled.
+ * We clear pmd to possibly replace it with page table pointer in
+ * different code paths. So make sure we wait for the parallel
+ * find_linux_pte_or_hugepage to finish.
+ */
+ kick_all_cpus_sync();
+ return old_pmd;
+}
+
+int radix__has_transparent_hugepage(void)
+{
+ /* For radix 2M at PMD level means thp */
+ if (mmu_psize_defs[MMU_PAGE_2M].shift == PMD_SHIFT)
+ return 1;
+ return 0;
+}
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c
index de37ff4..88a3075 100644
--- a/arch/powerpc/mm/pgtable.c
+++ b/arch/powerpc/mm/pgtable.c
@@ -38,14 +38,25 @@
/* We only try to do i/d cache coherency on stuff that looks like
* reasonably "normal" PTEs. We currently require a PTE to be present
- * and we avoid _PAGE_SPECIAL and _PAGE_NO_CACHE. We also only do that
+ * and we avoid _PAGE_SPECIAL and cache inhibited pte. We also only do that
* on userspace PTEs
*/
static inline int pte_looks_normal(pte_t pte)
{
+
+#if defined(CONFIG_PPC_BOOK3S_64)
+ if ((pte_val(pte) & (_PAGE_PRESENT | _PAGE_SPECIAL)) == _PAGE_PRESENT) {
+ if (pte_ci(pte))
+ return 0;
+ if (pte_user(pte))
+ return 1;
+ }
+ return 0;
+#else
return (pte_val(pte) &
- (_PAGE_PRESENT | _PAGE_SPECIAL | _PAGE_NO_CACHE | _PAGE_USER)) ==
- (_PAGE_PRESENT | _PAGE_USER);
+ (_PAGE_PRESENT | _PAGE_SPECIAL | _PAGE_NO_CACHE | _PAGE_USER)) ==
+ (_PAGE_PRESENT | _PAGE_USER);
+#endif
}
static struct page *maybe_pte_to_page(pte_t pte)
@@ -71,6 +82,9 @@
static pte_t set_pte_filter(pte_t pte)
{
+ if (radix_enabled())
+ return pte;
+
pte = __pte(pte_val(pte) & ~_PAGE_HPTEFLAGS);
if (pte_looks_normal(pte) && !(cpu_has_feature(CPU_FTR_COHERENT_ICACHE) ||
cpu_has_feature(CPU_FTR_NOEXECUTE))) {
@@ -177,8 +191,8 @@
* _PAGE_PRESENT, but we can be sure that it is not in hpte.
* Hence we can use set_pte_at for them.
*/
- VM_WARN_ON((pte_val(*ptep) & (_PAGE_PRESENT | _PAGE_USER)) ==
- (_PAGE_PRESENT | _PAGE_USER));
+ VM_WARN_ON(pte_present(*ptep) && !pte_protnone(*ptep));
+
/*
* Add the pte bit when tryint set a pte
*/
diff --git a/arch/powerpc/mm/pgtable_64.c b/arch/powerpc/mm/pgtable_64.c
index 3471060..e009e06 100644
--- a/arch/powerpc/mm/pgtable_64.c
+++ b/arch/powerpc/mm/pgtable_64.c
@@ -55,104 +55,63 @@
#include "mmu_decl.h"
-#define CREATE_TRACE_POINTS
-#include <trace/events/thp.h>
-
-/* Some sanity checking */
-#if TASK_SIZE_USER64 > PGTABLE_RANGE
-#error TASK_SIZE_USER64 exceeds pagetable range
-#endif
-
#ifdef CONFIG_PPC_STD_MMU_64
#if TASK_SIZE_USER64 > (1UL << (ESID_BITS + SID_SHIFT))
#error TASK_SIZE_USER64 exceeds user VSID range
#endif
#endif
-unsigned long ioremap_bot = IOREMAP_BASE;
-
-#ifdef CONFIG_PPC_MMU_NOHASH
-static __ref void *early_alloc_pgtable(unsigned long size)
-{
- void *pt;
-
- pt = __va(memblock_alloc_base(size, size, __pa(MAX_DMA_ADDRESS)));
- memset(pt, 0, size);
-
- return pt;
-}
-#endif /* CONFIG_PPC_MMU_NOHASH */
-
+#ifdef CONFIG_PPC_BOOK3S_64
/*
- * map_kernel_page currently only called by __ioremap
- * map_kernel_page adds an entry to the ioremap page table
- * and adds an entry to the HPT, possibly bolting it
+ * partition table and process table for ISA 3.0
*/
-int map_kernel_page(unsigned long ea, unsigned long pa, unsigned long flags)
-{
- pgd_t *pgdp;
- pud_t *pudp;
- pmd_t *pmdp;
- pte_t *ptep;
-
- if (slab_is_available()) {
- pgdp = pgd_offset_k(ea);
- pudp = pud_alloc(&init_mm, pgdp, ea);
- if (!pudp)
- return -ENOMEM;
- pmdp = pmd_alloc(&init_mm, pudp, ea);
- if (!pmdp)
- return -ENOMEM;
- ptep = pte_alloc_kernel(pmdp, ea);
- if (!ptep)
- return -ENOMEM;
- set_pte_at(&init_mm, ea, ptep, pfn_pte(pa >> PAGE_SHIFT,
- __pgprot(flags)));
- } else {
-#ifdef CONFIG_PPC_MMU_NOHASH
- pgdp = pgd_offset_k(ea);
-#ifdef PUD_TABLE_SIZE
- if (pgd_none(*pgdp)) {
- pudp = early_alloc_pgtable(PUD_TABLE_SIZE);
- BUG_ON(pudp == NULL);
- pgd_populate(&init_mm, pgdp, pudp);
- }
-#endif /* PUD_TABLE_SIZE */
- pudp = pud_offset(pgdp, ea);
- if (pud_none(*pudp)) {
- pmdp = early_alloc_pgtable(PMD_TABLE_SIZE);
- BUG_ON(pmdp == NULL);
- pud_populate(&init_mm, pudp, pmdp);
- }
- pmdp = pmd_offset(pudp, ea);
- if (!pmd_present(*pmdp)) {
- ptep = early_alloc_pgtable(PAGE_SIZE);
- BUG_ON(ptep == NULL);
- pmd_populate_kernel(&init_mm, pmdp, ptep);
- }
- ptep = pte_offset_kernel(pmdp, ea);
- set_pte_at(&init_mm, ea, ptep, pfn_pte(pa >> PAGE_SHIFT,
- __pgprot(flags)));
-#else /* CONFIG_PPC_MMU_NOHASH */
- /*
- * If the mm subsystem is not fully up, we cannot create a
- * linux page table entry for this mapping. Simply bolt an
- * entry in the hardware page table.
- *
- */
- if (htab_bolt_mapping(ea, ea + PAGE_SIZE, pa, flags,
- mmu_io_psize, mmu_kernel_ssize)) {
- printk(KERN_ERR "Failed to do bolted mapping IO "
- "memory at %016lx !\n", pa);
- return -ENOMEM;
- }
-#endif /* !CONFIG_PPC_MMU_NOHASH */
- }
-
- smp_wmb();
- return 0;
-}
-
+struct prtb_entry *process_tb;
+struct patb_entry *partition_tb;
+/*
+ * page table size
+ */
+unsigned long __pte_index_size;
+EXPORT_SYMBOL(__pte_index_size);
+unsigned long __pmd_index_size;
+EXPORT_SYMBOL(__pmd_index_size);
+unsigned long __pud_index_size;
+EXPORT_SYMBOL(__pud_index_size);
+unsigned long __pgd_index_size;
+EXPORT_SYMBOL(__pgd_index_size);
+unsigned long __pmd_cache_index;
+EXPORT_SYMBOL(__pmd_cache_index);
+unsigned long __pte_table_size;
+EXPORT_SYMBOL(__pte_table_size);
+unsigned long __pmd_table_size;
+EXPORT_SYMBOL(__pmd_table_size);
+unsigned long __pud_table_size;
+EXPORT_SYMBOL(__pud_table_size);
+unsigned long __pgd_table_size;
+EXPORT_SYMBOL(__pgd_table_size);
+unsigned long __pmd_val_bits;
+EXPORT_SYMBOL(__pmd_val_bits);
+unsigned long __pud_val_bits;
+EXPORT_SYMBOL(__pud_val_bits);
+unsigned long __pgd_val_bits;
+EXPORT_SYMBOL(__pgd_val_bits);
+unsigned long __kernel_virt_start;
+EXPORT_SYMBOL(__kernel_virt_start);
+unsigned long __kernel_virt_size;
+EXPORT_SYMBOL(__kernel_virt_size);
+unsigned long __vmalloc_start;
+EXPORT_SYMBOL(__vmalloc_start);
+unsigned long __vmalloc_end;
+EXPORT_SYMBOL(__vmalloc_end);
+struct page *vmemmap;
+EXPORT_SYMBOL(vmemmap);
+unsigned long __pte_frag_nr;
+EXPORT_SYMBOL(__pte_frag_nr);
+unsigned long __pte_frag_size_shift;
+EXPORT_SYMBOL(__pte_frag_size_shift);
+unsigned long ioremap_bot;
+#else /* !CONFIG_PPC_BOOK3S_64 */
+unsigned long ioremap_bot = IOREMAP_BASE;
+#endif
/**
* __ioremap_at - Low level function to establish the page tables
@@ -167,12 +126,8 @@
if ((flags & _PAGE_PRESENT) == 0)
flags |= pgprot_val(PAGE_KERNEL);
- /* Non-cacheable page cannot be coherent */
- if (flags & _PAGE_NO_CACHE)
- flags &= ~_PAGE_COHERENT;
-
/* We don't support the 4K PFN hack with ioremap */
- if (flags & _PAGE_4K_PFN)
+ if (flags & H_PAGE_4K_PFN)
return NULL;
WARN_ON(pa & ~PAGE_MASK);
@@ -253,7 +208,7 @@
void __iomem * ioremap(phys_addr_t addr, unsigned long size)
{
- unsigned long flags = _PAGE_NO_CACHE | _PAGE_GUARDED;
+ unsigned long flags = pgprot_val(pgprot_noncached(__pgprot(0)));
void *caller = __builtin_return_address(0);
if (ppc_md.ioremap)
@@ -263,7 +218,7 @@
void __iomem * ioremap_wc(phys_addr_t addr, unsigned long size)
{
- unsigned long flags = _PAGE_NO_CACHE;
+ unsigned long flags = pgprot_val(pgprot_noncached_wc(__pgprot(0)));
void *caller = __builtin_return_address(0);
if (ppc_md.ioremap)
@@ -277,11 +232,20 @@
void *caller = __builtin_return_address(0);
/* writeable implies dirty for kernel addresses */
- if (flags & _PAGE_RW)
+ if (flags & _PAGE_WRITE)
flags |= _PAGE_DIRTY;
- /* we don't want to let _PAGE_USER and _PAGE_EXEC leak out */
- flags &= ~(_PAGE_USER | _PAGE_EXEC);
+ /* we don't want to let _PAGE_EXEC leak out */
+ flags &= ~_PAGE_EXEC;
+ /*
+ * Force kernel mapping.
+ */
+#if defined(CONFIG_PPC_BOOK3S_64)
+ flags |= _PAGE_PRIVILEGED;
+#else
+ flags &= ~_PAGE_USER;
+#endif
+
#ifdef _PAGE_BAP_SR
/* _PAGE_USER contains _PAGE_BAP_SR on BookE using the new PTE format
@@ -411,7 +375,7 @@
return (pte_t *)ret;
}
-pte_t *page_table_alloc(struct mm_struct *mm, unsigned long vmaddr, int kernel)
+pte_t *pte_fragment_alloc(struct mm_struct *mm, unsigned long vmaddr, int kernel)
{
pte_t *pte;
@@ -421,8 +385,9 @@
return __alloc_for_cache(mm, kernel);
}
+#endif /* CONFIG_PPC_64K_PAGES */
-void page_table_free(struct mm_struct *mm, unsigned long *table, int kernel)
+void pte_fragment_free(unsigned long *table, int kernel)
{
struct page *page = virt_to_page(table);
if (put_page_testzero(page)) {
@@ -433,15 +398,6 @@
}
#ifdef CONFIG_SMP
-static void page_table_free_rcu(void *table)
-{
- struct page *page = virt_to_page(table);
- if (put_page_testzero(page)) {
- pgtable_page_dtor(page);
- free_hot_cold_page(page, 0);
- }
-}
-
void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int shift)
{
unsigned long pgf = (unsigned long)table;
@@ -458,7 +414,7 @@
if (!shift)
/* PTE page needs special handling */
- page_table_free_rcu(table);
+ pte_fragment_free(table, 0);
else {
BUG_ON(shift > MAX_PGTABLE_INDEX_SIZE);
kmem_cache_free(PGT_CACHE(shift), table);
@@ -469,385 +425,10 @@
{
if (!shift) {
/* PTE page needs special handling */
- struct page *page = virt_to_page(table);
- if (put_page_testzero(page)) {
- pgtable_page_dtor(page);
- free_hot_cold_page(page, 0);
- }
+ pte_fragment_free(table, 0);
} else {
BUG_ON(shift > MAX_PGTABLE_INDEX_SIZE);
kmem_cache_free(PGT_CACHE(shift), table);
}
}
#endif
-#endif /* CONFIG_PPC_64K_PAGES */
-
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-
-/*
- * This is called when relaxing access to a hugepage. It's also called in the page
- * fault path when we don't hit any of the major fault cases, ie, a minor
- * update of _PAGE_ACCESSED, _PAGE_DIRTY, etc... The generic code will have
- * handled those two for us, we additionally deal with missing execute
- * permission here on some processors
- */
-int pmdp_set_access_flags(struct vm_area_struct *vma, unsigned long address,
- pmd_t *pmdp, pmd_t entry, int dirty)
-{
- int changed;
-#ifdef CONFIG_DEBUG_VM
- WARN_ON(!pmd_trans_huge(*pmdp));
- assert_spin_locked(&vma->vm_mm->page_table_lock);
-#endif
- changed = !pmd_same(*(pmdp), entry);
- if (changed) {
- __ptep_set_access_flags(pmdp_ptep(pmdp), pmd_pte(entry));
- /*
- * Since we are not supporting SW TLB systems, we don't
- * have any thing similar to flush_tlb_page_nohash()
- */
- }
- return changed;
-}
-
-unsigned long pmd_hugepage_update(struct mm_struct *mm, unsigned long addr,
- pmd_t *pmdp, unsigned long clr,
- unsigned long set)
-{
-
- unsigned long old, tmp;
-
-#ifdef CONFIG_DEBUG_VM
- WARN_ON(!pmd_trans_huge(*pmdp));
- assert_spin_locked(&mm->page_table_lock);
-#endif
-
-#ifdef PTE_ATOMIC_UPDATES
- __asm__ __volatile__(
- "1: ldarx %0,0,%3\n\
- andi. %1,%0,%6\n\
- bne- 1b \n\
- andc %1,%0,%4 \n\
- or %1,%1,%7\n\
- stdcx. %1,0,%3 \n\
- bne- 1b"
- : "=&r" (old), "=&r" (tmp), "=m" (*pmdp)
- : "r" (pmdp), "r" (clr), "m" (*pmdp), "i" (_PAGE_BUSY), "r" (set)
- : "cc" );
-#else
- old = pmd_val(*pmdp);
- *pmdp = __pmd((old & ~clr) | set);
-#endif
- trace_hugepage_update(addr, old, clr, set);
- if (old & _PAGE_HASHPTE)
- hpte_do_hugepage_flush(mm, addr, pmdp, old);
- return old;
-}
-
-pmd_t pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long address,
- pmd_t *pmdp)
-{
- pmd_t pmd;
-
- VM_BUG_ON(address & ~HPAGE_PMD_MASK);
- VM_BUG_ON(pmd_trans_huge(*pmdp));
-
- pmd = *pmdp;
- pmd_clear(pmdp);
- /*
- * Wait for all pending hash_page to finish. This is needed
- * in case of subpage collapse. When we collapse normal pages
- * to hugepage, we first clear the pmd, then invalidate all
- * the PTE entries. The assumption here is that any low level
- * page fault will see a none pmd and take the slow path that
- * will wait on mmap_sem. But we could very well be in a
- * hash_page with local ptep pointer value. Such a hash page
- * can result in adding new HPTE entries for normal subpages.
- * That means we could be modifying the page content as we
- * copy them to a huge page. So wait for parallel hash_page
- * to finish before invalidating HPTE entries. We can do this
- * by sending an IPI to all the cpus and executing a dummy
- * function there.
- */
- kick_all_cpus_sync();
- /*
- * Now invalidate the hpte entries in the range
- * covered by pmd. This make sure we take a
- * fault and will find the pmd as none, which will
- * result in a major fault which takes mmap_sem and
- * hence wait for collapse to complete. Without this
- * the __collapse_huge_page_copy can result in copying
- * the old content.
- */
- flush_tlb_pmd_range(vma->vm_mm, &pmd, address);
- return pmd;
-}
-
-int pmdp_test_and_clear_young(struct vm_area_struct *vma,
- unsigned long address, pmd_t *pmdp)
-{
- return __pmdp_test_and_clear_young(vma->vm_mm, address, pmdp);
-}
-
-/*
- * We currently remove entries from the hashtable regardless of whether
- * the entry was young or dirty. The generic routines only flush if the
- * entry was young or dirty which is not good enough.
- *
- * We should be more intelligent about this but for the moment we override
- * these functions and force a tlb flush unconditionally
- */
-int pmdp_clear_flush_young(struct vm_area_struct *vma,
- unsigned long address, pmd_t *pmdp)
-{
- return __pmdp_test_and_clear_young(vma->vm_mm, address, pmdp);
-}
-
-/*
- * We want to put the pgtable in pmd and use pgtable for tracking
- * the base page size hptes
- */
-void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,
- pgtable_t pgtable)
-{
- pgtable_t *pgtable_slot;
- assert_spin_locked(&mm->page_table_lock);
- /*
- * we store the pgtable in the second half of PMD
- */
- pgtable_slot = (pgtable_t *)pmdp + PTRS_PER_PMD;
- *pgtable_slot = pgtable;
- /*
- * expose the deposited pgtable to other cpus.
- * before we set the hugepage PTE at pmd level
- * hash fault code looks at the deposted pgtable
- * to store hash index values.
- */
- smp_wmb();
-}
-
-pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp)
-{
- pgtable_t pgtable;
- pgtable_t *pgtable_slot;
-
- assert_spin_locked(&mm->page_table_lock);
- pgtable_slot = (pgtable_t *)pmdp + PTRS_PER_PMD;
- pgtable = *pgtable_slot;
- /*
- * Once we withdraw, mark the entry NULL.
- */
- *pgtable_slot = NULL;
- /*
- * We store HPTE information in the deposited PTE fragment.
- * zero out the content on withdraw.
- */
- memset(pgtable, 0, PTE_FRAG_SIZE);
- return pgtable;
-}
-
-void pmdp_huge_split_prepare(struct vm_area_struct *vma,
- unsigned long address, pmd_t *pmdp)
-{
- VM_BUG_ON(address & ~HPAGE_PMD_MASK);
- VM_BUG_ON(REGION_ID(address) != USER_REGION_ID);
-
- /*
- * We can't mark the pmd none here, because that will cause a race
- * against exit_mmap. We need to continue mark pmd TRANS HUGE, while
- * we spilt, but at the same time we wan't rest of the ppc64 code
- * not to insert hash pte on this, because we will be modifying
- * the deposited pgtable in the caller of this function. Hence
- * clear the _PAGE_USER so that we move the fault handling to
- * higher level function and that will serialize against ptl.
- * We need to flush existing hash pte entries here even though,
- * the translation is still valid, because we will withdraw
- * pgtable_t after this.
- */
- pmd_hugepage_update(vma->vm_mm, address, pmdp, _PAGE_USER, 0);
-}
-
-
-/*
- * set a new huge pmd. We should not be called for updating
- * an existing pmd entry. That should go via pmd_hugepage_update.
- */
-void set_pmd_at(struct mm_struct *mm, unsigned long addr,
- pmd_t *pmdp, pmd_t pmd)
-{
-#ifdef CONFIG_DEBUG_VM
- WARN_ON((pmd_val(*pmdp) & (_PAGE_PRESENT | _PAGE_USER)) ==
- (_PAGE_PRESENT | _PAGE_USER));
- assert_spin_locked(&mm->page_table_lock);
- WARN_ON(!pmd_trans_huge(pmd));
-#endif
- trace_hugepage_set_pmd(addr, pmd_val(pmd));
- return set_pte_at(mm, addr, pmdp_ptep(pmdp), pmd_pte(pmd));
-}
-
-/*
- * We use this to invalidate a pmdp entry before switching from a
- * hugepte to regular pmd entry.
- */
-void pmdp_invalidate(struct vm_area_struct *vma, unsigned long address,
- pmd_t *pmdp)
-{
- pmd_hugepage_update(vma->vm_mm, address, pmdp, _PAGE_PRESENT, 0);
-
- /*
- * This ensures that generic code that rely on IRQ disabling
- * to prevent a parallel THP split work as expected.
- */
- kick_all_cpus_sync();
-}
-
-/*
- * A linux hugepage PMD was changed and the corresponding hash table entries
- * neesd to be flushed.
- */
-void hpte_do_hugepage_flush(struct mm_struct *mm, unsigned long addr,
- pmd_t *pmdp, unsigned long old_pmd)
-{
- int ssize;
- unsigned int psize;
- unsigned long vsid;
- unsigned long flags = 0;
- const struct cpumask *tmp;
-
- /* get the base page size,vsid and segment size */
-#ifdef CONFIG_DEBUG_VM
- psize = get_slice_psize(mm, addr);
- BUG_ON(psize == MMU_PAGE_16M);
-#endif
- if (old_pmd & _PAGE_COMBO)
- psize = MMU_PAGE_4K;
- else
- psize = MMU_PAGE_64K;
-
- if (!is_kernel_addr(addr)) {
- ssize = user_segment_size(addr);
- vsid = get_vsid(mm->context.id, addr, ssize);
- WARN_ON(vsid == 0);
- } else {
- vsid = get_kernel_vsid(addr, mmu_kernel_ssize);
- ssize = mmu_kernel_ssize;
- }
-
- tmp = cpumask_of(smp_processor_id());
- if (cpumask_equal(mm_cpumask(mm), tmp))
- flags |= HPTE_LOCAL_UPDATE;
-
- return flush_hash_hugepage(vsid, addr, pmdp, psize, ssize, flags);
-}
-
-static pmd_t pmd_set_protbits(pmd_t pmd, pgprot_t pgprot)
-{
- return __pmd(pmd_val(pmd) | pgprot_val(pgprot));
-}
-
-pmd_t pfn_pmd(unsigned long pfn, pgprot_t pgprot)
-{
- unsigned long pmdv;
-
- pmdv = (pfn << PTE_RPN_SHIFT) & PTE_RPN_MASK;
- return pmd_set_protbits(__pmd(pmdv), pgprot);
-}
-
-pmd_t mk_pmd(struct page *page, pgprot_t pgprot)
-{
- return pfn_pmd(page_to_pfn(page), pgprot);
-}
-
-pmd_t pmd_modify(pmd_t pmd, pgprot_t newprot)
-{
- unsigned long pmdv;
-
- pmdv = pmd_val(pmd);
- pmdv &= _HPAGE_CHG_MASK;
- return pmd_set_protbits(__pmd(pmdv), newprot);
-}
-
-/*
- * This is called at the end of handling a user page fault, when the
- * fault has been handled by updating a HUGE PMD entry in the linux page tables.
- * We use it to preload an HPTE into the hash table corresponding to
- * the updated linux HUGE PMD entry.
- */
-void update_mmu_cache_pmd(struct vm_area_struct *vma, unsigned long addr,
- pmd_t *pmd)
-{
- return;
-}
-
-pmd_t pmdp_huge_get_and_clear(struct mm_struct *mm,
- unsigned long addr, pmd_t *pmdp)
-{
- pmd_t old_pmd;
- pgtable_t pgtable;
- unsigned long old;
- pgtable_t *pgtable_slot;
-
- old = pmd_hugepage_update(mm, addr, pmdp, ~0UL, 0);
- old_pmd = __pmd(old);
- /*
- * We have pmd == none and we are holding page_table_lock.
- * So we can safely go and clear the pgtable hash
- * index info.
- */
- pgtable_slot = (pgtable_t *)pmdp + PTRS_PER_PMD;
- pgtable = *pgtable_slot;
- /*
- * Let's zero out old valid and hash index details
- * hash fault look at them.
- */
- memset(pgtable, 0, PTE_FRAG_SIZE);
- /*
- * Serialize against find_linux_pte_or_hugepte which does lock-less
- * lookup in page tables with local interrupts disabled. For huge pages
- * it casts pmd_t to pte_t. Since format of pte_t is different from
- * pmd_t we want to prevent transit from pmd pointing to page table
- * to pmd pointing to huge page (and back) while interrupts are disabled.
- * We clear pmd to possibly replace it with page table pointer in
- * different code paths. So make sure we wait for the parallel
- * find_linux_pte_or_hugepage to finish.
- */
- kick_all_cpus_sync();
- return old_pmd;
-}
-
-int has_transparent_hugepage(void)
-{
-
- BUILD_BUG_ON_MSG((PMD_SHIFT - PAGE_SHIFT) >= MAX_ORDER,
- "hugepages can't be allocated by the buddy allocator");
-
- BUILD_BUG_ON_MSG((PMD_SHIFT - PAGE_SHIFT) < 2,
- "We need more than 2 pages to do deferred thp split");
-
- if (!mmu_has_feature(MMU_FTR_16M_PAGE))
- return 0;
- /*
- * We support THP only if PMD_SIZE is 16MB.
- */
- if (mmu_psize_defs[MMU_PAGE_16M].shift != PMD_SHIFT)
- return 0;
- /*
- * We need to make sure that we support 16MB hugepage in a segement
- * with base page size 64K or 4K. We only enable THP with a PAGE_SIZE
- * of 64K.
- */
- /*
- * If we have 64K HPTE, we will be using that by default
- */
- if (mmu_psize_defs[MMU_PAGE_64K].shift &&
- (mmu_psize_defs[MMU_PAGE_64K].penc[MMU_PAGE_16M] == -1))
- return 0;
- /*
- * Ok we only have 4K HPTE
- */
- if (mmu_psize_defs[MMU_PAGE_4K].penc[MMU_PAGE_16M] == -1)
- return 0;
-
- return 1;
-}
-#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
diff --git a/arch/powerpc/mm/slb.c b/arch/powerpc/mm/slb.c
index 825b687..48fc28b 100644
--- a/arch/powerpc/mm/slb.c
+++ b/arch/powerpc/mm/slb.c
@@ -32,7 +32,6 @@
};
extern void slb_allocate_realmode(unsigned long ea);
-extern void slb_allocate_user(unsigned long ea);
static void slb_allocate(unsigned long ea)
{
diff --git a/arch/powerpc/mm/slb_low.S b/arch/powerpc/mm/slb_low.S
index 736d18b..dfdb90c 100644
--- a/arch/powerpc/mm/slb_low.S
+++ b/arch/powerpc/mm/slb_low.S
@@ -35,7 +35,7 @@
* check for bad kernel/user address
* (ea & ~REGION_MASK) >= PGTABLE_RANGE
*/
- rldicr. r9,r3,4,(63 - PGTABLE_EADDR_SIZE - 4)
+ rldicr. r9,r3,4,(63 - H_PGTABLE_EADDR_SIZE - 4)
bne- 8f
srdi r9,r3,60 /* get region */
@@ -91,7 +91,7 @@
* can be demoted from 64K -> 4K dynamically on some machines
*/
clrldi r11,r10,48
- cmpldi r11,(VMALLOC_SIZE >> 28) - 1
+ cmpldi r11,(H_VMALLOC_SIZE >> 28) - 1
bgt 5f
lhz r11,PACAVMALLOCSLLP(r13)
b 6f
@@ -179,56 +179,6 @@
li r11,SLB_VSID_USER /* flags don't much matter */
b slb_finish_load
-#ifdef __DISABLED__
-
-/* void slb_allocate_user(unsigned long ea);
- *
- * Create an SLB entry for the given EA (user or kernel).
- * r3 = faulting address, r13 = PACA
- * r9, r10, r11 are clobbered by this function
- * No other registers are examined or changed.
- *
- * It is called with translation enabled in order to be able to walk the
- * page tables. This is not currently used.
- */
-_GLOBAL(slb_allocate_user)
- /* r3 = faulting address */
- srdi r10,r3,28 /* get esid */
-
- crset 4*cr7+lt /* set "user" flag for later */
-
- /* check if we fit in the range covered by the pagetables*/
- srdi. r9,r3,PGTABLE_EADDR_SIZE
- crnot 4*cr0+eq,4*cr0+eq
- beqlr
-
- /* now we need to get to the page tables in order to get the page
- * size encoding from the PMD. In the future, we'll be able to deal
- * with 1T segments too by getting the encoding from the PGD instead
- */
- ld r9,PACAPGDIR(r13)
- cmpldi cr0,r9,0
- beqlr
- rlwinm r11,r10,8,25,28
- ldx r9,r9,r11 /* get pgd_t */
- cmpldi cr0,r9,0
- beqlr
- rlwinm r11,r10,3,17,28
- ldx r9,r9,r11 /* get pmd_t */
- cmpldi cr0,r9,0
- beqlr
-
- /* build vsid flags */
- andi. r11,r9,SLB_VSID_LLP
- ori r11,r11,SLB_VSID_USER
-
- /* get context to calculate proto-VSID */
- ld r9,PACACONTEXTID(r13)
- /* fall through slb_finish_load */
-
-#endif /* __DISABLED__ */
-
-
/*
* Finish loading of an SLB entry and return
*
diff --git a/arch/powerpc/mm/slice.c b/arch/powerpc/mm/slice.c
index 42954f0..2b27458 100644
--- a/arch/powerpc/mm/slice.c
+++ b/arch/powerpc/mm/slice.c
@@ -37,8 +37,8 @@
#include <asm/hugetlb.h>
/* some sanity checks */
-#if (PGTABLE_RANGE >> 43) > SLICE_MASK_SIZE
-#error PGTABLE_RANGE exceeds slice_mask high_slices size
+#if (H_PGTABLE_RANGE >> 43) > SLICE_MASK_SIZE
+#error H_PGTABLE_RANGE exceeds slice_mask high_slices size
#endif
static DEFINE_SPINLOCK(slice_convert_lock);
@@ -395,6 +395,7 @@
/* Sanity checks */
BUG_ON(mm->task_size == 0);
+ VM_BUG_ON(radix_enabled());
slice_dbg("slice_get_unmapped_area(mm=%p, psize=%d...\n", mm, psize);
slice_dbg(" addr=%lx, len=%lx, flags=%lx, topdown=%d\n",
@@ -568,6 +569,16 @@
unsigned char *hpsizes;
int index, mask_index;
+ /*
+ * Radix doesn't use slice, but can get enabled along with MMU_SLICE
+ */
+ if (radix_enabled()) {
+#ifdef CONFIG_PPC_64K_PAGES
+ return MMU_PAGE_64K;
+#else
+ return MMU_PAGE_4K;
+#endif
+ }
if (addr < SLICE_LOW_TOP) {
u64 lpsizes;
lpsizes = mm->context.low_slices_psize;
@@ -605,6 +616,7 @@
slice_dbg("slice_set_user_psize(mm=%p, psize=%d)\n", mm, psize);
+ VM_BUG_ON(radix_enabled());
spin_lock_irqsave(&slice_convert_lock, flags);
old_psize = mm->context.user_psize;
@@ -649,6 +661,7 @@
{
struct slice_mask mask = slice_range_to_mask(start, len);
+ VM_BUG_ON(radix_enabled());
slice_convert(mm, mask, psize);
}
@@ -678,6 +691,9 @@
struct slice_mask mask, available;
unsigned int psize = mm->context.user_psize;
+ if (radix_enabled())
+ return 0;
+
mask = slice_range_to_mask(addr, len);
available = slice_mask_for_size(mm, psize);
#ifdef CONFIG_PPC_64K_PAGES
diff --git a/arch/powerpc/mm/tlb-radix.c b/arch/powerpc/mm/tlb-radix.c
new file mode 100644
index 0000000..0fdaf93
--- /dev/null
+++ b/arch/powerpc/mm/tlb-radix.c
@@ -0,0 +1,251 @@
+/*
+ * TLB flush routines for radix kernels.
+ *
+ * Copyright 2015-2016, Aneesh Kumar K.V, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <linux/mm.h>
+#include <linux/hugetlb.h>
+#include <linux/memblock.h>
+
+#include <asm/tlb.h>
+#include <asm/tlbflush.h>
+
+static DEFINE_RAW_SPINLOCK(native_tlbie_lock);
+
+static inline void __tlbiel_pid(unsigned long pid, int set)
+{
+ unsigned long rb,rs,ric,prs,r;
+
+ rb = PPC_BIT(53); /* IS = 1 */
+ rb |= set << PPC_BITLSHIFT(51);
+ rs = ((unsigned long)pid) << PPC_BITLSHIFT(31);
+ prs = 1; /* process scoped */
+ r = 1; /* raidx format */
+ ric = 2; /* invalidate all the caches */
+
+ asm volatile("ptesync": : :"memory");
+ asm volatile(".long 0x7c000224 | (%0 << 11) | (%1 << 16) |"
+ "(%2 << 17) | (%3 << 18) | (%4 << 21)"
+ : : "r"(rb), "i"(r), "i"(prs), "i"(ric), "r"(rs) : "memory");
+ asm volatile("ptesync": : :"memory");
+}
+
+/*
+ * We use 128 set in radix mode and 256 set in hpt mode.
+ */
+static inline void _tlbiel_pid(unsigned long pid)
+{
+ int set;
+
+ for (set = 0; set < POWER9_TLB_SETS_RADIX ; set++) {
+ __tlbiel_pid(pid, set);
+ }
+ return;
+}
+
+static inline void _tlbie_pid(unsigned long pid)
+{
+ unsigned long rb,rs,ric,prs,r;
+
+ rb = PPC_BIT(53); /* IS = 1 */
+ rs = pid << PPC_BITLSHIFT(31);
+ prs = 1; /* process scoped */
+ r = 1; /* raidx format */
+ ric = 2; /* invalidate all the caches */
+
+ asm volatile("ptesync": : :"memory");
+ asm volatile(".long 0x7c000264 | (%0 << 11) | (%1 << 16) |"
+ "(%2 << 17) | (%3 << 18) | (%4 << 21)"
+ : : "r"(rb), "i"(r), "i"(prs), "i"(ric), "r"(rs) : "memory");
+ asm volatile("eieio; tlbsync; ptesync": : :"memory");
+}
+
+static inline void _tlbiel_va(unsigned long va, unsigned long pid,
+ unsigned long ap)
+{
+ unsigned long rb,rs,ric,prs,r;
+
+ rb = va & ~(PPC_BITMASK(52, 63));
+ rb |= ap << PPC_BITLSHIFT(58);
+ rs = pid << PPC_BITLSHIFT(31);
+ prs = 1; /* process scoped */
+ r = 1; /* raidx format */
+ ric = 0; /* no cluster flush yet */
+
+ asm volatile("ptesync": : :"memory");
+ asm volatile(".long 0x7c000224 | (%0 << 11) | (%1 << 16) |"
+ "(%2 << 17) | (%3 << 18) | (%4 << 21)"
+ : : "r"(rb), "i"(r), "i"(prs), "i"(ric), "r"(rs) : "memory");
+ asm volatile("ptesync": : :"memory");
+}
+
+static inline void _tlbie_va(unsigned long va, unsigned long pid,
+ unsigned long ap)
+{
+ unsigned long rb,rs,ric,prs,r;
+
+ rb = va & ~(PPC_BITMASK(52, 63));
+ rb |= ap << PPC_BITLSHIFT(58);
+ rs = pid << PPC_BITLSHIFT(31);
+ prs = 1; /* process scoped */
+ r = 1; /* raidx format */
+ ric = 0; /* no cluster flush yet */
+
+ asm volatile("ptesync": : :"memory");
+ asm volatile(".long 0x7c000264 | (%0 << 11) | (%1 << 16) |"
+ "(%2 << 17) | (%3 << 18) | (%4 << 21)"
+ : : "r"(rb), "i"(r), "i"(prs), "i"(ric), "r"(rs) : "memory");
+ asm volatile("eieio; tlbsync; ptesync": : :"memory");
+}
+
+/*
+ * Base TLB flushing operations:
+ *
+ * - flush_tlb_mm(mm) flushes the specified mm context TLB's
+ * - flush_tlb_page(vma, vmaddr) flushes one page
+ * - flush_tlb_range(vma, start, end) flushes a range of pages
+ * - flush_tlb_kernel_range(start, end) flushes kernel pages
+ *
+ * - local_* variants of page and mm only apply to the current
+ * processor
+ */
+void radix__local_flush_tlb_mm(struct mm_struct *mm)
+{
+ unsigned int pid;
+
+ preempt_disable();
+ pid = mm->context.id;
+ if (pid != MMU_NO_CONTEXT)
+ _tlbiel_pid(pid);
+ preempt_enable();
+}
+EXPORT_SYMBOL(radix__local_flush_tlb_mm);
+
+void radix___local_flush_tlb_page(struct mm_struct *mm, unsigned long vmaddr,
+ unsigned long ap, int nid)
+{
+ unsigned int pid;
+
+ preempt_disable();
+ pid = mm ? mm->context.id : 0;
+ if (pid != MMU_NO_CONTEXT)
+ _tlbiel_va(vmaddr, pid, ap);
+ preempt_enable();
+}
+
+void radix__local_flush_tlb_page(struct vm_area_struct *vma, unsigned long vmaddr)
+{
+#ifdef CONFIG_HUGETLB_PAGE
+ /* need the return fix for nohash.c */
+ if (vma && is_vm_hugetlb_page(vma))
+ return __local_flush_hugetlb_page(vma, vmaddr);
+#endif
+ radix___local_flush_tlb_page(vma ? vma->vm_mm : NULL, vmaddr,
+ mmu_get_ap(mmu_virtual_psize), 0);
+}
+EXPORT_SYMBOL(radix__local_flush_tlb_page);
+
+#ifdef CONFIG_SMP
+static int mm_is_core_local(struct mm_struct *mm)
+{
+ return cpumask_subset(mm_cpumask(mm),
+ topology_sibling_cpumask(smp_processor_id()));
+}
+
+void radix__flush_tlb_mm(struct mm_struct *mm)
+{
+ unsigned int pid;
+
+ preempt_disable();
+ pid = mm->context.id;
+ if (unlikely(pid == MMU_NO_CONTEXT))
+ goto no_context;
+
+ if (!mm_is_core_local(mm)) {
+ int lock_tlbie = !mmu_has_feature(MMU_FTR_LOCKLESS_TLBIE);
+
+ if (lock_tlbie)
+ raw_spin_lock(&native_tlbie_lock);
+ _tlbie_pid(pid);
+ if (lock_tlbie)
+ raw_spin_unlock(&native_tlbie_lock);
+ } else
+ _tlbiel_pid(pid);
+no_context:
+ preempt_enable();
+}
+EXPORT_SYMBOL(radix__flush_tlb_mm);
+
+void radix___flush_tlb_page(struct mm_struct *mm, unsigned long vmaddr,
+ unsigned long ap, int nid)
+{
+ unsigned int pid;
+
+ preempt_disable();
+ pid = mm ? mm->context.id : 0;
+ if (unlikely(pid == MMU_NO_CONTEXT))
+ goto bail;
+ if (!mm_is_core_local(mm)) {
+ int lock_tlbie = !mmu_has_feature(MMU_FTR_LOCKLESS_TLBIE);
+
+ if (lock_tlbie)
+ raw_spin_lock(&native_tlbie_lock);
+ _tlbie_va(vmaddr, pid, ap);
+ if (lock_tlbie)
+ raw_spin_unlock(&native_tlbie_lock);
+ } else
+ _tlbiel_va(vmaddr, pid, ap);
+bail:
+ preempt_enable();
+}
+
+void radix__flush_tlb_page(struct vm_area_struct *vma, unsigned long vmaddr)
+{
+#ifdef CONFIG_HUGETLB_PAGE
+ if (vma && is_vm_hugetlb_page(vma))
+ return flush_hugetlb_page(vma, vmaddr);
+#endif
+ radix___flush_tlb_page(vma ? vma->vm_mm : NULL, vmaddr,
+ mmu_get_ap(mmu_virtual_psize), 0);
+}
+EXPORT_SYMBOL(radix__flush_tlb_page);
+
+#endif /* CONFIG_SMP */
+
+void radix__flush_tlb_kernel_range(unsigned long start, unsigned long end)
+{
+ int lock_tlbie = !mmu_has_feature(MMU_FTR_LOCKLESS_TLBIE);
+
+ if (lock_tlbie)
+ raw_spin_lock(&native_tlbie_lock);
+ _tlbie_pid(0);
+ if (lock_tlbie)
+ raw_spin_unlock(&native_tlbie_lock);
+}
+EXPORT_SYMBOL(radix__flush_tlb_kernel_range);
+
+/*
+ * Currently, for range flushing, we just do a full mm flush. Because
+ * we use this in code path where we don' track the page size.
+ */
+void radix__flush_tlb_range(struct vm_area_struct *vma, unsigned long start,
+ unsigned long end)
+
+{
+ struct mm_struct *mm = vma->vm_mm;
+ radix__flush_tlb_mm(mm);
+}
+EXPORT_SYMBOL(radix__flush_tlb_range);
+
+
+void radix__tlb_flush(struct mmu_gather *tlb)
+{
+ struct mm_struct *mm = tlb->mm;
+ radix__flush_tlb_mm(mm);
+}
diff --git a/arch/powerpc/mm/tlb_hash64.c b/arch/powerpc/mm/tlb_hash64.c
index f7b8039..4517aa4 100644
--- a/arch/powerpc/mm/tlb_hash64.c
+++ b/arch/powerpc/mm/tlb_hash64.c
@@ -155,7 +155,7 @@
batch->index = 0;
}
-void tlb_flush(struct mmu_gather *tlb)
+void hash__tlb_flush(struct mmu_gather *tlb)
{
struct ppc64_tlb_batch *tlbbatch = &get_cpu_var(ppc64_tlb_batch);
@@ -218,7 +218,7 @@
pte = pte_val(*ptep);
if (is_thp)
trace_hugepage_invalidate(start, pte);
- if (!(pte & _PAGE_HASHPTE))
+ if (!(pte & H_PAGE_HASHPTE))
continue;
if (unlikely(is_thp))
hpte_do_hugepage_flush(mm, start, (pmd_t *)ptep, pte);
@@ -248,7 +248,7 @@
start_pte = pte_offset_map(pmd, addr);
for (pte = start_pte; pte < start_pte + PTRS_PER_PTE; pte++) {
unsigned long pteval = pte_val(*pte);
- if (pteval & _PAGE_HASHPTE)
+ if (pteval & H_PAGE_HASHPTE)
hpte_need_flush(mm, addr, pte, pteval, 0);
addr += PAGE_SIZE;
}
diff --git a/arch/powerpc/perf/Makefile b/arch/powerpc/perf/Makefile
index f9c083a..77b6394 100644
--- a/arch/powerpc/perf/Makefile
+++ b/arch/powerpc/perf/Makefile
@@ -1,6 +1,6 @@
subdir-ccflags-$(CONFIG_PPC_WERROR) := -Werror
-obj-$(CONFIG_PERF_EVENTS) += callchain.o
+obj-$(CONFIG_PERF_EVENTS) += callchain.o perf_regs.o
obj-$(CONFIG_PPC_PERF_CTRS) += core-book3s.o bhrb.o
obj64-$(CONFIG_PPC_PERF_CTRS) += power4-pmu.o ppc970-pmu.o power5-pmu.o \
diff --git a/arch/powerpc/perf/callchain.c b/arch/powerpc/perf/callchain.c
index 22d9015..26d37e6 100644
--- a/arch/powerpc/perf/callchain.c
+++ b/arch/powerpc/perf/callchain.c
@@ -137,7 +137,7 @@
offset = addr & ((1UL << shift) - 1);
pte = READ_ONCE(*ptep);
- if (!pte_present(pte) || !(pte_val(pte) & _PAGE_USER))
+ if (!pte_present(pte) || !pte_user(pte))
goto err_out;
pfn = pte_pfn(pte);
if (!page_is_ram(pfn))
diff --git a/arch/powerpc/perf/perf_regs.c b/arch/powerpc/perf/perf_regs.c
new file mode 100644
index 0000000..d24a8a3
--- /dev/null
+++ b/arch/powerpc/perf/perf_regs.c
@@ -0,0 +1,104 @@
+/*
+ * Copyright 2016 Anju T, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <linux/errno.h>
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <linux/perf_event.h>
+#include <linux/bug.h>
+#include <linux/stddef.h>
+#include <asm/ptrace.h>
+#include <asm/perf_regs.h>
+
+#define PT_REGS_OFFSET(id, r) [id] = offsetof(struct pt_regs, r)
+
+#define REG_RESERVED (~((1ULL << PERF_REG_POWERPC_MAX) - 1))
+
+static unsigned int pt_regs_offset[PERF_REG_POWERPC_MAX] = {
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R0, gpr[0]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R1, gpr[1]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R2, gpr[2]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R3, gpr[3]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R4, gpr[4]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R5, gpr[5]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R6, gpr[6]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R7, gpr[7]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R8, gpr[8]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R9, gpr[9]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R10, gpr[10]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R11, gpr[11]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R12, gpr[12]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R13, gpr[13]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R14, gpr[14]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R15, gpr[15]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R16, gpr[16]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R17, gpr[17]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R18, gpr[18]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R19, gpr[19]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R20, gpr[20]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R21, gpr[21]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R22, gpr[22]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R23, gpr[23]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R24, gpr[24]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R25, gpr[25]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R26, gpr[26]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R27, gpr[27]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R28, gpr[28]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R29, gpr[29]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R30, gpr[30]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_R31, gpr[31]),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_NIP, nip),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_MSR, msr),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_ORIG_R3, orig_gpr3),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_CTR, ctr),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_LINK, link),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_XER, xer),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_CCR, ccr),
+#ifdef CONFIG_PPC64
+ PT_REGS_OFFSET(PERF_REG_POWERPC_SOFTE, softe),
+#else
+ PT_REGS_OFFSET(PERF_REG_POWERPC_SOFTE, mq),
+#endif
+ PT_REGS_OFFSET(PERF_REG_POWERPC_TRAP, trap),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_DAR, dar),
+ PT_REGS_OFFSET(PERF_REG_POWERPC_DSISR, dsisr),
+};
+
+u64 perf_reg_value(struct pt_regs *regs, int idx)
+{
+ if (WARN_ON_ONCE(idx >= PERF_REG_POWERPC_MAX))
+ return 0;
+
+ return regs_get_register(regs, pt_regs_offset[idx]);
+}
+
+int perf_reg_validate(u64 mask)
+{
+ if (!mask || mask & REG_RESERVED)
+ return -EINVAL;
+ return 0;
+}
+
+u64 perf_reg_abi(struct task_struct *task)
+{
+#ifdef CONFIG_PPC64
+ if (!test_tsk_thread_flag(task, TIF_32BIT))
+ return PERF_SAMPLE_REGS_ABI_64;
+ else
+#endif
+ return PERF_SAMPLE_REGS_ABI_32;
+}
+
+void perf_get_regs_user(struct perf_regs *regs_user,
+ struct pt_regs *regs,
+ struct pt_regs *regs_user_copy)
+{
+ regs_user->regs = task_pt_regs(current);
+ regs_user->abi = perf_reg_abi(current);
+}
diff --git a/arch/powerpc/perf/power8-events-list.h b/arch/powerpc/perf/power8-events-list.h
index 741b77e..3a2e6e8eb 100644
--- a/arch/powerpc/perf/power8-events-list.h
+++ b/arch/powerpc/perf/power8-events-list.h
@@ -49,3 +49,43 @@
EVENT(PM_DTLB_MISS, 0x300fc)
/* ITLB Reloaded */
EVENT(PM_ITLB_MISS, 0x400fc)
+/* Run_Instructions */
+EVENT(PM_RUN_INST_CMPL, 0x500fa)
+/* Alternate event code for PM_RUN_INST_CMPL */
+EVENT(PM_RUN_INST_CMPL_ALT, 0x400fa)
+/* Run_cycles */
+EVENT(PM_RUN_CYC, 0x600f4)
+/* Alternate event code for Run_cycles */
+EVENT(PM_RUN_CYC_ALT, 0x200f4)
+/* Marked store completed */
+EVENT(PM_MRK_ST_CMPL, 0x10134)
+/* Alternate event code for Marked store completed */
+EVENT(PM_MRK_ST_CMPL_ALT, 0x301e2)
+/* Marked two path branch */
+EVENT(PM_BR_MRK_2PATH, 0x10138)
+/* Alternate event code for PM_BR_MRK_2PATH */
+EVENT(PM_BR_MRK_2PATH_ALT, 0x40138)
+/* L3 castouts in Mepf state */
+EVENT(PM_L3_CO_MEPF, 0x18082)
+/* Alternate event code for PM_L3_CO_MEPF */
+EVENT(PM_L3_CO_MEPF_ALT, 0x3e05e)
+/* Data cache was reloaded from a location other than L2 due to a marked load */
+EVENT(PM_MRK_DATA_FROM_L2MISS, 0x1d14e)
+/* Alternate event code for PM_MRK_DATA_FROM_L2MISS */
+EVENT(PM_MRK_DATA_FROM_L2MISS_ALT, 0x401e8)
+/* Alternate event code for PM_CMPLU_STALL */
+EVENT(PM_CMPLU_STALL_ALT, 0x1e054)
+/* Two path branch */
+EVENT(PM_BR_2PATH, 0x20036)
+/* Alternate event code for PM_BR_2PATH */
+EVENT(PM_BR_2PATH_ALT, 0x40036)
+/* # PPC Dispatched */
+EVENT(PM_INST_DISP, 0x200f2)
+/* Alternate event code for PM_INST_DISP */
+EVENT(PM_INST_DISP_ALT, 0x300f2)
+/* Marked filter Match */
+EVENT(PM_MRK_FILT_MATCH, 0x2013c)
+/* Alternate event code for PM_MRK_FILT_MATCH */
+EVENT(PM_MRK_FILT_MATCH_ALT, 0x3012e)
+/* Alternate event code for PM_LD_MISS_L1 */
+EVENT(PM_LD_MISS_L1_ALT, 0x400f0)
diff --git a/arch/powerpc/perf/power8-pmu.c b/arch/powerpc/perf/power8-pmu.c
index 690d918..7cf3b43 100644
--- a/arch/powerpc/perf/power8-pmu.c
+++ b/arch/powerpc/perf/power8-pmu.c
@@ -274,7 +274,8 @@
/* Ignore Linux defined bits when checking event below */
base_event = event & ~EVENT_LINUX_MASK;
- if (pmc >= 5 && base_event != 0x500fa && base_event != 0x600f4)
+ if (pmc >= 5 && base_event != PM_RUN_INST_CMPL &&
+ base_event != PM_RUN_CYC)
return -1;
mask |= CNST_PMC_MASK(pmc);
@@ -488,17 +489,17 @@
/* Table of alternatives, sorted by column 0 */
static const unsigned int event_alternatives[][MAX_ALT] = {
- { 0x10134, 0x301e2 }, /* PM_MRK_ST_CMPL */
- { 0x10138, 0x40138 }, /* PM_BR_MRK_2PATH */
- { 0x18082, 0x3e05e }, /* PM_L3_CO_MEPF */
- { 0x1d14e, 0x401e8 }, /* PM_MRK_DATA_FROM_L2MISS */
- { 0x1e054, 0x4000a }, /* PM_CMPLU_STALL */
- { 0x20036, 0x40036 }, /* PM_BR_2PATH */
- { 0x200f2, 0x300f2 }, /* PM_INST_DISP */
- { 0x200f4, 0x600f4 }, /* PM_RUN_CYC */
- { 0x2013c, 0x3012e }, /* PM_MRK_FILT_MATCH */
- { 0x3e054, 0x400f0 }, /* PM_LD_MISS_L1 */
- { 0x400fa, 0x500fa }, /* PM_RUN_INST_CMPL */
+ { PM_MRK_ST_CMPL, PM_MRK_ST_CMPL_ALT },
+ { PM_BR_MRK_2PATH, PM_BR_MRK_2PATH_ALT },
+ { PM_L3_CO_MEPF, PM_L3_CO_MEPF_ALT },
+ { PM_MRK_DATA_FROM_L2MISS, PM_MRK_DATA_FROM_L2MISS_ALT },
+ { PM_CMPLU_STALL_ALT, PM_CMPLU_STALL },
+ { PM_BR_2PATH, PM_BR_2PATH_ALT },
+ { PM_INST_DISP, PM_INST_DISP_ALT },
+ { PM_RUN_CYC_ALT, PM_RUN_CYC },
+ { PM_MRK_FILT_MATCH, PM_MRK_FILT_MATCH_ALT },
+ { PM_LD_MISS_L1, PM_LD_MISS_L1_ALT },
+ { PM_RUN_INST_CMPL_ALT, PM_RUN_INST_CMPL },
};
/*
@@ -546,17 +547,17 @@
j = num_alt;
for (i = 0; i < num_alt; ++i) {
switch (alt[i]) {
- case 0x1e: /* PM_CYC */
- alt[j++] = 0x600f4; /* PM_RUN_CYC */
+ case PM_CYC:
+ alt[j++] = PM_RUN_CYC;
break;
- case 0x600f4: /* PM_RUN_CYC */
- alt[j++] = 0x1e;
+ case PM_RUN_CYC:
+ alt[j++] = PM_CYC;
break;
- case 0x2: /* PM_PPC_CMPL */
- alt[j++] = 0x500fa; /* PM_RUN_INST_CMPL */
+ case PM_INST_CMPL:
+ alt[j++] = PM_RUN_INST_CMPL;
break;
- case 0x500fa: /* PM_RUN_INST_CMPL */
- alt[j++] = 0x2; /* PM_PPC_CMPL */
+ case PM_RUN_INST_CMPL:
+ alt[j++] = PM_INST_CMPL;
break;
}
}
diff --git a/arch/powerpc/platforms/Kconfig.cputype b/arch/powerpc/platforms/Kconfig.cputype
index 142dff5..77e9b8d 100644
--- a/arch/powerpc/platforms/Kconfig.cputype
+++ b/arch/powerpc/platforms/Kconfig.cputype
@@ -72,7 +72,7 @@
select PPC_FPU
select PPC_HAVE_PMU_SUPPORT
select SYS_SUPPORTS_HUGETLBFS
- select HAVE_ARCH_TRANSPARENT_HUGEPAGE if PPC_64K_PAGES
+ select HAVE_ARCH_TRANSPARENT_HUGEPAGE
select ARCH_SUPPORTS_NUMA_BALANCING
select IRQ_WORK
@@ -331,6 +331,15 @@
def_bool y
depends on PPC_STD_MMU && PPC64
+config PPC_RADIX_MMU
+ bool "Radix MMU Support"
+ depends on PPC_BOOK3S_64
+ default y
+ help
+ Enable support for the Power ISA 3.0 Radix style MMU. Currently this
+ is only implemented by IBM Power9 CPUs, if you don't have one of them
+ you can probably disable this.
+
config PPC_MMU_NOHASH
def_bool y
depends on !PPC_STD_MMU
diff --git a/arch/powerpc/platforms/cell/spu_base.c b/arch/powerpc/platforms/cell/spu_base.c
index f7af74f..3cbe38f 100644
--- a/arch/powerpc/platforms/cell/spu_base.c
+++ b/arch/powerpc/platforms/cell/spu_base.c
@@ -24,7 +24,7 @@
#include <linux/interrupt.h>
#include <linux/list.h>
-#include <linux/module.h>
+#include <linux/init.h>
#include <linux/ptrace.h>
#include <linux/slab.h>
#include <linux/wait.h>
@@ -197,7 +197,7 @@
(REGION_ID(ea) != USER_REGION_ID)) {
spin_unlock(&spu->register_lock);
- ret = hash_page(ea, _PAGE_PRESENT, 0x300, dsisr);
+ ret = hash_page(ea, _PAGE_PRESENT | _PAGE_READ, 0x300, dsisr);
spin_lock(&spu->register_lock);
if (!ret) {
@@ -805,7 +805,4 @@
out:
return ret;
}
-module_init(init_spu_base);
-
-MODULE_LICENSE("GPL");
-MODULE_AUTHOR("Arnd Bergmann <arndb@de.ibm.com>");
+device_initcall(init_spu_base);
diff --git a/arch/powerpc/platforms/cell/spufs/fault.c b/arch/powerpc/platforms/cell/spufs/fault.c
index d98f845..e29e4d5 100644
--- a/arch/powerpc/platforms/cell/spufs/fault.c
+++ b/arch/powerpc/platforms/cell/spufs/fault.c
@@ -141,8 +141,8 @@
/* we must not hold the lock when entering copro_handle_mm_fault */
spu_release(ctx);
- access = (_PAGE_PRESENT | _PAGE_USER);
- access |= (dsisr & MFC_DSISR_ACCESS_PUT) ? _PAGE_RW : 0UL;
+ access = (_PAGE_PRESENT | _PAGE_READ);
+ access |= (dsisr & MFC_DSISR_ACCESS_PUT) ? _PAGE_WRITE : 0UL;
local_irq_save(flags);
ret = hash_page(ea, access, 0x300, dsisr);
local_irq_restore(flags);
diff --git a/arch/powerpc/platforms/powernv/eeh-powernv.c b/arch/powerpc/platforms/powernv/eeh-powernv.c
index 950b3e5..9226df1 100644
--- a/arch/powerpc/platforms/powernv/eeh-powernv.c
+++ b/arch/powerpc/platforms/powernv/eeh-powernv.c
@@ -75,7 +75,7 @@
* and P7IOC separately. So we should regard
* PE#0 as valid for PHB3 and P7IOC.
*/
- if (phb->ioda.reserved_pe != 0)
+ if (phb->ioda.reserved_pe_idx != 0)
eeh_add_flag(EEH_VALID_PE_ZERO);
break;
@@ -1009,8 +1009,9 @@
static int pnv_eeh_reset(struct eeh_pe *pe, int option)
{
struct pci_controller *hose = pe->phb;
+ struct pnv_phb *phb;
struct pci_bus *bus;
- int ret;
+ int64_t rc;
/*
* For PHB reset, we always have complete reset. For those PEs whose
@@ -1026,45 +1027,39 @@
* reset. The side effect is that EEH core has to clear the frozen
* state explicitly after BAR restore.
*/
- if (pe->type & EEH_PE_PHB) {
- ret = pnv_eeh_phb_reset(hose, option);
- } else {
- struct pnv_phb *phb;
- s64 rc;
+ if (pe->type & EEH_PE_PHB)
+ return pnv_eeh_phb_reset(hose, option);
- /*
- * The frozen PE might be caused by PAPR error injection
- * registers, which are expected to be cleared after hitting
- * frozen PE as stated in the hardware spec. Unfortunately,
- * that's not true on P7IOC. So we have to clear it manually
- * to avoid recursive EEH errors during recovery.
- */
- phb = hose->private_data;
- if (phb->model == PNV_PHB_MODEL_P7IOC &&
- (option == EEH_RESET_HOT ||
- option == EEH_RESET_FUNDAMENTAL)) {
- rc = opal_pci_reset(phb->opal_id,
- OPAL_RESET_PHB_ERROR,
- OPAL_ASSERT_RESET);
- if (rc != OPAL_SUCCESS) {
- pr_warn("%s: Failure %lld clearing "
- "error injection registers\n",
- __func__, rc);
- return -EIO;
- }
+ /*
+ * The frozen PE might be caused by PAPR error injection
+ * registers, which are expected to be cleared after hitting
+ * frozen PE as stated in the hardware spec. Unfortunately,
+ * that's not true on P7IOC. So we have to clear it manually
+ * to avoid recursive EEH errors during recovery.
+ */
+ phb = hose->private_data;
+ if (phb->model == PNV_PHB_MODEL_P7IOC &&
+ (option == EEH_RESET_HOT ||
+ option == EEH_RESET_FUNDAMENTAL)) {
+ rc = opal_pci_reset(phb->opal_id,
+ OPAL_RESET_PHB_ERROR,
+ OPAL_ASSERT_RESET);
+ if (rc != OPAL_SUCCESS) {
+ pr_warn("%s: Failure %lld clearing error injection registers\n",
+ __func__, rc);
+ return -EIO;
}
-
- bus = eeh_pe_bus_get(pe);
- if (pe->type & EEH_PE_VF)
- ret = pnv_eeh_reset_vf_pe(pe, option);
- else if (pci_is_root_bus(bus) ||
- pci_is_root_bus(bus->parent))
- ret = pnv_eeh_root_reset(hose, option);
- else
- ret = pnv_eeh_bridge_reset(bus->self, option);
}
- return ret;
+ bus = eeh_pe_bus_get(pe);
+ if (pe->type & EEH_PE_VF)
+ return pnv_eeh_reset_vf_pe(pe, option);
+
+ if (pci_is_root_bus(bus) ||
+ pci_is_root_bus(bus->parent))
+ return pnv_eeh_root_reset(hose, option);
+
+ return pnv_eeh_bridge_reset(bus->self, option);
}
/**
diff --git a/arch/powerpc/platforms/powernv/npu-dma.c b/arch/powerpc/platforms/powernv/npu-dma.c
index 7229acd..0459e100 100644
--- a/arch/powerpc/platforms/powernv/npu-dma.c
+++ b/arch/powerpc/platforms/powernv/npu-dma.c
@@ -12,6 +12,7 @@
#include <linux/export.h>
#include <linux/pci.h>
#include <linux/memblock.h>
+#include <linux/iommu.h>
#include <asm/iommu.h>
#include <asm/pnv-pci.h>
@@ -25,8 +26,6 @@
* Other types of TCE cache invalidation are not functional in the
* hardware.
*/
-#define TCE_KILL_INVAL_ALL PPC_BIT(0)
-
static struct pci_dev *get_pci_dev(struct device_node *dn)
{
return PCI_DN(dn)->pcidev;
@@ -138,22 +137,17 @@
struct pnv_ioda_pe *pe;
struct pci_dn *pdn;
- if (npe->flags & PNV_IODA_PE_PEER) {
- pe = npe->peers[0];
- pdev = pe->pdev;
- } else {
- pdev = pnv_pci_get_gpu_dev(npe->pdev);
- if (!pdev)
- return NULL;
+ pdev = pnv_pci_get_gpu_dev(npe->pdev);
+ if (!pdev)
+ return NULL;
- pdn = pci_get_pdn(pdev);
- if (WARN_ON(!pdn || pdn->pe_number == IODA_INVALID_PE))
- return NULL;
+ pdn = pci_get_pdn(pdev);
+ if (WARN_ON(!pdn || pdn->pe_number == IODA_INVALID_PE))
+ return NULL;
- hose = pci_bus_to_host(pdev->bus);
- phb = hose->private_data;
- pe = &phb->ioda.pe_array[pdn->pe_number];
- }
+ hose = pci_bus_to_host(pdev->bus);
+ phb = hose->private_data;
+ pe = &phb->ioda.pe_array[pdn->pe_number];
if (gpdev)
*gpdev = pdev;
@@ -161,92 +155,70 @@
return pe;
}
-void pnv_npu_tce_invalidate_entire(struct pnv_ioda_pe *npe)
+long pnv_npu_set_window(struct pnv_ioda_pe *npe, int num,
+ struct iommu_table *tbl)
{
struct pnv_phb *phb = npe->phb;
+ int64_t rc;
+ const unsigned long size = tbl->it_indirect_levels ?
+ tbl->it_level_size : tbl->it_size;
+ const __u64 start_addr = tbl->it_offset << tbl->it_page_shift;
+ const __u64 win_size = tbl->it_size << tbl->it_page_shift;
- if (WARN_ON(phb->type != PNV_PHB_NPU ||
- !phb->ioda.tce_inval_reg ||
- !(npe->flags & PNV_IODA_PE_DEV)))
- return;
+ pe_info(npe, "Setting up window %llx..%llx pg=%lx\n",
+ start_addr, start_addr + win_size - 1,
+ IOMMU_PAGE_SIZE(tbl));
- mb(); /* Ensure previous TCE table stores are visible */
- __raw_writeq(cpu_to_be64(TCE_KILL_INVAL_ALL),
- phb->ioda.tce_inval_reg);
-}
-
-void pnv_npu_tce_invalidate(struct pnv_ioda_pe *npe,
- struct iommu_table *tbl,
- unsigned long index,
- unsigned long npages,
- bool rm)
-{
- struct pnv_phb *phb = npe->phb;
-
- /* We can only invalidate the whole cache on NPU */
- unsigned long val = TCE_KILL_INVAL_ALL;
-
- if (WARN_ON(phb->type != PNV_PHB_NPU ||
- !phb->ioda.tce_inval_reg ||
- !(npe->flags & PNV_IODA_PE_DEV)))
- return;
-
- mb(); /* Ensure previous TCE table stores are visible */
- if (rm)
- __raw_rm_writeq(cpu_to_be64(val),
- (__be64 __iomem *) phb->ioda.tce_inval_reg_phys);
- else
- __raw_writeq(cpu_to_be64(val),
- phb->ioda.tce_inval_reg);
-}
-
-void pnv_npu_init_dma_pe(struct pnv_ioda_pe *npe)
-{
- struct pnv_ioda_pe *gpe;
- struct pci_dev *gpdev;
- int i, avail = -1;
-
- if (!npe->pdev || !(npe->flags & PNV_IODA_PE_DEV))
- return;
-
- gpe = get_gpu_pci_dev_and_pe(npe, &gpdev);
- if (!gpe)
- return;
-
- for (i = 0; i < PNV_IODA_MAX_PEER_PES; i++) {
- /* Nothing to do if the PE is already connected. */
- if (gpe->peers[i] == npe)
- return;
-
- if (!gpe->peers[i])
- avail = i;
+ rc = opal_pci_map_pe_dma_window(phb->opal_id,
+ npe->pe_number,
+ npe->pe_number,
+ tbl->it_indirect_levels + 1,
+ __pa(tbl->it_base),
+ size << 3,
+ IOMMU_PAGE_SIZE(tbl));
+ if (rc) {
+ pe_err(npe, "Failed to configure TCE table, err %lld\n", rc);
+ return rc;
}
+ pnv_pci_ioda2_tce_invalidate_entire(phb, false);
- if (WARN_ON(avail < 0))
- return;
+ /* Add the table to the list so its TCE cache will get invalidated */
+ pnv_pci_link_table_and_group(phb->hose->node, num,
+ tbl, &npe->table_group);
- gpe->peers[avail] = npe;
- gpe->flags |= PNV_IODA_PE_PEER;
+ return 0;
+}
- /*
- * We assume that the NPU devices only have a single peer PE
- * (the GPU PCIe device PE).
- */
- npe->peers[0] = gpe;
- npe->flags |= PNV_IODA_PE_PEER;
+long pnv_npu_unset_window(struct pnv_ioda_pe *npe, int num)
+{
+ struct pnv_phb *phb = npe->phb;
+ int64_t rc;
+
+ pe_info(npe, "Removing DMA window\n");
+
+ rc = opal_pci_map_pe_dma_window(phb->opal_id, npe->pe_number,
+ npe->pe_number,
+ 0/* levels */, 0/* table address */,
+ 0/* table size */, 0/* page size */);
+ if (rc) {
+ pe_err(npe, "Unmapping failed, ret = %lld\n", rc);
+ return rc;
+ }
+ pnv_pci_ioda2_tce_invalidate_entire(phb, false);
+
+ pnv_pci_unlink_table_and_group(npe->table_group.tables[num],
+ &npe->table_group);
+
+ return 0;
}
/*
- * For the NPU we want to point the TCE table at the same table as the
- * real PCI device.
+ * Enables 32 bit DMA on NPU.
*/
-static void pnv_npu_disable_bypass(struct pnv_ioda_pe *npe)
+static void pnv_npu_dma_set_32(struct pnv_ioda_pe *npe)
{
- struct pnv_phb *phb = npe->phb;
struct pci_dev *gpdev;
struct pnv_ioda_pe *gpe;
- void *addr;
- unsigned int size;
int64_t rc;
/*
@@ -260,14 +232,7 @@
if (!gpe)
return;
- addr = (void *)gpe->table_group.tables[0]->it_base;
- size = gpe->table_group.tables[0]->it_size << 3;
- rc = opal_pci_map_pe_dma_window(phb->opal_id, npe->pe_number,
- npe->pe_number, 1, __pa(addr),
- size, 0x1000);
- if (rc != OPAL_SUCCESS)
- pr_warn("%s: Error %lld setting DMA window on PHB#%d-PE#%d\n",
- __func__, rc, phb->hose->global_number, npe->pe_number);
+ rc = pnv_npu_set_window(npe, 0, gpe->table_group.tables[0]);
/*
* We don't initialise npu_pe->tce32_table as we always use
@@ -277,72 +242,120 @@
}
/*
- * Enable/disable bypass mode on the NPU. The NPU only supports one
+ * Enables bypass mode on the NPU. The NPU only supports one
* window per link, so bypass needs to be explicitly enabled or
* disabled. Unlike for a PHB3 bypass and non-bypass modes can't be
* active at the same time.
*/
-int pnv_npu_dma_set_bypass(struct pnv_ioda_pe *npe, bool enable)
+static int pnv_npu_dma_set_bypass(struct pnv_ioda_pe *npe)
{
struct pnv_phb *phb = npe->phb;
int64_t rc = 0;
+ phys_addr_t top = memblock_end_of_DRAM();
if (phb->type != PNV_PHB_NPU || !npe->pdev)
return -EINVAL;
- if (enable) {
- /* Enable the bypass window */
- phys_addr_t top = memblock_end_of_DRAM();
+ rc = pnv_npu_unset_window(npe, 0);
+ if (rc != OPAL_SUCCESS)
+ return rc;
- npe->tce_bypass_base = 0;
- top = roundup_pow_of_two(top);
- dev_info(&npe->pdev->dev, "Enabling bypass for PE %d\n",
- npe->pe_number);
- rc = opal_pci_map_pe_dma_window_real(phb->opal_id,
- npe->pe_number, npe->pe_number,
- npe->tce_bypass_base, top);
- } else {
- /*
- * Disable the bypass window by replacing it with the
- * TCE32 window.
- */
- pnv_npu_disable_bypass(npe);
- }
+ /* Enable the bypass window */
+
+ top = roundup_pow_of_two(top);
+ dev_info(&npe->pdev->dev, "Enabling bypass for PE %d\n",
+ npe->pe_number);
+ rc = opal_pci_map_pe_dma_window_real(phb->opal_id,
+ npe->pe_number, npe->pe_number,
+ 0 /* bypass base */, top);
+
+ if (rc == OPAL_SUCCESS)
+ pnv_pci_ioda2_tce_invalidate_entire(phb, false);
return rc;
}
-int pnv_npu_dma_set_mask(struct pci_dev *npdev, u64 dma_mask)
+void pnv_npu_try_dma_set_bypass(struct pci_dev *gpdev, bool bypass)
{
- struct pci_controller *hose = pci_bus_to_host(npdev->bus);
- struct pnv_phb *phb = hose->private_data;
- struct pci_dn *pdn = pci_get_pdn(npdev);
- struct pnv_ioda_pe *npe, *gpe;
- struct pci_dev *gpdev;
- uint64_t top;
- bool bypass = false;
+ int i;
+ struct pnv_phb *phb;
+ struct pci_dn *pdn;
+ struct pnv_ioda_pe *npe;
+ struct pci_dev *npdev;
- if (WARN_ON(!pdn || pdn->pe_number == IODA_INVALID_PE))
- return -ENXIO;
+ for (i = 0; ; ++i) {
+ npdev = pnv_pci_get_npu_dev(gpdev, i);
- /* We only do bypass if it's enabled on the linked device */
- npe = &phb->ioda.pe_array[pdn->pe_number];
- gpe = get_gpu_pci_dev_and_pe(npe, &gpdev);
- if (!gpe)
- return -ENODEV;
+ if (!npdev)
+ break;
- if (gpe->tce_bypass_enabled) {
- top = gpe->tce_bypass_base + memblock_end_of_DRAM() - 1;
- bypass = (dma_mask >= top);
+ pdn = pci_get_pdn(npdev);
+ if (WARN_ON(!pdn || pdn->pe_number == IODA_INVALID_PE))
+ return;
+
+ phb = pci_bus_to_host(npdev->bus)->private_data;
+
+ /* We only do bypass if it's enabled on the linked device */
+ npe = &phb->ioda.pe_array[pdn->pe_number];
+
+ if (bypass) {
+ dev_info(&npdev->dev,
+ "Using 64-bit DMA iommu bypass\n");
+ pnv_npu_dma_set_bypass(npe);
+ } else {
+ dev_info(&npdev->dev, "Using 32-bit DMA via iommu\n");
+ pnv_npu_dma_set_32(npe);
+ }
+ }
+}
+
+/* Switch ownership from platform code to external user (e.g. VFIO) */
+void pnv_npu_take_ownership(struct pnv_ioda_pe *npe)
+{
+ struct pnv_phb *phb = npe->phb;
+ int64_t rc;
+
+ /*
+ * Note: NPU has just a single TVE in the hardware which means that
+ * while used by the kernel, it can have either 32bit window or
+ * DMA bypass but never both. So we deconfigure 32bit window only
+ * if it was enabled at the moment of ownership change.
+ */
+ if (npe->table_group.tables[0]) {
+ pnv_npu_unset_window(npe, 0);
+ return;
}
- if (bypass)
- dev_info(&npdev->dev, "Using 64-bit DMA iommu bypass\n");
- else
- dev_info(&npdev->dev, "Using 32-bit DMA via iommu\n");
+ /* Disable bypass */
+ rc = opal_pci_map_pe_dma_window_real(phb->opal_id,
+ npe->pe_number, npe->pe_number,
+ 0 /* bypass base */, 0);
+ if (rc) {
+ pe_err(npe, "Failed to disable bypass, err %lld\n", rc);
+ return;
+ }
+ pnv_pci_ioda2_tce_invalidate_entire(npe->phb, false);
+}
- pnv_npu_dma_set_bypass(npe, bypass);
- *npdev->dev.dma_mask = dma_mask;
+struct pnv_ioda_pe *pnv_pci_npu_setup_iommu(struct pnv_ioda_pe *npe)
+{
+ struct pnv_phb *phb = npe->phb;
+ struct pci_bus *pbus = phb->hose->bus;
+ struct pci_dev *npdev, *gpdev = NULL, *gptmp;
+ struct pnv_ioda_pe *gpe = get_gpu_pci_dev_and_pe(npe, &gpdev);
- return 0;
+ if (!gpe || !gpdev)
+ return NULL;
+
+ list_for_each_entry(npdev, &pbus->devices, bus_list) {
+ gptmp = pnv_pci_get_gpu_dev(npdev);
+
+ if (gptmp != gpdev)
+ continue;
+
+ pe_info(gpe, "Attached NPU %s\n", dev_name(&npdev->dev));
+ iommu_group_add_device(gpe->table_group.group, &npdev->dev);
+ }
+
+ return gpe;
}
diff --git a/arch/powerpc/platforms/powernv/opal-hmi.c b/arch/powerpc/platforms/powernv/opal-hmi.c
index d000f4e..c0a8201 100644
--- a/arch/powerpc/platforms/powernv/opal-hmi.c
+++ b/arch/powerpc/platforms/powernv/opal-hmi.c
@@ -150,15 +150,17 @@
static void print_checkstop_reason(const char *level,
struct OpalHMIEvent *hmi_evt)
{
- switch (hmi_evt->u.xstop_error.xstop_type) {
+ uint8_t type = hmi_evt->u.xstop_error.xstop_type;
+ switch (type) {
case CHECKSTOP_TYPE_CORE:
print_core_checkstop_reason(level, hmi_evt);
break;
case CHECKSTOP_TYPE_NX:
print_nx_checkstop_reason(level, hmi_evt);
break;
- case CHECKSTOP_TYPE_UNKNOWN:
- printk("%s Unknown Malfunction Alert.\n", level);
+ default:
+ printk("%s Unknown Malfunction Alert of type %d\n",
+ level, type);
break;
}
}
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index c5baaf3..3a5ea82 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -48,15 +48,16 @@
#include "powernv.h"
#include "pci.h"
-/* 256M DMA window, 4K TCE pages, 8 bytes TCE */
-#define TCE32_TABLE_SIZE ((0x10000000 / 0x1000) * 8)
+#define PNV_IODA1_M64_NUM 16 /* Number of M64 BARs */
+#define PNV_IODA1_M64_SEGS 8 /* Segments per M64 BAR */
+#define PNV_IODA1_DMA32_SEGSIZE 0x10000000
#define POWERNV_IOMMU_DEFAULT_LEVELS 1
#define POWERNV_IOMMU_MAX_LEVELS 5
static void pnv_pci_ioda2_table_free_pages(struct iommu_table *tbl);
-static void pe_level_printk(const struct pnv_ioda_pe *pe, const char *level,
+void pe_level_printk(const struct pnv_ioda_pe *pe, const char *level,
const char *fmt, ...)
{
struct va_format vaf;
@@ -87,13 +88,6 @@
va_end(args);
}
-#define pe_err(pe, fmt, ...) \
- pe_level_printk(pe, KERN_ERR, fmt, ##__VA_ARGS__)
-#define pe_warn(pe, fmt, ...) \
- pe_level_printk(pe, KERN_WARNING, fmt, ##__VA_ARGS__)
-#define pe_info(pe, fmt, ...) \
- pe_level_printk(pe, KERN_INFO, fmt, ##__VA_ARGS__)
-
static bool pnv_iommu_bypass_disabled __read_mostly;
static int __init iommu_setup(char *str)
@@ -122,9 +116,17 @@
(IORESOURCE_MEM_64 | IORESOURCE_PREFETCH));
}
+static struct pnv_ioda_pe *pnv_ioda_init_pe(struct pnv_phb *phb, int pe_no)
+{
+ phb->ioda.pe_array[pe_no].phb = phb;
+ phb->ioda.pe_array[pe_no].pe_number = pe_no;
+
+ return &phb->ioda.pe_array[pe_no];
+}
+
static void pnv_ioda_reserve_pe(struct pnv_phb *phb, int pe_no)
{
- if (!(pe_no >= 0 && pe_no < phb->ioda.total_pe)) {
+ if (!(pe_no >= 0 && pe_no < phb->ioda.total_pe_num)) {
pr_warn("%s: Invalid PE %d on PHB#%x\n",
__func__, pe_no, phb->hose->global_number);
return;
@@ -134,32 +136,31 @@
pr_debug("%s: PE %d was reserved on PHB#%x\n",
__func__, pe_no, phb->hose->global_number);
- phb->ioda.pe_array[pe_no].phb = phb;
- phb->ioda.pe_array[pe_no].pe_number = pe_no;
+ pnv_ioda_init_pe(phb, pe_no);
}
-static int pnv_ioda_alloc_pe(struct pnv_phb *phb)
+static struct pnv_ioda_pe *pnv_ioda_alloc_pe(struct pnv_phb *phb)
{
unsigned long pe;
do {
pe = find_next_zero_bit(phb->ioda.pe_alloc,
- phb->ioda.total_pe, 0);
- if (pe >= phb->ioda.total_pe)
- return IODA_INVALID_PE;
+ phb->ioda.total_pe_num, 0);
+ if (pe >= phb->ioda.total_pe_num)
+ return NULL;
} while(test_and_set_bit(pe, phb->ioda.pe_alloc));
- phb->ioda.pe_array[pe].phb = phb;
- phb->ioda.pe_array[pe].pe_number = pe;
- return pe;
+ return pnv_ioda_init_pe(phb, pe);
}
-static void pnv_ioda_free_pe(struct pnv_phb *phb, int pe)
+static void pnv_ioda_free_pe(struct pnv_ioda_pe *pe)
{
- WARN_ON(phb->ioda.pe_array[pe].pdev);
+ struct pnv_phb *phb = pe->phb;
- memset(&phb->ioda.pe_array[pe], 0, sizeof(struct pnv_ioda_pe));
- clear_bit(pe, phb->ioda.pe_alloc);
+ WARN_ON(pe->pdev);
+
+ memset(pe, 0, sizeof(struct pnv_ioda_pe));
+ clear_bit(pe->pe_number, phb->ioda.pe_alloc);
}
/* The default M64 BAR is shared by all PEs */
@@ -199,13 +200,13 @@
* expected to be 0 or last one of PE capabicity.
*/
r = &phb->hose->mem_resources[1];
- if (phb->ioda.reserved_pe == 0)
+ if (phb->ioda.reserved_pe_idx == 0)
r->start += phb->ioda.m64_segsize;
- else if (phb->ioda.reserved_pe == (phb->ioda.total_pe - 1))
+ else if (phb->ioda.reserved_pe_idx == (phb->ioda.total_pe_num - 1))
r->end -= phb->ioda.m64_segsize;
else
pr_warn(" Cannot strip M64 segment for reserved PE#%d\n",
- phb->ioda.reserved_pe);
+ phb->ioda.reserved_pe_idx);
return 0;
@@ -219,7 +220,7 @@
return -EIO;
}
-static void pnv_ioda2_reserve_dev_m64_pe(struct pci_dev *pdev,
+static void pnv_ioda_reserve_dev_m64_pe(struct pci_dev *pdev,
unsigned long *pe_bitmap)
{
struct pci_controller *hose = pci_bus_to_host(pdev->bus);
@@ -246,22 +247,80 @@
}
}
-static void pnv_ioda2_reserve_m64_pe(struct pci_bus *bus,
- unsigned long *pe_bitmap,
- bool all)
+static int pnv_ioda1_init_m64(struct pnv_phb *phb)
+{
+ struct resource *r;
+ int index;
+
+ /*
+ * There are 16 M64 BARs, each of which has 8 segments. So
+ * there are as many M64 segments as the maximum number of
+ * PEs, which is 128.
+ */
+ for (index = 0; index < PNV_IODA1_M64_NUM; index++) {
+ unsigned long base, segsz = phb->ioda.m64_segsize;
+ int64_t rc;
+
+ base = phb->ioda.m64_base +
+ index * PNV_IODA1_M64_SEGS * segsz;
+ rc = opal_pci_set_phb_mem_window(phb->opal_id,
+ OPAL_M64_WINDOW_TYPE, index, base, 0,
+ PNV_IODA1_M64_SEGS * segsz);
+ if (rc != OPAL_SUCCESS) {
+ pr_warn(" Error %lld setting M64 PHB#%d-BAR#%d\n",
+ rc, phb->hose->global_number, index);
+ goto fail;
+ }
+
+ rc = opal_pci_phb_mmio_enable(phb->opal_id,
+ OPAL_M64_WINDOW_TYPE, index,
+ OPAL_ENABLE_M64_SPLIT);
+ if (rc != OPAL_SUCCESS) {
+ pr_warn(" Error %lld enabling M64 PHB#%d-BAR#%d\n",
+ rc, phb->hose->global_number, index);
+ goto fail;
+ }
+ }
+
+ /*
+ * Exclude the segment used by the reserved PE, which
+ * is expected to be 0 or last supported PE#.
+ */
+ r = &phb->hose->mem_resources[1];
+ if (phb->ioda.reserved_pe_idx == 0)
+ r->start += phb->ioda.m64_segsize;
+ else if (phb->ioda.reserved_pe_idx == (phb->ioda.total_pe_num - 1))
+ r->end -= phb->ioda.m64_segsize;
+ else
+ WARN(1, "Wrong reserved PE#%d on PHB#%d\n",
+ phb->ioda.reserved_pe_idx, phb->hose->global_number);
+
+ return 0;
+
+fail:
+ for ( ; index >= 0; index--)
+ opal_pci_phb_mmio_enable(phb->opal_id,
+ OPAL_M64_WINDOW_TYPE, index, OPAL_DISABLE_M64);
+
+ return -EIO;
+}
+
+static void pnv_ioda_reserve_m64_pe(struct pci_bus *bus,
+ unsigned long *pe_bitmap,
+ bool all)
{
struct pci_dev *pdev;
list_for_each_entry(pdev, &bus->devices, bus_list) {
- pnv_ioda2_reserve_dev_m64_pe(pdev, pe_bitmap);
+ pnv_ioda_reserve_dev_m64_pe(pdev, pe_bitmap);
if (all && pdev->subordinate)
- pnv_ioda2_reserve_m64_pe(pdev->subordinate,
- pe_bitmap, all);
+ pnv_ioda_reserve_m64_pe(pdev->subordinate,
+ pe_bitmap, all);
}
}
-static int pnv_ioda2_pick_m64_pe(struct pci_bus *bus, bool all)
+static struct pnv_ioda_pe *pnv_ioda_pick_m64_pe(struct pci_bus *bus, bool all)
{
struct pci_controller *hose = pci_bus_to_host(bus);
struct pnv_phb *phb = hose->private_data;
@@ -271,28 +330,28 @@
/* Root bus shouldn't use M64 */
if (pci_is_root_bus(bus))
- return IODA_INVALID_PE;
+ return NULL;
/* Allocate bitmap */
- size = _ALIGN_UP(phb->ioda.total_pe / 8, sizeof(unsigned long));
+ size = _ALIGN_UP(phb->ioda.total_pe_num / 8, sizeof(unsigned long));
pe_alloc = kzalloc(size, GFP_KERNEL);
if (!pe_alloc) {
pr_warn("%s: Out of memory !\n",
__func__);
- return IODA_INVALID_PE;
+ return NULL;
}
/* Figure out reserved PE numbers by the PE */
- pnv_ioda2_reserve_m64_pe(bus, pe_alloc, all);
+ pnv_ioda_reserve_m64_pe(bus, pe_alloc, all);
/*
* the current bus might not own M64 window and that's all
* contributed by its child buses. For the case, we needn't
* pick M64 dependent PE#.
*/
- if (bitmap_empty(pe_alloc, phb->ioda.total_pe)) {
+ if (bitmap_empty(pe_alloc, phb->ioda.total_pe_num)) {
kfree(pe_alloc);
- return IODA_INVALID_PE;
+ return NULL;
}
/*
@@ -301,10 +360,11 @@
*/
master_pe = NULL;
i = -1;
- while ((i = find_next_bit(pe_alloc, phb->ioda.total_pe, i + 1)) <
- phb->ioda.total_pe) {
+ while ((i = find_next_bit(pe_alloc, phb->ioda.total_pe_num, i + 1)) <
+ phb->ioda.total_pe_num) {
pe = &phb->ioda.pe_array[i];
+ phb->ioda.m64_segmap[pe->pe_number] = pe->pe_number;
if (!master_pe) {
pe->flags |= PNV_IODA_PE_MASTER;
INIT_LIST_HEAD(&pe->slaves);
@@ -314,10 +374,30 @@
pe->master = master_pe;
list_add_tail(&pe->list, &master_pe->slaves);
}
+
+ /*
+ * P7IOC supports M64DT, which helps mapping M64 segment
+ * to one particular PE#. However, PHB3 has fixed mapping
+ * between M64 segment and PE#. In order to have same logic
+ * for P7IOC and PHB3, we enforce fixed mapping between M64
+ * segment and PE# on P7IOC.
+ */
+ if (phb->type == PNV_PHB_IODA1) {
+ int64_t rc;
+
+ rc = opal_pci_map_pe_mmio_window(phb->opal_id,
+ pe->pe_number, OPAL_M64_WINDOW_TYPE,
+ pe->pe_number / PNV_IODA1_M64_SEGS,
+ pe->pe_number % PNV_IODA1_M64_SEGS);
+ if (rc != OPAL_SUCCESS)
+ pr_warn("%s: Error %lld mapping M64 for PHB#%d-PE#%d\n",
+ __func__, rc, phb->hose->global_number,
+ pe->pe_number);
+ }
}
kfree(pe_alloc);
- return master_pe->pe_number;
+ return master_pe;
}
static void __init pnv_ioda_parse_m64_window(struct pnv_phb *phb)
@@ -328,8 +408,7 @@
const u32 *r;
u64 pci_addr;
- /* FIXME: Support M64 for P7IOC */
- if (phb->type != PNV_PHB_IODA2) {
+ if (phb->type != PNV_PHB_IODA1 && phb->type != PNV_PHB_IODA2) {
pr_info(" Not support M64 window\n");
return;
}
@@ -355,7 +434,7 @@
hose->mem_offset[1] = res->start - pci_addr;
phb->ioda.m64_size = resource_size(res);
- phb->ioda.m64_segsize = phb->ioda.m64_size / phb->ioda.total_pe;
+ phb->ioda.m64_segsize = phb->ioda.m64_size / phb->ioda.total_pe_num;
phb->ioda.m64_base = pci_addr;
pr_info(" MEM64 0x%016llx..0x%016llx -> 0x%016llx\n",
@@ -363,9 +442,12 @@
/* Use last M64 BAR to cover M64 window */
phb->ioda.m64_bar_idx = 15;
- phb->init_m64 = pnv_ioda2_init_m64;
- phb->reserve_m64_pe = pnv_ioda2_reserve_m64_pe;
- phb->pick_m64_pe = pnv_ioda2_pick_m64_pe;
+ if (phb->type == PNV_PHB_IODA1)
+ phb->init_m64 = pnv_ioda1_init_m64;
+ else
+ phb->init_m64 = pnv_ioda2_init_m64;
+ phb->reserve_m64_pe = pnv_ioda_reserve_m64_pe;
+ phb->pick_m64_pe = pnv_ioda_pick_m64_pe;
}
static void pnv_ioda_freeze_pe(struct pnv_phb *phb, int pe_no)
@@ -456,7 +538,7 @@
s64 rc;
/* Sanity check on PE number */
- if (pe_no < 0 || pe_no >= phb->ioda.total_pe)
+ if (pe_no < 0 || pe_no >= phb->ioda.total_pe_num)
return OPAL_EEH_STOPPED_PERM_UNAVAIL;
/*
@@ -808,44 +890,6 @@
return 0;
}
-static void pnv_ioda_link_pe_by_weight(struct pnv_phb *phb,
- struct pnv_ioda_pe *pe)
-{
- struct pnv_ioda_pe *lpe;
-
- list_for_each_entry(lpe, &phb->ioda.pe_dma_list, dma_link) {
- if (lpe->dma_weight < pe->dma_weight) {
- list_add_tail(&pe->dma_link, &lpe->dma_link);
- return;
- }
- }
- list_add_tail(&pe->dma_link, &phb->ioda.pe_dma_list);
-}
-
-static unsigned int pnv_ioda_dma_weight(struct pci_dev *dev)
-{
- /* This is quite simplistic. The "base" weight of a device
- * is 10. 0 means no DMA is to be accounted for it.
- */
-
- /* If it's a bridge, no DMA */
- if (dev->hdr_type != PCI_HEADER_TYPE_NORMAL)
- return 0;
-
- /* Reduce the weight of slow USB controllers */
- if (dev->class == PCI_CLASS_SERIAL_USB_UHCI ||
- dev->class == PCI_CLASS_SERIAL_USB_OHCI ||
- dev->class == PCI_CLASS_SERIAL_USB_EHCI)
- return 3;
-
- /* Increase the weight of RAID (includes Obsidian) */
- if ((dev->class >> 8) == PCI_CLASS_STORAGE_RAID)
- return 15;
-
- /* Default */
- return 10;
-}
-
#ifdef CONFIG_PCI_IOV
static int pnv_pci_vf_resource_shift(struct pci_dev *dev, int offset)
{
@@ -919,7 +963,6 @@
struct pnv_phb *phb = hose->private_data;
struct pci_dn *pdn = pci_get_pdn(dev);
struct pnv_ioda_pe *pe;
- int pe_num;
if (!pdn) {
pr_err("%s: Device tree node not associated properly\n",
@@ -929,8 +972,8 @@
if (pdn->pe_number != IODA_INVALID_PE)
return NULL;
- pe_num = pnv_ioda_alloc_pe(phb);
- if (pe_num == IODA_INVALID_PE) {
+ pe = pnv_ioda_alloc_pe(phb);
+ if (!pe) {
pr_warning("%s: Not enough PE# available, disabling device\n",
pci_name(dev));
return NULL;
@@ -943,14 +986,12 @@
*
* At some point we want to remove the PDN completely anyways
*/
- pe = &phb->ioda.pe_array[pe_num];
pci_dev_get(dev);
pdn->pcidev = dev;
- pdn->pe_number = pe_num;
+ pdn->pe_number = pe->pe_number;
pe->flags = PNV_IODA_PE_DEV;
pe->pdev = dev;
pe->pbus = NULL;
- pe->tce32_seg = -1;
pe->mve_number = -1;
pe->rid = dev->bus->number << 8 | pdn->devfn;
@@ -958,23 +999,15 @@
if (pnv_ioda_configure_pe(phb, pe)) {
/* XXX What do we do here ? */
- if (pe_num)
- pnv_ioda_free_pe(phb, pe_num);
+ pnv_ioda_free_pe(pe);
pdn->pe_number = IODA_INVALID_PE;
pe->pdev = NULL;
pci_dev_put(dev);
return NULL;
}
- /* Assign a DMA weight to the device */
- pe->dma_weight = pnv_ioda_dma_weight(dev);
- if (pe->dma_weight != 0) {
- phb->ioda.dma_weight += pe->dma_weight;
- phb->ioda.dma_pe_count++;
- }
-
- /* Link the PE */
- pnv_ioda_link_pe_by_weight(phb, pe);
+ /* Put PE to the list */
+ list_add_tail(&pe->list, &phb->ioda.pe_list);
return pe;
}
@@ -993,7 +1026,6 @@
}
pdn->pcidev = dev;
pdn->pe_number = pe->pe_number;
- pe->dma_weight += pnv_ioda_dma_weight(dev);
if ((pe->flags & PNV_IODA_PE_BUS_ALL) && dev->subordinate)
pnv_ioda_setup_same_PE(dev->subordinate, pe);
}
@@ -1005,49 +1037,44 @@
* subordinate PCI devices and buses. The second type of PE is normally
* orgiriated by PCIe-to-PCI bridge or PLX switch downstream ports.
*/
-static void pnv_ioda_setup_bus_PE(struct pci_bus *bus, bool all)
+static struct pnv_ioda_pe *pnv_ioda_setup_bus_PE(struct pci_bus *bus, bool all)
{
struct pci_controller *hose = pci_bus_to_host(bus);
struct pnv_phb *phb = hose->private_data;
- struct pnv_ioda_pe *pe;
- int pe_num = IODA_INVALID_PE;
+ struct pnv_ioda_pe *pe = NULL;
/* Check if PE is determined by M64 */
if (phb->pick_m64_pe)
- pe_num = phb->pick_m64_pe(bus, all);
+ pe = phb->pick_m64_pe(bus, all);
/* The PE number isn't pinned by M64 */
- if (pe_num == IODA_INVALID_PE)
- pe_num = pnv_ioda_alloc_pe(phb);
+ if (!pe)
+ pe = pnv_ioda_alloc_pe(phb);
- if (pe_num == IODA_INVALID_PE) {
+ if (!pe) {
pr_warning("%s: Not enough PE# available for PCI bus %04x:%02x\n",
__func__, pci_domain_nr(bus), bus->number);
- return;
+ return NULL;
}
- pe = &phb->ioda.pe_array[pe_num];
pe->flags |= (all ? PNV_IODA_PE_BUS_ALL : PNV_IODA_PE_BUS);
pe->pbus = bus;
pe->pdev = NULL;
- pe->tce32_seg = -1;
pe->mve_number = -1;
pe->rid = bus->busn_res.start << 8;
- pe->dma_weight = 0;
if (all)
pe_info(pe, "Secondary bus %d..%d associated with PE#%d\n",
- bus->busn_res.start, bus->busn_res.end, pe_num);
+ bus->busn_res.start, bus->busn_res.end, pe->pe_number);
else
pe_info(pe, "Secondary bus %d associated with PE#%d\n",
- bus->busn_res.start, pe_num);
+ bus->busn_res.start, pe->pe_number);
if (pnv_ioda_configure_pe(phb, pe)) {
/* XXX What do we do here ? */
- if (pe_num)
- pnv_ioda_free_pe(phb, pe_num);
+ pnv_ioda_free_pe(pe);
pe->pbus = NULL;
- return;
+ return NULL;
}
/* Associate it with all child devices */
@@ -1056,16 +1083,7 @@
/* Put PE to the list */
list_add_tail(&pe->list, &phb->ioda.pe_list);
- /* Account for one DMA PE if at least one DMA capable device exist
- * below the bridge
- */
- if (pe->dma_weight != 0) {
- phb->ioda.dma_weight += pe->dma_weight;
- phb->ioda.dma_pe_count++;
- }
-
- /* Link the PE */
- pnv_ioda_link_pe_by_weight(phb, pe);
+ return pe;
}
static struct pnv_ioda_pe *pnv_ioda_setup_npu_PE(struct pci_dev *npu_pdev)
@@ -1088,7 +1106,7 @@
* same GPU get assigned the same PE.
*/
gpu_pdev = pnv_pci_get_gpu_dev(npu_pdev);
- for (pe_num = 0; pe_num < phb->ioda.total_pe; pe_num++) {
+ for (pe_num = 0; pe_num < phb->ioda.total_pe_num; pe_num++) {
pe = &phb->ioda.pe_array[pe_num];
if (!pe->pdev)
continue;
@@ -1106,7 +1124,6 @@
rid = npu_pdev->bus->number << 8 | npu_pdn->devfn;
npu_pdn->pcidev = npu_pdev;
npu_pdn->pe_number = pe_num;
- pe->dma_weight += pnv_ioda_dma_weight(npu_pdev);
phb->ioda.pe_rmap[rid] = pe->pe_number;
/* Map the PE to this link */
@@ -1378,7 +1395,7 @@
pnv_ioda_deconfigure_pe(phb, pe);
- pnv_ioda_free_pe(phb, pe->pe_number);
+ pnv_ioda_free_pe(pe);
}
}
@@ -1387,6 +1404,7 @@
struct pci_bus *bus;
struct pci_controller *hose;
struct pnv_phb *phb;
+ struct pnv_ioda_pe *pe;
struct pci_dn *pdn;
struct pci_sriov *iov;
u16 num_vfs, i;
@@ -1411,8 +1429,11 @@
/* Release PE numbers */
if (pdn->m64_single_mode) {
for (i = 0; i < num_vfs; i++) {
- if (pdn->pe_num_map[i] != IODA_INVALID_PE)
- pnv_ioda_free_pe(phb, pdn->pe_num_map[i]);
+ if (pdn->pe_num_map[i] == IODA_INVALID_PE)
+ continue;
+
+ pe = &phb->ioda.pe_array[pdn->pe_num_map[i]];
+ pnv_ioda_free_pe(pe);
}
} else
bitmap_clear(phb->ioda.pe_alloc, *pdn->pe_num_map, num_vfs);
@@ -1454,7 +1475,6 @@
pe->flags = PNV_IODA_PE_VF;
pe->pbus = NULL;
pe->parent_dev = pdev;
- pe->tce32_seg = -1;
pe->mve_number = -1;
pe->rid = (pci_iov_virtfn_bus(pdev, vf_index) << 8) |
pci_iov_virtfn_devfn(pdev, vf_index);
@@ -1466,8 +1486,7 @@
if (pnv_ioda_configure_pe(phb, pe)) {
/* XXX What do we do here ? */
- if (pe_num)
- pnv_ioda_free_pe(phb, pe_num);
+ pnv_ioda_free_pe(pe);
pe->pdev = NULL;
continue;
}
@@ -1486,6 +1505,7 @@
struct pci_bus *bus;
struct pci_controller *hose;
struct pnv_phb *phb;
+ struct pnv_ioda_pe *pe;
struct pci_dn *pdn;
int ret;
u16 i;
@@ -1528,18 +1548,20 @@
/* Calculate available PE for required VFs */
if (pdn->m64_single_mode) {
for (i = 0; i < num_vfs; i++) {
- pdn->pe_num_map[i] = pnv_ioda_alloc_pe(phb);
- if (pdn->pe_num_map[i] == IODA_INVALID_PE) {
+ pe = pnv_ioda_alloc_pe(phb);
+ if (!pe) {
ret = -EBUSY;
goto m64_failed;
}
+
+ pdn->pe_num_map[i] = pe->pe_number;
}
} else {
mutex_lock(&phb->ioda.pe_alloc_mutex);
*pdn->pe_num_map = bitmap_find_next_zero_area(
- phb->ioda.pe_alloc, phb->ioda.total_pe,
+ phb->ioda.pe_alloc, phb->ioda.total_pe_num,
0, num_vfs, 0);
- if (*pdn->pe_num_map >= phb->ioda.total_pe) {
+ if (*pdn->pe_num_map >= phb->ioda.total_pe_num) {
mutex_unlock(&phb->ioda.pe_alloc_mutex);
dev_info(&pdev->dev, "Failed to enable VF%d\n", num_vfs);
kfree(pdn->pe_num_map);
@@ -1577,8 +1599,11 @@
m64_failed:
if (pdn->m64_single_mode) {
for (i = 0; i < num_vfs; i++) {
- if (pdn->pe_num_map[i] != IODA_INVALID_PE)
- pnv_ioda_free_pe(phb, pdn->pe_num_map[i]);
+ if (pdn->pe_num_map[i] == IODA_INVALID_PE)
+ continue;
+
+ pe = &phb->ioda.pe_array[pdn->pe_num_map[i]];
+ pnv_ioda_free_pe(pe);
}
} else
bitmap_clear(phb->ioda.pe_alloc, *pdn->pe_num_map, num_vfs);
@@ -1640,8 +1665,6 @@
struct pnv_ioda_pe *pe;
uint64_t top;
bool bypass = false;
- struct pci_dev *linked_npu_dev;
- int i;
if (WARN_ON(!pdn || pdn->pe_number == IODA_INVALID_PE))
return -ENODEV;;
@@ -1662,15 +1685,7 @@
*pdev->dev.dma_mask = dma_mask;
/* Update peer npu devices */
- if (pe->flags & PNV_IODA_PE_PEER)
- for (i = 0; i < PNV_IODA_MAX_PEER_PES; i++) {
- if (!pe->peers[i])
- continue;
-
- linked_npu_dev = pe->peers[i]->pdev;
- if (dma_get_mask(&linked_npu_dev->dev) != dma_mask)
- dma_set_mask(&linked_npu_dev->dev, dma_mask);
- }
+ pnv_npu_try_dma_set_bypass(pdev, bypass);
return 0;
}
@@ -1811,28 +1826,34 @@
.get = pnv_tce_get,
};
-static inline void pnv_pci_ioda2_tce_invalidate_entire(struct pnv_ioda_pe *pe)
+#define TCE_KILL_INVAL_ALL PPC_BIT(0)
+#define TCE_KILL_INVAL_PE PPC_BIT(1)
+#define TCE_KILL_INVAL_TCE PPC_BIT(2)
+
+void pnv_pci_ioda2_tce_invalidate_entire(struct pnv_phb *phb, bool rm)
+{
+ const unsigned long val = TCE_KILL_INVAL_ALL;
+
+ mb(); /* Ensure previous TCE table stores are visible */
+ if (rm)
+ __raw_rm_writeq(cpu_to_be64(val),
+ (__be64 __iomem *)
+ phb->ioda.tce_inval_reg_phys);
+ else
+ __raw_writeq(cpu_to_be64(val), phb->ioda.tce_inval_reg);
+}
+
+static inline void pnv_pci_ioda2_tce_invalidate_pe(struct pnv_ioda_pe *pe)
{
/* 01xb - invalidate TCEs that match the specified PE# */
- unsigned long val = (0x4ull << 60) | (pe->pe_number & 0xFF);
+ unsigned long val = TCE_KILL_INVAL_PE | (pe->pe_number & 0xFF);
struct pnv_phb *phb = pe->phb;
- struct pnv_ioda_pe *npe;
- int i;
if (!phb->ioda.tce_inval_reg)
return;
mb(); /* Ensure above stores are visible */
__raw_writeq(cpu_to_be64(val), phb->ioda.tce_inval_reg);
-
- if (pe->flags & PNV_IODA_PE_PEER)
- for (i = 0; i < PNV_IODA_MAX_PEER_PES; i++) {
- npe = pe->peers[i];
- if (!npe || npe->phb->type != PNV_PHB_NPU)
- continue;
-
- pnv_npu_tce_invalidate_entire(npe);
- }
}
static void pnv_pci_ioda2_do_tce_invalidate(unsigned pe_number, bool rm,
@@ -1842,7 +1863,7 @@
unsigned long start, end, inc;
/* We'll invalidate DMA address in PE scope */
- start = 0x2ull << 60;
+ start = TCE_KILL_INVAL_TCE;
start |= (pe_number & 0xFF);
end = start;
@@ -1867,28 +1888,24 @@
struct iommu_table_group_link *tgl;
list_for_each_entry_rcu(tgl, &tbl->it_group_list, next) {
- struct pnv_ioda_pe *npe;
struct pnv_ioda_pe *pe = container_of(tgl->table_group,
struct pnv_ioda_pe, table_group);
__be64 __iomem *invalidate = rm ?
(__be64 __iomem *)pe->phb->ioda.tce_inval_reg_phys :
pe->phb->ioda.tce_inval_reg;
- int i;
+ if (pe->phb->type == PNV_PHB_NPU) {
+ /*
+ * The NVLink hardware does not support TCE kill
+ * per TCE entry so we have to invalidate
+ * the entire cache for it.
+ */
+ pnv_pci_ioda2_tce_invalidate_entire(pe->phb, rm);
+ continue;
+ }
pnv_pci_ioda2_do_tce_invalidate(pe->pe_number, rm,
invalidate, tbl->it_page_shift,
index, npages);
-
- if (pe->flags & PNV_IODA_PE_PEER)
- /* Invalidate PEs using the same TCE table */
- for (i = 0; i < PNV_IODA_MAX_PEER_PES; i++) {
- npe = pe->peers[i];
- if (!npe || npe->phb->type != PNV_PHB_NPU)
- continue;
-
- pnv_npu_tce_invalidate(npe, tbl, index,
- npages, rm);
- }
}
}
@@ -1945,56 +1962,140 @@
.free = pnv_ioda2_table_free,
};
-static void pnv_pci_ioda_setup_dma_pe(struct pnv_phb *phb,
- struct pnv_ioda_pe *pe, unsigned int base,
- unsigned int segs)
+static int pnv_pci_ioda_dev_dma_weight(struct pci_dev *dev, void *data)
+{
+ unsigned int *weight = (unsigned int *)data;
+
+ /* This is quite simplistic. The "base" weight of a device
+ * is 10. 0 means no DMA is to be accounted for it.
+ */
+ if (dev->hdr_type != PCI_HEADER_TYPE_NORMAL)
+ return 0;
+
+ if (dev->class == PCI_CLASS_SERIAL_USB_UHCI ||
+ dev->class == PCI_CLASS_SERIAL_USB_OHCI ||
+ dev->class == PCI_CLASS_SERIAL_USB_EHCI)
+ *weight += 3;
+ else if ((dev->class >> 8) == PCI_CLASS_STORAGE_RAID)
+ *weight += 15;
+ else
+ *weight += 10;
+
+ return 0;
+}
+
+static unsigned int pnv_pci_ioda_pe_dma_weight(struct pnv_ioda_pe *pe)
+{
+ unsigned int weight = 0;
+
+ /* SRIOV VF has same DMA32 weight as its PF */
+#ifdef CONFIG_PCI_IOV
+ if ((pe->flags & PNV_IODA_PE_VF) && pe->parent_dev) {
+ pnv_pci_ioda_dev_dma_weight(pe->parent_dev, &weight);
+ return weight;
+ }
+#endif
+
+ if ((pe->flags & PNV_IODA_PE_DEV) && pe->pdev) {
+ pnv_pci_ioda_dev_dma_weight(pe->pdev, &weight);
+ } else if ((pe->flags & PNV_IODA_PE_BUS) && pe->pbus) {
+ struct pci_dev *pdev;
+
+ list_for_each_entry(pdev, &pe->pbus->devices, bus_list)
+ pnv_pci_ioda_dev_dma_weight(pdev, &weight);
+ } else if ((pe->flags & PNV_IODA_PE_BUS_ALL) && pe->pbus) {
+ pci_walk_bus(pe->pbus, pnv_pci_ioda_dev_dma_weight, &weight);
+ }
+
+ return weight;
+}
+
+static void pnv_pci_ioda1_setup_dma_pe(struct pnv_phb *phb,
+ struct pnv_ioda_pe *pe)
{
struct page *tce_mem = NULL;
struct iommu_table *tbl;
- unsigned int i;
+ unsigned int weight, total_weight = 0;
+ unsigned int tce32_segsz, base, segs, avail, i;
int64_t rc;
void *addr;
/* XXX FIXME: Handle 64-bit only DMA devices */
/* XXX FIXME: Provide 64-bit DMA facilities & non-4K TCE tables etc.. */
/* XXX FIXME: Allocate multi-level tables on PHB3 */
-
- /* We shouldn't already have a 32-bit DMA associated */
- if (WARN_ON(pe->tce32_seg >= 0))
+ weight = pnv_pci_ioda_pe_dma_weight(pe);
+ if (!weight)
return;
+ pci_walk_bus(phb->hose->bus, pnv_pci_ioda_dev_dma_weight,
+ &total_weight);
+ segs = (weight * phb->ioda.dma32_count) / total_weight;
+ if (!segs)
+ segs = 1;
+
+ /*
+ * Allocate contiguous DMA32 segments. We begin with the expected
+ * number of segments. With one more attempt, the number of DMA32
+ * segments to be allocated is decreased by one until one segment
+ * is allocated successfully.
+ */
+ do {
+ for (base = 0; base <= phb->ioda.dma32_count - segs; base++) {
+ for (avail = 0, i = base; i < base + segs; i++) {
+ if (phb->ioda.dma32_segmap[i] ==
+ IODA_INVALID_PE)
+ avail++;
+ }
+
+ if (avail == segs)
+ goto found;
+ }
+ } while (--segs);
+
+ if (!segs) {
+ pe_warn(pe, "No available DMA32 segments\n");
+ return;
+ }
+
+found:
tbl = pnv_pci_table_alloc(phb->hose->node);
iommu_register_group(&pe->table_group, phb->hose->global_number,
pe->pe_number);
pnv_pci_link_table_and_group(phb->hose->node, 0, tbl, &pe->table_group);
/* Grab a 32-bit TCE table */
- pe->tce32_seg = base;
+ pe_info(pe, "DMA weight %d (%d), assigned (%d) %d DMA32 segments\n",
+ weight, total_weight, base, segs);
pe_info(pe, " Setting up 32-bit TCE table at %08x..%08x\n",
- (base << 28), ((base + segs) << 28) - 1);
+ base * PNV_IODA1_DMA32_SEGSIZE,
+ (base + segs) * PNV_IODA1_DMA32_SEGSIZE - 1);
/* XXX Currently, we allocate one big contiguous table for the
* TCEs. We only really need one chunk per 256M of TCE space
* (ie per segment) but that's an optimization for later, it
* requires some added smarts with our get/put_tce implementation
+ *
+ * Each TCE page is 4KB in size and each TCE entry occupies 8
+ * bytes
*/
+ tce32_segsz = PNV_IODA1_DMA32_SEGSIZE >> (IOMMU_PAGE_SHIFT_4K - 3);
tce_mem = alloc_pages_node(phb->hose->node, GFP_KERNEL,
- get_order(TCE32_TABLE_SIZE * segs));
+ get_order(tce32_segsz * segs));
if (!tce_mem) {
pe_err(pe, " Failed to allocate a 32-bit TCE memory\n");
goto fail;
}
addr = page_address(tce_mem);
- memset(addr, 0, TCE32_TABLE_SIZE * segs);
+ memset(addr, 0, tce32_segsz * segs);
/* Configure HW */
for (i = 0; i < segs; i++) {
rc = opal_pci_map_pe_dma_window(phb->opal_id,
pe->pe_number,
base + i, 1,
- __pa(addr) + TCE32_TABLE_SIZE * i,
- TCE32_TABLE_SIZE, 0x1000);
+ __pa(addr) + tce32_segsz * i,
+ tce32_segsz, IOMMU_PAGE_SIZE_4K);
if (rc) {
pe_err(pe, " Failed to configure 32-bit TCE table,"
" err %ld\n", rc);
@@ -2002,9 +2103,14 @@
}
}
+ /* Setup DMA32 segment mapping */
+ for (i = base; i < base + segs; i++)
+ phb->ioda.dma32_segmap[i] = pe->pe_number;
+
/* Setup linux iommu table */
- pnv_pci_setup_iommu_table(tbl, addr, TCE32_TABLE_SIZE * segs,
- base << 28, IOMMU_PAGE_SHIFT_4K);
+ pnv_pci_setup_iommu_table(tbl, addr, tce32_segsz * segs,
+ base * PNV_IODA1_DMA32_SEGSIZE,
+ IOMMU_PAGE_SHIFT_4K);
/* OPAL variant of P7IOC SW invalidated TCEs */
if (phb->ioda.tce_inval_reg)
@@ -2031,10 +2137,8 @@
return;
fail:
/* XXX Failure: Try to fallback to 64-bit only ? */
- if (pe->tce32_seg >= 0)
- pe->tce32_seg = -1;
if (tce_mem)
- __free_pages(tce_mem, get_order(TCE32_TABLE_SIZE * segs));
+ __free_pages(tce_mem, get_order(tce32_segsz * segs));
if (tbl) {
pnv_pci_unlink_table_and_group(tbl, &pe->table_group);
iommu_free_table(tbl, "pnv");
@@ -2075,7 +2179,7 @@
pnv_pci_link_table_and_group(phb->hose->node, num,
tbl, &pe->table_group);
- pnv_pci_ioda2_tce_invalidate_entire(pe);
+ pnv_pci_ioda2_tce_invalidate_pe(pe);
return 0;
}
@@ -2219,7 +2323,7 @@
if (ret)
pe_warn(pe, "Unmapping failed, ret = %ld\n", ret);
else
- pnv_pci_ioda2_tce_invalidate_entire(pe);
+ pnv_pci_ioda2_tce_invalidate_pe(pe);
pnv_pci_unlink_table_and_group(table_group->tables[num], table_group);
@@ -2288,6 +2392,116 @@
.take_ownership = pnv_ioda2_take_ownership,
.release_ownership = pnv_ioda2_release_ownership,
};
+
+static int gpe_table_group_to_npe_cb(struct device *dev, void *opaque)
+{
+ struct pci_controller *hose;
+ struct pnv_phb *phb;
+ struct pnv_ioda_pe **ptmppe = opaque;
+ struct pci_dev *pdev = container_of(dev, struct pci_dev, dev);
+ struct pci_dn *pdn = pci_get_pdn(pdev);
+
+ if (!pdn || pdn->pe_number == IODA_INVALID_PE)
+ return 0;
+
+ hose = pci_bus_to_host(pdev->bus);
+ phb = hose->private_data;
+ if (phb->type != PNV_PHB_NPU)
+ return 0;
+
+ *ptmppe = &phb->ioda.pe_array[pdn->pe_number];
+
+ return 1;
+}
+
+/*
+ * This returns PE of associated NPU.
+ * This assumes that NPU is in the same IOMMU group with GPU and there is
+ * no other PEs.
+ */
+static struct pnv_ioda_pe *gpe_table_group_to_npe(
+ struct iommu_table_group *table_group)
+{
+ struct pnv_ioda_pe *npe = NULL;
+ int ret = iommu_group_for_each_dev(table_group->group, &npe,
+ gpe_table_group_to_npe_cb);
+
+ BUG_ON(!ret || !npe);
+
+ return npe;
+}
+
+static long pnv_pci_ioda2_npu_set_window(struct iommu_table_group *table_group,
+ int num, struct iommu_table *tbl)
+{
+ long ret = pnv_pci_ioda2_set_window(table_group, num, tbl);
+
+ if (ret)
+ return ret;
+
+ ret = pnv_npu_set_window(gpe_table_group_to_npe(table_group), num, tbl);
+ if (ret)
+ pnv_pci_ioda2_unset_window(table_group, num);
+
+ return ret;
+}
+
+static long pnv_pci_ioda2_npu_unset_window(
+ struct iommu_table_group *table_group,
+ int num)
+{
+ long ret = pnv_pci_ioda2_unset_window(table_group, num);
+
+ if (ret)
+ return ret;
+
+ return pnv_npu_unset_window(gpe_table_group_to_npe(table_group), num);
+}
+
+static void pnv_ioda2_npu_take_ownership(struct iommu_table_group *table_group)
+{
+ /*
+ * Detach NPU first as pnv_ioda2_take_ownership() will destroy
+ * the iommu_table if 32bit DMA is enabled.
+ */
+ pnv_npu_take_ownership(gpe_table_group_to_npe(table_group));
+ pnv_ioda2_take_ownership(table_group);
+}
+
+static struct iommu_table_group_ops pnv_pci_ioda2_npu_ops = {
+ .get_table_size = pnv_pci_ioda2_get_table_size,
+ .create_table = pnv_pci_ioda2_create_table,
+ .set_window = pnv_pci_ioda2_npu_set_window,
+ .unset_window = pnv_pci_ioda2_npu_unset_window,
+ .take_ownership = pnv_ioda2_npu_take_ownership,
+ .release_ownership = pnv_ioda2_release_ownership,
+};
+
+static void pnv_pci_ioda_setup_iommu_api(void)
+{
+ struct pci_controller *hose, *tmp;
+ struct pnv_phb *phb;
+ struct pnv_ioda_pe *pe, *gpe;
+
+ /*
+ * Now we have all PHBs discovered, time to add NPU devices to
+ * the corresponding IOMMU groups.
+ */
+ list_for_each_entry_safe(hose, tmp, &hose_list, list_node) {
+ phb = hose->private_data;
+
+ if (phb->type != PNV_PHB_NPU)
+ continue;
+
+ list_for_each_entry(pe, &phb->ioda.pe_list, list) {
+ gpe = pnv_pci_npu_setup_iommu(pe);
+ if (gpe)
+ gpe->table_group.ops = &pnv_pci_ioda2_npu_ops;
+ }
+ }
+}
+#else /* !CONFIG_IOMMU_API */
+static void pnv_pci_ioda_setup_iommu_api(void) { };
#endif
static void pnv_pci_ioda_setup_opal_tce_kill(struct pnv_phb *phb)
@@ -2443,10 +2657,6 @@
{
int64_t rc;
- /* We shouldn't already have a 32-bit DMA associated */
- if (WARN_ON(pe->tce32_seg >= 0))
- return;
-
/* TVE #1 is selected by PCI address bit 59 */
pe->tce_bypass_base = 1ull << 59;
@@ -2454,7 +2664,6 @@
pe->pe_number);
/* The PE will reserve all possible 32-bits space */
- pe->tce32_seg = 0;
pe_info(pe, "Setting up 32-bit TCE table at 0..%08x\n",
phb->ioda.m32_pci_base);
@@ -2470,11 +2679,8 @@
#endif
rc = pnv_pci_ioda2_setup_default_config(pe);
- if (rc) {
- if (pe->tce32_seg >= 0)
- pe->tce32_seg = -1;
+ if (rc)
return;
- }
if (pe->flags & PNV_IODA_PE_DEV)
iommu_add_device(&pe->pdev->dev);
@@ -2485,47 +2691,24 @@
static void pnv_ioda_setup_dma(struct pnv_phb *phb)
{
struct pci_controller *hose = phb->hose;
- unsigned int residual, remaining, segs, tw, base;
struct pnv_ioda_pe *pe;
+ unsigned int weight;
/* If we have more PE# than segments available, hand out one
* per PE until we run out and let the rest fail. If not,
* then we assign at least one segment per PE, plus more based
* on the amount of devices under that PE
*/
- if (phb->ioda.dma_pe_count > phb->ioda.tce32_count)
- residual = 0;
- else
- residual = phb->ioda.tce32_count -
- phb->ioda.dma_pe_count;
-
- pr_info("PCI: Domain %04x has %ld available 32-bit DMA segments\n",
- hose->global_number, phb->ioda.tce32_count);
- pr_info("PCI: %d PE# for a total weight of %d\n",
- phb->ioda.dma_pe_count, phb->ioda.dma_weight);
+ pr_info("PCI: Domain %04x has %d available 32-bit DMA segments\n",
+ hose->global_number, phb->ioda.dma32_count);
pnv_pci_ioda_setup_opal_tce_kill(phb);
- /* Walk our PE list and configure their DMA segments, hand them
- * out one base segment plus any residual segments based on
- * weight
- */
- remaining = phb->ioda.tce32_count;
- tw = phb->ioda.dma_weight;
- base = 0;
- list_for_each_entry(pe, &phb->ioda.pe_dma_list, dma_link) {
- if (!pe->dma_weight)
+ /* Walk our PE list and configure their DMA segments */
+ list_for_each_entry(pe, &phb->ioda.pe_list, list) {
+ weight = pnv_pci_ioda_pe_dma_weight(pe);
+ if (!weight)
continue;
- if (!remaining) {
- pe_warn(pe, "No DMA32 resources available\n");
- continue;
- }
- segs = 1;
- if (residual) {
- segs += ((pe->dma_weight * residual) + (tw / 2)) / tw;
- if (segs > remaining)
- segs = remaining;
- }
/*
* For IODA2 compliant PHB3, we needn't care about the weight.
@@ -2533,12 +2716,9 @@
* the specific PE.
*/
if (phb->type == PNV_PHB_IODA1) {
- pe_info(pe, "DMA weight %d, assigned %d DMA32 segments\n",
- pe->dma_weight, segs);
- pnv_pci_ioda_setup_dma_pe(phb, pe, base, segs);
+ pnv_pci_ioda1_setup_dma_pe(phb, pe);
} else if (phb->type == PNV_PHB_IODA2) {
pe_info(pe, "Assign DMA32 space\n");
- segs = 0;
pnv_pci_ioda2_setup_dma_pe(phb, pe);
} else if (phb->type == PNV_PHB_NPU) {
/*
@@ -2548,9 +2728,6 @@
* as the PHB3 TVT.
*/
}
-
- remaining -= segs;
- base += segs;
}
}
@@ -2858,7 +3035,7 @@
pdn->m64_single_mode = false;
total_vfs = pci_sriov_get_totalvfs(pdev);
- mul = phb->ioda.total_pe;
+ mul = phb->ioda.total_pe_num;
total_vf_bar_sz = 0;
for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
@@ -2929,19 +3106,72 @@
}
#endif /* CONFIG_PCI_IOV */
+static void pnv_ioda_setup_pe_res(struct pnv_ioda_pe *pe,
+ struct resource *res)
+{
+ struct pnv_phb *phb = pe->phb;
+ struct pci_bus_region region;
+ int index;
+ int64_t rc;
+
+ if (!res || !res->flags || res->start > res->end)
+ return;
+
+ if (res->flags & IORESOURCE_IO) {
+ region.start = res->start - phb->ioda.io_pci_base;
+ region.end = res->end - phb->ioda.io_pci_base;
+ index = region.start / phb->ioda.io_segsize;
+
+ while (index < phb->ioda.total_pe_num &&
+ region.start <= region.end) {
+ phb->ioda.io_segmap[index] = pe->pe_number;
+ rc = opal_pci_map_pe_mmio_window(phb->opal_id,
+ pe->pe_number, OPAL_IO_WINDOW_TYPE, 0, index);
+ if (rc != OPAL_SUCCESS) {
+ pr_err("%s: Error %lld mapping IO segment#%d to PE#%d\n",
+ __func__, rc, index, pe->pe_number);
+ break;
+ }
+
+ region.start += phb->ioda.io_segsize;
+ index++;
+ }
+ } else if ((res->flags & IORESOURCE_MEM) &&
+ !pnv_pci_is_mem_pref_64(res->flags)) {
+ region.start = res->start -
+ phb->hose->mem_offset[0] -
+ phb->ioda.m32_pci_base;
+ region.end = res->end -
+ phb->hose->mem_offset[0] -
+ phb->ioda.m32_pci_base;
+ index = region.start / phb->ioda.m32_segsize;
+
+ while (index < phb->ioda.total_pe_num &&
+ region.start <= region.end) {
+ phb->ioda.m32_segmap[index] = pe->pe_number;
+ rc = opal_pci_map_pe_mmio_window(phb->opal_id,
+ pe->pe_number, OPAL_M32_WINDOW_TYPE, 0, index);
+ if (rc != OPAL_SUCCESS) {
+ pr_err("%s: Error %lld mapping M32 segment#%d to PE#%d",
+ __func__, rc, index, pe->pe_number);
+ break;
+ }
+
+ region.start += phb->ioda.m32_segsize;
+ index++;
+ }
+ }
+}
+
/*
* This function is supposed to be called on basis of PE from top
* to bottom style. So the the I/O or MMIO segment assigned to
* parent PE could be overrided by its child PEs if necessary.
*/
-static void pnv_ioda_setup_pe_seg(struct pci_controller *hose,
- struct pnv_ioda_pe *pe)
+static void pnv_ioda_setup_pe_seg(struct pnv_ioda_pe *pe)
{
- struct pnv_phb *phb = hose->private_data;
- struct pci_bus_region region;
- struct resource *res;
- int i, index;
- int rc;
+ struct pci_dev *pdev;
+ int i;
/*
* NOTE: We only care PCI bus based PE for now. For PCI
@@ -2950,57 +3180,20 @@
*/
BUG_ON(!(pe->flags & (PNV_IODA_PE_BUS | PNV_IODA_PE_BUS_ALL)));
- pci_bus_for_each_resource(pe->pbus, res, i) {
- if (!res || !res->flags ||
- res->start > res->end)
+ list_for_each_entry(pdev, &pe->pbus->devices, bus_list) {
+ for (i = 0; i <= PCI_ROM_RESOURCE; i++)
+ pnv_ioda_setup_pe_res(pe, &pdev->resource[i]);
+
+ /*
+ * If the PE contains all subordinate PCI buses, the
+ * windows of the child bridges should be mapped to
+ * the PE as well.
+ */
+ if (!(pe->flags & PNV_IODA_PE_BUS_ALL) || !pci_is_bridge(pdev))
continue;
-
- if (res->flags & IORESOURCE_IO) {
- region.start = res->start - phb->ioda.io_pci_base;
- region.end = res->end - phb->ioda.io_pci_base;
- index = region.start / phb->ioda.io_segsize;
-
- while (index < phb->ioda.total_pe &&
- region.start <= region.end) {
- phb->ioda.io_segmap[index] = pe->pe_number;
- rc = opal_pci_map_pe_mmio_window(phb->opal_id,
- pe->pe_number, OPAL_IO_WINDOW_TYPE, 0, index);
- if (rc != OPAL_SUCCESS) {
- pr_err("%s: OPAL error %d when mapping IO "
- "segment #%d to PE#%d\n",
- __func__, rc, index, pe->pe_number);
- break;
- }
-
- region.start += phb->ioda.io_segsize;
- index++;
- }
- } else if ((res->flags & IORESOURCE_MEM) &&
- !pnv_pci_is_mem_pref_64(res->flags)) {
- region.start = res->start -
- hose->mem_offset[0] -
- phb->ioda.m32_pci_base;
- region.end = res->end -
- hose->mem_offset[0] -
- phb->ioda.m32_pci_base;
- index = region.start / phb->ioda.m32_segsize;
-
- while (index < phb->ioda.total_pe &&
- region.start <= region.end) {
- phb->ioda.m32_segmap[index] = pe->pe_number;
- rc = opal_pci_map_pe_mmio_window(phb->opal_id,
- pe->pe_number, OPAL_M32_WINDOW_TYPE, 0, index);
- if (rc != OPAL_SUCCESS) {
- pr_err("%s: OPAL error %d when mapping M32 "
- "segment#%d to PE#%d",
- __func__, rc, index, pe->pe_number);
- break;
- }
-
- region.start += phb->ioda.m32_segsize;
- index++;
- }
- }
+ for (i = 0; i < PCI_BRIDGE_RESOURCE_NUM; i++)
+ pnv_ioda_setup_pe_res(pe,
+ &pdev->resource[PCI_BRIDGE_RESOURCES + i]);
}
}
@@ -3018,7 +3211,7 @@
continue;
list_for_each_entry(pe, &phb->ioda.pe_list, list) {
- pnv_ioda_setup_pe_seg(hose, pe);
+ pnv_ioda_setup_pe_seg(pe);
}
}
}
@@ -3035,6 +3228,8 @@
phb = hose->private_data;
phb->initialized = 1;
}
+
+ pnv_pci_ioda_setup_iommu_api();
}
static void pnv_pci_ioda_create_dbgfs(void)
@@ -3056,27 +3251,6 @@
#endif /* CONFIG_DEBUG_FS */
}
-static void pnv_npu_ioda_fixup(void)
-{
- bool enable_bypass;
- struct pci_controller *hose, *tmp;
- struct pnv_phb *phb;
- struct pnv_ioda_pe *pe;
-
- list_for_each_entry_safe(hose, tmp, &hose_list, list_node) {
- phb = hose->private_data;
- if (phb->type != PNV_PHB_NPU)
- continue;
-
- list_for_each_entry(pe, &phb->ioda.pe_dma_list, dma_link) {
- enable_bypass = dma_get_mask(&pe->pdev->dev) ==
- DMA_BIT_MASK(64);
- pnv_npu_init_dma_pe(pe);
- pnv_npu_dma_set_bypass(pe, enable_bypass);
- }
- }
-}
-
static void pnv_pci_ioda_fixup(void)
{
pnv_pci_ioda_setup_PEs();
@@ -3089,9 +3263,6 @@
eeh_init();
eeh_addr_cache_build();
#endif
-
- /* Link NPU IODA tables to their PCI devices. */
- pnv_npu_ioda_fixup();
}
/*
@@ -3195,12 +3366,6 @@
return true;
}
-static u32 pnv_ioda_bdfn_to_pe(struct pnv_phb *phb, struct pci_bus *bus,
- u32 devfn)
-{
- return phb->ioda.pe_rmap[(bus->number << 8) | devfn];
-}
-
static void pnv_pci_ioda_shutdown(struct pci_controller *hose)
{
struct pnv_phb *phb = hose->private_data;
@@ -3210,31 +3375,39 @@
}
static const struct pci_controller_ops pnv_pci_ioda_controller_ops = {
- .dma_dev_setup = pnv_pci_dma_dev_setup,
- .dma_bus_setup = pnv_pci_dma_bus_setup,
+ .dma_dev_setup = pnv_pci_dma_dev_setup,
+ .dma_bus_setup = pnv_pci_dma_bus_setup,
#ifdef CONFIG_PCI_MSI
- .setup_msi_irqs = pnv_setup_msi_irqs,
- .teardown_msi_irqs = pnv_teardown_msi_irqs,
+ .setup_msi_irqs = pnv_setup_msi_irqs,
+ .teardown_msi_irqs = pnv_teardown_msi_irqs,
#endif
- .enable_device_hook = pnv_pci_enable_device_hook,
- .window_alignment = pnv_pci_window_alignment,
- .reset_secondary_bus = pnv_pci_reset_secondary_bus,
- .dma_set_mask = pnv_pci_ioda_dma_set_mask,
- .dma_get_required_mask = pnv_pci_ioda_dma_get_required_mask,
- .shutdown = pnv_pci_ioda_shutdown,
+ .enable_device_hook = pnv_pci_enable_device_hook,
+ .window_alignment = pnv_pci_window_alignment,
+ .reset_secondary_bus = pnv_pci_reset_secondary_bus,
+ .dma_set_mask = pnv_pci_ioda_dma_set_mask,
+ .dma_get_required_mask = pnv_pci_ioda_dma_get_required_mask,
+ .shutdown = pnv_pci_ioda_shutdown,
};
+static int pnv_npu_dma_set_mask(struct pci_dev *npdev, u64 dma_mask)
+{
+ dev_err_once(&npdev->dev,
+ "%s operation unsupported for NVLink devices\n",
+ __func__);
+ return -EPERM;
+}
+
static const struct pci_controller_ops pnv_npu_ioda_controller_ops = {
- .dma_dev_setup = pnv_pci_dma_dev_setup,
+ .dma_dev_setup = pnv_pci_dma_dev_setup,
#ifdef CONFIG_PCI_MSI
- .setup_msi_irqs = pnv_setup_msi_irqs,
- .teardown_msi_irqs = pnv_teardown_msi_irqs,
+ .setup_msi_irqs = pnv_setup_msi_irqs,
+ .teardown_msi_irqs = pnv_teardown_msi_irqs,
#endif
- .enable_device_hook = pnv_pci_enable_device_hook,
- .window_alignment = pnv_pci_window_alignment,
- .reset_secondary_bus = pnv_pci_reset_secondary_bus,
- .dma_set_mask = pnv_npu_dma_set_mask,
- .shutdown = pnv_pci_ioda_shutdown,
+ .enable_device_hook = pnv_pci_enable_device_hook,
+ .window_alignment = pnv_pci_window_alignment,
+ .reset_secondary_bus = pnv_pci_reset_secondary_bus,
+ .dma_set_mask = pnv_npu_dma_set_mask,
+ .shutdown = pnv_pci_ioda_shutdown,
};
static void __init pnv_pci_init_ioda_phb(struct device_node *np,
@@ -3242,10 +3415,12 @@
{
struct pci_controller *hose;
struct pnv_phb *phb;
- unsigned long size, m32map_off, pemap_off, iomap_off = 0;
+ unsigned long size, m64map_off, m32map_off, pemap_off;
+ unsigned long iomap_off = 0, dma32map_off = 0;
const __be64 *prop64;
const __be32 *prop32;
int len;
+ unsigned int segno;
u64 phb_id;
void *aux;
long rc;
@@ -3306,13 +3481,13 @@
pr_err(" Failed to map registers !\n");
/* Initialize more IODA stuff */
- phb->ioda.total_pe = 1;
+ phb->ioda.total_pe_num = 1;
prop32 = of_get_property(np, "ibm,opal-num-pes", NULL);
if (prop32)
- phb->ioda.total_pe = be32_to_cpup(prop32);
+ phb->ioda.total_pe_num = be32_to_cpup(prop32);
prop32 = of_get_property(np, "ibm,opal-reserved-pe", NULL);
if (prop32)
- phb->ioda.reserved_pe = be32_to_cpup(prop32);
+ phb->ioda.reserved_pe_idx = be32_to_cpup(prop32);
/* Parse 64-bit MMIO range */
pnv_ioda_parse_m64_window(phb);
@@ -3321,36 +3496,58 @@
/* FW Has already off top 64k of M32 space (MSI space) */
phb->ioda.m32_size += 0x10000;
- phb->ioda.m32_segsize = phb->ioda.m32_size / phb->ioda.total_pe;
+ phb->ioda.m32_segsize = phb->ioda.m32_size / phb->ioda.total_pe_num;
phb->ioda.m32_pci_base = hose->mem_resources[0].start - hose->mem_offset[0];
phb->ioda.io_size = hose->pci_io_size;
- phb->ioda.io_segsize = phb->ioda.io_size / phb->ioda.total_pe;
+ phb->ioda.io_segsize = phb->ioda.io_size / phb->ioda.total_pe_num;
phb->ioda.io_pci_base = 0; /* XXX calculate this ? */
+ /* Calculate how many 32-bit TCE segments we have */
+ phb->ioda.dma32_count = phb->ioda.m32_pci_base /
+ PNV_IODA1_DMA32_SEGSIZE;
+
/* Allocate aux data & arrays. We don't have IO ports on PHB3 */
- size = _ALIGN_UP(phb->ioda.total_pe / 8, sizeof(unsigned long));
+ size = _ALIGN_UP(max_t(unsigned, phb->ioda.total_pe_num, 8) / 8,
+ sizeof(unsigned long));
+ m64map_off = size;
+ size += phb->ioda.total_pe_num * sizeof(phb->ioda.m64_segmap[0]);
m32map_off = size;
- size += phb->ioda.total_pe * sizeof(phb->ioda.m32_segmap[0]);
+ size += phb->ioda.total_pe_num * sizeof(phb->ioda.m32_segmap[0]);
if (phb->type == PNV_PHB_IODA1) {
iomap_off = size;
- size += phb->ioda.total_pe * sizeof(phb->ioda.io_segmap[0]);
+ size += phb->ioda.total_pe_num * sizeof(phb->ioda.io_segmap[0]);
+ dma32map_off = size;
+ size += phb->ioda.dma32_count *
+ sizeof(phb->ioda.dma32_segmap[0]);
}
pemap_off = size;
- size += phb->ioda.total_pe * sizeof(struct pnv_ioda_pe);
+ size += phb->ioda.total_pe_num * sizeof(struct pnv_ioda_pe);
aux = memblock_virt_alloc(size, 0);
phb->ioda.pe_alloc = aux;
+ phb->ioda.m64_segmap = aux + m64map_off;
phb->ioda.m32_segmap = aux + m32map_off;
- if (phb->type == PNV_PHB_IODA1)
+ for (segno = 0; segno < phb->ioda.total_pe_num; segno++) {
+ phb->ioda.m64_segmap[segno] = IODA_INVALID_PE;
+ phb->ioda.m32_segmap[segno] = IODA_INVALID_PE;
+ }
+ if (phb->type == PNV_PHB_IODA1) {
phb->ioda.io_segmap = aux + iomap_off;
- phb->ioda.pe_array = aux + pemap_off;
- set_bit(phb->ioda.reserved_pe, phb->ioda.pe_alloc);
+ for (segno = 0; segno < phb->ioda.total_pe_num; segno++)
+ phb->ioda.io_segmap[segno] = IODA_INVALID_PE;
- INIT_LIST_HEAD(&phb->ioda.pe_dma_list);
+ phb->ioda.dma32_segmap = aux + dma32map_off;
+ for (segno = 0; segno < phb->ioda.dma32_count; segno++)
+ phb->ioda.dma32_segmap[segno] = IODA_INVALID_PE;
+ }
+ phb->ioda.pe_array = aux + pemap_off;
+ set_bit(phb->ioda.reserved_pe_idx, phb->ioda.pe_alloc);
+
INIT_LIST_HEAD(&phb->ioda.pe_list);
mutex_init(&phb->ioda.pe_list_mutex);
/* Calculate how many 32-bit TCE segments we have */
- phb->ioda.tce32_count = phb->ioda.m32_pci_base >> 28;
+ phb->ioda.dma32_count = phb->ioda.m32_pci_base /
+ PNV_IODA1_DMA32_SEGSIZE;
#if 0 /* We should really do that ... */
rc = opal_pci_set_phb_mem_window(opal->phb_id,
@@ -3362,7 +3559,7 @@
#endif
pr_info(" %03d (%03d) PE's M32: 0x%x [segment=0x%x]\n",
- phb->ioda.total_pe, phb->ioda.reserved_pe,
+ phb->ioda.total_pe_num, phb->ioda.reserved_pe_idx,
phb->ioda.m32_size, phb->ioda.m32_segsize);
if (phb->ioda.m64_size)
pr_info(" M64: 0x%lx [segment=0x%lx]\n",
@@ -3377,12 +3574,6 @@
phb->freeze_pe = pnv_ioda_freeze_pe;
phb->unfreeze_pe = pnv_ioda_unfreeze_pe;
- /* Setup RID -> PE mapping function */
- phb->bdfn_to_pe = pnv_ioda_bdfn_to_pe;
-
- /* Setup TCEs */
- phb->dma_dev_setup = pnv_pci_ioda_dma_dev_setup;
-
/* Setup MSI support */
pnv_pci_init_ioda_msis(phb);
@@ -3395,10 +3586,12 @@
*/
ppc_md.pcibios_fixup = pnv_pci_ioda_fixup;
- if (phb->type == PNV_PHB_NPU)
+ if (phb->type == PNV_PHB_NPU) {
hose->controller_ops = pnv_npu_ioda_controller_ops;
- else
+ } else {
+ phb->dma_dev_setup = pnv_pci_ioda_dma_dev_setup;
hose->controller_ops = pnv_pci_ioda_controller_ops;
+ }
#ifdef CONFIG_PCI_IOV
ppc_md.pcibios_fixup_sriov = pnv_pci_ioda_fixup_iov_resources;
diff --git a/arch/powerpc/platforms/powernv/pci.c b/arch/powerpc/platforms/powernv/pci.c
index 73c8dc2..1d92bd9 100644
--- a/arch/powerpc/platforms/powernv/pci.c
+++ b/arch/powerpc/platforms/powernv/pci.c
@@ -39,9 +39,6 @@
/* Delay in usec */
#define PCI_RESET_DELAY_US 3000000
-#define cfg_dbg(fmt...) do { } while(0)
-//#define cfg_dbg(fmt...) printk(fmt)
-
#ifdef CONFIG_PCI_MSI
int pnv_setup_msi_irqs(struct pci_dev *pdev, int nvec, int type)
{
@@ -370,7 +367,7 @@
struct pnv_phb *phb = pdn->phb->private_data;
u8 fstate;
__be16 pcierr;
- int pe_no;
+ unsigned int pe_no;
s64 rc;
/*
@@ -380,7 +377,7 @@
*/
pe_no = pdn->pe_number;
if (pe_no == IODA_INVALID_PE) {
- pe_no = phb->ioda.reserved_pe;
+ pe_no = phb->ioda.reserved_pe_idx;
}
/*
@@ -402,8 +399,8 @@
}
}
- cfg_dbg(" -> EEH check, bdfn=%04x PE#%d fstate=%x\n",
- (pdn->busno << 8) | (pdn->devfn), pe_no, fstate);
+ pr_devel(" -> EEH check, bdfn=%04x PE#%d fstate=%x\n",
+ (pdn->busno << 8) | (pdn->devfn), pe_no, fstate);
/* Clear the frozen state if applicable */
if (fstate == OPAL_EEH_STOPPED_MMIO_FREEZE ||
@@ -451,8 +448,8 @@
return PCIBIOS_FUNC_NOT_SUPPORTED;
}
- cfg_dbg("%s: bus: %x devfn: %x +%x/%x -> %08x\n",
- __func__, pdn->busno, pdn->devfn, where, size, *val);
+ pr_devel("%s: bus: %x devfn: %x +%x/%x -> %08x\n",
+ __func__, pdn->busno, pdn->devfn, where, size, *val);
return PCIBIOS_SUCCESSFUL;
}
@@ -462,8 +459,8 @@
struct pnv_phb *phb = pdn->phb->private_data;
u32 bdfn = (pdn->busno << 8) | pdn->devfn;
- cfg_dbg("%s: bus: %x devfn: %x +%x/%x -> %08x\n",
- pdn->busno, pdn->devfn, where, size, val);
+ pr_devel("%s: bus: %x devfn: %x +%x/%x -> %08x\n",
+ __func__, pdn->busno, pdn->devfn, where, size, val);
switch (size) {
case 1:
opal_pci_config_write_byte(phb->opal_id, bdfn, where, val);
diff --git a/arch/powerpc/platforms/powernv/pci.h b/arch/powerpc/platforms/powernv/pci.h
index 3f814f3..7dee25e 100644
--- a/arch/powerpc/platforms/powernv/pci.h
+++ b/arch/powerpc/platforms/powernv/pci.h
@@ -24,7 +24,6 @@
#define PNV_IODA_PE_MASTER (1 << 3) /* Master PE in compound case */
#define PNV_IODA_PE_SLAVE (1 << 4) /* Slave PE in compound case */
#define PNV_IODA_PE_VF (1 << 5) /* PE for one VF */
-#define PNV_IODA_PE_PEER (1 << 6) /* PE has peers */
/* Data associated with a PE, including IOMMU tracking etc.. */
struct pnv_phb;
@@ -32,9 +31,6 @@
unsigned long flags;
struct pnv_phb *phb;
-#define PNV_IODA_MAX_PEER_PES 8
- struct pnv_ioda_pe *peers[PNV_IODA_MAX_PEER_PES];
-
/* A PE can be associated with a single device or an
* entire bus (& children). In the former case, pdev
* is populated, in the later case, pbus is.
@@ -53,14 +49,7 @@
/* PE number */
unsigned int pe_number;
- /* "Weight" assigned to the PE for the sake of DMA resource
- * allocations
- */
- unsigned int dma_weight;
-
/* "Base" iommu table, ie, 4K TCEs, 32-bit DMA */
- int tce32_seg;
- int tce32_segcount;
struct iommu_table_group table_group;
/* 64-bit TCE bypass region */
@@ -78,7 +67,6 @@
struct list_head slaves;
/* Link in list of PE#s */
- struct list_head dma_link;
struct list_head list;
};
@@ -110,19 +98,18 @@
unsigned int is_64, struct msi_msg *msg);
void (*dma_dev_setup)(struct pnv_phb *phb, struct pci_dev *pdev);
void (*fixup_phb)(struct pci_controller *hose);
- u32 (*bdfn_to_pe)(struct pnv_phb *phb, struct pci_bus *bus, u32 devfn);
int (*init_m64)(struct pnv_phb *phb);
void (*reserve_m64_pe)(struct pci_bus *bus,
unsigned long *pe_bitmap, bool all);
- int (*pick_m64_pe)(struct pci_bus *bus, bool all);
+ struct pnv_ioda_pe *(*pick_m64_pe)(struct pci_bus *bus, bool all);
int (*get_pe_state)(struct pnv_phb *phb, int pe_no);
void (*freeze_pe)(struct pnv_phb *phb, int pe_no);
int (*unfreeze_pe)(struct pnv_phb *phb, int pe_no, int opt);
struct {
/* Global bridge info */
- unsigned int total_pe;
- unsigned int reserved_pe;
+ unsigned int total_pe_num;
+ unsigned int reserved_pe_idx;
/* 32-bit MMIO window */
unsigned int m32_size;
@@ -141,15 +128,19 @@
unsigned int io_segsize;
unsigned int io_pci_base;
- /* PE allocation bitmap */
- unsigned long *pe_alloc;
- /* PE allocation mutex */
+ /* PE allocation */
struct mutex pe_alloc_mutex;
+ unsigned long *pe_alloc;
+ struct pnv_ioda_pe *pe_array;
/* M32 & IO segment maps */
+ unsigned int *m64_segmap;
unsigned int *m32_segmap;
unsigned int *io_segmap;
- struct pnv_ioda_pe *pe_array;
+
+ /* DMA32 segment maps - IODA1 only */
+ unsigned int dma32_count;
+ unsigned int *dma32_segmap;
/* IRQ chip */
int irq_chip_init;
@@ -167,20 +158,6 @@
*/
unsigned char pe_rmap[0x10000];
- /* 32-bit TCE tables allocation */
- unsigned long tce32_count;
-
- /* Total "weight" for the sake of DMA resources
- * allocation
- */
- unsigned int dma_weight;
- unsigned int dma_pe_count;
-
- /* Sorted list of used PE's, sorted at
- * boot for resource allocation purposes
- */
- struct list_head pe_dma_list;
-
/* TCE cache invalidate registers (physical and
* remapped)
*/
@@ -236,16 +213,23 @@
extern int pnv_setup_msi_irqs(struct pci_dev *pdev, int nvec, int type);
extern void pnv_teardown_msi_irqs(struct pci_dev *pdev);
+extern void pe_level_printk(const struct pnv_ioda_pe *pe, const char *level,
+ const char *fmt, ...);
+#define pe_err(pe, fmt, ...) \
+ pe_level_printk(pe, KERN_ERR, fmt, ##__VA_ARGS__)
+#define pe_warn(pe, fmt, ...) \
+ pe_level_printk(pe, KERN_WARNING, fmt, ##__VA_ARGS__)
+#define pe_info(pe, fmt, ...) \
+ pe_level_printk(pe, KERN_INFO, fmt, ##__VA_ARGS__)
+
/* Nvlink functions */
-extern void pnv_npu_tce_invalidate_entire(struct pnv_ioda_pe *npe);
-extern void pnv_npu_tce_invalidate(struct pnv_ioda_pe *npe,
- struct iommu_table *tbl,
- unsigned long index,
- unsigned long npages,
- bool rm);
-extern void pnv_npu_init_dma_pe(struct pnv_ioda_pe *npe);
-extern void pnv_npu_setup_dma_pe(struct pnv_ioda_pe *npe);
-extern int pnv_npu_dma_set_bypass(struct pnv_ioda_pe *npe, bool enabled);
-extern int pnv_npu_dma_set_mask(struct pci_dev *npdev, u64 dma_mask);
+extern void pnv_npu_try_dma_set_bypass(struct pci_dev *gpdev, bool bypass);
+extern void pnv_pci_ioda2_tce_invalidate_entire(struct pnv_phb *phb, bool rm);
+extern struct pnv_ioda_pe *pnv_pci_npu_setup_iommu(struct pnv_ioda_pe *npe);
+extern long pnv_npu_set_window(struct pnv_ioda_pe *npe, int num,
+ struct iommu_table *tbl);
+extern long pnv_npu_unset_window(struct pnv_ioda_pe *npe, int num);
+extern void pnv_npu_take_ownership(struct pnv_ioda_pe *npe);
+extern void pnv_npu_release_ownership(struct pnv_ioda_pe *npe);
#endif /* __POWERNV_PCI_H */
diff --git a/arch/powerpc/platforms/powernv/setup.c b/arch/powerpc/platforms/powernv/setup.c
index 1acb0c7..ee6430b 100644
--- a/arch/powerpc/platforms/powernv/setup.c
+++ b/arch/powerpc/platforms/powernv/setup.c
@@ -273,7 +273,10 @@
if (!of_flat_dt_is_compatible(root, "ibm,powernv"))
return 0;
- hpte_init_native();
+ if (IS_ENABLED(CONFIG_PPC_RADIX_MMU) && radix_enabled())
+ radix_init_native();
+ else if (IS_ENABLED(CONFIG_PPC_STD_MMU_64))
+ hpte_init_native();
if (firmware_has_feature(FW_FEATURE_OPAL))
pnv_setup_machdep_opal();
diff --git a/arch/powerpc/platforms/ps3/htab.c b/arch/powerpc/platforms/ps3/htab.c
index 2f95d33..c9a3e67 100644
--- a/arch/powerpc/platforms/ps3/htab.c
+++ b/arch/powerpc/platforms/ps3/htab.c
@@ -63,7 +63,7 @@
vflags &= ~HPTE_V_SECONDARY;
hpte_v = hpte_encode_v(vpn, psize, apsize, ssize) | vflags | HPTE_V_VALID;
- hpte_r = hpte_encode_r(ps3_mm_phys_to_lpar(pa), psize, apsize) | rflags;
+ hpte_r = hpte_encode_r(ps3_mm_phys_to_lpar(pa), psize, apsize, ssize) | rflags;
spin_lock_irqsave(&ps3_htab_lock, flags);
diff --git a/arch/powerpc/platforms/ps3/spu.c b/arch/powerpc/platforms/ps3/spu.c
index a0bca05..492b257 100644
--- a/arch/powerpc/platforms/ps3/spu.c
+++ b/arch/powerpc/platforms/ps3/spu.c
@@ -205,7 +205,7 @@
static int __init setup_areas(struct spu *spu)
{
struct table {char* name; unsigned long addr; unsigned long size;};
- static const unsigned long shadow_flags = _PAGE_NO_CACHE | 3;
+ unsigned long shadow_flags = pgprot_val(pgprot_noncached_wc(PAGE_KERNEL_RO));
spu_pdata(spu)->shadow = __ioremap(spu_pdata(spu)->shadow_addr,
sizeof(struct spe_shadow),
@@ -216,7 +216,7 @@
}
spu->local_store = (__force void *)ioremap_prot(spu->local_store_phys,
- LS_SIZE, _PAGE_NO_CACHE);
+ LS_SIZE, pgprot_val(pgprot_noncached_wc(__pgprot(0))));
if (!spu->local_store) {
pr_debug("%s:%d: ioremap local_store failed\n",
diff --git a/arch/powerpc/platforms/pseries/hotplug-memory.c b/arch/powerpc/platforms/pseries/hotplug-memory.c
index e9ff44c..2ce1385 100644
--- a/arch/powerpc/platforms/pseries/hotplug-memory.c
+++ b/arch/powerpc/platforms/pseries/hotplug-memory.c
@@ -116,6 +116,155 @@
return new_prop;
}
+static void dlpar_update_drconf_property(struct device_node *dn,
+ struct property *prop)
+{
+ struct of_drconf_cell *lmbs;
+ u32 num_lmbs, *p;
+ int i;
+
+ /* Convert the property back to BE */
+ p = prop->value;
+ num_lmbs = *p;
+ *p = cpu_to_be32(*p);
+ p++;
+
+ lmbs = (struct of_drconf_cell *)p;
+ for (i = 0; i < num_lmbs; i++) {
+ lmbs[i].base_addr = cpu_to_be64(lmbs[i].base_addr);
+ lmbs[i].drc_index = cpu_to_be32(lmbs[i].drc_index);
+ lmbs[i].flags = cpu_to_be32(lmbs[i].flags);
+ }
+
+ rtas_hp_event = true;
+ of_update_property(dn, prop);
+ rtas_hp_event = false;
+}
+
+static int dlpar_update_device_tree_lmb(struct of_drconf_cell *lmb)
+{
+ struct device_node *dn;
+ struct property *prop;
+ struct of_drconf_cell *lmbs;
+ u32 *p, num_lmbs;
+ int i;
+
+ dn = of_find_node_by_path("/ibm,dynamic-reconfiguration-memory");
+ if (!dn)
+ return -ENODEV;
+
+ prop = dlpar_clone_drconf_property(dn);
+ if (!prop) {
+ of_node_put(dn);
+ return -ENODEV;
+ }
+
+ p = prop->value;
+ num_lmbs = *p++;
+ lmbs = (struct of_drconf_cell *)p;
+
+ for (i = 0; i < num_lmbs; i++) {
+ if (lmbs[i].drc_index == lmb->drc_index) {
+ lmbs[i].flags = lmb->flags;
+ lmbs[i].aa_index = lmb->aa_index;
+
+ dlpar_update_drconf_property(dn, prop);
+ break;
+ }
+ }
+
+ of_node_put(dn);
+ return 0;
+}
+
+static u32 lookup_lmb_associativity_index(struct of_drconf_cell *lmb)
+{
+ struct device_node *parent, *lmb_node, *dr_node;
+ const u32 *lmb_assoc;
+ const u32 *assoc_arrays;
+ u32 aa_index;
+ int aa_arrays, aa_array_entries, aa_array_sz;
+ int i;
+
+ parent = of_find_node_by_path("/");
+ if (!parent)
+ return -ENODEV;
+
+ lmb_node = dlpar_configure_connector(cpu_to_be32(lmb->drc_index),
+ parent);
+ of_node_put(parent);
+ if (!lmb_node)
+ return -EINVAL;
+
+ lmb_assoc = of_get_property(lmb_node, "ibm,associativity", NULL);
+ if (!lmb_assoc) {
+ dlpar_free_cc_nodes(lmb_node);
+ return -ENODEV;
+ }
+
+ dr_node = of_find_node_by_path("/ibm,dynamic-reconfiguration-memory");
+ if (!dr_node) {
+ dlpar_free_cc_nodes(lmb_node);
+ return -ENODEV;
+ }
+
+ assoc_arrays = of_get_property(dr_node,
+ "ibm,associativity-lookup-arrays",
+ NULL);
+ of_node_put(dr_node);
+ if (!assoc_arrays) {
+ dlpar_free_cc_nodes(lmb_node);
+ return -ENODEV;
+ }
+
+ /* The ibm,associativity-lookup-arrays property is defined to be
+ * a 32-bit value specifying the number of associativity arrays
+ * followed by a 32-bitvalue specifying the number of entries per
+ * array, followed by the associativity arrays.
+ */
+ aa_arrays = be32_to_cpu(assoc_arrays[0]);
+ aa_array_entries = be32_to_cpu(assoc_arrays[1]);
+ aa_array_sz = aa_array_entries * sizeof(u32);
+
+ aa_index = -1;
+ for (i = 0; i < aa_arrays; i++) {
+ int indx = (i * aa_array_entries) + 2;
+
+ if (memcmp(&assoc_arrays[indx], &lmb_assoc[1], aa_array_sz))
+ continue;
+
+ aa_index = i;
+ break;
+ }
+
+ dlpar_free_cc_nodes(lmb_node);
+ return aa_index;
+}
+
+static int dlpar_add_device_tree_lmb(struct of_drconf_cell *lmb)
+{
+ int aa_index;
+
+ lmb->flags |= DRCONF_MEM_ASSIGNED;
+
+ aa_index = lookup_lmb_associativity_index(lmb);
+ if (aa_index < 0) {
+ pr_err("Couldn't find associativity index for drc index %x\n",
+ lmb->drc_index);
+ return aa_index;
+ }
+
+ lmb->aa_index = aa_index;
+ return dlpar_update_device_tree_lmb(lmb);
+}
+
+static int dlpar_remove_device_tree_lmb(struct of_drconf_cell *lmb)
+{
+ lmb->flags &= ~DRCONF_MEM_ASSIGNED;
+ lmb->aa_index = 0xffffffff;
+ return dlpar_update_device_tree_lmb(lmb);
+}
+
static struct memory_block *lmb_to_memblock(struct of_drconf_cell *lmb)
{
unsigned long section_nr;
@@ -243,8 +392,8 @@
memblock_remove(lmb->base_addr, block_sz);
dlpar_release_drc(lmb->drc_index);
+ dlpar_remove_device_tree_lmb(lmb);
- lmb->flags &= ~DRCONF_MEM_ASSIGNED;
return 0;
}
@@ -384,43 +533,32 @@
#endif /* CONFIG_MEMORY_HOTREMOVE */
-static int dlpar_add_lmb(struct of_drconf_cell *lmb)
+static int dlpar_add_lmb_memory(struct of_drconf_cell *lmb)
{
struct memory_block *mem_block;
unsigned long block_sz;
int nid, rc;
- if (lmb->flags & DRCONF_MEM_ASSIGNED)
- return -EINVAL;
-
block_sz = memory_block_size_bytes();
- rc = dlpar_acquire_drc(lmb->drc_index);
- if (rc)
- return rc;
-
/* Find the node id for this address */
nid = memory_add_physaddr_to_nid(lmb->base_addr);
/* Add the memory */
rc = add_memory(nid, lmb->base_addr, block_sz);
- if (rc) {
- dlpar_release_drc(lmb->drc_index);
+ if (rc)
return rc;
- }
/* Register this block of memory */
rc = memblock_add(lmb->base_addr, block_sz);
if (rc) {
remove_memory(nid, lmb->base_addr, block_sz);
- dlpar_release_drc(lmb->drc_index);
return rc;
}
mem_block = lmb_to_memblock(lmb);
if (!mem_block) {
remove_memory(nid, lmb->base_addr, block_sz);
- dlpar_release_drc(lmb->drc_index);
return -EINVAL;
}
@@ -428,7 +566,6 @@
put_device(&mem_block->dev);
if (rc) {
remove_memory(nid, lmb->base_addr, block_sz);
- dlpar_release_drc(lmb->drc_index);
return rc;
}
@@ -436,6 +573,34 @@
return 0;
}
+static int dlpar_add_lmb(struct of_drconf_cell *lmb)
+{
+ int rc;
+
+ if (lmb->flags & DRCONF_MEM_ASSIGNED)
+ return -EINVAL;
+
+ rc = dlpar_acquire_drc(lmb->drc_index);
+ if (rc)
+ return rc;
+
+ rc = dlpar_add_device_tree_lmb(lmb);
+ if (rc) {
+ pr_err("Couldn't update device tree for drc index %x\n",
+ lmb->drc_index);
+ dlpar_release_drc(lmb->drc_index);
+ return rc;
+ }
+
+ rc = dlpar_add_lmb_memory(lmb);
+ if (rc) {
+ dlpar_remove_device_tree_lmb(lmb);
+ dlpar_release_drc(lmb->drc_index);
+ }
+
+ return rc;
+}
+
static int dlpar_memory_add_by_count(u32 lmbs_to_add, struct property *prop)
{
struct of_drconf_cell *lmbs;
@@ -536,31 +701,6 @@
return rc;
}
-static void dlpar_update_drconf_property(struct device_node *dn,
- struct property *prop)
-{
- struct of_drconf_cell *lmbs;
- u32 num_lmbs, *p;
- int i;
-
- /* Convert the property back to BE */
- p = prop->value;
- num_lmbs = *p;
- *p = cpu_to_be32(*p);
- p++;
-
- lmbs = (struct of_drconf_cell *)p;
- for (i = 0; i < num_lmbs; i++) {
- lmbs[i].base_addr = cpu_to_be64(lmbs[i].base_addr);
- lmbs[i].drc_index = cpu_to_be32(lmbs[i].drc_index);
- lmbs[i].flags = cpu_to_be32(lmbs[i].flags);
- }
-
- rtas_hp_event = true;
- of_update_property(dn, prop);
- rtas_hp_event = false;
-}
-
int dlpar_memory(struct pseries_hp_errorlog *hp_elog)
{
struct device_node *dn;
@@ -608,10 +748,7 @@
break;
}
- if (rc)
- dlpar_free_drconf_property(prop);
- else
- dlpar_update_drconf_property(dn, prop);
+ dlpar_free_drconf_property(prop);
dlpar_memory_out:
of_node_put(dn);
diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c
index bd98ce2..b7dfc13 100644
--- a/arch/powerpc/platforms/pseries/iommu.c
+++ b/arch/powerpc/platforms/pseries/iommu.c
@@ -912,7 +912,8 @@
static int query_ddw(struct pci_dev *dev, const u32 *ddw_avail,
struct ddw_query_response *query)
{
- struct eeh_dev *edev;
+ struct device_node *dn;
+ struct pci_dn *pdn;
u32 cfg_addr;
u64 buid;
int ret;
@@ -923,11 +924,10 @@
* Retrieve them from the pci device, not the node with the
* dma-window property
*/
- edev = pci_dev_to_eeh_dev(dev);
- cfg_addr = edev->config_addr;
- if (edev->pe_config_addr)
- cfg_addr = edev->pe_config_addr;
- buid = edev->phb->buid;
+ dn = pci_device_to_OF_node(dev);
+ pdn = PCI_DN(dn);
+ buid = pdn->phb->buid;
+ cfg_addr = (pdn->busno << 8) | pdn->devfn;
ret = rtas_call(ddw_avail[0], 3, 5, (u32 *)query,
cfg_addr, BUID_HI(buid), BUID_LO(buid));
@@ -941,7 +941,8 @@
struct ddw_create_response *create, int page_shift,
int window_shift)
{
- struct eeh_dev *edev;
+ struct device_node *dn;
+ struct pci_dn *pdn;
u32 cfg_addr;
u64 buid;
int ret;
@@ -952,11 +953,10 @@
* Retrieve them from the pci device, not the node with the
* dma-window property
*/
- edev = pci_dev_to_eeh_dev(dev);
- cfg_addr = edev->config_addr;
- if (edev->pe_config_addr)
- cfg_addr = edev->pe_config_addr;
- buid = edev->phb->buid;
+ dn = pci_device_to_OF_node(dev);
+ pdn = PCI_DN(dn);
+ buid = pdn->phb->buid;
+ cfg_addr = (pdn->busno << 8) | pdn->devfn;
do {
/* extra outputs are LIOBN and dma-addr (hi, lo) */
diff --git a/arch/powerpc/platforms/pseries/lpar.c b/arch/powerpc/platforms/pseries/lpar.c
index 2415a0d..7f6100d 100644
--- a/arch/powerpc/platforms/pseries/lpar.c
+++ b/arch/powerpc/platforms/pseries/lpar.c
@@ -89,18 +89,21 @@
"%lx failed with %ld\n", cpu, hwcpu, addr, ret);
return;
}
+
+#ifdef CONFIG_PPC_STD_MMU_64
/*
* PAPR says this feature is SLB-Buffer but firmware never
* reports that. All SPLPAR support SLB shadow buffer.
*/
- addr = __pa(paca[cpu].slb_shadow_ptr);
- if (firmware_has_feature(FW_FEATURE_SPLPAR)) {
+ if (!radix_enabled() && firmware_has_feature(FW_FEATURE_SPLPAR)) {
+ addr = __pa(paca[cpu].slb_shadow_ptr);
ret = register_slb_shadow(hwcpu, addr);
if (ret)
pr_err("WARNING: SLB shadow buffer registration for "
"cpu %d (hw %d) of area %lx failed with %ld\n",
cpu, hwcpu, addr, ret);
}
+#endif /* CONFIG_PPC_STD_MMU_64 */
/*
* Register dispatch trace log, if one has been allocated.
@@ -123,6 +126,8 @@
}
}
+#ifdef CONFIG_PPC_STD_MMU_64
+
static long pSeries_lpar_hpte_insert(unsigned long hpte_group,
unsigned long vpn, unsigned long pa,
unsigned long rflags, unsigned long vflags,
@@ -139,7 +144,7 @@
hpte_group, vpn, pa, rflags, vflags, psize);
hpte_v = hpte_encode_v(vpn, psize, apsize, ssize) | vflags | HPTE_V_VALID;
- hpte_r = hpte_encode_r(pa, psize, apsize) | rflags;
+ hpte_r = hpte_encode_r(pa, psize, apsize, ssize) | rflags;
if (!(vflags & HPTE_V_BOLTED))
pr_devel(" hpte_v=%016lx, hpte_r=%016lx\n", hpte_v, hpte_r);
@@ -152,10 +157,6 @@
/* Exact = 0 */
flags = 0;
- /* Make pHyp happy */
- if ((rflags & _PAGE_NO_CACHE) && !(rflags & _PAGE_WRITETHRU))
- hpte_r &= ~HPTE_R_M;
-
if (firmware_has_feature(FW_FEATURE_XCMO) && !(hpte_r & HPTE_R_N))
flags |= H_COALESCE_CAND;
@@ -659,6 +660,8 @@
void arch_free_page(struct page *page, int order)
{
+ if (radix_enabled())
+ return;
if (!cmo_free_hint_flag || !firmware_has_feature(FW_FEATURE_CMO))
return;
@@ -666,7 +669,8 @@
}
EXPORT_SYMBOL(arch_free_page);
-#endif
+#endif /* CONFIG_PPC_SMLPAR */
+#endif /* CONFIG_PPC_STD_MMU_64 */
#ifdef CONFIG_TRACEPOINTS
#ifdef HAVE_JUMP_LABEL
diff --git a/arch/powerpc/platforms/pseries/lparcfg.c b/arch/powerpc/platforms/pseries/lparcfg.c
index c9fecf0..afa05a2 100644
--- a/arch/powerpc/platforms/pseries/lparcfg.c
+++ b/arch/powerpc/platforms/pseries/lparcfg.c
@@ -484,8 +484,9 @@
seq_printf(m, "shared_processor_mode=%d\n",
lppaca_shared_proc(get_lppaca()));
+#ifdef CONFIG_PPC_STD_MMU_64
seq_printf(m, "slb_size=%d\n", mmu_slb_size);
-
+#endif
parse_em_data(m);
return 0;
diff --git a/arch/powerpc/platforms/pseries/mobility.c b/arch/powerpc/platforms/pseries/mobility.c
index ceb18d3..a560a98 100644
--- a/arch/powerpc/platforms/pseries/mobility.c
+++ b/arch/powerpc/platforms/pseries/mobility.c
@@ -191,8 +191,8 @@
break;
case 0x80000000:
- prop = of_find_property(dn, prop_name, NULL);
- of_remove_property(dn, prop);
+ of_remove_property(dn, of_find_property(dn,
+ prop_name, NULL));
prop = NULL;
break;
diff --git a/arch/powerpc/platforms/pseries/msi.c b/arch/powerpc/platforms/pseries/msi.c
index 272e9ec..543a638 100644
--- a/arch/powerpc/platforms/pseries/msi.c
+++ b/arch/powerpc/platforms/pseries/msi.c
@@ -305,7 +305,7 @@
memset(&counts, 0, sizeof(struct msi_counts));
/* Work out how many devices we have below this PE */
- traverse_pci_devices(pe_dn, count_non_bridge_devices, &counts);
+ pci_traverse_device_nodes(pe_dn, count_non_bridge_devices, &counts);
if (counts.num_devices == 0) {
pr_err("rtas_msi: found 0 devices under PE for %s\n",
@@ -320,7 +320,7 @@
/* else, we have some more calculating to do */
counts.requestor = pci_device_to_OF_node(dev);
counts.request = request;
- traverse_pci_devices(pe_dn, count_spare_msis, &counts);
+ pci_traverse_device_nodes(pe_dn, count_spare_msis, &counts);
/* If the quota isn't an integer multiple of the total, we can
* use the remainder as spare MSIs for anyone that wants them. */
diff --git a/arch/powerpc/platforms/pseries/pci_dlpar.c b/arch/powerpc/platforms/pseries/pci_dlpar.c
index 5d4a3df..906dbaa 100644
--- a/arch/powerpc/platforms/pseries/pci_dlpar.c
+++ b/arch/powerpc/platforms/pseries/pci_dlpar.c
@@ -34,38 +34,6 @@
#include "pseries.h"
-static struct pci_bus *
-find_bus_among_children(struct pci_bus *bus,
- struct device_node *dn)
-{
- struct pci_bus *child = NULL;
- struct pci_bus *tmp;
- struct device_node *busdn;
-
- busdn = pci_bus_to_OF_node(bus);
- if (busdn == dn)
- return bus;
-
- list_for_each_entry(tmp, &bus->children, node) {
- child = find_bus_among_children(tmp, dn);
- if (child)
- break;
- };
- return child;
-}
-
-struct pci_bus *
-pcibios_find_pci_bus(struct device_node *dn)
-{
- struct pci_dn *pdn = dn->data;
-
- if (!pdn || !pdn->phb || !pdn->phb->bus)
- return NULL;
-
- return find_bus_among_children(pdn->phb->bus, dn);
-}
-EXPORT_SYMBOL_GPL(pcibios_find_pci_bus);
-
struct pci_controller *init_phb_dynamic(struct device_node *dn)
{
struct pci_controller *phb;
diff --git a/arch/powerpc/platforms/pseries/reconfig.c b/arch/powerpc/platforms/pseries/reconfig.c
index 7c7fcc0..cc66c49 100644
--- a/arch/powerpc/platforms/pseries/reconfig.c
+++ b/arch/powerpc/platforms/pseries/reconfig.c
@@ -303,7 +303,6 @@
{
struct device_node *np;
char *tmp;
- struct property *prop;
buf = parse_node(buf, bufsize, &np);
if (!np)
@@ -316,9 +315,7 @@
if (strlen(buf) == 0)
return -EINVAL;
- prop = of_find_property(np, buf, NULL);
-
- return of_remove_property(np, prop);
+ return of_remove_property(np, of_find_property(np, buf, NULL));
}
static int do_update_property(char *buf, size_t bufsize)
diff --git a/arch/powerpc/platforms/pseries/setup.c b/arch/powerpc/platforms/pseries/setup.c
index 6e944fc..9883bc7 100644
--- a/arch/powerpc/platforms/pseries/setup.c
+++ b/arch/powerpc/platforms/pseries/setup.c
@@ -235,6 +235,8 @@
for_each_node_by_name(np, "interrupt-controller") {
typep = of_get_property(np, "compatible", NULL);
+ if (!typep)
+ continue;
if (strstr(typep, "open-pic")) {
pSeries_mpic_node = of_node_get(np);
ppc_md.init_IRQ = pseries_mpic_init_IRQ;
@@ -265,7 +267,7 @@
pdn = parent ? PCI_DN(parent) : NULL;
if (pdn) {
/* Create pdn and EEH device */
- update_dn_pci_info(np, pdn->phb);
+ pci_add_device_node_info(pdn->phb, np);
eeh_dev_init(PCI_DN(np), pdn->phb);
}
diff --git a/arch/powerpc/sysdev/fsl_pci.c b/arch/powerpc/sysdev/fsl_pci.c
index 85729f4..0ef9df4 100644
--- a/arch/powerpc/sysdev/fsl_pci.c
+++ b/arch/powerpc/sysdev/fsl_pci.c
@@ -37,6 +37,7 @@
#include <asm/pci-bridge.h>
#include <asm/ppc-pci.h>
#include <asm/machdep.h>
+#include <asm/mpc85xx.h>
#include <asm/disassemble.h>
#include <asm/ppc-opcode.h>
#include <sysdev/fsl_soc.h>
@@ -527,6 +528,8 @@
u8 hdr_type, progif;
struct device_node *dev;
struct ccsr_pci __iomem *pci;
+ u16 temp;
+ u32 svr = mfspr(SPRN_SVR);
dev = pdev->dev.of_node;
@@ -596,6 +599,27 @@
PPC_INDIRECT_TYPE_SURPRESS_PRIMARY_BUS;
if (fsl_pcie_check_link(hose))
hose->indirect_type |= PPC_INDIRECT_TYPE_NO_PCIE_LINK;
+ } else {
+ /*
+ * Set PBFR(PCI Bus Function Register)[10] = 1 to
+ * disable the combining of crossing cacheline
+ * boundary requests into one burst transaction.
+ * PCI-X operation is not affected.
+ * Fix erratum PCI 5 on MPC8548
+ */
+#define PCI_BUS_FUNCTION 0x44
+#define PCI_BUS_FUNCTION_MDS 0x400 /* Master disable streaming */
+ if (((SVR_SOC_VER(svr) == SVR_8543) ||
+ (SVR_SOC_VER(svr) == SVR_8545) ||
+ (SVR_SOC_VER(svr) == SVR_8547) ||
+ (SVR_SOC_VER(svr) == SVR_8548)) &&
+ !early_find_capability(hose, 0, 0, PCI_CAP_ID_PCIX)) {
+ early_read_config_word(hose, 0, 0,
+ PCI_BUS_FUNCTION, &temp);
+ temp |= PCI_BUS_FUNCTION_MDS;
+ early_write_config_word(hose, 0, 0,
+ PCI_BUS_FUNCTION, temp);
+ }
}
printk(KERN_INFO "Found FSL PCI host bridge at 0x%016llx. "
diff --git a/arch/powerpc/sysdev/mpic.c b/arch/powerpc/sysdev/mpic.c
index afe3c7c..7de45b2 100644
--- a/arch/powerpc/sysdev/mpic.c
+++ b/arch/powerpc/sysdev/mpic.c
@@ -2004,8 +2004,15 @@
static int mpic_init_sys(void)
{
+ int rc;
+
register_syscore_ops(&mpic_syscore_ops);
- subsys_system_register(&mpic_subsys, NULL);
+ rc = subsys_system_register(&mpic_subsys, NULL);
+ if (rc) {
+ unregister_syscore_ops(&mpic_syscore_ops);
+ pr_err("mpic: Failed to register subsystem!\n");
+ return rc;
+ }
return 0;
}
diff --git a/arch/powerpc/xmon/Makefile b/arch/powerpc/xmon/Makefile
index 436062d..0b2f771 100644
--- a/arch/powerpc/xmon/Makefile
+++ b/arch/powerpc/xmon/Makefile
@@ -7,7 +7,7 @@
ccflags-$(CONFIG_PPC64) := $(NO_MINIMAL_TOC)
-obj-y += xmon.o nonstdio.o
+obj-y += xmon.o nonstdio.o spr_access.o
ifdef CONFIG_XMON_DISASSEMBLY
obj-y += ppc-dis.o ppc-opc.o
diff --git a/arch/powerpc/xmon/spr_access.S b/arch/powerpc/xmon/spr_access.S
new file mode 100644
index 0000000..84ad7421
--- /dev/null
+++ b/arch/powerpc/xmon/spr_access.S
@@ -0,0 +1,45 @@
+#include <asm/ppc_asm.h>
+
+/* unsigned long xmon_mfspr(sprn, default_value) */
+_GLOBAL(xmon_mfspr)
+ ld r5, .Lmfspr_table@got(r2)
+ b xmon_mxspr
+
+/* void xmon_mtspr(sprn, new_value) */
+_GLOBAL(xmon_mtspr)
+ ld r5, .Lmtspr_table@got(r2)
+ b xmon_mxspr
+
+/*
+ * r3 = sprn
+ * r4 = default or new value
+ * r5 = table base
+ */
+xmon_mxspr:
+ /*
+ * To index into the table of mxsprs we need:
+ * i = (sprn & 0x3ff) * 8
+ * or using rwlinm:
+ * i = (sprn << 3) & (0x3ff << 3)
+ */
+ rlwinm r3, r3, 3, 0x3ff << 3
+ add r5, r5, r3
+ mtctr r5
+ mr r3, r4 /* put default_value in r3 for mfspr */
+ bctr
+
+.Lmfspr_table:
+ spr = 0
+ .rept 1024
+ mfspr r3, spr
+ blr
+ spr = spr + 1
+ .endr
+
+.Lmtspr_table:
+ spr = 0
+ .rept 1024
+ mtspr spr, r4
+ blr
+ spr = spr + 1
+ .endr
diff --git a/arch/powerpc/xmon/xmon.c b/arch/powerpc/xmon/xmon.c
index 942796f..c5e1551 100644
--- a/arch/powerpc/xmon/xmon.c
+++ b/arch/powerpc/xmon/xmon.c
@@ -86,6 +86,7 @@
static long bus_error_jmp[JMP_BUF_LEN];
static int catch_memory_errors;
+static int catch_spr_faults;
static long *xmon_fault_jmp[NR_CPUS];
/* Breakpoint stuff */
@@ -147,7 +148,7 @@
static void flush_input(void);
static int inchar(void);
static void take_input(char *);
-static unsigned long read_spr(int);
+static int read_spr(int, unsigned long *);
static void write_spr(int, unsigned long);
static void super_regs(void);
static void remove_bpts(void);
@@ -250,6 +251,9 @@
sdi # disassemble spu local store for spu # (in hex)\n"
#endif
" S print special registers\n\
+ Sa print all SPRs\n\
+ Sr # read SPR #\n\
+ Sw #v write v to SPR #\n\
t print backtrace\n\
x exit monitor and recover\n\
X exit monitor and don't recover\n"
@@ -442,6 +446,12 @@
#ifdef CONFIG_SMP
cpu = smp_processor_id();
if (cpumask_test_cpu(cpu, &cpus_in_xmon)) {
+ /*
+ * We catch SPR read/write faults here because the 0x700, 0xf60
+ * etc. handlers don't call debugger_fault_handler().
+ */
+ if (catch_spr_faults)
+ longjmp(bus_error_jmp, 1);
get_output_lock();
excprint(regs);
printf("cpu 0x%x: Exception %lx %s in xmon, "
@@ -1635,89 +1645,87 @@
catch_memory_errors = 0;
}
-static unsigned long
-read_spr(int n)
+extern unsigned long xmon_mfspr(int spr, unsigned long default_value);
+extern void xmon_mtspr(int spr, unsigned long value);
+
+static int
+read_spr(int n, unsigned long *vp)
{
- unsigned int instrs[2];
- unsigned long (*code)(void);
unsigned long ret = -1UL;
-#ifdef CONFIG_PPC64
- unsigned long opd[3];
-
- opd[0] = (unsigned long)instrs;
- opd[1] = 0;
- opd[2] = 0;
- code = (unsigned long (*)(void)) opd;
-#else
- code = (unsigned long (*)(void)) instrs;
-#endif
-
- /* mfspr r3,n; blr */
- instrs[0] = 0x7c6002a6 + ((n & 0x1F) << 16) + ((n & 0x3e0) << 6);
- instrs[1] = 0x4e800020;
- store_inst(instrs);
- store_inst(instrs+1);
+ int ok = 0;
if (setjmp(bus_error_jmp) == 0) {
- catch_memory_errors = 1;
+ catch_spr_faults = 1;
sync();
- ret = code();
+ ret = xmon_mfspr(n, *vp);
sync();
- /* wait a little while to see if we get a machine check */
- __delay(200);
- n = size;
+ *vp = ret;
+ ok = 1;
}
+ catch_spr_faults = 0;
- return ret;
+ return ok;
}
static void
write_spr(int n, unsigned long val)
{
- unsigned int instrs[2];
- unsigned long (*code)(unsigned long);
-#ifdef CONFIG_PPC64
- unsigned long opd[3];
-
- opd[0] = (unsigned long)instrs;
- opd[1] = 0;
- opd[2] = 0;
- code = (unsigned long (*)(unsigned long)) opd;
-#else
- code = (unsigned long (*)(unsigned long)) instrs;
-#endif
-
- instrs[0] = 0x7c6003a6 + ((n & 0x1F) << 16) + ((n & 0x3e0) << 6);
- instrs[1] = 0x4e800020;
- store_inst(instrs);
- store_inst(instrs+1);
-
if (setjmp(bus_error_jmp) == 0) {
- catch_memory_errors = 1;
+ catch_spr_faults = 1;
sync();
- code(val);
+ xmon_mtspr(n, val);
sync();
- /* wait a little while to see if we get a machine check */
- __delay(200);
- n = size;
+ } else {
+ printf("SPR 0x%03x (%4d) Faulted during write\n", n, n);
}
+ catch_spr_faults = 0;
}
static unsigned long regno;
extern char exc_prolog;
extern char dec_exc;
+static void dump_one_spr(int spr, bool show_unimplemented)
+{
+ unsigned long val;
+
+ val = 0xdeadbeef;
+ if (!read_spr(spr, &val)) {
+ printf("SPR 0x%03x (%4d) Faulted during read\n", spr, spr);
+ return;
+ }
+
+ if (val == 0xdeadbeef) {
+ /* Looks like read was a nop, confirm */
+ val = 0x0badcafe;
+ if (!read_spr(spr, &val)) {
+ printf("SPR 0x%03x (%4d) Faulted during read\n", spr, spr);
+ return;
+ }
+
+ if (val == 0x0badcafe) {
+ if (show_unimplemented)
+ printf("SPR 0x%03x (%4d) Unimplemented\n", spr, spr);
+ return;
+ }
+ }
+
+ printf("SPR 0x%03x (%4d) = 0x%lx\n", spr, spr, val);
+}
+
static void super_regs(void)
{
int cmd;
- unsigned long val;
+ int spr;
cmd = skipbl();
- if (cmd == '\n') {
+
+ switch (cmd) {
+ case '\n': {
unsigned long sp, toc;
asm("mr %0,1" : "=r" (sp) :);
asm("mr %0,2" : "=r" (toc) :);
@@ -1730,21 +1738,29 @@
mfspr(SPRN_DEC), mfspr(SPRN_SPRG2));
printf("sp = "REG" sprg3= "REG"\n", sp, mfspr(SPRN_SPRG3));
printf("toc = "REG" dar = "REG"\n", toc, mfspr(SPRN_DAR));
-
return;
}
-
- scanhex(®no);
- switch (cmd) {
- case 'w':
- val = read_spr(regno);
+ case 'w': {
+ unsigned long val;
+ scanhex(®no);
+ val = 0;
+ read_spr(regno, &val);
scanhex(&val);
write_spr(regno, val);
- /* fall through */
- case 'r':
- printf("spr %lx = %lx\n", regno, read_spr(regno));
+ dump_one_spr(regno, true);
break;
}
+ case 'r':
+ scanhex(®no);
+ dump_one_spr(regno, true);
+ break;
+ case 'a':
+ /* dump ALL SPRs */
+ for (spr = 1; spr < 1024; ++spr)
+ dump_one_spr(spr, false);
+ break;
+ }
+
scannl();
}
@@ -2913,7 +2929,7 @@
printf("%s", after);
}
-#ifdef CONFIG_PPC_BOOK3S_64
+#ifdef CONFIG_PPC_STD_MMU_64
void dump_segments(void)
{
int i;
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index de0fcc0..a8c2590 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -123,6 +123,7 @@
select HAVE_ARCH_AUDITSYSCALL
select HAVE_ARCH_EARLY_PFN_TO_NID
select HAVE_ARCH_JUMP_LABEL
+ select CPU_NO_EFFICIENT_FFS if !HAVE_MARCH_Z9_109_FEATURES
select HAVE_ARCH_SECCOMP_FILTER
select HAVE_ARCH_SOFT_DIRTY
select HAVE_ARCH_TRACEHOOK
@@ -134,6 +135,7 @@
select HAVE_DMA_API_DEBUG
select HAVE_DYNAMIC_FTRACE
select HAVE_DYNAMIC_FTRACE_WITH_REGS
+ select HAVE_EXIT_THREAD
select HAVE_FTRACE_MCOUNT_RECORD
select HAVE_FUNCTION_GRAPH_TRACER
select HAVE_FUNCTION_TRACER
@@ -165,6 +167,7 @@
select TTY
select VIRT_CPU_ACCOUNTING
select VIRT_TO_BUS
+ select HAVE_NMI
config SCHED_OMIT_FRAME_POINTER
diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h
index 6da41fa..37b9017 100644
--- a/arch/s390/include/asm/kvm_host.h
+++ b/arch/s390/include/asm/kvm_host.h
@@ -38,7 +38,7 @@
*/
#define KVM_NR_IRQCHIPS 1
#define KVM_IRQCHIP_NUM_PINS 4096
-#define KVM_HALT_POLL_NS_DEFAULT 0
+#define KVM_HALT_POLL_NS_DEFAULT 80000
/* s390-specific vcpu->requests bit members */
#define KVM_REQ_ENABLE_IBS 8
@@ -247,6 +247,7 @@
u32 exit_instruction;
u32 halt_successful_poll;
u32 halt_attempted_poll;
+ u32 halt_poll_invalid;
u32 halt_wakeup;
u32 instruction_lctl;
u32 instruction_lctlg;
@@ -544,10 +545,6 @@
struct kvm_s390_local_interrupt local_int;
struct hrtimer ckc_timer;
struct kvm_s390_pgm_info pgm;
- union {
- struct cpuid cpu_id;
- u64 stidp_data;
- };
struct gmap *gmap;
struct kvm_guestdbg_info_arch guestdbg;
unsigned long pfault_token;
@@ -605,7 +602,7 @@
__u64 fac_mask[S390_ARCH_FAC_LIST_SIZE_U64];
/* facility list requested by guest (in dma page) */
__u64 *fac_list;
- struct cpuid cpu_id;
+ u64 cpuid;
unsigned short ibc;
};
@@ -700,4 +697,6 @@
static inline void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu) {}
static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {}
+void kvm_arch_vcpu_block_finish(struct kvm_vcpu *vcpu);
+
#endif
diff --git a/arch/s390/include/asm/pgtable.h b/arch/s390/include/asm/pgtable.h
index 2f66645..18d2beb 100644
--- a/arch/s390/include/asm/pgtable.h
+++ b/arch/s390/include/asm/pgtable.h
@@ -1223,6 +1223,7 @@
return pmd_val(pmd) & _SEGMENT_ENTRY_LARGE;
}
+#define has_transparent_hugepage has_transparent_hugepage
static inline int has_transparent_hugepage(void)
{
return MACHINE_HAS_HPAGE ? 1 : 0;
diff --git a/arch/s390/include/asm/sclp.h b/arch/s390/include/asm/sclp.h
index bd7893d..e4f6f73 100644
--- a/arch/s390/include/asm/sclp.h
+++ b/arch/s390/include/asm/sclp.h
@@ -69,6 +69,7 @@
unsigned int max_cores;
unsigned long hsa_size;
unsigned long facilities;
+ unsigned int hmfai;
};
extern struct sclp_info sclp;
diff --git a/arch/s390/include/asm/sigp.h b/arch/s390/include/asm/sigp.h
index ec60cf7..1c8f33f 100644
--- a/arch/s390/include/asm/sigp.h
+++ b/arch/s390/include/asm/sigp.h
@@ -27,6 +27,7 @@
/* SIGP cpu status bits */
+#define SIGP_STATUS_INVALID_ORDER 0x00000002UL
#define SIGP_STATUS_CHECK_STOP 0x00000010UL
#define SIGP_STATUS_STOPPED 0x00000040UL
#define SIGP_STATUS_EXT_CALL_PENDING 0x00000080UL
diff --git a/arch/s390/include/uapi/asm/kvm.h b/arch/s390/include/uapi/asm/kvm.h
index 347fe5a..3b8e99e 100644
--- a/arch/s390/include/uapi/asm/kvm.h
+++ b/arch/s390/include/uapi/asm/kvm.h
@@ -25,6 +25,7 @@
#define KVM_DEV_FLIC_APF_DISABLE_WAIT 5
#define KVM_DEV_FLIC_ADAPTER_REGISTER 6
#define KVM_DEV_FLIC_ADAPTER_MODIFY 7
+#define KVM_DEV_FLIC_CLEAR_IO_IRQ 8
/*
* We can have up to 4*64k pending subchannels + 8 adapter interrupts,
* as well as up to ASYNC_PF_PER_VCPU*KVM_MAX_VCPUS pfault done interrupts.
diff --git a/arch/s390/include/uapi/asm/sie.h b/arch/s390/include/uapi/asm/sie.h
index 5dbaa72..8fb5d4a 100644
--- a/arch/s390/include/uapi/asm/sie.h
+++ b/arch/s390/include/uapi/asm/sie.h
@@ -16,14 +16,19 @@
{ 0x01, "SIGP sense" }, \
{ 0x02, "SIGP external call" }, \
{ 0x03, "SIGP emergency signal" }, \
+ { 0x04, "SIGP start" }, \
{ 0x05, "SIGP stop" }, \
{ 0x06, "SIGP restart" }, \
{ 0x09, "SIGP stop and store status" }, \
{ 0x0b, "SIGP initial cpu reset" }, \
+ { 0x0c, "SIGP cpu reset" }, \
{ 0x0d, "SIGP set prefix" }, \
{ 0x0e, "SIGP store status at address" }, \
{ 0x12, "SIGP set architecture" }, \
- { 0x15, "SIGP sense running" }
+ { 0x13, "SIGP conditional emergency signal" }, \
+ { 0x15, "SIGP sense running" }, \
+ { 0x16, "SIGP set multithreading"}, \
+ { 0x17, "SIGP store additional status ait address"}
#define icpt_prog_codes \
{ 0x0001, "Prog Operation" }, \
diff --git a/arch/s390/kernel/machine_kexec.c b/arch/s390/kernel/machine_kexec.c
index 2f1b721..0e64f08 100644
--- a/arch/s390/kernel/machine_kexec.c
+++ b/arch/s390/kernel/machine_kexec.c
@@ -43,13 +43,13 @@
switch (action) {
case PM_SUSPEND_PREPARE:
case PM_HIBERNATION_PREPARE:
- if (crashk_res.start)
- crash_map_reserved_pages();
+ if (kexec_crash_image)
+ arch_kexec_unprotect_crashkres();
break;
case PM_POST_SUSPEND:
case PM_POST_HIBERNATION:
- if (crashk_res.start)
- crash_unmap_reserved_pages();
+ if (kexec_crash_image)
+ arch_kexec_protect_crashkres();
break;
default:
return NOTIFY_DONE;
@@ -60,6 +60,8 @@
static int __init machine_kdump_pm_init(void)
{
pm_notifier(machine_kdump_pm_cb, 0);
+ /* Create initial mapping for crashkernel memory */
+ arch_kexec_unprotect_crashkres();
return 0;
}
arch_initcall(machine_kdump_pm_init);
@@ -146,6 +148,8 @@
#endif
}
+#ifdef CONFIG_CRASH_DUMP
+
/*
* Map or unmap crashkernel memory
*/
@@ -167,21 +171,25 @@
}
/*
- * Map crashkernel memory
+ * Unmap crashkernel memory
*/
-void crash_map_reserved_pages(void)
+void arch_kexec_protect_crashkres(void)
{
- crash_map_pages(1);
+ if (crashk_res.end)
+ crash_map_pages(0);
}
/*
- * Unmap crashkernel memory
+ * Map crashkernel memory
*/
-void crash_unmap_reserved_pages(void)
+void arch_kexec_unprotect_crashkres(void)
{
- crash_map_pages(0);
+ if (crashk_res.end)
+ crash_map_pages(1);
}
+#endif
+
/*
* Give back memory to hypervisor before new kdump is loaded
*/
diff --git a/arch/s390/kernel/process.c b/arch/s390/kernel/process.c
index 481d7a83..bba4fa7 100644
--- a/arch/s390/kernel/process.c
+++ b/arch/s390/kernel/process.c
@@ -68,9 +68,10 @@
/*
* Free current thread data structures etc..
*/
-void exit_thread(void)
+void exit_thread(struct task_struct *tsk)
{
- exit_thread_runtime_instr();
+ if (tsk == current)
+ exit_thread_runtime_instr();
}
void flush_thread(void)
diff --git a/arch/s390/kernel/vdso.c b/arch/s390/kernel/vdso.c
index 94495ca..5904abf 100644
--- a/arch/s390/kernel/vdso.c
+++ b/arch/s390/kernel/vdso.c
@@ -216,7 +216,8 @@
* it at vdso_base which is the "natural" base for it, but we might
* fail and end up putting it elsewhere.
*/
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
vdso_base = get_unmapped_area(NULL, 0, vdso_pages << PAGE_SHIFT, 0, 0);
if (IS_ERR_VALUE(vdso_base)) {
rc = vdso_base;
diff --git a/arch/s390/kvm/Kconfig b/arch/s390/kvm/Kconfig
index 5ea5af3..b190023 100644
--- a/arch/s390/kvm/Kconfig
+++ b/arch/s390/kvm/Kconfig
@@ -28,6 +28,7 @@
select HAVE_KVM_IRQCHIP
select HAVE_KVM_IRQFD
select HAVE_KVM_IRQ_ROUTING
+ select HAVE_KVM_INVALID_WAKEUPS
select SRCU
select KVM_VFIO
---help---
diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c
index 84efc2b..5a80af7 100644
--- a/arch/s390/kvm/interrupt.c
+++ b/arch/s390/kvm/interrupt.c
@@ -977,6 +977,11 @@
void kvm_s390_vcpu_wakeup(struct kvm_vcpu *vcpu)
{
+ /*
+ * We cannot move this into the if, as the CPU might be already
+ * in kvm_vcpu_block without having the waitqueue set (polling)
+ */
+ vcpu->valid_wakeup = true;
if (swait_active(&vcpu->wq)) {
/*
* The vcpu gave up the cpu voluntarily, mark it as a good
@@ -2034,6 +2039,27 @@
return ret;
}
+static int clear_io_irq(struct kvm *kvm, struct kvm_device_attr *attr)
+
+{
+ const u64 isc_mask = 0xffUL << 24; /* all iscs set */
+ u32 schid;
+
+ if (attr->flags)
+ return -EINVAL;
+ if (attr->attr != sizeof(schid))
+ return -EINVAL;
+ if (copy_from_user(&schid, (void __user *) attr->addr, sizeof(schid)))
+ return -EFAULT;
+ kfree(kvm_s390_get_io_int(kvm, isc_mask, schid));
+ /*
+ * If userspace is conforming to the architecture, we can have at most
+ * one pending I/O interrupt per subchannel, so this is effectively a
+ * clear all.
+ */
+ return 0;
+}
+
static int flic_set_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
{
int r = 0;
@@ -2067,6 +2093,9 @@
case KVM_DEV_FLIC_ADAPTER_MODIFY:
r = modify_io_adapter(dev, attr);
break;
+ case KVM_DEV_FLIC_CLEAR_IO_IRQ:
+ r = clear_io_irq(dev->kvm, attr);
+ break;
default:
r = -EINVAL;
}
@@ -2074,6 +2103,23 @@
return r;
}
+static int flic_has_attr(struct kvm_device *dev,
+ struct kvm_device_attr *attr)
+{
+ switch (attr->group) {
+ case KVM_DEV_FLIC_GET_ALL_IRQS:
+ case KVM_DEV_FLIC_ENQUEUE:
+ case KVM_DEV_FLIC_CLEAR_IRQS:
+ case KVM_DEV_FLIC_APF_ENABLE:
+ case KVM_DEV_FLIC_APF_DISABLE_WAIT:
+ case KVM_DEV_FLIC_ADAPTER_REGISTER:
+ case KVM_DEV_FLIC_ADAPTER_MODIFY:
+ case KVM_DEV_FLIC_CLEAR_IO_IRQ:
+ return 0;
+ }
+ return -ENXIO;
+}
+
static int flic_create(struct kvm_device *dev, u32 type)
{
if (!dev)
@@ -2095,6 +2141,7 @@
.name = "kvm-flic",
.get_attr = flic_get_attr,
.set_attr = flic_set_attr,
+ .has_attr = flic_has_attr,
.create = flic_create,
.destroy = flic_destroy,
};
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index 668c087..6d8ec3a 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -65,6 +65,7 @@
{ "exit_instr_and_program_int", VCPU_STAT(exit_instr_and_program) },
{ "halt_successful_poll", VCPU_STAT(halt_successful_poll) },
{ "halt_attempted_poll", VCPU_STAT(halt_attempted_poll) },
+ { "halt_poll_invalid", VCPU_STAT(halt_poll_invalid) },
{ "halt_wakeup", VCPU_STAT(halt_wakeup) },
{ "instruction_lctlg", VCPU_STAT(instruction_lctlg) },
{ "instruction_lctl", VCPU_STAT(instruction_lctl) },
@@ -118,9 +119,9 @@
};
/* upper facilities limit for kvm */
-unsigned long kvm_s390_fac_list_mask[] = {
- 0xffe6fffbfcfdfc40UL,
- 0x005e800000000000UL,
+unsigned long kvm_s390_fac_list_mask[16] = {
+ 0xffe6000000000000UL,
+ 0x005e000000000000UL,
};
unsigned long kvm_s390_fac_list_mask_size(void)
@@ -638,6 +639,7 @@
static int kvm_s390_set_processor(struct kvm *kvm, struct kvm_device_attr *attr)
{
struct kvm_s390_vm_cpu_processor *proc;
+ u16 lowest_ibc, unblocked_ibc;
int ret = 0;
mutex_lock(&kvm->lock);
@@ -652,9 +654,17 @@
}
if (!copy_from_user(proc, (void __user *)attr->addr,
sizeof(*proc))) {
- memcpy(&kvm->arch.model.cpu_id, &proc->cpuid,
- sizeof(struct cpuid));
- kvm->arch.model.ibc = proc->ibc;
+ kvm->arch.model.cpuid = proc->cpuid;
+ lowest_ibc = sclp.ibc >> 16 & 0xfff;
+ unblocked_ibc = sclp.ibc & 0xfff;
+ if (lowest_ibc) {
+ if (proc->ibc > unblocked_ibc)
+ kvm->arch.model.ibc = unblocked_ibc;
+ else if (proc->ibc < lowest_ibc)
+ kvm->arch.model.ibc = lowest_ibc;
+ else
+ kvm->arch.model.ibc = proc->ibc;
+ }
memcpy(kvm->arch.model.fac_list, proc->fac_list,
S390_ARCH_FAC_LIST_SIZE_BYTE);
} else
@@ -687,7 +697,7 @@
ret = -ENOMEM;
goto out;
}
- memcpy(&proc->cpuid, &kvm->arch.model.cpu_id, sizeof(struct cpuid));
+ proc->cpuid = kvm->arch.model.cpuid;
proc->ibc = kvm->arch.model.ibc;
memcpy(&proc->fac_list, kvm->arch.model.fac_list,
S390_ARCH_FAC_LIST_SIZE_BYTE);
@@ -1081,10 +1091,13 @@
kvm->arch.crypto.crycbd |= CRYCB_FORMAT1;
}
-static void kvm_s390_get_cpu_id(struct cpuid *cpu_id)
+static u64 kvm_s390_get_initial_cpuid(void)
{
- get_cpu_id(cpu_id);
- cpu_id->version = 0xff;
+ struct cpuid cpuid;
+
+ get_cpu_id(&cpuid);
+ cpuid.version = 0xff;
+ return *((u64 *) &cpuid);
}
static void kvm_s390_crypto_init(struct kvm *kvm)
@@ -1175,7 +1188,7 @@
memcpy(kvm->arch.model.fac_list, kvm->arch.model.fac_mask,
S390_ARCH_FAC_LIST_SIZE_BYTE);
- kvm_s390_get_cpu_id(&kvm->arch.model.cpu_id);
+ kvm->arch.model.cpuid = kvm_s390_get_initial_cpuid();
kvm->arch.model.ibc = sclp.ibc & 0x0fff;
kvm_s390_crypto_init(kvm);
@@ -1624,7 +1637,6 @@
{
struct kvm_s390_cpu_model *model = &vcpu->kvm->arch.model;
- vcpu->arch.cpu_id = model->cpu_id;
vcpu->arch.sie_block->ibc = model->ibc;
if (test_kvm_facility(vcpu->kvm, 7))
vcpu->arch.sie_block->fac = (u32)(u64) model->fac_list;
@@ -1645,11 +1657,14 @@
kvm_s390_vcpu_setup_model(vcpu);
- vcpu->arch.sie_block->ecb = 6;
+ vcpu->arch.sie_block->ecb = 0x02;
+ if (test_kvm_facility(vcpu->kvm, 9))
+ vcpu->arch.sie_block->ecb |= 0x04;
if (test_kvm_facility(vcpu->kvm, 50) && test_kvm_facility(vcpu->kvm, 73))
vcpu->arch.sie_block->ecb |= 0x10;
- vcpu->arch.sie_block->ecb2 = 8;
+ if (test_kvm_facility(vcpu->kvm, 8))
+ vcpu->arch.sie_block->ecb2 |= 0x08;
vcpu->arch.sie_block->eca = 0xC1002000U;
if (sclp.has_siif)
vcpu->arch.sie_block->eca |= 1;
@@ -2971,13 +2986,31 @@
return;
}
+static inline unsigned long nonhyp_mask(int i)
+{
+ unsigned int nonhyp_fai = (sclp.hmfai << i * 2) >> 30;
+
+ return 0x0000ffffffffffffUL >> (nonhyp_fai << 4);
+}
+
+void kvm_arch_vcpu_block_finish(struct kvm_vcpu *vcpu)
+{
+ vcpu->valid_wakeup = false;
+}
+
static int __init kvm_s390_init(void)
{
+ int i;
+
if (!sclp.has_sief2) {
pr_info("SIE not available\n");
return -ENODEV;
}
+ for (i = 0; i < 16; i++)
+ kvm_s390_fac_list_mask[i] |=
+ S390_lowcore.stfle_fac_list[i] & nonhyp_mask(i);
+
return kvm_init(NULL, sizeof(struct kvm_vcpu), 0, THIS_MODULE);
}
diff --git a/arch/s390/kvm/priv.c b/arch/s390/kvm/priv.c
index 0a1591d..95916fa 100644
--- a/arch/s390/kvm/priv.c
+++ b/arch/s390/kvm/priv.c
@@ -439,7 +439,7 @@
static int handle_stidp(struct kvm_vcpu *vcpu)
{
- u64 stidp_data = vcpu->arch.stidp_data;
+ u64 stidp_data = vcpu->kvm->arch.model.cpuid;
u64 operand2;
int rc;
ar_t ar;
@@ -670,8 +670,9 @@
if (vcpu->run->s.regs.gprs[reg1] & PFMF_RESERVED)
return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
- /* Only provide non-quiescing support if the host supports it */
- if (vcpu->run->s.regs.gprs[reg1] & PFMF_NQ && !test_facility(14))
+ /* Only provide non-quiescing support if enabled for the guest */
+ if (vcpu->run->s.regs.gprs[reg1] & PFMF_NQ &&
+ !test_kvm_facility(vcpu->kvm, 14))
return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
/* No support for conditional-SSKE */
@@ -744,7 +745,7 @@
{
/* entries expected to be 1FF */
int entries = (vcpu->arch.sie_block->cbrlo & ~PAGE_MASK) >> 3;
- unsigned long *cbrlo, cbrle;
+ unsigned long *cbrlo;
struct gmap *gmap;
int i;
@@ -765,17 +766,9 @@
vcpu->arch.sie_block->cbrlo &= PAGE_MASK; /* reset nceo */
cbrlo = phys_to_virt(vcpu->arch.sie_block->cbrlo);
down_read(&gmap->mm->mmap_sem);
- for (i = 0; i < entries; ++i) {
- cbrle = cbrlo[i];
- if (unlikely(cbrle & ~PAGE_MASK || cbrle < 2 * PAGE_SIZE))
- /* invalid entry */
- break;
- /* try to free backing */
- __gmap_zap(gmap, cbrle);
- }
+ for (i = 0; i < entries; ++i)
+ __gmap_zap(gmap, cbrlo[i]);
up_read(&gmap->mm->mmap_sem);
- if (i < entries)
- return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
return 0;
}
diff --git a/arch/s390/kvm/sigp.c b/arch/s390/kvm/sigp.c
index 77c22d6..28ea0ca 100644
--- a/arch/s390/kvm/sigp.c
+++ b/arch/s390/kvm/sigp.c
@@ -240,6 +240,12 @@
struct kvm_s390_local_interrupt *li;
int rc;
+ if (!test_kvm_facility(vcpu->kvm, 9)) {
+ *reg &= 0xffffffff00000000UL;
+ *reg |= SIGP_STATUS_INVALID_ORDER;
+ return SIGP_CC_STATUS_STORED;
+ }
+
li = &dst_vcpu->arch.local_int;
if (atomic_read(li->cpuflags) & CPUSTAT_RUNNING) {
/* running */
diff --git a/arch/score/Kconfig b/arch/score/Kconfig
index 366e1b5..507d631 100644
--- a/arch/score/Kconfig
+++ b/arch/score/Kconfig
@@ -14,6 +14,7 @@
select VIRT_TO_BUS
select MODULES_USE_ELF_REL
select CLONE_BACKWARDS
+ select CPU_NO_EFFICIENT_FFS
choice
prompt "System type"
diff --git a/arch/score/include/uapi/asm/unistd.h b/arch/score/include/uapi/asm/unistd.h
index 9cb4260..d4008c3 100644
--- a/arch/score/include/uapi/asm/unistd.h
+++ b/arch/score/include/uapi/asm/unistd.h
@@ -1,5 +1,6 @@
#define __ARCH_HAVE_MMU
+#define __ARCH_WANT_RENAMEAT
#define __ARCH_WANT_SYSCALL_NO_AT
#define __ARCH_WANT_SYSCALL_NO_FLAGS
#define __ARCH_WANT_SYSCALL_OFF_T
diff --git a/arch/score/kernel/process.c b/arch/score/kernel/process.c
index a1519ad3..aae9480 100644
--- a/arch/score/kernel/process.c
+++ b/arch/score/kernel/process.c
@@ -56,8 +56,6 @@
regs->regs[0] = sp;
}
-void exit_thread(void) {}
-
/*
* When a process does an "exec", machine state like FPU and debug
* registers need to be reset. This is a hook function for that.
diff --git a/arch/sh/Kconfig b/arch/sh/Kconfig
index 7ed20fc..e803a83 100644
--- a/arch/sh/Kconfig
+++ b/arch/sh/Kconfig
@@ -20,6 +20,7 @@
select PERF_USE_VMALLOC
select HAVE_DEBUG_KMEMLEAK
select HAVE_KERNEL_GZIP
+ select CPU_NO_EFFICIENT_FFS
select HAVE_KERNEL_BZIP2
select HAVE_KERNEL_LZMA
select HAVE_KERNEL_XZ
@@ -44,6 +45,7 @@
select OLD_SIGSUSPEND
select OLD_SIGACTION
select HAVE_ARCH_AUDITSYSCALL
+ select HAVE_NMI
help
The SuperH is a RISC processor targeted for use in embedded systems
and consumer electronics; it was also used in the Sega Dreamcast
@@ -71,6 +73,7 @@
config SUPERH64
def_bool ARCH = "sh64"
+ select HAVE_EXIT_THREAD
select KALLSYMS
config ARCH_DEFCONFIG
diff --git a/arch/sh/configs/apsh4ad0a_defconfig b/arch/sh/configs/apsh4ad0a_defconfig
index a8d9757..fe45d2c 100644
--- a/arch/sh/configs/apsh4ad0a_defconfig
+++ b/arch/sh/configs/apsh4ad0a_defconfig
@@ -10,7 +10,6 @@
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CGROUP_CPUACCT=y
-CONFIG_RESOURCE_COUNTERS=y
CONFIG_CGROUP_MEMCG=y
CONFIG_BLK_CGROUP=y
CONFIG_NAMESPACES=y
diff --git a/arch/sh/configs/sdk7786_defconfig b/arch/sh/configs/sdk7786_defconfig
index e7e56a4..36642ec2 100644
--- a/arch/sh/configs/sdk7786_defconfig
+++ b/arch/sh/configs/sdk7786_defconfig
@@ -17,7 +17,6 @@
CONFIG_CPUSETS=y
# CONFIG_PROC_PID_CPUSET is not set
CONFIG_CGROUP_CPUACCT=y
-CONFIG_RESOURCE_COUNTERS=y
CONFIG_CGROUP_MEMCG=y
CONFIG_CGROUP_MEMCG_SWAP=y
CONFIG_CGROUP_SCHED=y
diff --git a/arch/sh/configs/se7206_defconfig b/arch/sh/configs/se7206_defconfig
index 6bc30ab..91853a6 100644
--- a/arch/sh/configs/se7206_defconfig
+++ b/arch/sh/configs/se7206_defconfig
@@ -10,7 +10,6 @@
CONFIG_CGROUP_DEBUG=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CGROUP_CPUACCT=y
-CONFIG_RESOURCE_COUNTERS=y
CONFIG_CGROUP_MEMCG=y
CONFIG_RELAY=y
CONFIG_NAMESPACES=y
diff --git a/arch/sh/configs/shx3_defconfig b/arch/sh/configs/shx3_defconfig
index cd6c519..4a4269a 100644
--- a/arch/sh/configs/shx3_defconfig
+++ b/arch/sh/configs/shx3_defconfig
@@ -12,7 +12,6 @@
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CGROUP_CPUACCT=y
-CONFIG_RESOURCE_COUNTERS=y
CONFIG_CGROUP_MEMCG=y
CONFIG_RELAY=y
CONFIG_NAMESPACES=y
diff --git a/arch/sh/configs/urquell_defconfig b/arch/sh/configs/urquell_defconfig
index 1e843db..01c9a91 100644
--- a/arch/sh/configs/urquell_defconfig
+++ b/arch/sh/configs/urquell_defconfig
@@ -14,7 +14,6 @@
CONFIG_CPUSETS=y
# CONFIG_PROC_PID_CPUSET is not set
CONFIG_CGROUP_CPUACCT=y
-CONFIG_RESOURCE_COUNTERS=y
CONFIG_CGROUP_MEMCG=y
CONFIG_CGROUP_MEMCG_SWAP=y
CONFIG_CGROUP_SCHED=y
diff --git a/arch/sh/kernel/process_32.c b/arch/sh/kernel/process_32.c
index 2885fc9..ee12e94 100644
--- a/arch/sh/kernel/process_32.c
+++ b/arch/sh/kernel/process_32.c
@@ -76,13 +76,6 @@
}
EXPORT_SYMBOL(start_thread);
-/*
- * Free current thread data structures etc..
- */
-void exit_thread(void)
-{
-}
-
void flush_thread(void)
{
struct task_struct *tsk = current;
diff --git a/arch/sh/kernel/process_64.c b/arch/sh/kernel/process_64.c
index e2062e6..9d3e991 100644
--- a/arch/sh/kernel/process_64.c
+++ b/arch/sh/kernel/process_64.c
@@ -288,7 +288,7 @@
/*
* Free current thread data structures etc..
*/
-void exit_thread(void)
+void exit_thread(struct task_struct *tsk)
{
/*
* See arch/sparc/kernel/process.c for the precedent for doing
@@ -307,9 +307,8 @@
* which it would get safely nulled.
*/
#ifdef CONFIG_SH_FPU
- if (last_task_used_math == current) {
+ if (last_task_used_math == tsk)
last_task_used_math = NULL;
- }
#endif
}
diff --git a/arch/sh/kernel/vsyscall/vsyscall.c b/arch/sh/kernel/vsyscall/vsyscall.c
index ea2aa13..cc0cc5b 100644
--- a/arch/sh/kernel/vsyscall/vsyscall.c
+++ b/arch/sh/kernel/vsyscall/vsyscall.c
@@ -64,7 +64,9 @@
unsigned long addr;
int ret;
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
+
addr = get_unmapped_area(NULL, 0, PAGE_SIZE, 0, 0);
if (IS_ERR_VALUE(addr)) {
ret = addr;
diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
index db0a26c..546293d 100644
--- a/arch/sparc/Kconfig
+++ b/arch/sparc/Kconfig
@@ -20,6 +20,7 @@
select HAVE_OPROFILE
select HAVE_ARCH_KGDB if !SMP || SPARC64
select HAVE_ARCH_TRACEHOOK
+ select HAVE_EXIT_THREAD
select SYSCTL_EXCEPTION_TRACE
select ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE
select RTC_CLASS
@@ -41,6 +42,7 @@
select ODD_RT_SIGACTION
select OLD_SIGSUSPEND
select ARCH_HAS_SG_CHAIN
+ select CPU_NO_EFFICIENT_FFS
config SPARC32
def_bool !64BIT
@@ -78,6 +80,7 @@
select NO_BOOTMEM
select HAVE_ARCH_AUDITSYSCALL
select ARCH_SUPPORTS_ATOMIC_RMW
+ select HAVE_NMI
config ARCH_DEFCONFIG
string
diff --git a/arch/sparc/include/asm/head_32.h b/arch/sparc/include/asm/head_32.h
index 5f1dbe3..6fc60fd 100644
--- a/arch/sparc/include/asm/head_32.h
+++ b/arch/sparc/include/asm/head_32.h
@@ -43,10 +43,10 @@
nop;
#ifdef CONFIG_KGDB
-#define KGDB_TRAP(num) \
- b kgdb_trap_low; \
- rd %psr,%l0; \
- nop; \
+#define KGDB_TRAP(num) \
+ mov num, %l7; \
+ b kgdb_trap_low; \
+ rd %psr,%l0; \
nop;
#else
#define KGDB_TRAP(num) \
diff --git a/arch/sparc/include/asm/kgdb.h b/arch/sparc/include/asm/kgdb.h
index 47366af..a6ad7bf 100644
--- a/arch/sparc/include/asm/kgdb.h
+++ b/arch/sparc/include/asm/kgdb.h
@@ -28,10 +28,10 @@
#define NUMREGBYTES ((GDB_CSR + 1) * 4)
#else
#define NUMREGBYTES ((GDB_Y + 1) * 8)
+#endif
struct pt_regs;
asmlinkage void kgdb_trap(unsigned long trap_level, struct pt_regs *regs);
-#endif
void arch_kgdb_breakpoint(void);
diff --git a/arch/sparc/include/asm/page_32.h b/arch/sparc/include/asm/page_32.h
index f82a1f3..0efd058 100644
--- a/arch/sparc/include/asm/page_32.h
+++ b/arch/sparc/include/asm/page_32.h
@@ -69,7 +69,6 @@
#define __pte(x) ((pte_t) { (x) } )
#define __iopte(x) ((iopte_t) { (x) } )
-/* #define __pmd(x) ((pmd_t) { (x) } ) */ /* XXX procedure with loop */
#define __pgd(x) ((pgd_t) { (x) } )
#define __ctxd(x) ((ctxd_t) { (x) } )
#define __pgprot(x) ((pgprot_t) { (x) } )
@@ -97,7 +96,6 @@
#define __pte(x) (x)
#define __iopte(x) (x)
-/* #define __pmd(x) (x) */ /* XXX later */
#define __pgd(x) (x)
#define __ctxd(x) (x)
#define __pgprot(x) (x)
diff --git a/arch/sparc/include/asm/pgalloc_32.h b/arch/sparc/include/asm/pgalloc_32.h
index a3890da..0346c7e 100644
--- a/arch/sparc/include/asm/pgalloc_32.h
+++ b/arch/sparc/include/asm/pgalloc_32.h
@@ -29,9 +29,9 @@
static inline void pgd_set(pgd_t * pgdp, pmd_t * pmdp)
{
- unsigned long pa = __nocache_pa((unsigned long)pmdp);
+ unsigned long pa = __nocache_pa(pmdp);
- set_pte((pte_t *)pgdp, (SRMMU_ET_PTD | (pa >> 4)));
+ set_pte((pte_t *)pgdp, __pte((SRMMU_ET_PTD | (pa >> 4))));
}
#define pgd_populate(MM, PGD, PMD) pgd_set(PGD, PMD)
diff --git a/arch/sparc/include/asm/pgtable_32.h b/arch/sparc/include/asm/pgtable_32.h
index 91b963a..ce6f569 100644
--- a/arch/sparc/include/asm/pgtable_32.h
+++ b/arch/sparc/include/asm/pgtable_32.h
@@ -298,7 +298,7 @@
#define pgprot_noncached pgprot_noncached
static inline pgprot_t pgprot_noncached(pgprot_t prot)
{
- prot &= ~__pgprot(SRMMU_CACHE);
+ pgprot_val(prot) &= ~pgprot_val(__pgprot(SRMMU_CACHE));
return prot;
}
diff --git a/arch/sparc/include/asm/pgtable_64.h b/arch/sparc/include/asm/pgtable_64.h
index f089cfa..e7d8280 100644
--- a/arch/sparc/include/asm/pgtable_64.h
+++ b/arch/sparc/include/asm/pgtable_64.h
@@ -375,7 +375,7 @@
#define pgprot_noncached pgprot_noncached
#if defined(CONFIG_HUGETLB_PAGE) || defined(CONFIG_TRANSPARENT_HUGEPAGE)
-static inline pte_t pte_mkhuge(pte_t pte)
+static inline unsigned long __pte_huge_mask(void)
{
unsigned long mask;
@@ -390,8 +390,19 @@
: "=r" (mask)
: "i" (_PAGE_SZHUGE_4U), "i" (_PAGE_SZHUGE_4V));
- return __pte(pte_val(pte) | mask);
+ return mask;
}
+
+static inline pte_t pte_mkhuge(pte_t pte)
+{
+ return __pte(pte_val(pte) | __pte_huge_mask());
+}
+
+static inline bool is_hugetlb_pte(pte_t pte)
+{
+ return !!(pte_val(pte) & __pte_huge_mask());
+}
+
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
static inline pmd_t pmd_mkhuge(pmd_t pmd)
{
@@ -403,6 +414,11 @@
return __pmd(pte_val(pte));
}
#endif
+#else
+static inline bool is_hugetlb_pte(pte_t pte)
+{
+ return false;
+}
#endif
static inline pte_t pte_mkdirty(pte_t pte)
@@ -681,8 +697,6 @@
return pte_val(pte) & _PAGE_PMD_HUGE;
}
-#define has_transparent_hugepage() 1
-
static inline pmd_t pmd_mkold(pmd_t pmd)
{
pte_t pte = __pte(pmd_val(pmd));
@@ -858,6 +872,19 @@
void tlb_batch_add(struct mm_struct *mm, unsigned long vaddr,
pte_t *ptep, pte_t orig, int fullmm);
+static void maybe_tlb_batch_add(struct mm_struct *mm, unsigned long vaddr,
+ pte_t *ptep, pte_t orig, int fullmm)
+{
+ /* It is more efficient to let flush_tlb_kernel_range()
+ * handle init_mm tlb flushes.
+ *
+ * SUN4V NOTE: _PAGE_VALID is the same value in both the SUN4U
+ * and SUN4V pte layout, so this inline test is fine.
+ */
+ if (likely(mm != &init_mm) && pte_accessible(mm, orig))
+ tlb_batch_add(mm, vaddr, ptep, orig, fullmm);
+}
+
#define __HAVE_ARCH_PMDP_HUGE_GET_AND_CLEAR
static inline pmd_t pmdp_huge_get_and_clear(struct mm_struct *mm,
unsigned long addr,
@@ -874,15 +901,7 @@
pte_t orig = *ptep;
*ptep = pte;
-
- /* It is more efficient to let flush_tlb_kernel_range()
- * handle init_mm tlb flushes.
- *
- * SUN4V NOTE: _PAGE_VALID is the same value in both the SUN4U
- * and SUN4V pte layout, so this inline test is fine.
- */
- if (likely(mm != &init_mm) && pte_accessible(mm, orig))
- tlb_batch_add(mm, addr, ptep, orig, fullmm);
+ maybe_tlb_batch_add(mm, addr, ptep, orig, fullmm);
}
#define set_pte_at(mm,addr,ptep,pte) \
diff --git a/arch/sparc/include/asm/tlbflush_64.h b/arch/sparc/include/asm/tlbflush_64.h
index dea1cfa..a8e192e 100644
--- a/arch/sparc/include/asm/tlbflush_64.h
+++ b/arch/sparc/include/asm/tlbflush_64.h
@@ -8,6 +8,7 @@
#define TLB_BATCH_NR 192
struct tlb_batch {
+ bool huge;
struct mm_struct *mm;
unsigned long tlb_nr;
unsigned long active;
@@ -16,7 +17,7 @@
void flush_tsb_kernel_range(unsigned long start, unsigned long end);
void flush_tsb_user(struct tlb_batch *tb);
-void flush_tsb_user_page(struct mm_struct *mm, unsigned long vaddr);
+void flush_tsb_user_page(struct mm_struct *mm, unsigned long vaddr, bool huge);
/* TLB flush operations. */
diff --git a/arch/sparc/kernel/entry.S b/arch/sparc/kernel/entry.S
index 51aa6e8..07918ab 100644
--- a/arch/sparc/kernel/entry.S
+++ b/arch/sparc/kernel/entry.S
@@ -1225,20 +1225,18 @@
RESTORE_ALL
#ifdef CONFIG_KGDB
- .align 4
- .globl kgdb_trap_low
- .type kgdb_trap_low,#function
-kgdb_trap_low:
+ ENTRY(kgdb_trap_low)
rd %wim,%l3
SAVE_ALL
wr %l0, PSR_ET, %psr
WRITE_PAUSE
+ mov %l7, %o0 ! trap_level
call kgdb_trap
- add %sp, STACKFRAME_SZ, %o0
+ add %sp, STACKFRAME_SZ, %o1 ! struct pt_regs *regs
RESTORE_ALL
- .size kgdb_trap_low,.-kgdb_trap_low
+ ENDPROC(kgdb_trap_low)
#endif
.align 4
diff --git a/arch/sparc/kernel/kernel.h b/arch/sparc/kernel/kernel.h
index 5057ec2..c9804551 100644
--- a/arch/sparc/kernel/kernel.h
+++ b/arch/sparc/kernel/kernel.h
@@ -127,6 +127,7 @@
extern unsigned int linux_trap_ipi15_sun4d[];
extern unsigned int linux_trap_ipi15_sun4m[];
+extern struct tt_entry trapbase;
extern struct tt_entry trapbase_cpu1;
extern struct tt_entry trapbase_cpu2;
extern struct tt_entry trapbase_cpu3;
diff --git a/arch/sparc/kernel/kgdb_32.c b/arch/sparc/kernel/kgdb_32.c
index dcf2108..6e8e318 100644
--- a/arch/sparc/kernel/kgdb_32.c
+++ b/arch/sparc/kernel/kgdb_32.c
@@ -12,7 +12,8 @@
#include <asm/irq.h>
#include <asm/cacheflush.h>
-extern unsigned long trapbase;
+#include "kernel.h"
+#include "entry.h"
void pt_regs_to_gdb_regs(unsigned long *gdb_regs, struct pt_regs *regs)
{
@@ -133,21 +134,19 @@
return -1;
}
-extern void do_hw_interrupt(struct pt_regs *regs, unsigned long type);
-
-asmlinkage void kgdb_trap(struct pt_regs *regs)
+asmlinkage void kgdb_trap(unsigned long trap_level, struct pt_regs *regs)
{
unsigned long flags;
if (user_mode(regs)) {
- do_hw_interrupt(regs, 0xfd);
+ do_hw_interrupt(regs, trap_level);
return;
}
flushw_all();
local_irq_save(flags);
- kgdb_handle_exception(0x172, SIGTRAP, 0, regs);
+ kgdb_handle_exception(trap_level, SIGTRAP, 0, regs);
local_irq_restore(flags);
}
diff --git a/arch/sparc/kernel/process_32.c b/arch/sparc/kernel/process_32.c
index c5113c7..b7780a5 100644
--- a/arch/sparc/kernel/process_32.c
+++ b/arch/sparc/kernel/process_32.c
@@ -184,21 +184,21 @@
/*
* Free current thread data structures etc..
*/
-void exit_thread(void)
+void exit_thread(struct task_struct *tsk)
{
#ifndef CONFIG_SMP
- if(last_task_used_math == current) {
+ if (last_task_used_math == tsk) {
#else
- if (test_thread_flag(TIF_USEDFPU)) {
+ if (test_ti_thread_flag(task_thread_info(tsk), TIF_USEDFPU)) {
#endif
/* Keep process from leaving FPU in a bogon state. */
put_psr(get_psr() | PSR_EF);
- fpsave(¤t->thread.float_regs[0], ¤t->thread.fsr,
- ¤t->thread.fpqueue[0], ¤t->thread.fpqdepth);
+ fpsave(&tsk->thread.float_regs[0], &tsk->thread.fsr,
+ &tsk->thread.fpqueue[0], &tsk->thread.fpqdepth);
#ifndef CONFIG_SMP
last_task_used_math = NULL;
#else
- clear_thread_flag(TIF_USEDFPU);
+ clear_ti_thread_flag(task_thread_info(tsk), TIF_USEDFPU);
#endif
}
}
diff --git a/arch/sparc/kernel/process_64.c b/arch/sparc/kernel/process_64.c
index c16ef1a..fa14402 100644
--- a/arch/sparc/kernel/process_64.c
+++ b/arch/sparc/kernel/process_64.c
@@ -417,9 +417,9 @@
}
/* Free current thread data structures etc.. */
-void exit_thread(void)
+void exit_thread(struct task_struct *tsk)
{
- struct thread_info *t = current_thread_info();
+ struct thread_info *t = task_thread_info(tsk);
if (t->utraps) {
if (t->utraps[0] < 2)
diff --git a/arch/sparc/kernel/setup_32.c b/arch/sparc/kernel/setup_32.c
index 69d75ff1..c4e65cb 100644
--- a/arch/sparc/kernel/setup_32.c
+++ b/arch/sparc/kernel/setup_32.c
@@ -68,8 +68,6 @@
* prints out pretty messages and returns.
*/
-extern unsigned long trapbase;
-
/* Pretty sick eh? */
static void prom_sync_me(void)
{
@@ -300,7 +298,7 @@
int i;
unsigned long highest_paddr;
- sparc_ttable = (struct tt_entry *) &trapbase;
+ sparc_ttable = &trapbase;
/* Initialize PROM console and command line. */
*cmdline_p = prom_getbootargs();
diff --git a/arch/sparc/mm/hugetlbpage.c b/arch/sparc/mm/hugetlbpage.c
index 4977800..ba52e64 100644
--- a/arch/sparc/mm/hugetlbpage.c
+++ b/arch/sparc/mm/hugetlbpage.c
@@ -176,17 +176,31 @@
pte_t *ptep, pte_t entry)
{
int i;
+ pte_t orig[2];
+ unsigned long nptes;
if (!pte_present(*ptep) && pte_present(entry))
mm->context.huge_pte_count++;
addr &= HPAGE_MASK;
- for (i = 0; i < (1 << HUGETLB_PAGE_ORDER); i++) {
- set_pte_at(mm, addr, ptep, entry);
+
+ nptes = 1 << HUGETLB_PAGE_ORDER;
+ orig[0] = *ptep;
+ orig[1] = *(ptep + nptes / 2);
+ for (i = 0; i < nptes; i++) {
+ *ptep = entry;
ptep++;
addr += PAGE_SIZE;
pte_val(entry) += PAGE_SIZE;
}
+
+ /* Issue TLB flush at REAL_HPAGE_SIZE boundaries */
+ addr -= REAL_HPAGE_SIZE;
+ ptep -= nptes / 2;
+ maybe_tlb_batch_add(mm, addr, ptep, orig[1], 0);
+ addr -= REAL_HPAGE_SIZE;
+ ptep -= nptes / 2;
+ maybe_tlb_batch_add(mm, addr, ptep, orig[0], 0);
}
pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr,
@@ -194,19 +208,28 @@
{
pte_t entry;
int i;
+ unsigned long nptes;
entry = *ptep;
if (pte_present(entry))
mm->context.huge_pte_count--;
addr &= HPAGE_MASK;
-
- for (i = 0; i < (1 << HUGETLB_PAGE_ORDER); i++) {
- pte_clear(mm, addr, ptep);
+ nptes = 1 << HUGETLB_PAGE_ORDER;
+ for (i = 0; i < nptes; i++) {
+ *ptep = __pte(0UL);
addr += PAGE_SIZE;
ptep++;
}
+ /* Issue TLB flush at REAL_HPAGE_SIZE boundaries */
+ addr -= REAL_HPAGE_SIZE;
+ ptep -= nptes / 2;
+ maybe_tlb_batch_add(mm, addr, ptep, entry, 0);
+ addr -= REAL_HPAGE_SIZE;
+ ptep -= nptes / 2;
+ maybe_tlb_batch_add(mm, addr, ptep, entry, 0);
+
return entry;
}
diff --git a/arch/sparc/mm/init_64.c b/arch/sparc/mm/init_64.c
index 09e8388..652683c 100644
--- a/arch/sparc/mm/init_64.c
+++ b/arch/sparc/mm/init_64.c
@@ -324,18 +324,6 @@
tsb_insert(tsb, tag, tte);
}
-#if defined(CONFIG_HUGETLB_PAGE) || defined(CONFIG_TRANSPARENT_HUGEPAGE)
-static inline bool is_hugetlb_pte(pte_t pte)
-{
- if ((tlb_type == hypervisor &&
- (pte_val(pte) & _PAGE_SZALL_4V) == _PAGE_SZHUGE_4V) ||
- (tlb_type != hypervisor &&
- (pte_val(pte) & _PAGE_SZALL_4U) == _PAGE_SZHUGE_4U))
- return true;
- return false;
-}
-#endif
-
void update_mmu_cache(struct vm_area_struct *vma, unsigned long address, pte_t *ptep)
{
struct mm_struct *mm;
diff --git a/arch/sparc/mm/io-unit.c b/arch/sparc/mm/io-unit.c
index f311bf2..338fb71 100644
--- a/arch/sparc/mm/io-unit.c
+++ b/arch/sparc/mm/io-unit.c
@@ -133,7 +133,7 @@
vaddr = IOUNIT_DMA_BASE + (scan << PAGE_SHIFT) + (vaddr & ~PAGE_MASK);
for (k = 0; k < npages; k++, iopte = __iopte(iopte_val(iopte) + 0x100), scan++) {
set_bit(scan, iounit->bmap);
- sbus_writel(iopte, &iounit->page_table[scan]);
+ sbus_writel(iopte_val(iopte), &iounit->page_table[scan]);
}
IOD(("%08lx\n", vaddr));
return vaddr;
@@ -228,7 +228,7 @@
i = ((addr - IOUNIT_DMA_BASE) >> PAGE_SHIFT);
iopte = iounit->page_table + i;
- sbus_writel(MKIOPTE(__pa(page)), iopte);
+ sbus_writel(iopte_val(MKIOPTE(__pa(page))), iopte);
}
addr += PAGE_SIZE;
va += PAGE_SIZE;
diff --git a/arch/sparc/mm/srmmu.c b/arch/sparc/mm/srmmu.c
index 5cbc96d..c7f2a52 100644
--- a/arch/sparc/mm/srmmu.c
+++ b/arch/sparc/mm/srmmu.c
@@ -107,17 +107,22 @@
/* XXX should we hyper_flush_whole_icache here - Anton */
static inline void srmmu_ctxd_set(ctxd_t *ctxp, pgd_t *pgdp)
-{ set_pte((pte_t *)ctxp, (SRMMU_ET_PTD | (__nocache_pa((unsigned long) pgdp) >> 4))); }
+{
+ pte_t pte;
+
+ pte = __pte((SRMMU_ET_PTD | (__nocache_pa(pgdp) >> 4)));
+ set_pte((pte_t *)ctxp, pte);
+}
void pmd_set(pmd_t *pmdp, pte_t *ptep)
{
unsigned long ptp; /* Physical address, shifted right by 4 */
int i;
- ptp = __nocache_pa((unsigned long) ptep) >> 4;
+ ptp = __nocache_pa(ptep) >> 4;
for (i = 0; i < PTRS_PER_PTE/SRMMU_REAL_PTRS_PER_PTE; i++) {
- set_pte((pte_t *)&pmdp->pmdv[i], SRMMU_ET_PTD | ptp);
- ptp += (SRMMU_REAL_PTRS_PER_PTE*sizeof(pte_t) >> 4);
+ set_pte((pte_t *)&pmdp->pmdv[i], __pte(SRMMU_ET_PTD | ptp));
+ ptp += (SRMMU_REAL_PTRS_PER_PTE * sizeof(pte_t) >> 4);
}
}
@@ -128,8 +133,8 @@
ptp = page_to_pfn(ptep) << (PAGE_SHIFT-4); /* watch for overflow */
for (i = 0; i < PTRS_PER_PTE/SRMMU_REAL_PTRS_PER_PTE; i++) {
- set_pte((pte_t *)&pmdp->pmdv[i], SRMMU_ET_PTD | ptp);
- ptp += (SRMMU_REAL_PTRS_PER_PTE*sizeof(pte_t) >> 4);
+ set_pte((pte_t *)&pmdp->pmdv[i], __pte(SRMMU_ET_PTD | ptp));
+ ptp += (SRMMU_REAL_PTRS_PER_PTE * sizeof(pte_t) >> 4);
}
}
@@ -911,7 +916,7 @@
/* ctx table has to be physically aligned to its size */
srmmu_context_table = __srmmu_get_nocache(num_contexts * sizeof(ctxd_t), num_contexts * sizeof(ctxd_t));
- srmmu_ctx_table_phys = (ctxd_t *)__nocache_pa((unsigned long)srmmu_context_table);
+ srmmu_ctx_table_phys = (ctxd_t *)__nocache_pa(srmmu_context_table);
for (i = 0; i < num_contexts; i++)
srmmu_ctxd_set((ctxd_t *)__nocache_fix(&srmmu_context_table[i]), srmmu_swapper_pg_dir);
diff --git a/arch/sparc/mm/tlb.c b/arch/sparc/mm/tlb.c
index 9df2190c..f81cd97 100644
--- a/arch/sparc/mm/tlb.c
+++ b/arch/sparc/mm/tlb.c
@@ -67,7 +67,7 @@
}
static void tlb_batch_add_one(struct mm_struct *mm, unsigned long vaddr,
- bool exec)
+ bool exec, bool huge)
{
struct tlb_batch *tb = &get_cpu_var(tlb_batch);
unsigned long nr;
@@ -84,13 +84,21 @@
}
if (!tb->active) {
- flush_tsb_user_page(mm, vaddr);
+ flush_tsb_user_page(mm, vaddr, huge);
global_flush_tlb_page(mm, vaddr);
goto out;
}
- if (nr == 0)
+ if (nr == 0) {
tb->mm = mm;
+ tb->huge = huge;
+ }
+
+ if (tb->huge != huge) {
+ flush_tlb_pending();
+ tb->huge = huge;
+ nr = 0;
+ }
tb->vaddrs[nr] = vaddr;
tb->tlb_nr = ++nr;
@@ -104,6 +112,8 @@
void tlb_batch_add(struct mm_struct *mm, unsigned long vaddr,
pte_t *ptep, pte_t orig, int fullmm)
{
+ bool huge = is_hugetlb_pte(orig);
+
if (tlb_type != hypervisor &&
pte_dirty(orig)) {
unsigned long paddr, pfn = pte_pfn(orig);
@@ -129,7 +139,7 @@
no_cache_flush:
if (!fullmm)
- tlb_batch_add_one(mm, vaddr, pte_exec(orig));
+ tlb_batch_add_one(mm, vaddr, pte_exec(orig), huge);
}
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
@@ -145,7 +155,7 @@
if (pte_val(*pte) & _PAGE_VALID) {
bool exec = pte_exec(*pte);
- tlb_batch_add_one(mm, vaddr, exec);
+ tlb_batch_add_one(mm, vaddr, exec, false);
}
pte++;
vaddr += PAGE_SIZE;
@@ -185,8 +195,9 @@
pte_t orig_pte = __pte(pmd_val(orig));
bool exec = pte_exec(orig_pte);
- tlb_batch_add_one(mm, addr, exec);
- tlb_batch_add_one(mm, addr + REAL_HPAGE_SIZE, exec);
+ tlb_batch_add_one(mm, addr, exec, true);
+ tlb_batch_add_one(mm, addr + REAL_HPAGE_SIZE, exec,
+ true);
} else {
tlb_batch_pmd_scan(mm, addr, orig);
}
diff --git a/arch/sparc/mm/tsb.c b/arch/sparc/mm/tsb.c
index a065766..a0604a4 100644
--- a/arch/sparc/mm/tsb.c
+++ b/arch/sparc/mm/tsb.c
@@ -76,14 +76,15 @@
spin_lock_irqsave(&mm->context.lock, flags);
- base = (unsigned long) mm->context.tsb_block[MM_TSB_BASE].tsb;
- nentries = mm->context.tsb_block[MM_TSB_BASE].tsb_nentries;
- if (tlb_type == cheetah_plus || tlb_type == hypervisor)
- base = __pa(base);
- __flush_tsb_one(tb, PAGE_SHIFT, base, nentries);
-
+ if (!tb->huge) {
+ base = (unsigned long) mm->context.tsb_block[MM_TSB_BASE].tsb;
+ nentries = mm->context.tsb_block[MM_TSB_BASE].tsb_nentries;
+ if (tlb_type == cheetah_plus || tlb_type == hypervisor)
+ base = __pa(base);
+ __flush_tsb_one(tb, PAGE_SHIFT, base, nentries);
+ }
#if defined(CONFIG_HUGETLB_PAGE) || defined(CONFIG_TRANSPARENT_HUGEPAGE)
- if (mm->context.tsb_block[MM_TSB_HUGE].tsb) {
+ if (tb->huge && mm->context.tsb_block[MM_TSB_HUGE].tsb) {
base = (unsigned long) mm->context.tsb_block[MM_TSB_HUGE].tsb;
nentries = mm->context.tsb_block[MM_TSB_HUGE].tsb_nentries;
if (tlb_type == cheetah_plus || tlb_type == hypervisor)
@@ -94,20 +95,21 @@
spin_unlock_irqrestore(&mm->context.lock, flags);
}
-void flush_tsb_user_page(struct mm_struct *mm, unsigned long vaddr)
+void flush_tsb_user_page(struct mm_struct *mm, unsigned long vaddr, bool huge)
{
unsigned long nentries, base, flags;
spin_lock_irqsave(&mm->context.lock, flags);
- base = (unsigned long) mm->context.tsb_block[MM_TSB_BASE].tsb;
- nentries = mm->context.tsb_block[MM_TSB_BASE].tsb_nentries;
- if (tlb_type == cheetah_plus || tlb_type == hypervisor)
- base = __pa(base);
- __flush_tsb_one_entry(base, vaddr, PAGE_SHIFT, nentries);
-
+ if (!huge) {
+ base = (unsigned long) mm->context.tsb_block[MM_TSB_BASE].tsb;
+ nentries = mm->context.tsb_block[MM_TSB_BASE].tsb_nentries;
+ if (tlb_type == cheetah_plus || tlb_type == hypervisor)
+ base = __pa(base);
+ __flush_tsb_one_entry(base, vaddr, PAGE_SHIFT, nentries);
+ }
#if defined(CONFIG_HUGETLB_PAGE) || defined(CONFIG_TRANSPARENT_HUGEPAGE)
- if (mm->context.tsb_block[MM_TSB_HUGE].tsb) {
+ if (huge && mm->context.tsb_block[MM_TSB_HUGE].tsb) {
base = (unsigned long) mm->context.tsb_block[MM_TSB_HUGE].tsb;
nentries = mm->context.tsb_block[MM_TSB_HUGE].tsb_nentries;
if (tlb_type == cheetah_plus || tlb_type == hypervisor)
diff --git a/arch/tile/Kconfig b/arch/tile/Kconfig
index 8171930..4820a02 100644
--- a/arch/tile/Kconfig
+++ b/arch/tile/Kconfig
@@ -3,47 +3,38 @@
config TILE
def_bool y
- select HAVE_PERF_EVENTS
- select USE_PMC if PERF_EVENTS
- select HAVE_DMA_API_DEBUG
- select HAVE_KVM if !TILEGX
- select GENERIC_FIND_FIRST_BIT
- select SYSCTL_EXCEPTION_TRACE
- select CC_OPTIMIZE_FOR_SIZE
- select HAVE_DEBUG_KMEMLEAK
- select GENERIC_IRQ_PROBE
- select GENERIC_PENDING_IRQ if SMP
- select GENERIC_IRQ_SHOW
- select HAVE_DEBUG_BUGVERBOSE
- select VIRT_TO_BUS
- select SYS_HYPERVISOR
+ select ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE
select ARCH_HAS_DEBUG_STRICT_USER_COPY_CHECKS
select ARCH_HAS_DEVMEM_IS_ALLOWED
select ARCH_HAVE_NMI_SAFE_CMPXCHG
- select GENERIC_CLOCKEVENTS
- select MODULES_USE_ELF_RELA
- select HAVE_ARCH_TRACEHOOK
- select HAVE_SYSCALL_TRACEPOINTS
- select USER_STACKTRACE_SUPPORT
- select ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE
- select HAVE_DEBUG_STACKOVERFLOW
select ARCH_WANT_FRAME_POINTERS
- select HAVE_CONTEXT_TRACKING
+ select CC_OPTIMIZE_FOR_SIZE
select EDAC_SUPPORT
+ select GENERIC_CLOCKEVENTS
+ select GENERIC_FIND_FIRST_BIT
+ select GENERIC_IRQ_PROBE
+ select GENERIC_IRQ_SHOW
+ select GENERIC_PENDING_IRQ if SMP
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
select HAVE_ARCH_SECCOMP_FILTER
-
-# FIXME: investigate whether we need/want these options.
-# select HAVE_IOREMAP_PROT
-# select HAVE_OPTPROBES
-# select HAVE_REGS_AND_STACK_ACCESS_API
-# select HAVE_HW_BREAKPOINT
-# select PERF_EVENTS
-# select HAVE_USER_RETURN_NOTIFIER
-# config NO_BOOTMEM
-# config ARCH_SUPPORTS_DEBUG_PAGEALLOC
-# config HUGETLB_PAGE_SIZE_VARIABLE
+ select HAVE_ARCH_TRACEHOOK
+ select HAVE_CONTEXT_TRACKING
+ select HAVE_DEBUG_BUGVERBOSE
+ select HAVE_DEBUG_KMEMLEAK
+ select HAVE_DEBUG_STACKOVERFLOW
+ select HAVE_DMA_API_DEBUG
+ select HAVE_EXIT_THREAD
+ select HAVE_KVM if !TILEGX
+ select HAVE_NMI if USE_PMC
+ select HAVE_PERF_EVENTS
+ select HAVE_SYSCALL_TRACEPOINTS
+ select MODULES_USE_ELF_RELA
+ select SYSCTL_EXCEPTION_TRACE
+ select SYS_HYPERVISOR
+ select USER_STACKTRACE_SUPPORT
+ select USE_PMC if PERF_EVENTS
+ select VIRT_TO_BUS
config MMU
def_bool y
@@ -130,17 +121,17 @@
# 64-bit TILE-Gx toolchain, so force CONFIG_TILEGX on.
config TILEGX
def_bool ARCH != "tilepro"
- select SPARSE_IRQ
+ select ARCH_SUPPORTS_ATOMIC_RMW
select GENERIC_IRQ_LEGACY_ALLOC_HWIRQ
- select HAVE_FUNCTION_TRACER
- select HAVE_FUNCTION_GRAPH_TRACER
+ select HAVE_ARCH_JUMP_LABEL
+ select HAVE_ARCH_KGDB
select HAVE_DYNAMIC_FTRACE
select HAVE_FTRACE_MCOUNT_RECORD
+ select HAVE_FUNCTION_GRAPH_TRACER
+ select HAVE_FUNCTION_TRACER
select HAVE_KPROBES
select HAVE_KRETPROBES
- select HAVE_ARCH_KGDB
- select ARCH_SUPPORTS_ATOMIC_RMW
- select HAVE_ARCH_JUMP_LABEL
+ select SPARSE_IRQ
config TILEPRO
def_bool !TILEGX
diff --git a/arch/tile/configs/tilegx_defconfig b/arch/tile/configs/tilegx_defconfig
index 7189055..fd122ef 100644
--- a/arch/tile/configs/tilegx_defconfig
+++ b/arch/tile/configs/tilegx_defconfig
@@ -16,7 +16,6 @@
CONFIG_CGROUP_DEVICE=y
CONFIG_CPUSETS=y
CONFIG_CGROUP_CPUACCT=y
-CONFIG_RESOURCE_COUNTERS=y
CONFIG_CGROUP_SCHED=y
CONFIG_RT_GROUP_SCHED=y
CONFIG_BLK_CGROUP=y
@@ -89,7 +88,6 @@
CONFIG_TCP_CONG_ILLINOIS=m
CONFIG_TCP_MD5SIG=y
CONFIG_IPV6=y
-CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_IPV6_ROUTE_INFO=y
CONFIG_IPV6_OPTIMISTIC_DAD=y
diff --git a/arch/tile/configs/tilepro_defconfig b/arch/tile/configs/tilepro_defconfig
index dc85468..eb6a559 100644
--- a/arch/tile/configs/tilepro_defconfig
+++ b/arch/tile/configs/tilepro_defconfig
@@ -15,7 +15,6 @@
CONFIG_CGROUP_DEVICE=y
CONFIG_CPUSETS=y
CONFIG_CGROUP_CPUACCT=y
-CONFIG_RESOURCE_COUNTERS=y
CONFIG_CGROUP_SCHED=y
CONFIG_RT_GROUP_SCHED=y
CONFIG_BLK_CGROUP=y
@@ -85,7 +84,6 @@
CONFIG_TCP_CONG_ILLINOIS=m
CONFIG_TCP_MD5SIG=y
CONFIG_IPV6=y
-CONFIG_IPV6_PRIVACY=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_IPV6_ROUTE_INFO=y
CONFIG_IPV6_OPTIMISTIC_DAD=y
diff --git a/arch/tile/gxio/mpipe.c b/arch/tile/gxio/mpipe.c
index f102048..34de300 100644
--- a/arch/tile/gxio/mpipe.c
+++ b/arch/tile/gxio/mpipe.c
@@ -122,7 +122,7 @@
{
const int BUFFERS_PER_LINE = 12;
- /* Count the number of cachlines. */
+ /* Count the number of cachelines. */
unsigned long lines =
(buffers + BUFFERS_PER_LINE - 1) / BUFFERS_PER_LINE;
diff --git a/arch/tile/include/asm/atomic_64.h b/arch/tile/include/asm/atomic_64.h
index 51cabc2..b0531a6 100644
--- a/arch/tile/include/asm/atomic_64.h
+++ b/arch/tile/include/asm/atomic_64.h
@@ -37,12 +37,25 @@
__insn_fetchadd4((void *)&v->counter, i);
}
+/*
+ * Note a subtlety of the locking here. We are required to provide a
+ * full memory barrier before and after the operation. However, we
+ * only provide an explicit mb before the operation. After the
+ * operation, we use barrier() to get a full mb for free, because:
+ *
+ * (1) The barrier directive to the compiler prohibits any instructions
+ * being statically hoisted before the barrier;
+ * (2) the microarchitecture will not issue any further instructions
+ * until the fetchadd result is available for the "+ i" add instruction;
+ * (3) the smb_mb before the fetchadd ensures that no other memory
+ * operations are in flight at this point.
+ */
static inline int atomic_add_return(int i, atomic_t *v)
{
int val;
smp_mb(); /* barrier for proper semantics */
val = __insn_fetchadd4((void *)&v->counter, i) + i;
- barrier(); /* the "+ i" above will wait on memory */
+ barrier(); /* equivalent to smp_mb(); see block comment above */
return val;
}
@@ -95,7 +108,7 @@
int val;
smp_mb(); /* barrier for proper semantics */
val = __insn_fetchadd((void *)&v->counter, i) + i;
- barrier(); /* the "+ i" above will wait on memory */
+ barrier(); /* equivalent to smp_mb; see atomic_add_return() */
return val;
}
diff --git a/arch/tile/include/asm/pgtable.h b/arch/tile/include/asm/pgtable.h
index 96cecf5..2a26cc4 100644
--- a/arch/tile/include/asm/pgtable.h
+++ b/arch/tile/include/asm/pgtable.h
@@ -487,7 +487,6 @@
}
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-#define has_transparent_hugepage() 1
#define pmd_trans_huge pmd_huge_page
#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
diff --git a/arch/tile/include/uapi/asm/unistd.h b/arch/tile/include/uapi/asm/unistd.h
index 3866397..24e9187 100644
--- a/arch/tile/include/uapi/asm/unistd.h
+++ b/arch/tile/include/uapi/asm/unistd.h
@@ -12,6 +12,7 @@
* more details.
*/
+#define __ARCH_WANT_RENAMEAT
#if !defined(__LP64__) || defined(__SYSCALL_COMPAT)
/* Use the flavor of this syscall that matches the 32-bit API better. */
#define __ARCH_WANT_SYNC_FILE_RANGE2
diff --git a/arch/tile/kernel/pci_gx.c b/arch/tile/kernel/pci_gx.c
index aa2b44c..0e7a5d0 100644
--- a/arch/tile/kernel/pci_gx.c
+++ b/arch/tile/kernel/pci_gx.c
@@ -40,7 +40,7 @@
#include <arch/sim.h>
/*
- * This file containes the routines to search for PCI buses,
+ * This file contains the routines to search for PCI buses,
* enumerate the buses, and configure any attached devices.
*/
@@ -434,7 +434,7 @@
/*
* Now determine which PCIe ports are configured to operate in RC
- * mode. There is a differece in the port configuration capability
+ * mode. There is a difference in the port configuration capability
* between the Gx36 and Gx72 devices.
*
* The Gx36 has configuration capability for each of the 3 PCIe
diff --git a/arch/tile/kernel/process.c b/arch/tile/kernel/process.c
index b5f30d3..6b705cc 100644
--- a/arch/tile/kernel/process.c
+++ b/arch/tile/kernel/process.c
@@ -541,7 +541,7 @@
/*
* Free current thread data structures etc..
*/
-void exit_thread(void)
+void exit_thread(struct task_struct *tsk)
{
#ifdef CONFIG_HARDWALL
/*
@@ -550,7 +550,7 @@
* the last reference to a hardwall fd, it would already have
* been released and deactivated at this point.)
*/
- hardwall_deactivate_all(current);
+ hardwall_deactivate_all(tsk);
#endif
}
diff --git a/arch/tile/kernel/setup.c b/arch/tile/kernel/setup.c
index a992238..153020a 100644
--- a/arch/tile/kernel/setup.c
+++ b/arch/tile/kernel/setup.c
@@ -962,9 +962,7 @@
cpumask_set_cpu(best_cpu, &node_2_cpu_mask[node]);
cpu_2_node[best_cpu] = node;
cpumask_clear_cpu(best_cpu, &unbound_cpus);
- node = next_node(node, default_nodes);
- if (node == MAX_NUMNODES)
- node = first_node(default_nodes);
+ node = next_node_in(node, default_nodes);
}
/* Print out node assignments and set defaults for disabled cpus */
diff --git a/arch/tile/kernel/unaligned.c b/arch/tile/kernel/unaligned.c
index 0db5f7c..9772a35 100644
--- a/arch/tile/kernel/unaligned.c
+++ b/arch/tile/kernel/unaligned.c
@@ -188,7 +188,7 @@
* Parse fault bundle, find potential used registers and mark
* corresponding bits in reg_map and alias_map. These 2 bit maps
* are used to find the scratch registers and determine if there
- * is register alais.
+ * is register alias.
*/
if (bundle & TILEGX_BUNDLE_MODE_MASK) { /* Y Mode Bundle. */
@@ -1529,7 +1529,7 @@
}
- /* Read the bundle casued the exception! */
+ /* Read the bundle caused the exception! */
pc = (tilegx_bundle_bits __user *)(regs->pc);
if (get_user(bundle, pc) != 0) {
/* Probably never be here since pc is valid user address.*/
diff --git a/arch/tile/mm/hugetlbpage.c b/arch/tile/mm/hugetlbpage.c
index e212c64..77ceaa3 100644
--- a/arch/tile/mm/hugetlbpage.c
+++ b/arch/tile/mm/hugetlbpage.c
@@ -308,11 +308,16 @@
static __init int setup_hugepagesz(char *opt)
{
+ int rc;
+
if (!saw_hugepagesz) {
saw_hugepagesz = true;
memset(huge_shift, 0, sizeof(huge_shift));
}
- return __setup_hugepagesz(memparse(opt, NULL));
+ rc = __setup_hugepagesz(memparse(opt, NULL));
+ if (rc)
+ hugetlb_bad_size();
+ return rc;
}
__setup("hugepagesz=", setup_hugepagesz);
diff --git a/arch/tile/mm/init.c b/arch/tile/mm/init.c
index a0582b7..adce254 100644
--- a/arch/tile/mm/init.c
+++ b/arch/tile/mm/init.c
@@ -679,7 +679,7 @@
* Hacky direct set to avoid unnecessary
* lock take/release for EVERY page here.
*/
- p->_count.counter = 0;
+ p->_refcount.counter = 0;
p->_mapcount.counter = -1;
}
init_page_count(page);
diff --git a/arch/um/configs/i386_defconfig b/arch/um/configs/i386_defconfig
index a12bf68..5636221 100644
--- a/arch/um/configs/i386_defconfig
+++ b/arch/um/configs/i386_defconfig
@@ -17,7 +17,6 @@
CONFIG_CGROUP_DEVICE=y
CONFIG_CPUSETS=y
CONFIG_CGROUP_CPUACCT=y
-CONFIG_RESOURCE_COUNTERS=y
CONFIG_CGROUP_SCHED=y
CONFIG_BLK_CGROUP=y
# CONFIG_PID_NS is not set
diff --git a/arch/um/configs/x86_64_defconfig b/arch/um/configs/x86_64_defconfig
index 3aab117..7a67b7a 100644
--- a/arch/um/configs/x86_64_defconfig
+++ b/arch/um/configs/x86_64_defconfig
@@ -15,7 +15,6 @@
CONFIG_CGROUP_DEVICE=y
CONFIG_CPUSETS=y
CONFIG_CGROUP_CPUACCT=y
-CONFIG_RESOURCE_COUNTERS=y
CONFIG_CGROUP_SCHED=y
CONFIG_BLK_CGROUP=y
# CONFIG_PID_NS is not set
diff --git a/arch/um/kernel/process.c b/arch/um/kernel/process.c
index 48af59a..0b04711 100644
--- a/arch/um/kernel/process.c
+++ b/arch/um/kernel/process.c
@@ -103,10 +103,6 @@
tracehook_notify_resume(regs);
}
-void exit_thread(void)
-{
-}
-
int get_current_pid(void)
{
return task_pid_nr(current);
diff --git a/arch/unicore32/include/uapi/asm/unistd.h b/arch/unicore32/include/uapi/asm/unistd.h
index d4cc455..1f63c47 100644
--- a/arch/unicore32/include/uapi/asm/unistd.h
+++ b/arch/unicore32/include/uapi/asm/unistd.h
@@ -10,6 +10,8 @@
* published by the Free Software Foundation.
*/
+#define __ARCH_WANT_RENAMEAT
+
/* Use the standard ABI for syscalls. */
#include <asm-generic/unistd.h>
#define __ARCH_WANT_SYS_CLONE
diff --git a/arch/unicore32/kernel/process.c b/arch/unicore32/kernel/process.c
index b008e99..00299c9 100644
--- a/arch/unicore32/kernel/process.c
+++ b/arch/unicore32/kernel/process.c
@@ -201,13 +201,6 @@
__backtrace();
}
-/*
- * Free current thread data structures etc..
- */
-void exit_thread(void)
-{
-}
-
void flush_thread(void)
{
struct thread_info *thread = current_thread_info();
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index ace79d2..0a7b885 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -105,6 +105,7 @@
select HAVE_DYNAMIC_FTRACE
select HAVE_DYNAMIC_FTRACE_WITH_REGS
select HAVE_EFFICIENT_UNALIGNED_ACCESS
+ select HAVE_EXIT_THREAD
select HAVE_FENTRY if X86_64
select HAVE_FTRACE_MCOUNT_RECORD
select HAVE_FUNCTION_GRAPH_FP_TEST
@@ -130,6 +131,7 @@
select HAVE_MEMBLOCK
select HAVE_MEMBLOCK_NODE_MAP
select HAVE_MIXED_BREAKPOINTS_REGS
+ select HAVE_NMI
select HAVE_OPROFILE
select HAVE_OPTPROBES
select HAVE_PCSPKR_PLATFORM
diff --git a/arch/x86/configs/i386_defconfig b/arch/x86/configs/i386_defconfig
index 265901a..5fa6ee2 100644
--- a/arch/x86/configs/i386_defconfig
+++ b/arch/x86/configs/i386_defconfig
@@ -17,7 +17,6 @@
CONFIG_CGROUP_FREEZER=y
CONFIG_CPUSETS=y
CONFIG_CGROUP_CPUACCT=y
-CONFIG_RESOURCE_COUNTERS=y
CONFIG_CGROUP_SCHED=y
CONFIG_BLK_DEV_INITRD=y
# CONFIG_COMPAT_BRK is not set
diff --git a/arch/x86/configs/x86_64_defconfig b/arch/x86/configs/x86_64_defconfig
index 0c8d796..d28bdab 100644
--- a/arch/x86/configs/x86_64_defconfig
+++ b/arch/x86/configs/x86_64_defconfig
@@ -16,7 +16,6 @@
CONFIG_CGROUP_FREEZER=y
CONFIG_CPUSETS=y
CONFIG_CGROUP_CPUACCT=y
-CONFIG_RESOURCE_COUNTERS=y
CONFIG_CGROUP_SCHED=y
CONFIG_BLK_DEV_INITRD=y
# CONFIG_COMPAT_BRK is not set
diff --git a/arch/x86/crypto/sha-mb/sha1_x8_avx2.S b/arch/x86/crypto/sha-mb/sha1_x8_avx2.S
index 8e1b477..c9dae1c 100644
--- a/arch/x86/crypto/sha-mb/sha1_x8_avx2.S
+++ b/arch/x86/crypto/sha-mb/sha1_x8_avx2.S
@@ -296,7 +296,11 @@
#
ENTRY(sha1_x8_avx2)
- push RSP_SAVE
+ # save callee-saved clobbered registers to comply with C function ABI
+ push %r12
+ push %r13
+ push %r14
+ push %r15
#save rsp
mov %rsp, RSP_SAVE
@@ -446,7 +450,12 @@
## Postamble
mov RSP_SAVE, %rsp
- pop RSP_SAVE
+
+ # restore callee-saved clobbered registers
+ pop %r15
+ pop %r14
+ pop %r13
+ pop %r12
ret
ENDPROC(sha1_x8_avx2)
diff --git a/arch/x86/entry/vdso/vma.c b/arch/x86/entry/vdso/vma.c
index b3cf813..ab220ac 100644
--- a/arch/x86/entry/vdso/vma.c
+++ b/arch/x86/entry/vdso/vma.c
@@ -163,7 +163,8 @@
addr = 0;
}
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
addr = get_unmapped_area(NULL, addr,
image->size - image->sym_vvar_start, 0, 0);
diff --git a/arch/x86/ia32/ia32_aout.c b/arch/x86/ia32/ia32_aout.c
index ae6aad1..f5e737f 100644
--- a/arch/x86/ia32/ia32_aout.c
+++ b/arch/x86/ia32/ia32_aout.c
@@ -116,13 +116,13 @@
.min_coredump = PAGE_SIZE
};
-static void set_brk(unsigned long start, unsigned long end)
+static unsigned long set_brk(unsigned long start, unsigned long end)
{
start = PAGE_ALIGN(start);
end = PAGE_ALIGN(end);
if (end <= start)
- return;
- vm_brk(start, end - start);
+ return start;
+ return vm_brk(start, end - start);
}
#ifdef CONFIG_COREDUMP
@@ -349,7 +349,10 @@
#endif
if (!bprm->file->f_op->mmap || (fd_offset & ~PAGE_MASK) != 0) {
- vm_brk(N_TXTADDR(ex), ex.a_text+ex.a_data);
+ error = vm_brk(N_TXTADDR(ex), ex.a_text+ex.a_data);
+ if (IS_ERR_VALUE(error))
+ return error;
+
read_code(bprm->file, N_TXTADDR(ex), fd_offset,
ex.a_text+ex.a_data);
goto beyond_if;
@@ -372,10 +375,13 @@
if (error != N_DATADDR(ex))
return error;
}
-beyond_if:
- set_binfmt(&aout_format);
- set_brk(current->mm->start_brk, current->mm->brk);
+beyond_if:
+ error = set_brk(current->mm->start_brk, current->mm->brk);
+ if (IS_ERR_VALUE(error))
+ return error;
+
+ set_binfmt(&aout_format);
current->mm->start_stack =
(unsigned long)create_aout_tables((char __user *)bprm->p, bprm);
@@ -434,7 +440,9 @@
error_time = jiffies;
}
#endif
- vm_brk(start_addr, ex.a_text + ex.a_data + ex.a_bss);
+ retval = vm_brk(start_addr, ex.a_text + ex.a_data + ex.a_bss);
+ if (IS_ERR_VALUE(retval))
+ goto out;
read_code(file, start_addr, N_TXTOFF(ex),
ex.a_text + ex.a_data);
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index b7e3944..e0fbe7e 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -562,7 +562,6 @@
struct {
u64 msr_val;
u64 last_steal;
- u64 accum_steal;
struct gfn_to_hva_cache stime;
struct kvm_steal_time steal;
} st;
@@ -774,6 +773,11 @@
u8 nr_reserved_ioapic_pins;
bool disabled_lapic_found;
+
+ /* Struct members for AVIC */
+ u32 ldr_mode;
+ struct page *avic_logical_id_table_page;
+ struct page *avic_physical_id_table_page;
};
struct kvm_vm_stat {
@@ -804,6 +808,7 @@
u32 halt_exits;
u32 halt_successful_poll;
u32 halt_attempted_poll;
+ u32 halt_poll_invalid;
u32 halt_wakeup;
u32 request_irq_exits;
u32 irq_exits;
@@ -848,6 +853,9 @@
bool (*cpu_has_high_real_mode_segbase)(void);
void (*cpuid_update)(struct kvm_vcpu *vcpu);
+ int (*vm_init)(struct kvm *kvm);
+ void (*vm_destroy)(struct kvm *kvm);
+
/* Create, but do not attach this VCPU */
struct kvm_vcpu *(*vcpu_create)(struct kvm *kvm, unsigned id);
void (*vcpu_free)(struct kvm_vcpu *vcpu);
@@ -914,7 +922,7 @@
bool (*get_enable_apicv)(void);
void (*refresh_apicv_exec_ctrl)(struct kvm_vcpu *vcpu);
void (*hwapic_irr_update)(struct kvm_vcpu *vcpu, int max_irr);
- void (*hwapic_isr_update)(struct kvm *kvm, int isr);
+ void (*hwapic_isr_update)(struct kvm_vcpu *vcpu, int isr);
void (*load_eoi_exitmap)(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap);
void (*set_virtual_x2apic_mode)(struct kvm_vcpu *vcpu, bool set);
void (*set_apic_access_page_addr)(struct kvm_vcpu *vcpu, hpa_t hpa);
@@ -990,8 +998,13 @@
*/
int (*pre_block)(struct kvm_vcpu *vcpu);
void (*post_block)(struct kvm_vcpu *vcpu);
+
+ void (*vcpu_blocking)(struct kvm_vcpu *vcpu);
+ void (*vcpu_unblocking)(struct kvm_vcpu *vcpu);
+
int (*update_pi_irte)(struct kvm *kvm, unsigned int host_irq,
uint32_t guest_irq, bool set);
+ void (*apicv_post_state_restore)(struct kvm_vcpu *vcpu);
};
struct kvm_arch_async_pf {
@@ -1341,7 +1354,18 @@
void kvm_set_msi_irq(struct kvm_kernel_irq_routing_entry *e,
struct kvm_lapic_irq *irq);
-static inline void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu) {}
-static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) {}
+static inline void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu)
+{
+ if (kvm_x86_ops->vcpu_blocking)
+ kvm_x86_ops->vcpu_blocking(vcpu);
+}
+
+static inline void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu)
+{
+ if (kvm_x86_ops->vcpu_unblocking)
+ kvm_x86_ops->vcpu_unblocking(vcpu);
+}
+
+static inline void kvm_arch_vcpu_block_finish(struct kvm_vcpu *vcpu) {}
#endif /* _ASM_X86_KVM_HOST_H */
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index f86491a..1a27396 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -181,6 +181,7 @@
return (pmd_val(pmd) & (_PAGE_PSE|_PAGE_DEVMAP)) == _PAGE_PSE;
}
+#define has_transparent_hugepage has_transparent_hugepage
static inline int has_transparent_hugepage(void)
{
return boot_cpu_has(X86_FEATURE_PSE);
diff --git a/arch/x86/include/asm/svm.h b/arch/x86/include/asm/svm.h
index 6136d99..d0fe23e 100644
--- a/arch/x86/include/asm/svm.h
+++ b/arch/x86/include/asm/svm.h
@@ -78,7 +78,8 @@
u32 exit_int_info;
u32 exit_int_info_err;
u64 nested_ctl;
- u8 reserved_4[16];
+ u64 avic_vapic_bar;
+ u8 reserved_4[8];
u32 event_inj;
u32 event_inj_err;
u64 nested_cr3;
@@ -88,7 +89,11 @@
u64 next_rip;
u8 insn_len;
u8 insn_bytes[15];
- u8 reserved_6[800];
+ u64 avic_backing_page; /* Offset 0xe0 */
+ u8 reserved_6[8]; /* Offset 0xe8 */
+ u64 avic_logical_id; /* Offset 0xf0 */
+ u64 avic_physical_id; /* Offset 0xf8 */
+ u8 reserved_7[768];
};
@@ -111,6 +116,9 @@
#define V_INTR_MASKING_SHIFT 24
#define V_INTR_MASKING_MASK (1 << V_INTR_MASKING_SHIFT)
+#define AVIC_ENABLE_SHIFT 31
+#define AVIC_ENABLE_MASK (1 << AVIC_ENABLE_SHIFT)
+
#define SVM_INTERRUPT_SHADOW_MASK 1
#define SVM_IOIO_STR_SHIFT 2
diff --git a/arch/x86/include/asm/uaccess.h b/arch/x86/include/asm/uaccess.h
index 12f9653..2982387 100644
--- a/arch/x86/include/asm/uaccess.h
+++ b/arch/x86/include/asm/uaccess.h
@@ -5,6 +5,7 @@
*/
#include <linux/errno.h>
#include <linux/compiler.h>
+#include <linux/kasan-checks.h>
#include <linux/thread_info.h>
#include <linux/string.h>
#include <asm/asm.h>
@@ -721,6 +722,8 @@
might_fault();
+ kasan_check_write(to, n);
+
/*
* While we would like to have the compiler do the checking for us
* even in the non-constant size case, any false positives there are
@@ -754,6 +757,8 @@
{
int sz = __compiletime_object_size(from);
+ kasan_check_read(from, n);
+
might_fault();
/* See the comment in copy_from_user() above. */
diff --git a/arch/x86/include/asm/uaccess_32.h b/arch/x86/include/asm/uaccess_32.h
index 3fe0eac..4b32da24 100644
--- a/arch/x86/include/asm/uaccess_32.h
+++ b/arch/x86/include/asm/uaccess_32.h
@@ -33,46 +33,10 @@
* the specified block with access_ok() before calling this function.
* The caller should also make sure he pins the user space address
* so that we don't result in page fault and sleep.
- *
- * Here we special-case 1, 2 and 4-byte copy_*_user invocations. On a fault
- * we return the initial request size (1, 2 or 4), as copy_*_user should do.
- * If a store crosses a page boundary and gets a fault, the x86 will not write
- * anything, so this is accurate.
*/
-
static __always_inline unsigned long __must_check
__copy_to_user_inatomic(void __user *to, const void *from, unsigned long n)
{
- if (__builtin_constant_p(n)) {
- unsigned long ret;
-
- switch (n) {
- case 1:
- __uaccess_begin();
- __put_user_size(*(u8 *)from, (u8 __user *)to,
- 1, ret, 1);
- __uaccess_end();
- return ret;
- case 2:
- __uaccess_begin();
- __put_user_size(*(u16 *)from, (u16 __user *)to,
- 2, ret, 2);
- __uaccess_end();
- return ret;
- case 4:
- __uaccess_begin();
- __put_user_size(*(u32 *)from, (u32 __user *)to,
- 4, ret, 4);
- __uaccess_end();
- return ret;
- case 8:
- __uaccess_begin();
- __put_user_size(*(u64 *)from, (u64 __user *)to,
- 8, ret, 8);
- __uaccess_end();
- return ret;
- }
- }
return __copy_to_user_ll(to, from, n);
}
@@ -101,32 +65,6 @@
static __always_inline unsigned long
__copy_from_user_inatomic(void *to, const void __user *from, unsigned long n)
{
- /* Avoid zeroing the tail if the copy fails..
- * If 'n' is constant and 1, 2, or 4, we do still zero on a failure,
- * but as the zeroing behaviour is only significant when n is not
- * constant, that shouldn't be a problem.
- */
- if (__builtin_constant_p(n)) {
- unsigned long ret;
-
- switch (n) {
- case 1:
- __uaccess_begin();
- __get_user_size(*(u8 *)to, from, 1, ret, 1);
- __uaccess_end();
- return ret;
- case 2:
- __uaccess_begin();
- __get_user_size(*(u16 *)to, from, 2, ret, 2);
- __uaccess_end();
- return ret;
- case 4:
- __uaccess_begin();
- __get_user_size(*(u32 *)to, from, 4, ret, 4);
- __uaccess_end();
- return ret;
- }
- }
return __copy_from_user_ll_nozero(to, from, n);
}
diff --git a/arch/x86/include/asm/uaccess_64.h b/arch/x86/include/asm/uaccess_64.h
index 3076986..2eac2aa 100644
--- a/arch/x86/include/asm/uaccess_64.h
+++ b/arch/x86/include/asm/uaccess_64.h
@@ -7,6 +7,7 @@
#include <linux/compiler.h>
#include <linux/errno.h>
#include <linux/lockdep.h>
+#include <linux/kasan-checks.h>
#include <asm/alternative.h>
#include <asm/cpufeatures.h>
#include <asm/page.h>
@@ -109,6 +110,7 @@
int __copy_from_user(void *dst, const void __user *src, unsigned size)
{
might_fault();
+ kasan_check_write(dst, size);
return __copy_from_user_nocheck(dst, src, size);
}
@@ -175,6 +177,7 @@
int __copy_to_user(void __user *dst, const void *src, unsigned size)
{
might_fault();
+ kasan_check_read(src, size);
return __copy_to_user_nocheck(dst, src, size);
}
@@ -242,12 +245,14 @@
static __must_check __always_inline int
__copy_from_user_inatomic(void *dst, const void __user *src, unsigned size)
{
+ kasan_check_write(dst, size);
return __copy_from_user_nocheck(dst, src, size);
}
static __must_check __always_inline int
__copy_to_user_inatomic(void __user *dst, const void *src, unsigned size)
{
+ kasan_check_read(src, size);
return __copy_to_user_nocheck(dst, src, size);
}
@@ -258,6 +263,7 @@
__copy_from_user_nocache(void *dst, const void __user *src, unsigned size)
{
might_fault();
+ kasan_check_write(dst, size);
return __copy_user_nocache(dst, src, size, 1);
}
@@ -265,6 +271,7 @@
__copy_from_user_inatomic_nocache(void *dst, const void __user *src,
unsigned size)
{
+ kasan_check_write(dst, size);
return __copy_user_nocache(dst, src, size, 0);
}
diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h
index cd54147..739c0c5 100644
--- a/arch/x86/include/uapi/asm/kvm.h
+++ b/arch/x86/include/uapi/asm/kvm.h
@@ -216,9 +216,9 @@
__u32 padding[3];
};
-#define KVM_CPUID_FLAG_SIGNIFCANT_INDEX BIT(0)
-#define KVM_CPUID_FLAG_STATEFUL_FUNC BIT(1)
-#define KVM_CPUID_FLAG_STATE_READ_NEXT BIT(2)
+#define KVM_CPUID_FLAG_SIGNIFCANT_INDEX (1 << 0)
+#define KVM_CPUID_FLAG_STATEFUL_FUNC (1 << 1)
+#define KVM_CPUID_FLAG_STATE_READ_NEXT (1 << 2)
/* for KVM_SET_CPUID2 */
struct kvm_cpuid2 {
diff --git a/arch/x86/include/uapi/asm/svm.h b/arch/x86/include/uapi/asm/svm.h
index 8a4add8..b9e9bb2 100644
--- a/arch/x86/include/uapi/asm/svm.h
+++ b/arch/x86/include/uapi/asm/svm.h
@@ -73,6 +73,8 @@
#define SVM_EXIT_MWAIT_COND 0x08c
#define SVM_EXIT_XSETBV 0x08d
#define SVM_EXIT_NPF 0x400
+#define SVM_EXIT_AVIC_INCOMPLETE_IPI 0x401
+#define SVM_EXIT_AVIC_UNACCELERATED_ACCESS 0x402
#define SVM_EXIT_ERR -1
@@ -107,8 +109,10 @@
{ SVM_EXIT_SMI, "smi" }, \
{ SVM_EXIT_INIT, "init" }, \
{ SVM_EXIT_VINTR, "vintr" }, \
+ { SVM_EXIT_CR0_SEL_WRITE, "cr0_sel_write" }, \
{ SVM_EXIT_CPUID, "cpuid" }, \
{ SVM_EXIT_INVD, "invd" }, \
+ { SVM_EXIT_PAUSE, "pause" }, \
{ SVM_EXIT_HLT, "hlt" }, \
{ SVM_EXIT_INVLPG, "invlpg" }, \
{ SVM_EXIT_INVLPGA, "invlpga" }, \
@@ -127,7 +131,10 @@
{ SVM_EXIT_MONITOR, "monitor" }, \
{ SVM_EXIT_MWAIT, "mwait" }, \
{ SVM_EXIT_XSETBV, "xsetbv" }, \
- { SVM_EXIT_NPF, "npf" }
+ { SVM_EXIT_NPF, "npf" }, \
+ { SVM_EXIT_RSM, "rsm" }, \
+ { SVM_EXIT_AVIC_INCOMPLETE_IPI, "avic_incomplete_ipi" }, \
+ { SVM_EXIT_AVIC_UNACCELERATED_ACCESS, "avic_unaccelerated_access" }
#endif /* _UAPI__SVM_H */
diff --git a/arch/x86/kernel/apic/hw_nmi.c b/arch/x86/kernel/apic/hw_nmi.c
index 045e424..7788ce6 100644
--- a/arch/x86/kernel/apic/hw_nmi.c
+++ b/arch/x86/kernel/apic/hw_nmi.c
@@ -18,7 +18,6 @@
#include <linux/nmi.h>
#include <linux/module.h>
#include <linux/delay.h>
-#include <linux/seq_buf.h>
#ifdef CONFIG_HARDLOCKUP_DETECTOR
u64 hw_nmi_get_sample_period(int watchdog_thresh)
diff --git a/arch/x86/kernel/kexec-bzimage64.c b/arch/x86/kernel/kexec-bzimage64.c
index 2af478e..f2356bd 100644
--- a/arch/x86/kernel/kexec-bzimage64.c
+++ b/arch/x86/kernel/kexec-bzimage64.c
@@ -19,8 +19,7 @@
#include <linux/kernel.h>
#include <linux/mm.h>
#include <linux/efi.h>
-#include <linux/verify_pefile.h>
-#include <keys/system_keyring.h>
+#include <linux/verification.h>
#include <asm/bootparam.h>
#include <asm/setup.h>
@@ -529,18 +528,9 @@
#ifdef CONFIG_KEXEC_BZIMAGE_VERIFY_SIG
static int bzImage64_verify_sig(const char *kernel, unsigned long kernel_len)
{
- bool trusted;
- int ret;
-
- ret = verify_pefile_signature(kernel, kernel_len,
- system_trusted_keyring,
- VERIFYING_KEXEC_PE_SIGNATURE,
- &trusted);
- if (ret < 0)
- return ret;
- if (!trusted)
- return -EKEYREJECTED;
- return 0;
+ return verify_pefile_signature(kernel, kernel_len,
+ NULL,
+ VERIFYING_KEXEC_PE_SIGNATURE);
}
#endif
diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c
index ba7fbba..5a294e4 100644
--- a/arch/x86/kernel/machine_kexec_64.c
+++ b/arch/x86/kernel/machine_kexec_64.c
@@ -538,3 +538,48 @@
return -ENOEXEC;
}
#endif /* CONFIG_KEXEC_FILE */
+
+static int
+kexec_mark_range(unsigned long start, unsigned long end, bool protect)
+{
+ struct page *page;
+ unsigned int nr_pages;
+
+ /*
+ * For physical range: [start, end]. We must skip the unassigned
+ * crashk resource with zero-valued "end" member.
+ */
+ if (!end || start > end)
+ return 0;
+
+ page = pfn_to_page(start >> PAGE_SHIFT);
+ nr_pages = (end >> PAGE_SHIFT) - (start >> PAGE_SHIFT) + 1;
+ if (protect)
+ return set_pages_ro(page, nr_pages);
+ else
+ return set_pages_rw(page, nr_pages);
+}
+
+static void kexec_mark_crashkres(bool protect)
+{
+ unsigned long control;
+
+ kexec_mark_range(crashk_low_res.start, crashk_low_res.end, protect);
+
+ /* Don't touch the control code page used in crash_kexec().*/
+ control = PFN_PHYS(page_to_pfn(kexec_crash_image->control_code_page));
+ /* Control code page is located in the 2nd page. */
+ kexec_mark_range(crashk_res.start, control + PAGE_SIZE - 1, protect);
+ control += KEXEC_CONTROL_PAGE_SIZE;
+ kexec_mark_range(control, crashk_res.end, protect);
+}
+
+void arch_kexec_protect_crashkres(void)
+{
+ kexec_mark_crashkres(true);
+}
+
+void arch_kexec_unprotect_crashkres(void)
+{
+ kexec_mark_crashkres(false);
+}
diff --git a/arch/x86/kernel/mcount_64.S b/arch/x86/kernel/mcount_64.S
index ed48a9f..6192422 100644
--- a/arch/x86/kernel/mcount_64.S
+++ b/arch/x86/kernel/mcount_64.S
@@ -182,7 +182,8 @@
jmp ftrace_stub
#endif
-GLOBAL(ftrace_stub)
+/* This is weak to keep gas from relaxing the jumps */
+WEAK(ftrace_stub)
retq
END(ftrace_caller)
diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
index 2915d54..96becbb 100644
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -97,10 +97,9 @@
/*
* Free current thread data structures etc..
*/
-void exit_thread(void)
+void exit_thread(struct task_struct *tsk)
{
- struct task_struct *me = current;
- struct thread_struct *t = &me->thread;
+ struct thread_struct *t = &tsk->thread;
unsigned long *bp = t->io_bitmap_ptr;
struct fpu *fpu = &t->fpu;
diff --git a/arch/x86/kvm/ioapic.c b/arch/x86/kvm/ioapic.c
index 9db47090..5f42d03 100644
--- a/arch/x86/kvm/ioapic.c
+++ b/arch/x86/kvm/ioapic.c
@@ -443,7 +443,7 @@
spin_lock(&ioapic->lock);
if (trigger_mode != IOAPIC_LEVEL_TRIG ||
- kvm_apic_get_reg(apic, APIC_SPIV) & APIC_SPIV_DIRECTED_EOI)
+ kvm_lapic_get_reg(apic, APIC_SPIV) & APIC_SPIV_DIRECTED_EOI)
continue;
ASSERT(ent->fields.trig_mode == IOAPIC_LEVEL_TRIG);
diff --git a/arch/x86/kvm/irq_comm.c b/arch/x86/kvm/irq_comm.c
index 54ead79..dfb4c64 100644
--- a/arch/x86/kvm/irq_comm.c
+++ b/arch/x86/kvm/irq_comm.c
@@ -382,9 +382,6 @@
u32 i, nr_ioapic_pins;
int idx;
- /* kvm->irq_routing must be read after clearing
- * KVM_SCAN_IOAPIC. */
- smp_mb();
idx = srcu_read_lock(&kvm->irq_srcu);
table = srcu_dereference(kvm->irq_routing, &kvm->irq_srcu);
nr_ioapic_pins = min_t(u32, table->nr_rt_entries,
diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index 1a2da0e..bbb5b28 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -59,9 +59,8 @@
/* #define apic_debug(fmt,arg...) printk(KERN_WARNING fmt,##arg) */
#define apic_debug(fmt, arg...)
-#define APIC_LVT_NUM 6
/* 14 is the version for Xeon and Pentium 8.4.8*/
-#define APIC_VERSION (0x14UL | ((APIC_LVT_NUM - 1) << 16))
+#define APIC_VERSION (0x14UL | ((KVM_APIC_LVT_NUM - 1) << 16))
#define LAPIC_MMIO_LENGTH (1 << 12)
/* followed define is not in apicdef.h */
#define APIC_SHORT_MASK 0xc0000
@@ -73,14 +72,6 @@
#define APIC_BROADCAST 0xFF
#define X2APIC_BROADCAST 0xFFFFFFFFul
-#define VEC_POS(v) ((v) & (32 - 1))
-#define REG_POS(v) (((v) >> 5) << 4)
-
-static inline void apic_set_reg(struct kvm_lapic *apic, int reg_off, u32 val)
-{
- *((u32 *) (apic->regs + reg_off)) = val;
-}
-
static inline int apic_test_vector(int vec, void *bitmap)
{
return test_bit(VEC_POS(vec), (bitmap) + REG_POS(vec));
@@ -94,11 +85,6 @@
apic_test_vector(vector, apic->regs + APIC_IRR);
}
-static inline void apic_set_vector(int vec, void *bitmap)
-{
- set_bit(VEC_POS(vec), (bitmap) + REG_POS(vec));
-}
-
static inline void apic_clear_vector(int vec, void *bitmap)
{
clear_bit(VEC_POS(vec), (bitmap) + REG_POS(vec));
@@ -173,7 +159,7 @@
continue;
aid = kvm_apic_id(apic);
- ldr = kvm_apic_get_reg(apic, APIC_LDR);
+ ldr = kvm_lapic_get_reg(apic, APIC_LDR);
if (aid < ARRAY_SIZE(new->phys_map))
new->phys_map[aid] = apic;
@@ -182,7 +168,7 @@
new->mode |= KVM_APIC_MODE_X2APIC;
} else if (ldr) {
ldr = GET_APIC_LOGICAL_ID(ldr);
- if (kvm_apic_get_reg(apic, APIC_DFR) == APIC_DFR_FLAT)
+ if (kvm_lapic_get_reg(apic, APIC_DFR) == APIC_DFR_FLAT)
new->mode |= KVM_APIC_MODE_XAPIC_FLAT;
else
new->mode |= KVM_APIC_MODE_XAPIC_CLUSTER;
@@ -212,7 +198,7 @@
{
bool enabled = val & APIC_SPIV_APIC_ENABLED;
- apic_set_reg(apic, APIC_SPIV, val);
+ kvm_lapic_set_reg(apic, APIC_SPIV, val);
if (enabled != apic->sw_enabled) {
apic->sw_enabled = enabled;
@@ -226,13 +212,13 @@
static inline void kvm_apic_set_id(struct kvm_lapic *apic, u8 id)
{
- apic_set_reg(apic, APIC_ID, id << 24);
+ kvm_lapic_set_reg(apic, APIC_ID, id << 24);
recalculate_apic_map(apic->vcpu->kvm);
}
static inline void kvm_apic_set_ldr(struct kvm_lapic *apic, u32 id)
{
- apic_set_reg(apic, APIC_LDR, id);
+ kvm_lapic_set_reg(apic, APIC_LDR, id);
recalculate_apic_map(apic->vcpu->kvm);
}
@@ -240,19 +226,19 @@
{
u32 ldr = ((id >> 4) << 16) | (1 << (id & 0xf));
- apic_set_reg(apic, APIC_ID, id << 24);
- apic_set_reg(apic, APIC_LDR, ldr);
+ kvm_lapic_set_reg(apic, APIC_ID, id << 24);
+ kvm_lapic_set_reg(apic, APIC_LDR, ldr);
recalculate_apic_map(apic->vcpu->kvm);
}
static inline int apic_lvt_enabled(struct kvm_lapic *apic, int lvt_type)
{
- return !(kvm_apic_get_reg(apic, lvt_type) & APIC_LVT_MASKED);
+ return !(kvm_lapic_get_reg(apic, lvt_type) & APIC_LVT_MASKED);
}
static inline int apic_lvt_vector(struct kvm_lapic *apic, int lvt_type)
{
- return kvm_apic_get_reg(apic, lvt_type) & APIC_VECTOR_MASK;
+ return kvm_lapic_get_reg(apic, lvt_type) & APIC_VECTOR_MASK;
}
static inline int apic_lvtt_oneshot(struct kvm_lapic *apic)
@@ -287,10 +273,10 @@
feat = kvm_find_cpuid_entry(apic->vcpu, 0x1, 0);
if (feat && (feat->ecx & (1 << (X86_FEATURE_X2APIC & 31))))
v |= APIC_LVR_DIRECTED_EOI;
- apic_set_reg(apic, APIC_LVR, v);
+ kvm_lapic_set_reg(apic, APIC_LVR, v);
}
-static const unsigned int apic_lvt_mask[APIC_LVT_NUM] = {
+static const unsigned int apic_lvt_mask[KVM_APIC_LVT_NUM] = {
LVT_MASK , /* part LVTT mask, timer mode mask added at runtime */
LVT_MASK | APIC_MODE_MASK, /* LVTTHMR */
LVT_MASK | APIC_MODE_MASK, /* LVTPC */
@@ -349,16 +335,6 @@
}
EXPORT_SYMBOL_GPL(kvm_apic_update_irr);
-static inline void apic_set_irr(int vec, struct kvm_lapic *apic)
-{
- apic_set_vector(vec, apic->regs + APIC_IRR);
- /*
- * irr_pending must be true if any interrupt is pending; set it after
- * APIC_IRR to avoid race with apic_clear_irr
- */
- apic->irr_pending = true;
-}
-
static inline int apic_search_irr(struct kvm_lapic *apic)
{
return find_highest_vector(apic->regs + APIC_IRR);
@@ -416,7 +392,7 @@
* just set SVI.
*/
if (unlikely(vcpu->arch.apicv_active))
- kvm_x86_ops->hwapic_isr_update(vcpu->kvm, vec);
+ kvm_x86_ops->hwapic_isr_update(vcpu, vec);
else {
++apic->isr_count;
BUG_ON(apic->isr_count > MAX_APIC_VECTOR);
@@ -464,7 +440,7 @@
* and must be left alone.
*/
if (unlikely(vcpu->arch.apicv_active))
- kvm_x86_ops->hwapic_isr_update(vcpu->kvm,
+ kvm_x86_ops->hwapic_isr_update(vcpu,
apic_find_highest_isr(apic));
else {
--apic->isr_count;
@@ -549,8 +525,8 @@
u32 tpr, isrv, ppr, old_ppr;
int isr;
- old_ppr = kvm_apic_get_reg(apic, APIC_PROCPRI);
- tpr = kvm_apic_get_reg(apic, APIC_TASKPRI);
+ old_ppr = kvm_lapic_get_reg(apic, APIC_PROCPRI);
+ tpr = kvm_lapic_get_reg(apic, APIC_TASKPRI);
isr = apic_find_highest_isr(apic);
isrv = (isr != -1) ? isr : 0;
@@ -563,7 +539,7 @@
apic, ppr, isr, isrv);
if (old_ppr != ppr) {
- apic_set_reg(apic, APIC_PROCPRI, ppr);
+ kvm_lapic_set_reg(apic, APIC_PROCPRI, ppr);
if (ppr < old_ppr)
kvm_make_request(KVM_REQ_EVENT, apic->vcpu);
}
@@ -571,7 +547,7 @@
static void apic_set_tpr(struct kvm_lapic *apic, u32 tpr)
{
- apic_set_reg(apic, APIC_TASKPRI, tpr);
+ kvm_lapic_set_reg(apic, APIC_TASKPRI, tpr);
apic_update_ppr(apic);
}
@@ -601,7 +577,7 @@
if (kvm_apic_broadcast(apic, mda))
return true;
- logical_id = kvm_apic_get_reg(apic, APIC_LDR);
+ logical_id = kvm_lapic_get_reg(apic, APIC_LDR);
if (apic_x2apic_mode(apic))
return ((logical_id >> 16) == (mda >> 16))
@@ -610,7 +586,7 @@
logical_id = GET_APIC_LOGICAL_ID(logical_id);
mda = GET_APIC_DEST_FIELD(mda);
- switch (kvm_apic_get_reg(apic, APIC_DFR)) {
+ switch (kvm_lapic_get_reg(apic, APIC_DFR)) {
case APIC_DFR_FLAT:
return (logical_id & mda) != 0;
case APIC_DFR_CLUSTER:
@@ -618,7 +594,7 @@
&& (logical_id & mda & 0xf) != 0;
default:
apic_debug("Bad DFR vcpu %d: %08x\n",
- apic->vcpu->vcpu_id, kvm_apic_get_reg(apic, APIC_DFR));
+ apic->vcpu->vcpu_id, kvm_lapic_get_reg(apic, APIC_DFR));
return false;
}
}
@@ -668,6 +644,7 @@
return false;
}
}
+EXPORT_SYMBOL_GPL(kvm_apic_match_dest);
int kvm_vector_to_index(u32 vector, u32 dest_vcpus,
const unsigned long *bitmap, u32 bitmap_size)
@@ -921,7 +898,7 @@
if (apic_test_vector(vector, apic->regs + APIC_TMR) != !!trig_mode) {
if (trig_mode)
- apic_set_vector(vector, apic->regs + APIC_TMR);
+ kvm_lapic_set_vector(vector, apic->regs + APIC_TMR);
else
apic_clear_vector(vector, apic->regs + APIC_TMR);
}
@@ -929,7 +906,7 @@
if (vcpu->arch.apicv_active)
kvm_x86_ops->deliver_posted_interrupt(vcpu, vector);
else {
- apic_set_irr(vector, apic);
+ kvm_lapic_set_irr(vector, apic);
kvm_make_request(KVM_REQ_EVENT, vcpu);
kvm_vcpu_kick(vcpu);
@@ -1073,8 +1050,8 @@
static void apic_send_ipi(struct kvm_lapic *apic)
{
- u32 icr_low = kvm_apic_get_reg(apic, APIC_ICR);
- u32 icr_high = kvm_apic_get_reg(apic, APIC_ICR2);
+ u32 icr_low = kvm_lapic_get_reg(apic, APIC_ICR);
+ u32 icr_high = kvm_lapic_get_reg(apic, APIC_ICR2);
struct kvm_lapic_irq irq;
irq.vector = icr_low & APIC_VECTOR_MASK;
@@ -1111,7 +1088,7 @@
ASSERT(apic != NULL);
/* if initial count is 0, current count should also be 0 */
- if (kvm_apic_get_reg(apic, APIC_TMICT) == 0 ||
+ if (kvm_lapic_get_reg(apic, APIC_TMICT) == 0 ||
apic->lapic_timer.period == 0)
return 0;
@@ -1168,13 +1145,13 @@
break;
case APIC_PROCPRI:
apic_update_ppr(apic);
- val = kvm_apic_get_reg(apic, offset);
+ val = kvm_lapic_get_reg(apic, offset);
break;
case APIC_TASKPRI:
report_tpr_access(apic, false);
/* fall thru */
default:
- val = kvm_apic_get_reg(apic, offset);
+ val = kvm_lapic_get_reg(apic, offset);
break;
}
@@ -1186,7 +1163,7 @@
return container_of(dev, struct kvm_lapic, dev);
}
-static int apic_reg_read(struct kvm_lapic *apic, u32 offset, int len,
+int kvm_lapic_reg_read(struct kvm_lapic *apic, u32 offset, int len,
void *data)
{
unsigned char alignment = offset & 0xf;
@@ -1223,6 +1200,7 @@
}
return 0;
}
+EXPORT_SYMBOL_GPL(kvm_lapic_reg_read);
static int apic_mmio_in_range(struct kvm_lapic *apic, gpa_t addr)
{
@@ -1240,7 +1218,7 @@
if (!apic_mmio_in_range(apic, address))
return -EOPNOTSUPP;
- apic_reg_read(apic, offset, len, data);
+ kvm_lapic_reg_read(apic, offset, len, data);
return 0;
}
@@ -1249,7 +1227,7 @@
{
u32 tmp1, tmp2, tdcr;
- tdcr = kvm_apic_get_reg(apic, APIC_TDCR);
+ tdcr = kvm_lapic_get_reg(apic, APIC_TDCR);
tmp1 = tdcr & 0xf;
tmp2 = ((tmp1 & 0x3) | ((tmp1 & 0x8) >> 1)) + 1;
apic->divide_count = 0x1 << (tmp2 & 0x7);
@@ -1260,7 +1238,7 @@
static void apic_update_lvtt(struct kvm_lapic *apic)
{
- u32 timer_mode = kvm_apic_get_reg(apic, APIC_LVTT) &
+ u32 timer_mode = kvm_lapic_get_reg(apic, APIC_LVTT) &
apic->lapic_timer.timer_mode_mask;
if (apic->lapic_timer.timer_mode != timer_mode) {
@@ -1296,7 +1274,7 @@
static bool lapic_timer_int_injected(struct kvm_vcpu *vcpu)
{
struct kvm_lapic *apic = vcpu->arch.apic;
- u32 reg = kvm_apic_get_reg(apic, APIC_LVTT);
+ u32 reg = kvm_lapic_get_reg(apic, APIC_LVTT);
if (kvm_apic_hw_enabled(apic)) {
int vec = reg & APIC_VECTOR_MASK;
@@ -1344,7 +1322,7 @@
if (apic_lvtt_period(apic) || apic_lvtt_oneshot(apic)) {
/* lapic timer in oneshot or periodic mode */
now = apic->lapic_timer.timer.base->get_time();
- apic->lapic_timer.period = (u64)kvm_apic_get_reg(apic, APIC_TMICT)
+ apic->lapic_timer.period = (u64)kvm_lapic_get_reg(apic, APIC_TMICT)
* APIC_BUS_CYCLE_NS * apic->divide_count;
if (!apic->lapic_timer.period)
@@ -1376,7 +1354,7 @@
"timer initial count 0x%x, period %lldns, "
"expire @ 0x%016" PRIx64 ".\n", __func__,
APIC_BUS_CYCLE_NS, ktime_to_ns(now),
- kvm_apic_get_reg(apic, APIC_TMICT),
+ kvm_lapic_get_reg(apic, APIC_TMICT),
apic->lapic_timer.period,
ktime_to_ns(ktime_add_ns(now,
apic->lapic_timer.period)));
@@ -1425,7 +1403,7 @@
}
}
-static int apic_reg_write(struct kvm_lapic *apic, u32 reg, u32 val)
+int kvm_lapic_reg_write(struct kvm_lapic *apic, u32 reg, u32 val)
{
int ret = 0;
@@ -1457,7 +1435,7 @@
case APIC_DFR:
if (!apic_x2apic_mode(apic)) {
- apic_set_reg(apic, APIC_DFR, val | 0x0FFFFFFF);
+ kvm_lapic_set_reg(apic, APIC_DFR, val | 0x0FFFFFFF);
recalculate_apic_map(apic->vcpu->kvm);
} else
ret = 1;
@@ -1465,17 +1443,17 @@
case APIC_SPIV: {
u32 mask = 0x3ff;
- if (kvm_apic_get_reg(apic, APIC_LVR) & APIC_LVR_DIRECTED_EOI)
+ if (kvm_lapic_get_reg(apic, APIC_LVR) & APIC_LVR_DIRECTED_EOI)
mask |= APIC_SPIV_DIRECTED_EOI;
apic_set_spiv(apic, val & mask);
if (!(val & APIC_SPIV_APIC_ENABLED)) {
int i;
u32 lvt_val;
- for (i = 0; i < APIC_LVT_NUM; i++) {
- lvt_val = kvm_apic_get_reg(apic,
+ for (i = 0; i < KVM_APIC_LVT_NUM; i++) {
+ lvt_val = kvm_lapic_get_reg(apic,
APIC_LVTT + 0x10 * i);
- apic_set_reg(apic, APIC_LVTT + 0x10 * i,
+ kvm_lapic_set_reg(apic, APIC_LVTT + 0x10 * i,
lvt_val | APIC_LVT_MASKED);
}
apic_update_lvtt(apic);
@@ -1486,14 +1464,14 @@
}
case APIC_ICR:
/* No delay here, so we always clear the pending bit */
- apic_set_reg(apic, APIC_ICR, val & ~(1 << 12));
+ kvm_lapic_set_reg(apic, APIC_ICR, val & ~(1 << 12));
apic_send_ipi(apic);
break;
case APIC_ICR2:
if (!apic_x2apic_mode(apic))
val &= 0xff000000;
- apic_set_reg(apic, APIC_ICR2, val);
+ kvm_lapic_set_reg(apic, APIC_ICR2, val);
break;
case APIC_LVT0:
@@ -1507,7 +1485,7 @@
val |= APIC_LVT_MASKED;
val &= apic_lvt_mask[(reg - APIC_LVTT) >> 4];
- apic_set_reg(apic, reg, val);
+ kvm_lapic_set_reg(apic, reg, val);
break;
@@ -1515,7 +1493,7 @@
if (!kvm_apic_sw_enabled(apic))
val |= APIC_LVT_MASKED;
val &= (apic_lvt_mask[0] | apic->lapic_timer.timer_mode_mask);
- apic_set_reg(apic, APIC_LVTT, val);
+ kvm_lapic_set_reg(apic, APIC_LVTT, val);
apic_update_lvtt(apic);
break;
@@ -1524,14 +1502,14 @@
break;
hrtimer_cancel(&apic->lapic_timer.timer);
- apic_set_reg(apic, APIC_TMICT, val);
+ kvm_lapic_set_reg(apic, APIC_TMICT, val);
start_apic_timer(apic);
break;
case APIC_TDCR:
if (val & 4)
apic_debug("KVM_WRITE:TDCR %x\n", val);
- apic_set_reg(apic, APIC_TDCR, val);
+ kvm_lapic_set_reg(apic, APIC_TDCR, val);
update_divide_count(apic);
break;
@@ -1544,7 +1522,7 @@
case APIC_SELF_IPI:
if (apic_x2apic_mode(apic)) {
- apic_reg_write(apic, APIC_ICR, 0x40000 | (val & 0xff));
+ kvm_lapic_reg_write(apic, APIC_ICR, 0x40000 | (val & 0xff));
} else
ret = 1;
break;
@@ -1556,6 +1534,7 @@
apic_debug("Local APIC Write to read-only register %x\n", reg);
return ret;
}
+EXPORT_SYMBOL_GPL(kvm_lapic_reg_write);
static int apic_mmio_write(struct kvm_vcpu *vcpu, struct kvm_io_device *this,
gpa_t address, int len, const void *data)
@@ -1585,14 +1564,14 @@
apic_debug("%s: offset 0x%x with length 0x%x, and value is "
"0x%x\n", __func__, offset, len, val);
- apic_reg_write(apic, offset & 0xff0, val);
+ kvm_lapic_reg_write(apic, offset & 0xff0, val);
return 0;
}
void kvm_lapic_set_eoi(struct kvm_vcpu *vcpu)
{
- apic_reg_write(vcpu->arch.apic, APIC_EOI, 0);
+ kvm_lapic_reg_write(vcpu->arch.apic, APIC_EOI, 0);
}
EXPORT_SYMBOL_GPL(kvm_lapic_set_eoi);
@@ -1604,10 +1583,10 @@
/* hw has done the conditional check and inst decode */
offset &= 0xff0;
- apic_reg_read(vcpu->arch.apic, offset, 4, &val);
+ kvm_lapic_reg_read(vcpu->arch.apic, offset, 4, &val);
/* TODO: optimize to just emulate side effect w/o one more write */
- apic_reg_write(vcpu->arch.apic, offset, val);
+ kvm_lapic_reg_write(vcpu->arch.apic, offset, val);
}
EXPORT_SYMBOL_GPL(kvm_apic_write_nodecode);
@@ -1667,14 +1646,14 @@
struct kvm_lapic *apic = vcpu->arch.apic;
apic_set_tpr(apic, ((cr8 & 0x0f) << 4)
- | (kvm_apic_get_reg(apic, APIC_TASKPRI) & 4));
+ | (kvm_lapic_get_reg(apic, APIC_TASKPRI) & 4));
}
u64 kvm_lapic_get_cr8(struct kvm_vcpu *vcpu)
{
u64 tpr;
- tpr = (u64) kvm_apic_get_reg(vcpu->arch.apic, APIC_TASKPRI);
+ tpr = (u64) kvm_lapic_get_reg(vcpu->arch.apic, APIC_TASKPRI);
return (tpr & 0xf0) >> 4;
}
@@ -1740,28 +1719,28 @@
kvm_apic_set_id(apic, vcpu->vcpu_id);
kvm_apic_set_version(apic->vcpu);
- for (i = 0; i < APIC_LVT_NUM; i++)
- apic_set_reg(apic, APIC_LVTT + 0x10 * i, APIC_LVT_MASKED);
+ for (i = 0; i < KVM_APIC_LVT_NUM; i++)
+ kvm_lapic_set_reg(apic, APIC_LVTT + 0x10 * i, APIC_LVT_MASKED);
apic_update_lvtt(apic);
if (kvm_check_has_quirk(vcpu->kvm, KVM_X86_QUIRK_LINT0_REENABLED))
- apic_set_reg(apic, APIC_LVT0,
+ kvm_lapic_set_reg(apic, APIC_LVT0,
SET_APIC_DELIVERY_MODE(0, APIC_MODE_EXTINT));
- apic_manage_nmi_watchdog(apic, kvm_apic_get_reg(apic, APIC_LVT0));
+ apic_manage_nmi_watchdog(apic, kvm_lapic_get_reg(apic, APIC_LVT0));
- apic_set_reg(apic, APIC_DFR, 0xffffffffU);
+ kvm_lapic_set_reg(apic, APIC_DFR, 0xffffffffU);
apic_set_spiv(apic, 0xff);
- apic_set_reg(apic, APIC_TASKPRI, 0);
+ kvm_lapic_set_reg(apic, APIC_TASKPRI, 0);
if (!apic_x2apic_mode(apic))
kvm_apic_set_ldr(apic, 0);
- apic_set_reg(apic, APIC_ESR, 0);
- apic_set_reg(apic, APIC_ICR, 0);
- apic_set_reg(apic, APIC_ICR2, 0);
- apic_set_reg(apic, APIC_TDCR, 0);
- apic_set_reg(apic, APIC_TMICT, 0);
+ kvm_lapic_set_reg(apic, APIC_ESR, 0);
+ kvm_lapic_set_reg(apic, APIC_ICR, 0);
+ kvm_lapic_set_reg(apic, APIC_ICR2, 0);
+ kvm_lapic_set_reg(apic, APIC_TDCR, 0);
+ kvm_lapic_set_reg(apic, APIC_TMICT, 0);
for (i = 0; i < 8; i++) {
- apic_set_reg(apic, APIC_IRR + 0x10 * i, 0);
- apic_set_reg(apic, APIC_ISR + 0x10 * i, 0);
- apic_set_reg(apic, APIC_TMR + 0x10 * i, 0);
+ kvm_lapic_set_reg(apic, APIC_IRR + 0x10 * i, 0);
+ kvm_lapic_set_reg(apic, APIC_ISR + 0x10 * i, 0);
+ kvm_lapic_set_reg(apic, APIC_TMR + 0x10 * i, 0);
}
apic->irr_pending = vcpu->arch.apicv_active;
apic->isr_count = vcpu->arch.apicv_active ? 1 : 0;
@@ -1806,7 +1785,7 @@
int kvm_apic_local_deliver(struct kvm_lapic *apic, int lvt_type)
{
- u32 reg = kvm_apic_get_reg(apic, lvt_type);
+ u32 reg = kvm_lapic_get_reg(apic, lvt_type);
int vector, mode, trig_mode;
if (kvm_apic_hw_enabled(apic) && !(reg & APIC_LVT_MASKED)) {
@@ -1901,14 +1880,14 @@
apic_update_ppr(apic);
highest_irr = apic_find_highest_irr(apic);
if ((highest_irr == -1) ||
- ((highest_irr & 0xF0) <= kvm_apic_get_reg(apic, APIC_PROCPRI)))
+ ((highest_irr & 0xF0) <= kvm_lapic_get_reg(apic, APIC_PROCPRI)))
return -1;
return highest_irr;
}
int kvm_apic_accept_pic_intr(struct kvm_vcpu *vcpu)
{
- u32 lvt0 = kvm_apic_get_reg(vcpu->arch.apic, APIC_LVT0);
+ u32 lvt0 = kvm_lapic_get_reg(vcpu->arch.apic, APIC_LVT0);
int r = 0;
if (!kvm_apic_hw_enabled(vcpu->arch.apic))
@@ -1974,7 +1953,7 @@
apic_update_ppr(apic);
hrtimer_cancel(&apic->lapic_timer.timer);
apic_update_lvtt(apic);
- apic_manage_nmi_watchdog(apic, kvm_apic_get_reg(apic, APIC_LVT0));
+ apic_manage_nmi_watchdog(apic, kvm_lapic_get_reg(apic, APIC_LVT0));
update_divide_count(apic);
start_apic_timer(apic);
apic->irr_pending = true;
@@ -1982,9 +1961,11 @@
1 : count_vectors(apic->regs + APIC_ISR);
apic->highest_isr_cache = -1;
if (vcpu->arch.apicv_active) {
+ if (kvm_x86_ops->apicv_post_state_restore)
+ kvm_x86_ops->apicv_post_state_restore(vcpu);
kvm_x86_ops->hwapic_irr_update(vcpu,
apic_find_highest_irr(apic));
- kvm_x86_ops->hwapic_isr_update(vcpu->kvm,
+ kvm_x86_ops->hwapic_isr_update(vcpu,
apic_find_highest_isr(apic));
}
kvm_make_request(KVM_REQ_EVENT, vcpu);
@@ -2097,7 +2078,7 @@
if (!test_bit(KVM_APIC_CHECK_VAPIC, &vcpu->arch.apic_attention))
return;
- tpr = kvm_apic_get_reg(apic, APIC_TASKPRI) & 0xff;
+ tpr = kvm_lapic_get_reg(apic, APIC_TASKPRI) & 0xff;
max_irr = apic_find_highest_irr(apic);
if (max_irr < 0)
max_irr = 0;
@@ -2139,8 +2120,8 @@
/* if this is ICR write vector before command */
if (reg == APIC_ICR)
- apic_reg_write(apic, APIC_ICR2, (u32)(data >> 32));
- return apic_reg_write(apic, reg, (u32)data);
+ kvm_lapic_reg_write(apic, APIC_ICR2, (u32)(data >> 32));
+ return kvm_lapic_reg_write(apic, reg, (u32)data);
}
int kvm_x2apic_msr_read(struct kvm_vcpu *vcpu, u32 msr, u64 *data)
@@ -2157,10 +2138,10 @@
return 1;
}
- if (apic_reg_read(apic, reg, 4, &low))
+ if (kvm_lapic_reg_read(apic, reg, 4, &low))
return 1;
if (reg == APIC_ICR)
- apic_reg_read(apic, APIC_ICR2, 4, &high);
+ kvm_lapic_reg_read(apic, APIC_ICR2, 4, &high);
*data = (((u64)high) << 32) | low;
@@ -2176,8 +2157,8 @@
/* if this is ICR write vector before command */
if (reg == APIC_ICR)
- apic_reg_write(apic, APIC_ICR2, (u32)(data >> 32));
- return apic_reg_write(apic, reg, (u32)data);
+ kvm_lapic_reg_write(apic, APIC_ICR2, (u32)(data >> 32));
+ return kvm_lapic_reg_write(apic, reg, (u32)data);
}
int kvm_hv_vapic_msr_read(struct kvm_vcpu *vcpu, u32 reg, u64 *data)
@@ -2188,10 +2169,10 @@
if (!lapic_in_kernel(vcpu))
return 1;
- if (apic_reg_read(apic, reg, 4, &low))
+ if (kvm_lapic_reg_read(apic, reg, 4, &low))
return 1;
if (reg == APIC_ICR)
- apic_reg_read(apic, APIC_ICR2, 4, &high);
+ kvm_lapic_reg_read(apic, APIC_ICR2, 4, &high);
*data = (((u64)high) << 32) | low;
diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
index f71183e..891c6da 100644
--- a/arch/x86/kvm/lapic.h
+++ b/arch/x86/kvm/lapic.h
@@ -7,6 +7,10 @@
#define KVM_APIC_INIT 0
#define KVM_APIC_SIPI 1
+#define KVM_APIC_LVT_NUM 6
+
+#define KVM_APIC_SHORT_MASK 0xc0000
+#define KVM_APIC_DEST_MASK 0x800
struct kvm_timer {
struct hrtimer timer;
@@ -59,6 +63,11 @@
void kvm_lapic_set_base(struct kvm_vcpu *vcpu, u64 value);
u64 kvm_lapic_get_base(struct kvm_vcpu *vcpu);
void kvm_apic_set_version(struct kvm_vcpu *vcpu);
+int kvm_lapic_reg_write(struct kvm_lapic *apic, u32 reg, u32 val);
+int kvm_lapic_reg_read(struct kvm_lapic *apic, u32 offset, int len,
+ void *data);
+bool kvm_apic_match_dest(struct kvm_vcpu *vcpu, struct kvm_lapic *source,
+ int short_hand, unsigned int dest, int dest_mode);
void __kvm_apic_update_irr(u32 *pir, void *regs);
void kvm_apic_update_irr(struct kvm_vcpu *vcpu, u32 *pir);
@@ -99,9 +108,32 @@
int kvm_lapic_enable_pv_eoi(struct kvm_vcpu *vcpu, u64 data);
void kvm_lapic_init(void);
-static inline u32 kvm_apic_get_reg(struct kvm_lapic *apic, int reg_off)
+#define VEC_POS(v) ((v) & (32 - 1))
+#define REG_POS(v) (((v) >> 5) << 4)
+
+static inline void kvm_lapic_set_vector(int vec, void *bitmap)
{
- return *((u32 *) (apic->regs + reg_off));
+ set_bit(VEC_POS(vec), (bitmap) + REG_POS(vec));
+}
+
+static inline void kvm_lapic_set_irr(int vec, struct kvm_lapic *apic)
+{
+ kvm_lapic_set_vector(vec, apic->regs + APIC_IRR);
+ /*
+ * irr_pending must be true if any interrupt is pending; set it after
+ * APIC_IRR to avoid race with apic_clear_irr
+ */
+ apic->irr_pending = true;
+}
+
+static inline u32 kvm_lapic_get_reg(struct kvm_lapic *apic, int reg_off)
+{
+ return *((u32 *) (apic->regs + reg_off));
+}
+
+static inline void kvm_lapic_set_reg(struct kvm_lapic *apic, int reg_off, u32 val)
+{
+ *((u32 *) (apic->regs + reg_off)) = val;
}
extern struct static_key kvm_no_apic_vcpu;
@@ -169,7 +201,7 @@
static inline int kvm_apic_id(struct kvm_lapic *apic)
{
- return (kvm_apic_get_reg(apic, APIC_ID) >> 24) & 0xff;
+ return (kvm_lapic_get_reg(apic, APIC_ID) >> 24) & 0xff;
}
bool kvm_apic_pending_eoi(struct kvm_vcpu *vcpu, int vector);
diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
index 38c0c32..24e8001 100644
--- a/arch/x86/kvm/mmu.c
+++ b/arch/x86/kvm/mmu.c
@@ -1909,18 +1909,17 @@
* since it has been deleted from active_mmu_pages but still can be found
* at hast list.
*
- * for_each_gfn_indirect_valid_sp has skipped that kind of page and
- * kvm_mmu_get_page(), the only user of for_each_gfn_sp(), has skipped
- * all the obsolete pages.
+ * for_each_gfn_valid_sp() has skipped that kind of pages.
*/
-#define for_each_gfn_sp(_kvm, _sp, _gfn) \
+#define for_each_gfn_valid_sp(_kvm, _sp, _gfn) \
hlist_for_each_entry(_sp, \
&(_kvm)->arch.mmu_page_hash[kvm_page_table_hashfn(_gfn)], hash_link) \
- if ((_sp)->gfn != (_gfn)) {} else
+ if ((_sp)->gfn != (_gfn) || is_obsolete_sp((_kvm), (_sp)) \
+ || (_sp)->role.invalid) {} else
#define for_each_gfn_indirect_valid_sp(_kvm, _sp, _gfn) \
- for_each_gfn_sp(_kvm, _sp, _gfn) \
- if ((_sp)->role.direct || (_sp)->role.invalid) {} else
+ for_each_gfn_valid_sp(_kvm, _sp, _gfn) \
+ if ((_sp)->role.direct) {} else
/* @sp->gfn should be write-protected at the call site */
static bool __kvm_sync_page(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp,
@@ -1961,6 +1960,11 @@
static void mmu_audit_disable(void) { }
#endif
+static bool is_obsolete_sp(struct kvm *kvm, struct kvm_mmu_page *sp)
+{
+ return unlikely(sp->mmu_valid_gen != kvm->arch.mmu_valid_gen);
+}
+
static bool kvm_sync_page(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp,
struct list_head *invalid_list)
{
@@ -2105,11 +2109,6 @@
__clear_sp_write_flooding_count(sp);
}
-static bool is_obsolete_sp(struct kvm *kvm, struct kvm_mmu_page *sp)
-{
- return unlikely(sp->mmu_valid_gen != kvm->arch.mmu_valid_gen);
-}
-
static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu,
gfn_t gfn,
gva_t gaddr,
@@ -2136,10 +2135,7 @@
quadrant &= (1 << ((PT32_PT_BITS - PT64_PT_BITS) * level)) - 1;
role.quadrant = quadrant;
}
- for_each_gfn_sp(vcpu->kvm, sp, gfn) {
- if (is_obsolete_sp(vcpu->kvm, sp))
- continue;
-
+ for_each_gfn_valid_sp(vcpu->kvm, sp, gfn) {
if (!need_sync && sp->unsync)
need_sync = true;
diff --git a/arch/x86/kvm/mtrr.c b/arch/x86/kvm/mtrr.c
index 3f8c732..c146f3c 100644
--- a/arch/x86/kvm/mtrr.c
+++ b/arch/x86/kvm/mtrr.c
@@ -44,8 +44,6 @@
case MSR_MTRRdefType:
case MSR_IA32_CR_PAT:
return true;
- case 0x2f8:
- return true;
}
return false;
}
diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
index fafd720..2214214 100644
--- a/arch/x86/kvm/svm.c
+++ b/arch/x86/kvm/svm.c
@@ -14,6 +14,9 @@
* the COPYING file in the top-level directory.
*
*/
+
+#define pr_fmt(fmt) "SVM: " fmt
+
#include <linux/kvm_host.h>
#include "irq.h"
@@ -32,6 +35,7 @@
#include <linux/trace_events.h>
#include <linux/slab.h>
+#include <asm/apic.h>
#include <asm/perf_event.h>
#include <asm/tlbflush.h>
#include <asm/desc.h>
@@ -68,6 +72,8 @@
#define SVM_FEATURE_DECODE_ASSIST (1 << 7)
#define SVM_FEATURE_PAUSE_FILTER (1 << 10)
+#define SVM_AVIC_DOORBELL 0xc001011b
+
#define NESTED_EXIT_HOST 0 /* Exit handled on host level */
#define NESTED_EXIT_DONE 1 /* Exit caused nested vmexit */
#define NESTED_EXIT_CONTINUE 2 /* Further checks needed */
@@ -78,6 +84,18 @@
#define TSC_RATIO_MIN 0x0000000000000001ULL
#define TSC_RATIO_MAX 0x000000ffffffffffULL
+#define AVIC_HPA_MASK ~((0xFFFULL << 52) || 0xFFF)
+
+/*
+ * 0xff is broadcast, so the max index allowed for physical APIC ID
+ * table is 0xfe. APIC IDs above 0xff are reserved.
+ */
+#define AVIC_MAX_PHYSICAL_ID_COUNT 255
+
+#define AVIC_UNACCEL_ACCESS_WRITE_MASK 1
+#define AVIC_UNACCEL_ACCESS_OFFSET_MASK 0xFF0
+#define AVIC_UNACCEL_ACCESS_VECTOR_MASK 0xFFFFFFFF
+
static bool erratum_383_found __read_mostly;
static const u32 host_save_user_msrs[] = {
@@ -162,8 +180,21 @@
/* cached guest cpuid flags for faster access */
bool nrips_enabled : 1;
+
+ u32 ldr_reg;
+ struct page *avic_backing_page;
+ u64 *avic_physical_id_cache;
+ bool avic_is_running;
};
+#define AVIC_LOGICAL_ID_ENTRY_GUEST_PHYSICAL_ID_MASK (0xFF)
+#define AVIC_LOGICAL_ID_ENTRY_VALID_MASK (1 << 31)
+
+#define AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK (0xFFULL)
+#define AVIC_PHYSICAL_ID_ENTRY_BACKING_PAGE_MASK (0xFFFFFFFFFFULL << 12)
+#define AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK (1ULL << 62)
+#define AVIC_PHYSICAL_ID_ENTRY_VALID_MASK (1ULL << 63)
+
static DEFINE_PER_CPU(u64, current_tsc_ratio);
#define TSC_RATIO_DEFAULT 0x0100000000ULL
@@ -205,6 +236,10 @@
static int nested = true;
module_param(nested, int, S_IRUGO);
+/* enable / disable AVIC */
+static int avic;
+module_param(avic, int, S_IRUGO);
+
static void svm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0);
static void svm_flush_tlb(struct kvm_vcpu *vcpu);
static void svm_complete_interrupts(struct vcpu_svm *svm);
@@ -228,12 +263,18 @@
VMCB_SEG, /* CS, DS, SS, ES, CPL */
VMCB_CR2, /* CR2 only */
VMCB_LBR, /* DBGCTL, BR_FROM, BR_TO, LAST_EX_FROM, LAST_EX_TO */
+ VMCB_AVIC, /* AVIC APIC_BAR, AVIC APIC_BACKING_PAGE,
+ * AVIC PHYSICAL_TABLE pointer,
+ * AVIC LOGICAL_TABLE pointer
+ */
VMCB_DIRTY_MAX,
};
/* TPR and CR2 are always written before VMRUN */
#define VMCB_ALWAYS_DIRTY_MASK ((1U << VMCB_INTR) | (1U << VMCB_CR2))
+#define VMCB_AVIC_APIC_BAR_MASK 0xFFFFFFFFFF000ULL
+
static inline void mark_all_dirty(struct vmcb *vmcb)
{
vmcb->control.clean = 0;
@@ -255,6 +296,23 @@
return container_of(vcpu, struct vcpu_svm, vcpu);
}
+static inline void avic_update_vapic_bar(struct vcpu_svm *svm, u64 data)
+{
+ svm->vmcb->control.avic_vapic_bar = data & VMCB_AVIC_APIC_BAR_MASK;
+ mark_dirty(svm->vmcb, VMCB_AVIC);
+}
+
+static inline bool avic_vcpu_is_running(struct kvm_vcpu *vcpu)
+{
+ struct vcpu_svm *svm = to_svm(vcpu);
+ u64 *entry = svm->avic_physical_id_cache;
+
+ if (!entry)
+ return false;
+
+ return (READ_ONCE(*entry) & AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK);
+}
+
static void recalc_intercepts(struct vcpu_svm *svm)
{
struct vmcb_control_area *c, *h;
@@ -923,6 +981,12 @@
} else
kvm_disable_tdp();
+ if (avic && (!npt_enabled || !boot_cpu_has(X86_FEATURE_AVIC)))
+ avic = false;
+
+ if (avic)
+ pr_info("AVIC enabled\n");
+
return 0;
err:
@@ -1000,6 +1064,22 @@
mark_dirty(svm->vmcb, VMCB_INTERCEPTS);
}
+static void avic_init_vmcb(struct vcpu_svm *svm)
+{
+ struct vmcb *vmcb = svm->vmcb;
+ struct kvm_arch *vm_data = &svm->vcpu.kvm->arch;
+ phys_addr_t bpa = page_to_phys(svm->avic_backing_page);
+ phys_addr_t lpa = page_to_phys(vm_data->avic_logical_id_table_page);
+ phys_addr_t ppa = page_to_phys(vm_data->avic_physical_id_table_page);
+
+ vmcb->control.avic_backing_page = bpa & AVIC_HPA_MASK;
+ vmcb->control.avic_logical_id = lpa & AVIC_HPA_MASK;
+ vmcb->control.avic_physical_id = ppa & AVIC_HPA_MASK;
+ vmcb->control.avic_physical_id |= AVIC_MAX_PHYSICAL_ID_COUNT;
+ vmcb->control.int_ctl |= AVIC_ENABLE_MASK;
+ svm->vcpu.arch.apicv_active = true;
+}
+
static void init_vmcb(struct vcpu_svm *svm)
{
struct vmcb_control_area *control = &svm->vmcb->control;
@@ -1014,7 +1094,8 @@
set_cr_intercept(svm, INTERCEPT_CR0_WRITE);
set_cr_intercept(svm, INTERCEPT_CR3_WRITE);
set_cr_intercept(svm, INTERCEPT_CR4_WRITE);
- set_cr_intercept(svm, INTERCEPT_CR8_WRITE);
+ if (!kvm_vcpu_apicv_active(&svm->vcpu))
+ set_cr_intercept(svm, INTERCEPT_CR8_WRITE);
set_dr_intercepts(svm);
@@ -1110,9 +1191,197 @@
set_intercept(svm, INTERCEPT_PAUSE);
}
+ if (avic)
+ avic_init_vmcb(svm);
+
mark_all_dirty(svm->vmcb);
enable_gif(svm);
+
+}
+
+static u64 *avic_get_physical_id_entry(struct kvm_vcpu *vcpu, int index)
+{
+ u64 *avic_physical_id_table;
+ struct kvm_arch *vm_data = &vcpu->kvm->arch;
+
+ if (index >= AVIC_MAX_PHYSICAL_ID_COUNT)
+ return NULL;
+
+ avic_physical_id_table = page_address(vm_data->avic_physical_id_table_page);
+
+ return &avic_physical_id_table[index];
+}
+
+/**
+ * Note:
+ * AVIC hardware walks the nested page table to check permissions,
+ * but does not use the SPA address specified in the leaf page
+ * table entry since it uses address in the AVIC_BACKING_PAGE pointer
+ * field of the VMCB. Therefore, we set up the
+ * APIC_ACCESS_PAGE_PRIVATE_MEMSLOT (4KB) here.
+ */
+static int avic_init_access_page(struct kvm_vcpu *vcpu)
+{
+ struct kvm *kvm = vcpu->kvm;
+ int ret;
+
+ if (kvm->arch.apic_access_page_done)
+ return 0;
+
+ ret = x86_set_memory_region(kvm,
+ APIC_ACCESS_PAGE_PRIVATE_MEMSLOT,
+ APIC_DEFAULT_PHYS_BASE,
+ PAGE_SIZE);
+ if (ret)
+ return ret;
+
+ kvm->arch.apic_access_page_done = true;
+ return 0;
+}
+
+static int avic_init_backing_page(struct kvm_vcpu *vcpu)
+{
+ int ret;
+ u64 *entry, new_entry;
+ int id = vcpu->vcpu_id;
+ struct vcpu_svm *svm = to_svm(vcpu);
+
+ ret = avic_init_access_page(vcpu);
+ if (ret)
+ return ret;
+
+ if (id >= AVIC_MAX_PHYSICAL_ID_COUNT)
+ return -EINVAL;
+
+ if (!svm->vcpu.arch.apic->regs)
+ return -EINVAL;
+
+ svm->avic_backing_page = virt_to_page(svm->vcpu.arch.apic->regs);
+
+ /* Setting AVIC backing page address in the phy APIC ID table */
+ entry = avic_get_physical_id_entry(vcpu, id);
+ if (!entry)
+ return -EINVAL;
+
+ new_entry = READ_ONCE(*entry);
+ new_entry = (page_to_phys(svm->avic_backing_page) &
+ AVIC_PHYSICAL_ID_ENTRY_BACKING_PAGE_MASK) |
+ AVIC_PHYSICAL_ID_ENTRY_VALID_MASK;
+ WRITE_ONCE(*entry, new_entry);
+
+ svm->avic_physical_id_cache = entry;
+
+ return 0;
+}
+
+static void avic_vm_destroy(struct kvm *kvm)
+{
+ struct kvm_arch *vm_data = &kvm->arch;
+
+ if (vm_data->avic_logical_id_table_page)
+ __free_page(vm_data->avic_logical_id_table_page);
+ if (vm_data->avic_physical_id_table_page)
+ __free_page(vm_data->avic_physical_id_table_page);
+}
+
+static int avic_vm_init(struct kvm *kvm)
+{
+ int err = -ENOMEM;
+ struct kvm_arch *vm_data = &kvm->arch;
+ struct page *p_page;
+ struct page *l_page;
+
+ if (!avic)
+ return 0;
+
+ /* Allocating physical APIC ID table (4KB) */
+ p_page = alloc_page(GFP_KERNEL);
+ if (!p_page)
+ goto free_avic;
+
+ vm_data->avic_physical_id_table_page = p_page;
+ clear_page(page_address(p_page));
+
+ /* Allocating logical APIC ID table (4KB) */
+ l_page = alloc_page(GFP_KERNEL);
+ if (!l_page)
+ goto free_avic;
+
+ vm_data->avic_logical_id_table_page = l_page;
+ clear_page(page_address(l_page));
+
+ return 0;
+
+free_avic:
+ avic_vm_destroy(kvm);
+ return err;
+}
+
+/**
+ * This function is called during VCPU halt/unhalt.
+ */
+static void avic_set_running(struct kvm_vcpu *vcpu, bool is_run)
+{
+ u64 entry;
+ int h_physical_id = __default_cpu_present_to_apicid(vcpu->cpu);
+ struct vcpu_svm *svm = to_svm(vcpu);
+
+ if (!kvm_vcpu_apicv_active(vcpu))
+ return;
+
+ svm->avic_is_running = is_run;
+
+ /* ID = 0xff (broadcast), ID > 0xff (reserved) */
+ if (WARN_ON(h_physical_id >= AVIC_MAX_PHYSICAL_ID_COUNT))
+ return;
+
+ entry = READ_ONCE(*(svm->avic_physical_id_cache));
+ WARN_ON(is_run == !!(entry & AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK));
+
+ entry &= ~AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK;
+ if (is_run)
+ entry |= AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK;
+ WRITE_ONCE(*(svm->avic_physical_id_cache), entry);
+}
+
+static void avic_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
+{
+ u64 entry;
+ /* ID = 0xff (broadcast), ID > 0xff (reserved) */
+ int h_physical_id = __default_cpu_present_to_apicid(cpu);
+ struct vcpu_svm *svm = to_svm(vcpu);
+
+ if (!kvm_vcpu_apicv_active(vcpu))
+ return;
+
+ if (WARN_ON(h_physical_id >= AVIC_MAX_PHYSICAL_ID_COUNT))
+ return;
+
+ entry = READ_ONCE(*(svm->avic_physical_id_cache));
+ WARN_ON(entry & AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK);
+
+ entry &= ~AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK;
+ entry |= (h_physical_id & AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK);
+
+ entry &= ~AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK;
+ if (svm->avic_is_running)
+ entry |= AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK;
+
+ WRITE_ONCE(*(svm->avic_physical_id_cache), entry);
+}
+
+static void avic_vcpu_put(struct kvm_vcpu *vcpu)
+{
+ u64 entry;
+ struct vcpu_svm *svm = to_svm(vcpu);
+
+ if (!kvm_vcpu_apicv_active(vcpu))
+ return;
+
+ entry = READ_ONCE(*(svm->avic_physical_id_cache));
+ entry &= ~AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK;
+ WRITE_ONCE(*(svm->avic_physical_id_cache), entry);
}
static void svm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)
@@ -1131,6 +1400,9 @@
kvm_cpuid(vcpu, &eax, &dummy, &dummy, &dummy);
kvm_register_write(vcpu, VCPU_REGS_RDX, eax);
+
+ if (kvm_vcpu_apicv_active(vcpu) && !init_event)
+ avic_update_vapic_bar(svm, APIC_DEFAULT_PHYS_BASE);
}
static struct kvm_vcpu *svm_create_vcpu(struct kvm *kvm, unsigned int id)
@@ -1169,6 +1441,17 @@
if (!hsave_page)
goto free_page3;
+ if (avic) {
+ err = avic_init_backing_page(&svm->vcpu);
+ if (err)
+ goto free_page4;
+ }
+
+ /* We initialize this flag to true to make sure that the is_running
+ * bit would be set the first time the vcpu is loaded.
+ */
+ svm->avic_is_running = true;
+
svm->nested.hsave = page_address(hsave_page);
svm->msrpm = page_address(msrpm_pages);
@@ -1187,6 +1470,8 @@
return &svm->vcpu;
+free_page4:
+ __free_page(hsave_page);
free_page3:
__free_pages(nested_msrpm_pages, MSRPM_ALLOC_ORDER);
free_page2:
@@ -1243,6 +1528,8 @@
/* This assumes that the kernel never uses MSR_TSC_AUX */
if (static_cpu_has(X86_FEATURE_RDTSCP))
wrmsrl(MSR_TSC_AUX, svm->tsc_aux);
+
+ avic_vcpu_load(vcpu, cpu);
}
static void svm_vcpu_put(struct kvm_vcpu *vcpu)
@@ -1250,6 +1537,8 @@
struct vcpu_svm *svm = to_svm(vcpu);
int i;
+ avic_vcpu_put(vcpu);
+
++vcpu->stat.host_state_reload;
kvm_load_ldt(svm->host.ldt);
#ifdef CONFIG_X86_64
@@ -1265,6 +1554,16 @@
wrmsrl(host_save_user_msrs[i], svm->host_user_msrs[i]);
}
+static void svm_vcpu_blocking(struct kvm_vcpu *vcpu)
+{
+ avic_set_running(vcpu, false);
+}
+
+static void svm_vcpu_unblocking(struct kvm_vcpu *vcpu)
+{
+ avic_set_running(vcpu, true);
+}
+
static unsigned long svm_get_rflags(struct kvm_vcpu *vcpu)
{
return to_svm(vcpu)->vmcb->save.rflags;
@@ -2673,10 +2972,11 @@
disable_gif(svm);
/* After a CLGI no interrupts should come */
- svm_clear_vintr(svm);
- svm->vmcb->control.int_ctl &= ~V_IRQ_MASK;
-
- mark_dirty(svm->vmcb, VMCB_INTR);
+ if (!kvm_vcpu_apicv_active(&svm->vcpu)) {
+ svm_clear_vintr(svm);
+ svm->vmcb->control.int_ctl &= ~V_IRQ_MASK;
+ mark_dirty(svm->vmcb, VMCB_INTR);
+ }
return 1;
}
@@ -3212,6 +3512,10 @@
case MSR_VM_IGNNE:
vcpu_unimpl(vcpu, "unimplemented wrmsr: 0x%x data 0x%llx\n", ecx, data);
break;
+ case MSR_IA32_APICBASE:
+ if (kvm_vcpu_apicv_active(vcpu))
+ avic_update_vapic_bar(to_svm(vcpu), data);
+ /* Follow through */
default:
return kvm_set_msr_common(vcpu, msr);
}
@@ -3281,6 +3585,278 @@
return nop_interception(svm);
}
+enum avic_ipi_failure_cause {
+ AVIC_IPI_FAILURE_INVALID_INT_TYPE,
+ AVIC_IPI_FAILURE_TARGET_NOT_RUNNING,
+ AVIC_IPI_FAILURE_INVALID_TARGET,
+ AVIC_IPI_FAILURE_INVALID_BACKING_PAGE,
+};
+
+static int avic_incomplete_ipi_interception(struct vcpu_svm *svm)
+{
+ u32 icrh = svm->vmcb->control.exit_info_1 >> 32;
+ u32 icrl = svm->vmcb->control.exit_info_1;
+ u32 id = svm->vmcb->control.exit_info_2 >> 32;
+ u32 index = svm->vmcb->control.exit_info_2 && 0xFF;
+ struct kvm_lapic *apic = svm->vcpu.arch.apic;
+
+ trace_kvm_avic_incomplete_ipi(svm->vcpu.vcpu_id, icrh, icrl, id, index);
+
+ switch (id) {
+ case AVIC_IPI_FAILURE_INVALID_INT_TYPE:
+ /*
+ * AVIC hardware handles the generation of
+ * IPIs when the specified Message Type is Fixed
+ * (also known as fixed delivery mode) and
+ * the Trigger Mode is edge-triggered. The hardware
+ * also supports self and broadcast delivery modes
+ * specified via the Destination Shorthand(DSH)
+ * field of the ICRL. Logical and physical APIC ID
+ * formats are supported. All other IPI types cause
+ * a #VMEXIT, which needs to emulated.
+ */
+ kvm_lapic_reg_write(apic, APIC_ICR2, icrh);
+ kvm_lapic_reg_write(apic, APIC_ICR, icrl);
+ break;
+ case AVIC_IPI_FAILURE_TARGET_NOT_RUNNING: {
+ int i;
+ struct kvm_vcpu *vcpu;
+ struct kvm *kvm = svm->vcpu.kvm;
+ struct kvm_lapic *apic = svm->vcpu.arch.apic;
+
+ /*
+ * At this point, we expect that the AVIC HW has already
+ * set the appropriate IRR bits on the valid target
+ * vcpus. So, we just need to kick the appropriate vcpu.
+ */
+ kvm_for_each_vcpu(i, vcpu, kvm) {
+ bool m = kvm_apic_match_dest(vcpu, apic,
+ icrl & KVM_APIC_SHORT_MASK,
+ GET_APIC_DEST_FIELD(icrh),
+ icrl & KVM_APIC_DEST_MASK);
+
+ if (m && !avic_vcpu_is_running(vcpu))
+ kvm_vcpu_wake_up(vcpu);
+ }
+ break;
+ }
+ case AVIC_IPI_FAILURE_INVALID_TARGET:
+ break;
+ case AVIC_IPI_FAILURE_INVALID_BACKING_PAGE:
+ WARN_ONCE(1, "Invalid backing page\n");
+ break;
+ default:
+ pr_err("Unknown IPI interception\n");
+ }
+
+ return 1;
+}
+
+static u32 *avic_get_logical_id_entry(struct kvm_vcpu *vcpu, u32 ldr, bool flat)
+{
+ struct kvm_arch *vm_data = &vcpu->kvm->arch;
+ int index;
+ u32 *logical_apic_id_table;
+ int dlid = GET_APIC_LOGICAL_ID(ldr);
+
+ if (!dlid)
+ return NULL;
+
+ if (flat) { /* flat */
+ index = ffs(dlid) - 1;
+ if (index > 7)
+ return NULL;
+ } else { /* cluster */
+ int cluster = (dlid & 0xf0) >> 4;
+ int apic = ffs(dlid & 0x0f) - 1;
+
+ if ((apic < 0) || (apic > 7) ||
+ (cluster >= 0xf))
+ return NULL;
+ index = (cluster << 2) + apic;
+ }
+
+ logical_apic_id_table = (u32 *) page_address(vm_data->avic_logical_id_table_page);
+
+ return &logical_apic_id_table[index];
+}
+
+static int avic_ldr_write(struct kvm_vcpu *vcpu, u8 g_physical_id, u32 ldr,
+ bool valid)
+{
+ bool flat;
+ u32 *entry, new_entry;
+
+ flat = kvm_lapic_get_reg(vcpu->arch.apic, APIC_DFR) == APIC_DFR_FLAT;
+ entry = avic_get_logical_id_entry(vcpu, ldr, flat);
+ if (!entry)
+ return -EINVAL;
+
+ new_entry = READ_ONCE(*entry);
+ new_entry &= ~AVIC_LOGICAL_ID_ENTRY_GUEST_PHYSICAL_ID_MASK;
+ new_entry |= (g_physical_id & AVIC_LOGICAL_ID_ENTRY_GUEST_PHYSICAL_ID_MASK);
+ if (valid)
+ new_entry |= AVIC_LOGICAL_ID_ENTRY_VALID_MASK;
+ else
+ new_entry &= ~AVIC_LOGICAL_ID_ENTRY_VALID_MASK;
+ WRITE_ONCE(*entry, new_entry);
+
+ return 0;
+}
+
+static int avic_handle_ldr_update(struct kvm_vcpu *vcpu)
+{
+ int ret;
+ struct vcpu_svm *svm = to_svm(vcpu);
+ u32 ldr = kvm_lapic_get_reg(vcpu->arch.apic, APIC_LDR);
+
+ if (!ldr)
+ return 1;
+
+ ret = avic_ldr_write(vcpu, vcpu->vcpu_id, ldr, true);
+ if (ret && svm->ldr_reg) {
+ avic_ldr_write(vcpu, 0, svm->ldr_reg, false);
+ svm->ldr_reg = 0;
+ } else {
+ svm->ldr_reg = ldr;
+ }
+ return ret;
+}
+
+static int avic_handle_apic_id_update(struct kvm_vcpu *vcpu)
+{
+ u64 *old, *new;
+ struct vcpu_svm *svm = to_svm(vcpu);
+ u32 apic_id_reg = kvm_lapic_get_reg(vcpu->arch.apic, APIC_ID);
+ u32 id = (apic_id_reg >> 24) & 0xff;
+
+ if (vcpu->vcpu_id == id)
+ return 0;
+
+ old = avic_get_physical_id_entry(vcpu, vcpu->vcpu_id);
+ new = avic_get_physical_id_entry(vcpu, id);
+ if (!new || !old)
+ return 1;
+
+ /* We need to move physical_id_entry to new offset */
+ *new = *old;
+ *old = 0ULL;
+ to_svm(vcpu)->avic_physical_id_cache = new;
+
+ /*
+ * Also update the guest physical APIC ID in the logical
+ * APIC ID table entry if already setup the LDR.
+ */
+ if (svm->ldr_reg)
+ avic_handle_ldr_update(vcpu);
+
+ return 0;
+}
+
+static int avic_handle_dfr_update(struct kvm_vcpu *vcpu)
+{
+ struct vcpu_svm *svm = to_svm(vcpu);
+ struct kvm_arch *vm_data = &vcpu->kvm->arch;
+ u32 dfr = kvm_lapic_get_reg(vcpu->arch.apic, APIC_DFR);
+ u32 mod = (dfr >> 28) & 0xf;
+
+ /*
+ * We assume that all local APICs are using the same type.
+ * If this changes, we need to flush the AVIC logical
+ * APID id table.
+ */
+ if (vm_data->ldr_mode == mod)
+ return 0;
+
+ clear_page(page_address(vm_data->avic_logical_id_table_page));
+ vm_data->ldr_mode = mod;
+
+ if (svm->ldr_reg)
+ avic_handle_ldr_update(vcpu);
+ return 0;
+}
+
+static int avic_unaccel_trap_write(struct vcpu_svm *svm)
+{
+ struct kvm_lapic *apic = svm->vcpu.arch.apic;
+ u32 offset = svm->vmcb->control.exit_info_1 &
+ AVIC_UNACCEL_ACCESS_OFFSET_MASK;
+
+ switch (offset) {
+ case APIC_ID:
+ if (avic_handle_apic_id_update(&svm->vcpu))
+ return 0;
+ break;
+ case APIC_LDR:
+ if (avic_handle_ldr_update(&svm->vcpu))
+ return 0;
+ break;
+ case APIC_DFR:
+ avic_handle_dfr_update(&svm->vcpu);
+ break;
+ default:
+ break;
+ }
+
+ kvm_lapic_reg_write(apic, offset, kvm_lapic_get_reg(apic, offset));
+
+ return 1;
+}
+
+static bool is_avic_unaccelerated_access_trap(u32 offset)
+{
+ bool ret = false;
+
+ switch (offset) {
+ case APIC_ID:
+ case APIC_EOI:
+ case APIC_RRR:
+ case APIC_LDR:
+ case APIC_DFR:
+ case APIC_SPIV:
+ case APIC_ESR:
+ case APIC_ICR:
+ case APIC_LVTT:
+ case APIC_LVTTHMR:
+ case APIC_LVTPC:
+ case APIC_LVT0:
+ case APIC_LVT1:
+ case APIC_LVTERR:
+ case APIC_TMICT:
+ case APIC_TDCR:
+ ret = true;
+ break;
+ default:
+ break;
+ }
+ return ret;
+}
+
+static int avic_unaccelerated_access_interception(struct vcpu_svm *svm)
+{
+ int ret = 0;
+ u32 offset = svm->vmcb->control.exit_info_1 &
+ AVIC_UNACCEL_ACCESS_OFFSET_MASK;
+ u32 vector = svm->vmcb->control.exit_info_2 &
+ AVIC_UNACCEL_ACCESS_VECTOR_MASK;
+ bool write = (svm->vmcb->control.exit_info_1 >> 32) &
+ AVIC_UNACCEL_ACCESS_WRITE_MASK;
+ bool trap = is_avic_unaccelerated_access_trap(offset);
+
+ trace_kvm_avic_unaccelerated_access(svm->vcpu.vcpu_id, offset,
+ trap, write, vector);
+ if (trap) {
+ /* Handling Trap */
+ WARN_ONCE(!write, "svm: Handling trap read.\n");
+ ret = avic_unaccel_trap_write(svm);
+ } else {
+ /* Handling Fault */
+ ret = (emulate_instruction(&svm->vcpu, 0) == EMULATE_DONE);
+ }
+
+ return ret;
+}
+
static int (*const svm_exit_handlers[])(struct vcpu_svm *svm) = {
[SVM_EXIT_READ_CR0] = cr_interception,
[SVM_EXIT_READ_CR3] = cr_interception,
@@ -3344,6 +3920,8 @@
[SVM_EXIT_XSETBV] = xsetbv_interception,
[SVM_EXIT_NPF] = pf_interception,
[SVM_EXIT_RSM] = emulate_on_interception,
+ [SVM_EXIT_AVIC_INCOMPLETE_IPI] = avic_incomplete_ipi_interception,
+ [SVM_EXIT_AVIC_UNACCELERATED_ACCESS] = avic_unaccelerated_access_interception,
};
static void dump_vmcb(struct kvm_vcpu *vcpu)
@@ -3375,10 +3953,14 @@
pr_err("%-20s%08x\n", "exit_int_info_err:", control->exit_int_info_err);
pr_err("%-20s%lld\n", "nested_ctl:", control->nested_ctl);
pr_err("%-20s%016llx\n", "nested_cr3:", control->nested_cr3);
+ pr_err("%-20s%016llx\n", "avic_vapic_bar:", control->avic_vapic_bar);
pr_err("%-20s%08x\n", "event_inj:", control->event_inj);
pr_err("%-20s%08x\n", "event_inj_err:", control->event_inj_err);
pr_err("%-20s%lld\n", "lbr_ctl:", control->lbr_ctl);
pr_err("%-20s%016llx\n", "next_rip:", control->next_rip);
+ pr_err("%-20s%016llx\n", "avic_backing_page:", control->avic_backing_page);
+ pr_err("%-20s%016llx\n", "avic_logical_id:", control->avic_logical_id);
+ pr_err("%-20s%016llx\n", "avic_physical_id:", control->avic_physical_id);
pr_err("VMCB State Save Area:\n");
pr_err("%-5s s: %04x a: %04x l: %08x b: %016llx\n",
"es:",
@@ -3562,6 +4144,7 @@
{
struct vmcb_control_area *control;
+ /* The following fields are ignored when AVIC is enabled */
control = &svm->vmcb->control;
control->int_vector = irq;
control->int_ctl &= ~V_INTR_PRIO_MASK;
@@ -3583,11 +4166,17 @@
SVM_EVTINJ_VALID | SVM_EVTINJ_TYPE_INTR;
}
+static inline bool svm_nested_virtualize_tpr(struct kvm_vcpu *vcpu)
+{
+ return is_guest_mode(vcpu) && (vcpu->arch.hflags & HF_VINTR_MASK);
+}
+
static void update_cr8_intercept(struct kvm_vcpu *vcpu, int tpr, int irr)
{
struct vcpu_svm *svm = to_svm(vcpu);
- if (is_guest_mode(vcpu) && (vcpu->arch.hflags & HF_VINTR_MASK))
+ if (svm_nested_virtualize_tpr(vcpu) ||
+ kvm_vcpu_apicv_active(vcpu))
return;
clr_cr_intercept(svm, INTERCEPT_CR8_WRITE);
@@ -3606,11 +4195,28 @@
static bool svm_get_enable_apicv(void)
{
- return false;
+ return avic;
}
+static void svm_hwapic_irr_update(struct kvm_vcpu *vcpu, int max_irr)
+{
+}
+
+static void svm_hwapic_isr_update(struct kvm_vcpu *vcpu, int max_isr)
+{
+}
+
+/* Note: Currently only used by Hyper-V. */
static void svm_refresh_apicv_exec_ctrl(struct kvm_vcpu *vcpu)
{
+ struct vcpu_svm *svm = to_svm(vcpu);
+ struct vmcb *vmcb = svm->vmcb;
+
+ if (!avic)
+ return;
+
+ vmcb->control.int_ctl &= ~AVIC_ENABLE_MASK;
+ mark_dirty(vmcb, VMCB_INTR);
}
static void svm_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap)
@@ -3623,6 +4229,18 @@
return;
}
+static void svm_deliver_avic_intr(struct kvm_vcpu *vcpu, int vec)
+{
+ kvm_lapic_set_irr(vec, vcpu->arch.apic);
+ smp_mb__after_atomic();
+
+ if (avic_vcpu_is_running(vcpu))
+ wrmsrl(SVM_AVIC_DOORBELL,
+ __default_cpu_present_to_apicid(vcpu->cpu));
+ else
+ kvm_vcpu_wake_up(vcpu);
+}
+
static int svm_nmi_allowed(struct kvm_vcpu *vcpu)
{
struct vcpu_svm *svm = to_svm(vcpu);
@@ -3677,6 +4295,9 @@
{
struct vcpu_svm *svm = to_svm(vcpu);
+ if (kvm_vcpu_apicv_active(vcpu))
+ return;
+
/*
* In case GIF=0 we can't rely on the CPU to tell us when GIF becomes
* 1, because that's a separate STGI/VMRUN intercept. The next time we
@@ -3728,7 +4349,7 @@
{
struct vcpu_svm *svm = to_svm(vcpu);
- if (is_guest_mode(vcpu) && (vcpu->arch.hflags & HF_VINTR_MASK))
+ if (svm_nested_virtualize_tpr(vcpu))
return;
if (!is_cr_intercept(svm, INTERCEPT_CR8_WRITE)) {
@@ -3742,7 +4363,8 @@
struct vcpu_svm *svm = to_svm(vcpu);
u64 cr8;
- if (is_guest_mode(vcpu) && (vcpu->arch.hflags & HF_VINTR_MASK))
+ if (svm_nested_virtualize_tpr(vcpu) ||
+ kvm_vcpu_apicv_active(vcpu))
return;
cr8 = kvm_get_cr8(vcpu);
@@ -4045,14 +4667,26 @@
static void svm_cpuid_update(struct kvm_vcpu *vcpu)
{
struct vcpu_svm *svm = to_svm(vcpu);
+ struct kvm_cpuid_entry2 *entry;
/* Update nrips enabled cache */
svm->nrips_enabled = !!guest_cpuid_has_nrips(&svm->vcpu);
+
+ if (!kvm_vcpu_apicv_active(vcpu))
+ return;
+
+ entry = kvm_find_cpuid_entry(vcpu, 1, 0);
+ if (entry)
+ entry->ecx &= ~bit(X86_FEATURE_X2APIC);
}
static void svm_set_supported_cpuid(u32 func, struct kvm_cpuid_entry2 *entry)
{
switch (func) {
+ case 0x1:
+ if (avic)
+ entry->ecx &= ~bit(X86_FEATURE_X2APIC);
+ break;
case 0x80000001:
if (nested)
entry->ecx |= (1 << 2); /* Set SVM bit */
@@ -4307,6 +4941,15 @@
{
}
+static inline void avic_post_state_restore(struct kvm_vcpu *vcpu)
+{
+ if (avic_handle_apic_id_update(vcpu) != 0)
+ return;
+ if (avic_handle_dfr_update(vcpu) != 0)
+ return;
+ avic_handle_ldr_update(vcpu);
+}
+
static struct kvm_x86_ops svm_x86_ops = {
.cpu_has_kvm_support = has_svm,
.disabled_by_bios = is_disabled,
@@ -4322,9 +4965,14 @@
.vcpu_free = svm_free_vcpu,
.vcpu_reset = svm_vcpu_reset,
+ .vm_init = avic_vm_init,
+ .vm_destroy = avic_vm_destroy,
+
.prepare_guest_switch = svm_prepare_guest_switch,
.vcpu_load = svm_vcpu_load,
.vcpu_put = svm_vcpu_put,
+ .vcpu_blocking = svm_vcpu_blocking,
+ .vcpu_unblocking = svm_vcpu_unblocking,
.update_bp_intercept = update_bp_intercept,
.get_msr = svm_get_msr,
@@ -4382,6 +5030,9 @@
.refresh_apicv_exec_ctrl = svm_refresh_apicv_exec_ctrl,
.load_eoi_exitmap = svm_load_eoi_exitmap,
.sync_pir_to_irr = svm_sync_pir_to_irr,
+ .hwapic_irr_update = svm_hwapic_irr_update,
+ .hwapic_isr_update = svm_hwapic_isr_update,
+ .apicv_post_state_restore = avic_post_state_restore,
.set_tss_addr = svm_set_tss_addr,
.get_tdp_level = get_npt_level,
@@ -4415,6 +5066,7 @@
.sched_in = svm_sched_in,
.pmu_ops = &amd_pmu_ops,
+ .deliver_posted_interrupt = svm_deliver_avic_intr,
};
static int __init svm_init(void)
diff --git a/arch/x86/kvm/trace.h b/arch/x86/kvm/trace.h
index b72743c..8de9250 100644
--- a/arch/x86/kvm/trace.h
+++ b/arch/x86/kvm/trace.h
@@ -1291,6 +1291,63 @@
__entry->vcpu_id, __entry->timer_index)
);
+/*
+ * Tracepoint for AMD AVIC
+ */
+TRACE_EVENT(kvm_avic_incomplete_ipi,
+ TP_PROTO(u32 vcpu, u32 icrh, u32 icrl, u32 id, u32 index),
+ TP_ARGS(vcpu, icrh, icrl, id, index),
+
+ TP_STRUCT__entry(
+ __field(u32, vcpu)
+ __field(u32, icrh)
+ __field(u32, icrl)
+ __field(u32, id)
+ __field(u32, index)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu = vcpu;
+ __entry->icrh = icrh;
+ __entry->icrl = icrl;
+ __entry->id = id;
+ __entry->index = index;
+ ),
+
+ TP_printk("vcpu=%u, icrh:icrl=%#010x:%08x, id=%u, index=%u\n",
+ __entry->vcpu, __entry->icrh, __entry->icrl,
+ __entry->id, __entry->index)
+);
+
+TRACE_EVENT(kvm_avic_unaccelerated_access,
+ TP_PROTO(u32 vcpu, u32 offset, bool ft, bool rw, u32 vec),
+ TP_ARGS(vcpu, offset, ft, rw, vec),
+
+ TP_STRUCT__entry(
+ __field(u32, vcpu)
+ __field(u32, offset)
+ __field(bool, ft)
+ __field(bool, rw)
+ __field(u32, vec)
+ ),
+
+ TP_fast_assign(
+ __entry->vcpu = vcpu;
+ __entry->offset = offset;
+ __entry->ft = ft;
+ __entry->rw = rw;
+ __entry->vec = vec;
+ ),
+
+ TP_printk("vcpu=%u, offset=%#x(%s), %s, %s, vec=%#x\n",
+ __entry->vcpu,
+ __entry->offset,
+ __print_symbolic(__entry->offset, kvm_trace_symbol_apic),
+ __entry->ft ? "trap" : "fault",
+ __entry->rw ? "write" : "read",
+ __entry->vec)
+);
+
#endif /* _TRACE_KVM_H */
#undef TRACE_INCLUDE_PATH
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index cb47fe3..e605d1e 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -5050,8 +5050,8 @@
vmcs_write16(VIRTUAL_PROCESSOR_ID, vmx->vpid);
cr0 = X86_CR0_NW | X86_CR0_CD | X86_CR0_ET;
- vmx_set_cr0(vcpu, cr0); /* enter rmode */
vmx->vcpu.arch.cr0 = cr0;
+ vmx_set_cr0(vcpu, cr0); /* enter rmode */
vmx_set_cr4(vcpu, 0);
vmx_set_efer(vcpu, 0);
vmx_fpu_activate(vcpu);
@@ -8318,19 +8318,19 @@
vmcs_write64(APIC_ACCESS_ADDR, hpa);
}
-static void vmx_hwapic_isr_update(struct kvm *kvm, int isr)
+static void vmx_hwapic_isr_update(struct kvm_vcpu *vcpu, int max_isr)
{
u16 status;
u8 old;
- if (isr == -1)
- isr = 0;
+ if (max_isr == -1)
+ max_isr = 0;
status = vmcs_read16(GUEST_INTR_STATUS);
old = status >> 8;
- if (isr != old) {
+ if (max_isr != old) {
status &= 0xff;
- status |= isr << 8;
+ status |= max_isr << 8;
vmcs_write16(GUEST_INTR_STATUS, status);
}
}
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 12f33e6..c805cf4 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -161,6 +161,7 @@
{ "halt_exits", VCPU_STAT(halt_exits) },
{ "halt_successful_poll", VCPU_STAT(halt_successful_poll) },
{ "halt_attempted_poll", VCPU_STAT(halt_attempted_poll) },
+ { "halt_poll_invalid", VCPU_STAT(halt_poll_invalid) },
{ "halt_wakeup", VCPU_STAT(halt_wakeup) },
{ "hypercalls", VCPU_STAT(hypercalls) },
{ "request_irq", VCPU_STAT(request_irq_exits) },
@@ -2002,22 +2003,8 @@
vcpu->arch.pv_time_enabled = false;
}
-static void accumulate_steal_time(struct kvm_vcpu *vcpu)
-{
- u64 delta;
-
- if (!(vcpu->arch.st.msr_val & KVM_MSR_ENABLED))
- return;
-
- delta = current->sched_info.run_delay - vcpu->arch.st.last_steal;
- vcpu->arch.st.last_steal = current->sched_info.run_delay;
- vcpu->arch.st.accum_steal = delta;
-}
-
static void record_steal_time(struct kvm_vcpu *vcpu)
{
- accumulate_steal_time(vcpu);
-
if (!(vcpu->arch.st.msr_val & KVM_MSR_ENABLED))
return;
@@ -2025,9 +2012,26 @@
&vcpu->arch.st.steal, sizeof(struct kvm_steal_time))))
return;
- vcpu->arch.st.steal.steal += vcpu->arch.st.accum_steal;
- vcpu->arch.st.steal.version += 2;
- vcpu->arch.st.accum_steal = 0;
+ if (vcpu->arch.st.steal.version & 1)
+ vcpu->arch.st.steal.version += 1; /* first time write, random junk */
+
+ vcpu->arch.st.steal.version += 1;
+
+ kvm_write_guest_cached(vcpu->kvm, &vcpu->arch.st.stime,
+ &vcpu->arch.st.steal, sizeof(struct kvm_steal_time));
+
+ smp_wmb();
+
+ vcpu->arch.st.steal.steal += current->sched_info.run_delay -
+ vcpu->arch.st.last_steal;
+ vcpu->arch.st.last_steal = current->sched_info.run_delay;
+
+ kvm_write_guest_cached(vcpu->kvm, &vcpu->arch.st.stime,
+ &vcpu->arch.st.steal, sizeof(struct kvm_steal_time));
+
+ smp_wmb();
+
+ vcpu->arch.st.steal.version += 1;
kvm_write_guest_cached(vcpu->kvm, &vcpu->arch.st.stime,
&vcpu->arch.st.steal, sizeof(struct kvm_steal_time));
@@ -7752,6 +7756,9 @@
kvm_page_track_init(kvm);
kvm_mmu_init_vm(kvm);
+ if (kvm_x86_ops->vm_init)
+ return kvm_x86_ops->vm_init(kvm);
+
return 0;
}
@@ -7873,6 +7880,8 @@
x86_set_memory_region(kvm, IDENTITY_PAGETABLE_PRIVATE_MEMSLOT, 0, 0);
x86_set_memory_region(kvm, TSS_PRIVATE_MEMSLOT, 0, 0);
}
+ if (kvm_x86_ops->vm_destroy)
+ kvm_x86_ops->vm_destroy(kvm);
kvm_iommu_unmap_guest(kvm);
kfree(kvm->arch.vpic);
kfree(kvm->arch.vioapic);
@@ -8355,19 +8364,21 @@
}
EXPORT_SYMBOL_GPL(kvm_arch_has_noncoherent_dma);
+bool kvm_arch_has_irq_bypass(void)
+{
+ return kvm_x86_ops->update_pi_irte != NULL;
+}
+
int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
struct irq_bypass_producer *prod)
{
struct kvm_kernel_irqfd *irqfd =
container_of(cons, struct kvm_kernel_irqfd, consumer);
- if (kvm_x86_ops->update_pi_irte) {
- irqfd->producer = prod;
- return kvm_x86_ops->update_pi_irte(irqfd->kvm,
- prod->irq, irqfd->gsi, 1);
- }
+ irqfd->producer = prod;
- return -EINVAL;
+ return kvm_x86_ops->update_pi_irte(irqfd->kvm,
+ prod->irq, irqfd->gsi, 1);
}
void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
@@ -8377,11 +8388,6 @@
struct kvm_kernel_irqfd *irqfd =
container_of(cons, struct kvm_kernel_irqfd, consumer);
- if (!kvm_x86_ops->update_pi_irte) {
- WARN_ON(irqfd->producer != NULL);
- return;
- }
-
WARN_ON(irqfd->producer != prod);
irqfd->producer = NULL;
@@ -8429,3 +8435,5 @@
EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_ple_window);
EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_pml_full);
EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_pi_irte_update);
+EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_avic_unaccelerated_access);
+EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_avic_incomplete_ipi);
diff --git a/arch/x86/mm/hugetlbpage.c b/arch/x86/mm/hugetlbpage.c
index 14a9505..2ae8584 100644
--- a/arch/x86/mm/hugetlbpage.c
+++ b/arch/x86/mm/hugetlbpage.c
@@ -165,6 +165,7 @@
} else if (ps == PUD_SIZE && boot_cpu_has(X86_FEATURE_GBPAGES)) {
hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
} else {
+ hugetlb_bad_size();
printk(KERN_ERR "hugepagesz: Unsupported page size %lu M\n",
ps >> 20);
return 0;
diff --git a/arch/x86/mm/numa.c b/arch/x86/mm/numa.c
index f70c1ff..9c086c5 100644
--- a/arch/x86/mm/numa.c
+++ b/arch/x86/mm/numa.c
@@ -617,9 +617,7 @@
if (early_cpu_to_node(i) != NUMA_NO_NODE)
continue;
numa_set_node(i, rr);
- rr = next_node(rr, node_online_map);
- if (rr == MAX_NUMNODES)
- rr = first_node(node_online_map);
+ rr = next_node_in(rr, node_online_map);
}
}
diff --git a/arch/x86/pci/common.c b/arch/x86/pci/common.c
index 381a43c..8196054 100644
--- a/arch/x86/pci/common.c
+++ b/arch/x86/pci/common.c
@@ -516,7 +516,7 @@
int __init pcibios_init(void)
{
- if (!raw_pci_ops) {
+ if (!raw_pci_ops && !raw_pci_ext_ops) {
printk(KERN_WARNING "PCI: System does not support PCI\n");
return 0;
}
diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c
index b7de192..837ea36 100644
--- a/arch/x86/pci/fixup.c
+++ b/arch/x86/pci/fixup.c
@@ -552,9 +552,16 @@
}
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x27B9, twinhead_reserve_killing_zone);
+/*
+ * Broadwell EP Home Agent BARs erroneously return non-zero values when read.
+ *
+ * See http://www.intel.com/content/www/us/en/processors/xeon/xeon-e5-v4-spec-update.html
+ * entry BDF2.
+ */
static void pci_bdwep_bar(struct pci_dev *dev)
{
dev->non_compliant_bars = 1;
}
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6f60, pci_bdwep_bar);
DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fa0, pci_bdwep_bar);
DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fc0, pci_bdwep_bar);
diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
index 4bd08b0..99ddab7 100644
--- a/arch/x86/pci/xen.c
+++ b/arch/x86/pci/xen.c
@@ -491,8 +491,11 @@
#endif
__acpi_register_gsi = acpi_register_gsi_xen;
__acpi_unregister_gsi = NULL;
- /* Pre-allocate legacy irqs */
- for (irq = 0; irq < nr_legacy_irqs(); irq++) {
+ /*
+ * Pre-allocate the legacy IRQs. Use NR_LEGACY_IRQS here
+ * because we don't have a PIC and thus nr_legacy_irqs() is zero.
+ */
+ for (irq = 0; irq < NR_IRQS_LEGACY; irq++) {
int trigger, polarity;
if (acpi_get_override_irq(irq, &trigger, &polarity) == -1)
diff --git a/arch/x86/um/vdso/vma.c b/arch/x86/um/vdso/vma.c
index 237c683..6be22f9 100644
--- a/arch/x86/um/vdso/vma.c
+++ b/arch/x86/um/vdso/vma.c
@@ -61,7 +61,8 @@
if (!vdso_enabled)
return 0;
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
err = install_special_mapping(mm, um_vdso_addr, PAGE_SIZE,
VM_READ|VM_EXEC|
diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c
index 7ab2951..e345891 100644
--- a/arch/x86/xen/setup.c
+++ b/arch/x86/xen/setup.c
@@ -393,6 +393,9 @@
unsigned long i = 0;
unsigned long n = end_pfn - start_pfn;
+ if (remap_pfn == 0)
+ remap_pfn = nr_pages;
+
while (i < n) {
unsigned long cur_pfn = start_pfn + i;
unsigned long left = n - i;
@@ -438,17 +441,29 @@
return remap_pfn;
}
-static void __init xen_set_identity_and_remap(unsigned long nr_pages)
+static unsigned long __init xen_count_remap_pages(
+ unsigned long start_pfn, unsigned long end_pfn, unsigned long nr_pages,
+ unsigned long remap_pages)
+{
+ if (start_pfn >= nr_pages)
+ return remap_pages;
+
+ return remap_pages + min(end_pfn, nr_pages) - start_pfn;
+}
+
+static unsigned long __init xen_foreach_remap_area(unsigned long nr_pages,
+ unsigned long (*func)(unsigned long start_pfn, unsigned long end_pfn,
+ unsigned long nr_pages, unsigned long last_val))
{
phys_addr_t start = 0;
- unsigned long last_pfn = nr_pages;
+ unsigned long ret_val = 0;
const struct e820entry *entry = xen_e820_map;
int i;
/*
* Combine non-RAM regions and gaps until a RAM region (or the
- * end of the map) is reached, then set the 1:1 map and
- * remap the memory in those non-RAM regions.
+ * end of the map) is reached, then call the provided function
+ * to perform its duty on the non-RAM region.
*
* The combined non-RAM regions are rounded to a whole number
* of pages so any partial pages are accessible via the 1:1
@@ -466,14 +481,13 @@
end_pfn = PFN_UP(entry->addr);
if (start_pfn < end_pfn)
- last_pfn = xen_set_identity_and_remap_chunk(
- start_pfn, end_pfn, nr_pages,
- last_pfn);
+ ret_val = func(start_pfn, end_pfn, nr_pages,
+ ret_val);
start = end;
}
}
- pr_info("Released %ld page(s)\n", xen_released_pages);
+ return ret_val;
}
/*
@@ -596,35 +610,6 @@
}
}
-static unsigned long __init xen_count_remap_pages(unsigned long max_pfn)
-{
- unsigned long extra = 0;
- unsigned long start_pfn, end_pfn;
- const struct e820entry *entry = xen_e820_map;
- int i;
-
- end_pfn = 0;
- for (i = 0; i < xen_e820_map_entries; i++, entry++) {
- start_pfn = PFN_DOWN(entry->addr);
- /* Adjacent regions on non-page boundaries handling! */
- end_pfn = min(end_pfn, start_pfn);
-
- if (start_pfn >= max_pfn)
- return extra + max_pfn - end_pfn;
-
- /* Add any holes in map to result. */
- extra += start_pfn - end_pfn;
-
- end_pfn = PFN_UP(entry->addr + entry->size);
- end_pfn = min(end_pfn, max_pfn);
-
- if (entry->type != E820_RAM)
- extra += end_pfn - start_pfn;
- }
-
- return extra;
-}
-
bool __init xen_is_e820_reserved(phys_addr_t start, phys_addr_t size)
{
struct e820entry *entry;
@@ -804,7 +789,7 @@
max_pages = xen_get_max_pages();
/* How many extra pages do we need due to remapping? */
- max_pages += xen_count_remap_pages(max_pfn);
+ max_pages += xen_foreach_remap_area(max_pfn, xen_count_remap_pages);
if (max_pages > max_pfn)
extra_pages += max_pages - max_pfn;
@@ -922,7 +907,9 @@
* Set identity map on non-RAM pages and prepare remapping the
* underlying RAM.
*/
- xen_set_identity_and_remap(max_pfn);
+ xen_foreach_remap_area(max_pfn, xen_set_identity_and_remap_chunk);
+
+ pr_info("Released %ld page(s)\n", xen_released_pages);
return "Xen";
}
diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c
index a0a4e55..6deba5b 100644
--- a/arch/x86/xen/time.c
+++ b/arch/x86/xen/time.c
@@ -290,11 +290,11 @@
WARN_ON(!clockevent_state_oneshot(evt));
single.timeout_abs_ns = get_abs_timeout(delta);
- single.flags = VCPU_SSHOTTMR_future;
+ /* Get an event anyway, even if the timeout is already expired */
+ single.flags = 0;
ret = HYPERVISOR_vcpu_op(VCPUOP_set_singleshot_timer, cpu, &single);
-
- BUG_ON(ret != 0 && ret != -ETIME);
+ BUG_ON(ret != 0);
return ret;
}
diff --git a/arch/xtensa/Kconfig b/arch/xtensa/Kconfig
index 85257af..64336f6 100644
--- a/arch/xtensa/Kconfig
+++ b/arch/xtensa/Kconfig
@@ -14,6 +14,7 @@
select GENERIC_PCI_IOMAP
select GENERIC_SCHED_CLOCK
select HAVE_DMA_API_DEBUG
+ select HAVE_EXIT_THREAD
select HAVE_FUNCTION_TRACER
select HAVE_FUTEX_CMPXCHG if !MMU
select HAVE_HW_BREAKPOINT if PERF_EVENTS
diff --git a/arch/xtensa/configs/generic_kc705_defconfig b/arch/xtensa/configs/generic_kc705_defconfig
index f4b7b38..d9444f0 100644
--- a/arch/xtensa/configs/generic_kc705_defconfig
+++ b/arch/xtensa/configs/generic_kc705_defconfig
@@ -11,7 +11,6 @@
CONFIG_CGROUP_DEVICE=y
CONFIG_CPUSETS=y
CONFIG_CGROUP_CPUACCT=y
-CONFIG_RESOURCE_COUNTERS=y
CONFIG_MEMCG=y
CONFIG_NAMESPACES=y
CONFIG_SCHED_AUTOGROUP=y
diff --git a/arch/xtensa/configs/smp_lx200_defconfig b/arch/xtensa/configs/smp_lx200_defconfig
index 22eeacb..61f943c 100644
--- a/arch/xtensa/configs/smp_lx200_defconfig
+++ b/arch/xtensa/configs/smp_lx200_defconfig
@@ -11,7 +11,6 @@
CONFIG_CGROUP_DEVICE=y
CONFIG_CPUSETS=y
CONFIG_CGROUP_CPUACCT=y
-CONFIG_RESOURCE_COUNTERS=y
CONFIG_MEMCG=y
CONFIG_NAMESPACES=y
CONFIG_SCHED_AUTOGROUP=y
diff --git a/arch/xtensa/kernel/process.c b/arch/xtensa/kernel/process.c
index 5bbfed8..e0ded48 100644
--- a/arch/xtensa/kernel/process.c
+++ b/arch/xtensa/kernel/process.c
@@ -115,10 +115,10 @@
/*
* This is called when the thread calls exit().
*/
-void exit_thread(void)
+void exit_thread(struct task_struct *tsk)
{
#if XTENSA_HAVE_COPROCESSORS
- coprocessor_release_all(current_thread_info());
+ coprocessor_release_all(task_thread_info(tsk));
#endif
}
diff --git a/block/ioctl.c b/block/ioctl.c
index 4ff1f92..698c793 100644
--- a/block/ioctl.c
+++ b/block/ioctl.c
@@ -407,35 +407,6 @@
ret == -ENOIOCTLCMD;
}
-#ifdef CONFIG_FS_DAX
-bool blkdev_dax_capable(struct block_device *bdev)
-{
- struct gendisk *disk = bdev->bd_disk;
-
- if (!disk->fops->direct_access)
- return false;
-
- /*
- * If the partition is not aligned on a page boundary, we can't
- * do dax I/O to it.
- */
- if ((bdev->bd_part->start_sect % (PAGE_SIZE / 512))
- || (bdev->bd_part->nr_sects % (PAGE_SIZE / 512)))
- return false;
-
- /*
- * If the device has known bad blocks, force all I/O through the
- * driver / page cache.
- *
- * TODO: support finer grained dax error handling
- */
- if (disk->bb && disk->bb->count)
- return false;
-
- return true;
-}
-#endif
-
static int blkdev_flushbuf(struct block_device *bdev, fmode_t mode,
unsigned cmd, unsigned long arg)
{
@@ -598,9 +569,6 @@
case BLKTRACESETUP:
case BLKTRACETEARDOWN:
return blk_trace_ioctl(bdev, cmd, argp);
- case BLKDAXGET:
- return put_int(arg, !!(bdev->bd_inode->i_flags & S_DAX));
- break;
case IOC_PR_REGISTER:
return blkdev_pr_register(bdev, argp);
case IOC_PR_RESERVE:
diff --git a/block/partitions/ldm.c b/block/partitions/ldm.c
index e507cfb..edcea70 100644
--- a/block/partitions/ldm.c
+++ b/block/partitions/ldm.c
@@ -27,6 +27,8 @@
#include <linux/pagemap.h>
#include <linux/stringify.h>
#include <linux/kernel.h>
+#include <linux/uuid.h>
+
#include "ldm.h"
#include "check.h"
#include "msdos.h"
@@ -66,60 +68,6 @@
}
/**
- * ldm_parse_hexbyte - Convert a ASCII hex number to a byte
- * @src: Pointer to at least 2 characters to convert.
- *
- * Convert a two character ASCII hex string to a number.
- *
- * Return: 0-255 Success, the byte was parsed correctly
- * -1 Error, an invalid character was supplied
- */
-static int ldm_parse_hexbyte (const u8 *src)
-{
- unsigned int x; /* For correct wrapping */
- int h;
-
- /* high part */
- x = h = hex_to_bin(src[0]);
- if (h < 0)
- return -1;
-
- /* low part */
- h = hex_to_bin(src[1]);
- if (h < 0)
- return -1;
-
- return (x << 4) + h;
-}
-
-/**
- * ldm_parse_guid - Convert GUID from ASCII to binary
- * @src: 36 char string of the form fa50ff2b-f2e8-45de-83fa-65417f2f49ba
- * @dest: Memory block to hold binary GUID (16 bytes)
- *
- * N.B. The GUID need not be NULL terminated.
- *
- * Return: 'true' @dest contains binary GUID
- * 'false' @dest contents are undefined
- */
-static bool ldm_parse_guid (const u8 *src, u8 *dest)
-{
- static const int size[] = { 4, 2, 2, 2, 6 };
- int i, j, v;
-
- if (src[8] != '-' || src[13] != '-' ||
- src[18] != '-' || src[23] != '-')
- return false;
-
- for (j = 0; j < 5; j++, src++)
- for (i = 0; i < size[j]; i++, src+=2, *dest++ = v)
- if ((v = ldm_parse_hexbyte (src)) < 0)
- return false;
-
- return true;
-}
-
-/**
* ldm_parse_privhead - Read the LDM Database PRIVHEAD structure
* @data: Raw database PRIVHEAD structure loaded from the device
* @ph: In-memory privhead structure in which to return parsed information
@@ -167,7 +115,7 @@
ldm_error("PRIVHEAD disk size doesn't match real disk size");
return false;
}
- if (!ldm_parse_guid(data + 0x0030, ph->disk_id)) {
+ if (uuid_be_to_bin(data + 0x0030, (uuid_be *)ph->disk_id)) {
ldm_error("PRIVHEAD contains an invalid GUID.");
return false;
}
@@ -944,7 +892,7 @@
disk = &vb->vblk.disk;
ldm_get_vstr (buffer + 0x18 + r_diskid, disk->alt_name,
sizeof (disk->alt_name));
- if (!ldm_parse_guid (buffer + 0x19 + r_name, disk->disk_id))
+ if (uuid_be_to_bin(buffer + 0x19 + r_name, (uuid_be *)disk->disk_id))
return false;
return true;
diff --git a/certs/Kconfig b/certs/Kconfig
index f0f8a44..fc5955f 100644
--- a/certs/Kconfig
+++ b/certs/Kconfig
@@ -17,6 +17,7 @@
config SYSTEM_TRUSTED_KEYRING
bool "Provide system-wide ring of trusted keys"
depends on KEYS
+ depends on ASYMMETRIC_KEY_TYPE
help
Provide a system keyring to which trusted keys can be added. Keys in
the keyring are considered to be trusted. Keys may be added at will
@@ -55,4 +56,12 @@
This is the number of bytes reserved in the kernel image for a
certificate to be inserted.
+config SECONDARY_TRUSTED_KEYRING
+ bool "Provide a keyring to which extra trustable keys may be added"
+ depends on SYSTEM_TRUSTED_KEYRING
+ help
+ If set, provide a keyring to which extra keys may be added, provided
+ those keys are not blacklisted and are vouched for by a key built
+ into the kernel or already in the secondary trusted keyring.
+
endmenu
diff --git a/certs/system_keyring.c b/certs/system_keyring.c
index f418032..50979d6 100644
--- a/certs/system_keyring.c
+++ b/certs/system_keyring.c
@@ -18,29 +18,88 @@
#include <keys/system_keyring.h>
#include <crypto/pkcs7.h>
-struct key *system_trusted_keyring;
-EXPORT_SYMBOL_GPL(system_trusted_keyring);
+static struct key *builtin_trusted_keys;
+#ifdef CONFIG_SECONDARY_TRUSTED_KEYRING
+static struct key *secondary_trusted_keys;
+#endif
extern __initconst const u8 system_certificate_list[];
extern __initconst const unsigned long system_certificate_list_size;
+/**
+ * restrict_link_to_builtin_trusted - Restrict keyring addition by built in CA
+ *
+ * Restrict the addition of keys into a keyring based on the key-to-be-added
+ * being vouched for by a key in the built in system keyring.
+ */
+int restrict_link_by_builtin_trusted(struct key *keyring,
+ const struct key_type *type,
+ const union key_payload *payload)
+{
+ return restrict_link_by_signature(builtin_trusted_keys, type, payload);
+}
+
+#ifdef CONFIG_SECONDARY_TRUSTED_KEYRING
+/**
+ * restrict_link_by_builtin_and_secondary_trusted - Restrict keyring
+ * addition by both builtin and secondary keyrings
+ *
+ * Restrict the addition of keys into a keyring based on the key-to-be-added
+ * being vouched for by a key in either the built-in or the secondary system
+ * keyrings.
+ */
+int restrict_link_by_builtin_and_secondary_trusted(
+ struct key *keyring,
+ const struct key_type *type,
+ const union key_payload *payload)
+{
+ /* If we have a secondary trusted keyring, then that contains a link
+ * through to the builtin keyring and the search will follow that link.
+ */
+ if (type == &key_type_keyring &&
+ keyring == secondary_trusted_keys &&
+ payload == &builtin_trusted_keys->payload)
+ /* Allow the builtin keyring to be added to the secondary */
+ return 0;
+
+ return restrict_link_by_signature(secondary_trusted_keys, type, payload);
+}
+#endif
+
/*
- * Load the compiled-in keys
+ * Create the trusted keyrings
*/
static __init int system_trusted_keyring_init(void)
{
- pr_notice("Initialise system trusted keyring\n");
+ pr_notice("Initialise system trusted keyrings\n");
- system_trusted_keyring =
- keyring_alloc(".system_keyring",
+ builtin_trusted_keys =
+ keyring_alloc(".builtin_trusted_keys",
KUIDT_INIT(0), KGIDT_INIT(0), current_cred(),
((KEY_POS_ALL & ~KEY_POS_SETATTR) |
KEY_USR_VIEW | KEY_USR_READ | KEY_USR_SEARCH),
- KEY_ALLOC_NOT_IN_QUOTA, NULL);
- if (IS_ERR(system_trusted_keyring))
- panic("Can't allocate system trusted keyring\n");
+ KEY_ALLOC_NOT_IN_QUOTA,
+ NULL, NULL);
+ if (IS_ERR(builtin_trusted_keys))
+ panic("Can't allocate builtin trusted keyring\n");
- set_bit(KEY_FLAG_TRUSTED_ONLY, &system_trusted_keyring->flags);
+#ifdef CONFIG_SECONDARY_TRUSTED_KEYRING
+ secondary_trusted_keys =
+ keyring_alloc(".secondary_trusted_keys",
+ KUIDT_INIT(0), KGIDT_INIT(0), current_cred(),
+ ((KEY_POS_ALL & ~KEY_POS_SETATTR) |
+ KEY_USR_VIEW | KEY_USR_READ | KEY_USR_SEARCH |
+ KEY_USR_WRITE),
+ KEY_ALLOC_NOT_IN_QUOTA,
+ restrict_link_by_builtin_and_secondary_trusted,
+ NULL);
+ if (IS_ERR(secondary_trusted_keys))
+ panic("Can't allocate secondary trusted keyring\n");
+
+ if (key_link(secondary_trusted_keys, builtin_trusted_keys) < 0)
+ panic("Can't link trusted keyrings\n");
+#endif
+
return 0;
}
@@ -76,7 +135,7 @@
if (plen > end - p)
goto dodgy_cert;
- key = key_create_or_update(make_key_ref(system_trusted_keyring, 1),
+ key = key_create_or_update(make_key_ref(builtin_trusted_keys, 1),
"asymmetric",
NULL,
p,
@@ -84,8 +143,8 @@
((KEY_POS_ALL & ~KEY_POS_SETATTR) |
KEY_USR_VIEW | KEY_USR_READ),
KEY_ALLOC_NOT_IN_QUOTA |
- KEY_ALLOC_TRUSTED |
- KEY_ALLOC_BUILT_IN);
+ KEY_ALLOC_BUILT_IN |
+ KEY_ALLOC_BYPASS_RESTRICTION);
if (IS_ERR(key)) {
pr_err("Problem loading in-kernel X.509 certificate (%ld)\n",
PTR_ERR(key));
@@ -108,19 +167,27 @@
#ifdef CONFIG_SYSTEM_DATA_VERIFICATION
/**
- * Verify a PKCS#7-based signature on system data.
- * @data: The data to be verified.
+ * verify_pkcs7_signature - Verify a PKCS#7-based signature on system data.
+ * @data: The data to be verified (NULL if expecting internal data).
* @len: Size of @data.
* @raw_pkcs7: The PKCS#7 message that is the signature.
* @pkcs7_len: The size of @raw_pkcs7.
+ * @trusted_keys: Trusted keys to use (NULL for builtin trusted keys only,
+ * (void *)1UL for all trusted keys).
* @usage: The use to which the key is being put.
+ * @view_content: Callback to gain access to content.
+ * @ctx: Context for callback.
*/
-int system_verify_data(const void *data, unsigned long len,
- const void *raw_pkcs7, size_t pkcs7_len,
- enum key_being_used_for usage)
+int verify_pkcs7_signature(const void *data, size_t len,
+ const void *raw_pkcs7, size_t pkcs7_len,
+ struct key *trusted_keys,
+ enum key_being_used_for usage,
+ int (*view_content)(void *ctx,
+ const void *data, size_t len,
+ size_t asn1hdrlen),
+ void *ctx)
{
struct pkcs7_message *pkcs7;
- bool trusted;
int ret;
pkcs7 = pkcs7_parse_message(raw_pkcs7, pkcs7_len);
@@ -128,7 +195,7 @@
return PTR_ERR(pkcs7);
/* The data should be detached - so we need to supply it. */
- if (pkcs7_supply_detached_data(pkcs7, data, len) < 0) {
+ if (data && pkcs7_supply_detached_data(pkcs7, data, len) < 0) {
pr_err("PKCS#7 signature with non-detached data\n");
ret = -EBADMSG;
goto error;
@@ -138,13 +205,33 @@
if (ret < 0)
goto error;
- ret = pkcs7_validate_trust(pkcs7, system_trusted_keyring, &trusted);
- if (ret < 0)
+ if (!trusted_keys) {
+ trusted_keys = builtin_trusted_keys;
+ } else if (trusted_keys == (void *)1UL) {
+#ifdef CONFIG_SECONDARY_TRUSTED_KEYRING
+ trusted_keys = secondary_trusted_keys;
+#else
+ trusted_keys = builtin_trusted_keys;
+#endif
+ }
+ ret = pkcs7_validate_trust(pkcs7, trusted_keys);
+ if (ret < 0) {
+ if (ret == -ENOKEY)
+ pr_err("PKCS#7 signature not signed with a trusted key\n");
goto error;
+ }
- if (!trusted) {
- pr_err("PKCS#7 signature not signed with a trusted key\n");
- ret = -ENOKEY;
+ if (view_content) {
+ size_t asn1hdrlen;
+
+ ret = pkcs7_get_content_data(pkcs7, &data, &len, &asn1hdrlen);
+ if (ret < 0) {
+ if (ret == -ENODATA)
+ pr_devel("PKCS#7 message does not contain data\n");
+ goto error;
+ }
+
+ ret = view_content(ctx, data, len, asn1hdrlen);
}
error:
@@ -152,6 +239,6 @@
pr_devel("<==%s() = %d\n", __func__, ret);
return ret;
}
-EXPORT_SYMBOL_GPL(system_verify_data);
+EXPORT_SYMBOL_GPL(verify_pkcs7_signature);
#endif /* CONFIG_SYSTEM_DATA_VERIFICATION */
diff --git a/crypto/asymmetric_keys/Kconfig b/crypto/asymmetric_keys/Kconfig
index 91a7e04..e28e912 100644
--- a/crypto/asymmetric_keys/Kconfig
+++ b/crypto/asymmetric_keys/Kconfig
@@ -1,5 +1,5 @@
menuconfig ASYMMETRIC_KEY_TYPE
- tristate "Asymmetric (public-key cryptographic) key type"
+ bool "Asymmetric (public-key cryptographic) key type"
depends on KEYS
help
This option provides support for a key type that holds the data for
@@ -40,8 +40,7 @@
config PKCS7_TEST_KEY
tristate "PKCS#7 testing key type"
- depends on PKCS7_MESSAGE_PARSER
- select SYSTEM_TRUSTED_KEYRING
+ depends on SYSTEM_DATA_VERIFICATION
help
This option provides a type of key that can be loaded up from a
PKCS#7 message - provided the message is signed by a trusted key. If
@@ -54,6 +53,7 @@
config SIGNED_PE_FILE_VERIFICATION
bool "Support for PE file signature verification"
depends on PKCS7_MESSAGE_PARSER=y
+ depends on SYSTEM_DATA_VERIFICATION
select ASN1
select OID_REGISTRY
help
diff --git a/crypto/asymmetric_keys/Makefile b/crypto/asymmetric_keys/Makefile
index f904862..6516855 100644
--- a/crypto/asymmetric_keys/Makefile
+++ b/crypto/asymmetric_keys/Makefile
@@ -4,7 +4,10 @@
obj-$(CONFIG_ASYMMETRIC_KEY_TYPE) += asymmetric_keys.o
-asymmetric_keys-y := asymmetric_type.o signature.o
+asymmetric_keys-y := \
+ asymmetric_type.o \
+ restrict.o \
+ signature.o
obj-$(CONFIG_ASYMMETRIC_PUBLIC_KEY_SUBTYPE) += public_key.o
diff --git a/crypto/asymmetric_keys/asymmetric_keys.h b/crypto/asymmetric_keys/asymmetric_keys.h
index 1d450b5..ca8e9ac 100644
--- a/crypto/asymmetric_keys/asymmetric_keys.h
+++ b/crypto/asymmetric_keys/asymmetric_keys.h
@@ -9,6 +9,8 @@
* 2 of the Licence, or (at your option) any later version.
*/
+#include <keys/asymmetric-type.h>
+
extern struct asymmetric_key_id *asymmetric_key_hex_to_key_id(const char *id);
extern int __asymmetric_key_hex_to_key_id(const char *id,
diff --git a/crypto/asymmetric_keys/asymmetric_type.c b/crypto/asymmetric_keys/asymmetric_type.c
index 9f2165b..6600181 100644
--- a/crypto/asymmetric_keys/asymmetric_type.c
+++ b/crypto/asymmetric_keys/asymmetric_type.c
@@ -35,6 +35,95 @@
static DECLARE_RWSEM(asymmetric_key_parsers_sem);
/**
+ * find_asymmetric_key - Find a key by ID.
+ * @keyring: The keys to search.
+ * @id_0: The first ID to look for or NULL.
+ * @id_1: The second ID to look for or NULL.
+ * @partial: Use partial match if true, exact if false.
+ *
+ * Find a key in the given keyring by identifier. The preferred identifier is
+ * the id_0 and the fallback identifier is the id_1. If both are given, the
+ * lookup is by the former, but the latter must also match.
+ */
+struct key *find_asymmetric_key(struct key *keyring,
+ const struct asymmetric_key_id *id_0,
+ const struct asymmetric_key_id *id_1,
+ bool partial)
+{
+ struct key *key;
+ key_ref_t ref;
+ const char *lookup;
+ char *req, *p;
+ int len;
+
+ if (id_0) {
+ lookup = id_0->data;
+ len = id_0->len;
+ } else {
+ lookup = id_1->data;
+ len = id_1->len;
+ }
+
+ /* Construct an identifier "id:<keyid>". */
+ p = req = kmalloc(2 + 1 + len * 2 + 1, GFP_KERNEL);
+ if (!req)
+ return ERR_PTR(-ENOMEM);
+
+ if (partial) {
+ *p++ = 'i';
+ *p++ = 'd';
+ } else {
+ *p++ = 'e';
+ *p++ = 'x';
+ }
+ *p++ = ':';
+ p = bin2hex(p, lookup, len);
+ *p = 0;
+
+ pr_debug("Look up: \"%s\"\n", req);
+
+ ref = keyring_search(make_key_ref(keyring, 1),
+ &key_type_asymmetric, req);
+ if (IS_ERR(ref))
+ pr_debug("Request for key '%s' err %ld\n", req, PTR_ERR(ref));
+ kfree(req);
+
+ if (IS_ERR(ref)) {
+ switch (PTR_ERR(ref)) {
+ /* Hide some search errors */
+ case -EACCES:
+ case -ENOTDIR:
+ case -EAGAIN:
+ return ERR_PTR(-ENOKEY);
+ default:
+ return ERR_CAST(ref);
+ }
+ }
+
+ key = key_ref_to_ptr(ref);
+ if (id_0 && id_1) {
+ const struct asymmetric_key_ids *kids = asymmetric_key_ids(key);
+
+ if (!kids->id[0]) {
+ pr_debug("First ID matches, but second is missing\n");
+ goto reject;
+ }
+ if (!asymmetric_key_id_same(id_1, kids->id[1])) {
+ pr_debug("First ID matches, but second does not\n");
+ goto reject;
+ }
+ }
+
+ pr_devel("<==%s() = 0 [%x]\n", __func__, key_serial(key));
+ return key;
+
+reject:
+ key_put(key);
+ return ERR_PTR(-EKEYREJECTED);
+}
+EXPORT_SYMBOL_GPL(find_asymmetric_key);
+
+/**
* asymmetric_key_generate_id: Construct an asymmetric key ID
* @val_1: First binary blob
* @len_1: Length of first binary blob
@@ -331,7 +420,8 @@
pr_devel("==>%s()\n", __func__);
if (subtype) {
- subtype->destroy(prep->payload.data[asym_crypto]);
+ subtype->destroy(prep->payload.data[asym_crypto],
+ prep->payload.data[asym_auth]);
module_put(subtype->owner);
}
asymmetric_key_free_kids(kids);
@@ -346,13 +436,15 @@
struct asymmetric_key_subtype *subtype = asymmetric_key_subtype(key);
struct asymmetric_key_ids *kids = key->payload.data[asym_key_ids];
void *data = key->payload.data[asym_crypto];
+ void *auth = key->payload.data[asym_auth];
key->payload.data[asym_crypto] = NULL;
key->payload.data[asym_subtype] = NULL;
key->payload.data[asym_key_ids] = NULL;
+ key->payload.data[asym_auth] = NULL;
if (subtype) {
- subtype->destroy(data);
+ subtype->destroy(data, auth);
module_put(subtype->owner);
}
diff --git a/crypto/asymmetric_keys/mscode_parser.c b/crypto/asymmetric_keys/mscode_parser.c
index 3242cbf..6a76d5c 100644
--- a/crypto/asymmetric_keys/mscode_parser.c
+++ b/crypto/asymmetric_keys/mscode_parser.c
@@ -21,19 +21,13 @@
/*
* Parse a Microsoft Individual Code Signing blob
*/
-int mscode_parse(struct pefile_context *ctx)
+int mscode_parse(void *_ctx, const void *content_data, size_t data_len,
+ size_t asn1hdrlen)
{
- const void *content_data;
- size_t data_len;
- int ret;
+ struct pefile_context *ctx = _ctx;
- ret = pkcs7_get_content_data(ctx->pkcs7, &content_data, &data_len, 1);
-
- if (ret) {
- pr_debug("PKCS#7 message does not contain data\n");
- return ret;
- }
-
+ content_data -= asn1hdrlen;
+ data_len += asn1hdrlen;
pr_devel("Data: %zu [%*ph]\n", data_len, (unsigned)(data_len),
content_data);
@@ -129,7 +123,6 @@
{
struct pefile_context *ctx = context;
- ctx->digest = value;
- ctx->digest_len = vlen;
- return 0;
+ ctx->digest = kmemdup(value, vlen, GFP_KERNEL);
+ return ctx->digest ? 0 : -ENOMEM;
}
diff --git a/crypto/asymmetric_keys/pkcs7_key_type.c b/crypto/asymmetric_keys/pkcs7_key_type.c
index e2d0edb..1063b64 100644
--- a/crypto/asymmetric_keys/pkcs7_key_type.c
+++ b/crypto/asymmetric_keys/pkcs7_key_type.c
@@ -13,12 +13,9 @@
#include <linux/key.h>
#include <linux/err.h>
#include <linux/module.h>
+#include <linux/verification.h>
#include <linux/key-type.h>
-#include <keys/asymmetric-type.h>
-#include <crypto/pkcs7.h>
#include <keys/user-type.h>
-#include <keys/system_keyring.h>
-#include "pkcs7_parser.h"
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("PKCS#7 testing key type");
@@ -29,57 +26,44 @@
"Usage to specify when verifying the PKCS#7 message");
/*
+ * Retrieve the PKCS#7 message content.
+ */
+static int pkcs7_view_content(void *ctx, const void *data, size_t len,
+ size_t asn1hdrlen)
+{
+ struct key_preparsed_payload *prep = ctx;
+ const void *saved_prep_data;
+ size_t saved_prep_datalen;
+ int ret;
+
+ saved_prep_data = prep->data;
+ saved_prep_datalen = prep->datalen;
+ prep->data = data;
+ prep->datalen = len;
+
+ ret = user_preparse(prep);
+
+ prep->data = saved_prep_data;
+ prep->datalen = saved_prep_datalen;
+ return ret;
+}
+
+/*
* Preparse a PKCS#7 wrapped and validated data blob.
*/
static int pkcs7_preparse(struct key_preparsed_payload *prep)
{
enum key_being_used_for usage = pkcs7_usage;
- struct pkcs7_message *pkcs7;
- const void *data, *saved_prep_data;
- size_t datalen, saved_prep_datalen;
- bool trusted;
- int ret;
-
- kenter("");
if (usage >= NR__KEY_BEING_USED_FOR) {
pr_err("Invalid usage type %d\n", usage);
return -EINVAL;
}
- saved_prep_data = prep->data;
- saved_prep_datalen = prep->datalen;
- pkcs7 = pkcs7_parse_message(saved_prep_data, saved_prep_datalen);
- if (IS_ERR(pkcs7)) {
- ret = PTR_ERR(pkcs7);
- goto error;
- }
-
- ret = pkcs7_verify(pkcs7, usage);
- if (ret < 0)
- goto error_free;
-
- ret = pkcs7_validate_trust(pkcs7, system_trusted_keyring, &trusted);
- if (ret < 0)
- goto error_free;
- if (!trusted)
- pr_warn("PKCS#7 message doesn't chain back to a trusted key\n");
-
- ret = pkcs7_get_content_data(pkcs7, &data, &datalen, false);
- if (ret < 0)
- goto error_free;
-
- prep->data = data;
- prep->datalen = datalen;
- ret = user_preparse(prep);
- prep->data = saved_prep_data;
- prep->datalen = saved_prep_datalen;
-
-error_free:
- pkcs7_free_message(pkcs7);
-error:
- kleave(" = %d", ret);
- return ret;
+ return verify_pkcs7_signature(NULL, 0,
+ prep->data, prep->datalen,
+ (void *)1UL, usage,
+ pkcs7_view_content, prep);
}
/*
diff --git a/crypto/asymmetric_keys/pkcs7_parser.c b/crypto/asymmetric_keys/pkcs7_parser.c
index bdd0d753..af4cd86 100644
--- a/crypto/asymmetric_keys/pkcs7_parser.c
+++ b/crypto/asymmetric_keys/pkcs7_parser.c
@@ -44,9 +44,7 @@
static void pkcs7_free_signed_info(struct pkcs7_signed_info *sinfo)
{
if (sinfo) {
- kfree(sinfo->sig.s);
- kfree(sinfo->sig.digest);
- kfree(sinfo->signing_cert_id);
+ public_key_signature_free(sinfo->sig);
kfree(sinfo);
}
}
@@ -125,6 +123,10 @@
ctx->sinfo = kzalloc(sizeof(struct pkcs7_signed_info), GFP_KERNEL);
if (!ctx->sinfo)
goto out_no_sinfo;
+ ctx->sinfo->sig = kzalloc(sizeof(struct public_key_signature),
+ GFP_KERNEL);
+ if (!ctx->sinfo->sig)
+ goto out_no_sig;
ctx->data = (unsigned long)data;
ctx->ppcerts = &ctx->certs;
@@ -150,6 +152,7 @@
ctx->certs = cert->next;
x509_free_certificate(cert);
}
+out_no_sig:
pkcs7_free_signed_info(ctx->sinfo);
out_no_sinfo:
pkcs7_free_message(ctx->msg);
@@ -165,24 +168,25 @@
* @pkcs7: The preparsed PKCS#7 message to access
* @_data: Place to return a pointer to the data
* @_data_len: Place to return the data length
- * @want_wrapper: True if the ASN.1 object header should be included in the data
+ * @_headerlen: Size of ASN.1 header not included in _data
*
- * Get access to the data content of the PKCS#7 message, including, optionally,
- * the header of the ASN.1 object that contains it. Returns -ENODATA if the
- * data object was missing from the message.
+ * Get access to the data content of the PKCS#7 message. The size of the
+ * header of the ASN.1 object that contains it is also provided and can be used
+ * to adjust *_data and *_data_len to get the entire object.
+ *
+ * Returns -ENODATA if the data object was missing from the message.
*/
int pkcs7_get_content_data(const struct pkcs7_message *pkcs7,
const void **_data, size_t *_data_len,
- bool want_wrapper)
+ size_t *_headerlen)
{
- size_t wrapper;
-
if (!pkcs7->data)
return -ENODATA;
- wrapper = want_wrapper ? pkcs7->data_hdrlen : 0;
- *_data = pkcs7->data - wrapper;
- *_data_len = pkcs7->data_len + wrapper;
+ *_data = pkcs7->data;
+ *_data_len = pkcs7->data_len;
+ if (_headerlen)
+ *_headerlen = pkcs7->data_hdrlen;
return 0;
}
EXPORT_SYMBOL_GPL(pkcs7_get_content_data);
@@ -218,25 +222,25 @@
switch (ctx->last_oid) {
case OID_md4:
- ctx->sinfo->sig.hash_algo = "md4";
+ ctx->sinfo->sig->hash_algo = "md4";
break;
case OID_md5:
- ctx->sinfo->sig.hash_algo = "md5";
+ ctx->sinfo->sig->hash_algo = "md5";
break;
case OID_sha1:
- ctx->sinfo->sig.hash_algo = "sha1";
+ ctx->sinfo->sig->hash_algo = "sha1";
break;
case OID_sha256:
- ctx->sinfo->sig.hash_algo = "sha256";
+ ctx->sinfo->sig->hash_algo = "sha256";
break;
case OID_sha384:
- ctx->sinfo->sig.hash_algo = "sha384";
+ ctx->sinfo->sig->hash_algo = "sha384";
break;
case OID_sha512:
- ctx->sinfo->sig.hash_algo = "sha512";
+ ctx->sinfo->sig->hash_algo = "sha512";
break;
case OID_sha224:
- ctx->sinfo->sig.hash_algo = "sha224";
+ ctx->sinfo->sig->hash_algo = "sha224";
break;
default:
printk("Unsupported digest algo: %u\n", ctx->last_oid);
@@ -256,7 +260,7 @@
switch (ctx->last_oid) {
case OID_rsaEncryption:
- ctx->sinfo->sig.pkey_algo = "rsa";
+ ctx->sinfo->sig->pkey_algo = "rsa";
break;
default:
printk("Unsupported pkey algo: %u\n", ctx->last_oid);
@@ -616,11 +620,11 @@
{
struct pkcs7_parse_context *ctx = context;
- ctx->sinfo->sig.s = kmemdup(value, vlen, GFP_KERNEL);
- if (!ctx->sinfo->sig.s)
+ ctx->sinfo->sig->s = kmemdup(value, vlen, GFP_KERNEL);
+ if (!ctx->sinfo->sig->s)
return -ENOMEM;
- ctx->sinfo->sig.s_size = vlen;
+ ctx->sinfo->sig->s_size = vlen;
return 0;
}
@@ -656,12 +660,16 @@
pr_devel("SINFO KID: %u [%*phN]\n", kid->len, kid->len, kid->data);
- sinfo->signing_cert_id = kid;
+ sinfo->sig->auth_ids[0] = kid;
sinfo->index = ++ctx->sinfo_index;
*ctx->ppsinfo = sinfo;
ctx->ppsinfo = &sinfo->next;
ctx->sinfo = kzalloc(sizeof(struct pkcs7_signed_info), GFP_KERNEL);
if (!ctx->sinfo)
return -ENOMEM;
+ ctx->sinfo->sig = kzalloc(sizeof(struct public_key_signature),
+ GFP_KERNEL);
+ if (!ctx->sinfo->sig)
+ return -ENOMEM;
return 0;
}
diff --git a/crypto/asymmetric_keys/pkcs7_parser.h b/crypto/asymmetric_keys/pkcs7_parser.h
index a66b19e..f4e8107 100644
--- a/crypto/asymmetric_keys/pkcs7_parser.h
+++ b/crypto/asymmetric_keys/pkcs7_parser.h
@@ -22,7 +22,6 @@
struct pkcs7_signed_info *next;
struct x509_certificate *signer; /* Signing certificate (in msg->certs) */
unsigned index;
- bool trusted;
bool unsupported_crypto; /* T if not usable due to missing crypto */
/* Message digest - the digest of the Content Data (or NULL) */
@@ -41,19 +40,17 @@
#define sinfo_has_ms_statement_type 5
time64_t signing_time;
- /* Issuing cert serial number and issuer's name [PKCS#7 or CMS ver 1]
- * or issuing cert's SKID [CMS ver 3].
- */
- struct asymmetric_key_id *signing_cert_id;
-
/* Message signature.
*
* This contains the generated digest of _either_ the Content Data or
* the Authenticated Attributes [RFC2315 9.3]. If the latter, one of
* the attributes contains the digest of the the Content Data within
* it.
+ *
+ * THis also contains the issuing cert serial number and issuer's name
+ * [PKCS#7 or CMS ver 1] or issuing cert's SKID [CMS ver 3].
*/
- struct public_key_signature sig;
+ struct public_key_signature *sig;
};
struct pkcs7_message {
diff --git a/crypto/asymmetric_keys/pkcs7_trust.c b/crypto/asymmetric_keys/pkcs7_trust.c
index 7d7a39b4..f6a009d 100644
--- a/crypto/asymmetric_keys/pkcs7_trust.c
+++ b/crypto/asymmetric_keys/pkcs7_trust.c
@@ -27,10 +27,9 @@
struct pkcs7_signed_info *sinfo,
struct key *trust_keyring)
{
- struct public_key_signature *sig = &sinfo->sig;
+ struct public_key_signature *sig = sinfo->sig;
struct x509_certificate *x509, *last = NULL, *p;
struct key *key;
- bool trusted;
int ret;
kenter(",%u,", sinfo->index);
@@ -42,10 +41,8 @@
for (x509 = sinfo->signer; x509; x509 = x509->signer) {
if (x509->seen) {
- if (x509->verified) {
- trusted = x509->trusted;
+ if (x509->verified)
goto verified;
- }
kleave(" = -ENOKEY [cached]");
return -ENOKEY;
}
@@ -54,9 +51,8 @@
/* Look to see if this certificate is present in the trusted
* keys.
*/
- key = x509_request_asymmetric_key(trust_keyring,
- x509->id, x509->skid,
- false);
+ key = find_asymmetric_key(trust_keyring,
+ x509->id, x509->skid, false);
if (!IS_ERR(key)) {
/* One of the X.509 certificates in the PKCS#7 message
* is apparently the same as one we already trust.
@@ -80,17 +76,17 @@
might_sleep();
last = x509;
- sig = &last->sig;
+ sig = last->sig;
}
/* No match - see if the root certificate has a signer amongst the
* trusted keys.
*/
- if (last && (last->akid_id || last->akid_skid)) {
- key = x509_request_asymmetric_key(trust_keyring,
- last->akid_id,
- last->akid_skid,
- false);
+ if (last && (last->sig->auth_ids[0] || last->sig->auth_ids[1])) {
+ key = find_asymmetric_key(trust_keyring,
+ last->sig->auth_ids[0],
+ last->sig->auth_ids[1],
+ false);
if (!IS_ERR(key)) {
x509 = last;
pr_devel("sinfo %u: Root cert %u signer is key %x\n",
@@ -104,10 +100,8 @@
/* As a last resort, see if we have a trusted public key that matches
* the signed info directly.
*/
- key = x509_request_asymmetric_key(trust_keyring,
- sinfo->signing_cert_id,
- NULL,
- false);
+ key = find_asymmetric_key(trust_keyring,
+ sinfo->sig->auth_ids[0], NULL, false);
if (!IS_ERR(key)) {
pr_devel("sinfo %u: Direct signer is key %x\n",
sinfo->index, key_serial(key));
@@ -122,7 +116,6 @@
matched:
ret = verify_signature(key, sig);
- trusted = test_bit(KEY_FLAG_TRUSTED, &key->flags);
key_put(key);
if (ret < 0) {
if (ret == -ENOMEM)
@@ -134,12 +127,9 @@
verified:
if (x509) {
x509->verified = true;
- for (p = sinfo->signer; p != x509; p = p->signer) {
+ for (p = sinfo->signer; p != x509; p = p->signer)
p->verified = true;
- p->trusted = trusted;
- }
}
- sinfo->trusted = trusted;
kleave(" = 0");
return 0;
}
@@ -148,7 +138,6 @@
* pkcs7_validate_trust - Validate PKCS#7 trust chain
* @pkcs7: The PKCS#7 certificate to validate
* @trust_keyring: Signing certificates to use as starting points
- * @_trusted: Set to true if trustworth, false otherwise
*
* Validate that the certificate chain inside the PKCS#7 message intersects
* keys we already know and trust.
@@ -170,16 +159,13 @@
* May also return -ENOMEM.
*/
int pkcs7_validate_trust(struct pkcs7_message *pkcs7,
- struct key *trust_keyring,
- bool *_trusted)
+ struct key *trust_keyring)
{
struct pkcs7_signed_info *sinfo;
struct x509_certificate *p;
int cached_ret = -ENOKEY;
int ret;
- *_trusted = false;
-
for (p = pkcs7->certs; p; p = p->next)
p->seen = false;
@@ -193,7 +179,6 @@
cached_ret = -ENOPKG;
continue;
case 0:
- *_trusted |= sinfo->trusted;
cached_ret = 0;
continue;
default:
diff --git a/crypto/asymmetric_keys/pkcs7_verify.c b/crypto/asymmetric_keys/pkcs7_verify.c
index 50be2a1..44b746e 100644
--- a/crypto/asymmetric_keys/pkcs7_verify.c
+++ b/crypto/asymmetric_keys/pkcs7_verify.c
@@ -25,34 +25,36 @@
static int pkcs7_digest(struct pkcs7_message *pkcs7,
struct pkcs7_signed_info *sinfo)
{
+ struct public_key_signature *sig = sinfo->sig;
struct crypto_shash *tfm;
struct shash_desc *desc;
- size_t digest_size, desc_size;
- void *digest;
+ size_t desc_size;
int ret;
- kenter(",%u,%s", sinfo->index, sinfo->sig.hash_algo);
+ kenter(",%u,%s", sinfo->index, sinfo->sig->hash_algo);
- if (!sinfo->sig.hash_algo)
+ if (!sinfo->sig->hash_algo)
return -ENOPKG;
/* Allocate the hashing algorithm we're going to need and find out how
* big the hash operational data will be.
*/
- tfm = crypto_alloc_shash(sinfo->sig.hash_algo, 0, 0);
+ tfm = crypto_alloc_shash(sinfo->sig->hash_algo, 0, 0);
if (IS_ERR(tfm))
return (PTR_ERR(tfm) == -ENOENT) ? -ENOPKG : PTR_ERR(tfm);
desc_size = crypto_shash_descsize(tfm) + sizeof(*desc);
- sinfo->sig.digest_size = digest_size = crypto_shash_digestsize(tfm);
+ sig->digest_size = crypto_shash_digestsize(tfm);
ret = -ENOMEM;
- digest = kzalloc(ALIGN(digest_size, __alignof__(*desc)) + desc_size,
- GFP_KERNEL);
- if (!digest)
+ sig->digest = kmalloc(sig->digest_size, GFP_KERNEL);
+ if (!sig->digest)
goto error_no_desc;
- desc = PTR_ALIGN(digest + digest_size, __alignof__(*desc));
+ desc = kzalloc(desc_size, GFP_KERNEL);
+ if (!desc)
+ goto error_no_desc;
+
desc->tfm = tfm;
desc->flags = CRYPTO_TFM_REQ_MAY_SLEEP;
@@ -60,10 +62,11 @@
ret = crypto_shash_init(desc);
if (ret < 0)
goto error;
- ret = crypto_shash_finup(desc, pkcs7->data, pkcs7->data_len, digest);
+ ret = crypto_shash_finup(desc, pkcs7->data, pkcs7->data_len,
+ sig->digest);
if (ret < 0)
goto error;
- pr_devel("MsgDigest = [%*ph]\n", 8, digest);
+ pr_devel("MsgDigest = [%*ph]\n", 8, sig->digest);
/* However, if there are authenticated attributes, there must be a
* message digest attribute amongst them which corresponds to the
@@ -78,14 +81,15 @@
goto error;
}
- if (sinfo->msgdigest_len != sinfo->sig.digest_size) {
+ if (sinfo->msgdigest_len != sig->digest_size) {
pr_debug("Sig %u: Invalid digest size (%u)\n",
sinfo->index, sinfo->msgdigest_len);
ret = -EBADMSG;
goto error;
}
- if (memcmp(digest, sinfo->msgdigest, sinfo->msgdigest_len) != 0) {
+ if (memcmp(sig->digest, sinfo->msgdigest,
+ sinfo->msgdigest_len) != 0) {
pr_debug("Sig %u: Message digest doesn't match\n",
sinfo->index);
ret = -EKEYREJECTED;
@@ -97,7 +101,7 @@
* convert the attributes from a CONT.0 into a SET before we
* hash it.
*/
- memset(digest, 0, sinfo->sig.digest_size);
+ memset(sig->digest, 0, sig->digest_size);
ret = crypto_shash_init(desc);
if (ret < 0)
@@ -107,17 +111,14 @@
if (ret < 0)
goto error;
ret = crypto_shash_finup(desc, sinfo->authattrs,
- sinfo->authattrs_len, digest);
+ sinfo->authattrs_len, sig->digest);
if (ret < 0)
goto error;
- pr_devel("AADigest = [%*ph]\n", 8, digest);
+ pr_devel("AADigest = [%*ph]\n", 8, sig->digest);
}
- sinfo->sig.digest = digest;
- digest = NULL;
-
error:
- kfree(digest);
+ kfree(desc);
error_no_desc:
crypto_free_shash(tfm);
kleave(" = %d", ret);
@@ -144,12 +145,12 @@
* PKCS#7 message - but I can't be 100% sure of that. It's
* possible this will need element-by-element comparison.
*/
- if (!asymmetric_key_id_same(x509->id, sinfo->signing_cert_id))
+ if (!asymmetric_key_id_same(x509->id, sinfo->sig->auth_ids[0]))
continue;
pr_devel("Sig %u: Found cert serial match X.509[%u]\n",
sinfo->index, certix);
- if (x509->pub->pkey_algo != sinfo->sig.pkey_algo) {
+ if (x509->pub->pkey_algo != sinfo->sig->pkey_algo) {
pr_warn("Sig %u: X.509 algo and PKCS#7 sig algo don't match\n",
sinfo->index);
continue;
@@ -164,7 +165,7 @@
*/
pr_debug("Sig %u: Issuing X.509 cert not found (#%*phN)\n",
sinfo->index,
- sinfo->signing_cert_id->len, sinfo->signing_cert_id->data);
+ sinfo->sig->auth_ids[0]->len, sinfo->sig->auth_ids[0]->data);
return 0;
}
@@ -174,6 +175,7 @@
static int pkcs7_verify_sig_chain(struct pkcs7_message *pkcs7,
struct pkcs7_signed_info *sinfo)
{
+ struct public_key_signature *sig;
struct x509_certificate *x509 = sinfo->signer, *p;
struct asymmetric_key_id *auth;
int ret;
@@ -188,34 +190,26 @@
x509->subject,
x509->raw_serial_size, x509->raw_serial);
x509->seen = true;
- ret = x509_get_sig_params(x509);
- if (ret < 0)
- goto maybe_missing_crypto_in_x509;
+ if (x509->unsupported_key)
+ goto unsupported_crypto_in_x509;
pr_debug("- issuer %s\n", x509->issuer);
- if (x509->akid_id)
+ sig = x509->sig;
+ if (sig->auth_ids[0])
pr_debug("- authkeyid.id %*phN\n",
- x509->akid_id->len, x509->akid_id->data);
- if (x509->akid_skid)
+ sig->auth_ids[0]->len, sig->auth_ids[0]->data);
+ if (sig->auth_ids[1])
pr_debug("- authkeyid.skid %*phN\n",
- x509->akid_skid->len, x509->akid_skid->data);
+ sig->auth_ids[1]->len, sig->auth_ids[1]->data);
- if ((!x509->akid_id && !x509->akid_skid) ||
- strcmp(x509->subject, x509->issuer) == 0) {
+ if (x509->self_signed) {
/* If there's no authority certificate specified, then
* the certificate must be self-signed and is the root
* of the chain. Likewise if the cert is its own
* authority.
*/
- pr_debug("- no auth?\n");
- if (x509->raw_subject_size != x509->raw_issuer_size ||
- memcmp(x509->raw_subject, x509->raw_issuer,
- x509->raw_issuer_size) != 0)
- return 0;
-
- ret = x509_check_signature(x509->pub, x509);
- if (ret < 0)
- goto maybe_missing_crypto_in_x509;
+ if (x509->unsupported_sig)
+ goto unsupported_crypto_in_x509;
x509->signer = x509;
pr_debug("- self-signed\n");
return 0;
@@ -224,7 +218,7 @@
/* Look through the X.509 certificates in the PKCS#7 message's
* list to see if the next one is there.
*/
- auth = x509->akid_id;
+ auth = sig->auth_ids[0];
if (auth) {
pr_debug("- want %*phN\n", auth->len, auth->data);
for (p = pkcs7->certs; p; p = p->next) {
@@ -234,7 +228,7 @@
goto found_issuer_check_skid;
}
} else {
- auth = x509->akid_skid;
+ auth = sig->auth_ids[1];
pr_debug("- want %*phN\n", auth->len, auth->data);
for (p = pkcs7->certs; p; p = p->next) {
if (!p->skid)
@@ -254,8 +248,8 @@
/* We matched issuer + serialNumber, but if there's an
* authKeyId.keyId, that must match the CA subjKeyId also.
*/
- if (x509->akid_skid &&
- !asymmetric_key_id_same(p->skid, x509->akid_skid)) {
+ if (sig->auth_ids[1] &&
+ !asymmetric_key_id_same(p->skid, sig->auth_ids[1])) {
pr_warn("Sig %u: X.509 chain contains auth-skid nonmatch (%u->%u)\n",
sinfo->index, x509->index, p->index);
return -EKEYREJECTED;
@@ -267,7 +261,7 @@
sinfo->index);
return 0;
}
- ret = x509_check_signature(p->pub, x509);
+ ret = public_key_verify_signature(p->pub, p->sig);
if (ret < 0)
return ret;
x509->signer = p;
@@ -279,16 +273,14 @@
might_sleep();
}
-maybe_missing_crypto_in_x509:
+unsupported_crypto_in_x509:
/* Just prune the certificate chain at this point if we lack some
* crypto module to go further. Note, however, we don't want to set
- * sinfo->missing_crypto as the signed info block may still be
+ * sinfo->unsupported_crypto as the signed info block may still be
* validatable against an X.509 cert lower in the chain that we have a
* trusted copy of.
*/
- if (ret == -ENOPKG)
- return 0;
- return ret;
+ return 0;
}
/*
@@ -332,7 +324,7 @@
}
/* Verify the PKCS#7 binary against the key */
- ret = public_key_verify_signature(sinfo->signer->pub, &sinfo->sig);
+ ret = public_key_verify_signature(sinfo->signer->pub, sinfo->sig);
if (ret < 0)
return ret;
@@ -375,9 +367,8 @@
enum key_being_used_for usage)
{
struct pkcs7_signed_info *sinfo;
- struct x509_certificate *x509;
int enopkg = -ENOPKG;
- int ret, n;
+ int ret;
kenter("");
@@ -419,12 +410,6 @@
return -EINVAL;
}
- for (n = 0, x509 = pkcs7->certs; x509; x509 = x509->next, n++) {
- ret = x509_get_sig_params(x509);
- if (ret < 0)
- return ret;
- }
-
for (sinfo = pkcs7->signed_infos; sinfo; sinfo = sinfo->next) {
ret = pkcs7_verify_one(pkcs7, sinfo);
if (ret < 0) {
diff --git a/crypto/asymmetric_keys/public_key.c b/crypto/asymmetric_keys/public_key.c
index 0f8b264..fd76b5f 100644
--- a/crypto/asymmetric_keys/public_key.c
+++ b/crypto/asymmetric_keys/public_key.c
@@ -39,15 +39,23 @@
/*
* Destroy a public key algorithm key.
*/
-void public_key_destroy(void *payload)
+void public_key_free(struct public_key *key)
{
- struct public_key *key = payload;
-
- if (key)
+ if (key) {
kfree(key->key);
- kfree(key);
+ kfree(key);
+ }
}
-EXPORT_SYMBOL_GPL(public_key_destroy);
+EXPORT_SYMBOL_GPL(public_key_free);
+
+/*
+ * Destroy a public key algorithm key.
+ */
+static void public_key_destroy(void *payload0, void *payload3)
+{
+ public_key_free(payload0);
+ public_key_signature_free(payload3);
+}
struct public_key_completion {
struct completion completion;
diff --git a/crypto/asymmetric_keys/restrict.c b/crypto/asymmetric_keys/restrict.c
new file mode 100644
index 0000000..ac4bddf
--- /dev/null
+++ b/crypto/asymmetric_keys/restrict.c
@@ -0,0 +1,108 @@
+/* Instantiate a public key crypto key from an X.509 Certificate
+ *
+ * Copyright (C) 2012, 2016 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public Licence
+ * as published by the Free Software Foundation; either version
+ * 2 of the Licence, or (at your option) any later version.
+ */
+
+#define pr_fmt(fmt) "ASYM: "fmt
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/err.h>
+#include <crypto/public_key.h>
+#include "asymmetric_keys.h"
+
+static bool use_builtin_keys;
+static struct asymmetric_key_id *ca_keyid;
+
+#ifndef MODULE
+static struct {
+ struct asymmetric_key_id id;
+ unsigned char data[10];
+} cakey;
+
+static int __init ca_keys_setup(char *str)
+{
+ if (!str) /* default system keyring */
+ return 1;
+
+ if (strncmp(str, "id:", 3) == 0) {
+ struct asymmetric_key_id *p = &cakey.id;
+ size_t hexlen = (strlen(str) - 3) / 2;
+ int ret;
+
+ if (hexlen == 0 || hexlen > sizeof(cakey.data)) {
+ pr_err("Missing or invalid ca_keys id\n");
+ return 1;
+ }
+
+ ret = __asymmetric_key_hex_to_key_id(str + 3, p, hexlen);
+ if (ret < 0)
+ pr_err("Unparsable ca_keys id hex string\n");
+ else
+ ca_keyid = p; /* owner key 'id:xxxxxx' */
+ } else if (strcmp(str, "builtin") == 0) {
+ use_builtin_keys = true;
+ }
+
+ return 1;
+}
+__setup("ca_keys=", ca_keys_setup);
+#endif
+
+/**
+ * restrict_link_by_signature - Restrict additions to a ring of public keys
+ * @trust_keyring: A ring of keys that can be used to vouch for the new cert.
+ * @type: The type of key being added.
+ * @payload: The payload of the new key.
+ *
+ * Check the new certificate against the ones in the trust keyring. If one of
+ * those is the signing key and validates the new certificate, then mark the
+ * new certificate as being trusted.
+ *
+ * Returns 0 if the new certificate was accepted, -ENOKEY if we couldn't find a
+ * matching parent certificate in the trusted list, -EKEYREJECTED if the
+ * signature check fails or the key is blacklisted and some other error if
+ * there is a matching certificate but the signature check cannot be performed.
+ */
+int restrict_link_by_signature(struct key *trust_keyring,
+ const struct key_type *type,
+ const union key_payload *payload)
+{
+ const struct public_key_signature *sig;
+ struct key *key;
+ int ret;
+
+ pr_devel("==>%s()\n", __func__);
+
+ if (!trust_keyring)
+ return -ENOKEY;
+
+ if (type != &key_type_asymmetric)
+ return -EOPNOTSUPP;
+
+ sig = payload->data[asym_auth];
+ if (!sig->auth_ids[0] && !sig->auth_ids[1])
+ return 0;
+
+ if (ca_keyid && !asymmetric_key_id_partial(sig->auth_ids[1], ca_keyid))
+ return -EPERM;
+
+ /* See if we have a key that signed this one. */
+ key = find_asymmetric_key(trust_keyring,
+ sig->auth_ids[0], sig->auth_ids[1],
+ false);
+ if (IS_ERR(key))
+ return -ENOKEY;
+
+ if (use_builtin_keys && !test_bit(KEY_FLAG_BUILTIN, &key->flags))
+ ret = -ENOKEY;
+ else
+ ret = verify_signature(key, sig);
+ key_put(key);
+ return ret;
+}
diff --git a/crypto/asymmetric_keys/signature.c b/crypto/asymmetric_keys/signature.c
index 004d5fc..11b7ba17 100644
--- a/crypto/asymmetric_keys/signature.c
+++ b/crypto/asymmetric_keys/signature.c
@@ -15,9 +15,27 @@
#include <keys/asymmetric-subtype.h>
#include <linux/export.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include <crypto/public_key.h>
#include "asymmetric_keys.h"
+/*
+ * Destroy a public key signature.
+ */
+void public_key_signature_free(struct public_key_signature *sig)
+{
+ int i;
+
+ if (sig) {
+ for (i = 0; i < ARRAY_SIZE(sig->auth_ids); i++)
+ kfree(sig->auth_ids[i]);
+ kfree(sig->s);
+ kfree(sig->digest);
+ kfree(sig);
+ }
+}
+EXPORT_SYMBOL_GPL(public_key_signature_free);
+
/**
* verify_signature - Initiate the use of an asymmetric key to verify a signature
* @key: The asymmetric key to verify against
diff --git a/crypto/asymmetric_keys/verify_pefile.c b/crypto/asymmetric_keys/verify_pefile.c
index 7e8c233..672a94c 100644
--- a/crypto/asymmetric_keys/verify_pefile.c
+++ b/crypto/asymmetric_keys/verify_pefile.c
@@ -16,7 +16,7 @@
#include <linux/err.h>
#include <linux/pe.h>
#include <linux/asn1.h>
-#include <crypto/pkcs7.h>
+#include <linux/verification.h>
#include <crypto/hash.h>
#include "verify_pefile.h"
@@ -392,9 +392,8 @@
* verify_pefile_signature - Verify the signature on a PE binary image
* @pebuf: Buffer containing the PE binary image
* @pelen: Length of the binary image
- * @trust_keyring: Signing certificates to use as starting points
+ * @trust_keys: Signing certificate(s) to use as starting points
* @usage: The use to which the key is being put.
- * @_trusted: Set to true if trustworth, false otherwise
*
* Validate that the certificate chain inside the PKCS#7 message inside the PE
* binary image intersects keys we already know and trust.
@@ -418,14 +417,10 @@
* May also return -ENOMEM.
*/
int verify_pefile_signature(const void *pebuf, unsigned pelen,
- struct key *trusted_keyring,
- enum key_being_used_for usage,
- bool *_trusted)
+ struct key *trusted_keys,
+ enum key_being_used_for usage)
{
- struct pkcs7_message *pkcs7;
struct pefile_context ctx;
- const void *data;
- size_t datalen;
int ret;
kenter("");
@@ -439,19 +434,10 @@
if (ret < 0)
return ret;
- pkcs7 = pkcs7_parse_message(pebuf + ctx.sig_offset, ctx.sig_len);
- if (IS_ERR(pkcs7))
- return PTR_ERR(pkcs7);
- ctx.pkcs7 = pkcs7;
-
- ret = pkcs7_get_content_data(ctx.pkcs7, &data, &datalen, false);
- if (ret < 0 || datalen == 0) {
- pr_devel("PKCS#7 message does not contain data\n");
- ret = -EBADMSG;
- goto error;
- }
-
- ret = mscode_parse(&ctx);
+ ret = verify_pkcs7_signature(NULL, 0,
+ pebuf + ctx.sig_offset, ctx.sig_len,
+ trusted_keys, usage,
+ mscode_parse, &ctx);
if (ret < 0)
goto error;
@@ -462,16 +448,8 @@
* contents.
*/
ret = pefile_digest_pe(pebuf, pelen, &ctx);
- if (ret < 0)
- goto error;
-
- ret = pkcs7_verify(pkcs7, usage);
- if (ret < 0)
- goto error;
-
- ret = pkcs7_validate_trust(pkcs7, trusted_keyring, _trusted);
error:
- pkcs7_free_message(ctx.pkcs7);
+ kfree(ctx.digest);
return ret;
}
diff --git a/crypto/asymmetric_keys/verify_pefile.h b/crypto/asymmetric_keys/verify_pefile.h
index a133eb8..cd4d209 100644
--- a/crypto/asymmetric_keys/verify_pefile.h
+++ b/crypto/asymmetric_keys/verify_pefile.h
@@ -9,7 +9,6 @@
* 2 of the Licence, or (at your option) any later version.
*/
-#include <linux/verify_pefile.h>
#include <crypto/pkcs7.h>
#include <crypto/hash_info.h>
@@ -23,7 +22,6 @@
unsigned sig_offset;
unsigned sig_len;
const struct section_header *secs;
- struct pkcs7_message *pkcs7;
/* PKCS#7 MS Individual Code Signing content */
const void *digest; /* Digest */
@@ -39,4 +37,5 @@
/*
* mscode_parser.c
*/
-extern int mscode_parse(struct pefile_context *ctx);
+extern int mscode_parse(void *_ctx, const void *content_data, size_t data_len,
+ size_t asn1hdrlen);
diff --git a/crypto/asymmetric_keys/x509_cert_parser.c b/crypto/asymmetric_keys/x509_cert_parser.c
index 4a29bac..865f46e 100644
--- a/crypto/asymmetric_keys/x509_cert_parser.c
+++ b/crypto/asymmetric_keys/x509_cert_parser.c
@@ -47,15 +47,12 @@
void x509_free_certificate(struct x509_certificate *cert)
{
if (cert) {
- public_key_destroy(cert->pub);
+ public_key_free(cert->pub);
+ public_key_signature_free(cert->sig);
kfree(cert->issuer);
kfree(cert->subject);
kfree(cert->id);
kfree(cert->skid);
- kfree(cert->akid_id);
- kfree(cert->akid_skid);
- kfree(cert->sig.digest);
- kfree(cert->sig.s);
kfree(cert);
}
}
@@ -78,6 +75,9 @@
cert->pub = kzalloc(sizeof(struct public_key), GFP_KERNEL);
if (!cert->pub)
goto error_no_ctx;
+ cert->sig = kzalloc(sizeof(struct public_key_signature), GFP_KERNEL);
+ if (!cert->sig)
+ goto error_no_ctx;
ctx = kzalloc(sizeof(struct x509_parse_context), GFP_KERNEL);
if (!ctx)
goto error_no_ctx;
@@ -108,6 +108,11 @@
cert->pub->keylen = ctx->key_size;
+ /* Grab the signature bits */
+ ret = x509_get_sig_params(cert);
+ if (ret < 0)
+ goto error_decode;
+
/* Generate cert issuer + serial number key ID */
kid = asymmetric_key_generate_id(cert->raw_serial,
cert->raw_serial_size,
@@ -119,6 +124,11 @@
}
cert->id = kid;
+ /* Detect self-signed certificates */
+ ret = x509_check_for_self_signed(cert);
+ if (ret < 0)
+ goto error_decode;
+
kfree(ctx);
return cert;
@@ -188,33 +198,33 @@
return -ENOPKG; /* Unsupported combination */
case OID_md4WithRSAEncryption:
- ctx->cert->sig.hash_algo = "md4";
- ctx->cert->sig.pkey_algo = "rsa";
+ ctx->cert->sig->hash_algo = "md4";
+ ctx->cert->sig->pkey_algo = "rsa";
break;
case OID_sha1WithRSAEncryption:
- ctx->cert->sig.hash_algo = "sha1";
- ctx->cert->sig.pkey_algo = "rsa";
+ ctx->cert->sig->hash_algo = "sha1";
+ ctx->cert->sig->pkey_algo = "rsa";
break;
case OID_sha256WithRSAEncryption:
- ctx->cert->sig.hash_algo = "sha256";
- ctx->cert->sig.pkey_algo = "rsa";
+ ctx->cert->sig->hash_algo = "sha256";
+ ctx->cert->sig->pkey_algo = "rsa";
break;
case OID_sha384WithRSAEncryption:
- ctx->cert->sig.hash_algo = "sha384";
- ctx->cert->sig.pkey_algo = "rsa";
+ ctx->cert->sig->hash_algo = "sha384";
+ ctx->cert->sig->pkey_algo = "rsa";
break;
case OID_sha512WithRSAEncryption:
- ctx->cert->sig.hash_algo = "sha512";
- ctx->cert->sig.pkey_algo = "rsa";
+ ctx->cert->sig->hash_algo = "sha512";
+ ctx->cert->sig->pkey_algo = "rsa";
break;
case OID_sha224WithRSAEncryption:
- ctx->cert->sig.hash_algo = "sha224";
- ctx->cert->sig.pkey_algo = "rsa";
+ ctx->cert->sig->hash_algo = "sha224";
+ ctx->cert->sig->pkey_algo = "rsa";
break;
}
@@ -572,14 +582,14 @@
pr_debug("AKID: keyid: %*phN\n", (int)vlen, value);
- if (ctx->cert->akid_skid)
+ if (ctx->cert->sig->auth_ids[1])
return 0;
kid = asymmetric_key_generate_id(value, vlen, "", 0);
if (IS_ERR(kid))
return PTR_ERR(kid);
pr_debug("authkeyid %*phN\n", kid->len, kid->data);
- ctx->cert->akid_skid = kid;
+ ctx->cert->sig->auth_ids[1] = kid;
return 0;
}
@@ -611,7 +621,7 @@
pr_debug("AKID: serial: %*phN\n", (int)vlen, value);
- if (!ctx->akid_raw_issuer || ctx->cert->akid_id)
+ if (!ctx->akid_raw_issuer || ctx->cert->sig->auth_ids[0])
return 0;
kid = asymmetric_key_generate_id(value,
@@ -622,6 +632,6 @@
return PTR_ERR(kid);
pr_debug("authkeyid %*phN\n", kid->len, kid->data);
- ctx->cert->akid_id = kid;
+ ctx->cert->sig->auth_ids[0] = kid;
return 0;
}
diff --git a/crypto/asymmetric_keys/x509_parser.h b/crypto/asymmetric_keys/x509_parser.h
index dbeed60..05eef1c 100644
--- a/crypto/asymmetric_keys/x509_parser.h
+++ b/crypto/asymmetric_keys/x509_parser.h
@@ -17,13 +17,11 @@
struct x509_certificate *next;
struct x509_certificate *signer; /* Certificate that signed this one */
struct public_key *pub; /* Public key details */
- struct public_key_signature sig; /* Signature parameters */
+ struct public_key_signature *sig; /* Signature parameters */
char *issuer; /* Name of certificate issuer */
char *subject; /* Name of certificate subject */
struct asymmetric_key_id *id; /* Issuer + Serial number */
struct asymmetric_key_id *skid; /* Subject + subjectKeyId (optional) */
- struct asymmetric_key_id *akid_id; /* CA AuthKeyId matching ->id (optional) */
- struct asymmetric_key_id *akid_skid; /* CA AuthKeyId matching ->skid (optional) */
time64_t valid_from;
time64_t valid_to;
const void *tbs; /* Signed data */
@@ -41,8 +39,9 @@
unsigned index;
bool seen; /* Infinite recursion prevention */
bool verified;
- bool trusted;
- bool unsupported_crypto; /* T if can't be verified due to missing crypto */
+ bool self_signed; /* T if self-signed (check unsupported_sig too) */
+ bool unsupported_key; /* T if key uses unsupported crypto */
+ bool unsupported_sig; /* T if signature uses unsupported crypto */
};
/*
@@ -58,5 +57,4 @@
* x509_public_key.c
*/
extern int x509_get_sig_params(struct x509_certificate *cert);
-extern int x509_check_signature(const struct public_key *pub,
- struct x509_certificate *cert);
+extern int x509_check_for_self_signed(struct x509_certificate *cert);
diff --git a/crypto/asymmetric_keys/x509_public_key.c b/crypto/asymmetric_keys/x509_public_key.c
index 733c046..fb73229 100644
--- a/crypto/asymmetric_keys/x509_public_key.c
+++ b/crypto/asymmetric_keys/x509_public_key.c
@@ -20,256 +20,133 @@
#include "asymmetric_keys.h"
#include "x509_parser.h"
-static bool use_builtin_keys;
-static struct asymmetric_key_id *ca_keyid;
-
-#ifndef MODULE
-static struct {
- struct asymmetric_key_id id;
- unsigned char data[10];
-} cakey;
-
-static int __init ca_keys_setup(char *str)
-{
- if (!str) /* default system keyring */
- return 1;
-
- if (strncmp(str, "id:", 3) == 0) {
- struct asymmetric_key_id *p = &cakey.id;
- size_t hexlen = (strlen(str) - 3) / 2;
- int ret;
-
- if (hexlen == 0 || hexlen > sizeof(cakey.data)) {
- pr_err("Missing or invalid ca_keys id\n");
- return 1;
- }
-
- ret = __asymmetric_key_hex_to_key_id(str + 3, p, hexlen);
- if (ret < 0)
- pr_err("Unparsable ca_keys id hex string\n");
- else
- ca_keyid = p; /* owner key 'id:xxxxxx' */
- } else if (strcmp(str, "builtin") == 0) {
- use_builtin_keys = true;
- }
-
- return 1;
-}
-__setup("ca_keys=", ca_keys_setup);
-#endif
-
-/**
- * x509_request_asymmetric_key - Request a key by X.509 certificate params.
- * @keyring: The keys to search.
- * @id: The issuer & serialNumber to look for or NULL.
- * @skid: The subjectKeyIdentifier to look for or NULL.
- * @partial: Use partial match if true, exact if false.
- *
- * Find a key in the given keyring by identifier. The preferred identifier is
- * the issuer + serialNumber and the fallback identifier is the
- * subjectKeyIdentifier. If both are given, the lookup is by the former, but
- * the latter must also match.
- */
-struct key *x509_request_asymmetric_key(struct key *keyring,
- const struct asymmetric_key_id *id,
- const struct asymmetric_key_id *skid,
- bool partial)
-{
- struct key *key;
- key_ref_t ref;
- const char *lookup;
- char *req, *p;
- int len;
-
- if (id) {
- lookup = id->data;
- len = id->len;
- } else {
- lookup = skid->data;
- len = skid->len;
- }
-
- /* Construct an identifier "id:<keyid>". */
- p = req = kmalloc(2 + 1 + len * 2 + 1, GFP_KERNEL);
- if (!req)
- return ERR_PTR(-ENOMEM);
-
- if (partial) {
- *p++ = 'i';
- *p++ = 'd';
- } else {
- *p++ = 'e';
- *p++ = 'x';
- }
- *p++ = ':';
- p = bin2hex(p, lookup, len);
- *p = 0;
-
- pr_debug("Look up: \"%s\"\n", req);
-
- ref = keyring_search(make_key_ref(keyring, 1),
- &key_type_asymmetric, req);
- if (IS_ERR(ref))
- pr_debug("Request for key '%s' err %ld\n", req, PTR_ERR(ref));
- kfree(req);
-
- if (IS_ERR(ref)) {
- switch (PTR_ERR(ref)) {
- /* Hide some search errors */
- case -EACCES:
- case -ENOTDIR:
- case -EAGAIN:
- return ERR_PTR(-ENOKEY);
- default:
- return ERR_CAST(ref);
- }
- }
-
- key = key_ref_to_ptr(ref);
- if (id && skid) {
- const struct asymmetric_key_ids *kids = asymmetric_key_ids(key);
- if (!kids->id[1]) {
- pr_debug("issuer+serial match, but expected SKID missing\n");
- goto reject;
- }
- if (!asymmetric_key_id_same(skid, kids->id[1])) {
- pr_debug("issuer+serial match, but SKID does not\n");
- goto reject;
- }
- }
-
- pr_devel("<==%s() = 0 [%x]\n", __func__, key_serial(key));
- return key;
-
-reject:
- key_put(key);
- return ERR_PTR(-EKEYREJECTED);
-}
-EXPORT_SYMBOL_GPL(x509_request_asymmetric_key);
-
/*
* Set up the signature parameters in an X.509 certificate. This involves
* digesting the signed data and extracting the signature.
*/
int x509_get_sig_params(struct x509_certificate *cert)
{
+ struct public_key_signature *sig = cert->sig;
struct crypto_shash *tfm;
struct shash_desc *desc;
- size_t digest_size, desc_size;
- void *digest;
+ size_t desc_size;
int ret;
pr_devel("==>%s()\n", __func__);
- if (cert->unsupported_crypto)
- return -ENOPKG;
- if (cert->sig.s)
- return 0;
+ if (!cert->pub->pkey_algo)
+ cert->unsupported_key = true;
- cert->sig.s = kmemdup(cert->raw_sig, cert->raw_sig_size,
- GFP_KERNEL);
- if (!cert->sig.s)
+ if (!sig->pkey_algo)
+ cert->unsupported_sig = true;
+
+ /* We check the hash if we can - even if we can't then verify it */
+ if (!sig->hash_algo) {
+ cert->unsupported_sig = true;
+ return 0;
+ }
+
+ sig->s = kmemdup(cert->raw_sig, cert->raw_sig_size, GFP_KERNEL);
+ if (!sig->s)
return -ENOMEM;
- cert->sig.s_size = cert->raw_sig_size;
+ sig->s_size = cert->raw_sig_size;
/* Allocate the hashing algorithm we're going to need and find out how
* big the hash operational data will be.
*/
- tfm = crypto_alloc_shash(cert->sig.hash_algo, 0, 0);
+ tfm = crypto_alloc_shash(sig->hash_algo, 0, 0);
if (IS_ERR(tfm)) {
if (PTR_ERR(tfm) == -ENOENT) {
- cert->unsupported_crypto = true;
- return -ENOPKG;
+ cert->unsupported_sig = true;
+ return 0;
}
return PTR_ERR(tfm);
}
desc_size = crypto_shash_descsize(tfm) + sizeof(*desc);
- digest_size = crypto_shash_digestsize(tfm);
+ sig->digest_size = crypto_shash_digestsize(tfm);
- /* We allocate the hash operational data storage on the end of the
- * digest storage space.
- */
ret = -ENOMEM;
- digest = kzalloc(ALIGN(digest_size, __alignof__(*desc)) + desc_size,
- GFP_KERNEL);
- if (!digest)
+ sig->digest = kmalloc(sig->digest_size, GFP_KERNEL);
+ if (!sig->digest)
goto error;
- cert->sig.digest = digest;
- cert->sig.digest_size = digest_size;
+ desc = kzalloc(desc_size, GFP_KERNEL);
+ if (!desc)
+ goto error;
- desc = PTR_ALIGN(digest + digest_size, __alignof__(*desc));
desc->tfm = tfm;
desc->flags = CRYPTO_TFM_REQ_MAY_SLEEP;
ret = crypto_shash_init(desc);
if (ret < 0)
- goto error;
+ goto error_2;
might_sleep();
- ret = crypto_shash_finup(desc, cert->tbs, cert->tbs_size, digest);
+ ret = crypto_shash_finup(desc, cert->tbs, cert->tbs_size, sig->digest);
+
+error_2:
+ kfree(desc);
error:
crypto_free_shash(tfm);
pr_devel("<==%s() = %d\n", __func__, ret);
return ret;
}
-EXPORT_SYMBOL_GPL(x509_get_sig_params);
/*
- * Check the signature on a certificate using the provided public key
+ * Check for self-signedness in an X.509 cert and if found, check the signature
+ * immediately if we can.
*/
-int x509_check_signature(const struct public_key *pub,
- struct x509_certificate *cert)
+int x509_check_for_self_signed(struct x509_certificate *cert)
{
- int ret;
+ int ret = 0;
pr_devel("==>%s()\n", __func__);
- ret = x509_get_sig_params(cert);
- if (ret < 0)
- return ret;
+ if (cert->raw_subject_size != cert->raw_issuer_size ||
+ memcmp(cert->raw_subject, cert->raw_issuer,
+ cert->raw_issuer_size) != 0)
+ goto not_self_signed;
- ret = public_key_verify_signature(pub, &cert->sig);
- if (ret == -ENOPKG)
- cert->unsupported_crypto = true;
- pr_debug("Cert Verification: %d\n", ret);
- return ret;
-}
-EXPORT_SYMBOL_GPL(x509_check_signature);
+ if (cert->sig->auth_ids[0] || cert->sig->auth_ids[1]) {
+ /* If the AKID is present it may have one or two parts. If
+ * both are supplied, both must match.
+ */
+ bool a = asymmetric_key_id_same(cert->skid, cert->sig->auth_ids[1]);
+ bool b = asymmetric_key_id_same(cert->id, cert->sig->auth_ids[0]);
-/*
- * Check the new certificate against the ones in the trust keyring. If one of
- * those is the signing key and validates the new certificate, then mark the
- * new certificate as being trusted.
- *
- * Return 0 if the new certificate was successfully validated, 1 if we couldn't
- * find a matching parent certificate in the trusted list and an error if there
- * is a matching certificate but the signature check fails.
- */
-static int x509_validate_trust(struct x509_certificate *cert,
- struct key *trust_keyring)
-{
- struct key *key;
- int ret = 1;
+ if (!a && !b)
+ goto not_self_signed;
- if (!trust_keyring)
- return -EOPNOTSUPP;
-
- if (ca_keyid && !asymmetric_key_id_partial(cert->akid_skid, ca_keyid))
- return -EPERM;
-
- key = x509_request_asymmetric_key(trust_keyring,
- cert->akid_id, cert->akid_skid,
- false);
- if (!IS_ERR(key)) {
- if (!use_builtin_keys
- || test_bit(KEY_FLAG_BUILTIN, &key->flags))
- ret = x509_check_signature(key->payload.data[asym_crypto],
- cert);
- key_put(key);
+ ret = -EKEYREJECTED;
+ if (((a && !b) || (b && !a)) &&
+ cert->sig->auth_ids[0] && cert->sig->auth_ids[1])
+ goto out;
}
+
+ ret = -EKEYREJECTED;
+ if (cert->pub->pkey_algo != cert->sig->pkey_algo)
+ goto out;
+
+ ret = public_key_verify_signature(cert->pub, cert->sig);
+ if (ret < 0) {
+ if (ret == -ENOPKG) {
+ cert->unsupported_sig = true;
+ ret = 0;
+ }
+ goto out;
+ }
+
+ pr_devel("Cert Self-signature verified");
+ cert->self_signed = true;
+
+out:
+ pr_devel("<==%s() = %d\n", __func__, ret);
return ret;
+
+not_self_signed:
+ pr_devel("<==%s() = 0 [not]\n", __func__);
+ return 0;
}
/*
@@ -291,34 +168,22 @@
pr_devel("Cert Issuer: %s\n", cert->issuer);
pr_devel("Cert Subject: %s\n", cert->subject);
- if (!cert->pub->pkey_algo ||
- !cert->sig.pkey_algo ||
- !cert->sig.hash_algo) {
+ if (cert->unsupported_key) {
ret = -ENOPKG;
goto error_free_cert;
}
pr_devel("Cert Key Algo: %s\n", cert->pub->pkey_algo);
pr_devel("Cert Valid period: %lld-%lld\n", cert->valid_from, cert->valid_to);
- pr_devel("Cert Signature: %s + %s\n",
- cert->sig.pkey_algo,
- cert->sig.hash_algo);
cert->pub->id_type = "X509";
- /* Check the signature on the key if it appears to be self-signed */
- if ((!cert->akid_skid && !cert->akid_id) ||
- asymmetric_key_id_same(cert->skid, cert->akid_skid) ||
- asymmetric_key_id_same(cert->id, cert->akid_id)) {
- ret = x509_check_signature(cert->pub, cert); /* self-signed */
- if (ret < 0)
- goto error_free_cert;
- } else if (!prep->trusted) {
- ret = x509_validate_trust(cert, get_system_trusted_keyring());
- if (ret)
- ret = x509_validate_trust(cert, get_ima_mok_keyring());
- if (!ret)
- prep->trusted = 1;
+ if (cert->unsupported_sig) {
+ public_key_signature_free(cert->sig);
+ cert->sig = NULL;
+ } else {
+ pr_devel("Cert Signature: %s + %s\n",
+ cert->sig->pkey_algo, cert->sig->hash_algo);
}
/* Propose a description */
@@ -353,6 +218,7 @@
prep->payload.data[asym_subtype] = &public_key_subtype;
prep->payload.data[asym_key_ids] = kids;
prep->payload.data[asym_crypto] = cert->pub;
+ prep->payload.data[asym_auth] = cert->sig;
prep->description = desc;
prep->quotalen = 100;
@@ -360,6 +226,7 @@
cert->pub = NULL;
cert->id = NULL;
cert->skid = NULL;
+ cert->sig = NULL;
desc = NULL;
ret = 0;
diff --git a/drivers/Kconfig b/drivers/Kconfig
index d2ac339..e1e2066 100644
--- a/drivers/Kconfig
+++ b/drivers/Kconfig
@@ -114,6 +114,8 @@
source "drivers/dma/Kconfig"
+source "drivers/dma-buf/Kconfig"
+
source "drivers/dca/Kconfig"
source "drivers/auxdisplay/Kconfig"
@@ -190,6 +192,8 @@
source "drivers/nvdimm/Kconfig"
+source "drivers/dax/Kconfig"
+
source "drivers/nvmem/Kconfig"
source "drivers/hwtracing/stm/Kconfig"
diff --git a/drivers/Makefile b/drivers/Makefile
index 8f5d076..0b6f3d6 100644
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -66,6 +66,7 @@
obj-$(CONFIG_NVM) += lightnvm/
obj-y += base/ block/ misc/ mfd/ nfc/
obj-$(CONFIG_LIBNVDIMM) += nvdimm/
+obj-$(CONFIG_DEV_DAX) += dax/
obj-$(CONFIG_DMA_SHARED_BUFFER) += dma-buf/
obj-$(CONFIG_NUBUS) += nubus/
obj-y += macintosh/
diff --git a/drivers/acpi/nfit.c b/drivers/acpi/nfit.c
index 63cc9db..2215fc8 100644
--- a/drivers/acpi/nfit.c
+++ b/drivers/acpi/nfit.c
@@ -45,6 +45,11 @@
MODULE_PARM_DESC(scrub_overflow_abort,
"Number of times we overflow ARS results before abort");
+static bool disable_vendor_specific;
+module_param(disable_vendor_specific, bool, S_IRUGO);
+MODULE_PARM_DESC(disable_vendor_specific,
+ "Limit commands to the publicly specified set\n");
+
static struct workqueue_struct *nfit_wq;
struct nfit_table_prev {
@@ -171,33 +176,46 @@
unsigned int buf_len, int *cmd_rc)
{
struct acpi_nfit_desc *acpi_desc = to_acpi_nfit_desc(nd_desc);
- const struct nd_cmd_desc *desc = NULL;
union acpi_object in_obj, in_buf, *out_obj;
+ const struct nd_cmd_desc *desc = NULL;
struct device *dev = acpi_desc->dev;
+ struct nd_cmd_pkg *call_pkg = NULL;
const char *cmd_name, *dimm_name;
- unsigned long dsm_mask;
+ unsigned long cmd_mask, dsm_mask;
acpi_handle handle;
+ unsigned int func;
const u8 *uuid;
u32 offset;
int rc, i;
+ func = cmd;
+ if (cmd == ND_CMD_CALL) {
+ call_pkg = buf;
+ func = call_pkg->nd_command;
+ }
+
if (nvdimm) {
struct nfit_mem *nfit_mem = nvdimm_provider_data(nvdimm);
struct acpi_device *adev = nfit_mem->adev;
if (!adev)
return -ENOTTY;
+ if (call_pkg && nfit_mem->family != call_pkg->nd_family)
+ return -ENOTTY;
+
dimm_name = nvdimm_name(nvdimm);
cmd_name = nvdimm_cmd_name(cmd);
+ cmd_mask = nvdimm_cmd_mask(nvdimm);
dsm_mask = nfit_mem->dsm_mask;
desc = nd_cmd_dimm_desc(cmd);
- uuid = to_nfit_uuid(NFIT_DEV_DIMM);
+ uuid = to_nfit_uuid(nfit_mem->family);
handle = adev->handle;
} else {
struct acpi_device *adev = to_acpi_dev(acpi_desc);
cmd_name = nvdimm_bus_cmd_name(cmd);
- dsm_mask = nd_desc->dsm_mask;
+ cmd_mask = nd_desc->cmd_mask;
+ dsm_mask = cmd_mask;
desc = nd_cmd_bus_desc(cmd);
uuid = to_nfit_uuid(NFIT_DEV_BUS);
handle = adev->handle;
@@ -207,7 +225,7 @@
if (!desc || (cmd && (desc->out_num + desc->in_num == 0)))
return -ENOTTY;
- if (!test_bit(cmd, &dsm_mask))
+ if (!test_bit(cmd, &cmd_mask) || !test_bit(func, &dsm_mask))
return -ENOTTY;
in_obj.type = ACPI_TYPE_PACKAGE;
@@ -222,21 +240,44 @@
in_buf.buffer.length += nd_cmd_in_size(nvdimm, cmd, desc,
i, buf);
- if (IS_ENABLED(CONFIG_ACPI_NFIT_DEBUG)) {
- dev_dbg(dev, "%s:%s cmd: %s input length: %d\n", __func__,
- dimm_name, cmd_name, in_buf.buffer.length);
- print_hex_dump_debug(cmd_name, DUMP_PREFIX_OFFSET, 4,
- 4, in_buf.buffer.pointer, min_t(u32, 128,
- in_buf.buffer.length), true);
+ if (call_pkg) {
+ /* skip over package wrapper */
+ in_buf.buffer.pointer = (void *) &call_pkg->nd_payload;
+ in_buf.buffer.length = call_pkg->nd_size_in;
}
- out_obj = acpi_evaluate_dsm(handle, uuid, 1, cmd, &in_obj);
+ if (IS_ENABLED(CONFIG_ACPI_NFIT_DEBUG)) {
+ dev_dbg(dev, "%s:%s cmd: %d: func: %d input length: %d\n",
+ __func__, dimm_name, cmd, func,
+ in_buf.buffer.length);
+ print_hex_dump_debug("nvdimm in ", DUMP_PREFIX_OFFSET, 4, 4,
+ in_buf.buffer.pointer,
+ min_t(u32, 256, in_buf.buffer.length), true);
+ }
+
+ out_obj = acpi_evaluate_dsm(handle, uuid, 1, func, &in_obj);
if (!out_obj) {
dev_dbg(dev, "%s:%s _DSM failed cmd: %s\n", __func__, dimm_name,
cmd_name);
return -EINVAL;
}
+ if (call_pkg) {
+ call_pkg->nd_fw_size = out_obj->buffer.length;
+ memcpy(call_pkg->nd_payload + call_pkg->nd_size_in,
+ out_obj->buffer.pointer,
+ min(call_pkg->nd_fw_size, call_pkg->nd_size_out));
+
+ ACPI_FREE(out_obj);
+ /*
+ * Need to support FW function w/o known size in advance.
+ * Caller can determine required size based upon nd_fw_size.
+ * If we return an error (like elsewhere) then caller wouldn't
+ * be able to rely upon data returned to make calculation.
+ */
+ return 0;
+ }
+
if (out_obj->package.type != ACPI_TYPE_BUFFER) {
dev_dbg(dev, "%s:%s unexpected output object type cmd: %s type: %d\n",
__func__, dimm_name, cmd_name, out_obj->type);
@@ -658,6 +699,7 @@
if (!nfit_mem)
return -ENOMEM;
INIT_LIST_HEAD(&nfit_mem->list);
+ nfit_mem->acpi_desc = acpi_desc;
list_add(&nfit_mem->list, &acpi_desc->dimms);
}
@@ -819,7 +861,7 @@
{
struct acpi_nfit_control_region *dcr = to_nfit_dcr(dev);
- return sprintf(buf, "%#x\n", dcr->vendor_id);
+ return sprintf(buf, "0x%04x\n", be16_to_cpu(dcr->vendor_id));
}
static DEVICE_ATTR_RO(vendor);
@@ -828,7 +870,7 @@
{
struct acpi_nfit_control_region *dcr = to_nfit_dcr(dev);
- return sprintf(buf, "%#x\n", dcr->revision_id);
+ return sprintf(buf, "0x%04x\n", be16_to_cpu(dcr->revision_id));
}
static DEVICE_ATTR_RO(rev_id);
@@ -837,28 +879,142 @@
{
struct acpi_nfit_control_region *dcr = to_nfit_dcr(dev);
- return sprintf(buf, "%#x\n", dcr->device_id);
+ return sprintf(buf, "0x%04x\n", be16_to_cpu(dcr->device_id));
}
static DEVICE_ATTR_RO(device);
+static ssize_t subsystem_vendor_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct acpi_nfit_control_region *dcr = to_nfit_dcr(dev);
+
+ return sprintf(buf, "0x%04x\n", be16_to_cpu(dcr->subsystem_vendor_id));
+}
+static DEVICE_ATTR_RO(subsystem_vendor);
+
+static ssize_t subsystem_rev_id_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct acpi_nfit_control_region *dcr = to_nfit_dcr(dev);
+
+ return sprintf(buf, "0x%04x\n",
+ be16_to_cpu(dcr->subsystem_revision_id));
+}
+static DEVICE_ATTR_RO(subsystem_rev_id);
+
+static ssize_t subsystem_device_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct acpi_nfit_control_region *dcr = to_nfit_dcr(dev);
+
+ return sprintf(buf, "0x%04x\n", be16_to_cpu(dcr->subsystem_device_id));
+}
+static DEVICE_ATTR_RO(subsystem_device);
+
+static int num_nvdimm_formats(struct nvdimm *nvdimm)
+{
+ struct nfit_mem *nfit_mem = nvdimm_provider_data(nvdimm);
+ int formats = 0;
+
+ if (nfit_mem->memdev_pmem)
+ formats++;
+ if (nfit_mem->memdev_bdw)
+ formats++;
+ return formats;
+}
+
static ssize_t format_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct acpi_nfit_control_region *dcr = to_nfit_dcr(dev);
- return sprintf(buf, "%#x\n", dcr->code);
+ return sprintf(buf, "0x%04x\n", be16_to_cpu(dcr->code));
}
static DEVICE_ATTR_RO(format);
+static ssize_t format1_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ u32 handle;
+ ssize_t rc = -ENXIO;
+ struct nfit_mem *nfit_mem;
+ struct nfit_memdev *nfit_memdev;
+ struct acpi_nfit_desc *acpi_desc;
+ struct nvdimm *nvdimm = to_nvdimm(dev);
+ struct acpi_nfit_control_region *dcr = to_nfit_dcr(dev);
+
+ nfit_mem = nvdimm_provider_data(nvdimm);
+ acpi_desc = nfit_mem->acpi_desc;
+ handle = to_nfit_memdev(dev)->device_handle;
+
+ /* assumes DIMMs have at most 2 published interface codes */
+ mutex_lock(&acpi_desc->init_mutex);
+ list_for_each_entry(nfit_memdev, &acpi_desc->memdevs, list) {
+ struct acpi_nfit_memory_map *memdev = nfit_memdev->memdev;
+ struct nfit_dcr *nfit_dcr;
+
+ if (memdev->device_handle != handle)
+ continue;
+
+ list_for_each_entry(nfit_dcr, &acpi_desc->dcrs, list) {
+ if (nfit_dcr->dcr->region_index != memdev->region_index)
+ continue;
+ if (nfit_dcr->dcr->code == dcr->code)
+ continue;
+ rc = sprintf(buf, "%#x\n",
+ be16_to_cpu(nfit_dcr->dcr->code));
+ break;
+ }
+ if (rc != ENXIO)
+ break;
+ }
+ mutex_unlock(&acpi_desc->init_mutex);
+ return rc;
+}
+static DEVICE_ATTR_RO(format1);
+
+static ssize_t formats_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct nvdimm *nvdimm = to_nvdimm(dev);
+
+ return sprintf(buf, "%d\n", num_nvdimm_formats(nvdimm));
+}
+static DEVICE_ATTR_RO(formats);
+
static ssize_t serial_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct acpi_nfit_control_region *dcr = to_nfit_dcr(dev);
- return sprintf(buf, "%#x\n", dcr->serial_number);
+ return sprintf(buf, "0x%08x\n", be32_to_cpu(dcr->serial_number));
}
static DEVICE_ATTR_RO(serial);
+static ssize_t family_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct nvdimm *nvdimm = to_nvdimm(dev);
+ struct nfit_mem *nfit_mem = nvdimm_provider_data(nvdimm);
+
+ if (nfit_mem->family < 0)
+ return -ENXIO;
+ return sprintf(buf, "%d\n", nfit_mem->family);
+}
+static DEVICE_ATTR_RO(family);
+
+static ssize_t dsm_mask_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct nvdimm *nvdimm = to_nvdimm(dev);
+ struct nfit_mem *nfit_mem = nvdimm_provider_data(nvdimm);
+
+ if (nfit_mem->family < 0)
+ return -ENXIO;
+ return sprintf(buf, "%#lx\n", nfit_mem->dsm_mask);
+}
+static DEVICE_ATTR_RO(dsm_mask);
+
static ssize_t flags_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
@@ -873,15 +1029,41 @@
}
static DEVICE_ATTR_RO(flags);
+static ssize_t id_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct acpi_nfit_control_region *dcr = to_nfit_dcr(dev);
+
+ if (dcr->valid_fields & ACPI_NFIT_CONTROL_MFG_INFO_VALID)
+ return sprintf(buf, "%04x-%02x-%04x-%08x\n",
+ be16_to_cpu(dcr->vendor_id),
+ dcr->manufacturing_location,
+ be16_to_cpu(dcr->manufacturing_date),
+ be32_to_cpu(dcr->serial_number));
+ else
+ return sprintf(buf, "%04x-%08x\n",
+ be16_to_cpu(dcr->vendor_id),
+ be32_to_cpu(dcr->serial_number));
+}
+static DEVICE_ATTR_RO(id);
+
static struct attribute *acpi_nfit_dimm_attributes[] = {
&dev_attr_handle.attr,
&dev_attr_phys_id.attr,
&dev_attr_vendor.attr,
&dev_attr_device.attr,
- &dev_attr_format.attr,
- &dev_attr_serial.attr,
&dev_attr_rev_id.attr,
+ &dev_attr_subsystem_vendor.attr,
+ &dev_attr_subsystem_device.attr,
+ &dev_attr_subsystem_rev_id.attr,
+ &dev_attr_format.attr,
+ &dev_attr_formats.attr,
+ &dev_attr_format1.attr,
+ &dev_attr_serial.attr,
&dev_attr_flags.attr,
+ &dev_attr_id.attr,
+ &dev_attr_family.attr,
+ &dev_attr_dsm_mask.attr,
NULL,
};
@@ -889,11 +1071,13 @@
struct attribute *a, int n)
{
struct device *dev = container_of(kobj, struct device, kobj);
+ struct nvdimm *nvdimm = to_nvdimm(dev);
- if (to_nfit_dcr(dev))
- return a->mode;
- else
+ if (!to_nfit_dcr(dev))
return 0;
+ if (a == &dev_attr_format1.attr && num_nvdimm_formats(nvdimm) <= 1)
+ return 0;
+ return a->mode;
}
static struct attribute_group acpi_nfit_dimm_attribute_group = {
@@ -926,10 +1110,13 @@
{
struct acpi_device *adev, *adev_dimm;
struct device *dev = acpi_desc->dev;
- const u8 *uuid = to_nfit_uuid(NFIT_DEV_DIMM);
+ unsigned long dsm_mask;
+ const u8 *uuid;
int i;
- nfit_mem->dsm_mask = acpi_desc->dimm_dsm_force_en;
+ /* nfit test assumes 1:1 relationship between commands and dsms */
+ nfit_mem->dsm_mask = acpi_desc->dimm_cmd_force_en;
+ nfit_mem->family = NVDIMM_FAMILY_INTEL;
adev = to_acpi_dev(acpi_desc);
if (!adev)
return 0;
@@ -942,7 +1129,35 @@
return force_enable_dimms ? 0 : -ENODEV;
}
- for (i = ND_CMD_SMART; i <= ND_CMD_VENDOR; i++)
+ /*
+ * Until standardization materializes we need to consider up to 3
+ * different command sets. Note, that checking for function0 (bit0)
+ * tells us if any commands are reachable through this uuid.
+ */
+ for (i = NVDIMM_FAMILY_INTEL; i <= NVDIMM_FAMILY_HPE2; i++)
+ if (acpi_check_dsm(adev_dimm->handle, to_nfit_uuid(i), 1, 1))
+ break;
+
+ /* limit the supported commands to those that are publicly documented */
+ nfit_mem->family = i;
+ if (nfit_mem->family == NVDIMM_FAMILY_INTEL) {
+ dsm_mask = 0x3fe;
+ if (disable_vendor_specific)
+ dsm_mask &= ~(1 << ND_CMD_VENDOR);
+ } else if (nfit_mem->family == NVDIMM_FAMILY_HPE1)
+ dsm_mask = 0x1c3c76;
+ else if (nfit_mem->family == NVDIMM_FAMILY_HPE2) {
+ dsm_mask = 0x1fe;
+ if (disable_vendor_specific)
+ dsm_mask &= ~(1 << 8);
+ } else {
+ dev_err(dev, "unknown dimm command family\n");
+ nfit_mem->family = -1;
+ return force_enable_dimms ? 0 : -ENODEV;
+ }
+
+ uuid = to_nfit_uuid(nfit_mem->family);
+ for_each_set_bit(i, &dsm_mask, BITS_PER_LONG)
if (acpi_check_dsm(adev_dimm->handle, uuid, 1, 1ULL << i))
set_bit(i, &nfit_mem->dsm_mask);
@@ -955,8 +1170,8 @@
int dimm_count = 0;
list_for_each_entry(nfit_mem, &acpi_desc->dimms, list) {
+ unsigned long flags = 0, cmd_mask;
struct nvdimm *nvdimm;
- unsigned long flags = 0;
u32 device_handle;
u16 mem_flags;
int rc;
@@ -979,9 +1194,18 @@
if (rc)
continue;
+ /*
+ * TODO: provide translation for non-NVDIMM_FAMILY_INTEL
+ * devices (i.e. from nd_cmd to acpi_dsm) to standardize the
+ * userspace interface.
+ */
+ cmd_mask = 1UL << ND_CMD_CALL;
+ if (nfit_mem->family == NVDIMM_FAMILY_INTEL)
+ cmd_mask |= nfit_mem->dsm_mask;
+
nvdimm = nvdimm_create(acpi_desc->nvdimm_bus, nfit_mem,
acpi_nfit_dimm_attribute_groups,
- flags, &nfit_mem->dsm_mask);
+ flags, cmd_mask);
if (!nvdimm)
return -ENOMEM;
@@ -1010,14 +1234,14 @@
struct acpi_device *adev;
int i;
- nd_desc->dsm_mask = acpi_desc->bus_dsm_force_en;
+ nd_desc->cmd_mask = acpi_desc->bus_cmd_force_en;
adev = to_acpi_dev(acpi_desc);
if (!adev)
return;
for (i = ND_CMD_ARS_CAP; i <= ND_CMD_CLEAR_ERROR; i++)
if (acpi_check_dsm(adev->handle, uuid, 1, 1ULL << i))
- set_bit(i, &nd_desc->dsm_mask);
+ set_bit(i, &nd_desc->cmd_mask);
}
static ssize_t range_index_show(struct device *dev,
@@ -2309,7 +2533,7 @@
acpi_size sz;
int rc;
- status = acpi_get_table_with_size("NFIT", 0, &tbl, &sz);
+ status = acpi_get_table_with_size(ACPI_SIG_NFIT, 0, &tbl, &sz);
if (ACPI_FAILURE(status)) {
/* This is ok, we could have an nvdimm hotplugged later */
dev_dbg(dev, "failed to find NFIT at startup\n");
@@ -2466,6 +2690,8 @@
acpi_str_to_uuid(UUID_PERSISTENT_VIRTUAL_CD, nfit_uuid[NFIT_SPA_PCD]);
acpi_str_to_uuid(UUID_NFIT_BUS, nfit_uuid[NFIT_DEV_BUS]);
acpi_str_to_uuid(UUID_NFIT_DIMM, nfit_uuid[NFIT_DEV_DIMM]);
+ acpi_str_to_uuid(UUID_NFIT_DIMM_N_HPE1, nfit_uuid[NFIT_DEV_DIMM_N_HPE1]);
+ acpi_str_to_uuid(UUID_NFIT_DIMM_N_HPE2, nfit_uuid[NFIT_DEV_DIMM_N_HPE2]);
nfit_wq = create_singlethread_workqueue("nfit");
if (!nfit_wq)
diff --git a/drivers/acpi/nfit.h b/drivers/acpi/nfit.h
index c75576b..11cb383 100644
--- a/drivers/acpi/nfit.h
+++ b/drivers/acpi/nfit.h
@@ -21,13 +21,25 @@
#include <linux/acpi.h>
#include <acpi/acuuid.h>
+/* ACPI 6.1 */
#define UUID_NFIT_BUS "2f10e7a4-9e91-11e4-89d3-123b93f75cba"
+
+/* http://pmem.io/documents/NVDIMM_DSM_Interface_Example.pdf */
#define UUID_NFIT_DIMM "4309ac30-0d11-11e4-9191-0800200c9a66"
+
+/* https://github.com/HewlettPackard/hpe-nvm/blob/master/Documentation/ */
+#define UUID_NFIT_DIMM_N_HPE1 "9002c334-acf3-4c0e-9642-a235f0d53bc6"
+#define UUID_NFIT_DIMM_N_HPE2 "5008664b-b758-41a0-a03c-27c2f2d04f7e"
+
#define ACPI_NFIT_MEM_FAILED_MASK (ACPI_NFIT_MEM_SAVE_FAILED \
| ACPI_NFIT_MEM_RESTORE_FAILED | ACPI_NFIT_MEM_FLUSH_FAILED \
| ACPI_NFIT_MEM_NOT_ARMED)
enum nfit_uuids {
+ /* for simplicity alias the uuid index with the family id */
+ NFIT_DEV_DIMM = NVDIMM_FAMILY_INTEL,
+ NFIT_DEV_DIMM_N_HPE1 = NVDIMM_FAMILY_HPE1,
+ NFIT_DEV_DIMM_N_HPE2 = NVDIMM_FAMILY_HPE2,
NFIT_SPA_VOLATILE,
NFIT_SPA_PM,
NFIT_SPA_DCR,
@@ -37,15 +49,16 @@
NFIT_SPA_PDISK,
NFIT_SPA_PCD,
NFIT_DEV_BUS,
- NFIT_DEV_DIMM,
NFIT_UUID_MAX,
};
-enum nfit_fic {
- NFIT_FIC_BYTE = 0x101, /* byte-addressable energy backed */
- NFIT_FIC_BLK = 0x201, /* block-addressable non-energy backed */
- NFIT_FIC_BYTEN = 0x301, /* byte-addressable non-energy backed */
-};
+/*
+ * Region format interface codes are stored as an array of bytes in the
+ * NFIT DIMM Control Region structure
+ */
+#define NFIT_FIC_BYTE cpu_to_be16(0x101) /* byte-addressable energy backed */
+#define NFIT_FIC_BLK cpu_to_be16(0x201) /* block-addressable non-energy backed */
+#define NFIT_FIC_BYTEN cpu_to_be16(0x301) /* byte-addressable non-energy backed */
enum {
NFIT_BLK_READ_FLUSH = 1,
@@ -109,7 +122,9 @@
struct nfit_flush *nfit_flush;
struct list_head list;
struct acpi_device *adev;
+ struct acpi_nfit_desc *acpi_desc;
unsigned long dsm_mask;
+ int family;
};
struct acpi_nfit_desc {
@@ -132,8 +147,8 @@
size_t ars_status_size;
struct work_struct work;
unsigned int cancel:1;
- unsigned long dimm_dsm_force_en;
- unsigned long bus_dsm_force_en;
+ unsigned long dimm_cmd_force_en;
+ unsigned long bus_cmd_force_en;
int (*blk_do_io)(struct nd_blk_region *ndbr, resource_size_t dpa,
void *iobuf, u64 len, int rw);
};
diff --git a/drivers/acpi/utils.c b/drivers/acpi/utils.c
index ac832bf..22c0995 100644
--- a/drivers/acpi/utils.c
+++ b/drivers/acpi/utils.c
@@ -625,7 +625,7 @@
* some old BIOSes do expect a buffer or an integer etc.
*/
union acpi_object *
-acpi_evaluate_dsm(acpi_handle handle, const u8 *uuid, int rev, int func,
+acpi_evaluate_dsm(acpi_handle handle, const u8 *uuid, u64 rev, u64 func,
union acpi_object *argv4)
{
acpi_status ret;
@@ -674,7 +674,7 @@
* functions. Currently only support 64 functions at maximum, should be
* enough for now.
*/
-bool acpi_check_dsm(acpi_handle handle, const u8 *uuid, int rev, u64 funcs)
+bool acpi_check_dsm(acpi_handle handle, const u8 *uuid, u64 rev, u64 funcs)
{
int i;
u64 mask = 0;
diff --git a/drivers/amba/bus.c b/drivers/amba/bus.c
index f009936..a5b5c87 100644
--- a/drivers/amba/bus.c
+++ b/drivers/amba/bus.c
@@ -336,16 +336,7 @@
kfree(d);
}
-/**
- * amba_device_add - add a previously allocated AMBA device structure
- * @dev: AMBA device allocated by amba_device_alloc
- * @parent: resource parent for this devices resources
- *
- * Claim the resource, and read the device cell ID if not already
- * initialized. Register the AMBA device with the Linux device
- * manager.
- */
-int amba_device_add(struct amba_device *dev, struct resource *parent)
+static int amba_device_try_add(struct amba_device *dev, struct resource *parent)
{
u32 size;
void __iomem *tmp;
@@ -373,6 +364,12 @@
goto err_release;
}
+ ret = dev_pm_domain_attach(&dev->dev, true);
+ if (ret == -EPROBE_DEFER) {
+ iounmap(tmp);
+ goto err_release;
+ }
+
ret = amba_get_enable_pclk(dev);
if (ret == 0) {
u32 pid, cid;
@@ -398,6 +395,7 @@
}
iounmap(tmp);
+ dev_pm_domain_detach(&dev->dev, true);
if (ret)
goto err_release;
@@ -421,6 +419,88 @@
err_out:
return ret;
}
+
+/*
+ * Registration of AMBA device require reading its pid and cid registers.
+ * To do this, the device must be turned on (if it is a part of power domain)
+ * and have clocks enabled. However in some cases those resources might not be
+ * yet available. Returning EPROBE_DEFER is not a solution in such case,
+ * because callers don't handle this special error code. Instead such devices
+ * are added to the special list and their registration is retried from
+ * periodic worker, until all resources are available and registration succeeds.
+ */
+struct deferred_device {
+ struct amba_device *dev;
+ struct resource *parent;
+ struct list_head node;
+};
+
+static LIST_HEAD(deferred_devices);
+static DEFINE_MUTEX(deferred_devices_lock);
+
+static void amba_deferred_retry_func(struct work_struct *dummy);
+static DECLARE_DELAYED_WORK(deferred_retry_work, amba_deferred_retry_func);
+
+#define DEFERRED_DEVICE_TIMEOUT (msecs_to_jiffies(5 * 1000))
+
+static void amba_deferred_retry_func(struct work_struct *dummy)
+{
+ struct deferred_device *ddev, *tmp;
+
+ mutex_lock(&deferred_devices_lock);
+
+ list_for_each_entry_safe(ddev, tmp, &deferred_devices, node) {
+ int ret = amba_device_try_add(ddev->dev, ddev->parent);
+
+ if (ret == -EPROBE_DEFER)
+ continue;
+
+ list_del_init(&ddev->node);
+ kfree(ddev);
+ }
+
+ if (!list_empty(&deferred_devices))
+ schedule_delayed_work(&deferred_retry_work,
+ DEFERRED_DEVICE_TIMEOUT);
+
+ mutex_unlock(&deferred_devices_lock);
+}
+
+/**
+ * amba_device_add - add a previously allocated AMBA device structure
+ * @dev: AMBA device allocated by amba_device_alloc
+ * @parent: resource parent for this devices resources
+ *
+ * Claim the resource, and read the device cell ID if not already
+ * initialized. Register the AMBA device with the Linux device
+ * manager.
+ */
+int amba_device_add(struct amba_device *dev, struct resource *parent)
+{
+ int ret = amba_device_try_add(dev, parent);
+
+ if (ret == -EPROBE_DEFER) {
+ struct deferred_device *ddev;
+
+ ddev = kmalloc(sizeof(*ddev), GFP_KERNEL);
+ if (!ddev)
+ return -ENOMEM;
+
+ ddev->dev = dev;
+ ddev->parent = parent;
+ ret = 0;
+
+ mutex_lock(&deferred_devices_lock);
+
+ if (list_empty(&deferred_devices))
+ schedule_delayed_work(&deferred_retry_work,
+ DEFERRED_DEVICE_TIMEOUT);
+ list_add_tail(&ddev->node, &deferred_devices);
+
+ mutex_unlock(&deferred_devices_lock);
+ }
+ return ret;
+}
EXPORT_SYMBOL_GPL(amba_device_add);
static struct amba_device *
diff --git a/drivers/ata/Kconfig b/drivers/ata/Kconfig
index cfa936a..e2dc4c0 100644
--- a/drivers/ata/Kconfig
+++ b/drivers/ata/Kconfig
@@ -313,14 +313,23 @@
config SATA_DWC
tristate "DesignWare Cores SATA support"
- depends on 460EX
- select DW_DMAC
+ depends on DMADEVICES
+ select GENERIC_PHY
help
This option enables support for the on-chip SATA controller of the
AppliedMicro processor 460EX.
If unsure, say N.
+config SATA_DWC_OLD_DMA
+ bool "Support old device trees"
+ depends on SATA_DWC
+ select DW_DMAC_CORE
+ default y if 460EX
+ help
+ This option enables support for old device trees without the
+ "dmas" property.
+
config SATA_DWC_DEBUG
bool "Debugging driver version"
depends on SATA_DWC
diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
index 6f33ace..6be7770 100644
--- a/drivers/ata/libata-core.c
+++ b/drivers/ata/libata-core.c
@@ -66,6 +66,7 @@
#include <scsi/scsi_host.h>
#include <linux/libata.h>
#include <asm/byteorder.h>
+#include <asm/unaligned.h>
#include <linux/cdrom.h>
#include <linux/ratelimit.h>
#include <linux/pm_runtime.h>
@@ -695,7 +696,7 @@
* RETURNS:
* Block address read from @tf.
*/
-u64 ata_tf_read_block(struct ata_taskfile *tf, struct ata_device *dev)
+u64 ata_tf_read_block(const struct ata_taskfile *tf, struct ata_device *dev)
{
u64 block = 0;
@@ -720,7 +721,7 @@
if (!sect) {
ata_dev_warn(dev,
"device reported invalid CHS sector 0\n");
- sect = 1; /* oh well */
+ return U64_MAX;
}
block = (cyl * dev->heads + head) * dev->sectors + sect - 1;
@@ -2079,6 +2080,81 @@
return ((ap->cbl == ATA_CBL_SATA) && (!ata_id_is_sata(dev->id)));
}
+static void ata_dev_config_ncq_send_recv(struct ata_device *dev)
+{
+ struct ata_port *ap = dev->link->ap;
+ unsigned int err_mask;
+ int log_index = ATA_LOG_NCQ_SEND_RECV * 2;
+ u16 log_pages;
+
+ err_mask = ata_read_log_page(dev, ATA_LOG_DIRECTORY,
+ 0, ap->sector_buf, 1);
+ if (err_mask) {
+ ata_dev_dbg(dev,
+ "failed to get Log Directory Emask 0x%x\n",
+ err_mask);
+ return;
+ }
+ log_pages = get_unaligned_le16(&ap->sector_buf[log_index]);
+ if (!log_pages) {
+ ata_dev_warn(dev,
+ "NCQ Send/Recv Log not supported\n");
+ return;
+ }
+ err_mask = ata_read_log_page(dev, ATA_LOG_NCQ_SEND_RECV,
+ 0, ap->sector_buf, 1);
+ if (err_mask) {
+ ata_dev_dbg(dev,
+ "failed to get NCQ Send/Recv Log Emask 0x%x\n",
+ err_mask);
+ } else {
+ u8 *cmds = dev->ncq_send_recv_cmds;
+
+ dev->flags |= ATA_DFLAG_NCQ_SEND_RECV;
+ memcpy(cmds, ap->sector_buf, ATA_LOG_NCQ_SEND_RECV_SIZE);
+
+ if (dev->horkage & ATA_HORKAGE_NO_NCQ_TRIM) {
+ ata_dev_dbg(dev, "disabling queued TRIM support\n");
+ cmds[ATA_LOG_NCQ_SEND_RECV_DSM_OFFSET] &=
+ ~ATA_LOG_NCQ_SEND_RECV_DSM_TRIM;
+ }
+ }
+}
+
+static void ata_dev_config_ncq_non_data(struct ata_device *dev)
+{
+ struct ata_port *ap = dev->link->ap;
+ unsigned int err_mask;
+ int log_index = ATA_LOG_NCQ_NON_DATA * 2;
+ u16 log_pages;
+
+ err_mask = ata_read_log_page(dev, ATA_LOG_DIRECTORY,
+ 0, ap->sector_buf, 1);
+ if (err_mask) {
+ ata_dev_dbg(dev,
+ "failed to get Log Directory Emask 0x%x\n",
+ err_mask);
+ return;
+ }
+ log_pages = get_unaligned_le16(&ap->sector_buf[log_index]);
+ if (!log_pages) {
+ ata_dev_warn(dev,
+ "NCQ Send/Recv Log not supported\n");
+ return;
+ }
+ err_mask = ata_read_log_page(dev, ATA_LOG_NCQ_NON_DATA,
+ 0, ap->sector_buf, 1);
+ if (err_mask) {
+ ata_dev_dbg(dev,
+ "failed to get NCQ Non-Data Log Emask 0x%x\n",
+ err_mask);
+ } else {
+ u8 *cmds = dev->ncq_non_data_cmds;
+
+ memcpy(cmds, ap->sector_buf, ATA_LOG_NCQ_NON_DATA_SIZE);
+ }
+}
+
static int ata_dev_config_ncq(struct ata_device *dev,
char *desc, size_t desc_sz)
{
@@ -2123,31 +2199,127 @@
snprintf(desc, desc_sz, "NCQ (depth %d/%d)%s", hdepth,
ddepth, aa_desc);
- if ((ap->flags & ATA_FLAG_FPDMA_AUX) &&
- ata_id_has_ncq_send_and_recv(dev->id)) {
- err_mask = ata_read_log_page(dev, ATA_LOG_NCQ_SEND_RECV,
- 0, ap->sector_buf, 1);
- if (err_mask) {
- ata_dev_dbg(dev,
- "failed to get NCQ Send/Recv Log Emask 0x%x\n",
- err_mask);
- } else {
- u8 *cmds = dev->ncq_send_recv_cmds;
-
- dev->flags |= ATA_DFLAG_NCQ_SEND_RECV;
- memcpy(cmds, ap->sector_buf, ATA_LOG_NCQ_SEND_RECV_SIZE);
-
- if (dev->horkage & ATA_HORKAGE_NO_NCQ_TRIM) {
- ata_dev_dbg(dev, "disabling queued TRIM support\n");
- cmds[ATA_LOG_NCQ_SEND_RECV_DSM_OFFSET] &=
- ~ATA_LOG_NCQ_SEND_RECV_DSM_TRIM;
- }
- }
+ if ((ap->flags & ATA_FLAG_FPDMA_AUX)) {
+ if (ata_id_has_ncq_send_and_recv(dev->id))
+ ata_dev_config_ncq_send_recv(dev);
+ if (ata_id_has_ncq_non_data(dev->id))
+ ata_dev_config_ncq_non_data(dev);
}
return 0;
}
+static void ata_dev_config_sense_reporting(struct ata_device *dev)
+{
+ unsigned int err_mask;
+
+ if (!ata_id_has_sense_reporting(dev->id))
+ return;
+
+ if (ata_id_sense_reporting_enabled(dev->id))
+ return;
+
+ err_mask = ata_dev_set_feature(dev, SETFEATURE_SENSE_DATA, 0x1);
+ if (err_mask) {
+ ata_dev_dbg(dev,
+ "failed to enable Sense Data Reporting, Emask 0x%x\n",
+ err_mask);
+ }
+}
+
+static void ata_dev_config_zac(struct ata_device *dev)
+{
+ struct ata_port *ap = dev->link->ap;
+ unsigned int err_mask;
+ u8 *identify_buf = ap->sector_buf;
+ int log_index = ATA_LOG_SATA_ID_DEV_DATA * 2, i, found = 0;
+ u16 log_pages;
+
+ dev->zac_zones_optimal_open = U32_MAX;
+ dev->zac_zones_optimal_nonseq = U32_MAX;
+ dev->zac_zones_max_open = U32_MAX;
+
+ /*
+ * Always set the 'ZAC' flag for Host-managed devices.
+ */
+ if (dev->class == ATA_DEV_ZAC)
+ dev->flags |= ATA_DFLAG_ZAC;
+ else if (ata_id_zoned_cap(dev->id) == 0x01)
+ /*
+ * Check for host-aware devices.
+ */
+ dev->flags |= ATA_DFLAG_ZAC;
+
+ if (!(dev->flags & ATA_DFLAG_ZAC))
+ return;
+
+ /*
+ * Read Log Directory to figure out if IDENTIFY DEVICE log
+ * is supported.
+ */
+ err_mask = ata_read_log_page(dev, ATA_LOG_DIRECTORY,
+ 0, ap->sector_buf, 1);
+ if (err_mask) {
+ ata_dev_info(dev,
+ "failed to get Log Directory Emask 0x%x\n",
+ err_mask);
+ return;
+ }
+ log_pages = get_unaligned_le16(&ap->sector_buf[log_index]);
+ if (log_pages == 0) {
+ ata_dev_warn(dev,
+ "ATA Identify Device Log not supported\n");
+ return;
+ }
+ /*
+ * Read IDENTIFY DEVICE data log, page 0, to figure out
+ * if page 9 is supported.
+ */
+ err_mask = ata_read_log_page(dev, ATA_LOG_SATA_ID_DEV_DATA, 0,
+ identify_buf, 1);
+ if (err_mask) {
+ ata_dev_info(dev,
+ "failed to get Device Identify Log Emask 0x%x\n",
+ err_mask);
+ return;
+ }
+ log_pages = identify_buf[8];
+ for (i = 0; i < log_pages; i++) {
+ if (identify_buf[9 + i] == ATA_LOG_ZONED_INFORMATION) {
+ found++;
+ break;
+ }
+ }
+ if (!found) {
+ ata_dev_warn(dev,
+ "ATA Zoned Information Log not supported\n");
+ return;
+ }
+
+ /*
+ * Read IDENTIFY DEVICE data log, page 9 (Zoned-device information)
+ */
+ err_mask = ata_read_log_page(dev, ATA_LOG_SATA_ID_DEV_DATA,
+ ATA_LOG_ZONED_INFORMATION,
+ identify_buf, 1);
+ if (!err_mask) {
+ u64 zoned_cap, opt_open, opt_nonseq, max_open;
+
+ zoned_cap = get_unaligned_le64(&identify_buf[8]);
+ if ((zoned_cap >> 63))
+ dev->zac_zoned_cap = (zoned_cap & 1);
+ opt_open = get_unaligned_le64(&identify_buf[24]);
+ if ((opt_open >> 63))
+ dev->zac_zones_optimal_open = (u32)opt_open;
+ opt_nonseq = get_unaligned_le64(&identify_buf[32]);
+ if ((opt_nonseq >> 63))
+ dev->zac_zones_optimal_nonseq = (u32)opt_nonseq;
+ max_open = get_unaligned_le64(&identify_buf[40]);
+ if ((max_open >> 63))
+ dev->zac_zones_max_open = (u32)max_open;
+ }
+}
+
/**
* ata_dev_configure - Configure the specified ATA/ATAPI device
* @dev: Target device to configure
@@ -2370,7 +2542,8 @@
dev->devslp_timing[i] = sata_setting[j];
}
}
-
+ ata_dev_config_sense_reporting(dev);
+ ata_dev_config_zac(dev);
dev->cdb_len = 16;
}
diff --git a/drivers/ata/libata-eh.c b/drivers/ata/libata-eh.c
index 961acc7..61dc7a9 100644
--- a/drivers/ata/libata-eh.c
+++ b/drivers/ata/libata-eh.c
@@ -1600,6 +1600,8 @@
tf->hob_lbah = buf[10];
tf->nsect = buf[12];
tf->hob_nsect = buf[13];
+ if (ata_id_has_ncq_autosense(dev->id))
+ tf->auxiliary = buf[14] << 16 | buf[15] << 8 | buf[16];
return 0;
}
@@ -1636,6 +1638,56 @@
}
/**
+ * ata_eh_request_sense - perform REQUEST_SENSE_DATA_EXT
+ * @dev: device to perform REQUEST_SENSE_SENSE_DATA_EXT to
+ * @cmd: scsi command for which the sense code should be set
+ *
+ * Perform REQUEST_SENSE_DATA_EXT after the device reported CHECK
+ * SENSE. This function is an EH helper.
+ *
+ * LOCKING:
+ * Kernel thread context (may sleep).
+ */
+static void ata_eh_request_sense(struct ata_queued_cmd *qc,
+ struct scsi_cmnd *cmd)
+{
+ struct ata_device *dev = qc->dev;
+ struct ata_taskfile tf;
+ unsigned int err_mask;
+
+ if (qc->ap->pflags & ATA_PFLAG_FROZEN) {
+ ata_dev_warn(dev, "sense data available but port frozen\n");
+ return;
+ }
+
+ if (!cmd || qc->flags & ATA_QCFLAG_SENSE_VALID)
+ return;
+
+ if (!ata_id_sense_reporting_enabled(dev->id)) {
+ ata_dev_warn(qc->dev, "sense data reporting disabled\n");
+ return;
+ }
+
+ DPRINTK("ATA request sense\n");
+
+ ata_tf_init(dev, &tf);
+ tf.flags |= ATA_TFLAG_ISADDR | ATA_TFLAG_DEVICE;
+ tf.flags |= ATA_TFLAG_LBA | ATA_TFLAG_LBA48;
+ tf.command = ATA_CMD_REQ_SENSE_DATA;
+ tf.protocol = ATA_PROT_NODATA;
+
+ err_mask = ata_exec_internal(dev, &tf, NULL, DMA_NONE, NULL, 0, 0);
+ /* Ignore err_mask; ATA_ERR might be set */
+ if (tf.command & ATA_SENSE) {
+ ata_scsi_set_sense(dev, cmd, tf.lbah, tf.lbam, tf.lbal);
+ qc->flags |= ATA_QCFLAG_SENSE_VALID;
+ } else {
+ ata_dev_warn(dev, "request sense failed stat %02x emask %x\n",
+ tf.command, err_mask);
+ }
+}
+
+/**
* atapi_eh_request_sense - perform ATAPI REQUEST_SENSE
* @dev: device to perform REQUEST_SENSE to
* @sense_buf: result sense data buffer (SCSI_SENSE_BUFFERSIZE bytes long)
@@ -1797,6 +1849,18 @@
memcpy(&qc->result_tf, &tf, sizeof(tf));
qc->result_tf.flags = ATA_TFLAG_ISADDR | ATA_TFLAG_LBA | ATA_TFLAG_LBA48;
qc->err_mask |= AC_ERR_DEV | AC_ERR_NCQ;
+ if ((qc->result_tf.command & ATA_SENSE) || qc->result_tf.auxiliary) {
+ char sense_key, asc, ascq;
+
+ sense_key = (qc->result_tf.auxiliary >> 16) & 0xff;
+ asc = (qc->result_tf.auxiliary >> 8) & 0xff;
+ ascq = qc->result_tf.auxiliary & 0xff;
+ ata_scsi_set_sense(dev, qc->scsicmd, sense_key, asc, ascq);
+ ata_scsi_set_sense_information(dev, qc->scsicmd,
+ &qc->result_tf);
+ qc->flags |= ATA_QCFLAG_SENSE_VALID;
+ }
+
ehc->i.err_mask &= ~AC_ERR_DEV;
}
@@ -1826,14 +1890,23 @@
return ATA_EH_RESET;
}
- if (stat & (ATA_ERR | ATA_DF))
+ if (stat & (ATA_ERR | ATA_DF)) {
qc->err_mask |= AC_ERR_DEV;
- else
+ /*
+ * Sense data reporting does not work if the
+ * device fault bit is set.
+ */
+ if (stat & ATA_DF)
+ stat &= ~ATA_SENSE;
+ } else {
return 0;
+ }
switch (qc->dev->class) {
case ATA_DEV_ATA:
case ATA_DEV_ZAC:
+ if (stat & ATA_SENSE)
+ ata_eh_request_sense(qc, qc->scsicmd);
if (err & ATA_ICRC)
qc->err_mask |= AC_ERR_ATA_BUS;
if (err & (ATA_UNC | ATA_AMNF))
@@ -1847,20 +1920,31 @@
tmp = atapi_eh_request_sense(qc->dev,
qc->scsicmd->sense_buffer,
qc->result_tf.feature >> 4);
- if (!tmp) {
- /* ATA_QCFLAG_SENSE_VALID is used to
- * tell atapi_qc_complete() that sense
- * data is already valid.
- *
- * TODO: interpret sense data and set
- * appropriate err_mask.
- */
+ if (!tmp)
qc->flags |= ATA_QCFLAG_SENSE_VALID;
- } else
+ else
qc->err_mask |= tmp;
}
}
+ if (qc->flags & ATA_QCFLAG_SENSE_VALID) {
+ int ret = scsi_check_sense(qc->scsicmd);
+ /*
+ * SUCCESS here means that the sense code could
+ * evaluated and should be passed to the upper layers
+ * for correct evaluation.
+ * FAILED means the sense code could not interpreted
+ * and the device would need to be reset.
+ * NEEDS_RETRY and ADD_TO_MLQUEUE means that the
+ * command would need to be retried.
+ */
+ if (ret == NEEDS_RETRY || ret == ADD_TO_MLQUEUE) {
+ qc->flags |= ATA_QCFLAG_RETRY;
+ qc->err_mask |= AC_ERR_OTHER;
+ } else if (ret != SUCCESS) {
+ qc->err_mask |= AC_ERR_HSM;
+ }
+ }
if (qc->err_mask & (AC_ERR_HSM | AC_ERR_TIMEOUT | AC_ERR_ATA_BUS))
action |= ATA_EH_RESET;
@@ -2398,6 +2482,8 @@
{ ATA_CMD_CFA_WRITE_MULT_NE, "CFA WRITE MULTIPLE WITHOUT ERASE" },
{ ATA_CMD_REQ_SENSE_DATA, "REQUEST SENSE DATA EXT" },
{ ATA_CMD_SANITIZE_DEVICE, "SANITIZE DEVICE" },
+ { ATA_CMD_ZAC_MGMT_IN, "ZAC MANAGEMENT IN" },
+ { ATA_CMD_ZAC_MGMT_OUT, "ZAC MANAGEMENT OUT" },
{ ATA_CMD_READ_LONG, "READ LONG (with retries)" },
{ ATA_CMD_READ_LONG_ONCE, "READ LONG (without retries)" },
{ ATA_CMD_WRITE_LONG, "WRITE LONG (with retries)" },
@@ -2569,14 +2655,15 @@
#ifdef CONFIG_ATA_VERBOSE_ERROR
if (res->command & (ATA_BUSY | ATA_DRDY | ATA_DF | ATA_DRQ |
- ATA_ERR)) {
+ ATA_SENSE | ATA_ERR)) {
if (res->command & ATA_BUSY)
ata_dev_err(qc->dev, "status: { Busy }\n");
else
- ata_dev_err(qc->dev, "status: { %s%s%s%s}\n",
+ ata_dev_err(qc->dev, "status: { %s%s%s%s%s}\n",
res->command & ATA_DRDY ? "DRDY " : "",
res->command & ATA_DF ? "DF " : "",
res->command & ATA_DRQ ? "DRQ " : "",
+ res->command & ATA_SENSE ? "SENSE " : "",
res->command & ATA_ERR ? "ERR " : "");
}
diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c
index 7bcc870..bfec66f 100644
--- a/drivers/ata/libata-scsi.c
+++ b/drivers/ata/libata-scsi.c
@@ -270,11 +270,52 @@
ata_scsi_park_show, ata_scsi_park_store);
EXPORT_SYMBOL_GPL(dev_attr_unload_heads);
-static void ata_scsi_set_sense(struct scsi_cmnd *cmd, u8 sk, u8 asc, u8 ascq)
+void ata_scsi_set_sense(struct ata_device *dev, struct scsi_cmnd *cmd,
+ u8 sk, u8 asc, u8 ascq)
{
+ bool d_sense = (dev->flags & ATA_DFLAG_D_SENSE);
+
+ if (!cmd)
+ return;
+
cmd->result = (DRIVER_SENSE << 24) | SAM_STAT_CHECK_CONDITION;
- scsi_build_sense_buffer(0, cmd->sense_buffer, sk, asc, ascq);
+ scsi_build_sense_buffer(d_sense, cmd->sense_buffer, sk, asc, ascq);
+}
+
+void ata_scsi_set_sense_information(struct ata_device *dev,
+ struct scsi_cmnd *cmd,
+ const struct ata_taskfile *tf)
+{
+ u64 information;
+
+ if (!cmd)
+ return;
+
+ information = ata_tf_read_block(tf, dev);
+ if (information == U64_MAX)
+ return;
+
+ scsi_set_sense_information(cmd->sense_buffer,
+ SCSI_SENSE_BUFFERSIZE, information);
+}
+
+static void ata_scsi_set_invalid_field(struct ata_device *dev,
+ struct scsi_cmnd *cmd, u16 field, u8 bit)
+{
+ ata_scsi_set_sense(dev, cmd, ILLEGAL_REQUEST, 0x24, 0x0);
+ /* "Invalid field in cbd" */
+ scsi_set_sense_field_pointer(cmd->sense_buffer, SCSI_SENSE_BUFFERSIZE,
+ field, bit, 1);
+}
+
+static void ata_scsi_set_invalid_parameter(struct ata_device *dev,
+ struct scsi_cmnd *cmd, u16 field)
+{
+ /* "Invalid field in parameter list" */
+ ata_scsi_set_sense(dev, cmd, ILLEGAL_REQUEST, 0x26, 0x0);
+ scsi_set_sense_field_pointer(cmd->sense_buffer, SCSI_SENSE_BUFFERSIZE,
+ field, 0xff, 0);
}
static ssize_t
@@ -364,10 +405,10 @@
};
EXPORT_SYMBOL_GPL(ata_common_sdev_attrs);
-static void ata_scsi_invalid_field(struct scsi_cmnd *cmd)
+static void ata_scsi_invalid_field(struct ata_device *dev,
+ struct scsi_cmnd *cmd, u16 field)
{
- ata_scsi_set_sense(cmd, ILLEGAL_REQUEST, 0x24, 0x0);
- /* "Invalid field in cbd" */
+ ata_scsi_set_invalid_field(dev, cmd, field, 0xff);
cmd->scsi_done(cmd);
}
@@ -980,6 +1021,7 @@
unsigned char *sb = cmd->sense_buffer;
unsigned char *desc = sb + 8;
int verbose = qc->ap->ops->error_handler == NULL;
+ u8 sense_key, asc, ascq;
memset(sb, 0, SCSI_SENSE_BUFFERSIZE);
@@ -992,47 +1034,71 @@
if (qc->err_mask ||
tf->command & (ATA_BUSY | ATA_DF | ATA_ERR | ATA_DRQ)) {
ata_to_sense_error(qc->ap->print_id, tf->command, tf->feature,
- &sb[1], &sb[2], &sb[3], verbose);
- sb[1] &= 0x0f;
+ &sense_key, &asc, &ascq, verbose);
+ ata_scsi_set_sense(qc->dev, cmd, sense_key, asc, ascq);
} else {
- sb[1] = RECOVERED_ERROR;
- sb[2] = 0;
- sb[3] = 0x1D;
+ /*
+ * ATA PASS-THROUGH INFORMATION AVAILABLE
+ * Always in descriptor format sense.
+ */
+ scsi_build_sense_buffer(1, cmd->sense_buffer,
+ RECOVERED_ERROR, 0, 0x1D);
}
- /*
- * Sense data is current and format is descriptor.
- */
- sb[0] = 0x72;
+ if ((cmd->sense_buffer[0] & 0x7f) >= 0x72) {
+ u8 len;
- desc[0] = 0x09;
+ /* descriptor format */
+ len = sb[7];
+ desc = (char *)scsi_sense_desc_find(sb, len + 8, 9);
+ if (!desc) {
+ if (SCSI_SENSE_BUFFERSIZE < len + 14)
+ return;
+ sb[7] = len + 14;
+ desc = sb + 8 + len;
+ }
+ desc[0] = 9;
+ desc[1] = 12;
+ /*
+ * Copy registers into sense buffer.
+ */
+ desc[2] = 0x00;
+ desc[3] = tf->feature; /* == error reg */
+ desc[5] = tf->nsect;
+ desc[7] = tf->lbal;
+ desc[9] = tf->lbam;
+ desc[11] = tf->lbah;
+ desc[12] = tf->device;
+ desc[13] = tf->command; /* == status reg */
- /* set length of additional sense data */
- sb[7] = 14;
- desc[1] = 12;
-
- /*
- * Copy registers into sense buffer.
- */
- desc[2] = 0x00;
- desc[3] = tf->feature; /* == error reg */
- desc[5] = tf->nsect;
- desc[7] = tf->lbal;
- desc[9] = tf->lbam;
- desc[11] = tf->lbah;
- desc[12] = tf->device;
- desc[13] = tf->command; /* == status reg */
-
- /*
- * Fill in Extend bit, and the high order bytes
- * if applicable.
- */
- if (tf->flags & ATA_TFLAG_LBA48) {
- desc[2] |= 0x01;
- desc[4] = tf->hob_nsect;
- desc[6] = tf->hob_lbal;
- desc[8] = tf->hob_lbam;
- desc[10] = tf->hob_lbah;
+ /*
+ * Fill in Extend bit, and the high order bytes
+ * if applicable.
+ */
+ if (tf->flags & ATA_TFLAG_LBA48) {
+ desc[2] |= 0x01;
+ desc[4] = tf->hob_nsect;
+ desc[6] = tf->hob_lbal;
+ desc[8] = tf->hob_lbam;
+ desc[10] = tf->hob_lbah;
+ }
+ } else {
+ /* Fixed sense format */
+ desc[0] = tf->feature;
+ desc[1] = tf->command; /* status */
+ desc[2] = tf->device;
+ desc[3] = tf->nsect;
+ desc[0] = 0;
+ if (tf->flags & ATA_TFLAG_LBA48) {
+ desc[8] |= 0x80;
+ if (tf->hob_nsect)
+ desc[8] |= 0x40;
+ if (tf->hob_lbal || tf->hob_lbam || tf->hob_lbah)
+ desc[8] |= 0x20;
+ }
+ desc[9] = tf->lbal;
+ desc[10] = tf->lbam;
+ desc[11] = tf->lbah;
}
}
@@ -1052,41 +1118,41 @@
struct scsi_cmnd *cmd = qc->scsicmd;
struct ata_taskfile *tf = &qc->result_tf;
unsigned char *sb = cmd->sense_buffer;
- unsigned char *desc = sb + 8;
int verbose = qc->ap->ops->error_handler == NULL;
u64 block;
+ u8 sense_key, asc, ascq;
memset(sb, 0, SCSI_SENSE_BUFFERSIZE);
cmd->result = (DRIVER_SENSE << 24) | SAM_STAT_CHECK_CONDITION;
- /* sense data is current and format is descriptor */
- sb[0] = 0x72;
-
+ if (ata_dev_disabled(dev)) {
+ /* Device disabled after error recovery */
+ /* LOGICAL UNIT NOT READY, HARD RESET REQUIRED */
+ ata_scsi_set_sense(dev, cmd, NOT_READY, 0x04, 0x21);
+ return;
+ }
/* Use ata_to_sense_error() to map status register bits
* onto sense key, asc & ascq.
*/
if (qc->err_mask ||
tf->command & (ATA_BUSY | ATA_DF | ATA_ERR | ATA_DRQ)) {
ata_to_sense_error(qc->ap->print_id, tf->command, tf->feature,
- &sb[1], &sb[2], &sb[3], verbose);
- sb[1] &= 0x0f;
+ &sense_key, &asc, &ascq, verbose);
+ ata_scsi_set_sense(dev, cmd, sense_key, asc, ascq);
+ } else {
+ /* Could not decode error */
+ ata_dev_warn(dev, "could not decode error status 0x%x err_mask 0x%x\n",
+ tf->command, qc->err_mask);
+ ata_scsi_set_sense(dev, cmd, ABORTED_COMMAND, 0, 0);
+ return;
}
block = ata_tf_read_block(&qc->result_tf, dev);
+ if (block == U64_MAX)
+ return;
- /* information sense data descriptor */
- sb[7] = 12;
- desc[0] = 0x00;
- desc[1] = 10;
-
- desc[2] |= 0x80; /* valid */
- desc[6] = block >> 40;
- desc[7] = block >> 32;
- desc[8] = block >> 24;
- desc[9] = block >> 16;
- desc[10] = block >> 8;
- desc[11] = block;
+ scsi_set_sense_information(sb, SCSI_SENSE_BUFFERSIZE, block);
}
static void ata_scsi_sdev_config(struct scsi_device *sdev)
@@ -1343,19 +1409,29 @@
struct scsi_cmnd *scmd = qc->scsicmd;
struct ata_taskfile *tf = &qc->tf;
const u8 *cdb = scmd->cmnd;
+ u16 fp;
+ u8 bp = 0xff;
- if (scmd->cmd_len < 5)
+ if (scmd->cmd_len < 5) {
+ fp = 4;
goto invalid_fld;
+ }
tf->flags |= ATA_TFLAG_DEVICE | ATA_TFLAG_ISADDR;
tf->protocol = ATA_PROT_NODATA;
if (cdb[1] & 0x1) {
; /* ignore IMMED bit, violates sat-r05 */
}
- if (cdb[4] & 0x2)
+ if (cdb[4] & 0x2) {
+ fp = 4;
+ bp = 1;
goto invalid_fld; /* LOEJ bit set not supported */
- if (((cdb[4] >> 4) & 0xf) != 0)
+ }
+ if (((cdb[4] >> 4) & 0xf) != 0) {
+ fp = 4;
+ bp = 3;
goto invalid_fld; /* power conditions not supported */
+ }
if (cdb[4] & 0x1) {
tf->nsect = 1; /* 1 sector, lba=0 */
@@ -1401,8 +1477,7 @@
return 0;
invalid_fld:
- ata_scsi_set_sense(scmd, ILLEGAL_REQUEST, 0x24, 0x0);
- /* "Invalid field in cbd" */
+ ata_scsi_set_invalid_field(qc->dev, scmd, fp, bp);
return 1;
skip:
scmd->result = SAM_STAT_GOOD;
@@ -1553,20 +1628,27 @@
const u8 *cdb = scmd->cmnd;
u64 block;
u32 n_block;
+ u16 fp;
tf->flags |= ATA_TFLAG_ISADDR | ATA_TFLAG_DEVICE;
tf->protocol = ATA_PROT_NODATA;
if (cdb[0] == VERIFY) {
- if (scmd->cmd_len < 10)
+ if (scmd->cmd_len < 10) {
+ fp = 9;
goto invalid_fld;
+ }
scsi_10_lba_len(cdb, &block, &n_block);
} else if (cdb[0] == VERIFY_16) {
- if (scmd->cmd_len < 16)
+ if (scmd->cmd_len < 16) {
+ fp = 15;
goto invalid_fld;
+ }
scsi_16_lba_len(cdb, &block, &n_block);
- } else
+ } else {
+ fp = 0;
goto invalid_fld;
+ }
if (!n_block)
goto nothing_to_do;
@@ -1640,12 +1722,11 @@
return 0;
invalid_fld:
- ata_scsi_set_sense(scmd, ILLEGAL_REQUEST, 0x24, 0x0);
- /* "Invalid field in cbd" */
+ ata_scsi_set_invalid_field(qc->dev, scmd, fp, 0xff);
return 1;
out_of_range:
- ata_scsi_set_sense(scmd, ILLEGAL_REQUEST, 0x21, 0x0);
+ ata_scsi_set_sense(qc->dev, scmd, ILLEGAL_REQUEST, 0x21, 0x0);
/* "Logical Block Address out of range" */
return 1;
@@ -1680,6 +1761,7 @@
u64 block;
u32 n_block;
int rc;
+ u16 fp = 0;
if (cdb[0] == WRITE_10 || cdb[0] == WRITE_6 || cdb[0] == WRITE_16)
tf_flags |= ATA_TFLAG_WRITE;
@@ -1688,16 +1770,20 @@
switch (cdb[0]) {
case READ_10:
case WRITE_10:
- if (unlikely(scmd->cmd_len < 10))
+ if (unlikely(scmd->cmd_len < 10)) {
+ fp = 9;
goto invalid_fld;
+ }
scsi_10_lba_len(cdb, &block, &n_block);
if (cdb[1] & (1 << 3))
tf_flags |= ATA_TFLAG_FUA;
break;
case READ_6:
case WRITE_6:
- if (unlikely(scmd->cmd_len < 6))
+ if (unlikely(scmd->cmd_len < 6)) {
+ fp = 5;
goto invalid_fld;
+ }
scsi_6_lba_len(cdb, &block, &n_block);
/* for 6-byte r/w commands, transfer length 0
@@ -1708,14 +1794,17 @@
break;
case READ_16:
case WRITE_16:
- if (unlikely(scmd->cmd_len < 16))
+ if (unlikely(scmd->cmd_len < 16)) {
+ fp = 15;
goto invalid_fld;
+ }
scsi_16_lba_len(cdb, &block, &n_block);
if (cdb[1] & (1 << 3))
tf_flags |= ATA_TFLAG_FUA;
break;
default:
DPRINTK("no-byte command\n");
+ fp = 0;
goto invalid_fld;
}
@@ -1742,12 +1831,11 @@
goto out_of_range;
/* treat all other errors as -EINVAL, fall through */
invalid_fld:
- ata_scsi_set_sense(scmd, ILLEGAL_REQUEST, 0x24, 0x0);
- /* "Invalid field in cbd" */
+ ata_scsi_set_invalid_field(qc->dev, scmd, fp, 0xff);
return 1;
out_of_range:
- ata_scsi_set_sense(scmd, ILLEGAL_REQUEST, 0x21, 0x0);
+ ata_scsi_set_sense(qc->dev, scmd, ILLEGAL_REQUEST, 0x21, 0x0);
/* "Logical Block Address out of range" */
return 1;
@@ -1784,6 +1872,8 @@
if (((cdb[0] == ATA_16) || (cdb[0] == ATA_12)) &&
((cdb[2] & 0x20) || need_sense))
ata_gen_passthru_sense(qc);
+ else if (qc->flags & ATA_QCFLAG_SENSE_VALID)
+ cmd->result = SAM_STAT_CHECK_CONDITION;
else if (need_sense)
ata_gen_ata_sense(qc);
else
@@ -1992,14 +2082,14 @@
0x00,
0xA0, /* SAM-5 (no version claimed) */
- 0x04,
- 0xC0, /* SBC-3 (no version claimed) */
+ 0x06,
+ 0x00, /* SBC-4 (no version claimed) */
- 0x04,
- 0x60, /* SPC-4 (no version claimed) */
+ 0x05,
+ 0xC0, /* SPC-5 (no version claimed) */
0x60,
- 0x20, /* ZBC (no version claimed) */
+ 0x24, /* ZBC r05 */
};
u8 hdr[] = {
@@ -2019,10 +2109,8 @@
(args->dev->link->ap->pflags & ATA_PFLAG_EXTERNAL))
hdr[1] |= (1 << 7);
- if (args->dev->class == ATA_DEV_ZAC) {
+ if (args->dev->class == ATA_DEV_ZAC)
hdr[0] = TYPE_ZBC;
- hdr[2] = 0x6; /* ZBC is defined in SPC-4 */
- }
memcpy(rbuf, hdr, sizeof(hdr));
memcpy(&rbuf[8], "ATA ", 8);
@@ -2036,7 +2124,7 @@
if (rbuf[32] == 0 || rbuf[32] == ' ')
memcpy(&rbuf[32], "n/a ", 4);
- if (args->dev->class == ATA_DEV_ZAC)
+ if (ata_id_zoned_cap(args->id) || args->dev->class == ATA_DEV_ZAC)
memcpy(rbuf + 58, versions_zbc, sizeof(versions_zbc));
else
memcpy(rbuf + 58, versions, sizeof(versions));
@@ -2056,6 +2144,7 @@
*/
static unsigned int ata_scsiop_inq_00(struct ata_scsi_args *args, u8 *rbuf)
{
+ int num_pages;
const u8 pages[] = {
0x00, /* page 0x00, this page */
0x80, /* page 0x80, unit serial no page */
@@ -2064,10 +2153,14 @@
0xb0, /* page 0xb0, block limits page */
0xb1, /* page 0xb1, block device characteristics page */
0xb2, /* page 0xb2, thin provisioning page */
+ 0xb6, /* page 0xb6, zoned block device characteristics */
};
- rbuf[3] = sizeof(pages); /* number of supported VPD pages */
- memcpy(rbuf + 4, pages, sizeof(pages));
+ num_pages = sizeof(pages);
+ if (!(args->dev->flags & ATA_DFLAG_ZAC))
+ num_pages--;
+ rbuf[3] = num_pages; /* number of supported VPD pages */
+ memcpy(rbuf + 4, pages, num_pages);
return 0;
}
@@ -2232,12 +2325,15 @@
{
int form_factor = ata_id_form_factor(args->id);
int media_rotation_rate = ata_id_rotation_rate(args->id);
+ u8 zoned = ata_id_zoned_cap(args->id);
rbuf[1] = 0xb1;
rbuf[3] = 0x3c;
rbuf[4] = media_rotation_rate >> 8;
rbuf[5] = media_rotation_rate;
rbuf[7] = form_factor;
+ if (zoned)
+ rbuf[8] = (zoned << 4);
return 0;
}
@@ -2252,6 +2348,26 @@
return 0;
}
+static unsigned int ata_scsiop_inq_b6(struct ata_scsi_args *args, u8 *rbuf)
+{
+ /*
+ * zbc-r05 SCSI Zoned Block device characteristics VPD page
+ */
+ rbuf[1] = 0xb6;
+ rbuf[3] = 0x3C;
+
+ /*
+ * URSWRZ bit is only meaningful for host-managed ZAC drives
+ */
+ if (args->dev->zac_zoned_cap & 1)
+ rbuf[4] |= 1;
+ put_unaligned_be32(args->dev->zac_zones_optimal_open, &rbuf[8]);
+ put_unaligned_be32(args->dev->zac_zones_optimal_nonseq, &rbuf[12]);
+ put_unaligned_be32(args->dev->zac_zones_max_open, &rbuf[16]);
+
+ return 0;
+}
+
/**
* ata_scsiop_noop - Command handler that simply returns success.
* @args: device IDENTIFY data / SCSI command of interest.
@@ -2317,6 +2433,7 @@
/**
* ata_msense_ctl_mode - Simulate MODE SENSE control mode page
+ * @dev: ATA device of interest
* @buf: output buffer
* @changeable: whether changeable parameters are requested
*
@@ -2325,9 +2442,12 @@
* LOCKING:
* None.
*/
-static unsigned int ata_msense_ctl_mode(u8 *buf, bool changeable)
+static unsigned int ata_msense_ctl_mode(struct ata_device *dev, u8 *buf,
+ bool changeable)
{
modecpy(buf, def_control_mpage, sizeof(def_control_mpage), changeable);
+ if (changeable && (dev->flags & ATA_DFLAG_D_SENSE))
+ buf[2] |= (1 << 2); /* Descriptor sense requested */
return sizeof(def_control_mpage);
}
@@ -2395,7 +2515,8 @@
};
u8 pg, spg;
unsigned int ebd, page_control, six_byte;
- u8 dpofua;
+ u8 dpofua, bp = 0xff;
+ u16 fp;
VPRINTK("ENTER\n");
@@ -2414,6 +2535,8 @@
case 3: /* saved */
goto saving_not_supp;
default:
+ fp = 2;
+ bp = 6;
goto invalid_fld;
}
@@ -2428,8 +2551,10 @@
* No mode subpages supported (yet) but asking for _all_
* subpages may be valid
*/
- if (spg && (spg != ALL_SUB_MPAGES))
+ if (spg && (spg != ALL_SUB_MPAGES)) {
+ fp = 3;
goto invalid_fld;
+ }
switch(pg) {
case RW_RECOVERY_MPAGE:
@@ -2441,16 +2566,17 @@
break;
case CONTROL_MPAGE:
- p += ata_msense_ctl_mode(p, page_control == 1);
+ p += ata_msense_ctl_mode(args->dev, p, page_control == 1);
break;
case ALL_MPAGES:
p += ata_msense_rw_recovery(p, page_control == 1);
p += ata_msense_caching(args->id, p, page_control == 1);
- p += ata_msense_ctl_mode(p, page_control == 1);
+ p += ata_msense_ctl_mode(args->dev, p, page_control == 1);
break;
default: /* invalid page code */
+ fp = 2;
goto invalid_fld;
}
@@ -2480,12 +2606,11 @@
return 0;
invalid_fld:
- ata_scsi_set_sense(args->cmd, ILLEGAL_REQUEST, 0x24, 0x0);
- /* "Invalid field in cbd" */
+ ata_scsi_set_invalid_field(dev, args->cmd, fp, bp);
return 1;
saving_not_supp:
- ata_scsi_set_sense(args->cmd, ILLEGAL_REQUEST, 0x39, 0x0);
+ ata_scsi_set_sense(dev, args->cmd, ILLEGAL_REQUEST, 0x39, 0x0);
/* "Saving parameters not supported" */
return 1;
}
@@ -2561,6 +2686,9 @@
rbuf[14] |= 0x40; /* LBPRZ */
}
}
+ if (ata_id_zoned_cap(args->id) ||
+ args->dev->class == ATA_DEV_ZAC)
+ rbuf[12] = (1 << 4); /* RC_BASIS */
}
return 0;
}
@@ -2942,9 +3070,12 @@
struct scsi_cmnd *scmd = qc->scsicmd;
struct ata_device *dev = qc->dev;
const u8 *cdb = scmd->cmnd;
+ u16 fp;
- if ((tf->protocol = ata_scsi_map_proto(cdb[1])) == ATA_PROT_UNKNOWN)
+ if ((tf->protocol = ata_scsi_map_proto(cdb[1])) == ATA_PROT_UNKNOWN) {
+ fp = 1;
goto invalid_fld;
+ }
/* enable LBA */
tf->flags |= ATA_TFLAG_LBA;
@@ -3008,8 +3139,10 @@
case ATA_CMD_READ_LONG_ONCE:
case ATA_CMD_WRITE_LONG:
case ATA_CMD_WRITE_LONG_ONCE:
- if (tf->protocol != ATA_PROT_PIO || tf->nsect != 1)
+ if (tf->protocol != ATA_PROT_PIO || tf->nsect != 1) {
+ fp = 1;
goto invalid_fld;
+ }
qc->sect_size = scsi_bufflen(scmd);
break;
@@ -3072,12 +3205,16 @@
ata_qc_set_pc_nbytes(qc);
/* We may not issue DMA commands if no DMA mode is set */
- if (tf->protocol == ATA_PROT_DMA && dev->dma_mode == 0)
+ if (tf->protocol == ATA_PROT_DMA && dev->dma_mode == 0) {
+ fp = 1;
goto invalid_fld;
+ }
/* sanity check for pio multi commands */
- if ((cdb[1] & 0xe0) && !is_multi_taskfile(tf))
+ if ((cdb[1] & 0xe0) && !is_multi_taskfile(tf)) {
+ fp = 1;
goto invalid_fld;
+ }
if (is_multi_taskfile(tf)) {
unsigned int multi_count = 1 << (cdb[1] >> 5);
@@ -3098,8 +3235,10 @@
* ->set_dmamode(), and ->post_set_mode() hooks).
*/
if (tf->command == ATA_CMD_SET_FEATURES &&
- tf->feature == SETFEATURES_XFER)
+ tf->feature == SETFEATURES_XFER) {
+ fp = (cdb[0] == ATA_16) ? 4 : 3;
goto invalid_fld;
+ }
/*
* Filter TPM commands by default. These provide an
@@ -3116,14 +3255,15 @@
* so that we comply with the TC consortium stated goal that the user
* can turn off TC features of their system.
*/
- if (tf->command >= 0x5C && tf->command <= 0x5F && !libata_allow_tpm)
+ if (tf->command >= 0x5C && tf->command <= 0x5F && !libata_allow_tpm) {
+ fp = (cdb[0] == ATA_16) ? 14 : 9;
goto invalid_fld;
+ }
return 0;
invalid_fld:
- ata_scsi_set_sense(scmd, ILLEGAL_REQUEST, 0x24, 0x00);
- /* "Invalid field in cdb" */
+ ata_scsi_set_invalid_field(dev, scmd, fp, 0xff);
return 1;
}
@@ -3137,25 +3277,32 @@
u32 n_block;
u32 size;
void *buf;
+ u16 fp;
+ u8 bp = 0xff;
/* we may not issue DMA commands if no DMA mode is set */
if (unlikely(!dev->dma_mode))
- goto invalid_fld;
+ goto invalid_opcode;
- if (unlikely(scmd->cmd_len < 16))
+ if (unlikely(scmd->cmd_len < 16)) {
+ fp = 15;
goto invalid_fld;
+ }
scsi_16_lba_len(cdb, &block, &n_block);
/* for now we only support WRITE SAME with the unmap bit set */
- if (unlikely(!(cdb[1] & 0x8)))
+ if (unlikely(!(cdb[1] & 0x8))) {
+ fp = 1;
+ bp = 3;
goto invalid_fld;
+ }
/*
* WRITE SAME always has a sector sized buffer as payload, this
* should never be a multiple entry S/G list.
*/
if (!scsi_sg_count(scmd))
- goto invalid_fld;
+ goto invalid_param_len;
buf = page_address(sg_page(scsi_sglist(scmd)));
size = ata_set_lba_range_entries(buf, 512, block, n_block);
@@ -3186,9 +3333,242 @@
return 0;
+invalid_fld:
+ ata_scsi_set_invalid_field(dev, scmd, fp, bp);
+ return 1;
+invalid_param_len:
+ /* "Parameter list length error" */
+ ata_scsi_set_sense(dev, scmd, ILLEGAL_REQUEST, 0x1a, 0x0);
+ return 1;
+invalid_opcode:
+ /* "Invalid command operation code" */
+ ata_scsi_set_sense(dev, scmd, ILLEGAL_REQUEST, 0x20, 0x0);
+ return 1;
+}
+
+/**
+ * ata_scsi_report_zones_complete - convert ATA output
+ * @qc: command structure returning the data
+ *
+ * Convert T-13 little-endian field representation into
+ * T-10 big-endian field representation.
+ * What a mess.
+ */
+static void ata_scsi_report_zones_complete(struct ata_queued_cmd *qc)
+{
+ struct scsi_cmnd *scmd = qc->scsicmd;
+ struct sg_mapping_iter miter;
+ unsigned long flags;
+ unsigned int bytes = 0;
+
+ sg_miter_start(&miter, scsi_sglist(scmd), scsi_sg_count(scmd),
+ SG_MITER_TO_SG | SG_MITER_ATOMIC);
+
+ local_irq_save(flags);
+ while (sg_miter_next(&miter)) {
+ unsigned int offset = 0;
+
+ if (bytes == 0) {
+ char *hdr;
+ u32 list_length;
+ u64 max_lba, opt_lba;
+ u16 same;
+
+ /* Swizzle header */
+ hdr = miter.addr;
+ list_length = get_unaligned_le32(&hdr[0]);
+ same = get_unaligned_le16(&hdr[4]);
+ max_lba = get_unaligned_le64(&hdr[8]);
+ opt_lba = get_unaligned_le64(&hdr[16]);
+ put_unaligned_be32(list_length, &hdr[0]);
+ hdr[4] = same & 0xf;
+ put_unaligned_be64(max_lba, &hdr[8]);
+ put_unaligned_be64(opt_lba, &hdr[16]);
+ offset += 64;
+ bytes += 64;
+ }
+ while (offset < miter.length) {
+ char *rec;
+ u8 cond, type, non_seq, reset;
+ u64 size, start, wp;
+
+ /* Swizzle zone descriptor */
+ rec = miter.addr + offset;
+ type = rec[0] & 0xf;
+ cond = (rec[1] >> 4) & 0xf;
+ non_seq = (rec[1] & 2);
+ reset = (rec[1] & 1);
+ size = get_unaligned_le64(&rec[8]);
+ start = get_unaligned_le64(&rec[16]);
+ wp = get_unaligned_le64(&rec[24]);
+ rec[0] = type;
+ rec[1] = (cond << 4) | non_seq | reset;
+ put_unaligned_be64(size, &rec[8]);
+ put_unaligned_be64(start, &rec[16]);
+ put_unaligned_be64(wp, &rec[24]);
+ WARN_ON(offset + 64 > miter.length);
+ offset += 64;
+ bytes += 64;
+ }
+ }
+ sg_miter_stop(&miter);
+ local_irq_restore(flags);
+
+ ata_scsi_qc_complete(qc);
+}
+
+static unsigned int ata_scsi_zbc_in_xlat(struct ata_queued_cmd *qc)
+{
+ struct ata_taskfile *tf = &qc->tf;
+ struct scsi_cmnd *scmd = qc->scsicmd;
+ const u8 *cdb = scmd->cmnd;
+ u16 sect, fp = (u16)-1;
+ u8 sa, options, bp = 0xff;
+ u64 block;
+ u32 n_block;
+
+ if (unlikely(scmd->cmd_len < 16)) {
+ ata_dev_warn(qc->dev, "invalid cdb length %d\n",
+ scmd->cmd_len);
+ fp = 15;
+ goto invalid_fld;
+ }
+ scsi_16_lba_len(cdb, &block, &n_block);
+ if (n_block != scsi_bufflen(scmd)) {
+ ata_dev_warn(qc->dev, "non-matching transfer count (%d/%d)\n",
+ n_block, scsi_bufflen(scmd));
+ goto invalid_param_len;
+ }
+ sa = cdb[1] & 0x1f;
+ if (sa != ZI_REPORT_ZONES) {
+ ata_dev_warn(qc->dev, "invalid service action %d\n", sa);
+ fp = 1;
+ goto invalid_fld;
+ }
+ /*
+ * ZAC allows only for transfers in 512 byte blocks,
+ * and uses a 16 bit value for the transfer count.
+ */
+ if ((n_block / 512) > 0xffff || n_block < 512 || (n_block % 512)) {
+ ata_dev_warn(qc->dev, "invalid transfer count %d\n", n_block);
+ goto invalid_param_len;
+ }
+ sect = n_block / 512;
+ options = cdb[14];
+
+ if (ata_ncq_enabled(qc->dev) &&
+ ata_fpdma_zac_mgmt_in_supported(qc->dev)) {
+ tf->protocol = ATA_PROT_NCQ;
+ tf->command = ATA_CMD_FPDMA_RECV;
+ tf->hob_nsect = ATA_SUBCMD_FPDMA_RECV_ZAC_MGMT_IN & 0x1f;
+ tf->nsect = qc->tag << 3;
+ tf->feature = sect & 0xff;
+ tf->hob_feature = (sect >> 8) & 0xff;
+ tf->auxiliary = ATA_SUBCMD_ZAC_MGMT_IN_REPORT_ZONES;
+ } else {
+ tf->command = ATA_CMD_ZAC_MGMT_IN;
+ tf->feature = ATA_SUBCMD_ZAC_MGMT_IN_REPORT_ZONES;
+ tf->protocol = ATA_PROT_DMA;
+ tf->hob_feature = options;
+ tf->hob_nsect = (sect >> 8) & 0xff;
+ tf->nsect = sect & 0xff;
+ }
+ tf->device = ATA_LBA;
+ tf->lbah = (block >> 16) & 0xff;
+ tf->lbam = (block >> 8) & 0xff;
+ tf->lbal = block & 0xff;
+ tf->hob_lbah = (block >> 40) & 0xff;
+ tf->hob_lbam = (block >> 32) & 0xff;
+ tf->hob_lbal = (block >> 24) & 0xff;
+
+ tf->flags |= ATA_TFLAG_ISADDR | ATA_TFLAG_DEVICE | ATA_TFLAG_LBA48;
+ qc->flags |= ATA_QCFLAG_RESULT_TF;
+
+ ata_qc_set_pc_nbytes(qc);
+
+ qc->complete_fn = ata_scsi_report_zones_complete;
+
+ return 0;
+
+invalid_fld:
+ ata_scsi_set_invalid_field(qc->dev, scmd, fp, bp);
+ return 1;
+
+invalid_param_len:
+ /* "Parameter list length error" */
+ ata_scsi_set_sense(qc->dev, scmd, ILLEGAL_REQUEST, 0x1a, 0x0);
+ return 1;
+}
+
+static unsigned int ata_scsi_zbc_out_xlat(struct ata_queued_cmd *qc)
+{
+ struct ata_taskfile *tf = &qc->tf;
+ struct scsi_cmnd *scmd = qc->scsicmd;
+ struct ata_device *dev = qc->dev;
+ const u8 *cdb = scmd->cmnd;
+ u8 reset_all, sa;
+ u64 block;
+ u32 n_block;
+ u16 fp = (u16)-1;
+
+ if (unlikely(scmd->cmd_len < 16)) {
+ fp = 15;
+ goto invalid_fld;
+ }
+
+ sa = cdb[1] & 0x1f;
+ if ((sa != ZO_CLOSE_ZONE) && (sa != ZO_FINISH_ZONE) &&
+ (sa != ZO_OPEN_ZONE) && (sa != ZO_RESET_WRITE_POINTER)) {
+ fp = 1;
+ goto invalid_fld;
+ }
+
+ scsi_16_lba_len(cdb, &block, &n_block);
+ if (n_block) {
+ /*
+ * ZAC MANAGEMENT OUT doesn't define any length
+ */
+ goto invalid_param_len;
+ }
+ if (block > dev->n_sectors)
+ goto out_of_range;
+
+ reset_all = cdb[14] & 0x1;
+
+ if (ata_ncq_enabled(qc->dev) &&
+ ata_fpdma_zac_mgmt_out_supported(qc->dev)) {
+ tf->protocol = ATA_PROT_NCQ;
+ tf->command = ATA_CMD_NCQ_NON_DATA;
+ tf->hob_nsect = ATA_SUBCMD_NCQ_NON_DATA_ZAC_MGMT_OUT;
+ tf->nsect = qc->tag << 3;
+ tf->auxiliary = sa | (reset_all & 0x1) << 8;
+ } else {
+ tf->protocol = ATA_PROT_NODATA;
+ tf->command = ATA_CMD_ZAC_MGMT_OUT;
+ tf->feature = sa;
+ tf->hob_feature = reset_all & 0x1;
+ }
+ tf->lbah = (block >> 16) & 0xff;
+ tf->lbam = (block >> 8) & 0xff;
+ tf->lbal = block & 0xff;
+ tf->hob_lbah = (block >> 40) & 0xff;
+ tf->hob_lbam = (block >> 32) & 0xff;
+ tf->hob_lbal = (block >> 24) & 0xff;
+ tf->device = ATA_LBA;
+ tf->flags |= ATA_TFLAG_ISADDR | ATA_TFLAG_DEVICE | ATA_TFLAG_LBA48;
+
+ return 0;
+
invalid_fld:
- ata_scsi_set_sense(scmd, ILLEGAL_REQUEST, 0x24, 0x00);
- /* "Invalid field in cdb" */
+ ata_scsi_set_invalid_field(qc->dev, scmd, fp, 0xff);
+ return 1;
+ out_of_range:
+ /* "Logical Block Address out of range" */
+ ata_scsi_set_sense(qc->dev, scmd, ILLEGAL_REQUEST, 0x21, 0x00);
+ return 1;
+invalid_param_len:
+ /* "Parameter list length error" */
+ ata_scsi_set_sense(qc->dev, scmd, ILLEGAL_REQUEST, 0x1a, 0x0);
return 1;
}
@@ -3197,6 +3577,7 @@
* @qc: Storage for translated ATA taskfile
* @buf: input buffer
* @len: number of valid bytes in the input buffer
+ * @fp: out parameter for the failed field on error
*
* Prepare a taskfile to modify caching information for the device.
*
@@ -3204,20 +3585,26 @@
* None.
*/
static int ata_mselect_caching(struct ata_queued_cmd *qc,
- const u8 *buf, int len)
+ const u8 *buf, int len, u16 *fp)
{
struct ata_taskfile *tf = &qc->tf;
struct ata_device *dev = qc->dev;
char mpage[CACHE_MPAGE_LEN];
u8 wce;
+ int i;
/*
* The first two bytes of def_cache_mpage are a header, so offsets
* in mpage are off by 2 compared to buf. Same for len.
*/
- if (len != CACHE_MPAGE_LEN - 2)
+ if (len != CACHE_MPAGE_LEN - 2) {
+ if (len < CACHE_MPAGE_LEN - 2)
+ *fp = len;
+ else
+ *fp = CACHE_MPAGE_LEN - 2;
return -EINVAL;
+ }
wce = buf[0] & (1 << 2);
@@ -3225,10 +3612,14 @@
* Check that read-only bits are not modified.
*/
ata_msense_caching(dev->id, mpage, false);
- mpage[2] &= ~(1 << 2);
- mpage[2] |= wce;
- if (memcmp(mpage + 2, buf, CACHE_MPAGE_LEN - 2) != 0)
- return -EINVAL;
+ for (i = 0; i < CACHE_MPAGE_LEN - 2; i++) {
+ if (i == 0)
+ continue;
+ if (mpage[i + 2] != buf[i]) {
+ *fp = i;
+ return -EINVAL;
+ }
+ }
tf->flags |= ATA_TFLAG_DEVICE | ATA_TFLAG_ISADDR;
tf->protocol = ATA_PROT_NODATA;
@@ -3239,6 +3630,62 @@
}
/**
+ * ata_mselect_control - Simulate MODE SELECT for control page
+ * @qc: Storage for translated ATA taskfile
+ * @buf: input buffer
+ * @len: number of valid bytes in the input buffer
+ * @fp: out parameter for the failed field on error
+ *
+ * Prepare a taskfile to modify caching information for the device.
+ *
+ * LOCKING:
+ * None.
+ */
+static int ata_mselect_control(struct ata_queued_cmd *qc,
+ const u8 *buf, int len, u16 *fp)
+{
+ struct ata_device *dev = qc->dev;
+ char mpage[CONTROL_MPAGE_LEN];
+ u8 d_sense;
+ int i;
+
+ /*
+ * The first two bytes of def_control_mpage are a header, so offsets
+ * in mpage are off by 2 compared to buf. Same for len.
+ */
+
+ if (len != CONTROL_MPAGE_LEN - 2) {
+ if (len < CONTROL_MPAGE_LEN - 2)
+ *fp = len;
+ else
+ *fp = CONTROL_MPAGE_LEN - 2;
+ return -EINVAL;
+ }
+
+ d_sense = buf[0] & (1 << 2);
+
+ /*
+ * Check that read-only bits are not modified.
+ */
+ ata_msense_ctl_mode(dev, mpage, false);
+ for (i = 0; i < CONTROL_MPAGE_LEN - 2; i++) {
+ if (i == 0)
+ continue;
+ if (mpage[2 + i] != buf[i]) {
+ *fp = i;
+ return -EINVAL;
+ }
+ }
+ if (d_sense & (1 << 2))
+ dev->flags |= ATA_DFLAG_D_SENSE;
+ else
+ dev->flags &= ~ATA_DFLAG_D_SENSE;
+ qc->scsicmd->result = SAM_STAT_GOOD;
+ qc->scsicmd->scsi_done(qc->scsicmd);
+ return 0;
+}
+
+/**
* ata_scsiop_mode_select - Simulate MODE SELECT 6, 10 commands
* @qc: Storage for translated ATA taskfile
*
@@ -3257,27 +3704,36 @@
u8 pg, spg;
unsigned six_byte, pg_len, hdr_len, bd_len;
int len;
+ u16 fp = (u16)-1;
+ u8 bp = 0xff;
VPRINTK("ENTER\n");
six_byte = (cdb[0] == MODE_SELECT);
if (six_byte) {
- if (scmd->cmd_len < 5)
+ if (scmd->cmd_len < 5) {
+ fp = 4;
goto invalid_fld;
+ }
len = cdb[4];
hdr_len = 4;
} else {
- if (scmd->cmd_len < 9)
+ if (scmd->cmd_len < 9) {
+ fp = 8;
goto invalid_fld;
+ }
len = (cdb[7] << 8) + cdb[8];
hdr_len = 8;
}
/* We only support PF=1, SP=0. */
- if ((cdb[1] & 0x11) != 0x10)
+ if ((cdb[1] & 0x11) != 0x10) {
+ fp = 1;
+ bp = (cdb[1] & 0x01) ? 1 : 5;
goto invalid_fld;
+ }
/* Test early for possible overrun. */
if (!scsi_sg_count(scmd) || scsi_sglist(scmd)->length < len)
@@ -3298,8 +3754,11 @@
p += hdr_len;
if (len < bd_len)
goto invalid_param_len;
- if (bd_len != 0 && bd_len != 8)
+ if (bd_len != 0 && bd_len != 8) {
+ fp = (six_byte) ? 3 : 6;
+ fp += bd_len + hdr_len;
goto invalid_param;
+ }
len -= bd_len;
p += bd_len;
@@ -3330,18 +3789,29 @@
* No mode subpages supported (yet) but asking for _all_
* subpages may be valid
*/
- if (spg && (spg != ALL_SUB_MPAGES))
+ if (spg && (spg != ALL_SUB_MPAGES)) {
+ fp = (p[0] & 0x40) ? 1 : 0;
+ fp += hdr_len + bd_len;
goto invalid_param;
+ }
if (pg_len > len)
goto invalid_param_len;
switch (pg) {
case CACHE_MPAGE:
- if (ata_mselect_caching(qc, p, pg_len) < 0)
+ if (ata_mselect_caching(qc, p, pg_len, &fp) < 0) {
+ fp += hdr_len + bd_len;
goto invalid_param;
+ }
break;
-
+ case CONTROL_MPAGE:
+ if (ata_mselect_control(qc, p, pg_len, &fp) < 0) {
+ fp += hdr_len + bd_len;
+ goto invalid_param;
+ }
+ break;
default: /* invalid page code */
+ fp = bd_len + hdr_len;
goto invalid_param;
}
@@ -3355,18 +3825,16 @@
return 0;
invalid_fld:
- /* "Invalid field in CDB" */
- ata_scsi_set_sense(scmd, ILLEGAL_REQUEST, 0x24, 0x0);
+ ata_scsi_set_invalid_field(qc->dev, scmd, fp, bp);
return 1;
invalid_param:
- /* "Invalid field in parameter list" */
- ata_scsi_set_sense(scmd, ILLEGAL_REQUEST, 0x26, 0x0);
+ ata_scsi_set_invalid_parameter(qc->dev, scmd, fp);
return 1;
invalid_param_len:
/* "Parameter list length error" */
- ata_scsi_set_sense(scmd, ILLEGAL_REQUEST, 0x1a, 0x0);
+ ata_scsi_set_sense(qc->dev, scmd, ILLEGAL_REQUEST, 0x1a, 0x0);
return 1;
skip:
@@ -3419,6 +3887,12 @@
return ata_scsi_mode_select_xlat;
break;
+ case ZBC_IN:
+ return ata_scsi_zbc_in_xlat;
+
+ case ZBC_OUT:
+ return ata_scsi_zbc_out_xlat;
+
case START_STOP:
return ata_scsi_start_stop_xlat;
}
@@ -3567,12 +4041,12 @@
switch(scsicmd[0]) {
/* TODO: worth improving? */
case FORMAT_UNIT:
- ata_scsi_invalid_field(cmd);
+ ata_scsi_invalid_field(dev, cmd, 0);
break;
case INQUIRY:
- if (scsicmd[1] & 2) /* is CmdDt set? */
- ata_scsi_invalid_field(cmd);
+ if (scsicmd[1] & 2) /* is CmdDt set? */
+ ata_scsi_invalid_field(dev, cmd, 1);
else if ((scsicmd[1] & 1) == 0) /* is EVPD clear? */
ata_scsi_rbuf_fill(&args, ata_scsiop_inq_std);
else switch (scsicmd[2]) {
@@ -3597,8 +4071,14 @@
case 0xb2:
ata_scsi_rbuf_fill(&args, ata_scsiop_inq_b2);
break;
+ case 0xb6:
+ if (dev->flags & ATA_DFLAG_ZAC) {
+ ata_scsi_rbuf_fill(&args, ata_scsiop_inq_b6);
+ break;
+ }
+ /* Fallthrough */
default:
- ata_scsi_invalid_field(cmd);
+ ata_scsi_invalid_field(dev, cmd, 2);
break;
}
break;
@@ -3616,7 +4096,7 @@
if ((scsicmd[1] & 0x1f) == SAI_READ_CAPACITY_16)
ata_scsi_rbuf_fill(&args, ata_scsiop_read_cap);
else
- ata_scsi_invalid_field(cmd);
+ ata_scsi_invalid_field(dev, cmd, 1);
break;
case REPORT_LUNS:
@@ -3624,7 +4104,7 @@
break;
case REQUEST_SENSE:
- ata_scsi_set_sense(cmd, 0, 0, 0);
+ ata_scsi_set_sense(dev, cmd, 0, 0, 0);
cmd->result = (DRIVER_SENSE << 24);
cmd->scsi_done(cmd);
break;
@@ -3648,12 +4128,12 @@
if ((tmp8 == 0x4) && (!scsicmd[3]) && (!scsicmd[4]))
ata_scsi_rbuf_fill(&args, ata_scsiop_noop);
else
- ata_scsi_invalid_field(cmd);
+ ata_scsi_invalid_field(dev, cmd, 1);
break;
/* all other commands */
default:
- ata_scsi_set_sense(cmd, ILLEGAL_REQUEST, 0x20, 0x0);
+ ata_scsi_set_sense(dev, cmd, ILLEGAL_REQUEST, 0x20, 0x0);
/* "Invalid command operation code" */
cmd->scsi_done(cmd);
break;
diff --git a/drivers/ata/libata-trace.c b/drivers/ata/libata-trace.c
index fd30b8c..f8c550df 100644
--- a/drivers/ata/libata-trace.c
+++ b/drivers/ata/libata-trace.c
@@ -149,3 +149,75 @@
return ret;
}
+
+const char *
+libata_trace_parse_subcmd(struct trace_seq *p, unsigned char cmd,
+ unsigned char feature, unsigned char hob_nsect)
+{
+ const char *ret = trace_seq_buffer_ptr(p);
+
+ switch (cmd) {
+ case ATA_CMD_FPDMA_RECV:
+ switch (hob_nsect & 0x5f) {
+ case ATA_SUBCMD_FPDMA_RECV_RD_LOG_DMA_EXT:
+ trace_seq_printf(p, " READ_LOG_DMA_EXT");
+ break;
+ case ATA_SUBCMD_FPDMA_RECV_ZAC_MGMT_IN:
+ trace_seq_printf(p, " ZAC_MGMT_IN");
+ break;
+ }
+ break;
+ case ATA_CMD_FPDMA_SEND:
+ switch (hob_nsect & 0x5f) {
+ case ATA_SUBCMD_FPDMA_SEND_WR_LOG_DMA_EXT:
+ trace_seq_printf(p, " WRITE_LOG_DMA_EXT");
+ break;
+ case ATA_SUBCMD_FPDMA_SEND_DSM:
+ trace_seq_printf(p, " DATASET_MANAGEMENT");
+ break;
+ }
+ break;
+ case ATA_CMD_NCQ_NON_DATA:
+ switch (feature) {
+ case ATA_SUBCMD_NCQ_NON_DATA_ABORT_QUEUE:
+ trace_seq_printf(p, " ABORT_QUEUE");
+ break;
+ case ATA_SUBCMD_NCQ_NON_DATA_SET_FEATURES:
+ trace_seq_printf(p, " SET_FEATURES");
+ break;
+ case ATA_SUBCMD_NCQ_NON_DATA_ZERO_EXT:
+ trace_seq_printf(p, " ZERO_EXT");
+ break;
+ case ATA_SUBCMD_NCQ_NON_DATA_ZAC_MGMT_OUT:
+ trace_seq_printf(p, " ZAC_MGMT_OUT");
+ break;
+ }
+ break;
+ case ATA_CMD_ZAC_MGMT_IN:
+ switch (feature) {
+ case ATA_SUBCMD_ZAC_MGMT_IN_REPORT_ZONES:
+ trace_seq_printf(p, " REPORT_ZONES");
+ break;
+ }
+ break;
+ case ATA_CMD_ZAC_MGMT_OUT:
+ switch (feature) {
+ case ATA_SUBCMD_ZAC_MGMT_OUT_CLOSE_ZONE:
+ trace_seq_printf(p, " CLOSE_ZONE");
+ break;
+ case ATA_SUBCMD_ZAC_MGMT_OUT_FINISH_ZONE:
+ trace_seq_printf(p, " FINISH_ZONE");
+ break;
+ case ATA_SUBCMD_ZAC_MGMT_OUT_OPEN_ZONE:
+ trace_seq_printf(p, " OPEN_ZONE");
+ break;
+ case ATA_SUBCMD_ZAC_MGMT_OUT_RESET_WRITE_POINTER:
+ trace_seq_printf(p, " RESET_WRITE_POINTER");
+ break;
+ }
+ break;
+ }
+ trace_seq_putc(p, 0);
+
+ return ret;
+}
diff --git a/drivers/ata/libata.h b/drivers/ata/libata.h
index f840ca1..3b301a4 100644
--- a/drivers/ata/libata.h
+++ b/drivers/ata/libata.h
@@ -67,7 +67,8 @@
extern int ata_build_rw_tf(struct ata_taskfile *tf, struct ata_device *dev,
u64 block, u32 n_block, unsigned int tf_flags,
unsigned int tag);
-extern u64 ata_tf_read_block(struct ata_taskfile *tf, struct ata_device *dev);
+extern u64 ata_tf_read_block(const struct ata_taskfile *tf,
+ struct ata_device *dev);
extern unsigned ata_exec_internal(struct ata_device *dev,
struct ata_taskfile *tf, const u8 *cdb,
int dma_dir, void *buf, unsigned int buflen,
@@ -137,6 +138,11 @@
struct scsi_host_template *sht);
extern void ata_scsi_scan_host(struct ata_port *ap, int sync);
extern int ata_scsi_offline_dev(struct ata_device *dev);
+extern void ata_scsi_set_sense(struct ata_device *dev,
+ struct scsi_cmnd *cmd, u8 sk, u8 asc, u8 ascq);
+extern void ata_scsi_set_sense_information(struct ata_device *dev,
+ struct scsi_cmnd *cmd,
+ const struct ata_taskfile *tf);
extern void ata_scsi_media_change_notify(struct ata_device *dev);
extern void ata_scsi_hotplug(struct work_struct *work);
extern void ata_schedule_scsi_eh(struct Scsi_Host *shost);
diff --git a/drivers/ata/pata_icside.c b/drivers/ata/pata_icside.c
index d7c7320..188f2f2 100644
--- a/drivers/ata/pata_icside.c
+++ b/drivers/ata/pata_icside.c
@@ -294,7 +294,7 @@
static struct scsi_host_template pata_icside_sht = {
ATA_BASE_SHT(DRV_NAME),
- .sg_tablesize = SCSI_MAX_SG_CHAIN_SEGMENTS,
+ .sg_tablesize = SG_MAX_SEGMENTS,
.dma_boundary = IOMD_DMA_BOUNDARY,
};
diff --git a/drivers/ata/sata_dwc_460ex.c b/drivers/ata/sata_dwc_460ex.c
index 9020349..00c2af1 100644
--- a/drivers/ata/sata_dwc_460ex.c
+++ b/drivers/ata/sata_dwc_460ex.c
@@ -30,10 +30,12 @@
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/device.h>
+#include <linux/dmaengine.h>
#include <linux/of_address.h>
#include <linux/of_irq.h>
#include <linux/of_platform.h>
#include <linux/platform_device.h>
+#include <linux/phy/phy.h>
#include <linux/libata.h>
#include <linux/slab.h>
@@ -42,10 +44,6 @@
#include <scsi/scsi_host.h>
#include <scsi/scsi_cmnd.h>
-/* Supported DMA engine drivers */
-#include <linux/platform_data/dma-dw.h>
-#include <linux/dma/dw.h>
-
/* These two are defined in "libata.h" */
#undef DRV_NAME
#undef DRV_VERSION
@@ -53,19 +51,14 @@
#define DRV_NAME "sata-dwc"
#define DRV_VERSION "1.3"
-#ifndef out_le32
-#define out_le32(a, v) __raw_writel(__cpu_to_le32(v), (void __iomem *)(a))
-#endif
-
-#ifndef in_le32
-#define in_le32(a) __le32_to_cpu(__raw_readl((void __iomem *)(a)))
-#endif
+#define sata_dwc_writel(a, v) writel_relaxed(v, a)
+#define sata_dwc_readl(a) readl_relaxed(a)
#ifndef NO_IRQ
#define NO_IRQ 0
#endif
-#define AHB_DMA_BRST_DFLT 64 /* 16 data items burst length*/
+#define AHB_DMA_BRST_DFLT 64 /* 16 data items burst length */
enum {
SATA_DWC_MAX_PORTS = 1,
@@ -102,7 +95,7 @@
u32 versionr; /* Version Register */
u32 idr; /* ID Register */
u32 unimpl[192]; /* Unimplemented */
- u32 dmadr[256]; /* FIFO Locations in DMA Mode */
+ u32 dmadr[256]; /* FIFO Locations in DMA Mode */
};
enum {
@@ -146,9 +139,14 @@
struct device *dev; /* generic device struct */
struct ata_probe_ent *pe; /* ptr to probe-ent */
struct ata_host *host;
- u8 __iomem *reg_base;
- struct sata_dwc_regs *sata_dwc_regs; /* DW Synopsys SATA specific */
+ struct sata_dwc_regs __iomem *sata_dwc_regs; /* DW SATA specific */
+ u32 sactive_issued;
+ u32 sactive_queued;
+ struct phy *phy;
+ phys_addr_t dmadr;
+#ifdef CONFIG_SATA_DWC_OLD_DMA
struct dw_dma_chip *dma;
+#endif
};
#define SATA_DWC_QCMD_MAX 32
@@ -159,25 +157,19 @@
int dma_pending[SATA_DWC_QCMD_MAX];
/* DMA info */
- struct dw_dma_slave *dws;
struct dma_chan *chan;
struct dma_async_tx_descriptor *desc[SATA_DWC_QCMD_MAX];
u32 dma_interrupt_count;
};
/*
- * Commonly used DWC SATA driver Macros
+ * Commonly used DWC SATA driver macros
*/
-#define HSDEV_FROM_HOST(host) ((struct sata_dwc_device *)\
- (host)->private_data)
-#define HSDEV_FROM_AP(ap) ((struct sata_dwc_device *)\
- (ap)->host->private_data)
-#define HSDEVP_FROM_AP(ap) ((struct sata_dwc_device_port *)\
- (ap)->private_data)
-#define HSDEV_FROM_QC(qc) ((struct sata_dwc_device *)\
- (qc)->ap->host->private_data)
-#define HSDEV_FROM_HSDEVP(p) ((struct sata_dwc_device *)\
- (hsdevp)->hsdev)
+#define HSDEV_FROM_HOST(host) ((struct sata_dwc_device *)(host)->private_data)
+#define HSDEV_FROM_AP(ap) ((struct sata_dwc_device *)(ap)->host->private_data)
+#define HSDEVP_FROM_AP(ap) ((struct sata_dwc_device_port *)(ap)->private_data)
+#define HSDEV_FROM_QC(qc) ((struct sata_dwc_device *)(qc)->ap->host->private_data)
+#define HSDEV_FROM_HSDEVP(p) ((struct sata_dwc_device *)(p)->hsdev)
enum {
SATA_DWC_CMD_ISSUED_NOT = 0,
@@ -190,21 +182,6 @@
SATA_DWC_DMA_PENDING_RX = 2,
};
-struct sata_dwc_host_priv {
- void __iomem *scr_addr_sstatus;
- u32 sata_dwc_sactive_issued ;
- u32 sata_dwc_sactive_queued ;
-};
-
-static struct sata_dwc_host_priv host_pvt;
-
-static struct dw_dma_slave sata_dwc_dma_dws = {
- .src_id = 0,
- .dst_id = 0,
- .src_master = 0,
- .dst_master = 1,
-};
-
/*
* Prototypes
*/
@@ -215,6 +192,93 @@
static void sata_dwc_port_stop(struct ata_port *ap);
static void sata_dwc_clear_dmacr(struct sata_dwc_device_port *hsdevp, u8 tag);
+#ifdef CONFIG_SATA_DWC_OLD_DMA
+
+#include <linux/platform_data/dma-dw.h>
+#include <linux/dma/dw.h>
+
+static struct dw_dma_slave sata_dwc_dma_dws = {
+ .src_id = 0,
+ .dst_id = 0,
+ .m_master = 1,
+ .p_master = 0,
+};
+
+static bool sata_dwc_dma_filter(struct dma_chan *chan, void *param)
+{
+ struct dw_dma_slave *dws = &sata_dwc_dma_dws;
+
+ if (dws->dma_dev != chan->device->dev)
+ return false;
+
+ chan->private = dws;
+ return true;
+}
+
+static int sata_dwc_dma_get_channel_old(struct sata_dwc_device_port *hsdevp)
+{
+ struct sata_dwc_device *hsdev = hsdevp->hsdev;
+ struct dw_dma_slave *dws = &sata_dwc_dma_dws;
+ dma_cap_mask_t mask;
+
+ dws->dma_dev = hsdev->dev;
+
+ dma_cap_zero(mask);
+ dma_cap_set(DMA_SLAVE, mask);
+
+ /* Acquire DMA channel */
+ hsdevp->chan = dma_request_channel(mask, sata_dwc_dma_filter, hsdevp);
+ if (!hsdevp->chan) {
+ dev_err(hsdev->dev, "%s: dma channel unavailable\n",
+ __func__);
+ return -EAGAIN;
+ }
+
+ return 0;
+}
+
+static int sata_dwc_dma_init_old(struct platform_device *pdev,
+ struct sata_dwc_device *hsdev)
+{
+ struct device_node *np = pdev->dev.of_node;
+ struct resource *res;
+
+ hsdev->dma = devm_kzalloc(&pdev->dev, sizeof(*hsdev->dma), GFP_KERNEL);
+ if (!hsdev->dma)
+ return -ENOMEM;
+
+ hsdev->dma->dev = &pdev->dev;
+
+ /* Get SATA DMA interrupt number */
+ hsdev->dma->irq = irq_of_parse_and_map(np, 1);
+ if (hsdev->dma->irq == NO_IRQ) {
+ dev_err(&pdev->dev, "no SATA DMA irq\n");
+ return -ENODEV;
+ }
+
+ /* Get physical SATA DMA register base address */
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 1);
+ hsdev->dma->regs = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(hsdev->dma->regs)) {
+ dev_err(&pdev->dev,
+ "ioremap failed for AHBDMA register address\n");
+ return PTR_ERR(hsdev->dma->regs);
+ }
+
+ /* Initialize AHB DMAC */
+ return dw_dma_probe(hsdev->dma);
+}
+
+static void sata_dwc_dma_exit_old(struct sata_dwc_device *hsdev)
+{
+ if (!hsdev->dma)
+ return;
+
+ dw_dma_remove(hsdev->dma);
+}
+
+#endif
+
static const char *get_prot_descript(u8 protocol)
{
switch ((enum ata_tf_protocols)protocol) {
@@ -305,21 +369,20 @@
struct ata_port *ap = qc->ap;
struct sata_dwc_device_port *hsdevp = HSDEVP_FROM_AP(ap);
struct sata_dwc_device *hsdev = HSDEV_FROM_AP(ap);
- dma_addr_t addr = (dma_addr_t)&hsdev->sata_dwc_regs->dmadr;
struct dma_slave_config sconf;
struct dma_async_tx_descriptor *desc;
if (qc->dma_dir == DMA_DEV_TO_MEM) {
- sconf.src_addr = addr;
- sconf.device_fc = true;
+ sconf.src_addr = hsdev->dmadr;
+ sconf.device_fc = false;
} else { /* DMA_MEM_TO_DEV */
- sconf.dst_addr = addr;
+ sconf.dst_addr = hsdev->dmadr;
sconf.device_fc = false;
}
sconf.direction = qc->dma_dir;
- sconf.src_maxburst = AHB_DMA_BRST_DFLT;
- sconf.dst_maxburst = AHB_DMA_BRST_DFLT;
+ sconf.src_maxburst = AHB_DMA_BRST_DFLT / 4; /* in items */
+ sconf.dst_maxburst = AHB_DMA_BRST_DFLT / 4; /* in items */
sconf.src_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES;
sconf.dst_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES;
@@ -336,8 +399,8 @@
desc->callback = dma_dwc_xfer_done;
desc->callback_param = hsdev;
- dev_dbg(hsdev->dev, "%s sg: 0x%p, count: %d addr: %pad\n",
- __func__, qc->sg, qc->n_elem, &addr);
+ dev_dbg(hsdev->dev, "%s sg: 0x%p, count: %d addr: %pa\n", __func__,
+ qc->sg, qc->n_elem, &hsdev->dmadr);
return desc;
}
@@ -350,48 +413,38 @@
return -EINVAL;
}
- *val = in_le32(link->ap->ioaddr.scr_addr + (scr * 4));
- dev_dbg(link->ap->dev, "%s: id=%d reg=%d val=val=0x%08x\n",
- __func__, link->ap->print_id, scr, *val);
+ *val = sata_dwc_readl(link->ap->ioaddr.scr_addr + (scr * 4));
+ dev_dbg(link->ap->dev, "%s: id=%d reg=%d val=0x%08x\n", __func__,
+ link->ap->print_id, scr, *val);
return 0;
}
static int sata_dwc_scr_write(struct ata_link *link, unsigned int scr, u32 val)
{
- dev_dbg(link->ap->dev, "%s: id=%d reg=%d val=val=0x%08x\n",
- __func__, link->ap->print_id, scr, val);
+ dev_dbg(link->ap->dev, "%s: id=%d reg=%d val=0x%08x\n", __func__,
+ link->ap->print_id, scr, val);
if (scr > SCR_NOTIFICATION) {
dev_err(link->ap->dev, "%s: Incorrect SCR offset 0x%02x\n",
__func__, scr);
return -EINVAL;
}
- out_le32(link->ap->ioaddr.scr_addr + (scr * 4), val);
+ sata_dwc_writel(link->ap->ioaddr.scr_addr + (scr * 4), val);
return 0;
}
-static u32 core_scr_read(unsigned int scr)
-{
- return in_le32(host_pvt.scr_addr_sstatus + (scr * 4));
-}
-
-static void core_scr_write(unsigned int scr, u32 val)
-{
- out_le32(host_pvt.scr_addr_sstatus + (scr * 4), val);
-}
-
-static void clear_serror(void)
+static void clear_serror(struct ata_port *ap)
{
u32 val;
- val = core_scr_read(SCR_ERROR);
- core_scr_write(SCR_ERROR, val);
+ sata_dwc_scr_read(&ap->link, SCR_ERROR, &val);
+ sata_dwc_scr_write(&ap->link, SCR_ERROR, val);
}
static void clear_interrupt_bit(struct sata_dwc_device *hsdev, u32 bit)
{
- out_le32(&hsdev->sata_dwc_regs->intpr,
- in_le32(&hsdev->sata_dwc_regs->intpr));
+ sata_dwc_writel(&hsdev->sata_dwc_regs->intpr,
+ sata_dwc_readl(&hsdev->sata_dwc_regs->intpr));
}
static u32 qcmd_tag_to_mask(u8 tag)
@@ -412,7 +465,7 @@
ata_ehi_clear_desc(ehi);
- serror = core_scr_read(SCR_ERROR);
+ sata_dwc_scr_read(&ap->link, SCR_ERROR, &serror);
status = ap->ops->sff_check_status(ap);
tag = ap->link.active_tag;
@@ -423,7 +476,7 @@
hsdevp->dma_pending[tag], hsdevp->cmd_issued[tag]);
/* Clear error register and interrupt bit */
- clear_serror();
+ clear_serror(ap);
clear_interrupt_bit(hsdev, SATA_DWC_INTPR_ERR);
/* This is the only error happening now. TODO check for exact error */
@@ -462,12 +515,12 @@
int handled, num_processed, port = 0;
uint intpr, sactive, sactive2, tag_mask;
struct sata_dwc_device_port *hsdevp;
- host_pvt.sata_dwc_sactive_issued = 0;
+ hsdev->sactive_issued = 0;
spin_lock_irqsave(&host->lock, flags);
/* Read the interrupt register */
- intpr = in_le32(&hsdev->sata_dwc_regs->intpr);
+ intpr = sata_dwc_readl(&hsdev->sata_dwc_regs->intpr);
ap = host->ports[port];
hsdevp = HSDEVP_FROM_AP(ap);
@@ -486,12 +539,12 @@
if (intpr & SATA_DWC_INTPR_NEWFP) {
clear_interrupt_bit(hsdev, SATA_DWC_INTPR_NEWFP);
- tag = (u8)(in_le32(&hsdev->sata_dwc_regs->fptagr));
+ tag = (u8)(sata_dwc_readl(&hsdev->sata_dwc_regs->fptagr));
dev_dbg(ap->dev, "%s: NEWFP tag=%d\n", __func__, tag);
if (hsdevp->cmd_issued[tag] != SATA_DWC_CMD_ISSUED_PEND)
dev_warn(ap->dev, "CMD tag=%d not pending?\n", tag);
- host_pvt.sata_dwc_sactive_issued |= qcmd_tag_to_mask(tag);
+ hsdev->sactive_issued |= qcmd_tag_to_mask(tag);
qc = ata_qc_from_tag(ap, tag);
/*
@@ -505,11 +558,11 @@
handled = 1;
goto DONE;
}
- sactive = core_scr_read(SCR_ACTIVE);
- tag_mask = (host_pvt.sata_dwc_sactive_issued | sactive) ^ sactive;
+ sata_dwc_scr_read(&ap->link, SCR_ACTIVE, &sactive);
+ tag_mask = (hsdev->sactive_issued | sactive) ^ sactive;
/* If no sactive issued and tag_mask is zero then this is not NCQ */
- if (host_pvt.sata_dwc_sactive_issued == 0 && tag_mask == 0) {
+ if (hsdev->sactive_issued == 0 && tag_mask == 0) {
if (ap->link.active_tag == ATA_TAG_POISON)
tag = 0;
else
@@ -579,22 +632,19 @@
*/
/* process completed commands */
- sactive = core_scr_read(SCR_ACTIVE);
- tag_mask = (host_pvt.sata_dwc_sactive_issued | sactive) ^ sactive;
+ sata_dwc_scr_read(&ap->link, SCR_ACTIVE, &sactive);
+ tag_mask = (hsdev->sactive_issued | sactive) ^ sactive;
- if (sactive != 0 || (host_pvt.sata_dwc_sactive_issued) > 1 || \
- tag_mask > 1) {
+ if (sactive != 0 || hsdev->sactive_issued > 1 || tag_mask > 1) {
dev_dbg(ap->dev,
"%s NCQ:sactive=0x%08x sactive_issued=0x%08x tag_mask=0x%08x\n",
- __func__, sactive, host_pvt.sata_dwc_sactive_issued,
- tag_mask);
+ __func__, sactive, hsdev->sactive_issued, tag_mask);
}
- if ((tag_mask | (host_pvt.sata_dwc_sactive_issued)) != \
- (host_pvt.sata_dwc_sactive_issued)) {
+ if ((tag_mask | hsdev->sactive_issued) != hsdev->sactive_issued) {
dev_warn(ap->dev,
- "Bad tag mask? sactive=0x%08x (host_pvt.sata_dwc_sactive_issued)=0x%08x tag_mask=0x%08x\n",
- sactive, host_pvt.sata_dwc_sactive_issued, tag_mask);
+ "Bad tag mask? sactive=0x%08x sactive_issued=0x%08x tag_mask=0x%08x\n",
+ sactive, hsdev->sactive_issued, tag_mask);
}
/* read just to clear ... not bad if currently still busy */
@@ -656,7 +706,7 @@
* we were processing --we read status as part of processing a completed
* command).
*/
- sactive2 = core_scr_read(SCR_ACTIVE);
+ sata_dwc_scr_read(&ap->link, SCR_ACTIVE, &sactive2);
if (sactive2 != sactive) {
dev_dbg(ap->dev,
"More completed - sactive=0x%x sactive2=0x%x\n",
@@ -672,15 +722,14 @@
static void sata_dwc_clear_dmacr(struct sata_dwc_device_port *hsdevp, u8 tag)
{
struct sata_dwc_device *hsdev = HSDEV_FROM_HSDEVP(hsdevp);
+ u32 dmacr = sata_dwc_readl(&hsdev->sata_dwc_regs->dmacr);
if (hsdevp->dma_pending[tag] == SATA_DWC_DMA_PENDING_RX) {
- out_le32(&(hsdev->sata_dwc_regs->dmacr),
- SATA_DWC_DMACR_RX_CLEAR(
- in_le32(&(hsdev->sata_dwc_regs->dmacr))));
+ dmacr = SATA_DWC_DMACR_RX_CLEAR(dmacr);
+ sata_dwc_writel(&hsdev->sata_dwc_regs->dmacr, dmacr);
} else if (hsdevp->dma_pending[tag] == SATA_DWC_DMA_PENDING_TX) {
- out_le32(&(hsdev->sata_dwc_regs->dmacr),
- SATA_DWC_DMACR_TX_CLEAR(
- in_le32(&(hsdev->sata_dwc_regs->dmacr))));
+ dmacr = SATA_DWC_DMACR_TX_CLEAR(dmacr);
+ sata_dwc_writel(&hsdev->sata_dwc_regs->dmacr, dmacr);
} else {
/*
* This should not happen, it indicates the driver is out of
@@ -688,10 +737,9 @@
*/
dev_err(hsdev->dev,
"%s DMA protocol RX and TX DMA not pending tag=0x%02x pending=%d dmacr: 0x%08x\n",
- __func__, tag, hsdevp->dma_pending[tag],
- in_le32(&hsdev->sata_dwc_regs->dmacr));
- out_le32(&(hsdev->sata_dwc_regs->dmacr),
- SATA_DWC_DMACR_TXRXCH_CLEAR);
+ __func__, tag, hsdevp->dma_pending[tag], dmacr);
+ sata_dwc_writel(&hsdev->sata_dwc_regs->dmacr,
+ SATA_DWC_DMACR_TXRXCH_CLEAR);
}
}
@@ -716,7 +764,7 @@
__func__, qc->tag, qc->tf.command,
get_dma_dir_descript(qc->dma_dir),
get_prot_descript(qc->tf.protocol),
- in_le32(&(hsdev->sata_dwc_regs->dmacr)));
+ sata_dwc_readl(&hsdev->sata_dwc_regs->dmacr));
}
#endif
@@ -725,7 +773,7 @@
dev_err(ap->dev,
"%s DMA protocol RX and TX DMA not pending dmacr: 0x%08x\n",
__func__,
- in_le32(&(hsdev->sata_dwc_regs->dmacr)));
+ sata_dwc_readl(&hsdev->sata_dwc_regs->dmacr));
}
hsdevp->dma_pending[tag] = SATA_DWC_DMA_PENDING_NONE;
@@ -742,8 +790,9 @@
u8 status = 0;
u32 mask = 0x0;
u8 tag = qc->tag;
+ struct sata_dwc_device *hsdev = HSDEV_FROM_AP(ap);
struct sata_dwc_device_port *hsdevp = HSDEVP_FROM_AP(ap);
- host_pvt.sata_dwc_sactive_queued = 0;
+ hsdev->sactive_queued = 0;
dev_dbg(ap->dev, "%s checkstatus? %x\n", __func__, check_status);
if (hsdevp->dma_pending[tag] == SATA_DWC_DMA_PENDING_TX)
@@ -756,10 +805,8 @@
/* clear active bit */
mask = (~(qcmd_tag_to_mask(tag)));
- host_pvt.sata_dwc_sactive_queued = (host_pvt.sata_dwc_sactive_queued) \
- & mask;
- host_pvt.sata_dwc_sactive_issued = (host_pvt.sata_dwc_sactive_issued) \
- & mask;
+ hsdev->sactive_queued = hsdev->sactive_queued & mask;
+ hsdev->sactive_issued = hsdev->sactive_issued & mask;
ata_qc_complete(qc);
return 0;
}
@@ -767,54 +814,62 @@
static void sata_dwc_enable_interrupts(struct sata_dwc_device *hsdev)
{
/* Enable selective interrupts by setting the interrupt maskregister*/
- out_le32(&hsdev->sata_dwc_regs->intmr,
- SATA_DWC_INTMR_ERRM |
- SATA_DWC_INTMR_NEWFPM |
- SATA_DWC_INTMR_PMABRTM |
- SATA_DWC_INTMR_DMATM);
+ sata_dwc_writel(&hsdev->sata_dwc_regs->intmr,
+ SATA_DWC_INTMR_ERRM |
+ SATA_DWC_INTMR_NEWFPM |
+ SATA_DWC_INTMR_PMABRTM |
+ SATA_DWC_INTMR_DMATM);
/*
* Unmask the error bits that should trigger an error interrupt by
* setting the error mask register.
*/
- out_le32(&hsdev->sata_dwc_regs->errmr, SATA_DWC_SERROR_ERR_BITS);
+ sata_dwc_writel(&hsdev->sata_dwc_regs->errmr, SATA_DWC_SERROR_ERR_BITS);
dev_dbg(hsdev->dev, "%s: INTMR = 0x%08x, ERRMR = 0x%08x\n",
- __func__, in_le32(&hsdev->sata_dwc_regs->intmr),
- in_le32(&hsdev->sata_dwc_regs->errmr));
+ __func__, sata_dwc_readl(&hsdev->sata_dwc_regs->intmr),
+ sata_dwc_readl(&hsdev->sata_dwc_regs->errmr));
}
-static bool sata_dwc_dma_filter(struct dma_chan *chan, void *param)
+static void sata_dwc_setup_port(struct ata_ioports *port, void __iomem *base)
{
- struct sata_dwc_device_port *hsdevp = param;
- struct dw_dma_slave *dws = hsdevp->dws;
+ port->cmd_addr = base + 0x00;
+ port->data_addr = base + 0x00;
- if (dws->dma_dev != chan->device->dev)
- return false;
+ port->error_addr = base + 0x04;
+ port->feature_addr = base + 0x04;
- chan->private = dws;
- return true;
+ port->nsect_addr = base + 0x08;
+
+ port->lbal_addr = base + 0x0c;
+ port->lbam_addr = base + 0x10;
+ port->lbah_addr = base + 0x14;
+
+ port->device_addr = base + 0x18;
+ port->command_addr = base + 0x1c;
+ port->status_addr = base + 0x1c;
+
+ port->altstatus_addr = base + 0x20;
+ port->ctl_addr = base + 0x20;
}
-static void sata_dwc_setup_port(struct ata_ioports *port, unsigned long base)
+static int sata_dwc_dma_get_channel(struct sata_dwc_device_port *hsdevp)
{
- port->cmd_addr = (void __iomem *)base + 0x00;
- port->data_addr = (void __iomem *)base + 0x00;
+ struct sata_dwc_device *hsdev = hsdevp->hsdev;
+ struct device *dev = hsdev->dev;
- port->error_addr = (void __iomem *)base + 0x04;
- port->feature_addr = (void __iomem *)base + 0x04;
+#ifdef CONFIG_SATA_DWC_OLD_DMA
+ if (!of_find_property(dev->of_node, "dmas", NULL))
+ return sata_dwc_dma_get_channel_old(hsdevp);
+#endif
- port->nsect_addr = (void __iomem *)base + 0x08;
+ hsdevp->chan = dma_request_chan(dev, "sata-dma");
+ if (IS_ERR(hsdevp->chan)) {
+ dev_err(dev, "failed to allocate dma channel: %ld\n",
+ PTR_ERR(hsdevp->chan));
+ return PTR_ERR(hsdevp->chan);
+ }
- port->lbal_addr = (void __iomem *)base + 0x0c;
- port->lbam_addr = (void __iomem *)base + 0x10;
- port->lbah_addr = (void __iomem *)base + 0x14;
-
- port->device_addr = (void __iomem *)base + 0x18;
- port->command_addr = (void __iomem *)base + 0x1c;
- port->status_addr = (void __iomem *)base + 0x1c;
-
- port->altstatus_addr = (void __iomem *)base + 0x20;
- port->ctl_addr = (void __iomem *)base + 0x20;
+ return 0;
}
/*
@@ -829,7 +884,6 @@
struct sata_dwc_device *hsdev;
struct sata_dwc_device_port *hsdevp = NULL;
struct device *pdev;
- dma_cap_mask_t mask;
int i;
hsdev = HSDEV_FROM_AP(ap);
@@ -853,20 +907,13 @@
}
hsdevp->hsdev = hsdev;
- hsdevp->dws = &sata_dwc_dma_dws;
- hsdevp->dws->dma_dev = hsdev->dev;
-
- dma_cap_zero(mask);
- dma_cap_set(DMA_SLAVE, mask);
-
- /* Acquire DMA channel */
- hsdevp->chan = dma_request_channel(mask, sata_dwc_dma_filter, hsdevp);
- if (!hsdevp->chan) {
- dev_err(hsdev->dev, "%s: dma channel unavailable\n",
- __func__);
- err = -EAGAIN;
+ err = sata_dwc_dma_get_channel(hsdevp);
+ if (err)
goto CLEANUP_ALLOC;
- }
+
+ err = phy_power_on(hsdev->phy);
+ if (err)
+ goto CLEANUP_ALLOC;
for (i = 0; i < SATA_DWC_QCMD_MAX; i++)
hsdevp->cmd_issued[i] = SATA_DWC_CMD_ISSUED_NOT;
@@ -877,18 +924,18 @@
if (ap->port_no == 0) {
dev_dbg(ap->dev, "%s: clearing TXCHEN, RXCHEN in DMAC\n",
__func__);
- out_le32(&hsdev->sata_dwc_regs->dmacr,
- SATA_DWC_DMACR_TXRXCH_CLEAR);
+ sata_dwc_writel(&hsdev->sata_dwc_regs->dmacr,
+ SATA_DWC_DMACR_TXRXCH_CLEAR);
dev_dbg(ap->dev, "%s: setting burst size in DBTSR\n",
__func__);
- out_le32(&hsdev->sata_dwc_regs->dbtsr,
- (SATA_DWC_DBTSR_MWR(AHB_DMA_BRST_DFLT) |
- SATA_DWC_DBTSR_MRD(AHB_DMA_BRST_DFLT)));
+ sata_dwc_writel(&hsdev->sata_dwc_regs->dbtsr,
+ (SATA_DWC_DBTSR_MWR(AHB_DMA_BRST_DFLT) |
+ SATA_DWC_DBTSR_MRD(AHB_DMA_BRST_DFLT)));
}
/* Clear any error bits before libata starts issuing commands */
- clear_serror();
+ clear_serror(ap);
ap->private_data = hsdevp;
dev_dbg(ap->dev, "%s: done\n", __func__);
return 0;
@@ -903,11 +950,13 @@
static void sata_dwc_port_stop(struct ata_port *ap)
{
struct sata_dwc_device_port *hsdevp = HSDEVP_FROM_AP(ap);
+ struct sata_dwc_device *hsdev = HSDEV_FROM_AP(ap);
dev_dbg(ap->dev, "%s: ap->id = %d\n", __func__, ap->print_id);
- dmaengine_terminate_all(hsdevp->chan);
+ dmaengine_terminate_sync(hsdevp->chan);
dma_release_channel(hsdevp->chan);
+ phy_power_off(hsdev->phy);
kfree(hsdevp);
ap->private_data = NULL;
@@ -924,22 +973,20 @@
struct ata_taskfile *tf,
u8 tag, u32 cmd_issued)
{
- unsigned long flags;
struct sata_dwc_device_port *hsdevp = HSDEVP_FROM_AP(ap);
dev_dbg(ap->dev, "%s cmd(0x%02x): %s tag=%d\n", __func__, tf->command,
ata_get_cmd_descript(tf->command), tag);
- spin_lock_irqsave(&ap->host->lock, flags);
hsdevp->cmd_issued[tag] = cmd_issued;
- spin_unlock_irqrestore(&ap->host->lock, flags);
+
/*
* Clear SError before executing a new command.
* sata_dwc_scr_write and read can not be used here. Clearing the PM
* managed SError register for the disk needs to be done before the
* task file is loaded.
*/
- clear_serror();
+ clear_serror(ap);
ata_sff_exec_command(ap, tf);
}
@@ -992,18 +1039,18 @@
sata_dwc_tf_dump(ap, &qc->tf);
if (start_dma) {
- reg = core_scr_read(SCR_ERROR);
+ sata_dwc_scr_read(&ap->link, SCR_ERROR, ®);
if (reg & SATA_DWC_SERROR_ERR_BITS) {
dev_err(ap->dev, "%s: ****** SError=0x%08x ******\n",
__func__, reg);
}
if (dir == DMA_TO_DEVICE)
- out_le32(&hsdev->sata_dwc_regs->dmacr,
- SATA_DWC_DMACR_TXCHEN);
+ sata_dwc_writel(&hsdev->sata_dwc_regs->dmacr,
+ SATA_DWC_DMACR_TXCHEN);
else
- out_le32(&hsdev->sata_dwc_regs->dmacr,
- SATA_DWC_DMACR_RXCHEN);
+ sata_dwc_writel(&hsdev->sata_dwc_regs->dmacr,
+ SATA_DWC_DMACR_RXCHEN);
/* Enable AHB DMA transfer on the specified channel */
dmaengine_submit(desc);
@@ -1025,36 +1072,12 @@
sata_dwc_bmdma_start_by_tag(qc, tag);
}
-/*
- * Function : sata_dwc_qc_prep_by_tag
- * arguments : ata_queued_cmd *qc, u8 tag
- * Return value : None
- * qc_prep for a particular queued command based on tag
- */
-static void sata_dwc_qc_prep_by_tag(struct ata_queued_cmd *qc, u8 tag)
-{
- struct dma_async_tx_descriptor *desc;
- struct ata_port *ap = qc->ap;
- struct sata_dwc_device_port *hsdevp = HSDEVP_FROM_AP(ap);
-
- dev_dbg(ap->dev, "%s: port=%d dma dir=%s n_elem=%d\n",
- __func__, ap->port_no, get_dma_dir_descript(qc->dma_dir),
- qc->n_elem);
-
- desc = dma_dwc_xfer_setup(qc);
- if (!desc) {
- dev_err(ap->dev, "%s: dma_dwc_xfer_setup returns NULL\n",
- __func__);
- return;
- }
- hsdevp->desc[tag] = desc;
-}
-
static unsigned int sata_dwc_qc_issue(struct ata_queued_cmd *qc)
{
u32 sactive;
u8 tag = qc->tag;
struct ata_port *ap = qc->ap;
+ struct sata_dwc_device_port *hsdevp = HSDEVP_FROM_AP(ap);
#ifdef DEBUG_NCQ
if (qc->tag > 0 || ap->link.sactive > 1)
@@ -1068,47 +1091,33 @@
if (!ata_is_ncq(qc->tf.protocol))
tag = 0;
- sata_dwc_qc_prep_by_tag(qc, tag);
+
+ if (ata_is_dma(qc->tf.protocol)) {
+ hsdevp->desc[tag] = dma_dwc_xfer_setup(qc);
+ if (!hsdevp->desc[tag])
+ return AC_ERR_SYSTEM;
+ } else {
+ hsdevp->desc[tag] = NULL;
+ }
if (ata_is_ncq(qc->tf.protocol)) {
- sactive = core_scr_read(SCR_ACTIVE);
+ sata_dwc_scr_read(&ap->link, SCR_ACTIVE, &sactive);
sactive |= (0x00000001 << tag);
- core_scr_write(SCR_ACTIVE, sactive);
+ sata_dwc_scr_write(&ap->link, SCR_ACTIVE, sactive);
dev_dbg(qc->ap->dev,
"%s: tag=%d ap->link.sactive = 0x%08x sactive=0x%08x\n",
__func__, tag, qc->ap->link.sactive, sactive);
ap->ops->sff_tf_load(ap, &qc->tf);
- sata_dwc_exec_command_by_tag(ap, &qc->tf, qc->tag,
+ sata_dwc_exec_command_by_tag(ap, &qc->tf, tag,
SATA_DWC_CMD_ISSUED_PEND);
} else {
- ata_sff_qc_issue(qc);
+ return ata_bmdma_qc_issue(qc);
}
return 0;
}
-/*
- * Function : sata_dwc_qc_prep
- * arguments : ata_queued_cmd *qc
- * Return value : None
- * qc_prep for a particular queued command
- */
-
-static void sata_dwc_qc_prep(struct ata_queued_cmd *qc)
-{
- if ((qc->dma_dir == DMA_NONE) || (qc->tf.protocol == ATA_PROT_PIO))
- return;
-
-#ifdef DEBUG_NCQ
- if (qc->tag > 0)
- dev_info(qc->ap->dev, "%s: qc->tag=%d ap->active_tag=0x%08x\n",
- __func__, qc->tag, qc->ap->link.active_tag);
-
- return ;
-#endif
-}
-
static void sata_dwc_error_handler(struct ata_port *ap)
{
ata_sff_error_handler(ap);
@@ -1125,17 +1134,22 @@
sata_dwc_enable_interrupts(hsdev);
/* Reconfigure the DMA control register */
- out_le32(&hsdev->sata_dwc_regs->dmacr,
- SATA_DWC_DMACR_TXRXCH_CLEAR);
+ sata_dwc_writel(&hsdev->sata_dwc_regs->dmacr,
+ SATA_DWC_DMACR_TXRXCH_CLEAR);
/* Reconfigure the DMA Burst Transaction Size register */
- out_le32(&hsdev->sata_dwc_regs->dbtsr,
- SATA_DWC_DBTSR_MWR(AHB_DMA_BRST_DFLT) |
- SATA_DWC_DBTSR_MRD(AHB_DMA_BRST_DFLT));
+ sata_dwc_writel(&hsdev->sata_dwc_regs->dbtsr,
+ SATA_DWC_DBTSR_MWR(AHB_DMA_BRST_DFLT) |
+ SATA_DWC_DBTSR_MRD(AHB_DMA_BRST_DFLT));
return ret;
}
+static void sata_dwc_dev_select(struct ata_port *ap, unsigned int device)
+{
+ /* SATA DWC is master only */
+}
+
/*
* scsi mid-layer and libata interface structures
*/
@@ -1148,7 +1162,13 @@
*/
.sg_tablesize = LIBATA_MAX_PRD,
/* .can_queue = ATA_MAX_QUEUE, */
- .dma_boundary = ATA_DMA_BOUNDARY,
+ /*
+ * Make sure a LLI block is not created that will span 8K max FIS
+ * boundary. If the block spans such a FIS boundary, there is a chance
+ * that a DMA burst will cross that boundary -- this results in an
+ * error in the host controller.
+ */
+ .dma_boundary = 0x1fff /* ATA_DMA_BOUNDARY */,
};
static struct ata_port_operations sata_dwc_ops = {
@@ -1157,7 +1177,6 @@
.error_handler = sata_dwc_error_handler,
.hardreset = sata_dwc_hardreset,
- .qc_prep = sata_dwc_qc_prep,
.qc_issue = sata_dwc_qc_issue,
.scr_read = sata_dwc_scr_read,
@@ -1166,6 +1185,8 @@
.port_start = sata_dwc_port_start,
.port_stop = sata_dwc_port_stop,
+ .sff_dev_select = sata_dwc_dev_select,
+
.bmdma_setup = sata_dwc_bmdma_setup,
.bmdma_start = sata_dwc_bmdma_start,
};
@@ -1184,13 +1205,14 @@
struct sata_dwc_device *hsdev;
u32 idr, versionr;
char *ver = (char *)&versionr;
- u8 __iomem *base;
+ void __iomem *base;
int err = 0;
int irq;
struct ata_host *host;
struct ata_port_info pi = sata_dwc_port_info[0];
const struct ata_port_info *ppi[] = { &pi, NULL };
struct device_node *np = ofdev->dev.of_node;
+ struct resource *res;
/* Allocate DWC SATA device */
host = ata_host_alloc_pinfo(&ofdev->dev, ppi, SATA_DWC_MAX_PORTS);
@@ -1201,57 +1223,33 @@
host->private_data = hsdev;
/* Ioremap SATA registers */
- base = of_iomap(np, 0);
- if (!base) {
+ res = platform_get_resource(ofdev, IORESOURCE_MEM, 0);
+ base = devm_ioremap_resource(&ofdev->dev, res);
+ if (IS_ERR(base)) {
dev_err(&ofdev->dev,
"ioremap failed for SATA register address\n");
- return -ENODEV;
+ return PTR_ERR(base);
}
- hsdev->reg_base = base;
dev_dbg(&ofdev->dev, "ioremap done for SATA register address\n");
/* Synopsys DWC SATA specific Registers */
- hsdev->sata_dwc_regs = (void *__iomem)(base + SATA_DWC_REG_OFFSET);
+ hsdev->sata_dwc_regs = base + SATA_DWC_REG_OFFSET;
+ hsdev->dmadr = res->start + SATA_DWC_REG_OFFSET + offsetof(struct sata_dwc_regs, dmadr);
/* Setup port */
host->ports[0]->ioaddr.cmd_addr = base;
host->ports[0]->ioaddr.scr_addr = base + SATA_DWC_SCR_OFFSET;
- host_pvt.scr_addr_sstatus = base + SATA_DWC_SCR_OFFSET;
- sata_dwc_setup_port(&host->ports[0]->ioaddr, (unsigned long)base);
+ sata_dwc_setup_port(&host->ports[0]->ioaddr, base);
/* Read the ID and Version Registers */
- idr = in_le32(&hsdev->sata_dwc_regs->idr);
- versionr = in_le32(&hsdev->sata_dwc_regs->versionr);
+ idr = sata_dwc_readl(&hsdev->sata_dwc_regs->idr);
+ versionr = sata_dwc_readl(&hsdev->sata_dwc_regs->versionr);
dev_notice(&ofdev->dev, "id %d, controller version %c.%c%c\n",
idr, ver[0], ver[1], ver[2]);
- /* Get SATA DMA interrupt number */
- hsdev->dma->irq = irq_of_parse_and_map(np, 1);
- if (hsdev->dma->irq == NO_IRQ) {
- dev_err(&ofdev->dev, "no SATA DMA irq\n");
- err = -ENODEV;
- goto error_iomap;
- }
-
- /* Get physical SATA DMA register base address */
- hsdev->dma->regs = of_iomap(np, 1);
- if (!hsdev->dma->regs) {
- dev_err(&ofdev->dev,
- "ioremap failed for AHBDMA register address\n");
- err = -ENODEV;
- goto error_iomap;
- }
-
/* Save dev for later use in dev_xxx() routines */
hsdev->dev = &ofdev->dev;
- hsdev->dma->dev = &ofdev->dev;
-
- /* Initialize AHB DMAC */
- err = dw_dma_probe(hsdev->dma, NULL);
- if (err)
- goto error_dma_iomap;
-
/* Enable SATA Interrupts */
sata_dwc_enable_interrupts(hsdev);
@@ -1263,6 +1261,25 @@
goto error_out;
}
+#ifdef CONFIG_SATA_DWC_OLD_DMA
+ if (!of_find_property(np, "dmas", NULL)) {
+ err = sata_dwc_dma_init_old(ofdev, hsdev);
+ if (err)
+ goto error_out;
+ }
+#endif
+
+ hsdev->phy = devm_phy_optional_get(hsdev->dev, "sata-phy");
+ if (IS_ERR(hsdev->phy)) {
+ err = PTR_ERR(hsdev->phy);
+ hsdev->phy = NULL;
+ goto error_out;
+ }
+
+ err = phy_init(hsdev->phy);
+ if (err)
+ goto error_out;
+
/*
* Now, register with libATA core, this will also initiate the
* device discovery process, invoking our port_start() handler &
@@ -1276,12 +1293,7 @@
return 0;
error_out:
- /* Free SATA DMA resources */
- dw_dma_remove(hsdev->dma);
-error_dma_iomap:
- iounmap(hsdev->dma->regs);
-error_iomap:
- iounmap(base);
+ phy_exit(hsdev->phy);
return err;
}
@@ -1293,11 +1305,13 @@
ata_host_detach(host);
- /* Free SATA DMA resources */
- dw_dma_remove(hsdev->dma);
+ phy_exit(hsdev->phy);
- iounmap(hsdev->dma->regs);
- iounmap(hsdev->reg_base);
+#ifdef CONFIG_SATA_DWC_OLD_DMA
+ /* Free SATA DMA resources */
+ sata_dwc_dma_exit_old(hsdev);
+#endif
+
dev_dbg(&ofdev->dev, "done\n");
return 0;
}
diff --git a/drivers/base/devcoredump.c b/drivers/base/devcoredump.c
index 1bd120a..240374f 100644
--- a/drivers/base/devcoredump.c
+++ b/drivers/base/devcoredump.c
@@ -4,6 +4,7 @@
* GPL LICENSE SUMMARY
*
* Copyright(c) 2014 Intel Mobile Communications GmbH
+ * Copyright(c) 2015 Intel Deutschland GmbH
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
@@ -41,12 +42,12 @@
struct devcd_entry {
struct device devcd_dev;
- const void *data;
+ void *data;
size_t datalen;
struct module *owner;
ssize_t (*read)(char *buffer, loff_t offset, size_t count,
- const void *data, size_t datalen);
- void (*free)(const void *data);
+ void *data, size_t datalen);
+ void (*free)(void *data);
struct delayed_work del_wk;
struct device *failing_dev;
};
@@ -174,7 +175,7 @@
};
static ssize_t devcd_readv(char *buffer, loff_t offset, size_t count,
- const void *data, size_t datalen)
+ void *data, size_t datalen)
{
if (offset > datalen)
return -EINVAL;
@@ -188,6 +189,11 @@
return count;
}
+static void devcd_freev(void *data)
+{
+ vfree(data);
+}
+
/**
* dev_coredumpv - create device coredump with vmalloc data
* @dev: the struct device for the crashed device
@@ -198,10 +204,10 @@
* This function takes ownership of the vmalloc'ed data and will free
* it when it is no longer used. See dev_coredumpm() for more information.
*/
-void dev_coredumpv(struct device *dev, const void *data, size_t datalen,
+void dev_coredumpv(struct device *dev, void *data, size_t datalen,
gfp_t gfp)
{
- dev_coredumpm(dev, NULL, data, datalen, gfp, devcd_readv, vfree);
+ dev_coredumpm(dev, NULL, data, datalen, gfp, devcd_readv, devcd_freev);
}
EXPORT_SYMBOL_GPL(dev_coredumpv);
@@ -213,6 +219,44 @@
}
/**
+ * devcd_free_sgtable - free all the memory of the given scatterlist table
+ * (i.e. both pages and scatterlist instances)
+ * NOTE: if two tables allocated with devcd_alloc_sgtable and then chained
+ * using the sg_chain function then that function should be called only once
+ * on the chained table
+ * @table: pointer to sg_table to free
+ */
+static void devcd_free_sgtable(void *data)
+{
+ _devcd_free_sgtable(data);
+}
+
+/**
+ * devcd_read_from_table - copy data from sg_table to a given buffer
+ * and return the number of bytes read
+ * @buffer: the buffer to copy the data to it
+ * @buf_len: the length of the buffer
+ * @data: the scatterlist table to copy from
+ * @offset: start copy from @offset@ bytes from the head of the data
+ * in the given scatterlist
+ * @data_len: the length of the data in the sg_table
+ */
+static ssize_t devcd_read_from_sgtable(char *buffer, loff_t offset,
+ size_t buf_len, void *data,
+ size_t data_len)
+{
+ struct scatterlist *table = data;
+
+ if (offset > data_len)
+ return -EINVAL;
+
+ if (offset + buf_len > data_len)
+ buf_len = data_len - offset;
+ return sg_pcopy_to_buffer(table, sg_nents(table), buffer, buf_len,
+ offset);
+}
+
+/**
* dev_coredumpm - create device coredump with read/free methods
* @dev: the struct device for the crashed device
* @owner: the module that contains the read/free functions, use %THIS_MODULE
@@ -228,10 +272,10 @@
* function will be called to free the data.
*/
void dev_coredumpm(struct device *dev, struct module *owner,
- const void *data, size_t datalen, gfp_t gfp,
+ void *data, size_t datalen, gfp_t gfp,
ssize_t (*read)(char *buffer, loff_t offset, size_t count,
- const void *data, size_t datalen),
- void (*free)(const void *data))
+ void *data, size_t datalen),
+ void (*free)(void *data))
{
static atomic_t devcd_count = ATOMIC_INIT(0);
struct devcd_entry *devcd;
@@ -291,6 +335,27 @@
}
EXPORT_SYMBOL_GPL(dev_coredumpm);
+/**
+ * dev_coredumpmsg - create device coredump that uses scatterlist as data
+ * parameter
+ * @dev: the struct device for the crashed device
+ * @table: the dump data
+ * @datalen: length of the data
+ * @gfp: allocation flags
+ *
+ * Creates a new device coredump for the given device. If a previous one hasn't
+ * been read yet, the new coredump is discarded. The data lifetime is determined
+ * by the device coredump framework and when it is no longer needed
+ * it will free the data.
+ */
+void dev_coredumpsg(struct device *dev, struct scatterlist *table,
+ size_t datalen, gfp_t gfp)
+{
+ dev_coredumpm(dev, NULL, table, datalen, gfp, devcd_read_from_sgtable,
+ devcd_free_sgtable);
+}
+EXPORT_SYMBOL_GPL(dev_coredumpsg);
+
static int __init devcoredump_init(void)
{
return class_register(&devcd_class);
diff --git a/drivers/bcma/driver_chipcommon_sflash.c b/drivers/bcma/driver_chipcommon_sflash.c
index 04d706c..35b13a0 100644
--- a/drivers/bcma/driver_chipcommon_sflash.c
+++ b/drivers/bcma/driver_chipcommon_sflash.c
@@ -146,7 +146,6 @@
return -ENOTSUPP;
}
- sflash->window = BCMA_SOC_FLASH2;
sflash->blocksize = e->blocksize;
sflash->numblocks = e->numblocks;
sflash->size = sflash->blocksize * sflash->numblocks;
diff --git a/drivers/block/aoe/aoecmd.c b/drivers/block/aoe/aoecmd.c
index 437b3a8..d597e43 100644
--- a/drivers/block/aoe/aoecmd.c
+++ b/drivers/block/aoe/aoecmd.c
@@ -861,7 +861,7 @@
* discussion.
*
* We cannot use get_page in the workaround, because it insists on a
- * positive page count as a precondition. So we use _count directly.
+ * positive page count as a precondition. So we use _refcount directly.
*/
static void
bio_pageinc(struct bio *bio)
diff --git a/drivers/block/zram/zcomp.c b/drivers/block/zram/zcomp.c
index 3ef42e5..b51a816 100644
--- a/drivers/block/zram/zcomp.c
+++ b/drivers/block/zram/zcomp.c
@@ -13,6 +13,7 @@
#include <linux/slab.h>
#include <linux/wait.h>
#include <linux/sched.h>
+#include <linux/cpu.h>
#include "zcomp.h"
#include "zcomp_lzo.h"
@@ -20,29 +21,6 @@
#include "zcomp_lz4.h"
#endif
-/*
- * single zcomp_strm backend
- */
-struct zcomp_strm_single {
- struct mutex strm_lock;
- struct zcomp_strm *zstrm;
-};
-
-/*
- * multi zcomp_strm backend
- */
-struct zcomp_strm_multi {
- /* protect strm list */
- spinlock_t strm_lock;
- /* max possible number of zstrm streams */
- int max_strm;
- /* number of available zstrm streams */
- int avail_strm;
- /* list of available strms */
- struct list_head idle_strm;
- wait_queue_head_t strm_wait;
-};
-
static struct zcomp_backend *backends[] = {
&zcomp_lzo,
#ifdef CONFIG_ZRAM_LZ4_COMPRESS
@@ -93,188 +71,6 @@
return zstrm;
}
-/*
- * get idle zcomp_strm or wait until other process release
- * (zcomp_strm_release()) one for us
- */
-static struct zcomp_strm *zcomp_strm_multi_find(struct zcomp *comp)
-{
- struct zcomp_strm_multi *zs = comp->stream;
- struct zcomp_strm *zstrm;
-
- while (1) {
- spin_lock(&zs->strm_lock);
- if (!list_empty(&zs->idle_strm)) {
- zstrm = list_entry(zs->idle_strm.next,
- struct zcomp_strm, list);
- list_del(&zstrm->list);
- spin_unlock(&zs->strm_lock);
- return zstrm;
- }
- /* zstrm streams limit reached, wait for idle stream */
- if (zs->avail_strm >= zs->max_strm) {
- spin_unlock(&zs->strm_lock);
- wait_event(zs->strm_wait, !list_empty(&zs->idle_strm));
- continue;
- }
- /* allocate new zstrm stream */
- zs->avail_strm++;
- spin_unlock(&zs->strm_lock);
- /*
- * This function can be called in swapout/fs write path
- * so we can't use GFP_FS|IO. And it assumes we already
- * have at least one stream in zram initialization so we
- * don't do best effort to allocate more stream in here.
- * A default stream will work well without further multiple
- * streams. That's why we use NORETRY | NOWARN.
- */
- zstrm = zcomp_strm_alloc(comp, GFP_NOIO | __GFP_NORETRY |
- __GFP_NOWARN);
- if (!zstrm) {
- spin_lock(&zs->strm_lock);
- zs->avail_strm--;
- spin_unlock(&zs->strm_lock);
- wait_event(zs->strm_wait, !list_empty(&zs->idle_strm));
- continue;
- }
- break;
- }
- return zstrm;
-}
-
-/* add stream back to idle list and wake up waiter or free the stream */
-static void zcomp_strm_multi_release(struct zcomp *comp, struct zcomp_strm *zstrm)
-{
- struct zcomp_strm_multi *zs = comp->stream;
-
- spin_lock(&zs->strm_lock);
- if (zs->avail_strm <= zs->max_strm) {
- list_add(&zstrm->list, &zs->idle_strm);
- spin_unlock(&zs->strm_lock);
- wake_up(&zs->strm_wait);
- return;
- }
-
- zs->avail_strm--;
- spin_unlock(&zs->strm_lock);
- zcomp_strm_free(comp, zstrm);
-}
-
-/* change max_strm limit */
-static bool zcomp_strm_multi_set_max_streams(struct zcomp *comp, int num_strm)
-{
- struct zcomp_strm_multi *zs = comp->stream;
- struct zcomp_strm *zstrm;
-
- spin_lock(&zs->strm_lock);
- zs->max_strm = num_strm;
- /*
- * if user has lowered the limit and there are idle streams,
- * immediately free as much streams (and memory) as we can.
- */
- while (zs->avail_strm > num_strm && !list_empty(&zs->idle_strm)) {
- zstrm = list_entry(zs->idle_strm.next,
- struct zcomp_strm, list);
- list_del(&zstrm->list);
- zcomp_strm_free(comp, zstrm);
- zs->avail_strm--;
- }
- spin_unlock(&zs->strm_lock);
- return true;
-}
-
-static void zcomp_strm_multi_destroy(struct zcomp *comp)
-{
- struct zcomp_strm_multi *zs = comp->stream;
- struct zcomp_strm *zstrm;
-
- while (!list_empty(&zs->idle_strm)) {
- zstrm = list_entry(zs->idle_strm.next,
- struct zcomp_strm, list);
- list_del(&zstrm->list);
- zcomp_strm_free(comp, zstrm);
- }
- kfree(zs);
-}
-
-static int zcomp_strm_multi_create(struct zcomp *comp, int max_strm)
-{
- struct zcomp_strm *zstrm;
- struct zcomp_strm_multi *zs;
-
- comp->destroy = zcomp_strm_multi_destroy;
- comp->strm_find = zcomp_strm_multi_find;
- comp->strm_release = zcomp_strm_multi_release;
- comp->set_max_streams = zcomp_strm_multi_set_max_streams;
- zs = kmalloc(sizeof(struct zcomp_strm_multi), GFP_KERNEL);
- if (!zs)
- return -ENOMEM;
-
- comp->stream = zs;
- spin_lock_init(&zs->strm_lock);
- INIT_LIST_HEAD(&zs->idle_strm);
- init_waitqueue_head(&zs->strm_wait);
- zs->max_strm = max_strm;
- zs->avail_strm = 1;
-
- zstrm = zcomp_strm_alloc(comp, GFP_KERNEL);
- if (!zstrm) {
- kfree(zs);
- return -ENOMEM;
- }
- list_add(&zstrm->list, &zs->idle_strm);
- return 0;
-}
-
-static struct zcomp_strm *zcomp_strm_single_find(struct zcomp *comp)
-{
- struct zcomp_strm_single *zs = comp->stream;
- mutex_lock(&zs->strm_lock);
- return zs->zstrm;
-}
-
-static void zcomp_strm_single_release(struct zcomp *comp,
- struct zcomp_strm *zstrm)
-{
- struct zcomp_strm_single *zs = comp->stream;
- mutex_unlock(&zs->strm_lock);
-}
-
-static bool zcomp_strm_single_set_max_streams(struct zcomp *comp, int num_strm)
-{
- /* zcomp_strm_single support only max_comp_streams == 1 */
- return false;
-}
-
-static void zcomp_strm_single_destroy(struct zcomp *comp)
-{
- struct zcomp_strm_single *zs = comp->stream;
- zcomp_strm_free(comp, zs->zstrm);
- kfree(zs);
-}
-
-static int zcomp_strm_single_create(struct zcomp *comp)
-{
- struct zcomp_strm_single *zs;
-
- comp->destroy = zcomp_strm_single_destroy;
- comp->strm_find = zcomp_strm_single_find;
- comp->strm_release = zcomp_strm_single_release;
- comp->set_max_streams = zcomp_strm_single_set_max_streams;
- zs = kmalloc(sizeof(struct zcomp_strm_single), GFP_KERNEL);
- if (!zs)
- return -ENOMEM;
-
- comp->stream = zs;
- mutex_init(&zs->strm_lock);
- zs->zstrm = zcomp_strm_alloc(comp, GFP_KERNEL);
- if (!zs->zstrm) {
- kfree(zs);
- return -ENOMEM;
- }
- return 0;
-}
-
/* show available compressors */
ssize_t zcomp_available_show(const char *comp, char *buf)
{
@@ -299,19 +95,14 @@
return find_backend(comp) != NULL;
}
-bool zcomp_set_max_streams(struct zcomp *comp, int num_strm)
-{
- return comp->set_max_streams(comp, num_strm);
-}
-
struct zcomp_strm *zcomp_strm_find(struct zcomp *comp)
{
- return comp->strm_find(comp);
+ return *get_cpu_ptr(comp->stream);
}
void zcomp_strm_release(struct zcomp *comp, struct zcomp_strm *zstrm)
{
- comp->strm_release(comp, zstrm);
+ put_cpu_ptr(comp->stream);
}
int zcomp_compress(struct zcomp *comp, struct zcomp_strm *zstrm,
@@ -327,9 +118,83 @@
return comp->backend->decompress(src, src_len, dst);
}
+static int __zcomp_cpu_notifier(struct zcomp *comp,
+ unsigned long action, unsigned long cpu)
+{
+ struct zcomp_strm *zstrm;
+
+ switch (action) {
+ case CPU_UP_PREPARE:
+ if (WARN_ON(*per_cpu_ptr(comp->stream, cpu)))
+ break;
+ zstrm = zcomp_strm_alloc(comp, GFP_KERNEL);
+ if (IS_ERR_OR_NULL(zstrm)) {
+ pr_err("Can't allocate a compression stream\n");
+ return NOTIFY_BAD;
+ }
+ *per_cpu_ptr(comp->stream, cpu) = zstrm;
+ break;
+ case CPU_DEAD:
+ case CPU_UP_CANCELED:
+ zstrm = *per_cpu_ptr(comp->stream, cpu);
+ if (!IS_ERR_OR_NULL(zstrm))
+ zcomp_strm_free(comp, zstrm);
+ *per_cpu_ptr(comp->stream, cpu) = NULL;
+ break;
+ default:
+ break;
+ }
+ return NOTIFY_OK;
+}
+
+static int zcomp_cpu_notifier(struct notifier_block *nb,
+ unsigned long action, void *pcpu)
+{
+ unsigned long cpu = (unsigned long)pcpu;
+ struct zcomp *comp = container_of(nb, typeof(*comp), notifier);
+
+ return __zcomp_cpu_notifier(comp, action, cpu);
+}
+
+static int zcomp_init(struct zcomp *comp)
+{
+ unsigned long cpu;
+ int ret;
+
+ comp->notifier.notifier_call = zcomp_cpu_notifier;
+
+ comp->stream = alloc_percpu(struct zcomp_strm *);
+ if (!comp->stream)
+ return -ENOMEM;
+
+ cpu_notifier_register_begin();
+ for_each_online_cpu(cpu) {
+ ret = __zcomp_cpu_notifier(comp, CPU_UP_PREPARE, cpu);
+ if (ret == NOTIFY_BAD)
+ goto cleanup;
+ }
+ __register_cpu_notifier(&comp->notifier);
+ cpu_notifier_register_done();
+ return 0;
+
+cleanup:
+ for_each_online_cpu(cpu)
+ __zcomp_cpu_notifier(comp, CPU_UP_CANCELED, cpu);
+ cpu_notifier_register_done();
+ return -ENOMEM;
+}
+
void zcomp_destroy(struct zcomp *comp)
{
- comp->destroy(comp);
+ unsigned long cpu;
+
+ cpu_notifier_register_begin();
+ for_each_online_cpu(cpu)
+ __zcomp_cpu_notifier(comp, CPU_UP_CANCELED, cpu);
+ __unregister_cpu_notifier(&comp->notifier);
+ cpu_notifier_register_done();
+
+ free_percpu(comp->stream);
kfree(comp);
}
@@ -339,9 +204,9 @@
* backend pointer or ERR_PTR if things went bad. ERR_PTR(-EINVAL)
* if requested algorithm is not supported, ERR_PTR(-ENOMEM) in
* case of allocation error, or any other error potentially
- * returned by functions zcomp_strm_{multi,single}_create.
+ * returned by zcomp_init().
*/
-struct zcomp *zcomp_create(const char *compress, int max_strm)
+struct zcomp *zcomp_create(const char *compress)
{
struct zcomp *comp;
struct zcomp_backend *backend;
@@ -356,10 +221,7 @@
return ERR_PTR(-ENOMEM);
comp->backend = backend;
- if (max_strm > 1)
- error = zcomp_strm_multi_create(comp, max_strm);
- else
- error = zcomp_strm_single_create(comp);
+ error = zcomp_init(comp);
if (error) {
kfree(comp);
return ERR_PTR(error);
diff --git a/drivers/block/zram/zcomp.h b/drivers/block/zram/zcomp.h
index b7d2a4b..ffd88cb 100644
--- a/drivers/block/zram/zcomp.h
+++ b/drivers/block/zram/zcomp.h
@@ -10,8 +10,6 @@
#ifndef _ZCOMP_H_
#define _ZCOMP_H_
-#include <linux/mutex.h>
-
struct zcomp_strm {
/* compression/decompression buffer */
void *buffer;
@@ -21,8 +19,6 @@
* working memory)
*/
void *private;
- /* used in multi stream backend, protected by backend strm_lock */
- struct list_head list;
};
/* static compression backend */
@@ -41,19 +37,15 @@
/* dynamic per-device compression frontend */
struct zcomp {
- void *stream;
+ struct zcomp_strm * __percpu *stream;
struct zcomp_backend *backend;
-
- struct zcomp_strm *(*strm_find)(struct zcomp *comp);
- void (*strm_release)(struct zcomp *comp, struct zcomp_strm *zstrm);
- bool (*set_max_streams)(struct zcomp *comp, int num_strm);
- void (*destroy)(struct zcomp *comp);
+ struct notifier_block notifier;
};
ssize_t zcomp_available_show(const char *comp, char *buf);
bool zcomp_available_algorithm(const char *comp);
-struct zcomp *zcomp_create(const char *comp, int max_strm);
+struct zcomp *zcomp_create(const char *comp);
void zcomp_destroy(struct zcomp *comp);
struct zcomp_strm *zcomp_strm_find(struct zcomp *comp);
diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index 370c2f7..8fcad8b 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -304,46 +304,25 @@
return len;
}
+/*
+ * We switched to per-cpu streams and this attr is not needed anymore.
+ * However, we will keep it around for some time, because:
+ * a) we may revert per-cpu streams in the future
+ * b) it's visible to user space and we need to follow our 2 years
+ * retirement rule; but we already have a number of 'soon to be
+ * altered' attrs, so max_comp_streams need to wait for the next
+ * layoff cycle.
+ */
static ssize_t max_comp_streams_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
- int val;
- struct zram *zram = dev_to_zram(dev);
-
- down_read(&zram->init_lock);
- val = zram->max_comp_streams;
- up_read(&zram->init_lock);
-
- return scnprintf(buf, PAGE_SIZE, "%d\n", val);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", num_online_cpus());
}
static ssize_t max_comp_streams_store(struct device *dev,
struct device_attribute *attr, const char *buf, size_t len)
{
- int num;
- struct zram *zram = dev_to_zram(dev);
- int ret;
-
- ret = kstrtoint(buf, 0, &num);
- if (ret < 0)
- return ret;
- if (num < 1)
- return -EINVAL;
-
- down_write(&zram->init_lock);
- if (init_done(zram)) {
- if (!zcomp_set_max_streams(zram->comp, num)) {
- pr_info("Cannot change max compression streams\n");
- ret = -EINVAL;
- goto out;
- }
- }
-
- zram->max_comp_streams = num;
- ret = len;
-out:
- up_write(&zram->init_lock);
- return ret;
+ return len;
}
static ssize_t comp_algorithm_show(struct device *dev,
@@ -456,8 +435,26 @@
return ret;
}
+static ssize_t debug_stat_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ int version = 1;
+ struct zram *zram = dev_to_zram(dev);
+ ssize_t ret;
+
+ down_read(&zram->init_lock);
+ ret = scnprintf(buf, PAGE_SIZE,
+ "version: %d\n%8llu\n",
+ version,
+ (u64)atomic64_read(&zram->stats.writestall));
+ up_read(&zram->init_lock);
+
+ return ret;
+}
+
static DEVICE_ATTR_RO(io_stat);
static DEVICE_ATTR_RO(mm_stat);
+static DEVICE_ATTR_RO(debug_stat);
ZRAM_ATTR_RO(num_reads);
ZRAM_ATTR_RO(num_writes);
ZRAM_ATTR_RO(failed_reads);
@@ -514,7 +511,7 @@
goto out_error;
}
- meta->mem_pool = zs_create_pool(pool_name, GFP_NOIO | __GFP_HIGHMEM);
+ meta->mem_pool = zs_create_pool(pool_name);
if (!meta->mem_pool) {
pr_err("Error creating memory pool\n");
goto out_error;
@@ -650,7 +647,7 @@
{
int ret = 0;
size_t clen;
- unsigned long handle;
+ unsigned long handle = 0;
struct page *page;
unsigned char *user_mem, *cmem, *src, *uncmem = NULL;
struct zram_meta *meta = zram->meta;
@@ -673,9 +670,8 @@
goto out;
}
- zstrm = zcomp_strm_find(zram->comp);
+compress_again:
user_mem = kmap_atomic(page);
-
if (is_partial_io(bvec)) {
memcpy(uncmem + offset, user_mem + bvec->bv_offset,
bvec->bv_len);
@@ -699,6 +695,7 @@
goto out;
}
+ zstrm = zcomp_strm_find(zram->comp);
ret = zcomp_compress(zram->comp, zstrm, uncmem, &clen);
if (!is_partial_io(bvec)) {
kunmap_atomic(user_mem);
@@ -710,6 +707,7 @@
pr_err("Compression failed! err=%d\n", ret);
goto out;
}
+
src = zstrm->buffer;
if (unlikely(clen > max_zpage_size)) {
clen = PAGE_SIZE;
@@ -717,8 +715,35 @@
src = uncmem;
}
- handle = zs_malloc(meta->mem_pool, clen);
+ /*
+ * handle allocation has 2 paths:
+ * a) fast path is executed with preemption disabled (for
+ * per-cpu streams) and has __GFP_DIRECT_RECLAIM bit clear,
+ * since we can't sleep;
+ * b) slow path enables preemption and attempts to allocate
+ * the page with __GFP_DIRECT_RECLAIM bit set. we have to
+ * put per-cpu compression stream and, thus, to re-do
+ * the compression once handle is allocated.
+ *
+ * if we have a 'non-null' handle here then we are coming
+ * from the slow path and handle has already been allocated.
+ */
+ if (!handle)
+ handle = zs_malloc(meta->mem_pool, clen,
+ __GFP_KSWAPD_RECLAIM |
+ __GFP_NOWARN |
+ __GFP_HIGHMEM);
if (!handle) {
+ zcomp_strm_release(zram->comp, zstrm);
+ zstrm = NULL;
+
+ atomic64_inc(&zram->stats.writestall);
+
+ handle = zs_malloc(meta->mem_pool, clen,
+ GFP_NOIO | __GFP_HIGHMEM);
+ if (handle)
+ goto compress_again;
+
pr_err("Error allocating memory for compressed page: %u, size=%zu\n",
index, clen);
ret = -ENOMEM;
@@ -1009,7 +1034,6 @@
/* Reset stats */
memset(&zram->stats, 0, sizeof(zram->stats));
zram->disksize = 0;
- zram->max_comp_streams = 1;
set_capacity(zram->disk, 0);
part_stat_set_all(&zram->disk->part0, 0);
@@ -1038,7 +1062,7 @@
if (!meta)
return -ENOMEM;
- comp = zcomp_create(zram->compressor, zram->max_comp_streams);
+ comp = zcomp_create(zram->compressor);
if (IS_ERR(comp)) {
pr_err("Cannot initialise %s compressing backend\n",
zram->compressor);
@@ -1177,6 +1201,7 @@
&dev_attr_comp_algorithm.attr,
&dev_attr_io_stat.attr,
&dev_attr_mm_stat.attr,
+ &dev_attr_debug_stat.attr,
NULL,
};
@@ -1273,7 +1298,6 @@
}
strlcpy(zram->compressor, default_compressor, sizeof(zram->compressor));
zram->meta = NULL;
- zram->max_comp_streams = 1;
pr_info("Added device: %s\n", zram->disk->disk_name);
return device_id;
diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h
index 8e92339..3f5bf66 100644
--- a/drivers/block/zram/zram_drv.h
+++ b/drivers/block/zram/zram_drv.h
@@ -85,6 +85,7 @@
atomic64_t zero_pages; /* no. of zero filled pages */
atomic64_t pages_stored; /* no. of pages currently stored */
atomic_long_t max_used_pages; /* no. of maximum pages stored */
+ atomic64_t writestall; /* no. of write slow paths */
};
struct zram_meta {
@@ -102,7 +103,6 @@
* the number of pages zram can consume for storing compressed data
*/
unsigned long limit_pages;
- int max_comp_streams;
struct zram_stats stats;
atomic_t refcount; /* refcount for zram_meta */
diff --git a/drivers/bus/brcmstb_gisb.c b/drivers/bus/brcmstb_gisb.c
index f364fa4..72fe0a5 100644
--- a/drivers/bus/brcmstb_gisb.c
+++ b/drivers/bus/brcmstb_gisb.c
@@ -30,6 +30,10 @@
#include <asm/signal.h>
#endif
+#ifdef CONFIG_MIPS
+#include <asm/traps.h>
+#endif
+
#define ARB_ERR_CAP_CLEAR (1 << 0)
#define ARB_ERR_CAP_STATUS_TIMEOUT (1 << 12)
#define ARB_ERR_CAP_STATUS_TEA (1 << 11)
@@ -238,6 +242,29 @@
}
#endif
+#ifdef CONFIG_MIPS
+static int brcmstb_bus_error_handler(struct pt_regs *regs, int is_fixup)
+{
+ int ret = 0;
+ struct brcmstb_gisb_arb_device *gdev;
+ u32 cap_status;
+
+ list_for_each_entry(gdev, &brcmstb_gisb_arb_device_list, next) {
+ cap_status = gisb_read(gdev, ARB_ERR_CAP_STATUS);
+
+ /* Invalid captured address, bail out */
+ if (!(cap_status & ARB_ERR_CAP_STATUS_VALID)) {
+ is_fixup = 1;
+ goto out;
+ }
+
+ ret |= brcmstb_gisb_arb_decode_addr(gdev, "bus error");
+ }
+out:
+ return is_fixup ? MIPS_BE_FIXUP : MIPS_BE_FATAL;
+}
+#endif
+
static irqreturn_t brcmstb_gisb_timeout_handler(int irq, void *dev_id)
{
brcmstb_gisb_arb_decode_addr(dev_id, "timeout");
@@ -355,6 +382,9 @@
hook_fault_code(22, brcmstb_bus_error_handler, SIGBUS, 0,
"imprecise external abort");
#endif
+#ifdef CONFIG_MIPS
+ board_be_handler = brcmstb_bus_error_handler;
+#endif
dev_info(&pdev->dev, "registered mem: %p, irqs: %d, %d\n",
gdev->base, timeout_irq, tea_irq);
diff --git a/drivers/bus/mips_cdmm.c b/drivers/bus/mips_cdmm.c
index 1c543ef..cad49bc 100644
--- a/drivers/bus/mips_cdmm.c
+++ b/drivers/bus/mips_cdmm.c
@@ -599,8 +599,8 @@
* mips_cdmm_bus_down() - Tear down the CDMM bus.
* @data: Pointer to unsigned int CPU number.
*
- * This work_on_cpu callback function is executed on a given CPU to call the
- * CDMM driver cpu_down callback for all devices on that CPU.
+ * This function is executed on the hotplugged CPU and calls the CDMM
+ * driver cpu_down callback for all devices on that CPU.
*/
static long mips_cdmm_bus_down(void *data)
{
@@ -630,7 +630,9 @@
* CDMM devices on that CPU, or to call the CDMM driver cpu_up callback for all
* devices already discovered on that CPU.
*
- * It is used during initialisation and when CPUs are brought online.
+ * It is used as work_on_cpu callback function during
+ * initialisation. When CPUs are brought online the function is
+ * invoked directly on the hotplugged CPU.
*/
static long mips_cdmm_bus_up(void *data)
{
@@ -677,10 +679,10 @@
switch (action & ~CPU_TASKS_FROZEN) {
case CPU_ONLINE:
case CPU_DOWN_FAILED:
- work_on_cpu(cpu, mips_cdmm_bus_up, &cpu);
+ mips_cdmm_bus_up(&cpu);
break;
case CPU_DOWN_PREPARE:
- work_on_cpu(cpu, mips_cdmm_bus_down, &cpu);
+ mips_cdmm_bus_down(&cpu);
break;
default:
return NOTIFY_DONE;
diff --git a/drivers/char/Kconfig b/drivers/char/Kconfig
index 3ec0766..601f64f 100644
--- a/drivers/char/Kconfig
+++ b/drivers/char/Kconfig
@@ -279,8 +279,7 @@
config RTC
tristate "Enhanced Real Time Clock Support (legacy PC RTC driver)"
- depends on !PPC && !PARISC && !IA64 && !M68K && !SPARC && !FRV \
- && !ARM && !SUPERH && !S390 && !AVR32 && !BLACKFIN && !UML
+ depends on ALPHA || (MIPS && MACH_LOONGSON64) || MN10300
---help---
If you say Y here and create a character special file /dev/rtc with
major number 10 and minor number 135 using mknod ("man mknod"), you
@@ -585,7 +584,6 @@
config DEVPORT
bool
- depends on !M68K
depends on ISA || PCI
default y
diff --git a/drivers/char/pcmcia/synclink_cs.c b/drivers/char/pcmcia/synclink_cs.c
index e524e83..d28922d 100644
--- a/drivers/char/pcmcia/synclink_cs.c
+++ b/drivers/char/pcmcia/synclink_cs.c
@@ -1101,7 +1101,7 @@
wake_up_interruptible(&info->status_event_wait_q);
wake_up_interruptible(&info->event_wait_q);
- if (info->port.flags & ASYNC_CHECK_CD) {
+ if (tty_port_check_carrier(&info->port)) {
if (debug_level >= DEBUG_LEVEL_ISR)
printk("%s CD now %s...", info->device_name,
(info->serial_signals & SerialSignal_DCD) ? "on" : "off");
@@ -1272,7 +1272,7 @@
if (debug_level >= DEBUG_LEVEL_INFO)
printk("%s(%d):startup(%s)\n", __FILE__, __LINE__, info->device_name);
- if (info->port.flags & ASYNC_INITIALIZED)
+ if (tty_port_initialized(&info->port))
return 0;
if (!info->tx_buf) {
@@ -1311,7 +1311,7 @@
if (tty)
clear_bit(TTY_IO_ERROR, &tty->flags);
- info->port.flags |= ASYNC_INITIALIZED;
+ tty_port_set_initialized(&info->port, 1);
return 0;
}
@@ -1322,7 +1322,7 @@
{
unsigned long flags;
- if (!(info->port.flags & ASYNC_INITIALIZED))
+ if (!tty_port_initialized(&info->port))
return;
if (debug_level >= DEBUG_LEVEL_INFO)
@@ -1361,7 +1361,7 @@
if (tty)
set_bit(TTY_IO_ERROR, &tty->flags);
- info->port.flags &= ~ASYNC_INITIALIZED;
+ tty_port_set_initialized(&info->port, 0);
}
static void mgslpc_program_hw(MGSLPC_INFO *info, struct tty_struct *tty)
@@ -1466,15 +1466,8 @@
}
info->timeout += HZ/50; /* Add .02 seconds of slop */
- if (cflag & CRTSCTS)
- info->port.flags |= ASYNC_CTS_FLOW;
- else
- info->port.flags &= ~ASYNC_CTS_FLOW;
-
- if (cflag & CLOCAL)
- info->port.flags &= ~ASYNC_CHECK_CD;
- else
- info->port.flags |= ASYNC_CHECK_CD;
+ tty_port_set_cts_flow(&info->port, cflag & CRTSCTS);
+ tty_port_set_check_carrier(&info->port, ~cflag & CLOCAL);
/* process tty input control flags */
@@ -2246,7 +2239,7 @@
if ((cmd != TIOCGSERIAL) && (cmd != TIOCSSERIAL) &&
(cmd != TIOCMIWAIT)) {
- if (tty->flags & (1 << TTY_IO_ERROR))
+ if (tty_io_error(tty))
return -EIO;
}
@@ -2316,7 +2309,7 @@
/* Handle transition away from B0 status */
if (!(old_termios->c_cflag & CBAUD) && C_BAUD(tty)) {
info->serial_signals |= SerialSignal_DTR;
- if (!C_CRTSCTS(tty) || !test_bit(TTY_THROTTLED, &tty->flags))
+ if (!C_CRTSCTS(tty) || !tty_throttled(tty))
info->serial_signals |= SerialSignal_RTS;
spin_lock_irqsave(&info->lock, flags);
set_signals(info);
@@ -2345,7 +2338,7 @@
if (tty_port_close_start(port, tty, filp) == 0)
goto cleanup;
- if (port->flags & ASYNC_INITIALIZED)
+ if (tty_port_initialized(port))
mgslpc_wait_until_sent(tty, info->timeout);
mgslpc_flush_buffer(tty);
@@ -2378,7 +2371,7 @@
if (mgslpc_paranoia_check(info, tty->name, "mgslpc_wait_until_sent"))
return;
- if (!(info->port.flags & ASYNC_INITIALIZED))
+ if (!tty_port_initialized(&info->port))
goto exit;
orig_jiffies = jiffies;
diff --git a/drivers/char/random.c b/drivers/char/random.c
index b583e53..0158d3b 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -260,6 +260,7 @@
#include <linux/irq.h>
#include <linux/syscalls.h>
#include <linux/completion.h>
+#include <linux/uuid.h>
#include <asm/processor.h>
#include <asm/uaccess.h>
@@ -1621,26 +1622,6 @@
return urandom_read(NULL, buf, count, NULL);
}
-/***************************************************************
- * Random UUID interface
- *
- * Used here for a Boot ID, but can be useful for other kernel
- * drivers.
- ***************************************************************/
-
-/*
- * Generate random UUID
- */
-void generate_random_uuid(unsigned char uuid_out[16])
-{
- get_random_bytes(uuid_out, 16);
- /* Set UUID version to 4 --- truly random generation */
- uuid_out[6] = (uuid_out[6] & 0x0F) | 0x40;
- /* Set the UUID variant to DCE */
- uuid_out[8] = (uuid_out[8] & 0x3F) | 0x80;
-}
-EXPORT_SYMBOL(generate_random_uuid);
-
/********************************************************************
*
* Sysctl interface
diff --git a/drivers/char/xillybus/xillybus_of.c b/drivers/char/xillybus/xillybus_of.c
index 7818650..78a492f 100644
--- a/drivers/char/xillybus/xillybus_of.c
+++ b/drivers/char/xillybus/xillybus_of.c
@@ -81,7 +81,6 @@
{
dma_addr_t addr;
struct xilly_mapping *this;
- int rc;
this = kzalloc(sizeof(*this), GFP_KERNEL);
if (!this)
@@ -101,15 +100,7 @@
*ret_dma_handle = addr;
- rc = devm_add_action(ep->dev, xilly_of_unmap, this);
-
- if (rc) {
- dma_unmap_single(ep->dev, addr, size, direction);
- kfree(this);
- return rc;
- }
-
- return 0;
+ return devm_add_action_or_reset(ep->dev, xilly_of_unmap, this);
}
static struct xilly_endpoint_hardware of_hw = {
diff --git a/drivers/char/xillybus/xillybus_pcie.c b/drivers/char/xillybus/xillybus_pcie.c
index 9418300..dff2d15 100644
--- a/drivers/char/xillybus/xillybus_pcie.c
+++ b/drivers/char/xillybus/xillybus_pcie.c
@@ -98,7 +98,6 @@
int pci_direction;
dma_addr_t addr;
struct xilly_mapping *this;
- int rc;
this = kzalloc(sizeof(*this), GFP_KERNEL);
if (!this)
@@ -120,14 +119,7 @@
*ret_dma_handle = addr;
- rc = devm_add_action(ep->dev, xilly_pci_unmap, this);
- if (rc) {
- pci_unmap_single(ep->pdev, addr, size, pci_direction);
- kfree(this);
- return rc;
- }
-
- return 0;
+ return devm_add_action_or_reset(ep->dev, xilly_pci_unmap, this);
}
static struct xilly_endpoint_hardware pci_hw = {
diff --git a/drivers/clk/Kconfig b/drivers/clk/Kconfig
index c455549..53ddba2 100644
--- a/drivers/clk/Kconfig
+++ b/drivers/clk/Kconfig
@@ -197,6 +197,15 @@
---help---
Support for the Marvell PXA SoC.
+config COMMON_CLK_PIC32
+ def_bool COMMON_CLK && MACH_PIC32
+
+config COMMON_CLK_OXNAS
+ bool "Clock driver for the OXNAS SoC Family"
+ select MFD_SYSCON
+ ---help---
+ Support for the OXNAS SoC Family clocks.
+
source "drivers/clk/bcm/Kconfig"
source "drivers/clk/hisilicon/Kconfig"
source "drivers/clk/mvebu/Kconfig"
diff --git a/drivers/clk/Makefile b/drivers/clk/Makefile
index 46869d6..dcc5e69 100644
--- a/drivers/clk/Makefile
+++ b/drivers/clk/Makefile
@@ -33,6 +33,7 @@
obj-$(CONFIG_ARCH_MOXART) += clk-moxart.o
obj-$(CONFIG_ARCH_NOMADIK) += clk-nomadik.o
obj-$(CONFIG_ARCH_NSPIRE) += clk-nspire.o
+obj-$(CONFIG_COMMON_CLK_OXNAS) += clk-oxnas.o
obj-$(CONFIG_COMMON_CLK_PALMAS) += clk-palmas.o
obj-$(CONFIG_CLK_QORIQ) += clk-qoriq.o
obj-$(CONFIG_COMMON_CLK_RK808) += clk-rk808.o
@@ -51,6 +52,7 @@
obj-$(CONFIG_COMMON_CLK_XGENE) += clk-xgene.o
obj-$(CONFIG_COMMON_CLK_PWM) += clk-pwm.o
obj-$(CONFIG_COMMON_CLK_AT91) += at91/
+obj-$(CONFIG_ARCH_ARTPEC) += axis/
obj-y += bcm/
obj-$(CONFIG_ARCH_BERLIN) += berlin/
obj-$(CONFIG_ARCH_HISI) += hisilicon/
@@ -58,10 +60,11 @@
obj-$(CONFIG_MACH_INGENIC) += ingenic/
obj-$(CONFIG_COMMON_CLK_KEYSTONE) += keystone/
obj-$(CONFIG_ARCH_MEDIATEK) += mediatek/
+obj-$(CONFIG_MACH_PIC32) += microchip/
ifeq ($(CONFIG_COMMON_CLK), y)
obj-$(CONFIG_ARCH_MMP) += mmp/
endif
-obj-$(CONFIG_PLAT_ORION) += mvebu/
+obj-y += mvebu/
obj-$(CONFIG_ARCH_MESON) += meson/
obj-$(CONFIG_ARCH_MXS) += mxs/
obj-$(CONFIG_MACH_PISTACHIO) += pistachio/
@@ -84,3 +87,4 @@
obj-$(CONFIG_ARCH_ZX) += zte/
obj-$(CONFIG_ARCH_ZYNQ) += zynq/
obj-$(CONFIG_H8300) += h8300/
+obj-$(CONFIG_ARC_PLAT_AXS10X) += axs10x/
diff --git a/drivers/clk/at91/clk-h32mx.c b/drivers/clk/at91/clk-h32mx.c
index 819f584..8e20c8a 100644
--- a/drivers/clk/at91/clk-h32mx.c
+++ b/drivers/clk/at91/clk-h32mx.c
@@ -114,7 +114,7 @@
h32mxclk->regmap = regmap;
clk = clk_register(NULL, &h32mxclk->hw);
- if (!clk) {
+ if (IS_ERR(clk)) {
kfree(h32mxclk);
return;
}
diff --git a/drivers/clk/axis/Makefile b/drivers/clk/axis/Makefile
new file mode 100644
index 0000000..628c9d3
--- /dev/null
+++ b/drivers/clk/axis/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_MACH_ARTPEC6) += clk-artpec6.o
diff --git a/drivers/clk/axis/clk-artpec6.c b/drivers/clk/axis/clk-artpec6.c
new file mode 100644
index 0000000..ffc988b
--- /dev/null
+++ b/drivers/clk/axis/clk-artpec6.c
@@ -0,0 +1,242 @@
+/*
+ * ARTPEC-6 clock initialization
+ *
+ * Copyright 2015-2016 Axis Comunications AB.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/clk-provider.h>
+#include <linux/device.h>
+#include <linux/io.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+#include <dt-bindings/clock/axis,artpec6-clkctrl.h>
+
+#define NUM_I2S_CLOCKS 2
+
+struct artpec6_clkctrl_drvdata {
+ struct clk *clk_table[ARTPEC6_CLK_NUMCLOCKS];
+ void __iomem *syscon_base;
+ struct clk_onecell_data clk_data;
+ spinlock_t i2scfg_lock;
+};
+
+static struct artpec6_clkctrl_drvdata *clkdata;
+
+static const char *const i2s_clk_names[NUM_I2S_CLOCKS] = {
+ "i2s0",
+ "i2s1",
+};
+
+static const int i2s_clk_indexes[NUM_I2S_CLOCKS] = {
+ ARTPEC6_CLK_I2S0_CLK,
+ ARTPEC6_CLK_I2S1_CLK,
+};
+
+static void of_artpec6_clkctrl_setup(struct device_node *np)
+{
+ int i;
+ const char *sys_refclk_name;
+ u32 pll_mode, pll_m, pll_n;
+ struct clk **clks;
+
+ /* Mandatory parent clock. */
+ i = of_property_match_string(np, "clock-names", "sys_refclk");
+ if (i < 0)
+ return;
+
+ sys_refclk_name = of_clk_get_parent_name(np, i);
+
+ clkdata = kzalloc(sizeof(*clkdata), GFP_KERNEL);
+ if (!clkdata)
+ return;
+
+ clks = clkdata->clk_table;
+
+ for (i = 0; i < ARTPEC6_CLK_NUMCLOCKS; ++i)
+ clks[i] = ERR_PTR(-EPROBE_DEFER);
+
+ clkdata->syscon_base = of_iomap(np, 0);
+ BUG_ON(clkdata->syscon_base == NULL);
+
+ /* Read PLL1 factors configured by boot strap pins. */
+ pll_mode = (readl(clkdata->syscon_base) >> 6) & 3;
+ switch (pll_mode) {
+ case 0: /* DDR3-2133 mode */
+ pll_m = 4;
+ pll_n = 85;
+ break;
+ case 1: /* DDR3-1866 mode */
+ pll_m = 6;
+ pll_n = 112;
+ break;
+ case 2: /* DDR3-1600 mode */
+ pll_m = 4;
+ pll_n = 64;
+ break;
+ case 3: /* DDR3-1333 mode */
+ pll_m = 8;
+ pll_n = 106;
+ break;
+ }
+
+ clks[ARTPEC6_CLK_CPU] =
+ clk_register_fixed_factor(NULL, "cpu", sys_refclk_name, 0, pll_n,
+ pll_m);
+ clks[ARTPEC6_CLK_CPU_PERIPH] =
+ clk_register_fixed_factor(NULL, "cpu_periph", "cpu", 0, 1, 2);
+
+ /* EPROBE_DEFER on the apb_clock is not handled in amba devices. */
+ clks[ARTPEC6_CLK_UART_PCLK] =
+ clk_register_fixed_factor(NULL, "uart_pclk", "cpu", 0, 1, 8);
+ clks[ARTPEC6_CLK_UART_REFCLK] =
+ clk_register_fixed_rate(NULL, "uart_ref", sys_refclk_name, 0,
+ 50000000);
+
+ clks[ARTPEC6_CLK_SPI_PCLK] =
+ clk_register_fixed_factor(NULL, "spi_pclk", "cpu", 0, 1, 8);
+ clks[ARTPEC6_CLK_SPI_SSPCLK] =
+ clk_register_fixed_rate(NULL, "spi_sspclk", sys_refclk_name, 0,
+ 50000000);
+
+ clks[ARTPEC6_CLK_DBG_PCLK] =
+ clk_register_fixed_factor(NULL, "dbg_pclk", "cpu", 0, 1, 8);
+
+ clkdata->clk_data.clks = clkdata->clk_table;
+ clkdata->clk_data.clk_num = ARTPEC6_CLK_NUMCLOCKS;
+
+ of_clk_add_provider(np, of_clk_src_onecell_get, &clkdata->clk_data);
+}
+
+CLK_OF_DECLARE(artpec6_clkctrl, "axis,artpec6-clkctrl",
+ of_artpec6_clkctrl_setup);
+
+static int artpec6_clkctrl_probe(struct platform_device *pdev)
+{
+ int propidx;
+ struct device_node *np = pdev->dev.of_node;
+ struct device *dev = &pdev->dev;
+ struct clk **clks = clkdata->clk_table;
+ const char *sys_refclk_name;
+ const char *i2s_refclk_name = NULL;
+ const char *frac_clk_name[2] = { NULL, NULL };
+ const char *i2s_mux_parents[2];
+ u32 muxreg;
+ int i;
+ int err = 0;
+
+ /* Mandatory parent clock. */
+ propidx = of_property_match_string(np, "clock-names", "sys_refclk");
+ if (propidx < 0)
+ return -EINVAL;
+
+ sys_refclk_name = of_clk_get_parent_name(np, propidx);
+
+ /* Find clock names of optional parent clocks. */
+ propidx = of_property_match_string(np, "clock-names", "i2s_refclk");
+ if (propidx >= 0)
+ i2s_refclk_name = of_clk_get_parent_name(np, propidx);
+
+ propidx = of_property_match_string(np, "clock-names", "frac_clk0");
+ if (propidx >= 0)
+ frac_clk_name[0] = of_clk_get_parent_name(np, propidx);
+ propidx = of_property_match_string(np, "clock-names", "frac_clk1");
+ if (propidx >= 0)
+ frac_clk_name[1] = of_clk_get_parent_name(np, propidx);
+
+ spin_lock_init(&clkdata->i2scfg_lock);
+
+ clks[ARTPEC6_CLK_NAND_CLKA] =
+ clk_register_fixed_factor(dev, "nand_clka", "cpu", 0, 1, 8);
+ clks[ARTPEC6_CLK_NAND_CLKB] =
+ clk_register_fixed_rate(dev, "nand_clkb", sys_refclk_name, 0,
+ 100000000);
+ clks[ARTPEC6_CLK_ETH_ACLK] =
+ clk_register_fixed_factor(dev, "eth_aclk", "cpu", 0, 1, 4);
+ clks[ARTPEC6_CLK_DMA_ACLK] =
+ clk_register_fixed_factor(dev, "dma_aclk", "cpu", 0, 1, 4);
+ clks[ARTPEC6_CLK_PTP_REF] =
+ clk_register_fixed_rate(dev, "ptp_ref", sys_refclk_name, 0,
+ 100000000);
+ clks[ARTPEC6_CLK_SD_PCLK] =
+ clk_register_fixed_rate(dev, "sd_pclk", sys_refclk_name, 0,
+ 100000000);
+ clks[ARTPEC6_CLK_SD_IMCLK] =
+ clk_register_fixed_rate(dev, "sd_imclk", sys_refclk_name, 0,
+ 100000000);
+ clks[ARTPEC6_CLK_I2S_HST] =
+ clk_register_fixed_factor(dev, "i2s_hst", "cpu", 0, 1, 8);
+
+ for (i = 0; i < NUM_I2S_CLOCKS; ++i) {
+ if (i2s_refclk_name && frac_clk_name[i]) {
+ i2s_mux_parents[0] = frac_clk_name[i];
+ i2s_mux_parents[1] = i2s_refclk_name;
+
+ clks[i2s_clk_indexes[i]] =
+ clk_register_mux(dev, i2s_clk_names[i],
+ i2s_mux_parents, 2,
+ CLK_SET_RATE_NO_REPARENT |
+ CLK_SET_RATE_PARENT,
+ clkdata->syscon_base + 0x14, i, 1,
+ 0, &clkdata->i2scfg_lock);
+ } else if (frac_clk_name[i]) {
+ /* Lock the mux for internal clock reference. */
+ muxreg = readl(clkdata->syscon_base + 0x14);
+ muxreg &= ~BIT(i);
+ writel(muxreg, clkdata->syscon_base + 0x14);
+ clks[i2s_clk_indexes[i]] =
+ clk_register_fixed_factor(dev, i2s_clk_names[i],
+ frac_clk_name[i], 0, 1,
+ 1);
+ } else if (i2s_refclk_name) {
+ /* Lock the mux for external clock reference. */
+ muxreg = readl(clkdata->syscon_base + 0x14);
+ muxreg |= BIT(i);
+ writel(muxreg, clkdata->syscon_base + 0x14);
+ clks[i2s_clk_indexes[i]] =
+ clk_register_fixed_factor(dev, i2s_clk_names[i],
+ i2s_refclk_name, 0, 1, 1);
+ }
+ }
+
+ clks[ARTPEC6_CLK_I2C] =
+ clk_register_fixed_rate(dev, "i2c", sys_refclk_name, 0, 100000000);
+
+ clks[ARTPEC6_CLK_SYS_TIMER] =
+ clk_register_fixed_rate(dev, "timer", sys_refclk_name, 0,
+ 100000000);
+ clks[ARTPEC6_CLK_FRACDIV_IN] =
+ clk_register_fixed_rate(dev, "fracdiv_in", sys_refclk_name, 0,
+ 600000000);
+
+ for (i = 0; i < ARTPEC6_CLK_NUMCLOCKS; ++i) {
+ if (IS_ERR(clks[i]) && PTR_ERR(clks[i]) != -EPROBE_DEFER) {
+ dev_err(dev,
+ "Failed to register clock at index %d err=%ld\n",
+ i, PTR_ERR(clks[i]));
+ err = PTR_ERR(clks[i]);
+ }
+ }
+
+ return err;
+}
+
+static const struct of_device_id artpec_clkctrl_of_match[] = {
+ { .compatible = "axis,artpec6-clkctrl" },
+ {}
+};
+
+static struct platform_driver artpec6_clkctrl_driver = {
+ .probe = artpec6_clkctrl_probe,
+ .driver = {
+ .name = "artpec6_clkctrl",
+ .of_match_table = artpec_clkctrl_of_match,
+ },
+};
+
+builtin_platform_driver(artpec6_clkctrl_driver);
diff --git a/drivers/clk/axs10x/Makefile b/drivers/clk/axs10x/Makefile
new file mode 100644
index 0000000..01996b8
--- /dev/null
+++ b/drivers/clk/axs10x/Makefile
@@ -0,0 +1 @@
+obj-y += i2s_pll_clock.o
diff --git a/drivers/clk/axs10x/i2s_pll_clock.c b/drivers/clk/axs10x/i2s_pll_clock.c
new file mode 100644
index 0000000..411310d
--- /dev/null
+++ b/drivers/clk/axs10x/i2s_pll_clock.c
@@ -0,0 +1,228 @@
+/*
+ * Synopsys AXS10X SDP I2S PLL clock driver
+ *
+ * Copyright (C) 2016 Synopsys
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2. This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#include <linux/platform_device.h>
+#include <linux/module.h>
+#include <linux/clk-provider.h>
+#include <linux/err.h>
+#include <linux/device.h>
+#include <linux/of_address.h>
+#include <linux/slab.h>
+#include <linux/of.h>
+
+/* PLL registers addresses */
+#define PLL_IDIV_REG 0x0
+#define PLL_FBDIV_REG 0x4
+#define PLL_ODIV0_REG 0x8
+#define PLL_ODIV1_REG 0xC
+
+struct i2s_pll_cfg {
+ unsigned int rate;
+ unsigned int idiv;
+ unsigned int fbdiv;
+ unsigned int odiv0;
+ unsigned int odiv1;
+};
+
+static const struct i2s_pll_cfg i2s_pll_cfg_27m[] = {
+ /* 27 Mhz */
+ { 1024000, 0x104, 0x451, 0x10E38, 0x2000 },
+ { 1411200, 0x104, 0x596, 0x10D35, 0x2000 },
+ { 1536000, 0x208, 0xA28, 0x10B2C, 0x2000 },
+ { 2048000, 0x82, 0x451, 0x10E38, 0x2000 },
+ { 2822400, 0x82, 0x596, 0x10D35, 0x2000 },
+ { 3072000, 0x104, 0xA28, 0x10B2C, 0x2000 },
+ { 2116800, 0x82, 0x3CF, 0x10C30, 0x2000 },
+ { 2304000, 0x104, 0x79E, 0x10B2C, 0x2000 },
+ { 0, 0, 0, 0, 0 },
+};
+
+static const struct i2s_pll_cfg i2s_pll_cfg_28m[] = {
+ /* 28.224 Mhz */
+ { 1024000, 0x82, 0x105, 0x107DF, 0x2000 },
+ { 1411200, 0x28A, 0x1, 0x10001, 0x2000 },
+ { 1536000, 0xA28, 0x187, 0x10042, 0x2000 },
+ { 2048000, 0x41, 0x105, 0x107DF, 0x2000 },
+ { 2822400, 0x145, 0x1, 0x10001, 0x2000 },
+ { 3072000, 0x514, 0x187, 0x10042, 0x2000 },
+ { 2116800, 0x514, 0x42, 0x10001, 0x2000 },
+ { 2304000, 0x619, 0x82, 0x10001, 0x2000 },
+ { 0, 0, 0, 0, 0 },
+};
+
+struct i2s_pll_clk {
+ void __iomem *base;
+ struct clk_hw hw;
+ struct device *dev;
+};
+
+static inline void i2s_pll_write(struct i2s_pll_clk *clk, unsigned int reg,
+ unsigned int val)
+{
+ writel_relaxed(val, clk->base + reg);
+}
+
+static inline unsigned int i2s_pll_read(struct i2s_pll_clk *clk,
+ unsigned int reg)
+{
+ return readl_relaxed(clk->base + reg);
+}
+
+static inline struct i2s_pll_clk *to_i2s_pll_clk(struct clk_hw *hw)
+{
+ return container_of(hw, struct i2s_pll_clk, hw);
+}
+
+static inline unsigned int i2s_pll_get_value(unsigned int val)
+{
+ return (val & 0x3F) + ((val >> 6) & 0x3F);
+}
+
+static const struct i2s_pll_cfg *i2s_pll_get_cfg(unsigned long prate)
+{
+ switch (prate) {
+ case 27000000:
+ return i2s_pll_cfg_27m;
+ case 28224000:
+ return i2s_pll_cfg_28m;
+ default:
+ return NULL;
+ }
+}
+
+static unsigned long i2s_pll_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct i2s_pll_clk *clk = to_i2s_pll_clk(hw);
+ unsigned int idiv, fbdiv, odiv;
+
+ idiv = i2s_pll_get_value(i2s_pll_read(clk, PLL_IDIV_REG));
+ fbdiv = i2s_pll_get_value(i2s_pll_read(clk, PLL_FBDIV_REG));
+ odiv = i2s_pll_get_value(i2s_pll_read(clk, PLL_ODIV0_REG));
+
+ return ((parent_rate / idiv) * fbdiv) / odiv;
+}
+
+static long i2s_pll_round_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long *prate)
+{
+ struct i2s_pll_clk *clk = to_i2s_pll_clk(hw);
+ const struct i2s_pll_cfg *pll_cfg = i2s_pll_get_cfg(*prate);
+ int i;
+
+ if (!pll_cfg) {
+ dev_err(clk->dev, "invalid parent rate=%ld\n", *prate);
+ return -EINVAL;
+ }
+
+ for (i = 0; pll_cfg[i].rate != 0; i++)
+ if (pll_cfg[i].rate == rate)
+ return rate;
+
+ return -EINVAL;
+}
+
+static int i2s_pll_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct i2s_pll_clk *clk = to_i2s_pll_clk(hw);
+ const struct i2s_pll_cfg *pll_cfg = i2s_pll_get_cfg(parent_rate);
+ int i;
+
+ if (!pll_cfg) {
+ dev_err(clk->dev, "invalid parent rate=%ld\n", parent_rate);
+ return -EINVAL;
+ }
+
+ for (i = 0; pll_cfg[i].rate != 0; i++) {
+ if (pll_cfg[i].rate == rate) {
+ i2s_pll_write(clk, PLL_IDIV_REG, pll_cfg[i].idiv);
+ i2s_pll_write(clk, PLL_FBDIV_REG, pll_cfg[i].fbdiv);
+ i2s_pll_write(clk, PLL_ODIV0_REG, pll_cfg[i].odiv0);
+ i2s_pll_write(clk, PLL_ODIV1_REG, pll_cfg[i].odiv1);
+ return 0;
+ }
+ }
+
+ dev_err(clk->dev, "invalid rate=%ld, parent_rate=%ld\n", rate,
+ parent_rate);
+ return -EINVAL;
+}
+
+static const struct clk_ops i2s_pll_ops = {
+ .recalc_rate = i2s_pll_recalc_rate,
+ .round_rate = i2s_pll_round_rate,
+ .set_rate = i2s_pll_set_rate,
+};
+
+static int i2s_pll_clk_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *node = dev->of_node;
+ const char *clk_name;
+ const char *parent_name;
+ struct clk *clk;
+ struct i2s_pll_clk *pll_clk;
+ struct clk_init_data init;
+ struct resource *mem;
+
+ pll_clk = devm_kzalloc(dev, sizeof(*pll_clk), GFP_KERNEL);
+ if (!pll_clk)
+ return -ENOMEM;
+
+ mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ pll_clk->base = devm_ioremap_resource(dev, mem);
+ if (IS_ERR(pll_clk->base))
+ return PTR_ERR(pll_clk->base);
+
+ clk_name = node->name;
+ init.name = clk_name;
+ init.ops = &i2s_pll_ops;
+ parent_name = of_clk_get_parent_name(node, 0);
+ init.parent_names = &parent_name;
+ init.num_parents = 1;
+ pll_clk->hw.init = &init;
+ pll_clk->dev = dev;
+
+ clk = devm_clk_register(dev, &pll_clk->hw);
+ if (IS_ERR(clk)) {
+ dev_err(dev, "failed to register %s clock (%ld)\n",
+ clk_name, PTR_ERR(clk));
+ return PTR_ERR(clk);
+ }
+
+ return of_clk_add_provider(node, of_clk_src_simple_get, clk);
+}
+
+static int i2s_pll_clk_remove(struct platform_device *pdev)
+{
+ of_clk_del_provider(pdev->dev.of_node);
+ return 0;
+}
+
+static const struct of_device_id i2s_pll_clk_id[] = {
+ { .compatible = "snps,axs10x-i2s-pll-clock", },
+ { },
+};
+MODULE_DEVICE_TABLE(of, i2s_pll_clk_id);
+
+static struct platform_driver i2s_pll_clk_driver = {
+ .driver = {
+ .name = "axs10x-i2s-pll-clock",
+ .of_match_table = i2s_pll_clk_id,
+ },
+ .probe = i2s_pll_clk_probe,
+ .remove = i2s_pll_clk_remove,
+};
+module_platform_driver(i2s_pll_clk_driver);
+
+MODULE_AUTHOR("Jose Abreu <joabreu@synopsys.com>");
+MODULE_DESCRIPTION("Synopsys AXS10X SDP I2S PLL Clock Driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/clk/bcm/clk-bcm2835.c b/drivers/clk/bcm/clk-bcm2835.c
index c74ed3f..7a79708 100644
--- a/drivers/clk/bcm/clk-bcm2835.c
+++ b/drivers/clk/bcm/clk-bcm2835.c
@@ -12,9 +12,6 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
/**
@@ -40,6 +37,7 @@
#include <linux/clk-provider.h>
#include <linux/clkdev.h>
#include <linux/clk/bcm2835.h>
+#include <linux/debugfs.h>
#include <linux/module.h>
#include <linux/of.h>
#include <linux/platform_device.h>
@@ -51,6 +49,7 @@
#define CM_GNRICCTL 0x000
#define CM_GNRICDIV 0x004
# define CM_DIV_FRAC_BITS 12
+# define CM_DIV_FRAC_MASK GENMASK(CM_DIV_FRAC_BITS - 1, 0)
#define CM_VPUCTL 0x008
#define CM_VPUDIV 0x00c
@@ -118,6 +117,8 @@
#define CM_SDCCTL 0x1a8
#define CM_SDCDIV 0x1ac
#define CM_ARMCTL 0x1b0
+#define CM_AVEOCTL 0x1b8
+#define CM_AVEODIV 0x1bc
#define CM_EMMCCTL 0x1c0
#define CM_EMMCDIV 0x1c4
@@ -128,6 +129,7 @@
# define CM_GATE BIT(CM_GATE_BIT)
# define CM_BUSY BIT(7)
# define CM_BUSYD BIT(8)
+# define CM_FRAC BIT(9)
# define CM_SRC_SHIFT 0
# define CM_SRC_BITS 4
# define CM_SRC_MASK 0xf
@@ -297,11 +299,11 @@
struct bcm2835_cprman {
struct device *dev;
void __iomem *regs;
- spinlock_t regs_lock;
+ spinlock_t regs_lock; /* spinlock for all clocks */
const char *osc_name;
struct clk_onecell_data onecell;
- struct clk *clks[BCM2835_CLOCK_COUNT];
+ struct clk *clks[];
};
static inline void cprman_write(struct bcm2835_cprman *cprman, u32 reg, u32 val)
@@ -314,6 +316,27 @@
return readl(cprman->regs + reg);
}
+static int bcm2835_debugfs_regset(struct bcm2835_cprman *cprman, u32 base,
+ struct debugfs_reg32 *regs, size_t nregs,
+ struct dentry *dentry)
+{
+ struct dentry *regdump;
+ struct debugfs_regset32 *regset;
+
+ regset = devm_kzalloc(cprman->dev, sizeof(*regset), GFP_KERNEL);
+ if (!regset)
+ return -ENOMEM;
+
+ regset->regs = regs;
+ regset->nregs = nregs;
+ regset->base = cprman->regs + base;
+
+ regdump = debugfs_create_regset32("regdump", S_IRUGO, dentry,
+ regset);
+
+ return regdump ? 0 : -ENOMEM;
+}
+
/*
* These are fixed clocks. They're probably not all root clocks and it may
* be possible to turn them on and off but until this is mapped out better
@@ -377,132 +400,27 @@
static const struct bcm2835_pll_ana_bits bcm2835_ana_default = {
.mask0 = 0,
.set0 = 0,
- .mask1 = ~(A2W_PLL_KI_MASK | A2W_PLL_KP_MASK),
+ .mask1 = (u32)~(A2W_PLL_KI_MASK | A2W_PLL_KP_MASK),
.set1 = (2 << A2W_PLL_KI_SHIFT) | (8 << A2W_PLL_KP_SHIFT),
- .mask3 = ~A2W_PLL_KA_MASK,
+ .mask3 = (u32)~A2W_PLL_KA_MASK,
.set3 = (2 << A2W_PLL_KA_SHIFT),
.fb_prediv_mask = BIT(14),
};
static const struct bcm2835_pll_ana_bits bcm2835_ana_pllh = {
- .mask0 = ~(A2W_PLLH_KA_MASK | A2W_PLLH_KI_LOW_MASK),
+ .mask0 = (u32)~(A2W_PLLH_KA_MASK | A2W_PLLH_KI_LOW_MASK),
.set0 = (2 << A2W_PLLH_KA_SHIFT) | (2 << A2W_PLLH_KI_LOW_SHIFT),
- .mask1 = ~(A2W_PLLH_KI_HIGH_MASK | A2W_PLLH_KP_MASK),
+ .mask1 = (u32)~(A2W_PLLH_KI_HIGH_MASK | A2W_PLLH_KP_MASK),
.set1 = (6 << A2W_PLLH_KP_SHIFT),
.mask3 = 0,
.set3 = 0,
.fb_prediv_mask = BIT(11),
};
-/*
- * PLLA is the auxiliary PLL, used to drive the CCP2 (Compact Camera
- * Port 2) transmitter clock.
- *
- * It is in the PX LDO power domain, which is on when the AUDIO domain
- * is on.
- */
-static const struct bcm2835_pll_data bcm2835_plla_data = {
- .name = "plla",
- .cm_ctrl_reg = CM_PLLA,
- .a2w_ctrl_reg = A2W_PLLA_CTRL,
- .frac_reg = A2W_PLLA_FRAC,
- .ana_reg_base = A2W_PLLA_ANA0,
- .reference_enable_mask = A2W_XOSC_CTRL_PLLA_ENABLE,
- .lock_mask = CM_LOCK_FLOCKA,
-
- .ana = &bcm2835_ana_default,
-
- .min_rate = 600000000u,
- .max_rate = 2400000000u,
- .max_fb_rate = BCM2835_MAX_FB_RATE,
-};
-
-/* PLLB is used for the ARM's clock. */
-static const struct bcm2835_pll_data bcm2835_pllb_data = {
- .name = "pllb",
- .cm_ctrl_reg = CM_PLLB,
- .a2w_ctrl_reg = A2W_PLLB_CTRL,
- .frac_reg = A2W_PLLB_FRAC,
- .ana_reg_base = A2W_PLLB_ANA0,
- .reference_enable_mask = A2W_XOSC_CTRL_PLLB_ENABLE,
- .lock_mask = CM_LOCK_FLOCKB,
-
- .ana = &bcm2835_ana_default,
-
- .min_rate = 600000000u,
- .max_rate = 3000000000u,
- .max_fb_rate = BCM2835_MAX_FB_RATE,
-};
-
-/*
- * PLLC is the core PLL, used to drive the core VPU clock.
- *
- * It is in the PX LDO power domain, which is on when the AUDIO domain
- * is on.
-*/
-static const struct bcm2835_pll_data bcm2835_pllc_data = {
- .name = "pllc",
- .cm_ctrl_reg = CM_PLLC,
- .a2w_ctrl_reg = A2W_PLLC_CTRL,
- .frac_reg = A2W_PLLC_FRAC,
- .ana_reg_base = A2W_PLLC_ANA0,
- .reference_enable_mask = A2W_XOSC_CTRL_PLLC_ENABLE,
- .lock_mask = CM_LOCK_FLOCKC,
-
- .ana = &bcm2835_ana_default,
-
- .min_rate = 600000000u,
- .max_rate = 3000000000u,
- .max_fb_rate = BCM2835_MAX_FB_RATE,
-};
-
-/*
- * PLLD is the display PLL, used to drive DSI display panels.
- *
- * It is in the PX LDO power domain, which is on when the AUDIO domain
- * is on.
- */
-static const struct bcm2835_pll_data bcm2835_plld_data = {
- .name = "plld",
- .cm_ctrl_reg = CM_PLLD,
- .a2w_ctrl_reg = A2W_PLLD_CTRL,
- .frac_reg = A2W_PLLD_FRAC,
- .ana_reg_base = A2W_PLLD_ANA0,
- .reference_enable_mask = A2W_XOSC_CTRL_DDR_ENABLE,
- .lock_mask = CM_LOCK_FLOCKD,
-
- .ana = &bcm2835_ana_default,
-
- .min_rate = 600000000u,
- .max_rate = 2400000000u,
- .max_fb_rate = BCM2835_MAX_FB_RATE,
-};
-
-/*
- * PLLH is used to supply the pixel clock or the AUX clock for the TV
- * encoder.
- *
- * It is in the HDMI power domain.
- */
-static const struct bcm2835_pll_data bcm2835_pllh_data = {
- "pllh",
- .cm_ctrl_reg = CM_PLLH,
- .a2w_ctrl_reg = A2W_PLLH_CTRL,
- .frac_reg = A2W_PLLH_FRAC,
- .ana_reg_base = A2W_PLLH_ANA0,
- .reference_enable_mask = A2W_XOSC_CTRL_PLLC_ENABLE,
- .lock_mask = CM_LOCK_FLOCKH,
-
- .ana = &bcm2835_ana_pllh,
-
- .min_rate = 600000000u,
- .max_rate = 3000000000u,
- .max_fb_rate = BCM2835_MAX_FB_RATE,
-};
-
struct bcm2835_pll_divider_data {
const char *name;
- const struct bcm2835_pll_data *source_pll;
+ const char *source_pll;
+
u32 cm_reg;
u32 a2w_reg;
@@ -511,124 +429,6 @@
u32 fixed_divider;
};
-static const struct bcm2835_pll_divider_data bcm2835_plla_core_data = {
- .name = "plla_core",
- .source_pll = &bcm2835_plla_data,
- .cm_reg = CM_PLLA,
- .a2w_reg = A2W_PLLA_CORE,
- .load_mask = CM_PLLA_LOADCORE,
- .hold_mask = CM_PLLA_HOLDCORE,
- .fixed_divider = 1,
-};
-
-static const struct bcm2835_pll_divider_data bcm2835_plla_per_data = {
- .name = "plla_per",
- .source_pll = &bcm2835_plla_data,
- .cm_reg = CM_PLLA,
- .a2w_reg = A2W_PLLA_PER,
- .load_mask = CM_PLLA_LOADPER,
- .hold_mask = CM_PLLA_HOLDPER,
- .fixed_divider = 1,
-};
-
-static const struct bcm2835_pll_divider_data bcm2835_pllb_arm_data = {
- .name = "pllb_arm",
- .source_pll = &bcm2835_pllb_data,
- .cm_reg = CM_PLLB,
- .a2w_reg = A2W_PLLB_ARM,
- .load_mask = CM_PLLB_LOADARM,
- .hold_mask = CM_PLLB_HOLDARM,
- .fixed_divider = 1,
-};
-
-static const struct bcm2835_pll_divider_data bcm2835_pllc_core0_data = {
- .name = "pllc_core0",
- .source_pll = &bcm2835_pllc_data,
- .cm_reg = CM_PLLC,
- .a2w_reg = A2W_PLLC_CORE0,
- .load_mask = CM_PLLC_LOADCORE0,
- .hold_mask = CM_PLLC_HOLDCORE0,
- .fixed_divider = 1,
-};
-
-static const struct bcm2835_pll_divider_data bcm2835_pllc_core1_data = {
- .name = "pllc_core1", .source_pll = &bcm2835_pllc_data,
- .cm_reg = CM_PLLC, A2W_PLLC_CORE1,
- .load_mask = CM_PLLC_LOADCORE1,
- .hold_mask = CM_PLLC_HOLDCORE1,
- .fixed_divider = 1,
-};
-
-static const struct bcm2835_pll_divider_data bcm2835_pllc_core2_data = {
- .name = "pllc_core2",
- .source_pll = &bcm2835_pllc_data,
- .cm_reg = CM_PLLC,
- .a2w_reg = A2W_PLLC_CORE2,
- .load_mask = CM_PLLC_LOADCORE2,
- .hold_mask = CM_PLLC_HOLDCORE2,
- .fixed_divider = 1,
-};
-
-static const struct bcm2835_pll_divider_data bcm2835_pllc_per_data = {
- .name = "pllc_per",
- .source_pll = &bcm2835_pllc_data,
- .cm_reg = CM_PLLC,
- .a2w_reg = A2W_PLLC_PER,
- .load_mask = CM_PLLC_LOADPER,
- .hold_mask = CM_PLLC_HOLDPER,
- .fixed_divider = 1,
-};
-
-static const struct bcm2835_pll_divider_data bcm2835_plld_core_data = {
- .name = "plld_core",
- .source_pll = &bcm2835_plld_data,
- .cm_reg = CM_PLLD,
- .a2w_reg = A2W_PLLD_CORE,
- .load_mask = CM_PLLD_LOADCORE,
- .hold_mask = CM_PLLD_HOLDCORE,
- .fixed_divider = 1,
-};
-
-static const struct bcm2835_pll_divider_data bcm2835_plld_per_data = {
- .name = "plld_per",
- .source_pll = &bcm2835_plld_data,
- .cm_reg = CM_PLLD,
- .a2w_reg = A2W_PLLD_PER,
- .load_mask = CM_PLLD_LOADPER,
- .hold_mask = CM_PLLD_HOLDPER,
- .fixed_divider = 1,
-};
-
-static const struct bcm2835_pll_divider_data bcm2835_pllh_rcal_data = {
- .name = "pllh_rcal",
- .source_pll = &bcm2835_pllh_data,
- .cm_reg = CM_PLLH,
- .a2w_reg = A2W_PLLH_RCAL,
- .load_mask = CM_PLLH_LOADRCAL,
- .hold_mask = 0,
- .fixed_divider = 10,
-};
-
-static const struct bcm2835_pll_divider_data bcm2835_pllh_aux_data = {
- .name = "pllh_aux",
- .source_pll = &bcm2835_pllh_data,
- .cm_reg = CM_PLLH,
- .a2w_reg = A2W_PLLH_AUX,
- .load_mask = CM_PLLH_LOADAUX,
- .hold_mask = 0,
- .fixed_divider = 10,
-};
-
-static const struct bcm2835_pll_divider_data bcm2835_pllh_pix_data = {
- .name = "pllh_pix",
- .source_pll = &bcm2835_pllh_data,
- .cm_reg = CM_PLLH,
- .a2w_reg = A2W_PLLH_PIX,
- .load_mask = CM_PLLH_LOADPIX,
- .hold_mask = 0,
- .fixed_divider = 10,
-};
-
struct bcm2835_clock_data {
const char *name;
@@ -644,187 +444,14 @@
u32 frac_bits;
bool is_vpu_clock;
+ bool is_mash_clock;
};
-static const char *const bcm2835_clock_per_parents[] = {
- "gnd",
- "xosc",
- "testdebug0",
- "testdebug1",
- "plla_per",
- "pllc_per",
- "plld_per",
- "pllh_aux",
-};
+struct bcm2835_gate_data {
+ const char *name;
+ const char *parent;
-static const char *const bcm2835_clock_vpu_parents[] = {
- "gnd",
- "xosc",
- "testdebug0",
- "testdebug1",
- "plla_core",
- "pllc_core0",
- "plld_core",
- "pllh_aux",
- "pllc_core1",
- "pllc_core2",
-};
-
-static const char *const bcm2835_clock_osc_parents[] = {
- "gnd",
- "xosc",
- "testdebug0",
- "testdebug1"
-};
-
-/*
- * Used for a 1Mhz clock for the system clocksource, and also used by
- * the watchdog timer and the camera pulse generator.
- */
-static const struct bcm2835_clock_data bcm2835_clock_timer_data = {
- .name = "timer",
- .num_mux_parents = ARRAY_SIZE(bcm2835_clock_osc_parents),
- .parents = bcm2835_clock_osc_parents,
- .ctl_reg = CM_TIMERCTL,
- .div_reg = CM_TIMERDIV,
- .int_bits = 6,
- .frac_bits = 12,
-};
-
-/* One Time Programmable Memory clock. Maximum 10Mhz. */
-static const struct bcm2835_clock_data bcm2835_clock_otp_data = {
- .name = "otp",
- .num_mux_parents = ARRAY_SIZE(bcm2835_clock_osc_parents),
- .parents = bcm2835_clock_osc_parents,
- .ctl_reg = CM_OTPCTL,
- .div_reg = CM_OTPDIV,
- .int_bits = 4,
- .frac_bits = 0,
-};
-
-/*
- * VPU clock. This doesn't have an enable bit, since it drives the
- * bus for everything else, and is special so it doesn't need to be
- * gated for rate changes. It is also known as "clk_audio" in various
- * hardware documentation.
- */
-static const struct bcm2835_clock_data bcm2835_clock_vpu_data = {
- .name = "vpu",
- .num_mux_parents = ARRAY_SIZE(bcm2835_clock_vpu_parents),
- .parents = bcm2835_clock_vpu_parents,
- .ctl_reg = CM_VPUCTL,
- .div_reg = CM_VPUDIV,
- .int_bits = 12,
- .frac_bits = 8,
- .is_vpu_clock = true,
-};
-
-static const struct bcm2835_clock_data bcm2835_clock_v3d_data = {
- .name = "v3d",
- .num_mux_parents = ARRAY_SIZE(bcm2835_clock_vpu_parents),
- .parents = bcm2835_clock_vpu_parents,
- .ctl_reg = CM_V3DCTL,
- .div_reg = CM_V3DDIV,
- .int_bits = 4,
- .frac_bits = 8,
-};
-
-static const struct bcm2835_clock_data bcm2835_clock_isp_data = {
- .name = "isp",
- .num_mux_parents = ARRAY_SIZE(bcm2835_clock_vpu_parents),
- .parents = bcm2835_clock_vpu_parents,
- .ctl_reg = CM_ISPCTL,
- .div_reg = CM_ISPDIV,
- .int_bits = 4,
- .frac_bits = 8,
-};
-
-static const struct bcm2835_clock_data bcm2835_clock_h264_data = {
- .name = "h264",
- .num_mux_parents = ARRAY_SIZE(bcm2835_clock_vpu_parents),
- .parents = bcm2835_clock_vpu_parents,
- .ctl_reg = CM_H264CTL,
- .div_reg = CM_H264DIV,
- .int_bits = 4,
- .frac_bits = 8,
-};
-
-/* TV encoder clock. Only operating frequency is 108Mhz. */
-static const struct bcm2835_clock_data bcm2835_clock_vec_data = {
- .name = "vec",
- .num_mux_parents = ARRAY_SIZE(bcm2835_clock_per_parents),
- .parents = bcm2835_clock_per_parents,
- .ctl_reg = CM_VECCTL,
- .div_reg = CM_VECDIV,
- .int_bits = 4,
- .frac_bits = 0,
-};
-
-static const struct bcm2835_clock_data bcm2835_clock_uart_data = {
- .name = "uart",
- .num_mux_parents = ARRAY_SIZE(bcm2835_clock_per_parents),
- .parents = bcm2835_clock_per_parents,
- .ctl_reg = CM_UARTCTL,
- .div_reg = CM_UARTDIV,
- .int_bits = 10,
- .frac_bits = 12,
-};
-
-/* HDMI state machine */
-static const struct bcm2835_clock_data bcm2835_clock_hsm_data = {
- .name = "hsm",
- .num_mux_parents = ARRAY_SIZE(bcm2835_clock_per_parents),
- .parents = bcm2835_clock_per_parents,
- .ctl_reg = CM_HSMCTL,
- .div_reg = CM_HSMDIV,
- .int_bits = 4,
- .frac_bits = 8,
-};
-
-/*
- * Secondary SDRAM clock. Used for low-voltage modes when the PLL in
- * the SDRAM controller can't be used.
- */
-static const struct bcm2835_clock_data bcm2835_clock_sdram_data = {
- .name = "sdram",
- .num_mux_parents = ARRAY_SIZE(bcm2835_clock_vpu_parents),
- .parents = bcm2835_clock_vpu_parents,
- .ctl_reg = CM_SDCCTL,
- .div_reg = CM_SDCDIV,
- .int_bits = 6,
- .frac_bits = 0,
-};
-
-/* Clock for the temperature sensor. Generally run at 2Mhz, max 5Mhz. */
-static const struct bcm2835_clock_data bcm2835_clock_tsens_data = {
- .name = "tsens",
- .num_mux_parents = ARRAY_SIZE(bcm2835_clock_osc_parents),
- .parents = bcm2835_clock_osc_parents,
- .ctl_reg = CM_TSENSCTL,
- .div_reg = CM_TSENSDIV,
- .int_bits = 5,
- .frac_bits = 0,
-};
-
-/* Arasan EMMC clock */
-static const struct bcm2835_clock_data bcm2835_clock_emmc_data = {
- .name = "emmc",
- .num_mux_parents = ARRAY_SIZE(bcm2835_clock_per_parents),
- .parents = bcm2835_clock_per_parents,
- .ctl_reg = CM_EMMCCTL,
- .div_reg = CM_EMMCDIV,
- .int_bits = 4,
- .frac_bits = 8,
-};
-
-static const struct bcm2835_clock_data bcm2835_clock_pwm_data = {
- .name = "pwm",
- .num_mux_parents = ARRAY_SIZE(bcm2835_clock_per_parents),
- .parents = bcm2835_clock_per_parents,
- .ctl_reg = CM_PWMCTL,
- .div_reg = CM_PWMDIV,
- .int_bits = 12,
- .frac_bits = 12,
+ u32 ctl_reg;
};
struct bcm2835_pll {
@@ -910,8 +537,14 @@
struct bcm2835_cprman *cprman = pll->cprman;
const struct bcm2835_pll_data *data = pll->data;
- cprman_write(cprman, data->cm_ctrl_reg, CM_PLL_ANARST);
- cprman_write(cprman, data->a2w_ctrl_reg, A2W_PLL_CTRL_PWRDN);
+ spin_lock(&cprman->regs_lock);
+ cprman_write(cprman, data->cm_ctrl_reg,
+ cprman_read(cprman, data->cm_ctrl_reg) |
+ CM_PLL_ANARST);
+ cprman_write(cprman, data->a2w_ctrl_reg,
+ cprman_read(cprman, data->a2w_ctrl_reg) |
+ A2W_PLL_CTRL_PWRDN);
+ spin_unlock(&cprman->regs_lock);
}
static int bcm2835_pll_on(struct clk_hw *hw)
@@ -921,6 +554,10 @@
const struct bcm2835_pll_data *data = pll->data;
ktime_t timeout;
+ cprman_write(cprman, data->a2w_ctrl_reg,
+ cprman_read(cprman, data->a2w_ctrl_reg) &
+ ~A2W_PLL_CTRL_PWRDN);
+
/* Take the PLL out of reset. */
cprman_write(cprman, data->cm_ctrl_reg,
cprman_read(cprman, data->cm_ctrl_reg) & ~CM_PLL_ANARST);
@@ -1030,6 +667,36 @@
return 0;
}
+static int bcm2835_pll_debug_init(struct clk_hw *hw,
+ struct dentry *dentry)
+{
+ struct bcm2835_pll *pll = container_of(hw, struct bcm2835_pll, hw);
+ struct bcm2835_cprman *cprman = pll->cprman;
+ const struct bcm2835_pll_data *data = pll->data;
+ struct debugfs_reg32 *regs;
+
+ regs = devm_kzalloc(cprman->dev, 7 * sizeof(*regs), GFP_KERNEL);
+ if (!regs)
+ return -ENOMEM;
+
+ regs[0].name = "cm_ctrl";
+ regs[0].offset = data->cm_ctrl_reg;
+ regs[1].name = "a2w_ctrl";
+ regs[1].offset = data->a2w_ctrl_reg;
+ regs[2].name = "frac";
+ regs[2].offset = data->frac_reg;
+ regs[3].name = "ana0";
+ regs[3].offset = data->ana_reg_base + 0 * 4;
+ regs[4].name = "ana1";
+ regs[4].offset = data->ana_reg_base + 1 * 4;
+ regs[5].name = "ana2";
+ regs[5].offset = data->ana_reg_base + 2 * 4;
+ regs[6].name = "ana3";
+ regs[6].offset = data->ana_reg_base + 3 * 4;
+
+ return bcm2835_debugfs_regset(cprman, 0, regs, 7, dentry);
+}
+
static const struct clk_ops bcm2835_pll_clk_ops = {
.is_prepared = bcm2835_pll_is_on,
.prepare = bcm2835_pll_on,
@@ -1037,6 +704,7 @@
.recalc_rate = bcm2835_pll_get_rate,
.set_rate = bcm2835_pll_set_rate,
.round_rate = bcm2835_pll_round_rate,
+ .debug_init = bcm2835_pll_debug_init,
};
struct bcm2835_pll_divider {
@@ -1079,10 +747,12 @@
struct bcm2835_cprman *cprman = divider->cprman;
const struct bcm2835_pll_divider_data *data = divider->data;
+ spin_lock(&cprman->regs_lock);
cprman_write(cprman, data->cm_reg,
(cprman_read(cprman, data->cm_reg) &
~data->load_mask) | data->hold_mask);
cprman_write(cprman, data->a2w_reg, A2W_PLL_CHANNEL_DISABLE);
+ spin_unlock(&cprman->regs_lock);
}
static int bcm2835_pll_divider_on(struct clk_hw *hw)
@@ -1091,12 +761,14 @@
struct bcm2835_cprman *cprman = divider->cprman;
const struct bcm2835_pll_divider_data *data = divider->data;
+ spin_lock(&cprman->regs_lock);
cprman_write(cprman, data->a2w_reg,
cprman_read(cprman, data->a2w_reg) &
~A2W_PLL_CHANNEL_DISABLE);
cprman_write(cprman, data->cm_reg,
cprman_read(cprman, data->cm_reg) & ~data->hold_mask);
+ spin_unlock(&cprman->regs_lock);
return 0;
}
@@ -1124,6 +796,26 @@
return 0;
}
+static int bcm2835_pll_divider_debug_init(struct clk_hw *hw,
+ struct dentry *dentry)
+{
+ struct bcm2835_pll_divider *divider = bcm2835_pll_divider_from_hw(hw);
+ struct bcm2835_cprman *cprman = divider->cprman;
+ const struct bcm2835_pll_divider_data *data = divider->data;
+ struct debugfs_reg32 *regs;
+
+ regs = devm_kzalloc(cprman->dev, 7 * sizeof(*regs), GFP_KERNEL);
+ if (!regs)
+ return -ENOMEM;
+
+ regs[0].name = "cm";
+ regs[0].offset = data->cm_reg;
+ regs[1].name = "a2w";
+ regs[1].offset = data->a2w_reg;
+
+ return bcm2835_debugfs_regset(cprman, 0, regs, 2, dentry);
+}
+
static const struct clk_ops bcm2835_pll_divider_clk_ops = {
.is_prepared = bcm2835_pll_divider_is_on,
.prepare = bcm2835_pll_divider_on,
@@ -1131,6 +823,7 @@
.recalc_rate = bcm2835_pll_divider_get_rate,
.set_rate = bcm2835_pll_divider_set_rate,
.round_rate = bcm2835_pll_divider_round_rate,
+ .debug_init = bcm2835_pll_divider_debug_init,
};
/*
@@ -1170,7 +863,7 @@
GENMASK(CM_DIV_FRAC_BITS - data->frac_bits, 0) >> 1;
u64 temp = (u64)parent_rate << CM_DIV_FRAC_BITS;
u64 rem;
- u32 div;
+ u32 div, mindiv, maxdiv;
rem = do_div(temp, rate);
div = temp;
@@ -1180,10 +873,23 @@
div += unused_frac_mask + 1;
div &= ~unused_frac_mask;
- /* Clamp to the limits. */
- div = max(div, unused_frac_mask + 1);
- div = min_t(u32, div, GENMASK(data->int_bits + CM_DIV_FRAC_BITS - 1,
- CM_DIV_FRAC_BITS - data->frac_bits));
+ /* different clamping limits apply for a mash clock */
+ if (data->is_mash_clock) {
+ /* clamp to min divider of 2 */
+ mindiv = 2 << CM_DIV_FRAC_BITS;
+ /* clamp to the highest possible integer divider */
+ maxdiv = (BIT(data->int_bits) - 1) << CM_DIV_FRAC_BITS;
+ } else {
+ /* clamp to min divider of 1 */
+ mindiv = 1 << CM_DIV_FRAC_BITS;
+ /* clamp to the highest possible fractional divider */
+ maxdiv = GENMASK(data->int_bits + CM_DIV_FRAC_BITS - 1,
+ CM_DIV_FRAC_BITS - data->frac_bits);
+ }
+
+ /* apply the clamping limits */
+ div = max_t(u32, div, mindiv);
+ div = min_t(u32, div, maxdiv);
return div;
}
@@ -1277,14 +983,31 @@
struct bcm2835_cprman *cprman = clock->cprman;
const struct bcm2835_clock_data *data = clock->data;
u32 div = bcm2835_clock_choose_div(hw, rate, parent_rate, false);
+ u32 ctl;
+
+ spin_lock(&cprman->regs_lock);
+
+ /*
+ * Setting up frac support
+ *
+ * In principle it is recommended to stop/start the clock first,
+ * but as we set CLK_SET_RATE_GATE during registration of the
+ * clock this requirement should be take care of by the
+ * clk-framework.
+ */
+ ctl = cprman_read(cprman, data->ctl_reg) & ~CM_FRAC;
+ ctl |= (div & CM_DIV_FRAC_MASK) ? CM_FRAC : 0;
+ cprman_write(cprman, data->ctl_reg, ctl);
cprman_write(cprman, data->div_reg, div);
+ spin_unlock(&cprman->regs_lock);
+
return 0;
}
static int bcm2835_clock_determine_rate(struct clk_hw *hw,
- struct clk_rate_request *req)
+ struct clk_rate_request *req)
{
struct bcm2835_clock *clock = bcm2835_clock_from_hw(hw);
struct clk_hw *parent, *best_parent = NULL;
@@ -1342,6 +1065,30 @@
return (src & CM_SRC_MASK) >> CM_SRC_SHIFT;
}
+static struct debugfs_reg32 bcm2835_debugfs_clock_reg32[] = {
+ {
+ .name = "ctl",
+ .offset = 0,
+ },
+ {
+ .name = "div",
+ .offset = 4,
+ },
+};
+
+static int bcm2835_clock_debug_init(struct clk_hw *hw,
+ struct dentry *dentry)
+{
+ struct bcm2835_clock *clock = bcm2835_clock_from_hw(hw);
+ struct bcm2835_cprman *cprman = clock->cprman;
+ const struct bcm2835_clock_data *data = clock->data;
+
+ return bcm2835_debugfs_regset(
+ cprman, data->ctl_reg,
+ bcm2835_debugfs_clock_reg32,
+ ARRAY_SIZE(bcm2835_debugfs_clock_reg32),
+ dentry);
+}
static const struct clk_ops bcm2835_clock_clk_ops = {
.is_prepared = bcm2835_clock_is_on,
@@ -1352,6 +1099,7 @@
.determine_rate = bcm2835_clock_determine_rate,
.set_parent = bcm2835_clock_set_parent,
.get_parent = bcm2835_clock_get_parent,
+ .debug_init = bcm2835_clock_debug_init,
};
static int bcm2835_vpu_clock_is_on(struct clk_hw *hw)
@@ -1370,6 +1118,7 @@
.determine_rate = bcm2835_clock_determine_rate,
.set_parent = bcm2835_clock_set_parent,
.get_parent = bcm2835_clock_get_parent,
+ .debug_init = bcm2835_clock_debug_init,
};
static struct clk *bcm2835_register_pll(struct bcm2835_cprman *cprman,
@@ -1418,7 +1167,7 @@
memset(&init, 0, sizeof(init));
- init.parent_names = &data->source_pll->name;
+ init.parent_names = &data->source_pll;
init.num_parents = 1;
init.name = divider_name;
init.ops = &bcm2835_pll_divider_clk_ops;
@@ -1501,14 +1250,559 @@
return devm_clk_register(cprman->dev, &clock->hw);
}
+static struct clk *bcm2835_register_gate(struct bcm2835_cprman *cprman,
+ const struct bcm2835_gate_data *data)
+{
+ return clk_register_gate(cprman->dev, data->name, data->parent,
+ CLK_IGNORE_UNUSED | CLK_SET_RATE_GATE,
+ cprman->regs + data->ctl_reg,
+ CM_GATE_BIT, 0, &cprman->regs_lock);
+}
+
+typedef struct clk *(*bcm2835_clk_register)(struct bcm2835_cprman *cprman,
+ const void *data);
+struct bcm2835_clk_desc {
+ bcm2835_clk_register clk_register;
+ const void *data;
+};
+
+/* assignment helper macros for different clock types */
+#define _REGISTER(f, ...) { .clk_register = (bcm2835_clk_register)f, \
+ .data = __VA_ARGS__ }
+#define REGISTER_PLL(...) _REGISTER(&bcm2835_register_pll, \
+ &(struct bcm2835_pll_data) \
+ {__VA_ARGS__})
+#define REGISTER_PLL_DIV(...) _REGISTER(&bcm2835_register_pll_divider, \
+ &(struct bcm2835_pll_divider_data) \
+ {__VA_ARGS__})
+#define REGISTER_CLK(...) _REGISTER(&bcm2835_register_clock, \
+ &(struct bcm2835_clock_data) \
+ {__VA_ARGS__})
+#define REGISTER_GATE(...) _REGISTER(&bcm2835_register_gate, \
+ &(struct bcm2835_gate_data) \
+ {__VA_ARGS__})
+
+/* parent mux arrays plus helper macros */
+
+/* main oscillator parent mux */
+static const char *const bcm2835_clock_osc_parents[] = {
+ "gnd",
+ "xosc",
+ "testdebug0",
+ "testdebug1"
+};
+
+#define REGISTER_OSC_CLK(...) REGISTER_CLK( \
+ .num_mux_parents = ARRAY_SIZE(bcm2835_clock_osc_parents), \
+ .parents = bcm2835_clock_osc_parents, \
+ __VA_ARGS__)
+
+/* main peripherial parent mux */
+static const char *const bcm2835_clock_per_parents[] = {
+ "gnd",
+ "xosc",
+ "testdebug0",
+ "testdebug1",
+ "plla_per",
+ "pllc_per",
+ "plld_per",
+ "pllh_aux",
+};
+
+#define REGISTER_PER_CLK(...) REGISTER_CLK( \
+ .num_mux_parents = ARRAY_SIZE(bcm2835_clock_per_parents), \
+ .parents = bcm2835_clock_per_parents, \
+ __VA_ARGS__)
+
+/* main vpu parent mux */
+static const char *const bcm2835_clock_vpu_parents[] = {
+ "gnd",
+ "xosc",
+ "testdebug0",
+ "testdebug1",
+ "plla_core",
+ "pllc_core0",
+ "plld_core",
+ "pllh_aux",
+ "pllc_core1",
+ "pllc_core2",
+};
+
+#define REGISTER_VPU_CLK(...) REGISTER_CLK( \
+ .num_mux_parents = ARRAY_SIZE(bcm2835_clock_vpu_parents), \
+ .parents = bcm2835_clock_vpu_parents, \
+ __VA_ARGS__)
+
+/*
+ * the real definition of all the pll, pll_dividers and clocks
+ * these make use of the above REGISTER_* macros
+ */
+static const struct bcm2835_clk_desc clk_desc_array[] = {
+ /* the PLL + PLL dividers */
+
+ /*
+ * PLLA is the auxiliary PLL, used to drive the CCP2
+ * (Compact Camera Port 2) transmitter clock.
+ *
+ * It is in the PX LDO power domain, which is on when the
+ * AUDIO domain is on.
+ */
+ [BCM2835_PLLA] = REGISTER_PLL(
+ .name = "plla",
+ .cm_ctrl_reg = CM_PLLA,
+ .a2w_ctrl_reg = A2W_PLLA_CTRL,
+ .frac_reg = A2W_PLLA_FRAC,
+ .ana_reg_base = A2W_PLLA_ANA0,
+ .reference_enable_mask = A2W_XOSC_CTRL_PLLA_ENABLE,
+ .lock_mask = CM_LOCK_FLOCKA,
+
+ .ana = &bcm2835_ana_default,
+
+ .min_rate = 600000000u,
+ .max_rate = 2400000000u,
+ .max_fb_rate = BCM2835_MAX_FB_RATE),
+ [BCM2835_PLLA_CORE] = REGISTER_PLL_DIV(
+ .name = "plla_core",
+ .source_pll = "plla",
+ .cm_reg = CM_PLLA,
+ .a2w_reg = A2W_PLLA_CORE,
+ .load_mask = CM_PLLA_LOADCORE,
+ .hold_mask = CM_PLLA_HOLDCORE,
+ .fixed_divider = 1),
+ [BCM2835_PLLA_PER] = REGISTER_PLL_DIV(
+ .name = "plla_per",
+ .source_pll = "plla",
+ .cm_reg = CM_PLLA,
+ .a2w_reg = A2W_PLLA_PER,
+ .load_mask = CM_PLLA_LOADPER,
+ .hold_mask = CM_PLLA_HOLDPER,
+ .fixed_divider = 1),
+ [BCM2835_PLLA_DSI0] = REGISTER_PLL_DIV(
+ .name = "plla_dsi0",
+ .source_pll = "plla",
+ .cm_reg = CM_PLLA,
+ .a2w_reg = A2W_PLLA_DSI0,
+ .load_mask = CM_PLLA_LOADDSI0,
+ .hold_mask = CM_PLLA_HOLDDSI0,
+ .fixed_divider = 1),
+ [BCM2835_PLLA_CCP2] = REGISTER_PLL_DIV(
+ .name = "plla_ccp2",
+ .source_pll = "plla",
+ .cm_reg = CM_PLLA,
+ .a2w_reg = A2W_PLLA_CCP2,
+ .load_mask = CM_PLLA_LOADCCP2,
+ .hold_mask = CM_PLLA_HOLDCCP2,
+ .fixed_divider = 1),
+
+ /* PLLB is used for the ARM's clock. */
+ [BCM2835_PLLB] = REGISTER_PLL(
+ .name = "pllb",
+ .cm_ctrl_reg = CM_PLLB,
+ .a2w_ctrl_reg = A2W_PLLB_CTRL,
+ .frac_reg = A2W_PLLB_FRAC,
+ .ana_reg_base = A2W_PLLB_ANA0,
+ .reference_enable_mask = A2W_XOSC_CTRL_PLLB_ENABLE,
+ .lock_mask = CM_LOCK_FLOCKB,
+
+ .ana = &bcm2835_ana_default,
+
+ .min_rate = 600000000u,
+ .max_rate = 3000000000u,
+ .max_fb_rate = BCM2835_MAX_FB_RATE),
+ [BCM2835_PLLB_ARM] = REGISTER_PLL_DIV(
+ .name = "pllb_arm",
+ .source_pll = "pllb",
+ .cm_reg = CM_PLLB,
+ .a2w_reg = A2W_PLLB_ARM,
+ .load_mask = CM_PLLB_LOADARM,
+ .hold_mask = CM_PLLB_HOLDARM,
+ .fixed_divider = 1),
+
+ /*
+ * PLLC is the core PLL, used to drive the core VPU clock.
+ *
+ * It is in the PX LDO power domain, which is on when the
+ * AUDIO domain is on.
+ */
+ [BCM2835_PLLC] = REGISTER_PLL(
+ .name = "pllc",
+ .cm_ctrl_reg = CM_PLLC,
+ .a2w_ctrl_reg = A2W_PLLC_CTRL,
+ .frac_reg = A2W_PLLC_FRAC,
+ .ana_reg_base = A2W_PLLC_ANA0,
+ .reference_enable_mask = A2W_XOSC_CTRL_PLLC_ENABLE,
+ .lock_mask = CM_LOCK_FLOCKC,
+
+ .ana = &bcm2835_ana_default,
+
+ .min_rate = 600000000u,
+ .max_rate = 3000000000u,
+ .max_fb_rate = BCM2835_MAX_FB_RATE),
+ [BCM2835_PLLC_CORE0] = REGISTER_PLL_DIV(
+ .name = "pllc_core0",
+ .source_pll = "pllc",
+ .cm_reg = CM_PLLC,
+ .a2w_reg = A2W_PLLC_CORE0,
+ .load_mask = CM_PLLC_LOADCORE0,
+ .hold_mask = CM_PLLC_HOLDCORE0,
+ .fixed_divider = 1),
+ [BCM2835_PLLC_CORE1] = REGISTER_PLL_DIV(
+ .name = "pllc_core1",
+ .source_pll = "pllc",
+ .cm_reg = CM_PLLC,
+ .a2w_reg = A2W_PLLC_CORE1,
+ .load_mask = CM_PLLC_LOADCORE1,
+ .hold_mask = CM_PLLC_HOLDCORE1,
+ .fixed_divider = 1),
+ [BCM2835_PLLC_CORE2] = REGISTER_PLL_DIV(
+ .name = "pllc_core2",
+ .source_pll = "pllc",
+ .cm_reg = CM_PLLC,
+ .a2w_reg = A2W_PLLC_CORE2,
+ .load_mask = CM_PLLC_LOADCORE2,
+ .hold_mask = CM_PLLC_HOLDCORE2,
+ .fixed_divider = 1),
+ [BCM2835_PLLC_PER] = REGISTER_PLL_DIV(
+ .name = "pllc_per",
+ .source_pll = "pllc",
+ .cm_reg = CM_PLLC,
+ .a2w_reg = A2W_PLLC_PER,
+ .load_mask = CM_PLLC_LOADPER,
+ .hold_mask = CM_PLLC_HOLDPER,
+ .fixed_divider = 1),
+
+ /*
+ * PLLD is the display PLL, used to drive DSI display panels.
+ *
+ * It is in the PX LDO power domain, which is on when the
+ * AUDIO domain is on.
+ */
+ [BCM2835_PLLD] = REGISTER_PLL(
+ .name = "plld",
+ .cm_ctrl_reg = CM_PLLD,
+ .a2w_ctrl_reg = A2W_PLLD_CTRL,
+ .frac_reg = A2W_PLLD_FRAC,
+ .ana_reg_base = A2W_PLLD_ANA0,
+ .reference_enable_mask = A2W_XOSC_CTRL_DDR_ENABLE,
+ .lock_mask = CM_LOCK_FLOCKD,
+
+ .ana = &bcm2835_ana_default,
+
+ .min_rate = 600000000u,
+ .max_rate = 2400000000u,
+ .max_fb_rate = BCM2835_MAX_FB_RATE),
+ [BCM2835_PLLD_CORE] = REGISTER_PLL_DIV(
+ .name = "plld_core",
+ .source_pll = "plld",
+ .cm_reg = CM_PLLD,
+ .a2w_reg = A2W_PLLD_CORE,
+ .load_mask = CM_PLLD_LOADCORE,
+ .hold_mask = CM_PLLD_HOLDCORE,
+ .fixed_divider = 1),
+ [BCM2835_PLLD_PER] = REGISTER_PLL_DIV(
+ .name = "plld_per",
+ .source_pll = "plld",
+ .cm_reg = CM_PLLD,
+ .a2w_reg = A2W_PLLD_PER,
+ .load_mask = CM_PLLD_LOADPER,
+ .hold_mask = CM_PLLD_HOLDPER,
+ .fixed_divider = 1),
+ [BCM2835_PLLD_DSI0] = REGISTER_PLL_DIV(
+ .name = "plld_dsi0",
+ .source_pll = "plld",
+ .cm_reg = CM_PLLD,
+ .a2w_reg = A2W_PLLD_DSI0,
+ .load_mask = CM_PLLD_LOADDSI0,
+ .hold_mask = CM_PLLD_HOLDDSI0,
+ .fixed_divider = 1),
+ [BCM2835_PLLD_DSI1] = REGISTER_PLL_DIV(
+ .name = "plld_dsi1",
+ .source_pll = "plld",
+ .cm_reg = CM_PLLD,
+ .a2w_reg = A2W_PLLD_DSI1,
+ .load_mask = CM_PLLD_LOADDSI1,
+ .hold_mask = CM_PLLD_HOLDDSI1,
+ .fixed_divider = 1),
+
+ /*
+ * PLLH is used to supply the pixel clock or the AUX clock for the
+ * TV encoder.
+ *
+ * It is in the HDMI power domain.
+ */
+ [BCM2835_PLLH] = REGISTER_PLL(
+ "pllh",
+ .cm_ctrl_reg = CM_PLLH,
+ .a2w_ctrl_reg = A2W_PLLH_CTRL,
+ .frac_reg = A2W_PLLH_FRAC,
+ .ana_reg_base = A2W_PLLH_ANA0,
+ .reference_enable_mask = A2W_XOSC_CTRL_PLLC_ENABLE,
+ .lock_mask = CM_LOCK_FLOCKH,
+
+ .ana = &bcm2835_ana_pllh,
+
+ .min_rate = 600000000u,
+ .max_rate = 3000000000u,
+ .max_fb_rate = BCM2835_MAX_FB_RATE),
+ [BCM2835_PLLH_RCAL] = REGISTER_PLL_DIV(
+ .name = "pllh_rcal",
+ .source_pll = "pllh",
+ .cm_reg = CM_PLLH,
+ .a2w_reg = A2W_PLLH_RCAL,
+ .load_mask = CM_PLLH_LOADRCAL,
+ .hold_mask = 0,
+ .fixed_divider = 10),
+ [BCM2835_PLLH_AUX] = REGISTER_PLL_DIV(
+ .name = "pllh_aux",
+ .source_pll = "pllh",
+ .cm_reg = CM_PLLH,
+ .a2w_reg = A2W_PLLH_AUX,
+ .load_mask = CM_PLLH_LOADAUX,
+ .hold_mask = 0,
+ .fixed_divider = 10),
+ [BCM2835_PLLH_PIX] = REGISTER_PLL_DIV(
+ .name = "pllh_pix",
+ .source_pll = "pllh",
+ .cm_reg = CM_PLLH,
+ .a2w_reg = A2W_PLLH_PIX,
+ .load_mask = CM_PLLH_LOADPIX,
+ .hold_mask = 0,
+ .fixed_divider = 10),
+
+ /* the clocks */
+
+ /* clocks with oscillator parent mux */
+
+ /* One Time Programmable Memory clock. Maximum 10Mhz. */
+ [BCM2835_CLOCK_OTP] = REGISTER_OSC_CLK(
+ .name = "otp",
+ .ctl_reg = CM_OTPCTL,
+ .div_reg = CM_OTPDIV,
+ .int_bits = 4,
+ .frac_bits = 0),
+ /*
+ * Used for a 1Mhz clock for the system clocksource, and also used
+ * bythe watchdog timer and the camera pulse generator.
+ */
+ [BCM2835_CLOCK_TIMER] = REGISTER_OSC_CLK(
+ .name = "timer",
+ .ctl_reg = CM_TIMERCTL,
+ .div_reg = CM_TIMERDIV,
+ .int_bits = 6,
+ .frac_bits = 12),
+ /*
+ * Clock for the temperature sensor.
+ * Generally run at 2Mhz, max 5Mhz.
+ */
+ [BCM2835_CLOCK_TSENS] = REGISTER_OSC_CLK(
+ .name = "tsens",
+ .ctl_reg = CM_TSENSCTL,
+ .div_reg = CM_TSENSDIV,
+ .int_bits = 5,
+ .frac_bits = 0),
+ [BCM2835_CLOCK_TEC] = REGISTER_OSC_CLK(
+ .name = "tec",
+ .ctl_reg = CM_TECCTL,
+ .div_reg = CM_TECDIV,
+ .int_bits = 6,
+ .frac_bits = 0),
+
+ /* clocks with vpu parent mux */
+ [BCM2835_CLOCK_H264] = REGISTER_VPU_CLK(
+ .name = "h264",
+ .ctl_reg = CM_H264CTL,
+ .div_reg = CM_H264DIV,
+ .int_bits = 4,
+ .frac_bits = 8),
+ [BCM2835_CLOCK_ISP] = REGISTER_VPU_CLK(
+ .name = "isp",
+ .ctl_reg = CM_ISPCTL,
+ .div_reg = CM_ISPDIV,
+ .int_bits = 4,
+ .frac_bits = 8),
+
+ /*
+ * Secondary SDRAM clock. Used for low-voltage modes when the PLL
+ * in the SDRAM controller can't be used.
+ */
+ [BCM2835_CLOCK_SDRAM] = REGISTER_VPU_CLK(
+ .name = "sdram",
+ .ctl_reg = CM_SDCCTL,
+ .div_reg = CM_SDCDIV,
+ .int_bits = 6,
+ .frac_bits = 0),
+ [BCM2835_CLOCK_V3D] = REGISTER_VPU_CLK(
+ .name = "v3d",
+ .ctl_reg = CM_V3DCTL,
+ .div_reg = CM_V3DDIV,
+ .int_bits = 4,
+ .frac_bits = 8),
+ /*
+ * VPU clock. This doesn't have an enable bit, since it drives
+ * the bus for everything else, and is special so it doesn't need
+ * to be gated for rate changes. It is also known as "clk_audio"
+ * in various hardware documentation.
+ */
+ [BCM2835_CLOCK_VPU] = REGISTER_VPU_CLK(
+ .name = "vpu",
+ .ctl_reg = CM_VPUCTL,
+ .div_reg = CM_VPUDIV,
+ .int_bits = 12,
+ .frac_bits = 8,
+ .is_vpu_clock = true),
+
+ /* clocks with per parent mux */
+ [BCM2835_CLOCK_AVEO] = REGISTER_PER_CLK(
+ .name = "aveo",
+ .ctl_reg = CM_AVEOCTL,
+ .div_reg = CM_AVEODIV,
+ .int_bits = 4,
+ .frac_bits = 0),
+ [BCM2835_CLOCK_CAM0] = REGISTER_PER_CLK(
+ .name = "cam0",
+ .ctl_reg = CM_CAM0CTL,
+ .div_reg = CM_CAM0DIV,
+ .int_bits = 4,
+ .frac_bits = 8),
+ [BCM2835_CLOCK_CAM1] = REGISTER_PER_CLK(
+ .name = "cam1",
+ .ctl_reg = CM_CAM1CTL,
+ .div_reg = CM_CAM1DIV,
+ .int_bits = 4,
+ .frac_bits = 8),
+ [BCM2835_CLOCK_DFT] = REGISTER_PER_CLK(
+ .name = "dft",
+ .ctl_reg = CM_DFTCTL,
+ .div_reg = CM_DFTDIV,
+ .int_bits = 5,
+ .frac_bits = 0),
+ [BCM2835_CLOCK_DPI] = REGISTER_PER_CLK(
+ .name = "dpi",
+ .ctl_reg = CM_DPICTL,
+ .div_reg = CM_DPIDIV,
+ .int_bits = 4,
+ .frac_bits = 8),
+
+ /* Arasan EMMC clock */
+ [BCM2835_CLOCK_EMMC] = REGISTER_PER_CLK(
+ .name = "emmc",
+ .ctl_reg = CM_EMMCCTL,
+ .div_reg = CM_EMMCDIV,
+ .int_bits = 4,
+ .frac_bits = 8),
+
+ /* General purpose (GPIO) clocks */
+ [BCM2835_CLOCK_GP0] = REGISTER_PER_CLK(
+ .name = "gp0",
+ .ctl_reg = CM_GP0CTL,
+ .div_reg = CM_GP0DIV,
+ .int_bits = 12,
+ .frac_bits = 12,
+ .is_mash_clock = true),
+ [BCM2835_CLOCK_GP1] = REGISTER_PER_CLK(
+ .name = "gp1",
+ .ctl_reg = CM_GP1CTL,
+ .div_reg = CM_GP1DIV,
+ .int_bits = 12,
+ .frac_bits = 12,
+ .is_mash_clock = true),
+ [BCM2835_CLOCK_GP2] = REGISTER_PER_CLK(
+ .name = "gp2",
+ .ctl_reg = CM_GP2CTL,
+ .div_reg = CM_GP2DIV,
+ .int_bits = 12,
+ .frac_bits = 12),
+
+ /* HDMI state machine */
+ [BCM2835_CLOCK_HSM] = REGISTER_PER_CLK(
+ .name = "hsm",
+ .ctl_reg = CM_HSMCTL,
+ .div_reg = CM_HSMDIV,
+ .int_bits = 4,
+ .frac_bits = 8),
+ [BCM2835_CLOCK_PCM] = REGISTER_PER_CLK(
+ .name = "pcm",
+ .ctl_reg = CM_PCMCTL,
+ .div_reg = CM_PCMDIV,
+ .int_bits = 12,
+ .frac_bits = 12,
+ .is_mash_clock = true),
+ [BCM2835_CLOCK_PWM] = REGISTER_PER_CLK(
+ .name = "pwm",
+ .ctl_reg = CM_PWMCTL,
+ .div_reg = CM_PWMDIV,
+ .int_bits = 12,
+ .frac_bits = 12,
+ .is_mash_clock = true),
+ [BCM2835_CLOCK_SLIM] = REGISTER_PER_CLK(
+ .name = "slim",
+ .ctl_reg = CM_SLIMCTL,
+ .div_reg = CM_SLIMDIV,
+ .int_bits = 12,
+ .frac_bits = 12,
+ .is_mash_clock = true),
+ [BCM2835_CLOCK_SMI] = REGISTER_PER_CLK(
+ .name = "smi",
+ .ctl_reg = CM_SMICTL,
+ .div_reg = CM_SMIDIV,
+ .int_bits = 4,
+ .frac_bits = 8),
+ [BCM2835_CLOCK_UART] = REGISTER_PER_CLK(
+ .name = "uart",
+ .ctl_reg = CM_UARTCTL,
+ .div_reg = CM_UARTDIV,
+ .int_bits = 10,
+ .frac_bits = 12),
+
+ /* TV encoder clock. Only operating frequency is 108Mhz. */
+ [BCM2835_CLOCK_VEC] = REGISTER_PER_CLK(
+ .name = "vec",
+ .ctl_reg = CM_VECCTL,
+ .div_reg = CM_VECDIV,
+ .int_bits = 4,
+ .frac_bits = 0),
+
+ /* dsi clocks */
+ [BCM2835_CLOCK_DSI0E] = REGISTER_PER_CLK(
+ .name = "dsi0e",
+ .ctl_reg = CM_DSI0ECTL,
+ .div_reg = CM_DSI0EDIV,
+ .int_bits = 4,
+ .frac_bits = 8),
+ [BCM2835_CLOCK_DSI1E] = REGISTER_PER_CLK(
+ .name = "dsi1e",
+ .ctl_reg = CM_DSI1ECTL,
+ .div_reg = CM_DSI1EDIV,
+ .int_bits = 4,
+ .frac_bits = 8),
+
+ /* the gates */
+
+ /*
+ * CM_PERIICTL (and CM_PERIACTL, CM_SYSCTL and CM_VPUCTL if
+ * you have the debug bit set in the power manager, which we
+ * don't bother exposing) are individual gates off of the
+ * non-stop vpu clock.
+ */
+ [BCM2835_CLOCK_PERI_IMAGE] = REGISTER_GATE(
+ .name = "peri_image",
+ .parent = "vpu",
+ .ctl_reg = CM_PERIICTL),
+};
+
static int bcm2835_clk_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct clk **clks;
struct bcm2835_cprman *cprman;
struct resource *res;
+ const struct bcm2835_clk_desc *desc;
+ const size_t asize = ARRAY_SIZE(clk_desc_array);
+ size_t i;
- cprman = devm_kzalloc(dev, sizeof(*cprman), GFP_KERNEL);
+ cprman = devm_kzalloc(dev,
+ sizeof(*cprman) + asize * sizeof(*clks),
+ GFP_KERNEL);
if (!cprman)
return -ENOMEM;
@@ -1525,80 +1819,15 @@
platform_set_drvdata(pdev, cprman);
- cprman->onecell.clk_num = BCM2835_CLOCK_COUNT;
+ cprman->onecell.clk_num = asize;
cprman->onecell.clks = cprman->clks;
clks = cprman->clks;
- clks[BCM2835_PLLA] = bcm2835_register_pll(cprman, &bcm2835_plla_data);
- clks[BCM2835_PLLB] = bcm2835_register_pll(cprman, &bcm2835_pllb_data);
- clks[BCM2835_PLLC] = bcm2835_register_pll(cprman, &bcm2835_pllc_data);
- clks[BCM2835_PLLD] = bcm2835_register_pll(cprman, &bcm2835_plld_data);
- clks[BCM2835_PLLH] = bcm2835_register_pll(cprman, &bcm2835_pllh_data);
-
- clks[BCM2835_PLLA_CORE] =
- bcm2835_register_pll_divider(cprman, &bcm2835_plla_core_data);
- clks[BCM2835_PLLA_PER] =
- bcm2835_register_pll_divider(cprman, &bcm2835_plla_per_data);
- clks[BCM2835_PLLC_CORE0] =
- bcm2835_register_pll_divider(cprman, &bcm2835_pllc_core0_data);
- clks[BCM2835_PLLC_CORE1] =
- bcm2835_register_pll_divider(cprman, &bcm2835_pllc_core1_data);
- clks[BCM2835_PLLC_CORE2] =
- bcm2835_register_pll_divider(cprman, &bcm2835_pllc_core2_data);
- clks[BCM2835_PLLC_PER] =
- bcm2835_register_pll_divider(cprman, &bcm2835_pllc_per_data);
- clks[BCM2835_PLLD_CORE] =
- bcm2835_register_pll_divider(cprman, &bcm2835_plld_core_data);
- clks[BCM2835_PLLD_PER] =
- bcm2835_register_pll_divider(cprman, &bcm2835_plld_per_data);
- clks[BCM2835_PLLH_RCAL] =
- bcm2835_register_pll_divider(cprman, &bcm2835_pllh_rcal_data);
- clks[BCM2835_PLLH_AUX] =
- bcm2835_register_pll_divider(cprman, &bcm2835_pllh_aux_data);
- clks[BCM2835_PLLH_PIX] =
- bcm2835_register_pll_divider(cprman, &bcm2835_pllh_pix_data);
-
- clks[BCM2835_CLOCK_TIMER] =
- bcm2835_register_clock(cprman, &bcm2835_clock_timer_data);
- clks[BCM2835_CLOCK_OTP] =
- bcm2835_register_clock(cprman, &bcm2835_clock_otp_data);
- clks[BCM2835_CLOCK_TSENS] =
- bcm2835_register_clock(cprman, &bcm2835_clock_tsens_data);
- clks[BCM2835_CLOCK_VPU] =
- bcm2835_register_clock(cprman, &bcm2835_clock_vpu_data);
- clks[BCM2835_CLOCK_V3D] =
- bcm2835_register_clock(cprman, &bcm2835_clock_v3d_data);
- clks[BCM2835_CLOCK_ISP] =
- bcm2835_register_clock(cprman, &bcm2835_clock_isp_data);
- clks[BCM2835_CLOCK_H264] =
- bcm2835_register_clock(cprman, &bcm2835_clock_h264_data);
- clks[BCM2835_CLOCK_V3D] =
- bcm2835_register_clock(cprman, &bcm2835_clock_v3d_data);
- clks[BCM2835_CLOCK_SDRAM] =
- bcm2835_register_clock(cprman, &bcm2835_clock_sdram_data);
- clks[BCM2835_CLOCK_UART] =
- bcm2835_register_clock(cprman, &bcm2835_clock_uart_data);
- clks[BCM2835_CLOCK_VEC] =
- bcm2835_register_clock(cprman, &bcm2835_clock_vec_data);
- clks[BCM2835_CLOCK_HSM] =
- bcm2835_register_clock(cprman, &bcm2835_clock_hsm_data);
- clks[BCM2835_CLOCK_EMMC] =
- bcm2835_register_clock(cprman, &bcm2835_clock_emmc_data);
-
- /*
- * CM_PERIICTL (and CM_PERIACTL, CM_SYSCTL and CM_VPUCTL if
- * you have the debug bit set in the power manager, which we
- * don't bother exposing) are individual gates off of the
- * non-stop vpu clock.
- */
- clks[BCM2835_CLOCK_PERI_IMAGE] =
- clk_register_gate(dev, "peri_image", "vpu",
- CLK_IGNORE_UNUSED | CLK_SET_RATE_GATE,
- cprman->regs + CM_PERIICTL, CM_GATE_BIT,
- 0, &cprman->regs_lock);
-
- clks[BCM2835_CLOCK_PWM] =
- bcm2835_register_clock(cprman, &bcm2835_clock_pwm_data);
+ for (i = 0; i < asize; i++) {
+ desc = &clk_desc_array[i];
+ if (desc->clk_register && desc->data)
+ clks[i] = desc->clk_register(cprman, desc->data);
+ }
return of_clk_add_provider(dev->of_node, of_clk_src_onecell_get,
&cprman->onecell);
diff --git a/drivers/clk/bcm/clk-kona-setup.c b/drivers/clk/bcm/clk-kona-setup.c
index deaa7f9..526b0b0 100644
--- a/drivers/clk/bcm/clk-kona-setup.c
+++ b/drivers/clk/bcm/clk-kona-setup.c
@@ -577,7 +577,8 @@
* selector is not required, but we allocate space for the
* array anyway to keep things simple.
*/
- parent_names = kmalloc(parent_count * sizeof(parent_names), GFP_KERNEL);
+ parent_names = kmalloc_array(parent_count, sizeof(*parent_names),
+ GFP_KERNEL);
if (!parent_names) {
pr_err("%s: error allocating %u parent names\n", __func__,
parent_count);
diff --git a/drivers/clk/clk-clps711x.c b/drivers/clk/clk-clps711x.c
index ff4ef4f..1f60b02 100644
--- a/drivers/clk/clk-clps711x.c
+++ b/drivers/clk/clk-clps711x.c
@@ -107,16 +107,15 @@
writel(tmp, base + CLPS711X_SYSCON1);
clps711x_clk->clks[CLPS711X_CLK_DUMMY] =
- clk_register_fixed_rate(NULL, "dummy", NULL, CLK_IS_ROOT, 0);
+ clk_register_fixed_rate(NULL, "dummy", NULL, 0, 0);
clps711x_clk->clks[CLPS711X_CLK_CPU] =
- clk_register_fixed_rate(NULL, "cpu", NULL, CLK_IS_ROOT, f_cpu);
+ clk_register_fixed_rate(NULL, "cpu", NULL, 0, f_cpu);
clps711x_clk->clks[CLPS711X_CLK_BUS] =
- clk_register_fixed_rate(NULL, "bus", NULL, CLK_IS_ROOT, f_bus);
+ clk_register_fixed_rate(NULL, "bus", NULL, 0, f_bus);
clps711x_clk->clks[CLPS711X_CLK_PLL] =
- clk_register_fixed_rate(NULL, "pll", NULL, CLK_IS_ROOT, f_pll);
+ clk_register_fixed_rate(NULL, "pll", NULL, 0, f_pll);
clps711x_clk->clks[CLPS711X_CLK_TIMERREF] =
- clk_register_fixed_rate(NULL, "timer_ref", NULL, CLK_IS_ROOT,
- f_tim);
+ clk_register_fixed_rate(NULL, "timer_ref", NULL, 0, f_tim);
clps711x_clk->clks[CLPS711X_CLK_TIMER1] =
clk_register_divider_table(NULL, "timer1", "timer_ref", 0,
base + CLPS711X_SYSCON1, 5, 1, 0,
@@ -126,10 +125,9 @@
base + CLPS711X_SYSCON1, 7, 1, 0,
timer_div_table, &clps711x_clk->lock);
clps711x_clk->clks[CLPS711X_CLK_PWM] =
- clk_register_fixed_rate(NULL, "pwm", NULL, CLK_IS_ROOT, f_pwm);
+ clk_register_fixed_rate(NULL, "pwm", NULL, 0, f_pwm);
clps711x_clk->clks[CLPS711X_CLK_SPIREF] =
- clk_register_fixed_rate(NULL, "spi_ref", NULL, CLK_IS_ROOT,
- f_spi);
+ clk_register_fixed_rate(NULL, "spi_ref", NULL, 0, f_spi);
clps711x_clk->clks[CLPS711X_CLK_SPI] =
clk_register_divider_table(NULL, "spi", "spi_ref", 0,
base + CLPS711X_SYSCON1, 16, 2, 0,
@@ -137,8 +135,7 @@
clps711x_clk->clks[CLPS711X_CLK_UART] =
clk_register_fixed_factor(NULL, "uart", "bus", 0, 1, 10);
clps711x_clk->clks[CLPS711X_CLK_TICK] =
- clk_register_fixed_rate(NULL, "tick", NULL, CLK_IS_ROOT, 64);
-
+ clk_register_fixed_rate(NULL, "tick", NULL, 0, 64);
for (i = 0; i < CLPS711X_CLK_MAX; i++)
if (IS_ERR(clps711x_clk->clks[i]))
pr_err("clk %i: register failed with %ld\n",
diff --git a/drivers/clk/clk-composite.c b/drivers/clk/clk-composite.c
index 1f903e1f8..00269de 100644
--- a/drivers/clk/clk-composite.c
+++ b/drivers/clk/clk-composite.c
@@ -151,6 +151,33 @@
return rate_ops->set_rate(rate_hw, rate, parent_rate);
}
+static int clk_composite_set_rate_and_parent(struct clk_hw *hw,
+ unsigned long rate,
+ unsigned long parent_rate,
+ u8 index)
+{
+ struct clk_composite *composite = to_clk_composite(hw);
+ const struct clk_ops *rate_ops = composite->rate_ops;
+ const struct clk_ops *mux_ops = composite->mux_ops;
+ struct clk_hw *rate_hw = composite->rate_hw;
+ struct clk_hw *mux_hw = composite->mux_hw;
+ unsigned long temp_rate;
+
+ __clk_hw_set_clk(rate_hw, hw);
+ __clk_hw_set_clk(mux_hw, hw);
+
+ temp_rate = rate_ops->recalc_rate(rate_hw, parent_rate);
+ if (temp_rate > rate) {
+ rate_ops->set_rate(rate_hw, rate, parent_rate);
+ mux_ops->set_parent(mux_hw, index);
+ } else {
+ mux_ops->set_parent(mux_hw, index);
+ rate_ops->set_rate(rate_hw, rate, parent_rate);
+ }
+
+ return 0;
+}
+
static int clk_composite_is_enabled(struct clk_hw *hw)
{
struct clk_composite *composite = to_clk_composite(hw);
@@ -184,17 +211,18 @@
gate_ops->disable(gate_hw);
}
-struct clk *clk_register_composite(struct device *dev, const char *name,
+struct clk_hw *clk_hw_register_composite(struct device *dev, const char *name,
const char * const *parent_names, int num_parents,
struct clk_hw *mux_hw, const struct clk_ops *mux_ops,
struct clk_hw *rate_hw, const struct clk_ops *rate_ops,
struct clk_hw *gate_hw, const struct clk_ops *gate_ops,
unsigned long flags)
{
- struct clk *clk;
+ struct clk_hw *hw;
struct clk_init_data init;
struct clk_composite *composite;
struct clk_ops *clk_composite_ops;
+ int ret;
composite = kzalloc(sizeof(*composite), GFP_KERNEL);
if (!composite)
@@ -204,12 +232,13 @@
init.flags = flags | CLK_IS_BASIC;
init.parent_names = parent_names;
init.num_parents = num_parents;
+ hw = &composite->hw;
clk_composite_ops = &composite->ops;
if (mux_hw && mux_ops) {
if (!mux_ops->get_parent) {
- clk = ERR_PTR(-EINVAL);
+ hw = ERR_PTR(-EINVAL);
goto err;
}
@@ -224,7 +253,7 @@
if (rate_hw && rate_ops) {
if (!rate_ops->recalc_rate) {
- clk = ERR_PTR(-EINVAL);
+ hw = ERR_PTR(-EINVAL);
goto err;
}
clk_composite_ops->recalc_rate = clk_composite_recalc_rate;
@@ -250,10 +279,16 @@
composite->rate_ops = rate_ops;
}
+ if (mux_hw && mux_ops && rate_hw && rate_ops) {
+ if (mux_ops->set_parent && rate_ops->set_rate)
+ clk_composite_ops->set_rate_and_parent =
+ clk_composite_set_rate_and_parent;
+ }
+
if (gate_hw && gate_ops) {
if (!gate_ops->is_enabled || !gate_ops->enable ||
!gate_ops->disable) {
- clk = ERR_PTR(-EINVAL);
+ hw = ERR_PTR(-EINVAL);
goto err;
}
@@ -267,22 +302,56 @@
init.ops = clk_composite_ops;
composite->hw.init = &init;
- clk = clk_register(dev, &composite->hw);
- if (IS_ERR(clk))
+ ret = clk_hw_register(dev, hw);
+ if (ret) {
+ hw = ERR_PTR(ret);
goto err;
+ }
if (composite->mux_hw)
- composite->mux_hw->clk = clk;
+ composite->mux_hw->clk = hw->clk;
if (composite->rate_hw)
- composite->rate_hw->clk = clk;
+ composite->rate_hw->clk = hw->clk;
if (composite->gate_hw)
- composite->gate_hw->clk = clk;
+ composite->gate_hw->clk = hw->clk;
- return clk;
+ return hw;
err:
kfree(composite);
- return clk;
+ return hw;
+}
+
+struct clk *clk_register_composite(struct device *dev, const char *name,
+ const char * const *parent_names, int num_parents,
+ struct clk_hw *mux_hw, const struct clk_ops *mux_ops,
+ struct clk_hw *rate_hw, const struct clk_ops *rate_ops,
+ struct clk_hw *gate_hw, const struct clk_ops *gate_ops,
+ unsigned long flags)
+{
+ struct clk_hw *hw;
+
+ hw = clk_hw_register_composite(dev, name, parent_names, num_parents,
+ mux_hw, mux_ops, rate_hw, rate_ops, gate_hw, gate_ops,
+ flags);
+ if (IS_ERR(hw))
+ return ERR_CAST(hw);
+ return hw->clk;
+}
+
+void clk_unregister_composite(struct clk *clk)
+{
+ struct clk_composite *composite;
+ struct clk_hw *hw;
+
+ hw = __clk_get_hw(clk);
+ if (!hw)
+ return;
+
+ composite = to_clk_composite(hw);
+
+ clk_unregister(clk);
+ kfree(composite);
}
diff --git a/drivers/clk/clk-divider.c b/drivers/clk/clk-divider.c
index 00e035b..a0f55bc 100644
--- a/drivers/clk/clk-divider.c
+++ b/drivers/clk/clk-divider.c
@@ -426,15 +426,16 @@
};
EXPORT_SYMBOL_GPL(clk_divider_ro_ops);
-static struct clk *_register_divider(struct device *dev, const char *name,
+static struct clk_hw *_register_divider(struct device *dev, const char *name,
const char *parent_name, unsigned long flags,
void __iomem *reg, u8 shift, u8 width,
u8 clk_divider_flags, const struct clk_div_table *table,
spinlock_t *lock)
{
struct clk_divider *div;
- struct clk *clk;
+ struct clk_hw *hw;
struct clk_init_data init;
+ int ret;
if (clk_divider_flags & CLK_DIVIDER_HIWORD_MASK) {
if (width + shift > 16) {
@@ -467,12 +468,14 @@
div->table = table;
/* register the clock */
- clk = clk_register(dev, &div->hw);
-
- if (IS_ERR(clk))
+ hw = &div->hw;
+ ret = clk_hw_register(dev, hw);
+ if (ret) {
kfree(div);
+ hw = ERR_PTR(ret);
+ }
- return clk;
+ return hw;
}
/**
@@ -492,10 +495,37 @@
void __iomem *reg, u8 shift, u8 width,
u8 clk_divider_flags, spinlock_t *lock)
{
+ struct clk_hw *hw;
+
+ hw = _register_divider(dev, name, parent_name, flags, reg, shift,
+ width, clk_divider_flags, NULL, lock);
+ if (IS_ERR(hw))
+ return ERR_CAST(hw);
+ return hw->clk;
+}
+EXPORT_SYMBOL_GPL(clk_register_divider);
+
+/**
+ * clk_hw_register_divider - register a divider clock with the clock framework
+ * @dev: device registering this clock
+ * @name: name of this clock
+ * @parent_name: name of clock's parent
+ * @flags: framework-specific flags
+ * @reg: register address to adjust divider
+ * @shift: number of bits to shift the bitfield
+ * @width: width of the bitfield
+ * @clk_divider_flags: divider-specific flags for this clock
+ * @lock: shared register lock for this clock
+ */
+struct clk_hw *clk_hw_register_divider(struct device *dev, const char *name,
+ const char *parent_name, unsigned long flags,
+ void __iomem *reg, u8 shift, u8 width,
+ u8 clk_divider_flags, spinlock_t *lock)
+{
return _register_divider(dev, name, parent_name, flags, reg, shift,
width, clk_divider_flags, NULL, lock);
}
-EXPORT_SYMBOL_GPL(clk_register_divider);
+EXPORT_SYMBOL_GPL(clk_hw_register_divider);
/**
* clk_register_divider_table - register a table based divider clock with
@@ -517,10 +547,40 @@
u8 clk_divider_flags, const struct clk_div_table *table,
spinlock_t *lock)
{
+ struct clk_hw *hw;
+
+ hw = _register_divider(dev, name, parent_name, flags, reg, shift,
+ width, clk_divider_flags, table, lock);
+ if (IS_ERR(hw))
+ return ERR_CAST(hw);
+ return hw->clk;
+}
+EXPORT_SYMBOL_GPL(clk_register_divider_table);
+
+/**
+ * clk_hw_register_divider_table - register a table based divider clock with
+ * the clock framework
+ * @dev: device registering this clock
+ * @name: name of this clock
+ * @parent_name: name of clock's parent
+ * @flags: framework-specific flags
+ * @reg: register address to adjust divider
+ * @shift: number of bits to shift the bitfield
+ * @width: width of the bitfield
+ * @clk_divider_flags: divider-specific flags for this clock
+ * @table: array of divider/value pairs ending with a div set to 0
+ * @lock: shared register lock for this clock
+ */
+struct clk_hw *clk_hw_register_divider_table(struct device *dev,
+ const char *name, const char *parent_name, unsigned long flags,
+ void __iomem *reg, u8 shift, u8 width,
+ u8 clk_divider_flags, const struct clk_div_table *table,
+ spinlock_t *lock)
+{
return _register_divider(dev, name, parent_name, flags, reg, shift,
width, clk_divider_flags, table, lock);
}
-EXPORT_SYMBOL_GPL(clk_register_divider_table);
+EXPORT_SYMBOL_GPL(clk_hw_register_divider_table);
void clk_unregister_divider(struct clk *clk)
{
@@ -537,3 +597,18 @@
kfree(div);
}
EXPORT_SYMBOL_GPL(clk_unregister_divider);
+
+/**
+ * clk_hw_unregister_divider - unregister a clk divider
+ * @hw: hardware-specific clock data to unregister
+ */
+void clk_hw_unregister_divider(struct clk_hw *hw)
+{
+ struct clk_divider *div;
+
+ div = to_clk_divider(hw);
+
+ clk_hw_unregister(hw);
+ kfree(div);
+}
+EXPORT_SYMBOL_GPL(clk_hw_unregister_divider);
diff --git a/drivers/clk/clk-fixed-factor.c b/drivers/clk/clk-fixed-factor.c
index 053448e..75cd6c7 100644
--- a/drivers/clk/clk-fixed-factor.c
+++ b/drivers/clk/clk-fixed-factor.c
@@ -68,13 +68,14 @@
};
EXPORT_SYMBOL_GPL(clk_fixed_factor_ops);
-struct clk *clk_register_fixed_factor(struct device *dev, const char *name,
- const char *parent_name, unsigned long flags,
+struct clk_hw *clk_hw_register_fixed_factor(struct device *dev,
+ const char *name, const char *parent_name, unsigned long flags,
unsigned int mult, unsigned int div)
{
struct clk_fixed_factor *fix;
struct clk_init_data init;
- struct clk *clk;
+ struct clk_hw *hw;
+ int ret;
fix = kmalloc(sizeof(*fix), GFP_KERNEL);
if (!fix)
@@ -91,12 +92,28 @@
init.parent_names = &parent_name;
init.num_parents = 1;
- clk = clk_register(dev, &fix->hw);
-
- if (IS_ERR(clk))
+ hw = &fix->hw;
+ ret = clk_hw_register(dev, hw);
+ if (ret) {
kfree(fix);
+ hw = ERR_PTR(ret);
+ }
- return clk;
+ return hw;
+}
+EXPORT_SYMBOL_GPL(clk_hw_register_fixed_factor);
+
+struct clk *clk_register_fixed_factor(struct device *dev, const char *name,
+ const char *parent_name, unsigned long flags,
+ unsigned int mult, unsigned int div)
+{
+ struct clk_hw *hw;
+
+ hw = clk_hw_register_fixed_factor(dev, name, parent_name, flags, mult,
+ div);
+ if (IS_ERR(hw))
+ return ERR_CAST(hw);
+ return hw->clk;
}
EXPORT_SYMBOL_GPL(clk_register_fixed_factor);
@@ -113,6 +130,17 @@
}
EXPORT_SYMBOL_GPL(clk_unregister_fixed_factor);
+void clk_hw_unregister_fixed_factor(struct clk_hw *hw)
+{
+ struct clk_fixed_factor *fix;
+
+ fix = to_clk_fixed_factor(hw);
+
+ clk_hw_unregister(hw);
+ kfree(fix);
+}
+EXPORT_SYMBOL_GPL(clk_hw_unregister_fixed_factor);
+
#ifdef CONFIG_OF
/**
* of_fixed_factor_clk_setup() - Setup function for simple fixed factor clock
diff --git a/drivers/clk/clk-fixed-rate.c b/drivers/clk/clk-fixed-rate.c
index cd9dc92..8e4453e 100644
--- a/drivers/clk/clk-fixed-rate.c
+++ b/drivers/clk/clk-fixed-rate.c
@@ -45,8 +45,8 @@
EXPORT_SYMBOL_GPL(clk_fixed_rate_ops);
/**
- * clk_register_fixed_rate_with_accuracy - register fixed-rate clock with the
- * clock framework
+ * clk_hw_register_fixed_rate_with_accuracy - register fixed-rate clock with
+ * the clock framework
* @dev: device that is registering this clock
* @name: name of this clock
* @parent_name: name of clock's parent
@@ -54,13 +54,14 @@
* @fixed_rate: non-adjustable clock rate
* @fixed_accuracy: non-adjustable clock rate
*/
-struct clk *clk_register_fixed_rate_with_accuracy(struct device *dev,
+struct clk_hw *clk_hw_register_fixed_rate_with_accuracy(struct device *dev,
const char *name, const char *parent_name, unsigned long flags,
unsigned long fixed_rate, unsigned long fixed_accuracy)
{
struct clk_fixed_rate *fixed;
- struct clk *clk;
+ struct clk_hw *hw;
struct clk_init_data init;
+ int ret;
/* allocate fixed-rate clock */
fixed = kzalloc(sizeof(*fixed), GFP_KERNEL);
@@ -79,22 +80,49 @@
fixed->hw.init = &init;
/* register the clock */
- clk = clk_register(dev, &fixed->hw);
- if (IS_ERR(clk))
+ hw = &fixed->hw;
+ ret = clk_hw_register(dev, hw);
+ if (ret) {
kfree(fixed);
+ hw = ERR_PTR(ret);
+ }
- return clk;
+ return hw;
+}
+EXPORT_SYMBOL_GPL(clk_hw_register_fixed_rate_with_accuracy);
+
+struct clk *clk_register_fixed_rate_with_accuracy(struct device *dev,
+ const char *name, const char *parent_name, unsigned long flags,
+ unsigned long fixed_rate, unsigned long fixed_accuracy)
+{
+ struct clk_hw *hw;
+
+ hw = clk_hw_register_fixed_rate_with_accuracy(dev, name, parent_name,
+ flags, fixed_rate, fixed_accuracy);
+ if (IS_ERR(hw))
+ return ERR_CAST(hw);
+ return hw->clk;
}
EXPORT_SYMBOL_GPL(clk_register_fixed_rate_with_accuracy);
/**
- * clk_register_fixed_rate - register fixed-rate clock with the clock framework
+ * clk_hw_register_fixed_rate - register fixed-rate clock with the clock
+ * framework
* @dev: device that is registering this clock
* @name: name of this clock
* @parent_name: name of clock's parent
* @flags: framework-specific flags
* @fixed_rate: non-adjustable clock rate
*/
+struct clk_hw *clk_hw_register_fixed_rate(struct device *dev, const char *name,
+ const char *parent_name, unsigned long flags,
+ unsigned long fixed_rate)
+{
+ return clk_hw_register_fixed_rate_with_accuracy(dev, name, parent_name,
+ flags, fixed_rate, 0);
+}
+EXPORT_SYMBOL_GPL(clk_hw_register_fixed_rate);
+
struct clk *clk_register_fixed_rate(struct device *dev, const char *name,
const char *parent_name, unsigned long flags,
unsigned long fixed_rate)
diff --git a/drivers/clk/clk-fractional-divider.c b/drivers/clk/clk-fractional-divider.c
index 1abcd76..aab9046 100644
--- a/drivers/clk/clk-fractional-divider.c
+++ b/drivers/clk/clk-fractional-divider.c
@@ -116,14 +116,15 @@
};
EXPORT_SYMBOL_GPL(clk_fractional_divider_ops);
-struct clk *clk_register_fractional_divider(struct device *dev,
+struct clk_hw *clk_hw_register_fractional_divider(struct device *dev,
const char *name, const char *parent_name, unsigned long flags,
void __iomem *reg, u8 mshift, u8 mwidth, u8 nshift, u8 nwidth,
u8 clk_divider_flags, spinlock_t *lock)
{
struct clk_fractional_divider *fd;
struct clk_init_data init;
- struct clk *clk;
+ struct clk_hw *hw;
+ int ret;
fd = kzalloc(sizeof(*fd), GFP_KERNEL);
if (!fd)
@@ -146,10 +147,39 @@
fd->lock = lock;
fd->hw.init = &init;
- clk = clk_register(dev, &fd->hw);
- if (IS_ERR(clk))
+ hw = &fd->hw;
+ ret = clk_hw_register(dev, hw);
+ if (ret) {
kfree(fd);
+ hw = ERR_PTR(ret);
+ }
- return clk;
+ return hw;
+}
+EXPORT_SYMBOL_GPL(clk_hw_register_fractional_divider);
+
+struct clk *clk_register_fractional_divider(struct device *dev,
+ const char *name, const char *parent_name, unsigned long flags,
+ void __iomem *reg, u8 mshift, u8 mwidth, u8 nshift, u8 nwidth,
+ u8 clk_divider_flags, spinlock_t *lock)
+{
+ struct clk_hw *hw;
+
+ hw = clk_hw_register_fractional_divider(dev, name, parent_name, flags,
+ reg, mshift, mwidth, nshift, nwidth, clk_divider_flags,
+ lock);
+ if (IS_ERR(hw))
+ return ERR_CAST(hw);
+ return hw->clk;
}
EXPORT_SYMBOL_GPL(clk_register_fractional_divider);
+
+void clk_hw_unregister_fractional_divider(struct clk_hw *hw)
+{
+ struct clk_fractional_divider *fd;
+
+ fd = to_clk_fd(hw);
+
+ clk_hw_unregister(hw);
+ kfree(fd);
+}
diff --git a/drivers/clk/clk-gate.c b/drivers/clk/clk-gate.c
index d0d8ec8..4e691e3 100644
--- a/drivers/clk/clk-gate.c
+++ b/drivers/clk/clk-gate.c
@@ -110,7 +110,7 @@
EXPORT_SYMBOL_GPL(clk_gate_ops);
/**
- * clk_register_gate - register a gate clock with the clock framework
+ * clk_hw_register_gate - register a gate clock with the clock framework
* @dev: device that is registering this clock
* @name: name of this clock
* @parent_name: name of this clock's parent
@@ -120,14 +120,15 @@
* @clk_gate_flags: gate-specific flags for this clock
* @lock: shared register lock for this clock
*/
-struct clk *clk_register_gate(struct device *dev, const char *name,
+struct clk_hw *clk_hw_register_gate(struct device *dev, const char *name,
const char *parent_name, unsigned long flags,
void __iomem *reg, u8 bit_idx,
u8 clk_gate_flags, spinlock_t *lock)
{
struct clk_gate *gate;
- struct clk *clk;
+ struct clk_hw *hw;
struct clk_init_data init;
+ int ret;
if (clk_gate_flags & CLK_GATE_HIWORD_MASK) {
if (bit_idx > 15) {
@@ -154,12 +155,29 @@
gate->lock = lock;
gate->hw.init = &init;
- clk = clk_register(dev, &gate->hw);
-
- if (IS_ERR(clk))
+ hw = &gate->hw;
+ ret = clk_hw_register(dev, hw);
+ if (ret) {
kfree(gate);
+ hw = ERR_PTR(ret);
+ }
- return clk;
+ return hw;
+}
+EXPORT_SYMBOL_GPL(clk_hw_register_gate);
+
+struct clk *clk_register_gate(struct device *dev, const char *name,
+ const char *parent_name, unsigned long flags,
+ void __iomem *reg, u8 bit_idx,
+ u8 clk_gate_flags, spinlock_t *lock)
+{
+ struct clk_hw *hw;
+
+ hw = clk_hw_register_gate(dev, name, parent_name, flags, reg,
+ bit_idx, clk_gate_flags, lock);
+ if (IS_ERR(hw))
+ return ERR_CAST(hw);
+ return hw->clk;
}
EXPORT_SYMBOL_GPL(clk_register_gate);
@@ -178,3 +196,14 @@
kfree(gate);
}
EXPORT_SYMBOL_GPL(clk_unregister_gate);
+
+void clk_hw_unregister_gate(struct clk_hw *hw)
+{
+ struct clk_gate *gate;
+
+ gate = to_clk_gate(hw);
+
+ clk_hw_unregister(hw);
+ kfree(gate);
+}
+EXPORT_SYMBOL_GPL(clk_hw_unregister_gate);
diff --git a/drivers/clk/clk-gpio.c b/drivers/clk/clk-gpio.c
index 08f65ac..86b2457 100644
--- a/drivers/clk/clk-gpio.c
+++ b/drivers/clk/clk-gpio.c
@@ -94,13 +94,13 @@
};
EXPORT_SYMBOL_GPL(clk_gpio_mux_ops);
-static struct clk *clk_register_gpio(struct device *dev, const char *name,
+static struct clk_hw *clk_register_gpio(struct device *dev, const char *name,
const char * const *parent_names, u8 num_parents, unsigned gpio,
bool active_low, unsigned long flags,
const struct clk_ops *clk_gpio_ops)
{
struct clk_gpio *clk_gpio;
- struct clk *clk;
+ struct clk_hw *hw;
struct clk_init_data init = {};
unsigned long gpio_flags;
int err;
@@ -141,24 +141,26 @@
clk_gpio->gpiod = gpio_to_desc(gpio);
clk_gpio->hw.init = &init;
+ hw = &clk_gpio->hw;
if (dev)
- clk = devm_clk_register(dev, &clk_gpio->hw);
+ err = devm_clk_hw_register(dev, hw);
else
- clk = clk_register(NULL, &clk_gpio->hw);
+ err = clk_hw_register(NULL, hw);
- if (!IS_ERR(clk))
- return clk;
+ if (!err)
+ return hw;
if (!dev) {
gpiod_put(clk_gpio->gpiod);
kfree(clk_gpio);
}
- return clk;
+ return ERR_PTR(err);
}
/**
- * clk_register_gpio_gate - register a gpio clock gate with the clock framework
+ * clk_hw_register_gpio_gate - register a gpio clock gate with the clock
+ * framework
* @dev: device that is registering this clock
* @name: name of this clock
* @parent_name: name of this clock's parent
@@ -166,7 +168,7 @@
* @active_low: true if gpio should be set to 0 to enable clock
* @flags: clock flags
*/
-struct clk *clk_register_gpio_gate(struct device *dev, const char *name,
+struct clk_hw *clk_hw_register_gpio_gate(struct device *dev, const char *name,
const char *parent_name, unsigned gpio, bool active_low,
unsigned long flags)
{
@@ -175,10 +177,24 @@
(parent_name ? 1 : 0), gpio, active_low, flags,
&clk_gpio_gate_ops);
}
+EXPORT_SYMBOL_GPL(clk_hw_register_gpio_gate);
+
+struct clk *clk_register_gpio_gate(struct device *dev, const char *name,
+ const char *parent_name, unsigned gpio, bool active_low,
+ unsigned long flags)
+{
+ struct clk_hw *hw;
+
+ hw = clk_hw_register_gpio_gate(dev, name, parent_name, gpio, active_low,
+ flags);
+ if (IS_ERR(hw))
+ return ERR_CAST(hw);
+ return hw->clk;
+}
EXPORT_SYMBOL_GPL(clk_register_gpio_gate);
/**
- * clk_register_gpio_mux - register a gpio clock mux with the clock framework
+ * clk_hw_register_gpio_mux - register a gpio clock mux with the clock framework
* @dev: device that is registering this clock
* @name: name of this clock
* @parent_names: names of this clock's parents
@@ -187,7 +203,7 @@
* @active_low: true if gpio should be set to 0 to enable clock
* @flags: clock flags
*/
-struct clk *clk_register_gpio_mux(struct device *dev, const char *name,
+struct clk_hw *clk_hw_register_gpio_mux(struct device *dev, const char *name,
const char * const *parent_names, u8 num_parents, unsigned gpio,
bool active_low, unsigned long flags)
{
@@ -199,6 +215,20 @@
return clk_register_gpio(dev, name, parent_names, num_parents,
gpio, active_low, flags, &clk_gpio_mux_ops);
}
+EXPORT_SYMBOL_GPL(clk_hw_register_gpio_mux);
+
+struct clk *clk_register_gpio_mux(struct device *dev, const char *name,
+ const char * const *parent_names, u8 num_parents, unsigned gpio,
+ bool active_low, unsigned long flags)
+{
+ struct clk_hw *hw;
+
+ hw = clk_hw_register_gpio_mux(dev, name, parent_names, num_parents,
+ gpio, active_low, flags);
+ if (IS_ERR(hw))
+ return ERR_CAST(hw);
+ return hw->clk;
+}
EXPORT_SYMBOL_GPL(clk_register_gpio_mux);
static int gpio_clk_driver_probe(struct platform_device *pdev)
diff --git a/drivers/clk/clk-ls1x.c b/drivers/clk/clk-ls1x.c
index d4c6198..5097831 100644
--- a/drivers/clk/clk-ls1x.c
+++ b/drivers/clk/clk-ls1x.c
@@ -88,8 +88,7 @@
{
struct clk *clk;
- clk = clk_register_fixed_rate(NULL, "osc_33m_clk", NULL, CLK_IS_ROOT,
- OSC);
+ clk = clk_register_fixed_rate(NULL, "osc_33m_clk", NULL, 0, OSC);
clk_register_clkdev(clk, "osc_33m_clk", NULL);
/* clock derived from 33 MHz OSC clk */
diff --git a/drivers/clk/clk-mux.c b/drivers/clk/clk-mux.c
index 252188f..16a3d57 100644
--- a/drivers/clk/clk-mux.c
+++ b/drivers/clk/clk-mux.c
@@ -113,16 +113,17 @@
};
EXPORT_SYMBOL_GPL(clk_mux_ro_ops);
-struct clk *clk_register_mux_table(struct device *dev, const char *name,
+struct clk_hw *clk_hw_register_mux_table(struct device *dev, const char *name,
const char * const *parent_names, u8 num_parents,
unsigned long flags,
void __iomem *reg, u8 shift, u32 mask,
u8 clk_mux_flags, u32 *table, spinlock_t *lock)
{
struct clk_mux *mux;
- struct clk *clk;
+ struct clk_hw *hw;
struct clk_init_data init;
u8 width = 0;
+ int ret;
if (clk_mux_flags & CLK_MUX_HIWORD_MASK) {
width = fls(mask) - ffs(mask) + 1;
@@ -157,12 +158,31 @@
mux->table = table;
mux->hw.init = &init;
- clk = clk_register(dev, &mux->hw);
-
- if (IS_ERR(clk))
+ hw = &mux->hw;
+ ret = clk_hw_register(dev, hw);
+ if (ret) {
kfree(mux);
+ hw = ERR_PTR(ret);
+ }
- return clk;
+ return hw;
+}
+EXPORT_SYMBOL_GPL(clk_hw_register_mux_table);
+
+struct clk *clk_register_mux_table(struct device *dev, const char *name,
+ const char * const *parent_names, u8 num_parents,
+ unsigned long flags,
+ void __iomem *reg, u8 shift, u32 mask,
+ u8 clk_mux_flags, u32 *table, spinlock_t *lock)
+{
+ struct clk_hw *hw;
+
+ hw = clk_hw_register_mux_table(dev, name, parent_names, num_parents,
+ flags, reg, shift, mask, clk_mux_flags,
+ table, lock);
+ if (IS_ERR(hw))
+ return ERR_CAST(hw);
+ return hw->clk;
}
EXPORT_SYMBOL_GPL(clk_register_mux_table);
@@ -180,6 +200,20 @@
}
EXPORT_SYMBOL_GPL(clk_register_mux);
+struct clk_hw *clk_hw_register_mux(struct device *dev, const char *name,
+ const char * const *parent_names, u8 num_parents,
+ unsigned long flags,
+ void __iomem *reg, u8 shift, u8 width,
+ u8 clk_mux_flags, spinlock_t *lock)
+{
+ u32 mask = BIT(width) - 1;
+
+ return clk_hw_register_mux_table(dev, name, parent_names, num_parents,
+ flags, reg, shift, mask, clk_mux_flags,
+ NULL, lock);
+}
+EXPORT_SYMBOL_GPL(clk_hw_register_mux);
+
void clk_unregister_mux(struct clk *clk)
{
struct clk_mux *mux;
@@ -195,3 +229,14 @@
kfree(mux);
}
EXPORT_SYMBOL_GPL(clk_unregister_mux);
+
+void clk_hw_unregister_mux(struct clk_hw *hw)
+{
+ struct clk_mux *mux;
+
+ mux = to_clk_mux(hw);
+
+ clk_hw_unregister(hw);
+ kfree(mux);
+}
+EXPORT_SYMBOL_GPL(clk_hw_unregister_mux);
diff --git a/drivers/clk/clk-nspire.c b/drivers/clk/clk-nspire.c
index a378db7..64f196a 100644
--- a/drivers/clk/clk-nspire.c
+++ b/drivers/clk/clk-nspire.c
@@ -125,8 +125,7 @@
of_property_read_string(node, "clock-output-names", &clk_name);
- clk = clk_register_fixed_rate(NULL, clk_name, NULL, CLK_IS_ROOT,
- info.base_clock);
+ clk = clk_register_fixed_rate(NULL, clk_name, NULL, 0, info.base_clock);
if (!IS_ERR(clk))
of_clk_add_provider(node, of_clk_src_simple_get, clk);
else
diff --git a/drivers/clk/clk-oxnas.c b/drivers/clk/clk-oxnas.c
new file mode 100644
index 0000000..efba7d4
--- /dev/null
+++ b/drivers/clk/clk-oxnas.c
@@ -0,0 +1,195 @@
+/*
+ * Copyright (C) 2010 Broadcom
+ * Copyright (C) 2012 Stephen Warren
+ * Copyright (C) 2016 Neil Armstrong <narmstrong@baylibre.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/clk-provider.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/stringify.h>
+#include <linux/regmap.h>
+#include <linux/mfd/syscon.h>
+
+/* Standard regmap gate clocks */
+struct clk_oxnas {
+ struct clk_hw hw;
+ signed char bit;
+ struct regmap *regmap;
+};
+
+/* Regmap offsets */
+#define CLK_STAT_REGOFFSET 0x24
+#define CLK_SET_REGOFFSET 0x2c
+#define CLK_CLR_REGOFFSET 0x30
+
+static inline struct clk_oxnas *to_clk_oxnas(struct clk_hw *hw)
+{
+ return container_of(hw, struct clk_oxnas, hw);
+}
+
+static int oxnas_clk_is_enabled(struct clk_hw *hw)
+{
+ struct clk_oxnas *std = to_clk_oxnas(hw);
+ int ret;
+ unsigned int val;
+
+ ret = regmap_read(std->regmap, CLK_STAT_REGOFFSET, &val);
+ if (ret < 0)
+ return ret;
+
+ return val & BIT(std->bit);
+}
+
+static int oxnas_clk_enable(struct clk_hw *hw)
+{
+ struct clk_oxnas *std = to_clk_oxnas(hw);
+
+ regmap_write(std->regmap, CLK_SET_REGOFFSET, BIT(std->bit));
+
+ return 0;
+}
+
+static void oxnas_clk_disable(struct clk_hw *hw)
+{
+ struct clk_oxnas *std = to_clk_oxnas(hw);
+
+ regmap_write(std->regmap, CLK_CLR_REGOFFSET, BIT(std->bit));
+}
+
+static const struct clk_ops oxnas_clk_ops = {
+ .enable = oxnas_clk_enable,
+ .disable = oxnas_clk_disable,
+ .is_enabled = oxnas_clk_is_enabled,
+};
+
+static const char *const oxnas_clk_parents[] = {
+ "oscillator",
+};
+
+static const char *const eth_parents[] = {
+ "gmacclk",
+};
+
+#define DECLARE_STD_CLKP(__clk, __parent) \
+static const struct clk_init_data clk_##__clk##_init = { \
+ .name = __stringify(__clk), \
+ .ops = &oxnas_clk_ops, \
+ .parent_names = __parent, \
+ .num_parents = ARRAY_SIZE(__parent), \
+}
+
+#define DECLARE_STD_CLK(__clk) DECLARE_STD_CLKP(__clk, oxnas_clk_parents)
+
+/* Hardware Bit - Clock association */
+struct clk_oxnas_init_data {
+ unsigned long bit;
+ const struct clk_init_data *clk_init;
+};
+
+/* Clk init data declaration */
+DECLARE_STD_CLK(leon);
+DECLARE_STD_CLK(dma_sgdma);
+DECLARE_STD_CLK(cipher);
+DECLARE_STD_CLK(sata);
+DECLARE_STD_CLK(audio);
+DECLARE_STD_CLK(usbmph);
+DECLARE_STD_CLKP(etha, eth_parents);
+DECLARE_STD_CLK(pciea);
+DECLARE_STD_CLK(nand);
+
+/* Table index is clock indice */
+static const struct clk_oxnas_init_data clk_oxnas_init[] = {
+ [0] = {0, &clk_leon_init},
+ [1] = {1, &clk_dma_sgdma_init},
+ [2] = {2, &clk_cipher_init},
+ /* Skip & Do not touch to DDR clock */
+ [3] = {4, &clk_sata_init},
+ [4] = {5, &clk_audio_init},
+ [5] = {6, &clk_usbmph_init},
+ [6] = {7, &clk_etha_init},
+ [7] = {8, &clk_pciea_init},
+ [8] = {9, &clk_nand_init},
+};
+
+struct clk_oxnas_data {
+ struct clk_oxnas clk_oxnas[ARRAY_SIZE(clk_oxnas_init)];
+ struct clk_onecell_data onecell_data[ARRAY_SIZE(clk_oxnas_init)];
+ struct clk *clks[ARRAY_SIZE(clk_oxnas_init)];
+};
+
+static int oxnas_stdclk_probe(struct platform_device *pdev)
+{
+ struct device_node *np = pdev->dev.of_node;
+ struct clk_oxnas_data *clk_oxnas;
+ struct regmap *regmap;
+ int i;
+
+ clk_oxnas = devm_kzalloc(&pdev->dev, sizeof(*clk_oxnas), GFP_KERNEL);
+ if (!clk_oxnas)
+ return -ENOMEM;
+
+ regmap = syscon_node_to_regmap(of_get_parent(np));
+ if (!regmap) {
+ dev_err(&pdev->dev, "failed to have parent regmap\n");
+ return -EINVAL;
+ }
+
+ for (i = 0; i < ARRAY_SIZE(clk_oxnas_init); i++) {
+ struct clk_oxnas *_clk;
+
+ _clk = &clk_oxnas->clk_oxnas[i];
+ _clk->bit = clk_oxnas_init[i].bit;
+ _clk->hw.init = clk_oxnas_init[i].clk_init;
+ _clk->regmap = regmap;
+
+ clk_oxnas->clks[i] =
+ devm_clk_register(&pdev->dev, &_clk->hw);
+ if (WARN_ON(IS_ERR(clk_oxnas->clks[i])))
+ return PTR_ERR(clk_oxnas->clks[i]);
+ }
+
+ clk_oxnas->onecell_data->clks = clk_oxnas->clks;
+ clk_oxnas->onecell_data->clk_num = ARRAY_SIZE(clk_oxnas_init);
+
+ return of_clk_add_provider(np, of_clk_src_onecell_get,
+ clk_oxnas->onecell_data);
+}
+
+static int oxnas_stdclk_remove(struct platform_device *pdev)
+{
+ of_clk_del_provider(pdev->dev.of_node);
+
+ return 0;
+}
+
+static const struct of_device_id oxnas_stdclk_dt_ids[] = {
+ { .compatible = "oxsemi,ox810se-stdclk" },
+ { }
+};
+MODULE_DEVICE_TABLE(of, oxnas_stdclk_dt_ids);
+
+static struct platform_driver oxnas_stdclk_driver = {
+ .probe = oxnas_stdclk_probe,
+ .remove = oxnas_stdclk_remove,
+ .driver = {
+ .name = "oxnas-stdclk",
+ .of_match_table = oxnas_stdclk_dt_ids,
+ },
+};
+
+module_platform_driver(oxnas_stdclk_driver);
diff --git a/drivers/clk/clk-palmas.c b/drivers/clk/clk-palmas.c
index 9c0b8e6..8328863 100644
--- a/drivers/clk/clk-palmas.c
+++ b/drivers/clk/clk-palmas.c
@@ -132,7 +132,7 @@
.init = {
.name = "clk32kg",
.ops = &palmas_clks_ops,
- .flags = CLK_IS_ROOT | CLK_IGNORE_UNUSED,
+ .flags = CLK_IGNORE_UNUSED,
},
.desc = {
.clk_name = "clk32kg",
@@ -148,7 +148,7 @@
.init = {
.name = "clk32kgaudio",
.ops = &palmas_clks_ops,
- .flags = CLK_IS_ROOT | CLK_IGNORE_UNUSED,
+ .flags = CLK_IGNORE_UNUSED,
},
.desc = {
.clk_name = "clk32kgaudio",
diff --git a/drivers/clk/clk-qoriq.c b/drivers/clk/clk-qoriq.c
index 7bc1c45..58566a17 100644
--- a/drivers/clk/clk-qoriq.c
+++ b/drivers/clk/clk-qoriq.c
@@ -869,14 +869,15 @@
}
}
-static struct clk *sysclk_from_fixed(struct device_node *node, const char *name)
+static struct clk __init
+*sysclk_from_fixed(struct device_node *node, const char *name)
{
u32 rate;
if (of_property_read_u32(node, "clock-frequency", &rate))
return ERR_PTR(-ENODEV);
- return clk_register_fixed_rate(NULL, name, NULL, CLK_IS_ROOT, rate);
+ return clk_register_fixed_rate(NULL, name, NULL, 0, rate);
}
static struct clk *sysclk_from_parent(const char *name)
diff --git a/drivers/clk/clk-rk808.c b/drivers/clk/clk-rk808.c
index 0fee2f4..7438303 100644
--- a/drivers/clk/clk-rk808.c
+++ b/drivers/clk/clk-rk808.c
@@ -106,7 +106,6 @@
if (!clk_table)
return -ENOMEM;
- init.flags = CLK_IS_ROOT;
init.parent_names = NULL;
init.num_parents = 0;
init.name = "rk808-clkout1";
diff --git a/drivers/clk/clk-tango4.c b/drivers/clk/clk-tango4.c
index 004ab7d..eef75e3 100644
--- a/drivers/clk/clk-tango4.c
+++ b/drivers/clk/clk-tango4.c
@@ -4,17 +4,19 @@
#include <linux/init.h>
#include <linux/io.h>
-static struct clk *out[2];
-static struct clk_onecell_data clk_data = { out, 2 };
+#define CLK_COUNT 4 /* cpu_clk, sys_clk, usb_clk, sdio_clk */
+static struct clk *clks[CLK_COUNT];
+static struct clk_onecell_data clk_data = { clks, CLK_COUNT };
-#define SYSCLK_CTRL 0x20
-#define CPUCLK_CTRL 0x24
-#define LEGACY_DIV 0x3c
+#define SYSCLK_DIV 0x20
+#define CPUCLK_DIV 0x24
+#define DIV_BYPASS BIT(23)
-#define PLL_N(val) (((val) >> 0) & 0x7f)
-#define PLL_K(val) (((val) >> 13) & 0x7)
-#define PLL_M(val) (((val) >> 16) & 0x7)
-#define DIV_INDEX(val) (((val) >> 8) & 0xf)
+/*** CLKGEN_PLL ***/
+#define extract_pll_n(val) ((val >> 0) & ((1u << 7) - 1))
+#define extract_pll_k(val) ((val >> 13) & ((1u << 3) - 1))
+#define extract_pll_m(val) ((val >> 16) & ((1u << 3) - 1))
+#define extract_pll_isel(val) ((val >> 24) & ((1u << 3) - 1))
static void __init make_pll(int idx, const char *parent, void __iomem *base)
{
@@ -22,40 +24,61 @@
u32 val, mul, div;
sprintf(name, "pll%d", idx);
- val = readl_relaxed(base + idx*8);
- mul = PLL_N(val) + 1;
- div = (PLL_M(val) + 1) << PLL_K(val);
+ val = readl(base + idx * 8);
+ mul = extract_pll_n(val) + 1;
+ div = (extract_pll_m(val) + 1) << extract_pll_k(val);
clk_register_fixed_factor(NULL, name, parent, 0, mul, div);
+ if (extract_pll_isel(val) != 1)
+ panic("%s: input not set to XTAL_IN\n", name);
}
-static int __init get_div(void __iomem *base)
+static void __init make_cd(int idx, void __iomem *base)
{
- u8 sysclk_tab[16] = { 2, 4, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4 };
- int idx = DIV_INDEX(readl_relaxed(base + LEGACY_DIV));
+ char name[8];
+ u32 val, mul, div;
- return sysclk_tab[idx];
+ sprintf(name, "cd%d", idx);
+ val = readl(base + idx * 8);
+ mul = 1 << 27;
+ div = (2 << 27) + val;
+ clk_register_fixed_factor(NULL, name, "pll2", 0, mul, div);
+ if (val > 0xf0000000)
+ panic("%s: unsupported divider %x\n", name, val);
}
static void __init tango4_clkgen_setup(struct device_node *np)
{
- int div, ret;
+ struct clk **pp = clk_data.clks;
void __iomem *base = of_iomap(np, 0);
const char *parent = of_clk_get_parent_name(np, 0);
if (!base)
- panic("%s: invalid address\n", np->full_name);
+ panic("%s: invalid address\n", np->name);
+
+ if (readl(base + CPUCLK_DIV) & DIV_BYPASS)
+ panic("%s: unsupported cpuclk setup\n", np->name);
+
+ if (readl(base + SYSCLK_DIV) & DIV_BYPASS)
+ panic("%s: unsupported sysclk setup\n", np->name);
+
+ writel(0x100, base + CPUCLK_DIV); /* disable frequency ramping */
make_pll(0, parent, base);
make_pll(1, parent, base);
+ make_pll(2, parent, base);
+ make_cd(2, base + 0x80);
+ make_cd(6, base + 0x80);
- out[0] = clk_register_divider(NULL, "cpuclk", "pll0", 0,
- base + CPUCLK_CTRL, 8, 8, CLK_DIVIDER_ONE_BASED, NULL);
+ pp[0] = clk_register_divider(NULL, "cpu_clk", "pll0", 0,
+ base + CPUCLK_DIV, 8, 8, CLK_DIVIDER_ONE_BASED, NULL);
+ pp[1] = clk_register_fixed_factor(NULL, "sys_clk", "pll1", 0, 1, 4);
+ pp[2] = clk_register_fixed_factor(NULL, "usb_clk", "cd2", 0, 1, 2);
+ pp[3] = clk_register_fixed_factor(NULL, "sdio_clk", "cd6", 0, 1, 2);
- div = readl_relaxed(base + SYSCLK_CTRL) & BIT(23) ? get_div(base) : 4;
- out[1] = clk_register_fixed_factor(NULL, "sysclk", "pll1", 0, 1, div);
+ if (IS_ERR(pp[0]) || IS_ERR(pp[1]) || IS_ERR(pp[2]) || IS_ERR(pp[3]))
+ panic("%s: clk registration failed\n", np->name);
- ret = of_clk_add_provider(np, of_clk_src_onecell_get, &clk_data);
- if (IS_ERR(out[0]) || IS_ERR(out[1]) || ret < 0)
- panic("%s: clk registration failed\n", np->full_name);
+ if (of_clk_add_provider(np, of_clk_src_onecell_get, &clk_data))
+ panic("%s: clk provider registration failed\n", np->name);
}
CLK_OF_DECLARE(tango4_clkgen, "sigma,tango4-clkgen", tango4_clkgen_setup);
diff --git a/drivers/clk/clk-twl6040.c b/drivers/clk/clk-twl6040.c
index 8e5ed64..697c667 100644
--- a/drivers/clk/clk-twl6040.c
+++ b/drivers/clk/clk-twl6040.c
@@ -74,7 +74,6 @@
static struct clk_init_data wm831x_clkout_init = {
.name = "mcpdm_fclk",
.ops = &twl6040_mcpdm_ops,
- .flags = CLK_IS_ROOT,
};
static int twl6040_clk_probe(struct platform_device *pdev)
diff --git a/drivers/clk/clk-wm831x.c b/drivers/clk/clk-wm831x.c
index 43f9d15..88def4b 100644
--- a/drivers/clk/clk-wm831x.c
+++ b/drivers/clk/clk-wm831x.c
@@ -58,7 +58,6 @@
static struct clk_init_data wm831x_xtal_init = {
.name = "xtal",
.ops = &wm831x_xtal_ops,
- .flags = CLK_IS_ROOT,
};
static const unsigned long wm831x_fll_auto_rates[] = {
diff --git a/drivers/clk/clk-xgene.c b/drivers/clk/clk-xgene.c
index d73450b..3433132 100644
--- a/drivers/clk/clk-xgene.c
+++ b/drivers/clk/clk-xgene.c
@@ -198,7 +198,7 @@
of_property_read_string(np, "clock-output-names", &clk_name);
clk = xgene_register_clk_pll(NULL,
clk_name, of_clk_get_parent_name(np, 0),
- CLK_IS_ROOT, reg, 0, pll_type, &clk_lock,
+ 0, reg, 0, pll_type, &clk_lock,
version);
if (!IS_ERR(clk)) {
of_clk_add_provider(np, of_clk_src_simple_get, clk);
diff --git a/drivers/clk/clk.c b/drivers/clk/clk.c
index fb74dc1..d584004 100644
--- a/drivers/clk/clk.c
+++ b/drivers/clk/clk.c
@@ -574,6 +574,9 @@
if (WARN_ON(core->prepare_count == 0))
return;
+ if (WARN_ON(core->prepare_count == 1 && core->flags & CLK_IS_CRITICAL))
+ return;
+
if (--core->prepare_count > 0)
return;
@@ -679,6 +682,9 @@
if (WARN_ON(core->enable_count == 0))
return;
+ if (WARN_ON(core->enable_count == 1 && core->flags & CLK_IS_CRITICAL))
+ return;
+
if (--core->enable_count > 0)
return;
@@ -2397,6 +2403,16 @@
if (core->ops->init)
core->ops->init(core->hw);
+ if (core->flags & CLK_IS_CRITICAL) {
+ unsigned long flags;
+
+ clk_core_prepare(core);
+
+ flags = clk_enable_lock();
+ clk_core_enable(core);
+ clk_enable_unlock(flags);
+ }
+
kref_init(&core->ref);
out:
clk_prepare_unlock();
@@ -2536,6 +2552,22 @@
}
EXPORT_SYMBOL_GPL(clk_register);
+/**
+ * clk_hw_register - register a clk_hw and return an error code
+ * @dev: device that is registering this clock
+ * @hw: link to hardware-specific clock data
+ *
+ * clk_hw_register is the primary interface for populating the clock tree with
+ * new clock nodes. It returns an integer equal to zero indicating success or
+ * less than zero indicating failure. Drivers must test for an error code after
+ * calling clk_hw_register().
+ */
+int clk_hw_register(struct device *dev, struct clk_hw *hw)
+{
+ return PTR_ERR_OR_ZERO(clk_register(dev, hw));
+}
+EXPORT_SYMBOL_GPL(clk_hw_register);
+
/* Free memory allocated for a clock. */
static void __clk_release(struct kref *ref)
{
@@ -2637,11 +2669,26 @@
}
EXPORT_SYMBOL_GPL(clk_unregister);
+/**
+ * clk_hw_unregister - unregister a currently registered clk_hw
+ * @hw: hardware-specific clock data to unregister
+ */
+void clk_hw_unregister(struct clk_hw *hw)
+{
+ clk_unregister(hw->clk);
+}
+EXPORT_SYMBOL_GPL(clk_hw_unregister);
+
static void devm_clk_release(struct device *dev, void *res)
{
clk_unregister(*(struct clk **)res);
}
+static void devm_clk_hw_release(struct device *dev, void *res)
+{
+ clk_hw_unregister(*(struct clk_hw **)res);
+}
+
/**
* devm_clk_register - resource managed clk_register()
* @dev: device that is registering this clock
@@ -2672,6 +2719,36 @@
}
EXPORT_SYMBOL_GPL(devm_clk_register);
+/**
+ * devm_clk_hw_register - resource managed clk_hw_register()
+ * @dev: device that is registering this clock
+ * @hw: link to hardware-specific clock data
+ *
+ * Managed clk_hw_register(). Clocks registered by this function are
+ * automatically clk_hw_unregister()ed on driver detach. See clk_hw_register()
+ * for more information.
+ */
+int devm_clk_hw_register(struct device *dev, struct clk_hw *hw)
+{
+ struct clk_hw **hwp;
+ int ret;
+
+ hwp = devres_alloc(devm_clk_hw_release, sizeof(*hwp), GFP_KERNEL);
+ if (!hwp)
+ return -ENOMEM;
+
+ ret = clk_hw_register(dev, hw);
+ if (!ret) {
+ *hwp = hw;
+ devres_add(dev, hwp);
+ } else {
+ devres_free(hwp);
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(devm_clk_hw_register);
+
static int devm_clk_match(struct device *dev, void *res, void *data)
{
struct clk *c = res;
@@ -2680,6 +2757,15 @@
return c == data;
}
+static int devm_clk_hw_match(struct device *dev, void *res, void *data)
+{
+ struct clk_hw *hw = res;
+
+ if (WARN_ON(!hw))
+ return 0;
+ return hw == data;
+}
+
/**
* devm_clk_unregister - resource managed clk_unregister()
* @clk: clock to unregister
@@ -2694,6 +2780,22 @@
}
EXPORT_SYMBOL_GPL(devm_clk_unregister);
+/**
+ * devm_clk_hw_unregister - resource managed clk_hw_unregister()
+ * @dev: device that is unregistering the hardware-specific clock data
+ * @hw: link to hardware-specific clock data
+ *
+ * Unregister a clk_hw registered with devm_clk_hw_register(). Normally
+ * this function will not need to be called and the resource management
+ * code will ensure that the resource is freed.
+ */
+void devm_clk_hw_unregister(struct device *dev, struct clk_hw *hw)
+{
+ WARN_ON(devres_release(dev, devm_clk_hw_release, devm_clk_hw_match,
+ hw));
+}
+EXPORT_SYMBOL_GPL(devm_clk_hw_unregister);
+
/*
* clkdev helpers
*/
@@ -2855,6 +2957,7 @@
struct device_node *node;
struct clk *(*get)(struct of_phandle_args *clkspec, void *data);
+ struct clk_hw *(*get_hw)(struct of_phandle_args *clkspec, void *data);
void *data;
};
@@ -2871,6 +2974,12 @@
}
EXPORT_SYMBOL_GPL(of_clk_src_simple_get);
+struct clk_hw *of_clk_hw_simple_get(struct of_phandle_args *clkspec, void *data)
+{
+ return data;
+}
+EXPORT_SYMBOL_GPL(of_clk_hw_simple_get);
+
struct clk *of_clk_src_onecell_get(struct of_phandle_args *clkspec, void *data)
{
struct clk_onecell_data *clk_data = data;
@@ -2885,6 +2994,21 @@
}
EXPORT_SYMBOL_GPL(of_clk_src_onecell_get);
+struct clk_hw *
+of_clk_hw_onecell_get(struct of_phandle_args *clkspec, void *data)
+{
+ struct clk_hw_onecell_data *hw_data = data;
+ unsigned int idx = clkspec->args[0];
+
+ if (idx >= hw_data->num) {
+ pr_err("%s: invalid index %u\n", __func__, idx);
+ return ERR_PTR(-EINVAL);
+ }
+
+ return hw_data->hws[idx];
+}
+EXPORT_SYMBOL_GPL(of_clk_hw_onecell_get);
+
/**
* of_clk_add_provider() - Register a clock provider for a node
* @np: Device node pointer associated with clock provider
@@ -2921,6 +3045,41 @@
EXPORT_SYMBOL_GPL(of_clk_add_provider);
/**
+ * of_clk_add_hw_provider() - Register a clock provider for a node
+ * @np: Device node pointer associated with clock provider
+ * @get: callback for decoding clk_hw
+ * @data: context pointer for @get callback.
+ */
+int of_clk_add_hw_provider(struct device_node *np,
+ struct clk_hw *(*get)(struct of_phandle_args *clkspec,
+ void *data),
+ void *data)
+{
+ struct of_clk_provider *cp;
+ int ret;
+
+ cp = kzalloc(sizeof(*cp), GFP_KERNEL);
+ if (!cp)
+ return -ENOMEM;
+
+ cp->node = of_node_get(np);
+ cp->data = data;
+ cp->get_hw = get;
+
+ mutex_lock(&of_clk_mutex);
+ list_add(&cp->link, &of_clk_providers);
+ mutex_unlock(&of_clk_mutex);
+ pr_debug("Added clk_hw provider from %s\n", np->full_name);
+
+ ret = of_clk_set_defaults(np, true);
+ if (ret < 0)
+ of_clk_del_provider(np);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(of_clk_add_hw_provider);
+
+/**
* of_clk_del_provider() - Remove a previously registered clock provider
* @np: Device node pointer associated with clock provider
*/
@@ -2941,11 +3100,32 @@
}
EXPORT_SYMBOL_GPL(of_clk_del_provider);
+static struct clk_hw *
+__of_clk_get_hw_from_provider(struct of_clk_provider *provider,
+ struct of_phandle_args *clkspec)
+{
+ struct clk *clk;
+ struct clk_hw *hw = ERR_PTR(-EPROBE_DEFER);
+
+ if (provider->get_hw) {
+ hw = provider->get_hw(clkspec, provider->data);
+ } else if (provider->get) {
+ clk = provider->get(clkspec, provider->data);
+ if (!IS_ERR(clk))
+ hw = __clk_get_hw(clk);
+ else
+ hw = ERR_CAST(clk);
+ }
+
+ return hw;
+}
+
struct clk *__of_clk_get_from_provider(struct of_phandle_args *clkspec,
const char *dev_id, const char *con_id)
{
struct of_clk_provider *provider;
struct clk *clk = ERR_PTR(-EPROBE_DEFER);
+ struct clk_hw *hw = ERR_PTR(-EPROBE_DEFER);
if (!clkspec)
return ERR_PTR(-EINVAL);
@@ -2954,10 +3134,9 @@
mutex_lock(&of_clk_mutex);
list_for_each_entry(provider, &of_clk_providers, link) {
if (provider->node == clkspec->np)
- clk = provider->get(clkspec, provider->data);
- if (!IS_ERR(clk)) {
- clk = __clk_create_clk(__clk_get_hw(clk), dev_id,
- con_id);
+ hw = __of_clk_get_hw_from_provider(provider, clkspec);
+ if (!IS_ERR(hw)) {
+ clk = __clk_create_clk(hw, dev_id, con_id);
if (!IS_ERR(clk) && !__clk_get(clk)) {
__clk_free_clk(clk);
@@ -3127,6 +3306,41 @@
}
/**
+ * of_clk_detect_critical() - set CLK_IS_CRITICAL flag from Device Tree
+ * @np: Device node pointer associated with clock provider
+ * @index: clock index
+ * @flags: pointer to clk_core->flags
+ *
+ * Detects if the clock-critical property exists and, if so, sets the
+ * corresponding CLK_IS_CRITICAL flag.
+ *
+ * Do not use this function. It exists only for legacy Device Tree
+ * bindings, such as the one-clock-per-node style that are outdated.
+ * Those bindings typically put all clock data into .dts and the Linux
+ * driver has no clock data, thus making it impossible to set this flag
+ * correctly from the driver. Only those drivers may call
+ * of_clk_detect_critical from their setup functions.
+ *
+ * Return: error code or zero on success
+ */
+int of_clk_detect_critical(struct device_node *np,
+ int index, unsigned long *flags)
+{
+ struct property *prop;
+ const __be32 *cur;
+ uint32_t idx;
+
+ if (!np || !flags)
+ return -EINVAL;
+
+ of_property_for_each_u32(np, "clock-critical", prop, cur, idx)
+ if (index == idx)
+ *flags |= CLK_IS_CRITICAL;
+
+ return 0;
+}
+
+/**
* of_clk_init() - Scan and init clock providers from the DT
* @matches: array of compatible values and init functions for providers.
*
diff --git a/drivers/clk/clkdev.c b/drivers/clk/clkdev.c
index eb20b94..89cc700 100644
--- a/drivers/clk/clkdev.c
+++ b/drivers/clk/clkdev.c
@@ -301,6 +301,20 @@
}
EXPORT_SYMBOL(clkdev_alloc);
+struct clk_lookup *
+clkdev_hw_alloc(struct clk_hw *hw, const char *con_id, const char *dev_fmt, ...)
+{
+ struct clk_lookup *cl;
+ va_list ap;
+
+ va_start(ap, dev_fmt);
+ cl = vclkdev_alloc(hw, con_id, dev_fmt, ap);
+ va_end(ap);
+
+ return cl;
+}
+EXPORT_SYMBOL(clkdev_hw_alloc);
+
/**
* clkdev_create - allocate and add a clkdev lookup structure
* @clk: struct clk to associate with all clk_lookups
@@ -324,6 +338,29 @@
}
EXPORT_SYMBOL_GPL(clkdev_create);
+/**
+ * clkdev_hw_create - allocate and add a clkdev lookup structure
+ * @hw: struct clk_hw to associate with all clk_lookups
+ * @con_id: connection ID string on device
+ * @dev_fmt: format string describing device name
+ *
+ * Returns a clk_lookup structure, which can be later unregistered and
+ * freed.
+ */
+struct clk_lookup *clkdev_hw_create(struct clk_hw *hw, const char *con_id,
+ const char *dev_fmt, ...)
+{
+ struct clk_lookup *cl;
+ va_list ap;
+
+ va_start(ap, dev_fmt);
+ cl = vclkdev_create(hw, con_id, dev_fmt, ap);
+ va_end(ap);
+
+ return cl;
+}
+EXPORT_SYMBOL_GPL(clkdev_hw_create);
+
int clk_add_alias(const char *alias, const char *alias_dev_name,
const char *con_id, struct device *dev)
{
@@ -404,28 +441,28 @@
EXPORT_SYMBOL(clk_register_clkdev);
/**
- * clk_register_clkdevs - register a set of clk_lookup for a struct clk
- * @clk: struct clk to associate with all clk_lookups
- * @cl: array of clk_lookup structures with con_id and dev_id pre-initialized
- * @num: number of clk_lookup structures to register
+ * clk_hw_register_clkdev - register one clock lookup for a struct clk_hw
+ * @hw: struct clk_hw to associate with all clk_lookups
+ * @con_id: connection ID string on device
+ * @dev_id: format string describing device name
*
- * To make things easier for mass registration, we detect error clks
- * from a previous clk_register() call, and return the error code for
- * those. This is to permit this function to be called immediately
- * after clk_register().
+ * con_id or dev_id may be NULL as a wildcard, just as in the rest of
+ * clkdev.
*/
-int clk_register_clkdevs(struct clk *clk, struct clk_lookup *cl, size_t num)
+int clk_hw_register_clkdev(struct clk_hw *hw, const char *con_id,
+ const char *dev_id)
{
- unsigned i;
+ struct clk_lookup *cl;
- if (IS_ERR(clk))
- return PTR_ERR(clk);
+ /*
+ * Since dev_id can be NULL, and NULL is handled specially, we must
+ * pass it as either a NULL format string, or with "%s".
+ */
+ if (dev_id)
+ cl = __clk_register_clkdev(hw, con_id, "%s", dev_id);
+ else
+ cl = __clk_register_clkdev(hw, con_id, NULL);
- for (i = 0; i < num; i++, cl++) {
- cl->clk_hw = __clk_get_hw(clk);
- __clkdev_add(cl);
- }
-
- return 0;
+ return cl ? 0 : -ENOMEM;
}
-EXPORT_SYMBOL(clk_register_clkdevs);
+EXPORT_SYMBOL(clk_hw_register_clkdev);
diff --git a/drivers/clk/hisilicon/Kconfig b/drivers/clk/hisilicon/Kconfig
index e434854..3f537a0 100644
--- a/drivers/clk/hisilicon/Kconfig
+++ b/drivers/clk/hisilicon/Kconfig
@@ -1,3 +1,11 @@
+config COMMON_CLK_HI3519
+ tristate "Hi3519 Clock Driver"
+ depends on ARCH_HISI || COMPILE_TEST
+ select RESET_HISI
+ default ARCH_HISI
+ help
+ Build the clock driver for hi3519.
+
config COMMON_CLK_HI6220
bool "Hi6220 Clock Driver"
depends on ARCH_HISI || COMPILE_TEST
@@ -5,6 +13,13 @@
help
Build the Hisilicon Hi6220 clock driver based on the common clock framework.
+config RESET_HISI
+ bool "HiSilicon Reset Controller Driver"
+ depends on ARCH_HISI || COMPILE_TEST
+ select RESET_CONTROLLER
+ help
+ Build reset controller driver for HiSilicon device chipsets.
+
config STUB_CLK_HI6220
bool "Hi6220 Stub Clock Driver"
depends on COMMON_CLK_HI6220 && MAILBOX
diff --git a/drivers/clk/hisilicon/Makefile b/drivers/clk/hisilicon/Makefile
index 74dba31..e169ec7 100644
--- a/drivers/clk/hisilicon/Makefile
+++ b/drivers/clk/hisilicon/Makefile
@@ -7,5 +7,7 @@
obj-$(CONFIG_ARCH_HI3xxx) += clk-hi3620.o
obj-$(CONFIG_ARCH_HIP04) += clk-hip04.o
obj-$(CONFIG_ARCH_HIX5HD2) += clk-hix5hd2.o
+obj-$(CONFIG_COMMON_CLK_HI3519) += clk-hi3519.o
obj-$(CONFIG_COMMON_CLK_HI6220) += clk-hi6220.o
+obj-$(CONFIG_RESET_HISI) += reset.o
obj-$(CONFIG_STUB_CLK_HI6220) += clk-hi6220-stub.o
diff --git a/drivers/clk/hisilicon/clk-hi3519.c b/drivers/clk/hisilicon/clk-hi3519.c
new file mode 100644
index 0000000..715c730
--- /dev/null
+++ b/drivers/clk/hisilicon/clk-hi3519.c
@@ -0,0 +1,131 @@
+/*
+ * Hi3519 Clock Driver
+ *
+ * Copyright (c) 2015-2016 HiSilicon Technologies Co., Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <dt-bindings/clock/hi3519-clock.h>
+#include <linux/clk-provider.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include "clk.h"
+#include "reset.h"
+
+#define HI3519_INNER_CLK_OFFSET 64
+#define HI3519_FIXED_24M 65
+#define HI3519_FIXED_50M 66
+#define HI3519_FIXED_75M 67
+#define HI3519_FIXED_125M 68
+#define HI3519_FIXED_150M 69
+#define HI3519_FIXED_200M 70
+#define HI3519_FIXED_250M 71
+#define HI3519_FIXED_300M 72
+#define HI3519_FIXED_400M 73
+#define HI3519_FMC_MUX 74
+
+#define HI3519_NR_CLKS 128
+
+static const struct hisi_fixed_rate_clock hi3519_fixed_rate_clks[] = {
+ { HI3519_FIXED_24M, "24m", NULL, 0, 24000000, },
+ { HI3519_FIXED_50M, "50m", NULL, 0, 50000000, },
+ { HI3519_FIXED_75M, "75m", NULL, 0, 75000000, },
+ { HI3519_FIXED_125M, "125m", NULL, 0, 125000000, },
+ { HI3519_FIXED_150M, "150m", NULL, 0, 150000000, },
+ { HI3519_FIXED_200M, "200m", NULL, 0, 200000000, },
+ { HI3519_FIXED_250M, "250m", NULL, 0, 250000000, },
+ { HI3519_FIXED_300M, "300m", NULL, 0, 300000000, },
+ { HI3519_FIXED_400M, "400m", NULL, 0, 400000000, },
+};
+
+static const char *const fmc_mux_p[] = {
+ "24m", "75m", "125m", "150m", "200m", "250m", "300m", "400m", };
+static u32 fmc_mux_table[] = {0, 1, 2, 3, 4, 5, 6, 7};
+
+static const struct hisi_mux_clock hi3519_mux_clks[] = {
+ { HI3519_FMC_MUX, "fmc_mux", fmc_mux_p, ARRAY_SIZE(fmc_mux_p),
+ CLK_SET_RATE_PARENT, 0xc0, 2, 3, 0, fmc_mux_table, },
+};
+
+static const struct hisi_gate_clock hi3519_gate_clks[] = {
+ { HI3519_FMC_CLK, "clk_fmc", "fmc_mux",
+ CLK_SET_RATE_PARENT, 0xc0, 1, 0, },
+ { HI3519_UART0_CLK, "clk_uart0", "24m",
+ CLK_SET_RATE_PARENT, 0xe4, 20, 0, },
+ { HI3519_UART1_CLK, "clk_uart1", "24m",
+ CLK_SET_RATE_PARENT, 0xe4, 21, 0, },
+ { HI3519_UART2_CLK, "clk_uart2", "24m",
+ CLK_SET_RATE_PARENT, 0xe4, 22, 0, },
+ { HI3519_UART3_CLK, "clk_uart3", "24m",
+ CLK_SET_RATE_PARENT, 0xe4, 23, 0, },
+ { HI3519_UART4_CLK, "clk_uart4", "24m",
+ CLK_SET_RATE_PARENT, 0xe4, 24, 0, },
+ { HI3519_SPI0_CLK, "clk_spi0", "50m",
+ CLK_SET_RATE_PARENT, 0xe4, 16, 0, },
+ { HI3519_SPI1_CLK, "clk_spi1", "50m",
+ CLK_SET_RATE_PARENT, 0xe4, 17, 0, },
+ { HI3519_SPI2_CLK, "clk_spi2", "50m",
+ CLK_SET_RATE_PARENT, 0xe4, 18, 0, },
+};
+
+static int hi3519_clk_probe(struct platform_device *pdev)
+{
+ struct device_node *np = pdev->dev.of_node;
+ struct hisi_clock_data *clk_data;
+ struct hisi_reset_controller *rstc;
+
+ rstc = hisi_reset_init(np);
+ if (!rstc)
+ return -ENOMEM;
+
+ clk_data = hisi_clk_init(np, HI3519_NR_CLKS);
+ if (!clk_data) {
+ hisi_reset_exit(rstc);
+ return -ENODEV;
+ }
+
+ hisi_clk_register_fixed_rate(hi3519_fixed_rate_clks,
+ ARRAY_SIZE(hi3519_fixed_rate_clks),
+ clk_data);
+ hisi_clk_register_mux(hi3519_mux_clks, ARRAY_SIZE(hi3519_mux_clks),
+ clk_data);
+ hisi_clk_register_gate(hi3519_gate_clks,
+ ARRAY_SIZE(hi3519_gate_clks), clk_data);
+
+ return 0;
+}
+
+static const struct of_device_id hi3519_clk_match_table[] = {
+ { .compatible = "hisilicon,hi3519-crg" },
+ { }
+};
+MODULE_DEVICE_TABLE(of, hi3519_clk_match_table);
+
+static struct platform_driver hi3519_clk_driver = {
+ .probe = hi3519_clk_probe,
+ .driver = {
+ .name = "hi3519-clk",
+ .of_match_table = hi3519_clk_match_table,
+ },
+};
+
+static int __init hi3519_clk_init(void)
+{
+ return platform_driver_register(&hi3519_clk_driver);
+}
+core_initcall(hi3519_clk_init);
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("HiSilicon Hi3519 Clock Driver");
diff --git a/drivers/clk/hisilicon/clk.c b/drivers/clk/hisilicon/clk.c
index 9f8e766..9b15adb 100644
--- a/drivers/clk/hisilicon/clk.c
+++ b/drivers/clk/hisilicon/clk.c
@@ -37,7 +37,7 @@
static DEFINE_SPINLOCK(hisi_clk_lock);
-struct hisi_clock_data __init *hisi_clk_init(struct device_node *np,
+struct hisi_clock_data *hisi_clk_init(struct device_node *np,
int nr_clks)
{
struct hisi_clock_data *clk_data;
@@ -71,8 +71,9 @@
err:
return NULL;
}
+EXPORT_SYMBOL_GPL(hisi_clk_init);
-void __init hisi_clk_register_fixed_rate(struct hisi_fixed_rate_clock *clks,
+void hisi_clk_register_fixed_rate(const struct hisi_fixed_rate_clock *clks,
int nums, struct hisi_clock_data *data)
{
struct clk *clk;
@@ -91,8 +92,9 @@
data->clk_data.clks[clks[i].id] = clk;
}
}
+EXPORT_SYMBOL_GPL(hisi_clk_register_fixed_rate);
-void __init hisi_clk_register_fixed_factor(struct hisi_fixed_factor_clock *clks,
+void hisi_clk_register_fixed_factor(const struct hisi_fixed_factor_clock *clks,
int nums,
struct hisi_clock_data *data)
{
@@ -112,8 +114,9 @@
data->clk_data.clks[clks[i].id] = clk;
}
}
+EXPORT_SYMBOL_GPL(hisi_clk_register_fixed_factor);
-void __init hisi_clk_register_mux(struct hisi_mux_clock *clks,
+void hisi_clk_register_mux(const struct hisi_mux_clock *clks,
int nums, struct hisi_clock_data *data)
{
struct clk *clk;
@@ -141,8 +144,9 @@
data->clk_data.clks[clks[i].id] = clk;
}
}
+EXPORT_SYMBOL_GPL(hisi_clk_register_mux);
-void __init hisi_clk_register_divider(struct hisi_divider_clock *clks,
+void hisi_clk_register_divider(const struct hisi_divider_clock *clks,
int nums, struct hisi_clock_data *data)
{
struct clk *clk;
@@ -170,8 +174,9 @@
data->clk_data.clks[clks[i].id] = clk;
}
}
+EXPORT_SYMBOL_GPL(hisi_clk_register_divider);
-void __init hisi_clk_register_gate(struct hisi_gate_clock *clks,
+void hisi_clk_register_gate(const struct hisi_gate_clock *clks,
int nums, struct hisi_clock_data *data)
{
struct clk *clk;
@@ -198,8 +203,9 @@
data->clk_data.clks[clks[i].id] = clk;
}
}
+EXPORT_SYMBOL_GPL(hisi_clk_register_gate);
-void __init hisi_clk_register_gate_sep(struct hisi_gate_clock *clks,
+void hisi_clk_register_gate_sep(const struct hisi_gate_clock *clks,
int nums, struct hisi_clock_data *data)
{
struct clk *clk;
@@ -226,8 +232,9 @@
data->clk_data.clks[clks[i].id] = clk;
}
}
+EXPORT_SYMBOL_GPL(hisi_clk_register_gate_sep);
-void __init hi6220_clk_register_divider(struct hi6220_divider_clock *clks,
+void __init hi6220_clk_register_divider(const struct hi6220_divider_clock *clks,
int nums, struct hisi_clock_data *data)
{
struct clk *clk;
diff --git a/drivers/clk/hisilicon/clk.h b/drivers/clk/hisilicon/clk.h
index b56fbc1..20d64af 100644
--- a/drivers/clk/hisilicon/clk.h
+++ b/drivers/clk/hisilicon/clk.h
@@ -111,18 +111,18 @@
u8 shift, u8 width, u32 mask_bit, spinlock_t *lock);
struct hisi_clock_data *hisi_clk_init(struct device_node *, int);
-void hisi_clk_register_fixed_rate(struct hisi_fixed_rate_clock *,
+void hisi_clk_register_fixed_rate(const struct hisi_fixed_rate_clock *,
int, struct hisi_clock_data *);
-void hisi_clk_register_fixed_factor(struct hisi_fixed_factor_clock *,
+void hisi_clk_register_fixed_factor(const struct hisi_fixed_factor_clock *,
int, struct hisi_clock_data *);
-void hisi_clk_register_mux(struct hisi_mux_clock *, int,
+void hisi_clk_register_mux(const struct hisi_mux_clock *, int,
struct hisi_clock_data *);
-void hisi_clk_register_divider(struct hisi_divider_clock *,
+void hisi_clk_register_divider(const struct hisi_divider_clock *,
int, struct hisi_clock_data *);
-void hisi_clk_register_gate(struct hisi_gate_clock *,
+void hisi_clk_register_gate(const struct hisi_gate_clock *,
int, struct hisi_clock_data *);
-void hisi_clk_register_gate_sep(struct hisi_gate_clock *,
+void hisi_clk_register_gate_sep(const struct hisi_gate_clock *,
int, struct hisi_clock_data *);
-void hi6220_clk_register_divider(struct hi6220_divider_clock *,
+void hi6220_clk_register_divider(const struct hi6220_divider_clock *,
int, struct hisi_clock_data *);
#endif /* __HISI_CLK_H */
diff --git a/drivers/clk/hisilicon/reset.c b/drivers/clk/hisilicon/reset.c
new file mode 100644
index 0000000..6aa49c2
--- /dev/null
+++ b/drivers/clk/hisilicon/reset.c
@@ -0,0 +1,134 @@
+/*
+ * Hisilicon Reset Controller Driver
+ *
+ * Copyright (c) 2015-2016 HiSilicon Technologies Co., Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/io.h>
+#include <linux/of_address.h>
+#include <linux/reset-controller.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include "reset.h"
+
+#define HISI_RESET_BIT_MASK 0x1f
+#define HISI_RESET_OFFSET_SHIFT 8
+#define HISI_RESET_OFFSET_MASK 0xffff00
+
+struct hisi_reset_controller {
+ spinlock_t lock;
+ void __iomem *membase;
+ struct reset_controller_dev rcdev;
+};
+
+
+#define to_hisi_reset_controller(rcdev) \
+ container_of(rcdev, struct hisi_reset_controller, rcdev)
+
+static int hisi_reset_of_xlate(struct reset_controller_dev *rcdev,
+ const struct of_phandle_args *reset_spec)
+{
+ u32 offset;
+ u8 bit;
+
+ offset = (reset_spec->args[0] << HISI_RESET_OFFSET_SHIFT)
+ & HISI_RESET_OFFSET_MASK;
+ bit = reset_spec->args[1] & HISI_RESET_BIT_MASK;
+
+ return (offset | bit);
+}
+
+static int hisi_reset_assert(struct reset_controller_dev *rcdev,
+ unsigned long id)
+{
+ struct hisi_reset_controller *rstc = to_hisi_reset_controller(rcdev);
+ unsigned long flags;
+ u32 offset, reg;
+ u8 bit;
+
+ offset = (id & HISI_RESET_OFFSET_MASK) >> HISI_RESET_OFFSET_SHIFT;
+ bit = id & HISI_RESET_BIT_MASK;
+
+ spin_lock_irqsave(&rstc->lock, flags);
+
+ reg = readl(rstc->membase + offset);
+ writel(reg | BIT(bit), rstc->membase + offset);
+
+ spin_unlock_irqrestore(&rstc->lock, flags);
+
+ return 0;
+}
+
+static int hisi_reset_deassert(struct reset_controller_dev *rcdev,
+ unsigned long id)
+{
+ struct hisi_reset_controller *rstc = to_hisi_reset_controller(rcdev);
+ unsigned long flags;
+ u32 offset, reg;
+ u8 bit;
+
+ offset = (id & HISI_RESET_OFFSET_MASK) >> HISI_RESET_OFFSET_SHIFT;
+ bit = id & HISI_RESET_BIT_MASK;
+
+ spin_lock_irqsave(&rstc->lock, flags);
+
+ reg = readl(rstc->membase + offset);
+ writel(reg & ~BIT(bit), rstc->membase + offset);
+
+ spin_unlock_irqrestore(&rstc->lock, flags);
+
+ return 0;
+}
+
+static const struct reset_control_ops hisi_reset_ops = {
+ .assert = hisi_reset_assert,
+ .deassert = hisi_reset_deassert,
+};
+
+struct hisi_reset_controller *hisi_reset_init(struct device_node *np)
+{
+ struct hisi_reset_controller *rstc;
+
+ rstc = kzalloc(sizeof(*rstc), GFP_KERNEL);
+ if (!rstc)
+ return NULL;
+
+ rstc->membase = of_iomap(np, 0);
+ if (!rstc->membase) {
+ kfree(rstc);
+ return NULL;
+ }
+
+ spin_lock_init(&rstc->lock);
+
+ rstc->rcdev.owner = THIS_MODULE;
+ rstc->rcdev.ops = &hisi_reset_ops;
+ rstc->rcdev.of_node = np;
+ rstc->rcdev.of_reset_n_cells = 2;
+ rstc->rcdev.of_xlate = hisi_reset_of_xlate;
+ reset_controller_register(&rstc->rcdev);
+
+ return rstc;
+}
+EXPORT_SYMBOL_GPL(hisi_reset_init);
+
+void hisi_reset_exit(struct hisi_reset_controller *rstc)
+{
+ reset_controller_unregister(&rstc->rcdev);
+ iounmap(rstc->membase);
+ kfree(rstc);
+}
+EXPORT_SYMBOL_GPL(hisi_reset_exit);
diff --git a/drivers/clk/hisilicon/reset.h b/drivers/clk/hisilicon/reset.h
new file mode 100644
index 0000000..677d773
--- /dev/null
+++ b/drivers/clk/hisilicon/reset.h
@@ -0,0 +1,36 @@
+/*
+ * Copyright (c) 2015 HiSilicon Technologies Co., Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __HISI_RESET_H
+#define __HISI_RESET_H
+
+struct device_node;
+struct hisi_reset_controller;
+
+#ifdef CONFIG_RESET_CONTROLLER
+struct hisi_reset_controller *hisi_reset_init(struct device_node *np);
+void hisi_reset_exit(struct hisi_reset_controller *rstc);
+#else
+static inline hisi_reset_controller *hisi_reset_init(struct device_node *np)
+{
+ return 0;
+}
+static inline void hisi_reset_exit(struct hisi_reset_controller *rstc)
+{}
+#endif
+
+#endif /* __HISI_RESET_H */
diff --git a/drivers/clk/imx/clk-gate2.c b/drivers/clk/imx/clk-gate2.c
index 8935bff..db44a19 100644
--- a/drivers/clk/imx/clk-gate2.c
+++ b/drivers/clk/imx/clk-gate2.c
@@ -31,6 +31,7 @@
struct clk_hw hw;
void __iomem *reg;
u8 bit_idx;
+ u8 cgr_val;
u8 flags;
spinlock_t *lock;
unsigned int *share_count;
@@ -50,7 +51,8 @@
goto out;
reg = readl(gate->reg);
- reg |= 3 << gate->bit_idx;
+ reg &= ~(3 << gate->bit_idx);
+ reg |= gate->cgr_val << gate->bit_idx;
writel(reg, gate->reg);
out:
@@ -125,7 +127,7 @@
struct clk *clk_register_gate2(struct device *dev, const char *name,
const char *parent_name, unsigned long flags,
- void __iomem *reg, u8 bit_idx,
+ void __iomem *reg, u8 bit_idx, u8 cgr_val,
u8 clk_gate2_flags, spinlock_t *lock,
unsigned int *share_count)
{
@@ -140,6 +142,7 @@
/* struct clk_gate2 assignments */
gate->reg = reg;
gate->bit_idx = bit_idx;
+ gate->cgr_val = cgr_val;
gate->flags = clk_gate2_flags;
gate->lock = lock;
gate->share_count = share_count;
diff --git a/drivers/clk/imx/clk-imx35.c b/drivers/clk/imx/clk-imx35.c
index a71d24c..b0978d3 100644
--- a/drivers/clk/imx/clk-imx35.c
+++ b/drivers/clk/imx/clk-imx35.c
@@ -66,7 +66,7 @@
static const char *ipg_per_sel[] = {"ahb_per_div", "arm_per_div"};
enum mx35_clks {
- ckih, ckil, mpll, ppll, mpll_075, arm, hsp, hsp_div, hsp_sel, ahb, ipg,
+ ckih, mpll, ppll, mpll_075, arm, hsp, hsp_div, hsp_sel, ahb, ipg,
arm_per_div, ahb_per_div, ipg_per, uart_sel, uart_div, esdhc_sel,
esdhc1_div, esdhc2_div, esdhc3_div, spdif_sel, spdif_div_pre,
spdif_div_post, ssi_sel, ssi1_div_pre, ssi1_div_post, ssi2_div_pre,
@@ -79,7 +79,7 @@
rtc_gate, rtic_gate, scc_gate, sdma_gate, spba_gate, spdif_gate,
ssi1_gate, ssi2_gate, uart1_gate, uart2_gate, uart3_gate, usbotg_gate,
wdog_gate, max_gate, admux_gate, csi_gate, csi_div, csi_sel, iim_gate,
- gpu2d_gate, clk_max
+ gpu2d_gate, ckil, clk_max
};
static struct clk *clk[clk_max];
diff --git a/drivers/clk/imx/clk-imx6sx.c b/drivers/clk/imx/clk-imx6sx.c
index fea125e..97e742a 100644
--- a/drivers/clk/imx/clk-imx6sx.c
+++ b/drivers/clk/imx/clk-imx6sx.c
@@ -134,6 +134,8 @@
static u32 share_count_ssi1;
static u32 share_count_ssi2;
static u32 share_count_ssi3;
+static u32 share_count_sai1;
+static u32 share_count_sai2;
static struct clk ** const uart_clks[] __initconst = {
&clks[IMX6SX_CLK_UART_IPG],
@@ -469,10 +471,10 @@
clks[IMX6SX_CLK_SSI3] = imx_clk_gate2_shared("ssi3", "ssi3_podf", base + 0x7c, 22, &share_count_ssi3);
clks[IMX6SX_CLK_UART_IPG] = imx_clk_gate2("uart_ipg", "ipg", base + 0x7c, 24);
clks[IMX6SX_CLK_UART_SERIAL] = imx_clk_gate2("uart_serial", "uart_podf", base + 0x7c, 26);
- clks[IMX6SX_CLK_SAI1_IPG] = imx_clk_gate2("sai1_ipg", "ipg", base + 0x7c, 28);
- clks[IMX6SX_CLK_SAI2_IPG] = imx_clk_gate2("sai2_ipg", "ipg", base + 0x7c, 30);
- clks[IMX6SX_CLK_SAI1] = imx_clk_gate2("sai1", "ssi1_podf", base + 0x7c, 28);
- clks[IMX6SX_CLK_SAI2] = imx_clk_gate2("sai2", "ssi2_podf", base + 0x7c, 30);
+ clks[IMX6SX_CLK_SAI1_IPG] = imx_clk_gate2_shared("sai1_ipg", "ipg", base + 0x7c, 28, &share_count_sai1);
+ clks[IMX6SX_CLK_SAI2_IPG] = imx_clk_gate2_shared("sai2_ipg", "ipg", base + 0x7c, 30, &share_count_sai2);
+ clks[IMX6SX_CLK_SAI1] = imx_clk_gate2_shared("sai1", "ssi1_podf", base + 0x7c, 28, &share_count_sai1);
+ clks[IMX6SX_CLK_SAI2] = imx_clk_gate2_shared("sai2", "ssi2_podf", base + 0x7c, 30, &share_count_sai2);
/* CCGR6 */
clks[IMX6SX_CLK_USBOH3] = imx_clk_gate2("usboh3", "ipg", base + 0x80, 0);
diff --git a/drivers/clk/imx/clk-imx7d.c b/drivers/clk/imx/clk-imx7d.c
index fbb6a8c..5229968 100644
--- a/drivers/clk/imx/clk-imx7d.c
+++ b/drivers/clk/imx/clk-imx7d.c
@@ -56,7 +56,7 @@
"pll_sys_pfd2_135m_clk", "pll_sys_pfd6_clk", "pll_enet_250m_clk",
"pll_audio_main_clk", };
-static const char *ahb_channel_sel[] = { "osc", "pll_sys_pfd2_135m_clk",
+static const char *ahb_channel_sel[] = { "osc", "pll_sys_pfd2_270m_clk",
"pll_dram_533m_clk", "pll_sys_pfd0_392m_clk",
"pll_enet_125m_clk", "pll_usb_main_clk", "pll_audio_main_clk",
"pll_video_main_clk", };
@@ -342,7 +342,7 @@
static const char *clko2_sel[] = { "osc", "pll_sys_main_240m_clk",
"pll_sys_pfd0_392m_clk", "pll_sys_pfd1_166m_clk", "pll_sys_pfd4_clk",
- "pll_audio_main_clk", "pll_video_main_clk", "osc_32k_clk", };
+ "pll_audio_main_clk", "pll_video_main_clk", "ckil", };
static const char *lvds1_sel[] = { "pll_arm_main_clk",
"pll_sys_main_clk", "pll_sys_pfd0_392m_clk", "pll_sys_pfd1_332m_clk",
@@ -382,6 +382,7 @@
clks[IMX7D_CLK_DUMMY] = imx_clk_fixed("dummy", 0);
clks[IMX7D_OSC_24M_CLK] = of_clk_get_by_name(ccm_node, "osc");
+ clks[IMX7D_CKIL] = of_clk_get_by_name(ccm_node, "ckil");
np = of_find_compatible_node(NULL, NULL, "fsl,imx7d-anatop");
base = of_iomap(np, 0);
diff --git a/drivers/clk/imx/clk-pllv3.c b/drivers/clk/imx/clk-pllv3.c
index c05c43d..4826b3c 100644
--- a/drivers/clk/imx/clk-pllv3.c
+++ b/drivers/clk/imx/clk-pllv3.c
@@ -44,6 +44,7 @@
u32 powerdown;
u32 div_mask;
u32 div_shift;
+ unsigned long ref_clock;
};
#define to_clk_pllv3(_hw) container_of(_hw, struct clk_pllv3, hw)
@@ -286,7 +287,9 @@
static unsigned long clk_pllv3_enet_recalc_rate(struct clk_hw *hw,
unsigned long parent_rate)
{
- return 500000000;
+ struct clk_pllv3 *pll = to_clk_pllv3(hw);
+
+ return pll->ref_clock;
}
static const struct clk_ops clk_pllv3_enet_ops = {
@@ -326,7 +329,11 @@
break;
case IMX_PLLV3_ENET_IMX7:
pll->powerdown = IMX7_ENET_PLL_POWER;
+ pll->ref_clock = 1000000000;
+ ops = &clk_pllv3_enet_ops;
+ break;
case IMX_PLLV3_ENET:
+ pll->ref_clock = 500000000;
ops = &clk_pllv3_enet_ops;
break;
default:
diff --git a/drivers/clk/imx/clk-vf610.c b/drivers/clk/imx/clk-vf610.c
index 0a94d96..3a1f244 100644
--- a/drivers/clk/imx/clk-vf610.c
+++ b/drivers/clk/imx/clk-vf610.c
@@ -10,6 +10,7 @@
#include <linux/of_address.h>
#include <linux/clk.h>
+#include <linux/syscore_ops.h>
#include <dt-bindings/clock/vf610-clock.h>
#include "clk.h"
@@ -40,6 +41,7 @@
#define CCM_CCGR9 (ccm_base + 0x64)
#define CCM_CCGR10 (ccm_base + 0x68)
#define CCM_CCGR11 (ccm_base + 0x6c)
+#define CCM_CCGRx(x) (CCM_CCGR0 + (x) * 4)
#define CCM_CMEOR0 (ccm_base + 0x70)
#define CCM_CMEOR1 (ccm_base + 0x74)
#define CCM_CMEOR2 (ccm_base + 0x78)
@@ -115,10 +117,19 @@
static struct clk *clk[VF610_CLK_END];
static struct clk_onecell_data clk_data;
+static u32 cscmr1;
+static u32 cscmr2;
+static u32 cscdr1;
+static u32 cscdr2;
+static u32 cscdr3;
+static u32 ccgr[12];
+
static unsigned int const clks_init_on[] __initconst = {
VF610_CLK_SYS_BUS,
VF610_CLK_DDR_SEL,
VF610_CLK_DAP,
+ VF610_CLK_DDRMC,
+ VF610_CLK_WKPU,
};
static struct clk * __init vf610_get_fixed_clock(
@@ -132,6 +143,43 @@
return clk;
};
+static int vf610_clk_suspend(void)
+{
+ int i;
+
+ cscmr1 = readl_relaxed(CCM_CSCMR1);
+ cscmr2 = readl_relaxed(CCM_CSCMR2);
+
+ cscdr1 = readl_relaxed(CCM_CSCDR1);
+ cscdr2 = readl_relaxed(CCM_CSCDR2);
+ cscdr3 = readl_relaxed(CCM_CSCDR3);
+
+ for (i = 0; i < 12; i++)
+ ccgr[i] = readl_relaxed(CCM_CCGRx(i));
+
+ return 0;
+}
+
+static void vf610_clk_resume(void)
+{
+ int i;
+
+ writel_relaxed(cscmr1, CCM_CSCMR1);
+ writel_relaxed(cscmr2, CCM_CSCMR2);
+
+ writel_relaxed(cscdr1, CCM_CSCDR1);
+ writel_relaxed(cscdr2, CCM_CSCDR2);
+ writel_relaxed(cscdr3, CCM_CSCDR3);
+
+ for (i = 0; i < 12; i++)
+ writel_relaxed(ccgr[i], CCM_CCGRx(i));
+}
+
+static struct syscore_ops vf610_clk_syscore_ops = {
+ .suspend = vf610_clk_suspend,
+ .resume = vf610_clk_resume,
+};
+
static void __init vf610_clocks_init(struct device_node *ccm_node)
{
struct device_node *np;
@@ -233,6 +281,9 @@
clk[VF610_CLK_PLL4_MAIN_DIV] = clk_register_divider_table(NULL, "pll4_audio_div", "pll4_audio", 0, CCM_CACRR, 6, 3, 0, pll4_audio_div_table, &imx_ccm_lock);
clk[VF610_CLK_PLL6_MAIN_DIV] = imx_clk_divider("pll6_video_div", "pll6_video", CCM_CACRR, 21, 1);
+ clk[VF610_CLK_DDRMC] = imx_clk_gate2_cgr("ddrmc", "ddr_sel", CCM_CCGR6, CCM_CCGRx_CGn(14), 0x2);
+ clk[VF610_CLK_WKPU] = imx_clk_gate2_cgr("wkpu", "ipg_bus", CCM_CCGR4, CCM_CCGRx_CGn(10), 0x2);
+
clk[VF610_CLK_USBPHY0] = imx_clk_gate("usbphy0", "pll3_usb_otg", PLL3_CTRL, 6);
clk[VF610_CLK_USBPHY1] = imx_clk_gate("usbphy1", "pll7_usb_host", PLL7_CTRL, 6);
@@ -321,11 +372,14 @@
clk[VF610_CLK_DCU0_SEL] = imx_clk_mux("dcu0_sel", CCM_CSCMR1, 28, 1, dcu_sels, 2);
clk[VF610_CLK_DCU0_EN] = imx_clk_gate("dcu0_en", "dcu0_sel", CCM_CSCDR3, 19);
clk[VF610_CLK_DCU0_DIV] = imx_clk_divider("dcu0_div", "dcu0_en", CCM_CSCDR3, 16, 3);
- clk[VF610_CLK_DCU0] = imx_clk_gate2("dcu0", "dcu0_div", CCM_CCGR3, CCM_CCGRx_CGn(8));
+ clk[VF610_CLK_DCU0] = imx_clk_gate2("dcu0", "ipg_bus", CCM_CCGR3, CCM_CCGRx_CGn(8));
clk[VF610_CLK_DCU1_SEL] = imx_clk_mux("dcu1_sel", CCM_CSCMR1, 29, 1, dcu_sels, 2);
clk[VF610_CLK_DCU1_EN] = imx_clk_gate("dcu1_en", "dcu1_sel", CCM_CSCDR3, 23);
clk[VF610_CLK_DCU1_DIV] = imx_clk_divider("dcu1_div", "dcu1_en", CCM_CSCDR3, 20, 3);
- clk[VF610_CLK_DCU1] = imx_clk_gate2("dcu1", "dcu1_div", CCM_CCGR9, CCM_CCGRx_CGn(8));
+ clk[VF610_CLK_DCU1] = imx_clk_gate2("dcu1", "ipg_bus", CCM_CCGR9, CCM_CCGRx_CGn(8));
+
+ clk[VF610_CLK_TCON0] = imx_clk_gate2("tcon0", "platform_bus", CCM_CCGR1, CCM_CCGRx_CGn(13));
+ clk[VF610_CLK_TCON1] = imx_clk_gate2("tcon1", "platform_bus", CCM_CCGR7, CCM_CCGRx_CGn(13));
clk[VF610_CLK_ESAI_SEL] = imx_clk_mux("esai_sel", CCM_CSCMR1, 20, 2, esai_sels, 4);
clk[VF610_CLK_ESAI_EN] = imx_clk_gate("esai_en", "esai_sel", CCM_CSCDR2, 30);
@@ -409,6 +463,8 @@
for (i = 0; i < ARRAY_SIZE(clks_init_on); i++)
clk_prepare_enable(clk[clks_init_on[i]]);
+ register_syscore_ops(&vf610_clk_syscore_ops);
+
/* Add the clocks to provider list */
clk_data.clks = clk;
clk_data.clk_num = ARRAY_SIZE(clk);
diff --git a/drivers/clk/imx/clk.h b/drivers/clk/imx/clk.h
index d942f57..508d0fa 100644
--- a/drivers/clk/imx/clk.h
+++ b/drivers/clk/imx/clk.h
@@ -41,7 +41,7 @@
struct clk *clk_register_gate2(struct device *dev, const char *name,
const char *parent_name, unsigned long flags,
- void __iomem *reg, u8 bit_idx,
+ void __iomem *reg, u8 bit_idx, u8 cgr_val,
u8 clk_gate_flags, spinlock_t *lock,
unsigned int *share_count);
@@ -55,7 +55,7 @@
void __iomem *reg, u8 shift)
{
return clk_register_gate2(NULL, name, parent, CLK_SET_RATE_PARENT, reg,
- shift, 0, &imx_ccm_lock, NULL);
+ shift, 0x3, 0, &imx_ccm_lock, NULL);
}
static inline struct clk *imx_clk_gate2_shared(const char *name,
@@ -63,7 +63,14 @@
unsigned int *share_count)
{
return clk_register_gate2(NULL, name, parent, CLK_SET_RATE_PARENT, reg,
- shift, 0, &imx_ccm_lock, share_count);
+ shift, 0x3, 0, &imx_ccm_lock, share_count);
+}
+
+static inline struct clk *imx_clk_gate2_cgr(const char *name, const char *parent,
+ void __iomem *reg, u8 shift, u8 cgr_val)
+{
+ return clk_register_gate2(NULL, name, parent, CLK_SET_RATE_PARENT, reg,
+ shift, cgr_val, 0, &imx_ccm_lock, NULL);
}
struct clk *imx_clk_pfd(const char *name, const char *parent_name,
diff --git a/drivers/clk/ingenic/cgu.c b/drivers/clk/ingenic/cgu.c
index 7cfb7b2..e8248f9 100644
--- a/drivers/clk/ingenic/cgu.c
+++ b/drivers/clk/ingenic/cgu.c
@@ -325,6 +325,7 @@
div = (div_reg >> clk_info->div.shift) &
GENMASK(clk_info->div.bits - 1, 0);
div += 1;
+ div *= clk_info->div.div;
rate /= div;
}
@@ -345,6 +346,14 @@
div = min_t(unsigned, div, 1 << clk_info->div.bits);
div = max_t(unsigned, div, 1);
+ /*
+ * If the divider value itself must be divided before being written to
+ * the divider register, we must ensure we don't have any bits set that
+ * would be lost as a result of doing so.
+ */
+ div /= clk_info->div.div;
+ div *= clk_info->div.div;
+
return div;
}
@@ -395,7 +404,7 @@
/* update the divide */
mask = GENMASK(clk_info->div.bits - 1, 0);
reg &= ~(mask << clk_info->div.shift);
- reg |= (div - 1) << clk_info->div.shift;
+ reg |= ((div / clk_info->div.div) - 1) << clk_info->div.shift;
/* clear the stop bit */
if (clk_info->div.stop_bit != -1)
diff --git a/drivers/clk/ingenic/cgu.h b/drivers/clk/ingenic/cgu.h
index 99347e2..09700b2 100644
--- a/drivers/clk/ingenic/cgu.h
+++ b/drivers/clk/ingenic/cgu.h
@@ -76,8 +76,11 @@
/**
* struct ingenic_cgu_div_info - information about a divider
* @reg: offset of the divider control register within the CGU
- * @shift: number of bits to shift the divide value by (ie. the index of
+ * @shift: number of bits to left shift the divide value by (ie. the index of
* the lowest bit of the divide value within its control register)
+ * @div: number of bits to divide the divider value by (i.e. if the
+ * effective divider value is the value written to the register
+ * multiplied by some constant)
* @bits: the size of the divide value in bits
* @ce_bit: the index of the change enable bit within reg, or -1 if there
* isn't one
@@ -87,6 +90,7 @@
struct ingenic_cgu_div_info {
unsigned reg;
u8 shift;
+ u8 div;
u8 bits;
s8 ce_bit;
s8 busy_bit;
diff --git a/drivers/clk/ingenic/jz4740-cgu.c b/drivers/clk/ingenic/jz4740-cgu.c
index 305a26c2..510fe7e 100644
--- a/drivers/clk/ingenic/jz4740-cgu.c
+++ b/drivers/clk/ingenic/jz4740-cgu.c
@@ -90,51 +90,51 @@
[JZ4740_CLK_PLL_HALF] = {
"pll half", CGU_CLK_DIV,
.parents = { JZ4740_CLK_PLL, -1, -1, -1 },
- .div = { CGU_REG_CPCCR, 21, 1, -1, -1, -1 },
+ .div = { CGU_REG_CPCCR, 21, 1, 1, -1, -1, -1 },
},
[JZ4740_CLK_CCLK] = {
"cclk", CGU_CLK_DIV,
.parents = { JZ4740_CLK_PLL, -1, -1, -1 },
- .div = { CGU_REG_CPCCR, 0, 4, 22, -1, -1 },
+ .div = { CGU_REG_CPCCR, 0, 1, 4, 22, -1, -1 },
},
[JZ4740_CLK_HCLK] = {
"hclk", CGU_CLK_DIV,
.parents = { JZ4740_CLK_PLL, -1, -1, -1 },
- .div = { CGU_REG_CPCCR, 4, 4, 22, -1, -1 },
+ .div = { CGU_REG_CPCCR, 4, 1, 4, 22, -1, -1 },
},
[JZ4740_CLK_PCLK] = {
"pclk", CGU_CLK_DIV,
.parents = { JZ4740_CLK_PLL, -1, -1, -1 },
- .div = { CGU_REG_CPCCR, 8, 4, 22, -1, -1 },
+ .div = { CGU_REG_CPCCR, 8, 1, 4, 22, -1, -1 },
},
[JZ4740_CLK_MCLK] = {
"mclk", CGU_CLK_DIV,
.parents = { JZ4740_CLK_PLL, -1, -1, -1 },
- .div = { CGU_REG_CPCCR, 12, 4, 22, -1, -1 },
+ .div = { CGU_REG_CPCCR, 12, 1, 4, 22, -1, -1 },
},
[JZ4740_CLK_LCD] = {
"lcd", CGU_CLK_DIV | CGU_CLK_GATE,
.parents = { JZ4740_CLK_PLL_HALF, -1, -1, -1 },
- .div = { CGU_REG_CPCCR, 16, 5, 22, -1, -1 },
+ .div = { CGU_REG_CPCCR, 16, 1, 5, 22, -1, -1 },
.gate = { CGU_REG_CLKGR, 10 },
},
[JZ4740_CLK_LCD_PCLK] = {
"lcd_pclk", CGU_CLK_DIV,
.parents = { JZ4740_CLK_PLL_HALF, -1, -1, -1 },
- .div = { CGU_REG_LPCDR, 0, 11, -1, -1, -1 },
+ .div = { CGU_REG_LPCDR, 0, 1, 11, -1, -1, -1 },
},
[JZ4740_CLK_I2S] = {
"i2s", CGU_CLK_MUX | CGU_CLK_DIV | CGU_CLK_GATE,
.parents = { JZ4740_CLK_EXT, JZ4740_CLK_PLL_HALF, -1, -1 },
.mux = { CGU_REG_CPCCR, 31, 1 },
- .div = { CGU_REG_I2SCDR, 0, 8, -1, -1, -1 },
+ .div = { CGU_REG_I2SCDR, 0, 1, 8, -1, -1, -1 },
.gate = { CGU_REG_CLKGR, 6 },
},
@@ -142,21 +142,21 @@
"spi", CGU_CLK_MUX | CGU_CLK_DIV | CGU_CLK_GATE,
.parents = { JZ4740_CLK_EXT, JZ4740_CLK_PLL, -1, -1 },
.mux = { CGU_REG_SSICDR, 31, 1 },
- .div = { CGU_REG_SSICDR, 0, 4, -1, -1, -1 },
+ .div = { CGU_REG_SSICDR, 0, 1, 4, -1, -1, -1 },
.gate = { CGU_REG_CLKGR, 4 },
},
[JZ4740_CLK_MMC] = {
"mmc", CGU_CLK_DIV | CGU_CLK_GATE,
.parents = { JZ4740_CLK_PLL_HALF, -1, -1, -1 },
- .div = { CGU_REG_MSCCDR, 0, 5, -1, -1, -1 },
+ .div = { CGU_REG_MSCCDR, 0, 1, 5, -1, -1, -1 },
.gate = { CGU_REG_CLKGR, 7 },
},
[JZ4740_CLK_UHC] = {
"uhc", CGU_CLK_DIV | CGU_CLK_GATE,
.parents = { JZ4740_CLK_PLL_HALF, -1, -1, -1 },
- .div = { CGU_REG_UHCCDR, 0, 4, -1, -1, -1 },
+ .div = { CGU_REG_UHCCDR, 0, 1, 4, -1, -1, -1 },
.gate = { CGU_REG_CLKGR, 14 },
},
@@ -164,7 +164,7 @@
"udc", CGU_CLK_MUX | CGU_CLK_DIV,
.parents = { JZ4740_CLK_EXT, JZ4740_CLK_PLL_HALF, -1, -1 },
.mux = { CGU_REG_CPCCR, 29, 1 },
- .div = { CGU_REG_CPCCR, 23, 6, -1, -1, -1 },
+ .div = { CGU_REG_CPCCR, 23, 1, 6, -1, -1, -1 },
.gate = { CGU_REG_SCR, 6 },
},
diff --git a/drivers/clk/ingenic/jz4780-cgu.c b/drivers/clk/ingenic/jz4780-cgu.c
index 431f962..b35d6d9 100644
--- a/drivers/clk/ingenic/jz4780-cgu.c
+++ b/drivers/clk/ingenic/jz4780-cgu.c
@@ -296,13 +296,13 @@
[JZ4780_CLK_CPU] = {
"cpu", CGU_CLK_DIV,
.parents = { JZ4780_CLK_CPUMUX, -1, -1, -1 },
- .div = { CGU_REG_CLOCKCONTROL, 0, 4, 22, -1, -1 },
+ .div = { CGU_REG_CLOCKCONTROL, 0, 1, 4, 22, -1, -1 },
},
[JZ4780_CLK_L2CACHE] = {
"l2cache", CGU_CLK_DIV,
.parents = { JZ4780_CLK_CPUMUX, -1, -1, -1 },
- .div = { CGU_REG_CLOCKCONTROL, 4, 4, -1, -1, -1 },
+ .div = { CGU_REG_CLOCKCONTROL, 4, 1, 4, -1, -1, -1 },
},
[JZ4780_CLK_AHB0] = {
@@ -310,7 +310,7 @@
.parents = { -1, JZ4780_CLK_SCLKA, JZ4780_CLK_MPLL,
JZ4780_CLK_EPLL },
.mux = { CGU_REG_CLOCKCONTROL, 26, 2 },
- .div = { CGU_REG_CLOCKCONTROL, 8, 4, 21, -1, -1 },
+ .div = { CGU_REG_CLOCKCONTROL, 8, 1, 4, 21, -1, -1 },
},
[JZ4780_CLK_AHB2PMUX] = {
@@ -323,20 +323,20 @@
[JZ4780_CLK_AHB2] = {
"ahb2", CGU_CLK_DIV,
.parents = { JZ4780_CLK_AHB2PMUX, -1, -1, -1 },
- .div = { CGU_REG_CLOCKCONTROL, 12, 4, 20, -1, -1 },
+ .div = { CGU_REG_CLOCKCONTROL, 12, 1, 4, 20, -1, -1 },
},
[JZ4780_CLK_PCLK] = {
"pclk", CGU_CLK_DIV,
.parents = { JZ4780_CLK_AHB2PMUX, -1, -1, -1 },
- .div = { CGU_REG_CLOCKCONTROL, 16, 4, 20, -1, -1 },
+ .div = { CGU_REG_CLOCKCONTROL, 16, 1, 4, 20, -1, -1 },
},
[JZ4780_CLK_DDR] = {
"ddr", CGU_CLK_MUX | CGU_CLK_DIV,
.parents = { -1, JZ4780_CLK_SCLKA, JZ4780_CLK_MPLL, -1 },
.mux = { CGU_REG_DDRCDR, 30, 2 },
- .div = { CGU_REG_DDRCDR, 0, 4, 29, 28, 27 },
+ .div = { CGU_REG_DDRCDR, 0, 1, 4, 29, 28, 27 },
},
[JZ4780_CLK_VPU] = {
@@ -344,7 +344,7 @@
.parents = { JZ4780_CLK_SCLKA, JZ4780_CLK_MPLL,
JZ4780_CLK_EPLL, -1 },
.mux = { CGU_REG_VPUCDR, 30, 2 },
- .div = { CGU_REG_VPUCDR, 0, 4, 29, 28, 27 },
+ .div = { CGU_REG_VPUCDR, 0, 1, 4, 29, 28, 27 },
.gate = { CGU_REG_CLKGR1, 2 },
},
@@ -352,7 +352,7 @@
"i2s_pll", CGU_CLK_MUX | CGU_CLK_DIV,
.parents = { JZ4780_CLK_SCLKA, JZ4780_CLK_EPLL, -1, -1 },
.mux = { CGU_REG_I2SCDR, 30, 1 },
- .div = { CGU_REG_I2SCDR, 0, 8, 29, 28, 27 },
+ .div = { CGU_REG_I2SCDR, 0, 1, 8, 29, 28, 27 },
},
[JZ4780_CLK_I2S] = {
@@ -366,7 +366,7 @@
.parents = { JZ4780_CLK_SCLKA, JZ4780_CLK_MPLL,
JZ4780_CLK_VPLL, -1 },
.mux = { CGU_REG_LP0CDR, 30, 2 },
- .div = { CGU_REG_LP0CDR, 0, 8, 28, 27, 26 },
+ .div = { CGU_REG_LP0CDR, 0, 1, 8, 28, 27, 26 },
},
[JZ4780_CLK_LCD1PIXCLK] = {
@@ -374,7 +374,7 @@
.parents = { JZ4780_CLK_SCLKA, JZ4780_CLK_MPLL,
JZ4780_CLK_VPLL, -1 },
.mux = { CGU_REG_LP1CDR, 30, 2 },
- .div = { CGU_REG_LP1CDR, 0, 8, 28, 27, 26 },
+ .div = { CGU_REG_LP1CDR, 0, 1, 8, 28, 27, 26 },
},
[JZ4780_CLK_MSCMUX] = {
@@ -386,21 +386,21 @@
[JZ4780_CLK_MSC0] = {
"msc0", CGU_CLK_DIV | CGU_CLK_GATE,
.parents = { JZ4780_CLK_MSCMUX, -1, -1, -1 },
- .div = { CGU_REG_MSC0CDR, 0, 8, 29, 28, 27 },
+ .div = { CGU_REG_MSC0CDR, 0, 2, 8, 29, 28, 27 },
.gate = { CGU_REG_CLKGR0, 3 },
},
[JZ4780_CLK_MSC1] = {
"msc1", CGU_CLK_DIV | CGU_CLK_GATE,
.parents = { JZ4780_CLK_MSCMUX, -1, -1, -1 },
- .div = { CGU_REG_MSC1CDR, 0, 8, 29, 28, 27 },
+ .div = { CGU_REG_MSC1CDR, 0, 2, 8, 29, 28, 27 },
.gate = { CGU_REG_CLKGR0, 11 },
},
[JZ4780_CLK_MSC2] = {
"msc2", CGU_CLK_DIV | CGU_CLK_GATE,
.parents = { JZ4780_CLK_MSCMUX, -1, -1, -1 },
- .div = { CGU_REG_MSC2CDR, 0, 8, 29, 28, 27 },
+ .div = { CGU_REG_MSC2CDR, 0, 2, 8, 29, 28, 27 },
.gate = { CGU_REG_CLKGR0, 12 },
},
@@ -409,7 +409,7 @@
.parents = { JZ4780_CLK_SCLKA, JZ4780_CLK_MPLL,
JZ4780_CLK_EPLL, JZ4780_CLK_OTGPHY },
.mux = { CGU_REG_UHCCDR, 30, 2 },
- .div = { CGU_REG_UHCCDR, 0, 8, 29, 28, 27 },
+ .div = { CGU_REG_UHCCDR, 0, 1, 8, 29, 28, 27 },
.gate = { CGU_REG_CLKGR0, 24 },
},
@@ -417,7 +417,7 @@
"ssi_pll", CGU_CLK_MUX | CGU_CLK_DIV,
.parents = { JZ4780_CLK_SCLKA, JZ4780_CLK_MPLL, -1, -1 },
.mux = { CGU_REG_SSICDR, 30, 1 },
- .div = { CGU_REG_SSICDR, 0, 8, 29, 28, 27 },
+ .div = { CGU_REG_SSICDR, 0, 1, 8, 29, 28, 27 },
},
[JZ4780_CLK_SSI] = {
@@ -430,7 +430,7 @@
"cim_mclk", CGU_CLK_MUX | CGU_CLK_DIV,
.parents = { JZ4780_CLK_SCLKA, JZ4780_CLK_MPLL, -1, -1 },
.mux = { CGU_REG_CIMCDR, 31, 1 },
- .div = { CGU_REG_CIMCDR, 0, 8, 30, 29, 28 },
+ .div = { CGU_REG_CIMCDR, 0, 1, 8, 30, 29, 28 },
},
[JZ4780_CLK_PCMPLL] = {
@@ -438,7 +438,7 @@
.parents = { JZ4780_CLK_SCLKA, JZ4780_CLK_MPLL,
JZ4780_CLK_EPLL, JZ4780_CLK_VPLL },
.mux = { CGU_REG_PCMCDR, 29, 2 },
- .div = { CGU_REG_PCMCDR, 0, 8, 28, 27, 26 },
+ .div = { CGU_REG_PCMCDR, 0, 1, 8, 28, 27, 26 },
},
[JZ4780_CLK_PCM] = {
@@ -453,7 +453,7 @@
.parents = { -1, JZ4780_CLK_SCLKA, JZ4780_CLK_MPLL,
JZ4780_CLK_EPLL },
.mux = { CGU_REG_GPUCDR, 30, 2 },
- .div = { CGU_REG_GPUCDR, 0, 4, 29, 28, 27 },
+ .div = { CGU_REG_GPUCDR, 0, 1, 4, 29, 28, 27 },
.gate = { CGU_REG_CLKGR1, 4 },
},
@@ -462,7 +462,7 @@
.parents = { JZ4780_CLK_SCLKA, JZ4780_CLK_MPLL,
JZ4780_CLK_VPLL, -1 },
.mux = { CGU_REG_HDMICDR, 30, 2 },
- .div = { CGU_REG_HDMICDR, 0, 8, 29, 28, 26 },
+ .div = { CGU_REG_HDMICDR, 0, 1, 8, 29, 28, 26 },
.gate = { CGU_REG_CLKGR1, 9 },
},
@@ -471,7 +471,7 @@
.parents = { -1, JZ4780_CLK_SCLKA, JZ4780_CLK_MPLL,
JZ4780_CLK_EPLL },
.mux = { CGU_REG_BCHCDR, 30, 2 },
- .div = { CGU_REG_BCHCDR, 0, 4, 29, 28, 27 },
+ .div = { CGU_REG_BCHCDR, 0, 1, 4, 29, 28, 27 },
.gate = { CGU_REG_CLKGR0, 1 },
},
diff --git a/drivers/clk/mediatek/clk-mt8173.c b/drivers/clk/mediatek/clk-mt8173.c
index 227e356..10c9860 100644
--- a/drivers/clk/mediatek/clk-mt8173.c
+++ b/drivers/clk/mediatek/clk-mt8173.c
@@ -61,7 +61,6 @@
FACTOR(CLK_TOP_CLKRTC_INT, "clkrtc_int", "clk26m", 1, 793),
FACTOR(CLK_TOP_FPC, "fpc_ck", "clk26m", 1, 1),
- FACTOR(CLK_TOP_HDMITX_DIG_CTS, "hdmitx_dig_cts", "tvdpll_445p5m", 1, 3),
FACTOR(CLK_TOP_HDMITXPLL_D2, "hdmitxpll_d2", "hdmitx_dig_cts", 1, 2),
FACTOR(CLK_TOP_HDMITXPLL_D3, "hdmitxpll_d3", "hdmitx_dig_cts", 1, 3),
@@ -558,7 +557,11 @@
MUX_GATE(CLK_TOP_ATB_SEL, "atb_sel", atb_parents, 0x0090, 16, 2, 23),
MUX_GATE(CLK_TOP_VENC_LT_SEL, "venclt_sel", venc_lt_parents, 0x0090, 24, 4, 31),
/* CLK_CFG_6 */
- MUX_GATE(CLK_TOP_DPI0_SEL, "dpi0_sel", dpi0_parents, 0x00a0, 0, 3, 7),
+ /*
+ * The dpi0_sel clock should not propagate rate changes to its parent
+ * clock so the dpi driver can have full control over PLL and divider.
+ */
+ MUX_GATE_FLAGS(CLK_TOP_DPI0_SEL, "dpi0_sel", dpi0_parents, 0x00a0, 0, 3, 7, 0),
MUX_GATE(CLK_TOP_IRDA_SEL, "irda_sel", irda_parents, 0x00a0, 8, 2, 15),
MUX_GATE(CLK_TOP_CCI400_SEL, "cci400_sel", cci400_parents, 0x00a0, 16, 3, 23),
MUX_GATE(CLK_TOP_AUD_1_SEL, "aud_1_sel", aud_1_parents, 0x00a0, 24, 2, 31),
@@ -1091,6 +1094,11 @@
clk_data->clks[cku->id] = clk;
}
+ clk = clk_register_divider(NULL, "hdmi_ref", "tvdpll_594m", 0,
+ base + 0x40, 16, 3, CLK_DIVIDER_POWER_OF_TWO,
+ NULL);
+ clk_data->clks[CLK_APMIXED_HDMI_REF] = clk;
+
r = of_clk_add_provider(node, of_clk_src_onecell_get, clk_data);
if (r)
pr_err("%s(): could not register clock provider: %d\n",
diff --git a/drivers/clk/mediatek/clk-mtk.h b/drivers/clk/mediatek/clk-mtk.h
index 32d2e45..9f24fcf 100644
--- a/drivers/clk/mediatek/clk-mtk.h
+++ b/drivers/clk/mediatek/clk-mtk.h
@@ -83,7 +83,11 @@
signed char num_parents;
};
-#define MUX_GATE(_id, _name, _parents, _reg, _shift, _width, _gate) { \
+/*
+ * In case the rate change propagation to parent clocks is undesirable,
+ * this macro allows to specify the clock flags manually.
+ */
+#define MUX_GATE_FLAGS(_id, _name, _parents, _reg, _shift, _width, _gate, _flags) { \
.id = _id, \
.name = _name, \
.mux_reg = _reg, \
@@ -94,9 +98,16 @@
.divider_shift = -1, \
.parent_names = _parents, \
.num_parents = ARRAY_SIZE(_parents), \
- .flags = CLK_SET_RATE_PARENT, \
+ .flags = _flags, \
}
+/*
+ * Unless necessary, all MUX_GATE clocks propagate rate changes to their
+ * parent clock by default.
+ */
+#define MUX_GATE(_id, _name, _parents, _reg, _shift, _width, _gate) \
+ MUX_GATE_FLAGS(_id, _name, _parents, _reg, _shift, _width, _gate, CLK_SET_RATE_PARENT)
+
#define MUX(_id, _name, _parents, _reg, _shift, _width) { \
.id = _id, \
.name = _name, \
diff --git a/drivers/clk/meson/meson8b-clkc.c b/drivers/clk/meson/meson8b-clkc.c
index 61f6d55..4d057b3 100644
--- a/drivers/clk/meson/meson8b-clkc.c
+++ b/drivers/clk/meson/meson8b-clkc.c
@@ -141,11 +141,11 @@
};
static const struct clk_conf meson8b_xtal_conf __initconst =
- FIXED_RATE_P(MESON8B_REG_CTL0_ADDR, CLKID_XTAL, "xtal",
- CLK_IS_ROOT, PARM(0x00, 4, 7));
+ FIXED_RATE_P(MESON8B_REG_CTL0_ADDR, CLKID_XTAL, "xtal", 0,
+ PARM(0x00, 4, 7));
static const struct clk_conf meson8b_clk_confs[] __initconst = {
- FIXED_RATE(CLKID_ZERO, "zero", CLK_IS_ROOT, 0),
+ FIXED_RATE(CLKID_ZERO, "zero", 0, 0),
PLL(MESON8B_REG_PLL_FIXED, CLKID_PLL_FIXED, "fixed_pll",
p_xtal, 0, &pll_confs),
PLL(MESON8B_REG_PLL_VID, CLKID_PLL_VID, "vid_pll",
diff --git a/drivers/clk/microchip/Makefile b/drivers/clk/microchip/Makefile
new file mode 100644
index 0000000..2152f41
--- /dev/null
+++ b/drivers/clk/microchip/Makefile
@@ -0,0 +1,2 @@
+obj-$(CONFIG_COMMON_CLK_PIC32) += clk-core.o
+obj-$(CONFIG_PIC32MZDA) += clk-pic32mzda.o
diff --git a/drivers/clk/microchip/clk-core.c b/drivers/clk/microchip/clk-core.c
new file mode 100644
index 0000000..ca85cea
--- /dev/null
+++ b/drivers/clk/microchip/clk-core.c
@@ -0,0 +1,1031 @@
+/*
+ * Purna Chandra Mandal,<purna.mandal@microchip.com>
+ * Copyright (C) 2015 Microchip Technology Inc. All rights reserved.
+ *
+ * This program is free software; you can distribute it and/or modify it
+ * under the terms of the GNU General Public License (Version 2) as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+ * for more details.
+ */
+#include <linux/clk-provider.h>
+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/interrupt.h>
+#include <linux/iopoll.h>
+#include <asm/mach-pic32/pic32.h>
+#include <asm/traps.h>
+
+#include "clk-core.h"
+
+/* OSCCON Reg fields */
+#define OSC_CUR_MASK 0x07
+#define OSC_CUR_SHIFT 12
+#define OSC_NEW_MASK 0x07
+#define OSC_NEW_SHIFT 8
+#define OSC_SWEN BIT(0)
+
+/* SPLLCON Reg fields */
+#define PLL_RANGE_MASK 0x07
+#define PLL_RANGE_SHIFT 0
+#define PLL_ICLK_MASK 0x01
+#define PLL_ICLK_SHIFT 7
+#define PLL_IDIV_MASK 0x07
+#define PLL_IDIV_SHIFT 8
+#define PLL_ODIV_MASK 0x07
+#define PLL_ODIV_SHIFT 24
+#define PLL_MULT_MASK 0x7F
+#define PLL_MULT_SHIFT 16
+#define PLL_MULT_MAX 128
+#define PLL_ODIV_MIN 1
+#define PLL_ODIV_MAX 5
+
+/* Peripheral Bus Clock Reg Fields */
+#define PB_DIV_MASK 0x7f
+#define PB_DIV_SHIFT 0
+#define PB_DIV_READY BIT(11)
+#define PB_DIV_ENABLE BIT(15)
+#define PB_DIV_MAX 128
+#define PB_DIV_MIN 0
+
+/* Reference Oscillator Control Reg fields */
+#define REFO_SEL_MASK 0x0f
+#define REFO_SEL_SHIFT 0
+#define REFO_ACTIVE BIT(8)
+#define REFO_DIVSW_EN BIT(9)
+#define REFO_OE BIT(12)
+#define REFO_ON BIT(15)
+#define REFO_DIV_SHIFT 16
+#define REFO_DIV_MASK 0x7fff
+
+/* Reference Oscillator Trim Register Fields */
+#define REFO_TRIM_REG 0x10
+#define REFO_TRIM_MASK 0x1ff
+#define REFO_TRIM_SHIFT 23
+#define REFO_TRIM_MAX 511
+
+/* Mux Slew Control Register fields */
+#define SLEW_BUSY BIT(0)
+#define SLEW_DOWNEN BIT(1)
+#define SLEW_UPEN BIT(2)
+#define SLEW_DIV 0x07
+#define SLEW_DIV_SHIFT 8
+#define SLEW_SYSDIV 0x0f
+#define SLEW_SYSDIV_SHIFT 20
+
+/* Clock Poll Timeout */
+#define LOCK_TIMEOUT_US USEC_PER_MSEC
+
+/* SoC specific clock needed during SPLL clock rate switch */
+static struct clk_hw *pic32_sclk_hw;
+
+/* add instruction pipeline delay while CPU clock is in-transition. */
+#define cpu_nop5() \
+do { \
+ __asm__ __volatile__("nop"); \
+ __asm__ __volatile__("nop"); \
+ __asm__ __volatile__("nop"); \
+ __asm__ __volatile__("nop"); \
+ __asm__ __volatile__("nop"); \
+} while (0)
+
+/* Perpheral bus clocks */
+struct pic32_periph_clk {
+ struct clk_hw hw;
+ void __iomem *ctrl_reg;
+ struct pic32_clk_common *core;
+};
+
+#define clkhw_to_pbclk(_hw) container_of(_hw, struct pic32_periph_clk, hw)
+
+static int pbclk_is_enabled(struct clk_hw *hw)
+{
+ struct pic32_periph_clk *pb = clkhw_to_pbclk(hw);
+
+ return readl(pb->ctrl_reg) & PB_DIV_ENABLE;
+}
+
+static int pbclk_enable(struct clk_hw *hw)
+{
+ struct pic32_periph_clk *pb = clkhw_to_pbclk(hw);
+
+ writel(PB_DIV_ENABLE, PIC32_SET(pb->ctrl_reg));
+ return 0;
+}
+
+static void pbclk_disable(struct clk_hw *hw)
+{
+ struct pic32_periph_clk *pb = clkhw_to_pbclk(hw);
+
+ writel(PB_DIV_ENABLE, PIC32_CLR(pb->ctrl_reg));
+}
+
+static unsigned long calc_best_divided_rate(unsigned long rate,
+ unsigned long parent_rate,
+ u32 divider_max,
+ u32 divider_min)
+{
+ unsigned long divided_rate, divided_rate_down, best_rate;
+ unsigned long div, div_up;
+
+ /* eq. clk_rate = parent_rate / divider.
+ *
+ * Find best divider to produce closest of target divided rate.
+ */
+ div = parent_rate / rate;
+ div = clamp_val(div, divider_min, divider_max);
+ div_up = clamp_val(div + 1, divider_min, divider_max);
+
+ divided_rate = parent_rate / div;
+ divided_rate_down = parent_rate / div_up;
+ if (abs(rate - divided_rate_down) < abs(rate - divided_rate))
+ best_rate = divided_rate_down;
+ else
+ best_rate = divided_rate;
+
+ return best_rate;
+}
+
+static inline u32 pbclk_read_pbdiv(struct pic32_periph_clk *pb)
+{
+ return ((readl(pb->ctrl_reg) >> PB_DIV_SHIFT) & PB_DIV_MASK) + 1;
+}
+
+static unsigned long pbclk_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct pic32_periph_clk *pb = clkhw_to_pbclk(hw);
+
+ return parent_rate / pbclk_read_pbdiv(pb);
+}
+
+static long pbclk_round_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long *parent_rate)
+{
+ return calc_best_divided_rate(rate, *parent_rate,
+ PB_DIV_MAX, PB_DIV_MIN);
+}
+
+static int pbclk_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct pic32_periph_clk *pb = clkhw_to_pbclk(hw);
+ unsigned long flags;
+ u32 v, div;
+ int err;
+
+ /* check & wait for DIV_READY */
+ err = readl_poll_timeout(pb->ctrl_reg, v, v & PB_DIV_READY,
+ 1, LOCK_TIMEOUT_US);
+ if (err)
+ return err;
+
+ /* calculate clkdiv and best rate */
+ div = DIV_ROUND_CLOSEST(parent_rate, rate);
+
+ spin_lock_irqsave(&pb->core->reg_lock, flags);
+
+ /* apply new div */
+ v = readl(pb->ctrl_reg);
+ v &= ~PB_DIV_MASK;
+ v |= (div - 1);
+
+ pic32_syskey_unlock();
+
+ writel(v, pb->ctrl_reg);
+
+ spin_unlock_irqrestore(&pb->core->reg_lock, flags);
+
+ /* wait again, for pbdivready */
+ err = readl_poll_timeout_atomic(pb->ctrl_reg, v, v & PB_DIV_READY,
+ 1, LOCK_TIMEOUT_US);
+ if (err)
+ return err;
+
+ /* confirm that new div is applied correctly */
+ return (pbclk_read_pbdiv(pb) == div) ? 0 : -EBUSY;
+}
+
+const struct clk_ops pic32_pbclk_ops = {
+ .enable = pbclk_enable,
+ .disable = pbclk_disable,
+ .is_enabled = pbclk_is_enabled,
+ .recalc_rate = pbclk_recalc_rate,
+ .round_rate = pbclk_round_rate,
+ .set_rate = pbclk_set_rate,
+};
+
+struct clk *pic32_periph_clk_register(const struct pic32_periph_clk_data *desc,
+ struct pic32_clk_common *core)
+{
+ struct pic32_periph_clk *pbclk;
+ struct clk *clk;
+
+ pbclk = devm_kzalloc(core->dev, sizeof(*pbclk), GFP_KERNEL);
+ if (!pbclk)
+ return ERR_PTR(-ENOMEM);
+
+ pbclk->hw.init = &desc->init_data;
+ pbclk->core = core;
+ pbclk->ctrl_reg = desc->ctrl_reg + core->iobase;
+
+ clk = devm_clk_register(core->dev, &pbclk->hw);
+ if (IS_ERR(clk)) {
+ dev_err(core->dev, "%s: clk_register() failed\n", __func__);
+ devm_kfree(core->dev, pbclk);
+ }
+
+ return clk;
+}
+
+/* Reference oscillator operations */
+struct pic32_ref_osc {
+ struct clk_hw hw;
+ void __iomem *ctrl_reg;
+ const u32 *parent_map;
+ struct pic32_clk_common *core;
+};
+
+#define clkhw_to_refosc(_hw) container_of(_hw, struct pic32_ref_osc, hw)
+
+static int roclk_is_enabled(struct clk_hw *hw)
+{
+ struct pic32_ref_osc *refo = clkhw_to_refosc(hw);
+
+ return readl(refo->ctrl_reg) & REFO_ON;
+}
+
+static int roclk_enable(struct clk_hw *hw)
+{
+ struct pic32_ref_osc *refo = clkhw_to_refosc(hw);
+
+ writel(REFO_ON | REFO_OE, PIC32_SET(refo->ctrl_reg));
+ return 0;
+}
+
+static void roclk_disable(struct clk_hw *hw)
+{
+ struct pic32_ref_osc *refo = clkhw_to_refosc(hw);
+
+ writel(REFO_ON | REFO_OE, PIC32_CLR(refo->ctrl_reg));
+}
+
+static void roclk_init(struct clk_hw *hw)
+{
+ /* initialize clock in disabled state */
+ roclk_disable(hw);
+}
+
+static u8 roclk_get_parent(struct clk_hw *hw)
+{
+ struct pic32_ref_osc *refo = clkhw_to_refosc(hw);
+ u32 v, i;
+
+ v = (readl(refo->ctrl_reg) >> REFO_SEL_SHIFT) & REFO_SEL_MASK;
+
+ if (!refo->parent_map)
+ return v;
+
+ for (i = 0; i < clk_hw_get_num_parents(hw); i++)
+ if (refo->parent_map[i] == v)
+ return i;
+
+ return -EINVAL;
+}
+
+static unsigned long roclk_calc_rate(unsigned long parent_rate,
+ u32 rodiv, u32 rotrim)
+{
+ u64 rate64;
+
+ /* fout = fin / [2 * {div + (trim / 512)}]
+ * = fin * 512 / [1024 * div + 2 * trim]
+ * = fin * 256 / (512 * div + trim)
+ * = (fin << 8) / ((div << 9) + trim)
+ */
+ if (rotrim) {
+ rodiv = (rodiv << 9) + rotrim;
+ rate64 = parent_rate;
+ rate64 <<= 8;
+ do_div(rate64, rodiv);
+ } else if (rodiv) {
+ rate64 = parent_rate / (rodiv << 1);
+ } else {
+ rate64 = parent_rate;
+ }
+ return rate64;
+}
+
+static void roclk_calc_div_trim(unsigned long rate,
+ unsigned long parent_rate,
+ u32 *rodiv_p, u32 *rotrim_p)
+{
+ u32 div, rotrim, rodiv;
+ u64 frac;
+
+ /* Find integer approximation of floating-point arithmetic.
+ * fout = fin / [2 * {rodiv + (rotrim / 512)}] ... (1)
+ * i.e. fout = fin / 2 * DIV
+ * whereas DIV = rodiv + (rotrim / 512)
+ *
+ * Since kernel does not perform floating-point arithmatic so
+ * (rotrim/512) will be zero. And DIV & rodiv will result same.
+ *
+ * ie. fout = (fin * 256) / [(512 * rodiv) + rotrim] ... from (1)
+ * ie. rotrim = ((fin * 256) / fout) - (512 * DIV)
+ */
+ if (parent_rate <= rate) {
+ div = 0;
+ frac = 0;
+ rodiv = 0;
+ rotrim = 0;
+ } else {
+ div = parent_rate / (rate << 1);
+ frac = parent_rate;
+ frac <<= 8;
+ do_div(frac, rate);
+ frac -= (u64)(div << 9);
+
+ rodiv = (div > REFO_DIV_MASK) ? REFO_DIV_MASK : div;
+ rotrim = (frac >= REFO_TRIM_MAX) ? REFO_TRIM_MAX : frac;
+ }
+
+ if (rodiv_p)
+ *rodiv_p = rodiv;
+
+ if (rotrim_p)
+ *rotrim_p = rotrim;
+}
+
+static unsigned long roclk_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct pic32_ref_osc *refo = clkhw_to_refosc(hw);
+ u32 v, rodiv, rotrim;
+
+ /* get rodiv */
+ v = readl(refo->ctrl_reg);
+ rodiv = (v >> REFO_DIV_SHIFT) & REFO_DIV_MASK;
+
+ /* get trim */
+ v = readl(refo->ctrl_reg + REFO_TRIM_REG);
+ rotrim = (v >> REFO_TRIM_SHIFT) & REFO_TRIM_MASK;
+
+ return roclk_calc_rate(parent_rate, rodiv, rotrim);
+}
+
+static long roclk_round_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long *parent_rate)
+{
+ u32 rotrim, rodiv;
+
+ /* calculate dividers for new rate */
+ roclk_calc_div_trim(rate, *parent_rate, &rodiv, &rotrim);
+
+ /* caclulate new rate (rounding) based on new rodiv & rotrim */
+ return roclk_calc_rate(*parent_rate, rodiv, rotrim);
+}
+
+static int roclk_determine_rate(struct clk_hw *hw,
+ struct clk_rate_request *req)
+{
+ struct clk_hw *parent_clk, *best_parent_clk = NULL;
+ unsigned int i, delta, best_delta = -1;
+ unsigned long parent_rate, best_parent_rate = 0;
+ unsigned long best = 0, nearest_rate;
+
+ /* find a parent which can generate nearest clkrate >= rate */
+ for (i = 0; i < clk_hw_get_num_parents(hw); i++) {
+ /* get parent */
+ parent_clk = clk_hw_get_parent_by_index(hw, i);
+ if (!parent_clk)
+ continue;
+
+ /* skip if parent runs slower than target rate */
+ parent_rate = clk_hw_get_rate(parent_clk);
+ if (req->rate > parent_rate)
+ continue;
+
+ nearest_rate = roclk_round_rate(hw, req->rate, &parent_rate);
+ delta = abs(nearest_rate - req->rate);
+ if ((nearest_rate >= req->rate) && (delta < best_delta)) {
+ best_parent_clk = parent_clk;
+ best_parent_rate = parent_rate;
+ best = nearest_rate;
+ best_delta = delta;
+
+ if (delta == 0)
+ break;
+ }
+ }
+
+ /* if no match found, retain old rate */
+ if (!best_parent_clk) {
+ pr_err("%s:%s, no parent found for rate %lu.\n",
+ __func__, clk_hw_get_name(hw), req->rate);
+ return clk_hw_get_rate(hw);
+ }
+
+ pr_debug("%s,rate %lu, best_parent(%s, %lu), best %lu, delta %d\n",
+ clk_hw_get_name(hw), req->rate,
+ clk_hw_get_name(best_parent_clk), best_parent_rate,
+ best, best_delta);
+
+ if (req->best_parent_rate)
+ req->best_parent_rate = best_parent_rate;
+
+ if (req->best_parent_hw)
+ req->best_parent_hw = best_parent_clk;
+
+ return best;
+}
+
+static int roclk_set_parent(struct clk_hw *hw, u8 index)
+{
+ struct pic32_ref_osc *refo = clkhw_to_refosc(hw);
+ unsigned long flags;
+ u32 v;
+ int err;
+
+ if (refo->parent_map)
+ index = refo->parent_map[index];
+
+ /* wait until ACTIVE bit is zero or timeout */
+ err = readl_poll_timeout(refo->ctrl_reg, v, !(v & REFO_ACTIVE),
+ 1, LOCK_TIMEOUT_US);
+ if (err) {
+ pr_err("%s: poll failed, clk active\n", clk_hw_get_name(hw));
+ return err;
+ }
+
+ spin_lock_irqsave(&refo->core->reg_lock, flags);
+
+ pic32_syskey_unlock();
+
+ /* calculate & apply new */
+ v = readl(refo->ctrl_reg);
+ v &= ~(REFO_SEL_MASK << REFO_SEL_SHIFT);
+ v |= index << REFO_SEL_SHIFT;
+
+ writel(v, refo->ctrl_reg);
+
+ spin_unlock_irqrestore(&refo->core->reg_lock, flags);
+
+ return 0;
+}
+
+static int roclk_set_rate_and_parent(struct clk_hw *hw,
+ unsigned long rate,
+ unsigned long parent_rate,
+ u8 index)
+{
+ struct pic32_ref_osc *refo = clkhw_to_refosc(hw);
+ unsigned long flags;
+ u32 trim, rodiv, v;
+ int err;
+
+ /* calculate new rodiv & rotrim for new rate */
+ roclk_calc_div_trim(rate, parent_rate, &rodiv, &trim);
+
+ pr_debug("parent_rate = %lu, rate = %lu, div = %d, trim = %d\n",
+ parent_rate, rate, rodiv, trim);
+
+ /* wait till source change is active */
+ err = readl_poll_timeout(refo->ctrl_reg, v,
+ !(v & (REFO_ACTIVE | REFO_DIVSW_EN)),
+ 1, LOCK_TIMEOUT_US);
+ if (err) {
+ pr_err("%s: poll timedout, clock is still active\n", __func__);
+ return err;
+ }
+
+ spin_lock_irqsave(&refo->core->reg_lock, flags);
+ v = readl(refo->ctrl_reg);
+
+ pic32_syskey_unlock();
+
+ /* apply parent, if required */
+ if (refo->parent_map)
+ index = refo->parent_map[index];
+
+ v &= ~(REFO_SEL_MASK << REFO_SEL_SHIFT);
+ v |= index << REFO_SEL_SHIFT;
+
+ /* apply RODIV */
+ v &= ~(REFO_DIV_MASK << REFO_DIV_SHIFT);
+ v |= rodiv << REFO_DIV_SHIFT;
+ writel(v, refo->ctrl_reg);
+
+ /* apply ROTRIM */
+ v = readl(refo->ctrl_reg + REFO_TRIM_REG);
+ v &= ~(REFO_TRIM_MASK << REFO_TRIM_SHIFT);
+ v |= trim << REFO_TRIM_SHIFT;
+ writel(v, refo->ctrl_reg + REFO_TRIM_REG);
+
+ /* enable & activate divider switching */
+ writel(REFO_ON | REFO_DIVSW_EN, PIC32_SET(refo->ctrl_reg));
+
+ /* wait till divswen is in-progress */
+ err = readl_poll_timeout_atomic(refo->ctrl_reg, v, !(v & REFO_DIVSW_EN),
+ 1, LOCK_TIMEOUT_US);
+ /* leave the clk gated as it was */
+ writel(REFO_ON, PIC32_CLR(refo->ctrl_reg));
+
+ spin_unlock_irqrestore(&refo->core->reg_lock, flags);
+
+ return err;
+}
+
+static int roclk_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ u8 index = roclk_get_parent(hw);
+
+ return roclk_set_rate_and_parent(hw, rate, parent_rate, index);
+}
+
+const struct clk_ops pic32_roclk_ops = {
+ .enable = roclk_enable,
+ .disable = roclk_disable,
+ .is_enabled = roclk_is_enabled,
+ .get_parent = roclk_get_parent,
+ .set_parent = roclk_set_parent,
+ .determine_rate = roclk_determine_rate,
+ .recalc_rate = roclk_recalc_rate,
+ .set_rate_and_parent = roclk_set_rate_and_parent,
+ .set_rate = roclk_set_rate,
+ .init = roclk_init,
+};
+
+struct clk *pic32_refo_clk_register(const struct pic32_ref_osc_data *data,
+ struct pic32_clk_common *core)
+{
+ struct pic32_ref_osc *refo;
+ struct clk *clk;
+
+ refo = devm_kzalloc(core->dev, sizeof(*refo), GFP_KERNEL);
+ if (!refo)
+ return ERR_PTR(-ENOMEM);
+
+ refo->core = core;
+ refo->hw.init = &data->init_data;
+ refo->ctrl_reg = data->ctrl_reg + core->iobase;
+ refo->parent_map = data->parent_map;
+
+ clk = devm_clk_register(core->dev, &refo->hw);
+ if (IS_ERR(clk))
+ dev_err(core->dev, "%s: clk_register() failed\n", __func__);
+
+ return clk;
+}
+
+struct pic32_sys_pll {
+ struct clk_hw hw;
+ void __iomem *ctrl_reg;
+ void __iomem *status_reg;
+ u32 lock_mask;
+ u32 idiv; /* PLL iclk divider, treated fixed */
+ struct pic32_clk_common *core;
+};
+
+#define clkhw_to_spll(_hw) container_of(_hw, struct pic32_sys_pll, hw)
+
+static inline u32 spll_odiv_to_divider(u32 odiv)
+{
+ odiv = clamp_val(odiv, PLL_ODIV_MIN, PLL_ODIV_MAX);
+
+ return 1 << odiv;
+}
+
+static unsigned long spll_calc_mult_div(struct pic32_sys_pll *pll,
+ unsigned long rate,
+ unsigned long parent_rate,
+ u32 *mult_p, u32 *odiv_p)
+{
+ u32 mul, div, best_mul = 1, best_div = 1;
+ unsigned long new_rate, best_rate = rate;
+ unsigned int best_delta = -1, delta, match_found = 0;
+ u64 rate64;
+
+ parent_rate /= pll->idiv;
+
+ for (mul = 1; mul <= PLL_MULT_MAX; mul++) {
+ for (div = PLL_ODIV_MIN; div <= PLL_ODIV_MAX; div++) {
+ rate64 = parent_rate;
+ rate64 *= mul;
+ do_div(rate64, 1 << div);
+ new_rate = rate64;
+ delta = abs(rate - new_rate);
+ if ((new_rate >= rate) && (delta < best_delta)) {
+ best_delta = delta;
+ best_rate = new_rate;
+ best_mul = mul;
+ best_div = div;
+ match_found = 1;
+ }
+ }
+ }
+
+ if (!match_found) {
+ pr_warn("spll: no match found\n");
+ return 0;
+ }
+
+ pr_debug("rate %lu, par_rate %lu/mult %u, div %u, best_rate %lu\n",
+ rate, parent_rate, best_mul, best_div, best_rate);
+
+ if (mult_p)
+ *mult_p = best_mul - 1;
+
+ if (odiv_p)
+ *odiv_p = best_div;
+
+ return best_rate;
+}
+
+static unsigned long spll_clk_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct pic32_sys_pll *pll = clkhw_to_spll(hw);
+ unsigned long pll_in_rate;
+ u32 mult, odiv, div, v;
+ u64 rate64;
+
+ v = readl(pll->ctrl_reg);
+ odiv = ((v >> PLL_ODIV_SHIFT) & PLL_ODIV_MASK);
+ mult = ((v >> PLL_MULT_SHIFT) & PLL_MULT_MASK) + 1;
+ div = spll_odiv_to_divider(odiv);
+
+ /* pll_in_rate = parent_rate / idiv
+ * pll_out_rate = pll_in_rate * mult / div;
+ */
+ pll_in_rate = parent_rate / pll->idiv;
+ rate64 = pll_in_rate;
+ rate64 *= mult;
+ do_div(rate64, div);
+
+ return rate64;
+}
+
+static long spll_clk_round_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long *parent_rate)
+{
+ struct pic32_sys_pll *pll = clkhw_to_spll(hw);
+
+ return spll_calc_mult_div(pll, rate, *parent_rate, NULL, NULL);
+}
+
+static int spll_clk_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct pic32_sys_pll *pll = clkhw_to_spll(hw);
+ unsigned long ret, flags;
+ u32 mult, odiv, v;
+ int err;
+
+ ret = spll_calc_mult_div(pll, rate, parent_rate, &mult, &odiv);
+ if (!ret)
+ return -EINVAL;
+
+ /*
+ * We can't change SPLL counters when it is in-active use
+ * by SYSCLK. So check before applying new counters/rate.
+ */
+
+ /* Is spll_clk active parent of sys_clk ? */
+ if (unlikely(clk_hw_get_parent(pic32_sclk_hw) == hw)) {
+ pr_err("%s: failed, clk in-use\n", __func__);
+ return -EBUSY;
+ }
+
+ spin_lock_irqsave(&pll->core->reg_lock, flags);
+
+ /* apply new multiplier & divisor */
+ v = readl(pll->ctrl_reg);
+ v &= ~(PLL_MULT_MASK << PLL_MULT_SHIFT);
+ v &= ~(PLL_ODIV_MASK << PLL_ODIV_SHIFT);
+ v |= (mult << PLL_MULT_SHIFT) | (odiv << PLL_ODIV_SHIFT);
+
+ /* sys unlock before write */
+ pic32_syskey_unlock();
+
+ writel(v, pll->ctrl_reg);
+ cpu_relax();
+
+ /* insert few nops (5-stage) to ensure CPU does not hang */
+ cpu_nop5();
+ cpu_nop5();
+
+ /* Wait until PLL is locked (maximum 100 usecs). */
+ err = readl_poll_timeout_atomic(pll->status_reg, v,
+ v & pll->lock_mask, 1, 100);
+ spin_unlock_irqrestore(&pll->core->reg_lock, flags);
+
+ return err;
+}
+
+/* SPLL clock operation */
+const struct clk_ops pic32_spll_ops = {
+ .recalc_rate = spll_clk_recalc_rate,
+ .round_rate = spll_clk_round_rate,
+ .set_rate = spll_clk_set_rate,
+};
+
+struct clk *pic32_spll_clk_register(const struct pic32_sys_pll_data *data,
+ struct pic32_clk_common *core)
+{
+ struct pic32_sys_pll *spll;
+ struct clk *clk;
+
+ spll = devm_kzalloc(core->dev, sizeof(*spll), GFP_KERNEL);
+ if (!spll)
+ return ERR_PTR(-ENOMEM);
+
+ spll->core = core;
+ spll->hw.init = &data->init_data;
+ spll->ctrl_reg = data->ctrl_reg + core->iobase;
+ spll->status_reg = data->status_reg + core->iobase;
+ spll->lock_mask = data->lock_mask;
+
+ /* cache PLL idiv; PLL driver uses it as constant.*/
+ spll->idiv = (readl(spll->ctrl_reg) >> PLL_IDIV_SHIFT) & PLL_IDIV_MASK;
+ spll->idiv += 1;
+
+ clk = devm_clk_register(core->dev, &spll->hw);
+ if (IS_ERR(clk))
+ dev_err(core->dev, "sys_pll: clk_register() failed\n");
+
+ return clk;
+}
+
+/* System mux clock(aka SCLK) */
+
+struct pic32_sys_clk {
+ struct clk_hw hw;
+ void __iomem *mux_reg;
+ void __iomem *slew_reg;
+ u32 slew_div;
+ const u32 *parent_map;
+ struct pic32_clk_common *core;
+};
+
+#define clkhw_to_sys_clk(_hw) container_of(_hw, struct pic32_sys_clk, hw)
+
+static unsigned long sclk_get_rate(struct clk_hw *hw, unsigned long parent_rate)
+{
+ struct pic32_sys_clk *sclk = clkhw_to_sys_clk(hw);
+ u32 div;
+
+ div = (readl(sclk->slew_reg) >> SLEW_SYSDIV_SHIFT) & SLEW_SYSDIV;
+ div += 1; /* sys-div to divider */
+
+ return parent_rate / div;
+}
+
+static long sclk_round_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long *parent_rate)
+{
+ return calc_best_divided_rate(rate, *parent_rate, SLEW_SYSDIV, 1);
+}
+
+static int sclk_set_rate(struct clk_hw *hw,
+ unsigned long rate, unsigned long parent_rate)
+{
+ struct pic32_sys_clk *sclk = clkhw_to_sys_clk(hw);
+ unsigned long flags;
+ u32 v, div;
+ int err;
+
+ div = parent_rate / rate;
+
+ spin_lock_irqsave(&sclk->core->reg_lock, flags);
+
+ /* apply new div */
+ v = readl(sclk->slew_reg);
+ v &= ~(SLEW_SYSDIV << SLEW_SYSDIV_SHIFT);
+ v |= (div - 1) << SLEW_SYSDIV_SHIFT;
+
+ pic32_syskey_unlock();
+
+ writel(v, sclk->slew_reg);
+
+ /* wait until BUSY is cleared */
+ err = readl_poll_timeout_atomic(sclk->slew_reg, v,
+ !(v & SLEW_BUSY), 1, LOCK_TIMEOUT_US);
+
+ spin_unlock_irqrestore(&sclk->core->reg_lock, flags);
+
+ return err;
+}
+
+static u8 sclk_get_parent(struct clk_hw *hw)
+{
+ struct pic32_sys_clk *sclk = clkhw_to_sys_clk(hw);
+ u32 i, v;
+
+ v = (readl(sclk->mux_reg) >> OSC_CUR_SHIFT) & OSC_CUR_MASK;
+
+ if (!sclk->parent_map)
+ return v;
+
+ for (i = 0; i < clk_hw_get_num_parents(hw); i++)
+ if (sclk->parent_map[i] == v)
+ return i;
+ return -EINVAL;
+}
+
+static int sclk_set_parent(struct clk_hw *hw, u8 index)
+{
+ struct pic32_sys_clk *sclk = clkhw_to_sys_clk(hw);
+ unsigned long flags;
+ u32 nosc, cosc, v;
+ int err;
+
+ spin_lock_irqsave(&sclk->core->reg_lock, flags);
+
+ /* find new_osc */
+ nosc = sclk->parent_map ? sclk->parent_map[index] : index;
+
+ /* set new parent */
+ v = readl(sclk->mux_reg);
+ v &= ~(OSC_NEW_MASK << OSC_NEW_SHIFT);
+ v |= nosc << OSC_NEW_SHIFT;
+
+ pic32_syskey_unlock();
+
+ writel(v, sclk->mux_reg);
+
+ /* initate switch */
+ writel(OSC_SWEN, PIC32_SET(sclk->mux_reg));
+ cpu_relax();
+
+ /* add nop to flush pipeline (as cpu_clk is in-flux) */
+ cpu_nop5();
+
+ /* wait for SWEN bit to clear */
+ err = readl_poll_timeout_atomic(sclk->slew_reg, v,
+ !(v & OSC_SWEN), 1, LOCK_TIMEOUT_US);
+
+ spin_unlock_irqrestore(&sclk->core->reg_lock, flags);
+
+ /*
+ * SCLK clock-switching logic might reject a clock switching request
+ * if pre-requisites (like new clk_src not present or unstable) are
+ * not met.
+ * So confirm before claiming success.
+ */
+ cosc = (readl(sclk->mux_reg) >> OSC_CUR_SHIFT) & OSC_CUR_MASK;
+ if (cosc != nosc) {
+ pr_err("%s: err, failed to set_parent() to %d, current %d\n",
+ clk_hw_get_name(hw), nosc, cosc);
+ err = -EBUSY;
+ }
+
+ return err;
+}
+
+static void sclk_init(struct clk_hw *hw)
+{
+ struct pic32_sys_clk *sclk = clkhw_to_sys_clk(hw);
+ unsigned long flags;
+ u32 v;
+
+ /* Maintain reference to this clk, required in spll_clk_set_rate() */
+ pic32_sclk_hw = hw;
+
+ /* apply slew divider on both up and down scaling */
+ if (sclk->slew_div) {
+ spin_lock_irqsave(&sclk->core->reg_lock, flags);
+ v = readl(sclk->slew_reg);
+ v &= ~(SLEW_DIV << SLEW_DIV_SHIFT);
+ v |= sclk->slew_div << SLEW_DIV_SHIFT;
+ v |= SLEW_DOWNEN | SLEW_UPEN;
+ writel(v, sclk->slew_reg);
+ spin_unlock_irqrestore(&sclk->core->reg_lock, flags);
+ }
+}
+
+/* sclk with post-divider */
+const struct clk_ops pic32_sclk_ops = {
+ .get_parent = sclk_get_parent,
+ .set_parent = sclk_set_parent,
+ .round_rate = sclk_round_rate,
+ .set_rate = sclk_set_rate,
+ .recalc_rate = sclk_get_rate,
+ .init = sclk_init,
+ .determine_rate = __clk_mux_determine_rate,
+};
+
+/* sclk with no slew and no post-divider */
+const struct clk_ops pic32_sclk_no_div_ops = {
+ .get_parent = sclk_get_parent,
+ .set_parent = sclk_set_parent,
+ .init = sclk_init,
+ .determine_rate = __clk_mux_determine_rate,
+};
+
+struct clk *pic32_sys_clk_register(const struct pic32_sys_clk_data *data,
+ struct pic32_clk_common *core)
+{
+ struct pic32_sys_clk *sclk;
+ struct clk *clk;
+
+ sclk = devm_kzalloc(core->dev, sizeof(*sclk), GFP_KERNEL);
+ if (!sclk)
+ return ERR_PTR(-ENOMEM);
+
+ sclk->core = core;
+ sclk->hw.init = &data->init_data;
+ sclk->mux_reg = data->mux_reg + core->iobase;
+ sclk->slew_reg = data->slew_reg + core->iobase;
+ sclk->slew_div = data->slew_div;
+ sclk->parent_map = data->parent_map;
+
+ clk = devm_clk_register(core->dev, &sclk->hw);
+ if (IS_ERR(clk))
+ dev_err(core->dev, "%s: clk register failed\n", __func__);
+
+ return clk;
+}
+
+/* secondary oscillator */
+struct pic32_sec_osc {
+ struct clk_hw hw;
+ void __iomem *enable_reg;
+ void __iomem *status_reg;
+ u32 enable_mask;
+ u32 status_mask;
+ unsigned long fixed_rate;
+ struct pic32_clk_common *core;
+};
+
+#define clkhw_to_sosc(_hw) container_of(_hw, struct pic32_sec_osc, hw)
+static int sosc_clk_enable(struct clk_hw *hw)
+{
+ struct pic32_sec_osc *sosc = clkhw_to_sosc(hw);
+ u32 v;
+
+ /* enable SOSC */
+ pic32_syskey_unlock();
+ writel(sosc->enable_mask, PIC32_SET(sosc->enable_reg));
+
+ /* wait till warm-up period expires or ready-status is updated */
+ return readl_poll_timeout_atomic(sosc->status_reg, v,
+ v & sosc->status_mask, 1, 100);
+}
+
+static void sosc_clk_disable(struct clk_hw *hw)
+{
+ struct pic32_sec_osc *sosc = clkhw_to_sosc(hw);
+
+ pic32_syskey_unlock();
+ writel(sosc->enable_mask, PIC32_CLR(sosc->enable_reg));
+}
+
+static int sosc_clk_is_enabled(struct clk_hw *hw)
+{
+ struct pic32_sec_osc *sosc = clkhw_to_sosc(hw);
+ u32 enabled, ready;
+
+ /* check enabled and ready status */
+ enabled = readl(sosc->enable_reg) & sosc->enable_mask;
+ ready = readl(sosc->status_reg) & sosc->status_mask;
+
+ return enabled && ready;
+}
+
+static unsigned long sosc_clk_calc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ return clkhw_to_sosc(hw)->fixed_rate;
+}
+
+const struct clk_ops pic32_sosc_ops = {
+ .enable = sosc_clk_enable,
+ .disable = sosc_clk_disable,
+ .is_enabled = sosc_clk_is_enabled,
+ .recalc_rate = sosc_clk_calc_rate,
+};
+
+struct clk *pic32_sosc_clk_register(const struct pic32_sec_osc_data *data,
+ struct pic32_clk_common *core)
+{
+ struct pic32_sec_osc *sosc;
+
+ sosc = devm_kzalloc(core->dev, sizeof(*sosc), GFP_KERNEL);
+ if (!sosc)
+ return ERR_PTR(-ENOMEM);
+
+ sosc->core = core;
+ sosc->hw.init = &data->init_data;
+ sosc->fixed_rate = data->fixed_rate;
+ sosc->enable_mask = data->enable_mask;
+ sosc->status_mask = data->status_mask;
+ sosc->enable_reg = data->enable_reg + core->iobase;
+ sosc->status_reg = data->status_reg + core->iobase;
+
+ return devm_clk_register(core->dev, &sosc->hw);
+}
diff --git a/drivers/clk/microchip/clk-core.h b/drivers/clk/microchip/clk-core.h
new file mode 100644
index 0000000..8566642
--- /dev/null
+++ b/drivers/clk/microchip/clk-core.h
@@ -0,0 +1,84 @@
+/*
+ * Purna Chandra Mandal,<purna.mandal@microchip.com>
+ * Copyright (C) 2015 Microchip Technology Inc. All rights reserved.
+ *
+ * This program is free software; you can distribute it and/or modify it
+ * under the terms of the GNU General Public License (Version 2) as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+ * for more details.
+ */
+#ifndef __MICROCHIP_CLK_PIC32_H_
+#define __MICROCHIP_CLK_PIC32_H_
+
+#include <linux/clk-provider.h>
+
+/* PIC32 clock data */
+struct pic32_clk_common {
+ struct device *dev;
+ void __iomem *iobase;
+ spinlock_t reg_lock; /* clock lock */
+};
+
+/* System PLL clock */
+struct pic32_sys_pll_data {
+ struct clk_init_data init_data;
+ const u32 ctrl_reg;
+ const u32 status_reg;
+ const u32 lock_mask;
+};
+
+/* System clock */
+struct pic32_sys_clk_data {
+ struct clk_init_data init_data;
+ const u32 mux_reg;
+ const u32 slew_reg;
+ const u32 *parent_map;
+ const u32 slew_div;
+};
+
+/* Reference Oscillator clock */
+struct pic32_ref_osc_data {
+ struct clk_init_data init_data;
+ const u32 ctrl_reg;
+ const u32 *parent_map;
+};
+
+/* Peripheral Bus clock */
+struct pic32_periph_clk_data {
+ struct clk_init_data init_data;
+ const u32 ctrl_reg;
+};
+
+/* External Secondary Oscillator clock */
+struct pic32_sec_osc_data {
+ struct clk_init_data init_data;
+ const u32 enable_reg;
+ const u32 status_reg;
+ const u32 enable_mask;
+ const u32 status_mask;
+ const unsigned long fixed_rate;
+};
+
+extern const struct clk_ops pic32_pbclk_ops;
+extern const struct clk_ops pic32_sclk_ops;
+extern const struct clk_ops pic32_sclk_no_div_ops;
+extern const struct clk_ops pic32_spll_ops;
+extern const struct clk_ops pic32_roclk_ops;
+extern const struct clk_ops pic32_sosc_ops;
+
+struct clk *pic32_periph_clk_register(const struct pic32_periph_clk_data *data,
+ struct pic32_clk_common *core);
+struct clk *pic32_refo_clk_register(const struct pic32_ref_osc_data *data,
+ struct pic32_clk_common *core);
+struct clk *pic32_sys_clk_register(const struct pic32_sys_clk_data *data,
+ struct pic32_clk_common *core);
+struct clk *pic32_spll_clk_register(const struct pic32_sys_pll_data *data,
+ struct pic32_clk_common *core);
+struct clk *pic32_sosc_clk_register(const struct pic32_sec_osc_data *data,
+ struct pic32_clk_common *core);
+
+#endif /* __MICROCHIP_CLK_PIC32_H_*/
diff --git a/drivers/clk/microchip/clk-pic32mzda.c b/drivers/clk/microchip/clk-pic32mzda.c
new file mode 100644
index 0000000..020a29a
--- /dev/null
+++ b/drivers/clk/microchip/clk-pic32mzda.c
@@ -0,0 +1,275 @@
+/*
+ * Purna Chandra Mandal,<purna.mandal@microchip.com>
+ * Copyright (C) 2015 Microchip Technology Inc. All rights reserved.
+ *
+ * This program is free software; you can distribute it and/or modify it
+ * under the terms of the GNU General Public License (Version 2) as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+ * for more details.
+ */
+#include <dt-bindings/clock/microchip,pic32-clock.h>
+#include <linux/clk.h>
+#include <linux/clk-provider.h>
+#include <linux/clkdev.h>
+#include <linux/module.h>
+#include <linux/of_address.h>
+#include <linux/of_platform.h>
+#include <linux/platform_device.h>
+#include <asm/traps.h>
+
+#include "clk-core.h"
+
+/* FRC Postscaler */
+#define OSC_FRCDIV_MASK 0x07
+#define OSC_FRCDIV_SHIFT 24
+
+/* SPLL fields */
+#define PLL_ICLK_MASK 0x01
+#define PLL_ICLK_SHIFT 7
+
+#define DECLARE_PERIPHERAL_CLOCK(__clk_name, __reg, __flags) \
+ { \
+ .ctrl_reg = (__reg), \
+ .init_data = { \
+ .name = (__clk_name), \
+ .parent_names = (const char *[]) { \
+ "sys_clk" \
+ }, \
+ .num_parents = 1, \
+ .ops = &pic32_pbclk_ops, \
+ .flags = (__flags), \
+ }, \
+ }
+
+#define DECLARE_REFO_CLOCK(__clkid, __reg) \
+ { \
+ .ctrl_reg = (__reg), \
+ .init_data = { \
+ .name = "refo" #__clkid "_clk", \
+ .parent_names = (const char *[]) { \
+ "sys_clk", "pb1_clk", "posc_clk", \
+ "frc_clk", "lprc_clk", "sosc_clk", \
+ "sys_pll", "refi" #__clkid "_clk", \
+ "bfrc_clk", \
+ }, \
+ .num_parents = 9, \
+ .flags = CLK_SET_RATE_GATE | CLK_SET_PARENT_GATE,\
+ .ops = &pic32_roclk_ops, \
+ }, \
+ .parent_map = (const u32[]) { \
+ 0, 1, 2, 3, 4, 5, 7, 8, 9 \
+ }, \
+ }
+
+static const struct pic32_ref_osc_data ref_clks[] = {
+ DECLARE_REFO_CLOCK(1, 0x80),
+ DECLARE_REFO_CLOCK(2, 0xa0),
+ DECLARE_REFO_CLOCK(3, 0xc0),
+ DECLARE_REFO_CLOCK(4, 0xe0),
+ DECLARE_REFO_CLOCK(5, 0x100),
+};
+
+static const struct pic32_periph_clk_data periph_clocks[] = {
+ DECLARE_PERIPHERAL_CLOCK("pb1_clk", 0x140, 0),
+ DECLARE_PERIPHERAL_CLOCK("pb2_clk", 0x150, CLK_IGNORE_UNUSED),
+ DECLARE_PERIPHERAL_CLOCK("pb3_clk", 0x160, 0),
+ DECLARE_PERIPHERAL_CLOCK("pb4_clk", 0x170, 0),
+ DECLARE_PERIPHERAL_CLOCK("pb5_clk", 0x180, 0),
+ DECLARE_PERIPHERAL_CLOCK("pb6_clk", 0x190, 0),
+ DECLARE_PERIPHERAL_CLOCK("cpu_clk", 0x1a0, CLK_IGNORE_UNUSED),
+};
+
+static const struct pic32_sys_clk_data sys_mux_clk = {
+ .slew_reg = 0x1c0,
+ .slew_div = 2, /* step of div_4 -> div_2 -> no_div */
+ .init_data = {
+ .name = "sys_clk",
+ .parent_names = (const char *[]) {
+ "frcdiv_clk", "sys_pll", "posc_clk",
+ "sosc_clk", "lprc_clk", "frcdiv_clk",
+ },
+ .num_parents = 6,
+ .ops = &pic32_sclk_ops,
+ },
+ .parent_map = (const u32[]) {
+ 0, 1, 2, 4, 5, 7,
+ },
+};
+
+static const struct pic32_sys_pll_data sys_pll = {
+ .ctrl_reg = 0x020,
+ .status_reg = 0x1d0,
+ .lock_mask = BIT(7),
+ .init_data = {
+ .name = "sys_pll",
+ .parent_names = (const char *[]) {
+ "spll_mux_clk"
+ },
+ .num_parents = 1,
+ .ops = &pic32_spll_ops,
+ },
+};
+
+static const struct pic32_sec_osc_data sosc_clk = {
+ .status_reg = 0x1d0,
+ .enable_mask = BIT(1),
+ .status_mask = BIT(4),
+ .init_data = {
+ .name = "sosc_clk",
+ .parent_names = NULL,
+ .ops = &pic32_sosc_ops,
+ },
+};
+
+static int pic32mzda_critical_clks[] = {
+ PB2CLK, PB7CLK
+};
+
+/* PIC32MZDA clock data */
+struct pic32mzda_clk_data {
+ struct clk *clks[MAXCLKS];
+ struct pic32_clk_common core;
+ struct clk_onecell_data onecell_data;
+ struct notifier_block failsafe_notifier;
+};
+
+static int pic32_fscm_nmi(struct notifier_block *nb,
+ unsigned long action, void *data)
+{
+ struct pic32mzda_clk_data *cd;
+
+ cd = container_of(nb, struct pic32mzda_clk_data, failsafe_notifier);
+
+ /* SYSCLK is now running from BFRCCLK. Report clock failure. */
+ if (readl(cd->core.iobase) & BIT(2))
+ pr_alert("pic32-clk: FSCM detected clk failure.\n");
+
+ /* TODO: detect reason of failure and recover accordingly */
+
+ return NOTIFY_OK;
+}
+
+static int pic32mzda_clk_probe(struct platform_device *pdev)
+{
+ const char *const pll_mux_parents[] = {"posc_clk", "frc_clk"};
+ struct device_node *np = pdev->dev.of_node;
+ struct pic32mzda_clk_data *cd;
+ struct pic32_clk_common *core;
+ struct clk *pll_mux_clk, *clk;
+ struct clk **clks;
+ int nr_clks, i, ret;
+
+ cd = devm_kzalloc(&pdev->dev, sizeof(*cd), GFP_KERNEL);
+ if (!cd)
+ return -ENOMEM;
+
+ core = &cd->core;
+ core->iobase = of_io_request_and_map(np, 0, of_node_full_name(np));
+ if (IS_ERR(core->iobase)) {
+ dev_err(&pdev->dev, "pic32-clk: failed to map registers\n");
+ return PTR_ERR(core->iobase);
+ }
+
+ spin_lock_init(&core->reg_lock);
+ core->dev = &pdev->dev;
+ clks = &cd->clks[0];
+
+ /* register fixed rate clocks */
+ clks[POSCCLK] = clk_register_fixed_rate(&pdev->dev, "posc_clk", NULL,
+ CLK_IS_ROOT, 24000000);
+ clks[FRCCLK] = clk_register_fixed_rate(&pdev->dev, "frc_clk", NULL,
+ CLK_IS_ROOT, 8000000);
+ clks[BFRCCLK] = clk_register_fixed_rate(&pdev->dev, "bfrc_clk", NULL,
+ CLK_IS_ROOT, 8000000);
+ clks[LPRCCLK] = clk_register_fixed_rate(&pdev->dev, "lprc_clk", NULL,
+ CLK_IS_ROOT, 32000);
+ clks[UPLLCLK] = clk_register_fixed_rate(&pdev->dev, "usbphy_clk", NULL,
+ CLK_IS_ROOT, 24000000);
+ /* fixed rate (optional) clock */
+ if (of_find_property(np, "microchip,pic32mzda-sosc", NULL)) {
+ pr_info("pic32-clk: dt requests SOSC.\n");
+ clks[SOSCCLK] = pic32_sosc_clk_register(&sosc_clk, core);
+ }
+ /* divider clock */
+ clks[FRCDIVCLK] = clk_register_divider(&pdev->dev, "frcdiv_clk",
+ "frc_clk", 0,
+ core->iobase,
+ OSC_FRCDIV_SHIFT,
+ OSC_FRCDIV_MASK,
+ CLK_DIVIDER_POWER_OF_TWO,
+ &core->reg_lock);
+ /* PLL ICLK mux */
+ pll_mux_clk = clk_register_mux(&pdev->dev, "spll_mux_clk",
+ pll_mux_parents, 2, 0,
+ core->iobase + 0x020,
+ PLL_ICLK_SHIFT, 1, 0, &core->reg_lock);
+ if (IS_ERR(pll_mux_clk))
+ pr_err("spll_mux_clk: clk register failed\n");
+
+ /* PLL */
+ clks[PLLCLK] = pic32_spll_clk_register(&sys_pll, core);
+ /* SYSTEM clock */
+ clks[SCLK] = pic32_sys_clk_register(&sys_mux_clk, core);
+ /* Peripheral bus clocks */
+ for (nr_clks = PB1CLK, i = 0; nr_clks <= PB7CLK; i++, nr_clks++)
+ clks[nr_clks] = pic32_periph_clk_register(&periph_clocks[i],
+ core);
+ /* Reference oscillator clock */
+ for (nr_clks = REF1CLK, i = 0; nr_clks <= REF5CLK; i++, nr_clks++)
+ clks[nr_clks] = pic32_refo_clk_register(&ref_clks[i], core);
+
+ /* register clkdev */
+ for (i = 0; i < MAXCLKS; i++) {
+ if (IS_ERR(clks[i]))
+ continue;
+ clk_register_clkdev(clks[i], NULL, __clk_get_name(clks[i]));
+ }
+
+ /* register clock provider */
+ cd->onecell_data.clks = clks;
+ cd->onecell_data.clk_num = MAXCLKS;
+ ret = of_clk_add_provider(np, of_clk_src_onecell_get,
+ &cd->onecell_data);
+ if (ret)
+ return ret;
+
+ /* force enable critical clocks */
+ for (i = 0; i < ARRAY_SIZE(pic32mzda_critical_clks); i++) {
+ clk = clks[pic32mzda_critical_clks[i]];
+ if (clk_prepare_enable(clk))
+ dev_err(&pdev->dev, "clk_prepare_enable(%s) failed\n",
+ __clk_get_name(clk));
+ }
+
+ /* register NMI for failsafe clock monitor */
+ cd->failsafe_notifier.notifier_call = pic32_fscm_nmi;
+ return register_nmi_notifier(&cd->failsafe_notifier);
+}
+
+static const struct of_device_id pic32mzda_clk_match_table[] = {
+ { .compatible = "microchip,pic32mzda-clk", },
+ { }
+};
+MODULE_DEVICE_TABLE(of, pic32mzda_clk_match_table);
+
+static struct platform_driver pic32mzda_clk_driver = {
+ .probe = pic32mzda_clk_probe,
+ .driver = {
+ .name = "clk-pic32mzda",
+ .of_match_table = pic32mzda_clk_match_table,
+ },
+};
+
+static int __init microchip_pic32mzda_clk_init(void)
+{
+ return platform_driver_register(&pic32mzda_clk_driver);
+}
+core_initcall(microchip_pic32mzda_clk_init);
+
+MODULE_DESCRIPTION("Microchip PIC32MZDA Clock Driver");
+MODULE_LICENSE("GPL v2");
+MODULE_ALIAS("platform:clk-pic32mzda");
diff --git a/drivers/clk/mmp/clk-mmp2.c b/drivers/clk/mmp/clk-mmp2.c
index 38931db..383f6a4 100644
--- a/drivers/clk/mmp/clk-mmp2.c
+++ b/drivers/clk/mmp/clk-mmp2.c
@@ -99,23 +99,19 @@
return;
}
- clk = clk_register_fixed_rate(NULL, "clk32", NULL, CLK_IS_ROOT, 3200);
+ clk = clk_register_fixed_rate(NULL, "clk32", NULL, 0, 3200);
clk_register_clkdev(clk, "clk32", NULL);
- vctcxo = clk_register_fixed_rate(NULL, "vctcxo", NULL, CLK_IS_ROOT,
- 26000000);
+ vctcxo = clk_register_fixed_rate(NULL, "vctcxo", NULL, 0, 26000000);
clk_register_clkdev(vctcxo, "vctcxo", NULL);
- clk = clk_register_fixed_rate(NULL, "pll1", NULL, CLK_IS_ROOT,
- 800000000);
+ clk = clk_register_fixed_rate(NULL, "pll1", NULL, 0, 800000000);
clk_register_clkdev(clk, "pll1", NULL);
- clk = clk_register_fixed_rate(NULL, "usb_pll", NULL, CLK_IS_ROOT,
- 480000000);
+ clk = clk_register_fixed_rate(NULL, "usb_pll", NULL, 0, 480000000);
clk_register_clkdev(clk, "usb_pll", NULL);
- clk = clk_register_fixed_rate(NULL, "pll2", NULL, CLK_IS_ROOT,
- 960000000);
+ clk = clk_register_fixed_rate(NULL, "pll2", NULL, 0, 960000000);
clk_register_clkdev(clk, "pll2", NULL);
clk = clk_register_fixed_factor(NULL, "pll1_2", "pll1",
diff --git a/drivers/clk/mmp/clk-of-mmp2.c b/drivers/clk/mmp/clk-of-mmp2.c
index 251533d..3a51fff 100644
--- a/drivers/clk/mmp/clk-of-mmp2.c
+++ b/drivers/clk/mmp/clk-of-mmp2.c
@@ -63,11 +63,11 @@
};
static struct mmp_param_fixed_rate_clk fixed_rate_clks[] = {
- {MMP2_CLK_CLK32, "clk32", NULL, CLK_IS_ROOT, 32768},
- {MMP2_CLK_VCTCXO, "vctcxo", NULL, CLK_IS_ROOT, 26000000},
- {MMP2_CLK_PLL1, "pll1", NULL, CLK_IS_ROOT, 800000000},
- {MMP2_CLK_PLL2, "pll2", NULL, CLK_IS_ROOT, 960000000},
- {MMP2_CLK_USB_PLL, "usb_pll", NULL, CLK_IS_ROOT, 480000000},
+ {MMP2_CLK_CLK32, "clk32", NULL, 0, 32768},
+ {MMP2_CLK_VCTCXO, "vctcxo", NULL, 0, 26000000},
+ {MMP2_CLK_PLL1, "pll1", NULL, 0, 800000000},
+ {MMP2_CLK_PLL2, "pll2", NULL, 0, 960000000},
+ {MMP2_CLK_USB_PLL, "usb_pll", NULL, 0, 480000000},
};
static struct mmp_param_fixed_factor_clk fixed_factor_clks[] = {
diff --git a/drivers/clk/mmp/clk-of-pxa168.c b/drivers/clk/mmp/clk-of-pxa168.c
index 64eaf41..87f2317 100644
--- a/drivers/clk/mmp/clk-of-pxa168.c
+++ b/drivers/clk/mmp/clk-of-pxa168.c
@@ -56,10 +56,10 @@
};
static struct mmp_param_fixed_rate_clk fixed_rate_clks[] = {
- {PXA168_CLK_CLK32, "clk32", NULL, CLK_IS_ROOT, 32768},
- {PXA168_CLK_VCTCXO, "vctcxo", NULL, CLK_IS_ROOT, 26000000},
- {PXA168_CLK_PLL1, "pll1", NULL, CLK_IS_ROOT, 624000000},
- {PXA168_CLK_USB_PLL, "usb_pll", NULL, CLK_IS_ROOT, 480000000},
+ {PXA168_CLK_CLK32, "clk32", NULL, 0, 32768},
+ {PXA168_CLK_VCTCXO, "vctcxo", NULL, 0, 26000000},
+ {PXA168_CLK_PLL1, "pll1", NULL, 0, 624000000},
+ {PXA168_CLK_USB_PLL, "usb_pll", NULL, 0, 480000000},
};
static struct mmp_param_fixed_factor_clk fixed_factor_clks[] = {
diff --git a/drivers/clk/mmp/clk-of-pxa1928.c b/drivers/clk/mmp/clk-of-pxa1928.c
index 433a5ae..e478ff4 100644
--- a/drivers/clk/mmp/clk-of-pxa1928.c
+++ b/drivers/clk/mmp/clk-of-pxa1928.c
@@ -34,12 +34,12 @@
};
static struct mmp_param_fixed_rate_clk fixed_rate_clks[] = {
- {0, "clk32", NULL, CLK_IS_ROOT, 32768},
- {0, "vctcxo", NULL, CLK_IS_ROOT, 26000000},
- {0, "pll1_624", NULL, CLK_IS_ROOT, 624000000},
- {0, "pll5p", NULL, CLK_IS_ROOT, 832000000},
- {0, "pll5", NULL, CLK_IS_ROOT, 1248000000},
- {0, "usb_pll", NULL, CLK_IS_ROOT, 480000000},
+ {0, "clk32", NULL, 0, 32768},
+ {0, "vctcxo", NULL, 0, 26000000},
+ {0, "pll1_624", NULL, 0, 624000000},
+ {0, "pll5p", NULL, 0, 832000000},
+ {0, "pll5", NULL, 0, 1248000000},
+ {0, "usb_pll", NULL, 0, 480000000},
};
static struct mmp_param_fixed_factor_clk fixed_factor_clks[] = {
diff --git a/drivers/clk/mmp/clk-of-pxa910.c b/drivers/clk/mmp/clk-of-pxa910.c
index 13d6173..e22a67f 100644
--- a/drivers/clk/mmp/clk-of-pxa910.c
+++ b/drivers/clk/mmp/clk-of-pxa910.c
@@ -56,10 +56,10 @@
};
static struct mmp_param_fixed_rate_clk fixed_rate_clks[] = {
- {PXA910_CLK_CLK32, "clk32", NULL, CLK_IS_ROOT, 32768},
- {PXA910_CLK_VCTCXO, "vctcxo", NULL, CLK_IS_ROOT, 26000000},
- {PXA910_CLK_PLL1, "pll1", NULL, CLK_IS_ROOT, 624000000},
- {PXA910_CLK_USB_PLL, "usb_pll", NULL, CLK_IS_ROOT, 480000000},
+ {PXA910_CLK_CLK32, "clk32", NULL, 0, 32768},
+ {PXA910_CLK_VCTCXO, "vctcxo", NULL, 0, 26000000},
+ {PXA910_CLK_PLL1, "pll1", NULL, 0, 624000000},
+ {PXA910_CLK_USB_PLL, "usb_pll", NULL, 0, 480000000},
};
static struct mmp_param_fixed_factor_clk fixed_factor_clks[] = {
diff --git a/drivers/clk/mmp/clk-pxa168.c b/drivers/clk/mmp/clk-pxa168.c
index 0dd83fb..a9ef920 100644
--- a/drivers/clk/mmp/clk-pxa168.c
+++ b/drivers/clk/mmp/clk-pxa168.c
@@ -92,15 +92,13 @@
return;
}
- clk = clk_register_fixed_rate(NULL, "clk32", NULL, CLK_IS_ROOT, 3200);
+ clk = clk_register_fixed_rate(NULL, "clk32", NULL, 0, 3200);
clk_register_clkdev(clk, "clk32", NULL);
- clk = clk_register_fixed_rate(NULL, "vctcxo", NULL, CLK_IS_ROOT,
- 26000000);
+ clk = clk_register_fixed_rate(NULL, "vctcxo", NULL, 0, 26000000);
clk_register_clkdev(clk, "vctcxo", NULL);
- clk = clk_register_fixed_rate(NULL, "pll1", NULL, CLK_IS_ROOT,
- 624000000);
+ clk = clk_register_fixed_rate(NULL, "pll1", NULL, 0, 624000000);
clk_register_clkdev(clk, "pll1", NULL);
clk = clk_register_fixed_factor(NULL, "pll1_2", "pll1",
diff --git a/drivers/clk/mmp/clk-pxa910.c b/drivers/clk/mmp/clk-pxa910.c
index e1d2ce2..a520cf7 100644
--- a/drivers/clk/mmp/clk-pxa910.c
+++ b/drivers/clk/mmp/clk-pxa910.c
@@ -97,15 +97,13 @@
return;
}
- clk = clk_register_fixed_rate(NULL, "clk32", NULL, CLK_IS_ROOT, 3200);
+ clk = clk_register_fixed_rate(NULL, "clk32", NULL, 0, 3200);
clk_register_clkdev(clk, "clk32", NULL);
- clk = clk_register_fixed_rate(NULL, "vctcxo", NULL, CLK_IS_ROOT,
- 26000000);
+ clk = clk_register_fixed_rate(NULL, "vctcxo", NULL, 0, 26000000);
clk_register_clkdev(clk, "vctcxo", NULL);
- clk = clk_register_fixed_rate(NULL, "pll1", NULL, CLK_IS_ROOT,
- 624000000);
+ clk = clk_register_fixed_rate(NULL, "pll1", NULL, 0, 624000000);
clk_register_clkdev(clk, "pll1", NULL);
clk = clk_register_fixed_factor(NULL, "pll1_2", "pll1",
diff --git a/drivers/clk/mvebu/Kconfig b/drivers/clk/mvebu/Kconfig
index eaee8f0..3165da7 100644
--- a/drivers/clk/mvebu/Kconfig
+++ b/drivers/clk/mvebu/Kconfig
@@ -29,6 +29,12 @@
select MVEBU_CLK_COMMON
select MVEBU_CLK_CPU
+config ARMADA_AP806_SYSCON
+ bool
+
+config ARMADA_CP110_SYSCON
+ bool
+
config DOVE_CLK
bool
select MVEBU_CLK_COMMON
diff --git a/drivers/clk/mvebu/Makefile b/drivers/clk/mvebu/Makefile
index 8866115..7172ef6 100644
--- a/drivers/clk/mvebu/Makefile
+++ b/drivers/clk/mvebu/Makefile
@@ -7,6 +7,8 @@
obj-$(CONFIG_ARMADA_38X_CLK) += armada-38x.o
obj-$(CONFIG_ARMADA_39X_CLK) += armada-39x.o
obj-$(CONFIG_ARMADA_XP_CLK) += armada-xp.o
+obj-$(CONFIG_ARMADA_AP806_SYSCON) += ap806-system-controller.o
+obj-$(CONFIG_ARMADA_CP110_SYSCON) += cp110-system-controller.o
obj-$(CONFIG_DOVE_CLK) += dove.o dove-divider.o
obj-$(CONFIG_KIRKWOOD_CLK) += kirkwood.o
obj-$(CONFIG_ORION_CLK) += orion.o
diff --git a/drivers/clk/mvebu/ap806-system-controller.c b/drivers/clk/mvebu/ap806-system-controller.c
new file mode 100644
index 0000000..02023ba
--- /dev/null
+++ b/drivers/clk/mvebu/ap806-system-controller.c
@@ -0,0 +1,168 @@
+/*
+ * Marvell Armada AP806 System Controller
+ *
+ * Copyright (C) 2016 Marvell
+ *
+ * Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2. This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#define pr_fmt(fmt) "ap806-system-controller: " fmt
+
+#include <linux/clk-provider.h>
+#include <linux/mfd/syscon.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+
+#define AP806_SAR_REG 0x400
+#define AP806_SAR_CLKFREQ_MODE_MASK 0x1f
+
+#define AP806_CLK_NUM 4
+
+static struct clk *ap806_clks[AP806_CLK_NUM];
+
+static struct clk_onecell_data ap806_clk_data = {
+ .clks = ap806_clks,
+ .clk_num = AP806_CLK_NUM,
+};
+
+static int ap806_syscon_clk_probe(struct platform_device *pdev)
+{
+ unsigned int freq_mode, cpuclk_freq;
+ const char *name, *fixedclk_name;
+ struct device_node *np = pdev->dev.of_node;
+ struct regmap *regmap;
+ u32 reg;
+ int ret;
+
+ regmap = syscon_node_to_regmap(np);
+ if (IS_ERR(regmap)) {
+ dev_err(&pdev->dev, "cannot get regmap\n");
+ return PTR_ERR(regmap);
+ }
+
+ ret = regmap_read(regmap, AP806_SAR_REG, ®);
+ if (ret) {
+ dev_err(&pdev->dev, "cannot read from regmap\n");
+ return ret;
+ }
+
+ freq_mode = reg & AP806_SAR_CLKFREQ_MODE_MASK;
+ switch (freq_mode) {
+ case 0x0 ... 0x5:
+ cpuclk_freq = 2000;
+ break;
+ case 0x6 ... 0xB:
+ cpuclk_freq = 1800;
+ break;
+ case 0xC ... 0x11:
+ cpuclk_freq = 1600;
+ break;
+ case 0x12 ... 0x16:
+ cpuclk_freq = 1400;
+ break;
+ case 0x17 ... 0x19:
+ cpuclk_freq = 1300;
+ break;
+ default:
+ dev_err(&pdev->dev, "invalid SAR value\n");
+ return -EINVAL;
+ }
+
+ /* Convert to hertz */
+ cpuclk_freq *= 1000 * 1000;
+
+ /* CPU clocks depend on the Sample At Reset configuration */
+ of_property_read_string_index(np, "clock-output-names",
+ 0, &name);
+ ap806_clks[0] = clk_register_fixed_rate(&pdev->dev, name, NULL,
+ 0, cpuclk_freq);
+ if (IS_ERR(ap806_clks[0])) {
+ ret = PTR_ERR(ap806_clks[0]);
+ goto fail0;
+ }
+
+ of_property_read_string_index(np, "clock-output-names",
+ 1, &name);
+ ap806_clks[1] = clk_register_fixed_rate(&pdev->dev, name, NULL, 0,
+ cpuclk_freq);
+ if (IS_ERR(ap806_clks[1])) {
+ ret = PTR_ERR(ap806_clks[1]);
+ goto fail1;
+ }
+
+ /* Fixed clock is always 1200 Mhz */
+ of_property_read_string_index(np, "clock-output-names",
+ 2, &fixedclk_name);
+ ap806_clks[2] = clk_register_fixed_rate(&pdev->dev, fixedclk_name, NULL,
+ 0, 1200 * 1000 * 1000);
+ if (IS_ERR(ap806_clks[2])) {
+ ret = PTR_ERR(ap806_clks[2]);
+ goto fail2;
+ }
+
+ /* MSS Clock is fixed clock divided by 6 */
+ of_property_read_string_index(np, "clock-output-names",
+ 3, &name);
+ ap806_clks[3] = clk_register_fixed_factor(NULL, name, fixedclk_name,
+ 0, 1, 6);
+ if (IS_ERR(ap806_clks[3])) {
+ ret = PTR_ERR(ap806_clks[3]);
+ goto fail3;
+ }
+
+ ret = of_clk_add_provider(np, of_clk_src_onecell_get, &ap806_clk_data);
+ if (ret)
+ goto fail_clk_add;
+
+ return 0;
+
+fail_clk_add:
+ clk_unregister_fixed_factor(ap806_clks[3]);
+fail3:
+ clk_unregister_fixed_rate(ap806_clks[2]);
+fail2:
+ clk_unregister_fixed_rate(ap806_clks[1]);
+fail1:
+ clk_unregister_fixed_rate(ap806_clks[0]);
+fail0:
+ return ret;
+}
+
+static int ap806_syscon_clk_remove(struct platform_device *pdev)
+{
+ of_clk_del_provider(pdev->dev.of_node);
+ clk_unregister_fixed_factor(ap806_clks[3]);
+ clk_unregister_fixed_rate(ap806_clks[2]);
+ clk_unregister_fixed_rate(ap806_clks[1]);
+ clk_unregister_fixed_rate(ap806_clks[0]);
+
+ return 0;
+}
+
+static const struct of_device_id ap806_syscon_of_match[] = {
+ { .compatible = "marvell,ap806-system-controller", },
+ { }
+};
+MODULE_DEVICE_TABLE(of, armada8k_pcie_of_match);
+
+static struct platform_driver ap806_syscon_driver = {
+ .probe = ap806_syscon_clk_probe,
+ .remove = ap806_syscon_clk_remove,
+ .driver = {
+ .name = "marvell-ap806-system-controller",
+ .of_match_table = ap806_syscon_of_match,
+ },
+};
+
+module_platform_driver(ap806_syscon_driver);
+
+MODULE_DESCRIPTION("Marvell AP806 System Controller driver");
+MODULE_AUTHOR("Thomas Petazzoni <thomas.petazzoni@free-electrons.com>");
+MODULE_LICENSE("GPL");
diff --git a/drivers/clk/mvebu/cp110-system-controller.c b/drivers/clk/mvebu/cp110-system-controller.c
new file mode 100644
index 0000000..7fa42d6
--- /dev/null
+++ b/drivers/clk/mvebu/cp110-system-controller.c
@@ -0,0 +1,406 @@
+/*
+ * Marvell Armada CP110 System Controller
+ *
+ * Copyright (C) 2016 Marvell
+ *
+ * Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2. This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+/*
+ * CP110 has 5 core clocks:
+ *
+ * - APLL (1 Ghz)
+ * - PPv2 core (1/3 APLL)
+ * - EIP (1/2 APLL)
+ * - Core (1/2 EIP)
+ *
+ * - NAND clock, which is either:
+ * - Equal to the core clock
+ * - 2/5 APLL
+ *
+ * CP110 has 32 gatable clocks, for the various peripherals in the
+ * IP. They have fairly complicated parent/child relationships.
+ */
+
+#define pr_fmt(fmt) "cp110-system-controller: " fmt
+
+#include <linux/clk-provider.h>
+#include <linux/mfd/syscon.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+#include <linux/slab.h>
+
+#define CP110_PM_CLOCK_GATING_REG 0x220
+#define CP110_NAND_FLASH_CLK_CTRL_REG 0x700
+#define NF_CLOCK_SEL_400_MASK BIT(0)
+
+enum {
+ CP110_CLK_TYPE_CORE,
+ CP110_CLK_TYPE_GATABLE,
+};
+
+#define CP110_MAX_CORE_CLOCKS 5
+#define CP110_MAX_GATABLE_CLOCKS 32
+
+#define CP110_CLK_NUM \
+ (CP110_MAX_CORE_CLOCKS + CP110_MAX_GATABLE_CLOCKS)
+
+#define CP110_CORE_APLL 0
+#define CP110_CORE_PPV2 1
+#define CP110_CORE_EIP 2
+#define CP110_CORE_CORE 3
+#define CP110_CORE_NAND 4
+
+/* A number of gatable clocks need special handling */
+#define CP110_GATE_AUDIO 0
+#define CP110_GATE_COMM_UNIT 1
+#define CP110_GATE_NAND 2
+#define CP110_GATE_PPV2 3
+#define CP110_GATE_SDIO 4
+#define CP110_GATE_XOR1 7
+#define CP110_GATE_XOR0 8
+#define CP110_GATE_PCIE_X1_0 11
+#define CP110_GATE_PCIE_X1_1 12
+#define CP110_GATE_PCIE_X4 13
+#define CP110_GATE_PCIE_XOR 14
+#define CP110_GATE_SATA 15
+#define CP110_GATE_SATA_USB 16
+#define CP110_GATE_MAIN 17
+#define CP110_GATE_SDMMC 18
+#define CP110_GATE_SLOW_IO 21
+#define CP110_GATE_USB3H0 22
+#define CP110_GATE_USB3H1 23
+#define CP110_GATE_USB3DEV 24
+#define CP110_GATE_EIP150 25
+#define CP110_GATE_EIP197 26
+
+static struct clk *cp110_clks[CP110_CLK_NUM];
+
+static struct clk_onecell_data cp110_clk_data = {
+ .clks = cp110_clks,
+ .clk_num = CP110_CLK_NUM,
+};
+
+struct cp110_gate_clk {
+ struct clk_hw hw;
+ struct regmap *regmap;
+ u8 bit_idx;
+};
+
+#define to_cp110_gate_clk(clk) container_of(clk, struct cp110_gate_clk, hw)
+
+static int cp110_gate_enable(struct clk_hw *hw)
+{
+ struct cp110_gate_clk *gate = to_cp110_gate_clk(hw);
+
+ regmap_update_bits(gate->regmap, CP110_PM_CLOCK_GATING_REG,
+ BIT(gate->bit_idx), BIT(gate->bit_idx));
+
+ return 0;
+}
+
+static void cp110_gate_disable(struct clk_hw *hw)
+{
+ struct cp110_gate_clk *gate = to_cp110_gate_clk(hw);
+
+ regmap_update_bits(gate->regmap, CP110_PM_CLOCK_GATING_REG,
+ BIT(gate->bit_idx), 0);
+}
+
+static int cp110_gate_is_enabled(struct clk_hw *hw)
+{
+ struct cp110_gate_clk *gate = to_cp110_gate_clk(hw);
+ u32 val;
+
+ regmap_read(gate->regmap, CP110_PM_CLOCK_GATING_REG, &val);
+
+ return val & BIT(gate->bit_idx);
+}
+
+static const struct clk_ops cp110_gate_ops = {
+ .enable = cp110_gate_enable,
+ .disable = cp110_gate_disable,
+ .is_enabled = cp110_gate_is_enabled,
+};
+
+static struct clk *cp110_register_gate(const char *name,
+ const char *parent_name,
+ struct regmap *regmap, u8 bit_idx)
+{
+ struct cp110_gate_clk *gate;
+ struct clk *clk;
+ struct clk_init_data init;
+
+ gate = kzalloc(sizeof(*gate), GFP_KERNEL);
+ if (!gate)
+ return ERR_PTR(-ENOMEM);
+
+ init.name = name;
+ init.ops = &cp110_gate_ops;
+ init.parent_names = &parent_name;
+ init.num_parents = 1;
+
+ gate->regmap = regmap;
+ gate->bit_idx = bit_idx;
+ gate->hw.init = &init;
+
+ clk = clk_register(NULL, &gate->hw);
+ if (IS_ERR(clk))
+ kfree(gate);
+
+ return clk;
+}
+
+static void cp110_unregister_gate(struct clk *clk)
+{
+ struct clk_hw *hw;
+
+ hw = __clk_get_hw(clk);
+ if (!hw)
+ return;
+
+ clk_unregister(clk);
+ kfree(to_cp110_gate_clk(hw));
+}
+
+static struct clk *cp110_of_clk_get(struct of_phandle_args *clkspec, void *data)
+{
+ struct clk_onecell_data *clk_data = data;
+ unsigned int type = clkspec->args[0];
+ unsigned int idx = clkspec->args[1];
+
+ if (type == CP110_CLK_TYPE_CORE) {
+ if (idx > CP110_MAX_CORE_CLOCKS)
+ return ERR_PTR(-EINVAL);
+ return clk_data->clks[idx];
+ } else if (type == CP110_CLK_TYPE_GATABLE) {
+ if (idx > CP110_MAX_GATABLE_CLOCKS)
+ return ERR_PTR(-EINVAL);
+ return clk_data->clks[CP110_MAX_CORE_CLOCKS + idx];
+ }
+
+ return ERR_PTR(-EINVAL);
+}
+
+static int cp110_syscon_clk_probe(struct platform_device *pdev)
+{
+ struct regmap *regmap;
+ struct device_node *np = pdev->dev.of_node;
+ const char *ppv2_name, *apll_name, *core_name, *eip_name, *nand_name;
+ struct clk *clk;
+ u32 nand_clk_ctrl;
+ int i, ret;
+
+ regmap = syscon_node_to_regmap(np);
+ if (IS_ERR(regmap))
+ return PTR_ERR(regmap);
+
+ ret = regmap_read(regmap, CP110_NAND_FLASH_CLK_CTRL_REG,
+ &nand_clk_ctrl);
+ if (ret)
+ return ret;
+
+ /* Register the APLL which is the root of the clk tree */
+ of_property_read_string_index(np, "core-clock-output-names",
+ CP110_CORE_APLL, &apll_name);
+ clk = clk_register_fixed_rate(NULL, apll_name, NULL, 0,
+ 1000 * 1000 * 1000);
+ if (IS_ERR(clk)) {
+ ret = PTR_ERR(clk);
+ goto fail0;
+ }
+
+ cp110_clks[CP110_CORE_APLL] = clk;
+
+ /* PPv2 is APLL/3 */
+ of_property_read_string_index(np, "core-clock-output-names",
+ CP110_CORE_PPV2, &ppv2_name);
+ clk = clk_register_fixed_factor(NULL, ppv2_name, apll_name, 0, 1, 3);
+ if (IS_ERR(clk)) {
+ ret = PTR_ERR(clk);
+ goto fail1;
+ }
+
+ cp110_clks[CP110_CORE_PPV2] = clk;
+
+ /* EIP clock is APLL/2 */
+ of_property_read_string_index(np, "core-clock-output-names",
+ CP110_CORE_EIP, &eip_name);
+ clk = clk_register_fixed_factor(NULL, eip_name, apll_name, 0, 1, 2);
+ if (IS_ERR(clk)) {
+ ret = PTR_ERR(clk);
+ goto fail2;
+ }
+
+ cp110_clks[CP110_CORE_EIP] = clk;
+
+ /* Core clock is EIP/2 */
+ of_property_read_string_index(np, "core-clock-output-names",
+ CP110_CORE_CORE, &core_name);
+ clk = clk_register_fixed_factor(NULL, core_name, eip_name, 0, 1, 2);
+ if (IS_ERR(clk)) {
+ ret = PTR_ERR(clk);
+ goto fail3;
+ }
+
+ cp110_clks[CP110_CORE_CORE] = clk;
+
+ /* NAND can be either APLL/2.5 or core clock */
+ of_property_read_string_index(np, "core-clock-output-names",
+ CP110_CORE_NAND, &nand_name);
+ if (nand_clk_ctrl & NF_CLOCK_SEL_400_MASK)
+ clk = clk_register_fixed_factor(NULL, nand_name,
+ apll_name, 0, 2, 5);
+ else
+ clk = clk_register_fixed_factor(NULL, nand_name,
+ core_name, 0, 1, 1);
+ if (IS_ERR(clk)) {
+ ret = PTR_ERR(clk);
+ goto fail4;
+ }
+
+ cp110_clks[CP110_CORE_NAND] = clk;
+
+ for (i = 0; i < CP110_MAX_GATABLE_CLOCKS; i++) {
+ const char *parent, *name;
+ int ret;
+
+ ret = of_property_read_string_index(np,
+ "gate-clock-output-names",
+ i, &name);
+ /* Reached the end of the list? */
+ if (ret < 0)
+ break;
+
+ if (!strcmp(name, "none"))
+ continue;
+
+ switch (i) {
+ case CP110_GATE_AUDIO:
+ case CP110_GATE_COMM_UNIT:
+ case CP110_GATE_EIP150:
+ case CP110_GATE_EIP197:
+ case CP110_GATE_SLOW_IO:
+ of_property_read_string_index(np,
+ "gate-clock-output-names",
+ CP110_GATE_MAIN, &parent);
+ break;
+ case CP110_GATE_NAND:
+ parent = nand_name;
+ break;
+ case CP110_GATE_PPV2:
+ parent = ppv2_name;
+ break;
+ case CP110_GATE_SDIO:
+ of_property_read_string_index(np,
+ "gate-clock-output-names",
+ CP110_GATE_SDMMC, &parent);
+ break;
+ case CP110_GATE_XOR1:
+ case CP110_GATE_XOR0:
+ case CP110_GATE_PCIE_X1_0:
+ case CP110_GATE_PCIE_X1_1:
+ case CP110_GATE_PCIE_X4:
+ of_property_read_string_index(np,
+ "gate-clock-output-names",
+ CP110_GATE_PCIE_XOR, &parent);
+ break;
+ case CP110_GATE_SATA:
+ case CP110_GATE_USB3H0:
+ case CP110_GATE_USB3H1:
+ case CP110_GATE_USB3DEV:
+ of_property_read_string_index(np,
+ "gate-clock-output-names",
+ CP110_GATE_SATA_USB, &parent);
+ break;
+ default:
+ parent = core_name;
+ break;
+ }
+
+ clk = cp110_register_gate(name, parent, regmap, i);
+ if (IS_ERR(clk)) {
+ ret = PTR_ERR(clk);
+ goto fail_gate;
+ }
+
+ cp110_clks[CP110_MAX_CORE_CLOCKS + i] = clk;
+ }
+
+ ret = of_clk_add_provider(np, cp110_of_clk_get, &cp110_clk_data);
+ if (ret)
+ goto fail_clk_add;
+
+ return 0;
+
+fail_clk_add:
+fail_gate:
+ for (i = 0; i < CP110_MAX_GATABLE_CLOCKS; i++) {
+ clk = cp110_clks[CP110_MAX_CORE_CLOCKS + i];
+
+ if (clk)
+ cp110_unregister_gate(clk);
+ }
+
+ clk_unregister_fixed_factor(cp110_clks[CP110_CORE_NAND]);
+fail4:
+ clk_unregister_fixed_factor(cp110_clks[CP110_CORE_CORE]);
+fail3:
+ clk_unregister_fixed_factor(cp110_clks[CP110_CORE_EIP]);
+fail2:
+ clk_unregister_fixed_factor(cp110_clks[CP110_CORE_PPV2]);
+fail1:
+ clk_unregister_fixed_rate(cp110_clks[CP110_CORE_APLL]);
+fail0:
+ return ret;
+}
+
+static int cp110_syscon_clk_remove(struct platform_device *pdev)
+{
+ int i;
+
+ of_clk_del_provider(pdev->dev.of_node);
+
+ for (i = 0; i < CP110_MAX_GATABLE_CLOCKS; i++) {
+ struct clk *clk = cp110_clks[CP110_MAX_CORE_CLOCKS + i];
+
+ if (clk)
+ cp110_unregister_gate(clk);
+ }
+
+ clk_unregister_fixed_factor(cp110_clks[CP110_CORE_NAND]);
+ clk_unregister_fixed_factor(cp110_clks[CP110_CORE_CORE]);
+ clk_unregister_fixed_factor(cp110_clks[CP110_CORE_EIP]);
+ clk_unregister_fixed_factor(cp110_clks[CP110_CORE_PPV2]);
+ clk_unregister_fixed_rate(cp110_clks[CP110_CORE_APLL]);
+
+ return 0;
+}
+
+static const struct of_device_id cp110_syscon_of_match[] = {
+ { .compatible = "marvell,cp110-system-controller0", },
+ { }
+};
+MODULE_DEVICE_TABLE(of, armada8k_pcie_of_match);
+
+static struct platform_driver cp110_syscon_driver = {
+ .probe = cp110_syscon_clk_probe,
+ .remove = cp110_syscon_clk_remove,
+ .driver = {
+ .name = "marvell-cp110-system-controller0",
+ .of_match_table = cp110_syscon_of_match,
+ },
+};
+
+module_platform_driver(cp110_syscon_driver);
+
+MODULE_DESCRIPTION("Marvell CP110 System Controller 0 driver");
+MODULE_AUTHOR("Thomas Petazzoni <thomas.petazzoni@free-electrons.com>");
+MODULE_LICENSE("GPL");
diff --git a/drivers/clk/nxp/clk-lpc18xx-creg.c b/drivers/clk/nxp/clk-lpc18xx-creg.c
index d44b61a..9e35749 100644
--- a/drivers/clk/nxp/clk-lpc18xx-creg.c
+++ b/drivers/clk/nxp/clk-lpc18xx-creg.c
@@ -147,6 +147,7 @@
init.name = creg_clk->name;
init.parent_names = parent_name;
init.num_parents = 1;
+ init.flags = 0;
creg_clk->reg = syscon;
creg_clk->hw.init = &init;
diff --git a/drivers/clk/qcom/gcc-msm8916.c b/drivers/clk/qcom/gcc-msm8916.c
index 9c29080..5c4e193 100644
--- a/drivers/clk/qcom/gcc-msm8916.c
+++ b/drivers/clk/qcom/gcc-msm8916.c
@@ -2346,6 +2346,7 @@
"pcnoc_bfdcd_clk_src",
},
.num_parents = 1,
+ .flags = CLK_SET_RATE_PARENT,
.ops = &clk_branch2_ops,
},
},
@@ -2381,6 +2382,7 @@
"crypto_clk_src",
},
.num_parents = 1,
+ .flags = CLK_SET_RATE_PARENT,
.ops = &clk_branch2_ops,
},
},
diff --git a/drivers/clk/qcom/mmcc-msm8996.c b/drivers/clk/qcom/mmcc-msm8996.c
index 6df7ff3..847dd9da 100644
--- a/drivers/clk/qcom/mmcc-msm8996.c
+++ b/drivers/clk/qcom/mmcc-msm8996.c
@@ -1279,21 +1279,6 @@
},
};
-static struct clk_branch mmss_mmagic_axi_clk = {
- .halt_reg = 0x506c,
- .clkr = {
- .enable_reg = 0x506c,
- .enable_mask = BIT(0),
- .hw.init = &(struct clk_init_data){
- .name = "mmss_mmagic_axi_clk",
- .parent_names = (const char *[]){ "axi_clk_src" },
- .num_parents = 1,
- .flags = CLK_SET_RATE_PARENT,
- .ops = &clk_branch2_ops,
- },
- },
-};
-
static struct clk_branch mmss_mmagic_maxi_clk = {
.halt_reg = 0x5074,
.clkr = {
@@ -1579,21 +1564,6 @@
},
};
-static struct clk_branch mmagic_bimc_axi_clk = {
- .halt_reg = 0x5294,
- .clkr = {
- .enable_reg = 0x5294,
- .enable_mask = BIT(0),
- .hw.init = &(struct clk_init_data){
- .name = "mmagic_bimc_axi_clk",
- .parent_names = (const char *[]){ "axi_clk_src" },
- .num_parents = 1,
- .flags = CLK_SET_RATE_PARENT,
- .ops = &clk_branch2_ops,
- },
- },
-};
-
static struct clk_branch mmagic_bimc_noc_cfg_ahb_clk = {
.halt_reg = 0x5298,
.clkr = {
@@ -3121,7 +3091,6 @@
[MMSS_MMAGIC_CFG_AHB_CLK] = &mmss_mmagic_cfg_ahb_clk.clkr,
[MMSS_MISC_AHB_CLK] = &mmss_misc_ahb_clk.clkr,
[MMSS_MISC_CXO_CLK] = &mmss_misc_cxo_clk.clkr,
- [MMSS_MMAGIC_AXI_CLK] = &mmss_mmagic_axi_clk.clkr,
[MMSS_MMAGIC_MAXI_CLK] = &mmss_mmagic_maxi_clk.clkr,
[MMAGIC_CAMSS_AXI_CLK] = &mmagic_camss_axi_clk.clkr,
[MMAGIC_CAMSS_NOC_CFG_AHB_CLK] = &mmagic_camss_noc_cfg_ahb_clk.clkr,
@@ -3141,7 +3110,6 @@
[MMAGIC_VIDEO_NOC_CFG_AHB_CLK] = &mmagic_video_noc_cfg_ahb_clk.clkr,
[SMMU_VIDEO_AHB_CLK] = &smmu_video_ahb_clk.clkr,
[SMMU_VIDEO_AXI_CLK] = &smmu_video_axi_clk.clkr,
- [MMAGIC_BIMC_AXI_CLK] = &mmagic_bimc_axi_clk.clkr,
[MMAGIC_BIMC_NOC_CFG_AHB_CLK] = &mmagic_bimc_noc_cfg_ahb_clk.clkr,
[GPU_GX_GFX3D_CLK] = &gpu_gx_gfx3d_clk.clkr,
[GPU_GX_RBBMTIMER_CLK] = &gpu_gx_rbbmtimer_clk.clkr,
diff --git a/drivers/clk/renesas/clk-mstp.c b/drivers/clk/renesas/clk-mstp.c
index 8b597b9..5093a25 100644
--- a/drivers/clk/renesas/clk-mstp.c
+++ b/drivers/clk/renesas/clk-mstp.c
@@ -316,11 +316,10 @@
return;
pd->name = np->name;
-
pd->flags = GENPD_FLAG_PM_CLK;
- pm_genpd_init(pd, &simple_qos_governor, false);
pd->attach_dev = cpg_mstp_attach_dev;
pd->detach_dev = cpg_mstp_detach_dev;
+ pm_genpd_init(pd, &pm_domain_always_on_gov, false);
of_genpd_add_provider_simple(np, pd);
}
diff --git a/drivers/clk/renesas/r8a7795-cpg-mssr.c b/drivers/clk/renesas/r8a7795-cpg-mssr.c
index 6af7f5b..ca5519c 100644
--- a/drivers/clk/renesas/r8a7795-cpg-mssr.c
+++ b/drivers/clk/renesas/r8a7795-cpg-mssr.c
@@ -120,6 +120,7 @@
DEF_DIV6P1("mso", R8A7795_CLK_MSO, CLK_PLL1_DIV4, 0x014),
DEF_DIV6P1("hdmi", R8A7795_CLK_HDMI, CLK_PLL1_DIV2, 0x250),
DEF_DIV6P1("canfd", R8A7795_CLK_CANFD, CLK_PLL1_DIV4, 0x244),
+ DEF_DIV6P1("csi0", R8A7795_CLK_CSI0, CLK_PLL1_DIV4, 0x00c),
DEF_DIV6_RO("osc", R8A7795_CLK_OSC, CLK_EXTAL, CPG_RCKCR, 8),
DEF_DIV6_RO("r_int", CLK_RINT, CLK_EXTAL, CPG_RCKCR, 32),
@@ -190,6 +191,10 @@
DEF_MOD("ehci1", 702, R8A7795_CLK_S3D4),
DEF_MOD("ehci0", 703, R8A7795_CLK_S3D4),
DEF_MOD("hsusb", 704, R8A7795_CLK_S3D4),
+ DEF_MOD("csi21", 713, R8A7795_CLK_CSI0),
+ DEF_MOD("csi20", 714, R8A7795_CLK_CSI0),
+ DEF_MOD("csi41", 715, R8A7795_CLK_CSI0),
+ DEF_MOD("csi40", 716, R8A7795_CLK_CSI0),
DEF_MOD("du3", 721, R8A7795_CLK_S2D1),
DEF_MOD("du2", 722, R8A7795_CLK_S2D1),
DEF_MOD("du1", 723, R8A7795_CLK_S2D1),
@@ -197,6 +202,14 @@
DEF_MOD("lvds", 727, R8A7795_CLK_S2D1),
DEF_MOD("hdmi1", 728, R8A7795_CLK_HDMI),
DEF_MOD("hdmi0", 729, R8A7795_CLK_HDMI),
+ DEF_MOD("vin7", 804, R8A7795_CLK_S2D1),
+ DEF_MOD("vin6", 805, R8A7795_CLK_S2D1),
+ DEF_MOD("vin5", 806, R8A7795_CLK_S2D1),
+ DEF_MOD("vin4", 807, R8A7795_CLK_S2D1),
+ DEF_MOD("vin3", 808, R8A7795_CLK_S2D1),
+ DEF_MOD("vin2", 809, R8A7795_CLK_S2D1),
+ DEF_MOD("vin1", 810, R8A7795_CLK_S2D1),
+ DEF_MOD("vin0", 811, R8A7795_CLK_S2D1),
DEF_MOD("etheravb", 812, R8A7795_CLK_S3D2),
DEF_MOD("sata0", 815, R8A7795_CLK_S3D2),
DEF_MOD("gpio7", 905, R8A7795_CLK_CP),
diff --git a/drivers/clk/renesas/renesas-cpg-mssr.c b/drivers/clk/renesas/renesas-cpg-mssr.c
index 1f2dc362..210cd74 100644
--- a/drivers/clk/renesas/renesas-cpg-mssr.c
+++ b/drivers/clk/renesas/renesas-cpg-mssr.c
@@ -493,9 +493,9 @@
genpd = &pd->genpd;
genpd->name = np->name;
genpd->flags = GENPD_FLAG_PM_CLK;
- pm_genpd_init(genpd, &simple_qos_governor, false);
genpd->attach_dev = cpg_mssr_attach_dev;
genpd->detach_dev = cpg_mssr_detach_dev;
+ pm_genpd_init(genpd, &pm_domain_always_on_gov, false);
cpg_mssr_clk_domain = pd;
of_genpd_add_provider_simple(np, genpd);
diff --git a/drivers/clk/rockchip/Makefile b/drivers/clk/rockchip/Makefile
index 80b9a37..f47a2fa 100644
--- a/drivers/clk/rockchip/Makefile
+++ b/drivers/clk/rockchip/Makefile
@@ -15,3 +15,4 @@
obj-y += clk-rk3228.o
obj-y += clk-rk3288.o
obj-y += clk-rk3368.o
+obj-y += clk-rk3399.o
diff --git a/drivers/clk/rockchip/clk-cpu.c b/drivers/clk/rockchip/clk-cpu.c
index 4e73ed5..4bb130c 100644
--- a/drivers/clk/rockchip/clk-cpu.c
+++ b/drivers/clk/rockchip/clk-cpu.c
@@ -158,12 +158,16 @@
writel(HIWORD_UPDATE(alt_div, reg_data->div_core_mask,
reg_data->div_core_shift) |
- HIWORD_UPDATE(1, 1, reg_data->mux_core_shift),
+ HIWORD_UPDATE(reg_data->mux_core_alt,
+ reg_data->mux_core_mask,
+ reg_data->mux_core_shift),
cpuclk->reg_base + reg_data->core_reg);
} else {
/* select alternate parent */
- writel(HIWORD_UPDATE(1, 1, reg_data->mux_core_shift),
- cpuclk->reg_base + reg_data->core_reg);
+ writel(HIWORD_UPDATE(reg_data->mux_core_alt,
+ reg_data->mux_core_mask,
+ reg_data->mux_core_shift),
+ cpuclk->reg_base + reg_data->core_reg);
}
spin_unlock_irqrestore(cpuclk->lock, flags);
@@ -198,7 +202,9 @@
writel(HIWORD_UPDATE(0, reg_data->div_core_mask,
reg_data->div_core_shift) |
- HIWORD_UPDATE(0, 1, reg_data->mux_core_shift),
+ HIWORD_UPDATE(reg_data->mux_core_main,
+ reg_data->mux_core_mask,
+ reg_data->mux_core_shift),
cpuclk->reg_base + reg_data->core_reg);
if (ndata->old_rate > ndata->new_rate)
@@ -252,7 +258,7 @@
return ERR_PTR(-ENOMEM);
init.name = name;
- init.parent_names = &parent_names[0];
+ init.parent_names = &parent_names[reg_data->mux_core_main];
init.num_parents = 1;
init.ops = &rockchip_cpuclk_ops;
@@ -270,10 +276,10 @@
cpuclk->clk_nb.notifier_call = rockchip_cpuclk_notifier_cb;
cpuclk->hw.init = &init;
- cpuclk->alt_parent = __clk_lookup(parent_names[1]);
+ cpuclk->alt_parent = __clk_lookup(parent_names[reg_data->mux_core_alt]);
if (!cpuclk->alt_parent) {
- pr_err("%s: could not lookup alternate parent\n",
- __func__);
+ pr_err("%s: could not lookup alternate parent: (%d)\n",
+ __func__, reg_data->mux_core_alt);
ret = -EINVAL;
goto free_cpuclk;
}
@@ -285,10 +291,11 @@
goto free_cpuclk;
}
- clk = __clk_lookup(parent_names[0]);
+ clk = __clk_lookup(parent_names[reg_data->mux_core_main]);
if (!clk) {
- pr_err("%s: could not lookup parent clock %s\n",
- __func__, parent_names[0]);
+ pr_err("%s: could not lookup parent clock: (%d) %s\n",
+ __func__, reg_data->mux_core_main,
+ parent_names[reg_data->mux_core_main]);
ret = -EINVAL;
goto free_alt_parent;
}
diff --git a/drivers/clk/rockchip/clk-mmc-phase.c b/drivers/clk/rockchip/clk-mmc-phase.c
index e0dc7e8..bc856f2 100644
--- a/drivers/clk/rockchip/clk-mmc-phase.c
+++ b/drivers/clk/rockchip/clk-mmc-phase.c
@@ -123,7 +123,8 @@
raw_value = delay_num ? ROCKCHIP_MMC_DELAY_SEL : 0;
raw_value |= delay_num << ROCKCHIP_MMC_DELAYNUM_OFFSET;
raw_value |= nineties;
- writel(HIWORD_UPDATE(raw_value, 0x07ff, mmc_clock->shift), mmc_clock->reg);
+ writel(HIWORD_UPDATE(raw_value, 0x07ff, mmc_clock->shift),
+ mmc_clock->reg);
pr_debug("%s->set_phase(%d) delay_nums=%u reg[0x%p]=0x%03x actual_degrees=%d\n",
clk_hw_get_name(hw), degrees, delay_num,
diff --git a/drivers/clk/rockchip/clk-pll.c b/drivers/clk/rockchip/clk-pll.c
index 5de797e..db81e45 100644
--- a/drivers/clk/rockchip/clk-pll.c
+++ b/drivers/clk/rockchip/clk-pll.c
@@ -46,6 +46,8 @@
const struct rockchip_pll_rate_table *rate_table;
unsigned int rate_count;
spinlock_t *lock;
+
+ struct rockchip_clk_provider *ctx;
};
#define to_rockchip_clk_pll(_hw) container_of(_hw, struct rockchip_clk_pll, hw)
@@ -90,15 +92,10 @@
*/
static int rockchip_pll_wait_lock(struct rockchip_clk_pll *pll)
{
- struct regmap *grf = rockchip_clk_get_grf();
+ struct regmap *grf = pll->ctx->grf;
unsigned int val;
int delay = 24000000, ret;
- if (IS_ERR(grf)) {
- pr_err("%s: grf regmap not available\n", __func__);
- return PTR_ERR(grf);
- }
-
while (delay > 0) {
ret = regmap_read(grf, pll->lock_offset, &val);
if (ret) {
@@ -234,7 +231,7 @@
/* wait for the pll to lock */
ret = rockchip_pll_wait_lock(pll);
if (ret) {
- pr_warn("%s: pll update unsucessful, trying to restore old params\n",
+ pr_warn("%s: pll update unsuccessful, trying to restore old params\n",
__func__);
rockchip_rk3036_pll_set_params(pll, &cur);
}
@@ -250,17 +247,9 @@
{
struct rockchip_clk_pll *pll = to_rockchip_clk_pll(hw);
const struct rockchip_pll_rate_table *rate;
- unsigned long old_rate = rockchip_rk3036_pll_recalc_rate(hw, prate);
- struct regmap *grf = rockchip_clk_get_grf();
- if (IS_ERR(grf)) {
- pr_debug("%s: grf regmap not available, aborting rate change\n",
- __func__);
- return PTR_ERR(grf);
- }
-
- pr_debug("%s: changing %s from %lu to %lu with a parent rate of %lu\n",
- __func__, __clk_get_name(hw->clk), old_rate, drate, prate);
+ pr_debug("%s: changing %s to %lu with a parent rate of %lu\n",
+ __func__, __clk_get_name(hw->clk), drate, prate);
/* Get required rate settings from table */
rate = rockchip_get_pll_settings(pll, drate);
@@ -473,7 +462,7 @@
/* wait for the pll to lock */
ret = rockchip_pll_wait_lock(pll);
if (ret) {
- pr_warn("%s: pll update unsucessful, trying to restore old params\n",
+ pr_warn("%s: pll update unsuccessful, trying to restore old params\n",
__func__);
rockchip_rk3066_pll_set_params(pll, &cur);
}
@@ -489,17 +478,9 @@
{
struct rockchip_clk_pll *pll = to_rockchip_clk_pll(hw);
const struct rockchip_pll_rate_table *rate;
- unsigned long old_rate = rockchip_rk3066_pll_recalc_rate(hw, prate);
- struct regmap *grf = rockchip_clk_get_grf();
- if (IS_ERR(grf)) {
- pr_debug("%s: grf regmap not available, aborting rate change\n",
- __func__);
- return PTR_ERR(grf);
- }
-
- pr_debug("%s: changing %s from %lu to %lu with a parent rate of %lu\n",
- __func__, clk_hw_get_name(hw), old_rate, drate, prate);
+ pr_debug("%s: changing %s to %lu with a parent rate of %lu\n",
+ __func__, clk_hw_get_name(hw), drate, prate);
/* Get required rate settings from table */
rate = rockchip_get_pll_settings(pll, drate);
@@ -563,11 +544,6 @@
rate->no, cur.no, rate->nf, cur.nf, rate->nb, cur.nb);
if (rate->nr != cur.nr || rate->no != cur.no || rate->nf != cur.nf
|| rate->nb != cur.nb) {
- struct regmap *grf = rockchip_clk_get_grf();
-
- if (IS_ERR(grf))
- return;
-
pr_debug("%s: pll %s: rate params do not match rate table, adjusting\n",
__func__, clk_hw_get_name(hw));
rockchip_rk3066_pll_set_params(pll, rate);
@@ -591,16 +567,277 @@
.init = rockchip_rk3066_pll_init,
};
+/**
+ * PLL used in RK3399
+ */
+
+#define RK3399_PLLCON(i) (i * 0x4)
+#define RK3399_PLLCON0_FBDIV_MASK 0xfff
+#define RK3399_PLLCON0_FBDIV_SHIFT 0
+#define RK3399_PLLCON1_REFDIV_MASK 0x3f
+#define RK3399_PLLCON1_REFDIV_SHIFT 0
+#define RK3399_PLLCON1_POSTDIV1_MASK 0x7
+#define RK3399_PLLCON1_POSTDIV1_SHIFT 8
+#define RK3399_PLLCON1_POSTDIV2_MASK 0x7
+#define RK3399_PLLCON1_POSTDIV2_SHIFT 12
+#define RK3399_PLLCON2_FRAC_MASK 0xffffff
+#define RK3399_PLLCON2_FRAC_SHIFT 0
+#define RK3399_PLLCON2_LOCK_STATUS BIT(31)
+#define RK3399_PLLCON3_PWRDOWN BIT(0)
+#define RK3399_PLLCON3_DSMPD_MASK 0x1
+#define RK3399_PLLCON3_DSMPD_SHIFT 3
+
+static int rockchip_rk3399_pll_wait_lock(struct rockchip_clk_pll *pll)
+{
+ u32 pllcon;
+ int delay = 24000000;
+
+ /* poll check the lock status in rk3399 xPLLCON2 */
+ while (delay > 0) {
+ pllcon = readl_relaxed(pll->reg_base + RK3399_PLLCON(2));
+ if (pllcon & RK3399_PLLCON2_LOCK_STATUS)
+ return 0;
+
+ delay--;
+ }
+
+ pr_err("%s: timeout waiting for pll to lock\n", __func__);
+ return -ETIMEDOUT;
+}
+
+static void rockchip_rk3399_pll_get_params(struct rockchip_clk_pll *pll,
+ struct rockchip_pll_rate_table *rate)
+{
+ u32 pllcon;
+
+ pllcon = readl_relaxed(pll->reg_base + RK3399_PLLCON(0));
+ rate->fbdiv = ((pllcon >> RK3399_PLLCON0_FBDIV_SHIFT)
+ & RK3399_PLLCON0_FBDIV_MASK);
+
+ pllcon = readl_relaxed(pll->reg_base + RK3399_PLLCON(1));
+ rate->refdiv = ((pllcon >> RK3399_PLLCON1_REFDIV_SHIFT)
+ & RK3399_PLLCON1_REFDIV_MASK);
+ rate->postdiv1 = ((pllcon >> RK3399_PLLCON1_POSTDIV1_SHIFT)
+ & RK3399_PLLCON1_POSTDIV1_MASK);
+ rate->postdiv2 = ((pllcon >> RK3399_PLLCON1_POSTDIV2_SHIFT)
+ & RK3399_PLLCON1_POSTDIV2_MASK);
+
+ pllcon = readl_relaxed(pll->reg_base + RK3399_PLLCON(2));
+ rate->frac = ((pllcon >> RK3399_PLLCON2_FRAC_SHIFT)
+ & RK3399_PLLCON2_FRAC_MASK);
+
+ pllcon = readl_relaxed(pll->reg_base + RK3399_PLLCON(3));
+ rate->dsmpd = ((pllcon >> RK3399_PLLCON3_DSMPD_SHIFT)
+ & RK3399_PLLCON3_DSMPD_MASK);
+}
+
+static unsigned long rockchip_rk3399_pll_recalc_rate(struct clk_hw *hw,
+ unsigned long prate)
+{
+ struct rockchip_clk_pll *pll = to_rockchip_clk_pll(hw);
+ struct rockchip_pll_rate_table cur;
+ u64 rate64 = prate;
+
+ rockchip_rk3399_pll_get_params(pll, &cur);
+
+ rate64 *= cur.fbdiv;
+ do_div(rate64, cur.refdiv);
+
+ if (cur.dsmpd == 0) {
+ /* fractional mode */
+ u64 frac_rate64 = prate * cur.frac;
+
+ do_div(frac_rate64, cur.refdiv);
+ rate64 += frac_rate64 >> 24;
+ }
+
+ do_div(rate64, cur.postdiv1);
+ do_div(rate64, cur.postdiv2);
+
+ return (unsigned long)rate64;
+}
+
+static int rockchip_rk3399_pll_set_params(struct rockchip_clk_pll *pll,
+ const struct rockchip_pll_rate_table *rate)
+{
+ const struct clk_ops *pll_mux_ops = pll->pll_mux_ops;
+ struct clk_mux *pll_mux = &pll->pll_mux;
+ struct rockchip_pll_rate_table cur;
+ u32 pllcon;
+ int rate_change_remuxed = 0;
+ int cur_parent;
+ int ret;
+
+ pr_debug("%s: rate settings for %lu fbdiv: %d, postdiv1: %d, refdiv: %d, postdiv2: %d, dsmpd: %d, frac: %d\n",
+ __func__, rate->rate, rate->fbdiv, rate->postdiv1, rate->refdiv,
+ rate->postdiv2, rate->dsmpd, rate->frac);
+
+ rockchip_rk3399_pll_get_params(pll, &cur);
+ cur.rate = 0;
+
+ cur_parent = pll_mux_ops->get_parent(&pll_mux->hw);
+ if (cur_parent == PLL_MODE_NORM) {
+ pll_mux_ops->set_parent(&pll_mux->hw, PLL_MODE_SLOW);
+ rate_change_remuxed = 1;
+ }
+
+ /* update pll values */
+ writel_relaxed(HIWORD_UPDATE(rate->fbdiv, RK3399_PLLCON0_FBDIV_MASK,
+ RK3399_PLLCON0_FBDIV_SHIFT),
+ pll->reg_base + RK3399_PLLCON(0));
+
+ writel_relaxed(HIWORD_UPDATE(rate->refdiv, RK3399_PLLCON1_REFDIV_MASK,
+ RK3399_PLLCON1_REFDIV_SHIFT) |
+ HIWORD_UPDATE(rate->postdiv1, RK3399_PLLCON1_POSTDIV1_MASK,
+ RK3399_PLLCON1_POSTDIV1_SHIFT) |
+ HIWORD_UPDATE(rate->postdiv2, RK3399_PLLCON1_POSTDIV2_MASK,
+ RK3399_PLLCON1_POSTDIV2_SHIFT),
+ pll->reg_base + RK3399_PLLCON(1));
+
+ /* xPLL CON2 is not HIWORD_MASK */
+ pllcon = readl_relaxed(pll->reg_base + RK3399_PLLCON(2));
+ pllcon &= ~(RK3399_PLLCON2_FRAC_MASK << RK3399_PLLCON2_FRAC_SHIFT);
+ pllcon |= rate->frac << RK3399_PLLCON2_FRAC_SHIFT;
+ writel_relaxed(pllcon, pll->reg_base + RK3399_PLLCON(2));
+
+ writel_relaxed(HIWORD_UPDATE(rate->dsmpd, RK3399_PLLCON3_DSMPD_MASK,
+ RK3399_PLLCON3_DSMPD_SHIFT),
+ pll->reg_base + RK3399_PLLCON(3));
+
+ /* wait for the pll to lock */
+ ret = rockchip_rk3399_pll_wait_lock(pll);
+ if (ret) {
+ pr_warn("%s: pll update unsuccessful, trying to restore old params\n",
+ __func__);
+ rockchip_rk3399_pll_set_params(pll, &cur);
+ }
+
+ if (rate_change_remuxed)
+ pll_mux_ops->set_parent(&pll_mux->hw, PLL_MODE_NORM);
+
+ return ret;
+}
+
+static int rockchip_rk3399_pll_set_rate(struct clk_hw *hw, unsigned long drate,
+ unsigned long prate)
+{
+ struct rockchip_clk_pll *pll = to_rockchip_clk_pll(hw);
+ const struct rockchip_pll_rate_table *rate;
+
+ pr_debug("%s: changing %s to %lu with a parent rate of %lu\n",
+ __func__, __clk_get_name(hw->clk), drate, prate);
+
+ /* Get required rate settings from table */
+ rate = rockchip_get_pll_settings(pll, drate);
+ if (!rate) {
+ pr_err("%s: Invalid rate : %lu for pll clk %s\n", __func__,
+ drate, __clk_get_name(hw->clk));
+ return -EINVAL;
+ }
+
+ return rockchip_rk3399_pll_set_params(pll, rate);
+}
+
+static int rockchip_rk3399_pll_enable(struct clk_hw *hw)
+{
+ struct rockchip_clk_pll *pll = to_rockchip_clk_pll(hw);
+
+ writel(HIWORD_UPDATE(0, RK3399_PLLCON3_PWRDOWN, 0),
+ pll->reg_base + RK3399_PLLCON(3));
+
+ return 0;
+}
+
+static void rockchip_rk3399_pll_disable(struct clk_hw *hw)
+{
+ struct rockchip_clk_pll *pll = to_rockchip_clk_pll(hw);
+
+ writel(HIWORD_UPDATE(RK3399_PLLCON3_PWRDOWN,
+ RK3399_PLLCON3_PWRDOWN, 0),
+ pll->reg_base + RK3399_PLLCON(3));
+}
+
+static int rockchip_rk3399_pll_is_enabled(struct clk_hw *hw)
+{
+ struct rockchip_clk_pll *pll = to_rockchip_clk_pll(hw);
+ u32 pllcon = readl(pll->reg_base + RK3399_PLLCON(3));
+
+ return !(pllcon & RK3399_PLLCON3_PWRDOWN);
+}
+
+static void rockchip_rk3399_pll_init(struct clk_hw *hw)
+{
+ struct rockchip_clk_pll *pll = to_rockchip_clk_pll(hw);
+ const struct rockchip_pll_rate_table *rate;
+ struct rockchip_pll_rate_table cur;
+ unsigned long drate;
+
+ if (!(pll->flags & ROCKCHIP_PLL_SYNC_RATE))
+ return;
+
+ drate = clk_hw_get_rate(hw);
+ rate = rockchip_get_pll_settings(pll, drate);
+
+ /* when no rate setting for the current rate, rely on clk_set_rate */
+ if (!rate)
+ return;
+
+ rockchip_rk3399_pll_get_params(pll, &cur);
+
+ pr_debug("%s: pll %s@%lu: Hz\n", __func__, __clk_get_name(hw->clk),
+ drate);
+ pr_debug("old - fbdiv: %d, postdiv1: %d, refdiv: %d, postdiv2: %d, dsmpd: %d, frac: %d\n",
+ cur.fbdiv, cur.postdiv1, cur.refdiv, cur.postdiv2,
+ cur.dsmpd, cur.frac);
+ pr_debug("new - fbdiv: %d, postdiv1: %d, refdiv: %d, postdiv2: %d, dsmpd: %d, frac: %d\n",
+ rate->fbdiv, rate->postdiv1, rate->refdiv, rate->postdiv2,
+ rate->dsmpd, rate->frac);
+
+ if (rate->fbdiv != cur.fbdiv || rate->postdiv1 != cur.postdiv1 ||
+ rate->refdiv != cur.refdiv || rate->postdiv2 != cur.postdiv2 ||
+ rate->dsmpd != cur.dsmpd || rate->frac != cur.frac) {
+ struct clk *parent = clk_get_parent(hw->clk);
+
+ if (!parent) {
+ pr_warn("%s: parent of %s not available\n",
+ __func__, __clk_get_name(hw->clk));
+ return;
+ }
+
+ pr_debug("%s: pll %s: rate params do not match rate table, adjusting\n",
+ __func__, __clk_get_name(hw->clk));
+ rockchip_rk3399_pll_set_params(pll, rate);
+ }
+}
+
+static const struct clk_ops rockchip_rk3399_pll_clk_norate_ops = {
+ .recalc_rate = rockchip_rk3399_pll_recalc_rate,
+ .enable = rockchip_rk3399_pll_enable,
+ .disable = rockchip_rk3399_pll_disable,
+ .is_enabled = rockchip_rk3399_pll_is_enabled,
+};
+
+static const struct clk_ops rockchip_rk3399_pll_clk_ops = {
+ .recalc_rate = rockchip_rk3399_pll_recalc_rate,
+ .round_rate = rockchip_pll_round_rate,
+ .set_rate = rockchip_rk3399_pll_set_rate,
+ .enable = rockchip_rk3399_pll_enable,
+ .disable = rockchip_rk3399_pll_disable,
+ .is_enabled = rockchip_rk3399_pll_is_enabled,
+ .init = rockchip_rk3399_pll_init,
+};
+
/*
* Common registering of pll clocks
*/
-struct clk *rockchip_clk_register_pll(enum rockchip_pll_type pll_type,
+struct clk *rockchip_clk_register_pll(struct rockchip_clk_provider *ctx,
+ enum rockchip_pll_type pll_type,
const char *name, const char *const *parent_names,
- u8 num_parents, void __iomem *base, int con_offset,
- int grf_lock_offset, int lock_shift, int mode_offset,
- int mode_shift, struct rockchip_pll_rate_table *rate_table,
- u8 clk_pll_flags, spinlock_t *lock)
+ u8 num_parents, int con_offset, int grf_lock_offset,
+ int lock_shift, int mode_offset, int mode_shift,
+ struct rockchip_pll_rate_table *rate_table,
+ u8 clk_pll_flags)
{
const char *pll_parents[3];
struct clk_init_data init;
@@ -624,14 +861,16 @@
/* create the mux on top of the real pll */
pll->pll_mux_ops = &clk_mux_ops;
pll_mux = &pll->pll_mux;
- pll_mux->reg = base + mode_offset;
+ pll_mux->reg = ctx->reg_base + mode_offset;
pll_mux->shift = mode_shift;
pll_mux->mask = PLL_MODE_MASK;
pll_mux->flags = 0;
- pll_mux->lock = lock;
+ pll_mux->lock = &ctx->lock;
pll_mux->hw.init = &init;
- if (pll_type == pll_rk3036 || pll_type == pll_rk3066)
+ if (pll_type == pll_rk3036 ||
+ pll_type == pll_rk3066 ||
+ pll_type == pll_rk3399)
pll_mux->flags |= CLK_MUX_HIWORD_MASK;
/* the actual muxing is xin24m, pll-output, xin32k */
@@ -677,17 +916,23 @@
switch (pll_type) {
case pll_rk3036:
- if (!pll->rate_table)
+ if (!pll->rate_table || IS_ERR(ctx->grf))
init.ops = &rockchip_rk3036_pll_clk_norate_ops;
else
init.ops = &rockchip_rk3036_pll_clk_ops;
break;
case pll_rk3066:
- if (!pll->rate_table)
+ if (!pll->rate_table || IS_ERR(ctx->grf))
init.ops = &rockchip_rk3066_pll_clk_norate_ops;
else
init.ops = &rockchip_rk3066_pll_clk_ops;
break;
+ case pll_rk3399:
+ if (!pll->rate_table)
+ init.ops = &rockchip_rk3399_pll_clk_norate_ops;
+ else
+ init.ops = &rockchip_rk3399_pll_clk_ops;
+ break;
default:
pr_warn("%s: Unknown pll type for pll clk %s\n",
__func__, name);
@@ -695,11 +940,12 @@
pll->hw.init = &init;
pll->type = pll_type;
- pll->reg_base = base + con_offset;
+ pll->reg_base = ctx->reg_base + con_offset;
pll->lock_offset = grf_lock_offset;
pll->lock_shift = lock_shift;
pll->flags = clk_pll_flags;
- pll->lock = lock;
+ pll->lock = &ctx->lock;
+ pll->ctx = ctx;
pll_clk = clk_register(NULL, &pll->hw);
if (IS_ERR(pll_clk)) {
diff --git a/drivers/clk/rockchip/clk-rk3036.c b/drivers/clk/rockchip/clk-rk3036.c
index 7cdb2d6..924f560 100644
--- a/drivers/clk/rockchip/clk-rk3036.c
+++ b/drivers/clk/rockchip/clk-rk3036.c
@@ -113,7 +113,10 @@
.core_reg = RK2928_CLKSEL_CON(0),
.div_core_shift = 0,
.div_core_mask = 0x1f,
+ .mux_core_alt = 1,
+ .mux_core_main = 0,
.mux_core_shift = 7,
+ .mux_core_mask = 0x1,
};
PNAME(mux_pll_p) = { "xin24m", "xin24m" };
@@ -437,6 +440,7 @@
static void __init rk3036_clk_init(struct device_node *np)
{
+ struct rockchip_clk_provider *ctx;
void __iomem *reg_base;
struct clk *clk;
@@ -446,22 +450,27 @@
return;
}
- rockchip_clk_init(np, reg_base, CLK_NR_CLKS);
+ ctx = rockchip_clk_init(np, reg_base, CLK_NR_CLKS);
+ if (IS_ERR(ctx)) {
+ pr_err("%s: rockchip clk init failed\n", __func__);
+ iounmap(reg_base);
+ return;
+ }
clk = clk_register_fixed_factor(NULL, "usb480m", "xin24m", 0, 20, 1);
if (IS_ERR(clk))
pr_warn("%s: could not register clock usb480m: %ld\n",
__func__, PTR_ERR(clk));
- rockchip_clk_register_plls(rk3036_pll_clks,
+ rockchip_clk_register_plls(ctx, rk3036_pll_clks,
ARRAY_SIZE(rk3036_pll_clks),
RK3036_GRF_SOC_STATUS0);
- rockchip_clk_register_branches(rk3036_clk_branches,
+ rockchip_clk_register_branches(ctx, rk3036_clk_branches,
ARRAY_SIZE(rk3036_clk_branches));
rockchip_clk_protect_critical(rk3036_critical_clocks,
ARRAY_SIZE(rk3036_critical_clocks));
- rockchip_clk_register_armclk(ARMCLK, "armclk",
+ rockchip_clk_register_armclk(ctx, ARMCLK, "armclk",
mux_armclk_p, ARRAY_SIZE(mux_armclk_p),
&rk3036_cpuclk_data, rk3036_cpuclk_rates,
ARRAY_SIZE(rk3036_cpuclk_rates));
@@ -469,6 +478,8 @@
rockchip_register_softrst(np, 9, reg_base + RK2928_SOFTRST_CON(0),
ROCKCHIP_SOFTRST_HIWORD_MASK);
- rockchip_register_restart_notifier(RK2928_GLB_SRST_FST, NULL);
+ rockchip_register_restart_notifier(ctx, RK2928_GLB_SRST_FST, NULL);
+
+ rockchip_clk_of_add_provider(np, ctx);
}
CLK_OF_DECLARE(rk3036_cru, "rockchip,rk3036-cru", rk3036_clk_init);
diff --git a/drivers/clk/rockchip/clk-rk3188.c b/drivers/clk/rockchip/clk-rk3188.c
index 40bab39..d0e722a 100644
--- a/drivers/clk/rockchip/clk-rk3188.c
+++ b/drivers/clk/rockchip/clk-rk3188.c
@@ -155,7 +155,10 @@
.core_reg = RK2928_CLKSEL_CON(0),
.div_core_shift = 0,
.div_core_mask = 0x1f,
+ .mux_core_alt = 1,
+ .mux_core_main = 0,
.mux_core_shift = 8,
+ .mux_core_mask = 0x1,
};
#define RK3188_DIV_ACLK_CORE_MASK 0x7
@@ -191,7 +194,10 @@
.core_reg = RK2928_CLKSEL_CON(0),
.div_core_shift = 9,
.div_core_mask = 0x1f,
+ .mux_core_alt = 1,
+ .mux_core_main = 0,
.mux_core_shift = 8,
+ .mux_core_mask = 0x1,
};
PNAME(mux_pll_p) = { "xin24m", "xin32k" };
@@ -753,57 +759,75 @@
"hclk_cpubus"
};
-static void __init rk3188_common_clk_init(struct device_node *np)
+static struct rockchip_clk_provider *__init rk3188_common_clk_init(struct device_node *np)
{
+ struct rockchip_clk_provider *ctx;
void __iomem *reg_base;
reg_base = of_iomap(np, 0);
if (!reg_base) {
pr_err("%s: could not map cru region\n", __func__);
- return;
+ return ERR_PTR(-ENOMEM);
}
- rockchip_clk_init(np, reg_base, CLK_NR_CLKS);
+ ctx = rockchip_clk_init(np, reg_base, CLK_NR_CLKS);
+ if (IS_ERR(ctx)) {
+ pr_err("%s: rockchip clk init failed\n", __func__);
+ iounmap(reg_base);
+ return ERR_PTR(-ENOMEM);
+ }
- rockchip_clk_register_branches(common_clk_branches,
+ rockchip_clk_register_branches(ctx, common_clk_branches,
ARRAY_SIZE(common_clk_branches));
rockchip_register_softrst(np, 9, reg_base + RK2928_SOFTRST_CON(0),
ROCKCHIP_SOFTRST_HIWORD_MASK);
- rockchip_register_restart_notifier(RK2928_GLB_SRST_FST, NULL);
+ rockchip_register_restart_notifier(ctx, RK2928_GLB_SRST_FST, NULL);
+
+ return ctx;
}
static void __init rk3066a_clk_init(struct device_node *np)
{
- rk3188_common_clk_init(np);
- rockchip_clk_register_plls(rk3066_pll_clks,
+ struct rockchip_clk_provider *ctx;
+
+ ctx = rk3188_common_clk_init(np);
+ if (IS_ERR(ctx))
+ return;
+
+ rockchip_clk_register_plls(ctx, rk3066_pll_clks,
ARRAY_SIZE(rk3066_pll_clks),
RK3066_GRF_SOC_STATUS);
- rockchip_clk_register_branches(rk3066a_clk_branches,
+ rockchip_clk_register_branches(ctx, rk3066a_clk_branches,
ARRAY_SIZE(rk3066a_clk_branches));
- rockchip_clk_register_armclk(ARMCLK, "armclk",
+ rockchip_clk_register_armclk(ctx, ARMCLK, "armclk",
mux_armclk_p, ARRAY_SIZE(mux_armclk_p),
&rk3066_cpuclk_data, rk3066_cpuclk_rates,
ARRAY_SIZE(rk3066_cpuclk_rates));
rockchip_clk_protect_critical(rk3188_critical_clocks,
ARRAY_SIZE(rk3188_critical_clocks));
+ rockchip_clk_of_add_provider(np, ctx);
}
CLK_OF_DECLARE(rk3066a_cru, "rockchip,rk3066a-cru", rk3066a_clk_init);
static void __init rk3188a_clk_init(struct device_node *np)
{
+ struct rockchip_clk_provider *ctx;
struct clk *clk1, *clk2;
unsigned long rate;
int ret;
- rk3188_common_clk_init(np);
- rockchip_clk_register_plls(rk3188_pll_clks,
+ ctx = rk3188_common_clk_init(np);
+ if (IS_ERR(ctx))
+ return;
+
+ rockchip_clk_register_plls(ctx, rk3188_pll_clks,
ARRAY_SIZE(rk3188_pll_clks),
RK3188_GRF_SOC_STATUS);
- rockchip_clk_register_branches(rk3188_clk_branches,
+ rockchip_clk_register_branches(ctx, rk3188_clk_branches,
ARRAY_SIZE(rk3188_clk_branches));
- rockchip_clk_register_armclk(ARMCLK, "armclk",
+ rockchip_clk_register_armclk(ctx, ARMCLK, "armclk",
mux_armclk_p, ARRAY_SIZE(mux_armclk_p),
&rk3188_cpuclk_data, rk3188_cpuclk_rates,
ARRAY_SIZE(rk3188_cpuclk_rates));
@@ -827,6 +851,7 @@
rockchip_clk_protect_critical(rk3188_critical_clocks,
ARRAY_SIZE(rk3188_critical_clocks));
+ rockchip_clk_of_add_provider(np, ctx);
}
CLK_OF_DECLARE(rk3188a_cru, "rockchip,rk3188a-cru", rk3188a_clk_init);
diff --git a/drivers/clk/rockchip/clk-rk3228.c b/drivers/clk/rockchip/clk-rk3228.c
index 7702d28..016bdb0 100644
--- a/drivers/clk/rockchip/clk-rk3228.c
+++ b/drivers/clk/rockchip/clk-rk3228.c
@@ -111,7 +111,10 @@
.core_reg = RK2928_CLKSEL_CON(0),
.div_core_shift = 0,
.div_core_mask = 0x1f,
+ .mux_core_alt = 1,
+ .mux_core_main = 0,
.mux_core_shift = 6,
+ .mux_core_mask = 0x1,
};
PNAME(mux_pll_p) = { "clk_24m", "xin24m" };
@@ -625,6 +628,7 @@
static void __init rk3228_clk_init(struct device_node *np)
{
+ struct rockchip_clk_provider *ctx;
void __iomem *reg_base;
reg_base = of_iomap(np, 0);
@@ -633,17 +637,22 @@
return;
}
- rockchip_clk_init(np, reg_base, CLK_NR_CLKS);
+ ctx = rockchip_clk_init(np, reg_base, CLK_NR_CLKS);
+ if (IS_ERR(ctx)) {
+ pr_err("%s: rockchip clk init failed\n", __func__);
+ iounmap(reg_base);
+ return;
+ }
- rockchip_clk_register_plls(rk3228_pll_clks,
+ rockchip_clk_register_plls(ctx, rk3228_pll_clks,
ARRAY_SIZE(rk3228_pll_clks),
RK3228_GRF_SOC_STATUS0);
- rockchip_clk_register_branches(rk3228_clk_branches,
+ rockchip_clk_register_branches(ctx, rk3228_clk_branches,
ARRAY_SIZE(rk3228_clk_branches));
rockchip_clk_protect_critical(rk3228_critical_clocks,
ARRAY_SIZE(rk3228_critical_clocks));
- rockchip_clk_register_armclk(ARMCLK, "armclk",
+ rockchip_clk_register_armclk(ctx, ARMCLK, "armclk",
mux_armclk_p, ARRAY_SIZE(mux_armclk_p),
&rk3228_cpuclk_data, rk3228_cpuclk_rates,
ARRAY_SIZE(rk3228_cpuclk_rates));
@@ -651,6 +660,8 @@
rockchip_register_softrst(np, 9, reg_base + RK2928_SOFTRST_CON(0),
ROCKCHIP_SOFTRST_HIWORD_MASK);
- rockchip_register_restart_notifier(RK3228_GLB_SRST_FST, NULL);
+ rockchip_register_restart_notifier(ctx, RK3228_GLB_SRST_FST, NULL);
+
+ rockchip_clk_of_add_provider(np, ctx);
}
CLK_OF_DECLARE(rk3228_cru, "rockchip,rk3228-cru", rk3228_clk_init);
diff --git a/drivers/clk/rockchip/clk-rk3288.c b/drivers/clk/rockchip/clk-rk3288.c
index 3cb7216..39af05a 100644
--- a/drivers/clk/rockchip/clk-rk3288.c
+++ b/drivers/clk/rockchip/clk-rk3288.c
@@ -165,7 +165,10 @@
.core_reg = RK3288_CLKSEL_CON(0),
.div_core_shift = 8,
.div_core_mask = 0x1f,
+ .mux_core_alt = 1,
+ .mux_core_main = 0,
.mux_core_shift = 15,
+ .mux_core_mask = 0x1,
};
PNAME(mux_pll_p) = { "xin24m", "xin32k" };
@@ -878,6 +881,7 @@
static void __init rk3288_clk_init(struct device_node *np)
{
+ struct rockchip_clk_provider *ctx;
struct clk *clk;
rk3288_cru_base = of_iomap(np, 0);
@@ -886,7 +890,12 @@
return;
}
- rockchip_clk_init(np, rk3288_cru_base, CLK_NR_CLKS);
+ ctx = rockchip_clk_init(np, rk3288_cru_base, CLK_NR_CLKS);
+ if (IS_ERR(ctx)) {
+ pr_err("%s: rockchip clk init failed\n", __func__);
+ iounmap(rk3288_cru_base);
+ return;
+ }
/* Watchdog pclk is controlled by RK3288_SGRF_SOC_CON0[1]. */
clk = clk_register_fixed_factor(NULL, "pclk_wdt", "pclk_pd_alive", 0, 1, 1);
@@ -894,17 +903,17 @@
pr_warn("%s: could not register clock pclk_wdt: %ld\n",
__func__, PTR_ERR(clk));
else
- rockchip_clk_add_lookup(clk, PCLK_WDT);
+ rockchip_clk_add_lookup(ctx, clk, PCLK_WDT);
- rockchip_clk_register_plls(rk3288_pll_clks,
+ rockchip_clk_register_plls(ctx, rk3288_pll_clks,
ARRAY_SIZE(rk3288_pll_clks),
RK3288_GRF_SOC_STATUS1);
- rockchip_clk_register_branches(rk3288_clk_branches,
+ rockchip_clk_register_branches(ctx, rk3288_clk_branches,
ARRAY_SIZE(rk3288_clk_branches));
rockchip_clk_protect_critical(rk3288_critical_clocks,
ARRAY_SIZE(rk3288_critical_clocks));
- rockchip_clk_register_armclk(ARMCLK, "armclk",
+ rockchip_clk_register_armclk(ctx, ARMCLK, "armclk",
mux_armclk_p, ARRAY_SIZE(mux_armclk_p),
&rk3288_cpuclk_data, rk3288_cpuclk_rates,
ARRAY_SIZE(rk3288_cpuclk_rates));
@@ -913,8 +922,10 @@
rk3288_cru_base + RK3288_SOFTRST_CON(0),
ROCKCHIP_SOFTRST_HIWORD_MASK);
- rockchip_register_restart_notifier(RK3288_GLB_SRST_FST,
+ rockchip_register_restart_notifier(ctx, RK3288_GLB_SRST_FST,
rk3288_clk_shutdown);
register_syscore_ops(&rk3288_clk_syscore_ops);
+
+ rockchip_clk_of_add_provider(np, ctx);
}
CLK_OF_DECLARE(rk3288_cru, "rockchip,rk3288-cru", rk3288_clk_init);
diff --git a/drivers/clk/rockchip/clk-rk3368.c b/drivers/clk/rockchip/clk-rk3368.c
index a2bb122..6cb474c 100644
--- a/drivers/clk/rockchip/clk-rk3368.c
+++ b/drivers/clk/rockchip/clk-rk3368.c
@@ -165,14 +165,20 @@
.core_reg = RK3368_CLKSEL_CON(0),
.div_core_shift = 0,
.div_core_mask = 0x1f,
+ .mux_core_alt = 1,
+ .mux_core_main = 0,
.mux_core_shift = 7,
+ .mux_core_mask = 0x1,
};
static const struct rockchip_cpuclk_reg_data rk3368_cpuclkl_data = {
.core_reg = RK3368_CLKSEL_CON(2),
.div_core_shift = 0,
+ .mux_core_alt = 1,
+ .mux_core_main = 0,
.div_core_mask = 0x1f,
.mux_core_shift = 7,
+ .mux_core_mask = 0x1,
};
#define RK3368_DIV_ACLKM_MASK 0x1f
@@ -856,6 +862,7 @@
static void __init rk3368_clk_init(struct device_node *np)
{
+ struct rockchip_clk_provider *ctx;
void __iomem *reg_base;
struct clk *clk;
@@ -865,7 +872,12 @@
return;
}
- rockchip_clk_init(np, reg_base, CLK_NR_CLKS);
+ ctx = rockchip_clk_init(np, reg_base, CLK_NR_CLKS);
+ if (IS_ERR(ctx)) {
+ pr_err("%s: rockchip clk init failed\n", __func__);
+ iounmap(reg_base);
+ return;
+ }
/* Watchdog pclk is controlled by sgrf_soc_con3[7]. */
clk = clk_register_fixed_factor(NULL, "pclk_wdt", "pclk_pd_alive", 0, 1, 1);
@@ -873,22 +885,22 @@
pr_warn("%s: could not register clock pclk_wdt: %ld\n",
__func__, PTR_ERR(clk));
else
- rockchip_clk_add_lookup(clk, PCLK_WDT);
+ rockchip_clk_add_lookup(ctx, clk, PCLK_WDT);
- rockchip_clk_register_plls(rk3368_pll_clks,
+ rockchip_clk_register_plls(ctx, rk3368_pll_clks,
ARRAY_SIZE(rk3368_pll_clks),
RK3368_GRF_SOC_STATUS0);
- rockchip_clk_register_branches(rk3368_clk_branches,
+ rockchip_clk_register_branches(ctx, rk3368_clk_branches,
ARRAY_SIZE(rk3368_clk_branches));
rockchip_clk_protect_critical(rk3368_critical_clocks,
ARRAY_SIZE(rk3368_critical_clocks));
- rockchip_clk_register_armclk(ARMCLKB, "armclkb",
+ rockchip_clk_register_armclk(ctx, ARMCLKB, "armclkb",
mux_armclkb_p, ARRAY_SIZE(mux_armclkb_p),
&rk3368_cpuclkb_data, rk3368_cpuclkb_rates,
ARRAY_SIZE(rk3368_cpuclkb_rates));
- rockchip_clk_register_armclk(ARMCLKL, "armclkl",
+ rockchip_clk_register_armclk(ctx, ARMCLKL, "armclkl",
mux_armclkl_p, ARRAY_SIZE(mux_armclkl_p),
&rk3368_cpuclkl_data, rk3368_cpuclkl_rates,
ARRAY_SIZE(rk3368_cpuclkl_rates));
@@ -896,6 +908,8 @@
rockchip_register_softrst(np, 15, reg_base + RK3368_SOFTRST_CON(0),
ROCKCHIP_SOFTRST_HIWORD_MASK);
- rockchip_register_restart_notifier(RK3368_GLB_SRST_FST, NULL);
+ rockchip_register_restart_notifier(ctx, RK3368_GLB_SRST_FST, NULL);
+
+ rockchip_clk_of_add_provider(np, ctx);
}
CLK_OF_DECLARE(rk3368_cru, "rockchip,rk3368-cru", rk3368_clk_init);
diff --git a/drivers/clk/rockchip/clk-rk3399.c b/drivers/clk/rockchip/clk-rk3399.c
new file mode 100644
index 0000000..291543f
--- /dev/null
+++ b/drivers/clk/rockchip/clk-rk3399.c
@@ -0,0 +1,1573 @@
+/*
+ * Copyright (c) 2016 Rockchip Electronics Co. Ltd.
+ * Author: Xing Zheng <zhengxing@rock-chips.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/clk-provider.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+#include <dt-bindings/clock/rk3399-cru.h>
+#include "clk.h"
+
+enum rk3399_plls {
+ lpll, bpll, dpll, cpll, gpll, npll, vpll,
+};
+
+enum rk3399_pmu_plls {
+ ppll,
+};
+
+static struct rockchip_pll_rate_table rk3399_pll_rates[] = {
+ /* _mhz, _refdiv, _fbdiv, _postdiv1, _postdiv2, _dsmpd, _frac */
+ RK3036_PLL_RATE(2208000000, 1, 92, 1, 1, 1, 0),
+ RK3036_PLL_RATE(2184000000, 1, 91, 1, 1, 1, 0),
+ RK3036_PLL_RATE(2160000000, 1, 90, 1, 1, 1, 0),
+ RK3036_PLL_RATE(2136000000, 1, 89, 1, 1, 1, 0),
+ RK3036_PLL_RATE(2112000000, 1, 88, 1, 1, 1, 0),
+ RK3036_PLL_RATE(2088000000, 1, 87, 1, 1, 1, 0),
+ RK3036_PLL_RATE(2064000000, 1, 86, 1, 1, 1, 0),
+ RK3036_PLL_RATE(2040000000, 1, 85, 1, 1, 1, 0),
+ RK3036_PLL_RATE(2016000000, 1, 84, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1992000000, 1, 83, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1968000000, 1, 82, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1944000000, 1, 81, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1920000000, 1, 80, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1896000000, 1, 79, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1872000000, 1, 78, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1848000000, 1, 77, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1824000000, 1, 76, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1800000000, 1, 75, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1776000000, 1, 74, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1752000000, 1, 73, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1728000000, 1, 72, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1704000000, 1, 71, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1680000000, 1, 70, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1656000000, 1, 69, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1632000000, 1, 68, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1608000000, 1, 67, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1584000000, 1, 66, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1560000000, 1, 65, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1536000000, 1, 64, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1512000000, 1, 63, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1488000000, 1, 62, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1464000000, 1, 61, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1440000000, 1, 60, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1416000000, 1, 59, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1392000000, 1, 58, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1368000000, 1, 57, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1344000000, 1, 56, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1320000000, 1, 55, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1296000000, 1, 54, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1272000000, 1, 53, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1248000000, 1, 52, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1200000000, 1, 50, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1188000000, 2, 99, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1104000000, 1, 46, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1100000000, 12, 550, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1008000000, 1, 84, 2, 1, 1, 0),
+ RK3036_PLL_RATE(1000000000, 6, 500, 2, 1, 1, 0),
+ RK3036_PLL_RATE( 984000000, 1, 82, 2, 1, 1, 0),
+ RK3036_PLL_RATE( 960000000, 1, 80, 2, 1, 1, 0),
+ RK3036_PLL_RATE( 936000000, 1, 78, 2, 1, 1, 0),
+ RK3036_PLL_RATE( 912000000, 1, 76, 2, 1, 1, 0),
+ RK3036_PLL_RATE( 900000000, 4, 300, 2, 1, 1, 0),
+ RK3036_PLL_RATE( 888000000, 1, 74, 2, 1, 1, 0),
+ RK3036_PLL_RATE( 864000000, 1, 72, 2, 1, 1, 0),
+ RK3036_PLL_RATE( 840000000, 1, 70, 2, 1, 1, 0),
+ RK3036_PLL_RATE( 816000000, 1, 68, 2, 1, 1, 0),
+ RK3036_PLL_RATE( 800000000, 6, 400, 2, 1, 1, 0),
+ RK3036_PLL_RATE( 700000000, 6, 350, 2, 1, 1, 0),
+ RK3036_PLL_RATE( 696000000, 1, 58, 2, 1, 1, 0),
+ RK3036_PLL_RATE( 676000000, 3, 169, 2, 1, 1, 0),
+ RK3036_PLL_RATE( 600000000, 1, 75, 3, 1, 1, 0),
+ RK3036_PLL_RATE( 594000000, 1, 99, 4, 1, 1, 0),
+ RK3036_PLL_RATE( 504000000, 1, 63, 3, 1, 1, 0),
+ RK3036_PLL_RATE( 500000000, 6, 250, 2, 1, 1, 0),
+ RK3036_PLL_RATE( 408000000, 1, 68, 2, 2, 1, 0),
+ RK3036_PLL_RATE( 312000000, 1, 52, 2, 2, 1, 0),
+ RK3036_PLL_RATE( 297000000, 1, 99, 4, 2, 1, 0),
+ RK3036_PLL_RATE( 216000000, 1, 72, 4, 2, 1, 0),
+ RK3036_PLL_RATE( 148500000, 1, 99, 4, 4, 1, 0),
+ RK3036_PLL_RATE( 96000000, 1, 64, 4, 4, 1, 0),
+ RK3036_PLL_RATE( 74250000, 2, 99, 4, 4, 1, 0),
+ RK3036_PLL_RATE( 54000000, 1, 54, 6, 4, 1, 0),
+ RK3036_PLL_RATE( 27000000, 1, 27, 6, 4, 1, 0),
+ { /* sentinel */ },
+};
+
+/* CRU parents */
+PNAME(mux_pll_p) = { "xin24m", "xin32k" };
+
+PNAME(mux_armclkl_p) = { "clk_core_l_lpll_src",
+ "clk_core_l_bpll_src",
+ "clk_core_l_dpll_src",
+ "clk_core_l_gpll_src" };
+PNAME(mux_armclkb_p) = { "clk_core_b_lpll_src",
+ "clk_core_b_bpll_src",
+ "clk_core_b_dpll_src",
+ "clk_core_b_gpll_src" };
+PNAME(mux_aclk_cci_p) = { "cpll_aclk_cci_src",
+ "gpll_aclk_cci_src",
+ "npll_aclk_cci_src",
+ "vpll_aclk_cci_src" };
+PNAME(mux_cci_trace_p) = { "cpll_cci_trace",
+ "gpll_cci_trace" };
+PNAME(mux_cs_p) = { "cpll_cs", "gpll_cs",
+ "npll_cs"};
+PNAME(mux_aclk_perihp_p) = { "cpll_aclk_perihp_src",
+ "gpll_aclk_perihp_src" };
+
+PNAME(mux_pll_src_cpll_gpll_p) = { "cpll", "gpll" };
+PNAME(mux_pll_src_cpll_gpll_npll_p) = { "cpll", "gpll", "npll" };
+PNAME(mux_pll_src_cpll_gpll_ppll_p) = { "cpll", "gpll", "ppll" };
+PNAME(mux_pll_src_cpll_gpll_upll_p) = { "cpll", "gpll", "upll" };
+PNAME(mux_pll_src_npll_cpll_gpll_p) = { "npll", "cpll", "gpll" };
+PNAME(mux_pll_src_cpll_gpll_npll_ppll_p) = { "cpll", "gpll", "npll",
+ "ppll" };
+PNAME(mux_pll_src_cpll_gpll_npll_24m_p) = { "cpll", "gpll", "npll",
+ "xin24m" };
+PNAME(mux_pll_src_cpll_gpll_npll_usbphy480m_p) = { "cpll", "gpll", "npll",
+ "clk_usbphy_480m" };
+PNAME(mux_pll_src_ppll_cpll_gpll_npll_p) = { "ppll", "cpll", "gpll",
+ "npll", "upll" };
+PNAME(mux_pll_src_cpll_gpll_npll_upll_24m_p) = { "cpll", "gpll", "npll",
+ "upll", "xin24m" };
+PNAME(mux_pll_src_cpll_gpll_npll_ppll_upll_24m_p) = { "cpll", "gpll", "npll",
+ "ppll", "upll", "xin24m" };
+
+PNAME(mux_pll_src_vpll_cpll_gpll_p) = { "vpll", "cpll", "gpll" };
+PNAME(mux_pll_src_vpll_cpll_gpll_npll_p) = { "vpll", "cpll", "gpll",
+ "npll" };
+PNAME(mux_pll_src_vpll_cpll_gpll_24m_p) = { "vpll", "cpll", "gpll",
+ "xin24m" };
+
+PNAME(mux_dclk_vop0_p) = { "dclk_vop0_div",
+ "dclk_vop0_frac" };
+PNAME(mux_dclk_vop1_p) = { "dclk_vop1_div",
+ "dclk_vop1_frac" };
+
+PNAME(mux_clk_cif_p) = { "clk_cifout_src", "xin24m" };
+
+PNAME(mux_pll_src_24m_usbphy480m_p) = { "xin24m", "clk_usbphy_480m" };
+PNAME(mux_pll_src_24m_pciephy_p) = { "xin24m", "clk_pciephy_ref100m" };
+PNAME(mux_pll_src_24m_32k_cpll_gpll_p) = { "xin24m", "xin32k",
+ "cpll", "gpll" };
+PNAME(mux_pciecore_cru_phy_p) = { "clk_pcie_core_cru",
+ "clk_pcie_core_phy" };
+
+PNAME(mux_aclk_emmc_p) = { "cpll_aclk_emmc_src",
+ "gpll_aclk_emmc_src" };
+
+PNAME(mux_aclk_perilp0_p) = { "cpll_aclk_perilp0_src",
+ "gpll_aclk_perilp0_src" };
+
+PNAME(mux_fclk_cm0s_p) = { "cpll_fclk_cm0s_src",
+ "gpll_fclk_cm0s_src" };
+
+PNAME(mux_hclk_perilp1_p) = { "cpll_hclk_perilp1_src",
+ "gpll_hclk_perilp1_src" };
+
+PNAME(mux_clk_testout1_p) = { "clk_testout1_pll_src", "xin24m" };
+PNAME(mux_clk_testout2_p) = { "clk_testout2_pll_src", "xin24m" };
+
+PNAME(mux_usbphy_480m_p) = { "clk_usbphy0_480m_src",
+ "clk_usbphy1_480m_src" };
+PNAME(mux_aclk_gmac_p) = { "cpll_aclk_gmac_src",
+ "gpll_aclk_gmac_src" };
+PNAME(mux_rmii_p) = { "clk_gmac", "clkin_gmac" };
+PNAME(mux_spdif_p) = { "clk_spdif_div", "clk_spdif_frac",
+ "clkin_i2s", "xin12m" };
+PNAME(mux_i2s0_p) = { "clk_i2s0_div", "clk_i2s0_frac",
+ "clkin_i2s", "xin12m" };
+PNAME(mux_i2s1_p) = { "clk_i2s1_div", "clk_i2s1_frac",
+ "clkin_i2s", "xin12m" };
+PNAME(mux_i2s2_p) = { "clk_i2s2_div", "clk_i2s2_frac",
+ "clkin_i2s", "xin12m" };
+PNAME(mux_i2sch_p) = { "clk_i2s0", "clk_i2s1",
+ "clk_i2s2" };
+PNAME(mux_i2sout_p) = { "clk_i2sout_src", "xin12m" };
+
+PNAME(mux_uart0_p) = { "clk_uart0_div", "clk_uart0_frac", "xin24m" };
+PNAME(mux_uart1_p) = { "clk_uart1_div", "clk_uart1_frac", "xin24m" };
+PNAME(mux_uart2_p) = { "clk_uart2_div", "clk_uart2_frac", "xin24m" };
+PNAME(mux_uart3_p) = { "clk_uart3_div", "clk_uart3_frac", "xin24m" };
+
+/* PMU CRU parents */
+PNAME(mux_ppll_24m_p) = { "ppll", "xin24m" };
+PNAME(mux_24m_ppll_p) = { "xin24m", "ppll" };
+PNAME(mux_fclk_cm0s_pmu_ppll_p) = { "fclk_cm0s_pmu_ppll_src", "xin24m" };
+PNAME(mux_wifi_pmu_p) = { "clk_wifi_div", "clk_wifi_frac" };
+PNAME(mux_uart4_pmu_p) = { "clk_uart4_div", "clk_uart4_frac",
+ "xin24m" };
+PNAME(mux_clk_testout2_2io_p) = { "clk_testout2", "clk_32k_suspend_pmu" };
+
+static struct rockchip_pll_clock rk3399_pll_clks[] __initdata = {
+ [lpll] = PLL(pll_rk3399, PLL_APLLL, "lpll", mux_pll_p, 0, RK3399_PLL_CON(0),
+ RK3399_PLL_CON(3), 8, 31, 0, rk3399_pll_rates),
+ [bpll] = PLL(pll_rk3399, PLL_APLLB, "bpll", mux_pll_p, 0, RK3399_PLL_CON(8),
+ RK3399_PLL_CON(11), 8, 31, 0, rk3399_pll_rates),
+ [dpll] = PLL(pll_rk3399, PLL_DPLL, "dpll", mux_pll_p, 0, RK3399_PLL_CON(16),
+ RK3399_PLL_CON(19), 8, 31, 0, NULL),
+ [cpll] = PLL(pll_rk3399, PLL_CPLL, "cpll", mux_pll_p, 0, RK3399_PLL_CON(24),
+ RK3399_PLL_CON(27), 8, 31, ROCKCHIP_PLL_SYNC_RATE, rk3399_pll_rates),
+ [gpll] = PLL(pll_rk3399, PLL_GPLL, "gpll", mux_pll_p, 0, RK3399_PLL_CON(32),
+ RK3399_PLL_CON(35), 8, 31, ROCKCHIP_PLL_SYNC_RATE, rk3399_pll_rates),
+ [npll] = PLL(pll_rk3399, PLL_NPLL, "npll", mux_pll_p, 0, RK3399_PLL_CON(40),
+ RK3399_PLL_CON(43), 8, 31, ROCKCHIP_PLL_SYNC_RATE, rk3399_pll_rates),
+ [vpll] = PLL(pll_rk3399, PLL_VPLL, "vpll", mux_pll_p, 0, RK3399_PLL_CON(48),
+ RK3399_PLL_CON(51), 8, 31, ROCKCHIP_PLL_SYNC_RATE, rk3399_pll_rates),
+};
+
+static struct rockchip_pll_clock rk3399_pmu_pll_clks[] __initdata = {
+ [ppll] = PLL(pll_rk3399, PLL_PPLL, "ppll", mux_pll_p, 0, RK3399_PMU_PLL_CON(0),
+ RK3399_PMU_PLL_CON(3), 8, 31, ROCKCHIP_PLL_SYNC_RATE, rk3399_pll_rates),
+};
+
+#define MFLAGS CLK_MUX_HIWORD_MASK
+#define DFLAGS CLK_DIVIDER_HIWORD_MASK
+#define GFLAGS (CLK_GATE_HIWORD_MASK | CLK_GATE_SET_TO_DISABLE)
+#define IFLAGS ROCKCHIP_INVERTER_HIWORD_MASK
+
+static struct rockchip_clk_branch rk3399_spdif_fracmux __initdata =
+ MUX(0, "clk_spdif_mux", mux_spdif_p, CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(32), 13, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3399_i2s0_fracmux __initdata =
+ MUX(0, "clk_i2s0_mux", mux_i2s0_p, CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(28), 8, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3399_i2s1_fracmux __initdata =
+ MUX(0, "clk_i2s1_mux", mux_i2s1_p, CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(29), 8, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3399_i2s2_fracmux __initdata =
+ MUX(0, "clk_i2s2_mux", mux_i2s2_p, CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(30), 8, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3399_uart0_fracmux __initdata =
+ MUX(SCLK_UART0, "clk_uart0", mux_uart0_p, CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(33), 8, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3399_uart1_fracmux __initdata =
+ MUX(SCLK_UART1, "clk_uart1", mux_uart1_p, CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(34), 8, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3399_uart2_fracmux __initdata =
+ MUX(SCLK_UART2, "clk_uart2", mux_uart2_p, CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(35), 8, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3399_uart3_fracmux __initdata =
+ MUX(SCLK_UART3, "clk_uart3", mux_uart3_p, CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(36), 8, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3399_uart4_pmu_fracmux __initdata =
+ MUX(SCLK_UART4_PMU, "clk_uart4_pmu", mux_uart4_pmu_p, CLK_SET_RATE_PARENT,
+ RK3399_PMU_CLKSEL_CON(5), 8, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3399_dclk_vop0_fracmux __initdata =
+ MUX(DCLK_VOP0, "dclk_vop0", mux_dclk_vop0_p, CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(49), 11, 1, MFLAGS);
+
+static struct rockchip_clk_branch rk3399_dclk_vop1_fracmux __initdata =
+ MUX(DCLK_VOP1, "dclk_vop1", mux_dclk_vop1_p, CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(50), 11, 1, MFLAGS);
+
+static struct rockchip_clk_branch rk3399_pmuclk_wifi_fracmux __initdata =
+ MUX(SCLK_WIFI_PMU, "clk_wifi_pmu", mux_wifi_pmu_p, CLK_SET_RATE_PARENT,
+ RK3399_PMU_CLKSEL_CON(1), 14, 1, MFLAGS);
+
+static const struct rockchip_cpuclk_reg_data rk3399_cpuclkl_data = {
+ .core_reg = RK3399_CLKSEL_CON(0),
+ .div_core_shift = 0,
+ .div_core_mask = 0x1f,
+ .mux_core_alt = 3,
+ .mux_core_main = 0,
+ .mux_core_shift = 6,
+ .mux_core_mask = 0x3,
+};
+
+static const struct rockchip_cpuclk_reg_data rk3399_cpuclkb_data = {
+ .core_reg = RK3399_CLKSEL_CON(2),
+ .div_core_shift = 0,
+ .div_core_mask = 0x1f,
+ .mux_core_alt = 3,
+ .mux_core_main = 1,
+ .mux_core_shift = 6,
+ .mux_core_mask = 0x3,
+};
+
+#define RK3399_DIV_ACLKM_MASK 0x1f
+#define RK3399_DIV_ACLKM_SHIFT 8
+#define RK3399_DIV_ATCLK_MASK 0x1f
+#define RK3399_DIV_ATCLK_SHIFT 0
+#define RK3399_DIV_PCLK_DBG_MASK 0x1f
+#define RK3399_DIV_PCLK_DBG_SHIFT 8
+
+#define RK3399_CLKSEL0(_offs, _aclkm) \
+ { \
+ .reg = RK3399_CLKSEL_CON(0 + _offs), \
+ .val = HIWORD_UPDATE(_aclkm, RK3399_DIV_ACLKM_MASK, \
+ RK3399_DIV_ACLKM_SHIFT), \
+ }
+#define RK3399_CLKSEL1(_offs, _atclk, _pdbg) \
+ { \
+ .reg = RK3399_CLKSEL_CON(1 + _offs), \
+ .val = HIWORD_UPDATE(_atclk, RK3399_DIV_ATCLK_MASK, \
+ RK3399_DIV_ATCLK_SHIFT) | \
+ HIWORD_UPDATE(_pdbg, RK3399_DIV_PCLK_DBG_MASK, \
+ RK3399_DIV_PCLK_DBG_SHIFT), \
+ }
+
+/* cluster_l: aclkm in clksel0, rest in clksel1 */
+#define RK3399_CPUCLKL_RATE(_prate, _aclkm, _atclk, _pdbg) \
+ { \
+ .prate = _prate##U, \
+ .divs = { \
+ RK3399_CLKSEL0(0, _aclkm), \
+ RK3399_CLKSEL1(0, _atclk, _pdbg), \
+ }, \
+ }
+
+/* cluster_b: aclkm in clksel2, rest in clksel3 */
+#define RK3399_CPUCLKB_RATE(_prate, _aclkm, _atclk, _pdbg) \
+ { \
+ .prate = _prate##U, \
+ .divs = { \
+ RK3399_CLKSEL0(2, _aclkm), \
+ RK3399_CLKSEL1(2, _atclk, _pdbg), \
+ }, \
+ }
+
+static struct rockchip_cpuclk_rate_table rk3399_cpuclkl_rates[] __initdata = {
+ RK3399_CPUCLKL_RATE(1800000000, 1, 8, 8),
+ RK3399_CPUCLKL_RATE(1704000000, 1, 8, 8),
+ RK3399_CPUCLKL_RATE(1608000000, 1, 7, 7),
+ RK3399_CPUCLKL_RATE(1512000000, 1, 7, 7),
+ RK3399_CPUCLKL_RATE(1488000000, 1, 6, 6),
+ RK3399_CPUCLKL_RATE(1416000000, 1, 6, 6),
+ RK3399_CPUCLKL_RATE(1200000000, 1, 5, 5),
+ RK3399_CPUCLKL_RATE(1008000000, 1, 5, 5),
+ RK3399_CPUCLKL_RATE( 816000000, 1, 4, 4),
+ RK3399_CPUCLKL_RATE( 696000000, 1, 3, 3),
+ RK3399_CPUCLKL_RATE( 600000000, 1, 3, 3),
+ RK3399_CPUCLKL_RATE( 408000000, 1, 2, 2),
+ RK3399_CPUCLKL_RATE( 312000000, 1, 1, 1),
+ RK3399_CPUCLKL_RATE( 216000000, 1, 1, 1),
+ RK3399_CPUCLKL_RATE( 96000000, 1, 1, 1),
+};
+
+static struct rockchip_cpuclk_rate_table rk3399_cpuclkb_rates[] __initdata = {
+ RK3399_CPUCLKB_RATE(2208000000, 1, 11, 11),
+ RK3399_CPUCLKB_RATE(2184000000, 1, 11, 11),
+ RK3399_CPUCLKB_RATE(2088000000, 1, 10, 10),
+ RK3399_CPUCLKB_RATE(2040000000, 1, 10, 10),
+ RK3399_CPUCLKB_RATE(1992000000, 1, 9, 9),
+ RK3399_CPUCLKB_RATE(1896000000, 1, 9, 9),
+ RK3399_CPUCLKB_RATE(1800000000, 1, 8, 8),
+ RK3399_CPUCLKB_RATE(1704000000, 1, 8, 8),
+ RK3399_CPUCLKB_RATE(1608000000, 1, 7, 7),
+ RK3399_CPUCLKB_RATE(1512000000, 1, 7, 7),
+ RK3399_CPUCLKB_RATE(1488000000, 1, 6, 6),
+ RK3399_CPUCLKB_RATE(1416000000, 1, 6, 6),
+ RK3399_CPUCLKB_RATE(1200000000, 1, 5, 5),
+ RK3399_CPUCLKB_RATE(1008000000, 1, 5, 5),
+ RK3399_CPUCLKB_RATE( 816000000, 1, 4, 4),
+ RK3399_CPUCLKB_RATE( 696000000, 1, 3, 3),
+ RK3399_CPUCLKB_RATE( 600000000, 1, 3, 3),
+ RK3399_CPUCLKB_RATE( 408000000, 1, 2, 2),
+ RK3399_CPUCLKB_RATE( 312000000, 1, 1, 1),
+ RK3399_CPUCLKB_RATE( 216000000, 1, 1, 1),
+ RK3399_CPUCLKB_RATE( 96000000, 1, 1, 1),
+};
+
+static struct rockchip_clk_branch rk3399_clk_branches[] __initdata = {
+ /*
+ * CRU Clock-Architecture
+ */
+
+ /* usbphy */
+ GATE(SCLK_USB2PHY0_REF, "clk_usb2phy0_ref", "xin24m", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(6), 5, GFLAGS),
+ GATE(SCLK_USB2PHY1_REF, "clk_usb2phy1_ref", "xin24m", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(6), 6, GFLAGS),
+
+ GATE(0, "clk_usbphy0_480m_src", "clk_usbphy0_480m", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(13), 12, GFLAGS),
+ GATE(0, "clk_usbphy1_480m_src", "clk_usbphy1_480m", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(13), 12, GFLAGS),
+ MUX(0, "clk_usbphy_480m", mux_usbphy_480m_p, CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(14), 6, 1, MFLAGS),
+
+ MUX(0, "upll", mux_pll_src_24m_usbphy480m_p, 0,
+ RK3399_CLKSEL_CON(14), 15, 1, MFLAGS),
+
+ COMPOSITE_NODIV(SCLK_HSICPHY, "clk_hsicphy", mux_pll_src_cpll_gpll_npll_usbphy480m_p, 0,
+ RK3399_CLKSEL_CON(19), 0, 2, MFLAGS,
+ RK3399_CLKGATE_CON(6), 4, GFLAGS),
+
+ COMPOSITE(ACLK_USB3, "aclk_usb3", mux_pll_src_cpll_gpll_npll_p, 0,
+ RK3399_CLKSEL_CON(39), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(12), 0, GFLAGS),
+ GATE(ACLK_USB3_NOC, "aclk_usb3_noc", "aclk_usb3", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(30), 0, GFLAGS),
+ GATE(ACLK_USB3OTG0, "aclk_usb3otg0", "aclk_usb3", 0,
+ RK3399_CLKGATE_CON(30), 1, GFLAGS),
+ GATE(ACLK_USB3OTG1, "aclk_usb3otg1", "aclk_usb3", 0,
+ RK3399_CLKGATE_CON(30), 2, GFLAGS),
+ GATE(ACLK_USB3_RKSOC_AXI_PERF, "aclk_usb3_rksoc_axi_perf", "aclk_usb3", 0,
+ RK3399_CLKGATE_CON(30), 3, GFLAGS),
+ GATE(ACLK_USB3_GRF, "aclk_usb3_grf", "aclk_usb3", 0,
+ RK3399_CLKGATE_CON(30), 4, GFLAGS),
+
+ GATE(SCLK_USB3OTG0_REF, "clk_usb3otg0_ref", "xin24m", 0,
+ RK3399_CLKGATE_CON(12), 1, GFLAGS),
+ GATE(SCLK_USB3OTG1_REF, "clk_usb3otg1_ref", "xin24m", 0,
+ RK3399_CLKGATE_CON(12), 2, GFLAGS),
+
+ COMPOSITE(SCLK_USB3OTG0_SUSPEND, "clk_usb3otg0_suspend", mux_pll_p, 0,
+ RK3399_CLKSEL_CON(40), 15, 1, MFLAGS, 0, 10, DFLAGS,
+ RK3399_CLKGATE_CON(12), 3, GFLAGS),
+
+ COMPOSITE(SCLK_USB3OTG1_SUSPEND, "clk_usb3otg1_suspend", mux_pll_p, 0,
+ RK3399_CLKSEL_CON(41), 15, 1, MFLAGS, 0, 10, DFLAGS,
+ RK3399_CLKGATE_CON(12), 4, GFLAGS),
+
+ COMPOSITE(SCLK_UPHY0_TCPDPHY_REF, "clk_uphy0_tcpdphy_ref", mux_pll_p, 0,
+ RK3399_CLKSEL_CON(64), 15, 1, MFLAGS, 8, 5, DFLAGS,
+ RK3399_CLKGATE_CON(13), 4, GFLAGS),
+
+ COMPOSITE(SCLK_UPHY0_TCPDCORE, "clk_uphy0_tcpdcore", mux_pll_src_24m_32k_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(64), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(13), 5, GFLAGS),
+
+ COMPOSITE(SCLK_UPHY1_TCPDPHY_REF, "clk_uphy1_tcpdphy_ref", mux_pll_p, 0,
+ RK3399_CLKSEL_CON(65), 15, 1, MFLAGS, 8, 5, DFLAGS,
+ RK3399_CLKGATE_CON(13), 6, GFLAGS),
+
+ COMPOSITE(SCLK_UPHY1_TCPDCORE, "clk_uphy1_tcpdcore", mux_pll_src_24m_32k_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(65), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(13), 7, GFLAGS),
+
+ /* little core */
+ GATE(0, "clk_core_l_lpll_src", "lpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(0), 0, GFLAGS),
+ GATE(0, "clk_core_l_bpll_src", "bpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(0), 1, GFLAGS),
+ GATE(0, "clk_core_l_dpll_src", "dpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(0), 2, GFLAGS),
+ GATE(0, "clk_core_l_gpll_src", "gpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(0), 3, GFLAGS),
+
+ COMPOSITE_NOMUX(0, "aclkm_core_l", "armclkl", CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(0), 8, 5, DFLAGS | CLK_DIVIDER_READ_ONLY,
+ RK3399_CLKGATE_CON(0), 4, GFLAGS),
+ COMPOSITE_NOMUX(0, "atclk_core_l", "armclkl", CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(1), 0, 5, DFLAGS | CLK_DIVIDER_READ_ONLY,
+ RK3399_CLKGATE_CON(0), 5, GFLAGS),
+ COMPOSITE_NOMUX(0, "pclk_dbg_core_l", "armclkl", CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(1), 8, 5, DFLAGS | CLK_DIVIDER_READ_ONLY,
+ RK3399_CLKGATE_CON(0), 6, GFLAGS),
+
+ GATE(ACLK_CORE_ADB400_CORE_L_2_CCI500, "aclk_core_adb400_core_l_2_cci500", "aclkm_core_l", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(14), 12, GFLAGS),
+ GATE(ACLK_PERF_CORE_L, "aclk_perf_core_l", "aclkm_core_l", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(14), 13, GFLAGS),
+
+ GATE(0, "clk_dbg_pd_core_l", "armclkl", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(14), 9, GFLAGS),
+ GATE(ACLK_GIC_ADB400_GIC_2_CORE_L, "aclk_core_adb400_gic_2_core_l", "armclkl", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(14), 10, GFLAGS),
+ GATE(ACLK_GIC_ADB400_CORE_L_2_GIC, "aclk_core_adb400_core_l_2_gic", "armclkl", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(14), 11, GFLAGS),
+ GATE(SCLK_PVTM_CORE_L, "clk_pvtm_core_l", "xin24m", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(0), 7, GFLAGS),
+
+ /* big core */
+ GATE(0, "clk_core_b_lpll_src", "lpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(1), 0, GFLAGS),
+ GATE(0, "clk_core_b_bpll_src", "bpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(1), 1, GFLAGS),
+ GATE(0, "clk_core_b_dpll_src", "dpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(1), 2, GFLAGS),
+ GATE(0, "clk_core_b_gpll_src", "gpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(1), 3, GFLAGS),
+
+ COMPOSITE_NOMUX(0, "aclkm_core_b", "armclkb", CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(2), 8, 5, DFLAGS | CLK_DIVIDER_READ_ONLY,
+ RK3399_CLKGATE_CON(1), 4, GFLAGS),
+ COMPOSITE_NOMUX(0, "atclk_core_b", "armclkb", CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(3), 0, 5, DFLAGS | CLK_DIVIDER_READ_ONLY,
+ RK3399_CLKGATE_CON(1), 5, GFLAGS),
+ COMPOSITE_NOMUX(0, "pclk_dbg_core_b", "armclkb", CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(3), 8, 5, DFLAGS | CLK_DIVIDER_READ_ONLY,
+ RK3399_CLKGATE_CON(1), 6, GFLAGS),
+
+ GATE(ACLK_CORE_ADB400_CORE_B_2_CCI500, "aclk_core_adb400_core_b_2_cci500", "aclkm_core_b", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(14), 5, GFLAGS),
+ GATE(ACLK_PERF_CORE_B, "aclk_perf_core_b", "aclkm_core_b", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(14), 6, GFLAGS),
+
+ GATE(0, "clk_dbg_pd_core_b", "armclkb", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(14), 1, GFLAGS),
+ GATE(ACLK_GIC_ADB400_GIC_2_CORE_B, "aclk_core_adb400_gic_2_core_b", "armclkb", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(14), 3, GFLAGS),
+ GATE(ACLK_GIC_ADB400_CORE_B_2_GIC, "aclk_core_adb400_core_b_2_gic", "armclkb", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(14), 4, GFLAGS),
+
+ DIV(0, "pclken_dbg_core_b", "pclk_dbg_core_b", CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(3), 13, 2, DFLAGS | CLK_DIVIDER_READ_ONLY),
+
+ GATE(0, "pclk_dbg_cxcs_pd_core_b", "pclk_dbg_core_b", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(14), 2, GFLAGS),
+
+ GATE(SCLK_PVTM_CORE_B, "clk_pvtm_core_b", "xin24m", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(1), 7, GFLAGS),
+
+ /* gmac */
+ GATE(0, "cpll_aclk_gmac_src", "cpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(6), 9, GFLAGS),
+ GATE(0, "gpll_aclk_gmac_src", "gpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(6), 8, GFLAGS),
+ COMPOSITE(0, "aclk_gmac_pre", mux_aclk_gmac_p, 0,
+ RK3399_CLKSEL_CON(20), 7, 1, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(6), 10, GFLAGS),
+
+ GATE(ACLK_GMAC, "aclk_gmac", "aclk_gmac_pre", 0,
+ RK3399_CLKGATE_CON(32), 0, GFLAGS),
+ GATE(ACLK_GMAC_NOC, "aclk_gmac_noc", "aclk_gmac_pre", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(32), 1, GFLAGS),
+ GATE(ACLK_PERF_GMAC, "aclk_perf_gmac", "aclk_gmac_pre", 0,
+ RK3399_CLKGATE_CON(32), 4, GFLAGS),
+
+ COMPOSITE_NOMUX(0, "pclk_gmac_pre", "aclk_gmac_pre", 0,
+ RK3399_CLKSEL_CON(19), 8, 3, DFLAGS,
+ RK3399_CLKGATE_CON(6), 11, GFLAGS),
+ GATE(PCLK_GMAC, "pclk_gmac", "pclk_gmac_pre", 0,
+ RK3399_CLKGATE_CON(32), 2, GFLAGS),
+ GATE(PCLK_GMAC_NOC, "pclk_gmac_noc", "pclk_gmac_pre", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(32), 3, GFLAGS),
+
+ COMPOSITE(SCLK_MAC, "clk_gmac", mux_pll_src_cpll_gpll_npll_p, 0,
+ RK3399_CLKSEL_CON(20), 14, 2, MFLAGS, 8, 5, DFLAGS,
+ RK3399_CLKGATE_CON(5), 5, GFLAGS),
+
+ MUX(SCLK_RMII_SRC, "clk_rmii_src", mux_rmii_p, CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(19), 4, 1, MFLAGS),
+ GATE(SCLK_MACREF_OUT, "clk_mac_refout", "clk_rmii_src", 0,
+ RK3399_CLKGATE_CON(5), 6, GFLAGS),
+ GATE(SCLK_MACREF, "clk_mac_ref", "clk_rmii_src", 0,
+ RK3399_CLKGATE_CON(5), 7, GFLAGS),
+ GATE(SCLK_MAC_RX, "clk_rmii_rx", "clk_rmii_src", 0,
+ RK3399_CLKGATE_CON(5), 8, GFLAGS),
+ GATE(SCLK_MAC_TX, "clk_rmii_tx", "clk_rmii_src", 0,
+ RK3399_CLKGATE_CON(5), 9, GFLAGS),
+
+ /* spdif */
+ COMPOSITE(0, "clk_spdif_div", mux_pll_src_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(32), 7, 1, MFLAGS, 0, 7, DFLAGS,
+ RK3399_CLKGATE_CON(8), 13, GFLAGS),
+ COMPOSITE_FRACMUX(0, "clk_spdif_frac", "clk_spdif_div", CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(99), 0,
+ RK3399_CLKGATE_CON(8), 14, GFLAGS,
+ &rk3399_spdif_fracmux),
+ GATE(SCLK_SPDIF_8CH, "clk_spdif", "clk_spdif_mux", CLK_SET_RATE_PARENT,
+ RK3399_CLKGATE_CON(8), 15, GFLAGS),
+
+ COMPOSITE(SCLK_SPDIF_REC_DPTX, "clk_spdif_rec_dptx", mux_pll_src_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(32), 15, 1, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(10), 6, GFLAGS),
+ /* i2s */
+ COMPOSITE(0, "clk_i2s0_div", mux_pll_src_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(28), 7, 1, MFLAGS, 0, 7, DFLAGS,
+ RK3399_CLKGATE_CON(8), 3, GFLAGS),
+ COMPOSITE_FRACMUX(0, "clk_i2s0_frac", "clk_i2s0_div", CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(96), 0,
+ RK3399_CLKGATE_CON(8), 4, GFLAGS,
+ &rk3399_i2s0_fracmux),
+ GATE(SCLK_I2S0_8CH, "clk_i2s0", "clk_i2s0_mux", CLK_SET_RATE_PARENT,
+ RK3399_CLKGATE_CON(8), 5, GFLAGS),
+
+ COMPOSITE(0, "clk_i2s1_div", mux_pll_src_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(29), 7, 1, MFLAGS, 0, 7, DFLAGS,
+ RK3399_CLKGATE_CON(8), 6, GFLAGS),
+ COMPOSITE_FRACMUX(0, "clk_i2s1_frac", "clk_i2s1_div", CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(97), 0,
+ RK3399_CLKGATE_CON(8), 7, GFLAGS,
+ &rk3399_i2s1_fracmux),
+ GATE(SCLK_I2S1_8CH, "clk_i2s1", "clk_i2s1_mux", CLK_SET_RATE_PARENT,
+ RK3399_CLKGATE_CON(8), 8, GFLAGS),
+
+ COMPOSITE(0, "clk_i2s2_div", mux_pll_src_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(30), 7, 1, MFLAGS, 0, 7, DFLAGS,
+ RK3399_CLKGATE_CON(8), 9, GFLAGS),
+ COMPOSITE_FRACMUX(0, "clk_i2s2_frac", "clk_i2s2_div", CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(98), 0,
+ RK3399_CLKGATE_CON(8), 10, GFLAGS,
+ &rk3399_i2s2_fracmux),
+ GATE(SCLK_I2S2_8CH, "clk_i2s2", "clk_i2s2_mux", CLK_SET_RATE_PARENT,
+ RK3399_CLKGATE_CON(8), 11, GFLAGS),
+
+ MUX(0, "clk_i2sout_src", mux_i2sch_p, CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(31), 0, 2, MFLAGS),
+ COMPOSITE_NODIV(SCLK_I2S_8CH_OUT, "clk_i2sout", mux_i2sout_p, CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(30), 8, 2, MFLAGS,
+ RK3399_CLKGATE_CON(8), 12, GFLAGS),
+
+ /* uart */
+ MUX(0, "clk_uart0_src", mux_pll_src_cpll_gpll_upll_p, 0,
+ RK3399_CLKSEL_CON(33), 12, 2, MFLAGS),
+ COMPOSITE_NOMUX(0, "clk_uart0_div", "clk_uart0_src", 0,
+ RK3399_CLKSEL_CON(33), 0, 7, DFLAGS,
+ RK3399_CLKGATE_CON(9), 0, GFLAGS),
+ COMPOSITE_FRACMUX(0, "clk_uart0_frac", "clk_uart0_div", CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(100), 0,
+ RK3399_CLKGATE_CON(9), 1, GFLAGS,
+ &rk3399_uart0_fracmux),
+
+ MUX(0, "clk_uart_src", mux_pll_src_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(33), 15, 1, MFLAGS),
+ COMPOSITE_NOMUX(0, "clk_uart1_div", "clk_uart_src", 0,
+ RK3399_CLKSEL_CON(34), 0, 7, DFLAGS,
+ RK3399_CLKGATE_CON(9), 2, GFLAGS),
+ COMPOSITE_FRACMUX(0, "clk_uart1_frac", "clk_uart1_div", CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(101), 0,
+ RK3399_CLKGATE_CON(9), 3, GFLAGS,
+ &rk3399_uart1_fracmux),
+
+ COMPOSITE_NOMUX(0, "clk_uart2_div", "clk_uart_src", 0,
+ RK3399_CLKSEL_CON(35), 0, 7, DFLAGS,
+ RK3399_CLKGATE_CON(9), 4, GFLAGS),
+ COMPOSITE_FRACMUX(0, "clk_uart2_frac", "clk_uart2_div", CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(102), 0,
+ RK3399_CLKGATE_CON(9), 5, GFLAGS,
+ &rk3399_uart2_fracmux),
+
+ COMPOSITE_NOMUX(0, "clk_uart3_div", "clk_uart_src", 0,
+ RK3399_CLKSEL_CON(36), 0, 7, DFLAGS,
+ RK3399_CLKGATE_CON(9), 6, GFLAGS),
+ COMPOSITE_FRACMUX(0, "clk_uart3_frac", "clk_uart3_div", CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(103), 0,
+ RK3399_CLKGATE_CON(9), 7, GFLAGS,
+ &rk3399_uart3_fracmux),
+
+ COMPOSITE(0, "pclk_ddr", mux_pll_src_cpll_gpll_p, CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(6), 15, 1, MFLAGS, 8, 5, DFLAGS,
+ RK3399_CLKGATE_CON(3), 4, GFLAGS),
+
+ GATE(PCLK_CENTER_MAIN_NOC, "pclk_center_main_noc", "pclk_ddr", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(18), 10, GFLAGS),
+ GATE(PCLK_DDR_MON, "pclk_ddr_mon", "pclk_ddr", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(18), 12, GFLAGS),
+ GATE(PCLK_CIC, "pclk_cic", "pclk_ddr", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(18), 15, GFLAGS),
+ GATE(PCLK_DDR_SGRF, "pclk_ddr_sgrf", "pclk_ddr", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(19), 2, GFLAGS),
+
+ GATE(SCLK_PVTM_DDR, "clk_pvtm_ddr", "xin24m", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(4), 11, GFLAGS),
+ GATE(SCLK_DFIMON0_TIMER, "clk_dfimon0_timer", "xin24m", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(3), 5, GFLAGS),
+ GATE(SCLK_DFIMON1_TIMER, "clk_dfimon1_timer", "xin24m", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(3), 6, GFLAGS),
+
+ /* cci */
+ GATE(0, "cpll_aclk_cci_src", "cpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(2), 0, GFLAGS),
+ GATE(0, "gpll_aclk_cci_src", "gpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(2), 1, GFLAGS),
+ GATE(0, "npll_aclk_cci_src", "npll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(2), 2, GFLAGS),
+ GATE(0, "vpll_aclk_cci_src", "vpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(2), 3, GFLAGS),
+
+ COMPOSITE(0, "aclk_cci_pre", mux_aclk_cci_p, CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(5), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(2), 4, GFLAGS),
+
+ GATE(ACLK_ADB400M_PD_CORE_L, "aclk_adb400m_pd_core_l", "aclk_cci_pre", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(15), 0, GFLAGS),
+ GATE(ACLK_ADB400M_PD_CORE_B, "aclk_adb400m_pd_core_b", "aclk_cci_pre", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(15), 1, GFLAGS),
+ GATE(ACLK_CCI, "aclk_cci", "aclk_cci_pre", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(15), 2, GFLAGS),
+ GATE(ACLK_CCI_NOC0, "aclk_cci_noc0", "aclk_cci_pre", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(15), 3, GFLAGS),
+ GATE(ACLK_CCI_NOC1, "aclk_cci_noc1", "aclk_cci_pre", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(15), 4, GFLAGS),
+ GATE(ACLK_CCI_GRF, "aclk_cci_grf", "aclk_cci_pre", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(15), 7, GFLAGS),
+
+ GATE(0, "cpll_cci_trace", "cpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(2), 5, GFLAGS),
+ GATE(0, "gpll_cci_trace", "gpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(2), 6, GFLAGS),
+ COMPOSITE(SCLK_CCI_TRACE, "clk_cci_trace", mux_cci_trace_p, CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(5), 15, 2, MFLAGS, 8, 5, DFLAGS,
+ RK3399_CLKGATE_CON(2), 7, GFLAGS),
+
+ GATE(0, "cpll_cs", "cpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(2), 8, GFLAGS),
+ GATE(0, "gpll_cs", "gpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(2), 9, GFLAGS),
+ GATE(0, "npll_cs", "npll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(2), 10, GFLAGS),
+ COMPOSITE_NOGATE(0, "clk_cs", mux_cs_p, CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(4), 6, 2, MFLAGS, 0, 5, DFLAGS),
+ GATE(0, "clk_dbg_cxcs", "clk_cs", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(15), 5, GFLAGS),
+ GATE(0, "clk_dbg_noc", "clk_cs", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(15), 6, GFLAGS),
+
+ /* vcodec */
+ COMPOSITE(0, "aclk_vcodec_pre", mux_pll_src_cpll_gpll_npll_ppll_p, 0,
+ RK3399_CLKSEL_CON(7), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(4), 0, GFLAGS),
+ COMPOSITE_NOMUX(0, "hclk_vcodec_pre", "aclk_vcodec_pre", 0,
+ RK3399_CLKSEL_CON(7), 8, 5, DFLAGS,
+ RK3399_CLKGATE_CON(4), 1, GFLAGS),
+ GATE(HCLK_VCODEC, "hclk_vcodec", "hclk_vcodec_pre", 0,
+ RK3399_CLKGATE_CON(17), 2, GFLAGS),
+ GATE(0, "hclk_vcodec_noc", "hclk_vcodec_pre", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(17), 3, GFLAGS),
+
+ GATE(ACLK_VCODEC, "aclk_vcodec", "aclk_vcodec_pre", 0,
+ RK3399_CLKGATE_CON(17), 0, GFLAGS),
+ GATE(0, "aclk_vcodec_noc", "aclk_vcodec_pre", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(17), 1, GFLAGS),
+
+ /* vdu */
+ COMPOSITE(SCLK_VDU_CORE, "clk_vdu_core", mux_pll_src_cpll_gpll_npll_p, 0,
+ RK3399_CLKSEL_CON(9), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(4), 4, GFLAGS),
+ COMPOSITE(SCLK_VDU_CA, "clk_vdu_ca", mux_pll_src_cpll_gpll_npll_p, 0,
+ RK3399_CLKSEL_CON(9), 14, 2, MFLAGS, 8, 5, DFLAGS,
+ RK3399_CLKGATE_CON(4), 5, GFLAGS),
+
+ COMPOSITE(0, "aclk_vdu_pre", mux_pll_src_cpll_gpll_npll_ppll_p, 0,
+ RK3399_CLKSEL_CON(8), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(4), 2, GFLAGS),
+ COMPOSITE_NOMUX(0, "hclk_vdu_pre", "aclk_vdu_pre", 0,
+ RK3399_CLKSEL_CON(8), 8, 5, DFLAGS,
+ RK3399_CLKGATE_CON(4), 3, GFLAGS),
+ GATE(HCLK_VDU, "hclk_vdu", "hclk_vdu_pre", 0,
+ RK3399_CLKGATE_CON(17), 10, GFLAGS),
+ GATE(HCLK_VDU_NOC, "hclk_vdu_noc", "hclk_vdu_pre", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(17), 11, GFLAGS),
+
+ GATE(ACLK_VDU, "aclk_vdu", "aclk_vdu_pre", 0,
+ RK3399_CLKGATE_CON(17), 8, GFLAGS),
+ GATE(ACLK_VDU_NOC, "aclk_vdu_noc", "aclk_vdu_pre", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(17), 9, GFLAGS),
+
+ /* iep */
+ COMPOSITE(0, "aclk_iep_pre", mux_pll_src_cpll_gpll_npll_ppll_p, 0,
+ RK3399_CLKSEL_CON(10), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(4), 6, GFLAGS),
+ COMPOSITE_NOMUX(0, "hclk_iep_pre", "aclk_iep_pre", 0,
+ RK3399_CLKSEL_CON(10), 8, 5, DFLAGS,
+ RK3399_CLKGATE_CON(4), 7, GFLAGS),
+ GATE(HCLK_IEP, "hclk_iep", "hclk_iep_pre", 0,
+ RK3399_CLKGATE_CON(16), 2, GFLAGS),
+ GATE(HCLK_IEP_NOC, "hclk_iep_noc", "hclk_iep_pre", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(16), 3, GFLAGS),
+
+ GATE(ACLK_IEP, "aclk_iep", "aclk_iep_pre", 0,
+ RK3399_CLKGATE_CON(16), 0, GFLAGS),
+ GATE(ACLK_IEP_NOC, "aclk_iep_noc", "aclk_iep_pre", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(16), 1, GFLAGS),
+
+ /* rga */
+ COMPOSITE(SCLK_RGA_CORE, "clk_rga_core", mux_pll_src_cpll_gpll_npll_ppll_p, 0,
+ RK3399_CLKSEL_CON(12), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(4), 10, GFLAGS),
+
+ COMPOSITE(0, "aclk_rga_pre", mux_pll_src_cpll_gpll_npll_ppll_p, 0,
+ RK3399_CLKSEL_CON(11), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(4), 8, GFLAGS),
+ COMPOSITE_NOMUX(0, "hclk_rga_pre", "aclk_rga_pre", 0,
+ RK3399_CLKSEL_CON(11), 8, 5, DFLAGS,
+ RK3399_CLKGATE_CON(4), 9, GFLAGS),
+ GATE(HCLK_RGA, "hclk_rga", "hclk_rga_pre", 0,
+ RK3399_CLKGATE_CON(16), 10, GFLAGS),
+ GATE(HCLK_RGA_NOC, "hclk_rga_noc", "hclk_rga_pre", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(16), 11, GFLAGS),
+
+ GATE(ACLK_RGA, "aclk_rga", "aclk_rga_pre", 0,
+ RK3399_CLKGATE_CON(16), 8, GFLAGS),
+ GATE(ACLK_RGA_NOC, "aclk_rga_noc", "aclk_rga_pre", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(16), 9, GFLAGS),
+
+ /* center */
+ COMPOSITE(0, "aclk_center", mux_pll_src_cpll_gpll_npll_p, CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(12), 14, 2, MFLAGS, 8, 5, DFLAGS,
+ RK3399_CLKGATE_CON(3), 7, GFLAGS),
+ GATE(ACLK_CENTER_MAIN_NOC, "aclk_center_main_noc", "aclk_center", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(19), 0, GFLAGS),
+ GATE(ACLK_CENTER_PERI_NOC, "aclk_center_peri_noc", "aclk_center", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(19), 1, GFLAGS),
+
+ /* gpu */
+ COMPOSITE(0, "aclk_gpu_pre", mux_pll_src_ppll_cpll_gpll_npll_p, CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(13), 5, 3, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(13), 0, GFLAGS),
+ GATE(ACLK_GPU, "aclk_gpu", "aclk_gpu_pre", 0,
+ RK3399_CLKGATE_CON(30), 8, GFLAGS),
+ GATE(ACLK_PERF_GPU, "aclk_perf_gpu", "aclk_gpu_pre", 0,
+ RK3399_CLKGATE_CON(30), 10, GFLAGS),
+ GATE(ACLK_GPU_GRF, "aclk_gpu_grf", "aclk_gpu_pre", 0,
+ RK3399_CLKGATE_CON(30), 11, GFLAGS),
+ GATE(SCLK_PVTM_GPU, "aclk_pvtm_gpu", "xin24m", 0,
+ RK3399_CLKGATE_CON(13), 1, GFLAGS),
+
+ /* perihp */
+ GATE(0, "cpll_aclk_perihp_src", "gpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(5), 0, GFLAGS),
+ GATE(0, "gpll_aclk_perihp_src", "cpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(5), 1, GFLAGS),
+ COMPOSITE(ACLK_PERIHP, "aclk_perihp", mux_aclk_perihp_p, CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(14), 7, 1, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(5), 2, GFLAGS),
+ COMPOSITE_NOMUX(HCLK_PERIHP, "hclk_perihp", "aclk_perihp", CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(14), 8, 2, DFLAGS,
+ RK3399_CLKGATE_CON(5), 3, GFLAGS),
+ COMPOSITE_NOMUX(PCLK_PERIHP, "pclk_perihp", "aclk_perihp", CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(14), 12, 2, DFLAGS,
+ RK3399_CLKGATE_CON(5), 4, GFLAGS),
+
+ GATE(ACLK_PERF_PCIE, "aclk_perf_pcie", "aclk_perihp", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(20), 2, GFLAGS),
+ GATE(ACLK_PCIE, "aclk_pcie", "aclk_perihp", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(20), 10, GFLAGS),
+ GATE(0, "aclk_perihp_noc", "aclk_perihp", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(20), 12, GFLAGS),
+
+ GATE(HCLK_HOST0, "hclk_host0", "hclk_perihp", 0,
+ RK3399_CLKGATE_CON(20), 5, GFLAGS),
+ GATE(HCLK_HOST0_ARB, "hclk_host0_arb", "hclk_perihp", 0,
+ RK3399_CLKGATE_CON(20), 6, GFLAGS),
+ GATE(HCLK_HOST1, "hclk_host1", "hclk_perihp", 0,
+ RK3399_CLKGATE_CON(20), 7, GFLAGS),
+ GATE(HCLK_HOST1_ARB, "hclk_host1_arb", "hclk_perihp", 0,
+ RK3399_CLKGATE_CON(20), 8, GFLAGS),
+ GATE(HCLK_HSIC, "hclk_hsic", "hclk_perihp", 0,
+ RK3399_CLKGATE_CON(20), 9, GFLAGS),
+ GATE(0, "hclk_perihp_noc", "hclk_perihp", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(20), 13, GFLAGS),
+ GATE(0, "hclk_ahb1tom", "hclk_perihp", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(20), 15, GFLAGS),
+
+ GATE(PCLK_PERIHP_GRF, "pclk_perihp_grf", "pclk_perihp", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(20), 4, GFLAGS),
+ GATE(PCLK_PCIE, "pclk_pcie", "pclk_perihp", 0,
+ RK3399_CLKGATE_CON(20), 11, GFLAGS),
+ GATE(0, "pclk_perihp_noc", "pclk_perihp", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(20), 14, GFLAGS),
+ GATE(PCLK_HSICPHY, "pclk_hsicphy", "pclk_perihp", 0,
+ RK3399_CLKGATE_CON(31), 8, GFLAGS),
+
+ /* sdio & sdmmc */
+ COMPOSITE(0, "hclk_sd", mux_pll_src_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(13), 15, 1, MFLAGS, 8, 5, DFLAGS,
+ RK3399_CLKGATE_CON(12), 13, GFLAGS),
+ GATE(HCLK_SDMMC, "hclk_sdmmc", "hclk_sd", 0,
+ RK3399_CLKGATE_CON(33), 8, GFLAGS),
+ GATE(0, "hclk_sdmmc_noc", "hclk_sd", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(33), 9, GFLAGS),
+
+ COMPOSITE(SCLK_SDIO, "clk_sdio", mux_pll_src_cpll_gpll_npll_ppll_upll_24m_p, 0,
+ RK3399_CLKSEL_CON(15), 8, 3, MFLAGS, 0, 7, DFLAGS,
+ RK3399_CLKGATE_CON(6), 0, GFLAGS),
+
+ COMPOSITE(SCLK_SDMMC, "clk_sdmmc", mux_pll_src_cpll_gpll_npll_ppll_upll_24m_p, 0,
+ RK3399_CLKSEL_CON(16), 8, 3, MFLAGS, 0, 7, DFLAGS,
+ RK3399_CLKGATE_CON(6), 1, GFLAGS),
+
+ MMC(SCLK_SDMMC_DRV, "sdmmc_drv", "clk_sdmmc", RK3399_SDMMC_CON0, 1),
+ MMC(SCLK_SDMMC_SAMPLE, "sdmmc_sample", "clk_sdmmc", RK3399_SDMMC_CON1, 1),
+
+ MMC(SCLK_SDIO_DRV, "sdio_drv", "clk_sdio", RK3399_SDIO_CON0, 1),
+ MMC(SCLK_SDIO_SAMPLE, "sdio_sample", "clk_sdio", RK3399_SDIO_CON1, 1),
+
+ /* pcie */
+ COMPOSITE(SCLK_PCIE_PM, "clk_pcie_pm", mux_pll_src_cpll_gpll_npll_24m_p, 0,
+ RK3399_CLKSEL_CON(17), 8, 3, MFLAGS, 0, 7, DFLAGS,
+ RK3399_CLKGATE_CON(6), 2, GFLAGS),
+
+ COMPOSITE_NOMUX(SCLK_PCIEPHY_REF100M, "clk_pciephy_ref100m", "npll", 0,
+ RK3399_CLKSEL_CON(18), 11, 5, DFLAGS,
+ RK3399_CLKGATE_CON(12), 6, GFLAGS),
+ MUX(SCLK_PCIEPHY_REF, "clk_pciephy_ref", mux_pll_src_24m_pciephy_p, CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(18), 10, 1, MFLAGS),
+
+ COMPOSITE(0, "clk_pcie_core_cru", mux_pll_src_cpll_gpll_npll_p, 0,
+ RK3399_CLKSEL_CON(18), 8, 2, MFLAGS, 0, 7, DFLAGS,
+ RK3399_CLKGATE_CON(6), 3, GFLAGS),
+ MUX(SCLK_PCIE_CORE, "clk_pcie_core", mux_pciecore_cru_phy_p, CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(18), 7, 1, MFLAGS),
+
+ /* emmc */
+ COMPOSITE(SCLK_EMMC, "clk_emmc", mux_pll_src_cpll_gpll_npll_upll_24m_p, 0,
+ RK3399_CLKSEL_CON(22), 8, 3, MFLAGS, 0, 7, DFLAGS,
+ RK3399_CLKGATE_CON(6), 14, GFLAGS),
+
+ GATE(0, "cpll_aclk_emmc_src", "cpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(6), 12, GFLAGS),
+ GATE(0, "gpll_aclk_emmc_src", "gpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(6), 13, GFLAGS),
+ COMPOSITE_NOGATE(ACLK_EMMC, "aclk_emmc", mux_aclk_emmc_p, CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(21), 7, 1, MFLAGS, 0, 5, DFLAGS),
+ GATE(ACLK_EMMC_CORE, "aclk_emmccore", "aclk_emmc", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(32), 8, GFLAGS),
+ GATE(ACLK_EMMC_NOC, "aclk_emmc_noc", "aclk_emmc", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(32), 9, GFLAGS),
+ GATE(ACLK_EMMC_GRF, "aclk_emmcgrf", "aclk_emmc", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(32), 10, GFLAGS),
+
+ /* perilp0 */
+ GATE(0, "cpll_aclk_perilp0_src", "cpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(7), 1, GFLAGS),
+ GATE(0, "gpll_aclk_perilp0_src", "gpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(7), 0, GFLAGS),
+ COMPOSITE(ACLK_PERILP0, "aclk_perilp0", mux_aclk_perilp0_p, CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(23), 7, 1, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(7), 2, GFLAGS),
+ COMPOSITE_NOMUX(HCLK_PERILP0, "hclk_perilp0", "aclk_perilp0", CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(23), 8, 2, DFLAGS,
+ RK3399_CLKGATE_CON(7), 3, GFLAGS),
+ COMPOSITE_NOMUX(PCLK_PERILP0, "pclk_perilp0", "aclk_perilp0", 0,
+ RK3399_CLKSEL_CON(23), 12, 3, DFLAGS,
+ RK3399_CLKGATE_CON(7), 4, GFLAGS),
+
+ /* aclk_perilp0 gates */
+ GATE(ACLK_INTMEM, "aclk_intmem", "aclk_perilp0", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(23), 0, GFLAGS),
+ GATE(ACLK_TZMA, "aclk_tzma", "aclk_perilp0", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(23), 1, GFLAGS),
+ GATE(SCLK_INTMEM0, "clk_intmem0", "aclk_perilp0", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(23), 2, GFLAGS),
+ GATE(SCLK_INTMEM1, "clk_intmem1", "aclk_perilp0", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(23), 3, GFLAGS),
+ GATE(SCLK_INTMEM2, "clk_intmem2", "aclk_perilp0", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(23), 4, GFLAGS),
+ GATE(SCLK_INTMEM3, "clk_intmem3", "aclk_perilp0", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(23), 5, GFLAGS),
+ GATE(SCLK_INTMEM4, "clk_intmem4", "aclk_perilp0", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(23), 6, GFLAGS),
+ GATE(SCLK_INTMEM5, "clk_intmem5", "aclk_perilp0", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(23), 7, GFLAGS),
+ GATE(ACLK_DCF, "aclk_dcf", "aclk_perilp0", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(23), 8, GFLAGS),
+ GATE(ACLK_DMAC0_PERILP, "aclk_dmac0_perilp", "aclk_perilp0", 0, RK3399_CLKGATE_CON(25), 5, GFLAGS),
+ GATE(ACLK_DMAC1_PERILP, "aclk_dmac1_perilp", "aclk_perilp0", 0, RK3399_CLKGATE_CON(25), 6, GFLAGS),
+ GATE(ACLK_PERILP0_NOC, "aclk_perilp0_noc", "aclk_perilp0", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(25), 7, GFLAGS),
+
+ /* hclk_perilp0 gates */
+ GATE(HCLK_ROM, "hclk_rom", "hclk_perilp0", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(24), 4, GFLAGS),
+ GATE(HCLK_M_CRYPTO0, "hclk_m_crypto0", "hclk_perilp0", 0, RK3399_CLKGATE_CON(24), 5, GFLAGS),
+ GATE(HCLK_S_CRYPTO0, "hclk_s_crypto0", "hclk_perilp0", 0, RK3399_CLKGATE_CON(24), 6, GFLAGS),
+ GATE(HCLK_M_CRYPTO1, "hclk_m_crypto1", "hclk_perilp0", 0, RK3399_CLKGATE_CON(24), 14, GFLAGS),
+ GATE(HCLK_S_CRYPTO1, "hclk_s_crypto1", "hclk_perilp0", 0, RK3399_CLKGATE_CON(24), 15, GFLAGS),
+ GATE(HCLK_PERILP0_NOC, "hclk_perilp0_noc", "hclk_perilp0", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(25), 8, GFLAGS),
+
+ /* pclk_perilp0 gates */
+ GATE(PCLK_DCF, "pclk_dcf", "pclk_perilp0", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(23), 9, GFLAGS),
+
+ /* crypto */
+ COMPOSITE(SCLK_CRYPTO0, "clk_crypto0", mux_pll_src_cpll_gpll_ppll_p, 0,
+ RK3399_CLKSEL_CON(24), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(7), 7, GFLAGS),
+
+ COMPOSITE(SCLK_CRYPTO1, "clk_crypto1", mux_pll_src_cpll_gpll_ppll_p, 0,
+ RK3399_CLKSEL_CON(26), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(7), 8, GFLAGS),
+
+ /* cm0s_perilp */
+ GATE(0, "cpll_fclk_cm0s_src", "cpll", 0,
+ RK3399_CLKGATE_CON(7), 6, GFLAGS),
+ GATE(0, "gpll_fclk_cm0s_src", "gpll", 0,
+ RK3399_CLKGATE_CON(7), 5, GFLAGS),
+ COMPOSITE(FCLK_CM0S, "fclk_cm0s", mux_fclk_cm0s_p, 0,
+ RK3399_CLKSEL_CON(24), 15, 1, MFLAGS, 8, 5, DFLAGS,
+ RK3399_CLKGATE_CON(7), 9, GFLAGS),
+
+ /* fclk_cm0s gates */
+ GATE(SCLK_M0_PERILP, "sclk_m0_perilp", "fclk_cm0s", 0, RK3399_CLKGATE_CON(24), 8, GFLAGS),
+ GATE(HCLK_M0_PERILP, "hclk_m0_perilp", "fclk_cm0s", 0, RK3399_CLKGATE_CON(24), 9, GFLAGS),
+ GATE(DCLK_M0_PERILP, "dclk_m0_perilp", "fclk_cm0s", 0, RK3399_CLKGATE_CON(24), 10, GFLAGS),
+ GATE(SCLK_M0_PERILP_DEC, "clk_m0_perilp_dec", "fclk_cm0s", 0, RK3399_CLKGATE_CON(24), 11, GFLAGS),
+ GATE(HCLK_M0_PERILP_NOC, "hclk_m0_perilp_noc", "fclk_cm0s", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(25), 11, GFLAGS),
+
+ /* perilp1 */
+ GATE(0, "cpll_hclk_perilp1_src", "cpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(8), 1, GFLAGS),
+ GATE(0, "gpll_hclk_perilp1_src", "gpll", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(8), 0, GFLAGS),
+ COMPOSITE_NOGATE(HCLK_PERILP1, "hclk_perilp1", mux_hclk_perilp1_p, CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(25), 7, 1, MFLAGS, 0, 5, DFLAGS),
+ COMPOSITE_NOMUX(PCLK_PERILP1, "pclk_perilp1", "hclk_perilp1", CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(25), 8, 3, DFLAGS,
+ RK3399_CLKGATE_CON(8), 2, GFLAGS),
+
+ /* hclk_perilp1 gates */
+ GATE(0, "hclk_perilp1_noc", "hclk_perilp1", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(25), 9, GFLAGS),
+ GATE(0, "hclk_sdio_noc", "hclk_perilp1", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(25), 12, GFLAGS),
+ GATE(HCLK_I2S0_8CH, "hclk_i2s0", "hclk_perilp1", 0, RK3399_CLKGATE_CON(34), 0, GFLAGS),
+ GATE(HCLK_I2S1_8CH, "hclk_i2s1", "hclk_perilp1", 0, RK3399_CLKGATE_CON(34), 1, GFLAGS),
+ GATE(HCLK_I2S2_8CH, "hclk_i2s2", "hclk_perilp1", 0, RK3399_CLKGATE_CON(34), 2, GFLAGS),
+ GATE(HCLK_SPDIF, "hclk_spdif", "hclk_perilp1", 0, RK3399_CLKGATE_CON(34), 3, GFLAGS),
+ GATE(HCLK_SDIO, "hclk_sdio", "hclk_perilp1", 0, RK3399_CLKGATE_CON(34), 4, GFLAGS),
+ GATE(PCLK_SPI5, "pclk_spi5", "hclk_perilp1", 0, RK3399_CLKGATE_CON(34), 5, GFLAGS),
+ GATE(0, "hclk_sdioaudio_noc", "hclk_perilp1", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(34), 6, GFLAGS),
+
+ /* pclk_perilp1 gates */
+ GATE(PCLK_UART0, "pclk_uart0", "pclk_perilp1", 0, RK3399_CLKGATE_CON(22), 0, GFLAGS),
+ GATE(PCLK_UART1, "pclk_uart1", "pclk_perilp1", 0, RK3399_CLKGATE_CON(22), 1, GFLAGS),
+ GATE(PCLK_UART2, "pclk_uart2", "pclk_perilp1", 0, RK3399_CLKGATE_CON(22), 2, GFLAGS),
+ GATE(PCLK_UART3, "pclk_uart3", "pclk_perilp1", 0, RK3399_CLKGATE_CON(22), 3, GFLAGS),
+ GATE(PCLK_I2C7, "pclk_rki2c7", "pclk_perilp1", 0, RK3399_CLKGATE_CON(22), 5, GFLAGS),
+ GATE(PCLK_I2C1, "pclk_rki2c1", "pclk_perilp1", 0, RK3399_CLKGATE_CON(22), 6, GFLAGS),
+ GATE(PCLK_I2C5, "pclk_rki2c5", "pclk_perilp1", 0, RK3399_CLKGATE_CON(22), 7, GFLAGS),
+ GATE(PCLK_I2C6, "pclk_rki2c6", "pclk_perilp1", 0, RK3399_CLKGATE_CON(22), 8, GFLAGS),
+ GATE(PCLK_I2C2, "pclk_rki2c2", "pclk_perilp1", 0, RK3399_CLKGATE_CON(22), 9, GFLAGS),
+ GATE(PCLK_I2C3, "pclk_rki2c3", "pclk_perilp1", 0, RK3399_CLKGATE_CON(22), 10, GFLAGS),
+ GATE(PCLK_MAILBOX0, "pclk_mailbox0", "pclk_perilp1", 0, RK3399_CLKGATE_CON(22), 11, GFLAGS),
+ GATE(PCLK_SARADC, "pclk_saradc", "pclk_perilp1", 0, RK3399_CLKGATE_CON(22), 12, GFLAGS),
+ GATE(PCLK_TSADC, "pclk_tsadc", "pclk_perilp1", 0, RK3399_CLKGATE_CON(22), 13, GFLAGS),
+ GATE(PCLK_EFUSE1024NS, "pclk_efuse1024ns", "pclk_perilp1", 0, RK3399_CLKGATE_CON(22), 14, GFLAGS),
+ GATE(PCLK_EFUSE1024S, "pclk_efuse1024s", "pclk_perilp1", 0, RK3399_CLKGATE_CON(22), 15, GFLAGS),
+ GATE(PCLK_SPI0, "pclk_spi0", "pclk_perilp1", 0, RK3399_CLKGATE_CON(23), 10, GFLAGS),
+ GATE(PCLK_SPI1, "pclk_spi1", "pclk_perilp1", 0, RK3399_CLKGATE_CON(23), 11, GFLAGS),
+ GATE(PCLK_SPI2, "pclk_spi2", "pclk_perilp1", 0, RK3399_CLKGATE_CON(23), 12, GFLAGS),
+ GATE(PCLK_SPI4, "pclk_spi4", "pclk_perilp1", 0, RK3399_CLKGATE_CON(23), 13, GFLAGS),
+ GATE(PCLK_PERIHP_GRF, "pclk_perilp_sgrf", "pclk_perilp1", 0, RK3399_CLKGATE_CON(24), 13, GFLAGS),
+ GATE(0, "pclk_perilp1_noc", "pclk_perilp1", 0, RK3399_CLKGATE_CON(25), 10, GFLAGS),
+
+ /* saradc */
+ COMPOSITE_NOMUX(SCLK_SARADC, "clk_saradc", "xin24m", 0,
+ RK3399_CLKSEL_CON(26), 8, 8, DFLAGS,
+ RK3399_CLKGATE_CON(9), 11, GFLAGS),
+
+ /* tsadc */
+ COMPOSITE(SCLK_TSADC, "clk_tsadc", mux_pll_p, 0,
+ RK3399_CLKSEL_CON(27), 15, 1, MFLAGS, 0, 10, DFLAGS,
+ RK3399_CLKGATE_CON(9), 10, GFLAGS),
+
+ /* cif_testout */
+ MUX(0, "clk_testout1_pll_src", mux_pll_src_cpll_gpll_npll_p, 0,
+ RK3399_CLKSEL_CON(38), 6, 2, MFLAGS),
+ COMPOSITE(0, "clk_testout1", mux_clk_testout1_p, 0,
+ RK3399_CLKSEL_CON(38), 5, 1, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(13), 14, GFLAGS),
+
+ MUX(0, "clk_testout2_pll_src", mux_pll_src_cpll_gpll_npll_p, 0,
+ RK3399_CLKSEL_CON(38), 14, 2, MFLAGS),
+ COMPOSITE(0, "clk_testout2", mux_clk_testout2_p, 0,
+ RK3399_CLKSEL_CON(38), 13, 1, MFLAGS, 8, 5, DFLAGS,
+ RK3399_CLKGATE_CON(13), 15, GFLAGS),
+
+ /* vio */
+ COMPOSITE(ACLK_VIO, "aclk_vio", mux_pll_src_cpll_gpll_ppll_p, CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(42), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(11), 10, GFLAGS),
+ COMPOSITE_NOMUX(PCLK_VIO, "pclk_vio", "aclk_vio", 0,
+ RK3399_CLKSEL_CON(43), 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(11), 1, GFLAGS),
+
+ GATE(ACLK_VIO_NOC, "aclk_vio_noc", "aclk_vio", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(29), 0, GFLAGS),
+
+ GATE(PCLK_MIPI_DSI0, "pclk_mipi_dsi0", "pclk_vio", 0,
+ RK3399_CLKGATE_CON(29), 1, GFLAGS),
+ GATE(PCLK_MIPI_DSI1, "pclk_mipi_dsi1", "pclk_vio", 0,
+ RK3399_CLKGATE_CON(29), 2, GFLAGS),
+ GATE(PCLK_VIO_GRF, "pclk_vio_grf", "pclk_vio", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(29), 12, GFLAGS),
+
+ /* hdcp */
+ COMPOSITE(ACLK_HDCP, "aclk_hdcp", mux_pll_src_cpll_gpll_ppll_p, 0,
+ RK3399_CLKSEL_CON(42), 14, 2, MFLAGS, 8, 5, DFLAGS,
+ RK3399_CLKGATE_CON(11), 12, GFLAGS),
+ COMPOSITE_NOMUX(HCLK_HDCP, "hclk_hdcp", "aclk_hdcp", 0,
+ RK3399_CLKSEL_CON(43), 5, 5, DFLAGS,
+ RK3399_CLKGATE_CON(11), 3, GFLAGS),
+ COMPOSITE_NOMUX(PCLK_HDCP, "pclk_hdcp", "aclk_hdcp", 0,
+ RK3399_CLKSEL_CON(43), 10, 5, DFLAGS,
+ RK3399_CLKGATE_CON(11), 10, GFLAGS),
+
+ GATE(ACLK_HDCP_NOC, "aclk_hdcp_noc", "aclk_hdcp", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(29), 4, GFLAGS),
+ GATE(ACLK_HDCP22, "aclk_hdcp22", "aclk_hdcp", 0,
+ RK3399_CLKGATE_CON(29), 10, GFLAGS),
+
+ GATE(HCLK_HDCP_NOC, "hclk_hdcp_noc", "hclk_hdcp", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(29), 5, GFLAGS),
+ GATE(HCLK_HDCP22, "hclk_hdcp22", "hclk_hdcp", 0,
+ RK3399_CLKGATE_CON(29), 9, GFLAGS),
+
+ GATE(PCLK_HDCP_NOC, "pclk_hdcp_noc", "pclk_hdcp", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(29), 3, GFLAGS),
+ GATE(PCLK_HDMI_CTRL, "pclk_hdmi_ctrl", "pclk_hdcp", 0,
+ RK3399_CLKGATE_CON(29), 6, GFLAGS),
+ GATE(PCLK_DP_CTRL, "pclk_dp_ctrl", "pclk_hdcp", 0,
+ RK3399_CLKGATE_CON(29), 7, GFLAGS),
+ GATE(PCLK_HDCP22, "pclk_hdcp22", "pclk_hdcp", 0,
+ RK3399_CLKGATE_CON(29), 8, GFLAGS),
+ GATE(PCLK_GASKET, "pclk_gasket", "pclk_hdcp", 0,
+ RK3399_CLKGATE_CON(29), 11, GFLAGS),
+
+ /* edp */
+ COMPOSITE(SCLK_DP_CORE, "clk_dp_core", mux_pll_src_npll_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(46), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(11), 8, GFLAGS),
+
+ COMPOSITE(PCLK_EDP, "pclk_edp", mux_pll_src_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(44), 15, 1, MFLAGS, 8, 5, DFLAGS,
+ RK3399_CLKGATE_CON(11), 11, GFLAGS),
+ GATE(PCLK_EDP_NOC, "pclk_edp_noc", "pclk_edp", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(32), 12, GFLAGS),
+ GATE(PCLK_EDP_CTRL, "pclk_edp_ctrl", "pclk_edp", 0,
+ RK3399_CLKGATE_CON(32), 13, GFLAGS),
+
+ /* hdmi */
+ GATE(SCLK_HDMI_SFR, "clk_hdmi_sfr", "xin24m", 0,
+ RK3399_CLKGATE_CON(11), 6, GFLAGS),
+
+ COMPOSITE(SCLK_HDMI_CEC, "clk_hdmi_cec", mux_pll_p, 0,
+ RK3399_CLKSEL_CON(45), 15, 1, MFLAGS, 0, 10, DFLAGS,
+ RK3399_CLKGATE_CON(11), 7, GFLAGS),
+
+ /* vop0 */
+ COMPOSITE(ACLK_VOP0_PRE, "aclk_vop0_pre", mux_pll_src_vpll_cpll_gpll_npll_p, 0,
+ RK3399_CLKSEL_CON(47), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(10), 8, GFLAGS),
+ COMPOSITE_NOMUX(0, "hclk_vop0_pre", "aclk_vop0_pre", 0,
+ RK3399_CLKSEL_CON(47), 8, 5, DFLAGS,
+ RK3399_CLKGATE_CON(10), 9, GFLAGS),
+
+ GATE(ACLK_VOP0, "aclk_vop0", "aclk_vop0_pre", 0,
+ RK3399_CLKGATE_CON(28), 3, GFLAGS),
+ GATE(ACLK_VOP0_NOC, "aclk_vop0_noc", "aclk_vop0_pre", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(28), 1, GFLAGS),
+
+ GATE(HCLK_VOP0, "hclk_vop0", "hclk_vop0_pre", 0,
+ RK3399_CLKGATE_CON(28), 2, GFLAGS),
+ GATE(HCLK_VOP0_NOC, "hclk_vop0_noc", "hclk_vop0_pre", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(28), 0, GFLAGS),
+
+ COMPOSITE(DCLK_VOP0_DIV, "dclk_vop0_div", mux_pll_src_vpll_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(49), 8, 2, MFLAGS, 0, 8, DFLAGS,
+ RK3399_CLKGATE_CON(10), 12, GFLAGS),
+
+ COMPOSITE_FRACMUX_NOGATE(0, "dclk_vop0_frac", "dclk_vop0_div", CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(106), 0,
+ &rk3399_dclk_vop0_fracmux),
+
+ COMPOSITE(SCLK_VOP0_PWM, "clk_vop0_pwm", mux_pll_src_vpll_cpll_gpll_24m_p, 0,
+ RK3399_CLKSEL_CON(51), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(10), 14, GFLAGS),
+
+ /* vop1 */
+ COMPOSITE(ACLK_VOP1_PRE, "aclk_vop1_pre", mux_pll_src_vpll_cpll_gpll_npll_p, 0,
+ RK3399_CLKSEL_CON(48), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(10), 10, GFLAGS),
+ COMPOSITE_NOMUX(0, "hclk_vop1_pre", "aclk_vop1_pre", 0,
+ RK3399_CLKSEL_CON(48), 8, 5, DFLAGS,
+ RK3399_CLKGATE_CON(10), 11, GFLAGS),
+
+ GATE(ACLK_VOP1, "aclk_vop1", "aclk_vop1_pre", 0,
+ RK3399_CLKGATE_CON(28), 7, GFLAGS),
+ GATE(ACLK_VOP1_NOC, "aclk_vop1_noc", "aclk_vop1_pre", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(28), 5, GFLAGS),
+
+ GATE(HCLK_VOP1, "hclk_vop1", "hclk_vop1_pre", 0,
+ RK3399_CLKGATE_CON(28), 6, GFLAGS),
+ GATE(HCLK_VOP1_NOC, "hclk_vop1_noc", "hclk_vop1_pre", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(28), 4, GFLAGS),
+
+ COMPOSITE(DCLK_VOP1_DIV, "dclk_vop1_div", mux_pll_src_vpll_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(50), 8, 2, MFLAGS, 0, 8, DFLAGS,
+ RK3399_CLKGATE_CON(10), 13, GFLAGS),
+
+ COMPOSITE_FRACMUX_NOGATE(0, "dclk_vop1_frac", "dclk_vop1_div", CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(107), 0,
+ &rk3399_dclk_vop1_fracmux),
+
+ COMPOSITE(SCLK_VOP1_PWM, "clk_vop1_pwm", mux_pll_src_vpll_cpll_gpll_24m_p, CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(52), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(10), 15, GFLAGS),
+
+ /* isp */
+ COMPOSITE(ACLK_ISP0, "aclk_isp0", mux_pll_src_cpll_gpll_ppll_p, 0,
+ RK3399_CLKSEL_CON(53), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(12), 8, GFLAGS),
+ COMPOSITE_NOMUX(HCLK_ISP0, "hclk_isp0", "aclk_isp0", 0,
+ RK3399_CLKSEL_CON(53), 8, 5, DFLAGS,
+ RK3399_CLKGATE_CON(12), 9, GFLAGS),
+
+ GATE(ACLK_ISP0_NOC, "aclk_isp0_noc", "aclk_isp0", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(27), 1, GFLAGS),
+ GATE(ACLK_ISP0_WRAPPER, "aclk_isp0_wrapper", "aclk_isp0", 0,
+ RK3399_CLKGATE_CON(27), 5, GFLAGS),
+ GATE(HCLK_ISP1_WRAPPER, "hclk_isp1_wrapper", "aclk_isp0", 0,
+ RK3399_CLKGATE_CON(27), 7, GFLAGS),
+
+ GATE(HCLK_ISP0_NOC, "hclk_isp0_noc", "hclk_isp0", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(27), 0, GFLAGS),
+ GATE(HCLK_ISP0_WRAPPER, "hclk_isp0_wrapper", "hclk_isp0", 0,
+ RK3399_CLKGATE_CON(27), 4, GFLAGS),
+
+ COMPOSITE(SCLK_ISP0, "clk_isp0", mux_pll_src_cpll_gpll_npll_p, 0,
+ RK3399_CLKSEL_CON(55), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(11), 4, GFLAGS),
+
+ COMPOSITE(ACLK_ISP1, "aclk_isp1", mux_pll_src_cpll_gpll_ppll_p, 0,
+ RK3399_CLKSEL_CON(54), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3399_CLKGATE_CON(12), 10, GFLAGS),
+ COMPOSITE_NOMUX(HCLK_ISP1, "hclk_isp1", "aclk_isp1", 0,
+ RK3399_CLKSEL_CON(54), 8, 5, DFLAGS,
+ RK3399_CLKGATE_CON(12), 11, GFLAGS),
+
+ GATE(ACLK_ISP1_NOC, "aclk_isp1_noc", "aclk_isp1", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(27), 3, GFLAGS),
+
+ GATE(HCLK_ISP1_NOC, "hclk_isp1_noc", "hclk_isp1", CLK_IGNORE_UNUSED,
+ RK3399_CLKGATE_CON(27), 2, GFLAGS),
+ GATE(ACLK_ISP1_WRAPPER, "aclk_isp1_wrapper", "hclk_isp1", 0,
+ RK3399_CLKGATE_CON(27), 8, GFLAGS),
+
+ COMPOSITE(SCLK_ISP1, "clk_isp1", mux_pll_src_cpll_gpll_npll_p, 0,
+ RK3399_CLKSEL_CON(55), 14, 2, MFLAGS, 8, 5, DFLAGS,
+ RK3399_CLKGATE_CON(11), 5, GFLAGS),
+
+ /*
+ * We use pclkin_cifinv by default GRF_SOC_CON20[9] (GSC20_9) setting in system,
+ * so we ignore the mux and make clocks nodes as following,
+ *
+ * pclkin_cifinv --|-------\
+ * |GSC20_9|-- pclkin_cifmux -- |G27_6| -- pclkin_isp1_wrapper
+ * pclkin_cif --|-------/
+ */
+ GATE(PCLK_ISP1_WRAPPER, "pclkin_isp1_wrapper", "pclkin_cif", 0,
+ RK3399_CLKGATE_CON(27), 6, GFLAGS),
+
+ /* cif */
+ COMPOSITE_NODIV(0, "clk_cifout_src", mux_pll_src_cpll_gpll_npll_p, 0,
+ RK3399_CLKSEL_CON(56), 6, 2, MFLAGS,
+ RK3399_CLKGATE_CON(10), 7, GFLAGS),
+
+ COMPOSITE_NOGATE(SCLK_CIF_OUT, "clk_cifout", mux_clk_cif_p, 0,
+ RK3399_CLKSEL_CON(56), 5, 1, MFLAGS, 0, 5, DFLAGS),
+
+ /* gic */
+ COMPOSITE(ACLK_GIC_PRE, "aclk_gic_pre", mux_pll_src_cpll_gpll_p, CLK_IGNORE_UNUSED,
+ RK3399_CLKSEL_CON(56), 15, 1, MFLAGS, 8, 5, DFLAGS,
+ RK3399_CLKGATE_CON(12), 12, GFLAGS),
+
+ GATE(ACLK_GIC, "aclk_gic", "aclk_gic_pre", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(33), 0, GFLAGS),
+ GATE(ACLK_GIC_NOC, "aclk_gic_noc", "aclk_gic_pre", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(33), 1, GFLAGS),
+ GATE(ACLK_GIC_ADB400_CORE_L_2_GIC, "aclk_gic_adb400_core_l_2_gic", "aclk_gic_pre", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(33), 2, GFLAGS),
+ GATE(ACLK_GIC_ADB400_CORE_B_2_GIC, "aclk_gic_adb400_core_b_2_gic", "aclk_gic_pre", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(33), 3, GFLAGS),
+ GATE(ACLK_GIC_ADB400_GIC_2_CORE_L, "aclk_gic_adb400_gic_2_core_l", "aclk_gic_pre", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(33), 4, GFLAGS),
+ GATE(ACLK_GIC_ADB400_GIC_2_CORE_B, "aclk_gic_adb400_gic_2_core_b", "aclk_gic_pre", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(33), 5, GFLAGS),
+
+ /* alive */
+ /* pclk_alive_gpll_src is controlled by PMUGRF_SOC_CON0[6] */
+ DIV(PCLK_ALIVE, "pclk_alive", "gpll", 0,
+ RK3399_CLKSEL_CON(57), 0, 5, DFLAGS),
+
+ GATE(PCLK_USBPHY_MUX_G, "pclk_usbphy_mux_g", "pclk_alive", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(21), 4, GFLAGS),
+ GATE(PCLK_UPHY0_TCPHY_G, "pclk_uphy0_tcphy_g", "pclk_alive", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(21), 5, GFLAGS),
+ GATE(PCLK_UPHY0_TCPD_G, "pclk_uphy0_tcpd_g", "pclk_alive", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(21), 6, GFLAGS),
+ GATE(PCLK_UPHY1_TCPHY_G, "pclk_uphy1_tcphy_g", "pclk_alive", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(21), 8, GFLAGS),
+ GATE(PCLK_UPHY1_TCPD_G, "pclk_uphy1_tcpd_g", "pclk_alive", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(21), 9, GFLAGS),
+
+ GATE(PCLK_GRF, "pclk_grf", "pclk_alive", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(31), 1, GFLAGS),
+ GATE(PCLK_INTR_ARB, "pclk_intr_arb", "pclk_alive", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(31), 2, GFLAGS),
+ GATE(PCLK_GPIO2, "pclk_gpio2", "pclk_alive", 0, RK3399_CLKGATE_CON(31), 3, GFLAGS),
+ GATE(PCLK_GPIO3, "pclk_gpio3", "pclk_alive", 0, RK3399_CLKGATE_CON(31), 4, GFLAGS),
+ GATE(PCLK_GPIO4, "pclk_gpio4", "pclk_alive", 0, RK3399_CLKGATE_CON(31), 5, GFLAGS),
+ GATE(PCLK_TIMER0, "pclk_timer0", "pclk_alive", 0, RK3399_CLKGATE_CON(31), 6, GFLAGS),
+ GATE(PCLK_TIMER1, "pclk_timer1", "pclk_alive", 0, RK3399_CLKGATE_CON(31), 7, GFLAGS),
+ GATE(PCLK_PMU_INTR_ARB, "pclk_pmu_intr_arb", "pclk_alive", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(31), 9, GFLAGS),
+ GATE(PCLK_SGRF, "pclk_sgrf", "pclk_alive", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(31), 10, GFLAGS),
+
+ GATE(SCLK_MIPIDPHY_REF, "clk_mipidphy_ref", "xin24m", 0, RK3399_CLKGATE_CON(11), 14, GFLAGS),
+ GATE(SCLK_DPHY_PLL, "clk_dphy_pll", "clk_mipidphy_ref", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(21), 0, GFLAGS),
+
+ GATE(SCLK_MIPIDPHY_CFG, "clk_mipidphy_cfg", "xin24m", 0, RK3399_CLKGATE_CON(11), 15, GFLAGS),
+ GATE(SCLK_DPHY_TX0_CFG, "clk_dphy_tx0_cfg", "clk_mipidphy_cfg", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(21), 1, GFLAGS),
+ GATE(SCLK_DPHY_TX1RX1_CFG, "clk_dphy_tx1rx1_cfg", "clk_mipidphy_cfg", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(21), 2, GFLAGS),
+ GATE(SCLK_DPHY_RX0_CFG, "clk_dphy_rx0_cfg", "clk_mipidphy_cfg", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(21), 3, GFLAGS),
+
+ /* testout */
+ MUX(0, "clk_test_pre", mux_pll_src_cpll_gpll_p, CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(58), 7, 1, MFLAGS),
+ COMPOSITE_FRAC(0, "clk_test_frac", "clk_test_pre", CLK_SET_RATE_PARENT,
+ RK3399_CLKSEL_CON(105), 0,
+ RK3399_CLKGATE_CON(13), 9, GFLAGS),
+
+ DIV(0, "clk_test_24m", "xin24m", 0,
+ RK3399_CLKSEL_CON(57), 6, 10, DFLAGS),
+
+ /* spi */
+ COMPOSITE(SCLK_SPI0, "clk_spi0", mux_pll_src_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(59), 7, 1, MFLAGS, 0, 7, DFLAGS,
+ RK3399_CLKGATE_CON(9), 12, GFLAGS),
+
+ COMPOSITE(SCLK_SPI1, "clk_spi1", mux_pll_src_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(59), 15, 1, MFLAGS, 8, 7, DFLAGS,
+ RK3399_CLKGATE_CON(9), 13, GFLAGS),
+
+ COMPOSITE(SCLK_SPI2, "clk_spi2", mux_pll_src_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(60), 7, 1, MFLAGS, 0, 7, DFLAGS,
+ RK3399_CLKGATE_CON(9), 14, GFLAGS),
+
+ COMPOSITE(SCLK_SPI4, "clk_spi4", mux_pll_src_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(60), 15, 1, MFLAGS, 8, 7, DFLAGS,
+ RK3399_CLKGATE_CON(9), 15, GFLAGS),
+
+ COMPOSITE(SCLK_SPI5, "clk_spi5", mux_pll_src_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(58), 15, 1, MFLAGS, 8, 7, DFLAGS,
+ RK3399_CLKGATE_CON(13), 13, GFLAGS),
+
+ /* i2c */
+ COMPOSITE(SCLK_I2C1, "clk_i2c1", mux_pll_src_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(61), 7, 1, MFLAGS, 0, 7, DFLAGS,
+ RK3399_CLKGATE_CON(10), 0, GFLAGS),
+
+ COMPOSITE(SCLK_I2C2, "clk_i2c2", mux_pll_src_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(62), 7, 1, MFLAGS, 0, 7, DFLAGS,
+ RK3399_CLKGATE_CON(10), 2, GFLAGS),
+
+ COMPOSITE(SCLK_I2C3, "clk_i2c3", mux_pll_src_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(63), 7, 1, MFLAGS, 0, 7, DFLAGS,
+ RK3399_CLKGATE_CON(10), 4, GFLAGS),
+
+ COMPOSITE(SCLK_I2C5, "clk_i2c5", mux_pll_src_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(61), 15, 1, MFLAGS, 8, 7, DFLAGS,
+ RK3399_CLKGATE_CON(10), 1, GFLAGS),
+
+ COMPOSITE(SCLK_I2C6, "clk_i2c6", mux_pll_src_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(62), 15, 1, MFLAGS, 8, 7, DFLAGS,
+ RK3399_CLKGATE_CON(10), 3, GFLAGS),
+
+ COMPOSITE(SCLK_I2C7, "clk_i2c7", mux_pll_src_cpll_gpll_p, 0,
+ RK3399_CLKSEL_CON(63), 15, 1, MFLAGS, 8, 7, DFLAGS,
+ RK3399_CLKGATE_CON(10), 5, GFLAGS),
+
+ /* timer */
+ GATE(SCLK_TIMER00, "clk_timer00", "xin24m", 0, RK3399_CLKGATE_CON(26), 0, GFLAGS),
+ GATE(SCLK_TIMER01, "clk_timer01", "xin24m", 0, RK3399_CLKGATE_CON(26), 1, GFLAGS),
+ GATE(SCLK_TIMER02, "clk_timer02", "xin24m", 0, RK3399_CLKGATE_CON(26), 2, GFLAGS),
+ GATE(SCLK_TIMER03, "clk_timer03", "xin24m", 0, RK3399_CLKGATE_CON(26), 3, GFLAGS),
+ GATE(SCLK_TIMER04, "clk_timer04", "xin24m", 0, RK3399_CLKGATE_CON(26), 4, GFLAGS),
+ GATE(SCLK_TIMER05, "clk_timer05", "xin24m", 0, RK3399_CLKGATE_CON(26), 5, GFLAGS),
+ GATE(SCLK_TIMER06, "clk_timer06", "xin24m", 0, RK3399_CLKGATE_CON(26), 6, GFLAGS),
+ GATE(SCLK_TIMER07, "clk_timer07", "xin24m", 0, RK3399_CLKGATE_CON(26), 7, GFLAGS),
+ GATE(SCLK_TIMER08, "clk_timer08", "xin24m", 0, RK3399_CLKGATE_CON(26), 8, GFLAGS),
+ GATE(SCLK_TIMER09, "clk_timer09", "xin24m", 0, RK3399_CLKGATE_CON(26), 9, GFLAGS),
+ GATE(SCLK_TIMER10, "clk_timer10", "xin24m", 0, RK3399_CLKGATE_CON(26), 10, GFLAGS),
+ GATE(SCLK_TIMER11, "clk_timer11", "xin24m", 0, RK3399_CLKGATE_CON(26), 11, GFLAGS),
+
+ /* clk_test */
+ /* clk_test_pre is controlled by CRU_MISC_CON[3] */
+ COMPOSITE_NOMUX(0, "clk_test", "clk_test_pre", CLK_IGNORE_UNUSED,
+ RK3368_CLKSEL_CON(58), 0, 5, DFLAGS,
+ RK3368_CLKGATE_CON(13), 11, GFLAGS),
+};
+
+static struct rockchip_clk_branch rk3399_clk_pmu_branches[] __initdata = {
+ /*
+ * PMU CRU Clock-Architecture
+ */
+
+ GATE(0, "fclk_cm0s_pmu_ppll_src", "ppll", 0,
+ RK3399_PMU_CLKGATE_CON(0), 1, GFLAGS),
+
+ COMPOSITE_NOGATE(FCLK_CM0S_SRC_PMU, "fclk_cm0s_src_pmu", mux_fclk_cm0s_pmu_ppll_p, 0,
+ RK3399_PMU_CLKSEL_CON(0), 15, 1, MFLAGS, 8, 5, DFLAGS),
+
+ COMPOSITE(SCLK_SPI3_PMU, "clk_spi3_pmu", mux_24m_ppll_p, 0,
+ RK3399_PMU_CLKSEL_CON(1), 7, 1, MFLAGS, 0, 7, DFLAGS,
+ RK3399_PMU_CLKGATE_CON(0), 2, GFLAGS),
+
+ COMPOSITE(0, "clk_wifi_div", mux_ppll_24m_p, CLK_IGNORE_UNUSED,
+ RK3399_PMU_CLKSEL_CON(1), 13, 1, MFLAGS, 8, 5, DFLAGS,
+ RK3399_PMU_CLKGATE_CON(0), 8, GFLAGS),
+
+ COMPOSITE_FRACMUX_NOGATE(0, "clk_wifi_frac", "clk_wifi_div", CLK_SET_RATE_PARENT,
+ RK3399_PMU_CLKSEL_CON(7), 0,
+ &rk3399_pmuclk_wifi_fracmux),
+
+ MUX(0, "clk_timer_src_pmu", mux_pll_p, CLK_IGNORE_UNUSED,
+ RK3399_PMU_CLKSEL_CON(1), 15, 1, MFLAGS),
+
+ COMPOSITE_NOMUX(SCLK_I2C0_PMU, "clk_i2c0_pmu", "ppll", 0,
+ RK3399_PMU_CLKSEL_CON(2), 0, 7, DFLAGS,
+ RK3399_PMU_CLKGATE_CON(0), 9, GFLAGS),
+
+ COMPOSITE_NOMUX(SCLK_I2C4_PMU, "clk_i2c4_pmu", "ppll", 0,
+ RK3399_PMU_CLKSEL_CON(3), 0, 7, DFLAGS,
+ RK3399_PMU_CLKGATE_CON(0), 10, GFLAGS),
+
+ COMPOSITE_NOMUX(SCLK_I2C8_PMU, "clk_i2c8_pmu", "ppll", 0,
+ RK3399_PMU_CLKSEL_CON(2), 8, 7, DFLAGS,
+ RK3399_PMU_CLKGATE_CON(0), 11, GFLAGS),
+
+ DIV(0, "clk_32k_suspend_pmu", "xin24m", CLK_IGNORE_UNUSED,
+ RK3399_PMU_CLKSEL_CON(4), 0, 10, DFLAGS),
+ MUX(0, "clk_testout_2io", mux_clk_testout2_2io_p, CLK_IGNORE_UNUSED,
+ RK3399_PMU_CLKSEL_CON(4), 15, 1, MFLAGS),
+
+ COMPOSITE(0, "clk_uart4_div", mux_24m_ppll_p, 0,
+ RK3399_PMU_CLKSEL_CON(5), 10, 1, MFLAGS, 0, 7, DFLAGS,
+ RK3399_PMU_CLKGATE_CON(0), 5, GFLAGS),
+
+ COMPOSITE_FRACMUX(0, "clk_uart4_frac", "clk_uart4_div", CLK_SET_RATE_PARENT,
+ RK3399_PMU_CLKSEL_CON(6), 0,
+ RK3399_PMU_CLKGATE_CON(0), 6, GFLAGS,
+ &rk3399_uart4_pmu_fracmux),
+
+ DIV(PCLK_SRC_PMU, "pclk_pmu_src", "ppll", CLK_IGNORE_UNUSED,
+ RK3399_PMU_CLKSEL_CON(0), 0, 5, DFLAGS),
+
+ /* pmu clock gates */
+ GATE(SCLK_TIMER12_PMU, "clk_timer0_pmu", "clk_timer_src_pmu", 0, RK3399_PMU_CLKGATE_CON(0), 3, GFLAGS),
+ GATE(SCLK_TIMER13_PMU, "clk_timer1_pmu", "clk_timer_src_pmu", 0, RK3399_PMU_CLKGATE_CON(0), 4, GFLAGS),
+
+ GATE(SCLK_PVTM_PMU, "clk_pvtm_pmu", "xin24m", CLK_IGNORE_UNUSED, RK3399_PMU_CLKGATE_CON(0), 7, GFLAGS),
+
+ GATE(PCLK_PMU, "pclk_pmu", "pclk_pmu_src", CLK_IGNORE_UNUSED, RK3399_PMU_CLKGATE_CON(1), 0, GFLAGS),
+ GATE(PCLK_PMUGRF_PMU, "pclk_pmugrf_pmu", "pclk_pmu_src", CLK_IGNORE_UNUSED, RK3399_PMU_CLKGATE_CON(1), 1, GFLAGS),
+ GATE(PCLK_INTMEM1_PMU, "pclk_intmem1_pmu", "pclk_pmu_src", CLK_IGNORE_UNUSED, RK3399_PMU_CLKGATE_CON(1), 2, GFLAGS),
+ GATE(PCLK_GPIO0_PMU, "pclk_gpio0_pmu", "pclk_pmu_src", 0, RK3399_PMU_CLKGATE_CON(1), 3, GFLAGS),
+ GATE(PCLK_GPIO1_PMU, "pclk_gpio1_pmu", "pclk_pmu_src", 0, RK3399_PMU_CLKGATE_CON(1), 4, GFLAGS),
+ GATE(PCLK_SGRF_PMU, "pclk_sgrf_pmu", "pclk_pmu_src", CLK_IGNORE_UNUSED, RK3399_PMU_CLKGATE_CON(1), 5, GFLAGS),
+ GATE(PCLK_NOC_PMU, "pclk_noc_pmu", "pclk_pmu_src", CLK_IGNORE_UNUSED, RK3399_PMU_CLKGATE_CON(1), 6, GFLAGS),
+ GATE(PCLK_I2C0_PMU, "pclk_i2c0_pmu", "pclk_pmu_src", 0, RK3399_PMU_CLKGATE_CON(1), 7, GFLAGS),
+ GATE(PCLK_I2C4_PMU, "pclk_i2c4_pmu", "pclk_pmu_src", 0, RK3399_PMU_CLKGATE_CON(1), 8, GFLAGS),
+ GATE(PCLK_I2C8_PMU, "pclk_i2c8_pmu", "pclk_pmu_src", 0, RK3399_PMU_CLKGATE_CON(1), 9, GFLAGS),
+ GATE(PCLK_RKPWM_PMU, "pclk_rkpwm_pmu", "pclk_pmu_src", 0, RK3399_PMU_CLKGATE_CON(1), 10, GFLAGS),
+ GATE(PCLK_SPI3_PMU, "pclk_spi3_pmu", "pclk_pmu_src", 0, RK3399_PMU_CLKGATE_CON(1), 11, GFLAGS),
+ GATE(PCLK_TIMER_PMU, "pclk_timer_pmu", "pclk_pmu_src", 0, RK3399_PMU_CLKGATE_CON(1), 12, GFLAGS),
+ GATE(PCLK_MAILBOX_PMU, "pclk_mailbox_pmu", "pclk_pmu_src", 0, RK3399_PMU_CLKGATE_CON(1), 13, GFLAGS),
+ GATE(PCLK_UART4_PMU, "pclk_uart4_pmu", "pclk_pmu_src", 0, RK3399_PMU_CLKGATE_CON(1), 14, GFLAGS),
+ GATE(PCLK_WDT_M0_PMU, "pclk_wdt_m0_pmu", "pclk_pmu_src", 0, RK3399_PMU_CLKGATE_CON(1), 15, GFLAGS),
+
+ GATE(FCLK_CM0S_PMU, "fclk_cm0s_pmu", "fclk_cm0s_src_pmu", 0, RK3399_PMU_CLKGATE_CON(2), 0, GFLAGS),
+ GATE(SCLK_CM0S_PMU, "sclk_cm0s_pmu", "fclk_cm0s_src_pmu", 0, RK3399_PMU_CLKGATE_CON(2), 1, GFLAGS),
+ GATE(HCLK_CM0S_PMU, "hclk_cm0s_pmu", "fclk_cm0s_src_pmu", 0, RK3399_PMU_CLKGATE_CON(2), 2, GFLAGS),
+ GATE(DCLK_CM0S_PMU, "dclk_cm0s_pmu", "fclk_cm0s_src_pmu", 0, RK3399_PMU_CLKGATE_CON(2), 3, GFLAGS),
+ GATE(HCLK_NOC_PMU, "hclk_noc_pmu", "fclk_cm0s_src_pmu", CLK_IGNORE_UNUSED, RK3399_PMU_CLKGATE_CON(2), 5, GFLAGS),
+};
+
+static const char *const rk3399_cru_critical_clocks[] __initconst = {
+ "aclk_cci_pre",
+ "pclk_perilp0",
+ "pclk_perilp0",
+ "hclk_perilp0",
+ "hclk_perilp0_noc",
+ "pclk_perilp1",
+ "pclk_perilp1_noc",
+ "pclk_perihp",
+ "pclk_perihp_noc",
+ "hclk_perihp",
+ "aclk_perihp",
+ "aclk_perihp_noc",
+ "aclk_perilp0",
+ "aclk_perilp0_noc",
+ "hclk_perilp1",
+ "hclk_perilp1_noc",
+ "aclk_dmac0_perilp",
+ "gpll_hclk_perilp1_src",
+ "gpll_aclk_perilp0_src",
+ "gpll_aclk_perihp_src",
+};
+
+static const char *const rk3399_pmucru_critical_clocks[] __initconst = {
+ "ppll",
+ "pclk_pmu_src",
+ "fclk_cm0s_src_pmu",
+ "clk_timer_src_pmu",
+};
+
+static void __init rk3399_clk_init(struct device_node *np)
+{
+ struct rockchip_clk_provider *ctx;
+ void __iomem *reg_base;
+
+ reg_base = of_iomap(np, 0);
+ if (!reg_base) {
+ pr_err("%s: could not map cru region\n", __func__);
+ return;
+ }
+
+ ctx = rockchip_clk_init(np, reg_base, CLK_NR_CLKS);
+ if (IS_ERR(ctx)) {
+ pr_err("%s: rockchip clk init failed\n", __func__);
+ return;
+ }
+
+ rockchip_clk_register_plls(ctx, rk3399_pll_clks,
+ ARRAY_SIZE(rk3399_pll_clks), -1);
+
+ rockchip_clk_register_branches(ctx, rk3399_clk_branches,
+ ARRAY_SIZE(rk3399_clk_branches));
+
+ rockchip_clk_protect_critical(rk3399_cru_critical_clocks,
+ ARRAY_SIZE(rk3399_cru_critical_clocks));
+
+ rockchip_clk_register_armclk(ctx, ARMCLKL, "armclkl",
+ mux_armclkl_p, ARRAY_SIZE(mux_armclkl_p),
+ &rk3399_cpuclkl_data, rk3399_cpuclkl_rates,
+ ARRAY_SIZE(rk3399_cpuclkl_rates));
+
+ rockchip_clk_register_armclk(ctx, ARMCLKB, "armclkb",
+ mux_armclkb_p, ARRAY_SIZE(mux_armclkb_p),
+ &rk3399_cpuclkb_data, rk3399_cpuclkb_rates,
+ ARRAY_SIZE(rk3399_cpuclkb_rates));
+
+ rockchip_register_softrst(np, 21, reg_base + RK3399_SOFTRST_CON(0),
+ ROCKCHIP_SOFTRST_HIWORD_MASK);
+
+ rockchip_register_restart_notifier(ctx, RK3399_GLB_SRST_FST, NULL);
+
+ rockchip_clk_of_add_provider(np, ctx);
+}
+CLK_OF_DECLARE(rk3399_cru, "rockchip,rk3399-cru", rk3399_clk_init);
+
+static void __init rk3399_pmu_clk_init(struct device_node *np)
+{
+ struct rockchip_clk_provider *ctx;
+ void __iomem *reg_base;
+
+ reg_base = of_iomap(np, 0);
+ if (!reg_base) {
+ pr_err("%s: could not map cru pmu region\n", __func__);
+ return;
+ }
+
+ ctx = rockchip_clk_init(np, reg_base, CLKPMU_NR_CLKS);
+ if (IS_ERR(ctx)) {
+ pr_err("%s: rockchip pmu clk init failed\n", __func__);
+ return;
+ }
+
+ rockchip_clk_register_plls(ctx, rk3399_pmu_pll_clks,
+ ARRAY_SIZE(rk3399_pmu_pll_clks), -1);
+
+ rockchip_clk_register_branches(ctx, rk3399_clk_pmu_branches,
+ ARRAY_SIZE(rk3399_clk_pmu_branches));
+
+ rockchip_clk_protect_critical(rk3399_pmucru_critical_clocks,
+ ARRAY_SIZE(rk3399_pmucru_critical_clocks));
+
+ rockchip_register_softrst(np, 2, reg_base + RK3399_PMU_SOFTRST_CON(0),
+ ROCKCHIP_SOFTRST_HIWORD_MASK);
+
+ rockchip_clk_of_add_provider(np, ctx);
+}
+CLK_OF_DECLARE(rk3399_cru_pmu, "rockchip,rk3399-pmucru", rk3399_pmu_clk_init);
diff --git a/drivers/clk/rockchip/clk.c b/drivers/clk/rockchip/clk.c
index ec06350..7ffd134 100644
--- a/drivers/clk/rockchip/clk.c
+++ b/drivers/clk/rockchip/clk.c
@@ -2,6 +2,9 @@
* Copyright (c) 2014 MundoReader S.L.
* Author: Heiko Stuebner <heiko@sntech.de>
*
+ * Copyright (c) 2016 Rockchip Electronics Co. Ltd.
+ * Author: Xing Zheng <zhengxing@rock-chips.com>
+ *
* based on
*
* samsung/clk.c
@@ -39,7 +42,8 @@
* sometimes without one of those components.
*/
static struct clk *rockchip_clk_register_branch(const char *name,
- const char *const *parent_names, u8 num_parents, void __iomem *base,
+ const char *const *parent_names, u8 num_parents,
+ void __iomem *base,
int muxdiv_offset, u8 mux_shift, u8 mux_width, u8 mux_flags,
u8 div_shift, u8 div_width, u8 div_flags,
struct clk_div_table *div_table, int gate_offset,
@@ -136,9 +140,11 @@
pr_debug("%s: event %lu, old_rate %lu, new_rate: %lu\n",
__func__, event, ndata->old_rate, ndata->new_rate);
if (event == PRE_RATE_CHANGE) {
- frac->rate_change_idx = frac->mux_ops->get_parent(&frac_mux->hw);
+ frac->rate_change_idx =
+ frac->mux_ops->get_parent(&frac_mux->hw);
if (frac->rate_change_idx != frac->mux_frac_idx) {
- frac->mux_ops->set_parent(&frac_mux->hw, frac->mux_frac_idx);
+ frac->mux_ops->set_parent(&frac_mux->hw,
+ frac->mux_frac_idx);
frac->rate_change_remuxed = 1;
}
} else if (event == POST_RATE_CHANGE) {
@@ -149,7 +155,8 @@
* reaches the mux itself.
*/
if (frac->rate_change_remuxed) {
- frac->mux_ops->set_parent(&frac_mux->hw, frac->rate_change_idx);
+ frac->mux_ops->set_parent(&frac_mux->hw,
+ frac->rate_change_idx);
frac->rate_change_remuxed = 0;
}
}
@@ -157,7 +164,8 @@
return notifier_from_errno(ret);
}
-static struct clk *rockchip_clk_register_frac_branch(const char *name,
+static struct clk *rockchip_clk_register_frac_branch(
+ struct rockchip_clk_provider *ctx, const char *name,
const char *const *parent_names, u8 num_parents,
void __iomem *base, int muxdiv_offset, u8 div_flags,
int gate_offset, u8 gate_shift, u8 gate_flags,
@@ -250,7 +258,7 @@
if (IS_ERR(mux_clk))
return clk;
- rockchip_clk_add_lookup(mux_clk, child->id);
+ rockchip_clk_add_lookup(ctx, mux_clk, child->id);
/* notifier on the fraction divider to catch rate changes */
if (frac->mux_frac_idx >= 0) {
@@ -314,66 +322,82 @@
return clk;
}
-static DEFINE_SPINLOCK(clk_lock);
-static struct clk **clk_table;
-static void __iomem *reg_base;
-static struct clk_onecell_data clk_data;
-static struct device_node *cru_node;
-static struct regmap *grf;
-
-void __init rockchip_clk_init(struct device_node *np, void __iomem *base,
- unsigned long nr_clks)
+struct rockchip_clk_provider * __init rockchip_clk_init(struct device_node *np,
+ void __iomem *base, unsigned long nr_clks)
{
- reg_base = base;
- cru_node = np;
- grf = ERR_PTR(-EPROBE_DEFER);
+ struct rockchip_clk_provider *ctx;
+ struct clk **clk_table;
+ int i;
+
+ ctx = kzalloc(sizeof(struct rockchip_clk_provider), GFP_KERNEL);
+ if (!ctx)
+ return ERR_PTR(-ENOMEM);
clk_table = kcalloc(nr_clks, sizeof(struct clk *), GFP_KERNEL);
if (!clk_table)
- pr_err("%s: could not allocate clock lookup table\n", __func__);
+ goto err_free;
- clk_data.clks = clk_table;
- clk_data.clk_num = nr_clks;
- of_clk_add_provider(np, of_clk_src_onecell_get, &clk_data);
+ for (i = 0; i < nr_clks; ++i)
+ clk_table[i] = ERR_PTR(-ENOENT);
+
+ ctx->reg_base = base;
+ ctx->clk_data.clks = clk_table;
+ ctx->clk_data.clk_num = nr_clks;
+ ctx->cru_node = np;
+ ctx->grf = ERR_PTR(-EPROBE_DEFER);
+ spin_lock_init(&ctx->lock);
+
+ ctx->grf = syscon_regmap_lookup_by_phandle(ctx->cru_node,
+ "rockchip,grf");
+
+ return ctx;
+
+err_free:
+ kfree(ctx);
+ return ERR_PTR(-ENOMEM);
}
-struct regmap *rockchip_clk_get_grf(void)
+void __init rockchip_clk_of_add_provider(struct device_node *np,
+ struct rockchip_clk_provider *ctx)
{
- if (IS_ERR(grf))
- grf = syscon_regmap_lookup_by_phandle(cru_node, "rockchip,grf");
- return grf;
+ if (of_clk_add_provider(np, of_clk_src_onecell_get,
+ &ctx->clk_data))
+ pr_err("%s: could not register clk provider\n", __func__);
}
-void rockchip_clk_add_lookup(struct clk *clk, unsigned int id)
+void rockchip_clk_add_lookup(struct rockchip_clk_provider *ctx,
+ struct clk *clk, unsigned int id)
{
- if (clk_table && id)
- clk_table[id] = clk;
+ if (ctx->clk_data.clks && id)
+ ctx->clk_data.clks[id] = clk;
}
-void __init rockchip_clk_register_plls(struct rockchip_pll_clock *list,
+void __init rockchip_clk_register_plls(struct rockchip_clk_provider *ctx,
+ struct rockchip_pll_clock *list,
unsigned int nr_pll, int grf_lock_offset)
{
struct clk *clk;
int idx;
for (idx = 0; idx < nr_pll; idx++, list++) {
- clk = rockchip_clk_register_pll(list->type, list->name,
+ clk = rockchip_clk_register_pll(ctx, list->type, list->name,
list->parent_names, list->num_parents,
- reg_base, list->con_offset, grf_lock_offset,
+ list->con_offset, grf_lock_offset,
list->lock_shift, list->mode_offset,
list->mode_shift, list->rate_table,
- list->pll_flags, &clk_lock);
+ list->pll_flags);
if (IS_ERR(clk)) {
pr_err("%s: failed to register clock %s\n", __func__,
list->name);
continue;
}
- rockchip_clk_add_lookup(clk, list->id);
+ rockchip_clk_add_lookup(ctx, clk, list->id);
}
}
void __init rockchip_clk_register_branches(
+ struct rockchip_clk_provider *ctx,
struct rockchip_clk_branch *list,
unsigned int nr_clk)
{
@@ -389,56 +413,59 @@
case branch_mux:
clk = clk_register_mux(NULL, list->name,
list->parent_names, list->num_parents,
- flags, reg_base + list->muxdiv_offset,
+ flags, ctx->reg_base + list->muxdiv_offset,
list->mux_shift, list->mux_width,
- list->mux_flags, &clk_lock);
+ list->mux_flags, &ctx->lock);
break;
case branch_divider:
if (list->div_table)
clk = clk_register_divider_table(NULL,
list->name, list->parent_names[0],
- flags, reg_base + list->muxdiv_offset,
+ flags,
+ ctx->reg_base + list->muxdiv_offset,
list->div_shift, list->div_width,
list->div_flags, list->div_table,
- &clk_lock);
+ &ctx->lock);
else
clk = clk_register_divider(NULL, list->name,
list->parent_names[0], flags,
- reg_base + list->muxdiv_offset,
+ ctx->reg_base + list->muxdiv_offset,
list->div_shift, list->div_width,
- list->div_flags, &clk_lock);
+ list->div_flags, &ctx->lock);
break;
case branch_fraction_divider:
- clk = rockchip_clk_register_frac_branch(list->name,
+ clk = rockchip_clk_register_frac_branch(ctx, list->name,
list->parent_names, list->num_parents,
- reg_base, list->muxdiv_offset, list->div_flags,
+ ctx->reg_base, list->muxdiv_offset,
+ list->div_flags,
list->gate_offset, list->gate_shift,
list->gate_flags, flags, list->child,
- &clk_lock);
+ &ctx->lock);
break;
case branch_gate:
flags |= CLK_SET_RATE_PARENT;
clk = clk_register_gate(NULL, list->name,
list->parent_names[0], flags,
- reg_base + list->gate_offset,
- list->gate_shift, list->gate_flags, &clk_lock);
+ ctx->reg_base + list->gate_offset,
+ list->gate_shift, list->gate_flags, &ctx->lock);
break;
case branch_composite:
clk = rockchip_clk_register_branch(list->name,
list->parent_names, list->num_parents,
- reg_base, list->muxdiv_offset, list->mux_shift,
+ ctx->reg_base, list->muxdiv_offset,
+ list->mux_shift,
list->mux_width, list->mux_flags,
list->div_shift, list->div_width,
list->div_flags, list->div_table,
list->gate_offset, list->gate_shift,
- list->gate_flags, flags, &clk_lock);
+ list->gate_flags, flags, &ctx->lock);
break;
case branch_mmc:
clk = rockchip_clk_register_mmc(
list->name,
list->parent_names, list->num_parents,
- reg_base + list->muxdiv_offset,
+ ctx->reg_base + list->muxdiv_offset,
list->div_shift
);
break;
@@ -446,16 +473,16 @@
clk = rockchip_clk_register_inverter(
list->name, list->parent_names,
list->num_parents,
- reg_base + list->muxdiv_offset,
- list->div_shift, list->div_flags, &clk_lock);
+ ctx->reg_base + list->muxdiv_offset,
+ list->div_shift, list->div_flags, &ctx->lock);
break;
case branch_factor:
clk = rockchip_clk_register_factor_branch(
list->name, list->parent_names,
- list->num_parents, reg_base,
+ list->num_parents, ctx->reg_base,
list->div_shift, list->div_width,
list->gate_offset, list->gate_shift,
- list->gate_flags, flags, &clk_lock);
+ list->gate_flags, flags, &ctx->lock);
break;
}
@@ -472,11 +499,12 @@
continue;
}
- rockchip_clk_add_lookup(clk, list->id);
+ rockchip_clk_add_lookup(ctx, clk, list->id);
}
}
-void __init rockchip_clk_register_armclk(unsigned int lookup_id,
+void __init rockchip_clk_register_armclk(struct rockchip_clk_provider *ctx,
+ unsigned int lookup_id,
const char *name, const char *const *parent_names,
u8 num_parents,
const struct rockchip_cpuclk_reg_data *reg_data,
@@ -486,15 +514,15 @@
struct clk *clk;
clk = rockchip_clk_register_cpuclk(name, parent_names, num_parents,
- reg_data, rates, nrates, reg_base,
- &clk_lock);
+ reg_data, rates, nrates,
+ ctx->reg_base, &ctx->lock);
if (IS_ERR(clk)) {
pr_err("%s: failed to register clock %s: %ld\n",
__func__, name, PTR_ERR(clk));
return;
}
- rockchip_clk_add_lookup(clk, lookup_id);
+ rockchip_clk_add_lookup(ctx, clk, lookup_id);
}
void __init rockchip_clk_protect_critical(const char *const clocks[],
@@ -511,6 +539,7 @@
}
}
+static void __iomem *rst_base;
static unsigned int reg_restart;
static void (*cb_restart)(void);
static int rockchip_restart_notify(struct notifier_block *this,
@@ -519,7 +548,7 @@
if (cb_restart)
cb_restart();
- writel(0xfdb9, reg_base + reg_restart);
+ writel(0xfdb9, rst_base + reg_restart);
return NOTIFY_DONE;
}
@@ -528,10 +557,14 @@
.priority = 128,
};
-void __init rockchip_register_restart_notifier(unsigned int reg, void (*cb)(void))
+void __init
+rockchip_register_restart_notifier(struct rockchip_clk_provider *ctx,
+ unsigned int reg,
+ void (*cb)(void))
{
int ret;
+ rst_base = ctx->reg_base;
reg_restart = reg;
cb_restart = cb;
ret = register_restart_handler(&rockchip_restart_handler);
diff --git a/drivers/clk/rockchip/clk.h b/drivers/clk/rockchip/clk.h
index 39c198b..2194ffa 100644
--- a/drivers/clk/rockchip/clk.h
+++ b/drivers/clk/rockchip/clk.h
@@ -27,13 +27,13 @@
#define CLK_ROCKCHIP_CLK_H
#include <linux/io.h>
+#include <linux/clk-provider.h>
struct clk;
#define HIWORD_UPDATE(val, mask, shift) \
((val) << (shift) | (mask) << ((shift) + 16))
-/* register positions shared by RK2928, RK3036, RK3066, RK3188 and RK3228 */
#define RK2928_PLL_CON(x) ((x) * 0x4)
#define RK2928_MODE_CON 0x40
#define RK2928_CLKSEL_CON(x) ((x) * 0x4 + 0x44)
@@ -92,9 +92,30 @@
#define RK3368_EMMC_CON0 0x418
#define RK3368_EMMC_CON1 0x41c
+#define RK3399_PLL_CON(x) RK2928_PLL_CON(x)
+#define RK3399_CLKSEL_CON(x) ((x) * 0x4 + 0x100)
+#define RK3399_CLKGATE_CON(x) ((x) * 0x4 + 0x300)
+#define RK3399_SOFTRST_CON(x) ((x) * 0x4 + 0x400)
+#define RK3399_GLB_SRST_FST 0x500
+#define RK3399_GLB_SRST_SND 0x504
+#define RK3399_GLB_CNT_TH 0x508
+#define RK3399_MISC_CON 0x50c
+#define RK3399_RST_CON 0x510
+#define RK3399_RST_ST 0x514
+#define RK3399_SDMMC_CON0 0x580
+#define RK3399_SDMMC_CON1 0x584
+#define RK3399_SDIO_CON0 0x588
+#define RK3399_SDIO_CON1 0x58c
+
+#define RK3399_PMU_PLL_CON(x) RK2928_PLL_CON(x)
+#define RK3399_PMU_CLKSEL_CON(x) ((x) * 0x4 + 0x80)
+#define RK3399_PMU_CLKGATE_CON(x) ((x) * 0x4 + 0x100)
+#define RK3399_PMU_SOFTRST_CON(x) ((x) * 0x4 + 0x110)
+
enum rockchip_pll_type {
pll_rk3036,
pll_rk3066,
+ pll_rk3399,
};
#define RK3036_PLL_RATE(_rate, _refdiv, _fbdiv, _postdiv1, \
@@ -127,13 +148,29 @@
.nb = _nb, \
}
+/**
+ * struct rockchip_clk_provider - information about clock provider
+ * @reg_base: virtual address for the register base.
+ * @clk_data: holds clock related data like clk* and number of clocks.
+ * @cru_node: device-node of the clock-provider
+ * @grf: regmap of the general-register-files syscon
+ * @lock: maintains exclusion between callbacks for a given clock-provider.
+ */
+struct rockchip_clk_provider {
+ void __iomem *reg_base;
+ struct clk_onecell_data clk_data;
+ struct device_node *cru_node;
+ struct regmap *grf;
+ spinlock_t lock;
+};
+
struct rockchip_pll_rate_table {
unsigned long rate;
unsigned int nr;
unsigned int nf;
unsigned int no;
unsigned int nb;
- /* for RK3036 */
+ /* for RK3036/RK3399 */
unsigned int fbdiv;
unsigned int postdiv1;
unsigned int refdiv;
@@ -143,10 +180,11 @@
};
/**
- * struct rockchip_pll_clock: information about pll clock
+ * struct rockchip_pll_clock - information about pll clock
* @id: platform specific id of the clock.
* @name: name of this pll clock.
- * @parent_name: name of the parent clock.
+ * @parent_names: name of the parent clock.
+ * @num_parents: number of parents
* @flags: optional flags for basic clock.
* @con_offset: offset of the register for configuring the PLL.
* @mode_offset: offset of the register for configuring the PLL-mode.
@@ -194,12 +232,13 @@
.rate_table = _rtable, \
}
-struct clk *rockchip_clk_register_pll(enum rockchip_pll_type pll_type,
+struct clk *rockchip_clk_register_pll(struct rockchip_clk_provider *ctx,
+ enum rockchip_pll_type pll_type,
const char *name, const char *const *parent_names,
- u8 num_parents, void __iomem *base, int con_offset,
- int grf_lock_offset, int lock_shift, int reg_mode,
- int mode_shift, struct rockchip_pll_rate_table *rate_table,
- u8 clk_pll_flags, spinlock_t *lock);
+ u8 num_parents, int con_offset, int grf_lock_offset,
+ int lock_shift, int mode_offset, int mode_shift,
+ struct rockchip_pll_rate_table *rate_table,
+ u8 clk_pll_flags);
struct rockchip_cpuclk_clksel {
int reg;
@@ -213,18 +252,23 @@
};
/**
- * struct rockchip_cpuclk_reg_data: describes register offsets and masks of the cpuclock
+ * struct rockchip_cpuclk_reg_data - register offsets and masks of the cpuclock
* @core_reg: register offset of the core settings register
* @div_core_shift: core divider offset used to divide the pll value
* @div_core_mask: core divider mask
+ * @mux_core_alt: mux value to select alternate parent
+ * @mux_core_main: mux value to select main parent of core
* @mux_core_shift: offset of the core multiplexer
+ * @mux_core_mask: core multiplexer mask
*/
struct rockchip_cpuclk_reg_data {
int core_reg;
u8 div_core_shift;
u32 div_core_mask;
- int mux_core_reg;
+ u8 mux_core_alt;
+ u8 mux_core_main;
u8 mux_core_shift;
+ u32 mux_core_mask;
};
struct clk *rockchip_clk_register_cpuclk(const char *name,
@@ -428,6 +472,22 @@
.child = ch, \
}
+#define COMPOSITE_FRACMUX_NOGATE(_id, cname, pname, f, mo, df, ch) \
+ { \
+ .id = _id, \
+ .branch_type = branch_fraction_divider, \
+ .name = cname, \
+ .parent_names = (const char *[]){ pname }, \
+ .num_parents = 1, \
+ .flags = f, \
+ .muxdiv_offset = mo, \
+ .div_shift = 16, \
+ .div_width = 16, \
+ .div_flags = df, \
+ .gate_offset = -1, \
+ .child = ch, \
+ }
+
#define MUX(_id, cname, pnames, f, o, s, w, mf) \
{ \
.id = _id, \
@@ -536,21 +596,27 @@
.gate_flags = gf, \
}
-void rockchip_clk_init(struct device_node *np, void __iomem *base,
- unsigned long nr_clks);
-struct regmap *rockchip_clk_get_grf(void);
-void rockchip_clk_add_lookup(struct clk *clk, unsigned int id);
-void rockchip_clk_register_branches(struct rockchip_clk_branch *clk_list,
+struct rockchip_clk_provider *rockchip_clk_init(struct device_node *np,
+ void __iomem *base, unsigned long nr_clks);
+void rockchip_clk_of_add_provider(struct device_node *np,
+ struct rockchip_clk_provider *ctx);
+void rockchip_clk_add_lookup(struct rockchip_clk_provider *ctx,
+ struct clk *clk, unsigned int id);
+void rockchip_clk_register_branches(struct rockchip_clk_provider *ctx,
+ struct rockchip_clk_branch *list,
unsigned int nr_clk);
-void rockchip_clk_register_plls(struct rockchip_pll_clock *pll_list,
+void rockchip_clk_register_plls(struct rockchip_clk_provider *ctx,
+ struct rockchip_pll_clock *pll_list,
unsigned int nr_pll, int grf_lock_offset);
-void rockchip_clk_register_armclk(unsigned int lookup_id, const char *name,
+void rockchip_clk_register_armclk(struct rockchip_clk_provider *ctx,
+ unsigned int lookup_id, const char *name,
const char *const *parent_names, u8 num_parents,
const struct rockchip_cpuclk_reg_data *reg_data,
const struct rockchip_cpuclk_rate_table *rates,
int nrates);
void rockchip_clk_protect_critical(const char *const clocks[], int nclocks);
-void rockchip_register_restart_notifier(unsigned int reg, void (*cb)(void));
+void rockchip_register_restart_notifier(struct rockchip_clk_provider *ctx,
+ unsigned int reg, void (*cb)(void));
#define ROCKCHIP_SOFTRST_HIWORD_MASK BIT(0)
diff --git a/drivers/clk/samsung/clk-exynos5420.c b/drivers/clk/samsung/clk-exynos5420.c
index be03ed0..92382ce 100644
--- a/drivers/clk/samsung/clk-exynos5420.c
+++ b/drivers/clk/samsung/clk-exynos5420.c
@@ -554,8 +554,8 @@
};
static struct samsung_div_clock exynos5800_div_clks[] __initdata = {
- DIV(0, "dout_aclk400_wcore", "mout_aclk400_wcore", DIV_TOP0, 16, 3),
-
+ DIV(CLK_DOUT_ACLK400_WCORE, "dout_aclk400_wcore",
+ "mout_aclk400_wcore", DIV_TOP0, 16, 3),
DIV(0, "dout_aclk550_cam", "mout_aclk550_cam",
DIV_TOP8, 16, 3),
DIV(0, "dout_aclkfl1_550_cam", "mout_aclkfl1_550_cam",
@@ -607,8 +607,8 @@
};
static struct samsung_div_clock exynos5420_div_clks[] __initdata = {
- DIV(0, "dout_aclk400_wcore", "mout_aclk400_wcore_bpll",
- DIV_TOP0, 16, 3),
+ DIV(CLK_DOUT_ACLK400_WCORE, "dout_aclk400_wcore",
+ "mout_aclk400_wcore_bpll", DIV_TOP0, 16, 3),
};
static struct samsung_mux_clock exynos5x_mux_clks[] __initdata = {
@@ -785,31 +785,47 @@
DIV(0, "div_kfc", "mout_kfc", DIV_KFC0, 0, 3),
DIV(0, "sclk_kpll", "mout_kpll", DIV_KFC0, 24, 3),
- DIV(0, "dout_aclk400_isp", "mout_aclk400_isp", DIV_TOP0, 0, 3),
- DIV(0, "dout_aclk400_mscl", "mout_aclk400_mscl", DIV_TOP0, 4, 3),
- DIV(0, "dout_aclk200", "mout_aclk200", DIV_TOP0, 8, 3),
- DIV(0, "dout_aclk200_fsys2", "mout_aclk200_fsys2", DIV_TOP0, 12, 3),
- DIV(0, "dout_aclk100_noc", "mout_aclk100_noc", DIV_TOP0, 20, 3),
- DIV(0, "dout_pclk200_fsys", "mout_pclk200_fsys", DIV_TOP0, 24, 3),
- DIV(0, "dout_aclk200_fsys", "mout_aclk200_fsys", DIV_TOP0, 28, 3),
+ DIV(CLK_DOUT_ACLK400_ISP, "dout_aclk400_isp", "mout_aclk400_isp",
+ DIV_TOP0, 0, 3),
+ DIV(CLK_DOUT_ACLK400_MSCL, "dout_aclk400_mscl", "mout_aclk400_mscl",
+ DIV_TOP0, 4, 3),
+ DIV(CLK_DOUT_ACLK200, "dout_aclk200", "mout_aclk200",
+ DIV_TOP0, 8, 3),
+ DIV(CLK_DOUT_ACLK200_FSYS2, "dout_aclk200_fsys2", "mout_aclk200_fsys2",
+ DIV_TOP0, 12, 3),
+ DIV(CLK_DOUT_ACLK100_NOC, "dout_aclk100_noc", "mout_aclk100_noc",
+ DIV_TOP0, 20, 3),
+ DIV(CLK_DOUT_PCLK200_FSYS, "dout_pclk200_fsys", "mout_pclk200_fsys",
+ DIV_TOP0, 24, 3),
+ DIV(CLK_DOUT_ACLK200_FSYS, "dout_aclk200_fsys", "mout_aclk200_fsys",
+ DIV_TOP0, 28, 3),
+ DIV(CLK_DOUT_ACLK333_432_GSCL, "dout_aclk333_432_gscl",
+ "mout_aclk333_432_gscl", DIV_TOP1, 0, 3),
+ DIV(CLK_DOUT_ACLK333_432_ISP, "dout_aclk333_432_isp",
+ "mout_aclk333_432_isp", DIV_TOP1, 4, 3),
+ DIV(CLK_DOUT_ACLK66, "dout_aclk66", "mout_aclk66",
+ DIV_TOP1, 8, 6),
+ DIV(CLK_DOUT_ACLK333_432_ISP0, "dout_aclk333_432_isp0",
+ "mout_aclk333_432_isp0", DIV_TOP1, 16, 3),
+ DIV(CLK_DOUT_ACLK266, "dout_aclk266", "mout_aclk266",
+ DIV_TOP1, 20, 3),
+ DIV(CLK_DOUT_ACLK166, "dout_aclk166", "mout_aclk166",
+ DIV_TOP1, 24, 3),
+ DIV(CLK_DOUT_ACLK333, "dout_aclk333", "mout_aclk333",
+ DIV_TOP1, 28, 3),
- DIV(0, "dout_aclk333_432_gscl", "mout_aclk333_432_gscl",
- DIV_TOP1, 0, 3),
- DIV(0, "dout_aclk333_432_isp", "mout_aclk333_432_isp",
- DIV_TOP1, 4, 3),
- DIV(0, "dout_aclk66", "mout_aclk66", DIV_TOP1, 8, 6),
- DIV(0, "dout_aclk333_432_isp0", "mout_aclk333_432_isp0",
- DIV_TOP1, 16, 3),
- DIV(0, "dout_aclk266", "mout_aclk266", DIV_TOP1, 20, 3),
- DIV(0, "dout_aclk166", "mout_aclk166", DIV_TOP1, 24, 3),
- DIV(0, "dout_aclk333", "mout_aclk333", DIV_TOP1, 28, 3),
-
- DIV(0, "dout_aclk333_g2d", "mout_aclk333_g2d", DIV_TOP2, 8, 3),
- DIV(0, "dout_aclk266_g2d", "mout_aclk266_g2d", DIV_TOP2, 12, 3),
- DIV(0, "dout_aclk_g3d", "mout_aclk_g3d", DIV_TOP2, 16, 3),
- DIV(0, "dout_aclk300_jpeg", "mout_aclk300_jpeg", DIV_TOP2, 20, 3),
- DIV(0, "dout_aclk300_disp1", "mout_aclk300_disp1", DIV_TOP2, 24, 3),
- DIV(0, "dout_aclk300_gscl", "mout_aclk300_gscl", DIV_TOP2, 28, 3),
+ DIV(CLK_DOUT_ACLK333_G2D, "dout_aclk333_g2d", "mout_aclk333_g2d",
+ DIV_TOP2, 8, 3),
+ DIV(CLK_DOUT_ACLK266_G2D, "dout_aclk266_g2d", "mout_aclk266_g2d",
+ DIV_TOP2, 12, 3),
+ DIV(CLK_DOUT_ACLK_G3D, "dout_aclk_g3d", "mout_aclk_g3d", DIV_TOP2,
+ 16, 3),
+ DIV(CLK_DOUT_ACLK300_JPEG, "dout_aclk300_jpeg", "mout_aclk300_jpeg",
+ DIV_TOP2, 20, 3),
+ DIV(CLK_DOUT_ACLK300_DISP1, "dout_aclk300_disp1",
+ "mout_aclk300_disp1", DIV_TOP2, 24, 3),
+ DIV(CLK_DOUT_ACLK300_GSCL, "dout_aclk300_gscl", "mout_aclk300_gscl",
+ DIV_TOP2, 28, 3),
/* DISP1 Block */
DIV(0, "dout_fimd1", "mout_fimd1_final", DIV_DISP10, 0, 4),
@@ -817,7 +833,8 @@
DIV(0, "dout_dp1", "mout_dp1", DIV_DISP10, 24, 4),
DIV(CLK_DOUT_PIXEL, "dout_hdmi_pixel", "mout_pixel", DIV_DISP10, 28, 4),
DIV(0, "dout_disp1_blk", "aclk200_disp1", DIV2_RATIO0, 16, 2),
- DIV(0, "dout_aclk400_disp1", "mout_aclk400_disp1", DIV_TOP2, 4, 3),
+ DIV(CLK_DOUT_ACLK400_DISP1, "dout_aclk400_disp1",
+ "mout_aclk400_disp1", DIV_TOP2, 4, 3),
/* Audio Block */
DIV(0, "dout_maudio0", "mout_maudio0", DIV_MAU, 20, 4),
diff --git a/drivers/clk/sirf/clk-atlas6.c b/drivers/clk/sirf/clk-atlas6.c
index c5eaa9d..665fa68 100644
--- a/drivers/clk/sirf/clk-atlas6.c
+++ b/drivers/clk/sirf/clk-atlas6.c
@@ -130,10 +130,9 @@
panic("unable to map clkc registers\n");
/* These are always available (RTC and 26MHz OSC)*/
- atlas6_clks[rtc] = clk_register_fixed_rate(NULL, "rtc", NULL,
- CLK_IS_ROOT, 32768);
- atlas6_clks[osc] = clk_register_fixed_rate(NULL, "osc", NULL,
- CLK_IS_ROOT, 26000000);
+ atlas6_clks[rtc] = clk_register_fixed_rate(NULL, "rtc", NULL, 0, 32768);
+ atlas6_clks[osc] = clk_register_fixed_rate(NULL, "osc", NULL, 0,
+ 26000000);
for (i = pll1; i < maxclk; i++) {
atlas6_clks[i] = clk_register(NULL, atlas6_clk_hw_array[i]);
diff --git a/drivers/clk/sirf/clk-prima2.c b/drivers/clk/sirf/clk-prima2.c
index f92c402..aac1c8ec 100644
--- a/drivers/clk/sirf/clk-prima2.c
+++ b/drivers/clk/sirf/clk-prima2.c
@@ -129,10 +129,9 @@
panic("unable to map clkc registers\n");
/* These are always available (RTC and 26MHz OSC)*/
- prima2_clks[rtc] = clk_register_fixed_rate(NULL, "rtc", NULL,
- CLK_IS_ROOT, 32768);
- prima2_clks[osc] = clk_register_fixed_rate(NULL, "osc", NULL,
- CLK_IS_ROOT, 26000000);
+ prima2_clks[rtc] = clk_register_fixed_rate(NULL, "rtc", NULL, 0, 32768);
+ prima2_clks[osc] = clk_register_fixed_rate(NULL, "osc", NULL, 0,
+ 26000000);
for (i = pll1; i < maxclk; i++) {
prima2_clks[i] = clk_register(NULL, prima2_clk_hw_array[i]);
diff --git a/drivers/clk/sunxi/Makefile b/drivers/clk/sunxi/Makefile
index 3fd7901..39d2044 100644
--- a/drivers/clk/sunxi/Makefile
+++ b/drivers/clk/sunxi/Makefile
@@ -11,6 +11,9 @@
obj-y += clk-a20-gmac.o
obj-y += clk-mod0.o
obj-y += clk-simple-gates.o
+obj-y += clk-sun4i-display.o
+obj-y += clk-sun4i-pll3.o
+obj-y += clk-sun4i-tcon-ch1.o
obj-y += clk-sun8i-bus-gates.o
obj-y += clk-sun8i-mbus.o
obj-y += clk-sun9i-core.o
diff --git a/drivers/clk/sunxi/clk-a10-hosc.c b/drivers/clk/sunxi/clk-a10-hosc.c
index 6b598c6..dca5324 100644
--- a/drivers/clk/sunxi/clk-a10-hosc.c
+++ b/drivers/clk/sunxi/clk-a10-hosc.c
@@ -54,8 +54,7 @@
NULL, 0,
NULL, NULL,
&fixed->hw, &clk_fixed_rate_ops,
- &gate->hw, &clk_gate_ops,
- CLK_IS_ROOT);
+ &gate->hw, &clk_gate_ops, 0);
if (IS_ERR(clk))
goto err_free_gate;
diff --git a/drivers/clk/sunxi/clk-a10-mod1.c b/drivers/clk/sunxi/clk-a10-mod1.c
index e9d870d..e2819fa 100644
--- a/drivers/clk/sunxi/clk-a10-mod1.c
+++ b/drivers/clk/sunxi/clk-a10-mod1.c
@@ -62,7 +62,7 @@
clk = clk_register_composite(NULL, clk_name, parents, i,
&mux->hw, &clk_mux_ops,
NULL, NULL,
- &gate->hw, &clk_gate_ops, 0);
+ &gate->hw, &clk_gate_ops, CLK_SET_RATE_PARENT);
if (IS_ERR(clk))
goto err_free_gate;
diff --git a/drivers/clk/sunxi/clk-sun4i-display.c b/drivers/clk/sunxi/clk-sun4i-display.c
new file mode 100644
index 0000000..445a749
--- /dev/null
+++ b/drivers/clk/sunxi/clk-sun4i-display.c
@@ -0,0 +1,261 @@
+/*
+ * Copyright 2015 Maxime Ripard
+ *
+ * Maxime Ripard <maxime.ripard@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/clk-provider.h>
+#include <linux/kernel.h>
+#include <linux/of_address.h>
+#include <linux/reset-controller.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+
+struct sun4i_a10_display_clk_data {
+ bool has_div;
+ u8 num_rst;
+ u8 parents;
+
+ u8 offset_en;
+ u8 offset_div;
+ u8 offset_mux;
+ u8 offset_rst;
+
+ u8 width_div;
+ u8 width_mux;
+};
+
+struct reset_data {
+ void __iomem *reg;
+ spinlock_t *lock;
+ struct reset_controller_dev rcdev;
+ u8 offset;
+};
+
+static DEFINE_SPINLOCK(sun4i_a10_display_lock);
+
+static inline struct reset_data *rcdev_to_reset_data(struct reset_controller_dev *rcdev)
+{
+ return container_of(rcdev, struct reset_data, rcdev);
+};
+
+static int sun4i_a10_display_assert(struct reset_controller_dev *rcdev,
+ unsigned long id)
+{
+ struct reset_data *data = rcdev_to_reset_data(rcdev);
+ unsigned long flags;
+ u32 reg;
+
+ spin_lock_irqsave(data->lock, flags);
+
+ reg = readl(data->reg);
+ writel(reg & ~BIT(data->offset + id), data->reg);
+
+ spin_unlock_irqrestore(data->lock, flags);
+
+ return 0;
+}
+
+static int sun4i_a10_display_deassert(struct reset_controller_dev *rcdev,
+ unsigned long id)
+{
+ struct reset_data *data = rcdev_to_reset_data(rcdev);
+ unsigned long flags;
+ u32 reg;
+
+ spin_lock_irqsave(data->lock, flags);
+
+ reg = readl(data->reg);
+ writel(reg | BIT(data->offset + id), data->reg);
+
+ spin_unlock_irqrestore(data->lock, flags);
+
+ return 0;
+}
+
+static int sun4i_a10_display_status(struct reset_controller_dev *rcdev,
+ unsigned long id)
+{
+ struct reset_data *data = rcdev_to_reset_data(rcdev);
+
+ return !(readl(data->reg) & BIT(data->offset + id));
+}
+
+static const struct reset_control_ops sun4i_a10_display_reset_ops = {
+ .assert = sun4i_a10_display_assert,
+ .deassert = sun4i_a10_display_deassert,
+ .status = sun4i_a10_display_status,
+};
+
+static int sun4i_a10_display_reset_xlate(struct reset_controller_dev *rcdev,
+ const struct of_phandle_args *spec)
+{
+ /* We only have a single reset signal */
+ return 0;
+}
+
+static void __init sun4i_a10_display_init(struct device_node *node,
+ const struct sun4i_a10_display_clk_data *data)
+{
+ const char *parents[4];
+ const char *clk_name = node->name;
+ struct reset_data *reset_data;
+ struct clk_divider *div = NULL;
+ struct clk_gate *gate;
+ struct resource res;
+ struct clk_mux *mux;
+ void __iomem *reg;
+ struct clk *clk;
+ int ret;
+
+ of_property_read_string(node, "clock-output-names", &clk_name);
+
+ reg = of_io_request_and_map(node, 0, of_node_full_name(node));
+ if (IS_ERR(reg)) {
+ pr_err("%s: Could not map the clock registers\n", clk_name);
+ return;
+ }
+
+ ret = of_clk_parent_fill(node, parents, data->parents);
+ if (ret != data->parents) {
+ pr_err("%s: Could not retrieve the parents\n", clk_name);
+ goto unmap;
+ }
+
+ mux = kzalloc(sizeof(*mux), GFP_KERNEL);
+ if (!mux)
+ goto unmap;
+
+ mux->reg = reg;
+ mux->shift = data->offset_mux;
+ mux->mask = (1 << data->width_mux) - 1;
+ mux->lock = &sun4i_a10_display_lock;
+
+ gate = kzalloc(sizeof(*gate), GFP_KERNEL);
+ if (!gate)
+ goto free_mux;
+
+ gate->reg = reg;
+ gate->bit_idx = data->offset_en;
+ gate->lock = &sun4i_a10_display_lock;
+
+ if (data->has_div) {
+ div = kzalloc(sizeof(*div), GFP_KERNEL);
+ if (!div)
+ goto free_gate;
+
+ div->reg = reg;
+ div->shift = data->offset_div;
+ div->width = data->width_div;
+ div->lock = &sun4i_a10_display_lock;
+ }
+
+ clk = clk_register_composite(NULL, clk_name,
+ parents, data->parents,
+ &mux->hw, &clk_mux_ops,
+ data->has_div ? &div->hw : NULL,
+ data->has_div ? &clk_divider_ops : NULL,
+ &gate->hw, &clk_gate_ops,
+ 0);
+ if (IS_ERR(clk)) {
+ pr_err("%s: Couldn't register the clock\n", clk_name);
+ goto free_div;
+ }
+
+ ret = of_clk_add_provider(node, of_clk_src_simple_get, clk);
+ if (ret) {
+ pr_err("%s: Couldn't register DT provider\n", clk_name);
+ goto free_clk;
+ }
+
+ if (!data->num_rst)
+ return;
+
+ reset_data = kzalloc(sizeof(*reset_data), GFP_KERNEL);
+ if (!reset_data)
+ goto free_of_clk;
+
+ reset_data->reg = reg;
+ reset_data->offset = data->offset_rst;
+ reset_data->lock = &sun4i_a10_display_lock;
+ reset_data->rcdev.nr_resets = data->num_rst;
+ reset_data->rcdev.ops = &sun4i_a10_display_reset_ops;
+ reset_data->rcdev.of_node = node;
+
+ if (data->num_rst == 1) {
+ reset_data->rcdev.of_reset_n_cells = 0;
+ reset_data->rcdev.of_xlate = &sun4i_a10_display_reset_xlate;
+ } else {
+ reset_data->rcdev.of_reset_n_cells = 1;
+ }
+
+ if (reset_controller_register(&reset_data->rcdev)) {
+ pr_err("%s: Couldn't register the reset controller\n",
+ clk_name);
+ goto free_reset;
+ }
+
+ return;
+
+free_reset:
+ kfree(reset_data);
+free_of_clk:
+ of_clk_del_provider(node);
+free_clk:
+ clk_unregister_composite(clk);
+free_div:
+ kfree(div);
+free_gate:
+ kfree(gate);
+free_mux:
+ kfree(mux);
+unmap:
+ iounmap(reg);
+ of_address_to_resource(node, 0, &res);
+ release_mem_region(res.start, resource_size(&res));
+}
+
+static const struct sun4i_a10_display_clk_data sun4i_a10_tcon_ch0_data __initconst = {
+ .num_rst = 2,
+ .parents = 4,
+ .offset_en = 31,
+ .offset_rst = 29,
+ .offset_mux = 24,
+ .width_mux = 2,
+};
+
+static void __init sun4i_a10_tcon_ch0_setup(struct device_node *node)
+{
+ sun4i_a10_display_init(node, &sun4i_a10_tcon_ch0_data);
+}
+CLK_OF_DECLARE(sun4i_a10_tcon_ch0, "allwinner,sun4i-a10-tcon-ch0-clk",
+ sun4i_a10_tcon_ch0_setup);
+
+static const struct sun4i_a10_display_clk_data sun4i_a10_display_data __initconst = {
+ .has_div = true,
+ .num_rst = 1,
+ .parents = 3,
+ .offset_en = 31,
+ .offset_rst = 30,
+ .offset_mux = 24,
+ .offset_div = 0,
+ .width_mux = 2,
+ .width_div = 4,
+};
+
+static void __init sun4i_a10_display_setup(struct device_node *node)
+{
+ sun4i_a10_display_init(node, &sun4i_a10_display_data);
+}
+CLK_OF_DECLARE(sun4i_a10_display, "allwinner,sun4i-a10-display-clk",
+ sun4i_a10_display_setup);
diff --git a/drivers/clk/sunxi/clk-sun4i-pll3.c b/drivers/clk/sunxi/clk-sun4i-pll3.c
new file mode 100644
index 0000000..f66267e
--- /dev/null
+++ b/drivers/clk/sunxi/clk-sun4i-pll3.c
@@ -0,0 +1,98 @@
+/*
+ * Copyright 2015 Maxime Ripard
+ *
+ * Maxime Ripard <maxime.ripard@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/clk-provider.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+
+#define SUN4I_A10_PLL3_GATE_BIT 31
+#define SUN4I_A10_PLL3_DIV_WIDTH 7
+#define SUN4I_A10_PLL3_DIV_SHIFT 0
+
+static DEFINE_SPINLOCK(sun4i_a10_pll3_lock);
+
+static void __init sun4i_a10_pll3_setup(struct device_node *node)
+{
+ const char *clk_name = node->name, *parent;
+ struct clk_multiplier *mult;
+ struct clk_gate *gate;
+ struct resource res;
+ void __iomem *reg;
+ struct clk *clk;
+ int ret;
+
+ of_property_read_string(node, "clock-output-names", &clk_name);
+ parent = of_clk_get_parent_name(node, 0);
+
+ reg = of_io_request_and_map(node, 0, of_node_full_name(node));
+ if (IS_ERR(reg)) {
+ pr_err("%s: Could not map the clock registers\n", clk_name);
+ return;
+ }
+
+ gate = kzalloc(sizeof(*gate), GFP_KERNEL);
+ if (!gate)
+ goto err_unmap;
+
+ gate->reg = reg;
+ gate->bit_idx = SUN4I_A10_PLL3_GATE_BIT;
+ gate->lock = &sun4i_a10_pll3_lock;
+
+ mult = kzalloc(sizeof(*mult), GFP_KERNEL);
+ if (!mult)
+ goto err_free_gate;
+
+ mult->reg = reg;
+ mult->shift = SUN4I_A10_PLL3_DIV_SHIFT;
+ mult->width = SUN4I_A10_PLL3_DIV_WIDTH;
+ mult->lock = &sun4i_a10_pll3_lock;
+
+ clk = clk_register_composite(NULL, clk_name,
+ &parent, 1,
+ NULL, NULL,
+ &mult->hw, &clk_multiplier_ops,
+ &gate->hw, &clk_gate_ops,
+ 0);
+ if (IS_ERR(clk)) {
+ pr_err("%s: Couldn't register the clock\n", clk_name);
+ goto err_free_mult;
+ }
+
+ ret = of_clk_add_provider(node, of_clk_src_simple_get, clk);
+ if (ret) {
+ pr_err("%s: Couldn't register DT provider\n",
+ clk_name);
+ goto err_clk_unregister;
+ }
+
+ return;
+
+err_clk_unregister:
+ clk_unregister_composite(clk);
+err_free_mult:
+ kfree(mult);
+err_free_gate:
+ kfree(gate);
+err_unmap:
+ iounmap(reg);
+ of_address_to_resource(node, 0, &res);
+ release_mem_region(res.start, resource_size(&res));
+}
+
+CLK_OF_DECLARE(sun4i_a10_pll3, "allwinner,sun4i-a10-pll3-clk",
+ sun4i_a10_pll3_setup);
diff --git a/drivers/clk/sunxi/clk-sun4i-tcon-ch1.c b/drivers/clk/sunxi/clk-sun4i-tcon-ch1.c
new file mode 100644
index 0000000..98a4582
--- /dev/null
+++ b/drivers/clk/sunxi/clk-sun4i-tcon-ch1.c
@@ -0,0 +1,300 @@
+/*
+ * Copyright 2015 Maxime Ripard
+ *
+ * Maxime Ripard <maxime.ripard@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/clk-provider.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+
+#define TCON_CH1_SCLK2_PARENTS 4
+
+#define TCON_CH1_SCLK2_GATE_BIT BIT(31)
+#define TCON_CH1_SCLK2_MUX_MASK 3
+#define TCON_CH1_SCLK2_MUX_SHIFT 24
+#define TCON_CH1_SCLK2_DIV_MASK 0xf
+#define TCON_CH1_SCLK2_DIV_SHIFT 0
+
+#define TCON_CH1_SCLK1_GATE_BIT BIT(15)
+#define TCON_CH1_SCLK1_HALF_BIT BIT(11)
+
+struct tcon_ch1_clk {
+ struct clk_hw hw;
+ spinlock_t lock;
+ void __iomem *reg;
+};
+
+#define hw_to_tclk(hw) container_of(hw, struct tcon_ch1_clk, hw)
+
+static void tcon_ch1_disable(struct clk_hw *hw)
+{
+ struct tcon_ch1_clk *tclk = hw_to_tclk(hw);
+ unsigned long flags;
+ u32 reg;
+
+ spin_lock_irqsave(&tclk->lock, flags);
+ reg = readl(tclk->reg);
+ reg &= ~(TCON_CH1_SCLK2_GATE_BIT | TCON_CH1_SCLK1_GATE_BIT);
+ writel(reg, tclk->reg);
+ spin_unlock_irqrestore(&tclk->lock, flags);
+}
+
+static int tcon_ch1_enable(struct clk_hw *hw)
+{
+ struct tcon_ch1_clk *tclk = hw_to_tclk(hw);
+ unsigned long flags;
+ u32 reg;
+
+ spin_lock_irqsave(&tclk->lock, flags);
+ reg = readl(tclk->reg);
+ reg |= TCON_CH1_SCLK2_GATE_BIT | TCON_CH1_SCLK1_GATE_BIT;
+ writel(reg, tclk->reg);
+ spin_unlock_irqrestore(&tclk->lock, flags);
+
+ return 0;
+}
+
+static int tcon_ch1_is_enabled(struct clk_hw *hw)
+{
+ struct tcon_ch1_clk *tclk = hw_to_tclk(hw);
+ u32 reg;
+
+ reg = readl(tclk->reg);
+ return reg & (TCON_CH1_SCLK2_GATE_BIT | TCON_CH1_SCLK1_GATE_BIT);
+}
+
+static u8 tcon_ch1_get_parent(struct clk_hw *hw)
+{
+ struct tcon_ch1_clk *tclk = hw_to_tclk(hw);
+ int num_parents = clk_hw_get_num_parents(hw);
+ u32 reg;
+
+ reg = readl(tclk->reg) >> TCON_CH1_SCLK2_MUX_SHIFT;
+ reg &= reg >> TCON_CH1_SCLK2_MUX_MASK;
+
+ if (reg >= num_parents)
+ return -EINVAL;
+
+ return reg;
+}
+
+static int tcon_ch1_set_parent(struct clk_hw *hw, u8 index)
+{
+ struct tcon_ch1_clk *tclk = hw_to_tclk(hw);
+ unsigned long flags;
+ u32 reg;
+
+ spin_lock_irqsave(&tclk->lock, flags);
+ reg = readl(tclk->reg);
+ reg &= ~(TCON_CH1_SCLK2_MUX_MASK << TCON_CH1_SCLK2_MUX_SHIFT);
+ reg |= index << TCON_CH1_SCLK2_MUX_SHIFT;
+ writel(reg, tclk->reg);
+ spin_unlock_irqrestore(&tclk->lock, flags);
+
+ return 0;
+};
+
+static unsigned long tcon_ch1_calc_divider(unsigned long rate,
+ unsigned long parent_rate,
+ u8 *div,
+ bool *half)
+{
+ unsigned long best_rate = 0;
+ u8 best_m = 0, m;
+ bool is_double;
+
+ for (m = 1; m < 16; m++) {
+ u8 d;
+
+ for (d = 1; d < 3; d++) {
+ unsigned long tmp_rate;
+
+ tmp_rate = parent_rate / m / d;
+
+ if (tmp_rate > rate)
+ continue;
+
+ if (!best_rate ||
+ (rate - tmp_rate) < (rate - best_rate)) {
+ best_rate = tmp_rate;
+ best_m = m;
+ is_double = d;
+ }
+ }
+ }
+
+ if (div && half) {
+ *div = best_m;
+ *half = is_double;
+ }
+
+ return best_rate;
+}
+
+static int tcon_ch1_determine_rate(struct clk_hw *hw,
+ struct clk_rate_request *req)
+{
+ long best_rate = -EINVAL;
+ int i;
+
+ for (i = 0; i < clk_hw_get_num_parents(hw); i++) {
+ unsigned long parent_rate;
+ unsigned long tmp_rate;
+ struct clk_hw *parent;
+
+ parent = clk_hw_get_parent_by_index(hw, i);
+ if (!parent)
+ continue;
+
+ parent_rate = clk_hw_get_rate(parent);
+
+ tmp_rate = tcon_ch1_calc_divider(req->rate, parent_rate,
+ NULL, NULL);
+
+ if (best_rate < 0 ||
+ (req->rate - tmp_rate) < (req->rate - best_rate)) {
+ best_rate = tmp_rate;
+ req->best_parent_rate = parent_rate;
+ req->best_parent_hw = parent;
+ }
+ }
+
+ if (best_rate < 0)
+ return best_rate;
+
+ req->rate = best_rate;
+ return 0;
+}
+
+static unsigned long tcon_ch1_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct tcon_ch1_clk *tclk = hw_to_tclk(hw);
+ u32 reg;
+
+ reg = readl(tclk->reg);
+
+ parent_rate /= (reg & TCON_CH1_SCLK2_DIV_MASK) + 1;
+
+ if (reg & TCON_CH1_SCLK1_HALF_BIT)
+ parent_rate /= 2;
+
+ return parent_rate;
+}
+
+static int tcon_ch1_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct tcon_ch1_clk *tclk = hw_to_tclk(hw);
+ unsigned long flags;
+ bool half;
+ u8 div_m;
+ u32 reg;
+
+ tcon_ch1_calc_divider(rate, parent_rate, &div_m, &half);
+
+ spin_lock_irqsave(&tclk->lock, flags);
+ reg = readl(tclk->reg);
+ reg &= ~(TCON_CH1_SCLK2_DIV_MASK | TCON_CH1_SCLK1_HALF_BIT);
+ reg |= (div_m - 1) & TCON_CH1_SCLK2_DIV_MASK;
+
+ if (half)
+ reg |= TCON_CH1_SCLK1_HALF_BIT;
+
+ writel(reg, tclk->reg);
+ spin_unlock_irqrestore(&tclk->lock, flags);
+
+ return 0;
+}
+
+static const struct clk_ops tcon_ch1_ops = {
+ .disable = tcon_ch1_disable,
+ .enable = tcon_ch1_enable,
+ .is_enabled = tcon_ch1_is_enabled,
+
+ .get_parent = tcon_ch1_get_parent,
+ .set_parent = tcon_ch1_set_parent,
+
+ .determine_rate = tcon_ch1_determine_rate,
+ .recalc_rate = tcon_ch1_recalc_rate,
+ .set_rate = tcon_ch1_set_rate,
+};
+
+static void __init tcon_ch1_setup(struct device_node *node)
+{
+ const char *parents[TCON_CH1_SCLK2_PARENTS];
+ const char *clk_name = node->name;
+ struct clk_init_data init;
+ struct tcon_ch1_clk *tclk;
+ struct resource res;
+ struct clk *clk;
+ void __iomem *reg;
+ int ret;
+
+ of_property_read_string(node, "clock-output-names", &clk_name);
+
+ reg = of_io_request_and_map(node, 0, of_node_full_name(node));
+ if (IS_ERR(reg)) {
+ pr_err("%s: Could not map the clock registers\n", clk_name);
+ return;
+ }
+
+ ret = of_clk_parent_fill(node, parents, TCON_CH1_SCLK2_PARENTS);
+ if (ret != TCON_CH1_SCLK2_PARENTS) {
+ pr_err("%s Could not retrieve the parents\n", clk_name);
+ goto err_unmap;
+ }
+
+ tclk = kzalloc(sizeof(*tclk), GFP_KERNEL);
+ if (!tclk)
+ goto err_unmap;
+
+ init.name = clk_name;
+ init.ops = &tcon_ch1_ops;
+ init.parent_names = parents;
+ init.num_parents = TCON_CH1_SCLK2_PARENTS;
+ init.flags = CLK_SET_RATE_PARENT;
+
+ tclk->reg = reg;
+ tclk->hw.init = &init;
+ spin_lock_init(&tclk->lock);
+
+ clk = clk_register(NULL, &tclk->hw);
+ if (IS_ERR(clk)) {
+ pr_err("%s: Couldn't register the clock\n", clk_name);
+ goto err_free_data;
+ }
+
+ ret = of_clk_add_provider(node, of_clk_src_simple_get, clk);
+ if (ret) {
+ pr_err("%s: Couldn't register our clock provider\n", clk_name);
+ goto err_unregister_clk;
+ }
+
+ return;
+
+err_unregister_clk:
+ clk_unregister(clk);
+err_free_data:
+ kfree(tclk);
+err_unmap:
+ iounmap(reg);
+ of_address_to_resource(node, 0, &res);
+ release_mem_region(res.start, resource_size(&res));
+}
+
+CLK_OF_DECLARE(tcon_ch1, "allwinner,sun4i-a10-tcon-ch1-clk",
+ tcon_ch1_setup);
diff --git a/drivers/clk/sunxi/clk-sun9i-mmc.c b/drivers/clk/sunxi/clk-sun9i-mmc.c
index 028dd83..7167373 100644
--- a/drivers/clk/sunxi/clk-sun9i-mmc.c
+++ b/drivers/clk/sunxi/clk-sun9i-mmc.c
@@ -106,7 +106,7 @@
r = platform_get_resource(pdev, IORESOURCE_MEM, 0);
/* one clock/reset pair per word */
- count = DIV_ROUND_UP((r->end - r->start + 1), SUN9I_MMC_WIDTH);
+ count = DIV_ROUND_UP((resource_size(r)), SUN9I_MMC_WIDTH);
data->membase = devm_ioremap_resource(&pdev->dev, r);
if (IS_ERR(data->membase))
return PTR_ERR(data->membase);
diff --git a/drivers/clk/sunxi/clk-sunxi.c b/drivers/clk/sunxi/clk-sunxi.c
index 91de0a0..838b22a 100644
--- a/drivers/clk/sunxi/clk-sunxi.c
+++ b/drivers/clk/sunxi/clk-sunxi.c
@@ -523,21 +523,12 @@
.enable = 31,
.table = &sun4i_pll5_config,
.getter = sun4i_get_pll5_factors,
- .name = "pll5",
-};
-
-static const struct factors_data sun4i_pll6_data __initconst = {
- .enable = 31,
- .table = &sun4i_pll5_config,
- .getter = sun4i_get_pll5_factors,
- .name = "pll6",
};
static const struct factors_data sun6i_a31_pll6_data __initconst = {
.enable = 31,
.table = &sun6i_a31_pll6_config,
.getter = sun6i_a31_get_pll6_factors,
- .name = "pll6x2",
};
static const struct factors_data sun5i_a13_ahb_data __initconst = {
@@ -933,7 +924,7 @@
};
static const struct divs_data pll6_divs_data __initconst = {
- .factors = &sun4i_pll6_data,
+ .factors = &sun4i_pll5_data,
.ndivs = 4,
.div = {
{ .shift = 0, .table = pll6_sata_tbl, .gate = 14 }, /* M, SATA */
@@ -975,6 +966,8 @@
struct clk_gate *gate = NULL;
struct clk_fixed_factor *fix_factor;
struct clk_divider *divider;
+ struct factors_data factors = *data->factors;
+ char *derived_name = NULL;
void __iomem *reg;
int ndivs = SUNXI_DIVS_MAX_QTY, i = 0;
int flags, clkflags;
@@ -983,11 +976,37 @@
if (data->ndivs)
ndivs = data->ndivs;
+ /* Try to find a name for base factor clock */
+ for (i = 0; i < ndivs; i++) {
+ if (data->div[i].self) {
+ of_property_read_string_index(node, "clock-output-names",
+ i, &factors.name);
+ break;
+ }
+ }
+ /* If we don't have a .self clk use the first output-name up to '_' */
+ if (factors.name == NULL) {
+ char *endp;
+
+ of_property_read_string_index(node, "clock-output-names",
+ 0, &clk_name);
+ endp = strchr(clk_name, '_');
+ if (endp) {
+ derived_name = kstrndup(clk_name, endp - clk_name,
+ GFP_KERNEL);
+ factors.name = derived_name;
+ } else {
+ factors.name = clk_name;
+ }
+ }
+
/* Set up factor clock that we will be dividing */
- pclk = sunxi_factors_clk_setup(node, data->factors);
+ pclk = sunxi_factors_clk_setup(node, &factors);
if (!pclk)
return NULL;
+
parent = __clk_get_name(pclk);
+ kfree(derived_name);
reg = of_iomap(node, 0);
if (!reg) {
@@ -1127,3 +1146,41 @@
}
CLK_OF_DECLARE(sun6i_pll6, "allwinner,sun6i-a31-pll6-clk",
sun6i_pll6_clk_setup);
+
+/*
+ * sun6i display
+ *
+ * rate = parent_rate / (m + 1);
+ */
+static void sun6i_display_factors(struct factors_request *req)
+{
+ u8 m;
+
+ if (req->rate > req->parent_rate)
+ req->rate = req->parent_rate;
+
+ m = DIV_ROUND_UP(req->parent_rate, req->rate);
+
+ req->rate = req->parent_rate / m;
+ req->m = m - 1;
+}
+
+static const struct clk_factors_config sun6i_display_config = {
+ .mshift = 0,
+ .mwidth = 4,
+};
+
+static const struct factors_data sun6i_display_data __initconst = {
+ .enable = 31,
+ .mux = 24,
+ .muxmask = BIT(2) | BIT(1) | BIT(0),
+ .table = &sun6i_display_config,
+ .getter = sun6i_display_factors,
+};
+
+static void __init sun6i_display_setup(struct device_node *node)
+{
+ sunxi_factors_clk_setup(node, &sun6i_display_data);
+}
+CLK_OF_DECLARE(sun6i_display, "allwinner,sun6i-a31-display-clk",
+ sun6i_display_setup);
diff --git a/drivers/clk/tegra/Makefile b/drivers/clk/tegra/Makefile
index 97984c5..33fd093 100644
--- a/drivers/clk/tegra/Makefile
+++ b/drivers/clk/tegra/Makefile
@@ -3,6 +3,7 @@
obj-y += clk-dfll.o
obj-y += clk-divider.o
obj-y += clk-periph.o
+obj-y += clk-periph-fixed.o
obj-y += clk-periph-gate.o
obj-y += clk-pll.o
obj-y += clk-pll-out.o
diff --git a/drivers/clk/tegra/clk-dfll.c b/drivers/clk/tegra/clk-dfll.c
index 19bfa07..f010562 100644
--- a/drivers/clk/tegra/clk-dfll.c
+++ b/drivers/clk/tegra/clk-dfll.c
@@ -55,6 +55,7 @@
#include <linux/seq_file.h>
#include "clk-dfll.h"
+#include "cvb.h"
/*
* DFLL control registers - access via dfll_{readl,writel}
@@ -442,8 +443,8 @@
{
td->tune_range = DFLL_TUNE_LOW;
- dfll_writel(td, td->soc->tune0_low, DFLL_TUNE0);
- dfll_writel(td, td->soc->tune1, DFLL_TUNE1);
+ dfll_writel(td, td->soc->cvb->cpu_dfll_data.tune0_low, DFLL_TUNE0);
+ dfll_writel(td, td->soc->cvb->cpu_dfll_data.tune1, DFLL_TUNE1);
dfll_wmb(td);
if (td->soc->set_clock_trimmers_low)
@@ -1449,7 +1450,7 @@
}
v_max = dev_pm_opp_get_voltage(opp);
- v = td->soc->min_millivolts * 1000;
+ v = td->soc->cvb->min_millivolts * 1000;
lut = find_vdd_map_entry_exact(td, v);
if (lut < 0)
goto out;
@@ -1461,7 +1462,7 @@
break;
v_opp = dev_pm_opp_get_voltage(opp);
- if (v_opp <= td->soc->min_millivolts * 1000)
+ if (v_opp <= td->soc->cvb->min_millivolts * 1000)
td->dvco_rate_min = dev_pm_opp_get_freq(opp);
for (;;) {
@@ -1490,7 +1491,7 @@
if (!td->dvco_rate_min)
dev_err(td->dev, "no opp above DFLL minimum voltage %d mV\n",
- td->soc->min_millivolts);
+ td->soc->cvb->min_millivolts);
else
ret = 0;
diff --git a/drivers/clk/tegra/clk-dfll.h b/drivers/clk/tegra/clk-dfll.h
index 2e4c077..ed2ad88 100644
--- a/drivers/clk/tegra/clk-dfll.h
+++ b/drivers/clk/tegra/clk-dfll.h
@@ -24,22 +24,18 @@
/**
* struct tegra_dfll_soc_data - SoC-specific hooks/integration for the DFLL driver
- * @opp_dev: struct device * that holds the OPP table for the DFLL
- * @min_millivolts: minimum voltage (in mV) that the DFLL can operate
- * @tune0_low: DFLL tuning register 0 (low voltage range)
- * @tune0_high: DFLL tuning register 0 (high voltage range)
- * @tune1: DFLL tuning register 1
- * @assert_dvco_reset: fn ptr to place the DVCO in reset
- * @deassert_dvco_reset: fn ptr to release the DVCO reset
- * @set_clock_trimmers_high: fn ptr to tune clock trimmers for high voltage
- * @set_clock_trimmers_low: fn ptr to tune clock trimmers for low voltage
+ * @dev: struct device * that holds the OPP table for the DFLL
+ * @max_freq: maximum frequency supported on this SoC
+ * @cvb: CPU frequency table for this SoC
+ * @init_clock_trimmers: callback to initialize clock trimmers
+ * @set_clock_trimmers_high: callback to tune clock trimmers for high voltage
+ * @set_clock_trimmers_low: callback to tune clock trimmers for low voltage
*/
struct tegra_dfll_soc_data {
struct device *dev;
- unsigned int min_millivolts;
- u32 tune0_low;
- u32 tune0_high;
- u32 tune1;
+ unsigned long max_freq;
+ const struct cvb_table *cvb;
+
void (*init_clock_trimmers)(void);
void (*set_clock_trimmers_high)(void);
void (*set_clock_trimmers_low)(void);
diff --git a/drivers/clk/tegra/clk-id.h b/drivers/clk/tegra/clk-id.h
index 62ea381..36c9749 100644
--- a/drivers/clk/tegra/clk-id.h
+++ b/drivers/clk/tegra/clk-id.h
@@ -71,6 +71,7 @@
tegra_clk_disp2_8,
tegra_clk_dp2,
tegra_clk_dpaux,
+ tegra_clk_dpaux1,
tegra_clk_dsialp,
tegra_clk_dsia_mux,
tegra_clk_dsiblp,
@@ -306,6 +307,7 @@
tegra_clk_xusb_ss_div2,
tegra_clk_xusb_ssp_src,
tegra_clk_sclk_mux,
+ tegra_clk_sor_safe,
tegra_clk_max,
};
diff --git a/drivers/clk/tegra/clk-periph-fixed.c b/drivers/clk/tegra/clk-periph-fixed.c
new file mode 100644
index 0000000..c57dfb0
--- /dev/null
+++ b/drivers/clk/tegra/clk-periph-fixed.c
@@ -0,0 +1,120 @@
+/*
+ * Copyright (c) 2015, NVIDIA CORPORATION. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/clk-provider.h>
+
+#include "clk.h"
+
+static inline struct tegra_clk_periph_fixed *
+to_tegra_clk_periph_fixed(struct clk_hw *hw)
+{
+ return container_of(hw, struct tegra_clk_periph_fixed, hw);
+}
+
+static int tegra_clk_periph_fixed_is_enabled(struct clk_hw *hw)
+{
+ struct tegra_clk_periph_fixed *fixed = to_tegra_clk_periph_fixed(hw);
+ u32 mask = 1 << (fixed->num % 32), value;
+
+ value = readl(fixed->base + fixed->regs->enb_reg);
+ if (value & mask) {
+ value = readl(fixed->base + fixed->regs->rst_reg);
+ if ((value & mask) == 0)
+ return 1;
+ }
+
+ return 0;
+}
+
+static int tegra_clk_periph_fixed_enable(struct clk_hw *hw)
+{
+ struct tegra_clk_periph_fixed *fixed = to_tegra_clk_periph_fixed(hw);
+ u32 mask = 1 << (fixed->num % 32);
+
+ writel(mask, fixed->base + fixed->regs->enb_set_reg);
+
+ return 0;
+}
+
+static void tegra_clk_periph_fixed_disable(struct clk_hw *hw)
+{
+ struct tegra_clk_periph_fixed *fixed = to_tegra_clk_periph_fixed(hw);
+ u32 mask = 1 << (fixed->num % 32);
+
+ writel(mask, fixed->base + fixed->regs->enb_clr_reg);
+}
+
+static unsigned long
+tegra_clk_periph_fixed_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct tegra_clk_periph_fixed *fixed = to_tegra_clk_periph_fixed(hw);
+ unsigned long long rate;
+
+ rate = (unsigned long long)parent_rate * fixed->mul;
+ do_div(rate, fixed->div);
+
+ return (unsigned long)rate;
+}
+
+static const struct clk_ops tegra_clk_periph_fixed_ops = {
+ .is_enabled = tegra_clk_periph_fixed_is_enabled,
+ .enable = tegra_clk_periph_fixed_enable,
+ .disable = tegra_clk_periph_fixed_disable,
+ .recalc_rate = tegra_clk_periph_fixed_recalc_rate,
+};
+
+struct clk *tegra_clk_register_periph_fixed(const char *name,
+ const char *parent,
+ unsigned long flags,
+ void __iomem *base,
+ unsigned int mul,
+ unsigned int div,
+ unsigned int num)
+{
+ const struct tegra_clk_periph_regs *regs;
+ struct tegra_clk_periph_fixed *fixed;
+ struct clk_init_data init;
+ struct clk *clk;
+
+ regs = get_reg_bank(num);
+ if (!regs)
+ return ERR_PTR(-EINVAL);
+
+ fixed = kzalloc(sizeof(*fixed), GFP_KERNEL);
+ if (!fixed)
+ return ERR_PTR(-ENOMEM);
+
+ init.name = name;
+ init.flags = flags;
+ init.parent_names = parent ? &parent : NULL;
+ init.num_parents = parent ? 1 : 0;
+ init.ops = &tegra_clk_periph_fixed_ops;
+
+ fixed->base = base;
+ fixed->regs = regs;
+ fixed->mul = mul;
+ fixed->div = div;
+ fixed->num = num;
+
+ fixed->hw.init = &init;
+
+ clk = clk_register(NULL, &fixed->hw);
+ if (IS_ERR(clk))
+ kfree(fixed);
+
+ return clk;
+}
diff --git a/drivers/clk/tegra/clk-periph-gate.c b/drivers/clk/tegra/clk-periph-gate.c
index d28d6e9..8812782 100644
--- a/drivers/clk/tegra/clk-periph-gate.c
+++ b/drivers/clk/tegra/clk-periph-gate.c
@@ -134,7 +134,7 @@
struct tegra_clk_periph_gate *gate;
struct clk *clk;
struct clk_init_data init;
- struct tegra_clk_periph_regs *pregs;
+ const struct tegra_clk_periph_regs *pregs;
pregs = get_reg_bank(clk_num);
if (!pregs)
diff --git a/drivers/clk/tegra/clk-periph.c b/drivers/clk/tegra/clk-periph.c
index ec5b611..a17ca6d 100644
--- a/drivers/clk/tegra/clk-periph.c
+++ b/drivers/clk/tegra/clk-periph.c
@@ -145,7 +145,7 @@
{
struct clk *clk;
struct clk_init_data init;
- struct tegra_clk_periph_regs *bank;
+ const struct tegra_clk_periph_regs *bank;
bool div = !(periph->gate.flags & TEGRA_PERIPH_NO_DIV);
if (periph->gate.flags & TEGRA_PERIPH_NO_DIV) {
diff --git a/drivers/clk/tegra/clk-pll.c b/drivers/clk/tegra/clk-pll.c
index 6ac3f84..4e194ec 100644
--- a/drivers/clk/tegra/clk-pll.c
+++ b/drivers/clk/tegra/clk-pll.c
@@ -2013,6 +2013,52 @@
#endif
#if defined(CONFIG_ARCH_TEGRA_210_SOC)
+struct clk *tegra_clk_register_pllre_tegra210(const char *name,
+ const char *parent_name, void __iomem *clk_base,
+ void __iomem *pmc, unsigned long flags,
+ struct tegra_clk_pll_params *pll_params,
+ spinlock_t *lock, unsigned long parent_rate)
+{
+ u32 val;
+ struct tegra_clk_pll *pll;
+ struct clk *clk;
+
+ pll_params->vco_min = _clip_vco_min(pll_params->vco_min, parent_rate);
+
+ if (pll_params->adjust_vco)
+ pll_params->vco_min = pll_params->adjust_vco(pll_params,
+ parent_rate);
+
+ pll = _tegra_init_pll(clk_base, pmc, pll_params, lock);
+ if (IS_ERR(pll))
+ return ERR_CAST(pll);
+
+ /* program minimum rate by default */
+
+ val = pll_readl_base(pll);
+ if (val & PLL_BASE_ENABLE)
+ WARN_ON(readl_relaxed(clk_base + pll_params->iddq_reg) &
+ BIT(pll_params->iddq_bit_idx));
+ else {
+ val = 0x4 << divm_shift(pll);
+ val |= 0x41 << divn_shift(pll);
+ pll_writel_base(val, pll);
+ }
+
+ /* disable lock override */
+
+ val = pll_readl_misc(pll);
+ val &= ~BIT(29);
+ pll_writel_misc(val, pll);
+
+ clk = _tegra_clk_register_pll(pll, name, parent_name, flags,
+ &tegra_clk_pllre_ops);
+ if (IS_ERR(clk))
+ kfree(pll);
+
+ return clk;
+}
+
static int clk_plle_tegra210_enable(struct clk_hw *hw)
{
struct tegra_clk_pll *pll = to_clk_pll(hw);
diff --git a/drivers/clk/tegra/clk-tegra-fixed.c b/drivers/clk/tegra/clk-tegra-fixed.c
index d64ec7a..91c38f1 100644
--- a/drivers/clk/tegra/clk-tegra-fixed.c
+++ b/drivers/clk/tegra/clk-tegra-fixed.c
@@ -107,4 +107,3 @@
*dt_clk = clk;
}
}
-
diff --git a/drivers/clk/tegra/clk-tegra-periph.c b/drivers/clk/tegra/clk-tegra-periph.c
index ea2b9cbf..29d04c6 100644
--- a/drivers/clk/tegra/clk-tegra-periph.c
+++ b/drivers/clk/tegra/clk-tegra-periph.c
@@ -803,7 +803,7 @@
GATE("hda2hdmi", "clk_m", 128, TEGRA_PERIPH_ON_APB, tegra_clk_hda2hdmi, 0),
GATE("bsea", "clk_m", 62, 0, tegra_clk_bsea, 0),
GATE("bsev", "clk_m", 63, 0, tegra_clk_bsev, 0),
- GATE("mipi-cal", "clk_m", 56, 0, tegra_clk_mipi_cal, 0),
+ GATE("mipi-cal", "clk72mhz", 56, 0, tegra_clk_mipi_cal, 0),
GATE("usbd", "clk_m", 22, 0, tegra_clk_usbd, 0),
GATE("usb2", "clk_m", 58, 0, tegra_clk_usb2, 0),
GATE("usb3", "clk_m", 59, 0, tegra_clk_usb3, 0),
@@ -821,7 +821,6 @@
GATE("ispb", "clk_m", 3, 0, tegra_clk_ispb, 0),
GATE("vim2_clk", "clk_m", 11, 0, tegra_clk_vim2_clk, 0),
GATE("pcie", "clk_m", 70, 0, tegra_clk_pcie, 0),
- GATE("dpaux", "clk_m", 181, 0, tegra_clk_dpaux, 0),
GATE("gpu", "pll_ref", 184, 0, tegra_clk_gpu, 0),
GATE("pllg_ref", "pll_ref", 189, 0, tegra_clk_pll_g_ref, 0),
GATE("hsic_trk", "usb2_hsic_trk", 209, TEGRA_PERIPH_NO_RESET, tegra_clk_hsic_trk, 0),
@@ -877,7 +876,7 @@
struct clk **dt_clk;
for (i = 0; i < ARRAY_SIZE(periph_clks); i++) {
- struct tegra_clk_periph_regs *bank;
+ const struct tegra_clk_periph_regs *bank;
struct tegra_periph_init_data *data;
data = periph_clks + i;
diff --git a/drivers/clk/tegra/clk-tegra114.c b/drivers/clk/tegra/clk-tegra114.c
index df47ec3..b78054f 100644
--- a/drivers/clk/tegra/clk-tegra114.c
+++ b/drivers/clk/tegra/clk-tegra114.c
@@ -743,7 +743,6 @@
[tegra_clk_csi] = { .dt_id = TEGRA114_CLK_CSI, .present = true },
[tegra_clk_i2c2] = { .dt_id = TEGRA114_CLK_I2C2, .present = true },
[tegra_clk_uartc] = { .dt_id = TEGRA114_CLK_UARTC, .present = true },
- [tegra_clk_mipi_cal] = { .dt_id = TEGRA114_CLK_MIPI_CAL, .present = true },
[tegra_clk_emc] = { .dt_id = TEGRA114_CLK_EMC, .present = true },
[tegra_clk_usb2] = { .dt_id = TEGRA114_CLK_USB2, .present = true },
[tegra_clk_usb3] = { .dt_id = TEGRA114_CLK_USB3, .present = true },
@@ -1237,6 +1236,11 @@
&emc_lock);
clks[TEGRA114_CLK_MC] = clk;
+ clk = tegra_clk_register_periph_gate("mipi-cal", "clk_m", 0, clk_base,
+ CLK_SET_RATE_PARENT, 56,
+ periph_clk_enb_refcnt);
+ clks[TEGRA114_CLK_MIPI_CAL] = clk;
+
for (i = 0; i < ARRAY_SIZE(tegra_periph_clk_list); i++) {
data = &tegra_periph_clk_list[i];
clk = tegra_clk_register_periph(data->name,
diff --git a/drivers/clk/tegra/clk-tegra124-dfll-fcpu.c b/drivers/clk/tegra/clk-tegra124-dfll-fcpu.c
index 6125333..c205809 100644
--- a/drivers/clk/tegra/clk-tegra124-dfll-fcpu.c
+++ b/drivers/clk/tegra/clk-tegra124-dfll-fcpu.c
@@ -47,32 +47,32 @@
},
.speedo_scale = 100,
.voltage_scale = 1000,
- .cvb_table = {
- {204000000UL, {1112619, -29295, 402} },
- {306000000UL, {1150460, -30585, 402} },
- {408000000UL, {1190122, -31865, 402} },
- {510000000UL, {1231606, -33155, 402} },
- {612000000UL, {1274912, -34435, 402} },
- {714000000UL, {1320040, -35725, 402} },
- {816000000UL, {1366990, -37005, 402} },
- {918000000UL, {1415762, -38295, 402} },
- {1020000000UL, {1466355, -39575, 402} },
- {1122000000UL, {1518771, -40865, 402} },
- {1224000000UL, {1573009, -42145, 402} },
- {1326000000UL, {1629068, -43435, 402} },
- {1428000000UL, {1686950, -44715, 402} },
- {1530000000UL, {1746653, -46005, 402} },
- {1632000000UL, {1808179, -47285, 402} },
- {1734000000UL, {1871526, -48575, 402} },
- {1836000000UL, {1936696, -49855, 402} },
- {1938000000UL, {2003687, -51145, 402} },
- {2014500000UL, {2054787, -52095, 402} },
- {2116500000UL, {2124957, -53385, 402} },
- {2218500000UL, {2196950, -54665, 402} },
- {2320500000UL, {2270765, -55955, 402} },
- {2422500000UL, {2346401, -57235, 402} },
- {2524500000UL, {2437299, -58535, 402} },
- {0, { 0, 0, 0} },
+ .entries = {
+ { 204000000UL, { 1112619, -29295, 402 } },
+ { 306000000UL, { 1150460, -30585, 402 } },
+ { 408000000UL, { 1190122, -31865, 402 } },
+ { 510000000UL, { 1231606, -33155, 402 } },
+ { 612000000UL, { 1274912, -34435, 402 } },
+ { 714000000UL, { 1320040, -35725, 402 } },
+ { 816000000UL, { 1366990, -37005, 402 } },
+ { 918000000UL, { 1415762, -38295, 402 } },
+ { 1020000000UL, { 1466355, -39575, 402 } },
+ { 1122000000UL, { 1518771, -40865, 402 } },
+ { 1224000000UL, { 1573009, -42145, 402 } },
+ { 1326000000UL, { 1629068, -43435, 402 } },
+ { 1428000000UL, { 1686950, -44715, 402 } },
+ { 1530000000UL, { 1746653, -46005, 402 } },
+ { 1632000000UL, { 1808179, -47285, 402 } },
+ { 1734000000UL, { 1871526, -48575, 402 } },
+ { 1836000000UL, { 1936696, -49855, 402 } },
+ { 1938000000UL, { 2003687, -51145, 402 } },
+ { 2014500000UL, { 2054787, -52095, 402 } },
+ { 2116500000UL, { 2124957, -53385, 402 } },
+ { 2218500000UL, { 2196950, -54665, 402 } },
+ { 2320500000UL, { 2270765, -55955, 402 } },
+ { 2422500000UL, { 2346401, -57235, 402 } },
+ { 2524500000UL, { 2437299, -58535, 402 } },
+ { 0UL, { 0, 0, 0 } },
},
.cpu_dfll_data = {
.tune0_low = 0x005020ff,
@@ -84,9 +84,8 @@
static int tegra124_dfll_fcpu_probe(struct platform_device *pdev)
{
- int process_id, speedo_id, speedo_value;
+ int process_id, speedo_id, speedo_value, err;
struct tegra_dfll_soc_data *soc;
- const struct cvb_table *cvb;
process_id = tegra_sku_info.cpu_process_id;
speedo_id = tegra_sku_info.cpu_speedo_id;
@@ -108,23 +107,41 @@
return -ENODEV;
}
- cvb = tegra_cvb_build_opp_table(tegra124_cpu_cvb_tables,
- ARRAY_SIZE(tegra124_cpu_cvb_tables),
- process_id, speedo_id, speedo_value,
- cpu_max_freq_table[speedo_id],
- soc->dev);
- if (IS_ERR(cvb)) {
- dev_err(&pdev->dev, "couldn't build OPP table: %ld\n",
- PTR_ERR(cvb));
- return PTR_ERR(cvb);
+ soc->max_freq = cpu_max_freq_table[speedo_id];
+
+ soc->cvb = tegra_cvb_add_opp_table(soc->dev, tegra124_cpu_cvb_tables,
+ ARRAY_SIZE(tegra124_cpu_cvb_tables),
+ process_id, speedo_id, speedo_value,
+ soc->max_freq);
+ if (IS_ERR(soc->cvb)) {
+ dev_err(&pdev->dev, "couldn't add OPP table: %ld\n",
+ PTR_ERR(soc->cvb));
+ return PTR_ERR(soc->cvb);
}
- soc->min_millivolts = cvb->min_millivolts;
- soc->tune0_low = cvb->cpu_dfll_data.tune0_low;
- soc->tune0_high = cvb->cpu_dfll_data.tune0_high;
- soc->tune1 = cvb->cpu_dfll_data.tune1;
+ err = tegra_dfll_register(pdev, soc);
+ if (err < 0) {
+ tegra_cvb_remove_opp_table(soc->dev, soc->cvb, soc->max_freq);
+ return err;
+ }
- return tegra_dfll_register(pdev, soc);
+ platform_set_drvdata(pdev, soc);
+
+ return 0;
+}
+
+static int tegra124_dfll_fcpu_remove(struct platform_device *pdev)
+{
+ struct tegra_dfll_soc_data *soc = platform_get_drvdata(pdev);
+ int err;
+
+ err = tegra_dfll_unregister(pdev);
+ if (err < 0)
+ dev_err(&pdev->dev, "failed to unregister DFLL: %d\n", err);
+
+ tegra_cvb_remove_opp_table(soc->dev, soc->cvb, soc->max_freq);
+
+ return 0;
}
static const struct of_device_id tegra124_dfll_fcpu_of_match[] = {
@@ -140,7 +157,7 @@
static struct platform_driver tegra124_dfll_fcpu_driver = {
.probe = tegra124_dfll_fcpu_probe,
- .remove = tegra_dfll_unregister,
+ .remove = tegra124_dfll_fcpu_remove,
.driver = {
.name = "tegra124-dfll",
.of_match_table = tegra124_dfll_fcpu_of_match,
diff --git a/drivers/clk/tegra/clk-tegra124.c b/drivers/clk/tegra/clk-tegra124.c
index 1627258..f4fbbf1 100644
--- a/drivers/clk/tegra/clk-tegra124.c
+++ b/drivers/clk/tegra/clk-tegra124.c
@@ -1155,6 +1155,10 @@
1, 2);
clks[TEGRA124_CLK_XUSB_SS_DIV2] = clk;
+ clk = tegra_clk_register_periph_fixed("dpaux", "pll_p", 0, clk_base,
+ 1, 17, 181);
+ clks[TEGRA124_CLK_DPAUX] = clk;
+
clk = clk_register_gate(NULL, "pll_d_dsi_out", "pll_d_out0", 0,
clk_base + PLLD_MISC, 30, 0, &pll_d_lock);
clks[TEGRA124_CLK_PLL_D_DSI_OUT] = clk;
diff --git a/drivers/clk/tegra/clk-tegra210.c b/drivers/clk/tegra/clk-tegra210.c
index 3d0edee..b855181 100644
--- a/drivers/clk/tegra/clk-tegra210.c
+++ b/drivers/clk/tegra/clk-tegra210.c
@@ -92,6 +92,7 @@
#define PLLE_AUX 0x48c
#define PLLRE_BASE 0x4c4
#define PLLRE_MISC0 0x4c8
+#define PLLRE_OUT1 0x4cc
#define PLLDP_BASE 0x590
#define PLLDP_MISC 0x594
@@ -2150,6 +2151,7 @@
[tegra_clk_clk72Mhz_8] = { .dt_id = TEGRA210_CLK_CLK72MHZ, .present = true },
[tegra_clk_vic03_8] = { .dt_id = TEGRA210_CLK_VIC03, .present = true },
[tegra_clk_dpaux] = { .dt_id = TEGRA210_CLK_DPAUX, .present = true },
+ [tegra_clk_dpaux1] = { .dt_id = TEGRA210_CLK_DPAUX1, .present = true },
[tegra_clk_sor0] = { .dt_id = TEGRA210_CLK_SOR0, .present = true },
[tegra_clk_sor0_lvds] = { .dt_id = TEGRA210_CLK_SOR0_LVDS, .present = true },
[tegra_clk_gpu] = { .dt_id = TEGRA210_CLK_GPU, .present = true },
@@ -2461,6 +2463,18 @@
1, 2);
clks[TEGRA210_CLK_XUSB_SS_DIV2] = clk;
+ clk = tegra_clk_register_periph_fixed("dpaux", "pll_p", 0, clk_base,
+ 1, 17, 181);
+ clks[TEGRA210_CLK_DPAUX] = clk;
+
+ clk = tegra_clk_register_periph_fixed("dpaux1", "pll_p", 0, clk_base,
+ 1, 17, 207);
+ clks[TEGRA210_CLK_DPAUX1] = clk;
+
+ clk = tegra_clk_register_periph_fixed("sor_safe", "pll_p", 0, clk_base,
+ 1, 17, 222);
+ clks[TEGRA210_CLK_SOR_SAFE] = clk;
+
/* pll_d_dsi_out */
clk = clk_register_gate(NULL, "pll_d_dsi_out", "pll_d_out0", 0,
clk_base + PLLD_MISC0, 21, 0, &pll_d_lock);
@@ -2640,8 +2654,10 @@
clks[TEGRA210_CLK_PLL_D_OUT0] = clk;
/* PLLRE */
- clk = tegra_clk_register_pllre("pll_re_vco", "pll_ref", clk_base, pmc,
- 0, &pll_re_vco_params, &pll_re_lock, pll_ref_freq);
+ clk = tegra_clk_register_pllre_tegra210("pll_re_vco", "pll_ref",
+ clk_base, pmc, 0,
+ &pll_re_vco_params,
+ &pll_re_lock, pll_ref_freq);
clk_register_clkdev(clk, "pll_re_vco", NULL);
clks[TEGRA210_CLK_PLL_RE_VCO] = clk;
@@ -2651,6 +2667,15 @@
clk_register_clkdev(clk, "pll_re_out", NULL);
clks[TEGRA210_CLK_PLL_RE_OUT] = clk;
+ clk = tegra_clk_register_divider("pll_re_out1_div", "pll_re_vco",
+ clk_base + PLLRE_OUT1, 0,
+ TEGRA_DIVIDER_ROUND_UP,
+ 8, 8, 1, NULL);
+ clk = tegra_clk_register_pll_out("pll_re_out1", "pll_re_out1_div",
+ clk_base + PLLRE_OUT1, 1, 0,
+ CLK_SET_RATE_PARENT, 0, NULL);
+ clks[TEGRA210_CLK_PLL_RE_OUT1] = clk;
+
/* PLLE */
clk = tegra_clk_register_plle_tegra210("pll_e", "pll_ref",
clk_base, 0, &pll_e_params, NULL);
diff --git a/drivers/clk/tegra/clk-tegra30.c b/drivers/clk/tegra/clk-tegra30.c
index 0478565..9396f49 100644
--- a/drivers/clk/tegra/clk-tegra30.c
+++ b/drivers/clk/tegra/clk-tegra30.c
@@ -339,11 +339,11 @@
};
static struct tegra_clk_pll_freq_table pll_u_freq_table[] = {
- { 12000000, 480000000, 960, 12, 1, 12 },
- { 13000000, 480000000, 960, 13, 1, 12 },
- { 16800000, 480000000, 400, 7, 1, 5 },
- { 19200000, 480000000, 200, 4, 1, 3 },
- { 26000000, 480000000, 960, 26, 1, 12 },
+ { 12000000, 480000000, 960, 12, 2, 12 },
+ { 13000000, 480000000, 960, 13, 2, 12 },
+ { 16800000, 480000000, 400, 7, 2, 5 },
+ { 19200000, 480000000, 200, 4, 2, 3 },
+ { 26000000, 480000000, 960, 26, 2, 12 },
{ 0, 0, 0, 0, 0, 0 },
};
@@ -1372,6 +1372,7 @@
{ TEGRA30_CLK_SBC4, TEGRA30_CLK_PLL_P, 100000000, 0 },
{ TEGRA30_CLK_SBC5, TEGRA30_CLK_PLL_P, 100000000, 0 },
{ TEGRA30_CLK_SBC6, TEGRA30_CLK_PLL_P, 100000000, 0 },
+ { TEGRA30_CLK_PLL_C, TEGRA30_CLK_CLK_MAX, 600000000, 0 },
{ TEGRA30_CLK_HOST1X, TEGRA30_CLK_PLL_C, 150000000, 0 },
{ TEGRA30_CLK_DISP1, TEGRA30_CLK_PLL_P, 600000000, 0 },
{ TEGRA30_CLK_DISP2, TEGRA30_CLK_PLL_P, 600000000, 0 },
@@ -1379,6 +1380,7 @@
{ TEGRA30_CLK_GR2D, TEGRA30_CLK_PLL_C, 300000000, 0 },
{ TEGRA30_CLK_GR3D, TEGRA30_CLK_PLL_C, 300000000, 0 },
{ TEGRA30_CLK_GR3D2, TEGRA30_CLK_PLL_C, 300000000, 0 },
+ { TEGRA30_CLK_PLL_U, TEGRA30_CLK_CLK_MAX, 480000000, 0 },
/* must be the last entry */
{ TEGRA30_CLK_CLK_MAX, TEGRA30_CLK_CLK_MAX, 0, 0 },
};
diff --git a/drivers/clk/tegra/clk.c b/drivers/clk/tegra/clk.c
index f60fe2e..b2cdd9a 100644
--- a/drivers/clk/tegra/clk.c
+++ b/drivers/clk/tegra/clk.c
@@ -84,7 +84,7 @@
static int (*special_reset_deassert)(unsigned long);
static unsigned int num_special_reset;
-static struct tegra_clk_periph_regs periph_regs[] = {
+static const struct tegra_clk_periph_regs periph_regs[] = {
[0] = {
.enb_reg = CLK_OUT_ENB_L,
.enb_set_reg = CLK_OUT_ENB_SET_L,
@@ -182,7 +182,7 @@
return -EINVAL;
}
-struct tegra_clk_periph_regs *get_reg_bank(int clkid)
+const struct tegra_clk_periph_regs *get_reg_bank(int clkid)
{
int reg_bank = clkid / 32;
diff --git a/drivers/clk/tegra/clk.h b/drivers/clk/tegra/clk.h
index 4dbcfae..9421f03 100644
--- a/drivers/clk/tegra/clk.h
+++ b/drivers/clk/tegra/clk.h
@@ -386,6 +386,12 @@
struct tegra_clk_pll_params *pll_params,
spinlock_t *lock, unsigned long parent_rate);
+struct clk *tegra_clk_register_pllre_tegra210(const char *name,
+ const char *parent_name, void __iomem *clk_base,
+ void __iomem *pmc, unsigned long flags,
+ struct tegra_clk_pll_params *pll_params,
+ spinlock_t *lock, unsigned long parent_rate);
+
struct clk *tegra_clk_register_plle_tegra114(const char *name,
const char *parent_name,
void __iomem *clk_base, unsigned long flags,
@@ -496,7 +502,7 @@
u8 flags;
int clk_num;
int *enable_refcnt;
- struct tegra_clk_periph_regs *regs;
+ const struct tegra_clk_periph_regs *regs;
};
#define to_clk_periph_gate(_hw) \
@@ -516,6 +522,23 @@
const char *parent_name, u8 gate_flags, void __iomem *clk_base,
unsigned long flags, int clk_num, int *enable_refcnt);
+struct tegra_clk_periph_fixed {
+ struct clk_hw hw;
+ void __iomem *base;
+ const struct tegra_clk_periph_regs *regs;
+ unsigned int mul;
+ unsigned int div;
+ unsigned int num;
+};
+
+struct clk *tegra_clk_register_periph_fixed(const char *name,
+ const char *parent,
+ unsigned long flags,
+ void __iomem *base,
+ unsigned int mul,
+ unsigned int div,
+ unsigned int num);
+
/**
* struct clk-periph - peripheral clock
*
@@ -716,7 +739,7 @@
void tegra_init_dup_clks(struct tegra_clk_duplicate *dup_list,
struct clk *clks[], int clk_max);
-struct tegra_clk_periph_regs *get_reg_bank(int clkid);
+const struct tegra_clk_periph_regs *get_reg_bank(int clkid);
struct clk **tegra_clk_init(void __iomem *clk_base, int num, int periph_banks);
struct clk **tegra_lookup_dt_id(int clk_id, struct tegra_clk *tegra_clk);
diff --git a/drivers/clk/tegra/cvb.c b/drivers/clk/tegra/cvb.c
index 69c74ee..624115e 100644
--- a/drivers/clk/tegra/cvb.c
+++ b/drivers/clk/tegra/cvb.c
@@ -61,29 +61,28 @@
return mv;
}
-static int build_opp_table(const struct cvb_table *d,
- int speedo_value,
- unsigned long max_freq,
- struct device *opp_dev)
+static int build_opp_table(struct device *dev, const struct cvb_table *table,
+ int speedo_value, unsigned long max_freq)
{
+ const struct rail_alignment *align = &table->alignment;
int i, ret, dfll_mv, min_mv, max_mv;
- const struct cvb_table_freq_entry *table = NULL;
- const struct rail_alignment *align = &d->alignment;
- min_mv = round_voltage(d->min_millivolts, align, UP);
- max_mv = round_voltage(d->max_millivolts, align, DOWN);
+ min_mv = round_voltage(table->min_millivolts, align, UP);
+ max_mv = round_voltage(table->max_millivolts, align, DOWN);
for (i = 0; i < MAX_DVFS_FREQS; i++) {
- table = &d->cvb_table[i];
- if (!table->freq || (table->freq > max_freq))
+ const struct cvb_table_freq_entry *entry = &table->entries[i];
+
+ if (!entry->freq || (entry->freq > max_freq))
break;
- dfll_mv = get_cvb_voltage(
- speedo_value, d->speedo_scale, &table->coefficients);
- dfll_mv = round_cvb_voltage(dfll_mv, d->voltage_scale, align);
+ dfll_mv = get_cvb_voltage(speedo_value, table->speedo_scale,
+ &entry->coefficients);
+ dfll_mv = round_cvb_voltage(dfll_mv, table->voltage_scale,
+ align);
dfll_mv = clamp(dfll_mv, min_mv, max_mv);
- ret = dev_pm_opp_add(opp_dev, table->freq, dfll_mv * 1000);
+ ret = dev_pm_opp_add(dev, entry->freq, dfll_mv * 1000);
if (ret)
return ret;
}
@@ -92,7 +91,7 @@
}
/**
- * tegra_cvb_build_opp_table - build OPP table from Tegra CVB tables
+ * tegra_cvb_add_opp_table - build OPP table from Tegra CVB tables
* @cvb_tables: array of CVB tables
* @sz: size of the previously mentioned array
* @process_id: process id of the HW module
@@ -108,26 +107,42 @@
* given @opp_dev. Returns a pointer to the struct cvb_table that matched
* or an ERR_PTR on failure.
*/
-const struct cvb_table *tegra_cvb_build_opp_table(
- const struct cvb_table *cvb_tables,
- size_t sz, int process_id,
- int speedo_id, int speedo_value,
- unsigned long max_rate,
- struct device *opp_dev)
+const struct cvb_table *
+tegra_cvb_add_opp_table(struct device *dev, const struct cvb_table *tables,
+ size_t count, int process_id, int speedo_id,
+ int speedo_value, unsigned long max_freq)
{
- int i, ret;
+ size_t i;
+ int ret;
- for (i = 0; i < sz; i++) {
- const struct cvb_table *d = &cvb_tables[i];
+ for (i = 0; i < count; i++) {
+ const struct cvb_table *table = &tables[i];
- if (d->speedo_id != -1 && d->speedo_id != speedo_id)
- continue;
- if (d->process_id != -1 && d->process_id != process_id)
+ if (table->speedo_id != -1 && table->speedo_id != speedo_id)
continue;
- ret = build_opp_table(d, speedo_value, max_rate, opp_dev);
- return ret ? ERR_PTR(ret) : d;
+ if (table->process_id != -1 && table->process_id != process_id)
+ continue;
+
+ ret = build_opp_table(dev, table, speedo_value, max_freq);
+ return ret ? ERR_PTR(ret) : table;
}
return ERR_PTR(-EINVAL);
}
+
+void tegra_cvb_remove_opp_table(struct device *dev,
+ const struct cvb_table *table,
+ unsigned long max_freq)
+{
+ unsigned int i;
+
+ for (i = 0; i < MAX_DVFS_FREQS; i++) {
+ const struct cvb_table_freq_entry *entry = &table->entries[i];
+
+ if (!entry->freq || (entry->freq > max_freq))
+ break;
+
+ dev_pm_opp_remove(dev, entry->freq);
+ }
+}
diff --git a/drivers/clk/tegra/cvb.h b/drivers/clk/tegra/cvb.h
index f62cdc4..c1f0779 100644
--- a/drivers/clk/tegra/cvb.h
+++ b/drivers/clk/tegra/cvb.h
@@ -53,15 +53,16 @@
int speedo_scale;
int voltage_scale;
- struct cvb_table_freq_entry cvb_table[MAX_DVFS_FREQS];
+ struct cvb_table_freq_entry entries[MAX_DVFS_FREQS];
struct cvb_cpu_dfll_data cpu_dfll_data;
};
-const struct cvb_table *tegra_cvb_build_opp_table(
- const struct cvb_table *cvb_tables,
- size_t sz, int process_id,
- int speedo_id, int speedo_value,
- unsigned long max_rate,
- struct device *opp_dev);
+const struct cvb_table *
+tegra_cvb_add_opp_table(struct device *dev, const struct cvb_table *cvb_tables,
+ size_t count, int process_id, int speedo_id,
+ int speedo_value, unsigned long max_freq);
+void tegra_cvb_remove_opp_table(struct device *dev,
+ const struct cvb_table *table,
+ unsigned long max_freq);
#endif
diff --git a/drivers/clk/ti/clk-54xx.c b/drivers/clk/ti/clk-54xx.c
index 59ce2fa..294bc03 100644
--- a/drivers/clk/ti/clk-54xx.c
+++ b/drivers/clk/ti/clk-54xx.c
@@ -210,6 +210,7 @@
DT_CLK("usbhs_omap", "usbtll_fck", "dummy_ck"),
DT_CLK("omap_wdt", "ick", "dummy_ck"),
DT_CLK(NULL, "timer_32k_ck", "sys_32k_ck"),
+ DT_CLK(NULL, "sys_clkin_ck", "sys_clkin"),
DT_CLK("4ae18000.timer", "timer_sys_ck", "sys_clkin"),
DT_CLK("48032000.timer", "timer_sys_ck", "sys_clkin"),
DT_CLK("48034000.timer", "timer_sys_ck", "sys_clkin"),
diff --git a/drivers/clk/ti/clk-7xx.c b/drivers/clk/ti/clk-7xx.c
index 6b5a309..bfa17d3 100644
--- a/drivers/clk/ti/clk-7xx.c
+++ b/drivers/clk/ti/clk-7xx.c
@@ -289,6 +289,7 @@
DT_CLK("usbhs_omap", "usbtll_fck", "dummy_ck"),
DT_CLK("omap_wdt", "ick", "dummy_ck"),
DT_CLK(NULL, "timer_32k_ck", "sys_32k_ck"),
+ DT_CLK(NULL, "sys_clkin_ck", "timer_sys_clk_div"),
DT_CLK("4ae18000.timer", "timer_sys_ck", "timer_sys_clk_div"),
DT_CLK("48032000.timer", "timer_sys_ck", "timer_sys_clk_div"),
DT_CLK("48034000.timer", "timer_sys_ck", "timer_sys_clk_div"),
diff --git a/drivers/clk/ti/clk-dra7-atl.c b/drivers/clk/ti/clk-dra7-atl.c
index 2e14dfb..c773332 100644
--- a/drivers/clk/ti/clk-dra7-atl.c
+++ b/drivers/clk/ti/clk-dra7-atl.c
@@ -265,6 +265,7 @@
/* Get configuration for the ATL instances */
snprintf(prop, sizeof(prop), "atl%u", i);
+ of_node_get(node);
cfg_node = of_find_node_by_name(node, prop);
if (cfg_node) {
ret = of_property_read_u32(cfg_node, "bws",
@@ -278,6 +279,7 @@
atl_write(cinfo, DRA7_ATL_AWSMUX_REG(i),
cdesc->aws);
}
+ of_node_put(cfg_node);
}
cdesc->probed = true;
diff --git a/drivers/clk/ti/clkt_dflt.c b/drivers/clk/ti/clkt_dflt.c
index 1ddc288..c6ae563 100644
--- a/drivers/clk/ti/clkt_dflt.c
+++ b/drivers/clk/ti/clkt_dflt.c
@@ -222,7 +222,7 @@
}
}
- if (unlikely(IS_ERR(clk->enable_reg))) {
+ if (IS_ERR(clk->enable_reg)) {
pr_err("%s: %s missing enable_reg\n", __func__,
clk_hw_get_name(hw));
ret = -EINVAL;
diff --git a/drivers/clk/ti/clkt_dpll.c b/drivers/clk/ti/clkt_dpll.c
index 032c658..b919fdf 100644
--- a/drivers/clk/ti/clkt_dpll.c
+++ b/drivers/clk/ti/clkt_dpll.c
@@ -301,6 +301,9 @@
dd = clk->dpll_data;
+ if (dd->max_rate && target_rate > dd->max_rate)
+ target_rate = dd->max_rate;
+
ref_rate = clk_hw_get_rate(dd->clk_ref);
clk_name = clk_hw_get_name(hw);
pr_debug("clock: %s: starting DPLL round_rate, target rate %lu\n",
diff --git a/drivers/clk/ti/dpll.c b/drivers/clk/ti/dpll.c
index 3bc9959..9fc8754 100644
--- a/drivers/clk/ti/dpll.c
+++ b/drivers/clk/ti/dpll.c
@@ -655,6 +655,7 @@
.max_multiplier = 2047,
.max_divider = 128,
.min_divider = 1,
+ .max_rate = 1000000000,
.modes = (1 << DPLL_LOW_POWER_BYPASS) | (1 << DPLL_LOCKED),
};
@@ -674,6 +675,7 @@
.max_divider = 256,
.min_divider = 2,
.flags = DPLL_J_TYPE,
+ .max_rate = 2000000000,
.modes = (1 << DPLL_LOW_POWER_BYPASS) | (1 << DPLL_LOCKED),
};
@@ -692,6 +694,7 @@
.max_multiplier = 2047,
.max_divider = 128,
.min_divider = 1,
+ .max_rate = 2000000000,
.flags = DPLL_J_TYPE,
.modes = (1 << DPLL_LOW_POWER_BYPASS) | (1 << DPLL_LOCKED),
};
@@ -712,6 +715,7 @@
.max_multiplier = 2047,
.max_divider = 128,
.min_divider = 1,
+ .max_rate = 1000000000,
.modes = (1 << DPLL_LOW_POWER_BYPASS) | (1 << DPLL_LOCKED),
};
@@ -729,6 +733,7 @@
.max_multiplier = 2047,
.max_divider = 128,
.min_divider = 1,
+ .max_rate = 1000000000,
.modes = (1 << DPLL_LOW_POWER_BYPASS) | (1 << DPLL_LOCKED),
};
diff --git a/drivers/clk/zte/clk-zx296702.c b/drivers/clk/zte/clk-zx296702.c
index ebd20d8..76e967c 100644
--- a/drivers/clk/zte/clk-zx296702.c
+++ b/drivers/clk/zte/clk-zx296702.c
@@ -234,8 +234,7 @@
WARN_ON(!topcrm_base);
clk[ZX296702_OSC] =
- clk_register_fixed_rate(NULL, "osc", NULL, CLK_IS_ROOT,
- 30000000);
+ clk_register_fixed_rate(NULL, "osc", NULL, 0, 30000000);
clk[ZX296702_PLL_A9] =
clk_register_zx_pll("pll_a9", "osc", 0, topcrm_base
+ 0x01c, pll_a9_config,
diff --git a/drivers/clocksource/Kconfig b/drivers/clocksource/Kconfig
index 6ff327a..47352d2 100644
--- a/drivers/clocksource/Kconfig
+++ b/drivers/clocksource/Kconfig
@@ -181,6 +181,16 @@
This option enables support for Texas Instruments 32.768 Hz clocksource
available on many OMAP-like platforms.
+config CLKSRC_NPS
+ bool "NPS400 clocksource driver" if COMPILE_TEST
+ depends on !PHYS_ADDR_T_64BIT
+ select CLKSRC_MMIO
+ select CLKSRC_OF if OF
+ help
+ NPS400 clocksource support.
+ Got 64 bit counter with update rate up to 1000MHz.
+ This counter is accessed via couple of 32 bit memory mapped registers.
+
config CLKSRC_STM32
bool "Clocksource for STM32 SoCs" if !ARCH_STM32
depends on OF && ARM && (ARCH_STM32 || COMPILE_TEST)
diff --git a/drivers/clocksource/Makefile b/drivers/clocksource/Makefile
index b0a3c96..473974f 100644
--- a/drivers/clocksource/Makefile
+++ b/drivers/clocksource/Makefile
@@ -47,6 +47,7 @@
obj-$(CONFIG_MTK_TIMER) += mtk_timer.o
obj-$(CONFIG_CLKSRC_PISTACHIO) += time-pistachio.o
obj-$(CONFIG_CLKSRC_TI_32K) += timer-ti-32k.o
+obj-$(CONFIG_CLKSRC_NPS) += timer-nps.o
obj-$(CONFIG_ARM_ARCH_TIMER) += arm_arch_timer.o
obj-$(CONFIG_ARM_GLOBAL_TIMER) += arm_global_timer.o
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 5152b38..4814446 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -468,11 +468,11 @@
.mask = CLOCKSOURCE_MASK(56),
};
-static struct timecounter timecounter;
+static struct arch_timer_kvm_info arch_timer_kvm_info;
-struct timecounter *arch_timer_get_timecounter(void)
+struct arch_timer_kvm_info *arch_timer_get_kvm_info(void)
{
- return &timecounter;
+ return &arch_timer_kvm_info;
}
static void __init arch_counter_register(unsigned type)
@@ -500,7 +500,8 @@
clocksource_register_hz(&clocksource_counter, arch_timer_rate);
cyclecounter.mult = clocksource_counter.mult;
cyclecounter.shift = clocksource_counter.shift;
- timecounter_init(&timecounter, &cyclecounter, start_count);
+ timecounter_init(&arch_timer_kvm_info.timecounter,
+ &cyclecounter, start_count);
/* 56 bits minimum, so we assume worst case rollover */
sched_clock_register(arch_timer_read_counter, 56, arch_timer_rate);
@@ -744,6 +745,8 @@
arch_timer_register();
arch_timer_common_init();
+
+ arch_timer_kvm_info.virtual_irq = arch_timer_ppi[VIRT_PPI];
}
static void __init arch_timer_of_init(struct device_node *np)
diff --git a/drivers/clocksource/timer-nps.c b/drivers/clocksource/timer-nps.c
new file mode 100644
index 0000000..d461089
--- /dev/null
+++ b/drivers/clocksource/timer-nps.c
@@ -0,0 +1,98 @@
+/*
+ * Copyright (c) 2016, Mellanox Technologies. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/clocksource.h>
+#include <linux/clockchips.h>
+#include <linux/clk.h>
+#include <linux/of.h>
+#include <linux/of_irq.h>
+#include <linux/cpu.h>
+#include <soc/nps/common.h>
+
+#define NPS_MSU_TICK_LOW 0xC8
+#define NPS_CLUSTER_OFFSET 8
+#define NPS_CLUSTER_NUM 16
+
+/* This array is per cluster of CPUs (Each NPS400 cluster got 256 CPUs) */
+static void *nps_msu_reg_low_addr[NPS_CLUSTER_NUM] __read_mostly;
+
+static unsigned long nps_timer_rate;
+
+static cycle_t nps_clksrc_read(struct clocksource *clksrc)
+{
+ int cluster = raw_smp_processor_id() >> NPS_CLUSTER_OFFSET;
+
+ return (cycle_t)ioread32be(nps_msu_reg_low_addr[cluster]);
+}
+
+static void __init nps_setup_clocksource(struct device_node *node,
+ struct clk *clk)
+{
+ int ret, cluster;
+
+ for (cluster = 0; cluster < NPS_CLUSTER_NUM; cluster++)
+ nps_msu_reg_low_addr[cluster] =
+ nps_host_reg((cluster << NPS_CLUSTER_OFFSET),
+ NPS_MSU_BLKID, NPS_MSU_TICK_LOW);
+
+ ret = clk_prepare_enable(clk);
+ if (ret) {
+ pr_err("Couldn't enable parent clock\n");
+ return;
+ }
+
+ nps_timer_rate = clk_get_rate(clk);
+
+ ret = clocksource_mmio_init(nps_msu_reg_low_addr, "EZnps-tick",
+ nps_timer_rate, 301, 32, nps_clksrc_read);
+ if (ret) {
+ pr_err("Couldn't register clock source.\n");
+ clk_disable_unprepare(clk);
+ }
+}
+
+static void __init nps_timer_init(struct device_node *node)
+{
+ struct clk *clk;
+
+ clk = of_clk_get(node, 0);
+ if (IS_ERR(clk)) {
+ pr_err("Can't get timer clock.\n");
+ return;
+ }
+
+ nps_setup_clocksource(node, clk);
+}
+
+CLOCKSOURCE_OF_DECLARE(ezchip_nps400_clksrc, "ezchip,nps400-timer",
+ nps_timer_init);
diff --git a/drivers/cpufreq/Makefile b/drivers/cpufreq/Makefile
index e1eb11e..0a9b6a09 100644
--- a/drivers/cpufreq/Makefile
+++ b/drivers/cpufreq/Makefile
@@ -102,7 +102,7 @@
obj-$(CONFIG_ETRAXFS) += cris-etraxfs-cpufreq.o
obj-$(CONFIG_IA64_ACPI_CPUFREQ) += ia64-acpi-cpufreq.o
obj-$(CONFIG_LOONGSON2_CPUFREQ) += loongson2_cpufreq.o
-obj-$(CONFIG_LOONGSON1_CPUFREQ) += ls1x-cpufreq.o
+obj-$(CONFIG_LOONGSON1_CPUFREQ) += loongson1-cpufreq.o
obj-$(CONFIG_SH_CPU_FREQ) += sh-cpufreq.o
obj-$(CONFIG_SPARC_US2E_CPUFREQ) += sparc-us2e-cpufreq.o
obj-$(CONFIG_SPARC_US3_CPUFREQ) += sparc-us3-cpufreq.o
diff --git a/drivers/cpufreq/loongson1-cpufreq.c b/drivers/cpufreq/loongson1-cpufreq.c
new file mode 100644
index 0000000..be89416
--- /dev/null
+++ b/drivers/cpufreq/loongson1-cpufreq.c
@@ -0,0 +1,222 @@
+/*
+ * CPU Frequency Scaling for Loongson 1 SoC
+ *
+ * Copyright (C) 2014-2016 Zhang, Keguang <keguang.zhang@gmail.com>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2. This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#include <linux/clk.h>
+#include <linux/clk-provider.h>
+#include <linux/cpu.h>
+#include <linux/cpufreq.h>
+#include <linux/delay.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+
+#include <cpufreq.h>
+#include <loongson1.h>
+
+struct ls1x_cpufreq {
+ struct device *dev;
+ struct clk *clk; /* CPU clk */
+ struct clk *mux_clk; /* MUX of CPU clk */
+ struct clk *pll_clk; /* PLL clk */
+ struct clk *osc_clk; /* OSC clk */
+ unsigned int max_freq;
+ unsigned int min_freq;
+};
+
+static struct ls1x_cpufreq *cpufreq;
+
+static int ls1x_cpufreq_notifier(struct notifier_block *nb,
+ unsigned long val, void *data)
+{
+ if (val == CPUFREQ_POSTCHANGE)
+ current_cpu_data.udelay_val = loops_per_jiffy;
+
+ return NOTIFY_OK;
+}
+
+static struct notifier_block ls1x_cpufreq_notifier_block = {
+ .notifier_call = ls1x_cpufreq_notifier
+};
+
+static int ls1x_cpufreq_target(struct cpufreq_policy *policy,
+ unsigned int index)
+{
+ struct device *cpu_dev = get_cpu_device(policy->cpu);
+ unsigned int old_freq, new_freq;
+
+ old_freq = policy->cur;
+ new_freq = policy->freq_table[index].frequency;
+
+ /*
+ * The procedure of reconfiguring CPU clk is as below.
+ *
+ * - Reparent CPU clk to OSC clk
+ * - Reset CPU clock (very important)
+ * - Reconfigure CPU DIV
+ * - Reparent CPU clk back to CPU DIV clk
+ */
+
+ clk_set_parent(policy->clk, cpufreq->osc_clk);
+ __raw_writel(__raw_readl(LS1X_CLK_PLL_DIV) | RST_CPU_EN | RST_CPU,
+ LS1X_CLK_PLL_DIV);
+ __raw_writel(__raw_readl(LS1X_CLK_PLL_DIV) & ~(RST_CPU_EN | RST_CPU),
+ LS1X_CLK_PLL_DIV);
+ clk_set_rate(cpufreq->mux_clk, new_freq * 1000);
+ clk_set_parent(policy->clk, cpufreq->mux_clk);
+ dev_dbg(cpu_dev, "%u KHz --> %u KHz\n", old_freq, new_freq);
+
+ return 0;
+}
+
+static int ls1x_cpufreq_init(struct cpufreq_policy *policy)
+{
+ struct device *cpu_dev = get_cpu_device(policy->cpu);
+ struct cpufreq_frequency_table *freq_tbl;
+ unsigned int pll_freq, freq;
+ int steps, i, ret;
+
+ pll_freq = clk_get_rate(cpufreq->pll_clk) / 1000;
+
+ steps = 1 << DIV_CPU_WIDTH;
+ freq_tbl = kcalloc(steps, sizeof(*freq_tbl), GFP_KERNEL);
+ if (!freq_tbl)
+ return -ENOMEM;
+
+ for (i = 0; i < (steps - 1); i++) {
+ freq = pll_freq / (i + 1);
+ if ((freq < cpufreq->min_freq) || (freq > cpufreq->max_freq))
+ freq_tbl[i].frequency = CPUFREQ_ENTRY_INVALID;
+ else
+ freq_tbl[i].frequency = freq;
+ dev_dbg(cpu_dev,
+ "cpufreq table: index %d: frequency %d\n", i,
+ freq_tbl[i].frequency);
+ }
+ freq_tbl[i].frequency = CPUFREQ_TABLE_END;
+
+ policy->clk = cpufreq->clk;
+ ret = cpufreq_generic_init(policy, freq_tbl, 0);
+ if (ret)
+ kfree(freq_tbl);
+
+ return ret;
+}
+
+static int ls1x_cpufreq_exit(struct cpufreq_policy *policy)
+{
+ kfree(policy->freq_table);
+ return 0;
+}
+
+static struct cpufreq_driver ls1x_cpufreq_driver = {
+ .name = "cpufreq-ls1x",
+ .flags = CPUFREQ_STICKY | CPUFREQ_NEED_INITIAL_FREQ_CHECK,
+ .verify = cpufreq_generic_frequency_table_verify,
+ .target_index = ls1x_cpufreq_target,
+ .get = cpufreq_generic_get,
+ .init = ls1x_cpufreq_init,
+ .exit = ls1x_cpufreq_exit,
+ .attr = cpufreq_generic_attr,
+};
+
+static int ls1x_cpufreq_remove(struct platform_device *pdev)
+{
+ cpufreq_unregister_notifier(&ls1x_cpufreq_notifier_block,
+ CPUFREQ_TRANSITION_NOTIFIER);
+ cpufreq_unregister_driver(&ls1x_cpufreq_driver);
+
+ return 0;
+}
+
+static int ls1x_cpufreq_probe(struct platform_device *pdev)
+{
+ struct plat_ls1x_cpufreq *pdata = dev_get_platdata(&pdev->dev);
+ struct clk *clk;
+ int ret;
+
+ if (!pdata || !pdata->clk_name || !pdata->osc_clk_name) {
+ dev_err(&pdev->dev, "platform data missing\n");
+ return -EINVAL;
+ }
+
+ cpufreq =
+ devm_kzalloc(&pdev->dev, sizeof(struct ls1x_cpufreq), GFP_KERNEL);
+ if (!cpufreq)
+ return -ENOMEM;
+
+ cpufreq->dev = &pdev->dev;
+
+ clk = devm_clk_get(&pdev->dev, pdata->clk_name);
+ if (IS_ERR(clk)) {
+ dev_err(&pdev->dev, "unable to get %s clock\n",
+ pdata->clk_name);
+ return PTR_ERR(clk);
+ }
+ cpufreq->clk = clk;
+
+ clk = clk_get_parent(clk);
+ if (IS_ERR(clk)) {
+ dev_err(&pdev->dev, "unable to get parent of %s clock\n",
+ __clk_get_name(cpufreq->clk));
+ return PTR_ERR(clk);
+ }
+ cpufreq->mux_clk = clk;
+
+ clk = clk_get_parent(clk);
+ if (IS_ERR(clk)) {
+ dev_err(&pdev->dev, "unable to get parent of %s clock\n",
+ __clk_get_name(cpufreq->mux_clk));
+ return PTR_ERR(clk);
+ }
+ cpufreq->pll_clk = clk;
+
+ clk = devm_clk_get(&pdev->dev, pdata->osc_clk_name);
+ if (IS_ERR(clk)) {
+ dev_err(&pdev->dev, "unable to get %s clock\n",
+ pdata->osc_clk_name);
+ return PTR_ERR(clk);
+ }
+ cpufreq->osc_clk = clk;
+
+ cpufreq->max_freq = pdata->max_freq;
+ cpufreq->min_freq = pdata->min_freq;
+
+ ret = cpufreq_register_driver(&ls1x_cpufreq_driver);
+ if (ret) {
+ dev_err(&pdev->dev,
+ "failed to register CPUFreq driver: %d\n", ret);
+ return ret;
+ }
+
+ ret = cpufreq_register_notifier(&ls1x_cpufreq_notifier_block,
+ CPUFREQ_TRANSITION_NOTIFIER);
+
+ if (ret) {
+ dev_err(&pdev->dev,
+ "failed to register CPUFreq notifier: %d\n",ret);
+ cpufreq_unregister_driver(&ls1x_cpufreq_driver);
+ }
+
+ return ret;
+}
+
+static struct platform_driver ls1x_cpufreq_platdrv = {
+ .probe = ls1x_cpufreq_probe,
+ .remove = ls1x_cpufreq_remove,
+ .driver = {
+ .name = "ls1x-cpufreq",
+ },
+};
+
+module_platform_driver(ls1x_cpufreq_platdrv);
+
+MODULE_AUTHOR("Kelvin Cheung <keguang.zhang@gmail.com>");
+MODULE_DESCRIPTION("Loongson1 CPUFreq driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/cpufreq/ls1x-cpufreq.c b/drivers/cpufreq/ls1x-cpufreq.c
deleted file mode 100644
index 262581b..0000000
--- a/drivers/cpufreq/ls1x-cpufreq.c
+++ /dev/null
@@ -1,222 +0,0 @@
-/*
- * CPU Frequency Scaling for Loongson 1 SoC
- *
- * Copyright (C) 2014 Zhang, Keguang <keguang.zhang@gmail.com>
- *
- * This file is licensed under the terms of the GNU General Public
- * License version 2. This program is licensed "as is" without any
- * warranty of any kind, whether express or implied.
- */
-
-#include <linux/clk.h>
-#include <linux/clk-provider.h>
-#include <linux/cpu.h>
-#include <linux/cpufreq.h>
-#include <linux/delay.h>
-#include <linux/module.h>
-#include <linux/platform_device.h>
-#include <linux/slab.h>
-
-#include <cpufreq.h>
-#include <loongson1.h>
-
-static struct {
- struct device *dev;
- struct clk *clk; /* CPU clk */
- struct clk *mux_clk; /* MUX of CPU clk */
- struct clk *pll_clk; /* PLL clk */
- struct clk *osc_clk; /* OSC clk */
- unsigned int max_freq;
- unsigned int min_freq;
-} ls1x_cpufreq;
-
-static int ls1x_cpufreq_notifier(struct notifier_block *nb,
- unsigned long val, void *data)
-{
- if (val == CPUFREQ_POSTCHANGE)
- current_cpu_data.udelay_val = loops_per_jiffy;
-
- return NOTIFY_OK;
-}
-
-static struct notifier_block ls1x_cpufreq_notifier_block = {
- .notifier_call = ls1x_cpufreq_notifier
-};
-
-static int ls1x_cpufreq_target(struct cpufreq_policy *policy,
- unsigned int index)
-{
- unsigned int old_freq, new_freq;
-
- old_freq = policy->cur;
- new_freq = policy->freq_table[index].frequency;
-
- /*
- * The procedure of reconfiguring CPU clk is as below.
- *
- * - Reparent CPU clk to OSC clk
- * - Reset CPU clock (very important)
- * - Reconfigure CPU DIV
- * - Reparent CPU clk back to CPU DIV clk
- */
-
- dev_dbg(ls1x_cpufreq.dev, "%u KHz --> %u KHz\n", old_freq, new_freq);
- clk_set_parent(policy->clk, ls1x_cpufreq.osc_clk);
- __raw_writel(__raw_readl(LS1X_CLK_PLL_DIV) | RST_CPU_EN | RST_CPU,
- LS1X_CLK_PLL_DIV);
- __raw_writel(__raw_readl(LS1X_CLK_PLL_DIV) & ~(RST_CPU_EN | RST_CPU),
- LS1X_CLK_PLL_DIV);
- clk_set_rate(ls1x_cpufreq.mux_clk, new_freq * 1000);
- clk_set_parent(policy->clk, ls1x_cpufreq.mux_clk);
-
- return 0;
-}
-
-static int ls1x_cpufreq_init(struct cpufreq_policy *policy)
-{
- struct cpufreq_frequency_table *freq_tbl;
- unsigned int pll_freq, freq;
- int steps, i, ret;
-
- pll_freq = clk_get_rate(ls1x_cpufreq.pll_clk) / 1000;
-
- steps = 1 << DIV_CPU_WIDTH;
- freq_tbl = kzalloc(sizeof(*freq_tbl) * steps, GFP_KERNEL);
- if (!freq_tbl) {
- dev_err(ls1x_cpufreq.dev,
- "failed to alloc cpufreq_frequency_table\n");
- ret = -ENOMEM;
- goto out;
- }
-
- for (i = 0; i < (steps - 1); i++) {
- freq = pll_freq / (i + 1);
- if ((freq < ls1x_cpufreq.min_freq) ||
- (freq > ls1x_cpufreq.max_freq))
- freq_tbl[i].frequency = CPUFREQ_ENTRY_INVALID;
- else
- freq_tbl[i].frequency = freq;
- dev_dbg(ls1x_cpufreq.dev,
- "cpufreq table: index %d: frequency %d\n", i,
- freq_tbl[i].frequency);
- }
- freq_tbl[i].frequency = CPUFREQ_TABLE_END;
-
- policy->clk = ls1x_cpufreq.clk;
- ret = cpufreq_generic_init(policy, freq_tbl, 0);
- if (ret)
- kfree(freq_tbl);
-out:
- return ret;
-}
-
-static int ls1x_cpufreq_exit(struct cpufreq_policy *policy)
-{
- kfree(policy->freq_table);
- return 0;
-}
-
-static struct cpufreq_driver ls1x_cpufreq_driver = {
- .name = "cpufreq-ls1x",
- .flags = CPUFREQ_STICKY | CPUFREQ_NEED_INITIAL_FREQ_CHECK,
- .verify = cpufreq_generic_frequency_table_verify,
- .target_index = ls1x_cpufreq_target,
- .get = cpufreq_generic_get,
- .init = ls1x_cpufreq_init,
- .exit = ls1x_cpufreq_exit,
- .attr = cpufreq_generic_attr,
-};
-
-static int ls1x_cpufreq_remove(struct platform_device *pdev)
-{
- cpufreq_unregister_notifier(&ls1x_cpufreq_notifier_block,
- CPUFREQ_TRANSITION_NOTIFIER);
- cpufreq_unregister_driver(&ls1x_cpufreq_driver);
-
- return 0;
-}
-
-static int ls1x_cpufreq_probe(struct platform_device *pdev)
-{
- struct plat_ls1x_cpufreq *pdata = pdev->dev.platform_data;
- struct clk *clk;
- int ret;
-
- if (!pdata || !pdata->clk_name || !pdata->osc_clk_name)
- return -EINVAL;
-
- ls1x_cpufreq.dev = &pdev->dev;
-
- clk = devm_clk_get(&pdev->dev, pdata->clk_name);
- if (IS_ERR(clk)) {
- dev_err(ls1x_cpufreq.dev, "unable to get %s clock\n",
- pdata->clk_name);
- ret = PTR_ERR(clk);
- goto out;
- }
- ls1x_cpufreq.clk = clk;
-
- clk = clk_get_parent(clk);
- if (IS_ERR(clk)) {
- dev_err(ls1x_cpufreq.dev, "unable to get parent of %s clock\n",
- __clk_get_name(ls1x_cpufreq.clk));
- ret = PTR_ERR(clk);
- goto out;
- }
- ls1x_cpufreq.mux_clk = clk;
-
- clk = clk_get_parent(clk);
- if (IS_ERR(clk)) {
- dev_err(ls1x_cpufreq.dev, "unable to get parent of %s clock\n",
- __clk_get_name(ls1x_cpufreq.mux_clk));
- ret = PTR_ERR(clk);
- goto out;
- }
- ls1x_cpufreq.pll_clk = clk;
-
- clk = devm_clk_get(&pdev->dev, pdata->osc_clk_name);
- if (IS_ERR(clk)) {
- dev_err(ls1x_cpufreq.dev, "unable to get %s clock\n",
- pdata->osc_clk_name);
- ret = PTR_ERR(clk);
- goto out;
- }
- ls1x_cpufreq.osc_clk = clk;
-
- ls1x_cpufreq.max_freq = pdata->max_freq;
- ls1x_cpufreq.min_freq = pdata->min_freq;
-
- ret = cpufreq_register_driver(&ls1x_cpufreq_driver);
- if (ret) {
- dev_err(ls1x_cpufreq.dev,
- "failed to register cpufreq driver: %d\n", ret);
- goto out;
- }
-
- ret = cpufreq_register_notifier(&ls1x_cpufreq_notifier_block,
- CPUFREQ_TRANSITION_NOTIFIER);
-
- if (!ret)
- goto out;
-
- dev_err(ls1x_cpufreq.dev, "failed to register cpufreq notifier: %d\n",
- ret);
-
- cpufreq_unregister_driver(&ls1x_cpufreq_driver);
-out:
- return ret;
-}
-
-static struct platform_driver ls1x_cpufreq_platdrv = {
- .driver = {
- .name = "ls1x-cpufreq",
- },
- .probe = ls1x_cpufreq_probe,
- .remove = ls1x_cpufreq_remove,
-};
-
-module_platform_driver(ls1x_cpufreq_platdrv);
-
-MODULE_AUTHOR("Kelvin Cheung <keguang.zhang@gmail.com>");
-MODULE_DESCRIPTION("Loongson 1 CPUFreq driver");
-MODULE_LICENSE("GPL");
diff --git a/drivers/cpufreq/pmac32-cpufreq.c b/drivers/cpufreq/pmac32-cpufreq.c
index b7b576e..ff44016 100644
--- a/drivers/cpufreq/pmac32-cpufreq.c
+++ b/drivers/cpufreq/pmac32-cpufreq.c
@@ -300,7 +300,7 @@
_set_L3CR(save_l3cr);
/* Restore userland MMU context */
- switch_mmu_context(NULL, current->active_mm);
+ switch_mmu_context(NULL, current->active_mm, NULL);
#ifdef DEBUG_FREQ
printk(KERN_DEBUG "HID1, after: %x\n", mfspr(SPRN_HID1));
diff --git a/drivers/dax/Kconfig b/drivers/dax/Kconfig
new file mode 100644
index 0000000..cedab75
--- /dev/null
+++ b/drivers/dax/Kconfig
@@ -0,0 +1,26 @@
+menuconfig DEV_DAX
+ tristate "DAX: direct access to differentiated memory"
+ default m if NVDIMM_DAX
+ depends on TRANSPARENT_HUGEPAGE
+ help
+ Support raw access to differentiated (persistence, bandwidth,
+ latency...) memory via an mmap(2) capable character
+ device. Platform firmware or a device driver may identify a
+ platform memory resource that is differentiated from the
+ baseline memory pool. Mappings of a /dev/daxX.Y device impose
+ restrictions that make the mapping behavior deterministic.
+
+if DEV_DAX
+
+config DEV_DAX_PMEM
+ tristate "PMEM DAX: direct access to persistent memory"
+ depends on NVDIMM_DAX
+ default DEV_DAX
+ help
+ Support raw access to persistent memory. Note that this
+ driver consumes memory ranges allocated and exported by the
+ libnvdimm sub-system.
+
+ Say Y if unsure
+
+endif
diff --git a/drivers/dax/Makefile b/drivers/dax/Makefile
new file mode 100644
index 0000000..27c54e3
--- /dev/null
+++ b/drivers/dax/Makefile
@@ -0,0 +1,4 @@
+obj-$(CONFIG_DEV_DAX) += dax.o
+obj-$(CONFIG_DEV_DAX_PMEM) += dax_pmem.o
+
+dax_pmem-y := pmem.o
diff --git a/drivers/dax/dax.c b/drivers/dax/dax.c
new file mode 100644
index 0000000..b891a12
--- /dev/null
+++ b/drivers/dax/dax.c
@@ -0,0 +1,575 @@
+/*
+ * Copyright(c) 2016 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ */
+#include <linux/pagemap.h>
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/pfn_t.h>
+#include <linux/slab.h>
+#include <linux/dax.h>
+#include <linux/fs.h>
+#include <linux/mm.h>
+
+static int dax_major;
+static struct class *dax_class;
+static DEFINE_IDA(dax_minor_ida);
+
+/**
+ * struct dax_region - mapping infrastructure for dax devices
+ * @id: kernel-wide unique region for a memory range
+ * @base: linear address corresponding to @res
+ * @kref: to pin while other agents have a need to do lookups
+ * @dev: parent device backing this region
+ * @align: allocation and mapping alignment for child dax devices
+ * @res: physical address range of the region
+ * @pfn_flags: identify whether the pfns are paged back or not
+ */
+struct dax_region {
+ int id;
+ struct ida ida;
+ void *base;
+ struct kref kref;
+ struct device *dev;
+ unsigned int align;
+ struct resource res;
+ unsigned long pfn_flags;
+};
+
+/**
+ * struct dax_dev - subdivision of a dax region
+ * @region - parent region
+ * @dev - device backing the character device
+ * @kref - enable this data to be tracked in filp->private_data
+ * @alive - !alive + rcu grace period == no new mappings can be established
+ * @id - child id in the region
+ * @num_resources - number of physical address extents in this device
+ * @res - array of physical address ranges
+ */
+struct dax_dev {
+ struct dax_region *region;
+ struct device *dev;
+ struct kref kref;
+ bool alive;
+ int id;
+ int num_resources;
+ struct resource res[0];
+};
+
+static void dax_region_free(struct kref *kref)
+{
+ struct dax_region *dax_region;
+
+ dax_region = container_of(kref, struct dax_region, kref);
+ kfree(dax_region);
+}
+
+void dax_region_put(struct dax_region *dax_region)
+{
+ kref_put(&dax_region->kref, dax_region_free);
+}
+EXPORT_SYMBOL_GPL(dax_region_put);
+
+static void dax_dev_free(struct kref *kref)
+{
+ struct dax_dev *dax_dev;
+
+ dax_dev = container_of(kref, struct dax_dev, kref);
+ dax_region_put(dax_dev->region);
+ kfree(dax_dev);
+}
+
+static void dax_dev_put(struct dax_dev *dax_dev)
+{
+ kref_put(&dax_dev->kref, dax_dev_free);
+}
+
+struct dax_region *alloc_dax_region(struct device *parent, int region_id,
+ struct resource *res, unsigned int align, void *addr,
+ unsigned long pfn_flags)
+{
+ struct dax_region *dax_region;
+
+ dax_region = kzalloc(sizeof(*dax_region), GFP_KERNEL);
+
+ if (!dax_region)
+ return NULL;
+
+ memcpy(&dax_region->res, res, sizeof(*res));
+ dax_region->pfn_flags = pfn_flags;
+ kref_init(&dax_region->kref);
+ dax_region->id = region_id;
+ ida_init(&dax_region->ida);
+ dax_region->align = align;
+ dax_region->dev = parent;
+ dax_region->base = addr;
+
+ return dax_region;
+}
+EXPORT_SYMBOL_GPL(alloc_dax_region);
+
+static ssize_t size_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct dax_dev *dax_dev = dev_get_drvdata(dev);
+ unsigned long long size = 0;
+ int i;
+
+ for (i = 0; i < dax_dev->num_resources; i++)
+ size += resource_size(&dax_dev->res[i]);
+
+ return sprintf(buf, "%llu\n", size);
+}
+static DEVICE_ATTR_RO(size);
+
+static struct attribute *dax_device_attributes[] = {
+ &dev_attr_size.attr,
+ NULL,
+};
+
+static const struct attribute_group dax_device_attribute_group = {
+ .attrs = dax_device_attributes,
+};
+
+static const struct attribute_group *dax_attribute_groups[] = {
+ &dax_device_attribute_group,
+ NULL,
+};
+
+static void unregister_dax_dev(void *_dev)
+{
+ struct device *dev = _dev;
+ struct dax_dev *dax_dev = dev_get_drvdata(dev);
+ struct dax_region *dax_region = dax_dev->region;
+
+ dev_dbg(dev, "%s\n", __func__);
+
+ /*
+ * Note, rcu is not protecting the liveness of dax_dev, rcu is
+ * ensuring that any fault handlers that might have seen
+ * dax_dev->alive == true, have completed. Any fault handlers
+ * that start after synchronize_rcu() has started will abort
+ * upon seeing dax_dev->alive == false.
+ */
+ dax_dev->alive = false;
+ synchronize_rcu();
+
+ get_device(dev);
+ device_unregister(dev);
+ ida_simple_remove(&dax_region->ida, dax_dev->id);
+ ida_simple_remove(&dax_minor_ida, MINOR(dev->devt));
+ put_device(dev);
+ dax_dev_put(dax_dev);
+}
+
+int devm_create_dax_dev(struct dax_region *dax_region, struct resource *res,
+ int count)
+{
+ struct device *parent = dax_region->dev;
+ struct dax_dev *dax_dev;
+ struct device *dev;
+ int rc, minor;
+ dev_t dev_t;
+
+ dax_dev = kzalloc(sizeof(*dax_dev) + sizeof(*res) * count, GFP_KERNEL);
+ if (!dax_dev)
+ return -ENOMEM;
+ memcpy(dax_dev->res, res, sizeof(*res) * count);
+ dax_dev->num_resources = count;
+ kref_init(&dax_dev->kref);
+ dax_dev->alive = true;
+ dax_dev->region = dax_region;
+ kref_get(&dax_region->kref);
+
+ dax_dev->id = ida_simple_get(&dax_region->ida, 0, 0, GFP_KERNEL);
+ if (dax_dev->id < 0) {
+ rc = dax_dev->id;
+ goto err_id;
+ }
+
+ minor = ida_simple_get(&dax_minor_ida, 0, 0, GFP_KERNEL);
+ if (minor < 0) {
+ rc = minor;
+ goto err_minor;
+ }
+
+ dev_t = MKDEV(dax_major, minor);
+ dev = device_create_with_groups(dax_class, parent, dev_t, dax_dev,
+ dax_attribute_groups, "dax%d.%d", dax_region->id,
+ dax_dev->id);
+ if (IS_ERR(dev)) {
+ rc = PTR_ERR(dev);
+ goto err_create;
+ }
+ dax_dev->dev = dev;
+
+ rc = devm_add_action(dax_region->dev, unregister_dax_dev, dev);
+ if (rc) {
+ unregister_dax_dev(dev);
+ return rc;
+ }
+
+ return 0;
+
+ err_create:
+ ida_simple_remove(&dax_minor_ida, minor);
+ err_minor:
+ ida_simple_remove(&dax_region->ida, dax_dev->id);
+ err_id:
+ dax_dev_put(dax_dev);
+
+ return rc;
+}
+EXPORT_SYMBOL_GPL(devm_create_dax_dev);
+
+/* return an unmapped area aligned to the dax region specified alignment */
+static unsigned long dax_dev_get_unmapped_area(struct file *filp,
+ unsigned long addr, unsigned long len, unsigned long pgoff,
+ unsigned long flags)
+{
+ unsigned long off, off_end, off_align, len_align, addr_align, align;
+ struct dax_dev *dax_dev = filp ? filp->private_data : NULL;
+ struct dax_region *dax_region;
+
+ if (!dax_dev || addr)
+ goto out;
+
+ dax_region = dax_dev->region;
+ align = dax_region->align;
+ off = pgoff << PAGE_SHIFT;
+ off_end = off + len;
+ off_align = round_up(off, align);
+
+ if ((off_end <= off_align) || ((off_end - off_align) < align))
+ goto out;
+
+ len_align = len + align;
+ if ((off + len_align) < off)
+ goto out;
+
+ addr_align = current->mm->get_unmapped_area(filp, addr, len_align,
+ pgoff, flags);
+ if (!IS_ERR_VALUE(addr_align)) {
+ addr_align += (off - addr_align) & (align - 1);
+ return addr_align;
+ }
+ out:
+ return current->mm->get_unmapped_area(filp, addr, len, pgoff, flags);
+}
+
+static int __match_devt(struct device *dev, const void *data)
+{
+ const dev_t *devt = data;
+
+ return dev->devt == *devt;
+}
+
+static struct device *dax_dev_find(dev_t dev_t)
+{
+ return class_find_device(dax_class, NULL, &dev_t, __match_devt);
+}
+
+static int dax_dev_open(struct inode *inode, struct file *filp)
+{
+ struct dax_dev *dax_dev = NULL;
+ struct device *dev;
+
+ dev = dax_dev_find(inode->i_rdev);
+ if (!dev)
+ return -ENXIO;
+
+ device_lock(dev);
+ dax_dev = dev_get_drvdata(dev);
+ if (dax_dev) {
+ dev_dbg(dev, "%s\n", __func__);
+ filp->private_data = dax_dev;
+ kref_get(&dax_dev->kref);
+ inode->i_flags = S_DAX;
+ }
+ device_unlock(dev);
+
+ if (!dax_dev) {
+ put_device(dev);
+ return -ENXIO;
+ }
+ return 0;
+}
+
+static int dax_dev_release(struct inode *inode, struct file *filp)
+{
+ struct dax_dev *dax_dev = filp->private_data;
+ struct device *dev = dax_dev->dev;
+
+ dev_dbg(dax_dev->dev, "%s\n", __func__);
+ dax_dev_put(dax_dev);
+ put_device(dev);
+
+ return 0;
+}
+
+static int check_vma(struct dax_dev *dax_dev, struct vm_area_struct *vma,
+ const char *func)
+{
+ struct dax_region *dax_region = dax_dev->region;
+ struct device *dev = dax_dev->dev;
+ unsigned long mask;
+
+ if (!dax_dev->alive)
+ return -ENXIO;
+
+ /* prevent private / writable mappings from being established */
+ if ((vma->vm_flags & (VM_NORESERVE|VM_SHARED|VM_WRITE)) == VM_WRITE) {
+ dev_info(dev, "%s: %s: fail, attempted private mapping\n",
+ current->comm, func);
+ return -EINVAL;
+ }
+
+ mask = dax_region->align - 1;
+ if (vma->vm_start & mask || vma->vm_end & mask) {
+ dev_info(dev, "%s: %s: fail, unaligned vma (%#lx - %#lx, %#lx)\n",
+ current->comm, func, vma->vm_start, vma->vm_end,
+ mask);
+ return -EINVAL;
+ }
+
+ if ((dax_region->pfn_flags & (PFN_DEV|PFN_MAP)) == PFN_DEV
+ && (vma->vm_flags & VM_DONTCOPY) == 0) {
+ dev_info(dev, "%s: %s: fail, dax range requires MADV_DONTFORK\n",
+ current->comm, func);
+ return -EINVAL;
+ }
+
+ if (!vma_is_dax(vma)) {
+ dev_info(dev, "%s: %s: fail, vma is not DAX capable\n",
+ current->comm, func);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static phys_addr_t pgoff_to_phys(struct dax_dev *dax_dev, pgoff_t pgoff,
+ unsigned long size)
+{
+ struct resource *res;
+ phys_addr_t phys;
+ int i;
+
+ for (i = 0; i < dax_dev->num_resources; i++) {
+ res = &dax_dev->res[i];
+ phys = pgoff * PAGE_SIZE + res->start;
+ if (phys >= res->start && phys <= res->end)
+ break;
+ pgoff -= PHYS_PFN(resource_size(res));
+ }
+
+ if (i < dax_dev->num_resources) {
+ res = &dax_dev->res[i];
+ if (phys + size - 1 <= res->end)
+ return phys;
+ }
+
+ return -1;
+}
+
+static int __dax_dev_fault(struct dax_dev *dax_dev, struct vm_area_struct *vma,
+ struct vm_fault *vmf)
+{
+ unsigned long vaddr = (unsigned long) vmf->virtual_address;
+ struct device *dev = dax_dev->dev;
+ struct dax_region *dax_region;
+ int rc = VM_FAULT_SIGBUS;
+ phys_addr_t phys;
+ pfn_t pfn;
+
+ if (check_vma(dax_dev, vma, __func__))
+ return VM_FAULT_SIGBUS;
+
+ dax_region = dax_dev->region;
+ if (dax_region->align > PAGE_SIZE) {
+ dev_dbg(dev, "%s: alignment > fault size\n", __func__);
+ return VM_FAULT_SIGBUS;
+ }
+
+ phys = pgoff_to_phys(dax_dev, vmf->pgoff, PAGE_SIZE);
+ if (phys == -1) {
+ dev_dbg(dev, "%s: phys_to_pgoff(%#lx) failed\n", __func__,
+ vmf->pgoff);
+ return VM_FAULT_SIGBUS;
+ }
+
+ pfn = phys_to_pfn_t(phys, dax_region->pfn_flags);
+
+ rc = vm_insert_mixed(vma, vaddr, pfn);
+
+ if (rc == -ENOMEM)
+ return VM_FAULT_OOM;
+ if (rc < 0 && rc != -EBUSY)
+ return VM_FAULT_SIGBUS;
+
+ return VM_FAULT_NOPAGE;
+}
+
+static int dax_dev_fault(struct vm_area_struct *vma, struct vm_fault *vmf)
+{
+ int rc;
+ struct file *filp = vma->vm_file;
+ struct dax_dev *dax_dev = filp->private_data;
+
+ dev_dbg(dax_dev->dev, "%s: %s: %s (%#lx - %#lx)\n", __func__,
+ current->comm, (vmf->flags & FAULT_FLAG_WRITE)
+ ? "write" : "read", vma->vm_start, vma->vm_end);
+ rcu_read_lock();
+ rc = __dax_dev_fault(dax_dev, vma, vmf);
+ rcu_read_unlock();
+
+ return rc;
+}
+
+static int __dax_dev_pmd_fault(struct dax_dev *dax_dev,
+ struct vm_area_struct *vma, unsigned long addr, pmd_t *pmd,
+ unsigned int flags)
+{
+ unsigned long pmd_addr = addr & PMD_MASK;
+ struct device *dev = dax_dev->dev;
+ struct dax_region *dax_region;
+ phys_addr_t phys;
+ pgoff_t pgoff;
+ pfn_t pfn;
+
+ if (check_vma(dax_dev, vma, __func__))
+ return VM_FAULT_SIGBUS;
+
+ dax_region = dax_dev->region;
+ if (dax_region->align > PMD_SIZE) {
+ dev_dbg(dev, "%s: alignment > fault size\n", __func__);
+ return VM_FAULT_SIGBUS;
+ }
+
+ /* dax pmd mappings require pfn_t_devmap() */
+ if ((dax_region->pfn_flags & (PFN_DEV|PFN_MAP)) != (PFN_DEV|PFN_MAP)) {
+ dev_dbg(dev, "%s: alignment > fault size\n", __func__);
+ return VM_FAULT_SIGBUS;
+ }
+
+ pgoff = linear_page_index(vma, pmd_addr);
+ phys = pgoff_to_phys(dax_dev, pgoff, PAGE_SIZE);
+ if (phys == -1) {
+ dev_dbg(dev, "%s: phys_to_pgoff(%#lx) failed\n", __func__,
+ pgoff);
+ return VM_FAULT_SIGBUS;
+ }
+
+ pfn = phys_to_pfn_t(phys, dax_region->pfn_flags);
+
+ return vmf_insert_pfn_pmd(vma, addr, pmd, pfn,
+ flags & FAULT_FLAG_WRITE);
+}
+
+static int dax_dev_pmd_fault(struct vm_area_struct *vma, unsigned long addr,
+ pmd_t *pmd, unsigned int flags)
+{
+ int rc;
+ struct file *filp = vma->vm_file;
+ struct dax_dev *dax_dev = filp->private_data;
+
+ dev_dbg(dax_dev->dev, "%s: %s: %s (%#lx - %#lx)\n", __func__,
+ current->comm, (flags & FAULT_FLAG_WRITE)
+ ? "write" : "read", vma->vm_start, vma->vm_end);
+
+ rcu_read_lock();
+ rc = __dax_dev_pmd_fault(dax_dev, vma, addr, pmd, flags);
+ rcu_read_unlock();
+
+ return rc;
+}
+
+static void dax_dev_vm_open(struct vm_area_struct *vma)
+{
+ struct file *filp = vma->vm_file;
+ struct dax_dev *dax_dev = filp->private_data;
+
+ dev_dbg(dax_dev->dev, "%s\n", __func__);
+ kref_get(&dax_dev->kref);
+}
+
+static void dax_dev_vm_close(struct vm_area_struct *vma)
+{
+ struct file *filp = vma->vm_file;
+ struct dax_dev *dax_dev = filp->private_data;
+
+ dev_dbg(dax_dev->dev, "%s\n", __func__);
+ dax_dev_put(dax_dev);
+}
+
+static const struct vm_operations_struct dax_dev_vm_ops = {
+ .fault = dax_dev_fault,
+ .pmd_fault = dax_dev_pmd_fault,
+ .open = dax_dev_vm_open,
+ .close = dax_dev_vm_close,
+};
+
+static int dax_dev_mmap(struct file *filp, struct vm_area_struct *vma)
+{
+ struct dax_dev *dax_dev = filp->private_data;
+ int rc;
+
+ dev_dbg(dax_dev->dev, "%s\n", __func__);
+
+ rc = check_vma(dax_dev, vma, __func__);
+ if (rc)
+ return rc;
+
+ kref_get(&dax_dev->kref);
+ vma->vm_ops = &dax_dev_vm_ops;
+ vma->vm_flags |= VM_MIXEDMAP | VM_HUGEPAGE;
+ return 0;
+
+}
+
+static const struct file_operations dax_fops = {
+ .llseek = noop_llseek,
+ .owner = THIS_MODULE,
+ .open = dax_dev_open,
+ .release = dax_dev_release,
+ .get_unmapped_area = dax_dev_get_unmapped_area,
+ .mmap = dax_dev_mmap,
+};
+
+static int __init dax_init(void)
+{
+ int rc;
+
+ rc = register_chrdev(0, "dax", &dax_fops);
+ if (rc < 0)
+ return rc;
+ dax_major = rc;
+
+ dax_class = class_create(THIS_MODULE, "dax");
+ if (IS_ERR(dax_class)) {
+ unregister_chrdev(dax_major, "dax");
+ return PTR_ERR(dax_class);
+ }
+
+ return 0;
+}
+
+static void __exit dax_exit(void)
+{
+ class_destroy(dax_class);
+ unregister_chrdev(dax_major, "dax");
+ ida_destroy(&dax_minor_ida);
+}
+
+MODULE_AUTHOR("Intel Corporation");
+MODULE_LICENSE("GPL v2");
+subsys_initcall(dax_init);
+module_exit(dax_exit);
diff --git a/drivers/dax/dax.h b/drivers/dax/dax.h
new file mode 100644
index 0000000..d8b8f1f
--- /dev/null
+++ b/drivers/dax/dax.h
@@ -0,0 +1,24 @@
+/*
+ * Copyright(c) 2016 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ */
+#ifndef __DAX_H__
+#define __DAX_H__
+struct device;
+struct resource;
+struct dax_region;
+void dax_region_put(struct dax_region *dax_region);
+struct dax_region *alloc_dax_region(struct device *parent,
+ int region_id, struct resource *res, unsigned int align,
+ void *addr, unsigned long flags);
+int devm_create_dax_dev(struct dax_region *dax_region, struct resource *res,
+ int count);
+#endif /* __DAX_H__ */
diff --git a/drivers/dax/pmem.c b/drivers/dax/pmem.c
new file mode 100644
index 0000000..55d510e
--- /dev/null
+++ b/drivers/dax/pmem.c
@@ -0,0 +1,158 @@
+/*
+ * Copyright(c) 2016 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ */
+#include <linux/percpu-refcount.h>
+#include <linux/memremap.h>
+#include <linux/module.h>
+#include <linux/pfn_t.h>
+#include "../nvdimm/pfn.h"
+#include "../nvdimm/nd.h"
+#include "dax.h"
+
+struct dax_pmem {
+ struct device *dev;
+ struct percpu_ref ref;
+ struct completion cmp;
+};
+
+struct dax_pmem *to_dax_pmem(struct percpu_ref *ref)
+{
+ return container_of(ref, struct dax_pmem, ref);
+}
+
+static void dax_pmem_percpu_release(struct percpu_ref *ref)
+{
+ struct dax_pmem *dax_pmem = to_dax_pmem(ref);
+
+ dev_dbg(dax_pmem->dev, "%s\n", __func__);
+ complete(&dax_pmem->cmp);
+}
+
+static void dax_pmem_percpu_exit(void *data)
+{
+ struct percpu_ref *ref = data;
+ struct dax_pmem *dax_pmem = to_dax_pmem(ref);
+
+ dev_dbg(dax_pmem->dev, "%s\n", __func__);
+ percpu_ref_exit(ref);
+ wait_for_completion(&dax_pmem->cmp);
+}
+
+static void dax_pmem_percpu_kill(void *data)
+{
+ struct percpu_ref *ref = data;
+ struct dax_pmem *dax_pmem = to_dax_pmem(ref);
+
+ dev_dbg(dax_pmem->dev, "%s\n", __func__);
+ percpu_ref_kill(ref);
+}
+
+static int dax_pmem_probe(struct device *dev)
+{
+ int rc;
+ void *addr;
+ struct resource res;
+ struct nd_pfn_sb *pfn_sb;
+ struct dax_pmem *dax_pmem;
+ struct nd_region *nd_region;
+ struct nd_namespace_io *nsio;
+ struct dax_region *dax_region;
+ struct nd_namespace_common *ndns;
+ struct nd_dax *nd_dax = to_nd_dax(dev);
+ struct nd_pfn *nd_pfn = &nd_dax->nd_pfn;
+ struct vmem_altmap __altmap, *altmap = NULL;
+
+ ndns = nvdimm_namespace_common_probe(dev);
+ if (IS_ERR(ndns))
+ return PTR_ERR(ndns);
+ nsio = to_nd_namespace_io(&ndns->dev);
+
+ /* parse the 'pfn' info block via ->rw_bytes */
+ devm_nsio_enable(dev, nsio);
+ altmap = nvdimm_setup_pfn(nd_pfn, &res, &__altmap);
+ if (IS_ERR(altmap))
+ return PTR_ERR(altmap);
+ devm_nsio_disable(dev, nsio);
+
+ pfn_sb = nd_pfn->pfn_sb;
+
+ if (!devm_request_mem_region(dev, nsio->res.start,
+ resource_size(&nsio->res), dev_name(dev))) {
+ dev_warn(dev, "could not reserve region %pR\n", &nsio->res);
+ return -EBUSY;
+ }
+
+ dax_pmem = devm_kzalloc(dev, sizeof(*dax_pmem), GFP_KERNEL);
+ if (!dax_pmem)
+ return -ENOMEM;
+
+ dax_pmem->dev = dev;
+ init_completion(&dax_pmem->cmp);
+ rc = percpu_ref_init(&dax_pmem->ref, dax_pmem_percpu_release, 0,
+ GFP_KERNEL);
+ if (rc)
+ return rc;
+
+ rc = devm_add_action(dev, dax_pmem_percpu_exit, &dax_pmem->ref);
+ if (rc) {
+ dax_pmem_percpu_exit(&dax_pmem->ref);
+ return rc;
+ }
+
+ addr = devm_memremap_pages(dev, &res, &dax_pmem->ref, altmap);
+ if (IS_ERR(addr))
+ return PTR_ERR(addr);
+
+ rc = devm_add_action(dev, dax_pmem_percpu_kill, &dax_pmem->ref);
+ if (rc) {
+ dax_pmem_percpu_kill(&dax_pmem->ref);
+ return rc;
+ }
+
+ nd_region = to_nd_region(dev->parent);
+ dax_region = alloc_dax_region(dev, nd_region->id, &res,
+ le32_to_cpu(pfn_sb->align), addr, PFN_DEV|PFN_MAP);
+ if (!dax_region)
+ return -ENOMEM;
+
+ /* TODO: support for subdividing a dax region... */
+ rc = devm_create_dax_dev(dax_region, &res, 1);
+
+ /* child dax_dev instances now own the lifetime of the dax_region */
+ dax_region_put(dax_region);
+
+ return rc;
+}
+
+static struct nd_device_driver dax_pmem_driver = {
+ .probe = dax_pmem_probe,
+ .drv = {
+ .name = "dax_pmem",
+ },
+ .type = ND_DRIVER_DAX_PMEM,
+};
+
+static int __init dax_pmem_init(void)
+{
+ return nd_driver_register(&dax_pmem_driver);
+}
+module_init(dax_pmem_init);
+
+static void __exit dax_pmem_exit(void)
+{
+ driver_unregister(&dax_pmem_driver.drv);
+}
+module_exit(dax_pmem_exit);
+
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("Intel Corporation");
+MODULE_ALIAS_ND_DEVICE(ND_DEVICE_DAX_PMEM);
diff --git a/drivers/dma-buf/Kconfig b/drivers/dma-buf/Kconfig
new file mode 100644
index 0000000..9824bc4
--- /dev/null
+++ b/drivers/dma-buf/Kconfig
@@ -0,0 +1,11 @@
+menu "DMABUF options"
+
+config SYNC_FILE
+ bool "sync_file support for fences"
+ default n
+ select ANON_INODES
+ select DMA_SHARED_BUFFER
+ ---help---
+ This option enables the fence framework synchronization to export
+ sync_files to userspace that can represent one or more fences.
+endmenu
diff --git a/drivers/dma-buf/Makefile b/drivers/dma-buf/Makefile
index 57a675f..4a424ec 100644
--- a/drivers/dma-buf/Makefile
+++ b/drivers/dma-buf/Makefile
@@ -1 +1,2 @@
obj-y := dma-buf.o fence.o reservation.o seqno-fence.o
+obj-$(CONFIG_SYNC_FILE) += sync_file.o
diff --git a/drivers/dma-buf/sync_file.c b/drivers/dma-buf/sync_file.c
new file mode 100644
index 0000000..f08cf2d
--- /dev/null
+++ b/drivers/dma-buf/sync_file.c
@@ -0,0 +1,395 @@
+/*
+ * drivers/dma-buf/sync_file.c
+ *
+ * Copyright (C) 2012 Google, Inc.
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include <linux/export.h>
+#include <linux/file.h>
+#include <linux/fs.h>
+#include <linux/kernel.h>
+#include <linux/poll.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/uaccess.h>
+#include <linux/anon_inodes.h>
+#include <linux/sync_file.h>
+#include <uapi/linux/sync_file.h>
+
+static const struct file_operations sync_file_fops;
+
+static struct sync_file *sync_file_alloc(int size)
+{
+ struct sync_file *sync_file;
+
+ sync_file = kzalloc(size, GFP_KERNEL);
+ if (!sync_file)
+ return NULL;
+
+ sync_file->file = anon_inode_getfile("sync_file", &sync_file_fops,
+ sync_file, 0);
+ if (IS_ERR(sync_file->file))
+ goto err;
+
+ kref_init(&sync_file->kref);
+
+ init_waitqueue_head(&sync_file->wq);
+
+ return sync_file;
+
+err:
+ kfree(sync_file);
+ return NULL;
+}
+
+static void fence_check_cb_func(struct fence *f, struct fence_cb *cb)
+{
+ struct sync_file_cb *check;
+ struct sync_file *sync_file;
+
+ check = container_of(cb, struct sync_file_cb, cb);
+ sync_file = check->sync_file;
+
+ if (atomic_dec_and_test(&sync_file->status))
+ wake_up_all(&sync_file->wq);
+}
+
+/**
+ * sync_file_create() - creates a sync file
+ * @fence: fence to add to the sync_fence
+ *
+ * Creates a sync_file containg @fence. Once this is called, the sync_file
+ * takes ownership of @fence. The sync_file can be released with
+ * fput(sync_file->file). Returns the sync_file or NULL in case of error.
+ */
+struct sync_file *sync_file_create(struct fence *fence)
+{
+ struct sync_file *sync_file;
+
+ sync_file = sync_file_alloc(offsetof(struct sync_file, cbs[1]));
+ if (!sync_file)
+ return NULL;
+
+ sync_file->num_fences = 1;
+ atomic_set(&sync_file->status, 1);
+ snprintf(sync_file->name, sizeof(sync_file->name), "%s-%s%d-%d",
+ fence->ops->get_driver_name(fence),
+ fence->ops->get_timeline_name(fence), fence->context,
+ fence->seqno);
+
+ sync_file->cbs[0].fence = fence;
+ sync_file->cbs[0].sync_file = sync_file;
+ if (fence_add_callback(fence, &sync_file->cbs[0].cb,
+ fence_check_cb_func))
+ atomic_dec(&sync_file->status);
+
+ return sync_file;
+}
+EXPORT_SYMBOL(sync_file_create);
+
+/**
+ * sync_file_fdget() - get a sync_file from an fd
+ * @fd: fd referencing a fence
+ *
+ * Ensures @fd references a valid sync_file, increments the refcount of the
+ * backing file. Returns the sync_file or NULL in case of error.
+ */
+static struct sync_file *sync_file_fdget(int fd)
+{
+ struct file *file = fget(fd);
+
+ if (!file)
+ return NULL;
+
+ if (file->f_op != &sync_file_fops)
+ goto err;
+
+ return file->private_data;
+
+err:
+ fput(file);
+ return NULL;
+}
+
+static void sync_file_add_pt(struct sync_file *sync_file, int *i,
+ struct fence *fence)
+{
+ sync_file->cbs[*i].fence = fence;
+ sync_file->cbs[*i].sync_file = sync_file;
+
+ if (!fence_add_callback(fence, &sync_file->cbs[*i].cb,
+ fence_check_cb_func)) {
+ fence_get(fence);
+ (*i)++;
+ }
+}
+
+/**
+ * sync_file_merge() - merge two sync_files
+ * @name: name of new fence
+ * @a: sync_file a
+ * @b: sync_file b
+ *
+ * Creates a new sync_file which contains copies of all the fences in both
+ * @a and @b. @a and @b remain valid, independent sync_file. Returns the
+ * new merged sync_file or NULL in case of error.
+ */
+static struct sync_file *sync_file_merge(const char *name, struct sync_file *a,
+ struct sync_file *b)
+{
+ int num_fences = a->num_fences + b->num_fences;
+ struct sync_file *sync_file;
+ int i, i_a, i_b;
+ unsigned long size = offsetof(struct sync_file, cbs[num_fences]);
+
+ sync_file = sync_file_alloc(size);
+ if (!sync_file)
+ return NULL;
+
+ atomic_set(&sync_file->status, num_fences);
+
+ /*
+ * Assume sync_file a and b are both ordered and have no
+ * duplicates with the same context.
+ *
+ * If a sync_file can only be created with sync_file_merge
+ * and sync_file_create, this is a reasonable assumption.
+ */
+ for (i = i_a = i_b = 0; i_a < a->num_fences && i_b < b->num_fences; ) {
+ struct fence *pt_a = a->cbs[i_a].fence;
+ struct fence *pt_b = b->cbs[i_b].fence;
+
+ if (pt_a->context < pt_b->context) {
+ sync_file_add_pt(sync_file, &i, pt_a);
+
+ i_a++;
+ } else if (pt_a->context > pt_b->context) {
+ sync_file_add_pt(sync_file, &i, pt_b);
+
+ i_b++;
+ } else {
+ if (pt_a->seqno - pt_b->seqno <= INT_MAX)
+ sync_file_add_pt(sync_file, &i, pt_a);
+ else
+ sync_file_add_pt(sync_file, &i, pt_b);
+
+ i_a++;
+ i_b++;
+ }
+ }
+
+ for (; i_a < a->num_fences; i_a++)
+ sync_file_add_pt(sync_file, &i, a->cbs[i_a].fence);
+
+ for (; i_b < b->num_fences; i_b++)
+ sync_file_add_pt(sync_file, &i, b->cbs[i_b].fence);
+
+ if (num_fences > i)
+ atomic_sub(num_fences - i, &sync_file->status);
+ sync_file->num_fences = i;
+
+ strlcpy(sync_file->name, name, sizeof(sync_file->name));
+ return sync_file;
+}
+
+static void sync_file_free(struct kref *kref)
+{
+ struct sync_file *sync_file = container_of(kref, struct sync_file,
+ kref);
+ int i;
+
+ for (i = 0; i < sync_file->num_fences; ++i) {
+ fence_remove_callback(sync_file->cbs[i].fence,
+ &sync_file->cbs[i].cb);
+ fence_put(sync_file->cbs[i].fence);
+ }
+
+ kfree(sync_file);
+}
+
+static int sync_file_release(struct inode *inode, struct file *file)
+{
+ struct sync_file *sync_file = file->private_data;
+
+ kref_put(&sync_file->kref, sync_file_free);
+ return 0;
+}
+
+static unsigned int sync_file_poll(struct file *file, poll_table *wait)
+{
+ struct sync_file *sync_file = file->private_data;
+ int status;
+
+ poll_wait(file, &sync_file->wq, wait);
+
+ status = atomic_read(&sync_file->status);
+
+ if (!status)
+ return POLLIN;
+ if (status < 0)
+ return POLLERR;
+ return 0;
+}
+
+static long sync_file_ioctl_merge(struct sync_file *sync_file,
+ unsigned long arg)
+{
+ int fd = get_unused_fd_flags(O_CLOEXEC);
+ int err;
+ struct sync_file *fence2, *fence3;
+ struct sync_merge_data data;
+
+ if (fd < 0)
+ return fd;
+
+ if (copy_from_user(&data, (void __user *)arg, sizeof(data))) {
+ err = -EFAULT;
+ goto err_put_fd;
+ }
+
+ if (data.flags || data.pad) {
+ err = -EINVAL;
+ goto err_put_fd;
+ }
+
+ fence2 = sync_file_fdget(data.fd2);
+ if (!fence2) {
+ err = -ENOENT;
+ goto err_put_fd;
+ }
+
+ data.name[sizeof(data.name) - 1] = '\0';
+ fence3 = sync_file_merge(data.name, sync_file, fence2);
+ if (!fence3) {
+ err = -ENOMEM;
+ goto err_put_fence2;
+ }
+
+ data.fence = fd;
+ if (copy_to_user((void __user *)arg, &data, sizeof(data))) {
+ err = -EFAULT;
+ goto err_put_fence3;
+ }
+
+ fd_install(fd, fence3->file);
+ fput(fence2->file);
+ return 0;
+
+err_put_fence3:
+ fput(fence3->file);
+
+err_put_fence2:
+ fput(fence2->file);
+
+err_put_fd:
+ put_unused_fd(fd);
+ return err;
+}
+
+static void sync_fill_fence_info(struct fence *fence,
+ struct sync_fence_info *info)
+{
+ strlcpy(info->obj_name, fence->ops->get_timeline_name(fence),
+ sizeof(info->obj_name));
+ strlcpy(info->driver_name, fence->ops->get_driver_name(fence),
+ sizeof(info->driver_name));
+ if (fence_is_signaled(fence))
+ info->status = fence->status >= 0 ? 1 : fence->status;
+ else
+ info->status = 0;
+ info->timestamp_ns = ktime_to_ns(fence->timestamp);
+}
+
+static long sync_file_ioctl_fence_info(struct sync_file *sync_file,
+ unsigned long arg)
+{
+ struct sync_file_info info;
+ struct sync_fence_info *fence_info = NULL;
+ __u32 size;
+ int ret, i;
+
+ if (copy_from_user(&info, (void __user *)arg, sizeof(info)))
+ return -EFAULT;
+
+ if (info.flags || info.pad)
+ return -EINVAL;
+
+ /*
+ * Passing num_fences = 0 means that userspace doesn't want to
+ * retrieve any sync_fence_info. If num_fences = 0 we skip filling
+ * sync_fence_info and return the actual number of fences on
+ * info->num_fences.
+ */
+ if (!info.num_fences)
+ goto no_fences;
+
+ if (info.num_fences < sync_file->num_fences)
+ return -EINVAL;
+
+ size = sync_file->num_fences * sizeof(*fence_info);
+ fence_info = kzalloc(size, GFP_KERNEL);
+ if (!fence_info)
+ return -ENOMEM;
+
+ for (i = 0; i < sync_file->num_fences; ++i)
+ sync_fill_fence_info(sync_file->cbs[i].fence, &fence_info[i]);
+
+ if (copy_to_user(u64_to_user_ptr(info.sync_fence_info), fence_info,
+ size)) {
+ ret = -EFAULT;
+ goto out;
+ }
+
+no_fences:
+ strlcpy(info.name, sync_file->name, sizeof(info.name));
+ info.status = atomic_read(&sync_file->status);
+ if (info.status >= 0)
+ info.status = !info.status;
+
+ info.num_fences = sync_file->num_fences;
+
+ if (copy_to_user((void __user *)arg, &info, sizeof(info)))
+ ret = -EFAULT;
+ else
+ ret = 0;
+
+out:
+ kfree(fence_info);
+
+ return ret;
+}
+
+static long sync_file_ioctl(struct file *file, unsigned int cmd,
+ unsigned long arg)
+{
+ struct sync_file *sync_file = file->private_data;
+
+ switch (cmd) {
+ case SYNC_IOC_MERGE:
+ return sync_file_ioctl_merge(sync_file, arg);
+
+ case SYNC_IOC_FILE_INFO:
+ return sync_file_ioctl_fence_info(sync_file, arg);
+
+ default:
+ return -ENOTTY;
+ }
+}
+
+static const struct file_operations sync_file_fops = {
+ .release = sync_file_release,
+ .poll = sync_file_poll,
+ .unlocked_ioctl = sync_file_ioctl,
+ .compat_ioctl = sync_file_ioctl,
+};
+
diff --git a/drivers/dma/Kconfig b/drivers/dma/Kconfig
index d96d87c..8c98779 100644
--- a/drivers/dma/Kconfig
+++ b/drivers/dma/Kconfig
@@ -332,7 +332,7 @@
config MV_XOR
bool "Marvell XOR engine support"
- depends on PLAT_ORION
+ depends on PLAT_ORION || ARCH_MVEBU || COMPILE_TEST
select DMA_ENGINE
select DMA_ENGINE_RAID
select ASYNC_TX_ENABLE_CHANNEL_SWITCH
@@ -467,6 +467,20 @@
This DMA controller transfers data from memory to peripheral fifo
or vice versa. It does not support memory to memory data transfer.
+config TEGRA210_ADMA
+ bool "NVIDIA Tegra210 ADMA support"
+ depends on ARCH_TEGRA_210_SOC
+ select DMA_ENGINE
+ select DMA_VIRTUAL_CHANNELS
+ select PM_CLK
+ help
+ Support for the NVIDIA Tegra210 ADMA controller driver. The
+ DMA controller has multiple DMA channels and is used to service
+ various audio clients in the Tegra210 audio processing engine
+ (APE). This DMA controller transfers data from memory to
+ peripheral and vice versa. It does not support memory to
+ memory data transfer.
+
config TIMB_DMA
tristate "Timberdale FPGA DMA support"
depends on MFD_TIMBERDALE
@@ -507,7 +521,7 @@
config XILINX_VDMA
tristate "Xilinx AXI VDMA Engine"
- depends on (ARCH_ZYNQ || MICROBLAZE)
+ depends on (ARCH_ZYNQ || MICROBLAZE || ARM64)
select DMA_ENGINE
help
Enable support for Xilinx AXI VDMA Soft IP.
diff --git a/drivers/dma/Makefile b/drivers/dma/Makefile
index 6084127..614f28b 100644
--- a/drivers/dma/Makefile
+++ b/drivers/dma/Makefile
@@ -59,6 +59,7 @@
obj-$(CONFIG_S3C24XX_DMAC) += s3c24xx-dma.o
obj-$(CONFIG_TXX9_DMAC) += txx9dmac.o
obj-$(CONFIG_TEGRA20_APB_DMA) += tegra20-apb-dma.o
+obj-$(CONFIG_TEGRA210_ADMA) += tegra210-adma.o
obj-$(CONFIG_TIMB_DMA) += timb_dma.o
obj-$(CONFIG_TI_CPPI41) += cppi41.o
obj-$(CONFIG_TI_DMA_CROSSBAR) += ti-dma-crossbar.o
diff --git a/drivers/dma/amba-pl08x.c b/drivers/dma/amba-pl08x.c
index 9b42c05..81db1c4 100644
--- a/drivers/dma/amba-pl08x.c
+++ b/drivers/dma/amba-pl08x.c
@@ -107,16 +107,20 @@
/**
* struct vendor_data - vendor-specific config parameters for PL08x derivatives
* @channels: the number of channels available in this variant
+ * @signals: the number of request signals available from the hardware
* @dualmaster: whether this version supports dual AHB masters or not.
* @nomadik: whether the channels have Nomadik security extension bits
* that need to be checked for permission before use and some registers are
* missing
* @pl080s: whether this version is a PL080S, which has separate register and
* LLI word for transfer size.
+ * @max_transfer_size: the maximum single element transfer size for this
+ * PL08x variant.
*/
struct vendor_data {
u8 config_offset;
u8 channels;
+ u8 signals;
bool dualmaster;
bool nomadik;
bool pl080s;
@@ -235,7 +239,7 @@
struct virt_dma_chan vc;
struct pl08x_phy_chan *phychan;
const char *name;
- const struct pl08x_channel_data *cd;
+ struct pl08x_channel_data *cd;
struct dma_slave_config cfg;
struct pl08x_txd *at;
struct pl08x_driver_data *host;
@@ -1909,6 +1913,12 @@
if (slave) {
chan->cd = &pl08x->pd->slave_channels[i];
+ /*
+ * Some implementations have muxed signals, whereas some
+ * use a mux in front of the signals and need dynamic
+ * assignment of signals.
+ */
+ chan->signal = i;
pl08x_dma_slave_init(chan);
} else {
chan->cd = &pl08x->pd->memcpy_channel;
@@ -2050,40 +2060,33 @@
struct of_dma *ofdma)
{
struct pl08x_driver_data *pl08x = ofdma->of_dma_data;
- struct pl08x_channel_data *data;
- struct pl08x_dma_chan *chan;
struct dma_chan *dma_chan;
+ struct pl08x_dma_chan *plchan;
if (!pl08x)
return NULL;
- if (dma_spec->args_count != 2)
+ if (dma_spec->args_count != 2) {
+ dev_err(&pl08x->adev->dev,
+ "DMA channel translation requires two cells\n");
return NULL;
+ }
dma_chan = pl08x_find_chan_id(pl08x, dma_spec->args[0]);
- if (dma_chan)
- return dma_get_slave_channel(dma_chan);
-
- chan = devm_kzalloc(pl08x->slave.dev, sizeof(*chan) + sizeof(*data),
- GFP_KERNEL);
- if (!chan)
+ if (!dma_chan) {
+ dev_err(&pl08x->adev->dev,
+ "DMA slave channel not found\n");
return NULL;
+ }
- data = (void *)&chan[1];
- data->bus_id = "(none)";
- data->periph_buses = dma_spec->args[1];
+ plchan = to_pl08x_chan(dma_chan);
+ dev_dbg(&pl08x->adev->dev,
+ "translated channel for signal %d\n",
+ dma_spec->args[0]);
- chan->cd = data;
- chan->host = pl08x;
- chan->slave = true;
- chan->name = data->bus_id;
- chan->state = PL08X_CHAN_IDLE;
- chan->signal = dma_spec->args[0];
- chan->vc.desc_free = pl08x_desc_free;
-
- vchan_init(&chan->vc, &pl08x->slave);
-
- return dma_get_slave_channel(&chan->vc.chan);
+ /* Augment channel data for applicable AHB buses */
+ plchan->cd->periph_buses = dma_spec->args[1];
+ return dma_get_slave_channel(dma_chan);
}
static int pl08x_of_probe(struct amba_device *adev,
@@ -2091,9 +2094,11 @@
struct device_node *np)
{
struct pl08x_platform_data *pd;
+ struct pl08x_channel_data *chanp = NULL;
u32 cctl_memcpy = 0;
u32 val;
int ret;
+ int i;
pd = devm_kzalloc(&adev->dev, sizeof(*pd), GFP_KERNEL);
if (!pd)
@@ -2195,6 +2200,27 @@
/* Use the buses that can access memory, obviously */
pd->memcpy_channel.periph_buses = pd->mem_buses;
+ /*
+ * Allocate channel data for all possible slave channels (one
+ * for each possible signal), channels will then be allocated
+ * for a device and have it's AHB interfaces set up at
+ * translation time.
+ */
+ chanp = devm_kcalloc(&adev->dev,
+ pl08x->vd->signals,
+ sizeof(struct pl08x_channel_data),
+ GFP_KERNEL);
+ if (!chanp)
+ return -ENOMEM;
+
+ pd->slave_channels = chanp;
+ for (i = 0; i < pl08x->vd->signals; i++) {
+ /* chanp->periph_buses will be assigned at translation */
+ chanp->bus_id = kasprintf(GFP_KERNEL, "slave%d", i);
+ chanp++;
+ }
+ pd->num_slave_channels = pl08x->vd->signals;
+
pl08x->pd = pd;
return of_dma_controller_register(adev->dev.of_node, pl08x_of_xlate,
@@ -2234,6 +2260,10 @@
goto out_no_pl08x;
}
+ /* Assign useful pointers to the driver state */
+ pl08x->adev = adev;
+ pl08x->vd = vd;
+
/* Initialize memcpy engine */
dma_cap_set(DMA_MEMCPY, pl08x->memcpy.cap_mask);
pl08x->memcpy.dev = &adev->dev;
@@ -2284,10 +2314,6 @@
}
}
- /* Assign useful pointers to the driver state */
- pl08x->adev = adev;
- pl08x->vd = vd;
-
/* By default, AHB1 only. If dualmaster, from platform */
pl08x->lli_buses = PL08X_AHB1;
pl08x->mem_buses = PL08X_AHB1;
@@ -2438,6 +2464,7 @@
static struct vendor_data vendor_pl080 = {
.config_offset = PL080_CH_CONFIG,
.channels = 8,
+ .signals = 16,
.dualmaster = true,
.max_transfer_size = PL080_CONTROL_TRANSFER_SIZE_MASK,
};
@@ -2445,6 +2472,7 @@
static struct vendor_data vendor_nomadik = {
.config_offset = PL080_CH_CONFIG,
.channels = 8,
+ .signals = 32,
.dualmaster = true,
.nomadik = true,
.max_transfer_size = PL080_CONTROL_TRANSFER_SIZE_MASK,
@@ -2453,6 +2481,7 @@
static struct vendor_data vendor_pl080s = {
.config_offset = PL080S_CH_CONFIG,
.channels = 8,
+ .signals = 32,
.pl080s = true,
.max_transfer_size = PL080S_CONTROL_TRANSFER_SIZE_MASK,
};
@@ -2460,6 +2489,7 @@
static struct vendor_data vendor_pl081 = {
.config_offset = PL080_CH_CONFIG,
.channels = 2,
+ .signals = 16,
.dualmaster = false,
.max_transfer_size = PL080_CONTROL_TRANSFER_SIZE_MASK,
};
diff --git a/drivers/dma/bcm2835-dma.c b/drivers/dma/bcm2835-dma.c
index 996c4b0..6149b27 100644
--- a/drivers/dma/bcm2835-dma.c
+++ b/drivers/dma/bcm2835-dma.c
@@ -46,6 +46,9 @@
#include "virt-dma.h"
+#define BCM2835_DMA_MAX_DMA_CHAN_SUPPORTED 14
+#define BCM2835_DMA_CHAN_NAME_SIZE 8
+
struct bcm2835_dmadev {
struct dma_device ddev;
spinlock_t lock;
@@ -73,7 +76,6 @@
struct list_head node;
struct dma_slave_config cfg;
- bool cyclic;
unsigned int dreq;
int ch;
@@ -82,6 +84,9 @@
void __iomem *chan_base;
int irq_number;
+ unsigned int irq_flags;
+
+ bool is_lite_channel;
};
struct bcm2835_desc {
@@ -89,47 +94,104 @@
struct virt_dma_desc vd;
enum dma_transfer_direction dir;
- struct bcm2835_cb_entry *cb_list;
-
unsigned int frames;
size_t size;
+
+ bool cyclic;
+
+ struct bcm2835_cb_entry cb_list[];
};
#define BCM2835_DMA_CS 0x00
#define BCM2835_DMA_ADDR 0x04
+#define BCM2835_DMA_TI 0x08
#define BCM2835_DMA_SOURCE_AD 0x0c
#define BCM2835_DMA_DEST_AD 0x10
-#define BCM2835_DMA_NEXTCB 0x1C
+#define BCM2835_DMA_LEN 0x14
+#define BCM2835_DMA_STRIDE 0x18
+#define BCM2835_DMA_NEXTCB 0x1c
+#define BCM2835_DMA_DEBUG 0x20
/* DMA CS Control and Status bits */
-#define BCM2835_DMA_ACTIVE BIT(0)
-#define BCM2835_DMA_INT BIT(2)
+#define BCM2835_DMA_ACTIVE BIT(0) /* activate the DMA */
+#define BCM2835_DMA_END BIT(1) /* current CB has ended */
+#define BCM2835_DMA_INT BIT(2) /* interrupt status */
+#define BCM2835_DMA_DREQ BIT(3) /* DREQ state */
#define BCM2835_DMA_ISPAUSED BIT(4) /* Pause requested or not active */
#define BCM2835_DMA_ISHELD BIT(5) /* Is held by DREQ flow control */
-#define BCM2835_DMA_ERR BIT(8)
+#define BCM2835_DMA_WAITING_FOR_WRITES BIT(6) /* waiting for last
+ * AXI-write to ack
+ */
+#define BCM2835_DMA_ERR BIT(8)
+#define BCM2835_DMA_PRIORITY(x) ((x & 15) << 16) /* AXI priority */
+#define BCM2835_DMA_PANIC_PRIORITY(x) ((x & 15) << 20) /* panic priority */
+/* current value of TI.BCM2835_DMA_WAIT_RESP */
+#define BCM2835_DMA_WAIT_FOR_WRITES BIT(28)
+#define BCM2835_DMA_DIS_DEBUG BIT(29) /* disable debug pause signal */
#define BCM2835_DMA_ABORT BIT(30) /* Stop current CB, go to next, WO */
#define BCM2835_DMA_RESET BIT(31) /* WO, self clearing */
+/* Transfer information bits - also bcm2835_cb.info field */
#define BCM2835_DMA_INT_EN BIT(0)
+#define BCM2835_DMA_TDMODE BIT(1) /* 2D-Mode */
+#define BCM2835_DMA_WAIT_RESP BIT(3) /* wait for AXI-write to be acked */
#define BCM2835_DMA_D_INC BIT(4)
-#define BCM2835_DMA_D_DREQ BIT(6)
+#define BCM2835_DMA_D_WIDTH BIT(5) /* 128bit writes if set */
+#define BCM2835_DMA_D_DREQ BIT(6) /* enable DREQ for destination */
+#define BCM2835_DMA_D_IGNORE BIT(7) /* ignore destination writes */
#define BCM2835_DMA_S_INC BIT(8)
-#define BCM2835_DMA_S_DREQ BIT(10)
+#define BCM2835_DMA_S_WIDTH BIT(9) /* 128bit writes if set */
+#define BCM2835_DMA_S_DREQ BIT(10) /* enable SREQ for source */
+#define BCM2835_DMA_S_IGNORE BIT(11) /* ignore source reads - read 0 */
+#define BCM2835_DMA_BURST_LENGTH(x) ((x & 15) << 12)
+#define BCM2835_DMA_PER_MAP(x) ((x & 31) << 16) /* REQ source */
+#define BCM2835_DMA_WAIT(x) ((x & 31) << 21) /* add DMA-wait cycles */
+#define BCM2835_DMA_NO_WIDE_BURSTS BIT(26) /* no 2 beat write bursts */
-#define BCM2835_DMA_PER_MAP(x) ((x) << 16)
+/* debug register bits */
+#define BCM2835_DMA_DEBUG_LAST_NOT_SET_ERR BIT(0)
+#define BCM2835_DMA_DEBUG_FIFO_ERR BIT(1)
+#define BCM2835_DMA_DEBUG_READ_ERR BIT(2)
+#define BCM2835_DMA_DEBUG_OUTSTANDING_WRITES_SHIFT 4
+#define BCM2835_DMA_DEBUG_OUTSTANDING_WRITES_BITS 4
+#define BCM2835_DMA_DEBUG_ID_SHIFT 16
+#define BCM2835_DMA_DEBUG_ID_BITS 9
+#define BCM2835_DMA_DEBUG_STATE_SHIFT 16
+#define BCM2835_DMA_DEBUG_STATE_BITS 9
+#define BCM2835_DMA_DEBUG_VERSION_SHIFT 25
+#define BCM2835_DMA_DEBUG_VERSION_BITS 3
+#define BCM2835_DMA_DEBUG_LITE BIT(28)
+
+/* shared registers for all dma channels */
+#define BCM2835_DMA_INT_STATUS 0xfe0
+#define BCM2835_DMA_ENABLE 0xff0
#define BCM2835_DMA_DATA_TYPE_S8 1
#define BCM2835_DMA_DATA_TYPE_S16 2
#define BCM2835_DMA_DATA_TYPE_S32 4
#define BCM2835_DMA_DATA_TYPE_S128 16
-#define BCM2835_DMA_BULK_MASK BIT(0)
-#define BCM2835_DMA_FIQ_MASK (BIT(2) | BIT(3))
-
/* Valid only for channels 0 - 14, 15 has its own base address */
#define BCM2835_DMA_CHAN(n) ((n) << 8) /* Base address */
#define BCM2835_DMA_CHANIO(base, n) ((base) + BCM2835_DMA_CHAN(n))
+/* the max dma length for different channels */
+#define MAX_DMA_LEN SZ_1G
+#define MAX_LITE_DMA_LEN (SZ_64K - 4)
+
+static inline size_t bcm2835_dma_max_frame_length(struct bcm2835_chan *c)
+{
+ /* lite and normal channels have different max frame length */
+ return c->is_lite_channel ? MAX_LITE_DMA_LEN : MAX_DMA_LEN;
+}
+
+/* how many frames of max_len size do we need to transfer len bytes */
+static inline size_t bcm2835_dma_frames_for_length(size_t len,
+ size_t max_len)
+{
+ return DIV_ROUND_UP(len, max_len);
+}
+
static inline struct bcm2835_dmadev *to_bcm2835_dma_dev(struct dma_device *d)
{
return container_of(d, struct bcm2835_dmadev, ddev);
@@ -146,19 +208,209 @@
return container_of(t, struct bcm2835_desc, vd.tx);
}
-static void bcm2835_dma_desc_free(struct virt_dma_desc *vd)
+static void bcm2835_dma_free_cb_chain(struct bcm2835_desc *desc)
{
- struct bcm2835_desc *desc = container_of(vd, struct bcm2835_desc, vd);
- int i;
+ size_t i;
for (i = 0; i < desc->frames; i++)
dma_pool_free(desc->c->cb_pool, desc->cb_list[i].cb,
desc->cb_list[i].paddr);
- kfree(desc->cb_list);
kfree(desc);
}
+static void bcm2835_dma_desc_free(struct virt_dma_desc *vd)
+{
+ bcm2835_dma_free_cb_chain(
+ container_of(vd, struct bcm2835_desc, vd));
+}
+
+static void bcm2835_dma_create_cb_set_length(
+ struct bcm2835_chan *chan,
+ struct bcm2835_dma_cb *control_block,
+ size_t len,
+ size_t period_len,
+ size_t *total_len,
+ u32 finalextrainfo)
+{
+ size_t max_len = bcm2835_dma_max_frame_length(chan);
+
+ /* set the length taking lite-channel limitations into account */
+ control_block->length = min_t(u32, len, max_len);
+
+ /* finished if we have no period_length */
+ if (!period_len)
+ return;
+
+ /*
+ * period_len means: that we need to generate
+ * transfers that are terminating at every
+ * multiple of period_len - this is typically
+ * used to set the interrupt flag in info
+ * which is required during cyclic transfers
+ */
+
+ /* have we filled in period_length yet? */
+ if (*total_len + control_block->length < period_len)
+ return;
+
+ /* calculate the length that remains to reach period_length */
+ control_block->length = period_len - *total_len;
+
+ /* reset total_length for next period */
+ *total_len = 0;
+
+ /* add extrainfo bits in info */
+ control_block->info |= finalextrainfo;
+}
+
+static inline size_t bcm2835_dma_count_frames_for_sg(
+ struct bcm2835_chan *c,
+ struct scatterlist *sgl,
+ unsigned int sg_len)
+{
+ size_t frames = 0;
+ struct scatterlist *sgent;
+ unsigned int i;
+ size_t plength = bcm2835_dma_max_frame_length(c);
+
+ for_each_sg(sgl, sgent, sg_len, i)
+ frames += bcm2835_dma_frames_for_length(
+ sg_dma_len(sgent), plength);
+
+ return frames;
+}
+
+/**
+ * bcm2835_dma_create_cb_chain - create a control block and fills data in
+ *
+ * @chan: the @dma_chan for which we run this
+ * @direction: the direction in which we transfer
+ * @cyclic: it is a cyclic transfer
+ * @info: the default info bits to apply per controlblock
+ * @frames: number of controlblocks to allocate
+ * @src: the src address to assign (if the S_INC bit is set
+ * in @info, then it gets incremented)
+ * @dst: the dst address to assign (if the D_INC bit is set
+ * in @info, then it gets incremented)
+ * @buf_len: the full buffer length (may also be 0)
+ * @period_len: the period length when to apply @finalextrainfo
+ * in addition to the last transfer
+ * this will also break some control-blocks early
+ * @finalextrainfo: additional bits in last controlblock
+ * (or when period_len is reached in case of cyclic)
+ * @gfp: the GFP flag to use for allocation
+ */
+static struct bcm2835_desc *bcm2835_dma_create_cb_chain(
+ struct dma_chan *chan, enum dma_transfer_direction direction,
+ bool cyclic, u32 info, u32 finalextrainfo, size_t frames,
+ dma_addr_t src, dma_addr_t dst, size_t buf_len,
+ size_t period_len, gfp_t gfp)
+{
+ struct bcm2835_chan *c = to_bcm2835_dma_chan(chan);
+ size_t len = buf_len, total_len;
+ size_t frame;
+ struct bcm2835_desc *d;
+ struct bcm2835_cb_entry *cb_entry;
+ struct bcm2835_dma_cb *control_block;
+
+ if (!frames)
+ return NULL;
+
+ /* allocate and setup the descriptor. */
+ d = kzalloc(sizeof(*d) + frames * sizeof(struct bcm2835_cb_entry),
+ gfp);
+ if (!d)
+ return NULL;
+
+ d->c = c;
+ d->dir = direction;
+ d->cyclic = cyclic;
+
+ /*
+ * Iterate over all frames, create a control block
+ * for each frame and link them together.
+ */
+ for (frame = 0, total_len = 0; frame < frames; d->frames++, frame++) {
+ cb_entry = &d->cb_list[frame];
+ cb_entry->cb = dma_pool_alloc(c->cb_pool, gfp,
+ &cb_entry->paddr);
+ if (!cb_entry->cb)
+ goto error_cb;
+
+ /* fill in the control block */
+ control_block = cb_entry->cb;
+ control_block->info = info;
+ control_block->src = src;
+ control_block->dst = dst;
+ control_block->stride = 0;
+ control_block->next = 0;
+ /* set up length in control_block if requested */
+ if (buf_len) {
+ /* calculate length honoring period_length */
+ bcm2835_dma_create_cb_set_length(
+ c, control_block,
+ len, period_len, &total_len,
+ cyclic ? finalextrainfo : 0);
+
+ /* calculate new remaining length */
+ len -= control_block->length;
+ }
+
+ /* link this the last controlblock */
+ if (frame)
+ d->cb_list[frame - 1].cb->next = cb_entry->paddr;
+
+ /* update src and dst and length */
+ if (src && (info & BCM2835_DMA_S_INC))
+ src += control_block->length;
+ if (dst && (info & BCM2835_DMA_D_INC))
+ dst += control_block->length;
+
+ /* Length of total transfer */
+ d->size += control_block->length;
+ }
+
+ /* the last frame requires extra flags */
+ d->cb_list[d->frames - 1].cb->info |= finalextrainfo;
+
+ /* detect a size missmatch */
+ if (buf_len && (d->size != buf_len))
+ goto error_cb;
+
+ return d;
+error_cb:
+ bcm2835_dma_free_cb_chain(d);
+
+ return NULL;
+}
+
+static void bcm2835_dma_fill_cb_chain_with_sg(
+ struct dma_chan *chan,
+ enum dma_transfer_direction direction,
+ struct bcm2835_cb_entry *cb,
+ struct scatterlist *sgl,
+ unsigned int sg_len)
+{
+ struct bcm2835_chan *c = to_bcm2835_dma_chan(chan);
+ size_t max_len = bcm2835_dma_max_frame_length(c);
+ unsigned int i, len;
+ dma_addr_t addr;
+ struct scatterlist *sgent;
+
+ for_each_sg(sgl, sgent, sg_len, i) {
+ for (addr = sg_dma_address(sgent), len = sg_dma_len(sgent);
+ len > 0;
+ addr += cb->cb->length, len -= cb->cb->length, cb++) {
+ if (direction == DMA_DEV_TO_MEM)
+ cb->cb->dst = addr;
+ else
+ cb->cb->src = addr;
+ cb->cb->length = min(len, max_len);
+ }
+ }
+}
+
static int bcm2835_dma_abort(void __iomem *chan_base)
{
unsigned long cs;
@@ -218,6 +470,15 @@
struct bcm2835_desc *d;
unsigned long flags;
+ /* check the shared interrupt */
+ if (c->irq_flags & IRQF_SHARED) {
+ /* check if the interrupt is enabled */
+ flags = readl(c->chan_base + BCM2835_DMA_CS);
+ /* if not set then we are not the reason for the irq */
+ if (!(flags & BCM2835_DMA_INT))
+ return IRQ_NONE;
+ }
+
spin_lock_irqsave(&c->vc.lock, flags);
/* Acknowledge interrupt */
@@ -226,12 +487,18 @@
d = c->desc;
if (d) {
- /* TODO Only works for cyclic DMA */
- vchan_cyclic_callback(&d->vd);
- }
+ if (d->cyclic) {
+ /* call the cyclic callback */
+ vchan_cyclic_callback(&d->vd);
- /* Keep the DMA engine running */
- writel(BCM2835_DMA_ACTIVE, c->chan_base + BCM2835_DMA_CS);
+ /* Keep the DMA engine running */
+ writel(BCM2835_DMA_ACTIVE,
+ c->chan_base + BCM2835_DMA_CS);
+ } else {
+ vchan_cookie_complete(&c->desc->vd);
+ bcm2835_dma_start_desc(c);
+ }
+ }
spin_unlock_irqrestore(&c->vc.lock, flags);
@@ -252,8 +519,8 @@
return -ENOMEM;
}
- return request_irq(c->irq_number,
- bcm2835_dma_callback, 0, "DMA IRQ", c);
+ return request_irq(c->irq_number, bcm2835_dma_callback,
+ c->irq_flags, "DMA IRQ", c);
}
static void bcm2835_dma_free_chan_resources(struct dma_chan *chan)
@@ -339,8 +606,6 @@
struct bcm2835_chan *c = to_bcm2835_dma_chan(chan);
unsigned long flags;
- c->cyclic = true; /* Nothing else is implemented */
-
spin_lock_irqsave(&c->vc.lock, flags);
if (vchan_issue_pending(&c->vc) && !c->desc)
bcm2835_dma_start_desc(c);
@@ -348,18 +613,98 @@
spin_unlock_irqrestore(&c->vc.lock, flags);
}
+struct dma_async_tx_descriptor *bcm2835_dma_prep_dma_memcpy(
+ struct dma_chan *chan, dma_addr_t dst, dma_addr_t src,
+ size_t len, unsigned long flags)
+{
+ struct bcm2835_chan *c = to_bcm2835_dma_chan(chan);
+ struct bcm2835_desc *d;
+ u32 info = BCM2835_DMA_D_INC | BCM2835_DMA_S_INC;
+ u32 extra = BCM2835_DMA_INT_EN | BCM2835_DMA_WAIT_RESP;
+ size_t max_len = bcm2835_dma_max_frame_length(c);
+ size_t frames;
+
+ /* if src, dst or len is not given return with an error */
+ if (!src || !dst || !len)
+ return NULL;
+
+ /* calculate number of frames */
+ frames = bcm2835_dma_frames_for_length(len, max_len);
+
+ /* allocate the CB chain - this also fills in the pointers */
+ d = bcm2835_dma_create_cb_chain(chan, DMA_MEM_TO_MEM, false,
+ info, extra, frames,
+ src, dst, len, 0, GFP_KERNEL);
+ if (!d)
+ return NULL;
+
+ return vchan_tx_prep(&c->vc, &d->vd, flags);
+}
+
+static struct dma_async_tx_descriptor *bcm2835_dma_prep_slave_sg(
+ struct dma_chan *chan,
+ struct scatterlist *sgl, unsigned int sg_len,
+ enum dma_transfer_direction direction,
+ unsigned long flags, void *context)
+{
+ struct bcm2835_chan *c = to_bcm2835_dma_chan(chan);
+ struct bcm2835_desc *d;
+ dma_addr_t src = 0, dst = 0;
+ u32 info = BCM2835_DMA_WAIT_RESP;
+ u32 extra = BCM2835_DMA_INT_EN;
+ size_t frames;
+
+ if (!is_slave_direction(direction)) {
+ dev_err(chan->device->dev,
+ "%s: bad direction?\n", __func__);
+ return NULL;
+ }
+
+ if (c->dreq != 0)
+ info |= BCM2835_DMA_PER_MAP(c->dreq);
+
+ if (direction == DMA_DEV_TO_MEM) {
+ if (c->cfg.src_addr_width != DMA_SLAVE_BUSWIDTH_4_BYTES)
+ return NULL;
+ src = c->cfg.src_addr;
+ info |= BCM2835_DMA_S_DREQ | BCM2835_DMA_D_INC;
+ } else {
+ if (c->cfg.dst_addr_width != DMA_SLAVE_BUSWIDTH_4_BYTES)
+ return NULL;
+ dst = c->cfg.dst_addr;
+ info |= BCM2835_DMA_D_DREQ | BCM2835_DMA_S_INC;
+ }
+
+ /* count frames in sg list */
+ frames = bcm2835_dma_count_frames_for_sg(c, sgl, sg_len);
+
+ /* allocate the CB chain */
+ d = bcm2835_dma_create_cb_chain(chan, direction, false,
+ info, extra,
+ frames, src, dst, 0, 0,
+ GFP_KERNEL);
+ if (!d)
+ return NULL;
+
+ /* fill in frames with scatterlist pointers */
+ bcm2835_dma_fill_cb_chain_with_sg(chan, direction, d->cb_list,
+ sgl, sg_len);
+
+ return vchan_tx_prep(&c->vc, &d->vd, flags);
+}
+
static struct dma_async_tx_descriptor *bcm2835_dma_prep_dma_cyclic(
struct dma_chan *chan, dma_addr_t buf_addr, size_t buf_len,
size_t period_len, enum dma_transfer_direction direction,
unsigned long flags)
{
struct bcm2835_chan *c = to_bcm2835_dma_chan(chan);
- enum dma_slave_buswidth dev_width;
struct bcm2835_desc *d;
- dma_addr_t dev_addr;
- unsigned int es, sync_type;
- unsigned int frame;
- int i;
+ dma_addr_t src, dst;
+ u32 info = BCM2835_DMA_WAIT_RESP;
+ u32 extra = BCM2835_DMA_INT_EN;
+ size_t max_len = bcm2835_dma_max_frame_length(c);
+ size_t frames;
/* Grab configuration */
if (!is_slave_direction(direction)) {
@@ -367,103 +712,61 @@
return NULL;
}
- if (direction == DMA_DEV_TO_MEM) {
- dev_addr = c->cfg.src_addr;
- dev_width = c->cfg.src_addr_width;
- sync_type = BCM2835_DMA_S_DREQ;
- } else {
- dev_addr = c->cfg.dst_addr;
- dev_width = c->cfg.dst_addr_width;
- sync_type = BCM2835_DMA_D_DREQ;
- }
-
- /* Bus width translates to the element size (ES) */
- switch (dev_width) {
- case DMA_SLAVE_BUSWIDTH_4_BYTES:
- es = BCM2835_DMA_DATA_TYPE_S32;
- break;
- default:
+ if (!buf_len) {
+ dev_err(chan->device->dev,
+ "%s: bad buffer length (= 0)\n", __func__);
return NULL;
}
- /* Now allocate and setup the descriptor. */
- d = kzalloc(sizeof(*d), GFP_NOWAIT);
- if (!d)
- return NULL;
-
- d->c = c;
- d->dir = direction;
- d->frames = buf_len / period_len;
-
- d->cb_list = kcalloc(d->frames, sizeof(*d->cb_list), GFP_KERNEL);
- if (!d->cb_list) {
- kfree(d);
- return NULL;
- }
- /* Allocate memory for control blocks */
- for (i = 0; i < d->frames; i++) {
- struct bcm2835_cb_entry *cb_entry = &d->cb_list[i];
-
- cb_entry->cb = dma_pool_zalloc(c->cb_pool, GFP_ATOMIC,
- &cb_entry->paddr);
- if (!cb_entry->cb)
- goto error_cb;
- }
-
/*
- * Iterate over all frames, create a control block
- * for each frame and link them together.
+ * warn if buf_len is not a multiple of period_len - this may leed
+ * to unexpected latencies for interrupts and thus audiable clicks
*/
- for (frame = 0; frame < d->frames; frame++) {
- struct bcm2835_dma_cb *control_block = d->cb_list[frame].cb;
+ if (buf_len % period_len)
+ dev_warn_once(chan->device->dev,
+ "%s: buffer_length (%zd) is not a multiple of period_len (%zd)\n",
+ __func__, buf_len, period_len);
- /* Setup adresses */
- if (d->dir == DMA_DEV_TO_MEM) {
- control_block->info = BCM2835_DMA_D_INC;
- control_block->src = dev_addr;
- control_block->dst = buf_addr + frame * period_len;
- } else {
- control_block->info = BCM2835_DMA_S_INC;
- control_block->src = buf_addr + frame * period_len;
- control_block->dst = dev_addr;
- }
+ /* Setup DREQ channel */
+ if (c->dreq != 0)
+ info |= BCM2835_DMA_PER_MAP(c->dreq);
- /* Enable interrupt */
- control_block->info |= BCM2835_DMA_INT_EN;
-
- /* Setup synchronization */
- if (sync_type != 0)
- control_block->info |= sync_type;
-
- /* Setup DREQ channel */
- if (c->dreq != 0)
- control_block->info |=
- BCM2835_DMA_PER_MAP(c->dreq);
-
- /* Length of a frame */
- control_block->length = period_len;
- d->size += control_block->length;
-
- /*
- * Next block is the next frame.
- * This DMA engine driver currently only supports cyclic DMA.
- * Therefore, wrap around at number of frames.
- */
- control_block->next = d->cb_list[((frame + 1) % d->frames)].paddr;
+ if (direction == DMA_DEV_TO_MEM) {
+ if (c->cfg.src_addr_width != DMA_SLAVE_BUSWIDTH_4_BYTES)
+ return NULL;
+ src = c->cfg.src_addr;
+ dst = buf_addr;
+ info |= BCM2835_DMA_S_DREQ | BCM2835_DMA_D_INC;
+ } else {
+ if (c->cfg.dst_addr_width != DMA_SLAVE_BUSWIDTH_4_BYTES)
+ return NULL;
+ dst = c->cfg.dst_addr;
+ src = buf_addr;
+ info |= BCM2835_DMA_D_DREQ | BCM2835_DMA_S_INC;
}
+ /* calculate number of frames */
+ frames = /* number of periods */
+ DIV_ROUND_UP(buf_len, period_len) *
+ /* number of frames per period */
+ bcm2835_dma_frames_for_length(period_len, max_len);
+
+ /*
+ * allocate the CB chain
+ * note that we need to use GFP_NOWAIT, as the ALSA i2s dmaengine
+ * implementation calls prep_dma_cyclic with interrupts disabled.
+ */
+ d = bcm2835_dma_create_cb_chain(chan, direction, true,
+ info, extra,
+ frames, src, dst, buf_len,
+ period_len, GFP_NOWAIT);
+ if (!d)
+ return NULL;
+
+ /* wrap around into a loop */
+ d->cb_list[d->frames - 1].cb->next = d->cb_list[0].paddr;
+
return vchan_tx_prep(&c->vc, &d->vd, flags);
-error_cb:
- i--;
- for (; i >= 0; i--) {
- struct bcm2835_cb_entry *cb_entry = &d->cb_list[i];
-
- dma_pool_free(c->cb_pool, cb_entry->cb, cb_entry->paddr);
- }
-
- kfree(d->cb_list);
- kfree(d);
- return NULL;
}
static int bcm2835_dma_slave_config(struct dma_chan *chan,
@@ -529,7 +832,8 @@
return 0;
}
-static int bcm2835_dma_chan_init(struct bcm2835_dmadev *d, int chan_id, int irq)
+static int bcm2835_dma_chan_init(struct bcm2835_dmadev *d, int chan_id,
+ int irq, unsigned int irq_flags)
{
struct bcm2835_chan *c;
@@ -544,6 +848,12 @@
c->chan_base = BCM2835_DMA_CHANIO(d->base, chan_id);
c->ch = chan_id;
c->irq_number = irq;
+ c->irq_flags = irq_flags;
+
+ /* check in DEBUG register if this is a LITE channel */
+ if (readl(c->chan_base + BCM2835_DMA_DEBUG) &
+ BCM2835_DMA_DEBUG_LITE)
+ c->is_lite_channel = true;
return 0;
}
@@ -587,9 +897,11 @@
struct resource *res;
void __iomem *base;
int rc;
- int i;
- int irq;
+ int i, j;
+ int irq[BCM2835_DMA_MAX_DMA_CHAN_SUPPORTED + 1];
+ int irq_flags;
uint32_t chans_available;
+ char chan_name[BCM2835_DMA_CHAN_NAME_SIZE];
if (!pdev->dev.dma_mask)
pdev->dev.dma_mask = &pdev->dev.coherent_dma_mask;
@@ -615,16 +927,22 @@
dma_cap_set(DMA_SLAVE, od->ddev.cap_mask);
dma_cap_set(DMA_PRIVATE, od->ddev.cap_mask);
dma_cap_set(DMA_CYCLIC, od->ddev.cap_mask);
+ dma_cap_set(DMA_SLAVE, od->ddev.cap_mask);
+ dma_cap_set(DMA_MEMCPY, od->ddev.cap_mask);
od->ddev.device_alloc_chan_resources = bcm2835_dma_alloc_chan_resources;
od->ddev.device_free_chan_resources = bcm2835_dma_free_chan_resources;
od->ddev.device_tx_status = bcm2835_dma_tx_status;
od->ddev.device_issue_pending = bcm2835_dma_issue_pending;
od->ddev.device_prep_dma_cyclic = bcm2835_dma_prep_dma_cyclic;
+ od->ddev.device_prep_slave_sg = bcm2835_dma_prep_slave_sg;
+ od->ddev.device_prep_dma_memcpy = bcm2835_dma_prep_dma_memcpy;
od->ddev.device_config = bcm2835_dma_slave_config;
od->ddev.device_terminate_all = bcm2835_dma_terminate_all;
od->ddev.src_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_4_BYTES);
od->ddev.dst_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_4_BYTES);
- od->ddev.directions = BIT(DMA_DEV_TO_MEM) | BIT(DMA_MEM_TO_DEV);
+ od->ddev.directions = BIT(DMA_DEV_TO_MEM) | BIT(DMA_MEM_TO_DEV) |
+ BIT(DMA_MEM_TO_MEM);
+ od->ddev.residue_granularity = DMA_RESIDUE_GRANULARITY_BURST;
od->ddev.dev = &pdev->dev;
INIT_LIST_HEAD(&od->ddev.channels);
spin_lock_init(&od->lock);
@@ -640,22 +958,48 @@
goto err_no_dma;
}
- /*
- * Do not use the FIQ and BULK channels,
- * because they are used by the GPU.
- */
- chans_available &= ~(BCM2835_DMA_FIQ_MASK | BCM2835_DMA_BULK_MASK);
-
- for (i = 0; i < pdev->num_resources; i++) {
- irq = platform_get_irq(pdev, i);
- if (irq < 0)
- break;
-
- if (chans_available & (1 << i)) {
- rc = bcm2835_dma_chan_init(od, i, irq);
- if (rc)
- goto err_no_dma;
+ /* get irqs for each channel that we support */
+ for (i = 0; i <= BCM2835_DMA_MAX_DMA_CHAN_SUPPORTED; i++) {
+ /* skip masked out channels */
+ if (!(chans_available & (1 << i))) {
+ irq[i] = -1;
+ continue;
}
+
+ /* get the named irq */
+ snprintf(chan_name, sizeof(chan_name), "dma%i", i);
+ irq[i] = platform_get_irq_byname(pdev, chan_name);
+ if (irq[i] >= 0)
+ continue;
+
+ /* legacy device tree case handling */
+ dev_warn_once(&pdev->dev,
+ "missing interrupt-names property in device tree - legacy interpretation is used\n");
+ /*
+ * in case of channel >= 11
+ * use the 11th interrupt and that is shared
+ */
+ irq[i] = platform_get_irq(pdev, i < 11 ? i : 11);
+ }
+
+ /* get irqs for each channel */
+ for (i = 0; i <= BCM2835_DMA_MAX_DMA_CHAN_SUPPORTED; i++) {
+ /* skip channels without irq */
+ if (irq[i] < 0)
+ continue;
+
+ /* check if there are other channels that also use this irq */
+ irq_flags = 0;
+ for (j = 0; j <= BCM2835_DMA_MAX_DMA_CHAN_SUPPORTED; j++)
+ if ((i != j) && (irq[j] == irq[i])) {
+ irq_flags = IRQF_SHARED;
+ break;
+ }
+
+ /* initialize the channel */
+ rc = bcm2835_dma_chan_init(od, i, irq[i], irq_flags);
+ if (rc)
+ goto err_no_dma;
}
dev_dbg(&pdev->dev, "Initialized %i DMA channels\n", i);
diff --git a/drivers/dma/dmaengine.c b/drivers/dma/dmaengine.c
index 0cb259c5..8c9f45f 100644
--- a/drivers/dma/dmaengine.c
+++ b/drivers/dma/dmaengine.c
@@ -289,7 +289,7 @@
do {
status = dma_async_is_tx_complete(chan, cookie, NULL, NULL);
if (time_after_eq(jiffies, dma_sync_wait_timeout)) {
- pr_err("%s: timeout!\n", __func__);
+ dev_err(chan->device->dev, "%s: timeout!\n", __func__);
return DMA_ERROR;
}
if (status != DMA_IN_PROGRESS)
@@ -482,7 +482,8 @@
device = chan->device;
/* check if the channel supports slave transactions */
- if (!test_bit(DMA_SLAVE, device->cap_mask.bits))
+ if (!(test_bit(DMA_SLAVE, device->cap_mask.bits) ||
+ test_bit(DMA_CYCLIC, device->cap_mask.bits)))
return -ENXIO;
/*
@@ -518,7 +519,7 @@
struct dma_chan *chan;
if (mask && !__dma_device_satisfies_mask(dev, mask)) {
- pr_debug("%s: wrong capabilities\n", __func__);
+ dev_dbg(dev->dev, "%s: wrong capabilities\n", __func__);
return NULL;
}
/* devices with multiple channels need special handling as we need to
@@ -533,12 +534,12 @@
list_for_each_entry(chan, &dev->channels, device_node) {
if (chan->client_count) {
- pr_debug("%s: %s busy\n",
+ dev_dbg(dev->dev, "%s: %s busy\n",
__func__, dma_chan_name(chan));
continue;
}
if (fn && !fn(chan, fn_param)) {
- pr_debug("%s: %s filter said false\n",
+ dev_dbg(dev->dev, "%s: %s filter said false\n",
__func__, dma_chan_name(chan));
continue;
}
@@ -567,11 +568,12 @@
if (err) {
if (err == -ENODEV) {
- pr_debug("%s: %s module removed\n", __func__,
- dma_chan_name(chan));
+ dev_dbg(device->dev, "%s: %s module removed\n",
+ __func__, dma_chan_name(chan));
list_del_rcu(&device->global_node);
} else
- pr_debug("%s: failed to get %s: (%d)\n",
+ dev_dbg(device->dev,
+ "%s: failed to get %s: (%d)\n",
__func__, dma_chan_name(chan), err);
if (--device->privatecnt == 0)
@@ -602,7 +604,8 @@
device->privatecnt++;
err = dma_chan_get(chan);
if (err) {
- pr_debug("%s: failed to get %s: (%d)\n",
+ dev_dbg(chan->device->dev,
+ "%s: failed to get %s: (%d)\n",
__func__, dma_chan_name(chan), err);
chan = NULL;
if (--device->privatecnt == 0)
@@ -814,8 +817,9 @@
list_del_rcu(&device->global_node);
break;
} else if (err)
- pr_debug("%s: failed to get %s: (%d)\n",
- __func__, dma_chan_name(chan), err);
+ dev_dbg(chan->device->dev,
+ "%s: failed to get %s: (%d)\n",
+ __func__, dma_chan_name(chan), err);
}
}
@@ -862,12 +866,12 @@
return false;
#endif
- #if defined(CONFIG_ASYNC_MEMCPY) || defined(CONFIG_ASYNC_MEMCPY_MODULE)
+ #if IS_ENABLED(CONFIG_ASYNC_MEMCPY)
if (!dma_has_cap(DMA_MEMCPY, device->cap_mask))
return false;
#endif
- #if defined(CONFIG_ASYNC_XOR) || defined(CONFIG_ASYNC_XOR_MODULE)
+ #if IS_ENABLED(CONFIG_ASYNC_XOR)
if (!dma_has_cap(DMA_XOR, device->cap_mask))
return false;
@@ -877,7 +881,7 @@
#endif
#endif
- #if defined(CONFIG_ASYNC_PQ) || defined(CONFIG_ASYNC_PQ_MODULE)
+ #if IS_ENABLED(CONFIG_ASYNC_PQ)
if (!dma_has_cap(DMA_PQ, device->cap_mask))
return false;
@@ -1222,8 +1226,9 @@
while (tx->cookie == -EBUSY) {
if (time_after_eq(jiffies, dma_sync_wait_timeout)) {
- pr_err("%s timeout waiting for descriptor submission\n",
- __func__);
+ dev_err(tx->chan->device->dev,
+ "%s timeout waiting for descriptor submission\n",
+ __func__);
return DMA_ERROR;
}
cpu_relax();
diff --git a/drivers/dma/dw/core.c b/drivers/dma/dw/core.c
index 97199b3..edf053f 100644
--- a/drivers/dma/dw/core.c
+++ b/drivers/dma/dw/core.c
@@ -45,22 +45,19 @@
DW_DMA_MSIZE_16; \
u8 _dmsize = _is_slave ? _sconfig->dst_maxburst : \
DW_DMA_MSIZE_16; \
+ u8 _dms = (_dwc->direction == DMA_MEM_TO_DEV) ? \
+ _dwc->p_master : _dwc->m_master; \
+ u8 _sms = (_dwc->direction == DMA_DEV_TO_MEM) ? \
+ _dwc->p_master : _dwc->m_master; \
\
(DWC_CTLL_DST_MSIZE(_dmsize) \
| DWC_CTLL_SRC_MSIZE(_smsize) \
| DWC_CTLL_LLP_D_EN \
| DWC_CTLL_LLP_S_EN \
- | DWC_CTLL_DMS(_dwc->dst_master) \
- | DWC_CTLL_SMS(_dwc->src_master)); \
+ | DWC_CTLL_DMS(_dms) \
+ | DWC_CTLL_SMS(_sms)); \
})
-/*
- * Number of descriptors to allocate for each channel. This should be
- * made configurable somehow; preferably, the clients (at least the
- * ones using slave transfers) should be able to give us a hint.
- */
-#define NR_DESCS_PER_CHANNEL 64
-
/* The set of bus widths supported by the DMA controller */
#define DW_DMA_BUSWIDTHS \
BIT(DMA_SLAVE_BUSWIDTH_UNDEFINED) | \
@@ -80,51 +77,65 @@
return to_dw_desc(dwc->active_list.next);
}
-static struct dw_desc *dwc_desc_get(struct dw_dma_chan *dwc)
+static dma_cookie_t dwc_tx_submit(struct dma_async_tx_descriptor *tx)
{
- struct dw_desc *desc, *_desc;
- struct dw_desc *ret = NULL;
- unsigned int i = 0;
- unsigned long flags;
+ struct dw_desc *desc = txd_to_dw_desc(tx);
+ struct dw_dma_chan *dwc = to_dw_dma_chan(tx->chan);
+ dma_cookie_t cookie;
+ unsigned long flags;
spin_lock_irqsave(&dwc->lock, flags);
- list_for_each_entry_safe(desc, _desc, &dwc->free_list, desc_node) {
- i++;
- if (async_tx_test_ack(&desc->txd)) {
- list_del(&desc->desc_node);
- ret = desc;
- break;
- }
- dev_dbg(chan2dev(&dwc->chan), "desc %p not ACKed\n", desc);
- }
+ cookie = dma_cookie_assign(tx);
+
+ /*
+ * REVISIT: We should attempt to chain as many descriptors as
+ * possible, perhaps even appending to those already submitted
+ * for DMA. But this is hard to do in a race-free manner.
+ */
+
+ list_add_tail(&desc->desc_node, &dwc->queue);
spin_unlock_irqrestore(&dwc->lock, flags);
+ dev_vdbg(chan2dev(tx->chan), "%s: queued %u\n",
+ __func__, desc->txd.cookie);
- dev_vdbg(chan2dev(&dwc->chan), "scanned %u descriptors on freelist\n", i);
-
- return ret;
+ return cookie;
}
-/*
- * Move a descriptor, including any children, to the free list.
- * `desc' must not be on any lists.
- */
+static struct dw_desc *dwc_desc_get(struct dw_dma_chan *dwc)
+{
+ struct dw_dma *dw = to_dw_dma(dwc->chan.device);
+ struct dw_desc *desc;
+ dma_addr_t phys;
+
+ desc = dma_pool_zalloc(dw->desc_pool, GFP_ATOMIC, &phys);
+ if (!desc)
+ return NULL;
+
+ dwc->descs_allocated++;
+ INIT_LIST_HEAD(&desc->tx_list);
+ dma_async_tx_descriptor_init(&desc->txd, &dwc->chan);
+ desc->txd.tx_submit = dwc_tx_submit;
+ desc->txd.flags = DMA_CTRL_ACK;
+ desc->txd.phys = phys;
+ return desc;
+}
+
static void dwc_desc_put(struct dw_dma_chan *dwc, struct dw_desc *desc)
{
- unsigned long flags;
+ struct dw_dma *dw = to_dw_dma(dwc->chan.device);
+ struct dw_desc *child, *_next;
- if (desc) {
- struct dw_desc *child;
+ if (unlikely(!desc))
+ return;
- spin_lock_irqsave(&dwc->lock, flags);
- list_for_each_entry(child, &desc->tx_list, desc_node)
- dev_vdbg(chan2dev(&dwc->chan),
- "moving child desc %p to freelist\n",
- child);
- list_splice_init(&desc->tx_list, &dwc->free_list);
- dev_vdbg(chan2dev(&dwc->chan), "moving desc %p to freelist\n", desc);
- list_add(&desc->desc_node, &dwc->free_list);
- spin_unlock_irqrestore(&dwc->lock, flags);
+ list_for_each_entry_safe(child, _next, &desc->tx_list, desc_node) {
+ list_del(&child->desc_node);
+ dma_pool_free(dw->desc_pool, child, child->txd.phys);
+ dwc->descs_allocated--;
}
+
+ dma_pool_free(dw->desc_pool, desc, desc->txd.phys);
+ dwc->descs_allocated--;
}
static void dwc_initialize(struct dw_dma_chan *dwc)
@@ -133,7 +144,7 @@
u32 cfghi = DWC_CFGH_FIFO_MODE;
u32 cfglo = DWC_CFGL_CH_PRIOR(dwc->priority);
- if (dwc->initialized == true)
+ if (test_bit(DW_DMA_IS_INITIALIZED, &dwc->flags))
return;
cfghi |= DWC_CFGH_DST_PER(dwc->dst_id);
@@ -146,26 +157,11 @@
channel_set_bit(dw, MASK.XFER, dwc->mask);
channel_set_bit(dw, MASK.ERROR, dwc->mask);
- dwc->initialized = true;
+ set_bit(DW_DMA_IS_INITIALIZED, &dwc->flags);
}
/*----------------------------------------------------------------------*/
-static inline unsigned int dwc_fast_ffs(unsigned long long v)
-{
- /*
- * We can be a lot more clever here, but this should take care
- * of the most common optimization.
- */
- if (!(v & 7))
- return 3;
- else if (!(v & 3))
- return 2;
- else if (!(v & 1))
- return 1;
- return 0;
-}
-
static inline void dwc_dump_chan_regs(struct dw_dma_chan *dwc)
{
dev_err(chan2dev(&dwc->chan),
@@ -197,12 +193,12 @@
* Software emulation of LLP mode relies on interrupts to continue
* multi block transfer.
*/
- ctllo = desc->lli.ctllo | DWC_CTLL_INT_EN;
+ ctllo = lli_read(desc, ctllo) | DWC_CTLL_INT_EN;
- channel_writel(dwc, SAR, desc->lli.sar);
- channel_writel(dwc, DAR, desc->lli.dar);
+ channel_writel(dwc, SAR, lli_read(desc, sar));
+ channel_writel(dwc, DAR, lli_read(desc, dar));
channel_writel(dwc, CTL_LO, ctllo);
- channel_writel(dwc, CTL_HI, desc->lli.ctlhi);
+ channel_writel(dwc, CTL_HI, lli_read(desc, ctlhi));
channel_set_bit(dw, CH_EN, dwc->mask);
/* Move pointer to next descriptor */
@@ -213,6 +209,7 @@
static void dwc_dostart(struct dw_dma_chan *dwc, struct dw_desc *first)
{
struct dw_dma *dw = to_dw_dma(dwc->chan.device);
+ u8 lms = DWC_LLP_LMS(dwc->m_master);
unsigned long was_soft_llp;
/* ASSERT: channel is idle */
@@ -237,7 +234,7 @@
dwc_initialize(dwc);
- dwc->residue = first->total_len;
+ first->residue = first->total_len;
dwc->tx_node_active = &first->tx_list;
/* Submit first block */
@@ -248,9 +245,8 @@
dwc_initialize(dwc);
- channel_writel(dwc, LLP, first->txd.phys);
- channel_writel(dwc, CTL_LO,
- DWC_CTLL_LLP_D_EN | DWC_CTLL_LLP_S_EN);
+ channel_writel(dwc, LLP, first->txd.phys | lms);
+ channel_writel(dwc, CTL_LO, DWC_CTLL_LLP_D_EN | DWC_CTLL_LLP_S_EN);
channel_writel(dwc, CTL_HI, 0);
channel_set_bit(dw, CH_EN, dwc->mask);
}
@@ -293,11 +289,7 @@
list_for_each_entry(child, &desc->tx_list, desc_node)
async_tx_ack(&child->txd);
async_tx_ack(&desc->txd);
-
- list_splice_init(&desc->tx_list, &dwc->free_list);
- list_move(&desc->desc_node, &dwc->free_list);
-
- dma_descriptor_unmap(txd);
+ dwc_desc_put(dwc, desc);
spin_unlock_irqrestore(&dwc->lock, flags);
if (callback)
@@ -368,11 +360,11 @@
head = &desc->tx_list;
if (active != head) {
- /* Update desc to reflect last sent one */
- if (active != head->next)
- desc = to_dw_desc(active->prev);
-
- dwc->residue -= desc->len;
+ /* Update residue to reflect last sent descriptor */
+ if (active == head->next)
+ desc->residue -= desc->len;
+ else
+ desc->residue -= to_dw_desc(active->prev)->len;
child = to_dw_desc(active);
@@ -387,8 +379,6 @@
clear_bit(DW_DMA_IS_SOFT_LLP, &dwc->flags);
}
- dwc->residue = 0;
-
spin_unlock_irqrestore(&dwc->lock, flags);
dwc_complete_all(dw, dwc);
@@ -396,7 +386,6 @@
}
if (list_empty(&dwc->active_list)) {
- dwc->residue = 0;
spin_unlock_irqrestore(&dwc->lock, flags);
return;
}
@@ -411,31 +400,31 @@
list_for_each_entry_safe(desc, _desc, &dwc->active_list, desc_node) {
/* Initial residue value */
- dwc->residue = desc->total_len;
+ desc->residue = desc->total_len;
/* Check first descriptors addr */
- if (desc->txd.phys == llp) {
+ if (desc->txd.phys == DWC_LLP_LOC(llp)) {
spin_unlock_irqrestore(&dwc->lock, flags);
return;
}
/* Check first descriptors llp */
- if (desc->lli.llp == llp) {
+ if (lli_read(desc, llp) == llp) {
/* This one is currently in progress */
- dwc->residue -= dwc_get_sent(dwc);
+ desc->residue -= dwc_get_sent(dwc);
spin_unlock_irqrestore(&dwc->lock, flags);
return;
}
- dwc->residue -= desc->len;
+ desc->residue -= desc->len;
list_for_each_entry(child, &desc->tx_list, desc_node) {
- if (child->lli.llp == llp) {
+ if (lli_read(child, llp) == llp) {
/* Currently in progress */
- dwc->residue -= dwc_get_sent(dwc);
+ desc->residue -= dwc_get_sent(dwc);
spin_unlock_irqrestore(&dwc->lock, flags);
return;
}
- dwc->residue -= child->len;
+ desc->residue -= child->len;
}
/*
@@ -457,10 +446,14 @@
spin_unlock_irqrestore(&dwc->lock, flags);
}
-static inline void dwc_dump_lli(struct dw_dma_chan *dwc, struct dw_lli *lli)
+static inline void dwc_dump_lli(struct dw_dma_chan *dwc, struct dw_desc *desc)
{
dev_crit(chan2dev(&dwc->chan), " desc: s0x%x d0x%x l0x%x c0x%x:%x\n",
- lli->sar, lli->dar, lli->llp, lli->ctlhi, lli->ctllo);
+ lli_read(desc, sar),
+ lli_read(desc, dar),
+ lli_read(desc, llp),
+ lli_read(desc, ctlhi),
+ lli_read(desc, ctllo));
}
static void dwc_handle_error(struct dw_dma *dw, struct dw_dma_chan *dwc)
@@ -496,9 +489,9 @@
*/
dev_WARN(chan2dev(&dwc->chan), "Bad descriptor submitted for DMA!\n"
" cookie: %d\n", bad_desc->txd.cookie);
- dwc_dump_lli(dwc, &bad_desc->lli);
+ dwc_dump_lli(dwc, bad_desc);
list_for_each_entry(child, &bad_desc->tx_list, desc_node)
- dwc_dump_lli(dwc, &child->lli);
+ dwc_dump_lli(dwc, child);
spin_unlock_irqrestore(&dwc->lock, flags);
@@ -549,7 +542,7 @@
*/
if (unlikely(status_err & dwc->mask) ||
unlikely(status_xfer & dwc->mask)) {
- int i;
+ unsigned int i;
dev_err(chan2dev(&dwc->chan),
"cyclic DMA unexpected %s interrupt, stopping DMA transfer\n",
@@ -571,7 +564,7 @@
dma_writel(dw, CLEAR.XFER, dwc->mask);
for (i = 0; i < dwc->cdesc->periods; i++)
- dwc_dump_lli(dwc, &dwc->cdesc->desc[i]->lli);
+ dwc_dump_lli(dwc, dwc->cdesc->desc[i]);
spin_unlock_irqrestore(&dwc->lock, flags);
}
@@ -589,7 +582,7 @@
u32 status_block;
u32 status_xfer;
u32 status_err;
- int i;
+ unsigned int i;
status_block = dma_readl(dw, RAW.BLOCK);
status_xfer = dma_readl(dw, RAW.XFER);
@@ -658,30 +651,6 @@
/*----------------------------------------------------------------------*/
-static dma_cookie_t dwc_tx_submit(struct dma_async_tx_descriptor *tx)
-{
- struct dw_desc *desc = txd_to_dw_desc(tx);
- struct dw_dma_chan *dwc = to_dw_dma_chan(tx->chan);
- dma_cookie_t cookie;
- unsigned long flags;
-
- spin_lock_irqsave(&dwc->lock, flags);
- cookie = dma_cookie_assign(tx);
-
- /*
- * REVISIT: We should attempt to chain as many descriptors as
- * possible, perhaps even appending to those already submitted
- * for DMA. But this is hard to do in a race-free manner.
- */
-
- dev_vdbg(chan2dev(tx->chan), "%s: queued %u\n", __func__, desc->txd.cookie);
- list_add_tail(&desc->desc_node, &dwc->queue);
-
- spin_unlock_irqrestore(&dwc->lock, flags);
-
- return cookie;
-}
-
static struct dma_async_tx_descriptor *
dwc_prep_dma_memcpy(struct dma_chan *chan, dma_addr_t dest, dma_addr_t src,
size_t len, unsigned long flags)
@@ -693,10 +662,12 @@
struct dw_desc *prev;
size_t xfer_count;
size_t offset;
+ u8 m_master = dwc->m_master;
unsigned int src_width;
unsigned int dst_width;
- unsigned int data_width;
+ unsigned int data_width = dw->pdata->data_width[m_master];
u32 ctllo;
+ u8 lms = DWC_LLP_LMS(m_master);
dev_vdbg(chan2dev(chan),
"%s: d%pad s%pad l0x%zx f0x%lx\n", __func__,
@@ -709,11 +680,7 @@
dwc->direction = DMA_MEM_TO_MEM;
- data_width = min_t(unsigned int, dw->data_width[dwc->src_master],
- dw->data_width[dwc->dst_master]);
-
- src_width = dst_width = min_t(unsigned int, data_width,
- dwc_fast_ffs(src | dest | len));
+ src_width = dst_width = __ffs(data_width | src | dest | len);
ctllo = DWC_DEFAULT_CTLLO(chan)
| DWC_CTLL_DST_WIDTH(dst_width)
@@ -731,27 +698,27 @@
if (!desc)
goto err_desc_get;
- desc->lli.sar = src + offset;
- desc->lli.dar = dest + offset;
- desc->lli.ctllo = ctllo;
- desc->lli.ctlhi = xfer_count;
+ lli_write(desc, sar, src + offset);
+ lli_write(desc, dar, dest + offset);
+ lli_write(desc, ctllo, ctllo);
+ lli_write(desc, ctlhi, xfer_count);
desc->len = xfer_count << src_width;
if (!first) {
first = desc;
} else {
- prev->lli.llp = desc->txd.phys;
- list_add_tail(&desc->desc_node,
- &first->tx_list);
+ lli_write(prev, llp, desc->txd.phys | lms);
+ list_add_tail(&desc->desc_node, &first->tx_list);
}
prev = desc;
}
if (flags & DMA_PREP_INTERRUPT)
/* Trigger interrupt after last block */
- prev->lli.ctllo |= DWC_CTLL_INT_EN;
+ lli_set(prev, ctllo, DWC_CTLL_INT_EN);
prev->lli.llp = 0;
+ lli_clear(prev, ctllo, DWC_CTLL_LLP_D_EN | DWC_CTLL_LLP_S_EN);
first->txd.flags = flags;
first->total_len = len;
@@ -773,10 +740,12 @@
struct dw_desc *prev;
struct dw_desc *first;
u32 ctllo;
+ u8 m_master = dwc->m_master;
+ u8 lms = DWC_LLP_LMS(m_master);
dma_addr_t reg;
unsigned int reg_width;
unsigned int mem_width;
- unsigned int data_width;
+ unsigned int data_width = dw->pdata->data_width[m_master];
unsigned int i;
struct scatterlist *sg;
size_t total_len = 0;
@@ -802,8 +771,6 @@
ctllo |= sconfig->device_fc ? DWC_CTLL_FC(DW_DMA_FC_P_M2P) :
DWC_CTLL_FC(DW_DMA_FC_D_M2P);
- data_width = dw->data_width[dwc->src_master];
-
for_each_sg(sgl, sg, sg_len, i) {
struct dw_desc *desc;
u32 len, dlen, mem;
@@ -811,17 +778,16 @@
mem = sg_dma_address(sg);
len = sg_dma_len(sg);
- mem_width = min_t(unsigned int,
- data_width, dwc_fast_ffs(mem | len));
+ mem_width = __ffs(data_width | mem | len);
slave_sg_todev_fill_desc:
desc = dwc_desc_get(dwc);
if (!desc)
goto err_desc_get;
- desc->lli.sar = mem;
- desc->lli.dar = reg;
- desc->lli.ctllo = ctllo | DWC_CTLL_SRC_WIDTH(mem_width);
+ lli_write(desc, sar, mem);
+ lli_write(desc, dar, reg);
+ lli_write(desc, ctllo, ctllo | DWC_CTLL_SRC_WIDTH(mem_width));
if ((len >> mem_width) > dwc->block_size) {
dlen = dwc->block_size << mem_width;
mem += dlen;
@@ -831,15 +797,14 @@
len = 0;
}
- desc->lli.ctlhi = dlen >> mem_width;
+ lli_write(desc, ctlhi, dlen >> mem_width);
desc->len = dlen;
if (!first) {
first = desc;
} else {
- prev->lli.llp = desc->txd.phys;
- list_add_tail(&desc->desc_node,
- &first->tx_list);
+ lli_write(prev, llp, desc->txd.phys | lms);
+ list_add_tail(&desc->desc_node, &first->tx_list);
}
prev = desc;
total_len += dlen;
@@ -859,8 +824,6 @@
ctllo |= sconfig->device_fc ? DWC_CTLL_FC(DW_DMA_FC_P_P2M) :
DWC_CTLL_FC(DW_DMA_FC_D_P2M);
- data_width = dw->data_width[dwc->dst_master];
-
for_each_sg(sgl, sg, sg_len, i) {
struct dw_desc *desc;
u32 len, dlen, mem;
@@ -868,17 +831,16 @@
mem = sg_dma_address(sg);
len = sg_dma_len(sg);
- mem_width = min_t(unsigned int,
- data_width, dwc_fast_ffs(mem | len));
+ mem_width = __ffs(data_width | mem | len);
slave_sg_fromdev_fill_desc:
desc = dwc_desc_get(dwc);
if (!desc)
goto err_desc_get;
- desc->lli.sar = reg;
- desc->lli.dar = mem;
- desc->lli.ctllo = ctllo | DWC_CTLL_DST_WIDTH(mem_width);
+ lli_write(desc, sar, reg);
+ lli_write(desc, dar, mem);
+ lli_write(desc, ctllo, ctllo | DWC_CTLL_DST_WIDTH(mem_width));
if ((len >> reg_width) > dwc->block_size) {
dlen = dwc->block_size << reg_width;
mem += dlen;
@@ -887,15 +849,14 @@
dlen = len;
len = 0;
}
- desc->lli.ctlhi = dlen >> reg_width;
+ lli_write(desc, ctlhi, dlen >> reg_width);
desc->len = dlen;
if (!first) {
first = desc;
} else {
- prev->lli.llp = desc->txd.phys;
- list_add_tail(&desc->desc_node,
- &first->tx_list);
+ lli_write(prev, llp, desc->txd.phys | lms);
+ list_add_tail(&desc->desc_node, &first->tx_list);
}
prev = desc;
total_len += dlen;
@@ -910,9 +871,10 @@
if (flags & DMA_PREP_INTERRUPT)
/* Trigger interrupt after last block */
- prev->lli.ctllo |= DWC_CTLL_INT_EN;
+ lli_set(prev, ctllo, DWC_CTLL_INT_EN);
prev->lli.llp = 0;
+ lli_clear(prev, ctllo, DWC_CTLL_LLP_D_EN | DWC_CTLL_LLP_S_EN);
first->total_len = total_len;
return &first->txd;
@@ -937,8 +899,8 @@
dwc->src_id = dws->src_id;
dwc->dst_id = dws->dst_id;
- dwc->src_master = dws->src_master;
- dwc->dst_master = dws->dst_master;
+ dwc->m_master = dws->m_master;
+ dwc->p_master = dws->p_master;
return true;
}
@@ -991,7 +953,7 @@
while (!(channel_readl(dwc, CFG_LO) & DWC_CFGL_FIFO_EMPTY) && count--)
udelay(2);
- dwc->paused = true;
+ set_bit(DW_DMA_IS_PAUSED, &dwc->flags);
spin_unlock_irqrestore(&dwc->lock, flags);
@@ -1004,7 +966,7 @@
channel_writel(dwc, CFG_LO, cfglo & ~DWC_CFGL_CH_SUSP);
- dwc->paused = false;
+ clear_bit(DW_DMA_IS_PAUSED, &dwc->flags);
}
static int dwc_resume(struct dma_chan *chan)
@@ -1012,12 +974,10 @@
struct dw_dma_chan *dwc = to_dw_dma_chan(chan);
unsigned long flags;
- if (!dwc->paused)
- return 0;
-
spin_lock_irqsave(&dwc->lock, flags);
- dwc_chan_resume(dwc);
+ if (test_bit(DW_DMA_IS_PAUSED, &dwc->flags))
+ dwc_chan_resume(dwc);
spin_unlock_irqrestore(&dwc->lock, flags);
@@ -1053,16 +1013,37 @@
return 0;
}
-static inline u32 dwc_get_residue(struct dw_dma_chan *dwc)
+static struct dw_desc *dwc_find_desc(struct dw_dma_chan *dwc, dma_cookie_t c)
{
+ struct dw_desc *desc;
+
+ list_for_each_entry(desc, &dwc->active_list, desc_node)
+ if (desc->txd.cookie == c)
+ return desc;
+
+ return NULL;
+}
+
+static u32 dwc_get_residue(struct dw_dma_chan *dwc, dma_cookie_t cookie)
+{
+ struct dw_desc *desc;
unsigned long flags;
u32 residue;
spin_lock_irqsave(&dwc->lock, flags);
- residue = dwc->residue;
- if (test_bit(DW_DMA_IS_SOFT_LLP, &dwc->flags) && residue)
- residue -= dwc_get_sent(dwc);
+ desc = dwc_find_desc(dwc, cookie);
+ if (desc) {
+ if (desc == dwc_first_active(dwc)) {
+ residue = desc->residue;
+ if (test_bit(DW_DMA_IS_SOFT_LLP, &dwc->flags) && residue)
+ residue -= dwc_get_sent(dwc);
+ } else {
+ residue = desc->total_len;
+ }
+ } else {
+ residue = 0;
+ }
spin_unlock_irqrestore(&dwc->lock, flags);
return residue;
@@ -1083,10 +1064,12 @@
dwc_scan_descriptors(to_dw_dma(chan->device), dwc);
ret = dma_cookie_status(chan, cookie, txstate);
- if (ret != DMA_COMPLETE)
- dma_set_residue(txstate, dwc_get_residue(dwc));
+ if (ret == DMA_COMPLETE)
+ return ret;
- if (dwc->paused && ret == DMA_IN_PROGRESS)
+ dma_set_residue(txstate, dwc_get_residue(dwc, cookie));
+
+ if (test_bit(DW_DMA_IS_PAUSED, &dwc->flags) && ret == DMA_IN_PROGRESS)
return DMA_PAUSED;
return ret;
@@ -1107,7 +1090,7 @@
static void dw_dma_off(struct dw_dma *dw)
{
- int i;
+ unsigned int i;
dma_writel(dw, CFG, 0);
@@ -1121,7 +1104,7 @@
cpu_relax();
for (i = 0; i < dw->dma.chancnt; i++)
- dw->chan[i].initialized = false;
+ clear_bit(DW_DMA_IS_INITIALIZED, &dw->chan[i].flags);
}
static void dw_dma_on(struct dw_dma *dw)
@@ -1133,9 +1116,6 @@
{
struct dw_dma_chan *dwc = to_dw_dma_chan(chan);
struct dw_dma *dw = to_dw_dma(chan->device);
- struct dw_desc *desc;
- int i;
- unsigned long flags;
dev_vdbg(chan2dev(chan), "%s\n", __func__);
@@ -1166,48 +1146,13 @@
dw_dma_on(dw);
dw->in_use |= dwc->mask;
- spin_lock_irqsave(&dwc->lock, flags);
- i = dwc->descs_allocated;
- while (dwc->descs_allocated < NR_DESCS_PER_CHANNEL) {
- dma_addr_t phys;
-
- spin_unlock_irqrestore(&dwc->lock, flags);
-
- desc = dma_pool_alloc(dw->desc_pool, GFP_ATOMIC, &phys);
- if (!desc)
- goto err_desc_alloc;
-
- memset(desc, 0, sizeof(struct dw_desc));
-
- INIT_LIST_HEAD(&desc->tx_list);
- dma_async_tx_descriptor_init(&desc->txd, chan);
- desc->txd.tx_submit = dwc_tx_submit;
- desc->txd.flags = DMA_CTRL_ACK;
- desc->txd.phys = phys;
-
- dwc_desc_put(dwc, desc);
-
- spin_lock_irqsave(&dwc->lock, flags);
- i = ++dwc->descs_allocated;
- }
-
- spin_unlock_irqrestore(&dwc->lock, flags);
-
- dev_dbg(chan2dev(chan), "%s: allocated %d descriptors\n", __func__, i);
-
- return i;
-
-err_desc_alloc:
- dev_info(chan2dev(chan), "only allocated %d descriptors\n", i);
-
- return i;
+ return 0;
}
static void dwc_free_chan_resources(struct dma_chan *chan)
{
struct dw_dma_chan *dwc = to_dw_dma_chan(chan);
struct dw_dma *dw = to_dw_dma(chan->device);
- struct dw_desc *desc, *_desc;
unsigned long flags;
LIST_HEAD(list);
@@ -1220,17 +1165,15 @@
BUG_ON(dma_readl(to_dw_dma(chan->device), CH_EN) & dwc->mask);
spin_lock_irqsave(&dwc->lock, flags);
- list_splice_init(&dwc->free_list, &list);
- dwc->descs_allocated = 0;
/* Clear custom channel configuration */
dwc->src_id = 0;
dwc->dst_id = 0;
- dwc->src_master = 0;
- dwc->dst_master = 0;
+ dwc->m_master = 0;
+ dwc->p_master = 0;
- dwc->initialized = false;
+ clear_bit(DW_DMA_IS_INITIALIZED, &dwc->flags);
/* Disable interrupts */
channel_clear_bit(dw, MASK.XFER, dwc->mask);
@@ -1244,11 +1187,6 @@
if (!dw->in_use)
dw_dma_off(dw);
- list_for_each_entry_safe(desc, _desc, &list, desc_node) {
- dev_vdbg(chan2dev(chan), " freeing descriptor %p\n", desc);
- dma_pool_free(dw->desc_pool, desc, desc->txd.phys);
- }
-
dev_vdbg(chan2dev(chan), "%s: done\n", __func__);
}
@@ -1326,6 +1264,7 @@
struct dw_cyclic_desc *retval = NULL;
struct dw_desc *desc;
struct dw_desc *last = NULL;
+ u8 lms = DWC_LLP_LMS(dwc->m_master);
unsigned long was_cyclic;
unsigned int reg_width;
unsigned int periods;
@@ -1379,9 +1318,6 @@
retval = ERR_PTR(-ENOMEM);
- if (periods > NR_DESCS_PER_CHANNEL)
- goto out_err;
-
cdesc = kzalloc(sizeof(struct dw_cyclic_desc), GFP_KERNEL);
if (!cdesc)
goto out_err;
@@ -1397,50 +1333,50 @@
switch (direction) {
case DMA_MEM_TO_DEV:
- desc->lli.dar = sconfig->dst_addr;
- desc->lli.sar = buf_addr + (period_len * i);
- desc->lli.ctllo = (DWC_DEFAULT_CTLLO(chan)
- | DWC_CTLL_DST_WIDTH(reg_width)
- | DWC_CTLL_SRC_WIDTH(reg_width)
- | DWC_CTLL_DST_FIX
- | DWC_CTLL_SRC_INC
- | DWC_CTLL_INT_EN);
+ lli_write(desc, dar, sconfig->dst_addr);
+ lli_write(desc, sar, buf_addr + period_len * i);
+ lli_write(desc, ctllo, (DWC_DEFAULT_CTLLO(chan)
+ | DWC_CTLL_DST_WIDTH(reg_width)
+ | DWC_CTLL_SRC_WIDTH(reg_width)
+ | DWC_CTLL_DST_FIX
+ | DWC_CTLL_SRC_INC
+ | DWC_CTLL_INT_EN));
- desc->lli.ctllo |= sconfig->device_fc ?
- DWC_CTLL_FC(DW_DMA_FC_P_M2P) :
- DWC_CTLL_FC(DW_DMA_FC_D_M2P);
+ lli_set(desc, ctllo, sconfig->device_fc ?
+ DWC_CTLL_FC(DW_DMA_FC_P_M2P) :
+ DWC_CTLL_FC(DW_DMA_FC_D_M2P));
break;
case DMA_DEV_TO_MEM:
- desc->lli.dar = buf_addr + (period_len * i);
- desc->lli.sar = sconfig->src_addr;
- desc->lli.ctllo = (DWC_DEFAULT_CTLLO(chan)
- | DWC_CTLL_SRC_WIDTH(reg_width)
- | DWC_CTLL_DST_WIDTH(reg_width)
- | DWC_CTLL_DST_INC
- | DWC_CTLL_SRC_FIX
- | DWC_CTLL_INT_EN);
+ lli_write(desc, dar, buf_addr + period_len * i);
+ lli_write(desc, sar, sconfig->src_addr);
+ lli_write(desc, ctllo, (DWC_DEFAULT_CTLLO(chan)
+ | DWC_CTLL_SRC_WIDTH(reg_width)
+ | DWC_CTLL_DST_WIDTH(reg_width)
+ | DWC_CTLL_DST_INC
+ | DWC_CTLL_SRC_FIX
+ | DWC_CTLL_INT_EN));
- desc->lli.ctllo |= sconfig->device_fc ?
- DWC_CTLL_FC(DW_DMA_FC_P_P2M) :
- DWC_CTLL_FC(DW_DMA_FC_D_P2M);
+ lli_set(desc, ctllo, sconfig->device_fc ?
+ DWC_CTLL_FC(DW_DMA_FC_P_P2M) :
+ DWC_CTLL_FC(DW_DMA_FC_D_P2M));
break;
default:
break;
}
- desc->lli.ctlhi = (period_len >> reg_width);
+ lli_write(desc, ctlhi, period_len >> reg_width);
cdesc->desc[i] = desc;
if (last)
- last->lli.llp = desc->txd.phys;
+ lli_write(last, llp, desc->txd.phys | lms);
last = desc;
}
/* Let's make a cyclic list */
- last->lli.llp = cdesc->desc[0]->txd.phys;
+ lli_write(last, llp, cdesc->desc[0]->txd.phys | lms);
dev_dbg(chan2dev(&dwc->chan),
"cyclic prepared buf %pad len %zu period %zu periods %d\n",
@@ -1471,7 +1407,7 @@
struct dw_dma_chan *dwc = to_dw_dma_chan(chan);
struct dw_dma *dw = to_dw_dma(dwc->chan.device);
struct dw_cyclic_desc *cdesc = dwc->cdesc;
- int i;
+ unsigned int i;
unsigned long flags;
dev_dbg(chan2dev(&dwc->chan), "%s\n", __func__);
@@ -1495,32 +1431,38 @@
kfree(cdesc->desc);
kfree(cdesc);
+ dwc->cdesc = NULL;
+
clear_bit(DW_DMA_IS_CYCLIC, &dwc->flags);
}
EXPORT_SYMBOL(dw_dma_cyclic_free);
/*----------------------------------------------------------------------*/
-int dw_dma_probe(struct dw_dma_chip *chip, struct dw_dma_platform_data *pdata)
+int dw_dma_probe(struct dw_dma_chip *chip)
{
+ struct dw_dma_platform_data *pdata;
struct dw_dma *dw;
bool autocfg = false;
unsigned int dw_params;
- unsigned int max_blk_size = 0;
+ unsigned int i;
int err;
- int i;
dw = devm_kzalloc(chip->dev, sizeof(*dw), GFP_KERNEL);
if (!dw)
return -ENOMEM;
+ dw->pdata = devm_kzalloc(chip->dev, sizeof(*dw->pdata), GFP_KERNEL);
+ if (!dw->pdata)
+ return -ENOMEM;
+
dw->regs = chip->regs;
chip->dw = dw;
pm_runtime_get_sync(chip->dev);
- if (!pdata) {
- dw_params = dma_read_byaddr(chip->regs, DW_PARAMS);
+ if (!chip->pdata) {
+ dw_params = dma_readl(dw, DW_PARAMS);
dev_dbg(chip->dev, "DW_PARAMS: 0x%08x\n", dw_params);
autocfg = dw_params >> DW_PARAMS_EN & 1;
@@ -1529,29 +1471,31 @@
goto err_pdata;
}
- pdata = devm_kzalloc(chip->dev, sizeof(*pdata), GFP_KERNEL);
- if (!pdata) {
- err = -ENOMEM;
- goto err_pdata;
- }
+ /* Reassign the platform data pointer */
+ pdata = dw->pdata;
/* Get hardware configuration parameters */
pdata->nr_channels = (dw_params >> DW_PARAMS_NR_CHAN & 7) + 1;
pdata->nr_masters = (dw_params >> DW_PARAMS_NR_MASTER & 3) + 1;
for (i = 0; i < pdata->nr_masters; i++) {
pdata->data_width[i] =
- (dw_params >> DW_PARAMS_DATA_WIDTH(i) & 3) + 2;
+ 4 << (dw_params >> DW_PARAMS_DATA_WIDTH(i) & 3);
}
- max_blk_size = dma_readl(dw, MAX_BLK_SIZE);
+ pdata->block_size = dma_readl(dw, MAX_BLK_SIZE);
/* Fill platform data with the default values */
pdata->is_private = true;
pdata->is_memcpy = true;
pdata->chan_allocation_order = CHAN_ALLOCATION_ASCENDING;
pdata->chan_priority = CHAN_PRIORITY_ASCENDING;
- } else if (pdata->nr_channels > DW_DMA_MAX_NR_CHANNELS) {
+ } else if (chip->pdata->nr_channels > DW_DMA_MAX_NR_CHANNELS) {
err = -EINVAL;
goto err_pdata;
+ } else {
+ memcpy(dw->pdata, chip->pdata, sizeof(*dw->pdata));
+
+ /* Reassign the platform data pointer */
+ pdata = dw->pdata;
}
dw->chan = devm_kcalloc(chip->dev, pdata->nr_channels, sizeof(*dw->chan),
@@ -1561,11 +1505,6 @@
goto err_pdata;
}
- /* Get hardware configuration parameters */
- dw->nr_masters = pdata->nr_masters;
- for (i = 0; i < dw->nr_masters; i++)
- dw->data_width[i] = pdata->data_width[i];
-
/* Calculate all channel mask before DMA setup */
dw->all_chan_mask = (1 << pdata->nr_channels) - 1;
@@ -1612,7 +1551,6 @@
INIT_LIST_HEAD(&dwc->active_list);
INIT_LIST_HEAD(&dwc->queue);
- INIT_LIST_HEAD(&dwc->free_list);
channel_clear_bit(dw, CH_EN, dwc->mask);
@@ -1620,11 +1558,9 @@
/* Hardware configuration */
if (autocfg) {
- unsigned int dwc_params;
unsigned int r = DW_DMA_MAX_NR_CHANNELS - i - 1;
- void __iomem *addr = chip->regs + r * sizeof(u32);
-
- dwc_params = dma_read_byaddr(addr, DWC_PARAMS);
+ void __iomem *addr = &__dw_regs(dw)->DWC_PARAMS[r];
+ unsigned int dwc_params = dma_readl_native(addr);
dev_dbg(chip->dev, "DWC_PARAMS[%d]: 0x%08x\n", i,
dwc_params);
@@ -1635,16 +1571,15 @@
* up to 0x0a for 4095.
*/
dwc->block_size =
- (4 << ((max_blk_size >> 4 * i) & 0xf)) - 1;
+ (4 << ((pdata->block_size >> 4 * i) & 0xf)) - 1;
dwc->nollp =
(dwc_params >> DWC_PARAMS_MBLK_EN & 0x1) == 0;
} else {
dwc->block_size = pdata->block_size;
/* Check if channel supports multi block transfer */
- channel_writel(dwc, LLP, 0xfffffffc);
- dwc->nollp =
- (channel_readl(dwc, LLP) & 0xfffffffc) == 0;
+ channel_writel(dwc, LLP, DWC_LLP_LOC(0xffffffff));
+ dwc->nollp = DWC_LLP_LOC(channel_readl(dwc, LLP)) == 0;
channel_writel(dwc, LLP, 0);
}
}
diff --git a/drivers/dma/dw/pci.c b/drivers/dma/dw/pci.c
index 358f968..0ae6c3b 100644
--- a/drivers/dma/dw/pci.c
+++ b/drivers/dma/dw/pci.c
@@ -17,8 +17,8 @@
static int dw_pci_probe(struct pci_dev *pdev, const struct pci_device_id *pid)
{
+ const struct dw_dma_platform_data *pdata = (void *)pid->driver_data;
struct dw_dma_chip *chip;
- struct dw_dma_platform_data *pdata = (void *)pid->driver_data;
int ret;
ret = pcim_enable_device(pdev);
@@ -49,8 +49,9 @@
chip->dev = &pdev->dev;
chip->regs = pcim_iomap_table(pdev)[0];
chip->irq = pdev->irq;
+ chip->pdata = pdata;
- ret = dw_dma_probe(chip, pdata);
+ ret = dw_dma_probe(chip);
if (ret)
return ret;
diff --git a/drivers/dma/dw/platform.c b/drivers/dma/dw/platform.c
index 26edbe3..5bda0eb 100644
--- a/drivers/dma/dw/platform.c
+++ b/drivers/dma/dw/platform.c
@@ -42,13 +42,13 @@
slave.src_id = dma_spec->args[0];
slave.dst_id = dma_spec->args[0];
- slave.src_master = dma_spec->args[1];
- slave.dst_master = dma_spec->args[2];
+ slave.m_master = dma_spec->args[1];
+ slave.p_master = dma_spec->args[2];
if (WARN_ON(slave.src_id >= DW_DMA_MAX_NR_REQUESTS ||
slave.dst_id >= DW_DMA_MAX_NR_REQUESTS ||
- slave.src_master >= dw->nr_masters ||
- slave.dst_master >= dw->nr_masters))
+ slave.m_master >= dw->pdata->nr_masters ||
+ slave.p_master >= dw->pdata->nr_masters))
return NULL;
dma_cap_zero(cap);
@@ -66,8 +66,8 @@
.dma_dev = dma_spec->dev,
.src_id = dma_spec->slave_id,
.dst_id = dma_spec->slave_id,
- .src_master = 1,
- .dst_master = 0,
+ .m_master = 0,
+ .p_master = 1,
};
return dw_dma_filter(chan, &slave);
@@ -103,6 +103,7 @@
struct device_node *np = pdev->dev.of_node;
struct dw_dma_platform_data *pdata;
u32 tmp, arr[DW_DMA_MAX_NR_MASTERS];
+ u32 nr_masters;
u32 nr_channels;
if (!np) {
@@ -110,6 +111,11 @@
return NULL;
}
+ if (of_property_read_u32(np, "dma-masters", &nr_masters))
+ return NULL;
+ if (nr_masters < 1 || nr_masters > DW_DMA_MAX_NR_MASTERS)
+ return NULL;
+
if (of_property_read_u32(np, "dma-channels", &nr_channels))
return NULL;
@@ -117,6 +123,7 @@
if (!pdata)
return NULL;
+ pdata->nr_masters = nr_masters;
pdata->nr_channels = nr_channels;
if (of_property_read_bool(np, "is_private"))
@@ -131,17 +138,13 @@
if (!of_property_read_u32(np, "block_size", &tmp))
pdata->block_size = tmp;
- if (!of_property_read_u32(np, "dma-masters", &tmp)) {
- if (tmp > DW_DMA_MAX_NR_MASTERS)
- return NULL;
-
- pdata->nr_masters = tmp;
- }
-
- if (!of_property_read_u32_array(np, "data_width", arr,
- pdata->nr_masters))
- for (tmp = 0; tmp < pdata->nr_masters; tmp++)
+ if (!of_property_read_u32_array(np, "data-width", arr, nr_masters)) {
+ for (tmp = 0; tmp < nr_masters; tmp++)
pdata->data_width[tmp] = arr[tmp];
+ } else if (!of_property_read_u32_array(np, "data_width", arr, nr_masters)) {
+ for (tmp = 0; tmp < nr_masters; tmp++)
+ pdata->data_width[tmp] = BIT(arr[tmp] & 0x07);
+ }
return pdata;
}
@@ -158,7 +161,7 @@
struct dw_dma_chip *chip;
struct device *dev = &pdev->dev;
struct resource *mem;
- struct dw_dma_platform_data *pdata;
+ const struct dw_dma_platform_data *pdata;
int err;
chip = devm_kzalloc(dev, sizeof(*chip), GFP_KERNEL);
@@ -183,6 +186,7 @@
pdata = dw_dma_parse_dt(pdev);
chip->dev = dev;
+ chip->pdata = pdata;
chip->clk = devm_clk_get(chip->dev, "hclk");
if (IS_ERR(chip->clk))
@@ -193,7 +197,7 @@
pm_runtime_enable(&pdev->dev);
- err = dw_dma_probe(chip, pdata);
+ err = dw_dma_probe(chip);
if (err)
goto err_dw_dma_probe;
diff --git a/drivers/dma/dw/regs.h b/drivers/dma/dw/regs.h
index 0a50c18..4b7bd78 100644
--- a/drivers/dma/dw/regs.h
+++ b/drivers/dma/dw/regs.h
@@ -114,10 +114,6 @@
#define dma_writel_native writel
#endif
-/* To access the registers in early stage of probe */
-#define dma_read_byaddr(addr, name) \
- dma_readl_native((addr) + offsetof(struct dw_dma_regs, name))
-
/* Bitfields in DW_PARAMS */
#define DW_PARAMS_NR_CHAN 8 /* number of channels */
#define DW_PARAMS_NR_MASTER 11 /* number of AHB masters */
@@ -143,6 +139,10 @@
DW_DMA_MSIZE_256,
};
+/* Bitfields in LLP */
+#define DWC_LLP_LMS(x) ((x) & 3) /* list master select */
+#define DWC_LLP_LOC(x) ((x) & ~3) /* next lli */
+
/* Bitfields in CTL_LO */
#define DWC_CTLL_INT_EN (1 << 0) /* irqs enabled? */
#define DWC_CTLL_DST_WIDTH(n) ((n)<<1) /* bytes per element */
@@ -216,6 +216,8 @@
enum dw_dmac_flags {
DW_DMA_IS_CYCLIC = 0,
DW_DMA_IS_SOFT_LLP = 1,
+ DW_DMA_IS_PAUSED = 2,
+ DW_DMA_IS_INITIALIZED = 3,
};
struct dw_dma_chan {
@@ -224,8 +226,6 @@
u8 mask;
u8 priority;
enum dma_transfer_direction direction;
- bool paused;
- bool initialized;
/* software emulation of the LLP transfers */
struct list_head *tx_node_active;
@@ -236,8 +236,6 @@
unsigned long flags;
struct list_head active_list;
struct list_head queue;
- struct list_head free_list;
- u32 residue;
struct dw_cyclic_desc *cdesc;
unsigned int descs_allocated;
@@ -249,8 +247,8 @@
/* custom slave configuration */
u8 src_id;
u8 dst_id;
- u8 src_master;
- u8 dst_master;
+ u8 m_master;
+ u8 p_master;
/* configuration passed via .device_config */
struct dma_slave_config dma_sconfig;
@@ -283,9 +281,8 @@
u8 all_chan_mask;
u8 in_use;
- /* hardware configuration */
- unsigned char nr_masters;
- unsigned char data_width[DW_DMA_MAX_NR_MASTERS];
+ /* platform data */
+ struct dw_dma_platform_data *pdata;
};
static inline struct dw_dma_regs __iomem *__dw_regs(struct dw_dma *dw)
@@ -308,32 +305,51 @@
return container_of(ddev, struct dw_dma, dma);
}
+#ifdef CONFIG_DW_DMAC_BIG_ENDIAN_IO
+typedef __be32 __dw32;
+#else
+typedef __le32 __dw32;
+#endif
+
/* LLI == Linked List Item; a.k.a. DMA block descriptor */
struct dw_lli {
/* values that are not changed by hardware */
- u32 sar;
- u32 dar;
- u32 llp; /* chain to next lli */
- u32 ctllo;
+ __dw32 sar;
+ __dw32 dar;
+ __dw32 llp; /* chain to next lli */
+ __dw32 ctllo;
/* values that may get written back: */
- u32 ctlhi;
+ __dw32 ctlhi;
/* sstat and dstat can snapshot peripheral register state.
* silicon config may discard either or both...
*/
- u32 sstat;
- u32 dstat;
+ __dw32 sstat;
+ __dw32 dstat;
};
struct dw_desc {
/* FIRST values the hardware uses */
struct dw_lli lli;
+#ifdef CONFIG_DW_DMAC_BIG_ENDIAN_IO
+#define lli_set(d, reg, v) ((d)->lli.reg |= cpu_to_be32(v))
+#define lli_clear(d, reg, v) ((d)->lli.reg &= ~cpu_to_be32(v))
+#define lli_read(d, reg) be32_to_cpu((d)->lli.reg)
+#define lli_write(d, reg, v) ((d)->lli.reg = cpu_to_be32(v))
+#else
+#define lli_set(d, reg, v) ((d)->lli.reg |= cpu_to_le32(v))
+#define lli_clear(d, reg, v) ((d)->lli.reg &= ~cpu_to_le32(v))
+#define lli_read(d, reg) le32_to_cpu((d)->lli.reg)
+#define lli_write(d, reg, v) ((d)->lli.reg = cpu_to_le32(v))
+#endif
+
/* THEN values for driver housekeeping */
struct list_head desc_node;
struct list_head tx_list;
struct dma_async_tx_descriptor txd;
size_t len;
size_t total_len;
+ u32 residue;
};
#define to_dw_desc(h) list_entry(h, struct dw_desc, desc_node)
diff --git a/drivers/dma/edma.c b/drivers/dma/edma.c
index 04070ba..8181ed1 100644
--- a/drivers/dma/edma.c
+++ b/drivers/dma/edma.c
@@ -1537,8 +1537,17 @@
dev_vdbg(ecc->dev, "dma_ccerr_handler\n");
- if (!edma_error_pending(ecc))
+ if (!edma_error_pending(ecc)) {
+ /*
+ * The registers indicate no pending error event but the irq
+ * handler has been called.
+ * Ask eDMA to re-evaluate the error registers.
+ */
+ dev_err(ecc->dev, "%s: Error interrupt without error event!\n",
+ __func__);
+ edma_write(ecc, EDMA_EEVAL, 1);
return IRQ_NONE;
+ }
while (1) {
/* Event missed register(s) */
diff --git a/drivers/dma/fsldma.c b/drivers/dma/fsldma.c
index aac85c3..a8828ed 100644
--- a/drivers/dma/fsldma.c
+++ b/drivers/dma/fsldma.c
@@ -462,13 +462,12 @@
struct fsl_desc_sw *desc;
dma_addr_t pdesc;
- desc = dma_pool_alloc(chan->desc_pool, GFP_ATOMIC, &pdesc);
+ desc = dma_pool_zalloc(chan->desc_pool, GFP_ATOMIC, &pdesc);
if (!desc) {
chan_dbg(chan, "out of memory for link descriptor\n");
return NULL;
}
- memset(desc, 0, sizeof(*desc));
INIT_LIST_HEAD(&desc->tx_list);
dma_async_tx_descriptor_init(&desc->async_tx, &chan->common);
desc->async_tx.tx_submit = fsl_dma_tx_submit;
diff --git a/drivers/dma/hsu/hsu.c b/drivers/dma/hsu/hsu.c
index ee51051..f8c5cd5 100644
--- a/drivers/dma/hsu/hsu.c
+++ b/drivers/dma/hsu/hsu.c
@@ -77,8 +77,8 @@
hsu_chan_writel(hsuc, HSU_CH_MTSR, mtsr);
/* Set descriptors */
- count = (desc->nents - desc->active) % HSU_DMA_CHAN_NR_DESC;
- for (i = 0; i < count; i++) {
+ count = desc->nents - desc->active;
+ for (i = 0; i < count && i < HSU_DMA_CHAN_NR_DESC; i++) {
hsu_chan_writel(hsuc, HSU_CH_DxSAR(i), desc->sg[i].addr);
hsu_chan_writel(hsuc, HSU_CH_DxTSR(i), desc->sg[i].len);
@@ -160,7 +160,7 @@
return IRQ_NONE;
/* Timeout IRQ, need wait some time, see Errata 2 */
- if (hsuc->direction == DMA_DEV_TO_MEM && (sr & HSU_CH_SR_DESCTO_ANY))
+ if (sr & HSU_CH_SR_DESCTO_ANY)
udelay(2);
sr &= ~HSU_CH_SR_DESCTO_ANY;
@@ -420,6 +420,8 @@
hsu->dma.dev = chip->dev;
+ dma_set_max_seg_size(hsu->dma.dev, HSU_CH_DxTSR_MASK);
+
ret = dma_async_device_register(&hsu->dma);
if (ret)
return ret;
diff --git a/drivers/dma/hsu/hsu.h b/drivers/dma/hsu/hsu.h
index 6b070c2..486b023b 100644
--- a/drivers/dma/hsu/hsu.h
+++ b/drivers/dma/hsu/hsu.h
@@ -58,6 +58,10 @@
#define HSU_CH_DCR_CHEI BIT(23)
#define HSU_CH_DCR_CHTOI(x) BIT(24 + (x))
+/* Bits in HSU_CH_DxTSR */
+#define HSU_CH_DxTSR_MASK GENMASK(15, 0)
+#define HSU_CH_DxTSR_TSR(x) ((x) & HSU_CH_DxTSR_MASK)
+
struct hsu_dma_sg {
dma_addr_t addr;
unsigned int len;
diff --git a/drivers/dma/ioat/init.c b/drivers/dma/ioat/init.c
index efdee1a..d406056 100644
--- a/drivers/dma/ioat/init.c
+++ b/drivers/dma/ioat/init.c
@@ -690,12 +690,11 @@
/* allocate a completion writeback area */
/* doing 2 32bit writes to mmio since 1 64b write doesn't work */
ioat_chan->completion =
- dma_pool_alloc(ioat_chan->ioat_dma->completion_pool,
- GFP_KERNEL, &ioat_chan->completion_dma);
+ dma_pool_zalloc(ioat_chan->ioat_dma->completion_pool,
+ GFP_KERNEL, &ioat_chan->completion_dma);
if (!ioat_chan->completion)
return -ENOMEM;
- memset(ioat_chan->completion, 0, sizeof(*ioat_chan->completion));
writel(((u64)ioat_chan->completion_dma) & 0x00000000FFFFFFFF,
ioat_chan->reg_base + IOAT_CHANCMP_OFFSET_LOW);
writel(((u64)ioat_chan->completion_dma) >> 32,
@@ -1074,6 +1073,7 @@
struct ioatdma_chan *ioat_chan;
bool is_raid_device = false;
int err;
+ u16 val16;
dma = &ioat_dma->dma_dev;
dma->device_prep_dma_memcpy = ioat_dma_prep_memcpy_lock;
@@ -1173,6 +1173,17 @@
if (dca)
ioat_dma->dca = ioat_dca_init(pdev, ioat_dma->reg_base);
+ /* disable relaxed ordering */
+ err = pcie_capability_read_word(pdev, IOAT_DEVCTRL_OFFSET, &val16);
+ if (err)
+ return err;
+
+ /* clear relaxed ordering enable */
+ val16 &= ~IOAT_DEVCTRL_ROE;
+ err = pcie_capability_write_word(pdev, IOAT_DEVCTRL_OFFSET, val16);
+ if (err)
+ return err;
+
return 0;
}
diff --git a/drivers/dma/ioat/registers.h b/drivers/dma/ioat/registers.h
index 4994a36..7053498 100644
--- a/drivers/dma/ioat/registers.h
+++ b/drivers/dma/ioat/registers.h
@@ -26,6 +26,13 @@
#define IOAT_PCI_CHANERR_INT_OFFSET 0x180
#define IOAT_PCI_CHANERRMASK_INT_OFFSET 0x184
+/* PCIe config registers */
+
+/* EXPCAPID + N */
+#define IOAT_DEVCTRL_OFFSET 0x8
+/* relaxed ordering enable */
+#define IOAT_DEVCTRL_ROE 0x10
+
/* MMIO Device Registers */
#define IOAT_CHANCNT_OFFSET 0x00 /* 8-bit */
diff --git a/drivers/dma/mmp_pdma.c b/drivers/dma/mmp_pdma.c
index e39457f..56f1fd6 100644
--- a/drivers/dma/mmp_pdma.c
+++ b/drivers/dma/mmp_pdma.c
@@ -364,13 +364,12 @@
struct mmp_pdma_desc_sw *desc;
dma_addr_t pdesc;
- desc = dma_pool_alloc(chan->desc_pool, GFP_ATOMIC, &pdesc);
+ desc = dma_pool_zalloc(chan->desc_pool, GFP_ATOMIC, &pdesc);
if (!desc) {
dev_err(chan->dev, "out of memory for link descriptor\n");
return NULL;
}
- memset(desc, 0, sizeof(*desc));
INIT_LIST_HEAD(&desc->tx_list);
dma_async_tx_descriptor_init(&desc->async_tx, &chan->chan);
/* each desc has submit */
diff --git a/drivers/dma/mpc512x_dma.c b/drivers/dma/mpc512x_dma.c
index aae76fb..ccadafa 100644
--- a/drivers/dma/mpc512x_dma.c
+++ b/drivers/dma/mpc512x_dma.c
@@ -3,6 +3,7 @@
* Copyright (C) Semihalf 2009
* Copyright (C) Ilya Yanok, Emcraft Systems 2010
* Copyright (C) Alexander Popov, Promcontroller 2014
+ * Copyright (C) Mario Six, Guntermann & Drunck GmbH, 2016
*
* Written by Piotr Ziecik <kosmo@semihalf.com>. Hardware description
* (defines, structures and comments) was taken from MPC5121 DMA driver
@@ -26,18 +27,19 @@
*/
/*
- * MPC512x and MPC8308 DMA driver. It supports
- * memory to memory data transfers (tested using dmatest module) and
- * data transfers between memory and peripheral I/O memory
- * by means of slave scatter/gather with these limitations:
- * - chunked transfers (described by s/g lists with more than one item)
- * are refused as long as proper support for scatter/gather is missing;
- * - transfers on MPC8308 always start from software as this SoC appears
- * not to have external request lines for peripheral flow control;
- * - only peripheral devices with 4-byte FIFO access register are supported;
- * - minimal memory <-> I/O memory transfer chunk is 4 bytes and consequently
- * source and destination addresses must be 4-byte aligned
- * and transfer size must be aligned on (4 * maxburst) boundary;
+ * MPC512x and MPC8308 DMA driver. It supports memory to memory data transfers
+ * (tested using dmatest module) and data transfers between memory and
+ * peripheral I/O memory by means of slave scatter/gather with these
+ * limitations:
+ * - chunked transfers (described by s/g lists with more than one item) are
+ * refused as long as proper support for scatter/gather is missing
+ * - transfers on MPC8308 always start from software as this SoC does not have
+ * external request lines for peripheral flow control
+ * - memory <-> I/O memory transfer chunks of sizes of 1, 2, 4, 16 (for
+ * MPC512x), and 32 bytes are supported, and, consequently, source
+ * addresses and destination addresses must be aligned accordingly;
+ * furthermore, for MPC512x SoCs, the transfer size must be aligned on
+ * (chunk size * maxburst)
*/
#include <linux/module.h>
@@ -213,8 +215,10 @@
/* Settings for access to peripheral FIFO */
dma_addr_t src_per_paddr;
u32 src_tcd_nunits;
+ u8 swidth;
dma_addr_t dst_per_paddr;
u32 dst_tcd_nunits;
+ u8 dwidth;
/* Lock for this structure */
spinlock_t lock;
@@ -247,6 +251,7 @@
static inline struct mpc_dma *dma_chan_to_mpc_dma(struct dma_chan *c)
{
struct mpc_dma_chan *mchan = dma_chan_to_mpc_dma_chan(c);
+
return container_of(mchan, struct mpc_dma, channels[c->chan_id]);
}
@@ -254,9 +259,9 @@
* Execute all queued DMA descriptors.
*
* Following requirements must be met while calling mpc_dma_execute():
- * a) mchan->lock is acquired,
- * b) mchan->active list is empty,
- * c) mchan->queued list contains at least one entry.
+ * a) mchan->lock is acquired,
+ * b) mchan->active list is empty,
+ * c) mchan->queued list contains at least one entry.
*/
static void mpc_dma_execute(struct mpc_dma_chan *mchan)
{
@@ -446,20 +451,15 @@
if (es & MPC_DMA_DMAES_SAE)
dev_err(mdma->dma.dev, "- Source Address Error\n");
if (es & MPC_DMA_DMAES_SOE)
- dev_err(mdma->dma.dev, "- Source Offset"
- " Configuration Error\n");
+ dev_err(mdma->dma.dev, "- Source Offset Configuration Error\n");
if (es & MPC_DMA_DMAES_DAE)
- dev_err(mdma->dma.dev, "- Destination Address"
- " Error\n");
+ dev_err(mdma->dma.dev, "- Destination Address Error\n");
if (es & MPC_DMA_DMAES_DOE)
- dev_err(mdma->dma.dev, "- Destination Offset"
- " Configuration Error\n");
+ dev_err(mdma->dma.dev, "- Destination Offset Configuration Error\n");
if (es & MPC_DMA_DMAES_NCE)
- dev_err(mdma->dma.dev, "- NBytes/Citter"
- " Configuration Error\n");
+ dev_err(mdma->dma.dev, "- NBytes/Citter Configuration Error\n");
if (es & MPC_DMA_DMAES_SGE)
- dev_err(mdma->dma.dev, "- Scatter/Gather"
- " Configuration Error\n");
+ dev_err(mdma->dma.dev, "- Scatter/Gather Configuration Error\n");
if (es & MPC_DMA_DMAES_SBE)
dev_err(mdma->dma.dev, "- Source Bus Error\n");
if (es & MPC_DMA_DMAES_DBE)
@@ -518,8 +518,8 @@
for (i = 0; i < MPC_DMA_DESCRIPTORS; i++) {
mdesc = kzalloc(sizeof(struct mpc_dma_desc), GFP_KERNEL);
if (!mdesc) {
- dev_notice(mdma->dma.dev, "Memory allocation error. "
- "Allocated only %u descriptors\n", i);
+ dev_notice(mdma->dma.dev,
+ "Memory allocation error. Allocated only %u descriptors\n", i);
break;
}
@@ -684,6 +684,15 @@
return &mdesc->desc;
}
+inline u8 buswidth_to_dmatsize(u8 buswidth)
+{
+ u8 res;
+
+ for (res = 0; buswidth > 1; buswidth /= 2)
+ res++;
+ return res;
+}
+
static struct dma_async_tx_descriptor *
mpc_dma_prep_slave_sg(struct dma_chan *chan, struct scatterlist *sgl,
unsigned int sg_len, enum dma_transfer_direction direction,
@@ -742,39 +751,54 @@
memset(tcd, 0, sizeof(struct mpc_dma_tcd));
- if (!IS_ALIGNED(sg_dma_address(sg), 4))
- goto err_prep;
-
if (direction == DMA_DEV_TO_MEM) {
tcd->saddr = per_paddr;
tcd->daddr = sg_dma_address(sg);
+
+ if (!IS_ALIGNED(sg_dma_address(sg), mchan->dwidth))
+ goto err_prep;
+
tcd->soff = 0;
- tcd->doff = 4;
+ tcd->doff = mchan->dwidth;
} else {
tcd->saddr = sg_dma_address(sg);
tcd->daddr = per_paddr;
- tcd->soff = 4;
+
+ if (!IS_ALIGNED(sg_dma_address(sg), mchan->swidth))
+ goto err_prep;
+
+ tcd->soff = mchan->swidth;
tcd->doff = 0;
}
- tcd->ssize = MPC_DMA_TSIZE_4;
- tcd->dsize = MPC_DMA_TSIZE_4;
+ tcd->ssize = buswidth_to_dmatsize(mchan->swidth);
+ tcd->dsize = buswidth_to_dmatsize(mchan->dwidth);
- len = sg_dma_len(sg);
- tcd->nbytes = tcd_nunits * 4;
- if (!IS_ALIGNED(len, tcd->nbytes))
- goto err_prep;
+ if (mdma->is_mpc8308) {
+ tcd->nbytes = sg_dma_len(sg);
+ if (!IS_ALIGNED(tcd->nbytes, mchan->swidth))
+ goto err_prep;
- iter = len / tcd->nbytes;
- if (iter >= 1 << 15) {
- /* len is too big */
- goto err_prep;
+ /* No major loops for MPC8303 */
+ tcd->biter = 1;
+ tcd->citer = 1;
+ } else {
+ len = sg_dma_len(sg);
+ tcd->nbytes = tcd_nunits * tcd->ssize;
+ if (!IS_ALIGNED(len, tcd->nbytes))
+ goto err_prep;
+
+ iter = len / tcd->nbytes;
+ if (iter >= 1 << 15) {
+ /* len is too big */
+ goto err_prep;
+ }
+ /* citer_linkch contains the high bits of iter */
+ tcd->biter = iter & 0x1ff;
+ tcd->biter_linkch = iter >> 9;
+ tcd->citer = tcd->biter;
+ tcd->citer_linkch = tcd->biter_linkch;
}
- /* citer_linkch contains the high bits of iter */
- tcd->biter = iter & 0x1ff;
- tcd->biter_linkch = iter >> 9;
- tcd->citer = tcd->biter;
- tcd->citer_linkch = tcd->biter_linkch;
tcd->e_sg = 0;
tcd->d_req = 1;
@@ -796,40 +820,62 @@
return NULL;
}
+inline bool is_buswidth_valid(u8 buswidth, bool is_mpc8308)
+{
+ switch (buswidth) {
+ case 16:
+ if (is_mpc8308)
+ return false;
+ case 1:
+ case 2:
+ case 4:
+ case 32:
+ break;
+ default:
+ return false;
+ }
+
+ return true;
+}
+
static int mpc_dma_device_config(struct dma_chan *chan,
struct dma_slave_config *cfg)
{
struct mpc_dma_chan *mchan = dma_chan_to_mpc_dma_chan(chan);
+ struct mpc_dma *mdma = dma_chan_to_mpc_dma(&mchan->chan);
unsigned long flags;
/*
* Software constraints:
- * - only transfers between a peripheral device and
- * memory are supported;
- * - only peripheral devices with 4-byte FIFO access register
- * are supported;
- * - minimal transfer chunk is 4 bytes and consequently
- * source and destination addresses must be 4-byte aligned
- * and transfer size must be aligned on (4 * maxburst)
- * boundary;
- * - during the transfer RAM address is being incremented by
- * the size of minimal transfer chunk;
- * - peripheral port's address is constant during the transfer.
+ * - only transfers between a peripheral device and memory are
+ * supported
+ * - transfer chunk sizes of 1, 2, 4, 16 (for MPC512x), and 32 bytes
+ * are supported, and, consequently, source addresses and
+ * destination addresses; must be aligned accordingly; furthermore,
+ * for MPC512x SoCs, the transfer size must be aligned on (chunk
+ * size * maxburst)
+ * - during the transfer, the RAM address is incremented by the size
+ * of transfer chunk
+ * - the peripheral port's address is constant during the transfer.
*/
- if (cfg->src_addr_width != DMA_SLAVE_BUSWIDTH_4_BYTES ||
- cfg->dst_addr_width != DMA_SLAVE_BUSWIDTH_4_BYTES ||
- !IS_ALIGNED(cfg->src_addr, 4) ||
- !IS_ALIGNED(cfg->dst_addr, 4)) {
+ if (!IS_ALIGNED(cfg->src_addr, cfg->src_addr_width) ||
+ !IS_ALIGNED(cfg->dst_addr, cfg->dst_addr_width)) {
return -EINVAL;
}
+ if (!is_buswidth_valid(cfg->src_addr_width, mdma->is_mpc8308) ||
+ !is_buswidth_valid(cfg->dst_addr_width, mdma->is_mpc8308))
+ return -EINVAL;
+
spin_lock_irqsave(&mchan->lock, flags);
mchan->src_per_paddr = cfg->src_addr;
mchan->src_tcd_nunits = cfg->src_maxburst;
+ mchan->swidth = cfg->src_addr_width;
mchan->dst_per_paddr = cfg->dst_addr;
mchan->dst_tcd_nunits = cfg->dst_maxburst;
+ mchan->dwidth = cfg->dst_addr_width;
/* Apply defaults */
if (mchan->src_tcd_nunits == 0)
@@ -875,7 +921,6 @@
mdma = devm_kzalloc(dev, sizeof(struct mpc_dma), GFP_KERNEL);
if (!mdma) {
- dev_err(dev, "Memory exhausted!\n");
retval = -ENOMEM;
goto err;
}
@@ -999,7 +1044,8 @@
out_be32(&mdma->regs->dmaerrl, 0xFFFF);
} else {
out_be32(&mdma->regs->dmacr, MPC_DMA_DMACR_EDCG |
- MPC_DMA_DMACR_ERGA | MPC_DMA_DMACR_ERCA);
+ MPC_DMA_DMACR_ERGA |
+ MPC_DMA_DMACR_ERCA);
/* Disable hardware DMA requests */
out_be32(&mdma->regs->dmaerqh, 0);
diff --git a/drivers/dma/mv_xor.c b/drivers/dma/mv_xor.c
index 3922a5d..25d1dad 100644
--- a/drivers/dma/mv_xor.c
+++ b/drivers/dma/mv_xor.c
@@ -31,6 +31,12 @@
#include "dmaengine.h"
#include "mv_xor.h"
+enum mv_xor_type {
+ XOR_ORION,
+ XOR_ARMADA_38X,
+ XOR_ARMADA_37XX,
+};
+
enum mv_xor_mode {
XOR_MODE_IN_REG,
XOR_MODE_IN_DESC,
@@ -477,7 +483,7 @@
BUG_ON(len > MV_XOR_MAX_BYTE_COUNT);
dev_dbg(mv_chan_to_devp(mv_chan),
- "%s src_cnt: %d len: %u dest %pad flags: %ld\n",
+ "%s src_cnt: %d len: %zu dest %pad flags: %ld\n",
__func__, src_cnt, len, &dest, flags);
sw_desc = mv_chan_alloc_slot(mv_chan);
@@ -933,7 +939,7 @@
static struct mv_xor_chan *
mv_xor_channel_add(struct mv_xor_device *xordev,
struct platform_device *pdev,
- int idx, dma_cap_mask_t cap_mask, int irq, int op_in_desc)
+ int idx, dma_cap_mask_t cap_mask, int irq)
{
int ret = 0;
struct mv_xor_chan *mv_chan;
@@ -945,7 +951,10 @@
mv_chan->idx = idx;
mv_chan->irq = irq;
- mv_chan->op_in_desc = op_in_desc;
+ if (xordev->xor_type == XOR_ORION)
+ mv_chan->op_in_desc = XOR_MODE_IN_REG;
+ else
+ mv_chan->op_in_desc = XOR_MODE_IN_DESC;
dma_dev = &mv_chan->dmadev;
@@ -1085,6 +1094,33 @@
writel(0, base + WINDOW_OVERRIDE_CTRL(1));
}
+static void
+mv_xor_conf_mbus_windows_a3700(struct mv_xor_device *xordev)
+{
+ void __iomem *base = xordev->xor_high_base;
+ u32 win_enable = 0;
+ int i;
+
+ for (i = 0; i < 8; i++) {
+ writel(0, base + WINDOW_BASE(i));
+ writel(0, base + WINDOW_SIZE(i));
+ if (i < 4)
+ writel(0, base + WINDOW_REMAP_HIGH(i));
+ }
+ /*
+ * For Armada3700 open default 4GB Mbus window. The dram
+ * related configuration are done at AXIS level.
+ */
+ writel(0xffff0000, base + WINDOW_SIZE(0));
+ win_enable |= 1;
+ win_enable |= 3 << 16;
+
+ writel(win_enable, base + WINDOW_BAR_ENABLE(0));
+ writel(win_enable, base + WINDOW_BAR_ENABLE(1));
+ writel(0, base + WINDOW_OVERRIDE_CTRL(0));
+ writel(0, base + WINDOW_OVERRIDE_CTRL(1));
+}
+
/*
* Since this XOR driver is basically used only for RAID5, we don't
* need to care about synchronizing ->suspend with DMA activity,
@@ -1129,6 +1165,11 @@
XOR_INTR_MASK(mv_chan));
}
+ if (xordev->xor_type == XOR_ARMADA_37XX) {
+ mv_xor_conf_mbus_windows_a3700(xordev);
+ return 0;
+ }
+
dram = mv_mbus_dram_info();
if (dram)
mv_xor_conf_mbus_windows(xordev, dram);
@@ -1137,8 +1178,9 @@
}
static const struct of_device_id mv_xor_dt_ids[] = {
- { .compatible = "marvell,orion-xor", .data = (void *)XOR_MODE_IN_REG },
- { .compatible = "marvell,armada-380-xor", .data = (void *)XOR_MODE_IN_DESC },
+ { .compatible = "marvell,orion-xor", .data = (void *)XOR_ORION },
+ { .compatible = "marvell,armada-380-xor", .data = (void *)XOR_ARMADA_38X },
+ { .compatible = "marvell,armada-3700-xor", .data = (void *)XOR_ARMADA_37XX },
{},
};
@@ -1152,7 +1194,6 @@
struct resource *res;
unsigned int max_engines, max_channels;
int i, ret;
- int op_in_desc;
dev_notice(&pdev->dev, "Marvell shared XOR driver\n");
@@ -1180,12 +1221,30 @@
platform_set_drvdata(pdev, xordev);
+
+ /*
+ * We need to know which type of XOR device we use before
+ * setting up. In non-dt case it can only be the legacy one.
+ */
+ xordev->xor_type = XOR_ORION;
+ if (pdev->dev.of_node) {
+ const struct of_device_id *of_id =
+ of_match_device(mv_xor_dt_ids,
+ &pdev->dev);
+
+ xordev->xor_type = (uintptr_t)of_id->data;
+ }
+
/*
* (Re-)program MBUS remapping windows if we are asked to.
*/
- dram = mv_mbus_dram_info();
- if (dram)
- mv_xor_conf_mbus_windows(xordev, dram);
+ if (xordev->xor_type == XOR_ARMADA_37XX) {
+ mv_xor_conf_mbus_windows_a3700(xordev);
+ } else {
+ dram = mv_mbus_dram_info();
+ if (dram)
+ mv_xor_conf_mbus_windows(xordev, dram);
+ }
/* Not all platforms can gate the clock, so it is not
* an error if the clock does not exists.
@@ -1199,12 +1258,16 @@
* order for async_tx to perform well. So we limit the number
* of engines and channels so that we take into account this
* constraint. Note that we also want to use channels from
- * separate engines when possible.
+ * separate engines when possible. For dual-CPU Armada 3700
+ * SoC with single XOR engine allow using its both channels.
*/
max_engines = num_present_cpus();
- max_channels = min_t(unsigned int,
- MV_XOR_MAX_CHANNELS,
- DIV_ROUND_UP(num_present_cpus(), 2));
+ if (xordev->xor_type == XOR_ARMADA_37XX)
+ max_channels = num_present_cpus();
+ else
+ max_channels = min_t(unsigned int,
+ MV_XOR_MAX_CHANNELS,
+ DIV_ROUND_UP(num_present_cpus(), 2));
if (mv_xor_engine_count >= max_engines)
return 0;
@@ -1212,15 +1275,11 @@
if (pdev->dev.of_node) {
struct device_node *np;
int i = 0;
- const struct of_device_id *of_id =
- of_match_device(mv_xor_dt_ids,
- &pdev->dev);
for_each_child_of_node(pdev->dev.of_node, np) {
struct mv_xor_chan *chan;
dma_cap_mask_t cap_mask;
int irq;
- op_in_desc = (int)of_id->data;
if (i >= max_channels)
continue;
@@ -1237,7 +1296,7 @@
}
chan = mv_xor_channel_add(xordev, pdev, i,
- cap_mask, irq, op_in_desc);
+ cap_mask, irq);
if (IS_ERR(chan)) {
ret = PTR_ERR(chan);
irq_dispose_mapping(irq);
@@ -1266,8 +1325,7 @@
}
chan = mv_xor_channel_add(xordev, pdev, i,
- cd->cap_mask, irq,
- XOR_MODE_IN_REG);
+ cd->cap_mask, irq);
if (IS_ERR(chan)) {
ret = PTR_ERR(chan);
goto err_channel_add;
diff --git a/drivers/dma/mv_xor.h b/drivers/dma/mv_xor.h
index c19fe30..bf56e08 100644
--- a/drivers/dma/mv_xor.h
+++ b/drivers/dma/mv_xor.h
@@ -85,6 +85,7 @@
void __iomem *xor_high_base;
struct clk *clk;
struct mv_xor_chan *channels[MV_XOR_MAX_CHANNELS];
+ int xor_type;
};
/**
diff --git a/drivers/dma/of-dma.c b/drivers/dma/of-dma.c
index 1e1f298..faae0bf 100644
--- a/drivers/dma/of-dma.c
+++ b/drivers/dma/of-dma.c
@@ -240,8 +240,9 @@
struct of_phandle_args dma_spec;
struct of_dma *ofdma;
struct dma_chan *chan;
- int count, i;
+ int count, i, start;
int ret_no_channel = -ENODEV;
+ static atomic_t last_index;
if (!np || !name) {
pr_err("%s: not enough information provided\n", __func__);
@@ -259,8 +260,15 @@
return ERR_PTR(-ENODEV);
}
+ /*
+ * approximate an average distribution across multiple
+ * entries with the same name
+ */
+ start = atomic_inc_return(&last_index);
for (i = 0; i < count; i++) {
- if (of_dma_match_channel(np, name, i, &dma_spec))
+ if (of_dma_match_channel(np, name,
+ (i + start) % count,
+ &dma_spec))
continue;
mutex_lock(&of_dma_lock);
diff --git a/drivers/dma/pxa_dma.c b/drivers/dma/pxa_dma.c
index 77c1c44..e756a30c 100644
--- a/drivers/dma/pxa_dma.c
+++ b/drivers/dma/pxa_dma.c
@@ -117,6 +117,7 @@
/* protected by vc->lock */
struct pxad_phy *phy;
struct dma_pool *desc_pool; /* Descriptors pool */
+ dma_cookie_t bus_error;
};
struct pxad_device {
@@ -563,6 +564,7 @@
return;
}
}
+ chan->bus_error = 0;
/*
* Program the descriptor's address into the DMA controller,
@@ -666,6 +668,7 @@
struct virt_dma_desc *vd, *tmp;
unsigned int dcsr;
unsigned long flags;
+ dma_cookie_t last_started = 0;
BUG_ON(!chan);
@@ -678,6 +681,7 @@
dev_dbg(&chan->vc.chan.dev->device,
"%s(): checking txd %p[%x]: completed=%d\n",
__func__, vd, vd->tx.cookie, is_desc_completed(vd));
+ last_started = vd->tx.cookie;
if (to_pxad_sw_desc(vd)->cyclic) {
vchan_cyclic_callback(vd);
break;
@@ -690,7 +694,12 @@
}
}
- if (dcsr & PXA_DCSR_STOPSTATE) {
+ if (dcsr & PXA_DCSR_BUSERR) {
+ chan->bus_error = last_started;
+ phy_disable(phy);
+ }
+
+ if (!chan->bus_error && dcsr & PXA_DCSR_STOPSTATE) {
dev_dbg(&chan->vc.chan.dev->device,
"%s(): channel stopped, submitted_empty=%d issued_empty=%d",
__func__,
@@ -1249,6 +1258,9 @@
struct pxad_chan *chan = to_pxad_chan(dchan);
enum dma_status ret;
+ if (cookie == chan->bus_error)
+ return DMA_ERROR;
+
ret = dma_cookie_status(dchan, cookie, txstate);
if (likely(txstate && (ret != DMA_ERROR)))
dma_set_residue(txstate, pxad_residue(chan, cookie));
@@ -1321,7 +1333,7 @@
return 0;
}
-static const struct of_device_id const pxad_dt_ids[] = {
+static const struct of_device_id pxad_dt_ids[] = {
{ .compatible = "marvell,pdma-1.0", },
{}
};
diff --git a/drivers/dma/qcom/Makefile b/drivers/dma/qcom/Makefile
index bfea699..4bfc38b 100644
--- a/drivers/dma/qcom/Makefile
+++ b/drivers/dma/qcom/Makefile
@@ -1,3 +1,5 @@
obj-$(CONFIG_QCOM_BAM_DMA) += bam_dma.o
obj-$(CONFIG_QCOM_HIDMA_MGMT) += hdma_mgmt.o
hdma_mgmt-objs := hidma_mgmt.o hidma_mgmt_sys.o
+obj-$(CONFIG_QCOM_HIDMA) += hdma.o
+hdma-objs := hidma_ll.o hidma.o hidma_dbg.o
diff --git a/drivers/dma/qcom/bam_dma.c b/drivers/dma/qcom/bam_dma.c
index d5e0a9c..969b481 100644
--- a/drivers/dma/qcom/bam_dma.c
+++ b/drivers/dma/qcom/bam_dma.c
@@ -342,7 +342,7 @@
#define BAM_DESC_FIFO_SIZE SZ_32K
#define MAX_DESCRIPTORS (BAM_DESC_FIFO_SIZE / sizeof(struct bam_desc_hw) - 1)
-#define BAM_MAX_DATA_SIZE (SZ_32K - 8)
+#define BAM_FIFO_SIZE (SZ_32K - 8)
struct bam_chan {
struct virt_dma_chan vc;
@@ -387,6 +387,7 @@
/* execution environment ID, from DT */
u32 ee;
+ bool controlled_remotely;
const struct reg_offset_data *layout;
@@ -458,7 +459,7 @@
*/
writel_relaxed(ALIGN(bchan->fifo_phys, sizeof(struct bam_desc_hw)),
bam_addr(bdev, bchan->id, BAM_P_DESC_FIFO_ADDR));
- writel_relaxed(BAM_DESC_FIFO_SIZE,
+ writel_relaxed(BAM_FIFO_SIZE,
bam_addr(bdev, bchan->id, BAM_P_FIFO_SIZES));
/* enable the per pipe interrupts, enable EOT, ERR, and INT irqs */
@@ -604,7 +605,7 @@
/* calculate number of required entries */
for_each_sg(sgl, sg, sg_len, i)
- num_alloc += DIV_ROUND_UP(sg_dma_len(sg), BAM_MAX_DATA_SIZE);
+ num_alloc += DIV_ROUND_UP(sg_dma_len(sg), BAM_FIFO_SIZE);
/* allocate enough room to accomodate the number of entries */
async_desc = kzalloc(sizeof(*async_desc) +
@@ -635,10 +636,10 @@
desc->addr = cpu_to_le32(sg_dma_address(sg) +
curr_offset);
- if (remainder > BAM_MAX_DATA_SIZE) {
- desc->size = cpu_to_le16(BAM_MAX_DATA_SIZE);
- remainder -= BAM_MAX_DATA_SIZE;
- curr_offset += BAM_MAX_DATA_SIZE;
+ if (remainder > BAM_FIFO_SIZE) {
+ desc->size = cpu_to_le16(BAM_FIFO_SIZE);
+ remainder -= BAM_FIFO_SIZE;
+ curr_offset += BAM_FIFO_SIZE;
} else {
desc->size = cpu_to_le16(remainder);
remainder = 0;
@@ -801,13 +802,17 @@
if (srcs & P_IRQ)
tasklet_schedule(&bdev->task);
- if (srcs & BAM_IRQ)
+ if (srcs & BAM_IRQ) {
clr_mask = readl_relaxed(bam_addr(bdev, 0, BAM_IRQ_STTS));
- /* don't allow reorder of the various accesses to the BAM registers */
- mb();
+ /*
+ * don't allow reorder of the various accesses to the BAM
+ * registers
+ */
+ mb();
- writel_relaxed(clr_mask, bam_addr(bdev, 0, BAM_IRQ_CLR));
+ writel_relaxed(clr_mask, bam_addr(bdev, 0, BAM_IRQ_CLR));
+ }
return IRQ_HANDLED;
}
@@ -1038,6 +1043,9 @@
val = readl_relaxed(bam_addr(bdev, 0, BAM_NUM_PIPES));
bdev->num_channels = val & BAM_NUM_PIPES_MASK;
+ if (bdev->controlled_remotely)
+ return 0;
+
/* s/w reset bam */
/* after reset all pipes are disabled and idle */
val = readl_relaxed(bam_addr(bdev, 0, BAM_CTRL));
@@ -1125,6 +1133,9 @@
return ret;
}
+ bdev->controlled_remotely = of_property_read_bool(pdev->dev.of_node,
+ "qcom,controlled-remotely");
+
bdev->bamclk = devm_clk_get(bdev->dev, "bam_clk");
if (IS_ERR(bdev->bamclk))
return PTR_ERR(bdev->bamclk);
@@ -1163,7 +1174,7 @@
/* set max dma segment size */
bdev->common.dev = bdev->dev;
bdev->common.dev->dma_parms = &bdev->dma_parms;
- ret = dma_set_max_seg_size(bdev->common.dev, BAM_MAX_DATA_SIZE);
+ ret = dma_set_max_seg_size(bdev->common.dev, BAM_FIFO_SIZE);
if (ret) {
dev_err(bdev->dev, "cannot set maximum segment size\n");
goto err_bam_channel_exit;
@@ -1234,6 +1245,9 @@
bam_dma_terminate_all(&bdev->channels[i].vc.chan);
tasklet_kill(&bdev->channels[i].vc.task);
+ if (!bdev->channels[i].fifo_virt)
+ continue;
+
dma_free_wc(bdev->dev, BAM_DESC_FIFO_SIZE,
bdev->channels[i].fifo_virt,
bdev->channels[i].fifo_phys);
diff --git a/drivers/dma/qcom/hidma.c b/drivers/dma/qcom/hidma.c
index cccc78e..41b5c6d 100644
--- a/drivers/dma/qcom/hidma.c
+++ b/drivers/dma/qcom/hidma.c
@@ -1,7 +1,7 @@
/*
* Qualcomm Technologies HIDMA DMA engine interface
*
- * Copyright (c) 2015, The Linux Foundation. All rights reserved.
+ * Copyright (c) 2015-2016, The Linux Foundation. All rights reserved.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 and
@@ -404,7 +404,7 @@
spin_unlock_irqrestore(&mchan->lock, irqflags);
/* this suspends the existing transfer */
- rc = hidma_ll_pause(dmadev->lldev);
+ rc = hidma_ll_disable(dmadev->lldev);
if (rc) {
dev_err(dmadev->ddev.dev, "channel did not pause\n");
goto out;
@@ -427,7 +427,7 @@
list_move(&mdesc->node, &mchan->free);
}
- rc = hidma_ll_resume(dmadev->lldev);
+ rc = hidma_ll_enable(dmadev->lldev);
out:
pm_runtime_mark_last_busy(dmadev->ddev.dev);
pm_runtime_put_autosuspend(dmadev->ddev.dev);
@@ -488,7 +488,7 @@
dmadev = to_hidma_dev(mchan->chan.device);
if (!mchan->paused) {
pm_runtime_get_sync(dmadev->ddev.dev);
- if (hidma_ll_pause(dmadev->lldev))
+ if (hidma_ll_disable(dmadev->lldev))
dev_warn(dmadev->ddev.dev, "channel did not stop\n");
mchan->paused = true;
pm_runtime_mark_last_busy(dmadev->ddev.dev);
@@ -507,7 +507,7 @@
dmadev = to_hidma_dev(mchan->chan.device);
if (mchan->paused) {
pm_runtime_get_sync(dmadev->ddev.dev);
- rc = hidma_ll_resume(dmadev->lldev);
+ rc = hidma_ll_enable(dmadev->lldev);
if (!rc)
mchan->paused = false;
else
@@ -530,6 +530,43 @@
return hidma_ll_inthandler(chirq, lldev);
}
+static ssize_t hidma_show_values(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct hidma_dev *mdev = platform_get_drvdata(pdev);
+
+ buf[0] = 0;
+
+ if (strcmp(attr->attr.name, "chid") == 0)
+ sprintf(buf, "%d\n", mdev->chidx);
+
+ return strlen(buf);
+}
+
+static int hidma_create_sysfs_entry(struct hidma_dev *dev, char *name,
+ int mode)
+{
+ struct device_attribute *attrs;
+ char *name_copy;
+
+ attrs = devm_kmalloc(dev->ddev.dev, sizeof(struct device_attribute),
+ GFP_KERNEL);
+ if (!attrs)
+ return -ENOMEM;
+
+ name_copy = devm_kstrdup(dev->ddev.dev, name, GFP_KERNEL);
+ if (!name_copy)
+ return -ENOMEM;
+
+ attrs->attr.name = name_copy;
+ attrs->attr.mode = mode;
+ attrs->show = hidma_show_values;
+ sysfs_attr_init(&attrs->attr);
+
+ return device_create_file(dev->ddev.dev, attrs);
+}
+
static int hidma_probe(struct platform_device *pdev)
{
struct hidma_dev *dmadev;
@@ -644,6 +681,8 @@
dmadev->irq = chirq;
tasklet_init(&dmadev->task, hidma_issue_task, (unsigned long)dmadev);
+ hidma_debug_init(dmadev);
+ hidma_create_sysfs_entry(dmadev, "chid", S_IRUGO);
dev_info(&pdev->dev, "HI-DMA engine driver registration complete\n");
platform_set_drvdata(pdev, dmadev);
pm_runtime_mark_last_busy(dmadev->ddev.dev);
@@ -651,6 +690,7 @@
return 0;
uninit:
+ hidma_debug_uninit(dmadev);
hidma_ll_uninit(dmadev->lldev);
dmafree:
if (dmadev)
@@ -668,6 +708,7 @@
pm_runtime_get_sync(dmadev->ddev.dev);
dma_async_device_unregister(&dmadev->ddev);
devm_free_irq(dmadev->ddev.dev, dmadev->irq, dmadev->lldev);
+ hidma_debug_uninit(dmadev);
hidma_ll_uninit(dmadev->lldev);
hidma_free(dmadev);
@@ -689,7 +730,6 @@
{.compatible = "qcom,hidma-1.0",},
{},
};
-
MODULE_DEVICE_TABLE(of, hidma_match);
static struct platform_driver hidma_driver = {
diff --git a/drivers/dma/qcom/hidma.h b/drivers/dma/qcom/hidma.h
index 231e306..db413a5 100644
--- a/drivers/dma/qcom/hidma.h
+++ b/drivers/dma/qcom/hidma.h
@@ -1,7 +1,7 @@
/*
* Qualcomm Technologies HIDMA data structures
*
- * Copyright (c) 2014, The Linux Foundation. All rights reserved.
+ * Copyright (c) 2014-2016, The Linux Foundation. All rights reserved.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 and
@@ -20,32 +20,29 @@
#include <linux/interrupt.h>
#include <linux/dmaengine.h>
-#define TRE_SIZE 32 /* each TRE is 32 bytes */
-#define TRE_CFG_IDX 0
-#define TRE_LEN_IDX 1
-#define TRE_SRC_LOW_IDX 2
-#define TRE_SRC_HI_IDX 3
-#define TRE_DEST_LOW_IDX 4
-#define TRE_DEST_HI_IDX 5
-
-struct hidma_tx_status {
- u8 err_info; /* error record in this transfer */
- u8 err_code; /* completion code */
-};
+#define HIDMA_TRE_SIZE 32 /* each TRE is 32 bytes */
+#define HIDMA_TRE_CFG_IDX 0
+#define HIDMA_TRE_LEN_IDX 1
+#define HIDMA_TRE_SRC_LOW_IDX 2
+#define HIDMA_TRE_SRC_HI_IDX 3
+#define HIDMA_TRE_DEST_LOW_IDX 4
+#define HIDMA_TRE_DEST_HI_IDX 5
struct hidma_tre {
atomic_t allocated; /* if this channel is allocated */
bool queued; /* flag whether this is pending */
u16 status; /* status */
- u32 chidx; /* index of the tre */
+ u32 idx; /* index of the tre */
u32 dma_sig; /* signature of the tre */
const char *dev_name; /* name of the device */
void (*callback)(void *data); /* requester callback */
void *data; /* Data associated with this channel*/
struct hidma_lldev *lldev; /* lldma device pointer */
- u32 tre_local[TRE_SIZE / sizeof(u32) + 1]; /* TRE local copy */
+ u32 tre_local[HIDMA_TRE_SIZE / sizeof(u32) + 1]; /* TRE local copy */
u32 tre_index; /* the offset where this was written*/
u32 int_flags; /* interrupt flags */
+ u8 err_info; /* error record in this transfer */
+ u8 err_code; /* completion code */
};
struct hidma_lldev {
@@ -61,22 +58,21 @@
void __iomem *evca; /* Event Channel address */
struct hidma_tre
**pending_tre_list; /* Pointers to pending TREs */
- struct hidma_tx_status
- *tx_status_list; /* Pointers to pending TREs status*/
s32 pending_tre_count; /* Number of TREs pending */
void *tre_ring; /* TRE ring */
- dma_addr_t tre_ring_handle; /* TRE ring to be shared with HW */
+ dma_addr_t tre_dma; /* TRE ring to be shared with HW */
u32 tre_ring_size; /* Byte size of the ring */
u32 tre_processed_off; /* last processed TRE */
void *evre_ring; /* EVRE ring */
- dma_addr_t evre_ring_handle; /* EVRE ring to be shared with HW */
+ dma_addr_t evre_dma; /* EVRE ring to be shared with HW */
u32 evre_ring_size; /* Byte size of the ring */
u32 evre_processed_off; /* last processed EVRE */
u32 tre_write_offset; /* TRE write location */
struct tasklet_struct task; /* task delivering notifications */
+ struct tasklet_struct rst_task; /* task to reset HW */
DECLARE_KFIFO_PTR(handoff_fifo,
struct hidma_tre *); /* pending TREs FIFO */
};
@@ -145,8 +141,8 @@
bool hidma_ll_isenabled(struct hidma_lldev *llhndl);
void hidma_ll_queue_request(struct hidma_lldev *llhndl, u32 tre_ch);
void hidma_ll_start(struct hidma_lldev *llhndl);
-int hidma_ll_pause(struct hidma_lldev *llhndl);
-int hidma_ll_resume(struct hidma_lldev *llhndl);
+int hidma_ll_disable(struct hidma_lldev *lldev);
+int hidma_ll_enable(struct hidma_lldev *llhndl);
void hidma_ll_set_transfer_params(struct hidma_lldev *llhndl, u32 tre_ch,
dma_addr_t src, dma_addr_t dest, u32 len, u32 flags);
int hidma_ll_setup(struct hidma_lldev *lldev);
@@ -157,4 +153,6 @@
irqreturn_t hidma_ll_inthandler(int irq, void *arg);
void hidma_cleanup_pending_tre(struct hidma_lldev *llhndl, u8 err_info,
u8 err_code);
+int hidma_debug_init(struct hidma_dev *dmadev);
+void hidma_debug_uninit(struct hidma_dev *dmadev);
#endif
diff --git a/drivers/dma/qcom/hidma_dbg.c b/drivers/dma/qcom/hidma_dbg.c
new file mode 100644
index 0000000..fa827e5
--- /dev/null
+++ b/drivers/dma/qcom/hidma_dbg.c
@@ -0,0 +1,217 @@
+/*
+ * Qualcomm Technologies HIDMA debug file
+ *
+ * Copyright (c) 2015-2016, The Linux Foundation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/debugfs.h>
+#include <linux/device.h>
+#include <linux/list.h>
+#include <linux/pm_runtime.h>
+
+#include "hidma.h"
+
+static void hidma_ll_chstats(struct seq_file *s, void *llhndl, u32 tre_ch)
+{
+ struct hidma_lldev *lldev = llhndl;
+ struct hidma_tre *tre;
+ u32 length;
+ dma_addr_t src_start;
+ dma_addr_t dest_start;
+ u32 *tre_local;
+
+ if (tre_ch >= lldev->nr_tres) {
+ dev_err(lldev->dev, "invalid TRE number in chstats:%d", tre_ch);
+ return;
+ }
+ tre = &lldev->trepool[tre_ch];
+ seq_printf(s, "------Channel %d -----\n", tre_ch);
+ seq_printf(s, "allocated=%d\n", atomic_read(&tre->allocated));
+ seq_printf(s, "queued = 0x%x\n", tre->queued);
+ seq_printf(s, "err_info = 0x%x\n", tre->err_info);
+ seq_printf(s, "err_code = 0x%x\n", tre->err_code);
+ seq_printf(s, "status = 0x%x\n", tre->status);
+ seq_printf(s, "idx = 0x%x\n", tre->idx);
+ seq_printf(s, "dma_sig = 0x%x\n", tre->dma_sig);
+ seq_printf(s, "dev_name=%s\n", tre->dev_name);
+ seq_printf(s, "callback=%p\n", tre->callback);
+ seq_printf(s, "data=%p\n", tre->data);
+ seq_printf(s, "tre_index = 0x%x\n", tre->tre_index);
+
+ tre_local = &tre->tre_local[0];
+ src_start = tre_local[HIDMA_TRE_SRC_LOW_IDX];
+ src_start = ((u64) (tre_local[HIDMA_TRE_SRC_HI_IDX]) << 32) + src_start;
+ dest_start = tre_local[HIDMA_TRE_DEST_LOW_IDX];
+ dest_start += ((u64) (tre_local[HIDMA_TRE_DEST_HI_IDX]) << 32);
+ length = tre_local[HIDMA_TRE_LEN_IDX];
+
+ seq_printf(s, "src=%pap\n", &src_start);
+ seq_printf(s, "dest=%pap\n", &dest_start);
+ seq_printf(s, "length = 0x%x\n", length);
+}
+
+static void hidma_ll_devstats(struct seq_file *s, void *llhndl)
+{
+ struct hidma_lldev *lldev = llhndl;
+
+ seq_puts(s, "------Device -----\n");
+ seq_printf(s, "lldev init = 0x%x\n", lldev->initialized);
+ seq_printf(s, "trch_state = 0x%x\n", lldev->trch_state);
+ seq_printf(s, "evch_state = 0x%x\n", lldev->evch_state);
+ seq_printf(s, "chidx = 0x%x\n", lldev->chidx);
+ seq_printf(s, "nr_tres = 0x%x\n", lldev->nr_tres);
+ seq_printf(s, "trca=%p\n", lldev->trca);
+ seq_printf(s, "tre_ring=%p\n", lldev->tre_ring);
+ seq_printf(s, "tre_ring_handle=%pap\n", &lldev->tre_dma);
+ seq_printf(s, "tre_ring_size = 0x%x\n", lldev->tre_ring_size);
+ seq_printf(s, "tre_processed_off = 0x%x\n", lldev->tre_processed_off);
+ seq_printf(s, "pending_tre_count=%d\n", lldev->pending_tre_count);
+ seq_printf(s, "evca=%p\n", lldev->evca);
+ seq_printf(s, "evre_ring=%p\n", lldev->evre_ring);
+ seq_printf(s, "evre_ring_handle=%pap\n", &lldev->evre_dma);
+ seq_printf(s, "evre_ring_size = 0x%x\n", lldev->evre_ring_size);
+ seq_printf(s, "evre_processed_off = 0x%x\n", lldev->evre_processed_off);
+ seq_printf(s, "tre_write_offset = 0x%x\n", lldev->tre_write_offset);
+}
+
+/*
+ * hidma_chan_stats: display HIDMA channel statistics
+ *
+ * Display the statistics for the current HIDMA virtual channel device.
+ */
+static int hidma_chan_stats(struct seq_file *s, void *unused)
+{
+ struct hidma_chan *mchan = s->private;
+ struct hidma_desc *mdesc;
+ struct hidma_dev *dmadev = mchan->dmadev;
+
+ pm_runtime_get_sync(dmadev->ddev.dev);
+ seq_printf(s, "paused=%u\n", mchan->paused);
+ seq_printf(s, "dma_sig=%u\n", mchan->dma_sig);
+ seq_puts(s, "prepared\n");
+ list_for_each_entry(mdesc, &mchan->prepared, node)
+ hidma_ll_chstats(s, mchan->dmadev->lldev, mdesc->tre_ch);
+
+ seq_puts(s, "active\n");
+ list_for_each_entry(mdesc, &mchan->active, node)
+ hidma_ll_chstats(s, mchan->dmadev->lldev, mdesc->tre_ch);
+
+ seq_puts(s, "completed\n");
+ list_for_each_entry(mdesc, &mchan->completed, node)
+ hidma_ll_chstats(s, mchan->dmadev->lldev, mdesc->tre_ch);
+
+ hidma_ll_devstats(s, mchan->dmadev->lldev);
+ pm_runtime_mark_last_busy(dmadev->ddev.dev);
+ pm_runtime_put_autosuspend(dmadev->ddev.dev);
+ return 0;
+}
+
+/*
+ * hidma_dma_info: display HIDMA device info
+ *
+ * Display the info for the current HIDMA device.
+ */
+static int hidma_dma_info(struct seq_file *s, void *unused)
+{
+ struct hidma_dev *dmadev = s->private;
+ resource_size_t sz;
+
+ seq_printf(s, "nr_descriptors=%d\n", dmadev->nr_descriptors);
+ seq_printf(s, "dev_trca=%p\n", &dmadev->dev_trca);
+ seq_printf(s, "dev_trca_phys=%pa\n", &dmadev->trca_resource->start);
+ sz = resource_size(dmadev->trca_resource);
+ seq_printf(s, "dev_trca_size=%pa\n", &sz);
+ seq_printf(s, "dev_evca=%p\n", &dmadev->dev_evca);
+ seq_printf(s, "dev_evca_phys=%pa\n", &dmadev->evca_resource->start);
+ sz = resource_size(dmadev->evca_resource);
+ seq_printf(s, "dev_evca_size=%pa\n", &sz);
+ return 0;
+}
+
+static int hidma_chan_stats_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, hidma_chan_stats, inode->i_private);
+}
+
+static int hidma_dma_info_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, hidma_dma_info, inode->i_private);
+}
+
+static const struct file_operations hidma_chan_fops = {
+ .open = hidma_chan_stats_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
+static const struct file_operations hidma_dma_fops = {
+ .open = hidma_dma_info_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
+void hidma_debug_uninit(struct hidma_dev *dmadev)
+{
+ debugfs_remove_recursive(dmadev->debugfs);
+ debugfs_remove_recursive(dmadev->stats);
+}
+
+int hidma_debug_init(struct hidma_dev *dmadev)
+{
+ int rc = 0;
+ int chidx = 0;
+ struct list_head *position = NULL;
+
+ dmadev->debugfs = debugfs_create_dir(dev_name(dmadev->ddev.dev), NULL);
+ if (!dmadev->debugfs) {
+ rc = -ENODEV;
+ return rc;
+ }
+
+ /* walk through the virtual channel list */
+ list_for_each(position, &dmadev->ddev.channels) {
+ struct hidma_chan *chan;
+
+ chan = list_entry(position, struct hidma_chan,
+ chan.device_node);
+ sprintf(chan->dbg_name, "chan%d", chidx);
+ chan->debugfs = debugfs_create_dir(chan->dbg_name,
+ dmadev->debugfs);
+ if (!chan->debugfs) {
+ rc = -ENOMEM;
+ goto cleanup;
+ }
+ chan->stats = debugfs_create_file("stats", S_IRUGO,
+ chan->debugfs, chan,
+ &hidma_chan_fops);
+ if (!chan->stats) {
+ rc = -ENOMEM;
+ goto cleanup;
+ }
+ chidx++;
+ }
+
+ dmadev->stats = debugfs_create_file("stats", S_IRUGO,
+ dmadev->debugfs, dmadev,
+ &hidma_dma_fops);
+ if (!dmadev->stats) {
+ rc = -ENOMEM;
+ goto cleanup;
+ }
+
+ return 0;
+cleanup:
+ hidma_debug_uninit(dmadev);
+ return rc;
+}
diff --git a/drivers/dma/qcom/hidma_ll.c b/drivers/dma/qcom/hidma_ll.c
new file mode 100644
index 0000000..f392900
--- /dev/null
+++ b/drivers/dma/qcom/hidma_ll.c
@@ -0,0 +1,872 @@
+/*
+ * Qualcomm Technologies HIDMA DMA engine low level code
+ *
+ * Copyright (c) 2015-2016, The Linux Foundation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/dmaengine.h>
+#include <linux/slab.h>
+#include <linux/interrupt.h>
+#include <linux/mm.h>
+#include <linux/highmem.h>
+#include <linux/dma-mapping.h>
+#include <linux/delay.h>
+#include <linux/atomic.h>
+#include <linux/iopoll.h>
+#include <linux/kfifo.h>
+#include <linux/bitops.h>
+
+#include "hidma.h"
+
+#define HIDMA_EVRE_SIZE 16 /* each EVRE is 16 bytes */
+
+#define HIDMA_TRCA_CTRLSTS_REG 0x000
+#define HIDMA_TRCA_RING_LOW_REG 0x008
+#define HIDMA_TRCA_RING_HIGH_REG 0x00C
+#define HIDMA_TRCA_RING_LEN_REG 0x010
+#define HIDMA_TRCA_DOORBELL_REG 0x400
+
+#define HIDMA_EVCA_CTRLSTS_REG 0x000
+#define HIDMA_EVCA_INTCTRL_REG 0x004
+#define HIDMA_EVCA_RING_LOW_REG 0x008
+#define HIDMA_EVCA_RING_HIGH_REG 0x00C
+#define HIDMA_EVCA_RING_LEN_REG 0x010
+#define HIDMA_EVCA_WRITE_PTR_REG 0x020
+#define HIDMA_EVCA_DOORBELL_REG 0x400
+
+#define HIDMA_EVCA_IRQ_STAT_REG 0x100
+#define HIDMA_EVCA_IRQ_CLR_REG 0x108
+#define HIDMA_EVCA_IRQ_EN_REG 0x110
+
+#define HIDMA_EVRE_CFG_IDX 0
+
+#define HIDMA_EVRE_ERRINFO_BIT_POS 24
+#define HIDMA_EVRE_CODE_BIT_POS 28
+
+#define HIDMA_EVRE_ERRINFO_MASK GENMASK(3, 0)
+#define HIDMA_EVRE_CODE_MASK GENMASK(3, 0)
+
+#define HIDMA_CH_CONTROL_MASK GENMASK(7, 0)
+#define HIDMA_CH_STATE_MASK GENMASK(7, 0)
+#define HIDMA_CH_STATE_BIT_POS 0x8
+
+#define HIDMA_IRQ_EV_CH_EOB_IRQ_BIT_POS 0
+#define HIDMA_IRQ_EV_CH_WR_RESP_BIT_POS 1
+#define HIDMA_IRQ_TR_CH_TRE_RD_RSP_ER_BIT_POS 9
+#define HIDMA_IRQ_TR_CH_DATA_RD_ER_BIT_POS 10
+#define HIDMA_IRQ_TR_CH_DATA_WR_ER_BIT_POS 11
+#define HIDMA_IRQ_TR_CH_INVALID_TRE_BIT_POS 14
+
+#define ENABLE_IRQS (BIT(HIDMA_IRQ_EV_CH_EOB_IRQ_BIT_POS) | \
+ BIT(HIDMA_IRQ_EV_CH_WR_RESP_BIT_POS) | \
+ BIT(HIDMA_IRQ_TR_CH_TRE_RD_RSP_ER_BIT_POS) | \
+ BIT(HIDMA_IRQ_TR_CH_DATA_RD_ER_BIT_POS) | \
+ BIT(HIDMA_IRQ_TR_CH_DATA_WR_ER_BIT_POS) | \
+ BIT(HIDMA_IRQ_TR_CH_INVALID_TRE_BIT_POS))
+
+#define HIDMA_INCREMENT_ITERATOR(iter, size, ring_size) \
+do { \
+ iter += size; \
+ if (iter >= ring_size) \
+ iter -= ring_size; \
+} while (0)
+
+#define HIDMA_CH_STATE(val) \
+ ((val >> HIDMA_CH_STATE_BIT_POS) & HIDMA_CH_STATE_MASK)
+
+#define HIDMA_ERR_INT_MASK \
+ (BIT(HIDMA_IRQ_TR_CH_INVALID_TRE_BIT_POS) | \
+ BIT(HIDMA_IRQ_TR_CH_TRE_RD_RSP_ER_BIT_POS) | \
+ BIT(HIDMA_IRQ_EV_CH_WR_RESP_BIT_POS) | \
+ BIT(HIDMA_IRQ_TR_CH_DATA_RD_ER_BIT_POS) | \
+ BIT(HIDMA_IRQ_TR_CH_DATA_WR_ER_BIT_POS))
+
+enum ch_command {
+ HIDMA_CH_DISABLE = 0,
+ HIDMA_CH_ENABLE = 1,
+ HIDMA_CH_SUSPEND = 2,
+ HIDMA_CH_RESET = 9,
+};
+
+enum ch_state {
+ HIDMA_CH_DISABLED = 0,
+ HIDMA_CH_ENABLED = 1,
+ HIDMA_CH_RUNNING = 2,
+ HIDMA_CH_SUSPENDED = 3,
+ HIDMA_CH_STOPPED = 4,
+};
+
+enum tre_type {
+ HIDMA_TRE_MEMCPY = 3,
+};
+
+enum err_code {
+ HIDMA_EVRE_STATUS_COMPLETE = 1,
+ HIDMA_EVRE_STATUS_ERROR = 4,
+};
+
+static int hidma_is_chan_enabled(int state)
+{
+ switch (state) {
+ case HIDMA_CH_ENABLED:
+ case HIDMA_CH_RUNNING:
+ return true;
+ default:
+ return false;
+ }
+}
+
+void hidma_ll_free(struct hidma_lldev *lldev, u32 tre_ch)
+{
+ struct hidma_tre *tre;
+
+ if (tre_ch >= lldev->nr_tres) {
+ dev_err(lldev->dev, "invalid TRE number in free:%d", tre_ch);
+ return;
+ }
+
+ tre = &lldev->trepool[tre_ch];
+ if (atomic_read(&tre->allocated) != true) {
+ dev_err(lldev->dev, "trying to free an unused TRE:%d", tre_ch);
+ return;
+ }
+
+ atomic_set(&tre->allocated, 0);
+}
+
+int hidma_ll_request(struct hidma_lldev *lldev, u32 sig, const char *dev_name,
+ void (*callback)(void *data), void *data, u32 *tre_ch)
+{
+ unsigned int i;
+ struct hidma_tre *tre;
+ u32 *tre_local;
+
+ if (!tre_ch || !lldev)
+ return -EINVAL;
+
+ /* need to have at least one empty spot in the queue */
+ for (i = 0; i < lldev->nr_tres - 1; i++) {
+ if (atomic_add_unless(&lldev->trepool[i].allocated, 1, 1))
+ break;
+ }
+
+ if (i == (lldev->nr_tres - 1))
+ return -ENOMEM;
+
+ tre = &lldev->trepool[i];
+ tre->dma_sig = sig;
+ tre->dev_name = dev_name;
+ tre->callback = callback;
+ tre->data = data;
+ tre->idx = i;
+ tre->status = 0;
+ tre->queued = 0;
+ tre->err_code = 0;
+ tre->err_info = 0;
+ tre->lldev = lldev;
+ tre_local = &tre->tre_local[0];
+ tre_local[HIDMA_TRE_CFG_IDX] = HIDMA_TRE_MEMCPY;
+ tre_local[HIDMA_TRE_CFG_IDX] |= (lldev->chidx & 0xFF) << 8;
+ tre_local[HIDMA_TRE_CFG_IDX] |= BIT(16); /* set IEOB */
+ *tre_ch = i;
+ if (callback)
+ callback(data);
+ return 0;
+}
+
+/*
+ * Multiple TREs may be queued and waiting in the pending queue.
+ */
+static void hidma_ll_tre_complete(unsigned long arg)
+{
+ struct hidma_lldev *lldev = (struct hidma_lldev *)arg;
+ struct hidma_tre *tre;
+
+ while (kfifo_out(&lldev->handoff_fifo, &tre, 1)) {
+ /* call the user if it has been read by the hardware */
+ if (tre->callback)
+ tre->callback(tre->data);
+ }
+}
+
+static int hidma_post_completed(struct hidma_lldev *lldev, int tre_iterator,
+ u8 err_info, u8 err_code)
+{
+ struct hidma_tre *tre;
+ unsigned long flags;
+
+ spin_lock_irqsave(&lldev->lock, flags);
+ tre = lldev->pending_tre_list[tre_iterator / HIDMA_TRE_SIZE];
+ if (!tre) {
+ spin_unlock_irqrestore(&lldev->lock, flags);
+ dev_warn(lldev->dev, "tre_index [%d] and tre out of sync\n",
+ tre_iterator / HIDMA_TRE_SIZE);
+ return -EINVAL;
+ }
+ lldev->pending_tre_list[tre->tre_index] = NULL;
+
+ /*
+ * Keep track of pending TREs that SW is expecting to receive
+ * from HW. We got one now. Decrement our counter.
+ */
+ lldev->pending_tre_count--;
+ if (lldev->pending_tre_count < 0) {
+ dev_warn(lldev->dev, "tre count mismatch on completion");
+ lldev->pending_tre_count = 0;
+ }
+
+ spin_unlock_irqrestore(&lldev->lock, flags);
+
+ tre->err_info = err_info;
+ tre->err_code = err_code;
+ tre->queued = 0;
+
+ kfifo_put(&lldev->handoff_fifo, tre);
+ tasklet_schedule(&lldev->task);
+
+ return 0;
+}
+
+/*
+ * Called to handle the interrupt for the channel.
+ * Return a positive number if TRE or EVRE were consumed on this run.
+ * Return a positive number if there are pending TREs or EVREs.
+ * Return 0 if there is nothing to consume or no pending TREs/EVREs found.
+ */
+static int hidma_handle_tre_completion(struct hidma_lldev *lldev)
+{
+ u32 evre_ring_size = lldev->evre_ring_size;
+ u32 tre_ring_size = lldev->tre_ring_size;
+ u32 err_info, err_code, evre_write_off;
+ u32 tre_iterator, evre_iterator;
+ u32 num_completed = 0;
+
+ evre_write_off = readl_relaxed(lldev->evca + HIDMA_EVCA_WRITE_PTR_REG);
+ tre_iterator = lldev->tre_processed_off;
+ evre_iterator = lldev->evre_processed_off;
+
+ if ((evre_write_off > evre_ring_size) ||
+ (evre_write_off % HIDMA_EVRE_SIZE)) {
+ dev_err(lldev->dev, "HW reports invalid EVRE write offset\n");
+ return 0;
+ }
+
+ /*
+ * By the time control reaches here the number of EVREs and TREs
+ * may not match. Only consume the ones that hardware told us.
+ */
+ while ((evre_iterator != evre_write_off)) {
+ u32 *current_evre = lldev->evre_ring + evre_iterator;
+ u32 cfg;
+
+ cfg = current_evre[HIDMA_EVRE_CFG_IDX];
+ err_info = cfg >> HIDMA_EVRE_ERRINFO_BIT_POS;
+ err_info &= HIDMA_EVRE_ERRINFO_MASK;
+ err_code =
+ (cfg >> HIDMA_EVRE_CODE_BIT_POS) & HIDMA_EVRE_CODE_MASK;
+
+ if (hidma_post_completed(lldev, tre_iterator, err_info,
+ err_code))
+ break;
+
+ HIDMA_INCREMENT_ITERATOR(tre_iterator, HIDMA_TRE_SIZE,
+ tre_ring_size);
+ HIDMA_INCREMENT_ITERATOR(evre_iterator, HIDMA_EVRE_SIZE,
+ evre_ring_size);
+
+ /*
+ * Read the new event descriptor written by the HW.
+ * As we are processing the delivered events, other events
+ * get queued to the SW for processing.
+ */
+ evre_write_off =
+ readl_relaxed(lldev->evca + HIDMA_EVCA_WRITE_PTR_REG);
+ num_completed++;
+ }
+
+ if (num_completed) {
+ u32 evre_read_off = (lldev->evre_processed_off +
+ HIDMA_EVRE_SIZE * num_completed);
+ u32 tre_read_off = (lldev->tre_processed_off +
+ HIDMA_TRE_SIZE * num_completed);
+
+ evre_read_off = evre_read_off % evre_ring_size;
+ tre_read_off = tre_read_off % tre_ring_size;
+
+ writel(evre_read_off, lldev->evca + HIDMA_EVCA_DOORBELL_REG);
+
+ /* record the last processed tre offset */
+ lldev->tre_processed_off = tre_read_off;
+ lldev->evre_processed_off = evre_read_off;
+ }
+
+ return num_completed;
+}
+
+void hidma_cleanup_pending_tre(struct hidma_lldev *lldev, u8 err_info,
+ u8 err_code)
+{
+ u32 tre_iterator;
+ u32 tre_ring_size = lldev->tre_ring_size;
+ int num_completed = 0;
+ u32 tre_read_off;
+
+ tre_iterator = lldev->tre_processed_off;
+ while (lldev->pending_tre_count) {
+ if (hidma_post_completed(lldev, tre_iterator, err_info,
+ err_code))
+ break;
+ HIDMA_INCREMENT_ITERATOR(tre_iterator, HIDMA_TRE_SIZE,
+ tre_ring_size);
+ num_completed++;
+ }
+ tre_read_off = (lldev->tre_processed_off +
+ HIDMA_TRE_SIZE * num_completed);
+
+ tre_read_off = tre_read_off % tre_ring_size;
+
+ /* record the last processed tre offset */
+ lldev->tre_processed_off = tre_read_off;
+}
+
+static int hidma_ll_reset(struct hidma_lldev *lldev)
+{
+ u32 val;
+ int ret;
+
+ val = readl(lldev->trca + HIDMA_TRCA_CTRLSTS_REG);
+ val &= ~(HIDMA_CH_CONTROL_MASK << 16);
+ val |= HIDMA_CH_RESET << 16;
+ writel(val, lldev->trca + HIDMA_TRCA_CTRLSTS_REG);
+
+ /*
+ * Delay 10ms after reset to allow DMA logic to quiesce.
+ * Do a polled read up to 1ms and 10ms maximum.
+ */
+ ret = readl_poll_timeout(lldev->trca + HIDMA_TRCA_CTRLSTS_REG, val,
+ HIDMA_CH_STATE(val) == HIDMA_CH_DISABLED,
+ 1000, 10000);
+ if (ret) {
+ dev_err(lldev->dev, "transfer channel did not reset\n");
+ return ret;
+ }
+
+ val = readl(lldev->evca + HIDMA_EVCA_CTRLSTS_REG);
+ val &= ~(HIDMA_CH_CONTROL_MASK << 16);
+ val |= HIDMA_CH_RESET << 16;
+ writel(val, lldev->evca + HIDMA_EVCA_CTRLSTS_REG);
+
+ /*
+ * Delay 10ms after reset to allow DMA logic to quiesce.
+ * Do a polled read up to 1ms and 10ms maximum.
+ */
+ ret = readl_poll_timeout(lldev->evca + HIDMA_EVCA_CTRLSTS_REG, val,
+ HIDMA_CH_STATE(val) == HIDMA_CH_DISABLED,
+ 1000, 10000);
+ if (ret)
+ return ret;
+
+ lldev->trch_state = HIDMA_CH_DISABLED;
+ lldev->evch_state = HIDMA_CH_DISABLED;
+ return 0;
+}
+
+/*
+ * Abort all transactions and perform a reset.
+ */
+static void hidma_ll_abort(unsigned long arg)
+{
+ struct hidma_lldev *lldev = (struct hidma_lldev *)arg;
+ u8 err_code = HIDMA_EVRE_STATUS_ERROR;
+ u8 err_info = 0xFF;
+ int rc;
+
+ hidma_cleanup_pending_tre(lldev, err_info, err_code);
+
+ /* reset the channel for recovery */
+ rc = hidma_ll_setup(lldev);
+ if (rc) {
+ dev_err(lldev->dev, "channel reinitialize failed after error\n");
+ return;
+ }
+ writel(ENABLE_IRQS, lldev->evca + HIDMA_EVCA_IRQ_EN_REG);
+}
+
+/*
+ * The interrupt handler for HIDMA will try to consume as many pending
+ * EVRE from the event queue as possible. Each EVRE has an associated
+ * TRE that holds the user interface parameters. EVRE reports the
+ * result of the transaction. Hardware guarantees ordering between EVREs
+ * and TREs. We use last processed offset to figure out which TRE is
+ * associated with which EVRE. If two TREs are consumed by HW, the EVREs
+ * are in order in the event ring.
+ *
+ * This handler will do a one pass for consuming EVREs. Other EVREs may
+ * be delivered while we are working. It will try to consume incoming
+ * EVREs one more time and return.
+ *
+ * For unprocessed EVREs, hardware will trigger another interrupt until
+ * all the interrupt bits are cleared.
+ *
+ * Hardware guarantees that by the time interrupt is observed, all data
+ * transactions in flight are delivered to their respective places and
+ * are visible to the CPU.
+ *
+ * On demand paging for IOMMU is only supported for PCIe via PRI
+ * (Page Request Interface) not for HIDMA. All other hardware instances
+ * including HIDMA work on pinned DMA addresses.
+ *
+ * HIDMA is not aware of IOMMU presence since it follows the DMA API. All
+ * IOMMU latency will be built into the data movement time. By the time
+ * interrupt happens, IOMMU lookups + data movement has already taken place.
+ *
+ * While the first read in a typical PCI endpoint ISR flushes all outstanding
+ * requests traditionally to the destination, this concept does not apply
+ * here for this HW.
+ */
+irqreturn_t hidma_ll_inthandler(int chirq, void *arg)
+{
+ struct hidma_lldev *lldev = arg;
+ u32 status;
+ u32 enable;
+ u32 cause;
+
+ /*
+ * Fine tuned for this HW...
+ *
+ * This ISR has been designed for this particular hardware. Relaxed
+ * read and write accessors are used for performance reasons due to
+ * interrupt delivery guarantees. Do not copy this code blindly and
+ * expect that to work.
+ */
+ status = readl_relaxed(lldev->evca + HIDMA_EVCA_IRQ_STAT_REG);
+ enable = readl_relaxed(lldev->evca + HIDMA_EVCA_IRQ_EN_REG);
+ cause = status & enable;
+
+ while (cause) {
+ if (cause & HIDMA_ERR_INT_MASK) {
+ dev_err(lldev->dev, "error 0x%x, resetting...\n",
+ cause);
+
+ /* Clear out pending interrupts */
+ writel(cause, lldev->evca + HIDMA_EVCA_IRQ_CLR_REG);
+
+ tasklet_schedule(&lldev->rst_task);
+ goto out;
+ }
+
+ /*
+ * Try to consume as many EVREs as possible.
+ */
+ hidma_handle_tre_completion(lldev);
+
+ /* We consumed TREs or there are pending TREs or EVREs. */
+ writel_relaxed(cause, lldev->evca + HIDMA_EVCA_IRQ_CLR_REG);
+
+ /*
+ * Another interrupt might have arrived while we are
+ * processing this one. Read the new cause.
+ */
+ status = readl_relaxed(lldev->evca + HIDMA_EVCA_IRQ_STAT_REG);
+ enable = readl_relaxed(lldev->evca + HIDMA_EVCA_IRQ_EN_REG);
+ cause = status & enable;
+ }
+
+out:
+ return IRQ_HANDLED;
+}
+
+int hidma_ll_enable(struct hidma_lldev *lldev)
+{
+ u32 val;
+ int ret;
+
+ val = readl(lldev->evca + HIDMA_EVCA_CTRLSTS_REG);
+ val &= ~(HIDMA_CH_CONTROL_MASK << 16);
+ val |= HIDMA_CH_ENABLE << 16;
+ writel(val, lldev->evca + HIDMA_EVCA_CTRLSTS_REG);
+
+ ret = readl_poll_timeout(lldev->evca + HIDMA_EVCA_CTRLSTS_REG, val,
+ hidma_is_chan_enabled(HIDMA_CH_STATE(val)),
+ 1000, 10000);
+ if (ret) {
+ dev_err(lldev->dev, "event channel did not get enabled\n");
+ return ret;
+ }
+
+ val = readl(lldev->trca + HIDMA_TRCA_CTRLSTS_REG);
+ val &= ~(HIDMA_CH_CONTROL_MASK << 16);
+ val |= HIDMA_CH_ENABLE << 16;
+ writel(val, lldev->trca + HIDMA_TRCA_CTRLSTS_REG);
+
+ ret = readl_poll_timeout(lldev->trca + HIDMA_TRCA_CTRLSTS_REG, val,
+ hidma_is_chan_enabled(HIDMA_CH_STATE(val)),
+ 1000, 10000);
+ if (ret) {
+ dev_err(lldev->dev, "transfer channel did not get enabled\n");
+ return ret;
+ }
+
+ lldev->trch_state = HIDMA_CH_ENABLED;
+ lldev->evch_state = HIDMA_CH_ENABLED;
+
+ return 0;
+}
+
+void hidma_ll_start(struct hidma_lldev *lldev)
+{
+ unsigned long irqflags;
+
+ spin_lock_irqsave(&lldev->lock, irqflags);
+ writel(lldev->tre_write_offset, lldev->trca + HIDMA_TRCA_DOORBELL_REG);
+ spin_unlock_irqrestore(&lldev->lock, irqflags);
+}
+
+bool hidma_ll_isenabled(struct hidma_lldev *lldev)
+{
+ u32 val;
+
+ val = readl(lldev->trca + HIDMA_TRCA_CTRLSTS_REG);
+ lldev->trch_state = HIDMA_CH_STATE(val);
+ val = readl(lldev->evca + HIDMA_EVCA_CTRLSTS_REG);
+ lldev->evch_state = HIDMA_CH_STATE(val);
+
+ /* both channels have to be enabled before calling this function */
+ if (hidma_is_chan_enabled(lldev->trch_state) &&
+ hidma_is_chan_enabled(lldev->evch_state))
+ return true;
+
+ return false;
+}
+
+void hidma_ll_queue_request(struct hidma_lldev *lldev, u32 tre_ch)
+{
+ struct hidma_tre *tre;
+ unsigned long flags;
+
+ tre = &lldev->trepool[tre_ch];
+
+ /* copy the TRE into its location in the TRE ring */
+ spin_lock_irqsave(&lldev->lock, flags);
+ tre->tre_index = lldev->tre_write_offset / HIDMA_TRE_SIZE;
+ lldev->pending_tre_list[tre->tre_index] = tre;
+ memcpy(lldev->tre_ring + lldev->tre_write_offset,
+ &tre->tre_local[0], HIDMA_TRE_SIZE);
+ tre->err_code = 0;
+ tre->err_info = 0;
+ tre->queued = 1;
+ lldev->pending_tre_count++;
+ lldev->tre_write_offset = (lldev->tre_write_offset + HIDMA_TRE_SIZE)
+ % lldev->tre_ring_size;
+ spin_unlock_irqrestore(&lldev->lock, flags);
+}
+
+/*
+ * Note that even though we stop this channel if there is a pending transaction
+ * in flight it will complete and follow the callback. This request will
+ * prevent further requests to be made.
+ */
+int hidma_ll_disable(struct hidma_lldev *lldev)
+{
+ u32 val;
+ int ret;
+
+ val = readl(lldev->evca + HIDMA_EVCA_CTRLSTS_REG);
+ lldev->evch_state = HIDMA_CH_STATE(val);
+ val = readl(lldev->trca + HIDMA_TRCA_CTRLSTS_REG);
+ lldev->trch_state = HIDMA_CH_STATE(val);
+
+ /* already suspended by this OS */
+ if ((lldev->trch_state == HIDMA_CH_SUSPENDED) ||
+ (lldev->evch_state == HIDMA_CH_SUSPENDED))
+ return 0;
+
+ /* already stopped by the manager */
+ if ((lldev->trch_state == HIDMA_CH_STOPPED) ||
+ (lldev->evch_state == HIDMA_CH_STOPPED))
+ return 0;
+
+ val = readl(lldev->trca + HIDMA_TRCA_CTRLSTS_REG);
+ val &= ~(HIDMA_CH_CONTROL_MASK << 16);
+ val |= HIDMA_CH_SUSPEND << 16;
+ writel(val, lldev->trca + HIDMA_TRCA_CTRLSTS_REG);
+
+ /*
+ * Start the wait right after the suspend is confirmed.
+ * Do a polled read up to 1ms and 10ms maximum.
+ */
+ ret = readl_poll_timeout(lldev->trca + HIDMA_TRCA_CTRLSTS_REG, val,
+ HIDMA_CH_STATE(val) == HIDMA_CH_SUSPENDED,
+ 1000, 10000);
+ if (ret)
+ return ret;
+
+ val = readl(lldev->evca + HIDMA_EVCA_CTRLSTS_REG);
+ val &= ~(HIDMA_CH_CONTROL_MASK << 16);
+ val |= HIDMA_CH_SUSPEND << 16;
+ writel(val, lldev->evca + HIDMA_EVCA_CTRLSTS_REG);
+
+ /*
+ * Start the wait right after the suspend is confirmed
+ * Delay up to 10ms after reset to allow DMA logic to quiesce.
+ */
+ ret = readl_poll_timeout(lldev->evca + HIDMA_EVCA_CTRLSTS_REG, val,
+ HIDMA_CH_STATE(val) == HIDMA_CH_SUSPENDED,
+ 1000, 10000);
+ if (ret)
+ return ret;
+
+ lldev->trch_state = HIDMA_CH_SUSPENDED;
+ lldev->evch_state = HIDMA_CH_SUSPENDED;
+ return 0;
+}
+
+void hidma_ll_set_transfer_params(struct hidma_lldev *lldev, u32 tre_ch,
+ dma_addr_t src, dma_addr_t dest, u32 len,
+ u32 flags)
+{
+ struct hidma_tre *tre;
+ u32 *tre_local;
+
+ if (tre_ch >= lldev->nr_tres) {
+ dev_err(lldev->dev, "invalid TRE number in transfer params:%d",
+ tre_ch);
+ return;
+ }
+
+ tre = &lldev->trepool[tre_ch];
+ if (atomic_read(&tre->allocated) != true) {
+ dev_err(lldev->dev, "trying to set params on an unused TRE:%d",
+ tre_ch);
+ return;
+ }
+
+ tre_local = &tre->tre_local[0];
+ tre_local[HIDMA_TRE_LEN_IDX] = len;
+ tre_local[HIDMA_TRE_SRC_LOW_IDX] = lower_32_bits(src);
+ tre_local[HIDMA_TRE_SRC_HI_IDX] = upper_32_bits(src);
+ tre_local[HIDMA_TRE_DEST_LOW_IDX] = lower_32_bits(dest);
+ tre_local[HIDMA_TRE_DEST_HI_IDX] = upper_32_bits(dest);
+ tre->int_flags = flags;
+}
+
+/*
+ * Called during initialization and after an error condition
+ * to restore hardware state.
+ */
+int hidma_ll_setup(struct hidma_lldev *lldev)
+{
+ int rc;
+ u64 addr;
+ u32 val;
+ u32 nr_tres = lldev->nr_tres;
+
+ lldev->pending_tre_count = 0;
+ lldev->tre_processed_off = 0;
+ lldev->evre_processed_off = 0;
+ lldev->tre_write_offset = 0;
+
+ /* disable interrupts */
+ writel(0, lldev->evca + HIDMA_EVCA_IRQ_EN_REG);
+
+ /* clear all pending interrupts */
+ val = readl(lldev->evca + HIDMA_EVCA_IRQ_STAT_REG);
+ writel(val, lldev->evca + HIDMA_EVCA_IRQ_CLR_REG);
+
+ rc = hidma_ll_reset(lldev);
+ if (rc)
+ return rc;
+
+ /*
+ * Clear all pending interrupts again.
+ * Otherwise, we observe reset complete interrupts.
+ */
+ val = readl(lldev->evca + HIDMA_EVCA_IRQ_STAT_REG);
+ writel(val, lldev->evca + HIDMA_EVCA_IRQ_CLR_REG);
+
+ /* disable interrupts again after reset */
+ writel(0, lldev->evca + HIDMA_EVCA_IRQ_EN_REG);
+
+ addr = lldev->tre_dma;
+ writel(lower_32_bits(addr), lldev->trca + HIDMA_TRCA_RING_LOW_REG);
+ writel(upper_32_bits(addr), lldev->trca + HIDMA_TRCA_RING_HIGH_REG);
+ writel(lldev->tre_ring_size, lldev->trca + HIDMA_TRCA_RING_LEN_REG);
+
+ addr = lldev->evre_dma;
+ writel(lower_32_bits(addr), lldev->evca + HIDMA_EVCA_RING_LOW_REG);
+ writel(upper_32_bits(addr), lldev->evca + HIDMA_EVCA_RING_HIGH_REG);
+ writel(HIDMA_EVRE_SIZE * nr_tres,
+ lldev->evca + HIDMA_EVCA_RING_LEN_REG);
+
+ /* support IRQ only for now */
+ val = readl(lldev->evca + HIDMA_EVCA_INTCTRL_REG);
+ val &= ~0xF;
+ val |= 0x1;
+ writel(val, lldev->evca + HIDMA_EVCA_INTCTRL_REG);
+
+ /* clear all pending interrupts and enable them */
+ writel(ENABLE_IRQS, lldev->evca + HIDMA_EVCA_IRQ_CLR_REG);
+ writel(ENABLE_IRQS, lldev->evca + HIDMA_EVCA_IRQ_EN_REG);
+
+ return hidma_ll_enable(lldev);
+}
+
+struct hidma_lldev *hidma_ll_init(struct device *dev, u32 nr_tres,
+ void __iomem *trca, void __iomem *evca,
+ u8 chidx)
+{
+ u32 required_bytes;
+ struct hidma_lldev *lldev;
+ int rc;
+ size_t sz;
+
+ if (!trca || !evca || !dev || !nr_tres)
+ return NULL;
+
+ /* need at least four TREs */
+ if (nr_tres < 4)
+ return NULL;
+
+ /* need an extra space */
+ nr_tres += 1;
+
+ lldev = devm_kzalloc(dev, sizeof(struct hidma_lldev), GFP_KERNEL);
+ if (!lldev)
+ return NULL;
+
+ lldev->evca = evca;
+ lldev->trca = trca;
+ lldev->dev = dev;
+ sz = sizeof(struct hidma_tre);
+ lldev->trepool = devm_kcalloc(lldev->dev, nr_tres, sz, GFP_KERNEL);
+ if (!lldev->trepool)
+ return NULL;
+
+ required_bytes = sizeof(lldev->pending_tre_list[0]);
+ lldev->pending_tre_list = devm_kcalloc(dev, nr_tres, required_bytes,
+ GFP_KERNEL);
+ if (!lldev->pending_tre_list)
+ return NULL;
+
+ sz = (HIDMA_TRE_SIZE + 1) * nr_tres;
+ lldev->tre_ring = dmam_alloc_coherent(dev, sz, &lldev->tre_dma,
+ GFP_KERNEL);
+ if (!lldev->tre_ring)
+ return NULL;
+
+ memset(lldev->tre_ring, 0, (HIDMA_TRE_SIZE + 1) * nr_tres);
+ lldev->tre_ring_size = HIDMA_TRE_SIZE * nr_tres;
+ lldev->nr_tres = nr_tres;
+
+ /* the TRE ring has to be TRE_SIZE aligned */
+ if (!IS_ALIGNED(lldev->tre_dma, HIDMA_TRE_SIZE)) {
+ u8 tre_ring_shift;
+
+ tre_ring_shift = lldev->tre_dma % HIDMA_TRE_SIZE;
+ tre_ring_shift = HIDMA_TRE_SIZE - tre_ring_shift;
+ lldev->tre_dma += tre_ring_shift;
+ lldev->tre_ring += tre_ring_shift;
+ }
+
+ sz = (HIDMA_EVRE_SIZE + 1) * nr_tres;
+ lldev->evre_ring = dmam_alloc_coherent(dev, sz, &lldev->evre_dma,
+ GFP_KERNEL);
+ if (!lldev->evre_ring)
+ return NULL;
+
+ memset(lldev->evre_ring, 0, (HIDMA_EVRE_SIZE + 1) * nr_tres);
+ lldev->evre_ring_size = HIDMA_EVRE_SIZE * nr_tres;
+
+ /* the EVRE ring has to be EVRE_SIZE aligned */
+ if (!IS_ALIGNED(lldev->evre_dma, HIDMA_EVRE_SIZE)) {
+ u8 evre_ring_shift;
+
+ evre_ring_shift = lldev->evre_dma % HIDMA_EVRE_SIZE;
+ evre_ring_shift = HIDMA_EVRE_SIZE - evre_ring_shift;
+ lldev->evre_dma += evre_ring_shift;
+ lldev->evre_ring += evre_ring_shift;
+ }
+ lldev->nr_tres = nr_tres;
+ lldev->chidx = chidx;
+
+ sz = nr_tres * sizeof(struct hidma_tre *);
+ rc = kfifo_alloc(&lldev->handoff_fifo, sz, GFP_KERNEL);
+ if (rc)
+ return NULL;
+
+ rc = hidma_ll_setup(lldev);
+ if (rc)
+ return NULL;
+
+ spin_lock_init(&lldev->lock);
+ tasklet_init(&lldev->rst_task, hidma_ll_abort, (unsigned long)lldev);
+ tasklet_init(&lldev->task, hidma_ll_tre_complete, (unsigned long)lldev);
+ lldev->initialized = 1;
+ writel(ENABLE_IRQS, lldev->evca + HIDMA_EVCA_IRQ_EN_REG);
+ return lldev;
+}
+
+int hidma_ll_uninit(struct hidma_lldev *lldev)
+{
+ u32 required_bytes;
+ int rc = 0;
+ u32 val;
+
+ if (!lldev)
+ return -ENODEV;
+
+ if (!lldev->initialized)
+ return 0;
+
+ lldev->initialized = 0;
+
+ required_bytes = sizeof(struct hidma_tre) * lldev->nr_tres;
+ tasklet_kill(&lldev->task);
+ memset(lldev->trepool, 0, required_bytes);
+ lldev->trepool = NULL;
+ lldev->pending_tre_count = 0;
+ lldev->tre_write_offset = 0;
+
+ rc = hidma_ll_reset(lldev);
+
+ /*
+ * Clear all pending interrupts again.
+ * Otherwise, we observe reset complete interrupts.
+ */
+ val = readl(lldev->evca + HIDMA_EVCA_IRQ_STAT_REG);
+ writel(val, lldev->evca + HIDMA_EVCA_IRQ_CLR_REG);
+ writel(0, lldev->evca + HIDMA_EVCA_IRQ_EN_REG);
+ return rc;
+}
+
+enum dma_status hidma_ll_status(struct hidma_lldev *lldev, u32 tre_ch)
+{
+ enum dma_status ret = DMA_ERROR;
+ struct hidma_tre *tre;
+ unsigned long flags;
+ u8 err_code;
+
+ spin_lock_irqsave(&lldev->lock, flags);
+
+ tre = &lldev->trepool[tre_ch];
+ err_code = tre->err_code;
+
+ if (err_code & HIDMA_EVRE_STATUS_COMPLETE)
+ ret = DMA_COMPLETE;
+ else if (err_code & HIDMA_EVRE_STATUS_ERROR)
+ ret = DMA_ERROR;
+ else
+ ret = DMA_IN_PROGRESS;
+ spin_unlock_irqrestore(&lldev->lock, flags);
+
+ return ret;
+}
diff --git a/drivers/dma/qcom/hidma_mgmt.c b/drivers/dma/qcom/hidma_mgmt.c
index ef491b8..c0e3653 100644
--- a/drivers/dma/qcom/hidma_mgmt.c
+++ b/drivers/dma/qcom/hidma_mgmt.c
@@ -1,7 +1,7 @@
/*
* Qualcomm Technologies HIDMA DMA engine Management interface
*
- * Copyright (c) 2015, The Linux Foundation. All rights reserved.
+ * Copyright (c) 2015-2016, The Linux Foundation. All rights reserved.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 and
@@ -17,13 +17,14 @@
#include <linux/acpi.h>
#include <linux/of.h>
#include <linux/property.h>
-#include <linux/interrupt.h>
-#include <linux/platform_device.h>
+#include <linux/of_irq.h>
+#include <linux/of_platform.h>
#include <linux/module.h>
#include <linux/uaccess.h>
#include <linux/slab.h>
#include <linux/pm_runtime.h>
#include <linux/bitops.h>
+#include <linux/dma-mapping.h>
#include "hidma_mgmt.h"
@@ -298,5 +299,109 @@
},
};
-module_platform_driver(hidma_mgmt_driver);
+#if defined(CONFIG_OF) && defined(CONFIG_OF_IRQ)
+static int object_counter;
+
+static int __init hidma_mgmt_of_populate_channels(struct device_node *np)
+{
+ struct platform_device *pdev_parent = of_find_device_by_node(np);
+ struct platform_device_info pdevinfo;
+ struct of_phandle_args out_irq;
+ struct device_node *child;
+ struct resource *res;
+ const __be32 *cell;
+ int ret = 0, size, i, num;
+ u64 addr, addr_size;
+
+ for_each_available_child_of_node(np, child) {
+ struct resource *res_iter;
+ struct platform_device *new_pdev;
+
+ cell = of_get_property(child, "reg", &size);
+ if (!cell) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ size /= sizeof(*cell);
+ num = size /
+ (of_n_addr_cells(child) + of_n_size_cells(child)) + 1;
+
+ /* allocate a resource array */
+ res = kcalloc(num, sizeof(*res), GFP_KERNEL);
+ if (!res) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ /* read each reg value */
+ i = 0;
+ res_iter = res;
+ while (i < size) {
+ addr = of_read_number(&cell[i],
+ of_n_addr_cells(child));
+ i += of_n_addr_cells(child);
+
+ addr_size = of_read_number(&cell[i],
+ of_n_size_cells(child));
+ i += of_n_size_cells(child);
+
+ res_iter->start = addr;
+ res_iter->end = res_iter->start + addr_size - 1;
+ res_iter->flags = IORESOURCE_MEM;
+ res_iter++;
+ }
+
+ ret = of_irq_parse_one(child, 0, &out_irq);
+ if (ret)
+ goto out;
+
+ res_iter->start = irq_create_of_mapping(&out_irq);
+ res_iter->name = "hidma event irq";
+ res_iter->flags = IORESOURCE_IRQ;
+
+ memset(&pdevinfo, 0, sizeof(pdevinfo));
+ pdevinfo.fwnode = &child->fwnode;
+ pdevinfo.parent = pdev_parent ? &pdev_parent->dev : NULL;
+ pdevinfo.name = child->name;
+ pdevinfo.id = object_counter++;
+ pdevinfo.res = res;
+ pdevinfo.num_res = num;
+ pdevinfo.data = NULL;
+ pdevinfo.size_data = 0;
+ pdevinfo.dma_mask = DMA_BIT_MASK(64);
+ new_pdev = platform_device_register_full(&pdevinfo);
+ if (!new_pdev) {
+ ret = -ENODEV;
+ goto out;
+ }
+ of_dma_configure(&new_pdev->dev, child);
+
+ kfree(res);
+ res = NULL;
+ }
+out:
+ kfree(res);
+
+ return ret;
+}
+#endif
+
+static int __init hidma_mgmt_init(void)
+{
+#if defined(CONFIG_OF) && defined(CONFIG_OF_IRQ)
+ struct device_node *child;
+
+ for (child = of_find_matching_node(NULL, hidma_mgmt_match); child;
+ child = of_find_matching_node(child, hidma_mgmt_match)) {
+ /* device tree based firmware here */
+ hidma_mgmt_of_populate_channels(child);
+ of_node_put(child);
+ }
+#endif
+ platform_driver_register(&hidma_mgmt_driver);
+
+ return 0;
+}
+module_init(hidma_mgmt_init);
MODULE_LICENSE("GPL v2");
diff --git a/drivers/dma/sun6i-dma.c b/drivers/dma/sun6i-dma.c
index 2db12e4..5065ca4 100644
--- a/drivers/dma/sun6i-dma.c
+++ b/drivers/dma/sun6i-dma.c
@@ -146,6 +146,8 @@
struct dma_slave_config cfg;
struct sun6i_pchan *phy;
u8 port;
+ u8 irq_type;
+ bool cyclic;
};
struct sun6i_dma_dev {
@@ -254,6 +256,30 @@
return addr_width >> 1;
}
+static size_t sun6i_get_chan_size(struct sun6i_pchan *pchan)
+{
+ struct sun6i_desc *txd = pchan->desc;
+ struct sun6i_dma_lli *lli;
+ size_t bytes;
+ dma_addr_t pos;
+
+ pos = readl(pchan->base + DMA_CHAN_LLI_ADDR);
+ bytes = readl(pchan->base + DMA_CHAN_CUR_CNT);
+
+ if (pos == LLI_LAST_ITEM)
+ return bytes;
+
+ for (lli = txd->v_lli; lli; lli = lli->v_lli_next) {
+ if (lli->p_lli_next == pos) {
+ for (lli = lli->v_lli_next; lli; lli = lli->v_lli_next)
+ bytes += lli->len;
+ break;
+ }
+ }
+
+ return bytes;
+}
+
static void *sun6i_dma_lli_add(struct sun6i_dma_lli *prev,
struct sun6i_dma_lli *next,
dma_addr_t next_phy,
@@ -276,45 +302,6 @@
return next;
}
-static inline int sun6i_dma_cfg_lli(struct sun6i_dma_lli *lli,
- dma_addr_t src,
- dma_addr_t dst, u32 len,
- struct dma_slave_config *config)
-{
- u8 src_width, dst_width, src_burst, dst_burst;
-
- if (!config)
- return -EINVAL;
-
- src_burst = convert_burst(config->src_maxburst);
- if (src_burst)
- return src_burst;
-
- dst_burst = convert_burst(config->dst_maxburst);
- if (dst_burst)
- return dst_burst;
-
- src_width = convert_buswidth(config->src_addr_width);
- if (src_width)
- return src_width;
-
- dst_width = convert_buswidth(config->dst_addr_width);
- if (dst_width)
- return dst_width;
-
- lli->cfg = DMA_CHAN_CFG_SRC_BURST(src_burst) |
- DMA_CHAN_CFG_SRC_WIDTH(src_width) |
- DMA_CHAN_CFG_DST_BURST(dst_burst) |
- DMA_CHAN_CFG_DST_WIDTH(dst_width);
-
- lli->src = src;
- lli->dst = dst;
- lli->len = len;
- lli->para = NORMAL_WAIT;
-
- return 0;
-}
-
static inline void sun6i_dma_dump_lli(struct sun6i_vchan *vchan,
struct sun6i_dma_lli *lli)
{
@@ -381,9 +368,13 @@
irq_reg = pchan->idx / DMA_IRQ_CHAN_NR;
irq_offset = pchan->idx % DMA_IRQ_CHAN_NR;
- irq_val = readl(sdev->base + DMA_IRQ_EN(irq_offset));
- irq_val |= DMA_IRQ_QUEUE << (irq_offset * DMA_IRQ_CHAN_WIDTH);
- writel(irq_val, sdev->base + DMA_IRQ_EN(irq_offset));
+ vchan->irq_type = vchan->cyclic ? DMA_IRQ_PKG : DMA_IRQ_QUEUE;
+
+ irq_val = readl(sdev->base + DMA_IRQ_EN(irq_reg));
+ irq_val &= ~((DMA_IRQ_HALF | DMA_IRQ_PKG | DMA_IRQ_QUEUE) <<
+ (irq_offset * DMA_IRQ_CHAN_WIDTH));
+ irq_val |= vchan->irq_type << (irq_offset * DMA_IRQ_CHAN_WIDTH);
+ writel(irq_val, sdev->base + DMA_IRQ_EN(irq_reg));
writel(pchan->desc->p_lli, pchan->base + DMA_CHAN_LLI_ADDR);
writel(DMA_CHAN_ENABLE_START, pchan->base + DMA_CHAN_ENABLE);
@@ -479,11 +470,12 @@
writel(status, sdev->base + DMA_IRQ_STAT(i));
for (j = 0; (j < DMA_IRQ_CHAN_NR) && status; j++) {
- if (status & DMA_IRQ_QUEUE) {
- pchan = sdev->pchans + j;
- vchan = pchan->vchan;
-
- if (vchan) {
+ pchan = sdev->pchans + j;
+ vchan = pchan->vchan;
+ if (vchan && (status & vchan->irq_type)) {
+ if (vchan->cyclic) {
+ vchan_cyclic_callback(&pchan->desc->vd);
+ } else {
spin_lock(&vchan->vc.lock);
vchan_cookie_complete(&pchan->desc->vd);
pchan->done = pchan->desc;
@@ -502,6 +494,55 @@
return ret;
}
+static int set_config(struct sun6i_dma_dev *sdev,
+ struct dma_slave_config *sconfig,
+ enum dma_transfer_direction direction,
+ u32 *p_cfg)
+{
+ s8 src_width, dst_width, src_burst, dst_burst;
+
+ switch (direction) {
+ case DMA_MEM_TO_DEV:
+ src_burst = convert_burst(sconfig->src_maxburst ?
+ sconfig->src_maxburst : 8);
+ src_width = convert_buswidth(sconfig->src_addr_width !=
+ DMA_SLAVE_BUSWIDTH_UNDEFINED ?
+ sconfig->src_addr_width :
+ DMA_SLAVE_BUSWIDTH_4_BYTES);
+ dst_burst = convert_burst(sconfig->dst_maxburst);
+ dst_width = convert_buswidth(sconfig->dst_addr_width);
+ break;
+ case DMA_DEV_TO_MEM:
+ src_burst = convert_burst(sconfig->src_maxburst);
+ src_width = convert_buswidth(sconfig->src_addr_width);
+ dst_burst = convert_burst(sconfig->dst_maxburst ?
+ sconfig->dst_maxburst : 8);
+ dst_width = convert_buswidth(sconfig->dst_addr_width !=
+ DMA_SLAVE_BUSWIDTH_UNDEFINED ?
+ sconfig->dst_addr_width :
+ DMA_SLAVE_BUSWIDTH_4_BYTES);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ if (src_burst < 0)
+ return src_burst;
+ if (src_width < 0)
+ return src_width;
+ if (dst_burst < 0)
+ return dst_burst;
+ if (dst_width < 0)
+ return dst_width;
+
+ *p_cfg = DMA_CHAN_CFG_SRC_BURST(src_burst) |
+ DMA_CHAN_CFG_SRC_WIDTH(src_width) |
+ DMA_CHAN_CFG_DST_BURST(dst_burst) |
+ DMA_CHAN_CFG_DST_WIDTH(dst_width);
+
+ return 0;
+}
+
static struct dma_async_tx_descriptor *sun6i_dma_prep_dma_memcpy(
struct dma_chan *chan, dma_addr_t dest, dma_addr_t src,
size_t len, unsigned long flags)
@@ -569,13 +610,15 @@
struct sun6i_desc *txd;
struct scatterlist *sg;
dma_addr_t p_lli;
+ u32 lli_cfg;
int i, ret;
if (!sgl)
return NULL;
- if (!is_slave_direction(dir)) {
- dev_err(chan2dev(chan), "Invalid DMA direction\n");
+ ret = set_config(sdev, sconfig, dir, &lli_cfg);
+ if (ret) {
+ dev_err(chan2dev(chan), "Invalid DMA configuration\n");
return NULL;
}
@@ -588,14 +631,14 @@
if (!v_lli)
goto err_lli_free;
- if (dir == DMA_MEM_TO_DEV) {
- ret = sun6i_dma_cfg_lli(v_lli, sg_dma_address(sg),
- sconfig->dst_addr, sg_dma_len(sg),
- sconfig);
- if (ret)
- goto err_cur_lli_free;
+ v_lli->len = sg_dma_len(sg);
+ v_lli->para = NORMAL_WAIT;
- v_lli->cfg |= DMA_CHAN_CFG_DST_IO_MODE |
+ if (dir == DMA_MEM_TO_DEV) {
+ v_lli->src = sg_dma_address(sg);
+ v_lli->dst = sconfig->dst_addr;
+ v_lli->cfg = lli_cfg |
+ DMA_CHAN_CFG_DST_IO_MODE |
DMA_CHAN_CFG_SRC_LINEAR_MODE |
DMA_CHAN_CFG_SRC_DRQ(DRQ_SDRAM) |
DMA_CHAN_CFG_DST_DRQ(vchan->port);
@@ -607,13 +650,10 @@
sg_dma_len(sg), flags);
} else {
- ret = sun6i_dma_cfg_lli(v_lli, sconfig->src_addr,
- sg_dma_address(sg), sg_dma_len(sg),
- sconfig);
- if (ret)
- goto err_cur_lli_free;
-
- v_lli->cfg |= DMA_CHAN_CFG_DST_LINEAR_MODE |
+ v_lli->src = sconfig->src_addr;
+ v_lli->dst = sg_dma_address(sg);
+ v_lli->cfg = lli_cfg |
+ DMA_CHAN_CFG_DST_LINEAR_MODE |
DMA_CHAN_CFG_SRC_IO_MODE |
DMA_CHAN_CFG_DST_DRQ(DRQ_SDRAM) |
DMA_CHAN_CFG_SRC_DRQ(vchan->port);
@@ -634,8 +674,78 @@
return vchan_tx_prep(&vchan->vc, &txd->vd, flags);
-err_cur_lli_free:
- dma_pool_free(sdev->pool, v_lli, p_lli);
+err_lli_free:
+ for (prev = txd->v_lli; prev; prev = prev->v_lli_next)
+ dma_pool_free(sdev->pool, prev, virt_to_phys(prev));
+ kfree(txd);
+ return NULL;
+}
+
+static struct dma_async_tx_descriptor *sun6i_dma_prep_dma_cyclic(
+ struct dma_chan *chan,
+ dma_addr_t buf_addr,
+ size_t buf_len,
+ size_t period_len,
+ enum dma_transfer_direction dir,
+ unsigned long flags)
+{
+ struct sun6i_dma_dev *sdev = to_sun6i_dma_dev(chan->device);
+ struct sun6i_vchan *vchan = to_sun6i_vchan(chan);
+ struct dma_slave_config *sconfig = &vchan->cfg;
+ struct sun6i_dma_lli *v_lli, *prev = NULL;
+ struct sun6i_desc *txd;
+ dma_addr_t p_lli;
+ u32 lli_cfg;
+ unsigned int i, periods = buf_len / period_len;
+ int ret;
+
+ ret = set_config(sdev, sconfig, dir, &lli_cfg);
+ if (ret) {
+ dev_err(chan2dev(chan), "Invalid DMA configuration\n");
+ return NULL;
+ }
+
+ txd = kzalloc(sizeof(*txd), GFP_NOWAIT);
+ if (!txd)
+ return NULL;
+
+ for (i = 0; i < periods; i++) {
+ v_lli = dma_pool_alloc(sdev->pool, GFP_NOWAIT, &p_lli);
+ if (!v_lli) {
+ dev_err(sdev->slave.dev, "Failed to alloc lli memory\n");
+ goto err_lli_free;
+ }
+
+ v_lli->len = period_len;
+ v_lli->para = NORMAL_WAIT;
+
+ if (dir == DMA_MEM_TO_DEV) {
+ v_lli->src = buf_addr + period_len * i;
+ v_lli->dst = sconfig->dst_addr;
+ v_lli->cfg = lli_cfg |
+ DMA_CHAN_CFG_DST_IO_MODE |
+ DMA_CHAN_CFG_SRC_LINEAR_MODE |
+ DMA_CHAN_CFG_SRC_DRQ(DRQ_SDRAM) |
+ DMA_CHAN_CFG_DST_DRQ(vchan->port);
+ } else {
+ v_lli->src = sconfig->src_addr;
+ v_lli->dst = buf_addr + period_len * i;
+ v_lli->cfg = lli_cfg |
+ DMA_CHAN_CFG_DST_LINEAR_MODE |
+ DMA_CHAN_CFG_SRC_IO_MODE |
+ DMA_CHAN_CFG_DST_DRQ(DRQ_SDRAM) |
+ DMA_CHAN_CFG_SRC_DRQ(vchan->port);
+ }
+
+ prev = sun6i_dma_lli_add(prev, v_lli, p_lli, txd);
+ }
+
+ prev->p_lli_next = txd->p_lli; /* cyclic list */
+
+ vchan->cyclic = true;
+
+ return vchan_tx_prep(&vchan->vc, &txd->vd, flags);
+
err_lli_free:
for (prev = txd->v_lli; prev; prev = prev->v_lli_next)
dma_pool_free(sdev->pool, prev, virt_to_phys(prev));
@@ -712,6 +822,16 @@
spin_lock_irqsave(&vchan->vc.lock, flags);
+ if (vchan->cyclic) {
+ vchan->cyclic = false;
+ if (pchan && pchan->desc) {
+ struct virt_dma_desc *vd = &pchan->desc->vd;
+ struct virt_dma_chan *vc = &vchan->vc;
+
+ list_add_tail(&vd->node, &vc->desc_completed);
+ }
+ }
+
vchan_get_all_descriptors(&vchan->vc, &head);
if (pchan) {
@@ -759,7 +879,7 @@
} else if (!pchan || !pchan->desc) {
bytes = 0;
} else {
- bytes = readl(pchan->base + DMA_CHAN_CUR_CNT);
+ bytes = sun6i_get_chan_size(pchan);
}
spin_unlock_irqrestore(&vchan->vc.lock, flags);
@@ -963,6 +1083,7 @@
dma_cap_set(DMA_PRIVATE, sdc->slave.cap_mask);
dma_cap_set(DMA_MEMCPY, sdc->slave.cap_mask);
dma_cap_set(DMA_SLAVE, sdc->slave.cap_mask);
+ dma_cap_set(DMA_CYCLIC, sdc->slave.cap_mask);
INIT_LIST_HEAD(&sdc->slave.channels);
sdc->slave.device_free_chan_resources = sun6i_dma_free_chan_resources;
@@ -970,6 +1091,7 @@
sdc->slave.device_issue_pending = sun6i_dma_issue_pending;
sdc->slave.device_prep_slave_sg = sun6i_dma_prep_slave_sg;
sdc->slave.device_prep_dma_memcpy = sun6i_dma_prep_dma_memcpy;
+ sdc->slave.device_prep_dma_cyclic = sun6i_dma_prep_dma_cyclic;
sdc->slave.copy_align = DMAENGINE_ALIGN_4_BYTES;
sdc->slave.device_config = sun6i_dma_config;
sdc->slave.device_pause = sun6i_dma_pause;
diff --git a/drivers/dma/tegra20-apb-dma.c b/drivers/dma/tegra20-apb-dma.c
index 3871f29..01e316f 100644
--- a/drivers/dma/tegra20-apb-dma.c
+++ b/drivers/dma/tegra20-apb-dma.c
@@ -54,6 +54,7 @@
#define TEGRA_APBDMA_CSR_ONCE BIT(27)
#define TEGRA_APBDMA_CSR_FLOW BIT(21)
#define TEGRA_APBDMA_CSR_REQ_SEL_SHIFT 16
+#define TEGRA_APBDMA_CSR_REQ_SEL_MASK 0x1F
#define TEGRA_APBDMA_CSR_WCOUNT_MASK 0xFFFC
/* STATUS register */
@@ -114,6 +115,8 @@
/* Channel base address offset from APBDMA base address */
#define TEGRA_APBDMA_CHANNEL_BASE_ADD_OFFSET 0x1000
+#define TEGRA_APBDMA_SLAVE_ID_INVALID (TEGRA_APBDMA_CSR_REQ_SEL_MASK + 1)
+
struct tegra_dma;
/*
@@ -353,8 +356,11 @@
}
memcpy(&tdc->dma_sconfig, sconfig, sizeof(*sconfig));
- if (!tdc->slave_id)
+ if (tdc->slave_id == TEGRA_APBDMA_SLAVE_ID_INVALID) {
+ if (sconfig->slave_id > TEGRA_APBDMA_CSR_REQ_SEL_MASK)
+ return -EINVAL;
tdc->slave_id = sconfig->slave_id;
+ }
tdc->config_init = true;
return 0;
}
@@ -1236,7 +1242,7 @@
}
pm_runtime_put(tdma->dev);
- tdc->slave_id = 0;
+ tdc->slave_id = TEGRA_APBDMA_SLAVE_ID_INVALID;
}
static struct dma_chan *tegra_dma_of_xlate(struct of_phandle_args *dma_spec,
@@ -1246,6 +1252,11 @@
struct dma_chan *chan;
struct tegra_dma_channel *tdc;
+ if (dma_spec->args[0] > TEGRA_APBDMA_CSR_REQ_SEL_MASK) {
+ dev_err(tdma->dev, "Invalid slave id: %d\n", dma_spec->args[0]);
+ return NULL;
+ }
+
chan = dma_get_any_slave_channel(&tdma->dma_dev);
if (!chan)
return NULL;
@@ -1389,6 +1400,7 @@
&tdma->dma_dev.channels);
tdc->tdma = tdma;
tdc->id = i;
+ tdc->slave_id = TEGRA_APBDMA_SLAVE_ID_INVALID;
tasklet_init(&tdc->tasklet, tegra_dma_tasklet,
(unsigned long)tdc);
diff --git a/drivers/dma/tegra210-adma.c b/drivers/dma/tegra210-adma.c
new file mode 100644
index 0000000..c4b121c
--- /dev/null
+++ b/drivers/dma/tegra210-adma.c
@@ -0,0 +1,840 @@
+/*
+ * ADMA driver for Nvidia's Tegra210 ADMA controller.
+ *
+ * Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/clk.h>
+#include <linux/iopoll.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/of_dma.h>
+#include <linux/of_irq.h>
+#include <linux/pm_clock.h>
+#include <linux/pm_runtime.h>
+#include <linux/slab.h>
+
+#include "virt-dma.h"
+
+#define ADMA_CH_CMD 0x00
+#define ADMA_CH_STATUS 0x0c
+#define ADMA_CH_STATUS_XFER_EN BIT(0)
+
+#define ADMA_CH_INT_STATUS 0x10
+#define ADMA_CH_INT_STATUS_XFER_DONE BIT(0)
+
+#define ADMA_CH_INT_CLEAR 0x1c
+#define ADMA_CH_CTRL 0x24
+#define ADMA_CH_CTRL_TX_REQ(val) (((val) & 0xf) << 28)
+#define ADMA_CH_CTRL_TX_REQ_MAX 10
+#define ADMA_CH_CTRL_RX_REQ(val) (((val) & 0xf) << 24)
+#define ADMA_CH_CTRL_RX_REQ_MAX 10
+#define ADMA_CH_CTRL_DIR(val) (((val) & 0xf) << 12)
+#define ADMA_CH_CTRL_DIR_AHUB2MEM 2
+#define ADMA_CH_CTRL_DIR_MEM2AHUB 4
+#define ADMA_CH_CTRL_MODE_CONTINUOUS (2 << 8)
+#define ADMA_CH_CTRL_FLOWCTRL_EN BIT(1)
+
+#define ADMA_CH_CONFIG 0x28
+#define ADMA_CH_CONFIG_SRC_BUF(val) (((val) & 0x7) << 28)
+#define ADMA_CH_CONFIG_TRG_BUF(val) (((val) & 0x7) << 24)
+#define ADMA_CH_CONFIG_BURST_SIZE(val) (((val) & 0x7) << 20)
+#define ADMA_CH_CONFIG_BURST_16 5
+#define ADMA_CH_CONFIG_WEIGHT_FOR_WRR(val) ((val) & 0xf)
+#define ADMA_CH_CONFIG_MAX_BUFS 8
+
+#define ADMA_CH_FIFO_CTRL 0x2c
+#define ADMA_CH_FIFO_CTRL_OVRFW_THRES(val) (((val) & 0xf) << 24)
+#define ADMA_CH_FIFO_CTRL_STARV_THRES(val) (((val) & 0xf) << 16)
+#define ADMA_CH_FIFO_CTRL_TX_SIZE(val) (((val) & 0xf) << 8)
+#define ADMA_CH_FIFO_CTRL_RX_SIZE(val) ((val) & 0xf)
+
+#define ADMA_CH_LOWER_SRC_ADDR 0x34
+#define ADMA_CH_LOWER_TRG_ADDR 0x3c
+#define ADMA_CH_TC 0x44
+#define ADMA_CH_TC_COUNT_MASK 0x3ffffffc
+
+#define ADMA_CH_XFER_STATUS 0x54
+#define ADMA_CH_XFER_STATUS_COUNT_MASK 0xffff
+
+#define ADMA_GLOBAL_CMD 0xc00
+#define ADMA_GLOBAL_SOFT_RESET 0xc04
+#define ADMA_GLOBAL_INT_CLEAR 0xc20
+#define ADMA_GLOBAL_CTRL 0xc24
+
+#define ADMA_CH_REG_OFFSET(a) (a * 0x80)
+
+#define ADMA_CH_FIFO_CTRL_DEFAULT (ADMA_CH_FIFO_CTRL_OVRFW_THRES(1) | \
+ ADMA_CH_FIFO_CTRL_STARV_THRES(1) | \
+ ADMA_CH_FIFO_CTRL_TX_SIZE(3) | \
+ ADMA_CH_FIFO_CTRL_RX_SIZE(3))
+struct tegra_adma;
+
+/*
+ * struct tegra_adma_chip_data - Tegra chip specific data
+ * @nr_channels: Number of DMA channels available.
+ */
+struct tegra_adma_chip_data {
+ int nr_channels;
+};
+
+/*
+ * struct tegra_adma_chan_regs - Tegra ADMA channel registers
+ */
+struct tegra_adma_chan_regs {
+ unsigned int ctrl;
+ unsigned int config;
+ unsigned int src_addr;
+ unsigned int trg_addr;
+ unsigned int fifo_ctrl;
+ unsigned int tc;
+};
+
+/*
+ * struct tegra_adma_desc - Tegra ADMA descriptor to manage transfer requests.
+ */
+struct tegra_adma_desc {
+ struct virt_dma_desc vd;
+ struct tegra_adma_chan_regs ch_regs;
+ size_t buf_len;
+ size_t period_len;
+ size_t num_periods;
+};
+
+/*
+ * struct tegra_adma_chan - Tegra ADMA channel information
+ */
+struct tegra_adma_chan {
+ struct virt_dma_chan vc;
+ struct tegra_adma_desc *desc;
+ struct tegra_adma *tdma;
+ int irq;
+ void __iomem *chan_addr;
+
+ /* Slave channel configuration info */
+ struct dma_slave_config sconfig;
+ enum dma_transfer_direction sreq_dir;
+ unsigned int sreq_index;
+ bool sreq_reserved;
+
+ /* Transfer count and position info */
+ unsigned int tx_buf_count;
+ unsigned int tx_buf_pos;
+};
+
+/*
+ * struct tegra_adma - Tegra ADMA controller information
+ */
+struct tegra_adma {
+ struct dma_device dma_dev;
+ struct device *dev;
+ void __iomem *base_addr;
+ unsigned int nr_channels;
+ unsigned long rx_requests_reserved;
+ unsigned long tx_requests_reserved;
+
+ /* Used to store global command register state when suspending */
+ unsigned int global_cmd;
+
+ /* Last member of the structure */
+ struct tegra_adma_chan channels[0];
+};
+
+static inline void tdma_write(struct tegra_adma *tdma, u32 reg, u32 val)
+{
+ writel(val, tdma->base_addr + reg);
+}
+
+static inline u32 tdma_read(struct tegra_adma *tdma, u32 reg)
+{
+ return readl(tdma->base_addr + reg);
+}
+
+static inline void tdma_ch_write(struct tegra_adma_chan *tdc, u32 reg, u32 val)
+{
+ writel(val, tdc->chan_addr + reg);
+}
+
+static inline u32 tdma_ch_read(struct tegra_adma_chan *tdc, u32 reg)
+{
+ return readl(tdc->chan_addr + reg);
+}
+
+static inline struct tegra_adma_chan *to_tegra_adma_chan(struct dma_chan *dc)
+{
+ return container_of(dc, struct tegra_adma_chan, vc.chan);
+}
+
+static inline struct tegra_adma_desc *to_tegra_adma_desc(
+ struct dma_async_tx_descriptor *td)
+{
+ return container_of(td, struct tegra_adma_desc, vd.tx);
+}
+
+static inline struct device *tdc2dev(struct tegra_adma_chan *tdc)
+{
+ return tdc->tdma->dev;
+}
+
+static void tegra_adma_desc_free(struct virt_dma_desc *vd)
+{
+ kfree(container_of(vd, struct tegra_adma_desc, vd));
+}
+
+static int tegra_adma_slave_config(struct dma_chan *dc,
+ struct dma_slave_config *sconfig)
+{
+ struct tegra_adma_chan *tdc = to_tegra_adma_chan(dc);
+
+ memcpy(&tdc->sconfig, sconfig, sizeof(*sconfig));
+
+ return 0;
+}
+
+static int tegra_adma_init(struct tegra_adma *tdma)
+{
+ u32 status;
+ int ret;
+
+ /* Clear any interrupts */
+ tdma_write(tdma, ADMA_GLOBAL_INT_CLEAR, 0x1);
+
+ /* Assert soft reset */
+ tdma_write(tdma, ADMA_GLOBAL_SOFT_RESET, 0x1);
+
+ /* Wait for reset to clear */
+ ret = readx_poll_timeout(readl,
+ tdma->base_addr + ADMA_GLOBAL_SOFT_RESET,
+ status, status == 0, 20, 10000);
+ if (ret)
+ return ret;
+
+ /* Enable global ADMA registers */
+ tdma_write(tdma, ADMA_GLOBAL_CMD, 1);
+
+ return 0;
+}
+
+static int tegra_adma_request_alloc(struct tegra_adma_chan *tdc,
+ enum dma_transfer_direction direction)
+{
+ struct tegra_adma *tdma = tdc->tdma;
+ unsigned int sreq_index = tdc->sreq_index;
+
+ if (tdc->sreq_reserved)
+ return tdc->sreq_dir == direction ? 0 : -EINVAL;
+
+ switch (direction) {
+ case DMA_MEM_TO_DEV:
+ if (sreq_index > ADMA_CH_CTRL_TX_REQ_MAX) {
+ dev_err(tdma->dev, "invalid DMA request\n");
+ return -EINVAL;
+ }
+
+ if (test_and_set_bit(sreq_index, &tdma->tx_requests_reserved)) {
+ dev_err(tdma->dev, "DMA request reserved\n");
+ return -EINVAL;
+ }
+ break;
+
+ case DMA_DEV_TO_MEM:
+ if (sreq_index > ADMA_CH_CTRL_RX_REQ_MAX) {
+ dev_err(tdma->dev, "invalid DMA request\n");
+ return -EINVAL;
+ }
+
+ if (test_and_set_bit(sreq_index, &tdma->rx_requests_reserved)) {
+ dev_err(tdma->dev, "DMA request reserved\n");
+ return -EINVAL;
+ }
+ break;
+
+ default:
+ dev_WARN(tdma->dev, "channel %s has invalid transfer type\n",
+ dma_chan_name(&tdc->vc.chan));
+ return -EINVAL;
+ }
+
+ tdc->sreq_dir = direction;
+ tdc->sreq_reserved = true;
+
+ return 0;
+}
+
+static void tegra_adma_request_free(struct tegra_adma_chan *tdc)
+{
+ struct tegra_adma *tdma = tdc->tdma;
+
+ if (!tdc->sreq_reserved)
+ return;
+
+ switch (tdc->sreq_dir) {
+ case DMA_MEM_TO_DEV:
+ clear_bit(tdc->sreq_index, &tdma->tx_requests_reserved);
+ break;
+
+ case DMA_DEV_TO_MEM:
+ clear_bit(tdc->sreq_index, &tdma->rx_requests_reserved);
+ break;
+
+ default:
+ dev_WARN(tdma->dev, "channel %s has invalid transfer type\n",
+ dma_chan_name(&tdc->vc.chan));
+ return;
+ }
+
+ tdc->sreq_reserved = false;
+}
+
+static u32 tegra_adma_irq_status(struct tegra_adma_chan *tdc)
+{
+ u32 status = tdma_ch_read(tdc, ADMA_CH_INT_STATUS);
+
+ return status & ADMA_CH_INT_STATUS_XFER_DONE;
+}
+
+static u32 tegra_adma_irq_clear(struct tegra_adma_chan *tdc)
+{
+ u32 status = tegra_adma_irq_status(tdc);
+
+ if (status)
+ tdma_ch_write(tdc, ADMA_CH_INT_CLEAR, status);
+
+ return status;
+}
+
+static void tegra_adma_stop(struct tegra_adma_chan *tdc)
+{
+ unsigned int status;
+
+ /* Disable ADMA */
+ tdma_ch_write(tdc, ADMA_CH_CMD, 0);
+
+ /* Clear interrupt status */
+ tegra_adma_irq_clear(tdc);
+
+ if (readx_poll_timeout_atomic(readl, tdc->chan_addr + ADMA_CH_STATUS,
+ status, !(status & ADMA_CH_STATUS_XFER_EN),
+ 20, 10000)) {
+ dev_err(tdc2dev(tdc), "unable to stop DMA channel\n");
+ return;
+ }
+
+ kfree(tdc->desc);
+ tdc->desc = NULL;
+}
+
+static void tegra_adma_start(struct tegra_adma_chan *tdc)
+{
+ struct virt_dma_desc *vd = vchan_next_desc(&tdc->vc);
+ struct tegra_adma_chan_regs *ch_regs;
+ struct tegra_adma_desc *desc;
+
+ if (!vd)
+ return;
+
+ list_del(&vd->node);
+
+ desc = to_tegra_adma_desc(&vd->tx);
+
+ if (!desc) {
+ dev_warn(tdc2dev(tdc), "unable to start DMA, no descriptor\n");
+ return;
+ }
+
+ ch_regs = &desc->ch_regs;
+
+ tdc->tx_buf_pos = 0;
+ tdc->tx_buf_count = 0;
+ tdma_ch_write(tdc, ADMA_CH_TC, ch_regs->tc);
+ tdma_ch_write(tdc, ADMA_CH_CTRL, ch_regs->ctrl);
+ tdma_ch_write(tdc, ADMA_CH_LOWER_SRC_ADDR, ch_regs->src_addr);
+ tdma_ch_write(tdc, ADMA_CH_LOWER_TRG_ADDR, ch_regs->trg_addr);
+ tdma_ch_write(tdc, ADMA_CH_FIFO_CTRL, ch_regs->fifo_ctrl);
+ tdma_ch_write(tdc, ADMA_CH_CONFIG, ch_regs->config);
+
+ /* Start ADMA */
+ tdma_ch_write(tdc, ADMA_CH_CMD, 1);
+
+ tdc->desc = desc;
+}
+
+static unsigned int tegra_adma_get_residue(struct tegra_adma_chan *tdc)
+{
+ struct tegra_adma_desc *desc = tdc->desc;
+ unsigned int max = ADMA_CH_XFER_STATUS_COUNT_MASK + 1;
+ unsigned int pos = tdma_ch_read(tdc, ADMA_CH_XFER_STATUS);
+ unsigned int periods_remaining;
+
+ /*
+ * Handle wrap around of buffer count register
+ */
+ if (pos < tdc->tx_buf_pos)
+ tdc->tx_buf_count += pos + (max - tdc->tx_buf_pos);
+ else
+ tdc->tx_buf_count += pos - tdc->tx_buf_pos;
+
+ periods_remaining = tdc->tx_buf_count % desc->num_periods;
+ tdc->tx_buf_pos = pos;
+
+ return desc->buf_len - (periods_remaining * desc->period_len);
+}
+
+static irqreturn_t tegra_adma_isr(int irq, void *dev_id)
+{
+ struct tegra_adma_chan *tdc = dev_id;
+ unsigned long status;
+ unsigned long flags;
+
+ spin_lock_irqsave(&tdc->vc.lock, flags);
+
+ status = tegra_adma_irq_clear(tdc);
+ if (status == 0 || !tdc->desc) {
+ spin_unlock_irqrestore(&tdc->vc.lock, flags);
+ return IRQ_NONE;
+ }
+
+ vchan_cyclic_callback(&tdc->desc->vd);
+
+ spin_unlock_irqrestore(&tdc->vc.lock, flags);
+
+ return IRQ_HANDLED;
+}
+
+static void tegra_adma_issue_pending(struct dma_chan *dc)
+{
+ struct tegra_adma_chan *tdc = to_tegra_adma_chan(dc);
+ unsigned long flags;
+
+ spin_lock_irqsave(&tdc->vc.lock, flags);
+
+ if (vchan_issue_pending(&tdc->vc)) {
+ if (!tdc->desc)
+ tegra_adma_start(tdc);
+ }
+
+ spin_unlock_irqrestore(&tdc->vc.lock, flags);
+}
+
+static int tegra_adma_terminate_all(struct dma_chan *dc)
+{
+ struct tegra_adma_chan *tdc = to_tegra_adma_chan(dc);
+ unsigned long flags;
+ LIST_HEAD(head);
+
+ spin_lock_irqsave(&tdc->vc.lock, flags);
+
+ if (tdc->desc)
+ tegra_adma_stop(tdc);
+
+ tegra_adma_request_free(tdc);
+ vchan_get_all_descriptors(&tdc->vc, &head);
+ spin_unlock_irqrestore(&tdc->vc.lock, flags);
+ vchan_dma_desc_free_list(&tdc->vc, &head);
+
+ return 0;
+}
+
+static enum dma_status tegra_adma_tx_status(struct dma_chan *dc,
+ dma_cookie_t cookie,
+ struct dma_tx_state *txstate)
+{
+ struct tegra_adma_chan *tdc = to_tegra_adma_chan(dc);
+ struct tegra_adma_desc *desc;
+ struct virt_dma_desc *vd;
+ enum dma_status ret;
+ unsigned long flags;
+ unsigned int residual;
+
+ ret = dma_cookie_status(dc, cookie, txstate);
+ if (ret == DMA_COMPLETE || !txstate)
+ return ret;
+
+ spin_lock_irqsave(&tdc->vc.lock, flags);
+
+ vd = vchan_find_desc(&tdc->vc, cookie);
+ if (vd) {
+ desc = to_tegra_adma_desc(&vd->tx);
+ residual = desc->ch_regs.tc;
+ } else if (tdc->desc && tdc->desc->vd.tx.cookie == cookie) {
+ residual = tegra_adma_get_residue(tdc);
+ } else {
+ residual = 0;
+ }
+
+ spin_unlock_irqrestore(&tdc->vc.lock, flags);
+
+ dma_set_residue(txstate, residual);
+
+ return ret;
+}
+
+static int tegra_adma_set_xfer_params(struct tegra_adma_chan *tdc,
+ struct tegra_adma_desc *desc,
+ dma_addr_t buf_addr,
+ enum dma_transfer_direction direction)
+{
+ struct tegra_adma_chan_regs *ch_regs = &desc->ch_regs;
+ unsigned int burst_size, adma_dir;
+
+ if (desc->num_periods > ADMA_CH_CONFIG_MAX_BUFS)
+ return -EINVAL;
+
+ switch (direction) {
+ case DMA_MEM_TO_DEV:
+ adma_dir = ADMA_CH_CTRL_DIR_MEM2AHUB;
+ burst_size = fls(tdc->sconfig.dst_maxburst);
+ ch_regs->config = ADMA_CH_CONFIG_SRC_BUF(desc->num_periods - 1);
+ ch_regs->ctrl = ADMA_CH_CTRL_TX_REQ(tdc->sreq_index);
+ ch_regs->src_addr = buf_addr;
+ break;
+
+ case DMA_DEV_TO_MEM:
+ adma_dir = ADMA_CH_CTRL_DIR_AHUB2MEM;
+ burst_size = fls(tdc->sconfig.src_maxburst);
+ ch_regs->config = ADMA_CH_CONFIG_TRG_BUF(desc->num_periods - 1);
+ ch_regs->ctrl = ADMA_CH_CTRL_RX_REQ(tdc->sreq_index);
+ ch_regs->trg_addr = buf_addr;
+ break;
+
+ default:
+ dev_err(tdc2dev(tdc), "DMA direction is not supported\n");
+ return -EINVAL;
+ }
+
+ if (!burst_size || burst_size > ADMA_CH_CONFIG_BURST_16)
+ burst_size = ADMA_CH_CONFIG_BURST_16;
+
+ ch_regs->ctrl |= ADMA_CH_CTRL_DIR(adma_dir) |
+ ADMA_CH_CTRL_MODE_CONTINUOUS |
+ ADMA_CH_CTRL_FLOWCTRL_EN;
+ ch_regs->config |= ADMA_CH_CONFIG_BURST_SIZE(burst_size);
+ ch_regs->config |= ADMA_CH_CONFIG_WEIGHT_FOR_WRR(1);
+ ch_regs->fifo_ctrl = ADMA_CH_FIFO_CTRL_DEFAULT;
+ ch_regs->tc = desc->period_len & ADMA_CH_TC_COUNT_MASK;
+
+ return tegra_adma_request_alloc(tdc, direction);
+}
+
+static struct dma_async_tx_descriptor *tegra_adma_prep_dma_cyclic(
+ struct dma_chan *dc, dma_addr_t buf_addr, size_t buf_len,
+ size_t period_len, enum dma_transfer_direction direction,
+ unsigned long flags)
+{
+ struct tegra_adma_chan *tdc = to_tegra_adma_chan(dc);
+ struct tegra_adma_desc *desc = NULL;
+
+ if (!buf_len || !period_len || period_len > ADMA_CH_TC_COUNT_MASK) {
+ dev_err(tdc2dev(tdc), "invalid buffer/period len\n");
+ return NULL;
+ }
+
+ if (buf_len % period_len) {
+ dev_err(tdc2dev(tdc), "buf_len not a multiple of period_len\n");
+ return NULL;
+ }
+
+ if (!IS_ALIGNED(buf_addr, 4)) {
+ dev_err(tdc2dev(tdc), "invalid buffer alignment\n");
+ return NULL;
+ }
+
+ desc = kzalloc(sizeof(*desc), GFP_NOWAIT);
+ if (!desc)
+ return NULL;
+
+ desc->buf_len = buf_len;
+ desc->period_len = period_len;
+ desc->num_periods = buf_len / period_len;
+
+ if (tegra_adma_set_xfer_params(tdc, desc, buf_addr, direction)) {
+ kfree(desc);
+ return NULL;
+ }
+
+ return vchan_tx_prep(&tdc->vc, &desc->vd, flags);
+}
+
+static int tegra_adma_alloc_chan_resources(struct dma_chan *dc)
+{
+ struct tegra_adma_chan *tdc = to_tegra_adma_chan(dc);
+ int ret;
+
+ ret = request_irq(tdc->irq, tegra_adma_isr, 0, dma_chan_name(dc), tdc);
+ if (ret) {
+ dev_err(tdc2dev(tdc), "failed to get interrupt for %s\n",
+ dma_chan_name(dc));
+ return ret;
+ }
+
+ ret = pm_runtime_get_sync(tdc2dev(tdc));
+ if (ret < 0) {
+ free_irq(tdc->irq, tdc);
+ return ret;
+ }
+
+ dma_cookie_init(&tdc->vc.chan);
+
+ return 0;
+}
+
+static void tegra_adma_free_chan_resources(struct dma_chan *dc)
+{
+ struct tegra_adma_chan *tdc = to_tegra_adma_chan(dc);
+
+ tegra_adma_terminate_all(dc);
+ vchan_free_chan_resources(&tdc->vc);
+ tasklet_kill(&tdc->vc.task);
+ free_irq(tdc->irq, tdc);
+ pm_runtime_put(tdc2dev(tdc));
+
+ tdc->sreq_index = 0;
+ tdc->sreq_dir = DMA_TRANS_NONE;
+}
+
+static struct dma_chan *tegra_dma_of_xlate(struct of_phandle_args *dma_spec,
+ struct of_dma *ofdma)
+{
+ struct tegra_adma *tdma = ofdma->of_dma_data;
+ struct tegra_adma_chan *tdc;
+ struct dma_chan *chan;
+ unsigned int sreq_index;
+
+ if (dma_spec->args_count != 1)
+ return NULL;
+
+ sreq_index = dma_spec->args[0];
+
+ if (sreq_index == 0) {
+ dev_err(tdma->dev, "DMA request must not be 0\n");
+ return NULL;
+ }
+
+ chan = dma_get_any_slave_channel(&tdma->dma_dev);
+ if (!chan)
+ return NULL;
+
+ tdc = to_tegra_adma_chan(chan);
+ tdc->sreq_index = sreq_index;
+
+ return chan;
+}
+
+static int tegra_adma_runtime_suspend(struct device *dev)
+{
+ struct tegra_adma *tdma = dev_get_drvdata(dev);
+
+ tdma->global_cmd = tdma_read(tdma, ADMA_GLOBAL_CMD);
+
+ return pm_clk_suspend(dev);
+}
+
+static int tegra_adma_runtime_resume(struct device *dev)
+{
+ struct tegra_adma *tdma = dev_get_drvdata(dev);
+ int ret;
+
+ ret = pm_clk_resume(dev);
+ if (ret)
+ return ret;
+
+ tdma_write(tdma, ADMA_GLOBAL_CMD, tdma->global_cmd);
+
+ return 0;
+}
+
+static const struct tegra_adma_chip_data tegra210_chip_data = {
+ .nr_channels = 22,
+};
+
+static const struct of_device_id tegra_adma_of_match[] = {
+ { .compatible = "nvidia,tegra210-adma", .data = &tegra210_chip_data },
+ { },
+};
+MODULE_DEVICE_TABLE(of, tegra_adma_of_match);
+
+static int tegra_adma_probe(struct platform_device *pdev)
+{
+ const struct tegra_adma_chip_data *cdata;
+ struct tegra_adma *tdma;
+ struct resource *res;
+ struct clk *clk;
+ int ret, i;
+
+ cdata = of_device_get_match_data(&pdev->dev);
+ if (!cdata) {
+ dev_err(&pdev->dev, "device match data not found\n");
+ return -ENODEV;
+ }
+
+ tdma = devm_kzalloc(&pdev->dev, sizeof(*tdma) + cdata->nr_channels *
+ sizeof(struct tegra_adma_chan), GFP_KERNEL);
+ if (!tdma)
+ return -ENOMEM;
+
+ tdma->dev = &pdev->dev;
+ tdma->nr_channels = cdata->nr_channels;
+ platform_set_drvdata(pdev, tdma);
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ tdma->base_addr = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(tdma->base_addr))
+ return PTR_ERR(tdma->base_addr);
+
+ ret = pm_clk_create(&pdev->dev);
+ if (ret)
+ return ret;
+
+ clk = clk_get(&pdev->dev, "d_audio");
+ if (IS_ERR(clk)) {
+ dev_err(&pdev->dev, "ADMA clock not found\n");
+ ret = PTR_ERR(clk);
+ goto clk_destroy;
+ }
+
+ ret = pm_clk_add_clk(&pdev->dev, clk);
+ if (ret) {
+ clk_put(clk);
+ goto clk_destroy;
+ }
+
+ pm_runtime_enable(&pdev->dev);
+
+ ret = pm_runtime_get_sync(&pdev->dev);
+ if (ret < 0)
+ goto rpm_disable;
+
+ ret = tegra_adma_init(tdma);
+ if (ret)
+ goto rpm_put;
+
+ INIT_LIST_HEAD(&tdma->dma_dev.channels);
+ for (i = 0; i < tdma->nr_channels; i++) {
+ struct tegra_adma_chan *tdc = &tdma->channels[i];
+
+ tdc->chan_addr = tdma->base_addr + ADMA_CH_REG_OFFSET(i);
+
+ tdc->irq = of_irq_get(pdev->dev.of_node, i);
+ if (tdc->irq < 0) {
+ ret = tdc->irq;
+ goto irq_dispose;
+ }
+
+ vchan_init(&tdc->vc, &tdma->dma_dev);
+ tdc->vc.desc_free = tegra_adma_desc_free;
+ tdc->tdma = tdma;
+ }
+
+ dma_cap_set(DMA_SLAVE, tdma->dma_dev.cap_mask);
+ dma_cap_set(DMA_PRIVATE, tdma->dma_dev.cap_mask);
+ dma_cap_set(DMA_CYCLIC, tdma->dma_dev.cap_mask);
+
+ tdma->dma_dev.dev = &pdev->dev;
+ tdma->dma_dev.device_alloc_chan_resources =
+ tegra_adma_alloc_chan_resources;
+ tdma->dma_dev.device_free_chan_resources =
+ tegra_adma_free_chan_resources;
+ tdma->dma_dev.device_issue_pending = tegra_adma_issue_pending;
+ tdma->dma_dev.device_prep_dma_cyclic = tegra_adma_prep_dma_cyclic;
+ tdma->dma_dev.device_config = tegra_adma_slave_config;
+ tdma->dma_dev.device_tx_status = tegra_adma_tx_status;
+ tdma->dma_dev.device_terminate_all = tegra_adma_terminate_all;
+ tdma->dma_dev.src_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_4_BYTES);
+ tdma->dma_dev.dst_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_4_BYTES);
+ tdma->dma_dev.directions = BIT(DMA_DEV_TO_MEM) | BIT(DMA_MEM_TO_DEV);
+ tdma->dma_dev.residue_granularity = DMA_RESIDUE_GRANULARITY_SEGMENT;
+
+ ret = dma_async_device_register(&tdma->dma_dev);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "ADMA registration failed: %d\n", ret);
+ goto irq_dispose;
+ }
+
+ ret = of_dma_controller_register(pdev->dev.of_node,
+ tegra_dma_of_xlate, tdma);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "ADMA OF registration failed %d\n", ret);
+ goto dma_remove;
+ }
+
+ pm_runtime_put(&pdev->dev);
+
+ dev_info(&pdev->dev, "Tegra210 ADMA driver registered %d channels\n",
+ tdma->nr_channels);
+
+ return 0;
+
+dma_remove:
+ dma_async_device_unregister(&tdma->dma_dev);
+irq_dispose:
+ while (--i >= 0)
+ irq_dispose_mapping(tdma->channels[i].irq);
+rpm_put:
+ pm_runtime_put_sync(&pdev->dev);
+rpm_disable:
+ pm_runtime_disable(&pdev->dev);
+clk_destroy:
+ pm_clk_destroy(&pdev->dev);
+
+ return ret;
+}
+
+static int tegra_adma_remove(struct platform_device *pdev)
+{
+ struct tegra_adma *tdma = platform_get_drvdata(pdev);
+ int i;
+
+ dma_async_device_unregister(&tdma->dma_dev);
+
+ for (i = 0; i < tdma->nr_channels; ++i)
+ irq_dispose_mapping(tdma->channels[i].irq);
+
+ pm_runtime_put_sync(&pdev->dev);
+ pm_runtime_disable(&pdev->dev);
+ pm_clk_destroy(&pdev->dev);
+
+ return 0;
+}
+
+#ifdef CONFIG_PM_SLEEP
+static int tegra_adma_pm_suspend(struct device *dev)
+{
+ return pm_runtime_suspended(dev) == false;
+}
+#endif
+
+static const struct dev_pm_ops tegra_adma_dev_pm_ops = {
+ SET_RUNTIME_PM_OPS(tegra_adma_runtime_suspend,
+ tegra_adma_runtime_resume, NULL)
+ SET_SYSTEM_SLEEP_PM_OPS(tegra_adma_pm_suspend, NULL)
+};
+
+static struct platform_driver tegra_admac_driver = {
+ .driver = {
+ .name = "tegra-adma",
+ .pm = &tegra_adma_dev_pm_ops,
+ .of_match_table = tegra_adma_of_match,
+ },
+ .probe = tegra_adma_probe,
+ .remove = tegra_adma_remove,
+};
+
+module_platform_driver(tegra_admac_driver);
+
+MODULE_ALIAS("platform:tegra210-adma");
+MODULE_DESCRIPTION("NVIDIA Tegra ADMA driver");
+MODULE_AUTHOR("Dara Ramesh <dramesh@nvidia.com>");
+MODULE_AUTHOR("Jon Hunter <jonathanh@nvidia.com>");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/dma/xilinx/xilinx_vdma.c b/drivers/dma/xilinx/xilinx_vdma.c
index ef67f27..df91185 100644
--- a/drivers/dma/xilinx/xilinx_vdma.c
+++ b/drivers/dma/xilinx/xilinx_vdma.c
@@ -16,6 +16,15 @@
* video device (S2MM). Initialization, status, interrupt and management
* registers are accessed through an AXI4-Lite slave interface.
*
+ * The AXI Direct Memory Access (AXI DMA) core is a soft Xilinx IP core that
+ * provides high-bandwidth one dimensional direct memory access between memory
+ * and AXI4-Stream target peripherals. It supports one receive and one
+ * transmit channel, both of them optional at synthesis time.
+ *
+ * The AXI CDMA, is a soft IP, which provides high-bandwidth Direct Memory
+ * Access (DMA) between a memory-mapped source address and a memory-mapped
+ * destination address.
+ *
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 2 of the License, or
@@ -35,116 +44,138 @@
#include <linux/of_platform.h>
#include <linux/of_irq.h>
#include <linux/slab.h>
+#include <linux/clk.h>
#include "../dmaengine.h"
/* Register/Descriptor Offsets */
-#define XILINX_VDMA_MM2S_CTRL_OFFSET 0x0000
-#define XILINX_VDMA_S2MM_CTRL_OFFSET 0x0030
+#define XILINX_DMA_MM2S_CTRL_OFFSET 0x0000
+#define XILINX_DMA_S2MM_CTRL_OFFSET 0x0030
#define XILINX_VDMA_MM2S_DESC_OFFSET 0x0050
#define XILINX_VDMA_S2MM_DESC_OFFSET 0x00a0
/* Control Registers */
-#define XILINX_VDMA_REG_DMACR 0x0000
-#define XILINX_VDMA_DMACR_DELAY_MAX 0xff
-#define XILINX_VDMA_DMACR_DELAY_SHIFT 24
-#define XILINX_VDMA_DMACR_FRAME_COUNT_MAX 0xff
-#define XILINX_VDMA_DMACR_FRAME_COUNT_SHIFT 16
-#define XILINX_VDMA_DMACR_ERR_IRQ BIT(14)
-#define XILINX_VDMA_DMACR_DLY_CNT_IRQ BIT(13)
-#define XILINX_VDMA_DMACR_FRM_CNT_IRQ BIT(12)
-#define XILINX_VDMA_DMACR_MASTER_SHIFT 8
-#define XILINX_VDMA_DMACR_FSYNCSRC_SHIFT 5
-#define XILINX_VDMA_DMACR_FRAMECNT_EN BIT(4)
-#define XILINX_VDMA_DMACR_GENLOCK_EN BIT(3)
-#define XILINX_VDMA_DMACR_RESET BIT(2)
-#define XILINX_VDMA_DMACR_CIRC_EN BIT(1)
-#define XILINX_VDMA_DMACR_RUNSTOP BIT(0)
-#define XILINX_VDMA_DMACR_FSYNCSRC_MASK GENMASK(6, 5)
+#define XILINX_DMA_REG_DMACR 0x0000
+#define XILINX_DMA_DMACR_DELAY_MAX 0xff
+#define XILINX_DMA_DMACR_DELAY_SHIFT 24
+#define XILINX_DMA_DMACR_FRAME_COUNT_MAX 0xff
+#define XILINX_DMA_DMACR_FRAME_COUNT_SHIFT 16
+#define XILINX_DMA_DMACR_ERR_IRQ BIT(14)
+#define XILINX_DMA_DMACR_DLY_CNT_IRQ BIT(13)
+#define XILINX_DMA_DMACR_FRM_CNT_IRQ BIT(12)
+#define XILINX_DMA_DMACR_MASTER_SHIFT 8
+#define XILINX_DMA_DMACR_FSYNCSRC_SHIFT 5
+#define XILINX_DMA_DMACR_FRAMECNT_EN BIT(4)
+#define XILINX_DMA_DMACR_GENLOCK_EN BIT(3)
+#define XILINX_DMA_DMACR_RESET BIT(2)
+#define XILINX_DMA_DMACR_CIRC_EN BIT(1)
+#define XILINX_DMA_DMACR_RUNSTOP BIT(0)
+#define XILINX_DMA_DMACR_FSYNCSRC_MASK GENMASK(6, 5)
-#define XILINX_VDMA_REG_DMASR 0x0004
-#define XILINX_VDMA_DMASR_EOL_LATE_ERR BIT(15)
-#define XILINX_VDMA_DMASR_ERR_IRQ BIT(14)
-#define XILINX_VDMA_DMASR_DLY_CNT_IRQ BIT(13)
-#define XILINX_VDMA_DMASR_FRM_CNT_IRQ BIT(12)
-#define XILINX_VDMA_DMASR_SOF_LATE_ERR BIT(11)
-#define XILINX_VDMA_DMASR_SG_DEC_ERR BIT(10)
-#define XILINX_VDMA_DMASR_SG_SLV_ERR BIT(9)
-#define XILINX_VDMA_DMASR_EOF_EARLY_ERR BIT(8)
-#define XILINX_VDMA_DMASR_SOF_EARLY_ERR BIT(7)
-#define XILINX_VDMA_DMASR_DMA_DEC_ERR BIT(6)
-#define XILINX_VDMA_DMASR_DMA_SLAVE_ERR BIT(5)
-#define XILINX_VDMA_DMASR_DMA_INT_ERR BIT(4)
-#define XILINX_VDMA_DMASR_IDLE BIT(1)
-#define XILINX_VDMA_DMASR_HALTED BIT(0)
-#define XILINX_VDMA_DMASR_DELAY_MASK GENMASK(31, 24)
-#define XILINX_VDMA_DMASR_FRAME_COUNT_MASK GENMASK(23, 16)
+#define XILINX_DMA_REG_DMASR 0x0004
+#define XILINX_DMA_DMASR_EOL_LATE_ERR BIT(15)
+#define XILINX_DMA_DMASR_ERR_IRQ BIT(14)
+#define XILINX_DMA_DMASR_DLY_CNT_IRQ BIT(13)
+#define XILINX_DMA_DMASR_FRM_CNT_IRQ BIT(12)
+#define XILINX_DMA_DMASR_SOF_LATE_ERR BIT(11)
+#define XILINX_DMA_DMASR_SG_DEC_ERR BIT(10)
+#define XILINX_DMA_DMASR_SG_SLV_ERR BIT(9)
+#define XILINX_DMA_DMASR_EOF_EARLY_ERR BIT(8)
+#define XILINX_DMA_DMASR_SOF_EARLY_ERR BIT(7)
+#define XILINX_DMA_DMASR_DMA_DEC_ERR BIT(6)
+#define XILINX_DMA_DMASR_DMA_SLAVE_ERR BIT(5)
+#define XILINX_DMA_DMASR_DMA_INT_ERR BIT(4)
+#define XILINX_DMA_DMASR_IDLE BIT(1)
+#define XILINX_DMA_DMASR_HALTED BIT(0)
+#define XILINX_DMA_DMASR_DELAY_MASK GENMASK(31, 24)
+#define XILINX_DMA_DMASR_FRAME_COUNT_MASK GENMASK(23, 16)
-#define XILINX_VDMA_REG_CURDESC 0x0008
-#define XILINX_VDMA_REG_TAILDESC 0x0010
-#define XILINX_VDMA_REG_REG_INDEX 0x0014
-#define XILINX_VDMA_REG_FRMSTORE 0x0018
-#define XILINX_VDMA_REG_THRESHOLD 0x001c
-#define XILINX_VDMA_REG_FRMPTR_STS 0x0024
-#define XILINX_VDMA_REG_PARK_PTR 0x0028
-#define XILINX_VDMA_PARK_PTR_WR_REF_SHIFT 8
-#define XILINX_VDMA_PARK_PTR_RD_REF_SHIFT 0
-#define XILINX_VDMA_REG_VDMA_VERSION 0x002c
+#define XILINX_DMA_REG_CURDESC 0x0008
+#define XILINX_DMA_REG_TAILDESC 0x0010
+#define XILINX_DMA_REG_REG_INDEX 0x0014
+#define XILINX_DMA_REG_FRMSTORE 0x0018
+#define XILINX_DMA_REG_THRESHOLD 0x001c
+#define XILINX_DMA_REG_FRMPTR_STS 0x0024
+#define XILINX_DMA_REG_PARK_PTR 0x0028
+#define XILINX_DMA_PARK_PTR_WR_REF_SHIFT 8
+#define XILINX_DMA_PARK_PTR_RD_REF_SHIFT 0
+#define XILINX_DMA_REG_VDMA_VERSION 0x002c
/* Register Direct Mode Registers */
-#define XILINX_VDMA_REG_VSIZE 0x0000
-#define XILINX_VDMA_REG_HSIZE 0x0004
+#define XILINX_DMA_REG_VSIZE 0x0000
+#define XILINX_DMA_REG_HSIZE 0x0004
-#define XILINX_VDMA_REG_FRMDLY_STRIDE 0x0008
-#define XILINX_VDMA_FRMDLY_STRIDE_FRMDLY_SHIFT 24
-#define XILINX_VDMA_FRMDLY_STRIDE_STRIDE_SHIFT 0
+#define XILINX_DMA_REG_FRMDLY_STRIDE 0x0008
+#define XILINX_DMA_FRMDLY_STRIDE_FRMDLY_SHIFT 24
+#define XILINX_DMA_FRMDLY_STRIDE_STRIDE_SHIFT 0
#define XILINX_VDMA_REG_START_ADDRESS(n) (0x000c + 4 * (n))
+#define XILINX_VDMA_REG_START_ADDRESS_64(n) (0x000c + 8 * (n))
/* HW specific definitions */
-#define XILINX_VDMA_MAX_CHANS_PER_DEVICE 0x2
+#define XILINX_DMA_MAX_CHANS_PER_DEVICE 0x2
-#define XILINX_VDMA_DMAXR_ALL_IRQ_MASK \
- (XILINX_VDMA_DMASR_FRM_CNT_IRQ | \
- XILINX_VDMA_DMASR_DLY_CNT_IRQ | \
- XILINX_VDMA_DMASR_ERR_IRQ)
+#define XILINX_DMA_DMAXR_ALL_IRQ_MASK \
+ (XILINX_DMA_DMASR_FRM_CNT_IRQ | \
+ XILINX_DMA_DMASR_DLY_CNT_IRQ | \
+ XILINX_DMA_DMASR_ERR_IRQ)
-#define XILINX_VDMA_DMASR_ALL_ERR_MASK \
- (XILINX_VDMA_DMASR_EOL_LATE_ERR | \
- XILINX_VDMA_DMASR_SOF_LATE_ERR | \
- XILINX_VDMA_DMASR_SG_DEC_ERR | \
- XILINX_VDMA_DMASR_SG_SLV_ERR | \
- XILINX_VDMA_DMASR_EOF_EARLY_ERR | \
- XILINX_VDMA_DMASR_SOF_EARLY_ERR | \
- XILINX_VDMA_DMASR_DMA_DEC_ERR | \
- XILINX_VDMA_DMASR_DMA_SLAVE_ERR | \
- XILINX_VDMA_DMASR_DMA_INT_ERR)
+#define XILINX_DMA_DMASR_ALL_ERR_MASK \
+ (XILINX_DMA_DMASR_EOL_LATE_ERR | \
+ XILINX_DMA_DMASR_SOF_LATE_ERR | \
+ XILINX_DMA_DMASR_SG_DEC_ERR | \
+ XILINX_DMA_DMASR_SG_SLV_ERR | \
+ XILINX_DMA_DMASR_EOF_EARLY_ERR | \
+ XILINX_DMA_DMASR_SOF_EARLY_ERR | \
+ XILINX_DMA_DMASR_DMA_DEC_ERR | \
+ XILINX_DMA_DMASR_DMA_SLAVE_ERR | \
+ XILINX_DMA_DMASR_DMA_INT_ERR)
/*
* Recoverable errors are DMA Internal error, SOF Early, EOF Early
* and SOF Late. They are only recoverable when C_FLUSH_ON_FSYNC
* is enabled in the h/w system.
*/
-#define XILINX_VDMA_DMASR_ERR_RECOVER_MASK \
- (XILINX_VDMA_DMASR_SOF_LATE_ERR | \
- XILINX_VDMA_DMASR_EOF_EARLY_ERR | \
- XILINX_VDMA_DMASR_SOF_EARLY_ERR | \
- XILINX_VDMA_DMASR_DMA_INT_ERR)
+#define XILINX_DMA_DMASR_ERR_RECOVER_MASK \
+ (XILINX_DMA_DMASR_SOF_LATE_ERR | \
+ XILINX_DMA_DMASR_EOF_EARLY_ERR | \
+ XILINX_DMA_DMASR_SOF_EARLY_ERR | \
+ XILINX_DMA_DMASR_DMA_INT_ERR)
/* Axi VDMA Flush on Fsync bits */
-#define XILINX_VDMA_FLUSH_S2MM 3
-#define XILINX_VDMA_FLUSH_MM2S 2
-#define XILINX_VDMA_FLUSH_BOTH 1
+#define XILINX_DMA_FLUSH_S2MM 3
+#define XILINX_DMA_FLUSH_MM2S 2
+#define XILINX_DMA_FLUSH_BOTH 1
/* Delay loop counter to prevent hardware failure */
-#define XILINX_VDMA_LOOP_COUNT 1000000
+#define XILINX_DMA_LOOP_COUNT 1000000
+
+/* AXI DMA Specific Registers/Offsets */
+#define XILINX_DMA_REG_SRCDSTADDR 0x18
+#define XILINX_DMA_REG_BTT 0x28
+
+/* AXI DMA Specific Masks/Bit fields */
+#define XILINX_DMA_MAX_TRANS_LEN GENMASK(22, 0)
+#define XILINX_DMA_CR_COALESCE_MAX GENMASK(23, 16)
+#define XILINX_DMA_CR_COALESCE_SHIFT 16
+#define XILINX_DMA_BD_SOP BIT(27)
+#define XILINX_DMA_BD_EOP BIT(26)
+#define XILINX_DMA_COALESCE_MAX 255
+#define XILINX_DMA_NUM_APP_WORDS 5
+
+/* AXI CDMA Specific Registers/Offsets */
+#define XILINX_CDMA_REG_SRCADDR 0x18
+#define XILINX_CDMA_REG_DSTADDR 0x20
+
+/* AXI CDMA Specific Masks */
+#define XILINX_CDMA_CR_SGMODE BIT(3)
/**
* struct xilinx_vdma_desc_hw - Hardware Descriptor
* @next_desc: Next Descriptor Pointer @0x00
* @pad1: Reserved @0x04
* @buf_addr: Buffer address @0x08
- * @pad2: Reserved @0x0C
+ * @buf_addr_msb: MSB of Buffer address @0x0C
* @vsize: Vertical Size @0x10
* @hsize: Horizontal Size @0x14
* @stride: Number of bytes between the first
@@ -154,13 +185,59 @@
u32 next_desc;
u32 pad1;
u32 buf_addr;
- u32 pad2;
+ u32 buf_addr_msb;
u32 vsize;
u32 hsize;
u32 stride;
} __aligned(64);
/**
+ * struct xilinx_axidma_desc_hw - Hardware Descriptor for AXI DMA
+ * @next_desc: Next Descriptor Pointer @0x00
+ * @pad1: Reserved @0x04
+ * @buf_addr: Buffer address @0x08
+ * @pad2: Reserved @0x0C
+ * @pad3: Reserved @0x10
+ * @pad4: Reserved @0x14
+ * @control: Control field @0x18
+ * @status: Status field @0x1C
+ * @app: APP Fields @0x20 - 0x30
+ */
+struct xilinx_axidma_desc_hw {
+ u32 next_desc;
+ u32 pad1;
+ u32 buf_addr;
+ u32 pad2;
+ u32 pad3;
+ u32 pad4;
+ u32 control;
+ u32 status;
+ u32 app[XILINX_DMA_NUM_APP_WORDS];
+} __aligned(64);
+
+/**
+ * struct xilinx_cdma_desc_hw - Hardware Descriptor
+ * @next_desc: Next Descriptor Pointer @0x00
+ * @pad1: Reserved @0x04
+ * @src_addr: Source address @0x08
+ * @pad2: Reserved @0x0C
+ * @dest_addr: Destination address @0x10
+ * @pad3: Reserved @0x14
+ * @control: Control field @0x18
+ * @status: Status field @0x1C
+ */
+struct xilinx_cdma_desc_hw {
+ u32 next_desc;
+ u32 pad1;
+ u32 src_addr;
+ u32 pad2;
+ u32 dest_addr;
+ u32 pad3;
+ u32 control;
+ u32 status;
+} __aligned(64);
+
+/**
* struct xilinx_vdma_tx_segment - Descriptor segment
* @hw: Hardware descriptor
* @node: Node in the descriptor segments list
@@ -173,19 +250,43 @@
} __aligned(64);
/**
- * struct xilinx_vdma_tx_descriptor - Per Transaction structure
+ * struct xilinx_axidma_tx_segment - Descriptor segment
+ * @hw: Hardware descriptor
+ * @node: Node in the descriptor segments list
+ * @phys: Physical address of segment
+ */
+struct xilinx_axidma_tx_segment {
+ struct xilinx_axidma_desc_hw hw;
+ struct list_head node;
+ dma_addr_t phys;
+} __aligned(64);
+
+/**
+ * struct xilinx_cdma_tx_segment - Descriptor segment
+ * @hw: Hardware descriptor
+ * @node: Node in the descriptor segments list
+ * @phys: Physical address of segment
+ */
+struct xilinx_cdma_tx_segment {
+ struct xilinx_cdma_desc_hw hw;
+ struct list_head node;
+ dma_addr_t phys;
+} __aligned(64);
+
+/**
+ * struct xilinx_dma_tx_descriptor - Per Transaction structure
* @async_tx: Async transaction descriptor
* @segments: TX segments list
* @node: Node in the channel descriptors list
*/
-struct xilinx_vdma_tx_descriptor {
+struct xilinx_dma_tx_descriptor {
struct dma_async_tx_descriptor async_tx;
struct list_head segments;
struct list_head node;
};
/**
- * struct xilinx_vdma_chan - Driver specific VDMA channel structure
+ * struct xilinx_dma_chan - Driver specific DMA channel structure
* @xdev: Driver specific device structure
* @ctrl_offset: Control registers offset
* @desc_offset: TX descriptor registers offset
@@ -207,9 +308,14 @@
* @config: Device configuration info
* @flush_on_fsync: Flush on Frame sync
* @desc_pendingcount: Descriptor pending count
+ * @ext_addr: Indicates 64 bit addressing is supported by dma channel
+ * @desc_submitcount: Descriptor h/w submitted count
+ * @residue: Residue for AXI DMA
+ * @seg_v: Statically allocated segments base
+ * @start_transfer: Differentiate b/w DMA IP's transfer
*/
-struct xilinx_vdma_chan {
- struct xilinx_vdma_device *xdev;
+struct xilinx_dma_chan {
+ struct xilinx_dma_device *xdev;
u32 ctrl_offset;
u32 desc_offset;
spinlock_t lock;
@@ -230,73 +336,122 @@
struct xilinx_vdma_config config;
bool flush_on_fsync;
u32 desc_pendingcount;
+ bool ext_addr;
+ u32 desc_submitcount;
+ u32 residue;
+ struct xilinx_axidma_tx_segment *seg_v;
+ void (*start_transfer)(struct xilinx_dma_chan *chan);
+};
+
+struct xilinx_dma_config {
+ enum xdma_ip_type dmatype;
+ int (*clk_init)(struct platform_device *pdev, struct clk **axi_clk,
+ struct clk **tx_clk, struct clk **txs_clk,
+ struct clk **rx_clk, struct clk **rxs_clk);
};
/**
- * struct xilinx_vdma_device - VDMA device structure
+ * struct xilinx_dma_device - DMA device structure
* @regs: I/O mapped base address
* @dev: Device Structure
* @common: DMA device structure
- * @chan: Driver specific VDMA channel
+ * @chan: Driver specific DMA channel
* @has_sg: Specifies whether Scatter-Gather is present or not
* @flush_on_fsync: Flush on frame sync
+ * @ext_addr: Indicates 64 bit addressing is supported by dma device
+ * @pdev: Platform device structure pointer
+ * @dma_config: DMA config structure
+ * @axi_clk: DMA Axi4-lite interace clock
+ * @tx_clk: DMA mm2s clock
+ * @txs_clk: DMA mm2s stream clock
+ * @rx_clk: DMA s2mm clock
+ * @rxs_clk: DMA s2mm stream clock
*/
-struct xilinx_vdma_device {
+struct xilinx_dma_device {
void __iomem *regs;
struct device *dev;
struct dma_device common;
- struct xilinx_vdma_chan *chan[XILINX_VDMA_MAX_CHANS_PER_DEVICE];
+ struct xilinx_dma_chan *chan[XILINX_DMA_MAX_CHANS_PER_DEVICE];
bool has_sg;
u32 flush_on_fsync;
+ bool ext_addr;
+ struct platform_device *pdev;
+ const struct xilinx_dma_config *dma_config;
+ struct clk *axi_clk;
+ struct clk *tx_clk;
+ struct clk *txs_clk;
+ struct clk *rx_clk;
+ struct clk *rxs_clk;
};
/* Macros */
#define to_xilinx_chan(chan) \
- container_of(chan, struct xilinx_vdma_chan, common)
-#define to_vdma_tx_descriptor(tx) \
- container_of(tx, struct xilinx_vdma_tx_descriptor, async_tx)
-#define xilinx_vdma_poll_timeout(chan, reg, val, cond, delay_us, timeout_us) \
+ container_of(chan, struct xilinx_dma_chan, common)
+#define to_dma_tx_descriptor(tx) \
+ container_of(tx, struct xilinx_dma_tx_descriptor, async_tx)
+#define xilinx_dma_poll_timeout(chan, reg, val, cond, delay_us, timeout_us) \
readl_poll_timeout(chan->xdev->regs + chan->ctrl_offset + reg, val, \
cond, delay_us, timeout_us)
/* IO accessors */
-static inline u32 vdma_read(struct xilinx_vdma_chan *chan, u32 reg)
+static inline u32 dma_read(struct xilinx_dma_chan *chan, u32 reg)
{
return ioread32(chan->xdev->regs + reg);
}
-static inline void vdma_write(struct xilinx_vdma_chan *chan, u32 reg, u32 value)
+static inline void dma_write(struct xilinx_dma_chan *chan, u32 reg, u32 value)
{
iowrite32(value, chan->xdev->regs + reg);
}
-static inline void vdma_desc_write(struct xilinx_vdma_chan *chan, u32 reg,
+static inline void vdma_desc_write(struct xilinx_dma_chan *chan, u32 reg,
u32 value)
{
- vdma_write(chan, chan->desc_offset + reg, value);
+ dma_write(chan, chan->desc_offset + reg, value);
}
-static inline u32 vdma_ctrl_read(struct xilinx_vdma_chan *chan, u32 reg)
+static inline u32 dma_ctrl_read(struct xilinx_dma_chan *chan, u32 reg)
{
- return vdma_read(chan, chan->ctrl_offset + reg);
+ return dma_read(chan, chan->ctrl_offset + reg);
}
-static inline void vdma_ctrl_write(struct xilinx_vdma_chan *chan, u32 reg,
+static inline void dma_ctrl_write(struct xilinx_dma_chan *chan, u32 reg,
u32 value)
{
- vdma_write(chan, chan->ctrl_offset + reg, value);
+ dma_write(chan, chan->ctrl_offset + reg, value);
}
-static inline void vdma_ctrl_clr(struct xilinx_vdma_chan *chan, u32 reg,
+static inline void dma_ctrl_clr(struct xilinx_dma_chan *chan, u32 reg,
u32 clr)
{
- vdma_ctrl_write(chan, reg, vdma_ctrl_read(chan, reg) & ~clr);
+ dma_ctrl_write(chan, reg, dma_ctrl_read(chan, reg) & ~clr);
}
-static inline void vdma_ctrl_set(struct xilinx_vdma_chan *chan, u32 reg,
+static inline void dma_ctrl_set(struct xilinx_dma_chan *chan, u32 reg,
u32 set)
{
- vdma_ctrl_write(chan, reg, vdma_ctrl_read(chan, reg) | set);
+ dma_ctrl_write(chan, reg, dma_ctrl_read(chan, reg) | set);
+}
+
+/**
+ * vdma_desc_write_64 - 64-bit descriptor write
+ * @chan: Driver specific VDMA channel
+ * @reg: Register to write
+ * @value_lsb: lower address of the descriptor.
+ * @value_msb: upper address of the descriptor.
+ *
+ * Since vdma driver is trying to write to a register offset which is not a
+ * multiple of 64 bits(ex : 0x5c), we are writing as two separate 32 bits
+ * instead of a single 64 bit register write.
+ */
+static inline void vdma_desc_write_64(struct xilinx_dma_chan *chan, u32 reg,
+ u32 value_lsb, u32 value_msb)
+{
+ /* Write the lsb 32 bits*/
+ writel(value_lsb, chan->xdev->regs + chan->desc_offset + reg);
+
+ /* Write the msb 32 bits */
+ writel(value_msb, chan->xdev->regs + chan->desc_offset + reg + 4);
}
/* -----------------------------------------------------------------------------
@@ -305,16 +460,37 @@
/**
* xilinx_vdma_alloc_tx_segment - Allocate transaction segment
- * @chan: Driver specific VDMA channel
+ * @chan: Driver specific DMA channel
*
* Return: The allocated segment on success and NULL on failure.
*/
static struct xilinx_vdma_tx_segment *
-xilinx_vdma_alloc_tx_segment(struct xilinx_vdma_chan *chan)
+xilinx_vdma_alloc_tx_segment(struct xilinx_dma_chan *chan)
{
struct xilinx_vdma_tx_segment *segment;
dma_addr_t phys;
+ segment = dma_pool_zalloc(chan->desc_pool, GFP_ATOMIC, &phys);
+ if (!segment)
+ return NULL;
+
+ segment->phys = phys;
+
+ return segment;
+}
+
+/**
+ * xilinx_cdma_alloc_tx_segment - Allocate transaction segment
+ * @chan: Driver specific DMA channel
+ *
+ * Return: The allocated segment on success and NULL on failure.
+ */
+static struct xilinx_cdma_tx_segment *
+xilinx_cdma_alloc_tx_segment(struct xilinx_dma_chan *chan)
+{
+ struct xilinx_cdma_tx_segment *segment;
+ dma_addr_t phys;
+
segment = dma_pool_alloc(chan->desc_pool, GFP_ATOMIC, &phys);
if (!segment)
return NULL;
@@ -326,26 +502,70 @@
}
/**
- * xilinx_vdma_free_tx_segment - Free transaction segment
- * @chan: Driver specific VDMA channel
- * @segment: VDMA transaction segment
+ * xilinx_axidma_alloc_tx_segment - Allocate transaction segment
+ * @chan: Driver specific DMA channel
+ *
+ * Return: The allocated segment on success and NULL on failure.
*/
-static void xilinx_vdma_free_tx_segment(struct xilinx_vdma_chan *chan,
+static struct xilinx_axidma_tx_segment *
+xilinx_axidma_alloc_tx_segment(struct xilinx_dma_chan *chan)
+{
+ struct xilinx_axidma_tx_segment *segment;
+ dma_addr_t phys;
+
+ segment = dma_pool_alloc(chan->desc_pool, GFP_ATOMIC, &phys);
+ if (!segment)
+ return NULL;
+
+ memset(segment, 0, sizeof(*segment));
+ segment->phys = phys;
+
+ return segment;
+}
+
+/**
+ * xilinx_dma_free_tx_segment - Free transaction segment
+ * @chan: Driver specific DMA channel
+ * @segment: DMA transaction segment
+ */
+static void xilinx_dma_free_tx_segment(struct xilinx_dma_chan *chan,
+ struct xilinx_axidma_tx_segment *segment)
+{
+ dma_pool_free(chan->desc_pool, segment, segment->phys);
+}
+
+/**
+ * xilinx_cdma_free_tx_segment - Free transaction segment
+ * @chan: Driver specific DMA channel
+ * @segment: DMA transaction segment
+ */
+static void xilinx_cdma_free_tx_segment(struct xilinx_dma_chan *chan,
+ struct xilinx_cdma_tx_segment *segment)
+{
+ dma_pool_free(chan->desc_pool, segment, segment->phys);
+}
+
+/**
+ * xilinx_vdma_free_tx_segment - Free transaction segment
+ * @chan: Driver specific DMA channel
+ * @segment: DMA transaction segment
+ */
+static void xilinx_vdma_free_tx_segment(struct xilinx_dma_chan *chan,
struct xilinx_vdma_tx_segment *segment)
{
dma_pool_free(chan->desc_pool, segment, segment->phys);
}
/**
- * xilinx_vdma_tx_descriptor - Allocate transaction descriptor
- * @chan: Driver specific VDMA channel
+ * xilinx_dma_tx_descriptor - Allocate transaction descriptor
+ * @chan: Driver specific DMA channel
*
* Return: The allocated descriptor on success and NULL on failure.
*/
-static struct xilinx_vdma_tx_descriptor *
-xilinx_vdma_alloc_tx_descriptor(struct xilinx_vdma_chan *chan)
+static struct xilinx_dma_tx_descriptor *
+xilinx_dma_alloc_tx_descriptor(struct xilinx_dma_chan *chan)
{
- struct xilinx_vdma_tx_descriptor *desc;
+ struct xilinx_dma_tx_descriptor *desc;
desc = kzalloc(sizeof(*desc), GFP_KERNEL);
if (!desc)
@@ -357,22 +577,38 @@
}
/**
- * xilinx_vdma_free_tx_descriptor - Free transaction descriptor
- * @chan: Driver specific VDMA channel
- * @desc: VDMA transaction descriptor
+ * xilinx_dma_free_tx_descriptor - Free transaction descriptor
+ * @chan: Driver specific DMA channel
+ * @desc: DMA transaction descriptor
*/
static void
-xilinx_vdma_free_tx_descriptor(struct xilinx_vdma_chan *chan,
- struct xilinx_vdma_tx_descriptor *desc)
+xilinx_dma_free_tx_descriptor(struct xilinx_dma_chan *chan,
+ struct xilinx_dma_tx_descriptor *desc)
{
struct xilinx_vdma_tx_segment *segment, *next;
+ struct xilinx_cdma_tx_segment *cdma_segment, *cdma_next;
+ struct xilinx_axidma_tx_segment *axidma_segment, *axidma_next;
if (!desc)
return;
- list_for_each_entry_safe(segment, next, &desc->segments, node) {
- list_del(&segment->node);
- xilinx_vdma_free_tx_segment(chan, segment);
+ if (chan->xdev->dma_config->dmatype == XDMA_TYPE_VDMA) {
+ list_for_each_entry_safe(segment, next, &desc->segments, node) {
+ list_del(&segment->node);
+ xilinx_vdma_free_tx_segment(chan, segment);
+ }
+ } else if (chan->xdev->dma_config->dmatype == XDMA_TYPE_CDMA) {
+ list_for_each_entry_safe(cdma_segment, cdma_next,
+ &desc->segments, node) {
+ list_del(&cdma_segment->node);
+ xilinx_cdma_free_tx_segment(chan, cdma_segment);
+ }
+ } else {
+ list_for_each_entry_safe(axidma_segment, axidma_next,
+ &desc->segments, node) {
+ list_del(&axidma_segment->node);
+ xilinx_dma_free_tx_segment(chan, axidma_segment);
+ }
}
kfree(desc);
@@ -381,60 +617,62 @@
/* Required functions */
/**
- * xilinx_vdma_free_desc_list - Free descriptors list
- * @chan: Driver specific VDMA channel
+ * xilinx_dma_free_desc_list - Free descriptors list
+ * @chan: Driver specific DMA channel
* @list: List to parse and delete the descriptor
*/
-static void xilinx_vdma_free_desc_list(struct xilinx_vdma_chan *chan,
+static void xilinx_dma_free_desc_list(struct xilinx_dma_chan *chan,
struct list_head *list)
{
- struct xilinx_vdma_tx_descriptor *desc, *next;
+ struct xilinx_dma_tx_descriptor *desc, *next;
list_for_each_entry_safe(desc, next, list, node) {
list_del(&desc->node);
- xilinx_vdma_free_tx_descriptor(chan, desc);
+ xilinx_dma_free_tx_descriptor(chan, desc);
}
}
/**
- * xilinx_vdma_free_descriptors - Free channel descriptors
- * @chan: Driver specific VDMA channel
+ * xilinx_dma_free_descriptors - Free channel descriptors
+ * @chan: Driver specific DMA channel
*/
-static void xilinx_vdma_free_descriptors(struct xilinx_vdma_chan *chan)
+static void xilinx_dma_free_descriptors(struct xilinx_dma_chan *chan)
{
unsigned long flags;
spin_lock_irqsave(&chan->lock, flags);
- xilinx_vdma_free_desc_list(chan, &chan->pending_list);
- xilinx_vdma_free_desc_list(chan, &chan->done_list);
- xilinx_vdma_free_desc_list(chan, &chan->active_list);
+ xilinx_dma_free_desc_list(chan, &chan->pending_list);
+ xilinx_dma_free_desc_list(chan, &chan->done_list);
+ xilinx_dma_free_desc_list(chan, &chan->active_list);
spin_unlock_irqrestore(&chan->lock, flags);
}
/**
- * xilinx_vdma_free_chan_resources - Free channel resources
+ * xilinx_dma_free_chan_resources - Free channel resources
* @dchan: DMA channel
*/
-static void xilinx_vdma_free_chan_resources(struct dma_chan *dchan)
+static void xilinx_dma_free_chan_resources(struct dma_chan *dchan)
{
- struct xilinx_vdma_chan *chan = to_xilinx_chan(dchan);
+ struct xilinx_dma_chan *chan = to_xilinx_chan(dchan);
dev_dbg(chan->dev, "Free all channel resources.\n");
- xilinx_vdma_free_descriptors(chan);
+ xilinx_dma_free_descriptors(chan);
+ if (chan->xdev->dma_config->dmatype == XDMA_TYPE_AXIDMA)
+ xilinx_dma_free_tx_segment(chan, chan->seg_v);
dma_pool_destroy(chan->desc_pool);
chan->desc_pool = NULL;
}
/**
- * xilinx_vdma_chan_desc_cleanup - Clean channel descriptors
- * @chan: Driver specific VDMA channel
+ * xilinx_dma_chan_desc_cleanup - Clean channel descriptors
+ * @chan: Driver specific DMA channel
*/
-static void xilinx_vdma_chan_desc_cleanup(struct xilinx_vdma_chan *chan)
+static void xilinx_dma_chan_desc_cleanup(struct xilinx_dma_chan *chan)
{
- struct xilinx_vdma_tx_descriptor *desc, *next;
+ struct xilinx_dma_tx_descriptor *desc, *next;
unsigned long flags;
spin_lock_irqsave(&chan->lock, flags);
@@ -457,32 +695,32 @@
/* Run any dependencies, then free the descriptor */
dma_run_dependencies(&desc->async_tx);
- xilinx_vdma_free_tx_descriptor(chan, desc);
+ xilinx_dma_free_tx_descriptor(chan, desc);
}
spin_unlock_irqrestore(&chan->lock, flags);
}
/**
- * xilinx_vdma_do_tasklet - Schedule completion tasklet
- * @data: Pointer to the Xilinx VDMA channel structure
+ * xilinx_dma_do_tasklet - Schedule completion tasklet
+ * @data: Pointer to the Xilinx DMA channel structure
*/
-static void xilinx_vdma_do_tasklet(unsigned long data)
+static void xilinx_dma_do_tasklet(unsigned long data)
{
- struct xilinx_vdma_chan *chan = (struct xilinx_vdma_chan *)data;
+ struct xilinx_dma_chan *chan = (struct xilinx_dma_chan *)data;
- xilinx_vdma_chan_desc_cleanup(chan);
+ xilinx_dma_chan_desc_cleanup(chan);
}
/**
- * xilinx_vdma_alloc_chan_resources - Allocate channel resources
+ * xilinx_dma_alloc_chan_resources - Allocate channel resources
* @dchan: DMA channel
*
* Return: '0' on success and failure value on error
*/
-static int xilinx_vdma_alloc_chan_resources(struct dma_chan *dchan)
+static int xilinx_dma_alloc_chan_resources(struct dma_chan *dchan)
{
- struct xilinx_vdma_chan *chan = to_xilinx_chan(dchan);
+ struct xilinx_dma_chan *chan = to_xilinx_chan(dchan);
/* Has this channel already been allocated? */
if (chan->desc_pool)
@@ -492,10 +730,26 @@
* We need the descriptor to be aligned to 64bytes
* for meeting Xilinx VDMA specification requirement.
*/
- chan->desc_pool = dma_pool_create("xilinx_vdma_desc_pool",
- chan->dev,
- sizeof(struct xilinx_vdma_tx_segment),
- __alignof__(struct xilinx_vdma_tx_segment), 0);
+ if (chan->xdev->dma_config->dmatype == XDMA_TYPE_AXIDMA) {
+ chan->desc_pool = dma_pool_create("xilinx_dma_desc_pool",
+ chan->dev,
+ sizeof(struct xilinx_axidma_tx_segment),
+ __alignof__(struct xilinx_axidma_tx_segment),
+ 0);
+ } else if (chan->xdev->dma_config->dmatype == XDMA_TYPE_CDMA) {
+ chan->desc_pool = dma_pool_create("xilinx_cdma_desc_pool",
+ chan->dev,
+ sizeof(struct xilinx_cdma_tx_segment),
+ __alignof__(struct xilinx_cdma_tx_segment),
+ 0);
+ } else {
+ chan->desc_pool = dma_pool_create("xilinx_vdma_desc_pool",
+ chan->dev,
+ sizeof(struct xilinx_vdma_tx_segment),
+ __alignof__(struct xilinx_vdma_tx_segment),
+ 0);
+ }
+
if (!chan->desc_pool) {
dev_err(chan->dev,
"unable to allocate channel %d descriptor pool\n",
@@ -503,110 +757,160 @@
return -ENOMEM;
}
+ if (chan->xdev->dma_config->dmatype == XDMA_TYPE_AXIDMA)
+ /*
+ * For AXI DMA case after submitting a pending_list, keep
+ * an extra segment allocated so that the "next descriptor"
+ * pointer on the tail descriptor always points to a
+ * valid descriptor, even when paused after reaching taildesc.
+ * This way, it is possible to issue additional
+ * transfers without halting and restarting the channel.
+ */
+ chan->seg_v = xilinx_axidma_alloc_tx_segment(chan);
+
dma_cookie_init(dchan);
+
+ if (chan->xdev->dma_config->dmatype == XDMA_TYPE_AXIDMA) {
+ /* For AXI DMA resetting once channel will reset the
+ * other channel as well so enable the interrupts here.
+ */
+ dma_ctrl_set(chan, XILINX_DMA_REG_DMACR,
+ XILINX_DMA_DMAXR_ALL_IRQ_MASK);
+ }
+
+ if ((chan->xdev->dma_config->dmatype == XDMA_TYPE_CDMA) && chan->has_sg)
+ dma_ctrl_set(chan, XILINX_DMA_REG_DMACR,
+ XILINX_CDMA_CR_SGMODE);
+
return 0;
}
/**
- * xilinx_vdma_tx_status - Get VDMA transaction status
+ * xilinx_dma_tx_status - Get DMA transaction status
* @dchan: DMA channel
* @cookie: Transaction identifier
* @txstate: Transaction state
*
* Return: DMA transaction status
*/
-static enum dma_status xilinx_vdma_tx_status(struct dma_chan *dchan,
+static enum dma_status xilinx_dma_tx_status(struct dma_chan *dchan,
dma_cookie_t cookie,
struct dma_tx_state *txstate)
{
- return dma_cookie_status(dchan, cookie, txstate);
+ struct xilinx_dma_chan *chan = to_xilinx_chan(dchan);
+ struct xilinx_dma_tx_descriptor *desc;
+ struct xilinx_axidma_tx_segment *segment;
+ struct xilinx_axidma_desc_hw *hw;
+ enum dma_status ret;
+ unsigned long flags;
+ u32 residue = 0;
+
+ ret = dma_cookie_status(dchan, cookie, txstate);
+ if (ret == DMA_COMPLETE || !txstate)
+ return ret;
+
+ if (chan->xdev->dma_config->dmatype == XDMA_TYPE_AXIDMA) {
+ spin_lock_irqsave(&chan->lock, flags);
+
+ desc = list_last_entry(&chan->active_list,
+ struct xilinx_dma_tx_descriptor, node);
+ if (chan->has_sg) {
+ list_for_each_entry(segment, &desc->segments, node) {
+ hw = &segment->hw;
+ residue += (hw->control - hw->status) &
+ XILINX_DMA_MAX_TRANS_LEN;
+ }
+ }
+ spin_unlock_irqrestore(&chan->lock, flags);
+
+ chan->residue = residue;
+ dma_set_residue(txstate, chan->residue);
+ }
+
+ return ret;
}
/**
- * xilinx_vdma_is_running - Check if VDMA channel is running
- * @chan: Driver specific VDMA channel
+ * xilinx_dma_is_running - Check if DMA channel is running
+ * @chan: Driver specific DMA channel
*
* Return: '1' if running, '0' if not.
*/
-static bool xilinx_vdma_is_running(struct xilinx_vdma_chan *chan)
+static bool xilinx_dma_is_running(struct xilinx_dma_chan *chan)
{
- return !(vdma_ctrl_read(chan, XILINX_VDMA_REG_DMASR) &
- XILINX_VDMA_DMASR_HALTED) &&
- (vdma_ctrl_read(chan, XILINX_VDMA_REG_DMACR) &
- XILINX_VDMA_DMACR_RUNSTOP);
+ return !(dma_ctrl_read(chan, XILINX_DMA_REG_DMASR) &
+ XILINX_DMA_DMASR_HALTED) &&
+ (dma_ctrl_read(chan, XILINX_DMA_REG_DMACR) &
+ XILINX_DMA_DMACR_RUNSTOP);
}
/**
- * xilinx_vdma_is_idle - Check if VDMA channel is idle
- * @chan: Driver specific VDMA channel
+ * xilinx_dma_is_idle - Check if DMA channel is idle
+ * @chan: Driver specific DMA channel
*
* Return: '1' if idle, '0' if not.
*/
-static bool xilinx_vdma_is_idle(struct xilinx_vdma_chan *chan)
+static bool xilinx_dma_is_idle(struct xilinx_dma_chan *chan)
{
- return vdma_ctrl_read(chan, XILINX_VDMA_REG_DMASR) &
- XILINX_VDMA_DMASR_IDLE;
+ return dma_ctrl_read(chan, XILINX_DMA_REG_DMASR) &
+ XILINX_DMA_DMASR_IDLE;
}
/**
- * xilinx_vdma_halt - Halt VDMA channel
- * @chan: Driver specific VDMA channel
+ * xilinx_dma_halt - Halt DMA channel
+ * @chan: Driver specific DMA channel
*/
-static void xilinx_vdma_halt(struct xilinx_vdma_chan *chan)
+static void xilinx_dma_halt(struct xilinx_dma_chan *chan)
{
int err;
u32 val;
- vdma_ctrl_clr(chan, XILINX_VDMA_REG_DMACR, XILINX_VDMA_DMACR_RUNSTOP);
+ dma_ctrl_clr(chan, XILINX_DMA_REG_DMACR, XILINX_DMA_DMACR_RUNSTOP);
/* Wait for the hardware to halt */
- err = xilinx_vdma_poll_timeout(chan, XILINX_VDMA_REG_DMASR, val,
- (val & XILINX_VDMA_DMASR_HALTED), 0,
- XILINX_VDMA_LOOP_COUNT);
+ err = xilinx_dma_poll_timeout(chan, XILINX_DMA_REG_DMASR, val,
+ (val & XILINX_DMA_DMASR_HALTED), 0,
+ XILINX_DMA_LOOP_COUNT);
if (err) {
dev_err(chan->dev, "Cannot stop channel %p: %x\n",
- chan, vdma_ctrl_read(chan, XILINX_VDMA_REG_DMASR));
+ chan, dma_ctrl_read(chan, XILINX_DMA_REG_DMASR));
chan->err = true;
}
-
- return;
}
/**
- * xilinx_vdma_start - Start VDMA channel
- * @chan: Driver specific VDMA channel
+ * xilinx_dma_start - Start DMA channel
+ * @chan: Driver specific DMA channel
*/
-static void xilinx_vdma_start(struct xilinx_vdma_chan *chan)
+static void xilinx_dma_start(struct xilinx_dma_chan *chan)
{
int err;
u32 val;
- vdma_ctrl_set(chan, XILINX_VDMA_REG_DMACR, XILINX_VDMA_DMACR_RUNSTOP);
+ dma_ctrl_set(chan, XILINX_DMA_REG_DMACR, XILINX_DMA_DMACR_RUNSTOP);
/* Wait for the hardware to start */
- err = xilinx_vdma_poll_timeout(chan, XILINX_VDMA_REG_DMASR, val,
- !(val & XILINX_VDMA_DMASR_HALTED), 0,
- XILINX_VDMA_LOOP_COUNT);
+ err = xilinx_dma_poll_timeout(chan, XILINX_DMA_REG_DMASR, val,
+ !(val & XILINX_DMA_DMASR_HALTED), 0,
+ XILINX_DMA_LOOP_COUNT);
if (err) {
dev_err(chan->dev, "Cannot start channel %p: %x\n",
- chan, vdma_ctrl_read(chan, XILINX_VDMA_REG_DMASR));
+ chan, dma_ctrl_read(chan, XILINX_DMA_REG_DMASR));
chan->err = true;
}
-
- return;
}
/**
* xilinx_vdma_start_transfer - Starts VDMA transfer
* @chan: Driver specific channel struct pointer
*/
-static void xilinx_vdma_start_transfer(struct xilinx_vdma_chan *chan)
+static void xilinx_vdma_start_transfer(struct xilinx_dma_chan *chan)
{
struct xilinx_vdma_config *config = &chan->config;
- struct xilinx_vdma_tx_descriptor *desc, *tail_desc;
+ struct xilinx_dma_tx_descriptor *desc, *tail_desc;
u32 reg;
struct xilinx_vdma_tx_segment *tail_segment;
@@ -618,16 +922,16 @@
return;
desc = list_first_entry(&chan->pending_list,
- struct xilinx_vdma_tx_descriptor, node);
+ struct xilinx_dma_tx_descriptor, node);
tail_desc = list_last_entry(&chan->pending_list,
- struct xilinx_vdma_tx_descriptor, node);
+ struct xilinx_dma_tx_descriptor, node);
tail_segment = list_last_entry(&tail_desc->segments,
struct xilinx_vdma_tx_segment, node);
/* If it is SG mode and hardware is busy, cannot submit */
- if (chan->has_sg && xilinx_vdma_is_running(chan) &&
- !xilinx_vdma_is_idle(chan)) {
+ if (chan->has_sg && xilinx_dma_is_running(chan) &&
+ !xilinx_dma_is_idle(chan)) {
dev_dbg(chan->dev, "DMA controller still busy\n");
return;
}
@@ -637,19 +941,19 @@
* done, start new transfers
*/
if (chan->has_sg)
- vdma_ctrl_write(chan, XILINX_VDMA_REG_CURDESC,
+ dma_ctrl_write(chan, XILINX_DMA_REG_CURDESC,
desc->async_tx.phys);
/* Configure the hardware using info in the config structure */
- reg = vdma_ctrl_read(chan, XILINX_VDMA_REG_DMACR);
+ reg = dma_ctrl_read(chan, XILINX_DMA_REG_DMACR);
if (config->frm_cnt_en)
- reg |= XILINX_VDMA_DMACR_FRAMECNT_EN;
+ reg |= XILINX_DMA_DMACR_FRAMECNT_EN;
else
- reg &= ~XILINX_VDMA_DMACR_FRAMECNT_EN;
+ reg &= ~XILINX_DMA_DMACR_FRAMECNT_EN;
/* Configure channel to allow number frame buffers */
- vdma_ctrl_write(chan, XILINX_VDMA_REG_FRMSTORE,
+ dma_ctrl_write(chan, XILINX_DMA_REG_FRMSTORE,
chan->desc_pendingcount);
/*
@@ -657,45 +961,53 @@
* In direct register mode, if not parking, enable circular mode
*/
if (chan->has_sg || !config->park)
- reg |= XILINX_VDMA_DMACR_CIRC_EN;
+ reg |= XILINX_DMA_DMACR_CIRC_EN;
if (config->park)
- reg &= ~XILINX_VDMA_DMACR_CIRC_EN;
+ reg &= ~XILINX_DMA_DMACR_CIRC_EN;
- vdma_ctrl_write(chan, XILINX_VDMA_REG_DMACR, reg);
+ dma_ctrl_write(chan, XILINX_DMA_REG_DMACR, reg);
if (config->park && (config->park_frm >= 0) &&
(config->park_frm < chan->num_frms)) {
if (chan->direction == DMA_MEM_TO_DEV)
- vdma_write(chan, XILINX_VDMA_REG_PARK_PTR,
+ dma_write(chan, XILINX_DMA_REG_PARK_PTR,
config->park_frm <<
- XILINX_VDMA_PARK_PTR_RD_REF_SHIFT);
+ XILINX_DMA_PARK_PTR_RD_REF_SHIFT);
else
- vdma_write(chan, XILINX_VDMA_REG_PARK_PTR,
+ dma_write(chan, XILINX_DMA_REG_PARK_PTR,
config->park_frm <<
- XILINX_VDMA_PARK_PTR_WR_REF_SHIFT);
+ XILINX_DMA_PARK_PTR_WR_REF_SHIFT);
}
/* Start the hardware */
- xilinx_vdma_start(chan);
+ xilinx_dma_start(chan);
if (chan->err)
return;
/* Start the transfer */
if (chan->has_sg) {
- vdma_ctrl_write(chan, XILINX_VDMA_REG_TAILDESC,
+ dma_ctrl_write(chan, XILINX_DMA_REG_TAILDESC,
tail_segment->phys);
} else {
struct xilinx_vdma_tx_segment *segment, *last = NULL;
int i = 0;
- list_for_each_entry(desc, &chan->pending_list, node) {
- segment = list_first_entry(&desc->segments,
- struct xilinx_vdma_tx_segment, node);
- vdma_desc_write(chan,
+ if (chan->desc_submitcount < chan->num_frms)
+ i = chan->desc_submitcount;
+
+ list_for_each_entry(segment, &desc->segments, node) {
+ if (chan->ext_addr)
+ vdma_desc_write_64(chan,
+ XILINX_VDMA_REG_START_ADDRESS_64(i++),
+ segment->hw.buf_addr,
+ segment->hw.buf_addr_msb);
+ else
+ vdma_desc_write(chan,
XILINX_VDMA_REG_START_ADDRESS(i++),
segment->hw.buf_addr);
+
last = segment;
}
@@ -703,10 +1015,79 @@
return;
/* HW expects these parameters to be same for one transaction */
- vdma_desc_write(chan, XILINX_VDMA_REG_HSIZE, last->hw.hsize);
- vdma_desc_write(chan, XILINX_VDMA_REG_FRMDLY_STRIDE,
+ vdma_desc_write(chan, XILINX_DMA_REG_HSIZE, last->hw.hsize);
+ vdma_desc_write(chan, XILINX_DMA_REG_FRMDLY_STRIDE,
last->hw.stride);
- vdma_desc_write(chan, XILINX_VDMA_REG_VSIZE, last->hw.vsize);
+ vdma_desc_write(chan, XILINX_DMA_REG_VSIZE, last->hw.vsize);
+ }
+
+ if (!chan->has_sg) {
+ list_del(&desc->node);
+ list_add_tail(&desc->node, &chan->active_list);
+ chan->desc_submitcount++;
+ chan->desc_pendingcount--;
+ if (chan->desc_submitcount == chan->num_frms)
+ chan->desc_submitcount = 0;
+ } else {
+ list_splice_tail_init(&chan->pending_list, &chan->active_list);
+ chan->desc_pendingcount = 0;
+ }
+}
+
+/**
+ * xilinx_cdma_start_transfer - Starts cdma transfer
+ * @chan: Driver specific channel struct pointer
+ */
+static void xilinx_cdma_start_transfer(struct xilinx_dma_chan *chan)
+{
+ struct xilinx_dma_tx_descriptor *head_desc, *tail_desc;
+ struct xilinx_cdma_tx_segment *tail_segment;
+ u32 ctrl_reg = dma_read(chan, XILINX_DMA_REG_DMACR);
+
+ if (chan->err)
+ return;
+
+ if (list_empty(&chan->pending_list))
+ return;
+
+ head_desc = list_first_entry(&chan->pending_list,
+ struct xilinx_dma_tx_descriptor, node);
+ tail_desc = list_last_entry(&chan->pending_list,
+ struct xilinx_dma_tx_descriptor, node);
+ tail_segment = list_last_entry(&tail_desc->segments,
+ struct xilinx_cdma_tx_segment, node);
+
+ if (chan->desc_pendingcount <= XILINX_DMA_COALESCE_MAX) {
+ ctrl_reg &= ~XILINX_DMA_CR_COALESCE_MAX;
+ ctrl_reg |= chan->desc_pendingcount <<
+ XILINX_DMA_CR_COALESCE_SHIFT;
+ dma_ctrl_write(chan, XILINX_DMA_REG_DMACR, ctrl_reg);
+ }
+
+ if (chan->has_sg) {
+ dma_ctrl_write(chan, XILINX_DMA_REG_CURDESC,
+ head_desc->async_tx.phys);
+
+ /* Update tail ptr register which will start the transfer */
+ dma_ctrl_write(chan, XILINX_DMA_REG_TAILDESC,
+ tail_segment->phys);
+ } else {
+ /* In simple mode */
+ struct xilinx_cdma_tx_segment *segment;
+ struct xilinx_cdma_desc_hw *hw;
+
+ segment = list_first_entry(&head_desc->segments,
+ struct xilinx_cdma_tx_segment,
+ node);
+
+ hw = &segment->hw;
+
+ dma_ctrl_write(chan, XILINX_CDMA_REG_SRCADDR, hw->src_addr);
+ dma_ctrl_write(chan, XILINX_CDMA_REG_DSTADDR, hw->dest_addr);
+
+ /* Start the transfer */
+ dma_ctrl_write(chan, XILINX_DMA_REG_BTT,
+ hw->control & XILINX_DMA_MAX_TRANS_LEN);
}
list_splice_tail_init(&chan->pending_list, &chan->active_list);
@@ -714,28 +1095,113 @@
}
/**
- * xilinx_vdma_issue_pending - Issue pending transactions
+ * xilinx_dma_start_transfer - Starts DMA transfer
+ * @chan: Driver specific channel struct pointer
+ */
+static void xilinx_dma_start_transfer(struct xilinx_dma_chan *chan)
+{
+ struct xilinx_dma_tx_descriptor *head_desc, *tail_desc;
+ struct xilinx_axidma_tx_segment *tail_segment, *old_head, *new_head;
+ u32 reg;
+
+ if (chan->err)
+ return;
+
+ if (list_empty(&chan->pending_list))
+ return;
+
+ /* If it is SG mode and hardware is busy, cannot submit */
+ if (chan->has_sg && xilinx_dma_is_running(chan) &&
+ !xilinx_dma_is_idle(chan)) {
+ dev_dbg(chan->dev, "DMA controller still busy\n");
+ return;
+ }
+
+ head_desc = list_first_entry(&chan->pending_list,
+ struct xilinx_dma_tx_descriptor, node);
+ tail_desc = list_last_entry(&chan->pending_list,
+ struct xilinx_dma_tx_descriptor, node);
+ tail_segment = list_last_entry(&tail_desc->segments,
+ struct xilinx_axidma_tx_segment, node);
+
+ old_head = list_first_entry(&head_desc->segments,
+ struct xilinx_axidma_tx_segment, node);
+ new_head = chan->seg_v;
+ /* Copy Buffer Descriptor fields. */
+ new_head->hw = old_head->hw;
+
+ /* Swap and save new reserve */
+ list_replace_init(&old_head->node, &new_head->node);
+ chan->seg_v = old_head;
+
+ tail_segment->hw.next_desc = chan->seg_v->phys;
+ head_desc->async_tx.phys = new_head->phys;
+
+ reg = dma_ctrl_read(chan, XILINX_DMA_REG_DMACR);
+
+ if (chan->desc_pendingcount <= XILINX_DMA_COALESCE_MAX) {
+ reg &= ~XILINX_DMA_CR_COALESCE_MAX;
+ reg |= chan->desc_pendingcount <<
+ XILINX_DMA_CR_COALESCE_SHIFT;
+ dma_ctrl_write(chan, XILINX_DMA_REG_DMACR, reg);
+ }
+
+ if (chan->has_sg)
+ dma_ctrl_write(chan, XILINX_DMA_REG_CURDESC,
+ head_desc->async_tx.phys);
+
+ xilinx_dma_start(chan);
+
+ if (chan->err)
+ return;
+
+ /* Start the transfer */
+ if (chan->has_sg) {
+ dma_ctrl_write(chan, XILINX_DMA_REG_TAILDESC,
+ tail_segment->phys);
+ } else {
+ struct xilinx_axidma_tx_segment *segment;
+ struct xilinx_axidma_desc_hw *hw;
+
+ segment = list_first_entry(&head_desc->segments,
+ struct xilinx_axidma_tx_segment,
+ node);
+ hw = &segment->hw;
+
+ dma_ctrl_write(chan, XILINX_DMA_REG_SRCDSTADDR, hw->buf_addr);
+
+ /* Start the transfer */
+ dma_ctrl_write(chan, XILINX_DMA_REG_BTT,
+ hw->control & XILINX_DMA_MAX_TRANS_LEN);
+ }
+
+ list_splice_tail_init(&chan->pending_list, &chan->active_list);
+ chan->desc_pendingcount = 0;
+}
+
+/**
+ * xilinx_dma_issue_pending - Issue pending transactions
* @dchan: DMA channel
*/
-static void xilinx_vdma_issue_pending(struct dma_chan *dchan)
+static void xilinx_dma_issue_pending(struct dma_chan *dchan)
{
- struct xilinx_vdma_chan *chan = to_xilinx_chan(dchan);
+ struct xilinx_dma_chan *chan = to_xilinx_chan(dchan);
unsigned long flags;
spin_lock_irqsave(&chan->lock, flags);
- xilinx_vdma_start_transfer(chan);
+ chan->start_transfer(chan);
spin_unlock_irqrestore(&chan->lock, flags);
}
/**
- * xilinx_vdma_complete_descriptor - Mark the active descriptor as complete
+ * xilinx_dma_complete_descriptor - Mark the active descriptor as complete
* @chan : xilinx DMA channel
*
* CONTEXT: hardirq
*/
-static void xilinx_vdma_complete_descriptor(struct xilinx_vdma_chan *chan)
+static void xilinx_dma_complete_descriptor(struct xilinx_dma_chan *chan)
{
- struct xilinx_vdma_tx_descriptor *desc, *next;
+ struct xilinx_dma_tx_descriptor *desc, *next;
/* This function was invoked with lock held */
if (list_empty(&chan->active_list))
@@ -749,27 +1215,27 @@
}
/**
- * xilinx_vdma_reset - Reset VDMA channel
- * @chan: Driver specific VDMA channel
+ * xilinx_dma_reset - Reset DMA channel
+ * @chan: Driver specific DMA channel
*
* Return: '0' on success and failure value on error
*/
-static int xilinx_vdma_reset(struct xilinx_vdma_chan *chan)
+static int xilinx_dma_reset(struct xilinx_dma_chan *chan)
{
int err;
u32 tmp;
- vdma_ctrl_set(chan, XILINX_VDMA_REG_DMACR, XILINX_VDMA_DMACR_RESET);
+ dma_ctrl_set(chan, XILINX_DMA_REG_DMACR, XILINX_DMA_DMACR_RESET);
/* Wait for the hardware to finish reset */
- err = xilinx_vdma_poll_timeout(chan, XILINX_VDMA_REG_DMACR, tmp,
- !(tmp & XILINX_VDMA_DMACR_RESET), 0,
- XILINX_VDMA_LOOP_COUNT);
+ err = xilinx_dma_poll_timeout(chan, XILINX_DMA_REG_DMACR, tmp,
+ !(tmp & XILINX_DMA_DMACR_RESET), 0,
+ XILINX_DMA_LOOP_COUNT);
if (err) {
dev_err(chan->dev, "reset timeout, cr %x, sr %x\n",
- vdma_ctrl_read(chan, XILINX_VDMA_REG_DMACR),
- vdma_ctrl_read(chan, XILINX_VDMA_REG_DMASR));
+ dma_ctrl_read(chan, XILINX_DMA_REG_DMACR),
+ dma_ctrl_read(chan, XILINX_DMA_REG_DMASR));
return -ETIMEDOUT;
}
@@ -779,48 +1245,48 @@
}
/**
- * xilinx_vdma_chan_reset - Reset VDMA channel and enable interrupts
- * @chan: Driver specific VDMA channel
+ * xilinx_dma_chan_reset - Reset DMA channel and enable interrupts
+ * @chan: Driver specific DMA channel
*
* Return: '0' on success and failure value on error
*/
-static int xilinx_vdma_chan_reset(struct xilinx_vdma_chan *chan)
+static int xilinx_dma_chan_reset(struct xilinx_dma_chan *chan)
{
int err;
/* Reset VDMA */
- err = xilinx_vdma_reset(chan);
+ err = xilinx_dma_reset(chan);
if (err)
return err;
/* Enable interrupts */
- vdma_ctrl_set(chan, XILINX_VDMA_REG_DMACR,
- XILINX_VDMA_DMAXR_ALL_IRQ_MASK);
+ dma_ctrl_set(chan, XILINX_DMA_REG_DMACR,
+ XILINX_DMA_DMAXR_ALL_IRQ_MASK);
return 0;
}
/**
- * xilinx_vdma_irq_handler - VDMA Interrupt handler
+ * xilinx_dma_irq_handler - DMA Interrupt handler
* @irq: IRQ number
- * @data: Pointer to the Xilinx VDMA channel structure
+ * @data: Pointer to the Xilinx DMA channel structure
*
* Return: IRQ_HANDLED/IRQ_NONE
*/
-static irqreturn_t xilinx_vdma_irq_handler(int irq, void *data)
+static irqreturn_t xilinx_dma_irq_handler(int irq, void *data)
{
- struct xilinx_vdma_chan *chan = data;
+ struct xilinx_dma_chan *chan = data;
u32 status;
/* Read the status and ack the interrupts. */
- status = vdma_ctrl_read(chan, XILINX_VDMA_REG_DMASR);
- if (!(status & XILINX_VDMA_DMAXR_ALL_IRQ_MASK))
+ status = dma_ctrl_read(chan, XILINX_DMA_REG_DMASR);
+ if (!(status & XILINX_DMA_DMAXR_ALL_IRQ_MASK))
return IRQ_NONE;
- vdma_ctrl_write(chan, XILINX_VDMA_REG_DMASR,
- status & XILINX_VDMA_DMAXR_ALL_IRQ_MASK);
+ dma_ctrl_write(chan, XILINX_DMA_REG_DMASR,
+ status & XILINX_DMA_DMAXR_ALL_IRQ_MASK);
- if (status & XILINX_VDMA_DMASR_ERR_IRQ) {
+ if (status & XILINX_DMA_DMASR_ERR_IRQ) {
/*
* An error occurred. If C_FLUSH_ON_FSYNC is enabled and the
* error is recoverable, ignore it. Otherwise flag the error.
@@ -828,22 +1294,23 @@
* Only recoverable errors can be cleared in the DMASR register,
* make sure not to write to other error bits to 1.
*/
- u32 errors = status & XILINX_VDMA_DMASR_ALL_ERR_MASK;
- vdma_ctrl_write(chan, XILINX_VDMA_REG_DMASR,
- errors & XILINX_VDMA_DMASR_ERR_RECOVER_MASK);
+ u32 errors = status & XILINX_DMA_DMASR_ALL_ERR_MASK;
+
+ dma_ctrl_write(chan, XILINX_DMA_REG_DMASR,
+ errors & XILINX_DMA_DMASR_ERR_RECOVER_MASK);
if (!chan->flush_on_fsync ||
- (errors & ~XILINX_VDMA_DMASR_ERR_RECOVER_MASK)) {
+ (errors & ~XILINX_DMA_DMASR_ERR_RECOVER_MASK)) {
dev_err(chan->dev,
"Channel %p has errors %x, cdr %x tdr %x\n",
chan, errors,
- vdma_ctrl_read(chan, XILINX_VDMA_REG_CURDESC),
- vdma_ctrl_read(chan, XILINX_VDMA_REG_TAILDESC));
+ dma_ctrl_read(chan, XILINX_DMA_REG_CURDESC),
+ dma_ctrl_read(chan, XILINX_DMA_REG_TAILDESC));
chan->err = true;
}
}
- if (status & XILINX_VDMA_DMASR_DLY_CNT_IRQ) {
+ if (status & XILINX_DMA_DMASR_DLY_CNT_IRQ) {
/*
* Device takes too long to do the transfer when user requires
* responsiveness.
@@ -851,10 +1318,10 @@
dev_dbg(chan->dev, "Inter-packet latency too long\n");
}
- if (status & XILINX_VDMA_DMASR_FRM_CNT_IRQ) {
+ if (status & XILINX_DMA_DMASR_FRM_CNT_IRQ) {
spin_lock(&chan->lock);
- xilinx_vdma_complete_descriptor(chan);
- xilinx_vdma_start_transfer(chan);
+ xilinx_dma_complete_descriptor(chan);
+ chan->start_transfer(chan);
spin_unlock(&chan->lock);
}
@@ -867,11 +1334,13 @@
* @chan: Driver specific dma channel
* @desc: dma transaction descriptor
*/
-static void append_desc_queue(struct xilinx_vdma_chan *chan,
- struct xilinx_vdma_tx_descriptor *desc)
+static void append_desc_queue(struct xilinx_dma_chan *chan,
+ struct xilinx_dma_tx_descriptor *desc)
{
struct xilinx_vdma_tx_segment *tail_segment;
- struct xilinx_vdma_tx_descriptor *tail_desc;
+ struct xilinx_dma_tx_descriptor *tail_desc;
+ struct xilinx_axidma_tx_segment *axidma_tail_segment;
+ struct xilinx_cdma_tx_segment *cdma_tail_segment;
if (list_empty(&chan->pending_list))
goto append;
@@ -881,10 +1350,23 @@
* that already exists in memory.
*/
tail_desc = list_last_entry(&chan->pending_list,
- struct xilinx_vdma_tx_descriptor, node);
- tail_segment = list_last_entry(&tail_desc->segments,
- struct xilinx_vdma_tx_segment, node);
- tail_segment->hw.next_desc = (u32)desc->async_tx.phys;
+ struct xilinx_dma_tx_descriptor, node);
+ if (chan->xdev->dma_config->dmatype == XDMA_TYPE_VDMA) {
+ tail_segment = list_last_entry(&tail_desc->segments,
+ struct xilinx_vdma_tx_segment,
+ node);
+ tail_segment->hw.next_desc = (u32)desc->async_tx.phys;
+ } else if (chan->xdev->dma_config->dmatype == XDMA_TYPE_CDMA) {
+ cdma_tail_segment = list_last_entry(&tail_desc->segments,
+ struct xilinx_cdma_tx_segment,
+ node);
+ cdma_tail_segment->hw.next_desc = (u32)desc->async_tx.phys;
+ } else {
+ axidma_tail_segment = list_last_entry(&tail_desc->segments,
+ struct xilinx_axidma_tx_segment,
+ node);
+ axidma_tail_segment->hw.next_desc = (u32)desc->async_tx.phys;
+ }
/*
* Add the software descriptor and all children to the list
@@ -894,22 +1376,23 @@
list_add_tail(&desc->node, &chan->pending_list);
chan->desc_pendingcount++;
- if (unlikely(chan->desc_pendingcount > chan->num_frms)) {
+ if (chan->has_sg && (chan->xdev->dma_config->dmatype == XDMA_TYPE_VDMA)
+ && unlikely(chan->desc_pendingcount > chan->num_frms)) {
dev_dbg(chan->dev, "desc pendingcount is too high\n");
chan->desc_pendingcount = chan->num_frms;
}
}
/**
- * xilinx_vdma_tx_submit - Submit DMA transaction
+ * xilinx_dma_tx_submit - Submit DMA transaction
* @tx: Async transaction descriptor
*
* Return: cookie value on success and failure value on error
*/
-static dma_cookie_t xilinx_vdma_tx_submit(struct dma_async_tx_descriptor *tx)
+static dma_cookie_t xilinx_dma_tx_submit(struct dma_async_tx_descriptor *tx)
{
- struct xilinx_vdma_tx_descriptor *desc = to_vdma_tx_descriptor(tx);
- struct xilinx_vdma_chan *chan = to_xilinx_chan(tx->chan);
+ struct xilinx_dma_tx_descriptor *desc = to_dma_tx_descriptor(tx);
+ struct xilinx_dma_chan *chan = to_xilinx_chan(tx->chan);
dma_cookie_t cookie;
unsigned long flags;
int err;
@@ -919,7 +1402,7 @@
* If reset fails, need to hard reset the system.
* Channel is no longer functional
*/
- err = xilinx_vdma_chan_reset(chan);
+ err = xilinx_dma_chan_reset(chan);
if (err < 0)
return err;
}
@@ -950,8 +1433,8 @@
struct dma_interleaved_template *xt,
unsigned long flags)
{
- struct xilinx_vdma_chan *chan = to_xilinx_chan(dchan);
- struct xilinx_vdma_tx_descriptor *desc;
+ struct xilinx_dma_chan *chan = to_xilinx_chan(dchan);
+ struct xilinx_dma_tx_descriptor *desc;
struct xilinx_vdma_tx_segment *segment, *prev = NULL;
struct xilinx_vdma_desc_hw *hw;
@@ -965,12 +1448,12 @@
return NULL;
/* Allocate a transaction descriptor. */
- desc = xilinx_vdma_alloc_tx_descriptor(chan);
+ desc = xilinx_dma_alloc_tx_descriptor(chan);
if (!desc)
return NULL;
dma_async_tx_descriptor_init(&desc->async_tx, &chan->common);
- desc->async_tx.tx_submit = xilinx_vdma_tx_submit;
+ desc->async_tx.tx_submit = xilinx_dma_tx_submit;
async_tx_ack(&desc->async_tx);
/* Allocate the link descriptor from DMA pool */
@@ -983,14 +1466,25 @@
hw->vsize = xt->numf;
hw->hsize = xt->sgl[0].size;
hw->stride = (xt->sgl[0].icg + xt->sgl[0].size) <<
- XILINX_VDMA_FRMDLY_STRIDE_STRIDE_SHIFT;
+ XILINX_DMA_FRMDLY_STRIDE_STRIDE_SHIFT;
hw->stride |= chan->config.frm_dly <<
- XILINX_VDMA_FRMDLY_STRIDE_FRMDLY_SHIFT;
+ XILINX_DMA_FRMDLY_STRIDE_FRMDLY_SHIFT;
- if (xt->dir != DMA_MEM_TO_DEV)
- hw->buf_addr = xt->dst_start;
- else
- hw->buf_addr = xt->src_start;
+ if (xt->dir != DMA_MEM_TO_DEV) {
+ if (chan->ext_addr) {
+ hw->buf_addr = lower_32_bits(xt->dst_start);
+ hw->buf_addr_msb = upper_32_bits(xt->dst_start);
+ } else {
+ hw->buf_addr = xt->dst_start;
+ }
+ } else {
+ if (chan->ext_addr) {
+ hw->buf_addr = lower_32_bits(xt->src_start);
+ hw->buf_addr_msb = upper_32_bits(xt->src_start);
+ } else {
+ hw->buf_addr = xt->src_start;
+ }
+ }
/* Insert the segment into the descriptor segments list. */
list_add_tail(&segment->node, &desc->segments);
@@ -1005,29 +1499,194 @@
return &desc->async_tx;
error:
- xilinx_vdma_free_tx_descriptor(chan, desc);
+ xilinx_dma_free_tx_descriptor(chan, desc);
return NULL;
}
/**
- * xilinx_vdma_terminate_all - Halt the channel and free descriptors
- * @chan: Driver specific VDMA Channel pointer
+ * xilinx_cdma_prep_memcpy - prepare descriptors for a memcpy transaction
+ * @dchan: DMA channel
+ * @dma_dst: destination address
+ * @dma_src: source address
+ * @len: transfer length
+ * @flags: transfer ack flags
+ *
+ * Return: Async transaction descriptor on success and NULL on failure
*/
-static int xilinx_vdma_terminate_all(struct dma_chan *dchan)
+static struct dma_async_tx_descriptor *
+xilinx_cdma_prep_memcpy(struct dma_chan *dchan, dma_addr_t dma_dst,
+ dma_addr_t dma_src, size_t len, unsigned long flags)
{
- struct xilinx_vdma_chan *chan = to_xilinx_chan(dchan);
+ struct xilinx_dma_chan *chan = to_xilinx_chan(dchan);
+ struct xilinx_dma_tx_descriptor *desc;
+ struct xilinx_cdma_tx_segment *segment, *prev;
+ struct xilinx_cdma_desc_hw *hw;
+
+ if (!len || len > XILINX_DMA_MAX_TRANS_LEN)
+ return NULL;
+
+ desc = xilinx_dma_alloc_tx_descriptor(chan);
+ if (!desc)
+ return NULL;
+
+ dma_async_tx_descriptor_init(&desc->async_tx, &chan->common);
+ desc->async_tx.tx_submit = xilinx_dma_tx_submit;
+
+ /* Allocate the link descriptor from DMA pool */
+ segment = xilinx_cdma_alloc_tx_segment(chan);
+ if (!segment)
+ goto error;
+
+ hw = &segment->hw;
+ hw->control = len;
+ hw->src_addr = dma_src;
+ hw->dest_addr = dma_dst;
+
+ /* Fill the previous next descriptor with current */
+ prev = list_last_entry(&desc->segments,
+ struct xilinx_cdma_tx_segment, node);
+ prev->hw.next_desc = segment->phys;
+
+ /* Insert the segment into the descriptor segments list. */
+ list_add_tail(&segment->node, &desc->segments);
+
+ prev = segment;
+
+ /* Link the last hardware descriptor with the first. */
+ segment = list_first_entry(&desc->segments,
+ struct xilinx_cdma_tx_segment, node);
+ desc->async_tx.phys = segment->phys;
+ prev->hw.next_desc = segment->phys;
+
+ return &desc->async_tx;
+
+error:
+ xilinx_dma_free_tx_descriptor(chan, desc);
+ return NULL;
+}
+
+/**
+ * xilinx_dma_prep_slave_sg - prepare descriptors for a DMA_SLAVE transaction
+ * @dchan: DMA channel
+ * @sgl: scatterlist to transfer to/from
+ * @sg_len: number of entries in @scatterlist
+ * @direction: DMA direction
+ * @flags: transfer ack flags
+ * @context: APP words of the descriptor
+ *
+ * Return: Async transaction descriptor on success and NULL on failure
+ */
+static struct dma_async_tx_descriptor *xilinx_dma_prep_slave_sg(
+ struct dma_chan *dchan, struct scatterlist *sgl, unsigned int sg_len,
+ enum dma_transfer_direction direction, unsigned long flags,
+ void *context)
+{
+ struct xilinx_dma_chan *chan = to_xilinx_chan(dchan);
+ struct xilinx_dma_tx_descriptor *desc;
+ struct xilinx_axidma_tx_segment *segment = NULL, *prev = NULL;
+ u32 *app_w = (u32 *)context;
+ struct scatterlist *sg;
+ size_t copy;
+ size_t sg_used;
+ unsigned int i;
+
+ if (!is_slave_direction(direction))
+ return NULL;
+
+ /* Allocate a transaction descriptor. */
+ desc = xilinx_dma_alloc_tx_descriptor(chan);
+ if (!desc)
+ return NULL;
+
+ dma_async_tx_descriptor_init(&desc->async_tx, &chan->common);
+ desc->async_tx.tx_submit = xilinx_dma_tx_submit;
+
+ /* Build transactions using information in the scatter gather list */
+ for_each_sg(sgl, sg, sg_len, i) {
+ sg_used = 0;
+
+ /* Loop until the entire scatterlist entry is used */
+ while (sg_used < sg_dma_len(sg)) {
+ struct xilinx_axidma_desc_hw *hw;
+
+ /* Get a free segment */
+ segment = xilinx_axidma_alloc_tx_segment(chan);
+ if (!segment)
+ goto error;
+
+ /*
+ * Calculate the maximum number of bytes to transfer,
+ * making sure it is less than the hw limit
+ */
+ copy = min_t(size_t, sg_dma_len(sg) - sg_used,
+ XILINX_DMA_MAX_TRANS_LEN);
+ hw = &segment->hw;
+
+ /* Fill in the descriptor */
+ hw->buf_addr = sg_dma_address(sg) + sg_used;
+
+ hw->control = copy;
+
+ if (chan->direction == DMA_MEM_TO_DEV) {
+ if (app_w)
+ memcpy(hw->app, app_w, sizeof(u32) *
+ XILINX_DMA_NUM_APP_WORDS);
+ }
+
+ if (prev)
+ prev->hw.next_desc = segment->phys;
+
+ prev = segment;
+ sg_used += copy;
+
+ /*
+ * Insert the segment into the descriptor segments
+ * list.
+ */
+ list_add_tail(&segment->node, &desc->segments);
+ }
+ }
+
+ segment = list_first_entry(&desc->segments,
+ struct xilinx_axidma_tx_segment, node);
+ desc->async_tx.phys = segment->phys;
+ prev->hw.next_desc = segment->phys;
+
+ /* For the last DMA_MEM_TO_DEV transfer, set EOP */
+ if (chan->direction == DMA_MEM_TO_DEV) {
+ segment->hw.control |= XILINX_DMA_BD_SOP;
+ segment = list_last_entry(&desc->segments,
+ struct xilinx_axidma_tx_segment,
+ node);
+ segment->hw.control |= XILINX_DMA_BD_EOP;
+ }
+
+ return &desc->async_tx;
+
+error:
+ xilinx_dma_free_tx_descriptor(chan, desc);
+ return NULL;
+}
+
+/**
+ * xilinx_dma_terminate_all - Halt the channel and free descriptors
+ * @chan: Driver specific DMA Channel pointer
+ */
+static int xilinx_dma_terminate_all(struct dma_chan *dchan)
+{
+ struct xilinx_dma_chan *chan = to_xilinx_chan(dchan);
/* Halt the DMA engine */
- xilinx_vdma_halt(chan);
+ xilinx_dma_halt(chan);
/* Remove and free all of the descriptors in the lists */
- xilinx_vdma_free_descriptors(chan);
+ xilinx_dma_free_descriptors(chan);
return 0;
}
/**
- * xilinx_vdma_channel_set_config - Configure VDMA channel
+ * xilinx_dma_channel_set_config - Configure VDMA channel
* Run-time configuration for Axi VDMA, supports:
* . halt the channel
* . configure interrupt coalescing and inter-packet delay threshold
@@ -1042,13 +1701,13 @@
int xilinx_vdma_channel_set_config(struct dma_chan *dchan,
struct xilinx_vdma_config *cfg)
{
- struct xilinx_vdma_chan *chan = to_xilinx_chan(dchan);
+ struct xilinx_dma_chan *chan = to_xilinx_chan(dchan);
u32 dmacr;
if (cfg->reset)
- return xilinx_vdma_chan_reset(chan);
+ return xilinx_dma_chan_reset(chan);
- dmacr = vdma_ctrl_read(chan, XILINX_VDMA_REG_DMACR);
+ dmacr = dma_ctrl_read(chan, XILINX_DMA_REG_DMACR);
chan->config.frm_dly = cfg->frm_dly;
chan->config.park = cfg->park;
@@ -1058,8 +1717,8 @@
chan->config.master = cfg->master;
if (cfg->gen_lock && chan->genlock) {
- dmacr |= XILINX_VDMA_DMACR_GENLOCK_EN;
- dmacr |= cfg->master << XILINX_VDMA_DMACR_MASTER_SHIFT;
+ dmacr |= XILINX_DMA_DMACR_GENLOCK_EN;
+ dmacr |= cfg->master << XILINX_DMA_DMACR_MASTER_SHIFT;
}
chan->config.frm_cnt_en = cfg->frm_cnt_en;
@@ -1071,21 +1730,21 @@
chan->config.coalesc = cfg->coalesc;
chan->config.delay = cfg->delay;
- if (cfg->coalesc <= XILINX_VDMA_DMACR_FRAME_COUNT_MAX) {
- dmacr |= cfg->coalesc << XILINX_VDMA_DMACR_FRAME_COUNT_SHIFT;
+ if (cfg->coalesc <= XILINX_DMA_DMACR_FRAME_COUNT_MAX) {
+ dmacr |= cfg->coalesc << XILINX_DMA_DMACR_FRAME_COUNT_SHIFT;
chan->config.coalesc = cfg->coalesc;
}
- if (cfg->delay <= XILINX_VDMA_DMACR_DELAY_MAX) {
- dmacr |= cfg->delay << XILINX_VDMA_DMACR_DELAY_SHIFT;
+ if (cfg->delay <= XILINX_DMA_DMACR_DELAY_MAX) {
+ dmacr |= cfg->delay << XILINX_DMA_DMACR_DELAY_SHIFT;
chan->config.delay = cfg->delay;
}
/* FSync Source selection */
- dmacr &= ~XILINX_VDMA_DMACR_FSYNCSRC_MASK;
- dmacr |= cfg->ext_fsync << XILINX_VDMA_DMACR_FSYNCSRC_SHIFT;
+ dmacr &= ~XILINX_DMA_DMACR_FSYNCSRC_MASK;
+ dmacr |= cfg->ext_fsync << XILINX_DMA_DMACR_FSYNCSRC_SHIFT;
- vdma_ctrl_write(chan, XILINX_VDMA_REG_DMACR, dmacr);
+ dma_ctrl_write(chan, XILINX_DMA_REG_DMACR, dmacr);
return 0;
}
@@ -1096,14 +1755,14 @@
*/
/**
- * xilinx_vdma_chan_remove - Per Channel remove function
- * @chan: Driver specific VDMA channel
+ * xilinx_dma_chan_remove - Per Channel remove function
+ * @chan: Driver specific DMA channel
*/
-static void xilinx_vdma_chan_remove(struct xilinx_vdma_chan *chan)
+static void xilinx_dma_chan_remove(struct xilinx_dma_chan *chan)
{
/* Disable all interrupts */
- vdma_ctrl_clr(chan, XILINX_VDMA_REG_DMACR,
- XILINX_VDMA_DMAXR_ALL_IRQ_MASK);
+ dma_ctrl_clr(chan, XILINX_DMA_REG_DMACR,
+ XILINX_DMA_DMAXR_ALL_IRQ_MASK);
if (chan->irq > 0)
free_irq(chan->irq, chan);
@@ -1113,8 +1772,197 @@
list_del(&chan->common.device_node);
}
+static int axidma_clk_init(struct platform_device *pdev, struct clk **axi_clk,
+ struct clk **tx_clk, struct clk **rx_clk,
+ struct clk **sg_clk, struct clk **tmp_clk)
+{
+ int err;
+
+ *tmp_clk = NULL;
+
+ *axi_clk = devm_clk_get(&pdev->dev, "s_axi_lite_aclk");
+ if (IS_ERR(*axi_clk)) {
+ err = PTR_ERR(*axi_clk);
+ dev_err(&pdev->dev, "failed to get axi_aclk (%u)\n", err);
+ return err;
+ }
+
+ *tx_clk = devm_clk_get(&pdev->dev, "m_axi_mm2s_aclk");
+ if (IS_ERR(*tx_clk))
+ *tx_clk = NULL;
+
+ *rx_clk = devm_clk_get(&pdev->dev, "m_axi_s2mm_aclk");
+ if (IS_ERR(*rx_clk))
+ *rx_clk = NULL;
+
+ *sg_clk = devm_clk_get(&pdev->dev, "m_axi_sg_aclk");
+ if (IS_ERR(*sg_clk))
+ *sg_clk = NULL;
+
+ err = clk_prepare_enable(*axi_clk);
+ if (err) {
+ dev_err(&pdev->dev, "failed to enable axi_clk (%u)\n", err);
+ return err;
+ }
+
+ err = clk_prepare_enable(*tx_clk);
+ if (err) {
+ dev_err(&pdev->dev, "failed to enable tx_clk (%u)\n", err);
+ goto err_disable_axiclk;
+ }
+
+ err = clk_prepare_enable(*rx_clk);
+ if (err) {
+ dev_err(&pdev->dev, "failed to enable rx_clk (%u)\n", err);
+ goto err_disable_txclk;
+ }
+
+ err = clk_prepare_enable(*sg_clk);
+ if (err) {
+ dev_err(&pdev->dev, "failed to enable sg_clk (%u)\n", err);
+ goto err_disable_rxclk;
+ }
+
+ return 0;
+
+err_disable_rxclk:
+ clk_disable_unprepare(*rx_clk);
+err_disable_txclk:
+ clk_disable_unprepare(*tx_clk);
+err_disable_axiclk:
+ clk_disable_unprepare(*axi_clk);
+
+ return err;
+}
+
+static int axicdma_clk_init(struct platform_device *pdev, struct clk **axi_clk,
+ struct clk **dev_clk, struct clk **tmp_clk,
+ struct clk **tmp1_clk, struct clk **tmp2_clk)
+{
+ int err;
+
+ *tmp_clk = NULL;
+ *tmp1_clk = NULL;
+ *tmp2_clk = NULL;
+
+ *axi_clk = devm_clk_get(&pdev->dev, "s_axi_lite_aclk");
+ if (IS_ERR(*axi_clk)) {
+ err = PTR_ERR(*axi_clk);
+ dev_err(&pdev->dev, "failed to get axi_clk (%u)\n", err);
+ return err;
+ }
+
+ *dev_clk = devm_clk_get(&pdev->dev, "m_axi_aclk");
+ if (IS_ERR(*dev_clk)) {
+ err = PTR_ERR(*dev_clk);
+ dev_err(&pdev->dev, "failed to get dev_clk (%u)\n", err);
+ return err;
+ }
+
+ err = clk_prepare_enable(*axi_clk);
+ if (err) {
+ dev_err(&pdev->dev, "failed to enable axi_clk (%u)\n", err);
+ return err;
+ }
+
+ err = clk_prepare_enable(*dev_clk);
+ if (err) {
+ dev_err(&pdev->dev, "failed to enable dev_clk (%u)\n", err);
+ goto err_disable_axiclk;
+ }
+
+ return 0;
+
+err_disable_axiclk:
+ clk_disable_unprepare(*axi_clk);
+
+ return err;
+}
+
+static int axivdma_clk_init(struct platform_device *pdev, struct clk **axi_clk,
+ struct clk **tx_clk, struct clk **txs_clk,
+ struct clk **rx_clk, struct clk **rxs_clk)
+{
+ int err;
+
+ *axi_clk = devm_clk_get(&pdev->dev, "s_axi_lite_aclk");
+ if (IS_ERR(*axi_clk)) {
+ err = PTR_ERR(*axi_clk);
+ dev_err(&pdev->dev, "failed to get axi_aclk (%u)\n", err);
+ return err;
+ }
+
+ *tx_clk = devm_clk_get(&pdev->dev, "m_axi_mm2s_aclk");
+ if (IS_ERR(*tx_clk))
+ *tx_clk = NULL;
+
+ *txs_clk = devm_clk_get(&pdev->dev, "m_axis_mm2s_aclk");
+ if (IS_ERR(*txs_clk))
+ *txs_clk = NULL;
+
+ *rx_clk = devm_clk_get(&pdev->dev, "m_axi_s2mm_aclk");
+ if (IS_ERR(*rx_clk))
+ *rx_clk = NULL;
+
+ *rxs_clk = devm_clk_get(&pdev->dev, "s_axis_s2mm_aclk");
+ if (IS_ERR(*rxs_clk))
+ *rxs_clk = NULL;
+
+ err = clk_prepare_enable(*axi_clk);
+ if (err) {
+ dev_err(&pdev->dev, "failed to enable axi_clk (%u)\n", err);
+ return err;
+ }
+
+ err = clk_prepare_enable(*tx_clk);
+ if (err) {
+ dev_err(&pdev->dev, "failed to enable tx_clk (%u)\n", err);
+ goto err_disable_axiclk;
+ }
+
+ err = clk_prepare_enable(*txs_clk);
+ if (err) {
+ dev_err(&pdev->dev, "failed to enable txs_clk (%u)\n", err);
+ goto err_disable_txclk;
+ }
+
+ err = clk_prepare_enable(*rx_clk);
+ if (err) {
+ dev_err(&pdev->dev, "failed to enable rx_clk (%u)\n", err);
+ goto err_disable_txsclk;
+ }
+
+ err = clk_prepare_enable(*rxs_clk);
+ if (err) {
+ dev_err(&pdev->dev, "failed to enable rxs_clk (%u)\n", err);
+ goto err_disable_rxclk;
+ }
+
+ return 0;
+
+err_disable_rxclk:
+ clk_disable_unprepare(*rx_clk);
+err_disable_txsclk:
+ clk_disable_unprepare(*txs_clk);
+err_disable_txclk:
+ clk_disable_unprepare(*tx_clk);
+err_disable_axiclk:
+ clk_disable_unprepare(*axi_clk);
+
+ return err;
+}
+
+static void xdma_disable_allclks(struct xilinx_dma_device *xdev)
+{
+ clk_disable_unprepare(xdev->rxs_clk);
+ clk_disable_unprepare(xdev->rx_clk);
+ clk_disable_unprepare(xdev->txs_clk);
+ clk_disable_unprepare(xdev->tx_clk);
+ clk_disable_unprepare(xdev->axi_clk);
+}
+
/**
- * xilinx_vdma_chan_probe - Per Channel Probing
+ * xilinx_dma_chan_probe - Per Channel Probing
* It get channel features from the device tree entry and
* initialize special channel handling routines
*
@@ -1123,10 +1971,10 @@
*
* Return: '0' on success and failure value on error
*/
-static int xilinx_vdma_chan_probe(struct xilinx_vdma_device *xdev,
+static int xilinx_dma_chan_probe(struct xilinx_dma_device *xdev,
struct device_node *node)
{
- struct xilinx_vdma_chan *chan;
+ struct xilinx_dma_chan *chan;
bool has_dre = false;
u32 value, width;
int err;
@@ -1140,6 +1988,7 @@
chan->xdev = xdev;
chan->has_sg = xdev->has_sg;
chan->desc_pendingcount = 0x0;
+ chan->ext_addr = xdev->ext_addr;
spin_lock_init(&chan->lock);
INIT_LIST_HEAD(&chan->pending_list);
@@ -1169,23 +2018,27 @@
chan->direction = DMA_MEM_TO_DEV;
chan->id = 0;
- chan->ctrl_offset = XILINX_VDMA_MM2S_CTRL_OFFSET;
- chan->desc_offset = XILINX_VDMA_MM2S_DESC_OFFSET;
+ chan->ctrl_offset = XILINX_DMA_MM2S_CTRL_OFFSET;
+ if (xdev->dma_config->dmatype == XDMA_TYPE_VDMA) {
+ chan->desc_offset = XILINX_VDMA_MM2S_DESC_OFFSET;
- if (xdev->flush_on_fsync == XILINX_VDMA_FLUSH_BOTH ||
- xdev->flush_on_fsync == XILINX_VDMA_FLUSH_MM2S)
- chan->flush_on_fsync = true;
+ if (xdev->flush_on_fsync == XILINX_DMA_FLUSH_BOTH ||
+ xdev->flush_on_fsync == XILINX_DMA_FLUSH_MM2S)
+ chan->flush_on_fsync = true;
+ }
} else if (of_device_is_compatible(node,
"xlnx,axi-vdma-s2mm-channel")) {
chan->direction = DMA_DEV_TO_MEM;
chan->id = 1;
- chan->ctrl_offset = XILINX_VDMA_S2MM_CTRL_OFFSET;
- chan->desc_offset = XILINX_VDMA_S2MM_DESC_OFFSET;
+ chan->ctrl_offset = XILINX_DMA_S2MM_CTRL_OFFSET;
+ if (xdev->dma_config->dmatype == XDMA_TYPE_VDMA) {
+ chan->desc_offset = XILINX_VDMA_S2MM_DESC_OFFSET;
- if (xdev->flush_on_fsync == XILINX_VDMA_FLUSH_BOTH ||
- xdev->flush_on_fsync == XILINX_VDMA_FLUSH_S2MM)
- chan->flush_on_fsync = true;
+ if (xdev->flush_on_fsync == XILINX_DMA_FLUSH_BOTH ||
+ xdev->flush_on_fsync == XILINX_DMA_FLUSH_S2MM)
+ chan->flush_on_fsync = true;
+ }
} else {
dev_err(xdev->dev, "Invalid channel compatible node\n");
return -EINVAL;
@@ -1193,15 +2046,22 @@
/* Request the interrupt */
chan->irq = irq_of_parse_and_map(node, 0);
- err = request_irq(chan->irq, xilinx_vdma_irq_handler, IRQF_SHARED,
- "xilinx-vdma-controller", chan);
+ err = request_irq(chan->irq, xilinx_dma_irq_handler, IRQF_SHARED,
+ "xilinx-dma-controller", chan);
if (err) {
dev_err(xdev->dev, "unable to request IRQ %d\n", chan->irq);
return err;
}
+ if (xdev->dma_config->dmatype == XDMA_TYPE_AXIDMA)
+ chan->start_transfer = xilinx_dma_start_transfer;
+ else if (xdev->dma_config->dmatype == XDMA_TYPE_CDMA)
+ chan->start_transfer = xilinx_cdma_start_transfer;
+ else
+ chan->start_transfer = xilinx_vdma_start_transfer;
+
/* Initialize the tasklet */
- tasklet_init(&chan->tasklet, xilinx_vdma_do_tasklet,
+ tasklet_init(&chan->tasklet, xilinx_dma_do_tasklet,
(unsigned long)chan);
/*
@@ -1214,7 +2074,7 @@
xdev->chan[chan->id] = chan;
/* Reset the channel */
- err = xilinx_vdma_chan_reset(chan);
+ err = xilinx_dma_chan_reset(chan);
if (err < 0) {
dev_err(xdev->dev, "Reset channel failed\n");
return err;
@@ -1233,28 +2093,54 @@
static struct dma_chan *of_dma_xilinx_xlate(struct of_phandle_args *dma_spec,
struct of_dma *ofdma)
{
- struct xilinx_vdma_device *xdev = ofdma->of_dma_data;
+ struct xilinx_dma_device *xdev = ofdma->of_dma_data;
int chan_id = dma_spec->args[0];
- if (chan_id >= XILINX_VDMA_MAX_CHANS_PER_DEVICE || !xdev->chan[chan_id])
+ if (chan_id >= XILINX_DMA_MAX_CHANS_PER_DEVICE || !xdev->chan[chan_id])
return NULL;
return dma_get_slave_channel(&xdev->chan[chan_id]->common);
}
+static const struct xilinx_dma_config axidma_config = {
+ .dmatype = XDMA_TYPE_AXIDMA,
+ .clk_init = axidma_clk_init,
+};
+
+static const struct xilinx_dma_config axicdma_config = {
+ .dmatype = XDMA_TYPE_CDMA,
+ .clk_init = axicdma_clk_init,
+};
+
+static const struct xilinx_dma_config axivdma_config = {
+ .dmatype = XDMA_TYPE_VDMA,
+ .clk_init = axivdma_clk_init,
+};
+
+static const struct of_device_id xilinx_dma_of_ids[] = {
+ { .compatible = "xlnx,axi-dma-1.00.a", .data = &axidma_config },
+ { .compatible = "xlnx,axi-cdma-1.00.a", .data = &axicdma_config },
+ { .compatible = "xlnx,axi-vdma-1.00.a", .data = &axivdma_config },
+ {}
+};
+MODULE_DEVICE_TABLE(of, xilinx_dma_of_ids);
+
/**
- * xilinx_vdma_probe - Driver probe function
+ * xilinx_dma_probe - Driver probe function
* @pdev: Pointer to the platform_device structure
*
* Return: '0' on success and failure value on error
*/
-static int xilinx_vdma_probe(struct platform_device *pdev)
+static int xilinx_dma_probe(struct platform_device *pdev)
{
+ int (*clk_init)(struct platform_device *, struct clk **, struct clk **,
+ struct clk **, struct clk **, struct clk **)
+ = axivdma_clk_init;
struct device_node *node = pdev->dev.of_node;
- struct xilinx_vdma_device *xdev;
- struct device_node *child;
+ struct xilinx_dma_device *xdev;
+ struct device_node *child, *np = pdev->dev.of_node;
struct resource *io;
- u32 num_frames;
+ u32 num_frames, addr_width;
int i, err;
/* Allocate and initialize the DMA engine structure */
@@ -1263,6 +2149,20 @@
return -ENOMEM;
xdev->dev = &pdev->dev;
+ if (np) {
+ const struct of_device_id *match;
+
+ match = of_match_node(xilinx_dma_of_ids, np);
+ if (match && match->data) {
+ xdev->dma_config = match->data;
+ clk_init = xdev->dma_config->clk_init;
+ }
+ }
+
+ err = clk_init(pdev, &xdev->axi_clk, &xdev->tx_clk, &xdev->txs_clk,
+ &xdev->rx_clk, &xdev->rxs_clk);
+ if (err)
+ return err;
/* Request and map I/O memory */
io = platform_get_resource(pdev, IORESOURCE_MEM, 0);
@@ -1273,46 +2173,77 @@
/* Retrieve the DMA engine properties from the device tree */
xdev->has_sg = of_property_read_bool(node, "xlnx,include-sg");
- err = of_property_read_u32(node, "xlnx,num-fstores", &num_frames);
- if (err < 0) {
- dev_err(xdev->dev, "missing xlnx,num-fstores property\n");
- return err;
+ if (xdev->dma_config->dmatype == XDMA_TYPE_VDMA) {
+ err = of_property_read_u32(node, "xlnx,num-fstores",
+ &num_frames);
+ if (err < 0) {
+ dev_err(xdev->dev,
+ "missing xlnx,num-fstores property\n");
+ return err;
+ }
+
+ err = of_property_read_u32(node, "xlnx,flush-fsync",
+ &xdev->flush_on_fsync);
+ if (err < 0)
+ dev_warn(xdev->dev,
+ "missing xlnx,flush-fsync property\n");
}
- err = of_property_read_u32(node, "xlnx,flush-fsync",
- &xdev->flush_on_fsync);
+ err = of_property_read_u32(node, "xlnx,addrwidth", &addr_width);
if (err < 0)
- dev_warn(xdev->dev, "missing xlnx,flush-fsync property\n");
+ dev_warn(xdev->dev, "missing xlnx,addrwidth property\n");
+
+ if (addr_width > 32)
+ xdev->ext_addr = true;
+ else
+ xdev->ext_addr = false;
+
+ /* Set the dma mask bits */
+ dma_set_mask(xdev->dev, DMA_BIT_MASK(addr_width));
/* Initialize the DMA engine */
xdev->common.dev = &pdev->dev;
INIT_LIST_HEAD(&xdev->common.channels);
- dma_cap_set(DMA_SLAVE, xdev->common.cap_mask);
- dma_cap_set(DMA_PRIVATE, xdev->common.cap_mask);
+ if (!(xdev->dma_config->dmatype == XDMA_TYPE_CDMA)) {
+ dma_cap_set(DMA_SLAVE, xdev->common.cap_mask);
+ dma_cap_set(DMA_PRIVATE, xdev->common.cap_mask);
+ }
xdev->common.device_alloc_chan_resources =
- xilinx_vdma_alloc_chan_resources;
+ xilinx_dma_alloc_chan_resources;
xdev->common.device_free_chan_resources =
- xilinx_vdma_free_chan_resources;
- xdev->common.device_prep_interleaved_dma =
+ xilinx_dma_free_chan_resources;
+ xdev->common.device_terminate_all = xilinx_dma_terminate_all;
+ xdev->common.device_tx_status = xilinx_dma_tx_status;
+ xdev->common.device_issue_pending = xilinx_dma_issue_pending;
+ if (xdev->dma_config->dmatype == XDMA_TYPE_AXIDMA) {
+ xdev->common.device_prep_slave_sg = xilinx_dma_prep_slave_sg;
+ /* Residue calculation is supported by only AXI DMA */
+ xdev->common.residue_granularity =
+ DMA_RESIDUE_GRANULARITY_SEGMENT;
+ } else if (xdev->dma_config->dmatype == XDMA_TYPE_CDMA) {
+ dma_cap_set(DMA_MEMCPY, xdev->common.cap_mask);
+ xdev->common.device_prep_dma_memcpy = xilinx_cdma_prep_memcpy;
+ } else {
+ xdev->common.device_prep_interleaved_dma =
xilinx_vdma_dma_prep_interleaved;
- xdev->common.device_terminate_all = xilinx_vdma_terminate_all;
- xdev->common.device_tx_status = xilinx_vdma_tx_status;
- xdev->common.device_issue_pending = xilinx_vdma_issue_pending;
+ }
platform_set_drvdata(pdev, xdev);
/* Initialize the channels */
for_each_child_of_node(node, child) {
- err = xilinx_vdma_chan_probe(xdev, child);
+ err = xilinx_dma_chan_probe(xdev, child);
if (err < 0)
- goto error;
+ goto disable_clks;
}
- for (i = 0; i < XILINX_VDMA_MAX_CHANS_PER_DEVICE; i++)
- if (xdev->chan[i])
- xdev->chan[i]->num_frms = num_frames;
+ if (xdev->dma_config->dmatype == XDMA_TYPE_VDMA) {
+ for (i = 0; i < XILINX_DMA_MAX_CHANS_PER_DEVICE; i++)
+ if (xdev->chan[i])
+ xdev->chan[i]->num_frms = num_frames;
+ }
/* Register the DMA engine with the core */
dma_async_device_register(&xdev->common);
@@ -1329,49 +2260,47 @@
return 0;
+disable_clks:
+ xdma_disable_allclks(xdev);
error:
- for (i = 0; i < XILINX_VDMA_MAX_CHANS_PER_DEVICE; i++)
+ for (i = 0; i < XILINX_DMA_MAX_CHANS_PER_DEVICE; i++)
if (xdev->chan[i])
- xilinx_vdma_chan_remove(xdev->chan[i]);
+ xilinx_dma_chan_remove(xdev->chan[i]);
return err;
}
/**
- * xilinx_vdma_remove - Driver remove function
+ * xilinx_dma_remove - Driver remove function
* @pdev: Pointer to the platform_device structure
*
* Return: Always '0'
*/
-static int xilinx_vdma_remove(struct platform_device *pdev)
+static int xilinx_dma_remove(struct platform_device *pdev)
{
- struct xilinx_vdma_device *xdev = platform_get_drvdata(pdev);
+ struct xilinx_dma_device *xdev = platform_get_drvdata(pdev);
int i;
of_dma_controller_free(pdev->dev.of_node);
dma_async_device_unregister(&xdev->common);
- for (i = 0; i < XILINX_VDMA_MAX_CHANS_PER_DEVICE; i++)
+ for (i = 0; i < XILINX_DMA_MAX_CHANS_PER_DEVICE; i++)
if (xdev->chan[i])
- xilinx_vdma_chan_remove(xdev->chan[i]);
+ xilinx_dma_chan_remove(xdev->chan[i]);
+
+ xdma_disable_allclks(xdev);
return 0;
}
-static const struct of_device_id xilinx_vdma_of_ids[] = {
- { .compatible = "xlnx,axi-vdma-1.00.a",},
- {}
-};
-MODULE_DEVICE_TABLE(of, xilinx_vdma_of_ids);
-
static struct platform_driver xilinx_vdma_driver = {
.driver = {
.name = "xilinx-vdma",
- .of_match_table = xilinx_vdma_of_ids,
+ .of_match_table = xilinx_dma_of_ids,
},
- .probe = xilinx_vdma_probe,
- .remove = xilinx_vdma_remove,
+ .probe = xilinx_dma_probe,
+ .remove = xilinx_dma_remove,
};
module_platform_driver(xilinx_vdma_driver);
diff --git a/drivers/firmware/broadcom/Kconfig b/drivers/firmware/broadcom/Kconfig
index 6bed1199..3c7e5b7 100644
--- a/drivers/firmware/broadcom/Kconfig
+++ b/drivers/firmware/broadcom/Kconfig
@@ -9,3 +9,14 @@
This driver provides an easy way to get value of requested parameter.
It simply reads content of NVRAM and parses it. It doesn't control any
hardware part itself.
+
+config BCM47XX_SPROM
+ bool "Broadcom SPROM driver"
+ depends on BCM47XX_NVRAM
+ help
+ Broadcom devices store configuration data in SPROM. Accessing it is
+ specific to the bus host type, e.g. PCI(e) devices have it mapped in
+ a PCI BAR.
+ In case of SoC devices SPROM content is stored on a flash used by
+ bootloader firmware CFE. This driver provides method to ssb and bcma
+ drivers to read SPROM on SoC.
diff --git a/drivers/firmware/broadcom/Makefile b/drivers/firmware/broadcom/Makefile
index d0e6835..f93efc4 100644
--- a/drivers/firmware/broadcom/Makefile
+++ b/drivers/firmware/broadcom/Makefile
@@ -1 +1,2 @@
obj-$(CONFIG_BCM47XX_NVRAM) += bcm47xx_nvram.o
+obj-$(CONFIG_BCM47XX_SPROM) += bcm47xx_sprom.o
diff --git a/drivers/firmware/broadcom/bcm47xx_sprom.c b/drivers/firmware/broadcom/bcm47xx_sprom.c
new file mode 100644
index 0000000..b6eb875
--- /dev/null
+++ b/drivers/firmware/broadcom/bcm47xx_sprom.c
@@ -0,0 +1,737 @@
+/*
+ * Copyright (C) 2004 Florian Schirmer <jolt@tuxbox.org>
+ * Copyright (C) 2006 Felix Fietkau <nbd@openwrt.org>
+ * Copyright (C) 2006 Michael Buesch <m@bues.ch>
+ * Copyright (C) 2010 Waldemar Brodkorb <wbx@openadk.org>
+ * Copyright (C) 2010-2012 Hauke Mehrtens <hauke@hauke-m.de>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ *
+ * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED
+ * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN
+ * NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
+ * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+ * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ * ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+ * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write to the Free Software Foundation, Inc.,
+ * 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#include <linux/bcm47xx_nvram.h>
+#include <linux/bcma/bcma.h>
+#include <linux/etherdevice.h>
+#include <linux/if_ether.h>
+#include <linux/ssb/ssb.h>
+
+static void create_key(const char *prefix, const char *postfix,
+ const char *name, char *buf, int len)
+{
+ if (prefix && postfix)
+ snprintf(buf, len, "%s%s%s", prefix, name, postfix);
+ else if (prefix)
+ snprintf(buf, len, "%s%s", prefix, name);
+ else if (postfix)
+ snprintf(buf, len, "%s%s", name, postfix);
+ else
+ snprintf(buf, len, "%s", name);
+}
+
+static int get_nvram_var(const char *prefix, const char *postfix,
+ const char *name, char *buf, int len, bool fallback)
+{
+ char key[40];
+ int err;
+
+ create_key(prefix, postfix, name, key, sizeof(key));
+
+ err = bcm47xx_nvram_getenv(key, buf, len);
+ if (fallback && err == -ENOENT && prefix) {
+ create_key(NULL, postfix, name, key, sizeof(key));
+ err = bcm47xx_nvram_getenv(key, buf, len);
+ }
+ return err;
+}
+
+#define NVRAM_READ_VAL(type) \
+static void nvram_read_ ## type(const char *prefix, \
+ const char *postfix, const char *name, \
+ type *val, type allset, bool fallback) \
+{ \
+ char buf[100]; \
+ int err; \
+ type var; \
+ \
+ err = get_nvram_var(prefix, postfix, name, buf, sizeof(buf), \
+ fallback); \
+ if (err < 0) \
+ return; \
+ err = kstrto ## type(strim(buf), 0, &var); \
+ if (err) { \
+ pr_warn("can not parse nvram name %s%s%s with value %s got %i\n", \
+ prefix, name, postfix, buf, err); \
+ return; \
+ } \
+ if (allset && var == allset) \
+ return; \
+ *val = var; \
+}
+
+NVRAM_READ_VAL(u8)
+NVRAM_READ_VAL(s8)
+NVRAM_READ_VAL(u16)
+NVRAM_READ_VAL(u32)
+
+#undef NVRAM_READ_VAL
+
+static void nvram_read_u32_2(const char *prefix, const char *name,
+ u16 *val_lo, u16 *val_hi, bool fallback)
+{
+ char buf[100];
+ int err;
+ u32 val;
+
+ err = get_nvram_var(prefix, NULL, name, buf, sizeof(buf), fallback);
+ if (err < 0)
+ return;
+ err = kstrtou32(strim(buf), 0, &val);
+ if (err) {
+ pr_warn("can not parse nvram name %s%s with value %s got %i\n",
+ prefix, name, buf, err);
+ return;
+ }
+ *val_lo = (val & 0x0000FFFFU);
+ *val_hi = (val & 0xFFFF0000U) >> 16;
+}
+
+static void nvram_read_leddc(const char *prefix, const char *name,
+ u8 *leddc_on_time, u8 *leddc_off_time,
+ bool fallback)
+{
+ char buf[100];
+ int err;
+ u32 val;
+
+ err = get_nvram_var(prefix, NULL, name, buf, sizeof(buf), fallback);
+ if (err < 0)
+ return;
+ err = kstrtou32(strim(buf), 0, &val);
+ if (err) {
+ pr_warn("can not parse nvram name %s%s with value %s got %i\n",
+ prefix, name, buf, err);
+ return;
+ }
+
+ if (val == 0xffff || val == 0xffffffff)
+ return;
+
+ *leddc_on_time = val & 0xff;
+ *leddc_off_time = (val >> 16) & 0xff;
+}
+
+static void bcm47xx_nvram_parse_macaddr(char *buf, u8 macaddr[6])
+{
+ if (strchr(buf, ':'))
+ sscanf(buf, "%hhx:%hhx:%hhx:%hhx:%hhx:%hhx", &macaddr[0],
+ &macaddr[1], &macaddr[2], &macaddr[3], &macaddr[4],
+ &macaddr[5]);
+ else if (strchr(buf, '-'))
+ sscanf(buf, "%hhx-%hhx-%hhx-%hhx-%hhx-%hhx", &macaddr[0],
+ &macaddr[1], &macaddr[2], &macaddr[3], &macaddr[4],
+ &macaddr[5]);
+ else
+ pr_warn("Can not parse mac address: %s\n", buf);
+}
+
+static void nvram_read_macaddr(const char *prefix, const char *name,
+ u8 val[6], bool fallback)
+{
+ char buf[100];
+ int err;
+
+ err = get_nvram_var(prefix, NULL, name, buf, sizeof(buf), fallback);
+ if (err < 0)
+ return;
+
+ bcm47xx_nvram_parse_macaddr(buf, val);
+}
+
+static void nvram_read_alpha2(const char *prefix, const char *name,
+ char val[2], bool fallback)
+{
+ char buf[10];
+ int err;
+
+ err = get_nvram_var(prefix, NULL, name, buf, sizeof(buf), fallback);
+ if (err < 0)
+ return;
+ if (buf[0] == '0')
+ return;
+ if (strlen(buf) > 2) {
+ pr_warn("alpha2 is too long %s\n", buf);
+ return;
+ }
+ memcpy(val, buf, 2);
+}
+
+/* This is one-function-only macro, it uses local "sprom" variable! */
+#define ENTRY(_revmask, _type, _prefix, _name, _val, _allset, _fallback) \
+ if (_revmask & BIT(sprom->revision)) \
+ nvram_read_ ## _type(_prefix, NULL, _name, &sprom->_val, \
+ _allset, _fallback)
+/*
+ * Special version of filling function that can be safely called for any SPROM
+ * revision. For every NVRAM to SPROM mapping it contains bitmask of revisions
+ * for which the mapping is valid.
+ * It obviously requires some hexadecimal/bitmasks knowledge, but allows
+ * writing cleaner code (easy revisions handling).
+ * Note that while SPROM revision 0 was never used, we still keep BIT(0)
+ * reserved for it, just to keep numbering sane.
+ */
+static void bcm47xx_sprom_fill_auto(struct ssb_sprom *sprom,
+ const char *prefix, bool fallback)
+{
+ const char *pre = prefix;
+ bool fb = fallback;
+
+ /* Broadcom extracts it for rev 8+ but it was found on 2 and 4 too */
+ ENTRY(0xfffffffe, u16, pre, "devid", dev_id, 0, fallback);
+
+ ENTRY(0xfffffffe, u16, pre, "boardrev", board_rev, 0, true);
+ ENTRY(0xfffffffe, u32, pre, "boardflags", boardflags, 0, fb);
+ ENTRY(0xfffffff0, u32, pre, "boardflags2", boardflags2, 0, fb);
+ ENTRY(0xfffff800, u32, pre, "boardflags3", boardflags3, 0, fb);
+ ENTRY(0x00000002, u16, pre, "boardflags", boardflags_lo, 0, fb);
+ ENTRY(0xfffffffc, u16, pre, "boardtype", board_type, 0, true);
+ ENTRY(0xfffffffe, u16, pre, "boardnum", board_num, 0, fb);
+ ENTRY(0x00000002, u8, pre, "cc", country_code, 0, fb);
+ ENTRY(0xfffffff8, u8, pre, "regrev", regrev, 0, fb);
+
+ ENTRY(0xfffffffe, u8, pre, "ledbh0", gpio0, 0xff, fb);
+ ENTRY(0xfffffffe, u8, pre, "ledbh1", gpio1, 0xff, fb);
+ ENTRY(0xfffffffe, u8, pre, "ledbh2", gpio2, 0xff, fb);
+ ENTRY(0xfffffffe, u8, pre, "ledbh3", gpio3, 0xff, fb);
+
+ ENTRY(0x0000070e, u16, pre, "pa0b0", pa0b0, 0, fb);
+ ENTRY(0x0000070e, u16, pre, "pa0b1", pa0b1, 0, fb);
+ ENTRY(0x0000070e, u16, pre, "pa0b2", pa0b2, 0, fb);
+ ENTRY(0x0000070e, u8, pre, "pa0itssit", itssi_bg, 0, fb);
+ ENTRY(0x0000070e, u8, pre, "pa0maxpwr", maxpwr_bg, 0, fb);
+
+ ENTRY(0x0000070c, u8, pre, "opo", opo, 0, fb);
+ ENTRY(0xfffffffe, u8, pre, "aa2g", ant_available_bg, 0, fb);
+ ENTRY(0xfffffffe, u8, pre, "aa5g", ant_available_a, 0, fb);
+ ENTRY(0x000007fe, s8, pre, "ag0", antenna_gain.a0, 0, fb);
+ ENTRY(0x000007fe, s8, pre, "ag1", antenna_gain.a1, 0, fb);
+ ENTRY(0x000007f0, s8, pre, "ag2", antenna_gain.a2, 0, fb);
+ ENTRY(0x000007f0, s8, pre, "ag3", antenna_gain.a3, 0, fb);
+
+ ENTRY(0x0000070e, u16, pre, "pa1b0", pa1b0, 0, fb);
+ ENTRY(0x0000070e, u16, pre, "pa1b1", pa1b1, 0, fb);
+ ENTRY(0x0000070e, u16, pre, "pa1b2", pa1b2, 0, fb);
+ ENTRY(0x0000070c, u16, pre, "pa1lob0", pa1lob0, 0, fb);
+ ENTRY(0x0000070c, u16, pre, "pa1lob1", pa1lob1, 0, fb);
+ ENTRY(0x0000070c, u16, pre, "pa1lob2", pa1lob2, 0, fb);
+ ENTRY(0x0000070c, u16, pre, "pa1hib0", pa1hib0, 0, fb);
+ ENTRY(0x0000070c, u16, pre, "pa1hib1", pa1hib1, 0, fb);
+ ENTRY(0x0000070c, u16, pre, "pa1hib2", pa1hib2, 0, fb);
+ ENTRY(0x0000070e, u8, pre, "pa1itssit", itssi_a, 0, fb);
+ ENTRY(0x0000070e, u8, pre, "pa1maxpwr", maxpwr_a, 0, fb);
+ ENTRY(0x0000070c, u8, pre, "pa1lomaxpwr", maxpwr_al, 0, fb);
+ ENTRY(0x0000070c, u8, pre, "pa1himaxpwr", maxpwr_ah, 0, fb);
+
+ ENTRY(0x00000708, u8, pre, "bxa2g", bxa2g, 0, fb);
+ ENTRY(0x00000708, u8, pre, "rssisav2g", rssisav2g, 0, fb);
+ ENTRY(0x00000708, u8, pre, "rssismc2g", rssismc2g, 0, fb);
+ ENTRY(0x00000708, u8, pre, "rssismf2g", rssismf2g, 0, fb);
+ ENTRY(0x00000708, u8, pre, "bxa5g", bxa5g, 0, fb);
+ ENTRY(0x00000708, u8, pre, "rssisav5g", rssisav5g, 0, fb);
+ ENTRY(0x00000708, u8, pre, "rssismc5g", rssismc5g, 0, fb);
+ ENTRY(0x00000708, u8, pre, "rssismf5g", rssismf5g, 0, fb);
+ ENTRY(0x00000708, u8, pre, "tri2g", tri2g, 0, fb);
+ ENTRY(0x00000708, u8, pre, "tri5g", tri5g, 0, fb);
+ ENTRY(0x00000708, u8, pre, "tri5gl", tri5gl, 0, fb);
+ ENTRY(0x00000708, u8, pre, "tri5gh", tri5gh, 0, fb);
+ ENTRY(0x00000708, s8, pre, "rxpo2g", rxpo2g, 0, fb);
+ ENTRY(0x00000708, s8, pre, "rxpo5g", rxpo5g, 0, fb);
+ ENTRY(0xfffffff0, u8, pre, "txchain", txchain, 0xf, fb);
+ ENTRY(0xfffffff0, u8, pre, "rxchain", rxchain, 0xf, fb);
+ ENTRY(0xfffffff0, u8, pre, "antswitch", antswitch, 0xff, fb);
+ ENTRY(0x00000700, u8, pre, "tssipos2g", fem.ghz2.tssipos, 0, fb);
+ ENTRY(0x00000700, u8, pre, "extpagain2g", fem.ghz2.extpa_gain, 0, fb);
+ ENTRY(0x00000700, u8, pre, "pdetrange2g", fem.ghz2.pdet_range, 0, fb);
+ ENTRY(0x00000700, u8, pre, "triso2g", fem.ghz2.tr_iso, 0, fb);
+ ENTRY(0x00000700, u8, pre, "antswctl2g", fem.ghz2.antswlut, 0, fb);
+ ENTRY(0x00000700, u8, pre, "tssipos5g", fem.ghz5.tssipos, 0, fb);
+ ENTRY(0x00000700, u8, pre, "extpagain5g", fem.ghz5.extpa_gain, 0, fb);
+ ENTRY(0x00000700, u8, pre, "pdetrange5g", fem.ghz5.pdet_range, 0, fb);
+ ENTRY(0x00000700, u8, pre, "triso5g", fem.ghz5.tr_iso, 0, fb);
+ ENTRY(0x00000700, u8, pre, "antswctl5g", fem.ghz5.antswlut, 0, fb);
+ ENTRY(0x000000f0, u8, pre, "txpid2ga0", txpid2g[0], 0, fb);
+ ENTRY(0x000000f0, u8, pre, "txpid2ga1", txpid2g[1], 0, fb);
+ ENTRY(0x000000f0, u8, pre, "txpid2ga2", txpid2g[2], 0, fb);
+ ENTRY(0x000000f0, u8, pre, "txpid2ga3", txpid2g[3], 0, fb);
+ ENTRY(0x000000f0, u8, pre, "txpid5ga0", txpid5g[0], 0, fb);
+ ENTRY(0x000000f0, u8, pre, "txpid5ga1", txpid5g[1], 0, fb);
+ ENTRY(0x000000f0, u8, pre, "txpid5ga2", txpid5g[2], 0, fb);
+ ENTRY(0x000000f0, u8, pre, "txpid5ga3", txpid5g[3], 0, fb);
+ ENTRY(0x000000f0, u8, pre, "txpid5gla0", txpid5gl[0], 0, fb);
+ ENTRY(0x000000f0, u8, pre, "txpid5gla1", txpid5gl[1], 0, fb);
+ ENTRY(0x000000f0, u8, pre, "txpid5gla2", txpid5gl[2], 0, fb);
+ ENTRY(0x000000f0, u8, pre, "txpid5gla3", txpid5gl[3], 0, fb);
+ ENTRY(0x000000f0, u8, pre, "txpid5gha0", txpid5gh[0], 0, fb);
+ ENTRY(0x000000f0, u8, pre, "txpid5gha1", txpid5gh[1], 0, fb);
+ ENTRY(0x000000f0, u8, pre, "txpid5gha2", txpid5gh[2], 0, fb);
+ ENTRY(0x000000f0, u8, pre, "txpid5gha3", txpid5gh[3], 0, fb);
+
+ ENTRY(0xffffff00, u8, pre, "tempthresh", tempthresh, 0, fb);
+ ENTRY(0xffffff00, u8, pre, "tempoffset", tempoffset, 0, fb);
+ ENTRY(0xffffff00, u16, pre, "rawtempsense", rawtempsense, 0, fb);
+ ENTRY(0xffffff00, u8, pre, "measpower", measpower, 0, fb);
+ ENTRY(0xffffff00, u8, pre, "tempsense_slope", tempsense_slope, 0, fb);
+ ENTRY(0xffffff00, u8, pre, "tempcorrx", tempcorrx, 0, fb);
+ ENTRY(0xffffff00, u8, pre, "tempsense_option", tempsense_option, 0, fb);
+ ENTRY(0x00000700, u8, pre, "freqoffset_corr", freqoffset_corr, 0, fb);
+ ENTRY(0x00000700, u8, pre, "iqcal_swp_dis", iqcal_swp_dis, 0, fb);
+ ENTRY(0x00000700, u8, pre, "hw_iqcal_en", hw_iqcal_en, 0, fb);
+ ENTRY(0x00000700, u8, pre, "elna2g", elna2g, 0, fb);
+ ENTRY(0x00000700, u8, pre, "elna5g", elna5g, 0, fb);
+ ENTRY(0xffffff00, u8, pre, "phycal_tempdelta", phycal_tempdelta, 0, fb);
+ ENTRY(0xffffff00, u8, pre, "temps_period", temps_period, 0, fb);
+ ENTRY(0xffffff00, u8, pre, "temps_hysteresis", temps_hysteresis, 0, fb);
+ ENTRY(0xffffff00, u8, pre, "measpower1", measpower1, 0, fb);
+ ENTRY(0xffffff00, u8, pre, "measpower2", measpower2, 0, fb);
+
+ ENTRY(0x000001f0, u16, pre, "cck2gpo", cck2gpo, 0, fb);
+ ENTRY(0x000001f0, u32, pre, "ofdm2gpo", ofdm2gpo, 0, fb);
+ ENTRY(0x000001f0, u32, pre, "ofdm5gpo", ofdm5gpo, 0, fb);
+ ENTRY(0x000001f0, u32, pre, "ofdm5glpo", ofdm5glpo, 0, fb);
+ ENTRY(0x000001f0, u32, pre, "ofdm5ghpo", ofdm5ghpo, 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs2gpo0", mcs2gpo[0], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs2gpo1", mcs2gpo[1], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs2gpo2", mcs2gpo[2], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs2gpo3", mcs2gpo[3], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs2gpo4", mcs2gpo[4], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs2gpo5", mcs2gpo[5], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs2gpo6", mcs2gpo[6], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs2gpo7", mcs2gpo[7], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5gpo0", mcs5gpo[0], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5gpo1", mcs5gpo[1], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5gpo2", mcs5gpo[2], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5gpo3", mcs5gpo[3], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5gpo4", mcs5gpo[4], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5gpo5", mcs5gpo[5], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5gpo6", mcs5gpo[6], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5gpo7", mcs5gpo[7], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5glpo0", mcs5glpo[0], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5glpo1", mcs5glpo[1], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5glpo2", mcs5glpo[2], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5glpo3", mcs5glpo[3], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5glpo4", mcs5glpo[4], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5glpo5", mcs5glpo[5], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5glpo6", mcs5glpo[6], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5glpo7", mcs5glpo[7], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5ghpo0", mcs5ghpo[0], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5ghpo1", mcs5ghpo[1], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5ghpo2", mcs5ghpo[2], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5ghpo3", mcs5ghpo[3], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5ghpo4", mcs5ghpo[4], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5ghpo5", mcs5ghpo[5], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5ghpo6", mcs5ghpo[6], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "mcs5ghpo7", mcs5ghpo[7], 0, fb);
+ ENTRY(0x000001f0, u16, pre, "cddpo", cddpo, 0, fb);
+ ENTRY(0x000001f0, u16, pre, "stbcpo", stbcpo, 0, fb);
+ ENTRY(0x000001f0, u16, pre, "bw40po", bw40po, 0, fb);
+ ENTRY(0x000001f0, u16, pre, "bwduppo", bwduppo, 0, fb);
+
+ ENTRY(0xfffffe00, u16, pre, "cckbw202gpo", cckbw202gpo, 0, fb);
+ ENTRY(0xfffffe00, u16, pre, "cckbw20ul2gpo", cckbw20ul2gpo, 0, fb);
+ ENTRY(0x00000600, u32, pre, "legofdmbw202gpo", legofdmbw202gpo, 0, fb);
+ ENTRY(0x00000600, u32, pre, "legofdmbw20ul2gpo", legofdmbw20ul2gpo, 0, fb);
+ ENTRY(0x00000600, u32, pre, "legofdmbw205glpo", legofdmbw205glpo, 0, fb);
+ ENTRY(0x00000600, u32, pre, "legofdmbw20ul5glpo", legofdmbw20ul5glpo, 0, fb);
+ ENTRY(0x00000600, u32, pre, "legofdmbw205gmpo", legofdmbw205gmpo, 0, fb);
+ ENTRY(0x00000600, u32, pre, "legofdmbw20ul5gmpo", legofdmbw20ul5gmpo, 0, fb);
+ ENTRY(0x00000600, u32, pre, "legofdmbw205ghpo", legofdmbw205ghpo, 0, fb);
+ ENTRY(0x00000600, u32, pre, "legofdmbw20ul5ghpo", legofdmbw20ul5ghpo, 0, fb);
+ ENTRY(0xfffffe00, u32, pre, "mcsbw202gpo", mcsbw202gpo, 0, fb);
+ ENTRY(0x00000600, u32, pre, "mcsbw20ul2gpo", mcsbw20ul2gpo, 0, fb);
+ ENTRY(0xfffffe00, u32, pre, "mcsbw402gpo", mcsbw402gpo, 0, fb);
+ ENTRY(0xfffffe00, u32, pre, "mcsbw205glpo", mcsbw205glpo, 0, fb);
+ ENTRY(0x00000600, u32, pre, "mcsbw20ul5glpo", mcsbw20ul5glpo, 0, fb);
+ ENTRY(0xfffffe00, u32, pre, "mcsbw405glpo", mcsbw405glpo, 0, fb);
+ ENTRY(0xfffffe00, u32, pre, "mcsbw205gmpo", mcsbw205gmpo, 0, fb);
+ ENTRY(0x00000600, u32, pre, "mcsbw20ul5gmpo", mcsbw20ul5gmpo, 0, fb);
+ ENTRY(0xfffffe00, u32, pre, "mcsbw405gmpo", mcsbw405gmpo, 0, fb);
+ ENTRY(0xfffffe00, u32, pre, "mcsbw205ghpo", mcsbw205ghpo, 0, fb);
+ ENTRY(0x00000600, u32, pre, "mcsbw20ul5ghpo", mcsbw20ul5ghpo, 0, fb);
+ ENTRY(0xfffffe00, u32, pre, "mcsbw405ghpo", mcsbw405ghpo, 0, fb);
+ ENTRY(0x00000600, u16, pre, "mcs32po", mcs32po, 0, fb);
+ ENTRY(0x00000600, u16, pre, "legofdm40duppo", legofdm40duppo, 0, fb);
+ ENTRY(0x00000700, u8, pre, "pcieingress_war", pcieingress_war, 0, fb);
+
+ /* TODO: rev 11 support */
+ ENTRY(0x00000700, u8, pre, "rxgainerr2ga0", rxgainerr2ga[0], 0, fb);
+ ENTRY(0x00000700, u8, pre, "rxgainerr2ga1", rxgainerr2ga[1], 0, fb);
+ ENTRY(0x00000700, u8, pre, "rxgainerr2ga2", rxgainerr2ga[2], 0, fb);
+ ENTRY(0x00000700, u8, pre, "rxgainerr5gla0", rxgainerr5gla[0], 0, fb);
+ ENTRY(0x00000700, u8, pre, "rxgainerr5gla1", rxgainerr5gla[1], 0, fb);
+ ENTRY(0x00000700, u8, pre, "rxgainerr5gla2", rxgainerr5gla[2], 0, fb);
+ ENTRY(0x00000700, u8, pre, "rxgainerr5gma0", rxgainerr5gma[0], 0, fb);
+ ENTRY(0x00000700, u8, pre, "rxgainerr5gma1", rxgainerr5gma[1], 0, fb);
+ ENTRY(0x00000700, u8, pre, "rxgainerr5gma2", rxgainerr5gma[2], 0, fb);
+ ENTRY(0x00000700, u8, pre, "rxgainerr5gha0", rxgainerr5gha[0], 0, fb);
+ ENTRY(0x00000700, u8, pre, "rxgainerr5gha1", rxgainerr5gha[1], 0, fb);
+ ENTRY(0x00000700, u8, pre, "rxgainerr5gha2", rxgainerr5gha[2], 0, fb);
+ ENTRY(0x00000700, u8, pre, "rxgainerr5gua0", rxgainerr5gua[0], 0, fb);
+ ENTRY(0x00000700, u8, pre, "rxgainerr5gua1", rxgainerr5gua[1], 0, fb);
+ ENTRY(0x00000700, u8, pre, "rxgainerr5gua2", rxgainerr5gua[2], 0, fb);
+
+ ENTRY(0xfffffe00, u8, pre, "sar2g", sar2g, 0, fb);
+ ENTRY(0xfffffe00, u8, pre, "sar5g", sar5g, 0, fb);
+
+ /* TODO: rev 11 support */
+ ENTRY(0x00000700, u8, pre, "noiselvl2ga0", noiselvl2ga[0], 0, fb);
+ ENTRY(0x00000700, u8, pre, "noiselvl2ga1", noiselvl2ga[1], 0, fb);
+ ENTRY(0x00000700, u8, pre, "noiselvl2ga2", noiselvl2ga[2], 0, fb);
+ ENTRY(0x00000700, u8, pre, "noiselvl5gla0", noiselvl5gla[0], 0, fb);
+ ENTRY(0x00000700, u8, pre, "noiselvl5gla1", noiselvl5gla[1], 0, fb);
+ ENTRY(0x00000700, u8, pre, "noiselvl5gla2", noiselvl5gla[2], 0, fb);
+ ENTRY(0x00000700, u8, pre, "noiselvl5gma0", noiselvl5gma[0], 0, fb);
+ ENTRY(0x00000700, u8, pre, "noiselvl5gma1", noiselvl5gma[1], 0, fb);
+ ENTRY(0x00000700, u8, pre, "noiselvl5gma2", noiselvl5gma[2], 0, fb);
+ ENTRY(0x00000700, u8, pre, "noiselvl5gha0", noiselvl5gha[0], 0, fb);
+ ENTRY(0x00000700, u8, pre, "noiselvl5gha1", noiselvl5gha[1], 0, fb);
+ ENTRY(0x00000700, u8, pre, "noiselvl5gha2", noiselvl5gha[2], 0, fb);
+ ENTRY(0x00000700, u8, pre, "noiselvl5gua0", noiselvl5gua[0], 0, fb);
+ ENTRY(0x00000700, u8, pre, "noiselvl5gua1", noiselvl5gua[1], 0, fb);
+ ENTRY(0x00000700, u8, pre, "noiselvl5gua2", noiselvl5gua[2], 0, fb);
+}
+#undef ENTRY /* It's specififc, uses local variable, don't use it (again). */
+
+static void bcm47xx_fill_sprom_path_r4589(struct ssb_sprom *sprom,
+ const char *prefix, bool fallback)
+{
+ char postfix[2];
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(sprom->core_pwr_info); i++) {
+ struct ssb_sprom_core_pwr_info *pwr_info;
+
+ pwr_info = &sprom->core_pwr_info[i];
+
+ snprintf(postfix, sizeof(postfix), "%i", i);
+ nvram_read_u8(prefix, postfix, "maxp2ga",
+ &pwr_info->maxpwr_2g, 0, fallback);
+ nvram_read_u8(prefix, postfix, "itt2ga",
+ &pwr_info->itssi_2g, 0, fallback);
+ nvram_read_u8(prefix, postfix, "itt5ga",
+ &pwr_info->itssi_5g, 0, fallback);
+ nvram_read_u16(prefix, postfix, "pa2gw0a",
+ &pwr_info->pa_2g[0], 0, fallback);
+ nvram_read_u16(prefix, postfix, "pa2gw1a",
+ &pwr_info->pa_2g[1], 0, fallback);
+ nvram_read_u16(prefix, postfix, "pa2gw2a",
+ &pwr_info->pa_2g[2], 0, fallback);
+ nvram_read_u8(prefix, postfix, "maxp5ga",
+ &pwr_info->maxpwr_5g, 0, fallback);
+ nvram_read_u8(prefix, postfix, "maxp5gha",
+ &pwr_info->maxpwr_5gh, 0, fallback);
+ nvram_read_u8(prefix, postfix, "maxp5gla",
+ &pwr_info->maxpwr_5gl, 0, fallback);
+ nvram_read_u16(prefix, postfix, "pa5gw0a",
+ &pwr_info->pa_5g[0], 0, fallback);
+ nvram_read_u16(prefix, postfix, "pa5gw1a",
+ &pwr_info->pa_5g[1], 0, fallback);
+ nvram_read_u16(prefix, postfix, "pa5gw2a",
+ &pwr_info->pa_5g[2], 0, fallback);
+ nvram_read_u16(prefix, postfix, "pa5glw0a",
+ &pwr_info->pa_5gl[0], 0, fallback);
+ nvram_read_u16(prefix, postfix, "pa5glw1a",
+ &pwr_info->pa_5gl[1], 0, fallback);
+ nvram_read_u16(prefix, postfix, "pa5glw2a",
+ &pwr_info->pa_5gl[2], 0, fallback);
+ nvram_read_u16(prefix, postfix, "pa5ghw0a",
+ &pwr_info->pa_5gh[0], 0, fallback);
+ nvram_read_u16(prefix, postfix, "pa5ghw1a",
+ &pwr_info->pa_5gh[1], 0, fallback);
+ nvram_read_u16(prefix, postfix, "pa5ghw2a",
+ &pwr_info->pa_5gh[2], 0, fallback);
+ }
+}
+
+static void bcm47xx_fill_sprom_path_r45(struct ssb_sprom *sprom,
+ const char *prefix, bool fallback)
+{
+ char postfix[2];
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(sprom->core_pwr_info); i++) {
+ struct ssb_sprom_core_pwr_info *pwr_info;
+
+ pwr_info = &sprom->core_pwr_info[i];
+
+ snprintf(postfix, sizeof(postfix), "%i", i);
+ nvram_read_u16(prefix, postfix, "pa2gw3a",
+ &pwr_info->pa_2g[3], 0, fallback);
+ nvram_read_u16(prefix, postfix, "pa5gw3a",
+ &pwr_info->pa_5g[3], 0, fallback);
+ nvram_read_u16(prefix, postfix, "pa5glw3a",
+ &pwr_info->pa_5gl[3], 0, fallback);
+ nvram_read_u16(prefix, postfix, "pa5ghw3a",
+ &pwr_info->pa_5gh[3], 0, fallback);
+ }
+}
+
+static bool bcm47xx_is_valid_mac(u8 *mac)
+{
+ return mac && !(mac[0] == 0x00 && mac[1] == 0x90 && mac[2] == 0x4c);
+}
+
+static int bcm47xx_increase_mac_addr(u8 *mac, u8 num)
+{
+ u8 *oui = mac + ETH_ALEN/2 - 1;
+ u8 *p = mac + ETH_ALEN - 1;
+
+ do {
+ (*p) += num;
+ if (*p > num)
+ break;
+ p--;
+ num = 1;
+ } while (p != oui);
+
+ if (p == oui) {
+ pr_err("unable to fetch mac address\n");
+ return -ENOENT;
+ }
+ return 0;
+}
+
+static int mac_addr_used = 2;
+
+static void bcm47xx_fill_sprom_ethernet(struct ssb_sprom *sprom,
+ const char *prefix, bool fallback)
+{
+ bool fb = fallback;
+
+ nvram_read_macaddr(prefix, "et0macaddr", sprom->et0mac, fallback);
+ nvram_read_u8(prefix, NULL, "et0mdcport", &sprom->et0mdcport, 0,
+ fallback);
+ nvram_read_u8(prefix, NULL, "et0phyaddr", &sprom->et0phyaddr, 0,
+ fallback);
+
+ nvram_read_macaddr(prefix, "et1macaddr", sprom->et1mac, fallback);
+ nvram_read_u8(prefix, NULL, "et1mdcport", &sprom->et1mdcport, 0,
+ fallback);
+ nvram_read_u8(prefix, NULL, "et1phyaddr", &sprom->et1phyaddr, 0,
+ fallback);
+
+ nvram_read_macaddr(prefix, "et2macaddr", sprom->et2mac, fb);
+ nvram_read_u8(prefix, NULL, "et2mdcport", &sprom->et2mdcport, 0, fb);
+ nvram_read_u8(prefix, NULL, "et2phyaddr", &sprom->et2phyaddr, 0, fb);
+
+ nvram_read_macaddr(prefix, "macaddr", sprom->il0mac, fallback);
+ nvram_read_macaddr(prefix, "il0macaddr", sprom->il0mac, fallback);
+
+ /* The address prefix 00:90:4C is used by Broadcom in their initial
+ * configuration. When a mac address with the prefix 00:90:4C is used
+ * all devices from the same series are sharing the same mac address.
+ * To prevent mac address collisions we replace them with a mac address
+ * based on the base address.
+ */
+ if (!bcm47xx_is_valid_mac(sprom->il0mac)) {
+ u8 mac[6];
+
+ nvram_read_macaddr(NULL, "et0macaddr", mac, false);
+ if (bcm47xx_is_valid_mac(mac)) {
+ int err = bcm47xx_increase_mac_addr(mac, mac_addr_used);
+
+ if (!err) {
+ ether_addr_copy(sprom->il0mac, mac);
+ mac_addr_used++;
+ }
+ }
+ }
+}
+
+static void bcm47xx_fill_board_data(struct ssb_sprom *sprom, const char *prefix,
+ bool fallback)
+{
+ nvram_read_u32_2(prefix, "boardflags", &sprom->boardflags_lo,
+ &sprom->boardflags_hi, fallback);
+ nvram_read_u32_2(prefix, "boardflags2", &sprom->boardflags2_lo,
+ &sprom->boardflags2_hi, fallback);
+}
+
+void bcm47xx_fill_sprom(struct ssb_sprom *sprom, const char *prefix,
+ bool fallback)
+{
+ bcm47xx_fill_sprom_ethernet(sprom, prefix, fallback);
+ bcm47xx_fill_board_data(sprom, prefix, fallback);
+
+ nvram_read_u8(prefix, NULL, "sromrev", &sprom->revision, 0, fallback);
+
+ /* Entries requiring custom functions */
+ nvram_read_alpha2(prefix, "ccode", sprom->alpha2, fallback);
+ if (sprom->revision >= 3)
+ nvram_read_leddc(prefix, "leddc", &sprom->leddc_on_time,
+ &sprom->leddc_off_time, fallback);
+
+ switch (sprom->revision) {
+ case 4:
+ case 5:
+ bcm47xx_fill_sprom_path_r4589(sprom, prefix, fallback);
+ bcm47xx_fill_sprom_path_r45(sprom, prefix, fallback);
+ break;
+ case 8:
+ case 9:
+ bcm47xx_fill_sprom_path_r4589(sprom, prefix, fallback);
+ break;
+ }
+
+ bcm47xx_sprom_fill_auto(sprom, prefix, fallback);
+}
+
+#if IS_BUILTIN(CONFIG_SSB) && IS_ENABLED(CONFIG_SSB_SPROM)
+static int bcm47xx_get_sprom_ssb(struct ssb_bus *bus, struct ssb_sprom *out)
+{
+ char prefix[10];
+
+ switch (bus->bustype) {
+ case SSB_BUSTYPE_SSB:
+ bcm47xx_fill_sprom(out, NULL, false);
+ return 0;
+ case SSB_BUSTYPE_PCI:
+ memset(out, 0, sizeof(struct ssb_sprom));
+ snprintf(prefix, sizeof(prefix), "pci/%u/%u/",
+ bus->host_pci->bus->number + 1,
+ PCI_SLOT(bus->host_pci->devfn));
+ bcm47xx_fill_sprom(out, prefix, false);
+ return 0;
+ default:
+ pr_warn("Unable to fill SPROM for given bustype.\n");
+ return -EINVAL;
+ }
+}
+#endif
+
+#if IS_BUILTIN(CONFIG_BCMA)
+/*
+ * Having many NVRAM entries for PCI devices led to repeating prefixes like
+ * pci/1/1/ all the time and wasting flash space. So at some point Broadcom
+ * decided to introduce prefixes like 0: 1: 2: etc.
+ * If we find e.g. devpath0=pci/2/1 or devpath0=pci/2/1/ we should use 0:
+ * instead of pci/2/1/.
+ */
+static void bcm47xx_sprom_apply_prefix_alias(char *prefix, size_t prefix_size)
+{
+ size_t prefix_len = strlen(prefix);
+ size_t short_len = prefix_len - 1;
+ char nvram_var[10];
+ char buf[20];
+ int i;
+
+ /* Passed prefix has to end with a slash */
+ if (prefix_len <= 0 || prefix[prefix_len - 1] != '/')
+ return;
+
+ for (i = 0; i < 3; i++) {
+ if (snprintf(nvram_var, sizeof(nvram_var), "devpath%d", i) <= 0)
+ continue;
+ if (bcm47xx_nvram_getenv(nvram_var, buf, sizeof(buf)) < 0)
+ continue;
+ if (!strcmp(buf, prefix) ||
+ (short_len && strlen(buf) == short_len && !strncmp(buf, prefix, short_len))) {
+ snprintf(prefix, prefix_size, "%d:", i);
+ return;
+ }
+ }
+}
+
+static int bcm47xx_get_sprom_bcma(struct bcma_bus *bus, struct ssb_sprom *out)
+{
+ struct bcma_boardinfo *binfo = &bus->boardinfo;
+ struct bcma_device *core;
+ char buf[10];
+ char *prefix;
+ bool fallback = false;
+
+ switch (bus->hosttype) {
+ case BCMA_HOSTTYPE_PCI:
+ memset(out, 0, sizeof(struct ssb_sprom));
+ /* On BCM47XX all PCI buses share the same domain */
+ if (config_enabled(CONFIG_BCM47XX))
+ snprintf(buf, sizeof(buf), "pci/%u/%u/",
+ bus->host_pci->bus->number + 1,
+ PCI_SLOT(bus->host_pci->devfn));
+ else
+ snprintf(buf, sizeof(buf), "pci/%u/%u/",
+ pci_domain_nr(bus->host_pci->bus) + 1,
+ bus->host_pci->bus->number);
+ bcm47xx_sprom_apply_prefix_alias(buf, sizeof(buf));
+ prefix = buf;
+ break;
+ case BCMA_HOSTTYPE_SOC:
+ memset(out, 0, sizeof(struct ssb_sprom));
+ core = bcma_find_core(bus, BCMA_CORE_80211);
+ if (core) {
+ snprintf(buf, sizeof(buf), "sb/%u/",
+ core->core_index);
+ prefix = buf;
+ fallback = true;
+ } else {
+ prefix = NULL;
+ }
+ break;
+ default:
+ pr_warn("Unable to fill SPROM for given bustype.\n");
+ return -EINVAL;
+ }
+
+ nvram_read_u16(prefix, NULL, "boardvendor", &binfo->vendor, 0, true);
+ if (!binfo->vendor)
+ binfo->vendor = SSB_BOARDVENDOR_BCM;
+ nvram_read_u16(prefix, NULL, "boardtype", &binfo->type, 0, true);
+
+ bcm47xx_fill_sprom(out, prefix, fallback);
+
+ return 0;
+}
+#endif
+
+static unsigned int bcm47xx_sprom_registered;
+
+/*
+ * On bcm47xx we need to register SPROM fallback handler very early, so we can't
+ * use anything like platform device / driver for this.
+ */
+int bcm47xx_sprom_register_fallbacks(void)
+{
+ if (bcm47xx_sprom_registered)
+ return 0;
+
+#if IS_BUILTIN(CONFIG_SSB) && IS_ENABLED(CONFIG_SSB_SPROM)
+ if (ssb_arch_register_fallback_sprom(&bcm47xx_get_sprom_ssb))
+ pr_warn("Failed to register ssb SPROM handler\n");
+#endif
+
+#if IS_BUILTIN(CONFIG_BCMA)
+ if (bcma_arch_register_fallback_sprom(&bcm47xx_get_sprom_bcma))
+ pr_warn("Failed to register bcma SPROM handler\n");
+#endif
+
+ bcm47xx_sprom_registered = 1;
+
+ return 0;
+}
+
+fs_initcall(bcm47xx_sprom_register_fallbacks);
diff --git a/drivers/firmware/iscsi_ibft.c b/drivers/firmware/iscsi_ibft.c
index 81037e5..14042a6 100644
--- a/drivers/firmware/iscsi_ibft.c
+++ b/drivers/firmware/iscsi_ibft.c
@@ -418,6 +418,31 @@
return str - buf;
}
+static ssize_t ibft_attr_show_acpitbl(void *data, int type, char *buf)
+{
+ struct ibft_kobject *entry = data;
+ char *str = buf;
+
+ switch (type) {
+ case ISCSI_BOOT_ACPITBL_SIGNATURE:
+ str += sprintf_string(str, ACPI_NAME_SIZE,
+ entry->header->header.signature);
+ break;
+ case ISCSI_BOOT_ACPITBL_OEM_ID:
+ str += sprintf_string(str, ACPI_OEM_ID_SIZE,
+ entry->header->header.oem_id);
+ break;
+ case ISCSI_BOOT_ACPITBL_OEM_TABLE_ID:
+ str += sprintf_string(str, ACPI_OEM_TABLE_ID_SIZE,
+ entry->header->header.oem_table_id);
+ break;
+ default:
+ break;
+ }
+
+ return str - buf;
+}
+
static int __init ibft_check_device(void)
{
int len;
@@ -576,6 +601,24 @@
return rc;
}
+static umode_t __init ibft_check_acpitbl_for(void *data, int type)
+{
+
+ umode_t rc = 0;
+
+ switch (type) {
+ case ISCSI_BOOT_ACPITBL_SIGNATURE:
+ case ISCSI_BOOT_ACPITBL_OEM_ID:
+ case ISCSI_BOOT_ACPITBL_OEM_TABLE_ID:
+ rc = S_IRUGO;
+ break;
+ default:
+ break;
+ }
+
+ return rc;
+}
+
static void ibft_kobj_release(void *data)
{
kfree(data);
@@ -699,6 +742,8 @@
static int __init ibft_register_kobjects(struct acpi_table_ibft *header)
{
struct ibft_control *control = NULL;
+ struct iscsi_boot_kobj *boot_kobj;
+ struct ibft_kobject *ibft_kobj;
void *ptr, *end;
int rc = 0;
u16 offset;
@@ -726,6 +771,25 @@
break;
}
}
+ if (rc)
+ return rc;
+
+ ibft_kobj = kzalloc(sizeof(*ibft_kobj), GFP_KERNEL);
+ if (!ibft_kobj)
+ return -ENOMEM;
+
+ ibft_kobj->header = header;
+ ibft_kobj->hdr = NULL; /*for ibft_unregister*/
+
+ boot_kobj = iscsi_boot_create_acpitbl(boot_kset, 0,
+ ibft_kobj,
+ ibft_attr_show_acpitbl,
+ ibft_check_acpitbl_for,
+ ibft_kobj_release);
+ if (!boot_kobj) {
+ kfree(ibft_kobj);
+ rc = -ENOMEM;
+ }
return rc;
}
@@ -738,7 +802,7 @@
list_for_each_entry_safe(boot_kobj, tmp_kobj,
&boot_kset->kobj_list, list) {
ibft_kobj = boot_kobj->data;
- if (ibft_kobj->hdr->id == id_nic)
+ if (ibft_kobj->hdr && ibft_kobj->hdr->id == id_nic)
sysfs_remove_link(&boot_kobj->kobj, "device");
};
}
diff --git a/drivers/firmware/qemu_fw_cfg.c b/drivers/firmware/qemu_fw_cfg.c
index 1b95475..0e20116 100644
--- a/drivers/firmware/qemu_fw_cfg.c
+++ b/drivers/firmware/qemu_fw_cfg.c
@@ -125,9 +125,7 @@
# define FW_CFG_CTRL_OFF 0x00
# define FW_CFG_DATA_OFF 0x01
# else
-# warning "QEMU FW_CFG may not be available on this architecture!"
-# define FW_CFG_CTRL_OFF 0x00
-# define FW_CFG_DATA_OFF 0x01
+# error "QEMU FW_CFG not available on this architecture!"
# endif
#endif
diff --git a/drivers/gpio/Kconfig b/drivers/gpio/Kconfig
index d00e7b6..48da857 100644
--- a/drivers/gpio/Kconfig
+++ b/drivers/gpio/Kconfig
@@ -530,30 +530,35 @@
config GPIO_104_DIO_48E
tristate "ACCES 104-DIO-48E GPIO support"
+ depends on ISA
select GPIOLIB_IRQCHIP
help
- Enables GPIO support for the ACCES 104-DIO-48E family. The base port
- address for the device may be configured via the dio_48e_base module
- parameter. The interrupt line number for the device may be configured
- via the dio_48e_irq module parameter.
+ Enables GPIO support for the ACCES 104-DIO-48E series (104-DIO-48E,
+ 104-DIO-24E). The base port addresses for the devices may be
+ configured via the base module parameter. The interrupt line numbers
+ for the devices may be configured via the irq module parameter.
config GPIO_104_IDIO_16
tristate "ACCES 104-IDIO-16 GPIO support"
+ depends on ISA
select GPIOLIB_IRQCHIP
help
- Enables GPIO support for the ACCES 104-IDIO-16 family. The base port
- address for the device may be set via the idio_16_base module
- parameter. The interrupt line number for the device may be set via the
- idio_16_irq module parameter.
+ Enables GPIO support for the ACCES 104-IDIO-16 family (104-IDIO-16,
+ 104-IDIO-16E, 104-IDO-16, 104-IDIO-8, 104-IDIO-8E, 104-IDO-8). The
+ base port addresses for the devices may be configured via the base
+ module parameter. The interrupt line numbers for the devices may be
+ configured via the irq module parameter.
config GPIO_104_IDI_48
tristate "ACCES 104-IDI-48 GPIO support"
+ depends on ISA
select GPIOLIB_IRQCHIP
help
- Enables GPIO support for the ACCES 104-IDI-48 family. The base port
- address for the device may be configured via the idi_48_base module
- parameter. The interrupt line number for the device may be configured
- via the idi_48_irq module parameter.
+ Enables GPIO support for the ACCES 104-IDI-48 family (104-IDI-48A,
+ 104-IDI-48AC, 104-IDI-48B, 104-IDI-48BC). The base port addresses for
+ the devices may be configured via the base module parameter. The
+ interrupt line numbers for the devices may be configured via the irq
+ module parameter.
config GPIO_F7188X
tristate "F71869, F71869A, F71882FG, F71889F and F81866 GPIO support"
@@ -622,12 +627,13 @@
config GPIO_WS16C48
tristate "WinSystems WS16C48 GPIO support"
+ depends on ISA
select GPIOLIB_IRQCHIP
help
- Enables GPIO support for the WinSystems WS16C48. The base port address
- for the device may be configured via the ws16c48_base module
- parameter. The interrupt line number for the device may be configured
- via the ws16c48_irq module parameter.
+ Enables GPIO support for the WinSystems WS16C48. The base port
+ addresses for the devices may be configured via the base module
+ parameter. The interrupt line numbers for the devices may be
+ configured via the irq module parameter.
endmenu
diff --git a/drivers/gpio/gpio-104-dio-48e.c b/drivers/gpio/gpio-104-dio-48e.c
index 448a903..1a647c0 100644
--- a/drivers/gpio/gpio-104-dio-48e.c
+++ b/drivers/gpio/gpio-104-dio-48e.c
@@ -1,5 +1,5 @@
/*
- * GPIO driver for the ACCES 104-DIO-48E
+ * GPIO driver for the ACCES 104-DIO-48E series
* Copyright (C) 2016 William Breathitt Gray
*
* This program is free software; you can redistribute it and/or modify
@@ -10,6 +10,9 @@
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* General Public License for more details.
+ *
+ * This driver supports the following ACCES devices: 104-DIO-48E and
+ * 104-DIO-24E.
*/
#include <linux/bitops.h>
#include <linux/device.h>
@@ -19,18 +22,23 @@
#include <linux/ioport.h>
#include <linux/interrupt.h>
#include <linux/irqdesc.h>
+#include <linux/isa.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
-#include <linux/platform_device.h>
#include <linux/spinlock.h>
-static unsigned dio_48e_base;
-module_param(dio_48e_base, uint, 0);
-MODULE_PARM_DESC(dio_48e_base, "ACCES 104-DIO-48E base address");
-static unsigned dio_48e_irq;
-module_param(dio_48e_irq, uint, 0);
-MODULE_PARM_DESC(dio_48e_irq, "ACCES 104-DIO-48E interrupt line number");
+#define DIO48E_EXTENT 16
+#define MAX_NUM_DIO48E max_num_isa_dev(DIO48E_EXTENT)
+
+static unsigned int base[MAX_NUM_DIO48E];
+static unsigned int num_dio48e;
+module_param_array(base, uint, &num_dio48e, 0);
+MODULE_PARM_DESC(base, "ACCES 104-DIO-48E base addresses");
+
+static unsigned int irq[MAX_NUM_DIO48E];
+module_param_array(irq, uint, NULL, 0);
+MODULE_PARM_DESC(irq, "ACCES 104-DIO-48E interrupt line numbers");
/**
* struct dio48e_gpio - GPIO device private data structure
@@ -294,23 +302,19 @@
return IRQ_HANDLED;
}
-static int __init dio48e_probe(struct platform_device *pdev)
+static int dio48e_probe(struct device *dev, unsigned int id)
{
- struct device *dev = &pdev->dev;
struct dio48e_gpio *dio48egpio;
- const unsigned base = dio_48e_base;
- const unsigned extent = 16;
const char *const name = dev_name(dev);
int err;
- const unsigned irq = dio_48e_irq;
dio48egpio = devm_kzalloc(dev, sizeof(*dio48egpio), GFP_KERNEL);
if (!dio48egpio)
return -ENOMEM;
- if (!devm_request_region(dev, base, extent, name)) {
+ if (!devm_request_region(dev, base[id], DIO48E_EXTENT, name)) {
dev_err(dev, "Unable to lock port addresses (0x%X-0x%X)\n",
- base, base + extent);
+ base[id], base[id] + DIO48E_EXTENT);
return -EBUSY;
}
@@ -324,8 +328,8 @@
dio48egpio->chip.direction_output = dio48e_gpio_direction_output;
dio48egpio->chip.get = dio48e_gpio_get;
dio48egpio->chip.set = dio48e_gpio_set;
- dio48egpio->base = base;
- dio48egpio->irq = irq;
+ dio48egpio->base = base[id];
+ dio48egpio->irq = irq[id];
spin_lock_init(&dio48egpio->lock);
@@ -338,19 +342,19 @@
}
/* initialize all GPIO as output */
- outb(0x80, base + 3);
- outb(0x00, base);
- outb(0x00, base + 1);
- outb(0x00, base + 2);
- outb(0x00, base + 3);
- outb(0x80, base + 7);
- outb(0x00, base + 4);
- outb(0x00, base + 5);
- outb(0x00, base + 6);
- outb(0x00, base + 7);
+ outb(0x80, base[id] + 3);
+ outb(0x00, base[id]);
+ outb(0x00, base[id] + 1);
+ outb(0x00, base[id] + 2);
+ outb(0x00, base[id] + 3);
+ outb(0x80, base[id] + 7);
+ outb(0x00, base[id] + 4);
+ outb(0x00, base[id] + 5);
+ outb(0x00, base[id] + 6);
+ outb(0x00, base[id] + 7);
/* disable IRQ by default */
- inb(base + 0xB);
+ inb(base[id] + 0xB);
err = gpiochip_irqchip_add(&dio48egpio->chip, &dio48e_irqchip, 0,
handle_edge_irq, IRQ_TYPE_NONE);
@@ -359,7 +363,7 @@
goto err_gpiochip_remove;
}
- err = request_irq(irq, dio48e_irq_handler, 0, name, dio48egpio);
+ err = request_irq(irq[id], dio48e_irq_handler, 0, name, dio48egpio);
if (err) {
dev_err(dev, "IRQ handler registering failed (%d)\n", err);
goto err_gpiochip_remove;
@@ -372,9 +376,9 @@
return err;
}
-static int dio48e_remove(struct platform_device *pdev)
+static int dio48e_remove(struct device *dev, unsigned int id)
{
- struct dio48e_gpio *const dio48egpio = platform_get_drvdata(pdev);
+ struct dio48e_gpio *const dio48egpio = dev_get_drvdata(dev);
free_irq(dio48egpio->irq, dio48egpio);
gpiochip_remove(&dio48egpio->chip);
@@ -382,48 +386,14 @@
return 0;
}
-static struct platform_device *dio48e_device;
-
-static struct platform_driver dio48e_driver = {
+static struct isa_driver dio48e_driver = {
+ .probe = dio48e_probe,
.driver = {
.name = "104-dio-48e"
},
.remove = dio48e_remove
};
-
-static void __exit dio48e_exit(void)
-{
- platform_device_unregister(dio48e_device);
- platform_driver_unregister(&dio48e_driver);
-}
-
-static int __init dio48e_init(void)
-{
- int err;
-
- dio48e_device = platform_device_alloc(dio48e_driver.driver.name, -1);
- if (!dio48e_device)
- return -ENOMEM;
-
- err = platform_device_add(dio48e_device);
- if (err)
- goto err_platform_device;
-
- err = platform_driver_probe(&dio48e_driver, dio48e_probe);
- if (err)
- goto err_platform_driver;
-
- return 0;
-
-err_platform_driver:
- platform_device_del(dio48e_device);
-err_platform_device:
- platform_device_put(dio48e_device);
- return err;
-}
-
-module_init(dio48e_init);
-module_exit(dio48e_exit);
+module_isa_driver(dio48e_driver, num_dio48e);
MODULE_AUTHOR("William Breathitt Gray <vilhelm.gray@gmail.com>");
MODULE_DESCRIPTION("ACCES 104-DIO-48E GPIO driver");
diff --git a/drivers/gpio/gpio-104-idi-48.c b/drivers/gpio/gpio-104-idi-48.c
index e37cd4c..6c75c83 100644
--- a/drivers/gpio/gpio-104-idi-48.c
+++ b/drivers/gpio/gpio-104-idi-48.c
@@ -10,6 +10,9 @@
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* General Public License for more details.
+ *
+ * This driver supports the following ACCES devices: 104-IDI-48A,
+ * 104-IDI-48AC, 104-IDI-48B, and 104-IDI-48BC.
*/
#include <linux/bitops.h>
#include <linux/device.h>
@@ -19,18 +22,23 @@
#include <linux/ioport.h>
#include <linux/interrupt.h>
#include <linux/irqdesc.h>
+#include <linux/isa.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
-#include <linux/platform_device.h>
#include <linux/spinlock.h>
-static unsigned idi_48_base;
-module_param(idi_48_base, uint, 0);
-MODULE_PARM_DESC(idi_48_base, "ACCES 104-IDI-48 base address");
-static unsigned idi_48_irq;
-module_param(idi_48_irq, uint, 0);
-MODULE_PARM_DESC(idi_48_irq, "ACCES 104-IDI-48 interrupt line number");
+#define IDI_48_EXTENT 8
+#define MAX_NUM_IDI_48 max_num_isa_dev(IDI_48_EXTENT)
+
+static unsigned int base[MAX_NUM_IDI_48];
+static unsigned int num_idi_48;
+module_param_array(base, uint, &num_idi_48, 0);
+MODULE_PARM_DESC(base, "ACCES 104-IDI-48 base addresses");
+
+static unsigned int irq[MAX_NUM_IDI_48];
+module_param_array(irq, uint, NULL, 0);
+MODULE_PARM_DESC(irq, "ACCES 104-IDI-48 interrupt line numbers");
/**
* struct idi_48_gpio - GPIO device private data structure
@@ -211,23 +219,19 @@
return IRQ_HANDLED;
}
-static int __init idi_48_probe(struct platform_device *pdev)
+static int idi_48_probe(struct device *dev, unsigned int id)
{
- struct device *dev = &pdev->dev;
struct idi_48_gpio *idi48gpio;
- const unsigned base = idi_48_base;
- const unsigned extent = 8;
const char *const name = dev_name(dev);
int err;
- const unsigned irq = idi_48_irq;
idi48gpio = devm_kzalloc(dev, sizeof(*idi48gpio), GFP_KERNEL);
if (!idi48gpio)
return -ENOMEM;
- if (!devm_request_region(dev, base, extent, name)) {
+ if (!devm_request_region(dev, base[id], IDI_48_EXTENT, name)) {
dev_err(dev, "Unable to lock port addresses (0x%X-0x%X)\n",
- base, base + extent);
+ base[id], base[id] + IDI_48_EXTENT);
return -EBUSY;
}
@@ -239,8 +243,8 @@
idi48gpio->chip.get_direction = idi_48_gpio_get_direction;
idi48gpio->chip.direction_input = idi_48_gpio_direction_input;
idi48gpio->chip.get = idi_48_gpio_get;
- idi48gpio->base = base;
- idi48gpio->irq = irq;
+ idi48gpio->base = base[id];
+ idi48gpio->irq = irq[id];
spin_lock_init(&idi48gpio->lock);
@@ -253,8 +257,8 @@
}
/* Disable IRQ by default */
- outb(0, base + 7);
- inb(base + 7);
+ outb(0, base[id] + 7);
+ inb(base[id] + 7);
err = gpiochip_irqchip_add(&idi48gpio->chip, &idi_48_irqchip, 0,
handle_edge_irq, IRQ_TYPE_NONE);
@@ -263,7 +267,7 @@
goto err_gpiochip_remove;
}
- err = request_irq(irq, idi_48_irq_handler, IRQF_SHARED, name,
+ err = request_irq(irq[id], idi_48_irq_handler, IRQF_SHARED, name,
idi48gpio);
if (err) {
dev_err(dev, "IRQ handler registering failed (%d)\n", err);
@@ -277,9 +281,9 @@
return err;
}
-static int idi_48_remove(struct platform_device *pdev)
+static int idi_48_remove(struct device *dev, unsigned int id)
{
- struct idi_48_gpio *const idi48gpio = platform_get_drvdata(pdev);
+ struct idi_48_gpio *const idi48gpio = dev_get_drvdata(dev);
free_irq(idi48gpio->irq, idi48gpio);
gpiochip_remove(&idi48gpio->chip);
@@ -287,48 +291,14 @@
return 0;
}
-static struct platform_device *idi_48_device;
-
-static struct platform_driver idi_48_driver = {
+static struct isa_driver idi_48_driver = {
+ .probe = idi_48_probe,
.driver = {
.name = "104-idi-48"
},
.remove = idi_48_remove
};
-
-static void __exit idi_48_exit(void)
-{
- platform_device_unregister(idi_48_device);
- platform_driver_unregister(&idi_48_driver);
-}
-
-static int __init idi_48_init(void)
-{
- int err;
-
- idi_48_device = platform_device_alloc(idi_48_driver.driver.name, -1);
- if (!idi_48_device)
- return -ENOMEM;
-
- err = platform_device_add(idi_48_device);
- if (err)
- goto err_platform_device;
-
- err = platform_driver_probe(&idi_48_driver, idi_48_probe);
- if (err)
- goto err_platform_driver;
-
- return 0;
-
-err_platform_driver:
- platform_device_del(idi_48_device);
-err_platform_device:
- platform_device_put(idi_48_device);
- return err;
-}
-
-module_init(idi_48_init);
-module_exit(idi_48_exit);
+module_isa_driver(idi_48_driver, num_idi_48);
MODULE_AUTHOR("William Breathitt Gray <vilhelm.gray@gmail.com>");
MODULE_DESCRIPTION("ACCES 104-IDI-48 GPIO driver");
diff --git a/drivers/gpio/gpio-104-idio-16.c b/drivers/gpio/gpio-104-idio-16.c
index ecc85fe..6787b8f 100644
--- a/drivers/gpio/gpio-104-idio-16.c
+++ b/drivers/gpio/gpio-104-idio-16.c
@@ -10,6 +10,9 @@
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* General Public License for more details.
+ *
+ * This driver supports the following ACCES devices: 104-IDIO-16,
+ * 104-IDIO-16E, 104-IDO-16, 104-IDIO-8, 104-IDIO-8E, and 104-IDO-8.
*/
#include <linux/bitops.h>
#include <linux/device.h>
@@ -19,18 +22,23 @@
#include <linux/ioport.h>
#include <linux/interrupt.h>
#include <linux/irqdesc.h>
+#include <linux/isa.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
-#include <linux/platform_device.h>
#include <linux/spinlock.h>
-static unsigned idio_16_base;
-module_param(idio_16_base, uint, 0);
-MODULE_PARM_DESC(idio_16_base, "ACCES 104-IDIO-16 base address");
-static unsigned idio_16_irq;
-module_param(idio_16_irq, uint, 0);
-MODULE_PARM_DESC(idio_16_irq, "ACCES 104-IDIO-16 interrupt line number");
+#define IDIO_16_EXTENT 8
+#define MAX_NUM_IDIO_16 max_num_isa_dev(IDIO_16_EXTENT)
+
+static unsigned int base[MAX_NUM_IDIO_16];
+static unsigned int num_idio_16;
+module_param_array(base, uint, &num_idio_16, 0);
+MODULE_PARM_DESC(base, "ACCES 104-IDIO-16 base addresses");
+
+static unsigned int irq[MAX_NUM_IDIO_16];
+module_param_array(irq, uint, NULL, 0);
+MODULE_PARM_DESC(irq, "ACCES 104-IDIO-16 interrupt line numbers");
/**
* struct idio_16_gpio - GPIO device private data structure
@@ -185,23 +193,19 @@
return IRQ_HANDLED;
}
-static int __init idio_16_probe(struct platform_device *pdev)
+static int idio_16_probe(struct device *dev, unsigned int id)
{
- struct device *dev = &pdev->dev;
struct idio_16_gpio *idio16gpio;
- const unsigned base = idio_16_base;
- const unsigned extent = 8;
const char *const name = dev_name(dev);
int err;
- const unsigned irq = idio_16_irq;
idio16gpio = devm_kzalloc(dev, sizeof(*idio16gpio), GFP_KERNEL);
if (!idio16gpio)
return -ENOMEM;
- if (!devm_request_region(dev, base, extent, name)) {
+ if (!devm_request_region(dev, base[id], IDIO_16_EXTENT, name)) {
dev_err(dev, "Unable to lock port addresses (0x%X-0x%X)\n",
- base, base + extent);
+ base[id], base[id] + IDIO_16_EXTENT);
return -EBUSY;
}
@@ -215,8 +219,8 @@
idio16gpio->chip.direction_output = idio_16_gpio_direction_output;
idio16gpio->chip.get = idio_16_gpio_get;
idio16gpio->chip.set = idio_16_gpio_set;
- idio16gpio->base = base;
- idio16gpio->irq = irq;
+ idio16gpio->base = base[id];
+ idio16gpio->irq = irq[id];
idio16gpio->out_state = 0xFFFF;
spin_lock_init(&idio16gpio->lock);
@@ -230,8 +234,8 @@
}
/* Disable IRQ by default */
- outb(0, base + 2);
- outb(0, base + 1);
+ outb(0, base[id] + 2);
+ outb(0, base[id] + 1);
err = gpiochip_irqchip_add(&idio16gpio->chip, &idio_16_irqchip, 0,
handle_edge_irq, IRQ_TYPE_NONE);
@@ -240,7 +244,7 @@
goto err_gpiochip_remove;
}
- err = request_irq(irq, idio_16_irq_handler, 0, name, idio16gpio);
+ err = request_irq(irq[id], idio_16_irq_handler, 0, name, idio16gpio);
if (err) {
dev_err(dev, "IRQ handler registering failed (%d)\n", err);
goto err_gpiochip_remove;
@@ -253,9 +257,9 @@
return err;
}
-static int idio_16_remove(struct platform_device *pdev)
+static int idio_16_remove(struct device *dev, unsigned int id)
{
- struct idio_16_gpio *const idio16gpio = platform_get_drvdata(pdev);
+ struct idio_16_gpio *const idio16gpio = dev_get_drvdata(dev);
free_irq(idio16gpio->irq, idio16gpio);
gpiochip_remove(&idio16gpio->chip);
@@ -263,48 +267,15 @@
return 0;
}
-static struct platform_device *idio_16_device;
-
-static struct platform_driver idio_16_driver = {
+static struct isa_driver idio_16_driver = {
+ .probe = idio_16_probe,
.driver = {
.name = "104-idio-16"
},
.remove = idio_16_remove
};
-static void __exit idio_16_exit(void)
-{
- platform_device_unregister(idio_16_device);
- platform_driver_unregister(&idio_16_driver);
-}
-
-static int __init idio_16_init(void)
-{
- int err;
-
- idio_16_device = platform_device_alloc(idio_16_driver.driver.name, -1);
- if (!idio_16_device)
- return -ENOMEM;
-
- err = platform_device_add(idio_16_device);
- if (err)
- goto err_platform_device;
-
- err = platform_driver_probe(&idio_16_driver, idio_16_probe);
- if (err)
- goto err_platform_driver;
-
- return 0;
-
-err_platform_driver:
- platform_device_del(idio_16_device);
-err_platform_device:
- platform_device_put(idio_16_device);
- return err;
-}
-
-module_init(idio_16_init);
-module_exit(idio_16_exit);
+module_isa_driver(idio_16_driver, num_idio_16);
MODULE_AUTHOR("William Breathitt Gray <vilhelm.gray@gmail.com>");
MODULE_DESCRIPTION("ACCES 104-IDIO-16 GPIO driver");
diff --git a/drivers/gpio/gpio-ws16c48.c b/drivers/gpio/gpio-ws16c48.c
index 51f41e8..eaa71d4 100644
--- a/drivers/gpio/gpio-ws16c48.c
+++ b/drivers/gpio/gpio-ws16c48.c
@@ -19,18 +19,23 @@
#include <linux/ioport.h>
#include <linux/interrupt.h>
#include <linux/irqdesc.h>
+#include <linux/isa.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
-#include <linux/platform_device.h>
#include <linux/spinlock.h>
-static unsigned ws16c48_base;
-module_param(ws16c48_base, uint, 0);
-MODULE_PARM_DESC(ws16c48_base, "WinSystems WS16C48 base address");
-static unsigned ws16c48_irq;
-module_param(ws16c48_irq, uint, 0);
-MODULE_PARM_DESC(ws16c48_irq, "WinSystems WS16C48 interrupt line number");
+#define WS16C48_EXTENT 16
+#define MAX_NUM_WS16C48 max_num_isa_dev(WS16C48_EXTENT)
+
+static unsigned int base[MAX_NUM_WS16C48];
+static unsigned int num_ws16c48;
+module_param_array(base, uint, &num_ws16c48, 0);
+MODULE_PARM_DESC(base, "WinSystems WS16C48 base addresses");
+
+static unsigned int irq[MAX_NUM_WS16C48];
+module_param_array(irq, uint, NULL, 0);
+MODULE_PARM_DESC(irq, "WinSystems WS16C48 interrupt line numbers");
/**
* struct ws16c48_gpio - GPIO device private data structure
@@ -298,23 +303,19 @@
return IRQ_HANDLED;
}
-static int __init ws16c48_probe(struct platform_device *pdev)
+static int ws16c48_probe(struct device *dev, unsigned int id)
{
- struct device *dev = &pdev->dev;
struct ws16c48_gpio *ws16c48gpio;
- const unsigned base = ws16c48_base;
- const unsigned extent = 16;
const char *const name = dev_name(dev);
int err;
- const unsigned irq = ws16c48_irq;
ws16c48gpio = devm_kzalloc(dev, sizeof(*ws16c48gpio), GFP_KERNEL);
if (!ws16c48gpio)
return -ENOMEM;
- if (!devm_request_region(dev, base, extent, name)) {
+ if (!devm_request_region(dev, base[id], WS16C48_EXTENT, name)) {
dev_err(dev, "Unable to lock port addresses (0x%X-0x%X)\n",
- base, base + extent);
+ base[id], base[id] + WS16C48_EXTENT);
return -EBUSY;
}
@@ -328,8 +329,8 @@
ws16c48gpio->chip.direction_output = ws16c48_gpio_direction_output;
ws16c48gpio->chip.get = ws16c48_gpio_get;
ws16c48gpio->chip.set = ws16c48_gpio_set;
- ws16c48gpio->base = base;
- ws16c48gpio->irq = irq;
+ ws16c48gpio->base = base[id];
+ ws16c48gpio->irq = irq[id];
spin_lock_init(&ws16c48gpio->lock);
@@ -342,11 +343,11 @@
}
/* Disable IRQ by default */
- outb(0x80, base + 7);
- outb(0, base + 8);
- outb(0, base + 9);
- outb(0, base + 10);
- outb(0xC0, base + 7);
+ outb(0x80, base[id] + 7);
+ outb(0, base[id] + 8);
+ outb(0, base[id] + 9);
+ outb(0, base[id] + 10);
+ outb(0xC0, base[id] + 7);
err = gpiochip_irqchip_add(&ws16c48gpio->chip, &ws16c48_irqchip, 0,
handle_edge_irq, IRQ_TYPE_NONE);
@@ -355,7 +356,7 @@
goto err_gpiochip_remove;
}
- err = request_irq(irq, ws16c48_irq_handler, IRQF_SHARED, name,
+ err = request_irq(irq[id], ws16c48_irq_handler, IRQF_SHARED, name,
ws16c48gpio);
if (err) {
dev_err(dev, "IRQ handler registering failed (%d)\n", err);
@@ -369,9 +370,9 @@
return err;
}
-static int ws16c48_remove(struct platform_device *pdev)
+static int ws16c48_remove(struct device *dev, unsigned int id)
{
- struct ws16c48_gpio *const ws16c48gpio = platform_get_drvdata(pdev);
+ struct ws16c48_gpio *const ws16c48gpio = dev_get_drvdata(dev);
free_irq(ws16c48gpio->irq, ws16c48gpio);
gpiochip_remove(&ws16c48gpio->chip);
@@ -379,48 +380,15 @@
return 0;
}
-static struct platform_device *ws16c48_device;
-
-static struct platform_driver ws16c48_driver = {
+static struct isa_driver ws16c48_driver = {
+ .probe = ws16c48_probe,
.driver = {
.name = "ws16c48"
},
.remove = ws16c48_remove
};
-static void __exit ws16c48_exit(void)
-{
- platform_device_unregister(ws16c48_device);
- platform_driver_unregister(&ws16c48_driver);
-}
-
-static int __init ws16c48_init(void)
-{
- int err;
-
- ws16c48_device = platform_device_alloc(ws16c48_driver.driver.name, -1);
- if (!ws16c48_device)
- return -ENOMEM;
-
- err = platform_device_add(ws16c48_device);
- if (err)
- goto err_platform_device;
-
- err = platform_driver_probe(&ws16c48_driver, ws16c48_probe);
- if (err)
- goto err_platform_driver;
-
- return 0;
-
-err_platform_driver:
- platform_device_del(ws16c48_device);
-err_platform_device:
- platform_device_put(ws16c48_device);
- return err;
-}
-
-module_init(ws16c48_init);
-module_exit(ws16c48_exit);
+module_isa_driver(ws16c48_driver, num_ws16c48);
MODULE_AUTHOR("William Breathitt Gray <vilhelm.gray@gmail.com>");
MODULE_DESCRIPTION("WinSystems WS16C48 GPIO driver");
diff --git a/drivers/gpu/drm/Kconfig b/drivers/gpu/drm/Kconfig
index f2a74d0..fc35731 100644
--- a/drivers/gpu/drm/Kconfig
+++ b/drivers/gpu/drm/Kconfig
@@ -52,6 +52,7 @@
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
+ select FB_DEFERRED_IO
help
FBDEV helpers for KMS drivers.
@@ -252,6 +253,8 @@
source "drivers/gpu/drm/shmobile/Kconfig"
+source "drivers/gpu/drm/sun4i/Kconfig"
+
source "drivers/gpu/drm/omapdrm/Kconfig"
source "drivers/gpu/drm/tilcdc/Kconfig"
@@ -281,3 +284,9 @@
source "drivers/gpu/drm/vc4/Kconfig"
source "drivers/gpu/drm/etnaviv/Kconfig"
+
+source "drivers/gpu/drm/arc/Kconfig"
+
+source "drivers/gpu/drm/hisilicon/Kconfig"
+
+source "drivers/gpu/drm/mediatek/Kconfig"
diff --git a/drivers/gpu/drm/Makefile b/drivers/gpu/drm/Makefile
index 6eb94fc..2bd3e5a 100644
--- a/drivers/gpu/drm/Makefile
+++ b/drivers/gpu/drm/Makefile
@@ -1,4 +1,4 @@
-#
+
# Makefile for the drm device driver. This driver provides support for the
# Direct Rendering Infrastructure (DRI) in XFree86 4.1.0 and higher.
@@ -65,6 +65,7 @@
obj-$(CONFIG_DRM_RCAR_DU) += rcar-du/
obj-$(CONFIG_DRM_SHMOBILE) +=shmobile/
obj-y += omapdrm/
+obj-$(CONFIG_DRM_SUN4I) += sun4i/
obj-y += tilcdc/
obj-$(CONFIG_DRM_QXL) += qxl/
obj-$(CONFIG_DRM_BOCHS) += bochs/
@@ -73,8 +74,11 @@
obj-$(CONFIG_DRM_TEGRA) += tegra/
obj-$(CONFIG_DRM_STI) += sti/
obj-$(CONFIG_DRM_IMX) += imx/
+obj-$(CONFIG_DRM_MEDIATEK) += mediatek/
obj-y += i2c/
obj-y += panel/
obj-y += bridge/
obj-$(CONFIG_DRM_FSL_DCU) += fsl-dcu/
obj-$(CONFIG_DRM_ETNAVIV) += etnaviv/
+obj-$(CONFIG_DRM_ARCPGU)+= arc/
+obj-y += hisilicon/
diff --git a/drivers/gpu/drm/amd/acp/acp_hw.c b/drivers/gpu/drm/amd/acp/acp_hw.c
index 7af83f1..c7d7205 100644
--- a/drivers/gpu/drm/amd/acp/acp_hw.c
+++ b/drivers/gpu/drm/amd/acp/acp_hw.c
@@ -34,7 +34,7 @@
#define mmACP_AZALIA_I2S_SELECT 0x51d4
-int amd_acp_hw_init(void *cgs_device,
+int amd_acp_hw_init(struct cgs_device *cgs_device,
unsigned acp_version_major, unsigned acp_version_minor)
{
unsigned int acp_mode = ACP_MODE_I2S;
diff --git a/drivers/gpu/drm/amd/acp/include/acp_gfx_if.h b/drivers/gpu/drm/amd/acp/include/acp_gfx_if.h
index bccf47b..a72ddb2f 100644
--- a/drivers/gpu/drm/amd/acp/include/acp_gfx_if.h
+++ b/drivers/gpu/drm/amd/acp/include/acp_gfx_if.h
@@ -28,7 +28,7 @@
#include "cgs_linux.h"
#include "cgs_common.h"
-int amd_acp_hw_init(void *cgs_device,
+int amd_acp_hw_init(struct cgs_device *cgs_device,
unsigned acp_version_major, unsigned acp_version_minor);
#endif /* _ACP_GFX_IF_H */
diff --git a/drivers/gpu/drm/amd/amdgpu/Kconfig b/drivers/gpu/drm/amd/amdgpu/Kconfig
index b30fcfa..7335c04 100644
--- a/drivers/gpu/drm/amd/amdgpu/Kconfig
+++ b/drivers/gpu/drm/amd/amdgpu/Kconfig
@@ -15,3 +15,13 @@
help
This option selects CONFIG_MMU_NOTIFIER if it isn't already
selected to enabled full userptr support.
+
+config DRM_AMDGPU_GART_DEBUGFS
+ bool "Allow GART access through debugfs"
+ depends on DRM_AMDGPU
+ depends on DEBUG_FS
+ default n
+ help
+ Selecting this option creates a debugfs file to inspect the mapped
+ pages. Uses more memory for housekeeping, enable only for debugging.
+
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index 1bcbade4..2a009c3 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -283,7 +283,8 @@
int (*parse_cs)(struct amdgpu_cs_parser *p, uint32_t ib_idx);
/* command emit functions */
void (*emit_ib)(struct amdgpu_ring *ring,
- struct amdgpu_ib *ib);
+ struct amdgpu_ib *ib,
+ unsigned vm_id, bool ctx_switch);
void (*emit_fence)(struct amdgpu_ring *ring, uint64_t addr,
uint64_t seq, unsigned flags);
void (*emit_pipeline_sync)(struct amdgpu_ring *ring);
@@ -302,6 +303,8 @@
void (*insert_nop)(struct amdgpu_ring *ring, uint32_t count);
/* pad the indirect buffer to the necessary number of dw */
void (*pad_ib)(struct amdgpu_ring *ring, struct amdgpu_ib *ib);
+ unsigned (*init_cond_exec)(struct amdgpu_ring *ring);
+ void (*patch_cond_exec)(struct amdgpu_ring *ring, unsigned offset);
};
/*
@@ -365,13 +368,6 @@
#define AMDGPU_FENCE_FLAG_64BIT (1 << 0)
#define AMDGPU_FENCE_FLAG_INT (1 << 1)
-struct amdgpu_user_fence {
- /* write-back bo */
- struct amdgpu_bo *bo;
- /* write-back address offset to bo start */
- uint32_t offset;
-};
-
int amdgpu_fence_driver_init(struct amdgpu_device *adev);
void amdgpu_fence_driver_fini(struct amdgpu_device *adev);
void amdgpu_fence_driver_force_completion(struct amdgpu_device *adev);
@@ -391,6 +387,14 @@
/*
* TTM.
*/
+
+#define AMDGPU_TTM_LRU_SIZE 20
+
+struct amdgpu_mman_lru {
+ struct list_head *lru[TTM_NUM_MEM_TYPES];
+ struct list_head *swap_lru;
+};
+
struct amdgpu_mman {
struct ttm_bo_global_ref bo_global_ref;
struct drm_global_reference mem_global_ref;
@@ -408,6 +412,9 @@
struct amdgpu_ring *buffer_funcs_ring;
/* Scheduler entity for buffer moves */
struct amd_sched_entity entity;
+
+ /* custom LRU management */
+ struct amdgpu_mman_lru log2_size[AMDGPU_TTM_LRU_SIZE];
};
int amdgpu_copy_buffer(struct amdgpu_ring *ring,
@@ -494,9 +501,10 @@
struct drm_file *file_priv);
unsigned long amdgpu_gem_timeout(uint64_t timeout_ns);
struct sg_table *amdgpu_gem_prime_get_sg_table(struct drm_gem_object *obj);
-struct drm_gem_object *amdgpu_gem_prime_import_sg_table(struct drm_device *dev,
- struct dma_buf_attachment *attach,
- struct sg_table *sg);
+struct drm_gem_object *
+amdgpu_gem_prime_import_sg_table(struct drm_device *dev,
+ struct dma_buf_attachment *attach,
+ struct sg_table *sg);
struct dma_buf *amdgpu_gem_prime_export(struct drm_device *dev,
struct drm_gem_object *gobj,
int flags);
@@ -586,6 +594,9 @@
struct amdgpu_sync *sync,
struct reservation_object *resv,
void *owner);
+bool amdgpu_sync_is_idle(struct amdgpu_sync *sync);
+int amdgpu_sync_cycle_fences(struct amdgpu_sync *dst, struct amdgpu_sync *src,
+ struct fence *fence);
struct fence *amdgpu_sync_get_fence(struct amdgpu_sync *sync);
int amdgpu_sync_wait(struct amdgpu_sync *sync);
void amdgpu_sync_free(struct amdgpu_sync *sync);
@@ -609,8 +620,9 @@
unsigned num_gpu_pages;
unsigned num_cpu_pages;
unsigned table_size;
+#ifdef CONFIG_DRM_AMDGPU_GART_DEBUGFS
struct page **pages;
- dma_addr_t *pages_addr;
+#endif
bool ready;
const struct amdgpu_gart_funcs *gart_funcs;
};
@@ -709,6 +721,7 @@
unsigned shared_count;
struct fence **shared;
struct fence_cb cb;
+ bool async;
};
@@ -721,17 +734,7 @@
uint32_t length_dw;
uint64_t gpu_addr;
uint32_t *ptr;
- struct amdgpu_user_fence *user;
- struct amdgpu_vm *vm;
- unsigned vm_id;
- uint64_t vm_pd_addr;
- struct amdgpu_ctx *ctx;
- uint32_t gds_base, gds_size;
- uint32_t gws_base, gws_size;
- uint32_t oa_base, oa_size;
uint32_t flags;
- /* resulting sequence number */
- uint64_t sequence;
};
enum amdgpu_ring_type {
@@ -742,22 +745,25 @@
AMDGPU_RING_TYPE_VCE
};
-extern struct amd_sched_backend_ops amdgpu_sched_ops;
+extern const struct amd_sched_backend_ops amdgpu_sched_ops;
int amdgpu_job_alloc(struct amdgpu_device *adev, unsigned num_ibs,
- struct amdgpu_job **job);
+ struct amdgpu_job **job, struct amdgpu_vm *vm);
int amdgpu_job_alloc_with_ib(struct amdgpu_device *adev, unsigned size,
struct amdgpu_job **job);
+
void amdgpu_job_free(struct amdgpu_job *job);
+void amdgpu_job_free_func(struct kref *refcount);
int amdgpu_job_submit(struct amdgpu_job *job, struct amdgpu_ring *ring,
struct amd_sched_entity *entity, void *owner,
struct fence **f);
+void amdgpu_job_timeout_func(struct work_struct *work);
struct amdgpu_ring {
struct amdgpu_device *adev;
const struct amdgpu_ring_funcs *funcs;
struct amdgpu_fence_driver fence_drv;
- struct amd_gpu_scheduler sched;
+ struct amd_gpu_scheduler sched;
spinlock_t fence_lock;
struct amdgpu_bo *ring_obj;
@@ -785,9 +791,12 @@
unsigned wptr_offs;
unsigned next_rptr_offs;
unsigned fence_offs;
- struct amdgpu_ctx *current_ctx;
+ uint64_t current_ctx;
enum amdgpu_ring_type type;
char name[16];
+ unsigned cond_exe_offs;
+ u64 cond_exe_gpu_addr;
+ volatile u32 *cond_exe_cpu_addr;
};
/*
@@ -830,13 +839,6 @@
uint64_t addr;
};
-struct amdgpu_vm_id {
- struct amdgpu_vm_manager_id *mgr_id;
- uint64_t pd_gpu_addr;
- /* last flushed PD/PT update */
- struct fence *flushed_updates;
-};
-
struct amdgpu_vm {
/* tree of virtual addresses mapped */
struct rb_root va;
@@ -862,19 +864,29 @@
struct amdgpu_vm_pt *page_tables;
/* for id and flush management per ring */
- struct amdgpu_vm_id ids[AMDGPU_MAX_RINGS];
+ struct amdgpu_vm_id *ids[AMDGPU_MAX_RINGS];
/* protecting freed */
spinlock_t freed_lock;
/* Scheduler entity for page table updates */
struct amd_sched_entity entity;
+
+ /* client id */
+ u64 client_id;
};
-struct amdgpu_vm_manager_id {
+struct amdgpu_vm_id {
struct list_head list;
- struct fence *active;
- atomic_long_t owner;
+ struct fence *first;
+ struct amdgpu_sync active;
+ struct fence *last_flush;
+ struct amdgpu_ring *last_user;
+ atomic64_t owner;
+
+ uint64_t pd_gpu_addr;
+ /* last flushed PD/PT update */
+ struct fence *flushed_updates;
uint32_t gds_base;
uint32_t gds_size;
@@ -889,7 +901,7 @@
struct mutex lock;
unsigned num_ids;
struct list_head ids_lru;
- struct amdgpu_vm_manager_id ids[AMDGPU_NUM_VM];
+ struct amdgpu_vm_id ids[AMDGPU_NUM_VM];
uint32_t max_pfn;
/* vram base address for page table entry */
@@ -901,6 +913,8 @@
struct amdgpu_ring *vm_pte_rings[AMDGPU_MAX_RINGS];
unsigned vm_pte_num_rings;
atomic_t vm_pte_next_ring;
+ /* client id counter */
+ atomic64_t client_counter;
};
void amdgpu_vm_manager_init(struct amdgpu_device *adev);
@@ -916,11 +930,11 @@
int amdgpu_vm_grab_id(struct amdgpu_vm *vm, struct amdgpu_ring *ring,
struct amdgpu_sync *sync, struct fence *fence,
unsigned *vm_id, uint64_t *vm_pd_addr);
-void amdgpu_vm_flush(struct amdgpu_ring *ring,
- unsigned vm_id, uint64_t pd_addr,
- uint32_t gds_base, uint32_t gds_size,
- uint32_t gws_base, uint32_t gws_size,
- uint32_t oa_base, uint32_t oa_size);
+int amdgpu_vm_flush(struct amdgpu_ring *ring,
+ unsigned vm_id, uint64_t pd_addr,
+ uint32_t gds_base, uint32_t gds_size,
+ uint32_t gws_base, uint32_t gws_size,
+ uint32_t oa_base, uint32_t oa_size);
void amdgpu_vm_reset_id(struct amdgpu_device *adev, unsigned vm_id);
uint64_t amdgpu_vm_map_gart(const dma_addr_t *pages_addr, uint64_t addr);
int amdgpu_vm_update_page_directory(struct amdgpu_device *adev,
@@ -1026,6 +1040,11 @@
*/
#include "clearstate_defs.h"
+struct amdgpu_rlc_funcs {
+ void (*enter_safe_mode)(struct amdgpu_device *adev);
+ void (*exit_safe_mode)(struct amdgpu_device *adev);
+};
+
struct amdgpu_rlc {
/* for power gating */
struct amdgpu_bo *save_restore_obj;
@@ -1044,6 +1063,24 @@
uint64_t cp_table_gpu_addr;
volatile uint32_t *cp_table_ptr;
u32 cp_table_size;
+
+ /* safe mode for updating CG/PG state */
+ bool in_safe_mode;
+ const struct amdgpu_rlc_funcs *funcs;
+
+ /* for firmware data */
+ u32 save_and_restore_offset;
+ u32 clear_state_descriptor_offset;
+ u32 avail_scratch_ram_locations;
+ u32 reg_restore_list_size;
+ u32 reg_list_format_start;
+ u32 reg_list_format_separate_start;
+ u32 starting_offsets_start;
+ u32 reg_list_format_size_bytes;
+ u32 reg_list_size_bytes;
+
+ u32 *register_list_format;
+ u32 *register_restore;
};
struct amdgpu_mec {
@@ -1097,6 +1134,12 @@
uint32_t macrotile_mode_array[16];
};
+struct amdgpu_cu_info {
+ uint32_t number; /* total active CU number */
+ uint32_t ao_cu_mask;
+ uint32_t bitmap[4][4];
+};
+
struct amdgpu_gfx {
struct mutex gpu_clock_mutex;
struct amdgpu_gca_config config;
@@ -1129,17 +1172,19 @@
struct amdgpu_irq_src priv_reg_irq;
struct amdgpu_irq_src priv_inst_irq;
/* gfx status */
- uint32_t gfx_current_status;
+ uint32_t gfx_current_status;
/* ce ram size*/
- unsigned ce_ram_size;
+ unsigned ce_ram_size;
+ struct amdgpu_cu_info cu_info;
};
int amdgpu_ib_get(struct amdgpu_device *adev, struct amdgpu_vm *vm,
unsigned size, struct amdgpu_ib *ib);
-void amdgpu_ib_free(struct amdgpu_device *adev, struct amdgpu_ib *ib, struct fence *f);
+void amdgpu_ib_free(struct amdgpu_device *adev, struct amdgpu_ib *ib,
+ struct fence *f);
int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs,
struct amdgpu_ib *ib, struct fence *last_vm_update,
- struct fence **f);
+ struct amdgpu_job *job, struct fence **f);
int amdgpu_ib_pool_init(struct amdgpu_device *adev);
void amdgpu_ib_pool_fini(struct amdgpu_device *adev);
int amdgpu_ib_ring_tests(struct amdgpu_device *adev);
@@ -1164,7 +1209,7 @@
struct amdgpu_cs_chunk {
uint32_t chunk_id;
uint32_t length_dw;
- uint32_t *kdata;
+ void *kdata;
};
struct amdgpu_cs_parser {
@@ -1195,13 +1240,25 @@
struct amdgpu_job {
struct amd_sched_job base;
struct amdgpu_device *adev;
+ struct amdgpu_vm *vm;
struct amdgpu_ring *ring;
struct amdgpu_sync sync;
struct amdgpu_ib *ibs;
struct fence *fence; /* the hw fence */
uint32_t num_ibs;
void *owner;
- struct amdgpu_user_fence uf;
+ uint64_t ctx;
+ unsigned vm_id;
+ uint64_t vm_pd_addr;
+ uint32_t gds_base, gds_size;
+ uint32_t gws_base, gws_size;
+ uint32_t oa_base, oa_size;
+
+ /* user fence handling */
+ struct amdgpu_bo *uf_bo;
+ uint32_t uf_offset;
+ uint64_t uf_sequence;
+
};
#define to_amdgpu_job(sched_job) \
container_of((sched_job), struct amdgpu_job, base)
@@ -1582,10 +1639,12 @@
/*
* UVD
*/
-#define AMDGPU_MAX_UVD_HANDLES 10
-#define AMDGPU_UVD_STACK_SIZE (1024*1024)
-#define AMDGPU_UVD_HEAP_SIZE (1024*1024)
-#define AMDGPU_UVD_FIRMWARE_OFFSET 256
+#define AMDGPU_DEFAULT_UVD_HANDLES 10
+#define AMDGPU_MAX_UVD_HANDLES 40
+#define AMDGPU_UVD_STACK_SIZE (200*1024)
+#define AMDGPU_UVD_HEAP_SIZE (256*1024)
+#define AMDGPU_UVD_SESSION_SIZE (50*1024)
+#define AMDGPU_UVD_FIRMWARE_OFFSET 256
struct amdgpu_uvd {
struct amdgpu_bo *vcpu_bo;
@@ -1593,6 +1652,7 @@
uint64_t gpu_addr;
unsigned fw_version;
void *saved_bo;
+ unsigned max_handles;
atomic_t handles[AMDGPU_MAX_UVD_HANDLES];
struct drm_file *filp[AMDGPU_MAX_UVD_HANDLES];
struct delayed_work idle_work;
@@ -1645,7 +1705,7 @@
struct amdgpu_sdma_instance instance[AMDGPU_MAX_SDMA_INSTANCES];
struct amdgpu_irq_src trap_irq;
struct amdgpu_irq_src illegal_inst_irq;
- int num_instances;
+ int num_instances;
};
/*
@@ -1691,12 +1751,12 @@
* Debugfs
*/
struct amdgpu_debugfs {
- struct drm_info_list *files;
+ const struct drm_info_list *files;
unsigned num_files;
};
int amdgpu_debugfs_add_files(struct amdgpu_device *adev,
- struct drm_info_list *files,
+ const struct drm_info_list *files,
unsigned nfiles);
int amdgpu_debugfs_fence_init(struct amdgpu_device *adev);
@@ -1738,13 +1798,6 @@
bool grbm_indexed;
};
-struct amdgpu_cu_info {
- uint32_t number; /* total active CU number */
- uint32_t ao_cu_mask;
- uint32_t bitmap[4][4];
-};
-
-
/*
* ASIC specific functions.
*/
@@ -1762,7 +1815,6 @@
u32 (*get_xclk)(struct amdgpu_device *adev);
/* get the gpu clock counter */
uint64_t (*get_gpu_clock_counter)(struct amdgpu_device *adev);
- int (*get_cu_info)(struct amdgpu_device *adev, struct amdgpu_cu_info *info);
/* MM block clocks */
int (*set_uvd_clocks)(struct amdgpu_device *adev, u32 vclk, u32 dclk);
int (*set_vce_clocks)(struct amdgpu_device *adev, u32 evclk, u32 ecclk);
@@ -1855,15 +1907,8 @@
/*
* CGS
*/
-void *amdgpu_cgs_create_device(struct amdgpu_device *adev);
-void amdgpu_cgs_destroy_device(void *cgs_device);
-
-
-/*
- * CGS
- */
-void *amdgpu_cgs_create_device(struct amdgpu_device *adev);
-void amdgpu_cgs_destroy_device(void *cgs_device);
+struct cgs_device *amdgpu_cgs_create_device(struct amdgpu_device *adev);
+void amdgpu_cgs_destroy_device(struct cgs_device *cgs_device);
/* GPU virtualization */
@@ -1904,16 +1949,15 @@
int usec_timeout;
const struct amdgpu_asic_funcs *asic_funcs;
bool shutdown;
- bool suspend;
bool need_dma32;
bool accel_working;
- struct work_struct reset_work;
+ struct work_struct reset_work;
struct notifier_block acpi_nb;
struct amdgpu_i2c_chan *i2c_bus[AMDGPU_MAX_I2C_BUS];
struct amdgpu_debugfs debugfs[AMDGPU_DEBUGFS_MAX_COMPONENTS];
- unsigned debugfs_count;
+ unsigned debugfs_count;
#if defined(CONFIG_DEBUG_FS)
- struct dentry *debugfs_regs;
+ struct dentry *debugfs_regs[AMDGPU_DEBUGFS_MAX_COMPONENTS];
#endif
struct amdgpu_atif atif;
struct amdgpu_atcs atcs;
@@ -1926,7 +1970,6 @@
/* BIOS */
uint8_t *bios;
bool is_atom_bios;
- uint16_t bios_header_start;
struct amdgpu_bo *stollen_vga_memory;
uint32_t bios_scratch[AMDGPU_BIOS_NUM_SCRATCH];
@@ -2163,7 +2206,6 @@
#define amdgpu_asic_read_disabled_bios(adev) (adev)->asic_funcs->read_disabled_bios((adev))
#define amdgpu_asic_read_bios_from_rom(adev, b, l) (adev)->asic_funcs->read_bios_from_rom((adev), (b), (l))
#define amdgpu_asic_read_register(adev, se, sh, offset, v)((adev)->asic_funcs->read_register((adev), (se), (sh), (offset), (v)))
-#define amdgpu_asic_get_cu_info(adev, info) (adev)->asic_funcs->get_cu_info((adev), (info))
#define amdgpu_gart_flush_gpu_tlb(adev, vmid) (adev)->gart.gart_funcs->flush_gpu_tlb((adev), (vmid))
#define amdgpu_gart_set_pte_pde(adev, pt, idx, addr, flags) (adev)->gart.gart_funcs->set_pte_pde((adev), (pt), (idx), (addr), (flags))
#define amdgpu_vm_copy_pte(adev, ib, pe, src, count) ((adev)->vm_manager.vm_pte_funcs->copy_pte((ib), (pe), (src), (count)))
@@ -2175,7 +2217,7 @@
#define amdgpu_ring_get_rptr(r) (r)->funcs->get_rptr((r))
#define amdgpu_ring_get_wptr(r) (r)->funcs->get_wptr((r))
#define amdgpu_ring_set_wptr(r) (r)->funcs->set_wptr((r))
-#define amdgpu_ring_emit_ib(r, ib) (r)->funcs->emit_ib((r), (ib))
+#define amdgpu_ring_emit_ib(r, ib, vm_id, c) (r)->funcs->emit_ib((r), (ib), (vm_id), (c))
#define amdgpu_ring_emit_pipeline_sync(r) (r)->funcs->emit_pipeline_sync((r))
#define amdgpu_ring_emit_vm_flush(r, vmid, addr) (r)->funcs->emit_vm_flush((r), (vmid), (addr))
#define amdgpu_ring_emit_fence(r, addr, seq, flags) (r)->funcs->emit_fence((r), (addr), (seq), (flags))
@@ -2183,6 +2225,8 @@
#define amdgpu_ring_emit_hdp_flush(r) (r)->funcs->emit_hdp_flush((r))
#define amdgpu_ring_emit_hdp_invalidate(r) (r)->funcs->emit_hdp_invalidate((r))
#define amdgpu_ring_pad_ib(r, ib) ((r)->funcs->pad_ib((r), (ib)))
+#define amdgpu_ring_init_cond_exec(r) (r)->funcs->init_cond_exec((r))
+#define amdgpu_ring_patch_cond_exec(r,o) (r)->funcs->patch_cond_exec((r),(o))
#define amdgpu_ih_get_wptr(adev) (adev)->irq.ih_funcs->get_wptr((adev))
#define amdgpu_ih_decode_iv(adev, iv) (adev)->irq.ih_funcs->decode_iv((adev), (iv))
#define amdgpu_ih_set_rptr(adev) (adev)->irq.ih_funcs->set_rptr((adev))
@@ -2196,7 +2240,7 @@
#define amdgpu_display_hpd_set_polarity(adev, h) (adev)->mode_info.funcs->hpd_set_polarity((adev), (h))
#define amdgpu_display_hpd_get_gpio_reg(adev) (adev)->mode_info.funcs->hpd_get_gpio_reg((adev))
#define amdgpu_display_bandwidth_update(adev) (adev)->mode_info.funcs->bandwidth_update((adev))
-#define amdgpu_display_page_flip(adev, crtc, base) (adev)->mode_info.funcs->page_flip((adev), (crtc), (base))
+#define amdgpu_display_page_flip(adev, crtc, base, async) (adev)->mode_info.funcs->page_flip((adev), (crtc), (base), (async))
#define amdgpu_display_page_flip_get_scanoutpos(adev, crtc, vbl, pos) (adev)->mode_info.funcs->page_flip_get_scanoutpos((adev), (crtc), (vbl), (pos))
#define amdgpu_display_add_encoder(adev, e, s, c) (adev)->mode_info.funcs->add_encoder((adev), (e), (s), (c))
#define amdgpu_display_add_connector(adev, ci, sd, ct, ib, coi, h, r) (adev)->mode_info.funcs->add_connector((adev), (ci), (sd), (ct), (ib), (coi), (h), (r))
@@ -2339,7 +2383,7 @@
* KMS
*/
extern const struct drm_ioctl_desc amdgpu_ioctls_kms[];
-extern int amdgpu_max_kms_ioctl;
+extern const int amdgpu_max_kms_ioctl;
int amdgpu_driver_load_kms(struct drm_device *dev, unsigned long flags);
int amdgpu_driver_unload_kms(struct drm_device *dev);
@@ -2398,5 +2442,4 @@
uint64_t addr, struct amdgpu_bo **bo);
#include "amdgpu_object.h"
-
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_acp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_acp.c
index b7b583c..252edba 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_acp.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_acp.c
@@ -467,13 +467,6 @@
return 0;
}
-static void acp_print_status(void *handle)
-{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- dev_info(adev->dev, "ACP STATUS\n");
-}
-
static int acp_set_clockgating_state(void *handle,
enum amd_clockgating_state state)
{
@@ -487,6 +480,7 @@
}
const struct amd_ip_funcs acp_ip_funcs = {
+ .name = "acp_ip",
.early_init = acp_early_init,
.late_init = NULL,
.sw_init = acp_sw_init,
@@ -498,7 +492,6 @@
.is_idle = acp_is_idle,
.wait_for_idle = acp_wait_for_idle,
.soft_reset = acp_soft_reset,
- .print_status = acp_print_status,
.set_clockgating_state = acp_set_clockgating_state,
.set_powergating_state = acp_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_acp.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_acp.h
index f6e32a6..8a39631 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_acp.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_acp.h
@@ -30,7 +30,7 @@
struct amdgpu_acp {
struct device *parent;
- void *cgs_device;
+ struct cgs_device *cgs_device;
struct amd_acp_private *private;
struct mfd_cell *acp_cell;
struct resource *acp_res;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
index 84b0ce3..9df1bcb 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
@@ -234,16 +234,6 @@
return hpd;
}
-static bool amdgpu_atombios_apply_quirks(struct amdgpu_device *adev,
- uint32_t supported_device,
- int *connector_type,
- struct amdgpu_i2c_bus_rec *i2c_bus,
- uint16_t *line_mux,
- struct amdgpu_hpd *hpd)
-{
- return true;
-}
-
static const int object_connector_convert[] = {
DRM_MODE_CONNECTOR_Unknown,
DRM_MODE_CONNECTOR_DVII,
@@ -514,11 +504,6 @@
conn_id = le16_to_cpu(path->usConnObjectId);
- if (!amdgpu_atombios_apply_quirks
- (adev, le16_to_cpu(path->usDeviceTag), &connector_type,
- &ddc_bus, &conn_id, &hpd))
- continue;
-
amdgpu_display_add_connector(adev,
conn_id,
le16_to_cpu(path->usDeviceTag),
@@ -699,6 +684,36 @@
return ret;
}
+union gfx_info {
+ ATOM_GFX_INFO_V2_1 info;
+};
+
+int amdgpu_atombios_get_gfx_info(struct amdgpu_device *adev)
+{
+ struct amdgpu_mode_info *mode_info = &adev->mode_info;
+ int index = GetIndexIntoMasterTable(DATA, GFX_Info);
+ uint8_t frev, crev;
+ uint16_t data_offset;
+ int ret = -EINVAL;
+
+ if (amdgpu_atom_parse_data_header(mode_info->atom_context, index, NULL,
+ &frev, &crev, &data_offset)) {
+ union gfx_info *gfx_info = (union gfx_info *)
+ (mode_info->atom_context->bios + data_offset);
+
+ adev->gfx.config.max_shader_engines = gfx_info->info.max_shader_engines;
+ adev->gfx.config.max_tile_pipes = gfx_info->info.max_tile_pipes;
+ adev->gfx.config.max_cu_per_sh = gfx_info->info.max_cu_per_sh;
+ adev->gfx.config.max_sh_per_se = gfx_info->info.max_sh_per_se;
+ adev->gfx.config.max_backends_per_se = gfx_info->info.max_backends_per_se;
+ adev->gfx.config.max_texture_channel_caches =
+ gfx_info->info.max_texture_channel_caches;
+
+ ret = 0;
+ }
+ return ret;
+}
+
union igp_info {
struct _ATOM_INTEGRATED_SYSTEM_INFO info;
struct _ATOM_INTEGRATED_SYSTEM_INFO_V2 info_2;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.h
index 9e14420..8c2e696 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.h
@@ -144,6 +144,8 @@
int amdgpu_atombios_get_clock_info(struct amdgpu_device *adev);
+int amdgpu_atombios_get_gfx_info(struct amdgpu_device *adev);
+
bool amdgpu_atombios_get_asic_ss_info(struct amdgpu_device *adev,
struct amdgpu_atom_ss *ss,
int id, u32 clock);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_benchmark.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_benchmark.c
index cd639c3..33e47a4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_benchmark.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_benchmark.c
@@ -141,7 +141,7 @@
void amdgpu_benchmark(struct amdgpu_device *adev, int test_number)
{
int i;
- int common_modes[AMDGPU_BENCHMARK_COMMON_MODES_N] = {
+ static const int common_modes[AMDGPU_BENCHMARK_COMMON_MODES_N] = {
640 * 480 * 4,
720 * 480 * 4,
800 * 600 * 4,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_bios.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_bios.c
index 80add22..99ca75b 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_bios.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_bios.c
@@ -349,7 +349,7 @@
bool amdgpu_get_bios(struct amdgpu_device *adev)
{
bool r;
- uint16_t tmp;
+ uint16_t tmp, bios_header_start;
r = amdgpu_atrm_get_bios(adev);
if (r == false)
@@ -383,11 +383,11 @@
goto free_bios;
}
- adev->bios_header_start = RBIOS16(0x48);
- if (!adev->bios_header_start) {
+ bios_header_start = RBIOS16(0x48);
+ if (!bios_header_start) {
goto free_bios;
}
- tmp = adev->bios_header_start + 4;
+ tmp = bios_header_start + 4;
if (!memcmp(adev->bios + tmp, "ATOM", 4) ||
!memcmp(adev->bios + tmp, "MOTA", 4)) {
adev->is_atom_bios = true;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c
index eacd810..823bf5e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.c
@@ -106,7 +106,7 @@
struct amdgpu_bo *bo;
struct mm_struct *usermm;
- gobj = drm_gem_object_lookup(adev->ddev, filp, info[i].bo_handle);
+ gobj = drm_gem_object_lookup(filp, info[i].bo_handle);
if (!gobj) {
r = -ENOENT;
goto error_free;
@@ -263,7 +263,7 @@
for (i = 0; i < args->in.bo_number; ++i) {
if (copy_from_user(&info[i], uptr, bytes))
goto error_free;
-
+
uptr += args->in.bo_info_size;
}
}
@@ -271,7 +271,7 @@
switch (args->in.operation) {
case AMDGPU_BO_LIST_OP_CREATE:
r = amdgpu_bo_list_create(fpriv, &list, &handle);
- if (r)
+ if (r)
goto error_free;
r = amdgpu_bo_list_set(adev, filp, list, info,
@@ -281,7 +281,7 @@
goto error_free;
break;
-
+
case AMDGPU_BO_LIST_OP_DESTROY:
amdgpu_bo_list_destroy(fpriv, handle);
handle = 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cgs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cgs.c
index 6043dc7..199f76b 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cgs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cgs.c
@@ -42,7 +42,7 @@
struct amdgpu_device *adev = \
((struct amdgpu_cgs_device *)cgs_device)->adev
-static int amdgpu_cgs_gpu_mem_info(void *cgs_device, enum cgs_gpu_mem_type type,
+static int amdgpu_cgs_gpu_mem_info(struct cgs_device *cgs_device, enum cgs_gpu_mem_type type,
uint64_t *mc_start, uint64_t *mc_size,
uint64_t *mem_size)
{
@@ -73,7 +73,7 @@
return 0;
}
-static int amdgpu_cgs_gmap_kmem(void *cgs_device, void *kmem,
+static int amdgpu_cgs_gmap_kmem(struct cgs_device *cgs_device, void *kmem,
uint64_t size,
uint64_t min_offset, uint64_t max_offset,
cgs_handle_t *kmem_handle, uint64_t *mcaddr)
@@ -102,7 +102,7 @@
return ret;
}
-static int amdgpu_cgs_gunmap_kmem(void *cgs_device, cgs_handle_t kmem_handle)
+static int amdgpu_cgs_gunmap_kmem(struct cgs_device *cgs_device, cgs_handle_t kmem_handle)
{
struct amdgpu_bo *obj = (struct amdgpu_bo *)kmem_handle;
@@ -118,7 +118,7 @@
return 0;
}
-static int amdgpu_cgs_alloc_gpu_mem(void *cgs_device,
+static int amdgpu_cgs_alloc_gpu_mem(struct cgs_device *cgs_device,
enum cgs_gpu_mem_type type,
uint64_t size, uint64_t align,
uint64_t min_offset, uint64_t max_offset,
@@ -208,7 +208,7 @@
return ret;
}
-static int amdgpu_cgs_free_gpu_mem(void *cgs_device, cgs_handle_t handle)
+static int amdgpu_cgs_free_gpu_mem(struct cgs_device *cgs_device, cgs_handle_t handle)
{
struct amdgpu_bo *obj = (struct amdgpu_bo *)handle;
@@ -225,7 +225,7 @@
return 0;
}
-static int amdgpu_cgs_gmap_gpu_mem(void *cgs_device, cgs_handle_t handle,
+static int amdgpu_cgs_gmap_gpu_mem(struct cgs_device *cgs_device, cgs_handle_t handle,
uint64_t *mcaddr)
{
int r;
@@ -246,7 +246,7 @@
return r;
}
-static int amdgpu_cgs_gunmap_gpu_mem(void *cgs_device, cgs_handle_t handle)
+static int amdgpu_cgs_gunmap_gpu_mem(struct cgs_device *cgs_device, cgs_handle_t handle)
{
int r;
struct amdgpu_bo *obj = (struct amdgpu_bo *)handle;
@@ -258,7 +258,7 @@
return r;
}
-static int amdgpu_cgs_kmap_gpu_mem(void *cgs_device, cgs_handle_t handle,
+static int amdgpu_cgs_kmap_gpu_mem(struct cgs_device *cgs_device, cgs_handle_t handle,
void **map)
{
int r;
@@ -271,7 +271,7 @@
return r;
}
-static int amdgpu_cgs_kunmap_gpu_mem(void *cgs_device, cgs_handle_t handle)
+static int amdgpu_cgs_kunmap_gpu_mem(struct cgs_device *cgs_device, cgs_handle_t handle)
{
int r;
struct amdgpu_bo *obj = (struct amdgpu_bo *)handle;
@@ -283,20 +283,20 @@
return r;
}
-static uint32_t amdgpu_cgs_read_register(void *cgs_device, unsigned offset)
+static uint32_t amdgpu_cgs_read_register(struct cgs_device *cgs_device, unsigned offset)
{
CGS_FUNC_ADEV;
return RREG32(offset);
}
-static void amdgpu_cgs_write_register(void *cgs_device, unsigned offset,
+static void amdgpu_cgs_write_register(struct cgs_device *cgs_device, unsigned offset,
uint32_t value)
{
CGS_FUNC_ADEV;
WREG32(offset, value);
}
-static uint32_t amdgpu_cgs_read_ind_register(void *cgs_device,
+static uint32_t amdgpu_cgs_read_ind_register(struct cgs_device *cgs_device,
enum cgs_ind_reg space,
unsigned index)
{
@@ -320,7 +320,7 @@
return 0;
}
-static void amdgpu_cgs_write_ind_register(void *cgs_device,
+static void amdgpu_cgs_write_ind_register(struct cgs_device *cgs_device,
enum cgs_ind_reg space,
unsigned index, uint32_t value)
{
@@ -343,7 +343,7 @@
WARN(1, "Invalid indirect register space");
}
-static uint8_t amdgpu_cgs_read_pci_config_byte(void *cgs_device, unsigned addr)
+static uint8_t amdgpu_cgs_read_pci_config_byte(struct cgs_device *cgs_device, unsigned addr)
{
CGS_FUNC_ADEV;
uint8_t val;
@@ -353,7 +353,7 @@
return val;
}
-static uint16_t amdgpu_cgs_read_pci_config_word(void *cgs_device, unsigned addr)
+static uint16_t amdgpu_cgs_read_pci_config_word(struct cgs_device *cgs_device, unsigned addr)
{
CGS_FUNC_ADEV;
uint16_t val;
@@ -363,7 +363,7 @@
return val;
}
-static uint32_t amdgpu_cgs_read_pci_config_dword(void *cgs_device,
+static uint32_t amdgpu_cgs_read_pci_config_dword(struct cgs_device *cgs_device,
unsigned addr)
{
CGS_FUNC_ADEV;
@@ -374,7 +374,7 @@
return val;
}
-static void amdgpu_cgs_write_pci_config_byte(void *cgs_device, unsigned addr,
+static void amdgpu_cgs_write_pci_config_byte(struct cgs_device *cgs_device, unsigned addr,
uint8_t value)
{
CGS_FUNC_ADEV;
@@ -382,7 +382,7 @@
WARN(ret, "pci_write_config_byte error");
}
-static void amdgpu_cgs_write_pci_config_word(void *cgs_device, unsigned addr,
+static void amdgpu_cgs_write_pci_config_word(struct cgs_device *cgs_device, unsigned addr,
uint16_t value)
{
CGS_FUNC_ADEV;
@@ -390,7 +390,7 @@
WARN(ret, "pci_write_config_word error");
}
-static void amdgpu_cgs_write_pci_config_dword(void *cgs_device, unsigned addr,
+static void amdgpu_cgs_write_pci_config_dword(struct cgs_device *cgs_device, unsigned addr,
uint32_t value)
{
CGS_FUNC_ADEV;
@@ -399,7 +399,7 @@
}
-static int amdgpu_cgs_get_pci_resource(void *cgs_device,
+static int amdgpu_cgs_get_pci_resource(struct cgs_device *cgs_device,
enum cgs_resource_type resource_type,
uint64_t size,
uint64_t offset,
@@ -433,7 +433,7 @@
}
}
-static const void *amdgpu_cgs_atom_get_data_table(void *cgs_device,
+static const void *amdgpu_cgs_atom_get_data_table(struct cgs_device *cgs_device,
unsigned table, uint16_t *size,
uint8_t *frev, uint8_t *crev)
{
@@ -449,7 +449,7 @@
return NULL;
}
-static int amdgpu_cgs_atom_get_cmd_table_revs(void *cgs_device, unsigned table,
+static int amdgpu_cgs_atom_get_cmd_table_revs(struct cgs_device *cgs_device, unsigned table,
uint8_t *frev, uint8_t *crev)
{
CGS_FUNC_ADEV;
@@ -462,7 +462,7 @@
return -EINVAL;
}
-static int amdgpu_cgs_atom_exec_cmd_table(void *cgs_device, unsigned table,
+static int amdgpu_cgs_atom_exec_cmd_table(struct cgs_device *cgs_device, unsigned table,
void *args)
{
CGS_FUNC_ADEV;
@@ -471,33 +471,33 @@
adev->mode_info.atom_context, table, args);
}
-static int amdgpu_cgs_create_pm_request(void *cgs_device, cgs_handle_t *request)
+static int amdgpu_cgs_create_pm_request(struct cgs_device *cgs_device, cgs_handle_t *request)
{
/* TODO */
return 0;
}
-static int amdgpu_cgs_destroy_pm_request(void *cgs_device, cgs_handle_t request)
+static int amdgpu_cgs_destroy_pm_request(struct cgs_device *cgs_device, cgs_handle_t request)
{
/* TODO */
return 0;
}
-static int amdgpu_cgs_set_pm_request(void *cgs_device, cgs_handle_t request,
+static int amdgpu_cgs_set_pm_request(struct cgs_device *cgs_device, cgs_handle_t request,
int active)
{
/* TODO */
return 0;
}
-static int amdgpu_cgs_pm_request_clock(void *cgs_device, cgs_handle_t request,
+static int amdgpu_cgs_pm_request_clock(struct cgs_device *cgs_device, cgs_handle_t request,
enum cgs_clock clock, unsigned freq)
{
/* TODO */
return 0;
}
-static int amdgpu_cgs_pm_request_engine(void *cgs_device, cgs_handle_t request,
+static int amdgpu_cgs_pm_request_engine(struct cgs_device *cgs_device, cgs_handle_t request,
enum cgs_engine engine, int powered)
{
/* TODO */
@@ -506,7 +506,7 @@
-static int amdgpu_cgs_pm_query_clock_limits(void *cgs_device,
+static int amdgpu_cgs_pm_query_clock_limits(struct cgs_device *cgs_device,
enum cgs_clock clock,
struct cgs_clock_limits *limits)
{
@@ -514,7 +514,7 @@
return 0;
}
-static int amdgpu_cgs_set_camera_voltages(void *cgs_device, uint32_t mask,
+static int amdgpu_cgs_set_camera_voltages(struct cgs_device *cgs_device, uint32_t mask,
const uint32_t *voltages)
{
DRM_ERROR("not implemented");
@@ -565,7 +565,7 @@
.process = cgs_process_irq,
};
-static int amdgpu_cgs_add_irq_source(void *cgs_device, unsigned src_id,
+static int amdgpu_cgs_add_irq_source(struct cgs_device *cgs_device, unsigned src_id,
unsigned num_types,
cgs_irq_source_set_func_t set,
cgs_irq_handler_func_t handler,
@@ -600,19 +600,19 @@
return ret;
}
-static int amdgpu_cgs_irq_get(void *cgs_device, unsigned src_id, unsigned type)
+static int amdgpu_cgs_irq_get(struct cgs_device *cgs_device, unsigned src_id, unsigned type)
{
CGS_FUNC_ADEV;
return amdgpu_irq_get(adev, adev->irq.sources[src_id], type);
}
-static int amdgpu_cgs_irq_put(void *cgs_device, unsigned src_id, unsigned type)
+static int amdgpu_cgs_irq_put(struct cgs_device *cgs_device, unsigned src_id, unsigned type)
{
CGS_FUNC_ADEV;
return amdgpu_irq_put(adev, adev->irq.sources[src_id], type);
}
-int amdgpu_cgs_set_clockgating_state(void *cgs_device,
+int amdgpu_cgs_set_clockgating_state(struct cgs_device *cgs_device,
enum amd_ip_block_type block_type,
enum amd_clockgating_state state)
{
@@ -633,7 +633,7 @@
return r;
}
-int amdgpu_cgs_set_powergating_state(void *cgs_device,
+int amdgpu_cgs_set_powergating_state(struct cgs_device *cgs_device,
enum amd_ip_block_type block_type,
enum amd_powergating_state state)
{
@@ -655,7 +655,7 @@
}
-static uint32_t fw_type_convert(void *cgs_device, uint32_t fw_type)
+static uint32_t fw_type_convert(struct cgs_device *cgs_device, uint32_t fw_type)
{
CGS_FUNC_ADEV;
enum AMDGPU_UCODE_ID result = AMDGPU_UCODE_ID_MAXIMUM;
@@ -681,9 +681,10 @@
result = AMDGPU_UCODE_ID_CP_MEC1;
break;
case CGS_UCODE_ID_CP_MEC_JT2:
- if (adev->asic_type == CHIP_TONGA)
+ if (adev->asic_type == CHIP_TONGA || adev->asic_type == CHIP_POLARIS11
+ || adev->asic_type == CHIP_POLARIS10)
result = AMDGPU_UCODE_ID_CP_MEC2;
- else if (adev->asic_type == CHIP_CARRIZO)
+ else
result = AMDGPU_UCODE_ID_CP_MEC1;
break;
case CGS_UCODE_ID_RLC_G:
@@ -695,13 +696,13 @@
return result;
}
-static int amdgpu_cgs_get_firmware_info(void *cgs_device,
+static int amdgpu_cgs_get_firmware_info(struct cgs_device *cgs_device,
enum cgs_ucode_id type,
struct cgs_firmware_info *info)
{
CGS_FUNC_ADEV;
- if (CGS_UCODE_ID_SMU != type) {
+ if ((CGS_UCODE_ID_SMU != type) && (CGS_UCODE_ID_SMU_SK != type)) {
uint64_t gpu_addr;
uint32_t data_size;
const struct gfx_firmware_header_v1_0 *header;
@@ -734,30 +735,44 @@
const uint8_t *src;
const struct smc_firmware_header_v1_0 *hdr;
- switch (adev->asic_type) {
- case CHIP_TONGA:
- strcpy(fw_name, "amdgpu/tonga_smc.bin");
- break;
- case CHIP_FIJI:
- strcpy(fw_name, "amdgpu/fiji_smc.bin");
- break;
- default:
- DRM_ERROR("SMC firmware not supported\n");
- return -EINVAL;
- }
+ if (!adev->pm.fw) {
+ switch (adev->asic_type) {
+ case CHIP_TONGA:
+ strcpy(fw_name, "amdgpu/tonga_smc.bin");
+ break;
+ case CHIP_FIJI:
+ strcpy(fw_name, "amdgpu/fiji_smc.bin");
+ break;
+ case CHIP_POLARIS11:
+ if (type == CGS_UCODE_ID_SMU)
+ strcpy(fw_name, "amdgpu/polaris11_smc.bin");
+ else if (type == CGS_UCODE_ID_SMU_SK)
+ strcpy(fw_name, "amdgpu/polaris11_smc_sk.bin");
+ break;
+ case CHIP_POLARIS10:
+ if (type == CGS_UCODE_ID_SMU)
+ strcpy(fw_name, "amdgpu/polaris10_smc.bin");
+ else if (type == CGS_UCODE_ID_SMU_SK)
+ strcpy(fw_name, "amdgpu/polaris10_smc_sk.bin");
+ break;
+ default:
+ DRM_ERROR("SMC firmware not supported\n");
+ return -EINVAL;
+ }
- err = request_firmware(&adev->pm.fw, fw_name, adev->dev);
- if (err) {
- DRM_ERROR("Failed to request firmware\n");
- return err;
- }
+ err = request_firmware(&adev->pm.fw, fw_name, adev->dev);
+ if (err) {
+ DRM_ERROR("Failed to request firmware\n");
+ return err;
+ }
- err = amdgpu_ucode_validate(adev->pm.fw);
- if (err) {
- DRM_ERROR("Failed to load firmware \"%s\"", fw_name);
- release_firmware(adev->pm.fw);
- adev->pm.fw = NULL;
- return err;
+ err = amdgpu_ucode_validate(adev->pm.fw);
+ if (err) {
+ DRM_ERROR("Failed to load firmware \"%s\"", fw_name);
+ release_firmware(adev->pm.fw);
+ adev->pm.fw = NULL;
+ return err;
+ }
}
hdr = (const struct smc_firmware_header_v1_0 *) adev->pm.fw->data;
@@ -774,7 +789,7 @@
return 0;
}
-static int amdgpu_cgs_query_system_info(void *cgs_device,
+static int amdgpu_cgs_query_system_info(struct cgs_device *cgs_device,
struct cgs_system_info *sys_info)
{
CGS_FUNC_ADEV;
@@ -801,6 +816,9 @@
case CGS_SYSTEM_INFO_PG_FLAGS:
sys_info->value = adev->pg_flags;
break;
+ case CGS_SYSTEM_INFO_GFX_CU_INFO:
+ sys_info->value = adev->gfx.cu_info.number;
+ break;
default:
return -ENODEV;
}
@@ -808,7 +826,7 @@
return 0;
}
-static int amdgpu_cgs_get_active_displays_info(void *cgs_device,
+static int amdgpu_cgs_get_active_displays_info(struct cgs_device *cgs_device,
struct cgs_display_info *info)
{
CGS_FUNC_ADEV;
@@ -851,7 +869,7 @@
}
-static int amdgpu_cgs_notify_dpm_enabled(void *cgs_device, bool enabled)
+static int amdgpu_cgs_notify_dpm_enabled(struct cgs_device *cgs_device, bool enabled)
{
CGS_FUNC_ADEV;
@@ -867,7 +885,7 @@
*/
#if defined(CONFIG_ACPI)
-static int amdgpu_cgs_acpi_eval_object(void *cgs_device,
+static int amdgpu_cgs_acpi_eval_object(struct cgs_device *cgs_device,
struct cgs_acpi_method_info *info)
{
CGS_FUNC_ADEV;
@@ -1030,14 +1048,14 @@
return result;
}
#else
-static int amdgpu_cgs_acpi_eval_object(void *cgs_device,
+static int amdgpu_cgs_acpi_eval_object(struct cgs_device *cgs_device,
struct cgs_acpi_method_info *info)
{
return -EIO;
}
#endif
-int amdgpu_cgs_call_acpi_method(void *cgs_device,
+int amdgpu_cgs_call_acpi_method(struct cgs_device *cgs_device,
uint32_t acpi_method,
uint32_t acpi_function,
void *pinput, void *poutput,
@@ -1121,7 +1139,7 @@
amdgpu_cgs_irq_put
};
-void *amdgpu_cgs_create_device(struct amdgpu_device *adev)
+struct cgs_device *amdgpu_cgs_create_device(struct amdgpu_device *adev)
{
struct amdgpu_cgs_device *cgs_device =
kmalloc(sizeof(*cgs_device), GFP_KERNEL);
@@ -1135,10 +1153,10 @@
cgs_device->base.os_ops = &amdgpu_cgs_os_ops;
cgs_device->adev = adev;
- return cgs_device;
+ return (struct cgs_device *)cgs_device;
}
-void amdgpu_cgs_destroy_device(void *cgs_device)
+void amdgpu_cgs_destroy_device(struct cgs_device *cgs_device)
{
kfree(cgs_device);
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_connectors.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_connectors.c
index 119cdc2..60a0c9a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_connectors.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_connectors.c
@@ -439,7 +439,7 @@
struct drm_display_mode *mode = NULL;
struct drm_display_mode *native_mode = &amdgpu_encoder->native_mode;
int i;
- struct mode_size {
+ static const struct mode_size {
int w;
int h;
} common_modes[17] = {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index 9392e50..9bc8f1d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -24,7 +24,6 @@
* Authors:
* Jerome Glisse <glisse@freedesktop.org>
*/
-#include <linux/list_sort.h>
#include <linux/pagemap.h>
#include <drm/drmP.h>
#include <drm/amdgpu_drm.h>
@@ -88,44 +87,41 @@
}
static int amdgpu_cs_user_fence_chunk(struct amdgpu_cs_parser *p,
- struct amdgpu_user_fence *uf,
- struct drm_amdgpu_cs_chunk_fence *fence_data)
+ struct drm_amdgpu_cs_chunk_fence *data,
+ uint32_t *offset)
{
struct drm_gem_object *gobj;
- uint32_t handle;
- handle = fence_data->handle;
- gobj = drm_gem_object_lookup(p->adev->ddev, p->filp,
- fence_data->handle);
+ gobj = drm_gem_object_lookup(p->filp, data->handle);
if (gobj == NULL)
return -EINVAL;
- uf->bo = amdgpu_bo_ref(gem_to_amdgpu_bo(gobj));
- uf->offset = fence_data->offset;
-
- if (amdgpu_ttm_tt_get_usermm(uf->bo->tbo.ttm)) {
- drm_gem_object_unreference_unlocked(gobj);
- return -EINVAL;
- }
-
- p->uf_entry.robj = amdgpu_bo_ref(uf->bo);
+ p->uf_entry.robj = amdgpu_bo_ref(gem_to_amdgpu_bo(gobj));
p->uf_entry.priority = 0;
p->uf_entry.tv.bo = &p->uf_entry.robj->tbo;
p->uf_entry.tv.shared = true;
p->uf_entry.user_pages = NULL;
+ *offset = data->offset;
drm_gem_object_unreference_unlocked(gobj);
+
+ if (amdgpu_ttm_tt_get_usermm(p->uf_entry.robj->tbo.ttm)) {
+ amdgpu_bo_unref(&p->uf_entry.robj);
+ return -EINVAL;
+ }
+
return 0;
}
int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, void *data)
{
struct amdgpu_fpriv *fpriv = p->filp->driver_priv;
+ struct amdgpu_vm *vm = &fpriv->vm;
union drm_amdgpu_cs *cs = data;
uint64_t *chunk_array_user;
uint64_t *chunk_array;
- struct amdgpu_user_fence uf = {};
unsigned size, num_ibs = 0;
+ uint32_t uf_offset = 0;
int i;
int ret;
@@ -200,7 +196,8 @@
goto free_partial_kdata;
}
- ret = amdgpu_cs_user_fence_chunk(p, &uf, (void *)p->chunks[i].kdata);
+ ret = amdgpu_cs_user_fence_chunk(p, p->chunks[i].kdata,
+ &uf_offset);
if (ret)
goto free_partial_kdata;
@@ -215,11 +212,14 @@
}
}
- ret = amdgpu_job_alloc(p->adev, num_ibs, &p->job);
+ ret = amdgpu_job_alloc(p->adev, num_ibs, &p->job, vm);
if (ret)
goto free_all_kdata;
- p->job->uf = uf;
+ if (p->uf_entry.robj) {
+ p->job->uf_bo = amdgpu_bo_ref(p->uf_entry.robj);
+ p->job->uf_offset = uf_offset;
+ }
kfree(chunk_array);
return 0;
@@ -377,7 +377,7 @@
INIT_LIST_HEAD(&duplicates);
amdgpu_vm_get_pd_bo(&fpriv->vm, &p->validated, &p->vm_pd);
- if (p->job->uf.bo)
+ if (p->uf_entry.robj)
list_add(&p->uf_entry.tv.head, &p->validated);
if (need_mmap_lock)
@@ -473,6 +473,9 @@
goto error_validate;
if (p->bo_list) {
+ struct amdgpu_bo *gds = p->bo_list->gds_obj;
+ struct amdgpu_bo *gws = p->bo_list->gws_obj;
+ struct amdgpu_bo *oa = p->bo_list->oa_obj;
struct amdgpu_vm *vm = &fpriv->vm;
unsigned i;
@@ -481,6 +484,19 @@
p->bo_list->array[i].bo_va = amdgpu_vm_bo_find(vm, bo);
}
+
+ if (gds) {
+ p->job->gds_base = amdgpu_bo_gpu_offset(gds);
+ p->job->gds_size = amdgpu_bo_size(gds);
+ }
+ if (gws) {
+ p->job->gws_base = amdgpu_bo_gpu_offset(gws);
+ p->job->gws_size = amdgpu_bo_size(gws);
+ }
+ if (oa) {
+ p->job->oa_base = amdgpu_bo_gpu_offset(oa);
+ p->job->oa_size = amdgpu_bo_size(oa);
+ }
}
error_validate:
@@ -527,16 +543,6 @@
return 0;
}
-static int cmp_size_smaller_first(void *priv, struct list_head *a,
- struct list_head *b)
-{
- struct amdgpu_bo_list_entry *la = list_entry(a, struct amdgpu_bo_list_entry, tv.head);
- struct amdgpu_bo_list_entry *lb = list_entry(b, struct amdgpu_bo_list_entry, tv.head);
-
- /* Sort A before B if A is smaller. */
- return (int)la->robj->tbo.num_pages - (int)lb->robj->tbo.num_pages;
-}
-
/**
* cs_parser_fini() - clean parser states
* @parser: parser structure holding parsing context.
@@ -553,18 +559,6 @@
if (!error) {
amdgpu_vm_move_pt_bos_in_lru(parser->adev, &fpriv->vm);
- /* Sort the buffer list from the smallest to largest buffer,
- * which affects the order of buffers in the LRU list.
- * This assures that the smallest buffers are added first
- * to the LRU list, so they are likely to be later evicted
- * first, instead of large buffers whose eviction is more
- * expensive.
- *
- * This slightly lowers the number of bytes moved by TTM
- * per frame under memory pressure.
- */
- list_sort(NULL, &parser->validated, cmp_size_smaller_first);
-
ttm_eu_fence_buffer_objects(&parser->ticket,
&parser->validated,
parser->fence);
@@ -763,41 +757,14 @@
ib->length_dw = chunk_ib->ib_bytes / 4;
ib->flags = chunk_ib->flags;
- ib->ctx = parser->ctx;
j++;
}
- /* add GDS resources to first IB */
- if (parser->bo_list) {
- struct amdgpu_bo *gds = parser->bo_list->gds_obj;
- struct amdgpu_bo *gws = parser->bo_list->gws_obj;
- struct amdgpu_bo *oa = parser->bo_list->oa_obj;
- struct amdgpu_ib *ib = &parser->job->ibs[0];
-
- if (gds) {
- ib->gds_base = amdgpu_bo_gpu_offset(gds);
- ib->gds_size = amdgpu_bo_size(gds);
- }
- if (gws) {
- ib->gws_base = amdgpu_bo_gpu_offset(gws);
- ib->gws_size = amdgpu_bo_size(gws);
- }
- if (oa) {
- ib->oa_base = amdgpu_bo_gpu_offset(oa);
- ib->oa_size = amdgpu_bo_size(oa);
- }
- }
- /* wrap the last IB with user fence */
- if (parser->job->uf.bo) {
- struct amdgpu_ib *ib = &parser->job->ibs[parser->job->num_ibs - 1];
-
- /* UVD & VCE fw doesn't support user fences */
- if (parser->job->ring->type == AMDGPU_RING_TYPE_UVD ||
- parser->job->ring->type == AMDGPU_RING_TYPE_VCE)
- return -EINVAL;
-
- ib->user = &parser->job->uf;
- }
+ /* UVD & VCE fw doesn't support user fences */
+ if (parser->job->uf_bo && (
+ parser->job->ring->type == AMDGPU_RING_TYPE_UVD ||
+ parser->job->ring->type == AMDGPU_RING_TYPE_VCE))
+ return -EINVAL;
return 0;
}
@@ -862,28 +829,28 @@
union drm_amdgpu_cs *cs)
{
struct amdgpu_ring *ring = p->job->ring;
- struct amd_sched_fence *fence;
+ struct amd_sched_entity *entity = &p->ctx->rings[ring->idx].entity;
+ struct fence *fence;
struct amdgpu_job *job;
+ int r;
job = p->job;
p->job = NULL;
- job->base.sched = &ring->sched;
- job->base.s_entity = &p->ctx->rings[ring->idx].entity;
- job->owner = p->filp;
-
- fence = amd_sched_fence_create(job->base.s_entity, p->filp);
- if (!fence) {
+ r = amd_sched_job_init(&job->base, &ring->sched,
+ entity, amdgpu_job_timeout_func,
+ amdgpu_job_free_func,
+ p->filp, &fence);
+ if (r) {
amdgpu_job_free(job);
- return -ENOMEM;
+ return r;
}
- job->base.s_fence = fence;
- p->fence = fence_get(&fence->base);
-
- cs->out.handle = amdgpu_ctx_add_fence(p->ctx, ring,
- &fence->base);
- job->ibs[job->num_ibs - 1].sequence = cs->out.handle;
+ job->owner = p->filp;
+ job->ctx = entity->fence_context;
+ p->fence = fence_get(fence);
+ cs->out.handle = amdgpu_ctx_add_fence(p->ctx, ring, fence);
+ job->uf_sequence = cs->out.handle;
trace_amdgpu_cs_ioctl(job);
amd_sched_entity_push_job(&job->base);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 2139da7..bb8b149 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -59,6 +59,8 @@
"FIJI",
"CARRIZO",
"STONEY",
+ "POLARIS10",
+ "POLARIS11",
"LAST",
};
@@ -346,7 +348,7 @@
adev->doorbell.base = pci_resource_start(adev->pdev, 2);
adev->doorbell.size = pci_resource_len(adev->pdev, 2);
- adev->doorbell.num_doorbells = min_t(u32, adev->doorbell.size / sizeof(u32),
+ adev->doorbell.num_doorbells = min_t(u32, adev->doorbell.size / sizeof(u32),
AMDGPU_DOORBELL_MAX_ASSIGNMENT+1);
if (adev->doorbell.num_doorbells == 0)
return -EINVAL;
@@ -936,15 +938,11 @@
}
if (amdgpu_gart_size != -1) {
- /* gtt size must be power of two and greater or equal to 32M */
+ /* gtt size must be greater or equal to 32M */
if (amdgpu_gart_size < 32) {
dev_warn(adev->dev, "gart size (%d) too small\n",
amdgpu_gart_size);
amdgpu_gart_size = -1;
- } else if (!amdgpu_check_pot_argument(amdgpu_gart_size)) {
- dev_warn(adev->dev, "gart size (%d) must be a power of 2\n",
- amdgpu_gart_size);
- amdgpu_gart_size = -1;
}
}
@@ -1144,6 +1142,8 @@
case CHIP_TOPAZ:
case CHIP_TONGA:
case CHIP_FIJI:
+ case CHIP_POLARIS11:
+ case CHIP_POLARIS10:
case CHIP_CARRIZO:
case CHIP_STONEY:
if (adev->asic_type == CHIP_CARRIZO || adev->asic_type == CHIP_STONEY)
@@ -1196,7 +1196,7 @@
if (r == -ENOENT) {
adev->ip_block_status[i].valid = false;
} else if (r) {
- DRM_ERROR("early_init %d failed %d\n", i, r);
+ DRM_ERROR("early_init of IP block <%s> failed %d\n", adev->ip_blocks[i].funcs->name, r);
return r;
} else {
adev->ip_block_status[i].valid = true;
@@ -1219,7 +1219,7 @@
continue;
r = adev->ip_blocks[i].funcs->sw_init((void *)adev);
if (r) {
- DRM_ERROR("sw_init %d failed %d\n", i, r);
+ DRM_ERROR("sw_init of IP block <%s> failed %d\n", adev->ip_blocks[i].funcs->name, r);
return r;
}
adev->ip_block_status[i].sw = true;
@@ -1252,7 +1252,7 @@
continue;
r = adev->ip_blocks[i].funcs->hw_init((void *)adev);
if (r) {
- DRM_ERROR("hw_init %d failed %d\n", i, r);
+ DRM_ERROR("hw_init of IP block <%s> failed %d\n", adev->ip_blocks[i].funcs->name, r);
return r;
}
adev->ip_block_status[i].hw = true;
@@ -1272,13 +1272,13 @@
r = adev->ip_blocks[i].funcs->set_clockgating_state((void *)adev,
AMD_CG_STATE_GATE);
if (r) {
- DRM_ERROR("set_clockgating_state(gate) %d failed %d\n", i, r);
+ DRM_ERROR("set_clockgating_state(gate) of IP block <%s> failed %d\n", adev->ip_blocks[i].funcs->name, r);
return r;
}
if (adev->ip_blocks[i].funcs->late_init) {
r = adev->ip_blocks[i].funcs->late_init((void *)adev);
if (r) {
- DRM_ERROR("late_init %d failed %d\n", i, r);
+ DRM_ERROR("late_init of IP block <%s> failed %d\n", adev->ip_blocks[i].funcs->name, r);
return r;
}
}
@@ -1302,13 +1302,13 @@
r = adev->ip_blocks[i].funcs->set_clockgating_state((void *)adev,
AMD_CG_STATE_UNGATE);
if (r) {
- DRM_ERROR("set_clockgating_state(ungate) %d failed %d\n", i, r);
+ DRM_ERROR("set_clockgating_state(ungate) of IP block <%s> failed %d\n", adev->ip_blocks[i].funcs->name, r);
return r;
}
r = adev->ip_blocks[i].funcs->hw_fini((void *)adev);
/* XXX handle errors */
if (r) {
- DRM_DEBUG("hw_fini %d failed %d\n", i, r);
+ DRM_DEBUG("hw_fini of IP block <%s> failed %d\n", adev->ip_blocks[i].funcs->name, r);
}
adev->ip_block_status[i].hw = false;
}
@@ -1319,7 +1319,7 @@
r = adev->ip_blocks[i].funcs->sw_fini((void *)adev);
/* XXX handle errors */
if (r) {
- DRM_DEBUG("sw_fini %d failed %d\n", i, r);
+ DRM_DEBUG("sw_fini of IP block <%s> failed %d\n", adev->ip_blocks[i].funcs->name, r);
}
adev->ip_block_status[i].sw = false;
adev->ip_block_status[i].valid = false;
@@ -1332,20 +1332,29 @@
{
int i, r;
+ /* ungate SMC block first */
+ r = amdgpu_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_SMC,
+ AMD_CG_STATE_UNGATE);
+ if (r) {
+ DRM_ERROR("set_clockgating_state(ungate) SMC failed %d\n",r);
+ }
+
for (i = adev->num_ip_blocks - 1; i >= 0; i--) {
if (!adev->ip_block_status[i].valid)
continue;
/* ungate blocks so that suspend can properly shut them down */
- r = adev->ip_blocks[i].funcs->set_clockgating_state((void *)adev,
- AMD_CG_STATE_UNGATE);
- if (r) {
- DRM_ERROR("set_clockgating_state(ungate) %d failed %d\n", i, r);
+ if (i != AMD_IP_BLOCK_TYPE_SMC) {
+ r = adev->ip_blocks[i].funcs->set_clockgating_state((void *)adev,
+ AMD_CG_STATE_UNGATE);
+ if (r) {
+ DRM_ERROR("set_clockgating_state(ungate) of IP block <%s> failed %d\n", adev->ip_blocks[i].funcs->name, r);
+ }
}
/* XXX handle errors */
r = adev->ip_blocks[i].funcs->suspend(adev);
/* XXX handle errors */
if (r) {
- DRM_ERROR("suspend %d failed %d\n", i, r);
+ DRM_ERROR("suspend of IP block <%s> failed %d\n", adev->ip_blocks[i].funcs->name, r);
}
}
@@ -1361,7 +1370,7 @@
continue;
r = adev->ip_blocks[i].funcs->resume(adev);
if (r) {
- DRM_ERROR("resume %d failed %d\n", i, r);
+ DRM_ERROR("resume of IP block <%s> failed %d\n", adev->ip_blocks[i].funcs->name, r);
return r;
}
}
@@ -2007,7 +2016,7 @@
* Debugfs
*/
int amdgpu_debugfs_add_files(struct amdgpu_device *adev,
- struct drm_info_list *files,
+ const struct drm_info_list *files,
unsigned nfiles)
{
unsigned i;
@@ -2119,32 +2128,246 @@
return result;
}
+static ssize_t amdgpu_debugfs_regs_pcie_read(struct file *f, char __user *buf,
+ size_t size, loff_t *pos)
+{
+ struct amdgpu_device *adev = f->f_inode->i_private;
+ ssize_t result = 0;
+ int r;
+
+ if (size & 0x3 || *pos & 0x3)
+ return -EINVAL;
+
+ while (size) {
+ uint32_t value;
+
+ value = RREG32_PCIE(*pos >> 2);
+ r = put_user(value, (uint32_t *)buf);
+ if (r)
+ return r;
+
+ result += 4;
+ buf += 4;
+ *pos += 4;
+ size -= 4;
+ }
+
+ return result;
+}
+
+static ssize_t amdgpu_debugfs_regs_pcie_write(struct file *f, const char __user *buf,
+ size_t size, loff_t *pos)
+{
+ struct amdgpu_device *adev = f->f_inode->i_private;
+ ssize_t result = 0;
+ int r;
+
+ if (size & 0x3 || *pos & 0x3)
+ return -EINVAL;
+
+ while (size) {
+ uint32_t value;
+
+ r = get_user(value, (uint32_t *)buf);
+ if (r)
+ return r;
+
+ WREG32_PCIE(*pos >> 2, value);
+
+ result += 4;
+ buf += 4;
+ *pos += 4;
+ size -= 4;
+ }
+
+ return result;
+}
+
+static ssize_t amdgpu_debugfs_regs_didt_read(struct file *f, char __user *buf,
+ size_t size, loff_t *pos)
+{
+ struct amdgpu_device *adev = f->f_inode->i_private;
+ ssize_t result = 0;
+ int r;
+
+ if (size & 0x3 || *pos & 0x3)
+ return -EINVAL;
+
+ while (size) {
+ uint32_t value;
+
+ value = RREG32_DIDT(*pos >> 2);
+ r = put_user(value, (uint32_t *)buf);
+ if (r)
+ return r;
+
+ result += 4;
+ buf += 4;
+ *pos += 4;
+ size -= 4;
+ }
+
+ return result;
+}
+
+static ssize_t amdgpu_debugfs_regs_didt_write(struct file *f, const char __user *buf,
+ size_t size, loff_t *pos)
+{
+ struct amdgpu_device *adev = f->f_inode->i_private;
+ ssize_t result = 0;
+ int r;
+
+ if (size & 0x3 || *pos & 0x3)
+ return -EINVAL;
+
+ while (size) {
+ uint32_t value;
+
+ r = get_user(value, (uint32_t *)buf);
+ if (r)
+ return r;
+
+ WREG32_DIDT(*pos >> 2, value);
+
+ result += 4;
+ buf += 4;
+ *pos += 4;
+ size -= 4;
+ }
+
+ return result;
+}
+
+static ssize_t amdgpu_debugfs_regs_smc_read(struct file *f, char __user *buf,
+ size_t size, loff_t *pos)
+{
+ struct amdgpu_device *adev = f->f_inode->i_private;
+ ssize_t result = 0;
+ int r;
+
+ if (size & 0x3 || *pos & 0x3)
+ return -EINVAL;
+
+ while (size) {
+ uint32_t value;
+
+ value = RREG32_SMC(*pos >> 2);
+ r = put_user(value, (uint32_t *)buf);
+ if (r)
+ return r;
+
+ result += 4;
+ buf += 4;
+ *pos += 4;
+ size -= 4;
+ }
+
+ return result;
+}
+
+static ssize_t amdgpu_debugfs_regs_smc_write(struct file *f, const char __user *buf,
+ size_t size, loff_t *pos)
+{
+ struct amdgpu_device *adev = f->f_inode->i_private;
+ ssize_t result = 0;
+ int r;
+
+ if (size & 0x3 || *pos & 0x3)
+ return -EINVAL;
+
+ while (size) {
+ uint32_t value;
+
+ r = get_user(value, (uint32_t *)buf);
+ if (r)
+ return r;
+
+ WREG32_SMC(*pos >> 2, value);
+
+ result += 4;
+ buf += 4;
+ *pos += 4;
+ size -= 4;
+ }
+
+ return result;
+}
+
static const struct file_operations amdgpu_debugfs_regs_fops = {
.owner = THIS_MODULE,
.read = amdgpu_debugfs_regs_read,
.write = amdgpu_debugfs_regs_write,
.llseek = default_llseek
};
+static const struct file_operations amdgpu_debugfs_regs_didt_fops = {
+ .owner = THIS_MODULE,
+ .read = amdgpu_debugfs_regs_didt_read,
+ .write = amdgpu_debugfs_regs_didt_write,
+ .llseek = default_llseek
+};
+static const struct file_operations amdgpu_debugfs_regs_pcie_fops = {
+ .owner = THIS_MODULE,
+ .read = amdgpu_debugfs_regs_pcie_read,
+ .write = amdgpu_debugfs_regs_pcie_write,
+ .llseek = default_llseek
+};
+static const struct file_operations amdgpu_debugfs_regs_smc_fops = {
+ .owner = THIS_MODULE,
+ .read = amdgpu_debugfs_regs_smc_read,
+ .write = amdgpu_debugfs_regs_smc_write,
+ .llseek = default_llseek
+};
+
+static const struct file_operations *debugfs_regs[] = {
+ &amdgpu_debugfs_regs_fops,
+ &amdgpu_debugfs_regs_didt_fops,
+ &amdgpu_debugfs_regs_pcie_fops,
+ &amdgpu_debugfs_regs_smc_fops,
+};
+
+static const char *debugfs_regs_names[] = {
+ "amdgpu_regs",
+ "amdgpu_regs_didt",
+ "amdgpu_regs_pcie",
+ "amdgpu_regs_smc",
+};
static int amdgpu_debugfs_regs_init(struct amdgpu_device *adev)
{
struct drm_minor *minor = adev->ddev->primary;
struct dentry *ent, *root = minor->debugfs_root;
+ unsigned i, j;
- ent = debugfs_create_file("amdgpu_regs", S_IFREG | S_IRUGO, root,
- adev, &amdgpu_debugfs_regs_fops);
- if (IS_ERR(ent))
- return PTR_ERR(ent);
- i_size_write(ent->d_inode, adev->rmmio_size);
- adev->debugfs_regs = ent;
+ for (i = 0; i < ARRAY_SIZE(debugfs_regs); i++) {
+ ent = debugfs_create_file(debugfs_regs_names[i],
+ S_IFREG | S_IRUGO, root,
+ adev, debugfs_regs[i]);
+ if (IS_ERR(ent)) {
+ for (j = 0; j < i; j++) {
+ debugfs_remove(adev->debugfs_regs[i]);
+ adev->debugfs_regs[i] = NULL;
+ }
+ return PTR_ERR(ent);
+ }
+
+ if (!i)
+ i_size_write(ent->d_inode, adev->rmmio_size);
+ adev->debugfs_regs[i] = ent;
+ }
return 0;
}
static void amdgpu_debugfs_regs_cleanup(struct amdgpu_device *adev)
{
- debugfs_remove(adev->debugfs_regs);
- adev->debugfs_regs = NULL;
+ unsigned i;
+
+ for (i = 0; i < ARRAY_SIZE(debugfs_regs); i++) {
+ if (adev->debugfs_regs[i]) {
+ debugfs_remove(adev->debugfs_regs[i]);
+ adev->debugfs_regs[i] = NULL;
+ }
+ }
}
int amdgpu_debugfs_init(struct drm_minor *minor)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_display.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_display.c
index 3fb405b..b0832da 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_display.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_display.c
@@ -131,12 +131,17 @@
vblank->framedur_ns / 1000,
vblank->linedur_ns / 1000, stat, vpos, hpos);
- /* set the flip status */
+ /* Do the flip (mmio) */
+ adev->mode_info.funcs->page_flip(adev, work->crtc_id, work->base, work->async);
+
+ /* Set the flip status */
amdgpuCrtc->pflip_status = AMDGPU_FLIP_SUBMITTED;
spin_unlock_irqrestore(&crtc->dev->event_lock, flags);
- /* Do the flip (mmio) */
- adev->mode_info.funcs->page_flip(adev, work->crtc_id, work->base);
+
+ DRM_DEBUG_DRIVER("crtc:%d[%p], pflip_stat:AMDGPU_FLIP_SUBMITTED, work: %p,\n",
+ amdgpuCrtc->crtc_id, amdgpuCrtc, work);
+
}
/*
@@ -192,6 +197,7 @@
work->event = event;
work->adev = adev;
work->crtc_id = amdgpu_crtc->crtc_id;
+ work->async = (page_flip_flags & DRM_MODE_PAGE_FLIP_ASYNC) != 0;
/* schedule unpin of the old buffer */
old_amdgpu_fb = to_amdgpu_framebuffer(crtc->primary->fb);
@@ -252,6 +258,9 @@
amdgpu_crtc->pflip_status = AMDGPU_FLIP_PENDING;
amdgpu_crtc->pflip_works = work;
+
+ DRM_DEBUG_DRIVER("crtc:%d[%p], pflip_stat:AMDGPU_FLIP_PENDING, work: %p,\n",
+ amdgpu_crtc->crtc_id, amdgpu_crtc, work);
/* update crtc fb */
crtc->primary->fb = fb;
spin_unlock_irqrestore(&crtc->dev->event_lock, flags);
@@ -554,7 +563,7 @@
struct amdgpu_framebuffer *amdgpu_fb;
int ret;
- obj = drm_gem_object_lookup(dev, file_priv, mode_cmd->handles[0]);
+ obj = drm_gem_object_lookup(file_priv, mode_cmd->handles[0]);
if (obj == NULL) {
dev_err(&dev->pdev->dev, "No GEM object associated to handle 0x%08X, "
"can't create framebuffer\n", mode_cmd->handles[0]);
@@ -588,20 +597,20 @@
.output_poll_changed = amdgpu_output_poll_changed
};
-static struct drm_prop_enum_list amdgpu_underscan_enum_list[] =
+static const struct drm_prop_enum_list amdgpu_underscan_enum_list[] =
{ { UNDERSCAN_OFF, "off" },
{ UNDERSCAN_ON, "on" },
{ UNDERSCAN_AUTO, "auto" },
};
-static struct drm_prop_enum_list amdgpu_audio_enum_list[] =
+static const struct drm_prop_enum_list amdgpu_audio_enum_list[] =
{ { AMDGPU_AUDIO_DISABLE, "off" },
{ AMDGPU_AUDIO_ENABLE, "on" },
{ AMDGPU_AUDIO_AUTO, "auto" },
};
/* XXX support different dither options? spatial, temporal, both, etc. */
-static struct drm_prop_enum_list amdgpu_dither_enum_list[] =
+static const struct drm_prop_enum_list amdgpu_dither_enum_list[] =
{ { AMDGPU_FMT_DITHER_DISABLE, "off" },
{ AMDGPU_FMT_DITHER_ENABLE, "on" },
};
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dpm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dpm.c
index 7b7f4ab..fe36caf 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dpm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dpm.c
@@ -150,7 +150,7 @@
list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) {
amdgpu_crtc = to_amdgpu_crtc(crtc);
if (crtc->enabled && amdgpu_crtc->enabled && amdgpu_crtc->hw_mode.clock) {
- vrefresh = amdgpu_crtc->hw_mode.vrefresh;
+ vrefresh = drm_mode_vrefresh(&amdgpu_crtc->hw_mode);
break;
}
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index f1e17d6..1dab5f2 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -166,7 +166,7 @@
MODULE_PARM_DESC(pcie_lane_cap, "PCIE Lane Caps (0: autodetect (default))");
module_param_named(pcie_lane_cap, amdgpu_pcie_lane_cap, uint, 0444);
-static struct pci_device_id pciidlist[] = {
+static const struct pci_device_id pciidlist[] = {
#ifdef CONFIG_DRM_AMDGPU_CIK
/* Kaveri */
{0x1002, 0x1304, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_KAVERI|AMD_IS_MOBILITY|AMD_IS_APU},
@@ -277,6 +277,16 @@
{0x1002, 0x9877, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_CARRIZO|AMD_IS_APU},
/* stoney */
{0x1002, 0x98E4, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_STONEY|AMD_IS_APU},
+ /* Polaris11 */
+ {0x1002, 0x67E0, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS11},
+ {0x1002, 0x67E1, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS11},
+ {0x1002, 0x67E8, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS11},
+ {0x1002, 0x67E9, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS11},
+ {0x1002, 0x67EB, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS11},
+ {0x1002, 0x67FF, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS11},
+ /* Polaris10 */
+ {0x1002, 0x67C0, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS10},
+ {0x1002, 0x67DF, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS10},
{0, 0, 0}
};
@@ -514,7 +524,7 @@
.irq_uninstall = amdgpu_irq_uninstall,
.irq_handler = amdgpu_irq_handler,
.ioctls = amdgpu_ioctls_kms,
- .gem_free_object = amdgpu_gem_object_free,
+ .gem_free_object_unlocked = amdgpu_gem_object_free,
.gem_open_object = amdgpu_gem_object_open,
.gem_close_object = amdgpu_gem_object_close,
.dumb_create = amdgpu_mode_dumb_create,
@@ -556,12 +566,10 @@
static int __init amdgpu_init(void)
{
amdgpu_sync_init();
-#ifdef CONFIG_VGA_CONSOLE
if (vgacon_text_force()) {
DRM_ERROR("VGACON disables amdgpu kernel modesetting.\n");
return -EINVAL;
}
-#endif
DRM_INFO("amdgpu kernel modesetting enabled.\n");
driver = &kms_driver;
pdriver = &amdgpu_kms_pci_driver;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
index d81f1f4..ba9c042 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
@@ -198,7 +198,7 @@
/* There is always exactly one thread signaling this fence slot */
fence = rcu_dereference_protected(*ptr, 1);
- rcu_assign_pointer(*ptr, NULL);
+ RCU_INIT_POINTER(*ptr, NULL);
BUG_ON(!fence);
@@ -352,9 +352,9 @@
setup_timer(&ring->fence_drv.fallback_timer, amdgpu_fence_fallback,
(unsigned long)ring);
- ring->fence_drv.num_fences_mask = num_hw_submission - 1;
+ ring->fence_drv.num_fences_mask = num_hw_submission * 2 - 1;
spin_lock_init(&ring->fence_drv.lock);
- ring->fence_drv.fences = kcalloc(num_hw_submission, sizeof(void *),
+ ring->fence_drv.fences = kcalloc(num_hw_submission * 2, sizeof(void *),
GFP_KERNEL);
if (!ring->fence_drv.fences)
return -ENOMEM;
@@ -639,7 +639,7 @@
return 0;
}
-static struct drm_info_list amdgpu_debugfs_fence_list[] = {
+static const struct drm_info_list amdgpu_debugfs_fence_list[] = {
{"amdgpu_fence_info", &amdgpu_debugfs_fence_info, 0, NULL},
{"amdgpu_gpu_reset", &amdgpu_debugfs_gpu_reset, 0, NULL}
};
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
index 7312d72..921bce2 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
@@ -238,18 +238,17 @@
t = offset / AMDGPU_GPU_PAGE_SIZE;
p = t / (PAGE_SIZE / AMDGPU_GPU_PAGE_SIZE);
for (i = 0; i < pages; i++, p++) {
- if (adev->gart.pages[p]) {
- adev->gart.pages[p] = NULL;
- adev->gart.pages_addr[p] = adev->dummy_page.addr;
- page_base = adev->gart.pages_addr[p];
- if (!adev->gart.ptr)
- continue;
+#ifdef CONFIG_AMDGPU_GART_DEBUGFS
+ adev->gart.pages[p] = NULL;
+#endif
+ page_base = adev->dummy_page.addr;
+ if (!adev->gart.ptr)
+ continue;
- for (j = 0; j < (PAGE_SIZE / AMDGPU_GPU_PAGE_SIZE); j++, t++) {
- amdgpu_gart_set_pte_pde(adev, adev->gart.ptr,
- t, page_base, flags);
- page_base += AMDGPU_GPU_PAGE_SIZE;
- }
+ for (j = 0; j < (PAGE_SIZE / AMDGPU_GPU_PAGE_SIZE); j++, t++) {
+ amdgpu_gart_set_pte_pde(adev, adev->gart.ptr,
+ t, page_base, flags);
+ page_base += AMDGPU_GPU_PAGE_SIZE;
}
}
mb();
@@ -287,10 +286,11 @@
p = t / (PAGE_SIZE / AMDGPU_GPU_PAGE_SIZE);
for (i = 0; i < pages; i++, p++) {
- adev->gart.pages_addr[p] = dma_addr[i];
+#ifdef CONFIG_AMDGPU_GART_DEBUGFS
adev->gart.pages[p] = pagelist[i];
+#endif
if (adev->gart.ptr) {
- page_base = adev->gart.pages_addr[p];
+ page_base = dma_addr[i];
for (j = 0; j < (PAGE_SIZE / AMDGPU_GPU_PAGE_SIZE); j++, t++) {
amdgpu_gart_set_pte_pde(adev, adev->gart.ptr, t, page_base, flags);
page_base += AMDGPU_GPU_PAGE_SIZE;
@@ -312,11 +312,11 @@
*/
int amdgpu_gart_init(struct amdgpu_device *adev)
{
- int r, i;
+ int r;
- if (adev->gart.pages) {
+ if (adev->dummy_page.page)
return 0;
- }
+
/* We need PAGE_SIZE >= AMDGPU_GPU_PAGE_SIZE */
if (PAGE_SIZE < AMDGPU_GPU_PAGE_SIZE) {
DRM_ERROR("Page size is smaller than GPU page size!\n");
@@ -330,22 +330,16 @@
adev->gart.num_gpu_pages = adev->mc.gtt_size / AMDGPU_GPU_PAGE_SIZE;
DRM_INFO("GART: num cpu pages %u, num gpu pages %u\n",
adev->gart.num_cpu_pages, adev->gart.num_gpu_pages);
+
+#ifdef CONFIG_AMDGPU_GART_DEBUGFS
/* Allocate pages table */
adev->gart.pages = vzalloc(sizeof(void *) * adev->gart.num_cpu_pages);
if (adev->gart.pages == NULL) {
amdgpu_gart_fini(adev);
return -ENOMEM;
}
- adev->gart.pages_addr = vzalloc(sizeof(dma_addr_t) *
- adev->gart.num_cpu_pages);
- if (adev->gart.pages_addr == NULL) {
- amdgpu_gart_fini(adev);
- return -ENOMEM;
- }
- /* set GART entry to point to the dummy page by default */
- for (i = 0; i < adev->gart.num_cpu_pages; i++) {
- adev->gart.pages_addr[i] = adev->dummy_page.addr;
- }
+#endif
+
return 0;
}
@@ -358,15 +352,14 @@
*/
void amdgpu_gart_fini(struct amdgpu_device *adev)
{
- if (adev->gart.pages && adev->gart.pages_addr && adev->gart.ready) {
+ if (adev->gart.ready) {
/* unbind pages */
amdgpu_gart_unbind(adev, 0, adev->gart.num_cpu_pages);
}
adev->gart.ready = false;
+#ifdef CONFIG_AMDGPU_GART_DEBUGFS
vfree(adev->gart.pages);
- vfree(adev->gart.pages_addr);
adev->gart.pages = NULL;
- adev->gart.pages_addr = NULL;
-
+#endif
amdgpu_dummy_page_fini(adev);
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gds.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gds.h
index c3f4e85..503d540 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gds.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gds.h
@@ -43,7 +43,7 @@
struct amdgpu_bo;
struct amdgpu_gds_asic_info {
- uint32_t total_size;
+ uint32_t total_size;
uint32_t gfx_partition_size;
uint32_t cs_partition_size;
};
@@ -52,8 +52,8 @@
struct amdgpu_gds_asic_info mem;
struct amdgpu_gds_asic_info gws;
struct amdgpu_gds_asic_info oa;
- /* At present, GDS, GWS and OA resources for gfx (graphics)
- * is always pre-allocated and available for graphics operation.
+ /* At present, GDS, GWS and OA resources for gfx (graphics)
+ * is always pre-allocated and available for graphics operation.
* Such resource is shared between all gfx clients.
* TODO: move this operation to user space
* */
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
index fa6a27b..8fab648 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
@@ -93,7 +93,7 @@
struct drm_device *ddev = adev->ddev;
struct drm_file *file;
- mutex_lock(&ddev->struct_mutex);
+ mutex_lock(&ddev->filelist_mutex);
list_for_each_entry(file, &ddev->filelist, lhead) {
struct drm_gem_object *gobj;
@@ -103,13 +103,13 @@
spin_lock(&file->table_lock);
idr_for_each_entry(&file->object_idr, gobj, handle) {
WARN_ONCE(1, "And also active allocations!\n");
- drm_gem_object_unreference(gobj);
+ drm_gem_object_unreference_unlocked(gobj);
}
idr_destroy(&file->object_idr);
spin_unlock(&file->table_lock);
}
- mutex_unlock(&ddev->struct_mutex);
+ mutex_unlock(&ddev->filelist_mutex);
}
/*
@@ -338,7 +338,7 @@
struct drm_gem_object *gobj;
struct amdgpu_bo *robj;
- gobj = drm_gem_object_lookup(dev, filp, handle);
+ gobj = drm_gem_object_lookup(filp, handle);
if (gobj == NULL) {
return -ENOENT;
}
@@ -402,7 +402,7 @@
int r = 0;
long ret;
- gobj = drm_gem_object_lookup(dev, filp, handle);
+ gobj = drm_gem_object_lookup(filp, handle);
if (gobj == NULL) {
return -ENOENT;
}
@@ -436,7 +436,7 @@
int r = -1;
DRM_DEBUG("%d \n", args->handle);
- gobj = drm_gem_object_lookup(dev, filp, args->handle);
+ gobj = drm_gem_object_lookup(filp, args->handle);
if (gobj == NULL)
return -ENOENT;
robj = gem_to_amdgpu_bo(gobj);
@@ -584,7 +584,7 @@
return -EINVAL;
}
- gobj = drm_gem_object_lookup(dev, filp, args->handle);
+ gobj = drm_gem_object_lookup(filp, args->handle);
if (gobj == NULL)
return -ENOENT;
rbo = gem_to_amdgpu_bo(gobj);
@@ -646,7 +646,7 @@
struct amdgpu_bo *robj;
int r;
- gobj = drm_gem_object_lookup(dev, filp, args->handle);
+ gobj = drm_gem_object_lookup(filp, args->handle);
if (gobj == NULL) {
return -ENOENT;
}
@@ -769,7 +769,7 @@
struct drm_file *file;
int r;
- r = mutex_lock_interruptible(&dev->struct_mutex);
+ r = mutex_lock_interruptible(&dev->filelist_mutex);
if (r)
return r;
@@ -793,11 +793,11 @@
spin_unlock(&file->table_lock);
}
- mutex_unlock(&dev->struct_mutex);
+ mutex_unlock(&dev->filelist_mutex);
return 0;
}
-static struct drm_info_list amdgpu_debugfs_gem_list[] = {
+static const struct drm_info_list amdgpu_debugfs_gem_list[] = {
{"amdgpu_gem_info", &amdgpu_debugfs_gem_info, 0, NULL},
};
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c
index 8443cea..34e3542 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c
@@ -74,9 +74,6 @@
ib->gpu_addr = amdgpu_sa_bo_gpu_addr(ib->sa_bo);
}
- ib->vm = vm;
- ib->vm_id = 0;
-
return 0;
}
@@ -89,7 +86,8 @@
*
* Free an IB (all asics).
*/
-void amdgpu_ib_free(struct amdgpu_device *adev, struct amdgpu_ib *ib, struct fence *f)
+void amdgpu_ib_free(struct amdgpu_device *adev, struct amdgpu_ib *ib,
+ struct fence *f)
{
amdgpu_sa_bo_free(adev, &ib->sa_bo, f);
}
@@ -117,28 +115,37 @@
*/
int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs,
struct amdgpu_ib *ibs, struct fence *last_vm_update,
- struct fence **f)
+ struct amdgpu_job *job, struct fence **f)
{
struct amdgpu_device *adev = ring->adev;
struct amdgpu_ib *ib = &ibs[0];
- struct amdgpu_ctx *ctx, *old_ctx;
+ bool skip_preamble, need_ctx_switch;
+ unsigned patch_offset = ~0;
struct amdgpu_vm *vm;
struct fence *hwf;
+ uint64_t ctx;
+
unsigned i;
int r = 0;
if (num_ibs == 0)
return -EINVAL;
- ctx = ibs->ctx;
- vm = ibs->vm;
+ /* ring tests don't use a job */
+ if (job) {
+ vm = job->vm;
+ ctx = job->ctx;
+ } else {
+ vm = NULL;
+ ctx = 0;
+ }
if (!ring->ready) {
dev_err(adev->dev, "couldn't schedule ib\n");
return -EINVAL;
}
- if (vm && !ibs->vm_id) {
+ if (vm && !job->vm_id) {
dev_err(adev->dev, "VM IB without ID\n");
return -EINVAL;
}
@@ -149,58 +156,68 @@
return r;
}
- if (vm) {
- /* do context switch */
- amdgpu_vm_flush(ring, ib->vm_id, ib->vm_pd_addr,
- ib->gds_base, ib->gds_size,
- ib->gws_base, ib->gws_size,
- ib->oa_base, ib->oa_size);
+ if (ring->type == AMDGPU_RING_TYPE_SDMA && ring->funcs->init_cond_exec)
+ patch_offset = amdgpu_ring_init_cond_exec(ring);
- if (ring->funcs->emit_hdp_flush)
- amdgpu_ring_emit_hdp_flush(ring);
+ if (vm) {
+ r = amdgpu_vm_flush(ring, job->vm_id, job->vm_pd_addr,
+ job->gds_base, job->gds_size,
+ job->gws_base, job->gws_size,
+ job->oa_base, job->oa_size);
+ if (r) {
+ amdgpu_ring_undo(ring);
+ return r;
+ }
}
- old_ctx = ring->current_ctx;
+ if (ring->funcs->emit_hdp_flush)
+ amdgpu_ring_emit_hdp_flush(ring);
+
+ /* always set cond_exec_polling to CONTINUE */
+ *ring->cond_exe_cpu_addr = 1;
+
+ skip_preamble = ring->current_ctx == ctx;
+ need_ctx_switch = ring->current_ctx != ctx;
for (i = 0; i < num_ibs; ++i) {
ib = &ibs[i];
- if (ib->ctx != ctx || ib->vm != vm) {
- ring->current_ctx = old_ctx;
- if (ib->vm_id)
- amdgpu_vm_reset_id(adev, ib->vm_id);
- amdgpu_ring_undo(ring);
- return -EINVAL;
- }
- amdgpu_ring_emit_ib(ring, ib);
- ring->current_ctx = ctx;
+ /* drop preamble IBs if we don't have a context switch */
+ if ((ib->flags & AMDGPU_IB_FLAG_PREAMBLE) && skip_preamble)
+ continue;
+
+ amdgpu_ring_emit_ib(ring, ib, job ? job->vm_id : 0,
+ need_ctx_switch);
+ need_ctx_switch = false;
}
- if (vm) {
- if (ring->funcs->emit_hdp_invalidate)
- amdgpu_ring_emit_hdp_invalidate(ring);
- }
+ if (ring->funcs->emit_hdp_invalidate)
+ amdgpu_ring_emit_hdp_invalidate(ring);
r = amdgpu_fence_emit(ring, &hwf);
if (r) {
dev_err(adev->dev, "failed to emit fence (%d)\n", r);
- ring->current_ctx = old_ctx;
- if (ib->vm_id)
- amdgpu_vm_reset_id(adev, ib->vm_id);
+ if (job && job->vm_id)
+ amdgpu_vm_reset_id(adev, job->vm_id);
amdgpu_ring_undo(ring);
return r;
}
/* wrap the last IB with fence */
- if (ib->user) {
- uint64_t addr = amdgpu_bo_gpu_offset(ib->user->bo);
- addr += ib->user->offset;
- amdgpu_ring_emit_fence(ring, addr, ib->sequence,
+ if (job && job->uf_bo) {
+ uint64_t addr = amdgpu_bo_gpu_offset(job->uf_bo);
+
+ addr += job->uf_offset;
+ amdgpu_ring_emit_fence(ring, addr, job->uf_sequence,
AMDGPU_FENCE_FLAG_64BIT);
}
if (f)
*f = fence_get(hwf);
+ if (patch_offset != ~0 && ring->funcs->patch_cond_exec)
+ amdgpu_ring_patch_cond_exec(ring, patch_offset);
+
+ ring->current_ctx = ctx;
amdgpu_ring_commit(ring);
return 0;
}
@@ -315,7 +332,7 @@
}
-static struct drm_info_list amdgpu_debugfs_sa_list[] = {
+static const struct drm_info_list amdgpu_debugfs_sa_list[] = {
{"amdgpu_sa_info", &amdgpu_debugfs_sa_info, 0, NULL},
};
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
index 762cfdb..835a3fa 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
@@ -219,7 +219,6 @@
if (r) {
return r;
}
- adev->ddev->vblank_disable_allowed = true;
/* enable msi */
adev->irq.msi_enabled = false;
@@ -498,7 +497,7 @@
return 0;
}
-static struct irq_domain_ops amdgpu_hw_irqdomain_ops = {
+static const struct irq_domain_ops amdgpu_hw_irqdomain_ops = {
.map = amdgpu_irqdomain_map,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
index 9c9b19e..f0dafa5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
@@ -28,8 +28,25 @@
#include "amdgpu.h"
#include "amdgpu_trace.h"
+static void amdgpu_job_free_handler(struct work_struct *ws)
+{
+ struct amdgpu_job *job = container_of(ws, struct amdgpu_job, base.work_free_job);
+ amd_sched_job_put(&job->base);
+}
+
+void amdgpu_job_timeout_func(struct work_struct *work)
+{
+ struct amdgpu_job *job = container_of(work, struct amdgpu_job, base.work_tdr.work);
+ DRM_ERROR("ring %s timeout, last signaled seq=%u, last emitted seq=%u\n",
+ job->base.sched->name,
+ (uint32_t)atomic_read(&job->ring->fence_drv.last_seq),
+ job->ring->fence_drv.sync_seq);
+
+ amd_sched_job_put(&job->base);
+}
+
int amdgpu_job_alloc(struct amdgpu_device *adev, unsigned num_ibs,
- struct amdgpu_job **job)
+ struct amdgpu_job **job, struct amdgpu_vm *vm)
{
size_t size = sizeof(struct amdgpu_job);
@@ -43,8 +60,10 @@
return -ENOMEM;
(*job)->adev = adev;
+ (*job)->vm = vm;
(*job)->ibs = (void *)&(*job)[1];
(*job)->num_ibs = num_ibs;
+ INIT_WORK(&(*job)->base.work_free_job, amdgpu_job_free_handler);
amdgpu_sync_create(&(*job)->sync);
@@ -56,7 +75,7 @@
{
int r;
- r = amdgpu_job_alloc(adev, 1, job);
+ r = amdgpu_job_alloc(adev, 1, job, NULL);
if (r)
return r;
@@ -78,8 +97,16 @@
amdgpu_sa_bo_free(job->adev, &job->ibs[i].sa_bo, f);
fence_put(job->fence);
- amdgpu_bo_unref(&job->uf.bo);
+ amdgpu_bo_unref(&job->uf_bo);
amdgpu_sync_free(&job->sync);
+
+ if (!job->base.use_sched)
+ kfree(job);
+}
+
+void amdgpu_job_free_func(struct kref *refcount)
+{
+ struct amdgpu_job *job = container_of(refcount, struct amdgpu_job, base.refcount);
kfree(job);
}
@@ -87,16 +114,22 @@
struct amd_sched_entity *entity, void *owner,
struct fence **f)
{
+ struct fence *fence;
+ int r;
job->ring = ring;
- job->base.sched = &ring->sched;
- job->base.s_entity = entity;
- job->base.s_fence = amd_sched_fence_create(job->base.s_entity, owner);
- if (!job->base.s_fence)
- return -ENOMEM;
- *f = fence_get(&job->base.s_fence->base);
+ if (!f)
+ return -EINVAL;
+
+ r = amd_sched_job_init(&job->base, &ring->sched,
+ entity, amdgpu_job_timeout_func,
+ amdgpu_job_free_func, owner, &fence);
+ if (r)
+ return r;
job->owner = owner;
+ job->ctx = entity->fence_context;
+ *f = fence_get(fence);
amd_sched_entity_push_job(&job->base);
return 0;
@@ -105,27 +138,19 @@
static struct fence *amdgpu_job_dependency(struct amd_sched_job *sched_job)
{
struct amdgpu_job *job = to_amdgpu_job(sched_job);
- struct amdgpu_vm *vm = job->ibs->vm;
+ struct amdgpu_vm *vm = job->vm;
struct fence *fence = amdgpu_sync_get_fence(&job->sync);
- if (fence == NULL && vm && !job->ibs->vm_id) {
+ if (fence == NULL && vm && !job->vm_id) {
struct amdgpu_ring *ring = job->ring;
- unsigned i, vm_id;
- uint64_t vm_pd_addr;
int r;
r = amdgpu_vm_grab_id(vm, ring, &job->sync,
&job->base.s_fence->base,
- &vm_id, &vm_pd_addr);
+ &job->vm_id, &job->vm_pd_addr);
if (r)
DRM_ERROR("Error getting VM ID (%d)\n", r);
- else {
- for (i = 0; i < job->num_ibs; ++i) {
- job->ibs[i].vm_id = vm_id;
- job->ibs[i].vm_pd_addr = vm_pd_addr;
- }
- }
fence = amdgpu_sync_get_fence(&job->sync);
}
@@ -153,7 +178,7 @@
trace_amdgpu_sched_run_job(job);
r = amdgpu_ib_schedule(job->ring, job->num_ibs, job->ibs,
- job->sync.last_vm_update, &fence);
+ job->sync.last_vm_update, job, &fence);
if (r) {
DRM_ERROR("Error scheduling IBs (%d)\n", r);
goto err;
@@ -165,7 +190,9 @@
return fence;
}
-struct amd_sched_backend_ops amdgpu_sched_ops = {
+const struct amd_sched_backend_ops amdgpu_sched_ops = {
.dependency = amdgpu_job_dependency,
.run_job = amdgpu_job_run,
+ .begin_job = amd_sched_job_begin,
+ .finish_job = amd_sched_job_finish,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
index b04337d..40a2370 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
@@ -427,7 +427,6 @@
}
case AMDGPU_INFO_DEV_INFO: {
struct drm_amdgpu_info_device dev_info = {};
- struct amdgpu_cu_info cu_info;
dev_info.device_id = dev->pdev->device;
dev_info.chip_rev = adev->rev_id;
@@ -461,11 +460,11 @@
AMDGPU_GPU_PAGE_SIZE;
dev_info.gart_page_size = AMDGPU_GPU_PAGE_SIZE;
- amdgpu_asic_get_cu_info(adev, &cu_info);
- dev_info.cu_active_number = cu_info.number;
- dev_info.cu_ao_mask = cu_info.ao_cu_mask;
+ dev_info.cu_active_number = adev->gfx.cu_info.number;
+ dev_info.cu_ao_mask = adev->gfx.cu_info.ao_cu_mask;
dev_info.ce_ram_size = adev->gfx.ce_ram_size;
- memcpy(&dev_info.cu_bitmap[0], &cu_info.bitmap[0], sizeof(cu_info.bitmap));
+ memcpy(&dev_info.cu_bitmap[0], &adev->gfx.cu_info.bitmap[0],
+ sizeof(adev->gfx.cu_info.bitmap));
dev_info.vram_type = adev->mc.vram_type;
dev_info.vram_bit_width = adev->mc.vram_width;
dev_info.vce_harvest_config = adev->vce.harvest_config;
@@ -755,4 +754,4 @@
DRM_IOCTL_DEF_DRV(AMDGPU_GEM_OP, amdgpu_gem_op_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(AMDGPU_GEM_USERPTR, amdgpu_gem_userptr_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
};
-int amdgpu_max_kms_ioctl = ARRAY_SIZE(amdgpu_ioctls_kms);
+const int amdgpu_max_kms_ioctl = ARRAY_SIZE(amdgpu_ioctls_kms);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
index 9f4a45c..32fa7b7 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c
@@ -232,7 +232,10 @@
int r;
mutex_lock(&adev->mn_lock);
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem)) {
+ mutex_unlock(&adev->mn_lock);
+ return ERR_PTR(-EINTR);
+ }
hash_for_each_possible(adev->mn_hash, rmn, node, (unsigned long)mm)
if (rmn->mm == mm)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h
index 81bd964..6b1d7d3 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h
@@ -283,7 +283,7 @@
u32 (*hpd_get_gpio_reg)(struct amdgpu_device *adev);
/* pageflipping */
void (*page_flip)(struct amdgpu_device *adev,
- int crtc_id, u64 crtc_base);
+ int crtc_id, u64 crtc_base, bool async);
int (*page_flip_get_scanoutpos)(struct amdgpu_device *adev, int crtc,
u32 *vbl, u32 *position);
/* display topology setup */
@@ -530,7 +530,7 @@
((em) == ATOM_ENCODER_MODE_DP_MST))
/* Driver internal use only flags of amdgpu_get_crtc_scanoutpos() */
-#define USE_REAL_VBLANKSTART (1 << 30)
+#define USE_REAL_VBLANKSTART (1 << 30)
#define GET_DISTANCE_TO_VBLANKSTART (1 << 31)
void amdgpu_link_encoder_connector(struct drm_device *dev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
index acc0801..bdb01d9 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
@@ -71,7 +71,7 @@
{
int r;
- r = ttm_bo_reserve(&bo->tbo, !no_intr, false, false, 0);
+ r = ttm_bo_reserve(&bo->tbo, !no_intr, false, NULL);
if (unlikely(r != 0)) {
if (r != -ERESTARTSYS)
dev_err(bo->adev->dev, "%p reserve failed\n", bo);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_pm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_pm.c
index ff9597c..589b36e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_pm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_pm.c
@@ -362,16 +362,23 @@
struct amdgpu_device *adev = ddev->dev_private;
int ret;
long level;
+ uint32_t i, mask = 0;
+ char sub_str[2];
- ret = kstrtol(buf, 0, &level);
+ for (i = 0; i < strlen(buf) - 1; i++) {
+ sub_str[0] = *(buf + i);
+ sub_str[1] = '\0';
+ ret = kstrtol(sub_str, 0, &level);
- if (ret) {
- count = -EINVAL;
- goto fail;
+ if (ret) {
+ count = -EINVAL;
+ goto fail;
+ }
+ mask |= 1 << level;
}
if (adev->pp_enabled)
- amdgpu_dpm_force_clock_level(adev, PP_SCLK, level);
+ amdgpu_dpm_force_clock_level(adev, PP_SCLK, mask);
fail:
return count;
}
@@ -399,16 +406,23 @@
struct amdgpu_device *adev = ddev->dev_private;
int ret;
long level;
+ uint32_t i, mask = 0;
+ char sub_str[2];
- ret = kstrtol(buf, 0, &level);
+ for (i = 0; i < strlen(buf) - 1; i++) {
+ sub_str[0] = *(buf + i);
+ sub_str[1] = '\0';
+ ret = kstrtol(sub_str, 0, &level);
- if (ret) {
- count = -EINVAL;
- goto fail;
+ if (ret) {
+ count = -EINVAL;
+ goto fail;
+ }
+ mask |= 1 << level;
}
if (adev->pp_enabled)
- amdgpu_dpm_force_clock_level(adev, PP_MCLK, level);
+ amdgpu_dpm_force_clock_level(adev, PP_MCLK, mask);
fail:
return count;
}
@@ -436,16 +450,23 @@
struct amdgpu_device *adev = ddev->dev_private;
int ret;
long level;
+ uint32_t i, mask = 0;
+ char sub_str[2];
- ret = kstrtol(buf, 0, &level);
+ for (i = 0; i < strlen(buf) - 1; i++) {
+ sub_str[0] = *(buf + i);
+ sub_str[1] = '\0';
+ ret = kstrtol(sub_str, 0, &level);
- if (ret) {
- count = -EINVAL;
- goto fail;
+ if (ret) {
+ count = -EINVAL;
+ goto fail;
+ }
+ mask |= 1 << level;
}
if (adev->pp_enabled)
- amdgpu_dpm_force_clock_level(adev, PP_PCIE, level);
+ amdgpu_dpm_force_clock_level(adev, PP_PCIE, mask);
fail:
return count;
}
@@ -1212,7 +1233,7 @@
return 0;
}
-static struct drm_info_list amdgpu_pm_info_list[] = {
+static const struct drm_info_list amdgpu_pm_info_list[] = {
{"amdgpu_pm_info", amdgpu_debugfs_pm_info, 0, NULL},
};
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_powerplay.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_powerplay.c
index e9c6ae6..6bd961f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_powerplay.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_powerplay.c
@@ -99,6 +99,10 @@
#ifdef CONFIG_DRM_AMD_POWERPLAY
switch (adev->asic_type) {
+ case CHIP_POLARIS11:
+ case CHIP_POLARIS10:
+ adev->pp_enabled = true;
+ break;
case CHIP_TONGA:
case CHIP_FIJI:
adev->pp_enabled = (amdgpu_powerplay == 0) ? false : true;
@@ -299,16 +303,8 @@
return ret;
}
-static void amdgpu_pp_print_status(void *handle)
-{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- if (adev->powerplay.ip_funcs->print_status)
- adev->powerplay.ip_funcs->print_status(
- adev->powerplay.pp_handle);
-}
-
const struct amd_ip_funcs amdgpu_pp_ip_funcs = {
+ .name = "amdgpu_powerplay",
.early_init = amdgpu_pp_early_init,
.late_init = amdgpu_pp_late_init,
.sw_init = amdgpu_pp_sw_init,
@@ -320,7 +316,6 @@
.is_idle = amdgpu_pp_is_idle,
.wait_for_idle = amdgpu_pp_wait_for_idle,
.soft_reset = amdgpu_pp_soft_reset,
- .print_status = amdgpu_pp_print_status,
.set_clockgating_state = amdgpu_pp_set_clockgating_state,
.set_powergating_state = amdgpu_pp_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c
index be6388f..7700dc2 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c
@@ -57,9 +57,10 @@
ttm_bo_kunmap(&bo->dma_buf_vmap);
}
-struct drm_gem_object *amdgpu_gem_prime_import_sg_table(struct drm_device *dev,
- struct dma_buf_attachment *attach,
- struct sg_table *sg)
+struct drm_gem_object *
+amdgpu_gem_prime_import_sg_table(struct drm_device *dev,
+ struct dma_buf_attachment *attach,
+ struct sg_table *sg)
{
struct reservation_object *resv = attach->dmabuf->resv;
struct amdgpu_device *adev = dev->dev_private;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c
index 972eed2..3b02272 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c
@@ -46,7 +46,8 @@
* wptr. The GPU then starts fetching commands and executes
* them until the pointers are equal again.
*/
-static int amdgpu_debugfs_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring);
+static int amdgpu_debugfs_ring_init(struct amdgpu_device *adev,
+ struct amdgpu_ring *ring);
/**
* amdgpu_ring_alloc - allocate space on the ring buffer
@@ -215,18 +216,17 @@
*
* @adev: amdgpu_device pointer
* @ring: amdgpu_ring structure holding ring information
- * @ring_size: size of the ring
+ * @max_ndw: maximum number of dw for ring alloc
* @nop: nop packet for this ring
*
* Initialize the driver information for the selected ring (all asics).
* Returns 0 on success, error on failure.
*/
int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring,
- unsigned ring_size, u32 nop, u32 align_mask,
+ unsigned max_dw, u32 nop, u32 align_mask,
struct amdgpu_irq_src *irq_src, unsigned irq_type,
enum amdgpu_ring_type ring_type)
{
- u32 rb_bufsz;
int r;
if (ring->adev == NULL) {
@@ -265,8 +265,17 @@
dev_err(adev->dev, "(%d) ring next_rptr wb alloc failed\n", r);
return r;
}
- ring->next_rptr_gpu_addr = adev->wb.gpu_addr + (ring->next_rptr_offs * 4);
+ ring->next_rptr_gpu_addr = adev->wb.gpu_addr + ring->next_rptr_offs * 4;
ring->next_rptr_cpu_addr = &adev->wb.wb[ring->next_rptr_offs];
+
+ r = amdgpu_wb_get(adev, &ring->cond_exe_offs);
+ if (r) {
+ dev_err(adev->dev, "(%d) ring cond_exec_polling wb alloc failed\n", r);
+ return r;
+ }
+ ring->cond_exe_gpu_addr = adev->wb.gpu_addr + (ring->cond_exe_offs * 4);
+ ring->cond_exe_cpu_addr = &adev->wb.wb[ring->cond_exe_offs];
+
spin_lock_init(&ring->fence_lock);
r = amdgpu_fence_driver_start_ring(ring, irq_src, irq_type);
if (r) {
@@ -274,10 +283,8 @@
return r;
}
- /* Align ring size */
- rb_bufsz = order_base_2(ring_size / 8);
- ring_size = (1 << (rb_bufsz + 1)) * 4;
- ring->ring_size = ring_size;
+ ring->ring_size = roundup_pow_of_two(max_dw * 4 *
+ amdgpu_sched_hw_submission);
ring->align_mask = align_mask;
ring->nop = nop;
ring->type = ring_type;
@@ -310,8 +317,7 @@
}
}
ring->ptr_mask = (ring->ring_size / 4) - 1;
- ring->max_dw = DIV_ROUND_UP(ring->ring_size / 4,
- amdgpu_sched_hw_submission);
+ ring->max_dw = max_dw;
if (amdgpu_debugfs_ring_init(adev, ring)) {
DRM_ERROR("Failed to register debugfs file for rings !\n");
@@ -363,9 +369,8 @@
struct drm_info_node *node = (struct drm_info_node *) m->private;
struct drm_device *dev = node->minor->dev;
struct amdgpu_device *adev = dev->dev_private;
- int roffset = *(int*)node->info_ent->data;
+ int roffset = (unsigned long)node->info_ent->data;
struct amdgpu_ring *ring = (void *)(((uint8_t*)adev) + roffset);
-
uint32_t rptr, wptr, rptr_next;
unsigned i;
@@ -408,46 +413,37 @@
return 0;
}
-/* TODO: clean this up !*/
-static int amdgpu_gfx_index = offsetof(struct amdgpu_device, gfx.gfx_ring[0]);
-static int cayman_cp1_index = offsetof(struct amdgpu_device, gfx.compute_ring[0]);
-static int cayman_cp2_index = offsetof(struct amdgpu_device, gfx.compute_ring[1]);
-static int amdgpu_dma1_index = offsetof(struct amdgpu_device, sdma.instance[0].ring);
-static int amdgpu_dma2_index = offsetof(struct amdgpu_device, sdma.instance[1].ring);
-static int r600_uvd_index = offsetof(struct amdgpu_device, uvd.ring);
-static int si_vce1_index = offsetof(struct amdgpu_device, vce.ring[0]);
-static int si_vce2_index = offsetof(struct amdgpu_device, vce.ring[1]);
-
-static struct drm_info_list amdgpu_debugfs_ring_info_list[] = {
- {"amdgpu_ring_gfx", amdgpu_debugfs_ring_info, 0, &amdgpu_gfx_index},
- {"amdgpu_ring_cp1", amdgpu_debugfs_ring_info, 0, &cayman_cp1_index},
- {"amdgpu_ring_cp2", amdgpu_debugfs_ring_info, 0, &cayman_cp2_index},
- {"amdgpu_ring_dma1", amdgpu_debugfs_ring_info, 0, &amdgpu_dma1_index},
- {"amdgpu_ring_dma2", amdgpu_debugfs_ring_info, 0, &amdgpu_dma2_index},
- {"amdgpu_ring_uvd", amdgpu_debugfs_ring_info, 0, &r600_uvd_index},
- {"amdgpu_ring_vce1", amdgpu_debugfs_ring_info, 0, &si_vce1_index},
- {"amdgpu_ring_vce2", amdgpu_debugfs_ring_info, 0, &si_vce2_index},
-};
+static struct drm_info_list amdgpu_debugfs_ring_info_list[AMDGPU_MAX_RINGS];
+static char amdgpu_debugfs_ring_names[AMDGPU_MAX_RINGS][32];
#endif
-static int amdgpu_debugfs_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring)
+static int amdgpu_debugfs_ring_init(struct amdgpu_device *adev,
+ struct amdgpu_ring *ring)
{
#if defined(CONFIG_DEBUG_FS)
+ unsigned offset = (uint8_t*)ring - (uint8_t*)adev;
unsigned i;
+ struct drm_info_list *info;
+ char *name;
+
for (i = 0; i < ARRAY_SIZE(amdgpu_debugfs_ring_info_list); ++i) {
- struct drm_info_list *info = &amdgpu_debugfs_ring_info_list[i];
- int roffset = *(int*)amdgpu_debugfs_ring_info_list[i].data;
- struct amdgpu_ring *other = (void *)(((uint8_t*)adev) + roffset);
- unsigned r;
-
- if (other != ring)
- continue;
-
- r = amdgpu_debugfs_add_files(adev, info, 1);
- if (r)
- return r;
+ info = &amdgpu_debugfs_ring_info_list[i];
+ if (!info->data)
+ break;
}
+
+ if (i == ARRAY_SIZE(amdgpu_debugfs_ring_info_list))
+ return -ENOSPC;
+
+ name = &amdgpu_debugfs_ring_names[i][0];
+ sprintf(name, "amdgpu_ring_%s", ring->name);
+ info->name = name;
+ info->show = amdgpu_debugfs_ring_info;
+ info->driver_features = 0;
+ info->data = (void*)(uintptr_t)offset;
+
+ return amdgpu_debugfs_add_files(adev, info, 1);
#endif
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
index c48b4fc..34a9280 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
@@ -109,6 +109,29 @@
}
/**
+ * amdgpu_sync_add_later - add the fence to the hash
+ *
+ * @sync: sync object to add the fence to
+ * @f: fence to add
+ *
+ * Tries to add the fence to an existing hash entry. Returns true when an entry
+ * was found, false otherwise.
+ */
+static bool amdgpu_sync_add_later(struct amdgpu_sync *sync, struct fence *f)
+{
+ struct amdgpu_sync_entry *e;
+
+ hash_for_each_possible(sync->fences, e, node, f->context) {
+ if (unlikely(e->fence->context != f->context))
+ continue;
+
+ amdgpu_sync_keep_later(&e->fence, f);
+ return true;
+ }
+ return false;
+}
+
+/**
* amdgpu_sync_fence - remember to sync to this fence
*
* @sync: sync object to add fence to
@@ -127,13 +150,8 @@
amdgpu_sync_get_owner(f) == AMDGPU_FENCE_OWNER_VM)
amdgpu_sync_keep_later(&sync->last_vm_update, f);
- hash_for_each_possible(sync->fences, e, node, f->context) {
- if (unlikely(e->fence->context != f->context))
- continue;
-
- amdgpu_sync_keep_later(&e->fence, f);
+ if (amdgpu_sync_add_later(sync, f))
return 0;
- }
e = kmem_cache_alloc(amdgpu_sync_slab, GFP_KERNEL);
if (!e)
@@ -204,6 +222,81 @@
return r;
}
+/**
+ * amdgpu_sync_is_idle - test if all fences are signaled
+ *
+ * @sync: the sync object
+ *
+ * Returns true if all fences in the sync object are signaled.
+ */
+bool amdgpu_sync_is_idle(struct amdgpu_sync *sync)
+{
+ struct amdgpu_sync_entry *e;
+ struct hlist_node *tmp;
+ int i;
+
+ hash_for_each_safe(sync->fences, i, tmp, e, node) {
+ struct fence *f = e->fence;
+
+ if (fence_is_signaled(f)) {
+ hash_del(&e->node);
+ fence_put(f);
+ kmem_cache_free(amdgpu_sync_slab, e);
+ continue;
+ }
+
+ return false;
+ }
+
+ return true;
+}
+
+/**
+ * amdgpu_sync_cycle_fences - move fences from one sync object into another
+ *
+ * @dst: the destination sync object
+ * @src: the source sync object
+ * @fence: fence to add to source
+ *
+ * Remove all fences from source and put them into destination and add
+ * fence as new one into source.
+ */
+int amdgpu_sync_cycle_fences(struct amdgpu_sync *dst, struct amdgpu_sync *src,
+ struct fence *fence)
+{
+ struct amdgpu_sync_entry *e, *newone;
+ struct hlist_node *tmp;
+ int i;
+
+ /* Allocate the new entry before moving the old ones */
+ newone = kmem_cache_alloc(amdgpu_sync_slab, GFP_KERNEL);
+ if (!newone)
+ return -ENOMEM;
+
+ hash_for_each_safe(src->fences, i, tmp, e, node) {
+ struct fence *f = e->fence;
+
+ hash_del(&e->node);
+ if (fence_is_signaled(f)) {
+ fence_put(f);
+ kmem_cache_free(amdgpu_sync_slab, e);
+ continue;
+ }
+
+ if (amdgpu_sync_add_later(dst, f)) {
+ kmem_cache_free(amdgpu_sync_slab, e);
+ continue;
+ }
+
+ hash_add(dst->fences, &e->node, f->context);
+ }
+
+ hash_add(src->fences, &newone->node, fence->context);
+ newone->fence = fence_get(fence);
+
+ return 0;
+}
+
struct fence *amdgpu_sync_get_fence(struct amdgpu_sync *sync)
{
struct amdgpu_sync_entry *e;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
index 11af449..3b9053a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -911,6 +911,52 @@
return flags;
}
+static void amdgpu_ttm_lru_removal(struct ttm_buffer_object *tbo)
+{
+ struct amdgpu_device *adev = amdgpu_get_adev(tbo->bdev);
+ unsigned i, j;
+
+ for (i = 0; i < AMDGPU_TTM_LRU_SIZE; ++i) {
+ struct amdgpu_mman_lru *lru = &adev->mman.log2_size[i];
+
+ for (j = 0; j < TTM_NUM_MEM_TYPES; ++j)
+ if (&tbo->lru == lru->lru[j])
+ lru->lru[j] = tbo->lru.prev;
+
+ if (&tbo->swap == lru->swap_lru)
+ lru->swap_lru = tbo->swap.prev;
+ }
+}
+
+static struct amdgpu_mman_lru *amdgpu_ttm_lru(struct ttm_buffer_object *tbo)
+{
+ struct amdgpu_device *adev = amdgpu_get_adev(tbo->bdev);
+ unsigned log2_size = min(ilog2(tbo->num_pages),
+ AMDGPU_TTM_LRU_SIZE - 1);
+
+ return &adev->mman.log2_size[log2_size];
+}
+
+static struct list_head *amdgpu_ttm_lru_tail(struct ttm_buffer_object *tbo)
+{
+ struct amdgpu_mman_lru *lru = amdgpu_ttm_lru(tbo);
+ struct list_head *res = lru->lru[tbo->mem.mem_type];
+
+ lru->lru[tbo->mem.mem_type] = &tbo->lru;
+
+ return res;
+}
+
+static struct list_head *amdgpu_ttm_swap_lru_tail(struct ttm_buffer_object *tbo)
+{
+ struct amdgpu_mman_lru *lru = amdgpu_ttm_lru(tbo);
+ struct list_head *res = lru->swap_lru;
+
+ lru->swap_lru = &tbo->swap;
+
+ return res;
+}
+
static struct ttm_bo_driver amdgpu_bo_driver = {
.ttm_tt_create = &amdgpu_ttm_tt_create,
.ttm_tt_populate = &amdgpu_ttm_tt_populate,
@@ -924,10 +970,14 @@
.fault_reserve_notify = &amdgpu_bo_fault_reserve_notify,
.io_mem_reserve = &amdgpu_ttm_io_mem_reserve,
.io_mem_free = &amdgpu_ttm_io_mem_free,
+ .lru_removal = &amdgpu_ttm_lru_removal,
+ .lru_tail = &amdgpu_ttm_lru_tail,
+ .swap_lru_tail = &amdgpu_ttm_swap_lru_tail,
};
int amdgpu_ttm_init(struct amdgpu_device *adev)
{
+ unsigned i, j;
int r;
r = amdgpu_ttm_global_init(adev);
@@ -945,6 +995,15 @@
DRM_ERROR("failed initializing buffer object driver(%d).\n", r);
return r;
}
+
+ for (i = 0; i < AMDGPU_TTM_LRU_SIZE; ++i) {
+ struct amdgpu_mman_lru *lru = &adev->mman.log2_size[i];
+
+ for (j = 0; j < TTM_NUM_MEM_TYPES; ++j)
+ lru->lru[j] = &adev->mman.bdev.man[j].lru;
+ lru->swap_lru = &adev->mman.bdev.glob->swap_lru;
+ }
+
adev->mman.initialized = true;
r = ttm_bo_init_mm(&adev->mman.bdev, TTM_PL_VRAM,
adev->mc.real_vram_size >> PAGE_SHIFT);
@@ -1167,7 +1226,7 @@
static int ttm_pl_vram = TTM_PL_VRAM;
static int ttm_pl_tt = TTM_PL_TT;
-static struct drm_info_list amdgpu_ttm_debugfs_list[] = {
+static const struct drm_info_list amdgpu_ttm_debugfs_list[] = {
{"amdgpu_vram_mm", amdgpu_mm_dump_table, 0, &ttm_pl_vram},
{"amdgpu_gtt_mm", amdgpu_mm_dump_table, 0, &ttm_pl_tt},
{"ttm_page_pool", ttm_page_alloc_debugfs, 0, NULL},
@@ -1218,6 +1277,8 @@
.llseek = default_llseek
};
+#ifdef CONFIG_DRM_AMDGPU_GART_DEBUGFS
+
static ssize_t amdgpu_ttm_gtt_read(struct file *f, char __user *buf,
size_t size, loff_t *pos)
{
@@ -1265,6 +1326,8 @@
#endif
+#endif
+
static int amdgpu_ttm_debugfs_init(struct amdgpu_device *adev)
{
#if defined(CONFIG_DEBUG_FS)
@@ -1280,6 +1343,7 @@
i_size_write(ent->d_inode, adev->mc.mc_vram_size);
adev->mman.vram = ent;
+#ifdef CONFIG_DRM_AMDGPU_GART_DEBUGFS
ent = debugfs_create_file("amdgpu_gtt", S_IFREG | S_IRUGO, root,
adev, &amdgpu_ttm_gtt_fops);
if (IS_ERR(ent))
@@ -1287,6 +1351,7 @@
i_size_write(ent->d_inode, adev->mc.gtt_size);
adev->mman.gtt = ent;
+#endif
count = ARRAY_SIZE(amdgpu_ttm_debugfs_list);
#ifdef CONFIG_SWIOTLB
@@ -1308,7 +1373,10 @@
debugfs_remove(adev->mman.vram);
adev->mman.vram = NULL;
+#ifdef CONFIG_DRM_AMDGPU_GART_DEBUGFS
debugfs_remove(adev->mman.gtt);
adev->mman.gtt = NULL;
#endif
+
+#endif
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c
index 871018c..01abfc2 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c
@@ -41,19 +41,23 @@
/* 1 second timeout */
#define UVD_IDLE_TIMEOUT_MS 1000
+/* Polaris10/11 firmware version */
+#define FW_1_66_16 ((1 << 24) | (66 << 16) | (16 << 8))
/* Firmware Names */
#ifdef CONFIG_DRM_AMDGPU_CIK
#define FIRMWARE_BONAIRE "radeon/bonaire_uvd.bin"
-#define FIRMWARE_KABINI "radeon/kabini_uvd.bin"
-#define FIRMWARE_KAVERI "radeon/kaveri_uvd.bin"
-#define FIRMWARE_HAWAII "radeon/hawaii_uvd.bin"
+#define FIRMWARE_KABINI "radeon/kabini_uvd.bin"
+#define FIRMWARE_KAVERI "radeon/kaveri_uvd.bin"
+#define FIRMWARE_HAWAII "radeon/hawaii_uvd.bin"
#define FIRMWARE_MULLINS "radeon/mullins_uvd.bin"
#endif
#define FIRMWARE_TONGA "amdgpu/tonga_uvd.bin"
#define FIRMWARE_CARRIZO "amdgpu/carrizo_uvd.bin"
#define FIRMWARE_FIJI "amdgpu/fiji_uvd.bin"
#define FIRMWARE_STONEY "amdgpu/stoney_uvd.bin"
+#define FIRMWARE_POLARIS10 "amdgpu/polaris10_uvd.bin"
+#define FIRMWARE_POLARIS11 "amdgpu/polaris11_uvd.bin"
/**
* amdgpu_uvd_cs_ctx - Command submission parser context
@@ -85,6 +89,8 @@
MODULE_FIRMWARE(FIRMWARE_CARRIZO);
MODULE_FIRMWARE(FIRMWARE_FIJI);
MODULE_FIRMWARE(FIRMWARE_STONEY);
+MODULE_FIRMWARE(FIRMWARE_POLARIS10);
+MODULE_FIRMWARE(FIRMWARE_POLARIS11);
static void amdgpu_uvd_note_usage(struct amdgpu_device *adev);
static void amdgpu_uvd_idle_work_handler(struct work_struct *work);
@@ -131,6 +137,12 @@
case CHIP_STONEY:
fw_name = FIRMWARE_STONEY;
break;
+ case CHIP_POLARIS10:
+ fw_name = FIRMWARE_POLARIS10;
+ break;
+ case CHIP_POLARIS11:
+ fw_name = FIRMWARE_POLARIS11;
+ break;
default:
return -EINVAL;
}
@@ -151,6 +163,9 @@
return r;
}
+ /* Set the default UVD handles that the firmware can handle */
+ adev->uvd.max_handles = AMDGPU_DEFAULT_UVD_HANDLES;
+
hdr = (const struct common_firmware_header *)adev->uvd.fw->data;
family_id = le32_to_cpu(hdr->ucode_version) & 0xff;
version_major = (le32_to_cpu(hdr->ucode_version) >> 24) & 0xff;
@@ -158,11 +173,28 @@
DRM_INFO("Found UVD firmware Version: %hu.%hu Family ID: %hu\n",
version_major, version_minor, family_id);
+ /*
+ * Limit the number of UVD handles depending on microcode major
+ * and minor versions. The firmware version which has 40 UVD
+ * instances support is 1.80. So all subsequent versions should
+ * also have the same support.
+ */
+ if ((version_major > 0x01) ||
+ ((version_major == 0x01) && (version_minor >= 0x50)))
+ adev->uvd.max_handles = AMDGPU_MAX_UVD_HANDLES;
+
adev->uvd.fw_version = ((version_major << 24) | (version_minor << 16) |
(family_id << 8));
+ if ((adev->asic_type == CHIP_POLARIS10 ||
+ adev->asic_type == CHIP_POLARIS11) &&
+ (adev->uvd.fw_version < FW_1_66_16))
+ DRM_ERROR("POLARIS10/11 UVD firmware version %hu.%hu is too old.\n",
+ version_major, version_minor);
+
bo_size = AMDGPU_GPU_PAGE_ALIGN(le32_to_cpu(hdr->ucode_size_bytes) + 8)
- + AMDGPU_UVD_STACK_SIZE + AMDGPU_UVD_HEAP_SIZE;
+ + AMDGPU_UVD_STACK_SIZE + AMDGPU_UVD_HEAP_SIZE
+ + AMDGPU_UVD_SESSION_SIZE * adev->uvd.max_handles;
r = amdgpu_bo_create(adev, bo_size, PAGE_SIZE, true,
AMDGPU_GEM_DOMAIN_VRAM,
AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED,
@@ -205,7 +237,7 @@
return r;
}
- for (i = 0; i < AMDGPU_MAX_UVD_HANDLES; ++i) {
+ for (i = 0; i < adev->uvd.max_handles; ++i) {
atomic_set(&adev->uvd.handles[i], 0);
adev->uvd.filp[i] = NULL;
}
@@ -251,7 +283,7 @@
if (adev->uvd.vcpu_bo == NULL)
return 0;
- for (i = 0; i < AMDGPU_MAX_UVD_HANDLES; ++i)
+ for (i = 0; i < adev->uvd.max_handles; ++i)
if (atomic_read(&adev->uvd.handles[i]))
break;
@@ -308,7 +340,7 @@
struct amdgpu_ring *ring = &adev->uvd.ring;
int i, r;
- for (i = 0; i < AMDGPU_MAX_UVD_HANDLES; ++i) {
+ for (i = 0; i < adev->uvd.max_handles; ++i) {
uint32_t handle = atomic_read(&adev->uvd.handles[i]);
if (handle != 0 && adev->uvd.filp[i] == filp) {
struct fence *fence;
@@ -390,7 +422,8 @@
*
* Peek into the decode message and calculate the necessary buffer sizes.
*/
-static int amdgpu_uvd_cs_msg_decode(uint32_t *msg, unsigned buf_sizes[])
+static int amdgpu_uvd_cs_msg_decode(struct amdgpu_device *adev, uint32_t *msg,
+ unsigned buf_sizes[])
{
unsigned stream_type = msg[4];
unsigned width = msg[6];
@@ -412,7 +445,6 @@
switch (stream_type) {
case 0: /* H264 */
- case 7: /* H264 Perf */
switch(level) {
case 30:
num_dpb_buffer = 8100 / fs_in_mb;
@@ -490,6 +522,54 @@
min_dpb_size += ALIGN(width_in_mb * height_in_mb * 32, 64);
break;
+ case 7: /* H264 Perf */
+ switch(level) {
+ case 30:
+ num_dpb_buffer = 8100 / fs_in_mb;
+ break;
+ case 31:
+ num_dpb_buffer = 18000 / fs_in_mb;
+ break;
+ case 32:
+ num_dpb_buffer = 20480 / fs_in_mb;
+ break;
+ case 41:
+ num_dpb_buffer = 32768 / fs_in_mb;
+ break;
+ case 42:
+ num_dpb_buffer = 34816 / fs_in_mb;
+ break;
+ case 50:
+ num_dpb_buffer = 110400 / fs_in_mb;
+ break;
+ case 51:
+ num_dpb_buffer = 184320 / fs_in_mb;
+ break;
+ default:
+ num_dpb_buffer = 184320 / fs_in_mb;
+ break;
+ }
+ num_dpb_buffer++;
+ if (num_dpb_buffer > 17)
+ num_dpb_buffer = 17;
+
+ /* reference picture buffer */
+ min_dpb_size = image_size * num_dpb_buffer;
+
+ if (adev->asic_type < CHIP_POLARIS10){
+ /* macroblock context buffer */
+ min_dpb_size +=
+ width_in_mb * height_in_mb * num_dpb_buffer * 192;
+
+ /* IT surface buffer */
+ min_dpb_size += width_in_mb * height_in_mb * 32;
+ } else {
+ /* macroblock context buffer */
+ min_ctx_size =
+ width_in_mb * height_in_mb * num_dpb_buffer * 192;
+ }
+ break;
+
case 16: /* H265 */
image_size = (ALIGN(width, 16) * ALIGN(height, 16) * 3) / 2;
image_size = ALIGN(image_size, 256);
@@ -568,7 +648,7 @@
amdgpu_bo_kunmap(bo);
/* try to alloc a new handle */
- for (i = 0; i < AMDGPU_MAX_UVD_HANDLES; ++i) {
+ for (i = 0; i < adev->uvd.max_handles; ++i) {
if (atomic_read(&adev->uvd.handles[i]) == handle) {
DRM_ERROR("Handle 0x%x already in use!\n", handle);
return -EINVAL;
@@ -585,13 +665,13 @@
case 1:
/* it's a decode msg, calc buffer sizes */
- r = amdgpu_uvd_cs_msg_decode(msg, ctx->buf_sizes);
+ r = amdgpu_uvd_cs_msg_decode(adev, msg, ctx->buf_sizes);
amdgpu_bo_kunmap(bo);
if (r)
return r;
/* validate the handle */
- for (i = 0; i < AMDGPU_MAX_UVD_HANDLES; ++i) {
+ for (i = 0; i < adev->uvd.max_handles; ++i) {
if (atomic_read(&adev->uvd.handles[i]) == handle) {
if (adev->uvd.filp[i] != ctx->parser->filp) {
DRM_ERROR("UVD handle collision detected!\n");
@@ -606,7 +686,7 @@
case 2:
/* it's a destroy msg, free the handle */
- for (i = 0; i < AMDGPU_MAX_UVD_HANDLES; ++i)
+ for (i = 0; i < adev->uvd.max_handles; ++i)
atomic_cmpxchg(&adev->uvd.handles[i], handle, 0);
amdgpu_bo_kunmap(bo);
return 0;
@@ -886,7 +966,7 @@
ib->length_dw = 16;
if (direct) {
- r = amdgpu_ib_schedule(ring, 1, ib, NULL, &f);
+ r = amdgpu_ib_schedule(ring, 1, ib, NULL, NULL, &f);
job->fence = f;
if (r)
goto err_free;
@@ -1018,7 +1098,7 @@
fences = amdgpu_fence_count_emitted(&adev->uvd.ring);
- for (i = 0; i < AMDGPU_MAX_UVD_HANDLES; ++i)
+ for (i = 0; i < adev->uvd.max_handles; ++i)
if (atomic_read(&adev->uvd.handles[i]))
++handles;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c
index 481a64f..875626a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c
@@ -41,15 +41,17 @@
/* Firmware Names */
#ifdef CONFIG_DRM_AMDGPU_CIK
#define FIRMWARE_BONAIRE "radeon/bonaire_vce.bin"
-#define FIRMWARE_KABINI "radeon/kabini_vce.bin"
-#define FIRMWARE_KAVERI "radeon/kaveri_vce.bin"
-#define FIRMWARE_HAWAII "radeon/hawaii_vce.bin"
+#define FIRMWARE_KABINI "radeon/kabini_vce.bin"
+#define FIRMWARE_KAVERI "radeon/kaveri_vce.bin"
+#define FIRMWARE_HAWAII "radeon/hawaii_vce.bin"
#define FIRMWARE_MULLINS "radeon/mullins_vce.bin"
#endif
#define FIRMWARE_TONGA "amdgpu/tonga_vce.bin"
#define FIRMWARE_CARRIZO "amdgpu/carrizo_vce.bin"
#define FIRMWARE_FIJI "amdgpu/fiji_vce.bin"
#define FIRMWARE_STONEY "amdgpu/stoney_vce.bin"
+#define FIRMWARE_POLARIS10 "amdgpu/polaris10_vce.bin"
+#define FIRMWARE_POLARIS11 "amdgpu/polaris11_vce.bin"
#ifdef CONFIG_DRM_AMDGPU_CIK
MODULE_FIRMWARE(FIRMWARE_BONAIRE);
@@ -62,6 +64,8 @@
MODULE_FIRMWARE(FIRMWARE_CARRIZO);
MODULE_FIRMWARE(FIRMWARE_FIJI);
MODULE_FIRMWARE(FIRMWARE_STONEY);
+MODULE_FIRMWARE(FIRMWARE_POLARIS10);
+MODULE_FIRMWARE(FIRMWARE_POLARIS11);
static void amdgpu_vce_idle_work_handler(struct work_struct *work);
@@ -113,6 +117,12 @@
case CHIP_STONEY:
fw_name = FIRMWARE_STONEY;
break;
+ case CHIP_POLARIS10:
+ fw_name = FIRMWARE_POLARIS10;
+ break;
+ case CHIP_POLARIS11:
+ fw_name = FIRMWARE_POLARIS11;
+ break;
default:
return -EINVAL;
@@ -426,7 +436,7 @@
for (i = ib->length_dw; i < ib_size_dw; ++i)
ib->ptr[i] = 0x0;
- r = amdgpu_ib_schedule(ring, 1, ib, NULL, &f);
+ r = amdgpu_ib_schedule(ring, 1, ib, NULL, NULL, &f);
job->fence = f;
if (r)
goto err;
@@ -488,7 +498,7 @@
ib->ptr[i] = 0x0;
if (direct) {
- r = amdgpu_ib_schedule(ring, 1, ib, NULL, &f);
+ r = amdgpu_ib_schedule(ring, 1, ib, NULL, NULL, &f);
job->fence = f;
if (r)
goto err;
@@ -752,7 +762,8 @@
* @ib: the IB to execute
*
*/
-void amdgpu_vce_ring_emit_ib(struct amdgpu_ring *ring, struct amdgpu_ib *ib)
+void amdgpu_vce_ring_emit_ib(struct amdgpu_ring *ring, struct amdgpu_ib *ib,
+ unsigned vm_id, bool ctx_switch)
{
amdgpu_ring_write(ring, VCE_CMD_IB);
amdgpu_ring_write(ring, lower_32_bits(ib->gpu_addr));
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.h
index ef99d23..f40cf76 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.h
@@ -34,7 +34,8 @@
bool direct, struct fence **fence);
void amdgpu_vce_free_handles(struct amdgpu_device *adev, struct drm_file *filp);
int amdgpu_vce_ring_parse_cs(struct amdgpu_cs_parser *p, uint32_t ib_idx);
-void amdgpu_vce_ring_emit_ib(struct amdgpu_ring *ring, struct amdgpu_ib *ib);
+void amdgpu_vce_ring_emit_ib(struct amdgpu_ring *ring, struct amdgpu_ib *ib,
+ unsigned vm_id, bool ctx_switch);
void amdgpu_vce_ring_emit_fence(struct amdgpu_ring *ring, u64 addr, u64 seq,
unsigned flags);
int amdgpu_vce_ring_test_ring(struct amdgpu_ring *ring);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index b6c011b..ea708cb9 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -166,74 +166,109 @@
{
uint64_t pd_addr = amdgpu_bo_gpu_offset(vm->page_directory);
struct amdgpu_device *adev = ring->adev;
- struct amdgpu_vm_id *id = &vm->ids[ring->idx];
struct fence *updates = sync->last_vm_update;
+ struct amdgpu_vm_id *id;
+ unsigned i = ring->idx;
int r;
mutex_lock(&adev->vm_manager.lock);
- /* check if the id is still valid */
- if (id->mgr_id) {
- struct fence *flushed = id->flushed_updates;
- bool is_later;
- long owner;
+ /* Check if we can use a VMID already assigned to this VM */
+ do {
+ struct fence *flushed;
- if (!flushed)
- is_later = true;
- else if (!updates)
- is_later = false;
- else
- is_later = fence_is_later(updates, flushed);
+ id = vm->ids[i++];
+ if (i == AMDGPU_MAX_RINGS)
+ i = 0;
- owner = atomic_long_read(&id->mgr_id->owner);
- if (!is_later && owner == (long)id &&
- pd_addr == id->pd_gpu_addr) {
+ /* Check all the prerequisites to using this VMID */
+ if (!id)
+ continue;
+ if (atomic64_read(&id->owner) != vm->client_id)
+ continue;
+
+ if (pd_addr != id->pd_gpu_addr)
+ continue;
+
+ if (id->last_user != ring &&
+ (!id->last_flush || !fence_is_signaled(id->last_flush)))
+ continue;
+
+ flushed = id->flushed_updates;
+ if (updates && (!flushed || fence_is_later(updates, flushed)))
+ continue;
+
+ /* Good we can use this VMID */
+ if (id->last_user == ring) {
r = amdgpu_sync_fence(ring->adev, sync,
- id->mgr_id->active);
- if (r) {
- mutex_unlock(&adev->vm_manager.lock);
- return r;
- }
-
- fence_put(id->mgr_id->active);
- id->mgr_id->active = fence_get(fence);
-
- list_move_tail(&id->mgr_id->list,
- &adev->vm_manager.ids_lru);
-
- *vm_id = id->mgr_id - adev->vm_manager.ids;
- *vm_pd_addr = AMDGPU_VM_NO_FLUSH;
- trace_amdgpu_vm_grab_id(vm, ring->idx, *vm_id,
- *vm_pd_addr);
-
- mutex_unlock(&adev->vm_manager.lock);
- return 0;
+ id->first);
+ if (r)
+ goto error;
}
- }
- id->mgr_id = list_first_entry(&adev->vm_manager.ids_lru,
- struct amdgpu_vm_manager_id,
- list);
+ /* And remember this submission as user of the VMID */
+ r = amdgpu_sync_fence(ring->adev, &id->active, fence);
+ if (r)
+ goto error;
- r = amdgpu_sync_fence(ring->adev, sync, id->mgr_id->active);
- if (!r) {
- fence_put(id->mgr_id->active);
- id->mgr_id->active = fence_get(fence);
+ list_move_tail(&id->list, &adev->vm_manager.ids_lru);
+ vm->ids[ring->idx] = id;
- fence_put(id->flushed_updates);
- id->flushed_updates = fence_get(updates);
-
- id->pd_gpu_addr = pd_addr;
-
- list_move_tail(&id->mgr_id->list, &adev->vm_manager.ids_lru);
- atomic_long_set(&id->mgr_id->owner, (long)id);
-
- *vm_id = id->mgr_id - adev->vm_manager.ids;
- *vm_pd_addr = pd_addr;
+ *vm_id = id - adev->vm_manager.ids;
+ *vm_pd_addr = AMDGPU_VM_NO_FLUSH;
trace_amdgpu_vm_grab_id(vm, ring->idx, *vm_id, *vm_pd_addr);
+
+ mutex_unlock(&adev->vm_manager.lock);
+ return 0;
+
+ } while (i != ring->idx);
+
+ id = list_first_entry(&adev->vm_manager.ids_lru,
+ struct amdgpu_vm_id,
+ list);
+
+ if (!amdgpu_sync_is_idle(&id->active)) {
+ struct list_head *head = &adev->vm_manager.ids_lru;
+ struct amdgpu_vm_id *tmp;
+
+ list_for_each_entry_safe(id, tmp, &adev->vm_manager.ids_lru,
+ list) {
+ if (amdgpu_sync_is_idle(&id->active)) {
+ list_move(&id->list, head);
+ head = &id->list;
+ }
+ }
+ id = list_first_entry(&adev->vm_manager.ids_lru,
+ struct amdgpu_vm_id,
+ list);
}
+ r = amdgpu_sync_cycle_fences(sync, &id->active, fence);
+ if (r)
+ goto error;
+
+ fence_put(id->first);
+ id->first = fence_get(fence);
+
+ fence_put(id->last_flush);
+ id->last_flush = NULL;
+
+ fence_put(id->flushed_updates);
+ id->flushed_updates = fence_get(updates);
+
+ id->pd_gpu_addr = pd_addr;
+
+ list_move_tail(&id->list, &adev->vm_manager.ids_lru);
+ id->last_user = ring;
+ atomic64_set(&id->owner, vm->client_id);
+ vm->ids[ring->idx] = id;
+
+ *vm_id = id - adev->vm_manager.ids;
+ *vm_pd_addr = pd_addr;
+ trace_amdgpu_vm_grab_id(vm, ring->idx, *vm_id, *vm_pd_addr);
+
+error:
mutex_unlock(&adev->vm_manager.lock);
return r;
}
@@ -247,43 +282,62 @@
*
* Emit a VM flush when it is necessary.
*/
-void amdgpu_vm_flush(struct amdgpu_ring *ring,
- unsigned vm_id, uint64_t pd_addr,
- uint32_t gds_base, uint32_t gds_size,
- uint32_t gws_base, uint32_t gws_size,
- uint32_t oa_base, uint32_t oa_size)
+int amdgpu_vm_flush(struct amdgpu_ring *ring,
+ unsigned vm_id, uint64_t pd_addr,
+ uint32_t gds_base, uint32_t gds_size,
+ uint32_t gws_base, uint32_t gws_size,
+ uint32_t oa_base, uint32_t oa_size)
{
struct amdgpu_device *adev = ring->adev;
- struct amdgpu_vm_manager_id *mgr_id = &adev->vm_manager.ids[vm_id];
+ struct amdgpu_vm_id *id = &adev->vm_manager.ids[vm_id];
bool gds_switch_needed = ring->funcs->emit_gds_switch && (
- mgr_id->gds_base != gds_base ||
- mgr_id->gds_size != gds_size ||
- mgr_id->gws_base != gws_base ||
- mgr_id->gws_size != gws_size ||
- mgr_id->oa_base != oa_base ||
- mgr_id->oa_size != oa_size);
+ id->gds_base != gds_base ||
+ id->gds_size != gds_size ||
+ id->gws_base != gws_base ||
+ id->gws_size != gws_size ||
+ id->oa_base != oa_base ||
+ id->oa_size != oa_size);
+ int r;
if (ring->funcs->emit_pipeline_sync && (
- pd_addr != AMDGPU_VM_NO_FLUSH || gds_switch_needed))
+ pd_addr != AMDGPU_VM_NO_FLUSH || gds_switch_needed ||
+ ring->type == AMDGPU_RING_TYPE_COMPUTE))
amdgpu_ring_emit_pipeline_sync(ring);
- if (pd_addr != AMDGPU_VM_NO_FLUSH) {
+ if (ring->funcs->emit_vm_flush &&
+ pd_addr != AMDGPU_VM_NO_FLUSH) {
+ struct fence *fence;
+
trace_amdgpu_vm_flush(pd_addr, ring->idx, vm_id);
amdgpu_ring_emit_vm_flush(ring, vm_id, pd_addr);
+
+ mutex_lock(&adev->vm_manager.lock);
+ if ((id->pd_gpu_addr == pd_addr) && (id->last_user == ring)) {
+ r = amdgpu_fence_emit(ring, &fence);
+ if (r) {
+ mutex_unlock(&adev->vm_manager.lock);
+ return r;
+ }
+ fence_put(id->last_flush);
+ id->last_flush = fence;
+ }
+ mutex_unlock(&adev->vm_manager.lock);
}
if (gds_switch_needed) {
- mgr_id->gds_base = gds_base;
- mgr_id->gds_size = gds_size;
- mgr_id->gws_base = gws_base;
- mgr_id->gws_size = gws_size;
- mgr_id->oa_base = oa_base;
- mgr_id->oa_size = oa_size;
+ id->gds_base = gds_base;
+ id->gds_size = gds_size;
+ id->gws_base = gws_base;
+ id->gws_size = gws_size;
+ id->oa_base = oa_base;
+ id->oa_size = oa_size;
amdgpu_ring_emit_gds_switch(ring, vm_id,
gds_base, gds_size,
gws_base, gws_size,
oa_base, oa_size);
}
+
+ return 0;
}
/**
@@ -296,14 +350,14 @@
*/
void amdgpu_vm_reset_id(struct amdgpu_device *adev, unsigned vm_id)
{
- struct amdgpu_vm_manager_id *mgr_id = &adev->vm_manager.ids[vm_id];
+ struct amdgpu_vm_id *id = &adev->vm_manager.ids[vm_id];
- mgr_id->gds_base = 0;
- mgr_id->gds_size = 0;
- mgr_id->gws_base = 0;
- mgr_id->gws_size = 0;
- mgr_id->oa_base = 0;
- mgr_id->oa_size = 0;
+ id->gds_base = 0;
+ id->gds_size = 0;
+ id->gws_base = 0;
+ id->gws_size = 0;
+ id->oa_base = 0;
+ id->oa_size = 0;
}
/**
@@ -335,8 +389,8 @@
* amdgpu_vm_update_pages - helper to call the right asic function
*
* @adev: amdgpu_device pointer
- * @gtt: GART instance to use for mapping
- * @gtt_flags: GTT hw access flags
+ * @src: address where to copy page table entries from
+ * @pages_addr: DMA addresses to use for mapping
* @ib: indirect buffer to fill with commands
* @pe: addr of the page entry
* @addr: dst addr to write into pe
@@ -348,8 +402,8 @@
* to setup the page table using the DMA.
*/
static void amdgpu_vm_update_pages(struct amdgpu_device *adev,
- struct amdgpu_gart *gtt,
- uint32_t gtt_flags,
+ uint64_t src,
+ dma_addr_t *pages_addr,
struct amdgpu_ib *ib,
uint64_t pe, uint64_t addr,
unsigned count, uint32_t incr,
@@ -357,12 +411,11 @@
{
trace_amdgpu_vm_set_page(pe, addr, count, incr, flags);
- if ((gtt == &adev->gart) && (flags == gtt_flags)) {
- uint64_t src = gtt->table_addr + (addr >> 12) * 8;
+ if (src) {
+ src += (addr >> 12) * 8;
amdgpu_vm_copy_pte(adev, ib, pe, src, count);
- } else if (gtt) {
- dma_addr_t *pages_addr = gtt->pages_addr;
+ } else if (pages_addr) {
amdgpu_vm_write_pte(adev, ib, pages_addr, pe, addr,
count, incr, flags);
@@ -412,7 +465,7 @@
if (r)
goto error;
- amdgpu_vm_update_pages(adev, NULL, 0, &job->ibs[0], addr, 0, entries,
+ amdgpu_vm_update_pages(adev, 0, NULL, &job->ibs[0], addr, 0, entries,
0, 0);
amdgpu_ring_pad_ib(ring, &job->ibs[0]);
@@ -522,7 +575,7 @@
((last_pt + incr * count) != pt)) {
if (count) {
- amdgpu_vm_update_pages(adev, NULL, 0, ib,
+ amdgpu_vm_update_pages(adev, 0, NULL, ib,
last_pde, last_pt,
count, incr,
AMDGPU_PTE_VALID);
@@ -537,7 +590,7 @@
}
if (count)
- amdgpu_vm_update_pages(adev, NULL, 0, ib, last_pde, last_pt,
+ amdgpu_vm_update_pages(adev, 0, NULL, ib, last_pde, last_pt,
count, incr, AMDGPU_PTE_VALID);
if (ib->length_dw != 0) {
@@ -570,8 +623,8 @@
* amdgpu_vm_frag_ptes - add fragment information to PTEs
*
* @adev: amdgpu_device pointer
- * @gtt: GART instance to use for mapping
- * @gtt_flags: GTT hw mapping flags
+ * @src: address where to copy page table entries from
+ * @pages_addr: DMA addresses to use for mapping
* @ib: IB for the update
* @pe_start: first PTE to handle
* @pe_end: last PTE to handle
@@ -579,8 +632,8 @@
* @flags: hw mapping flags
*/
static void amdgpu_vm_frag_ptes(struct amdgpu_device *adev,
- struct amdgpu_gart *gtt,
- uint32_t gtt_flags,
+ uint64_t src,
+ dma_addr_t *pages_addr,
struct amdgpu_ib *ib,
uint64_t pe_start, uint64_t pe_end,
uint64_t addr, uint32_t flags)
@@ -618,10 +671,11 @@
return;
/* system pages are non continuously */
- if (gtt || !(flags & AMDGPU_PTE_VALID) || (frag_start >= frag_end)) {
+ if (src || pages_addr || !(flags & AMDGPU_PTE_VALID) ||
+ (frag_start >= frag_end)) {
count = (pe_end - pe_start) / 8;
- amdgpu_vm_update_pages(adev, gtt, gtt_flags, ib, pe_start,
+ amdgpu_vm_update_pages(adev, src, pages_addr, ib, pe_start,
addr, count, AMDGPU_GPU_PAGE_SIZE,
flags);
return;
@@ -630,21 +684,21 @@
/* handle the 4K area at the beginning */
if (pe_start != frag_start) {
count = (frag_start - pe_start) / 8;
- amdgpu_vm_update_pages(adev, NULL, 0, ib, pe_start, addr,
+ amdgpu_vm_update_pages(adev, 0, NULL, ib, pe_start, addr,
count, AMDGPU_GPU_PAGE_SIZE, flags);
addr += AMDGPU_GPU_PAGE_SIZE * count;
}
/* handle the area in the middle */
count = (frag_end - frag_start) / 8;
- amdgpu_vm_update_pages(adev, NULL, 0, ib, frag_start, addr, count,
+ amdgpu_vm_update_pages(adev, 0, NULL, ib, frag_start, addr, count,
AMDGPU_GPU_PAGE_SIZE, flags | frag_flags);
/* handle the 4K area at the end */
if (frag_end != pe_end) {
addr += AMDGPU_GPU_PAGE_SIZE * count;
count = (pe_end - frag_end) / 8;
- amdgpu_vm_update_pages(adev, NULL, 0, ib, frag_end, addr,
+ amdgpu_vm_update_pages(adev, 0, NULL, ib, frag_end, addr,
count, AMDGPU_GPU_PAGE_SIZE, flags);
}
}
@@ -653,8 +707,8 @@
* amdgpu_vm_update_ptes - make sure that page tables are valid
*
* @adev: amdgpu_device pointer
- * @gtt: GART instance to use for mapping
- * @gtt_flags: GTT hw mapping flags
+ * @src: address where to copy page table entries from
+ * @pages_addr: DMA addresses to use for mapping
* @vm: requested vm
* @start: start of GPU address range
* @end: end of GPU address range
@@ -664,8 +718,8 @@
* Update the page tables in the range @start - @end.
*/
static void amdgpu_vm_update_ptes(struct amdgpu_device *adev,
- struct amdgpu_gart *gtt,
- uint32_t gtt_flags,
+ uint64_t src,
+ dma_addr_t *pages_addr,
struct amdgpu_vm *vm,
struct amdgpu_ib *ib,
uint64_t start, uint64_t end,
@@ -693,7 +747,7 @@
if (last_pe_end != pe_start) {
- amdgpu_vm_frag_ptes(adev, gtt, gtt_flags, ib,
+ amdgpu_vm_frag_ptes(adev, src, pages_addr, ib,
last_pe_start, last_pe_end,
last_dst, flags);
@@ -708,17 +762,16 @@
dst += nptes * AMDGPU_GPU_PAGE_SIZE;
}
- amdgpu_vm_frag_ptes(adev, gtt, gtt_flags, ib,
- last_pe_start, last_pe_end,
- last_dst, flags);
+ amdgpu_vm_frag_ptes(adev, src, pages_addr, ib, last_pe_start,
+ last_pe_end, last_dst, flags);
}
/**
* amdgpu_vm_bo_update_mapping - update a mapping in the vm page table
*
* @adev: amdgpu_device pointer
- * @gtt: GART instance to use for mapping
- * @gtt_flags: flags as they are used for GTT
+ * @src: address where to copy page table entries from
+ * @pages_addr: DMA addresses to use for mapping
* @vm: requested vm
* @start: start of mapped range
* @last: last mapped entry
@@ -730,8 +783,8 @@
* Returns 0 for success, -EINVAL for failure.
*/
static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev,
- struct amdgpu_gart *gtt,
- uint32_t gtt_flags,
+ uint64_t src,
+ dma_addr_t *pages_addr,
struct amdgpu_vm *vm,
uint64_t start, uint64_t last,
uint32_t flags, uint64_t addr,
@@ -762,11 +815,11 @@
/* padding, etc. */
ndw = 64;
- if ((gtt == &adev->gart) && (flags == gtt_flags)) {
+ if (src) {
/* only copy commands needed */
ndw += ncmds * 7;
- } else if (gtt) {
+ } else if (pages_addr) {
/* header for write data commands */
ndw += ncmds * 4;
@@ -796,8 +849,8 @@
if (r)
goto error_free;
- amdgpu_vm_update_ptes(adev, gtt, gtt_flags, vm, ib, start, last + 1,
- addr, flags);
+ amdgpu_vm_update_ptes(adev, src, pages_addr, vm, ib, start,
+ last + 1, addr, flags);
amdgpu_ring_pad_ib(ring, ib);
WARN_ON(ib->length_dw > ndw);
@@ -823,11 +876,12 @@
* amdgpu_vm_bo_split_mapping - split a mapping into smaller chunks
*
* @adev: amdgpu_device pointer
- * @gtt: GART instance to use for mapping
+ * @gtt_flags: flags as they are used for GTT
+ * @pages_addr: DMA addresses to use for mapping
* @vm: requested vm
* @mapping: mapped range and flags to use for the update
* @addr: addr to set the area to
- * @gtt_flags: flags as they are used for GTT
+ * @flags: HW flags for the mapping
* @fence: optional resulting fence
*
* Split the mapping into smaller chunks so that each update fits
@@ -835,16 +889,16 @@
* Returns 0 for success, -EINVAL for failure.
*/
static int amdgpu_vm_bo_split_mapping(struct amdgpu_device *adev,
- struct amdgpu_gart *gtt,
uint32_t gtt_flags,
+ dma_addr_t *pages_addr,
struct amdgpu_vm *vm,
struct amdgpu_bo_va_mapping *mapping,
- uint64_t addr, struct fence **fence)
+ uint32_t flags, uint64_t addr,
+ struct fence **fence)
{
const uint64_t max_size = 64ULL * 1024ULL * 1024ULL / AMDGPU_GPU_PAGE_SIZE;
- uint64_t start = mapping->it.start;
- uint32_t flags = gtt_flags;
+ uint64_t src = 0, start = mapping->it.start;
int r;
/* normally,bo_va->flags only contians READABLE and WIRTEABLE bit go here
@@ -857,10 +911,15 @@
trace_amdgpu_vm_bo_update(mapping);
+ if (pages_addr) {
+ if (flags == gtt_flags)
+ src = adev->gart.table_addr + (addr >> 12) * 8;
+ addr = 0;
+ }
addr += mapping->offset;
- if (!gtt || ((gtt == &adev->gart) && (flags == gtt_flags)))
- return amdgpu_vm_bo_update_mapping(adev, gtt, gtt_flags, vm,
+ if (!pages_addr || src)
+ return amdgpu_vm_bo_update_mapping(adev, src, pages_addr, vm,
start, mapping->it.last,
flags, addr, fence);
@@ -868,7 +927,7 @@
uint64_t last;
last = min((uint64_t)mapping->it.last, start + max_size - 1);
- r = amdgpu_vm_bo_update_mapping(adev, gtt, gtt_flags, vm,
+ r = amdgpu_vm_bo_update_mapping(adev, src, pages_addr, vm,
start, last, flags, addr,
fence);
if (r)
@@ -899,16 +958,20 @@
{
struct amdgpu_vm *vm = bo_va->vm;
struct amdgpu_bo_va_mapping *mapping;
- struct amdgpu_gart *gtt = NULL;
- uint32_t flags;
+ dma_addr_t *pages_addr = NULL;
+ uint32_t gtt_flags, flags;
uint64_t addr;
int r;
if (mem) {
+ struct ttm_dma_tt *ttm;
+
addr = (u64)mem->start << PAGE_SHIFT;
switch (mem->mem_type) {
case TTM_PL_TT:
- gtt = &bo_va->bo->adev->gart;
+ ttm = container_of(bo_va->bo->tbo.ttm, struct
+ ttm_dma_tt, ttm);
+ pages_addr = ttm->dma_address;
break;
case TTM_PL_VRAM:
@@ -923,6 +986,7 @@
}
flags = amdgpu_ttm_tt_pte_flags(adev, bo_va->bo->tbo.ttm, mem);
+ gtt_flags = (adev == bo_va->bo->adev) ? flags : 0;
spin_lock(&vm->status_lock);
if (!list_empty(&bo_va->vm_status))
@@ -930,7 +994,8 @@
spin_unlock(&vm->status_lock);
list_for_each_entry(mapping, &bo_va->invalids, list) {
- r = amdgpu_vm_bo_split_mapping(adev, gtt, flags, vm, mapping, addr,
+ r = amdgpu_vm_bo_split_mapping(adev, gtt_flags, pages_addr, vm,
+ mapping, flags, addr,
&bo_va->last_pt_update);
if (r)
return r;
@@ -976,8 +1041,8 @@
struct amdgpu_bo_va_mapping, list);
list_del(&mapping->list);
- r = amdgpu_vm_bo_split_mapping(adev, NULL, 0, vm, mapping,
- 0, NULL);
+ r = amdgpu_vm_bo_split_mapping(adev, 0, NULL, vm, mapping,
+ 0, 0, NULL);
kfree(mapping);
if (r)
return r;
@@ -1320,11 +1385,10 @@
struct amd_sched_rq *rq;
int i, r;
- for (i = 0; i < AMDGPU_MAX_RINGS; ++i) {
- vm->ids[i].mgr_id = NULL;
- vm->ids[i].flushed_updates = NULL;
- }
+ for (i = 0; i < AMDGPU_MAX_RINGS; ++i)
+ vm->ids[i] = NULL;
vm->va = RB_ROOT;
+ vm->client_id = atomic64_inc_return(&adev->vm_manager.client_counter);
spin_lock_init(&vm->status_lock);
INIT_LIST_HEAD(&vm->invalidated);
INIT_LIST_HEAD(&vm->cleared);
@@ -1416,15 +1480,6 @@
amdgpu_bo_unref(&vm->page_directory);
fence_put(vm->page_directory_fence);
-
- for (i = 0; i < AMDGPU_MAX_RINGS; ++i) {
- struct amdgpu_vm_id *id = &vm->ids[i];
-
- if (id->mgr_id)
- atomic_long_cmpxchg(&id->mgr_id->owner,
- (long)id, 0);
- fence_put(id->flushed_updates);
- }
}
/**
@@ -1443,11 +1498,13 @@
/* skip over VMID 0, since it is the system VM */
for (i = 1; i < adev->vm_manager.num_ids; ++i) {
amdgpu_vm_reset_id(adev, i);
+ amdgpu_sync_create(&adev->vm_manager.ids[i].active);
list_add_tail(&adev->vm_manager.ids[i].list,
&adev->vm_manager.ids_lru);
}
atomic_set(&adev->vm_manager.vm_pte_next_ring, 0);
+ atomic64_set(&adev->vm_manager.client_counter, 0);
}
/**
@@ -1461,6 +1518,11 @@
{
unsigned i;
- for (i = 0; i < AMDGPU_NUM_VM; ++i)
- fence_put(adev->vm_manager.ids[i].active);
+ for (i = 0; i < AMDGPU_NUM_VM; ++i) {
+ struct amdgpu_vm_id *id = &adev->vm_manager.ids[i];
+
+ fence_put(adev->vm_manager.ids[i].first);
+ amdgpu_sync_free(&adev->vm_manager.ids[i].active);
+ fence_put(id->flushed_updates);
+ }
}
diff --git a/drivers/gpu/drm/amd/amdgpu/atom.h b/drivers/gpu/drm/amd/amdgpu/atom.h
index fece8f4..49daf6d 100644
--- a/drivers/gpu/drm/amd/amdgpu/atom.h
+++ b/drivers/gpu/drm/amd/amdgpu/atom.h
@@ -92,7 +92,7 @@
#define ATOM_WS_AND_MASK 0x45
#define ATOM_WS_FB_WINDOW 0x46
#define ATOM_WS_ATTRIBUTES 0x47
-#define ATOM_WS_REGPTR 0x48
+#define ATOM_WS_REGPTR 0x48
#define ATOM_IIO_NOP 0
#define ATOM_IIO_START 1
diff --git a/drivers/gpu/drm/amd/amdgpu/atombios_crtc.c b/drivers/gpu/drm/amd/amdgpu/atombios_crtc.c
index 49aa350..49a39b1 100644
--- a/drivers/gpu/drm/amd/amdgpu/atombios_crtc.c
+++ b/drivers/gpu/drm/amd/amdgpu/atombios_crtc.c
@@ -461,13 +461,14 @@
PIXEL_CLOCK_PARAMETERS_V3 v3;
PIXEL_CLOCK_PARAMETERS_V5 v5;
PIXEL_CLOCK_PARAMETERS_V6 v6;
+ PIXEL_CLOCK_PARAMETERS_V7 v7;
};
/* on DCE5, make sure the voltage is high enough to support the
* required disp clk.
*/
void amdgpu_atombios_crtc_set_disp_eng_pll(struct amdgpu_device *adev,
- u32 dispclk)
+ u32 dispclk)
{
u8 frev, crev;
int index;
@@ -510,6 +511,49 @@
amdgpu_atom_execute_table(adev->mode_info.atom_context, index, (uint32_t *)&args);
}
+union set_dce_clock {
+ SET_DCE_CLOCK_PS_ALLOCATION_V1_1 v1_1;
+ SET_DCE_CLOCK_PS_ALLOCATION_V2_1 v2_1;
+};
+
+u32 amdgpu_atombios_crtc_set_dce_clock(struct amdgpu_device *adev,
+ u32 freq, u8 clk_type, u8 clk_src)
+{
+ u8 frev, crev;
+ int index;
+ union set_dce_clock args;
+ u32 ret_freq = 0;
+
+ memset(&args, 0, sizeof(args));
+
+ index = GetIndexIntoMasterTable(COMMAND, SetDCEClock);
+ if (!amdgpu_atom_parse_cmd_header(adev->mode_info.atom_context, index, &frev,
+ &crev))
+ return 0;
+
+ switch (frev) {
+ case 2:
+ switch (crev) {
+ case 1:
+ args.v2_1.asParam.ulDCEClkFreq = cpu_to_le32(freq); /* 10kHz units */
+ args.v2_1.asParam.ucDCEClkType = clk_type;
+ args.v2_1.asParam.ucDCEClkSrc = clk_src;
+ amdgpu_atom_execute_table(adev->mode_info.atom_context, index, (uint32_t *)&args);
+ ret_freq = le32_to_cpu(args.v2_1.asParam.ulDCEClkFreq) * 10;
+ break;
+ default:
+ DRM_ERROR("Unknown table version %d %d\n", frev, crev);
+ return 0;
+ }
+ break;
+ default:
+ DRM_ERROR("Unknown table version %d %d\n", frev, crev);
+ return 0;
+ }
+
+ return ret_freq;
+}
+
static bool is_pixel_clock_source_from_pll(u32 encoder_mode, int pll_id)
{
if (ENCODER_MODE_IS_DP(encoder_mode)) {
@@ -523,18 +567,18 @@
}
void amdgpu_atombios_crtc_program_pll(struct drm_crtc *crtc,
- u32 crtc_id,
- int pll_id,
- u32 encoder_mode,
- u32 encoder_id,
- u32 clock,
- u32 ref_div,
- u32 fb_div,
- u32 frac_fb_div,
- u32 post_div,
- int bpc,
- bool ss_enabled,
- struct amdgpu_atom_ss *ss)
+ u32 crtc_id,
+ int pll_id,
+ u32 encoder_mode,
+ u32 encoder_id,
+ u32 clock,
+ u32 ref_div,
+ u32 fb_div,
+ u32 frac_fb_div,
+ u32 post_div,
+ int bpc,
+ bool ss_enabled,
+ struct amdgpu_atom_ss *ss)
{
struct drm_device *dev = crtc->dev;
struct amdgpu_device *adev = dev->dev_private;
@@ -652,6 +696,34 @@
args.v6.ucEncoderMode = encoder_mode;
args.v6.ucPpll = pll_id;
break;
+ case 7:
+ args.v7.ulPixelClock = cpu_to_le32(clock * 10); /* 100 hz units */
+ args.v7.ucMiscInfo = 0;
+ if ((encoder_mode == ATOM_ENCODER_MODE_DVI) &&
+ (clock > 165000))
+ args.v7.ucMiscInfo |= PIXEL_CLOCK_V7_MISC_DVI_DUALLINK_EN;
+ args.v7.ucCRTC = crtc_id;
+ if (encoder_mode == ATOM_ENCODER_MODE_HDMI) {
+ switch (bpc) {
+ case 8:
+ default:
+ args.v7.ucDeepColorRatio = PIXEL_CLOCK_V7_DEEPCOLOR_RATIO_DIS;
+ break;
+ case 10:
+ args.v7.ucDeepColorRatio = PIXEL_CLOCK_V7_DEEPCOLOR_RATIO_5_4;
+ break;
+ case 12:
+ args.v7.ucDeepColorRatio = PIXEL_CLOCK_V7_DEEPCOLOR_RATIO_3_2;
+ break;
+ case 16:
+ args.v7.ucDeepColorRatio = PIXEL_CLOCK_V7_DEEPCOLOR_RATIO_2_1;
+ break;
+ }
+ }
+ args.v7.ucTransmitterID = encoder_id;
+ args.v7.ucEncoderMode = encoder_mode;
+ args.v7.ucPpll = pll_id;
+ break;
default:
DRM_ERROR("Unknown table version %d %d\n", frev, crev);
return;
diff --git a/drivers/gpu/drm/amd/amdgpu/atombios_crtc.h b/drivers/gpu/drm/amd/amdgpu/atombios_crtc.h
index c670833..0eeda8e 100644
--- a/drivers/gpu/drm/amd/amdgpu/atombios_crtc.h
+++ b/drivers/gpu/drm/amd/amdgpu/atombios_crtc.h
@@ -37,6 +37,8 @@
struct drm_display_mode *mode);
void amdgpu_atombios_crtc_set_disp_eng_pll(struct amdgpu_device *adev,
u32 dispclk);
+u32 amdgpu_atombios_crtc_set_dce_clock(struct amdgpu_device *adev,
+ u32 freq, u8 clk_type, u8 clk_src);
void amdgpu_atombios_crtc_program_pll(struct drm_crtc *crtc,
u32 crtc_id,
int pll_id,
diff --git a/drivers/gpu/drm/amd/amdgpu/atombios_encoders.c b/drivers/gpu/drm/amd/amdgpu/atombios_encoders.c
index 1cd6de5..48b6bd6 100644
--- a/drivers/gpu/drm/amd/amdgpu/atombios_encoders.c
+++ b/drivers/gpu/drm/amd/amdgpu/atombios_encoders.c
@@ -567,6 +567,7 @@
DIG_ENCODER_CONTROL_PARAMETERS_V2 v2;
DIG_ENCODER_CONTROL_PARAMETERS_V3 v3;
DIG_ENCODER_CONTROL_PARAMETERS_V4 v4;
+ DIG_ENCODER_CONTROL_PARAMETERS_V5 v5;
};
void
@@ -694,6 +695,47 @@
else
args.v4.ucHPD_ID = hpd_id + 1;
break;
+ case 5:
+ switch (action) {
+ case ATOM_ENCODER_CMD_SETUP_PANEL_MODE:
+ args.v5.asDPPanelModeParam.ucAction = action;
+ args.v5.asDPPanelModeParam.ucPanelMode = panel_mode;
+ args.v5.asDPPanelModeParam.ucDigId = dig->dig_encoder;
+ break;
+ case ATOM_ENCODER_CMD_STREAM_SETUP:
+ args.v5.asStreamParam.ucAction = action;
+ args.v5.asStreamParam.ucDigId = dig->dig_encoder;
+ args.v5.asStreamParam.ucDigMode =
+ amdgpu_atombios_encoder_get_encoder_mode(encoder);
+ if (ENCODER_MODE_IS_DP(args.v5.asStreamParam.ucDigMode))
+ args.v5.asStreamParam.ucLaneNum = dp_lane_count;
+ else if (amdgpu_dig_monitor_is_duallink(encoder,
+ amdgpu_encoder->pixel_clock))
+ args.v5.asStreamParam.ucLaneNum = 8;
+ else
+ args.v5.asStreamParam.ucLaneNum = 4;
+ args.v5.asStreamParam.ulPixelClock =
+ cpu_to_le32(amdgpu_encoder->pixel_clock / 10);
+ args.v5.asStreamParam.ucBitPerColor =
+ amdgpu_atombios_encoder_get_bpc(encoder);
+ args.v5.asStreamParam.ucLinkRateIn270Mhz = dp_clock / 27000;
+ break;
+ case ATOM_ENCODER_CMD_DP_LINK_TRAINING_START:
+ case ATOM_ENCODER_CMD_DP_LINK_TRAINING_PATTERN1:
+ case ATOM_ENCODER_CMD_DP_LINK_TRAINING_PATTERN2:
+ case ATOM_ENCODER_CMD_DP_LINK_TRAINING_PATTERN3:
+ case ATOM_ENCODER_CMD_DP_LINK_TRAINING_PATTERN4:
+ case ATOM_ENCODER_CMD_DP_LINK_TRAINING_COMPLETE:
+ case ATOM_ENCODER_CMD_DP_VIDEO_OFF:
+ case ATOM_ENCODER_CMD_DP_VIDEO_ON:
+ args.v5.asCmdParam.ucAction = action;
+ args.v5.asCmdParam.ucDigId = dig->dig_encoder;
+ break;
+ default:
+ DRM_ERROR("Unsupported action 0x%x\n", action);
+ break;
+ }
+ break;
default:
DRM_ERROR("Unknown table version %d, %d\n", frev, crev);
break;
@@ -714,11 +756,12 @@
DIG_TRANSMITTER_CONTROL_PARAMETERS_V3 v3;
DIG_TRANSMITTER_CONTROL_PARAMETERS_V4 v4;
DIG_TRANSMITTER_CONTROL_PARAMETERS_V1_5 v5;
+ DIG_TRANSMITTER_CONTROL_PARAMETERS_V1_6 v6;
};
void
amdgpu_atombios_encoder_setup_dig_transmitter(struct drm_encoder *encoder, int action,
- uint8_t lane_num, uint8_t lane_set)
+ uint8_t lane_num, uint8_t lane_set)
{
struct drm_device *dev = encoder->dev;
struct amdgpu_device *adev = dev->dev_private;
@@ -1070,6 +1113,54 @@
args.v5.ucDigEncoderSel = 1 << dig_encoder;
args.v5.ucDPLaneSet = lane_set;
break;
+ case 6:
+ args.v6.ucAction = action;
+ if (is_dp)
+ args.v6.ulSymClock = cpu_to_le32(dp_clock / 10);
+ else
+ args.v6.ulSymClock = cpu_to_le32(amdgpu_encoder->pixel_clock / 10);
+
+ switch (amdgpu_encoder->encoder_id) {
+ case ENCODER_OBJECT_ID_INTERNAL_UNIPHY:
+ if (dig->linkb)
+ args.v6.ucPhyId = ATOM_PHY_ID_UNIPHYB;
+ else
+ args.v6.ucPhyId = ATOM_PHY_ID_UNIPHYA;
+ break;
+ case ENCODER_OBJECT_ID_INTERNAL_UNIPHY1:
+ if (dig->linkb)
+ args.v6.ucPhyId = ATOM_PHY_ID_UNIPHYD;
+ else
+ args.v6.ucPhyId = ATOM_PHY_ID_UNIPHYC;
+ break;
+ case ENCODER_OBJECT_ID_INTERNAL_UNIPHY2:
+ if (dig->linkb)
+ args.v6.ucPhyId = ATOM_PHY_ID_UNIPHYF;
+ else
+ args.v6.ucPhyId = ATOM_PHY_ID_UNIPHYE;
+ break;
+ case ENCODER_OBJECT_ID_INTERNAL_UNIPHY3:
+ args.v6.ucPhyId = ATOM_PHY_ID_UNIPHYG;
+ break;
+ }
+ if (is_dp)
+ args.v6.ucLaneNum = dp_lane_count;
+ else if (amdgpu_dig_monitor_is_duallink(encoder, amdgpu_encoder->pixel_clock))
+ args.v6.ucLaneNum = 8;
+ else
+ args.v6.ucLaneNum = 4;
+ args.v6.ucConnObjId = connector_object_id;
+ if (action == ATOM_TRANSMITTER_ACTION_SETUP_VSEMPH)
+ args.v6.ucDPLaneSet = lane_set;
+ else
+ args.v6.ucDigMode = amdgpu_atombios_encoder_get_encoder_mode(encoder);
+
+ if (hpd_id == AMDGPU_HPD_NONE)
+ args.v6.ucHPDSel = 0;
+ else
+ args.v6.ucHPDSel = hpd_id + 1;
+ args.v6.ucDigEncoderSel = 1 << dig_encoder;
+ break;
default:
DRM_ERROR("Unknown table version %d, %d\n", frev, crev);
break;
diff --git a/drivers/gpu/drm/amd/amdgpu/ci_dpm.c b/drivers/gpu/drm/amd/amdgpu/ci_dpm.c
index 1f9109d..ea407db 100644
--- a/drivers/gpu/drm/amd/amdgpu/ci_dpm.c
+++ b/drivers/gpu/drm/amd/amdgpu/ci_dpm.c
@@ -2549,19 +2549,17 @@
return 0;
}
-static u8 ci_get_sleep_divider_id_from_clock(struct amdgpu_device *adev,
- u32 sclk, u32 min_sclk_in_sr)
+static u8 ci_get_sleep_divider_id_from_clock(u32 sclk, u32 min_sclk_in_sr)
{
u32 i;
u32 tmp;
- u32 min = (min_sclk_in_sr > CISLAND_MINIMUM_ENGINE_CLOCK) ?
- min_sclk_in_sr : CISLAND_MINIMUM_ENGINE_CLOCK;
+ u32 min = max(min_sclk_in_sr, (u32)CISLAND_MINIMUM_ENGINE_CLOCK);
if (sclk < min)
return 0;
for (i = CISLAND_MAX_DEEPSLEEP_DIVIDER_ID; ; i--) {
- tmp = sclk / (1 << i);
+ tmp = sclk >> i;
if (tmp >= min || i == 0)
break;
}
@@ -3358,8 +3356,7 @@
graphic_level->PowerThrottle = 0;
if (pi->caps_sclk_ds)
- graphic_level->DeepSleepDivId = ci_get_sleep_divider_id_from_clock(adev,
- engine_clock,
+ graphic_level->DeepSleepDivId = ci_get_sleep_divider_id_from_clock(engine_clock,
CISLAND_MINIMUM_ENGINE_CLOCK);
graphic_level->DisplayWatermark = PPSMC_DISPLAY_WATERMARK_LOW;
@@ -6309,215 +6306,6 @@
return 0;
}
-static void ci_dpm_print_status(void *handle)
-{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- dev_info(adev->dev, "CIK DPM registers\n");
- dev_info(adev->dev, " BIOS_SCRATCH_4=0x%08X\n",
- RREG32(mmBIOS_SCRATCH_4));
- dev_info(adev->dev, " MC_ARB_DRAM_TIMING=0x%08X\n",
- RREG32(mmMC_ARB_DRAM_TIMING));
- dev_info(adev->dev, " MC_ARB_DRAM_TIMING2=0x%08X\n",
- RREG32(mmMC_ARB_DRAM_TIMING2));
- dev_info(adev->dev, " MC_ARB_BURST_TIME=0x%08X\n",
- RREG32(mmMC_ARB_BURST_TIME));
- dev_info(adev->dev, " MC_ARB_DRAM_TIMING_1=0x%08X\n",
- RREG32(mmMC_ARB_DRAM_TIMING_1));
- dev_info(adev->dev, " MC_ARB_DRAM_TIMING2_1=0x%08X\n",
- RREG32(mmMC_ARB_DRAM_TIMING2_1));
- dev_info(adev->dev, " MC_CG_CONFIG=0x%08X\n",
- RREG32(mmMC_CG_CONFIG));
- dev_info(adev->dev, " MC_ARB_CG=0x%08X\n",
- RREG32(mmMC_ARB_CG));
- dev_info(adev->dev, " DIDT_SQ_CTRL0=0x%08X\n",
- RREG32_DIDT(ixDIDT_SQ_CTRL0));
- dev_info(adev->dev, " DIDT_DB_CTRL0=0x%08X\n",
- RREG32_DIDT(ixDIDT_DB_CTRL0));
- dev_info(adev->dev, " DIDT_TD_CTRL0=0x%08X\n",
- RREG32_DIDT(ixDIDT_TD_CTRL0));
- dev_info(adev->dev, " DIDT_TCP_CTRL0=0x%08X\n",
- RREG32_DIDT(ixDIDT_TCP_CTRL0));
- dev_info(adev->dev, " CG_THERMAL_INT=0x%08X\n",
- RREG32_SMC(ixCG_THERMAL_INT));
- dev_info(adev->dev, " CG_THERMAL_CTRL=0x%08X\n",
- RREG32_SMC(ixCG_THERMAL_CTRL));
- dev_info(adev->dev, " GENERAL_PWRMGT=0x%08X\n",
- RREG32_SMC(ixGENERAL_PWRMGT));
- dev_info(adev->dev, " MC_SEQ_CNTL_3=0x%08X\n",
- RREG32(mmMC_SEQ_CNTL_3));
- dev_info(adev->dev, " LCAC_MC0_CNTL=0x%08X\n",
- RREG32_SMC(ixLCAC_MC0_CNTL));
- dev_info(adev->dev, " LCAC_MC1_CNTL=0x%08X\n",
- RREG32_SMC(ixLCAC_MC1_CNTL));
- dev_info(adev->dev, " LCAC_CPL_CNTL=0x%08X\n",
- RREG32_SMC(ixLCAC_CPL_CNTL));
- dev_info(adev->dev, " SCLK_PWRMGT_CNTL=0x%08X\n",
- RREG32_SMC(ixSCLK_PWRMGT_CNTL));
- dev_info(adev->dev, " BIF_LNCNT_RESET=0x%08X\n",
- RREG32(mmBIF_LNCNT_RESET));
- dev_info(adev->dev, " FIRMWARE_FLAGS=0x%08X\n",
- RREG32_SMC(ixFIRMWARE_FLAGS));
- dev_info(adev->dev, " CG_SPLL_FUNC_CNTL=0x%08X\n",
- RREG32_SMC(ixCG_SPLL_FUNC_CNTL));
- dev_info(adev->dev, " CG_SPLL_FUNC_CNTL_2=0x%08X\n",
- RREG32_SMC(ixCG_SPLL_FUNC_CNTL_2));
- dev_info(adev->dev, " CG_SPLL_FUNC_CNTL_3=0x%08X\n",
- RREG32_SMC(ixCG_SPLL_FUNC_CNTL_3));
- dev_info(adev->dev, " CG_SPLL_FUNC_CNTL_4=0x%08X\n",
- RREG32_SMC(ixCG_SPLL_FUNC_CNTL_4));
- dev_info(adev->dev, " CG_SPLL_SPREAD_SPECTRUM=0x%08X\n",
- RREG32_SMC(ixCG_SPLL_SPREAD_SPECTRUM));
- dev_info(adev->dev, " CG_SPLL_SPREAD_SPECTRUM_2=0x%08X\n",
- RREG32_SMC(ixCG_SPLL_SPREAD_SPECTRUM_2));
- dev_info(adev->dev, " DLL_CNTL=0x%08X\n",
- RREG32(mmDLL_CNTL));
- dev_info(adev->dev, " MCLK_PWRMGT_CNTL=0x%08X\n",
- RREG32(mmMCLK_PWRMGT_CNTL));
- dev_info(adev->dev, " MPLL_AD_FUNC_CNTL=0x%08X\n",
- RREG32(mmMPLL_AD_FUNC_CNTL));
- dev_info(adev->dev, " MPLL_DQ_FUNC_CNTL=0x%08X\n",
- RREG32(mmMPLL_DQ_FUNC_CNTL));
- dev_info(adev->dev, " MPLL_FUNC_CNTL=0x%08X\n",
- RREG32(mmMPLL_FUNC_CNTL));
- dev_info(adev->dev, " MPLL_FUNC_CNTL_1=0x%08X\n",
- RREG32(mmMPLL_FUNC_CNTL_1));
- dev_info(adev->dev, " MPLL_FUNC_CNTL_2=0x%08X\n",
- RREG32(mmMPLL_FUNC_CNTL_2));
- dev_info(adev->dev, " MPLL_SS1=0x%08X\n",
- RREG32(mmMPLL_SS1));
- dev_info(adev->dev, " MPLL_SS2=0x%08X\n",
- RREG32(mmMPLL_SS2));
- dev_info(adev->dev, " CG_DISPLAY_GAP_CNTL=0x%08X\n",
- RREG32_SMC(ixCG_DISPLAY_GAP_CNTL));
- dev_info(adev->dev, " CG_DISPLAY_GAP_CNTL2=0x%08X\n",
- RREG32_SMC(ixCG_DISPLAY_GAP_CNTL2));
- dev_info(adev->dev, " CG_STATIC_SCREEN_PARAMETER=0x%08X\n",
- RREG32_SMC(ixCG_STATIC_SCREEN_PARAMETER));
- dev_info(adev->dev, " CG_FREQ_TRAN_VOTING_0=0x%08X\n",
- RREG32_SMC(ixCG_FREQ_TRAN_VOTING_0));
- dev_info(adev->dev, " CG_FREQ_TRAN_VOTING_1=0x%08X\n",
- RREG32_SMC(ixCG_FREQ_TRAN_VOTING_1));
- dev_info(adev->dev, " CG_FREQ_TRAN_VOTING_2=0x%08X\n",
- RREG32_SMC(ixCG_FREQ_TRAN_VOTING_2));
- dev_info(adev->dev, " CG_FREQ_TRAN_VOTING_3=0x%08X\n",
- RREG32_SMC(ixCG_FREQ_TRAN_VOTING_3));
- dev_info(adev->dev, " CG_FREQ_TRAN_VOTING_4=0x%08X\n",
- RREG32_SMC(ixCG_FREQ_TRAN_VOTING_4));
- dev_info(adev->dev, " CG_FREQ_TRAN_VOTING_5=0x%08X\n",
- RREG32_SMC(ixCG_FREQ_TRAN_VOTING_5));
- dev_info(adev->dev, " CG_FREQ_TRAN_VOTING_6=0x%08X\n",
- RREG32_SMC(ixCG_FREQ_TRAN_VOTING_6));
- dev_info(adev->dev, " CG_FREQ_TRAN_VOTING_7=0x%08X\n",
- RREG32_SMC(ixCG_FREQ_TRAN_VOTING_7));
- dev_info(adev->dev, " RCU_UC_EVENTS=0x%08X\n",
- RREG32_SMC(ixRCU_UC_EVENTS));
- dev_info(adev->dev, " DPM_TABLE_475=0x%08X\n",
- RREG32_SMC(ixDPM_TABLE_475));
- dev_info(adev->dev, " MC_SEQ_RAS_TIMING_LP=0x%08X\n",
- RREG32(mmMC_SEQ_RAS_TIMING_LP));
- dev_info(adev->dev, " MC_SEQ_RAS_TIMING=0x%08X\n",
- RREG32(mmMC_SEQ_RAS_TIMING));
- dev_info(adev->dev, " MC_SEQ_CAS_TIMING_LP=0x%08X\n",
- RREG32(mmMC_SEQ_CAS_TIMING_LP));
- dev_info(adev->dev, " MC_SEQ_CAS_TIMING=0x%08X\n",
- RREG32(mmMC_SEQ_CAS_TIMING));
- dev_info(adev->dev, " MC_SEQ_DLL_STBY_LP=0x%08X\n",
- RREG32(mmMC_SEQ_DLL_STBY_LP));
- dev_info(adev->dev, " MC_SEQ_DLL_STBY=0x%08X\n",
- RREG32(mmMC_SEQ_DLL_STBY));
- dev_info(adev->dev, " MC_SEQ_G5PDX_CMD0_LP=0x%08X\n",
- RREG32(mmMC_SEQ_G5PDX_CMD0_LP));
- dev_info(adev->dev, " MC_SEQ_G5PDX_CMD0=0x%08X\n",
- RREG32(mmMC_SEQ_G5PDX_CMD0));
- dev_info(adev->dev, " MC_SEQ_G5PDX_CMD1_LP=0x%08X\n",
- RREG32(mmMC_SEQ_G5PDX_CMD1_LP));
- dev_info(adev->dev, " MC_SEQ_G5PDX_CMD1=0x%08X\n",
- RREG32(mmMC_SEQ_G5PDX_CMD1));
- dev_info(adev->dev, " MC_SEQ_G5PDX_CTRL_LP=0x%08X\n",
- RREG32(mmMC_SEQ_G5PDX_CTRL_LP));
- dev_info(adev->dev, " MC_SEQ_G5PDX_CTRL=0x%08X\n",
- RREG32(mmMC_SEQ_G5PDX_CTRL));
- dev_info(adev->dev, " MC_SEQ_PMG_DVS_CMD_LP=0x%08X\n",
- RREG32(mmMC_SEQ_PMG_DVS_CMD_LP));
- dev_info(adev->dev, " MC_SEQ_PMG_DVS_CMD=0x%08X\n",
- RREG32(mmMC_SEQ_PMG_DVS_CMD));
- dev_info(adev->dev, " MC_SEQ_PMG_DVS_CTL_LP=0x%08X\n",
- RREG32(mmMC_SEQ_PMG_DVS_CTL_LP));
- dev_info(adev->dev, " MC_SEQ_PMG_DVS_CTL=0x%08X\n",
- RREG32(mmMC_SEQ_PMG_DVS_CTL));
- dev_info(adev->dev, " MC_SEQ_MISC_TIMING_LP=0x%08X\n",
- RREG32(mmMC_SEQ_MISC_TIMING_LP));
- dev_info(adev->dev, " MC_SEQ_MISC_TIMING=0x%08X\n",
- RREG32(mmMC_SEQ_MISC_TIMING));
- dev_info(adev->dev, " MC_SEQ_MISC_TIMING2_LP=0x%08X\n",
- RREG32(mmMC_SEQ_MISC_TIMING2_LP));
- dev_info(adev->dev, " MC_SEQ_MISC_TIMING2=0x%08X\n",
- RREG32(mmMC_SEQ_MISC_TIMING2));
- dev_info(adev->dev, " MC_SEQ_PMG_CMD_EMRS_LP=0x%08X\n",
- RREG32(mmMC_SEQ_PMG_CMD_EMRS_LP));
- dev_info(adev->dev, " MC_PMG_CMD_EMRS=0x%08X\n",
- RREG32(mmMC_PMG_CMD_EMRS));
- dev_info(adev->dev, " MC_SEQ_PMG_CMD_MRS_LP=0x%08X\n",
- RREG32(mmMC_SEQ_PMG_CMD_MRS_LP));
- dev_info(adev->dev, " MC_PMG_CMD_MRS=0x%08X\n",
- RREG32(mmMC_PMG_CMD_MRS));
- dev_info(adev->dev, " MC_SEQ_PMG_CMD_MRS1_LP=0x%08X\n",
- RREG32(mmMC_SEQ_PMG_CMD_MRS1_LP));
- dev_info(adev->dev, " MC_PMG_CMD_MRS1=0x%08X\n",
- RREG32(mmMC_PMG_CMD_MRS1));
- dev_info(adev->dev, " MC_SEQ_WR_CTL_D0_LP=0x%08X\n",
- RREG32(mmMC_SEQ_WR_CTL_D0_LP));
- dev_info(adev->dev, " MC_SEQ_WR_CTL_D0=0x%08X\n",
- RREG32(mmMC_SEQ_WR_CTL_D0));
- dev_info(adev->dev, " MC_SEQ_WR_CTL_D1_LP=0x%08X\n",
- RREG32(mmMC_SEQ_WR_CTL_D1_LP));
- dev_info(adev->dev, " MC_SEQ_WR_CTL_D1=0x%08X\n",
- RREG32(mmMC_SEQ_WR_CTL_D1));
- dev_info(adev->dev, " MC_SEQ_RD_CTL_D0_LP=0x%08X\n",
- RREG32(mmMC_SEQ_RD_CTL_D0_LP));
- dev_info(adev->dev, " MC_SEQ_RD_CTL_D0=0x%08X\n",
- RREG32(mmMC_SEQ_RD_CTL_D0));
- dev_info(adev->dev, " MC_SEQ_RD_CTL_D1_LP=0x%08X\n",
- RREG32(mmMC_SEQ_RD_CTL_D1_LP));
- dev_info(adev->dev, " MC_SEQ_RD_CTL_D1=0x%08X\n",
- RREG32(mmMC_SEQ_RD_CTL_D1));
- dev_info(adev->dev, " MC_SEQ_PMG_TIMING_LP=0x%08X\n",
- RREG32(mmMC_SEQ_PMG_TIMING_LP));
- dev_info(adev->dev, " MC_SEQ_PMG_TIMING=0x%08X\n",
- RREG32(mmMC_SEQ_PMG_TIMING));
- dev_info(adev->dev, " MC_SEQ_PMG_CMD_MRS2_LP=0x%08X\n",
- RREG32(mmMC_SEQ_PMG_CMD_MRS2_LP));
- dev_info(adev->dev, " MC_PMG_CMD_MRS2=0x%08X\n",
- RREG32(mmMC_PMG_CMD_MRS2));
- dev_info(adev->dev, " MC_SEQ_WR_CTL_2_LP=0x%08X\n",
- RREG32(mmMC_SEQ_WR_CTL_2_LP));
- dev_info(adev->dev, " MC_SEQ_WR_CTL_2=0x%08X\n",
- RREG32(mmMC_SEQ_WR_CTL_2));
- dev_info(adev->dev, " PCIE_LC_SPEED_CNTL=0x%08X\n",
- RREG32_PCIE(ixPCIE_LC_SPEED_CNTL));
- dev_info(adev->dev, " PCIE_LC_LINK_WIDTH_CNTL=0x%08X\n",
- RREG32_PCIE(ixPCIE_LC_LINK_WIDTH_CNTL));
- dev_info(adev->dev, " SMC_IND_INDEX_0=0x%08X\n",
- RREG32(mmSMC_IND_INDEX_0));
- dev_info(adev->dev, " SMC_IND_DATA_0=0x%08X\n",
- RREG32(mmSMC_IND_DATA_0));
- dev_info(adev->dev, " SMC_IND_ACCESS_CNTL=0x%08X\n",
- RREG32(mmSMC_IND_ACCESS_CNTL));
- dev_info(adev->dev, " SMC_RESP_0=0x%08X\n",
- RREG32(mmSMC_RESP_0));
- dev_info(adev->dev, " SMC_MESSAGE_0=0x%08X\n",
- RREG32(mmSMC_MESSAGE_0));
- dev_info(adev->dev, " SMC_SYSCON_RESET_CNTL=0x%08X\n",
- RREG32_SMC(ixSMC_SYSCON_RESET_CNTL));
- dev_info(adev->dev, " SMC_SYSCON_CLOCK_CNTL_0=0x%08X\n",
- RREG32_SMC(ixSMC_SYSCON_CLOCK_CNTL_0));
- dev_info(adev->dev, " SMC_SYSCON_MISC_CNTL=0x%08X\n",
- RREG32_SMC(ixSMC_SYSCON_MISC_CNTL));
- dev_info(adev->dev, " SMC_PC_C=0x%08X\n",
- RREG32_SMC(ixSMC_PC_C));
-}
-
static int ci_dpm_soft_reset(void *handle)
{
return 0;
@@ -6572,7 +6360,7 @@
}
static int ci_dpm_process_interrupt(struct amdgpu_device *adev,
- struct amdgpu_irq_src *source,
+ struct amdgpu_irq_src *source,
struct amdgpu_iv_entry *entry)
{
bool queue_thermal = false;
@@ -6614,6 +6402,7 @@
}
const struct amd_ip_funcs ci_dpm_ip_funcs = {
+ .name = "ci_dpm",
.early_init = ci_dpm_early_init,
.late_init = ci_dpm_late_init,
.sw_init = ci_dpm_sw_init,
@@ -6625,7 +6414,6 @@
.is_idle = ci_dpm_is_idle,
.wait_for_idle = ci_dpm_wait_for_idle,
.soft_reset = ci_dpm_soft_reset,
- .print_status = ci_dpm_print_status,
.set_clockgating_state = ci_dpm_set_clockgating_state,
.set_powergating_state = ci_dpm_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/cik.c b/drivers/gpu/drm/amd/amdgpu/cik.c
index bddc9ba..07bc795 100644
--- a/drivers/gpu/drm/amd/amdgpu/cik.c
+++ b/drivers/gpu/drm/amd/amdgpu/cik.c
@@ -962,7 +962,7 @@
return true;
}
-static struct amdgpu_allowed_register_entry cik_allowed_read_registers[] = {
+static const struct amdgpu_allowed_register_entry cik_allowed_read_registers[] = {
{mmGRBM_STATUS, false},
{mmGB_ADDR_CONFIG, false},
{mmMC_ARB_RAMCFG, false},
@@ -2007,7 +2007,6 @@
.get_xclk = &cik_get_xclk,
.set_uvd_clocks = &cik_set_uvd_clocks,
.set_vce_clocks = &cik_set_vce_clocks,
- .get_cu_info = &gfx_v7_0_get_cu_info,
/* these should be moved to their own ip modules */
.get_gpu_clock_counter = &gfx_v7_0_get_gpu_clock_counter,
.wait_for_mc_idle = &gmc_v7_0_mc_wait_for_idle,
@@ -2214,11 +2213,6 @@
return 0;
}
-static void cik_common_print_status(void *handle)
-{
-
-}
-
static int cik_common_soft_reset(void *handle)
{
/* XXX hard reset?? */
@@ -2238,6 +2232,7 @@
}
const struct amd_ip_funcs cik_common_ip_funcs = {
+ .name = "cik_common",
.early_init = cik_common_early_init,
.late_init = NULL,
.sw_init = cik_common_sw_init,
@@ -2249,7 +2244,6 @@
.is_idle = cik_common_is_idle,
.wait_for_idle = cik_common_wait_for_idle,
.soft_reset = cik_common_soft_reset,
- .print_status = cik_common_print_status,
.set_clockgating_state = cik_common_set_clockgating_state,
.set_powergating_state = cik_common_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
index 30c9b3be..845c21b 100644
--- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
@@ -243,7 +243,7 @@
/* wptr/rptr are in bytes! */
u32 ring_index = adev->irq.ih.rptr >> 2;
uint32_t dw[4];
-
+
dw[0] = le32_to_cpu(adev->irq.ih.ring[ring_index + 0]);
dw[1] = le32_to_cpu(adev->irq.ih.ring[ring_index + 1]);
dw[2] = le32_to_cpu(adev->irq.ih.ring[ring_index + 2]);
@@ -372,35 +372,6 @@
return -ETIMEDOUT;
}
-static void cik_ih_print_status(void *handle)
-{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- dev_info(adev->dev, "CIK IH registers\n");
- dev_info(adev->dev, " SRBM_STATUS=0x%08X\n",
- RREG32(mmSRBM_STATUS));
- dev_info(adev->dev, " SRBM_STATUS2=0x%08X\n",
- RREG32(mmSRBM_STATUS2));
- dev_info(adev->dev, " INTERRUPT_CNTL=0x%08X\n",
- RREG32(mmINTERRUPT_CNTL));
- dev_info(adev->dev, " INTERRUPT_CNTL2=0x%08X\n",
- RREG32(mmINTERRUPT_CNTL2));
- dev_info(adev->dev, " IH_CNTL=0x%08X\n",
- RREG32(mmIH_CNTL));
- dev_info(adev->dev, " IH_RB_CNTL=0x%08X\n",
- RREG32(mmIH_RB_CNTL));
- dev_info(adev->dev, " IH_RB_BASE=0x%08X\n",
- RREG32(mmIH_RB_BASE));
- dev_info(adev->dev, " IH_RB_WPTR_ADDR_LO=0x%08X\n",
- RREG32(mmIH_RB_WPTR_ADDR_LO));
- dev_info(adev->dev, " IH_RB_WPTR_ADDR_HI=0x%08X\n",
- RREG32(mmIH_RB_WPTR_ADDR_HI));
- dev_info(adev->dev, " IH_RB_RPTR=0x%08X\n",
- RREG32(mmIH_RB_RPTR));
- dev_info(adev->dev, " IH_RB_WPTR=0x%08X\n",
- RREG32(mmIH_RB_WPTR));
-}
-
static int cik_ih_soft_reset(void *handle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
@@ -412,8 +383,6 @@
srbm_soft_reset |= SRBM_SOFT_RESET__SOFT_RESET_IH_MASK;
if (srbm_soft_reset) {
- cik_ih_print_status((void *)adev);
-
tmp = RREG32(mmSRBM_SOFT_RESET);
tmp |= srbm_soft_reset;
dev_info(adev->dev, "SRBM_SOFT_RESET=0x%08X\n", tmp);
@@ -428,8 +397,6 @@
/* Wait a little for things to settle down */
udelay(50);
-
- cik_ih_print_status((void *)adev);
}
return 0;
@@ -448,6 +415,7 @@
}
const struct amd_ip_funcs cik_ih_ip_funcs = {
+ .name = "cik_ih",
.early_init = cik_ih_early_init,
.late_init = NULL,
.sw_init = cik_ih_sw_init,
@@ -459,7 +427,6 @@
.is_idle = cik_ih_is_idle,
.wait_for_idle = cik_ih_wait_for_idle,
.soft_reset = cik_ih_soft_reset,
- .print_status = cik_ih_print_status,
.set_clockgating_state = cik_ih_set_clockgating_state,
.set_powergating_state = cik_ih_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/cik_sdma.c b/drivers/gpu/drm/amd/amdgpu/cik_sdma.c
index d3ac329..518dca4 100644
--- a/drivers/gpu/drm/amd/amdgpu/cik_sdma.c
+++ b/drivers/gpu/drm/amd/amdgpu/cik_sdma.c
@@ -210,9 +210,10 @@
* Schedule an IB in the DMA ring (CIK).
*/
static void cik_sdma_ring_emit_ib(struct amdgpu_ring *ring,
- struct amdgpu_ib *ib)
+ struct amdgpu_ib *ib,
+ unsigned vm_id, bool ctx_switch)
{
- u32 extra_bits = ib->vm_id & 0xf;
+ u32 extra_bits = vm_id & 0xf;
u32 next_rptr = ring->wptr + 5;
while ((next_rptr & 7) != 4)
@@ -643,7 +644,7 @@
ib.ptr[3] = 1;
ib.ptr[4] = 0xDEADBEEF;
ib.length_dw = 5;
- r = amdgpu_ib_schedule(ring, 1, &ib, NULL, &f);
+ r = amdgpu_ib_schedule(ring, 1, &ib, NULL, NULL, &f);
if (r)
goto err1;
@@ -976,7 +977,7 @@
ring = &adev->sdma.instance[i].ring;
ring->ring_obj = NULL;
sprintf(ring->name, "sdma%d", i);
- r = amdgpu_ring_init(adev, ring, 256 * 1024,
+ r = amdgpu_ring_init(adev, ring, 1024,
SDMA_PACKET(SDMA_OPCODE_NOP, 0, 0), 0xf,
&adev->sdma.trap_irq,
(i == 0) ?
@@ -1064,57 +1065,6 @@
return -ETIMEDOUT;
}
-static void cik_sdma_print_status(void *handle)
-{
- int i, j;
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- dev_info(adev->dev, "CIK SDMA registers\n");
- dev_info(adev->dev, " SRBM_STATUS2=0x%08X\n",
- RREG32(mmSRBM_STATUS2));
- for (i = 0; i < adev->sdma.num_instances; i++) {
- dev_info(adev->dev, " SDMA%d_STATUS_REG=0x%08X\n",
- i, RREG32(mmSDMA0_STATUS_REG + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_ME_CNTL=0x%08X\n",
- i, RREG32(mmSDMA0_F32_CNTL + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_CNTL=0x%08X\n",
- i, RREG32(mmSDMA0_CNTL + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_SEM_INCOMPLETE_TIMER_CNTL=0x%08X\n",
- i, RREG32(mmSDMA0_SEM_INCOMPLETE_TIMER_CNTL + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_SEM_WAIT_FAIL_TIMER_CNTL=0x%08X\n",
- i, RREG32(mmSDMA0_SEM_WAIT_FAIL_TIMER_CNTL + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_IB_CNTL=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_IB_CNTL + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_RB_CNTL=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_RB_CNTL + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_RB_RPTR=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_RB_RPTR + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_RB_WPTR=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_RB_WPTR + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_RB_RPTR_ADDR_HI=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_RB_RPTR_ADDR_HI + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_RB_RPTR_ADDR_LO=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_RB_RPTR_ADDR_LO + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_RB_BASE=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_RB_BASE + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_RB_BASE_HI=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_RB_BASE_HI + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_TILING_CONFIG=0x%08X\n",
- i, RREG32(mmSDMA0_TILING_CONFIG + sdma_offsets[i]));
- mutex_lock(&adev->srbm_mutex);
- for (j = 0; j < 16; j++) {
- cik_srbm_select(adev, 0, 0, 0, j);
- dev_info(adev->dev, " VM %d:\n", j);
- dev_info(adev->dev, " SDMA0_GFX_VIRTUAL_ADDR=0x%08X\n",
- RREG32(mmSDMA0_GFX_VIRTUAL_ADDR + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA0_GFX_APE1_CNTL=0x%08X\n",
- RREG32(mmSDMA0_GFX_APE1_CNTL + sdma_offsets[i]));
- }
- cik_srbm_select(adev, 0, 0, 0, 0);
- mutex_unlock(&adev->srbm_mutex);
- }
-}
-
static int cik_sdma_soft_reset(void *handle)
{
u32 srbm_soft_reset = 0;
@@ -1137,8 +1087,6 @@
}
if (srbm_soft_reset) {
- cik_sdma_print_status((void *)adev);
-
tmp = RREG32(mmSRBM_SOFT_RESET);
tmp |= srbm_soft_reset;
dev_info(adev->dev, "SRBM_SOFT_RESET=0x%08X\n", tmp);
@@ -1153,8 +1101,6 @@
/* Wait a little for things to settle down */
udelay(50);
-
- cik_sdma_print_status((void *)adev);
}
return 0;
@@ -1278,6 +1224,7 @@
}
const struct amd_ip_funcs cik_sdma_ip_funcs = {
+ .name = "cik_sdma",
.early_init = cik_sdma_early_init,
.late_init = NULL,
.sw_init = cik_sdma_sw_init,
@@ -1289,7 +1236,6 @@
.is_idle = cik_sdma_is_idle,
.wait_for_idle = cik_sdma_wait_for_idle,
.soft_reset = cik_sdma_soft_reset,
- .print_status = cik_sdma_print_status,
.set_clockgating_state = cik_sdma_set_clockgating_state,
.set_powergating_state = cik_sdma_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/cikd.h b/drivers/gpu/drm/amd/amdgpu/cikd.h
index 60d4493..c4f6f00 100644
--- a/drivers/gpu/drm/amd/amdgpu/cikd.h
+++ b/drivers/gpu/drm/amd/amdgpu/cikd.h
@@ -190,8 +190,8 @@
# define MACRO_TILE_ASPECT(x) ((x) << 4)
# define NUM_BANKS(x) ((x) << 6)
-#define MSG_ENTER_RLC_SAFE_MODE 1
-#define MSG_EXIT_RLC_SAFE_MODE 0
+#define MSG_ENTER_RLC_SAFE_MODE 1
+#define MSG_EXIT_RLC_SAFE_MODE 0
/*
* PM4
diff --git a/drivers/gpu/drm/amd/amdgpu/cz_dpm.c b/drivers/gpu/drm/amd/amdgpu/cz_dpm.c
index e7ef226..fa4449e 100644
--- a/drivers/gpu/drm/amd/amdgpu/cz_dpm.c
+++ b/drivers/gpu/drm/amd/amdgpu/cz_dpm.c
@@ -2230,6 +2230,7 @@
}
const struct amd_ip_funcs cz_dpm_ip_funcs = {
+ .name = "cz_dpm",
.early_init = cz_dpm_early_init,
.late_init = cz_dpm_late_init,
.sw_init = cz_dpm_sw_init,
@@ -2241,7 +2242,6 @@
.is_idle = NULL,
.wait_for_idle = NULL,
.soft_reset = NULL,
- .print_status = NULL,
.set_clockgating_state = cz_dpm_set_clockgating_state,
.set_powergating_state = cz_dpm_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
index c79638f..863cb16 100644
--- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
@@ -222,7 +222,7 @@
/* wptr/rptr are in bytes! */
u32 ring_index = adev->irq.ih.rptr >> 2;
uint32_t dw[4];
-
+
dw[0] = le32_to_cpu(adev->irq.ih.ring[ring_index + 0]);
dw[1] = le32_to_cpu(adev->irq.ih.ring[ring_index + 1]);
dw[2] = le32_to_cpu(adev->irq.ih.ring[ring_index + 2]);
@@ -351,35 +351,6 @@
return -ETIMEDOUT;
}
-static void cz_ih_print_status(void *handle)
-{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- dev_info(adev->dev, "CZ IH registers\n");
- dev_info(adev->dev, " SRBM_STATUS=0x%08X\n",
- RREG32(mmSRBM_STATUS));
- dev_info(adev->dev, " SRBM_STATUS2=0x%08X\n",
- RREG32(mmSRBM_STATUS2));
- dev_info(adev->dev, " INTERRUPT_CNTL=0x%08X\n",
- RREG32(mmINTERRUPT_CNTL));
- dev_info(adev->dev, " INTERRUPT_CNTL2=0x%08X\n",
- RREG32(mmINTERRUPT_CNTL2));
- dev_info(adev->dev, " IH_CNTL=0x%08X\n",
- RREG32(mmIH_CNTL));
- dev_info(adev->dev, " IH_RB_CNTL=0x%08X\n",
- RREG32(mmIH_RB_CNTL));
- dev_info(adev->dev, " IH_RB_BASE=0x%08X\n",
- RREG32(mmIH_RB_BASE));
- dev_info(adev->dev, " IH_RB_WPTR_ADDR_LO=0x%08X\n",
- RREG32(mmIH_RB_WPTR_ADDR_LO));
- dev_info(adev->dev, " IH_RB_WPTR_ADDR_HI=0x%08X\n",
- RREG32(mmIH_RB_WPTR_ADDR_HI));
- dev_info(adev->dev, " IH_RB_RPTR=0x%08X\n",
- RREG32(mmIH_RB_RPTR));
- dev_info(adev->dev, " IH_RB_WPTR=0x%08X\n",
- RREG32(mmIH_RB_WPTR));
-}
-
static int cz_ih_soft_reset(void *handle)
{
u32 srbm_soft_reset = 0;
@@ -391,8 +362,6 @@
SOFT_RESET_IH, 1);
if (srbm_soft_reset) {
- cz_ih_print_status((void *)adev);
-
tmp = RREG32(mmSRBM_SOFT_RESET);
tmp |= srbm_soft_reset;
dev_info(adev->dev, "SRBM_SOFT_RESET=0x%08X\n", tmp);
@@ -407,8 +376,6 @@
/* Wait a little for things to settle down */
udelay(50);
-
- cz_ih_print_status((void *)adev);
}
return 0;
@@ -429,6 +396,7 @@
}
const struct amd_ip_funcs cz_ih_ip_funcs = {
+ .name = "cz_ih",
.early_init = cz_ih_early_init,
.late_init = NULL,
.sw_init = cz_ih_sw_init,
@@ -440,7 +408,6 @@
.is_idle = cz_ih_is_idle,
.wait_for_idle = cz_ih_wait_for_idle,
.soft_reset = cz_ih_soft_reset,
- .print_status = cz_ih_print_status,
.set_clockgating_state = cz_ih_set_clockgating_state,
.set_powergating_state = cz_ih_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/cz_smumgr.h b/drivers/gpu/drm/amd/amdgpu/cz_smumgr.h
index 924d355..026342f 100644
--- a/drivers/gpu/drm/amd/amdgpu/cz_smumgr.h
+++ b/drivers/gpu/drm/amd/amdgpu/cz_smumgr.h
@@ -77,7 +77,7 @@
uint8_t driver_buffer_length;
uint8_t scratch_buffer_length;
uint16_t toc_entry_used_count;
- uint16_t toc_entry_initialize_index;
+ uint16_t toc_entry_initialize_index;
uint16_t toc_entry_power_profiling_index;
uint16_t toc_entry_aram;
uint16_t toc_entry_ih_register_restore_task_index;
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
index 6de2ce53..8227344 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
@@ -284,10 +284,16 @@
* surface base address.
*/
static void dce_v10_0_page_flip(struct amdgpu_device *adev,
- int crtc_id, u64 crtc_base)
+ int crtc_id, u64 crtc_base, bool async)
{
struct amdgpu_crtc *amdgpu_crtc = adev->mode_info.crtcs[crtc_id];
+ u32 tmp;
+ /* flip at hsync for async, default is vsync */
+ tmp = RREG32(mmGRPH_FLIP_CONTROL + amdgpu_crtc->crtc_offset);
+ tmp = REG_SET_FIELD(tmp, GRPH_FLIP_CONTROL,
+ GRPH_SURFACE_UPDATE_H_RETRACE_EN, async ? 1 : 0);
+ WREG32(mmGRPH_FLIP_CONTROL + amdgpu_crtc->crtc_offset, tmp);
/* update the primary scanout address */
WREG32(mmGRPH_PRIMARY_SURFACE_ADDRESS_HIGH + amdgpu_crtc->crtc_offset,
upper_32_bits(crtc_base));
@@ -2211,6 +2217,14 @@
dce_v10_0_vga_enable(crtc, false);
+ /* Make sure surface address is updated at vertical blank rather than
+ * horizontal blank
+ */
+ tmp = RREG32(mmGRPH_FLIP_CONTROL + amdgpu_crtc->crtc_offset);
+ tmp = REG_SET_FIELD(tmp, GRPH_FLIP_CONTROL,
+ GRPH_SURFACE_UPDATE_H_RETRACE_EN, 0);
+ WREG32(mmGRPH_FLIP_CONTROL + amdgpu_crtc->crtc_offset, tmp);
+
WREG32(mmGRPH_PRIMARY_SURFACE_ADDRESS_HIGH + amdgpu_crtc->crtc_offset,
upper_32_bits(fb_location));
WREG32(mmGRPH_SECONDARY_SURFACE_ADDRESS_HIGH + amdgpu_crtc->crtc_offset,
@@ -2261,13 +2275,6 @@
WREG32(mmVIEWPORT_SIZE + amdgpu_crtc->crtc_offset,
(viewport_w << 16) | viewport_h);
- /* pageflip setup */
- /* make sure flip is at vb rather than hb */
- tmp = RREG32(mmGRPH_FLIP_CONTROL + amdgpu_crtc->crtc_offset);
- tmp = REG_SET_FIELD(tmp, GRPH_FLIP_CONTROL,
- GRPH_SURFACE_UPDATE_H_RETRACE_EN, 0);
- WREG32(mmGRPH_FLIP_CONTROL + amdgpu_crtc->crtc_offset, tmp);
-
/* set pageflip to happen only at start of vblank interval (front porch) */
WREG32(mmMASTER_UPDATE_MODE + amdgpu_crtc->crtc_offset, 3);
@@ -2587,7 +2594,7 @@
return -EINVAL;
}
- obj = drm_gem_object_lookup(crtc->dev, file_priv, handle);
+ obj = drm_gem_object_lookup(file_priv, handle);
if (!obj) {
DRM_ERROR("Cannot find cursor object %x for crtc %d\n", handle, amdgpu_crtc->crtc_id);
return -ENOENT;
@@ -2992,6 +2999,8 @@
adev->ddev->mode_config.funcs = &amdgpu_mode_funcs;
+ adev->ddev->mode_config.async_page_flip = true;
+
adev->ddev->mode_config.max_width = 16384;
adev->ddev->mode_config.max_height = 16384;
@@ -3130,14 +3139,6 @@
return 0;
}
-static void dce_v10_0_print_status(void *handle)
-{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- dev_info(adev->dev, "DCE 10.x registers\n");
- /* XXX todo */
-}
-
static int dce_v10_0_soft_reset(void *handle)
{
u32 srbm_soft_reset = 0, tmp;
@@ -3147,8 +3148,6 @@
srbm_soft_reset |= SRBM_SOFT_RESET__SOFT_RESET_DC_MASK;
if (srbm_soft_reset) {
- dce_v10_0_print_status((void *)adev);
-
tmp = RREG32(mmSRBM_SOFT_RESET);
tmp |= srbm_soft_reset;
dev_info(adev->dev, "SRBM_SOFT_RESET=0x%08X\n", tmp);
@@ -3163,7 +3162,6 @@
/* Wait a little for things to settle down */
udelay(50);
- dce_v10_0_print_status((void *)adev);
}
return 0;
}
@@ -3370,7 +3368,7 @@
/* wakeup usersapce */
if (works->event)
- drm_send_vblank_event(adev->ddev, crtc_id, works->event);
+ drm_crtc_send_vblank_event(&amdgpu_crtc->base, works->event);
spin_unlock_irqrestore(&adev->ddev->event_lock, flags);
@@ -3501,6 +3499,7 @@
}
const struct amd_ip_funcs dce_v10_0_ip_funcs = {
+ .name = "dce_v10_0",
.early_init = dce_v10_0_early_init,
.late_init = NULL,
.sw_init = dce_v10_0_sw_init,
@@ -3512,7 +3511,6 @@
.is_idle = dce_v10_0_is_idle,
.wait_for_idle = dce_v10_0_wait_for_idle,
.soft_reset = dce_v10_0_soft_reset,
- .print_status = dce_v10_0_print_status,
.set_clockgating_state = dce_v10_0_set_clockgating_state,
.set_powergating_state = dce_v10_0_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v11_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v11_0.c
index e9ccc6b..c11b600 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v11_0.c
@@ -132,6 +132,22 @@
mmFBC_MISC, 0x1f311fff, 0x14302000,
};
+static const u32 polaris11_golden_settings_a11[] =
+{
+ mmDCI_CLK_CNTL, 0x00000080, 0x00000000,
+ mmFBC_DEBUG_COMP, 0x000000f0, 0x00000070,
+ mmFBC_DEBUG1, 0xffffffff, 0x00000008,
+ mmFBC_MISC, 0x9f313fff, 0x14300008,
+ mmHDMI_CONTROL, 0x313f031f, 0x00000011,
+};
+
+static const u32 polaris10_golden_settings_a11[] =
+{
+ mmDCI_CLK_CNTL, 0x00000080, 0x00000000,
+ mmFBC_DEBUG_COMP, 0x000000f0, 0x00000070,
+ mmFBC_MISC, 0x9f313fff, 0x14300008,
+ mmHDMI_CONTROL, 0x313f031f, 0x00000011,
+};
static void dce_v11_0_init_golden_registers(struct amdgpu_device *adev)
{
@@ -149,6 +165,16 @@
stoney_golden_settings_a11,
(const u32)ARRAY_SIZE(stoney_golden_settings_a11));
break;
+ case CHIP_POLARIS11:
+ amdgpu_program_register_sequence(adev,
+ polaris11_golden_settings_a11,
+ (const u32)ARRAY_SIZE(polaris11_golden_settings_a11));
+ break;
+ case CHIP_POLARIS10:
+ amdgpu_program_register_sequence(adev,
+ polaris10_golden_settings_a11,
+ (const u32)ARRAY_SIZE(polaris10_golden_settings_a11));
+ break;
default:
break;
}
@@ -276,10 +302,17 @@
* surface base address.
*/
static void dce_v11_0_page_flip(struct amdgpu_device *adev,
- int crtc_id, u64 crtc_base)
+ int crtc_id, u64 crtc_base, bool async)
{
struct amdgpu_crtc *amdgpu_crtc = adev->mode_info.crtcs[crtc_id];
+ u32 tmp;
+ /* flip at hsync for async, default is vsync */
+ /* use UPDATE_IMMEDIATE_EN instead for async? */
+ tmp = RREG32(mmGRPH_FLIP_CONTROL + amdgpu_crtc->crtc_offset);
+ tmp = REG_SET_FIELD(tmp, GRPH_FLIP_CONTROL,
+ GRPH_SURFACE_UPDATE_H_RETRACE_EN, async ? 1 : 0);
+ WREG32(mmGRPH_FLIP_CONTROL + amdgpu_crtc->crtc_offset, tmp);
/* update the scanout addresses */
WREG32(mmGRPH_PRIMARY_SURFACE_ADDRESS_HIGH + amdgpu_crtc->crtc_offset,
upper_32_bits(crtc_base));
@@ -565,35 +598,14 @@
crtc_enabled = REG_GET_FIELD(RREG32(mmCRTC_CONTROL + crtc_offsets[i]),
CRTC_CONTROL, CRTC_MASTER_EN);
if (crtc_enabled) {
-#if 0
- u32 frame_count;
- int j;
-
+#if 1
save->crtc_enabled[i] = true;
tmp = RREG32(mmCRTC_BLANK_CONTROL + crtc_offsets[i]);
if (REG_GET_FIELD(tmp, CRTC_BLANK_CONTROL, CRTC_BLANK_DATA_EN) == 0) {
- amdgpu_display_vblank_wait(adev, i);
- WREG32(mmCRTC_UPDATE_LOCK + crtc_offsets[i], 1);
+ /*it is correct only for RGB ; black is 0*/
+ WREG32(mmCRTC_BLANK_DATA_COLOR + crtc_offsets[i], 0);
tmp = REG_SET_FIELD(tmp, CRTC_BLANK_CONTROL, CRTC_BLANK_DATA_EN, 1);
WREG32(mmCRTC_BLANK_CONTROL + crtc_offsets[i], tmp);
- WREG32(mmCRTC_UPDATE_LOCK + crtc_offsets[i], 0);
- }
- /* wait for the next frame */
- frame_count = amdgpu_display_vblank_get_counter(adev, i);
- for (j = 0; j < adev->usec_timeout; j++) {
- if (amdgpu_display_vblank_get_counter(adev, i) != frame_count)
- break;
- udelay(1);
- }
- tmp = RREG32(mmGRPH_UPDATE + crtc_offsets[i]);
- if (REG_GET_FIELD(tmp, GRPH_UPDATE, GRPH_UPDATE_LOCK) == 0) {
- tmp = REG_SET_FIELD(tmp, GRPH_UPDATE, GRPH_UPDATE_LOCK, 1);
- WREG32(mmGRPH_UPDATE + crtc_offsets[i], tmp);
- }
- tmp = RREG32(mmCRTC_MASTER_UPDATE_LOCK + crtc_offsets[i]);
- if (REG_GET_FIELD(tmp, CRTC_MASTER_UPDATE_LOCK, MASTER_UPDATE_LOCK) == 0) {
- tmp = REG_SET_FIELD(tmp, CRTC_MASTER_UPDATE_LOCK, MASTER_UPDATE_LOCK, 1);
- WREG32(mmCRTC_MASTER_UPDATE_LOCK + crtc_offsets[i], tmp);
}
#else
/* XXX this is a hack to avoid strange behavior with EFI on certain systems */
@@ -614,54 +626,20 @@
static void dce_v11_0_resume_mc_access(struct amdgpu_device *adev,
struct amdgpu_mode_mc_save *save)
{
- u32 tmp, frame_count;
- int i, j;
+ u32 tmp;
+ int i;
/* update crtc base addresses */
for (i = 0; i < adev->mode_info.num_crtc; i++) {
WREG32(mmGRPH_PRIMARY_SURFACE_ADDRESS_HIGH + crtc_offsets[i],
upper_32_bits(adev->mc.vram_start));
- WREG32(mmGRPH_SECONDARY_SURFACE_ADDRESS_HIGH + crtc_offsets[i],
- upper_32_bits(adev->mc.vram_start));
WREG32(mmGRPH_PRIMARY_SURFACE_ADDRESS + crtc_offsets[i],
(u32)adev->mc.vram_start);
- WREG32(mmGRPH_SECONDARY_SURFACE_ADDRESS + crtc_offsets[i],
- (u32)adev->mc.vram_start);
if (save->crtc_enabled[i]) {
- tmp = RREG32(mmCRTC_MASTER_UPDATE_MODE + crtc_offsets[i]);
- if (REG_GET_FIELD(tmp, CRTC_MASTER_UPDATE_MODE, MASTER_UPDATE_MODE) != 3) {
- tmp = REG_SET_FIELD(tmp, CRTC_MASTER_UPDATE_MODE, MASTER_UPDATE_MODE, 3);
- WREG32(mmCRTC_MASTER_UPDATE_MODE + crtc_offsets[i], tmp);
- }
- tmp = RREG32(mmGRPH_UPDATE + crtc_offsets[i]);
- if (REG_GET_FIELD(tmp, GRPH_UPDATE, GRPH_UPDATE_LOCK)) {
- tmp = REG_SET_FIELD(tmp, GRPH_UPDATE, GRPH_UPDATE_LOCK, 0);
- WREG32(mmGRPH_UPDATE + crtc_offsets[i], tmp);
- }
- tmp = RREG32(mmCRTC_MASTER_UPDATE_LOCK + crtc_offsets[i]);
- if (REG_GET_FIELD(tmp, CRTC_MASTER_UPDATE_LOCK, MASTER_UPDATE_LOCK)) {
- tmp = REG_SET_FIELD(tmp, CRTC_MASTER_UPDATE_LOCK, MASTER_UPDATE_LOCK, 0);
- WREG32(mmCRTC_MASTER_UPDATE_LOCK + crtc_offsets[i], tmp);
- }
- for (j = 0; j < adev->usec_timeout; j++) {
- tmp = RREG32(mmGRPH_UPDATE + crtc_offsets[i]);
- if (REG_GET_FIELD(tmp, GRPH_UPDATE, GRPH_SURFACE_UPDATE_PENDING) == 0)
- break;
- udelay(1);
- }
tmp = RREG32(mmCRTC_BLANK_CONTROL + crtc_offsets[i]);
tmp = REG_SET_FIELD(tmp, CRTC_BLANK_CONTROL, CRTC_BLANK_DATA_EN, 0);
- WREG32(mmCRTC_UPDATE_LOCK + crtc_offsets[i], 1);
WREG32(mmCRTC_BLANK_CONTROL + crtc_offsets[i], tmp);
- WREG32(mmCRTC_UPDATE_LOCK + crtc_offsets[i], 0);
- /* wait for the next frame */
- frame_count = amdgpu_display_vblank_get_counter(adev, i);
- for (j = 0; j < adev->usec_timeout; j++) {
- if (amdgpu_display_vblank_get_counter(adev, i) != frame_count)
- break;
- udelay(1);
- }
}
}
@@ -1624,6 +1602,7 @@
AUD4_REGISTER_OFFSET,
AUD5_REGISTER_OFFSET,
AUD6_REGISTER_OFFSET,
+ AUD7_REGISTER_OFFSET,
};
static int dce_v11_0_audio_init(struct amdgpu_device *adev)
@@ -1635,7 +1614,20 @@
adev->mode_info.audio.enabled = true;
- adev->mode_info.audio.num_pins = 7;
+ switch (adev->asic_type) {
+ case CHIP_CARRIZO:
+ case CHIP_STONEY:
+ adev->mode_info.audio.num_pins = 7;
+ break;
+ case CHIP_POLARIS10:
+ adev->mode_info.audio.num_pins = 8;
+ break;
+ case CHIP_POLARIS11:
+ adev->mode_info.audio.num_pins = 6;
+ break;
+ default:
+ return -EINVAL;
+ }
for (i = 0; i < adev->mode_info.audio.num_pins; i++) {
adev->mode_info.audio.pin[i].channels = -1;
@@ -2201,6 +2193,14 @@
dce_v11_0_vga_enable(crtc, false);
+ /* Make sure surface address is updated at vertical blank rather than
+ * horizontal blank
+ */
+ tmp = RREG32(mmGRPH_FLIP_CONTROL + amdgpu_crtc->crtc_offset);
+ tmp = REG_SET_FIELD(tmp, GRPH_FLIP_CONTROL,
+ GRPH_SURFACE_UPDATE_H_RETRACE_EN, 0);
+ WREG32(mmGRPH_FLIP_CONTROL + amdgpu_crtc->crtc_offset, tmp);
+
WREG32(mmGRPH_PRIMARY_SURFACE_ADDRESS_HIGH + amdgpu_crtc->crtc_offset,
upper_32_bits(fb_location));
WREG32(mmGRPH_SECONDARY_SURFACE_ADDRESS_HIGH + amdgpu_crtc->crtc_offset,
@@ -2251,13 +2251,6 @@
WREG32(mmVIEWPORT_SIZE + amdgpu_crtc->crtc_offset,
(viewport_w << 16) | viewport_h);
- /* pageflip setup */
- /* make sure flip is at vb rather than hb */
- tmp = RREG32(mmGRPH_FLIP_CONTROL + amdgpu_crtc->crtc_offset);
- tmp = REG_SET_FIELD(tmp, GRPH_FLIP_CONTROL,
- GRPH_SURFACE_UPDATE_H_RETRACE_EN, 0);
- WREG32(mmGRPH_FLIP_CONTROL + amdgpu_crtc->crtc_offset, tmp);
-
/* set pageflip to happen only at start of vblank interval (front porch) */
WREG32(mmCRTC_MASTER_UPDATE_MODE + amdgpu_crtc->crtc_offset, 3);
@@ -2427,6 +2420,40 @@
u32 pll_in_use;
int pll;
+ if ((adev->asic_type == CHIP_POLARIS10) ||
+ (adev->asic_type == CHIP_POLARIS11)) {
+ struct amdgpu_encoder *amdgpu_encoder =
+ to_amdgpu_encoder(amdgpu_crtc->encoder);
+ struct amdgpu_encoder_atom_dig *dig = amdgpu_encoder->enc_priv;
+
+ if (ENCODER_MODE_IS_DP(amdgpu_atombios_encoder_get_encoder_mode(amdgpu_crtc->encoder)))
+ return ATOM_DP_DTO;
+
+ switch (amdgpu_encoder->encoder_id) {
+ case ENCODER_OBJECT_ID_INTERNAL_UNIPHY:
+ if (dig->linkb)
+ return ATOM_COMBOPHY_PLL1;
+ else
+ return ATOM_COMBOPHY_PLL0;
+ break;
+ case ENCODER_OBJECT_ID_INTERNAL_UNIPHY1:
+ if (dig->linkb)
+ return ATOM_COMBOPHY_PLL3;
+ else
+ return ATOM_COMBOPHY_PLL2;
+ break;
+ case ENCODER_OBJECT_ID_INTERNAL_UNIPHY2:
+ if (dig->linkb)
+ return ATOM_COMBOPHY_PLL5;
+ else
+ return ATOM_COMBOPHY_PLL4;
+ break;
+ default:
+ DRM_ERROR("invalid encoder_id: 0x%x\n", amdgpu_encoder->encoder_id);
+ return ATOM_PPLL_INVALID;
+ }
+ }
+
if (ENCODER_MODE_IS_DP(amdgpu_atombios_encoder_get_encoder_mode(amdgpu_crtc->encoder))) {
if (adev->clock.dp_extclk)
/* skip PPLL programming if using ext clock */
@@ -2578,7 +2605,7 @@
return -EINVAL;
}
- obj = drm_gem_object_lookup(crtc->dev, file_priv, handle);
+ obj = drm_gem_object_lookup(file_priv, handle);
if (!obj) {
DRM_ERROR("Cannot find cursor object %x for crtc %d\n", handle, amdgpu_crtc->crtc_id);
return -ENOENT;
@@ -2782,7 +2809,17 @@
case ATOM_PPLL2:
/* disable the ppll */
amdgpu_atombios_crtc_program_pll(crtc, amdgpu_crtc->crtc_id, amdgpu_crtc->pll_id,
- 0, 0, ATOM_DISABLE, 0, 0, 0, 0, 0, false, &ss);
+ 0, 0, ATOM_DISABLE, 0, 0, 0, 0, 0, false, &ss);
+ break;
+ case ATOM_COMBOPHY_PLL0:
+ case ATOM_COMBOPHY_PLL1:
+ case ATOM_COMBOPHY_PLL2:
+ case ATOM_COMBOPHY_PLL3:
+ case ATOM_COMBOPHY_PLL4:
+ case ATOM_COMBOPHY_PLL5:
+ /* disable the ppll */
+ amdgpu_atombios_crtc_program_pll(crtc, ATOM_CRTC_INVALID, amdgpu_crtc->pll_id,
+ 0, 0, ATOM_DISABLE, 0, 0, 0, 0, 0, false, &ss);
break;
default:
break;
@@ -2800,11 +2837,28 @@
int x, int y, struct drm_framebuffer *old_fb)
{
struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
+ struct drm_device *dev = crtc->dev;
+ struct amdgpu_device *adev = dev->dev_private;
if (!amdgpu_crtc->adjusted_clock)
return -EINVAL;
- amdgpu_atombios_crtc_set_pll(crtc, adjusted_mode);
+ if ((adev->asic_type == CHIP_POLARIS10) ||
+ (adev->asic_type == CHIP_POLARIS11)) {
+ struct amdgpu_encoder *amdgpu_encoder =
+ to_amdgpu_encoder(amdgpu_crtc->encoder);
+ int encoder_mode =
+ amdgpu_atombios_encoder_get_encoder_mode(amdgpu_crtc->encoder);
+
+ /* SetPixelClock calculates the plls and ss values now */
+ amdgpu_atombios_crtc_program_pll(crtc, amdgpu_crtc->crtc_id,
+ amdgpu_crtc->pll_id,
+ encoder_mode, amdgpu_encoder->encoder_id,
+ adjusted_mode->clock, 0, 0, 0, 0,
+ amdgpu_crtc->bpc, amdgpu_crtc->ss_enabled, &amdgpu_crtc->ss);
+ } else {
+ amdgpu_atombios_crtc_set_pll(crtc, adjusted_mode);
+ }
amdgpu_atombios_crtc_set_dtd_timing(crtc, adjusted_mode);
dce_v11_0_crtc_do_set_base(crtc, old_fb, x, y, 0);
amdgpu_atombios_crtc_overscan_setup(crtc, mode, adjusted_mode);
@@ -2955,6 +3009,16 @@
adev->mode_info.num_hpd = 6;
adev->mode_info.num_dig = 9;
break;
+ case CHIP_POLARIS10:
+ adev->mode_info.num_crtc = 6;
+ adev->mode_info.num_hpd = 6;
+ adev->mode_info.num_dig = 6;
+ break;
+ case CHIP_POLARIS11:
+ adev->mode_info.num_crtc = 5;
+ adev->mode_info.num_hpd = 5;
+ adev->mode_info.num_dig = 5;
+ break;
default:
/* FIXME: not supported yet */
return -EINVAL;
@@ -2987,6 +3051,8 @@
adev->ddev->mode_config.funcs = &amdgpu_mode_funcs;
+ adev->ddev->mode_config.async_page_flip = true;
+
adev->ddev->mode_config.max_width = 16384;
adev->ddev->mode_config.max_height = 16384;
@@ -3057,7 +3123,15 @@
/* init dig PHYs, disp eng pll */
amdgpu_atombios_crtc_powergate_init(adev);
amdgpu_atombios_encoder_init_dig(adev);
- amdgpu_atombios_crtc_set_disp_eng_pll(adev, adev->clock.default_dispclk);
+ if ((adev->asic_type == CHIP_POLARIS10) ||
+ (adev->asic_type == CHIP_POLARIS11)) {
+ amdgpu_atombios_crtc_set_dce_clock(adev, adev->clock.default_dispclk,
+ DCE_CLOCK_TYPE_DISPCLK, ATOM_GCK_DFS);
+ amdgpu_atombios_crtc_set_dce_clock(adev, 0,
+ DCE_CLOCK_TYPE_DPREFCLK, ATOM_GCK_DFS);
+ } else {
+ amdgpu_atombios_crtc_set_disp_eng_pll(adev, adev->clock.default_dispclk);
+ }
/* initialize hpd */
dce_v11_0_hpd_init(adev);
@@ -3126,14 +3200,6 @@
return 0;
}
-static void dce_v11_0_print_status(void *handle)
-{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- dev_info(adev->dev, "DCE 10.x registers\n");
- /* XXX todo */
-}
-
static int dce_v11_0_soft_reset(void *handle)
{
u32 srbm_soft_reset = 0, tmp;
@@ -3143,8 +3209,6 @@
srbm_soft_reset |= SRBM_SOFT_RESET__SOFT_RESET_DC_MASK;
if (srbm_soft_reset) {
- dce_v11_0_print_status((void *)adev);
-
tmp = RREG32(mmSRBM_SOFT_RESET);
tmp |= srbm_soft_reset;
dev_info(adev->dev, "SRBM_SOFT_RESET=0x%08X\n", tmp);
@@ -3159,7 +3223,6 @@
/* Wait a little for things to settle down */
udelay(50);
- dce_v11_0_print_status((void *)adev);
}
return 0;
}
@@ -3366,7 +3429,7 @@
/* wakeup usersapce */
if(works->event)
- drm_send_vblank_event(adev->ddev, crtc_id, works->event);
+ drm_crtc_send_vblank_event(&amdgpu_crtc->base, works->event);
spin_unlock_irqrestore(&adev->ddev->event_lock, flags);
@@ -3497,6 +3560,7 @@
}
const struct amd_ip_funcs dce_v11_0_ip_funcs = {
+ .name = "dce_v11_0",
.early_init = dce_v11_0_early_init,
.late_init = NULL,
.sw_init = dce_v11_0_sw_init,
@@ -3508,7 +3572,6 @@
.is_idle = dce_v11_0_is_idle,
.wait_for_idle = dce_v11_0_wait_for_idle,
.soft_reset = dce_v11_0_soft_reset,
- .print_status = dce_v11_0_print_status,
.set_clockgating_state = dce_v11_0_set_clockgating_state,
.set_powergating_state = dce_v11_0_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
index e56b55d..3fb65e4 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
@@ -233,10 +233,13 @@
* surface base address.
*/
static void dce_v8_0_page_flip(struct amdgpu_device *adev,
- int crtc_id, u64 crtc_base)
+ int crtc_id, u64 crtc_base, bool async)
{
struct amdgpu_crtc *amdgpu_crtc = adev->mode_info.crtcs[crtc_id];
+ /* flip at hsync for async, default is vsync */
+ WREG32(mmGRPH_FLIP_CONTROL + amdgpu_crtc->crtc_offset, async ?
+ GRPH_FLIP_CONTROL__GRPH_SURFACE_UPDATE_H_RETRACE_EN_MASK : 0);
/* update the primary scanout addresses */
WREG32(mmGRPH_PRIMARY_SURFACE_ADDRESS_HIGH + amdgpu_crtc->crtc_offset,
upper_32_bits(crtc_base));
@@ -1999,7 +2002,7 @@
uint32_t fb_format, fb_pitch_pixels;
u32 fb_swap = (GRPH_ENDIAN_NONE << GRPH_SWAP_CNTL__GRPH_ENDIAN_SWAP__SHIFT);
u32 pipe_config;
- u32 tmp, viewport_w, viewport_h;
+ u32 viewport_w, viewport_h;
int r;
bool bypass_lut = false;
@@ -2135,6 +2138,11 @@
dce_v8_0_vga_enable(crtc, false);
+ /* Make sure surface address is updated at vertical blank rather than
+ * horizontal blank
+ */
+ WREG32(mmGRPH_FLIP_CONTROL + amdgpu_crtc->crtc_offset, 0);
+
WREG32(mmGRPH_PRIMARY_SURFACE_ADDRESS_HIGH + amdgpu_crtc->crtc_offset,
upper_32_bits(fb_location));
WREG32(mmGRPH_SECONDARY_SURFACE_ADDRESS_HIGH + amdgpu_crtc->crtc_offset,
@@ -2182,12 +2190,6 @@
WREG32(mmVIEWPORT_SIZE + amdgpu_crtc->crtc_offset,
(viewport_w << 16) | viewport_h);
- /* pageflip setup */
- /* make sure flip is at vb rather than hb */
- tmp = RREG32(mmGRPH_FLIP_CONTROL + amdgpu_crtc->crtc_offset);
- tmp &= ~GRPH_FLIP_CONTROL__GRPH_SURFACE_UPDATE_H_RETRACE_EN_MASK;
- WREG32(mmGRPH_FLIP_CONTROL + amdgpu_crtc->crtc_offset, tmp);
-
/* set pageflip to happen only at start of vblank interval (front porch) */
WREG32(mmMASTER_UPDATE_MODE + amdgpu_crtc->crtc_offset, 3);
@@ -2499,7 +2501,7 @@
return -EINVAL;
}
- obj = drm_gem_object_lookup(crtc->dev, file_priv, handle);
+ obj = drm_gem_object_lookup(file_priv, handle);
if (!obj) {
DRM_ERROR("Cannot find cursor object %x for crtc %d\n", handle, amdgpu_crtc->crtc_id);
return -ENOENT;
@@ -2902,6 +2904,8 @@
adev->ddev->mode_config.funcs = &amdgpu_mode_funcs;
+ adev->ddev->mode_config.async_page_flip = true;
+
adev->ddev->mode_config.max_width = 16384;
adev->ddev->mode_config.max_height = 16384;
@@ -3038,14 +3042,6 @@
return 0;
}
-static void dce_v8_0_print_status(void *handle)
-{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- dev_info(adev->dev, "DCE 8.x registers\n");
- /* XXX todo */
-}
-
static int dce_v8_0_soft_reset(void *handle)
{
u32 srbm_soft_reset = 0, tmp;
@@ -3055,8 +3051,6 @@
srbm_soft_reset |= SRBM_SOFT_RESET__SOFT_RESET_DC_MASK;
if (srbm_soft_reset) {
- dce_v8_0_print_status((void *)adev);
-
tmp = RREG32(mmSRBM_SOFT_RESET);
tmp |= srbm_soft_reset;
dev_info(adev->dev, "SRBM_SOFT_RESET=0x%08X\n", tmp);
@@ -3071,7 +3065,6 @@
/* Wait a little for things to settle down */
udelay(50);
- dce_v8_0_print_status((void *)adev);
}
return 0;
}
@@ -3379,7 +3372,7 @@
/* wakeup usersapce */
if (works->event)
- drm_send_vblank_event(adev->ddev, crtc_id, works->event);
+ drm_crtc_send_vblank_event(&amdgpu_crtc->base, works->event);
spin_unlock_irqrestore(&adev->ddev->event_lock, flags);
@@ -3431,6 +3424,7 @@
}
const struct amd_ip_funcs dce_v8_0_ip_funcs = {
+ .name = "dce_v8_0",
.early_init = dce_v8_0_early_init,
.late_init = NULL,
.sw_init = dce_v8_0_sw_init,
@@ -3442,7 +3436,6 @@
.is_idle = dce_v8_0_is_idle,
.wait_for_idle = dce_v8_0_wait_for_idle,
.soft_reset = dce_v8_0_soft_reset,
- .print_status = dce_v8_0_print_status,
.set_clockgating_state = dce_v8_0_set_clockgating_state,
.set_powergating_state = dce_v8_0_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/fiji_dpm.c b/drivers/gpu/drm/amd/amdgpu/fiji_dpm.c
index 4b0e45a..245cabf 100644
--- a/drivers/gpu/drm/amd/amdgpu/fiji_dpm.c
+++ b/drivers/gpu/drm/amd/amdgpu/fiji_dpm.c
@@ -143,6 +143,7 @@
}
const struct amd_ip_funcs fiji_dpm_ip_funcs = {
+ .name = "fiji_dpm",
.early_init = fiji_dpm_early_init,
.late_init = NULL,
.sw_init = fiji_dpm_sw_init,
@@ -154,7 +155,6 @@
.is_idle = NULL,
.wait_for_idle = NULL,
.soft_reset = NULL,
- .print_status = NULL,
.set_clockgating_state = fiji_dpm_set_clockgating_state,
.set_powergating_state = fiji_dpm_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c
index bb8709066..7f18a53 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c
@@ -53,7 +53,6 @@
static void gfx_v7_0_set_ring_funcs(struct amdgpu_device *adev);
static void gfx_v7_0_set_irq_funcs(struct amdgpu_device *adev);
static void gfx_v7_0_set_gds_init(struct amdgpu_device *adev);
-int gfx_v7_0_get_cu_info(struct amdgpu_device *, struct amdgpu_cu_info *);
MODULE_FIRMWARE("radeon/bonaire_pfp.bin");
MODULE_FIRMWARE("radeon/bonaire_me.bin");
@@ -882,6 +881,7 @@
static void gfx_v7_0_get_csb_buffer(struct amdgpu_device *adev, volatile u32 *buffer);
static void gfx_v7_0_init_cp_pg_table(struct amdgpu_device *adev);
static void gfx_v7_0_init_pg(struct amdgpu_device *adev);
+static void gfx_v7_0_get_cu_info(struct amdgpu_device *adev);
/*
* Core functions
@@ -1718,6 +1718,7 @@
gfx_v7_0_tiling_mode_table_init(adev);
gfx_v7_0_setup_rb(adev);
+ gfx_v7_0_get_cu_info(adev);
/* set HW defaults for 3D engine */
WREG32(mmCP_MEQ_THRESHOLDS,
@@ -2029,17 +2030,13 @@
* on the gfx ring for execution by the GPU.
*/
static void gfx_v7_0_ring_emit_ib_gfx(struct amdgpu_ring *ring,
- struct amdgpu_ib *ib)
+ struct amdgpu_ib *ib,
+ unsigned vm_id, bool ctx_switch)
{
- bool need_ctx_switch = ring->current_ctx != ib->ctx;
u32 header, control = 0;
u32 next_rptr = ring->wptr + 5;
- /* drop the CE preamble IB for the same context */
- if ((ib->flags & AMDGPU_IB_FLAG_PREAMBLE) && !need_ctx_switch)
- return;
-
- if (need_ctx_switch)
+ if (ctx_switch)
next_rptr += 2;
next_rptr += 4;
@@ -2050,7 +2047,7 @@
amdgpu_ring_write(ring, next_rptr);
/* insert SWITCH_BUFFER packet before first IB in the ring frame */
- if (need_ctx_switch) {
+ if (ctx_switch) {
amdgpu_ring_write(ring, PACKET3(PACKET3_SWITCH_BUFFER, 0));
amdgpu_ring_write(ring, 0);
}
@@ -2060,7 +2057,7 @@
else
header = PACKET3(PACKET3_INDIRECT_BUFFER, 2);
- control |= ib->length_dw | (ib->vm_id << 24);
+ control |= ib->length_dw | (vm_id << 24);
amdgpu_ring_write(ring, header);
amdgpu_ring_write(ring,
@@ -2073,7 +2070,8 @@
}
static void gfx_v7_0_ring_emit_ib_compute(struct amdgpu_ring *ring,
- struct amdgpu_ib *ib)
+ struct amdgpu_ib *ib,
+ unsigned vm_id, bool ctx_switch)
{
u32 header, control = 0;
u32 next_rptr = ring->wptr + 5;
@@ -2088,7 +2086,7 @@
header = PACKET3(PACKET3_INDIRECT_BUFFER, 2);
- control |= ib->length_dw | (ib->vm_id << 24);
+ control |= ib->length_dw | (vm_id << 24);
amdgpu_ring_write(ring, header);
amdgpu_ring_write(ring,
@@ -2136,7 +2134,7 @@
ib.ptr[2] = 0xDEADBEEF;
ib.length_dw = 3;
- r = amdgpu_ib_schedule(ring, 1, &ib, NULL, &f);
+ r = amdgpu_ib_schedule(ring, 1, &ib, NULL, NULL, &f);
if (r)
goto err2;
@@ -3053,6 +3051,19 @@
static void gfx_v7_0_ring_emit_pipeline_sync(struct amdgpu_ring *ring)
{
int usepfp = (ring->type == AMDGPU_RING_TYPE_GFX);
+ uint32_t seq = ring->fence_drv.sync_seq;
+ uint64_t addr = ring->fence_drv.gpu_addr;
+
+ amdgpu_ring_write(ring, PACKET3(PACKET3_WAIT_REG_MEM, 5));
+ amdgpu_ring_write(ring, (WAIT_REG_MEM_MEM_SPACE(1) | /* memory */
+ WAIT_REG_MEM_FUNCTION(3) | /* equal */
+ WAIT_REG_MEM_ENGINE(usepfp))); /* pfp or me */
+ amdgpu_ring_write(ring, addr & 0xfffffffc);
+ amdgpu_ring_write(ring, upper_32_bits(addr) & 0xffffffff);
+ amdgpu_ring_write(ring, seq);
+ amdgpu_ring_write(ring, 0xffffffff);
+ amdgpu_ring_write(ring, 4); /* poll interval */
+
if (usepfp) {
/* synce CE with ME to prevent CE fetch CEIB before context switch done */
amdgpu_ring_write(ring, PACKET3(PACKET3_SWITCH_BUFFER, 0));
@@ -3080,18 +3091,6 @@
unsigned vm_id, uint64_t pd_addr)
{
int usepfp = (ring->type == AMDGPU_RING_TYPE_GFX);
- uint32_t seq = ring->fence_drv.sync_seq;
- uint64_t addr = ring->fence_drv.gpu_addr;
-
- amdgpu_ring_write(ring, PACKET3(PACKET3_WAIT_REG_MEM, 5));
- amdgpu_ring_write(ring, (WAIT_REG_MEM_MEM_SPACE(1) | /* memory */
- WAIT_REG_MEM_FUNCTION(3) | /* equal */
- WAIT_REG_MEM_ENGINE(usepfp))); /* pfp or me */
- amdgpu_ring_write(ring, addr & 0xfffffffc);
- amdgpu_ring_write(ring, upper_32_bits(addr) & 0xffffffff);
- amdgpu_ring_write(ring, seq);
- amdgpu_ring_write(ring, 0xffffffff);
- amdgpu_ring_write(ring, 4); /* poll interval */
amdgpu_ring_write(ring, PACKET3(PACKET3_WRITE_DATA, 3));
amdgpu_ring_write(ring, (WRITE_DATA_ENGINE_SEL(usepfp) |
@@ -3869,18 +3868,13 @@
static void gfx_v7_0_init_ao_cu_mask(struct amdgpu_device *adev)
{
- uint32_t tmp, active_cu_number;
- struct amdgpu_cu_info cu_info;
+ u32 tmp;
- gfx_v7_0_get_cu_info(adev, &cu_info);
- tmp = cu_info.ao_cu_mask;
- active_cu_number = cu_info.number;
-
- WREG32(mmRLC_PG_ALWAYS_ON_CU_MASK, tmp);
+ WREG32(mmRLC_PG_ALWAYS_ON_CU_MASK, adev->gfx.cu_info.ao_cu_mask);
tmp = RREG32(mmRLC_MAX_PG_CU);
tmp &= ~RLC_MAX_PG_CU__MAX_POWERED_UP_CU_MASK;
- tmp |= (active_cu_number << RLC_MAX_PG_CU__MAX_POWERED_UP_CU__SHIFT);
+ tmp |= (adev->gfx.cu_info.number << RLC_MAX_PG_CU__MAX_POWERED_UP_CU__SHIFT);
WREG32(mmRLC_MAX_PG_CU, tmp);
}
@@ -4414,7 +4408,7 @@
ring = &adev->gfx.gfx_ring[i];
ring->ring_obj = NULL;
sprintf(ring->name, "gfx");
- r = amdgpu_ring_init(adev, ring, 1024 * 1024,
+ r = amdgpu_ring_init(adev, ring, 1024,
PACKET3(PACKET3_NOP, 0x3FFF), 0xf,
&adev->gfx.eop_irq, AMDGPU_CP_IRQ_GFX_EOP,
AMDGPU_RING_TYPE_GFX);
@@ -4438,10 +4432,10 @@
ring->me = 1; /* first MEC */
ring->pipe = i / 8;
ring->queue = i % 8;
- sprintf(ring->name, "comp %d.%d.%d", ring->me, ring->pipe, ring->queue);
+ sprintf(ring->name, "comp_%d.%d.%d", ring->me, ring->pipe, ring->queue);
irq_type = AMDGPU_CP_IRQ_COMPUTE_MEC1_PIPE0_EOP + ring->pipe;
/* type-2 packets are deprecated on MEC, use type-3 instead */
- r = amdgpu_ring_init(adev, ring, 1024 * 1024,
+ r = amdgpu_ring_init(adev, ring, 1024,
PACKET3(PACKET3_NOP, 0x3FFF), 0xf,
&adev->gfx.eop_irq, irq_type,
AMDGPU_RING_TYPE_COMPUTE);
@@ -4572,256 +4566,6 @@
return -ETIMEDOUT;
}
-static void gfx_v7_0_print_status(void *handle)
-{
- int i;
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- dev_info(adev->dev, "GFX 7.x registers\n");
- dev_info(adev->dev, " GRBM_STATUS=0x%08X\n",
- RREG32(mmGRBM_STATUS));
- dev_info(adev->dev, " GRBM_STATUS2=0x%08X\n",
- RREG32(mmGRBM_STATUS2));
- dev_info(adev->dev, " GRBM_STATUS_SE0=0x%08X\n",
- RREG32(mmGRBM_STATUS_SE0));
- dev_info(adev->dev, " GRBM_STATUS_SE1=0x%08X\n",
- RREG32(mmGRBM_STATUS_SE1));
- dev_info(adev->dev, " GRBM_STATUS_SE2=0x%08X\n",
- RREG32(mmGRBM_STATUS_SE2));
- dev_info(adev->dev, " GRBM_STATUS_SE3=0x%08X\n",
- RREG32(mmGRBM_STATUS_SE3));
- dev_info(adev->dev, " CP_STAT = 0x%08x\n", RREG32(mmCP_STAT));
- dev_info(adev->dev, " CP_STALLED_STAT1 = 0x%08x\n",
- RREG32(mmCP_STALLED_STAT1));
- dev_info(adev->dev, " CP_STALLED_STAT2 = 0x%08x\n",
- RREG32(mmCP_STALLED_STAT2));
- dev_info(adev->dev, " CP_STALLED_STAT3 = 0x%08x\n",
- RREG32(mmCP_STALLED_STAT3));
- dev_info(adev->dev, " CP_CPF_BUSY_STAT = 0x%08x\n",
- RREG32(mmCP_CPF_BUSY_STAT));
- dev_info(adev->dev, " CP_CPF_STALLED_STAT1 = 0x%08x\n",
- RREG32(mmCP_CPF_STALLED_STAT1));
- dev_info(adev->dev, " CP_CPF_STATUS = 0x%08x\n", RREG32(mmCP_CPF_STATUS));
- dev_info(adev->dev, " CP_CPC_BUSY_STAT = 0x%08x\n", RREG32(mmCP_CPC_BUSY_STAT));
- dev_info(adev->dev, " CP_CPC_STALLED_STAT1 = 0x%08x\n",
- RREG32(mmCP_CPC_STALLED_STAT1));
- dev_info(adev->dev, " CP_CPC_STATUS = 0x%08x\n", RREG32(mmCP_CPC_STATUS));
-
- for (i = 0; i < 32; i++) {
- dev_info(adev->dev, " GB_TILE_MODE%d=0x%08X\n",
- i, RREG32(mmGB_TILE_MODE0 + (i * 4)));
- }
- for (i = 0; i < 16; i++) {
- dev_info(adev->dev, " GB_MACROTILE_MODE%d=0x%08X\n",
- i, RREG32(mmGB_MACROTILE_MODE0 + (i * 4)));
- }
- for (i = 0; i < adev->gfx.config.max_shader_engines; i++) {
- dev_info(adev->dev, " se: %d\n", i);
- gfx_v7_0_select_se_sh(adev, i, 0xffffffff);
- dev_info(adev->dev, " PA_SC_RASTER_CONFIG=0x%08X\n",
- RREG32(mmPA_SC_RASTER_CONFIG));
- dev_info(adev->dev, " PA_SC_RASTER_CONFIG_1=0x%08X\n",
- RREG32(mmPA_SC_RASTER_CONFIG_1));
- }
- gfx_v7_0_select_se_sh(adev, 0xffffffff, 0xffffffff);
-
- dev_info(adev->dev, " GB_ADDR_CONFIG=0x%08X\n",
- RREG32(mmGB_ADDR_CONFIG));
- dev_info(adev->dev, " HDP_ADDR_CONFIG=0x%08X\n",
- RREG32(mmHDP_ADDR_CONFIG));
- dev_info(adev->dev, " DMIF_ADDR_CALC=0x%08X\n",
- RREG32(mmDMIF_ADDR_CALC));
-
- dev_info(adev->dev, " CP_MEQ_THRESHOLDS=0x%08X\n",
- RREG32(mmCP_MEQ_THRESHOLDS));
- dev_info(adev->dev, " SX_DEBUG_1=0x%08X\n",
- RREG32(mmSX_DEBUG_1));
- dev_info(adev->dev, " TA_CNTL_AUX=0x%08X\n",
- RREG32(mmTA_CNTL_AUX));
- dev_info(adev->dev, " SPI_CONFIG_CNTL=0x%08X\n",
- RREG32(mmSPI_CONFIG_CNTL));
- dev_info(adev->dev, " SQ_CONFIG=0x%08X\n",
- RREG32(mmSQ_CONFIG));
- dev_info(adev->dev, " DB_DEBUG=0x%08X\n",
- RREG32(mmDB_DEBUG));
- dev_info(adev->dev, " DB_DEBUG2=0x%08X\n",
- RREG32(mmDB_DEBUG2));
- dev_info(adev->dev, " DB_DEBUG3=0x%08X\n",
- RREG32(mmDB_DEBUG3));
- dev_info(adev->dev, " CB_HW_CONTROL=0x%08X\n",
- RREG32(mmCB_HW_CONTROL));
- dev_info(adev->dev, " SPI_CONFIG_CNTL_1=0x%08X\n",
- RREG32(mmSPI_CONFIG_CNTL_1));
- dev_info(adev->dev, " PA_SC_FIFO_SIZE=0x%08X\n",
- RREG32(mmPA_SC_FIFO_SIZE));
- dev_info(adev->dev, " VGT_NUM_INSTANCES=0x%08X\n",
- RREG32(mmVGT_NUM_INSTANCES));
- dev_info(adev->dev, " CP_PERFMON_CNTL=0x%08X\n",
- RREG32(mmCP_PERFMON_CNTL));
- dev_info(adev->dev, " PA_SC_FORCE_EOV_MAX_CNTS=0x%08X\n",
- RREG32(mmPA_SC_FORCE_EOV_MAX_CNTS));
- dev_info(adev->dev, " VGT_CACHE_INVALIDATION=0x%08X\n",
- RREG32(mmVGT_CACHE_INVALIDATION));
- dev_info(adev->dev, " VGT_GS_VERTEX_REUSE=0x%08X\n",
- RREG32(mmVGT_GS_VERTEX_REUSE));
- dev_info(adev->dev, " PA_SC_LINE_STIPPLE_STATE=0x%08X\n",
- RREG32(mmPA_SC_LINE_STIPPLE_STATE));
- dev_info(adev->dev, " PA_CL_ENHANCE=0x%08X\n",
- RREG32(mmPA_CL_ENHANCE));
- dev_info(adev->dev, " PA_SC_ENHANCE=0x%08X\n",
- RREG32(mmPA_SC_ENHANCE));
-
- dev_info(adev->dev, " CP_ME_CNTL=0x%08X\n",
- RREG32(mmCP_ME_CNTL));
- dev_info(adev->dev, " CP_MAX_CONTEXT=0x%08X\n",
- RREG32(mmCP_MAX_CONTEXT));
- dev_info(adev->dev, " CP_ENDIAN_SWAP=0x%08X\n",
- RREG32(mmCP_ENDIAN_SWAP));
- dev_info(adev->dev, " CP_DEVICE_ID=0x%08X\n",
- RREG32(mmCP_DEVICE_ID));
-
- dev_info(adev->dev, " CP_SEM_WAIT_TIMER=0x%08X\n",
- RREG32(mmCP_SEM_WAIT_TIMER));
- if (adev->asic_type != CHIP_HAWAII)
- dev_info(adev->dev, " CP_SEM_INCOMPLETE_TIMER_CNTL=0x%08X\n",
- RREG32(mmCP_SEM_INCOMPLETE_TIMER_CNTL));
-
- dev_info(adev->dev, " CP_RB_WPTR_DELAY=0x%08X\n",
- RREG32(mmCP_RB_WPTR_DELAY));
- dev_info(adev->dev, " CP_RB_VMID=0x%08X\n",
- RREG32(mmCP_RB_VMID));
- dev_info(adev->dev, " CP_RB0_CNTL=0x%08X\n",
- RREG32(mmCP_RB0_CNTL));
- dev_info(adev->dev, " CP_RB0_WPTR=0x%08X\n",
- RREG32(mmCP_RB0_WPTR));
- dev_info(adev->dev, " CP_RB0_RPTR_ADDR=0x%08X\n",
- RREG32(mmCP_RB0_RPTR_ADDR));
- dev_info(adev->dev, " CP_RB0_RPTR_ADDR_HI=0x%08X\n",
- RREG32(mmCP_RB0_RPTR_ADDR_HI));
- dev_info(adev->dev, " CP_RB0_CNTL=0x%08X\n",
- RREG32(mmCP_RB0_CNTL));
- dev_info(adev->dev, " CP_RB0_BASE=0x%08X\n",
- RREG32(mmCP_RB0_BASE));
- dev_info(adev->dev, " CP_RB0_BASE_HI=0x%08X\n",
- RREG32(mmCP_RB0_BASE_HI));
- dev_info(adev->dev, " CP_MEC_CNTL=0x%08X\n",
- RREG32(mmCP_MEC_CNTL));
- dev_info(adev->dev, " CP_CPF_DEBUG=0x%08X\n",
- RREG32(mmCP_CPF_DEBUG));
-
- dev_info(adev->dev, " SCRATCH_ADDR=0x%08X\n",
- RREG32(mmSCRATCH_ADDR));
- dev_info(adev->dev, " SCRATCH_UMSK=0x%08X\n",
- RREG32(mmSCRATCH_UMSK));
-
- /* init the pipes */
- mutex_lock(&adev->srbm_mutex);
- for (i = 0; i < (adev->gfx.mec.num_pipe * adev->gfx.mec.num_mec); i++) {
- int me = (i < 4) ? 1 : 2;
- int pipe = (i < 4) ? i : (i - 4);
- int queue;
-
- dev_info(adev->dev, " me: %d, pipe: %d\n", me, pipe);
- cik_srbm_select(adev, me, pipe, 0, 0);
- dev_info(adev->dev, " CP_HPD_EOP_BASE_ADDR=0x%08X\n",
- RREG32(mmCP_HPD_EOP_BASE_ADDR));
- dev_info(adev->dev, " CP_HPD_EOP_BASE_ADDR_HI=0x%08X\n",
- RREG32(mmCP_HPD_EOP_BASE_ADDR_HI));
- dev_info(adev->dev, " CP_HPD_EOP_VMID=0x%08X\n",
- RREG32(mmCP_HPD_EOP_VMID));
- dev_info(adev->dev, " CP_HPD_EOP_CONTROL=0x%08X\n",
- RREG32(mmCP_HPD_EOP_CONTROL));
-
- for (queue = 0; queue < 8; queue++) {
- cik_srbm_select(adev, me, pipe, queue, 0);
- dev_info(adev->dev, " queue: %d\n", queue);
- dev_info(adev->dev, " CP_PQ_WPTR_POLL_CNTL=0x%08X\n",
- RREG32(mmCP_PQ_WPTR_POLL_CNTL));
- dev_info(adev->dev, " CP_HQD_PQ_DOORBELL_CONTROL=0x%08X\n",
- RREG32(mmCP_HQD_PQ_DOORBELL_CONTROL));
- dev_info(adev->dev, " CP_HQD_ACTIVE=0x%08X\n",
- RREG32(mmCP_HQD_ACTIVE));
- dev_info(adev->dev, " CP_HQD_DEQUEUE_REQUEST=0x%08X\n",
- RREG32(mmCP_HQD_DEQUEUE_REQUEST));
- dev_info(adev->dev, " CP_HQD_PQ_RPTR=0x%08X\n",
- RREG32(mmCP_HQD_PQ_RPTR));
- dev_info(adev->dev, " CP_HQD_PQ_WPTR=0x%08X\n",
- RREG32(mmCP_HQD_PQ_WPTR));
- dev_info(adev->dev, " CP_HQD_PQ_BASE=0x%08X\n",
- RREG32(mmCP_HQD_PQ_BASE));
- dev_info(adev->dev, " CP_HQD_PQ_BASE_HI=0x%08X\n",
- RREG32(mmCP_HQD_PQ_BASE_HI));
- dev_info(adev->dev, " CP_HQD_PQ_CONTROL=0x%08X\n",
- RREG32(mmCP_HQD_PQ_CONTROL));
- dev_info(adev->dev, " CP_HQD_PQ_WPTR_POLL_ADDR=0x%08X\n",
- RREG32(mmCP_HQD_PQ_WPTR_POLL_ADDR));
- dev_info(adev->dev, " CP_HQD_PQ_WPTR_POLL_ADDR_HI=0x%08X\n",
- RREG32(mmCP_HQD_PQ_WPTR_POLL_ADDR_HI));
- dev_info(adev->dev, " CP_HQD_PQ_RPTR_REPORT_ADDR=0x%08X\n",
- RREG32(mmCP_HQD_PQ_RPTR_REPORT_ADDR));
- dev_info(adev->dev, " CP_HQD_PQ_RPTR_REPORT_ADDR_HI=0x%08X\n",
- RREG32(mmCP_HQD_PQ_RPTR_REPORT_ADDR_HI));
- dev_info(adev->dev, " CP_HQD_PQ_DOORBELL_CONTROL=0x%08X\n",
- RREG32(mmCP_HQD_PQ_DOORBELL_CONTROL));
- dev_info(adev->dev, " CP_HQD_PQ_WPTR=0x%08X\n",
- RREG32(mmCP_HQD_PQ_WPTR));
- dev_info(adev->dev, " CP_HQD_VMID=0x%08X\n",
- RREG32(mmCP_HQD_VMID));
- dev_info(adev->dev, " CP_MQD_BASE_ADDR=0x%08X\n",
- RREG32(mmCP_MQD_BASE_ADDR));
- dev_info(adev->dev, " CP_MQD_BASE_ADDR_HI=0x%08X\n",
- RREG32(mmCP_MQD_BASE_ADDR_HI));
- dev_info(adev->dev, " CP_MQD_CONTROL=0x%08X\n",
- RREG32(mmCP_MQD_CONTROL));
- }
- }
- cik_srbm_select(adev, 0, 0, 0, 0);
- mutex_unlock(&adev->srbm_mutex);
-
- dev_info(adev->dev, " CP_INT_CNTL_RING0=0x%08X\n",
- RREG32(mmCP_INT_CNTL_RING0));
- dev_info(adev->dev, " RLC_LB_CNTL=0x%08X\n",
- RREG32(mmRLC_LB_CNTL));
- dev_info(adev->dev, " RLC_CNTL=0x%08X\n",
- RREG32(mmRLC_CNTL));
- dev_info(adev->dev, " RLC_CGCG_CGLS_CTRL=0x%08X\n",
- RREG32(mmRLC_CGCG_CGLS_CTRL));
- dev_info(adev->dev, " RLC_LB_CNTR_INIT=0x%08X\n",
- RREG32(mmRLC_LB_CNTR_INIT));
- dev_info(adev->dev, " RLC_LB_CNTR_MAX=0x%08X\n",
- RREG32(mmRLC_LB_CNTR_MAX));
- dev_info(adev->dev, " RLC_LB_INIT_CU_MASK=0x%08X\n",
- RREG32(mmRLC_LB_INIT_CU_MASK));
- dev_info(adev->dev, " RLC_LB_PARAMS=0x%08X\n",
- RREG32(mmRLC_LB_PARAMS));
- dev_info(adev->dev, " RLC_LB_CNTL=0x%08X\n",
- RREG32(mmRLC_LB_CNTL));
- dev_info(adev->dev, " RLC_MC_CNTL=0x%08X\n",
- RREG32(mmRLC_MC_CNTL));
- dev_info(adev->dev, " RLC_UCODE_CNTL=0x%08X\n",
- RREG32(mmRLC_UCODE_CNTL));
-
- if (adev->asic_type == CHIP_BONAIRE)
- dev_info(adev->dev, " RLC_DRIVER_CPDMA_STATUS=0x%08X\n",
- RREG32(mmRLC_DRIVER_CPDMA_STATUS));
-
- mutex_lock(&adev->srbm_mutex);
- for (i = 0; i < 16; i++) {
- cik_srbm_select(adev, 0, 0, 0, i);
- dev_info(adev->dev, " VM %d:\n", i);
- dev_info(adev->dev, " SH_MEM_CONFIG=0x%08X\n",
- RREG32(mmSH_MEM_CONFIG));
- dev_info(adev->dev, " SH_MEM_APE1_BASE=0x%08X\n",
- RREG32(mmSH_MEM_APE1_BASE));
- dev_info(adev->dev, " SH_MEM_APE1_LIMIT=0x%08X\n",
- RREG32(mmSH_MEM_APE1_LIMIT));
- dev_info(adev->dev, " SH_MEM_BASES=0x%08X\n",
- RREG32(mmSH_MEM_BASES));
- }
- cik_srbm_select(adev, 0, 0, 0, 0);
- mutex_unlock(&adev->srbm_mutex);
-}
-
static int gfx_v7_0_soft_reset(void *handle)
{
u32 grbm_soft_reset = 0, srbm_soft_reset = 0;
@@ -4855,7 +4599,6 @@
srbm_soft_reset |= SRBM_SOFT_RESET__SOFT_RESET_GRBM_MASK;
if (grbm_soft_reset || srbm_soft_reset) {
- gfx_v7_0_print_status((void *)adev);
/* disable CG/PG */
gfx_v7_0_fini_pg(adev);
gfx_v7_0_update_cg(adev, false);
@@ -4898,7 +4641,6 @@
}
/* Wait a little for things to settle down */
udelay(50);
- gfx_v7_0_print_status((void *)adev);
}
return 0;
}
@@ -5150,6 +4892,7 @@
}
const struct amd_ip_funcs gfx_v7_0_ip_funcs = {
+ .name = "gfx_v7_0",
.early_init = gfx_v7_0_early_init,
.late_init = gfx_v7_0_late_init,
.sw_init = gfx_v7_0_sw_init,
@@ -5161,7 +4904,6 @@
.is_idle = gfx_v7_0_is_idle,
.wait_for_idle = gfx_v7_0_wait_for_idle,
.soft_reset = gfx_v7_0_soft_reset,
- .print_status = gfx_v7_0_print_status,
.set_clockgating_state = gfx_v7_0_set_clockgating_state,
.set_powergating_state = gfx_v7_0_set_powergating_state,
};
@@ -5268,14 +5010,11 @@
}
-int gfx_v7_0_get_cu_info(struct amdgpu_device *adev,
- struct amdgpu_cu_info *cu_info)
+static void gfx_v7_0_get_cu_info(struct amdgpu_device *adev)
{
int i, j, k, counter, active_cu_number = 0;
u32 mask, bitmap, ao_bitmap, ao_cu_mask = 0;
-
- if (!adev || !cu_info)
- return -EINVAL;
+ struct amdgpu_cu_info *cu_info = &adev->gfx.cu_info;
memset(cu_info, 0, sizeof(*cu_info));
@@ -5306,6 +5045,4 @@
cu_info->number = active_cu_number;
cu_info->ao_cu_mask = ao_cu_mask;
-
- return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.h b/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.h
index c04bfba..e747aa9 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.h
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.h
@@ -32,6 +32,5 @@
void gfx_v7_0_rlc_stop(struct amdgpu_device *adev);
uint64_t gfx_v7_0_get_gpu_clock_counter(struct amdgpu_device *adev);
void gfx_v7_0_select_se_sh(struct amdgpu_device *adev, u32 se_num, u32 sh_num);
-int gfx_v7_0_get_cu_info(struct amdgpu_device *adev, struct amdgpu_cu_info *cu_info);
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c
index f0c7b359..92647fb 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c
@@ -27,6 +27,7 @@
#include "vi.h"
#include "vid.h"
#include "amdgpu_ucode.h"
+#include "amdgpu_atombios.h"
#include "clearstate_vi.h"
#include "gmc/gmc_8_2_d.h"
@@ -51,6 +52,7 @@
#define TOPAZ_GB_ADDR_CONFIG_GOLDEN 0x22010001
#define CARRIZO_GB_ADDR_CONFIG_GOLDEN 0x22010001
+#define POLARIS11_GB_ADDR_CONFIG_GOLDEN 0x22011002
#define TONGA_GB_ADDR_CONFIG_GOLDEN 0x22011003
#define ARRAY_MODE(x) ((x) << GB_TILE_MODE0__ARRAY_MODE__SHIFT)
@@ -84,6 +86,8 @@
BPM_REG_FGCG_MAX
};
+#define RLC_FormatDirectRegListLength 14
+
MODULE_FIRMWARE("amdgpu/carrizo_ce.bin");
MODULE_FIRMWARE("amdgpu/carrizo_pfp.bin");
MODULE_FIRMWARE("amdgpu/carrizo_me.bin");
@@ -117,6 +121,20 @@
MODULE_FIRMWARE("amdgpu/fiji_mec2.bin");
MODULE_FIRMWARE("amdgpu/fiji_rlc.bin");
+MODULE_FIRMWARE("amdgpu/polaris11_ce.bin");
+MODULE_FIRMWARE("amdgpu/polaris11_pfp.bin");
+MODULE_FIRMWARE("amdgpu/polaris11_me.bin");
+MODULE_FIRMWARE("amdgpu/polaris11_mec.bin");
+MODULE_FIRMWARE("amdgpu/polaris11_mec2.bin");
+MODULE_FIRMWARE("amdgpu/polaris11_rlc.bin");
+
+MODULE_FIRMWARE("amdgpu/polaris10_ce.bin");
+MODULE_FIRMWARE("amdgpu/polaris10_pfp.bin");
+MODULE_FIRMWARE("amdgpu/polaris10_me.bin");
+MODULE_FIRMWARE("amdgpu/polaris10_mec.bin");
+MODULE_FIRMWARE("amdgpu/polaris10_mec2.bin");
+MODULE_FIRMWARE("amdgpu/polaris10_rlc.bin");
+
static const struct amdgpu_gds_reg_offset amdgpu_gds_reg_offset[] =
{
{mmGDS_VMID0_BASE, mmGDS_VMID0_SIZE, mmGDS_GWS_VMID0, mmGDS_OA_VMID0},
@@ -247,6 +265,64 @@
mmCP_MEM_SLP_CNTL, 0x00000001, 0x00000001,
};
+static const u32 golden_settings_polaris11_a11[] =
+{
+ mmCB_HW_CONTROL_3, 0x000001ff, 0x00000040,
+ mmDB_DEBUG2, 0xf00fffff, 0x00000400,
+ mmPA_SC_ENHANCE, 0xffffffff, 0x20000001,
+ mmPA_SC_LINE_STIPPLE_STATE, 0x0000ff0f, 0x00000000,
+ mmRLC_CGCG_CGLS_CTRL, 0x00000003, 0x0001003c,
+ mmRLC_CGCG_CGLS_CTRL_3D, 0xffffffff, 0x0001003c,
+ mmSQ_CONFIG, 0x07f80000, 0x07180000,
+ mmTA_CNTL_AUX, 0x000f000f, 0x000b0000,
+ mmTCC_CTRL, 0x00100000, 0xf31fff7f,
+ mmTCP_ADDR_CONFIG, 0x000003ff, 0x000000f3,
+ mmTCP_CHAN_STEER_HI, 0xffffffff, 0x00000000,
+ mmTCP_CHAN_STEER_LO, 0xffffffff, 0x00003210,
+};
+
+static const u32 polaris11_golden_common_all[] =
+{
+ mmGRBM_GFX_INDEX, 0xffffffff, 0xe0000000,
+ mmPA_SC_RASTER_CONFIG, 0xffffffff, 0x16000012,
+ mmPA_SC_RASTER_CONFIG_1, 0xffffffff, 0x00000000,
+ mmGB_ADDR_CONFIG, 0xffffffff, 0x22011002,
+ mmSPI_RESOURCE_RESERVE_CU_0, 0xffffffff, 0x00000800,
+ mmSPI_RESOURCE_RESERVE_CU_1, 0xffffffff, 0x00000800,
+ mmSPI_RESOURCE_RESERVE_EN_CU_0, 0xffffffff, 0x00007FBF,
+ mmSPI_RESOURCE_RESERVE_EN_CU_1, 0xffffffff, 0x00007FAF,
+};
+
+static const u32 golden_settings_polaris10_a11[] =
+{
+ mmATC_MISC_CG, 0x000c0fc0, 0x000c0200,
+ mmCB_HW_CONTROL_3, 0x000001ff, 0x00000040,
+ mmDB_DEBUG2, 0xf00fffff, 0x00000400,
+ mmPA_SC_ENHANCE, 0xffffffff, 0x20000001,
+ mmPA_SC_LINE_STIPPLE_STATE, 0x0000ff0f, 0x00000000,
+ mmPA_SC_RASTER_CONFIG, 0x3f3fffff, 0x16000012,
+ mmPA_SC_RASTER_CONFIG_1, 0x0000003f, 0x0000002a,
+ mmRLC_CGCG_CGLS_CTRL, 0x00000003, 0x0001003c,
+ mmRLC_CGCG_CGLS_CTRL_3D, 0xffffffff, 0x0001003c,
+ mmSQ_CONFIG, 0x07f80000, 0x07180000,
+ mmTA_CNTL_AUX, 0x000f000f, 0x000b0000,
+ mmTCC_CTRL, 0x00100000, 0xf31fff7f,
+ mmTCP_ADDR_CONFIG, 0x000003ff, 0x000000f7,
+ mmTCP_CHAN_STEER_HI, 0xffffffff, 0x00000000,
+};
+
+static const u32 polaris10_golden_common_all[] =
+{
+ mmGRBM_GFX_INDEX, 0xffffffff, 0xe0000000,
+ mmPA_SC_RASTER_CONFIG, 0xffffffff, 0x16000012,
+ mmPA_SC_RASTER_CONFIG_1, 0xffffffff, 0x0000002A,
+ mmGB_ADDR_CONFIG, 0xffffffff, 0x22011003,
+ mmSPI_RESOURCE_RESERVE_CU_0, 0xffffffff, 0x00000800,
+ mmSPI_RESOURCE_RESERVE_CU_1, 0xffffffff, 0x00000800,
+ mmSPI_RESOURCE_RESERVE_EN_CU_0, 0xffffffff, 0x00007FBF,
+ mmSPI_RESOURCE_RESERVE_EN_CU_1, 0xffffffff, 0x00007FAF,
+};
+
static const u32 fiji_golden_common_all[] =
{
mmGRBM_GFX_INDEX, 0xffffffff, 0xe0000000,
@@ -527,7 +603,7 @@
mmPA_SC_LINE_STIPPLE_STATE, 0x0000ff0f, 0x00000000,
mmRLC_CGCG_CGLS_CTRL, 0x00000003, 0x0001003c,
mmTA_CNTL_AUX, 0x000f000f, 0x000b0000,
- mmTCC_CTRL, 0x00100000, 0xf31fff7f,
+ mmTCC_CTRL, 0x00100000, 0xf31fff7f,
mmTCC_EXE_DISABLE, 0x00000002, 0x00000002,
mmTCP_ADDR_CONFIG, 0x0000000f, 0x000000f1,
mmTCP_CHAN_STEER_LO, 0xffffffff, 0x10101010,
@@ -558,6 +634,9 @@
static void gfx_v8_0_set_ring_funcs(struct amdgpu_device *adev);
static void gfx_v8_0_set_irq_funcs(struct amdgpu_device *adev);
static void gfx_v8_0_set_gds_init(struct amdgpu_device *adev);
+static void gfx_v8_0_set_rlc_funcs(struct amdgpu_device *adev);
+static u32 gfx_v8_0_get_csb_size(struct amdgpu_device *adev);
+static void gfx_v8_0_get_cu_info(struct amdgpu_device *adev);
static void gfx_v8_0_init_golden_registers(struct amdgpu_device *adev)
{
@@ -596,6 +675,22 @@
tonga_golden_common_all,
(const u32)ARRAY_SIZE(tonga_golden_common_all));
break;
+ case CHIP_POLARIS11:
+ amdgpu_program_register_sequence(adev,
+ golden_settings_polaris11_a11,
+ (const u32)ARRAY_SIZE(golden_settings_polaris11_a11));
+ amdgpu_program_register_sequence(adev,
+ polaris11_golden_common_all,
+ (const u32)ARRAY_SIZE(polaris11_golden_common_all));
+ break;
+ case CHIP_POLARIS10:
+ amdgpu_program_register_sequence(adev,
+ golden_settings_polaris10_a11,
+ (const u32)ARRAY_SIZE(golden_settings_polaris10_a11));
+ amdgpu_program_register_sequence(adev,
+ polaris10_golden_common_all,
+ (const u32)ARRAY_SIZE(polaris10_golden_common_all));
+ break;
case CHIP_CARRIZO:
amdgpu_program_register_sequence(adev,
cz_mgcg_cgcg_init,
@@ -706,7 +801,7 @@
ib.ptr[2] = 0xDEADBEEF;
ib.length_dw = 3;
- r = amdgpu_ib_schedule(ring, 1, &ib, NULL, &f);
+ r = amdgpu_ib_schedule(ring, 1, &ib, NULL, NULL, &f);
if (r)
goto err2;
@@ -747,6 +842,8 @@
struct amdgpu_firmware_info *info = NULL;
const struct common_firmware_header *header = NULL;
const struct gfx_firmware_header_v1_0 *cp_hdr;
+ const struct rlc_firmware_header_v2_0 *rlc_hdr;
+ unsigned int *tmp = NULL, i;
DRM_DEBUG("\n");
@@ -763,6 +860,12 @@
case CHIP_FIJI:
chip_name = "fiji";
break;
+ case CHIP_POLARIS11:
+ chip_name = "polaris11";
+ break;
+ case CHIP_POLARIS10:
+ chip_name = "polaris10";
+ break;
case CHIP_STONEY:
chip_name = "stoney";
break;
@@ -808,9 +911,49 @@
if (err)
goto out;
err = amdgpu_ucode_validate(adev->gfx.rlc_fw);
- cp_hdr = (const struct gfx_firmware_header_v1_0 *)adev->gfx.rlc_fw->data;
- adev->gfx.rlc_fw_version = le32_to_cpu(cp_hdr->header.ucode_version);
- adev->gfx.rlc_feature_version = le32_to_cpu(cp_hdr->ucode_feature_version);
+ rlc_hdr = (const struct rlc_firmware_header_v2_0 *)adev->gfx.rlc_fw->data;
+ adev->gfx.rlc_fw_version = le32_to_cpu(rlc_hdr->header.ucode_version);
+ adev->gfx.rlc_feature_version = le32_to_cpu(rlc_hdr->ucode_feature_version);
+
+ adev->gfx.rlc.save_and_restore_offset =
+ le32_to_cpu(rlc_hdr->save_and_restore_offset);
+ adev->gfx.rlc.clear_state_descriptor_offset =
+ le32_to_cpu(rlc_hdr->clear_state_descriptor_offset);
+ adev->gfx.rlc.avail_scratch_ram_locations =
+ le32_to_cpu(rlc_hdr->avail_scratch_ram_locations);
+ adev->gfx.rlc.reg_restore_list_size =
+ le32_to_cpu(rlc_hdr->reg_restore_list_size);
+ adev->gfx.rlc.reg_list_format_start =
+ le32_to_cpu(rlc_hdr->reg_list_format_start);
+ adev->gfx.rlc.reg_list_format_separate_start =
+ le32_to_cpu(rlc_hdr->reg_list_format_separate_start);
+ adev->gfx.rlc.starting_offsets_start =
+ le32_to_cpu(rlc_hdr->starting_offsets_start);
+ adev->gfx.rlc.reg_list_format_size_bytes =
+ le32_to_cpu(rlc_hdr->reg_list_format_size_bytes);
+ adev->gfx.rlc.reg_list_size_bytes =
+ le32_to_cpu(rlc_hdr->reg_list_size_bytes);
+
+ adev->gfx.rlc.register_list_format =
+ kmalloc(adev->gfx.rlc.reg_list_format_size_bytes +
+ adev->gfx.rlc.reg_list_size_bytes, GFP_KERNEL);
+
+ if (!adev->gfx.rlc.register_list_format) {
+ err = -ENOMEM;
+ goto out;
+ }
+
+ tmp = (unsigned int *)((uintptr_t)rlc_hdr +
+ le32_to_cpu(rlc_hdr->reg_list_format_array_offset_bytes));
+ for (i = 0 ; i < (rlc_hdr->reg_list_format_size_bytes >> 2); i++)
+ adev->gfx.rlc.register_list_format[i] = le32_to_cpu(tmp[i]);
+
+ adev->gfx.rlc.register_restore = adev->gfx.rlc.register_list_format + i;
+
+ tmp = (unsigned int *)((uintptr_t)rlc_hdr +
+ le32_to_cpu(rlc_hdr->reg_list_array_offset_bytes));
+ for (i = 0 ; i < (rlc_hdr->reg_list_size_bytes >> 2); i++)
+ adev->gfx.rlc.register_restore[i] = le32_to_cpu(tmp[i]);
snprintf(fw_name, sizeof(fw_name), "amdgpu/%s_mec.bin", chip_name);
err = request_firmware(&adev->gfx.mec_fw, fw_name, adev->dev);
@@ -911,6 +1054,153 @@
return err;
}
+static void gfx_v8_0_get_csb_buffer(struct amdgpu_device *adev,
+ volatile u32 *buffer)
+{
+ u32 count = 0, i;
+ const struct cs_section_def *sect = NULL;
+ const struct cs_extent_def *ext = NULL;
+
+ if (adev->gfx.rlc.cs_data == NULL)
+ return;
+ if (buffer == NULL)
+ return;
+
+ buffer[count++] = cpu_to_le32(PACKET3(PACKET3_PREAMBLE_CNTL, 0));
+ buffer[count++] = cpu_to_le32(PACKET3_PREAMBLE_BEGIN_CLEAR_STATE);
+
+ buffer[count++] = cpu_to_le32(PACKET3(PACKET3_CONTEXT_CONTROL, 1));
+ buffer[count++] = cpu_to_le32(0x80000000);
+ buffer[count++] = cpu_to_le32(0x80000000);
+
+ for (sect = adev->gfx.rlc.cs_data; sect->section != NULL; ++sect) {
+ for (ext = sect->section; ext->extent != NULL; ++ext) {
+ if (sect->id == SECT_CONTEXT) {
+ buffer[count++] =
+ cpu_to_le32(PACKET3(PACKET3_SET_CONTEXT_REG, ext->reg_count));
+ buffer[count++] = cpu_to_le32(ext->reg_index -
+ PACKET3_SET_CONTEXT_REG_START);
+ for (i = 0; i < ext->reg_count; i++)
+ buffer[count++] = cpu_to_le32(ext->extent[i]);
+ } else {
+ return;
+ }
+ }
+ }
+
+ buffer[count++] = cpu_to_le32(PACKET3(PACKET3_SET_CONTEXT_REG, 2));
+ buffer[count++] = cpu_to_le32(mmPA_SC_RASTER_CONFIG -
+ PACKET3_SET_CONTEXT_REG_START);
+ switch (adev->asic_type) {
+ case CHIP_TONGA:
+ case CHIP_POLARIS10:
+ buffer[count++] = cpu_to_le32(0x16000012);
+ buffer[count++] = cpu_to_le32(0x0000002A);
+ break;
+ case CHIP_POLARIS11:
+ buffer[count++] = cpu_to_le32(0x16000012);
+ buffer[count++] = cpu_to_le32(0x00000000);
+ break;
+ case CHIP_FIJI:
+ buffer[count++] = cpu_to_le32(0x3a00161a);
+ buffer[count++] = cpu_to_le32(0x0000002e);
+ break;
+ case CHIP_TOPAZ:
+ case CHIP_CARRIZO:
+ buffer[count++] = cpu_to_le32(0x00000002);
+ buffer[count++] = cpu_to_le32(0x00000000);
+ break;
+ case CHIP_STONEY:
+ buffer[count++] = cpu_to_le32(0x00000000);
+ buffer[count++] = cpu_to_le32(0x00000000);
+ break;
+ default:
+ buffer[count++] = cpu_to_le32(0x00000000);
+ buffer[count++] = cpu_to_le32(0x00000000);
+ break;
+ }
+
+ buffer[count++] = cpu_to_le32(PACKET3(PACKET3_PREAMBLE_CNTL, 0));
+ buffer[count++] = cpu_to_le32(PACKET3_PREAMBLE_END_CLEAR_STATE);
+
+ buffer[count++] = cpu_to_le32(PACKET3(PACKET3_CLEAR_STATE, 0));
+ buffer[count++] = cpu_to_le32(0);
+}
+
+static void gfx_v8_0_rlc_fini(struct amdgpu_device *adev)
+{
+ int r;
+
+ /* clear state block */
+ if (adev->gfx.rlc.clear_state_obj) {
+ r = amdgpu_bo_reserve(adev->gfx.rlc.clear_state_obj, false);
+ if (unlikely(r != 0))
+ dev_warn(adev->dev, "(%d) reserve RLC c bo failed\n", r);
+ amdgpu_bo_unpin(adev->gfx.rlc.clear_state_obj);
+ amdgpu_bo_unreserve(adev->gfx.rlc.clear_state_obj);
+
+ amdgpu_bo_unref(&adev->gfx.rlc.clear_state_obj);
+ adev->gfx.rlc.clear_state_obj = NULL;
+ }
+}
+
+static int gfx_v8_0_rlc_init(struct amdgpu_device *adev)
+{
+ volatile u32 *dst_ptr;
+ u32 dws;
+ const struct cs_section_def *cs_data;
+ int r;
+
+ adev->gfx.rlc.cs_data = vi_cs_data;
+
+ cs_data = adev->gfx.rlc.cs_data;
+
+ if (cs_data) {
+ /* clear state block */
+ adev->gfx.rlc.clear_state_size = dws = gfx_v8_0_get_csb_size(adev);
+
+ if (adev->gfx.rlc.clear_state_obj == NULL) {
+ r = amdgpu_bo_create(adev, dws * 4, PAGE_SIZE, true,
+ AMDGPU_GEM_DOMAIN_VRAM,
+ AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED,
+ NULL, NULL,
+ &adev->gfx.rlc.clear_state_obj);
+ if (r) {
+ dev_warn(adev->dev, "(%d) create RLC c bo failed\n", r);
+ gfx_v8_0_rlc_fini(adev);
+ return r;
+ }
+ }
+ r = amdgpu_bo_reserve(adev->gfx.rlc.clear_state_obj, false);
+ if (unlikely(r != 0)) {
+ gfx_v8_0_rlc_fini(adev);
+ return r;
+ }
+ r = amdgpu_bo_pin(adev->gfx.rlc.clear_state_obj, AMDGPU_GEM_DOMAIN_VRAM,
+ &adev->gfx.rlc.clear_state_gpu_addr);
+ if (r) {
+ amdgpu_bo_unreserve(adev->gfx.rlc.clear_state_obj);
+ dev_warn(adev->dev, "(%d) pin RLC c bo failed\n", r);
+ gfx_v8_0_rlc_fini(adev);
+ return r;
+ }
+
+ r = amdgpu_bo_kmap(adev->gfx.rlc.clear_state_obj, (void **)&adev->gfx.rlc.cs_ptr);
+ if (r) {
+ dev_warn(adev->dev, "(%d) map RLC c bo failed\n", r);
+ gfx_v8_0_rlc_fini(adev);
+ return r;
+ }
+ /* set up the cs buffer */
+ dst_ptr = adev->gfx.rlc.cs_ptr;
+ gfx_v8_0_get_csb_buffer(adev, dst_ptr);
+ amdgpu_bo_kunmap(adev->gfx.rlc.clear_state_obj);
+ amdgpu_bo_unreserve(adev->gfx.rlc.clear_state_obj);
+ }
+
+ return 0;
+}
+
static void gfx_v8_0_mec_fini(struct amdgpu_device *adev)
{
int r;
@@ -1262,7 +1552,7 @@
ib.ptr[ib.length_dw++] = EVENT_TYPE(7) | EVENT_INDEX(4);
/* shedule the ib on the ring */
- r = amdgpu_ib_schedule(ring, 1, &ib, NULL, &f);
+ r = amdgpu_ib_schedule(ring, 1, &ib, NULL, NULL, &f);
if (r) {
DRM_ERROR("amdgpu: ib submit failed (%d).\n", r);
goto fail;
@@ -1296,12 +1586,13 @@
return r;
}
-static void gfx_v8_0_gpu_early_init(struct amdgpu_device *adev)
+static int gfx_v8_0_gpu_early_init(struct amdgpu_device *adev)
{
u32 gb_addr_config;
u32 mc_shared_chmap, mc_arb_ramcfg;
u32 dimm00_addr_map, dimm01_addr_map, dimm10_addr_map, dimm11_addr_map;
u32 tmp;
+ int ret;
switch (adev->asic_type) {
case CHIP_TOPAZ:
@@ -1338,6 +1629,34 @@
adev->gfx.config.sc_earlyz_tile_fifo_size = 0x130;
gb_addr_config = TONGA_GB_ADDR_CONFIG_GOLDEN;
break;
+ case CHIP_POLARIS11:
+ ret = amdgpu_atombios_get_gfx_info(adev);
+ if (ret)
+ return ret;
+ adev->gfx.config.max_gprs = 256;
+ adev->gfx.config.max_gs_threads = 32;
+ adev->gfx.config.max_hw_contexts = 8;
+
+ adev->gfx.config.sc_prim_fifo_size_frontend = 0x20;
+ adev->gfx.config.sc_prim_fifo_size_backend = 0x100;
+ adev->gfx.config.sc_hiz_tile_fifo_size = 0x30;
+ adev->gfx.config.sc_earlyz_tile_fifo_size = 0x130;
+ gb_addr_config = POLARIS11_GB_ADDR_CONFIG_GOLDEN;
+ break;
+ case CHIP_POLARIS10:
+ ret = amdgpu_atombios_get_gfx_info(adev);
+ if (ret)
+ return ret;
+ adev->gfx.config.max_gprs = 256;
+ adev->gfx.config.max_gs_threads = 32;
+ adev->gfx.config.max_hw_contexts = 8;
+
+ adev->gfx.config.sc_prim_fifo_size_frontend = 0x20;
+ adev->gfx.config.sc_prim_fifo_size_backend = 0x100;
+ adev->gfx.config.sc_hiz_tile_fifo_size = 0x30;
+ adev->gfx.config.sc_earlyz_tile_fifo_size = 0x130;
+ gb_addr_config = TONGA_GB_ADDR_CONFIG_GOLDEN;
+ break;
case CHIP_TONGA:
adev->gfx.config.max_shader_engines = 4;
adev->gfx.config.max_tile_pipes = 8;
@@ -1520,6 +1839,8 @@
break;
}
adev->gfx.config.gb_addr_config = gb_addr_config;
+
+ return 0;
}
static int gfx_v8_0_sw_init(void *handle)
@@ -1553,6 +1874,12 @@
return r;
}
+ r = gfx_v8_0_rlc_init(adev);
+ if (r) {
+ DRM_ERROR("Failed to init rlc BOs!\n");
+ return r;
+ }
+
r = gfx_v8_0_mec_init(adev);
if (r) {
DRM_ERROR("Failed to init MEC BOs!\n");
@@ -1570,7 +1897,7 @@
ring->doorbell_index = AMDGPU_DOORBELL_GFX_RING0;
}
- r = amdgpu_ring_init(adev, ring, 1024 * 1024,
+ r = amdgpu_ring_init(adev, ring, 1024,
PACKET3(PACKET3_NOP, 0x3FFF), 0xf,
&adev->gfx.eop_irq, AMDGPU_CP_IRQ_GFX_EOP,
AMDGPU_RING_TYPE_GFX);
@@ -1594,10 +1921,10 @@
ring->me = 1; /* first MEC */
ring->pipe = i / 8;
ring->queue = i % 8;
- sprintf(ring->name, "comp %d.%d.%d", ring->me, ring->pipe, ring->queue);
+ sprintf(ring->name, "comp_%d.%d.%d", ring->me, ring->pipe, ring->queue);
irq_type = AMDGPU_CP_IRQ_COMPUTE_MEC1_PIPE0_EOP + ring->pipe;
/* type-2 packets are deprecated on MEC, use type-3 instead */
- r = amdgpu_ring_init(adev, ring, 1024 * 1024,
+ r = amdgpu_ring_init(adev, ring, 1024,
PACKET3(PACKET3_NOP, 0x3FFF), 0xf,
&adev->gfx.eop_irq, irq_type,
AMDGPU_RING_TYPE_COMPUTE);
@@ -1629,7 +1956,9 @@
adev->gfx.ce_ram_size = 0x8000;
- gfx_v8_0_gpu_early_init(adev);
+ r = gfx_v8_0_gpu_early_init(adev);
+ if (r)
+ return r;
return 0;
}
@@ -1650,6 +1979,10 @@
gfx_v8_0_mec_fini(adev);
+ gfx_v8_0_rlc_fini(adev);
+
+ kfree(adev->gfx.rlc.register_list_format);
+
return 0;
}
@@ -2219,6 +2552,410 @@
WREG32(mmGB_MACROTILE_MODE0 + reg_offset, mod2array[reg_offset]);
break;
+ case CHIP_POLARIS11:
+ modearray[0] = (ARRAY_MODE(ARRAY_2D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ TILE_SPLIT(ADDR_SURF_TILE_SPLIT_64B) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DEPTH_MICRO_TILING));
+ modearray[1] = (ARRAY_MODE(ARRAY_2D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ TILE_SPLIT(ADDR_SURF_TILE_SPLIT_128B) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DEPTH_MICRO_TILING));
+ modearray[2] = (ARRAY_MODE(ARRAY_2D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ TILE_SPLIT(ADDR_SURF_TILE_SPLIT_256B) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DEPTH_MICRO_TILING));
+ modearray[3] = (ARRAY_MODE(ARRAY_2D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ TILE_SPLIT(ADDR_SURF_TILE_SPLIT_512B) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DEPTH_MICRO_TILING));
+ modearray[4] = (ARRAY_MODE(ARRAY_2D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ TILE_SPLIT(ADDR_SURF_TILE_SPLIT_2KB) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DEPTH_MICRO_TILING));
+ modearray[5] = (ARRAY_MODE(ARRAY_1D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ TILE_SPLIT(ADDR_SURF_TILE_SPLIT_2KB) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DEPTH_MICRO_TILING));
+ modearray[6] = (ARRAY_MODE(ARRAY_PRT_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ TILE_SPLIT(ADDR_SURF_TILE_SPLIT_2KB) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DEPTH_MICRO_TILING));
+ modearray[7] = (ARRAY_MODE(ARRAY_PRT_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ TILE_SPLIT(ADDR_SURF_TILE_SPLIT_2KB) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DEPTH_MICRO_TILING));
+ modearray[8] = (ARRAY_MODE(ARRAY_LINEAR_ALIGNED) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16));
+ modearray[9] = (ARRAY_MODE(ARRAY_1D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DISPLAY_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_2));
+ modearray[10] = (ARRAY_MODE(ARRAY_2D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DISPLAY_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_2));
+ modearray[11] = (ARRAY_MODE(ARRAY_PRT_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DISPLAY_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_8));
+ modearray[12] = (ARRAY_MODE(ARRAY_PRT_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DISPLAY_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_8));
+ modearray[13] = (ARRAY_MODE(ARRAY_1D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THIN_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_2));
+ modearray[14] = (ARRAY_MODE(ARRAY_2D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THIN_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_2));
+ modearray[15] = (ARRAY_MODE(ARRAY_3D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THIN_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_2));
+ modearray[16] = (ARRAY_MODE(ARRAY_PRT_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THIN_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_8));
+ modearray[17] = (ARRAY_MODE(ARRAY_PRT_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THIN_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_8));
+ modearray[18] = (ARRAY_MODE(ARRAY_1D_TILED_THICK) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THIN_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_1));
+ modearray[19] = (ARRAY_MODE(ARRAY_1D_TILED_THICK) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THICK_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_1));
+ modearray[20] = (ARRAY_MODE(ARRAY_2D_TILED_THICK) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THICK_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_1));
+ modearray[21] = (ARRAY_MODE(ARRAY_3D_TILED_THICK) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THICK_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_1));
+ modearray[22] = (ARRAY_MODE(ARRAY_PRT_TILED_THICK) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THICK_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_1));
+ modearray[23] = (ARRAY_MODE(ARRAY_PRT_TILED_THICK) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THICK_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_1));
+ modearray[24] = (ARRAY_MODE(ARRAY_2D_TILED_THICK) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THIN_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_1));
+ modearray[25] = (ARRAY_MODE(ARRAY_2D_TILED_XTHICK) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THICK_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_1));
+ modearray[26] = (ARRAY_MODE(ARRAY_3D_TILED_XTHICK) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THICK_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_1));
+ modearray[27] = (ARRAY_MODE(ARRAY_1D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_ROTATED_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_2));
+ modearray[28] = (ARRAY_MODE(ARRAY_2D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_ROTATED_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_2));
+ modearray[29] = (ARRAY_MODE(ARRAY_PRT_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_ROTATED_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_8));
+ modearray[30] = (ARRAY_MODE(ARRAY_PRT_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_ROTATED_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_8));
+
+ mod2array[0] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_4) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_4) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[1] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_4) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_4) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[2] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_4) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_4) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[3] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_2) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_4) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[4] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_1) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_2) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[5] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_1) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_2) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[6] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_1) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_2) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[8] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_2) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_8) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_4) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[9] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_2) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_4) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_4) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[10] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_4) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_4) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[11] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_2) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_4) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[12] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_1) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_2) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[13] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_1) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_2) |
+ NUM_BANKS(ADDR_SURF_8_BANK));
+
+ mod2array[14] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_1) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_1) |
+ NUM_BANKS(ADDR_SURF_4_BANK));
+
+ for (reg_offset = 0; reg_offset < num_tile_mode_states; reg_offset++)
+ WREG32(mmGB_TILE_MODE0 + reg_offset, modearray[reg_offset]);
+
+ for (reg_offset = 0; reg_offset < num_secondary_tile_mode_states; reg_offset++)
+ if (reg_offset != 7)
+ WREG32(mmGB_MACROTILE_MODE0 + reg_offset, mod2array[reg_offset]);
+
+ break;
+ case CHIP_POLARIS10:
+ modearray[0] = (ARRAY_MODE(ARRAY_2D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ TILE_SPLIT(ADDR_SURF_TILE_SPLIT_64B) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DEPTH_MICRO_TILING));
+ modearray[1] = (ARRAY_MODE(ARRAY_2D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ TILE_SPLIT(ADDR_SURF_TILE_SPLIT_128B) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DEPTH_MICRO_TILING));
+ modearray[2] = (ARRAY_MODE(ARRAY_2D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ TILE_SPLIT(ADDR_SURF_TILE_SPLIT_256B) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DEPTH_MICRO_TILING));
+ modearray[3] = (ARRAY_MODE(ARRAY_2D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ TILE_SPLIT(ADDR_SURF_TILE_SPLIT_512B) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DEPTH_MICRO_TILING));
+ modearray[4] = (ARRAY_MODE(ARRAY_2D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ TILE_SPLIT(ADDR_SURF_TILE_SPLIT_2KB) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DEPTH_MICRO_TILING));
+ modearray[5] = (ARRAY_MODE(ARRAY_1D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ TILE_SPLIT(ADDR_SURF_TILE_SPLIT_2KB) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DEPTH_MICRO_TILING));
+ modearray[6] = (ARRAY_MODE(ARRAY_PRT_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ TILE_SPLIT(ADDR_SURF_TILE_SPLIT_2KB) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DEPTH_MICRO_TILING));
+ modearray[7] = (ARRAY_MODE(ARRAY_PRT_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ TILE_SPLIT(ADDR_SURF_TILE_SPLIT_2KB) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DEPTH_MICRO_TILING));
+ modearray[8] = (ARRAY_MODE(ARRAY_LINEAR_ALIGNED) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16));
+ modearray[9] = (ARRAY_MODE(ARRAY_1D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DISPLAY_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_2));
+ modearray[10] = (ARRAY_MODE(ARRAY_2D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DISPLAY_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_2));
+ modearray[11] = (ARRAY_MODE(ARRAY_PRT_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DISPLAY_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_8));
+ modearray[12] = (ARRAY_MODE(ARRAY_PRT_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_DISPLAY_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_8));
+ modearray[13] = (ARRAY_MODE(ARRAY_1D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THIN_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_2));
+ modearray[14] = (ARRAY_MODE(ARRAY_2D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THIN_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_2));
+ modearray[15] = (ARRAY_MODE(ARRAY_3D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THIN_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_2));
+ modearray[16] = (ARRAY_MODE(ARRAY_PRT_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THIN_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_8));
+ modearray[17] = (ARRAY_MODE(ARRAY_PRT_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THIN_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_8));
+ modearray[18] = (ARRAY_MODE(ARRAY_1D_TILED_THICK) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THIN_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_1));
+ modearray[19] = (ARRAY_MODE(ARRAY_1D_TILED_THICK) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THICK_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_1));
+ modearray[20] = (ARRAY_MODE(ARRAY_2D_TILED_THICK) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THICK_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_1));
+ modearray[21] = (ARRAY_MODE(ARRAY_3D_TILED_THICK) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THICK_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_1));
+ modearray[22] = (ARRAY_MODE(ARRAY_PRT_TILED_THICK) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THICK_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_1));
+ modearray[23] = (ARRAY_MODE(ARRAY_PRT_TILED_THICK) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THICK_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_1));
+ modearray[24] = (ARRAY_MODE(ARRAY_2D_TILED_THICK) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THIN_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_1));
+ modearray[25] = (ARRAY_MODE(ARRAY_2D_TILED_XTHICK) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THICK_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_1));
+ modearray[26] = (ARRAY_MODE(ARRAY_3D_TILED_XTHICK) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_THICK_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_1));
+ modearray[27] = (ARRAY_MODE(ARRAY_1D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_ROTATED_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_2));
+ modearray[28] = (ARRAY_MODE(ARRAY_2D_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_ROTATED_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_2));
+ modearray[29] = (ARRAY_MODE(ARRAY_PRT_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P8_32x32_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_ROTATED_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_8));
+ modearray[30] = (ARRAY_MODE(ARRAY_PRT_TILED_THIN1) |
+ PIPE_CONFIG(ADDR_SURF_P4_16x16) |
+ MICRO_TILE_MODE_NEW(ADDR_SURF_ROTATED_MICRO_TILING) |
+ SAMPLE_SPLIT(ADDR_SURF_SAMPLE_SPLIT_8));
+
+ mod2array[0] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_4) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_4) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[1] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_4) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_4) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[2] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_4) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_4) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[3] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_4) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_4) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[4] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_2) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_2) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[5] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_1) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_1) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[6] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_1) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_1) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[8] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_8) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_4) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[9] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_4) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_4) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[10] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_2) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_2) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[11] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_1) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_2) |
+ NUM_BANKS(ADDR_SURF_16_BANK));
+
+ mod2array[12] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_1) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_1) |
+ NUM_BANKS(ADDR_SURF_8_BANK));
+
+ mod2array[13] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_1) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_1) |
+ NUM_BANKS(ADDR_SURF_4_BANK));
+
+ mod2array[14] = (BANK_WIDTH(ADDR_SURF_BANK_WIDTH_1) |
+ BANK_HEIGHT(ADDR_SURF_BANK_HEIGHT_1) |
+ MACRO_TILE_ASPECT(ADDR_SURF_MACRO_ASPECT_1) |
+ NUM_BANKS(ADDR_SURF_4_BANK));
+
+ for (reg_offset = 0; reg_offset < num_tile_mode_states; reg_offset++)
+ WREG32(mmGB_TILE_MODE0 + reg_offset, modearray[reg_offset]);
+
+ for (reg_offset = 0; reg_offset < num_secondary_tile_mode_states; reg_offset++)
+ if (reg_offset != 7)
+ WREG32(mmGB_MACROTILE_MODE0 + reg_offset, mod2array[reg_offset]);
+
+ break;
case CHIP_STONEY:
modearray[0] = (ARRAY_MODE(ARRAY_2D_TILED_THIN1) |
PIPE_CONFIG(ADDR_SURF_P2) |
@@ -2695,6 +3432,7 @@
gfx_v8_0_tiling_mode_table_init(adev);
gfx_v8_0_setup_rb(adev);
+ gfx_v8_0_get_cu_info(adev);
/* XXX SH_MEM regs */
/* where to put LDS, scratch, GPUVM in FSA64 space */
@@ -2788,6 +3526,188 @@
WREG32(mmCP_INT_CNTL_RING0, tmp);
}
+static void gfx_v8_0_init_csb(struct amdgpu_device *adev)
+{
+ /* csib */
+ WREG32(mmRLC_CSIB_ADDR_HI,
+ adev->gfx.rlc.clear_state_gpu_addr >> 32);
+ WREG32(mmRLC_CSIB_ADDR_LO,
+ adev->gfx.rlc.clear_state_gpu_addr & 0xfffffffc);
+ WREG32(mmRLC_CSIB_LENGTH,
+ adev->gfx.rlc.clear_state_size);
+}
+
+static void gfx_v8_0_parse_ind_reg_list(int *register_list_format,
+ int ind_offset,
+ int list_size,
+ int *unique_indices,
+ int *indices_count,
+ int max_indices,
+ int *ind_start_offsets,
+ int *offset_count,
+ int max_offset)
+{
+ int indices;
+ bool new_entry = true;
+
+ for (; ind_offset < list_size; ind_offset++) {
+
+ if (new_entry) {
+ new_entry = false;
+ ind_start_offsets[*offset_count] = ind_offset;
+ *offset_count = *offset_count + 1;
+ BUG_ON(*offset_count >= max_offset);
+ }
+
+ if (register_list_format[ind_offset] == 0xFFFFFFFF) {
+ new_entry = true;
+ continue;
+ }
+
+ ind_offset += 2;
+
+ /* look for the matching indice */
+ for (indices = 0;
+ indices < *indices_count;
+ indices++) {
+ if (unique_indices[indices] ==
+ register_list_format[ind_offset])
+ break;
+ }
+
+ if (indices >= *indices_count) {
+ unique_indices[*indices_count] =
+ register_list_format[ind_offset];
+ indices = *indices_count;
+ *indices_count = *indices_count + 1;
+ BUG_ON(*indices_count >= max_indices);
+ }
+
+ register_list_format[ind_offset] = indices;
+ }
+}
+
+static int gfx_v8_0_init_save_restore_list(struct amdgpu_device *adev)
+{
+ int i, temp, data;
+ int unique_indices[] = {0, 0, 0, 0, 0, 0, 0, 0};
+ int indices_count = 0;
+ int indirect_start_offsets[] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
+ int offset_count = 0;
+
+ int list_size;
+ unsigned int *register_list_format =
+ kmalloc(adev->gfx.rlc.reg_list_format_size_bytes, GFP_KERNEL);
+ if (register_list_format == NULL)
+ return -ENOMEM;
+ memcpy(register_list_format, adev->gfx.rlc.register_list_format,
+ adev->gfx.rlc.reg_list_format_size_bytes);
+
+ gfx_v8_0_parse_ind_reg_list(register_list_format,
+ RLC_FormatDirectRegListLength,
+ adev->gfx.rlc.reg_list_format_size_bytes >> 2,
+ unique_indices,
+ &indices_count,
+ sizeof(unique_indices) / sizeof(int),
+ indirect_start_offsets,
+ &offset_count,
+ sizeof(indirect_start_offsets)/sizeof(int));
+
+ /* save and restore list */
+ temp = RREG32(mmRLC_SRM_CNTL);
+ temp |= RLC_SRM_CNTL__AUTO_INCR_ADDR_MASK;
+ WREG32(mmRLC_SRM_CNTL, temp);
+
+ WREG32(mmRLC_SRM_ARAM_ADDR, 0);
+ for (i = 0; i < adev->gfx.rlc.reg_list_size_bytes >> 2; i++)
+ WREG32(mmRLC_SRM_ARAM_DATA, adev->gfx.rlc.register_restore[i]);
+
+ /* indirect list */
+ WREG32(mmRLC_GPM_SCRATCH_ADDR, adev->gfx.rlc.reg_list_format_start);
+ for (i = 0; i < adev->gfx.rlc.reg_list_format_size_bytes >> 2; i++)
+ WREG32(mmRLC_GPM_SCRATCH_DATA, register_list_format[i]);
+
+ list_size = adev->gfx.rlc.reg_list_size_bytes >> 2;
+ list_size = list_size >> 1;
+ WREG32(mmRLC_GPM_SCRATCH_ADDR, adev->gfx.rlc.reg_restore_list_size);
+ WREG32(mmRLC_GPM_SCRATCH_DATA, list_size);
+
+ /* starting offsets starts */
+ WREG32(mmRLC_GPM_SCRATCH_ADDR,
+ adev->gfx.rlc.starting_offsets_start);
+ for (i = 0; i < sizeof(indirect_start_offsets)/sizeof(int); i++)
+ WREG32(mmRLC_GPM_SCRATCH_DATA,
+ indirect_start_offsets[i]);
+
+ /* unique indices */
+ temp = mmRLC_SRM_INDEX_CNTL_ADDR_0;
+ data = mmRLC_SRM_INDEX_CNTL_DATA_0;
+ for (i = 0; i < sizeof(unique_indices) / sizeof(int); i++) {
+ amdgpu_mm_wreg(adev, temp + i, unique_indices[i] & 0x3FFFF, false);
+ amdgpu_mm_wreg(adev, data + i, unique_indices[i] >> 20, false);
+ }
+ kfree(register_list_format);
+
+ return 0;
+}
+
+static void gfx_v8_0_enable_save_restore_machine(struct amdgpu_device *adev)
+{
+ uint32_t data;
+
+ data = RREG32(mmRLC_SRM_CNTL);
+ data |= RLC_SRM_CNTL__SRM_ENABLE_MASK;
+ WREG32(mmRLC_SRM_CNTL, data);
+}
+
+static void polaris11_init_power_gating(struct amdgpu_device *adev)
+{
+ uint32_t data;
+
+ if (adev->pg_flags & (AMD_PG_SUPPORT_GFX_PG |
+ AMD_PG_SUPPORT_GFX_SMG |
+ AMD_PG_SUPPORT_GFX_DMG)) {
+ data = RREG32(mmCP_RB_WPTR_POLL_CNTL);
+ data &= ~CP_RB_WPTR_POLL_CNTL__IDLE_POLL_COUNT_MASK;
+ data |= (0x60 << CP_RB_WPTR_POLL_CNTL__IDLE_POLL_COUNT__SHIFT);
+ WREG32(mmCP_RB_WPTR_POLL_CNTL, data);
+
+ data = 0;
+ data |= (0x10 << RLC_PG_DELAY__POWER_UP_DELAY__SHIFT);
+ data |= (0x10 << RLC_PG_DELAY__POWER_DOWN_DELAY__SHIFT);
+ data |= (0x10 << RLC_PG_DELAY__CMD_PROPAGATE_DELAY__SHIFT);
+ data |= (0x10 << RLC_PG_DELAY__MEM_SLEEP_DELAY__SHIFT);
+ WREG32(mmRLC_PG_DELAY, data);
+
+ data = RREG32(mmRLC_PG_DELAY_2);
+ data &= ~RLC_PG_DELAY_2__SERDES_CMD_DELAY_MASK;
+ data |= (0x3 << RLC_PG_DELAY_2__SERDES_CMD_DELAY__SHIFT);
+ WREG32(mmRLC_PG_DELAY_2, data);
+
+ data = RREG32(mmRLC_AUTO_PG_CTRL);
+ data &= ~RLC_AUTO_PG_CTRL__GRBM_REG_SAVE_GFX_IDLE_THRESHOLD_MASK;
+ data |= (0x55f0 << RLC_AUTO_PG_CTRL__GRBM_REG_SAVE_GFX_IDLE_THRESHOLD__SHIFT);
+ WREG32(mmRLC_AUTO_PG_CTRL, data);
+ }
+}
+
+static void gfx_v8_0_init_pg(struct amdgpu_device *adev)
+{
+ if (adev->pg_flags & (AMD_PG_SUPPORT_GFX_PG |
+ AMD_PG_SUPPORT_GFX_SMG |
+ AMD_PG_SUPPORT_GFX_DMG |
+ AMD_PG_SUPPORT_CP |
+ AMD_PG_SUPPORT_GDS |
+ AMD_PG_SUPPORT_RLC_SMU_HS)) {
+ gfx_v8_0_init_csb(adev);
+ gfx_v8_0_init_save_restore_list(adev);
+ gfx_v8_0_enable_save_restore_machine(adev);
+
+ if (adev->asic_type == CHIP_POLARIS11)
+ polaris11_init_power_gating(adev);
+ }
+}
+
void gfx_v8_0_rlc_stop(struct amdgpu_device *adev)
{
u32 tmp = RREG32(mmRLC_CNTL);
@@ -2858,12 +3778,17 @@
/* disable CG */
WREG32(mmRLC_CGCG_CGLS_CTRL, 0);
+ if (adev->asic_type == CHIP_POLARIS11 ||
+ adev->asic_type == CHIP_POLARIS10)
+ WREG32(mmRLC_CGCG_CGLS_CTRL_3D, 0);
/* disable PG */
WREG32(mmRLC_PG_CNTL, 0);
gfx_v8_0_rlc_reset(adev);
+ gfx_v8_0_init_pg(adev);
+
if (!adev->pp_enabled) {
if (!adev->firmware.smu_load) {
/* legacy rlc firmware loading */
@@ -3035,9 +3960,14 @@
amdgpu_ring_write(ring, mmPA_SC_RASTER_CONFIG - PACKET3_SET_CONTEXT_REG_START);
switch (adev->asic_type) {
case CHIP_TONGA:
+ case CHIP_POLARIS10:
amdgpu_ring_write(ring, 0x16000012);
amdgpu_ring_write(ring, 0x0000002A);
break;
+ case CHIP_POLARIS11:
+ amdgpu_ring_write(ring, 0x16000012);
+ amdgpu_ring_write(ring, 0x00000000);
+ break;
case CHIP_FIJI:
amdgpu_ring_write(ring, 0x3a00161a);
amdgpu_ring_write(ring, 0x0000002e);
@@ -3122,6 +4052,8 @@
tmp = REG_SET_FIELD(tmp, CP_RB_DOORBELL_CONTROL,
DOORBELL_OFFSET, ring->doorbell_index);
tmp = REG_SET_FIELD(tmp, CP_RB_DOORBELL_CONTROL,
+ DOORBELL_HIT, 0);
+ tmp = REG_SET_FIELD(tmp, CP_RB_DOORBELL_CONTROL,
DOORBELL_EN, 1);
} else {
tmp = REG_SET_FIELD(tmp, CP_RB_DOORBELL_CONTROL,
@@ -3679,7 +4611,9 @@
if (use_doorbell) {
if ((adev->asic_type == CHIP_CARRIZO) ||
(adev->asic_type == CHIP_FIJI) ||
- (adev->asic_type == CHIP_STONEY)) {
+ (adev->asic_type == CHIP_STONEY) ||
+ (adev->asic_type == CHIP_POLARIS11) ||
+ (adev->asic_type == CHIP_POLARIS10)) {
WREG32(mmCP_MEC_DOORBELL_RANGE_LOWER,
AMDGPU_DOORBELL_KIQ << 2);
WREG32(mmCP_MEC_DOORBELL_RANGE_UPPER,
@@ -3713,7 +4647,9 @@
tmp = REG_SET_FIELD(tmp, CP_HQD_PERSISTENT_STATE, PRELOAD_SIZE, 0x53);
WREG32(mmCP_HQD_PERSISTENT_STATE, tmp);
mqd->cp_hqd_persistent_state = tmp;
- if (adev->asic_type == CHIP_STONEY) {
+ if (adev->asic_type == CHIP_STONEY ||
+ adev->asic_type == CHIP_POLARIS11 ||
+ adev->asic_type == CHIP_POLARIS10) {
tmp = RREG32(mmCP_ME1_PIPE3_INT_CNTL);
tmp = REG_SET_FIELD(tmp, CP_ME1_PIPE3_INT_CNTL, GENERIC2_INT_ENABLE, 1);
WREG32(mmCP_ME1_PIPE3_INT_CNTL, tmp);
@@ -3845,6 +4781,9 @@
gfx_v8_0_rlc_stop(adev);
gfx_v8_0_cp_compute_fini(adev);
+ amdgpu_set_powergating_state(adev,
+ AMD_IP_BLOCK_TYPE_GFX, AMD_PG_STATE_UNGATE);
+
return 0;
}
@@ -3889,185 +4828,6 @@
return -ETIMEDOUT;
}
-static void gfx_v8_0_print_status(void *handle)
-{
- int i;
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- dev_info(adev->dev, "GFX 8.x registers\n");
- dev_info(adev->dev, " GRBM_STATUS=0x%08X\n",
- RREG32(mmGRBM_STATUS));
- dev_info(adev->dev, " GRBM_STATUS2=0x%08X\n",
- RREG32(mmGRBM_STATUS2));
- dev_info(adev->dev, " GRBM_STATUS_SE0=0x%08X\n",
- RREG32(mmGRBM_STATUS_SE0));
- dev_info(adev->dev, " GRBM_STATUS_SE1=0x%08X\n",
- RREG32(mmGRBM_STATUS_SE1));
- dev_info(adev->dev, " GRBM_STATUS_SE2=0x%08X\n",
- RREG32(mmGRBM_STATUS_SE2));
- dev_info(adev->dev, " GRBM_STATUS_SE3=0x%08X\n",
- RREG32(mmGRBM_STATUS_SE3));
- dev_info(adev->dev, " CP_STAT = 0x%08x\n", RREG32(mmCP_STAT));
- dev_info(adev->dev, " CP_STALLED_STAT1 = 0x%08x\n",
- RREG32(mmCP_STALLED_STAT1));
- dev_info(adev->dev, " CP_STALLED_STAT2 = 0x%08x\n",
- RREG32(mmCP_STALLED_STAT2));
- dev_info(adev->dev, " CP_STALLED_STAT3 = 0x%08x\n",
- RREG32(mmCP_STALLED_STAT3));
- dev_info(adev->dev, " CP_CPF_BUSY_STAT = 0x%08x\n",
- RREG32(mmCP_CPF_BUSY_STAT));
- dev_info(adev->dev, " CP_CPF_STALLED_STAT1 = 0x%08x\n",
- RREG32(mmCP_CPF_STALLED_STAT1));
- dev_info(adev->dev, " CP_CPF_STATUS = 0x%08x\n", RREG32(mmCP_CPF_STATUS));
- dev_info(adev->dev, " CP_CPC_BUSY_STAT = 0x%08x\n", RREG32(mmCP_CPC_BUSY_STAT));
- dev_info(adev->dev, " CP_CPC_STALLED_STAT1 = 0x%08x\n",
- RREG32(mmCP_CPC_STALLED_STAT1));
- dev_info(adev->dev, " CP_CPC_STATUS = 0x%08x\n", RREG32(mmCP_CPC_STATUS));
-
- for (i = 0; i < 32; i++) {
- dev_info(adev->dev, " GB_TILE_MODE%d=0x%08X\n",
- i, RREG32(mmGB_TILE_MODE0 + (i * 4)));
- }
- for (i = 0; i < 16; i++) {
- dev_info(adev->dev, " GB_MACROTILE_MODE%d=0x%08X\n",
- i, RREG32(mmGB_MACROTILE_MODE0 + (i * 4)));
- }
- for (i = 0; i < adev->gfx.config.max_shader_engines; i++) {
- dev_info(adev->dev, " se: %d\n", i);
- gfx_v8_0_select_se_sh(adev, i, 0xffffffff);
- dev_info(adev->dev, " PA_SC_RASTER_CONFIG=0x%08X\n",
- RREG32(mmPA_SC_RASTER_CONFIG));
- dev_info(adev->dev, " PA_SC_RASTER_CONFIG_1=0x%08X\n",
- RREG32(mmPA_SC_RASTER_CONFIG_1));
- }
- gfx_v8_0_select_se_sh(adev, 0xffffffff, 0xffffffff);
-
- dev_info(adev->dev, " GB_ADDR_CONFIG=0x%08X\n",
- RREG32(mmGB_ADDR_CONFIG));
- dev_info(adev->dev, " HDP_ADDR_CONFIG=0x%08X\n",
- RREG32(mmHDP_ADDR_CONFIG));
- dev_info(adev->dev, " DMIF_ADDR_CALC=0x%08X\n",
- RREG32(mmDMIF_ADDR_CALC));
-
- dev_info(adev->dev, " CP_MEQ_THRESHOLDS=0x%08X\n",
- RREG32(mmCP_MEQ_THRESHOLDS));
- dev_info(adev->dev, " SX_DEBUG_1=0x%08X\n",
- RREG32(mmSX_DEBUG_1));
- dev_info(adev->dev, " TA_CNTL_AUX=0x%08X\n",
- RREG32(mmTA_CNTL_AUX));
- dev_info(adev->dev, " SPI_CONFIG_CNTL=0x%08X\n",
- RREG32(mmSPI_CONFIG_CNTL));
- dev_info(adev->dev, " SQ_CONFIG=0x%08X\n",
- RREG32(mmSQ_CONFIG));
- dev_info(adev->dev, " DB_DEBUG=0x%08X\n",
- RREG32(mmDB_DEBUG));
- dev_info(adev->dev, " DB_DEBUG2=0x%08X\n",
- RREG32(mmDB_DEBUG2));
- dev_info(adev->dev, " DB_DEBUG3=0x%08X\n",
- RREG32(mmDB_DEBUG3));
- dev_info(adev->dev, " CB_HW_CONTROL=0x%08X\n",
- RREG32(mmCB_HW_CONTROL));
- dev_info(adev->dev, " SPI_CONFIG_CNTL_1=0x%08X\n",
- RREG32(mmSPI_CONFIG_CNTL_1));
- dev_info(adev->dev, " PA_SC_FIFO_SIZE=0x%08X\n",
- RREG32(mmPA_SC_FIFO_SIZE));
- dev_info(adev->dev, " VGT_NUM_INSTANCES=0x%08X\n",
- RREG32(mmVGT_NUM_INSTANCES));
- dev_info(adev->dev, " CP_PERFMON_CNTL=0x%08X\n",
- RREG32(mmCP_PERFMON_CNTL));
- dev_info(adev->dev, " PA_SC_FORCE_EOV_MAX_CNTS=0x%08X\n",
- RREG32(mmPA_SC_FORCE_EOV_MAX_CNTS));
- dev_info(adev->dev, " VGT_CACHE_INVALIDATION=0x%08X\n",
- RREG32(mmVGT_CACHE_INVALIDATION));
- dev_info(adev->dev, " VGT_GS_VERTEX_REUSE=0x%08X\n",
- RREG32(mmVGT_GS_VERTEX_REUSE));
- dev_info(adev->dev, " PA_SC_LINE_STIPPLE_STATE=0x%08X\n",
- RREG32(mmPA_SC_LINE_STIPPLE_STATE));
- dev_info(adev->dev, " PA_CL_ENHANCE=0x%08X\n",
- RREG32(mmPA_CL_ENHANCE));
- dev_info(adev->dev, " PA_SC_ENHANCE=0x%08X\n",
- RREG32(mmPA_SC_ENHANCE));
-
- dev_info(adev->dev, " CP_ME_CNTL=0x%08X\n",
- RREG32(mmCP_ME_CNTL));
- dev_info(adev->dev, " CP_MAX_CONTEXT=0x%08X\n",
- RREG32(mmCP_MAX_CONTEXT));
- dev_info(adev->dev, " CP_ENDIAN_SWAP=0x%08X\n",
- RREG32(mmCP_ENDIAN_SWAP));
- dev_info(adev->dev, " CP_DEVICE_ID=0x%08X\n",
- RREG32(mmCP_DEVICE_ID));
-
- dev_info(adev->dev, " CP_SEM_WAIT_TIMER=0x%08X\n",
- RREG32(mmCP_SEM_WAIT_TIMER));
-
- dev_info(adev->dev, " CP_RB_WPTR_DELAY=0x%08X\n",
- RREG32(mmCP_RB_WPTR_DELAY));
- dev_info(adev->dev, " CP_RB_VMID=0x%08X\n",
- RREG32(mmCP_RB_VMID));
- dev_info(adev->dev, " CP_RB0_CNTL=0x%08X\n",
- RREG32(mmCP_RB0_CNTL));
- dev_info(adev->dev, " CP_RB0_WPTR=0x%08X\n",
- RREG32(mmCP_RB0_WPTR));
- dev_info(adev->dev, " CP_RB0_RPTR_ADDR=0x%08X\n",
- RREG32(mmCP_RB0_RPTR_ADDR));
- dev_info(adev->dev, " CP_RB0_RPTR_ADDR_HI=0x%08X\n",
- RREG32(mmCP_RB0_RPTR_ADDR_HI));
- dev_info(adev->dev, " CP_RB0_CNTL=0x%08X\n",
- RREG32(mmCP_RB0_CNTL));
- dev_info(adev->dev, " CP_RB0_BASE=0x%08X\n",
- RREG32(mmCP_RB0_BASE));
- dev_info(adev->dev, " CP_RB0_BASE_HI=0x%08X\n",
- RREG32(mmCP_RB0_BASE_HI));
- dev_info(adev->dev, " CP_MEC_CNTL=0x%08X\n",
- RREG32(mmCP_MEC_CNTL));
- dev_info(adev->dev, " CP_CPF_DEBUG=0x%08X\n",
- RREG32(mmCP_CPF_DEBUG));
-
- dev_info(adev->dev, " SCRATCH_ADDR=0x%08X\n",
- RREG32(mmSCRATCH_ADDR));
- dev_info(adev->dev, " SCRATCH_UMSK=0x%08X\n",
- RREG32(mmSCRATCH_UMSK));
-
- dev_info(adev->dev, " CP_INT_CNTL_RING0=0x%08X\n",
- RREG32(mmCP_INT_CNTL_RING0));
- dev_info(adev->dev, " RLC_LB_CNTL=0x%08X\n",
- RREG32(mmRLC_LB_CNTL));
- dev_info(adev->dev, " RLC_CNTL=0x%08X\n",
- RREG32(mmRLC_CNTL));
- dev_info(adev->dev, " RLC_CGCG_CGLS_CTRL=0x%08X\n",
- RREG32(mmRLC_CGCG_CGLS_CTRL));
- dev_info(adev->dev, " RLC_LB_CNTR_INIT=0x%08X\n",
- RREG32(mmRLC_LB_CNTR_INIT));
- dev_info(adev->dev, " RLC_LB_CNTR_MAX=0x%08X\n",
- RREG32(mmRLC_LB_CNTR_MAX));
- dev_info(adev->dev, " RLC_LB_INIT_CU_MASK=0x%08X\n",
- RREG32(mmRLC_LB_INIT_CU_MASK));
- dev_info(adev->dev, " RLC_LB_PARAMS=0x%08X\n",
- RREG32(mmRLC_LB_PARAMS));
- dev_info(adev->dev, " RLC_LB_CNTL=0x%08X\n",
- RREG32(mmRLC_LB_CNTL));
- dev_info(adev->dev, " RLC_MC_CNTL=0x%08X\n",
- RREG32(mmRLC_MC_CNTL));
- dev_info(adev->dev, " RLC_UCODE_CNTL=0x%08X\n",
- RREG32(mmRLC_UCODE_CNTL));
-
- mutex_lock(&adev->srbm_mutex);
- for (i = 0; i < 16; i++) {
- vi_srbm_select(adev, 0, 0, 0, i);
- dev_info(adev->dev, " VM %d:\n", i);
- dev_info(adev->dev, " SH_MEM_CONFIG=0x%08X\n",
- RREG32(mmSH_MEM_CONFIG));
- dev_info(adev->dev, " SH_MEM_APE1_BASE=0x%08X\n",
- RREG32(mmSH_MEM_APE1_BASE));
- dev_info(adev->dev, " SH_MEM_APE1_LIMIT=0x%08X\n",
- RREG32(mmSH_MEM_APE1_LIMIT));
- dev_info(adev->dev, " SH_MEM_BASES=0x%08X\n",
- RREG32(mmSH_MEM_BASES));
- }
- vi_srbm_select(adev, 0, 0, 0, 0);
- mutex_unlock(&adev->srbm_mutex);
-}
-
static int gfx_v8_0_soft_reset(void *handle)
{
u32 grbm_soft_reset = 0, srbm_soft_reset = 0;
@@ -4108,7 +4868,6 @@
SRBM_SOFT_RESET, SOFT_RESET_GRBM, 1);
if (grbm_soft_reset || srbm_soft_reset) {
- gfx_v8_0_print_status((void *)adev);
/* stop the rlc */
gfx_v8_0_rlc_stop(adev);
@@ -4168,7 +4927,6 @@
/* Wait a little for things to settle down */
udelay(50);
- gfx_v8_0_print_status((void *)adev);
}
return 0;
}
@@ -4250,6 +5008,7 @@
gfx_v8_0_set_ring_funcs(adev);
gfx_v8_0_set_irq_funcs(adev);
gfx_v8_0_set_gds_init(adev);
+ gfx_v8_0_set_rlc_funcs(adev);
return 0;
}
@@ -4272,17 +5031,109 @@
if (r)
return r;
+ amdgpu_set_powergating_state(adev,
+ AMD_IP_BLOCK_TYPE_GFX, AMD_PG_STATE_GATE);
+
return 0;
}
+static void polaris11_enable_gfx_static_mg_power_gating(struct amdgpu_device *adev,
+ bool enable)
+{
+ uint32_t data, temp;
+
+ /* Send msg to SMU via Powerplay */
+ amdgpu_set_powergating_state(adev,
+ AMD_IP_BLOCK_TYPE_SMC,
+ enable ? AMD_PG_STATE_GATE : AMD_PG_STATE_UNGATE);
+
+ if (enable) {
+ /* Enable static MGPG */
+ temp = data = RREG32(mmRLC_PG_CNTL);
+ data |= RLC_PG_CNTL__STATIC_PER_CU_PG_ENABLE_MASK;
+
+ if (temp != data)
+ WREG32(mmRLC_PG_CNTL, data);
+ } else {
+ temp = data = RREG32(mmRLC_PG_CNTL);
+ data &= ~RLC_PG_CNTL__STATIC_PER_CU_PG_ENABLE_MASK;
+
+ if (temp != data)
+ WREG32(mmRLC_PG_CNTL, data);
+ }
+}
+
+static void polaris11_enable_gfx_dynamic_mg_power_gating(struct amdgpu_device *adev,
+ bool enable)
+{
+ uint32_t data, temp;
+
+ if (enable) {
+ /* Enable dynamic MGPG */
+ temp = data = RREG32(mmRLC_PG_CNTL);
+ data |= RLC_PG_CNTL__DYN_PER_CU_PG_ENABLE_MASK;
+
+ if (temp != data)
+ WREG32(mmRLC_PG_CNTL, data);
+ } else {
+ temp = data = RREG32(mmRLC_PG_CNTL);
+ data &= ~RLC_PG_CNTL__DYN_PER_CU_PG_ENABLE_MASK;
+
+ if (temp != data)
+ WREG32(mmRLC_PG_CNTL, data);
+ }
+}
+
+static void polaris11_enable_gfx_quick_mg_power_gating(struct amdgpu_device *adev,
+ bool enable)
+{
+ uint32_t data, temp;
+
+ if (enable) {
+ /* Enable quick PG */
+ temp = data = RREG32(mmRLC_PG_CNTL);
+ data |= 0x100000;
+
+ if (temp != data)
+ WREG32(mmRLC_PG_CNTL, data);
+ } else {
+ temp = data = RREG32(mmRLC_PG_CNTL);
+ data &= ~0x100000;
+
+ if (temp != data)
+ WREG32(mmRLC_PG_CNTL, data);
+ }
+}
+
static int gfx_v8_0_set_powergating_state(void *handle,
enum amd_powergating_state state)
{
+ struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+
+ if (!(adev->pg_flags & AMD_PG_SUPPORT_GFX_PG))
+ return 0;
+
+ switch (adev->asic_type) {
+ case CHIP_POLARIS11:
+ if (adev->pg_flags & AMD_PG_SUPPORT_GFX_SMG)
+ polaris11_enable_gfx_static_mg_power_gating(adev,
+ state == AMD_PG_STATE_GATE ? true : false);
+ else if (adev->pg_flags & AMD_PG_SUPPORT_GFX_DMG)
+ polaris11_enable_gfx_dynamic_mg_power_gating(adev,
+ state == AMD_PG_STATE_GATE ? true : false);
+ else
+ polaris11_enable_gfx_quick_mg_power_gating(adev,
+ state == AMD_PG_STATE_GATE ? true : false);
+ break;
+ default:
+ break;
+ }
+
return 0;
}
-static void fiji_send_serdes_cmd(struct amdgpu_device *adev,
- uint32_t reg_addr, uint32_t cmd)
+static void gfx_v8_0_send_serdes_cmd(struct amdgpu_device *adev,
+ uint32_t reg_addr, uint32_t cmd)
{
uint32_t data;
@@ -4292,7 +5143,8 @@
WREG32(mmRLC_SERDES_WR_NONCU_MASTER_MASK, 0xffffffff);
data = RREG32(mmRLC_SERDES_WR_CTRL);
- data &= ~(RLC_SERDES_WR_CTRL__WRITE_COMMAND_MASK |
+ if (adev->asic_type == CHIP_STONEY)
+ data &= ~(RLC_SERDES_WR_CTRL__WRITE_COMMAND_MASK |
RLC_SERDES_WR_CTRL__READ_COMMAND_MASK |
RLC_SERDES_WR_CTRL__P1_SELECT_MASK |
RLC_SERDES_WR_CTRL__P2_SELECT_MASK |
@@ -4300,42 +5152,218 @@
RLC_SERDES_WR_CTRL__POWER_DOWN_MASK |
RLC_SERDES_WR_CTRL__POWER_UP_MASK |
RLC_SERDES_WR_CTRL__SHORT_FORMAT_MASK |
- RLC_SERDES_WR_CTRL__BPM_DATA_MASK |
- RLC_SERDES_WR_CTRL__REG_ADDR_MASK |
RLC_SERDES_WR_CTRL__SRBM_OVERRIDE_MASK);
+ else
+ data &= ~(RLC_SERDES_WR_CTRL__WRITE_COMMAND_MASK |
+ RLC_SERDES_WR_CTRL__READ_COMMAND_MASK |
+ RLC_SERDES_WR_CTRL__P1_SELECT_MASK |
+ RLC_SERDES_WR_CTRL__P2_SELECT_MASK |
+ RLC_SERDES_WR_CTRL__RDDATA_RESET_MASK |
+ RLC_SERDES_WR_CTRL__POWER_DOWN_MASK |
+ RLC_SERDES_WR_CTRL__POWER_UP_MASK |
+ RLC_SERDES_WR_CTRL__SHORT_FORMAT_MASK |
+ RLC_SERDES_WR_CTRL__BPM_DATA_MASK |
+ RLC_SERDES_WR_CTRL__REG_ADDR_MASK |
+ RLC_SERDES_WR_CTRL__SRBM_OVERRIDE_MASK);
data |= (RLC_SERDES_WR_CTRL__RSVD_BPM_ADDR_MASK |
- (cmd << RLC_SERDES_WR_CTRL__BPM_DATA__SHIFT) |
- (reg_addr << RLC_SERDES_WR_CTRL__REG_ADDR__SHIFT) |
- (0xff << RLC_SERDES_WR_CTRL__BPM_ADDR__SHIFT));
+ (cmd << RLC_SERDES_WR_CTRL__BPM_DATA__SHIFT) |
+ (reg_addr << RLC_SERDES_WR_CTRL__REG_ADDR__SHIFT) |
+ (0xff << RLC_SERDES_WR_CTRL__BPM_ADDR__SHIFT));
WREG32(mmRLC_SERDES_WR_CTRL, data);
}
-static void fiji_update_medium_grain_clock_gating(struct amdgpu_device *adev,
- bool enable)
+#define MSG_ENTER_RLC_SAFE_MODE 1
+#define MSG_EXIT_RLC_SAFE_MODE 0
+
+#define RLC_GPR_REG2__REQ_MASK 0x00000001
+#define RLC_GPR_REG2__MESSAGE__SHIFT 0x00000001
+#define RLC_GPR_REG2__MESSAGE_MASK 0x0000001e
+
+static void cz_enter_rlc_safe_mode(struct amdgpu_device *adev)
+{
+ u32 data = 0;
+ unsigned i;
+
+ data = RREG32(mmRLC_CNTL);
+ if ((data & RLC_CNTL__RLC_ENABLE_F32_MASK) == 0)
+ return;
+
+ if ((adev->cg_flags & (AMD_CG_SUPPORT_GFX_CGCG | AMD_CG_SUPPORT_GFX_MGCG)) ||
+ (adev->pg_flags & (AMD_PG_SUPPORT_GFX_PG | AMD_PG_SUPPORT_GFX_SMG |
+ AMD_PG_SUPPORT_GFX_DMG))) {
+ data |= RLC_GPR_REG2__REQ_MASK;
+ data &= ~RLC_GPR_REG2__MESSAGE_MASK;
+ data |= (MSG_ENTER_RLC_SAFE_MODE << RLC_GPR_REG2__MESSAGE__SHIFT);
+ WREG32(mmRLC_GPR_REG2, data);
+
+ for (i = 0; i < adev->usec_timeout; i++) {
+ if ((RREG32(mmRLC_GPM_STAT) &
+ (RLC_GPM_STAT__GFX_CLOCK_STATUS_MASK |
+ RLC_GPM_STAT__GFX_POWER_STATUS_MASK)) ==
+ (RLC_GPM_STAT__GFX_CLOCK_STATUS_MASK |
+ RLC_GPM_STAT__GFX_POWER_STATUS_MASK))
+ break;
+ udelay(1);
+ }
+
+ for (i = 0; i < adev->usec_timeout; i++) {
+ if ((RREG32(mmRLC_GPR_REG2) & RLC_GPR_REG2__REQ_MASK) == 0)
+ break;
+ udelay(1);
+ }
+ adev->gfx.rlc.in_safe_mode = true;
+ }
+}
+
+static void cz_exit_rlc_safe_mode(struct amdgpu_device *adev)
+{
+ u32 data;
+ unsigned i;
+
+ data = RREG32(mmRLC_CNTL);
+ if ((data & RLC_CNTL__RLC_ENABLE_F32_MASK) == 0)
+ return;
+
+ if ((adev->cg_flags & (AMD_CG_SUPPORT_GFX_CGCG | AMD_CG_SUPPORT_GFX_MGCG)) ||
+ (adev->pg_flags & (AMD_PG_SUPPORT_GFX_PG | AMD_PG_SUPPORT_GFX_SMG |
+ AMD_PG_SUPPORT_GFX_DMG))) {
+ data |= RLC_GPR_REG2__REQ_MASK;
+ data &= ~RLC_GPR_REG2__MESSAGE_MASK;
+ data |= (MSG_EXIT_RLC_SAFE_MODE << RLC_GPR_REG2__MESSAGE__SHIFT);
+ WREG32(mmRLC_GPR_REG2, data);
+ adev->gfx.rlc.in_safe_mode = false;
+ }
+
+ for (i = 0; i < adev->usec_timeout; i++) {
+ if ((RREG32(mmRLC_GPR_REG2) & RLC_GPR_REG2__REQ_MASK) == 0)
+ break;
+ udelay(1);
+ }
+}
+
+static void iceland_enter_rlc_safe_mode(struct amdgpu_device *adev)
+{
+ u32 data;
+ unsigned i;
+
+ data = RREG32(mmRLC_CNTL);
+ if (!(data & RLC_CNTL__RLC_ENABLE_F32_MASK))
+ return;
+
+ if (adev->cg_flags & (AMD_CG_SUPPORT_GFX_CGCG | AMD_CG_SUPPORT_GFX_MGCG)) {
+ data |= RLC_SAFE_MODE__CMD_MASK;
+ data &= ~RLC_SAFE_MODE__MESSAGE_MASK;
+ data |= (1 << RLC_SAFE_MODE__MESSAGE__SHIFT);
+ WREG32(mmRLC_SAFE_MODE, data);
+
+ for (i = 0; i < adev->usec_timeout; i++) {
+ if ((RREG32(mmRLC_GPM_STAT) &
+ (RLC_GPM_STAT__GFX_CLOCK_STATUS_MASK |
+ RLC_GPM_STAT__GFX_POWER_STATUS_MASK)) ==
+ (RLC_GPM_STAT__GFX_CLOCK_STATUS_MASK |
+ RLC_GPM_STAT__GFX_POWER_STATUS_MASK))
+ break;
+ udelay(1);
+ }
+
+ for (i = 0; i < adev->usec_timeout; i++) {
+ if ((RREG32(mmRLC_SAFE_MODE) & RLC_SAFE_MODE__CMD_MASK) == 0)
+ break;
+ udelay(1);
+ }
+ adev->gfx.rlc.in_safe_mode = true;
+ }
+}
+
+static void iceland_exit_rlc_safe_mode(struct amdgpu_device *adev)
+{
+ u32 data = 0;
+ unsigned i;
+
+ data = RREG32(mmRLC_CNTL);
+ if (!(data & RLC_CNTL__RLC_ENABLE_F32_MASK))
+ return;
+
+ if (adev->cg_flags & (AMD_CG_SUPPORT_GFX_CGCG | AMD_CG_SUPPORT_GFX_MGCG)) {
+ if (adev->gfx.rlc.in_safe_mode) {
+ data |= RLC_SAFE_MODE__CMD_MASK;
+ data &= ~RLC_SAFE_MODE__MESSAGE_MASK;
+ WREG32(mmRLC_SAFE_MODE, data);
+ adev->gfx.rlc.in_safe_mode = false;
+ }
+ }
+
+ for (i = 0; i < adev->usec_timeout; i++) {
+ if ((RREG32(mmRLC_SAFE_MODE) & RLC_SAFE_MODE__CMD_MASK) == 0)
+ break;
+ udelay(1);
+ }
+}
+
+static void gfx_v8_0_nop_enter_rlc_safe_mode(struct amdgpu_device *adev)
+{
+ adev->gfx.rlc.in_safe_mode = true;
+}
+
+static void gfx_v8_0_nop_exit_rlc_safe_mode(struct amdgpu_device *adev)
+{
+ adev->gfx.rlc.in_safe_mode = false;
+}
+
+static const struct amdgpu_rlc_funcs cz_rlc_funcs = {
+ .enter_safe_mode = cz_enter_rlc_safe_mode,
+ .exit_safe_mode = cz_exit_rlc_safe_mode
+};
+
+static const struct amdgpu_rlc_funcs iceland_rlc_funcs = {
+ .enter_safe_mode = iceland_enter_rlc_safe_mode,
+ .exit_safe_mode = iceland_exit_rlc_safe_mode
+};
+
+static const struct amdgpu_rlc_funcs gfx_v8_0_nop_rlc_funcs = {
+ .enter_safe_mode = gfx_v8_0_nop_enter_rlc_safe_mode,
+ .exit_safe_mode = gfx_v8_0_nop_exit_rlc_safe_mode
+};
+
+static void gfx_v8_0_update_medium_grain_clock_gating(struct amdgpu_device *adev,
+ bool enable)
{
uint32_t temp, data;
- /* It is disabled by HW by default */
- if (enable) {
- /* 1 - RLC memory Light sleep */
- temp = data = RREG32(mmRLC_MEM_SLP_CNTL);
- data |= RLC_MEM_SLP_CNTL__RLC_MEM_LS_EN_MASK;
- if (temp != data)
- WREG32(mmRLC_MEM_SLP_CNTL, data);
+ adev->gfx.rlc.funcs->enter_safe_mode(adev);
- /* 2 - CP memory Light sleep */
- temp = data = RREG32(mmCP_MEM_SLP_CNTL);
- data |= CP_MEM_SLP_CNTL__CP_MEM_LS_EN_MASK;
- if (temp != data)
- WREG32(mmCP_MEM_SLP_CNTL, data);
+ /* It is disabled by HW by default */
+ if (enable && (adev->cg_flags & AMD_CG_SUPPORT_GFX_MGCG)) {
+ if (adev->cg_flags & AMD_CG_SUPPORT_GFX_MGLS) {
+ if (adev->cg_flags & AMD_CG_SUPPORT_GFX_RLC_LS) {
+ /* 1 - RLC memory Light sleep */
+ temp = data = RREG32(mmRLC_MEM_SLP_CNTL);
+ data |= RLC_MEM_SLP_CNTL__RLC_MEM_LS_EN_MASK;
+ if (temp != data)
+ WREG32(mmRLC_MEM_SLP_CNTL, data);
+ }
+
+ if (adev->cg_flags & AMD_CG_SUPPORT_GFX_CP_LS) {
+ /* 2 - CP memory Light sleep */
+ temp = data = RREG32(mmCP_MEM_SLP_CNTL);
+ data |= CP_MEM_SLP_CNTL__CP_MEM_LS_EN_MASK;
+ if (temp != data)
+ WREG32(mmCP_MEM_SLP_CNTL, data);
+ }
+ }
/* 3 - RLC_CGTT_MGCG_OVERRIDE */
temp = data = RREG32(mmRLC_CGTT_MGCG_OVERRIDE);
- data &= ~(RLC_CGTT_MGCG_OVERRIDE__CPF_MASK |
- RLC_CGTT_MGCG_OVERRIDE__RLC_MASK |
- RLC_CGTT_MGCG_OVERRIDE__MGCG_MASK |
- RLC_CGTT_MGCG_OVERRIDE__GRBM_MASK);
+ if (adev->flags & AMD_IS_APU)
+ data &= ~(RLC_CGTT_MGCG_OVERRIDE__CPF_MASK |
+ RLC_CGTT_MGCG_OVERRIDE__RLC_MASK |
+ RLC_CGTT_MGCG_OVERRIDE__MGCG_MASK);
+ else
+ data &= ~(RLC_CGTT_MGCG_OVERRIDE__CPF_MASK |
+ RLC_CGTT_MGCG_OVERRIDE__RLC_MASK |
+ RLC_CGTT_MGCG_OVERRIDE__MGCG_MASK |
+ RLC_CGTT_MGCG_OVERRIDE__GRBM_MASK);
if (temp != data)
WREG32(mmRLC_CGTT_MGCG_OVERRIDE, data);
@@ -4344,19 +5372,23 @@
gfx_v8_0_wait_for_rlc_serdes(adev);
/* 5 - clear mgcg override */
- fiji_send_serdes_cmd(adev, BPM_REG_MGCG_OVERRIDE, CLE_BPM_SERDES_CMD);
+ gfx_v8_0_send_serdes_cmd(adev, BPM_REG_MGCG_OVERRIDE, CLE_BPM_SERDES_CMD);
- /* 6 - Enable CGTS(Tree Shade) MGCG /MGLS */
- temp = data = RREG32(mmCGTS_SM_CTRL_REG);
- data &= ~(CGTS_SM_CTRL_REG__SM_MODE_MASK);
- data |= (0x2 << CGTS_SM_CTRL_REG__SM_MODE__SHIFT);
- data |= CGTS_SM_CTRL_REG__SM_MODE_ENABLE_MASK;
- data &= ~CGTS_SM_CTRL_REG__OVERRIDE_MASK;
- data &= ~CGTS_SM_CTRL_REG__LS_OVERRIDE_MASK;
- data |= CGTS_SM_CTRL_REG__ON_MONITOR_ADD_EN_MASK;
- data |= (0x96 << CGTS_SM_CTRL_REG__ON_MONITOR_ADD__SHIFT);
- if (temp != data)
- WREG32(mmCGTS_SM_CTRL_REG, data);
+ if (adev->cg_flags & AMD_CG_SUPPORT_GFX_CGTS) {
+ /* 6 - Enable CGTS(Tree Shade) MGCG /MGLS */
+ temp = data = RREG32(mmCGTS_SM_CTRL_REG);
+ data &= ~(CGTS_SM_CTRL_REG__SM_MODE_MASK);
+ data |= (0x2 << CGTS_SM_CTRL_REG__SM_MODE__SHIFT);
+ data |= CGTS_SM_CTRL_REG__SM_MODE_ENABLE_MASK;
+ data &= ~CGTS_SM_CTRL_REG__OVERRIDE_MASK;
+ if ((adev->cg_flags & AMD_CG_SUPPORT_GFX_MGLS) &&
+ (adev->cg_flags & AMD_CG_SUPPORT_GFX_CGTS_LS))
+ data &= ~CGTS_SM_CTRL_REG__LS_OVERRIDE_MASK;
+ data |= CGTS_SM_CTRL_REG__ON_MONITOR_ADD_EN_MASK;
+ data |= (0x96 << CGTS_SM_CTRL_REG__ON_MONITOR_ADD__SHIFT);
+ if (temp != data)
+ WREG32(mmCGTS_SM_CTRL_REG, data);
+ }
udelay(50);
/* 7 - wait for RLC_SERDES_CU_MASTER & RLC_SERDES_NONCU_MASTER idle */
@@ -4396,23 +5428,27 @@
gfx_v8_0_wait_for_rlc_serdes(adev);
/* 6 - set mgcg override */
- fiji_send_serdes_cmd(adev, BPM_REG_MGCG_OVERRIDE, SET_BPM_SERDES_CMD);
+ gfx_v8_0_send_serdes_cmd(adev, BPM_REG_MGCG_OVERRIDE, SET_BPM_SERDES_CMD);
udelay(50);
/* 7- wait for RLC_SERDES_CU_MASTER & RLC_SERDES_NONCU_MASTER idle */
gfx_v8_0_wait_for_rlc_serdes(adev);
}
+
+ adev->gfx.rlc.funcs->exit_safe_mode(adev);
}
-static void fiji_update_coarse_grain_clock_gating(struct amdgpu_device *adev,
- bool enable)
+static void gfx_v8_0_update_coarse_grain_clock_gating(struct amdgpu_device *adev,
+ bool enable)
{
uint32_t temp, temp1, data, data1;
temp = data = RREG32(mmRLC_CGCG_CGLS_CTRL);
- if (enable) {
+ adev->gfx.rlc.funcs->enter_safe_mode(adev);
+
+ if (enable && (adev->cg_flags & AMD_CG_SUPPORT_GFX_CGCG)) {
/* 1 enable cntx_empty_int_enable/cntx_busy_int_enable/
* Cmp_busy/GFX_Idle interrupts
*/
@@ -4427,25 +5463,29 @@
gfx_v8_0_wait_for_rlc_serdes(adev);
/* 3 - clear cgcg override */
- fiji_send_serdes_cmd(adev, BPM_REG_CGCG_OVERRIDE, CLE_BPM_SERDES_CMD);
+ gfx_v8_0_send_serdes_cmd(adev, BPM_REG_CGCG_OVERRIDE, CLE_BPM_SERDES_CMD);
/* wait for RLC_SERDES_CU_MASTER & RLC_SERDES_NONCU_MASTER idle */
gfx_v8_0_wait_for_rlc_serdes(adev);
/* 4 - write cmd to set CGLS */
- fiji_send_serdes_cmd(adev, BPM_REG_CGLS_EN, SET_BPM_SERDES_CMD);
+ gfx_v8_0_send_serdes_cmd(adev, BPM_REG_CGLS_EN, SET_BPM_SERDES_CMD);
/* 5 - enable cgcg */
data |= RLC_CGCG_CGLS_CTRL__CGCG_EN_MASK;
- /* enable cgls*/
- data |= RLC_CGCG_CGLS_CTRL__CGLS_EN_MASK;
+ if (adev->cg_flags & AMD_CG_SUPPORT_GFX_CGLS) {
+ /* enable cgls*/
+ data |= RLC_CGCG_CGLS_CTRL__CGLS_EN_MASK;
- temp1 = data1 = RREG32(mmRLC_CGTT_MGCG_OVERRIDE);
- data1 &= ~RLC_CGTT_MGCG_OVERRIDE__CGLS_MASK;
+ temp1 = data1 = RREG32(mmRLC_CGTT_MGCG_OVERRIDE);
+ data1 &= ~RLC_CGTT_MGCG_OVERRIDE__CGLS_MASK;
- if (temp1 != data1)
- WREG32(mmRLC_CGTT_MGCG_OVERRIDE, data1);
+ if (temp1 != data1)
+ WREG32(mmRLC_CGTT_MGCG_OVERRIDE, data1);
+ } else {
+ data &= ~RLC_CGCG_CGLS_CTRL__CGLS_EN_MASK;
+ }
if (temp != data)
WREG32(mmRLC_CGCG_CGLS_CTRL, data);
@@ -4470,36 +5510,38 @@
gfx_v8_0_wait_for_rlc_serdes(adev);
/* write cmd to Set CGCG Overrride */
- fiji_send_serdes_cmd(adev, BPM_REG_CGCG_OVERRIDE, SET_BPM_SERDES_CMD);
+ gfx_v8_0_send_serdes_cmd(adev, BPM_REG_CGCG_OVERRIDE, SET_BPM_SERDES_CMD);
/* wait for RLC_SERDES_CU_MASTER & RLC_SERDES_NONCU_MASTER idle */
gfx_v8_0_wait_for_rlc_serdes(adev);
/* write cmd to Clear CGLS */
- fiji_send_serdes_cmd(adev, BPM_REG_CGLS_EN, CLE_BPM_SERDES_CMD);
+ gfx_v8_0_send_serdes_cmd(adev, BPM_REG_CGLS_EN, CLE_BPM_SERDES_CMD);
/* disable cgcg, cgls should be disabled too. */
data &= ~(RLC_CGCG_CGLS_CTRL__CGCG_EN_MASK |
- RLC_CGCG_CGLS_CTRL__CGLS_EN_MASK);
+ RLC_CGCG_CGLS_CTRL__CGLS_EN_MASK);
if (temp != data)
WREG32(mmRLC_CGCG_CGLS_CTRL, data);
}
+
+ adev->gfx.rlc.funcs->exit_safe_mode(adev);
}
-static int fiji_update_gfx_clock_gating(struct amdgpu_device *adev,
- bool enable)
+static int gfx_v8_0_update_gfx_clock_gating(struct amdgpu_device *adev,
+ bool enable)
{
if (enable) {
/* CGCG/CGLS should be enabled after MGCG/MGLS/TS(CG/LS)
* === MGCG + MGLS + TS(CG/LS) ===
*/
- fiji_update_medium_grain_clock_gating(adev, enable);
- fiji_update_coarse_grain_clock_gating(adev, enable);
+ gfx_v8_0_update_medium_grain_clock_gating(adev, enable);
+ gfx_v8_0_update_coarse_grain_clock_gating(adev, enable);
} else {
/* CGCG/CGLS should be disabled before MGCG/MGLS/TS(CG/LS)
* === CGCG + CGLS ===
*/
- fiji_update_coarse_grain_clock_gating(adev, enable);
- fiji_update_medium_grain_clock_gating(adev, enable);
+ gfx_v8_0_update_coarse_grain_clock_gating(adev, enable);
+ gfx_v8_0_update_medium_grain_clock_gating(adev, enable);
}
return 0;
}
@@ -4511,8 +5553,10 @@
switch (adev->asic_type) {
case CHIP_FIJI:
- fiji_update_gfx_clock_gating(adev,
- state == AMD_CG_STATE_GATE ? true : false);
+ case CHIP_CARRIZO:
+ case CHIP_STONEY:
+ gfx_v8_0_update_gfx_clock_gating(adev,
+ state == AMD_CG_STATE_GATE ? true : false);
break;
default:
break;
@@ -4602,17 +5646,13 @@
}
static void gfx_v8_0_ring_emit_ib_gfx(struct amdgpu_ring *ring,
- struct amdgpu_ib *ib)
+ struct amdgpu_ib *ib,
+ unsigned vm_id, bool ctx_switch)
{
- bool need_ctx_switch = ring->current_ctx != ib->ctx;
u32 header, control = 0;
u32 next_rptr = ring->wptr + 5;
- /* drop the CE preamble IB for the same context */
- if ((ib->flags & AMDGPU_IB_FLAG_PREAMBLE) && !need_ctx_switch)
- return;
-
- if (need_ctx_switch)
+ if (ctx_switch)
next_rptr += 2;
next_rptr += 4;
@@ -4623,7 +5663,7 @@
amdgpu_ring_write(ring, next_rptr);
/* insert SWITCH_BUFFER packet before first IB in the ring frame */
- if (need_ctx_switch) {
+ if (ctx_switch) {
amdgpu_ring_write(ring, PACKET3(PACKET3_SWITCH_BUFFER, 0));
amdgpu_ring_write(ring, 0);
}
@@ -4633,7 +5673,7 @@
else
header = PACKET3(PACKET3_INDIRECT_BUFFER, 2);
- control |= ib->length_dw | (ib->vm_id << 24);
+ control |= ib->length_dw | (vm_id << 24);
amdgpu_ring_write(ring, header);
amdgpu_ring_write(ring,
@@ -4646,7 +5686,8 @@
}
static void gfx_v8_0_ring_emit_ib_compute(struct amdgpu_ring *ring,
- struct amdgpu_ib *ib)
+ struct amdgpu_ib *ib,
+ unsigned vm_id, bool ctx_switch)
{
u32 header, control = 0;
u32 next_rptr = ring->wptr + 5;
@@ -4662,7 +5703,7 @@
header = PACKET3(PACKET3_INDIRECT_BUFFER, 2);
- control |= ib->length_dw | (ib->vm_id << 24);
+ control |= ib->length_dw | (vm_id << 24);
amdgpu_ring_write(ring, header);
amdgpu_ring_write(ring,
@@ -5022,6 +6063,7 @@
}
const struct amd_ip_funcs gfx_v8_0_ip_funcs = {
+ .name = "gfx_v8_0",
.early_init = gfx_v8_0_early_init,
.late_init = gfx_v8_0_late_init,
.sw_init = gfx_v8_0_sw_init,
@@ -5033,7 +6075,6 @@
.is_idle = gfx_v8_0_is_idle,
.wait_for_idle = gfx_v8_0_wait_for_idle,
.soft_reset = gfx_v8_0_soft_reset,
- .print_status = gfx_v8_0_print_status,
.set_clockgating_state = gfx_v8_0_set_clockgating_state,
.set_powergating_state = gfx_v8_0_set_powergating_state,
};
@@ -5112,6 +6153,22 @@
adev->gfx.priv_inst_irq.funcs = &gfx_v8_0_priv_inst_irq_funcs;
}
+static void gfx_v8_0_set_rlc_funcs(struct amdgpu_device *adev)
+{
+ switch (adev->asic_type) {
+ case CHIP_TOPAZ:
+ case CHIP_STONEY:
+ adev->gfx.rlc.funcs = &iceland_rlc_funcs;
+ break;
+ case CHIP_CARRIZO:
+ adev->gfx.rlc.funcs = &cz_rlc_funcs;
+ break;
+ default:
+ adev->gfx.rlc.funcs = &gfx_v8_0_nop_rlc_funcs;
+ break;
+ }
+}
+
static void gfx_v8_0_set_gds_init(struct amdgpu_device *adev)
{
/* init asci gds info */
@@ -5155,14 +6212,11 @@
return (~data) & mask;
}
-int gfx_v8_0_get_cu_info(struct amdgpu_device *adev,
- struct amdgpu_cu_info *cu_info)
+static void gfx_v8_0_get_cu_info(struct amdgpu_device *adev)
{
int i, j, k, counter, active_cu_number = 0;
u32 mask, bitmap, ao_bitmap, ao_cu_mask = 0;
-
- if (!adev || !cu_info)
- return -EINVAL;
+ struct amdgpu_cu_info *cu_info = &adev->gfx.cu_info;
memset(cu_info, 0, sizeof(*cu_info));
@@ -5193,6 +6247,4 @@
cu_info->number = active_cu_number;
cu_info->ao_cu_mask = ao_cu_mask;
-
- return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.h b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.h
index 021e051..16a49f5 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.h
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.h
@@ -28,6 +28,5 @@
uint64_t gfx_v8_0_get_gpu_clock_counter(struct amdgpu_device *adev);
void gfx_v8_0_select_se_sh(struct amdgpu_device *adev, u32 se_num, u32 sh_num);
-int gfx_v8_0_get_cu_info(struct amdgpu_device *adev, struct amdgpu_cu_info *cu_info);
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
index a4a2e6c..1feb643 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
@@ -1117,114 +1117,6 @@
}
-static void gmc_v7_0_print_status(void *handle)
-{
- int i, j;
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- dev_info(adev->dev, "GMC 8.x registers\n");
- dev_info(adev->dev, " SRBM_STATUS=0x%08X\n",
- RREG32(mmSRBM_STATUS));
- dev_info(adev->dev, " SRBM_STATUS2=0x%08X\n",
- RREG32(mmSRBM_STATUS2));
-
- dev_info(adev->dev, " VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x%08X\n",
- RREG32(mmVM_CONTEXT1_PROTECTION_FAULT_ADDR));
- dev_info(adev->dev, " VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x%08X\n",
- RREG32(mmVM_CONTEXT1_PROTECTION_FAULT_STATUS));
- dev_info(adev->dev, " MC_VM_MX_L1_TLB_CNTL=0x%08X\n",
- RREG32(mmMC_VM_MX_L1_TLB_CNTL));
- dev_info(adev->dev, " VM_L2_CNTL=0x%08X\n",
- RREG32(mmVM_L2_CNTL));
- dev_info(adev->dev, " VM_L2_CNTL2=0x%08X\n",
- RREG32(mmVM_L2_CNTL2));
- dev_info(adev->dev, " VM_L2_CNTL3=0x%08X\n",
- RREG32(mmVM_L2_CNTL3));
- dev_info(adev->dev, " VM_CONTEXT0_PAGE_TABLE_START_ADDR=0x%08X\n",
- RREG32(mmVM_CONTEXT0_PAGE_TABLE_START_ADDR));
- dev_info(adev->dev, " VM_CONTEXT0_PAGE_TABLE_END_ADDR=0x%08X\n",
- RREG32(mmVM_CONTEXT0_PAGE_TABLE_END_ADDR));
- dev_info(adev->dev, " VM_CONTEXT0_PROTECTION_FAULT_DEFAULT_ADDR=0x%08X\n",
- RREG32(mmVM_CONTEXT0_PROTECTION_FAULT_DEFAULT_ADDR));
- dev_info(adev->dev, " VM_CONTEXT0_CNTL2=0x%08X\n",
- RREG32(mmVM_CONTEXT0_CNTL2));
- dev_info(adev->dev, " VM_CONTEXT0_CNTL=0x%08X\n",
- RREG32(mmVM_CONTEXT0_CNTL));
- dev_info(adev->dev, " 0x15D4=0x%08X\n",
- RREG32(0x575));
- dev_info(adev->dev, " 0x15D8=0x%08X\n",
- RREG32(0x576));
- dev_info(adev->dev, " 0x15DC=0x%08X\n",
- RREG32(0x577));
- dev_info(adev->dev, " VM_CONTEXT1_PAGE_TABLE_START_ADDR=0x%08X\n",
- RREG32(mmVM_CONTEXT1_PAGE_TABLE_START_ADDR));
- dev_info(adev->dev, " VM_CONTEXT1_PAGE_TABLE_END_ADDR=0x%08X\n",
- RREG32(mmVM_CONTEXT1_PAGE_TABLE_END_ADDR));
- dev_info(adev->dev, " VM_CONTEXT1_PROTECTION_FAULT_DEFAULT_ADDR=0x%08X\n",
- RREG32(mmVM_CONTEXT1_PROTECTION_FAULT_DEFAULT_ADDR));
- dev_info(adev->dev, " VM_CONTEXT1_CNTL2=0x%08X\n",
- RREG32(mmVM_CONTEXT1_CNTL2));
- dev_info(adev->dev, " VM_CONTEXT1_CNTL=0x%08X\n",
- RREG32(mmVM_CONTEXT1_CNTL));
- for (i = 0; i < 16; i++) {
- if (i < 8)
- dev_info(adev->dev, " VM_CONTEXT%d_PAGE_TABLE_BASE_ADDR=0x%08X\n",
- i, RREG32(mmVM_CONTEXT0_PAGE_TABLE_BASE_ADDR + i));
- else
- dev_info(adev->dev, " VM_CONTEXT%d_PAGE_TABLE_BASE_ADDR=0x%08X\n",
- i, RREG32(mmVM_CONTEXT8_PAGE_TABLE_BASE_ADDR + i - 8));
- }
- dev_info(adev->dev, " MC_VM_SYSTEM_APERTURE_LOW_ADDR=0x%08X\n",
- RREG32(mmMC_VM_SYSTEM_APERTURE_LOW_ADDR));
- dev_info(adev->dev, " MC_VM_SYSTEM_APERTURE_HIGH_ADDR=0x%08X\n",
- RREG32(mmMC_VM_SYSTEM_APERTURE_HIGH_ADDR));
- dev_info(adev->dev, " MC_VM_SYSTEM_APERTURE_DEFAULT_ADDR=0x%08X\n",
- RREG32(mmMC_VM_SYSTEM_APERTURE_DEFAULT_ADDR));
- dev_info(adev->dev, " MC_VM_FB_LOCATION=0x%08X\n",
- RREG32(mmMC_VM_FB_LOCATION));
- dev_info(adev->dev, " MC_VM_AGP_BASE=0x%08X\n",
- RREG32(mmMC_VM_AGP_BASE));
- dev_info(adev->dev, " MC_VM_AGP_TOP=0x%08X\n",
- RREG32(mmMC_VM_AGP_TOP));
- dev_info(adev->dev, " MC_VM_AGP_BOT=0x%08X\n",
- RREG32(mmMC_VM_AGP_BOT));
-
- if (adev->asic_type == CHIP_KAVERI) {
- dev_info(adev->dev, " CHUB_CONTROL=0x%08X\n",
- RREG32(mmCHUB_CONTROL));
- }
-
- dev_info(adev->dev, " HDP_REG_COHERENCY_FLUSH_CNTL=0x%08X\n",
- RREG32(mmHDP_REG_COHERENCY_FLUSH_CNTL));
- dev_info(adev->dev, " HDP_NONSURFACE_BASE=0x%08X\n",
- RREG32(mmHDP_NONSURFACE_BASE));
- dev_info(adev->dev, " HDP_NONSURFACE_INFO=0x%08X\n",
- RREG32(mmHDP_NONSURFACE_INFO));
- dev_info(adev->dev, " HDP_NONSURFACE_SIZE=0x%08X\n",
- RREG32(mmHDP_NONSURFACE_SIZE));
- dev_info(adev->dev, " HDP_MISC_CNTL=0x%08X\n",
- RREG32(mmHDP_MISC_CNTL));
- dev_info(adev->dev, " HDP_HOST_PATH_CNTL=0x%08X\n",
- RREG32(mmHDP_HOST_PATH_CNTL));
-
- for (i = 0, j = 0; i < 32; i++, j += 0x6) {
- dev_info(adev->dev, " %d:\n", i);
- dev_info(adev->dev, " 0x%04X=0x%08X\n",
- 0xb05 + j, RREG32(0xb05 + j));
- dev_info(adev->dev, " 0x%04X=0x%08X\n",
- 0xb06 + j, RREG32(0xb06 + j));
- dev_info(adev->dev, " 0x%04X=0x%08X\n",
- 0xb07 + j, RREG32(0xb07 + j));
- dev_info(adev->dev, " 0x%04X=0x%08X\n",
- 0xb08 + j, RREG32(0xb08 + j));
- dev_info(adev->dev, " 0x%04X=0x%08X\n",
- 0xb09 + j, RREG32(0xb09 + j));
- }
-
- dev_info(adev->dev, " BIF_FB_EN=0x%08X\n",
- RREG32(mmBIF_FB_EN));
-}
-
static int gmc_v7_0_soft_reset(void *handle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
@@ -1244,8 +1136,6 @@
}
if (srbm_soft_reset) {
- gmc_v7_0_print_status((void *)adev);
-
gmc_v7_0_mc_stop(adev, &save);
if (gmc_v7_0_wait_for_idle(adev)) {
dev_warn(adev->dev, "Wait for GMC idle timed out !\n");
@@ -1269,8 +1159,6 @@
gmc_v7_0_mc_resume(adev, &save);
udelay(50);
-
- gmc_v7_0_print_status((void *)adev);
}
return 0;
@@ -1373,6 +1261,7 @@
}
const struct amd_ip_funcs gmc_v7_0_ip_funcs = {
+ .name = "gmc_v7_0",
.early_init = gmc_v7_0_early_init,
.late_init = gmc_v7_0_late_init,
.sw_init = gmc_v7_0_sw_init,
@@ -1384,7 +1273,6 @@
.is_idle = gmc_v7_0_is_idle,
.wait_for_idle = gmc_v7_0_wait_for_idle,
.soft_reset = gmc_v7_0_soft_reset,
- .print_status = gmc_v7_0_print_status,
.set_clockgating_state = gmc_v7_0_set_clockgating_state,
.set_powergating_state = gmc_v7_0_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
index 7a9db2c..9945d5b 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
@@ -43,6 +43,8 @@
static void gmc_v8_0_set_irq_funcs(struct amdgpu_device *adev);
MODULE_FIRMWARE("amdgpu/tonga_mc.bin");
+MODULE_FIRMWARE("amdgpu/polaris11_mc.bin");
+MODULE_FIRMWARE("amdgpu/polaris10_mc.bin");
static const u32 golden_settings_tonga_a11[] =
{
@@ -73,6 +75,23 @@
mmMC_MEM_POWER_LS, 0xffffffff, 0x00000104
};
+static const u32 golden_settings_polaris11_a11[] =
+{
+ mmVM_PRT_APERTURE0_LOW_ADDR, 0x0fffffff, 0x0fffffff,
+ mmVM_PRT_APERTURE1_LOW_ADDR, 0x0fffffff, 0x0fffffff,
+ mmVM_PRT_APERTURE2_LOW_ADDR, 0x0fffffff, 0x0fffffff,
+ mmVM_PRT_APERTURE3_LOW_ADDR, 0x0fffffff, 0x0fffffff
+};
+
+static const u32 golden_settings_polaris10_a11[] =
+{
+ mmMC_ARB_WTM_GRPWT_RD, 0x00000003, 0x00000000,
+ mmVM_PRT_APERTURE0_LOW_ADDR, 0x0fffffff, 0x0fffffff,
+ mmVM_PRT_APERTURE1_LOW_ADDR, 0x0fffffff, 0x0fffffff,
+ mmVM_PRT_APERTURE2_LOW_ADDR, 0x0fffffff, 0x0fffffff,
+ mmVM_PRT_APERTURE3_LOW_ADDR, 0x0fffffff, 0x0fffffff
+};
+
static const u32 cz_mgcg_cgcg_init[] =
{
mmMC_MEM_POWER_LS, 0xffffffff, 0x00000104
@@ -103,6 +122,16 @@
golden_settings_tonga_a11,
(const u32)ARRAY_SIZE(golden_settings_tonga_a11));
break;
+ case CHIP_POLARIS11:
+ amdgpu_program_register_sequence(adev,
+ golden_settings_polaris11_a11,
+ (const u32)ARRAY_SIZE(golden_settings_polaris11_a11));
+ break;
+ case CHIP_POLARIS10:
+ amdgpu_program_register_sequence(adev,
+ golden_settings_polaris10_a11,
+ (const u32)ARRAY_SIZE(golden_settings_polaris10_a11));
+ break;
case CHIP_CARRIZO:
amdgpu_program_register_sequence(adev,
cz_mgcg_cgcg_init,
@@ -209,6 +238,12 @@
case CHIP_TONGA:
chip_name = "tonga";
break;
+ case CHIP_POLARIS11:
+ chip_name = "polaris11";
+ break;
+ case CHIP_POLARIS10:
+ chip_name = "polaris10";
+ break;
case CHIP_FIJI:
case CHIP_CARRIZO:
case CHIP_STONEY:
@@ -1085,111 +1120,6 @@
}
-static void gmc_v8_0_print_status(void *handle)
-{
- int i, j;
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- dev_info(adev->dev, "GMC 8.x registers\n");
- dev_info(adev->dev, " SRBM_STATUS=0x%08X\n",
- RREG32(mmSRBM_STATUS));
- dev_info(adev->dev, " SRBM_STATUS2=0x%08X\n",
- RREG32(mmSRBM_STATUS2));
-
- dev_info(adev->dev, " VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x%08X\n",
- RREG32(mmVM_CONTEXT1_PROTECTION_FAULT_ADDR));
- dev_info(adev->dev, " VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x%08X\n",
- RREG32(mmVM_CONTEXT1_PROTECTION_FAULT_STATUS));
- dev_info(adev->dev, " MC_VM_MX_L1_TLB_CNTL=0x%08X\n",
- RREG32(mmMC_VM_MX_L1_TLB_CNTL));
- dev_info(adev->dev, " VM_L2_CNTL=0x%08X\n",
- RREG32(mmVM_L2_CNTL));
- dev_info(adev->dev, " VM_L2_CNTL2=0x%08X\n",
- RREG32(mmVM_L2_CNTL2));
- dev_info(adev->dev, " VM_L2_CNTL3=0x%08X\n",
- RREG32(mmVM_L2_CNTL3));
- dev_info(adev->dev, " VM_L2_CNTL4=0x%08X\n",
- RREG32(mmVM_L2_CNTL4));
- dev_info(adev->dev, " VM_CONTEXT0_PAGE_TABLE_START_ADDR=0x%08X\n",
- RREG32(mmVM_CONTEXT0_PAGE_TABLE_START_ADDR));
- dev_info(adev->dev, " VM_CONTEXT0_PAGE_TABLE_END_ADDR=0x%08X\n",
- RREG32(mmVM_CONTEXT0_PAGE_TABLE_END_ADDR));
- dev_info(adev->dev, " VM_CONTEXT0_PROTECTION_FAULT_DEFAULT_ADDR=0x%08X\n",
- RREG32(mmVM_CONTEXT0_PROTECTION_FAULT_DEFAULT_ADDR));
- dev_info(adev->dev, " VM_CONTEXT0_CNTL2=0x%08X\n",
- RREG32(mmVM_CONTEXT0_CNTL2));
- dev_info(adev->dev, " VM_CONTEXT0_CNTL=0x%08X\n",
- RREG32(mmVM_CONTEXT0_CNTL));
- dev_info(adev->dev, " VM_L2_CONTEXT1_IDENTITY_APERTURE_LOW_ADDR=0x%08X\n",
- RREG32(mmVM_L2_CONTEXT1_IDENTITY_APERTURE_LOW_ADDR));
- dev_info(adev->dev, " VM_L2_CONTEXT1_IDENTITY_APERTURE_HIGH_ADDR=0x%08X\n",
- RREG32(mmVM_L2_CONTEXT1_IDENTITY_APERTURE_HIGH_ADDR));
- dev_info(adev->dev, " mmVM_L2_CONTEXT_IDENTITY_PHYSICAL_OFFSET=0x%08X\n",
- RREG32(mmVM_L2_CONTEXT_IDENTITY_PHYSICAL_OFFSET));
- dev_info(adev->dev, " VM_CONTEXT1_PAGE_TABLE_START_ADDR=0x%08X\n",
- RREG32(mmVM_CONTEXT1_PAGE_TABLE_START_ADDR));
- dev_info(adev->dev, " VM_CONTEXT1_PAGE_TABLE_END_ADDR=0x%08X\n",
- RREG32(mmVM_CONTEXT1_PAGE_TABLE_END_ADDR));
- dev_info(adev->dev, " VM_CONTEXT1_PROTECTION_FAULT_DEFAULT_ADDR=0x%08X\n",
- RREG32(mmVM_CONTEXT1_PROTECTION_FAULT_DEFAULT_ADDR));
- dev_info(adev->dev, " VM_CONTEXT1_CNTL2=0x%08X\n",
- RREG32(mmVM_CONTEXT1_CNTL2));
- dev_info(adev->dev, " VM_CONTEXT1_CNTL=0x%08X\n",
- RREG32(mmVM_CONTEXT1_CNTL));
- for (i = 0; i < 16; i++) {
- if (i < 8)
- dev_info(adev->dev, " VM_CONTEXT%d_PAGE_TABLE_BASE_ADDR=0x%08X\n",
- i, RREG32(mmVM_CONTEXT0_PAGE_TABLE_BASE_ADDR + i));
- else
- dev_info(adev->dev, " VM_CONTEXT%d_PAGE_TABLE_BASE_ADDR=0x%08X\n",
- i, RREG32(mmVM_CONTEXT8_PAGE_TABLE_BASE_ADDR + i - 8));
- }
- dev_info(adev->dev, " MC_VM_SYSTEM_APERTURE_LOW_ADDR=0x%08X\n",
- RREG32(mmMC_VM_SYSTEM_APERTURE_LOW_ADDR));
- dev_info(adev->dev, " MC_VM_SYSTEM_APERTURE_HIGH_ADDR=0x%08X\n",
- RREG32(mmMC_VM_SYSTEM_APERTURE_HIGH_ADDR));
- dev_info(adev->dev, " MC_VM_SYSTEM_APERTURE_DEFAULT_ADDR=0x%08X\n",
- RREG32(mmMC_VM_SYSTEM_APERTURE_DEFAULT_ADDR));
- dev_info(adev->dev, " MC_VM_FB_LOCATION=0x%08X\n",
- RREG32(mmMC_VM_FB_LOCATION));
- dev_info(adev->dev, " MC_VM_AGP_BASE=0x%08X\n",
- RREG32(mmMC_VM_AGP_BASE));
- dev_info(adev->dev, " MC_VM_AGP_TOP=0x%08X\n",
- RREG32(mmMC_VM_AGP_TOP));
- dev_info(adev->dev, " MC_VM_AGP_BOT=0x%08X\n",
- RREG32(mmMC_VM_AGP_BOT));
-
- dev_info(adev->dev, " HDP_REG_COHERENCY_FLUSH_CNTL=0x%08X\n",
- RREG32(mmHDP_REG_COHERENCY_FLUSH_CNTL));
- dev_info(adev->dev, " HDP_NONSURFACE_BASE=0x%08X\n",
- RREG32(mmHDP_NONSURFACE_BASE));
- dev_info(adev->dev, " HDP_NONSURFACE_INFO=0x%08X\n",
- RREG32(mmHDP_NONSURFACE_INFO));
- dev_info(adev->dev, " HDP_NONSURFACE_SIZE=0x%08X\n",
- RREG32(mmHDP_NONSURFACE_SIZE));
- dev_info(adev->dev, " HDP_MISC_CNTL=0x%08X\n",
- RREG32(mmHDP_MISC_CNTL));
- dev_info(adev->dev, " HDP_HOST_PATH_CNTL=0x%08X\n",
- RREG32(mmHDP_HOST_PATH_CNTL));
-
- for (i = 0, j = 0; i < 32; i++, j += 0x6) {
- dev_info(adev->dev, " %d:\n", i);
- dev_info(adev->dev, " 0x%04X=0x%08X\n",
- 0xb05 + j, RREG32(0xb05 + j));
- dev_info(adev->dev, " 0x%04X=0x%08X\n",
- 0xb06 + j, RREG32(0xb06 + j));
- dev_info(adev->dev, " 0x%04X=0x%08X\n",
- 0xb07 + j, RREG32(0xb07 + j));
- dev_info(adev->dev, " 0x%04X=0x%08X\n",
- 0xb08 + j, RREG32(0xb08 + j));
- dev_info(adev->dev, " 0x%04X=0x%08X\n",
- 0xb09 + j, RREG32(0xb09 + j));
- }
-
- dev_info(adev->dev, " BIF_FB_EN=0x%08X\n",
- RREG32(mmBIF_FB_EN));
-}
-
static int gmc_v8_0_soft_reset(void *handle)
{
struct amdgpu_mode_mc_save save;
@@ -1209,8 +1139,6 @@
}
if (srbm_soft_reset) {
- gmc_v8_0_print_status((void *)adev);
-
gmc_v8_0_mc_stop(adev, &save);
if (gmc_v8_0_wait_for_idle(adev)) {
dev_warn(adev->dev, "Wait for GMC idle timed out !\n");
@@ -1234,8 +1162,6 @@
gmc_v8_0_mc_resume(adev, &save);
udelay(50);
-
- gmc_v8_0_print_status((void *)adev);
}
return 0;
@@ -1313,11 +1239,11 @@
}
static void fiji_update_mc_medium_grain_clock_gating(struct amdgpu_device *adev,
- bool enable)
+ bool enable)
{
uint32_t data;
- if (enable) {
+ if (enable && (adev->cg_flags & AMD_CG_SUPPORT_MC_MGCG)) {
data = RREG32(mmMC_HUB_MISC_HUB_CG);
data |= MC_HUB_MISC_HUB_CG__ENABLE_MASK;
WREG32(mmMC_HUB_MISC_HUB_CG, data);
@@ -1393,11 +1319,11 @@
}
static void fiji_update_mc_light_sleep(struct amdgpu_device *adev,
- bool enable)
+ bool enable)
{
uint32_t data;
- if (enable) {
+ if (enable && (adev->cg_flags & AMD_CG_SUPPORT_MC_LS)) {
data = RREG32(mmMC_HUB_MISC_HUB_CG);
data |= MC_HUB_MISC_HUB_CG__MEM_LS_ENABLE_MASK;
WREG32(mmMC_HUB_MISC_HUB_CG, data);
@@ -1497,6 +1423,7 @@
}
const struct amd_ip_funcs gmc_v8_0_ip_funcs = {
+ .name = "gmc_v8_0",
.early_init = gmc_v8_0_early_init,
.late_init = gmc_v8_0_late_init,
.sw_init = gmc_v8_0_sw_init,
@@ -1508,7 +1435,6 @@
.is_idle = gmc_v8_0_is_idle,
.wait_for_idle = gmc_v8_0_wait_for_idle,
.soft_reset = gmc_v8_0_soft_reset,
- .print_status = gmc_v8_0_print_status,
.set_clockgating_state = gmc_v8_0_set_clockgating_state,
.set_powergating_state = gmc_v8_0_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_dpm.c b/drivers/gpu/drm/amd/amdgpu/iceland_dpm.c
index 208d55f4..460bc8a 100644
--- a/drivers/gpu/drm/amd/amdgpu/iceland_dpm.c
+++ b/drivers/gpu/drm/amd/amdgpu/iceland_dpm.c
@@ -157,6 +157,7 @@
}
const struct amd_ip_funcs iceland_dpm_ip_funcs = {
+ .name = "iceland_dpm",
.early_init = iceland_dpm_early_init,
.late_init = NULL,
.sw_init = iceland_dpm_sw_init,
@@ -168,7 +169,6 @@
.is_idle = NULL,
.wait_for_idle = NULL,
.soft_reset = NULL,
- .print_status = NULL,
.set_clockgating_state = iceland_dpm_set_clockgating_state,
.set_powergating_state = iceland_dpm_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
index 679e739..39bfc52 100644
--- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
@@ -351,35 +351,6 @@
return -ETIMEDOUT;
}
-static void iceland_ih_print_status(void *handle)
-{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- dev_info(adev->dev, "ICELAND IH registers\n");
- dev_info(adev->dev, " SRBM_STATUS=0x%08X\n",
- RREG32(mmSRBM_STATUS));
- dev_info(adev->dev, " SRBM_STATUS2=0x%08X\n",
- RREG32(mmSRBM_STATUS2));
- dev_info(adev->dev, " INTERRUPT_CNTL=0x%08X\n",
- RREG32(mmINTERRUPT_CNTL));
- dev_info(adev->dev, " INTERRUPT_CNTL2=0x%08X\n",
- RREG32(mmINTERRUPT_CNTL2));
- dev_info(adev->dev, " IH_CNTL=0x%08X\n",
- RREG32(mmIH_CNTL));
- dev_info(adev->dev, " IH_RB_CNTL=0x%08X\n",
- RREG32(mmIH_RB_CNTL));
- dev_info(adev->dev, " IH_RB_BASE=0x%08X\n",
- RREG32(mmIH_RB_BASE));
- dev_info(adev->dev, " IH_RB_WPTR_ADDR_LO=0x%08X\n",
- RREG32(mmIH_RB_WPTR_ADDR_LO));
- dev_info(adev->dev, " IH_RB_WPTR_ADDR_HI=0x%08X\n",
- RREG32(mmIH_RB_WPTR_ADDR_HI));
- dev_info(adev->dev, " IH_RB_RPTR=0x%08X\n",
- RREG32(mmIH_RB_RPTR));
- dev_info(adev->dev, " IH_RB_WPTR=0x%08X\n",
- RREG32(mmIH_RB_WPTR));
-}
-
static int iceland_ih_soft_reset(void *handle)
{
u32 srbm_soft_reset = 0;
@@ -391,8 +362,6 @@
SOFT_RESET_IH, 1);
if (srbm_soft_reset) {
- iceland_ih_print_status((void *)adev);
-
tmp = RREG32(mmSRBM_SOFT_RESET);
tmp |= srbm_soft_reset;
dev_info(adev->dev, "SRBM_SOFT_RESET=0x%08X\n", tmp);
@@ -407,8 +376,6 @@
/* Wait a little for things to settle down */
udelay(50);
-
- iceland_ih_print_status((void *)adev);
}
return 0;
@@ -427,6 +394,7 @@
}
const struct amd_ip_funcs iceland_ih_ip_funcs = {
+ .name = "iceland_ih",
.early_init = iceland_ih_early_init,
.late_init = NULL,
.sw_init = iceland_ih_sw_init,
@@ -438,7 +406,6 @@
.is_idle = iceland_ih_is_idle,
.wait_for_idle = iceland_ih_wait_for_idle,
.soft_reset = iceland_ih_soft_reset,
- .print_status = iceland_ih_print_status,
.set_clockgating_state = iceland_ih_set_clockgating_state,
.set_powergating_state = iceland_ih_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/kv_dpm.c b/drivers/gpu/drm/amd/amdgpu/kv_dpm.c
index 654d767..b45f547 100644
--- a/drivers/gpu/drm/amd/amdgpu/kv_dpm.c
+++ b/drivers/gpu/drm/amd/amdgpu/kv_dpm.c
@@ -135,11 +135,6 @@
#endif
}
-static u32 sumo_get_sleep_divider_from_id(u32 id)
-{
- return 1 << id;
-}
-
static void sumo_construct_sclk_voltage_mapping_table(struct amdgpu_device *adev,
struct sumo_sclk_voltage_mapping_table *sclk_voltage_mapping_table,
ATOM_AVAILABLE_SCLK_LIST *table)
@@ -2176,8 +2171,7 @@
struct kv_power_info *pi = kv_get_pi(adev);
u32 i;
u32 temp;
- u32 min = (min_sclk_in_sr > KV_MINIMUM_ENGINE_CLOCK) ?
- min_sclk_in_sr : KV_MINIMUM_ENGINE_CLOCK;
+ u32 min = max(min_sclk_in_sr, (u32)KV_MINIMUM_ENGINE_CLOCK);
if (sclk < min)
return 0;
@@ -2186,7 +2180,7 @@
return 0;
for (i = KV_MAX_DEEPSLEEP_DIVIDER_ID; i > 0; i--) {
- temp = sclk / sumo_get_sleep_divider_from_id(i);
+ temp = sclk >> i;
if (temp >= min)
break;
}
@@ -3147,62 +3141,6 @@
return 0;
}
-static void kv_dpm_print_status(void *handle)
-{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- dev_info(adev->dev, "KV/KB DPM registers\n");
- dev_info(adev->dev, " DIDT_SQ_CTRL0=0x%08X\n",
- RREG32_DIDT(ixDIDT_SQ_CTRL0));
- dev_info(adev->dev, " DIDT_DB_CTRL0=0x%08X\n",
- RREG32_DIDT(ixDIDT_DB_CTRL0));
- dev_info(adev->dev, " DIDT_TD_CTRL0=0x%08X\n",
- RREG32_DIDT(ixDIDT_TD_CTRL0));
- dev_info(adev->dev, " DIDT_TCP_CTRL0=0x%08X\n",
- RREG32_DIDT(ixDIDT_TCP_CTRL0));
- dev_info(adev->dev, " LCAC_SX0_OVR_SEL=0x%08X\n",
- RREG32_SMC(ixLCAC_SX0_OVR_SEL));
- dev_info(adev->dev, " LCAC_SX0_OVR_VAL=0x%08X\n",
- RREG32_SMC(ixLCAC_SX0_OVR_VAL));
- dev_info(adev->dev, " LCAC_MC0_OVR_SEL=0x%08X\n",
- RREG32_SMC(ixLCAC_MC0_OVR_SEL));
- dev_info(adev->dev, " LCAC_MC0_OVR_VAL=0x%08X\n",
- RREG32_SMC(ixLCAC_MC0_OVR_VAL));
- dev_info(adev->dev, " LCAC_MC1_OVR_SEL=0x%08X\n",
- RREG32_SMC(ixLCAC_MC1_OVR_SEL));
- dev_info(adev->dev, " LCAC_MC1_OVR_VAL=0x%08X\n",
- RREG32_SMC(ixLCAC_MC1_OVR_VAL));
- dev_info(adev->dev, " LCAC_MC2_OVR_SEL=0x%08X\n",
- RREG32_SMC(ixLCAC_MC2_OVR_SEL));
- dev_info(adev->dev, " LCAC_MC2_OVR_VAL=0x%08X\n",
- RREG32_SMC(ixLCAC_MC2_OVR_VAL));
- dev_info(adev->dev, " LCAC_MC3_OVR_SEL=0x%08X\n",
- RREG32_SMC(ixLCAC_MC3_OVR_SEL));
- dev_info(adev->dev, " LCAC_MC3_OVR_VAL=0x%08X\n",
- RREG32_SMC(ixLCAC_MC3_OVR_VAL));
- dev_info(adev->dev, " LCAC_CPL_OVR_SEL=0x%08X\n",
- RREG32_SMC(ixLCAC_CPL_OVR_SEL));
- dev_info(adev->dev, " LCAC_CPL_OVR_VAL=0x%08X\n",
- RREG32_SMC(ixLCAC_CPL_OVR_VAL));
- dev_info(adev->dev, " CG_FREQ_TRAN_VOTING_0=0x%08X\n",
- RREG32_SMC(ixCG_FREQ_TRAN_VOTING_0));
- dev_info(adev->dev, " GENERAL_PWRMGT=0x%08X\n",
- RREG32_SMC(ixGENERAL_PWRMGT));
- dev_info(adev->dev, " SCLK_PWRMGT_CNTL=0x%08X\n",
- RREG32_SMC(ixSCLK_PWRMGT_CNTL));
- dev_info(adev->dev, " SMC_MESSAGE_0=0x%08X\n",
- RREG32(mmSMC_MESSAGE_0));
- dev_info(adev->dev, " SMC_RESP_0=0x%08X\n",
- RREG32(mmSMC_RESP_0));
- dev_info(adev->dev, " SMC_MSG_ARG_0=0x%08X\n",
- RREG32(mmSMC_MSG_ARG_0));
- dev_info(adev->dev, " SMC_IND_INDEX_0=0x%08X\n",
- RREG32(mmSMC_IND_INDEX_0));
- dev_info(adev->dev, " SMC_IND_DATA_0=0x%08X\n",
- RREG32(mmSMC_IND_DATA_0));
- dev_info(adev->dev, " SMC_IND_ACCESS_CNTL=0x%08X\n",
- RREG32(mmSMC_IND_ACCESS_CNTL));
-}
static int kv_dpm_soft_reset(void *handle)
{
@@ -3300,6 +3238,7 @@
}
const struct amd_ip_funcs kv_dpm_ip_funcs = {
+ .name = "kv_dpm",
.early_init = kv_dpm_early_init,
.late_init = kv_dpm_late_init,
.sw_init = kv_dpm_sw_init,
@@ -3311,7 +3250,6 @@
.is_idle = kv_dpm_is_idle,
.wait_for_idle = kv_dpm_wait_for_idle,
.soft_reset = kv_dpm_soft_reset,
- .print_status = kv_dpm_print_status,
.set_clockgating_state = kv_dpm_set_clockgating_state,
.set_powergating_state = kv_dpm_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v2_4.c b/drivers/gpu/drm/amd/amdgpu/sdma_v2_4.c
index 6e0a86a..f4c3130 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v2_4.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v2_4.c
@@ -242,9 +242,10 @@
* Schedule an IB in the DMA ring (VI).
*/
static void sdma_v2_4_ring_emit_ib(struct amdgpu_ring *ring,
- struct amdgpu_ib *ib)
+ struct amdgpu_ib *ib,
+ unsigned vm_id, bool ctx_switch)
{
- u32 vmid = ib->vm_id & 0xf;
+ u32 vmid = vm_id & 0xf;
u32 next_rptr = ring->wptr + 5;
while ((next_rptr & 7) != 2)
@@ -701,7 +702,7 @@
ib.ptr[7] = SDMA_PKT_HEADER_OP(SDMA_OP_NOP);
ib.length_dw = 8;
- r = amdgpu_ib_schedule(ring, 1, &ib, NULL, &f);
+ r = amdgpu_ib_schedule(ring, 1, &ib, NULL, NULL, &f);
if (r)
goto err1;
@@ -990,7 +991,7 @@
ring->ring_obj = NULL;
ring->use_doorbell = false;
sprintf(ring->name, "sdma%d", i);
- r = amdgpu_ring_init(adev, ring, 256 * 1024,
+ r = amdgpu_ring_init(adev, ring, 1024,
SDMA_PKT_NOP_HEADER_OP(SDMA_OP_NOP), 0xf,
&adev->sdma.trap_irq,
(i == 0) ?
@@ -1080,55 +1081,6 @@
return -ETIMEDOUT;
}
-static void sdma_v2_4_print_status(void *handle)
-{
- int i, j;
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- dev_info(adev->dev, "VI SDMA registers\n");
- dev_info(adev->dev, " SRBM_STATUS2=0x%08X\n",
- RREG32(mmSRBM_STATUS2));
- for (i = 0; i < adev->sdma.num_instances; i++) {
- dev_info(adev->dev, " SDMA%d_STATUS_REG=0x%08X\n",
- i, RREG32(mmSDMA0_STATUS_REG + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_F32_CNTL=0x%08X\n",
- i, RREG32(mmSDMA0_F32_CNTL + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_CNTL=0x%08X\n",
- i, RREG32(mmSDMA0_CNTL + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_SEM_WAIT_FAIL_TIMER_CNTL=0x%08X\n",
- i, RREG32(mmSDMA0_SEM_WAIT_FAIL_TIMER_CNTL + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_IB_CNTL=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_IB_CNTL + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_RB_CNTL=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_RB_CNTL + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_RB_RPTR=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_RB_RPTR + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_RB_WPTR=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_RB_WPTR + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_RB_RPTR_ADDR_HI=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_RB_RPTR_ADDR_HI + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_RB_RPTR_ADDR_LO=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_RB_RPTR_ADDR_LO + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_RB_BASE=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_RB_BASE + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_RB_BASE_HI=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_RB_BASE_HI + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_TILING_CONFIG=0x%08X\n",
- i, RREG32(mmSDMA0_TILING_CONFIG + sdma_offsets[i]));
- mutex_lock(&adev->srbm_mutex);
- for (j = 0; j < 16; j++) {
- vi_srbm_select(adev, 0, 0, 0, j);
- dev_info(adev->dev, " VM %d:\n", j);
- dev_info(adev->dev, " SDMA%d_GFX_VIRTUAL_ADDR=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_VIRTUAL_ADDR + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_APE1_CNTL=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_APE1_CNTL + sdma_offsets[i]));
- }
- vi_srbm_select(adev, 0, 0, 0, 0);
- mutex_unlock(&adev->srbm_mutex);
- }
-}
-
static int sdma_v2_4_soft_reset(void *handle)
{
u32 srbm_soft_reset = 0;
@@ -1151,8 +1103,6 @@
}
if (srbm_soft_reset) {
- sdma_v2_4_print_status((void *)adev);
-
tmp = RREG32(mmSRBM_SOFT_RESET);
tmp |= srbm_soft_reset;
dev_info(adev->dev, "SRBM_SOFT_RESET=0x%08X\n", tmp);
@@ -1167,8 +1117,6 @@
/* Wait a little for things to settle down */
udelay(50);
-
- sdma_v2_4_print_status((void *)adev);
}
return 0;
@@ -1283,6 +1231,7 @@
}
const struct amd_ip_funcs sdma_v2_4_ip_funcs = {
+ .name = "sdma_v2_4",
.early_init = sdma_v2_4_early_init,
.late_init = NULL,
.sw_init = sdma_v2_4_sw_init,
@@ -1294,7 +1243,6 @@
.is_idle = sdma_v2_4_is_idle,
.wait_for_idle = sdma_v2_4_wait_for_idle,
.soft_reset = sdma_v2_4_soft_reset,
- .print_status = sdma_v2_4_print_status,
.set_clockgating_state = sdma_v2_4_set_clockgating_state,
.set_powergating_state = sdma_v2_4_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v3_0.c b/drivers/gpu/drm/amd/amdgpu/sdma_v3_0.c
index 8c8ca98..063f08a 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v3_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v3_0.c
@@ -56,6 +56,11 @@
MODULE_FIRMWARE("amdgpu/fiji_sdma.bin");
MODULE_FIRMWARE("amdgpu/fiji_sdma1.bin");
MODULE_FIRMWARE("amdgpu/stoney_sdma.bin");
+MODULE_FIRMWARE("amdgpu/polaris10_sdma.bin");
+MODULE_FIRMWARE("amdgpu/polaris10_sdma1.bin");
+MODULE_FIRMWARE("amdgpu/polaris11_sdma.bin");
+MODULE_FIRMWARE("amdgpu/polaris11_sdma1.bin");
+
static const u32 sdma_offsets[SDMA_MAX_INSTANCE] =
{
@@ -101,6 +106,32 @@
mmSDMA1_CLK_CTRL, 0xff000ff0, 0x00000100
};
+static const u32 golden_settings_polaris11_a11[] =
+{
+ mmSDMA0_CHICKEN_BITS, 0xfc910007, 0x00810007,
+ mmSDMA0_GFX_IB_CNTL, 0x800f0111, 0x00000100,
+ mmSDMA0_RLC0_IB_CNTL, 0x800f0111, 0x00000100,
+ mmSDMA0_RLC1_IB_CNTL, 0x800f0111, 0x00000100,
+ mmSDMA1_CHICKEN_BITS, 0xfc910007, 0x00810007,
+ mmSDMA1_GFX_IB_CNTL, 0x800f0111, 0x00000100,
+ mmSDMA1_RLC0_IB_CNTL, 0x800f0111, 0x00000100,
+ mmSDMA1_RLC1_IB_CNTL, 0x800f0111, 0x00000100,
+};
+
+static const u32 golden_settings_polaris10_a11[] =
+{
+ mmSDMA0_CHICKEN_BITS, 0xfc910007, 0x00810007,
+ mmSDMA0_CLK_CTRL, 0xff000fff, 0x00000000,
+ mmSDMA0_GFX_IB_CNTL, 0x800f0111, 0x00000100,
+ mmSDMA0_RLC0_IB_CNTL, 0x800f0111, 0x00000100,
+ mmSDMA0_RLC1_IB_CNTL, 0x800f0111, 0x00000100,
+ mmSDMA1_CHICKEN_BITS, 0xfc910007, 0x00810007,
+ mmSDMA1_CLK_CTRL, 0xff000fff, 0x00000000,
+ mmSDMA1_GFX_IB_CNTL, 0x800f0111, 0x00000100,
+ mmSDMA1_RLC0_IB_CNTL, 0x800f0111, 0x00000100,
+ mmSDMA1_RLC1_IB_CNTL, 0x800f0111, 0x00000100,
+};
+
static const u32 cz_golden_settings_a11[] =
{
mmSDMA0_CHICKEN_BITS, 0xfc910007, 0x00810007,
@@ -172,6 +203,16 @@
golden_settings_tonga_a11,
(const u32)ARRAY_SIZE(golden_settings_tonga_a11));
break;
+ case CHIP_POLARIS11:
+ amdgpu_program_register_sequence(adev,
+ golden_settings_polaris11_a11,
+ (const u32)ARRAY_SIZE(golden_settings_polaris11_a11));
+ break;
+ case CHIP_POLARIS10:
+ amdgpu_program_register_sequence(adev,
+ golden_settings_polaris10_a11,
+ (const u32)ARRAY_SIZE(golden_settings_polaris10_a11));
+ break;
case CHIP_CARRIZO:
amdgpu_program_register_sequence(adev,
cz_mgcg_cgcg_init,
@@ -220,6 +261,12 @@
case CHIP_FIJI:
chip_name = "fiji";
break;
+ case CHIP_POLARIS11:
+ chip_name = "polaris11";
+ break;
+ case CHIP_POLARIS10:
+ chip_name = "polaris10";
+ break;
case CHIP_CARRIZO:
chip_name = "carrizo";
break;
@@ -353,9 +400,10 @@
* Schedule an IB in the DMA ring (VI).
*/
static void sdma_v3_0_ring_emit_ib(struct amdgpu_ring *ring,
- struct amdgpu_ib *ib)
+ struct amdgpu_ib *ib,
+ unsigned vm_id, bool ctx_switch)
{
- u32 vmid = ib->vm_id & 0xf;
+ u32 vmid = vm_id & 0xf;
u32 next_rptr = ring->wptr + 5;
while ((next_rptr & 7) != 2)
@@ -452,6 +500,31 @@
amdgpu_ring_write(ring, SDMA_PKT_TRAP_INT_CONTEXT_INT_CONTEXT(0));
}
+unsigned init_cond_exec(struct amdgpu_ring *ring)
+{
+ unsigned ret;
+ amdgpu_ring_write(ring, SDMA_PKT_HEADER_OP(SDMA_OP_COND_EXE));
+ amdgpu_ring_write(ring, lower_32_bits(ring->cond_exe_gpu_addr));
+ amdgpu_ring_write(ring, upper_32_bits(ring->cond_exe_gpu_addr));
+ amdgpu_ring_write(ring, 1);
+ ret = ring->wptr;/* this is the offset we need patch later */
+ amdgpu_ring_write(ring, 0x55aa55aa);/* insert dummy here and patch it later */
+ return ret;
+}
+
+void patch_cond_exec(struct amdgpu_ring *ring, unsigned offset)
+{
+ unsigned cur;
+ BUG_ON(ring->ring[offset] != 0x55aa55aa);
+
+ cur = ring->wptr - 1;
+ if (likely(cur > offset))
+ ring->ring[offset] = cur - offset;
+ else
+ ring->ring[offset] = (ring->ring_size>>2) - offset + cur;
+}
+
+
/**
* sdma_v3_0_gfx_stop - stop the gfx async dma engines
*
@@ -853,7 +926,7 @@
ib.ptr[7] = SDMA_PKT_NOP_HEADER_OP(SDMA_OP_NOP);
ib.length_dw = 8;
- r = amdgpu_ib_schedule(ring, 1, &ib, NULL, &f);
+ r = amdgpu_ib_schedule(ring, 1, &ib, NULL, NULL, &f);
if (r)
goto err1;
@@ -1151,7 +1224,7 @@
AMDGPU_DOORBELL_sDMA_ENGINE0 : AMDGPU_DOORBELL_sDMA_ENGINE1;
sprintf(ring->name, "sdma%d", i);
- r = amdgpu_ring_init(adev, ring, 256 * 1024,
+ r = amdgpu_ring_init(adev, ring, 1024,
SDMA_PKT_NOP_HEADER_OP(SDMA_OP_NOP), 0xf,
&adev->sdma.trap_irq,
(i == 0) ?
@@ -1242,57 +1315,6 @@
return -ETIMEDOUT;
}
-static void sdma_v3_0_print_status(void *handle)
-{
- int i, j;
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- dev_info(adev->dev, "VI SDMA registers\n");
- dev_info(adev->dev, " SRBM_STATUS2=0x%08X\n",
- RREG32(mmSRBM_STATUS2));
- for (i = 0; i < adev->sdma.num_instances; i++) {
- dev_info(adev->dev, " SDMA%d_STATUS_REG=0x%08X\n",
- i, RREG32(mmSDMA0_STATUS_REG + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_F32_CNTL=0x%08X\n",
- i, RREG32(mmSDMA0_F32_CNTL + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_CNTL=0x%08X\n",
- i, RREG32(mmSDMA0_CNTL + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_SEM_WAIT_FAIL_TIMER_CNTL=0x%08X\n",
- i, RREG32(mmSDMA0_SEM_WAIT_FAIL_TIMER_CNTL + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_IB_CNTL=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_IB_CNTL + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_RB_CNTL=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_RB_CNTL + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_RB_RPTR=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_RB_RPTR + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_RB_WPTR=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_RB_WPTR + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_RB_RPTR_ADDR_HI=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_RB_RPTR_ADDR_HI + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_RB_RPTR_ADDR_LO=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_RB_RPTR_ADDR_LO + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_RB_BASE=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_RB_BASE + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_RB_BASE_HI=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_RB_BASE_HI + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_DOORBELL=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_DOORBELL + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_TILING_CONFIG=0x%08X\n",
- i, RREG32(mmSDMA0_TILING_CONFIG + sdma_offsets[i]));
- mutex_lock(&adev->srbm_mutex);
- for (j = 0; j < 16; j++) {
- vi_srbm_select(adev, 0, 0, 0, j);
- dev_info(adev->dev, " VM %d:\n", j);
- dev_info(adev->dev, " SDMA%d_GFX_VIRTUAL_ADDR=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_VIRTUAL_ADDR + sdma_offsets[i]));
- dev_info(adev->dev, " SDMA%d_GFX_APE1_CNTL=0x%08X\n",
- i, RREG32(mmSDMA0_GFX_APE1_CNTL + sdma_offsets[i]));
- }
- vi_srbm_select(adev, 0, 0, 0, 0);
- mutex_unlock(&adev->srbm_mutex);
- }
-}
-
static int sdma_v3_0_soft_reset(void *handle)
{
u32 srbm_soft_reset = 0;
@@ -1315,8 +1337,6 @@
}
if (srbm_soft_reset) {
- sdma_v3_0_print_status((void *)adev);
-
tmp = RREG32(mmSRBM_SOFT_RESET);
tmp |= srbm_soft_reset;
dev_info(adev->dev, "SRBM_SOFT_RESET=0x%08X\n", tmp);
@@ -1331,8 +1351,6 @@
/* Wait a little for things to settle down */
udelay(50);
-
- sdma_v3_0_print_status((void *)adev);
}
return 0;
@@ -1433,40 +1451,31 @@
return 0;
}
-static void fiji_update_sdma_medium_grain_clock_gating(
+static void sdma_v3_0_update_sdma_medium_grain_clock_gating(
struct amdgpu_device *adev,
bool enable)
{
uint32_t temp, data;
+ int i;
- if (enable) {
- temp = data = RREG32(mmSDMA0_CLK_CTRL);
- data &= ~(SDMA0_CLK_CTRL__SOFT_OVERRIDE7_MASK |
- SDMA0_CLK_CTRL__SOFT_OVERRIDE6_MASK |
- SDMA0_CLK_CTRL__SOFT_OVERRIDE5_MASK |
- SDMA0_CLK_CTRL__SOFT_OVERRIDE4_MASK |
- SDMA0_CLK_CTRL__SOFT_OVERRIDE3_MASK |
- SDMA0_CLK_CTRL__SOFT_OVERRIDE2_MASK |
- SDMA0_CLK_CTRL__SOFT_OVERRIDE1_MASK |
- SDMA0_CLK_CTRL__SOFT_OVERRIDE0_MASK);
- if (data != temp)
- WREG32(mmSDMA0_CLK_CTRL, data);
-
- temp = data = RREG32(mmSDMA1_CLK_CTRL);
- data &= ~(SDMA1_CLK_CTRL__SOFT_OVERRIDE7_MASK |
- SDMA1_CLK_CTRL__SOFT_OVERRIDE6_MASK |
- SDMA1_CLK_CTRL__SOFT_OVERRIDE5_MASK |
- SDMA1_CLK_CTRL__SOFT_OVERRIDE4_MASK |
- SDMA1_CLK_CTRL__SOFT_OVERRIDE3_MASK |
- SDMA1_CLK_CTRL__SOFT_OVERRIDE2_MASK |
- SDMA1_CLK_CTRL__SOFT_OVERRIDE1_MASK |
- SDMA1_CLK_CTRL__SOFT_OVERRIDE0_MASK);
-
- if (data != temp)
- WREG32(mmSDMA1_CLK_CTRL, data);
+ if (enable && (adev->cg_flags & AMD_CG_SUPPORT_SDMA_MGCG)) {
+ for (i = 0; i < adev->sdma.num_instances; i++) {
+ temp = data = RREG32(mmSDMA0_CLK_CTRL + sdma_offsets[i]);
+ data &= ~(SDMA0_CLK_CTRL__SOFT_OVERRIDE7_MASK |
+ SDMA0_CLK_CTRL__SOFT_OVERRIDE6_MASK |
+ SDMA0_CLK_CTRL__SOFT_OVERRIDE5_MASK |
+ SDMA0_CLK_CTRL__SOFT_OVERRIDE4_MASK |
+ SDMA0_CLK_CTRL__SOFT_OVERRIDE3_MASK |
+ SDMA0_CLK_CTRL__SOFT_OVERRIDE2_MASK |
+ SDMA0_CLK_CTRL__SOFT_OVERRIDE1_MASK |
+ SDMA0_CLK_CTRL__SOFT_OVERRIDE0_MASK);
+ if (data != temp)
+ WREG32(mmSDMA0_CLK_CTRL + sdma_offsets[i], data);
+ }
} else {
- temp = data = RREG32(mmSDMA0_CLK_CTRL);
- data |= SDMA0_CLK_CTRL__SOFT_OVERRIDE7_MASK |
+ for (i = 0; i < adev->sdma.num_instances; i++) {
+ temp = data = RREG32(mmSDMA0_CLK_CTRL + sdma_offsets[i]);
+ data |= SDMA0_CLK_CTRL__SOFT_OVERRIDE7_MASK |
SDMA0_CLK_CTRL__SOFT_OVERRIDE6_MASK |
SDMA0_CLK_CTRL__SOFT_OVERRIDE5_MASK |
SDMA0_CLK_CTRL__SOFT_OVERRIDE4_MASK |
@@ -1475,54 +1484,35 @@
SDMA0_CLK_CTRL__SOFT_OVERRIDE1_MASK |
SDMA0_CLK_CTRL__SOFT_OVERRIDE0_MASK;
- if (data != temp)
- WREG32(mmSDMA0_CLK_CTRL, data);
-
- temp = data = RREG32(mmSDMA1_CLK_CTRL);
- data |= SDMA1_CLK_CTRL__SOFT_OVERRIDE7_MASK |
- SDMA1_CLK_CTRL__SOFT_OVERRIDE6_MASK |
- SDMA1_CLK_CTRL__SOFT_OVERRIDE5_MASK |
- SDMA1_CLK_CTRL__SOFT_OVERRIDE4_MASK |
- SDMA1_CLK_CTRL__SOFT_OVERRIDE3_MASK |
- SDMA1_CLK_CTRL__SOFT_OVERRIDE2_MASK |
- SDMA1_CLK_CTRL__SOFT_OVERRIDE1_MASK |
- SDMA1_CLK_CTRL__SOFT_OVERRIDE0_MASK;
-
- if (data != temp)
- WREG32(mmSDMA1_CLK_CTRL, data);
+ if (data != temp)
+ WREG32(mmSDMA0_CLK_CTRL + sdma_offsets[i], data);
+ }
}
}
-static void fiji_update_sdma_medium_grain_light_sleep(
+static void sdma_v3_0_update_sdma_medium_grain_light_sleep(
struct amdgpu_device *adev,
bool enable)
{
uint32_t temp, data;
+ int i;
- if (enable) {
- temp = data = RREG32(mmSDMA0_POWER_CNTL);
- data |= SDMA0_POWER_CNTL__MEM_POWER_OVERRIDE_MASK;
+ if (enable && (adev->cg_flags & AMD_CG_SUPPORT_SDMA_LS)) {
+ for (i = 0; i < adev->sdma.num_instances; i++) {
+ temp = data = RREG32(mmSDMA0_POWER_CNTL + sdma_offsets[i]);
+ data |= SDMA0_POWER_CNTL__MEM_POWER_OVERRIDE_MASK;
- if (temp != data)
- WREG32(mmSDMA0_POWER_CNTL, data);
-
- temp = data = RREG32(mmSDMA1_POWER_CNTL);
- data |= SDMA1_POWER_CNTL__MEM_POWER_OVERRIDE_MASK;
-
- if (temp != data)
- WREG32(mmSDMA1_POWER_CNTL, data);
+ if (temp != data)
+ WREG32(mmSDMA0_POWER_CNTL + sdma_offsets[i], data);
+ }
} else {
- temp = data = RREG32(mmSDMA0_POWER_CNTL);
- data &= ~SDMA0_POWER_CNTL__MEM_POWER_OVERRIDE_MASK;
+ for (i = 0; i < adev->sdma.num_instances; i++) {
+ temp = data = RREG32(mmSDMA0_POWER_CNTL + sdma_offsets[i]);
+ data &= ~SDMA0_POWER_CNTL__MEM_POWER_OVERRIDE_MASK;
- if (temp != data)
- WREG32(mmSDMA0_POWER_CNTL, data);
-
- temp = data = RREG32(mmSDMA1_POWER_CNTL);
- data &= ~SDMA1_POWER_CNTL__MEM_POWER_OVERRIDE_MASK;
-
- if (temp != data)
- WREG32(mmSDMA1_POWER_CNTL, data);
+ if (temp != data)
+ WREG32(mmSDMA0_POWER_CNTL + sdma_offsets[i], data);
+ }
}
}
@@ -1533,9 +1523,11 @@
switch (adev->asic_type) {
case CHIP_FIJI:
- fiji_update_sdma_medium_grain_clock_gating(adev,
+ case CHIP_CARRIZO:
+ case CHIP_STONEY:
+ sdma_v3_0_update_sdma_medium_grain_clock_gating(adev,
state == AMD_CG_STATE_GATE ? true : false);
- fiji_update_sdma_medium_grain_light_sleep(adev,
+ sdma_v3_0_update_sdma_medium_grain_light_sleep(adev,
state == AMD_CG_STATE_GATE ? true : false);
break;
default:
@@ -1551,6 +1543,7 @@
}
const struct amd_ip_funcs sdma_v3_0_ip_funcs = {
+ .name = "sdma_v3_0",
.early_init = sdma_v3_0_early_init,
.late_init = NULL,
.sw_init = sdma_v3_0_sw_init,
@@ -1562,7 +1555,6 @@
.is_idle = sdma_v3_0_is_idle,
.wait_for_idle = sdma_v3_0_wait_for_idle,
.soft_reset = sdma_v3_0_soft_reset,
- .print_status = sdma_v3_0_print_status,
.set_clockgating_state = sdma_v3_0_set_clockgating_state,
.set_powergating_state = sdma_v3_0_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/smu_ucode_xfer_vi.h b/drivers/gpu/drm/amd/amdgpu/smu_ucode_xfer_vi.h
index c24a81e..880152c 100644
--- a/drivers/gpu/drm/amd/amdgpu/smu_ucode_xfer_vi.h
+++ b/drivers/gpu/drm/amd/amdgpu/smu_ucode_xfer_vi.h
@@ -44,6 +44,7 @@
#define UCODE_ID_IH_REG_RESTORE 11
#define UCODE_ID_VBIOS 12
#define UCODE_ID_MISC_METADATA 13
+#define UCODE_ID_SMU_SK 14
#define UCODE_ID_RLC_SCRATCH 32
#define UCODE_ID_RLC_SRM_ARAM 33
#define UCODE_ID_RLC_SRM_DRAM 34
diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_dpm.c b/drivers/gpu/drm/amd/amdgpu/tonga_dpm.c
index 0497784..b7615ce 100644
--- a/drivers/gpu/drm/amd/amdgpu/tonga_dpm.c
+++ b/drivers/gpu/drm/amd/amdgpu/tonga_dpm.c
@@ -143,6 +143,7 @@
}
const struct amd_ip_funcs tonga_dpm_ip_funcs = {
+ .name = "tonga_dpm",
.early_init = tonga_dpm_early_init,
.late_init = NULL,
.sw_init = tonga_dpm_sw_init,
@@ -154,7 +155,6 @@
.is_idle = NULL,
.wait_for_idle = NULL,
.soft_reset = NULL,
- .print_status = NULL,
.set_clockgating_state = tonga_dpm_set_clockgating_state,
.set_powergating_state = tonga_dpm_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
index 0f14199..f036af93 100644
--- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
@@ -374,35 +374,6 @@
return -ETIMEDOUT;
}
-static void tonga_ih_print_status(void *handle)
-{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- dev_info(adev->dev, "TONGA IH registers\n");
- dev_info(adev->dev, " SRBM_STATUS=0x%08X\n",
- RREG32(mmSRBM_STATUS));
- dev_info(adev->dev, " SRBM_STATUS2=0x%08X\n",
- RREG32(mmSRBM_STATUS2));
- dev_info(adev->dev, " INTERRUPT_CNTL=0x%08X\n",
- RREG32(mmINTERRUPT_CNTL));
- dev_info(adev->dev, " INTERRUPT_CNTL2=0x%08X\n",
- RREG32(mmINTERRUPT_CNTL2));
- dev_info(adev->dev, " IH_CNTL=0x%08X\n",
- RREG32(mmIH_CNTL));
- dev_info(adev->dev, " IH_RB_CNTL=0x%08X\n",
- RREG32(mmIH_RB_CNTL));
- dev_info(adev->dev, " IH_RB_BASE=0x%08X\n",
- RREG32(mmIH_RB_BASE));
- dev_info(adev->dev, " IH_RB_WPTR_ADDR_LO=0x%08X\n",
- RREG32(mmIH_RB_WPTR_ADDR_LO));
- dev_info(adev->dev, " IH_RB_WPTR_ADDR_HI=0x%08X\n",
- RREG32(mmIH_RB_WPTR_ADDR_HI));
- dev_info(adev->dev, " IH_RB_RPTR=0x%08X\n",
- RREG32(mmIH_RB_RPTR));
- dev_info(adev->dev, " IH_RB_WPTR=0x%08X\n",
- RREG32(mmIH_RB_WPTR));
-}
-
static int tonga_ih_soft_reset(void *handle)
{
u32 srbm_soft_reset = 0;
@@ -414,8 +385,6 @@
SOFT_RESET_IH, 1);
if (srbm_soft_reset) {
- tonga_ih_print_status(adev);
-
tmp = RREG32(mmSRBM_SOFT_RESET);
tmp |= srbm_soft_reset;
dev_info(adev->dev, "SRBM_SOFT_RESET=0x%08X\n", tmp);
@@ -430,8 +399,6 @@
/* Wait a little for things to settle down */
udelay(50);
-
- tonga_ih_print_status(adev);
}
return 0;
@@ -450,6 +417,7 @@
}
const struct amd_ip_funcs tonga_ih_ip_funcs = {
+ .name = "tonga_ih",
.early_init = tonga_ih_early_init,
.late_init = NULL,
.sw_init = tonga_ih_sw_init,
@@ -461,7 +429,6 @@
.is_idle = tonga_ih_is_idle,
.wait_for_idle = tonga_ih_wait_for_idle,
.soft_reset = tonga_ih_soft_reset,
- .print_status = tonga_ih_print_status,
.set_clockgating_state = tonga_ih_set_clockgating_state,
.set_powergating_state = tonga_ih_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/uvd_v4_2.c b/drivers/gpu/drm/amd/amdgpu/uvd_v4_2.c
index cb46375..f075514 100644
--- a/drivers/gpu/drm/amd/amdgpu/uvd_v4_2.c
+++ b/drivers/gpu/drm/amd/amdgpu/uvd_v4_2.c
@@ -114,7 +114,7 @@
ring = &adev->uvd.ring;
sprintf(ring->name, "uvd");
- r = amdgpu_ring_init(adev, ring, 4096, CP_PACKET2, 0xf,
+ r = amdgpu_ring_init(adev, ring, 512, CP_PACKET2, 0xf,
&adev->uvd.irq, 0, AMDGPU_RING_TYPE_UVD);
return r;
@@ -489,7 +489,8 @@
* Write ring commands to execute the indirect buffer
*/
static void uvd_v4_2_ring_emit_ib(struct amdgpu_ring *ring,
- struct amdgpu_ib *ib)
+ struct amdgpu_ib *ib,
+ unsigned vm_id, bool ctx_switch)
{
amdgpu_ring_write(ring, PACKET0(mmUVD_RBC_IB_BASE, 0));
amdgpu_ring_write(ring, ib->gpu_addr);
@@ -559,12 +560,13 @@
WREG32(mmUVD_VCPU_CACHE_SIZE0, size);
addr += size;
- size = AMDGPU_UVD_STACK_SIZE >> 3;
+ size = AMDGPU_UVD_HEAP_SIZE >> 3;
WREG32(mmUVD_VCPU_CACHE_OFFSET1, addr);
WREG32(mmUVD_VCPU_CACHE_SIZE1, size);
addr += size;
- size = AMDGPU_UVD_HEAP_SIZE >> 3;
+ size = (AMDGPU_UVD_STACK_SIZE +
+ (AMDGPU_UVD_SESSION_SIZE * adev->uvd.max_handles)) >> 3;
WREG32(mmUVD_VCPU_CACHE_OFFSET2, addr);
WREG32(mmUVD_VCPU_CACHE_SIZE2, size);
@@ -679,117 +681,6 @@
return uvd_v4_2_start(adev);
}
-static void uvd_v4_2_print_status(void *handle)
-{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- dev_info(adev->dev, "UVD 4.2 registers\n");
- dev_info(adev->dev, " UVD_SEMA_ADDR_LOW=0x%08X\n",
- RREG32(mmUVD_SEMA_ADDR_LOW));
- dev_info(adev->dev, " UVD_SEMA_ADDR_HIGH=0x%08X\n",
- RREG32(mmUVD_SEMA_ADDR_HIGH));
- dev_info(adev->dev, " UVD_SEMA_CMD=0x%08X\n",
- RREG32(mmUVD_SEMA_CMD));
- dev_info(adev->dev, " UVD_GPCOM_VCPU_CMD=0x%08X\n",
- RREG32(mmUVD_GPCOM_VCPU_CMD));
- dev_info(adev->dev, " UVD_GPCOM_VCPU_DATA0=0x%08X\n",
- RREG32(mmUVD_GPCOM_VCPU_DATA0));
- dev_info(adev->dev, " UVD_GPCOM_VCPU_DATA1=0x%08X\n",
- RREG32(mmUVD_GPCOM_VCPU_DATA1));
- dev_info(adev->dev, " UVD_ENGINE_CNTL=0x%08X\n",
- RREG32(mmUVD_ENGINE_CNTL));
- dev_info(adev->dev, " UVD_UDEC_ADDR_CONFIG=0x%08X\n",
- RREG32(mmUVD_UDEC_ADDR_CONFIG));
- dev_info(adev->dev, " UVD_UDEC_DB_ADDR_CONFIG=0x%08X\n",
- RREG32(mmUVD_UDEC_DB_ADDR_CONFIG));
- dev_info(adev->dev, " UVD_UDEC_DBW_ADDR_CONFIG=0x%08X\n",
- RREG32(mmUVD_UDEC_DBW_ADDR_CONFIG));
- dev_info(adev->dev, " UVD_SEMA_CNTL=0x%08X\n",
- RREG32(mmUVD_SEMA_CNTL));
- dev_info(adev->dev, " UVD_LMI_EXT40_ADDR=0x%08X\n",
- RREG32(mmUVD_LMI_EXT40_ADDR));
- dev_info(adev->dev, " UVD_CTX_INDEX=0x%08X\n",
- RREG32(mmUVD_CTX_INDEX));
- dev_info(adev->dev, " UVD_CTX_DATA=0x%08X\n",
- RREG32(mmUVD_CTX_DATA));
- dev_info(adev->dev, " UVD_CGC_GATE=0x%08X\n",
- RREG32(mmUVD_CGC_GATE));
- dev_info(adev->dev, " UVD_CGC_CTRL=0x%08X\n",
- RREG32(mmUVD_CGC_CTRL));
- dev_info(adev->dev, " UVD_LMI_CTRL2=0x%08X\n",
- RREG32(mmUVD_LMI_CTRL2));
- dev_info(adev->dev, " UVD_MASTINT_EN=0x%08X\n",
- RREG32(mmUVD_MASTINT_EN));
- dev_info(adev->dev, " UVD_LMI_ADDR_EXT=0x%08X\n",
- RREG32(mmUVD_LMI_ADDR_EXT));
- dev_info(adev->dev, " UVD_LMI_CTRL=0x%08X\n",
- RREG32(mmUVD_LMI_CTRL));
- dev_info(adev->dev, " UVD_LMI_SWAP_CNTL=0x%08X\n",
- RREG32(mmUVD_LMI_SWAP_CNTL));
- dev_info(adev->dev, " UVD_MP_SWAP_CNTL=0x%08X\n",
- RREG32(mmUVD_MP_SWAP_CNTL));
- dev_info(adev->dev, " UVD_MPC_SET_MUXA0=0x%08X\n",
- RREG32(mmUVD_MPC_SET_MUXA0));
- dev_info(adev->dev, " UVD_MPC_SET_MUXA1=0x%08X\n",
- RREG32(mmUVD_MPC_SET_MUXA1));
- dev_info(adev->dev, " UVD_MPC_SET_MUXB0=0x%08X\n",
- RREG32(mmUVD_MPC_SET_MUXB0));
- dev_info(adev->dev, " UVD_MPC_SET_MUXB1=0x%08X\n",
- RREG32(mmUVD_MPC_SET_MUXB1));
- dev_info(adev->dev, " UVD_MPC_SET_MUX=0x%08X\n",
- RREG32(mmUVD_MPC_SET_MUX));
- dev_info(adev->dev, " UVD_MPC_SET_ALU=0x%08X\n",
- RREG32(mmUVD_MPC_SET_ALU));
- dev_info(adev->dev, " UVD_VCPU_CACHE_OFFSET0=0x%08X\n",
- RREG32(mmUVD_VCPU_CACHE_OFFSET0));
- dev_info(adev->dev, " UVD_VCPU_CACHE_SIZE0=0x%08X\n",
- RREG32(mmUVD_VCPU_CACHE_SIZE0));
- dev_info(adev->dev, " UVD_VCPU_CACHE_OFFSET1=0x%08X\n",
- RREG32(mmUVD_VCPU_CACHE_OFFSET1));
- dev_info(adev->dev, " UVD_VCPU_CACHE_SIZE1=0x%08X\n",
- RREG32(mmUVD_VCPU_CACHE_SIZE1));
- dev_info(adev->dev, " UVD_VCPU_CACHE_OFFSET2=0x%08X\n",
- RREG32(mmUVD_VCPU_CACHE_OFFSET2));
- dev_info(adev->dev, " UVD_VCPU_CACHE_SIZE2=0x%08X\n",
- RREG32(mmUVD_VCPU_CACHE_SIZE2));
- dev_info(adev->dev, " UVD_VCPU_CNTL=0x%08X\n",
- RREG32(mmUVD_VCPU_CNTL));
- dev_info(adev->dev, " UVD_SOFT_RESET=0x%08X\n",
- RREG32(mmUVD_SOFT_RESET));
- dev_info(adev->dev, " UVD_RBC_IB_BASE=0x%08X\n",
- RREG32(mmUVD_RBC_IB_BASE));
- dev_info(adev->dev, " UVD_RBC_IB_SIZE=0x%08X\n",
- RREG32(mmUVD_RBC_IB_SIZE));
- dev_info(adev->dev, " UVD_RBC_RB_BASE=0x%08X\n",
- RREG32(mmUVD_RBC_RB_BASE));
- dev_info(adev->dev, " UVD_RBC_RB_RPTR=0x%08X\n",
- RREG32(mmUVD_RBC_RB_RPTR));
- dev_info(adev->dev, " UVD_RBC_RB_WPTR=0x%08X\n",
- RREG32(mmUVD_RBC_RB_WPTR));
- dev_info(adev->dev, " UVD_RBC_RB_WPTR_CNTL=0x%08X\n",
- RREG32(mmUVD_RBC_RB_WPTR_CNTL));
- dev_info(adev->dev, " UVD_RBC_RB_CNTL=0x%08X\n",
- RREG32(mmUVD_RBC_RB_CNTL));
- dev_info(adev->dev, " UVD_STATUS=0x%08X\n",
- RREG32(mmUVD_STATUS));
- dev_info(adev->dev, " UVD_SEMA_TIMEOUT_STATUS=0x%08X\n",
- RREG32(mmUVD_SEMA_TIMEOUT_STATUS));
- dev_info(adev->dev, " UVD_SEMA_WAIT_INCOMPLETE_TIMEOUT_CNTL=0x%08X\n",
- RREG32(mmUVD_SEMA_WAIT_INCOMPLETE_TIMEOUT_CNTL));
- dev_info(adev->dev, " UVD_SEMA_WAIT_FAULT_TIMEOUT_CNTL=0x%08X\n",
- RREG32(mmUVD_SEMA_WAIT_FAULT_TIMEOUT_CNTL));
- dev_info(adev->dev, " UVD_SEMA_SIGNAL_INCOMPLETE_TIMEOUT_CNTL=0x%08X\n",
- RREG32(mmUVD_SEMA_SIGNAL_INCOMPLETE_TIMEOUT_CNTL));
- dev_info(adev->dev, " UVD_CONTEXT_ID=0x%08X\n",
- RREG32(mmUVD_CONTEXT_ID));
- dev_info(adev->dev, " UVD_UDEC_ADDR_CONFIG=0x%08X\n",
- RREG32(mmUVD_UDEC_ADDR_CONFIG));
- dev_info(adev->dev, " UVD_UDEC_DB_ADDR_CONFIG=0x%08X\n",
- RREG32(mmUVD_UDEC_DB_ADDR_CONFIG));
- dev_info(adev->dev, " UVD_UDEC_DBW_ADDR_CONFIG=0x%08X\n",
- RREG32(mmUVD_UDEC_DBW_ADDR_CONFIG));
-
-}
-
static int uvd_v4_2_set_interrupt_state(struct amdgpu_device *adev,
struct amdgpu_irq_src *source,
unsigned type,
@@ -849,6 +740,7 @@
}
const struct amd_ip_funcs uvd_v4_2_ip_funcs = {
+ .name = "uvd_v4_2",
.early_init = uvd_v4_2_early_init,
.late_init = NULL,
.sw_init = uvd_v4_2_sw_init,
@@ -860,7 +752,6 @@
.is_idle = uvd_v4_2_is_idle,
.wait_for_idle = uvd_v4_2_wait_for_idle,
.soft_reset = uvd_v4_2_soft_reset,
- .print_status = uvd_v4_2_print_status,
.set_clockgating_state = uvd_v4_2_set_clockgating_state,
.set_powergating_state = uvd_v4_2_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c b/drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c
index 16476d8..e0a76a8 100644
--- a/drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c
@@ -31,6 +31,7 @@
#include "uvd/uvd_5_0_sh_mask.h"
#include "oss/oss_2_0_d.h"
#include "oss/oss_2_0_sh_mask.h"
+#include "vi.h"
static void uvd_v5_0_set_ring_funcs(struct amdgpu_device *adev);
static void uvd_v5_0_set_irq_funcs(struct amdgpu_device *adev);
@@ -110,7 +111,7 @@
ring = &adev->uvd.ring;
sprintf(ring->name, "uvd");
- r = amdgpu_ring_init(adev, ring, 4096, CP_PACKET2, 0xf,
+ r = amdgpu_ring_init(adev, ring, 512, CP_PACKET2, 0xf,
&adev->uvd.irq, 0, AMDGPU_RING_TYPE_UVD);
return r;
@@ -271,12 +272,13 @@
WREG32(mmUVD_VCPU_CACHE_SIZE0, size);
offset += size;
- size = AMDGPU_UVD_STACK_SIZE;
+ size = AMDGPU_UVD_HEAP_SIZE;
WREG32(mmUVD_VCPU_CACHE_OFFSET1, offset >> 3);
WREG32(mmUVD_VCPU_CACHE_SIZE1, size);
offset += size;
- size = AMDGPU_UVD_HEAP_SIZE;
+ size = AMDGPU_UVD_STACK_SIZE +
+ (AMDGPU_UVD_SESSION_SIZE * adev->uvd.max_handles);
WREG32(mmUVD_VCPU_CACHE_OFFSET2, offset >> 3);
WREG32(mmUVD_VCPU_CACHE_SIZE2, size);
@@ -537,7 +539,8 @@
* Write ring commands to execute the indirect buffer
*/
static void uvd_v5_0_ring_emit_ib(struct amdgpu_ring *ring,
- struct amdgpu_ib *ib)
+ struct amdgpu_ib *ib,
+ unsigned vm_id, bool ctx_switch)
{
amdgpu_ring_write(ring, PACKET0(mmUVD_LMI_RBC_IB_64BIT_BAR_LOW, 0));
amdgpu_ring_write(ring, lower_32_bits(ib->gpu_addr));
@@ -622,120 +625,6 @@
return uvd_v5_0_start(adev);
}
-static void uvd_v5_0_print_status(void *handle)
-{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- dev_info(adev->dev, "UVD 5.0 registers\n");
- dev_info(adev->dev, " UVD_SEMA_ADDR_LOW=0x%08X\n",
- RREG32(mmUVD_SEMA_ADDR_LOW));
- dev_info(adev->dev, " UVD_SEMA_ADDR_HIGH=0x%08X\n",
- RREG32(mmUVD_SEMA_ADDR_HIGH));
- dev_info(adev->dev, " UVD_SEMA_CMD=0x%08X\n",
- RREG32(mmUVD_SEMA_CMD));
- dev_info(adev->dev, " UVD_GPCOM_VCPU_CMD=0x%08X\n",
- RREG32(mmUVD_GPCOM_VCPU_CMD));
- dev_info(adev->dev, " UVD_GPCOM_VCPU_DATA0=0x%08X\n",
- RREG32(mmUVD_GPCOM_VCPU_DATA0));
- dev_info(adev->dev, " UVD_GPCOM_VCPU_DATA1=0x%08X\n",
- RREG32(mmUVD_GPCOM_VCPU_DATA1));
- dev_info(adev->dev, " UVD_ENGINE_CNTL=0x%08X\n",
- RREG32(mmUVD_ENGINE_CNTL));
- dev_info(adev->dev, " UVD_UDEC_ADDR_CONFIG=0x%08X\n",
- RREG32(mmUVD_UDEC_ADDR_CONFIG));
- dev_info(adev->dev, " UVD_UDEC_DB_ADDR_CONFIG=0x%08X\n",
- RREG32(mmUVD_UDEC_DB_ADDR_CONFIG));
- dev_info(adev->dev, " UVD_UDEC_DBW_ADDR_CONFIG=0x%08X\n",
- RREG32(mmUVD_UDEC_DBW_ADDR_CONFIG));
- dev_info(adev->dev, " UVD_SEMA_CNTL=0x%08X\n",
- RREG32(mmUVD_SEMA_CNTL));
- dev_info(adev->dev, " UVD_LMI_EXT40_ADDR=0x%08X\n",
- RREG32(mmUVD_LMI_EXT40_ADDR));
- dev_info(adev->dev, " UVD_CTX_INDEX=0x%08X\n",
- RREG32(mmUVD_CTX_INDEX));
- dev_info(adev->dev, " UVD_CTX_DATA=0x%08X\n",
- RREG32(mmUVD_CTX_DATA));
- dev_info(adev->dev, " UVD_CGC_GATE=0x%08X\n",
- RREG32(mmUVD_CGC_GATE));
- dev_info(adev->dev, " UVD_CGC_CTRL=0x%08X\n",
- RREG32(mmUVD_CGC_CTRL));
- dev_info(adev->dev, " UVD_LMI_CTRL2=0x%08X\n",
- RREG32(mmUVD_LMI_CTRL2));
- dev_info(adev->dev, " UVD_MASTINT_EN=0x%08X\n",
- RREG32(mmUVD_MASTINT_EN));
- dev_info(adev->dev, " UVD_LMI_ADDR_EXT=0x%08X\n",
- RREG32(mmUVD_LMI_ADDR_EXT));
- dev_info(adev->dev, " UVD_LMI_CTRL=0x%08X\n",
- RREG32(mmUVD_LMI_CTRL));
- dev_info(adev->dev, " UVD_LMI_SWAP_CNTL=0x%08X\n",
- RREG32(mmUVD_LMI_SWAP_CNTL));
- dev_info(adev->dev, " UVD_MP_SWAP_CNTL=0x%08X\n",
- RREG32(mmUVD_MP_SWAP_CNTL));
- dev_info(adev->dev, " UVD_MPC_SET_MUXA0=0x%08X\n",
- RREG32(mmUVD_MPC_SET_MUXA0));
- dev_info(adev->dev, " UVD_MPC_SET_MUXA1=0x%08X\n",
- RREG32(mmUVD_MPC_SET_MUXA1));
- dev_info(adev->dev, " UVD_MPC_SET_MUXB0=0x%08X\n",
- RREG32(mmUVD_MPC_SET_MUXB0));
- dev_info(adev->dev, " UVD_MPC_SET_MUXB1=0x%08X\n",
- RREG32(mmUVD_MPC_SET_MUXB1));
- dev_info(adev->dev, " UVD_MPC_SET_MUX=0x%08X\n",
- RREG32(mmUVD_MPC_SET_MUX));
- dev_info(adev->dev, " UVD_MPC_SET_ALU=0x%08X\n",
- RREG32(mmUVD_MPC_SET_ALU));
- dev_info(adev->dev, " UVD_VCPU_CACHE_OFFSET0=0x%08X\n",
- RREG32(mmUVD_VCPU_CACHE_OFFSET0));
- dev_info(adev->dev, " UVD_VCPU_CACHE_SIZE0=0x%08X\n",
- RREG32(mmUVD_VCPU_CACHE_SIZE0));
- dev_info(adev->dev, " UVD_VCPU_CACHE_OFFSET1=0x%08X\n",
- RREG32(mmUVD_VCPU_CACHE_OFFSET1));
- dev_info(adev->dev, " UVD_VCPU_CACHE_SIZE1=0x%08X\n",
- RREG32(mmUVD_VCPU_CACHE_SIZE1));
- dev_info(adev->dev, " UVD_VCPU_CACHE_OFFSET2=0x%08X\n",
- RREG32(mmUVD_VCPU_CACHE_OFFSET2));
- dev_info(adev->dev, " UVD_VCPU_CACHE_SIZE2=0x%08X\n",
- RREG32(mmUVD_VCPU_CACHE_SIZE2));
- dev_info(adev->dev, " UVD_VCPU_CNTL=0x%08X\n",
- RREG32(mmUVD_VCPU_CNTL));
- dev_info(adev->dev, " UVD_SOFT_RESET=0x%08X\n",
- RREG32(mmUVD_SOFT_RESET));
- dev_info(adev->dev, " UVD_LMI_RBC_IB_64BIT_BAR_LOW=0x%08X\n",
- RREG32(mmUVD_LMI_RBC_IB_64BIT_BAR_LOW));
- dev_info(adev->dev, " UVD_LMI_RBC_IB_64BIT_BAR_HIGH=0x%08X\n",
- RREG32(mmUVD_LMI_RBC_IB_64BIT_BAR_HIGH));
- dev_info(adev->dev, " UVD_RBC_IB_SIZE=0x%08X\n",
- RREG32(mmUVD_RBC_IB_SIZE));
- dev_info(adev->dev, " UVD_LMI_RBC_RB_64BIT_BAR_LOW=0x%08X\n",
- RREG32(mmUVD_LMI_RBC_RB_64BIT_BAR_LOW));
- dev_info(adev->dev, " UVD_LMI_RBC_RB_64BIT_BAR_HIGH=0x%08X\n",
- RREG32(mmUVD_LMI_RBC_RB_64BIT_BAR_HIGH));
- dev_info(adev->dev, " UVD_RBC_RB_RPTR=0x%08X\n",
- RREG32(mmUVD_RBC_RB_RPTR));
- dev_info(adev->dev, " UVD_RBC_RB_WPTR=0x%08X\n",
- RREG32(mmUVD_RBC_RB_WPTR));
- dev_info(adev->dev, " UVD_RBC_RB_WPTR_CNTL=0x%08X\n",
- RREG32(mmUVD_RBC_RB_WPTR_CNTL));
- dev_info(adev->dev, " UVD_RBC_RB_CNTL=0x%08X\n",
- RREG32(mmUVD_RBC_RB_CNTL));
- dev_info(adev->dev, " UVD_STATUS=0x%08X\n",
- RREG32(mmUVD_STATUS));
- dev_info(adev->dev, " UVD_SEMA_TIMEOUT_STATUS=0x%08X\n",
- RREG32(mmUVD_SEMA_TIMEOUT_STATUS));
- dev_info(adev->dev, " UVD_SEMA_WAIT_INCOMPLETE_TIMEOUT_CNTL=0x%08X\n",
- RREG32(mmUVD_SEMA_WAIT_INCOMPLETE_TIMEOUT_CNTL));
- dev_info(adev->dev, " UVD_SEMA_WAIT_FAULT_TIMEOUT_CNTL=0x%08X\n",
- RREG32(mmUVD_SEMA_WAIT_FAULT_TIMEOUT_CNTL));
- dev_info(adev->dev, " UVD_SEMA_SIGNAL_INCOMPLETE_TIMEOUT_CNTL=0x%08X\n",
- RREG32(mmUVD_SEMA_SIGNAL_INCOMPLETE_TIMEOUT_CNTL));
- dev_info(adev->dev, " UVD_CONTEXT_ID=0x%08X\n",
- RREG32(mmUVD_CONTEXT_ID));
- dev_info(adev->dev, " UVD_UDEC_ADDR_CONFIG=0x%08X\n",
- RREG32(mmUVD_UDEC_ADDR_CONFIG));
- dev_info(adev->dev, " UVD_UDEC_DB_ADDR_CONFIG=0x%08X\n",
- RREG32(mmUVD_UDEC_DB_ADDR_CONFIG));
- dev_info(adev->dev, " UVD_UDEC_DBW_ADDR_CONFIG=0x%08X\n",
- RREG32(mmUVD_UDEC_DBW_ADDR_CONFIG));
-}
-
static int uvd_v5_0_set_interrupt_state(struct amdgpu_device *adev,
struct amdgpu_irq_src *source,
unsigned type,
@@ -754,14 +643,128 @@
return 0;
}
+static void uvd_v5_0_set_sw_clock_gating(struct amdgpu_device *adev)
+{
+ uint32_t data, data1, data2, suvd_flags;
+
+ data = RREG32(mmUVD_CGC_CTRL);
+ data1 = RREG32(mmUVD_SUVD_CGC_GATE);
+ data2 = RREG32(mmUVD_SUVD_CGC_CTRL);
+
+ data &= ~(UVD_CGC_CTRL__CLK_OFF_DELAY_MASK |
+ UVD_CGC_CTRL__CLK_GATE_DLY_TIMER_MASK);
+
+ suvd_flags = UVD_SUVD_CGC_GATE__SRE_MASK |
+ UVD_SUVD_CGC_GATE__SIT_MASK |
+ UVD_SUVD_CGC_GATE__SMP_MASK |
+ UVD_SUVD_CGC_GATE__SCM_MASK |
+ UVD_SUVD_CGC_GATE__SDB_MASK;
+
+ data |= UVD_CGC_CTRL__DYN_CLOCK_MODE_MASK |
+ (1 << REG_FIELD_SHIFT(UVD_CGC_CTRL, CLK_GATE_DLY_TIMER)) |
+ (4 << REG_FIELD_SHIFT(UVD_CGC_CTRL, CLK_OFF_DELAY));
+
+ data &= ~(UVD_CGC_CTRL__UDEC_RE_MODE_MASK |
+ UVD_CGC_CTRL__UDEC_CM_MODE_MASK |
+ UVD_CGC_CTRL__UDEC_IT_MODE_MASK |
+ UVD_CGC_CTRL__UDEC_DB_MODE_MASK |
+ UVD_CGC_CTRL__UDEC_MP_MODE_MASK |
+ UVD_CGC_CTRL__SYS_MODE_MASK |
+ UVD_CGC_CTRL__UDEC_MODE_MASK |
+ UVD_CGC_CTRL__MPEG2_MODE_MASK |
+ UVD_CGC_CTRL__REGS_MODE_MASK |
+ UVD_CGC_CTRL__RBC_MODE_MASK |
+ UVD_CGC_CTRL__LMI_MC_MODE_MASK |
+ UVD_CGC_CTRL__LMI_UMC_MODE_MASK |
+ UVD_CGC_CTRL__IDCT_MODE_MASK |
+ UVD_CGC_CTRL__MPRD_MODE_MASK |
+ UVD_CGC_CTRL__MPC_MODE_MASK |
+ UVD_CGC_CTRL__LBSI_MODE_MASK |
+ UVD_CGC_CTRL__LRBBM_MODE_MASK |
+ UVD_CGC_CTRL__WCB_MODE_MASK |
+ UVD_CGC_CTRL__VCPU_MODE_MASK |
+ UVD_CGC_CTRL__JPEG_MODE_MASK |
+ UVD_CGC_CTRL__SCPU_MODE_MASK);
+ data2 &= ~(UVD_SUVD_CGC_CTRL__SRE_MODE_MASK |
+ UVD_SUVD_CGC_CTRL__SIT_MODE_MASK |
+ UVD_SUVD_CGC_CTRL__SMP_MODE_MASK |
+ UVD_SUVD_CGC_CTRL__SCM_MODE_MASK |
+ UVD_SUVD_CGC_CTRL__SDB_MODE_MASK);
+ data1 |= suvd_flags;
+
+ WREG32(mmUVD_CGC_CTRL, data);
+ WREG32(mmUVD_CGC_GATE, 0);
+ WREG32(mmUVD_SUVD_CGC_GATE, data1);
+ WREG32(mmUVD_SUVD_CGC_CTRL, data2);
+}
+
+#if 0
+static void uvd_v5_0_set_hw_clock_gating(struct amdgpu_device *adev)
+{
+ uint32_t data, data1, cgc_flags, suvd_flags;
+
+ data = RREG32(mmUVD_CGC_GATE);
+ data1 = RREG32(mmUVD_SUVD_CGC_GATE);
+
+ cgc_flags = UVD_CGC_GATE__SYS_MASK |
+ UVD_CGC_GATE__UDEC_MASK |
+ UVD_CGC_GATE__MPEG2_MASK |
+ UVD_CGC_GATE__RBC_MASK |
+ UVD_CGC_GATE__LMI_MC_MASK |
+ UVD_CGC_GATE__IDCT_MASK |
+ UVD_CGC_GATE__MPRD_MASK |
+ UVD_CGC_GATE__MPC_MASK |
+ UVD_CGC_GATE__LBSI_MASK |
+ UVD_CGC_GATE__LRBBM_MASK |
+ UVD_CGC_GATE__UDEC_RE_MASK |
+ UVD_CGC_GATE__UDEC_CM_MASK |
+ UVD_CGC_GATE__UDEC_IT_MASK |
+ UVD_CGC_GATE__UDEC_DB_MASK |
+ UVD_CGC_GATE__UDEC_MP_MASK |
+ UVD_CGC_GATE__WCB_MASK |
+ UVD_CGC_GATE__VCPU_MASK |
+ UVD_CGC_GATE__SCPU_MASK;
+
+ suvd_flags = UVD_SUVD_CGC_GATE__SRE_MASK |
+ UVD_SUVD_CGC_GATE__SIT_MASK |
+ UVD_SUVD_CGC_GATE__SMP_MASK |
+ UVD_SUVD_CGC_GATE__SCM_MASK |
+ UVD_SUVD_CGC_GATE__SDB_MASK;
+
+ data |= cgc_flags;
+ data1 |= suvd_flags;
+
+ WREG32(mmUVD_CGC_GATE, data);
+ WREG32(mmUVD_SUVD_CGC_GATE, data1);
+}
+#endif
+
static int uvd_v5_0_set_clockgating_state(void *handle,
enum amd_clockgating_state state)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ bool enable = (state == AMD_CG_STATE_GATE) ? true : false;
+ static int curstate = -1;
if (!(adev->cg_flags & AMD_CG_SUPPORT_UVD_MGCG))
return 0;
+ if (curstate == state)
+ return 0;
+
+ curstate = state;
+ if (enable) {
+ /* disable HW gating and enable Sw gating */
+ uvd_v5_0_set_sw_clock_gating(adev);
+ } else {
+ /* wait for STATUS to clear */
+ if (uvd_v5_0_wait_for_idle(handle))
+ return -EBUSY;
+
+ /* enable HW gates because UVD is idle */
+/* uvd_v5_0_set_hw_clock_gating(adev); */
+ }
+
return 0;
}
@@ -789,6 +792,7 @@
}
const struct amd_ip_funcs uvd_v5_0_ip_funcs = {
+ .name = "uvd_v5_0",
.early_init = uvd_v5_0_early_init,
.late_init = NULL,
.sw_init = uvd_v5_0_sw_init,
@@ -800,7 +804,6 @@
.is_idle = uvd_v5_0_is_idle,
.wait_for_idle = uvd_v5_0_wait_for_idle,
.soft_reset = uvd_v5_0_soft_reset,
- .print_status = uvd_v5_0_print_status,
.set_clockgating_state = uvd_v5_0_set_clockgating_state,
.set_powergating_state = uvd_v5_0_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c b/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c
index d493791..c9929d6 100644
--- a/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c
@@ -31,11 +31,15 @@
#include "uvd/uvd_6_0_sh_mask.h"
#include "oss/oss_2_0_d.h"
#include "oss/oss_2_0_sh_mask.h"
+#include "smu/smu_7_1_3_d.h"
+#include "smu/smu_7_1_3_sh_mask.h"
+#include "vi.h"
static void uvd_v6_0_set_ring_funcs(struct amdgpu_device *adev);
static void uvd_v6_0_set_irq_funcs(struct amdgpu_device *adev);
static int uvd_v6_0_start(struct amdgpu_device *adev);
static void uvd_v6_0_stop(struct amdgpu_device *adev);
+static void uvd_v6_0_set_sw_clock_gating(struct amdgpu_device *adev);
/**
* uvd_v6_0_ring_get_rptr - get read pointer
@@ -110,7 +114,7 @@
ring = &adev->uvd.ring;
sprintf(ring->name, "uvd");
- r = amdgpu_ring_init(adev, ring, 4096, CP_PACKET2, 0xf,
+ r = amdgpu_ring_init(adev, ring, 512, CP_PACKET2, 0xf,
&adev->uvd.irq, 0, AMDGPU_RING_TYPE_UVD);
return r;
@@ -270,20 +274,24 @@
WREG32(mmUVD_VCPU_CACHE_SIZE0, size);
offset += size;
- size = AMDGPU_UVD_STACK_SIZE;
+ size = AMDGPU_UVD_HEAP_SIZE;
WREG32(mmUVD_VCPU_CACHE_OFFSET1, offset >> 3);
WREG32(mmUVD_VCPU_CACHE_SIZE1, size);
offset += size;
- size = AMDGPU_UVD_HEAP_SIZE;
+ size = AMDGPU_UVD_STACK_SIZE +
+ (AMDGPU_UVD_SESSION_SIZE * adev->uvd.max_handles);
WREG32(mmUVD_VCPU_CACHE_OFFSET2, offset >> 3);
WREG32(mmUVD_VCPU_CACHE_SIZE2, size);
WREG32(mmUVD_UDEC_ADDR_CONFIG, adev->gfx.config.gb_addr_config);
WREG32(mmUVD_UDEC_DB_ADDR_CONFIG, adev->gfx.config.gb_addr_config);
WREG32(mmUVD_UDEC_DBW_ADDR_CONFIG, adev->gfx.config.gb_addr_config);
+
+ WREG32(mmUVD_GP_SCRATCH4, adev->uvd.max_handles);
}
+#if 0
static void cz_set_uvd_clock_gating_branches(struct amdgpu_device *adev,
bool enable)
{
@@ -360,157 +368,7 @@
WREG32(mmUVD_CGC_GATE, data);
WREG32(mmUVD_SUVD_CGC_GATE, data1);
}
-
-static void tonga_set_uvd_clock_gating_branches(struct amdgpu_device *adev,
- bool enable)
-{
- u32 data, data1;
-
- data = RREG32(mmUVD_CGC_GATE);
- data1 = RREG32(mmUVD_SUVD_CGC_GATE);
- if (enable) {
- data |= UVD_CGC_GATE__SYS_MASK |
- UVD_CGC_GATE__UDEC_MASK |
- UVD_CGC_GATE__MPEG2_MASK |
- UVD_CGC_GATE__RBC_MASK |
- UVD_CGC_GATE__LMI_MC_MASK |
- UVD_CGC_GATE__IDCT_MASK |
- UVD_CGC_GATE__MPRD_MASK |
- UVD_CGC_GATE__MPC_MASK |
- UVD_CGC_GATE__LBSI_MASK |
- UVD_CGC_GATE__LRBBM_MASK |
- UVD_CGC_GATE__UDEC_RE_MASK |
- UVD_CGC_GATE__UDEC_CM_MASK |
- UVD_CGC_GATE__UDEC_IT_MASK |
- UVD_CGC_GATE__UDEC_DB_MASK |
- UVD_CGC_GATE__UDEC_MP_MASK |
- UVD_CGC_GATE__WCB_MASK |
- UVD_CGC_GATE__VCPU_MASK |
- UVD_CGC_GATE__SCPU_MASK;
- data1 |= UVD_SUVD_CGC_GATE__SRE_MASK |
- UVD_SUVD_CGC_GATE__SIT_MASK |
- UVD_SUVD_CGC_GATE__SMP_MASK |
- UVD_SUVD_CGC_GATE__SCM_MASK |
- UVD_SUVD_CGC_GATE__SDB_MASK;
- } else {
- data &= ~(UVD_CGC_GATE__SYS_MASK |
- UVD_CGC_GATE__UDEC_MASK |
- UVD_CGC_GATE__MPEG2_MASK |
- UVD_CGC_GATE__RBC_MASK |
- UVD_CGC_GATE__LMI_MC_MASK |
- UVD_CGC_GATE__LMI_UMC_MASK |
- UVD_CGC_GATE__IDCT_MASK |
- UVD_CGC_GATE__MPRD_MASK |
- UVD_CGC_GATE__MPC_MASK |
- UVD_CGC_GATE__LBSI_MASK |
- UVD_CGC_GATE__LRBBM_MASK |
- UVD_CGC_GATE__UDEC_RE_MASK |
- UVD_CGC_GATE__UDEC_CM_MASK |
- UVD_CGC_GATE__UDEC_IT_MASK |
- UVD_CGC_GATE__UDEC_DB_MASK |
- UVD_CGC_GATE__UDEC_MP_MASK |
- UVD_CGC_GATE__WCB_MASK |
- UVD_CGC_GATE__VCPU_MASK |
- UVD_CGC_GATE__SCPU_MASK);
- data1 &= ~(UVD_SUVD_CGC_GATE__SRE_MASK |
- UVD_SUVD_CGC_GATE__SIT_MASK |
- UVD_SUVD_CGC_GATE__SMP_MASK |
- UVD_SUVD_CGC_GATE__SCM_MASK |
- UVD_SUVD_CGC_GATE__SDB_MASK);
- }
- WREG32(mmUVD_CGC_GATE, data);
- WREG32(mmUVD_SUVD_CGC_GATE, data1);
-}
-
-static void uvd_v6_0_set_uvd_dynamic_clock_mode(struct amdgpu_device *adev,
- bool swmode)
-{
- u32 data, data1 = 0, data2;
-
- /* Always un-gate UVD REGS bit */
- data = RREG32(mmUVD_CGC_GATE);
- data &= ~(UVD_CGC_GATE__REGS_MASK);
- WREG32(mmUVD_CGC_GATE, data);
-
- data = RREG32(mmUVD_CGC_CTRL);
- data &= ~(UVD_CGC_CTRL__CLK_OFF_DELAY_MASK |
- UVD_CGC_CTRL__CLK_GATE_DLY_TIMER_MASK);
- data |= UVD_CGC_CTRL__DYN_CLOCK_MODE_MASK |
- 1 << REG_FIELD_SHIFT(UVD_CGC_CTRL, CLK_GATE_DLY_TIMER) |
- 4 << REG_FIELD_SHIFT(UVD_CGC_CTRL, CLK_OFF_DELAY);
-
- data2 = RREG32(mmUVD_SUVD_CGC_CTRL);
- if (swmode) {
- data &= ~(UVD_CGC_CTRL__UDEC_RE_MODE_MASK |
- UVD_CGC_CTRL__UDEC_CM_MODE_MASK |
- UVD_CGC_CTRL__UDEC_IT_MODE_MASK |
- UVD_CGC_CTRL__UDEC_DB_MODE_MASK |
- UVD_CGC_CTRL__UDEC_MP_MODE_MASK |
- UVD_CGC_CTRL__SYS_MODE_MASK |
- UVD_CGC_CTRL__UDEC_MODE_MASK |
- UVD_CGC_CTRL__MPEG2_MODE_MASK |
- UVD_CGC_CTRL__REGS_MODE_MASK |
- UVD_CGC_CTRL__RBC_MODE_MASK |
- UVD_CGC_CTRL__LMI_MC_MODE_MASK |
- UVD_CGC_CTRL__LMI_UMC_MODE_MASK |
- UVD_CGC_CTRL__IDCT_MODE_MASK |
- UVD_CGC_CTRL__MPRD_MODE_MASK |
- UVD_CGC_CTRL__MPC_MODE_MASK |
- UVD_CGC_CTRL__LBSI_MODE_MASK |
- UVD_CGC_CTRL__LRBBM_MODE_MASK |
- UVD_CGC_CTRL__WCB_MODE_MASK |
- UVD_CGC_CTRL__VCPU_MODE_MASK |
- UVD_CGC_CTRL__JPEG_MODE_MASK |
- UVD_CGC_CTRL__SCPU_MODE_MASK);
- data1 |= UVD_CGC_CTRL2__DYN_OCLK_RAMP_EN_MASK |
- UVD_CGC_CTRL2__DYN_RCLK_RAMP_EN_MASK;
- data1 &= ~UVD_CGC_CTRL2__GATER_DIV_ID_MASK;
- data1 |= 7 << REG_FIELD_SHIFT(UVD_CGC_CTRL2, GATER_DIV_ID);
- data2 &= ~(UVD_SUVD_CGC_CTRL__SRE_MODE_MASK |
- UVD_SUVD_CGC_CTRL__SIT_MODE_MASK |
- UVD_SUVD_CGC_CTRL__SMP_MODE_MASK |
- UVD_SUVD_CGC_CTRL__SCM_MODE_MASK |
- UVD_SUVD_CGC_CTRL__SDB_MODE_MASK);
- } else {
- data |= UVD_CGC_CTRL__UDEC_RE_MODE_MASK |
- UVD_CGC_CTRL__UDEC_CM_MODE_MASK |
- UVD_CGC_CTRL__UDEC_IT_MODE_MASK |
- UVD_CGC_CTRL__UDEC_DB_MODE_MASK |
- UVD_CGC_CTRL__UDEC_MP_MODE_MASK |
- UVD_CGC_CTRL__SYS_MODE_MASK |
- UVD_CGC_CTRL__UDEC_MODE_MASK |
- UVD_CGC_CTRL__MPEG2_MODE_MASK |
- UVD_CGC_CTRL__REGS_MODE_MASK |
- UVD_CGC_CTRL__RBC_MODE_MASK |
- UVD_CGC_CTRL__LMI_MC_MODE_MASK |
- UVD_CGC_CTRL__LMI_UMC_MODE_MASK |
- UVD_CGC_CTRL__IDCT_MODE_MASK |
- UVD_CGC_CTRL__MPRD_MODE_MASK |
- UVD_CGC_CTRL__MPC_MODE_MASK |
- UVD_CGC_CTRL__LBSI_MODE_MASK |
- UVD_CGC_CTRL__LRBBM_MODE_MASK |
- UVD_CGC_CTRL__WCB_MODE_MASK |
- UVD_CGC_CTRL__VCPU_MODE_MASK |
- UVD_CGC_CTRL__SCPU_MODE_MASK;
- data2 |= UVD_SUVD_CGC_CTRL__SRE_MODE_MASK |
- UVD_SUVD_CGC_CTRL__SIT_MODE_MASK |
- UVD_SUVD_CGC_CTRL__SMP_MODE_MASK |
- UVD_SUVD_CGC_CTRL__SCM_MODE_MASK |
- UVD_SUVD_CGC_CTRL__SDB_MODE_MASK;
- }
- WREG32(mmUVD_CGC_CTRL, data);
- WREG32(mmUVD_SUVD_CGC_CTRL, data2);
-
- data = RREG32_UVD_CTX(ixUVD_CGC_CTRL2);
- data &= ~(REG_FIELD_MASK(UVD_CGC_CTRL2, DYN_OCLK_RAMP_EN) |
- REG_FIELD_MASK(UVD_CGC_CTRL2, DYN_RCLK_RAMP_EN) |
- REG_FIELD_MASK(UVD_CGC_CTRL2, GATER_DIV_ID));
- data1 &= (REG_FIELD_MASK(UVD_CGC_CTRL2, DYN_OCLK_RAMP_EN) |
- REG_FIELD_MASK(UVD_CGC_CTRL2, DYN_RCLK_RAMP_EN) |
- REG_FIELD_MASK(UVD_CGC_CTRL2, GATER_DIV_ID));
- data |= data1;
- WREG32_UVD_CTX(ixUVD_CGC_CTRL2, data);
-}
+#endif
/**
* uvd_v6_0_start - start UVD block
@@ -538,11 +396,7 @@
/* Set dynamic clock gating in S/W control mode */
if (adev->cg_flags & AMD_CG_SUPPORT_UVD_MGCG) {
- if (adev->flags & AMD_IS_APU)
- cz_set_uvd_clock_gating_branches(adev, false);
- else
- tonga_set_uvd_clock_gating_branches(adev, false);
- uvd_v6_0_set_uvd_dynamic_clock_mode(adev, true);
+ uvd_v6_0_set_sw_clock_gating(adev);
} else {
/* disable clock gating */
uint32_t data = RREG32(mmUVD_CGC_CTRL);
@@ -777,7 +631,8 @@
* Write ring commands to execute the indirect buffer
*/
static void uvd_v6_0_ring_emit_ib(struct amdgpu_ring *ring,
- struct amdgpu_ib *ib)
+ struct amdgpu_ib *ib,
+ unsigned vm_id, bool ctx_switch)
{
amdgpu_ring_write(ring, PACKET0(mmUVD_LMI_RBC_IB_64BIT_BAR_LOW, 0));
amdgpu_ring_write(ring, lower_32_bits(ib->gpu_addr));
@@ -854,112 +709,6 @@
return uvd_v6_0_start(adev);
}
-static void uvd_v6_0_print_status(void *handle)
-{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- dev_info(adev->dev, "UVD 6.0 registers\n");
- dev_info(adev->dev, " UVD_SEMA_ADDR_LOW=0x%08X\n",
- RREG32(mmUVD_SEMA_ADDR_LOW));
- dev_info(adev->dev, " UVD_SEMA_ADDR_HIGH=0x%08X\n",
- RREG32(mmUVD_SEMA_ADDR_HIGH));
- dev_info(adev->dev, " UVD_SEMA_CMD=0x%08X\n",
- RREG32(mmUVD_SEMA_CMD));
- dev_info(adev->dev, " UVD_GPCOM_VCPU_CMD=0x%08X\n",
- RREG32(mmUVD_GPCOM_VCPU_CMD));
- dev_info(adev->dev, " UVD_GPCOM_VCPU_DATA0=0x%08X\n",
- RREG32(mmUVD_GPCOM_VCPU_DATA0));
- dev_info(adev->dev, " UVD_GPCOM_VCPU_DATA1=0x%08X\n",
- RREG32(mmUVD_GPCOM_VCPU_DATA1));
- dev_info(adev->dev, " UVD_ENGINE_CNTL=0x%08X\n",
- RREG32(mmUVD_ENGINE_CNTL));
- dev_info(adev->dev, " UVD_UDEC_ADDR_CONFIG=0x%08X\n",
- RREG32(mmUVD_UDEC_ADDR_CONFIG));
- dev_info(adev->dev, " UVD_UDEC_DB_ADDR_CONFIG=0x%08X\n",
- RREG32(mmUVD_UDEC_DB_ADDR_CONFIG));
- dev_info(adev->dev, " UVD_UDEC_DBW_ADDR_CONFIG=0x%08X\n",
- RREG32(mmUVD_UDEC_DBW_ADDR_CONFIG));
- dev_info(adev->dev, " UVD_SEMA_CNTL=0x%08X\n",
- RREG32(mmUVD_SEMA_CNTL));
- dev_info(adev->dev, " UVD_LMI_EXT40_ADDR=0x%08X\n",
- RREG32(mmUVD_LMI_EXT40_ADDR));
- dev_info(adev->dev, " UVD_CTX_INDEX=0x%08X\n",
- RREG32(mmUVD_CTX_INDEX));
- dev_info(adev->dev, " UVD_CTX_DATA=0x%08X\n",
- RREG32(mmUVD_CTX_DATA));
- dev_info(adev->dev, " UVD_CGC_GATE=0x%08X\n",
- RREG32(mmUVD_CGC_GATE));
- dev_info(adev->dev, " UVD_CGC_CTRL=0x%08X\n",
- RREG32(mmUVD_CGC_CTRL));
- dev_info(adev->dev, " UVD_LMI_CTRL2=0x%08X\n",
- RREG32(mmUVD_LMI_CTRL2));
- dev_info(adev->dev, " UVD_MASTINT_EN=0x%08X\n",
- RREG32(mmUVD_MASTINT_EN));
- dev_info(adev->dev, " UVD_LMI_ADDR_EXT=0x%08X\n",
- RREG32(mmUVD_LMI_ADDR_EXT));
- dev_info(adev->dev, " UVD_LMI_CTRL=0x%08X\n",
- RREG32(mmUVD_LMI_CTRL));
- dev_info(adev->dev, " UVD_LMI_SWAP_CNTL=0x%08X\n",
- RREG32(mmUVD_LMI_SWAP_CNTL));
- dev_info(adev->dev, " UVD_MP_SWAP_CNTL=0x%08X\n",
- RREG32(mmUVD_MP_SWAP_CNTL));
- dev_info(adev->dev, " UVD_MPC_SET_MUXA0=0x%08X\n",
- RREG32(mmUVD_MPC_SET_MUXA0));
- dev_info(adev->dev, " UVD_MPC_SET_MUXA1=0x%08X\n",
- RREG32(mmUVD_MPC_SET_MUXA1));
- dev_info(adev->dev, " UVD_MPC_SET_MUXB0=0x%08X\n",
- RREG32(mmUVD_MPC_SET_MUXB0));
- dev_info(adev->dev, " UVD_MPC_SET_MUXB1=0x%08X\n",
- RREG32(mmUVD_MPC_SET_MUXB1));
- dev_info(adev->dev, " UVD_MPC_SET_MUX=0x%08X\n",
- RREG32(mmUVD_MPC_SET_MUX));
- dev_info(adev->dev, " UVD_MPC_SET_ALU=0x%08X\n",
- RREG32(mmUVD_MPC_SET_ALU));
- dev_info(adev->dev, " UVD_VCPU_CACHE_OFFSET0=0x%08X\n",
- RREG32(mmUVD_VCPU_CACHE_OFFSET0));
- dev_info(adev->dev, " UVD_VCPU_CACHE_SIZE0=0x%08X\n",
- RREG32(mmUVD_VCPU_CACHE_SIZE0));
- dev_info(adev->dev, " UVD_VCPU_CACHE_OFFSET1=0x%08X\n",
- RREG32(mmUVD_VCPU_CACHE_OFFSET1));
- dev_info(adev->dev, " UVD_VCPU_CACHE_SIZE1=0x%08X\n",
- RREG32(mmUVD_VCPU_CACHE_SIZE1));
- dev_info(adev->dev, " UVD_VCPU_CACHE_OFFSET2=0x%08X\n",
- RREG32(mmUVD_VCPU_CACHE_OFFSET2));
- dev_info(adev->dev, " UVD_VCPU_CACHE_SIZE2=0x%08X\n",
- RREG32(mmUVD_VCPU_CACHE_SIZE2));
- dev_info(adev->dev, " UVD_VCPU_CNTL=0x%08X\n",
- RREG32(mmUVD_VCPU_CNTL));
- dev_info(adev->dev, " UVD_SOFT_RESET=0x%08X\n",
- RREG32(mmUVD_SOFT_RESET));
- dev_info(adev->dev, " UVD_RBC_IB_SIZE=0x%08X\n",
- RREG32(mmUVD_RBC_IB_SIZE));
- dev_info(adev->dev, " UVD_RBC_RB_RPTR=0x%08X\n",
- RREG32(mmUVD_RBC_RB_RPTR));
- dev_info(adev->dev, " UVD_RBC_RB_WPTR=0x%08X\n",
- RREG32(mmUVD_RBC_RB_WPTR));
- dev_info(adev->dev, " UVD_RBC_RB_WPTR_CNTL=0x%08X\n",
- RREG32(mmUVD_RBC_RB_WPTR_CNTL));
- dev_info(adev->dev, " UVD_RBC_RB_CNTL=0x%08X\n",
- RREG32(mmUVD_RBC_RB_CNTL));
- dev_info(adev->dev, " UVD_STATUS=0x%08X\n",
- RREG32(mmUVD_STATUS));
- dev_info(adev->dev, " UVD_SEMA_TIMEOUT_STATUS=0x%08X\n",
- RREG32(mmUVD_SEMA_TIMEOUT_STATUS));
- dev_info(adev->dev, " UVD_SEMA_WAIT_INCOMPLETE_TIMEOUT_CNTL=0x%08X\n",
- RREG32(mmUVD_SEMA_WAIT_INCOMPLETE_TIMEOUT_CNTL));
- dev_info(adev->dev, " UVD_SEMA_WAIT_FAULT_TIMEOUT_CNTL=0x%08X\n",
- RREG32(mmUVD_SEMA_WAIT_FAULT_TIMEOUT_CNTL));
- dev_info(adev->dev, " UVD_SEMA_SIGNAL_INCOMPLETE_TIMEOUT_CNTL=0x%08X\n",
- RREG32(mmUVD_SEMA_SIGNAL_INCOMPLETE_TIMEOUT_CNTL));
- dev_info(adev->dev, " UVD_CONTEXT_ID=0x%08X\n",
- RREG32(mmUVD_CONTEXT_ID));
- dev_info(adev->dev, " UVD_UDEC_ADDR_CONFIG=0x%08X\n",
- RREG32(mmUVD_UDEC_ADDR_CONFIG));
- dev_info(adev->dev, " UVD_UDEC_DB_ADDR_CONFIG=0x%08X\n",
- RREG32(mmUVD_UDEC_DB_ADDR_CONFIG));
- dev_info(adev->dev, " UVD_UDEC_DBW_ADDR_CONFIG=0x%08X\n",
- RREG32(mmUVD_UDEC_DBW_ADDR_CONFIG));
-}
-
static int uvd_v6_0_set_interrupt_state(struct amdgpu_device *adev,
struct amdgpu_irq_src *source,
unsigned type,
@@ -978,25 +727,146 @@
return 0;
}
+static void uvd_v6_0_set_sw_clock_gating(struct amdgpu_device *adev)
+{
+ uint32_t data, data1, data2, suvd_flags;
+
+ data = RREG32(mmUVD_CGC_CTRL);
+ data1 = RREG32(mmUVD_SUVD_CGC_GATE);
+ data2 = RREG32(mmUVD_SUVD_CGC_CTRL);
+
+ data &= ~(UVD_CGC_CTRL__CLK_OFF_DELAY_MASK |
+ UVD_CGC_CTRL__CLK_GATE_DLY_TIMER_MASK);
+
+ suvd_flags = UVD_SUVD_CGC_GATE__SRE_MASK |
+ UVD_SUVD_CGC_GATE__SIT_MASK |
+ UVD_SUVD_CGC_GATE__SMP_MASK |
+ UVD_SUVD_CGC_GATE__SCM_MASK |
+ UVD_SUVD_CGC_GATE__SDB_MASK;
+
+ data |= UVD_CGC_CTRL__DYN_CLOCK_MODE_MASK |
+ (1 << REG_FIELD_SHIFT(UVD_CGC_CTRL, CLK_GATE_DLY_TIMER)) |
+ (4 << REG_FIELD_SHIFT(UVD_CGC_CTRL, CLK_OFF_DELAY));
+
+ data &= ~(UVD_CGC_CTRL__UDEC_RE_MODE_MASK |
+ UVD_CGC_CTRL__UDEC_CM_MODE_MASK |
+ UVD_CGC_CTRL__UDEC_IT_MODE_MASK |
+ UVD_CGC_CTRL__UDEC_DB_MODE_MASK |
+ UVD_CGC_CTRL__UDEC_MP_MODE_MASK |
+ UVD_CGC_CTRL__SYS_MODE_MASK |
+ UVD_CGC_CTRL__UDEC_MODE_MASK |
+ UVD_CGC_CTRL__MPEG2_MODE_MASK |
+ UVD_CGC_CTRL__REGS_MODE_MASK |
+ UVD_CGC_CTRL__RBC_MODE_MASK |
+ UVD_CGC_CTRL__LMI_MC_MODE_MASK |
+ UVD_CGC_CTRL__LMI_UMC_MODE_MASK |
+ UVD_CGC_CTRL__IDCT_MODE_MASK |
+ UVD_CGC_CTRL__MPRD_MODE_MASK |
+ UVD_CGC_CTRL__MPC_MODE_MASK |
+ UVD_CGC_CTRL__LBSI_MODE_MASK |
+ UVD_CGC_CTRL__LRBBM_MODE_MASK |
+ UVD_CGC_CTRL__WCB_MODE_MASK |
+ UVD_CGC_CTRL__VCPU_MODE_MASK |
+ UVD_CGC_CTRL__JPEG_MODE_MASK |
+ UVD_CGC_CTRL__SCPU_MODE_MASK |
+ UVD_CGC_CTRL__JPEG2_MODE_MASK);
+ data2 &= ~(UVD_SUVD_CGC_CTRL__SRE_MODE_MASK |
+ UVD_SUVD_CGC_CTRL__SIT_MODE_MASK |
+ UVD_SUVD_CGC_CTRL__SMP_MODE_MASK |
+ UVD_SUVD_CGC_CTRL__SCM_MODE_MASK |
+ UVD_SUVD_CGC_CTRL__SDB_MODE_MASK);
+ data1 |= suvd_flags;
+
+ WREG32(mmUVD_CGC_CTRL, data);
+ WREG32(mmUVD_CGC_GATE, 0);
+ WREG32(mmUVD_SUVD_CGC_GATE, data1);
+ WREG32(mmUVD_SUVD_CGC_CTRL, data2);
+}
+
+#if 0
+static void uvd_v6_0_set_hw_clock_gating(struct amdgpu_device *adev)
+{
+ uint32_t data, data1, cgc_flags, suvd_flags;
+
+ data = RREG32(mmUVD_CGC_GATE);
+ data1 = RREG32(mmUVD_SUVD_CGC_GATE);
+
+ cgc_flags = UVD_CGC_GATE__SYS_MASK |
+ UVD_CGC_GATE__UDEC_MASK |
+ UVD_CGC_GATE__MPEG2_MASK |
+ UVD_CGC_GATE__RBC_MASK |
+ UVD_CGC_GATE__LMI_MC_MASK |
+ UVD_CGC_GATE__IDCT_MASK |
+ UVD_CGC_GATE__MPRD_MASK |
+ UVD_CGC_GATE__MPC_MASK |
+ UVD_CGC_GATE__LBSI_MASK |
+ UVD_CGC_GATE__LRBBM_MASK |
+ UVD_CGC_GATE__UDEC_RE_MASK |
+ UVD_CGC_GATE__UDEC_CM_MASK |
+ UVD_CGC_GATE__UDEC_IT_MASK |
+ UVD_CGC_GATE__UDEC_DB_MASK |
+ UVD_CGC_GATE__UDEC_MP_MASK |
+ UVD_CGC_GATE__WCB_MASK |
+ UVD_CGC_GATE__VCPU_MASK |
+ UVD_CGC_GATE__SCPU_MASK |
+ UVD_CGC_GATE__JPEG_MASK |
+ UVD_CGC_GATE__JPEG2_MASK;
+
+ suvd_flags = UVD_SUVD_CGC_GATE__SRE_MASK |
+ UVD_SUVD_CGC_GATE__SIT_MASK |
+ UVD_SUVD_CGC_GATE__SMP_MASK |
+ UVD_SUVD_CGC_GATE__SCM_MASK |
+ UVD_SUVD_CGC_GATE__SDB_MASK;
+
+ data |= cgc_flags;
+ data1 |= suvd_flags;
+
+ WREG32(mmUVD_CGC_GATE, data);
+ WREG32(mmUVD_SUVD_CGC_GATE, data1);
+}
+#endif
+
+static void uvd_v6_set_bypass_mode(struct amdgpu_device *adev, bool enable)
+{
+ u32 tmp = RREG32_SMC(ixGCK_DFS_BYPASS_CNTL);
+
+ if (enable)
+ tmp |= (GCK_DFS_BYPASS_CNTL__BYPASSDCLK_MASK |
+ GCK_DFS_BYPASS_CNTL__BYPASSVCLK_MASK);
+ else
+ tmp &= ~(GCK_DFS_BYPASS_CNTL__BYPASSDCLK_MASK |
+ GCK_DFS_BYPASS_CNTL__BYPASSVCLK_MASK);
+
+ WREG32_SMC(ixGCK_DFS_BYPASS_CNTL, tmp);
+}
+
static int uvd_v6_0_set_clockgating_state(void *handle,
enum amd_clockgating_state state)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
bool enable = (state == AMD_CG_STATE_GATE) ? true : false;
+ static int curstate = -1;
+
+ if (adev->asic_type == CHIP_FIJI)
+ uvd_v6_set_bypass_mode(adev, enable);
if (!(adev->cg_flags & AMD_CG_SUPPORT_UVD_MGCG))
return 0;
+ if (curstate == state)
+ return 0;
+
+ curstate = state;
if (enable) {
- if (adev->flags & AMD_IS_APU)
- cz_set_uvd_clock_gating_branches(adev, enable);
- else
- tonga_set_uvd_clock_gating_branches(adev, enable);
- uvd_v6_0_set_uvd_dynamic_clock_mode(adev, true);
+ /* disable HW gating and enable Sw gating */
+ uvd_v6_0_set_sw_clock_gating(adev);
} else {
- uint32_t data = RREG32(mmUVD_CGC_CTRL);
- data &= ~UVD_CGC_CTRL__DYN_CLOCK_MODE_MASK;
- WREG32(mmUVD_CGC_CTRL, data);
+ /* wait for STATUS to clear */
+ if (uvd_v6_0_wait_for_idle(handle))
+ return -EBUSY;
+
+ /* enable HW gates because UVD is idle */
+/* uvd_v6_0_set_hw_clock_gating(adev); */
}
return 0;
@@ -1026,6 +896,7 @@
}
const struct amd_ip_funcs uvd_v6_0_ip_funcs = {
+ .name = "uvd_v6_0",
.early_init = uvd_v6_0_early_init,
.late_init = NULL,
.sw_init = uvd_v6_0_sw_init,
@@ -1037,7 +908,6 @@
.is_idle = uvd_v6_0_is_idle,
.wait_for_idle = uvd_v6_0_wait_for_idle,
.soft_reset = uvd_v6_0_soft_reset,
- .print_status = uvd_v6_0_print_status,
.set_clockgating_state = uvd_v6_0_set_clockgating_state,
.set_powergating_state = uvd_v6_0_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/vce_v2_0.c b/drivers/gpu/drm/amd/amdgpu/vce_v2_0.c
index c7e885b..45d92ac 100644
--- a/drivers/gpu/drm/amd/amdgpu/vce_v2_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/vce_v2_0.c
@@ -44,7 +44,7 @@
static void vce_v2_0_mc_resume(struct amdgpu_device *adev);
static void vce_v2_0_set_ring_funcs(struct amdgpu_device *adev);
static void vce_v2_0_set_irq_funcs(struct amdgpu_device *adev);
-
+static int vce_v2_0_wait_for_idle(void *handle);
/**
* vce_v2_0_ring_get_rptr - get read pointer
*
@@ -201,14 +201,14 @@
ring = &adev->vce.ring[0];
sprintf(ring->name, "vce0");
- r = amdgpu_ring_init(adev, ring, 4096, VCE_CMD_NO_OP, 0xf,
+ r = amdgpu_ring_init(adev, ring, 512, VCE_CMD_NO_OP, 0xf,
&adev->vce.irq, 0, AMDGPU_RING_TYPE_VCE);
if (r)
return r;
ring = &adev->vce.ring[1];
sprintf(ring->name, "vce1");
- r = amdgpu_ring_init(adev, ring, 4096, VCE_CMD_NO_OP, 0xf,
+ r = amdgpu_ring_init(adev, ring, 512, VCE_CMD_NO_OP, 0xf,
&adev->vce.irq, 0, AMDGPU_RING_TYPE_VCE);
if (r)
return r;
@@ -240,7 +240,8 @@
r = vce_v2_0_start(adev);
if (r)
- return r;
+/* this error mean vcpu not in running state, so just skip ring test, not stop driver initialize */
+ return 0;
ring = &adev->vce.ring[0];
ring->ready = true;
@@ -318,7 +319,7 @@
WREG32(mmVCE_UENC_REG_CLOCK_GATING, tmp);
WREG32(mmVCE_CGTT_CLK_OVERRIDE, 0);
- } else {
+ } else {
tmp = RREG32(mmVCE_CLOCK_GATING_B);
tmp |= 0xe7;
tmp &= ~0xe70000;
@@ -339,6 +340,21 @@
{
u32 orig, tmp;
+ if (gated) {
+ if (vce_v2_0_wait_for_idle(adev)) {
+ DRM_INFO("VCE is busy, Can't set clock gateing");
+ return;
+ }
+ WREG32_P(mmVCE_VCPU_CNTL, 0, ~VCE_VCPU_CNTL__CLK_EN_MASK);
+ WREG32_P(mmVCE_SOFT_RESET, VCE_SOFT_RESET__ECPU_SOFT_RESET_MASK, ~VCE_SOFT_RESET__ECPU_SOFT_RESET_MASK);
+ mdelay(100);
+ WREG32(mmVCE_STATUS, 0);
+ } else {
+ WREG32_P(mmVCE_VCPU_CNTL, VCE_VCPU_CNTL__CLK_EN_MASK, ~VCE_VCPU_CNTL__CLK_EN_MASK);
+ WREG32_P(mmVCE_SOFT_RESET, VCE_SOFT_RESET__ECPU_SOFT_RESET_MASK, ~VCE_SOFT_RESET__ECPU_SOFT_RESET_MASK);
+ mdelay(100);
+ }
+
tmp = RREG32(mmVCE_CLOCK_GATING_B);
tmp &= ~0x00060006;
if (gated) {
@@ -362,6 +378,7 @@
if (gated)
WREG32(mmVCE_CGTT_CLK_OVERRIDE, 0);
+ WREG32_P(mmVCE_SOFT_RESET, 0, ~VCE_SOFT_RESET__ECPU_SOFT_RESET_MASK);
}
static void vce_v2_0_disable_cg(struct amdgpu_device *adev)
@@ -478,75 +495,6 @@
return vce_v2_0_start(adev);
}
-static void vce_v2_0_print_status(void *handle)
-{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- dev_info(adev->dev, "VCE 2.0 registers\n");
- dev_info(adev->dev, " VCE_STATUS=0x%08X\n",
- RREG32(mmVCE_STATUS));
- dev_info(adev->dev, " VCE_VCPU_CNTL=0x%08X\n",
- RREG32(mmVCE_VCPU_CNTL));
- dev_info(adev->dev, " VCE_VCPU_CACHE_OFFSET0=0x%08X\n",
- RREG32(mmVCE_VCPU_CACHE_OFFSET0));
- dev_info(adev->dev, " VCE_VCPU_CACHE_SIZE0=0x%08X\n",
- RREG32(mmVCE_VCPU_CACHE_SIZE0));
- dev_info(adev->dev, " VCE_VCPU_CACHE_OFFSET1=0x%08X\n",
- RREG32(mmVCE_VCPU_CACHE_OFFSET1));
- dev_info(adev->dev, " VCE_VCPU_CACHE_SIZE1=0x%08X\n",
- RREG32(mmVCE_VCPU_CACHE_SIZE1));
- dev_info(adev->dev, " VCE_VCPU_CACHE_OFFSET2=0x%08X\n",
- RREG32(mmVCE_VCPU_CACHE_OFFSET2));
- dev_info(adev->dev, " VCE_VCPU_CACHE_SIZE2=0x%08X\n",
- RREG32(mmVCE_VCPU_CACHE_SIZE2));
- dev_info(adev->dev, " VCE_SOFT_RESET=0x%08X\n",
- RREG32(mmVCE_SOFT_RESET));
- dev_info(adev->dev, " VCE_RB_BASE_LO2=0x%08X\n",
- RREG32(mmVCE_RB_BASE_LO2));
- dev_info(adev->dev, " VCE_RB_BASE_HI2=0x%08X\n",
- RREG32(mmVCE_RB_BASE_HI2));
- dev_info(adev->dev, " VCE_RB_SIZE2=0x%08X\n",
- RREG32(mmVCE_RB_SIZE2));
- dev_info(adev->dev, " VCE_RB_RPTR2=0x%08X\n",
- RREG32(mmVCE_RB_RPTR2));
- dev_info(adev->dev, " VCE_RB_WPTR2=0x%08X\n",
- RREG32(mmVCE_RB_WPTR2));
- dev_info(adev->dev, " VCE_RB_BASE_LO=0x%08X\n",
- RREG32(mmVCE_RB_BASE_LO));
- dev_info(adev->dev, " VCE_RB_BASE_HI=0x%08X\n",
- RREG32(mmVCE_RB_BASE_HI));
- dev_info(adev->dev, " VCE_RB_SIZE=0x%08X\n",
- RREG32(mmVCE_RB_SIZE));
- dev_info(adev->dev, " VCE_RB_RPTR=0x%08X\n",
- RREG32(mmVCE_RB_RPTR));
- dev_info(adev->dev, " VCE_RB_WPTR=0x%08X\n",
- RREG32(mmVCE_RB_WPTR));
- dev_info(adev->dev, " VCE_CLOCK_GATING_A=0x%08X\n",
- RREG32(mmVCE_CLOCK_GATING_A));
- dev_info(adev->dev, " VCE_CLOCK_GATING_B=0x%08X\n",
- RREG32(mmVCE_CLOCK_GATING_B));
- dev_info(adev->dev, " VCE_CGTT_CLK_OVERRIDE=0x%08X\n",
- RREG32(mmVCE_CGTT_CLK_OVERRIDE));
- dev_info(adev->dev, " VCE_UENC_CLOCK_GATING=0x%08X\n",
- RREG32(mmVCE_UENC_CLOCK_GATING));
- dev_info(adev->dev, " VCE_UENC_REG_CLOCK_GATING=0x%08X\n",
- RREG32(mmVCE_UENC_REG_CLOCK_GATING));
- dev_info(adev->dev, " VCE_SYS_INT_EN=0x%08X\n",
- RREG32(mmVCE_SYS_INT_EN));
- dev_info(adev->dev, " VCE_LMI_CTRL2=0x%08X\n",
- RREG32(mmVCE_LMI_CTRL2));
- dev_info(adev->dev, " VCE_LMI_CTRL=0x%08X\n",
- RREG32(mmVCE_LMI_CTRL));
- dev_info(adev->dev, " VCE_LMI_VM_CTRL=0x%08X\n",
- RREG32(mmVCE_LMI_VM_CTRL));
- dev_info(adev->dev, " VCE_LMI_SWAP_CNTL=0x%08X\n",
- RREG32(mmVCE_LMI_SWAP_CNTL));
- dev_info(adev->dev, " VCE_LMI_SWAP_CNTL1=0x%08X\n",
- RREG32(mmVCE_LMI_SWAP_CNTL1));
- dev_info(adev->dev, " VCE_LMI_CACHE_CTRL=0x%08X\n",
- RREG32(mmVCE_LMI_CACHE_CTRL));
-}
-
static int vce_v2_0_set_interrupt_state(struct amdgpu_device *adev,
struct amdgpu_irq_src *source,
unsigned type,
@@ -619,6 +567,7 @@
}
const struct amd_ip_funcs vce_v2_0_ip_funcs = {
+ .name = "vce_v2_0",
.early_init = vce_v2_0_early_init,
.late_init = NULL,
.sw_init = vce_v2_0_sw_init,
@@ -630,7 +579,6 @@
.is_idle = vce_v2_0_is_idle,
.wait_for_idle = vce_v2_0_wait_for_idle,
.soft_reset = vce_v2_0_soft_reset,
- .print_status = vce_v2_0_print_status,
.set_clockgating_state = vce_v2_0_set_clockgating_state,
.set_powergating_state = vce_v2_0_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/vce_v3_0.c b/drivers/gpu/drm/amd/amdgpu/vce_v3_0.c
index ce468ee..30e8099 100644
--- a/drivers/gpu/drm/amd/amdgpu/vce_v3_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/vce_v3_0.c
@@ -40,9 +40,9 @@
#define GRBM_GFX_INDEX__VCE_INSTANCE__SHIFT 0x04
#define GRBM_GFX_INDEX__VCE_INSTANCE_MASK 0x10
-#define mmVCE_LMI_VCPU_CACHE_40BIT_BAR0 0x8616
-#define mmVCE_LMI_VCPU_CACHE_40BIT_BAR1 0x8617
-#define mmVCE_LMI_VCPU_CACHE_40BIT_BAR2 0x8618
+#define mmVCE_LMI_VCPU_CACHE_40BIT_BAR0 0x8616
+#define mmVCE_LMI_VCPU_CACHE_40BIT_BAR1 0x8617
+#define mmVCE_LMI_VCPU_CACHE_40BIT_BAR2 0x8618
#define VCE_V3_0_FW_SIZE (384 * 1024)
#define VCE_V3_0_STACK_SIZE (64 * 1024)
@@ -315,9 +315,11 @@
{
u32 tmp;
- /* Fiji, Stoney are single pipe */
+ /* Fiji, Stoney, Polaris10, Polaris11 are single pipe */
if ((adev->asic_type == CHIP_FIJI) ||
- (adev->asic_type == CHIP_STONEY))
+ (adev->asic_type == CHIP_STONEY) ||
+ (adev->asic_type == CHIP_POLARIS10) ||
+ (adev->asic_type == CHIP_POLARIS11))
return AMDGPU_VCE_HARVEST_VCE1;
/* Tonga and CZ are dual or single pipe */
@@ -381,14 +383,14 @@
ring = &adev->vce.ring[0];
sprintf(ring->name, "vce0");
- r = amdgpu_ring_init(adev, ring, 4096, VCE_CMD_NO_OP, 0xf,
+ r = amdgpu_ring_init(adev, ring, 512, VCE_CMD_NO_OP, 0xf,
&adev->vce.irq, 0, AMDGPU_RING_TYPE_VCE);
if (r)
return r;
ring = &adev->vce.ring[1];
sprintf(ring->name, "vce1");
- r = amdgpu_ring_init(adev, ring, 4096, VCE_CMD_NO_OP, 0xf,
+ r = amdgpu_ring_init(adev, ring, 512, VCE_CMD_NO_OP, 0xf,
&adev->vce.irq, 0, AMDGPU_RING_TYPE_VCE);
if (r)
return r;
@@ -564,73 +566,6 @@
return vce_v3_0_start(adev);
}
-static void vce_v3_0_print_status(void *handle)
-{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- dev_info(adev->dev, "VCE 3.0 registers\n");
- dev_info(adev->dev, " VCE_STATUS=0x%08X\n",
- RREG32(mmVCE_STATUS));
- dev_info(adev->dev, " VCE_VCPU_CNTL=0x%08X\n",
- RREG32(mmVCE_VCPU_CNTL));
- dev_info(adev->dev, " VCE_VCPU_CACHE_OFFSET0=0x%08X\n",
- RREG32(mmVCE_VCPU_CACHE_OFFSET0));
- dev_info(adev->dev, " VCE_VCPU_CACHE_SIZE0=0x%08X\n",
- RREG32(mmVCE_VCPU_CACHE_SIZE0));
- dev_info(adev->dev, " VCE_VCPU_CACHE_OFFSET1=0x%08X\n",
- RREG32(mmVCE_VCPU_CACHE_OFFSET1));
- dev_info(adev->dev, " VCE_VCPU_CACHE_SIZE1=0x%08X\n",
- RREG32(mmVCE_VCPU_CACHE_SIZE1));
- dev_info(adev->dev, " VCE_VCPU_CACHE_OFFSET2=0x%08X\n",
- RREG32(mmVCE_VCPU_CACHE_OFFSET2));
- dev_info(adev->dev, " VCE_VCPU_CACHE_SIZE2=0x%08X\n",
- RREG32(mmVCE_VCPU_CACHE_SIZE2));
- dev_info(adev->dev, " VCE_SOFT_RESET=0x%08X\n",
- RREG32(mmVCE_SOFT_RESET));
- dev_info(adev->dev, " VCE_RB_BASE_LO2=0x%08X\n",
- RREG32(mmVCE_RB_BASE_LO2));
- dev_info(adev->dev, " VCE_RB_BASE_HI2=0x%08X\n",
- RREG32(mmVCE_RB_BASE_HI2));
- dev_info(adev->dev, " VCE_RB_SIZE2=0x%08X\n",
- RREG32(mmVCE_RB_SIZE2));
- dev_info(adev->dev, " VCE_RB_RPTR2=0x%08X\n",
- RREG32(mmVCE_RB_RPTR2));
- dev_info(adev->dev, " VCE_RB_WPTR2=0x%08X\n",
- RREG32(mmVCE_RB_WPTR2));
- dev_info(adev->dev, " VCE_RB_BASE_LO=0x%08X\n",
- RREG32(mmVCE_RB_BASE_LO));
- dev_info(adev->dev, " VCE_RB_BASE_HI=0x%08X\n",
- RREG32(mmVCE_RB_BASE_HI));
- dev_info(adev->dev, " VCE_RB_SIZE=0x%08X\n",
- RREG32(mmVCE_RB_SIZE));
- dev_info(adev->dev, " VCE_RB_RPTR=0x%08X\n",
- RREG32(mmVCE_RB_RPTR));
- dev_info(adev->dev, " VCE_RB_WPTR=0x%08X\n",
- RREG32(mmVCE_RB_WPTR));
- dev_info(adev->dev, " VCE_CLOCK_GATING_A=0x%08X\n",
- RREG32(mmVCE_CLOCK_GATING_A));
- dev_info(adev->dev, " VCE_CLOCK_GATING_B=0x%08X\n",
- RREG32(mmVCE_CLOCK_GATING_B));
- dev_info(adev->dev, " VCE_UENC_CLOCK_GATING=0x%08X\n",
- RREG32(mmVCE_UENC_CLOCK_GATING));
- dev_info(adev->dev, " VCE_UENC_REG_CLOCK_GATING=0x%08X\n",
- RREG32(mmVCE_UENC_REG_CLOCK_GATING));
- dev_info(adev->dev, " VCE_SYS_INT_EN=0x%08X\n",
- RREG32(mmVCE_SYS_INT_EN));
- dev_info(adev->dev, " VCE_LMI_CTRL2=0x%08X\n",
- RREG32(mmVCE_LMI_CTRL2));
- dev_info(adev->dev, " VCE_LMI_CTRL=0x%08X\n",
- RREG32(mmVCE_LMI_CTRL));
- dev_info(adev->dev, " VCE_LMI_VM_CTRL=0x%08X\n",
- RREG32(mmVCE_LMI_VM_CTRL));
- dev_info(adev->dev, " VCE_LMI_SWAP_CNTL=0x%08X\n",
- RREG32(mmVCE_LMI_SWAP_CNTL));
- dev_info(adev->dev, " VCE_LMI_SWAP_CNTL1=0x%08X\n",
- RREG32(mmVCE_LMI_SWAP_CNTL1));
- dev_info(adev->dev, " VCE_LMI_CACHE_CTRL=0x%08X\n",
- RREG32(mmVCE_LMI_CACHE_CTRL));
-}
-
static int vce_v3_0_set_interrupt_state(struct amdgpu_device *adev,
struct amdgpu_irq_src *source,
unsigned type,
@@ -739,6 +674,7 @@
}
const struct amd_ip_funcs vce_v3_0_ip_funcs = {
+ .name = "vce_v3_0",
.early_init = vce_v3_0_early_init,
.late_init = NULL,
.sw_init = vce_v3_0_sw_init,
@@ -750,7 +686,6 @@
.is_idle = vce_v3_0_is_idle,
.wait_for_idle = vce_v3_0_wait_for_idle,
.soft_reset = vce_v3_0_soft_reset,
- .print_status = vce_v3_0_print_status,
.set_clockgating_state = vce_v3_0_set_clockgating_state,
.set_powergating_state = vce_v3_0_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/vi.c b/drivers/gpu/drm/amd/amdgpu/vi.c
index 1c120ef..2c88d0b 100644
--- a/drivers/gpu/drm/amd/amdgpu/vi.c
+++ b/drivers/gpu/drm/amd/amdgpu/vi.c
@@ -78,6 +78,11 @@
#include "amdgpu_acp.h"
#endif
+MODULE_FIRMWARE("amdgpu/polaris10_smc.bin");
+MODULE_FIRMWARE("amdgpu/polaris10_smc_sk.bin");
+MODULE_FIRMWARE("amdgpu/polaris11_smc.bin");
+MODULE_FIRMWARE("amdgpu/polaris11_smc_sk.bin");
+
/*
* Indirect registers accessor
*/
@@ -276,6 +281,8 @@
stoney_mgcg_cgcg_init,
(const u32)ARRAY_SIZE(stoney_mgcg_cgcg_init));
break;
+ case CHIP_POLARIS11:
+ case CHIP_POLARIS10:
default:
break;
}
@@ -414,11 +421,11 @@
return true;
}
-static struct amdgpu_allowed_register_entry tonga_allowed_read_registers[] = {
+static const struct amdgpu_allowed_register_entry tonga_allowed_read_registers[] = {
{mmGB_MACROTILE_MODE7, true},
};
-static struct amdgpu_allowed_register_entry cz_allowed_read_registers[] = {
+static const struct amdgpu_allowed_register_entry cz_allowed_read_registers[] = {
{mmGB_TILE_MODE7, true},
{mmGB_TILE_MODE12, true},
{mmGB_TILE_MODE17, true},
@@ -426,7 +433,7 @@
{mmGB_MACROTILE_MODE7, true},
};
-static struct amdgpu_allowed_register_entry vi_allowed_read_registers[] = {
+static const struct amdgpu_allowed_register_entry vi_allowed_read_registers[] = {
{mmGRBM_STATUS, false},
{mmGRBM_STATUS2, false},
{mmGRBM_STATUS_SE0, false},
@@ -525,8 +532,8 @@
static int vi_read_register(struct amdgpu_device *adev, u32 se_num,
u32 sh_num, u32 reg_offset, u32 *value)
{
- struct amdgpu_allowed_register_entry *asic_register_table = NULL;
- struct amdgpu_allowed_register_entry *asic_register_entry;
+ const struct amdgpu_allowed_register_entry *asic_register_table = NULL;
+ const struct amdgpu_allowed_register_entry *asic_register_entry;
uint32_t size, i;
*value = 0;
@@ -537,6 +544,8 @@
break;
case CHIP_FIJI:
case CHIP_TONGA:
+ case CHIP_POLARIS11:
+ case CHIP_POLARIS10:
case CHIP_CARRIZO:
case CHIP_STONEY:
asic_register_table = cz_allowed_read_registers;
@@ -907,6 +916,74 @@
},
};
+static const struct amdgpu_ip_block_version polaris11_ip_blocks[] =
+{
+ /* ORDER MATTERS! */
+ {
+ .type = AMD_IP_BLOCK_TYPE_COMMON,
+ .major = 2,
+ .minor = 0,
+ .rev = 0,
+ .funcs = &vi_common_ip_funcs,
+ },
+ {
+ .type = AMD_IP_BLOCK_TYPE_GMC,
+ .major = 8,
+ .minor = 1,
+ .rev = 0,
+ .funcs = &gmc_v8_0_ip_funcs,
+ },
+ {
+ .type = AMD_IP_BLOCK_TYPE_IH,
+ .major = 3,
+ .minor = 1,
+ .rev = 0,
+ .funcs = &tonga_ih_ip_funcs,
+ },
+ {
+ .type = AMD_IP_BLOCK_TYPE_SMC,
+ .major = 7,
+ .minor = 2,
+ .rev = 0,
+ .funcs = &amdgpu_pp_ip_funcs,
+ },
+ {
+ .type = AMD_IP_BLOCK_TYPE_DCE,
+ .major = 11,
+ .minor = 2,
+ .rev = 0,
+ .funcs = &dce_v11_0_ip_funcs,
+ },
+ {
+ .type = AMD_IP_BLOCK_TYPE_GFX,
+ .major = 8,
+ .minor = 0,
+ .rev = 0,
+ .funcs = &gfx_v8_0_ip_funcs,
+ },
+ {
+ .type = AMD_IP_BLOCK_TYPE_SDMA,
+ .major = 3,
+ .minor = 1,
+ .rev = 0,
+ .funcs = &sdma_v3_0_ip_funcs,
+ },
+ {
+ .type = AMD_IP_BLOCK_TYPE_UVD,
+ .major = 6,
+ .minor = 3,
+ .rev = 0,
+ .funcs = &uvd_v6_0_ip_funcs,
+ },
+ {
+ .type = AMD_IP_BLOCK_TYPE_VCE,
+ .major = 3,
+ .minor = 4,
+ .rev = 0,
+ .funcs = &vce_v3_0_ip_funcs,
+ },
+};
+
static const struct amdgpu_ip_block_version cz_ip_blocks[] =
{
/* ORDER MATTERS! */
@@ -999,6 +1076,11 @@
adev->ip_blocks = tonga_ip_blocks;
adev->num_ip_blocks = ARRAY_SIZE(tonga_ip_blocks);
break;
+ case CHIP_POLARIS11:
+ case CHIP_POLARIS10:
+ adev->ip_blocks = polaris11_ip_blocks;
+ adev->num_ip_blocks = ARRAY_SIZE(polaris11_ip_blocks);
+ break;
case CHIP_CARRIZO:
case CHIP_STONEY:
adev->ip_blocks = cz_ip_blocks;
@@ -1036,7 +1118,6 @@
.get_xclk = &vi_get_xclk,
.set_uvd_clocks = &vi_set_uvd_clocks,
.set_vce_clocks = &vi_set_vce_clocks,
- .get_cu_info = &gfx_v8_0_get_cu_info,
/* these should be moved to their own ip modules */
.get_gpu_clock_counter = &gfx_v8_0_get_gpu_clock_counter,
.wait_for_mc_idle = &gmc_v8_0_mc_wait_for_idle,
@@ -1076,19 +1157,69 @@
adev->external_rev_id = 0x1;
break;
case CHIP_FIJI:
- adev->cg_flags = 0;
+ adev->cg_flags = AMD_CG_SUPPORT_GFX_MGCG |
+ AMD_CG_SUPPORT_GFX_MGLS |
+ AMD_CG_SUPPORT_GFX_RLC_LS |
+ AMD_CG_SUPPORT_GFX_CP_LS |
+ AMD_CG_SUPPORT_GFX_CGTS |
+ AMD_CG_SUPPORT_GFX_CGTS_LS |
+ AMD_CG_SUPPORT_GFX_CGCG |
+ AMD_CG_SUPPORT_GFX_CGLS |
+ AMD_CG_SUPPORT_SDMA_MGCG |
+ AMD_CG_SUPPORT_SDMA_LS |
+ AMD_CG_SUPPORT_BIF_LS |
+ AMD_CG_SUPPORT_HDP_MGCG |
+ AMD_CG_SUPPORT_HDP_LS |
+ AMD_CG_SUPPORT_ROM_MGCG |
+ AMD_CG_SUPPORT_MC_MGCG |
+ AMD_CG_SUPPORT_MC_LS;
adev->pg_flags = 0;
adev->external_rev_id = adev->rev_id + 0x3c;
break;
case CHIP_TONGA:
- adev->cg_flags = 0;
+ adev->cg_flags = AMD_CG_SUPPORT_UVD_MGCG;
adev->pg_flags = 0;
adev->external_rev_id = adev->rev_id + 0x14;
break;
- case CHIP_CARRIZO:
- case CHIP_STONEY:
+ case CHIP_POLARIS11:
adev->cg_flags = 0;
adev->pg_flags = 0;
+ adev->external_rev_id = adev->rev_id + 0x5A;
+ break;
+ case CHIP_POLARIS10:
+ adev->cg_flags = 0;
+ adev->pg_flags = 0;
+ adev->external_rev_id = adev->rev_id + 0x50;
+ break;
+ case CHIP_CARRIZO:
+ adev->cg_flags = AMD_CG_SUPPORT_UVD_MGCG |
+ AMD_CG_SUPPORT_GFX_MGCG |
+ AMD_CG_SUPPORT_GFX_MGLS |
+ AMD_CG_SUPPORT_GFX_RLC_LS |
+ AMD_CG_SUPPORT_GFX_CP_LS |
+ AMD_CG_SUPPORT_GFX_CGTS |
+ AMD_CG_SUPPORT_GFX_MGLS |
+ AMD_CG_SUPPORT_GFX_CGTS_LS |
+ AMD_CG_SUPPORT_GFX_CGCG |
+ AMD_CG_SUPPORT_GFX_CGLS |
+ AMD_CG_SUPPORT_BIF_LS |
+ AMD_CG_SUPPORT_HDP_MGCG |
+ AMD_CG_SUPPORT_HDP_LS |
+ AMD_CG_SUPPORT_SDMA_MGCG |
+ AMD_CG_SUPPORT_SDMA_LS;
+ adev->pg_flags = 0;
+ adev->external_rev_id = adev->rev_id + 0x1;
+ break;
+ case CHIP_STONEY:
+ adev->cg_flags = AMD_CG_SUPPORT_UVD_MGCG |
+ AMD_CG_SUPPORT_GFX_MGCG |
+ AMD_CG_SUPPORT_GFX_MGLS |
+ AMD_CG_SUPPORT_BIF_LS |
+ AMD_CG_SUPPORT_HDP_MGCG |
+ AMD_CG_SUPPORT_HDP_LS |
+ AMD_CG_SUPPORT_SDMA_MGCG |
+ AMD_CG_SUPPORT_SDMA_LS;
+ adev->pg_flags = 0;
adev->external_rev_id = adev->rev_id + 0x1;
break;
default:
@@ -1164,24 +1295,19 @@
return 0;
}
-static void vi_common_print_status(void *handle)
-{
- return;
-}
-
static int vi_common_soft_reset(void *handle)
{
return 0;
}
-static void fiji_update_bif_medium_grain_light_sleep(struct amdgpu_device *adev,
- bool enable)
+static void vi_update_bif_medium_grain_light_sleep(struct amdgpu_device *adev,
+ bool enable)
{
uint32_t temp, data;
temp = data = RREG32_PCIE(ixPCIE_CNTL2);
- if (enable)
+ if (enable && (adev->cg_flags & AMD_CG_SUPPORT_BIF_LS))
data |= PCIE_CNTL2__SLV_MEM_LS_EN_MASK |
PCIE_CNTL2__MST_MEM_LS_EN_MASK |
PCIE_CNTL2__REPLAY_MEM_LS_EN_MASK;
@@ -1194,14 +1320,14 @@
WREG32_PCIE(ixPCIE_CNTL2, data);
}
-static void fiji_update_hdp_medium_grain_clock_gating(struct amdgpu_device *adev,
- bool enable)
+static void vi_update_hdp_medium_grain_clock_gating(struct amdgpu_device *adev,
+ bool enable)
{
uint32_t temp, data;
temp = data = RREG32(mmHDP_HOST_PATH_CNTL);
- if (enable)
+ if (enable && (adev->cg_flags & AMD_CG_SUPPORT_HDP_MGCG))
data &= ~HDP_HOST_PATH_CNTL__CLOCK_GATING_DIS_MASK;
else
data |= HDP_HOST_PATH_CNTL__CLOCK_GATING_DIS_MASK;
@@ -1210,14 +1336,14 @@
WREG32(mmHDP_HOST_PATH_CNTL, data);
}
-static void fiji_update_hdp_light_sleep(struct amdgpu_device *adev,
- bool enable)
+static void vi_update_hdp_light_sleep(struct amdgpu_device *adev,
+ bool enable)
{
uint32_t temp, data;
temp = data = RREG32(mmHDP_MEM_POWER_LS);
- if (enable)
+ if (enable && (adev->cg_flags & AMD_CG_SUPPORT_HDP_LS))
data |= HDP_MEM_POWER_LS__LS_ENABLE_MASK;
else
data &= ~HDP_MEM_POWER_LS__LS_ENABLE_MASK;
@@ -1226,14 +1352,14 @@
WREG32(mmHDP_MEM_POWER_LS, data);
}
-static void fiji_update_rom_medium_grain_clock_gating(struct amdgpu_device *adev,
- bool enable)
+static void vi_update_rom_medium_grain_clock_gating(struct amdgpu_device *adev,
+ bool enable)
{
uint32_t temp, data;
temp = data = RREG32_SMC(ixCGTT_ROM_CLK_CTRL0);
- if (enable)
+ if (enable && (adev->cg_flags & AMD_CG_SUPPORT_ROM_MGCG))
data &= ~(CGTT_ROM_CLK_CTRL0__SOFT_OVERRIDE0_MASK |
CGTT_ROM_CLK_CTRL0__SOFT_OVERRIDE1_MASK);
else
@@ -1245,19 +1371,28 @@
}
static int vi_common_set_clockgating_state(void *handle,
- enum amd_clockgating_state state)
+ enum amd_clockgating_state state)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
switch (adev->asic_type) {
case CHIP_FIJI:
- fiji_update_bif_medium_grain_light_sleep(adev,
+ vi_update_bif_medium_grain_light_sleep(adev,
state == AMD_CG_STATE_GATE ? true : false);
- fiji_update_hdp_medium_grain_clock_gating(adev,
+ vi_update_hdp_medium_grain_clock_gating(adev,
state == AMD_CG_STATE_GATE ? true : false);
- fiji_update_hdp_light_sleep(adev,
+ vi_update_hdp_light_sleep(adev,
state == AMD_CG_STATE_GATE ? true : false);
- fiji_update_rom_medium_grain_clock_gating(adev,
+ vi_update_rom_medium_grain_clock_gating(adev,
+ state == AMD_CG_STATE_GATE ? true : false);
+ break;
+ case CHIP_CARRIZO:
+ case CHIP_STONEY:
+ vi_update_bif_medium_grain_light_sleep(adev,
+ state == AMD_CG_STATE_GATE ? true : false);
+ vi_update_hdp_medium_grain_clock_gating(adev,
+ state == AMD_CG_STATE_GATE ? true : false);
+ vi_update_hdp_light_sleep(adev,
state == AMD_CG_STATE_GATE ? true : false);
break;
default:
@@ -1273,6 +1408,7 @@
}
const struct amd_ip_funcs vi_common_ip_funcs = {
+ .name = "vi_common",
.early_init = vi_common_early_init,
.late_init = NULL,
.sw_init = vi_common_sw_init,
@@ -1284,7 +1420,6 @@
.is_idle = vi_common_is_idle,
.wait_for_idle = vi_common_wait_for_idle,
.soft_reset = vi_common_soft_reset,
- .print_status = vi_common_print_status,
.set_clockgating_state = vi_common_set_clockgating_state,
.set_powergating_state = vi_common_set_powergating_state,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/vid.h b/drivers/gpu/drm/amd/amdgpu/vid.h
index ace4997..062ee16 100644
--- a/drivers/gpu/drm/amd/amdgpu/vid.h
+++ b/drivers/gpu/drm/amd/amdgpu/vid.h
@@ -54,7 +54,8 @@
#define AUD3_REGISTER_OFFSET (0x17b4 - 0x17a8)
#define AUD4_REGISTER_OFFSET (0x17b8 - 0x17a8)
#define AUD5_REGISTER_OFFSET (0x17bc - 0x17a8)
-#define AUD6_REGISTER_OFFSET (0x17c4 - 0x17a8)
+#define AUD6_REGISTER_OFFSET (0x17c0 - 0x17a8)
+#define AUD7_REGISTER_OFFSET (0x17c4 - 0x17a8)
/* hpd instance offsets */
#define HPD0_REGISTER_OFFSET (0x1898 - 0x1898)
@@ -365,7 +366,7 @@
#define VCE_CMD_IB 0x00000002
#define VCE_CMD_FENCE 0x00000003
#define VCE_CMD_TRAP 0x00000004
-#define VCE_CMD_IB_AUTO 0x00000005
+#define VCE_CMD_IB_AUTO 0x00000005
#define VCE_CMD_SEMAPHORE 0x00000006
#endif
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index 07ac724..ee3e04e 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -109,7 +109,7 @@
is_32bit_user_mode = in_compat_syscall();
- if (is_32bit_user_mode == true) {
+ if (is_32bit_user_mode) {
dev_warn(kfd_device,
"Process %d (32-bit) failed to open /dev/kfd\n"
"32-bit processes are not supported by amdkfd\n",
@@ -131,12 +131,11 @@
void *data)
{
struct kfd_ioctl_get_version_args *args = data;
- int err = 0;
args->major_version = KFD_IOCTL_MAJOR_VERSION;
args->minor_version = KFD_IOCTL_MINOR_VERSION;
- return err;
+ return 0;
}
static int set_queue_properties_from_user(struct queue_properties *q_properties,
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index 4bb7f42..f49c551 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -216,7 +216,7 @@
}
}
- if (set == false)
+ if (!set)
return -EBUSY;
pr_debug("kfd: DQM %s hqd slot - pipe (%d) queue(%d)\n",
@@ -354,7 +354,7 @@
return -ENOMEM;
}
- if (q->properties.is_active == true)
+ if (q->properties.is_active)
prev_active = true;
/*
@@ -363,9 +363,9 @@
* and modify counter accordingly
*/
retval = mqd->update_mqd(mqd, q->mqd, &q->properties);
- if ((q->properties.is_active == true) && (prev_active == false))
+ if ((q->properties.is_active) && (!prev_active))
dqm->queue_count++;
- else if ((q->properties.is_active == false) && (prev_active == true))
+ else if ((!q->properties.is_active) && (prev_active))
dqm->queue_count--;
if (sched_policy != KFD_SCHED_POLICY_NO_HWS)
@@ -954,7 +954,7 @@
if (lock)
mutex_lock(&dqm->lock);
- if (dqm->active_runlist == false)
+ if (!dqm->active_runlist)
goto out;
pr_debug("kfd: Before destroying queues, sdma queue count is : %u\n",
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_events.c b/drivers/gpu/drm/amd/amdkfd/kfd_events.c
index b6e28dc..a6a4b2b 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_events.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_events.c
@@ -177,9 +177,9 @@
bool ret;
ret = allocate_free_slot(p, page, signal_slot_index);
- if (ret == false) {
+ if (!ret) {
ret = allocate_signal_page(devkfd, p);
- if (ret == true)
+ if (ret)
ret = allocate_free_slot(p, page, signal_slot_index);
}
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
index 8fa8941..9beae87 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
@@ -300,7 +300,7 @@
break;
}
- if (kq->ops.initialize(kq, dev, type, KFD_KERNEL_QUEUE_SIZE) == false) {
+ if (!kq->ops.initialize(kq, dev, type, KFD_KERNEL_QUEUE_SIZE)) {
pr_err("amdkfd: failed to init kernel queue\n");
kfree(kq);
return NULL;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager.c
index 90f3914..ca8c093 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager.c
@@ -98,7 +98,7 @@
int retval;
BUG_ON(!pm);
- BUG_ON(pm->allocated == true);
+ BUG_ON(pm->allocated);
BUG_ON(is_over_subscription == NULL);
pm_calc_rlib_size(pm, rl_buffer_size, is_over_subscription);
@@ -292,7 +292,7 @@
q->properties.doorbell_off;
packet->mes_map_queues_ordinals[0].bitfields3.is_static =
- (use_static == true) ? 1 : 0;
+ (use_static) ? 1 : 0;
packet->mes_map_queues_ordinals[0].mqd_addr_lo =
lower_32_bits(q->gart_mqd_addr);
@@ -357,7 +357,7 @@
alloc_size_bytes);
list_for_each_entry(kq, &qpd->priv_queue_list, list) {
- if (kq->queue->properties.is_active != true)
+ if (!kq->queue->properties.is_active)
continue;
pr_debug("kfd: static_queue, mapping kernel q %d, is debug status %d\n",
@@ -383,7 +383,7 @@
}
list_for_each_entry(q, &qpd->queues_list, list) {
- if (q->properties.is_active != true)
+ if (!q->properties.is_active)
continue;
pr_debug("kfd: static_queue, mapping user queue %d, is debug status %d\n",
@@ -531,7 +531,7 @@
fail_acquire_packet_buffer:
mutex_unlock(&pm->lock);
fail_create_runlist_ib:
- if (pm->allocated == true)
+ if (pm->allocated)
pm_release_ib(pm);
return retval;
}
@@ -647,7 +647,7 @@
default:
BUG();
break;
- };
+ }
pm->priv_queue->ops.submit_packet(pm->priv_queue);
diff --git a/drivers/gpu/drm/amd/include/amd_shared.h b/drivers/gpu/drm/amd/include/amd_shared.h
index 04e40906..6080951 100644
--- a/drivers/gpu/drm/amd/include/amd_shared.h
+++ b/drivers/gpu/drm/amd/include/amd_shared.h
@@ -48,6 +48,8 @@
CHIP_FIJI,
CHIP_CARRIZO,
CHIP_STONEY,
+ CHIP_POLARIS10,
+ CHIP_POLARIS11,
CHIP_LAST,
};
@@ -104,6 +106,7 @@
#define AMD_CG_SUPPORT_VCE_MGCG (1 << 14)
#define AMD_CG_SUPPORT_HDP_LS (1 << 15)
#define AMD_CG_SUPPORT_HDP_MGCG (1 << 16)
+#define AMD_CG_SUPPORT_ROM_MGCG (1 << 17)
/* PG flags */
#define AMD_PG_SUPPORT_GFX_PG (1 << 0)
@@ -140,6 +143,8 @@
};
struct amd_ip_funcs {
+ /* Name of IP block */
+ char *name;
/* sets up early driver state (pre sw_init), does not configure hw - Optional */
int (*early_init)(void *handle);
/* sets up late driver/hw state (post hw_init) - Optional */
@@ -162,8 +167,6 @@
int (*wait_for_idle)(void *handle);
/* soft reset the IP block */
int (*soft_reset)(void *handle);
- /* dump the IP block status registers */
- void (*print_status)(void *handle);
/* enable/disable cg for the IP block */
int (*set_clockgating_state)(void *handle,
enum amd_clockgating_state state);
diff --git a/drivers/gpu/drm/amd/include/asic_reg/dce/dce_11_2_d.h b/drivers/gpu/drm/amd/include/asic_reg/dce/dce_11_2_d.h
new file mode 100755
index 0000000..09a7df1
--- /dev/null
+++ b/drivers/gpu/drm/amd/include/asic_reg/dce/dce_11_2_d.h
@@ -0,0 +1,10075 @@
+/*
+ * DCE_11_2 Register documentation
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included
+ * in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+ * OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
+ * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+
+#ifndef DCE_11_2_D_H
+#define DCE_11_2_D_H
+
+#define mmPIPE0_PG_CONFIG 0x2c0
+#define mmPIPE0_PG_ENABLE 0x2c1
+#define mmPIPE0_PG_STATUS 0x2c2
+#define mmPIPE1_PG_CONFIG 0x2c3
+#define mmPIPE1_PG_ENABLE 0x2c4
+#define mmPIPE1_PG_STATUS 0x2c5
+#define mmPIPE2_PG_CONFIG 0x2c6
+#define mmPIPE2_PG_ENABLE 0x2c7
+#define mmPIPE2_PG_STATUS 0x2c8
+#define mmPIPE3_PG_CONFIG 0x2c9
+#define mmPIPE3_PG_ENABLE 0x2ca
+#define mmPIPE3_PG_STATUS 0x2cb
+#define mmPIPE4_PG_CONFIG 0x2cc
+#define mmPIPE4_PG_ENABLE 0x2cd
+#define mmPIPE4_PG_STATUS 0x2ce
+#define mmPIPE5_PG_CONFIG 0x2cf
+#define mmPIPE5_PG_ENABLE 0x2d0
+#define mmPIPE5_PG_STATUS 0x2d1
+#define mmDCPG_INTERRUPT_STATUS 0x2de
+#define mmDCPG_INTERRUPT_CONTROL 0x2df
+#define mmDCPG_INTERRUPT_CONTROL2 0x2e0
+#define mmDC_IP_REQUEST_CNTL 0x2d2
+#define mmDC_PGFSM_CONFIG_REG 0x2d3
+#define mmDC_PGFSM_WRITE_REG 0x2d4
+#define mmDC_PGCNTL_STATUS_REG 0x2d5
+#define mmDCPG_TEST_DEBUG_INDEX 0x2d6
+#define mmDCPG_TEST_DEBUG_DATA 0x2d7
+#define mmBL1_PWM_AMBIENT_LIGHT_LEVEL 0x1628
+#define mmBL1_PWM_USER_LEVEL 0x1629
+#define mmBL1_PWM_TARGET_ABM_LEVEL 0x162a
+#define mmBL1_PWM_CURRENT_ABM_LEVEL 0x162b
+#define mmBL1_PWM_FINAL_DUTY_CYCLE 0x162c
+#define mmBL1_PWM_MINIMUM_DUTY_CYCLE 0x162d
+#define mmBL1_PWM_ABM_CNTL 0x162e
+#define mmBL1_PWM_BL_UPDATE_SAMPLE_RATE 0x162f
+#define mmBL1_PWM_GRP2_REG_LOCK 0x1630
+#define mmDC_ABM1_CNTL 0x1638
+#define mmDC_ABM1_IPCSC_COEFF_SEL 0x1639
+#define mmDC_ABM1_ACE_OFFSET_SLOPE_0 0x163a
+#define mmDC_ABM1_ACE_OFFSET_SLOPE_1 0x163b
+#define mmDC_ABM1_ACE_OFFSET_SLOPE_2 0x163c
+#define mmDC_ABM1_ACE_OFFSET_SLOPE_3 0x163d
+#define mmDC_ABM1_ACE_OFFSET_SLOPE_4 0x163e
+#define mmDC_ABM1_ACE_THRES_12 0x163f
+#define mmDC_ABM1_ACE_THRES_34 0x1640
+#define mmDC_ABM1_ACE_CNTL_MISC 0x1641
+#define mmDC_ABM1_DEBUG_MISC 0x1649
+#define mmDC_ABM1_HGLS_REG_READ_PROGRESS 0x164a
+#define mmDC_ABM1_HG_MISC_CTRL 0x164b
+#define mmDC_ABM1_LS_SUM_OF_LUMA 0x164c
+#define mmDC_ABM1_LS_MIN_MAX_LUMA 0x164d
+#define mmDC_ABM1_LS_FILTERED_MIN_MAX_LUMA 0x164e
+#define mmDC_ABM1_LS_PIXEL_COUNT 0x164f
+#define mmDC_ABM1_LS_OVR_SCAN_BIN 0x1650
+#define mmDC_ABM1_LS_MIN_MAX_PIXEL_VALUE_THRES 0x1651
+#define mmDC_ABM1_LS_MIN_PIXEL_VALUE_COUNT 0x1652
+#define mmDC_ABM1_LS_MAX_PIXEL_VALUE_COUNT 0x1653
+#define mmDC_ABM1_HG_SAMPLE_RATE 0x1654
+#define mmDC_ABM1_LS_SAMPLE_RATE 0x1655
+#define mmDC_ABM1_HG_BIN_1_32_SHIFT_FLAG 0x1656
+#define mmDC_ABM1_HG_BIN_1_8_SHIFT_INDEX 0x1657
+#define mmDC_ABM1_HG_BIN_9_16_SHIFT_INDEX 0x1658
+#define mmDC_ABM1_HG_BIN_17_24_SHIFT_INDEX 0x1659
+#define mmDC_ABM1_HG_BIN_25_32_SHIFT_INDEX 0x165a
+#define mmDC_ABM1_HG_RESULT_1 0x165b
+#define mmDC_ABM1_HG_RESULT_2 0x165c
+#define mmDC_ABM1_HG_RESULT_3 0x165d
+#define mmDC_ABM1_HG_RESULT_4 0x165e
+#define mmDC_ABM1_HG_RESULT_5 0x165f
+#define mmDC_ABM1_HG_RESULT_6 0x1660
+#define mmDC_ABM1_HG_RESULT_7 0x1661
+#define mmDC_ABM1_HG_RESULT_8 0x1662
+#define mmDC_ABM1_HG_RESULT_9 0x1663
+#define mmDC_ABM1_HG_RESULT_10 0x1664
+#define mmDC_ABM1_HG_RESULT_11 0x1665
+#define mmDC_ABM1_HG_RESULT_12 0x1666
+#define mmDC_ABM1_HG_RESULT_13 0x1667
+#define mmDC_ABM1_HG_RESULT_14 0x1668
+#define mmDC_ABM1_HG_RESULT_15 0x1669
+#define mmDC_ABM1_HG_RESULT_16 0x166a
+#define mmDC_ABM1_HG_RESULT_17 0x166b
+#define mmDC_ABM1_HG_RESULT_18 0x166c
+#define mmDC_ABM1_HG_RESULT_19 0x166d
+#define mmDC_ABM1_HG_RESULT_20 0x166e
+#define mmDC_ABM1_HG_RESULT_21 0x166f
+#define mmDC_ABM1_HG_RESULT_22 0x1670
+#define mmDC_ABM1_HG_RESULT_23 0x1671
+#define mmDC_ABM1_HG_RESULT_24 0x1672
+#define mmDC_ABM1_OVERSCAN_PIXEL_VALUE 0x169b
+#define mmDC_ABM1_BL_MASTER_LOCK 0x169c
+#define mmABM_TEST_DEBUG_INDEX 0x169e
+#define mmABM_TEST_DEBUG_DATA 0x169f
+#define mmCRTC_H_BLANK_EARLY_NUM 0x1b7d
+#define mmCRTC0_CRTC_H_BLANK_EARLY_NUM 0x1b7d
+#define mmCRTC1_CRTC_H_BLANK_EARLY_NUM 0x1d7d
+#define mmCRTC2_CRTC_H_BLANK_EARLY_NUM 0x1f7d
+#define mmCRTC3_CRTC_H_BLANK_EARLY_NUM 0x417d
+#define mmCRTC4_CRTC_H_BLANK_EARLY_NUM 0x437d
+#define mmCRTC5_CRTC_H_BLANK_EARLY_NUM 0x457d
+#define mmCRTC_H_TOTAL 0x1b80
+#define mmCRTC0_CRTC_H_TOTAL 0x1b80
+#define mmCRTC1_CRTC_H_TOTAL 0x1d80
+#define mmCRTC2_CRTC_H_TOTAL 0x1f80
+#define mmCRTC3_CRTC_H_TOTAL 0x4180
+#define mmCRTC4_CRTC_H_TOTAL 0x4380
+#define mmCRTC5_CRTC_H_TOTAL 0x4580
+#define mmCRTC_H_BLANK_START_END 0x1b81
+#define mmCRTC0_CRTC_H_BLANK_START_END 0x1b81
+#define mmCRTC1_CRTC_H_BLANK_START_END 0x1d81
+#define mmCRTC2_CRTC_H_BLANK_START_END 0x1f81
+#define mmCRTC3_CRTC_H_BLANK_START_END 0x4181
+#define mmCRTC4_CRTC_H_BLANK_START_END 0x4381
+#define mmCRTC5_CRTC_H_BLANK_START_END 0x4581
+#define mmCRTC_H_SYNC_A 0x1b82
+#define mmCRTC0_CRTC_H_SYNC_A 0x1b82
+#define mmCRTC1_CRTC_H_SYNC_A 0x1d82
+#define mmCRTC2_CRTC_H_SYNC_A 0x1f82
+#define mmCRTC3_CRTC_H_SYNC_A 0x4182
+#define mmCRTC4_CRTC_H_SYNC_A 0x4382
+#define mmCRTC5_CRTC_H_SYNC_A 0x4582
+#define mmCRTC_H_SYNC_A_CNTL 0x1b83
+#define mmCRTC0_CRTC_H_SYNC_A_CNTL 0x1b83
+#define mmCRTC1_CRTC_H_SYNC_A_CNTL 0x1d83
+#define mmCRTC2_CRTC_H_SYNC_A_CNTL 0x1f83
+#define mmCRTC3_CRTC_H_SYNC_A_CNTL 0x4183
+#define mmCRTC4_CRTC_H_SYNC_A_CNTL 0x4383
+#define mmCRTC5_CRTC_H_SYNC_A_CNTL 0x4583
+#define mmCRTC_H_SYNC_B 0x1b84
+#define mmCRTC0_CRTC_H_SYNC_B 0x1b84
+#define mmCRTC1_CRTC_H_SYNC_B 0x1d84
+#define mmCRTC2_CRTC_H_SYNC_B 0x1f84
+#define mmCRTC3_CRTC_H_SYNC_B 0x4184
+#define mmCRTC4_CRTC_H_SYNC_B 0x4384
+#define mmCRTC5_CRTC_H_SYNC_B 0x4584
+#define mmCRTC_H_SYNC_B_CNTL 0x1b85
+#define mmCRTC0_CRTC_H_SYNC_B_CNTL 0x1b85
+#define mmCRTC1_CRTC_H_SYNC_B_CNTL 0x1d85
+#define mmCRTC2_CRTC_H_SYNC_B_CNTL 0x1f85
+#define mmCRTC3_CRTC_H_SYNC_B_CNTL 0x4185
+#define mmCRTC4_CRTC_H_SYNC_B_CNTL 0x4385
+#define mmCRTC5_CRTC_H_SYNC_B_CNTL 0x4585
+#define mmCRTC_VBI_END 0x1b86
+#define mmCRTC0_CRTC_VBI_END 0x1b86
+#define mmCRTC1_CRTC_VBI_END 0x1d86
+#define mmCRTC2_CRTC_VBI_END 0x1f86
+#define mmCRTC3_CRTC_VBI_END 0x4186
+#define mmCRTC4_CRTC_VBI_END 0x4386
+#define mmCRTC5_CRTC_VBI_END 0x4586
+#define mmCRTC_V_TOTAL 0x1b87
+#define mmCRTC0_CRTC_V_TOTAL 0x1b87
+#define mmCRTC1_CRTC_V_TOTAL 0x1d87
+#define mmCRTC2_CRTC_V_TOTAL 0x1f87
+#define mmCRTC3_CRTC_V_TOTAL 0x4187
+#define mmCRTC4_CRTC_V_TOTAL 0x4387
+#define mmCRTC5_CRTC_V_TOTAL 0x4587
+#define mmCRTC_V_TOTAL_MIN 0x1b88
+#define mmCRTC0_CRTC_V_TOTAL_MIN 0x1b88
+#define mmCRTC1_CRTC_V_TOTAL_MIN 0x1d88
+#define mmCRTC2_CRTC_V_TOTAL_MIN 0x1f88
+#define mmCRTC3_CRTC_V_TOTAL_MIN 0x4188
+#define mmCRTC4_CRTC_V_TOTAL_MIN 0x4388
+#define mmCRTC5_CRTC_V_TOTAL_MIN 0x4588
+#define mmCRTC_V_TOTAL_MAX 0x1b89
+#define mmCRTC0_CRTC_V_TOTAL_MAX 0x1b89
+#define mmCRTC1_CRTC_V_TOTAL_MAX 0x1d89
+#define mmCRTC2_CRTC_V_TOTAL_MAX 0x1f89
+#define mmCRTC3_CRTC_V_TOTAL_MAX 0x4189
+#define mmCRTC4_CRTC_V_TOTAL_MAX 0x4389
+#define mmCRTC5_CRTC_V_TOTAL_MAX 0x4589
+#define mmCRTC_V_TOTAL_CONTROL 0x1b8a
+#define mmCRTC0_CRTC_V_TOTAL_CONTROL 0x1b8a
+#define mmCRTC1_CRTC_V_TOTAL_CONTROL 0x1d8a
+#define mmCRTC2_CRTC_V_TOTAL_CONTROL 0x1f8a
+#define mmCRTC3_CRTC_V_TOTAL_CONTROL 0x418a
+#define mmCRTC4_CRTC_V_TOTAL_CONTROL 0x438a
+#define mmCRTC5_CRTC_V_TOTAL_CONTROL 0x458a
+#define mmCRTC_V_TOTAL_INT_STATUS 0x1b8b
+#define mmCRTC0_CRTC_V_TOTAL_INT_STATUS 0x1b8b
+#define mmCRTC1_CRTC_V_TOTAL_INT_STATUS 0x1d8b
+#define mmCRTC2_CRTC_V_TOTAL_INT_STATUS 0x1f8b
+#define mmCRTC3_CRTC_V_TOTAL_INT_STATUS 0x418b
+#define mmCRTC4_CRTC_V_TOTAL_INT_STATUS 0x438b
+#define mmCRTC5_CRTC_V_TOTAL_INT_STATUS 0x458b
+#define mmCRTC_VSYNC_NOM_INT_STATUS 0x1b8c
+#define mmCRTC0_CRTC_VSYNC_NOM_INT_STATUS 0x1b8c
+#define mmCRTC1_CRTC_VSYNC_NOM_INT_STATUS 0x1d8c
+#define mmCRTC2_CRTC_VSYNC_NOM_INT_STATUS 0x1f8c
+#define mmCRTC3_CRTC_VSYNC_NOM_INT_STATUS 0x418c
+#define mmCRTC4_CRTC_VSYNC_NOM_INT_STATUS 0x438c
+#define mmCRTC5_CRTC_VSYNC_NOM_INT_STATUS 0x458c
+#define mmCRTC_V_BLANK_START_END 0x1b8d
+#define mmCRTC0_CRTC_V_BLANK_START_END 0x1b8d
+#define mmCRTC1_CRTC_V_BLANK_START_END 0x1d8d
+#define mmCRTC2_CRTC_V_BLANK_START_END 0x1f8d
+#define mmCRTC3_CRTC_V_BLANK_START_END 0x418d
+#define mmCRTC4_CRTC_V_BLANK_START_END 0x438d
+#define mmCRTC5_CRTC_V_BLANK_START_END 0x458d
+#define mmCRTC_V_SYNC_A 0x1b8e
+#define mmCRTC0_CRTC_V_SYNC_A 0x1b8e
+#define mmCRTC1_CRTC_V_SYNC_A 0x1d8e
+#define mmCRTC2_CRTC_V_SYNC_A 0x1f8e
+#define mmCRTC3_CRTC_V_SYNC_A 0x418e
+#define mmCRTC4_CRTC_V_SYNC_A 0x438e
+#define mmCRTC5_CRTC_V_SYNC_A 0x458e
+#define mmCRTC_V_SYNC_A_CNTL 0x1b8f
+#define mmCRTC0_CRTC_V_SYNC_A_CNTL 0x1b8f
+#define mmCRTC1_CRTC_V_SYNC_A_CNTL 0x1d8f
+#define mmCRTC2_CRTC_V_SYNC_A_CNTL 0x1f8f
+#define mmCRTC3_CRTC_V_SYNC_A_CNTL 0x418f
+#define mmCRTC4_CRTC_V_SYNC_A_CNTL 0x438f
+#define mmCRTC5_CRTC_V_SYNC_A_CNTL 0x458f
+#define mmCRTC_V_SYNC_B 0x1b90
+#define mmCRTC0_CRTC_V_SYNC_B 0x1b90
+#define mmCRTC1_CRTC_V_SYNC_B 0x1d90
+#define mmCRTC2_CRTC_V_SYNC_B 0x1f90
+#define mmCRTC3_CRTC_V_SYNC_B 0x4190
+#define mmCRTC4_CRTC_V_SYNC_B 0x4390
+#define mmCRTC5_CRTC_V_SYNC_B 0x4590
+#define mmCRTC_V_SYNC_B_CNTL 0x1b91
+#define mmCRTC0_CRTC_V_SYNC_B_CNTL 0x1b91
+#define mmCRTC1_CRTC_V_SYNC_B_CNTL 0x1d91
+#define mmCRTC2_CRTC_V_SYNC_B_CNTL 0x1f91
+#define mmCRTC3_CRTC_V_SYNC_B_CNTL 0x4191
+#define mmCRTC4_CRTC_V_SYNC_B_CNTL 0x4391
+#define mmCRTC5_CRTC_V_SYNC_B_CNTL 0x4591
+#define mmCRTC_DTMTEST_CNTL 0x1b92
+#define mmCRTC0_CRTC_DTMTEST_CNTL 0x1b92
+#define mmCRTC1_CRTC_DTMTEST_CNTL 0x1d92
+#define mmCRTC2_CRTC_DTMTEST_CNTL 0x1f92
+#define mmCRTC3_CRTC_DTMTEST_CNTL 0x4192
+#define mmCRTC4_CRTC_DTMTEST_CNTL 0x4392
+#define mmCRTC5_CRTC_DTMTEST_CNTL 0x4592
+#define mmCRTC_DTMTEST_STATUS_POSITION 0x1b93
+#define mmCRTC0_CRTC_DTMTEST_STATUS_POSITION 0x1b93
+#define mmCRTC1_CRTC_DTMTEST_STATUS_POSITION 0x1d93
+#define mmCRTC2_CRTC_DTMTEST_STATUS_POSITION 0x1f93
+#define mmCRTC3_CRTC_DTMTEST_STATUS_POSITION 0x4193
+#define mmCRTC4_CRTC_DTMTEST_STATUS_POSITION 0x4393
+#define mmCRTC5_CRTC_DTMTEST_STATUS_POSITION 0x4593
+#define mmCRTC_TRIGA_CNTL 0x1b94
+#define mmCRTC0_CRTC_TRIGA_CNTL 0x1b94
+#define mmCRTC1_CRTC_TRIGA_CNTL 0x1d94
+#define mmCRTC2_CRTC_TRIGA_CNTL 0x1f94
+#define mmCRTC3_CRTC_TRIGA_CNTL 0x4194
+#define mmCRTC4_CRTC_TRIGA_CNTL 0x4394
+#define mmCRTC5_CRTC_TRIGA_CNTL 0x4594
+#define mmCRTC_TRIGA_MANUAL_TRIG 0x1b95
+#define mmCRTC0_CRTC_TRIGA_MANUAL_TRIG 0x1b95
+#define mmCRTC1_CRTC_TRIGA_MANUAL_TRIG 0x1d95
+#define mmCRTC2_CRTC_TRIGA_MANUAL_TRIG 0x1f95
+#define mmCRTC3_CRTC_TRIGA_MANUAL_TRIG 0x4195
+#define mmCRTC4_CRTC_TRIGA_MANUAL_TRIG 0x4395
+#define mmCRTC5_CRTC_TRIGA_MANUAL_TRIG 0x4595
+#define mmCRTC_TRIGB_CNTL 0x1b96
+#define mmCRTC0_CRTC_TRIGB_CNTL 0x1b96
+#define mmCRTC1_CRTC_TRIGB_CNTL 0x1d96
+#define mmCRTC2_CRTC_TRIGB_CNTL 0x1f96
+#define mmCRTC3_CRTC_TRIGB_CNTL 0x4196
+#define mmCRTC4_CRTC_TRIGB_CNTL 0x4396
+#define mmCRTC5_CRTC_TRIGB_CNTL 0x4596
+#define mmCRTC_TRIGB_MANUAL_TRIG 0x1b97
+#define mmCRTC0_CRTC_TRIGB_MANUAL_TRIG 0x1b97
+#define mmCRTC1_CRTC_TRIGB_MANUAL_TRIG 0x1d97
+#define mmCRTC2_CRTC_TRIGB_MANUAL_TRIG 0x1f97
+#define mmCRTC3_CRTC_TRIGB_MANUAL_TRIG 0x4197
+#define mmCRTC4_CRTC_TRIGB_MANUAL_TRIG 0x4397
+#define mmCRTC5_CRTC_TRIGB_MANUAL_TRIG 0x4597
+#define mmCRTC_FORCE_COUNT_NOW_CNTL 0x1b98
+#define mmCRTC0_CRTC_FORCE_COUNT_NOW_CNTL 0x1b98
+#define mmCRTC1_CRTC_FORCE_COUNT_NOW_CNTL 0x1d98
+#define mmCRTC2_CRTC_FORCE_COUNT_NOW_CNTL 0x1f98
+#define mmCRTC3_CRTC_FORCE_COUNT_NOW_CNTL 0x4198
+#define mmCRTC4_CRTC_FORCE_COUNT_NOW_CNTL 0x4398
+#define mmCRTC5_CRTC_FORCE_COUNT_NOW_CNTL 0x4598
+#define mmCRTC_FLOW_CONTROL 0x1b99
+#define mmCRTC0_CRTC_FLOW_CONTROL 0x1b99
+#define mmCRTC1_CRTC_FLOW_CONTROL 0x1d99
+#define mmCRTC2_CRTC_FLOW_CONTROL 0x1f99
+#define mmCRTC3_CRTC_FLOW_CONTROL 0x4199
+#define mmCRTC4_CRTC_FLOW_CONTROL 0x4399
+#define mmCRTC5_CRTC_FLOW_CONTROL 0x4599
+#define mmCRTC_STEREO_FORCE_NEXT_EYE 0x1b9a
+#define mmCRTC0_CRTC_STEREO_FORCE_NEXT_EYE 0x1b9a
+#define mmCRTC1_CRTC_STEREO_FORCE_NEXT_EYE 0x1d9a
+#define mmCRTC2_CRTC_STEREO_FORCE_NEXT_EYE 0x1f9a
+#define mmCRTC3_CRTC_STEREO_FORCE_NEXT_EYE 0x419a
+#define mmCRTC4_CRTC_STEREO_FORCE_NEXT_EYE 0x439a
+#define mmCRTC5_CRTC_STEREO_FORCE_NEXT_EYE 0x459a
+#define mmCRTC_AVSYNC_COUNTER 0x1b9b
+#define mmCRTC0_CRTC_AVSYNC_COUNTER 0x1b9b
+#define mmCRTC1_CRTC_AVSYNC_COUNTER 0x1d9b
+#define mmCRTC2_CRTC_AVSYNC_COUNTER 0x1f9b
+#define mmCRTC3_CRTC_AVSYNC_COUNTER 0x419b
+#define mmCRTC4_CRTC_AVSYNC_COUNTER 0x439b
+#define mmCRTC5_CRTC_AVSYNC_COUNTER 0x459b
+#define mmCRTC_CONTROL 0x1b9c
+#define mmCRTC0_CRTC_CONTROL 0x1b9c
+#define mmCRTC1_CRTC_CONTROL 0x1d9c
+#define mmCRTC2_CRTC_CONTROL 0x1f9c
+#define mmCRTC3_CRTC_CONTROL 0x419c
+#define mmCRTC4_CRTC_CONTROL 0x439c
+#define mmCRTC5_CRTC_CONTROL 0x459c
+#define mmCRTC_BLANK_CONTROL 0x1b9d
+#define mmCRTC0_CRTC_BLANK_CONTROL 0x1b9d
+#define mmCRTC1_CRTC_BLANK_CONTROL 0x1d9d
+#define mmCRTC2_CRTC_BLANK_CONTROL 0x1f9d
+#define mmCRTC3_CRTC_BLANK_CONTROL 0x419d
+#define mmCRTC4_CRTC_BLANK_CONTROL 0x439d
+#define mmCRTC5_CRTC_BLANK_CONTROL 0x459d
+#define mmCRTC_INTERLACE_CONTROL 0x1b9e
+#define mmCRTC0_CRTC_INTERLACE_CONTROL 0x1b9e
+#define mmCRTC1_CRTC_INTERLACE_CONTROL 0x1d9e
+#define mmCRTC2_CRTC_INTERLACE_CONTROL 0x1f9e
+#define mmCRTC3_CRTC_INTERLACE_CONTROL 0x419e
+#define mmCRTC4_CRTC_INTERLACE_CONTROL 0x439e
+#define mmCRTC5_CRTC_INTERLACE_CONTROL 0x459e
+#define mmCRTC_INTERLACE_STATUS 0x1b9f
+#define mmCRTC0_CRTC_INTERLACE_STATUS 0x1b9f
+#define mmCRTC1_CRTC_INTERLACE_STATUS 0x1d9f
+#define mmCRTC2_CRTC_INTERLACE_STATUS 0x1f9f
+#define mmCRTC3_CRTC_INTERLACE_STATUS 0x419f
+#define mmCRTC4_CRTC_INTERLACE_STATUS 0x439f
+#define mmCRTC5_CRTC_INTERLACE_STATUS 0x459f
+#define mmCRTC_FIELD_INDICATION_CONTROL 0x1ba0
+#define mmCRTC0_CRTC_FIELD_INDICATION_CONTROL 0x1ba0
+#define mmCRTC1_CRTC_FIELD_INDICATION_CONTROL 0x1da0
+#define mmCRTC2_CRTC_FIELD_INDICATION_CONTROL 0x1fa0
+#define mmCRTC3_CRTC_FIELD_INDICATION_CONTROL 0x41a0
+#define mmCRTC4_CRTC_FIELD_INDICATION_CONTROL 0x43a0
+#define mmCRTC5_CRTC_FIELD_INDICATION_CONTROL 0x45a0
+#define mmCRTC_PIXEL_DATA_READBACK0 0x1ba1
+#define mmCRTC0_CRTC_PIXEL_DATA_READBACK0 0x1ba1
+#define mmCRTC1_CRTC_PIXEL_DATA_READBACK0 0x1da1
+#define mmCRTC2_CRTC_PIXEL_DATA_READBACK0 0x1fa1
+#define mmCRTC3_CRTC_PIXEL_DATA_READBACK0 0x41a1
+#define mmCRTC4_CRTC_PIXEL_DATA_READBACK0 0x43a1
+#define mmCRTC5_CRTC_PIXEL_DATA_READBACK0 0x45a1
+#define mmCRTC_PIXEL_DATA_READBACK1 0x1ba2
+#define mmCRTC0_CRTC_PIXEL_DATA_READBACK1 0x1ba2
+#define mmCRTC1_CRTC_PIXEL_DATA_READBACK1 0x1da2
+#define mmCRTC2_CRTC_PIXEL_DATA_READBACK1 0x1fa2
+#define mmCRTC3_CRTC_PIXEL_DATA_READBACK1 0x41a2
+#define mmCRTC4_CRTC_PIXEL_DATA_READBACK1 0x43a2
+#define mmCRTC5_CRTC_PIXEL_DATA_READBACK1 0x45a2
+#define mmCRTC_STATUS 0x1ba3
+#define mmCRTC0_CRTC_STATUS 0x1ba3
+#define mmCRTC1_CRTC_STATUS 0x1da3
+#define mmCRTC2_CRTC_STATUS 0x1fa3
+#define mmCRTC3_CRTC_STATUS 0x41a3
+#define mmCRTC4_CRTC_STATUS 0x43a3
+#define mmCRTC5_CRTC_STATUS 0x45a3
+#define mmCRTC_STATUS_POSITION 0x1ba4
+#define mmCRTC0_CRTC_STATUS_POSITION 0x1ba4
+#define mmCRTC1_CRTC_STATUS_POSITION 0x1da4
+#define mmCRTC2_CRTC_STATUS_POSITION 0x1fa4
+#define mmCRTC3_CRTC_STATUS_POSITION 0x41a4
+#define mmCRTC4_CRTC_STATUS_POSITION 0x43a4
+#define mmCRTC5_CRTC_STATUS_POSITION 0x45a4
+#define mmCRTC_NOM_VERT_POSITION 0x1ba5
+#define mmCRTC0_CRTC_NOM_VERT_POSITION 0x1ba5
+#define mmCRTC1_CRTC_NOM_VERT_POSITION 0x1da5
+#define mmCRTC2_CRTC_NOM_VERT_POSITION 0x1fa5
+#define mmCRTC3_CRTC_NOM_VERT_POSITION 0x41a5
+#define mmCRTC4_CRTC_NOM_VERT_POSITION 0x43a5
+#define mmCRTC5_CRTC_NOM_VERT_POSITION 0x45a5
+#define mmCRTC_STATUS_FRAME_COUNT 0x1ba6
+#define mmCRTC0_CRTC_STATUS_FRAME_COUNT 0x1ba6
+#define mmCRTC1_CRTC_STATUS_FRAME_COUNT 0x1da6
+#define mmCRTC2_CRTC_STATUS_FRAME_COUNT 0x1fa6
+#define mmCRTC3_CRTC_STATUS_FRAME_COUNT 0x41a6
+#define mmCRTC4_CRTC_STATUS_FRAME_COUNT 0x43a6
+#define mmCRTC5_CRTC_STATUS_FRAME_COUNT 0x45a6
+#define mmCRTC_STATUS_VF_COUNT 0x1ba7
+#define mmCRTC0_CRTC_STATUS_VF_COUNT 0x1ba7
+#define mmCRTC1_CRTC_STATUS_VF_COUNT 0x1da7
+#define mmCRTC2_CRTC_STATUS_VF_COUNT 0x1fa7
+#define mmCRTC3_CRTC_STATUS_VF_COUNT 0x41a7
+#define mmCRTC4_CRTC_STATUS_VF_COUNT 0x43a7
+#define mmCRTC5_CRTC_STATUS_VF_COUNT 0x45a7
+#define mmCRTC_STATUS_HV_COUNT 0x1ba8
+#define mmCRTC0_CRTC_STATUS_HV_COUNT 0x1ba8
+#define mmCRTC1_CRTC_STATUS_HV_COUNT 0x1da8
+#define mmCRTC2_CRTC_STATUS_HV_COUNT 0x1fa8
+#define mmCRTC3_CRTC_STATUS_HV_COUNT 0x41a8
+#define mmCRTC4_CRTC_STATUS_HV_COUNT 0x43a8
+#define mmCRTC5_CRTC_STATUS_HV_COUNT 0x45a8
+#define mmCRTC_COUNT_CONTROL 0x1ba9
+#define mmCRTC0_CRTC_COUNT_CONTROL 0x1ba9
+#define mmCRTC1_CRTC_COUNT_CONTROL 0x1da9
+#define mmCRTC2_CRTC_COUNT_CONTROL 0x1fa9
+#define mmCRTC3_CRTC_COUNT_CONTROL 0x41a9
+#define mmCRTC4_CRTC_COUNT_CONTROL 0x43a9
+#define mmCRTC5_CRTC_COUNT_CONTROL 0x45a9
+#define mmCRTC_COUNT_RESET 0x1baa
+#define mmCRTC0_CRTC_COUNT_RESET 0x1baa
+#define mmCRTC1_CRTC_COUNT_RESET 0x1daa
+#define mmCRTC2_CRTC_COUNT_RESET 0x1faa
+#define mmCRTC3_CRTC_COUNT_RESET 0x41aa
+#define mmCRTC4_CRTC_COUNT_RESET 0x43aa
+#define mmCRTC5_CRTC_COUNT_RESET 0x45aa
+#define mmCRTC_MANUAL_FORCE_VSYNC_NEXT_LINE 0x1bab
+#define mmCRTC0_CRTC_MANUAL_FORCE_VSYNC_NEXT_LINE 0x1bab
+#define mmCRTC1_CRTC_MANUAL_FORCE_VSYNC_NEXT_LINE 0x1dab
+#define mmCRTC2_CRTC_MANUAL_FORCE_VSYNC_NEXT_LINE 0x1fab
+#define mmCRTC3_CRTC_MANUAL_FORCE_VSYNC_NEXT_LINE 0x41ab
+#define mmCRTC4_CRTC_MANUAL_FORCE_VSYNC_NEXT_LINE 0x43ab
+#define mmCRTC5_CRTC_MANUAL_FORCE_VSYNC_NEXT_LINE 0x45ab
+#define mmCRTC_VERT_SYNC_CONTROL 0x1bac
+#define mmCRTC0_CRTC_VERT_SYNC_CONTROL 0x1bac
+#define mmCRTC1_CRTC_VERT_SYNC_CONTROL 0x1dac
+#define mmCRTC2_CRTC_VERT_SYNC_CONTROL 0x1fac
+#define mmCRTC3_CRTC_VERT_SYNC_CONTROL 0x41ac
+#define mmCRTC4_CRTC_VERT_SYNC_CONTROL 0x43ac
+#define mmCRTC5_CRTC_VERT_SYNC_CONTROL 0x45ac
+#define mmCRTC_STEREO_STATUS 0x1bad
+#define mmCRTC0_CRTC_STEREO_STATUS 0x1bad
+#define mmCRTC1_CRTC_STEREO_STATUS 0x1dad
+#define mmCRTC2_CRTC_STEREO_STATUS 0x1fad
+#define mmCRTC3_CRTC_STEREO_STATUS 0x41ad
+#define mmCRTC4_CRTC_STEREO_STATUS 0x43ad
+#define mmCRTC5_CRTC_STEREO_STATUS 0x45ad
+#define mmCRTC_STEREO_CONTROL 0x1bae
+#define mmCRTC0_CRTC_STEREO_CONTROL 0x1bae
+#define mmCRTC1_CRTC_STEREO_CONTROL 0x1dae
+#define mmCRTC2_CRTC_STEREO_CONTROL 0x1fae
+#define mmCRTC3_CRTC_STEREO_CONTROL 0x41ae
+#define mmCRTC4_CRTC_STEREO_CONTROL 0x43ae
+#define mmCRTC5_CRTC_STEREO_CONTROL 0x45ae
+#define mmCRTC_SNAPSHOT_STATUS 0x1baf
+#define mmCRTC0_CRTC_SNAPSHOT_STATUS 0x1baf
+#define mmCRTC1_CRTC_SNAPSHOT_STATUS 0x1daf
+#define mmCRTC2_CRTC_SNAPSHOT_STATUS 0x1faf
+#define mmCRTC3_CRTC_SNAPSHOT_STATUS 0x41af
+#define mmCRTC4_CRTC_SNAPSHOT_STATUS 0x43af
+#define mmCRTC5_CRTC_SNAPSHOT_STATUS 0x45af
+#define mmCRTC_SNAPSHOT_CONTROL 0x1bb0
+#define mmCRTC0_CRTC_SNAPSHOT_CONTROL 0x1bb0
+#define mmCRTC1_CRTC_SNAPSHOT_CONTROL 0x1db0
+#define mmCRTC2_CRTC_SNAPSHOT_CONTROL 0x1fb0
+#define mmCRTC3_CRTC_SNAPSHOT_CONTROL 0x41b0
+#define mmCRTC4_CRTC_SNAPSHOT_CONTROL 0x43b0
+#define mmCRTC5_CRTC_SNAPSHOT_CONTROL 0x45b0
+#define mmCRTC_SNAPSHOT_POSITION 0x1bb1
+#define mmCRTC0_CRTC_SNAPSHOT_POSITION 0x1bb1
+#define mmCRTC1_CRTC_SNAPSHOT_POSITION 0x1db1
+#define mmCRTC2_CRTC_SNAPSHOT_POSITION 0x1fb1
+#define mmCRTC3_CRTC_SNAPSHOT_POSITION 0x41b1
+#define mmCRTC4_CRTC_SNAPSHOT_POSITION 0x43b1
+#define mmCRTC5_CRTC_SNAPSHOT_POSITION 0x45b1
+#define mmCRTC_SNAPSHOT_FRAME 0x1bb2
+#define mmCRTC0_CRTC_SNAPSHOT_FRAME 0x1bb2
+#define mmCRTC1_CRTC_SNAPSHOT_FRAME 0x1db2
+#define mmCRTC2_CRTC_SNAPSHOT_FRAME 0x1fb2
+#define mmCRTC3_CRTC_SNAPSHOT_FRAME 0x41b2
+#define mmCRTC4_CRTC_SNAPSHOT_FRAME 0x43b2
+#define mmCRTC5_CRTC_SNAPSHOT_FRAME 0x45b2
+#define mmCRTC_START_LINE_CONTROL 0x1bb3
+#define mmCRTC0_CRTC_START_LINE_CONTROL 0x1bb3
+#define mmCRTC1_CRTC_START_LINE_CONTROL 0x1db3
+#define mmCRTC2_CRTC_START_LINE_CONTROL 0x1fb3
+#define mmCRTC3_CRTC_START_LINE_CONTROL 0x41b3
+#define mmCRTC4_CRTC_START_LINE_CONTROL 0x43b3
+#define mmCRTC5_CRTC_START_LINE_CONTROL 0x45b3
+#define mmCRTC_INTERRUPT_CONTROL 0x1bb4
+#define mmCRTC0_CRTC_INTERRUPT_CONTROL 0x1bb4
+#define mmCRTC1_CRTC_INTERRUPT_CONTROL 0x1db4
+#define mmCRTC2_CRTC_INTERRUPT_CONTROL 0x1fb4
+#define mmCRTC3_CRTC_INTERRUPT_CONTROL 0x41b4
+#define mmCRTC4_CRTC_INTERRUPT_CONTROL 0x43b4
+#define mmCRTC5_CRTC_INTERRUPT_CONTROL 0x45b4
+#define mmCRTC_UPDATE_LOCK 0x1bb5
+#define mmCRTC0_CRTC_UPDATE_LOCK 0x1bb5
+#define mmCRTC1_CRTC_UPDATE_LOCK 0x1db5
+#define mmCRTC2_CRTC_UPDATE_LOCK 0x1fb5
+#define mmCRTC3_CRTC_UPDATE_LOCK 0x41b5
+#define mmCRTC4_CRTC_UPDATE_LOCK 0x43b5
+#define mmCRTC5_CRTC_UPDATE_LOCK 0x45b5
+#define mmCRTC_DOUBLE_BUFFER_CONTROL 0x1bb6
+#define mmCRTC0_CRTC_DOUBLE_BUFFER_CONTROL 0x1bb6
+#define mmCRTC1_CRTC_DOUBLE_BUFFER_CONTROL 0x1db6
+#define mmCRTC2_CRTC_DOUBLE_BUFFER_CONTROL 0x1fb6
+#define mmCRTC3_CRTC_DOUBLE_BUFFER_CONTROL 0x41b6
+#define mmCRTC4_CRTC_DOUBLE_BUFFER_CONTROL 0x43b6
+#define mmCRTC5_CRTC_DOUBLE_BUFFER_CONTROL 0x45b6
+#define mmCRTC_VGA_PARAMETER_CAPTURE_MODE 0x1bb7
+#define mmCRTC0_CRTC_VGA_PARAMETER_CAPTURE_MODE 0x1bb7
+#define mmCRTC1_CRTC_VGA_PARAMETER_CAPTURE_MODE 0x1db7
+#define mmCRTC2_CRTC_VGA_PARAMETER_CAPTURE_MODE 0x1fb7
+#define mmCRTC3_CRTC_VGA_PARAMETER_CAPTURE_MODE 0x41b7
+#define mmCRTC4_CRTC_VGA_PARAMETER_CAPTURE_MODE 0x43b7
+#define mmCRTC5_CRTC_VGA_PARAMETER_CAPTURE_MODE 0x45b7
+#define mmCRTC_TEST_PATTERN_CONTROL 0x1bba
+#define mmCRTC0_CRTC_TEST_PATTERN_CONTROL 0x1bba
+#define mmCRTC1_CRTC_TEST_PATTERN_CONTROL 0x1dba
+#define mmCRTC2_CRTC_TEST_PATTERN_CONTROL 0x1fba
+#define mmCRTC3_CRTC_TEST_PATTERN_CONTROL 0x41ba
+#define mmCRTC4_CRTC_TEST_PATTERN_CONTROL 0x43ba
+#define mmCRTC5_CRTC_TEST_PATTERN_CONTROL 0x45ba
+#define mmCRTC_TEST_PATTERN_PARAMETERS 0x1bbb
+#define mmCRTC0_CRTC_TEST_PATTERN_PARAMETERS 0x1bbb
+#define mmCRTC1_CRTC_TEST_PATTERN_PARAMETERS 0x1dbb
+#define mmCRTC2_CRTC_TEST_PATTERN_PARAMETERS 0x1fbb
+#define mmCRTC3_CRTC_TEST_PATTERN_PARAMETERS 0x41bb
+#define mmCRTC4_CRTC_TEST_PATTERN_PARAMETERS 0x43bb
+#define mmCRTC5_CRTC_TEST_PATTERN_PARAMETERS 0x45bb
+#define mmCRTC_TEST_PATTERN_COLOR 0x1bbc
+#define mmCRTC0_CRTC_TEST_PATTERN_COLOR 0x1bbc
+#define mmCRTC1_CRTC_TEST_PATTERN_COLOR 0x1dbc
+#define mmCRTC2_CRTC_TEST_PATTERN_COLOR 0x1fbc
+#define mmCRTC3_CRTC_TEST_PATTERN_COLOR 0x41bc
+#define mmCRTC4_CRTC_TEST_PATTERN_COLOR 0x43bc
+#define mmCRTC5_CRTC_TEST_PATTERN_COLOR 0x45bc
+#define mmCRTC_MASTER_UPDATE_LOCK 0x1bbd
+#define mmCRTC0_CRTC_MASTER_UPDATE_LOCK 0x1bbd
+#define mmCRTC1_CRTC_MASTER_UPDATE_LOCK 0x1dbd
+#define mmCRTC2_CRTC_MASTER_UPDATE_LOCK 0x1fbd
+#define mmCRTC3_CRTC_MASTER_UPDATE_LOCK 0x41bd
+#define mmCRTC4_CRTC_MASTER_UPDATE_LOCK 0x43bd
+#define mmCRTC5_CRTC_MASTER_UPDATE_LOCK 0x45bd
+#define mmCRTC_MASTER_UPDATE_MODE 0x1bbe
+#define mmCRTC0_CRTC_MASTER_UPDATE_MODE 0x1bbe
+#define mmCRTC1_CRTC_MASTER_UPDATE_MODE 0x1dbe
+#define mmCRTC2_CRTC_MASTER_UPDATE_MODE 0x1fbe
+#define mmCRTC3_CRTC_MASTER_UPDATE_MODE 0x41be
+#define mmCRTC4_CRTC_MASTER_UPDATE_MODE 0x43be
+#define mmCRTC5_CRTC_MASTER_UPDATE_MODE 0x45be
+#define mmCRTC_MVP_INBAND_CNTL_INSERT 0x1bbf
+#define mmCRTC0_CRTC_MVP_INBAND_CNTL_INSERT 0x1bbf
+#define mmCRTC1_CRTC_MVP_INBAND_CNTL_INSERT 0x1dbf
+#define mmCRTC2_CRTC_MVP_INBAND_CNTL_INSERT 0x1fbf
+#define mmCRTC3_CRTC_MVP_INBAND_CNTL_INSERT 0x41bf
+#define mmCRTC4_CRTC_MVP_INBAND_CNTL_INSERT 0x43bf
+#define mmCRTC5_CRTC_MVP_INBAND_CNTL_INSERT 0x45bf
+#define mmCRTC_MVP_INBAND_CNTL_INSERT_TIMER 0x1bc0
+#define mmCRTC0_CRTC_MVP_INBAND_CNTL_INSERT_TIMER 0x1bc0
+#define mmCRTC1_CRTC_MVP_INBAND_CNTL_INSERT_TIMER 0x1dc0
+#define mmCRTC2_CRTC_MVP_INBAND_CNTL_INSERT_TIMER 0x1fc0
+#define mmCRTC3_CRTC_MVP_INBAND_CNTL_INSERT_TIMER 0x41c0
+#define mmCRTC4_CRTC_MVP_INBAND_CNTL_INSERT_TIMER 0x43c0
+#define mmCRTC5_CRTC_MVP_INBAND_CNTL_INSERT_TIMER 0x45c0
+#define mmCRTC_MVP_STATUS 0x1bc1
+#define mmCRTC0_CRTC_MVP_STATUS 0x1bc1
+#define mmCRTC1_CRTC_MVP_STATUS 0x1dc1
+#define mmCRTC2_CRTC_MVP_STATUS 0x1fc1
+#define mmCRTC3_CRTC_MVP_STATUS 0x41c1
+#define mmCRTC4_CRTC_MVP_STATUS 0x43c1
+#define mmCRTC5_CRTC_MVP_STATUS 0x45c1
+#define mmCRTC_MASTER_EN 0x1bc2
+#define mmCRTC0_CRTC_MASTER_EN 0x1bc2
+#define mmCRTC1_CRTC_MASTER_EN 0x1dc2
+#define mmCRTC2_CRTC_MASTER_EN 0x1fc2
+#define mmCRTC3_CRTC_MASTER_EN 0x41c2
+#define mmCRTC4_CRTC_MASTER_EN 0x43c2
+#define mmCRTC5_CRTC_MASTER_EN 0x45c2
+#define mmCRTC_ALLOW_STOP_OFF_V_CNT 0x1bc3
+#define mmCRTC0_CRTC_ALLOW_STOP_OFF_V_CNT 0x1bc3
+#define mmCRTC1_CRTC_ALLOW_STOP_OFF_V_CNT 0x1dc3
+#define mmCRTC2_CRTC_ALLOW_STOP_OFF_V_CNT 0x1fc3
+#define mmCRTC3_CRTC_ALLOW_STOP_OFF_V_CNT 0x41c3
+#define mmCRTC4_CRTC_ALLOW_STOP_OFF_V_CNT 0x43c3
+#define mmCRTC5_CRTC_ALLOW_STOP_OFF_V_CNT 0x45c3
+#define mmCRTC_V_UPDATE_INT_STATUS 0x1bc4
+#define mmCRTC0_CRTC_V_UPDATE_INT_STATUS 0x1bc4
+#define mmCRTC1_CRTC_V_UPDATE_INT_STATUS 0x1dc4
+#define mmCRTC2_CRTC_V_UPDATE_INT_STATUS 0x1fc4
+#define mmCRTC3_CRTC_V_UPDATE_INT_STATUS 0x41c4
+#define mmCRTC4_CRTC_V_UPDATE_INT_STATUS 0x43c4
+#define mmCRTC5_CRTC_V_UPDATE_INT_STATUS 0x45c4
+#define mmCRTC_OVERSCAN_COLOR 0x1bc8
+#define mmCRTC0_CRTC_OVERSCAN_COLOR 0x1bc8
+#define mmCRTC1_CRTC_OVERSCAN_COLOR 0x1dc8
+#define mmCRTC2_CRTC_OVERSCAN_COLOR 0x1fc8
+#define mmCRTC3_CRTC_OVERSCAN_COLOR 0x41c8
+#define mmCRTC4_CRTC_OVERSCAN_COLOR 0x43c8
+#define mmCRTC5_CRTC_OVERSCAN_COLOR 0x45c8
+#define mmCRTC_OVERSCAN_COLOR_EXT 0x1bc9
+#define mmCRTC0_CRTC_OVERSCAN_COLOR_EXT 0x1bc9
+#define mmCRTC1_CRTC_OVERSCAN_COLOR_EXT 0x1dc9
+#define mmCRTC2_CRTC_OVERSCAN_COLOR_EXT 0x1fc9
+#define mmCRTC3_CRTC_OVERSCAN_COLOR_EXT 0x41c9
+#define mmCRTC4_CRTC_OVERSCAN_COLOR_EXT 0x43c9
+#define mmCRTC5_CRTC_OVERSCAN_COLOR_EXT 0x45c9
+#define mmCRTC_BLANK_DATA_COLOR 0x1bca
+#define mmCRTC0_CRTC_BLANK_DATA_COLOR 0x1bca
+#define mmCRTC1_CRTC_BLANK_DATA_COLOR 0x1dca
+#define mmCRTC2_CRTC_BLANK_DATA_COLOR 0x1fca
+#define mmCRTC3_CRTC_BLANK_DATA_COLOR 0x41ca
+#define mmCRTC4_CRTC_BLANK_DATA_COLOR 0x43ca
+#define mmCRTC5_CRTC_BLANK_DATA_COLOR 0x45ca
+#define mmCRTC_BLANK_DATA_COLOR_EXT 0x1bcb
+#define mmCRTC0_CRTC_BLANK_DATA_COLOR_EXT 0x1bcb
+#define mmCRTC1_CRTC_BLANK_DATA_COLOR_EXT 0x1dcb
+#define mmCRTC2_CRTC_BLANK_DATA_COLOR_EXT 0x1fcb
+#define mmCRTC3_CRTC_BLANK_DATA_COLOR_EXT 0x41cb
+#define mmCRTC4_CRTC_BLANK_DATA_COLOR_EXT 0x43cb
+#define mmCRTC5_CRTC_BLANK_DATA_COLOR_EXT 0x45cb
+#define mmCRTC_BLACK_COLOR 0x1bcc
+#define mmCRTC0_CRTC_BLACK_COLOR 0x1bcc
+#define mmCRTC1_CRTC_BLACK_COLOR 0x1dcc
+#define mmCRTC2_CRTC_BLACK_COLOR 0x1fcc
+#define mmCRTC3_CRTC_BLACK_COLOR 0x41cc
+#define mmCRTC4_CRTC_BLACK_COLOR 0x43cc
+#define mmCRTC5_CRTC_BLACK_COLOR 0x45cc
+#define mmCRTC_BLACK_COLOR_EXT 0x1bcd
+#define mmCRTC0_CRTC_BLACK_COLOR_EXT 0x1bcd
+#define mmCRTC1_CRTC_BLACK_COLOR_EXT 0x1dcd
+#define mmCRTC2_CRTC_BLACK_COLOR_EXT 0x1fcd
+#define mmCRTC3_CRTC_BLACK_COLOR_EXT 0x41cd
+#define mmCRTC4_CRTC_BLACK_COLOR_EXT 0x43cd
+#define mmCRTC5_CRTC_BLACK_COLOR_EXT 0x45cd
+#define mmCRTC_VERTICAL_INTERRUPT0_POSITION 0x1bce
+#define mmCRTC0_CRTC_VERTICAL_INTERRUPT0_POSITION 0x1bce
+#define mmCRTC1_CRTC_VERTICAL_INTERRUPT0_POSITION 0x1dce
+#define mmCRTC2_CRTC_VERTICAL_INTERRUPT0_POSITION 0x1fce
+#define mmCRTC3_CRTC_VERTICAL_INTERRUPT0_POSITION 0x41ce
+#define mmCRTC4_CRTC_VERTICAL_INTERRUPT0_POSITION 0x43ce
+#define mmCRTC5_CRTC_VERTICAL_INTERRUPT0_POSITION 0x45ce
+#define mmCRTC_VERTICAL_INTERRUPT0_CONTROL 0x1bcf
+#define mmCRTC0_CRTC_VERTICAL_INTERRUPT0_CONTROL 0x1bcf
+#define mmCRTC1_CRTC_VERTICAL_INTERRUPT0_CONTROL 0x1dcf
+#define mmCRTC2_CRTC_VERTICAL_INTERRUPT0_CONTROL 0x1fcf
+#define mmCRTC3_CRTC_VERTICAL_INTERRUPT0_CONTROL 0x41cf
+#define mmCRTC4_CRTC_VERTICAL_INTERRUPT0_CONTROL 0x43cf
+#define mmCRTC5_CRTC_VERTICAL_INTERRUPT0_CONTROL 0x45cf
+#define mmCRTC_VERTICAL_INTERRUPT1_POSITION 0x1bd0
+#define mmCRTC0_CRTC_VERTICAL_INTERRUPT1_POSITION 0x1bd0
+#define mmCRTC1_CRTC_VERTICAL_INTERRUPT1_POSITION 0x1dd0
+#define mmCRTC2_CRTC_VERTICAL_INTERRUPT1_POSITION 0x1fd0
+#define mmCRTC3_CRTC_VERTICAL_INTERRUPT1_POSITION 0x41d0
+#define mmCRTC4_CRTC_VERTICAL_INTERRUPT1_POSITION 0x43d0
+#define mmCRTC5_CRTC_VERTICAL_INTERRUPT1_POSITION 0x45d0
+#define mmCRTC_VERTICAL_INTERRUPT1_CONTROL 0x1bd1
+#define mmCRTC0_CRTC_VERTICAL_INTERRUPT1_CONTROL 0x1bd1
+#define mmCRTC1_CRTC_VERTICAL_INTERRUPT1_CONTROL 0x1dd1
+#define mmCRTC2_CRTC_VERTICAL_INTERRUPT1_CONTROL 0x1fd1
+#define mmCRTC3_CRTC_VERTICAL_INTERRUPT1_CONTROL 0x41d1
+#define mmCRTC4_CRTC_VERTICAL_INTERRUPT1_CONTROL 0x43d1
+#define mmCRTC5_CRTC_VERTICAL_INTERRUPT1_CONTROL 0x45d1
+#define mmCRTC_VERTICAL_INTERRUPT2_POSITION 0x1bd2
+#define mmCRTC0_CRTC_VERTICAL_INTERRUPT2_POSITION 0x1bd2
+#define mmCRTC1_CRTC_VERTICAL_INTERRUPT2_POSITION 0x1dd2
+#define mmCRTC2_CRTC_VERTICAL_INTERRUPT2_POSITION 0x1fd2
+#define mmCRTC3_CRTC_VERTICAL_INTERRUPT2_POSITION 0x41d2
+#define mmCRTC4_CRTC_VERTICAL_INTERRUPT2_POSITION 0x43d2
+#define mmCRTC5_CRTC_VERTICAL_INTERRUPT2_POSITION 0x45d2
+#define mmCRTC_VERTICAL_INTERRUPT2_CONTROL 0x1bd3
+#define mmCRTC0_CRTC_VERTICAL_INTERRUPT2_CONTROL 0x1bd3
+#define mmCRTC1_CRTC_VERTICAL_INTERRUPT2_CONTROL 0x1dd3
+#define mmCRTC2_CRTC_VERTICAL_INTERRUPT2_CONTROL 0x1fd3
+#define mmCRTC3_CRTC_VERTICAL_INTERRUPT2_CONTROL 0x41d3
+#define mmCRTC4_CRTC_VERTICAL_INTERRUPT2_CONTROL 0x43d3
+#define mmCRTC5_CRTC_VERTICAL_INTERRUPT2_CONTROL 0x45d3
+#define mmCRTC_CRC_CNTL 0x1bd4
+#define mmCRTC0_CRTC_CRC_CNTL 0x1bd4
+#define mmCRTC1_CRTC_CRC_CNTL 0x1dd4
+#define mmCRTC2_CRTC_CRC_CNTL 0x1fd4
+#define mmCRTC3_CRTC_CRC_CNTL 0x41d4
+#define mmCRTC4_CRTC_CRC_CNTL 0x43d4
+#define mmCRTC5_CRTC_CRC_CNTL 0x45d4
+#define mmCRTC_CRC0_WINDOWA_X_CONTROL 0x1bd5
+#define mmCRTC0_CRTC_CRC0_WINDOWA_X_CONTROL 0x1bd5
+#define mmCRTC1_CRTC_CRC0_WINDOWA_X_CONTROL 0x1dd5
+#define mmCRTC2_CRTC_CRC0_WINDOWA_X_CONTROL 0x1fd5
+#define mmCRTC3_CRTC_CRC0_WINDOWA_X_CONTROL 0x41d5
+#define mmCRTC4_CRTC_CRC0_WINDOWA_X_CONTROL 0x43d5
+#define mmCRTC5_CRTC_CRC0_WINDOWA_X_CONTROL 0x45d5
+#define mmCRTC_CRC0_WINDOWA_Y_CONTROL 0x1bd6
+#define mmCRTC0_CRTC_CRC0_WINDOWA_Y_CONTROL 0x1bd6
+#define mmCRTC1_CRTC_CRC0_WINDOWA_Y_CONTROL 0x1dd6
+#define mmCRTC2_CRTC_CRC0_WINDOWA_Y_CONTROL 0x1fd6
+#define mmCRTC3_CRTC_CRC0_WINDOWA_Y_CONTROL 0x41d6
+#define mmCRTC4_CRTC_CRC0_WINDOWA_Y_CONTROL 0x43d6
+#define mmCRTC5_CRTC_CRC0_WINDOWA_Y_CONTROL 0x45d6
+#define mmCRTC_CRC0_WINDOWB_X_CONTROL 0x1bd7
+#define mmCRTC0_CRTC_CRC0_WINDOWB_X_CONTROL 0x1bd7
+#define mmCRTC1_CRTC_CRC0_WINDOWB_X_CONTROL 0x1dd7
+#define mmCRTC2_CRTC_CRC0_WINDOWB_X_CONTROL 0x1fd7
+#define mmCRTC3_CRTC_CRC0_WINDOWB_X_CONTROL 0x41d7
+#define mmCRTC4_CRTC_CRC0_WINDOWB_X_CONTROL 0x43d7
+#define mmCRTC5_CRTC_CRC0_WINDOWB_X_CONTROL 0x45d7
+#define mmCRTC_CRC0_WINDOWB_Y_CONTROL 0x1bd8
+#define mmCRTC0_CRTC_CRC0_WINDOWB_Y_CONTROL 0x1bd8
+#define mmCRTC1_CRTC_CRC0_WINDOWB_Y_CONTROL 0x1dd8
+#define mmCRTC2_CRTC_CRC0_WINDOWB_Y_CONTROL 0x1fd8
+#define mmCRTC3_CRTC_CRC0_WINDOWB_Y_CONTROL 0x41d8
+#define mmCRTC4_CRTC_CRC0_WINDOWB_Y_CONTROL 0x43d8
+#define mmCRTC5_CRTC_CRC0_WINDOWB_Y_CONTROL 0x45d8
+#define mmCRTC_CRC0_DATA_RG 0x1bd9
+#define mmCRTC0_CRTC_CRC0_DATA_RG 0x1bd9
+#define mmCRTC1_CRTC_CRC0_DATA_RG 0x1dd9
+#define mmCRTC2_CRTC_CRC0_DATA_RG 0x1fd9
+#define mmCRTC3_CRTC_CRC0_DATA_RG 0x41d9
+#define mmCRTC4_CRTC_CRC0_DATA_RG 0x43d9
+#define mmCRTC5_CRTC_CRC0_DATA_RG 0x45d9
+#define mmCRTC_CRC0_DATA_B 0x1bda
+#define mmCRTC0_CRTC_CRC0_DATA_B 0x1bda
+#define mmCRTC1_CRTC_CRC0_DATA_B 0x1dda
+#define mmCRTC2_CRTC_CRC0_DATA_B 0x1fda
+#define mmCRTC3_CRTC_CRC0_DATA_B 0x41da
+#define mmCRTC4_CRTC_CRC0_DATA_B 0x43da
+#define mmCRTC5_CRTC_CRC0_DATA_B 0x45da
+#define mmCRTC_CRC1_WINDOWA_X_CONTROL 0x1bdb
+#define mmCRTC0_CRTC_CRC1_WINDOWA_X_CONTROL 0x1bdb
+#define mmCRTC1_CRTC_CRC1_WINDOWA_X_CONTROL 0x1ddb
+#define mmCRTC2_CRTC_CRC1_WINDOWA_X_CONTROL 0x1fdb
+#define mmCRTC3_CRTC_CRC1_WINDOWA_X_CONTROL 0x41db
+#define mmCRTC4_CRTC_CRC1_WINDOWA_X_CONTROL 0x43db
+#define mmCRTC5_CRTC_CRC1_WINDOWA_X_CONTROL 0x45db
+#define mmCRTC_CRC1_WINDOWA_Y_CONTROL 0x1bdc
+#define mmCRTC0_CRTC_CRC1_WINDOWA_Y_CONTROL 0x1bdc
+#define mmCRTC1_CRTC_CRC1_WINDOWA_Y_CONTROL 0x1ddc
+#define mmCRTC2_CRTC_CRC1_WINDOWA_Y_CONTROL 0x1fdc
+#define mmCRTC3_CRTC_CRC1_WINDOWA_Y_CONTROL 0x41dc
+#define mmCRTC4_CRTC_CRC1_WINDOWA_Y_CONTROL 0x43dc
+#define mmCRTC5_CRTC_CRC1_WINDOWA_Y_CONTROL 0x45dc
+#define mmCRTC_CRC1_WINDOWB_X_CONTROL 0x1bdd
+#define mmCRTC0_CRTC_CRC1_WINDOWB_X_CONTROL 0x1bdd
+#define mmCRTC1_CRTC_CRC1_WINDOWB_X_CONTROL 0x1ddd
+#define mmCRTC2_CRTC_CRC1_WINDOWB_X_CONTROL 0x1fdd
+#define mmCRTC3_CRTC_CRC1_WINDOWB_X_CONTROL 0x41dd
+#define mmCRTC4_CRTC_CRC1_WINDOWB_X_CONTROL 0x43dd
+#define mmCRTC5_CRTC_CRC1_WINDOWB_X_CONTROL 0x45dd
+#define mmCRTC_CRC1_WINDOWB_Y_CONTROL 0x1bde
+#define mmCRTC0_CRTC_CRC1_WINDOWB_Y_CONTROL 0x1bde
+#define mmCRTC1_CRTC_CRC1_WINDOWB_Y_CONTROL 0x1dde
+#define mmCRTC2_CRTC_CRC1_WINDOWB_Y_CONTROL 0x1fde
+#define mmCRTC3_CRTC_CRC1_WINDOWB_Y_CONTROL 0x41de
+#define mmCRTC4_CRTC_CRC1_WINDOWB_Y_CONTROL 0x43de
+#define mmCRTC5_CRTC_CRC1_WINDOWB_Y_CONTROL 0x45de
+#define mmCRTC_CRC1_DATA_RG 0x1bdf
+#define mmCRTC0_CRTC_CRC1_DATA_RG 0x1bdf
+#define mmCRTC1_CRTC_CRC1_DATA_RG 0x1ddf
+#define mmCRTC2_CRTC_CRC1_DATA_RG 0x1fdf
+#define mmCRTC3_CRTC_CRC1_DATA_RG 0x41df
+#define mmCRTC4_CRTC_CRC1_DATA_RG 0x43df
+#define mmCRTC5_CRTC_CRC1_DATA_RG 0x45df
+#define mmCRTC_CRC1_DATA_B 0x1be0
+#define mmCRTC0_CRTC_CRC1_DATA_B 0x1be0
+#define mmCRTC1_CRTC_CRC1_DATA_B 0x1de0
+#define mmCRTC2_CRTC_CRC1_DATA_B 0x1fe0
+#define mmCRTC3_CRTC_CRC1_DATA_B 0x41e0
+#define mmCRTC4_CRTC_CRC1_DATA_B 0x43e0
+#define mmCRTC5_CRTC_CRC1_DATA_B 0x45e0
+#define mmCRTC_EXT_TIMING_SYNC_CONTROL 0x1be1
+#define mmCRTC0_CRTC_EXT_TIMING_SYNC_CONTROL 0x1be1
+#define mmCRTC1_CRTC_EXT_TIMING_SYNC_CONTROL 0x1de1
+#define mmCRTC2_CRTC_EXT_TIMING_SYNC_CONTROL 0x1fe1
+#define mmCRTC3_CRTC_EXT_TIMING_SYNC_CONTROL 0x41e1
+#define mmCRTC4_CRTC_EXT_TIMING_SYNC_CONTROL 0x43e1
+#define mmCRTC5_CRTC_EXT_TIMING_SYNC_CONTROL 0x45e1
+#define mmCRTC_EXT_TIMING_SYNC_WINDOW_START 0x1be2
+#define mmCRTC0_CRTC_EXT_TIMING_SYNC_WINDOW_START 0x1be2
+#define mmCRTC1_CRTC_EXT_TIMING_SYNC_WINDOW_START 0x1de2
+#define mmCRTC2_CRTC_EXT_TIMING_SYNC_WINDOW_START 0x1fe2
+#define mmCRTC3_CRTC_EXT_TIMING_SYNC_WINDOW_START 0x41e2
+#define mmCRTC4_CRTC_EXT_TIMING_SYNC_WINDOW_START 0x43e2
+#define mmCRTC5_CRTC_EXT_TIMING_SYNC_WINDOW_START 0x45e2
+#define mmCRTC_EXT_TIMING_SYNC_WINDOW_END 0x1be3
+#define mmCRTC0_CRTC_EXT_TIMING_SYNC_WINDOW_END 0x1be3
+#define mmCRTC1_CRTC_EXT_TIMING_SYNC_WINDOW_END 0x1de3
+#define mmCRTC2_CRTC_EXT_TIMING_SYNC_WINDOW_END 0x1fe3
+#define mmCRTC3_CRTC_EXT_TIMING_SYNC_WINDOW_END 0x41e3
+#define mmCRTC4_CRTC_EXT_TIMING_SYNC_WINDOW_END 0x43e3
+#define mmCRTC5_CRTC_EXT_TIMING_SYNC_WINDOW_END 0x45e3
+#define mmCRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL 0x1be4
+#define mmCRTC0_CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL 0x1be4
+#define mmCRTC1_CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL 0x1de4
+#define mmCRTC2_CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL 0x1fe4
+#define mmCRTC3_CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL 0x41e4
+#define mmCRTC4_CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL 0x43e4
+#define mmCRTC5_CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL 0x45e4
+#define mmCRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL 0x1be5
+#define mmCRTC0_CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL 0x1be5
+#define mmCRTC1_CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL 0x1de5
+#define mmCRTC2_CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL 0x1fe5
+#define mmCRTC3_CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL 0x41e5
+#define mmCRTC4_CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL 0x43e5
+#define mmCRTC5_CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL 0x45e5
+#define mmCRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL 0x1be6
+#define mmCRTC0_CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL 0x1be6
+#define mmCRTC1_CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL 0x1de6
+#define mmCRTC2_CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL 0x1fe6
+#define mmCRTC3_CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL 0x41e6
+#define mmCRTC4_CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL 0x43e6
+#define mmCRTC5_CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL 0x45e6
+#define mmCRTC_STATIC_SCREEN_CONTROL 0x1be7
+#define mmCRTC0_CRTC_STATIC_SCREEN_CONTROL 0x1be7
+#define mmCRTC1_CRTC_STATIC_SCREEN_CONTROL 0x1de7
+#define mmCRTC2_CRTC_STATIC_SCREEN_CONTROL 0x1fe7
+#define mmCRTC3_CRTC_STATIC_SCREEN_CONTROL 0x41e7
+#define mmCRTC4_CRTC_STATIC_SCREEN_CONTROL 0x43e7
+#define mmCRTC5_CRTC_STATIC_SCREEN_CONTROL 0x45e7
+#define mmCRTC_3D_STRUCTURE_CONTROL 0x1b78
+#define mmCRTC0_CRTC_3D_STRUCTURE_CONTROL 0x1b78
+#define mmCRTC1_CRTC_3D_STRUCTURE_CONTROL 0x1d78
+#define mmCRTC2_CRTC_3D_STRUCTURE_CONTROL 0x1f78
+#define mmCRTC3_CRTC_3D_STRUCTURE_CONTROL 0x4178
+#define mmCRTC4_CRTC_3D_STRUCTURE_CONTROL 0x4378
+#define mmCRTC5_CRTC_3D_STRUCTURE_CONTROL 0x4578
+#define mmCRTC_GSL_VSYNC_GAP 0x1b79
+#define mmCRTC0_CRTC_GSL_VSYNC_GAP 0x1b79
+#define mmCRTC1_CRTC_GSL_VSYNC_GAP 0x1d79
+#define mmCRTC2_CRTC_GSL_VSYNC_GAP 0x1f79
+#define mmCRTC3_CRTC_GSL_VSYNC_GAP 0x4179
+#define mmCRTC4_CRTC_GSL_VSYNC_GAP 0x4379
+#define mmCRTC5_CRTC_GSL_VSYNC_GAP 0x4579
+#define mmCRTC_GSL_WINDOW 0x1b7a
+#define mmCRTC0_CRTC_GSL_WINDOW 0x1b7a
+#define mmCRTC1_CRTC_GSL_WINDOW 0x1d7a
+#define mmCRTC2_CRTC_GSL_WINDOW 0x1f7a
+#define mmCRTC3_CRTC_GSL_WINDOW 0x417a
+#define mmCRTC4_CRTC_GSL_WINDOW 0x437a
+#define mmCRTC5_CRTC_GSL_WINDOW 0x457a
+#define mmCRTC_GSL_CONTROL 0x1b7b
+#define mmCRTC0_CRTC_GSL_CONTROL 0x1b7b
+#define mmCRTC1_CRTC_GSL_CONTROL 0x1d7b
+#define mmCRTC2_CRTC_GSL_CONTROL 0x1f7b
+#define mmCRTC3_CRTC_GSL_CONTROL 0x417b
+#define mmCRTC4_CRTC_GSL_CONTROL 0x437b
+#define mmCRTC5_CRTC_GSL_CONTROL 0x457b
+#define mmCRTC_TEST_DEBUG_INDEX 0x1bc6
+#define mmCRTC0_CRTC_TEST_DEBUG_INDEX 0x1bc6
+#define mmCRTC1_CRTC_TEST_DEBUG_INDEX 0x1dc6
+#define mmCRTC2_CRTC_TEST_DEBUG_INDEX 0x1fc6
+#define mmCRTC3_CRTC_TEST_DEBUG_INDEX 0x41c6
+#define mmCRTC4_CRTC_TEST_DEBUG_INDEX 0x43c6
+#define mmCRTC5_CRTC_TEST_DEBUG_INDEX 0x45c6
+#define mmCRTC_TEST_DEBUG_DATA 0x1bc7
+#define mmCRTC0_CRTC_TEST_DEBUG_DATA 0x1bc7
+#define mmCRTC1_CRTC_TEST_DEBUG_DATA 0x1dc7
+#define mmCRTC2_CRTC_TEST_DEBUG_DATA 0x1fc7
+#define mmCRTC3_CRTC_TEST_DEBUG_DATA 0x41c7
+#define mmCRTC4_CRTC_TEST_DEBUG_DATA 0x43c7
+#define mmCRTC5_CRTC_TEST_DEBUG_DATA 0x45c7
+#define mmDAC_ENABLE 0x16aa
+#define mmDAC_SOURCE_SELECT 0x16ab
+#define mmDAC_CRC_EN 0x16ac
+#define mmDAC_CRC_CONTROL 0x16ad
+#define mmDAC_CRC_SIG_RGB_MASK 0x16ae
+#define mmDAC_CRC_SIG_CONTROL_MASK 0x16af
+#define mmDAC_CRC_SIG_RGB 0x16b0
+#define mmDAC_CRC_SIG_CONTROL 0x16b1
+#define mmDAC_SYNC_TRISTATE_CONTROL 0x16b2
+#define mmDAC_STEREOSYNC_SELECT 0x16b3
+#define mmDAC_AUTODETECT_CONTROL 0x16b4
+#define mmDAC_AUTODETECT_CONTROL2 0x16b5
+#define mmDAC_AUTODETECT_CONTROL3 0x16b6
+#define mmDAC_AUTODETECT_STATUS 0x16b7
+#define mmDAC_AUTODETECT_INT_CONTROL 0x16b8
+#define mmDAC_FORCE_OUTPUT_CNTL 0x16b9
+#define mmDAC_FORCE_DATA 0x16ba
+#define mmDAC_POWERDOWN 0x16bb
+#define mmDAC_CONTROL 0x16bc
+#define mmDAC_COMPARATOR_ENABLE 0x16bd
+#define mmDAC_COMPARATOR_OUTPUT 0x16be
+#define mmDAC_PWR_CNTL 0x16bf
+#define mmDAC_DFT_CONFIG 0x16c0
+#define mmDAC_FIFO_STATUS 0x16c1
+#define mmDAC_TEST_DEBUG_INDEX 0x16c2
+#define mmDAC_TEST_DEBUG_DATA 0x16c3
+#define mmPERFCOUNTER_CNTL 0x170
+#define mmDC_PERFMON0_PERFCOUNTER_CNTL 0x170
+#define mmDC_PERFMON1_PERFCOUNTER_CNTL 0x358
+#define mmDC_PERFMON2_PERFCOUNTER_CNTL 0x364
+#define mmDC_PERFMON3_PERFCOUNTER_CNTL 0x18c8
+#define mmDC_PERFMON4_PERFCOUNTER_CNTL 0x1b24
+#define mmDC_PERFMON5_PERFCOUNTER_CNTL 0x1d24
+#define mmDC_PERFMON6_PERFCOUNTER_CNTL 0x1f24
+#define mmDC_PERFMON7_PERFCOUNTER_CNTL 0x4124
+#define mmDC_PERFMON8_PERFCOUNTER_CNTL 0x4324
+#define mmDC_PERFMON9_PERFCOUNTER_CNTL 0x4524
+#define mmDC_PERFMON10_PERFCOUNTER_CNTL 0x4724
+#define mmDC_PERFMON11_PERFCOUNTER_CNTL 0x59a0
+#define mmDC_PERFMON12_PERFCOUNTER_CNTL 0x5f68
+#define mmDC_PERFMON13_PERFCOUNTER_CNTL 0x9924
+#define mmPERFCOUNTER_STATE 0x171
+#define mmDC_PERFMON0_PERFCOUNTER_STATE 0x171
+#define mmDC_PERFMON1_PERFCOUNTER_STATE 0x359
+#define mmDC_PERFMON2_PERFCOUNTER_STATE 0x365
+#define mmDC_PERFMON3_PERFCOUNTER_STATE 0x18c9
+#define mmDC_PERFMON4_PERFCOUNTER_STATE 0x1b25
+#define mmDC_PERFMON5_PERFCOUNTER_STATE 0x1d25
+#define mmDC_PERFMON6_PERFCOUNTER_STATE 0x1f25
+#define mmDC_PERFMON7_PERFCOUNTER_STATE 0x4125
+#define mmDC_PERFMON8_PERFCOUNTER_STATE 0x4325
+#define mmDC_PERFMON9_PERFCOUNTER_STATE 0x4525
+#define mmDC_PERFMON10_PERFCOUNTER_STATE 0x4725
+#define mmDC_PERFMON11_PERFCOUNTER_STATE 0x59a1
+#define mmDC_PERFMON12_PERFCOUNTER_STATE 0x5f69
+#define mmDC_PERFMON13_PERFCOUNTER_STATE 0x9925
+#define mmPERFMON_CNTL 0x173
+#define mmDC_PERFMON0_PERFMON_CNTL 0x173
+#define mmDC_PERFMON1_PERFMON_CNTL 0x35b
+#define mmDC_PERFMON2_PERFMON_CNTL 0x367
+#define mmDC_PERFMON3_PERFMON_CNTL 0x18cb
+#define mmDC_PERFMON4_PERFMON_CNTL 0x1b27
+#define mmDC_PERFMON5_PERFMON_CNTL 0x1d27
+#define mmDC_PERFMON6_PERFMON_CNTL 0x1f27
+#define mmDC_PERFMON7_PERFMON_CNTL 0x4127
+#define mmDC_PERFMON8_PERFMON_CNTL 0x4327
+#define mmDC_PERFMON9_PERFMON_CNTL 0x4527
+#define mmDC_PERFMON10_PERFMON_CNTL 0x4727
+#define mmDC_PERFMON11_PERFMON_CNTL 0x59a3
+#define mmDC_PERFMON12_PERFMON_CNTL 0x5f6b
+#define mmDC_PERFMON13_PERFMON_CNTL 0x9927
+#define mmPERFMON_CNTL2 0x17a
+#define mmDC_PERFMON0_PERFMON_CNTL2 0x17a
+#define mmDC_PERFMON1_PERFMON_CNTL2 0x362
+#define mmDC_PERFMON2_PERFMON_CNTL2 0x36e
+#define mmDC_PERFMON3_PERFMON_CNTL2 0x18d2
+#define mmDC_PERFMON4_PERFMON_CNTL2 0x1b2e
+#define mmDC_PERFMON5_PERFMON_CNTL2 0x1d2e
+#define mmDC_PERFMON6_PERFMON_CNTL2 0x1f2e
+#define mmDC_PERFMON7_PERFMON_CNTL2 0x412e
+#define mmDC_PERFMON8_PERFMON_CNTL2 0x432e
+#define mmDC_PERFMON9_PERFMON_CNTL2 0x452e
+#define mmDC_PERFMON10_PERFMON_CNTL2 0x472e
+#define mmDC_PERFMON11_PERFMON_CNTL2 0x59aa
+#define mmDC_PERFMON12_PERFMON_CNTL2 0x5f72
+#define mmDC_PERFMON13_PERFMON_CNTL2 0x992e
+#define mmPERFMON_CVALUE_INT_MISC 0x172
+#define mmDC_PERFMON0_PERFMON_CVALUE_INT_MISC 0x172
+#define mmDC_PERFMON1_PERFMON_CVALUE_INT_MISC 0x35a
+#define mmDC_PERFMON2_PERFMON_CVALUE_INT_MISC 0x366
+#define mmDC_PERFMON3_PERFMON_CVALUE_INT_MISC 0x18ca
+#define mmDC_PERFMON4_PERFMON_CVALUE_INT_MISC 0x1b26
+#define mmDC_PERFMON5_PERFMON_CVALUE_INT_MISC 0x1d26
+#define mmDC_PERFMON6_PERFMON_CVALUE_INT_MISC 0x1f26
+#define mmDC_PERFMON7_PERFMON_CVALUE_INT_MISC 0x4126
+#define mmDC_PERFMON8_PERFMON_CVALUE_INT_MISC 0x4326
+#define mmDC_PERFMON9_PERFMON_CVALUE_INT_MISC 0x4526
+#define mmDC_PERFMON10_PERFMON_CVALUE_INT_MISC 0x4726
+#define mmDC_PERFMON11_PERFMON_CVALUE_INT_MISC 0x59a2
+#define mmDC_PERFMON12_PERFMON_CVALUE_INT_MISC 0x5f6a
+#define mmDC_PERFMON13_PERFMON_CVALUE_INT_MISC 0x9926
+#define mmPERFMON_CVALUE_LOW 0x174
+#define mmDC_PERFMON0_PERFMON_CVALUE_LOW 0x174
+#define mmDC_PERFMON1_PERFMON_CVALUE_LOW 0x35c
+#define mmDC_PERFMON2_PERFMON_CVALUE_LOW 0x368
+#define mmDC_PERFMON3_PERFMON_CVALUE_LOW 0x18cc
+#define mmDC_PERFMON4_PERFMON_CVALUE_LOW 0x1b28
+#define mmDC_PERFMON5_PERFMON_CVALUE_LOW 0x1d28
+#define mmDC_PERFMON6_PERFMON_CVALUE_LOW 0x1f28
+#define mmDC_PERFMON7_PERFMON_CVALUE_LOW 0x4128
+#define mmDC_PERFMON8_PERFMON_CVALUE_LOW 0x4328
+#define mmDC_PERFMON9_PERFMON_CVALUE_LOW 0x4528
+#define mmDC_PERFMON10_PERFMON_CVALUE_LOW 0x4728
+#define mmDC_PERFMON11_PERFMON_CVALUE_LOW 0x59a4
+#define mmDC_PERFMON12_PERFMON_CVALUE_LOW 0x5f6c
+#define mmDC_PERFMON13_PERFMON_CVALUE_LOW 0x9928
+#define mmPERFMON_HI 0x175
+#define mmDC_PERFMON0_PERFMON_HI 0x175
+#define mmDC_PERFMON1_PERFMON_HI 0x35d
+#define mmDC_PERFMON2_PERFMON_HI 0x369
+#define mmDC_PERFMON3_PERFMON_HI 0x18cd
+#define mmDC_PERFMON4_PERFMON_HI 0x1b29
+#define mmDC_PERFMON5_PERFMON_HI 0x1d29
+#define mmDC_PERFMON6_PERFMON_HI 0x1f29
+#define mmDC_PERFMON7_PERFMON_HI 0x4129
+#define mmDC_PERFMON8_PERFMON_HI 0x4329
+#define mmDC_PERFMON9_PERFMON_HI 0x4529
+#define mmDC_PERFMON10_PERFMON_HI 0x4729
+#define mmDC_PERFMON11_PERFMON_HI 0x59a5
+#define mmDC_PERFMON12_PERFMON_HI 0x5f6d
+#define mmDC_PERFMON13_PERFMON_HI 0x9929
+#define mmPERFMON_LOW 0x176
+#define mmDC_PERFMON0_PERFMON_LOW 0x176
+#define mmDC_PERFMON1_PERFMON_LOW 0x35e
+#define mmDC_PERFMON2_PERFMON_LOW 0x36a
+#define mmDC_PERFMON3_PERFMON_LOW 0x18ce
+#define mmDC_PERFMON4_PERFMON_LOW 0x1b2a
+#define mmDC_PERFMON5_PERFMON_LOW 0x1d2a
+#define mmDC_PERFMON6_PERFMON_LOW 0x1f2a
+#define mmDC_PERFMON7_PERFMON_LOW 0x412a
+#define mmDC_PERFMON8_PERFMON_LOW 0x432a
+#define mmDC_PERFMON9_PERFMON_LOW 0x452a
+#define mmDC_PERFMON10_PERFMON_LOW 0x472a
+#define mmDC_PERFMON11_PERFMON_LOW 0x59a6
+#define mmDC_PERFMON12_PERFMON_LOW 0x5f6e
+#define mmDC_PERFMON13_PERFMON_LOW 0x992a
+#define mmPERFMON_TEST_DEBUG_INDEX 0x177
+#define mmDC_PERFMON0_PERFMON_TEST_DEBUG_INDEX 0x177
+#define mmDC_PERFMON1_PERFMON_TEST_DEBUG_INDEX 0x35f
+#define mmDC_PERFMON2_PERFMON_TEST_DEBUG_INDEX 0x36b
+#define mmDC_PERFMON3_PERFMON_TEST_DEBUG_INDEX 0x18cf
+#define mmDC_PERFMON4_PERFMON_TEST_DEBUG_INDEX 0x1b2b
+#define mmDC_PERFMON5_PERFMON_TEST_DEBUG_INDEX 0x1d2b
+#define mmDC_PERFMON6_PERFMON_TEST_DEBUG_INDEX 0x1f2b
+#define mmDC_PERFMON7_PERFMON_TEST_DEBUG_INDEX 0x412b
+#define mmDC_PERFMON8_PERFMON_TEST_DEBUG_INDEX 0x432b
+#define mmDC_PERFMON9_PERFMON_TEST_DEBUG_INDEX 0x452b
+#define mmDC_PERFMON10_PERFMON_TEST_DEBUG_INDEX 0x472b
+#define mmDC_PERFMON11_PERFMON_TEST_DEBUG_INDEX 0x59a7
+#define mmDC_PERFMON12_PERFMON_TEST_DEBUG_INDEX 0x5f6f
+#define mmDC_PERFMON13_PERFMON_TEST_DEBUG_INDEX 0x992b
+#define mmPERFMON_TEST_DEBUG_DATA 0x178
+#define mmDC_PERFMON0_PERFMON_TEST_DEBUG_DATA 0x178
+#define mmDC_PERFMON1_PERFMON_TEST_DEBUG_DATA 0x360
+#define mmDC_PERFMON2_PERFMON_TEST_DEBUG_DATA 0x36c
+#define mmDC_PERFMON3_PERFMON_TEST_DEBUG_DATA 0x18d0
+#define mmDC_PERFMON4_PERFMON_TEST_DEBUG_DATA 0x1b2c
+#define mmDC_PERFMON5_PERFMON_TEST_DEBUG_DATA 0x1d2c
+#define mmDC_PERFMON6_PERFMON_TEST_DEBUG_DATA 0x1f2c
+#define mmDC_PERFMON7_PERFMON_TEST_DEBUG_DATA 0x412c
+#define mmDC_PERFMON8_PERFMON_TEST_DEBUG_DATA 0x432c
+#define mmDC_PERFMON9_PERFMON_TEST_DEBUG_DATA 0x452c
+#define mmDC_PERFMON10_PERFMON_TEST_DEBUG_DATA 0x472c
+#define mmDC_PERFMON11_PERFMON_TEST_DEBUG_DATA 0x59a8
+#define mmDC_PERFMON12_PERFMON_TEST_DEBUG_DATA 0x5f70
+#define mmDC_PERFMON13_PERFMON_TEST_DEBUG_DATA 0x992c
+#define mmREFCLK_CNTL 0x109
+#define mmDCCG_CBUS_ANTIGLITCH_RESETB 0x15c
+#define mmDCCG_CBUS_SPARE 0x15d
+#define mmDCCG_CBUS_WRCMD_DELAY 0x110
+#define mmDPREFCLK_CNTL 0x118
+#define mmDCE_VERSION 0x11e
+#define mmAVSYNC_COUNTER_WRITE 0x12a
+#define mmAVSYNC_COUNTER_CONTROL 0x12b
+#define mmAVSYNC_COUNTER_READ 0x12f
+#define mmDCCG_GTC_CNTL 0x120
+#define mmDCCG_GTC_DTO_INCR 0x121
+#define mmDCCG_GTC_DTO_MODULO 0x122
+#define mmDCCG_GTC_CURRENT 0x123
+#define mmDCCG_DS_DTO_INCR 0x113
+#define mmDCCG_DS_DTO_MODULO 0x114
+#define mmDCCG_DS_CNTL 0x115
+#define mmDCCG_DS_HW_CAL_INTERVAL 0x116
+#define mmDCCG_DS_DEBUG_CNTL 0x112
+#define mmDMCU_SMU_INTERRUPT_CNTL 0x12c
+#define mmSMU_CONTROL 0x12d
+#define mmSMU_INTERRUPT_CONTROL 0x12e
+#define mmDAC_CLK_ENABLE 0x128
+#define mmDVO_CLK_ENABLE 0x129
+#define mmDCCG_GATE_DISABLE_CNTL 0x134
+#define mmDCCG_GATE_DISABLE_CNTL2 0x13c
+#define mmDISPCLK_CGTT_BLK_CTRL_REG 0x135
+#define mmSCLK_CGTT_BLK_CTRL_REG 0x136
+#define mmDPREFCLK_CGTT_BLK_CTRL_REG 0x108
+#define mmREFCLK_CGTT_BLK_CTRL_REG 0x10b
+#define mmSYMCLK_CGTT_BLK_CTRL_REG 0x13d
+#define mmDCCG_CAC_STATUS 0x137
+#define mmPIXCLK0_RESYNC_CNTL 0x13a
+#define mmPHYPLLA_PIXCLK_RESYNC_CNTL 0x100
+#define mmPHYPLLB_PIXCLK_RESYNC_CNTL 0x101
+#define mmPHYPLLC_PIXCLK_RESYNC_CNTL 0x102
+#define mmPHYPLLD_PIXCLK_RESYNC_CNTL 0x103
+#define mmPHYPLLE_PIXCLK_RESYNC_CNTL 0x10c
+#define mmPHYPLLF_PIXCLK_RESYNC_CNTL 0x13e
+#define mmMICROSECOND_TIME_BASE_DIV 0x13b
+#define mmDCCG_DISP_CNTL_REG 0x13f
+#define mmMILLISECOND_TIME_BASE_DIV 0x130
+#define mmDISPCLK_FREQ_CHANGE_CNTL 0x131
+#define mmDC_MEM_GLOBAL_PWR_REQ_CNTL 0x132
+#define mmDCCG_PERFMON_CNTL 0x133
+#define mmDCCG_PERFMON_CNTL2 0x10e
+#define mmCRTC0_PIXEL_RATE_CNTL 0x140
+#define mmDP_DTO0_PHASE 0x141
+#define mmDP_DTO0_MODULO 0x142
+#define mmCRTC0_PHYPLL_PIXEL_RATE_CNTL 0x143
+#define mmCRTC1_PIXEL_RATE_CNTL 0x144
+#define mmDP_DTO1_PHASE 0x145
+#define mmDP_DTO1_MODULO 0x146
+#define mmCRTC1_PHYPLL_PIXEL_RATE_CNTL 0x147
+#define mmCRTC2_PIXEL_RATE_CNTL 0x148
+#define mmDP_DTO2_PHASE 0x149
+#define mmDP_DTO2_MODULO 0x14a
+#define mmCRTC2_PHYPLL_PIXEL_RATE_CNTL 0x14b
+#define mmCRTC3_PIXEL_RATE_CNTL 0x14c
+#define mmDP_DTO3_PHASE 0x14d
+#define mmDP_DTO3_MODULO 0x14e
+#define mmCRTC3_PHYPLL_PIXEL_RATE_CNTL 0x14f
+#define mmCRTC4_PIXEL_RATE_CNTL 0x150
+#define mmDP_DTO4_PHASE 0x151
+#define mmDP_DTO4_MODULO 0x152
+#define mmCRTC4_PHYPLL_PIXEL_RATE_CNTL 0x153
+#define mmCRTC5_PIXEL_RATE_CNTL 0x154
+#define mmDP_DTO5_PHASE 0x155
+#define mmDP_DTO5_MODULO 0x156
+#define mmCRTC5_PHYPLL_PIXEL_RATE_CNTL 0x157
+#define mmDCCG_SOFT_RESET 0x15f
+#define mmSYMCLKA_CLOCK_ENABLE 0x160
+#define mmSYMCLKB_CLOCK_ENABLE 0x161
+#define mmSYMCLKC_CLOCK_ENABLE 0x162
+#define mmSYMCLKD_CLOCK_ENABLE 0x163
+#define mmSYMCLKE_CLOCK_ENABLE 0x164
+#define mmSYMCLKF_CLOCK_ENABLE 0x165
+#define mmDPDBG_CLK_FORCE_CONTROL 0x10d
+#define mmDCCG_AUDIO_DTO_SOURCE 0x16b
+#define mmDCCG_AUDIO_DTO0_PHASE 0x16c
+#define mmDCCG_AUDIO_DTO0_MODULE 0x16d
+#define mmDCCG_AUDIO_DTO1_PHASE 0x16e
+#define mmDCCG_AUDIO_DTO1_MODULE 0x16f
+#define mmDCCG_TEST_DEBUG_INDEX 0x17c
+#define mmDCCG_TEST_DEBUG_DATA 0x17d
+#define mmDCCG_TEST_CLK_SEL 0x17e
+#define mmCPLL_MACRO_CNTL_RESERVED0 0x5fd0
+#define mmDCCG_CPLL0_CPLL_MACRO_CNTL_RESERVED0 0x5fd0
+#define mmDCCG_CPLL1_CPLL_MACRO_CNTL_RESERVED0 0x5fdc
+#define mmDCCG_CPLL2_CPLL_MACRO_CNTL_RESERVED0 0x5fe8
+#define mmDCCG_CPLL3_CPLL_MACRO_CNTL_RESERVED0 0x5ff4
+#define mmCPLL_MACRO_CNTL_RESERVED1 0x5fd1
+#define mmDCCG_CPLL0_CPLL_MACRO_CNTL_RESERVED1 0x5fd1
+#define mmDCCG_CPLL1_CPLL_MACRO_CNTL_RESERVED1 0x5fdd
+#define mmDCCG_CPLL2_CPLL_MACRO_CNTL_RESERVED1 0x5fe9
+#define mmDCCG_CPLL3_CPLL_MACRO_CNTL_RESERVED1 0x5ff5
+#define mmCPLL_MACRO_CNTL_RESERVED2 0x5fd2
+#define mmDCCG_CPLL0_CPLL_MACRO_CNTL_RESERVED2 0x5fd2
+#define mmDCCG_CPLL1_CPLL_MACRO_CNTL_RESERVED2 0x5fde
+#define mmDCCG_CPLL2_CPLL_MACRO_CNTL_RESERVED2 0x5fea
+#define mmDCCG_CPLL3_CPLL_MACRO_CNTL_RESERVED2 0x5ff6
+#define mmCPLL_MACRO_CNTL_RESERVED3 0x5fd3
+#define mmDCCG_CPLL0_CPLL_MACRO_CNTL_RESERVED3 0x5fd3
+#define mmDCCG_CPLL1_CPLL_MACRO_CNTL_RESERVED3 0x5fdf
+#define mmDCCG_CPLL2_CPLL_MACRO_CNTL_RESERVED3 0x5feb
+#define mmDCCG_CPLL3_CPLL_MACRO_CNTL_RESERVED3 0x5ff7
+#define mmCPLL_MACRO_CNTL_RESERVED4 0x5fd4
+#define mmDCCG_CPLL0_CPLL_MACRO_CNTL_RESERVED4 0x5fd4
+#define mmDCCG_CPLL1_CPLL_MACRO_CNTL_RESERVED4 0x5fe0
+#define mmDCCG_CPLL2_CPLL_MACRO_CNTL_RESERVED4 0x5fec
+#define mmDCCG_CPLL3_CPLL_MACRO_CNTL_RESERVED4 0x5ff8
+#define mmCPLL_MACRO_CNTL_RESERVED5 0x5fd5
+#define mmDCCG_CPLL0_CPLL_MACRO_CNTL_RESERVED5 0x5fd5
+#define mmDCCG_CPLL1_CPLL_MACRO_CNTL_RESERVED5 0x5fe1
+#define mmDCCG_CPLL2_CPLL_MACRO_CNTL_RESERVED5 0x5fed
+#define mmDCCG_CPLL3_CPLL_MACRO_CNTL_RESERVED5 0x5ff9
+#define mmCPLL_MACRO_CNTL_RESERVED6 0x5fd6
+#define mmDCCG_CPLL0_CPLL_MACRO_CNTL_RESERVED6 0x5fd6
+#define mmDCCG_CPLL1_CPLL_MACRO_CNTL_RESERVED6 0x5fe2
+#define mmDCCG_CPLL2_CPLL_MACRO_CNTL_RESERVED6 0x5fee
+#define mmDCCG_CPLL3_CPLL_MACRO_CNTL_RESERVED6 0x5ffa
+#define mmCPLL_MACRO_CNTL_RESERVED7 0x5fd7
+#define mmDCCG_CPLL0_CPLL_MACRO_CNTL_RESERVED7 0x5fd7
+#define mmDCCG_CPLL1_CPLL_MACRO_CNTL_RESERVED7 0x5fe3
+#define mmDCCG_CPLL2_CPLL_MACRO_CNTL_RESERVED7 0x5fef
+#define mmDCCG_CPLL3_CPLL_MACRO_CNTL_RESERVED7 0x5ffb
+#define mmCPLL_MACRO_CNTL_RESERVED8 0x5fd8
+#define mmDCCG_CPLL0_CPLL_MACRO_CNTL_RESERVED8 0x5fd8
+#define mmDCCG_CPLL1_CPLL_MACRO_CNTL_RESERVED8 0x5fe4
+#define mmDCCG_CPLL2_CPLL_MACRO_CNTL_RESERVED8 0x5ff0
+#define mmDCCG_CPLL3_CPLL_MACRO_CNTL_RESERVED8 0x5ffc
+#define mmCPLL_MACRO_CNTL_RESERVED9 0x5fd9
+#define mmDCCG_CPLL0_CPLL_MACRO_CNTL_RESERVED9 0x5fd9
+#define mmDCCG_CPLL1_CPLL_MACRO_CNTL_RESERVED9 0x5fe5
+#define mmDCCG_CPLL2_CPLL_MACRO_CNTL_RESERVED9 0x5ff1
+#define mmDCCG_CPLL3_CPLL_MACRO_CNTL_RESERVED9 0x5ffd
+#define mmCPLL_MACRO_CNTL_RESERVED10 0x5fda
+#define mmDCCG_CPLL0_CPLL_MACRO_CNTL_RESERVED10 0x5fda
+#define mmDCCG_CPLL1_CPLL_MACRO_CNTL_RESERVED10 0x5fe6
+#define mmDCCG_CPLL2_CPLL_MACRO_CNTL_RESERVED10 0x5ff2
+#define mmDCCG_CPLL3_CPLL_MACRO_CNTL_RESERVED10 0x5ffe
+#define mmCPLL_MACRO_CNTL_RESERVED11 0x5fdb
+#define mmDCCG_CPLL0_CPLL_MACRO_CNTL_RESERVED11 0x5fdb
+#define mmDCCG_CPLL1_CPLL_MACRO_CNTL_RESERVED11 0x5fe7
+#define mmDCCG_CPLL2_CPLL_MACRO_CNTL_RESERVED11 0x5ff3
+#define mmDCCG_CPLL3_CPLL_MACRO_CNTL_RESERVED11 0x5fff
+#define mmPLL_MACRO_CNTL_RESERVED0 0x1700
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED0 0x1700
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED0 0x172a
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED0 0x1754
+#define mmPLL_MACRO_CNTL_RESERVED1 0x1701
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED1 0x1701
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED1 0x172b
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED1 0x1755
+#define mmPLL_MACRO_CNTL_RESERVED2 0x1702
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED2 0x1702
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED2 0x172c
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED2 0x1756
+#define mmPLL_MACRO_CNTL_RESERVED3 0x1703
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED3 0x1703
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED3 0x172d
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED3 0x1757
+#define mmPLL_MACRO_CNTL_RESERVED4 0x1704
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED4 0x1704
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED4 0x172e
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED4 0x1758
+#define mmPLL_MACRO_CNTL_RESERVED5 0x1705
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED5 0x1705
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED5 0x172f
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED5 0x1759
+#define mmPLL_MACRO_CNTL_RESERVED6 0x1706
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED6 0x1706
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED6 0x1730
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED6 0x175a
+#define mmPLL_MACRO_CNTL_RESERVED7 0x1707
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED7 0x1707
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED7 0x1731
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED7 0x175b
+#define mmPLL_MACRO_CNTL_RESERVED8 0x1708
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED8 0x1708
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED8 0x1732
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED8 0x175c
+#define mmPLL_MACRO_CNTL_RESERVED9 0x1709
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED9 0x1709
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED9 0x1733
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED9 0x175d
+#define mmPLL_MACRO_CNTL_RESERVED10 0x170a
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED10 0x170a
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED10 0x1734
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED10 0x175e
+#define mmPLL_MACRO_CNTL_RESERVED11 0x170b
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED11 0x170b
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED11 0x1735
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED11 0x175f
+#define mmPLL_MACRO_CNTL_RESERVED12 0x170c
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED12 0x170c
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED12 0x1736
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED12 0x1760
+#define mmPLL_MACRO_CNTL_RESERVED13 0x170d
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED13 0x170d
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED13 0x1737
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED13 0x1761
+#define mmPLL_MACRO_CNTL_RESERVED14 0x170e
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED14 0x170e
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED14 0x1738
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED14 0x1762
+#define mmPLL_MACRO_CNTL_RESERVED15 0x170f
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED15 0x170f
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED15 0x1739
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED15 0x1763
+#define mmPLL_MACRO_CNTL_RESERVED16 0x1710
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED16 0x1710
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED16 0x173a
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED16 0x1764
+#define mmPLL_MACRO_CNTL_RESERVED17 0x1711
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED17 0x1711
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED17 0x173b
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED17 0x1765
+#define mmPLL_MACRO_CNTL_RESERVED18 0x1712
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED18 0x1712
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED18 0x173c
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED18 0x1766
+#define mmPLL_MACRO_CNTL_RESERVED19 0x1713
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED19 0x1713
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED19 0x173d
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED19 0x1767
+#define mmPLL_MACRO_CNTL_RESERVED20 0x1714
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED20 0x1714
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED20 0x173e
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED20 0x1768
+#define mmPLL_MACRO_CNTL_RESERVED21 0x1715
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED21 0x1715
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED21 0x173f
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED21 0x1769
+#define mmPLL_MACRO_CNTL_RESERVED22 0x1716
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED22 0x1716
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED22 0x1740
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED22 0x176a
+#define mmPLL_MACRO_CNTL_RESERVED23 0x1717
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED23 0x1717
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED23 0x1741
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED23 0x176b
+#define mmPLL_MACRO_CNTL_RESERVED24 0x1718
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED24 0x1718
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED24 0x1742
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED24 0x176c
+#define mmPLL_MACRO_CNTL_RESERVED25 0x1719
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED25 0x1719
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED25 0x1743
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED25 0x176d
+#define mmPLL_MACRO_CNTL_RESERVED26 0x171a
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED26 0x171a
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED26 0x1744
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED26 0x176e
+#define mmPLL_MACRO_CNTL_RESERVED27 0x171b
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED27 0x171b
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED27 0x1745
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED27 0x176f
+#define mmPLL_MACRO_CNTL_RESERVED28 0x171c
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED28 0x171c
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED28 0x1746
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED28 0x1770
+#define mmPLL_MACRO_CNTL_RESERVED29 0x171d
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED29 0x171d
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED29 0x1747
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED29 0x1771
+#define mmPLL_MACRO_CNTL_RESERVED30 0x171e
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED30 0x171e
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED30 0x1748
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED30 0x1772
+#define mmPLL_MACRO_CNTL_RESERVED31 0x171f
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED31 0x171f
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED31 0x1749
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED31 0x1773
+#define mmPLL_MACRO_CNTL_RESERVED32 0x1720
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED32 0x1720
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED32 0x174a
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED32 0x1774
+#define mmPLL_MACRO_CNTL_RESERVED33 0x1721
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED33 0x1721
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED33 0x174b
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED33 0x1775
+#define mmPLL_MACRO_CNTL_RESERVED34 0x1722
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED34 0x1722
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED34 0x174c
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED34 0x1776
+#define mmPLL_MACRO_CNTL_RESERVED35 0x1723
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED35 0x1723
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED35 0x174d
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED35 0x1777
+#define mmPLL_MACRO_CNTL_RESERVED36 0x1724
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED36 0x1724
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED36 0x174e
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED36 0x1778
+#define mmPLL_MACRO_CNTL_RESERVED37 0x1725
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED37 0x1725
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED37 0x174f
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED37 0x1779
+#define mmPLL_MACRO_CNTL_RESERVED38 0x1726
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED38 0x1726
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED38 0x1750
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED38 0x177a
+#define mmPLL_MACRO_CNTL_RESERVED39 0x1727
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED39 0x1727
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED39 0x1751
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED39 0x177b
+#define mmPLL_MACRO_CNTL_RESERVED40 0x1728
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED40 0x1728
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED40 0x1752
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED40 0x177c
+#define mmPLL_MACRO_CNTL_RESERVED41 0x1729
+#define mmDCCG_PLL0_PLL_MACRO_CNTL_RESERVED41 0x1729
+#define mmDCCG_PLL1_PLL_MACRO_CNTL_RESERVED41 0x1753
+#define mmDCCG_PLL2_PLL_MACRO_CNTL_RESERVED41 0x177d
+#define mmDENTIST_DISPCLK_CNTL 0x124
+#define mmDCDEBUG_BUS_CLK1_SEL 0x16c4
+#define mmDCDEBUG_BUS_CLK2_SEL 0x16c5
+#define mmDCDEBUG_BUS_CLK3_SEL 0x16c6
+#define mmDCDEBUG_BUS_CLK4_SEL 0x16c7
+#define mmDCDEBUG_BUS_CLK5_SEL 0x16c8
+#define mmDCDEBUG_OUT_PIN_OVERRIDE 0x16c9
+#define mmDCDEBUG_OUT_CNTL 0x16ca
+#define mmDCDEBUG_OUT_DATA 0x16cb
+#define mmDMIF_CONTROL 0x2f6
+#define mmDMIF_STATUS 0x2f7
+#define mmDMIFV_STATUS 0x2f5
+#define mmDMIF_HW_DEBUG 0x2f8
+#define mmDMIF_ARBITRATION_CONTROL 0x2f9
+#define mmPIPE0_ARBITRATION_CONTROL3 0x2fa
+#define mmPIPE1_ARBITRATION_CONTROL3 0x2fb
+#define mmPIPE2_ARBITRATION_CONTROL3 0x2fc
+#define mmPIPE3_ARBITRATION_CONTROL3 0x2fd
+#define mmPIPE4_ARBITRATION_CONTROL3 0x2fe
+#define mmPIPE5_ARBITRATION_CONTROL3 0x2ff
+#define mmPIPE6_ARBITRATION_CONTROL3 0x32a
+#define mmPIPE7_ARBITRATION_CONTROL3 0x32b
+#define mmDMIF_P_VMID 0x300
+#define mmDMIF_URG_OVERRIDE 0x329
+#define mmDMIF_TEST_DEBUG_INDEX 0x301
+#define mmDMIF_TEST_DEBUG_DATA 0x302
+#define ixDMIF_DEBUG02_CORE0 0x2
+#define ixDMIF_DEBUG02_CORE1 0xa
+#define mmDMIF_ADDR_CALC 0x303
+#define mmDMIF_STATUS2 0x304
+#define mmPIPE0_MAX_REQUESTS 0x305
+#define mmPIPE1_MAX_REQUESTS 0x306
+#define mmPIPE2_MAX_REQUESTS 0x307
+#define mmPIPE3_MAX_REQUESTS 0x308
+#define mmPIPE4_MAX_REQUESTS 0x309
+#define mmPIPE5_MAX_REQUESTS 0x30a
+#define mmPIPE6_MAX_REQUESTS 0x32c
+#define mmPIPE7_MAX_REQUESTS 0x32d
+#define mmDVMM_REG_RD_STATUS 0x32e
+#define mmDVMM_REG_RD_DATA 0x32f
+#define mmDVMM_PTE_REQ 0x330
+#define mmDVMM_CNTL 0x331
+#define mmDVMM_FAULT_STATUS 0x332
+#define mmDVMM_FAULT_ADDR 0x333
+#define mmLOW_POWER_TILING_CONTROL 0x30b
+#define mmMCIF_CONTROL 0x30c
+#define mmMCIF_WRITE_COMBINE_CONTROL 0x30d
+#define mmMCIF_TEST_DEBUG_INDEX 0x30e
+#define mmMCIF_TEST_DEBUG_DATA 0x30f
+#define ixIDDCCIF02_DBG_DCCIF_C 0x9
+#define ixIDDCCIF04_DBG_DCCIF_E 0xb
+#define ixIDDCCIF05_DBG_DCCIF_F 0xc
+#define mmMCIF_VMID 0x310
+#define mmMCIF_MEM_CONTROL 0x311
+#define mmCC_DC_PIPE_DIS 0x312
+#define mmMC_DC_INTERFACE_NACK_STATUS 0x313
+#define mmRBBMIF_TIMEOUT 0x314
+#define mmRBBMIF_STATUS 0x315
+#define mmRBBMIF_TIMEOUT_DIS 0x316
+#define mmRBBMIF_STATUS_FLAG 0x327
+#define mmDCI_MEM_PWR_STATUS 0x317
+#define mmDCI_MEM_PWR_STATUS2 0x318
+#define mmDCI_MEM_PWR_STATUS3 0x33d
+#define mmDCI_CLK_CNTL 0x319
+#define mmDCI_CLK_RAMP_CNTL 0x31a
+#define mmDCI_MEM_PWR_CNTL 0x31b
+#define mmDCI_MEM_PWR_CNTL2 0x31c
+#define mmDCI_MEM_PWR_CNTL3 0x31d
+#define mmDCI_MEM_PWR_CNTL4 0x33b
+#define mmDVMM_PTE_PGMEM_CONTROL 0x335
+#define mmDVMM_PTE_PGMEM_STATE 0x336
+#define mmDCI_SOFT_RESET 0x328
+#define mmDCI_MISC 0x33c
+#define mmDCI_TEST_DEBUG_INDEX 0x31e
+#define mmDCI_TEST_DEBUG_DATA 0x31f
+#define mmDCI_DEBUG_CONFIG 0x320
+#define mmPIPE0_DMIF_BUFFER_CONTROL 0x321
+#define mmPIPE1_DMIF_BUFFER_CONTROL 0x322
+#define mmPIPE2_DMIF_BUFFER_CONTROL 0x323
+#define mmPIPE3_DMIF_BUFFER_CONTROL 0x324
+#define mmPIPE4_DMIF_BUFFER_CONTROL 0x325
+#define mmPIPE5_DMIF_BUFFER_CONTROL 0x326
+#define mmDC_GENERICA 0x4800
+#define mmDC_GENERICB 0x4801
+#define mmDC_PAD_EXTERN_SIG 0x4802
+#define mmDC_REF_CLK_CNTL 0x4803
+#define mmDC_GPIO_DEBUG 0x4804
+#define mmUNIPHYA_LINK_CNTL 0x4805
+#define mmUNIPHYB_LINK_CNTL 0x4807
+#define mmUNIPHYC_LINK_CNTL 0x4809
+#define mmUNIPHYD_LINK_CNTL 0x480b
+#define mmUNIPHYE_LINK_CNTL 0x480d
+#define mmUNIPHYF_LINK_CNTL 0x480f
+#define mmUNIPHYG_LINK_CNTL 0x4811
+#define mmUNIPHYA_CHANNEL_XBAR_CNTL 0x4806
+#define mmUNIPHYB_CHANNEL_XBAR_CNTL 0x4808
+#define mmUNIPHYC_CHANNEL_XBAR_CNTL 0x480a
+#define mmUNIPHYD_CHANNEL_XBAR_CNTL 0x480c
+#define mmUNIPHYE_CHANNEL_XBAR_CNTL 0x480e
+#define mmUNIPHYF_CHANNEL_XBAR_CNTL 0x4810
+#define mmUNIPHYG_CHANNEL_XBAR_CNTL 0x4812
+#define mmUNIPHYLPA_LINK_CNTL 0x4847
+#define mmUNIPHYLPB_LINK_CNTL 0x4848
+#define mmUNIPHYLPA_CHANNEL_XBAR_CNTL 0x4849
+#define mmUNIPHYLPB_CHANNEL_XBAR_CNTL 0x484a
+#define mmUNIPHY_IMPCAL_LINKA 0x4838
+#define mmUNIPHY_IMPCAL_LINKB 0x4839
+#define mmUNIPHY_IMPCAL_LINKC 0x483f
+#define mmUNIPHY_IMPCAL_LINKD 0x4840
+#define mmUNIPHY_IMPCAL_LINKE 0x4843
+#define mmUNIPHY_IMPCAL_LINKF 0x4844
+#define mmUNIPHY_IMPCAL_PERIOD 0x483a
+#define mmAUXP_IMPCAL 0x483b
+#define mmAUXN_IMPCAL 0x483c
+#define mmDCIO_IMPCAL_CNTL 0x483d
+#define mmUNIPHY_IMPCAL_PSW_AB 0x483e
+#define mmDCIO_IMPCAL_CNTL_CD 0x4841
+#define mmUNIPHY_IMPCAL_PSW_CD 0x4842
+#define mmDCIO_IMPCAL_CNTL_EF 0x4845
+#define mmUNIPHY_IMPCAL_PSW_EF 0x4846
+#define mmDCIO_WRCMD_DELAY 0x4816
+#define mmDC_PINSTRAPS 0x4818
+#define mmDC_DVODATA_CONFIG 0x481a
+#define mmLVTMA_PWRSEQ_CNTL 0x481b
+#define mmLVTMA_PWRSEQ_STATE 0x481c
+#define mmLVTMA_PWRSEQ_REF_DIV 0x481d
+#define mmLVTMA_PWRSEQ_DELAY1 0x481e
+#define mmLVTMA_PWRSEQ_DELAY2 0x481f
+#define mmBL_PWM_CNTL 0x4820
+#define mmBL_PWM_CNTL2 0x4821
+#define mmBL_PWM_PERIOD_CNTL 0x4822
+#define mmBL_PWM_GRP1_REG_LOCK 0x4823
+#define mmDCIO_GSL_GENLK_PAD_CNTL 0x4824
+#define mmDCIO_GSL_SWAPLOCK_PAD_CNTL 0x4825
+#define mmDCIO_GSL0_CNTL 0x4826
+#define mmDCIO_GSL1_CNTL 0x4827
+#define mmDCIO_GSL2_CNTL 0x4828
+#define mmDC_GPU_TIMER_START_POSITION_V_UPDATE 0x4829
+#define mmDC_GPU_TIMER_START_POSITION_P_FLIP 0x482a
+#define mmDC_GPU_TIMER_READ 0x482b
+#define mmDC_GPU_TIMER_READ_CNTL 0x482c
+#define mmDCIO_CLOCK_CNTL 0x482d
+#define mmDCIO_DEBUG 0x482f
+#define mmDCO_DCFE_EXT_VSYNC_CNTL 0x4830
+#define mmDBG_OUT_CNTL 0x4834
+#define mmDCIO_DEBUG_CONFIG 0x4835
+#define mmDCIO_SOFT_RESET 0x4836
+#define mmDCIO_DPHY_SEL 0x4837
+#define mmDCIO_DPCS_TX_INTERRUPT 0x484b
+#define mmDCIO_DPCS_RX_INTERRUPT 0x484c
+#define mmDCIO_SEMAPHORE0 0x484d
+#define mmDCIO_SEMAPHORE1 0x484e
+#define mmDCIO_SEMAPHORE2 0x484f
+#define mmDCIO_SEMAPHORE3 0x4850
+#define mmDCIO_SEMAPHORE4 0x4851
+#define mmDCIO_SEMAPHORE5 0x4852
+#define mmDCIO_SEMAPHORE6 0x4853
+#define mmDCIO_SEMAPHORE7 0x4854
+#define mmDCIO_TEST_DEBUG_INDEX 0x4831
+#define mmDCIO_TEST_DEBUG_DATA 0x4832
+#define ixDCIO_DEBUG1 0x1
+#define ixDCIO_DEBUG2 0x2
+#define ixDCIO_DEBUG3 0x3
+#define ixDCIO_DEBUG4 0x4
+#define ixDCIO_DEBUG5 0x5
+#define ixDCIO_DEBUG6 0x6
+#define ixDCIO_DEBUG7 0x7
+#define ixDCIO_DEBUG8 0x8
+#define ixDCIO_DEBUG9 0x9
+#define ixDCIO_DEBUGA 0xa
+#define ixDCIO_DEBUGB 0xb
+#define ixDCIO_DEBUGC 0xc
+#define ixDCIO_DEBUGD 0xd
+#define ixDCIO_DEBUGE 0xe
+#define ixDCIO_DEBUGF 0xf
+#define ixDCIO_DEBUG10 0x10
+#define ixDCIO_DEBUG11 0x11
+#define ixDCIO_DEBUG12 0x12
+#define ixDCIO_DEBUG13 0x13
+#define ixDCIO_DEBUG14 0x14
+#define ixDCIO_DEBUG15 0x15
+#define ixDCIO_DEBUG16 0x16
+#define ixDCIO_DEBUG17 0x17
+#define ixDCIO_DEBUG18 0x18
+#define ixDCIO_DEBUG19 0x19
+#define ixDCIO_DEBUG1A 0x1a
+#define ixDCIO_DEBUG1B 0x1b
+#define ixDCIO_DEBUG1C 0x1c
+#define ixDCIO_DEBUG1D 0x1d
+#define ixDCIO_DEBUG1E 0x1e
+#define ixDCIO_DEBUG1F 0x1f
+#define ixDCIO_DEBUG20 0x20
+#define ixDCIO_DEBUG21 0x21
+#define ixDCIO_DEBUG22 0x22
+#define ixDCIO_DEBUG23 0x23
+#define ixDCIO_DEBUG24 0x24
+#define ixDCIO_DEBUG25 0x25
+#define ixDCIO_DEBUG26 0x26
+#define ixDCIO_DEBUG27 0x27
+#define ixDCIO_DEBUG28 0x28
+#define ixDCIO_DEBUG_ID 0x0
+#define mmDC_GPIO_GENERIC_MASK 0x4860
+#define mmDC_GPIO_GENERIC_A 0x4861
+#define mmDC_GPIO_GENERIC_EN 0x4862
+#define mmDC_GPIO_GENERIC_Y 0x4863
+#define mmDC_GPIO_DDC1_MASK 0x4868
+#define mmDC_GPIO_DDC1_A 0x4869
+#define mmDC_GPIO_DDC1_EN 0x486a
+#define mmDC_GPIO_DDC1_Y 0x486b
+#define mmDC_GPIO_DDC2_MASK 0x486c
+#define mmDC_GPIO_DDC2_A 0x486d
+#define mmDC_GPIO_DDC2_EN 0x486e
+#define mmDC_GPIO_DDC2_Y 0x486f
+#define mmDC_GPIO_DDC3_MASK 0x4870
+#define mmDC_GPIO_DDC3_A 0x4871
+#define mmDC_GPIO_DDC3_EN 0x4872
+#define mmDC_GPIO_DDC3_Y 0x4873
+#define mmDC_GPIO_DDC4_MASK 0x4874
+#define mmDC_GPIO_DDC4_A 0x4875
+#define mmDC_GPIO_DDC4_EN 0x4876
+#define mmDC_GPIO_DDC4_Y 0x4877
+#define mmDC_GPIO_DDC5_MASK 0x4878
+#define mmDC_GPIO_DDC5_A 0x4879
+#define mmDC_GPIO_DDC5_EN 0x487a
+#define mmDC_GPIO_DDC5_Y 0x487b
+#define mmDC_GPIO_DDC6_MASK 0x487c
+#define mmDC_GPIO_DDC6_A 0x487d
+#define mmDC_GPIO_DDC6_EN 0x487e
+#define mmDC_GPIO_DDC6_Y 0x487f
+#define mmDC_GPIO_DDCVGA_MASK 0x4880
+#define mmDC_GPIO_DDCVGA_A 0x4881
+#define mmDC_GPIO_DDCVGA_EN 0x4882
+#define mmDC_GPIO_DDCVGA_Y 0x4883
+#define mmDC_GPIO_SYNCA_MASK 0x4884
+#define mmDC_GPIO_SYNCA_A 0x4885
+#define mmDC_GPIO_SYNCA_EN 0x4886
+#define mmDC_GPIO_SYNCA_Y 0x4887
+#define mmDC_GPIO_GENLK_MASK 0x4888
+#define mmDC_GPIO_GENLK_A 0x4889
+#define mmDC_GPIO_GENLK_EN 0x488a
+#define mmDC_GPIO_GENLK_Y 0x488b
+#define mmDC_GPIO_HPD_MASK 0x488c
+#define mmDC_GPIO_HPD_A 0x488d
+#define mmDC_GPIO_HPD_EN 0x488e
+#define mmDC_GPIO_HPD_Y 0x488f
+#define mmDC_GPIO_PWRSEQ_MASK 0x4890
+#define mmDC_GPIO_PWRSEQ_A 0x4891
+#define mmDC_GPIO_PWRSEQ_EN 0x4892
+#define mmDC_GPIO_PWRSEQ_Y 0x4893
+#define mmDC_GPIO_PAD_STRENGTH_1 0x4894
+#define mmDC_GPIO_PAD_STRENGTH_2 0x4895
+#define mmPHY_AUX_CNTL 0x4897
+#define mmDC_GPIO_I2CPAD_A 0x4899
+#define mmDC_GPIO_I2CPAD_EN 0x489a
+#define mmDC_GPIO_I2CPAD_Y 0x489b
+#define mmDC_GPIO_I2CPAD_STRENGTH 0x489c
+#define mmDVO_VREF_CONTROL 0x489e
+#define mmDVO_SKEW_ADJUST 0x489f
+#define mmDC_GPIO_RECEIVER_EN0 0x48a0
+#define mmDC_GPIO_RECEIVER_EN1 0x48a1
+#define mmDC_GPIO_I2S_SPDIF_MASK 0x48a8
+#define mmDC_GPIO_I2S_SPDIF_A 0x48a9
+#define mmDC_GPIO_I2S_SPDIF_EN 0x48aa
+#define mmDC_GPIO_I2S_SPDIF_Y 0x48ab
+#define mmDC_GPIO_I2S_SPDIF_STRENGTH 0x48ac
+#define mmDC_GPIO_TX12_EN 0x48ad
+#define mmDC_GPIO_AUX_CTRL_0 0x48ae
+#define mmDC_GPIO_AUX_CTRL_1 0x48af
+#define mmDC_GPIO_AUX_CTRL_2 0x48b0
+#define mmDC_GPIO_HPD_CTRL_0 0x48b1
+#define mmDC_GPIO_HPD_CTRL_1 0x48b2
+#define mmDAC_MACRO_CNTL_RESERVED0 0x48b8
+#define mmDAC_MACRO_CNTL_RESERVED1 0x48b9
+#define mmDAC_MACRO_CNTL_RESERVED2 0x48ba
+#define mmDAC_MACRO_CNTL_RESERVED3 0x48bb
+#define mmUNIPHY_MACRO_CNTL_RESERVED0 0x48c0
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED0 0x48c0
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED0 0x4960
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED0 0x9a00
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED0 0x9aa0
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED0 0x9b40
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED0 0x9be0
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED0 0x9c80
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED0 0x9d20
+#define mmUNIPHY_MACRO_CNTL_RESERVED1 0x48c1
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED1 0x48c1
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED1 0x4961
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED1 0x9a01
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED1 0x9aa1
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED1 0x9b41
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED1 0x9be1
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED1 0x9c81
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED1 0x9d21
+#define mmUNIPHY_MACRO_CNTL_RESERVED2 0x48c2
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED2 0x48c2
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED2 0x4962
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED2 0x9a02
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED2 0x9aa2
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED2 0x9b42
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED2 0x9be2
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED2 0x9c82
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED2 0x9d22
+#define mmUNIPHY_MACRO_CNTL_RESERVED3 0x48c3
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED3 0x48c3
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED3 0x4963
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED3 0x9a03
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED3 0x9aa3
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED3 0x9b43
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED3 0x9be3
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED3 0x9c83
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED3 0x9d23
+#define mmUNIPHY_MACRO_CNTL_RESERVED4 0x48c4
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED4 0x48c4
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED4 0x4964
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED4 0x9a04
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED4 0x9aa4
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED4 0x9b44
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED4 0x9be4
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED4 0x9c84
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED4 0x9d24
+#define mmUNIPHY_MACRO_CNTL_RESERVED5 0x48c5
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED5 0x48c5
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED5 0x4965
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED5 0x9a05
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED5 0x9aa5
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED5 0x9b45
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED5 0x9be5
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED5 0x9c85
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED5 0x9d25
+#define mmUNIPHY_MACRO_CNTL_RESERVED6 0x48c6
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED6 0x48c6
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED6 0x4966
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED6 0x9a06
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED6 0x9aa6
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED6 0x9b46
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED6 0x9be6
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED6 0x9c86
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED6 0x9d26
+#define mmUNIPHY_MACRO_CNTL_RESERVED7 0x48c7
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED7 0x48c7
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED7 0x4967
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED7 0x9a07
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED7 0x9aa7
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED7 0x9b47
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED7 0x9be7
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED7 0x9c87
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED7 0x9d27
+#define mmUNIPHY_MACRO_CNTL_RESERVED8 0x48c8
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED8 0x48c8
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED8 0x4968
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED8 0x9a08
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED8 0x9aa8
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED8 0x9b48
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED8 0x9be8
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED8 0x9c88
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED8 0x9d28
+#define mmUNIPHY_MACRO_CNTL_RESERVED9 0x48c9
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED9 0x48c9
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED9 0x4969
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED9 0x9a09
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED9 0x9aa9
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED9 0x9b49
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED9 0x9be9
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED9 0x9c89
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED9 0x9d29
+#define mmUNIPHY_MACRO_CNTL_RESERVED10 0x48ca
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED10 0x48ca
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED10 0x496a
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED10 0x9a0a
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED10 0x9aaa
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED10 0x9b4a
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED10 0x9bea
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED10 0x9c8a
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED10 0x9d2a
+#define mmUNIPHY_MACRO_CNTL_RESERVED11 0x48cb
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED11 0x48cb
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED11 0x496b
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED11 0x9a0b
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED11 0x9aab
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED11 0x9b4b
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED11 0x9beb
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED11 0x9c8b
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED11 0x9d2b
+#define mmUNIPHY_MACRO_CNTL_RESERVED12 0x48cc
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED12 0x48cc
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED12 0x496c
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED12 0x9a0c
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED12 0x9aac
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED12 0x9b4c
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED12 0x9bec
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED12 0x9c8c
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED12 0x9d2c
+#define mmUNIPHY_MACRO_CNTL_RESERVED13 0x48cd
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED13 0x48cd
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED13 0x496d
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED13 0x9a0d
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED13 0x9aad
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED13 0x9b4d
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED13 0x9bed
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED13 0x9c8d
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED13 0x9d2d
+#define mmUNIPHY_MACRO_CNTL_RESERVED14 0x48ce
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED14 0x48ce
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED14 0x496e
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED14 0x9a0e
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED14 0x9aae
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED14 0x9b4e
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED14 0x9bee
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED14 0x9c8e
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED14 0x9d2e
+#define mmUNIPHY_MACRO_CNTL_RESERVED15 0x48cf
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED15 0x48cf
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED15 0x496f
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED15 0x9a0f
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED15 0x9aaf
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED15 0x9b4f
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED15 0x9bef
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED15 0x9c8f
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED15 0x9d2f
+#define mmUNIPHY_MACRO_CNTL_RESERVED16 0x48d0
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED16 0x48d0
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED16 0x4970
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED16 0x9a10
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED16 0x9ab0
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED16 0x9b50
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED16 0x9bf0
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED16 0x9c90
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED16 0x9d30
+#define mmUNIPHY_MACRO_CNTL_RESERVED17 0x48d1
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED17 0x48d1
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED17 0x4971
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED17 0x9a11
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED17 0x9ab1
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED17 0x9b51
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED17 0x9bf1
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED17 0x9c91
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED17 0x9d31
+#define mmUNIPHY_MACRO_CNTL_RESERVED18 0x48d2
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED18 0x48d2
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED18 0x4972
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED18 0x9a12
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED18 0x9ab2
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED18 0x9b52
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED18 0x9bf2
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED18 0x9c92
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED18 0x9d32
+#define mmUNIPHY_MACRO_CNTL_RESERVED19 0x48d3
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED19 0x48d3
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED19 0x4973
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED19 0x9a13
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED19 0x9ab3
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED19 0x9b53
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED19 0x9bf3
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED19 0x9c93
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED19 0x9d33
+#define mmUNIPHY_MACRO_CNTL_RESERVED20 0x48d4
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED20 0x48d4
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED20 0x4974
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED20 0x9a14
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED20 0x9ab4
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED20 0x9b54
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED20 0x9bf4
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED20 0x9c94
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED20 0x9d34
+#define mmUNIPHY_MACRO_CNTL_RESERVED21 0x48d5
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED21 0x48d5
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED21 0x4975
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED21 0x9a15
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED21 0x9ab5
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED21 0x9b55
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED21 0x9bf5
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED21 0x9c95
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED21 0x9d35
+#define mmUNIPHY_MACRO_CNTL_RESERVED22 0x48d6
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED22 0x48d6
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED22 0x4976
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED22 0x9a16
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED22 0x9ab6
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED22 0x9b56
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED22 0x9bf6
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED22 0x9c96
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED22 0x9d36
+#define mmUNIPHY_MACRO_CNTL_RESERVED23 0x48d7
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED23 0x48d7
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED23 0x4977
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED23 0x9a17
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED23 0x9ab7
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED23 0x9b57
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED23 0x9bf7
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED23 0x9c97
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED23 0x9d37
+#define mmUNIPHY_MACRO_CNTL_RESERVED24 0x48d8
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED24 0x48d8
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED24 0x4978
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED24 0x9a18
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED24 0x9ab8
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED24 0x9b58
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED24 0x9bf8
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED24 0x9c98
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED24 0x9d38
+#define mmUNIPHY_MACRO_CNTL_RESERVED25 0x48d9
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED25 0x48d9
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED25 0x4979
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED25 0x9a19
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED25 0x9ab9
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED25 0x9b59
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED25 0x9bf9
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED25 0x9c99
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED25 0x9d39
+#define mmUNIPHY_MACRO_CNTL_RESERVED26 0x48da
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED26 0x48da
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED26 0x497a
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED26 0x9a1a
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED26 0x9aba
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED26 0x9b5a
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED26 0x9bfa
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED26 0x9c9a
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED26 0x9d3a
+#define mmUNIPHY_MACRO_CNTL_RESERVED27 0x48db
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED27 0x48db
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED27 0x497b
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED27 0x9a1b
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED27 0x9abb
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED27 0x9b5b
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED27 0x9bfb
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED27 0x9c9b
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED27 0x9d3b
+#define mmUNIPHY_MACRO_CNTL_RESERVED28 0x48dc
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED28 0x48dc
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED28 0x497c
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED28 0x9a1c
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED28 0x9abc
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED28 0x9b5c
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED28 0x9bfc
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED28 0x9c9c
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED28 0x9d3c
+#define mmUNIPHY_MACRO_CNTL_RESERVED29 0x48dd
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED29 0x48dd
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED29 0x497d
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED29 0x9a1d
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED29 0x9abd
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED29 0x9b5d
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED29 0x9bfd
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED29 0x9c9d
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED29 0x9d3d
+#define mmUNIPHY_MACRO_CNTL_RESERVED30 0x48de
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED30 0x48de
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED30 0x497e
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED30 0x9a1e
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED30 0x9abe
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED30 0x9b5e
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED30 0x9bfe
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED30 0x9c9e
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED30 0x9d3e
+#define mmUNIPHY_MACRO_CNTL_RESERVED31 0x48df
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED31 0x48df
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED31 0x497f
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED31 0x9a1f
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED31 0x9abf
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED31 0x9b5f
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED31 0x9bff
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED31 0x9c9f
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED31 0x9d3f
+#define mmUNIPHY_MACRO_CNTL_RESERVED32 0x48e0
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED32 0x48e0
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED32 0x4980
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED32 0x9a20
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED32 0x9ac0
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED32 0x9b60
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED32 0x9c00
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED32 0x9ca0
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED32 0x9d40
+#define mmUNIPHY_MACRO_CNTL_RESERVED33 0x48e1
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED33 0x48e1
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED33 0x4981
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED33 0x9a21
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED33 0x9ac1
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED33 0x9b61
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED33 0x9c01
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED33 0x9ca1
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED33 0x9d41
+#define mmUNIPHY_MACRO_CNTL_RESERVED34 0x48e2
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED34 0x48e2
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED34 0x4982
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED34 0x9a22
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED34 0x9ac2
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED34 0x9b62
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED34 0x9c02
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED34 0x9ca2
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED34 0x9d42
+#define mmUNIPHY_MACRO_CNTL_RESERVED35 0x48e3
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED35 0x48e3
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED35 0x4983
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED35 0x9a23
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED35 0x9ac3
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED35 0x9b63
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED35 0x9c03
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED35 0x9ca3
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED35 0x9d43
+#define mmUNIPHY_MACRO_CNTL_RESERVED36 0x48e4
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED36 0x48e4
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED36 0x4984
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED36 0x9a24
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED36 0x9ac4
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED36 0x9b64
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED36 0x9c04
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED36 0x9ca4
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED36 0x9d44
+#define mmUNIPHY_MACRO_CNTL_RESERVED37 0x48e5
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED37 0x48e5
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED37 0x4985
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED37 0x9a25
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED37 0x9ac5
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED37 0x9b65
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED37 0x9c05
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED37 0x9ca5
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED37 0x9d45
+#define mmUNIPHY_MACRO_CNTL_RESERVED38 0x48e6
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED38 0x48e6
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED38 0x4986
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED38 0x9a26
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED38 0x9ac6
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED38 0x9b66
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED38 0x9c06
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED38 0x9ca6
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED38 0x9d46
+#define mmUNIPHY_MACRO_CNTL_RESERVED39 0x48e7
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED39 0x48e7
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED39 0x4987
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED39 0x9a27
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED39 0x9ac7
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED39 0x9b67
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED39 0x9c07
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED39 0x9ca7
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED39 0x9d47
+#define mmUNIPHY_MACRO_CNTL_RESERVED40 0x48e8
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED40 0x48e8
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED40 0x4988
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED40 0x9a28
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED40 0x9ac8
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED40 0x9b68
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED40 0x9c08
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED40 0x9ca8
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED40 0x9d48
+#define mmUNIPHY_MACRO_CNTL_RESERVED41 0x48e9
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED41 0x48e9
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED41 0x4989
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED41 0x9a29
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED41 0x9ac9
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED41 0x9b69
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED41 0x9c09
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED41 0x9ca9
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED41 0x9d49
+#define mmUNIPHY_MACRO_CNTL_RESERVED42 0x48ea
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED42 0x48ea
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED42 0x498a
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED42 0x9a2a
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED42 0x9aca
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED42 0x9b6a
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED42 0x9c0a
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED42 0x9caa
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED42 0x9d4a
+#define mmUNIPHY_MACRO_CNTL_RESERVED43 0x48eb
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED43 0x48eb
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED43 0x498b
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED43 0x9a2b
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED43 0x9acb
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED43 0x9b6b
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED43 0x9c0b
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED43 0x9cab
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED43 0x9d4b
+#define mmUNIPHY_MACRO_CNTL_RESERVED44 0x48ec
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED44 0x48ec
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED44 0x498c
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED44 0x9a2c
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED44 0x9acc
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED44 0x9b6c
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED44 0x9c0c
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED44 0x9cac
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED44 0x9d4c
+#define mmUNIPHY_MACRO_CNTL_RESERVED45 0x48ed
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED45 0x48ed
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED45 0x498d
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED45 0x9a2d
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED45 0x9acd
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED45 0x9b6d
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED45 0x9c0d
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED45 0x9cad
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED45 0x9d4d
+#define mmUNIPHY_MACRO_CNTL_RESERVED46 0x48ee
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED46 0x48ee
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED46 0x498e
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED46 0x9a2e
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED46 0x9ace
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED46 0x9b6e
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED46 0x9c0e
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED46 0x9cae
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED46 0x9d4e
+#define mmUNIPHY_MACRO_CNTL_RESERVED47 0x48ef
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED47 0x48ef
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED47 0x498f
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED47 0x9a2f
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED47 0x9acf
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED47 0x9b6f
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED47 0x9c0f
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED47 0x9caf
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED47 0x9d4f
+#define mmUNIPHY_MACRO_CNTL_RESERVED48 0x48f0
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED48 0x48f0
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED48 0x4990
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED48 0x9a30
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED48 0x9ad0
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED48 0x9b70
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED48 0x9c10
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED48 0x9cb0
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED48 0x9d50
+#define mmUNIPHY_MACRO_CNTL_RESERVED49 0x48f1
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED49 0x48f1
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED49 0x4991
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED49 0x9a31
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED49 0x9ad1
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED49 0x9b71
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED49 0x9c11
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED49 0x9cb1
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED49 0x9d51
+#define mmUNIPHY_MACRO_CNTL_RESERVED50 0x48f2
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED50 0x48f2
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED50 0x4992
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED50 0x9a32
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED50 0x9ad2
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED50 0x9b72
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED50 0x9c12
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED50 0x9cb2
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED50 0x9d52
+#define mmUNIPHY_MACRO_CNTL_RESERVED51 0x48f3
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED51 0x48f3
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED51 0x4993
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED51 0x9a33
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED51 0x9ad3
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED51 0x9b73
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED51 0x9c13
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED51 0x9cb3
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED51 0x9d53
+#define mmUNIPHY_MACRO_CNTL_RESERVED52 0x48f4
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED52 0x48f4
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED52 0x4994
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED52 0x9a34
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED52 0x9ad4
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED52 0x9b74
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED52 0x9c14
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED52 0x9cb4
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED52 0x9d54
+#define mmUNIPHY_MACRO_CNTL_RESERVED53 0x48f5
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED53 0x48f5
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED53 0x4995
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED53 0x9a35
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED53 0x9ad5
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED53 0x9b75
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED53 0x9c15
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED53 0x9cb5
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED53 0x9d55
+#define mmUNIPHY_MACRO_CNTL_RESERVED54 0x48f6
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED54 0x48f6
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED54 0x4996
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED54 0x9a36
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED54 0x9ad6
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED54 0x9b76
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED54 0x9c16
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED54 0x9cb6
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED54 0x9d56
+#define mmUNIPHY_MACRO_CNTL_RESERVED55 0x48f7
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED55 0x48f7
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED55 0x4997
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED55 0x9a37
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED55 0x9ad7
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED55 0x9b77
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED55 0x9c17
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED55 0x9cb7
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED55 0x9d57
+#define mmUNIPHY_MACRO_CNTL_RESERVED56 0x48f8
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED56 0x48f8
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED56 0x4998
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED56 0x9a38
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED56 0x9ad8
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED56 0x9b78
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED56 0x9c18
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED56 0x9cb8
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED56 0x9d58
+#define mmUNIPHY_MACRO_CNTL_RESERVED57 0x48f9
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED57 0x48f9
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED57 0x4999
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED57 0x9a39
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED57 0x9ad9
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED57 0x9b79
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED57 0x9c19
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED57 0x9cb9
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED57 0x9d59
+#define mmUNIPHY_MACRO_CNTL_RESERVED58 0x48fa
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED58 0x48fa
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED58 0x499a
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED58 0x9a3a
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED58 0x9ada
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED58 0x9b7a
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED58 0x9c1a
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED58 0x9cba
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED58 0x9d5a
+#define mmUNIPHY_MACRO_CNTL_RESERVED59 0x48fb
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED59 0x48fb
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED59 0x499b
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED59 0x9a3b
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED59 0x9adb
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED59 0x9b7b
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED59 0x9c1b
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED59 0x9cbb
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED59 0x9d5b
+#define mmUNIPHY_MACRO_CNTL_RESERVED60 0x48fc
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED60 0x48fc
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED60 0x499c
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED60 0x9a3c
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED60 0x9adc
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED60 0x9b7c
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED60 0x9c1c
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED60 0x9cbc
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED60 0x9d5c
+#define mmUNIPHY_MACRO_CNTL_RESERVED61 0x48fd
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED61 0x48fd
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED61 0x499d
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED61 0x9a3d
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED61 0x9add
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED61 0x9b7d
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED61 0x9c1d
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED61 0x9cbd
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED61 0x9d5d
+#define mmUNIPHY_MACRO_CNTL_RESERVED62 0x48fe
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED62 0x48fe
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED62 0x499e
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED62 0x9a3e
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED62 0x9ade
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED62 0x9b7e
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED62 0x9c1e
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED62 0x9cbe
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED62 0x9d5e
+#define mmUNIPHY_MACRO_CNTL_RESERVED63 0x48ff
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED63 0x48ff
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED63 0x499f
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED63 0x9a3f
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED63 0x9adf
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED63 0x9b7f
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED63 0x9c1f
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED63 0x9cbf
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED63 0x9d5f
+#define mmUNIPHY_MACRO_CNTL_RESERVED64 0x4900
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED64 0x4900
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED64 0x49a0
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED64 0x9a40
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED64 0x9ae0
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED64 0x9b80
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED64 0x9c20
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED64 0x9cc0
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED64 0x9d60
+#define mmUNIPHY_MACRO_CNTL_RESERVED65 0x4901
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED65 0x4901
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED65 0x49a1
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED65 0x9a41
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED65 0x9ae1
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED65 0x9b81
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED65 0x9c21
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED65 0x9cc1
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED65 0x9d61
+#define mmUNIPHY_MACRO_CNTL_RESERVED66 0x4902
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED66 0x4902
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED66 0x49a2
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED66 0x9a42
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED66 0x9ae2
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED66 0x9b82
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED66 0x9c22
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED66 0x9cc2
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED66 0x9d62
+#define mmUNIPHY_MACRO_CNTL_RESERVED67 0x4903
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED67 0x4903
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED67 0x49a3
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED67 0x9a43
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED67 0x9ae3
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED67 0x9b83
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED67 0x9c23
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED67 0x9cc3
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED67 0x9d63
+#define mmUNIPHY_MACRO_CNTL_RESERVED68 0x4904
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED68 0x4904
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED68 0x49a4
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED68 0x9a44
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED68 0x9ae4
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED68 0x9b84
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED68 0x9c24
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED68 0x9cc4
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED68 0x9d64
+#define mmUNIPHY_MACRO_CNTL_RESERVED69 0x4905
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED69 0x4905
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED69 0x49a5
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED69 0x9a45
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED69 0x9ae5
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED69 0x9b85
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED69 0x9c25
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED69 0x9cc5
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED69 0x9d65
+#define mmUNIPHY_MACRO_CNTL_RESERVED70 0x4906
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED70 0x4906
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED70 0x49a6
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED70 0x9a46
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED70 0x9ae6
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED70 0x9b86
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED70 0x9c26
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED70 0x9cc6
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED70 0x9d66
+#define mmUNIPHY_MACRO_CNTL_RESERVED71 0x4907
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED71 0x4907
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED71 0x49a7
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED71 0x9a47
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED71 0x9ae7
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED71 0x9b87
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED71 0x9c27
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED71 0x9cc7
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED71 0x9d67
+#define mmUNIPHY_MACRO_CNTL_RESERVED72 0x4908
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED72 0x4908
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED72 0x49a8
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED72 0x9a48
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED72 0x9ae8
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED72 0x9b88
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED72 0x9c28
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED72 0x9cc8
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED72 0x9d68
+#define mmUNIPHY_MACRO_CNTL_RESERVED73 0x4909
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED73 0x4909
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED73 0x49a9
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED73 0x9a49
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED73 0x9ae9
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED73 0x9b89
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED73 0x9c29
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED73 0x9cc9
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED73 0x9d69
+#define mmUNIPHY_MACRO_CNTL_RESERVED74 0x490a
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED74 0x490a
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED74 0x49aa
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED74 0x9a4a
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED74 0x9aea
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED74 0x9b8a
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED74 0x9c2a
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED74 0x9cca
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED74 0x9d6a
+#define mmUNIPHY_MACRO_CNTL_RESERVED75 0x490b
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED75 0x490b
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED75 0x49ab
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED75 0x9a4b
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED75 0x9aeb
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED75 0x9b8b
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED75 0x9c2b
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED75 0x9ccb
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED75 0x9d6b
+#define mmUNIPHY_MACRO_CNTL_RESERVED76 0x490c
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED76 0x490c
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED76 0x49ac
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED76 0x9a4c
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED76 0x9aec
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED76 0x9b8c
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED76 0x9c2c
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED76 0x9ccc
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED76 0x9d6c
+#define mmUNIPHY_MACRO_CNTL_RESERVED77 0x490d
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED77 0x490d
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED77 0x49ad
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED77 0x9a4d
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED77 0x9aed
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED77 0x9b8d
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED77 0x9c2d
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED77 0x9ccd
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED77 0x9d6d
+#define mmUNIPHY_MACRO_CNTL_RESERVED78 0x490e
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED78 0x490e
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED78 0x49ae
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED78 0x9a4e
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED78 0x9aee
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED78 0x9b8e
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED78 0x9c2e
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED78 0x9cce
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED78 0x9d6e
+#define mmUNIPHY_MACRO_CNTL_RESERVED79 0x490f
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED79 0x490f
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED79 0x49af
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED79 0x9a4f
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED79 0x9aef
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED79 0x9b8f
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED79 0x9c2f
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED79 0x9ccf
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED79 0x9d6f
+#define mmUNIPHY_MACRO_CNTL_RESERVED80 0x4910
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED80 0x4910
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED80 0x49b0
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED80 0x9a50
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED80 0x9af0
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED80 0x9b90
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED80 0x9c30
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED80 0x9cd0
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED80 0x9d70
+#define mmUNIPHY_MACRO_CNTL_RESERVED81 0x4911
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED81 0x4911
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED81 0x49b1
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED81 0x9a51
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED81 0x9af1
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED81 0x9b91
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED81 0x9c31
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED81 0x9cd1
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED81 0x9d71
+#define mmUNIPHY_MACRO_CNTL_RESERVED82 0x4912
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED82 0x4912
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED82 0x49b2
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED82 0x9a52
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED82 0x9af2
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED82 0x9b92
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED82 0x9c32
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED82 0x9cd2
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED82 0x9d72
+#define mmUNIPHY_MACRO_CNTL_RESERVED83 0x4913
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED83 0x4913
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED83 0x49b3
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED83 0x9a53
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED83 0x9af3
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED83 0x9b93
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED83 0x9c33
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED83 0x9cd3
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED83 0x9d73
+#define mmUNIPHY_MACRO_CNTL_RESERVED84 0x4914
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED84 0x4914
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED84 0x49b4
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED84 0x9a54
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED84 0x9af4
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED84 0x9b94
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED84 0x9c34
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED84 0x9cd4
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED84 0x9d74
+#define mmUNIPHY_MACRO_CNTL_RESERVED85 0x4915
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED85 0x4915
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED85 0x49b5
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED85 0x9a55
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED85 0x9af5
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED85 0x9b95
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED85 0x9c35
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED85 0x9cd5
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED85 0x9d75
+#define mmUNIPHY_MACRO_CNTL_RESERVED86 0x4916
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED86 0x4916
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED86 0x49b6
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED86 0x9a56
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED86 0x9af6
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED86 0x9b96
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED86 0x9c36
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED86 0x9cd6
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED86 0x9d76
+#define mmUNIPHY_MACRO_CNTL_RESERVED87 0x4917
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED87 0x4917
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED87 0x49b7
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED87 0x9a57
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED87 0x9af7
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED87 0x9b97
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED87 0x9c37
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED87 0x9cd7
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED87 0x9d77
+#define mmUNIPHY_MACRO_CNTL_RESERVED88 0x4918
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED88 0x4918
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED88 0x49b8
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED88 0x9a58
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED88 0x9af8
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED88 0x9b98
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED88 0x9c38
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED88 0x9cd8
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED88 0x9d78
+#define mmUNIPHY_MACRO_CNTL_RESERVED89 0x4919
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED89 0x4919
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED89 0x49b9
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED89 0x9a59
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED89 0x9af9
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED89 0x9b99
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED89 0x9c39
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED89 0x9cd9
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED89 0x9d79
+#define mmUNIPHY_MACRO_CNTL_RESERVED90 0x491a
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED90 0x491a
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED90 0x49ba
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED90 0x9a5a
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED90 0x9afa
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED90 0x9b9a
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED90 0x9c3a
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED90 0x9cda
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED90 0x9d7a
+#define mmUNIPHY_MACRO_CNTL_RESERVED91 0x491b
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED91 0x491b
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED91 0x49bb
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED91 0x9a5b
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED91 0x9afb
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED91 0x9b9b
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED91 0x9c3b
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED91 0x9cdb
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED91 0x9d7b
+#define mmUNIPHY_MACRO_CNTL_RESERVED92 0x491c
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED92 0x491c
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED92 0x49bc
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED92 0x9a5c
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED92 0x9afc
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED92 0x9b9c
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED92 0x9c3c
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED92 0x9cdc
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED92 0x9d7c
+#define mmUNIPHY_MACRO_CNTL_RESERVED93 0x491d
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED93 0x491d
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED93 0x49bd
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED93 0x9a5d
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED93 0x9afd
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED93 0x9b9d
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED93 0x9c3d
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED93 0x9cdd
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED93 0x9d7d
+#define mmUNIPHY_MACRO_CNTL_RESERVED94 0x491e
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED94 0x491e
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED94 0x49be
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED94 0x9a5e
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED94 0x9afe
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED94 0x9b9e
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED94 0x9c3e
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED94 0x9cde
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED94 0x9d7e
+#define mmUNIPHY_MACRO_CNTL_RESERVED95 0x491f
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED95 0x491f
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED95 0x49bf
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED95 0x9a5f
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED95 0x9aff
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED95 0x9b9f
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED95 0x9c3f
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED95 0x9cdf
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED95 0x9d7f
+#define mmUNIPHY_MACRO_CNTL_RESERVED96 0x4920
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED96 0x4920
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED96 0x49c0
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED96 0x9a60
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED96 0x9b00
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED96 0x9ba0
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED96 0x9c40
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED96 0x9ce0
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED96 0x9d80
+#define mmUNIPHY_MACRO_CNTL_RESERVED97 0x4921
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED97 0x4921
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED97 0x49c1
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED97 0x9a61
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED97 0x9b01
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED97 0x9ba1
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED97 0x9c41
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED97 0x9ce1
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED97 0x9d81
+#define mmUNIPHY_MACRO_CNTL_RESERVED98 0x4922
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED98 0x4922
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED98 0x49c2
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED98 0x9a62
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED98 0x9b02
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED98 0x9ba2
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED98 0x9c42
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED98 0x9ce2
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED98 0x9d82
+#define mmUNIPHY_MACRO_CNTL_RESERVED99 0x4923
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED99 0x4923
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED99 0x49c3
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED99 0x9a63
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED99 0x9b03
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED99 0x9ba3
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED99 0x9c43
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED99 0x9ce3
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED99 0x9d83
+#define mmUNIPHY_MACRO_CNTL_RESERVED100 0x4924
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED100 0x4924
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED100 0x49c4
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED100 0x9a64
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED100 0x9b04
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED100 0x9ba4
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED100 0x9c44
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED100 0x9ce4
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED100 0x9d84
+#define mmUNIPHY_MACRO_CNTL_RESERVED101 0x4925
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED101 0x4925
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED101 0x49c5
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED101 0x9a65
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED101 0x9b05
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED101 0x9ba5
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED101 0x9c45
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED101 0x9ce5
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED101 0x9d85
+#define mmUNIPHY_MACRO_CNTL_RESERVED102 0x4926
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED102 0x4926
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED102 0x49c6
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED102 0x9a66
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED102 0x9b06
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED102 0x9ba6
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED102 0x9c46
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED102 0x9ce6
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED102 0x9d86
+#define mmUNIPHY_MACRO_CNTL_RESERVED103 0x4927
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED103 0x4927
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED103 0x49c7
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED103 0x9a67
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED103 0x9b07
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED103 0x9ba7
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED103 0x9c47
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED103 0x9ce7
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED103 0x9d87
+#define mmUNIPHY_MACRO_CNTL_RESERVED104 0x4928
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED104 0x4928
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED104 0x49c8
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED104 0x9a68
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED104 0x9b08
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED104 0x9ba8
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED104 0x9c48
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED104 0x9ce8
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED104 0x9d88
+#define mmUNIPHY_MACRO_CNTL_RESERVED105 0x4929
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED105 0x4929
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED105 0x49c9
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED105 0x9a69
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED105 0x9b09
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED105 0x9ba9
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED105 0x9c49
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED105 0x9ce9
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED105 0x9d89
+#define mmUNIPHY_MACRO_CNTL_RESERVED106 0x492a
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED106 0x492a
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED106 0x49ca
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED106 0x9a6a
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED106 0x9b0a
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED106 0x9baa
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED106 0x9c4a
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED106 0x9cea
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED106 0x9d8a
+#define mmUNIPHY_MACRO_CNTL_RESERVED107 0x492b
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED107 0x492b
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED107 0x49cb
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED107 0x9a6b
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED107 0x9b0b
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED107 0x9bab
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED107 0x9c4b
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED107 0x9ceb
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED107 0x9d8b
+#define mmUNIPHY_MACRO_CNTL_RESERVED108 0x492c
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED108 0x492c
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED108 0x49cc
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED108 0x9a6c
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED108 0x9b0c
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED108 0x9bac
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED108 0x9c4c
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED108 0x9cec
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED108 0x9d8c
+#define mmUNIPHY_MACRO_CNTL_RESERVED109 0x492d
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED109 0x492d
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED109 0x49cd
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED109 0x9a6d
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED109 0x9b0d
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED109 0x9bad
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED109 0x9c4d
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED109 0x9ced
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED109 0x9d8d
+#define mmUNIPHY_MACRO_CNTL_RESERVED110 0x492e
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED110 0x492e
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED110 0x49ce
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED110 0x9a6e
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED110 0x9b0e
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED110 0x9bae
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED110 0x9c4e
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED110 0x9cee
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED110 0x9d8e
+#define mmUNIPHY_MACRO_CNTL_RESERVED111 0x492f
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED111 0x492f
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED111 0x49cf
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED111 0x9a6f
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED111 0x9b0f
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED111 0x9baf
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED111 0x9c4f
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED111 0x9cef
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED111 0x9d8f
+#define mmUNIPHY_MACRO_CNTL_RESERVED112 0x4930
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED112 0x4930
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED112 0x49d0
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED112 0x9a70
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED112 0x9b10
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED112 0x9bb0
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED112 0x9c50
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED112 0x9cf0
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED112 0x9d90
+#define mmUNIPHY_MACRO_CNTL_RESERVED113 0x4931
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED113 0x4931
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED113 0x49d1
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED113 0x9a71
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED113 0x9b11
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED113 0x9bb1
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED113 0x9c51
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED113 0x9cf1
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED113 0x9d91
+#define mmUNIPHY_MACRO_CNTL_RESERVED114 0x4932
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED114 0x4932
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED114 0x49d2
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED114 0x9a72
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED114 0x9b12
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED114 0x9bb2
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED114 0x9c52
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED114 0x9cf2
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED114 0x9d92
+#define mmUNIPHY_MACRO_CNTL_RESERVED115 0x4933
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED115 0x4933
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED115 0x49d3
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED115 0x9a73
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED115 0x9b13
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED115 0x9bb3
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED115 0x9c53
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED115 0x9cf3
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED115 0x9d93
+#define mmUNIPHY_MACRO_CNTL_RESERVED116 0x4934
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED116 0x4934
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED116 0x49d4
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED116 0x9a74
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED116 0x9b14
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED116 0x9bb4
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED116 0x9c54
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED116 0x9cf4
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED116 0x9d94
+#define mmUNIPHY_MACRO_CNTL_RESERVED117 0x4935
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED117 0x4935
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED117 0x49d5
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED117 0x9a75
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED117 0x9b15
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED117 0x9bb5
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED117 0x9c55
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED117 0x9cf5
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED117 0x9d95
+#define mmUNIPHY_MACRO_CNTL_RESERVED118 0x4936
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED118 0x4936
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED118 0x49d6
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED118 0x9a76
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED118 0x9b16
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED118 0x9bb6
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED118 0x9c56
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED118 0x9cf6
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED118 0x9d96
+#define mmUNIPHY_MACRO_CNTL_RESERVED119 0x4937
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED119 0x4937
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED119 0x49d7
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED119 0x9a77
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED119 0x9b17
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED119 0x9bb7
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED119 0x9c57
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED119 0x9cf7
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED119 0x9d97
+#define mmUNIPHY_MACRO_CNTL_RESERVED120 0x4938
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED120 0x4938
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED120 0x49d8
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED120 0x9a78
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED120 0x9b18
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED120 0x9bb8
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED120 0x9c58
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED120 0x9cf8
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED120 0x9d98
+#define mmUNIPHY_MACRO_CNTL_RESERVED121 0x4939
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED121 0x4939
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED121 0x49d9
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED121 0x9a79
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED121 0x9b19
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED121 0x9bb9
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED121 0x9c59
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED121 0x9cf9
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED121 0x9d99
+#define mmUNIPHY_MACRO_CNTL_RESERVED122 0x493a
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED122 0x493a
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED122 0x49da
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED122 0x9a7a
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED122 0x9b1a
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED122 0x9bba
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED122 0x9c5a
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED122 0x9cfa
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED122 0x9d9a
+#define mmUNIPHY_MACRO_CNTL_RESERVED123 0x493b
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED123 0x493b
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED123 0x49db
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED123 0x9a7b
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED123 0x9b1b
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED123 0x9bbb
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED123 0x9c5b
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED123 0x9cfb
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED123 0x9d9b
+#define mmUNIPHY_MACRO_CNTL_RESERVED124 0x493c
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED124 0x493c
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED124 0x49dc
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED124 0x9a7c
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED124 0x9b1c
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED124 0x9bbc
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED124 0x9c5c
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED124 0x9cfc
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED124 0x9d9c
+#define mmUNIPHY_MACRO_CNTL_RESERVED125 0x493d
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED125 0x493d
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED125 0x49dd
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED125 0x9a7d
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED125 0x9b1d
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED125 0x9bbd
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED125 0x9c5d
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED125 0x9cfd
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED125 0x9d9d
+#define mmUNIPHY_MACRO_CNTL_RESERVED126 0x493e
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED126 0x493e
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED126 0x49de
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED126 0x9a7e
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED126 0x9b1e
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED126 0x9bbe
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED126 0x9c5e
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED126 0x9cfe
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED126 0x9d9e
+#define mmUNIPHY_MACRO_CNTL_RESERVED127 0x493f
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED127 0x493f
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED127 0x49df
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED127 0x9a7f
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED127 0x9b1f
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED127 0x9bbf
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED127 0x9c5f
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED127 0x9cff
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED127 0x9d9f
+#define mmUNIPHY_MACRO_CNTL_RESERVED128 0x4940
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED128 0x4940
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED128 0x49e0
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED128 0x9a80
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED128 0x9b20
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED128 0x9bc0
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED128 0x9c60
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED128 0x9d00
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED128 0x9da0
+#define mmUNIPHY_MACRO_CNTL_RESERVED129 0x4941
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED129 0x4941
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED129 0x49e1
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED129 0x9a81
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED129 0x9b21
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED129 0x9bc1
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED129 0x9c61
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED129 0x9d01
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED129 0x9da1
+#define mmUNIPHY_MACRO_CNTL_RESERVED130 0x4942
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED130 0x4942
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED130 0x49e2
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED130 0x9a82
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED130 0x9b22
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED130 0x9bc2
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED130 0x9c62
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED130 0x9d02
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED130 0x9da2
+#define mmUNIPHY_MACRO_CNTL_RESERVED131 0x4943
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED131 0x4943
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED131 0x49e3
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED131 0x9a83
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED131 0x9b23
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED131 0x9bc3
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED131 0x9c63
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED131 0x9d03
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED131 0x9da3
+#define mmUNIPHY_MACRO_CNTL_RESERVED132 0x4944
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED132 0x4944
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED132 0x49e4
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED132 0x9a84
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED132 0x9b24
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED132 0x9bc4
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED132 0x9c64
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED132 0x9d04
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED132 0x9da4
+#define mmUNIPHY_MACRO_CNTL_RESERVED133 0x4945
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED133 0x4945
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED133 0x49e5
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED133 0x9a85
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED133 0x9b25
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED133 0x9bc5
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED133 0x9c65
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED133 0x9d05
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED133 0x9da5
+#define mmUNIPHY_MACRO_CNTL_RESERVED134 0x4946
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED134 0x4946
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED134 0x49e6
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED134 0x9a86
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED134 0x9b26
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED134 0x9bc6
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED134 0x9c66
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED134 0x9d06
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED134 0x9da6
+#define mmUNIPHY_MACRO_CNTL_RESERVED135 0x4947
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED135 0x4947
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED135 0x49e7
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED135 0x9a87
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED135 0x9b27
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED135 0x9bc7
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED135 0x9c67
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED135 0x9d07
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED135 0x9da7
+#define mmUNIPHY_MACRO_CNTL_RESERVED136 0x4948
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED136 0x4948
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED136 0x49e8
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED136 0x9a88
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED136 0x9b28
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED136 0x9bc8
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED136 0x9c68
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED136 0x9d08
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED136 0x9da8
+#define mmUNIPHY_MACRO_CNTL_RESERVED137 0x4949
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED137 0x4949
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED137 0x49e9
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED137 0x9a89
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED137 0x9b29
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED137 0x9bc9
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED137 0x9c69
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED137 0x9d09
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED137 0x9da9
+#define mmUNIPHY_MACRO_CNTL_RESERVED138 0x494a
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED138 0x494a
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED138 0x49ea
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED138 0x9a8a
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED138 0x9b2a
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED138 0x9bca
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED138 0x9c6a
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED138 0x9d0a
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED138 0x9daa
+#define mmUNIPHY_MACRO_CNTL_RESERVED139 0x494b
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED139 0x494b
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED139 0x49eb
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED139 0x9a8b
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED139 0x9b2b
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED139 0x9bcb
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED139 0x9c6b
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED139 0x9d0b
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED139 0x9dab
+#define mmUNIPHY_MACRO_CNTL_RESERVED140 0x494c
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED140 0x494c
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED140 0x49ec
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED140 0x9a8c
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED140 0x9b2c
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED140 0x9bcc
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED140 0x9c6c
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED140 0x9d0c
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED140 0x9dac
+#define mmUNIPHY_MACRO_CNTL_RESERVED141 0x494d
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED141 0x494d
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED141 0x49ed
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED141 0x9a8d
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED141 0x9b2d
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED141 0x9bcd
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED141 0x9c6d
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED141 0x9d0d
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED141 0x9dad
+#define mmUNIPHY_MACRO_CNTL_RESERVED142 0x494e
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED142 0x494e
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED142 0x49ee
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED142 0x9a8e
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED142 0x9b2e
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED142 0x9bce
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED142 0x9c6e
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED142 0x9d0e
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED142 0x9dae
+#define mmUNIPHY_MACRO_CNTL_RESERVED143 0x494f
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED143 0x494f
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED143 0x49ef
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED143 0x9a8f
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED143 0x9b2f
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED143 0x9bcf
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED143 0x9c6f
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED143 0x9d0f
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED143 0x9daf
+#define mmUNIPHY_MACRO_CNTL_RESERVED144 0x4950
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED144 0x4950
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED144 0x49f0
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED144 0x9a90
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED144 0x9b30
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED144 0x9bd0
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED144 0x9c70
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED144 0x9d10
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED144 0x9db0
+#define mmUNIPHY_MACRO_CNTL_RESERVED145 0x4951
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED145 0x4951
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED145 0x49f1
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED145 0x9a91
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED145 0x9b31
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED145 0x9bd1
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED145 0x9c71
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED145 0x9d11
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED145 0x9db1
+#define mmUNIPHY_MACRO_CNTL_RESERVED146 0x4952
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED146 0x4952
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED146 0x49f2
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED146 0x9a92
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED146 0x9b32
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED146 0x9bd2
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED146 0x9c72
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED146 0x9d12
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED146 0x9db2
+#define mmUNIPHY_MACRO_CNTL_RESERVED147 0x4953
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED147 0x4953
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED147 0x49f3
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED147 0x9a93
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED147 0x9b33
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED147 0x9bd3
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED147 0x9c73
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED147 0x9d13
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED147 0x9db3
+#define mmUNIPHY_MACRO_CNTL_RESERVED148 0x4954
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED148 0x4954
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED148 0x49f4
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED148 0x9a94
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED148 0x9b34
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED148 0x9bd4
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED148 0x9c74
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED148 0x9d14
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED148 0x9db4
+#define mmUNIPHY_MACRO_CNTL_RESERVED149 0x4955
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED149 0x4955
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED149 0x49f5
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED149 0x9a95
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED149 0x9b35
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED149 0x9bd5
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED149 0x9c75
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED149 0x9d15
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED149 0x9db5
+#define mmUNIPHY_MACRO_CNTL_RESERVED150 0x4956
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED150 0x4956
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED150 0x49f6
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED150 0x9a96
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED150 0x9b36
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED150 0x9bd6
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED150 0x9c76
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED150 0x9d16
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED150 0x9db6
+#define mmUNIPHY_MACRO_CNTL_RESERVED151 0x4957
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED151 0x4957
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED151 0x49f7
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED151 0x9a97
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED151 0x9b37
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED151 0x9bd7
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED151 0x9c77
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED151 0x9d17
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED151 0x9db7
+#define mmUNIPHY_MACRO_CNTL_RESERVED152 0x4958
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED152 0x4958
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED152 0x49f8
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED152 0x9a98
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED152 0x9b38
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED152 0x9bd8
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED152 0x9c78
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED152 0x9d18
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED152 0x9db8
+#define mmUNIPHY_MACRO_CNTL_RESERVED153 0x4959
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED153 0x4959
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED153 0x49f9
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED153 0x9a99
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED153 0x9b39
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED153 0x9bd9
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED153 0x9c79
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED153 0x9d19
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED153 0x9db9
+#define mmUNIPHY_MACRO_CNTL_RESERVED154 0x495a
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED154 0x495a
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED154 0x49fa
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED154 0x9a9a
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED154 0x9b3a
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED154 0x9bda
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED154 0x9c7a
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED154 0x9d1a
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED154 0x9dba
+#define mmUNIPHY_MACRO_CNTL_RESERVED155 0x495b
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED155 0x495b
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED155 0x49fb
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED155 0x9a9b
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED155 0x9b3b
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED155 0x9bdb
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED155 0x9c7b
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED155 0x9d1b
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED155 0x9dbb
+#define mmUNIPHY_MACRO_CNTL_RESERVED156 0x495c
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED156 0x495c
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED156 0x49fc
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED156 0x9a9c
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED156 0x9b3c
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED156 0x9bdc
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED156 0x9c7c
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED156 0x9d1c
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED156 0x9dbc
+#define mmUNIPHY_MACRO_CNTL_RESERVED157 0x495d
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED157 0x495d
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED157 0x49fd
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED157 0x9a9d
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED157 0x9b3d
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED157 0x9bdd
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED157 0x9c7d
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED157 0x9d1d
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED157 0x9dbd
+#define mmUNIPHY_MACRO_CNTL_RESERVED158 0x495e
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED158 0x495e
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED158 0x49fe
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED158 0x9a9e
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED158 0x9b3e
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED158 0x9bde
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED158 0x9c7e
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED158 0x9d1e
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED158 0x9dbe
+#define mmUNIPHY_MACRO_CNTL_RESERVED159 0x495f
+#define mmDCIO_UNIPHY0_UNIPHY_MACRO_CNTL_RESERVED159 0x495f
+#define mmDCIO_UNIPHY1_UNIPHY_MACRO_CNTL_RESERVED159 0x49ff
+#define mmDCIO_UNIPHY2_UNIPHY_MACRO_CNTL_RESERVED159 0x9a9f
+#define mmDCIO_UNIPHY3_UNIPHY_MACRO_CNTL_RESERVED159 0x9b3f
+#define mmDCIO_UNIPHY4_UNIPHY_MACRO_CNTL_RESERVED159 0x9bdf
+#define mmDCIO_UNIPHY5_UNIPHY_MACRO_CNTL_RESERVED159 0x9c7f
+#define mmDCIO_UNIPHY6_UNIPHY_MACRO_CNTL_RESERVED159 0x9d1f
+#define mmDCIO_UNIPHY7_UNIPHY_MACRO_CNTL_RESERVED159 0x9dbf
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED0 0x5a84
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED1 0x5a85
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED2 0x5a86
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED3 0x5a87
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED4 0x5a88
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED5 0x5a89
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED6 0x5a8a
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED7 0x5a8b
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED8 0x5a8c
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED9 0x5a8d
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED10 0x5a8e
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED11 0x5a8f
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED12 0x5a90
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED13 0x5a91
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED14 0x5a92
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED15 0x5a93
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED16 0x5a94
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED17 0x5a95
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED18 0x5a96
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED19 0x5a97
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED20 0x5a98
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED21 0x5a99
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED22 0x5a9a
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED23 0x5a9b
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED24 0x5a9c
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED25 0x5a9d
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED26 0x5a9e
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED27 0x5a9f
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED28 0x5aa0
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED29 0x5aa1
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED30 0x5aa2
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED31 0x5aa3
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED32 0x5aa4
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED33 0x5aa5
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED34 0x5aa6
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED35 0x5aa7
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED36 0x5aa8
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED37 0x5aa9
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED38 0x5aaa
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED39 0x5aab
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED40 0x5aac
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED41 0x5aad
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED42 0x5aae
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED43 0x5aaf
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED44 0x5ab0
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED45 0x5ab1
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED46 0x5ab2
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED47 0x5ab3
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED48 0x5ab4
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED49 0x5ab5
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED50 0x5ab6
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED51 0x5ab7
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED52 0x5ab8
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED53 0x5ab9
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED54 0x5aba
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED55 0x5abb
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED56 0x5abc
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED57 0x5abd
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED58 0x5abe
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED59 0x5abf
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED60 0x5ac0
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED61 0x5ac1
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED62 0x5ac2
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED63 0x5ac3
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED64 0x5ac4
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED65 0x5ac5
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED66 0x5ac6
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED67 0x5ac7
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED68 0x5ac8
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED69 0x5ac9
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED70 0x5aca
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED71 0x5acb
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED72 0x5acc
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED73 0x5acd
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED74 0x5ace
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED75 0x5acf
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED76 0x5ad0
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED77 0x5ad1
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED78 0x5ad2
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED79 0x5ad3
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED80 0x5ad4
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED81 0x5ad5
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED82 0x5ad6
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED83 0x5ad7
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED84 0x5ad8
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED85 0x5ad9
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED86 0x5ada
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED87 0x5adb
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED88 0x5adc
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED89 0x5add
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED90 0x5ade
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED91 0x5adf
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED92 0x5ae0
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED93 0x5ae1
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED94 0x5ae2
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED95 0x5ae3
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED96 0x5ae4
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED97 0x5ae5
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED98 0x5ae6
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED99 0x5ae7
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED100 0x5ae8
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED101 0x5ae9
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED102 0x5aea
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED103 0x5aeb
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED104 0x5aec
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED105 0x5aed
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED106 0x5aee
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED107 0x5aef
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED108 0x5af0
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED109 0x5af1
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED110 0x5af2
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED111 0x5af3
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED112 0x5af4
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED113 0x5af5
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED114 0x5af6
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED115 0x5af7
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED116 0x5af8
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED117 0x5af9
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED118 0x5afa
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED119 0x5afb
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED120 0x5afc
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED121 0x5afd
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED122 0x5afe
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED123 0x5aff
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED124 0x5b00
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED125 0x5b01
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED126 0x5b02
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED127 0x5b03
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED128 0x5b04
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED129 0x5b05
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED130 0x5b06
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED131 0x5b07
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED132 0x5b08
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED133 0x5b09
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED134 0x5b0a
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED135 0x5b0b
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED136 0x5b0c
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED137 0x5b0d
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED138 0x5b0e
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED139 0x5b0f
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED140 0x5b10
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED141 0x5b11
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED142 0x5b12
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED143 0x5b13
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED144 0x5b14
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED145 0x5b15
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED146 0x5b16
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED147 0x5b17
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED148 0x5b18
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED149 0x5b19
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED150 0x5b1a
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED151 0x5b1b
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED152 0x5b1c
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED153 0x5b1d
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED154 0x5b1e
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED155 0x5b1f
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED156 0x5b20
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED157 0x5b21
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED158 0x5b22
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED159 0x5b23
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED160 0x5b24
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED161 0x5b25
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED162 0x5b26
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED163 0x5b27
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED164 0x5b28
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED165 0x5b29
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED166 0x5b2a
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED167 0x5b2b
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED168 0x5b2c
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED169 0x5b2d
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED170 0x5b2e
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED171 0x5b2f
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED172 0x5b30
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED173 0x5b31
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED174 0x5b32
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED175 0x5b33
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED176 0x5b34
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED177 0x5b35
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED178 0x5b36
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED179 0x5b37
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED180 0x5b38
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED181 0x5b39
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED182 0x5b3a
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED183 0x5b3b
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED184 0x5b3c
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED185 0x5b3d
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED186 0x5b3e
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED187 0x5b3f
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED188 0x5b40
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED189 0x5b41
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED190 0x5b42
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED191 0x5b43
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED192 0x5b44
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED193 0x5b45
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED194 0x5b46
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED195 0x5b47
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED196 0x5b48
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED197 0x5b49
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED198 0x5b4a
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED199 0x5b4b
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED200 0x5b4c
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED201 0x5b4d
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED202 0x5b4e
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED203 0x5b4f
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED204 0x5b50
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED205 0x5b51
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED206 0x5b52
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED207 0x5b53
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED208 0x5b54
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED209 0x5b55
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED210 0x5b56
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED211 0x5b57
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED212 0x5b58
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED213 0x5b59
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED214 0x5b5a
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED215 0x5b5b
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED216 0x5b5c
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED217 0x5b5d
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED218 0x5b5e
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED219 0x5b5f
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED220 0x5b60
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED221 0x5b61
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED222 0x5b62
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED223 0x5b63
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED224 0x5b64
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED225 0x5b65
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED226 0x5b66
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED227 0x5b67
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED228 0x5b68
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED229 0x5b69
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED230 0x5b6a
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED231 0x5b6b
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED232 0x5b6c
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED233 0x5b6d
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED234 0x5b6e
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED235 0x5b6f
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED236 0x5b70
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED237 0x5b71
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED238 0x5b72
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED239 0x5b73
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED240 0x5b74
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED241 0x5b75
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED242 0x5b76
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED243 0x5b77
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED244 0x5b78
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED245 0x5b79
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED246 0x5b7a
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED247 0x5b7b
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED248 0x5b7c
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED249 0x5b7d
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED250 0x5b7e
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED251 0x5b7f
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED252 0x5b80
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED253 0x5b81
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED254 0x5b82
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED255 0x5b83
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED256 0x5b84
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED257 0x5b85
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED258 0x5b86
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED259 0x5b87
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED260 0x5b88
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED261 0x5b89
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED262 0x5b8a
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED263 0x5b8b
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED264 0x5b8c
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED265 0x5b8d
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED266 0x5b8e
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED267 0x5b8f
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED268 0x5b90
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED269 0x5b91
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED270 0x5b92
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED271 0x5b93
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED272 0x5b94
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED273 0x5b95
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED274 0x5b96
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED275 0x5b97
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED276 0x5b98
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED277 0x5b99
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED278 0x5b9a
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED279 0x5b9b
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED280 0x5b9c
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED281 0x5b9d
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED282 0x5b9e
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED283 0x5b9f
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED284 0x5ba0
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED285 0x5ba1
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED286 0x5ba2
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED287 0x5ba3
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED288 0x5ba4
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED289 0x5ba5
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED290 0x5ba6
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED291 0x5ba7
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED292 0x5ba8
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED293 0x5ba9
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED294 0x5baa
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED295 0x5bab
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED296 0x5bac
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED297 0x5bad
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED298 0x5bae
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED299 0x5baf
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED300 0x5bb0
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED301 0x5bb1
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED302 0x5bb2
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED303 0x5bb3
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED304 0x5bb4
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED305 0x5bb5
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED306 0x5bb6
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED307 0x5bb7
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED308 0x5bb8
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED309 0x5bb9
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED310 0x5bba
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED311 0x5bbb
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED312 0x5bbc
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED313 0x5bbd
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED314 0x5bbe
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED315 0x5bbf
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED316 0x5bc0
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED317 0x5bc1
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED318 0x5bc2
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED319 0x5bc3
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED320 0x5bc4
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED321 0x5bc5
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED322 0x5bc6
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED323 0x5bc7
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED324 0x5bc8
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED325 0x5bc9
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED326 0x5bca
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED327 0x5bcb
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED328 0x5bcc
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED329 0x5bcd
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED330 0x5bce
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED331 0x5bcf
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED332 0x5bd0
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED333 0x5bd1
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED334 0x5bd2
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED335 0x5bd3
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED336 0x5bd4
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED337 0x5bd5
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED338 0x5bd6
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED339 0x5bd7
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED340 0x5bd8
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED341 0x5bd9
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED342 0x5bda
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED343 0x5bdb
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED344 0x5bdc
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED345 0x5bdd
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED346 0x5bde
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED347 0x5bdf
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED348 0x5be0
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED349 0x5be1
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED350 0x5be2
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED351 0x5be3
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED352 0x5be4
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED353 0x5be5
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED354 0x5be6
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED355 0x5be7
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED356 0x5be8
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED357 0x5be9
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED358 0x5bea
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED359 0x5beb
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED360 0x5bec
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED361 0x5bed
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED362 0x5bee
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED363 0x5bef
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED364 0x5bf0
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED365 0x5bf1
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED366 0x5bf2
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED367 0x5bf3
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED368 0x5bf4
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED369 0x5bf5
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED370 0x5bf6
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED371 0x5bf7
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED372 0x5bf8
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED373 0x5bf9
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED374 0x5bfa
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED375 0x5bfb
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED376 0x5bfc
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED377 0x5bfd
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED378 0x5bfe
+#define mmDCRX_PHY_MACRO_CNTL_RESERVED379 0x5bff
+#define mmDPHY_MACRO_CNTL_RESERVED0 0x5d98
+#define mmDPHY_MACRO_CNTL_RESERVED1 0x5d99
+#define mmDPHY_MACRO_CNTL_RESERVED2 0x5d9a
+#define mmDPHY_MACRO_CNTL_RESERVED3 0x5d9b
+#define mmDPHY_MACRO_CNTL_RESERVED4 0x5d9c
+#define mmDPHY_MACRO_CNTL_RESERVED5 0x5d9d
+#define mmDPHY_MACRO_CNTL_RESERVED6 0x5d9e
+#define mmDPHY_MACRO_CNTL_RESERVED7 0x5d9f
+#define mmDPHY_MACRO_CNTL_RESERVED8 0x5da0
+#define mmDPHY_MACRO_CNTL_RESERVED9 0x5da1
+#define mmDPHY_MACRO_CNTL_RESERVED10 0x5da2
+#define mmDPHY_MACRO_CNTL_RESERVED11 0x5da3
+#define mmDPHY_MACRO_CNTL_RESERVED12 0x5da4
+#define mmDPHY_MACRO_CNTL_RESERVED13 0x5da5
+#define mmDPHY_MACRO_CNTL_RESERVED14 0x5da6
+#define mmDPHY_MACRO_CNTL_RESERVED15 0x5da7
+#define mmDPHY_MACRO_CNTL_RESERVED16 0x5da8
+#define mmDPHY_MACRO_CNTL_RESERVED17 0x5da9
+#define mmDPHY_MACRO_CNTL_RESERVED18 0x5daa
+#define mmDPHY_MACRO_CNTL_RESERVED19 0x5dab
+#define mmDPHY_MACRO_CNTL_RESERVED20 0x5dac
+#define mmDPHY_MACRO_CNTL_RESERVED21 0x5dad
+#define mmDPHY_MACRO_CNTL_RESERVED22 0x5dae
+#define mmDPHY_MACRO_CNTL_RESERVED23 0x5daf
+#define mmDPHY_MACRO_CNTL_RESERVED24 0x5db0
+#define mmDPHY_MACRO_CNTL_RESERVED25 0x5db1
+#define mmDPHY_MACRO_CNTL_RESERVED26 0x5db2
+#define mmDPHY_MACRO_CNTL_RESERVED27 0x5db3
+#define mmDPHY_MACRO_CNTL_RESERVED28 0x5db4
+#define mmDPHY_MACRO_CNTL_RESERVED29 0x5db5
+#define mmDPHY_MACRO_CNTL_RESERVED30 0x5db6
+#define mmDPHY_MACRO_CNTL_RESERVED31 0x5db7
+#define mmDPHY_MACRO_CNTL_RESERVED32 0x5db8
+#define mmDPHY_MACRO_CNTL_RESERVED33 0x5db9
+#define mmDPHY_MACRO_CNTL_RESERVED34 0x5dba
+#define mmDPHY_MACRO_CNTL_RESERVED35 0x5dbb
+#define mmDPHY_MACRO_CNTL_RESERVED36 0x5dbc
+#define mmDPHY_MACRO_CNTL_RESERVED37 0x5dbd
+#define mmDPHY_MACRO_CNTL_RESERVED38 0x5dbe
+#define mmDPHY_MACRO_CNTL_RESERVED39 0x5dbf
+#define mmDPHY_MACRO_CNTL_RESERVED40 0x5dc0
+#define mmDPHY_MACRO_CNTL_RESERVED41 0x5dc1
+#define mmDPHY_MACRO_CNTL_RESERVED42 0x5dc2
+#define mmDPHY_MACRO_CNTL_RESERVED43 0x5dc3
+#define mmDPHY_MACRO_CNTL_RESERVED44 0x5dc4
+#define mmDPHY_MACRO_CNTL_RESERVED45 0x5dc5
+#define mmDPHY_MACRO_CNTL_RESERVED46 0x5dc6
+#define mmDPHY_MACRO_CNTL_RESERVED47 0x5dc7
+#define mmDPHY_MACRO_CNTL_RESERVED48 0x5dc8
+#define mmDPHY_MACRO_CNTL_RESERVED49 0x5dc9
+#define mmDPHY_MACRO_CNTL_RESERVED50 0x5dca
+#define mmDPHY_MACRO_CNTL_RESERVED51 0x5dcb
+#define mmDPHY_MACRO_CNTL_RESERVED52 0x5dcc
+#define mmDPHY_MACRO_CNTL_RESERVED53 0x5dcd
+#define mmDPHY_MACRO_CNTL_RESERVED54 0x5dce
+#define mmDPHY_MACRO_CNTL_RESERVED55 0x5dcf
+#define mmDPHY_MACRO_CNTL_RESERVED56 0x5dd0
+#define mmDPHY_MACRO_CNTL_RESERVED57 0x5dd1
+#define mmDPHY_MACRO_CNTL_RESERVED58 0x5dd2
+#define mmDPHY_MACRO_CNTL_RESERVED59 0x5dd3
+#define mmDPHY_MACRO_CNTL_RESERVED60 0x5dd4
+#define mmDPHY_MACRO_CNTL_RESERVED61 0x5dd5
+#define mmDPHY_MACRO_CNTL_RESERVED62 0x5dd6
+#define mmDPHY_MACRO_CNTL_RESERVED63 0x5dd7
+#define mmGRPH_ENABLE 0x1a00
+#define mmDCP0_GRPH_ENABLE 0x1a00
+#define mmDCP1_GRPH_ENABLE 0x1c00
+#define mmDCP2_GRPH_ENABLE 0x1e00
+#define mmDCP3_GRPH_ENABLE 0x4000
+#define mmDCP4_GRPH_ENABLE 0x4200
+#define mmDCP5_GRPH_ENABLE 0x4400
+#define mmGRPH_CONTROL 0x1a01
+#define mmDCP0_GRPH_CONTROL 0x1a01
+#define mmDCP1_GRPH_CONTROL 0x1c01
+#define mmDCP2_GRPH_CONTROL 0x1e01
+#define mmDCP3_GRPH_CONTROL 0x4001
+#define mmDCP4_GRPH_CONTROL 0x4201
+#define mmDCP5_GRPH_CONTROL 0x4401
+#define mmGRPH_LUT_10BIT_BYPASS 0x1a02
+#define mmDCP0_GRPH_LUT_10BIT_BYPASS 0x1a02
+#define mmDCP1_GRPH_LUT_10BIT_BYPASS 0x1c02
+#define mmDCP2_GRPH_LUT_10BIT_BYPASS 0x1e02
+#define mmDCP3_GRPH_LUT_10BIT_BYPASS 0x4002
+#define mmDCP4_GRPH_LUT_10BIT_BYPASS 0x4202
+#define mmDCP5_GRPH_LUT_10BIT_BYPASS 0x4402
+#define mmGRPH_SWAP_CNTL 0x1a03
+#define mmDCP0_GRPH_SWAP_CNTL 0x1a03
+#define mmDCP1_GRPH_SWAP_CNTL 0x1c03
+#define mmDCP2_GRPH_SWAP_CNTL 0x1e03
+#define mmDCP3_GRPH_SWAP_CNTL 0x4003
+#define mmDCP4_GRPH_SWAP_CNTL 0x4203
+#define mmDCP5_GRPH_SWAP_CNTL 0x4403
+#define mmGRPH_PRIMARY_SURFACE_ADDRESS 0x1a04
+#define mmDCP0_GRPH_PRIMARY_SURFACE_ADDRESS 0x1a04
+#define mmDCP1_GRPH_PRIMARY_SURFACE_ADDRESS 0x1c04
+#define mmDCP2_GRPH_PRIMARY_SURFACE_ADDRESS 0x1e04
+#define mmDCP3_GRPH_PRIMARY_SURFACE_ADDRESS 0x4004
+#define mmDCP4_GRPH_PRIMARY_SURFACE_ADDRESS 0x4204
+#define mmDCP5_GRPH_PRIMARY_SURFACE_ADDRESS 0x4404
+#define mmGRPH_SECONDARY_SURFACE_ADDRESS 0x1a05
+#define mmDCP0_GRPH_SECONDARY_SURFACE_ADDRESS 0x1a05
+#define mmDCP1_GRPH_SECONDARY_SURFACE_ADDRESS 0x1c05
+#define mmDCP2_GRPH_SECONDARY_SURFACE_ADDRESS 0x1e05
+#define mmDCP3_GRPH_SECONDARY_SURFACE_ADDRESS 0x4005
+#define mmDCP4_GRPH_SECONDARY_SURFACE_ADDRESS 0x4205
+#define mmDCP5_GRPH_SECONDARY_SURFACE_ADDRESS 0x4405
+#define mmGRPH_PITCH 0x1a06
+#define mmDCP0_GRPH_PITCH 0x1a06
+#define mmDCP1_GRPH_PITCH 0x1c06
+#define mmDCP2_GRPH_PITCH 0x1e06
+#define mmDCP3_GRPH_PITCH 0x4006
+#define mmDCP4_GRPH_PITCH 0x4206
+#define mmDCP5_GRPH_PITCH 0x4406
+#define mmGRPH_PRIMARY_SURFACE_ADDRESS_HIGH 0x1a07
+#define mmDCP0_GRPH_PRIMARY_SURFACE_ADDRESS_HIGH 0x1a07
+#define mmDCP1_GRPH_PRIMARY_SURFACE_ADDRESS_HIGH 0x1c07
+#define mmDCP2_GRPH_PRIMARY_SURFACE_ADDRESS_HIGH 0x1e07
+#define mmDCP3_GRPH_PRIMARY_SURFACE_ADDRESS_HIGH 0x4007
+#define mmDCP4_GRPH_PRIMARY_SURFACE_ADDRESS_HIGH 0x4207
+#define mmDCP5_GRPH_PRIMARY_SURFACE_ADDRESS_HIGH 0x4407
+#define mmGRPH_SECONDARY_SURFACE_ADDRESS_HIGH 0x1a08
+#define mmDCP0_GRPH_SECONDARY_SURFACE_ADDRESS_HIGH 0x1a08
+#define mmDCP1_GRPH_SECONDARY_SURFACE_ADDRESS_HIGH 0x1c08
+#define mmDCP2_GRPH_SECONDARY_SURFACE_ADDRESS_HIGH 0x1e08
+#define mmDCP3_GRPH_SECONDARY_SURFACE_ADDRESS_HIGH 0x4008
+#define mmDCP4_GRPH_SECONDARY_SURFACE_ADDRESS_HIGH 0x4208
+#define mmDCP5_GRPH_SECONDARY_SURFACE_ADDRESS_HIGH 0x4408
+#define mmGRPH_SURFACE_OFFSET_X 0x1a09
+#define mmDCP0_GRPH_SURFACE_OFFSET_X 0x1a09
+#define mmDCP1_GRPH_SURFACE_OFFSET_X 0x1c09
+#define mmDCP2_GRPH_SURFACE_OFFSET_X 0x1e09
+#define mmDCP3_GRPH_SURFACE_OFFSET_X 0x4009
+#define mmDCP4_GRPH_SURFACE_OFFSET_X 0x4209
+#define mmDCP5_GRPH_SURFACE_OFFSET_X 0x4409
+#define mmGRPH_SURFACE_OFFSET_Y 0x1a0a
+#define mmDCP0_GRPH_SURFACE_OFFSET_Y 0x1a0a
+#define mmDCP1_GRPH_SURFACE_OFFSET_Y 0x1c0a
+#define mmDCP2_GRPH_SURFACE_OFFSET_Y 0x1e0a
+#define mmDCP3_GRPH_SURFACE_OFFSET_Y 0x400a
+#define mmDCP4_GRPH_SURFACE_OFFSET_Y 0x420a
+#define mmDCP5_GRPH_SURFACE_OFFSET_Y 0x440a
+#define mmGRPH_X_START 0x1a0b
+#define mmDCP0_GRPH_X_START 0x1a0b
+#define mmDCP1_GRPH_X_START 0x1c0b
+#define mmDCP2_GRPH_X_START 0x1e0b
+#define mmDCP3_GRPH_X_START 0x400b
+#define mmDCP4_GRPH_X_START 0x420b
+#define mmDCP5_GRPH_X_START 0x440b
+#define mmGRPH_Y_START 0x1a0c
+#define mmDCP0_GRPH_Y_START 0x1a0c
+#define mmDCP1_GRPH_Y_START 0x1c0c
+#define mmDCP2_GRPH_Y_START 0x1e0c
+#define mmDCP3_GRPH_Y_START 0x400c
+#define mmDCP4_GRPH_Y_START 0x420c
+#define mmDCP5_GRPH_Y_START 0x440c
+#define mmGRPH_X_END 0x1a0d
+#define mmDCP0_GRPH_X_END 0x1a0d
+#define mmDCP1_GRPH_X_END 0x1c0d
+#define mmDCP2_GRPH_X_END 0x1e0d
+#define mmDCP3_GRPH_X_END 0x400d
+#define mmDCP4_GRPH_X_END 0x420d
+#define mmDCP5_GRPH_X_END 0x440d
+#define mmGRPH_Y_END 0x1a0e
+#define mmDCP0_GRPH_Y_END 0x1a0e
+#define mmDCP1_GRPH_Y_END 0x1c0e
+#define mmDCP2_GRPH_Y_END 0x1e0e
+#define mmDCP3_GRPH_Y_END 0x400e
+#define mmDCP4_GRPH_Y_END 0x420e
+#define mmDCP5_GRPH_Y_END 0x440e
+#define mmINPUT_GAMMA_CONTROL 0x1a10
+#define mmDCP0_INPUT_GAMMA_CONTROL 0x1a10
+#define mmDCP1_INPUT_GAMMA_CONTROL 0x1c10
+#define mmDCP2_INPUT_GAMMA_CONTROL 0x1e10
+#define mmDCP3_INPUT_GAMMA_CONTROL 0x4010
+#define mmDCP4_INPUT_GAMMA_CONTROL 0x4210
+#define mmDCP5_INPUT_GAMMA_CONTROL 0x4410
+#define mmGRPH_UPDATE 0x1a11
+#define mmDCP0_GRPH_UPDATE 0x1a11
+#define mmDCP1_GRPH_UPDATE 0x1c11
+#define mmDCP2_GRPH_UPDATE 0x1e11
+#define mmDCP3_GRPH_UPDATE 0x4011
+#define mmDCP4_GRPH_UPDATE 0x4211
+#define mmDCP5_GRPH_UPDATE 0x4411
+#define mmGRPH_FLIP_CONTROL 0x1a12
+#define mmDCP0_GRPH_FLIP_CONTROL 0x1a12
+#define mmDCP1_GRPH_FLIP_CONTROL 0x1c12
+#define mmDCP2_GRPH_FLIP_CONTROL 0x1e12
+#define mmDCP3_GRPH_FLIP_CONTROL 0x4012
+#define mmDCP4_GRPH_FLIP_CONTROL 0x4212
+#define mmDCP5_GRPH_FLIP_CONTROL 0x4412
+#define mmGRPH_SURFACE_ADDRESS_INUSE 0x1a13
+#define mmDCP0_GRPH_SURFACE_ADDRESS_INUSE 0x1a13
+#define mmDCP1_GRPH_SURFACE_ADDRESS_INUSE 0x1c13
+#define mmDCP2_GRPH_SURFACE_ADDRESS_INUSE 0x1e13
+#define mmDCP3_GRPH_SURFACE_ADDRESS_INUSE 0x4013
+#define mmDCP4_GRPH_SURFACE_ADDRESS_INUSE 0x4213
+#define mmDCP5_GRPH_SURFACE_ADDRESS_INUSE 0x4413
+#define mmGRPH_DFQ_CONTROL 0x1a14
+#define mmDCP0_GRPH_DFQ_CONTROL 0x1a14
+#define mmDCP1_GRPH_DFQ_CONTROL 0x1c14
+#define mmDCP2_GRPH_DFQ_CONTROL 0x1e14
+#define mmDCP3_GRPH_DFQ_CONTROL 0x4014
+#define mmDCP4_GRPH_DFQ_CONTROL 0x4214
+#define mmDCP5_GRPH_DFQ_CONTROL 0x4414
+#define mmGRPH_DFQ_STATUS 0x1a15
+#define mmDCP0_GRPH_DFQ_STATUS 0x1a15
+#define mmDCP1_GRPH_DFQ_STATUS 0x1c15
+#define mmDCP2_GRPH_DFQ_STATUS 0x1e15
+#define mmDCP3_GRPH_DFQ_STATUS 0x4015
+#define mmDCP4_GRPH_DFQ_STATUS 0x4215
+#define mmDCP5_GRPH_DFQ_STATUS 0x4415
+#define mmGRPH_INTERRUPT_STATUS 0x1a16
+#define mmDCP0_GRPH_INTERRUPT_STATUS 0x1a16
+#define mmDCP1_GRPH_INTERRUPT_STATUS 0x1c16
+#define mmDCP2_GRPH_INTERRUPT_STATUS 0x1e16
+#define mmDCP3_GRPH_INTERRUPT_STATUS 0x4016
+#define mmDCP4_GRPH_INTERRUPT_STATUS 0x4216
+#define mmDCP5_GRPH_INTERRUPT_STATUS 0x4416
+#define mmGRPH_INTERRUPT_CONTROL 0x1a17
+#define mmDCP0_GRPH_INTERRUPT_CONTROL 0x1a17
+#define mmDCP1_GRPH_INTERRUPT_CONTROL 0x1c17
+#define mmDCP2_GRPH_INTERRUPT_CONTROL 0x1e17
+#define mmDCP3_GRPH_INTERRUPT_CONTROL 0x4017
+#define mmDCP4_GRPH_INTERRUPT_CONTROL 0x4217
+#define mmDCP5_GRPH_INTERRUPT_CONTROL 0x4417
+#define mmGRPH_SURFACE_ADDRESS_HIGH_INUSE 0x1a18
+#define mmDCP0_GRPH_SURFACE_ADDRESS_HIGH_INUSE 0x1a18
+#define mmDCP1_GRPH_SURFACE_ADDRESS_HIGH_INUSE 0x1c18
+#define mmDCP2_GRPH_SURFACE_ADDRESS_HIGH_INUSE 0x1e18
+#define mmDCP3_GRPH_SURFACE_ADDRESS_HIGH_INUSE 0x4018
+#define mmDCP4_GRPH_SURFACE_ADDRESS_HIGH_INUSE 0x4218
+#define mmDCP5_GRPH_SURFACE_ADDRESS_HIGH_INUSE 0x4418
+#define mmGRPH_COMPRESS_SURFACE_ADDRESS 0x1a19
+#define mmDCP0_GRPH_COMPRESS_SURFACE_ADDRESS 0x1a19
+#define mmDCP1_GRPH_COMPRESS_SURFACE_ADDRESS 0x1c19
+#define mmDCP2_GRPH_COMPRESS_SURFACE_ADDRESS 0x1e19
+#define mmDCP3_GRPH_COMPRESS_SURFACE_ADDRESS 0x4019
+#define mmDCP4_GRPH_COMPRESS_SURFACE_ADDRESS 0x4219
+#define mmDCP5_GRPH_COMPRESS_SURFACE_ADDRESS 0x4419
+#define mmGRPH_COMPRESS_PITCH 0x1a1a
+#define mmDCP0_GRPH_COMPRESS_PITCH 0x1a1a
+#define mmDCP1_GRPH_COMPRESS_PITCH 0x1c1a
+#define mmDCP2_GRPH_COMPRESS_PITCH 0x1e1a
+#define mmDCP3_GRPH_COMPRESS_PITCH 0x401a
+#define mmDCP4_GRPH_COMPRESS_PITCH 0x421a
+#define mmDCP5_GRPH_COMPRESS_PITCH 0x441a
+#define mmGRPH_COMPRESS_SURFACE_ADDRESS_HIGH 0x1a1b
+#define mmDCP0_GRPH_COMPRESS_SURFACE_ADDRESS_HIGH 0x1a1b
+#define mmDCP1_GRPH_COMPRESS_SURFACE_ADDRESS_HIGH 0x1c1b
+#define mmDCP2_GRPH_COMPRESS_SURFACE_ADDRESS_HIGH 0x1e1b
+#define mmDCP3_GRPH_COMPRESS_SURFACE_ADDRESS_HIGH 0x401b
+#define mmDCP4_GRPH_COMPRESS_SURFACE_ADDRESS_HIGH 0x421b
+#define mmDCP5_GRPH_COMPRESS_SURFACE_ADDRESS_HIGH 0x441b
+#define mmGRPH_PIPE_OUTSTANDING_REQUEST_LIMIT 0x1a1c
+#define mmDCP0_GRPH_PIPE_OUTSTANDING_REQUEST_LIMIT 0x1a1c
+#define mmDCP1_GRPH_PIPE_OUTSTANDING_REQUEST_LIMIT 0x1c1c
+#define mmDCP2_GRPH_PIPE_OUTSTANDING_REQUEST_LIMIT 0x1e1c
+#define mmDCP3_GRPH_PIPE_OUTSTANDING_REQUEST_LIMIT 0x401c
+#define mmDCP4_GRPH_PIPE_OUTSTANDING_REQUEST_LIMIT 0x421c
+#define mmDCP5_GRPH_PIPE_OUTSTANDING_REQUEST_LIMIT 0x441c
+#define mmPRESCALE_GRPH_CONTROL 0x1a2d
+#define mmDCP0_PRESCALE_GRPH_CONTROL 0x1a2d
+#define mmDCP1_PRESCALE_GRPH_CONTROL 0x1c2d
+#define mmDCP2_PRESCALE_GRPH_CONTROL 0x1e2d
+#define mmDCP3_PRESCALE_GRPH_CONTROL 0x402d
+#define mmDCP4_PRESCALE_GRPH_CONTROL 0x422d
+#define mmDCP5_PRESCALE_GRPH_CONTROL 0x442d
+#define mmPRESCALE_VALUES_GRPH_R 0x1a2e
+#define mmDCP0_PRESCALE_VALUES_GRPH_R 0x1a2e
+#define mmDCP1_PRESCALE_VALUES_GRPH_R 0x1c2e
+#define mmDCP2_PRESCALE_VALUES_GRPH_R 0x1e2e
+#define mmDCP3_PRESCALE_VALUES_GRPH_R 0x402e
+#define mmDCP4_PRESCALE_VALUES_GRPH_R 0x422e
+#define mmDCP5_PRESCALE_VALUES_GRPH_R 0x442e
+#define mmPRESCALE_VALUES_GRPH_G 0x1a2f
+#define mmDCP0_PRESCALE_VALUES_GRPH_G 0x1a2f
+#define mmDCP1_PRESCALE_VALUES_GRPH_G 0x1c2f
+#define mmDCP2_PRESCALE_VALUES_GRPH_G 0x1e2f
+#define mmDCP3_PRESCALE_VALUES_GRPH_G 0x402f
+#define mmDCP4_PRESCALE_VALUES_GRPH_G 0x422f
+#define mmDCP5_PRESCALE_VALUES_GRPH_G 0x442f
+#define mmPRESCALE_VALUES_GRPH_B 0x1a30
+#define mmDCP0_PRESCALE_VALUES_GRPH_B 0x1a30
+#define mmDCP1_PRESCALE_VALUES_GRPH_B 0x1c30
+#define mmDCP2_PRESCALE_VALUES_GRPH_B 0x1e30
+#define mmDCP3_PRESCALE_VALUES_GRPH_B 0x4030
+#define mmDCP4_PRESCALE_VALUES_GRPH_B 0x4230
+#define mmDCP5_PRESCALE_VALUES_GRPH_B 0x4430
+#define mmINPUT_CSC_CONTROL 0x1a35
+#define mmDCP0_INPUT_CSC_CONTROL 0x1a35
+#define mmDCP1_INPUT_CSC_CONTROL 0x1c35
+#define mmDCP2_INPUT_CSC_CONTROL 0x1e35
+#define mmDCP3_INPUT_CSC_CONTROL 0x4035
+#define mmDCP4_INPUT_CSC_CONTROL 0x4235
+#define mmDCP5_INPUT_CSC_CONTROL 0x4435
+#define mmINPUT_CSC_C11_C12 0x1a36
+#define mmDCP0_INPUT_CSC_C11_C12 0x1a36
+#define mmDCP1_INPUT_CSC_C11_C12 0x1c36
+#define mmDCP2_INPUT_CSC_C11_C12 0x1e36
+#define mmDCP3_INPUT_CSC_C11_C12 0x4036
+#define mmDCP4_INPUT_CSC_C11_C12 0x4236
+#define mmDCP5_INPUT_CSC_C11_C12 0x4436
+#define mmINPUT_CSC_C13_C14 0x1a37
+#define mmDCP0_INPUT_CSC_C13_C14 0x1a37
+#define mmDCP1_INPUT_CSC_C13_C14 0x1c37
+#define mmDCP2_INPUT_CSC_C13_C14 0x1e37
+#define mmDCP3_INPUT_CSC_C13_C14 0x4037
+#define mmDCP4_INPUT_CSC_C13_C14 0x4237
+#define mmDCP5_INPUT_CSC_C13_C14 0x4437
+#define mmINPUT_CSC_C21_C22 0x1a38
+#define mmDCP0_INPUT_CSC_C21_C22 0x1a38
+#define mmDCP1_INPUT_CSC_C21_C22 0x1c38
+#define mmDCP2_INPUT_CSC_C21_C22 0x1e38
+#define mmDCP3_INPUT_CSC_C21_C22 0x4038
+#define mmDCP4_INPUT_CSC_C21_C22 0x4238
+#define mmDCP5_INPUT_CSC_C21_C22 0x4438
+#define mmINPUT_CSC_C23_C24 0x1a39
+#define mmDCP0_INPUT_CSC_C23_C24 0x1a39
+#define mmDCP1_INPUT_CSC_C23_C24 0x1c39
+#define mmDCP2_INPUT_CSC_C23_C24 0x1e39
+#define mmDCP3_INPUT_CSC_C23_C24 0x4039
+#define mmDCP4_INPUT_CSC_C23_C24 0x4239
+#define mmDCP5_INPUT_CSC_C23_C24 0x4439
+#define mmINPUT_CSC_C31_C32 0x1a3a
+#define mmDCP0_INPUT_CSC_C31_C32 0x1a3a
+#define mmDCP1_INPUT_CSC_C31_C32 0x1c3a
+#define mmDCP2_INPUT_CSC_C31_C32 0x1e3a
+#define mmDCP3_INPUT_CSC_C31_C32 0x403a
+#define mmDCP4_INPUT_CSC_C31_C32 0x423a
+#define mmDCP5_INPUT_CSC_C31_C32 0x443a
+#define mmINPUT_CSC_C33_C34 0x1a3b
+#define mmDCP0_INPUT_CSC_C33_C34 0x1a3b
+#define mmDCP1_INPUT_CSC_C33_C34 0x1c3b
+#define mmDCP2_INPUT_CSC_C33_C34 0x1e3b
+#define mmDCP3_INPUT_CSC_C33_C34 0x403b
+#define mmDCP4_INPUT_CSC_C33_C34 0x423b
+#define mmDCP5_INPUT_CSC_C33_C34 0x443b
+#define mmOUTPUT_CSC_CONTROL 0x1a3c
+#define mmDCP0_OUTPUT_CSC_CONTROL 0x1a3c
+#define mmDCP1_OUTPUT_CSC_CONTROL 0x1c3c
+#define mmDCP2_OUTPUT_CSC_CONTROL 0x1e3c
+#define mmDCP3_OUTPUT_CSC_CONTROL 0x403c
+#define mmDCP4_OUTPUT_CSC_CONTROL 0x423c
+#define mmDCP5_OUTPUT_CSC_CONTROL 0x443c
+#define mmOUTPUT_CSC_C11_C12 0x1a3d
+#define mmDCP0_OUTPUT_CSC_C11_C12 0x1a3d
+#define mmDCP1_OUTPUT_CSC_C11_C12 0x1c3d
+#define mmDCP2_OUTPUT_CSC_C11_C12 0x1e3d
+#define mmDCP3_OUTPUT_CSC_C11_C12 0x403d
+#define mmDCP4_OUTPUT_CSC_C11_C12 0x423d
+#define mmDCP5_OUTPUT_CSC_C11_C12 0x443d
+#define mmOUTPUT_CSC_C13_C14 0x1a3e
+#define mmDCP0_OUTPUT_CSC_C13_C14 0x1a3e
+#define mmDCP1_OUTPUT_CSC_C13_C14 0x1c3e
+#define mmDCP2_OUTPUT_CSC_C13_C14 0x1e3e
+#define mmDCP3_OUTPUT_CSC_C13_C14 0x403e
+#define mmDCP4_OUTPUT_CSC_C13_C14 0x423e
+#define mmDCP5_OUTPUT_CSC_C13_C14 0x443e
+#define mmOUTPUT_CSC_C21_C22 0x1a3f
+#define mmDCP0_OUTPUT_CSC_C21_C22 0x1a3f
+#define mmDCP1_OUTPUT_CSC_C21_C22 0x1c3f
+#define mmDCP2_OUTPUT_CSC_C21_C22 0x1e3f
+#define mmDCP3_OUTPUT_CSC_C21_C22 0x403f
+#define mmDCP4_OUTPUT_CSC_C21_C22 0x423f
+#define mmDCP5_OUTPUT_CSC_C21_C22 0x443f
+#define mmOUTPUT_CSC_C23_C24 0x1a40
+#define mmDCP0_OUTPUT_CSC_C23_C24 0x1a40
+#define mmDCP1_OUTPUT_CSC_C23_C24 0x1c40
+#define mmDCP2_OUTPUT_CSC_C23_C24 0x1e40
+#define mmDCP3_OUTPUT_CSC_C23_C24 0x4040
+#define mmDCP4_OUTPUT_CSC_C23_C24 0x4240
+#define mmDCP5_OUTPUT_CSC_C23_C24 0x4440
+#define mmOUTPUT_CSC_C31_C32 0x1a41
+#define mmDCP0_OUTPUT_CSC_C31_C32 0x1a41
+#define mmDCP1_OUTPUT_CSC_C31_C32 0x1c41
+#define mmDCP2_OUTPUT_CSC_C31_C32 0x1e41
+#define mmDCP3_OUTPUT_CSC_C31_C32 0x4041
+#define mmDCP4_OUTPUT_CSC_C31_C32 0x4241
+#define mmDCP5_OUTPUT_CSC_C31_C32 0x4441
+#define mmOUTPUT_CSC_C33_C34 0x1a42
+#define mmDCP0_OUTPUT_CSC_C33_C34 0x1a42
+#define mmDCP1_OUTPUT_CSC_C33_C34 0x1c42
+#define mmDCP2_OUTPUT_CSC_C33_C34 0x1e42
+#define mmDCP3_OUTPUT_CSC_C33_C34 0x4042
+#define mmDCP4_OUTPUT_CSC_C33_C34 0x4242
+#define mmDCP5_OUTPUT_CSC_C33_C34 0x4442
+#define mmCOMM_MATRIXA_TRANS_C11_C12 0x1a43
+#define mmDCP0_COMM_MATRIXA_TRANS_C11_C12 0x1a43
+#define mmDCP1_COMM_MATRIXA_TRANS_C11_C12 0x1c43
+#define mmDCP2_COMM_MATRIXA_TRANS_C11_C12 0x1e43
+#define mmDCP3_COMM_MATRIXA_TRANS_C11_C12 0x4043
+#define mmDCP4_COMM_MATRIXA_TRANS_C11_C12 0x4243
+#define mmDCP5_COMM_MATRIXA_TRANS_C11_C12 0x4443
+#define mmCOMM_MATRIXA_TRANS_C13_C14 0x1a44
+#define mmDCP0_COMM_MATRIXA_TRANS_C13_C14 0x1a44
+#define mmDCP1_COMM_MATRIXA_TRANS_C13_C14 0x1c44
+#define mmDCP2_COMM_MATRIXA_TRANS_C13_C14 0x1e44
+#define mmDCP3_COMM_MATRIXA_TRANS_C13_C14 0x4044
+#define mmDCP4_COMM_MATRIXA_TRANS_C13_C14 0x4244
+#define mmDCP5_COMM_MATRIXA_TRANS_C13_C14 0x4444
+#define mmCOMM_MATRIXA_TRANS_C21_C22 0x1a45
+#define mmDCP0_COMM_MATRIXA_TRANS_C21_C22 0x1a45
+#define mmDCP1_COMM_MATRIXA_TRANS_C21_C22 0x1c45
+#define mmDCP2_COMM_MATRIXA_TRANS_C21_C22 0x1e45
+#define mmDCP3_COMM_MATRIXA_TRANS_C21_C22 0x4045
+#define mmDCP4_COMM_MATRIXA_TRANS_C21_C22 0x4245
+#define mmDCP5_COMM_MATRIXA_TRANS_C21_C22 0x4445
+#define mmCOMM_MATRIXA_TRANS_C23_C24 0x1a46
+#define mmDCP0_COMM_MATRIXA_TRANS_C23_C24 0x1a46
+#define mmDCP1_COMM_MATRIXA_TRANS_C23_C24 0x1c46
+#define mmDCP2_COMM_MATRIXA_TRANS_C23_C24 0x1e46
+#define mmDCP3_COMM_MATRIXA_TRANS_C23_C24 0x4046
+#define mmDCP4_COMM_MATRIXA_TRANS_C23_C24 0x4246
+#define mmDCP5_COMM_MATRIXA_TRANS_C23_C24 0x4446
+#define mmCOMM_MATRIXA_TRANS_C31_C32 0x1a47
+#define mmDCP0_COMM_MATRIXA_TRANS_C31_C32 0x1a47
+#define mmDCP1_COMM_MATRIXA_TRANS_C31_C32 0x1c47
+#define mmDCP2_COMM_MATRIXA_TRANS_C31_C32 0x1e47
+#define mmDCP3_COMM_MATRIXA_TRANS_C31_C32 0x4047
+#define mmDCP4_COMM_MATRIXA_TRANS_C31_C32 0x4247
+#define mmDCP5_COMM_MATRIXA_TRANS_C31_C32 0x4447
+#define mmCOMM_MATRIXA_TRANS_C33_C34 0x1a48
+#define mmDCP0_COMM_MATRIXA_TRANS_C33_C34 0x1a48
+#define mmDCP1_COMM_MATRIXA_TRANS_C33_C34 0x1c48
+#define mmDCP2_COMM_MATRIXA_TRANS_C33_C34 0x1e48
+#define mmDCP3_COMM_MATRIXA_TRANS_C33_C34 0x4048
+#define mmDCP4_COMM_MATRIXA_TRANS_C33_C34 0x4248
+#define mmDCP5_COMM_MATRIXA_TRANS_C33_C34 0x4448
+#define mmCOMM_MATRIXB_TRANS_C11_C12 0x1a49
+#define mmDCP0_COMM_MATRIXB_TRANS_C11_C12 0x1a49
+#define mmDCP1_COMM_MATRIXB_TRANS_C11_C12 0x1c49
+#define mmDCP2_COMM_MATRIXB_TRANS_C11_C12 0x1e49
+#define mmDCP3_COMM_MATRIXB_TRANS_C11_C12 0x4049
+#define mmDCP4_COMM_MATRIXB_TRANS_C11_C12 0x4249
+#define mmDCP5_COMM_MATRIXB_TRANS_C11_C12 0x4449
+#define mmCOMM_MATRIXB_TRANS_C13_C14 0x1a4a
+#define mmDCP0_COMM_MATRIXB_TRANS_C13_C14 0x1a4a
+#define mmDCP1_COMM_MATRIXB_TRANS_C13_C14 0x1c4a
+#define mmDCP2_COMM_MATRIXB_TRANS_C13_C14 0x1e4a
+#define mmDCP3_COMM_MATRIXB_TRANS_C13_C14 0x404a
+#define mmDCP4_COMM_MATRIXB_TRANS_C13_C14 0x424a
+#define mmDCP5_COMM_MATRIXB_TRANS_C13_C14 0x444a
+#define mmCOMM_MATRIXB_TRANS_C21_C22 0x1a4b
+#define mmDCP0_COMM_MATRIXB_TRANS_C21_C22 0x1a4b
+#define mmDCP1_COMM_MATRIXB_TRANS_C21_C22 0x1c4b
+#define mmDCP2_COMM_MATRIXB_TRANS_C21_C22 0x1e4b
+#define mmDCP3_COMM_MATRIXB_TRANS_C21_C22 0x404b
+#define mmDCP4_COMM_MATRIXB_TRANS_C21_C22 0x424b
+#define mmDCP5_COMM_MATRIXB_TRANS_C21_C22 0x444b
+#define mmCOMM_MATRIXB_TRANS_C23_C24 0x1a4c
+#define mmDCP0_COMM_MATRIXB_TRANS_C23_C24 0x1a4c
+#define mmDCP1_COMM_MATRIXB_TRANS_C23_C24 0x1c4c
+#define mmDCP2_COMM_MATRIXB_TRANS_C23_C24 0x1e4c
+#define mmDCP3_COMM_MATRIXB_TRANS_C23_C24 0x404c
+#define mmDCP4_COMM_MATRIXB_TRANS_C23_C24 0x424c
+#define mmDCP5_COMM_MATRIXB_TRANS_C23_C24 0x444c
+#define mmCOMM_MATRIXB_TRANS_C31_C32 0x1a4d
+#define mmDCP0_COMM_MATRIXB_TRANS_C31_C32 0x1a4d
+#define mmDCP1_COMM_MATRIXB_TRANS_C31_C32 0x1c4d
+#define mmDCP2_COMM_MATRIXB_TRANS_C31_C32 0x1e4d
+#define mmDCP3_COMM_MATRIXB_TRANS_C31_C32 0x404d
+#define mmDCP4_COMM_MATRIXB_TRANS_C31_C32 0x424d
+#define mmDCP5_COMM_MATRIXB_TRANS_C31_C32 0x444d
+#define mmCOMM_MATRIXB_TRANS_C33_C34 0x1a4e
+#define mmDCP0_COMM_MATRIXB_TRANS_C33_C34 0x1a4e
+#define mmDCP1_COMM_MATRIXB_TRANS_C33_C34 0x1c4e
+#define mmDCP2_COMM_MATRIXB_TRANS_C33_C34 0x1e4e
+#define mmDCP3_COMM_MATRIXB_TRANS_C33_C34 0x404e
+#define mmDCP4_COMM_MATRIXB_TRANS_C33_C34 0x424e
+#define mmDCP5_COMM_MATRIXB_TRANS_C33_C34 0x444e
+#define mmDENORM_CONTROL 0x1a50
+#define mmDCP0_DENORM_CONTROL 0x1a50
+#define mmDCP1_DENORM_CONTROL 0x1c50
+#define mmDCP2_DENORM_CONTROL 0x1e50
+#define mmDCP3_DENORM_CONTROL 0x4050
+#define mmDCP4_DENORM_CONTROL 0x4250
+#define mmDCP5_DENORM_CONTROL 0x4450
+#define mmOUT_ROUND_CONTROL 0x1a51
+#define mmDCP0_OUT_ROUND_CONTROL 0x1a51
+#define mmDCP1_OUT_ROUND_CONTROL 0x1c51
+#define mmDCP2_OUT_ROUND_CONTROL 0x1e51
+#define mmDCP3_OUT_ROUND_CONTROL 0x4051
+#define mmDCP4_OUT_ROUND_CONTROL 0x4251
+#define mmDCP5_OUT_ROUND_CONTROL 0x4451
+#define mmOUT_CLAMP_CONTROL_R_CR 0x1a52
+#define mmDCP0_OUT_CLAMP_CONTROL_R_CR 0x1a52
+#define mmDCP1_OUT_CLAMP_CONTROL_R_CR 0x1c52
+#define mmDCP2_OUT_CLAMP_CONTROL_R_CR 0x1e52
+#define mmDCP3_OUT_CLAMP_CONTROL_R_CR 0x4052
+#define mmDCP4_OUT_CLAMP_CONTROL_R_CR 0x4252
+#define mmDCP5_OUT_CLAMP_CONTROL_R_CR 0x4452
+#define mmOUT_CLAMP_CONTROL_G_Y 0x1a9c
+#define mmDCP0_OUT_CLAMP_CONTROL_G_Y 0x1a9c
+#define mmDCP1_OUT_CLAMP_CONTROL_G_Y 0x1c9c
+#define mmDCP2_OUT_CLAMP_CONTROL_G_Y 0x1e9c
+#define mmDCP3_OUT_CLAMP_CONTROL_G_Y 0x409c
+#define mmDCP4_OUT_CLAMP_CONTROL_G_Y 0x429c
+#define mmDCP5_OUT_CLAMP_CONTROL_G_Y 0x449c
+#define mmOUT_CLAMP_CONTROL_B_CB 0x1a9d
+#define mmDCP0_OUT_CLAMP_CONTROL_B_CB 0x1a9d
+#define mmDCP1_OUT_CLAMP_CONTROL_B_CB 0x1c9d
+#define mmDCP2_OUT_CLAMP_CONTROL_B_CB 0x1e9d
+#define mmDCP3_OUT_CLAMP_CONTROL_B_CB 0x409d
+#define mmDCP4_OUT_CLAMP_CONTROL_B_CB 0x429d
+#define mmDCP5_OUT_CLAMP_CONTROL_B_CB 0x449d
+#define mmKEY_CONTROL 0x1a53
+#define mmDCP0_KEY_CONTROL 0x1a53
+#define mmDCP1_KEY_CONTROL 0x1c53
+#define mmDCP2_KEY_CONTROL 0x1e53
+#define mmDCP3_KEY_CONTROL 0x4053
+#define mmDCP4_KEY_CONTROL 0x4253
+#define mmDCP5_KEY_CONTROL 0x4453
+#define mmKEY_RANGE_ALPHA 0x1a54
+#define mmDCP0_KEY_RANGE_ALPHA 0x1a54
+#define mmDCP1_KEY_RANGE_ALPHA 0x1c54
+#define mmDCP2_KEY_RANGE_ALPHA 0x1e54
+#define mmDCP3_KEY_RANGE_ALPHA 0x4054
+#define mmDCP4_KEY_RANGE_ALPHA 0x4254
+#define mmDCP5_KEY_RANGE_ALPHA 0x4454
+#define mmKEY_RANGE_RED 0x1a55
+#define mmDCP0_KEY_RANGE_RED 0x1a55
+#define mmDCP1_KEY_RANGE_RED 0x1c55
+#define mmDCP2_KEY_RANGE_RED 0x1e55
+#define mmDCP3_KEY_RANGE_RED 0x4055
+#define mmDCP4_KEY_RANGE_RED 0x4255
+#define mmDCP5_KEY_RANGE_RED 0x4455
+#define mmKEY_RANGE_GREEN 0x1a56
+#define mmDCP0_KEY_RANGE_GREEN 0x1a56
+#define mmDCP1_KEY_RANGE_GREEN 0x1c56
+#define mmDCP2_KEY_RANGE_GREEN 0x1e56
+#define mmDCP3_KEY_RANGE_GREEN 0x4056
+#define mmDCP4_KEY_RANGE_GREEN 0x4256
+#define mmDCP5_KEY_RANGE_GREEN 0x4456
+#define mmKEY_RANGE_BLUE 0x1a57
+#define mmDCP0_KEY_RANGE_BLUE 0x1a57
+#define mmDCP1_KEY_RANGE_BLUE 0x1c57
+#define mmDCP2_KEY_RANGE_BLUE 0x1e57
+#define mmDCP3_KEY_RANGE_BLUE 0x4057
+#define mmDCP4_KEY_RANGE_BLUE 0x4257
+#define mmDCP5_KEY_RANGE_BLUE 0x4457
+#define mmDEGAMMA_CONTROL 0x1a58
+#define mmDCP0_DEGAMMA_CONTROL 0x1a58
+#define mmDCP1_DEGAMMA_CONTROL 0x1c58
+#define mmDCP2_DEGAMMA_CONTROL 0x1e58
+#define mmDCP3_DEGAMMA_CONTROL 0x4058
+#define mmDCP4_DEGAMMA_CONTROL 0x4258
+#define mmDCP5_DEGAMMA_CONTROL 0x4458
+#define mmGAMUT_REMAP_CONTROL 0x1a59
+#define mmDCP0_GAMUT_REMAP_CONTROL 0x1a59
+#define mmDCP1_GAMUT_REMAP_CONTROL 0x1c59
+#define mmDCP2_GAMUT_REMAP_CONTROL 0x1e59
+#define mmDCP3_GAMUT_REMAP_CONTROL 0x4059
+#define mmDCP4_GAMUT_REMAP_CONTROL 0x4259
+#define mmDCP5_GAMUT_REMAP_CONTROL 0x4459
+#define mmGAMUT_REMAP_C11_C12 0x1a5a
+#define mmDCP0_GAMUT_REMAP_C11_C12 0x1a5a
+#define mmDCP1_GAMUT_REMAP_C11_C12 0x1c5a
+#define mmDCP2_GAMUT_REMAP_C11_C12 0x1e5a
+#define mmDCP3_GAMUT_REMAP_C11_C12 0x405a
+#define mmDCP4_GAMUT_REMAP_C11_C12 0x425a
+#define mmDCP5_GAMUT_REMAP_C11_C12 0x445a
+#define mmGAMUT_REMAP_C13_C14 0x1a5b
+#define mmDCP0_GAMUT_REMAP_C13_C14 0x1a5b
+#define mmDCP1_GAMUT_REMAP_C13_C14 0x1c5b
+#define mmDCP2_GAMUT_REMAP_C13_C14 0x1e5b
+#define mmDCP3_GAMUT_REMAP_C13_C14 0x405b
+#define mmDCP4_GAMUT_REMAP_C13_C14 0x425b
+#define mmDCP5_GAMUT_REMAP_C13_C14 0x445b
+#define mmGAMUT_REMAP_C21_C22 0x1a5c
+#define mmDCP0_GAMUT_REMAP_C21_C22 0x1a5c
+#define mmDCP1_GAMUT_REMAP_C21_C22 0x1c5c
+#define mmDCP2_GAMUT_REMAP_C21_C22 0x1e5c
+#define mmDCP3_GAMUT_REMAP_C21_C22 0x405c
+#define mmDCP4_GAMUT_REMAP_C21_C22 0x425c
+#define mmDCP5_GAMUT_REMAP_C21_C22 0x445c
+#define mmGAMUT_REMAP_C23_C24 0x1a5d
+#define mmDCP0_GAMUT_REMAP_C23_C24 0x1a5d
+#define mmDCP1_GAMUT_REMAP_C23_C24 0x1c5d
+#define mmDCP2_GAMUT_REMAP_C23_C24 0x1e5d
+#define mmDCP3_GAMUT_REMAP_C23_C24 0x405d
+#define mmDCP4_GAMUT_REMAP_C23_C24 0x425d
+#define mmDCP5_GAMUT_REMAP_C23_C24 0x445d
+#define mmGAMUT_REMAP_C31_C32 0x1a5e
+#define mmDCP0_GAMUT_REMAP_C31_C32 0x1a5e
+#define mmDCP1_GAMUT_REMAP_C31_C32 0x1c5e
+#define mmDCP2_GAMUT_REMAP_C31_C32 0x1e5e
+#define mmDCP3_GAMUT_REMAP_C31_C32 0x405e
+#define mmDCP4_GAMUT_REMAP_C31_C32 0x425e
+#define mmDCP5_GAMUT_REMAP_C31_C32 0x445e
+#define mmGAMUT_REMAP_C33_C34 0x1a5f
+#define mmDCP0_GAMUT_REMAP_C33_C34 0x1a5f
+#define mmDCP1_GAMUT_REMAP_C33_C34 0x1c5f
+#define mmDCP2_GAMUT_REMAP_C33_C34 0x1e5f
+#define mmDCP3_GAMUT_REMAP_C33_C34 0x405f
+#define mmDCP4_GAMUT_REMAP_C33_C34 0x425f
+#define mmDCP5_GAMUT_REMAP_C33_C34 0x445f
+#define mmDCP_SPATIAL_DITHER_CNTL 0x1a60
+#define mmDCP0_DCP_SPATIAL_DITHER_CNTL 0x1a60
+#define mmDCP1_DCP_SPATIAL_DITHER_CNTL 0x1c60
+#define mmDCP2_DCP_SPATIAL_DITHER_CNTL 0x1e60
+#define mmDCP3_DCP_SPATIAL_DITHER_CNTL 0x4060
+#define mmDCP4_DCP_SPATIAL_DITHER_CNTL 0x4260
+#define mmDCP5_DCP_SPATIAL_DITHER_CNTL 0x4460
+#define mmDCP_RANDOM_SEEDS 0x1a61
+#define mmDCP0_DCP_RANDOM_SEEDS 0x1a61
+#define mmDCP1_DCP_RANDOM_SEEDS 0x1c61
+#define mmDCP2_DCP_RANDOM_SEEDS 0x1e61
+#define mmDCP3_DCP_RANDOM_SEEDS 0x4061
+#define mmDCP4_DCP_RANDOM_SEEDS 0x4261
+#define mmDCP5_DCP_RANDOM_SEEDS 0x4461
+#define mmDCP_FP_CONVERTED_FIELD 0x1a65
+#define mmDCP0_DCP_FP_CONVERTED_FIELD 0x1a65
+#define mmDCP1_DCP_FP_CONVERTED_FIELD 0x1c65
+#define mmDCP2_DCP_FP_CONVERTED_FIELD 0x1e65
+#define mmDCP3_DCP_FP_CONVERTED_FIELD 0x4065
+#define mmDCP4_DCP_FP_CONVERTED_FIELD 0x4265
+#define mmDCP5_DCP_FP_CONVERTED_FIELD 0x4465
+#define mmCUR_CONTROL 0x1a66
+#define mmDCP0_CUR_CONTROL 0x1a66
+#define mmDCP1_CUR_CONTROL 0x1c66
+#define mmDCP2_CUR_CONTROL 0x1e66
+#define mmDCP3_CUR_CONTROL 0x4066
+#define mmDCP4_CUR_CONTROL 0x4266
+#define mmDCP5_CUR_CONTROL 0x4466
+#define mmCUR_SURFACE_ADDRESS 0x1a67
+#define mmDCP0_CUR_SURFACE_ADDRESS 0x1a67
+#define mmDCP1_CUR_SURFACE_ADDRESS 0x1c67
+#define mmDCP2_CUR_SURFACE_ADDRESS 0x1e67
+#define mmDCP3_CUR_SURFACE_ADDRESS 0x4067
+#define mmDCP4_CUR_SURFACE_ADDRESS 0x4267
+#define mmDCP5_CUR_SURFACE_ADDRESS 0x4467
+#define mmCUR_SIZE 0x1a68
+#define mmDCP0_CUR_SIZE 0x1a68
+#define mmDCP1_CUR_SIZE 0x1c68
+#define mmDCP2_CUR_SIZE 0x1e68
+#define mmDCP3_CUR_SIZE 0x4068
+#define mmDCP4_CUR_SIZE 0x4268
+#define mmDCP5_CUR_SIZE 0x4468
+#define mmCUR_SURFACE_ADDRESS_HIGH 0x1a69
+#define mmDCP0_CUR_SURFACE_ADDRESS_HIGH 0x1a69
+#define mmDCP1_CUR_SURFACE_ADDRESS_HIGH 0x1c69
+#define mmDCP2_CUR_SURFACE_ADDRESS_HIGH 0x1e69
+#define mmDCP3_CUR_SURFACE_ADDRESS_HIGH 0x4069
+#define mmDCP4_CUR_SURFACE_ADDRESS_HIGH 0x4269
+#define mmDCP5_CUR_SURFACE_ADDRESS_HIGH 0x4469
+#define mmCUR_POSITION 0x1a6a
+#define mmDCP0_CUR_POSITION 0x1a6a
+#define mmDCP1_CUR_POSITION 0x1c6a
+#define mmDCP2_CUR_POSITION 0x1e6a
+#define mmDCP3_CUR_POSITION 0x406a
+#define mmDCP4_CUR_POSITION 0x426a
+#define mmDCP5_CUR_POSITION 0x446a
+#define mmCUR_HOT_SPOT 0x1a6b
+#define mmDCP0_CUR_HOT_SPOT 0x1a6b
+#define mmDCP1_CUR_HOT_SPOT 0x1c6b
+#define mmDCP2_CUR_HOT_SPOT 0x1e6b
+#define mmDCP3_CUR_HOT_SPOT 0x406b
+#define mmDCP4_CUR_HOT_SPOT 0x426b
+#define mmDCP5_CUR_HOT_SPOT 0x446b
+#define mmCUR_COLOR1 0x1a6c
+#define mmDCP0_CUR_COLOR1 0x1a6c
+#define mmDCP1_CUR_COLOR1 0x1c6c
+#define mmDCP2_CUR_COLOR1 0x1e6c
+#define mmDCP3_CUR_COLOR1 0x406c
+#define mmDCP4_CUR_COLOR1 0x426c
+#define mmDCP5_CUR_COLOR1 0x446c
+#define mmCUR_COLOR2 0x1a6d
+#define mmDCP0_CUR_COLOR2 0x1a6d
+#define mmDCP1_CUR_COLOR2 0x1c6d
+#define mmDCP2_CUR_COLOR2 0x1e6d
+#define mmDCP3_CUR_COLOR2 0x406d
+#define mmDCP4_CUR_COLOR2 0x426d
+#define mmDCP5_CUR_COLOR2 0x446d
+#define mmCUR_UPDATE 0x1a6e
+#define mmDCP0_CUR_UPDATE 0x1a6e
+#define mmDCP1_CUR_UPDATE 0x1c6e
+#define mmDCP2_CUR_UPDATE 0x1e6e
+#define mmDCP3_CUR_UPDATE 0x406e
+#define mmDCP4_CUR_UPDATE 0x426e
+#define mmDCP5_CUR_UPDATE 0x446e
+#define mmCUR_REQUEST_FILTER_CNTL 0x1a99
+#define mmDCP0_CUR_REQUEST_FILTER_CNTL 0x1a99
+#define mmDCP1_CUR_REQUEST_FILTER_CNTL 0x1c99
+#define mmDCP2_CUR_REQUEST_FILTER_CNTL 0x1e99
+#define mmDCP3_CUR_REQUEST_FILTER_CNTL 0x4099
+#define mmDCP4_CUR_REQUEST_FILTER_CNTL 0x4299
+#define mmDCP5_CUR_REQUEST_FILTER_CNTL 0x4499
+#define mmCUR_STEREO_CONTROL 0x1a9a
+#define mmDCP0_CUR_STEREO_CONTROL 0x1a9a
+#define mmDCP1_CUR_STEREO_CONTROL 0x1c9a
+#define mmDCP2_CUR_STEREO_CONTROL 0x1e9a
+#define mmDCP3_CUR_STEREO_CONTROL 0x409a
+#define mmDCP4_CUR_STEREO_CONTROL 0x429a
+#define mmDCP5_CUR_STEREO_CONTROL 0x449a
+#define mmDC_LUT_RW_MODE 0x1a78
+#define mmDCP0_DC_LUT_RW_MODE 0x1a78
+#define mmDCP1_DC_LUT_RW_MODE 0x1c78
+#define mmDCP2_DC_LUT_RW_MODE 0x1e78
+#define mmDCP3_DC_LUT_RW_MODE 0x4078
+#define mmDCP4_DC_LUT_RW_MODE 0x4278
+#define mmDCP5_DC_LUT_RW_MODE 0x4478
+#define mmDC_LUT_RW_INDEX 0x1a79
+#define mmDCP0_DC_LUT_RW_INDEX 0x1a79
+#define mmDCP1_DC_LUT_RW_INDEX 0x1c79
+#define mmDCP2_DC_LUT_RW_INDEX 0x1e79
+#define mmDCP3_DC_LUT_RW_INDEX 0x4079
+#define mmDCP4_DC_LUT_RW_INDEX 0x4279
+#define mmDCP5_DC_LUT_RW_INDEX 0x4479
+#define mmDC_LUT_SEQ_COLOR 0x1a7a
+#define mmDCP0_DC_LUT_SEQ_COLOR 0x1a7a
+#define mmDCP1_DC_LUT_SEQ_COLOR 0x1c7a
+#define mmDCP2_DC_LUT_SEQ_COLOR 0x1e7a
+#define mmDCP3_DC_LUT_SEQ_COLOR 0x407a
+#define mmDCP4_DC_LUT_SEQ_COLOR 0x427a
+#define mmDCP5_DC_LUT_SEQ_COLOR 0x447a
+#define mmDC_LUT_PWL_DATA 0x1a7b
+#define mmDCP0_DC_LUT_PWL_DATA 0x1a7b
+#define mmDCP1_DC_LUT_PWL_DATA 0x1c7b
+#define mmDCP2_DC_LUT_PWL_DATA 0x1e7b
+#define mmDCP3_DC_LUT_PWL_DATA 0x407b
+#define mmDCP4_DC_LUT_PWL_DATA 0x427b
+#define mmDCP5_DC_LUT_PWL_DATA 0x447b
+#define mmDC_LUT_30_COLOR 0x1a7c
+#define mmDCP0_DC_LUT_30_COLOR 0x1a7c
+#define mmDCP1_DC_LUT_30_COLOR 0x1c7c
+#define mmDCP2_DC_LUT_30_COLOR 0x1e7c
+#define mmDCP3_DC_LUT_30_COLOR 0x407c
+#define mmDCP4_DC_LUT_30_COLOR 0x427c
+#define mmDCP5_DC_LUT_30_COLOR 0x447c
+#define mmDC_LUT_VGA_ACCESS_ENABLE 0x1a7d
+#define mmDCP0_DC_LUT_VGA_ACCESS_ENABLE 0x1a7d
+#define mmDCP1_DC_LUT_VGA_ACCESS_ENABLE 0x1c7d
+#define mmDCP2_DC_LUT_VGA_ACCESS_ENABLE 0x1e7d
+#define mmDCP3_DC_LUT_VGA_ACCESS_ENABLE 0x407d
+#define mmDCP4_DC_LUT_VGA_ACCESS_ENABLE 0x427d
+#define mmDCP5_DC_LUT_VGA_ACCESS_ENABLE 0x447d
+#define mmDC_LUT_WRITE_EN_MASK 0x1a7e
+#define mmDCP0_DC_LUT_WRITE_EN_MASK 0x1a7e
+#define mmDCP1_DC_LUT_WRITE_EN_MASK 0x1c7e
+#define mmDCP2_DC_LUT_WRITE_EN_MASK 0x1e7e
+#define mmDCP3_DC_LUT_WRITE_EN_MASK 0x407e
+#define mmDCP4_DC_LUT_WRITE_EN_MASK 0x427e
+#define mmDCP5_DC_LUT_WRITE_EN_MASK 0x447e
+#define mmDC_LUT_AUTOFILL 0x1a7f
+#define mmDCP0_DC_LUT_AUTOFILL 0x1a7f
+#define mmDCP1_DC_LUT_AUTOFILL 0x1c7f
+#define mmDCP2_DC_LUT_AUTOFILL 0x1e7f
+#define mmDCP3_DC_LUT_AUTOFILL 0x407f
+#define mmDCP4_DC_LUT_AUTOFILL 0x427f
+#define mmDCP5_DC_LUT_AUTOFILL 0x447f
+#define mmDC_LUT_CONTROL 0x1a80
+#define mmDCP0_DC_LUT_CONTROL 0x1a80
+#define mmDCP1_DC_LUT_CONTROL 0x1c80
+#define mmDCP2_DC_LUT_CONTROL 0x1e80
+#define mmDCP3_DC_LUT_CONTROL 0x4080
+#define mmDCP4_DC_LUT_CONTROL 0x4280
+#define mmDCP5_DC_LUT_CONTROL 0x4480
+#define mmDC_LUT_BLACK_OFFSET_BLUE 0x1a81
+#define mmDCP0_DC_LUT_BLACK_OFFSET_BLUE 0x1a81
+#define mmDCP1_DC_LUT_BLACK_OFFSET_BLUE 0x1c81
+#define mmDCP2_DC_LUT_BLACK_OFFSET_BLUE 0x1e81
+#define mmDCP3_DC_LUT_BLACK_OFFSET_BLUE 0x4081
+#define mmDCP4_DC_LUT_BLACK_OFFSET_BLUE 0x4281
+#define mmDCP5_DC_LUT_BLACK_OFFSET_BLUE 0x4481
+#define mmDC_LUT_BLACK_OFFSET_GREEN 0x1a82
+#define mmDCP0_DC_LUT_BLACK_OFFSET_GREEN 0x1a82
+#define mmDCP1_DC_LUT_BLACK_OFFSET_GREEN 0x1c82
+#define mmDCP2_DC_LUT_BLACK_OFFSET_GREEN 0x1e82
+#define mmDCP3_DC_LUT_BLACK_OFFSET_GREEN 0x4082
+#define mmDCP4_DC_LUT_BLACK_OFFSET_GREEN 0x4282
+#define mmDCP5_DC_LUT_BLACK_OFFSET_GREEN 0x4482
+#define mmDC_LUT_BLACK_OFFSET_RED 0x1a83
+#define mmDCP0_DC_LUT_BLACK_OFFSET_RED 0x1a83
+#define mmDCP1_DC_LUT_BLACK_OFFSET_RED 0x1c83
+#define mmDCP2_DC_LUT_BLACK_OFFSET_RED 0x1e83
+#define mmDCP3_DC_LUT_BLACK_OFFSET_RED 0x4083
+#define mmDCP4_DC_LUT_BLACK_OFFSET_RED 0x4283
+#define mmDCP5_DC_LUT_BLACK_OFFSET_RED 0x4483
+#define mmDC_LUT_WHITE_OFFSET_BLUE 0x1a84
+#define mmDCP0_DC_LUT_WHITE_OFFSET_BLUE 0x1a84
+#define mmDCP1_DC_LUT_WHITE_OFFSET_BLUE 0x1c84
+#define mmDCP2_DC_LUT_WHITE_OFFSET_BLUE 0x1e84
+#define mmDCP3_DC_LUT_WHITE_OFFSET_BLUE 0x4084
+#define mmDCP4_DC_LUT_WHITE_OFFSET_BLUE 0x4284
+#define mmDCP5_DC_LUT_WHITE_OFFSET_BLUE 0x4484
+#define mmDC_LUT_WHITE_OFFSET_GREEN 0x1a85
+#define mmDCP0_DC_LUT_WHITE_OFFSET_GREEN 0x1a85
+#define mmDCP1_DC_LUT_WHITE_OFFSET_GREEN 0x1c85
+#define mmDCP2_DC_LUT_WHITE_OFFSET_GREEN 0x1e85
+#define mmDCP3_DC_LUT_WHITE_OFFSET_GREEN 0x4085
+#define mmDCP4_DC_LUT_WHITE_OFFSET_GREEN 0x4285
+#define mmDCP5_DC_LUT_WHITE_OFFSET_GREEN 0x4485
+#define mmDC_LUT_WHITE_OFFSET_RED 0x1a86
+#define mmDCP0_DC_LUT_WHITE_OFFSET_RED 0x1a86
+#define mmDCP1_DC_LUT_WHITE_OFFSET_RED 0x1c86
+#define mmDCP2_DC_LUT_WHITE_OFFSET_RED 0x1e86
+#define mmDCP3_DC_LUT_WHITE_OFFSET_RED 0x4086
+#define mmDCP4_DC_LUT_WHITE_OFFSET_RED 0x4286
+#define mmDCP5_DC_LUT_WHITE_OFFSET_RED 0x4486
+#define mmDCP_CRC_CONTROL 0x1a87
+#define mmDCP0_DCP_CRC_CONTROL 0x1a87
+#define mmDCP1_DCP_CRC_CONTROL 0x1c87
+#define mmDCP2_DCP_CRC_CONTROL 0x1e87
+#define mmDCP3_DCP_CRC_CONTROL 0x4087
+#define mmDCP4_DCP_CRC_CONTROL 0x4287
+#define mmDCP5_DCP_CRC_CONTROL 0x4487
+#define mmDCP_CRC_MASK 0x1a88
+#define mmDCP0_DCP_CRC_MASK 0x1a88
+#define mmDCP1_DCP_CRC_MASK 0x1c88
+#define mmDCP2_DCP_CRC_MASK 0x1e88
+#define mmDCP3_DCP_CRC_MASK 0x4088
+#define mmDCP4_DCP_CRC_MASK 0x4288
+#define mmDCP5_DCP_CRC_MASK 0x4488
+#define mmDCP_CRC_CURRENT 0x1a89
+#define mmDCP0_DCP_CRC_CURRENT 0x1a89
+#define mmDCP1_DCP_CRC_CURRENT 0x1c89
+#define mmDCP2_DCP_CRC_CURRENT 0x1e89
+#define mmDCP3_DCP_CRC_CURRENT 0x4089
+#define mmDCP4_DCP_CRC_CURRENT 0x4289
+#define mmDCP5_DCP_CRC_CURRENT 0x4489
+#define mmDVMM_PTE_CONTROL 0x1a8a
+#define mmDCP0_DVMM_PTE_CONTROL 0x1a8a
+#define mmDCP1_DVMM_PTE_CONTROL 0x1c8a
+#define mmDCP2_DVMM_PTE_CONTROL 0x1e8a
+#define mmDCP3_DVMM_PTE_CONTROL 0x408a
+#define mmDCP4_DVMM_PTE_CONTROL 0x428a
+#define mmDCP5_DVMM_PTE_CONTROL 0x448a
+#define mmDCP_CRC_LAST 0x1a8b
+#define mmDCP0_DCP_CRC_LAST 0x1a8b
+#define mmDCP1_DCP_CRC_LAST 0x1c8b
+#define mmDCP2_DCP_CRC_LAST 0x1e8b
+#define mmDCP3_DCP_CRC_LAST 0x408b
+#define mmDCP4_DCP_CRC_LAST 0x428b
+#define mmDCP5_DCP_CRC_LAST 0x448b
+#define mmDCP_DEBUG 0x1a8d
+#define mmDCP0_DCP_DEBUG 0x1a8d
+#define mmDCP1_DCP_DEBUG 0x1c8d
+#define mmDCP2_DCP_DEBUG 0x1e8d
+#define mmDCP3_DCP_DEBUG 0x408d
+#define mmDCP4_DCP_DEBUG 0x428d
+#define mmDCP5_DCP_DEBUG 0x448d
+#define mmGRPH_FLIP_RATE_CNTL 0x1a8e
+#define mmDCP0_GRPH_FLIP_RATE_CNTL 0x1a8e
+#define mmDCP1_GRPH_FLIP_RATE_CNTL 0x1c8e
+#define mmDCP2_GRPH_FLIP_RATE_CNTL 0x1e8e
+#define mmDCP3_GRPH_FLIP_RATE_CNTL 0x408e
+#define mmDCP4_GRPH_FLIP_RATE_CNTL 0x428e
+#define mmDCP5_GRPH_FLIP_RATE_CNTL 0x448e
+#define mmDCP_GSL_CONTROL 0x1a90
+#define mmDCP0_DCP_GSL_CONTROL 0x1a90
+#define mmDCP1_DCP_GSL_CONTROL 0x1c90
+#define mmDCP2_DCP_GSL_CONTROL 0x1e90
+#define mmDCP3_DCP_GSL_CONTROL 0x4090
+#define mmDCP4_DCP_GSL_CONTROL 0x4290
+#define mmDCP5_DCP_GSL_CONTROL 0x4490
+#define mmDCP_LB_DATA_GAP_BETWEEN_CHUNK 0x1a91
+#define mmDCP0_DCP_LB_DATA_GAP_BETWEEN_CHUNK 0x1a91
+#define mmDCP1_DCP_LB_DATA_GAP_BETWEEN_CHUNK 0x1c91
+#define mmDCP2_DCP_LB_DATA_GAP_BETWEEN_CHUNK 0x1e91
+#define mmDCP3_DCP_LB_DATA_GAP_BETWEEN_CHUNK 0x4091
+#define mmDCP4_DCP_LB_DATA_GAP_BETWEEN_CHUNK 0x4291
+#define mmDCP5_DCP_LB_DATA_GAP_BETWEEN_CHUNK 0x4491
+#define mmDCP_DEBUG_SG 0x1a92
+#define mmDCP0_DCP_DEBUG_SG 0x1a92
+#define mmDCP1_DCP_DEBUG_SG 0x1c92
+#define mmDCP2_DCP_DEBUG_SG 0x1e92
+#define mmDCP3_DCP_DEBUG_SG 0x4092
+#define mmDCP4_DCP_DEBUG_SG 0x4292
+#define mmDCP5_DCP_DEBUG_SG 0x4492
+#define mmDCP_DEBUG_SG2 0x1a94
+#define mmDCP0_DCP_DEBUG_SG2 0x1a94
+#define mmDCP1_DCP_DEBUG_SG2 0x1c94
+#define mmDCP2_DCP_DEBUG_SG2 0x1e94
+#define mmDCP3_DCP_DEBUG_SG2 0x4094
+#define mmDCP4_DCP_DEBUG_SG2 0x4294
+#define mmDCP5_DCP_DEBUG_SG2 0x4494
+#define mmDCP_DVMM_DEBUG 0x1a93
+#define mmDCP0_DCP_DVMM_DEBUG 0x1a93
+#define mmDCP1_DCP_DVMM_DEBUG 0x1c93
+#define mmDCP2_DCP_DVMM_DEBUG 0x1e93
+#define mmDCP3_DCP_DVMM_DEBUG 0x4093
+#define mmDCP4_DCP_DVMM_DEBUG 0x4293
+#define mmDCP5_DCP_DVMM_DEBUG 0x4493
+#define mmDCP_TEST_DEBUG_INDEX 0x1a95
+#define mmDCP0_DCP_TEST_DEBUG_INDEX 0x1a95
+#define mmDCP1_DCP_TEST_DEBUG_INDEX 0x1c95
+#define mmDCP2_DCP_TEST_DEBUG_INDEX 0x1e95
+#define mmDCP3_DCP_TEST_DEBUG_INDEX 0x4095
+#define mmDCP4_DCP_TEST_DEBUG_INDEX 0x4295
+#define mmDCP5_DCP_TEST_DEBUG_INDEX 0x4495
+#define mmDCP_TEST_DEBUG_DATA 0x1a96
+#define mmDCP0_DCP_TEST_DEBUG_DATA 0x1a96
+#define mmDCP1_DCP_TEST_DEBUG_DATA 0x1c96
+#define mmDCP2_DCP_TEST_DEBUG_DATA 0x1e96
+#define mmDCP3_DCP_TEST_DEBUG_DATA 0x4096
+#define mmDCP4_DCP_TEST_DEBUG_DATA 0x4296
+#define mmDCP5_DCP_TEST_DEBUG_DATA 0x4496
+#define mmGRPH_STEREOSYNC_FLIP 0x1a97
+#define mmDCP0_GRPH_STEREOSYNC_FLIP 0x1a97
+#define mmDCP1_GRPH_STEREOSYNC_FLIP 0x1c97
+#define mmDCP2_GRPH_STEREOSYNC_FLIP 0x1e97
+#define mmDCP3_GRPH_STEREOSYNC_FLIP 0x4097
+#define mmDCP4_GRPH_STEREOSYNC_FLIP 0x4297
+#define mmDCP5_GRPH_STEREOSYNC_FLIP 0x4497
+#define mmDCP_DEBUG2 0x1a98
+#define mmDCP0_DCP_DEBUG2 0x1a98
+#define mmDCP1_DCP_DEBUG2 0x1c98
+#define mmDCP2_DCP_DEBUG2 0x1e98
+#define mmDCP3_DCP_DEBUG2 0x4098
+#define mmDCP4_DCP_DEBUG2 0x4298
+#define mmDCP5_DCP_DEBUG2 0x4498
+#define mmHW_ROTATION 0x1a9e
+#define mmDCP0_HW_ROTATION 0x1a9e
+#define mmDCP1_HW_ROTATION 0x1c9e
+#define mmDCP2_HW_ROTATION 0x1e9e
+#define mmDCP3_HW_ROTATION 0x409e
+#define mmDCP4_HW_ROTATION 0x429e
+#define mmDCP5_HW_ROTATION 0x449e
+#define mmGRPH_XDMA_CACHE_UNDERFLOW_DET_CNTL 0x1a9f
+#define mmDCP0_GRPH_XDMA_CACHE_UNDERFLOW_DET_CNTL 0x1a9f
+#define mmDCP1_GRPH_XDMA_CACHE_UNDERFLOW_DET_CNTL 0x1c9f
+#define mmDCP2_GRPH_XDMA_CACHE_UNDERFLOW_DET_CNTL 0x1e9f
+#define mmDCP3_GRPH_XDMA_CACHE_UNDERFLOW_DET_CNTL 0x409f
+#define mmDCP4_GRPH_XDMA_CACHE_UNDERFLOW_DET_CNTL 0x429f
+#define mmDCP5_GRPH_XDMA_CACHE_UNDERFLOW_DET_CNTL 0x449f
+#define mmREGAMMA_CONTROL 0x1aa0
+#define mmDCP0_REGAMMA_CONTROL 0x1aa0
+#define mmDCP1_REGAMMA_CONTROL 0x1ca0
+#define mmDCP2_REGAMMA_CONTROL 0x1ea0
+#define mmDCP3_REGAMMA_CONTROL 0x40a0
+#define mmDCP4_REGAMMA_CONTROL 0x42a0
+#define mmDCP5_REGAMMA_CONTROL 0x44a0
+#define mmREGAMMA_LUT_INDEX 0x1aa1
+#define mmDCP0_REGAMMA_LUT_INDEX 0x1aa1
+#define mmDCP1_REGAMMA_LUT_INDEX 0x1ca1
+#define mmDCP2_REGAMMA_LUT_INDEX 0x1ea1
+#define mmDCP3_REGAMMA_LUT_INDEX 0x40a1
+#define mmDCP4_REGAMMA_LUT_INDEX 0x42a1
+#define mmDCP5_REGAMMA_LUT_INDEX 0x44a1
+#define mmREGAMMA_LUT_DATA 0x1aa2
+#define mmDCP0_REGAMMA_LUT_DATA 0x1aa2
+#define mmDCP1_REGAMMA_LUT_DATA 0x1ca2
+#define mmDCP2_REGAMMA_LUT_DATA 0x1ea2
+#define mmDCP3_REGAMMA_LUT_DATA 0x40a2
+#define mmDCP4_REGAMMA_LUT_DATA 0x42a2
+#define mmDCP5_REGAMMA_LUT_DATA 0x44a2
+#define mmREGAMMA_LUT_WRITE_EN_MASK 0x1aa3
+#define mmDCP0_REGAMMA_LUT_WRITE_EN_MASK 0x1aa3
+#define mmDCP1_REGAMMA_LUT_WRITE_EN_MASK 0x1ca3
+#define mmDCP2_REGAMMA_LUT_WRITE_EN_MASK 0x1ea3
+#define mmDCP3_REGAMMA_LUT_WRITE_EN_MASK 0x40a3
+#define mmDCP4_REGAMMA_LUT_WRITE_EN_MASK 0x42a3
+#define mmDCP5_REGAMMA_LUT_WRITE_EN_MASK 0x44a3
+#define mmREGAMMA_CNTLA_START_CNTL 0x1aa4
+#define mmDCP0_REGAMMA_CNTLA_START_CNTL 0x1aa4
+#define mmDCP1_REGAMMA_CNTLA_START_CNTL 0x1ca4
+#define mmDCP2_REGAMMA_CNTLA_START_CNTL 0x1ea4
+#define mmDCP3_REGAMMA_CNTLA_START_CNTL 0x40a4
+#define mmDCP4_REGAMMA_CNTLA_START_CNTL 0x42a4
+#define mmDCP5_REGAMMA_CNTLA_START_CNTL 0x44a4
+#define mmREGAMMA_CNTLA_SLOPE_CNTL 0x1aa5
+#define mmDCP0_REGAMMA_CNTLA_SLOPE_CNTL 0x1aa5
+#define mmDCP1_REGAMMA_CNTLA_SLOPE_CNTL 0x1ca5
+#define mmDCP2_REGAMMA_CNTLA_SLOPE_CNTL 0x1ea5
+#define mmDCP3_REGAMMA_CNTLA_SLOPE_CNTL 0x40a5
+#define mmDCP4_REGAMMA_CNTLA_SLOPE_CNTL 0x42a5
+#define mmDCP5_REGAMMA_CNTLA_SLOPE_CNTL 0x44a5
+#define mmREGAMMA_CNTLA_END_CNTL1 0x1aa6
+#define mmDCP0_REGAMMA_CNTLA_END_CNTL1 0x1aa6
+#define mmDCP1_REGAMMA_CNTLA_END_CNTL1 0x1ca6
+#define mmDCP2_REGAMMA_CNTLA_END_CNTL1 0x1ea6
+#define mmDCP3_REGAMMA_CNTLA_END_CNTL1 0x40a6
+#define mmDCP4_REGAMMA_CNTLA_END_CNTL1 0x42a6
+#define mmDCP5_REGAMMA_CNTLA_END_CNTL1 0x44a6
+#define mmREGAMMA_CNTLA_END_CNTL2 0x1aa7
+#define mmDCP0_REGAMMA_CNTLA_END_CNTL2 0x1aa7
+#define mmDCP1_REGAMMA_CNTLA_END_CNTL2 0x1ca7
+#define mmDCP2_REGAMMA_CNTLA_END_CNTL2 0x1ea7
+#define mmDCP3_REGAMMA_CNTLA_END_CNTL2 0x40a7
+#define mmDCP4_REGAMMA_CNTLA_END_CNTL2 0x42a7
+#define mmDCP5_REGAMMA_CNTLA_END_CNTL2 0x44a7
+#define mmREGAMMA_CNTLA_REGION_0_1 0x1aa8
+#define mmDCP0_REGAMMA_CNTLA_REGION_0_1 0x1aa8
+#define mmDCP1_REGAMMA_CNTLA_REGION_0_1 0x1ca8
+#define mmDCP2_REGAMMA_CNTLA_REGION_0_1 0x1ea8
+#define mmDCP3_REGAMMA_CNTLA_REGION_0_1 0x40a8
+#define mmDCP4_REGAMMA_CNTLA_REGION_0_1 0x42a8
+#define mmDCP5_REGAMMA_CNTLA_REGION_0_1 0x44a8
+#define mmREGAMMA_CNTLA_REGION_2_3 0x1aa9
+#define mmDCP0_REGAMMA_CNTLA_REGION_2_3 0x1aa9
+#define mmDCP1_REGAMMA_CNTLA_REGION_2_3 0x1ca9
+#define mmDCP2_REGAMMA_CNTLA_REGION_2_3 0x1ea9
+#define mmDCP3_REGAMMA_CNTLA_REGION_2_3 0x40a9
+#define mmDCP4_REGAMMA_CNTLA_REGION_2_3 0x42a9
+#define mmDCP5_REGAMMA_CNTLA_REGION_2_3 0x44a9
+#define mmREGAMMA_CNTLA_REGION_4_5 0x1aaa
+#define mmDCP0_REGAMMA_CNTLA_REGION_4_5 0x1aaa
+#define mmDCP1_REGAMMA_CNTLA_REGION_4_5 0x1caa
+#define mmDCP2_REGAMMA_CNTLA_REGION_4_5 0x1eaa
+#define mmDCP3_REGAMMA_CNTLA_REGION_4_5 0x40aa
+#define mmDCP4_REGAMMA_CNTLA_REGION_4_5 0x42aa
+#define mmDCP5_REGAMMA_CNTLA_REGION_4_5 0x44aa
+#define mmREGAMMA_CNTLA_REGION_6_7 0x1aab
+#define mmDCP0_REGAMMA_CNTLA_REGION_6_7 0x1aab
+#define mmDCP1_REGAMMA_CNTLA_REGION_6_7 0x1cab
+#define mmDCP2_REGAMMA_CNTLA_REGION_6_7 0x1eab
+#define mmDCP3_REGAMMA_CNTLA_REGION_6_7 0x40ab
+#define mmDCP4_REGAMMA_CNTLA_REGION_6_7 0x42ab
+#define mmDCP5_REGAMMA_CNTLA_REGION_6_7 0x44ab
+#define mmREGAMMA_CNTLA_REGION_8_9 0x1aac
+#define mmDCP0_REGAMMA_CNTLA_REGION_8_9 0x1aac
+#define mmDCP1_REGAMMA_CNTLA_REGION_8_9 0x1cac
+#define mmDCP2_REGAMMA_CNTLA_REGION_8_9 0x1eac
+#define mmDCP3_REGAMMA_CNTLA_REGION_8_9 0x40ac
+#define mmDCP4_REGAMMA_CNTLA_REGION_8_9 0x42ac
+#define mmDCP5_REGAMMA_CNTLA_REGION_8_9 0x44ac
+#define mmREGAMMA_CNTLA_REGION_10_11 0x1aad
+#define mmDCP0_REGAMMA_CNTLA_REGION_10_11 0x1aad
+#define mmDCP1_REGAMMA_CNTLA_REGION_10_11 0x1cad
+#define mmDCP2_REGAMMA_CNTLA_REGION_10_11 0x1ead
+#define mmDCP3_REGAMMA_CNTLA_REGION_10_11 0x40ad
+#define mmDCP4_REGAMMA_CNTLA_REGION_10_11 0x42ad
+#define mmDCP5_REGAMMA_CNTLA_REGION_10_11 0x44ad
+#define mmREGAMMA_CNTLA_REGION_12_13 0x1aae
+#define mmDCP0_REGAMMA_CNTLA_REGION_12_13 0x1aae
+#define mmDCP1_REGAMMA_CNTLA_REGION_12_13 0x1cae
+#define mmDCP2_REGAMMA_CNTLA_REGION_12_13 0x1eae
+#define mmDCP3_REGAMMA_CNTLA_REGION_12_13 0x40ae
+#define mmDCP4_REGAMMA_CNTLA_REGION_12_13 0x42ae
+#define mmDCP5_REGAMMA_CNTLA_REGION_12_13 0x44ae
+#define mmREGAMMA_CNTLA_REGION_14_15 0x1aaf
+#define mmDCP0_REGAMMA_CNTLA_REGION_14_15 0x1aaf
+#define mmDCP1_REGAMMA_CNTLA_REGION_14_15 0x1caf
+#define mmDCP2_REGAMMA_CNTLA_REGION_14_15 0x1eaf
+#define mmDCP3_REGAMMA_CNTLA_REGION_14_15 0x40af
+#define mmDCP4_REGAMMA_CNTLA_REGION_14_15 0x42af
+#define mmDCP5_REGAMMA_CNTLA_REGION_14_15 0x44af
+#define mmREGAMMA_CNTLB_START_CNTL 0x1ab0
+#define mmDCP0_REGAMMA_CNTLB_START_CNTL 0x1ab0
+#define mmDCP1_REGAMMA_CNTLB_START_CNTL 0x1cb0
+#define mmDCP2_REGAMMA_CNTLB_START_CNTL 0x1eb0
+#define mmDCP3_REGAMMA_CNTLB_START_CNTL 0x40b0
+#define mmDCP4_REGAMMA_CNTLB_START_CNTL 0x42b0
+#define mmDCP5_REGAMMA_CNTLB_START_CNTL 0x44b0
+#define mmREGAMMA_CNTLB_SLOPE_CNTL 0x1ab1
+#define mmDCP0_REGAMMA_CNTLB_SLOPE_CNTL 0x1ab1
+#define mmDCP1_REGAMMA_CNTLB_SLOPE_CNTL 0x1cb1
+#define mmDCP2_REGAMMA_CNTLB_SLOPE_CNTL 0x1eb1
+#define mmDCP3_REGAMMA_CNTLB_SLOPE_CNTL 0x40b1
+#define mmDCP4_REGAMMA_CNTLB_SLOPE_CNTL 0x42b1
+#define mmDCP5_REGAMMA_CNTLB_SLOPE_CNTL 0x44b1
+#define mmREGAMMA_CNTLB_END_CNTL1 0x1ab2
+#define mmDCP0_REGAMMA_CNTLB_END_CNTL1 0x1ab2
+#define mmDCP1_REGAMMA_CNTLB_END_CNTL1 0x1cb2
+#define mmDCP2_REGAMMA_CNTLB_END_CNTL1 0x1eb2
+#define mmDCP3_REGAMMA_CNTLB_END_CNTL1 0x40b2
+#define mmDCP4_REGAMMA_CNTLB_END_CNTL1 0x42b2
+#define mmDCP5_REGAMMA_CNTLB_END_CNTL1 0x44b2
+#define mmREGAMMA_CNTLB_END_CNTL2 0x1ab3
+#define mmDCP0_REGAMMA_CNTLB_END_CNTL2 0x1ab3
+#define mmDCP1_REGAMMA_CNTLB_END_CNTL2 0x1cb3
+#define mmDCP2_REGAMMA_CNTLB_END_CNTL2 0x1eb3
+#define mmDCP3_REGAMMA_CNTLB_END_CNTL2 0x40b3
+#define mmDCP4_REGAMMA_CNTLB_END_CNTL2 0x42b3
+#define mmDCP5_REGAMMA_CNTLB_END_CNTL2 0x44b3
+#define mmREGAMMA_CNTLB_REGION_0_1 0x1ab4
+#define mmDCP0_REGAMMA_CNTLB_REGION_0_1 0x1ab4
+#define mmDCP1_REGAMMA_CNTLB_REGION_0_1 0x1cb4
+#define mmDCP2_REGAMMA_CNTLB_REGION_0_1 0x1eb4
+#define mmDCP3_REGAMMA_CNTLB_REGION_0_1 0x40b4
+#define mmDCP4_REGAMMA_CNTLB_REGION_0_1 0x42b4
+#define mmDCP5_REGAMMA_CNTLB_REGION_0_1 0x44b4
+#define mmREGAMMA_CNTLB_REGION_2_3 0x1ab5
+#define mmDCP0_REGAMMA_CNTLB_REGION_2_3 0x1ab5
+#define mmDCP1_REGAMMA_CNTLB_REGION_2_3 0x1cb5
+#define mmDCP2_REGAMMA_CNTLB_REGION_2_3 0x1eb5
+#define mmDCP3_REGAMMA_CNTLB_REGION_2_3 0x40b5
+#define mmDCP4_REGAMMA_CNTLB_REGION_2_3 0x42b5
+#define mmDCP5_REGAMMA_CNTLB_REGION_2_3 0x44b5
+#define mmREGAMMA_CNTLB_REGION_4_5 0x1ab6
+#define mmDCP0_REGAMMA_CNTLB_REGION_4_5 0x1ab6
+#define mmDCP1_REGAMMA_CNTLB_REGION_4_5 0x1cb6
+#define mmDCP2_REGAMMA_CNTLB_REGION_4_5 0x1eb6
+#define mmDCP3_REGAMMA_CNTLB_REGION_4_5 0x40b6
+#define mmDCP4_REGAMMA_CNTLB_REGION_4_5 0x42b6
+#define mmDCP5_REGAMMA_CNTLB_REGION_4_5 0x44b6
+#define mmREGAMMA_CNTLB_REGION_6_7 0x1ab7
+#define mmDCP0_REGAMMA_CNTLB_REGION_6_7 0x1ab7
+#define mmDCP1_REGAMMA_CNTLB_REGION_6_7 0x1cb7
+#define mmDCP2_REGAMMA_CNTLB_REGION_6_7 0x1eb7
+#define mmDCP3_REGAMMA_CNTLB_REGION_6_7 0x40b7
+#define mmDCP4_REGAMMA_CNTLB_REGION_6_7 0x42b7
+#define mmDCP5_REGAMMA_CNTLB_REGION_6_7 0x44b7
+#define mmREGAMMA_CNTLB_REGION_8_9 0x1ab8
+#define mmDCP0_REGAMMA_CNTLB_REGION_8_9 0x1ab8
+#define mmDCP1_REGAMMA_CNTLB_REGION_8_9 0x1cb8
+#define mmDCP2_REGAMMA_CNTLB_REGION_8_9 0x1eb8
+#define mmDCP3_REGAMMA_CNTLB_REGION_8_9 0x40b8
+#define mmDCP4_REGAMMA_CNTLB_REGION_8_9 0x42b8
+#define mmDCP5_REGAMMA_CNTLB_REGION_8_9 0x44b8
+#define mmREGAMMA_CNTLB_REGION_10_11 0x1ab9
+#define mmDCP0_REGAMMA_CNTLB_REGION_10_11 0x1ab9
+#define mmDCP1_REGAMMA_CNTLB_REGION_10_11 0x1cb9
+#define mmDCP2_REGAMMA_CNTLB_REGION_10_11 0x1eb9
+#define mmDCP3_REGAMMA_CNTLB_REGION_10_11 0x40b9
+#define mmDCP4_REGAMMA_CNTLB_REGION_10_11 0x42b9
+#define mmDCP5_REGAMMA_CNTLB_REGION_10_11 0x44b9
+#define mmREGAMMA_CNTLB_REGION_12_13 0x1aba
+#define mmDCP0_REGAMMA_CNTLB_REGION_12_13 0x1aba
+#define mmDCP1_REGAMMA_CNTLB_REGION_12_13 0x1cba
+#define mmDCP2_REGAMMA_CNTLB_REGION_12_13 0x1eba
+#define mmDCP3_REGAMMA_CNTLB_REGION_12_13 0x40ba
+#define mmDCP4_REGAMMA_CNTLB_REGION_12_13 0x42ba
+#define mmDCP5_REGAMMA_CNTLB_REGION_12_13 0x44ba
+#define mmREGAMMA_CNTLB_REGION_14_15 0x1abb
+#define mmDCP0_REGAMMA_CNTLB_REGION_14_15 0x1abb
+#define mmDCP1_REGAMMA_CNTLB_REGION_14_15 0x1cbb
+#define mmDCP2_REGAMMA_CNTLB_REGION_14_15 0x1ebb
+#define mmDCP3_REGAMMA_CNTLB_REGION_14_15 0x40bb
+#define mmDCP4_REGAMMA_CNTLB_REGION_14_15 0x42bb
+#define mmDCP5_REGAMMA_CNTLB_REGION_14_15 0x44bb
+#define mmALPHA_CONTROL 0x1abc
+#define mmDCP0_ALPHA_CONTROL 0x1abc
+#define mmDCP1_ALPHA_CONTROL 0x1cbc
+#define mmDCP2_ALPHA_CONTROL 0x1ebc
+#define mmDCP3_ALPHA_CONTROL 0x40bc
+#define mmDCP4_ALPHA_CONTROL 0x42bc
+#define mmDCP5_ALPHA_CONTROL 0x44bc
+#define mmGRPH_XDMA_RECOVERY_SURFACE_ADDRESS 0x1abd
+#define mmDCP0_GRPH_XDMA_RECOVERY_SURFACE_ADDRESS 0x1abd
+#define mmDCP1_GRPH_XDMA_RECOVERY_SURFACE_ADDRESS 0x1cbd
+#define mmDCP2_GRPH_XDMA_RECOVERY_SURFACE_ADDRESS 0x1ebd
+#define mmDCP3_GRPH_XDMA_RECOVERY_SURFACE_ADDRESS 0x40bd
+#define mmDCP4_GRPH_XDMA_RECOVERY_SURFACE_ADDRESS 0x42bd
+#define mmDCP5_GRPH_XDMA_RECOVERY_SURFACE_ADDRESS 0x44bd
+#define mmGRPH_XDMA_RECOVERY_SURFACE_ADDRESS_HIGH 0x1abe
+#define mmDCP0_GRPH_XDMA_RECOVERY_SURFACE_ADDRESS_HIGH 0x1abe
+#define mmDCP1_GRPH_XDMA_RECOVERY_SURFACE_ADDRESS_HIGH 0x1cbe
+#define mmDCP2_GRPH_XDMA_RECOVERY_SURFACE_ADDRESS_HIGH 0x1ebe
+#define mmDCP3_GRPH_XDMA_RECOVERY_SURFACE_ADDRESS_HIGH 0x40be
+#define mmDCP4_GRPH_XDMA_RECOVERY_SURFACE_ADDRESS_HIGH 0x42be
+#define mmDCP5_GRPH_XDMA_RECOVERY_SURFACE_ADDRESS_HIGH 0x44be
+#define mmGRPH_XDMA_CACHE_UNDERFLOW_DET_STATUS 0x1abf
+#define mmDCP0_GRPH_XDMA_CACHE_UNDERFLOW_DET_STATUS 0x1abf
+#define mmDCP1_GRPH_XDMA_CACHE_UNDERFLOW_DET_STATUS 0x1cbf
+#define mmDCP2_GRPH_XDMA_CACHE_UNDERFLOW_DET_STATUS 0x1ebf
+#define mmDCP3_GRPH_XDMA_CACHE_UNDERFLOW_DET_STATUS 0x40bf
+#define mmDCP4_GRPH_XDMA_CACHE_UNDERFLOW_DET_STATUS 0x42bf
+#define mmDCP5_GRPH_XDMA_CACHE_UNDERFLOW_DET_STATUS 0x44bf
+#define mmGRPH_SURFACE_COUNTER_CONTROL 0x1a0f
+#define mmDCP0_GRPH_SURFACE_COUNTER_CONTROL 0x1a0f
+#define mmDCP1_GRPH_SURFACE_COUNTER_CONTROL 0x1c0f
+#define mmDCP2_GRPH_SURFACE_COUNTER_CONTROL 0x1e0f
+#define mmDCP3_GRPH_SURFACE_COUNTER_CONTROL 0x400f
+#define mmDCP4_GRPH_SURFACE_COUNTER_CONTROL 0x420f
+#define mmDCP5_GRPH_SURFACE_COUNTER_CONTROL 0x440f
+#define mmGRPH_SURFACE_COUNTER_OUTPUT 0x1a1d
+#define mmDCP0_GRPH_SURFACE_COUNTER_OUTPUT 0x1a1d
+#define mmDCP1_GRPH_SURFACE_COUNTER_OUTPUT 0x1c1d
+#define mmDCP2_GRPH_SURFACE_COUNTER_OUTPUT 0x1e1d
+#define mmDCP3_GRPH_SURFACE_COUNTER_OUTPUT 0x401d
+#define mmDCP4_GRPH_SURFACE_COUNTER_OUTPUT 0x421d
+#define mmDCP5_GRPH_SURFACE_COUNTER_OUTPUT 0x441d
+#define mmDIG_FE_CNTL 0x4a00
+#define mmDIG0_DIG_FE_CNTL 0x4a00
+#define mmDIG1_DIG_FE_CNTL 0x4b00
+#define mmDIG2_DIG_FE_CNTL 0x4c00
+#define mmDIG3_DIG_FE_CNTL 0x4d00
+#define mmDIG4_DIG_FE_CNTL 0x4e00
+#define mmDIG5_DIG_FE_CNTL 0x4f00
+#define mmDIG6_DIG_FE_CNTL 0x5400
+#define mmDIG7_DIG_FE_CNTL 0x5600
+#define mmDIG8_DIG_FE_CNTL 0x5700
+#define mmDIG_OUTPUT_CRC_CNTL 0x4a01
+#define mmDIG0_DIG_OUTPUT_CRC_CNTL 0x4a01
+#define mmDIG1_DIG_OUTPUT_CRC_CNTL 0x4b01
+#define mmDIG2_DIG_OUTPUT_CRC_CNTL 0x4c01
+#define mmDIG3_DIG_OUTPUT_CRC_CNTL 0x4d01
+#define mmDIG4_DIG_OUTPUT_CRC_CNTL 0x4e01
+#define mmDIG5_DIG_OUTPUT_CRC_CNTL 0x4f01
+#define mmDIG6_DIG_OUTPUT_CRC_CNTL 0x5401
+#define mmDIG7_DIG_OUTPUT_CRC_CNTL 0x5601
+#define mmDIG8_DIG_OUTPUT_CRC_CNTL 0x5701
+#define mmDIG_OUTPUT_CRC_RESULT 0x4a02
+#define mmDIG0_DIG_OUTPUT_CRC_RESULT 0x4a02
+#define mmDIG1_DIG_OUTPUT_CRC_RESULT 0x4b02
+#define mmDIG2_DIG_OUTPUT_CRC_RESULT 0x4c02
+#define mmDIG3_DIG_OUTPUT_CRC_RESULT 0x4d02
+#define mmDIG4_DIG_OUTPUT_CRC_RESULT 0x4e02
+#define mmDIG5_DIG_OUTPUT_CRC_RESULT 0x4f02
+#define mmDIG6_DIG_OUTPUT_CRC_RESULT 0x5402
+#define mmDIG7_DIG_OUTPUT_CRC_RESULT 0x5602
+#define mmDIG8_DIG_OUTPUT_CRC_RESULT 0x5702
+#define mmDIG_CLOCK_PATTERN 0x4a03
+#define mmDIG0_DIG_CLOCK_PATTERN 0x4a03
+#define mmDIG1_DIG_CLOCK_PATTERN 0x4b03
+#define mmDIG2_DIG_CLOCK_PATTERN 0x4c03
+#define mmDIG3_DIG_CLOCK_PATTERN 0x4d03
+#define mmDIG4_DIG_CLOCK_PATTERN 0x4e03
+#define mmDIG5_DIG_CLOCK_PATTERN 0x4f03
+#define mmDIG6_DIG_CLOCK_PATTERN 0x5403
+#define mmDIG7_DIG_CLOCK_PATTERN 0x5603
+#define mmDIG8_DIG_CLOCK_PATTERN 0x5703
+#define mmDIG_TEST_PATTERN 0x4a04
+#define mmDIG0_DIG_TEST_PATTERN 0x4a04
+#define mmDIG1_DIG_TEST_PATTERN 0x4b04
+#define mmDIG2_DIG_TEST_PATTERN 0x4c04
+#define mmDIG3_DIG_TEST_PATTERN 0x4d04
+#define mmDIG4_DIG_TEST_PATTERN 0x4e04
+#define mmDIG5_DIG_TEST_PATTERN 0x4f04
+#define mmDIG6_DIG_TEST_PATTERN 0x5404
+#define mmDIG7_DIG_TEST_PATTERN 0x5604
+#define mmDIG8_DIG_TEST_PATTERN 0x5704
+#define mmDIG_RANDOM_PATTERN_SEED 0x4a05
+#define mmDIG0_DIG_RANDOM_PATTERN_SEED 0x4a05
+#define mmDIG1_DIG_RANDOM_PATTERN_SEED 0x4b05
+#define mmDIG2_DIG_RANDOM_PATTERN_SEED 0x4c05
+#define mmDIG3_DIG_RANDOM_PATTERN_SEED 0x4d05
+#define mmDIG4_DIG_RANDOM_PATTERN_SEED 0x4e05
+#define mmDIG5_DIG_RANDOM_PATTERN_SEED 0x4f05
+#define mmDIG6_DIG_RANDOM_PATTERN_SEED 0x5405
+#define mmDIG7_DIG_RANDOM_PATTERN_SEED 0x5605
+#define mmDIG8_DIG_RANDOM_PATTERN_SEED 0x5705
+#define mmDIG_FIFO_STATUS 0x4a06
+#define mmDIG0_DIG_FIFO_STATUS 0x4a06
+#define mmDIG1_DIG_FIFO_STATUS 0x4b06
+#define mmDIG2_DIG_FIFO_STATUS 0x4c06
+#define mmDIG3_DIG_FIFO_STATUS 0x4d06
+#define mmDIG4_DIG_FIFO_STATUS 0x4e06
+#define mmDIG5_DIG_FIFO_STATUS 0x4f06
+#define mmDIG6_DIG_FIFO_STATUS 0x5406
+#define mmDIG7_DIG_FIFO_STATUS 0x5606
+#define mmDIG8_DIG_FIFO_STATUS 0x5706
+#define mmDIG_DISPCLK_SWITCH_CNTL 0x4a07
+#define mmDIG0_DIG_DISPCLK_SWITCH_CNTL 0x4a07
+#define mmDIG1_DIG_DISPCLK_SWITCH_CNTL 0x4b07
+#define mmDIG2_DIG_DISPCLK_SWITCH_CNTL 0x4c07
+#define mmDIG3_DIG_DISPCLK_SWITCH_CNTL 0x4d07
+#define mmDIG4_DIG_DISPCLK_SWITCH_CNTL 0x4e07
+#define mmDIG5_DIG_DISPCLK_SWITCH_CNTL 0x4f07
+#define mmDIG6_DIG_DISPCLK_SWITCH_CNTL 0x5407
+#define mmDIG7_DIG_DISPCLK_SWITCH_CNTL 0x5607
+#define mmDIG8_DIG_DISPCLK_SWITCH_CNTL 0x5707
+#define mmDIG_DISPCLK_SWITCH_STATUS 0x4a08
+#define mmDIG0_DIG_DISPCLK_SWITCH_STATUS 0x4a08
+#define mmDIG1_DIG_DISPCLK_SWITCH_STATUS 0x4b08
+#define mmDIG2_DIG_DISPCLK_SWITCH_STATUS 0x4c08
+#define mmDIG3_DIG_DISPCLK_SWITCH_STATUS 0x4d08
+#define mmDIG4_DIG_DISPCLK_SWITCH_STATUS 0x4e08
+#define mmDIG5_DIG_DISPCLK_SWITCH_STATUS 0x4f08
+#define mmDIG6_DIG_DISPCLK_SWITCH_STATUS 0x5408
+#define mmDIG7_DIG_DISPCLK_SWITCH_STATUS 0x5608
+#define mmDIG8_DIG_DISPCLK_SWITCH_STATUS 0x5708
+#define mmHDMI_CONTROL 0x4a09
+#define mmDIG0_HDMI_CONTROL 0x4a09
+#define mmDIG1_HDMI_CONTROL 0x4b09
+#define mmDIG2_HDMI_CONTROL 0x4c09
+#define mmDIG3_HDMI_CONTROL 0x4d09
+#define mmDIG4_HDMI_CONTROL 0x4e09
+#define mmDIG5_HDMI_CONTROL 0x4f09
+#define mmDIG6_HDMI_CONTROL 0x5409
+#define mmDIG7_HDMI_CONTROL 0x5609
+#define mmDIG8_HDMI_CONTROL 0x5709
+#define mmHDMI_STATUS 0x4a0a
+#define mmDIG0_HDMI_STATUS 0x4a0a
+#define mmDIG1_HDMI_STATUS 0x4b0a
+#define mmDIG2_HDMI_STATUS 0x4c0a
+#define mmDIG3_HDMI_STATUS 0x4d0a
+#define mmDIG4_HDMI_STATUS 0x4e0a
+#define mmDIG5_HDMI_STATUS 0x4f0a
+#define mmDIG6_HDMI_STATUS 0x540a
+#define mmDIG7_HDMI_STATUS 0x560a
+#define mmDIG8_HDMI_STATUS 0x570a
+#define mmHDMI_AUDIO_PACKET_CONTROL 0x4a0b
+#define mmDIG0_HDMI_AUDIO_PACKET_CONTROL 0x4a0b
+#define mmDIG1_HDMI_AUDIO_PACKET_CONTROL 0x4b0b
+#define mmDIG2_HDMI_AUDIO_PACKET_CONTROL 0x4c0b
+#define mmDIG3_HDMI_AUDIO_PACKET_CONTROL 0x4d0b
+#define mmDIG4_HDMI_AUDIO_PACKET_CONTROL 0x4e0b
+#define mmDIG5_HDMI_AUDIO_PACKET_CONTROL 0x4f0b
+#define mmDIG6_HDMI_AUDIO_PACKET_CONTROL 0x540b
+#define mmDIG7_HDMI_AUDIO_PACKET_CONTROL 0x560b
+#define mmDIG8_HDMI_AUDIO_PACKET_CONTROL 0x570b
+#define mmHDMI_ACR_PACKET_CONTROL 0x4a0c
+#define mmDIG0_HDMI_ACR_PACKET_CONTROL 0x4a0c
+#define mmDIG1_HDMI_ACR_PACKET_CONTROL 0x4b0c
+#define mmDIG2_HDMI_ACR_PACKET_CONTROL 0x4c0c
+#define mmDIG3_HDMI_ACR_PACKET_CONTROL 0x4d0c
+#define mmDIG4_HDMI_ACR_PACKET_CONTROL 0x4e0c
+#define mmDIG5_HDMI_ACR_PACKET_CONTROL 0x4f0c
+#define mmDIG6_HDMI_ACR_PACKET_CONTROL 0x540c
+#define mmDIG7_HDMI_ACR_PACKET_CONTROL 0x560c
+#define mmDIG8_HDMI_ACR_PACKET_CONTROL 0x570c
+#define mmHDMI_VBI_PACKET_CONTROL 0x4a0d
+#define mmDIG0_HDMI_VBI_PACKET_CONTROL 0x4a0d
+#define mmDIG1_HDMI_VBI_PACKET_CONTROL 0x4b0d
+#define mmDIG2_HDMI_VBI_PACKET_CONTROL 0x4c0d
+#define mmDIG3_HDMI_VBI_PACKET_CONTROL 0x4d0d
+#define mmDIG4_HDMI_VBI_PACKET_CONTROL 0x4e0d
+#define mmDIG5_HDMI_VBI_PACKET_CONTROL 0x4f0d
+#define mmDIG6_HDMI_VBI_PACKET_CONTROL 0x540d
+#define mmDIG7_HDMI_VBI_PACKET_CONTROL 0x560d
+#define mmDIG8_HDMI_VBI_PACKET_CONTROL 0x570d
+#define mmHDMI_INFOFRAME_CONTROL0 0x4a0e
+#define mmDIG0_HDMI_INFOFRAME_CONTROL0 0x4a0e
+#define mmDIG1_HDMI_INFOFRAME_CONTROL0 0x4b0e
+#define mmDIG2_HDMI_INFOFRAME_CONTROL0 0x4c0e
+#define mmDIG3_HDMI_INFOFRAME_CONTROL0 0x4d0e
+#define mmDIG4_HDMI_INFOFRAME_CONTROL0 0x4e0e
+#define mmDIG5_HDMI_INFOFRAME_CONTROL0 0x4f0e
+#define mmDIG6_HDMI_INFOFRAME_CONTROL0 0x540e
+#define mmDIG7_HDMI_INFOFRAME_CONTROL0 0x560e
+#define mmDIG8_HDMI_INFOFRAME_CONTROL0 0x570e
+#define mmHDMI_INFOFRAME_CONTROL1 0x4a0f
+#define mmDIG0_HDMI_INFOFRAME_CONTROL1 0x4a0f
+#define mmDIG1_HDMI_INFOFRAME_CONTROL1 0x4b0f
+#define mmDIG2_HDMI_INFOFRAME_CONTROL1 0x4c0f
+#define mmDIG3_HDMI_INFOFRAME_CONTROL1 0x4d0f
+#define mmDIG4_HDMI_INFOFRAME_CONTROL1 0x4e0f
+#define mmDIG5_HDMI_INFOFRAME_CONTROL1 0x4f0f
+#define mmDIG6_HDMI_INFOFRAME_CONTROL1 0x540f
+#define mmDIG7_HDMI_INFOFRAME_CONTROL1 0x560f
+#define mmDIG8_HDMI_INFOFRAME_CONTROL1 0x570f
+#define mmHDMI_GENERIC_PACKET_CONTROL0 0x4a10
+#define mmDIG0_HDMI_GENERIC_PACKET_CONTROL0 0x4a10
+#define mmDIG1_HDMI_GENERIC_PACKET_CONTROL0 0x4b10
+#define mmDIG2_HDMI_GENERIC_PACKET_CONTROL0 0x4c10
+#define mmDIG3_HDMI_GENERIC_PACKET_CONTROL0 0x4d10
+#define mmDIG4_HDMI_GENERIC_PACKET_CONTROL0 0x4e10
+#define mmDIG5_HDMI_GENERIC_PACKET_CONTROL0 0x4f10
+#define mmDIG6_HDMI_GENERIC_PACKET_CONTROL0 0x5410
+#define mmDIG7_HDMI_GENERIC_PACKET_CONTROL0 0x5610
+#define mmDIG8_HDMI_GENERIC_PACKET_CONTROL0 0x5710
+#define mmAFMT_INTERRUPT_STATUS 0x4a11
+#define mmDIG0_AFMT_INTERRUPT_STATUS 0x4a11
+#define mmDIG1_AFMT_INTERRUPT_STATUS 0x4b11
+#define mmDIG2_AFMT_INTERRUPT_STATUS 0x4c11
+#define mmDIG3_AFMT_INTERRUPT_STATUS 0x4d11
+#define mmDIG4_AFMT_INTERRUPT_STATUS 0x4e11
+#define mmDIG5_AFMT_INTERRUPT_STATUS 0x4f11
+#define mmDIG6_AFMT_INTERRUPT_STATUS 0x5411
+#define mmDIG7_AFMT_INTERRUPT_STATUS 0x5611
+#define mmDIG8_AFMT_INTERRUPT_STATUS 0x5711
+#define mmHDMI_GC 0x4a13
+#define mmDIG0_HDMI_GC 0x4a13
+#define mmDIG1_HDMI_GC 0x4b13
+#define mmDIG2_HDMI_GC 0x4c13
+#define mmDIG3_HDMI_GC 0x4d13
+#define mmDIG4_HDMI_GC 0x4e13
+#define mmDIG5_HDMI_GC 0x4f13
+#define mmDIG6_HDMI_GC 0x5413
+#define mmDIG7_HDMI_GC 0x5613
+#define mmDIG8_HDMI_GC 0x5713
+#define mmAFMT_AUDIO_PACKET_CONTROL2 0x4a14
+#define mmDIG0_AFMT_AUDIO_PACKET_CONTROL2 0x4a14
+#define mmDIG1_AFMT_AUDIO_PACKET_CONTROL2 0x4b14
+#define mmDIG2_AFMT_AUDIO_PACKET_CONTROL2 0x4c14
+#define mmDIG3_AFMT_AUDIO_PACKET_CONTROL2 0x4d14
+#define mmDIG4_AFMT_AUDIO_PACKET_CONTROL2 0x4e14
+#define mmDIG5_AFMT_AUDIO_PACKET_CONTROL2 0x4f14
+#define mmDIG6_AFMT_AUDIO_PACKET_CONTROL2 0x5414
+#define mmDIG7_AFMT_AUDIO_PACKET_CONTROL2 0x5614
+#define mmDIG8_AFMT_AUDIO_PACKET_CONTROL2 0x5714
+#define mmAFMT_ISRC1_0 0x4a15
+#define mmDIG0_AFMT_ISRC1_0 0x4a15
+#define mmDIG1_AFMT_ISRC1_0 0x4b15
+#define mmDIG2_AFMT_ISRC1_0 0x4c15
+#define mmDIG3_AFMT_ISRC1_0 0x4d15
+#define mmDIG4_AFMT_ISRC1_0 0x4e15
+#define mmDIG5_AFMT_ISRC1_0 0x4f15
+#define mmDIG6_AFMT_ISRC1_0 0x5415
+#define mmDIG7_AFMT_ISRC1_0 0x5615
+#define mmDIG8_AFMT_ISRC1_0 0x5715
+#define mmAFMT_ISRC1_1 0x4a16
+#define mmDIG0_AFMT_ISRC1_1 0x4a16
+#define mmDIG1_AFMT_ISRC1_1 0x4b16
+#define mmDIG2_AFMT_ISRC1_1 0x4c16
+#define mmDIG3_AFMT_ISRC1_1 0x4d16
+#define mmDIG4_AFMT_ISRC1_1 0x4e16
+#define mmDIG5_AFMT_ISRC1_1 0x4f16
+#define mmDIG6_AFMT_ISRC1_1 0x5416
+#define mmDIG7_AFMT_ISRC1_1 0x5616
+#define mmDIG8_AFMT_ISRC1_1 0x5716
+#define mmAFMT_ISRC1_2 0x4a17
+#define mmDIG0_AFMT_ISRC1_2 0x4a17
+#define mmDIG1_AFMT_ISRC1_2 0x4b17
+#define mmDIG2_AFMT_ISRC1_2 0x4c17
+#define mmDIG3_AFMT_ISRC1_2 0x4d17
+#define mmDIG4_AFMT_ISRC1_2 0x4e17
+#define mmDIG5_AFMT_ISRC1_2 0x4f17
+#define mmDIG6_AFMT_ISRC1_2 0x5417
+#define mmDIG7_AFMT_ISRC1_2 0x5617
+#define mmDIG8_AFMT_ISRC1_2 0x5717
+#define mmAFMT_ISRC1_3 0x4a18
+#define mmDIG0_AFMT_ISRC1_3 0x4a18
+#define mmDIG1_AFMT_ISRC1_3 0x4b18
+#define mmDIG2_AFMT_ISRC1_3 0x4c18
+#define mmDIG3_AFMT_ISRC1_3 0x4d18
+#define mmDIG4_AFMT_ISRC1_3 0x4e18
+#define mmDIG5_AFMT_ISRC1_3 0x4f18
+#define mmDIG6_AFMT_ISRC1_3 0x5418
+#define mmDIG7_AFMT_ISRC1_3 0x5618
+#define mmDIG8_AFMT_ISRC1_3 0x5718
+#define mmAFMT_ISRC1_4 0x4a19
+#define mmDIG0_AFMT_ISRC1_4 0x4a19
+#define mmDIG1_AFMT_ISRC1_4 0x4b19
+#define mmDIG2_AFMT_ISRC1_4 0x4c19
+#define mmDIG3_AFMT_ISRC1_4 0x4d19
+#define mmDIG4_AFMT_ISRC1_4 0x4e19
+#define mmDIG5_AFMT_ISRC1_4 0x4f19
+#define mmDIG6_AFMT_ISRC1_4 0x5419
+#define mmDIG7_AFMT_ISRC1_4 0x5619
+#define mmDIG8_AFMT_ISRC1_4 0x5719
+#define mmAFMT_ISRC2_0 0x4a1a
+#define mmDIG0_AFMT_ISRC2_0 0x4a1a
+#define mmDIG1_AFMT_ISRC2_0 0x4b1a
+#define mmDIG2_AFMT_ISRC2_0 0x4c1a
+#define mmDIG3_AFMT_ISRC2_0 0x4d1a
+#define mmDIG4_AFMT_ISRC2_0 0x4e1a
+#define mmDIG5_AFMT_ISRC2_0 0x4f1a
+#define mmDIG6_AFMT_ISRC2_0 0x541a
+#define mmDIG7_AFMT_ISRC2_0 0x561a
+#define mmDIG8_AFMT_ISRC2_0 0x571a
+#define mmAFMT_ISRC2_1 0x4a1b
+#define mmDIG0_AFMT_ISRC2_1 0x4a1b
+#define mmDIG1_AFMT_ISRC2_1 0x4b1b
+#define mmDIG2_AFMT_ISRC2_1 0x4c1b
+#define mmDIG3_AFMT_ISRC2_1 0x4d1b
+#define mmDIG4_AFMT_ISRC2_1 0x4e1b
+#define mmDIG5_AFMT_ISRC2_1 0x4f1b
+#define mmDIG6_AFMT_ISRC2_1 0x541b
+#define mmDIG7_AFMT_ISRC2_1 0x561b
+#define mmDIG8_AFMT_ISRC2_1 0x571b
+#define mmAFMT_ISRC2_2 0x4a1c
+#define mmDIG0_AFMT_ISRC2_2 0x4a1c
+#define mmDIG1_AFMT_ISRC2_2 0x4b1c
+#define mmDIG2_AFMT_ISRC2_2 0x4c1c
+#define mmDIG3_AFMT_ISRC2_2 0x4d1c
+#define mmDIG4_AFMT_ISRC2_2 0x4e1c
+#define mmDIG5_AFMT_ISRC2_2 0x4f1c
+#define mmDIG6_AFMT_ISRC2_2 0x541c
+#define mmDIG7_AFMT_ISRC2_2 0x561c
+#define mmDIG8_AFMT_ISRC2_2 0x571c
+#define mmAFMT_ISRC2_3 0x4a1d
+#define mmDIG0_AFMT_ISRC2_3 0x4a1d
+#define mmDIG1_AFMT_ISRC2_3 0x4b1d
+#define mmDIG2_AFMT_ISRC2_3 0x4c1d
+#define mmDIG3_AFMT_ISRC2_3 0x4d1d
+#define mmDIG4_AFMT_ISRC2_3 0x4e1d
+#define mmDIG5_AFMT_ISRC2_3 0x4f1d
+#define mmDIG6_AFMT_ISRC2_3 0x541d
+#define mmDIG7_AFMT_ISRC2_3 0x561d
+#define mmDIG8_AFMT_ISRC2_3 0x571d
+#define mmAFMT_AVI_INFO0 0x4a1e
+#define mmDIG0_AFMT_AVI_INFO0 0x4a1e
+#define mmDIG1_AFMT_AVI_INFO0 0x4b1e
+#define mmDIG2_AFMT_AVI_INFO0 0x4c1e
+#define mmDIG3_AFMT_AVI_INFO0 0x4d1e
+#define mmDIG4_AFMT_AVI_INFO0 0x4e1e
+#define mmDIG5_AFMT_AVI_INFO0 0x4f1e
+#define mmDIG6_AFMT_AVI_INFO0 0x541e
+#define mmDIG7_AFMT_AVI_INFO0 0x561e
+#define mmDIG8_AFMT_AVI_INFO0 0x571e
+#define mmAFMT_AVI_INFO1 0x4a1f
+#define mmDIG0_AFMT_AVI_INFO1 0x4a1f
+#define mmDIG1_AFMT_AVI_INFO1 0x4b1f
+#define mmDIG2_AFMT_AVI_INFO1 0x4c1f
+#define mmDIG3_AFMT_AVI_INFO1 0x4d1f
+#define mmDIG4_AFMT_AVI_INFO1 0x4e1f
+#define mmDIG5_AFMT_AVI_INFO1 0x4f1f
+#define mmDIG6_AFMT_AVI_INFO1 0x541f
+#define mmDIG7_AFMT_AVI_INFO1 0x561f
+#define mmDIG8_AFMT_AVI_INFO1 0x571f
+#define mmAFMT_AVI_INFO2 0x4a20
+#define mmDIG0_AFMT_AVI_INFO2 0x4a20
+#define mmDIG1_AFMT_AVI_INFO2 0x4b20
+#define mmDIG2_AFMT_AVI_INFO2 0x4c20
+#define mmDIG3_AFMT_AVI_INFO2 0x4d20
+#define mmDIG4_AFMT_AVI_INFO2 0x4e20
+#define mmDIG5_AFMT_AVI_INFO2 0x4f20
+#define mmDIG6_AFMT_AVI_INFO2 0x5420
+#define mmDIG7_AFMT_AVI_INFO2 0x5620
+#define mmDIG8_AFMT_AVI_INFO2 0x5720
+#define mmAFMT_AVI_INFO3 0x4a21
+#define mmDIG0_AFMT_AVI_INFO3 0x4a21
+#define mmDIG1_AFMT_AVI_INFO3 0x4b21
+#define mmDIG2_AFMT_AVI_INFO3 0x4c21
+#define mmDIG3_AFMT_AVI_INFO3 0x4d21
+#define mmDIG4_AFMT_AVI_INFO3 0x4e21
+#define mmDIG5_AFMT_AVI_INFO3 0x4f21
+#define mmDIG6_AFMT_AVI_INFO3 0x5421
+#define mmDIG7_AFMT_AVI_INFO3 0x5621
+#define mmDIG8_AFMT_AVI_INFO3 0x5721
+#define mmAFMT_MPEG_INFO0 0x4a22
+#define mmDIG0_AFMT_MPEG_INFO0 0x4a22
+#define mmDIG1_AFMT_MPEG_INFO0 0x4b22
+#define mmDIG2_AFMT_MPEG_INFO0 0x4c22
+#define mmDIG3_AFMT_MPEG_INFO0 0x4d22
+#define mmDIG4_AFMT_MPEG_INFO0 0x4e22
+#define mmDIG5_AFMT_MPEG_INFO0 0x4f22
+#define mmDIG6_AFMT_MPEG_INFO0 0x5422
+#define mmDIG7_AFMT_MPEG_INFO0 0x5622
+#define mmDIG8_AFMT_MPEG_INFO0 0x5722
+#define mmAFMT_MPEG_INFO1 0x4a23
+#define mmDIG0_AFMT_MPEG_INFO1 0x4a23
+#define mmDIG1_AFMT_MPEG_INFO1 0x4b23
+#define mmDIG2_AFMT_MPEG_INFO1 0x4c23
+#define mmDIG3_AFMT_MPEG_INFO1 0x4d23
+#define mmDIG4_AFMT_MPEG_INFO1 0x4e23
+#define mmDIG5_AFMT_MPEG_INFO1 0x4f23
+#define mmDIG6_AFMT_MPEG_INFO1 0x5423
+#define mmDIG7_AFMT_MPEG_INFO1 0x5623
+#define mmDIG8_AFMT_MPEG_INFO1 0x5723
+#define mmAFMT_GENERIC_HDR 0x4a24
+#define mmDIG0_AFMT_GENERIC_HDR 0x4a24
+#define mmDIG1_AFMT_GENERIC_HDR 0x4b24
+#define mmDIG2_AFMT_GENERIC_HDR 0x4c24
+#define mmDIG3_AFMT_GENERIC_HDR 0x4d24
+#define mmDIG4_AFMT_GENERIC_HDR 0x4e24
+#define mmDIG5_AFMT_GENERIC_HDR 0x4f24
+#define mmDIG6_AFMT_GENERIC_HDR 0x5424
+#define mmDIG7_AFMT_GENERIC_HDR 0x5624
+#define mmDIG8_AFMT_GENERIC_HDR 0x5724
+#define mmAFMT_GENERIC_0 0x4a25
+#define mmDIG0_AFMT_GENERIC_0 0x4a25
+#define mmDIG1_AFMT_GENERIC_0 0x4b25
+#define mmDIG2_AFMT_GENERIC_0 0x4c25
+#define mmDIG3_AFMT_GENERIC_0 0x4d25
+#define mmDIG4_AFMT_GENERIC_0 0x4e25
+#define mmDIG5_AFMT_GENERIC_0 0x4f25
+#define mmDIG6_AFMT_GENERIC_0 0x5425
+#define mmDIG7_AFMT_GENERIC_0 0x5625
+#define mmDIG8_AFMT_GENERIC_0 0x5725
+#define mmAFMT_GENERIC_1 0x4a26
+#define mmDIG0_AFMT_GENERIC_1 0x4a26
+#define mmDIG1_AFMT_GENERIC_1 0x4b26
+#define mmDIG2_AFMT_GENERIC_1 0x4c26
+#define mmDIG3_AFMT_GENERIC_1 0x4d26
+#define mmDIG4_AFMT_GENERIC_1 0x4e26
+#define mmDIG5_AFMT_GENERIC_1 0x4f26
+#define mmDIG6_AFMT_GENERIC_1 0x5426
+#define mmDIG7_AFMT_GENERIC_1 0x5626
+#define mmDIG8_AFMT_GENERIC_1 0x5726
+#define mmAFMT_GENERIC_2 0x4a27
+#define mmDIG0_AFMT_GENERIC_2 0x4a27
+#define mmDIG1_AFMT_GENERIC_2 0x4b27
+#define mmDIG2_AFMT_GENERIC_2 0x4c27
+#define mmDIG3_AFMT_GENERIC_2 0x4d27
+#define mmDIG4_AFMT_GENERIC_2 0x4e27
+#define mmDIG5_AFMT_GENERIC_2 0x4f27
+#define mmDIG6_AFMT_GENERIC_2 0x5427
+#define mmDIG7_AFMT_GENERIC_2 0x5627
+#define mmDIG8_AFMT_GENERIC_2 0x5727
+#define mmAFMT_GENERIC_3 0x4a28
+#define mmDIG0_AFMT_GENERIC_3 0x4a28
+#define mmDIG1_AFMT_GENERIC_3 0x4b28
+#define mmDIG2_AFMT_GENERIC_3 0x4c28
+#define mmDIG3_AFMT_GENERIC_3 0x4d28
+#define mmDIG4_AFMT_GENERIC_3 0x4e28
+#define mmDIG5_AFMT_GENERIC_3 0x4f28
+#define mmDIG6_AFMT_GENERIC_3 0x5428
+#define mmDIG7_AFMT_GENERIC_3 0x5628
+#define mmDIG8_AFMT_GENERIC_3 0x5728
+#define mmAFMT_GENERIC_4 0x4a29
+#define mmDIG0_AFMT_GENERIC_4 0x4a29
+#define mmDIG1_AFMT_GENERIC_4 0x4b29
+#define mmDIG2_AFMT_GENERIC_4 0x4c29
+#define mmDIG3_AFMT_GENERIC_4 0x4d29
+#define mmDIG4_AFMT_GENERIC_4 0x4e29
+#define mmDIG5_AFMT_GENERIC_4 0x4f29
+#define mmDIG6_AFMT_GENERIC_4 0x5429
+#define mmDIG7_AFMT_GENERIC_4 0x5629
+#define mmDIG8_AFMT_GENERIC_4 0x5729
+#define mmAFMT_GENERIC_5 0x4a2a
+#define mmDIG0_AFMT_GENERIC_5 0x4a2a
+#define mmDIG1_AFMT_GENERIC_5 0x4b2a
+#define mmDIG2_AFMT_GENERIC_5 0x4c2a
+#define mmDIG3_AFMT_GENERIC_5 0x4d2a
+#define mmDIG4_AFMT_GENERIC_5 0x4e2a
+#define mmDIG5_AFMT_GENERIC_5 0x4f2a
+#define mmDIG6_AFMT_GENERIC_5 0x542a
+#define mmDIG7_AFMT_GENERIC_5 0x562a
+#define mmDIG8_AFMT_GENERIC_5 0x572a
+#define mmAFMT_GENERIC_6 0x4a2b
+#define mmDIG0_AFMT_GENERIC_6 0x4a2b
+#define mmDIG1_AFMT_GENERIC_6 0x4b2b
+#define mmDIG2_AFMT_GENERIC_6 0x4c2b
+#define mmDIG3_AFMT_GENERIC_6 0x4d2b
+#define mmDIG4_AFMT_GENERIC_6 0x4e2b
+#define mmDIG5_AFMT_GENERIC_6 0x4f2b
+#define mmDIG6_AFMT_GENERIC_6 0x542b
+#define mmDIG7_AFMT_GENERIC_6 0x562b
+#define mmDIG8_AFMT_GENERIC_6 0x572b
+#define mmAFMT_GENERIC_7 0x4a2c
+#define mmDIG0_AFMT_GENERIC_7 0x4a2c
+#define mmDIG1_AFMT_GENERIC_7 0x4b2c
+#define mmDIG2_AFMT_GENERIC_7 0x4c2c
+#define mmDIG3_AFMT_GENERIC_7 0x4d2c
+#define mmDIG4_AFMT_GENERIC_7 0x4e2c
+#define mmDIG5_AFMT_GENERIC_7 0x4f2c
+#define mmDIG6_AFMT_GENERIC_7 0x542c
+#define mmDIG7_AFMT_GENERIC_7 0x562c
+#define mmDIG8_AFMT_GENERIC_7 0x572c
+#define mmHDMI_GENERIC_PACKET_CONTROL1 0x4a2d
+#define mmDIG0_HDMI_GENERIC_PACKET_CONTROL1 0x4a2d
+#define mmDIG1_HDMI_GENERIC_PACKET_CONTROL1 0x4b2d
+#define mmDIG2_HDMI_GENERIC_PACKET_CONTROL1 0x4c2d
+#define mmDIG3_HDMI_GENERIC_PACKET_CONTROL1 0x4d2d
+#define mmDIG4_HDMI_GENERIC_PACKET_CONTROL1 0x4e2d
+#define mmDIG5_HDMI_GENERIC_PACKET_CONTROL1 0x4f2d
+#define mmDIG6_HDMI_GENERIC_PACKET_CONTROL1 0x542d
+#define mmDIG7_HDMI_GENERIC_PACKET_CONTROL1 0x562d
+#define mmDIG8_HDMI_GENERIC_PACKET_CONTROL1 0x572d
+#define mmHDMI_ACR_32_0 0x4a2e
+#define mmDIG0_HDMI_ACR_32_0 0x4a2e
+#define mmDIG1_HDMI_ACR_32_0 0x4b2e
+#define mmDIG2_HDMI_ACR_32_0 0x4c2e
+#define mmDIG3_HDMI_ACR_32_0 0x4d2e
+#define mmDIG4_HDMI_ACR_32_0 0x4e2e
+#define mmDIG5_HDMI_ACR_32_0 0x4f2e
+#define mmDIG6_HDMI_ACR_32_0 0x542e
+#define mmDIG7_HDMI_ACR_32_0 0x562e
+#define mmDIG8_HDMI_ACR_32_0 0x572e
+#define mmHDMI_ACR_32_1 0x4a2f
+#define mmDIG0_HDMI_ACR_32_1 0x4a2f
+#define mmDIG1_HDMI_ACR_32_1 0x4b2f
+#define mmDIG2_HDMI_ACR_32_1 0x4c2f
+#define mmDIG3_HDMI_ACR_32_1 0x4d2f
+#define mmDIG4_HDMI_ACR_32_1 0x4e2f
+#define mmDIG5_HDMI_ACR_32_1 0x4f2f
+#define mmDIG6_HDMI_ACR_32_1 0x542f
+#define mmDIG7_HDMI_ACR_32_1 0x562f
+#define mmDIG8_HDMI_ACR_32_1 0x572f
+#define mmHDMI_ACR_44_0 0x4a30
+#define mmDIG0_HDMI_ACR_44_0 0x4a30
+#define mmDIG1_HDMI_ACR_44_0 0x4b30
+#define mmDIG2_HDMI_ACR_44_0 0x4c30
+#define mmDIG3_HDMI_ACR_44_0 0x4d30
+#define mmDIG4_HDMI_ACR_44_0 0x4e30
+#define mmDIG5_HDMI_ACR_44_0 0x4f30
+#define mmDIG6_HDMI_ACR_44_0 0x5430
+#define mmDIG7_HDMI_ACR_44_0 0x5630
+#define mmDIG8_HDMI_ACR_44_0 0x5730
+#define mmHDMI_ACR_44_1 0x4a31
+#define mmDIG0_HDMI_ACR_44_1 0x4a31
+#define mmDIG1_HDMI_ACR_44_1 0x4b31
+#define mmDIG2_HDMI_ACR_44_1 0x4c31
+#define mmDIG3_HDMI_ACR_44_1 0x4d31
+#define mmDIG4_HDMI_ACR_44_1 0x4e31
+#define mmDIG5_HDMI_ACR_44_1 0x4f31
+#define mmDIG6_HDMI_ACR_44_1 0x5431
+#define mmDIG7_HDMI_ACR_44_1 0x5631
+#define mmDIG8_HDMI_ACR_44_1 0x5731
+#define mmHDMI_ACR_48_0 0x4a32
+#define mmDIG0_HDMI_ACR_48_0 0x4a32
+#define mmDIG1_HDMI_ACR_48_0 0x4b32
+#define mmDIG2_HDMI_ACR_48_0 0x4c32
+#define mmDIG3_HDMI_ACR_48_0 0x4d32
+#define mmDIG4_HDMI_ACR_48_0 0x4e32
+#define mmDIG5_HDMI_ACR_48_0 0x4f32
+#define mmDIG6_HDMI_ACR_48_0 0x5432
+#define mmDIG7_HDMI_ACR_48_0 0x5632
+#define mmDIG8_HDMI_ACR_48_0 0x5732
+#define mmHDMI_ACR_48_1 0x4a33
+#define mmDIG0_HDMI_ACR_48_1 0x4a33
+#define mmDIG1_HDMI_ACR_48_1 0x4b33
+#define mmDIG2_HDMI_ACR_48_1 0x4c33
+#define mmDIG3_HDMI_ACR_48_1 0x4d33
+#define mmDIG4_HDMI_ACR_48_1 0x4e33
+#define mmDIG5_HDMI_ACR_48_1 0x4f33
+#define mmDIG6_HDMI_ACR_48_1 0x5433
+#define mmDIG7_HDMI_ACR_48_1 0x5633
+#define mmDIG8_HDMI_ACR_48_1 0x5733
+#define mmHDMI_ACR_STATUS_0 0x4a34
+#define mmDIG0_HDMI_ACR_STATUS_0 0x4a34
+#define mmDIG1_HDMI_ACR_STATUS_0 0x4b34
+#define mmDIG2_HDMI_ACR_STATUS_0 0x4c34
+#define mmDIG3_HDMI_ACR_STATUS_0 0x4d34
+#define mmDIG4_HDMI_ACR_STATUS_0 0x4e34
+#define mmDIG5_HDMI_ACR_STATUS_0 0x4f34
+#define mmDIG6_HDMI_ACR_STATUS_0 0x5434
+#define mmDIG7_HDMI_ACR_STATUS_0 0x5634
+#define mmDIG8_HDMI_ACR_STATUS_0 0x5734
+#define mmHDMI_ACR_STATUS_1 0x4a35
+#define mmDIG0_HDMI_ACR_STATUS_1 0x4a35
+#define mmDIG1_HDMI_ACR_STATUS_1 0x4b35
+#define mmDIG2_HDMI_ACR_STATUS_1 0x4c35
+#define mmDIG3_HDMI_ACR_STATUS_1 0x4d35
+#define mmDIG4_HDMI_ACR_STATUS_1 0x4e35
+#define mmDIG5_HDMI_ACR_STATUS_1 0x4f35
+#define mmDIG6_HDMI_ACR_STATUS_1 0x5435
+#define mmDIG7_HDMI_ACR_STATUS_1 0x5635
+#define mmDIG8_HDMI_ACR_STATUS_1 0x5735
+#define mmAFMT_AUDIO_INFO0 0x4a36
+#define mmDIG0_AFMT_AUDIO_INFO0 0x4a36
+#define mmDIG1_AFMT_AUDIO_INFO0 0x4b36
+#define mmDIG2_AFMT_AUDIO_INFO0 0x4c36
+#define mmDIG3_AFMT_AUDIO_INFO0 0x4d36
+#define mmDIG4_AFMT_AUDIO_INFO0 0x4e36
+#define mmDIG5_AFMT_AUDIO_INFO0 0x4f36
+#define mmDIG6_AFMT_AUDIO_INFO0 0x5436
+#define mmDIG7_AFMT_AUDIO_INFO0 0x5636
+#define mmDIG8_AFMT_AUDIO_INFO0 0x5736
+#define mmAFMT_AUDIO_INFO1 0x4a37
+#define mmDIG0_AFMT_AUDIO_INFO1 0x4a37
+#define mmDIG1_AFMT_AUDIO_INFO1 0x4b37
+#define mmDIG2_AFMT_AUDIO_INFO1 0x4c37
+#define mmDIG3_AFMT_AUDIO_INFO1 0x4d37
+#define mmDIG4_AFMT_AUDIO_INFO1 0x4e37
+#define mmDIG5_AFMT_AUDIO_INFO1 0x4f37
+#define mmDIG6_AFMT_AUDIO_INFO1 0x5437
+#define mmDIG7_AFMT_AUDIO_INFO1 0x5637
+#define mmDIG8_AFMT_AUDIO_INFO1 0x5737
+#define mmAFMT_60958_0 0x4a38
+#define mmDIG0_AFMT_60958_0 0x4a38
+#define mmDIG1_AFMT_60958_0 0x4b38
+#define mmDIG2_AFMT_60958_0 0x4c38
+#define mmDIG3_AFMT_60958_0 0x4d38
+#define mmDIG4_AFMT_60958_0 0x4e38
+#define mmDIG5_AFMT_60958_0 0x4f38
+#define mmDIG6_AFMT_60958_0 0x5438
+#define mmDIG7_AFMT_60958_0 0x5638
+#define mmDIG8_AFMT_60958_0 0x5738
+#define mmAFMT_60958_1 0x4a39
+#define mmDIG0_AFMT_60958_1 0x4a39
+#define mmDIG1_AFMT_60958_1 0x4b39
+#define mmDIG2_AFMT_60958_1 0x4c39
+#define mmDIG3_AFMT_60958_1 0x4d39
+#define mmDIG4_AFMT_60958_1 0x4e39
+#define mmDIG5_AFMT_60958_1 0x4f39
+#define mmDIG6_AFMT_60958_1 0x5439
+#define mmDIG7_AFMT_60958_1 0x5639
+#define mmDIG8_AFMT_60958_1 0x5739
+#define mmAFMT_AUDIO_CRC_CONTROL 0x4a3a
+#define mmDIG0_AFMT_AUDIO_CRC_CONTROL 0x4a3a
+#define mmDIG1_AFMT_AUDIO_CRC_CONTROL 0x4b3a
+#define mmDIG2_AFMT_AUDIO_CRC_CONTROL 0x4c3a
+#define mmDIG3_AFMT_AUDIO_CRC_CONTROL 0x4d3a
+#define mmDIG4_AFMT_AUDIO_CRC_CONTROL 0x4e3a
+#define mmDIG5_AFMT_AUDIO_CRC_CONTROL 0x4f3a
+#define mmDIG6_AFMT_AUDIO_CRC_CONTROL 0x543a
+#define mmDIG7_AFMT_AUDIO_CRC_CONTROL 0x563a
+#define mmDIG8_AFMT_AUDIO_CRC_CONTROL 0x573a
+#define mmAFMT_RAMP_CONTROL0 0x4a3b
+#define mmDIG0_AFMT_RAMP_CONTROL0 0x4a3b
+#define mmDIG1_AFMT_RAMP_CONTROL0 0x4b3b
+#define mmDIG2_AFMT_RAMP_CONTROL0 0x4c3b
+#define mmDIG3_AFMT_RAMP_CONTROL0 0x4d3b
+#define mmDIG4_AFMT_RAMP_CONTROL0 0x4e3b
+#define mmDIG5_AFMT_RAMP_CONTROL0 0x4f3b
+#define mmDIG6_AFMT_RAMP_CONTROL0 0x543b
+#define mmDIG7_AFMT_RAMP_CONTROL0 0x563b
+#define mmDIG8_AFMT_RAMP_CONTROL0 0x573b
+#define mmAFMT_RAMP_CONTROL1 0x4a3c
+#define mmDIG0_AFMT_RAMP_CONTROL1 0x4a3c
+#define mmDIG1_AFMT_RAMP_CONTROL1 0x4b3c
+#define mmDIG2_AFMT_RAMP_CONTROL1 0x4c3c
+#define mmDIG3_AFMT_RAMP_CONTROL1 0x4d3c
+#define mmDIG4_AFMT_RAMP_CONTROL1 0x4e3c
+#define mmDIG5_AFMT_RAMP_CONTROL1 0x4f3c
+#define mmDIG6_AFMT_RAMP_CONTROL1 0x543c
+#define mmDIG7_AFMT_RAMP_CONTROL1 0x563c
+#define mmDIG8_AFMT_RAMP_CONTROL1 0x573c
+#define mmAFMT_RAMP_CONTROL2 0x4a3d
+#define mmDIG0_AFMT_RAMP_CONTROL2 0x4a3d
+#define mmDIG1_AFMT_RAMP_CONTROL2 0x4b3d
+#define mmDIG2_AFMT_RAMP_CONTROL2 0x4c3d
+#define mmDIG3_AFMT_RAMP_CONTROL2 0x4d3d
+#define mmDIG4_AFMT_RAMP_CONTROL2 0x4e3d
+#define mmDIG5_AFMT_RAMP_CONTROL2 0x4f3d
+#define mmDIG6_AFMT_RAMP_CONTROL2 0x543d
+#define mmDIG7_AFMT_RAMP_CONTROL2 0x563d
+#define mmDIG8_AFMT_RAMP_CONTROL2 0x573d
+#define mmAFMT_RAMP_CONTROL3 0x4a3e
+#define mmDIG0_AFMT_RAMP_CONTROL3 0x4a3e
+#define mmDIG1_AFMT_RAMP_CONTROL3 0x4b3e
+#define mmDIG2_AFMT_RAMP_CONTROL3 0x4c3e
+#define mmDIG3_AFMT_RAMP_CONTROL3 0x4d3e
+#define mmDIG4_AFMT_RAMP_CONTROL3 0x4e3e
+#define mmDIG5_AFMT_RAMP_CONTROL3 0x4f3e
+#define mmDIG6_AFMT_RAMP_CONTROL3 0x543e
+#define mmDIG7_AFMT_RAMP_CONTROL3 0x563e
+#define mmDIG8_AFMT_RAMP_CONTROL3 0x573e
+#define mmAFMT_60958_2 0x4a3f
+#define mmDIG0_AFMT_60958_2 0x4a3f
+#define mmDIG1_AFMT_60958_2 0x4b3f
+#define mmDIG2_AFMT_60958_2 0x4c3f
+#define mmDIG3_AFMT_60958_2 0x4d3f
+#define mmDIG4_AFMT_60958_2 0x4e3f
+#define mmDIG5_AFMT_60958_2 0x4f3f
+#define mmDIG6_AFMT_60958_2 0x543f
+#define mmDIG7_AFMT_60958_2 0x563f
+#define mmDIG8_AFMT_60958_2 0x573f
+#define mmAFMT_AUDIO_CRC_RESULT 0x4a40
+#define mmDIG0_AFMT_AUDIO_CRC_RESULT 0x4a40
+#define mmDIG1_AFMT_AUDIO_CRC_RESULT 0x4b40
+#define mmDIG2_AFMT_AUDIO_CRC_RESULT 0x4c40
+#define mmDIG3_AFMT_AUDIO_CRC_RESULT 0x4d40
+#define mmDIG4_AFMT_AUDIO_CRC_RESULT 0x4e40
+#define mmDIG5_AFMT_AUDIO_CRC_RESULT 0x4f40
+#define mmDIG6_AFMT_AUDIO_CRC_RESULT 0x5440
+#define mmDIG7_AFMT_AUDIO_CRC_RESULT 0x5640
+#define mmDIG8_AFMT_AUDIO_CRC_RESULT 0x5740
+#define mmAFMT_STATUS 0x4a41
+#define mmDIG0_AFMT_STATUS 0x4a41
+#define mmDIG1_AFMT_STATUS 0x4b41
+#define mmDIG2_AFMT_STATUS 0x4c41
+#define mmDIG3_AFMT_STATUS 0x4d41
+#define mmDIG4_AFMT_STATUS 0x4e41
+#define mmDIG5_AFMT_STATUS 0x4f41
+#define mmDIG6_AFMT_STATUS 0x5441
+#define mmDIG7_AFMT_STATUS 0x5641
+#define mmDIG8_AFMT_STATUS 0x5741
+#define mmAFMT_AUDIO_PACKET_CONTROL 0x4a42
+#define mmDIG0_AFMT_AUDIO_PACKET_CONTROL 0x4a42
+#define mmDIG1_AFMT_AUDIO_PACKET_CONTROL 0x4b42
+#define mmDIG2_AFMT_AUDIO_PACKET_CONTROL 0x4c42
+#define mmDIG3_AFMT_AUDIO_PACKET_CONTROL 0x4d42
+#define mmDIG4_AFMT_AUDIO_PACKET_CONTROL 0x4e42
+#define mmDIG5_AFMT_AUDIO_PACKET_CONTROL 0x4f42
+#define mmDIG6_AFMT_AUDIO_PACKET_CONTROL 0x5442
+#define mmDIG7_AFMT_AUDIO_PACKET_CONTROL 0x5642
+#define mmDIG8_AFMT_AUDIO_PACKET_CONTROL 0x5742
+#define mmAFMT_VBI_PACKET_CONTROL 0x4a43
+#define mmDIG0_AFMT_VBI_PACKET_CONTROL 0x4a43
+#define mmDIG1_AFMT_VBI_PACKET_CONTROL 0x4b43
+#define mmDIG2_AFMT_VBI_PACKET_CONTROL 0x4c43
+#define mmDIG3_AFMT_VBI_PACKET_CONTROL 0x4d43
+#define mmDIG4_AFMT_VBI_PACKET_CONTROL 0x4e43
+#define mmDIG5_AFMT_VBI_PACKET_CONTROL 0x4f43
+#define mmDIG6_AFMT_VBI_PACKET_CONTROL 0x5443
+#define mmDIG7_AFMT_VBI_PACKET_CONTROL 0x5643
+#define mmDIG8_AFMT_VBI_PACKET_CONTROL 0x5743
+#define mmAFMT_INFOFRAME_CONTROL0 0x4a44
+#define mmDIG0_AFMT_INFOFRAME_CONTROL0 0x4a44
+#define mmDIG1_AFMT_INFOFRAME_CONTROL0 0x4b44
+#define mmDIG2_AFMT_INFOFRAME_CONTROL0 0x4c44
+#define mmDIG3_AFMT_INFOFRAME_CONTROL0 0x4d44
+#define mmDIG4_AFMT_INFOFRAME_CONTROL0 0x4e44
+#define mmDIG5_AFMT_INFOFRAME_CONTROL0 0x4f44
+#define mmDIG6_AFMT_INFOFRAME_CONTROL0 0x5444
+#define mmDIG7_AFMT_INFOFRAME_CONTROL0 0x5644
+#define mmDIG8_AFMT_INFOFRAME_CONTROL0 0x5744
+#define mmAFMT_AUDIO_SRC_CONTROL 0x4a45
+#define mmDIG0_AFMT_AUDIO_SRC_CONTROL 0x4a45
+#define mmDIG1_AFMT_AUDIO_SRC_CONTROL 0x4b45
+#define mmDIG2_AFMT_AUDIO_SRC_CONTROL 0x4c45
+#define mmDIG3_AFMT_AUDIO_SRC_CONTROL 0x4d45
+#define mmDIG4_AFMT_AUDIO_SRC_CONTROL 0x4e45
+#define mmDIG5_AFMT_AUDIO_SRC_CONTROL 0x4f45
+#define mmDIG6_AFMT_AUDIO_SRC_CONTROL 0x5445
+#define mmDIG7_AFMT_AUDIO_SRC_CONTROL 0x5645
+#define mmDIG8_AFMT_AUDIO_SRC_CONTROL 0x5745
+#define mmAFMT_AUDIO_DBG_DTO_CNTL 0x4a46
+#define mmDIG0_AFMT_AUDIO_DBG_DTO_CNTL 0x4a46
+#define mmDIG1_AFMT_AUDIO_DBG_DTO_CNTL 0x4b46
+#define mmDIG2_AFMT_AUDIO_DBG_DTO_CNTL 0x4c46
+#define mmDIG3_AFMT_AUDIO_DBG_DTO_CNTL 0x4d46
+#define mmDIG4_AFMT_AUDIO_DBG_DTO_CNTL 0x4e46
+#define mmDIG5_AFMT_AUDIO_DBG_DTO_CNTL 0x4f46
+#define mmDIG6_AFMT_AUDIO_DBG_DTO_CNTL 0x5446
+#define mmDIG7_AFMT_AUDIO_DBG_DTO_CNTL 0x5646
+#define mmDIG8_AFMT_AUDIO_DBG_DTO_CNTL 0x5746
+#define mmAFMT_CNTL 0x4a7e
+#define mmDIG0_AFMT_CNTL 0x4a7e
+#define mmDIG1_AFMT_CNTL 0x4b7e
+#define mmDIG2_AFMT_CNTL 0x4c7e
+#define mmDIG3_AFMT_CNTL 0x4d7e
+#define mmDIG4_AFMT_CNTL 0x4e7e
+#define mmDIG5_AFMT_CNTL 0x4f7e
+#define mmDIG6_AFMT_CNTL 0x547e
+#define mmDIG7_AFMT_CNTL 0x567e
+#define mmDIG8_AFMT_CNTL 0x577e
+#define mmDIG_BE_CNTL 0x4a47
+#define mmDIG0_DIG_BE_CNTL 0x4a47
+#define mmDIG1_DIG_BE_CNTL 0x4b47
+#define mmDIG2_DIG_BE_CNTL 0x4c47
+#define mmDIG3_DIG_BE_CNTL 0x4d47
+#define mmDIG4_DIG_BE_CNTL 0x4e47
+#define mmDIG5_DIG_BE_CNTL 0x4f47
+#define mmDIG6_DIG_BE_CNTL 0x5447
+#define mmDIG7_DIG_BE_CNTL 0x5647
+#define mmDIG8_DIG_BE_CNTL 0x5747
+#define mmDIG_BE_EN_CNTL 0x4a48
+#define mmDIG0_DIG_BE_EN_CNTL 0x4a48
+#define mmDIG1_DIG_BE_EN_CNTL 0x4b48
+#define mmDIG2_DIG_BE_EN_CNTL 0x4c48
+#define mmDIG3_DIG_BE_EN_CNTL 0x4d48
+#define mmDIG4_DIG_BE_EN_CNTL 0x4e48
+#define mmDIG5_DIG_BE_EN_CNTL 0x4f48
+#define mmDIG6_DIG_BE_EN_CNTL 0x5448
+#define mmDIG7_DIG_BE_EN_CNTL 0x5648
+#define mmDIG8_DIG_BE_EN_CNTL 0x5748
+#define mmTMDS_CNTL 0x4a6b
+#define mmDIG0_TMDS_CNTL 0x4a6b
+#define mmDIG1_TMDS_CNTL 0x4b6b
+#define mmDIG2_TMDS_CNTL 0x4c6b
+#define mmDIG3_TMDS_CNTL 0x4d6b
+#define mmDIG4_TMDS_CNTL 0x4e6b
+#define mmDIG5_TMDS_CNTL 0x4f6b
+#define mmDIG6_TMDS_CNTL 0x546b
+#define mmDIG7_TMDS_CNTL 0x566b
+#define mmDIG8_TMDS_CNTL 0x576b
+#define mmTMDS_CONTROL_CHAR 0x4a6c
+#define mmDIG0_TMDS_CONTROL_CHAR 0x4a6c
+#define mmDIG1_TMDS_CONTROL_CHAR 0x4b6c
+#define mmDIG2_TMDS_CONTROL_CHAR 0x4c6c
+#define mmDIG3_TMDS_CONTROL_CHAR 0x4d6c
+#define mmDIG4_TMDS_CONTROL_CHAR 0x4e6c
+#define mmDIG5_TMDS_CONTROL_CHAR 0x4f6c
+#define mmDIG6_TMDS_CONTROL_CHAR 0x546c
+#define mmDIG7_TMDS_CONTROL_CHAR 0x566c
+#define mmDIG8_TMDS_CONTROL_CHAR 0x576c
+#define mmTMDS_CONTROL0_FEEDBACK 0x4a6d
+#define mmDIG0_TMDS_CONTROL0_FEEDBACK 0x4a6d
+#define mmDIG1_TMDS_CONTROL0_FEEDBACK 0x4b6d
+#define mmDIG2_TMDS_CONTROL0_FEEDBACK 0x4c6d
+#define mmDIG3_TMDS_CONTROL0_FEEDBACK 0x4d6d
+#define mmDIG4_TMDS_CONTROL0_FEEDBACK 0x4e6d
+#define mmDIG5_TMDS_CONTROL0_FEEDBACK 0x4f6d
+#define mmDIG6_TMDS_CONTROL0_FEEDBACK 0x546d
+#define mmDIG7_TMDS_CONTROL0_FEEDBACK 0x566d
+#define mmDIG8_TMDS_CONTROL0_FEEDBACK 0x576d
+#define mmTMDS_STEREOSYNC_CTL_SEL 0x4a6e
+#define mmDIG0_TMDS_STEREOSYNC_CTL_SEL 0x4a6e
+#define mmDIG1_TMDS_STEREOSYNC_CTL_SEL 0x4b6e
+#define mmDIG2_TMDS_STEREOSYNC_CTL_SEL 0x4c6e
+#define mmDIG3_TMDS_STEREOSYNC_CTL_SEL 0x4d6e
+#define mmDIG4_TMDS_STEREOSYNC_CTL_SEL 0x4e6e
+#define mmDIG5_TMDS_STEREOSYNC_CTL_SEL 0x4f6e
+#define mmDIG6_TMDS_STEREOSYNC_CTL_SEL 0x546e
+#define mmDIG7_TMDS_STEREOSYNC_CTL_SEL 0x566e
+#define mmDIG8_TMDS_STEREOSYNC_CTL_SEL 0x576e
+#define mmTMDS_SYNC_CHAR_PATTERN_0_1 0x4a6f
+#define mmDIG0_TMDS_SYNC_CHAR_PATTERN_0_1 0x4a6f
+#define mmDIG1_TMDS_SYNC_CHAR_PATTERN_0_1 0x4b6f
+#define mmDIG2_TMDS_SYNC_CHAR_PATTERN_0_1 0x4c6f
+#define mmDIG3_TMDS_SYNC_CHAR_PATTERN_0_1 0x4d6f
+#define mmDIG4_TMDS_SYNC_CHAR_PATTERN_0_1 0x4e6f
+#define mmDIG5_TMDS_SYNC_CHAR_PATTERN_0_1 0x4f6f
+#define mmDIG6_TMDS_SYNC_CHAR_PATTERN_0_1 0x546f
+#define mmDIG7_TMDS_SYNC_CHAR_PATTERN_0_1 0x566f
+#define mmDIG8_TMDS_SYNC_CHAR_PATTERN_0_1 0x576f
+#define mmTMDS_SYNC_CHAR_PATTERN_2_3 0x4a70
+#define mmDIG0_TMDS_SYNC_CHAR_PATTERN_2_3 0x4a70
+#define mmDIG1_TMDS_SYNC_CHAR_PATTERN_2_3 0x4b70
+#define mmDIG2_TMDS_SYNC_CHAR_PATTERN_2_3 0x4c70
+#define mmDIG3_TMDS_SYNC_CHAR_PATTERN_2_3 0x4d70
+#define mmDIG4_TMDS_SYNC_CHAR_PATTERN_2_3 0x4e70
+#define mmDIG5_TMDS_SYNC_CHAR_PATTERN_2_3 0x4f70
+#define mmDIG6_TMDS_SYNC_CHAR_PATTERN_2_3 0x5470
+#define mmDIG7_TMDS_SYNC_CHAR_PATTERN_2_3 0x5670
+#define mmDIG8_TMDS_SYNC_CHAR_PATTERN_2_3 0x5770
+#define mmTMDS_DEBUG 0x4a71
+#define mmDIG0_TMDS_DEBUG 0x4a71
+#define mmDIG1_TMDS_DEBUG 0x4b71
+#define mmDIG2_TMDS_DEBUG 0x4c71
+#define mmDIG3_TMDS_DEBUG 0x4d71
+#define mmDIG4_TMDS_DEBUG 0x4e71
+#define mmDIG5_TMDS_DEBUG 0x4f71
+#define mmDIG6_TMDS_DEBUG 0x5471
+#define mmDIG7_TMDS_DEBUG 0x5671
+#define mmDIG8_TMDS_DEBUG 0x5771
+#define mmTMDS_CTL_BITS 0x4a72
+#define mmDIG0_TMDS_CTL_BITS 0x4a72
+#define mmDIG1_TMDS_CTL_BITS 0x4b72
+#define mmDIG2_TMDS_CTL_BITS 0x4c72
+#define mmDIG3_TMDS_CTL_BITS 0x4d72
+#define mmDIG4_TMDS_CTL_BITS 0x4e72
+#define mmDIG5_TMDS_CTL_BITS 0x4f72
+#define mmDIG6_TMDS_CTL_BITS 0x5472
+#define mmDIG7_TMDS_CTL_BITS 0x5672
+#define mmDIG8_TMDS_CTL_BITS 0x5772
+#define mmTMDS_DCBALANCER_CONTROL 0x4a73
+#define mmDIG0_TMDS_DCBALANCER_CONTROL 0x4a73
+#define mmDIG1_TMDS_DCBALANCER_CONTROL 0x4b73
+#define mmDIG2_TMDS_DCBALANCER_CONTROL 0x4c73
+#define mmDIG3_TMDS_DCBALANCER_CONTROL 0x4d73
+#define mmDIG4_TMDS_DCBALANCER_CONTROL 0x4e73
+#define mmDIG5_TMDS_DCBALANCER_CONTROL 0x4f73
+#define mmDIG6_TMDS_DCBALANCER_CONTROL 0x5473
+#define mmDIG7_TMDS_DCBALANCER_CONTROL 0x5673
+#define mmDIG8_TMDS_DCBALANCER_CONTROL 0x5773
+#define mmTMDS_CTL0_1_GEN_CNTL 0x4a75
+#define mmDIG0_TMDS_CTL0_1_GEN_CNTL 0x4a75
+#define mmDIG1_TMDS_CTL0_1_GEN_CNTL 0x4b75
+#define mmDIG2_TMDS_CTL0_1_GEN_CNTL 0x4c75
+#define mmDIG3_TMDS_CTL0_1_GEN_CNTL 0x4d75
+#define mmDIG4_TMDS_CTL0_1_GEN_CNTL 0x4e75
+#define mmDIG5_TMDS_CTL0_1_GEN_CNTL 0x4f75
+#define mmDIG6_TMDS_CTL0_1_GEN_CNTL 0x5475
+#define mmDIG7_TMDS_CTL0_1_GEN_CNTL 0x5675
+#define mmDIG8_TMDS_CTL0_1_GEN_CNTL 0x5775
+#define mmTMDS_CTL2_3_GEN_CNTL 0x4a76
+#define mmDIG0_TMDS_CTL2_3_GEN_CNTL 0x4a76
+#define mmDIG1_TMDS_CTL2_3_GEN_CNTL 0x4b76
+#define mmDIG2_TMDS_CTL2_3_GEN_CNTL 0x4c76
+#define mmDIG3_TMDS_CTL2_3_GEN_CNTL 0x4d76
+#define mmDIG4_TMDS_CTL2_3_GEN_CNTL 0x4e76
+#define mmDIG5_TMDS_CTL2_3_GEN_CNTL 0x4f76
+#define mmDIG6_TMDS_CTL2_3_GEN_CNTL 0x5476
+#define mmDIG7_TMDS_CTL2_3_GEN_CNTL 0x5676
+#define mmDIG8_TMDS_CTL2_3_GEN_CNTL 0x5776
+#define mmDIG_VERSION 0x4a78
+#define mmDIG0_DIG_VERSION 0x4a78
+#define mmDIG1_DIG_VERSION 0x4b78
+#define mmDIG2_DIG_VERSION 0x4c78
+#define mmDIG3_DIG_VERSION 0x4d78
+#define mmDIG4_DIG_VERSION 0x4e78
+#define mmDIG5_DIG_VERSION 0x4f78
+#define mmDIG6_DIG_VERSION 0x5478
+#define mmDIG7_DIG_VERSION 0x5678
+#define mmDIG8_DIG_VERSION 0x5778
+#define mmDIG_LANE_ENABLE 0x4a79
+#define mmDIG0_DIG_LANE_ENABLE 0x4a79
+#define mmDIG1_DIG_LANE_ENABLE 0x4b79
+#define mmDIG2_DIG_LANE_ENABLE 0x4c79
+#define mmDIG3_DIG_LANE_ENABLE 0x4d79
+#define mmDIG4_DIG_LANE_ENABLE 0x4e79
+#define mmDIG5_DIG_LANE_ENABLE 0x4f79
+#define mmDIG6_DIG_LANE_ENABLE 0x5479
+#define mmDIG7_DIG_LANE_ENABLE 0x5679
+#define mmDIG8_DIG_LANE_ENABLE 0x5779
+#define mmDIG_TEST_DEBUG_INDEX 0x4a7a
+#define mmDIG0_DIG_TEST_DEBUG_INDEX 0x4a7a
+#define mmDIG1_DIG_TEST_DEBUG_INDEX 0x4b7a
+#define mmDIG2_DIG_TEST_DEBUG_INDEX 0x4c7a
+#define mmDIG3_DIG_TEST_DEBUG_INDEX 0x4d7a
+#define mmDIG4_DIG_TEST_DEBUG_INDEX 0x4e7a
+#define mmDIG5_DIG_TEST_DEBUG_INDEX 0x4f7a
+#define mmDIG6_DIG_TEST_DEBUG_INDEX 0x547a
+#define mmDIG7_DIG_TEST_DEBUG_INDEX 0x567a
+#define mmDIG8_DIG_TEST_DEBUG_INDEX 0x577a
+#define mmDIG_TEST_DEBUG_DATA 0x4a7b
+#define mmDIG0_DIG_TEST_DEBUG_DATA 0x4a7b
+#define mmDIG1_DIG_TEST_DEBUG_DATA 0x4b7b
+#define mmDIG2_DIG_TEST_DEBUG_DATA 0x4c7b
+#define mmDIG3_DIG_TEST_DEBUG_DATA 0x4d7b
+#define mmDIG4_DIG_TEST_DEBUG_DATA 0x4e7b
+#define mmDIG5_DIG_TEST_DEBUG_DATA 0x4f7b
+#define mmDIG6_DIG_TEST_DEBUG_DATA 0x547b
+#define mmDIG7_DIG_TEST_DEBUG_DATA 0x567b
+#define mmDIG8_DIG_TEST_DEBUG_DATA 0x577b
+#define mmDIG_FE_TEST_DEBUG_INDEX 0x4a7c
+#define mmDIG0_DIG_FE_TEST_DEBUG_INDEX 0x4a7c
+#define mmDIG1_DIG_FE_TEST_DEBUG_INDEX 0x4b7c
+#define mmDIG2_DIG_FE_TEST_DEBUG_INDEX 0x4c7c
+#define mmDIG3_DIG_FE_TEST_DEBUG_INDEX 0x4d7c
+#define mmDIG4_DIG_FE_TEST_DEBUG_INDEX 0x4e7c
+#define mmDIG5_DIG_FE_TEST_DEBUG_INDEX 0x4f7c
+#define mmDIG6_DIG_FE_TEST_DEBUG_INDEX 0x547c
+#define mmDIG7_DIG_FE_TEST_DEBUG_INDEX 0x567c
+#define mmDIG8_DIG_FE_TEST_DEBUG_INDEX 0x577c
+#define mmDIG_FE_TEST_DEBUG_DATA 0x4a7d
+#define mmDIG0_DIG_FE_TEST_DEBUG_DATA 0x4a7d
+#define mmDIG1_DIG_FE_TEST_DEBUG_DATA 0x4b7d
+#define mmDIG2_DIG_FE_TEST_DEBUG_DATA 0x4c7d
+#define mmDIG3_DIG_FE_TEST_DEBUG_DATA 0x4d7d
+#define mmDIG4_DIG_FE_TEST_DEBUG_DATA 0x4e7d
+#define mmDIG5_DIG_FE_TEST_DEBUG_DATA 0x4f7d
+#define mmDIG6_DIG_FE_TEST_DEBUG_DATA 0x547d
+#define mmDIG7_DIG_FE_TEST_DEBUG_DATA 0x567d
+#define mmDIG8_DIG_FE_TEST_DEBUG_DATA 0x577d
+#define mmDMCU_CTRL 0x1600
+#define mmDMCU_STATUS 0x1601
+#define mmDMCU_PC_START_ADDR 0x1602
+#define mmDMCU_FW_START_ADDR 0x1603
+#define mmDMCU_FW_END_ADDR 0x1604
+#define mmDMCU_FW_ISR_START_ADDR 0x1605
+#define mmDMCU_FW_CS_HI 0x1606
+#define mmDMCU_FW_CS_LO 0x1607
+#define mmDMCU_RAM_ACCESS_CTRL 0x1608
+#define mmDMCU_ERAM_WR_CTRL 0x1609
+#define mmDMCU_ERAM_WR_DATA 0x160a
+#define mmDMCU_ERAM_RD_CTRL 0x160b
+#define mmDMCU_ERAM_RD_DATA 0x160c
+#define mmDMCU_IRAM_WR_CTRL 0x160d
+#define mmDMCU_IRAM_WR_DATA 0x160e
+#define mmDMCU_IRAM_RD_CTRL 0x160f
+#define mmDMCU_IRAM_RD_DATA 0x1610
+#define mmDMCU_EVENT_TRIGGER 0x1611
+#define mmDMCU_UC_INTERNAL_INT_STATUS 0x1612
+#define mmDMCU_SS_INTERRUPT_CNTL_STATUS 0x1613
+#define mmDMCU_INTERRUPT_STATUS 0x1614
+#define mmDMCU_INTERRUPT_STATUS_1 0x1633
+#define mmDMCU_INTERRUPT_TO_HOST_EN_MASK 0x1615
+#define mmDMCU_INTERRUPT_TO_UC_EN_MASK 0x1616
+#define mmDMCU_INTERRUPT_TO_UC_EN_MASK_1 0x1631
+#define mmDMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL 0x1617
+#define mmDMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL_1 0x1632
+#define mmDC_DMCU_SCRATCH 0x1618
+#define mmDMCU_INT_CNT 0x1619
+#define mmDMCU_FW_CHECKSUM_SMPL_BYTE_POS 0x161a
+#define mmDMCU_UC_CLK_GATING_CNTL 0x161b
+#define mmMASTER_COMM_DATA_REG1 0x161c
+#define mmMASTER_COMM_DATA_REG2 0x161d
+#define mmMASTER_COMM_DATA_REG3 0x161e
+#define mmMASTER_COMM_CMD_REG 0x161f
+#define mmMASTER_COMM_CNTL_REG 0x1620
+#define mmSLAVE_COMM_DATA_REG1 0x1621
+#define mmSLAVE_COMM_DATA_REG2 0x1622
+#define mmSLAVE_COMM_DATA_REG3 0x1623
+#define mmSLAVE_COMM_CMD_REG 0x1624
+#define mmSLAVE_COMM_CNTL_REG 0x1625
+#define mmDMCU_TEST_DEBUG_INDEX 0x1626
+#define mmDMCU_TEST_DEBUG_DATA 0x1627
+#define mmDMCU_PERFMON_INTERRUPT_STATUS1 0x1644
+#define mmDMCU_PERFMON_INTERRUPT_STATUS2 0x1645
+#define mmDMCU_PERFMON_INTERRUPT_STATUS3 0x1646
+#define mmDMCU_PERFMON_INTERRUPT_STATUS4 0x1647
+#define mmDMCU_PERFMON_INTERRUPT_STATUS5 0x1642
+#define mmDMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1 0x1674
+#define mmDMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2 0x1675
+#define mmDMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3 0x1676
+#define mmDMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4 0x1677
+#define mmDMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5 0x1643
+#define mmDMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1 0x1678
+#define mmDMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2 0x1679
+#define mmDMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3 0x167a
+#define mmDMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4 0x167b
+#define mmDMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5 0x1673
+#define mmDMCU_DPRX_INTERRUPT_STATUS1 0x1634
+#define mmDMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1 0x1635
+#define mmDMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1 0x1636
+#define mmDP_LINK_CNTL 0x4aa0
+#define mmDP0_DP_LINK_CNTL 0x4aa0
+#define mmDP1_DP_LINK_CNTL 0x4ba0
+#define mmDP2_DP_LINK_CNTL 0x4ca0
+#define mmDP3_DP_LINK_CNTL 0x4da0
+#define mmDP4_DP_LINK_CNTL 0x4ea0
+#define mmDP5_DP_LINK_CNTL 0x4fa0
+#define mmDP6_DP_LINK_CNTL 0x54a0
+#define mmDP7_DP_LINK_CNTL 0x56a0
+#define mmDP8_DP_LINK_CNTL 0x57a0
+#define mmDP_PIXEL_FORMAT 0x4aa1
+#define mmDP0_DP_PIXEL_FORMAT 0x4aa1
+#define mmDP1_DP_PIXEL_FORMAT 0x4ba1
+#define mmDP2_DP_PIXEL_FORMAT 0x4ca1
+#define mmDP3_DP_PIXEL_FORMAT 0x4da1
+#define mmDP4_DP_PIXEL_FORMAT 0x4ea1
+#define mmDP5_DP_PIXEL_FORMAT 0x4fa1
+#define mmDP6_DP_PIXEL_FORMAT 0x54a1
+#define mmDP7_DP_PIXEL_FORMAT 0x56a1
+#define mmDP8_DP_PIXEL_FORMAT 0x57a1
+#define mmDP_MSA_COLORIMETRY 0x4aa2
+#define mmDP0_DP_MSA_COLORIMETRY 0x4aa2
+#define mmDP1_DP_MSA_COLORIMETRY 0x4ba2
+#define mmDP2_DP_MSA_COLORIMETRY 0x4ca2
+#define mmDP3_DP_MSA_COLORIMETRY 0x4da2
+#define mmDP4_DP_MSA_COLORIMETRY 0x4ea2
+#define mmDP5_DP_MSA_COLORIMETRY 0x4fa2
+#define mmDP6_DP_MSA_COLORIMETRY 0x54a2
+#define mmDP7_DP_MSA_COLORIMETRY 0x56a2
+#define mmDP8_DP_MSA_COLORIMETRY 0x57a2
+#define mmDP_CONFIG 0x4aa3
+#define mmDP0_DP_CONFIG 0x4aa3
+#define mmDP1_DP_CONFIG 0x4ba3
+#define mmDP2_DP_CONFIG 0x4ca3
+#define mmDP3_DP_CONFIG 0x4da3
+#define mmDP4_DP_CONFIG 0x4ea3
+#define mmDP5_DP_CONFIG 0x4fa3
+#define mmDP6_DP_CONFIG 0x54a3
+#define mmDP7_DP_CONFIG 0x56a3
+#define mmDP8_DP_CONFIG 0x57a3
+#define mmDP_VID_STREAM_CNTL 0x4aa4
+#define mmDP0_DP_VID_STREAM_CNTL 0x4aa4
+#define mmDP1_DP_VID_STREAM_CNTL 0x4ba4
+#define mmDP2_DP_VID_STREAM_CNTL 0x4ca4
+#define mmDP3_DP_VID_STREAM_CNTL 0x4da4
+#define mmDP4_DP_VID_STREAM_CNTL 0x4ea4
+#define mmDP5_DP_VID_STREAM_CNTL 0x4fa4
+#define mmDP6_DP_VID_STREAM_CNTL 0x54a4
+#define mmDP7_DP_VID_STREAM_CNTL 0x56a4
+#define mmDP8_DP_VID_STREAM_CNTL 0x57a4
+#define mmDP_STEER_FIFO 0x4aa5
+#define mmDP0_DP_STEER_FIFO 0x4aa5
+#define mmDP1_DP_STEER_FIFO 0x4ba5
+#define mmDP2_DP_STEER_FIFO 0x4ca5
+#define mmDP3_DP_STEER_FIFO 0x4da5
+#define mmDP4_DP_STEER_FIFO 0x4ea5
+#define mmDP5_DP_STEER_FIFO 0x4fa5
+#define mmDP6_DP_STEER_FIFO 0x54a5
+#define mmDP7_DP_STEER_FIFO 0x56a5
+#define mmDP8_DP_STEER_FIFO 0x57a5
+#define mmDP_MSA_MISC 0x4aa6
+#define mmDP0_DP_MSA_MISC 0x4aa6
+#define mmDP1_DP_MSA_MISC 0x4ba6
+#define mmDP2_DP_MSA_MISC 0x4ca6
+#define mmDP3_DP_MSA_MISC 0x4da6
+#define mmDP4_DP_MSA_MISC 0x4ea6
+#define mmDP5_DP_MSA_MISC 0x4fa6
+#define mmDP6_DP_MSA_MISC 0x54a6
+#define mmDP7_DP_MSA_MISC 0x56a6
+#define mmDP8_DP_MSA_MISC 0x57a6
+#define mmDP_VID_TIMING 0x4aa8
+#define mmDP0_DP_VID_TIMING 0x4aa8
+#define mmDP1_DP_VID_TIMING 0x4ba8
+#define mmDP2_DP_VID_TIMING 0x4ca8
+#define mmDP3_DP_VID_TIMING 0x4da8
+#define mmDP4_DP_VID_TIMING 0x4ea8
+#define mmDP5_DP_VID_TIMING 0x4fa8
+#define mmDP6_DP_VID_TIMING 0x54a8
+#define mmDP7_DP_VID_TIMING 0x56a8
+#define mmDP8_DP_VID_TIMING 0x57a8
+#define mmDP_VID_N 0x4aa9
+#define mmDP0_DP_VID_N 0x4aa9
+#define mmDP1_DP_VID_N 0x4ba9
+#define mmDP2_DP_VID_N 0x4ca9
+#define mmDP3_DP_VID_N 0x4da9
+#define mmDP4_DP_VID_N 0x4ea9
+#define mmDP5_DP_VID_N 0x4fa9
+#define mmDP6_DP_VID_N 0x54a9
+#define mmDP7_DP_VID_N 0x56a9
+#define mmDP8_DP_VID_N 0x57a9
+#define mmDP_VID_M 0x4aaa
+#define mmDP0_DP_VID_M 0x4aaa
+#define mmDP1_DP_VID_M 0x4baa
+#define mmDP2_DP_VID_M 0x4caa
+#define mmDP3_DP_VID_M 0x4daa
+#define mmDP4_DP_VID_M 0x4eaa
+#define mmDP5_DP_VID_M 0x4faa
+#define mmDP6_DP_VID_M 0x54aa
+#define mmDP7_DP_VID_M 0x56aa
+#define mmDP8_DP_VID_M 0x57aa
+#define mmDP_LINK_FRAMING_CNTL 0x4aab
+#define mmDP0_DP_LINK_FRAMING_CNTL 0x4aab
+#define mmDP1_DP_LINK_FRAMING_CNTL 0x4bab
+#define mmDP2_DP_LINK_FRAMING_CNTL 0x4cab
+#define mmDP3_DP_LINK_FRAMING_CNTL 0x4dab
+#define mmDP4_DP_LINK_FRAMING_CNTL 0x4eab
+#define mmDP5_DP_LINK_FRAMING_CNTL 0x4fab
+#define mmDP6_DP_LINK_FRAMING_CNTL 0x54ab
+#define mmDP7_DP_LINK_FRAMING_CNTL 0x56ab
+#define mmDP8_DP_LINK_FRAMING_CNTL 0x57ab
+#define mmDP_HBR2_EYE_PATTERN 0x4aac
+#define mmDP0_DP_HBR2_EYE_PATTERN 0x4aac
+#define mmDP1_DP_HBR2_EYE_PATTERN 0x4bac
+#define mmDP2_DP_HBR2_EYE_PATTERN 0x4cac
+#define mmDP3_DP_HBR2_EYE_PATTERN 0x4dac
+#define mmDP4_DP_HBR2_EYE_PATTERN 0x4eac
+#define mmDP5_DP_HBR2_EYE_PATTERN 0x4fac
+#define mmDP6_DP_HBR2_EYE_PATTERN 0x54ac
+#define mmDP7_DP_HBR2_EYE_PATTERN 0x56ac
+#define mmDP8_DP_HBR2_EYE_PATTERN 0x57ac
+#define mmDP_VID_MSA_VBID 0x4aad
+#define mmDP0_DP_VID_MSA_VBID 0x4aad
+#define mmDP1_DP_VID_MSA_VBID 0x4bad
+#define mmDP2_DP_VID_MSA_VBID 0x4cad
+#define mmDP3_DP_VID_MSA_VBID 0x4dad
+#define mmDP4_DP_VID_MSA_VBID 0x4ead
+#define mmDP5_DP_VID_MSA_VBID 0x4fad
+#define mmDP6_DP_VID_MSA_VBID 0x54ad
+#define mmDP7_DP_VID_MSA_VBID 0x56ad
+#define mmDP8_DP_VID_MSA_VBID 0x57ad
+#define mmDP_VID_INTERRUPT_CNTL 0x4aae
+#define mmDP0_DP_VID_INTERRUPT_CNTL 0x4aae
+#define mmDP1_DP_VID_INTERRUPT_CNTL 0x4bae
+#define mmDP2_DP_VID_INTERRUPT_CNTL 0x4cae
+#define mmDP3_DP_VID_INTERRUPT_CNTL 0x4dae
+#define mmDP4_DP_VID_INTERRUPT_CNTL 0x4eae
+#define mmDP5_DP_VID_INTERRUPT_CNTL 0x4fae
+#define mmDP6_DP_VID_INTERRUPT_CNTL 0x54ae
+#define mmDP7_DP_VID_INTERRUPT_CNTL 0x56ae
+#define mmDP8_DP_VID_INTERRUPT_CNTL 0x57ae
+#define mmDP_DPHY_CNTL 0x4aaf
+#define mmDP0_DP_DPHY_CNTL 0x4aaf
+#define mmDP1_DP_DPHY_CNTL 0x4baf
+#define mmDP2_DP_DPHY_CNTL 0x4caf
+#define mmDP3_DP_DPHY_CNTL 0x4daf
+#define mmDP4_DP_DPHY_CNTL 0x4eaf
+#define mmDP5_DP_DPHY_CNTL 0x4faf
+#define mmDP6_DP_DPHY_CNTL 0x54af
+#define mmDP7_DP_DPHY_CNTL 0x56af
+#define mmDP8_DP_DPHY_CNTL 0x57af
+#define mmDP_DPHY_TRAINING_PATTERN_SEL 0x4ab0
+#define mmDP0_DP_DPHY_TRAINING_PATTERN_SEL 0x4ab0
+#define mmDP1_DP_DPHY_TRAINING_PATTERN_SEL 0x4bb0
+#define mmDP2_DP_DPHY_TRAINING_PATTERN_SEL 0x4cb0
+#define mmDP3_DP_DPHY_TRAINING_PATTERN_SEL 0x4db0
+#define mmDP4_DP_DPHY_TRAINING_PATTERN_SEL 0x4eb0
+#define mmDP5_DP_DPHY_TRAINING_PATTERN_SEL 0x4fb0
+#define mmDP6_DP_DPHY_TRAINING_PATTERN_SEL 0x54b0
+#define mmDP7_DP_DPHY_TRAINING_PATTERN_SEL 0x56b0
+#define mmDP8_DP_DPHY_TRAINING_PATTERN_SEL 0x57b0
+#define mmDP_DPHY_SYM0 0x4ab1
+#define mmDP0_DP_DPHY_SYM0 0x4ab1
+#define mmDP1_DP_DPHY_SYM0 0x4bb1
+#define mmDP2_DP_DPHY_SYM0 0x4cb1
+#define mmDP3_DP_DPHY_SYM0 0x4db1
+#define mmDP4_DP_DPHY_SYM0 0x4eb1
+#define mmDP5_DP_DPHY_SYM0 0x4fb1
+#define mmDP6_DP_DPHY_SYM0 0x54b1
+#define mmDP7_DP_DPHY_SYM0 0x56b1
+#define mmDP8_DP_DPHY_SYM0 0x57b1
+#define mmDP_DPHY_SYM1 0x4ab2
+#define mmDP0_DP_DPHY_SYM1 0x4ab2
+#define mmDP1_DP_DPHY_SYM1 0x4bb2
+#define mmDP2_DP_DPHY_SYM1 0x4cb2
+#define mmDP3_DP_DPHY_SYM1 0x4db2
+#define mmDP4_DP_DPHY_SYM1 0x4eb2
+#define mmDP5_DP_DPHY_SYM1 0x4fb2
+#define mmDP6_DP_DPHY_SYM1 0x54b2
+#define mmDP7_DP_DPHY_SYM1 0x56b2
+#define mmDP8_DP_DPHY_SYM1 0x57b2
+#define mmDP_DPHY_SYM2 0x4ab3
+#define mmDP0_DP_DPHY_SYM2 0x4ab3
+#define mmDP1_DP_DPHY_SYM2 0x4bb3
+#define mmDP2_DP_DPHY_SYM2 0x4cb3
+#define mmDP3_DP_DPHY_SYM2 0x4db3
+#define mmDP4_DP_DPHY_SYM2 0x4eb3
+#define mmDP5_DP_DPHY_SYM2 0x4fb3
+#define mmDP6_DP_DPHY_SYM2 0x54b3
+#define mmDP7_DP_DPHY_SYM2 0x56b3
+#define mmDP8_DP_DPHY_SYM2 0x57b3
+#define mmDP_DPHY_8B10B_CNTL 0x4ab4
+#define mmDP0_DP_DPHY_8B10B_CNTL 0x4ab4
+#define mmDP1_DP_DPHY_8B10B_CNTL 0x4bb4
+#define mmDP2_DP_DPHY_8B10B_CNTL 0x4cb4
+#define mmDP3_DP_DPHY_8B10B_CNTL 0x4db4
+#define mmDP4_DP_DPHY_8B10B_CNTL 0x4eb4
+#define mmDP5_DP_DPHY_8B10B_CNTL 0x4fb4
+#define mmDP6_DP_DPHY_8B10B_CNTL 0x54b4
+#define mmDP7_DP_DPHY_8B10B_CNTL 0x56b4
+#define mmDP8_DP_DPHY_8B10B_CNTL 0x57b4
+#define mmDP_DPHY_PRBS_CNTL 0x4ab5
+#define mmDP0_DP_DPHY_PRBS_CNTL 0x4ab5
+#define mmDP1_DP_DPHY_PRBS_CNTL 0x4bb5
+#define mmDP2_DP_DPHY_PRBS_CNTL 0x4cb5
+#define mmDP3_DP_DPHY_PRBS_CNTL 0x4db5
+#define mmDP4_DP_DPHY_PRBS_CNTL 0x4eb5
+#define mmDP5_DP_DPHY_PRBS_CNTL 0x4fb5
+#define mmDP6_DP_DPHY_PRBS_CNTL 0x54b5
+#define mmDP7_DP_DPHY_PRBS_CNTL 0x56b5
+#define mmDP8_DP_DPHY_PRBS_CNTL 0x57b5
+#define mmDP_DPHY_BS_SR_SWAP_CNTL 0x4adc
+#define mmDP0_DP_DPHY_BS_SR_SWAP_CNTL 0x4adc
+#define mmDP1_DP_DPHY_BS_SR_SWAP_CNTL 0x4bdc
+#define mmDP2_DP_DPHY_BS_SR_SWAP_CNTL 0x4cdc
+#define mmDP3_DP_DPHY_BS_SR_SWAP_CNTL 0x4ddc
+#define mmDP4_DP_DPHY_BS_SR_SWAP_CNTL 0x4edc
+#define mmDP5_DP_DPHY_BS_SR_SWAP_CNTL 0x4fdc
+#define mmDP6_DP_DPHY_BS_SR_SWAP_CNTL 0x54dc
+#define mmDP7_DP_DPHY_BS_SR_SWAP_CNTL 0x56dc
+#define mmDP8_DP_DPHY_BS_SR_SWAP_CNTL 0x57dc
+#define mmDP_DPHY_CRC_EN 0x4ab7
+#define mmDP0_DP_DPHY_CRC_EN 0x4ab7
+#define mmDP1_DP_DPHY_CRC_EN 0x4bb7
+#define mmDP2_DP_DPHY_CRC_EN 0x4cb7
+#define mmDP3_DP_DPHY_CRC_EN 0x4db7
+#define mmDP4_DP_DPHY_CRC_EN 0x4eb7
+#define mmDP5_DP_DPHY_CRC_EN 0x4fb7
+#define mmDP6_DP_DPHY_CRC_EN 0x54b7
+#define mmDP7_DP_DPHY_CRC_EN 0x56b7
+#define mmDP8_DP_DPHY_CRC_EN 0x57b7
+#define mmDP_DPHY_CRC_CNTL 0x4ab8
+#define mmDP0_DP_DPHY_CRC_CNTL 0x4ab8
+#define mmDP1_DP_DPHY_CRC_CNTL 0x4bb8
+#define mmDP2_DP_DPHY_CRC_CNTL 0x4cb8
+#define mmDP3_DP_DPHY_CRC_CNTL 0x4db8
+#define mmDP4_DP_DPHY_CRC_CNTL 0x4eb8
+#define mmDP5_DP_DPHY_CRC_CNTL 0x4fb8
+#define mmDP6_DP_DPHY_CRC_CNTL 0x54b8
+#define mmDP7_DP_DPHY_CRC_CNTL 0x56b8
+#define mmDP8_DP_DPHY_CRC_CNTL 0x57b8
+#define mmDP_DPHY_CRC_RESULT 0x4ab9
+#define mmDP0_DP_DPHY_CRC_RESULT 0x4ab9
+#define mmDP1_DP_DPHY_CRC_RESULT 0x4bb9
+#define mmDP2_DP_DPHY_CRC_RESULT 0x4cb9
+#define mmDP3_DP_DPHY_CRC_RESULT 0x4db9
+#define mmDP4_DP_DPHY_CRC_RESULT 0x4eb9
+#define mmDP5_DP_DPHY_CRC_RESULT 0x4fb9
+#define mmDP6_DP_DPHY_CRC_RESULT 0x54b9
+#define mmDP7_DP_DPHY_CRC_RESULT 0x56b9
+#define mmDP8_DP_DPHY_CRC_RESULT 0x57b9
+#define mmDP_DPHY_CRC_MST_CNTL 0x4aba
+#define mmDP0_DP_DPHY_CRC_MST_CNTL 0x4aba
+#define mmDP1_DP_DPHY_CRC_MST_CNTL 0x4bba
+#define mmDP2_DP_DPHY_CRC_MST_CNTL 0x4cba
+#define mmDP3_DP_DPHY_CRC_MST_CNTL 0x4dba
+#define mmDP4_DP_DPHY_CRC_MST_CNTL 0x4eba
+#define mmDP5_DP_DPHY_CRC_MST_CNTL 0x4fba
+#define mmDP6_DP_DPHY_CRC_MST_CNTL 0x54ba
+#define mmDP7_DP_DPHY_CRC_MST_CNTL 0x56ba
+#define mmDP8_DP_DPHY_CRC_MST_CNTL 0x57ba
+#define mmDP_DPHY_CRC_MST_STATUS 0x4abb
+#define mmDP0_DP_DPHY_CRC_MST_STATUS 0x4abb
+#define mmDP1_DP_DPHY_CRC_MST_STATUS 0x4bbb
+#define mmDP2_DP_DPHY_CRC_MST_STATUS 0x4cbb
+#define mmDP3_DP_DPHY_CRC_MST_STATUS 0x4dbb
+#define mmDP4_DP_DPHY_CRC_MST_STATUS 0x4ebb
+#define mmDP5_DP_DPHY_CRC_MST_STATUS 0x4fbb
+#define mmDP6_DP_DPHY_CRC_MST_STATUS 0x54bb
+#define mmDP7_DP_DPHY_CRC_MST_STATUS 0x56bb
+#define mmDP8_DP_DPHY_CRC_MST_STATUS 0x57bb
+#define mmDP_DPHY_FAST_TRAINING 0x4abc
+#define mmDP0_DP_DPHY_FAST_TRAINING 0x4abc
+#define mmDP1_DP_DPHY_FAST_TRAINING 0x4bbc
+#define mmDP2_DP_DPHY_FAST_TRAINING 0x4cbc
+#define mmDP3_DP_DPHY_FAST_TRAINING 0x4dbc
+#define mmDP4_DP_DPHY_FAST_TRAINING 0x4ebc
+#define mmDP5_DP_DPHY_FAST_TRAINING 0x4fbc
+#define mmDP6_DP_DPHY_FAST_TRAINING 0x54bc
+#define mmDP7_DP_DPHY_FAST_TRAINING 0x56bc
+#define mmDP8_DP_DPHY_FAST_TRAINING 0x57bc
+#define mmDP_DPHY_FAST_TRAINING_STATUS 0x4abd
+#define mmDP0_DP_DPHY_FAST_TRAINING_STATUS 0x4abd
+#define mmDP1_DP_DPHY_FAST_TRAINING_STATUS 0x4bbd
+#define mmDP2_DP_DPHY_FAST_TRAINING_STATUS 0x4cbd
+#define mmDP3_DP_DPHY_FAST_TRAINING_STATUS 0x4dbd
+#define mmDP4_DP_DPHY_FAST_TRAINING_STATUS 0x4ebd
+#define mmDP5_DP_DPHY_FAST_TRAINING_STATUS 0x4fbd
+#define mmDP6_DP_DPHY_FAST_TRAINING_STATUS 0x54bd
+#define mmDP7_DP_DPHY_FAST_TRAINING_STATUS 0x56bd
+#define mmDP8_DP_DPHY_FAST_TRAINING_STATUS 0x57bd
+#define mmDP_DPHY_HBR2_PATTERN_CONTROL 0x4add
+#define mmDP0_DP_DPHY_HBR2_PATTERN_CONTROL 0x4add
+#define mmDP1_DP_DPHY_HBR2_PATTERN_CONTROL 0x4bdd
+#define mmDP2_DP_DPHY_HBR2_PATTERN_CONTROL 0x4cdd
+#define mmDP3_DP_DPHY_HBR2_PATTERN_CONTROL 0x4ddd
+#define mmDP4_DP_DPHY_HBR2_PATTERN_CONTROL 0x4edd
+#define mmDP5_DP_DPHY_HBR2_PATTERN_CONTROL 0x4fdd
+#define mmDP6_DP_DPHY_HBR2_PATTERN_CONTROL 0x54dd
+#define mmDP7_DP_DPHY_HBR2_PATTERN_CONTROL 0x56dd
+#define mmDP8_DP_DPHY_HBR2_PATTERN_CONTROL 0x57dd
+#define mmDP_MSA_V_TIMING_OVERRIDE1 0x4abe
+#define mmDP0_DP_MSA_V_TIMING_OVERRIDE1 0x4abe
+#define mmDP1_DP_MSA_V_TIMING_OVERRIDE1 0x4bbe
+#define mmDP2_DP_MSA_V_TIMING_OVERRIDE1 0x4cbe
+#define mmDP3_DP_MSA_V_TIMING_OVERRIDE1 0x4dbe
+#define mmDP4_DP_MSA_V_TIMING_OVERRIDE1 0x4ebe
+#define mmDP5_DP_MSA_V_TIMING_OVERRIDE1 0x4fbe
+#define mmDP6_DP_MSA_V_TIMING_OVERRIDE1 0x54be
+#define mmDP7_DP_MSA_V_TIMING_OVERRIDE1 0x56be
+#define mmDP8_DP_MSA_V_TIMING_OVERRIDE1 0x57be
+#define mmDP_MSA_V_TIMING_OVERRIDE2 0x4abf
+#define mmDP0_DP_MSA_V_TIMING_OVERRIDE2 0x4abf
+#define mmDP1_DP_MSA_V_TIMING_OVERRIDE2 0x4bbf
+#define mmDP2_DP_MSA_V_TIMING_OVERRIDE2 0x4cbf
+#define mmDP3_DP_MSA_V_TIMING_OVERRIDE2 0x4dbf
+#define mmDP4_DP_MSA_V_TIMING_OVERRIDE2 0x4ebf
+#define mmDP5_DP_MSA_V_TIMING_OVERRIDE2 0x4fbf
+#define mmDP6_DP_MSA_V_TIMING_OVERRIDE2 0x54bf
+#define mmDP7_DP_MSA_V_TIMING_OVERRIDE2 0x56bf
+#define mmDP8_DP_MSA_V_TIMING_OVERRIDE2 0x57bf
+#define mmDP_SEC_CNTL 0x4ac3
+#define mmDP0_DP_SEC_CNTL 0x4ac3
+#define mmDP1_DP_SEC_CNTL 0x4bc3
+#define mmDP2_DP_SEC_CNTL 0x4cc3
+#define mmDP3_DP_SEC_CNTL 0x4dc3
+#define mmDP4_DP_SEC_CNTL 0x4ec3
+#define mmDP5_DP_SEC_CNTL 0x4fc3
+#define mmDP6_DP_SEC_CNTL 0x54c3
+#define mmDP7_DP_SEC_CNTL 0x56c3
+#define mmDP8_DP_SEC_CNTL 0x57c3
+#define mmDP_SEC_CNTL1 0x4ac4
+#define mmDP0_DP_SEC_CNTL1 0x4ac4
+#define mmDP1_DP_SEC_CNTL1 0x4bc4
+#define mmDP2_DP_SEC_CNTL1 0x4cc4
+#define mmDP3_DP_SEC_CNTL1 0x4dc4
+#define mmDP4_DP_SEC_CNTL1 0x4ec4
+#define mmDP5_DP_SEC_CNTL1 0x4fc4
+#define mmDP6_DP_SEC_CNTL1 0x54c4
+#define mmDP7_DP_SEC_CNTL1 0x56c4
+#define mmDP8_DP_SEC_CNTL1 0x57c4
+#define mmDP_SEC_FRAMING1 0x4ac5
+#define mmDP0_DP_SEC_FRAMING1 0x4ac5
+#define mmDP1_DP_SEC_FRAMING1 0x4bc5
+#define mmDP2_DP_SEC_FRAMING1 0x4cc5
+#define mmDP3_DP_SEC_FRAMING1 0x4dc5
+#define mmDP4_DP_SEC_FRAMING1 0x4ec5
+#define mmDP5_DP_SEC_FRAMING1 0x4fc5
+#define mmDP6_DP_SEC_FRAMING1 0x54c5
+#define mmDP7_DP_SEC_FRAMING1 0x56c5
+#define mmDP8_DP_SEC_FRAMING1 0x57c5
+#define mmDP_SEC_FRAMING2 0x4ac6
+#define mmDP0_DP_SEC_FRAMING2 0x4ac6
+#define mmDP1_DP_SEC_FRAMING2 0x4bc6
+#define mmDP2_DP_SEC_FRAMING2 0x4cc6
+#define mmDP3_DP_SEC_FRAMING2 0x4dc6
+#define mmDP4_DP_SEC_FRAMING2 0x4ec6
+#define mmDP5_DP_SEC_FRAMING2 0x4fc6
+#define mmDP6_DP_SEC_FRAMING2 0x54c6
+#define mmDP7_DP_SEC_FRAMING2 0x56c6
+#define mmDP8_DP_SEC_FRAMING2 0x57c6
+#define mmDP_SEC_FRAMING3 0x4ac7
+#define mmDP0_DP_SEC_FRAMING3 0x4ac7
+#define mmDP1_DP_SEC_FRAMING3 0x4bc7
+#define mmDP2_DP_SEC_FRAMING3 0x4cc7
+#define mmDP3_DP_SEC_FRAMING3 0x4dc7
+#define mmDP4_DP_SEC_FRAMING3 0x4ec7
+#define mmDP5_DP_SEC_FRAMING3 0x4fc7
+#define mmDP6_DP_SEC_FRAMING3 0x54c7
+#define mmDP7_DP_SEC_FRAMING3 0x56c7
+#define mmDP8_DP_SEC_FRAMING3 0x57c7
+#define mmDP_SEC_FRAMING4 0x4ac8
+#define mmDP0_DP_SEC_FRAMING4 0x4ac8
+#define mmDP1_DP_SEC_FRAMING4 0x4bc8
+#define mmDP2_DP_SEC_FRAMING4 0x4cc8
+#define mmDP3_DP_SEC_FRAMING4 0x4dc8
+#define mmDP4_DP_SEC_FRAMING4 0x4ec8
+#define mmDP5_DP_SEC_FRAMING4 0x4fc8
+#define mmDP6_DP_SEC_FRAMING4 0x54c8
+#define mmDP7_DP_SEC_FRAMING4 0x56c8
+#define mmDP8_DP_SEC_FRAMING4 0x57c8
+#define mmDP_SEC_AUD_N 0x4ac9
+#define mmDP0_DP_SEC_AUD_N 0x4ac9
+#define mmDP1_DP_SEC_AUD_N 0x4bc9
+#define mmDP2_DP_SEC_AUD_N 0x4cc9
+#define mmDP3_DP_SEC_AUD_N 0x4dc9
+#define mmDP4_DP_SEC_AUD_N 0x4ec9
+#define mmDP5_DP_SEC_AUD_N 0x4fc9
+#define mmDP6_DP_SEC_AUD_N 0x54c9
+#define mmDP7_DP_SEC_AUD_N 0x56c9
+#define mmDP8_DP_SEC_AUD_N 0x57c9
+#define mmDP_SEC_AUD_N_READBACK 0x4aca
+#define mmDP0_DP_SEC_AUD_N_READBACK 0x4aca
+#define mmDP1_DP_SEC_AUD_N_READBACK 0x4bca
+#define mmDP2_DP_SEC_AUD_N_READBACK 0x4cca
+#define mmDP3_DP_SEC_AUD_N_READBACK 0x4dca
+#define mmDP4_DP_SEC_AUD_N_READBACK 0x4eca
+#define mmDP5_DP_SEC_AUD_N_READBACK 0x4fca
+#define mmDP6_DP_SEC_AUD_N_READBACK 0x54ca
+#define mmDP7_DP_SEC_AUD_N_READBACK 0x56ca
+#define mmDP8_DP_SEC_AUD_N_READBACK 0x57ca
+#define mmDP_SEC_AUD_M 0x4acb
+#define mmDP0_DP_SEC_AUD_M 0x4acb
+#define mmDP1_DP_SEC_AUD_M 0x4bcb
+#define mmDP2_DP_SEC_AUD_M 0x4ccb
+#define mmDP3_DP_SEC_AUD_M 0x4dcb
+#define mmDP4_DP_SEC_AUD_M 0x4ecb
+#define mmDP5_DP_SEC_AUD_M 0x4fcb
+#define mmDP6_DP_SEC_AUD_M 0x54cb
+#define mmDP7_DP_SEC_AUD_M 0x56cb
+#define mmDP8_DP_SEC_AUD_M 0x57cb
+#define mmDP_SEC_AUD_M_READBACK 0x4acc
+#define mmDP0_DP_SEC_AUD_M_READBACK 0x4acc
+#define mmDP1_DP_SEC_AUD_M_READBACK 0x4bcc
+#define mmDP2_DP_SEC_AUD_M_READBACK 0x4ccc
+#define mmDP3_DP_SEC_AUD_M_READBACK 0x4dcc
+#define mmDP4_DP_SEC_AUD_M_READBACK 0x4ecc
+#define mmDP5_DP_SEC_AUD_M_READBACK 0x4fcc
+#define mmDP6_DP_SEC_AUD_M_READBACK 0x54cc
+#define mmDP7_DP_SEC_AUD_M_READBACK 0x56cc
+#define mmDP8_DP_SEC_AUD_M_READBACK 0x57cc
+#define mmDP_SEC_TIMESTAMP 0x4acd
+#define mmDP0_DP_SEC_TIMESTAMP 0x4acd
+#define mmDP1_DP_SEC_TIMESTAMP 0x4bcd
+#define mmDP2_DP_SEC_TIMESTAMP 0x4ccd
+#define mmDP3_DP_SEC_TIMESTAMP 0x4dcd
+#define mmDP4_DP_SEC_TIMESTAMP 0x4ecd
+#define mmDP5_DP_SEC_TIMESTAMP 0x4fcd
+#define mmDP6_DP_SEC_TIMESTAMP 0x54cd
+#define mmDP7_DP_SEC_TIMESTAMP 0x56cd
+#define mmDP8_DP_SEC_TIMESTAMP 0x57cd
+#define mmDP_SEC_PACKET_CNTL 0x4ace
+#define mmDP0_DP_SEC_PACKET_CNTL 0x4ace
+#define mmDP1_DP_SEC_PACKET_CNTL 0x4bce
+#define mmDP2_DP_SEC_PACKET_CNTL 0x4cce
+#define mmDP3_DP_SEC_PACKET_CNTL 0x4dce
+#define mmDP4_DP_SEC_PACKET_CNTL 0x4ece
+#define mmDP5_DP_SEC_PACKET_CNTL 0x4fce
+#define mmDP6_DP_SEC_PACKET_CNTL 0x54ce
+#define mmDP7_DP_SEC_PACKET_CNTL 0x56ce
+#define mmDP8_DP_SEC_PACKET_CNTL 0x57ce
+#define mmDP_MSE_RATE_CNTL 0x4acf
+#define mmDP0_DP_MSE_RATE_CNTL 0x4acf
+#define mmDP1_DP_MSE_RATE_CNTL 0x4bcf
+#define mmDP2_DP_MSE_RATE_CNTL 0x4ccf
+#define mmDP3_DP_MSE_RATE_CNTL 0x4dcf
+#define mmDP4_DP_MSE_RATE_CNTL 0x4ecf
+#define mmDP5_DP_MSE_RATE_CNTL 0x4fcf
+#define mmDP6_DP_MSE_RATE_CNTL 0x54cf
+#define mmDP7_DP_MSE_RATE_CNTL 0x56cf
+#define mmDP8_DP_MSE_RATE_CNTL 0x57cf
+#define mmDP_MSE_RATE_UPDATE 0x4ad1
+#define mmDP0_DP_MSE_RATE_UPDATE 0x4ad1
+#define mmDP1_DP_MSE_RATE_UPDATE 0x4bd1
+#define mmDP2_DP_MSE_RATE_UPDATE 0x4cd1
+#define mmDP3_DP_MSE_RATE_UPDATE 0x4dd1
+#define mmDP4_DP_MSE_RATE_UPDATE 0x4ed1
+#define mmDP5_DP_MSE_RATE_UPDATE 0x4fd1
+#define mmDP6_DP_MSE_RATE_UPDATE 0x54d1
+#define mmDP7_DP_MSE_RATE_UPDATE 0x56d1
+#define mmDP8_DP_MSE_RATE_UPDATE 0x57d1
+#define mmDP_MSE_SAT0 0x4ad2
+#define mmDP0_DP_MSE_SAT0 0x4ad2
+#define mmDP1_DP_MSE_SAT0 0x4bd2
+#define mmDP2_DP_MSE_SAT0 0x4cd2
+#define mmDP3_DP_MSE_SAT0 0x4dd2
+#define mmDP4_DP_MSE_SAT0 0x4ed2
+#define mmDP5_DP_MSE_SAT0 0x4fd2
+#define mmDP6_DP_MSE_SAT0 0x54d2
+#define mmDP7_DP_MSE_SAT0 0x56d2
+#define mmDP8_DP_MSE_SAT0 0x57d2
+#define mmDP_MSE_SAT1 0x4ad3
+#define mmDP0_DP_MSE_SAT1 0x4ad3
+#define mmDP1_DP_MSE_SAT1 0x4bd3
+#define mmDP2_DP_MSE_SAT1 0x4cd3
+#define mmDP3_DP_MSE_SAT1 0x4dd3
+#define mmDP4_DP_MSE_SAT1 0x4ed3
+#define mmDP5_DP_MSE_SAT1 0x4fd3
+#define mmDP6_DP_MSE_SAT1 0x54d3
+#define mmDP7_DP_MSE_SAT1 0x56d3
+#define mmDP8_DP_MSE_SAT1 0x57d3
+#define mmDP_MSE_SAT2 0x4ad4
+#define mmDP0_DP_MSE_SAT2 0x4ad4
+#define mmDP1_DP_MSE_SAT2 0x4bd4
+#define mmDP2_DP_MSE_SAT2 0x4cd4
+#define mmDP3_DP_MSE_SAT2 0x4dd4
+#define mmDP4_DP_MSE_SAT2 0x4ed4
+#define mmDP5_DP_MSE_SAT2 0x4fd4
+#define mmDP6_DP_MSE_SAT2 0x54d4
+#define mmDP7_DP_MSE_SAT2 0x56d4
+#define mmDP8_DP_MSE_SAT2 0x57d4
+#define mmDP_MSE_SAT_UPDATE 0x4ad5
+#define mmDP0_DP_MSE_SAT_UPDATE 0x4ad5
+#define mmDP1_DP_MSE_SAT_UPDATE 0x4bd5
+#define mmDP2_DP_MSE_SAT_UPDATE 0x4cd5
+#define mmDP3_DP_MSE_SAT_UPDATE 0x4dd5
+#define mmDP4_DP_MSE_SAT_UPDATE 0x4ed5
+#define mmDP5_DP_MSE_SAT_UPDATE 0x4fd5
+#define mmDP6_DP_MSE_SAT_UPDATE 0x54d5
+#define mmDP7_DP_MSE_SAT_UPDATE 0x56d5
+#define mmDP8_DP_MSE_SAT_UPDATE 0x57d5
+#define mmDP_MSE_LINK_TIMING 0x4ad6
+#define mmDP0_DP_MSE_LINK_TIMING 0x4ad6
+#define mmDP1_DP_MSE_LINK_TIMING 0x4bd6
+#define mmDP2_DP_MSE_LINK_TIMING 0x4cd6
+#define mmDP3_DP_MSE_LINK_TIMING 0x4dd6
+#define mmDP4_DP_MSE_LINK_TIMING 0x4ed6
+#define mmDP5_DP_MSE_LINK_TIMING 0x4fd6
+#define mmDP6_DP_MSE_LINK_TIMING 0x54d6
+#define mmDP7_DP_MSE_LINK_TIMING 0x56d6
+#define mmDP8_DP_MSE_LINK_TIMING 0x57d6
+#define mmDP_MSE_MISC_CNTL 0x4ad7
+#define mmDP0_DP_MSE_MISC_CNTL 0x4ad7
+#define mmDP1_DP_MSE_MISC_CNTL 0x4bd7
+#define mmDP2_DP_MSE_MISC_CNTL 0x4cd7
+#define mmDP3_DP_MSE_MISC_CNTL 0x4dd7
+#define mmDP4_DP_MSE_MISC_CNTL 0x4ed7
+#define mmDP5_DP_MSE_MISC_CNTL 0x4fd7
+#define mmDP6_DP_MSE_MISC_CNTL 0x54d7
+#define mmDP7_DP_MSE_MISC_CNTL 0x56d7
+#define mmDP8_DP_MSE_MISC_CNTL 0x57d7
+#define mmDP_MSE_SAT0_STATUS 0x4adf
+#define mmDP0_DP_MSE_SAT0_STATUS 0x4adf
+#define mmDP1_DP_MSE_SAT0_STATUS 0x4bdf
+#define mmDP2_DP_MSE_SAT0_STATUS 0x4cdf
+#define mmDP3_DP_MSE_SAT0_STATUS 0x4ddf
+#define mmDP4_DP_MSE_SAT0_STATUS 0x4edf
+#define mmDP5_DP_MSE_SAT0_STATUS 0x4fdf
+#define mmDP6_DP_MSE_SAT0_STATUS 0x54df
+#define mmDP7_DP_MSE_SAT0_STATUS 0x56df
+#define mmDP8_DP_MSE_SAT0_STATUS 0x57df
+#define mmDP_MSE_SAT1_STATUS 0x4ae0
+#define mmDP0_DP_MSE_SAT1_STATUS 0x4ae0
+#define mmDP1_DP_MSE_SAT1_STATUS 0x4be0
+#define mmDP2_DP_MSE_SAT1_STATUS 0x4ce0
+#define mmDP3_DP_MSE_SAT1_STATUS 0x4de0
+#define mmDP4_DP_MSE_SAT1_STATUS 0x4ee0
+#define mmDP5_DP_MSE_SAT1_STATUS 0x4fe0
+#define mmDP6_DP_MSE_SAT1_STATUS 0x54e0
+#define mmDP7_DP_MSE_SAT1_STATUS 0x56e0
+#define mmDP8_DP_MSE_SAT1_STATUS 0x57e0
+#define mmDP_MSE_SAT2_STATUS 0x4ae1
+#define mmDP0_DP_MSE_SAT2_STATUS 0x4ae1
+#define mmDP1_DP_MSE_SAT2_STATUS 0x4be1
+#define mmDP2_DP_MSE_SAT2_STATUS 0x4ce1
+#define mmDP3_DP_MSE_SAT2_STATUS 0x4de1
+#define mmDP4_DP_MSE_SAT2_STATUS 0x4ee1
+#define mmDP5_DP_MSE_SAT2_STATUS 0x4fe1
+#define mmDP6_DP_MSE_SAT2_STATUS 0x54e1
+#define mmDP7_DP_MSE_SAT2_STATUS 0x56e1
+#define mmDP8_DP_MSE_SAT2_STATUS 0x57e1
+#define mmDP_TEST_DEBUG_INDEX 0x4ad8
+#define mmDP0_DP_TEST_DEBUG_INDEX 0x4ad8
+#define mmDP1_DP_TEST_DEBUG_INDEX 0x4bd8
+#define mmDP2_DP_TEST_DEBUG_INDEX 0x4cd8
+#define mmDP3_DP_TEST_DEBUG_INDEX 0x4dd8
+#define mmDP4_DP_TEST_DEBUG_INDEX 0x4ed8
+#define mmDP5_DP_TEST_DEBUG_INDEX 0x4fd8
+#define mmDP6_DP_TEST_DEBUG_INDEX 0x54d8
+#define mmDP7_DP_TEST_DEBUG_INDEX 0x56d8
+#define mmDP8_DP_TEST_DEBUG_INDEX 0x57d8
+#define mmDP_TEST_DEBUG_DATA 0x4ad9
+#define mmDP0_DP_TEST_DEBUG_DATA 0x4ad9
+#define mmDP1_DP_TEST_DEBUG_DATA 0x4bd9
+#define mmDP2_DP_TEST_DEBUG_DATA 0x4cd9
+#define mmDP3_DP_TEST_DEBUG_DATA 0x4dd9
+#define mmDP4_DP_TEST_DEBUG_DATA 0x4ed9
+#define mmDP5_DP_TEST_DEBUG_DATA 0x4fd9
+#define mmDP6_DP_TEST_DEBUG_DATA 0x54d9
+#define mmDP7_DP_TEST_DEBUG_DATA 0x56d9
+#define mmDP8_DP_TEST_DEBUG_DATA 0x57d9
+#define mmDP_FE_TEST_DEBUG_INDEX 0x4ada
+#define mmDP0_DP_FE_TEST_DEBUG_INDEX 0x4ada
+#define mmDP1_DP_FE_TEST_DEBUG_INDEX 0x4bda
+#define mmDP2_DP_FE_TEST_DEBUG_INDEX 0x4cda
+#define mmDP3_DP_FE_TEST_DEBUG_INDEX 0x4dda
+#define mmDP4_DP_FE_TEST_DEBUG_INDEX 0x4eda
+#define mmDP5_DP_FE_TEST_DEBUG_INDEX 0x4fda
+#define mmDP6_DP_FE_TEST_DEBUG_INDEX 0x54da
+#define mmDP7_DP_FE_TEST_DEBUG_INDEX 0x56da
+#define mmDP8_DP_FE_TEST_DEBUG_INDEX 0x57da
+#define mmDP_FE_TEST_DEBUG_DATA 0x4adb
+#define mmDP0_DP_FE_TEST_DEBUG_DATA 0x4adb
+#define mmDP1_DP_FE_TEST_DEBUG_DATA 0x4bdb
+#define mmDP2_DP_FE_TEST_DEBUG_DATA 0x4cdb
+#define mmDP3_DP_FE_TEST_DEBUG_DATA 0x4ddb
+#define mmDP4_DP_FE_TEST_DEBUG_DATA 0x4edb
+#define mmDP5_DP_FE_TEST_DEBUG_DATA 0x4fdb
+#define mmDP6_DP_FE_TEST_DEBUG_DATA 0x54db
+#define mmDP7_DP_FE_TEST_DEBUG_DATA 0x56db
+#define mmDP8_DP_FE_TEST_DEBUG_DATA 0x57db
+#define mmAUX_CONTROL 0x5c00
+#define mmDP_AUX0_AUX_CONTROL 0x5c00
+#define mmDP_AUX1_AUX_CONTROL 0x5c1c
+#define mmDP_AUX2_AUX_CONTROL 0x5c38
+#define mmDP_AUX3_AUX_CONTROL 0x5c54
+#define mmDP_AUX4_AUX_CONTROL 0x5c70
+#define mmDP_AUX5_AUX_CONTROL 0x5c8c
+#define mmAUX_SW_CONTROL 0x5c01
+#define mmDP_AUX0_AUX_SW_CONTROL 0x5c01
+#define mmDP_AUX1_AUX_SW_CONTROL 0x5c1d
+#define mmDP_AUX2_AUX_SW_CONTROL 0x5c39
+#define mmDP_AUX3_AUX_SW_CONTROL 0x5c55
+#define mmDP_AUX4_AUX_SW_CONTROL 0x5c71
+#define mmDP_AUX5_AUX_SW_CONTROL 0x5c8d
+#define mmAUX_ARB_CONTROL 0x5c02
+#define mmDP_AUX0_AUX_ARB_CONTROL 0x5c02
+#define mmDP_AUX1_AUX_ARB_CONTROL 0x5c1e
+#define mmDP_AUX2_AUX_ARB_CONTROL 0x5c3a
+#define mmDP_AUX3_AUX_ARB_CONTROL 0x5c56
+#define mmDP_AUX4_AUX_ARB_CONTROL 0x5c72
+#define mmDP_AUX5_AUX_ARB_CONTROL 0x5c8e
+#define mmAUX_INTERRUPT_CONTROL 0x5c03
+#define mmDP_AUX0_AUX_INTERRUPT_CONTROL 0x5c03
+#define mmDP_AUX1_AUX_INTERRUPT_CONTROL 0x5c1f
+#define mmDP_AUX2_AUX_INTERRUPT_CONTROL 0x5c3b
+#define mmDP_AUX3_AUX_INTERRUPT_CONTROL 0x5c57
+#define mmDP_AUX4_AUX_INTERRUPT_CONTROL 0x5c73
+#define mmDP_AUX5_AUX_INTERRUPT_CONTROL 0x5c8f
+#define mmAUX_SW_STATUS 0x5c04
+#define mmDP_AUX0_AUX_SW_STATUS 0x5c04
+#define mmDP_AUX1_AUX_SW_STATUS 0x5c20
+#define mmDP_AUX2_AUX_SW_STATUS 0x5c3c
+#define mmDP_AUX3_AUX_SW_STATUS 0x5c58
+#define mmDP_AUX4_AUX_SW_STATUS 0x5c74
+#define mmDP_AUX5_AUX_SW_STATUS 0x5c90
+#define mmAUX_LS_STATUS 0x5c05
+#define mmDP_AUX0_AUX_LS_STATUS 0x5c05
+#define mmDP_AUX1_AUX_LS_STATUS 0x5c21
+#define mmDP_AUX2_AUX_LS_STATUS 0x5c3d
+#define mmDP_AUX3_AUX_LS_STATUS 0x5c59
+#define mmDP_AUX4_AUX_LS_STATUS 0x5c75
+#define mmDP_AUX5_AUX_LS_STATUS 0x5c91
+#define mmAUX_SW_DATA 0x5c06
+#define mmDP_AUX0_AUX_SW_DATA 0x5c06
+#define mmDP_AUX1_AUX_SW_DATA 0x5c22
+#define mmDP_AUX2_AUX_SW_DATA 0x5c3e
+#define mmDP_AUX3_AUX_SW_DATA 0x5c5a
+#define mmDP_AUX4_AUX_SW_DATA 0x5c76
+#define mmDP_AUX5_AUX_SW_DATA 0x5c92
+#define mmAUX_LS_DATA 0x5c07
+#define mmDP_AUX0_AUX_LS_DATA 0x5c07
+#define mmDP_AUX1_AUX_LS_DATA 0x5c23
+#define mmDP_AUX2_AUX_LS_DATA 0x5c3f
+#define mmDP_AUX3_AUX_LS_DATA 0x5c5b
+#define mmDP_AUX4_AUX_LS_DATA 0x5c77
+#define mmDP_AUX5_AUX_LS_DATA 0x5c93
+#define mmAUX_DPHY_TX_REF_CONTROL 0x5c08
+#define mmDP_AUX0_AUX_DPHY_TX_REF_CONTROL 0x5c08
+#define mmDP_AUX1_AUX_DPHY_TX_REF_CONTROL 0x5c24
+#define mmDP_AUX2_AUX_DPHY_TX_REF_CONTROL 0x5c40
+#define mmDP_AUX3_AUX_DPHY_TX_REF_CONTROL 0x5c5c
+#define mmDP_AUX4_AUX_DPHY_TX_REF_CONTROL 0x5c78
+#define mmDP_AUX5_AUX_DPHY_TX_REF_CONTROL 0x5c94
+#define mmAUX_DPHY_TX_CONTROL 0x5c09
+#define mmDP_AUX0_AUX_DPHY_TX_CONTROL 0x5c09
+#define mmDP_AUX1_AUX_DPHY_TX_CONTROL 0x5c25
+#define mmDP_AUX2_AUX_DPHY_TX_CONTROL 0x5c41
+#define mmDP_AUX3_AUX_DPHY_TX_CONTROL 0x5c5d
+#define mmDP_AUX4_AUX_DPHY_TX_CONTROL 0x5c79
+#define mmDP_AUX5_AUX_DPHY_TX_CONTROL 0x5c95
+#define mmAUX_DPHY_RX_CONTROL0 0x5c0a
+#define mmDP_AUX0_AUX_DPHY_RX_CONTROL0 0x5c0a
+#define mmDP_AUX1_AUX_DPHY_RX_CONTROL0 0x5c26
+#define mmDP_AUX2_AUX_DPHY_RX_CONTROL0 0x5c42
+#define mmDP_AUX3_AUX_DPHY_RX_CONTROL0 0x5c5e
+#define mmDP_AUX4_AUX_DPHY_RX_CONTROL0 0x5c7a
+#define mmDP_AUX5_AUX_DPHY_RX_CONTROL0 0x5c96
+#define mmAUX_DPHY_RX_CONTROL1 0x5c0b
+#define mmDP_AUX0_AUX_DPHY_RX_CONTROL1 0x5c0b
+#define mmDP_AUX1_AUX_DPHY_RX_CONTROL1 0x5c27
+#define mmDP_AUX2_AUX_DPHY_RX_CONTROL1 0x5c43
+#define mmDP_AUX3_AUX_DPHY_RX_CONTROL1 0x5c5f
+#define mmDP_AUX4_AUX_DPHY_RX_CONTROL1 0x5c7b
+#define mmDP_AUX5_AUX_DPHY_RX_CONTROL1 0x5c97
+#define mmAUX_DPHY_TX_STATUS 0x5c0c
+#define mmDP_AUX0_AUX_DPHY_TX_STATUS 0x5c0c
+#define mmDP_AUX1_AUX_DPHY_TX_STATUS 0x5c28
+#define mmDP_AUX2_AUX_DPHY_TX_STATUS 0x5c44
+#define mmDP_AUX3_AUX_DPHY_TX_STATUS 0x5c60
+#define mmDP_AUX4_AUX_DPHY_TX_STATUS 0x5c7c
+#define mmDP_AUX5_AUX_DPHY_TX_STATUS 0x5c98
+#define mmAUX_DPHY_RX_STATUS 0x5c0d
+#define mmDP_AUX0_AUX_DPHY_RX_STATUS 0x5c0d
+#define mmDP_AUX1_AUX_DPHY_RX_STATUS 0x5c29
+#define mmDP_AUX2_AUX_DPHY_RX_STATUS 0x5c45
+#define mmDP_AUX3_AUX_DPHY_RX_STATUS 0x5c61
+#define mmDP_AUX4_AUX_DPHY_RX_STATUS 0x5c7d
+#define mmDP_AUX5_AUX_DPHY_RX_STATUS 0x5c99
+#define mmAUX_GTC_SYNC_ERROR_CONTROL 0x5c0f
+#define mmDP_AUX0_AUX_GTC_SYNC_ERROR_CONTROL 0x5c0f
+#define mmDP_AUX1_AUX_GTC_SYNC_ERROR_CONTROL 0x5c2b
+#define mmDP_AUX2_AUX_GTC_SYNC_ERROR_CONTROL 0x5c47
+#define mmDP_AUX3_AUX_GTC_SYNC_ERROR_CONTROL 0x5c63
+#define mmDP_AUX4_AUX_GTC_SYNC_ERROR_CONTROL 0x5c7f
+#define mmDP_AUX5_AUX_GTC_SYNC_ERROR_CONTROL 0x5c9b
+#define mmAUX_GTC_SYNC_CONTROLLER_STATUS 0x5c10
+#define mmDP_AUX0_AUX_GTC_SYNC_CONTROLLER_STATUS 0x5c10
+#define mmDP_AUX1_AUX_GTC_SYNC_CONTROLLER_STATUS 0x5c2c
+#define mmDP_AUX2_AUX_GTC_SYNC_CONTROLLER_STATUS 0x5c48
+#define mmDP_AUX3_AUX_GTC_SYNC_CONTROLLER_STATUS 0x5c64
+#define mmDP_AUX4_AUX_GTC_SYNC_CONTROLLER_STATUS 0x5c80
+#define mmDP_AUX5_AUX_GTC_SYNC_CONTROLLER_STATUS 0x5c9c
+#define mmAUX_GTC_SYNC_STATUS 0x5c11
+#define mmDP_AUX0_AUX_GTC_SYNC_STATUS 0x5c11
+#define mmDP_AUX1_AUX_GTC_SYNC_STATUS 0x5c2d
+#define mmDP_AUX2_AUX_GTC_SYNC_STATUS 0x5c49
+#define mmDP_AUX3_AUX_GTC_SYNC_STATUS 0x5c65
+#define mmDP_AUX4_AUX_GTC_SYNC_STATUS 0x5c81
+#define mmDP_AUX5_AUX_GTC_SYNC_STATUS 0x5c9d
+#define mmAUX_TEST_DEBUG_INDEX 0x5c14
+#define mmDP_AUX0_AUX_TEST_DEBUG_INDEX 0x5c14
+#define mmDP_AUX1_AUX_TEST_DEBUG_INDEX 0x5c30
+#define mmDP_AUX2_AUX_TEST_DEBUG_INDEX 0x5c4c
+#define mmDP_AUX3_AUX_TEST_DEBUG_INDEX 0x5c68
+#define mmDP_AUX4_AUX_TEST_DEBUG_INDEX 0x5c84
+#define mmDP_AUX5_AUX_TEST_DEBUG_INDEX 0x5ca0
+#define mmAUX_TEST_DEBUG_DATA 0x5c15
+#define mmDP_AUX0_AUX_TEST_DEBUG_DATA 0x5c15
+#define mmDP_AUX1_AUX_TEST_DEBUG_DATA 0x5c31
+#define mmDP_AUX2_AUX_TEST_DEBUG_DATA 0x5c4d
+#define mmDP_AUX3_AUX_TEST_DEBUG_DATA 0x5c69
+#define mmDP_AUX4_AUX_TEST_DEBUG_DATA 0x5c85
+#define mmDP_AUX5_AUX_TEST_DEBUG_DATA 0x5ca1
+#define ixDP_AUX_DEBUG_A 0x10
+#define ixDP_AUX_DEBUG_B 0x11
+#define ixDP_AUX_DEBUG_C 0x12
+#define ixDP_AUX_DEBUG_D 0x13
+#define ixDP_AUX_DEBUG_E 0x14
+#define ixDP_AUX_DEBUG_F 0x15
+#define ixDP_AUX_DEBUG_G 0x16
+#define ixDP_AUX_DEBUG_H 0x17
+#define ixDP_AUX_DEBUG_I 0x18
+#define ixDP_AUX_DEBUG_J 0x19
+#define ixDP_AUX_DEBUG_K 0x1a
+#define ixDP_AUX_DEBUG_L 0x1b
+#define ixDP_AUX_DEBUG_M 0x1c
+#define ixDP_AUX_DEBUG_N 0x1d
+#define ixDP_AUX_DEBUG_O 0x1e
+#define ixDP_AUX_DEBUG_P 0x1f
+#define ixDP_AUX_DEBUG_Q 0x20
+#define mmDVO_ENABLE 0x16a0
+#define mmDVO_SOURCE_SELECT 0x16a1
+#define mmDVO_OUTPUT 0x16a2
+#define mmDVO_CONTROL 0x16a3
+#define mmDVO_CRC_EN 0x16a4
+#define mmDVO_CRC2_SIG_MASK 0x16a5
+#define mmDVO_CRC2_SIG_RESULT 0x16a6
+#define mmDVO_FIFO_ERROR_STATUS 0x16a7
+#define mmDVO_TEST_DEBUG_INDEX 0x16a8
+#define mmDVO_TEST_DEBUG_DATA 0x16a9
+#define mmFBC_CNTL 0x280
+#define mmFBC_IDLE_FORCE_CLEAR_MASK 0x282
+#define mmFBC_START_STOP_DELAY 0x283
+#define mmFBC_COMP_CNTL 0x284
+#define mmFBC_COMP_MODE 0x285
+#define mmFBC_DEBUG0 0x286
+#define mmFBC_DEBUG1 0x287
+#define mmFBC_DEBUG2 0x288
+#define mmFBC_IND_LUT0 0x289
+#define mmFBC_IND_LUT1 0x28a
+#define mmFBC_IND_LUT2 0x28b
+#define mmFBC_IND_LUT3 0x28c
+#define mmFBC_IND_LUT4 0x28d
+#define mmFBC_IND_LUT5 0x28e
+#define mmFBC_IND_LUT6 0x28f
+#define mmFBC_IND_LUT7 0x290
+#define mmFBC_IND_LUT8 0x291
+#define mmFBC_IND_LUT9 0x292
+#define mmFBC_IND_LUT10 0x293
+#define mmFBC_IND_LUT11 0x294
+#define mmFBC_IND_LUT12 0x295
+#define mmFBC_IND_LUT13 0x296
+#define mmFBC_IND_LUT14 0x297
+#define mmFBC_IND_LUT15 0x298
+#define mmFBC_CSM_REGION_OFFSET_01 0x299
+#define mmFBC_CSM_REGION_OFFSET_23 0x29a
+#define mmFBC_CLIENT_REGION_MASK 0x29b
+#define mmFBC_DEBUG_COMP 0x29c
+#define mmFBC_DEBUG_CSR 0x29d
+#define mmFBC_DEBUG_CSR_RDATA 0x29e
+#define mmFBC_DEBUG_CSR_WDATA 0x29f
+#define mmFBC_DEBUG_CSR_RDATA_HI 0x2a0
+#define mmFBC_DEBUG_CSR_WDATA_HI 0x2a1
+#define mmFBC_MISC 0x2a2
+#define mmFBC_STATUS 0x2a3
+#define mmFBC_ALPHA_CNTL 0x2a6
+#define mmFBC_ALPHA_RGB_OVERRIDE 0x2a7
+#define mmFBC_TEST_DEBUG_INDEX 0x2a4
+#define mmFBC_TEST_DEBUG_DATA 0x2a5
+#define mmFMT_CLAMP_COMPONENT_R 0x1be8
+#define mmFMT0_FMT_CLAMP_COMPONENT_R 0x1be8
+#define mmFMT1_FMT_CLAMP_COMPONENT_R 0x1de8
+#define mmFMT2_FMT_CLAMP_COMPONENT_R 0x1fe8
+#define mmFMT3_FMT_CLAMP_COMPONENT_R 0x41e8
+#define mmFMT4_FMT_CLAMP_COMPONENT_R 0x43e8
+#define mmFMT5_FMT_CLAMP_COMPONENT_R 0x45e8
+#define mmFMT_CLAMP_COMPONENT_G 0x1be9
+#define mmFMT0_FMT_CLAMP_COMPONENT_G 0x1be9
+#define mmFMT1_FMT_CLAMP_COMPONENT_G 0x1de9
+#define mmFMT2_FMT_CLAMP_COMPONENT_G 0x1fe9
+#define mmFMT3_FMT_CLAMP_COMPONENT_G 0x41e9
+#define mmFMT4_FMT_CLAMP_COMPONENT_G 0x43e9
+#define mmFMT5_FMT_CLAMP_COMPONENT_G 0x45e9
+#define mmFMT_CLAMP_COMPONENT_B 0x1bea
+#define mmFMT0_FMT_CLAMP_COMPONENT_B 0x1bea
+#define mmFMT1_FMT_CLAMP_COMPONENT_B 0x1dea
+#define mmFMT2_FMT_CLAMP_COMPONENT_B 0x1fea
+#define mmFMT3_FMT_CLAMP_COMPONENT_B 0x41ea
+#define mmFMT4_FMT_CLAMP_COMPONENT_B 0x43ea
+#define mmFMT5_FMT_CLAMP_COMPONENT_B 0x45ea
+#define mmFMT_DYNAMIC_EXP_CNTL 0x1bed
+#define mmFMT0_FMT_DYNAMIC_EXP_CNTL 0x1bed
+#define mmFMT1_FMT_DYNAMIC_EXP_CNTL 0x1ded
+#define mmFMT2_FMT_DYNAMIC_EXP_CNTL 0x1fed
+#define mmFMT3_FMT_DYNAMIC_EXP_CNTL 0x41ed
+#define mmFMT4_FMT_DYNAMIC_EXP_CNTL 0x43ed
+#define mmFMT5_FMT_DYNAMIC_EXP_CNTL 0x45ed
+#define mmFMT_CONTROL 0x1bee
+#define mmFMT0_FMT_CONTROL 0x1bee
+#define mmFMT1_FMT_CONTROL 0x1dee
+#define mmFMT2_FMT_CONTROL 0x1fee
+#define mmFMT3_FMT_CONTROL 0x41ee
+#define mmFMT4_FMT_CONTROL 0x43ee
+#define mmFMT5_FMT_CONTROL 0x45ee
+#define mmFMT_BIT_DEPTH_CONTROL 0x1bf2
+#define mmFMT0_FMT_BIT_DEPTH_CONTROL 0x1bf2
+#define mmFMT1_FMT_BIT_DEPTH_CONTROL 0x1df2
+#define mmFMT2_FMT_BIT_DEPTH_CONTROL 0x1ff2
+#define mmFMT3_FMT_BIT_DEPTH_CONTROL 0x41f2
+#define mmFMT4_FMT_BIT_DEPTH_CONTROL 0x43f2
+#define mmFMT5_FMT_BIT_DEPTH_CONTROL 0x45f2
+#define mmFMT_DITHER_RAND_R_SEED 0x1bf3
+#define mmFMT0_FMT_DITHER_RAND_R_SEED 0x1bf3
+#define mmFMT1_FMT_DITHER_RAND_R_SEED 0x1df3
+#define mmFMT2_FMT_DITHER_RAND_R_SEED 0x1ff3
+#define mmFMT3_FMT_DITHER_RAND_R_SEED 0x41f3
+#define mmFMT4_FMT_DITHER_RAND_R_SEED 0x43f3
+#define mmFMT5_FMT_DITHER_RAND_R_SEED 0x45f3
+#define mmFMT_DITHER_RAND_G_SEED 0x1bf4
+#define mmFMT0_FMT_DITHER_RAND_G_SEED 0x1bf4
+#define mmFMT1_FMT_DITHER_RAND_G_SEED 0x1df4
+#define mmFMT2_FMT_DITHER_RAND_G_SEED 0x1ff4
+#define mmFMT3_FMT_DITHER_RAND_G_SEED 0x41f4
+#define mmFMT4_FMT_DITHER_RAND_G_SEED 0x43f4
+#define mmFMT5_FMT_DITHER_RAND_G_SEED 0x45f4
+#define mmFMT_DITHER_RAND_B_SEED 0x1bf5
+#define mmFMT0_FMT_DITHER_RAND_B_SEED 0x1bf5
+#define mmFMT1_FMT_DITHER_RAND_B_SEED 0x1df5
+#define mmFMT2_FMT_DITHER_RAND_B_SEED 0x1ff5
+#define mmFMT3_FMT_DITHER_RAND_B_SEED 0x41f5
+#define mmFMT4_FMT_DITHER_RAND_B_SEED 0x43f5
+#define mmFMT5_FMT_DITHER_RAND_B_SEED 0x45f5
+#define mmFMT_TEMPORAL_DITHER_PATTERN_CONTROL 0x1bf6
+#define mmFMT0_FMT_TEMPORAL_DITHER_PATTERN_CONTROL 0x1bf6
+#define mmFMT1_FMT_TEMPORAL_DITHER_PATTERN_CONTROL 0x1df6
+#define mmFMT2_FMT_TEMPORAL_DITHER_PATTERN_CONTROL 0x1ff6
+#define mmFMT3_FMT_TEMPORAL_DITHER_PATTERN_CONTROL 0x41f6
+#define mmFMT4_FMT_TEMPORAL_DITHER_PATTERN_CONTROL 0x43f6
+#define mmFMT5_FMT_TEMPORAL_DITHER_PATTERN_CONTROL 0x45f6
+#define mmFMT_TEMPORAL_DITHER_PROGRAMMABLE_PATTERN_S_MATRIX 0x1bf7
+#define mmFMT0_FMT_TEMPORAL_DITHER_PROGRAMMABLE_PATTERN_S_MATRIX 0x1bf7
+#define mmFMT1_FMT_TEMPORAL_DITHER_PROGRAMMABLE_PATTERN_S_MATRIX 0x1df7
+#define mmFMT2_FMT_TEMPORAL_DITHER_PROGRAMMABLE_PATTERN_S_MATRIX 0x1ff7
+#define mmFMT3_FMT_TEMPORAL_DITHER_PROGRAMMABLE_PATTERN_S_MATRIX 0x41f7
+#define mmFMT4_FMT_TEMPORAL_DITHER_PROGRAMMABLE_PATTERN_S_MATRIX 0x43f7
+#define mmFMT5_FMT_TEMPORAL_DITHER_PROGRAMMABLE_PATTERN_S_MATRIX 0x45f7
+#define mmFMT_TEMPORAL_DITHER_PROGRAMMABLE_PATTERN_T_MATRIX 0x1bf8
+#define mmFMT0_FMT_TEMPORAL_DITHER_PROGRAMMABLE_PATTERN_T_MATRIX 0x1bf8
+#define mmFMT1_FMT_TEMPORAL_DITHER_PROGRAMMABLE_PATTERN_T_MATRIX 0x1df8
+#define mmFMT2_FMT_TEMPORAL_DITHER_PROGRAMMABLE_PATTERN_T_MATRIX 0x1ff8
+#define mmFMT3_FMT_TEMPORAL_DITHER_PROGRAMMABLE_PATTERN_T_MATRIX 0x41f8
+#define mmFMT4_FMT_TEMPORAL_DITHER_PROGRAMMABLE_PATTERN_T_MATRIX 0x43f8
+#define mmFMT5_FMT_TEMPORAL_DITHER_PROGRAMMABLE_PATTERN_T_MATRIX 0x45f8
+#define mmFMT_CLAMP_CNTL 0x1bf9
+#define mmFMT0_FMT_CLAMP_CNTL 0x1bf9
+#define mmFMT1_FMT_CLAMP_CNTL 0x1df9
+#define mmFMT2_FMT_CLAMP_CNTL 0x1ff9
+#define mmFMT3_FMT_CLAMP_CNTL 0x41f9
+#define mmFMT4_FMT_CLAMP_CNTL 0x43f9
+#define mmFMT5_FMT_CLAMP_CNTL 0x45f9
+#define mmFMT_CRC_CNTL 0x1bfa
+#define mmFMT0_FMT_CRC_CNTL 0x1bfa
+#define mmFMT1_FMT_CRC_CNTL 0x1dfa
+#define mmFMT2_FMT_CRC_CNTL 0x1ffa
+#define mmFMT3_FMT_CRC_CNTL 0x41fa
+#define mmFMT4_FMT_CRC_CNTL 0x43fa
+#define mmFMT5_FMT_CRC_CNTL 0x45fa
+#define mmFMT_CRC_SIG_RED_GREEN_MASK 0x1bfb
+#define mmFMT0_FMT_CRC_SIG_RED_GREEN_MASK 0x1bfb
+#define mmFMT1_FMT_CRC_SIG_RED_GREEN_MASK 0x1dfb
+#define mmFMT2_FMT_CRC_SIG_RED_GREEN_MASK 0x1ffb
+#define mmFMT3_FMT_CRC_SIG_RED_GREEN_MASK 0x41fb
+#define mmFMT4_FMT_CRC_SIG_RED_GREEN_MASK 0x43fb
+#define mmFMT5_FMT_CRC_SIG_RED_GREEN_MASK 0x45fb
+#define mmFMT_CRC_SIG_BLUE_CONTROL_MASK 0x1bfc
+#define mmFMT0_FMT_CRC_SIG_BLUE_CONTROL_MASK 0x1bfc
+#define mmFMT1_FMT_CRC_SIG_BLUE_CONTROL_MASK 0x1dfc
+#define mmFMT2_FMT_CRC_SIG_BLUE_CONTROL_MASK 0x1ffc
+#define mmFMT3_FMT_CRC_SIG_BLUE_CONTROL_MASK 0x41fc
+#define mmFMT4_FMT_CRC_SIG_BLUE_CONTROL_MASK 0x43fc
+#define mmFMT5_FMT_CRC_SIG_BLUE_CONTROL_MASK 0x45fc
+#define mmFMT_CRC_SIG_RED_GREEN 0x1bfd
+#define mmFMT0_FMT_CRC_SIG_RED_GREEN 0x1bfd
+#define mmFMT1_FMT_CRC_SIG_RED_GREEN 0x1dfd
+#define mmFMT2_FMT_CRC_SIG_RED_GREEN 0x1ffd
+#define mmFMT3_FMT_CRC_SIG_RED_GREEN 0x41fd
+#define mmFMT4_FMT_CRC_SIG_RED_GREEN 0x43fd
+#define mmFMT5_FMT_CRC_SIG_RED_GREEN 0x45fd
+#define mmFMT_CRC_SIG_BLUE_CONTROL 0x1bfe
+#define mmFMT0_FMT_CRC_SIG_BLUE_CONTROL 0x1bfe
+#define mmFMT1_FMT_CRC_SIG_BLUE_CONTROL 0x1dfe
+#define mmFMT2_FMT_CRC_SIG_BLUE_CONTROL 0x1ffe
+#define mmFMT3_FMT_CRC_SIG_BLUE_CONTROL 0x41fe
+#define mmFMT4_FMT_CRC_SIG_BLUE_CONTROL 0x43fe
+#define mmFMT5_FMT_CRC_SIG_BLUE_CONTROL 0x45fe
+#define mmFMT_DEBUG_CNTL 0x1bff
+#define mmFMT0_FMT_DEBUG_CNTL 0x1bff
+#define mmFMT1_FMT_DEBUG_CNTL 0x1dff
+#define mmFMT2_FMT_DEBUG_CNTL 0x1fff
+#define mmFMT3_FMT_DEBUG_CNTL 0x41ff
+#define mmFMT4_FMT_DEBUG_CNTL 0x43ff
+#define mmFMT5_FMT_DEBUG_CNTL 0x45ff
+#define mmFMT_SIDE_BY_SIDE_STEREO_CONTROL 0x1bf0
+#define mmFMT0_FMT_SIDE_BY_SIDE_STEREO_CONTROL 0x1bf0
+#define mmFMT1_FMT_SIDE_BY_SIDE_STEREO_CONTROL 0x1df0
+#define mmFMT2_FMT_SIDE_BY_SIDE_STEREO_CONTROL 0x1ff0
+#define mmFMT3_FMT_SIDE_BY_SIDE_STEREO_CONTROL 0x41f0
+#define mmFMT4_FMT_SIDE_BY_SIDE_STEREO_CONTROL 0x43f0
+#define mmFMT5_FMT_SIDE_BY_SIDE_STEREO_CONTROL 0x45f0
+#define mmFMT_420_HBLANK_EARLY_START 0x1bf1
+#define mmFMT0_FMT_420_HBLANK_EARLY_START 0x1bf1
+#define mmFMT1_FMT_420_HBLANK_EARLY_START 0x1df1
+#define mmFMT2_FMT_420_HBLANK_EARLY_START 0x1ff1
+#define mmFMT3_FMT_420_HBLANK_EARLY_START 0x41f1
+#define mmFMT4_FMT_420_HBLANK_EARLY_START 0x43f1
+#define mmFMT5_FMT_420_HBLANK_EARLY_START 0x45f1
+#define mmFMT_TEST_DEBUG_INDEX 0x1beb
+#define mmFMT0_FMT_TEST_DEBUG_INDEX 0x1beb
+#define mmFMT1_FMT_TEST_DEBUG_INDEX 0x1deb
+#define mmFMT2_FMT_TEST_DEBUG_INDEX 0x1feb
+#define mmFMT3_FMT_TEST_DEBUG_INDEX 0x41eb
+#define mmFMT4_FMT_TEST_DEBUG_INDEX 0x43eb
+#define mmFMT5_FMT_TEST_DEBUG_INDEX 0x45eb
+#define mmFMT_TEST_DEBUG_DATA 0x1bec
+#define mmFMT0_FMT_TEST_DEBUG_DATA 0x1bec
+#define mmFMT1_FMT_TEST_DEBUG_DATA 0x1dec
+#define mmFMT2_FMT_TEST_DEBUG_DATA 0x1fec
+#define mmFMT3_FMT_TEST_DEBUG_DATA 0x41ec
+#define mmFMT4_FMT_TEST_DEBUG_DATA 0x43ec
+#define mmFMT5_FMT_TEST_DEBUG_DATA 0x45ec
+#define ixFMT_DEBUG0 0x1
+#define ixFMT_DEBUG1 0x2
+#define ixFMT_DEBUG2 0x3
+#define ixFMT_DEBUG3 0x4
+#define ixFMT_DEBUG_ID 0x0
+#define mmLB_DATA_FORMAT 0x1ac0
+#define mmLB0_LB_DATA_FORMAT 0x1ac0
+#define mmLB1_LB_DATA_FORMAT 0x1cc0
+#define mmLB2_LB_DATA_FORMAT 0x1ec0
+#define mmLB3_LB_DATA_FORMAT 0x40c0
+#define mmLB4_LB_DATA_FORMAT 0x42c0
+#define mmLB5_LB_DATA_FORMAT 0x44c0
+#define mmLB_MEMORY_CTRL 0x1ac1
+#define mmLB0_LB_MEMORY_CTRL 0x1ac1
+#define mmLB1_LB_MEMORY_CTRL 0x1cc1
+#define mmLB2_LB_MEMORY_CTRL 0x1ec1
+#define mmLB3_LB_MEMORY_CTRL 0x40c1
+#define mmLB4_LB_MEMORY_CTRL 0x42c1
+#define mmLB5_LB_MEMORY_CTRL 0x44c1
+#define mmLB_MEMORY_SIZE_STATUS 0x1ac2
+#define mmLB0_LB_MEMORY_SIZE_STATUS 0x1ac2
+#define mmLB1_LB_MEMORY_SIZE_STATUS 0x1cc2
+#define mmLB2_LB_MEMORY_SIZE_STATUS 0x1ec2
+#define mmLB3_LB_MEMORY_SIZE_STATUS 0x40c2
+#define mmLB4_LB_MEMORY_SIZE_STATUS 0x42c2
+#define mmLB5_LB_MEMORY_SIZE_STATUS 0x44c2
+#define mmLB_DESKTOP_HEIGHT 0x1ac3
+#define mmLB0_LB_DESKTOP_HEIGHT 0x1ac3
+#define mmLB1_LB_DESKTOP_HEIGHT 0x1cc3
+#define mmLB2_LB_DESKTOP_HEIGHT 0x1ec3
+#define mmLB3_LB_DESKTOP_HEIGHT 0x40c3
+#define mmLB4_LB_DESKTOP_HEIGHT 0x42c3
+#define mmLB5_LB_DESKTOP_HEIGHT 0x44c3
+#define mmLB_VLINE_START_END 0x1ac4
+#define mmLB0_LB_VLINE_START_END 0x1ac4
+#define mmLB1_LB_VLINE_START_END 0x1cc4
+#define mmLB2_LB_VLINE_START_END 0x1ec4
+#define mmLB3_LB_VLINE_START_END 0x40c4
+#define mmLB4_LB_VLINE_START_END 0x42c4
+#define mmLB5_LB_VLINE_START_END 0x44c4
+#define mmLB_VLINE2_START_END 0x1ac5
+#define mmLB0_LB_VLINE2_START_END 0x1ac5
+#define mmLB1_LB_VLINE2_START_END 0x1cc5
+#define mmLB2_LB_VLINE2_START_END 0x1ec5
+#define mmLB3_LB_VLINE2_START_END 0x40c5
+#define mmLB4_LB_VLINE2_START_END 0x42c5
+#define mmLB5_LB_VLINE2_START_END 0x44c5
+#define mmLB_V_COUNTER 0x1ac6
+#define mmLB0_LB_V_COUNTER 0x1ac6
+#define mmLB1_LB_V_COUNTER 0x1cc6
+#define mmLB2_LB_V_COUNTER 0x1ec6
+#define mmLB3_LB_V_COUNTER 0x40c6
+#define mmLB4_LB_V_COUNTER 0x42c6
+#define mmLB5_LB_V_COUNTER 0x44c6
+#define mmLB_SNAPSHOT_V_COUNTER 0x1ac7
+#define mmLB0_LB_SNAPSHOT_V_COUNTER 0x1ac7
+#define mmLB1_LB_SNAPSHOT_V_COUNTER 0x1cc7
+#define mmLB2_LB_SNAPSHOT_V_COUNTER 0x1ec7
+#define mmLB3_LB_SNAPSHOT_V_COUNTER 0x40c7
+#define mmLB4_LB_SNAPSHOT_V_COUNTER 0x42c7
+#define mmLB5_LB_SNAPSHOT_V_COUNTER 0x44c7
+#define mmLB_INTERRUPT_MASK 0x1ac8
+#define mmLB0_LB_INTERRUPT_MASK 0x1ac8
+#define mmLB1_LB_INTERRUPT_MASK 0x1cc8
+#define mmLB2_LB_INTERRUPT_MASK 0x1ec8
+#define mmLB3_LB_INTERRUPT_MASK 0x40c8
+#define mmLB4_LB_INTERRUPT_MASK 0x42c8
+#define mmLB5_LB_INTERRUPT_MASK 0x44c8
+#define mmLB_VLINE_STATUS 0x1ac9
+#define mmLB0_LB_VLINE_STATUS 0x1ac9
+#define mmLB1_LB_VLINE_STATUS 0x1cc9
+#define mmLB2_LB_VLINE_STATUS 0x1ec9
+#define mmLB3_LB_VLINE_STATUS 0x40c9
+#define mmLB4_LB_VLINE_STATUS 0x42c9
+#define mmLB5_LB_VLINE_STATUS 0x44c9
+#define mmLB_VLINE2_STATUS 0x1aca
+#define mmLB0_LB_VLINE2_STATUS 0x1aca
+#define mmLB1_LB_VLINE2_STATUS 0x1cca
+#define mmLB2_LB_VLINE2_STATUS 0x1eca
+#define mmLB3_LB_VLINE2_STATUS 0x40ca
+#define mmLB4_LB_VLINE2_STATUS 0x42ca
+#define mmLB5_LB_VLINE2_STATUS 0x44ca
+#define mmLB_VBLANK_STATUS 0x1acb
+#define mmLB0_LB_VBLANK_STATUS 0x1acb
+#define mmLB1_LB_VBLANK_STATUS 0x1ccb
+#define mmLB2_LB_VBLANK_STATUS 0x1ecb
+#define mmLB3_LB_VBLANK_STATUS 0x40cb
+#define mmLB4_LB_VBLANK_STATUS 0x42cb
+#define mmLB5_LB_VBLANK_STATUS 0x44cb
+#define mmLB_SYNC_RESET_SEL 0x1acc
+#define mmLB0_LB_SYNC_RESET_SEL 0x1acc
+#define mmLB1_LB_SYNC_RESET_SEL 0x1ccc
+#define mmLB2_LB_SYNC_RESET_SEL 0x1ecc
+#define mmLB3_LB_SYNC_RESET_SEL 0x40cc
+#define mmLB4_LB_SYNC_RESET_SEL 0x42cc
+#define mmLB5_LB_SYNC_RESET_SEL 0x44cc
+#define mmLB_BLACK_KEYER_R_CR 0x1acd
+#define mmLB0_LB_BLACK_KEYER_R_CR 0x1acd
+#define mmLB1_LB_BLACK_KEYER_R_CR 0x1ccd
+#define mmLB2_LB_BLACK_KEYER_R_CR 0x1ecd
+#define mmLB3_LB_BLACK_KEYER_R_CR 0x40cd
+#define mmLB4_LB_BLACK_KEYER_R_CR 0x42cd
+#define mmLB5_LB_BLACK_KEYER_R_CR 0x44cd
+#define mmLB_BLACK_KEYER_G_Y 0x1ace
+#define mmLB0_LB_BLACK_KEYER_G_Y 0x1ace
+#define mmLB1_LB_BLACK_KEYER_G_Y 0x1cce
+#define mmLB2_LB_BLACK_KEYER_G_Y 0x1ece
+#define mmLB3_LB_BLACK_KEYER_G_Y 0x40ce
+#define mmLB4_LB_BLACK_KEYER_G_Y 0x42ce
+#define mmLB5_LB_BLACK_KEYER_G_Y 0x44ce
+#define mmLB_BLACK_KEYER_B_CB 0x1acf
+#define mmLB0_LB_BLACK_KEYER_B_CB 0x1acf
+#define mmLB1_LB_BLACK_KEYER_B_CB 0x1ccf
+#define mmLB2_LB_BLACK_KEYER_B_CB 0x1ecf
+#define mmLB3_LB_BLACK_KEYER_B_CB 0x40cf
+#define mmLB4_LB_BLACK_KEYER_B_CB 0x42cf
+#define mmLB5_LB_BLACK_KEYER_B_CB 0x44cf
+#define mmLB_KEYER_COLOR_CTRL 0x1ad0
+#define mmLB0_LB_KEYER_COLOR_CTRL 0x1ad0
+#define mmLB1_LB_KEYER_COLOR_CTRL 0x1cd0
+#define mmLB2_LB_KEYER_COLOR_CTRL 0x1ed0
+#define mmLB3_LB_KEYER_COLOR_CTRL 0x40d0
+#define mmLB4_LB_KEYER_COLOR_CTRL 0x42d0
+#define mmLB5_LB_KEYER_COLOR_CTRL 0x44d0
+#define mmLB_KEYER_COLOR_R_CR 0x1ad1
+#define mmLB0_LB_KEYER_COLOR_R_CR 0x1ad1
+#define mmLB1_LB_KEYER_COLOR_R_CR 0x1cd1
+#define mmLB2_LB_KEYER_COLOR_R_CR 0x1ed1
+#define mmLB3_LB_KEYER_COLOR_R_CR 0x40d1
+#define mmLB4_LB_KEYER_COLOR_R_CR 0x42d1
+#define mmLB5_LB_KEYER_COLOR_R_CR 0x44d1
+#define mmLB_KEYER_COLOR_G_Y 0x1ad2
+#define mmLB0_LB_KEYER_COLOR_G_Y 0x1ad2
+#define mmLB1_LB_KEYER_COLOR_G_Y 0x1cd2
+#define mmLB2_LB_KEYER_COLOR_G_Y 0x1ed2
+#define mmLB3_LB_KEYER_COLOR_G_Y 0x40d2
+#define mmLB4_LB_KEYER_COLOR_G_Y 0x42d2
+#define mmLB5_LB_KEYER_COLOR_G_Y 0x44d2
+#define mmLB_KEYER_COLOR_B_CB 0x1ad3
+#define mmLB0_LB_KEYER_COLOR_B_CB 0x1ad3
+#define mmLB1_LB_KEYER_COLOR_B_CB 0x1cd3
+#define mmLB2_LB_KEYER_COLOR_B_CB 0x1ed3
+#define mmLB3_LB_KEYER_COLOR_B_CB 0x40d3
+#define mmLB4_LB_KEYER_COLOR_B_CB 0x42d3
+#define mmLB5_LB_KEYER_COLOR_B_CB 0x44d3
+#define mmLB_KEYER_COLOR_REP_R_CR 0x1ad4
+#define mmLB0_LB_KEYER_COLOR_REP_R_CR 0x1ad4
+#define mmLB1_LB_KEYER_COLOR_REP_R_CR 0x1cd4
+#define mmLB2_LB_KEYER_COLOR_REP_R_CR 0x1ed4
+#define mmLB3_LB_KEYER_COLOR_REP_R_CR 0x40d4
+#define mmLB4_LB_KEYER_COLOR_REP_R_CR 0x42d4
+#define mmLB5_LB_KEYER_COLOR_REP_R_CR 0x44d4
+#define mmLB_KEYER_COLOR_REP_G_Y 0x1ad5
+#define mmLB0_LB_KEYER_COLOR_REP_G_Y 0x1ad5
+#define mmLB1_LB_KEYER_COLOR_REP_G_Y 0x1cd5
+#define mmLB2_LB_KEYER_COLOR_REP_G_Y 0x1ed5
+#define mmLB3_LB_KEYER_COLOR_REP_G_Y 0x40d5
+#define mmLB4_LB_KEYER_COLOR_REP_G_Y 0x42d5
+#define mmLB5_LB_KEYER_COLOR_REP_G_Y 0x44d5
+#define mmLB_KEYER_COLOR_REP_B_CB 0x1ad6
+#define mmLB0_LB_KEYER_COLOR_REP_B_CB 0x1ad6
+#define mmLB1_LB_KEYER_COLOR_REP_B_CB 0x1cd6
+#define mmLB2_LB_KEYER_COLOR_REP_B_CB 0x1ed6
+#define mmLB3_LB_KEYER_COLOR_REP_B_CB 0x40d6
+#define mmLB4_LB_KEYER_COLOR_REP_B_CB 0x42d6
+#define mmLB5_LB_KEYER_COLOR_REP_B_CB 0x44d6
+#define mmLB_BUFFER_LEVEL_STATUS 0x1ad7
+#define mmLB0_LB_BUFFER_LEVEL_STATUS 0x1ad7
+#define mmLB1_LB_BUFFER_LEVEL_STATUS 0x1cd7
+#define mmLB2_LB_BUFFER_LEVEL_STATUS 0x1ed7
+#define mmLB3_LB_BUFFER_LEVEL_STATUS 0x40d7
+#define mmLB4_LB_BUFFER_LEVEL_STATUS 0x42d7
+#define mmLB5_LB_BUFFER_LEVEL_STATUS 0x44d7
+#define mmLB_BUFFER_URGENCY_CTRL 0x1ad8
+#define mmLB0_LB_BUFFER_URGENCY_CTRL 0x1ad8
+#define mmLB1_LB_BUFFER_URGENCY_CTRL 0x1cd8
+#define mmLB2_LB_BUFFER_URGENCY_CTRL 0x1ed8
+#define mmLB3_LB_BUFFER_URGENCY_CTRL 0x40d8
+#define mmLB4_LB_BUFFER_URGENCY_CTRL 0x42d8
+#define mmLB5_LB_BUFFER_URGENCY_CTRL 0x44d8
+#define mmLB_BUFFER_URGENCY_STATUS 0x1ad9
+#define mmLB0_LB_BUFFER_URGENCY_STATUS 0x1ad9
+#define mmLB1_LB_BUFFER_URGENCY_STATUS 0x1cd9
+#define mmLB2_LB_BUFFER_URGENCY_STATUS 0x1ed9
+#define mmLB3_LB_BUFFER_URGENCY_STATUS 0x40d9
+#define mmLB4_LB_BUFFER_URGENCY_STATUS 0x42d9
+#define mmLB5_LB_BUFFER_URGENCY_STATUS 0x44d9
+#define mmLB_BUFFER_STATUS 0x1ada
+#define mmLB0_LB_BUFFER_STATUS 0x1ada
+#define mmLB1_LB_BUFFER_STATUS 0x1cda
+#define mmLB2_LB_BUFFER_STATUS 0x1eda
+#define mmLB3_LB_BUFFER_STATUS 0x40da
+#define mmLB4_LB_BUFFER_STATUS 0x42da
+#define mmLB5_LB_BUFFER_STATUS 0x44da
+#define mmLB_NO_OUTSTANDING_REQ_STATUS 0x1adc
+#define mmLB0_LB_NO_OUTSTANDING_REQ_STATUS 0x1adc
+#define mmLB1_LB_NO_OUTSTANDING_REQ_STATUS 0x1cdc
+#define mmLB2_LB_NO_OUTSTANDING_REQ_STATUS 0x1edc
+#define mmLB3_LB_NO_OUTSTANDING_REQ_STATUS 0x40dc
+#define mmLB4_LB_NO_OUTSTANDING_REQ_STATUS 0x42dc
+#define mmLB5_LB_NO_OUTSTANDING_REQ_STATUS 0x44dc
+#define mmMVP_AFR_FLIP_MODE 0x1ae0
+#define mmLB0_MVP_AFR_FLIP_MODE 0x1ae0
+#define mmLB1_MVP_AFR_FLIP_MODE 0x1ce0
+#define mmLB2_MVP_AFR_FLIP_MODE 0x1ee0
+#define mmLB3_MVP_AFR_FLIP_MODE 0x40e0
+#define mmLB4_MVP_AFR_FLIP_MODE 0x42e0
+#define mmLB5_MVP_AFR_FLIP_MODE 0x44e0
+#define mmMVP_AFR_FLIP_FIFO_CNTL 0x1ae1
+#define mmLB0_MVP_AFR_FLIP_FIFO_CNTL 0x1ae1
+#define mmLB1_MVP_AFR_FLIP_FIFO_CNTL 0x1ce1
+#define mmLB2_MVP_AFR_FLIP_FIFO_CNTL 0x1ee1
+#define mmLB3_MVP_AFR_FLIP_FIFO_CNTL 0x40e1
+#define mmLB4_MVP_AFR_FLIP_FIFO_CNTL 0x42e1
+#define mmLB5_MVP_AFR_FLIP_FIFO_CNTL 0x44e1
+#define mmMVP_FLIP_LINE_NUM_INSERT 0x1ae2
+#define mmLB0_MVP_FLIP_LINE_NUM_INSERT 0x1ae2
+#define mmLB1_MVP_FLIP_LINE_NUM_INSERT 0x1ce2
+#define mmLB2_MVP_FLIP_LINE_NUM_INSERT 0x1ee2
+#define mmLB3_MVP_FLIP_LINE_NUM_INSERT 0x40e2
+#define mmLB4_MVP_FLIP_LINE_NUM_INSERT 0x42e2
+#define mmLB5_MVP_FLIP_LINE_NUM_INSERT 0x44e2
+#define mmDC_MVP_LB_CONTROL 0x1ae3
+#define mmLB0_DC_MVP_LB_CONTROL 0x1ae3
+#define mmLB1_DC_MVP_LB_CONTROL 0x1ce3
+#define mmLB2_DC_MVP_LB_CONTROL 0x1ee3
+#define mmLB3_DC_MVP_LB_CONTROL 0x40e3
+#define mmLB4_DC_MVP_LB_CONTROL 0x42e3
+#define mmLB5_DC_MVP_LB_CONTROL 0x44e3
+#define mmLB_DEBUG 0x1ae4
+#define mmLB0_LB_DEBUG 0x1ae4
+#define mmLB1_LB_DEBUG 0x1ce4
+#define mmLB2_LB_DEBUG 0x1ee4
+#define mmLB3_LB_DEBUG 0x40e4
+#define mmLB4_LB_DEBUG 0x42e4
+#define mmLB5_LB_DEBUG 0x44e4
+#define mmLB_DEBUG2 0x1ae5
+#define mmLB0_LB_DEBUG2 0x1ae5
+#define mmLB1_LB_DEBUG2 0x1ce5
+#define mmLB2_LB_DEBUG2 0x1ee5
+#define mmLB3_LB_DEBUG2 0x40e5
+#define mmLB4_LB_DEBUG2 0x42e5
+#define mmLB5_LB_DEBUG2 0x44e5
+#define mmLB_DEBUG3 0x1ae6
+#define mmLB0_LB_DEBUG3 0x1ae6
+#define mmLB1_LB_DEBUG3 0x1ce6
+#define mmLB2_LB_DEBUG3 0x1ee6
+#define mmLB3_LB_DEBUG3 0x40e6
+#define mmLB4_LB_DEBUG3 0x42e6
+#define mmLB5_LB_DEBUG3 0x44e6
+#define mmLB_TEST_DEBUG_INDEX 0x1afe
+#define mmLB0_LB_TEST_DEBUG_INDEX 0x1afe
+#define mmLB1_LB_TEST_DEBUG_INDEX 0x1cfe
+#define mmLB2_LB_TEST_DEBUG_INDEX 0x1efe
+#define mmLB3_LB_TEST_DEBUG_INDEX 0x40fe
+#define mmLB4_LB_TEST_DEBUG_INDEX 0x42fe
+#define mmLB5_LB_TEST_DEBUG_INDEX 0x44fe
+#define mmLB_TEST_DEBUG_DATA 0x1aff
+#define mmLB0_LB_TEST_DEBUG_DATA 0x1aff
+#define mmLB1_LB_TEST_DEBUG_DATA 0x1cff
+#define mmLB2_LB_TEST_DEBUG_DATA 0x1eff
+#define mmLB3_LB_TEST_DEBUG_DATA 0x40ff
+#define mmLB4_LB_TEST_DEBUG_DATA 0x42ff
+#define mmLB5_LB_TEST_DEBUG_DATA 0x44ff
+#define mmLBV_DATA_FORMAT 0x463c
+#define mmLBV0_LBV_DATA_FORMAT 0x463c
+#define mmLBV1_LBV_DATA_FORMAT 0x983c
+#define mmLBV_MEMORY_CTRL 0x463d
+#define mmLBV0_LBV_MEMORY_CTRL 0x463d
+#define mmLBV1_LBV_MEMORY_CTRL 0x983d
+#define mmLBV_MEMORY_SIZE_STATUS 0x463e
+#define mmLBV0_LBV_MEMORY_SIZE_STATUS 0x463e
+#define mmLBV1_LBV_MEMORY_SIZE_STATUS 0x983e
+#define mmLBV_DESKTOP_HEIGHT 0x463f
+#define mmLBV0_LBV_DESKTOP_HEIGHT 0x463f
+#define mmLBV1_LBV_DESKTOP_HEIGHT 0x983f
+#define mmLBV_VLINE_START_END 0x4640
+#define mmLBV0_LBV_VLINE_START_END 0x4640
+#define mmLBV1_LBV_VLINE_START_END 0x9840
+#define mmLBV_VLINE2_START_END 0x4641
+#define mmLBV0_LBV_VLINE2_START_END 0x4641
+#define mmLBV1_LBV_VLINE2_START_END 0x9841
+#define mmLBV_V_COUNTER 0x4642
+#define mmLBV0_LBV_V_COUNTER 0x4642
+#define mmLBV1_LBV_V_COUNTER 0x9842
+#define mmLBV_SNAPSHOT_V_COUNTER 0x4643
+#define mmLBV0_LBV_SNAPSHOT_V_COUNTER 0x4643
+#define mmLBV1_LBV_SNAPSHOT_V_COUNTER 0x9843
+#define mmLBV_V_COUNTER_CHROMA 0x4644
+#define mmLBV0_LBV_V_COUNTER_CHROMA 0x4644
+#define mmLBV1_LBV_V_COUNTER_CHROMA 0x9844
+#define mmLBV_SNAPSHOT_V_COUNTER_CHROMA 0x4645
+#define mmLBV0_LBV_SNAPSHOT_V_COUNTER_CHROMA 0x4645
+#define mmLBV1_LBV_SNAPSHOT_V_COUNTER_CHROMA 0x9845
+#define mmLBV_INTERRUPT_MASK 0x4646
+#define mmLBV0_LBV_INTERRUPT_MASK 0x4646
+#define mmLBV1_LBV_INTERRUPT_MASK 0x9846
+#define mmLBV_VLINE_STATUS 0x4647
+#define mmLBV0_LBV_VLINE_STATUS 0x4647
+#define mmLBV1_LBV_VLINE_STATUS 0x9847
+#define mmLBV_VLINE2_STATUS 0x4648
+#define mmLBV0_LBV_VLINE2_STATUS 0x4648
+#define mmLBV1_LBV_VLINE2_STATUS 0x9848
+#define mmLBV_VBLANK_STATUS 0x4649
+#define mmLBV0_LBV_VBLANK_STATUS 0x4649
+#define mmLBV1_LBV_VBLANK_STATUS 0x9849
+#define mmLBV_SYNC_RESET_SEL 0x464a
+#define mmLBV0_LBV_SYNC_RESET_SEL 0x464a
+#define mmLBV1_LBV_SYNC_RESET_SEL 0x984a
+#define mmLBV_BLACK_KEYER_R_CR 0x464b
+#define mmLBV0_LBV_BLACK_KEYER_R_CR 0x464b
+#define mmLBV1_LBV_BLACK_KEYER_R_CR 0x984b
+#define mmLBV_BLACK_KEYER_G_Y 0x464c
+#define mmLBV0_LBV_BLACK_KEYER_G_Y 0x464c
+#define mmLBV1_LBV_BLACK_KEYER_G_Y 0x984c
+#define mmLBV_BLACK_KEYER_B_CB 0x464d
+#define mmLBV0_LBV_BLACK_KEYER_B_CB 0x464d
+#define mmLBV1_LBV_BLACK_KEYER_B_CB 0x984d
+#define mmLBV_KEYER_COLOR_CTRL 0x464e
+#define mmLBV0_LBV_KEYER_COLOR_CTRL 0x464e
+#define mmLBV1_LBV_KEYER_COLOR_CTRL 0x984e
+#define mmLBV_KEYER_COLOR_R_CR 0x464f
+#define mmLBV0_LBV_KEYER_COLOR_R_CR 0x464f
+#define mmLBV1_LBV_KEYER_COLOR_R_CR 0x984f
+#define mmLBV_KEYER_COLOR_G_Y 0x4650
+#define mmLBV0_LBV_KEYER_COLOR_G_Y 0x4650
+#define mmLBV1_LBV_KEYER_COLOR_G_Y 0x9850
+#define mmLBV_KEYER_COLOR_B_CB 0x4651
+#define mmLBV0_LBV_KEYER_COLOR_B_CB 0x4651
+#define mmLBV1_LBV_KEYER_COLOR_B_CB 0x9851
+#define mmLBV_KEYER_COLOR_REP_R_CR 0x4652
+#define mmLBV0_LBV_KEYER_COLOR_REP_R_CR 0x4652
+#define mmLBV1_LBV_KEYER_COLOR_REP_R_CR 0x9852
+#define mmLBV_KEYER_COLOR_REP_G_Y 0x4653
+#define mmLBV0_LBV_KEYER_COLOR_REP_G_Y 0x4653
+#define mmLBV1_LBV_KEYER_COLOR_REP_G_Y 0x9853
+#define mmLBV_KEYER_COLOR_REP_B_CB 0x4654
+#define mmLBV0_LBV_KEYER_COLOR_REP_B_CB 0x4654
+#define mmLBV1_LBV_KEYER_COLOR_REP_B_CB 0x9854
+#define mmLBV_BUFFER_LEVEL_STATUS 0x4655
+#define mmLBV0_LBV_BUFFER_LEVEL_STATUS 0x4655
+#define mmLBV1_LBV_BUFFER_LEVEL_STATUS 0x9855
+#define mmLBV_BUFFER_URGENCY_CTRL 0x4656
+#define mmLBV0_LBV_BUFFER_URGENCY_CTRL 0x4656
+#define mmLBV1_LBV_BUFFER_URGENCY_CTRL 0x9856
+#define mmLBV_BUFFER_URGENCY_STATUS 0x4657
+#define mmLBV0_LBV_BUFFER_URGENCY_STATUS 0x4657
+#define mmLBV1_LBV_BUFFER_URGENCY_STATUS 0x9857
+#define mmLBV_BUFFER_STATUS 0x4658
+#define mmLBV0_LBV_BUFFER_STATUS 0x4658
+#define mmLBV1_LBV_BUFFER_STATUS 0x9858
+#define mmLBV_NO_OUTSTANDING_REQ_STATUS 0x4659
+#define mmLBV0_LBV_NO_OUTSTANDING_REQ_STATUS 0x4659
+#define mmLBV1_LBV_NO_OUTSTANDING_REQ_STATUS 0x9859
+#define mmLBV_DEBUG 0x465a
+#define mmLBV0_LBV_DEBUG 0x465a
+#define mmLBV1_LBV_DEBUG 0x985a
+#define mmLBV_DEBUG2 0x465b
+#define mmLBV0_LBV_DEBUG2 0x465b
+#define mmLBV1_LBV_DEBUG2 0x985b
+#define mmLBV_DEBUG3 0x465c
+#define mmLBV0_LBV_DEBUG3 0x465c
+#define mmLBV1_LBV_DEBUG3 0x985c
+#define mmLBV_TEST_DEBUG_INDEX 0x4666
+#define mmLBV0_LBV_TEST_DEBUG_INDEX 0x4666
+#define mmLBV1_LBV_TEST_DEBUG_INDEX 0x9866
+#define mmLBV_TEST_DEBUG_DATA 0x4667
+#define mmLBV0_LBV_TEST_DEBUG_DATA 0x4667
+#define mmLBV1_LBV_TEST_DEBUG_DATA 0x9867
+#define mmMVP_CONTROL1 0x2ac
+#define mmMVP_CONTROL2 0x2ad
+#define mmMVP_FIFO_CONTROL 0x2ae
+#define mmMVP_FIFO_STATUS 0x2af
+#define mmMVP_SLAVE_STATUS 0x2b0
+#define mmMVP_INBAND_CNTL_CAP 0x2b1
+#define mmMVP_BLACK_KEYER 0x2b2
+#define mmMVP_CRC_CNTL 0x2b3
+#define mmMVP_CRC_RESULT_BLUE_GREEN 0x2b4
+#define mmMVP_CRC_RESULT_RED 0x2b5
+#define mmMVP_CONTROL3 0x2b6
+#define mmMVP_RECEIVE_CNT_CNTL1 0x2b7
+#define mmMVP_RECEIVE_CNT_CNTL2 0x2b8
+#define mmMVP_DEBUG 0x2bb
+#define mmMVP_TEST_DEBUG_INDEX 0x2b9
+#define mmMVP_TEST_DEBUG_DATA 0x2ba
+#define ixMVP_DEBUG_12 0xc
+#define ixMVP_DEBUG_13 0xd
+#define ixMVP_DEBUG_14 0xe
+#define ixMVP_DEBUG_15 0xf
+#define ixMVP_DEBUG_16 0x10
+#define ixMVP_DEBUG_17 0x11
+#define mmSCL_COEF_RAM_SELECT 0x1b40
+#define mmSCL0_SCL_COEF_RAM_SELECT 0x1b40
+#define mmSCL1_SCL_COEF_RAM_SELECT 0x1d40
+#define mmSCL2_SCL_COEF_RAM_SELECT 0x1f40
+#define mmSCL3_SCL_COEF_RAM_SELECT 0x4140
+#define mmSCL4_SCL_COEF_RAM_SELECT 0x4340
+#define mmSCL5_SCL_COEF_RAM_SELECT 0x4540
+#define mmSCL_COEF_RAM_TAP_DATA 0x1b41
+#define mmSCL0_SCL_COEF_RAM_TAP_DATA 0x1b41
+#define mmSCL1_SCL_COEF_RAM_TAP_DATA 0x1d41
+#define mmSCL2_SCL_COEF_RAM_TAP_DATA 0x1f41
+#define mmSCL3_SCL_COEF_RAM_TAP_DATA 0x4141
+#define mmSCL4_SCL_COEF_RAM_TAP_DATA 0x4341
+#define mmSCL5_SCL_COEF_RAM_TAP_DATA 0x4541
+#define mmSCL_MODE 0x1b42
+#define mmSCL0_SCL_MODE 0x1b42
+#define mmSCL1_SCL_MODE 0x1d42
+#define mmSCL2_SCL_MODE 0x1f42
+#define mmSCL3_SCL_MODE 0x4142
+#define mmSCL4_SCL_MODE 0x4342
+#define mmSCL5_SCL_MODE 0x4542
+#define mmSCL_TAP_CONTROL 0x1b43
+#define mmSCL0_SCL_TAP_CONTROL 0x1b43
+#define mmSCL1_SCL_TAP_CONTROL 0x1d43
+#define mmSCL2_SCL_TAP_CONTROL 0x1f43
+#define mmSCL3_SCL_TAP_CONTROL 0x4143
+#define mmSCL4_SCL_TAP_CONTROL 0x4343
+#define mmSCL5_SCL_TAP_CONTROL 0x4543
+#define mmSCL_CONTROL 0x1b44
+#define mmSCL0_SCL_CONTROL 0x1b44
+#define mmSCL1_SCL_CONTROL 0x1d44
+#define mmSCL2_SCL_CONTROL 0x1f44
+#define mmSCL3_SCL_CONTROL 0x4144
+#define mmSCL4_SCL_CONTROL 0x4344
+#define mmSCL5_SCL_CONTROL 0x4544
+#define mmSCL_BYPASS_CONTROL 0x1b45
+#define mmSCL0_SCL_BYPASS_CONTROL 0x1b45
+#define mmSCL1_SCL_BYPASS_CONTROL 0x1d45
+#define mmSCL2_SCL_BYPASS_CONTROL 0x1f45
+#define mmSCL3_SCL_BYPASS_CONTROL 0x4145
+#define mmSCL4_SCL_BYPASS_CONTROL 0x4345
+#define mmSCL5_SCL_BYPASS_CONTROL 0x4545
+#define mmSCL_MANUAL_REPLICATE_CONTROL 0x1b46
+#define mmSCL0_SCL_MANUAL_REPLICATE_CONTROL 0x1b46
+#define mmSCL1_SCL_MANUAL_REPLICATE_CONTROL 0x1d46
+#define mmSCL2_SCL_MANUAL_REPLICATE_CONTROL 0x1f46
+#define mmSCL3_SCL_MANUAL_REPLICATE_CONTROL 0x4146
+#define mmSCL4_SCL_MANUAL_REPLICATE_CONTROL 0x4346
+#define mmSCL5_SCL_MANUAL_REPLICATE_CONTROL 0x4546
+#define mmSCL_AUTOMATIC_MODE_CONTROL 0x1b47
+#define mmSCL0_SCL_AUTOMATIC_MODE_CONTROL 0x1b47
+#define mmSCL1_SCL_AUTOMATIC_MODE_CONTROL 0x1d47
+#define mmSCL2_SCL_AUTOMATIC_MODE_CONTROL 0x1f47
+#define mmSCL3_SCL_AUTOMATIC_MODE_CONTROL 0x4147
+#define mmSCL4_SCL_AUTOMATIC_MODE_CONTROL 0x4347
+#define mmSCL5_SCL_AUTOMATIC_MODE_CONTROL 0x4547
+#define mmSCL_HORZ_FILTER_CONTROL 0x1b48
+#define mmSCL0_SCL_HORZ_FILTER_CONTROL 0x1b48
+#define mmSCL1_SCL_HORZ_FILTER_CONTROL 0x1d48
+#define mmSCL2_SCL_HORZ_FILTER_CONTROL 0x1f48
+#define mmSCL3_SCL_HORZ_FILTER_CONTROL 0x4148
+#define mmSCL4_SCL_HORZ_FILTER_CONTROL 0x4348
+#define mmSCL5_SCL_HORZ_FILTER_CONTROL 0x4548
+#define mmSCL_HORZ_FILTER_SCALE_RATIO 0x1b49
+#define mmSCL0_SCL_HORZ_FILTER_SCALE_RATIO 0x1b49
+#define mmSCL1_SCL_HORZ_FILTER_SCALE_RATIO 0x1d49
+#define mmSCL2_SCL_HORZ_FILTER_SCALE_RATIO 0x1f49
+#define mmSCL3_SCL_HORZ_FILTER_SCALE_RATIO 0x4149
+#define mmSCL4_SCL_HORZ_FILTER_SCALE_RATIO 0x4349
+#define mmSCL5_SCL_HORZ_FILTER_SCALE_RATIO 0x4549
+#define mmSCL_HORZ_FILTER_INIT 0x1b4a
+#define mmSCL0_SCL_HORZ_FILTER_INIT 0x1b4a
+#define mmSCL1_SCL_HORZ_FILTER_INIT 0x1d4a
+#define mmSCL2_SCL_HORZ_FILTER_INIT 0x1f4a
+#define mmSCL3_SCL_HORZ_FILTER_INIT 0x414a
+#define mmSCL4_SCL_HORZ_FILTER_INIT 0x434a
+#define mmSCL5_SCL_HORZ_FILTER_INIT 0x454a
+#define mmSCL_VERT_FILTER_CONTROL 0x1b4b
+#define mmSCL0_SCL_VERT_FILTER_CONTROL 0x1b4b
+#define mmSCL1_SCL_VERT_FILTER_CONTROL 0x1d4b
+#define mmSCL2_SCL_VERT_FILTER_CONTROL 0x1f4b
+#define mmSCL3_SCL_VERT_FILTER_CONTROL 0x414b
+#define mmSCL4_SCL_VERT_FILTER_CONTROL 0x434b
+#define mmSCL5_SCL_VERT_FILTER_CONTROL 0x454b
+#define mmSCL_VERT_FILTER_SCALE_RATIO 0x1b4c
+#define mmSCL0_SCL_VERT_FILTER_SCALE_RATIO 0x1b4c
+#define mmSCL1_SCL_VERT_FILTER_SCALE_RATIO 0x1d4c
+#define mmSCL2_SCL_VERT_FILTER_SCALE_RATIO 0x1f4c
+#define mmSCL3_SCL_VERT_FILTER_SCALE_RATIO 0x414c
+#define mmSCL4_SCL_VERT_FILTER_SCALE_RATIO 0x434c
+#define mmSCL5_SCL_VERT_FILTER_SCALE_RATIO 0x454c
+#define mmSCL_VERT_FILTER_INIT 0x1b4d
+#define mmSCL0_SCL_VERT_FILTER_INIT 0x1b4d
+#define mmSCL1_SCL_VERT_FILTER_INIT 0x1d4d
+#define mmSCL2_SCL_VERT_FILTER_INIT 0x1f4d
+#define mmSCL3_SCL_VERT_FILTER_INIT 0x414d
+#define mmSCL4_SCL_VERT_FILTER_INIT 0x434d
+#define mmSCL5_SCL_VERT_FILTER_INIT 0x454d
+#define mmSCL_VERT_FILTER_INIT_BOT 0x1b4e
+#define mmSCL0_SCL_VERT_FILTER_INIT_BOT 0x1b4e
+#define mmSCL1_SCL_VERT_FILTER_INIT_BOT 0x1d4e
+#define mmSCL2_SCL_VERT_FILTER_INIT_BOT 0x1f4e
+#define mmSCL3_SCL_VERT_FILTER_INIT_BOT 0x414e
+#define mmSCL4_SCL_VERT_FILTER_INIT_BOT 0x434e
+#define mmSCL5_SCL_VERT_FILTER_INIT_BOT 0x454e
+#define mmSCL_ROUND_OFFSET 0x1b4f
+#define mmSCL0_SCL_ROUND_OFFSET 0x1b4f
+#define mmSCL1_SCL_ROUND_OFFSET 0x1d4f
+#define mmSCL2_SCL_ROUND_OFFSET 0x1f4f
+#define mmSCL3_SCL_ROUND_OFFSET 0x414f
+#define mmSCL4_SCL_ROUND_OFFSET 0x434f
+#define mmSCL5_SCL_ROUND_OFFSET 0x454f
+#define mmSCL_UPDATE 0x1b51
+#define mmSCL0_SCL_UPDATE 0x1b51
+#define mmSCL1_SCL_UPDATE 0x1d51
+#define mmSCL2_SCL_UPDATE 0x1f51
+#define mmSCL3_SCL_UPDATE 0x4151
+#define mmSCL4_SCL_UPDATE 0x4351
+#define mmSCL5_SCL_UPDATE 0x4551
+#define mmSCL_F_SHARP_CONTROL 0x1b53
+#define mmSCL0_SCL_F_SHARP_CONTROL 0x1b53
+#define mmSCL1_SCL_F_SHARP_CONTROL 0x1d53
+#define mmSCL2_SCL_F_SHARP_CONTROL 0x1f53
+#define mmSCL3_SCL_F_SHARP_CONTROL 0x4153
+#define mmSCL4_SCL_F_SHARP_CONTROL 0x4353
+#define mmSCL5_SCL_F_SHARP_CONTROL 0x4553
+#define mmSCL_ALU_CONTROL 0x1b54
+#define mmSCL0_SCL_ALU_CONTROL 0x1b54
+#define mmSCL1_SCL_ALU_CONTROL 0x1d54
+#define mmSCL2_SCL_ALU_CONTROL 0x1f54
+#define mmSCL3_SCL_ALU_CONTROL 0x4154
+#define mmSCL4_SCL_ALU_CONTROL 0x4354
+#define mmSCL5_SCL_ALU_CONTROL 0x4554
+#define mmSCL_COEF_RAM_CONFLICT_STATUS 0x1b55
+#define mmSCL0_SCL_COEF_RAM_CONFLICT_STATUS 0x1b55
+#define mmSCL1_SCL_COEF_RAM_CONFLICT_STATUS 0x1d55
+#define mmSCL2_SCL_COEF_RAM_CONFLICT_STATUS 0x1f55
+#define mmSCL3_SCL_COEF_RAM_CONFLICT_STATUS 0x4155
+#define mmSCL4_SCL_COEF_RAM_CONFLICT_STATUS 0x4355
+#define mmSCL5_SCL_COEF_RAM_CONFLICT_STATUS 0x4555
+#define mmVIEWPORT_START_SECONDARY 0x1b5b
+#define mmSCL0_VIEWPORT_START_SECONDARY 0x1b5b
+#define mmSCL1_VIEWPORT_START_SECONDARY 0x1d5b
+#define mmSCL2_VIEWPORT_START_SECONDARY 0x1f5b
+#define mmSCL3_VIEWPORT_START_SECONDARY 0x415b
+#define mmSCL4_VIEWPORT_START_SECONDARY 0x435b
+#define mmSCL5_VIEWPORT_START_SECONDARY 0x455b
+#define mmVIEWPORT_START 0x1b5c
+#define mmSCL0_VIEWPORT_START 0x1b5c
+#define mmSCL1_VIEWPORT_START 0x1d5c
+#define mmSCL2_VIEWPORT_START 0x1f5c
+#define mmSCL3_VIEWPORT_START 0x415c
+#define mmSCL4_VIEWPORT_START 0x435c
+#define mmSCL5_VIEWPORT_START 0x455c
+#define mmVIEWPORT_SIZE 0x1b5d
+#define mmSCL0_VIEWPORT_SIZE 0x1b5d
+#define mmSCL1_VIEWPORT_SIZE 0x1d5d
+#define mmSCL2_VIEWPORT_SIZE 0x1f5d
+#define mmSCL3_VIEWPORT_SIZE 0x415d
+#define mmSCL4_VIEWPORT_SIZE 0x435d
+#define mmSCL5_VIEWPORT_SIZE 0x455d
+#define mmEXT_OVERSCAN_LEFT_RIGHT 0x1b5e
+#define mmSCL0_EXT_OVERSCAN_LEFT_RIGHT 0x1b5e
+#define mmSCL1_EXT_OVERSCAN_LEFT_RIGHT 0x1d5e
+#define mmSCL2_EXT_OVERSCAN_LEFT_RIGHT 0x1f5e
+#define mmSCL3_EXT_OVERSCAN_LEFT_RIGHT 0x415e
+#define mmSCL4_EXT_OVERSCAN_LEFT_RIGHT 0x435e
+#define mmSCL5_EXT_OVERSCAN_LEFT_RIGHT 0x455e
+#define mmEXT_OVERSCAN_TOP_BOTTOM 0x1b5f
+#define mmSCL0_EXT_OVERSCAN_TOP_BOTTOM 0x1b5f
+#define mmSCL1_EXT_OVERSCAN_TOP_BOTTOM 0x1d5f
+#define mmSCL2_EXT_OVERSCAN_TOP_BOTTOM 0x1f5f
+#define mmSCL3_EXT_OVERSCAN_TOP_BOTTOM 0x415f
+#define mmSCL4_EXT_OVERSCAN_TOP_BOTTOM 0x435f
+#define mmSCL5_EXT_OVERSCAN_TOP_BOTTOM 0x455f
+#define mmSCL_MODE_CHANGE_DET1 0x1b60
+#define mmSCL0_SCL_MODE_CHANGE_DET1 0x1b60
+#define mmSCL1_SCL_MODE_CHANGE_DET1 0x1d60
+#define mmSCL2_SCL_MODE_CHANGE_DET1 0x1f60
+#define mmSCL3_SCL_MODE_CHANGE_DET1 0x4160
+#define mmSCL4_SCL_MODE_CHANGE_DET1 0x4360
+#define mmSCL5_SCL_MODE_CHANGE_DET1 0x4560
+#define mmSCL_MODE_CHANGE_DET2 0x1b61
+#define mmSCL0_SCL_MODE_CHANGE_DET2 0x1b61
+#define mmSCL1_SCL_MODE_CHANGE_DET2 0x1d61
+#define mmSCL2_SCL_MODE_CHANGE_DET2 0x1f61
+#define mmSCL3_SCL_MODE_CHANGE_DET2 0x4161
+#define mmSCL4_SCL_MODE_CHANGE_DET2 0x4361
+#define mmSCL5_SCL_MODE_CHANGE_DET2 0x4561
+#define mmSCL_MODE_CHANGE_DET3 0x1b62
+#define mmSCL0_SCL_MODE_CHANGE_DET3 0x1b62
+#define mmSCL1_SCL_MODE_CHANGE_DET3 0x1d62
+#define mmSCL2_SCL_MODE_CHANGE_DET3 0x1f62
+#define mmSCL3_SCL_MODE_CHANGE_DET3 0x4162
+#define mmSCL4_SCL_MODE_CHANGE_DET3 0x4362
+#define mmSCL5_SCL_MODE_CHANGE_DET3 0x4562
+#define mmSCL_MODE_CHANGE_MASK 0x1b63
+#define mmSCL0_SCL_MODE_CHANGE_MASK 0x1b63
+#define mmSCL1_SCL_MODE_CHANGE_MASK 0x1d63
+#define mmSCL2_SCL_MODE_CHANGE_MASK 0x1f63
+#define mmSCL3_SCL_MODE_CHANGE_MASK 0x4163
+#define mmSCL4_SCL_MODE_CHANGE_MASK 0x4363
+#define mmSCL5_SCL_MODE_CHANGE_MASK 0x4563
+#define mmSCL_DEBUG2 0x1b69
+#define mmSCL0_SCL_DEBUG2 0x1b69
+#define mmSCL1_SCL_DEBUG2 0x1d69
+#define mmSCL2_SCL_DEBUG2 0x1f69
+#define mmSCL3_SCL_DEBUG2 0x4169
+#define mmSCL4_SCL_DEBUG2 0x4369
+#define mmSCL5_SCL_DEBUG2 0x4569
+#define mmSCL_DEBUG 0x1b6a
+#define mmSCL0_SCL_DEBUG 0x1b6a
+#define mmSCL1_SCL_DEBUG 0x1d6a
+#define mmSCL2_SCL_DEBUG 0x1f6a
+#define mmSCL3_SCL_DEBUG 0x416a
+#define mmSCL4_SCL_DEBUG 0x436a
+#define mmSCL5_SCL_DEBUG 0x456a
+#define mmSCL_TEST_DEBUG_INDEX 0x1b6b
+#define mmSCL0_SCL_TEST_DEBUG_INDEX 0x1b6b
+#define mmSCL1_SCL_TEST_DEBUG_INDEX 0x1d6b
+#define mmSCL2_SCL_TEST_DEBUG_INDEX 0x1f6b
+#define mmSCL3_SCL_TEST_DEBUG_INDEX 0x416b
+#define mmSCL4_SCL_TEST_DEBUG_INDEX 0x436b
+#define mmSCL5_SCL_TEST_DEBUG_INDEX 0x456b
+#define mmSCL_TEST_DEBUG_DATA 0x1b6c
+#define mmSCL0_SCL_TEST_DEBUG_DATA 0x1b6c
+#define mmSCL1_SCL_TEST_DEBUG_DATA 0x1d6c
+#define mmSCL2_SCL_TEST_DEBUG_DATA 0x1f6c
+#define mmSCL3_SCL_TEST_DEBUG_DATA 0x416c
+#define mmSCL4_SCL_TEST_DEBUG_DATA 0x436c
+#define mmSCL5_SCL_TEST_DEBUG_DATA 0x456c
+#define mmSCLV_COEF_RAM_SELECT 0x4670
+#define mmSCLV0_SCLV_COEF_RAM_SELECT 0x4670
+#define mmSCLV1_SCLV_COEF_RAM_SELECT 0x9870
+#define mmSCLV_COEF_RAM_TAP_DATA 0x4671
+#define mmSCLV0_SCLV_COEF_RAM_TAP_DATA 0x4671
+#define mmSCLV1_SCLV_COEF_RAM_TAP_DATA 0x9871
+#define mmSCLV_MODE 0x4672
+#define mmSCLV0_SCLV_MODE 0x4672
+#define mmSCLV1_SCLV_MODE 0x9872
+#define mmSCLV_TAP_CONTROL 0x4673
+#define mmSCLV0_SCLV_TAP_CONTROL 0x4673
+#define mmSCLV1_SCLV_TAP_CONTROL 0x9873
+#define mmSCLV_CONTROL 0x4674
+#define mmSCLV0_SCLV_CONTROL 0x4674
+#define mmSCLV1_SCLV_CONTROL 0x9874
+#define mmSCLV_MANUAL_REPLICATE_CONTROL 0x4675
+#define mmSCLV0_SCLV_MANUAL_REPLICATE_CONTROL 0x4675
+#define mmSCLV1_SCLV_MANUAL_REPLICATE_CONTROL 0x9875
+#define mmSCLV_AUTOMATIC_MODE_CONTROL 0x4676
+#define mmSCLV0_SCLV_AUTOMATIC_MODE_CONTROL 0x4676
+#define mmSCLV1_SCLV_AUTOMATIC_MODE_CONTROL 0x9876
+#define mmSCLV_HORZ_FILTER_CONTROL 0x4677
+#define mmSCLV0_SCLV_HORZ_FILTER_CONTROL 0x4677
+#define mmSCLV1_SCLV_HORZ_FILTER_CONTROL 0x9877
+#define mmSCLV_HORZ_FILTER_SCALE_RATIO 0x4678
+#define mmSCLV0_SCLV_HORZ_FILTER_SCALE_RATIO 0x4678
+#define mmSCLV1_SCLV_HORZ_FILTER_SCALE_RATIO 0x9878
+#define mmSCLV_HORZ_FILTER_INIT 0x4679
+#define mmSCLV0_SCLV_HORZ_FILTER_INIT 0x4679
+#define mmSCLV1_SCLV_HORZ_FILTER_INIT 0x9879
+#define mmSCLV_HORZ_FILTER_SCALE_RATIO_C 0x467a
+#define mmSCLV0_SCLV_HORZ_FILTER_SCALE_RATIO_C 0x467a
+#define mmSCLV1_SCLV_HORZ_FILTER_SCALE_RATIO_C 0x987a
+#define mmSCLV_HORZ_FILTER_INIT_C 0x467b
+#define mmSCLV0_SCLV_HORZ_FILTER_INIT_C 0x467b
+#define mmSCLV1_SCLV_HORZ_FILTER_INIT_C 0x987b
+#define mmSCLV_VERT_FILTER_CONTROL 0x467c
+#define mmSCLV0_SCLV_VERT_FILTER_CONTROL 0x467c
+#define mmSCLV1_SCLV_VERT_FILTER_CONTROL 0x987c
+#define mmSCLV_VERT_FILTER_SCALE_RATIO 0x467d
+#define mmSCLV0_SCLV_VERT_FILTER_SCALE_RATIO 0x467d
+#define mmSCLV1_SCLV_VERT_FILTER_SCALE_RATIO 0x987d
+#define mmSCLV_VERT_FILTER_INIT 0x467e
+#define mmSCLV0_SCLV_VERT_FILTER_INIT 0x467e
+#define mmSCLV1_SCLV_VERT_FILTER_INIT 0x987e
+#define mmSCLV_VERT_FILTER_INIT_BOT 0x467f
+#define mmSCLV0_SCLV_VERT_FILTER_INIT_BOT 0x467f
+#define mmSCLV1_SCLV_VERT_FILTER_INIT_BOT 0x987f
+#define mmSCLV_VERT_FILTER_SCALE_RATIO_C 0x4680
+#define mmSCLV0_SCLV_VERT_FILTER_SCALE_RATIO_C 0x4680
+#define mmSCLV1_SCLV_VERT_FILTER_SCALE_RATIO_C 0x9880
+#define mmSCLV_VERT_FILTER_INIT_C 0x4681
+#define mmSCLV0_SCLV_VERT_FILTER_INIT_C 0x4681
+#define mmSCLV1_SCLV_VERT_FILTER_INIT_C 0x9881
+#define mmSCLV_VERT_FILTER_INIT_BOT_C 0x4682
+#define mmSCLV0_SCLV_VERT_FILTER_INIT_BOT_C 0x4682
+#define mmSCLV1_SCLV_VERT_FILTER_INIT_BOT_C 0x9882
+#define mmSCLV_ROUND_OFFSET 0x4683
+#define mmSCLV0_SCLV_ROUND_OFFSET 0x4683
+#define mmSCLV1_SCLV_ROUND_OFFSET 0x9883
+#define mmSCLV_UPDATE 0x4684
+#define mmSCLV0_SCLV_UPDATE 0x4684
+#define mmSCLV1_SCLV_UPDATE 0x9884
+#define mmSCLV_ALU_CONTROL 0x4685
+#define mmSCLV0_SCLV_ALU_CONTROL 0x4685
+#define mmSCLV1_SCLV_ALU_CONTROL 0x9885
+#define mmSCLV_VIEWPORT_START 0x4686
+#define mmSCLV0_SCLV_VIEWPORT_START 0x4686
+#define mmSCLV1_SCLV_VIEWPORT_START 0x9886
+#define mmSCLV_VIEWPORT_START_SECONDARY 0x4687
+#define mmSCLV0_SCLV_VIEWPORT_START_SECONDARY 0x4687
+#define mmSCLV1_SCLV_VIEWPORT_START_SECONDARY 0x9887
+#define mmSCLV_VIEWPORT_SIZE 0x4688
+#define mmSCLV0_SCLV_VIEWPORT_SIZE 0x4688
+#define mmSCLV1_SCLV_VIEWPORT_SIZE 0x9888
+#define mmSCLV_VIEWPORT_START_C 0x4689
+#define mmSCLV0_SCLV_VIEWPORT_START_C 0x4689
+#define mmSCLV1_SCLV_VIEWPORT_START_C 0x9889
+#define mmSCLV_VIEWPORT_START_SECONDARY_C 0x468a
+#define mmSCLV0_SCLV_VIEWPORT_START_SECONDARY_C 0x468a
+#define mmSCLV1_SCLV_VIEWPORT_START_SECONDARY_C 0x988a
+#define mmSCLV_VIEWPORT_SIZE_C 0x468b
+#define mmSCLV0_SCLV_VIEWPORT_SIZE_C 0x468b
+#define mmSCLV1_SCLV_VIEWPORT_SIZE_C 0x988b
+#define mmSCLV_EXT_OVERSCAN_LEFT_RIGHT 0x468c
+#define mmSCLV0_SCLV_EXT_OVERSCAN_LEFT_RIGHT 0x468c
+#define mmSCLV1_SCLV_EXT_OVERSCAN_LEFT_RIGHT 0x988c
+#define mmSCLV_EXT_OVERSCAN_TOP_BOTTOM 0x468d
+#define mmSCLV0_SCLV_EXT_OVERSCAN_TOP_BOTTOM 0x468d
+#define mmSCLV1_SCLV_EXT_OVERSCAN_TOP_BOTTOM 0x988d
+#define mmSCLV_MODE_CHANGE_DET1 0x468e
+#define mmSCLV0_SCLV_MODE_CHANGE_DET1 0x468e
+#define mmSCLV1_SCLV_MODE_CHANGE_DET1 0x988e
+#define mmSCLV_MODE_CHANGE_DET2 0x468f
+#define mmSCLV0_SCLV_MODE_CHANGE_DET2 0x468f
+#define mmSCLV1_SCLV_MODE_CHANGE_DET2 0x988f
+#define mmSCLV_MODE_CHANGE_DET3 0x4690
+#define mmSCLV0_SCLV_MODE_CHANGE_DET3 0x4690
+#define mmSCLV1_SCLV_MODE_CHANGE_DET3 0x9890
+#define mmSCLV_MODE_CHANGE_MASK 0x4691
+#define mmSCLV0_SCLV_MODE_CHANGE_MASK 0x4691
+#define mmSCLV1_SCLV_MODE_CHANGE_MASK 0x9891
+#define mmSCLV_HORZ_FILTER_INIT_BOT 0x4692
+#define mmSCLV0_SCLV_HORZ_FILTER_INIT_BOT 0x4692
+#define mmSCLV1_SCLV_HORZ_FILTER_INIT_BOT 0x9892
+#define mmSCLV_HORZ_FILTER_INIT_BOT_C 0x4693
+#define mmSCLV0_SCLV_HORZ_FILTER_INIT_BOT_C 0x4693
+#define mmSCLV1_SCLV_HORZ_FILTER_INIT_BOT_C 0x9893
+#define mmSCLV_DEBUG2 0x4694
+#define mmSCLV0_SCLV_DEBUG2 0x4694
+#define mmSCLV1_SCLV_DEBUG2 0x9894
+#define mmSCLV_DEBUG 0x4695
+#define mmSCLV0_SCLV_DEBUG 0x4695
+#define mmSCLV1_SCLV_DEBUG 0x9895
+#define mmSCLV_TEST_DEBUG_INDEX 0x4696
+#define mmSCLV0_SCLV_TEST_DEBUG_INDEX 0x4696
+#define mmSCLV1_SCLV_TEST_DEBUG_INDEX 0x9896
+#define mmSCLV_TEST_DEBUG_DATA 0x4697
+#define mmSCLV0_SCLV_TEST_DEBUG_DATA 0x4697
+#define mmSCLV1_SCLV_TEST_DEBUG_DATA 0x9897
+#define mmCOL_MAN_UPDATE 0x46a4
+#define mmCOL_MAN0_COL_MAN_UPDATE 0x46a4
+#define mmCOL_MAN1_COL_MAN_UPDATE 0x98a4
+#define mmCOL_MAN_INPUT_CSC_CONTROL 0x46a5
+#define mmCOL_MAN0_COL_MAN_INPUT_CSC_CONTROL 0x46a5
+#define mmCOL_MAN1_COL_MAN_INPUT_CSC_CONTROL 0x98a5
+#define mmINPUT_CSC_C11_C12_A 0x46a6
+#define mmCOL_MAN0_INPUT_CSC_C11_C12_A 0x46a6
+#define mmCOL_MAN1_INPUT_CSC_C11_C12_A 0x98a6
+#define mmINPUT_CSC_C13_C14_A 0x46a7
+#define mmCOL_MAN0_INPUT_CSC_C13_C14_A 0x46a7
+#define mmCOL_MAN1_INPUT_CSC_C13_C14_A 0x98a7
+#define mmINPUT_CSC_C21_C22_A 0x46a8
+#define mmCOL_MAN0_INPUT_CSC_C21_C22_A 0x46a8
+#define mmCOL_MAN1_INPUT_CSC_C21_C22_A 0x98a8
+#define mmINPUT_CSC_C23_C24_A 0x46a9
+#define mmCOL_MAN0_INPUT_CSC_C23_C24_A 0x46a9
+#define mmCOL_MAN1_INPUT_CSC_C23_C24_A 0x98a9
+#define mmINPUT_CSC_C31_C32_A 0x46aa
+#define mmCOL_MAN0_INPUT_CSC_C31_C32_A 0x46aa
+#define mmCOL_MAN1_INPUT_CSC_C31_C32_A 0x98aa
+#define mmINPUT_CSC_C33_C34_A 0x46ab
+#define mmCOL_MAN0_INPUT_CSC_C33_C34_A 0x46ab
+#define mmCOL_MAN1_INPUT_CSC_C33_C34_A 0x98ab
+#define mmINPUT_CSC_C11_C12_B 0x46ac
+#define mmCOL_MAN0_INPUT_CSC_C11_C12_B 0x46ac
+#define mmCOL_MAN1_INPUT_CSC_C11_C12_B 0x98ac
+#define mmINPUT_CSC_C13_C14_B 0x46ad
+#define mmCOL_MAN0_INPUT_CSC_C13_C14_B 0x46ad
+#define mmCOL_MAN1_INPUT_CSC_C13_C14_B 0x98ad
+#define mmINPUT_CSC_C21_C22_B 0x46ae
+#define mmCOL_MAN0_INPUT_CSC_C21_C22_B 0x46ae
+#define mmCOL_MAN1_INPUT_CSC_C21_C22_B 0x98ae
+#define mmINPUT_CSC_C23_C24_B 0x46af
+#define mmCOL_MAN0_INPUT_CSC_C23_C24_B 0x46af
+#define mmCOL_MAN1_INPUT_CSC_C23_C24_B 0x98af
+#define mmINPUT_CSC_C31_C32_B 0x46b0
+#define mmCOL_MAN0_INPUT_CSC_C31_C32_B 0x46b0
+#define mmCOL_MAN1_INPUT_CSC_C31_C32_B 0x98b0
+#define mmINPUT_CSC_C33_C34_B 0x46b1
+#define mmCOL_MAN0_INPUT_CSC_C33_C34_B 0x46b1
+#define mmCOL_MAN1_INPUT_CSC_C33_C34_B 0x98b1
+#define mmPRESCALE_CONTROL 0x46b2
+#define mmCOL_MAN0_PRESCALE_CONTROL 0x46b2
+#define mmCOL_MAN1_PRESCALE_CONTROL 0x98b2
+#define mmPRESCALE_VALUES_R 0x46b3
+#define mmCOL_MAN0_PRESCALE_VALUES_R 0x46b3
+#define mmCOL_MAN1_PRESCALE_VALUES_R 0x98b3
+#define mmPRESCALE_VALUES_G 0x46b4
+#define mmCOL_MAN0_PRESCALE_VALUES_G 0x46b4
+#define mmCOL_MAN1_PRESCALE_VALUES_G 0x98b4
+#define mmPRESCALE_VALUES_B 0x46b5
+#define mmCOL_MAN0_PRESCALE_VALUES_B 0x46b5
+#define mmCOL_MAN1_PRESCALE_VALUES_B 0x98b5
+#define mmCOL_MAN_OUTPUT_CSC_CONTROL 0x46b6
+#define mmCOL_MAN0_COL_MAN_OUTPUT_CSC_CONTROL 0x46b6
+#define mmCOL_MAN1_COL_MAN_OUTPUT_CSC_CONTROL 0x98b6
+#define mmOUTPUT_CSC_C11_C12_A 0x46b7
+#define mmCOL_MAN0_OUTPUT_CSC_C11_C12_A 0x46b7
+#define mmCOL_MAN1_OUTPUT_CSC_C11_C12_A 0x98b7
+#define mmOUTPUT_CSC_C13_C14_A 0x46b8
+#define mmCOL_MAN0_OUTPUT_CSC_C13_C14_A 0x46b8
+#define mmCOL_MAN1_OUTPUT_CSC_C13_C14_A 0x98b8
+#define mmOUTPUT_CSC_C21_C22_A 0x46b9
+#define mmCOL_MAN0_OUTPUT_CSC_C21_C22_A 0x46b9
+#define mmCOL_MAN1_OUTPUT_CSC_C21_C22_A 0x98b9
+#define mmOUTPUT_CSC_C23_C24_A 0x46ba
+#define mmCOL_MAN0_OUTPUT_CSC_C23_C24_A 0x46ba
+#define mmCOL_MAN1_OUTPUT_CSC_C23_C24_A 0x98ba
+#define mmOUTPUT_CSC_C31_C32_A 0x46bb
+#define mmCOL_MAN0_OUTPUT_CSC_C31_C32_A 0x46bb
+#define mmCOL_MAN1_OUTPUT_CSC_C31_C32_A 0x98bb
+#define mmOUTPUT_CSC_C33_C34_A 0x46bc
+#define mmCOL_MAN0_OUTPUT_CSC_C33_C34_A 0x46bc
+#define mmCOL_MAN1_OUTPUT_CSC_C33_C34_A 0x98bc
+#define mmOUTPUT_CSC_C11_C12_B 0x46bd
+#define mmCOL_MAN0_OUTPUT_CSC_C11_C12_B 0x46bd
+#define mmCOL_MAN1_OUTPUT_CSC_C11_C12_B 0x98bd
+#define mmOUTPUT_CSC_C13_C14_B 0x46be
+#define mmCOL_MAN0_OUTPUT_CSC_C13_C14_B 0x46be
+#define mmCOL_MAN1_OUTPUT_CSC_C13_C14_B 0x98be
+#define mmOUTPUT_CSC_C21_C22_B 0x46bf
+#define mmCOL_MAN0_OUTPUT_CSC_C21_C22_B 0x46bf
+#define mmCOL_MAN1_OUTPUT_CSC_C21_C22_B 0x98bf
+#define mmOUTPUT_CSC_C23_C24_B 0x46c0
+#define mmCOL_MAN0_OUTPUT_CSC_C23_C24_B 0x46c0
+#define mmCOL_MAN1_OUTPUT_CSC_C23_C24_B 0x98c0
+#define mmOUTPUT_CSC_C31_C32_B 0x46c1
+#define mmCOL_MAN0_OUTPUT_CSC_C31_C32_B 0x46c1
+#define mmCOL_MAN1_OUTPUT_CSC_C31_C32_B 0x98c1
+#define mmOUTPUT_CSC_C33_C34_B 0x46c2
+#define mmCOL_MAN0_OUTPUT_CSC_C33_C34_B 0x46c2
+#define mmCOL_MAN1_OUTPUT_CSC_C33_C34_B 0x98c2
+#define mmDENORM_CLAMP_CONTROL 0x46c3
+#define mmCOL_MAN0_DENORM_CLAMP_CONTROL 0x46c3
+#define mmCOL_MAN1_DENORM_CLAMP_CONTROL 0x98c3
+#define mmDENORM_CLAMP_RANGE_R_CR 0x46c4
+#define mmCOL_MAN0_DENORM_CLAMP_RANGE_R_CR 0x46c4
+#define mmCOL_MAN1_DENORM_CLAMP_RANGE_R_CR 0x98c4
+#define mmDENORM_CLAMP_RANGE_G_Y 0x46c5
+#define mmCOL_MAN0_DENORM_CLAMP_RANGE_G_Y 0x46c5
+#define mmCOL_MAN1_DENORM_CLAMP_RANGE_G_Y 0x98c5
+#define mmDENORM_CLAMP_RANGE_B_CB 0x46c6
+#define mmCOL_MAN0_DENORM_CLAMP_RANGE_B_CB 0x46c6
+#define mmCOL_MAN1_DENORM_CLAMP_RANGE_B_CB 0x98c6
+#define mmCOL_MAN_FP_CONVERTED_FIELD 0x46c7
+#define mmCOL_MAN0_COL_MAN_FP_CONVERTED_FIELD 0x46c7
+#define mmCOL_MAN1_COL_MAN_FP_CONVERTED_FIELD 0x98c7
+#define mmGAMMA_CORR_CONTROL 0x46c8
+#define mmCOL_MAN0_GAMMA_CORR_CONTROL 0x46c8
+#define mmCOL_MAN1_GAMMA_CORR_CONTROL 0x98c8
+#define mmGAMMA_CORR_LUT_INDEX 0x46c9
+#define mmCOL_MAN0_GAMMA_CORR_LUT_INDEX 0x46c9
+#define mmCOL_MAN1_GAMMA_CORR_LUT_INDEX 0x98c9
+#define mmGAMMA_CORR_LUT_DATA 0x46ca
+#define mmCOL_MAN0_GAMMA_CORR_LUT_DATA 0x46ca
+#define mmCOL_MAN1_GAMMA_CORR_LUT_DATA 0x98ca
+#define mmGAMMA_CORR_LUT_WRITE_EN_MASK 0x46cb
+#define mmCOL_MAN0_GAMMA_CORR_LUT_WRITE_EN_MASK 0x46cb
+#define mmCOL_MAN1_GAMMA_CORR_LUT_WRITE_EN_MASK 0x98cb
+#define mmGAMMA_CORR_CNTLA_START_CNTL 0x46cc
+#define mmCOL_MAN0_GAMMA_CORR_CNTLA_START_CNTL 0x46cc
+#define mmCOL_MAN1_GAMMA_CORR_CNTLA_START_CNTL 0x98cc
+#define mmGAMMA_CORR_CNTLA_SLOPE_CNTL 0x46cd
+#define mmCOL_MAN0_GAMMA_CORR_CNTLA_SLOPE_CNTL 0x46cd
+#define mmCOL_MAN1_GAMMA_CORR_CNTLA_SLOPE_CNTL 0x98cd
+#define mmGAMMA_CORR_CNTLA_END_CNTL1 0x46ce
+#define mmCOL_MAN0_GAMMA_CORR_CNTLA_END_CNTL1 0x46ce
+#define mmCOL_MAN1_GAMMA_CORR_CNTLA_END_CNTL1 0x98ce
+#define mmGAMMA_CORR_CNTLA_END_CNTL2 0x46cf
+#define mmCOL_MAN0_GAMMA_CORR_CNTLA_END_CNTL2 0x46cf
+#define mmCOL_MAN1_GAMMA_CORR_CNTLA_END_CNTL2 0x98cf
+#define mmGAMMA_CORR_CNTLA_REGION_0_1 0x46d0
+#define mmCOL_MAN0_GAMMA_CORR_CNTLA_REGION_0_1 0x46d0
+#define mmCOL_MAN1_GAMMA_CORR_CNTLA_REGION_0_1 0x98d0
+#define mmGAMMA_CORR_CNTLA_REGION_2_3 0x46d1
+#define mmCOL_MAN0_GAMMA_CORR_CNTLA_REGION_2_3 0x46d1
+#define mmCOL_MAN1_GAMMA_CORR_CNTLA_REGION_2_3 0x98d1
+#define mmGAMMA_CORR_CNTLA_REGION_4_5 0x46d2
+#define mmCOL_MAN0_GAMMA_CORR_CNTLA_REGION_4_5 0x46d2
+#define mmCOL_MAN1_GAMMA_CORR_CNTLA_REGION_4_5 0x98d2
+#define mmGAMMA_CORR_CNTLA_REGION_6_7 0x46d3
+#define mmCOL_MAN0_GAMMA_CORR_CNTLA_REGION_6_7 0x46d3
+#define mmCOL_MAN1_GAMMA_CORR_CNTLA_REGION_6_7 0x98d3
+#define mmGAMMA_CORR_CNTLA_REGION_8_9 0x46d4
+#define mmCOL_MAN0_GAMMA_CORR_CNTLA_REGION_8_9 0x46d4
+#define mmCOL_MAN1_GAMMA_CORR_CNTLA_REGION_8_9 0x98d4
+#define mmGAMMA_CORR_CNTLA_REGION_10_11 0x46d5
+#define mmCOL_MAN0_GAMMA_CORR_CNTLA_REGION_10_11 0x46d5
+#define mmCOL_MAN1_GAMMA_CORR_CNTLA_REGION_10_11 0x98d5
+#define mmGAMMA_CORR_CNTLA_REGION_12_13 0x46d6
+#define mmCOL_MAN0_GAMMA_CORR_CNTLA_REGION_12_13 0x46d6
+#define mmCOL_MAN1_GAMMA_CORR_CNTLA_REGION_12_13 0x98d6
+#define mmGAMMA_CORR_CNTLA_REGION_14_15 0x46d7
+#define mmCOL_MAN0_GAMMA_CORR_CNTLA_REGION_14_15 0x46d7
+#define mmCOL_MAN1_GAMMA_CORR_CNTLA_REGION_14_15 0x98d7
+#define mmGAMMA_CORR_CNTLB_START_CNTL 0x46d8
+#define mmCOL_MAN0_GAMMA_CORR_CNTLB_START_CNTL 0x46d8
+#define mmCOL_MAN1_GAMMA_CORR_CNTLB_START_CNTL 0x98d8
+#define mmGAMMA_CORR_CNTLB_SLOPE_CNTL 0x46d9
+#define mmCOL_MAN0_GAMMA_CORR_CNTLB_SLOPE_CNTL 0x46d9
+#define mmCOL_MAN1_GAMMA_CORR_CNTLB_SLOPE_CNTL 0x98d9
+#define mmGAMMA_CORR_CNTLB_END_CNTL1 0x46da
+#define mmCOL_MAN0_GAMMA_CORR_CNTLB_END_CNTL1 0x46da
+#define mmCOL_MAN1_GAMMA_CORR_CNTLB_END_CNTL1 0x98da
+#define mmGAMMA_CORR_CNTLB_END_CNTL2 0x46db
+#define mmCOL_MAN0_GAMMA_CORR_CNTLB_END_CNTL2 0x46db
+#define mmCOL_MAN1_GAMMA_CORR_CNTLB_END_CNTL2 0x98db
+#define mmGAMMA_CORR_CNTLB_REGION_0_1 0x46dc
+#define mmCOL_MAN0_GAMMA_CORR_CNTLB_REGION_0_1 0x46dc
+#define mmCOL_MAN1_GAMMA_CORR_CNTLB_REGION_0_1 0x98dc
+#define mmGAMMA_CORR_CNTLB_REGION_2_3 0x46dd
+#define mmCOL_MAN0_GAMMA_CORR_CNTLB_REGION_2_3 0x46dd
+#define mmCOL_MAN1_GAMMA_CORR_CNTLB_REGION_2_3 0x98dd
+#define mmGAMMA_CORR_CNTLB_REGION_4_5 0x46de
+#define mmCOL_MAN0_GAMMA_CORR_CNTLB_REGION_4_5 0x46de
+#define mmCOL_MAN1_GAMMA_CORR_CNTLB_REGION_4_5 0x98de
+#define mmGAMMA_CORR_CNTLB_REGION_6_7 0x46df
+#define mmCOL_MAN0_GAMMA_CORR_CNTLB_REGION_6_7 0x46df
+#define mmCOL_MAN1_GAMMA_CORR_CNTLB_REGION_6_7 0x98df
+#define mmGAMMA_CORR_CNTLB_REGION_8_9 0x46e0
+#define mmCOL_MAN0_GAMMA_CORR_CNTLB_REGION_8_9 0x46e0
+#define mmCOL_MAN1_GAMMA_CORR_CNTLB_REGION_8_9 0x98e0
+#define mmGAMMA_CORR_CNTLB_REGION_10_11 0x46e1
+#define mmCOL_MAN0_GAMMA_CORR_CNTLB_REGION_10_11 0x46e1
+#define mmCOL_MAN1_GAMMA_CORR_CNTLB_REGION_10_11 0x98e1
+#define mmGAMMA_CORR_CNTLB_REGION_12_13 0x46e2
+#define mmCOL_MAN0_GAMMA_CORR_CNTLB_REGION_12_13 0x46e2
+#define mmCOL_MAN1_GAMMA_CORR_CNTLB_REGION_12_13 0x98e2
+#define mmGAMMA_CORR_CNTLB_REGION_14_15 0x46e3
+#define mmCOL_MAN0_GAMMA_CORR_CNTLB_REGION_14_15 0x46e3
+#define mmCOL_MAN1_GAMMA_CORR_CNTLB_REGION_14_15 0x98e3
+#define mmPACK_FIFO_ERROR 0x46e4
+#define mmCOL_MAN0_PACK_FIFO_ERROR 0x46e4
+#define mmCOL_MAN1_PACK_FIFO_ERROR 0x98e4
+#define mmOUTPUT_FIFO_ERROR 0x46e5
+#define mmCOL_MAN0_OUTPUT_FIFO_ERROR 0x46e5
+#define mmCOL_MAN1_OUTPUT_FIFO_ERROR 0x98e5
+#define mmINPUT_GAMMA_LUT_AUTOFILL 0x46e6
+#define mmCOL_MAN0_INPUT_GAMMA_LUT_AUTOFILL 0x46e6
+#define mmCOL_MAN1_INPUT_GAMMA_LUT_AUTOFILL 0x98e6
+#define mmINPUT_GAMMA_LUT_RW_INDEX 0x46e7
+#define mmCOL_MAN0_INPUT_GAMMA_LUT_RW_INDEX 0x46e7
+#define mmCOL_MAN1_INPUT_GAMMA_LUT_RW_INDEX 0x98e7
+#define mmINPUT_GAMMA_LUT_SEQ_COLOR 0x46e8
+#define mmCOL_MAN0_INPUT_GAMMA_LUT_SEQ_COLOR 0x46e8
+#define mmCOL_MAN1_INPUT_GAMMA_LUT_SEQ_COLOR 0x98e8
+#define mmINPUT_GAMMA_LUT_PWL_DATA 0x46e9
+#define mmCOL_MAN0_INPUT_GAMMA_LUT_PWL_DATA 0x46e9
+#define mmCOL_MAN1_INPUT_GAMMA_LUT_PWL_DATA 0x98e9
+#define mmINPUT_GAMMA_LUT_30_COLOR 0x46ea
+#define mmCOL_MAN0_INPUT_GAMMA_LUT_30_COLOR 0x46ea
+#define mmCOL_MAN1_INPUT_GAMMA_LUT_30_COLOR 0x98ea
+#define mmCOL_MAN_INPUT_GAMMA_CONTROL1 0x46eb
+#define mmCOL_MAN0_COL_MAN_INPUT_GAMMA_CONTROL1 0x46eb
+#define mmCOL_MAN1_COL_MAN_INPUT_GAMMA_CONTROL1 0x98eb
+#define mmCOL_MAN_INPUT_GAMMA_CONTROL2 0x46ec
+#define mmCOL_MAN0_COL_MAN_INPUT_GAMMA_CONTROL2 0x46ec
+#define mmCOL_MAN1_COL_MAN_INPUT_GAMMA_CONTROL2 0x98ec
+#define mmINPUT_GAMMA_BW_OFFSETS_B 0x46ed
+#define mmCOL_MAN0_INPUT_GAMMA_BW_OFFSETS_B 0x46ed
+#define mmCOL_MAN1_INPUT_GAMMA_BW_OFFSETS_B 0x98ed
+#define mmINPUT_GAMMA_BW_OFFSETS_G 0x46ee
+#define mmCOL_MAN0_INPUT_GAMMA_BW_OFFSETS_G 0x46ee
+#define mmCOL_MAN1_INPUT_GAMMA_BW_OFFSETS_G 0x98ee
+#define mmINPUT_GAMMA_BW_OFFSETS_R 0x46ef
+#define mmCOL_MAN0_INPUT_GAMMA_BW_OFFSETS_R 0x46ef
+#define mmCOL_MAN1_INPUT_GAMMA_BW_OFFSETS_R 0x98ef
+#define mmCOL_MAN_DEBUG_CONTROL 0x46f0
+#define mmCOL_MAN0_COL_MAN_DEBUG_CONTROL 0x46f0
+#define mmCOL_MAN1_COL_MAN_DEBUG_CONTROL 0x98f0
+#define mmCOL_MAN_TEST_DEBUG_INDEX 0x46f1
+#define mmCOL_MAN0_COL_MAN_TEST_DEBUG_INDEX 0x46f1
+#define mmCOL_MAN1_COL_MAN_TEST_DEBUG_INDEX 0x98f1
+#define mmCOL_MAN_TEST_DEBUG_DATA 0x46f3
+#define mmCOL_MAN0_COL_MAN_TEST_DEBUG_DATA 0x46f3
+#define mmCOL_MAN1_COL_MAN_TEST_DEBUG_DATA 0x98f3
+#define mmUNP_GRPH_ENABLE 0x4600
+#define mmUNP0_UNP_GRPH_ENABLE 0x4600
+#define mmUNP1_UNP_GRPH_ENABLE 0x9800
+#define mmUNP_GRPH_CONTROL 0x4601
+#define mmUNP0_UNP_GRPH_CONTROL 0x4601
+#define mmUNP1_UNP_GRPH_CONTROL 0x9801
+#define mmUNP_GRPH_CONTROL_C 0x4602
+#define mmUNP0_UNP_GRPH_CONTROL_C 0x4602
+#define mmUNP1_UNP_GRPH_CONTROL_C 0x9802
+#define mmUNP_GRPH_CONTROL_EXP 0x4603
+#define mmUNP0_UNP_GRPH_CONTROL_EXP 0x4603
+#define mmUNP1_UNP_GRPH_CONTROL_EXP 0x9803
+#define mmUNP_GRPH_SWAP_CNTL 0x4605
+#define mmUNP0_UNP_GRPH_SWAP_CNTL 0x4605
+#define mmUNP1_UNP_GRPH_SWAP_CNTL 0x9805
+#define mmUNP_GRPH_PRIMARY_SURFACE_ADDRESS_L 0x4606
+#define mmUNP0_UNP_GRPH_PRIMARY_SURFACE_ADDRESS_L 0x4606
+#define mmUNP1_UNP_GRPH_PRIMARY_SURFACE_ADDRESS_L 0x9806
+#define mmUNP_GRPH_PRIMARY_SURFACE_ADDRESS_C 0x4607
+#define mmUNP0_UNP_GRPH_PRIMARY_SURFACE_ADDRESS_C 0x4607
+#define mmUNP1_UNP_GRPH_PRIMARY_SURFACE_ADDRESS_C 0x9807
+#define mmUNP_GRPH_PRIMARY_SURFACE_ADDRESS_HIGH_L 0x4608
+#define mmUNP0_UNP_GRPH_PRIMARY_SURFACE_ADDRESS_HIGH_L 0x4608
+#define mmUNP1_UNP_GRPH_PRIMARY_SURFACE_ADDRESS_HIGH_L 0x9808
+#define mmUNP_GRPH_PRIMARY_SURFACE_ADDRESS_HIGH_C 0x4609
+#define mmUNP0_UNP_GRPH_PRIMARY_SURFACE_ADDRESS_HIGH_C 0x4609
+#define mmUNP1_UNP_GRPH_PRIMARY_SURFACE_ADDRESS_HIGH_C 0x9809
+#define mmUNP_GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_L 0x460a
+#define mmUNP0_UNP_GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_L 0x460a
+#define mmUNP1_UNP_GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_L 0x980a
+#define mmUNP_GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_C 0x460b
+#define mmUNP0_UNP_GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_C 0x460b
+#define mmUNP1_UNP_GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_C 0x980b
+#define mmUNP_GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_HIGH_L 0x460c
+#define mmUNP0_UNP_GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_HIGH_L 0x460c
+#define mmUNP1_UNP_GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_HIGH_L 0x980c
+#define mmUNP_GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_HIGH_C 0x460d
+#define mmUNP0_UNP_GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_HIGH_C 0x460d
+#define mmUNP1_UNP_GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_HIGH_C 0x980d
+#define mmUNP_GRPH_SECONDARY_SURFACE_ADDRESS_L 0x460e
+#define mmUNP0_UNP_GRPH_SECONDARY_SURFACE_ADDRESS_L 0x460e
+#define mmUNP1_UNP_GRPH_SECONDARY_SURFACE_ADDRESS_L 0x980e
+#define mmUNP_GRPH_SECONDARY_SURFACE_ADDRESS_C 0x460f
+#define mmUNP0_UNP_GRPH_SECONDARY_SURFACE_ADDRESS_C 0x460f
+#define mmUNP1_UNP_GRPH_SECONDARY_SURFACE_ADDRESS_C 0x980f
+#define mmUNP_GRPH_SECONDARY_SURFACE_ADDRESS_HIGH_L 0x4610
+#define mmUNP0_UNP_GRPH_SECONDARY_SURFACE_ADDRESS_HIGH_L 0x4610
+#define mmUNP1_UNP_GRPH_SECONDARY_SURFACE_ADDRESS_HIGH_L 0x9810
+#define mmUNP_GRPH_SECONDARY_SURFACE_ADDRESS_HIGH_C 0x4611
+#define mmUNP0_UNP_GRPH_SECONDARY_SURFACE_ADDRESS_HIGH_C 0x4611
+#define mmUNP1_UNP_GRPH_SECONDARY_SURFACE_ADDRESS_HIGH_C 0x9811
+#define mmUNP_GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_L 0x4612
+#define mmUNP0_UNP_GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_L 0x4612
+#define mmUNP1_UNP_GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_L 0x9812
+#define mmUNP_GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_C 0x4613
+#define mmUNP0_UNP_GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_C 0x4613
+#define mmUNP1_UNP_GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_C 0x9813
+#define mmUNP_GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_HIGH_L 0x4614
+#define mmUNP0_UNP_GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_HIGH_L 0x4614
+#define mmUNP1_UNP_GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_HIGH_L 0x9814
+#define mmUNP_GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_HIGH_C 0x4615
+#define mmUNP0_UNP_GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_HIGH_C 0x4615
+#define mmUNP1_UNP_GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_HIGH_C 0x9815
+#define mmUNP_GRPH_PITCH_L 0x4616
+#define mmUNP0_UNP_GRPH_PITCH_L 0x4616
+#define mmUNP1_UNP_GRPH_PITCH_L 0x9816
+#define mmUNP_GRPH_PITCH_C 0x4617
+#define mmUNP0_UNP_GRPH_PITCH_C 0x4617
+#define mmUNP1_UNP_GRPH_PITCH_C 0x9817
+#define mmUNP_GRPH_SURFACE_OFFSET_X_L 0x4618
+#define mmUNP0_UNP_GRPH_SURFACE_OFFSET_X_L 0x4618
+#define mmUNP1_UNP_GRPH_SURFACE_OFFSET_X_L 0x9818
+#define mmUNP_GRPH_SURFACE_OFFSET_X_C 0x4619
+#define mmUNP0_UNP_GRPH_SURFACE_OFFSET_X_C 0x4619
+#define mmUNP1_UNP_GRPH_SURFACE_OFFSET_X_C 0x9819
+#define mmUNP_GRPH_SURFACE_OFFSET_Y_L 0x461a
+#define mmUNP0_UNP_GRPH_SURFACE_OFFSET_Y_L 0x461a
+#define mmUNP1_UNP_GRPH_SURFACE_OFFSET_Y_L 0x981a
+#define mmUNP_GRPH_SURFACE_OFFSET_Y_C 0x461b
+#define mmUNP0_UNP_GRPH_SURFACE_OFFSET_Y_C 0x461b
+#define mmUNP1_UNP_GRPH_SURFACE_OFFSET_Y_C 0x981b
+#define mmUNP_GRPH_X_START_L 0x461c
+#define mmUNP0_UNP_GRPH_X_START_L 0x461c
+#define mmUNP1_UNP_GRPH_X_START_L 0x981c
+#define mmUNP_GRPH_X_START_C 0x461d
+#define mmUNP0_UNP_GRPH_X_START_C 0x461d
+#define mmUNP1_UNP_GRPH_X_START_C 0x981d
+#define mmUNP_GRPH_Y_START_L 0x461e
+#define mmUNP0_UNP_GRPH_Y_START_L 0x461e
+#define mmUNP1_UNP_GRPH_Y_START_L 0x981e
+#define mmUNP_GRPH_Y_START_C 0x461f
+#define mmUNP0_UNP_GRPH_Y_START_C 0x461f
+#define mmUNP1_UNP_GRPH_Y_START_C 0x981f
+#define mmUNP_GRPH_X_END_L 0x4620
+#define mmUNP0_UNP_GRPH_X_END_L 0x4620
+#define mmUNP1_UNP_GRPH_X_END_L 0x9820
+#define mmUNP_GRPH_X_END_C 0x4621
+#define mmUNP0_UNP_GRPH_X_END_C 0x4621
+#define mmUNP1_UNP_GRPH_X_END_C 0x9821
+#define mmUNP_GRPH_Y_END_L 0x4622
+#define mmUNP0_UNP_GRPH_Y_END_L 0x4622
+#define mmUNP1_UNP_GRPH_Y_END_L 0x9822
+#define mmUNP_GRPH_Y_END_C 0x4623
+#define mmUNP0_UNP_GRPH_Y_END_C 0x4623
+#define mmUNP1_UNP_GRPH_Y_END_C 0x9823
+#define mmUNP_GRPH_UPDATE 0x4624
+#define mmUNP0_UNP_GRPH_UPDATE 0x4624
+#define mmUNP1_UNP_GRPH_UPDATE 0x9824
+#define mmUNP_PIPE_OUTSTANDING_REQUEST_LIMIT 0x463a
+#define mmUNP0_UNP_PIPE_OUTSTANDING_REQUEST_LIMIT 0x463a
+#define mmUNP1_UNP_PIPE_OUTSTANDING_REQUEST_LIMIT 0x983a
+#define mmUNP_GRPH_SURFACE_ADDRESS_INUSE_L 0x4625
+#define mmUNP0_UNP_GRPH_SURFACE_ADDRESS_INUSE_L 0x4625
+#define mmUNP1_UNP_GRPH_SURFACE_ADDRESS_INUSE_L 0x9825
+#define mmUNP_GRPH_SURFACE_ADDRESS_INUSE_C 0x4626
+#define mmUNP0_UNP_GRPH_SURFACE_ADDRESS_INUSE_C 0x4626
+#define mmUNP1_UNP_GRPH_SURFACE_ADDRESS_INUSE_C 0x9826
+#define mmUNP_GRPH_SURFACE_ADDRESS_HIGH_INUSE_L 0x4627
+#define mmUNP0_UNP_GRPH_SURFACE_ADDRESS_HIGH_INUSE_L 0x4627
+#define mmUNP1_UNP_GRPH_SURFACE_ADDRESS_HIGH_INUSE_L 0x9827
+#define mmUNP_GRPH_SURFACE_ADDRESS_HIGH_INUSE_C 0x4628
+#define mmUNP0_UNP_GRPH_SURFACE_ADDRESS_HIGH_INUSE_C 0x4628
+#define mmUNP1_UNP_GRPH_SURFACE_ADDRESS_HIGH_INUSE_C 0x9828
+#define mmUNP_DVMM_PTE_CONTROL 0x4629
+#define mmUNP_GRPH_INTERRUPT_STATUS 0x462b
+#define mmUNP0_UNP_GRPH_INTERRUPT_STATUS 0x462b
+#define mmUNP1_UNP_GRPH_INTERRUPT_STATUS 0x982b
+#define mmUNP_GRPH_INTERRUPT_CONTROL 0x462c
+#define mmUNP0_UNP_GRPH_INTERRUPT_CONTROL 0x462c
+#define mmUNP1_UNP_GRPH_INTERRUPT_CONTROL 0x982c
+#define mmUNP_GRPH_STEREOSYNC_FLIP 0x462e
+#define mmUNP0_UNP_GRPH_STEREOSYNC_FLIP 0x462e
+#define mmUNP1_UNP_GRPH_STEREOSYNC_FLIP 0x982e
+#define mmUNP_FLIP_CONTROL 0x462f
+#define mmUNP0_UNP_FLIP_CONTROL 0x462f
+#define mmUNP1_UNP_FLIP_CONTROL 0x982f
+#define mmUNP_CRC_CONTROL 0x4630
+#define mmUNP0_UNP_CRC_CONTROL 0x4630
+#define mmUNP1_UNP_CRC_CONTROL 0x9830
+#define mmUNP_CRC_MASK 0x4631
+#define mmUNP0_UNP_CRC_MASK 0x4631
+#define mmUNP1_UNP_CRC_MASK 0x9831
+#define mmUNP_CRC_CURRENT 0x4632
+#define mmUNP0_UNP_CRC_CURRENT 0x4632
+#define mmUNP1_UNP_CRC_CURRENT 0x9832
+#define mmUNP_CRC_LAST 0x4633
+#define mmUNP0_UNP_CRC_LAST 0x4633
+#define mmUNP1_UNP_CRC_LAST 0x9833
+#define mmUNP_LB_DATA_GAP_BETWEEN_CHUNK 0x4634
+#define mmUNP0_UNP_LB_DATA_GAP_BETWEEN_CHUNK 0x4634
+#define mmUNP1_UNP_LB_DATA_GAP_BETWEEN_CHUNK 0x9834
+#define mmUNP_HW_ROTATION 0x4635
+#define mmUNP0_UNP_HW_ROTATION 0x4635
+#define mmUNP1_UNP_HW_ROTATION 0x9835
+#define mmUNP_DEBUG 0x4636
+#define mmUNP0_UNP_DEBUG 0x4636
+#define mmUNP1_UNP_DEBUG 0x9836
+#define mmUNP_DEBUG2 0x4637
+#define mmUNP0_UNP_DEBUG2 0x4637
+#define mmUNP1_UNP_DEBUG2 0x9837
+#define mmUNP_DVMM_DEBUG 0x463b
+#define mmUNP0_UNP_DVMM_DEBUG 0x463b
+#define mmUNP1_UNP_DVMM_DEBUG 0x983b
+#define mmUNP_TEST_DEBUG_INDEX 0x4638
+#define mmUNP0_UNP_TEST_DEBUG_INDEX 0x4638
+#define mmUNP1_UNP_TEST_DEBUG_INDEX 0x9838
+#define mmUNP_TEST_DEBUG_DATA 0x4639
+#define mmUNP0_UNP_TEST_DEBUG_DATA 0x4639
+#define mmUNP1_UNP_TEST_DEBUG_DATA 0x9839
+#define mmGENMO_WT 0xf0
+#define mmGENMO_RD 0xf3
+#define mmGENENB 0xf0
+#define mmGENFC_WT 0xee
+#define mmVGA0_GENFC_WT 0xee
+#define mmVGA1_GENFC_WT 0xf6
+#define mmGENFC_RD 0xf2
+#define mmGENS0 0xf0
+#define mmGENS1 0xee
+#define mmVGA0_GENS1 0xee
+#define mmVGA1_GENS1 0xf6
+#define mmDAC_DATA 0xf2
+#define mmDAC_MASK 0xf1
+#define mmDAC_R_INDEX 0xf1
+#define mmDAC_W_INDEX 0xf2
+#define mmSEQ8_IDX 0xf1
+#define mmSEQ8_DATA 0xf1
+#define ixSEQ00 0x0
+#define ixSEQ01 0x1
+#define ixSEQ02 0x2
+#define ixSEQ03 0x3
+#define ixSEQ04 0x4
+#define mmCRTC8_IDX 0xed
+#define mmVGA0_CRTC8_IDX 0xed
+#define mmVGA1_CRTC8_IDX 0xf5
+#define mmCRTC8_DATA 0xed
+#define mmVGA0_CRTC8_DATA 0xed
+#define mmVGA1_CRTC8_DATA 0xf5
+#define ixCRT00 0x0
+#define ixCRT01 0x1
+#define ixCRT02 0x2
+#define ixCRT03 0x3
+#define ixCRT04 0x4
+#define ixCRT05 0x5
+#define ixCRT06 0x6
+#define ixCRT07 0x7
+#define ixCRT08 0x8
+#define ixCRT09 0x9
+#define ixCRT0A 0xa
+#define ixCRT0B 0xb
+#define ixCRT0C 0xc
+#define ixCRT0D 0xd
+#define ixCRT0E 0xe
+#define ixCRT0F 0xf
+#define ixCRT10 0x10
+#define ixCRT11 0x11
+#define ixCRT12 0x12
+#define ixCRT13 0x13
+#define ixCRT14 0x14
+#define ixCRT15 0x15
+#define ixCRT16 0x16
+#define ixCRT17 0x17
+#define ixCRT18 0x18
+#define ixCRT1E 0x1e
+#define ixCRT1F 0x1f
+#define ixCRT22 0x22
+#define mmGRPH8_IDX 0xf3
+#define mmGRPH8_DATA 0xf3
+#define ixGRA00 0x0
+#define ixGRA01 0x1
+#define ixGRA02 0x2
+#define ixGRA03 0x3
+#define ixGRA04 0x4
+#define ixGRA05 0x5
+#define ixGRA06 0x6
+#define ixGRA07 0x7
+#define ixGRA08 0x8
+#define mmATTRX 0xf0
+#define mmATTRDW 0xf0
+#define mmATTRDR 0xf0
+#define ixATTR00 0x0
+#define ixATTR01 0x1
+#define ixATTR02 0x2
+#define ixATTR03 0x3
+#define ixATTR04 0x4
+#define ixATTR05 0x5
+#define ixATTR06 0x6
+#define ixATTR07 0x7
+#define ixATTR08 0x8
+#define ixATTR09 0x9
+#define ixATTR0A 0xa
+#define ixATTR0B 0xb
+#define ixATTR0C 0xc
+#define ixATTR0D 0xd
+#define ixATTR0E 0xe
+#define ixATTR0F 0xf
+#define ixATTR10 0x10
+#define ixATTR11 0x11
+#define ixATTR12 0x12
+#define ixATTR13 0x13
+#define ixATTR14 0x14
+#define mmVGA_RENDER_CONTROL 0xc0
+#define mmVGA_SOURCE_SELECT 0xfc
+#define mmVGA_SEQUENCER_RESET_CONTROL 0xc1
+#define mmVGA_MODE_CONTROL 0xc2
+#define mmVGA_SURFACE_PITCH_SELECT 0xc3
+#define mmVGA_MEMORY_BASE_ADDRESS 0xc4
+#define mmVGA_MEMORY_BASE_ADDRESS_HIGH 0xc9
+#define mmVGA_DISPBUF1_SURFACE_ADDR 0xc6
+#define mmVGA_DISPBUF2_SURFACE_ADDR 0xc8
+#define mmVGA_HDP_CONTROL 0xca
+#define mmVGA_CACHE_CONTROL 0xcb
+#define mmD1VGA_CONTROL 0xcc
+#define mmD2VGA_CONTROL 0xce
+#define mmD3VGA_CONTROL 0xf8
+#define mmD4VGA_CONTROL 0xf9
+#define mmD5VGA_CONTROL 0xfa
+#define mmD6VGA_CONTROL 0xfb
+#define mmVGA_HW_DEBUG 0xcf
+#define mmVGA_STATUS 0xd0
+#define mmVGA_INTERRUPT_CONTROL 0xd1
+#define mmVGA_STATUS_CLEAR 0xd2
+#define mmVGA_INTERRUPT_STATUS 0xd3
+#define mmVGA_MAIN_CONTROL 0xd4
+#define mmVGA_TEST_CONTROL 0xd5
+#define mmVGA_DEBUG_READBACK_INDEX 0xd6
+#define mmVGA_DEBUG_READBACK_DATA 0xd7
+#define mmVGA_MEM_WRITE_PAGE_ADDR 0x12
+#define mmVGA_MEM_READ_PAGE_ADDR 0x13
+#define mmVGA_TEST_DEBUG_INDEX 0xc5
+#define mmVGA_TEST_DEBUG_DATA 0xc7
+#define ixVGADCC_DBG_DCCIF_C 0x7e
+#define mmBPHYC_DAC_MACRO_CNTL 0x48b9
+#define mmBPHYC_DAC_AUTO_CALIB_CONTROL 0x48ba
+#define mmDPG_PIPE_ARBITRATION_CONTROL1 0x1b30
+#define mmDMIF_PG0_DPG_PIPE_ARBITRATION_CONTROL1 0x1b30
+#define mmDMIF_PG1_DPG_PIPE_ARBITRATION_CONTROL1 0x1d30
+#define mmDMIF_PG2_DPG_PIPE_ARBITRATION_CONTROL1 0x1f30
+#define mmDMIF_PG3_DPG_PIPE_ARBITRATION_CONTROL1 0x4130
+#define mmDMIF_PG4_DPG_PIPE_ARBITRATION_CONTROL1 0x4330
+#define mmDMIF_PG5_DPG_PIPE_ARBITRATION_CONTROL1 0x4530
+#define mmDPG_PIPE_ARBITRATION_CONTROL2 0x1b31
+#define mmDMIF_PG0_DPG_PIPE_ARBITRATION_CONTROL2 0x1b31
+#define mmDMIF_PG1_DPG_PIPE_ARBITRATION_CONTROL2 0x1d31
+#define mmDMIF_PG2_DPG_PIPE_ARBITRATION_CONTROL2 0x1f31
+#define mmDMIF_PG3_DPG_PIPE_ARBITRATION_CONTROL2 0x4131
+#define mmDMIF_PG4_DPG_PIPE_ARBITRATION_CONTROL2 0x4331
+#define mmDMIF_PG5_DPG_PIPE_ARBITRATION_CONTROL2 0x4531
+#define mmDPG_WATERMARK_MASK_CONTROL 0x1b32
+#define mmDMIF_PG0_DPG_WATERMARK_MASK_CONTROL 0x1b32
+#define mmDMIF_PG1_DPG_WATERMARK_MASK_CONTROL 0x1d32
+#define mmDMIF_PG2_DPG_WATERMARK_MASK_CONTROL 0x1f32
+#define mmDMIF_PG3_DPG_WATERMARK_MASK_CONTROL 0x4132
+#define mmDMIF_PG4_DPG_WATERMARK_MASK_CONTROL 0x4332
+#define mmDMIF_PG5_DPG_WATERMARK_MASK_CONTROL 0x4532
+#define mmDPG_PIPE_URGENCY_CONTROL 0x1b33
+#define mmDMIF_PG0_DPG_PIPE_URGENCY_CONTROL 0x1b33
+#define mmDMIF_PG1_DPG_PIPE_URGENCY_CONTROL 0x1d33
+#define mmDMIF_PG2_DPG_PIPE_URGENCY_CONTROL 0x1f33
+#define mmDMIF_PG3_DPG_PIPE_URGENCY_CONTROL 0x4133
+#define mmDMIF_PG4_DPG_PIPE_URGENCY_CONTROL 0x4333
+#define mmDMIF_PG5_DPG_PIPE_URGENCY_CONTROL 0x4533
+#define mmDPG_PIPE_DPM_CONTROL 0x1b34
+#define mmDMIF_PG0_DPG_PIPE_DPM_CONTROL 0x1b34
+#define mmDMIF_PG1_DPG_PIPE_DPM_CONTROL 0x1d34
+#define mmDMIF_PG2_DPG_PIPE_DPM_CONTROL 0x1f34
+#define mmDMIF_PG3_DPG_PIPE_DPM_CONTROL 0x4134
+#define mmDMIF_PG4_DPG_PIPE_DPM_CONTROL 0x4334
+#define mmDMIF_PG5_DPG_PIPE_DPM_CONTROL 0x4534
+#define mmDPG_PIPE_STUTTER_CONTROL 0x1b35
+#define mmDMIF_PG0_DPG_PIPE_STUTTER_CONTROL 0x1b35
+#define mmDMIF_PG1_DPG_PIPE_STUTTER_CONTROL 0x1d35
+#define mmDMIF_PG2_DPG_PIPE_STUTTER_CONTROL 0x1f35
+#define mmDMIF_PG3_DPG_PIPE_STUTTER_CONTROL 0x4135
+#define mmDMIF_PG4_DPG_PIPE_STUTTER_CONTROL 0x4335
+#define mmDMIF_PG5_DPG_PIPE_STUTTER_CONTROL 0x4535
+#define mmDPG_PIPE_NB_PSTATE_CHANGE_CONTROL 0x1b36
+#define mmDMIF_PG0_DPG_PIPE_NB_PSTATE_CHANGE_CONTROL 0x1b36
+#define mmDMIF_PG1_DPG_PIPE_NB_PSTATE_CHANGE_CONTROL 0x1d36
+#define mmDMIF_PG2_DPG_PIPE_NB_PSTATE_CHANGE_CONTROL 0x1f36
+#define mmDMIF_PG3_DPG_PIPE_NB_PSTATE_CHANGE_CONTROL 0x4136
+#define mmDMIF_PG4_DPG_PIPE_NB_PSTATE_CHANGE_CONTROL 0x4336
+#define mmDMIF_PG5_DPG_PIPE_NB_PSTATE_CHANGE_CONTROL 0x4536
+#define mmDPG_PIPE_STUTTER_CONTROL_NONLPTCH 0x1b37
+#define mmDMIF_PG0_DPG_PIPE_STUTTER_CONTROL_NONLPTCH 0x1b37
+#define mmDMIF_PG1_DPG_PIPE_STUTTER_CONTROL_NONLPTCH 0x1d37
+#define mmDMIF_PG2_DPG_PIPE_STUTTER_CONTROL_NONLPTCH 0x1f37
+#define mmDMIF_PG3_DPG_PIPE_STUTTER_CONTROL_NONLPTCH 0x4137
+#define mmDMIF_PG4_DPG_PIPE_STUTTER_CONTROL_NONLPTCH 0x4337
+#define mmDMIF_PG5_DPG_PIPE_STUTTER_CONTROL_NONLPTCH 0x4537
+#define mmDPG_REPEATER_PROGRAM 0x1b3a
+#define mmDMIF_PG0_DPG_REPEATER_PROGRAM 0x1b3a
+#define mmDMIF_PG1_DPG_REPEATER_PROGRAM 0x1d3a
+#define mmDMIF_PG2_DPG_REPEATER_PROGRAM 0x1f3a
+#define mmDMIF_PG3_DPG_REPEATER_PROGRAM 0x413a
+#define mmDMIF_PG4_DPG_REPEATER_PROGRAM 0x433a
+#define mmDMIF_PG5_DPG_REPEATER_PROGRAM 0x453a
+#define mmDPG_HW_DEBUG_A 0x1b3b
+#define mmDMIF_PG0_DPG_HW_DEBUG_A 0x1b3b
+#define mmDMIF_PG1_DPG_HW_DEBUG_A 0x1d3b
+#define mmDMIF_PG2_DPG_HW_DEBUG_A 0x1f3b
+#define mmDMIF_PG3_DPG_HW_DEBUG_A 0x413b
+#define mmDMIF_PG4_DPG_HW_DEBUG_A 0x433b
+#define mmDMIF_PG5_DPG_HW_DEBUG_A 0x453b
+#define mmDPG_HW_DEBUG_B 0x1b3c
+#define mmDMIF_PG0_DPG_HW_DEBUG_B 0x1b3c
+#define mmDMIF_PG1_DPG_HW_DEBUG_B 0x1d3c
+#define mmDMIF_PG2_DPG_HW_DEBUG_B 0x1f3c
+#define mmDMIF_PG3_DPG_HW_DEBUG_B 0x413c
+#define mmDMIF_PG4_DPG_HW_DEBUG_B 0x433c
+#define mmDMIF_PG5_DPG_HW_DEBUG_B 0x453c
+#define mmDPG_HW_DEBUG_11 0x1b3d
+#define mmDMIF_PG0_DPG_HW_DEBUG_11 0x1b3d
+#define mmDMIF_PG1_DPG_HW_DEBUG_11 0x1d3d
+#define mmDMIF_PG2_DPG_HW_DEBUG_11 0x1f3d
+#define mmDMIF_PG3_DPG_HW_DEBUG_11 0x413d
+#define mmDMIF_PG4_DPG_HW_DEBUG_11 0x433d
+#define mmDMIF_PG5_DPG_HW_DEBUG_11 0x453d
+#define mmDPG_CHK_PRE_PROC_CNTL 0x1b3e
+#define mmDMIF_PG0_DPG_CHK_PRE_PROC_CNTL 0x1b3e
+#define mmDMIF_PG1_DPG_CHK_PRE_PROC_CNTL 0x1d3e
+#define mmDMIF_PG2_DPG_CHK_PRE_PROC_CNTL 0x1f3e
+#define mmDMIF_PG3_DPG_CHK_PRE_PROC_CNTL 0x413e
+#define mmDMIF_PG4_DPG_CHK_PRE_PROC_CNTL 0x433e
+#define mmDMIF_PG5_DPG_CHK_PRE_PROC_CNTL 0x453e
+#define mmDPG_DVMM_STATUS 0x1b3f
+#define mmDMIF_PG0_DPG_DVMM_STATUS 0x1b3f
+#define mmDMIF_PG1_DPG_DVMM_STATUS 0x1d3f
+#define mmDMIF_PG2_DPG_DVMM_STATUS 0x1f3f
+#define mmDMIF_PG3_DPG_DVMM_STATUS 0x413f
+#define mmDMIF_PG4_DPG_DVMM_STATUS 0x433f
+#define mmDMIF_PG5_DPG_DVMM_STATUS 0x453f
+#define mmDPG_TEST_DEBUG_INDEX 0x1b38
+#define mmDMIF_PG0_DPG_TEST_DEBUG_INDEX 0x1b38
+#define mmDMIF_PG1_DPG_TEST_DEBUG_INDEX 0x1d38
+#define mmDMIF_PG2_DPG_TEST_DEBUG_INDEX 0x1f38
+#define mmDMIF_PG3_DPG_TEST_DEBUG_INDEX 0x4138
+#define mmDMIF_PG4_DPG_TEST_DEBUG_INDEX 0x4338
+#define mmDMIF_PG5_DPG_TEST_DEBUG_INDEX 0x4538
+#define mmDPG_TEST_DEBUG_DATA 0x1b39
+#define mmDMIF_PG0_DPG_TEST_DEBUG_DATA 0x1b39
+#define mmDMIF_PG1_DPG_TEST_DEBUG_DATA 0x1d39
+#define mmDMIF_PG2_DPG_TEST_DEBUG_DATA 0x1f39
+#define mmDMIF_PG3_DPG_TEST_DEBUG_DATA 0x4139
+#define mmDMIF_PG4_DPG_TEST_DEBUG_DATA 0x4339
+#define mmDMIF_PG5_DPG_TEST_DEBUG_DATA 0x4539
+#define mmDPGV0_PIPE_ARBITRATION_CONTROL1 0x4730
+#define mmDMIFV_PG0_DPGV0_PIPE_ARBITRATION_CONTROL1 0x4730
+#define mmDMIFV_PG1_DPGV0_PIPE_ARBITRATION_CONTROL1 0x9930
+#define mmDPGV1_PIPE_ARBITRATION_CONTROL1 0x473d
+#define mmDMIFV_PG0_DPGV1_PIPE_ARBITRATION_CONTROL1 0x473d
+#define mmDMIFV_PG1_DPGV1_PIPE_ARBITRATION_CONTROL1 0x993d
+#define mmDPGV0_PIPE_ARBITRATION_CONTROL2 0x4731
+#define mmDMIFV_PG0_DPGV0_PIPE_ARBITRATION_CONTROL2 0x4731
+#define mmDMIFV_PG1_DPGV0_PIPE_ARBITRATION_CONTROL2 0x9931
+#define mmDPGV1_PIPE_ARBITRATION_CONTROL2 0x473e
+#define mmDMIFV_PG0_DPGV1_PIPE_ARBITRATION_CONTROL2 0x473e
+#define mmDMIFV_PG1_DPGV1_PIPE_ARBITRATION_CONTROL2 0x993e
+#define mmDPGV0_WATERMARK_MASK_CONTROL 0x4732
+#define mmDMIFV_PG0_DPGV0_WATERMARK_MASK_CONTROL 0x4732
+#define mmDMIFV_PG1_DPGV0_WATERMARK_MASK_CONTROL 0x9932
+#define mmDPGV1_WATERMARK_MASK_CONTROL 0x473f
+#define mmDMIFV_PG0_DPGV1_WATERMARK_MASK_CONTROL 0x473f
+#define mmDMIFV_PG1_DPGV1_WATERMARK_MASK_CONTROL 0x993f
+#define mmDPGV0_PIPE_URGENCY_CONTROL 0x4733
+#define mmDMIFV_PG0_DPGV0_PIPE_URGENCY_CONTROL 0x4733
+#define mmDMIFV_PG1_DPGV0_PIPE_URGENCY_CONTROL 0x9933
+#define mmDPGV1_PIPE_URGENCY_CONTROL 0x4740
+#define mmDMIFV_PG0_DPGV1_PIPE_URGENCY_CONTROL 0x4740
+#define mmDMIFV_PG1_DPGV1_PIPE_URGENCY_CONTROL 0x9940
+#define mmDPGV0_PIPE_DPM_CONTROL 0x4734
+#define mmDMIFV_PG0_DPGV0_PIPE_DPM_CONTROL 0x4734
+#define mmDMIFV_PG1_DPGV0_PIPE_DPM_CONTROL 0x9934
+#define mmDPGV1_PIPE_DPM_CONTROL 0x4741
+#define mmDMIFV_PG0_DPGV1_PIPE_DPM_CONTROL 0x4741
+#define mmDMIFV_PG1_DPGV1_PIPE_DPM_CONTROL 0x9941
+#define mmDPGV0_PIPE_STUTTER_CONTROL 0x4735
+#define mmDMIFV_PG0_DPGV0_PIPE_STUTTER_CONTROL 0x4735
+#define mmDMIFV_PG1_DPGV0_PIPE_STUTTER_CONTROL 0x9935
+#define mmDPGV1_PIPE_STUTTER_CONTROL 0x4742
+#define mmDMIFV_PG0_DPGV1_PIPE_STUTTER_CONTROL 0x4742
+#define mmDMIFV_PG1_DPGV1_PIPE_STUTTER_CONTROL 0x9942
+#define mmDPGV0_PIPE_NB_PSTATE_CHANGE_CONTROL 0x4736
+#define mmDMIFV_PG0_DPGV0_PIPE_NB_PSTATE_CHANGE_CONTROL 0x4736
+#define mmDMIFV_PG1_DPGV0_PIPE_NB_PSTATE_CHANGE_CONTROL 0x9936
+#define mmDPGV1_PIPE_NB_PSTATE_CHANGE_CONTROL 0x4743
+#define mmDMIFV_PG0_DPGV1_PIPE_NB_PSTATE_CHANGE_CONTROL 0x4743
+#define mmDMIFV_PG1_DPGV1_PIPE_NB_PSTATE_CHANGE_CONTROL 0x9943
+#define mmDPGV0_PIPE_STUTTER_CONTROL_NONLPTCH 0x4737
+#define mmDMIFV_PG0_DPGV0_PIPE_STUTTER_CONTROL_NONLPTCH 0x4737
+#define mmDMIFV_PG1_DPGV0_PIPE_STUTTER_CONTROL_NONLPTCH 0x9937
+#define mmDPGV1_PIPE_STUTTER_CONTROL_NONLPTCH 0x4744
+#define mmDMIFV_PG0_DPGV1_PIPE_STUTTER_CONTROL_NONLPTCH 0x4744
+#define mmDMIFV_PG1_DPGV1_PIPE_STUTTER_CONTROL_NONLPTCH 0x9944
+#define mmDPGV0_REPEATER_PROGRAM 0x4738
+#define mmDMIFV_PG0_DPGV0_REPEATER_PROGRAM 0x4738
+#define mmDMIFV_PG1_DPGV0_REPEATER_PROGRAM 0x9938
+#define mmDPGV1_REPEATER_PROGRAM 0x4745
+#define mmDMIFV_PG0_DPGV1_REPEATER_PROGRAM 0x4745
+#define mmDMIFV_PG1_DPGV1_REPEATER_PROGRAM 0x9945
+#define mmDPGV0_HW_DEBUG_A 0x4739
+#define mmDMIFV_PG0_DPGV0_HW_DEBUG_A 0x4739
+#define mmDMIFV_PG1_DPGV0_HW_DEBUG_A 0x9939
+#define mmDPGV1_HW_DEBUG_A 0x4746
+#define mmDMIFV_PG0_DPGV1_HW_DEBUG_A 0x4746
+#define mmDMIFV_PG1_DPGV1_HW_DEBUG_A 0x9946
+#define mmDPGV0_HW_DEBUG_B 0x473a
+#define mmDMIFV_PG0_DPGV0_HW_DEBUG_B 0x473a
+#define mmDMIFV_PG1_DPGV0_HW_DEBUG_B 0x993a
+#define mmDPGV1_HW_DEBUG_B 0x4747
+#define mmDMIFV_PG0_DPGV1_HW_DEBUG_B 0x4747
+#define mmDMIFV_PG1_DPGV1_HW_DEBUG_B 0x9947
+#define mmDPGV0_HW_DEBUG_11 0x473b
+#define mmDMIFV_PG0_DPGV0_HW_DEBUG_11 0x473b
+#define mmDMIFV_PG1_DPGV0_HW_DEBUG_11 0x993b
+#define mmDPGV1_HW_DEBUG_11 0x4748
+#define mmDMIFV_PG0_DPGV1_HW_DEBUG_11 0x4748
+#define mmDMIFV_PG1_DPGV1_HW_DEBUG_11 0x9948
+#define mmDPGV0_CHK_PRE_PROC_CNTL 0x473c
+#define mmDMIFV_PG0_DPGV0_CHK_PRE_PROC_CNTL 0x473c
+#define mmDMIFV_PG1_DPGV0_CHK_PRE_PROC_CNTL 0x993c
+#define mmDPGV1_CHK_PRE_PROC_CNTL 0x4749
+#define mmDMIFV_PG0_DPGV1_CHK_PRE_PROC_CNTL 0x4749
+#define mmDMIFV_PG1_DPGV1_CHK_PRE_PROC_CNTL 0x9949
+#define mmDPGV_TEST_DEBUG_INDEX 0x474e
+#define mmDMIFV_PG0_DPGV_TEST_DEBUG_INDEX 0x474e
+#define mmDMIFV_PG1_DPGV_TEST_DEBUG_INDEX 0x994e
+#define mmDPGV_TEST_DEBUG_DATA 0x474f
+#define mmDMIFV_PG0_DPGV_TEST_DEBUG_DATA 0x474f
+#define mmDMIFV_PG1_DPGV_TEST_DEBUG_DATA 0x994f
+#define mmAZROOT_IMMEDIATE_COMMAND_OUTPUT_INTERFACE_INDEX 0x18
+#define mmAZROOT_IMMEDIATE_COMMAND_OUTPUT_INTERFACE_DATA 0x18
+#define ixAZALIA_F2_CODEC_ROOT_PARAMETER_VENDOR_AND_DEVICE_ID 0xf00
+#define ixAZALIA_F2_CODEC_ROOT_PARAMETER_REVISION_ID 0xf02
+#define ixAZALIA_F2_CODEC_ROOT_PARAMETER_SUBORDINATE_NODE_COUNT 0xf04
+#define ixAZALIA_F2_CODEC_FUNCTION_PARAMETER_SUBORDINATE_NODE_COUNT 0x1f04
+#define ixAZALIA_F2_CODEC_FUNCTION_PARAMETER_GROUP_TYPE 0x1f05
+#define ixAZALIA_F2_CODEC_FUNCTION_PARAMETER_SUPPORTED_SIZE_RATES 0x1f0a
+#define ixAZALIA_F2_CODEC_FUNCTION_PARAMETER_STREAM_FORMATS 0x1f0b
+#define ixAZALIA_F2_CODEC_FUNCTION_PARAMETER_POWER_STATES 0x1f0f
+#define ixAZALIA_F2_CODEC_FUNCTION_CONTROL_POWER_STATE 0x1705
+#define ixAZALIA_F2_CODEC_FUNCTION_CONTROL_RESET 0x17ff
+#define ixAZALIA_F2_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID 0x1720
+#define ixAZALIA_F2_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID_2 0x1721
+#define ixAZALIA_F2_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID_3 0x1722
+#define ixAZALIA_F2_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID_4 0x1723
+#define ixAZALIA_F2_CODEC_FUNCTION_CONTROL_CONVERTER_SYNCHRONIZATION 0x1770
+#define mmAZALIA_F0_CODEC_ROOT_PARAMETER_VENDOR_AND_DEVICE_ID 0x1828
+#define mmAZALIA_F0_CODEC_ROOT_PARAMETER_REVISION_ID 0x1829
+#define mmAZALIA_F0_CODEC_CHANNEL_COUNT_CONTROL 0x182a
+#define mmAZALIA_F0_CODEC_RESYNC_FIFO_CONTROL 0x182b
+#define mmAZALIA_F0_CODEC_FUNCTION_PARAMETER_GROUP_TYPE 0x182c
+#define mmAZALIA_F0_CODEC_FUNCTION_PARAMETER_SUPPORTED_SIZE_RATES 0x182d
+#define mmAZALIA_F0_CODEC_FUNCTION_PARAMETER_STREAM_FORMATS 0x182e
+#define mmAZALIA_F0_CODEC_FUNCTION_PARAMETER_POWER_STATES 0x182f
+#define mmAZALIA_F0_CODEC_FUNCTION_CONTROL_POWER_STATE 0x1830
+#define mmAZALIA_F0_CODEC_FUNCTION_CONTROL_RESET 0x1831
+#define mmAZALIA_F0_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID 0x1832
+#define mmAZALIA_F0_CODEC_FUNCTION_CONTROL_CONVERTER_SYNCHRONIZATION 0x1833
+#define mmCC_RCU_DC_AUDIO_PORT_CONNECTIVITY 0x1834
+#define mmCC_RCU_DC_AUDIO_INPUT_PORT_CONNECTIVITY 0x1835
+#define mmAZALIA_F0_CODEC_DEBUG 0x1836
+#define mmAZALIA_F0_GTC_GROUP_OFFSET0 0x1837
+#define mmAZALIA_F0_GTC_GROUP_OFFSET1 0x1838
+#define mmAZALIA_F0_GTC_GROUP_OFFSET2 0x1839
+#define mmAZALIA_F0_GTC_GROUP_OFFSET3 0x183a
+#define mmAZALIA_F0_GTC_GROUP_OFFSET4 0x183b
+#define mmAZALIA_F0_GTC_GROUP_OFFSET5 0x183c
+#define mmAZALIA_F0_GTC_GROUP_OFFSET6 0x183d
+#define mmGLOBAL_CAPABILITIES 0x0
+#define mmMINOR_VERSION 0x0
+#define mmMAJOR_VERSION 0x0
+#define mmOUTPUT_PAYLOAD_CAPABILITY 0x1
+#define mmINPUT_PAYLOAD_CAPABILITY 0x1
+#define mmGLOBAL_CONTROL 0x2
+#define mmWAKE_ENABLE 0x3
+#define mmSTATE_CHANGE_STATUS 0x3
+#define mmGLOBAL_STATUS 0x4
+#define mmOUTPUT_STREAM_PAYLOAD_CAPABILITY 0x6
+#define mmINPUT_STREAM_PAYLOAD_CAPABILITY 0x6
+#define mmINTERRUPT_CONTROL 0x8
+#define mmINTERRUPT_STATUS 0x9
+#define mmWALL_CLOCK_COUNTER 0xc
+#define mmSTREAM_SYNCHRONIZATION 0xe
+#define mmCORB_LOWER_BASE_ADDRESS 0x10
+#define mmCORB_UPPER_BASE_ADDRESS 0x11
+#define mmCORB_WRITE_POINTER 0x12
+#define mmCORB_READ_POINTER 0x12
+#define mmCORB_CONTROL 0x13
+#define mmCORB_STATUS 0x13
+#define mmCORB_SIZE 0x13
+#define mmRIRB_LOWER_BASE_ADDRESS 0x14
+#define mmRIRB_UPPER_BASE_ADDRESS 0x15
+#define mmRIRB_WRITE_POINTER 0x16
+#define mmRESPONSE_INTERRUPT_COUNT 0x16
+#define mmRIRB_CONTROL 0x17
+#define mmRIRB_STATUS 0x17
+#define mmRIRB_SIZE 0x17
+#define mmIMMEDIATE_COMMAND_OUTPUT_INTERFACE 0x18
+#define mmIMMEDIATE_COMMAND_OUTPUT_INTERFACE_INDEX 0x18
+#define mmIMMEDIATE_COMMAND_OUTPUT_INTERFACE_DATA 0x18
+#define mmIMMEDIATE_RESPONSE_INPUT_INTERFACE 0x19
+#define mmIMMEDIATE_COMMAND_STATUS 0x1a
+#define mmDMA_POSITION_LOWER_BASE_ADDRESS 0x1c
+#define mmDMA_POSITION_UPPER_BASE_ADDRESS 0x1d
+#define mmWALL_CLOCK_COUNTER_ALIAS 0x80c
+#define mmOUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS 0x20
+#define mmOUTPUT_STREAM_DESCRIPTOR_LINK_POSITION_IN_CURRENT_BUFFER 0x21
+#define mmOUTPUT_STREAM_DESCRIPTOR_CYCLIC_BUFFER_LENGTH 0x22
+#define mmOUTPUT_STREAM_DESCRIPTOR_LAST_VALID_INDEX 0x23
+#define mmOUTPUT_STREAM_DESCRIPTOR_FIFO_SIZE 0x24
+#define mmOUTPUT_STREAM_DESCRIPTOR_FORMAT 0x24
+#define mmOUTPUT_STREAM_DESCRIPTOR_BDL_POINTER_LOWER_BASE_ADDRESS 0x26
+#define mmOUTPUT_STREAM_DESCRIPTOR_BDL_POINTER_UPPER_BASE_ADDRESS 0x27
+#define mmOUTPUT_STREAM_DESCRIPTOR_LINK_POSITION_IN_CURRENT_BUFFER_ALIAS 0x821
+#define mmAZENDPOINT_IMMEDIATE_COMMAND_OUTPUT_INTERFACE_INDEX 0x18
+#define mmAZENDPOINT_IMMEDIATE_COMMAND_OUTPUT_INTERFACE_DATA 0x18
+#define ixAZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES 0x2f09
+#define ixAZALIA_F2_CODEC_CONVERTER_PARAMETER_SUPPORTED_SIZE_RATES 0x2f0a
+#define ixAZALIA_F2_CODEC_CONVERTER_PARAMETER_STREAM_FORMATS 0x2f0b
+#define ixAZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT 0x2200
+#define ixAZALIA_F2_CODEC_CONVERTER_CONTROL_CHANNEL_STREAM_ID 0x2706
+#define ixAZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER 0x270d
+#define ixAZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_2 0x270e
+#define ixAZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_3 0x273e
+#define ixAZALIA_F2_CODEC_CONVERTER_STRIPE_CONTROL 0x2724
+#define ixAZALIA_F2_CODEC_CONVERTER_CONTROL_RAMP_RATE 0x2770
+#define ixAZALIA_F2_CODEC_CONVERTER_CONTROL_GTC_EMBEDDING 0x2771
+#define ixAZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES 0x3f09
+#define ixAZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES 0x3f0c
+#define ixAZALIA_F2_CODEC_PIN_PARAMETER_CONNECTION_LIST_LENGTH 0x3f0e
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONNECTION_LIST_ENTRY 0x3702
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_WIDGET_CONTROL 0x3707
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_UNSOLICITED_RESPONSE 0x3708
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_PIN_SENSE 0x3709
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT 0x371c
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_2 0x371d
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_3 0x371e
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_4 0x371f
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_SPEAKER_ALLOCATION 0x3770
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_CHANNEL_ALLOCATION 0x3771
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_DOWN_MIX_INFO 0x3772
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR 0x3776
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR_DATA 0x3776
+#define ixAUDIO_DESCRIPTOR0 0x1
+#define ixAUDIO_DESCRIPTOR1 0x2
+#define ixAUDIO_DESCRIPTOR2 0x3
+#define ixAUDIO_DESCRIPTOR3 0x4
+#define ixAUDIO_DESCRIPTOR4 0x5
+#define ixAUDIO_DESCRIPTOR5 0x6
+#define ixAUDIO_DESCRIPTOR6 0x7
+#define ixAUDIO_DESCRIPTOR7 0x8
+#define ixAUDIO_DESCRIPTOR8 0x9
+#define ixAUDIO_DESCRIPTOR9 0xa
+#define ixAUDIO_DESCRIPTOR10 0xb
+#define ixAUDIO_DESCRIPTOR11 0xc
+#define ixAUDIO_DESCRIPTOR12 0xd
+#define ixAUDIO_DESCRIPTOR13 0xe
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL01_ENABLE 0x3777
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL23_ENABLE 0x3778
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL45_ENABLE 0x3779
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL67_ENABLE 0x377a
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_LIPSYNC 0x377b
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_HBR 0x377c
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_AUDIO_SINK_INFO_INDEX 0x3780
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_AUDIO_SINK_INFO_DATA 0x3781
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_MANUFACTURER_ID 0x0
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_PRODUCT_ID 0x1
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_SINK_DESCRIPTION_LEN 0x2
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_PORTID0 0x3
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_PORTID1 0x4
+#define ixSINK_DESCRIPTION0 0x5
+#define ixSINK_DESCRIPTION1 0x6
+#define ixSINK_DESCRIPTION2 0x7
+#define ixSINK_DESCRIPTION3 0x8
+#define ixSINK_DESCRIPTION4 0x9
+#define ixSINK_DESCRIPTION5 0xa
+#define ixSINK_DESCRIPTION6 0xb
+#define ixSINK_DESCRIPTION7 0xc
+#define ixSINK_DESCRIPTION8 0xd
+#define ixSINK_DESCRIPTION9 0xe
+#define ixSINK_DESCRIPTION10 0xf
+#define ixSINK_DESCRIPTION11 0x10
+#define ixSINK_DESCRIPTION12 0x11
+#define ixSINK_DESCRIPTION13 0x12
+#define ixSINK_DESCRIPTION14 0x13
+#define ixSINK_DESCRIPTION15 0x14
+#define ixSINK_DESCRIPTION16 0x15
+#define ixSINK_DESCRIPTION17 0x16
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL1_ENABLE 0x3785
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL3_ENABLE 0x3786
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL5_ENABLE 0x3787
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL7_ENABLE 0x3788
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL_MODE 0x3789
+#define ixAZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_0 0x378a
+#define ixAZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_1 0x378b
+#define ixAZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_2 0x378c
+#define ixAZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_3 0x378d
+#define ixAZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_4 0x378e
+#define ixAZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_5 0x378f
+#define ixAZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_6 0x3790
+#define ixAZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_7 0x3791
+#define ixAZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_8 0x3792
+#define ixAZALIA_F2_CODEC_PIN_ASSOCIATION_INFO 0x3793
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_DIGITAL_OUTPUT_STATUS 0x3797
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_LPIB_SNAPSHOT_CONTROL 0x3798
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_LPIB 0x3799
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_LPIB_TIMER_SNAPSHOT 0x379a
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_CODING_TYPE 0x379b
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_FORMAT_CHANGED 0x379c
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_WIRELESS_DISPLAY_IDENTIFICATION 0x379d
+#define ixAZALIA_F2_CODEC_PIN_CONTROL_REMOTE_KEEPALIVE 0x379e
+#define mmAZALIA_CONTROLLER_CLOCK_GATING 0x17e4
+#define mmAZALIA_AUDIO_DTO 0x17e5
+#define mmAZALIA_AUDIO_DTO_CONTROL 0x17e6
+#define mmAZALIA_SCLK_CONTROL 0x17e7
+#define mmAZALIA_UNDERFLOW_FILLER_SAMPLE 0x17e8
+#define mmAZALIA_DATA_DMA_CONTROL 0x17e9
+#define mmAZALIA_BDL_DMA_CONTROL 0x17ea
+#define mmAZALIA_RIRB_AND_DP_CONTROL 0x17eb
+#define mmAZALIA_CORB_DMA_CONTROL 0x17ec
+#define mmAZALIA_APPLICATION_POSITION_IN_CYCLIC_BUFFER 0x17f3
+#define mmAZALIA_CYCLIC_BUFFER_SYNC 0x17f4
+#define mmAZALIA_GLOBAL_CAPABILITIES 0x17f5
+#define mmAZALIA_OUTPUT_PAYLOAD_CAPABILITY 0x17f6
+#define mmAZALIA_OUTPUT_STREAM_ARBITER_CONTROL 0x17f7
+#define mmAZALIA_INPUT_PAYLOAD_CAPABILITY 0x17f8
+#define mmAZALIA_CONTROLLER_DEBUG 0x17f9
+#define mmAZALIA_MEM_PWR_CTRL 0x1810
+#define mmAZALIA_MEM_PWR_STATUS 0x1811
+#define mmDCI_PG_DEBUG_CONFIG 0x1812
+#define mmAZALIA_INPUT_CRC0_CONTROL0 0x17fb
+#define mmAZALIA_INPUT_CRC0_CONTROL1 0x17fc
+#define mmAZALIA_INPUT_CRC0_CONTROL2 0x17fd
+#define mmAZALIA_INPUT_CRC0_CONTROL3 0x17fe
+#define mmAZALIA_INPUT_CRC0_RESULT 0x17ff
+#define ixAZALIA_INPUT_CRC0_CHANNEL0 0x0
+#define ixAZALIA_INPUT_CRC0_CHANNEL1 0x1
+#define ixAZALIA_INPUT_CRC0_CHANNEL2 0x2
+#define ixAZALIA_INPUT_CRC0_CHANNEL3 0x3
+#define ixAZALIA_INPUT_CRC0_CHANNEL4 0x4
+#define ixAZALIA_INPUT_CRC0_CHANNEL5 0x5
+#define ixAZALIA_INPUT_CRC0_CHANNEL6 0x6
+#define ixAZALIA_INPUT_CRC0_CHANNEL7 0x7
+#define mmAZALIA_INPUT_CRC1_CONTROL0 0x1800
+#define mmAZALIA_INPUT_CRC1_CONTROL1 0x1801
+#define mmAZALIA_INPUT_CRC1_CONTROL2 0x1802
+#define mmAZALIA_INPUT_CRC1_CONTROL3 0x1803
+#define mmAZALIA_INPUT_CRC1_RESULT 0x1804
+#define ixAZALIA_INPUT_CRC1_CHANNEL0 0x0
+#define ixAZALIA_INPUT_CRC1_CHANNEL1 0x1
+#define ixAZALIA_INPUT_CRC1_CHANNEL2 0x2
+#define ixAZALIA_INPUT_CRC1_CHANNEL3 0x3
+#define ixAZALIA_INPUT_CRC1_CHANNEL4 0x4
+#define ixAZALIA_INPUT_CRC1_CHANNEL5 0x5
+#define ixAZALIA_INPUT_CRC1_CHANNEL6 0x6
+#define ixAZALIA_INPUT_CRC1_CHANNEL7 0x7
+#define mmAZALIA_CRC0_CONTROL0 0x1805
+#define mmAZALIA_CRC0_CONTROL1 0x1806
+#define mmAZALIA_CRC0_CONTROL2 0x1807
+#define mmAZALIA_CRC0_CONTROL3 0x1808
+#define mmAZALIA_CRC0_RESULT 0x1809
+#define ixAZALIA_CRC0_CHANNEL0 0x0
+#define ixAZALIA_CRC0_CHANNEL1 0x1
+#define ixAZALIA_CRC0_CHANNEL2 0x2
+#define ixAZALIA_CRC0_CHANNEL3 0x3
+#define ixAZALIA_CRC0_CHANNEL4 0x4
+#define ixAZALIA_CRC0_CHANNEL5 0x5
+#define ixAZALIA_CRC0_CHANNEL6 0x6
+#define ixAZALIA_CRC0_CHANNEL7 0x7
+#define mmAZALIA_CRC1_CONTROL0 0x180a
+#define mmAZALIA_CRC1_CONTROL1 0x180b
+#define mmAZALIA_CRC1_CONTROL2 0x180c
+#define mmAZALIA_CRC1_CONTROL3 0x180d
+#define mmAZALIA_CRC1_RESULT 0x180e
+#define ixAZALIA_CRC1_CHANNEL0 0x0
+#define ixAZALIA_CRC1_CHANNEL1 0x1
+#define ixAZALIA_CRC1_CHANNEL2 0x2
+#define ixAZALIA_CRC1_CHANNEL3 0x3
+#define ixAZALIA_CRC1_CHANNEL4 0x4
+#define ixAZALIA_CRC1_CHANNEL5 0x5
+#define ixAZALIA_CRC1_CHANNEL6 0x6
+#define ixAZALIA_CRC1_CHANNEL7 0x7
+#define mmAZ_TEST_DEBUG_INDEX 0x181f
+#define mmAZ_TEST_DEBUG_DATA 0x1820
+#define mmAZALIA_STREAM_INDEX 0x1780
+#define mmAZF0STREAM0_AZALIA_STREAM_INDEX 0x1780
+#define mmAZF0STREAM1_AZALIA_STREAM_INDEX 0x1782
+#define mmAZF0STREAM2_AZALIA_STREAM_INDEX 0x1784
+#define mmAZF0STREAM3_AZALIA_STREAM_INDEX 0x1786
+#define mmAZF0STREAM4_AZALIA_STREAM_INDEX 0x1788
+#define mmAZF0STREAM5_AZALIA_STREAM_INDEX 0x178a
+#define mmAZF0STREAM6_AZALIA_STREAM_INDEX 0x178c
+#define mmAZF0STREAM7_AZALIA_STREAM_INDEX 0x178e
+#define mmAZF0STREAM8_AZALIA_STREAM_INDEX 0x59c0
+#define mmAZF0STREAM9_AZALIA_STREAM_INDEX 0x59c2
+#define mmAZF0STREAM10_AZALIA_STREAM_INDEX 0x59c4
+#define mmAZF0STREAM11_AZALIA_STREAM_INDEX 0x59c6
+#define mmAZF0STREAM12_AZALIA_STREAM_INDEX 0x59c8
+#define mmAZF0STREAM13_AZALIA_STREAM_INDEX 0x59ca
+#define mmAZF0STREAM14_AZALIA_STREAM_INDEX 0x59cc
+#define mmAZF0STREAM15_AZALIA_STREAM_INDEX 0x59ce
+#define mmAZALIA_STREAM_DATA 0x1781
+#define mmAZF0STREAM0_AZALIA_STREAM_DATA 0x1781
+#define mmAZF0STREAM1_AZALIA_STREAM_DATA 0x1783
+#define mmAZF0STREAM2_AZALIA_STREAM_DATA 0x1785
+#define mmAZF0STREAM3_AZALIA_STREAM_DATA 0x1787
+#define mmAZF0STREAM4_AZALIA_STREAM_DATA 0x1789
+#define mmAZF0STREAM5_AZALIA_STREAM_DATA 0x178b
+#define mmAZF0STREAM6_AZALIA_STREAM_DATA 0x178d
+#define mmAZF0STREAM7_AZALIA_STREAM_DATA 0x178f
+#define mmAZF0STREAM8_AZALIA_STREAM_DATA 0x59c1
+#define mmAZF0STREAM9_AZALIA_STREAM_DATA 0x59c3
+#define mmAZF0STREAM10_AZALIA_STREAM_DATA 0x59c5
+#define mmAZF0STREAM11_AZALIA_STREAM_DATA 0x59c7
+#define mmAZF0STREAM12_AZALIA_STREAM_DATA 0x59c9
+#define mmAZF0STREAM13_AZALIA_STREAM_DATA 0x59cb
+#define mmAZF0STREAM14_AZALIA_STREAM_DATA 0x59cd
+#define mmAZF0STREAM15_AZALIA_STREAM_DATA 0x59cf
+#define ixAZALIA_FIFO_SIZE_CONTROL 0x0
+#define ixAZALIA_LATENCY_COUNTER_CONTROL 0x1
+#define ixAZALIA_WORSTCASE_LATENCY_COUNT 0x2
+#define ixAZALIA_CUMULATIVE_LATENCY_COUNT 0x3
+#define ixAZALIA_CUMULATIVE_REQUEST_COUNT 0x4
+#define ixAZALIA_STREAM_DEBUG 0x5
+#define mmAZALIA_F0_CODEC_ENDPOINT_INDEX 0x17a8
+#define mmAZF0ENDPOINT0_AZALIA_F0_CODEC_ENDPOINT_INDEX 0x17a8
+#define mmAZF0ENDPOINT1_AZALIA_F0_CODEC_ENDPOINT_INDEX 0x17ac
+#define mmAZF0ENDPOINT2_AZALIA_F0_CODEC_ENDPOINT_INDEX 0x17b0
+#define mmAZF0ENDPOINT3_AZALIA_F0_CODEC_ENDPOINT_INDEX 0x17b4
+#define mmAZF0ENDPOINT4_AZALIA_F0_CODEC_ENDPOINT_INDEX 0x17b8
+#define mmAZF0ENDPOINT5_AZALIA_F0_CODEC_ENDPOINT_INDEX 0x17bc
+#define mmAZF0ENDPOINT6_AZALIA_F0_CODEC_ENDPOINT_INDEX 0x17c0
+#define mmAZF0ENDPOINT7_AZALIA_F0_CODEC_ENDPOINT_INDEX 0x17c4
+#define mmAZALIA_F0_CODEC_ENDPOINT_DATA 0x17a9
+#define mmAZF0ENDPOINT0_AZALIA_F0_CODEC_ENDPOINT_DATA 0x17a9
+#define mmAZF0ENDPOINT1_AZALIA_F0_CODEC_ENDPOINT_DATA 0x17ad
+#define mmAZF0ENDPOINT2_AZALIA_F0_CODEC_ENDPOINT_DATA 0x17b1
+#define mmAZF0ENDPOINT3_AZALIA_F0_CODEC_ENDPOINT_DATA 0x17b5
+#define mmAZF0ENDPOINT4_AZALIA_F0_CODEC_ENDPOINT_DATA 0x17b9
+#define mmAZF0ENDPOINT5_AZALIA_F0_CODEC_ENDPOINT_DATA 0x17bd
+#define mmAZF0ENDPOINT6_AZALIA_F0_CODEC_ENDPOINT_DATA 0x17c1
+#define mmAZF0ENDPOINT7_AZALIA_F0_CODEC_ENDPOINT_DATA 0x17c5
+#define ixAZALIA_F0_CODEC_CONVERTER_PIN_DEBUG 0x0
+#define ixAZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES 0x1
+#define ixAZALIA_F0_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT 0x2
+#define ixAZALIA_F0_CODEC_CONVERTER_CONTROL_CHANNEL_STREAM_ID 0x3
+#define ixAZALIA_F0_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER 0x4
+#define ixAZALIA_F0_CODEC_CONVERTER_PARAMETER_STREAM_FORMATS 0x5
+#define ixAZALIA_F0_CODEC_CONVERTER_PARAMETER_SUPPORTED_SIZE_RATES 0x6
+#define ixAZALIA_F0_CODEC_CONVERTER_STRIPE_CONTROL 0x7
+#define ixAZALIA_F0_CODEC_CONVERTER_CONTROL_RAMP_RATE 0x8
+#define ixAZALIA_F0_CODEC_CONVERTER_CONTROL_GTC_EMBEDDING 0x9
+#define ixAZALIA_F0_CODEC_CONVERTER_CONTROL_GTC_OFFSET_DEBUG 0xa
+#define ixAZALIA_F0_CODEC_CONVERTER_GTC_COUNTER_DELTA 0xc
+#define ixAZALIA_F0_CODEC_CONVERTER_GTC_COUNTER_DELTA_MIN 0xd
+#define ixAZALIA_F0_CODEC_CONVERTER_GTC_COUNTER_DELTA_MAX 0xe
+#define ixAZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES 0x20
+#define ixAZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES 0x21
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_UNSOLICITED_RESPONSE 0x22
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_PIN_SENSE 0x23
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_WIDGET_CONTROL 0x24
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_CHANNEL_SPEAKER 0x25
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR0 0x28
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR1 0x29
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR2 0x2a
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR3 0x2b
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR4 0x2c
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR5 0x2d
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR6 0x2e
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR7 0x2f
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR8 0x30
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR9 0x31
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR10 0x32
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR11 0x33
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR12 0x34
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR13 0x35
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE 0x36
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2 0x57
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_MODE 0x58
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_LIPSYNC 0x37
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_HBR 0x38
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO0 0x3a
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO1 0x3b
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO2 0x3c
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO3 0x3d
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO4 0x3e
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO5 0x3f
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO6 0x40
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO7 0x41
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO8 0x42
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_HOT_PLUG_CONTROL 0x54
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_UNSOLICITED_RESPONSE_FORCE 0x55
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT 0x56
+#define ixAZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_0 0x59
+#define ixAZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_1 0x5a
+#define ixAZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_2 0x5b
+#define ixAZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_3 0x5c
+#define ixAZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_4 0x5d
+#define ixAZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_5 0x5e
+#define ixAZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_6 0x5f
+#define ixAZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_7 0x60
+#define ixAZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_8 0x61
+#define ixAZALIA_F0_CODEC_PIN_ASSOCIATION_INFO 0x62
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_DIGITAL_OUTPUT_STATUS 0x63
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_LPIB_SNAPSHOT_CONTROL 0x64
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_LPIB 0x65
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_LPIB_TIMER_SNAPSHOT 0x66
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_CODING_TYPE 0x67
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_FORMAT_CHANGED 0x68
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_WIRELESS_DISPLAY_IDENTIFICATION 0x69
+#define ixAZALIA_F0_CODEC_PIN_CONTROL_REMOTE_KEEPALIVE 0x6a
+#define ixAZALIA_F0_AUDIO_ENABLE_STATUS 0x6b
+#define ixAZALIA_F0_AUDIO_ENABLED_INT_STATUS 0x6c
+#define ixAZALIA_F0_AUDIO_DISABLED_INT_STATUS 0x6d
+#define ixAZALIA_F0_AUDIO_FORMAT_CHANGED_INT_STATUS 0x6e
+#define mmAZALIA_F0_CODEC_INPUT_ENDPOINT_INDEX 0x59d4
+#define mmAZF0INPUTENDPOINT0_AZALIA_F0_CODEC_INPUT_ENDPOINT_INDEX 0x59d4
+#define mmAZF0INPUTENDPOINT1_AZALIA_F0_CODEC_INPUT_ENDPOINT_INDEX 0x59d8
+#define mmAZF0INPUTENDPOINT2_AZALIA_F0_CODEC_INPUT_ENDPOINT_INDEX 0x59dc
+#define mmAZF0INPUTENDPOINT3_AZALIA_F0_CODEC_INPUT_ENDPOINT_INDEX 0x59e0
+#define mmAZF0INPUTENDPOINT4_AZALIA_F0_CODEC_INPUT_ENDPOINT_INDEX 0x59e4
+#define mmAZF0INPUTENDPOINT5_AZALIA_F0_CODEC_INPUT_ENDPOINT_INDEX 0x59e8
+#define mmAZF0INPUTENDPOINT6_AZALIA_F0_CODEC_INPUT_ENDPOINT_INDEX 0x59ec
+#define mmAZF0INPUTENDPOINT7_AZALIA_F0_CODEC_INPUT_ENDPOINT_INDEX 0x59f0
+#define mmAZALIA_F0_CODEC_INPUT_ENDPOINT_DATA 0x59d5
+#define mmAZF0INPUTENDPOINT0_AZALIA_F0_CODEC_INPUT_ENDPOINT_DATA 0x59d5
+#define mmAZF0INPUTENDPOINT1_AZALIA_F0_CODEC_INPUT_ENDPOINT_DATA 0x59d9
+#define mmAZF0INPUTENDPOINT2_AZALIA_F0_CODEC_INPUT_ENDPOINT_DATA 0x59dd
+#define mmAZF0INPUTENDPOINT3_AZALIA_F0_CODEC_INPUT_ENDPOINT_DATA 0x59e1
+#define mmAZF0INPUTENDPOINT4_AZALIA_F0_CODEC_INPUT_ENDPOINT_DATA 0x59e5
+#define mmAZF0INPUTENDPOINT5_AZALIA_F0_CODEC_INPUT_ENDPOINT_DATA 0x59e9
+#define mmAZF0INPUTENDPOINT6_AZALIA_F0_CODEC_INPUT_ENDPOINT_DATA 0x59ed
+#define mmAZF0INPUTENDPOINT7_AZALIA_F0_CODEC_INPUT_ENDPOINT_DATA 0x59f1
+#define ixAZALIA_F0_CODEC_INPUT_CONVERTER_PIN_DEBUG 0x0
+#define ixAZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES 0x1
+#define ixAZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT 0x2
+#define ixAZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_CHANNEL_STREAM_ID 0x3
+#define ixAZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER 0x4
+#define ixAZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_STREAM_FORMATS 0x5
+#define ixAZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_SUPPORTED_SIZE_RATES 0x6
+#define ixAZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES 0x20
+#define ixAZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES 0x21
+#define ixAZALIA_F0_CODEC_INPUT_PIN_CONTROL_UNSOLICITED_RESPONSE 0x22
+#define ixAZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_INPUT_PIN_SENSE 0x23
+#define ixAZALIA_F0_CODEC_INPUT_PIN_CONTROL_WIDGET_CONTROL 0x24
+#define ixAZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE 0x36
+#define ixAZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2 0x37
+#define ixAZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_HBR 0x38
+#define ixAZALIA_F0_CODEC_INPUT_PIN_CONTROL_CHANNEL_ALLOCATION 0x53
+#define ixAZALIA_F0_CODEC_INPUT_PIN_CONTROL_HOT_PLUG_CONTROL 0x54
+#define ixAZALIA_F0_CODEC_INPUT_PIN_CONTROL_UNSOLICITED_RESPONSE_FORCE 0x55
+#define ixAZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT 0x56
+#define ixAZALIA_F0_CODEC_INPUT_PIN_CONTROL_INPUT_STATUS_CONTROL 0x67
+#define ixAZALIA_F0_CODEC_INPUT_PIN_CONTROL_INFOFRAME 0x68
+#define ixAZALIA_F0_CODEC_INPUT_PIN_CONTROL_LPIB_SNAPSHOT_CONTROL 0x64
+#define ixAZALIA_F0_CODEC_INPUT_PIN_CONTROL_LPIB 0x65
+#define ixAZALIA_F0_CODEC_INPUT_PIN_CONTROL_LPIB_TIMER_SNAPSHOT 0x66
+#define mmAZENDPOINT_IMMEDIATE_COMMAND_INPUT_INTERFACE_INDEX 0x18
+#define mmAZENDPOINT_IMMEDIATE_COMMAND_INPUT_INTERFACE_DATA 0x18
+#define ixAZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES 0x6f09
+#define ixAZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_SUPPORTED_SIZE_RATES 0x6f0a
+#define ixAZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_STREAM_FORMATS 0x6f0b
+#define ixAZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT 0x6200
+#define ixAZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CHANNEL_STREAM_ID 0x6706
+#define ixAZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER 0x670d
+#define ixAZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES 0x7f09
+#define ixAZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES 0x7f0c
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_WIDGET_CONTROL 0x7707
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_UNSOLICITED_RESPONSE 0x7708
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_PIN_SENSE 0x7709
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT 0x771c
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_2 0x771d
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_3 0x771e
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_4 0x771f
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL0_ENABLE 0x7777
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL1_ENABLE 0x7785
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL2_ENABLE 0x7778
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL3_ENABLE 0x7786
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_HBR 0x777c
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL4_ENABLE 0x7779
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL5_ENABLE 0x7787
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL6_ENABLE 0x777a
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL7_ENABLE 0x7788
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_CHANNEL_ALLOCATION 0x7771
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_INPUT_STATUS_CONTROL 0x779b
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_INFOFRAME 0x779c
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_CHANNEL_STATUS_L 0x779d
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_CHANNEL_STATUS_H 0x779e
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_LPIB_SNAPSHOT_CONTROL 0x7798
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_LPIB 0x7799
+#define ixAZALIA_F2_CODEC_INPUT_PIN_CONTROL_LPIB_TIMER_SNAPSHOT 0x779a
+#define mmBLND_CONTROL 0x1b6d
+#define mmBLND0_BLND_CONTROL 0x1b6d
+#define mmBLND1_BLND_CONTROL 0x1d6d
+#define mmBLND2_BLND_CONTROL 0x1f6d
+#define mmBLND3_BLND_CONTROL 0x416d
+#define mmBLND4_BLND_CONTROL 0x436d
+#define mmBLND5_BLND_CONTROL 0x456d
+#define mmBLND_SM_CONTROL2 0x1b6e
+#define mmBLND0_BLND_SM_CONTROL2 0x1b6e
+#define mmBLND1_BLND_SM_CONTROL2 0x1d6e
+#define mmBLND2_BLND_SM_CONTROL2 0x1f6e
+#define mmBLND3_BLND_SM_CONTROL2 0x416e
+#define mmBLND4_BLND_SM_CONTROL2 0x436e
+#define mmBLND5_BLND_SM_CONTROL2 0x456e
+#define mmBLND_CONTROL2 0x1b6f
+#define mmBLND0_BLND_CONTROL2 0x1b6f
+#define mmBLND1_BLND_CONTROL2 0x1d6f
+#define mmBLND2_BLND_CONTROL2 0x1f6f
+#define mmBLND3_BLND_CONTROL2 0x416f
+#define mmBLND4_BLND_CONTROL2 0x436f
+#define mmBLND5_BLND_CONTROL2 0x456f
+#define mmBLND_UPDATE 0x1b70
+#define mmBLND0_BLND_UPDATE 0x1b70
+#define mmBLND1_BLND_UPDATE 0x1d70
+#define mmBLND2_BLND_UPDATE 0x1f70
+#define mmBLND3_BLND_UPDATE 0x4170
+#define mmBLND4_BLND_UPDATE 0x4370
+#define mmBLND5_BLND_UPDATE 0x4570
+#define mmBLND_UNDERFLOW_INTERRUPT 0x1b71
+#define mmBLND0_BLND_UNDERFLOW_INTERRUPT 0x1b71
+#define mmBLND1_BLND_UNDERFLOW_INTERRUPT 0x1d71
+#define mmBLND2_BLND_UNDERFLOW_INTERRUPT 0x1f71
+#define mmBLND3_BLND_UNDERFLOW_INTERRUPT 0x4171
+#define mmBLND4_BLND_UNDERFLOW_INTERRUPT 0x4371
+#define mmBLND5_BLND_UNDERFLOW_INTERRUPT 0x4571
+#define mmBLND_V_UPDATE_LOCK 0x1b73
+#define mmBLND0_BLND_V_UPDATE_LOCK 0x1b73
+#define mmBLND1_BLND_V_UPDATE_LOCK 0x1d73
+#define mmBLND2_BLND_V_UPDATE_LOCK 0x1f73
+#define mmBLND3_BLND_V_UPDATE_LOCK 0x4173
+#define mmBLND4_BLND_V_UPDATE_LOCK 0x4373
+#define mmBLND5_BLND_V_UPDATE_LOCK 0x4573
+#define mmBLND_REG_UPDATE_STATUS 0x1b77
+#define mmBLND0_BLND_REG_UPDATE_STATUS 0x1b77
+#define mmBLND1_BLND_REG_UPDATE_STATUS 0x1d77
+#define mmBLND2_BLND_REG_UPDATE_STATUS 0x1f77
+#define mmBLND3_BLND_REG_UPDATE_STATUS 0x4177
+#define mmBLND4_BLND_REG_UPDATE_STATUS 0x4377
+#define mmBLND5_BLND_REG_UPDATE_STATUS 0x4577
+#define mmBLND_DEBUG 0x1b74
+#define mmBLND0_BLND_DEBUG 0x1b74
+#define mmBLND1_BLND_DEBUG 0x1d74
+#define mmBLND2_BLND_DEBUG 0x1f74
+#define mmBLND3_BLND_DEBUG 0x4174
+#define mmBLND4_BLND_DEBUG 0x4374
+#define mmBLND5_BLND_DEBUG 0x4574
+#define mmBLND_TEST_DEBUG_INDEX 0x1b75
+#define mmBLND0_BLND_TEST_DEBUG_INDEX 0x1b75
+#define mmBLND1_BLND_TEST_DEBUG_INDEX 0x1d75
+#define mmBLND2_BLND_TEST_DEBUG_INDEX 0x1f75
+#define mmBLND3_BLND_TEST_DEBUG_INDEX 0x4175
+#define mmBLND4_BLND_TEST_DEBUG_INDEX 0x4375
+#define mmBLND5_BLND_TEST_DEBUG_INDEX 0x4575
+#define mmBLND_TEST_DEBUG_DATA 0x1b76
+#define mmBLND0_BLND_TEST_DEBUG_DATA 0x1b76
+#define mmBLND1_BLND_TEST_DEBUG_DATA 0x1d76
+#define mmBLND2_BLND_TEST_DEBUG_DATA 0x1f76
+#define mmBLND3_BLND_TEST_DEBUG_DATA 0x4176
+#define mmBLND4_BLND_TEST_DEBUG_DATA 0x4376
+#define mmBLND5_BLND_TEST_DEBUG_DATA 0x4576
+#define mmWB_ENABLE 0x5e18
+#define mmWB_EC_CONFIG 0x5e19
+#define mmCNV_MODE 0x5e1a
+#define mmCNV_WINDOW_START 0x5e1b
+#define mmCNV_WINDOW_SIZE 0x5e1c
+#define mmCNV_UPDATE 0x5e1d
+#define mmCNV_SOURCE_SIZE 0x5e1e
+#define mmCNV_CSC_CONTROL 0x5e1f
+#define mmCNV_CSC_C11_C12 0x5e20
+#define mmCNV_CSC_C13_C14 0x5e21
+#define mmCNV_CSC_C21_C22 0x5e22
+#define mmCNV_CSC_C23_C24 0x5e23
+#define mmCNV_CSC_C31_C32 0x5e24
+#define mmCNV_CSC_C33_C34 0x5e25
+#define mmCNV_CSC_ROUND_OFFSET_R 0x5e26
+#define mmCNV_CSC_ROUND_OFFSET_G 0x5e27
+#define mmCNV_CSC_ROUND_OFFSET_B 0x5e28
+#define mmCNV_CSC_CLAMP_R 0x5e29
+#define mmCNV_CSC_CLAMP_G 0x5e2a
+#define mmCNV_CSC_CLAMP_B 0x5e2b
+#define mmCNV_TEST_CNTL 0x5e2c
+#define mmCNV_TEST_CRC_RED 0x5e2d
+#define mmCNV_TEST_CRC_GREEN 0x5e2e
+#define mmCNV_TEST_CRC_BLUE 0x5e2f
+#define mmWB_DEBUG_CTRL 0x5e30
+#define mmWB_DBG_MODE 0x5e31
+#define mmWB_HW_DEBUG 0x5e32
+#define mmCNV_INPUT_SELECT 0x5e33
+#define mmWB_SOFT_RESET 0x5e36
+#define mmWB_WARM_UP_MODE_CTL1 0x5e37
+#define mmWB_WARM_UP_MODE_CTL2 0x5e38
+#define mmCNV_TEST_DEBUG_INDEX 0x5e34
+#define mmCNV_TEST_DEBUG_DATA 0x5e35
+#define mmDCFE_CLOCK_CONTROL 0x1b00
+#define mmDCFE0_DCFE_CLOCK_CONTROL 0x1b00
+#define mmDCFE1_DCFE_CLOCK_CONTROL 0x1d00
+#define mmDCFE2_DCFE_CLOCK_CONTROL 0x1f00
+#define mmDCFE3_DCFE_CLOCK_CONTROL 0x4100
+#define mmDCFE4_DCFE_CLOCK_CONTROL 0x4300
+#define mmDCFE5_DCFE_CLOCK_CONTROL 0x4500
+#define mmDCFE_SOFT_RESET 0x1b01
+#define mmDCFE0_DCFE_SOFT_RESET 0x1b01
+#define mmDCFE1_DCFE_SOFT_RESET 0x1d01
+#define mmDCFE2_DCFE_SOFT_RESET 0x1f01
+#define mmDCFE3_DCFE_SOFT_RESET 0x4101
+#define mmDCFE4_DCFE_SOFT_RESET 0x4301
+#define mmDCFE5_DCFE_SOFT_RESET 0x4501
+#define mmDCFE_DBG_CONFIG 0x1b02
+#define mmDCFE0_DCFE_DBG_CONFIG 0x1b02
+#define mmDCFE1_DCFE_DBG_CONFIG 0x1d02
+#define mmDCFE2_DCFE_DBG_CONFIG 0x1f02
+#define mmDCFE3_DCFE_DBG_CONFIG 0x4102
+#define mmDCFE4_DCFE_DBG_CONFIG 0x4302
+#define mmDCFE5_DCFE_DBG_CONFIG 0x4502
+#define mmDCFE_MEM_PWR_CTRL 0x1b03
+#define mmDCFE0_DCFE_MEM_PWR_CTRL 0x1b03
+#define mmDCFE1_DCFE_MEM_PWR_CTRL 0x1d03
+#define mmDCFE2_DCFE_MEM_PWR_CTRL 0x1f03
+#define mmDCFE3_DCFE_MEM_PWR_CTRL 0x4103
+#define mmDCFE4_DCFE_MEM_PWR_CTRL 0x4303
+#define mmDCFE5_DCFE_MEM_PWR_CTRL 0x4503
+#define mmDCFE_MEM_PWR_CTRL2 0x1b04
+#define mmDCFE0_DCFE_MEM_PWR_CTRL2 0x1b04
+#define mmDCFE1_DCFE_MEM_PWR_CTRL2 0x1d04
+#define mmDCFE2_DCFE_MEM_PWR_CTRL2 0x1f04
+#define mmDCFE3_DCFE_MEM_PWR_CTRL2 0x4104
+#define mmDCFE4_DCFE_MEM_PWR_CTRL2 0x4304
+#define mmDCFE5_DCFE_MEM_PWR_CTRL2 0x4504
+#define mmDCFE_MEM_PWR_STATUS 0x1b05
+#define mmDCFE0_DCFE_MEM_PWR_STATUS 0x1b05
+#define mmDCFE1_DCFE_MEM_PWR_STATUS 0x1d05
+#define mmDCFE2_DCFE_MEM_PWR_STATUS 0x1f05
+#define mmDCFE3_DCFE_MEM_PWR_STATUS 0x4105
+#define mmDCFE4_DCFE_MEM_PWR_STATUS 0x4305
+#define mmDCFE5_DCFE_MEM_PWR_STATUS 0x4505
+#define mmDCFE_MISC 0x1b06
+#define mmDCFE0_DCFE_MISC 0x1b06
+#define mmDCFE1_DCFE_MISC 0x1d06
+#define mmDCFE2_DCFE_MISC 0x1f06
+#define mmDCFE3_DCFE_MISC 0x4106
+#define mmDCFE4_DCFE_MISC 0x4306
+#define mmDCFE5_DCFE_MISC 0x4506
+#define mmDCFE_FLUSH 0x1b07
+#define mmDCFE0_DCFE_FLUSH 0x1b07
+#define mmDCFE1_DCFE_FLUSH 0x1d07
+#define mmDCFE2_DCFE_FLUSH 0x1f07
+#define mmDCFE3_DCFE_FLUSH 0x4107
+#define mmDCFE4_DCFE_FLUSH 0x4307
+#define mmDCFE5_DCFE_FLUSH 0x4507
+#define mmDCFEV_CLOCK_CONTROL 0x46f4
+#define mmDCFEV0_DCFEV_CLOCK_CONTROL 0x46f4
+#define mmDCFEV1_DCFEV_CLOCK_CONTROL 0x98f4
+#define mmDCFEV_SOFT_RESET 0x46f5
+#define mmDCFEV0_DCFEV_SOFT_RESET 0x46f5
+#define mmDCFEV1_DCFEV_SOFT_RESET 0x98f5
+#define mmDCFEV_DMIFV_CLOCK_CONTROL 0x46f6
+#define mmDCFEV0_DCFEV_DMIFV_CLOCK_CONTROL 0x46f6
+#define mmDCFEV1_DCFEV_DMIFV_CLOCK_CONTROL 0x98f6
+#define mmDCFEV_DBG_CONFIG 0x46f7
+#define mmDCFEV0_DCFEV_DBG_CONFIG 0x46f7
+#define mmDCFEV1_DCFEV_DBG_CONFIG 0x98f7
+#define mmDCFEV_DMIFV_MEM_PWR_CTRL 0x46f8
+#define mmDCFEV0_DCFEV_DMIFV_MEM_PWR_CTRL 0x46f8
+#define mmDCFEV1_DCFEV_DMIFV_MEM_PWR_CTRL 0x98f8
+#define mmDCFEV_DMIFV_MEM_PWR_STATUS 0x46f9
+#define mmDCFEV0_DCFEV_DMIFV_MEM_PWR_STATUS 0x46f9
+#define mmDCFEV1_DCFEV_DMIFV_MEM_PWR_STATUS 0x98f9
+#define mmDCFEV_MEM_PWR_CTRL 0x46fa
+#define mmDCFEV0_DCFEV_MEM_PWR_CTRL 0x46fa
+#define mmDCFEV1_DCFEV_MEM_PWR_CTRL 0x98fa
+#define mmDCFEV_MEM_PWR_CTRL2 0x46fb
+#define mmDCFEV0_DCFEV_MEM_PWR_CTRL2 0x46fb
+#define mmDCFEV1_DCFEV_MEM_PWR_CTRL2 0x98fb
+#define mmDCFEV_MEM_PWR_STATUS 0x46fc
+#define mmDCFEV0_DCFEV_MEM_PWR_STATUS 0x46fc
+#define mmDCFEV1_DCFEV_MEM_PWR_STATUS 0x98fc
+#define mmDCFEV_L_FLUSH 0x46ff
+#define mmDCFEV0_DCFEV_L_FLUSH 0x46ff
+#define mmDCFEV1_DCFEV_L_FLUSH 0x98ff
+#define mmDCFEV_C_FLUSH 0x4700
+#define mmDCFEV0_DCFEV_C_FLUSH 0x4700
+#define mmDCFEV1_DCFEV_C_FLUSH 0x9900
+#define mmDCFEV_DMIFV_DEBUG 0x46fd
+#define mmDCFEV0_DCFEV_DMIFV_DEBUG 0x46fd
+#define mmDCFEV1_DCFEV_DMIFV_DEBUG 0x98fd
+#define mmDCFEV_MISC 0x46fe
+#define mmDCFEV0_DCFEV_MISC 0x46fe
+#define mmDCFEV1_DCFEV_MISC 0x98fe
+#define mmDC_HPD_INT_STATUS 0x1898
+#define mmHPD0_DC_HPD_INT_STATUS 0x1898
+#define mmHPD1_DC_HPD_INT_STATUS 0x18a0
+#define mmHPD2_DC_HPD_INT_STATUS 0x18a8
+#define mmHPD3_DC_HPD_INT_STATUS 0x18b0
+#define mmHPD4_DC_HPD_INT_STATUS 0x18b8
+#define mmHPD5_DC_HPD_INT_STATUS 0x18c0
+#define mmDC_HPD_INT_CONTROL 0x1899
+#define mmHPD0_DC_HPD_INT_CONTROL 0x1899
+#define mmHPD1_DC_HPD_INT_CONTROL 0x18a1
+#define mmHPD2_DC_HPD_INT_CONTROL 0x18a9
+#define mmHPD3_DC_HPD_INT_CONTROL 0x18b1
+#define mmHPD4_DC_HPD_INT_CONTROL 0x18b9
+#define mmHPD5_DC_HPD_INT_CONTROL 0x18c1
+#define mmDC_HPD_CONTROL 0x189a
+#define mmHPD0_DC_HPD_CONTROL 0x189a
+#define mmHPD1_DC_HPD_CONTROL 0x18a2
+#define mmHPD2_DC_HPD_CONTROL 0x18aa
+#define mmHPD3_DC_HPD_CONTROL 0x18b2
+#define mmHPD4_DC_HPD_CONTROL 0x18ba
+#define mmHPD5_DC_HPD_CONTROL 0x18c2
+#define mmDC_HPD_FAST_TRAIN_CNTL 0x189b
+#define mmHPD0_DC_HPD_FAST_TRAIN_CNTL 0x189b
+#define mmHPD1_DC_HPD_FAST_TRAIN_CNTL 0x18a3
+#define mmHPD2_DC_HPD_FAST_TRAIN_CNTL 0x18ab
+#define mmHPD3_DC_HPD_FAST_TRAIN_CNTL 0x18b3
+#define mmHPD4_DC_HPD_FAST_TRAIN_CNTL 0x18bb
+#define mmHPD5_DC_HPD_FAST_TRAIN_CNTL 0x18c3
+#define mmDC_HPD_TOGGLE_FILT_CNTL 0x189c
+#define mmHPD0_DC_HPD_TOGGLE_FILT_CNTL 0x189c
+#define mmHPD1_DC_HPD_TOGGLE_FILT_CNTL 0x18a4
+#define mmHPD2_DC_HPD_TOGGLE_FILT_CNTL 0x18ac
+#define mmHPD3_DC_HPD_TOGGLE_FILT_CNTL 0x18b4
+#define mmHPD4_DC_HPD_TOGGLE_FILT_CNTL 0x18bc
+#define mmHPD5_DC_HPD_TOGGLE_FILT_CNTL 0x18c4
+#define mmDCO_SCRATCH0 0x184e
+#define mmDCO_SCRATCH1 0x184f
+#define mmDCO_SCRATCH2 0x1850
+#define mmDCO_SCRATCH3 0x1851
+#define mmDCO_SCRATCH4 0x1852
+#define mmDCO_SCRATCH5 0x1853
+#define mmDCO_SCRATCH6 0x1854
+#define mmDCO_SCRATCH7 0x1855
+#define mmDCE_VCE_CONTROL 0x1856
+#define mmDISP_INTERRUPT_STATUS 0x1857
+#define mmDISP_INTERRUPT_STATUS_CONTINUE 0x1858
+#define mmDISP_INTERRUPT_STATUS_CONTINUE2 0x1859
+#define mmDISP_INTERRUPT_STATUS_CONTINUE3 0x185a
+#define mmDISP_INTERRUPT_STATUS_CONTINUE4 0x185b
+#define mmDISP_INTERRUPT_STATUS_CONTINUE5 0x185c
+#define mmDISP_INTERRUPT_STATUS_CONTINUE6 0x185d
+#define mmDISP_INTERRUPT_STATUS_CONTINUE7 0x185e
+#define mmDISP_INTERRUPT_STATUS_CONTINUE8 0x185f
+#define mmDISP_INTERRUPT_STATUS_CONTINUE9 0x1860
+#define mmDISP_INTERRUPT_STATUS_CONTINUE10 0x1875
+#define mmDCO_MEM_PWR_STATUS 0x1861
+#define mmDCO_MEM_PWR_STATUS1 0x1874
+#define mmDCO_MEM_PWR_CTRL 0x1862
+#define mmDCO_MEM_PWR_CTRL2 0x1863
+#define mmFMT_MEMORY0_CONTROL 0x1888
+#define mmFMT_MEMORY1_CONTROL 0x1889
+#define mmFMT_MEMORY2_CONTROL 0x188a
+#define mmFMT_MEMORY3_CONTROL 0x188b
+#define mmFMT_MEMORY4_CONTROL 0x188c
+#define mmFMT_MEMORY5_CONTROL 0x188d
+#define mmDCO_CLK_CNTL 0x1864
+#define mmDCO_CLK_CNTL2 0x1876
+#define mmDCO_CLK_CNTL3 0x1877
+#define mmDPDBG_CNTL 0x1866
+#define mmDPDBG_INTERRUPT 0x1867
+#define mmDCO_POWER_MANAGEMENT_CNTL 0x1868
+#define mmDCO_SOFT_RESET 0x1871
+#define mmDIG_SOFT_RESET 0x1872
+#define mmDIG_SOFT_RESET_2 0x186a
+#define mmDCO_STEREOSYNC_SEL 0x186e
+#define mmDCO_HDMI_RXSTATUS_TIMER_CONTROL 0x1883
+#define mmDCO_PSP_INTERRUPT_STATUS 0x1884
+#define mmDCO_PSP_INTERRUPT_CLEAR 0x1885
+#define mmDCO_GENERIC_INTERRUPT_MESSAGE 0x1886
+#define mmDCO_GENERIC_INTERRUPT_CLEAR 0x1887
+#define mmDCO_TEST_DEBUG_INDEX 0x186f
+#define mmDCO_TEST_DEBUG_DATA 0x1870
+#define mmDC_I2C_CONTROL 0x16d4
+#define mmDC_I2C_ARBITRATION 0x16d5
+#define mmDC_I2C_INTERRUPT_CONTROL 0x16d6
+#define mmDC_I2C_SW_STATUS 0x16d7
+#define mmDC_I2C_DDC1_HW_STATUS 0x16d8
+#define mmDC_I2C_DDC2_HW_STATUS 0x16d9
+#define mmDC_I2C_DDC3_HW_STATUS 0x16da
+#define mmDC_I2C_DDC4_HW_STATUS 0x16db
+#define mmDC_I2C_DDC5_HW_STATUS 0x16dc
+#define mmDC_I2C_DDC6_HW_STATUS 0x16dd
+#define mmDC_I2C_DDC1_SPEED 0x16de
+#define mmDC_I2C_DDC1_SETUP 0x16df
+#define mmDC_I2C_DDC2_SPEED 0x16e0
+#define mmDC_I2C_DDC2_SETUP 0x16e1
+#define mmDC_I2C_DDC3_SPEED 0x16e2
+#define mmDC_I2C_DDC3_SETUP 0x16e3
+#define mmDC_I2C_DDC4_SPEED 0x16e4
+#define mmDC_I2C_DDC4_SETUP 0x16e5
+#define mmDC_I2C_DDC5_SPEED 0x16e6
+#define mmDC_I2C_DDC5_SETUP 0x16e7
+#define mmDC_I2C_DDC6_SPEED 0x16e8
+#define mmDC_I2C_DDC6_SETUP 0x16e9
+#define mmDC_I2C_TRANSACTION0 0x16ea
+#define mmDC_I2C_TRANSACTION1 0x16eb
+#define mmDC_I2C_TRANSACTION2 0x16ec
+#define mmDC_I2C_TRANSACTION3 0x16ed
+#define mmDC_I2C_DATA 0x16ee
+#define mmDC_I2C_DDCVGA_HW_STATUS 0x16ef
+#define mmDC_I2C_DDCVGA_SPEED 0x16f0
+#define mmDC_I2C_DDCVGA_SETUP 0x16f1
+#define mmDC_I2C_EDID_DETECT_CTRL 0x16f2
+#define mmDC_I2C_READ_REQUEST_INTERRUPT 0x16f3
+#define mmGENERIC_I2C_CONTROL 0x16f4
+#define mmGENERIC_I2C_INTERRUPT_CONTROL 0x16f5
+#define mmGENERIC_I2C_STATUS 0x16f6
+#define mmGENERIC_I2C_SPEED 0x16f7
+#define mmGENERIC_I2C_SETUP 0x16f8
+#define mmGENERIC_I2C_TRANSACTION 0x16f9
+#define mmGENERIC_I2C_DATA 0x16fa
+#define mmGENERIC_I2C_PIN_SELECTION 0x16fb
+#define mmGENERIC_I2C_PIN_DEBUG 0x16fc
+#define mmBLNDV_CONTROL 0x476d
+#define mmBLNDV0_BLNDV_CONTROL 0x476d
+#define mmBLNDV1_BLNDV_CONTROL 0x996d
+#define mmBLNDV_SM_CONTROL2 0x476e
+#define mmBLNDV0_BLNDV_SM_CONTROL2 0x476e
+#define mmBLNDV1_BLNDV_SM_CONTROL2 0x996e
+#define mmBLNDV_CONTROL2 0x476f
+#define mmBLNDV0_BLNDV_CONTROL2 0x476f
+#define mmBLNDV1_BLNDV_CONTROL2 0x996f
+#define mmBLNDV_UPDATE 0x4770
+#define mmBLNDV0_BLNDV_UPDATE 0x4770
+#define mmBLNDV1_BLNDV_UPDATE 0x9970
+#define mmBLNDV_UNDERFLOW_INTERRUPT 0x4771
+#define mmBLNDV0_BLNDV_UNDERFLOW_INTERRUPT 0x4771
+#define mmBLNDV1_BLNDV_UNDERFLOW_INTERRUPT 0x9971
+#define mmBLNDV_V_UPDATE_LOCK 0x4773
+#define mmBLNDV0_BLNDV_V_UPDATE_LOCK 0x4773
+#define mmBLNDV1_BLNDV_V_UPDATE_LOCK 0x9973
+#define mmBLNDV_REG_UPDATE_STATUS 0x4777
+#define mmBLNDV0_BLNDV_REG_UPDATE_STATUS 0x4777
+#define mmBLNDV1_BLNDV_REG_UPDATE_STATUS 0x9977
+#define mmBLNDV_DEBUG 0x4774
+#define mmBLNDV0_BLNDV_DEBUG 0x4774
+#define mmBLNDV1_BLNDV_DEBUG 0x9974
+#define mmBLNDV_TEST_DEBUG_INDEX 0x4775
+#define mmBLNDV0_BLNDV_TEST_DEBUG_INDEX 0x4775
+#define mmBLNDV1_BLNDV_TEST_DEBUG_INDEX 0x9975
+#define mmBLNDV_TEST_DEBUG_DATA 0x4776
+#define mmBLNDV0_BLNDV_TEST_DEBUG_DATA 0x4776
+#define mmBLNDV1_BLNDV_TEST_DEBUG_DATA 0x9976
+#define mmCRTCV_H_TOTAL 0x4780
+#define mmCRTCV0_CRTCV_H_TOTAL 0x4780
+#define mmCRTCV1_CRTCV_H_TOTAL 0x9980
+#define mmCRTCV_H_BLANK_START_END 0x4781
+#define mmCRTCV0_CRTCV_H_BLANK_START_END 0x4781
+#define mmCRTCV1_CRTCV_H_BLANK_START_END 0x9981
+#define mmCRTCV_H_SYNC_A 0x4782
+#define mmCRTCV0_CRTCV_H_SYNC_A 0x4782
+#define mmCRTCV1_CRTCV_H_SYNC_A 0x9982
+#define mmCRTCV_V_TOTAL 0x4787
+#define mmCRTCV0_CRTCV_V_TOTAL 0x4787
+#define mmCRTCV1_CRTCV_V_TOTAL 0x9987
+#define mmCRTCV_V_BLANK_START_END 0x478d
+#define mmCRTCV0_CRTCV_V_BLANK_START_END 0x478d
+#define mmCRTCV1_CRTCV_V_BLANK_START_END 0x998d
+#define mmCRTCV_V_SYNC_A 0x478e
+#define mmCRTCV0_CRTCV_V_SYNC_A 0x478e
+#define mmCRTCV1_CRTCV_V_SYNC_A 0x998e
+#define mmCRTCV_CONTROL 0x479c
+#define mmCRTCV0_CRTCV_CONTROL 0x479c
+#define mmCRTCV1_CRTCV_CONTROL 0x999c
+#define mmCRTCV_START_LINE_CONTROL 0x47b3
+#define mmCRTCV0_CRTCV_START_LINE_CONTROL 0x47b3
+#define mmCRTCV1_CRTCV_START_LINE_CONTROL 0x99b3
+#define mmCRTCV_OVERSCAN_COLOR 0x47c8
+#define mmCRTCV0_CRTCV_OVERSCAN_COLOR 0x47c8
+#define mmCRTCV1_CRTCV_OVERSCAN_COLOR 0x99c8
+#define mmCRTCV_OVERSCAN_COLOR_EXT 0x47c9
+#define mmCRTCV0_CRTCV_OVERSCAN_COLOR_EXT 0x47c9
+#define mmCRTCV1_CRTCV_OVERSCAN_COLOR_EXT 0x99c9
+#define mmCRTCV_BLACK_COLOR 0x47cc
+#define mmCRTCV0_CRTCV_BLACK_COLOR 0x47cc
+#define mmCRTCV1_CRTCV_BLACK_COLOR 0x99cc
+#define mmCRTCV_BLACK_COLOR_EXT 0x47cd
+#define mmCRTCV0_CRTCV_BLACK_COLOR_EXT 0x47cd
+#define mmCRTCV1_CRTCV_BLACK_COLOR_EXT 0x99cd
+#define mmCRTCV_CRC_CNTL 0x47d4
+#define mmCRTCV0_CRTCV_CRC_CNTL 0x47d4
+#define mmCRTCV1_CRTCV_CRC_CNTL 0x99d4
+#define mmCRTCV_CRC0_WINDOWA_X_CONTROL 0x47d5
+#define mmCRTCV0_CRTCV_CRC0_WINDOWA_X_CONTROL 0x47d5
+#define mmCRTCV1_CRTCV_CRC0_WINDOWA_X_CONTROL 0x99d5
+#define mmCRTCV_CRC0_WINDOWA_Y_CONTROL 0x47d6
+#define mmCRTCV0_CRTCV_CRC0_WINDOWA_Y_CONTROL 0x47d6
+#define mmCRTCV1_CRTCV_CRC0_WINDOWA_Y_CONTROL 0x99d6
+#define mmCRTCV_CRC0_WINDOWB_X_CONTROL 0x47d7
+#define mmCRTCV0_CRTCV_CRC0_WINDOWB_X_CONTROL 0x47d7
+#define mmCRTCV1_CRTCV_CRC0_WINDOWB_X_CONTROL 0x99d7
+#define mmCRTCV_CRC0_WINDOWB_Y_CONTROL 0x47d8
+#define mmCRTCV0_CRTCV_CRC0_WINDOWB_Y_CONTROL 0x47d8
+#define mmCRTCV1_CRTCV_CRC0_WINDOWB_Y_CONTROL 0x99d8
+#define mmCRTCV_CRC0_DATA_RG 0x47d9
+#define mmCRTCV0_CRTCV_CRC0_DATA_RG 0x47d9
+#define mmCRTCV1_CRTCV_CRC0_DATA_RG 0x99d9
+#define mmCRTCV_CRC0_DATA_B 0x47da
+#define mmCRTCV0_CRTCV_CRC0_DATA_B 0x47da
+#define mmCRTCV1_CRTCV_CRC0_DATA_B 0x99da
+#define mmCRTCV_CRC1_WINDOWA_X_CONTROL 0x47db
+#define mmCRTCV0_CRTCV_CRC1_WINDOWA_X_CONTROL 0x47db
+#define mmCRTCV1_CRTCV_CRC1_WINDOWA_X_CONTROL 0x99db
+#define mmCRTCV_CRC1_WINDOWA_Y_CONTROL 0x47dc
+#define mmCRTCV0_CRTCV_CRC1_WINDOWA_Y_CONTROL 0x47dc
+#define mmCRTCV1_CRTCV_CRC1_WINDOWA_Y_CONTROL 0x99dc
+#define mmCRTCV_CRC1_WINDOWB_X_CONTROL 0x47dd
+#define mmCRTCV0_CRTCV_CRC1_WINDOWB_X_CONTROL 0x47dd
+#define mmCRTCV1_CRTCV_CRC1_WINDOWB_X_CONTROL 0x99dd
+#define mmCRTCV_CRC1_WINDOWB_Y_CONTROL 0x47de
+#define mmCRTCV0_CRTCV_CRC1_WINDOWB_Y_CONTROL 0x47de
+#define mmCRTCV1_CRTCV_CRC1_WINDOWB_Y_CONTROL 0x99de
+#define mmCRTCV_CRC1_DATA_RG 0x47df
+#define mmCRTCV0_CRTCV_CRC1_DATA_RG 0x47df
+#define mmCRTCV1_CRTCV_CRC1_DATA_RG 0x99df
+#define mmCRTCV_CRC1_DATA_B 0x47e0
+#define mmCRTCV0_CRTCV_CRC1_DATA_B 0x47e0
+#define mmCRTCV1_CRTCV_CRC1_DATA_B 0x99e0
+#define mmCRTCV_TEST_DEBUG_INDEX 0x47c6
+#define mmCRTCV0_CRTCV_TEST_DEBUG_INDEX 0x47c6
+#define mmCRTCV1_CRTCV_TEST_DEBUG_INDEX 0x99c6
+#define mmCRTCV_TEST_DEBUG_DATA 0x47c7
+#define mmCRTCV0_CRTCV_TEST_DEBUG_DATA 0x47c7
+#define mmCRTCV1_CRTCV_TEST_DEBUG_DATA 0x99c7
+#define mmXDMA_MC_PCIE_CLIENT_CONFIG 0x3e0
+#define mmXDMA_LOCAL_SURFACE_TILING1 0x3e1
+#define mmXDMA_LOCAL_SURFACE_TILING2 0x3e2
+#define mmXDMA_INTERRUPT 0x3e3
+#define mmXDMA_CLOCK_GATING_CNTL 0x3e4
+#define mmXDMA_MEM_POWER_CNTL 0x3e6
+#define mmXDMA_IF_BIF_STATUS 0x3e7
+#define mmXDMA_PERF_MEAS_STATUS 0x3e8
+#define mmXDMA_IF_STATUS 0x3e9
+#define mmXDMA_TEST_DEBUG_INDEX 0x3ea
+#define mmXDMA_TEST_DEBUG_DATA 0x3eb
+#define mmXDMA_RBBMIF_RDWR_CNTL 0x3f8
+#define mmXDMA_PG_CONTROL 0x3f9
+#define mmXDMA_PG_WDATA 0x3fa
+#define mmXDMA_PG_STATUS 0x3fb
+#define mmXDMA_AON_TEST_DEBUG_INDEX 0x3fc
+#define mmXDMA_AON_TEST_DEBUG_DATA 0x3fd
+#define mmXDMA_MSTR_CNTL 0x3ec
+#define mmXDMA_MSTR_STATUS 0x3ed
+#define mmXDMA_MSTR_MEM_CLIENT_CONFIG 0x3ee
+#define mmXDMA_MSTR_LOCAL_SURFACE_BASE_ADDR 0x3ef
+#define mmXDMA_MSTR_LOCAL_SURFACE_BASE_ADDR_HIGH 0x3f0
+#define mmXDMA_MSTR_LOCAL_SURFACE_PITCH 0x3f1
+#define mmXDMA_MSTR_CMD_URGENT_CNTL 0x3f2
+#define mmXDMA_MSTR_MEM_URGENT_CNTL 0x3f3
+#define mmXDMA_MSTR_PCIE_NACK_STATUS 0x3f5
+#define mmXDMA_MSTR_MEM_NACK_STATUS 0x3f6
+#define mmXDMA_MSTR_VSYNC_GSL_CHECK 0x3f7
+#define mmXDMA_MSTR_PIPE_CNTL 0x400
+#define mmXDMA_MSTR_PIPE0_XDMA_MSTR_PIPE_CNTL 0x400
+#define mmXDMA_MSTR_PIPE1_XDMA_MSTR_PIPE_CNTL 0x410
+#define mmXDMA_MSTR_PIPE2_XDMA_MSTR_PIPE_CNTL 0x420
+#define mmXDMA_MSTR_PIPE3_XDMA_MSTR_PIPE_CNTL 0x430
+#define mmXDMA_MSTR_PIPE4_XDMA_MSTR_PIPE_CNTL 0x440
+#define mmXDMA_MSTR_PIPE5_XDMA_MSTR_PIPE_CNTL 0x450
+#define mmXDMA_MSTR_READ_COMMAND 0x401
+#define mmXDMA_MSTR_PIPE0_XDMA_MSTR_READ_COMMAND 0x401
+#define mmXDMA_MSTR_PIPE1_XDMA_MSTR_READ_COMMAND 0x411
+#define mmXDMA_MSTR_PIPE2_XDMA_MSTR_READ_COMMAND 0x421
+#define mmXDMA_MSTR_PIPE3_XDMA_MSTR_READ_COMMAND 0x431
+#define mmXDMA_MSTR_PIPE4_XDMA_MSTR_READ_COMMAND 0x441
+#define mmXDMA_MSTR_PIPE5_XDMA_MSTR_READ_COMMAND 0x451
+#define mmXDMA_MSTR_CHANNEL_DIM 0x402
+#define mmXDMA_MSTR_PIPE0_XDMA_MSTR_CHANNEL_DIM 0x402
+#define mmXDMA_MSTR_PIPE1_XDMA_MSTR_CHANNEL_DIM 0x412
+#define mmXDMA_MSTR_PIPE2_XDMA_MSTR_CHANNEL_DIM 0x422
+#define mmXDMA_MSTR_PIPE3_XDMA_MSTR_CHANNEL_DIM 0x432
+#define mmXDMA_MSTR_PIPE4_XDMA_MSTR_CHANNEL_DIM 0x442
+#define mmXDMA_MSTR_PIPE5_XDMA_MSTR_CHANNEL_DIM 0x452
+#define mmXDMA_MSTR_HEIGHT 0x403
+#define mmXDMA_MSTR_PIPE0_XDMA_MSTR_HEIGHT 0x403
+#define mmXDMA_MSTR_PIPE1_XDMA_MSTR_HEIGHT 0x413
+#define mmXDMA_MSTR_PIPE2_XDMA_MSTR_HEIGHT 0x423
+#define mmXDMA_MSTR_PIPE3_XDMA_MSTR_HEIGHT 0x433
+#define mmXDMA_MSTR_PIPE4_XDMA_MSTR_HEIGHT 0x443
+#define mmXDMA_MSTR_PIPE5_XDMA_MSTR_HEIGHT 0x453
+#define mmXDMA_MSTR_REMOTE_SURFACE_BASE 0x404
+#define mmXDMA_MSTR_PIPE0_XDMA_MSTR_REMOTE_SURFACE_BASE 0x404
+#define mmXDMA_MSTR_PIPE1_XDMA_MSTR_REMOTE_SURFACE_BASE 0x414
+#define mmXDMA_MSTR_PIPE2_XDMA_MSTR_REMOTE_SURFACE_BASE 0x424
+#define mmXDMA_MSTR_PIPE3_XDMA_MSTR_REMOTE_SURFACE_BASE 0x434
+#define mmXDMA_MSTR_PIPE4_XDMA_MSTR_REMOTE_SURFACE_BASE 0x444
+#define mmXDMA_MSTR_PIPE5_XDMA_MSTR_REMOTE_SURFACE_BASE 0x454
+#define mmXDMA_MSTR_REMOTE_SURFACE_BASE_HIGH 0x405
+#define mmXDMA_MSTR_PIPE0_XDMA_MSTR_REMOTE_SURFACE_BASE_HIGH 0x405
+#define mmXDMA_MSTR_PIPE1_XDMA_MSTR_REMOTE_SURFACE_BASE_HIGH 0x415
+#define mmXDMA_MSTR_PIPE2_XDMA_MSTR_REMOTE_SURFACE_BASE_HIGH 0x425
+#define mmXDMA_MSTR_PIPE3_XDMA_MSTR_REMOTE_SURFACE_BASE_HIGH 0x435
+#define mmXDMA_MSTR_PIPE4_XDMA_MSTR_REMOTE_SURFACE_BASE_HIGH 0x445
+#define mmXDMA_MSTR_PIPE5_XDMA_MSTR_REMOTE_SURFACE_BASE_HIGH 0x455
+#define mmXDMA_MSTR_REMOTE_GPU_ADDRESS 0x406
+#define mmXDMA_MSTR_PIPE0_XDMA_MSTR_REMOTE_GPU_ADDRESS 0x406
+#define mmXDMA_MSTR_PIPE1_XDMA_MSTR_REMOTE_GPU_ADDRESS 0x416
+#define mmXDMA_MSTR_PIPE2_XDMA_MSTR_REMOTE_GPU_ADDRESS 0x426
+#define mmXDMA_MSTR_PIPE3_XDMA_MSTR_REMOTE_GPU_ADDRESS 0x436
+#define mmXDMA_MSTR_PIPE4_XDMA_MSTR_REMOTE_GPU_ADDRESS 0x446
+#define mmXDMA_MSTR_PIPE5_XDMA_MSTR_REMOTE_GPU_ADDRESS 0x456
+#define mmXDMA_MSTR_REMOTE_GPU_ADDRESS_HIGH 0x407
+#define mmXDMA_MSTR_PIPE0_XDMA_MSTR_REMOTE_GPU_ADDRESS_HIGH 0x407
+#define mmXDMA_MSTR_PIPE1_XDMA_MSTR_REMOTE_GPU_ADDRESS_HIGH 0x417
+#define mmXDMA_MSTR_PIPE2_XDMA_MSTR_REMOTE_GPU_ADDRESS_HIGH 0x427
+#define mmXDMA_MSTR_PIPE3_XDMA_MSTR_REMOTE_GPU_ADDRESS_HIGH 0x437
+#define mmXDMA_MSTR_PIPE4_XDMA_MSTR_REMOTE_GPU_ADDRESS_HIGH 0x447
+#define mmXDMA_MSTR_PIPE5_XDMA_MSTR_REMOTE_GPU_ADDRESS_HIGH 0x457
+#define mmXDMA_MSTR_CACHE_BASE_ADDR 0x408
+#define mmXDMA_MSTR_PIPE0_XDMA_MSTR_CACHE_BASE_ADDR 0x408
+#define mmXDMA_MSTR_PIPE1_XDMA_MSTR_CACHE_BASE_ADDR 0x418
+#define mmXDMA_MSTR_PIPE2_XDMA_MSTR_CACHE_BASE_ADDR 0x428
+#define mmXDMA_MSTR_PIPE3_XDMA_MSTR_CACHE_BASE_ADDR 0x438
+#define mmXDMA_MSTR_PIPE4_XDMA_MSTR_CACHE_BASE_ADDR 0x448
+#define mmXDMA_MSTR_PIPE5_XDMA_MSTR_CACHE_BASE_ADDR 0x458
+#define mmXDMA_MSTR_CACHE_BASE_ADDR_HIGH 0x409
+#define mmXDMA_MSTR_PIPE0_XDMA_MSTR_CACHE_BASE_ADDR_HIGH 0x409
+#define mmXDMA_MSTR_PIPE1_XDMA_MSTR_CACHE_BASE_ADDR_HIGH 0x419
+#define mmXDMA_MSTR_PIPE2_XDMA_MSTR_CACHE_BASE_ADDR_HIGH 0x429
+#define mmXDMA_MSTR_PIPE3_XDMA_MSTR_CACHE_BASE_ADDR_HIGH 0x439
+#define mmXDMA_MSTR_PIPE4_XDMA_MSTR_CACHE_BASE_ADDR_HIGH 0x449
+#define mmXDMA_MSTR_PIPE5_XDMA_MSTR_CACHE_BASE_ADDR_HIGH 0x459
+#define mmXDMA_MSTR_CACHE 0x40a
+#define mmXDMA_MSTR_PIPE0_XDMA_MSTR_CACHE 0x40a
+#define mmXDMA_MSTR_PIPE1_XDMA_MSTR_CACHE 0x41a
+#define mmXDMA_MSTR_PIPE2_XDMA_MSTR_CACHE 0x42a
+#define mmXDMA_MSTR_PIPE3_XDMA_MSTR_CACHE 0x43a
+#define mmXDMA_MSTR_PIPE4_XDMA_MSTR_CACHE 0x44a
+#define mmXDMA_MSTR_PIPE5_XDMA_MSTR_CACHE 0x45a
+#define mmXDMA_MSTR_CHANNEL_START 0x40b
+#define mmXDMA_MSTR_PIPE0_XDMA_MSTR_CHANNEL_START 0x40b
+#define mmXDMA_MSTR_PIPE1_XDMA_MSTR_CHANNEL_START 0x41b
+#define mmXDMA_MSTR_PIPE2_XDMA_MSTR_CHANNEL_START 0x42b
+#define mmXDMA_MSTR_PIPE3_XDMA_MSTR_CHANNEL_START 0x43b
+#define mmXDMA_MSTR_PIPE4_XDMA_MSTR_CHANNEL_START 0x44b
+#define mmXDMA_MSTR_PIPE5_XDMA_MSTR_CHANNEL_START 0x45b
+#define mmXDMA_MSTR_PERFMEAS_STATUS 0x40e
+#define mmXDMA_MSTR_PIPE0_XDMA_MSTR_PERFMEAS_STATUS 0x40e
+#define mmXDMA_MSTR_PIPE1_XDMA_MSTR_PERFMEAS_STATUS 0x41e
+#define mmXDMA_MSTR_PIPE2_XDMA_MSTR_PERFMEAS_STATUS 0x42e
+#define mmXDMA_MSTR_PIPE3_XDMA_MSTR_PERFMEAS_STATUS 0x43e
+#define mmXDMA_MSTR_PIPE4_XDMA_MSTR_PERFMEAS_STATUS 0x44e
+#define mmXDMA_MSTR_PIPE5_XDMA_MSTR_PERFMEAS_STATUS 0x45e
+#define mmXDMA_MSTR_PERFMEAS_CNTL 0x40f
+#define mmXDMA_MSTR_PIPE0_XDMA_MSTR_PERFMEAS_CNTL 0x40f
+#define mmXDMA_MSTR_PIPE1_XDMA_MSTR_PERFMEAS_CNTL 0x41f
+#define mmXDMA_MSTR_PIPE2_XDMA_MSTR_PERFMEAS_CNTL 0x42f
+#define mmXDMA_MSTR_PIPE3_XDMA_MSTR_PERFMEAS_CNTL 0x43f
+#define mmXDMA_MSTR_PIPE4_XDMA_MSTR_PERFMEAS_CNTL 0x44f
+#define mmXDMA_MSTR_PIPE5_XDMA_MSTR_PERFMEAS_CNTL 0x45f
+#define mmXDMA_SLV_CNTL 0x460
+#define mmXDMA_SLV_MEM_CLIENT_CONFIG 0x461
+#define mmXDMA_SLV_SLS_PITCH 0x462
+#define mmXDMA_SLV_READ_URGENT_CNTL 0x463
+#define mmXDMA_SLV_WRITE_URGENT_CNTL 0x464
+#define mmXDMA_SLV_WB_RATE_CNTL 0x465
+#define mmXDMA_SLV_READ_LATENCY_MINMAX 0x466
+#define mmXDMA_SLV_READ_LATENCY_AVE 0x467
+#define mmXDMA_SLV_PCIE_NACK_STATUS 0x468
+#define mmXDMA_SLV_MEM_NACK_STATUS 0x469
+#define mmXDMA_SLV_RDRET_BUF_STATUS 0x46a
+#define mmXDMA_SLV_READ_LATENCY_TIMER 0x46b
+#define mmXDMA_SLV_FLIP_PENDING 0x46c
+#define mmXDMA_SLV_CHANNEL_CNTL 0x470
+#define mmXDMA_SLV_CHANNEL0_XDMA_SLV_CHANNEL_CNTL 0x470
+#define mmXDMA_SLV_CHANNEL1_XDMA_SLV_CHANNEL_CNTL 0x478
+#define mmXDMA_SLV_CHANNEL2_XDMA_SLV_CHANNEL_CNTL 0x480
+#define mmXDMA_SLV_CHANNEL3_XDMA_SLV_CHANNEL_CNTL 0x488
+#define mmXDMA_SLV_CHANNEL4_XDMA_SLV_CHANNEL_CNTL 0x490
+#define mmXDMA_SLV_CHANNEL5_XDMA_SLV_CHANNEL_CNTL 0x498
+#define mmXDMA_SLV_REMOTE_GPU_ADDRESS 0x471
+#define mmXDMA_SLV_CHANNEL0_XDMA_SLV_REMOTE_GPU_ADDRESS 0x471
+#define mmXDMA_SLV_CHANNEL1_XDMA_SLV_REMOTE_GPU_ADDRESS 0x479
+#define mmXDMA_SLV_CHANNEL2_XDMA_SLV_REMOTE_GPU_ADDRESS 0x481
+#define mmXDMA_SLV_CHANNEL3_XDMA_SLV_REMOTE_GPU_ADDRESS 0x489
+#define mmXDMA_SLV_CHANNEL4_XDMA_SLV_REMOTE_GPU_ADDRESS 0x491
+#define mmXDMA_SLV_CHANNEL5_XDMA_SLV_REMOTE_GPU_ADDRESS 0x499
+#define mmXDMA_SLV_REMOTE_GPU_ADDRESS_HIGH 0x472
+#define mmXDMA_SLV_CHANNEL0_XDMA_SLV_REMOTE_GPU_ADDRESS_HIGH 0x472
+#define mmXDMA_SLV_CHANNEL1_XDMA_SLV_REMOTE_GPU_ADDRESS_HIGH 0x47a
+#define mmXDMA_SLV_CHANNEL2_XDMA_SLV_REMOTE_GPU_ADDRESS_HIGH 0x482
+#define mmXDMA_SLV_CHANNEL3_XDMA_SLV_REMOTE_GPU_ADDRESS_HIGH 0x48a
+#define mmXDMA_SLV_CHANNEL4_XDMA_SLV_REMOTE_GPU_ADDRESS_HIGH 0x492
+#define mmXDMA_SLV_CHANNEL5_XDMA_SLV_REMOTE_GPU_ADDRESS_HIGH 0x49a
+#define mmCMD_BUS_TX_CONTROL_LANE0 0x48e0
+#define mmDC_COMBOPHYTXREGS0_CMD_BUS_TX_CONTROL_LANE0 0x48e0
+#define mmDC_COMBOPHYTXREGS1_CMD_BUS_TX_CONTROL_LANE0 0x4980
+#define mmDC_COMBOPHYTXREGS2_CMD_BUS_TX_CONTROL_LANE0 0x9a20
+#define mmDC_COMBOPHYTXREGS3_CMD_BUS_TX_CONTROL_LANE0 0x9ac0
+#define mmDC_COMBOPHYTXREGS4_CMD_BUS_TX_CONTROL_LANE0 0x9b60
+#define mmDC_COMBOPHYTXREGS5_CMD_BUS_TX_CONTROL_LANE0 0x9c00
+#define mmDC_COMBOPHYTXREGS6_CMD_BUS_TX_CONTROL_LANE0 0x9ca0
+#define mmDC_COMBOPHYTXREGS7_CMD_BUS_TX_CONTROL_LANE0 0x9d40
+#define mmCMD_BUS_TX_CONTROL_LANE1 0x48f0
+#define mmDC_COMBOPHYTXREGS0_CMD_BUS_TX_CONTROL_LANE1 0x48f0
+#define mmDC_COMBOPHYTXREGS1_CMD_BUS_TX_CONTROL_LANE1 0x4990
+#define mmDC_COMBOPHYTXREGS2_CMD_BUS_TX_CONTROL_LANE1 0x9a30
+#define mmDC_COMBOPHYTXREGS3_CMD_BUS_TX_CONTROL_LANE1 0x9ad0
+#define mmDC_COMBOPHYTXREGS4_CMD_BUS_TX_CONTROL_LANE1 0x9b70
+#define mmDC_COMBOPHYTXREGS5_CMD_BUS_TX_CONTROL_LANE1 0x9c10
+#define mmDC_COMBOPHYTXREGS6_CMD_BUS_TX_CONTROL_LANE1 0x9cb0
+#define mmDC_COMBOPHYTXREGS7_CMD_BUS_TX_CONTROL_LANE1 0x9d50
+#define mmCMD_BUS_TX_CONTROL_LANE2 0x4900
+#define mmDC_COMBOPHYTXREGS0_CMD_BUS_TX_CONTROL_LANE2 0x4900
+#define mmDC_COMBOPHYTXREGS1_CMD_BUS_TX_CONTROL_LANE2 0x49a0
+#define mmDC_COMBOPHYTXREGS2_CMD_BUS_TX_CONTROL_LANE2 0x9a40
+#define mmDC_COMBOPHYTXREGS3_CMD_BUS_TX_CONTROL_LANE2 0x9ae0
+#define mmDC_COMBOPHYTXREGS4_CMD_BUS_TX_CONTROL_LANE2 0x9b80
+#define mmDC_COMBOPHYTXREGS5_CMD_BUS_TX_CONTROL_LANE2 0x9c20
+#define mmDC_COMBOPHYTXREGS6_CMD_BUS_TX_CONTROL_LANE2 0x9cc0
+#define mmDC_COMBOPHYTXREGS7_CMD_BUS_TX_CONTROL_LANE2 0x9d60
+#define mmCMD_BUS_TX_CONTROL_LANE3 0x4910
+#define mmDC_COMBOPHYTXREGS0_CMD_BUS_TX_CONTROL_LANE3 0x4910
+#define mmDC_COMBOPHYTXREGS1_CMD_BUS_TX_CONTROL_LANE3 0x49b0
+#define mmDC_COMBOPHYTXREGS2_CMD_BUS_TX_CONTROL_LANE3 0x9a50
+#define mmDC_COMBOPHYTXREGS3_CMD_BUS_TX_CONTROL_LANE3 0x9af0
+#define mmDC_COMBOPHYTXREGS4_CMD_BUS_TX_CONTROL_LANE3 0x9b90
+#define mmDC_COMBOPHYTXREGS5_CMD_BUS_TX_CONTROL_LANE3 0x9c30
+#define mmDC_COMBOPHYTXREGS6_CMD_BUS_TX_CONTROL_LANE3 0x9cd0
+#define mmDC_COMBOPHYTXREGS7_CMD_BUS_TX_CONTROL_LANE3 0x9d70
+#define mmMARGIN_DEEMPH_LANE0 0x48e1
+#define mmDC_COMBOPHYTXREGS0_MARGIN_DEEMPH_LANE0 0x48e1
+#define mmDC_COMBOPHYTXREGS1_MARGIN_DEEMPH_LANE0 0x4981
+#define mmDC_COMBOPHYTXREGS2_MARGIN_DEEMPH_LANE0 0x9a21
+#define mmDC_COMBOPHYTXREGS3_MARGIN_DEEMPH_LANE0 0x9ac1
+#define mmDC_COMBOPHYTXREGS4_MARGIN_DEEMPH_LANE0 0x9b61
+#define mmDC_COMBOPHYTXREGS5_MARGIN_DEEMPH_LANE0 0x9c01
+#define mmDC_COMBOPHYTXREGS6_MARGIN_DEEMPH_LANE0 0x9ca1
+#define mmDC_COMBOPHYTXREGS7_MARGIN_DEEMPH_LANE0 0x9d41
+#define mmMARGIN_DEEMPH_LANE1 0x48f1
+#define mmDC_COMBOPHYTXREGS0_MARGIN_DEEMPH_LANE1 0x48f1
+#define mmDC_COMBOPHYTXREGS1_MARGIN_DEEMPH_LANE1 0x4991
+#define mmDC_COMBOPHYTXREGS2_MARGIN_DEEMPH_LANE1 0x9a31
+#define mmDC_COMBOPHYTXREGS3_MARGIN_DEEMPH_LANE1 0x9ad1
+#define mmDC_COMBOPHYTXREGS4_MARGIN_DEEMPH_LANE1 0x9b71
+#define mmDC_COMBOPHYTXREGS5_MARGIN_DEEMPH_LANE1 0x9c11
+#define mmDC_COMBOPHYTXREGS6_MARGIN_DEEMPH_LANE1 0x9cb1
+#define mmDC_COMBOPHYTXREGS7_MARGIN_DEEMPH_LANE1 0x9d51
+#define mmMARGIN_DEEMPH_LANE2 0x4901
+#define mmDC_COMBOPHYTXREGS0_MARGIN_DEEMPH_LANE2 0x4901
+#define mmDC_COMBOPHYTXREGS1_MARGIN_DEEMPH_LANE2 0x49a1
+#define mmDC_COMBOPHYTXREGS2_MARGIN_DEEMPH_LANE2 0x9a41
+#define mmDC_COMBOPHYTXREGS3_MARGIN_DEEMPH_LANE2 0x9ae1
+#define mmDC_COMBOPHYTXREGS4_MARGIN_DEEMPH_LANE2 0x9b81
+#define mmDC_COMBOPHYTXREGS5_MARGIN_DEEMPH_LANE2 0x9c21
+#define mmDC_COMBOPHYTXREGS6_MARGIN_DEEMPH_LANE2 0x9cc1
+#define mmDC_COMBOPHYTXREGS7_MARGIN_DEEMPH_LANE2 0x9d61
+#define mmMARGIN_DEEMPH_LANE3 0x4911
+#define mmDC_COMBOPHYTXREGS0_MARGIN_DEEMPH_LANE3 0x4911
+#define mmDC_COMBOPHYTXREGS1_MARGIN_DEEMPH_LANE3 0x49b1
+#define mmDC_COMBOPHYTXREGS2_MARGIN_DEEMPH_LANE3 0x9a51
+#define mmDC_COMBOPHYTXREGS3_MARGIN_DEEMPH_LANE3 0x9af1
+#define mmDC_COMBOPHYTXREGS4_MARGIN_DEEMPH_LANE3 0x9b91
+#define mmDC_COMBOPHYTXREGS5_MARGIN_DEEMPH_LANE3 0x9c31
+#define mmDC_COMBOPHYTXREGS6_MARGIN_DEEMPH_LANE3 0x9cd1
+#define mmDC_COMBOPHYTXREGS7_MARGIN_DEEMPH_LANE3 0x9d71
+#define mmCMD_BUS_GLOBAL_FOR_TX_LANE0 0x48e2
+#define mmDC_COMBOPHYTXREGS0_CMD_BUS_GLOBAL_FOR_TX_LANE0 0x48e2
+#define mmDC_COMBOPHYTXREGS1_CMD_BUS_GLOBAL_FOR_TX_LANE0 0x4982
+#define mmDC_COMBOPHYTXREGS2_CMD_BUS_GLOBAL_FOR_TX_LANE0 0x9a22
+#define mmDC_COMBOPHYTXREGS3_CMD_BUS_GLOBAL_FOR_TX_LANE0 0x9ac2
+#define mmDC_COMBOPHYTXREGS4_CMD_BUS_GLOBAL_FOR_TX_LANE0 0x9b62
+#define mmDC_COMBOPHYTXREGS5_CMD_BUS_GLOBAL_FOR_TX_LANE0 0x9c02
+#define mmDC_COMBOPHYTXREGS6_CMD_BUS_GLOBAL_FOR_TX_LANE0 0x9ca2
+#define mmDC_COMBOPHYTXREGS7_CMD_BUS_GLOBAL_FOR_TX_LANE0 0x9d42
+#define mmCMD_BUS_GLOBAL_FOR_TX_LANE1 0x48f2
+#define mmDC_COMBOPHYTXREGS0_CMD_BUS_GLOBAL_FOR_TX_LANE1 0x48f2
+#define mmDC_COMBOPHYTXREGS1_CMD_BUS_GLOBAL_FOR_TX_LANE1 0x4992
+#define mmDC_COMBOPHYTXREGS2_CMD_BUS_GLOBAL_FOR_TX_LANE1 0x9a32
+#define mmDC_COMBOPHYTXREGS3_CMD_BUS_GLOBAL_FOR_TX_LANE1 0x9ad2
+#define mmDC_COMBOPHYTXREGS4_CMD_BUS_GLOBAL_FOR_TX_LANE1 0x9b72
+#define mmDC_COMBOPHYTXREGS5_CMD_BUS_GLOBAL_FOR_TX_LANE1 0x9c12
+#define mmDC_COMBOPHYTXREGS6_CMD_BUS_GLOBAL_FOR_TX_LANE1 0x9cb2
+#define mmDC_COMBOPHYTXREGS7_CMD_BUS_GLOBAL_FOR_TX_LANE1 0x9d52
+#define mmCMD_BUS_GLOBAL_FOR_TX_LANE2 0x4902
+#define mmDC_COMBOPHYTXREGS0_CMD_BUS_GLOBAL_FOR_TX_LANE2 0x4902
+#define mmDC_COMBOPHYTXREGS1_CMD_BUS_GLOBAL_FOR_TX_LANE2 0x49a2
+#define mmDC_COMBOPHYTXREGS2_CMD_BUS_GLOBAL_FOR_TX_LANE2 0x9a42
+#define mmDC_COMBOPHYTXREGS3_CMD_BUS_GLOBAL_FOR_TX_LANE2 0x9ae2
+#define mmDC_COMBOPHYTXREGS4_CMD_BUS_GLOBAL_FOR_TX_LANE2 0x9b82
+#define mmDC_COMBOPHYTXREGS5_CMD_BUS_GLOBAL_FOR_TX_LANE2 0x9c22
+#define mmDC_COMBOPHYTXREGS6_CMD_BUS_GLOBAL_FOR_TX_LANE2 0x9cc2
+#define mmDC_COMBOPHYTXREGS7_CMD_BUS_GLOBAL_FOR_TX_LANE2 0x9d62
+#define mmCMD_BUS_GLOBAL_FOR_TX_LANE3 0x4912
+#define mmDC_COMBOPHYTXREGS0_CMD_BUS_GLOBAL_FOR_TX_LANE3 0x4912
+#define mmDC_COMBOPHYTXREGS1_CMD_BUS_GLOBAL_FOR_TX_LANE3 0x49b2
+#define mmDC_COMBOPHYTXREGS2_CMD_BUS_GLOBAL_FOR_TX_LANE3 0x9a52
+#define mmDC_COMBOPHYTXREGS3_CMD_BUS_GLOBAL_FOR_TX_LANE3 0x9af2
+#define mmDC_COMBOPHYTXREGS4_CMD_BUS_GLOBAL_FOR_TX_LANE3 0x9b92
+#define mmDC_COMBOPHYTXREGS5_CMD_BUS_GLOBAL_FOR_TX_LANE3 0x9c32
+#define mmDC_COMBOPHYTXREGS6_CMD_BUS_GLOBAL_FOR_TX_LANE3 0x9cd2
+#define mmDC_COMBOPHYTXREGS7_CMD_BUS_GLOBAL_FOR_TX_LANE3 0x9d72
+#define mmTX_DISP_RFU0_LANE0 0x48e3
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU0_LANE0 0x48e3
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU0_LANE0 0x4983
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU0_LANE0 0x9a23
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU0_LANE0 0x9ac3
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU0_LANE0 0x9b63
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU0_LANE0 0x9c03
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU0_LANE0 0x9ca3
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU0_LANE0 0x9d43
+#define mmTX_DISP_RFU0_LANE1 0x48f3
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU0_LANE1 0x48f3
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU0_LANE1 0x4993
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU0_LANE1 0x9a33
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU0_LANE1 0x9ad3
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU0_LANE1 0x9b73
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU0_LANE1 0x9c13
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU0_LANE1 0x9cb3
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU0_LANE1 0x9d53
+#define mmTX_DISP_RFU0_LANE2 0x4903
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU0_LANE2 0x4903
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU0_LANE2 0x49a3
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU0_LANE2 0x9a43
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU0_LANE2 0x9ae3
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU0_LANE2 0x9b83
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU0_LANE2 0x9c23
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU0_LANE2 0x9cc3
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU0_LANE2 0x9d63
+#define mmTX_DISP_RFU0_LANE3 0x4913
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU0_LANE3 0x4913
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU0_LANE3 0x49b3
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU0_LANE3 0x9a53
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU0_LANE3 0x9af3
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU0_LANE3 0x9b93
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU0_LANE3 0x9c33
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU0_LANE3 0x9cd3
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU0_LANE3 0x9d73
+#define mmTX_DISP_RFU1_LANE0 0x48e4
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU1_LANE0 0x48e4
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU1_LANE0 0x4984
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU1_LANE0 0x9a24
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU1_LANE0 0x9ac4
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU1_LANE0 0x9b64
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU1_LANE0 0x9c04
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU1_LANE0 0x9ca4
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU1_LANE0 0x9d44
+#define mmTX_DISP_RFU1_LANE1 0x48f4
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU1_LANE1 0x48f4
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU1_LANE1 0x4994
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU1_LANE1 0x9a34
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU1_LANE1 0x9ad4
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU1_LANE1 0x9b74
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU1_LANE1 0x9c14
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU1_LANE1 0x9cb4
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU1_LANE1 0x9d54
+#define mmTX_DISP_RFU1_LANE2 0x4904
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU1_LANE2 0x4904
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU1_LANE2 0x49a4
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU1_LANE2 0x9a44
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU1_LANE2 0x9ae4
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU1_LANE2 0x9b84
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU1_LANE2 0x9c24
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU1_LANE2 0x9cc4
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU1_LANE2 0x9d64
+#define mmTX_DISP_RFU1_LANE3 0x4914
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU1_LANE3 0x4914
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU1_LANE3 0x49b4
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU1_LANE3 0x9a54
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU1_LANE3 0x9af4
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU1_LANE3 0x9b94
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU1_LANE3 0x9c34
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU1_LANE3 0x9cd4
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU1_LANE3 0x9d74
+#define mmTX_DISP_RFU2_LANE0 0x48e5
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU2_LANE0 0x48e5
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU2_LANE0 0x4985
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU2_LANE0 0x9a25
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU2_LANE0 0x9ac5
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU2_LANE0 0x9b65
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU2_LANE0 0x9c05
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU2_LANE0 0x9ca5
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU2_LANE0 0x9d45
+#define mmTX_DISP_RFU2_LANE1 0x48f5
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU2_LANE1 0x48f5
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU2_LANE1 0x4995
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU2_LANE1 0x9a35
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU2_LANE1 0x9ad5
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU2_LANE1 0x9b75
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU2_LANE1 0x9c15
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU2_LANE1 0x9cb5
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU2_LANE1 0x9d55
+#define mmTX_DISP_RFU2_LANE2 0x4905
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU2_LANE2 0x4905
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU2_LANE2 0x49a5
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU2_LANE2 0x9a45
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU2_LANE2 0x9ae5
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU2_LANE2 0x9b85
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU2_LANE2 0x9c25
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU2_LANE2 0x9cc5
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU2_LANE2 0x9d65
+#define mmTX_DISP_RFU2_LANE3 0x4915
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU2_LANE3 0x4915
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU2_LANE3 0x49b5
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU2_LANE3 0x9a55
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU2_LANE3 0x9af5
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU2_LANE3 0x9b95
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU2_LANE3 0x9c35
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU2_LANE3 0x9cd5
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU2_LANE3 0x9d75
+#define mmTX_DISP_RFU3_LANE0 0x48e6
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU3_LANE0 0x48e6
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU3_LANE0 0x4986
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU3_LANE0 0x9a26
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU3_LANE0 0x9ac6
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU3_LANE0 0x9b66
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU3_LANE0 0x9c06
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU3_LANE0 0x9ca6
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU3_LANE0 0x9d46
+#define mmTX_DISP_RFU3_LANE1 0x48f6
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU3_LANE1 0x48f6
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU3_LANE1 0x4996
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU3_LANE1 0x9a36
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU3_LANE1 0x9ad6
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU3_LANE1 0x9b76
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU3_LANE1 0x9c16
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU3_LANE1 0x9cb6
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU3_LANE1 0x9d56
+#define mmTX_DISP_RFU3_LANE2 0x4906
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU3_LANE2 0x4906
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU3_LANE2 0x49a6
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU3_LANE2 0x9a46
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU3_LANE2 0x9ae6
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU3_LANE2 0x9b86
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU3_LANE2 0x9c26
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU3_LANE2 0x9cc6
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU3_LANE2 0x9d66
+#define mmTX_DISP_RFU3_LANE3 0x4916
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU3_LANE3 0x4916
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU3_LANE3 0x49b6
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU3_LANE3 0x9a56
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU3_LANE3 0x9af6
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU3_LANE3 0x9b96
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU3_LANE3 0x9c36
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU3_LANE3 0x9cd6
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU3_LANE3 0x9d76
+#define mmTX_DISP_RFU4_LANE0 0x48e7
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU4_LANE0 0x48e7
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU4_LANE0 0x4987
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU4_LANE0 0x9a27
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU4_LANE0 0x9ac7
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU4_LANE0 0x9b67
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU4_LANE0 0x9c07
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU4_LANE0 0x9ca7
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU4_LANE0 0x9d47
+#define mmTX_DISP_RFU4_LANE1 0x48f7
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU4_LANE1 0x48f7
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU4_LANE1 0x4997
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU4_LANE1 0x9a37
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU4_LANE1 0x9ad7
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU4_LANE1 0x9b77
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU4_LANE1 0x9c17
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU4_LANE1 0x9cb7
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU4_LANE1 0x9d57
+#define mmTX_DISP_RFU4_LANE2 0x4907
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU4_LANE2 0x4907
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU4_LANE2 0x49a7
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU4_LANE2 0x9a47
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU4_LANE2 0x9ae7
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU4_LANE2 0x9b87
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU4_LANE2 0x9c27
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU4_LANE2 0x9cc7
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU4_LANE2 0x9d67
+#define mmTX_DISP_RFU4_LANE3 0x4917
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU4_LANE3 0x4917
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU4_LANE3 0x49b7
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU4_LANE3 0x9a57
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU4_LANE3 0x9af7
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU4_LANE3 0x9b97
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU4_LANE3 0x9c37
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU4_LANE3 0x9cd7
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU4_LANE3 0x9d77
+#define mmTX_DISP_RFU5_LANE0 0x48e8
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU5_LANE0 0x48e8
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU5_LANE0 0x4988
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU5_LANE0 0x9a28
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU5_LANE0 0x9ac8
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU5_LANE0 0x9b68
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU5_LANE0 0x9c08
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU5_LANE0 0x9ca8
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU5_LANE0 0x9d48
+#define mmTX_DISP_RFU5_LANE1 0x48f8
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU5_LANE1 0x48f8
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU5_LANE1 0x4998
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU5_LANE1 0x9a38
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU5_LANE1 0x9ad8
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU5_LANE1 0x9b78
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU5_LANE1 0x9c18
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU5_LANE1 0x9cb8
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU5_LANE1 0x9d58
+#define mmTX_DISP_RFU5_LANE2 0x4908
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU5_LANE2 0x4908
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU5_LANE2 0x49a8
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU5_LANE2 0x9a48
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU5_LANE2 0x9ae8
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU5_LANE2 0x9b88
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU5_LANE2 0x9c28
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU5_LANE2 0x9cc8
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU5_LANE2 0x9d68
+#define mmTX_DISP_RFU5_LANE3 0x4918
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU5_LANE3 0x4918
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU5_LANE3 0x49b8
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU5_LANE3 0x9a58
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU5_LANE3 0x9af8
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU5_LANE3 0x9b98
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU5_LANE3 0x9c38
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU5_LANE3 0x9cd8
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU5_LANE3 0x9d78
+#define mmTX_DISP_RFU6_LANE0 0x48e9
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU6_LANE0 0x48e9
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU6_LANE0 0x4989
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU6_LANE0 0x9a29
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU6_LANE0 0x9ac9
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU6_LANE0 0x9b69
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU6_LANE0 0x9c09
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU6_LANE0 0x9ca9
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU6_LANE0 0x9d49
+#define mmTX_DISP_RFU6_LANE1 0x48f9
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU6_LANE1 0x48f9
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU6_LANE1 0x4999
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU6_LANE1 0x9a39
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU6_LANE1 0x9ad9
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU6_LANE1 0x9b79
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU6_LANE1 0x9c19
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU6_LANE1 0x9cb9
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU6_LANE1 0x9d59
+#define mmTX_DISP_RFU6_LANE2 0x4909
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU6_LANE2 0x4909
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU6_LANE2 0x49a9
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU6_LANE2 0x9a49
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU6_LANE2 0x9ae9
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU6_LANE2 0x9b89
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU6_LANE2 0x9c29
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU6_LANE2 0x9cc9
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU6_LANE2 0x9d69
+#define mmTX_DISP_RFU6_LANE3 0x4919
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU6_LANE3 0x4919
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU6_LANE3 0x49b9
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU6_LANE3 0x9a59
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU6_LANE3 0x9af9
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU6_LANE3 0x9b99
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU6_LANE3 0x9c39
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU6_LANE3 0x9cd9
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU6_LANE3 0x9d79
+#define mmTX_DISP_RFU7_LANE0 0x48ea
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU7_LANE0 0x48ea
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU7_LANE0 0x498a
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU7_LANE0 0x9a2a
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU7_LANE0 0x9aca
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU7_LANE0 0x9b6a
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU7_LANE0 0x9c0a
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU7_LANE0 0x9caa
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU7_LANE0 0x9d4a
+#define mmTX_DISP_RFU7_LANE1 0x48fa
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU7_LANE1 0x48fa
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU7_LANE1 0x499a
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU7_LANE1 0x9a3a
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU7_LANE1 0x9ada
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU7_LANE1 0x9b7a
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU7_LANE1 0x9c1a
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU7_LANE1 0x9cba
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU7_LANE1 0x9d5a
+#define mmTX_DISP_RFU7_LANE2 0x490a
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU7_LANE2 0x490a
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU7_LANE2 0x49aa
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU7_LANE2 0x9a4a
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU7_LANE2 0x9aea
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU7_LANE2 0x9b8a
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU7_LANE2 0x9c2a
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU7_LANE2 0x9cca
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU7_LANE2 0x9d6a
+#define mmTX_DISP_RFU7_LANE3 0x491a
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU7_LANE3 0x491a
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU7_LANE3 0x49ba
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU7_LANE3 0x9a5a
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU7_LANE3 0x9afa
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU7_LANE3 0x9b9a
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU7_LANE3 0x9c3a
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU7_LANE3 0x9cda
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU7_LANE3 0x9d7a
+#define mmTX_DISP_RFU8_LANE0 0x48eb
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU8_LANE0 0x48eb
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU8_LANE0 0x498b
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU8_LANE0 0x9a2b
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU8_LANE0 0x9acb
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU8_LANE0 0x9b6b
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU8_LANE0 0x9c0b
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU8_LANE0 0x9cab
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU8_LANE0 0x9d4b
+#define mmTX_DISP_RFU8_LANE1 0x48fb
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU8_LANE1 0x48fb
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU8_LANE1 0x499b
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU8_LANE1 0x9a3b
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU8_LANE1 0x9adb
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU8_LANE1 0x9b7b
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU8_LANE1 0x9c1b
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU8_LANE1 0x9cbb
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU8_LANE1 0x9d5b
+#define mmTX_DISP_RFU8_LANE2 0x490b
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU8_LANE2 0x490b
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU8_LANE2 0x49ab
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU8_LANE2 0x9a4b
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU8_LANE2 0x9aeb
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU8_LANE2 0x9b8b
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU8_LANE2 0x9c2b
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU8_LANE2 0x9ccb
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU8_LANE2 0x9d6b
+#define mmTX_DISP_RFU8_LANE3 0x491b
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU8_LANE3 0x491b
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU8_LANE3 0x49bb
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU8_LANE3 0x9a5b
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU8_LANE3 0x9afb
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU8_LANE3 0x9b9b
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU8_LANE3 0x9c3b
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU8_LANE3 0x9cdb
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU8_LANE3 0x9d7b
+#define mmTX_DISP_RFU9_LANE0 0x48ec
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU9_LANE0 0x48ec
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU9_LANE0 0x498c
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU9_LANE0 0x9a2c
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU9_LANE0 0x9acc
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU9_LANE0 0x9b6c
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU9_LANE0 0x9c0c
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU9_LANE0 0x9cac
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU9_LANE0 0x9d4c
+#define mmTX_DISP_RFU9_LANE1 0x48fc
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU9_LANE1 0x48fc
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU9_LANE1 0x499c
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU9_LANE1 0x9a3c
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU9_LANE1 0x9adc
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU9_LANE1 0x9b7c
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU9_LANE1 0x9c1c
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU9_LANE1 0x9cbc
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU9_LANE1 0x9d5c
+#define mmTX_DISP_RFU9_LANE2 0x490c
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU9_LANE2 0x490c
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU9_LANE2 0x49ac
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU9_LANE2 0x9a4c
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU9_LANE2 0x9aec
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU9_LANE2 0x9b8c
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU9_LANE2 0x9c2c
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU9_LANE2 0x9ccc
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU9_LANE2 0x9d6c
+#define mmTX_DISP_RFU9_LANE3 0x491c
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU9_LANE3 0x491c
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU9_LANE3 0x49bc
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU9_LANE3 0x9a5c
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU9_LANE3 0x9afc
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU9_LANE3 0x9b9c
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU9_LANE3 0x9c3c
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU9_LANE3 0x9cdc
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU9_LANE3 0x9d7c
+#define mmTX_DISP_RFU10_LANE0 0x48ed
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU10_LANE0 0x48ed
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU10_LANE0 0x498d
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU10_LANE0 0x9a2d
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU10_LANE0 0x9acd
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU10_LANE0 0x9b6d
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU10_LANE0 0x9c0d
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU10_LANE0 0x9cad
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU10_LANE0 0x9d4d
+#define mmTX_DISP_RFU10_LANE1 0x48fd
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU10_LANE1 0x48fd
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU10_LANE1 0x499d
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU10_LANE1 0x9a3d
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU10_LANE1 0x9add
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU10_LANE1 0x9b7d
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU10_LANE1 0x9c1d
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU10_LANE1 0x9cbd
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU10_LANE1 0x9d5d
+#define mmTX_DISP_RFU10_LANE2 0x490d
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU10_LANE2 0x490d
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU10_LANE2 0x49ad
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU10_LANE2 0x9a4d
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU10_LANE2 0x9aed
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU10_LANE2 0x9b8d
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU10_LANE2 0x9c2d
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU10_LANE2 0x9ccd
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU10_LANE2 0x9d6d
+#define mmTX_DISP_RFU10_LANE3 0x491d
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU10_LANE3 0x491d
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU10_LANE3 0x49bd
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU10_LANE3 0x9a5d
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU10_LANE3 0x9afd
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU10_LANE3 0x9b9d
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU10_LANE3 0x9c3d
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU10_LANE3 0x9cdd
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU10_LANE3 0x9d7d
+#define mmTX_DISP_RFU11_LANE0 0x48ee
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU11_LANE0 0x48ee
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU11_LANE0 0x498e
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU11_LANE0 0x9a2e
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU11_LANE0 0x9ace
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU11_LANE0 0x9b6e
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU11_LANE0 0x9c0e
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU11_LANE0 0x9cae
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU11_LANE0 0x9d4e
+#define mmTX_DISP_RFU11_LANE1 0x48fe
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU11_LANE1 0x48fe
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU11_LANE1 0x499e
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU11_LANE1 0x9a3e
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU11_LANE1 0x9ade
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU11_LANE1 0x9b7e
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU11_LANE1 0x9c1e
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU11_LANE1 0x9cbe
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU11_LANE1 0x9d5e
+#define mmTX_DISP_RFU11_LANE2 0x490e
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU11_LANE2 0x490e
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU11_LANE2 0x49ae
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU11_LANE2 0x9a4e
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU11_LANE2 0x9aee
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU11_LANE2 0x9b8e
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU11_LANE2 0x9c2e
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU11_LANE2 0x9cce
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU11_LANE2 0x9d6e
+#define mmTX_DISP_RFU11_LANE3 0x491e
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU11_LANE3 0x491e
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU11_LANE3 0x49be
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU11_LANE3 0x9a5e
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU11_LANE3 0x9afe
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU11_LANE3 0x9b9e
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU11_LANE3 0x9c3e
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU11_LANE3 0x9cde
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU11_LANE3 0x9d7e
+#define mmTX_DISP_RFU12_LANE0 0x48ef
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU12_LANE0 0x48ef
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU12_LANE0 0x498f
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU12_LANE0 0x9a2f
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU12_LANE0 0x9acf
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU12_LANE0 0x9b6f
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU12_LANE0 0x9c0f
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU12_LANE0 0x9caf
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU12_LANE0 0x9d4f
+#define mmTX_DISP_RFU12_LANE1 0x48ff
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU12_LANE1 0x48ff
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU12_LANE1 0x499f
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU12_LANE1 0x9a3f
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU12_LANE1 0x9adf
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU12_LANE1 0x9b7f
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU12_LANE1 0x9c1f
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU12_LANE1 0x9cbf
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU12_LANE1 0x9d5f
+#define mmTX_DISP_RFU12_LANE2 0x490f
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU12_LANE2 0x490f
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU12_LANE2 0x49af
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU12_LANE2 0x9a4f
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU12_LANE2 0x9aef
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU12_LANE2 0x9b8f
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU12_LANE2 0x9c2f
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU12_LANE2 0x9ccf
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU12_LANE2 0x9d6f
+#define mmTX_DISP_RFU12_LANE3 0x491f
+#define mmDC_COMBOPHYTXREGS0_TX_DISP_RFU12_LANE3 0x491f
+#define mmDC_COMBOPHYTXREGS1_TX_DISP_RFU12_LANE3 0x49bf
+#define mmDC_COMBOPHYTXREGS2_TX_DISP_RFU12_LANE3 0x9a5f
+#define mmDC_COMBOPHYTXREGS3_TX_DISP_RFU12_LANE3 0x9aff
+#define mmDC_COMBOPHYTXREGS4_TX_DISP_RFU12_LANE3 0x9b9f
+#define mmDC_COMBOPHYTXREGS5_TX_DISP_RFU12_LANE3 0x9c3f
+#define mmDC_COMBOPHYTXREGS6_TX_DISP_RFU12_LANE3 0x9cdf
+#define mmDC_COMBOPHYTXREGS7_TX_DISP_RFU12_LANE3 0x9d7f
+#define mmCOMMON_MAR_DEEMPH_NOM 0x48c3
+#define mmDC_COMBOPHYCMREGS0_COMMON_MAR_DEEMPH_NOM 0x48c3
+#define mmDC_COMBOPHYCMREGS1_COMMON_MAR_DEEMPH_NOM 0x4963
+#define mmDC_COMBOPHYCMREGS2_COMMON_MAR_DEEMPH_NOM 0x9a03
+#define mmDC_COMBOPHYCMREGS3_COMMON_MAR_DEEMPH_NOM 0x9aa3
+#define mmDC_COMBOPHYCMREGS4_COMMON_MAR_DEEMPH_NOM 0x9b43
+#define mmDC_COMBOPHYCMREGS5_COMMON_MAR_DEEMPH_NOM 0x9be3
+#define mmDC_COMBOPHYCMREGS6_COMMON_MAR_DEEMPH_NOM 0x9c83
+#define mmDC_COMBOPHYCMREGS7_COMMON_MAR_DEEMPH_NOM 0x9d23
+#define mmCOMMON_LANE_PWRMGMT 0x48c4
+#define mmDC_COMBOPHYCMREGS0_COMMON_LANE_PWRMGMT 0x48c4
+#define mmDC_COMBOPHYCMREGS1_COMMON_LANE_PWRMGMT 0x4964
+#define mmDC_COMBOPHYCMREGS2_COMMON_LANE_PWRMGMT 0x9a04
+#define mmDC_COMBOPHYCMREGS3_COMMON_LANE_PWRMGMT 0x9aa4
+#define mmDC_COMBOPHYCMREGS4_COMMON_LANE_PWRMGMT 0x9b44
+#define mmDC_COMBOPHYCMREGS5_COMMON_LANE_PWRMGMT 0x9be4
+#define mmDC_COMBOPHYCMREGS6_COMMON_LANE_PWRMGMT 0x9c84
+#define mmDC_COMBOPHYCMREGS7_COMMON_LANE_PWRMGMT 0x9d24
+#define mmCOMMON_TXCNTRL 0x48c5
+#define mmDC_COMBOPHYCMREGS0_COMMON_TXCNTRL 0x48c5
+#define mmDC_COMBOPHYCMREGS1_COMMON_TXCNTRL 0x4965
+#define mmDC_COMBOPHYCMREGS2_COMMON_TXCNTRL 0x9a05
+#define mmDC_COMBOPHYCMREGS3_COMMON_TXCNTRL 0x9aa5
+#define mmDC_COMBOPHYCMREGS4_COMMON_TXCNTRL 0x9b45
+#define mmDC_COMBOPHYCMREGS5_COMMON_TXCNTRL 0x9be5
+#define mmDC_COMBOPHYCMREGS6_COMMON_TXCNTRL 0x9c85
+#define mmDC_COMBOPHYCMREGS7_COMMON_TXCNTRL 0x9d25
+#define mmCOMMON_TMDP 0x48c6
+#define mmDC_COMBOPHYCMREGS0_COMMON_TMDP 0x48c6
+#define mmDC_COMBOPHYCMREGS1_COMMON_TMDP 0x4966
+#define mmDC_COMBOPHYCMREGS2_COMMON_TMDP 0x9a06
+#define mmDC_COMBOPHYCMREGS3_COMMON_TMDP 0x9aa6
+#define mmDC_COMBOPHYCMREGS4_COMMON_TMDP 0x9b46
+#define mmDC_COMBOPHYCMREGS5_COMMON_TMDP 0x9be6
+#define mmDC_COMBOPHYCMREGS6_COMMON_TMDP 0x9c86
+#define mmDC_COMBOPHYCMREGS7_COMMON_TMDP 0x9d26
+#define mmCOMMON_LANE_RESETS 0x48c7
+#define mmDC_COMBOPHYCMREGS0_COMMON_LANE_RESETS 0x48c7
+#define mmDC_COMBOPHYCMREGS1_COMMON_LANE_RESETS 0x4967
+#define mmDC_COMBOPHYCMREGS2_COMMON_LANE_RESETS 0x9a07
+#define mmDC_COMBOPHYCMREGS3_COMMON_LANE_RESETS 0x9aa7
+#define mmDC_COMBOPHYCMREGS4_COMMON_LANE_RESETS 0x9b47
+#define mmDC_COMBOPHYCMREGS5_COMMON_LANE_RESETS 0x9be7
+#define mmDC_COMBOPHYCMREGS6_COMMON_LANE_RESETS 0x9c87
+#define mmDC_COMBOPHYCMREGS7_COMMON_LANE_RESETS 0x9d27
+#define mmCOMMON_ZCALCODE_CTRL 0x48c8
+#define mmDC_COMBOPHYCMREGS0_COMMON_ZCALCODE_CTRL 0x48c8
+#define mmDC_COMBOPHYCMREGS1_COMMON_ZCALCODE_CTRL 0x4968
+#define mmDC_COMBOPHYCMREGS2_COMMON_ZCALCODE_CTRL 0x9a08
+#define mmDC_COMBOPHYCMREGS3_COMMON_ZCALCODE_CTRL 0x9aa8
+#define mmDC_COMBOPHYCMREGS4_COMMON_ZCALCODE_CTRL 0x9b48
+#define mmDC_COMBOPHYCMREGS5_COMMON_ZCALCODE_CTRL 0x9be8
+#define mmDC_COMBOPHYCMREGS6_COMMON_ZCALCODE_CTRL 0x9c88
+#define mmDC_COMBOPHYCMREGS7_COMMON_ZCALCODE_CTRL 0x9d28
+#define mmCOMMON_DISP_RFU1 0x48c9
+#define mmDC_COMBOPHYCMREGS0_COMMON_DISP_RFU1 0x48c9
+#define mmDC_COMBOPHYCMREGS1_COMMON_DISP_RFU1 0x4969
+#define mmDC_COMBOPHYCMREGS2_COMMON_DISP_RFU1 0x9a09
+#define mmDC_COMBOPHYCMREGS3_COMMON_DISP_RFU1 0x9aa9
+#define mmDC_COMBOPHYCMREGS4_COMMON_DISP_RFU1 0x9b49
+#define mmDC_COMBOPHYCMREGS5_COMMON_DISP_RFU1 0x9be9
+#define mmDC_COMBOPHYCMREGS6_COMMON_DISP_RFU1 0x9c89
+#define mmDC_COMBOPHYCMREGS7_COMMON_DISP_RFU1 0x9d29
+#define mmCOMMON_DISP_RFU2 0x48ca
+#define mmDC_COMBOPHYCMREGS0_COMMON_DISP_RFU2 0x48ca
+#define mmDC_COMBOPHYCMREGS1_COMMON_DISP_RFU2 0x496a
+#define mmDC_COMBOPHYCMREGS2_COMMON_DISP_RFU2 0x9a0a
+#define mmDC_COMBOPHYCMREGS3_COMMON_DISP_RFU2 0x9aaa
+#define mmDC_COMBOPHYCMREGS4_COMMON_DISP_RFU2 0x9b4a
+#define mmDC_COMBOPHYCMREGS5_COMMON_DISP_RFU2 0x9bea
+#define mmDC_COMBOPHYCMREGS6_COMMON_DISP_RFU2 0x9c8a
+#define mmDC_COMBOPHYCMREGS7_COMMON_DISP_RFU2 0x9d2a
+#define mmCOMMON_DISP_RFU3 0x48cb
+#define mmDC_COMBOPHYCMREGS0_COMMON_DISP_RFU3 0x48cb
+#define mmDC_COMBOPHYCMREGS1_COMMON_DISP_RFU3 0x496b
+#define mmDC_COMBOPHYCMREGS2_COMMON_DISP_RFU3 0x9a0b
+#define mmDC_COMBOPHYCMREGS3_COMMON_DISP_RFU3 0x9aab
+#define mmDC_COMBOPHYCMREGS4_COMMON_DISP_RFU3 0x9b4b
+#define mmDC_COMBOPHYCMREGS5_COMMON_DISP_RFU3 0x9beb
+#define mmDC_COMBOPHYCMREGS6_COMMON_DISP_RFU3 0x9c8b
+#define mmDC_COMBOPHYCMREGS7_COMMON_DISP_RFU3 0x9d2b
+#define mmCOMMON_DISP_RFU4 0x48cc
+#define mmDC_COMBOPHYCMREGS0_COMMON_DISP_RFU4 0x48cc
+#define mmDC_COMBOPHYCMREGS1_COMMON_DISP_RFU4 0x496c
+#define mmDC_COMBOPHYCMREGS2_COMMON_DISP_RFU4 0x9a0c
+#define mmDC_COMBOPHYCMREGS3_COMMON_DISP_RFU4 0x9aac
+#define mmDC_COMBOPHYCMREGS4_COMMON_DISP_RFU4 0x9b4c
+#define mmDC_COMBOPHYCMREGS5_COMMON_DISP_RFU4 0x9bec
+#define mmDC_COMBOPHYCMREGS6_COMMON_DISP_RFU4 0x9c8c
+#define mmDC_COMBOPHYCMREGS7_COMMON_DISP_RFU4 0x9d2c
+#define mmCOMMON_DISP_RFU5 0x48cd
+#define mmDC_COMBOPHYCMREGS0_COMMON_DISP_RFU5 0x48cd
+#define mmDC_COMBOPHYCMREGS1_COMMON_DISP_RFU5 0x496d
+#define mmDC_COMBOPHYCMREGS2_COMMON_DISP_RFU5 0x9a0d
+#define mmDC_COMBOPHYCMREGS3_COMMON_DISP_RFU5 0x9aad
+#define mmDC_COMBOPHYCMREGS4_COMMON_DISP_RFU5 0x9b4d
+#define mmDC_COMBOPHYCMREGS5_COMMON_DISP_RFU5 0x9bed
+#define mmDC_COMBOPHYCMREGS6_COMMON_DISP_RFU5 0x9c8d
+#define mmDC_COMBOPHYCMREGS7_COMMON_DISP_RFU5 0x9d2d
+#define mmCOMMON_DISP_RFU6 0x48ce
+#define mmDC_COMBOPHYCMREGS0_COMMON_DISP_RFU6 0x48ce
+#define mmDC_COMBOPHYCMREGS1_COMMON_DISP_RFU6 0x496e
+#define mmDC_COMBOPHYCMREGS2_COMMON_DISP_RFU6 0x9a0e
+#define mmDC_COMBOPHYCMREGS3_COMMON_DISP_RFU6 0x9aae
+#define mmDC_COMBOPHYCMREGS4_COMMON_DISP_RFU6 0x9b4e
+#define mmDC_COMBOPHYCMREGS5_COMMON_DISP_RFU6 0x9bee
+#define mmDC_COMBOPHYCMREGS6_COMMON_DISP_RFU6 0x9c8e
+#define mmDC_COMBOPHYCMREGS7_COMMON_DISP_RFU6 0x9d2e
+#define mmCOMMON_DISP_RFU7 0x48cf
+#define mmDC_COMBOPHYCMREGS0_COMMON_DISP_RFU7 0x48cf
+#define mmDC_COMBOPHYCMREGS1_COMMON_DISP_RFU7 0x496f
+#define mmDC_COMBOPHYCMREGS2_COMMON_DISP_RFU7 0x9a0f
+#define mmDC_COMBOPHYCMREGS3_COMMON_DISP_RFU7 0x9aaf
+#define mmDC_COMBOPHYCMREGS4_COMMON_DISP_RFU7 0x9b4f
+#define mmDC_COMBOPHYCMREGS5_COMMON_DISP_RFU7 0x9bef
+#define mmDC_COMBOPHYCMREGS6_COMMON_DISP_RFU7 0x9c8f
+#define mmDC_COMBOPHYCMREGS7_COMMON_DISP_RFU7 0x9d2f
+#define mmFREQ_CTRL0 0x4920
+#define mmDC_COMBOPHYPLLREGS0_FREQ_CTRL0 0x4920
+#define mmDC_COMBOPHYPLLREGS1_FREQ_CTRL0 0x49c0
+#define mmDC_COMBOPHYPLLREGS2_FREQ_CTRL0 0x9a60
+#define mmDC_COMBOPHYPLLREGS3_FREQ_CTRL0 0x9b00
+#define mmDC_COMBOPHYPLLREGS4_FREQ_CTRL0 0x9ba0
+#define mmDC_COMBOPHYPLLREGS5_FREQ_CTRL0 0x9c40
+#define mmDC_COMBOPHYPLLREGS6_FREQ_CTRL0 0x9ce0
+#define mmDC_COMBOPHYPLLREGS7_FREQ_CTRL0 0x9d80
+#define mmFREQ_CTRL1 0x4921
+#define mmDC_COMBOPHYPLLREGS0_FREQ_CTRL1 0x4921
+#define mmDC_COMBOPHYPLLREGS1_FREQ_CTRL1 0x49c1
+#define mmDC_COMBOPHYPLLREGS2_FREQ_CTRL1 0x9a61
+#define mmDC_COMBOPHYPLLREGS3_FREQ_CTRL1 0x9b01
+#define mmDC_COMBOPHYPLLREGS4_FREQ_CTRL1 0x9ba1
+#define mmDC_COMBOPHYPLLREGS5_FREQ_CTRL1 0x9c41
+#define mmDC_COMBOPHYPLLREGS6_FREQ_CTRL1 0x9ce1
+#define mmDC_COMBOPHYPLLREGS7_FREQ_CTRL1 0x9d81
+#define mmFREQ_CTRL2 0x4922
+#define mmDC_COMBOPHYPLLREGS0_FREQ_CTRL2 0x4922
+#define mmDC_COMBOPHYPLLREGS1_FREQ_CTRL2 0x49c2
+#define mmDC_COMBOPHYPLLREGS2_FREQ_CTRL2 0x9a62
+#define mmDC_COMBOPHYPLLREGS3_FREQ_CTRL2 0x9b02
+#define mmDC_COMBOPHYPLLREGS4_FREQ_CTRL2 0x9ba2
+#define mmDC_COMBOPHYPLLREGS5_FREQ_CTRL2 0x9c42
+#define mmDC_COMBOPHYPLLREGS6_FREQ_CTRL2 0x9ce2
+#define mmDC_COMBOPHYPLLREGS7_FREQ_CTRL2 0x9d82
+#define mmFREQ_CTRL3 0x4923
+#define mmDC_COMBOPHYPLLREGS0_FREQ_CTRL3 0x4923
+#define mmDC_COMBOPHYPLLREGS1_FREQ_CTRL3 0x49c3
+#define mmDC_COMBOPHYPLLREGS2_FREQ_CTRL3 0x9a63
+#define mmDC_COMBOPHYPLLREGS3_FREQ_CTRL3 0x9b03
+#define mmDC_COMBOPHYPLLREGS4_FREQ_CTRL3 0x9ba3
+#define mmDC_COMBOPHYPLLREGS5_FREQ_CTRL3 0x9c43
+#define mmDC_COMBOPHYPLLREGS6_FREQ_CTRL3 0x9ce3
+#define mmDC_COMBOPHYPLLREGS7_FREQ_CTRL3 0x9d83
+#define mmBW_CTRL_COARSE 0x4924
+#define mmDC_COMBOPHYPLLREGS0_BW_CTRL_COARSE 0x4924
+#define mmDC_COMBOPHYPLLREGS1_BW_CTRL_COARSE 0x49c4
+#define mmDC_COMBOPHYPLLREGS2_BW_CTRL_COARSE 0x9a64
+#define mmDC_COMBOPHYPLLREGS3_BW_CTRL_COARSE 0x9b04
+#define mmDC_COMBOPHYPLLREGS4_BW_CTRL_COARSE 0x9ba4
+#define mmDC_COMBOPHYPLLREGS5_BW_CTRL_COARSE 0x9c44
+#define mmDC_COMBOPHYPLLREGS6_BW_CTRL_COARSE 0x9ce4
+#define mmDC_COMBOPHYPLLREGS7_BW_CTRL_COARSE 0x9d84
+#define mmBW_CTRL_FINE 0x4925
+#define mmDC_COMBOPHYPLLREGS0_BW_CTRL_FINE 0x4925
+#define mmDC_COMBOPHYPLLREGS1_BW_CTRL_FINE 0x49c5
+#define mmDC_COMBOPHYPLLREGS2_BW_CTRL_FINE 0x9a65
+#define mmDC_COMBOPHYPLLREGS3_BW_CTRL_FINE 0x9b05
+#define mmDC_COMBOPHYPLLREGS4_BW_CTRL_FINE 0x9ba5
+#define mmDC_COMBOPHYPLLREGS5_BW_CTRL_FINE 0x9c45
+#define mmDC_COMBOPHYPLLREGS6_BW_CTRL_FINE 0x9ce5
+#define mmDC_COMBOPHYPLLREGS7_BW_CTRL_FINE 0x9d85
+#define mmCAL_CTRL 0x4926
+#define mmDC_COMBOPHYPLLREGS0_CAL_CTRL 0x4926
+#define mmDC_COMBOPHYPLLREGS1_CAL_CTRL 0x49c6
+#define mmDC_COMBOPHYPLLREGS2_CAL_CTRL 0x9a66
+#define mmDC_COMBOPHYPLLREGS3_CAL_CTRL 0x9b06
+#define mmDC_COMBOPHYPLLREGS4_CAL_CTRL 0x9ba6
+#define mmDC_COMBOPHYPLLREGS5_CAL_CTRL 0x9c46
+#define mmDC_COMBOPHYPLLREGS6_CAL_CTRL 0x9ce6
+#define mmDC_COMBOPHYPLLREGS7_CAL_CTRL 0x9d86
+#define mmLOOP_CTRL 0x4927
+#define mmDC_COMBOPHYPLLREGS0_LOOP_CTRL 0x4927
+#define mmDC_COMBOPHYPLLREGS1_LOOP_CTRL 0x49c7
+#define mmDC_COMBOPHYPLLREGS2_LOOP_CTRL 0x9a67
+#define mmDC_COMBOPHYPLLREGS3_LOOP_CTRL 0x9b07
+#define mmDC_COMBOPHYPLLREGS4_LOOP_CTRL 0x9ba7
+#define mmDC_COMBOPHYPLLREGS5_LOOP_CTRL 0x9c47
+#define mmDC_COMBOPHYPLLREGS6_LOOP_CTRL 0x9ce7
+#define mmDC_COMBOPHYPLLREGS7_LOOP_CTRL 0x9d87
+#define mmDEBUG0 0x4928
+#define mmDC_COMBOPHYPLLREGS0_DEBUG0 0x4928
+#define mmDC_COMBOPHYPLLREGS1_DEBUG0 0x49c8
+#define mmDC_COMBOPHYPLLREGS2_DEBUG0 0x9a68
+#define mmDC_COMBOPHYPLLREGS3_DEBUG0 0x9b08
+#define mmDC_COMBOPHYPLLREGS4_DEBUG0 0x9ba8
+#define mmDC_COMBOPHYPLLREGS5_DEBUG0 0x9c48
+#define mmDC_COMBOPHYPLLREGS6_DEBUG0 0x9ce8
+#define mmDC_COMBOPHYPLLREGS7_DEBUG0 0x9d88
+#define mmVREG_CFG 0x4929
+#define mmDC_COMBOPHYPLLREGS0_VREG_CFG 0x4929
+#define mmDC_COMBOPHYPLLREGS1_VREG_CFG 0x49c9
+#define mmDC_COMBOPHYPLLREGS2_VREG_CFG 0x9a69
+#define mmDC_COMBOPHYPLLREGS3_VREG_CFG 0x9b09
+#define mmDC_COMBOPHYPLLREGS4_VREG_CFG 0x9ba9
+#define mmDC_COMBOPHYPLLREGS5_VREG_CFG 0x9c49
+#define mmDC_COMBOPHYPLLREGS6_VREG_CFG 0x9ce9
+#define mmDC_COMBOPHYPLLREGS7_VREG_CFG 0x9d89
+#define mmOBSERVE0 0x492a
+#define mmDC_COMBOPHYPLLREGS0_OBSERVE0 0x492a
+#define mmDC_COMBOPHYPLLREGS1_OBSERVE0 0x49ca
+#define mmDC_COMBOPHYPLLREGS2_OBSERVE0 0x9a6a
+#define mmDC_COMBOPHYPLLREGS3_OBSERVE0 0x9b0a
+#define mmDC_COMBOPHYPLLREGS4_OBSERVE0 0x9baa
+#define mmDC_COMBOPHYPLLREGS5_OBSERVE0 0x9c4a
+#define mmDC_COMBOPHYPLLREGS6_OBSERVE0 0x9cea
+#define mmDC_COMBOPHYPLLREGS7_OBSERVE0 0x9d8a
+#define mmOBSERVE1 0x492b
+#define mmDC_COMBOPHYPLLREGS0_OBSERVE1 0x492b
+#define mmDC_COMBOPHYPLLREGS1_OBSERVE1 0x49cb
+#define mmDC_COMBOPHYPLLREGS2_OBSERVE1 0x9a6b
+#define mmDC_COMBOPHYPLLREGS3_OBSERVE1 0x9b0b
+#define mmDC_COMBOPHYPLLREGS4_OBSERVE1 0x9bab
+#define mmDC_COMBOPHYPLLREGS5_OBSERVE1 0x9c4b
+#define mmDC_COMBOPHYPLLREGS6_OBSERVE1 0x9ceb
+#define mmDC_COMBOPHYPLLREGS7_OBSERVE1 0x9d8b
+#define mmDFT_OUT 0x492c
+#define mmDC_COMBOPHYPLLREGS0_DFT_OUT 0x492c
+#define mmDC_COMBOPHYPLLREGS1_DFT_OUT 0x49cc
+#define mmDC_COMBOPHYPLLREGS2_DFT_OUT 0x9a6c
+#define mmDC_COMBOPHYPLLREGS3_DFT_OUT 0x9b0c
+#define mmDC_COMBOPHYPLLREGS4_DFT_OUT 0x9bac
+#define mmDC_COMBOPHYPLLREGS5_DFT_OUT 0x9c4c
+#define mmDC_COMBOPHYPLLREGS6_DFT_OUT 0x9cec
+#define mmDC_COMBOPHYPLLREGS7_DFT_OUT 0x9d8c
+#define mmPLL_WRAP_CNTRL1 0x495e
+#define mmDC_COMBOPHYPLLREGS0_PLL_WRAP_CNTRL1 0x495e
+#define mmDC_COMBOPHYPLLREGS1_PLL_WRAP_CNTRL1 0x49fe
+#define mmDC_COMBOPHYPLLREGS2_PLL_WRAP_CNTRL1 0x9a9e
+#define mmDC_COMBOPHYPLLREGS3_PLL_WRAP_CNTRL1 0x9b3e
+#define mmDC_COMBOPHYPLLREGS4_PLL_WRAP_CNTRL1 0x9bde
+#define mmDC_COMBOPHYPLLREGS5_PLL_WRAP_CNTRL1 0x9c7e
+#define mmDC_COMBOPHYPLLREGS6_PLL_WRAP_CNTRL1 0x9d1e
+#define mmDC_COMBOPHYPLLREGS7_PLL_WRAP_CNTRL1 0x9dbe
+#define mmPLL_WRAP_CNTRL 0x495f
+#define mmDC_COMBOPHYPLLREGS0_PLL_WRAP_CNTRL 0x495f
+#define mmDC_COMBOPHYPLLREGS1_PLL_WRAP_CNTRL 0x49ff
+#define mmDC_COMBOPHYPLLREGS2_PLL_WRAP_CNTRL 0x9a9f
+#define mmDC_COMBOPHYPLLREGS3_PLL_WRAP_CNTRL 0x9b3f
+#define mmDC_COMBOPHYPLLREGS4_PLL_WRAP_CNTRL 0x9bdf
+#define mmDC_COMBOPHYPLLREGS5_PLL_WRAP_CNTRL 0x9c7f
+#define mmDC_COMBOPHYPLLREGS6_PLL_WRAP_CNTRL 0x9d1f
+#define mmDC_COMBOPHYPLLREGS7_PLL_WRAP_CNTRL 0x9dbf
+#define mmPPLL_VREG_CFG 0x1700
+#define mmDC_DISPLAYPLLREGS0_PPLL_VREG_CFG 0x1700
+#define mmDC_DISPLAYPLLREGS1_PPLL_VREG_CFG 0x172a
+#define mmDC_DISPLAYPLLREGS2_PPLL_VREG_CFG 0x1754
+#define mmPPLL_MODE_CNTL 0x1701
+#define mmDC_DISPLAYPLLREGS0_PPLL_MODE_CNTL 0x1701
+#define mmDC_DISPLAYPLLREGS1_PPLL_MODE_CNTL 0x172b
+#define mmDC_DISPLAYPLLREGS2_PPLL_MODE_CNTL 0x1755
+#define mmPPLL_FREQ_CTRL0 0x1702
+#define mmDC_DISPLAYPLLREGS0_PPLL_FREQ_CTRL0 0x1702
+#define mmDC_DISPLAYPLLREGS1_PPLL_FREQ_CTRL0 0x172c
+#define mmDC_DISPLAYPLLREGS2_PPLL_FREQ_CTRL0 0x1756
+#define mmPPLL_FREQ_CTRL1 0x1703
+#define mmDC_DISPLAYPLLREGS0_PPLL_FREQ_CTRL1 0x1703
+#define mmDC_DISPLAYPLLREGS1_PPLL_FREQ_CTRL1 0x172d
+#define mmDC_DISPLAYPLLREGS2_PPLL_FREQ_CTRL1 0x1757
+#define mmPPLL_FREQ_CTRL2 0x1704
+#define mmDC_DISPLAYPLLREGS0_PPLL_FREQ_CTRL2 0x1704
+#define mmDC_DISPLAYPLLREGS1_PPLL_FREQ_CTRL2 0x172e
+#define mmDC_DISPLAYPLLREGS2_PPLL_FREQ_CTRL2 0x1758
+#define mmPPLL_FREQ_CTRL3 0x1705
+#define mmDC_DISPLAYPLLREGS0_PPLL_FREQ_CTRL3 0x1705
+#define mmDC_DISPLAYPLLREGS1_PPLL_FREQ_CTRL3 0x172f
+#define mmDC_DISPLAYPLLREGS2_PPLL_FREQ_CTRL3 0x1759
+#define mmPPLL_BW_CTRL_COARSE 0x1706
+#define mmDC_DISPLAYPLLREGS0_PPLL_BW_CTRL_COARSE 0x1706
+#define mmDC_DISPLAYPLLREGS1_PPLL_BW_CTRL_COARSE 0x1730
+#define mmDC_DISPLAYPLLREGS2_PPLL_BW_CTRL_COARSE 0x175a
+#define mmPPLL_BW_CTRL_FINE 0x1708
+#define mmDC_DISPLAYPLLREGS0_PPLL_BW_CTRL_FINE 0x1708
+#define mmDC_DISPLAYPLLREGS1_PPLL_BW_CTRL_FINE 0x1732
+#define mmDC_DISPLAYPLLREGS2_PPLL_BW_CTRL_FINE 0x175c
+#define mmPPLL_CAL_CTRL 0x1709
+#define mmDC_DISPLAYPLLREGS0_PPLL_CAL_CTRL 0x1709
+#define mmDC_DISPLAYPLLREGS1_PPLL_CAL_CTRL 0x1733
+#define mmDC_DISPLAYPLLREGS2_PPLL_CAL_CTRL 0x175d
+#define mmPPLL_LOOP_CTRL 0x170a
+#define mmDC_DISPLAYPLLREGS0_PPLL_LOOP_CTRL 0x170a
+#define mmDC_DISPLAYPLLREGS1_PPLL_LOOP_CTRL 0x1734
+#define mmDC_DISPLAYPLLREGS2_PPLL_LOOP_CTRL 0x175e
+#define mmPPLL_REFCLK_CNTL 0x1718
+#define mmDC_DISPLAYPLLREGS0_PPLL_REFCLK_CNTL 0x1718
+#define mmDC_DISPLAYPLLREGS1_PPLL_REFCLK_CNTL 0x1742
+#define mmDC_DISPLAYPLLREGS2_PPLL_REFCLK_CNTL 0x176c
+#define mmPPLL_CLKOUT_CNTL 0x1719
+#define mmDC_DISPLAYPLLREGS0_PPLL_CLKOUT_CNTL 0x1719
+#define mmDC_DISPLAYPLLREGS1_PPLL_CLKOUT_CNTL 0x1743
+#define mmDC_DISPLAYPLLREGS2_PPLL_CLKOUT_CNTL 0x176d
+#define mmPPLL_DFT_CNTL 0x171a
+#define mmDC_DISPLAYPLLREGS0_PPLL_DFT_CNTL 0x171a
+#define mmDC_DISPLAYPLLREGS1_PPLL_DFT_CNTL 0x1744
+#define mmDC_DISPLAYPLLREGS2_PPLL_DFT_CNTL 0x176e
+#define mmPPLL_ANALOG_CNTL 0x171b
+#define mmDC_DISPLAYPLLREGS0_PPLL_ANALOG_CNTL 0x171b
+#define mmDC_DISPLAYPLLREGS1_PPLL_ANALOG_CNTL 0x1745
+#define mmDC_DISPLAYPLLREGS2_PPLL_ANALOG_CNTL 0x176f
+#define mmPPLL_POSTDIV 0x171c
+#define mmDC_DISPLAYPLLREGS0_PPLL_POSTDIV 0x171c
+#define mmDC_DISPLAYPLLREGS1_PPLL_POSTDIV 0x1746
+#define mmDC_DISPLAYPLLREGS2_PPLL_POSTDIV 0x1770
+#define mmPPLL_DEBUG0 0x1720
+#define mmDC_DISPLAYPLLREGS0_PPLL_DEBUG0 0x1720
+#define mmDC_DISPLAYPLLREGS1_PPLL_DEBUG0 0x174a
+#define mmDC_DISPLAYPLLREGS2_PPLL_DEBUG0 0x1774
+#define mmPPLL_OBSERVE0 0x1721
+#define mmDC_DISPLAYPLLREGS0_PPLL_OBSERVE0 0x1721
+#define mmDC_DISPLAYPLLREGS1_PPLL_OBSERVE0 0x174b
+#define mmDC_DISPLAYPLLREGS2_PPLL_OBSERVE0 0x1775
+#define mmPPLL_OBSERVE1 0x1722
+#define mmDC_DISPLAYPLLREGS0_PPLL_OBSERVE1 0x1722
+#define mmDC_DISPLAYPLLREGS1_PPLL_OBSERVE1 0x174c
+#define mmDC_DISPLAYPLLREGS2_PPLL_OBSERVE1 0x1776
+#define mmPPLL_UPDATE_CNTL 0x1724
+#define mmDC_DISPLAYPLLREGS0_PPLL_UPDATE_CNTL 0x1724
+#define mmDC_DISPLAYPLLREGS1_PPLL_UPDATE_CNTL 0x174e
+#define mmDC_DISPLAYPLLREGS2_PPLL_UPDATE_CNTL 0x1778
+#define mmPPLL_OBSERVE0_OUT 0x1725
+#define mmDC_DISPLAYPLLREGS0_PPLL_OBSERVE0_OUT 0x1725
+#define mmDC_DISPLAYPLLREGS1_PPLL_OBSERVE0_OUT 0x174f
+#define mmDC_DISPLAYPLLREGS2_PPLL_OBSERVE0_OUT 0x1779
+#define mmPPLL_STATUS_DEBUG1 0x1726
+#define mmDC_DISPLAYPLLREGS0_PPLL_STATUS_DEBUG1 0x1726
+#define mmDC_DISPLAYPLLREGS1_PPLL_STATUS_DEBUG1 0x1750
+#define mmDC_DISPLAYPLLREGS2_PPLL_STATUS_DEBUG1 0x177a
+#define mmPPLL_DEBUG_MUX_CNTL 0x1727
+#define mmDC_DISPLAYPLLREGS0_PPLL_DEBUG_MUX_CNTL 0x1727
+#define mmDC_DISPLAYPLLREGS1_PPLL_DEBUG_MUX_CNTL 0x1751
+#define mmDC_DISPLAYPLLREGS2_PPLL_DEBUG_MUX_CNTL 0x177b
+#define mmPPLL_DIV_UPDATE_DEBUG 0x1728
+#define mmDC_DISPLAYPLLREGS0_PPLL_DIV_UPDATE_DEBUG 0x1728
+#define mmDC_DISPLAYPLLREGS1_PPLL_DIV_UPDATE_DEBUG 0x1752
+#define mmDC_DISPLAYPLLREGS2_PPLL_DIV_UPDATE_DEBUG 0x177c
+#define mmPPLL_STATUS_DEBUG0 0x1729
+#define mmDC_DISPLAYPLLREGS0_PPLL_STATUS_DEBUG0 0x1729
+#define mmDC_DISPLAYPLLREGS1_PPLL_STATUS_DEBUG0 0x1753
+#define mmDC_DISPLAYPLLREGS2_PPLL_STATUS_DEBUG0 0x177d
+#define mmCOMP_EN_CTL 0x9dc0
+#define mmDPCSTX_PHY_CNTL 0x48d0
+#define mmDPCSTX0_DPCSTX_PHY_CNTL 0x48d0
+#define mmDPCSTX1_DPCSTX_PHY_CNTL 0x4970
+#define mmDPCSTX2_DPCSTX_PHY_CNTL 0x9a10
+#define mmDPCSTX3_DPCSTX_PHY_CNTL 0x9ab0
+#define mmDPCSTX4_DPCSTX_PHY_CNTL 0x9b50
+#define mmDPCSTX5_DPCSTX_PHY_CNTL 0x9bf0
+#define mmDPCSTX6_DPCSTX_PHY_CNTL 0x9c90
+#define mmDPCSTX7_DPCSTX_PHY_CNTL 0x9d30
+#define mmDPCSTX_TX_CLOCK_CNTL 0x48d1
+#define mmDPCSTX0_DPCSTX_TX_CLOCK_CNTL 0x48d1
+#define mmDPCSTX1_DPCSTX_TX_CLOCK_CNTL 0x4971
+#define mmDPCSTX2_DPCSTX_TX_CLOCK_CNTL 0x9a11
+#define mmDPCSTX3_DPCSTX_TX_CLOCK_CNTL 0x9ab1
+#define mmDPCSTX4_DPCSTX_TX_CLOCK_CNTL 0x9b51
+#define mmDPCSTX5_DPCSTX_TX_CLOCK_CNTL 0x9bf1
+#define mmDPCSTX6_DPCSTX_TX_CLOCK_CNTL 0x9c91
+#define mmDPCSTX7_DPCSTX_TX_CLOCK_CNTL 0x9d31
+#define mmDPCSTX_TX_CNTL 0x48d3
+#define mmDPCSTX0_DPCSTX_TX_CNTL 0x48d3
+#define mmDPCSTX1_DPCSTX_TX_CNTL 0x4973
+#define mmDPCSTX2_DPCSTX_TX_CNTL 0x9a13
+#define mmDPCSTX3_DPCSTX_TX_CNTL 0x9ab3
+#define mmDPCSTX4_DPCSTX_TX_CNTL 0x9b53
+#define mmDPCSTX5_DPCSTX_TX_CNTL 0x9bf3
+#define mmDPCSTX6_DPCSTX_TX_CNTL 0x9c93
+#define mmDPCSTX7_DPCSTX_TX_CNTL 0x9d33
+#define mmDPCSTX_CBUS_CNTL 0x48d5
+#define mmDPCSTX0_DPCSTX_CBUS_CNTL 0x48d5
+#define mmDPCSTX1_DPCSTX_CBUS_CNTL 0x4975
+#define mmDPCSTX2_DPCSTX_CBUS_CNTL 0x9a15
+#define mmDPCSTX3_DPCSTX_CBUS_CNTL 0x9ab5
+#define mmDPCSTX4_DPCSTX_CBUS_CNTL 0x9b55
+#define mmDPCSTX5_DPCSTX_CBUS_CNTL 0x9bf5
+#define mmDPCSTX6_DPCSTX_CBUS_CNTL 0x9c95
+#define mmDPCSTX7_DPCSTX_CBUS_CNTL 0x9d35
+#define mmDPCSTX_REG_ERROR_STATUS 0x48d6
+#define mmDPCSTX0_DPCSTX_REG_ERROR_STATUS 0x48d6
+#define mmDPCSTX1_DPCSTX_REG_ERROR_STATUS 0x4976
+#define mmDPCSTX2_DPCSTX_REG_ERROR_STATUS 0x9a16
+#define mmDPCSTX3_DPCSTX_REG_ERROR_STATUS 0x9ab6
+#define mmDPCSTX4_DPCSTX_REG_ERROR_STATUS 0x9b56
+#define mmDPCSTX5_DPCSTX_REG_ERROR_STATUS 0x9bf6
+#define mmDPCSTX6_DPCSTX_REG_ERROR_STATUS 0x9c96
+#define mmDPCSTX7_DPCSTX_REG_ERROR_STATUS 0x9d36
+#define mmDPCSTX_TX_ERROR_STATUS 0x48d7
+#define mmDPCSTX0_DPCSTX_TX_ERROR_STATUS 0x48d7
+#define mmDPCSTX1_DPCSTX_TX_ERROR_STATUS 0x4977
+#define mmDPCSTX2_DPCSTX_TX_ERROR_STATUS 0x9a17
+#define mmDPCSTX3_DPCSTX_TX_ERROR_STATUS 0x9ab7
+#define mmDPCSTX4_DPCSTX_TX_ERROR_STATUS 0x9b57
+#define mmDPCSTX5_DPCSTX_TX_ERROR_STATUS 0x9bf7
+#define mmDPCSTX6_DPCSTX_TX_ERROR_STATUS 0x9c97
+#define mmDPCSTX7_DPCSTX_TX_ERROR_STATUS 0x9d37
+#define mmDPCSTX_PLL_UPDATE_ADDR 0x48d8
+#define mmDPCSTX0_DPCSTX_PLL_UPDATE_ADDR 0x48d8
+#define mmDPCSTX1_DPCSTX_PLL_UPDATE_ADDR 0x4978
+#define mmDPCSTX2_DPCSTX_PLL_UPDATE_ADDR 0x9a18
+#define mmDPCSTX3_DPCSTX_PLL_UPDATE_ADDR 0x9ab8
+#define mmDPCSTX4_DPCSTX_PLL_UPDATE_ADDR 0x9b58
+#define mmDPCSTX5_DPCSTX_PLL_UPDATE_ADDR 0x9bf8
+#define mmDPCSTX6_DPCSTX_PLL_UPDATE_ADDR 0x9c98
+#define mmDPCSTX7_DPCSTX_PLL_UPDATE_ADDR 0x9d38
+#define mmDPCSTX_PLL_UPDATE_DATA 0x48d9
+#define mmDPCSTX0_DPCSTX_PLL_UPDATE_DATA 0x48d9
+#define mmDPCSTX1_DPCSTX_PLL_UPDATE_DATA 0x4979
+#define mmDPCSTX2_DPCSTX_PLL_UPDATE_DATA 0x9a19
+#define mmDPCSTX3_DPCSTX_PLL_UPDATE_DATA 0x9ab9
+#define mmDPCSTX4_DPCSTX_PLL_UPDATE_DATA 0x9b59
+#define mmDPCSTX5_DPCSTX_PLL_UPDATE_DATA 0x9bf9
+#define mmDPCSTX6_DPCSTX_PLL_UPDATE_DATA 0x9c99
+#define mmDPCSTX7_DPCSTX_PLL_UPDATE_DATA 0x9d39
+#define mmDPCSTX_INDEX_MODE_ADDR 0x48da
+#define mmDPCSTX0_DPCSTX_INDEX_MODE_ADDR 0x48da
+#define mmDPCSTX1_DPCSTX_INDEX_MODE_ADDR 0x497a
+#define mmDPCSTX2_DPCSTX_INDEX_MODE_ADDR 0x9a1a
+#define mmDPCSTX3_DPCSTX_INDEX_MODE_ADDR 0x9aba
+#define mmDPCSTX4_DPCSTX_INDEX_MODE_ADDR 0x9b5a
+#define mmDPCSTX5_DPCSTX_INDEX_MODE_ADDR 0x9bfa
+#define mmDPCSTX6_DPCSTX_INDEX_MODE_ADDR 0x9c9a
+#define mmDPCSTX7_DPCSTX_INDEX_MODE_ADDR 0x9d3a
+#define mmDPCSTX_INDEX_MODE_DATA 0x48db
+#define mmDPCSTX0_DPCSTX_INDEX_MODE_DATA 0x48db
+#define mmDPCSTX1_DPCSTX_INDEX_MODE_DATA 0x497b
+#define mmDPCSTX2_DPCSTX_INDEX_MODE_DATA 0x9a1b
+#define mmDPCSTX3_DPCSTX_INDEX_MODE_DATA 0x9abb
+#define mmDPCSTX4_DPCSTX_INDEX_MODE_DATA 0x9b5b
+#define mmDPCSTX5_DPCSTX_INDEX_MODE_DATA 0x9bfb
+#define mmDPCSTX6_DPCSTX_INDEX_MODE_DATA 0x9c9b
+#define mmDPCSTX7_DPCSTX_INDEX_MODE_DATA 0x9d3b
+#define mmDPCSTX_DEBUG_CONFIG 0x48dc
+#define mmDPCSTX0_DPCSTX_DEBUG_CONFIG 0x48dc
+#define mmDPCSTX1_DPCSTX_DEBUG_CONFIG 0x497c
+#define mmDPCSTX2_DPCSTX_DEBUG_CONFIG 0x9a1c
+#define mmDPCSTX3_DPCSTX_DEBUG_CONFIG 0x9abc
+#define mmDPCSTX4_DPCSTX_DEBUG_CONFIG 0x9b5c
+#define mmDPCSTX5_DPCSTX_DEBUG_CONFIG 0x9bfc
+#define mmDPCSTX6_DPCSTX_DEBUG_CONFIG 0x9c9c
+#define mmDPCSTX7_DPCSTX_DEBUG_CONFIG 0x9d3c
+#define mmDPCSTX_TEST_DEBUG_DATA 0x48dd
+#define mmDPCSTX0_DPCSTX_TEST_DEBUG_DATA 0x48dd
+#define mmDPCSTX1_DPCSTX_TEST_DEBUG_DATA 0x497d
+#define mmDPCSTX2_DPCSTX_TEST_DEBUG_DATA 0x9a1d
+#define mmDPCSTX3_DPCSTX_TEST_DEBUG_DATA 0x9abd
+#define mmDPCSTX4_DPCSTX_TEST_DEBUG_DATA 0x9b5d
+#define mmDPCSTX5_DPCSTX_TEST_DEBUG_DATA 0x9bfd
+#define mmDPCSTX6_DPCSTX_TEST_DEBUG_DATA 0x9c9d
+#define mmDPCSTX7_DPCSTX_TEST_DEBUG_DATA 0x9d3d
+
+#endif /* DCE_11_2_D_H */
diff --git a/drivers/gpu/drm/amd/include/asic_reg/dce/dce_11_2_enum.h b/drivers/gpu/drm/amd/include/asic_reg/dce/dce_11_2_enum.h
new file mode 100644
index 0000000..b2ea420
--- /dev/null
+++ b/drivers/gpu/drm/amd/include/asic_reg/dce/dce_11_2_enum.h
@@ -0,0 +1,6813 @@
+/*
+ * DCE_11_2 Register documentation
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included
+ * in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+ * OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
+ * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+
+#ifndef DCE_11_2_ENUM_H
+#define DCE_11_2_ENUM_H
+
+typedef enum CRTC_CONTROL_CRTC_START_POINT_CNTL {
+ CRTC_CONTROL_CRTC_START_POINT_CNTL_NORMAL = 0x0,
+ CRTC_CONTROL_CRTC_START_POINT_CNTL_DP = 0x1,
+} CRTC_CONTROL_CRTC_START_POINT_CNTL;
+typedef enum CRTC_CONTROL_CRTC_FIELD_NUMBER_CNTL {
+ CRTC_CONTROL_CRTC_FIELD_NUMBER_CNTL_NORMAL = 0x0,
+ CRTC_CONTROL_CRTC_FIELD_NUMBER_CNTL_DP = 0x1,
+} CRTC_CONTROL_CRTC_FIELD_NUMBER_CNTL;
+typedef enum CRTC_CONTROL_CRTC_DISABLE_POINT_CNTL {
+ CRTC_CONTROL_CRTC_DISABLE_POINT_CNTL_DISABLE = 0x0,
+ CRTC_CONTROL_CRTC_DISABLE_POINT_CNTL_DISABLE_CURRENT= 0x1,
+ CRTC_CONTROL_CRTC_DISABLE_POINT_CNTL_RESERVED = 0x2,
+ CRTC_CONTROL_CRTC_DISABLE_POINT_CNTL_DISABLE_FIRST= 0x3,
+} CRTC_CONTROL_CRTC_DISABLE_POINT_CNTL;
+typedef enum CRTC_CONTROL_CRTC_FIELD_NUMBER_POLARITY {
+ CRTC_CONTROL_CRTC_FIELD_NUMBER_POLARITY_FALSE = 0x0,
+ CRTC_CONTROL_CRTC_FIELD_NUMBER_POLARITY_TRUE = 0x1,
+} CRTC_CONTROL_CRTC_FIELD_NUMBER_POLARITY;
+typedef enum CRTC_CONTROL_CRTC_DISP_READ_REQUEST_DISABLE {
+ CRTC_CONTROL_CRTC_DISP_READ_REQUEST_DISABLE_FALSE= 0x0,
+ CRTC_CONTROL_CRTC_DISP_READ_REQUEST_DISABLE_TRUE = 0x1,
+} CRTC_CONTROL_CRTC_DISP_READ_REQUEST_DISABLE;
+typedef enum CRTC_CONTROL_CRTC_SOF_PULL_EN {
+ CRTC_CONTROL_CRTC_SOF_PULL_EN_FALSE = 0x0,
+ CRTC_CONTROL_CRTC_SOF_PULL_EN_TRUE = 0x1,
+} CRTC_CONTROL_CRTC_SOF_PULL_EN;
+typedef enum CRTC_H_SYNC_B_CNTL_CRTC_H_SYNC_B_POL {
+ CRTC_H_SYNC_B_CNTL_CRTC_H_SYNC_B_POL_FALSE = 0x0,
+ CRTC_H_SYNC_B_CNTL_CRTC_H_SYNC_B_POL_TRUE = 0x1,
+} CRTC_H_SYNC_B_CNTL_CRTC_H_SYNC_B_POL;
+typedef enum CRTC_V_TOTAL_CONTROL_CRTC_V_TOTAL_MAX_SEL {
+ CRTC_V_TOTAL_CONTROL_CRTC_V_TOTAL_MAX_SEL_FALSE = 0x0,
+ CRTC_V_TOTAL_CONTROL_CRTC_V_TOTAL_MAX_SEL_TRUE = 0x1,
+} CRTC_V_TOTAL_CONTROL_CRTC_V_TOTAL_MAX_SEL;
+typedef enum CRTC_V_TOTAL_CONTROL_CRTC_V_TOTAL_MIN_SEL {
+ CRTC_V_TOTAL_CONTROL_CRTC_V_TOTAL_MIN_SEL_FALSE = 0x0,
+ CRTC_V_TOTAL_CONTROL_CRTC_V_TOTAL_MIN_SEL_TRUE = 0x1,
+} CRTC_V_TOTAL_CONTROL_CRTC_V_TOTAL_MIN_SEL;
+typedef enum CRTC_V_TOTAL_CONTROL_CRTC_SET_V_TOTAL_MIN_MASK_EN {
+ CRTC_V_TOTAL_CONTROL_CRTC_SET_V_TOTAL_MIN_MASK_EN_FALSE= 0x0,
+ CRTC_V_TOTAL_CONTROL_CRTC_SET_V_TOTAL_MIN_MASK_EN_TRUE= 0x1,
+} CRTC_V_TOTAL_CONTROL_CRTC_SET_V_TOTAL_MIN_MASK_EN;
+typedef enum CRTC_V_TOTAL_CONTROL_CRTC_FORCE_LOCK_TO_MASTER_VSYNC {
+ CRTC_V_TOTAL_CONTROL_CRTC_FORCE_LOCK_TO_MASTER_VSYNC_DISABLE= 0x0,
+ CRTC_V_TOTAL_CONTROL_CRTC_FORCE_LOCK_TO_MASTER_VSYNC_ENABLE= 0x1,
+} CRTC_V_TOTAL_CONTROL_CRTC_FORCE_LOCK_TO_MASTER_VSYNC;
+typedef enum CRTC_V_TOTAL_CONTROL_CRTC_FORCE_LOCK_ON_EVENT {
+ CRTC_V_TOTAL_CONTROL_CRTC_FORCE_LOCK_ON_EVENT_DISABLE= 0x0,
+ CRTC_V_TOTAL_CONTROL_CRTC_FORCE_LOCK_ON_EVENT_ENABLE= 0x1,
+} CRTC_V_TOTAL_CONTROL_CRTC_FORCE_LOCK_ON_EVENT;
+typedef enum CRTC_V_TOTAL_CONTROL_CRTC_SET_V_TOTAL_MIN_MASK {
+ CRTC_V_TOTAL_CONTROL_CRTC_SET_V_TOTAL_MIN_MASK_FRAME_START= 0x0,
+ CRTC_V_TOTAL_CONTROL_CRTC_SET_V_TOTAL_MIN_MASK_CRTC_TRIG_A= 0x1,
+ CRTC_V_TOTAL_CONTROL_CRTC_SET_V_TOTAL_MIN_MASK_CRTC_TRIG_B= 0x2,
+ CRTC_V_TOTAL_CONTROL_CRTC_SET_V_TOTAL_MIN_MASK_CURSOR_CHANGE= 0x3,
+ CRTC_V_TOTAL_CONTROL_CRTC_SET_V_TOTAL_MIN_MASK_OTHER_CLIENT= 0x4,
+ CRTC_V_TOTAL_CONTROL_CRTC_SET_V_TOTAL_MIN_MASK_MC_DC_REGION0= 0x5,
+ CRTC_V_TOTAL_CONTROL_CRTC_SET_V_TOTAL_MIN_MASK_MC_DC_REGION1= 0x6,
+ CRTC_V_TOTAL_CONTROL_CRTC_SET_V_TOTAL_MIN_MASK_MC_DC_REGION2= 0x7,
+ CRTC_V_TOTAL_CONTROL_CRTC_SET_V_TOTAL_MIN_MASK_MC_DC_REGION3= 0x8,
+ CRTC_V_TOTAL_CONTROL_CRTC_SET_V_TOTAL_MIN_MASK_GRAPHIC_UPDATE_PENDING= 0x9,
+ CRTC_V_TOTAL_CONTROL_CRTC_SET_V_TOTAL_MIN_MASK_RESERVED2= 0xa,
+ CRTC_V_TOTAL_CONTROL_CRTC_SET_V_TOTAL_MIN_MASK_INVALID= 0xb,
+ CRTC_V_TOTAL_CONTROL_CRTC_SET_V_TOTAL_MIN_MASK_DOUBLE_BUFFER= 0xc,
+ CRTC_V_TOTAL_CONTROL_CRTC_SET_V_TOTAL_MIN_MASK_D1CRTC_VERT_COUNT_NOM= 0xd,
+ CRTC_V_TOTAL_CONTROL_CRTC_SET_V_TOTAL_MIN_MASK_D1CRTC_VERT_COUNT= 0xe,
+ CRTC_V_TOTAL_CONTROL_CRTC_SET_V_TOTAL_MIN_MASK_RESERVED= 0xf,
+} CRTC_V_TOTAL_CONTROL_CRTC_SET_V_TOTAL_MIN_MASK;
+typedef enum CRTC_V_TOTAL_INT_STATUS_CRTC_SET_V_TOTAL_MIN_EVENT_OCCURED_ACK {
+ CRTC_V_TOTAL_INT_STATUS_CRTC_SET_V_TOTAL_MIN_EVENT_OCCURED_ACK_FALSE= 0x0,
+ CRTC_V_TOTAL_INT_STATUS_CRTC_SET_V_TOTAL_MIN_EVENT_OCCURED_ACK_TRUE= 0x1,
+} CRTC_V_TOTAL_INT_STATUS_CRTC_SET_V_TOTAL_MIN_EVENT_OCCURED_ACK;
+typedef enum CRTC_VSYNC_NOM_INT_STATUS_CRTC_VSYNC_NOM_INT_CLEAR {
+ CRTC_VSYNC_NOM_INT_STATUS_CRTC_VSYNC_NOM_INT_CLEAR_FALSE= 0x0,
+ CRTC_VSYNC_NOM_INT_STATUS_CRTC_VSYNC_NOM_INT_CLEAR_TRUE= 0x1,
+} CRTC_VSYNC_NOM_INT_STATUS_CRTC_VSYNC_NOM_INT_CLEAR;
+typedef enum CRTC_V_SYNC_B_CNTL_CRTC_V_SYNC_B_POL {
+ CRTC_V_SYNC_B_CNTL_CRTC_V_SYNC_B_POL_FALSE = 0x0,
+ CRTC_V_SYNC_B_CNTL_CRTC_V_SYNC_B_POL_TRUE = 0x1,
+} CRTC_V_SYNC_B_CNTL_CRTC_V_SYNC_B_POL;
+typedef enum CRTC_DTMTEST_CNTL_CRTC_DTMTEST_CRTC_EN {
+ CRTC_DTMTEST_CNTL_CRTC_DTMTEST_CRTC_EN_FALSE = 0x0,
+ CRTC_DTMTEST_CNTL_CRTC_DTMTEST_CRTC_EN_TRUE = 0x1,
+} CRTC_DTMTEST_CNTL_CRTC_DTMTEST_CRTC_EN;
+typedef enum CRTC_TRIGA_CNTL_CRTC_TRIGA_SOURCE_SELECT {
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_SOURCE_SELECT_VSYNCA_OTHER= 0x1,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_SOURCE_SELECT_HSYNCA_OTHER= 0x2,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_SOURCE_SELECT_GENERICF= 0x5,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_SOURCE_SELECT_GENERICE= 0x6,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_SOURCE_SELECT_VSYNCA = 0x7,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_SOURCE_SELECT_HSYNCA = 0x8,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_SOURCE_SELECT_VSYNCB = 0x9,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_SOURCE_SELECT_HSYNCB = 0xa,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_SOURCE_SELECT_HPD1 = 0xb,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_SOURCE_SELECT_HPD2 = 0xc,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_SOURCE_SELECT_GENERICD= 0xd,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_SOURCE_SELECT_GENERICC= 0xe,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_SOURCE_SELECT_IGSL0 = 0x10,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_SOURCE_SELECT_IGSL1 = 0x11,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_SOURCE_SELECT_IGSL2 = 0x12,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_SOURCE_SELECT_IBLON = 0x13,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_SOURCE_SELECT_GENERICA= 0x14,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_SOURCE_SELECT_GENERICB= 0x15,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_SOURCE_SELECT_IGSL_ALLOW= 0x16,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_SOURCE_SELECT_MANUAL_FLOW= 0x17,
+} CRTC_TRIGA_CNTL_CRTC_TRIGA_SOURCE_SELECT;
+typedef enum CRTC_TRIGA_CNTL_CRTC_TRIGA_POLARITY_SELECT {
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_POLARITY_SELECT_INTERLACE= 0x1,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_POLARITY_SELECT_GENERICA= 0x2,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_POLARITY_SELECT_GENERICB= 0x3,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_POLARITY_SELECT_HSYNCA= 0x4,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_POLARITY_SELECT_HSYNCB= 0x5,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_POLARITY_SELECT_VIDEO = 0x6,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_POLARITY_SELECT_GENERICC= 0x7,
+} CRTC_TRIGA_CNTL_CRTC_TRIGA_POLARITY_SELECT;
+typedef enum CRTC_TRIGA_CNTL_CRTC_TRIGA_RESYNC_BYPASS_EN {
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_RESYNC_BYPASS_EN_FALSE= 0x0,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_RESYNC_BYPASS_EN_TRUE = 0x1,
+} CRTC_TRIGA_CNTL_CRTC_TRIGA_RESYNC_BYPASS_EN;
+typedef enum CRTC_TRIGA_CNTL_CRTC_TRIGA_CLEAR {
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_CLEAR_FALSE = 0x0,
+ CRTC_TRIGA_CNTL_CRTC_TRIGA_CLEAR_TRUE = 0x1,
+} CRTC_TRIGA_CNTL_CRTC_TRIGA_CLEAR;
+typedef enum CRTC_TRIGB_CNTL_CRTC_TRIGB_SOURCE_SELECT {
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_SOURCE_SELECT_VSYNCA_OTHER= 0x1,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_SOURCE_SELECT_HSYNCA_OTHER= 0x2,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_SOURCE_SELECT_GENERICF= 0x5,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_SOURCE_SELECT_GENERICE= 0x6,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_SOURCE_SELECT_VSYNCA = 0x7,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_SOURCE_SELECT_HSYNCA = 0x8,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_SOURCE_SELECT_VSYNCB = 0x9,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_SOURCE_SELECT_HSYNCB = 0xa,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_SOURCE_SELECT_HPD1 = 0xb,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_SOURCE_SELECT_HPD2 = 0xc,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_SOURCE_SELECT_GENERICD= 0xd,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_SOURCE_SELECT_GENERICC= 0xe,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_SOURCE_SELECT_IGSL0 = 0x10,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_SOURCE_SELECT_IGSL1 = 0x11,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_SOURCE_SELECT_IGSL2 = 0x12,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_SOURCE_SELECT_IBLON = 0x13,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_SOURCE_SELECT_GENERICA= 0x14,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_SOURCE_SELECT_GENERICB= 0x15,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_SOURCE_SELECT_IGSL_ALLOW= 0x16,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_SOURCE_SELECT_MANUAL_FLOW= 0x17,
+} CRTC_TRIGB_CNTL_CRTC_TRIGB_SOURCE_SELECT;
+typedef enum CRTC_TRIGB_CNTL_CRTC_TRIGB_POLARITY_SELECT {
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_POLARITY_SELECT_INTERLACE= 0x1,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_POLARITY_SELECT_GENERICA= 0x2,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_POLARITY_SELECT_GENERICB= 0x3,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_POLARITY_SELECT_HSYNCA= 0x4,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_POLARITY_SELECT_HSYNCB= 0x5,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_POLARITY_SELECT_VIDEO = 0x6,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_POLARITY_SELECT_GENERICC= 0x7,
+} CRTC_TRIGB_CNTL_CRTC_TRIGB_POLARITY_SELECT;
+typedef enum CRTC_TRIGB_CNTL_CRTC_TRIGB_RESYNC_BYPASS_EN {
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_RESYNC_BYPASS_EN_FALSE= 0x0,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_RESYNC_BYPASS_EN_TRUE = 0x1,
+} CRTC_TRIGB_CNTL_CRTC_TRIGB_RESYNC_BYPASS_EN;
+typedef enum CRTC_TRIGB_CNTL_CRTC_TRIGB_CLEAR {
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_CLEAR_FALSE = 0x0,
+ CRTC_TRIGB_CNTL_CRTC_TRIGB_CLEAR_TRUE = 0x1,
+} CRTC_TRIGB_CNTL_CRTC_TRIGB_CLEAR;
+typedef enum CRTC_FORCE_COUNT_NOW_CNTL_CRTC_FORCE_COUNT_NOW_MODE {
+ CRTC_FORCE_COUNT_NOW_CNTL_CRTC_FORCE_COUNT_NOW_MODE_DISABLE= 0x0,
+ CRTC_FORCE_COUNT_NOW_CNTL_CRTC_FORCE_COUNT_NOW_MODE_HCOUNT= 0x1,
+ CRTC_FORCE_COUNT_NOW_CNTL_CRTC_FORCE_COUNT_NOW_MODE_HCOUNT_VCOUNT= 0x2,
+ CRTC_FORCE_COUNT_NOW_CNTL_CRTC_FORCE_COUNT_NOW_MODE_RESERVED= 0x3,
+} CRTC_FORCE_COUNT_NOW_CNTL_CRTC_FORCE_COUNT_NOW_MODE;
+typedef enum CRTC_FORCE_COUNT_NOW_CNTL_CRTC_FORCE_COUNT_NOW_CHECK {
+ CRTC_FORCE_COUNT_NOW_CNTL_CRTC_FORCE_COUNT_NOW_CHECK_FALSE= 0x0,
+ CRTC_FORCE_COUNT_NOW_CNTL_CRTC_FORCE_COUNT_NOW_CHECK_TRUE= 0x1,
+} CRTC_FORCE_COUNT_NOW_CNTL_CRTC_FORCE_COUNT_NOW_CHECK;
+typedef enum CRTC_FORCE_COUNT_NOW_CNTL_CRTC_FORCE_COUNT_NOW_TRIG_SEL {
+ CRTC_FORCE_COUNT_NOW_CNTL_CRTC_FORCE_COUNT_NOW_TRIG_SEL_FALSE= 0x0,
+ CRTC_FORCE_COUNT_NOW_CNTL_CRTC_FORCE_COUNT_NOW_TRIG_SEL_TRUE= 0x1,
+} CRTC_FORCE_COUNT_NOW_CNTL_CRTC_FORCE_COUNT_NOW_TRIG_SEL;
+typedef enum CRTC_FORCE_COUNT_NOW_CNTL_CRTC_FORCE_COUNT_NOW_CLEAR {
+ CRTC_FORCE_COUNT_NOW_CNTL_CRTC_FORCE_COUNT_NOW_CLEAR_FALSE= 0x0,
+ CRTC_FORCE_COUNT_NOW_CNTL_CRTC_FORCE_COUNT_NOW_CLEAR_TRUE= 0x1,
+} CRTC_FORCE_COUNT_NOW_CNTL_CRTC_FORCE_COUNT_NOW_CLEAR;
+typedef enum CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_SOURCE_SELECT {
+ CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_SOURCE_SELECT_LOGIC0= 0x0,
+ CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_SOURCE_SELECT_GENERICF= 0x1,
+ CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_SOURCE_SELECT_GENERICE= 0x2,
+ CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_SOURCE_SELECT_HPD1= 0x3,
+ CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_SOURCE_SELECT_HPD2= 0x4,
+ CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_SOURCE_SELECT_DDC1DATA= 0x5,
+ CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_SOURCE_SELECT_DDC1CLK= 0x6,
+ CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_SOURCE_SELECT_DDC2DATA= 0x7,
+ CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_SOURCE_SELECT_DDC2CLK= 0x8,
+ CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_SOURCE_SELECT_DVOCLK= 0x9,
+ CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_SOURCE_SELECT_MANUAL= 0xa,
+ CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_SOURCE_SELECT_LOGIC1= 0xb,
+ CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_SOURCE_SELECT_GENERICB= 0xc,
+ CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_SOURCE_SELECT_GENERICA= 0xd,
+ CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_SOURCE_SELECT_GENERICD= 0xe,
+ CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_SOURCE_SELECT_GENERICC= 0xf,
+} CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_SOURCE_SELECT;
+typedef enum CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_POLARITY {
+ CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_POLARITY_FALSE= 0x0,
+ CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_POLARITY_TRUE= 0x1,
+} CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_POLARITY;
+typedef enum CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_GRANULARITY {
+ CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_GRANULARITY_FALSE= 0x0,
+ CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_GRANULARITY_TRUE= 0x1,
+} CRTC_FLOW_CONTROL_CRTC_FLOW_CONTROL_GRANULARITY;
+typedef enum CRTC_STEREO_FORCE_NEXT_EYE_CRTC_STEREO_FORCE_NEXT_EYE {
+ CRTC_STEREO_FORCE_NEXT_EYE_CRTC_STEREO_FORCE_NEXT_EYE_NO= 0x0,
+ CRTC_STEREO_FORCE_NEXT_EYE_CRTC_STEREO_FORCE_NEXT_EYE_RIGHT= 0x1,
+ CRTC_STEREO_FORCE_NEXT_EYE_CRTC_STEREO_FORCE_NEXT_EYE_LEFT= 0x2,
+ CRTC_STEREO_FORCE_NEXT_EYE_CRTC_STEREO_FORCE_NEXT_EYE_RESERVED= 0x3,
+} CRTC_STEREO_FORCE_NEXT_EYE_CRTC_STEREO_FORCE_NEXT_EYE;
+typedef enum CRTC_CONTROL_CRTC_MASTER_EN {
+ CRTC_CONTROL_CRTC_MASTER_EN_FALSE = 0x0,
+ CRTC_CONTROL_CRTC_MASTER_EN_TRUE = 0x1,
+} CRTC_CONTROL_CRTC_MASTER_EN;
+typedef enum CRTC_BLANK_CONTROL_CRTC_BLANK_DATA_EN {
+ CRTC_BLANK_CONTROL_CRTC_BLANK_DATA_EN_FALSE = 0x0,
+ CRTC_BLANK_CONTROL_CRTC_BLANK_DATA_EN_TRUE = 0x1,
+} CRTC_BLANK_CONTROL_CRTC_BLANK_DATA_EN;
+typedef enum CRTC_BLANK_CONTROL_CRTC_BLANK_DE_MODE {
+ CRTC_BLANK_CONTROL_CRTC_BLANK_DE_MODE_FALSE = 0x0,
+ CRTC_BLANK_CONTROL_CRTC_BLANK_DE_MODE_TRUE = 0x1,
+} CRTC_BLANK_CONTROL_CRTC_BLANK_DE_MODE;
+typedef enum CRTC_INTERLACE_CONTROL_CRTC_INTERLACE_ENABLE {
+ CRTC_INTERLACE_CONTROL_CRTC_INTERLACE_ENABLE_FALSE= 0x0,
+ CRTC_INTERLACE_CONTROL_CRTC_INTERLACE_ENABLE_TRUE= 0x1,
+} CRTC_INTERLACE_CONTROL_CRTC_INTERLACE_ENABLE;
+typedef enum CRTC_INTERLACE_CONTROL_CRTC_INTERLACE_FORCE_NEXT_FIELD {
+ CRTC_INTERLACE_CONTROL_CRTC_INTERLACE_FORCE_NEXT_FIELD_NOT= 0x0,
+ CRTC_INTERLACE_CONTROL_CRTC_INTERLACE_FORCE_NEXT_FIELD_ODD= 0x1,
+ CRTC_INTERLACE_CONTROL_CRTC_INTERLACE_FORCE_NEXT_FIELD_EVEN= 0x2,
+ CRTC_INTERLACE_CONTROL_CRTC_INTERLACE_FORCE_NEXT_FIELD_NOT2= 0x3,
+} CRTC_INTERLACE_CONTROL_CRTC_INTERLACE_FORCE_NEXT_FIELD;
+typedef enum CRTC_FIELD_INDICATION_CONTROL_CRTC_FIELD_INDICATION_OUTPUT_POLARITY {
+ CRTC_FIELD_INDICATION_CONTROL_CRTC_FIELD_INDICATION_OUTPUT_POLARITY_FALSE= 0x0,
+ CRTC_FIELD_INDICATION_CONTROL_CRTC_FIELD_INDICATION_OUTPUT_POLARITY_TRUE= 0x1,
+} CRTC_FIELD_INDICATION_CONTROL_CRTC_FIELD_INDICATION_OUTPUT_POLARITY;
+typedef enum CRTC_FIELD_INDICATION_CONTROL_CRTC_FIELD_ALIGNMENT {
+ CRTC_FIELD_INDICATION_CONTROL_CRTC_FIELD_ALIGNMENT_FALSE= 0x0,
+ CRTC_FIELD_INDICATION_CONTROL_CRTC_FIELD_ALIGNMENT_TRUE= 0x1,
+} CRTC_FIELD_INDICATION_CONTROL_CRTC_FIELD_ALIGNMENT;
+typedef enum CRTC_COUNT_CONTROL_CRTC_HORZ_COUNT_BY2_EN {
+ CRTC_COUNT_CONTROL_CRTC_HORZ_COUNT_BY2_EN_FALSE = 0x0,
+ CRTC_COUNT_CONTROL_CRTC_HORZ_COUNT_BY2_EN_TRUE = 0x1,
+} CRTC_COUNT_CONTROL_CRTC_HORZ_COUNT_BY2_EN;
+typedef enum CRTC_MANUAL_FORCE_VSYNC_NEXT_LINE_CRTC_MANUAL_FORCE_VSYNC_NEXT_LINE {
+ CRTC_MANUAL_FORCE_VSYNC_NEXT_LINE_CRTC_MANUAL_FORCE_VSYNC_NEXT_LINE_FALSE= 0x0,
+ CRTC_MANUAL_FORCE_VSYNC_NEXT_LINE_CRTC_MANUAL_FORCE_VSYNC_NEXT_LINE_TRUE= 0x1,
+} CRTC_MANUAL_FORCE_VSYNC_NEXT_LINE_CRTC_MANUAL_FORCE_VSYNC_NEXT_LINE;
+typedef enum CRTC_VERT_SYNC_CONTROL_CRTC_FORCE_VSYNC_NEXT_LINE_CLEAR {
+ CRTC_VERT_SYNC_CONTROL_CRTC_FORCE_VSYNC_NEXT_LINE_CLEAR_FALSE= 0x0,
+ CRTC_VERT_SYNC_CONTROL_CRTC_FORCE_VSYNC_NEXT_LINE_CLEAR_TRUE= 0x1,
+} CRTC_VERT_SYNC_CONTROL_CRTC_FORCE_VSYNC_NEXT_LINE_CLEAR;
+typedef enum CRTC_VERT_SYNC_CONTROL_CRTC_AUTO_FORCE_VSYNC_MODE {
+ CRTC_VERT_SYNC_CONTROL_CRTC_AUTO_FORCE_VSYNC_MODE_DISABLE= 0x0,
+ CRTC_VERT_SYNC_CONTROL_CRTC_AUTO_FORCE_VSYNC_MODE_TRIGGERA= 0x1,
+ CRTC_VERT_SYNC_CONTROL_CRTC_AUTO_FORCE_VSYNC_MODE_TRIGGERB= 0x2,
+ CRTC_VERT_SYNC_CONTROL_CRTC_AUTO_FORCE_VSYNC_MODE_RESERVED= 0x3,
+} CRTC_VERT_SYNC_CONTROL_CRTC_AUTO_FORCE_VSYNC_MODE;
+typedef enum CRTC_STEREO_CONTROL_CRTC_STEREO_SYNC_OUTPUT_POLARITY {
+ CRTC_STEREO_CONTROL_CRTC_STEREO_SYNC_OUTPUT_POLARITY_FALSE= 0x0,
+ CRTC_STEREO_CONTROL_CRTC_STEREO_SYNC_OUTPUT_POLARITY_TRUE= 0x1,
+} CRTC_STEREO_CONTROL_CRTC_STEREO_SYNC_OUTPUT_POLARITY;
+typedef enum CRTC_STEREO_CONTROL_CRTC_STEREO_SYNC_SELECT_POLARITY {
+ CRTC_STEREO_CONTROL_CRTC_STEREO_SYNC_SELECT_POLARITY_FALSE= 0x0,
+ CRTC_STEREO_CONTROL_CRTC_STEREO_SYNC_SELECT_POLARITY_TRUE= 0x1,
+} CRTC_STEREO_CONTROL_CRTC_STEREO_SYNC_SELECT_POLARITY;
+typedef enum CRTC_STEREO_CONTROL_CRTC_STEREO_EYE_FLAG_POLARITY {
+ CRTC_STEREO_CONTROL_CRTC_STEREO_EYE_FLAG_POLARITY_FALSE= 0x0,
+ CRTC_STEREO_CONTROL_CRTC_STEREO_EYE_FLAG_POLARITY_TRUE= 0x1,
+} CRTC_STEREO_CONTROL_CRTC_STEREO_EYE_FLAG_POLARITY;
+typedef enum CRTC_STEREO_CONTROL_CRTC_STEREO_EN {
+ CRTC_STEREO_CONTROL_CRTC_STEREO_EN_FALSE = 0x0,
+ CRTC_STEREO_CONTROL_CRTC_STEREO_EN_TRUE = 0x1,
+} CRTC_STEREO_CONTROL_CRTC_STEREO_EN;
+typedef enum CRTC_SNAPSHOT_STATUS_CRTC_SNAPSHOT_CLEAR {
+ CRTC_SNAPSHOT_STATUS_CRTC_SNAPSHOT_CLEAR_FALSE = 0x0,
+ CRTC_SNAPSHOT_STATUS_CRTC_SNAPSHOT_CLEAR_TRUE = 0x1,
+} CRTC_SNAPSHOT_STATUS_CRTC_SNAPSHOT_CLEAR;
+typedef enum CRTC_SNAPSHOT_CONTROL_CRTC_AUTO_SNAPSHOT_TRIG_SEL {
+ CRTC_SNAPSHOT_CONTROL_CRTC_AUTO_SNAPSHOT_TRIG_SEL_DISABLE= 0x0,
+ CRTC_SNAPSHOT_CONTROL_CRTC_AUTO_SNAPSHOT_TRIG_SEL_TRIGGERA= 0x1,
+ CRTC_SNAPSHOT_CONTROL_CRTC_AUTO_SNAPSHOT_TRIG_SEL_TRIGGERB= 0x2,
+ CRTC_SNAPSHOT_CONTROL_CRTC_AUTO_SNAPSHOT_TRIG_SEL_RESERVED= 0x3,
+} CRTC_SNAPSHOT_CONTROL_CRTC_AUTO_SNAPSHOT_TRIG_SEL;
+typedef enum CRTC_START_LINE_CONTROL_CRTC_PROGRESSIVE_START_LINE_EARLY {
+ CRTC_START_LINE_CONTROL_CRTC_PROGRESSIVE_START_LINE_EARLY_FALSE= 0x0,
+ CRTC_START_LINE_CONTROL_CRTC_PROGRESSIVE_START_LINE_EARLY_TRUE= 0x1,
+} CRTC_START_LINE_CONTROL_CRTC_PROGRESSIVE_START_LINE_EARLY;
+typedef enum CRTC_START_LINE_CONTROL_CRTC_INTERLACE_START_LINE_EARLY {
+ CRTC_START_LINE_CONTROL_CRTC_INTERLACE_START_LINE_EARLY_FALSE= 0x0,
+ CRTC_START_LINE_CONTROL_CRTC_INTERLACE_START_LINE_EARLY_TRUE= 0x1,
+} CRTC_START_LINE_CONTROL_CRTC_INTERLACE_START_LINE_EARLY;
+typedef enum CRTC_START_LINE_CONTROL_CRTC_LEGACY_REQUESTOR_EN {
+ CRTC_START_LINE_CONTROL_CRTC_LEGACY_REQUESTOR_EN_FALSE= 0x0,
+ CRTC_START_LINE_CONTROL_CRTC_LEGACY_REQUESTOR_EN_TRUE= 0x1,
+} CRTC_START_LINE_CONTROL_CRTC_LEGACY_REQUESTOR_EN;
+typedef enum CRTC_START_LINE_CONTROL_CRTC_PREFETCH_EN {
+ CRTC_START_LINE_CONTROL_CRTC_PREFETCH_EN_FALSE = 0x0,
+ CRTC_START_LINE_CONTROL_CRTC_PREFETCH_EN_TRUE = 0x1,
+} CRTC_START_LINE_CONTROL_CRTC_PREFETCH_EN;
+typedef enum CRTC_INTERRUPT_CONTROL_CRTC_SNAPSHOT_INT_MSK {
+ CRTC_INTERRUPT_CONTROL_CRTC_SNAPSHOT_INT_MSK_FALSE= 0x0,
+ CRTC_INTERRUPT_CONTROL_CRTC_SNAPSHOT_INT_MSK_TRUE= 0x1,
+} CRTC_INTERRUPT_CONTROL_CRTC_SNAPSHOT_INT_MSK;
+typedef enum CRTC_INTERRUPT_CONTROL_CRTC_SNAPSHOT_INT_TYPE {
+ CRTC_INTERRUPT_CONTROL_CRTC_SNAPSHOT_INT_TYPE_FALSE= 0x0,
+ CRTC_INTERRUPT_CONTROL_CRTC_SNAPSHOT_INT_TYPE_TRUE= 0x1,
+} CRTC_INTERRUPT_CONTROL_CRTC_SNAPSHOT_INT_TYPE;
+typedef enum CRTC_INTERRUPT_CONTROL_CRTC_V_UPDATE_INT_MSK {
+ CRTC_INTERRUPT_CONTROL_CRTC_V_UPDATE_INT_MSK_FALSE= 0x0,
+ CRTC_INTERRUPT_CONTROL_CRTC_V_UPDATE_INT_MSK_TRUE= 0x1,
+} CRTC_INTERRUPT_CONTROL_CRTC_V_UPDATE_INT_MSK;
+typedef enum CRTC_INTERRUPT_CONTROL_CRTC_V_UPDATE_INT_TYPE {
+ CRTC_INTERRUPT_CONTROL_CRTC_V_UPDATE_INT_TYPE_FALSE= 0x0,
+ CRTC_INTERRUPT_CONTROL_CRTC_V_UPDATE_INT_TYPE_TRUE= 0x1,
+} CRTC_INTERRUPT_CONTROL_CRTC_V_UPDATE_INT_TYPE;
+typedef enum CRTC_INTERRUPT_CONTROL_CRTC_FORCE_COUNT_NOW_INT_MSK {
+ CRTC_INTERRUPT_CONTROL_CRTC_FORCE_COUNT_NOW_INT_MSK_FALSE= 0x0,
+ CRTC_INTERRUPT_CONTROL_CRTC_FORCE_COUNT_NOW_INT_MSK_TRUE= 0x1,
+} CRTC_INTERRUPT_CONTROL_CRTC_FORCE_COUNT_NOW_INT_MSK;
+typedef enum CRTC_INTERRUPT_CONTROL_CRTC_FORCE_COUNT_NOW_INT_TYPE {
+ CRTC_INTERRUPT_CONTROL_CRTC_FORCE_COUNT_NOW_INT_TYPE_FALSE= 0x0,
+ CRTC_INTERRUPT_CONTROL_CRTC_FORCE_COUNT_NOW_INT_TYPE_TRUE= 0x1,
+} CRTC_INTERRUPT_CONTROL_CRTC_FORCE_COUNT_NOW_INT_TYPE;
+typedef enum CRTC_INTERRUPT_CONTROL_CRTC_FORCE_VSYNC_NEXT_LINE_INT_MSK {
+ CRTC_INTERRUPT_CONTROL_CRTC_FORCE_VSYNC_NEXT_LINE_INT_MSK_FALSE= 0x0,
+ CRTC_INTERRUPT_CONTROL_CRTC_FORCE_VSYNC_NEXT_LINE_INT_MSK_TRUE= 0x1,
+} CRTC_INTERRUPT_CONTROL_CRTC_FORCE_VSYNC_NEXT_LINE_INT_MSK;
+typedef enum CRTC_INTERRUPT_CONTROL_CRTC_FORCE_VSYNC_NEXT_LINE_INT_TYPE {
+ CRTC_INTERRUPT_CONTROL_CRTC_FORCE_VSYNC_NEXT_LINE_INT_TYPE_FALSE= 0x0,
+ CRTC_INTERRUPT_CONTROL_CRTC_FORCE_VSYNC_NEXT_LINE_INT_TYPE_TRUE= 0x1,
+} CRTC_INTERRUPT_CONTROL_CRTC_FORCE_VSYNC_NEXT_LINE_INT_TYPE;
+typedef enum CRTC_INTERRUPT_CONTROL_CRTC_TRIGA_INT_MSK {
+ CRTC_INTERRUPT_CONTROL_CRTC_TRIGA_INT_MSK_FALSE = 0x0,
+ CRTC_INTERRUPT_CONTROL_CRTC_TRIGA_INT_MSK_TRUE = 0x1,
+} CRTC_INTERRUPT_CONTROL_CRTC_TRIGA_INT_MSK;
+typedef enum CRTC_INTERRUPT_CONTROL_CRTC_TRIGA_INT_TYPE {
+ CRTC_INTERRUPT_CONTROL_CRTC_TRIGA_INT_TYPE_FALSE = 0x0,
+ CRTC_INTERRUPT_CONTROL_CRTC_TRIGA_INT_TYPE_TRUE = 0x1,
+} CRTC_INTERRUPT_CONTROL_CRTC_TRIGA_INT_TYPE;
+typedef enum CRTC_INTERRUPT_CONTROL_CRTC_TRIGB_INT_MSK {
+ CRTC_INTERRUPT_CONTROL_CRTC_TRIGB_INT_MSK_FALSE = 0x0,
+ CRTC_INTERRUPT_CONTROL_CRTC_TRIGB_INT_MSK_TRUE = 0x1,
+} CRTC_INTERRUPT_CONTROL_CRTC_TRIGB_INT_MSK;
+typedef enum CRTC_INTERRUPT_CONTROL_CRTC_TRIGB_INT_TYPE {
+ CRTC_INTERRUPT_CONTROL_CRTC_TRIGB_INT_TYPE_FALSE = 0x0,
+ CRTC_INTERRUPT_CONTROL_CRTC_TRIGB_INT_TYPE_TRUE = 0x1,
+} CRTC_INTERRUPT_CONTROL_CRTC_TRIGB_INT_TYPE;
+typedef enum CRTC_INTERRUPT_CONTROL_CRTC_VSYNC_NOM_INT_MSK {
+ CRTC_INTERRUPT_CONTROL_CRTC_VSYNC_NOM_INT_MSK_FALSE= 0x0,
+ CRTC_INTERRUPT_CONTROL_CRTC_VSYNC_NOM_INT_MSK_TRUE= 0x1,
+} CRTC_INTERRUPT_CONTROL_CRTC_VSYNC_NOM_INT_MSK;
+typedef enum CRTC_INTERRUPT_CONTROL_CRTC_VSYNC_NOM_INT_TYPE {
+ CRTC_INTERRUPT_CONTROL_CRTC_VSYNC_NOM_INT_TYPE_FALSE= 0x0,
+ CRTC_INTERRUPT_CONTROL_CRTC_VSYNC_NOM_INT_TYPE_TRUE= 0x1,
+} CRTC_INTERRUPT_CONTROL_CRTC_VSYNC_NOM_INT_TYPE;
+typedef enum CRTC_INTERRUPT_CONTROL_CRTC_GSL_VSYNC_GAP_INT_MSK {
+ CRTC_INTERRUPT_CONTROL_CRTC_GSL_VSYNC_GAP_INT_MSK_FALSE= 0x0,
+ CRTC_INTERRUPT_CONTROL_CRTC_GSL_VSYNC_GAP_INT_MSK_TRUE= 0x1,
+} CRTC_INTERRUPT_CONTROL_CRTC_GSL_VSYNC_GAP_INT_MSK;
+typedef enum CRTC_INTERRUPT_CONTROL_CRTC_GSL_VSYNC_GAP_INT_TYPE {
+ CRTC_INTERRUPT_CONTROL_CRTC_GSL_VSYNC_GAP_INT_TYPE_FALSE= 0x0,
+ CRTC_INTERRUPT_CONTROL_CRTC_GSL_VSYNC_GAP_INT_TYPE_TRUE= 0x1,
+} CRTC_INTERRUPT_CONTROL_CRTC_GSL_VSYNC_GAP_INT_TYPE;
+typedef enum CRTC_UPDATE_LOCK_CRTC_UPDATE_LOCK {
+ CRTC_UPDATE_LOCK_CRTC_UPDATE_LOCK_FALSE = 0x0,
+ CRTC_UPDATE_LOCK_CRTC_UPDATE_LOCK_TRUE = 0x1,
+} CRTC_UPDATE_LOCK_CRTC_UPDATE_LOCK;
+typedef enum CRTC_DOUBLE_BUFFER_CONTROL_CRTC_UPDATE_INSTANTLY {
+ CRTC_DOUBLE_BUFFER_CONTROL_CRTC_UPDATE_INSTANTLY_FALSE= 0x0,
+ CRTC_DOUBLE_BUFFER_CONTROL_CRTC_UPDATE_INSTANTLY_TRUE= 0x1,
+} CRTC_DOUBLE_BUFFER_CONTROL_CRTC_UPDATE_INSTANTLY;
+typedef enum CRTC_DOUBLE_BUFFER_CONTROL_CRTC_BLANK_DATA_DOUBLE_BUFFER_EN {
+ CRTC_DOUBLE_BUFFER_CONTROL_CRTC_BLANK_DATA_DOUBLE_BUFFER_EN_FALSE= 0x0,
+ CRTC_DOUBLE_BUFFER_CONTROL_CRTC_BLANK_DATA_DOUBLE_BUFFER_EN_TRUE= 0x1,
+} CRTC_DOUBLE_BUFFER_CONTROL_CRTC_BLANK_DATA_DOUBLE_BUFFER_EN;
+typedef enum CRTC_VGA_PARAMETER_CAPTURE_MODE_CRTC_VGA_PARAMETER_CAPTURE_MODE {
+ CRTC_VGA_PARAMETER_CAPTURE_MODE_CRTC_VGA_PARAMETER_CAPTURE_MODE_FALSE= 0x0,
+ CRTC_VGA_PARAMETER_CAPTURE_MODE_CRTC_VGA_PARAMETER_CAPTURE_MODE_TRUE= 0x1,
+} CRTC_VGA_PARAMETER_CAPTURE_MODE_CRTC_VGA_PARAMETER_CAPTURE_MODE;
+typedef enum CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_EN {
+ CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_EN_FALSE= 0x0,
+ CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_EN_TRUE= 0x1,
+} CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_EN;
+typedef enum CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_MODE {
+ CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_MODE_RGB= 0x0,
+ CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_MODE_YCBCR601= 0x1,
+ CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_MODE_YCBCR709= 0x2,
+ CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_MODE_VBARS= 0x3,
+ CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_MODE_HBARS= 0x4,
+ CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_MODE_SRRGB= 0x5,
+ CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_MODE_DRRGB= 0x6,
+ CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_MODE_XRBIAS= 0x7,
+} CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_MODE;
+typedef enum CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_DYNAMIC_RANGE {
+ CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_DYNAMIC_RANGE_FALSE= 0x0,
+ CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_DYNAMIC_RANGE_TRUE= 0x1,
+} CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_DYNAMIC_RANGE;
+typedef enum CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_COLOR_FORMAT {
+ CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_COLOR_FORMAT_6BPC= 0x0,
+ CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_COLOR_FORMAT_8BPC= 0x1,
+ CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_COLOR_FORMAT_10BPC= 0x2,
+ CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_COLOR_FORMAT_RESERVED= 0x3,
+} CRTC_TEST_PATTERN_CONTROL_CRTC_TEST_PATTERN_COLOR_FORMAT;
+typedef enum MASTER_UPDATE_LOCK_MASTER_UPDATE_LOCK {
+ MASTER_UPDATE_LOCK_MASTER_UPDATE_LOCK_FALSE = 0x0,
+ MASTER_UPDATE_LOCK_MASTER_UPDATE_LOCK_TRUE = 0x1,
+} MASTER_UPDATE_LOCK_MASTER_UPDATE_LOCK;
+typedef enum MASTER_UPDATE_LOCK_GSL_CONTROL_MASTER_UPDATE_LOCK {
+ MASTER_UPDATE_LOCK_GSL_CONTROL_MASTER_UPDATE_LOCK_FALSE= 0x0,
+ MASTER_UPDATE_LOCK_GSL_CONTROL_MASTER_UPDATE_LOCK_TRUE= 0x1,
+} MASTER_UPDATE_LOCK_GSL_CONTROL_MASTER_UPDATE_LOCK;
+typedef enum MASTER_UPDATE_LOCK_UNDERFLOW_UPDATE_LOCK {
+ MASTER_UPDATE_LOCK_UNDERFLOW_UPDATE_LOCK_FALSE = 0x0,
+ MASTER_UPDATE_LOCK_UNDERFLOW_UPDATE_LOCK_TRUE = 0x1,
+} MASTER_UPDATE_LOCK_UNDERFLOW_UPDATE_LOCK;
+typedef enum MASTER_UPDATE_MODE_MASTER_UPDATE_MODE {
+ MASTER_UPDATE_MODE_MASTER_UPDATE_MODE_BETWEEN = 0x0,
+ MASTER_UPDATE_MODE_MASTER_UPDATE_MODE_HSYNCA = 0x1,
+ MASTER_UPDATE_MODE_MASTER_UPDATE_MODE_VSYNCA = 0x2,
+ MASTER_UPDATE_MODE_MASTER_UPDATE_MODE_BEFORE = 0x3,
+} MASTER_UPDATE_MODE_MASTER_UPDATE_MODE;
+typedef enum MASTER_UPDATE_MODE_MASTER_UPDATE_INTERLACED_MODE {
+ MASTER_UPDATE_MODE_MASTER_UPDATE_INTERLACED_MODE_BOTH= 0x0,
+ MASTER_UPDATE_MODE_MASTER_UPDATE_INTERLACED_MODE_EVEN= 0x1,
+ MASTER_UPDATE_MODE_MASTER_UPDATE_INTERLACED_MODE_ODD= 0x2,
+ MASTER_UPDATE_MODE_MASTER_UPDATE_INTERLACED_MODE_RESERVED= 0x3,
+} MASTER_UPDATE_MODE_MASTER_UPDATE_INTERLACED_MODE;
+typedef enum CRTC_MVP_INBAND_CNTL_INSERT_CRTC_MVP_INBAND_OUT_MODE {
+ CRTC_MVP_INBAND_CNTL_INSERT_CRTC_MVP_INBAND_OUT_MODE_DISABLE= 0x0,
+ CRTC_MVP_INBAND_CNTL_INSERT_CRTC_MVP_INBAND_OUT_MODE_DEBUG= 0x1,
+ CRTC_MVP_INBAND_CNTL_INSERT_CRTC_MVP_INBAND_OUT_MODE_NORMAL= 0x2,
+} CRTC_MVP_INBAND_CNTL_INSERT_CRTC_MVP_INBAND_OUT_MODE;
+typedef enum CRTC_MVP_STATUS_CRTC_FLIP_NOW_CLEAR {
+ CRTC_MVP_STATUS_CRTC_FLIP_NOW_CLEAR_FALSE = 0x0,
+ CRTC_MVP_STATUS_CRTC_FLIP_NOW_CLEAR_TRUE = 0x1,
+} CRTC_MVP_STATUS_CRTC_FLIP_NOW_CLEAR;
+typedef enum CRTC_MVP_STATUS_CRTC_AFR_HSYNC_SWITCH_DONE_CLEAR {
+ CRTC_MVP_STATUS_CRTC_AFR_HSYNC_SWITCH_DONE_CLEAR_FALSE= 0x0,
+ CRTC_MVP_STATUS_CRTC_AFR_HSYNC_SWITCH_DONE_CLEAR_TRUE= 0x1,
+} CRTC_MVP_STATUS_CRTC_AFR_HSYNC_SWITCH_DONE_CLEAR;
+typedef enum CRTC_V_UPDATE_INT_STATUS_CRTC_V_UPDATE_INT_CLEAR {
+ CRTC_V_UPDATE_INT_STATUS_CRTC_V_UPDATE_INT_CLEAR_FALSE= 0x0,
+ CRTC_V_UPDATE_INT_STATUS_CRTC_V_UPDATE_INT_CLEAR_TRUE= 0x1,
+} CRTC_V_UPDATE_INT_STATUS_CRTC_V_UPDATE_INT_CLEAR;
+typedef enum CRTC_VERTICAL_INTERRUPT0_CONTROL_CRTC_VERTICAL_INTERRUPT0_OUTPUT_POLARITY {
+ CRTC_VERTICAL_INTERRUPT0_CONTROL_CRTC_VERTICAL_INTERRUPT0_OUTPUT_POLARITY_FALSE= 0x0,
+ CRTC_VERTICAL_INTERRUPT0_CONTROL_CRTC_VERTICAL_INTERRUPT0_OUTPUT_POLARITY_TRUE= 0x1,
+} CRTC_VERTICAL_INTERRUPT0_CONTROL_CRTC_VERTICAL_INTERRUPT0_OUTPUT_POLARITY;
+typedef enum CRTC_VERTICAL_INTERRUPT0_CONTROL_CRTC_VERTICAL_INTERRUPT0_INT_ENABLE {
+ CRTC_VERTICAL_INTERRUPT0_CONTROL_CRTC_VERTICAL_INTERRUPT0_INT_ENABLE_FALSE= 0x0,
+ CRTC_VERTICAL_INTERRUPT0_CONTROL_CRTC_VERTICAL_INTERRUPT0_INT_ENABLE_TRUE= 0x1,
+} CRTC_VERTICAL_INTERRUPT0_CONTROL_CRTC_VERTICAL_INTERRUPT0_INT_ENABLE;
+typedef enum CRTC_VERTICAL_INTERRUPT0_CONTROL_CRTC_VERTICAL_INTERRUPT0_CLEAR {
+ CRTC_VERTICAL_INTERRUPT0_CONTROL_CRTC_VERTICAL_INTERRUPT0_CLEAR_FALSE= 0x0,
+ CRTC_VERTICAL_INTERRUPT0_CONTROL_CRTC_VERTICAL_INTERRUPT0_CLEAR_TRUE= 0x1,
+} CRTC_VERTICAL_INTERRUPT0_CONTROL_CRTC_VERTICAL_INTERRUPT0_CLEAR;
+typedef enum CRTC_VERTICAL_INTERRUPT0_CONTROL_CRTC_VERTICAL_INTERRUPT0_INT_TYPE {
+ CRTC_VERTICAL_INTERRUPT0_CONTROL_CRTC_VERTICAL_INTERRUPT0_INT_TYPE_FALSE= 0x0,
+ CRTC_VERTICAL_INTERRUPT0_CONTROL_CRTC_VERTICAL_INTERRUPT0_INT_TYPE_TRUE= 0x1,
+} CRTC_VERTICAL_INTERRUPT0_CONTROL_CRTC_VERTICAL_INTERRUPT0_INT_TYPE;
+typedef enum CRTC_VERTICAL_INTERRUPT1_CONTROL_CRTC_VERTICAL_INTERRUPT1_CLEAR {
+ CRTC_VERTICAL_INTERRUPT1_CONTROL_CRTC_VERTICAL_INTERRUPT1_CLEAR_CLEAR_FALSE= 0x0,
+ CRTC_VERTICAL_INTERRUPT1_CONTROL_CRTC_VERTICAL_INTERRUPT1_CLEAR_TRUE= 0x1,
+} CRTC_VERTICAL_INTERRUPT1_CONTROL_CRTC_VERTICAL_INTERRUPT1_CLEAR;
+typedef enum CRTC_VERTICAL_INTERRUPT1_CONTROL_CRTC_VERTICAL_INTERRUPT1_INT_ENABLE {
+ CRTC_VERTICAL_INTERRUPT1_CONTROL_CRTC_VERTICAL_INTERRUPT1_INT_ENABLE_FALSE= 0x0,
+ CRTC_VERTICAL_INTERRUPT1_CONTROL_CRTC_VERTICAL_INTERRUPT1_INT_ENABLE_TRUE= 0x1,
+} CRTC_VERTICAL_INTERRUPT1_CONTROL_CRTC_VERTICAL_INTERRUPT1_INT_ENABLE;
+typedef enum CRTC_VERTICAL_INTERRUPT1_CONTROL_CRTC_VERTICAL_INTERRUPT1_INT_TYPE {
+ CRTC_VERTICAL_INTERRUPT1_CONTROL_CRTC_VERTICAL_INTERRUPT1_INT_TYPE_FALSE= 0x0,
+ CRTC_VERTICAL_INTERRUPT1_CONTROL_CRTC_VERTICAL_INTERRUPT1_INT_TYPE_TRUE= 0x1,
+} CRTC_VERTICAL_INTERRUPT1_CONTROL_CRTC_VERTICAL_INTERRUPT1_INT_TYPE;
+typedef enum CRTC_VERTICAL_INTERRUPT2_CONTROL_CRTC_VERTICAL_INTERRUPT2_CLEAR {
+ CRTC_VERTICAL_INTERRUPT2_CONTROL_CRTC_VERTICAL_INTERRUPT2_CLEAR_CLEAR_FALSE= 0x0,
+ CRTC_VERTICAL_INTERRUPT2_CONTROL_CRTC_VERTICAL_INTERRUPT2_CLEAR_TRUE= 0x1,
+} CRTC_VERTICAL_INTERRUPT2_CONTROL_CRTC_VERTICAL_INTERRUPT2_CLEAR;
+typedef enum CRTC_VERTICAL_INTERRUPT2_CONTROL_CRTC_VERTICAL_INTERRUPT2_INT_ENABLE {
+ CRTC_VERTICAL_INTERRUPT2_CONTROL_CRTC_VERTICAL_INTERRUPT2_INT_ENABLE_FALSE= 0x0,
+ CRTC_VERTICAL_INTERRUPT2_CONTROL_CRTC_VERTICAL_INTERRUPT2_INT_ENABLE_TRUE= 0x1,
+} CRTC_VERTICAL_INTERRUPT2_CONTROL_CRTC_VERTICAL_INTERRUPT2_INT_ENABLE;
+typedef enum CRTC_VERTICAL_INTERRUPT2_CONTROL_CRTC_VERTICAL_INTERRUPT2_INT_TYPE {
+ CRTC_VERTICAL_INTERRUPT2_CONTROL_CRTC_VERTICAL_INTERRUPT2_INT_TYPE_FALSE= 0x0,
+ CRTC_VERTICAL_INTERRUPT2_CONTROL_CRTC_VERTICAL_INTERRUPT2_INT_TYPE_TRUE= 0x1,
+} CRTC_VERTICAL_INTERRUPT2_CONTROL_CRTC_VERTICAL_INTERRUPT2_INT_TYPE;
+typedef enum CRTC_CRC_CNTL_CRTC_CRC_EN {
+ CRTC_CRC_CNTL_CRTC_CRC_EN_FALSE = 0x0,
+ CRTC_CRC_CNTL_CRTC_CRC_EN_TRUE = 0x1,
+} CRTC_CRC_CNTL_CRTC_CRC_EN;
+typedef enum CRTC_CRC_CNTL_CRTC_CRC_CONT_EN {
+ CRTC_CRC_CNTL_CRTC_CRC_CONT_EN_FALSE = 0x0,
+ CRTC_CRC_CNTL_CRTC_CRC_CONT_EN_TRUE = 0x1,
+} CRTC_CRC_CNTL_CRTC_CRC_CONT_EN;
+typedef enum CRTC_CRC_CNTL_CRTC_CRC_STEREO_MODE {
+ CRTC_CRC_CNTL_CRTC_CRC_STEREO_MODE_LEFT = 0x0,
+ CRTC_CRC_CNTL_CRTC_CRC_STEREO_MODE_RIGHT = 0x1,
+ CRTC_CRC_CNTL_CRTC_CRC_STEREO_MODE_BOTH_EYES = 0x2,
+ CRTC_CRC_CNTL_CRTC_CRC_STEREO_MODE_BOTH_FIELDS = 0x3,
+} CRTC_CRC_CNTL_CRTC_CRC_STEREO_MODE;
+typedef enum CRTC_CRC_CNTL_CRTC_CRC_INTERLACE_MODE {
+ CRTC_CRC_CNTL_CRTC_CRC_INTERLACE_MODE_TOP = 0x0,
+ CRTC_CRC_CNTL_CRTC_CRC_INTERLACE_MODE_BOTTOM = 0x1,
+ CRTC_CRC_CNTL_CRTC_CRC_INTERLACE_MODE_BOTH_BOTTOM= 0x2,
+ CRTC_CRC_CNTL_CRTC_CRC_INTERLACE_MODE_BOTH_FIELD = 0x3,
+} CRTC_CRC_CNTL_CRTC_CRC_INTERLACE_MODE;
+typedef enum CRTC_CRC_CNTL_CRTC_CRC_USE_NEW_AND_REPEATED_PIXELS {
+ CRTC_CRC_CNTL_CRTC_CRC_USE_NEW_AND_REPEATED_PIXELS_FALSE= 0x0,
+ CRTC_CRC_CNTL_CRTC_CRC_USE_NEW_AND_REPEATED_PIXELS_TRUE= 0x1,
+} CRTC_CRC_CNTL_CRTC_CRC_USE_NEW_AND_REPEATED_PIXELS;
+typedef enum CRTC_CRC_CNTL_CRTC_CRTC_CRC0_SELECT {
+ CRTC_CRC_CNTL_CRTC_CRTC_CRC0_SELECT_UAB = 0x0,
+ CRTC_CRC_CNTL_CRTC_CRTC_CRC0_SELECT_UA_B = 0x1,
+ CRTC_CRC_CNTL_CRTC_CRTC_CRC0_SELECT_U_AB = 0x2,
+ CRTC_CRC_CNTL_CRTC_CRTC_CRC0_SELECT_U_A_B = 0x3,
+ CRTC_CRC_CNTL_CRTC_CRTC_CRC0_SELECT_IAB = 0x4,
+ CRTC_CRC_CNTL_CRTC_CRTC_CRC0_SELECT_IA_B = 0x5,
+ CRTC_CRC_CNTL_CRTC_CRTC_CRC0_SELECT_I_AB = 0x6,
+ CRTC_CRC_CNTL_CRTC_CRTC_CRC0_SELECT_I_A_B = 0x7,
+} CRTC_CRC_CNTL_CRTC_CRTC_CRC0_SELECT;
+typedef enum CRTC_CRC_CNTL_CRTC_CRTC_CRC1_SELECT {
+ CRTC_CRC_CNTL_CRTC_CRTC_CRC1_SELECT_UAB = 0x0,
+ CRTC_CRC_CNTL_CRTC_CRTC_CRC1_SELECT_UA_B = 0x1,
+ CRTC_CRC_CNTL_CRTC_CRTC_CRC1_SELECT_U_AB = 0x2,
+ CRTC_CRC_CNTL_CRTC_CRTC_CRC1_SELECT_U_A_B = 0x3,
+ CRTC_CRC_CNTL_CRTC_CRTC_CRC1_SELECT_IAB = 0x4,
+ CRTC_CRC_CNTL_CRTC_CRTC_CRC1_SELECT_IA_B = 0x5,
+ CRTC_CRC_CNTL_CRTC_CRTC_CRC1_SELECT_I_AB = 0x6,
+ CRTC_CRC_CNTL_CRTC_CRTC_CRC1_SELECT_I_A_B = 0x7,
+} CRTC_CRC_CNTL_CRTC_CRTC_CRC1_SELECT;
+typedef enum CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_ENABLE {
+ CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_ENABLE_DISABLE= 0x0,
+ CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_ENABLE_ONESHOT= 0x1,
+ CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_ENABLE_CONTINUOUS= 0x2,
+ CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_ENABLE_RESERVED= 0x3,
+} CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_ENABLE;
+typedef enum CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_HCOUNT_MODE_ENABLE {
+ CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_HCOUNT_MODE_ENABLE_FALSE= 0x0,
+ CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_HCOUNT_MODE_ENABLE_TRUE= 0x1,
+} CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_HCOUNT_MODE_ENABLE;
+typedef enum CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_JITTER_FILTERING_ENABLE {
+ CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_JITTER_FILTERING_ENABLE_FALSE= 0x0,
+ CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_JITTER_FILTERING_ENABLE_TRUE= 0x1,
+} CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_JITTER_FILTERING_ENABLE;
+typedef enum CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_JITTER_FILTERING_WINDOW {
+ CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_JITTER_FILTERING_WINDOW_1pixel= 0x0,
+ CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_JITTER_FILTERING_WINDOW_2pixel= 0x1,
+ CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_JITTER_FILTERING_WINDOW_3pixel= 0x2,
+ CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_JITTER_FILTERING_WINDOW_4pixel= 0x3,
+} CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_JITTER_FILTERING_WINDOW;
+typedef enum CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_WINDOW_ENABLE {
+ CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_WINDOW_ENABLE_FALSE= 0x0,
+ CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_WINDOW_ENABLE_TRUE= 0x1,
+} CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_WINDOW_ENABLE;
+typedef enum CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_WINDOW_UPDATE {
+ CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_WINDOW_UPDATE_FALSE= 0x0,
+ CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_WINDOW_UPDATE_TRUE= 0x1,
+} CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_WINDOW_UPDATE;
+typedef enum CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_VSYNC_POLARITY {
+ CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_VSYNC_POLARITY_FALSE= 0x0,
+ CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_VSYNC_POLARITY_TRUE= 0x1,
+} CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_VSYNC_POLARITY;
+typedef enum CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_HSYNC_POLARITY {
+ CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_HSYNC_POLARITY_FALSE= 0x0,
+ CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_HSYNC_POLARITY_TRUE= 0x1,
+} CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_HSYNC_POLARITY;
+typedef enum CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_INTERLACE_MODE {
+ CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_INTERLACE_MODE_FALSE= 0x0,
+ CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_INTERLACE_MODE_TRUE= 0x1,
+} CRTC_EXT_TIMING_SYNC_CONTROL_CRTC_EXT_TIMING_SYNC_INTERLACE_MODE;
+typedef enum CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_LOSS_INT_ENABLE {
+ CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_LOSS_INT_ENABLE_FALSE= 0x0,
+ CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_LOSS_INT_ENABLE_TRUE= 0x1,
+} CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_LOSS_INT_ENABLE;
+typedef enum CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_LOSS_CLEAR {
+ CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_LOSS_CLEAR_FALSE= 0x0,
+ CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_LOSS_CLEAR_TRUE= 0x1,
+} CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_LOSS_CLEAR;
+typedef enum CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_LOSS_INT_TYPE {
+ CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_LOSS_INT_TYPE_FALSE= 0x0,
+ CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_LOSS_INT_TYPE_TRUE= 0x1,
+} CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_LOSS_INT_TYPE;
+typedef enum CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_LOSS_FRAME_COUNT {
+ CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_LOSS_FRAME_COUNT_1FRAME= 0x0,
+ CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_LOSS_FRAME_COUNT_2FRAME= 0x1,
+ CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_LOSS_FRAME_COUNT_4FRAME= 0x2,
+ CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_LOSS_FRAME_COUNT_8FRAME= 0x3,
+ CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_LOSS_FRAME_COUNT_16FRAME= 0x4,
+ CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_LOSS_FRAME_COUNT_32FRAME= 0x5,
+ CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_LOSS_FRAME_COUNT_64FRAME= 0x6,
+ CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_LOSS_FRAME_COUNT_128FRAME= 0x7,
+} CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_LOSS_FRAME_COUNT;
+typedef enum CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_INT_ENABLE {
+ CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_INT_ENABLE_FALSE= 0x0,
+ CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_INT_ENABLE_TRUE= 0x1,
+} CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_INT_ENABLE;
+typedef enum CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_CLEAR {
+ CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_CLEAR_FALSE= 0x0,
+ CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_CLEAR_TRUE= 0x1,
+} CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_CLEAR;
+typedef enum CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_INT_TYPE {
+ CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_INT_TYPE_FALSE= 0x0,
+ CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_INT_TYPE_TRUE= 0x1,
+} CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_INT_TYPE;
+typedef enum CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_SIGNAL_INT_ENABLE {
+ CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_SIGNAL_INT_ENABLE_FALSE= 0x0,
+ CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_SIGNAL_INT_ENABLE_TRUE= 0x1,
+} CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_SIGNAL_INT_ENABLE;
+typedef enum CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_SIGNAL_CLEAR {
+ CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_SIGNAL_CLEAR_FALSE= 0x0,
+ CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_SIGNAL_CLEAR_TRUE= 0x1,
+} CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_SIGNAL_CLEAR;
+typedef enum CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_SIGNAL_INT_TYPE {
+ CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_SIGNAL_INT_TYPE_FALSE= 0x0,
+ CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_SIGNAL_INT_TYPE_TRUE= 0x1,
+} CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL_CRTC_EXT_TIMING_SYNC_SIGNAL_INT_TYPE;
+typedef enum CRTC_STATIC_SCREEN_CONTROL_CRTC_CPU_SS_INT_ENABLE {
+ CRTC_STATIC_SCREEN_CONTROL_CRTC_CPU_SS_INT_ENABLE_FALSE= 0x0,
+ CRTC_STATIC_SCREEN_CONTROL_CRTC_CPU_SS_INT_ENABLE_TRUE= 0x1,
+} CRTC_STATIC_SCREEN_CONTROL_CRTC_CPU_SS_INT_ENABLE;
+typedef enum CRTC_STATIC_SCREEN_CONTROL_CRTC_CPU_SS_INT_CLEAR {
+ CRTC_STATIC_SCREEN_CONTROL_CRTC_CPU_SS_INT_CLEAR_FALSE= 0x0,
+ CRTC_STATIC_SCREEN_CONTROL_CRTC_CPU_SS_INT_CLEAR_TRUE= 0x1,
+} CRTC_STATIC_SCREEN_CONTROL_CRTC_CPU_SS_INT_CLEAR;
+typedef enum CRTC_STATIC_SCREEN_CONTROL_CRTC_CPU_SS_INT_TYPE {
+ CRTC_STATIC_SCREEN_CONTROL_CRTC_CPU_SS_INT_TYPE_FALSE= 0x0,
+ CRTC_STATIC_SCREEN_CONTROL_CRTC_CPU_SS_INT_TYPE_TRUE= 0x1,
+} CRTC_STATIC_SCREEN_CONTROL_CRTC_CPU_SS_INT_TYPE;
+typedef enum CRTC_STATIC_SCREEN_CONTROL_CRTC_STATIC_SCREEN_OVERRIDE {
+ CRTC_STATIC_SCREEN_CONTROL_CRTC_STATIC_SCREEN_OVERRIDE_FALSE= 0x0,
+ CRTC_STATIC_SCREEN_CONTROL_CRTC_STATIC_SCREEN_OVERRIDE_TRUE= 0x1,
+} CRTC_STATIC_SCREEN_CONTROL_CRTC_STATIC_SCREEN_OVERRIDE;
+typedef enum CRTC_STATIC_SCREEN_CONTROL_CRTC_STATIC_SCREEN_OVERRIDE_VALUE {
+ CRTC_STATIC_SCREEN_CONTROL_CRTC_STATIC_SCREEN_OVERRIDE_VALUE_OFF= 0x0,
+ CRTC_STATIC_SCREEN_CONTROL_CRTC_STATIC_SCREEN_OVERRIDE_VALUE_ON= 0x1,
+} CRTC_STATIC_SCREEN_CONTROL_CRTC_STATIC_SCREEN_OVERRIDE_VALUE;
+typedef enum CRTC_3D_STRUCTURE_CONTROL_CRTC_3D_STRUCTURE_EN {
+ CRTC_3D_STRUCTURE_CONTROL_CRTC_3D_STRUCTURE_EN_FALSE= 0x0,
+ CRTC_3D_STRUCTURE_CONTROL_CRTC_3D_STRUCTURE_EN_TRUE= 0x1,
+} CRTC_3D_STRUCTURE_CONTROL_CRTC_3D_STRUCTURE_EN;
+typedef enum CRTC_3D_STRUCTURE_CONTROL_CRTC_3D_STRUCTURE_EN_DB {
+ CRTC_3D_STRUCTURE_CONTROL_CRTC_3D_STRUCTURE_EN_DB_FALSE= 0x0,
+ CRTC_3D_STRUCTURE_CONTROL_CRTC_3D_STRUCTURE_EN_DB_TRUE= 0x1,
+} CRTC_3D_STRUCTURE_CONTROL_CRTC_3D_STRUCTURE_EN_DB;
+typedef enum CRTC_3D_STRUCTURE_CONTROL_CRTC_3D_STRUCTURE_V_UPDATE_MODE {
+ CRTC_3D_STRUCTURE_CONTROL_CRTC_3D_STRUCTURE_V_UPDATE_MODE_BLOCK_BOTH= 0x0,
+ CRTC_3D_STRUCTURE_CONTROL_CRTC_3D_STRUCTURE_V_UPDATE_MODE_BLOCK_INTERLACE= 0x1,
+ CRTC_3D_STRUCTURE_CONTROL_CRTC_3D_STRUCTURE_V_UPDATE_MODE_BLOCK_PROGRASSIVE= 0x2,
+ CRTC_3D_STRUCTURE_CONTROL_CRTC_3D_STRUCTURE_V_UPDATE_MODE_RESERVED= 0x3,
+} CRTC_3D_STRUCTURE_CONTROL_CRTC_3D_STRUCTURE_V_UPDATE_MODE;
+typedef enum CRTC_3D_STRUCTURE_CONTROL_CRTC_3D_STRUCTURE_STEREO_SEL_OVR {
+ CRTC_3D_STRUCTURE_CONTROL_CRTC_3D_STRUCTURE_STEREO_SEL_OVR_FALSE= 0x0,
+ CRTC_3D_STRUCTURE_CONTROL_CRTC_3D_STRUCTURE_STEREO_SEL_OVR_TRUE= 0x1,
+} CRTC_3D_STRUCTURE_CONTROL_CRTC_3D_STRUCTURE_STEREO_SEL_OVR;
+typedef enum CRTC_V_SYNC_A_POL {
+ CRTC_V_SYNC_A_POL_HIGH = 0x0,
+ CRTC_V_SYNC_A_POL_LOW = 0x1,
+} CRTC_V_SYNC_A_POL;
+typedef enum CRTC_H_SYNC_A_POL {
+ CRTC_H_SYNC_A_POL_HIGH = 0x0,
+ CRTC_H_SYNC_A_POL_LOW = 0x1,
+} CRTC_H_SYNC_A_POL;
+typedef enum CRTC_HORZ_REPETITION_COUNT {
+ CRTC_HORZ_REPETITION_COUNT_0 = 0x0,
+ CRTC_HORZ_REPETITION_COUNT_1 = 0x1,
+ CRTC_HORZ_REPETITION_COUNT_2 = 0x2,
+ CRTC_HORZ_REPETITION_COUNT_3 = 0x3,
+ CRTC_HORZ_REPETITION_COUNT_4 = 0x4,
+ CRTC_HORZ_REPETITION_COUNT_5 = 0x5,
+ CRTC_HORZ_REPETITION_COUNT_6 = 0x6,
+ CRTC_HORZ_REPETITION_COUNT_7 = 0x7,
+ CRTC_HORZ_REPETITION_COUNT_8 = 0x8,
+ CRTC_HORZ_REPETITION_COUNT_9 = 0x9,
+ CRTC_HORZ_REPETITION_COUNT_10 = 0xa,
+ CRTC_HORZ_REPETITION_COUNT_11 = 0xb,
+ CRTC_HORZ_REPETITION_COUNT_12 = 0xc,
+ CRTC_HORZ_REPETITION_COUNT_13 = 0xd,
+ CRTC_HORZ_REPETITION_COUNT_14 = 0xe,
+ CRTC_HORZ_REPETITION_COUNT_15 = 0xf,
+} CRTC_HORZ_REPETITION_COUNT;
+typedef enum PERFCOUNTER_CVALUE_SEL {
+ PERFCOUNTER_CVALUE_SEL_47_0 = 0x0,
+ PERFCOUNTER_CVALUE_SEL_15_0 = 0x1,
+ PERFCOUNTER_CVALUE_SEL_31_16 = 0x2,
+ PERFCOUNTER_CVALUE_SEL_47_32 = 0x3,
+ PERFCOUNTER_CVALUE_SEL_11_0 = 0x4,
+ PERFCOUNTER_CVALUE_SEL_23_12 = 0x5,
+ PERFCOUNTER_CVALUE_SEL_35_24 = 0x6,
+ PERFCOUNTER_CVALUE_SEL_47_36 = 0x7,
+} PERFCOUNTER_CVALUE_SEL;
+typedef enum PERFCOUNTER_INC_MODE {
+ PERFCOUNTER_INC_MODE_MULTI_BIT = 0x0,
+ PERFCOUNTER_INC_MODE_BOTH_EDGE = 0x1,
+ PERFCOUNTER_INC_MODE_LSB = 0x2,
+ PERFCOUNTER_INC_MODE_POS_EDGE = 0x3,
+} PERFCOUNTER_INC_MODE;
+typedef enum PERFCOUNTER_HW_CNTL_SEL {
+ PERFCOUNTER_HW_CNTL_SEL_RUNEN = 0x0,
+ PERFCOUNTER_HW_CNTL_SEL_CNTOFF = 0x1,
+} PERFCOUNTER_HW_CNTL_SEL;
+typedef enum PERFCOUNTER_RUNEN_MODE {
+ PERFCOUNTER_RUNEN_MODE_LEVEL = 0x0,
+ PERFCOUNTER_RUNEN_MODE_EDGE = 0x1,
+} PERFCOUNTER_RUNEN_MODE;
+typedef enum PERFCOUNTER_CNTOFF_START_DIS {
+ PERFCOUNTER_CNTOFF_START_ENABLE = 0x0,
+ PERFCOUNTER_CNTOFF_START_DISABLE = 0x1,
+} PERFCOUNTER_CNTOFF_START_DIS;
+typedef enum PERFCOUNTER_RESTART_EN {
+ PERFCOUNTER_RESTART_DISABLE = 0x0,
+ PERFCOUNTER_RESTART_ENABLE = 0x1,
+} PERFCOUNTER_RESTART_EN;
+typedef enum PERFCOUNTER_INT_EN {
+ PERFCOUNTER_INT_DISABLE = 0x0,
+ PERFCOUNTER_INT_ENABLE = 0x1,
+} PERFCOUNTER_INT_EN;
+typedef enum PERFCOUNTER_OFF_MASK {
+ PERFCOUNTER_OFF_MASK_DISABLE = 0x0,
+ PERFCOUNTER_OFF_MASK_ENABLE = 0x1,
+} PERFCOUNTER_OFF_MASK;
+typedef enum PERFCOUNTER_ACTIVE {
+ PERFCOUNTER_IS_IDLE = 0x0,
+ PERFCOUNTER_IS_ACTIVE = 0x1,
+} PERFCOUNTER_ACTIVE;
+typedef enum PERFCOUNTER_INT_TYPE {
+ PERFCOUNTER_INT_TYPE_LEVEL = 0x0,
+ PERFCOUNTER_INT_TYPE_PULSE = 0x1,
+} PERFCOUNTER_INT_TYPE;
+typedef enum PERFCOUNTER_COUNTED_VALUE_TYPE {
+ PERFCOUNTER_COUNTED_VALUE_TYPE_ACC = 0x0,
+ PERFCOUNTER_COUNTED_VALUE_TYPE_MAX = 0x1,
+} PERFCOUNTER_COUNTED_VALUE_TYPE;
+typedef enum PERFCOUNTER_CNTL_SEL {
+ PERFCOUNTER_CNTL_SEL_0 = 0x0,
+ PERFCOUNTER_CNTL_SEL_1 = 0x1,
+ PERFCOUNTER_CNTL_SEL_2 = 0x2,
+ PERFCOUNTER_CNTL_SEL_3 = 0x3,
+ PERFCOUNTER_CNTL_SEL_4 = 0x4,
+ PERFCOUNTER_CNTL_SEL_5 = 0x5,
+ PERFCOUNTER_CNTL_SEL_6 = 0x6,
+ PERFCOUNTER_CNTL_SEL_7 = 0x7,
+} PERFCOUNTER_CNTL_SEL;
+typedef enum PERFCOUNTER_CNT0_STATE {
+ PERFCOUNTER_CNT0_STATE_RESET = 0x0,
+ PERFCOUNTER_CNT0_STATE_START = 0x1,
+ PERFCOUNTER_CNT0_STATE_FREEZE = 0x2,
+ PERFCOUNTER_CNT0_STATE_HW = 0x3,
+} PERFCOUNTER_CNT0_STATE;
+typedef enum PERFCOUNTER_STATE_SEL0 {
+ PERFCOUNTER_STATE_SEL0_GLOBAL = 0x0,
+ PERFCOUNTER_STATE_SEL0_LOCAL = 0x1,
+} PERFCOUNTER_STATE_SEL0;
+typedef enum PERFCOUNTER_CNT1_STATE {
+ PERFCOUNTER_CNT1_STATE_RESET = 0x0,
+ PERFCOUNTER_CNT1_STATE_START = 0x1,
+ PERFCOUNTER_CNT1_STATE_FREEZE = 0x2,
+ PERFCOUNTER_CNT1_STATE_HW = 0x3,
+} PERFCOUNTER_CNT1_STATE;
+typedef enum PERFCOUNTER_STATE_SEL1 {
+ PERFCOUNTER_STATE_SEL1_GLOBAL = 0x0,
+ PERFCOUNTER_STATE_SEL1_LOCAL = 0x1,
+} PERFCOUNTER_STATE_SEL1;
+typedef enum PERFCOUNTER_CNT2_STATE {
+ PERFCOUNTER_CNT2_STATE_RESET = 0x0,
+ PERFCOUNTER_CNT2_STATE_START = 0x1,
+ PERFCOUNTER_CNT2_STATE_FREEZE = 0x2,
+ PERFCOUNTER_CNT2_STATE_HW = 0x3,
+} PERFCOUNTER_CNT2_STATE;
+typedef enum PERFCOUNTER_STATE_SEL2 {
+ PERFCOUNTER_STATE_SEL2_GLOBAL = 0x0,
+ PERFCOUNTER_STATE_SEL2_LOCAL = 0x1,
+} PERFCOUNTER_STATE_SEL2;
+typedef enum PERFCOUNTER_CNT3_STATE {
+ PERFCOUNTER_CNT3_STATE_RESET = 0x0,
+ PERFCOUNTER_CNT3_STATE_START = 0x1,
+ PERFCOUNTER_CNT3_STATE_FREEZE = 0x2,
+ PERFCOUNTER_CNT3_STATE_HW = 0x3,
+} PERFCOUNTER_CNT3_STATE;
+typedef enum PERFCOUNTER_STATE_SEL3 {
+ PERFCOUNTER_STATE_SEL3_GLOBAL = 0x0,
+ PERFCOUNTER_STATE_SEL3_LOCAL = 0x1,
+} PERFCOUNTER_STATE_SEL3;
+typedef enum PERFCOUNTER_CNT4_STATE {
+ PERFCOUNTER_CNT4_STATE_RESET = 0x0,
+ PERFCOUNTER_CNT4_STATE_START = 0x1,
+ PERFCOUNTER_CNT4_STATE_FREEZE = 0x2,
+ PERFCOUNTER_CNT4_STATE_HW = 0x3,
+} PERFCOUNTER_CNT4_STATE;
+typedef enum PERFCOUNTER_STATE_SEL4 {
+ PERFCOUNTER_STATE_SEL4_GLOBAL = 0x0,
+ PERFCOUNTER_STATE_SEL4_LOCAL = 0x1,
+} PERFCOUNTER_STATE_SEL4;
+typedef enum PERFCOUNTER_CNT5_STATE {
+ PERFCOUNTER_CNT5_STATE_RESET = 0x0,
+ PERFCOUNTER_CNT5_STATE_START = 0x1,
+ PERFCOUNTER_CNT5_STATE_FREEZE = 0x2,
+ PERFCOUNTER_CNT5_STATE_HW = 0x3,
+} PERFCOUNTER_CNT5_STATE;
+typedef enum PERFCOUNTER_STATE_SEL5 {
+ PERFCOUNTER_STATE_SEL5_GLOBAL = 0x0,
+ PERFCOUNTER_STATE_SEL5_LOCAL = 0x1,
+} PERFCOUNTER_STATE_SEL5;
+typedef enum PERFCOUNTER_CNT6_STATE {
+ PERFCOUNTER_CNT6_STATE_RESET = 0x0,
+ PERFCOUNTER_CNT6_STATE_START = 0x1,
+ PERFCOUNTER_CNT6_STATE_FREEZE = 0x2,
+ PERFCOUNTER_CNT6_STATE_HW = 0x3,
+} PERFCOUNTER_CNT6_STATE;
+typedef enum PERFCOUNTER_STATE_SEL6 {
+ PERFCOUNTER_STATE_SEL6_GLOBAL = 0x0,
+ PERFCOUNTER_STATE_SEL6_LOCAL = 0x1,
+} PERFCOUNTER_STATE_SEL6;
+typedef enum PERFCOUNTER_CNT7_STATE {
+ PERFCOUNTER_CNT7_STATE_RESET = 0x0,
+ PERFCOUNTER_CNT7_STATE_START = 0x1,
+ PERFCOUNTER_CNT7_STATE_FREEZE = 0x2,
+ PERFCOUNTER_CNT7_STATE_HW = 0x3,
+} PERFCOUNTER_CNT7_STATE;
+typedef enum PERFCOUNTER_STATE_SEL7 {
+ PERFCOUNTER_STATE_SEL7_GLOBAL = 0x0,
+ PERFCOUNTER_STATE_SEL7_LOCAL = 0x1,
+} PERFCOUNTER_STATE_SEL7;
+typedef enum PERFMON_STATE {
+ PERFMON_STATE_RESET = 0x0,
+ PERFMON_STATE_START = 0x1,
+ PERFMON_STATE_FREEZE = 0x2,
+ PERFMON_STATE_HW = 0x3,
+} PERFMON_STATE;
+typedef enum PERFMON_CNTOFF_AND_OR {
+ PERFMON_CNTOFF_OR = 0x0,
+ PERFMON_CNTOFF_AND = 0x1,
+} PERFMON_CNTOFF_AND_OR;
+typedef enum PERFMON_CNTOFF_INT_EN {
+ PERFMON_CNTOFF_INT_DISABLE = 0x0,
+ PERFMON_CNTOFF_INT_ENABLE = 0x1,
+} PERFMON_CNTOFF_INT_EN;
+typedef enum PERFMON_CNTOFF_INT_TYPE {
+ PERFMON_CNTOFF_INT_TYPE_LEVEL = 0x0,
+ PERFMON_CNTOFF_INT_TYPE_PULSE = 0x1,
+} PERFMON_CNTOFF_INT_TYPE;
+typedef enum ENABLE {
+ DISABLE_THE_FEATURE = 0x0,
+ ENABLE_THE_FEATURE = 0x1,
+} ENABLE;
+typedef enum ENABLE_CLOCK {
+ DISABLE_THE_CLOCK = 0x0,
+ ENABLE_THE_CLOCK = 0x1,
+} ENABLE_CLOCK;
+typedef enum FORCE_VBI {
+ FORCE_VBI_LOW = 0x0,
+ FORCE_VBI_HIGH = 0x1,
+} FORCE_VBI;
+typedef enum OVERRIDE_CGTT_SCLK {
+ OVERRIDE_CGTT_SCLK_NOOP = 0x0,
+ SET_OVERRIDE_CGTT_SCLK = 0x1,
+} OVERRIDE_CGTT_SCLK;
+typedef enum CLEAR_SMU_INTR {
+ SMU_INTR_STATUS_NOOP = 0x0,
+ SMU_INTR_STATUS_CLEAR = 0x1,
+} CLEAR_SMU_INTR;
+typedef enum STATIC_SCREEN_SMU_INTR {
+ STATIC_SCREEN_SMU_INTR_NOOP = 0x0,
+ SET_STATIC_SCREEN_SMU_INTR = 0x1,
+} STATIC_SCREEN_SMU_INTR;
+typedef enum JITTER_REMOVE_DISABLE {
+ ENABLE_JITTER_REMOVAL = 0x0,
+ DISABLE_JITTER_REMOVAL = 0x1,
+} JITTER_REMOVE_DISABLE;
+typedef enum DISABLE_CLOCK_GATING {
+ CLOCK_GATING_ENABLED = 0x0,
+ CLOCK_GATING_DISABLED = 0x1,
+} DISABLE_CLOCK_GATING;
+typedef enum DISABLE_CLOCK_GATING_IN_DCO {
+ CLOCK_GATING_ENABLED_IN_DCO = 0x0,
+ CLOCK_GATING_DISABLED_IN_DCO = 0x1,
+} DISABLE_CLOCK_GATING_IN_DCO;
+typedef enum DCCG_DEEP_COLOR_CNTL {
+ DCCG_DEEP_COLOR_DTO_DISABLE = 0x0,
+ DCCG_DEEP_COLOR_DTO_5_4_RATIO = 0x1,
+ DCCG_DEEP_COLOR_DTO_3_2_RATIO = 0x2,
+ DCCG_DEEP_COLOR_DTO_2_1_RATIO = 0x3,
+} DCCG_DEEP_COLOR_CNTL;
+typedef enum REFCLK_CLOCK_EN {
+ REFCLK_CLOCK_EN_PCIE_REFCLK = 0x0,
+ REFCLK_CLOCK_EN_ALLOW_SRC = 0x1,
+} REFCLK_CLOCK_EN;
+typedef enum REFCLK_SRC_SEL {
+ REFCLK_SRC_SEL_XTALIN = 0x0,
+ REFCLK_SRC_SEL_DISPPLL = 0x1,
+} REFCLK_SRC_SEL;
+typedef enum DPREFCLK_SRC_SEL {
+ DPREFCLK_SRC_SEL_CK = 0x0,
+ DPREFCLK_SRC_SEL_P0PLL = 0x1,
+ DPREFCLK_SRC_SEL_P1PLL = 0x2,
+ DPREFCLK_SRC_SEL_P2PLL = 0x3,
+ DPREFCLK_SRC_SEL_P3PLL = 0x4,
+} DPREFCLK_SRC_SEL;
+typedef enum XTAL_REF_SEL {
+ XTAL_REF_SEL_1X = 0x0,
+ XTAL_REF_SEL_2X = 0x1,
+} XTAL_REF_SEL;
+typedef enum XTAL_REF_CLOCK_SOURCE_SEL {
+ XTAL_REF_CLOCK_SOURCE_SEL_XTALIN = 0x0,
+ XTAL_REF_CLOCK_SOURCE_SEL_PPLL = 0x1,
+} XTAL_REF_CLOCK_SOURCE_SEL;
+typedef enum MICROSECOND_TIME_BASE_CLOCK_SOURCE_SEL {
+ MICROSECOND_TIME_BASE_CLOCK_IS_XTALIN = 0x0,
+ MICROSECOND_TIME_BASE_CLOCK_IS_PPLL_REFCLK = 0x1,
+} MICROSECOND_TIME_BASE_CLOCK_SOURCE_SEL;
+typedef enum ALLOW_SR_ON_TRANS_REQ {
+ ALLOW_SR_ON_TRANS_REQ_ENABLE = 0x0,
+ ALLOW_SR_ON_TRANS_REQ_DISABLE = 0x1,
+} ALLOW_SR_ON_TRANS_REQ;
+typedef enum MILLISECOND_TIME_BASE_CLOCK_SOURCE_SEL {
+ MILLISECOND_TIME_BASE_CLOCK_IS_XTALIN = 0x0,
+ MILLISECOND_TIME_BASE_CLOCK_IS_PPLL_REFCLK = 0x1,
+} MILLISECOND_TIME_BASE_CLOCK_SOURCE_SEL;
+typedef enum PIPE_PIXEL_RATE_SOURCE {
+ PIPE_PIXEL_RATE_SOURCE_P0PLL = 0x0,
+ PIPE_PIXEL_RATE_SOURCE_P1PLL = 0x1,
+ PIPE_PIXEL_RATE_SOURCE_P2PLL = 0x2,
+} PIPE_PIXEL_RATE_SOURCE;
+typedef enum PIPE_PHYPLL_PIXEL_RATE_SOURCE {
+ PIPE_PHYPLL_PIXEL_RATE_SOURCE_UNIPHYA = 0x0,
+ PIPE_PHYPLL_PIXEL_RATE_SOURCE_UNIPHYB = 0x1,
+ PIPE_PHYPLL_PIXEL_RATE_SOURCE_UNIPHYC = 0x2,
+ PIPE_PHYPLL_PIXEL_RATE_SOURCE_UNIPHYD = 0x3,
+ PIPE_PHYPLL_PIXEL_RATE_SOURCE_UNIPHYE = 0x4,
+ PIPE_PHYPLL_PIXEL_RATE_SOURCE_UNIPHYF = 0x5,
+ PIPE_PHYPLL_PIXEL_RATE_SOURCE_UNIPHYG = 0x6,
+} PIPE_PHYPLL_PIXEL_RATE_SOURCE;
+typedef enum PIPE_PIXEL_RATE_PLL_SOURCE {
+ PIPE_PIXEL_RATE_PLL_SOURCE_PHYPLL = 0x0,
+ PIPE_PIXEL_RATE_PLL_SOURCE_DISPPLL = 0x1,
+} PIPE_PIXEL_RATE_PLL_SOURCE;
+typedef enum DP_DTO_DS_DISABLE {
+ DP_DTO_DESPREAD_DISABLE = 0x0,
+ DP_DTO_DESPREAD_ENABLE = 0x1,
+} DP_DTO_DS_DISABLE;
+typedef enum CRTC_ADD_PIXEL {
+ CRTC_ADD_PIXEL_NOOP = 0x0,
+ CRTC_ADD_PIXEL_FORCE = 0x1,
+} CRTC_ADD_PIXEL;
+typedef enum CRTC_DROP_PIXEL {
+ CRTC_DROP_PIXEL_NOOP = 0x0,
+ CRTC_DROP_PIXEL_FORCE = 0x1,
+} CRTC_DROP_PIXEL;
+typedef enum SYMCLK_FE_FORCE_EN {
+ SYMCLK_FE_FORCE_EN_DISABLE = 0x0,
+ SYMCLK_FE_FORCE_EN_ENABLE = 0x1,
+} SYMCLK_FE_FORCE_EN;
+typedef enum SYMCLK_FE_FORCE_SRC {
+ SYMCLK_FE_FORCE_SRC_UNIPHYA = 0x0,
+ SYMCLK_FE_FORCE_SRC_UNIPHYB = 0x1,
+ SYMCLK_FE_FORCE_SRC_UNIPHYC = 0x2,
+ SYMCLK_FE_FORCE_SRC_UNIPHYD = 0x3,
+ SYMCLK_FE_FORCE_SRC_UNIPHYE = 0x4,
+ SYMCLK_FE_FORCE_SRC_UNIPHYF = 0x5,
+ SYMCLK_FE_FORCE_SRC_UNIPHYG = 0x6,
+} SYMCLK_FE_FORCE_SRC;
+typedef enum DPDBG_CLK_FORCE_EN {
+ DPDBG_CLK_FORCE_EN_DISABLE = 0x0,
+ DPDBG_CLK_FORCE_EN_ENABLE = 0x1,
+} DPDBG_CLK_FORCE_EN;
+typedef enum DVOACLK_COARSE_SKEW_CNTL {
+ DVOACLK_COARSE_SKEW_CNTL_NO_ADJUSTMENT = 0x0,
+ DVOACLK_COARSE_SKEW_CNTL_DELAY_1_STEP = 0x1,
+ DVOACLK_COARSE_SKEW_CNTL_DELAY_2_STEPS = 0x2,
+ DVOACLK_COARSE_SKEW_CNTL_DELAY_3_STEPS = 0x3,
+ DVOACLK_COARSE_SKEW_CNTL_DELAY_4_STEPS = 0x4,
+ DVOACLK_COARSE_SKEW_CNTL_DELAY_5_STEPS = 0x5,
+ DVOACLK_COARSE_SKEW_CNTL_DELAY_6_STEPS = 0x6,
+ DVOACLK_COARSE_SKEW_CNTL_DELAY_7_STEPS = 0x7,
+ DVOACLK_COARSE_SKEW_CNTL_DELAY_8_STEPS = 0x8,
+ DVOACLK_COARSE_SKEW_CNTL_DELAY_9_STEPS = 0x9,
+ DVOACLK_COARSE_SKEW_CNTL_DELAY_10_STEPS = 0xa,
+ DVOACLK_COARSE_SKEW_CNTL_DELAY_11_STEPS = 0xb,
+ DVOACLK_COARSE_SKEW_CNTL_DELAY_12_STEPS = 0xc,
+ DVOACLK_COARSE_SKEW_CNTL_DELAY_13_STEPS = 0xd,
+ DVOACLK_COARSE_SKEW_CNTL_DELAY_14_STEPS = 0xe,
+ DVOACLK_COARSE_SKEW_CNTL_DELAY_15_STEPS = 0xf,
+ DVOACLK_COARSE_SKEW_CNTL_EARLY_1_STEP = 0x10,
+ DVOACLK_COARSE_SKEW_CNTL_EARLY_2_STEPS = 0x11,
+ DVOACLK_COARSE_SKEW_CNTL_EARLY_3_STEPS = 0x12,
+ DVOACLK_COARSE_SKEW_CNTL_EARLY_4_STEPS = 0x13,
+ DVOACLK_COARSE_SKEW_CNTL_EARLY_5_STEPS = 0x14,
+ DVOACLK_COARSE_SKEW_CNTL_EARLY_6_STEPS = 0x15,
+ DVOACLK_COARSE_SKEW_CNTL_EARLY_7_STEPS = 0x16,
+ DVOACLK_COARSE_SKEW_CNTL_EARLY_8_STEPS = 0x17,
+ DVOACLK_COARSE_SKEW_CNTL_EARLY_9_STEPS = 0x18,
+ DVOACLK_COARSE_SKEW_CNTL_EARLY_10_STEPS = 0x19,
+ DVOACLK_COARSE_SKEW_CNTL_EARLY_11_STEPS = 0x1a,
+ DVOACLK_COARSE_SKEW_CNTL_EARLY_12_STEPS = 0x1b,
+ DVOACLK_COARSE_SKEW_CNTL_EARLY_13_STEPS = 0x1c,
+ DVOACLK_COARSE_SKEW_CNTL_EARLY_14_STEPS = 0x1d,
+ DVOACLK_COARSE_SKEW_CNTL_EARLY_15_STEPS = 0x1e,
+} DVOACLK_COARSE_SKEW_CNTL;
+typedef enum DVOACLK_FINE_SKEW_CNTL {
+ DVOACLK_FINE_SKEW_CNTL_NO_ADJUSTMENT = 0x0,
+ DVOACLK_FINE_SKEW_CNTL_DELAY_1_STEP = 0x1,
+ DVOACLK_FINE_SKEW_CNTL_DELAY_2_STEPS = 0x2,
+ DVOACLK_FINE_SKEW_CNTL_DELAY_3_STEPS = 0x3,
+ DVOACLK_FINE_SKEW_CNTL_EARLY_1_STEP = 0x4,
+ DVOACLK_FINE_SKEW_CNTL_EARLY_2_STEPS = 0x5,
+ DVOACLK_FINE_SKEW_CNTL_EARLY_3_STEPS = 0x6,
+ DVOACLK_FINE_SKEW_CNTL_EARLY_4_STEPS = 0x7,
+} DVOACLK_FINE_SKEW_CNTL;
+typedef enum DVOACLKD_IN_PHASE {
+ DVOACLKD_IN_OPPOSITE_PHASE_WITH_PCLK_DVO = 0x0,
+ DVOACLKD_IN_PHASE_WITH_PCLK_DVO = 0x1,
+} DVOACLKD_IN_PHASE;
+typedef enum DVOACLKC_IN_PHASE {
+ DVOACLKC_IN_OPPOSITE_PHASE_WITH_PCLK_DVO = 0x0,
+ DVOACLKC_IN_PHASE_WITH_PCLK_DVO = 0x1,
+} DVOACLKC_IN_PHASE;
+typedef enum DVOACLKC_MVP_IN_PHASE {
+ DVOACLKC_MVP_IN_OPPOSITE_PHASE_WITH_PCLK_DVO = 0x0,
+ DVOACLKC_MVP_IN_PHASE_WITH_PCLK_DVO = 0x1,
+} DVOACLKC_MVP_IN_PHASE;
+typedef enum DVOACLKC_MVP_SKEW_PHASE_OVERRIDE {
+ DVOACLKC_MVP_SKEW_PHASE_OVERRIDE_DISABLE = 0x0,
+ DVOACLKC_MVP_SKEW_PHASE_OVERRIDE_ENABLE = 0x1,
+} DVOACLKC_MVP_SKEW_PHASE_OVERRIDE;
+typedef enum MVP_CLK_SRC_SEL {
+ MVP_CLK_SRC_SEL_RSRV = 0x0,
+ MVP_CLK_SRC_SEL_IO_1 = 0x1,
+ MVP_CLK_SRC_SEL_IO_2 = 0x2,
+ MVP_CLK_SRC_SEL_REFCLK = 0x3,
+} MVP_CLK_SRC_SEL;
+typedef enum DCCG_AUDIO_DTO0_SOURCE_SEL {
+ DCCG_AUDIO_DTO0_SOURCE_SEL_CRTC0 = 0x0,
+ DCCG_AUDIO_DTO0_SOURCE_SEL_CRTC1 = 0x1,
+ DCCG_AUDIO_DTO0_SOURCE_SEL_CRTC2 = 0x2,
+ DCCG_AUDIO_DTO0_SOURCE_SEL_CRTC3 = 0x3,
+ DCCG_AUDIO_DTO0_SOURCE_SEL_CRTC4 = 0x4,
+ DCCG_AUDIO_DTO0_SOURCE_SEL_CRTC5 = 0x5,
+ DCCG_AUDIO_DTO0_SOURCE_SEL_RESERVED = 0x6,
+} DCCG_AUDIO_DTO0_SOURCE_SEL;
+typedef enum DCCG_AUDIO_DTO_SEL {
+ DCCG_AUDIO_DTO_SEL_AUDIO_DTO0 = 0x0,
+ DCCG_AUDIO_DTO_SEL_AUDIO_DTO1 = 0x1,
+ DCCG_AUDIO_DTO_SEL_NO_AUDIO_DTO = 0x2,
+} DCCG_AUDIO_DTO_SEL;
+typedef enum DCCG_AUDIO_DTO2_SOURCE_SEL {
+ DCCG_AUDIO_DTO2_SOURCE_SEL_AMCLK0 = 0x0,
+ DCCG_AUDIO_DTO2_SOURCE_SEL_AMCLK1 = 0x1,
+} DCCG_AUDIO_DTO2_SOURCE_SEL;
+typedef enum DCCG_AUDIO_DTO_USE_512FBR_DTO {
+ DCCG_AUDIO_DTO_USE_128FBR_FOR_DP = 0x0,
+ DCCG_AUDIO_DTO_USE_512FBR_FOR_DP = 0x1,
+} DCCG_AUDIO_DTO_USE_512FBR_DTO;
+typedef enum DCCG_DBG_EN {
+ DCCG_DBG_EN_DISABLE = 0x0,
+ DCCG_DBG_EN_ENABLE = 0x1,
+} DCCG_DBG_EN;
+typedef enum DCCG_DBG_BLOCK_SEL {
+ DCCG_DBG_BLOCK_SEL_DCCG = 0x0,
+ DCCG_DBG_BLOCK_SEL_PMON = 0x1,
+ DCCG_DBG_BLOCK_SEL_PMON2 = 0x2,
+} DCCG_DBG_BLOCK_SEL;
+typedef enum DCCG_DBG_CLOCK_SEL {
+ DCCG_DBG_CLOCK_SEL_DISPCLK = 0x0,
+ DCCG_DBG_CLOCK_SEL_SCLK = 0x1,
+ DCCG_DBG_CLOCK_SEL_MVPCLK = 0x2,
+ DCCG_DBG_CLOCK_SEL_DVOCLK = 0x3,
+ DCCG_DBG_CLOCK_SEL_DACCLK = 0x4,
+ DCCG_DBG_CLOCK_SEL_REFCLK = 0x5,
+ DCCG_DBG_CLOCK_SEL_SYMCLKA = 0x6,
+ DCCG_DBG_CLOCK_SEL_SYMCLKB = 0x7,
+ DCCG_DBG_CLOCK_SEL_SYMCLKC = 0x8,
+ DCCG_DBG_CLOCK_SEL_SYMCLKD = 0x9,
+ DCCG_DBG_CLOCK_SEL_SYMCLKE = 0xa,
+ DCCG_DBG_CLOCK_SEL_SYMCLKG = 0xb,
+ DCCG_DBG_CLOCK_SEL_SYMCLKF = 0xc,
+ DCCG_DBG_CLOCK_SEL_RSRV = 0xd,
+ DCCG_DBG_CLOCK_SEL_AOMCLK0 = 0xe,
+ DCCG_DBG_CLOCK_SEL_AOMCLK1 = 0xf,
+ DCCG_DBG_CLOCK_SEL_AOMCLK2 = 0x10,
+ DCCG_DBG_CLOCK_SEL_DPREFCLK = 0x11,
+ DCCG_DBG_CLOCK_SEL_UNB_DB_CLK = 0x12,
+ DCCG_DBG_CLOCK_SEL_DSICLK = 0x13,
+ DCCG_DBG_CLOCK_SEL_BYTECLK = 0x14,
+ DCCG_DBG_CLOCK_SEL_ESCCLK = 0x15,
+ DCCG_DBG_CLOCK_SEL_SYMCLKLPA = 0x16,
+ DCCG_DBG_CLOCK_SEL_SYMCLKLPB = 0x17,
+} DCCG_DBG_CLOCK_SEL;
+typedef enum DCCG_DBG_OUT_BLOCK_SEL {
+ DCCG_DBG_OUT_BLOCK_SEL_DCCG = 0x0,
+ DCCG_DBG_OUT_BLOCK_SEL_DCO = 0x1,
+ DCCG_DBG_OUT_BLOCK_SEL_DCIO = 0x2,
+ DCCG_DBG_OUT_BLOCK_SEL_DSI = 0x3,
+} DCCG_DBG_OUT_BLOCK_SEL;
+typedef enum DISPCLK_FREQ_RAMP_DONE {
+ DISPCLK_FREQ_RAMP_IN_PROGRESS = 0x0,
+ DISPCLK_FREQ_RAMP_COMPLETED = 0x1,
+} DISPCLK_FREQ_RAMP_DONE;
+typedef enum DCCG_FIFO_ERRDET_RESET {
+ DCCG_FIFO_ERRDET_RESET_NOOP = 0x0,
+ DCCG_FIFO_ERRDET_RESET_FORCE = 0x1,
+} DCCG_FIFO_ERRDET_RESET;
+typedef enum DCCG_FIFO_ERRDET_STATE {
+ DCCG_FIFO_ERRDET_STATE_DETECTION = 0x0,
+ DCCG_FIFO_ERRDET_STATE_CALIBRATION = 0x1,
+} DCCG_FIFO_ERRDET_STATE;
+typedef enum DCCG_FIFO_ERRDET_OVR_EN {
+ DCCG_FIFO_ERRDET_OVR_DISABLE = 0x0,
+ DCCG_FIFO_ERRDET_OVR_ENABLE = 0x1,
+} DCCG_FIFO_ERRDET_OVR_EN;
+typedef enum DISPCLK_CHG_FWD_CORR_DISABLE {
+ DISPCLK_CHG_FWD_CORR_ENABLE_AT_BEGINNING = 0x0,
+ DISPCLK_CHG_FWD_CORR_DISABLE_AT_BEGINNING = 0x1,
+} DISPCLK_CHG_FWD_CORR_DISABLE;
+typedef enum DC_MEM_GLOBAL_PWR_REQ_DIS {
+ DC_MEM_GLOBAL_PWR_REQ_ENABLE = 0x0,
+ DC_MEM_GLOBAL_PWR_REQ_DISABLE = 0x1,
+} DC_MEM_GLOBAL_PWR_REQ_DIS;
+typedef enum DCCG_PERF_RUN {
+ DCCG_PERF_RUN_NOOP = 0x0,
+ DCCG_PERF_RUN_START = 0x1,
+} DCCG_PERF_RUN;
+typedef enum DCCG_PERF_MODE_VSYNC {
+ DCCG_PERF_MODE_VSYNC_NOOP = 0x0,
+ DCCG_PERF_MODE_VSYNC_START = 0x1,
+} DCCG_PERF_MODE_VSYNC;
+typedef enum DCCG_PERF_MODE_HSYNC {
+ DCCG_PERF_MODE_HSYNC_NOOP = 0x0,
+ DCCG_PERF_MODE_HSYNC_START = 0x1,
+} DCCG_PERF_MODE_HSYNC;
+typedef enum DCCG_PERF_CRTC_SELECT {
+ DCCG_PERF_SEL_CRTC0 = 0x0,
+ DCCG_PERF_SEL_CRTC1 = 0x1,
+ DCCG_PERF_SEL_CRTC2 = 0x2,
+ DCCG_PERF_SEL_CRTC3 = 0x3,
+ DCCG_PERF_SEL_CRTC4 = 0x4,
+ DCCG_PERF_SEL_CRTC5 = 0x5,
+} DCCG_PERF_CRTC_SELECT;
+typedef enum CLOCK_BRANCH_SOFT_RESET {
+ CLOCK_BRANCH_SOFT_RESET_NOOP = 0x0,
+ CLOCK_BRANCH_SOFT_RESET_FORCE = 0x1,
+} CLOCK_BRANCH_SOFT_RESET;
+typedef enum PLL_CFG_IF_SOFT_RESET {
+ PLL_CFG_IF_SOFT_RESET_NOOP = 0x0,
+ PLL_CFG_IF_SOFT_RESET_FORCE = 0x1,
+} PLL_CFG_IF_SOFT_RESET;
+typedef enum DVO_ENABLE_RST {
+ DVO_ENABLE_RST_DISABLE = 0x0,
+ DVO_ENABLE_RST_ENABLE = 0x1,
+} DVO_ENABLE_RST;
+typedef enum LptNumBanks {
+ LPT_NUM_BANKS_2BANK = 0x0,
+ LPT_NUM_BANKS_4BANK = 0x1,
+ LPT_NUM_BANKS_8BANK = 0x2,
+ LPT_NUM_BANKS_16BANK = 0x3,
+ LPT_NUM_BANKS_32BANK = 0x4,
+} LptNumBanks;
+typedef enum DCIO_DC_GENERICA_SEL {
+ DCIO_GENERICA_SEL_DACA_STEREOSYNC = 0x0,
+ DCIO_GENERICA_SEL_STEREOSYNC = 0x1,
+ DCIO_GENERICA_SEL_DACA_PIXCLK = 0x2,
+ DCIO_GENERICA_SEL_DACB_PIXCLK = 0x3,
+ DCIO_GENERICA_SEL_DVOA_CTL3 = 0x4,
+ DCIO_GENERICA_SEL_P1_PLLCLK = 0x5,
+ DCIO_GENERICA_SEL_P2_PLLCLK = 0x6,
+ DCIO_GENERICA_SEL_DVOA_STEREOSYNC = 0x7,
+ DCIO_GENERICA_SEL_DACA_FIELD_NUMBER = 0x8,
+ DCIO_GENERICA_SEL_DACB_FIELD_NUMBER = 0x9,
+ DCIO_GENERICA_SEL_GENERICA_DCCG = 0xa,
+ DCIO_GENERICA_SEL_SYNCEN = 0xb,
+ DCIO_GENERICA_SEL_GENERICA_SCG = 0xc,
+ DCIO_GENERICA_SEL_RESERVED_VALUE13 = 0xd,
+ DCIO_GENERICA_SEL_RESERVED_VALUE14 = 0xe,
+ DCIO_GENERICA_SEL_RESERVED_VALUE15 = 0xf,
+ DCIO_GENERICA_SEL_GENERICA_DPRX = 0x10,
+ DCIO_GENERICA_SEL_GENERICB_DPRX = 0x11,
+} DCIO_DC_GENERICA_SEL;
+typedef enum DCIO_DC_GENERIC_UNIPHY_REFDIV_CLK_SEL {
+ DCIO_UNIPHYA_TEST_REFDIV_CLK = 0x0,
+ DCIO_UNIPHYB_TEST_REFDIV_CLK = 0x1,
+ DCIO_UNIPHYC_TEST_REFDIV_CLK = 0x2,
+ DCIO_UNIPHYD_TEST_REFDIV_CLK = 0x3,
+ DCIO_UNIPHYE_TEST_REFDIV_CLK = 0x4,
+ DCIO_UNIPHYF_TEST_REFDIV_CLK = 0x5,
+ DCIO_UNIPHYG_TEST_REFDIV_CLK = 0x6,
+ DCIO_UNIPHYLPA_TEST_REFDIV_CLK = 0x7,
+ DCIO_UNIPHYLPB_TEST_REFDIV_CLK = 0x8,
+} DCIO_DC_GENERIC_UNIPHY_REFDIV_CLK_SEL;
+typedef enum DCIO_DC_GENERIC_UNIPHY_FBDIV_CLK_SEL {
+ DCIO_UNIPHYA_FBDIV_CLK = 0x0,
+ DCIO_UNIPHYB_FBDIV_CLK = 0x1,
+ DCIO_UNIPHYC_FBDIV_CLK = 0x2,
+ DCIO_UNIPHYD_FBDIV_CLK = 0x3,
+ DCIO_UNIPHYE_FBDIV_CLK = 0x4,
+ DCIO_UNIPHYF_FBDIV_CLK = 0x5,
+ DCIO_UNIPHYG_FBDIV_CLK = 0x6,
+ DCIO_UNIPHYLPA_FBDIV_CLK = 0x7,
+ DCIO_UNIPHYLPB_FBDIV_CLK = 0x8,
+} DCIO_DC_GENERIC_UNIPHY_FBDIV_CLK_SEL;
+typedef enum DCIO_DC_GENERIC_UNIPHY_FBDIV_SSC_CLK_SEL {
+ DCIO_UNIPHYA_FBDIV_SSC_CLK = 0x0,
+ DCIO_UNIPHYB_FBDIV_SSC_CLK = 0x1,
+ DCIO_UNIPHYC_FBDIV_SSC_CLK = 0x2,
+ DCIO_UNIPHYD_FBDIV_SSC_CLK = 0x3,
+ DCIO_UNIPHYE_FBDIV_SSC_CLK = 0x4,
+ DCIO_UNIPHYF_FBDIV_SSC_CLK = 0x5,
+ DCIO_UNIPHYG_FBDIV_SSC_CLK = 0x6,
+ DCIO_UNIPHYLPA_FBDIV_SSC_CLK = 0x7,
+ DCIO_UNIPHYLPB_FBDIV_SSC_CLK = 0x8,
+} DCIO_DC_GENERIC_UNIPHY_FBDIV_SSC_CLK_SEL;
+typedef enum DCIO_DC_GENERIC_UNIPHY_FBDIV_CLK_DIV2_SEL {
+ DCIO_UNIPHYA_TEST_FBDIV_CLK_DIV2 = 0x0,
+ DCIO_UNIPHYB_TEST_FBDIV_CLK_DIV2 = 0x1,
+ DCIO_UNIPHYC_TEST_FBDIV_CLK_DIV2 = 0x2,
+ DCIO_UNIPHYD_TEST_FBDIV_CLK_DIV2 = 0x3,
+ DCIO_UNIPHYE_TEST_FBDIV_CLK_DIV2 = 0x4,
+ DCIO_UNIPHYF_TEST_FBDIV_CLK_DIV2 = 0x5,
+ DCIO_UNIPHYG_TEST_FBDIV_CLK_DIV2 = 0x6,
+ DCIO_UNIPHYLPA_TEST_FBDIV_CLK_DIV2 = 0x7,
+ DCIO_UNIPHYLPB_TEST_FBDIV_CLK_DIV2 = 0x8,
+} DCIO_DC_GENERIC_UNIPHY_FBDIV_CLK_DIV2_SEL;
+typedef enum DCIO_DC_GENERICB_SEL {
+ DCIO_GENERICB_SEL_DACA_STEREOSYNC = 0x0,
+ DCIO_GENERICB_SEL_STEREOSYNC = 0x1,
+ DCIO_GENERICB_SEL_DACA_PIXCLK = 0x2,
+ DCIO_GENERICB_SEL_DACB_PIXCLK = 0x3,
+ DCIO_GENERICB_SEL_DVOA_CTL3 = 0x4,
+ DCIO_GENERICB_SEL_P1_PLLCLK = 0x5,
+ DCIO_GENERICB_SEL_P2_PLLCLK = 0x6,
+ DCIO_GENERICB_SEL_DVOA_STEREOSYNC = 0x7,
+ DCIO_GENERICB_SEL_DACA_FIELD_NUMBER = 0x8,
+ DCIO_GENERICB_SEL_DACB_FIELD_NUMBER = 0x9,
+ DCIO_GENERICB_SEL_GENERICB_DCCG = 0xa,
+ DCIO_GENERICB_SEL_SYNCEN = 0xb,
+ DCIO_GENERICB_SEL_GENERICA_SCG = 0xc,
+ DCIO_GENERICB_SEL_RESERVED_VALUE13 = 0xd,
+ DCIO_GENERICB_SEL_RESERVED_VALUE14 = 0xe,
+ DCIO_GENERICB_SEL_RESERVED_VALUE15 = 0xf,
+} DCIO_DC_GENERICB_SEL;
+typedef enum DCIO_DC_PAD_EXTERN_SIG_SEL {
+ DCIO_DC_PAD_EXTERN_SIG_SEL_MVP = 0x0,
+ DCIO_DC_PAD_EXTERN_SIG_SEL_VSYNCA = 0x1,
+ DCIO_DC_PAD_EXTERN_SIG_SEL_GENLK_CLK = 0x2,
+ DCIO_DC_PAD_EXTERN_SIG_SEL_GENLK_VSYNC = 0x3,
+ DCIO_DC_PAD_EXTERN_SIG_SEL_GENERICA = 0x4,
+ DCIO_DC_PAD_EXTERN_SIG_SEL_GENERICB = 0x5,
+ DCIO_DC_PAD_EXTERN_SIG_SEL_GENERICC = 0x6,
+ DCIO_DC_PAD_EXTERN_SIG_SEL_HPD1 = 0x7,
+ DCIO_DC_PAD_EXTERN_SIG_SEL_HPD2 = 0x8,
+ DCIO_DC_PAD_EXTERN_SIG_SEL_DDC1CLK = 0x9,
+ DCIO_DC_PAD_EXTERN_SIG_SEL_DDC1DATA = 0xa,
+ DCIO_DC_PAD_EXTERN_SIG_SEL_DDC2CLK = 0xb,
+ DCIO_DC_PAD_EXTERN_SIG_SEL_DDC2DATA = 0xc,
+ DCIO_DC_PAD_EXTERN_SIG_SEL_VHAD1 = 0xd,
+ DCIO_DC_PAD_EXTERN_SIG_SEL_VHAD0 = 0xe,
+ DCIO_DC_PAD_EXTERN_SIG_SEL_VPHCTL = 0xf,
+} DCIO_DC_PAD_EXTERN_SIG_SEL;
+typedef enum DCIO_DC_PAD_EXTERN_SIG_MVP_PIXEL_SRC_STATUS {
+ DCIO_MVP_PIXEL_SRC_STATUS_HSYNCA = 0x0,
+ DCIO_MVP_PIXEL_SRC_STATUS_HSYNCA_DUPLICATE = 0x1,
+ DCIO_MVP_PIXEL_SRC_STATUS_CRTC = 0x2,
+ DCIO_MVP_PIXEL_SRC_STATUS_LB = 0x3,
+} DCIO_DC_PAD_EXTERN_SIG_MVP_PIXEL_SRC_STATUS;
+typedef enum DCIO_DC_REF_CLK_CNTL_HSYNCA_OUTPUT_SEL {
+ DCIO_HSYNCA_OUTPUT_SEL_DISABLE = 0x0,
+ DCIO_HSYNCA_OUTPUT_SEL_PPLL1 = 0x1,
+ DCIO_HSYNCA_OUTPUT_SEL_PPLL2 = 0x2,
+ DCIO_HSYNCA_OUTPUT_SEL_RESERVED = 0x3,
+} DCIO_DC_REF_CLK_CNTL_HSYNCA_OUTPUT_SEL;
+typedef enum DCIO_DC_REF_CLK_CNTL_GENLK_CLK_OUTPUT_SEL {
+ DCIO_GENLK_CLK_OUTPUT_SEL_DISABLE = 0x0,
+ DCIO_GENLK_CLK_OUTPUT_SEL_PPLL1 = 0x1,
+ DCIO_GENLK_CLK_OUTPUT_SEL_PPLL2 = 0x2,
+ DCIO_GENLK_CLK_OUTPUT_SEL_RESERVED_VALUE3 = 0x3,
+} DCIO_DC_REF_CLK_CNTL_GENLK_CLK_OUTPUT_SEL;
+typedef enum DCIO_DC_GPIO_VIP_DEBUG {
+ DCIO_DC_GPIO_VIP_DEBUG_NORMAL = 0x0,
+ DCIO_DC_GPIO_VIP_DEBUG_CG_BIG = 0x1,
+} DCIO_DC_GPIO_VIP_DEBUG;
+typedef enum DCIO_DC_GPIO_MACRO_DEBUG {
+ DCIO_DC_GPIO_MACRO_DEBUG_NORMAL = 0x0,
+ DCIO_DC_GPIO_MACRO_DEBUG_CHIP_BIF = 0x1,
+ DCIO_DC_GPIO_MACRO_DEBUG_RESERVED_VALUE2 = 0x2,
+ DCIO_DC_GPIO_MACRO_DEBUG_RESERVED_VALUE3 = 0x3,
+} DCIO_DC_GPIO_MACRO_DEBUG;
+typedef enum DCIO_DC_GPIO_CHIP_DEBUG_OUT_PIN_SEL {
+ DCIO_DC_GPIO_CHIP_DEBUG_OUT_PIN_SEL_NORMAL = 0x0,
+ DCIO_DC_GPIO_CHIP_DEBUG_OUT_PIN_SEL_SWAP = 0x1,
+} DCIO_DC_GPIO_CHIP_DEBUG_OUT_PIN_SEL;
+typedef enum DCIO_DC_GPIO_DEBUG_BUS_FLOP_EN {
+ DCIO_DC_GPIO_DEBUG_BUS_FLOP_EN_BYPASS = 0x0,
+ DCIO_DC_GPIO_DEBUG_BUS_FLOP_EN_ENABLE = 0x1,
+} DCIO_DC_GPIO_DEBUG_BUS_FLOP_EN;
+typedef enum DCIO_DC_GPIO_DEBUG_DPRX_LOOPBACK_ENABLE {
+ DCIO_DPRX_LOOPBACK_ENABLE_NORMAL = 0x0,
+ DCIO_DPRX_LOOPBACK_ENABLE_LOOP = 0x1,
+} DCIO_DC_GPIO_DEBUG_DPRX_LOOPBACK_ENABLE;
+typedef enum DCIO_UNIPHY_LINK_CNTL_MINIMUM_PIXVLD_LOW_DURATION {
+ DCIO_UNIPHY_MINIMUM_PIXVLD_LOW_DURATION_3_CLOCKS = 0x0,
+ DCIO_UNIPHY_MINIMUM_PIXVLD_LOW_DURATION_7_CLOCKS = 0x1,
+ DCIO_UNIPHY_MINIMUM_PIXVLD_LOW_DURATION_11_CLOCKS= 0x2,
+ DCIO_UNIPHY_MINIMUM_PIXVLD_LOW_DURATION_15_CLOCKS= 0x3,
+ DCIO_UNIPHY_MINIMUM_PIXVLD_LOW_DURATION_19_CLOCKS= 0x4,
+ DCIO_UNIPHY_MINIMUM_PIXVLD_LOW_DURATION_23_CLOCKS= 0x5,
+ DCIO_UNIPHY_MINIMUM_PIXVLD_LOW_DURATION_27_CLOCKS= 0x6,
+ DCIO_UNIPHY_MINIMUM_PIXVLD_LOW_DURATION_31_CLOCKS= 0x7,
+} DCIO_UNIPHY_LINK_CNTL_MINIMUM_PIXVLD_LOW_DURATION;
+typedef enum DCIO_UNIPHY_LINK_CNTL_CHANNEL_INVERT {
+ DCIO_UNIPHY_CHANNEL_NO_INVERSION = 0x0,
+ DCIO_UNIPHY_CHANNEL_INVERTED = 0x1,
+} DCIO_UNIPHY_LINK_CNTL_CHANNEL_INVERT;
+typedef enum DCIO_UNIPHY_LINK_CNTL_ENABLE_HPD_MASK {
+ DCIO_UNIPHY_LINK_ENABLE_HPD_MASK_DISALLOW = 0x0,
+ DCIO_UNIPHY_LINK_ENABLE_HPD_MASK_ALLOW = 0x1,
+ DCIO_UNIPHY_LINK_ENABLE_HPD_MASK_ALLOW_DEBOUNCED = 0x2,
+ DCIO_UNIPHY_LINK_ENABLE_HPD_MASK_ALLOW_TOGGLE_FILTERED= 0x3,
+} DCIO_UNIPHY_LINK_CNTL_ENABLE_HPD_MASK;
+typedef enum DCIO_UNIPHY_CHANNEL_XBAR_SOURCE {
+ DCIO_UNIPHY_CHANNEL_XBAR_SOURCE_CH0 = 0x0,
+ DCIO_UNIPHY_CHANNEL_XBAR_SOURCE_CH1 = 0x1,
+ DCIO_UNIPHY_CHANNEL_XBAR_SOURCE_CH2 = 0x2,
+ DCIO_UNIPHY_CHANNEL_XBAR_SOURCE_CH3 = 0x3,
+} DCIO_UNIPHY_CHANNEL_XBAR_SOURCE;
+typedef enum DCIO_DC_DVODATA_CONFIG_VIP_MUX_EN {
+ DCIO_VIP_MUX_EN_DVO = 0x0,
+ DCIO_VIP_MUX_EN_VIP = 0x1,
+} DCIO_DC_DVODATA_CONFIG_VIP_MUX_EN;
+typedef enum DCIO_DC_DVODATA_CONFIG_VIP_ALTER_MAPPING_EN {
+ DCIO_VIP_ALTER_MAPPING_EN_DEFAULT = 0x0,
+ DCIO_VIP_ALTER_MAPPING_EN_ALTERNATIVE = 0x1,
+} DCIO_DC_DVODATA_CONFIG_VIP_ALTER_MAPPING_EN;
+typedef enum DCIO_DC_DVODATA_CONFIG_DVO_ALTER_MAPPING_EN {
+ DCIO_DVO_ALTER_MAPPING_EN_DEFAULT = 0x0,
+ DCIO_DVO_ALTER_MAPPING_EN_ALTERNATIVE = 0x1,
+} DCIO_DC_DVODATA_CONFIG_DVO_ALTER_MAPPING_EN;
+typedef enum DCIO_LVTMA_PWRSEQ_CNTL_DISABLE_SYNCEN_CONTROL_OF_TX_EN {
+ DCIO_LVTMA_PWRSEQ_DISABLE_SYNCEN_CONTROL_OF_TX_ENABLE= 0x0,
+ DCIO_LVTMA_PWRSEQ_DISABLE_SYNCEN_CONTROL_OF_TX_DISABLE= 0x1,
+} DCIO_LVTMA_PWRSEQ_CNTL_DISABLE_SYNCEN_CONTROL_OF_TX_EN;
+typedef enum DCIO_LVTMA_PWRSEQ_CNTL_TARGET_STATE {
+ DCIO_LVTMA_PWRSEQ_TARGET_STATE_LCD_OFF = 0x0,
+ DCIO_LVTMA_PWRSEQ_TARGET_STATE_LCD_ON = 0x1,
+} DCIO_LVTMA_PWRSEQ_CNTL_TARGET_STATE;
+typedef enum DCIO_LVTMA_PWRSEQ_CNTL_LVTMA_SYNCEN_POL {
+ DCIO_LVTMA_SYNCEN_POL_NON_INVERT = 0x0,
+ DCIO_LVTMA_SYNCEN_POL_INVERT = 0x1,
+} DCIO_LVTMA_PWRSEQ_CNTL_LVTMA_SYNCEN_POL;
+typedef enum DCIO_LVTMA_PWRSEQ_CNTL_LVTMA_DIGON {
+ DCIO_LVTMA_DIGON_OFF = 0x0,
+ DCIO_LVTMA_DIGON_ON = 0x1,
+} DCIO_LVTMA_PWRSEQ_CNTL_LVTMA_DIGON;
+typedef enum DCIO_LVTMA_PWRSEQ_CNTL_LVTMA_DIGON_POL {
+ DCIO_LVTMA_DIGON_POL_NON_INVERT = 0x0,
+ DCIO_LVTMA_DIGON_POL_INVERT = 0x1,
+} DCIO_LVTMA_PWRSEQ_CNTL_LVTMA_DIGON_POL;
+typedef enum DCIO_LVTMA_PWRSEQ_CNTL_LVTMA_BLON {
+ DCIO_LVTMA_BLON_OFF = 0x0,
+ DCIO_LVTMA_BLON_ON = 0x1,
+} DCIO_LVTMA_PWRSEQ_CNTL_LVTMA_BLON;
+typedef enum DCIO_LVTMA_PWRSEQ_CNTL_LVTMA_BLON_POL {
+ DCIO_LVTMA_BLON_POL_NON_INVERT = 0x0,
+ DCIO_LVTMA_BLON_POL_INVERT = 0x1,
+} DCIO_LVTMA_PWRSEQ_CNTL_LVTMA_BLON_POL;
+typedef enum DCIO_LVTMA_PWRSEQ_DELAY2_LVTMA_VARY_BL_OVERRIDE_EN {
+ DCIO_LVTMA_VARY_BL_OVERRIDE_EN_BLON = 0x0,
+ DCIO_LVTMA_VARY_BL_OVERRIDE_EN_SEPARATE = 0x1,
+} DCIO_LVTMA_PWRSEQ_DELAY2_LVTMA_VARY_BL_OVERRIDE_EN;
+typedef enum DCIO_BL_PWM_CNTL_BL_PWM_FRACTIONAL_EN {
+ DCIO_BL_PWM_FRACTIONAL_DISABLE = 0x0,
+ DCIO_BL_PWM_FRACTIONAL_ENABLE = 0x1,
+} DCIO_BL_PWM_CNTL_BL_PWM_FRACTIONAL_EN;
+typedef enum DCIO_BL_PWM_CNTL_BL_PWM_EN {
+ DCIO_BL_PWM_DISABLE = 0x0,
+ DCIO_BL_PWM_ENABLE = 0x1,
+} DCIO_BL_PWM_CNTL_BL_PWM_EN;
+typedef enum DCIO_BL_PWM_CNTL2_DBG_BL_PWM_INPUT_REFCLK_SELECT {
+ DCIO_DBG_BL_PWM_INPUT_REFCLK_SELECT_NORMAL = 0x0,
+ DCIO_DBG_BL_PWM_INPUT_REFCLK_SELECT_DEBUG1 = 0x1,
+ DCIO_DBG_BL_PWM_INPUT_REFCLK_SELECT_DEBUG2 = 0x2,
+ DCIO_DBG_BL_PWM_INPUT_REFCLK_SELECT_DEBUG3 = 0x3,
+} DCIO_BL_PWM_CNTL2_DBG_BL_PWM_INPUT_REFCLK_SELECT;
+typedef enum DCIO_BL_PWM_CNTL2_BL_PWM_OVERRIDE_BL_OUT_ENABLE {
+ DCIO_BL_PWM_OVERRIDE_BL_OUT_DISABLE = 0x0,
+ DCIO_BL_PWM_OVERRIDE_BL_OUT_ENABLE = 0x1,
+} DCIO_BL_PWM_CNTL2_BL_PWM_OVERRIDE_BL_OUT_ENABLE;
+typedef enum DCIO_BL_PWM_CNTL2_BL_PWM_OVERRIDE_LVTMA_PWRSEQ_EN {
+ DCIO_BL_PWM_OVERRIDE_LVTMA_PWRSEQ_EN_NORMAL = 0x0,
+ DCIO_BL_PWM_OVERRIDE_LVTMA_PWRSEQ_EN_PWM = 0x1,
+} DCIO_BL_PWM_CNTL2_BL_PWM_OVERRIDE_LVTMA_PWRSEQ_EN;
+typedef enum DCIO_BL_PWM_GRP1_REG_LOCK {
+ DCIO_BL_PWM_GRP1_REG_LOCK_DISABLE = 0x0,
+ DCIO_BL_PWM_GRP1_REG_LOCK_ENABLE = 0x1,
+} DCIO_BL_PWM_GRP1_REG_LOCK;
+typedef enum DCIO_BL_PWM_GRP1_UPDATE_AT_FRAME_START {
+ DCIO_BL_PWM_GRP1_UPDATE_AT_FRAME_START_DISABLE = 0x0,
+ DCIO_BL_PWM_GRP1_UPDATE_AT_FRAME_START_ENABLE = 0x1,
+} DCIO_BL_PWM_GRP1_UPDATE_AT_FRAME_START;
+typedef enum DCIO_BL_PWM_GRP1_FRAME_START_DISP_SEL {
+ DCIO_BL_PWM_GRP1_FRAME_START_DISP_SEL_CONTROLLER1= 0x0,
+ DCIO_BL_PWM_GRP1_FRAME_START_DISP_SEL_CONTROLLER2= 0x1,
+ DCIO_BL_PWM_GRP1_FRAME_START_DISP_SEL_CONTROLLER3= 0x2,
+ DCIO_BL_PWM_GRP1_FRAME_START_DISP_SEL_CONTROLLER4= 0x3,
+ DCIO_BL_PWM_GRP1_FRAME_START_DISP_SEL_CONTROLLER5= 0x4,
+ DCIO_BL_PWM_GRP1_FRAME_START_DISP_SEL_CONTROLLER6= 0x5,
+} DCIO_BL_PWM_GRP1_FRAME_START_DISP_SEL;
+typedef enum DCIO_BL_PWM_GRP1_READBACK_DB_REG_VALUE_EN {
+ DCIO_BL_PWM_GRP1_READBACK_DB_REG_VALUE_EN_BL_PWM = 0x0,
+ DCIO_BL_PWM_GRP1_READBACK_DB_REG_VALUE_EN_BL1_PWM= 0x1,
+} DCIO_BL_PWM_GRP1_READBACK_DB_REG_VALUE_EN;
+typedef enum DCIO_BL_PWM_GRP1_IGNORE_MASTER_LOCK_EN {
+ DCIO_BL_PWM_GRP1_IGNORE_MASTER_LOCK_ENABLE = 0x0,
+ DCIO_BL_PWM_GRP1_IGNORE_MASTER_LOCK_DISABLE = 0x1,
+} DCIO_BL_PWM_GRP1_IGNORE_MASTER_LOCK_EN;
+typedef enum DCIO_GSL_SEL {
+ DCIO_GSL_SEL_GROUP_0 = 0x0,
+ DCIO_GSL_SEL_GROUP_1 = 0x1,
+ DCIO_GSL_SEL_GROUP_2 = 0x2,
+} DCIO_GSL_SEL;
+typedef enum DCIO_GENLK_CLK_GSL_MASK {
+ DCIO_GENLK_CLK_GSL_MASK_NO = 0x0,
+ DCIO_GENLK_CLK_GSL_MASK_TIMING = 0x1,
+ DCIO_GENLK_CLK_GSL_MASK_STEREO = 0x2,
+} DCIO_GENLK_CLK_GSL_MASK;
+typedef enum DCIO_GENLK_VSYNC_GSL_MASK {
+ DCIO_GENLK_VSYNC_GSL_MASK_NO = 0x0,
+ DCIO_GENLK_VSYNC_GSL_MASK_TIMING = 0x1,
+ DCIO_GENLK_VSYNC_GSL_MASK_STEREO = 0x2,
+} DCIO_GENLK_VSYNC_GSL_MASK;
+typedef enum DCIO_SWAPLOCK_A_GSL_MASK {
+ DCIO_SWAPLOCK_A_GSL_MASK_NO = 0x0,
+ DCIO_SWAPLOCK_A_GSL_MASK_TIMING = 0x1,
+ DCIO_SWAPLOCK_A_GSL_MASK_STEREO = 0x2,
+} DCIO_SWAPLOCK_A_GSL_MASK;
+typedef enum DCIO_SWAPLOCK_B_GSL_MASK {
+ DCIO_SWAPLOCK_B_GSL_MASK_NO = 0x0,
+ DCIO_SWAPLOCK_B_GSL_MASK_TIMING = 0x1,
+ DCIO_SWAPLOCK_B_GSL_MASK_STEREO = 0x2,
+} DCIO_SWAPLOCK_B_GSL_MASK;
+typedef enum DCIO_GSL_VSYNC_SEL {
+ DCIO_GSL_VSYNC_SEL_PIPE0 = 0x0,
+ DCIO_GSL_VSYNC_SEL_PIPE1 = 0x1,
+ DCIO_GSL_VSYNC_SEL_PIPE2 = 0x2,
+ DCIO_GSL_VSYNC_SEL_PIPE3 = 0x3,
+ DCIO_GSL_VSYNC_SEL_PIPE4 = 0x4,
+ DCIO_GSL_VSYNC_SEL_PIPE5 = 0x5,
+} DCIO_GSL_VSYNC_SEL;
+typedef enum DCIO_GSL0_TIMING_SYNC_SEL {
+ DCIO_GSL0_TIMING_SYNC_SEL_PIPE = 0x0,
+ DCIO_GSL0_TIMING_SYNC_SEL_GENCLK_VSYNC = 0x1,
+ DCIO_GSL0_TIMING_SYNC_SEL_GENCLK_CLK = 0x2,
+ DCIO_GSL0_TIMING_SYNC_SEL_SWAPLOCK_A = 0x3,
+ DCIO_GSL0_TIMING_SYNC_SEL_SWAPLOCK_B = 0x4,
+} DCIO_GSL0_TIMING_SYNC_SEL;
+typedef enum DCIO_GSL0_GLOBAL_UNLOCK_SEL {
+ DCIO_GSL0_GLOBAL_UNLOCK_SEL_INVERSION = 0x0,
+ DCIO_GSL0_GLOBAL_UNLOCK_SEL_GENCLK_VSYNC = 0x1,
+ DCIO_GSL0_GLOBAL_UNLOCK_SEL_GENLK_CLK = 0x2,
+ DCIO_GSL0_GLOBAL_UNLOCK_SEL_SWAPLOCK_A = 0x3,
+ DCIO_GSL0_GLOBAL_UNLOCK_SEL_SWAPLOCK_B = 0x4,
+} DCIO_GSL0_GLOBAL_UNLOCK_SEL;
+typedef enum DCIO_GSL1_TIMING_SYNC_SEL {
+ DCIO_GSL1_TIMING_SYNC_SEL_PIPE = 0x0,
+ DCIO_GSL1_TIMING_SYNC_SEL_GENCLK_VSYNC = 0x1,
+ DCIO_GSL1_TIMING_SYNC_SEL_GENCLK_CLK = 0x2,
+ DCIO_GSL1_TIMING_SYNC_SEL_SWAPLOCK_A = 0x3,
+ DCIO_GSL1_TIMING_SYNC_SEL_SWAPLOCK_B = 0x4,
+} DCIO_GSL1_TIMING_SYNC_SEL;
+typedef enum DCIO_GSL1_GLOBAL_UNLOCK_SEL {
+ DCIO_GSL1_GLOBAL_UNLOCK_SEL_INVERSION = 0x0,
+ DCIO_GSL1_GLOBAL_UNLOCK_SEL_GENCLK_VSYNC = 0x1,
+ DCIO_GSL1_GLOBAL_UNLOCK_SEL_GENLK_CLK = 0x2,
+ DCIO_GSL1_GLOBAL_UNLOCK_SEL_SWAPLOCK_A = 0x3,
+ DCIO_GSL1_GLOBAL_UNLOCK_SEL_SWAPLOCK_B = 0x4,
+} DCIO_GSL1_GLOBAL_UNLOCK_SEL;
+typedef enum DCIO_GSL2_TIMING_SYNC_SEL {
+ DCIO_GSL2_TIMING_SYNC_SEL_PIPE = 0x0,
+ DCIO_GSL2_TIMING_SYNC_SEL_GENCLK_VSYNC = 0x1,
+ DCIO_GSL2_TIMING_SYNC_SEL_GENCLK_CLK = 0x2,
+ DCIO_GSL2_TIMING_SYNC_SEL_SWAPLOCK_A = 0x3,
+ DCIO_GSL2_TIMING_SYNC_SEL_SWAPLOCK_B = 0x4,
+} DCIO_GSL2_TIMING_SYNC_SEL;
+typedef enum DCIO_GSL2_GLOBAL_UNLOCK_SEL {
+ DCIO_GSL2_GLOBAL_UNLOCK_SEL_INVERSION = 0x0,
+ DCIO_GSL2_GLOBAL_UNLOCK_SEL_GENCLK_VSYNC = 0x1,
+ DCIO_GSL2_GLOBAL_UNLOCK_SEL_GENLK_CLK = 0x2,
+ DCIO_GSL2_GLOBAL_UNLOCK_SEL_SWAPLOCK_A = 0x3,
+ DCIO_GSL2_GLOBAL_UNLOCK_SEL_SWAPLOCK_B = 0x4,
+} DCIO_GSL2_GLOBAL_UNLOCK_SEL;
+typedef enum DCIO_DC_GPU_TIMER_START_POSITION {
+ DCIO_GPU_TIMER_START_0_END_27 = 0x0,
+ DCIO_GPU_TIMER_START_1_END_28 = 0x1,
+ DCIO_GPU_TIMER_START_2_END_29 = 0x2,
+ DCIO_GPU_TIMER_START_3_END_30 = 0x3,
+ DCIO_GPU_TIMER_START_4_END_31 = 0x4,
+ DCIO_GPU_TIMER_START_6_END_33 = 0x5,
+ DCIO_GPU_TIMER_START_8_END_35 = 0x6,
+ DCIO_GPU_TIMER_START_10_END_37 = 0x7,
+} DCIO_DC_GPU_TIMER_START_POSITION;
+typedef enum DCIO_CLOCK_CNTL_DCIO_TEST_CLK_SEL {
+ DCIO_TEST_CLK_SEL_DISPCLK = 0x0,
+ DCIO_TEST_CLK_SEL_GATED_DISPCLK = 0x1,
+ DCIO_TEST_CLK_SEL_SCLK = 0x2,
+} DCIO_CLOCK_CNTL_DCIO_TEST_CLK_SEL;
+typedef enum DCIO_CLOCK_CNTL_DISPCLK_R_DCIO_GATE_DIS {
+ DCIO_DISPCLK_R_DCIO_GATE_DISABLE = 0x0,
+ DCIO_DISPCLK_R_DCIO_GATE_ENABLE = 0x1,
+} DCIO_CLOCK_CNTL_DISPCLK_R_DCIO_GATE_DIS;
+typedef enum DCIO_DCO_DCFE_EXT_VSYNC_MUX {
+ DCIO_EXT_VSYNC_MUX_SWAPLOCKB = 0x0,
+ DCIO_EXT_VSYNC_MUX_CRTC0 = 0x1,
+ DCIO_EXT_VSYNC_MUX_CRTC1 = 0x2,
+ DCIO_EXT_VSYNC_MUX_CRTC2 = 0x3,
+ DCIO_EXT_VSYNC_MUX_CRTC3 = 0x4,
+ DCIO_EXT_VSYNC_MUX_CRTC4 = 0x5,
+ DCIO_EXT_VSYNC_MUX_CRTC5 = 0x6,
+ DCIO_EXT_VSYNC_MUX_GENERICB = 0x7,
+} DCIO_DCO_DCFE_EXT_VSYNC_MUX;
+typedef enum DCIO_DCO_EXT_VSYNC_MASK {
+ DCIO_EXT_VSYNC_MASK_NONE = 0x0,
+ DCIO_EXT_VSYNC_MASK_PIPE0 = 0x1,
+ DCIO_EXT_VSYNC_MASK_PIPE1 = 0x2,
+ DCIO_EXT_VSYNC_MASK_PIPE2 = 0x3,
+ DCIO_EXT_VSYNC_MASK_PIPE3 = 0x4,
+ DCIO_EXT_VSYNC_MASK_PIPE4 = 0x5,
+ DCIO_EXT_VSYNC_MASK_PIPE5 = 0x6,
+ DCIO_EXT_VSYNC_MASK_NONE_DUPLICATE = 0x7,
+} DCIO_DCO_EXT_VSYNC_MASK;
+typedef enum DCIO_DBG_OUT_PIN_SEL {
+ DCIO_DBG_OUT_PIN_SEL_LOW_12BIT = 0x0,
+ DCIO_DBG_OUT_PIN_SEL_HIGH_12BIT = 0x1,
+} DCIO_DBG_OUT_PIN_SEL;
+typedef enum DCIO_DBG_OUT_12BIT_SEL {
+ DCIO_DBG_OUT_12BIT_SEL_LOW_12BIT = 0x0,
+ DCIO_DBG_OUT_12BIT_SEL_MID_12BIT = 0x1,
+ DCIO_DBG_OUT_12BIT_SEL_HIGH_12BIT = 0x2,
+ DCIO_DBG_OUT_12BIT_SEL_OVERRIDE = 0x3,
+} DCIO_DBG_OUT_12BIT_SEL;
+typedef enum DCIO_DSYNC_SOFT_RESET {
+ DCIO_DSYNC_SOFT_RESET_DEASSERT = 0x0,
+ DCIO_DSYNC_SOFT_RESET_ASSERT = 0x1,
+} DCIO_DSYNC_SOFT_RESET;
+typedef enum DCIO_DACA_SOFT_RESET {
+ DCIO_DACA_SOFT_RESET_DEASSERT = 0x0,
+ DCIO_DACA_SOFT_RESET_ASSERT = 0x1,
+} DCIO_DACA_SOFT_RESET;
+typedef enum DCIO_DCRXPHY_SOFT_RESET {
+ DCIO_DCRXPHY_SOFT_RESET_DEASSERT = 0x0,
+ DCIO_DCRXPHY_SOFT_RESET_ASSERT = 0x1,
+} DCIO_DCRXPHY_SOFT_RESET;
+typedef enum DCIO_DPHY_LANE_SEL {
+ DCIO_DPHY_LANE_SEL_LANE0 = 0x0,
+ DCIO_DPHY_LANE_SEL_LANE1 = 0x1,
+ DCIO_DPHY_LANE_SEL_LANE2 = 0x2,
+ DCIO_DPHY_LANE_SEL_LANE3 = 0x3,
+} DCIO_DPHY_LANE_SEL;
+typedef enum DCIO_DPCS_INTERRUPT_TYPE {
+ DCIO_DPCS_INTERRUPT_TYPE_LEVEL_BASED = 0x0,
+ DCIO_DPCS_INTERRUPT_TYPE_PULSE_BASED = 0x1,
+} DCIO_DPCS_INTERRUPT_TYPE;
+typedef enum DCIO_DPCS_INTERRUPT_MASK {
+ DCIO_DPCS_INTERRUPT_DISABLE = 0x0,
+ DCIO_DPCS_INTERRUPT_ENABLE = 0x1,
+} DCIO_DPCS_INTERRUPT_MASK;
+typedef enum DCIO_DC_GPU_TIMER_READ_SELECT {
+ DCIO_GPU_TIMER_READ_SELECT_LOWER_D1_V_UPDATE = 0x0,
+ DCIO_GPU_TIMER_READ_SELECT_UPPER_D1_V_UPDATE = 0x1,
+ DCIO_GPU_TIMER_READ_SELECT_LOWER_D2_V_UPDATE = 0x2,
+ DCIO_GPU_TIMER_READ_SELECT_UPPER_D2_V_UPDATE = 0x3,
+ DCIO_GPU_TIMER_READ_SELECT_LOWER_D3_V_UPDATE = 0x4,
+ DCIO_GPU_TIMER_READ_SELECT_UPPER_D3_V_UPDATE = 0x5,
+ DCIO_GPU_TIMER_READ_SELECT_LOWER_D4_V_UPDATE = 0x6,
+ DCIO_GPU_TIMER_READ_SELECT_UPPER_D4_V_UPDATE = 0x7,
+ DCIO_GPU_TIMER_READ_SELECT_LOWER_D5_V_UPDATE = 0x8,
+ DCIO_GPU_TIMER_READ_SELECT_UPPER_D5_V_UPDATE = 0x9,
+ DCIO_GPU_TIMER_READ_SELECT_LOWER_D6_V_UPDATE = 0xa,
+ DCIO_GPU_TIMER_READ_SELECT_UPPER_D6_V_UPDATE = 0xb,
+ DCIO_GPU_TIMER_READ_SELECT_LOWER_D1_P_FLIP = 0xc,
+ DCIO_GPU_TIMER_READ_SELECT_UPPER_D1_P_FLIP = 0xd,
+ DCIO_GPU_TIMER_READ_SELECT_LOWER_D2_P_FLIP = 0xe,
+ DCIO_GPU_TIMER_READ_SELECT_UPPER_D2_P_FLIP = 0xf,
+ DCIO_GPU_TIMER_READ_SELECT_LOWER_D3_P_FLIP = 0x10,
+ DCIO_GPU_TIMER_READ_SELECT_UPPER_D3_P_FLIP = 0x11,
+ DCIO_GPU_TIMER_READ_SELECT_LOWER_D4_P_FLIP = 0x12,
+ DCIO_GPU_TIMER_READ_SELECT_UPPER_D4_P_FLIP = 0x13,
+ DCIO_GPU_TIMER_READ_SELECT_LOWER_D5_P_FLIP = 0x14,
+ DCIO_GPU_TIMER_READ_SELECT_UPPER_D5_P_FLIP = 0x15,
+ DCIO_GPU_TIMER_READ_SELECT_LOWER_D6_P_FLIP = 0x16,
+ DCIO_GPU_TIMER_READ_SELECT_UPPER_D6_P_FLIP = 0x17,
+ DCIO_GPU_TIMER_READ_SELECT_LOWER_D1_VSYNC_NOM = 0x18,
+ DCIO_GPU_TIMER_READ_SELECT_UPPER_D1_VSYNC_NOM = 0x19,
+ DCIO_GPU_TIMER_READ_SELECT_LOWER_D2_VSYNC_NOM = 0x1a,
+ DCIO_GPU_TIMER_READ_SELECT_UPPER_D2_VSYNC_NOM = 0x1b,
+ DCIO_GPU_TIMER_READ_SELECT_LOWER_D3_VSYNC_NOM = 0x1c,
+ DCIO_GPU_TIMER_READ_SELECT_UPPER_D3_VSYNC_NOM = 0x1d,
+ DCIO_GPU_TIMER_READ_SELECT_LOWER_D4_VSYNC_NOM = 0x1e,
+ DCIO_GPU_TIMER_READ_SELECT_UPPER_D4_VSYNC_NOM = 0x1f,
+ DCIO_GPU_TIMER_READ_SELECT_LOWER_D5_VSYNC_NOM = 0x20,
+ DCIO_GPU_TIMER_READ_SELECT_UPPER_D5_VSYNC_NOM = 0x21,
+ DCIO_GPU_TIMER_READ_SELECT_LOWER_D6_VSYNC_NOM = 0x22,
+ DCIO_GPU_TIMER_READ_SELECT_UPPER_D6_VSYNC_NOM = 0x23,
+} DCIO_DC_GPU_TIMER_READ_SELECT;
+typedef enum DCIO_IMPCAL_STEP_DELAY {
+ DCIO_IMPCAL_STEP_DELAY_1us = 0x0,
+ DCIO_IMPCAL_STEP_DELAY_2us = 0x1,
+ DCIO_IMPCAL_STEP_DELAY_3us = 0x2,
+ DCIO_IMPCAL_STEP_DELAY_4us = 0x3,
+ DCIO_IMPCAL_STEP_DELAY_5us = 0x4,
+ DCIO_IMPCAL_STEP_DELAY_6us = 0x5,
+ DCIO_IMPCAL_STEP_DELAY_7us = 0x6,
+ DCIO_IMPCAL_STEP_DELAY_8us = 0x7,
+ DCIO_IMPCAL_STEP_DELAY_9us = 0x8,
+ DCIO_IMPCAL_STEP_DELAY_10us = 0x9,
+ DCIO_IMPCAL_STEP_DELAY_11us = 0xa,
+ DCIO_IMPCAL_STEP_DELAY_12us = 0xb,
+ DCIO_IMPCAL_STEP_DELAY_13us = 0xc,
+ DCIO_IMPCAL_STEP_DELAY_14us = 0xd,
+ DCIO_IMPCAL_STEP_DELAY_15us = 0xe,
+ DCIO_IMPCAL_STEP_DELAY_16us = 0xf,
+} DCIO_IMPCAL_STEP_DELAY;
+typedef enum DCIO_UNIPHY_IMPCAL_SEL {
+ DCIO_UNIPHY_IMPCAL_SEL_TEMPERATURE = 0x0,
+ DCIO_UNIPHY_IMPCAL_SEL_BINARY = 0x1,
+} DCIO_UNIPHY_IMPCAL_SEL;
+typedef enum DCIO_DBG_CLOCK_SEL {
+ DCIO_DBG_CLOCK_SEL_DISPCLK = 0x0,
+ DCIO_DBG_CLOCK_SEL_SYMCLKA = 0x1,
+ DCIO_DBG_CLOCK_SEL_SYMCLKB = 0x2,
+ DCIO_DBG_CLOCK_SEL_SYMCLKC = 0x3,
+ DCIO_DBG_CLOCK_SEL_SYMCLKD = 0x4,
+ DCIO_DBG_CLOCK_SEL_SYMCLKE = 0x5,
+ DCIO_DBG_CLOCK_SEL_SYMCLKF = 0x6,
+ DCIO_DBG_CLOCK_SEL_REFCLK = 0xb,
+} DCIO_DBG_CLOCK_SEL;
+typedef enum DCIOCHIP_HPD_SEL {
+ DCIOCHIP_HPD_SEL_ASYNC = 0x0,
+ DCIOCHIP_HPD_SEL_CLOCKED = 0x1,
+} DCIOCHIP_HPD_SEL;
+typedef enum DCIOCHIP_PAD_MODE {
+ DCIOCHIP_PAD_MODE_DDC = 0x0,
+ DCIOCHIP_PAD_MODE_DP = 0x1,
+} DCIOCHIP_PAD_MODE;
+typedef enum DCIOCHIP_AUXSLAVE_PAD_MODE {
+ DCIOCHIP_AUXSLAVE_PAD_MODE_I2C = 0x0,
+ DCIOCHIP_AUXSLAVE_PAD_MODE_AUX = 0x1,
+} DCIOCHIP_AUXSLAVE_PAD_MODE;
+typedef enum DCIOCHIP_INVERT {
+ DCIOCHIP_POL_NON_INVERT = 0x0,
+ DCIOCHIP_POL_INVERT = 0x1,
+} DCIOCHIP_INVERT;
+typedef enum DCIOCHIP_PD_EN {
+ DCIOCHIP_PD_EN_NOTALLOW = 0x0,
+ DCIOCHIP_PD_EN_ALLOW = 0x1,
+} DCIOCHIP_PD_EN;
+typedef enum DCIOCHIP_GPIO_MASK_EN {
+ DCIOCHIP_GPIO_MASK_EN_HARDWARE = 0x0,
+ DCIOCHIP_GPIO_MASK_EN_SOFTWARE = 0x1,
+} DCIOCHIP_GPIO_MASK_EN;
+typedef enum DCIOCHIP_MASK {
+ DCIOCHIP_MASK_DISABLE = 0x0,
+ DCIOCHIP_MASK_ENABLE = 0x1,
+} DCIOCHIP_MASK;
+typedef enum DCIOCHIP_GPIO_I2C_MASK {
+ DCIOCHIP_GPIO_I2C_MASK_DISABLE = 0x0,
+ DCIOCHIP_GPIO_I2C_MASK_ENABLE = 0x1,
+} DCIOCHIP_GPIO_I2C_MASK;
+typedef enum DCIOCHIP_GPIO_I2C_DRIVE {
+ DCIOCHIP_GPIO_I2C_DRIVE_LOW = 0x0,
+ DCIOCHIP_GPIO_I2C_DRIVE_HIGH = 0x1,
+} DCIOCHIP_GPIO_I2C_DRIVE;
+typedef enum DCIOCHIP_GPIO_I2C_EN {
+ DCIOCHIP_GPIO_I2C_DISABLE = 0x0,
+ DCIOCHIP_GPIO_I2C_ENABLE = 0x1,
+} DCIOCHIP_GPIO_I2C_EN;
+typedef enum DCIOCHIP_MASK_4BIT {
+ DCIOCHIP_MASK_4BIT_DISABLE = 0x0,
+ DCIOCHIP_MASK_4BIT_ENABLE = 0xf,
+} DCIOCHIP_MASK_4BIT;
+typedef enum DCIOCHIP_ENABLE_4BIT {
+ DCIOCHIP_4BIT_DISABLE = 0x0,
+ DCIOCHIP_4BIT_ENABLE = 0xf,
+} DCIOCHIP_ENABLE_4BIT;
+typedef enum DCIOCHIP_MASK_5BIT {
+ DCIOCHIP_MASIK_5BIT_DISABLE = 0x0,
+ DCIOCHIP_MASIK_5BIT_ENABLE = 0x1f,
+} DCIOCHIP_MASK_5BIT;
+typedef enum DCIOCHIP_ENABLE_5BIT {
+ DCIOCHIP_5BIT_DISABLE = 0x0,
+ DCIOCHIP_5BIT_ENABLE = 0x1f,
+} DCIOCHIP_ENABLE_5BIT;
+typedef enum DCIOCHIP_MASK_2BIT {
+ DCIOCHIP_MASK_2BIT_DISABLE = 0x0,
+ DCIOCHIP_MASK_2BIT_ENABLE = 0x3,
+} DCIOCHIP_MASK_2BIT;
+typedef enum DCIOCHIP_ENABLE_2BIT {
+ DCIOCHIP_2BIT_DISABLE = 0x0,
+ DCIOCHIP_2BIT_ENABLE = 0x3,
+} DCIOCHIP_ENABLE_2BIT;
+typedef enum DCIOCHIP_REF_27_SRC_SEL {
+ DCIOCHIP_REF_27_SRC_SEL_XTAL_DIVIDER = 0x0,
+ DCIOCHIP_REF_27_SRC_SEL_DISP_CLKIN2_DIVIDER = 0x1,
+ DCIOCHIP_REF_27_SRC_SEL_XTAL_BYPASS = 0x2,
+ DCIOCHIP_REF_27_SRC_SEL_DISP_CLKIN2_BYPASS = 0x3,
+} DCIOCHIP_REF_27_SRC_SEL;
+typedef enum DCIOCHIP_DVO_VREFPON {
+ DCIOCHIP_DVO_VREFPON_DISABLE = 0x0,
+ DCIOCHIP_DVO_VREFPON_ENABLE = 0x1,
+} DCIOCHIP_DVO_VREFPON;
+typedef enum DCIOCHIP_DVO_VREFSEL {
+ DCIOCHIP_DVO_VREFSEL_ONCHIP = 0x0,
+ DCIOCHIP_DVO_VREFSEL_EXTERNAL = 0x1,
+} DCIOCHIP_DVO_VREFSEL;
+typedef enum DCIOCHIP_SPDIF1_IMODE {
+ DCIOCHIP_SPDIF1_IMODE_OE_A = 0x0,
+ DCIOCHIP_SPDIF1_IMODE_TSTE_TSTO = 0x1,
+} DCIOCHIP_SPDIF1_IMODE;
+typedef enum DCIOCHIP_AUX_FALLSLEWSEL {
+ DCIOCHIP_AUX_FALLSLEWSEL_LOW = 0x0,
+ DCIOCHIP_AUX_FALLSLEWSEL_HIGH0 = 0x1,
+ DCIOCHIP_AUX_FALLSLEWSEL_HIGH1 = 0x2,
+ DCIOCHIP_AUX_FALLSLEWSEL_ULTRAHIGH = 0x3,
+} DCIOCHIP_AUX_FALLSLEWSEL;
+typedef enum DCIOCHIP_AUX_SPIKESEL {
+ DCIOCHIP_AUX_SPIKESEL_50NS = 0x0,
+ DCIOCHIP_AUX_SPIKESEL_10NS = 0x1,
+} DCIOCHIP_AUX_SPIKESEL;
+typedef enum DCIOCHIP_AUX_CSEL0P9 {
+ DCIOCHIP_AUX_CSEL_DEC1P0 = 0x0,
+ DCIOCHIP_AUX_CSEL_DEC0P9 = 0x1,
+} DCIOCHIP_AUX_CSEL0P9;
+typedef enum DCIOCHIP_AUX_CSEL1P1 {
+ DCIOCHIP_AUX_CSEL_INC1P0 = 0x0,
+ DCIOCHIP_AUX_CSEL_INC1P1 = 0x1,
+} DCIOCHIP_AUX_CSEL1P1;
+typedef enum DCIOCHIP_AUX_RSEL0P9 {
+ DCIOCHIP_AUX_RSEL_DEC1P0 = 0x0,
+ DCIOCHIP_AUX_RSEL_DEC0P9 = 0x1,
+} DCIOCHIP_AUX_RSEL0P9;
+typedef enum DCIOCHIP_AUX_RSEL1P1 {
+ DCIOCHIP_AUX_RSEL_INC1P0 = 0x0,
+ DCIOCHIP_AUX_RSEL_INC1P1 = 0x1,
+} DCIOCHIP_AUX_RSEL1P1;
+typedef enum DCP_GRPH_ENABLE {
+ DCP_GRPH_ENABLE_FALSE = 0x0,
+ DCP_GRPH_ENABLE_TRUE = 0x1,
+} DCP_GRPH_ENABLE;
+typedef enum DCP_GRPH_KEYER_ALPHA_SEL {
+ DCP_GRPH_KEYER_ALPHA_SEL_FALSE = 0x0,
+ DCP_GRPH_KEYER_ALPHA_SEL_TRUE = 0x1,
+} DCP_GRPH_KEYER_ALPHA_SEL;
+typedef enum DCP_GRPH_DEPTH {
+ DCP_GRPH_DEPTH_8BPP = 0x0,
+ DCP_GRPH_DEPTH_16BPP = 0x1,
+ DCP_GRPH_DEPTH_32BPP = 0x2,
+ DCP_GRPH_DEPTH_64BPP = 0x3,
+} DCP_GRPH_DEPTH;
+typedef enum DCP_GRPH_NUM_BANKS {
+ DCP_GRPH_NUM_BANKS_2BANK = 0x0,
+ DCP_GRPH_NUM_BANKS_4BANK = 0x1,
+ DCP_GRPH_NUM_BANKS_8BANK = 0x2,
+ DCP_GRPH_NUM_BANKS_16BANK = 0x3,
+} DCP_GRPH_NUM_BANKS;
+typedef enum DCP_GRPH_BANK_WIDTH {
+ DCP_GRPH_BANK_WIDTH_1 = 0x0,
+ DCP_GRPH_BANK_WIDTH_2 = 0x1,
+ DCP_GRPH_BANK_WIDTH_4 = 0x2,
+ DCP_GRPH_BANK_WIDTH_8 = 0x3,
+} DCP_GRPH_BANK_WIDTH;
+typedef enum DCP_GRPH_FORMAT {
+ DCP_GRPH_FORMAT_8BPP = 0x0,
+ DCP_GRPH_FORMAT_16BPP = 0x1,
+ DCP_GRPH_FORMAT_32BPP = 0x2,
+ DCP_GRPH_FORMAT_64BPP = 0x3,
+} DCP_GRPH_FORMAT;
+typedef enum DCP_GRPH_BANK_HEIGHT {
+ DCP_GRPH_BANK_HEIGHT_1 = 0x0,
+ DCP_GRPH_BANK_HEIGHT_2 = 0x1,
+ DCP_GRPH_BANK_HEIGHT_4 = 0x2,
+ DCP_GRPH_BANK_HEIGHT_8 = 0x3,
+} DCP_GRPH_BANK_HEIGHT;
+typedef enum DCP_GRPH_TILE_SPLIT {
+ DCP_GRPH_TILE_SPLIT_64B = 0x0,
+ DCP_GRPH_TILE_SPLIT_128B = 0x1,
+ DCP_GRPH_TILE_SPLIT_256B = 0x2,
+ DCP_GRPH_TILE_SPLIT_512B = 0x3,
+ DCP_GRPH_TILE_SPLIT_1B = 0x4,
+ DCP_GRPH_TILE_SPLIT_2B = 0x5,
+ DCP_GRPH_TILE_SPLIT_4B = 0x6,
+} DCP_GRPH_TILE_SPLIT;
+typedef enum DCP_GRPH_ADDRESS_TRANSLATION_ENABLE {
+ DCP_GRPH_ADDRESS_TRANSLATION_ENABLE_FALSE = 0x0,
+ DCP_GRPH_ADDRESS_TRANSLATION_ENABLE_TRUE = 0x1,
+} DCP_GRPH_ADDRESS_TRANSLATION_ENABLE;
+typedef enum DCP_GRPH_PRIVILEGED_ACCESS_ENABLE {
+ DCP_GRPH_PRIVILEGED_ACCESS_ENABLE_FALSE = 0x0,
+ DCP_GRPH_PRIVILEGED_ACCESS_ENABLE_TRUE = 0x1,
+} DCP_GRPH_PRIVILEGED_ACCESS_ENABLE;
+typedef enum DCP_GRPH_MACRO_TILE_ASPECT {
+ DCP_GRPH_MACRO_TILE_ASPECT_1 = 0x0,
+ DCP_GRPH_MACRO_TILE_ASPECT_2 = 0x1,
+ DCP_GRPH_MACRO_TILE_ASPECT_4 = 0x2,
+ DCP_GRPH_MACRO_TILE_ASPECT_8 = 0x3,
+} DCP_GRPH_MACRO_TILE_ASPECT;
+typedef enum DCP_GRPH_ARRAY_MODE {
+ DCP_GRPH_ARRAY_MODE_0 = 0x0,
+ DCP_GRPH_ARRAY_MODE_1 = 0x1,
+ DCP_GRPH_ARRAY_MODE_2 = 0x2,
+ DCP_GRPH_ARRAY_MODE_3 = 0x3,
+ DCP_GRPH_ARRAY_MODE_4 = 0x4,
+ DCP_GRPH_ARRAY_MODE_7 = 0x7,
+ DCP_GRPH_ARRAY_MODE_12 = 0xc,
+ DCP_GRPH_ARRAY_MODE_13 = 0xd,
+} DCP_GRPH_ARRAY_MODE;
+typedef enum DCP_GRPH_MICRO_TILE_MODE {
+ DCP_GRPH_MICRO_TILE_MODE_0 = 0x0,
+ DCP_GRPH_MICRO_TILE_MODE_1 = 0x1,
+ DCP_GRPH_MICRO_TILE_MODE_2 = 0x2,
+ DCP_GRPH_MICRO_TILE_MODE_3 = 0x3,
+} DCP_GRPH_MICRO_TILE_MODE;
+typedef enum DCP_GRPH_COLOR_EXPANSION_MODE {
+ DCP_GRPH_COLOR_EXPANSION_MODE_DEXP = 0x0,
+ DCP_GRPH_COLOR_EXPANSION_MODE_ZEXP = 0x1,
+} DCP_GRPH_COLOR_EXPANSION_MODE;
+typedef enum DCP_GRPH_LUT_10BIT_BYPASS_EN {
+ DCP_GRPH_LUT_10BIT_BYPASS_EN_FALSE = 0x0,
+ DCP_GRPH_LUT_10BIT_BYPASS_EN_TRUE = 0x1,
+} DCP_GRPH_LUT_10BIT_BYPASS_EN;
+typedef enum DCP_GRPH_LUT_10BIT_BYPASS_DBL_BUF_EN {
+ DCP_GRPH_LUT_10BIT_BYPASS_DBL_BUF_EN_FALSE = 0x0,
+ DCP_GRPH_LUT_10BIT_BYPASS_DBL_BUF_EN_TRUE = 0x1,
+} DCP_GRPH_LUT_10BIT_BYPASS_DBL_BUF_EN;
+typedef enum DCP_GRPH_ENDIAN_SWAP {
+ DCP_GRPH_ENDIAN_SWAP_NONE = 0x0,
+ DCP_GRPH_ENDIAN_SWAP_8IN16 = 0x1,
+ DCP_GRPH_ENDIAN_SWAP_8IN32 = 0x2,
+ DCP_GRPH_ENDIAN_SWAP_8IN64 = 0x3,
+} DCP_GRPH_ENDIAN_SWAP;
+typedef enum DCP_GRPH_RED_CROSSBAR {
+ DCP_GRPH_RED_CROSSBAR_FROM_R = 0x0,
+ DCP_GRPH_RED_CROSSBAR_FROM_G = 0x1,
+ DCP_GRPH_RED_CROSSBAR_FROM_B = 0x2,
+ DCP_GRPH_RED_CROSSBAR_FROM_A = 0x3,
+} DCP_GRPH_RED_CROSSBAR;
+typedef enum DCP_GRPH_GREEN_CROSSBAR {
+ DCP_GRPH_GREEN_CROSSBAR_FROM_G = 0x0,
+ DCP_GRPH_GREEN_CROSSBAR_FROM_B = 0x1,
+ DCP_GRPH_GREEN_CROSSBAR_FROM_A = 0x2,
+ DCP_GRPH_GREEN_CROSSBAR_FROM_R = 0x3,
+} DCP_GRPH_GREEN_CROSSBAR;
+typedef enum DCP_GRPH_BLUE_CROSSBAR {
+ DCP_GRPH_BLUE_CROSSBAR_FROM_B = 0x0,
+ DCP_GRPH_BLUE_CROSSBAR_FROM_A = 0x1,
+ DCP_GRPH_BLUE_CROSSBAR_FROM_R = 0x2,
+ DCP_GRPH_BLUE_CROSSBAR_FROM_G = 0x3,
+} DCP_GRPH_BLUE_CROSSBAR;
+typedef enum DCP_GRPH_ALPHA_CROSSBAR {
+ DCP_GRPH_ALPHA_CROSSBAR_FROM_A = 0x0,
+ DCP_GRPH_ALPHA_CROSSBAR_FROM_R = 0x1,
+ DCP_GRPH_ALPHA_CROSSBAR_FROM_G = 0x2,
+ DCP_GRPH_ALPHA_CROSSBAR_FROM_B = 0x3,
+} DCP_GRPH_ALPHA_CROSSBAR;
+typedef enum DCP_GRPH_PRIMARY_DFQ_ENABLE {
+ DCP_GRPH_PRIMARY_DFQ_ENABLE_FALSE = 0x0,
+ DCP_GRPH_PRIMARY_DFQ_ENABLE_TRUE = 0x1,
+} DCP_GRPH_PRIMARY_DFQ_ENABLE;
+typedef enum DCP_GRPH_SECONDARY_DFQ_ENABLE {
+ DCP_GRPH_SECONDARY_DFQ_ENABLE_FALSE = 0x0,
+ DCP_GRPH_SECONDARY_DFQ_ENABLE_TRUE = 0x1,
+} DCP_GRPH_SECONDARY_DFQ_ENABLE;
+typedef enum DCP_GRPH_INPUT_GAMMA_MODE {
+ DCP_GRPH_INPUT_GAMMA_MODE_LUT = 0x0,
+ DCP_GRPH_INPUT_GAMMA_MODE_BYPASS = 0x1,
+} DCP_GRPH_INPUT_GAMMA_MODE;
+typedef enum DCP_GRPH_MODE_UPDATE_PENDING {
+ DCP_GRPH_MODE_UPDATE_PENDING_FALSE = 0x0,
+ DCP_GRPH_MODE_UPDATE_PENDING_TRUE = 0x1,
+} DCP_GRPH_MODE_UPDATE_PENDING;
+typedef enum DCP_GRPH_MODE_UPDATE_TAKEN {
+ DCP_GRPH_MODE_UPDATE_TAKEN_FALSE = 0x0,
+ DCP_GRPH_MODE_UPDATE_TAKEN_TRUE = 0x1,
+} DCP_GRPH_MODE_UPDATE_TAKEN;
+typedef enum DCP_GRPH_SURFACE_UPDATE_PENDING {
+ DCP_GRPH_SURFACE_UPDATE_PENDING_FALSE = 0x0,
+ DCP_GRPH_SURFACE_UPDATE_PENDING_TRUE = 0x1,
+} DCP_GRPH_SURFACE_UPDATE_PENDING;
+typedef enum DCP_GRPH_SURFACE_UPDATE_TAKEN {
+ DCP_GRPH_SURFACE_UPDATE_TAKEN_FALSE = 0x0,
+ DCP_GRPH_SURFACE_UPDATE_TAKEN_TRUE = 0x1,
+} DCP_GRPH_SURFACE_UPDATE_TAKEN;
+typedef enum DCP_GRPH_SURFACE_XDMA_PENDING_ENABLE {
+ DCP_GRPH_SURFACE_XDMA_PENDING_ENABLE_FALSE = 0x0,
+ DCP_GRPH_SURFACE_XDMA_PENDING_ENABLE_TRUE = 0x1,
+} DCP_GRPH_SURFACE_XDMA_PENDING_ENABLE;
+typedef enum DCP_GRPH_UPDATE_LOCK {
+ DCP_GRPH_UPDATE_LOCK_FALSE = 0x0,
+ DCP_GRPH_UPDATE_LOCK_TRUE = 0x1,
+} DCP_GRPH_UPDATE_LOCK;
+typedef enum DCP_GRPH_SURFACE_IGNORE_UPDATE_LOCK {
+ DCP_GRPH_SURFACE_IGNORE_UPDATE_LOCK_FALSE = 0x0,
+ DCP_GRPH_SURFACE_IGNORE_UPDATE_LOCK_TRUE = 0x1,
+} DCP_GRPH_SURFACE_IGNORE_UPDATE_LOCK;
+typedef enum DCP_GRPH_MODE_DISABLE_MULTIPLE_UPDATE {
+ DCP_GRPH_MODE_DISABLE_MULTIPLE_UPDATE_FALSE = 0x0,
+ DCP_GRPH_MODE_DISABLE_MULTIPLE_UPDATE_TRUE = 0x1,
+} DCP_GRPH_MODE_DISABLE_MULTIPLE_UPDATE;
+typedef enum DCP_GRPH_SURFACE_DISABLE_MULTIPLE_UPDATE {
+ DCP_GRPH_SURFACE_DISABLE_MULTIPLE_UPDATE_FALSE = 0x0,
+ DCP_GRPH_SURFACE_DISABLE_MULTIPLE_UPDATE_TRUE = 0x1,
+} DCP_GRPH_SURFACE_DISABLE_MULTIPLE_UPDATE;
+typedef enum DCP_GRPH_SURFACE_UPDATE_H_RETRACE_EN {
+ DCP_GRPH_SURFACE_UPDATE_H_RETRACE_EN_FALSE = 0x0,
+ DCP_GRPH_SURFACE_UPDATE_H_RETRACE_EN_TRUE = 0x1,
+} DCP_GRPH_SURFACE_UPDATE_H_RETRACE_EN;
+typedef enum DCP_GRPH_XDMA_SUPER_AA_EN {
+ DCP_GRPH_XDMA_SUPER_AA_EN_FALSE = 0x0,
+ DCP_GRPH_XDMA_SUPER_AA_EN_TRUE = 0x1,
+} DCP_GRPH_XDMA_SUPER_AA_EN;
+typedef enum DCP_GRPH_DFQ_RESET {
+ DCP_GRPH_DFQ_RESET_FALSE = 0x0,
+ DCP_GRPH_DFQ_RESET_TRUE = 0x1,
+} DCP_GRPH_DFQ_RESET;
+typedef enum DCP_GRPH_DFQ_SIZE {
+ DCP_GRPH_DFQ_SIZE_DEEP1 = 0x0,
+ DCP_GRPH_DFQ_SIZE_DEEP2 = 0x1,
+ DCP_GRPH_DFQ_SIZE_DEEP3 = 0x2,
+ DCP_GRPH_DFQ_SIZE_DEEP4 = 0x3,
+ DCP_GRPH_DFQ_SIZE_DEEP5 = 0x4,
+ DCP_GRPH_DFQ_SIZE_DEEP6 = 0x5,
+ DCP_GRPH_DFQ_SIZE_DEEP7 = 0x6,
+ DCP_GRPH_DFQ_SIZE_DEEP8 = 0x7,
+} DCP_GRPH_DFQ_SIZE;
+typedef enum DCP_GRPH_DFQ_MIN_FREE_ENTRIES {
+ DCP_GRPH_DFQ_MIN_FREE_ENTRIES_1 = 0x0,
+ DCP_GRPH_DFQ_MIN_FREE_ENTRIES_2 = 0x1,
+ DCP_GRPH_DFQ_MIN_FREE_ENTRIES_3 = 0x2,
+ DCP_GRPH_DFQ_MIN_FREE_ENTRIES_4 = 0x3,
+ DCP_GRPH_DFQ_MIN_FREE_ENTRIES_5 = 0x4,
+ DCP_GRPH_DFQ_MIN_FREE_ENTRIES_6 = 0x5,
+ DCP_GRPH_DFQ_MIN_FREE_ENTRIES_7 = 0x6,
+ DCP_GRPH_DFQ_MIN_FREE_ENTRIES_8 = 0x7,
+} DCP_GRPH_DFQ_MIN_FREE_ENTRIES;
+typedef enum DCP_GRPH_DFQ_RESET_ACK {
+ DCP_GRPH_DFQ_RESET_ACK_FALSE = 0x0,
+ DCP_GRPH_DFQ_RESET_ACK_TRUE = 0x1,
+} DCP_GRPH_DFQ_RESET_ACK;
+typedef enum DCP_GRPH_PFLIP_INT_CLEAR {
+ DCP_GRPH_PFLIP_INT_CLEAR_FALSE = 0x0,
+ DCP_GRPH_PFLIP_INT_CLEAR_TRUE = 0x1,
+} DCP_GRPH_PFLIP_INT_CLEAR;
+typedef enum DCP_GRPH_PFLIP_INT_MASK {
+ DCP_GRPH_PFLIP_INT_MASK_FALSE = 0x0,
+ DCP_GRPH_PFLIP_INT_MASK_TRUE = 0x1,
+} DCP_GRPH_PFLIP_INT_MASK;
+typedef enum DCP_GRPH_PFLIP_INT_TYPE {
+ DCP_GRPH_PFLIP_INT_TYPE_LEGACY_LEVEL = 0x0,
+ DCP_GRPH_PFLIP_INT_TYPE_PULSE = 0x1,
+} DCP_GRPH_PFLIP_INT_TYPE;
+typedef enum DCP_GRPH_PRESCALE_SELECT {
+ DCP_GRPH_PRESCALE_SELECT_FIXED = 0x0,
+ DCP_GRPH_PRESCALE_SELECT_FLOATING = 0x1,
+} DCP_GRPH_PRESCALE_SELECT;
+typedef enum DCP_GRPH_PRESCALE_R_SIGN {
+ DCP_GRPH_PRESCALE_R_SIGN_UNSIGNED = 0x0,
+ DCP_GRPH_PRESCALE_R_SIGN_SIGNED = 0x1,
+} DCP_GRPH_PRESCALE_R_SIGN;
+typedef enum DCP_GRPH_PRESCALE_G_SIGN {
+ DCP_GRPH_PRESCALE_G_SIGN_UNSIGNED = 0x0,
+ DCP_GRPH_PRESCALE_G_SIGN_SIGNED = 0x1,
+} DCP_GRPH_PRESCALE_G_SIGN;
+typedef enum DCP_GRPH_PRESCALE_B_SIGN {
+ DCP_GRPH_PRESCALE_B_SIGN_UNSIGNED = 0x0,
+ DCP_GRPH_PRESCALE_B_SIGN_SIGNED = 0x1,
+} DCP_GRPH_PRESCALE_B_SIGN;
+typedef enum DCP_GRPH_PRESCALE_BYPASS {
+ DCP_GRPH_PRESCALE_BYPASS_FALSE = 0x0,
+ DCP_GRPH_PRESCALE_BYPASS_TRUE = 0x1,
+} DCP_GRPH_PRESCALE_BYPASS;
+typedef enum DCP_INPUT_CSC_GRPH_MODE {
+ DCP_INPUT_CSC_GRPH_MODE_BYPASS = 0x0,
+ DCP_INPUT_CSC_GRPH_MODE_INPUT_CSC_COEF = 0x1,
+ DCP_INPUT_CSC_GRPH_MODE_SHARED_COEF = 0x2,
+ DCP_INPUT_CSC_GRPH_MODE_RESERVED = 0x3,
+} DCP_INPUT_CSC_GRPH_MODE;
+typedef enum DCP_OUTPUT_CSC_GRPH_MODE {
+ DCP_OUTPUT_CSC_GRPH_MODE_BYPASS = 0x0,
+ DCP_OUTPUT_CSC_GRPH_MODE_RGB = 0x1,
+ DCP_OUTPUT_CSC_GRPH_MODE_YCBCR601 = 0x2,
+ DCP_OUTPUT_CSC_GRPH_MODE_YCBCR709 = 0x3,
+ DCP_OUTPUT_CSC_GRPH_MODE_OUTPUT_CSC_COEF = 0x4,
+ DCP_OUTPUT_CSC_GRPH_MODE_SHARED_COEF = 0x5,
+ DCP_OUTPUT_CSC_GRPH_MODE_RESERVED0 = 0x6,
+ DCP_OUTPUT_CSC_GRPH_MODE_RESERVED1 = 0x7,
+} DCP_OUTPUT_CSC_GRPH_MODE;
+typedef enum DCP_DENORM_MODE {
+ DCP_DENORM_MODE_UNITY = 0x0,
+ DCP_DENORM_MODE_6BIT = 0x1,
+ DCP_DENORM_MODE_8BIT = 0x2,
+ DCP_DENORM_MODE_10BIT = 0x3,
+ DCP_DENORM_MODE_11BIT = 0x4,
+ DCP_DENORM_MODE_12BIT = 0x5,
+ DCP_DENORM_MODE_RESERVED0 = 0x6,
+ DCP_DENORM_MODE_RESERVED1 = 0x7,
+} DCP_DENORM_MODE;
+typedef enum DCP_DENORM_14BIT_OUT {
+ DCP_DENORM_14BIT_OUT_FALSE = 0x0,
+ DCP_DENORM_14BIT_OUT_TRUE = 0x1,
+} DCP_DENORM_14BIT_OUT;
+typedef enum DCP_OUT_ROUND_TRUNC_MODE {
+ DCP_OUT_ROUND_TRUNC_MODE_TRUNCATE_12 = 0x0,
+ DCP_OUT_ROUND_TRUNC_MODE_TRUNCATE_11 = 0x1,
+ DCP_OUT_ROUND_TRUNC_MODE_TRUNCATE_10 = 0x2,
+ DCP_OUT_ROUND_TRUNC_MODE_TRUNCATE_9 = 0x3,
+ DCP_OUT_ROUND_TRUNC_MODE_TRUNCATE_8 = 0x4,
+ DCP_OUT_ROUND_TRUNC_MODE_TRUNCATE_RESERVED = 0x5,
+ DCP_OUT_ROUND_TRUNC_MODE_TRUNCATE_14 = 0x6,
+ DCP_OUT_ROUND_TRUNC_MODE_TRUNCATE_13 = 0x7,
+ DCP_OUT_ROUND_TRUNC_MODE_ROUND_12 = 0x8,
+ DCP_OUT_ROUND_TRUNC_MODE_ROUND_11 = 0x9,
+ DCP_OUT_ROUND_TRUNC_MODE_ROUND_10 = 0xa,
+ DCP_OUT_ROUND_TRUNC_MODE_ROUND_9 = 0xb,
+ DCP_OUT_ROUND_TRUNC_MODE_ROUND_8 = 0xc,
+ DCP_OUT_ROUND_TRUNC_MODE_ROUND_RESERVED = 0xd,
+ DCP_OUT_ROUND_TRUNC_MODE_ROUND_14 = 0xe,
+ DCP_OUT_ROUND_TRUNC_MODE_ROUND_13 = 0xf,
+} DCP_OUT_ROUND_TRUNC_MODE;
+typedef enum DCP_KEY_MODE {
+ DCP_KEY_MODE_ALPHA0 = 0x0,
+ DCP_KEY_MODE_ALPHA1 = 0x1,
+ DCP_KEY_MODE_IN_RANGE_ALPHA1 = 0x2,
+ DCP_KEY_MODE_IN_RANGE_ALPHA0 = 0x3,
+} DCP_KEY_MODE;
+typedef enum DCP_GRPH_DEGAMMA_MODE {
+ DCP_GRPH_DEGAMMA_MODE_BYPASS = 0x0,
+ DCP_GRPH_DEGAMMA_MODE_ROMA = 0x1,
+ DCP_GRPH_DEGAMMA_MODE_ROMB = 0x2,
+ DCP_GRPH_DEGAMMA_MODE_RESERVED = 0x3,
+} DCP_GRPH_DEGAMMA_MODE;
+typedef enum DCP_CURSOR2_DEGAMMA_MODE {
+ DCP_CURSOR2_DEGAMMA_MODE_BYPASS = 0x0,
+ DCP_CURSOR2_DEGAMMA_MODE_ROMA = 0x1,
+ DCP_CURSOR2_DEGAMMA_MODE_ROMB = 0x2,
+ DCP_CURSOR2_DEGAMMA_MODE_RESERVED = 0x3,
+} DCP_CURSOR2_DEGAMMA_MODE;
+typedef enum DCP_CURSOR_DEGAMMA_MODE {
+ DCP_CURSOR_DEGAMMA_MODE_BYPASS = 0x0,
+ DCP_CURSOR_DEGAMMA_MODE_ROMA = 0x1,
+ DCP_CURSOR_DEGAMMA_MODE_ROMB = 0x2,
+ DCP_CURSOR_DEGAMMA_MODE_RESERVED = 0x3,
+} DCP_CURSOR_DEGAMMA_MODE;
+typedef enum DCP_GRPH_GAMUT_REMAP_MODE {
+ DCP_GRPH_GAMUT_REMAP_MODE_BYPASS = 0x0,
+ DCP_GRPH_GAMUT_REMAP_MODE_ROMA = 0x1,
+ DCP_GRPH_GAMUT_REMAP_MODE_ROMB = 0x2,
+ DCP_GRPH_GAMUT_REMAP_MODE_RESERVED = 0x3,
+} DCP_GRPH_GAMUT_REMAP_MODE;
+typedef enum DCP_SPATIAL_DITHER_EN {
+ DCP_SPATIAL_DITHER_EN_FALSE = 0x0,
+ DCP_SPATIAL_DITHER_EN_TRUE = 0x1,
+} DCP_SPATIAL_DITHER_EN;
+typedef enum DCP_SPATIAL_DITHER_MODE {
+ DCP_SPATIAL_DITHER_MODE_BYPASS = 0x0,
+ DCP_SPATIAL_DITHER_MODE_ROMA = 0x1,
+ DCP_SPATIAL_DITHER_MODE_ROMB = 0x2,
+ DCP_SPATIAL_DITHER_MODE_RESERVED = 0x3,
+} DCP_SPATIAL_DITHER_MODE;
+typedef enum DCP_SPATIAL_DITHER_DEPTH {
+ DCP_SPATIAL_DITHER_DEPTH_30BPP = 0x0,
+ DCP_SPATIAL_DITHER_DEPTH_24BPP = 0x1,
+ DCP_SPATIAL_DITHER_DEPTH_36BPP = 0x2,
+ DCP_SPATIAL_DITHER_DEPTH_UNDEFINED = 0x3,
+} DCP_SPATIAL_DITHER_DEPTH;
+typedef enum DCP_FRAME_RANDOM_ENABLE {
+ DCP_FRAME_RANDOM_ENABLE_FALSE = 0x0,
+ DCP_FRAME_RANDOM_ENABLE_TRUE = 0x1,
+} DCP_FRAME_RANDOM_ENABLE;
+typedef enum DCP_RGB_RANDOM_ENABLE {
+ DCP_RGB_RANDOM_ENABLE_FALSE = 0x0,
+ DCP_RGB_RANDOM_ENABLE_TRUE = 0x1,
+} DCP_RGB_RANDOM_ENABLE;
+typedef enum DCP_HIGHPASS_RANDOM_ENABLE {
+ DCP_HIGHPASS_RANDOM_ENABLE_FALSE = 0x0,
+ DCP_HIGHPASS_RANDOM_ENABLE_TRUE = 0x1,
+} DCP_HIGHPASS_RANDOM_ENABLE;
+typedef enum DCP_CURSOR_EN {
+ DCP_CURSOR_EN_FALSE = 0x0,
+ DCP_CURSOR_EN_TRUE = 0x1,
+} DCP_CURSOR_EN;
+typedef enum DCP_CUR_INV_TRANS_CLAMP {
+ DCP_CUR_INV_TRANS_CLAMP_FALSE = 0x0,
+ DCP_CUR_INV_TRANS_CLAMP_TRUE = 0x1,
+} DCP_CUR_INV_TRANS_CLAMP;
+typedef enum DCP_CURSOR_MODE {
+ DCP_CURSOR_MODE_MONO_2BPP = 0x0,
+ DCP_CURSOR_MODE_24BPP_1BIT = 0x1,
+ DCP_CURSOR_MODE_24BPP_8BIT_PREMULTI = 0x2,
+ DCP_CURSOR_MODE_24BPP_8BIT_UNPREMULTI = 0x3,
+} DCP_CURSOR_MODE;
+typedef enum DCP_CURSOR_2X_MAGNIFY {
+ DCP_CURSOR_2X_MAGNIFY_FALSE = 0x0,
+ DCP_CURSOR_2X_MAGNIFY_TRUE = 0x1,
+} DCP_CURSOR_2X_MAGNIFY;
+typedef enum DCP_CURSOR_FORCE_MC_ON {
+ DCP_CURSOR_FORCE_MC_ON_FALSE = 0x0,
+ DCP_CURSOR_FORCE_MC_ON_TRUE = 0x1,
+} DCP_CURSOR_FORCE_MC_ON;
+typedef enum DCP_CURSOR_URGENT_CONTROL {
+ DCP_CURSOR_URGENT_CONTROL_MODE_0 = 0x0,
+ DCP_CURSOR_URGENT_CONTROL_MODE_1 = 0x1,
+ DCP_CURSOR_URGENT_CONTROL_MODE_2 = 0x2,
+ DCP_CURSOR_URGENT_CONTROL_MODE_3 = 0x3,
+ DCP_CURSOR_URGENT_CONTROL_MODE_4 = 0x4,
+} DCP_CURSOR_URGENT_CONTROL;
+typedef enum DCP_CURSOR_UPDATE_PENDING {
+ DCP_CURSOR_UPDATE_PENDING_FALSE = 0x0,
+ DCP_CURSOR_UPDATE_PENDING_TRUE = 0x1,
+} DCP_CURSOR_UPDATE_PENDING;
+typedef enum DCP_CURSOR_UPDATE_TAKEN {
+ DCP_CURSOR_UPDATE_TAKEN_FALSE = 0x0,
+ DCP_CURSOR_UPDATE_TAKEN_TRUE = 0x1,
+} DCP_CURSOR_UPDATE_TAKEN;
+typedef enum DCP_CURSOR_UPDATE_LOCK {
+ DCP_CURSOR_UPDATE_LOCK_FALSE = 0x0,
+ DCP_CURSOR_UPDATE_LOCK_TRUE = 0x1,
+} DCP_CURSOR_UPDATE_LOCK;
+typedef enum DCP_CURSOR_DISABLE_MULTIPLE_UPDATE {
+ DCP_CURSOR_DISABLE_MULTIPLE_UPDATE_FALSE = 0x0,
+ DCP_CURSOR_DISABLE_MULTIPLE_UPDATE_TRUE = 0x1,
+} DCP_CURSOR_DISABLE_MULTIPLE_UPDATE;
+typedef enum DCP_CURSOR_UPDATE_STEREO_MODE {
+ DCP_CURSOR_UPDATE_STEREO_MODE_BOTH = 0x0,
+ DCP_CURSOR_UPDATE_STEREO_MODE_SECONDARY_ONLY = 0x1,
+ DCP_CURSOR_UPDATE_STEREO_MODE_UNDEFINED = 0x2,
+ DCP_CURSOR_UPDATE_STEREO_MODE_PRIMARY_ONLY = 0x3,
+} DCP_CURSOR_UPDATE_STEREO_MODE;
+typedef enum DCP_CURSOR2_EN {
+ DCP_CURSOR2_EN_FALSE = 0x0,
+ DCP_CURSOR2_EN_TRUE = 0x1,
+} DCP_CURSOR2_EN;
+typedef enum DCP_CUR2_INV_TRANS_CLAMP {
+ DCP_CUR2_INV_TRANS_CLAMP_FALSE = 0x0,
+ DCP_CUR2_INV_TRANS_CLAMP_TRUE = 0x1,
+} DCP_CUR2_INV_TRANS_CLAMP;
+typedef enum DCP_CURSOR2_MODE {
+ DCP_CURSOR2_MODE_MONO_2BPP = 0x0,
+ DCP_CURSOR2_MODE_24BPP_1BIT = 0x1,
+ DCP_CURSOR2_MODE_24BPP_8BIT_PREMULTI = 0x2,
+ DCP_CURSOR2_MODE_24BPP_8BIT_UNPREMULTI = 0x3,
+} DCP_CURSOR2_MODE;
+typedef enum DCP_CURSOR2_2X_MAGNIFY {
+ DCP_CURSOR2_2X_MAGNIFY_FALSE = 0x0,
+ DCP_CURSOR2_2X_MAGNIFY_TRUE = 0x1,
+} DCP_CURSOR2_2X_MAGNIFY;
+typedef enum DCP_CURSOR2_FORCE_MC_ON {
+ DCP_CURSOR2_FORCE_MC_ON_FALSE = 0x0,
+ DCP_CURSOR2_FORCE_MC_ON_TRUE = 0x1,
+} DCP_CURSOR2_FORCE_MC_ON;
+typedef enum DCP_CURSOR2_URGENT_CONTROL {
+ DCP_CURSOR2_URGENT_CONTROL_MODE_0 = 0x0,
+ DCP_CURSOR2_URGENT_CONTROL_MODE_1 = 0x1,
+ DCP_CURSOR2_URGENT_CONTROL_MODE_2 = 0x2,
+ DCP_CURSOR2_URGENT_CONTROL_MODE_3 = 0x3,
+ DCP_CURSOR2_URGENT_CONTROL_MODE_4 = 0x4,
+} DCP_CURSOR2_URGENT_CONTROL;
+typedef enum DCP_CURSOR2_UPDATE_PENDING {
+ DCP_CURSOR2_UPDATE_PENDING_FALSE = 0x0,
+ DCP_CURSOR2_UPDATE_PENDING_TRUE = 0x1,
+} DCP_CURSOR2_UPDATE_PENDING;
+typedef enum DCP_CURSOR2_UPDATE_TAKEN {
+ DCP_CURSOR2_UPDATE_TAKEN_FALSE = 0x0,
+ DCP_CURSOR2_UPDATE_TAKEN_TRUE = 0x1,
+} DCP_CURSOR2_UPDATE_TAKEN;
+typedef enum DCP_CURSOR2_UPDATE_LOCK {
+ DCP_CURSOR2_UPDATE_LOCK_FALSE = 0x0,
+ DCP_CURSOR2_UPDATE_LOCK_TRUE = 0x1,
+} DCP_CURSOR2_UPDATE_LOCK;
+typedef enum DCP_CURSOR2_DISABLE_MULTIPLE_UPDATE {
+ DCP_CURSOR2_DISABLE_MULTIPLE_UPDATE_FALSE = 0x0,
+ DCP_CURSOR2_DISABLE_MULTIPLE_UPDATE_TRUE = 0x1,
+} DCP_CURSOR2_DISABLE_MULTIPLE_UPDATE;
+typedef enum DCP_CURSOR2_UPDATE_STEREO_MODE {
+ DCP_CURSOR2_UPDATE_STEREO_MODE_BOTH = 0x0,
+ DCP_CURSOR2_UPDATE_STEREO_MODE_SECONDARY_ONLY = 0x1,
+ DCP_CURSOR2_UPDATE_STEREO_MODE_UNDEFINED = 0x2,
+ DCP_CURSOR2_UPDATE_STEREO_MODE_PRIMARY_ONLY = 0x3,
+} DCP_CURSOR2_UPDATE_STEREO_MODE;
+typedef enum DCP_CUR_REQUEST_FILTER_DIS {
+ DCP_CUR_REQUEST_FILTER_DIS_FALSE = 0x0,
+ DCP_CUR_REQUEST_FILTER_DIS_TRUE = 0x1,
+} DCP_CUR_REQUEST_FILTER_DIS;
+typedef enum DCP_CURSOR_STEREO_EN {
+ DCP_CURSOR_STEREO_EN_FALSE = 0x0,
+ DCP_CURSOR_STEREO_EN_TRUE = 0x1,
+} DCP_CURSOR_STEREO_EN;
+typedef enum DCP_CURSOR_STEREO_OFFSET_YNX {
+ DCP_CURSOR_STEREO_OFFSET_YNX_X_POSITION = 0x0,
+ DCP_CURSOR_STEREO_OFFSET_YNX_Y_POSITION = 0x1,
+} DCP_CURSOR_STEREO_OFFSET_YNX;
+typedef enum DCP_CURSOR2_STEREO_EN {
+ DCP_CURSOR2_STEREO_EN_FALSE = 0x0,
+ DCP_CURSOR2_STEREO_EN_TRUE = 0x1,
+} DCP_CURSOR2_STEREO_EN;
+typedef enum DCP_CURSOR2_STEREO_OFFSET_YNX {
+ DCP_CURSOR2_STEREO_OFFSET_YNX_X_POSITION = 0x0,
+ DCP_CURSOR2_STEREO_OFFSET_YNX_Y_POSITION = 0x1,
+} DCP_CURSOR2_STEREO_OFFSET_YNX;
+typedef enum DCP_DC_LUT_RW_MODE {
+ DCP_DC_LUT_RW_MODE_256_ENTRY = 0x0,
+ DCP_DC_LUT_RW_MODE_PWL = 0x1,
+} DCP_DC_LUT_RW_MODE;
+typedef enum DCP_DC_LUT_VGA_ACCESS_ENABLE {
+ DCP_DC_LUT_VGA_ACCESS_ENABLE_FALSE = 0x0,
+ DCP_DC_LUT_VGA_ACCESS_ENABLE_TRUE = 0x1,
+} DCP_DC_LUT_VGA_ACCESS_ENABLE;
+typedef enum DCP_DC_LUT_AUTOFILL {
+ DCP_DC_LUT_AUTOFILL_FALSE = 0x0,
+ DCP_DC_LUT_AUTOFILL_TRUE = 0x1,
+} DCP_DC_LUT_AUTOFILL;
+typedef enum DCP_DC_LUT_AUTOFILL_DONE {
+ DCP_DC_LUT_AUTOFILL_DONE_FALSE = 0x0,
+ DCP_DC_LUT_AUTOFILL_DONE_TRUE = 0x1,
+} DCP_DC_LUT_AUTOFILL_DONE;
+typedef enum DCP_DC_LUT_INC_B {
+ DCP_DC_LUT_INC_B_NA = 0x0,
+ DCP_DC_LUT_INC_B_2 = 0x1,
+ DCP_DC_LUT_INC_B_4 = 0x2,
+ DCP_DC_LUT_INC_B_8 = 0x3,
+ DCP_DC_LUT_INC_B_16 = 0x4,
+ DCP_DC_LUT_INC_B_32 = 0x5,
+ DCP_DC_LUT_INC_B_64 = 0x6,
+ DCP_DC_LUT_INC_B_128 = 0x7,
+ DCP_DC_LUT_INC_B_256 = 0x8,
+ DCP_DC_LUT_INC_B_512 = 0x9,
+} DCP_DC_LUT_INC_B;
+typedef enum DCP_DC_LUT_DATA_B_SIGNED_EN {
+ DCP_DC_LUT_DATA_B_SIGNED_EN_FALSE = 0x0,
+ DCP_DC_LUT_DATA_B_SIGNED_EN_TRUE = 0x1,
+} DCP_DC_LUT_DATA_B_SIGNED_EN;
+typedef enum DCP_DC_LUT_DATA_B_FLOAT_POINT_EN {
+ DCP_DC_LUT_DATA_B_FLOAT_POINT_EN_FALSE = 0x0,
+ DCP_DC_LUT_DATA_B_FLOAT_POINT_EN_TRUE = 0x1,
+} DCP_DC_LUT_DATA_B_FLOAT_POINT_EN;
+typedef enum DCP_DC_LUT_DATA_B_FORMAT {
+ DCP_DC_LUT_DATA_B_FORMAT_U0P10 = 0x0,
+ DCP_DC_LUT_DATA_B_FORMAT_S1P10 = 0x1,
+ DCP_DC_LUT_DATA_B_FORMAT_U1P11 = 0x2,
+ DCP_DC_LUT_DATA_B_FORMAT_U0P12 = 0x3,
+} DCP_DC_LUT_DATA_B_FORMAT;
+typedef enum DCP_DC_LUT_INC_G {
+ DCP_DC_LUT_INC_G_NA = 0x0,
+ DCP_DC_LUT_INC_G_2 = 0x1,
+ DCP_DC_LUT_INC_G_4 = 0x2,
+ DCP_DC_LUT_INC_G_8 = 0x3,
+ DCP_DC_LUT_INC_G_16 = 0x4,
+ DCP_DC_LUT_INC_G_32 = 0x5,
+ DCP_DC_LUT_INC_G_64 = 0x6,
+ DCP_DC_LUT_INC_G_128 = 0x7,
+ DCP_DC_LUT_INC_G_256 = 0x8,
+ DCP_DC_LUT_INC_G_512 = 0x9,
+} DCP_DC_LUT_INC_G;
+typedef enum DCP_DC_LUT_DATA_G_SIGNED_EN {
+ DCP_DC_LUT_DATA_G_SIGNED_EN_FALSE = 0x0,
+ DCP_DC_LUT_DATA_G_SIGNED_EN_TRUE = 0x1,
+} DCP_DC_LUT_DATA_G_SIGNED_EN;
+typedef enum DCP_DC_LUT_DATA_G_FLOAT_POINT_EN {
+ DCP_DC_LUT_DATA_G_FLOAT_POINT_EN_FALSE = 0x0,
+ DCP_DC_LUT_DATA_G_FLOAT_POINT_EN_TRUE = 0x1,
+} DCP_DC_LUT_DATA_G_FLOAT_POINT_EN;
+typedef enum DCP_DC_LUT_DATA_G_FORMAT {
+ DCP_DC_LUT_DATA_G_FORMAT_U0P10 = 0x0,
+ DCP_DC_LUT_DATA_G_FORMAT_S1P10 = 0x1,
+ DCP_DC_LUT_DATA_G_FORMAT_U1P11 = 0x2,
+ DCP_DC_LUT_DATA_G_FORMAT_U0P12 = 0x3,
+} DCP_DC_LUT_DATA_G_FORMAT;
+typedef enum DCP_DC_LUT_INC_R {
+ DCP_DC_LUT_INC_R_NA = 0x0,
+ DCP_DC_LUT_INC_R_2 = 0x1,
+ DCP_DC_LUT_INC_R_4 = 0x2,
+ DCP_DC_LUT_INC_R_8 = 0x3,
+ DCP_DC_LUT_INC_R_16 = 0x4,
+ DCP_DC_LUT_INC_R_32 = 0x5,
+ DCP_DC_LUT_INC_R_64 = 0x6,
+ DCP_DC_LUT_INC_R_128 = 0x7,
+ DCP_DC_LUT_INC_R_256 = 0x8,
+ DCP_DC_LUT_INC_R_512 = 0x9,
+} DCP_DC_LUT_INC_R;
+typedef enum DCP_DC_LUT_DATA_R_SIGNED_EN {
+ DCP_DC_LUT_DATA_R_SIGNED_EN_FALSE = 0x0,
+ DCP_DC_LUT_DATA_R_SIGNED_EN_TRUE = 0x1,
+} DCP_DC_LUT_DATA_R_SIGNED_EN;
+typedef enum DCP_DC_LUT_DATA_R_FLOAT_POINT_EN {
+ DCP_DC_LUT_DATA_R_FLOAT_POINT_EN_FALSE = 0x0,
+ DCP_DC_LUT_DATA_R_FLOAT_POINT_EN_TRUE = 0x1,
+} DCP_DC_LUT_DATA_R_FLOAT_POINT_EN;
+typedef enum DCP_DC_LUT_DATA_R_FORMAT {
+ DCP_DC_LUT_DATA_R_FORMAT_U0P10 = 0x0,
+ DCP_DC_LUT_DATA_R_FORMAT_S1P10 = 0x1,
+ DCP_DC_LUT_DATA_R_FORMAT_U1P11 = 0x2,
+ DCP_DC_LUT_DATA_R_FORMAT_U0P12 = 0x3,
+} DCP_DC_LUT_DATA_R_FORMAT;
+typedef enum DCP_CRC_ENABLE {
+ DCP_CRC_ENABLE_FALSE = 0x0,
+ DCP_CRC_ENABLE_TRUE = 0x1,
+} DCP_CRC_ENABLE;
+typedef enum DCP_CRC_SOURCE_SEL {
+ DCP_CRC_SOURCE_SEL_OUTPUT_PIX = 0x0,
+ DCP_CRC_SOURCE_SEL_INPUT_L32 = 0x1,
+ DCP_CRC_SOURCE_SEL_INPUT_H32 = 0x2,
+ DCP_CRC_SOURCE_SEL_OUTPUT_CNTL = 0x4,
+} DCP_CRC_SOURCE_SEL;
+typedef enum DCP_CRC_LINE_SEL {
+ DCP_CRC_LINE_SEL_RESERVED = 0x0,
+ DCP_CRC_LINE_SEL_EVEN = 0x1,
+ DCP_CRC_LINE_SEL_ODD = 0x2,
+ DCP_CRC_LINE_SEL_BOTH = 0x3,
+} DCP_CRC_LINE_SEL;
+typedef enum DCP_GRPH_FLIP_RATE {
+ DCP_GRPH_FLIP_RATE_1FRAME = 0x0,
+ DCP_GRPH_FLIP_RATE_2FRAME = 0x1,
+ DCP_GRPH_FLIP_RATE_3FRAME = 0x2,
+ DCP_GRPH_FLIP_RATE_4FRAME = 0x3,
+ DCP_GRPH_FLIP_RATE_5FRAME = 0x4,
+ DCP_GRPH_FLIP_RATE_6FRAME = 0x5,
+ DCP_GRPH_FLIP_RATE_7FRAME = 0x6,
+ DCP_GRPH_FLIP_RATE_8FRAME = 0x7,
+} DCP_GRPH_FLIP_RATE;
+typedef enum DCP_GRPH_FLIP_RATE_ENABLE {
+ DCP_GRPH_FLIP_RATE_ENABLE_FALSE = 0x0,
+ DCP_GRPH_FLIP_RATE_ENABLE_TRUE = 0x1,
+} DCP_GRPH_FLIP_RATE_ENABLE;
+typedef enum DCP_GSL0_EN {
+ DCP_GSL0_EN_FALSE = 0x0,
+ DCP_GSL0_EN_TRUE = 0x1,
+} DCP_GSL0_EN;
+typedef enum DCP_GSL1_EN {
+ DCP_GSL1_EN_FALSE = 0x0,
+ DCP_GSL1_EN_TRUE = 0x1,
+} DCP_GSL1_EN;
+typedef enum DCP_GSL2_EN {
+ DCP_GSL2_EN_FALSE = 0x0,
+ DCP_GSL2_EN_TRUE = 0x1,
+} DCP_GSL2_EN;
+typedef enum DCP_GSL_MASTER_EN {
+ DCP_GSL_MASTER_EN_FALSE = 0x0,
+ DCP_GSL_MASTER_EN_TRUE = 0x1,
+} DCP_GSL_MASTER_EN;
+typedef enum DCP_GSL_XDMA_GROUP {
+ DCP_GSL_XDMA_GROUP_VSYNC = 0x0,
+ DCP_GSL_XDMA_GROUP_HSYNC0 = 0x1,
+ DCP_GSL_XDMA_GROUP_HSYNC1 = 0x2,
+ DCP_GSL_XDMA_GROUP_HSYNC2 = 0x3,
+} DCP_GSL_XDMA_GROUP;
+typedef enum DCP_GSL_XDMA_GROUP_UNDERFLOW_EN {
+ DCP_GSL_XDMA_GROUP_UNDERFLOW_EN_FALSE = 0x0,
+ DCP_GSL_XDMA_GROUP_UNDERFLOW_EN_TRUE = 0x1,
+} DCP_GSL_XDMA_GROUP_UNDERFLOW_EN;
+typedef enum DCP_GSL_SYNC_SOURCE {
+ DCP_GSL_SYNC_SOURCE_FLIP = 0x0,
+ DCP_GSL_SYNC_SOURCE_PHASE0 = 0x1,
+ DCP_GSL_SYNC_SOURCE_RESET = 0x2,
+ DCP_GSL_SYNC_SOURCE_PHASE1 = 0x3,
+} DCP_GSL_SYNC_SOURCE;
+typedef enum DCP_GSL_DELAY_SURFACE_UPDATE_PENDING {
+ DCP_GSL_DELAY_SURFACE_UPDATE_PENDING_FALSE = 0x0,
+ DCP_GSL_DELAY_SURFACE_UPDATE_PENDING_TRUE = 0x1,
+} DCP_GSL_DELAY_SURFACE_UPDATE_PENDING;
+typedef enum DCP_TEST_DEBUG_WRITE_EN {
+ DCP_TEST_DEBUG_WRITE_EN_FALSE = 0x0,
+ DCP_TEST_DEBUG_WRITE_EN_TRUE = 0x1,
+} DCP_TEST_DEBUG_WRITE_EN;
+typedef enum DCP_GRPH_STEREOSYNC_FLIP_EN {
+ DCP_GRPH_STEREOSYNC_FLIP_EN_FALSE = 0x0,
+ DCP_GRPH_STEREOSYNC_FLIP_EN_TRUE = 0x1,
+} DCP_GRPH_STEREOSYNC_FLIP_EN;
+typedef enum DCP_GRPH_STEREOSYNC_FLIP_MODE {
+ DCP_GRPH_STEREOSYNC_FLIP_MODE_FLIP = 0x0,
+ DCP_GRPH_STEREOSYNC_FLIP_MODE_PHASE0 = 0x1,
+ DCP_GRPH_STEREOSYNC_FLIP_MODE_RESET = 0x2,
+ DCP_GRPH_STEREOSYNC_FLIP_MODE_PHASE1 = 0x3,
+} DCP_GRPH_STEREOSYNC_FLIP_MODE;
+typedef enum DCP_GRPH_STEREOSYNC_SELECT_DISABLE {
+ DCP_GRPH_STEREOSYNC_SELECT_DISABLE_FALSE = 0x0,
+ DCP_GRPH_STEREOSYNC_SELECT_DISABLE_TRUE = 0x1,
+} DCP_GRPH_STEREOSYNC_SELECT_DISABLE;
+typedef enum DCP_GRPH_ROTATION_ANGLE {
+ DCP_GRPH_ROTATION_ANGLE_0 = 0x0,
+ DCP_GRPH_ROTATION_ANGLE_90 = 0x1,
+ DCP_GRPH_ROTATION_ANGLE_180 = 0x2,
+ DCP_GRPH_ROTATION_ANGLE_270 = 0x3,
+} DCP_GRPH_ROTATION_ANGLE;
+typedef enum DCP_GRPH_XDMA_CACHE_UNDERFLOW_CNT_EN {
+ DCP_GRPH_XDMA_CACHE_UNDERFLOW_CNT_EN_FALSE = 0x0,
+ DCP_GRPH_XDMA_CACHE_UNDERFLOW_CNT_EN_TRUE = 0x1,
+} DCP_GRPH_XDMA_CACHE_UNDERFLOW_CNT_EN;
+typedef enum DCP_GRPH_XDMA_CACHE_UNDERFLOW_CNT_MODE {
+ DCP_GRPH_XDMA_CACHE_UNDERFLOW_CNT_MODE_RELY_NUM = 0x0,
+ DCP_GRPH_XDMA_CACHE_UNDERFLOW_CNT_MODE_RELY_ENABLE= 0x1,
+} DCP_GRPH_XDMA_CACHE_UNDERFLOW_CNT_MODE;
+typedef enum DCP_GRPH_REGAMMA_MODE {
+ DCP_GRPH_REGAMMA_MODE_BYPASS = 0x0,
+ DCP_GRPH_REGAMMA_MODE_SRGB = 0x1,
+ DCP_GRPH_REGAMMA_MODE_XVYCC = 0x2,
+ DCP_GRPH_REGAMMA_MODE_PROGA = 0x3,
+ DCP_GRPH_REGAMMA_MODE_PROGB = 0x4,
+} DCP_GRPH_REGAMMA_MODE;
+typedef enum DCP_ALPHA_ROUND_TRUNC_MODE {
+ DCP_ALPHA_ROUND_TRUNC_MODE_ROUND = 0x0,
+ DCP_ALPHA_ROUND_TRUNC_MODE_TRUNC = 0x1,
+} DCP_ALPHA_ROUND_TRUNC_MODE;
+typedef enum DCP_CURSOR_ALPHA_BLND_ENA {
+ DCP_CURSOR_ALPHA_BLND_ENA_FALSE = 0x0,
+ DCP_CURSOR_ALPHA_BLND_ENA_TRUE = 0x1,
+} DCP_CURSOR_ALPHA_BLND_ENA;
+typedef enum DCP_GRPH_XDMA_CACHE_UNDERFLOW_FRAME_MASK {
+ DCP_GRPH_XDMA_CACHE_UNDERFLOW_FRAME_MASK_FALSE = 0x0,
+ DCP_GRPH_XDMA_CACHE_UNDERFLOW_FRAME_MASK_TRUE = 0x1,
+} DCP_GRPH_XDMA_CACHE_UNDERFLOW_FRAME_MASK;
+typedef enum DCP_GRPH_XDMA_CACHE_UNDERFLOW_FRAME_ACK {
+ DCP_GRPH_XDMA_CACHE_UNDERFLOW_FRAME_ACK_FALSE = 0x0,
+ DCP_GRPH_XDMA_CACHE_UNDERFLOW_FRAME_ACK_TRUE = 0x1,
+} DCP_GRPH_XDMA_CACHE_UNDERFLOW_FRAME_ACK;
+typedef enum DCP_GRPH_XDMA_CACHE_UNDERFLOW_INT_MASK {
+ DCP_GRPH_XDMA_CACHE_UNDERFLOW_INT_MASK_FALSE = 0x0,
+ DCP_GRPH_XDMA_CACHE_UNDERFLOW_INT_MASK_TRUE = 0x1,
+} DCP_GRPH_XDMA_CACHE_UNDERFLOW_INT_MASK;
+typedef enum DCP_GRPH_XDMA_CACHE_UNDERFLOW_INT_ACK {
+ DCP_GRPH_XDMA_CACHE_UNDERFLOW_INT_ACK_FALSE = 0x0,
+ DCP_GRPH_XDMA_CACHE_UNDERFLOW_INT_ACK_TRUE = 0x1,
+} DCP_GRPH_XDMA_CACHE_UNDERFLOW_INT_ACK;
+typedef enum DCP_GRPH_SURFACE_COUNTER_EN {
+ DCP_GRPH_SURFACE_COUNTER_EN_DISABLE = 0x0,
+ DCP_GRPH_SURFACE_COUNTER_EN_ENABLE = 0x1,
+} DCP_GRPH_SURFACE_COUNTER_EN;
+typedef enum DCP_GRPH_SURFACE_COUNTER_EVENT_SELECT {
+ DCP_GRPH_SURFACE_COUNTER_EVENT_SELECT_0 = 0x0,
+ DCP_GRPH_SURFACE_COUNTER_EVENT_SELECT_1 = 0x1,
+ DCP_GRPH_SURFACE_COUNTER_EVENT_SELECT_2 = 0x2,
+ DCP_GRPH_SURFACE_COUNTER_EVENT_SELECT_3 = 0x3,
+ DCP_GRPH_SURFACE_COUNTER_EVENT_SELECT_4 = 0x4,
+ DCP_GRPH_SURFACE_COUNTER_EVENT_SELECT_5 = 0x5,
+ DCP_GRPH_SURFACE_COUNTER_EVENT_SELECT_6 = 0x6,
+ DCP_GRPH_SURFACE_COUNTER_EVENT_SELECT_7 = 0x7,
+ DCP_GRPH_SURFACE_COUNTER_EVENT_SELECT_8 = 0x8,
+ DCP_GRPH_SURFACE_COUNTER_EVENT_SELECT_9 = 0x9,
+ DCP_GRPH_SURFACE_COUNTER_EVENT_SELECT_10 = 0xa,
+ DCP_GRPH_SURFACE_COUNTER_EVENT_SELECT_11 = 0xb,
+} DCP_GRPH_SURFACE_COUNTER_EVENT_SELECT;
+typedef enum DCP_GRPH_SURFACE_COUNTER_ERR_WRAP_OCCURED {
+ DCP_GRPH_SURFACE_COUNTER_ERR_WRAP_OCCURED_NO = 0x0,
+ DCP_GRPH_SURFACE_COUNTER_ERR_WRAP_OCCURED_YES = 0x1,
+} DCP_GRPH_SURFACE_COUNTER_ERR_WRAP_OCCURED;
+typedef enum HDMI_KEEPOUT_MODE {
+ HDMI_KEEPOUT_0_650PIX_AFTER_VSYNC = 0x0,
+ HDMI_KEEPOUT_509_650PIX_AFTER_VSYNC = 0x1,
+} HDMI_KEEPOUT_MODE;
+typedef enum HDMI_CLOCK_CHANNEL_RATE {
+ HDMI_CLOCK_CHANNEL_FREQ_EQUAL_TO_CHAR_RATE = 0x0,
+ HDMI_CLOCK_CHANNEL_FREQ_QUARTER_TO_CHAR_RATE = 0x1,
+} HDMI_CLOCK_CHANNEL_RATE;
+typedef enum HDMI_NO_EXTRA_NULL_PACKET_FILLED {
+ HDMI_EXTRA_NULL_PACKET_FILLED_ENABLE = 0x0,
+ HDMI_EXTRA_NULL_PACKET_FILLED_DISABLE = 0x1,
+} HDMI_NO_EXTRA_NULL_PACKET_FILLED;
+typedef enum HDMI_PACKET_GEN_VERSION {
+ HDMI_PACKET_GEN_VERSION_OLD = 0x0,
+ HDMI_PACKET_GEN_VERSION_NEW = 0x1,
+} HDMI_PACKET_GEN_VERSION;
+typedef enum HDMI_ERROR_ACK {
+ HDMI_ERROR_ACK_INT = 0x0,
+ HDMI_ERROR_NOT_ACK = 0x1,
+} HDMI_ERROR_ACK;
+typedef enum HDMI_ERROR_MASK {
+ HDMI_ERROR_MASK_INT = 0x0,
+ HDMI_ERROR_NOT_MASK = 0x1,
+} HDMI_ERROR_MASK;
+typedef enum HDMI_DEEP_COLOR_DEPTH {
+ HDMI_DEEP_COLOR_DEPTH_24BPP = 0x0,
+ HDMI_DEEP_COLOR_DEPTH_30BPP = 0x1,
+ HDMI_DEEP_COLOR_DEPTH_36BPP = 0x2,
+ HDMI_DEEP_COLOR_DEPTH_RESERVED = 0x3,
+} HDMI_DEEP_COLOR_DEPTH;
+typedef enum HDMI_AUDIO_DELAY_EN {
+ HDMI_AUDIO_DELAY_DISABLE = 0x0,
+ HDMI_AUDIO_DELAY_58CLK = 0x1,
+ HDMI_AUDIO_DELAY_56CLK = 0x2,
+ HDMI_AUDIO_DELAY_RESERVED = 0x3,
+} HDMI_AUDIO_DELAY_EN;
+typedef enum HDMI_AUDIO_SEND_MAX_PACKETS {
+ HDMI_NOT_SEND_MAX_AUDIO_PACKETS = 0x0,
+ HDMI_SEND_MAX_AUDIO_PACKETS = 0x1,
+} HDMI_AUDIO_SEND_MAX_PACKETS;
+typedef enum HDMI_ACR_SEND {
+ HDMI_ACR_NOT_SEND = 0x0,
+ HDMI_ACR_PKT_SEND = 0x1,
+} HDMI_ACR_SEND;
+typedef enum HDMI_ACR_CONT {
+ HDMI_ACR_CONT_DISABLE = 0x0,
+ HDMI_ACR_CONT_ENABLE = 0x1,
+} HDMI_ACR_CONT;
+typedef enum HDMI_ACR_SELECT {
+ HDMI_ACR_SELECT_HW = 0x0,
+ HDMI_ACR_SELECT_32K = 0x1,
+ HDMI_ACR_SELECT_44K = 0x2,
+ HDMI_ACR_SELECT_48K = 0x3,
+} HDMI_ACR_SELECT;
+typedef enum HDMI_ACR_SOURCE {
+ HDMI_ACR_SOURCE_HW = 0x0,
+ HDMI_ACR_SOURCE_SW = 0x1,
+} HDMI_ACR_SOURCE;
+typedef enum HDMI_ACR_N_MULTIPLE {
+ HDMI_ACR_0_MULTIPLE_RESERVED = 0x0,
+ HDMI_ACR_1_MULTIPLE = 0x1,
+ HDMI_ACR_2_MULTIPLE = 0x2,
+ HDMI_ACR_3_MULTIPLE_RESERVED = 0x3,
+ HDMI_ACR_4_MULTIPLE = 0x4,
+ HDMI_ACR_5_MULTIPLE_RESERVED = 0x5,
+ HDMI_ACR_6_MULTIPLE_RESERVED = 0x6,
+ HDMI_ACR_7_MULTIPLE_RESERVED = 0x7,
+} HDMI_ACR_N_MULTIPLE;
+typedef enum HDMI_ACR_AUDIO_PRIORITY {
+ HDMI_ACR_PKT_HIGH_PRIORITY_THAN_AUDIO_SAMPLE = 0x0,
+ HDMI_AUDIO_SAMPLE_HIGH_PRIORITY_THAN_ACR_PKT = 0x1,
+} HDMI_ACR_AUDIO_PRIORITY;
+typedef enum HDMI_NULL_SEND {
+ HDMI_NULL_NOT_SEND = 0x0,
+ HDMI_NULL_PKT_SEND = 0x1,
+} HDMI_NULL_SEND;
+typedef enum HDMI_GC_SEND {
+ HDMI_GC_NOT_SEND = 0x0,
+ HDMI_GC_PKT_SEND = 0x1,
+} HDMI_GC_SEND;
+typedef enum HDMI_GC_CONT {
+ HDMI_GC_CONT_DISABLE = 0x0,
+ HDMI_GC_CONT_ENABLE = 0x1,
+} HDMI_GC_CONT;
+typedef enum HDMI_ISRC_SEND {
+ HDMI_ISRC_NOT_SEND = 0x0,
+ HDMI_ISRC_PKT_SEND = 0x1,
+} HDMI_ISRC_SEND;
+typedef enum HDMI_ISRC_CONT {
+ HDMI_ISRC_CONT_DISABLE = 0x0,
+ HDMI_ISRC_CONT_ENABLE = 0x1,
+} HDMI_ISRC_CONT;
+typedef enum HDMI_AVI_INFO_SEND {
+ HDMI_AVI_INFO_NOT_SEND = 0x0,
+ HDMI_AVI_INFO_PKT_SEND = 0x1,
+} HDMI_AVI_INFO_SEND;
+typedef enum HDMI_AVI_INFO_CONT {
+ HDMI_AVI_INFO_CONT_DISABLE = 0x0,
+ HDMI_AVI_INFO_CONT_ENABLE = 0x1,
+} HDMI_AVI_INFO_CONT;
+typedef enum HDMI_AUDIO_INFO_SEND {
+ HDMI_AUDIO_INFO_NOT_SEND = 0x0,
+ HDMI_AUDIO_INFO_PKT_SEND = 0x1,
+} HDMI_AUDIO_INFO_SEND;
+typedef enum HDMI_AUDIO_INFO_CONT {
+ HDMI_AUDIO_INFO_CONT_DISABLE = 0x0,
+ HDMI_AUDIO_INFO_CONT_ENABLE = 0x1,
+} HDMI_AUDIO_INFO_CONT;
+typedef enum HDMI_MPEG_INFO_SEND {
+ HDMI_MPEG_INFO_NOT_SEND = 0x0,
+ HDMI_MPEG_INFO_PKT_SEND = 0x1,
+} HDMI_MPEG_INFO_SEND;
+typedef enum HDMI_MPEG_INFO_CONT {
+ HDMI_MPEG_INFO_CONT_DISABLE = 0x0,
+ HDMI_MPEG_INFO_CONT_ENABLE = 0x1,
+} HDMI_MPEG_INFO_CONT;
+typedef enum HDMI_GENERIC0_SEND {
+ HDMI_GENERIC0_NOT_SEND = 0x0,
+ HDMI_GENERIC0_PKT_SEND = 0x1,
+} HDMI_GENERIC0_SEND;
+typedef enum HDMI_GENERIC0_CONT {
+ HDMI_GENERIC0_CONT_DISABLE = 0x0,
+ HDMI_GENERIC0_CONT_ENABLE = 0x1,
+} HDMI_GENERIC0_CONT;
+typedef enum HDMI_GENERIC1_SEND {
+ HDMI_GENERIC1_NOT_SEND = 0x0,
+ HDMI_GENERIC1_PKT_SEND = 0x1,
+} HDMI_GENERIC1_SEND;
+typedef enum HDMI_GENERIC1_CONT {
+ HDMI_GENERIC1_CONT_DISABLE = 0x0,
+ HDMI_GENERIC1_CONT_ENABLE = 0x1,
+} HDMI_GENERIC1_CONT;
+typedef enum HDMI_GC_AVMUTE_CONT {
+ HDMI_GC_AVMUTE_CONT_DISABLE = 0x0,
+ HDMI_GC_AVMUTE_CONT_ENABLE = 0x1,
+} HDMI_GC_AVMUTE_CONT;
+typedef enum HDMI_PACKING_PHASE_OVERRIDE {
+ HDMI_PACKING_PHASE_SET_BY_HW = 0x0,
+ HDMI_PACKING_PHASE_SET_BY_SW = 0x1,
+} HDMI_PACKING_PHASE_OVERRIDE;
+typedef enum HDMI_GENERIC2_SEND {
+ HDMI_GENERIC2_NOT_SEND = 0x0,
+ HDMI_GENERIC2_PKT_SEND = 0x1,
+} HDMI_GENERIC2_SEND;
+typedef enum HDMI_GENERIC2_CONT {
+ HDMI_GENERIC2_CONT_DISABLE = 0x0,
+ HDMI_GENERIC2_CONT_ENABLE = 0x1,
+} HDMI_GENERIC2_CONT;
+typedef enum HDMI_GENERIC3_SEND {
+ HDMI_GENERIC3_NOT_SEND = 0x0,
+ HDMI_GENERIC3_PKT_SEND = 0x1,
+} HDMI_GENERIC3_SEND;
+typedef enum HDMI_GENERIC3_CONT {
+ HDMI_GENERIC3_CONT_DISABLE = 0x0,
+ HDMI_GENERIC3_CONT_ENABLE = 0x1,
+} HDMI_GENERIC3_CONT;
+typedef enum TMDS_PIXEL_ENCODING {
+ TMDS_PIXEL_ENCODING_444_OR_420 = 0x0,
+ TMDS_PIXEL_ENCODING_422 = 0x1,
+} TMDS_PIXEL_ENCODING;
+typedef enum TMDS_COLOR_FORMAT {
+ TMDS_COLOR_FORMAT__24BPP__TWIN30BPP_MSB__DUAL48BPP= 0x0,
+ TMDS_COLOR_FORMAT_TWIN30BPP_LSB = 0x1,
+ TMDS_COLOR_FORMAT_DUAL30BPP = 0x2,
+ TMDS_COLOR_FORMAT_RESERVED = 0x3,
+} TMDS_COLOR_FORMAT;
+typedef enum TMDS_STEREOSYNC_CTL_SEL_REG {
+ TMDS_STEREOSYNC_CTL0 = 0x0,
+ TMDS_STEREOSYNC_CTL1 = 0x1,
+ TMDS_STEREOSYNC_CTL2 = 0x2,
+ TMDS_STEREOSYNC_CTL3 = 0x3,
+} TMDS_STEREOSYNC_CTL_SEL_REG;
+typedef enum TMDS_CTL0_DATA_SEL {
+ TMDS_CTL0_DATA_SEL0_RESERVED = 0x0,
+ TMDS_CTL0_DATA_SEL1_DISPLAY_ENABLE = 0x1,
+ TMDS_CTL0_DATA_SEL2_VSYNC = 0x2,
+ TMDS_CTL0_DATA_SEL3_RESERVED = 0x3,
+ TMDS_CTL0_DATA_SEL4_HSYNC = 0x4,
+ TMDS_CTL0_DATA_SEL5_SEL7_RESERVED = 0x5,
+ TMDS_CTL0_DATA_SEL8_RANDOM_DATA = 0x6,
+ TMDS_CTL0_DATA_SEL9_SEL15_RANDOM_DATA = 0x7,
+} TMDS_CTL0_DATA_SEL;
+typedef enum TMDS_CTL0_DATA_INVERT {
+ TMDS_CTL0_DATA_NORMAL = 0x0,
+ TMDS_CTL0_DATA_INVERT_EN = 0x1,
+} TMDS_CTL0_DATA_INVERT;
+typedef enum TMDS_CTL0_DATA_MODULATION {
+ TMDS_CTL0_DATA_MODULATION_DISABLE = 0x0,
+ TMDS_CTL0_DATA_MODULATION_BIT0 = 0x1,
+ TMDS_CTL0_DATA_MODULATION_BIT1 = 0x2,
+ TMDS_CTL0_DATA_MODULATION_BIT2 = 0x3,
+} TMDS_CTL0_DATA_MODULATION;
+typedef enum TMDS_CTL0_PATTERN_OUT_EN {
+ TMDS_CTL0_PATTERN_OUT_DISABLE = 0x0,
+ TMDS_CTL0_PATTERN_OUT_ENABLE = 0x1,
+} TMDS_CTL0_PATTERN_OUT_EN;
+typedef enum TMDS_CTL1_DATA_SEL {
+ TMDS_CTL1_DATA_SEL0_RESERVED = 0x0,
+ TMDS_CTL1_DATA_SEL1_DISPLAY_ENABLE = 0x1,
+ TMDS_CTL1_DATA_SEL2_VSYNC = 0x2,
+ TMDS_CTL1_DATA_SEL3_RESERVED = 0x3,
+ TMDS_CTL1_DATA_SEL4_HSYNC = 0x4,
+ TMDS_CTL1_DATA_SEL5_SEL7_RESERVED = 0x5,
+ TMDS_CTL1_DATA_SEL8_BLANK_TIME = 0x6,
+ TMDS_CTL1_DATA_SEL9_SEL15_RESERVED = 0x7,
+} TMDS_CTL1_DATA_SEL;
+typedef enum TMDS_CTL1_DATA_INVERT {
+ TMDS_CTL1_DATA_NORMAL = 0x0,
+ TMDS_CTL1_DATA_INVERT_EN = 0x1,
+} TMDS_CTL1_DATA_INVERT;
+typedef enum TMDS_CTL1_DATA_MODULATION {
+ TMDS_CTL1_DATA_MODULATION_DISABLE = 0x0,
+ TMDS_CTL1_DATA_MODULATION_BIT0 = 0x1,
+ TMDS_CTL1_DATA_MODULATION_BIT1 = 0x2,
+ TMDS_CTL1_DATA_MODULATION_BIT2 = 0x3,
+} TMDS_CTL1_DATA_MODULATION;
+typedef enum TMDS_CTL1_PATTERN_OUT_EN {
+ TMDS_CTL1_PATTERN_OUT_DISABLE = 0x0,
+ TMDS_CTL1_PATTERN_OUT_ENABLE = 0x1,
+} TMDS_CTL1_PATTERN_OUT_EN;
+typedef enum TMDS_CTL2_DATA_SEL {
+ TMDS_CTL2_DATA_SEL0_RESERVED = 0x0,
+ TMDS_CTL2_DATA_SEL1_DISPLAY_ENABLE = 0x1,
+ TMDS_CTL2_DATA_SEL2_VSYNC = 0x2,
+ TMDS_CTL2_DATA_SEL3_RESERVED = 0x3,
+ TMDS_CTL2_DATA_SEL4_HSYNC = 0x4,
+ TMDS_CTL2_DATA_SEL5_SEL7_RESERVED = 0x5,
+ TMDS_CTL2_DATA_SEL8_BLANK_TIME = 0x6,
+ TMDS_CTL2_DATA_SEL9_SEL15_RESERVED = 0x7,
+} TMDS_CTL2_DATA_SEL;
+typedef enum TMDS_CTL2_DATA_INVERT {
+ TMDS_CTL2_DATA_NORMAL = 0x0,
+ TMDS_CTL2_DATA_INVERT_EN = 0x1,
+} TMDS_CTL2_DATA_INVERT;
+typedef enum TMDS_CTL2_DATA_MODULATION {
+ TMDS_CTL2_DATA_MODULATION_DISABLE = 0x0,
+ TMDS_CTL2_DATA_MODULATION_BIT0 = 0x1,
+ TMDS_CTL2_DATA_MODULATION_BIT1 = 0x2,
+ TMDS_CTL2_DATA_MODULATION_BIT2 = 0x3,
+} TMDS_CTL2_DATA_MODULATION;
+typedef enum TMDS_CTL2_PATTERN_OUT_EN {
+ TMDS_CTL2_PATTERN_OUT_DISABLE = 0x0,
+ TMDS_CTL2_PATTERN_OUT_ENABLE = 0x1,
+} TMDS_CTL2_PATTERN_OUT_EN;
+typedef enum TMDS_CTL3_DATA_INVERT {
+ TMDS_CTL3_DATA_NORMAL = 0x0,
+ TMDS_CTL3_DATA_INVERT_EN = 0x1,
+} TMDS_CTL3_DATA_INVERT;
+typedef enum TMDS_CTL3_DATA_MODULATION {
+ TMDS_CTL3_DATA_MODULATION_DISABLE = 0x0,
+ TMDS_CTL3_DATA_MODULATION_BIT0 = 0x1,
+ TMDS_CTL3_DATA_MODULATION_BIT1 = 0x2,
+ TMDS_CTL3_DATA_MODULATION_BIT2 = 0x3,
+} TMDS_CTL3_DATA_MODULATION;
+typedef enum TMDS_CTL3_PATTERN_OUT_EN {
+ TMDS_CTL3_PATTERN_OUT_DISABLE = 0x0,
+ TMDS_CTL3_PATTERN_OUT_ENABLE = 0x1,
+} TMDS_CTL3_PATTERN_OUT_EN;
+typedef enum TMDS_CTL3_DATA_SEL {
+ TMDS_CTL3_DATA_SEL0_RESERVED = 0x0,
+ TMDS_CTL3_DATA_SEL1_DISPLAY_ENABLE = 0x1,
+ TMDS_CTL3_DATA_SEL2_VSYNC = 0x2,
+ TMDS_CTL3_DATA_SEL3_RESERVED = 0x3,
+ TMDS_CTL3_DATA_SEL4_HSYNC = 0x4,
+ TMDS_CTL3_DATA_SEL5_SEL7_RESERVED = 0x5,
+ TMDS_CTL3_DATA_SEL8_BLANK_TIME = 0x6,
+ TMDS_CTL3_DATA_SEL9_SEL15_RESERVED = 0x7,
+} TMDS_CTL3_DATA_SEL;
+typedef enum DIG_FE_CNTL_SOURCE_SELECT {
+ DIG_FE_SOURCE_FROM_FMT0 = 0x0,
+ DIG_FE_SOURCE_FROM_FMT1 = 0x1,
+ DIG_FE_SOURCE_FROM_FMT2 = 0x2,
+ DIG_FE_SOURCE_FROM_FMT3 = 0x3,
+ DIG_FE_SOURCE_FROM_FMT4 = 0x4,
+ DIG_FE_SOURCE_FROM_FMT5 = 0x5,
+} DIG_FE_CNTL_SOURCE_SELECT;
+typedef enum DIG_FE_CNTL_STEREOSYNC_SELECT {
+ DIG_FE_STEREOSYNC_FROM_FMT0 = 0x0,
+ DIG_FE_STEREOSYNC_FROM_FMT1 = 0x1,
+ DIG_FE_STEREOSYNC_FROM_FMT2 = 0x2,
+ DIG_FE_STEREOSYNC_FROM_FMT3 = 0x3,
+ DIG_FE_STEREOSYNC_FROM_FMT4 = 0x4,
+ DIG_FE_STEREOSYNC_FROM_FMT5 = 0x5,
+} DIG_FE_CNTL_STEREOSYNC_SELECT;
+typedef enum DIG_FIFO_READ_CLOCK_SRC {
+ DIG_FIFO_READ_CLOCK_SRC_FROM_DCCG = 0x0,
+ DIG_FIFO_READ_CLOCK_SRC_FROM_DISPLAY_PIPE = 0x1,
+} DIG_FIFO_READ_CLOCK_SRC;
+typedef enum DIG_OUTPUT_CRC_CNTL_LINK_SEL {
+ DIG_OUTPUT_CRC_ON_LINK0 = 0x0,
+ DIG_OUTPUT_CRC_ON_LINK1 = 0x1,
+} DIG_OUTPUT_CRC_CNTL_LINK_SEL;
+typedef enum DIG_OUTPUT_CRC_DATA_SEL {
+ DIG_OUTPUT_CRC_FOR_FULLFRAME = 0x0,
+ DIG_OUTPUT_CRC_FOR_ACTIVEONLY = 0x1,
+ DIG_OUTPUT_CRC_FOR_VBI = 0x2,
+ DIG_OUTPUT_CRC_FOR_AUDIO = 0x3,
+} DIG_OUTPUT_CRC_DATA_SEL;
+typedef enum DIG_TEST_PATTERN_TEST_PATTERN_OUT_EN {
+ DIG_IN_NORMAL_OPERATION = 0x0,
+ DIG_IN_DEBUG_MODE = 0x1,
+} DIG_TEST_PATTERN_TEST_PATTERN_OUT_EN;
+typedef enum DIG_TEST_PATTERN_HALF_CLOCK_PATTERN_SEL {
+ DIG_10BIT_TEST_PATTERN = 0x0,
+ DIG_ALTERNATING_TEST_PATTERN = 0x1,
+} DIG_TEST_PATTERN_HALF_CLOCK_PATTERN_SEL;
+typedef enum DIG_TEST_PATTERN_RANDOM_PATTERN_OUT_EN {
+ DIG_TEST_PATTERN_NORMAL = 0x0,
+ DIG_TEST_PATTERN_RANDOM = 0x1,
+} DIG_TEST_PATTERN_RANDOM_PATTERN_OUT_EN;
+typedef enum DIG_TEST_PATTERN_RANDOM_PATTERN_RESET {
+ DIG_RANDOM_PATTERN_ENABLED = 0x0,
+ DIG_RANDOM_PATTERN_RESETED = 0x1,
+} DIG_TEST_PATTERN_RANDOM_PATTERN_RESET;
+typedef enum DIG_TEST_PATTERN_EXTERNAL_RESET_EN {
+ DIG_TEST_PATTERN_EXTERNAL_RESET_ENABLE = 0x0,
+ DIG_TEST_PATTERN_EXTERNAL_RESET_BY_EXT_SIG = 0x1,
+} DIG_TEST_PATTERN_EXTERNAL_RESET_EN;
+typedef enum DIG_RANDOM_PATTERN_SEED_RAN_PAT {
+ DIG_RANDOM_PATTERN_SEED_RAN_PAT_ALL_PIXELS = 0x0,
+ DIG_RANDOM_PATTERN_SEED_RAN_PAT_DE_HIGH = 0x1,
+} DIG_RANDOM_PATTERN_SEED_RAN_PAT;
+typedef enum DIG_FIFO_STATUS_USE_OVERWRITE_LEVEL {
+ DIG_FIFO_USE_OVERWRITE_LEVEL = 0x0,
+ DIG_FIFO_USE_CAL_AVERAGE_LEVEL = 0x1,
+} DIG_FIFO_STATUS_USE_OVERWRITE_LEVEL;
+typedef enum DIG_FIFO_ERROR_ACK {
+ DIG_FIFO_ERROR_ACK_INT = 0x0,
+ DIG_FIFO_ERROR_NOT_ACK = 0x1,
+} DIG_FIFO_ERROR_ACK;
+typedef enum DIG_FIFO_STATUS_FORCE_RECAL_AVERAGE {
+ DIG_FIFO_NOT_FORCE_RECAL_AVERAGE = 0x0,
+ DIG_FIFO_FORCE_RECAL_AVERAGE_LEVEL = 0x1,
+} DIG_FIFO_STATUS_FORCE_RECAL_AVERAGE;
+typedef enum DIG_FIFO_STATUS_FORCE_RECOMP_MINMAX {
+ DIG_FIFO_NOT_FORCE_RECOMP_MINMAX = 0x0,
+ DIG_FIFO_FORCE_RECOMP_MINMAX = 0x1,
+} DIG_FIFO_STATUS_FORCE_RECOMP_MINMAX;
+typedef enum DIG_DISPCLK_SWITCH_CNTL_SWITCH_POINT {
+ DIG_DISPCLK_SWITCH_AT_EARLY_VBLANK = 0x0,
+ DIG_DISPCLK_SWITCH_AT_FIRST_HSYNC = 0x1,
+} DIG_DISPCLK_SWITCH_CNTL_SWITCH_POINT;
+typedef enum DIG_DISPCLK_SWITCH_ALLOWED_INT_ACK {
+ DIG_DISPCLK_SWITCH_ALLOWED_ACK_INT = 0x0,
+ DIG_DISPCLK_SWITCH_ALLOWED_INT_NOT_ACK = 0x1,
+} DIG_DISPCLK_SWITCH_ALLOWED_INT_ACK;
+typedef enum DIG_DISPCLK_SWITCH_ALLOWED_INT_MASK {
+ DIG_DISPCLK_SWITCH_ALLOWED_MASK_INT = 0x0,
+ DIG_DISPCLK_SWITCH_ALLOWED_INT_UNMASK = 0x1,
+} DIG_DISPCLK_SWITCH_ALLOWED_INT_MASK;
+typedef enum AFMT_INTERRUPT_STATUS_CHG_MASK {
+ AFMT_INTERRUPT_DISABLE = 0x0,
+ AFMT_INTERRUPT_ENABLE = 0x1,
+} AFMT_INTERRUPT_STATUS_CHG_MASK;
+typedef enum HDMI_GC_AVMUTE {
+ HDMI_GC_AVMUTE_SET = 0x0,
+ HDMI_GC_AVMUTE_UNSET = 0x1,
+} HDMI_GC_AVMUTE;
+typedef enum HDMI_DEFAULT_PAHSE {
+ HDMI_DEFAULT_PHASE_IS_0 = 0x0,
+ HDMI_DEFAULT_PHASE_IS_1 = 0x1,
+} HDMI_DEFAULT_PAHSE;
+typedef enum AFMT_AUDIO_PACKET_CONTROL2_AUDIO_LAYOUT_OVRD {
+ AFMT_AUDIO_LAYOUT_DETERMINED_BY_AZ_AUDIO_CHANNEL_STATUS= 0x0,
+ AFMT_AUDIO_LAYOUT_OVRD_BY_REGISTER = 0x1,
+} AFMT_AUDIO_PACKET_CONTROL2_AUDIO_LAYOUT_OVRD;
+typedef enum AUDIO_LAYOUT_SELECT {
+ AUDIO_LAYOUT_0 = 0x0,
+ AUDIO_LAYOUT_1 = 0x1,
+} AUDIO_LAYOUT_SELECT;
+typedef enum AFMT_AUDIO_CRC_CONTROL_CONT {
+ AFMT_AUDIO_CRC_ONESHOT = 0x0,
+ AFMT_AUDIO_CRC_AUTO_RESTART = 0x1,
+} AFMT_AUDIO_CRC_CONTROL_CONT;
+typedef enum AFMT_AUDIO_CRC_CONTROL_SOURCE {
+ AFMT_AUDIO_CRC_SOURCE_FROM_FIFO_INPUT = 0x0,
+ AFMT_AUDIO_CRC_SOURCE_FROM_FIFO_OUTPUT = 0x1,
+} AFMT_AUDIO_CRC_CONTROL_SOURCE;
+typedef enum AFMT_AUDIO_CRC_CONTROL_CH_SEL {
+ AFMT_AUDIO_CRC_CH0_SIG = 0x0,
+ AFMT_AUDIO_CRC_CH1_SIG = 0x1,
+ AFMT_AUDIO_CRC_CH2_SIG = 0x2,
+ AFMT_AUDIO_CRC_CH3_SIG = 0x3,
+ AFMT_AUDIO_CRC_CH4_SIG = 0x4,
+ AFMT_AUDIO_CRC_CH5_SIG = 0x5,
+ AFMT_AUDIO_CRC_CH6_SIG = 0x6,
+ AFMT_AUDIO_CRC_CH7_SIG = 0x7,
+ AFMT_AUDIO_CRC_RESERVED = 0x8,
+ AFMT_AUDIO_CRC_AUDIO_SAMPLE_COUNT = 0x9,
+} AFMT_AUDIO_CRC_CONTROL_CH_SEL;
+typedef enum AFMT_RAMP_CONTROL0_SIGN {
+ AFMT_RAMP_SIGNED = 0x0,
+ AFMT_RAMP_UNSIGNED = 0x1,
+} AFMT_RAMP_CONTROL0_SIGN;
+typedef enum AFMT_AUDIO_PACKET_CONTROL_AUDIO_SAMPLE_SEND {
+ AFMT_AUDIO_PACKET_SENT_DISABLED = 0x0,
+ AFMT_AUDIO_PACKET_SENT_ENABLED = 0x1,
+} AFMT_AUDIO_PACKET_CONTROL_AUDIO_SAMPLE_SEND;
+typedef enum AFMT_AUDIO_PACKET_CONTROL_RESET_FIFO_WHEN_AUDIO_DIS {
+ AFMT_NOT_RESET_AUDIO_FIFO_WHEN_AUDIO_DISABLED_RESERVED= 0x0,
+ AFMT_RESET_AUDIO_FIFO_WHEN_AUDIO_DISABLED = 0x1,
+} AFMT_AUDIO_PACKET_CONTROL_RESET_FIFO_WHEN_AUDIO_DIS;
+typedef enum AFMT_INFOFRAME_CONTROL0_AUDIO_INFO_SOURCE {
+ AFMT_INFOFRAME_SOURCE_FROM_AZALIA_BLOCK = 0x0,
+ AFMT_INFOFRAME_SOURCE_FROM_AFMT_REGISTERS = 0x1,
+} AFMT_INFOFRAME_CONTROL0_AUDIO_INFO_SOURCE;
+typedef enum AFMT_AUDIO_SRC_CONTROL_SELECT {
+ AFMT_AUDIO_SRC_FROM_AZ_STREAM0 = 0x0,
+ AFMT_AUDIO_SRC_FROM_AZ_STREAM1 = 0x1,
+ AFMT_AUDIO_SRC_FROM_AZ_STREAM2 = 0x2,
+ AFMT_AUDIO_SRC_FROM_AZ_STREAM3 = 0x3,
+ AFMT_AUDIO_SRC_FROM_AZ_STREAM4 = 0x4,
+ AFMT_AUDIO_SRC_FROM_AZ_STREAM5 = 0x5,
+ AFMT_AUDIO_SRC_RESERVED = 0x6,
+} AFMT_AUDIO_SRC_CONTROL_SELECT;
+typedef enum DIG_BE_CNTL_MODE {
+ DIG_BE_DP_SST_MODE = 0x0,
+ DIG_BE_RESERVED1 = 0x1,
+ DIG_BE_TMDS_DVI_MODE = 0x2,
+ DIG_BE_TMDS_HDMI_MODE = 0x3,
+ DIG_BE_SDVO_RESERVED = 0x4,
+ DIG_BE_DP_MST_MODE = 0x5,
+ DIG_BE_RESERVED2 = 0x6,
+ DIG_BE_RESERVED3 = 0x7,
+} DIG_BE_CNTL_MODE;
+typedef enum DIG_BE_CNTL_HPD_SELECT {
+ DIG_BE_CNTL_HPD1 = 0x0,
+ DIG_BE_CNTL_HPD2 = 0x1,
+ DIG_BE_CNTL_HPD3 = 0x2,
+ DIG_BE_CNTL_HPD4 = 0x3,
+ DIG_BE_CNTL_HPD5 = 0x4,
+ DIG_BE_CNTL_HPD6 = 0x5,
+} DIG_BE_CNTL_HPD_SELECT;
+typedef enum LVTMA_RANDOM_PATTERN_SEED_RAN_PAT {
+ LVTMA_RANDOM_PATTERN_SEED_ALL_PIXELS = 0x0,
+ LVTMA_RANDOM_PATTERN_SEED_ONLY_DE_HIGH = 0x1,
+} LVTMA_RANDOM_PATTERN_SEED_RAN_PAT;
+typedef enum TMDS_SYNC_PHASE {
+ TMDS_NOT_SYNC_PHASE_ON_FRAME_START = 0x0,
+ TMDS_SYNC_PHASE_ON_FRAME_START = 0x1,
+} TMDS_SYNC_PHASE;
+typedef enum TMDS_DATA_SYNCHRONIZATION_DSINTSEL {
+ TMDS_DATA_SYNCHRONIZATION_DSINTSEL_PCLK_TMDS = 0x0,
+ TMDS_DATA_SYNCHRONIZATION_DSINTSEL_TMDS_PLL = 0x1,
+} TMDS_DATA_SYNCHRONIZATION_DSINTSEL;
+typedef enum TMDS_TRANSMITTER_ENABLE_HPD_MASK {
+ TMDS_TRANSMITTER_HPD_MASK_NOT_OVERRIDE = 0x0,
+ TMDS_TRANSMITTER_HPD_MASK_OVERRIDE = 0x1,
+} TMDS_TRANSMITTER_ENABLE_HPD_MASK;
+typedef enum TMDS_TRANSMITTER_ENABLE_LNKCEN_HPD_MASK {
+ TMDS_TRANSMITTER_LNKCEN_HPD_MASK_NOT_OVERRIDE = 0x0,
+ TMDS_TRANSMITTER_LNKCEN_HPD_MASK_OVERRIDE = 0x1,
+} TMDS_TRANSMITTER_ENABLE_LNKCEN_HPD_MASK;
+typedef enum TMDS_TRANSMITTER_ENABLE_LNKDEN_HPD_MASK {
+ TMDS_TRANSMITTER_LNKDEN_HPD_MASK_NOT_OVERRIDE = 0x0,
+ TMDS_TRANSMITTER_LNKDEN_HPD_MASK_OVERRIDE = 0x1,
+} TMDS_TRANSMITTER_ENABLE_LNKDEN_HPD_MASK;
+typedef enum TMDS_TRANSMITTER_CONTROL_PLL_ENABLE_HPD_MASK {
+ TMDS_TRANSMITTER_HPD_NOT_OVERRIDE_PLL_ENABLE = 0x0,
+ TMDS_TRANSMITTER_HPD_OVERRIDE_PLL_ENABLE_ON_DISCON= 0x1,
+ TMDS_TRANSMITTER_HPD_OVERRIDE_PLL_ENABLE_ON_CON = 0x2,
+ TMDS_TRANSMITTER_HPD_OVERRIDE_PLL_ENABLE = 0x3,
+} TMDS_TRANSMITTER_CONTROL_PLL_ENABLE_HPD_MASK;
+typedef enum TMDS_TRANSMITTER_CONTROL_IDSCKSELA {
+ TMDS_TRANSMITTER_IDSCKSELA_USE_IPIXCLK = 0x0,
+ TMDS_TRANSMITTER_IDSCKSELA_USE_IDCLK = 0x1,
+} TMDS_TRANSMITTER_CONTROL_IDSCKSELA;
+typedef enum TMDS_TRANSMITTER_CONTROL_IDSCKSELB {
+ TMDS_TRANSMITTER_IDSCKSELB_USE_IPIXCLK = 0x0,
+ TMDS_TRANSMITTER_IDSCKSELB_USE_IDCLK = 0x1,
+} TMDS_TRANSMITTER_CONTROL_IDSCKSELB;
+typedef enum TMDS_TRANSMITTER_CONTROL_PLL_PWRUP_SEQ_EN {
+ TMDS_TRANSMITTER_PLL_PWRUP_SEQ_DISABLE = 0x0,
+ TMDS_TRANSMITTER_PLL_PWRUP_SEQ_ENABLE = 0x1,
+} TMDS_TRANSMITTER_CONTROL_PLL_PWRUP_SEQ_EN;
+typedef enum TMDS_TRANSMITTER_CONTROL_PLL_RESET_HPD_MASK {
+ TMDS_TRANSMITTER_PLL_NOT_RST_ON_HPD = 0x0,
+ TMDS_TRANSMITTER_PLL_RST_ON_HPD = 0x1,
+} TMDS_TRANSMITTER_CONTROL_PLL_RESET_HPD_MASK;
+typedef enum TMDS_TRANSMITTER_CONTROL_TMCLK_FROM_PADS {
+ TMDS_TRANSMITTER_TMCLK_FROM_TMDS_TMCLK = 0x0,
+ TMDS_TRANSMITTER_TMCLK_FROM_PADS = 0x1,
+} TMDS_TRANSMITTER_CONTROL_TMCLK_FROM_PADS;
+typedef enum TMDS_TRANSMITTER_CONTROL_TDCLK_FROM_PADS {
+ TMDS_TRANSMITTER_TDCLK_FROM_TMDS_TDCLK = 0x0,
+ TMDS_TRANSMITTER_TDCLK_FROM_PADS = 0x1,
+} TMDS_TRANSMITTER_CONTROL_TDCLK_FROM_PADS;
+typedef enum TMDS_TRANSMITTER_CONTROL_PLLSEL_OVERWRITE_EN {
+ TMDS_TRANSMITTER_PLLSEL_BY_HW = 0x0,
+ TMDS_TRANSMITTER_PLLSEL_OVERWRITE_BY_SW = 0x1,
+} TMDS_TRANSMITTER_CONTROL_PLLSEL_OVERWRITE_EN;
+typedef enum TMDS_TRANSMITTER_CONTROL_BYPASS_PLLA {
+ TMDS_TRANSMITTER_BYPASS_PLLA_COHERENT = 0x0,
+ TMDS_TRANSMITTER_BYPASS_PLLA_INCOHERENT = 0x1,
+} TMDS_TRANSMITTER_CONTROL_BYPASS_PLLA;
+typedef enum TMDS_TRANSMITTER_CONTROL_BYPASS_PLLB {
+ TMDS_TRANSMITTER_BYPASS_PLLB_COHERENT = 0x0,
+ TMDS_TRANSMITTER_BYPASS_PLLB_INCOHERENT = 0x1,
+} TMDS_TRANSMITTER_CONTROL_BYPASS_PLLB;
+typedef enum TMDS_REG_TEST_OUTPUTA_CNTLA {
+ TMDS_REG_TEST_OUTPUTA_CNTLA_OTDATA0 = 0x0,
+ TMDS_REG_TEST_OUTPUTA_CNTLA_OTDATA1 = 0x1,
+ TMDS_REG_TEST_OUTPUTA_CNTLA_OTDATA2 = 0x2,
+ TMDS_REG_TEST_OUTPUTA_CNTLA_NA = 0x3,
+} TMDS_REG_TEST_OUTPUTA_CNTLA;
+typedef enum TMDS_REG_TEST_OUTPUTB_CNTLB {
+ TMDS_REG_TEST_OUTPUTB_CNTLB_OTDATB0 = 0x0,
+ TMDS_REG_TEST_OUTPUTB_CNTLB_OTDATB1 = 0x1,
+ TMDS_REG_TEST_OUTPUTB_CNTLB_OTDATB2 = 0x2,
+ TMDS_REG_TEST_OUTPUTB_CNTLB_NA = 0x3,
+} TMDS_REG_TEST_OUTPUTB_CNTLB;
+typedef enum DP_LINK_TRAINING_COMPLETE {
+ DP_LINK_TRAINING_NOT_COMPLETE = 0x0,
+ DP_LINK_TRAINING_ALREADY_COMPLETE = 0x1,
+} DP_LINK_TRAINING_COMPLETE;
+typedef enum DP_EMBEDDED_PANEL_MODE {
+ DP_EXTERNAL_PANEL = 0x0,
+ DP_EMBEDDED_PANEL = 0x1,
+} DP_EMBEDDED_PANEL_MODE;
+typedef enum DP_PIXEL_ENCODING {
+ DP_PIXEL_ENCODING_RGB444 = 0x0,
+ DP_PIXEL_ENCODING_YCBCR422 = 0x1,
+ DP_PIXEL_ENCODING_YCBCR444 = 0x2,
+ DP_PIXEL_ENCODING_RGB_WIDE_GAMUT = 0x3,
+ DP_PIXEL_ENCODING_Y_ONLY = 0x4,
+ DP_PIXEL_ENCODING_YCBCR420 = 0x5,
+ DP_PIXEL_ENCODING_RESERVED = 0x6,
+} DP_PIXEL_ENCODING;
+typedef enum DP_DYN_RANGE {
+ DP_DYN_VESA_RANGE = 0x0,
+ DP_DYN_CEA_RANGE = 0x1,
+} DP_DYN_RANGE;
+typedef enum DP_YCBCR_RANGE {
+ DP_YCBCR_RANGE_BT601_5 = 0x0,
+ DP_YCBCR_RANGE_BT709_5 = 0x1,
+} DP_YCBCR_RANGE;
+typedef enum DP_COMPONENT_DEPTH {
+ DP_COMPONENT_DEPTH_6BPC = 0x0,
+ DP_COMPONENT_DEPTH_8BPC = 0x1,
+ DP_COMPONENT_DEPTH_10BPC = 0x2,
+ DP_COMPONENT_DEPTH_12BPC = 0x3,
+ DP_COMPONENT_DEPTH_16BPC = 0x4,
+ DP_COMPONENT_DEPTH_RESERVED = 0x5,
+} DP_COMPONENT_DEPTH;
+typedef enum DP_MSA_MISC0_OVERRIDE_ENABLE {
+ MSA_MISC0_OVERRIDE_DISABLE = 0x0,
+ MSA_MISC0_OVERRIDE_ENABLE = 0x1,
+} DP_MSA_MISC0_OVERRIDE_ENABLE;
+typedef enum DP_MSA_MISC1_BIT7_OVERRIDE_ENABLE {
+ MSA_MISC1_BIT7_OVERRIDE_DISABLE = 0x0,
+ MSA_MISC1_BIT7_OVERRIDE_ENABLE = 0x1,
+} DP_MSA_MISC1_BIT7_OVERRIDE_ENABLE;
+typedef enum DP_UDI_LANES {
+ DP_UDI_1_LANE = 0x0,
+ DP_UDI_2_LANES = 0x1,
+ DP_UDI_LANES_RESERVED = 0x2,
+ DP_UDI_4_LANES = 0x3,
+} DP_UDI_LANES;
+typedef enum DP_VID_STREAM_DIS_DEFER {
+ DP_VID_STREAM_DIS_NO_DEFER = 0x0,
+ DP_VID_STREAM_DIS_DEFER_TO_HBLANK = 0x1,
+ DP_VID_STREAM_DIS_DEFER_TO_VBLANK = 0x2,
+} DP_VID_STREAM_DIS_DEFER;
+typedef enum DP_STEER_OVERFLOW_ACK {
+ DP_STEER_OVERFLOW_ACK_NO_EFFECT = 0x0,
+ DP_STEER_OVERFLOW_ACK_CLR_INTERRUPT = 0x1,
+} DP_STEER_OVERFLOW_ACK;
+typedef enum DP_STEER_OVERFLOW_MASK {
+ DP_STEER_OVERFLOW_MASKED = 0x0,
+ DP_STEER_OVERFLOW_UNMASK = 0x1,
+} DP_STEER_OVERFLOW_MASK;
+typedef enum DP_TU_OVERFLOW_ACK {
+ DP_TU_OVERFLOW_ACK_NO_EFFECT = 0x0,
+ DP_TU_OVERFLOW_ACK_CLR_INTERRUPT = 0x1,
+} DP_TU_OVERFLOW_ACK;
+typedef enum DP_VID_TIMING_MODE {
+ DP_VID_TIMING_MODE_ASYNC = 0x0,
+ DP_VID_TIMING_MODE_SYNC = 0x1,
+} DP_VID_TIMING_MODE;
+typedef enum DP_VID_M_N_DOUBLE_BUFFER_MODE {
+ DP_VID_M_N_DOUBLE_BUFFER_AFTER_VID_M_UPDATE = 0x0,
+ DP_VID_M_N_DOUBLE_BUFFER_AT_FRAME_START = 0x1,
+} DP_VID_M_N_DOUBLE_BUFFER_MODE;
+typedef enum DP_VID_M_N_GEN_EN {
+ DP_VID_M_N_PROGRAMMED_VIA_REG = 0x0,
+ DP_VID_M_N_CALC_AUTO = 0x1,
+} DP_VID_M_N_GEN_EN;
+typedef enum DP_VID_M_DOUBLE_VALUE_EN {
+ DP_VID_M_INPUT_PIXEL_RATE = 0x0,
+ DP_VID_M_DOUBLE_INPUT_PIXEL_RATE = 0x1,
+} DP_VID_M_DOUBLE_VALUE_EN;
+typedef enum DP_VID_ENHANCED_FRAME_MODE {
+ VID_NORMAL_FRAME_MODE = 0x0,
+ VID_ENHANCED_MODE = 0x1,
+} DP_VID_ENHANCED_FRAME_MODE;
+typedef enum DP_VID_MSA_TOP_FIELD_MODE {
+ DP_TOP_FIELD_ONLY = 0x0,
+ DP_TOP_PLUS_BOTTOM_FIELD = 0x1,
+} DP_VID_MSA_TOP_FIELD_MODE;
+typedef enum DP_VID_VBID_FIELD_POL {
+ DP_VID_VBID_FIELD_POL_NORMAL = 0x0,
+ DP_VID_VBID_FIELD_POL_INV = 0x1,
+} DP_VID_VBID_FIELD_POL;
+typedef enum DP_VID_STREAM_DISABLE_ACK {
+ ID_STREAM_DISABLE_NO_ACK = 0x0,
+ ID_STREAM_DISABLE_ACKED = 0x1,
+} DP_VID_STREAM_DISABLE_ACK;
+typedef enum DP_VID_STREAM_DISABLE_MASK {
+ VID_STREAM_DISABLE_MASKED = 0x0,
+ VID_STREAM_DISABLE_UNMASK = 0x1,
+} DP_VID_STREAM_DISABLE_MASK;
+typedef enum DPHY_ATEST_SEL_LANE0 {
+ DPHY_ATEST_LANE0_PRBS_PATTERN = 0x0,
+ DPHY_ATEST_LANE0_REG_PATTERN = 0x1,
+} DPHY_ATEST_SEL_LANE0;
+typedef enum DPHY_ATEST_SEL_LANE1 {
+ DPHY_ATEST_LANE1_PRBS_PATTERN = 0x0,
+ DPHY_ATEST_LANE1_REG_PATTERN = 0x1,
+} DPHY_ATEST_SEL_LANE1;
+typedef enum DPHY_ATEST_SEL_LANE2 {
+ DPHY_ATEST_LANE2_PRBS_PATTERN = 0x0,
+ DPHY_ATEST_LANE2_REG_PATTERN = 0x1,
+} DPHY_ATEST_SEL_LANE2;
+typedef enum DPHY_ATEST_SEL_LANE3 {
+ DPHY_ATEST_LANE3_PRBS_PATTERN = 0x0,
+ DPHY_ATEST_LANE3_REG_PATTERN = 0x1,
+} DPHY_ATEST_SEL_LANE3;
+typedef enum DPHY_BYPASS {
+ DPHY_8B10B_OUTPUT = 0x0,
+ DPHY_DBG_OUTPUT = 0x1,
+} DPHY_BYPASS;
+typedef enum DPHY_SKEW_BYPASS {
+ DPHY_WITH_SKEW = 0x0,
+ DPHY_NO_SKEW = 0x1,
+} DPHY_SKEW_BYPASS;
+typedef enum DPHY_TRAINING_PATTERN_SEL {
+ DPHY_TRAINING_PATTERN_1 = 0x0,
+ DPHY_TRAINING_PATTERN_2 = 0x1,
+ DPHY_TRAINING_PATTERN_3 = 0x2,
+ DPHY_TRAINING_PATTERN_4 = 0x3,
+} DPHY_TRAINING_PATTERN_SEL;
+typedef enum DPHY_8B10B_RESET {
+ DPHY_8B10B_NOT_RESET = 0x0,
+ DPHY_8B10B_RESETET = 0x1,
+} DPHY_8B10B_RESET;
+typedef enum DP_DPHY_8B10B_EXT_DISP {
+ DP_DPHY_8B10B_EXT_DISP_ZERO = 0x0,
+ DP_DPHY_8B10B_EXT_DISP_ONE = 0x1,
+} DP_DPHY_8B10B_EXT_DISP;
+typedef enum DPHY_8B10B_CUR_DISP {
+ DPHY_8B10B_CUR_DISP_ZERO = 0x0,
+ DPHY_8B10B_CUR_DISP_ONE = 0x1,
+} DPHY_8B10B_CUR_DISP;
+typedef enum DPHY_PRBS_EN {
+ DPHY_PRBS_DISABLE = 0x0,
+ DPHY_PRBS_ENABLE = 0x1,
+} DPHY_PRBS_EN;
+typedef enum DPHY_PRBS_SEL {
+ DPHY_PRBS7_SELECTED = 0x0,
+ DPHY_PRBS23_SELECTED = 0x1,
+ DPHY_PRBS11_SELECTED = 0x2,
+} DPHY_PRBS_SEL;
+typedef enum DPHY_LOAD_BS_COUNT_START {
+ DPHY_LOAD_BS_COUNT_STARTED = 0x0,
+ DPHY_LOAD_BS_COUNT_NOT_STARTED = 0x1,
+} DPHY_LOAD_BS_COUNT_START;
+typedef enum DPHY_CRC_EN {
+ DPHY_CRC_DISABLED = 0x0,
+ DPHY_CRC_ENABLED = 0x1,
+} DPHY_CRC_EN;
+typedef enum DPHY_CRC_CONT_EN {
+ DPHY_CRC_ONE_SHOT = 0x0,
+ DPHY_CRC_CONTINUOUS = 0x1,
+} DPHY_CRC_CONT_EN;
+typedef enum DPHY_CRC_FIELD {
+ DPHY_CRC_START_FROM_TOP_FIELD = 0x0,
+ DPHY_CRC_START_FROM_BOTTOM_FIELD = 0x1,
+} DPHY_CRC_FIELD;
+typedef enum DPHY_CRC_SEL {
+ DPHY_CRC_LANE0_SELECTED = 0x0,
+ DPHY_CRC_LANE1_SELECTED = 0x1,
+ DPHY_CRC_LANE2_SELECTED = 0x2,
+ DPHY_CRC_LANE3_SELECTED = 0x3,
+} DPHY_CRC_SEL;
+typedef enum DPHY_RX_FAST_TRAINING_CAPABLE {
+ DPHY_FAST_TRAINING_NOT_CAPABLE_0 = 0x0,
+ DPHY_FAST_TRAINING_CAPABLE = 0x1,
+} DPHY_RX_FAST_TRAINING_CAPABLE;
+typedef enum DP_SEC_COLLISION_ACK {
+ DP_SEC_COLLISION_ACK_NO_EFFECT = 0x0,
+ DP_SEC_COLLISION_ACK_CLR_FLAG = 0x1,
+} DP_SEC_COLLISION_ACK;
+typedef enum DP_SEC_AUDIO_MUTE {
+ DP_SEC_AUDIO_MUTE_HW_CTRL = 0x0,
+ DP_SEC_AUDIO_MUTE_SW_CTRL = 0x1,
+} DP_SEC_AUDIO_MUTE;
+typedef enum DP_SEC_TIMESTAMP_MODE {
+ DP_SEC_TIMESTAMP_PROGRAMMABLE_MODE = 0x0,
+ DP_SEC_TIMESTAMP_AUTO_CALC_MODE = 0x1,
+} DP_SEC_TIMESTAMP_MODE;
+typedef enum DP_SEC_ASP_PRIORITY {
+ DP_SEC_ASP_LOW_PRIORITY = 0x0,
+ DP_SEC_ASP_HIGH_PRIORITY = 0x1,
+} DP_SEC_ASP_PRIORITY;
+typedef enum DP_SEC_ASP_CHANNEL_COUNT_OVERRIDE {
+ DP_SEC_ASP_CHANNEL_COUNT_FROM_AZ = 0x0,
+ DP_SEC_ASP_CHANNEL_COUNT_OVERRIDE_ENABLED = 0x1,
+} DP_SEC_ASP_CHANNEL_COUNT_OVERRIDE;
+typedef enum DP_MSE_SAT_UPDATE_ACT {
+ DP_MSE_SAT_UPDATE_NO_ACTION = 0x0,
+ DP_MSE_SAT_UPDATE_WITH_TRIGGER = 0x1,
+ DP_MSE_SAT_UPDATE_WITHOUT_TRIGGER = 0x2,
+} DP_MSE_SAT_UPDATE_ACT;
+typedef enum DP_MSE_LINK_LINE {
+ DP_MSE_LINK_LINE_32_MTP_LONG = 0x0,
+ DP_MSE_LINK_LINE_64_MTP_LONG = 0x1,
+ DP_MSE_LINK_LINE_128_MTP_LONG = 0x2,
+ DP_MSE_LINK_LINE_256_MTP_LONG = 0x3,
+} DP_MSE_LINK_LINE;
+typedef enum DP_MSE_BLANK_CODE {
+ DP_MSE_BLANK_CODE_SF_FILLED = 0x0,
+ DP_MSE_BLANK_CODE_ZERO_FILLED = 0x1,
+} DP_MSE_BLANK_CODE;
+typedef enum DP_MSE_TIMESTAMP_MODE {
+ DP_MSE_TIMESTAMP_CALC_BASED_ON_LINK_RATE = 0x0,
+ DP_MSE_TIMESTAMP_CALC_BASED_ON_VC_RATE = 0x1,
+} DP_MSE_TIMESTAMP_MODE;
+typedef enum DP_MSE_ZERO_ENCODER {
+ DP_MSE_NOT_ZERO_FE_ENCODER = 0x0,
+ DP_MSE_ZERO_FE_ENCODER = 0x1,
+} DP_MSE_ZERO_ENCODER;
+typedef enum DP_MSE_OUTPUT_DPDBG_DATA {
+ DP_MSE_OUTPUT_DPDBG_DATA_DIS = 0x0,
+ DP_MSE_OUTPUT_DPDBG_DATA_EN = 0x1,
+} DP_MSE_OUTPUT_DPDBG_DATA;
+typedef enum DP_DPHY_HBR2_PATTERN_CONTROL_MODE {
+ DP_DPHY_HBR2_PASS_THROUGH = 0x0,
+ DP_DPHY_HBR2_PATTERN_1 = 0x1,
+ DP_DPHY_HBR2_PATTERN_2_NEG = 0x2,
+ DP_DPHY_HBR2_PATTERN_3 = 0x3,
+ DP_DPHY_HBR2_PATTERN_2_POS = 0x6,
+} DP_DPHY_HBR2_PATTERN_CONTROL_MODE;
+typedef enum DPHY_CRC_MST_PHASE_ERROR_ACK {
+ DPHY_CRC_MST_PHASE_ERROR_NO_ACK = 0x0,
+ DPHY_CRC_MST_PHASE_ERROR_ACKED = 0x1,
+} DPHY_CRC_MST_PHASE_ERROR_ACK;
+typedef enum DPHY_SW_FAST_TRAINING_START {
+ DPHY_SW_FAST_TRAINING_NOT_STARTED = 0x0,
+ DPHY_SW_FAST_TRAINING_STARTED = 0x1,
+} DPHY_SW_FAST_TRAINING_START;
+typedef enum DP_DPHY_FAST_TRAINING_VBLANK_EDGE_DETECT_EN {
+ DP_DPHY_FAST_TRAINING_VBLANK_EDGE_DETECT_DISABLED= 0x0,
+ DP_DPHY_FAST_TRAINING_VBLANK_EDGE_DETECT_ENABLED = 0x1,
+} DP_DPHY_FAST_TRAINING_VBLANK_EDGE_DETECT_EN;
+typedef enum DP_DPHY_FAST_TRAINING_COMPLETE_MASK {
+ DP_DPHY_FAST_TRAINING_COMPLETE_MASKED = 0x0,
+ DP_DPHY_FAST_TRAINING_COMPLETE_NOT_MASKED = 0x1,
+} DP_DPHY_FAST_TRAINING_COMPLETE_MASK;
+typedef enum DP_DPHY_FAST_TRAINING_COMPLETE_ACK {
+ DP_DPHY_FAST_TRAINING_COMPLETE_NOT_ACKED = 0x0,
+ DP_DPHY_FAST_TRAINING_COMPLETE_ACKED = 0x1,
+} DP_DPHY_FAST_TRAINING_COMPLETE_ACK;
+typedef enum DP_MSA_V_TIMING_OVERRIDE_EN {
+ MSA_V_TIMING_OVERRIDE_DISABLED = 0x0,
+ MSA_V_TIMING_OVERRIDE_ENABLED = 0x1,
+} DP_MSA_V_TIMING_OVERRIDE_EN;
+typedef enum DP_SEC_GSP0_PRIORITY {
+ SEC_GSP0_PRIORITY_LOW = 0x0,
+ SEC_GSP0_PRIORITY_HIGH = 0x1,
+} DP_SEC_GSP0_PRIORITY;
+typedef enum DP_SEC_GSP0_SEND {
+ NOT_SENT = 0x0,
+ FORCE_SENT = 0x1,
+} DP_SEC_GSP0_SEND;
+typedef enum DP_AUX_CONTROL_HPD_SEL {
+ DP_AUX_CONTROL_HPD1_SELECTED = 0x0,
+ DP_AUX_CONTROL_HPD2_SELECTED = 0x1,
+ DP_AUX_CONTROL_HPD3_SELECTED = 0x2,
+ DP_AUX_CONTROL_HPD4_SELECTED = 0x3,
+ DP_AUX_CONTROL_HPD5_SELECTED = 0x4,
+ DP_AUX_CONTROL_HPD6_SELECTED = 0x5,
+} DP_AUX_CONTROL_HPD_SEL;
+typedef enum DP_AUX_CONTROL_TEST_MODE {
+ DP_AUX_CONTROL_TEST_MODE_DISABLE = 0x0,
+ DP_AUX_CONTROL_TEST_MODE_ENABLE = 0x1,
+} DP_AUX_CONTROL_TEST_MODE;
+typedef enum DP_AUX_SW_CONTROL_SW_GO {
+ DP_AUX_SW_CONTROL_SW__NOT_GO = 0x0,
+ DP_AUX_SW_CONTROL_SW__GO = 0x1,
+} DP_AUX_SW_CONTROL_SW_GO;
+typedef enum DP_AUX_SW_CONTROL_LS_READ_TRIG {
+ DP_AUX_SW_CONTROL_LS_READ__NOT_TRIG = 0x0,
+ DP_AUX_SW_CONTROL_LS_READ__TRIG = 0x1,
+} DP_AUX_SW_CONTROL_LS_READ_TRIG;
+typedef enum DP_AUX_ARB_CONTROL_ARB_PRIORITY {
+ DP_AUX_ARB_CONTROL_ARB_PRIORITY__GTC_LS_SW = 0x0,
+ DP_AUX_ARB_CONTROL_ARB_PRIORITY__LS_GTC_SW = 0x1,
+ DP_AUX_ARB_CONTROL_ARB_PRIORITY__SW_LS_GTC = 0x2,
+ DP_AUX_ARB_CONTROL_ARB_PRIORITY__SW_GTC_LS = 0x3,
+} DP_AUX_ARB_CONTROL_ARB_PRIORITY;
+typedef enum DP_AUX_ARB_CONTROL_USE_AUX_REG_REQ {
+ DP_AUX_ARB_CONTROL__NOT_USE_AUX_REG_REQ = 0x0,
+ DP_AUX_ARB_CONTROL__USE_AUX_REG_REQ = 0x1,
+} DP_AUX_ARB_CONTROL_USE_AUX_REG_REQ;
+typedef enum DP_AUX_ARB_CONTROL_DONE_USING_AUX_REG {
+ DP_AUX_ARB_CONTROL__DONE_NOT_USING_AUX_REG = 0x0,
+ DP_AUX_ARB_CONTROL__DONE_USING_AUX_REG = 0x1,
+} DP_AUX_ARB_CONTROL_DONE_USING_AUX_REG;
+typedef enum DP_AUX_INT_ACK {
+ DP_AUX_INT__NOT_ACK = 0x0,
+ DP_AUX_INT__ACK = 0x1,
+} DP_AUX_INT_ACK;
+typedef enum DP_AUX_LS_UPDATE_ACK {
+ DP_AUX_INT_LS_UPDATE_NOT_ACK = 0x0,
+ DP_AUX_INT_LS_UPDATE_ACK = 0x1,
+} DP_AUX_LS_UPDATE_ACK;
+typedef enum DP_AUX_DPHY_TX_REF_CONTROL_TX_REF_SEL {
+ DP_AUX_DPHY_TX_REF_CONTROL_TX_REF_SEL__DIVIDED_SYM_CLK= 0x0,
+ DP_AUX_DPHY_TX_REF_CONTROL_TX_REF_SEL__FROM_DCCG_MICROSECOND_REF= 0x1,
+} DP_AUX_DPHY_TX_REF_CONTROL_TX_REF_SEL;
+typedef enum DP_AUX_DPHY_TX_REF_CONTROL_TX_RATE {
+ DP_AUX_DPHY_TX_REF_CONTROL_TX_RATE__1MHZ = 0x0,
+ DP_AUX_DPHY_TX_REF_CONTROL_TX_RATE__2MHZ = 0x1,
+ DP_AUX_DPHY_TX_REF_CONTROL_TX_RATE__4MHZ = 0x2,
+ DP_AUX_DPHY_TX_REF_CONTROL_TX_RATE__8MHZ = 0x3,
+} DP_AUX_DPHY_TX_REF_CONTROL_TX_RATE;
+typedef enum DP_AUX_DPHY_TX_CONTROL_PRECHARGE_LEN {
+ DP_AUX_DPHY_TX_CONTROL_PRECHARGE_LEN__0US = 0x0,
+ DP_AUX_DPHY_TX_CONTROL_PRECHARGE_LEN__8US = 0x1,
+ DP_AUX_DPHY_TX_CONTROL_PRECHARGE_LEN__16US = 0x2,
+ DP_AUX_DPHY_TX_CONTROL_PRECHARGE_LEN__24US = 0x3,
+ DP_AUX_DPHY_TX_CONTROL_PRECHARGE_LEN__32US = 0x4,
+ DP_AUX_DPHY_TX_CONTROL_PRECHARGE_LEN__40US = 0x5,
+ DP_AUX_DPHY_TX_CONTROL_PRECHARGE_LEN__48US = 0x6,
+ DP_AUX_DPHY_TX_CONTROL_PRECHARGE_LEN__56US = 0x7,
+} DP_AUX_DPHY_TX_CONTROL_PRECHARGE_LEN;
+typedef enum DP_AUX_DPHY_TX_CONTROL_MODE_DET_CHECK_DELAY {
+ DP_AUX_DPHY_TX_CONTROL_MODE_DET_CHECK_DELAY__0 = 0x0,
+ DP_AUX_DPHY_TX_CONTROL_MODE_DET_CHECK_DELAY__16US= 0x1,
+ DP_AUX_DPHY_TX_CONTROL_MODE_DET_CHECK_DELAY__32US= 0x2,
+ DP_AUX_DPHY_TX_CONTROL_MODE_DET_CHECK_DELAY__64US= 0x3,
+ DP_AUX_DPHY_TX_CONTROL_MODE_DET_CHECK_DELAY__128US= 0x4,
+ DP_AUX_DPHY_TX_CONTROL_MODE_DET_CHECK_DELAY__256US= 0x5,
+} DP_AUX_DPHY_TX_CONTROL_MODE_DET_CHECK_DELAY;
+typedef enum DP_AUX_DPHY_RX_CONTROL_START_WINDOW {
+ DP_AUX_DPHY_RX_CONTROL_START_WINDOW__1TO2_PERIOD = 0x0,
+ DP_AUX_DPHY_RX_CONTROL_START_WINDOW__1TO4_PERIOD = 0x1,
+ DP_AUX_DPHY_RX_CONTROL_START_WINDOW__1TO8_PERIOD = 0x2,
+ DP_AUX_DPHY_RX_CONTROL_START_WINDOW__1TO16_PERIOD= 0x3,
+ DP_AUX_DPHY_RX_CONTROL_START_WINDOW__1TO32_PERIOD= 0x4,
+ DP_AUX_DPHY_RX_CONTROL_START_WINDOW__1TO64_PERIOD= 0x5,
+ DP_AUX_DPHY_RX_CONTROL_START_WINDOW__1TO128_PERIOD= 0x6,
+ DP_AUX_DPHY_RX_CONTROL_START_WINDOW__1TO256_PERIOD= 0x7,
+} DP_AUX_DPHY_RX_CONTROL_START_WINDOW;
+typedef enum DP_AUX_DPHY_RX_CONTROL_RECEIVE_WINDOW {
+ DP_AUX_DPHY_RX_CONTROL_RECEIVE_WINDOW__1TO2_PERIOD= 0x0,
+ DP_AUX_DPHY_RX_CONTROL_RECEIVE_WINDOW__1TO4_PERIOD= 0x1,
+ DP_AUX_DPHY_RX_CONTROL_RECEIVE_WINDOW__1TO8_PERIOD= 0x2,
+ DP_AUX_DPHY_RX_CONTROL_RECEIVE_WINDOW__1TO16_PERIOD= 0x3,
+ DP_AUX_DPHY_RX_CONTROL_RECEIVE_WINDOW__1TO32_PERIOD= 0x4,
+ DP_AUX_DPHY_RX_CONTROL_RECEIVE_WINDOW__1TO64_PERIOD= 0x5,
+ DP_AUX_DPHY_RX_CONTROL_RECEIVE_WINDOW__1TO128_PERIOD= 0x6,
+ DP_AUX_DPHY_RX_CONTROL_RECEIVE_WINDOW__1TO256_PERIOD= 0x7,
+} DP_AUX_DPHY_RX_CONTROL_RECEIVE_WINDOW;
+typedef enum DP_AUX_DPHY_RX_CONTROL_HALF_SYM_DETECT_LEN {
+ DP_AUX_DPHY_RX_CONTROL_HALF_SYM_DETECT_LEN__6_EDGES= 0x0,
+ DP_AUX_DPHY_RX_CONTROL_HALF_SYM_DETECT_LEN__10_EDGES= 0x1,
+ DP_AUX_DPHY_RX_CONTROL_HALF_SYM_DETECT_LEN__18_EDGES= 0x2,
+ DP_AUX_DPHY_RX_CONTROL_HALF_SYM_DETECT_LEN__RESERVED= 0x3,
+} DP_AUX_DPHY_RX_CONTROL_HALF_SYM_DETECT_LEN;
+typedef enum DP_AUX_DPHY_RX_CONTROL_ALLOW_BELOW_THRESHOLD_PHASE_DETECT {
+ DP_AUX_DPHY_RX_CONTROL__NOT_ALLOW_BELOW_THRESHOLD_PHASE_DETECT= 0x0,
+ DP_AUX_DPHY_RX_CONTROL__ALLOW_BELOW_THRESHOLD_PHASE_DETECT= 0x1,
+} DP_AUX_DPHY_RX_CONTROL_ALLOW_BELOW_THRESHOLD_PHASE_DETECT;
+typedef enum DP_AUX_DPHY_RX_CONTROL_ALLOW_BELOW_THRESHOLD_START {
+ DP_AUX_DPHY_RX_CONTROL__NOT_ALLOW_BELOW_THRESHOLD_START= 0x0,
+ DP_AUX_DPHY_RX_CONTROL__ALLOW_BELOW_THRESHOLD_START= 0x1,
+} DP_AUX_DPHY_RX_CONTROL_ALLOW_BELOW_THRESHOLD_START;
+typedef enum DP_AUX_DPHY_RX_CONTROL_ALLOW_BELOW_THRESHOLD_STOP {
+ DP_AUX_DPHY_RX_CONTROL__NOT_ALLOW_BELOW_THRESHOLD_STOP= 0x0,
+ DP_AUX_DPHY_RX_CONTROL__ALLOW_BELOW_THRESHOLD_STOP= 0x1,
+} DP_AUX_DPHY_RX_CONTROL_ALLOW_BELOW_THRESHOLD_STOP;
+typedef enum DP_AUX_DPHY_RX_CONTROL_PHASE_DETECT_LEN {
+ DP_AUX_DPHY_RX_CONTROL_PHASE_DETECT_LEN__2_HALF_SYMBOLS= 0x0,
+ DP_AUX_DPHY_RX_CONTROL_PHASE_DETECT_LEN__4_HALF_SYMBOLS= 0x1,
+ DP_AUX_DPHY_RX_CONTROL_PHASE_DETECT_LEN__6_HALF_SYMBOLS= 0x2,
+ DP_AUX_DPHY_RX_CONTROL_PHASE_DETECT_LEN__8_HALF_SYMBOLS= 0x3,
+} DP_AUX_DPHY_RX_CONTROL_PHASE_DETECT_LEN;
+typedef enum DP_AUX_DPHY_RX_CONTROL_TIMEOUT_LEN {
+ DP_AUX_DPHY_RX_CONTROL_TIMEOUT_LEN_450US = 0x0,
+ DP_AUX_DPHY_RX_CONTROL_TIMEOUT_LEN_500US = 0x1,
+ DP_AUX_DPHY_RX_CONTROL_TIMEOUT_LEN_550US = 0x2,
+ DP_AUX_DPHY_RX_CONTROL_TIMEOUT_LEN_600US = 0x3,
+ DP_AUX_DPHY_RX_CONTROL_TIMEOUT_LEN_650US = 0x4,
+ DP_AUX_DPHY_RX_CONTROL_TIMEOUT_LEN_700US = 0x5,
+ DP_AUX_DPHY_RX_CONTROL_TIMEOUT_LEN_750US = 0x6,
+ DP_AUX_DPHY_RX_CONTROL_TIMEOUT_LEN_800US = 0x7,
+} DP_AUX_DPHY_RX_CONTROL_TIMEOUT_LEN;
+typedef enum DP_AUX_DPHY_RX_DETECTION_THRESHOLD {
+ DP_AUX_DPHY_RX_DETECTION_THRESHOLD__1to2 = 0x0,
+ DP_AUX_DPHY_RX_DETECTION_THRESHOLD__3to4 = 0x1,
+ DP_AUX_DPHY_RX_DETECTION_THRESHOLD__7to8 = 0x2,
+ DP_AUX_DPHY_RX_DETECTION_THRESHOLD__15to16 = 0x3,
+ DP_AUX_DPHY_RX_DETECTION_THRESHOLD__31to32 = 0x4,
+ DP_AUX_DPHY_RX_DETECTION_THRESHOLD__63to64 = 0x5,
+ DP_AUX_DPHY_RX_DETECTION_THRESHOLD__127to128 = 0x6,
+ DP_AUX_DPHY_RX_DETECTION_THRESHOLD__255to256 = 0x7,
+} DP_AUX_DPHY_RX_DETECTION_THRESHOLD;
+typedef enum DP_AUX_GTC_SYNC_CONTROL_GTC_SYNC_BLOCK_REQ {
+ DP_AUX_GTC_SYNC_CONTROL_GTC_SYNC_ALLOW_REQ_FROM_OTHER_AUX= 0x0,
+ DP_AUX_GTC_SYNC_CONTROL_GTC_SYNC_BLOCK_REQ_FROM_OTHER_AUX= 0x1,
+} DP_AUX_GTC_SYNC_CONTROL_GTC_SYNC_BLOCK_REQ;
+typedef enum DP_AUX_GTC_SYNC_CONTROL_INTERVAL_RESET_WINDOW {
+ DP_AUX_GTC_SYNC_CONTROL_INTERVAL_RESET_WINDOW__300US= 0x0,
+ DP_AUX_GTC_SYNC_CONTROL_INTERVAL_RESET_WINDOW__400US= 0x1,
+ DP_AUX_GTC_SYNC_CONTROL_INTERVAL_RESET_WINDOW__500US= 0x2,
+ DP_AUX_GTC_SYNC_CONTROL_INTERVAL_RESET_WINDOW__600US= 0x3,
+} DP_AUX_GTC_SYNC_CONTROL_INTERVAL_RESET_WINDOW;
+typedef enum DP_AUX_GTC_SYNC_CONTROL_OFFSET_CALC_MAX_ATTEMPT {
+ DP_AUX_GTC_SYNC_CONTROL_OFFSET_CALC_MAX_ATTEMPT__4_ATTAMPS= 0x0,
+ DP_AUX_GTC_SYNC_CONTROL_OFFSET_CALC_MAX_ATTEMPT__8_ATTAMPS= 0x1,
+ DP_AUX_GTC_SYNC_CONTROL_OFFSET_CALC_MAX_ATTEMPT__16_ATTAMPS= 0x2,
+ DP_AUX_GTC_SYNC_CONTROL_OFFSET_CALC_MAX_ATTEMPT__RESERVED= 0x3,
+} DP_AUX_GTC_SYNC_CONTROL_OFFSET_CALC_MAX_ATTEMPT;
+typedef enum DP_AUX_GTC_SYNC_ERROR_CONTROL_LOCK_ACQ_TIMEOUT_LEN {
+ DP_AUX_GTC_SYNC_ERROR_CONTROL_LOCK_ACQ_TIMEOUT_LEN__0= 0x0,
+ DP_AUX_GTC_SYNC_ERROR_CONTROL_LOCK_ACQ_TIMEOUT_LEN__64= 0x1,
+ DP_AUX_GTC_SYNC_ERROR_CONTROL_LOCK_ACQ_TIMEOUT_LEN__128= 0x2,
+ DP_AUX_GTC_SYNC_ERROR_CONTROL_LOCK_ACQ_TIMEOUT_LEN__256= 0x3,
+} DP_AUX_GTC_SYNC_ERROR_CONTROL_LOCK_ACQ_TIMEOUT_LEN;
+typedef enum DP_AUX_ERR_OCCURRED_ACK {
+ DP_AUX_ERR_OCCURRED__NOT_ACK = 0x0,
+ DP_AUX_ERR_OCCURRED__ACK = 0x1,
+} DP_AUX_ERR_OCCURRED_ACK;
+typedef enum DP_AUX_POTENTIAL_ERR_REACHED_ACK {
+ DP_AUX_POTENTIAL_ERR_REACHED__NOT_ACK = 0x0,
+ DP_AUX_POTENTIAL_ERR_REACHED__ACK = 0x1,
+} DP_AUX_POTENTIAL_ERR_REACHED_ACK;
+typedef enum DP_AUX_DEFINITE_ERR_REACHED_ACK {
+ ALPHA_DP_AUX_DEFINITE_ERR_REACHED_NOT_ACK = 0x0,
+ ALPHA_DP_AUX_DEFINITE_ERR_REACHED_ACK = 0x1,
+} DP_AUX_DEFINITE_ERR_REACHED_ACK;
+typedef enum DP_AUX_RESET {
+ DP_AUX_RESET_DEASSERTED = 0x0,
+ DP_AUX_RESET_ASSERTED = 0x1,
+} DP_AUX_RESET;
+typedef enum DP_AUX_RESET_DONE {
+ DP_AUX_RESET_SEQUENCE_NOT_DONE = 0x0,
+ DP_AUX_RESET_SEQUENCE_DONE = 0x1,
+} DP_AUX_RESET_DONE;
+typedef enum FBC_IDLE_MASK_MASK_BITS {
+ FBC_IDLE_MASK_DISP_REG_UPDATE = 0x0,
+ FBC_IDLE_MASK_RESERVED1 = 0x1,
+ FBC_IDLE_MASK_FBC_GRPH_COMP_EN = 0x2,
+ FBC_IDLE_MASK_FBC_MIN_COMPRESSION = 0x3,
+ FBC_IDLE_MASK_FBC_ALPHA_COMP_EN = 0x4,
+ FBC_IDLE_MASK_FBC_ZERO_ALPHA_CHUNK_SKIP_EN = 0x5,
+ FBC_IDLE_MASK_FBC_FORCE_COPY_TO_COMP_BUF = 0x6,
+ FBC_IDLE_MASK_RESERVED7 = 0x7,
+ FBC_IDLE_MASK_RESERVED8 = 0x8,
+ FBC_IDLE_MASK_RESERVED9 = 0x9,
+ FBC_IDLE_MASK_RESERVED10 = 0xa,
+ FBC_IDLE_MASK_RESERVED11 = 0xb,
+ FBC_IDLE_MASK_RESERVED12 = 0xc,
+ FBC_IDLE_MASK_RESERVED13 = 0xd,
+ FBC_IDLE_MASK_RESERVED14 = 0xe,
+ FBC_IDLE_MASK_RESERVED15 = 0xf,
+ FBC_IDLE_MASK_RESERVED16 = 0x10,
+ FBC_IDLE_MASK_RESERVED17 = 0x11,
+ FBC_IDLE_MASK_RESERVED18 = 0x12,
+ FBC_IDLE_MASK_RESERVED19 = 0x13,
+ FBC_IDLE_MASK_RESERVED20 = 0x14,
+ FBC_IDLE_MASK_RESERVED21 = 0x15,
+ FBC_IDLE_MASK_RESERVED22 = 0x16,
+ FBC_IDLE_MASK_RESERVED23 = 0x17,
+ FBC_IDLE_MASK_MC_HIT_REGION_0 = 0x18,
+ FBC_IDLE_MASK_MC_HIT_REGION_1 = 0x19,
+ FBC_IDLE_MASK_MC_HIT_REGION_2 = 0x1a,
+ FBC_IDLE_MASK_MC_HIT_REGION_3 = 0x1b,
+ FBC_IDLE_MASK_MC_WRITE = 0x1c,
+ FBC_IDLE_MASK_CG_STATIC_SCREEN = 0x1d,
+ FBC_IDLE_MASK_RESERVED30 = 0x1e,
+ FBC_IDLE_MASK_RESERVED31 = 0x1f,
+} FBC_IDLE_MASK_MASK_BITS;
+typedef enum FMT_CONTROL_PIXEL_ENCODING {
+ FMT_CONTROL_PIXEL_ENCODING_RGB444_OR_YCBCR444 = 0x0,
+ FMT_CONTROL_PIXEL_ENCODING_YCBCR422 = 0x1,
+ FMT_CONTROL_PIXEL_ENCODING_YCBCR420 = 0x2,
+ FMT_CONTROL_PIXEL_ENCODING_RESERVED = 0x3,
+} FMT_CONTROL_PIXEL_ENCODING;
+typedef enum FMT_CONTROL_SUBSAMPLING_MODE {
+ FMT_CONTROL_SUBSAMPLING_MODE_DROP = 0x0,
+ FMT_CONTROL_SUBSAMPLING_MODE_AVERAGE = 0x1,
+ FMT_CONTROL_SUBSAMPLING_MODE_3_TAP = 0x2,
+ FMT_CONTROL_SUBSAMPLING_MODE_RESERVED = 0x3,
+} FMT_CONTROL_SUBSAMPLING_MODE;
+typedef enum FMT_CONTROL_SUBSAMPLING_ORDER {
+ FMT_CONTROL_SUBSAMPLING_ORDER_CB_BEFORE_CR = 0x0,
+ FMT_CONTROL_SUBSAMPLING_ORDER_CR_BEFORE_CB = 0x1,
+} FMT_CONTROL_SUBSAMPLING_ORDER;
+typedef enum FMT_CONTROL_CBCR_BIT_REDUCTION_BYPASS {
+ FMT_CONTROL_CBCR_BIT_REDUCTION_BYPASS_DISABLE = 0x0,
+ FMT_CONTROL_CBCR_BIT_REDUCTION_BYPASS_ENABLE = 0x1,
+} FMT_CONTROL_CBCR_BIT_REDUCTION_BYPASS;
+typedef enum FMT_BIT_DEPTH_CONTROL_TRUNCATE_MODE {
+ FMT_BIT_DEPTH_CONTROL_TRUNCATE_MODE_TRUNCATION = 0x0,
+ FMT_BIT_DEPTH_CONTROL_TRUNCATE_MODE_ROUNDING = 0x1,
+} FMT_BIT_DEPTH_CONTROL_TRUNCATE_MODE;
+typedef enum FMT_BIT_DEPTH_CONTROL_TRUNCATE_DEPTH {
+ FMT_BIT_DEPTH_CONTROL_TRUNCATE_DEPTH_18BPP = 0x0,
+ FMT_BIT_DEPTH_CONTROL_TRUNCATE_DEPTH_24BPP = 0x1,
+ FMT_BIT_DEPTH_CONTROL_TRUNCATE_DEPTH_30BPP = 0x2,
+} FMT_BIT_DEPTH_CONTROL_TRUNCATE_DEPTH;
+typedef enum FMT_BIT_DEPTH_CONTROL_SPATIAL_DITHER_DEPTH {
+ FMT_BIT_DEPTH_CONTROL_SPATIAL_DITHER_DEPTH_18BPP = 0x0,
+ FMT_BIT_DEPTH_CONTROL_SPATIAL_DITHER_DEPTH_24BPP = 0x1,
+ FMT_BIT_DEPTH_CONTROL_SPATIAL_DITHER_DEPTH_30BPP = 0x2,
+} FMT_BIT_DEPTH_CONTROL_SPATIAL_DITHER_DEPTH;
+typedef enum FMT_BIT_DEPTH_CONTROL_TEMPORAL_DITHER_DEPTH {
+ FMT_BIT_DEPTH_CONTROL_TEMPORAL_DITHER_DEPTH_18BPP= 0x0,
+ FMT_BIT_DEPTH_CONTROL_TEMPORAL_DITHER_DEPTH_24BPP= 0x1,
+ FMT_BIT_DEPTH_CONTROL_TEMPORAL_DITHER_DEPTH_30BPP= 0x2,
+} FMT_BIT_DEPTH_CONTROL_TEMPORAL_DITHER_DEPTH;
+typedef enum FMT_BIT_DEPTH_CONTROL_TEMPORAL_LEVEL {
+ FMT_BIT_DEPTH_CONTROL_TEMPORAL_LEVEL_GREY_LEVEL2 = 0x0,
+ FMT_BIT_DEPTH_CONTROL_TEMPORAL_LEVEL_GREY_LEVEL4 = 0x1,
+} FMT_BIT_DEPTH_CONTROL_TEMPORAL_LEVEL;
+typedef enum FMT_BIT_DEPTH_CONTROL_25FRC_SEL {
+ FMT_BIT_DEPTH_CONTROL_25FRC_SEL_Ei = 0x0,
+ FMT_BIT_DEPTH_CONTROL_25FRC_SEL_Fi = 0x1,
+ FMT_BIT_DEPTH_CONTROL_25FRC_SEL_Gi = 0x2,
+ FMT_BIT_DEPTH_CONTROL_25FRC_SEL_RESERVED = 0x3,
+} FMT_BIT_DEPTH_CONTROL_25FRC_SEL;
+typedef enum FMT_BIT_DEPTH_CONTROL_50FRC_SEL {
+ FMT_BIT_DEPTH_CONTROL_50FRC_SEL_A = 0x0,
+ FMT_BIT_DEPTH_CONTROL_50FRC_SEL_B = 0x1,
+ FMT_BIT_DEPTH_CONTROL_50FRC_SEL_C = 0x2,
+ FMT_BIT_DEPTH_CONTROL_50FRC_SEL_D = 0x3,
+} FMT_BIT_DEPTH_CONTROL_50FRC_SEL;
+typedef enum FMT_BIT_DEPTH_CONTROL_75FRC_SEL {
+ FMT_BIT_DEPTH_CONTROL_75FRC_SEL_E = 0x0,
+ FMT_BIT_DEPTH_CONTROL_75FRC_SEL_F = 0x1,
+ FMT_BIT_DEPTH_CONTROL_75FRC_SEL_G = 0x2,
+ FMT_BIT_DEPTH_CONTROL_75FRC_SEL_RESERVED = 0x3,
+} FMT_BIT_DEPTH_CONTROL_75FRC_SEL;
+typedef enum FMT_TEMPORAL_DITHER_PATTERN_CONTROL_SELECT {
+ FMT_TEMPORAL_DITHER_PATTERN_CONTROL_SELECT_LEGACY_HARDCODED_PATTERN= 0x0,
+ FMT_TEMPORAL_DITHER_PATTERN_CONTROL_SELECT_PROGRAMMABLE_PATTERN= 0x1,
+} FMT_TEMPORAL_DITHER_PATTERN_CONTROL_SELECT;
+typedef enum FMT_TEMPORAL_DITHER_PATTERN_CONTROL_RGB1_BGR0 {
+ FMT_TEMPORAL_DITHER_PATTERN_CONTROL_RGB1_BGR0_BGR= 0x0,
+ FMT_TEMPORAL_DITHER_PATTERN_CONTROL_RGB1_BGR0_RGB= 0x1,
+} FMT_TEMPORAL_DITHER_PATTERN_CONTROL_RGB1_BGR0;
+typedef enum FMT_CLAMP_CNTL_COLOR_FORMAT {
+ FMT_CLAMP_CNTL_COLOR_FORMAT_6BPC = 0x0,
+ FMT_CLAMP_CNTL_COLOR_FORMAT_8BPC = 0x1,
+ FMT_CLAMP_CNTL_COLOR_FORMAT_10BPC = 0x2,
+ FMT_CLAMP_CNTL_COLOR_FORMAT_12BPC = 0x3,
+ FMT_CLAMP_CNTL_COLOR_FORMAT_RESERVED1 = 0x4,
+ FMT_CLAMP_CNTL_COLOR_FORMAT_RESERVED2 = 0x5,
+ FMT_CLAMP_CNTL_COLOR_FORMAT_RESERVED3 = 0x6,
+ FMT_CLAMP_CNTL_COLOR_FORMAT_PROGRAMMABLE = 0x7,
+} FMT_CLAMP_CNTL_COLOR_FORMAT;
+typedef enum FMT_CRC_CNTL_CONT_EN {
+ FMT_CRC_CNTL_CONT_EN_ONE_SHOT = 0x0,
+ FMT_CRC_CNTL_CONT_EN_CONT = 0x1,
+} FMT_CRC_CNTL_CONT_EN;
+typedef enum FMT_CRC_CNTL_INCLUDE_OVERSCAN {
+ FMT_CRC_CNTL_INCLUDE_OVERSCAN_NOT_INCLUDE = 0x0,
+ FMT_CRC_CNTL_INCLUDE_OVERSCAN_INCLUDE = 0x1,
+} FMT_CRC_CNTL_INCLUDE_OVERSCAN;
+typedef enum FMT_CRC_CNTL_ONLY_BLANKB {
+ FMT_CRC_CNTL_ONLY_BLANKB_ENTIRE_FIELD = 0x0,
+ FMT_CRC_CNTL_ONLY_BLANKB_NON_BLANK = 0x1,
+} FMT_CRC_CNTL_ONLY_BLANKB;
+typedef enum FMT_CRC_CNTL_PSR_MODE_ENABLE {
+ FMT_CRC_CNTL_PSR_MODE_ENABLE_NORMAL = 0x0,
+ FMT_CRC_CNTL_PSR_MODE_ENABLE_EDP_PSR_CRC = 0x1,
+} FMT_CRC_CNTL_PSR_MODE_ENABLE;
+typedef enum FMT_CRC_CNTL_INTERLACE_MODE {
+ FMT_CRC_CNTL_INTERLACE_MODE_TOP = 0x0,
+ FMT_CRC_CNTL_INTERLACE_MODE_BOTTOM = 0x1,
+ FMT_CRC_CNTL_INTERLACE_MODE_BOTH_BOTTOM = 0x2,
+ FMT_CRC_CNTL_INTERLACE_MODE_BOTH_EACH = 0x3,
+} FMT_CRC_CNTL_INTERLACE_MODE;
+typedef enum FMT_CRC_CNTL_EVEN_ODD_PIX_ENABLE {
+ FMT_CRC_CNTL_EVEN_ODD_PIX_ENABLE_ALL = 0x0,
+ FMT_CRC_CNTL_EVEN_ODD_PIX_ENABLE_ODD_EVEN = 0x1,
+} FMT_CRC_CNTL_EVEN_ODD_PIX_ENABLE;
+typedef enum FMT_CRC_CNTL_EVEN_ODD_PIX_SELECT {
+ FMT_CRC_CNTL_EVEN_ODD_PIX_SELECT_EVEN = 0x0,
+ FMT_CRC_CNTL_EVEN_ODD_PIX_SELECT_ODD = 0x1,
+} FMT_CRC_CNTL_EVEN_ODD_PIX_SELECT;
+typedef enum FMT_DEBUG_CNTL_COLOR_SELECT {
+ FMT_DEBUG_CNTL_COLOR_SELECT_BLUE = 0x0,
+ FMT_DEBUG_CNTL_COLOR_SELECT_GREEN = 0x1,
+ FMT_DEBUG_CNTL_COLOR_SELECT_RED1 = 0x2,
+ FMT_DEBUG_CNTL_COLOR_SELECT_RED2 = 0x3,
+} FMT_DEBUG_CNTL_COLOR_SELECT;
+typedef enum FMT_SPATIAL_DITHER_MODE {
+ FMT_SPATIAL_DITHER_MODE_0 = 0x0,
+ FMT_SPATIAL_DITHER_MODE_1 = 0x1,
+ FMT_SPATIAL_DITHER_MODE_2 = 0x2,
+ FMT_SPATIAL_DITHER_MODE_3 = 0x3,
+} FMT_SPATIAL_DITHER_MODE;
+typedef enum FMT_STEREOSYNC_OVR_POL {
+ FMT_STEREOSYNC_OVR_POL_INVERTED = 0x0,
+ FMT_STEREOSYNC_OVR_POL_NOT_INVERTED = 0x1,
+} FMT_STEREOSYNC_OVR_POL;
+typedef enum FMT_DYNAMIC_EXP_MODE {
+ FMT_DYNAMIC_EXP_MODE_10to12 = 0x0,
+ FMT_DYNAMIC_EXP_MODE_8to12 = 0x1,
+} FMT_DYNAMIC_EXP_MODE;
+typedef enum LB_DATA_FORMAT_PIXEL_DEPTH {
+ LB_DATA_FORMAT_PIXEL_DEPTH_30BPP = 0x0,
+ LB_DATA_FORMAT_PIXEL_DEPTH_24BPP = 0x1,
+ LB_DATA_FORMAT_PIXEL_DEPTH_18BPP = 0x2,
+ LB_DATA_FORMAT_PIXEL_DEPTH_36BPP = 0x3,
+} LB_DATA_FORMAT_PIXEL_DEPTH;
+typedef enum LB_DATA_FORMAT_PIXEL_EXPAN_MODE {
+ LB_DATA_FORMAT_PIXEL_EXPAN_MODE_ZERO_PIXEL_EXPANSION= 0x0,
+ LB_DATA_FORMAT_PIXEL_EXPAN_MODE_DYNAMIC_PIXEL_EXPANSION= 0x1,
+} LB_DATA_FORMAT_PIXEL_EXPAN_MODE;
+typedef enum LB_DATA_FORMAT_PIXEL_REDUCE_MODE {
+ LB_DATA_FORMAT_PIXEL_REDUCE_MODE_TRUNCATION = 0x0,
+ LB_DATA_FORMAT_PIXEL_REDUCE_MODE_ROUNDING = 0x1,
+} LB_DATA_FORMAT_PIXEL_REDUCE_MODE;
+typedef enum LB_DATA_FORMAT_DYNAMIC_PIXEL_DEPTH {
+ LB_DATA_FORMAT_DYNAMIC_PIXEL_DEPTH_36BPP = 0x0,
+ LB_DATA_FORMAT_DYNAMIC_PIXEL_DEPTH_30BPP = 0x1,
+} LB_DATA_FORMAT_DYNAMIC_PIXEL_DEPTH;
+typedef enum LB_DATA_FORMAT_INTERLEAVE_EN {
+ LB_DATA_FORMAT_INTERLEAVE_DISABLE = 0x0,
+ LB_DATA_FORMAT_INTERLEAVE_ENABLE = 0x1,
+} LB_DATA_FORMAT_INTERLEAVE_EN;
+typedef enum LB_DATA_FORMAT_PREFILL_EN {
+ LB_DATA_FORMAT_PREFILL_DISABLE = 0x0,
+ LB_DATA_FORMAT_PREFILL_ENABLE = 0x1,
+} LB_DATA_FORMAT_PREFILL_EN;
+typedef enum LB_DATA_FORMAT_REQUEST_MODE {
+ LB_DATA_FORMAT_REQUEST_MODE_NORMAL = 0x0,
+ LB_DATA_FORMAT_REQUEST_MODE_START_OF_LINE = 0x1,
+} LB_DATA_FORMAT_REQUEST_MODE;
+typedef enum LB_DATA_FORMAT_ALPHA_EN {
+ LB_DATA_FORMAT_ALPHA_DISABLE = 0x0,
+ LB_DATA_FORMAT_ALPHA_ENABLE = 0x1,
+} LB_DATA_FORMAT_ALPHA_EN;
+typedef enum LB_VLINE_START_END_VLINE_INV {
+ LB_VLINE_START_END_VLINE_NORMAL = 0x0,
+ LB_VLINE_START_END_VLINE_INVERSE = 0x1,
+} LB_VLINE_START_END_VLINE_INV;
+typedef enum LB_VLINE2_START_END_VLINE2_INV {
+ LB_VLINE2_START_END_VLINE2_NORMAL = 0x0,
+ LB_VLINE2_START_END_VLINE2_INVERSE = 0x1,
+} LB_VLINE2_START_END_VLINE2_INV;
+typedef enum LB_INTERRUPT_MASK_VBLANK_INTERRUPT_MASK {
+ LB_INTERRUPT_MASK_VBLANK_INTERRUPT_DISABLE = 0x0,
+ LB_INTERRUPT_MASK_VBLANK_INTERRUPT_ENABLE = 0x1,
+} LB_INTERRUPT_MASK_VBLANK_INTERRUPT_MASK;
+typedef enum LB_INTERRUPT_MASK_VLINE_INTERRUPT_MASK {
+ LB_INTERRUPT_MASK_VLINE_INTERRUPT_DISABLE = 0x0,
+ LB_INTERRUPT_MASK_VLINE_INTERRUPT_ENABLE = 0x1,
+} LB_INTERRUPT_MASK_VLINE_INTERRUPT_MASK;
+typedef enum LB_INTERRUPT_MASK_VLINE2_INTERRUPT_MASK {
+ LB_INTERRUPT_MASK_VLINE2_INTERRUPT_DISABLE = 0x0,
+ LB_INTERRUPT_MASK_VLINE2_INTERRUPT_ENABLE = 0x1,
+} LB_INTERRUPT_MASK_VLINE2_INTERRUPT_MASK;
+typedef enum LB_VLINE_STATUS_VLINE_ACK {
+ LB_VLINE_STATUS_VLINE_NORMAL = 0x0,
+ LB_VLINE_STATUS_VLINE_CLEAR = 0x1,
+} LB_VLINE_STATUS_VLINE_ACK;
+typedef enum LB_VLINE_STATUS_VLINE_INTERRUPT_TYPE {
+ LB_VLINE_STATUS_VLINE_INTERRUPT_TYPE_LEVEL_BASED = 0x0,
+ LB_VLINE_STATUS_VLINE_INTERRUPT_TYPE_PULSE_BASED = 0x1,
+} LB_VLINE_STATUS_VLINE_INTERRUPT_TYPE;
+typedef enum LB_VLINE2_STATUS_VLINE2_ACK {
+ LB_VLINE2_STATUS_VLINE2_NORMAL = 0x0,
+ LB_VLINE2_STATUS_VLINE2_CLEAR = 0x1,
+} LB_VLINE2_STATUS_VLINE2_ACK;
+typedef enum LB_VLINE2_STATUS_VLINE2_INTERRUPT_TYPE {
+ LB_VLINE2_STATUS_VLINE2_INTERRUPT_TYPE_LEVEL_BASED= 0x0,
+ LB_VLINE2_STATUS_VLINE2_INTERRUPT_TYPE_PULSE_BASED= 0x1,
+} LB_VLINE2_STATUS_VLINE2_INTERRUPT_TYPE;
+typedef enum LB_VBLANK_STATUS_VBLANK_ACK {
+ LB_VBLANK_STATUS_VBLANK_NORMAL = 0x0,
+ LB_VBLANK_STATUS_VBLANK_CLEAR = 0x1,
+} LB_VBLANK_STATUS_VBLANK_ACK;
+typedef enum LB_VBLANK_STATUS_VBLANK_INTERRUPT_TYPE {
+ LB_VBLANK_STATUS_VBLANK_INTERRUPT_TYPE_LEVEL_BASED= 0x0,
+ LB_VBLANK_STATUS_VBLANK_INTERRUPT_TYPE_PULSE_BASED= 0x1,
+} LB_VBLANK_STATUS_VBLANK_INTERRUPT_TYPE;
+typedef enum LB_SYNC_RESET_SEL_LB_SYNC_RESET_SEL {
+ LB_SYNC_RESET_SEL_LB_SYNC_RESET_SEL_DISABLE = 0x0,
+ LB_SYNC_RESET_SEL_LB_SYNC_RESET_SEL_FROM_VSYNC_VBLANK= 0x1,
+ LB_SYNC_RESET_SEL_LB_SYNC_RESET_SEL_FROM_POWERDOWN_RESET= 0x2,
+ LB_SYNC_RESET_SEL_LB_SYNC_RESET_SEL_FROM_VSYNC_VBLANK_POWERDOWN_RESET= 0x3,
+} LB_SYNC_RESET_SEL_LB_SYNC_RESET_SEL;
+typedef enum LB_SYNC_RESET_SEL_LB_SYNC_RESET_SEL2 {
+ LB_SYNC_RESET_SEL_LB_SYNC_RESET_SEL2_USE_VBLANK = 0x0,
+ LB_SYNC_RESET_SEL_LB_SYNC_RESET_SEL2_USE_VSYNC = 0x1,
+} LB_SYNC_RESET_SEL_LB_SYNC_RESET_SEL2;
+typedef enum LB_SYNC_RESET_SEL_LB_SYNC_DURATION {
+ LB_SYNC_RESET_SEL_LB_SYNC_DURATION_16_CLOCKS = 0x0,
+ LB_SYNC_RESET_SEL_LB_SYNC_DURATION_32_CLOCKS = 0x1,
+ LB_SYNC_RESET_SEL_LB_SYNC_DURATION_64_CLOCKS = 0x2,
+ LB_SYNC_RESET_SEL_LB_SYNC_DURATION_128_CLOCKS = 0x3,
+} LB_SYNC_RESET_SEL_LB_SYNC_DURATION;
+typedef enum LB_KEYER_COLOR_CTRL_LB_KEYER_COLOR_EN {
+ LB_KEYER_COLOR_CTRL_LB_KEYER_COLOR_DISABLE = 0x0,
+ LB_KEYER_COLOR_CTRL_LB_KEYER_COLOR_ENABLE = 0x1,
+} LB_KEYER_COLOR_CTRL_LB_KEYER_COLOR_EN;
+typedef enum LB_KEYER_COLOR_CTRL_LB_KEYER_COLOR_REP_EN {
+ LB_KEYER_COLOR_CTRL_LB_KEYER_COLOR_REPLACEMENT_DISABLE= 0x0,
+ LB_KEYER_COLOR_CTRL_LB_KEYER_COLOR_REPLACEMENT_ENABLE= 0x1,
+} LB_KEYER_COLOR_CTRL_LB_KEYER_COLOR_REP_EN;
+typedef enum LB_BUFFER_STATUS_LB_BUFFER_EMPTY_ACK {
+ LB_BUFFER_STATUS_LB_BUFFER_EMPTY_NORMAL = 0x0,
+ LB_BUFFER_STATUS_LB_BUFFER_EMPTY_RESET = 0x1,
+} LB_BUFFER_STATUS_LB_BUFFER_EMPTY_ACK;
+typedef enum LB_BUFFER_STATUS_LB_BUFFER_FULL_ACK {
+ LB_BUFFER_STATUS_LB_BUFFER_FULL_NORMAL = 0x0,
+ LB_BUFFER_STATUS_LB_BUFFER_FULL_RESET = 0x1,
+} LB_BUFFER_STATUS_LB_BUFFER_FULL_ACK;
+typedef enum LB_MVP_AFR_FLIP_MODE_MVP_AFR_FLIP_MODE {
+ LB_MVP_AFR_FLIP_MODE_MVP_AFR_FLIP_MODE_REAL_FLIP = 0x2,
+ LB_MVP_AFR_FLIP_MODE_MVP_AFR_FLIP_MODE_DUMMY_FLIP= 0x3,
+} LB_MVP_AFR_FLIP_MODE_MVP_AFR_FLIP_MODE;
+typedef enum LB_MVP_AFR_FLIP_FIFO_CNTL_MVP_AFR_FLIP_FIFO_RESET {
+ LB_MVP_AFR_FLIP_FIFO_CNTL_MVP_AFR_FLIP_FIFO_NORMAL= 0x0,
+ LB_MVP_AFR_FLIP_FIFO_CNTL_MVP_AFR_FLIP_FIFO_RESET_ACTIVE= 0x1,
+} LB_MVP_AFR_FLIP_FIFO_CNTL_MVP_AFR_FLIP_FIFO_RESET;
+typedef enum LB_MVP_AFR_FLIP_FIFO_CNTL_MVP_AFR_FLIP_FIFO_RESET_ACK {
+ LB_MVP_AFR_FLIP_FIFO_CNTL_MVP_AFR_FLIP_FIFO_RESET_ACK_NOT_USED0= 0x0,
+ LB_MVP_AFR_FLIP_FIFO_CNTL_MVP_AFR_FLIP_FIFO_RESET_ACK_NOT_USED1= 0x1,
+} LB_MVP_AFR_FLIP_FIFO_CNTL_MVP_AFR_FLIP_FIFO_RESET_ACK;
+typedef enum LB_MVP_FLIP_LINE_NUM_INSERT_MVP_FLIP_LINE_NUM_INSERT_MODE {
+ LB_MVP_FLIP_LINE_NUM_INSERT_MVP_FLIP_LINE_NUM_INSERT_MODE_NO_INSERT= 0x0,
+ LB_MVP_FLIP_LINE_NUM_INSERT_MVP_FLIP_LINE_NUM_INSERT_MODE_DEBUG= 0x1,
+ LB_MVP_FLIP_LINE_NUM_INSERT_MVP_FLIP_LINE_NUM_INSERT_MODE_HSYNC_MODE= 0x2,
+} LB_MVP_FLIP_LINE_NUM_INSERT_MVP_FLIP_LINE_NUM_INSERT_MODE;
+typedef enum LB_MVP_FLIP_LINE_NUM_INSERT_MVP_FLIP_AUTO_ENABLE {
+ LB_MVP_FLIP_LINE_NUM_INSERT_MVP_FLIP_AUTO_DISABLE= 0x0,
+ LB_MVP_FLIP_LINE_NUM_INSERT_MVP_FLIP_AUTO_EN = 0x1,
+} LB_MVP_FLIP_LINE_NUM_INSERT_MVP_FLIP_AUTO_ENABLE;
+typedef enum LB_DC_MVP_LB_CONTROL_MVP_SWAP_LOCK_IN_MODE {
+ ALPHA_LB_DC_MVP_LB_CONTROL_MVP_SWAP_LOCK_IN_MODE_MASTER= 0x1,
+ ALPHA_LB_DC_MVP_LB_CONTROL_MVP_SWAP_LOCK_IN_MODE_SLAVE= 0x2,
+} LB_DC_MVP_LB_CONTROL_MVP_SWAP_LOCK_IN_MODE;
+typedef enum LB_DC_MVP_LB_CONTROL_DC_MVP_SWAP_LOCK_OUT_SEL {
+ LB_DC_MVP_LB_CONTROL_DC_MVP_SWAP_LOCK_OUT_SEL_NOT_USED0= 0x0,
+ LB_DC_MVP_LB_CONTROL_DC_MVP_SWAP_LOCK_OUT_SEL_NOT_USED1= 0x1,
+} LB_DC_MVP_LB_CONTROL_DC_MVP_SWAP_LOCK_OUT_SEL;
+typedef enum LB_DC_MVP_LB_CONTROL_DC_MVP_SWAP_LOCK_OUT_FORCE_ONE {
+ LB_DC_MVP_LB_CONTROL_DC_MVP_SWAP_LOCK_OUT_NO_FORCE_ONE= 0x0,
+ LB_DC_MVP_LB_CONTROL_DC_MVP_SWAP_LOCK_OUT_FORCE_TO_ONE= 0x1,
+} LB_DC_MVP_LB_CONTROL_DC_MVP_SWAP_LOCK_OUT_FORCE_ONE;
+typedef enum LB_DC_MVP_LB_CONTROL_DC_MVP_SWAP_LOCK_OUT_FORCE_ZERO {
+ LB_DC_MVP_LB_CONTROL_DC_MVP_SWAP_LOCK_OUT_NO_FORCE_ZERO= 0x0,
+ LB_DC_MVP_LB_CONTROL_DC_MVP_SWAP_LOCK_OUT_FORCE_TO_ZERO= 0x1,
+} LB_DC_MVP_LB_CONTROL_DC_MVP_SWAP_LOCK_OUT_FORCE_ZERO;
+typedef enum LB_TEST_DEBUG_INDEX_LB_TEST_DEBUG_WRITE_EN {
+ LB_TEST_DEBUG_INDEX_LB_TEST_DEBUG_WRITE_EN_NOT_USED0= 0x0,
+ LB_TEST_DEBUG_INDEX_LB_TEST_DEBUG_WRITE_EN_NOT_USED1= 0x1,
+} LB_TEST_DEBUG_INDEX_LB_TEST_DEBUG_WRITE_EN;
+typedef enum LBV_PIXEL_DEPTH {
+ PIXEL_DEPTH_30BPP = 0x0,
+ PIXEL_DEPTH_24BPP = 0x1,
+ PIXEL_DEPTH_18BPP = 0x2,
+ PIXEL_DEPTH_38BPP = 0x3,
+} LBV_PIXEL_DEPTH;
+typedef enum LBV_PIXEL_EXPAN_MODE {
+ PIXEL_EXPAN_MODE_ZERO_EXP = 0x0,
+ PIXEL_EXPAN_MODE_DYN_EXP = 0x1,
+} LBV_PIXEL_EXPAN_MODE;
+typedef enum LBV_INTERLEAVE_EN {
+ INTERLEAVE_DIS = 0x0,
+ INTERLEAVE_EN = 0x1,
+} LBV_INTERLEAVE_EN;
+typedef enum LBV_PIXEL_REDUCE_MODE {
+ PIXEL_REDUCE_MODE_TRUNCATION = 0x0,
+ PIXEL_REDUCE_MODE_ROUNDING = 0x1,
+} LBV_PIXEL_REDUCE_MODE;
+typedef enum LBV_DYNAMIC_PIXEL_DEPTH {
+ DYNAMIC_PIXEL_DEPTH_36BPP = 0x0,
+ DYNAMIC_PIXEL_DEPTH_30BPP = 0x1,
+} LBV_DYNAMIC_PIXEL_DEPTH;
+typedef enum LBV_DITHER_EN {
+ DITHER_DIS = 0x0,
+ DITHER_EN = 0x1,
+} LBV_DITHER_EN;
+typedef enum LBV_DOWNSCALE_PREFETCH_EN {
+ DOWNSCALE_PREFETCH_DIS = 0x0,
+ DOWNSCALE_PREFETCH_EN = 0x1,
+} LBV_DOWNSCALE_PREFETCH_EN;
+typedef enum LBV_MEMORY_CONFIG {
+ MEMORY_CONFIG_0 = 0x0,
+ MEMORY_CONFIG_1 = 0x1,
+ MEMORY_CONFIG_2 = 0x2,
+ MEMORY_CONFIG_3 = 0x3,
+} LBV_MEMORY_CONFIG;
+typedef enum LBV_SYNC_RESET_SEL2 {
+ SYNC_RESET_SEL2_VBLANK = 0x0,
+ SYNC_RESET_SEL2_VSYNC = 0x1,
+} LBV_SYNC_RESET_SEL2;
+typedef enum LBV_SYNC_DURATION {
+ SYNC_DURATION_16 = 0x0,
+ SYNC_DURATION_32 = 0x1,
+ SYNC_DURATION_64 = 0x2,
+ SYNC_DURATION_128 = 0x3,
+} LBV_SYNC_DURATION;
+typedef enum SCL_C_RAM_TAP_PAIR_IDX {
+ SCL_C_RAM_TAP_PAIR_ID0 = 0x0,
+ SCL_C_RAM_TAP_PAIR_ID1 = 0x1,
+ SCL_C_RAM_TAP_PAIR_ID2 = 0x2,
+ SCL_C_RAM_TAP_PAIR_ID3 = 0x3,
+ SCL_C_RAM_TAP_PAIR_ID4 = 0x4,
+} SCL_C_RAM_TAP_PAIR_IDX;
+typedef enum SCL_C_RAM_PHASE {
+ SCL_C_RAM_PHASE_0 = 0x0,
+ SCL_C_RAM_PHASE_1 = 0x1,
+ SCL_C_RAM_PHASE_2 = 0x2,
+ SCL_C_RAM_PHASE_3 = 0x3,
+ SCL_C_RAM_PHASE_4 = 0x4,
+ SCL_C_RAM_PHASE_5 = 0x5,
+ SCL_C_RAM_PHASE_6 = 0x6,
+ SCL_C_RAM_PHASE_7 = 0x7,
+ SCL_C_RAM_PHASE_8 = 0x8,
+} SCL_C_RAM_PHASE;
+typedef enum SCL_C_RAM_FILTER_TYPE {
+ SCL_C_RAM_FILTER_TYPE_VERT_LUMA_RGB_LUT = 0x0,
+ SCL_C_RAM_FILTER_TYPE_VERT_CHROMA_LUT = 0x1,
+ SCL_C_RAM_FILTER_TYPE_HORI_LUMA_RGB_LUT = 0x2,
+ SCL_C_RAM_FILTER_TYPE_HORI_CHROMA_LUT = 0x3,
+} SCL_C_RAM_FILTER_TYPE;
+typedef enum SCL_MODE_SEL {
+ SCL_MODE_RGB_BYPASS = 0x0,
+ SCL_MODE_RGB_SCALING = 0x1,
+ SCL_MODE_YCBCR_SCALING = 0x2,
+ SCL_MODE_YCBCR_BYPASS = 0x3,
+} SCL_MODE_SEL;
+typedef enum SCL_PSCL_EN {
+ SCL_PSCL_DISABLE = 0x0,
+ SCL_PSCL_ENANBLE = 0x1,
+} SCL_PSCL_EN;
+typedef enum SCL_V_NUM_OF_TAPS {
+ SCL_V_NUM_OF_TAPS_1 = 0x0,
+ SCL_V_NUM_OF_TAPS_2 = 0x1,
+ SCL_V_NUM_OF_TAPS_3 = 0x2,
+ SCL_V_NUM_OF_TAPS_4 = 0x3,
+ SCL_V_NUM_OF_TAPS_5 = 0x4,
+ SCL_V_NUM_OF_TAPS_6 = 0x5,
+} SCL_V_NUM_OF_TAPS;
+typedef enum SCL_H_NUM_OF_TAPS {
+ SCL_H_NUM_OF_TAPS_1 = 0x0,
+ SCL_H_NUM_OF_TAPS_2 = 0x1,
+ SCL_H_NUM_OF_TAPS_4 = 0x3,
+ SCL_H_NUM_OF_TAPS_6 = 0x5,
+ SCL_H_NUM_OF_TAPS_8 = 0x7,
+ SCL_H_NUM_OF_TAPS_10 = 0x9,
+} SCL_H_NUM_OF_TAPS;
+typedef enum SCL_BOUNDARY_MODE {
+ SCL_BOUNDARY_MODE_BLACK = 0x0,
+ SCL_BOUNDARY_MODE_EDGE = 0x1,
+} SCL_BOUNDARY_MODE;
+typedef enum SCL_EARLY_EOL_MOD {
+ SCL_EARLY_EOL_MODE_CRTC = 0x0,
+ SCL_EARLY_EOL_MODE_INTERNAL = 0x1,
+} SCL_EARLY_EOL_MOD;
+typedef enum SCL_BYPASS_MODE {
+ SCL_BYPASS_MODE_MC_MR = 0x0,
+ SCL_BYPASS_MODE_AC_NR = 0x1,
+ SCL_BYPASS_MODE_AC_AR = 0x2,
+ SCL_BYPASS_MODE_RESERVED = 0x3,
+} SCL_BYPASS_MODE;
+typedef enum SCL_V_MANUAL_REPLICATE_FACTOR {
+ SCL_V_MANUAL_REPLICATE_FACTOR_1 = 0x0,
+ SCL_V_MANUAL_REPLICATE_FACTOR_2 = 0x1,
+ SCL_V_MANUAL_REPLICATE_FACTOR_3 = 0x2,
+ SCL_V_MANUAL_REPLICATE_FACTOR_4 = 0x3,
+ SCL_V_MANUAL_REPLICATE_FACTOR_5 = 0x4,
+ SCL_V_MANUAL_REPLICATE_FACTOR_6 = 0x5,
+ SCL_V_MANUAL_REPLICATE_FACTOR_7 = 0x6,
+ SCL_V_MANUAL_REPLICATE_FACTOR_8 = 0x7,
+ SCL_V_MANUAL_REPLICATE_FACTOR_9 = 0x8,
+ SCL_V_MANUAL_REPLICATE_FACTOR_10 = 0x9,
+ SCL_V_MANUAL_REPLICATE_FACTOR_11 = 0xa,
+ SCL_V_MANUAL_REPLICATE_FACTOR_12 = 0xb,
+ SCL_V_MANUAL_REPLICATE_FACTOR_13 = 0xc,
+ SCL_V_MANUAL_REPLICATE_FACTOR_14 = 0xd,
+ SCL_V_MANUAL_REPLICATE_FACTOR_15 = 0xe,
+ SCL_V_MANUAL_REPLICATE_FACTOR_16 = 0xf,
+} SCL_V_MANUAL_REPLICATE_FACTOR;
+typedef enum SCL_H_MANUAL_REPLICATE_FACTOR {
+ SCL_H_MANUAL_REPLICATE_FACTOR_1 = 0x0,
+ SCL_H_MANUAL_REPLICATE_FACTOR_2 = 0x1,
+ SCL_H_MANUAL_REPLICATE_FACTOR_3 = 0x2,
+ SCL_H_MANUAL_REPLICATE_FACTOR_4 = 0x3,
+ SCL_H_MANUAL_REPLICATE_FACTOR_5 = 0x4,
+ SCL_H_MANUAL_REPLICATE_FACTOR_6 = 0x5,
+ SCL_H_MANUAL_REPLICATE_FACTOR_7 = 0x6,
+ SCL_H_MANUAL_REPLICATE_FACTOR_8 = 0x7,
+ SCL_H_MANUAL_REPLICATE_FACTOR_9 = 0x8,
+ SCL_H_MANUAL_REPLICATE_FACTOR_10 = 0x9,
+ SCL_H_MANUAL_REPLICATE_FACTOR_11 = 0xa,
+ SCL_H_MANUAL_REPLICATE_FACTOR_12 = 0xb,
+ SCL_H_MANUAL_REPLICATE_FACTOR_13 = 0xc,
+ SCL_H_MANUAL_REPLICATE_FACTOR_14 = 0xd,
+ SCL_H_MANUAL_REPLICATE_FACTOR_15 = 0xe,
+ SCL_H_MANUAL_REPLICATE_FACTOR_16 = 0xf,
+} SCL_H_MANUAL_REPLICATE_FACTOR;
+typedef enum SCL_V_CALC_AUTO_RATIO_EN {
+ SCL_V_CALC_AUTO_RATIO_DISABLE = 0x0,
+ SCL_V_CALC_AUTO_RATIO_ENABLE = 0x1,
+} SCL_V_CALC_AUTO_RATIO_EN;
+typedef enum SCL_H_CALC_AUTO_RATIO_EN {
+ SCL_H_CALC_AUTO_RATIO_DISABLE = 0x0,
+ SCL_H_CALC_AUTO_RATIO_ENABLE = 0x1,
+} SCL_H_CALC_AUTO_RATIO_EN;
+typedef enum SCL_H_FILTER_PICK_NEAREST {
+ SCL_H_FILTER_PICK_NEAREST_DISABLE = 0x0,
+ SCL_H_FILTER_PICK_NEAREST_ENABLE = 0x1,
+} SCL_H_FILTER_PICK_NEAREST;
+typedef enum SCL_H_2TAP_HARDCODE_COEF_EN {
+ SCL_H_2TAP_HARDCODE_COEF_DISABLE = 0x0,
+ SCL_H_2TAP_HARDCODE_COEF_ENABLE = 0x1,
+} SCL_H_2TAP_HARDCODE_COEF_EN;
+typedef enum SCL_V_FILTER_PICK_NEAREST {
+ SCL_V_FILTER_PICK_NEAREST_DISABLE = 0x0,
+ SCL_V_FILTER_PICK_NEAREST_ENABLE = 0x1,
+} SCL_V_FILTER_PICK_NEAREST;
+typedef enum SCL_V_2TAP_HARDCODE_COEF_EN {
+ SCL_V_2TAP_HARDCODE_COEF_DISABLE = 0x0,
+ SCL_V_2TAP_HARDCODE_COEF_ENABLE = 0x1,
+} SCL_V_2TAP_HARDCODE_COEF_EN;
+typedef enum SCL_UPDATE_TAKEN {
+ SCL_UPDATE_TAKEN_NO = 0x0,
+ SCL_UPDATE_TAKEN_YES = 0x1,
+} SCL_UPDATE_TAKEN;
+typedef enum SCL_UPDATE_LOCK {
+ SCL_UPDATE_UNLOCKED = 0x0,
+ SCL_UPDATE_LOCKED = 0x1,
+} SCL_UPDATE_LOCK;
+typedef enum SCL_COEF_UPDATE_COMPLETE {
+ SCL_COEF_UPDATE_NOT_COMPLETED = 0x0,
+ SCL_COEF_UPDATE_COMPLETED = 0x1,
+} SCL_COEF_UPDATE_COMPLETE;
+typedef enum SCL_HF_SHARP_SCALE_FACTOR {
+ SCL_HF_SHARP_SCALE_FACTOR_0 = 0x0,
+ SCL_HF_SHARP_SCALE_FACTOR_1 = 0x1,
+ SCL_HF_SHARP_SCALE_FACTOR_2 = 0x2,
+ SCL_HF_SHARP_SCALE_FACTOR_3 = 0x3,
+ SCL_HF_SHARP_SCALE_FACTOR_4 = 0x4,
+ SCL_HF_SHARP_SCALE_FACTOR_5 = 0x5,
+ SCL_HF_SHARP_SCALE_FACTOR_6 = 0x6,
+ SCL_HF_SHARP_SCALE_FACTOR_7 = 0x7,
+} SCL_HF_SHARP_SCALE_FACTOR;
+typedef enum SCL_HF_SHARP_EN {
+ SCL_HF_SHARP_DISABLE = 0x0,
+ SCL_HF_SHARP_ENABLE = 0x1,
+} SCL_HF_SHARP_EN;
+typedef enum SCL_VF_SHARP_SCALE_FACTOR {
+ SCL_VF_SHARP_SCALE_FACTOR_0 = 0x0,
+ SCL_VF_SHARP_SCALE_FACTOR_1 = 0x1,
+ SCL_VF_SHARP_SCALE_FACTOR_2 = 0x2,
+ SCL_VF_SHARP_SCALE_FACTOR_3 = 0x3,
+ SCL_VF_SHARP_SCALE_FACTOR_4 = 0x4,
+ SCL_VF_SHARP_SCALE_FACTOR_5 = 0x5,
+ SCL_VF_SHARP_SCALE_FACTOR_6 = 0x6,
+ SCL_VF_SHARP_SCALE_FACTOR_7 = 0x7,
+} SCL_VF_SHARP_SCALE_FACTOR;
+typedef enum SCL_VF_SHARP_EN {
+ SCL_VF_SHARP_DISABLE = 0x0,
+ SCL_VF_SHARP_ENABLE = 0x1,
+} SCL_VF_SHARP_EN;
+typedef enum SCL_ALU_DISABLE {
+ SCL_ALU_ENABLED = 0x0,
+ SCL_ALU_DISABLED = 0x1,
+} SCL_ALU_DISABLE;
+typedef enum SCL_HOST_CONFLICT_MASK {
+ SCL_HOST_CONFLICT_DISABLE_INTERRUPT = 0x0,
+ SCL_HOST_CONFLICT_ENABLE_INTERRUPT = 0x1,
+} SCL_HOST_CONFLICT_MASK;
+typedef enum SCL_SCL_MODE_CHANGE_MASK {
+ SCL_MODE_CHANGE_DISABLE_INTERRUPT = 0x0,
+ SCL_MODE_CHANGE_ENABLE_INTERRUPT = 0x1,
+} SCL_SCL_MODE_CHANGE_MASK;
+typedef enum SCLV_MODE_SEL {
+ SCLV_MODE_RGB_BYPASS = 0x0,
+ SCLV_MODE_RGB_SCALING = 0x1,
+ SCLV_MODE_YCBCR_SCALING = 0x2,
+ SCLV_MODE_YCBCR_BYPASS = 0x3,
+} SCLV_MODE_SEL;
+typedef enum SCLV_INTERLACE_SOURCE {
+ INTERLACE_SOURCE_PROGRESSIVE = 0x0,
+ INTERLACE_SOURCE_INTERLEAVE = 0x1,
+ INTERLACE_SOURCE_STACK = 0x2,
+} SCLV_INTERLACE_SOURCE;
+typedef enum SCLV_UPDATE_LOCK {
+ UPDATE_UNLOCKED = 0x0,
+ UPDATE_LOCKED = 0x1,
+} SCLV_UPDATE_LOCK;
+typedef enum SCLV_COEF_UPDATE_COMPLETE {
+ COEF_UPDATE_NOT_COMPLETE = 0x0,
+ COEF_UPDATE_COMPLETE = 0x1,
+} SCLV_COEF_UPDATE_COMPLETE;
+typedef enum COL_MAN_UPDATE_LOCK {
+ COL_MAN_UPDATE_UNLOCKED = 0x0,
+ COL_MAN_UPDATE_LOCKED = 0x1,
+} COL_MAN_UPDATE_LOCK;
+typedef enum COL_MAN_DISABLE_MULTIPLE_UPDATE {
+ COL_MAN_MULTIPLE_UPDATE = 0x0,
+ COL_MAN_MULTIPLE_UPDAT_EDISABLE = 0x1,
+} COL_MAN_DISABLE_MULTIPLE_UPDATE;
+typedef enum COL_MAN_INPUTCSC_MODE {
+ INPUTCSC_MODE_BYPASS = 0x0,
+ INPUTCSC_MODE_A = 0x1,
+ INPUTCSC_MODE_B = 0x2,
+ INPUTCSC_MODE_UNITY = 0x3,
+} COL_MAN_INPUTCSC_MODE;
+typedef enum COL_MAN_INPUTCSC_TYPE {
+ INPUTCSC_TYPE_12_0 = 0x0,
+ INPUTCSC_TYPE_10_2 = 0x1,
+ INPUTCSC_TYPE_8_4 = 0x2,
+} COL_MAN_INPUTCSC_TYPE;
+typedef enum COL_MAN_INPUTCSC_CONVERT {
+ INPUTCSC_ROUND = 0x0,
+ INPUTCSC_TRUNCATE = 0x1,
+} COL_MAN_INPUTCSC_CONVERT;
+typedef enum COL_MAN_PRESCALE_MODE {
+ PRESCALE_MODE_BYPASS = 0x0,
+ PRESCALE_MODE_PROGRAM = 0x1,
+ PRESCALE_MODE_UNITY = 0x2,
+} COL_MAN_PRESCALE_MODE;
+typedef enum COL_MAN_INPUT_GAMMA_MODE {
+ INGAMMA_MODE_BYPASS = 0x0,
+ INGAMMA_MODE_FIX = 0x1,
+ INGAMMA_MODE_FLOAT = 0x2,
+} COL_MAN_INPUT_GAMMA_MODE;
+typedef enum COL_MAN_OUTPUT_CSC_MODE {
+ COL_MAN_OUTPUT_CSC_BYPASS = 0x0,
+ COL_MAN_OUTPUT_CSC_RGB = 0x1,
+ COL_MAN_OUTPUT_CSC_YCrCb601 = 0x2,
+ COL_MAN_OUTPUT_CSC_YCrCb709 = 0x3,
+ COL_MAN_OUTPUT_CSC_A = 0x4,
+ COL_MAN_OUTPUT_CSC_B = 0x5,
+ COL_MAN_OUTPUT_CSC_UNITY = 0x6,
+} COL_MAN_OUTPUT_CSC_MODE;
+typedef enum COL_MAN_DENORM_CLAMP_CONTROL {
+ DENORM_CLAMP_MODE_UNITY = 0x0,
+ DENORM_CLAMP_MODE_8 = 0x1,
+ DENORM_CLAMP_MODE_10 = 0x2,
+ DENORM_CLAMP_MODE_12 = 0x3,
+} COL_MAN_DENORM_CLAMP_CONTROL;
+typedef enum COL_MAN_GAMMA_CORR_CONTROL {
+ GAMMA_CORR_MODE_BYPASS = 0x0,
+ GAMMA_CORR_MODE_A = 0x1,
+ GAMMA_CORR_MODE_B = 0x2,
+} COL_MAN_GAMMA_CORR_CONTROL;
+typedef enum COL_MAN_GLOBAL_PASSTHROUGH_ENABLE {
+ CM_GLOBAL_PASSTHROUGH_DISBALE = 0x0,
+ CM_GLOBAL_PASSTHROUGH_ENABLE = 0x1,
+} COL_MAN_GLOBAL_PASSTHROUGH_ENABLE;
+typedef enum UNP_GRPH_EN {
+ UNP_GRPH_DISABLED = 0x0,
+ UNP_GRPH_ENABLED = 0x1,
+} UNP_GRPH_EN;
+typedef enum UNP_GRPH_DEPTH {
+ UNP_GRPH_8BPP = 0x0,
+ UNP_GRPH_16BPP = 0x1,
+ UNP_GRPH_32BPP = 0x2,
+} UNP_GRPH_DEPTH;
+typedef enum UNP_GRPH_NUM_BANKS {
+ UNP_GRPH_ADDR_SURF_2_BANK = 0x0,
+ UNP_GRPH_ADDR_SURF_4_BANK = 0x1,
+ UNP_GRPH_ADDR_SURF_8_BANK = 0x2,
+ UNP_GRPH_ADDR_SURF_16_BANK = 0x3,
+} UNP_GRPH_NUM_BANKS;
+typedef enum UNP_GRPH_BANK_WIDTH {
+ UNP_GRPH_ADDR_SURF_BANK_WIDTH_1 = 0x0,
+ UNP_GRPH_ADDR_SURF_BANK_WIDTH_2 = 0x1,
+ UNP_GRPH_ADDR_SURF_BANK_WIDTH_4 = 0x2,
+ UNP_GRPH_ADDR_SURF_BANK_WIDTH_8 = 0x3,
+} UNP_GRPH_BANK_WIDTH;
+typedef enum UNP_GRPH_BANK_HEIGHT {
+ UNP_GRPH_ADDR_SURF_BANK_HEIGHT_1 = 0x0,
+ UNP_GRPH_ADDR_SURF_BANK_HEIGHT_2 = 0x1,
+ UNP_GRPH_ADDR_SURF_BANK_HEIGHT_4 = 0x2,
+ UNP_GRPH_ADDR_SURF_BANK_HEIGHT_8 = 0x3,
+} UNP_GRPH_BANK_HEIGHT;
+typedef enum UNP_GRPH_TILE_SPLIT {
+ UNP_ADDR_SURF_TILE_SPLIT_64B = 0x0,
+ UNP_ADDR_SURF_TILE_SPLIT_128B = 0x1,
+ UNP_ADDR_SURF_TILE_SPLIT_256B = 0x2,
+ UNP_ADDR_SURF_TILE_SPLIT_512B = 0x3,
+ UNP_ADDR_SURF_TILE_SPLIT_1KB = 0x4,
+ UNP_ADDR_SURF_TILE_SPLIT_2KB = 0x5,
+ UNP_ADDR_SURF_TILE_SPLIT_4KB = 0x6,
+} UNP_GRPH_TILE_SPLIT;
+typedef enum UNP_GRPH_ADDRESS_TRANSLATION_ENABLE {
+ UNP_GRPH_ADDRESS_TRANSLATION_ENABLE0 = 0x0,
+ UNP_GRPH_ADDRESS_TRANSLATION_ENABLE1 = 0x1,
+} UNP_GRPH_ADDRESS_TRANSLATION_ENABLE;
+typedef enum UNP_GRPH_PRIVILEGED_ACCESS_ENABLE {
+ UNP_GRPH_PRIVILEGED_ACCESS_DIS = 0x0,
+ UNP_GRPH_PRIVILEGED_ACCESS_EN = 0x1,
+} UNP_GRPH_PRIVILEGED_ACCESS_ENABLE;
+typedef enum UNP_GRPH_MACRO_TILE_ASPECT {
+ UNP_ADDR_SURF_MACRO_ASPECT_1 = 0x0,
+ UNP_ADDR_SURF_MACRO_ASPECT_2 = 0x1,
+ UNP_ADDR_SURF_MACRO_ASPECT_4 = 0x2,
+ UNP_ADDR_SURF_MACRO_ASPECT_8 = 0x3,
+} UNP_GRPH_MACRO_TILE_ASPECT;
+typedef enum UNP_GRPH_COLOR_EXPANSION_MODE {
+ UNP_GRPH_DYNAMIC_EXPANSION = 0x0,
+ UNP_GRPH_ZERO_EXPANSION = 0x1,
+} UNP_GRPH_COLOR_EXPANSION_MODE;
+typedef enum UNP_VIDEO_FORMAT {
+ UNP_VIDEO_FORMAT0 = 0x0,
+ UNP_VIDEO_FORMAT1 = 0x1,
+ UNP_VIDEO_FORMAT_YUV420_YCbCr = 0x2,
+ UNP_VIDEO_FORMAT_YUV420_YCrCb = 0x3,
+ UNP_VIDEO_FORMAT_YUV422_YCb = 0x4,
+ UNP_VIDEO_FORMAT_YUV422_YCr = 0x5,
+ UNP_VIDEO_FORMAT_YUV422_CbY = 0x6,
+ UNP_VIDEO_FORMAT_YUV422_CrY = 0x7,
+} UNP_VIDEO_FORMAT;
+typedef enum UNP_GRPH_ENDIAN_SWAP {
+ UNP_GRPH_ENDIAN_SWAP_NONE = 0x0,
+ UNP_GRPH_ENDIAN_SWAP_8IN16 = 0x1,
+ UNP_GRPH_ENDIAN_SWAP_8IN32 = 0x2,
+ UNP_GRPH_ENDIAN_SWAP_8IN43 = 0x3,
+} UNP_GRPH_ENDIAN_SWAP;
+typedef enum UNP_GRPH_RED_CROSSBAR {
+ UNP_GRPH_RED_CROSSBAR_R_Cr = 0x0,
+ UNP_GRPH_RED_CROSSBAR_G_Y = 0x1,
+ UNP_GRPH_RED_CROSSBAR_B_Cb = 0x2,
+ UNP_GRPH_RED_CROSSBAR_A = 0x3,
+} UNP_GRPH_RED_CROSSBAR;
+typedef enum UNP_GRPH_GREEN_CROSSBAR {
+ UNP_UNP_GRPH_GREEN_CROSSBAR_GY_AND_Y = 0x0,
+ UNP_UNP_GRPH_GREEN_CROSSBAR_B_Cb_AND_C = 0x1,
+ UNP_UNP_GRPH_GREEN_CROSSBAR_A = 0x2,
+ UNP_UNP_GRPH_GREEN_CROSSBAR_R_Cr = 0x3,
+} UNP_GRPH_GREEN_CROSSBAR;
+typedef enum UNP_GRPH_BLUE_CROSSBAR {
+ UNP_GRPH_BLUE_CROSSBAR_B_Cb_AND_C = 0x0,
+ UNP_GRPH_BLUE_CROSSBAR_A = 0x1,
+ UNP_GRPH_BLUE_CROSSBAR_R_Cr = 0x2,
+ UNP_GRPH_BLUE_CROSSBAR_GY_AND_Y = 0x3,
+} UNP_GRPH_BLUE_CROSSBAR;
+typedef enum UNP_GRPH_MODE_UPDATE_LOCKG {
+ UNP_GRPH_UPDATE_LOCK_0 = 0x0,
+ UNP_GRPH_UPDATE_LOCK_1 = 0x1,
+} UNP_GRPH_MODE_UPDATE_LOCKG;
+typedef enum UNP_GRPH_SURFACE_IGNORE_UPDATE_LOCK {
+ UNP_GRPH_SURFACE_IGNORE_UPDATE_LOCK_0 = 0x0,
+ UNP_GRPH_SURFACE_IGNORE_UPDATE_LOCK_1 = 0x1,
+} UNP_GRPH_SURFACE_IGNORE_UPDATE_LOCK;
+typedef enum UNP_GRPH_MODE_DISABLE_MULTIPLE_UPDATE {
+ UNP_GRPH_MODE_DISABLE_MULTIPLE_UPDATE_0 = 0x0,
+ UNP_GRPH_MODE_DISABLE_MULTIPLE_UPDATE_1 = 0x1,
+} UNP_GRPH_MODE_DISABLE_MULTIPLE_UPDATE;
+typedef enum UNP_GRPH_SURFACE_DISABLE_MULTIPLE_UPDATE {
+ UNP_GRPH_SURFACE_DISABLE_MULTIPLE_UPDATE_0 = 0x0,
+ UNP_GRPH_SURFACE_DISABLE_MULTIPLE_UPDATE_1 = 0x1,
+} UNP_GRPH_SURFACE_DISABLE_MULTIPLE_UPDATE;
+typedef enum UNP_GRPH_STEREOSYNC_FLIP_EN {
+ UNP_GRPH_STEREOSYNC_FLIP_DISABLE = 0x0,
+ UNP_GRPH_STEREOSYNC_FLIP_ENABLE = 0x1,
+} UNP_GRPH_STEREOSYNC_FLIP_EN;
+typedef enum UNP_GRPH_STEREOSYNC_FLIP_MODE {
+ UNP_GRPH_STEREOSYNC_FLIP_MODE_0 = 0x0,
+ UNP_GRPH_STEREOSYNC_FLIP_MODE_1 = 0x1,
+ UNP_GRPH_STEREOSYNC_FLIP_MODE_2 = 0x2,
+ UNP_GRPH_STEREOSYNC_FLIP_MODE_3 = 0x3,
+} UNP_GRPH_STEREOSYNC_FLIP_MODE;
+typedef enum UNP_GRPH_STACK_INTERLACE_FLIP_EN {
+ UNP_GRPH_STACK_INTERLACE_FLIP_DISABLE = 0x0,
+ UNP_GRPH_STACK_INTERLACE_FLIP_ENABLE = 0x1,
+} UNP_GRPH_STACK_INTERLACE_FLIP_EN;
+typedef enum UNP_GRPH_STACK_INTERLACE_FLIP_MODE {
+ UNP_GRPH_STACK_INTERLACE_FLIP_MODE_0 = 0x0,
+ UNP_GRPH_STACK_INTERLACE_FLIP_MODE_1 = 0x1,
+ UNP_GRPH_STACK_INTERLACE_FLIP_MODE_2 = 0x2,
+ UNP_GRPH_STACK_INTERLACE_FLIP_MODE_3 = 0x3,
+} UNP_GRPH_STACK_INTERLACE_FLIP_MODE;
+typedef enum UNP_GRPH_STEREOSYNC_SELECT_DISABLE {
+ UNP_GRPH_STEREOSYNC_SELECT_EN = 0x0,
+ UNP_GRPH_STEREOSYNC_SELECT_DIS = 0x1,
+} UNP_GRPH_STEREOSYNC_SELECT_DISABLE;
+typedef enum UNP_CRC_SOURCE_SEL {
+ UNP_CRC_SOURCE_SEL_NP_TO_LBV = 0x0,
+ UNP_CRC_SOURCE_SEL_LOWER32 = 0x1,
+ UNP_CRC_SOURCE_SEL_RESERVED = 0x2,
+ UNP_CRC_SOURCE_SEL_LOWER16 = 0x3,
+ UNP_CRC_SOURCE_SEL_UNP_TO_LBV = 0x4,
+} UNP_CRC_SOURCE_SEL;
+typedef enum UNP_CRC_LINE_SEL {
+ UNP_CRC_LINE_SEL_RESERVED = 0x0,
+ UNP_CRC_LINE_SEL_EVEN_ONLY = 0x1,
+ UNP_CRC_LINE_SEL_ODD_ONLY = 0x2,
+ UNP_CRC_LINE_SEL_ODD_EVEN = 0x3,
+} UNP_CRC_LINE_SEL;
+typedef enum UNP_ROTATION_ANGLE {
+ UNP_ROTATION_ANGLE_0 = 0x0,
+ UNP_ROTATION_ANGLE_90 = 0x1,
+ UNP_ROTATION_ANGLE_180 = 0x2,
+ UNP_ROTATION_ANGLE_270 = 0x3,
+ UNP_ROTATION_ANGLE_0m = 0x4,
+ UNP_ROTATION_ANGLE_90m = 0x5,
+ UNP_ROTATION_ANGLE_180m = 0x6,
+ UNP_ROTATION_ANGLE_270m = 0x7,
+} UNP_ROTATION_ANGLE;
+typedef enum UNP_PIXEL_DROP {
+ UNP_PIXEL_NO_DROP = 0x0,
+ UNP_PIXEL_DROPPING = 0x1,
+} UNP_PIXEL_DROP;
+typedef enum UNP_BUFFER_MODE {
+ UNP_BUFFER_MODE_LUMA = 0x0,
+ UNP_BUFFER_MODE_LUMA_CHROMA = 0x1,
+} UNP_BUFFER_MODE;
+typedef enum WATERMARK_MASK_CONTROL {
+ WM_MASK_CONTROL_SET_A = 0x0,
+ WM_MASK_CONTROL_SET_B = 0x1,
+ WM_MASK_CONTROL_SET_C = 0x2,
+ WM_MASK_CONTROL_SET_D = 0x3,
+ WM_MASK_CONTROL_RESERVED1 = 0x4,
+ WM_MASK_CONTROL_RESERVED2 = 0x5,
+ WM_MASK_CONTROL_RESERVED3 = 0x6,
+ WM_MASK_CONTROL_ACTIVE_SET = 0x7,
+} WATERMARK_MASK_CONTROL;
+typedef enum AZALIA_F2_CODEC_FUNCTION_CONTROL_RESET_CODEC_RESET {
+ AZALIA_F2_CODEC_FUNCTION_CONTROL_RESET_CODEC_NOT_RESET= 0x0,
+ AZALIA_F2_CODEC_FUNCTION_CONTROL_RESET_CODEC_DO_RESET= 0x1,
+} AZALIA_F2_CODEC_FUNCTION_CONTROL_RESET_CODEC_RESET;
+typedef enum CC_RCU_DC_AUDIO_PORT_CONNECTIVITY_PORT_CONNECTIVITY {
+ CC_RCU_DC_AUDIO_PORT_CONNECTIVITY_PORT_CONNECTIVITY_ALL= 0x0,
+ CC_RCU_DC_AUDIO_PORT_CONNECTIVITY_PORT_CONNECTIVITY_6= 0x1,
+ CC_RCU_DC_AUDIO_PORT_CONNECTIVITY_PORT_CONNECTIVITY_5= 0x2,
+ CC_RCU_DC_AUDIO_PORT_CONNECTIVITY_PORT_CONNECTIVITY_4= 0x3,
+ CC_RCU_DC_AUDIO_PORT_CONNECTIVITY_PORT_CONNECTIVITY_3= 0x4,
+ CC_RCU_DC_AUDIO_PORT_CONNECTIVITY_PORT_CONNECTIVITY_2= 0x5,
+ CC_RCU_DC_AUDIO_PORT_CONNECTIVITY_PORT_CONNECTIVITY_1= 0x6,
+ CC_RCU_DC_AUDIO_PORT_CONNECTIVITY_PORT_CONNECTIVITY_0= 0x7,
+} CC_RCU_DC_AUDIO_PORT_CONNECTIVITY_PORT_CONNECTIVITY;
+typedef enum CC_RCU_DC_AUDIO_INPUT_PORT_CONNECTIVITY_INPUT_PORT_CONNECTIVITY {
+ CC_RCU_DC_AUDIO_INPUT_PORT_CONNECTIVITY_INPUT_PORT_CONNECTIVITY_ALL= 0x0,
+ CC_RCU_DC_AUDIO_INPUT_PORT_CONNECTIVITY_INPUT_PORT_CONNECTIVITY_6= 0x1,
+ CC_RCU_DC_AUDIO_INPUT_PORT_CONNECTIVITY_INPUT_PORT_CONNECTIVITY_5= 0x2,
+ CC_RCU_DC_AUDIO_INPUT_PORT_CONNECTIVITY_INPUT_PORT_CONNECTIVITY_4= 0x3,
+ CC_RCU_DC_AUDIO_INPUT_PORT_CONNECTIVITY_INPUT_PORT_CONNECTIVITY_3= 0x4,
+ CC_RCU_DC_AUDIO_INPUT_PORT_CONNECTIVITY_INPUT_PORT_CONNECTIVITY_2= 0x5,
+ CC_RCU_DC_AUDIO_INPUT_PORT_CONNECTIVITY_INPUT_PORT_CONNECTIVITY_1= 0x6,
+ CC_RCU_DC_AUDIO_INPUT_PORT_CONNECTIVITY_INPUT_PORT_CONNECTIVITY_0= 0x7,
+} CC_RCU_DC_AUDIO_INPUT_PORT_CONNECTIVITY_INPUT_PORT_CONNECTIVITY;
+typedef enum GENERIC_AZ_CONTROLLER_REGISTER_ENABLE_CONTROL {
+ GENERIC_AZ_CONTROLLER_REGISTER_DISABLE = 0x0,
+ GENERIC_AZ_CONTROLLER_REGISTER_ENABLE = 0x1,
+} GENERIC_AZ_CONTROLLER_REGISTER_ENABLE_CONTROL;
+typedef enum GENERIC_AZ_CONTROLLER_REGISTER_ENABLE_CONTROL_RESERVED {
+ GENERIC_AZ_CONTROLLER_REGISTER_DISABLE_RESERVED = 0x0,
+ GENERIC_AZ_CONTROLLER_REGISTER_ENABLE_RESERVED = 0x1,
+} GENERIC_AZ_CONTROLLER_REGISTER_ENABLE_CONTROL_RESERVED;
+typedef enum GENERIC_AZ_CONTROLLER_REGISTER_STATUS {
+ GENERIC_AZ_CONTROLLER_REGISTER_STATUS_NOT_SET = 0x0,
+ GENERIC_AZ_CONTROLLER_REGISTER_STATUS_SET = 0x1,
+} GENERIC_AZ_CONTROLLER_REGISTER_STATUS;
+typedef enum GENERIC_AZ_CONTROLLER_REGISTER_STATUS_RESERVED {
+ GENERIC_AZ_CONTROLLER_REGISTER_STATUS_NOT_SET_RESERVED= 0x0,
+ GENERIC_AZ_CONTROLLER_REGISTER_STATUS_SET_RESERVED= 0x1,
+} GENERIC_AZ_CONTROLLER_REGISTER_STATUS_RESERVED;
+typedef enum AZ_GLOBAL_CAPABILITIES {
+ AZ_GLOBAL_CAPABILITIES_SIXTY_FOUR_BIT_ADDRESS_NOT_SUPPORTED= 0x0,
+ AZ_GLOBAL_CAPABILITIES_SIXTY_FOUR_BIT_ADDRESS_SUPPORTED= 0x1,
+} AZ_GLOBAL_CAPABILITIES;
+typedef enum GLOBAL_CONTROL_ACCEPT_UNSOLICITED_RESPONSE {
+ ACCEPT_UNSOLICITED_RESPONSE_NOT_ENABLE = 0x0,
+ ACCEPT_UNSOLICITED_RESPONSE_ENABLE = 0x1,
+} GLOBAL_CONTROL_ACCEPT_UNSOLICITED_RESPONSE;
+typedef enum GLOBAL_CONTROL_FLUSH_CONTROL {
+ FLUSH_CONTROL_FLUSH_NOT_STARTED = 0x0,
+ FLUSH_CONTROL_FLUSH_STARTED = 0x1,
+} GLOBAL_CONTROL_FLUSH_CONTROL;
+typedef enum GLOBAL_CONTROL_CONTROLLER_RESET {
+ CONTROLLER_RESET_AZ_CONTROLLER_IN_RESET = 0x0,
+ CONTROLLER_RESET_AZ_CONTROLLER_NOT_IN_RESET = 0x1,
+} GLOBAL_CONTROL_CONTROLLER_RESET;
+typedef enum AZ_STATE_CHANGE_STATUS {
+ AZ_STATE_CHANGE_STATUS_CODEC_NOT_PRESENT = 0x0,
+ AZ_STATE_CHANGE_STATUS_CODEC_PRESENT = 0x1,
+} AZ_STATE_CHANGE_STATUS;
+typedef enum GLOBAL_STATUS_FLUSH_STATUS {
+ GLOBAL_STATUS_FLUSH_STATUS_FLUSH_NOT_ENDED = 0x0,
+ GLOBAL_STATUS_FLUSH_STATUS_FLUSH_ENDED = 0x1,
+} GLOBAL_STATUS_FLUSH_STATUS;
+typedef enum STREAM_0_SYNCHRONIZATION {
+ STREAM_0_SYNCHRONIZATION_STEAM_NOT_STOPPED = 0x0,
+ STREAM_0_SYNCHRONIZATION_STEAM_STOPPED = 0x1,
+} STREAM_0_SYNCHRONIZATION;
+typedef enum STREAM_1_SYNCHRONIZATION {
+ STREAM_1_SYNCHRONIZATION_STEAM_NOT_STOPPED = 0x0,
+ STREAM_1_SYNCHRONIZATION_STEAM_STOPPED = 0x1,
+} STREAM_1_SYNCHRONIZATION;
+typedef enum STREAM_2_SYNCHRONIZATION {
+ STREAM_2_SYNCHRONIZATION_STEAM_NOT_STOPPED = 0x0,
+ STREAM_2_SYNCHRONIZATION_STEAM_STOPPED = 0x1,
+} STREAM_2_SYNCHRONIZATION;
+typedef enum STREAM_3_SYNCHRONIZATION {
+ STREAM_3_SYNCHRONIZATION_STEAM_NOT_STOPPED = 0x0,
+ STREAM_3_SYNCHRONIZATION_STEAM_STOPPED = 0x1,
+} STREAM_3_SYNCHRONIZATION;
+typedef enum STREAM_4_SYNCHRONIZATION {
+ STREAM_4_SYNCHRONIZATION_STEAM_NOT_STOPPED = 0x0,
+ STREAM_4_SYNCHRONIZATION_STEAM_STOPPED = 0x1,
+} STREAM_4_SYNCHRONIZATION;
+typedef enum STREAM_5_SYNCHRONIZATION {
+ STREAM_5_SYNCHRONIZATION_STEAM_NOT_STOPPED = 0x0,
+ STREAM_5_SYNCHRONIZATION_STEAM_STOPPED = 0x1,
+} STREAM_5_SYNCHRONIZATION;
+typedef enum STREAM_6_SYNCHRONIZATION {
+ STREAM_6_SYNCHRONIZATION_STEAM_NOT_STOPPED_RESERVED= 0x0,
+ STREAM_6_SYNCHRONIZATION_STEAM_STOPPED_RESERVED = 0x1,
+} STREAM_6_SYNCHRONIZATION;
+typedef enum STREAM_7_SYNCHRONIZATION {
+ STREAM_7_SYNCHRONIZATION_STEAM_NOT_STOPPED_RESERVED= 0x0,
+ STREAM_7_SYNCHRONIZATION_STEAM_STOPPED_RESERVED = 0x1,
+} STREAM_7_SYNCHRONIZATION;
+typedef enum STREAM_8_SYNCHRONIZATION {
+ STREAM_8_SYNCHRONIZATION_STEAM_NOT_STOPPED_RESERVED= 0x0,
+ STREAM_8_SYNCHRONIZATION_STEAM_STOPPED_RESERVED = 0x1,
+} STREAM_8_SYNCHRONIZATION;
+typedef enum STREAM_9_SYNCHRONIZATION {
+ STREAM_9_SYNCHRONIZATION_STEAM_NOT_STOPPED_RESERVED= 0x0,
+ STREAM_9_SYNCHRONIZATION_STEAM_STOPPED_RESERVED = 0x1,
+} STREAM_9_SYNCHRONIZATION;
+typedef enum STREAM_10_SYNCHRONIZATION {
+ STREAM_10_SYNCHRONIZATION_STEAM_NOT_STOPPED_RESERVED= 0x0,
+ STREAM_10_SYNCHRONIZATION_STEAM_STOPPED_RESERVED = 0x1,
+} STREAM_10_SYNCHRONIZATION;
+typedef enum STREAM_11_SYNCHRONIZATION {
+ STREAM_11_SYNCHRONIZATION_STEAM_NOT_STOPPED_RESERVED= 0x0,
+ STREAM_11_SYNCHRONIZATION_STEAM_STOPPED_RESERVED = 0x1,
+} STREAM_11_SYNCHRONIZATION;
+typedef enum STREAM_12_SYNCHRONIZATION {
+ STREAM_12_SYNCHRONIZATION_STEAM_NOT_STOPPED_RESERVED= 0x0,
+ STREAM_12_SYNCHRONIZATION_STEAM_STOPPED_RESERVED = 0x1,
+} STREAM_12_SYNCHRONIZATION;
+typedef enum STREAM_13_SYNCHRONIZATION {
+ STREAM_13_SYNCHRONIZATION_STEAM_NOT_STOPPED_RESERVED= 0x0,
+ STREAM_13_SYNCHRONIZATION_STEAM_STOPPED_RESERVED = 0x1,
+} STREAM_13_SYNCHRONIZATION;
+typedef enum STREAM_14_SYNCHRONIZATION {
+ STREAM_14_SYNCHRONIZATION_STEAM_NOT_STOPPED_RESERVED= 0x0,
+ STREAM_14_SYNCHRONIZATION_STEAM_STOPPED_RESERVED = 0x1,
+} STREAM_14_SYNCHRONIZATION;
+typedef enum STREAM_15_SYNCHRONIZATION {
+ STREAM_15_SYNCHRONIZATION_STEAM_NOT_STOPPED_RESERVED= 0x0,
+ STREAM_15_SYNCHRONIZATION_STEAM_STOPPED_RESERVED = 0x1,
+} STREAM_15_SYNCHRONIZATION;
+typedef enum CORB_READ_POINTER_RESET {
+ CORB_READ_POINTER_RESET_CORB_DMA_IS_NOT_RESET = 0x0,
+ CORB_READ_POINTER_RESET_CORB_DMA_IS_RESET = 0x1,
+} CORB_READ_POINTER_RESET;
+typedef enum AZ_CORB_SIZE {
+ AZ_CORB_SIZE_2ENTRIES_RESERVED = 0x0,
+ AZ_CORB_SIZE_16ENTRIES_RESERVED = 0x1,
+ AZ_CORB_SIZE_256ENTRIES = 0x2,
+ AZ_CORB_SIZE_RESERVED = 0x3,
+} AZ_CORB_SIZE;
+typedef enum AZ_RIRB_WRITE_POINTER_RESET {
+ AZ_RIRB_WRITE_POINTER_NOT_RESET = 0x0,
+ AZ_RIRB_WRITE_POINTER_DO_RESET = 0x1,
+} AZ_RIRB_WRITE_POINTER_RESET;
+typedef enum RIRB_CONTROL_RESPONSE_OVERRUN_INTERRUPT_CONTROL {
+ RIRB_CONTROL_RESPONSE_OVERRUN_INTERRUPT_CONTROL_INTERRUPT_DISABLED= 0x0,
+ RIRB_CONTROL_RESPONSE_OVERRUN_INTERRUPT_CONTROL_INTERRUPT_ENABLED= 0x1,
+} RIRB_CONTROL_RESPONSE_OVERRUN_INTERRUPT_CONTROL;
+typedef enum RIRB_CONTROL_RESPONSE_INTERRUPT_CONTROL {
+ RIRB_CONTROL_RESPONSE_INTERRUPT_CONTROL_INTERRUPT_DISABLED= 0x0,
+ RIRB_CONTROL_RESPONSE_INTERRUPT_CONTROL_INTERRUPT_ENABLED= 0x1,
+} RIRB_CONTROL_RESPONSE_INTERRUPT_CONTROL;
+typedef enum AZ_RIRB_SIZE {
+ AZ_RIRB_SIZE_2ENTRIES_RESERVED = 0x0,
+ AZ_RIRB_SIZE_16ENTRIES_RESERVED = 0x1,
+ AZ_RIRB_SIZE_256ENTRIES = 0x2,
+ AZ_RIRB_SIZE_UNDEFINED = 0x3,
+} AZ_RIRB_SIZE;
+typedef enum IMMEDIATE_COMMAND_STATUS_IMMEDIATE_RESULT_VALID {
+ IMMEDIATE_COMMAND_STATUS_IMMEDIATE_RESULT_VALID_NO_IMMEDIATE_RESPONSE_VALID= 0x0,
+ IMMEDIATE_COMMAND_STATUS_IMMEDIATE_RESULT_VALID_IMMEDIATE_RESPONSE_VALID= 0x1,
+} IMMEDIATE_COMMAND_STATUS_IMMEDIATE_RESULT_VALID;
+typedef enum IMMEDIATE_COMMAND_STATUS_IMMEDIATE_COMMAND_BUSY {
+ IMMEDIATE_COMMAND_STATUS_IMMEDIATE_COMMAND_NOT_BUSY= 0x0,
+ IMMEDIATE_COMMAND_STATUS_IMMEDIATE_COMMAND_IS_BUSY= 0x1,
+} IMMEDIATE_COMMAND_STATUS_IMMEDIATE_COMMAND_BUSY;
+typedef enum DMA_POSITION_LOWER_BASE_ADDRESS_BUFFER_ENABLE {
+ DMA_POSITION_LOWER_BASE_ADDRESS_BUFFER_ENABLE_DMA_DISABLE= 0x0,
+ DMA_POSITION_LOWER_BASE_ADDRESS_BUFFER_ENABLE_DMA_ENABLE= 0x1,
+} DMA_POSITION_LOWER_BASE_ADDRESS_BUFFER_ENABLE;
+typedef enum OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_DESCRIPTOR_ERROR {
+ OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_DESCRIPTOR_ERROR_STATUS_NOT_SET= 0x0,
+ OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_DESCRIPTOR_ERROR_STATUS_SET= 0x1,
+} OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_DESCRIPTOR_ERROR;
+typedef enum OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_FIFO_ERROR {
+ OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_FIFO_ERROR_STATUS_NOT_SET= 0x0,
+ OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_FIFO_ERROR_STATUS_SET= 0x1,
+} OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_FIFO_ERROR;
+typedef enum OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_BUFFER_COMPLETION_INTERRUPT_STATUS {
+ OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_BUFFER_COMPLETION_INTERRUPT_STATUS_NOT_SET= 0x0,
+ OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_BUFFER_COMPLETION_INTERRUPT_STATUS_SET= 0x1,
+} OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_BUFFER_COMPLETION_INTERRUPT_STATUS;
+typedef enum OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_TRAFFIC_PRIORITY {
+ OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_NO_TRAFFIC_PRIORITY= 0x0,
+ OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_YES_TRAFFIC_PRIORITY= 0x1,
+} OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_TRAFFIC_PRIORITY;
+typedef enum OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_DESCRIPTOR_ERROR_INTERRUPT_ENABLE {
+ OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_DESCRIPTOR_ERROR_INTERRUPT_DISABLED= 0x0,
+ OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_DESCRIPTOR_ERROR_INTERRUPT_ENABLED= 0x1,
+} OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_DESCRIPTOR_ERROR_INTERRUPT_ENABLE;
+typedef enum OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_FIFO_ERROR_INTERRUPT_ENABLE {
+ OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_FIFO_ERROR_INTERRUPT_DISABLED= 0x0,
+ OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_FIFO_ERROR_INTERRUPT_ENABLED= 0x1,
+} OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_FIFO_ERROR_INTERRUPT_ENABLE;
+typedef enum OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_INTERRUPT_ON_COMPLETION_ENABLE {
+ OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_INTERRUPT_ON_COMPLETION_ENABLE_INTERRUPT_DISABLED= 0x0,
+ OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_INTERRUPT_ON_COMPLETION_ENABLE_INTERRUPT_ENABLED= 0x1,
+} OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_INTERRUPT_ON_COMPLETION_ENABLE;
+typedef enum OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_STREAM_RUN {
+ OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_STREAM_NOT_RUN= 0x0,
+ OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_STREAM_DO_RUN= 0x1,
+} OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_STREAM_RUN;
+typedef enum OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_STREAM_RESET {
+ OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_STREAM_NOT_RESET= 0x0,
+ OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_STREAM_IS_RESET= 0x1,
+} OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS_STREAM_RESET;
+typedef enum OUTPUT_STREAM_DESCRIPTOR_FORMAT_SAMPLE_BASE_RATE {
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_SAMPLE_BASE_RATE_48KHZ= 0x0,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_SAMPLE_BASE_RATE_44P1KHZ= 0x1,
+} OUTPUT_STREAM_DESCRIPTOR_FORMAT_SAMPLE_BASE_RATE;
+typedef enum OUTPUT_STREAM_DESCRIPTOR_FORMAT_SAMPLE_BASE_MULTIPLE {
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_SAMPLE_BASE_MULTIPLE_BY1= 0x0,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_SAMPLE_BASE_MULTIPLE_BY2= 0x1,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_SAMPLE_BASE_MULTIPLE_BY3_RESERVED= 0x2,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_SAMPLE_BASE_MULTIPLE_BY4= 0x3,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_SAMPLE_BASE_MULTIPLE_RESERVED= 0x4,
+} OUTPUT_STREAM_DESCRIPTOR_FORMAT_SAMPLE_BASE_MULTIPLE;
+typedef enum OUTPUT_STREAM_DESCRIPTOR_FORMAT_SAMPLE_BASE_DIVISOR {
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_SAMPLE_BASE_DIVISOR_BY1= 0x0,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_SAMPLE_BASE_DIVISOR_BY2_RESERVED= 0x1,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_SAMPLE_BASE_DIVISOR_BY3= 0x2,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_SAMPLE_BASE_DIVISOR_BY4_RESERVED= 0x3,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_SAMPLE_BASE_DIVISOR_BY5_RESERVED= 0x4,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_SAMPLE_BASE_DIVISOR_BY6_RESERVED= 0x5,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_SAMPLE_BASE_DIVISOR_BY7_RESERVED= 0x6,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_SAMPLE_BASE_DIVISOR_BY8_RESERVED= 0x7,
+} OUTPUT_STREAM_DESCRIPTOR_FORMAT_SAMPLE_BASE_DIVISOR;
+typedef enum OUTPUT_STREAM_DESCRIPTOR_FORMAT_BITS_PER_SAMPLE {
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_BITS_PER_SAMPLE_8_RESERVED= 0x0,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_BITS_PER_SAMPLE_16= 0x1,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_BITS_PER_SAMPLE_20= 0x2,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_BITS_PER_SAMPLE_24= 0x3,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_BITS_PER_SAMPLE_32_RESERVED= 0x4,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_BITS_PER_SAMPLE_RESERVED= 0x5,
+} OUTPUT_STREAM_DESCRIPTOR_FORMAT_BITS_PER_SAMPLE;
+typedef enum OUTPUT_STREAM_DESCRIPTOR_FORMAT_NUMBER_OF_CHANNELS {
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_NUMBER_OF_CHANNELS_1= 0x0,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_NUMBER_OF_CHANNELS_2= 0x1,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_NUMBER_OF_CHANNELS_3= 0x2,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_NUMBER_OF_CHANNELS_4= 0x3,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_NUMBER_OF_CHANNELS_5= 0x4,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_NUMBER_OF_CHANNELS_6= 0x5,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_NUMBER_OF_CHANNELS_7= 0x6,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_NUMBER_OF_CHANNELS_8= 0x7,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_NUMBER_OF_CHANNELS_9_RESERVED= 0x8,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_NUMBER_OF_CHANNELS_10_RESERVED= 0x9,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_NUMBER_OF_CHANNELS_11_RESERVED= 0xa,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_NUMBER_OF_CHANNELS_12_RESERVED= 0xb,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_NUMBER_OF_CHANNELS_13_RESERVED= 0xc,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_NUMBER_OF_CHANNELS_14_RESERVED= 0xd,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_NUMBER_OF_CHANNELS_15_RESERVED= 0xe,
+ OUTPUT_STREAM_DESCRIPTOR_FORMAT_NUMBER_OF_CHANNELS_16_RESERVED= 0xf,
+} OUTPUT_STREAM_DESCRIPTOR_FORMAT_NUMBER_OF_CHANNELS;
+typedef enum AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_STREAM_TYPE {
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_STREAM_TYPE_PCM= 0x0,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_STREAM_TYPE_NOT_PCM= 0x1,
+} AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_STREAM_TYPE;
+typedef enum AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_RATE {
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_RATE_48KHZ= 0x0,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_RATE_44P1KHZ= 0x1,
+} AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_RATE;
+typedef enum AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_MULTIPLE {
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_MULTIPLE_BY1= 0x0,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_MULTIPLE_BY2= 0x1,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_MULTIPLE_BY3_RESERVED= 0x2,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_MULTIPLE_BY4= 0x3,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_MULTIPLE_RESERVED= 0x4,
+} AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_MULTIPLE;
+typedef enum AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_DIVISOR {
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_DIVISOR_BY1= 0x0,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_DIVISOR_BY2_RESERVED= 0x1,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_DIVISOR_BY3= 0x2,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_DIVISOR_BY4_RESERVED= 0x3,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_DIVISOR_BY5_RESERVED= 0x4,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_DIVISOR_BY6_RESERVED= 0x5,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_DIVISOR_BY7_RESERVED= 0x6,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_DIVISOR_BY8_RESERVED= 0x7,
+} AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_DIVISOR;
+typedef enum AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_BITS_PER_SAMPLE {
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_BITS_PER_SAMPLE_8_RESERVED= 0x0,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_BITS_PER_SAMPLE_16= 0x1,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_BITS_PER_SAMPLE_20= 0x2,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_BITS_PER_SAMPLE_24= 0x3,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_BITS_PER_SAMPLE_32_RESERVED= 0x4,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_BITS_PER_SAMPLE_RESERVED= 0x5,
+} AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_BITS_PER_SAMPLE;
+typedef enum AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_NUMBER_OF_CHANNELS {
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_NUMBER_OF_CHANNELS_1= 0x0,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_NUMBER_OF_CHANNELS_2= 0x1,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_NUMBER_OF_CHANNELS_3= 0x2,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_NUMBER_OF_CHANNELS_4= 0x3,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_NUMBER_OF_CHANNELS_5= 0x4,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_NUMBER_OF_CHANNELS_6= 0x5,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_NUMBER_OF_CHANNELS_7= 0x6,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_NUMBER_OF_CHANNELS_8= 0x7,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_NUMBER_OF_CHANNELS_RESERVED= 0x8,
+} AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT_NUMBER_OF_CHANNELS;
+typedef enum AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_L {
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_L_BIT7_NOT_SET= 0x0,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_L_BIT7_IS_SET= 0x1,
+} AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_L;
+typedef enum AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_PRO {
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_PRO_BIT_A_NOT_SET= 0x0,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_PRO_BIT_A_IS_SET= 0x1,
+} AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_PRO;
+typedef enum AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_NON_AUDIO {
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_NON_AUDIO_BIT_B_NOT_SET= 0x0,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_NON_AUDIO_BIT_B_IS_SET= 0x1,
+} AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_NON_AUDIO;
+typedef enum AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_COPY {
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_COPY_BIT_C_IS_SET= 0x0,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_COPY_BIT_C_NOT_SET= 0x1,
+} AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_COPY;
+typedef enum AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_PRE {
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_PRE_LSB_OF_D_NOT_SET= 0x0,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_PRE_LSB_OF_D_IS_SET= 0x1,
+} AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_PRE;
+typedef enum AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_VCFG {
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_VALIDITY_CFG_NOT_ON= 0x0,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_VALIDITY_CFG_ON= 0x1,
+} AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_VCFG;
+typedef enum AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_V {
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_V_BIT28_IS_ZERO= 0x0,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_V_BIT28_IS_ONE= 0x1,
+} AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_V;
+typedef enum AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_DIGEN {
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_DIGEN_DIGITAL_TRANSMISSION_DISABLED= 0x0,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_DIGEN_DIGITAL_TRANSMISSION_ENABLED= 0x1,
+} AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_DIGEN;
+typedef enum AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_3_KEEPALIVE {
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_3_KEEPALIVE_SILENT_STREAM_NOT_ENABLE= 0x0,
+ AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_3_KEEPALIVE_SILENT_STREAM_ENABLE= 0x1,
+} AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_3_KEEPALIVE;
+typedef enum AZALIA_F2_CODEC_PIN_CONTROL_WIDGET_CONTROL_OUT_ENABLE {
+ AZALIA_F2_CODEC_PIN_CONTROL_WIDGET_CONTROL_OUT_ENABLE_PIN_SHUT_OFF= 0x0,
+ AZALIA_F2_CODEC_PIN_CONTROL_WIDGET_CONTROL_OUT_ENABLE_PIN_DRIVEN= 0x1,
+} AZALIA_F2_CODEC_PIN_CONTROL_WIDGET_CONTROL_OUT_ENABLE;
+typedef enum AZALIA_F2_CODEC_PIN_CONTROL_UNSOLICITED_RESPONSE_ENABLE {
+ AZALIA_F2_CODEC_PIN_CONTROL_UNSOLICITED_RESPONSE_DISABLED= 0x0,
+ AZALIA_F2_CODEC_PIN_CONTROL_UNSOLICITED_RESPONSE_ENABLED= 0x1,
+} AZALIA_F2_CODEC_PIN_CONTROL_UNSOLICITED_RESPONSE_ENABLE;
+typedef enum AZALIA_F2_CODEC_PIN_CONTROL_DOWN_MIX_INFO_DOWN_MIX_INHIBIT {
+ AZALIA_F2_CODEC_PIN_CONTROL_DOWN_MIX_NO_INFO_OR_PERMITTED= 0x0,
+ AZALIA_F2_CODEC_PIN_CONTROL_DOWN_MIX_FORBIDDEN = 0x1,
+} AZALIA_F2_CODEC_PIN_CONTROL_DOWN_MIX_INFO_DOWN_MIX_INHIBIT;
+typedef enum AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL01_ENABLE_MULTICHANNEL01_MUTE {
+ AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL01_ENABLE_MULTICHANNEL01_NOT_MUTED= 0x0,
+ AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL01_ENABLE_MULTICHANNEL01_MUTED= 0x1,
+} AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL01_ENABLE_MULTICHANNEL01_MUTE;
+typedef enum AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL23_ENABLE_MULTICHANNEL23_MUTE {
+ AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL23_ENABLE_MULTICHANNEL23_NOT_MUTED= 0x0,
+ AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL23_ENABLE_MULTICHANNEL23_MUTED= 0x1,
+} AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL23_ENABLE_MULTICHANNEL23_MUTE;
+typedef enum AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL45_ENABLE_MULTICHANNEL45_MUTE {
+ AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL45_ENABLE_MULTICHANNEL45_NOT_MUTED= 0x0,
+ AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL45_ENABLE_MULTICHANNEL45_MUTED= 0x1,
+} AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL45_ENABLE_MULTICHANNEL45_MUTE;
+typedef enum AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL67_ENABLE_MULTICHANNEL67_MUTE {
+ AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL67_ENABLE_MULTICHANNEL67_NOT_MUTED= 0x0,
+ AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL67_ENABLE_MULTICHANNEL67_MUTED= 0x1,
+} AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL67_ENABLE_MULTICHANNEL67_MUTE;
+typedef enum AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL1_ENABLE_MULTICHANNEL1_MUTE {
+ AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL1_ENABLE_MULTICHANNEL1_NOT_MUTED= 0x0,
+ AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL1_ENABLE_MULTICHANNEL1_MUTED= 0x1,
+} AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL1_ENABLE_MULTICHANNEL1_MUTE;
+typedef enum AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL3_ENABLE_MULTICHANNEL3_MUTE {
+ AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL3_ENABLE_MULTICHANNEL3_NOT_MUTED= 0x0,
+ AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL3_ENABLE_MULTICHANNEL3_MUTED= 0x1,
+} AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL3_ENABLE_MULTICHANNEL3_MUTE;
+typedef enum AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL5_ENABLE_MULTICHANNEL5_MUTE {
+ AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL5_ENABLE_MULTICHANNEL5_NOT_MUTED= 0x0,
+ AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL5_ENABLE_MULTICHANNEL5_MUTED= 0x1,
+} AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL5_ENABLE_MULTICHANNEL5_MUTE;
+typedef enum AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL7_ENABLE_MULTICHANNEL7_MUTE {
+ AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL7_ENABLE_MULTICHANNEL7_NOT_MUTED= 0x0,
+ AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL7_ENABLE_MULTICHANNEL7_MUTED= 0x1,
+} AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL7_ENABLE_MULTICHANNEL7_MUTE;
+typedef enum AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL_MODE_MULTICHANNEL_MODE {
+ AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL_MODE_MULTICHANNEL_PAIR_MODE= 0x0,
+ AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL_MODE_MULTICHANNEL_SINGLE_MODE= 0x1,
+} AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL_MODE_MULTICHANNEL_MODE;
+typedef enum AZ_LATENCY_COUNTER_CONTROL {
+ AZ_LATENCY_COUNTER_NO_RESET = 0x0,
+ AZ_LATENCY_COUNTER_RESET_DONE = 0x1,
+} AZ_LATENCY_COUNTER_CONTROL;
+typedef enum AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE {
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_OUTPUT_CONVERTER_RESERVED= 0x0,
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_INPUT_CONVERTER_RESERVED= 0x1,
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_MIXER_RESERVED= 0x2,
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_SELECTOR_RESERVED= 0x3,
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_PIN_RESERVED= 0x4,
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_POWER_WIDGET_RESERVED= 0x5,
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_VOLUME_KNOB_RESERVED= 0x6,
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_BEEP_GENERATOR_RESERVED= 0x7,
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_RESERVED_RESERVED= 0x8,
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_VENDOR_DEFINED_RESERVED= 0x9,
+} AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE;
+typedef enum AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_LR_SWAP {
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_LR_SWAP_CAPABILITY= 0x0,
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_LR_SWAP_CAPABILITY= 0x1,
+} AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_LR_SWAP;
+typedef enum AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_POWER_CONTROL {
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_POWER_CONTROL_CAPABILITY= 0x0,
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_POWER_CONTROL_CAPABILITY= 0x1,
+} AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_POWER_CONTROL;
+typedef enum AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_DIGITAL {
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_IS_ANALOG= 0x0,
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_IS_DIGITAL= 0x1,
+} AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_DIGITAL;
+typedef enum AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_CONNECTION_LIST {
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_CONNECTION_LIST= 0x0,
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_CONNECTION_LIST= 0x1,
+} AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_CONNECTION_LIST;
+typedef enum AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_UNSOLICITED_RESPONSE_CAPABILITY {
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_UNSOLICITED_RESPONSE_CAPABILITY= 0x0,
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_UNSOLICITED_RESPONSE_CAPABILITY= 0x1,
+} AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_UNSOLICITED_RESPONSE_CAPABILITY;
+typedef enum AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_PROCESSING_WIDGET {
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_PROCESSING_WIDGET_NO_PROCESSING_CAPABILITIES= 0x0,
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_PROCESSING_WIDGET_HAVE_PROCESSING_CAPABILITIES= 0x1,
+} AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_PROCESSING_WIDGET;
+typedef enum AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_STRIPE {
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_SUPPORT_STRIPING= 0x0,
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_SUPPORT_STRIPING= 0x1,
+} AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_STRIPE;
+typedef enum AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_FORMAT_OVERRIDE {
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_FORMAT_OVERRIDE= 0x0,
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_SUPPORT_FORMAT_OVERRIDE= 0x1,
+} AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_FORMAT_OVERRIDE;
+typedef enum AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_AMPLIFIER_PARAMETER_OVERRIDE {
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_AMPLIFIER_PARAMETER= 0x0,
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_AMPLIFIER_PARAMETER_OVERRIDE= 0x1,
+} AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_AMPLIFIER_PARAMETER_OVERRIDE;
+typedef enum AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_OUTPUT_AMPLIFIER_PRESENT {
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_OUTPUT_AMPLIFIER= 0x0,
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_OUTPUT_AMPLIFIER= 0x1,
+} AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_OUTPUT_AMPLIFIER_PRESENT;
+typedef enum AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_INPUT_AMPLIFIER_PRESENT {
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_INPUT_AMPLIFIER= 0x0,
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_INPUT_AMPLIFIER= 0x1,
+} AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_INPUT_AMPLIFIER_PRESENT;
+typedef enum AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_AUDIO_CHANNEL_CAPABILITIES {
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_AUDIO_CHANNEL_CAPABILITIES_MONOPHONIC= 0x0,
+ AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_AUDIO_CHANNEL_CAPABILITIES_STEREO= 0x1,
+} AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_AUDIO_CHANNEL_CAPABILITIES;
+typedef enum AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE {
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_OUTPUT_CONVERTER_RESERVED= 0x0,
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_INPUT_CONVERTER_RESERVED= 0x1,
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_MIXER_RESERVED= 0x2,
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_SELECTOR_RESERVED= 0x3,
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_PIN_RESERVED= 0x4,
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_POWER_WIDGET_RESERVED= 0x5,
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_VOLUME_KNOB_RESERVED= 0x6,
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_BEEP_GENERATOR_RESERVED= 0x7,
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_RESERVED_RESERVED= 0x8,
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_VENDOR_DEFINED_RESERVED= 0x9,
+} AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE;
+typedef enum AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_LR_SWAP {
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_LR_SWAP_CAPABILITY= 0x0,
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_LR_SWAP_CAPABILITY= 0x1,
+} AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_LR_SWAP;
+typedef enum AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_POWER_CONTROL {
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_POWER_CONTROL_CAPABILITY= 0x0,
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_POWER_CONTROL_CAPABILITY= 0x1,
+} AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_POWER_CONTROL;
+typedef enum AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_DIGITAL {
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_IS_ANALOG= 0x0,
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_IS_DIGITAL= 0x1,
+} AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_DIGITAL;
+typedef enum AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_CONNECTION_LIST {
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_CONNECTION_LIST= 0x0,
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_CONNECTION_LIST= 0x1,
+} AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_CONNECTION_LIST;
+typedef enum AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_UNSOLICITED_RESPONSE_CAPABILITY {
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_UNSOLICITED_RESPONSE_CAPABILITY= 0x0,
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_UNSOLICITED_RESPONSE_CAPABILITY= 0x1,
+} AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_UNSOLICITED_RESPONSE_CAPABILITY;
+typedef enum AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_PROCESSING_WIDGET {
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_PROCESSING_WIDGET_NO_PROCESSING_CAPABILITIES= 0x0,
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_PROCESSING_WIDGET_HAVE_PROCESSING_CAPABILITIES= 0x1,
+} AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_PROCESSING_WIDGET;
+typedef enum AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_STRIPE {
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_SUPPORT_STRIPING= 0x0,
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_SUPPORT_STRIPING= 0x1,
+} AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_STRIPE;
+typedef enum AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_AMPLIFIER_PARAMETER_OVERRIDE {
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_AMPLIFIER_PARAMETER= 0x0,
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_AMPLIFIER_PARAMETER_OVERRIDE= 0x1,
+} AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_AMPLIFIER_PARAMETER_OVERRIDE;
+typedef enum AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_OUTPUT_AMPLIFIER_PRESENT {
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_OUTPUT_AMPLIFIER= 0x0,
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_OUTPUT_AMPLIFIER= 0x1,
+} AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_OUTPUT_AMPLIFIER_PRESENT;
+typedef enum AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_INPUT_AMPLIFIER_PRESENT {
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_INPUT_AMPLIFIER_PRESENT= 0x0,
+ AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_INPUT_AMPLIFIER= 0x1,
+} AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_INPUT_AMPLIFIER_PRESENT;
+typedef enum AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_EAPD_CAPABLE {
+ AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_NO_EAPD_PIN= 0x0,
+ AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_HAVE_EAPD_PIN= 0x1,
+} AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_EAPD_CAPABLE;
+typedef enum AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_BALANCED_I_O_PINS {
+ AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_I_O_PINS_ARE_NOT_BALANCED= 0x0,
+ AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_I_O_PINS_ARE_BALANCED= 0x1,
+} AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_BALANCED_I_O_PINS;
+typedef enum AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_INPUT_CAPABLE {
+ AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_NO_INPUT_PIN= 0x0,
+ AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_HAVE_INPUT_PIN= 0x1,
+} AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_INPUT_CAPABLE;
+typedef enum AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_OUTPUT_CAPABLE {
+ AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_NO_OUTPUT_PIN= 0x0,
+ AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_HAVE_OUTPUT_PIN= 0x1,
+} AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_OUTPUT_CAPABLE;
+typedef enum AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_HEADPHONE_DRIVE_CAPABLE {
+ AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_NO_HEADPHONE_DRIVE_CAPABILITY= 0x0,
+ AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_HAVE_HEADPHONE_DRIVE_CAPABILITY= 0x1,
+} AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_HEADPHONE_DRIVE_CAPABLE;
+typedef enum AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_JACK_DETECTION_CAPABILITY {
+ AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_NO_JACK_DETECTION_CAPABILITY= 0x0,
+ AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_HAVE_JACK_DETECTION_CAPABILITY= 0x1,
+} AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_JACK_DETECTION_CAPABILITY;
+typedef enum AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_TRIGGER_REQUIRED {
+ AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_NO_TRIGGER_REQUIRED_FOR_IMPEDANCE_MEASUREMENT= 0x0,
+ AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_TRIGGER_REQUIRED_FOR_IMPEDANCE_MEASUREMENT= 0x1,
+} AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_TRIGGER_REQUIRED;
+typedef enum AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_IMPEDANCE_SENSE_CAPABLE {
+ AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_NO_IMPEDANCE_SENSE_CAPABILITY= 0x0,
+ AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_HAVE_IMPEDANCE_SENSE_CAPABILITY= 0x1,
+} AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES_IMPEDANCE_SENSE_CAPABLE;
+typedef enum AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_MODE_MULTICHANNEL_MODE {
+ AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_MODE_MULTICHANNEL_PAIR_MODE= 0x0,
+ AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_MODE_MULTICHANNEL_SINGLE_MODE= 0x1,
+} AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_MODE_MULTICHANNEL_MODE;
+typedef enum AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_HBR_HBR_CAPABLE {
+ AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_HBR_NO_HBR_CAPABLILITY= 0x0,
+ AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_HBR_HAVE_HBR_CAPABLILITY= 0x1,
+} AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_HBR_HBR_CAPABLE;
+typedef enum AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE {
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_OUTPUT_CONVERTER_RESERVED= 0x0,
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_INPUT_CONVERTER_RESERVED= 0x1,
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_MIXER_RESERVED= 0x2,
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_SELECTOR_RESERVED= 0x3,
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_PIN_RESERVED= 0x4,
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_POWER_WIDGET_RESERVED= 0x5,
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_VOLUME_KNOB_RESERVED= 0x6,
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_BEEP_GENERATOR_RESERVED= 0x7,
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_RESERVED= 0x8,
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_VENDOR_DEFINED_RESERVED= 0x9,
+} AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE;
+typedef enum AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_LR_SWAP {
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_LR_SWAP_CAPABILITY= 0x0,
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_LR_SWAP_CAPABILITY= 0x1,
+} AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_LR_SWAP;
+typedef enum AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_POWER_CONTROL {
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_POWER_CONTROL_CAPABILITY= 0x0,
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_POWER_CONTROL_CAPABILITY= 0x1,
+} AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_POWER_CONTROL;
+typedef enum AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_DIGITAL {
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_CODEC_CONVERTER0_IS_ANALOG= 0x0,
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_CODEC_CONVERTER0_IS_DIGITAL= 0x1,
+} AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_DIGITAL;
+typedef enum AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_CONNECTION_LIST {
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_CONNECTION_LIST= 0x0,
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_CONNECTION_LIST= 0x1,
+} AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_CONNECTION_LIST;
+typedef enum AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_UNSOLICITED_RESPONSE_CAPABILITY {
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_UNSOLICITED_RESPONSE_CAPABILITY= 0x0,
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_UNSOLICITED_RESPONSE_CAPABILITY= 0x1,
+} AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_UNSOLICITED_RESPONSE_CAPABILITY;
+typedef enum AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_PROCESSING_WIDGET {
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_PROCESSING_WIDGET_CODEC_CONVERTER0_HAVE_NO_PROCESSING_CAPABILITIES= 0x0,
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_PROCESSING_WIDGET_CODEC_CONVERTER0_HAVE_PROCESSING_CAPABILITIES= 0x1,
+} AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_PROCESSING_WIDGET;
+typedef enum AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_STRIPE {
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NOT_SUPPORT_STRIPING= 0x0,
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_SUPPORT_STRIPING= 0x1,
+} AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_STRIPE;
+typedef enum AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_FORMAT_OVERRIDE {
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_FORMAT_OVERRIDE= 0x0,
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_FORMAT_OVERRIDE= 0x1,
+} AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_FORMAT_OVERRIDE;
+typedef enum AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_AMPLIFIER_PARAMETER_OVERRIDE {
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_AMPLIFIER_PARAMETER= 0x0,
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_AMPLIFIER_PARAMETER= 0x1,
+} AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_AMPLIFIER_PARAMETER_OVERRIDE;
+typedef enum AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_OUTPUT_AMPLIFIER_PRESENT {
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_OUTPUT_AMPLIFIER= 0x0,
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_OUTPUT_AMPLIFIER= 0x1,
+} AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_OUTPUT_AMPLIFIER_PRESENT;
+typedef enum AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_INPUT_AMPLIFIER_PRESENT {
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_INPUT_AMPLIFIER= 0x0,
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_INPUT_AMPLIFIER= 0x1,
+} AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_INPUT_AMPLIFIER_PRESENT;
+typedef enum AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_AUDIO_CHANNEL_CAPABILITIES {
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_AUDIO_CHANNEL_CAPABILITIES_MONOPHONIC= 0x0,
+ AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_AUDIO_CHANNEL_CAPABILITIES_STEREO= 0x1,
+} AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES_AUDIO_CHANNEL_CAPABILITIES;
+typedef enum AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE {
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_OUTPUT_CONVERTER_RESERVED= 0x0,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_INPUT_CONVERTER_RESERVED= 0x1,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_MIXER_RESERVED= 0x2,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_SELECTOR_RESERVED= 0x3,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_PIN_RESERVED= 0x4,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_POWER_WIDGET_RESERVED= 0x5,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_VOLUME_KNOB_RESERVED= 0x6,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_BEEP_GENERATOR_RESERVED= 0x7,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_RESERVED= 0x8,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE_VENDOR_DEFINED_RESERVED= 0x9,
+} AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_TYPE;
+typedef enum AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_LR_SWAP {
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_LR_SWAP= 0x0,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_LR_SWAP= 0x1,
+} AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_LR_SWAP;
+typedef enum AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_POWER_CONTROL {
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_POWER_CONTROL_CAPABILITY= 0x0,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_POWER_CONTROL_CAPABILITY= 0x1,
+} AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_POWER_CONTROL;
+typedef enum AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_DIGITAL {
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_IS_ANALOG= 0x0,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_IS_DIGITAL= 0x1,
+} AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_DIGITAL;
+typedef enum AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_CONNECTION_LIST {
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_CONNECTION_LIST= 0x0,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_CONNECTION_LIST= 0x1,
+} AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_CONNECTION_LIST;
+typedef enum AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_UNSOLICITED_RESPONSE_CAPABILITY {
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_UNSOLICITED_RESPONSE_CAPABILITY= 0x0,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_UNSOLICITED_RESPONSE_CAPABILITY= 0x1,
+} AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_UNSOLICITED_RESPONSE_CAPABILITY;
+typedef enum AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_PROCESSING_WIDGET {
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_PROCESSING_WIDGET_NO_PROCESING_CAPABILITIES= 0x0,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_PROCESSING_WIDGET_HAVE_PROCESING_CAPABILITIES= 0x1,
+} AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_PROCESSING_WIDGET;
+typedef enum AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_STRIPE {
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_SUPPORT_STRIPING= 0x0,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_SUPPORT_STRIPING= 0x1,
+} AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_STRIPE;
+typedef enum AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_AMPLIFIER_PARAMETER_OVERRIDE {
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_AMPLIFIER_PARAMETER= 0x0,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_AMPLIFIER_PARAMETER_OVERRIDE= 0x1,
+} AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_AMPLIFIER_PARAMETER_OVERRIDE;
+typedef enum AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_OUTPUT_AMPLIFIER_PRESENT {
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_OUTPUT_AMPLIFIER= 0x0,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_OUTPUT_AMPLIFIER= 0x1,
+} AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_OUTPUT_AMPLIFIER_PRESENT;
+typedef enum AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_INPUT_AMPLIFIER_PRESENT {
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_NO_INPUT_AMPLIFIER= 0x0,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_HAVE_INPUT_AMPLIFIER= 0x1,
+} AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES_INPUT_AMPLIFIER_PRESENT;
+typedef enum AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_DP {
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_DP_NOT_ENABLED= 0x0,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_DP_ENABLED= 0x1,
+} AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_DP;
+typedef enum AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_EAPD_CAPABLE {
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_EAPD_CAPABLE_NO_EAPD_PIN= 0x0,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_EAPD_CAPABLE_HAVE_EAPD_PIN= 0x1,
+} AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_EAPD_CAPABLE;
+typedef enum AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_HDMI {
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_HDMI_NOT_ENABLED= 0x0,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_HDMI_ENABLED= 0x1,
+} AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_HDMI;
+typedef enum AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_BALANCED_I_O_PINS {
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_I_O_PINS_NOT_BALANCED= 0x0,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_I_O_PINS_ARE_BALANCED= 0x1,
+} AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_BALANCED_I_O_PINS;
+typedef enum AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_INPUT_CAPABLE {
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_NO_INPUT_PIN= 0x0,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_HAVE_INPUT_PIN= 0x1,
+} AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_INPUT_CAPABLE;
+typedef enum AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_OUTPUT_CAPABLE {
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_NO_OUTPUT_PIN= 0x0,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_HAVE_OUTPUT_PIN= 0x1,
+} AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_OUTPUT_CAPABLE;
+typedef enum AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_HEADPHONE_DRIVE_CAPABLE {
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_NO_HEADPHONE_DRIVE_CAPABILITY= 0x0,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_HAVE_HEADPHONE_DRIVE_CAPABILITY= 0x1,
+} AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_HEADPHONE_DRIVE_CAPABLE;
+typedef enum AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_JACK_DETECTION_CAPABILITY {
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_NO_JACK_PRESENCE_DETECTION_CAPABILITY= 0x0,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_HAVE_JACK_PRESENCE_DETECTION_CAPABILITY= 0x1,
+} AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_JACK_DETECTION_CAPABILITY;
+typedef enum AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_TRIGGER_REQUIRED {
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_NO_TRIGGER_REQUIRED_FOR_IMPEDANCE_MEASUREMENT= 0x0,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_TRIGGER_REQUIRED_FOR_IMPEDANCE_MEASUREMENT= 0x1,
+} AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_TRIGGER_REQUIRED;
+typedef enum AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_IMPEDANCE_SENSE_CAPABLE {
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_NO_IMPEDANCE_SENSE_CAPABILITY= 0x0,
+ AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_HAVE_IMPEDANCE_SENSE_CAPABILITY= 0x1,
+} AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES_IMPEDANCE_SENSE_CAPABLE;
+typedef enum AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_HBR_HBR_CAPABLE {
+ AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_HBR_NO_HBR_CAPABILITY= 0x0,
+ AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_HBR_HAVE_HBR_CAPABILITY= 0x1,
+} AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_HBR_HBR_CAPABLE;
+typedef enum AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_STREAM_TYPE {
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_STREAM_TYPE_PCM= 0x0,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_STREAM_TYPE_NOT_PCM= 0x1,
+} AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_STREAM_TYPE;
+typedef enum AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_RATE {
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_RATE_48KHZ= 0x0,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_RATE_44P1KHZ= 0x1,
+} AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_RATE;
+typedef enum AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_MULTIPLE {
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_MULTIPLE_BY1= 0x0,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_MULTIPLE_BY2= 0x1,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_MULTIPLE_BY3_RESERVED= 0x2,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_MULTIPLE_BY4= 0x3,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_MULTIPLE_RESERVED= 0x4,
+} AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_MULTIPLE;
+typedef enum AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_DIVISOR {
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_DIVISOR_BY1= 0x0,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_DIVISOR_BY2_RESERVED= 0x1,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_DIVISOR_BY3= 0x2,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_DIVISOR_BY4_RESERVED= 0x3,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_DIVISOR_BY5_RESERVED= 0x4,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_DIVISOR_BY6_RESERVED= 0x5,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_DIVISOR_BY7_RESERVED= 0x6,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_DIVISOR_BY8_RESERVED= 0x7,
+} AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_SAMPLE_BASE_DIVISOR;
+typedef enum AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_BITS_PER_SAMPLE {
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_BITS_PER_SAMPLE_8_RESERVED= 0x0,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_BITS_PER_SAMPLE_16= 0x1,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_BITS_PER_SAMPLE_20= 0x2,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_BITS_PER_SAMPLE_24= 0x3,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_BITS_PER_SAMPLE_32_RESERVED= 0x4,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_BITS_PER_SAMPLE_RESERVED= 0x5,
+} AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_BITS_PER_SAMPLE;
+typedef enum AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_NUMBER_OF_CHANNELS {
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_NUMBER_OF_CHANNELS_1= 0x0,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_NUMBER_OF_CHANNELS_2= 0x1,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_NUMBER_OF_CHANNELS_3= 0x2,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_NUMBER_OF_CHANNELS_4= 0x3,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_NUMBER_OF_CHANNELS_5= 0x4,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_NUMBER_OF_CHANNELS_6= 0x5,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_NUMBER_OF_CHANNELS_7= 0x6,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_NUMBER_OF_CHANNELS_8= 0x7,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_NUMBER_OF_CHANNELS_RESERVED= 0x8,
+} AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT_NUMBER_OF_CHANNELS;
+typedef enum AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER_DIGEN {
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER_DIGEN_DIGITAL_TRANSMISSION_DISABLED= 0x0,
+ AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER_DIGEN_DIGITAL_TRANSMISSION_ENABLED= 0x1,
+} AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER_DIGEN;
+typedef enum AZALIA_F2_CODEC_INPUT_PIN_CONTROL_WIDGET_CONTROL_IN_ENABLE {
+ AZALIA_F2_CODEC_INPUT_PIN_CONTROL_WIDGET_CONTROL_IN_ENABLE_PIN_SHUT_OFF= 0x0,
+ AZALIA_F2_CODEC_INPUT_PIN_CONTROL_WIDGET_CONTROL_IN_ENABLE_PIN_DRIVEN= 0x1,
+} AZALIA_F2_CODEC_INPUT_PIN_CONTROL_WIDGET_CONTROL_IN_ENABLE;
+typedef enum AZALIA_F2_CODEC_INPUT_PIN_CONTROL_UNSOLICITED_RESPONSE_ENABLE {
+ AZALIA_F2_CODEC_INPUT_PIN_CONTROL_UNSOLICITED_RESPONSE_DISABLED= 0x0,
+ AZALIA_F2_CODEC_INPUT_PIN_CONTROL_UNSOLICITED_RESPONSE_ENABLED= 0x1,
+} AZALIA_F2_CODEC_INPUT_PIN_CONTROL_UNSOLICITED_RESPONSE_ENABLE;
+typedef enum AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL0_ENABLE_MULTICHANNEL0_MUTE {
+ AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL0_ENABLE_MULTICHANNEL0_NOT_MUTED= 0x0,
+ AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL0_ENABLE_MULTICHANNEL0_MUTED= 0x1,
+} AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL0_ENABLE_MULTICHANNEL0_MUTE;
+typedef enum AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL1_ENABLE_MULTICHANNEL1_MUTE {
+ AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL1_ENABLE_MULTICHANNEL1_NOT_MUTED= 0x0,
+ AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL1_ENABLE_MULTICHANNEL1_MUTED= 0x1,
+} AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL1_ENABLE_MULTICHANNEL1_MUTE;
+typedef enum AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL2_ENABLE_MULTICHANNEL2_MUTE {
+ AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL2_ENABLE_MULTICHANNEL2_NOT_MUTED= 0x0,
+ AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL2_ENABLE_MULTICHANNEL2_MUTED= 0x1,
+} AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL2_ENABLE_MULTICHANNEL2_MUTE;
+typedef enum AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL3_ENABLE_MULTICHANNEL3_MUTE {
+ AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL3_ENABLE_MULTICHANNEL3_NOT_MUTED= 0x0,
+ AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL3_ENABLE_MULTICHANNEL3_MUTED= 0x1,
+} AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL3_ENABLE_MULTICHANNEL3_MUTE;
+typedef enum AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL4_ENABLE_MULTICHANNEL4_MUTE {
+ AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL4_ENABLE_MULTICHANNEL4_NOT_MUTED= 0x0,
+ AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL4_ENABLE_MULTICHANNEL4_MUTED= 0x1,
+} AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL4_ENABLE_MULTICHANNEL4_MUTE;
+typedef enum AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL5_ENABLE_MULTICHANNEL5_MUTE {
+ AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL5_ENABLE_MULTICHANNEL5_NOT_MUTED= 0x0,
+ AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL5_ENABLE_MULTICHANNEL5_MUTED= 0x1,
+} AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL5_ENABLE_MULTICHANNEL5_MUTE;
+typedef enum AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL6_ENABLE_MULTICHANNEL6_MUTE {
+ AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL6_ENABLE_MULTICHANNEL6_NOT_MUTED= 0x0,
+ AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL6_ENABLE_MULTICHANNEL6_MUTED= 0x1,
+} AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL6_ENABLE_MULTICHANNEL6_MUTE;
+typedef enum AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL7_ENABLE_MULTICHANNEL7_MUTE {
+ AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL7_ENABLE_MULTICHANNEL7_NOT_MUTED= 0x0,
+ AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL7_ENABLE_MULTICHANNEL7_MUTED= 0x1,
+} AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL7_ENABLE_MULTICHANNEL7_MUTE;
+typedef enum BLND_CONTROL_BLND_MODE {
+ BLND_CONTROL_BLND_MODE_CURRENT_PIPE_ONLY = 0x0,
+ BLND_CONTROL_BLND_MODE_OTHER_PIPE_ONLY = 0x1,
+ BLND_CONTROL_BLND_MODE_ALPHA_BLENDING_MODE = 0x2,
+ BLND_CONTROL_BLND_MODE_OTHER_STEREO_TYPE = 0x3,
+} BLND_CONTROL_BLND_MODE;
+typedef enum BLND_CONTROL_BLND_STEREO_TYPE {
+ BLND_CONTROL_BLND_STEREO_TYPE_NON_SINGLE_PIPE_STEREO= 0x0,
+ BLND_CONTROL_BLND_STEREO_TYPE_SIDE_BY_SIDE_SINGLE_PIPE_STEREO= 0x1,
+ BLND_CONTROL_BLND_STEREO_TYPE_TOP_BOTTOM_SINGLE_PIPE_STEREO= 0x2,
+ BLND_CONTROL_BLND_STEREO_TYPE_UNUSED = 0x3,
+} BLND_CONTROL_BLND_STEREO_TYPE;
+typedef enum BLND_CONTROL_BLND_STEREO_POLARITY {
+ BLND_CONTROL_BLND_STEREO_POLARITY_LOW = 0x0,
+ BLND_CONTROL_BLND_STEREO_POLARITY_HIGH = 0x1,
+} BLND_CONTROL_BLND_STEREO_POLARITY;
+typedef enum BLND_CONTROL_BLND_FEEDTHROUGH_EN {
+ BLND_CONTROL_BLND_FEEDTHROUGH_EN_FALSE = 0x0,
+ BLND_CONTROL_BLND_FEEDTHROUGH_EN_TRUE = 0x1,
+} BLND_CONTROL_BLND_FEEDTHROUGH_EN;
+typedef enum BLND_CONTROL_BLND_ALPHA_MODE {
+ BLND_CONTROL_BLND_ALPHA_MODE_CURRENT_PIXEL_ALPHA = 0x0,
+ BLND_CONTROL_BLND_ALPHA_MODE_PIXEL_ALPHA_COMBINED_GLOBAL_GAIN= 0x1,
+ BLND_CONTROL_BLND_ALPHA_MODE_GLOBAL_ALPHA_ONLY = 0x2,
+ BLND_CONTROL_BLND_ALPHA_MODE_UNUSED = 0x3,
+} BLND_CONTROL_BLND_ALPHA_MODE;
+typedef enum BLND_CONTROL_BLND_ACTIVE_OVERLAP_ONLY {
+ BLND_CONTROL_BLND_ACTIVE_OVERLAY_ONLY_FALSE = 0x0,
+ BLND_CONTROL_BLND_ACTIVE_OVERLAY_ONLY_TRUE = 0x1,
+} BLND_CONTROL_BLND_ACTIVE_OVERLAP_ONLY;
+typedef enum BLND_CONTROL_BLND_MULTIPLIED_MODE {
+ BLND_CONTROL_BLND_MULTIPLIED_MODE_FALSE = 0x0,
+ BLND_CONTROL_BLND_MULTIPLIED_MODE_TRUE = 0x1,
+} BLND_CONTROL_BLND_MULTIPLIED_MODE;
+typedef enum BLND_SM_CONTROL2_SM_MODE {
+ BLND_SM_CONTROL2_SM_MODE_SINGLE_PLANE = 0x0,
+ BLND_SM_CONTROL2_SM_MODE_ROW_SUBSAMPLING = 0x2,
+ BLND_SM_CONTROL2_SM_MODE_COLUMN_SUBSAMPLING = 0x4,
+ BLND_SM_CONTROL2_SM_MODE_CHECKERBOARD_SUBSAMPLING= 0x6,
+} BLND_SM_CONTROL2_SM_MODE;
+typedef enum BLND_SM_CONTROL2_SM_FRAME_ALTERNATE {
+ BLND_SM_CONTROL2_SM_FRAME_ALTERNATE_FALSE = 0x0,
+ BLND_SM_CONTROL2_SM_FRAME_ALTERNATE_TRUE = 0x1,
+} BLND_SM_CONTROL2_SM_FRAME_ALTERNATE;
+typedef enum BLND_SM_CONTROL2_SM_FIELD_ALTERNATE {
+ BLND_SM_CONTROL2_SM_FIELD_ALTERNATE_FALSE = 0x0,
+ BLND_SM_CONTROL2_SM_FIELD_ALTERNATE_TRUE = 0x1,
+} BLND_SM_CONTROL2_SM_FIELD_ALTERNATE;
+typedef enum BLND_SM_CONTROL2_SM_FORCE_NEXT_FRAME_POL {
+ BLND_SM_CONTROL2_SM_FORCE_NEXT_FRAME_POL_NO_FORCE= 0x0,
+ BLND_SM_CONTROL2_SM_FORCE_NEXT_FRAME_POL_RESERVED= 0x1,
+ BLND_SM_CONTROL2_SM_FORCE_NEXT_FRAME_POL_FORCE_LOW= 0x2,
+ BLND_SM_CONTROL2_SM_FORCE_NEXT_FRAME_POL_FORCE_HIGH= 0x3,
+} BLND_SM_CONTROL2_SM_FORCE_NEXT_FRAME_POL;
+typedef enum BLND_SM_CONTROL2_SM_FORCE_NEXT_TOP_POL {
+ BLND_SM_CONTROL2_SM_FORCE_NEXT_TOP_POL_NO_FORCE = 0x0,
+ BLND_SM_CONTROL2_SM_FORCE_NEXT_TOP_POL_RESERVED = 0x1,
+ BLND_SM_CONTROL2_SM_FORCE_NEXT_TOP_POL_FORCE_LOW = 0x2,
+ BLND_SM_CONTROL2_SM_FORCE_NEXT_TOP_POL_FORCE_HIGH= 0x3,
+} BLND_SM_CONTROL2_SM_FORCE_NEXT_TOP_POL;
+typedef enum BLND_CONTROL2_PTI_ENABLE {
+ BLND_CONTROL2_PTI_ENABLE_FALSE = 0x0,
+ BLND_CONTROL2_PTI_ENABLE_TRUE = 0x1,
+} BLND_CONTROL2_PTI_ENABLE;
+typedef enum BLND_CONTROL2_BLND_SUPERAA_DEGAMMA_EN {
+ BLND_CONTROL2_BLND_SUPERAA_DEGAMMA_EN_FALSE = 0x0,
+ BLND_CONTROL2_BLND_SUPERAA_DEGAMMA_EN_TRUE = 0x1,
+} BLND_CONTROL2_BLND_SUPERAA_DEGAMMA_EN;
+typedef enum BLND_CONTROL2_BLND_SUPERAA_REGAMMA_EN {
+ BLND_CONTROL2_BLND_SUPERAA_REGAMMA_EN_FALSE = 0x0,
+ BLND_CONTROL2_BLND_SUPERAA_REGAMMA_EN_TRUE = 0x1,
+} BLND_CONTROL2_BLND_SUPERAA_REGAMMA_EN;
+typedef enum BLND_UNDERFLOW_INTERRUPT_BLND_UNDERFLOW_INT_ACK {
+ BLND_UNDERFLOW_INTERRUPT_BLND_UNDERFLOW_INT_ACK_FALSE= 0x0,
+ BLND_UNDERFLOW_INTERRUPT_BLND_UNDERFLOW_INT_ACK_TRUE= 0x1,
+} BLND_UNDERFLOW_INTERRUPT_BLND_UNDERFLOW_INT_ACK;
+typedef enum BLND_UNDERFLOW_INTERRUPT_BLND_UNDERFLOW_INT_MASK {
+ BLND_UNDERFLOW_INTERRUPT_BLND_UNDERFLOW_INT_MASK_FALSE= 0x0,
+ BLND_UNDERFLOW_INTERRUPT_BLND_UNDERFLOW_INT_MASK_TRUE= 0x1,
+} BLND_UNDERFLOW_INTERRUPT_BLND_UNDERFLOW_INT_MASK;
+typedef enum BLND_V_UPDATE_LOCK_BLND_DCP_GRPH_V_UPDATE_LOCK {
+ BLND_V_UPDATE_LOCK_BLND_DCP_GRPH_V_UPDATE_LOCK_FALSE= 0x0,
+ BLND_V_UPDATE_LOCK_BLND_DCP_GRPH_V_UPDATE_LOCK_TRUE= 0x1,
+} BLND_V_UPDATE_LOCK_BLND_DCP_GRPH_V_UPDATE_LOCK;
+typedef enum BLND_V_UPDATE_LOCK_BLND_DCP_GRPH_SURF_V_UPDATE_LOCK {
+ BLND_V_UPDATE_LOCK_BLND_DCP_GRPH_SURF_V_UPDATE_LOCK_FALSE= 0x0,
+ BLND_V_UPDATE_LOCK_BLND_DCP_GRPH_SURF_V_UPDATE_LOCK_TRUE= 0x1,
+} BLND_V_UPDATE_LOCK_BLND_DCP_GRPH_SURF_V_UPDATE_LOCK;
+typedef enum BLND_V_UPDATE_LOCK_BLND_DCP_CUR_V_UPDATE_LOCK {
+ BLND_V_UPDATE_LOCK_BLND_DCP_CUR_V_UPDATE_LOCK_FALSE= 0x0,
+ BLND_V_UPDATE_LOCK_BLND_DCP_CUR_V_UPDATE_LOCK_TRUE= 0x1,
+} BLND_V_UPDATE_LOCK_BLND_DCP_CUR_V_UPDATE_LOCK;
+typedef enum BLND_V_UPDATE_LOCK_BLND_DCP_CUR2_V_UPDATE_LOCK {
+ BLND_V_UPDATE_LOCK_BLND_DCP_CUR2_V_UPDATE_LOCK_FALSE= 0x0,
+ BLND_V_UPDATE_LOCK_BLND_DCP_CUR2_V_UPDATE_LOCK_TRUE= 0x1,
+} BLND_V_UPDATE_LOCK_BLND_DCP_CUR2_V_UPDATE_LOCK;
+typedef enum BLND_V_UPDATE_LOCK_BLND_SCL_V_UPDATE_LOCK {
+ BLND_V_UPDATE_LOCK_BLND_SCL_V_UPDATE_LOCK_FALSE = 0x0,
+ BLND_V_UPDATE_LOCK_BLND_SCL_V_UPDATE_LOCK_TRUE = 0x1,
+} BLND_V_UPDATE_LOCK_BLND_SCL_V_UPDATE_LOCK;
+typedef enum BLND_V_UPDATE_LOCK_BLND_BLND_V_UPDATE_LOCK {
+ BLND_V_UPDATE_LOCK_BLND_BLND_V_UPDATE_LOCK_FALSE = 0x0,
+ BLND_V_UPDATE_LOCK_BLND_BLND_V_UPDATE_LOCK_TRUE = 0x1,
+} BLND_V_UPDATE_LOCK_BLND_BLND_V_UPDATE_LOCK;
+typedef enum BLND_V_UPDATE_LOCK_BLND_V_UPDATE_LOCK_MODE {
+ BLND_V_UPDATE_LOCK_BLND_V_UPDATE_LOCK_MODE_FALSE = 0x0,
+ BLND_V_UPDATE_LOCK_BLND_V_UPDATE_LOCK_MODE_TRUE = 0x1,
+} BLND_V_UPDATE_LOCK_BLND_V_UPDATE_LOCK_MODE;
+typedef enum BLND_DEBUG_BLND_CNV_MUX_SELECT {
+ BLND_DEBUG_BLND_CNV_MUX_SELECT_LOW = 0x0,
+ BLND_DEBUG_BLND_CNV_MUX_SELECT_HIGH = 0x1,
+} BLND_DEBUG_BLND_CNV_MUX_SELECT;
+typedef enum BLND_TEST_DEBUG_INDEX_BLND_TEST_DEBUG_WRITE_EN {
+ BLND_TEST_DEBUG_INDEX_BLND_TEST_DEBUG_WRITE_EN_FALSE= 0x0,
+ BLND_TEST_DEBUG_INDEX_BLND_TEST_DEBUG_WRITE_EN_TRUE= 0x1,
+} BLND_TEST_DEBUG_INDEX_BLND_TEST_DEBUG_WRITE_EN;
+typedef enum SurfaceEndian {
+ ENDIAN_NONE = 0x0,
+ ENDIAN_8IN16 = 0x1,
+ ENDIAN_8IN32 = 0x2,
+ ENDIAN_8IN64 = 0x3,
+} SurfaceEndian;
+typedef enum ArrayMode {
+ ARRAY_LINEAR_GENERAL = 0x0,
+ ARRAY_LINEAR_ALIGNED = 0x1,
+ ARRAY_1D_TILED_THIN1 = 0x2,
+ ARRAY_1D_TILED_THICK = 0x3,
+ ARRAY_2D_TILED_THIN1 = 0x4,
+ ARRAY_PRT_TILED_THIN1 = 0x5,
+ ARRAY_PRT_2D_TILED_THIN1 = 0x6,
+ ARRAY_2D_TILED_THICK = 0x7,
+ ARRAY_2D_TILED_XTHICK = 0x8,
+ ARRAY_PRT_TILED_THICK = 0x9,
+ ARRAY_PRT_2D_TILED_THICK = 0xa,
+ ARRAY_PRT_3D_TILED_THIN1 = 0xb,
+ ARRAY_3D_TILED_THIN1 = 0xc,
+ ARRAY_3D_TILED_THICK = 0xd,
+ ARRAY_3D_TILED_XTHICK = 0xe,
+ ARRAY_PRT_3D_TILED_THICK = 0xf,
+} ArrayMode;
+typedef enum PipeTiling {
+ CONFIG_1_PIPE = 0x0,
+ CONFIG_2_PIPE = 0x1,
+ CONFIG_4_PIPE = 0x2,
+ CONFIG_8_PIPE = 0x3,
+} PipeTiling;
+typedef enum BankTiling {
+ CONFIG_4_BANK = 0x0,
+ CONFIG_8_BANK = 0x1,
+} BankTiling;
+typedef enum GroupInterleave {
+ CONFIG_256B_GROUP = 0x0,
+ CONFIG_512B_GROUP = 0x1,
+} GroupInterleave;
+typedef enum RowTiling {
+ CONFIG_1KB_ROW = 0x0,
+ CONFIG_2KB_ROW = 0x1,
+ CONFIG_4KB_ROW = 0x2,
+ CONFIG_8KB_ROW = 0x3,
+ CONFIG_1KB_ROW_OPT = 0x4,
+ CONFIG_2KB_ROW_OPT = 0x5,
+ CONFIG_4KB_ROW_OPT = 0x6,
+ CONFIG_8KB_ROW_OPT = 0x7,
+} RowTiling;
+typedef enum BankSwapBytes {
+ CONFIG_128B_SWAPS = 0x0,
+ CONFIG_256B_SWAPS = 0x1,
+ CONFIG_512B_SWAPS = 0x2,
+ CONFIG_1KB_SWAPS = 0x3,
+} BankSwapBytes;
+typedef enum SampleSplitBytes {
+ CONFIG_1KB_SPLIT = 0x0,
+ CONFIG_2KB_SPLIT = 0x1,
+ CONFIG_4KB_SPLIT = 0x2,
+ CONFIG_8KB_SPLIT = 0x3,
+} SampleSplitBytes;
+typedef enum NumPipes {
+ ADDR_CONFIG_1_PIPE = 0x0,
+ ADDR_CONFIG_2_PIPE = 0x1,
+ ADDR_CONFIG_4_PIPE = 0x2,
+ ADDR_CONFIG_8_PIPE = 0x3,
+} NumPipes;
+typedef enum PipeInterleaveSize {
+ ADDR_CONFIG_PIPE_INTERLEAVE_256B = 0x0,
+ ADDR_CONFIG_PIPE_INTERLEAVE_512B = 0x1,
+} PipeInterleaveSize;
+typedef enum BankInterleaveSize {
+ ADDR_CONFIG_BANK_INTERLEAVE_1 = 0x0,
+ ADDR_CONFIG_BANK_INTERLEAVE_2 = 0x1,
+ ADDR_CONFIG_BANK_INTERLEAVE_4 = 0x2,
+ ADDR_CONFIG_BANK_INTERLEAVE_8 = 0x3,
+} BankInterleaveSize;
+typedef enum NumShaderEngines {
+ ADDR_CONFIG_1_SHADER_ENGINE = 0x0,
+ ADDR_CONFIG_2_SHADER_ENGINE = 0x1,
+} NumShaderEngines;
+typedef enum ShaderEngineTileSize {
+ ADDR_CONFIG_SE_TILE_16 = 0x0,
+ ADDR_CONFIG_SE_TILE_32 = 0x1,
+} ShaderEngineTileSize;
+typedef enum NumGPUs {
+ ADDR_CONFIG_1_GPU = 0x0,
+ ADDR_CONFIG_2_GPU = 0x1,
+ ADDR_CONFIG_4_GPU = 0x2,
+} NumGPUs;
+typedef enum MultiGPUTileSize {
+ ADDR_CONFIG_GPU_TILE_16 = 0x0,
+ ADDR_CONFIG_GPU_TILE_32 = 0x1,
+ ADDR_CONFIG_GPU_TILE_64 = 0x2,
+ ADDR_CONFIG_GPU_TILE_128 = 0x3,
+} MultiGPUTileSize;
+typedef enum RowSize {
+ ADDR_CONFIG_1KB_ROW = 0x0,
+ ADDR_CONFIG_2KB_ROW = 0x1,
+ ADDR_CONFIG_4KB_ROW = 0x2,
+} RowSize;
+typedef enum NumLowerPipes {
+ ADDR_CONFIG_1_LOWER_PIPES = 0x0,
+ ADDR_CONFIG_2_LOWER_PIPES = 0x1,
+} NumLowerPipes;
+typedef enum DebugBlockId {
+ DBG_CLIENT_BLKID_RESERVED = 0x0,
+ DBG_CLIENT_BLKID_dbg = 0x1,
+ DBG_CLIENT_BLKID_scf2 = 0x2,
+ DBG_CLIENT_BLKID_mcd5 = 0x3,
+ DBG_CLIENT_BLKID_vmc = 0x4,
+ DBG_CLIENT_BLKID_sx30 = 0x5,
+ DBG_CLIENT_BLKID_mcd2 = 0x6,
+ DBG_CLIENT_BLKID_bci1 = 0x7,
+ DBG_CLIENT_BLKID_xdma_dbg_client_wrapper = 0x8,
+ DBG_CLIENT_BLKID_mcc0 = 0x9,
+ DBG_CLIENT_BLKID_uvdf_2 = 0xa,
+ DBG_CLIENT_BLKID_uvdf_3 = 0xb,
+ DBG_CLIENT_BLKID_uvdt_0 = 0xc,
+ DBG_CLIENT_BLKID_uvdi_0 = 0xd,
+ DBG_CLIENT_BLKID_bci0 = 0xe,
+ DBG_CLIENT_BLKID_vceb0_1 = 0xf,
+ DBG_CLIENT_BLKID_cb100 = 0x10,
+ DBG_CLIENT_BLKID_cb001 = 0x11,
+ DBG_CLIENT_BLKID_mcd4 = 0x12,
+ DBG_CLIENT_BLKID_tmonw00 = 0x13,
+ DBG_CLIENT_BLKID_cb101 = 0x14,
+ DBG_CLIENT_BLKID_sx10 = 0x15,
+ DBG_CLIENT_BLKID_cb301 = 0x16,
+ DBG_CLIENT_BLKID_tmonw01 = 0x17,
+ DBG_CLIENT_BLKID_vcea0_0 = 0x18,
+ DBG_CLIENT_BLKID_vcea0_1 = 0x19,
+ DBG_CLIENT_BLKID_vcea0_2 = 0x1a,
+ DBG_CLIENT_BLKID_vcea0_3 = 0x1b,
+ DBG_CLIENT_BLKID_scf1 = 0x1c,
+ DBG_CLIENT_BLKID_sx20 = 0x1d,
+ DBG_CLIENT_BLKID_spim1 = 0x1e,
+ DBG_CLIENT_BLKID_pa10 = 0x1f,
+ DBG_CLIENT_BLKID_pa00 = 0x20,
+ DBG_CLIENT_BLKID_gmcon = 0x21,
+ DBG_CLIENT_BLKID_mcb = 0x22,
+ DBG_CLIENT_BLKID_vgt0 = 0x23,
+ DBG_CLIENT_BLKID_pc0 = 0x24,
+ DBG_CLIENT_BLKID_bci2 = 0x25,
+ DBG_CLIENT_BLKID_uvdb_0 = 0x26,
+ DBG_CLIENT_BLKID_spim3 = 0x27,
+ DBG_CLIENT_BLKID_cpc_0 = 0x28,
+ DBG_CLIENT_BLKID_cpc_1 = 0x29,
+ DBG_CLIENT_BLKID_uvdm_0 = 0x2a,
+ DBG_CLIENT_BLKID_uvdm_1 = 0x2b,
+ DBG_CLIENT_BLKID_uvdm_2 = 0x2c,
+ DBG_CLIENT_BLKID_uvdm_3 = 0x2d,
+ DBG_CLIENT_BLKID_cb000 = 0x2e,
+ DBG_CLIENT_BLKID_spim0 = 0x2f,
+ DBG_CLIENT_BLKID_mcc2 = 0x30,
+ DBG_CLIENT_BLKID_ds0 = 0x31,
+ DBG_CLIENT_BLKID_srbm = 0x32,
+ DBG_CLIENT_BLKID_ih = 0x33,
+ DBG_CLIENT_BLKID_sem = 0x34,
+ DBG_CLIENT_BLKID_sdma_0 = 0x35,
+ DBG_CLIENT_BLKID_sdma_1 = 0x36,
+ DBG_CLIENT_BLKID_hdp = 0x37,
+ DBG_CLIENT_BLKID_cb200 = 0x38,
+ DBG_CLIENT_BLKID_scf3 = 0x39,
+ DBG_CLIENT_BLKID_vceb1_0 = 0x3a,
+ DBG_CLIENT_BLKID_vcea1_0 = 0x3b,
+ DBG_CLIENT_BLKID_vcea1_1 = 0x3c,
+ DBG_CLIENT_BLKID_vcea1_2 = 0x3d,
+ DBG_CLIENT_BLKID_vcea1_3 = 0x3e,
+ DBG_CLIENT_BLKID_bci3 = 0x3f,
+ DBG_CLIENT_BLKID_mcd0 = 0x40,
+ DBG_CLIENT_BLKID_pa11 = 0x41,
+ DBG_CLIENT_BLKID_pa01 = 0x42,
+ DBG_CLIENT_BLKID_cb201 = 0x43,
+ DBG_CLIENT_BLKID_spim2 = 0x44,
+ DBG_CLIENT_BLKID_vgt2 = 0x45,
+ DBG_CLIENT_BLKID_pc2 = 0x46,
+ DBG_CLIENT_BLKID_smu_0 = 0x47,
+ DBG_CLIENT_BLKID_smu_1 = 0x48,
+ DBG_CLIENT_BLKID_smu_2 = 0x49,
+ DBG_CLIENT_BLKID_cb1 = 0x4a,
+ DBG_CLIENT_BLKID_ia0 = 0x4b,
+ DBG_CLIENT_BLKID_wd = 0x4c,
+ DBG_CLIENT_BLKID_ia1 = 0x4d,
+ DBG_CLIENT_BLKID_vcec1_0 = 0x4e,
+ DBG_CLIENT_BLKID_scf0 = 0x4f,
+ DBG_CLIENT_BLKID_vgt1 = 0x50,
+ DBG_CLIENT_BLKID_pc1 = 0x51,
+ DBG_CLIENT_BLKID_cb0 = 0x52,
+ DBG_CLIENT_BLKID_gdc_one_0 = 0x53,
+ DBG_CLIENT_BLKID_gdc_one_1 = 0x54,
+ DBG_CLIENT_BLKID_gdc_one_2 = 0x55,
+ DBG_CLIENT_BLKID_gdc_one_3 = 0x56,
+ DBG_CLIENT_BLKID_gdc_one_4 = 0x57,
+ DBG_CLIENT_BLKID_gdc_one_5 = 0x58,
+ DBG_CLIENT_BLKID_gdc_one_6 = 0x59,
+ DBG_CLIENT_BLKID_gdc_one_7 = 0x5a,
+ DBG_CLIENT_BLKID_gdc_one_8 = 0x5b,
+ DBG_CLIENT_BLKID_gdc_one_9 = 0x5c,
+ DBG_CLIENT_BLKID_gdc_one_10 = 0x5d,
+ DBG_CLIENT_BLKID_gdc_one_11 = 0x5e,
+ DBG_CLIENT_BLKID_gdc_one_12 = 0x5f,
+ DBG_CLIENT_BLKID_gdc_one_13 = 0x60,
+ DBG_CLIENT_BLKID_gdc_one_14 = 0x61,
+ DBG_CLIENT_BLKID_gdc_one_15 = 0x62,
+ DBG_CLIENT_BLKID_gdc_one_16 = 0x63,
+ DBG_CLIENT_BLKID_gdc_one_17 = 0x64,
+ DBG_CLIENT_BLKID_gdc_one_18 = 0x65,
+ DBG_CLIENT_BLKID_gdc_one_19 = 0x66,
+ DBG_CLIENT_BLKID_gdc_one_20 = 0x67,
+ DBG_CLIENT_BLKID_gdc_one_21 = 0x68,
+ DBG_CLIENT_BLKID_gdc_one_22 = 0x69,
+ DBG_CLIENT_BLKID_gdc_one_23 = 0x6a,
+ DBG_CLIENT_BLKID_gdc_one_24 = 0x6b,
+ DBG_CLIENT_BLKID_gdc_one_25 = 0x6c,
+ DBG_CLIENT_BLKID_gdc_one_26 = 0x6d,
+ DBG_CLIENT_BLKID_gdc_one_27 = 0x6e,
+ DBG_CLIENT_BLKID_gdc_one_28 = 0x6f,
+ DBG_CLIENT_BLKID_gdc_one_29 = 0x70,
+ DBG_CLIENT_BLKID_gdc_one_30 = 0x71,
+ DBG_CLIENT_BLKID_gdc_one_31 = 0x72,
+ DBG_CLIENT_BLKID_gdc_one_32 = 0x73,
+ DBG_CLIENT_BLKID_gdc_one_33 = 0x74,
+ DBG_CLIENT_BLKID_gdc_one_34 = 0x75,
+ DBG_CLIENT_BLKID_gdc_one_35 = 0x76,
+ DBG_CLIENT_BLKID_vceb0_0 = 0x77,
+ DBG_CLIENT_BLKID_vgt3 = 0x78,
+ DBG_CLIENT_BLKID_pc3 = 0x79,
+ DBG_CLIENT_BLKID_mcd3 = 0x7a,
+ DBG_CLIENT_BLKID_uvdu_0 = 0x7b,
+ DBG_CLIENT_BLKID_uvdu_1 = 0x7c,
+ DBG_CLIENT_BLKID_uvdu_2 = 0x7d,
+ DBG_CLIENT_BLKID_uvdu_3 = 0x7e,
+ DBG_CLIENT_BLKID_uvdu_4 = 0x7f,
+ DBG_CLIENT_BLKID_uvdu_5 = 0x80,
+ DBG_CLIENT_BLKID_uvdu_6 = 0x81,
+ DBG_CLIENT_BLKID_cb300 = 0x82,
+ DBG_CLIENT_BLKID_mcd1 = 0x83,
+ DBG_CLIENT_BLKID_sx00 = 0x84,
+ DBG_CLIENT_BLKID_uvdf_0 = 0x85,
+ DBG_CLIENT_BLKID_uvdf_1 = 0x86,
+ DBG_CLIENT_BLKID_mcc3 = 0x87,
+ DBG_CLIENT_BLKID_cpg_0 = 0x88,
+ DBG_CLIENT_BLKID_cpg_1 = 0x89,
+ DBG_CLIENT_BLKID_gck = 0x8a,
+ DBG_CLIENT_BLKID_mcc1 = 0x8b,
+ DBG_CLIENT_BLKID_cpf_0 = 0x8c,
+ DBG_CLIENT_BLKID_cpf_1 = 0x8d,
+ DBG_CLIENT_BLKID_rlc = 0x8e,
+ DBG_CLIENT_BLKID_grbm = 0x8f,
+ DBG_CLIENT_BLKID_sammsp = 0x90,
+ DBG_CLIENT_BLKID_dci_pg = 0x91,
+ DBG_CLIENT_BLKID_dci_0 = 0x92,
+ DBG_CLIENT_BLKID_dccg0_0 = 0x93,
+ DBG_CLIENT_BLKID_dccg0_1 = 0x94,
+ DBG_CLIENT_BLKID_dccg0_2 = 0x95,
+ DBG_CLIENT_BLKID_dccg0_3 = 0x96,
+ DBG_CLIENT_BLKID_dccg0_4 = 0x97,
+ DBG_CLIENT_BLKID_dccg0_5 = 0x98,
+ DBG_CLIENT_BLKID_dccg0_6 = 0x99,
+ DBG_CLIENT_BLKID_dccg0_7 = 0x9a,
+ DBG_CLIENT_BLKID_dccg0_8 = 0x9b,
+ DBG_CLIENT_BLKID_dcfe01_0 = 0x9c,
+ DBG_CLIENT_BLKID_dcfe02_0 = 0x9d,
+ DBG_CLIENT_BLKID_dcfe03_0 = 0x9e,
+ DBG_CLIENT_BLKID_dcfe04_0 = 0x9f,
+ DBG_CLIENT_BLKID_dcfe05_0 = 0xa0,
+ DBG_CLIENT_BLKID_dcfe06_0 = 0xa1,
+ DBG_CLIENT_BLKID_uvde_0 = 0xa2,
+ DBG_CLIENT_BLKID_RESERVED_LAST = 0xa3,
+} DebugBlockId;
+typedef enum DebugBlockId_OLD {
+ DBG_BLOCK_ID_RESERVED = 0x0,
+ DBG_BLOCK_ID_DBG = 0x1,
+ DBG_BLOCK_ID_VMC = 0x2,
+ DBG_BLOCK_ID_PDMA = 0x3,
+ DBG_BLOCK_ID_CG = 0x4,
+ DBG_BLOCK_ID_SRBM = 0x5,
+ DBG_BLOCK_ID_GRBM = 0x6,
+ DBG_BLOCK_ID_RLC = 0x7,
+ DBG_BLOCK_ID_CSC = 0x8,
+ DBG_BLOCK_ID_SEM = 0x9,
+ DBG_BLOCK_ID_IH = 0xa,
+ DBG_BLOCK_ID_SC = 0xb,
+ DBG_BLOCK_ID_SQ = 0xc,
+ DBG_BLOCK_ID_AVP = 0xd,
+ DBG_BLOCK_ID_GMCON = 0xe,
+ DBG_BLOCK_ID_SMU = 0xf,
+ DBG_BLOCK_ID_DMA0 = 0x10,
+ DBG_BLOCK_ID_DMA1 = 0x11,
+ DBG_BLOCK_ID_SPIM = 0x12,
+ DBG_BLOCK_ID_GDS = 0x13,
+ DBG_BLOCK_ID_SPIS = 0x14,
+ DBG_BLOCK_ID_UNUSED0 = 0x15,
+ DBG_BLOCK_ID_PA0 = 0x16,
+ DBG_BLOCK_ID_PA1 = 0x17,
+ DBG_BLOCK_ID_CP0 = 0x18,
+ DBG_BLOCK_ID_CP1 = 0x19,
+ DBG_BLOCK_ID_CP2 = 0x1a,
+ DBG_BLOCK_ID_UNUSED1 = 0x1b,
+ DBG_BLOCK_ID_UVDU = 0x1c,
+ DBG_BLOCK_ID_UVDM = 0x1d,
+ DBG_BLOCK_ID_VCE = 0x1e,
+ DBG_BLOCK_ID_UNUSED2 = 0x1f,
+ DBG_BLOCK_ID_VGT0 = 0x20,
+ DBG_BLOCK_ID_VGT1 = 0x21,
+ DBG_BLOCK_ID_IA = 0x22,
+ DBG_BLOCK_ID_UNUSED3 = 0x23,
+ DBG_BLOCK_ID_SCT0 = 0x24,
+ DBG_BLOCK_ID_SCT1 = 0x25,
+ DBG_BLOCK_ID_SPM0 = 0x26,
+ DBG_BLOCK_ID_SPM1 = 0x27,
+ DBG_BLOCK_ID_TCAA = 0x28,
+ DBG_BLOCK_ID_TCAB = 0x29,
+ DBG_BLOCK_ID_TCCA = 0x2a,
+ DBG_BLOCK_ID_TCCB = 0x2b,
+ DBG_BLOCK_ID_MCC0 = 0x2c,
+ DBG_BLOCK_ID_MCC1 = 0x2d,
+ DBG_BLOCK_ID_MCC2 = 0x2e,
+ DBG_BLOCK_ID_MCC3 = 0x2f,
+ DBG_BLOCK_ID_SX0 = 0x30,
+ DBG_BLOCK_ID_SX1 = 0x31,
+ DBG_BLOCK_ID_SX2 = 0x32,
+ DBG_BLOCK_ID_SX3 = 0x33,
+ DBG_BLOCK_ID_UNUSED4 = 0x34,
+ DBG_BLOCK_ID_UNUSED5 = 0x35,
+ DBG_BLOCK_ID_UNUSED6 = 0x36,
+ DBG_BLOCK_ID_UNUSED7 = 0x37,
+ DBG_BLOCK_ID_PC0 = 0x38,
+ DBG_BLOCK_ID_PC1 = 0x39,
+ DBG_BLOCK_ID_UNUSED8 = 0x3a,
+ DBG_BLOCK_ID_UNUSED9 = 0x3b,
+ DBG_BLOCK_ID_UNUSED10 = 0x3c,
+ DBG_BLOCK_ID_UNUSED11 = 0x3d,
+ DBG_BLOCK_ID_MCB = 0x3e,
+ DBG_BLOCK_ID_UNUSED12 = 0x3f,
+ DBG_BLOCK_ID_SCB0 = 0x40,
+ DBG_BLOCK_ID_SCB1 = 0x41,
+ DBG_BLOCK_ID_UNUSED13 = 0x42,
+ DBG_BLOCK_ID_UNUSED14 = 0x43,
+ DBG_BLOCK_ID_SCF0 = 0x44,
+ DBG_BLOCK_ID_SCF1 = 0x45,
+ DBG_BLOCK_ID_UNUSED15 = 0x46,
+ DBG_BLOCK_ID_UNUSED16 = 0x47,
+ DBG_BLOCK_ID_BCI0 = 0x48,
+ DBG_BLOCK_ID_BCI1 = 0x49,
+ DBG_BLOCK_ID_BCI2 = 0x4a,
+ DBG_BLOCK_ID_BCI3 = 0x4b,
+ DBG_BLOCK_ID_UNUSED17 = 0x4c,
+ DBG_BLOCK_ID_UNUSED18 = 0x4d,
+ DBG_BLOCK_ID_UNUSED19 = 0x4e,
+ DBG_BLOCK_ID_UNUSED20 = 0x4f,
+ DBG_BLOCK_ID_CB00 = 0x50,
+ DBG_BLOCK_ID_CB01 = 0x51,
+ DBG_BLOCK_ID_CB02 = 0x52,
+ DBG_BLOCK_ID_CB03 = 0x53,
+ DBG_BLOCK_ID_CB04 = 0x54,
+ DBG_BLOCK_ID_UNUSED21 = 0x55,
+ DBG_BLOCK_ID_UNUSED22 = 0x56,
+ DBG_BLOCK_ID_UNUSED23 = 0x57,
+ DBG_BLOCK_ID_CB10 = 0x58,
+ DBG_BLOCK_ID_CB11 = 0x59,
+ DBG_BLOCK_ID_CB12 = 0x5a,
+ DBG_BLOCK_ID_CB13 = 0x5b,
+ DBG_BLOCK_ID_CB14 = 0x5c,
+ DBG_BLOCK_ID_UNUSED24 = 0x5d,
+ DBG_BLOCK_ID_UNUSED25 = 0x5e,
+ DBG_BLOCK_ID_UNUSED26 = 0x5f,
+ DBG_BLOCK_ID_TCP0 = 0x60,
+ DBG_BLOCK_ID_TCP1 = 0x61,
+ DBG_BLOCK_ID_TCP2 = 0x62,
+ DBG_BLOCK_ID_TCP3 = 0x63,
+ DBG_BLOCK_ID_TCP4 = 0x64,
+ DBG_BLOCK_ID_TCP5 = 0x65,
+ DBG_BLOCK_ID_TCP6 = 0x66,
+ DBG_BLOCK_ID_TCP7 = 0x67,
+ DBG_BLOCK_ID_TCP8 = 0x68,
+ DBG_BLOCK_ID_TCP9 = 0x69,
+ DBG_BLOCK_ID_TCP10 = 0x6a,
+ DBG_BLOCK_ID_TCP11 = 0x6b,
+ DBG_BLOCK_ID_TCP12 = 0x6c,
+ DBG_BLOCK_ID_TCP13 = 0x6d,
+ DBG_BLOCK_ID_TCP14 = 0x6e,
+ DBG_BLOCK_ID_TCP15 = 0x6f,
+ DBG_BLOCK_ID_TCP16 = 0x70,
+ DBG_BLOCK_ID_TCP17 = 0x71,
+ DBG_BLOCK_ID_TCP18 = 0x72,
+ DBG_BLOCK_ID_TCP19 = 0x73,
+ DBG_BLOCK_ID_TCP20 = 0x74,
+ DBG_BLOCK_ID_TCP21 = 0x75,
+ DBG_BLOCK_ID_TCP22 = 0x76,
+ DBG_BLOCK_ID_TCP23 = 0x77,
+ DBG_BLOCK_ID_TCP_RESERVED0 = 0x78,
+ DBG_BLOCK_ID_TCP_RESERVED1 = 0x79,
+ DBG_BLOCK_ID_TCP_RESERVED2 = 0x7a,
+ DBG_BLOCK_ID_TCP_RESERVED3 = 0x7b,
+ DBG_BLOCK_ID_TCP_RESERVED4 = 0x7c,
+ DBG_BLOCK_ID_TCP_RESERVED5 = 0x7d,
+ DBG_BLOCK_ID_TCP_RESERVED6 = 0x7e,
+ DBG_BLOCK_ID_TCP_RESERVED7 = 0x7f,
+ DBG_BLOCK_ID_DB00 = 0x80,
+ DBG_BLOCK_ID_DB01 = 0x81,
+ DBG_BLOCK_ID_DB02 = 0x82,
+ DBG_BLOCK_ID_DB03 = 0x83,
+ DBG_BLOCK_ID_DB04 = 0x84,
+ DBG_BLOCK_ID_UNUSED27 = 0x85,
+ DBG_BLOCK_ID_UNUSED28 = 0x86,
+ DBG_BLOCK_ID_UNUSED29 = 0x87,
+ DBG_BLOCK_ID_DB10 = 0x88,
+ DBG_BLOCK_ID_DB11 = 0x89,
+ DBG_BLOCK_ID_DB12 = 0x8a,
+ DBG_BLOCK_ID_DB13 = 0x8b,
+ DBG_BLOCK_ID_DB14 = 0x8c,
+ DBG_BLOCK_ID_UNUSED30 = 0x8d,
+ DBG_BLOCK_ID_UNUSED31 = 0x8e,
+ DBG_BLOCK_ID_UNUSED32 = 0x8f,
+ DBG_BLOCK_ID_TCC0 = 0x90,
+ DBG_BLOCK_ID_TCC1 = 0x91,
+ DBG_BLOCK_ID_TCC2 = 0x92,
+ DBG_BLOCK_ID_TCC3 = 0x93,
+ DBG_BLOCK_ID_TCC4 = 0x94,
+ DBG_BLOCK_ID_TCC5 = 0x95,
+ DBG_BLOCK_ID_TCC6 = 0x96,
+ DBG_BLOCK_ID_TCC7 = 0x97,
+ DBG_BLOCK_ID_SPS00 = 0x98,
+ DBG_BLOCK_ID_SPS01 = 0x99,
+ DBG_BLOCK_ID_SPS02 = 0x9a,
+ DBG_BLOCK_ID_SPS10 = 0x9b,
+ DBG_BLOCK_ID_SPS11 = 0x9c,
+ DBG_BLOCK_ID_SPS12 = 0x9d,
+ DBG_BLOCK_ID_UNUSED33 = 0x9e,
+ DBG_BLOCK_ID_UNUSED34 = 0x9f,
+ DBG_BLOCK_ID_TA00 = 0xa0,
+ DBG_BLOCK_ID_TA01 = 0xa1,
+ DBG_BLOCK_ID_TA02 = 0xa2,
+ DBG_BLOCK_ID_TA03 = 0xa3,
+ DBG_BLOCK_ID_TA04 = 0xa4,
+ DBG_BLOCK_ID_TA05 = 0xa5,
+ DBG_BLOCK_ID_TA06 = 0xa6,
+ DBG_BLOCK_ID_TA07 = 0xa7,
+ DBG_BLOCK_ID_TA08 = 0xa8,
+ DBG_BLOCK_ID_TA09 = 0xa9,
+ DBG_BLOCK_ID_TA0A = 0xaa,
+ DBG_BLOCK_ID_TA0B = 0xab,
+ DBG_BLOCK_ID_UNUSED35 = 0xac,
+ DBG_BLOCK_ID_UNUSED36 = 0xad,
+ DBG_BLOCK_ID_UNUSED37 = 0xae,
+ DBG_BLOCK_ID_UNUSED38 = 0xaf,
+ DBG_BLOCK_ID_TA10 = 0xb0,
+ DBG_BLOCK_ID_TA11 = 0xb1,
+ DBG_BLOCK_ID_TA12 = 0xb2,
+ DBG_BLOCK_ID_TA13 = 0xb3,
+ DBG_BLOCK_ID_TA14 = 0xb4,
+ DBG_BLOCK_ID_TA15 = 0xb5,
+ DBG_BLOCK_ID_TA16 = 0xb6,
+ DBG_BLOCK_ID_TA17 = 0xb7,
+ DBG_BLOCK_ID_TA18 = 0xb8,
+ DBG_BLOCK_ID_TA19 = 0xb9,
+ DBG_BLOCK_ID_TA1A = 0xba,
+ DBG_BLOCK_ID_TA1B = 0xbb,
+ DBG_BLOCK_ID_UNUSED39 = 0xbc,
+ DBG_BLOCK_ID_UNUSED40 = 0xbd,
+ DBG_BLOCK_ID_UNUSED41 = 0xbe,
+ DBG_BLOCK_ID_UNUSED42 = 0xbf,
+ DBG_BLOCK_ID_TD00 = 0xc0,
+ DBG_BLOCK_ID_TD01 = 0xc1,
+ DBG_BLOCK_ID_TD02 = 0xc2,
+ DBG_BLOCK_ID_TD03 = 0xc3,
+ DBG_BLOCK_ID_TD04 = 0xc4,
+ DBG_BLOCK_ID_TD05 = 0xc5,
+ DBG_BLOCK_ID_TD06 = 0xc6,
+ DBG_BLOCK_ID_TD07 = 0xc7,
+ DBG_BLOCK_ID_TD08 = 0xc8,
+ DBG_BLOCK_ID_TD09 = 0xc9,
+ DBG_BLOCK_ID_TD0A = 0xca,
+ DBG_BLOCK_ID_TD0B = 0xcb,
+ DBG_BLOCK_ID_UNUSED43 = 0xcc,
+ DBG_BLOCK_ID_UNUSED44 = 0xcd,
+ DBG_BLOCK_ID_UNUSED45 = 0xce,
+ DBG_BLOCK_ID_UNUSED46 = 0xcf,
+ DBG_BLOCK_ID_TD10 = 0xd0,
+ DBG_BLOCK_ID_TD11 = 0xd1,
+ DBG_BLOCK_ID_TD12 = 0xd2,
+ DBG_BLOCK_ID_TD13 = 0xd3,
+ DBG_BLOCK_ID_TD14 = 0xd4,
+ DBG_BLOCK_ID_TD15 = 0xd5,
+ DBG_BLOCK_ID_TD16 = 0xd6,
+ DBG_BLOCK_ID_TD17 = 0xd7,
+ DBG_BLOCK_ID_TD18 = 0xd8,
+ DBG_BLOCK_ID_TD19 = 0xd9,
+ DBG_BLOCK_ID_TD1A = 0xda,
+ DBG_BLOCK_ID_TD1B = 0xdb,
+ DBG_BLOCK_ID_UNUSED47 = 0xdc,
+ DBG_BLOCK_ID_UNUSED48 = 0xdd,
+ DBG_BLOCK_ID_UNUSED49 = 0xde,
+ DBG_BLOCK_ID_UNUSED50 = 0xdf,
+ DBG_BLOCK_ID_MCD0 = 0xe0,
+ DBG_BLOCK_ID_MCD1 = 0xe1,
+ DBG_BLOCK_ID_MCD2 = 0xe2,
+ DBG_BLOCK_ID_MCD3 = 0xe3,
+ DBG_BLOCK_ID_MCD4 = 0xe4,
+ DBG_BLOCK_ID_MCD5 = 0xe5,
+ DBG_BLOCK_ID_UNUSED51 = 0xe6,
+ DBG_BLOCK_ID_UNUSED52 = 0xe7,
+} DebugBlockId_OLD;
+typedef enum DebugBlockId_BY2 {
+ DBG_BLOCK_ID_RESERVED_BY2 = 0x0,
+ DBG_BLOCK_ID_VMC_BY2 = 0x1,
+ DBG_BLOCK_ID_CG_BY2 = 0x2,
+ DBG_BLOCK_ID_GRBM_BY2 = 0x3,
+ DBG_BLOCK_ID_CSC_BY2 = 0x4,
+ DBG_BLOCK_ID_IH_BY2 = 0x5,
+ DBG_BLOCK_ID_SQ_BY2 = 0x6,
+ DBG_BLOCK_ID_GMCON_BY2 = 0x7,
+ DBG_BLOCK_ID_DMA0_BY2 = 0x8,
+ DBG_BLOCK_ID_SPIM_BY2 = 0x9,
+ DBG_BLOCK_ID_SPIS_BY2 = 0xa,
+ DBG_BLOCK_ID_PA0_BY2 = 0xb,
+ DBG_BLOCK_ID_CP0_BY2 = 0xc,
+ DBG_BLOCK_ID_CP2_BY2 = 0xd,
+ DBG_BLOCK_ID_UVDU_BY2 = 0xe,
+ DBG_BLOCK_ID_VCE_BY2 = 0xf,
+ DBG_BLOCK_ID_VGT0_BY2 = 0x10,
+ DBG_BLOCK_ID_IA_BY2 = 0x11,
+ DBG_BLOCK_ID_SCT0_BY2 = 0x12,
+ DBG_BLOCK_ID_SPM0_BY2 = 0x13,
+ DBG_BLOCK_ID_TCAA_BY2 = 0x14,
+ DBG_BLOCK_ID_TCCA_BY2 = 0x15,
+ DBG_BLOCK_ID_MCC0_BY2 = 0x16,
+ DBG_BLOCK_ID_MCC2_BY2 = 0x17,
+ DBG_BLOCK_ID_SX0_BY2 = 0x18,
+ DBG_BLOCK_ID_SX2_BY2 = 0x19,
+ DBG_BLOCK_ID_UNUSED4_BY2 = 0x1a,
+ DBG_BLOCK_ID_UNUSED6_BY2 = 0x1b,
+ DBG_BLOCK_ID_PC0_BY2 = 0x1c,
+ DBG_BLOCK_ID_UNUSED8_BY2 = 0x1d,
+ DBG_BLOCK_ID_UNUSED10_BY2 = 0x1e,
+ DBG_BLOCK_ID_MCB_BY2 = 0x1f,
+ DBG_BLOCK_ID_SCB0_BY2 = 0x20,
+ DBG_BLOCK_ID_UNUSED13_BY2 = 0x21,
+ DBG_BLOCK_ID_SCF0_BY2 = 0x22,
+ DBG_BLOCK_ID_UNUSED15_BY2 = 0x23,
+ DBG_BLOCK_ID_BCI0_BY2 = 0x24,
+ DBG_BLOCK_ID_BCI2_BY2 = 0x25,
+ DBG_BLOCK_ID_UNUSED17_BY2 = 0x26,
+ DBG_BLOCK_ID_UNUSED19_BY2 = 0x27,
+ DBG_BLOCK_ID_CB00_BY2 = 0x28,
+ DBG_BLOCK_ID_CB02_BY2 = 0x29,
+ DBG_BLOCK_ID_CB04_BY2 = 0x2a,
+ DBG_BLOCK_ID_UNUSED22_BY2 = 0x2b,
+ DBG_BLOCK_ID_CB10_BY2 = 0x2c,
+ DBG_BLOCK_ID_CB12_BY2 = 0x2d,
+ DBG_BLOCK_ID_CB14_BY2 = 0x2e,
+ DBG_BLOCK_ID_UNUSED25_BY2 = 0x2f,
+ DBG_BLOCK_ID_TCP0_BY2 = 0x30,
+ DBG_BLOCK_ID_TCP2_BY2 = 0x31,
+ DBG_BLOCK_ID_TCP4_BY2 = 0x32,
+ DBG_BLOCK_ID_TCP6_BY2 = 0x33,
+ DBG_BLOCK_ID_TCP8_BY2 = 0x34,
+ DBG_BLOCK_ID_TCP10_BY2 = 0x35,
+ DBG_BLOCK_ID_TCP12_BY2 = 0x36,
+ DBG_BLOCK_ID_TCP14_BY2 = 0x37,
+ DBG_BLOCK_ID_TCP16_BY2 = 0x38,
+ DBG_BLOCK_ID_TCP18_BY2 = 0x39,
+ DBG_BLOCK_ID_TCP20_BY2 = 0x3a,
+ DBG_BLOCK_ID_TCP22_BY2 = 0x3b,
+ DBG_BLOCK_ID_TCP_RESERVED0_BY2 = 0x3c,
+ DBG_BLOCK_ID_TCP_RESERVED2_BY2 = 0x3d,
+ DBG_BLOCK_ID_TCP_RESERVED4_BY2 = 0x3e,
+ DBG_BLOCK_ID_TCP_RESERVED6_BY2 = 0x3f,
+ DBG_BLOCK_ID_DB00_BY2 = 0x40,
+ DBG_BLOCK_ID_DB02_BY2 = 0x41,
+ DBG_BLOCK_ID_DB04_BY2 = 0x42,
+ DBG_BLOCK_ID_UNUSED28_BY2 = 0x43,
+ DBG_BLOCK_ID_DB10_BY2 = 0x44,
+ DBG_BLOCK_ID_DB12_BY2 = 0x45,
+ DBG_BLOCK_ID_DB14_BY2 = 0x46,
+ DBG_BLOCK_ID_UNUSED31_BY2 = 0x47,
+ DBG_BLOCK_ID_TCC0_BY2 = 0x48,
+ DBG_BLOCK_ID_TCC2_BY2 = 0x49,
+ DBG_BLOCK_ID_TCC4_BY2 = 0x4a,
+ DBG_BLOCK_ID_TCC6_BY2 = 0x4b,
+ DBG_BLOCK_ID_SPS00_BY2 = 0x4c,
+ DBG_BLOCK_ID_SPS02_BY2 = 0x4d,
+ DBG_BLOCK_ID_SPS11_BY2 = 0x4e,
+ DBG_BLOCK_ID_UNUSED33_BY2 = 0x4f,
+ DBG_BLOCK_ID_TA00_BY2 = 0x50,
+ DBG_BLOCK_ID_TA02_BY2 = 0x51,
+ DBG_BLOCK_ID_TA04_BY2 = 0x52,
+ DBG_BLOCK_ID_TA06_BY2 = 0x53,
+ DBG_BLOCK_ID_TA08_BY2 = 0x54,
+ DBG_BLOCK_ID_TA0A_BY2 = 0x55,
+ DBG_BLOCK_ID_UNUSED35_BY2 = 0x56,
+ DBG_BLOCK_ID_UNUSED37_BY2 = 0x57,
+ DBG_BLOCK_ID_TA10_BY2 = 0x58,
+ DBG_BLOCK_ID_TA12_BY2 = 0x59,
+ DBG_BLOCK_ID_TA14_BY2 = 0x5a,
+ DBG_BLOCK_ID_TA16_BY2 = 0x5b,
+ DBG_BLOCK_ID_TA18_BY2 = 0x5c,
+ DBG_BLOCK_ID_TA1A_BY2 = 0x5d,
+ DBG_BLOCK_ID_UNUSED39_BY2 = 0x5e,
+ DBG_BLOCK_ID_UNUSED41_BY2 = 0x5f,
+ DBG_BLOCK_ID_TD00_BY2 = 0x60,
+ DBG_BLOCK_ID_TD02_BY2 = 0x61,
+ DBG_BLOCK_ID_TD04_BY2 = 0x62,
+ DBG_BLOCK_ID_TD06_BY2 = 0x63,
+ DBG_BLOCK_ID_TD08_BY2 = 0x64,
+ DBG_BLOCK_ID_TD0A_BY2 = 0x65,
+ DBG_BLOCK_ID_UNUSED43_BY2 = 0x66,
+ DBG_BLOCK_ID_UNUSED45_BY2 = 0x67,
+ DBG_BLOCK_ID_TD10_BY2 = 0x68,
+ DBG_BLOCK_ID_TD12_BY2 = 0x69,
+ DBG_BLOCK_ID_TD14_BY2 = 0x6a,
+ DBG_BLOCK_ID_TD16_BY2 = 0x6b,
+ DBG_BLOCK_ID_TD18_BY2 = 0x6c,
+ DBG_BLOCK_ID_TD1A_BY2 = 0x6d,
+ DBG_BLOCK_ID_UNUSED47_BY2 = 0x6e,
+ DBG_BLOCK_ID_UNUSED49_BY2 = 0x6f,
+ DBG_BLOCK_ID_MCD0_BY2 = 0x70,
+ DBG_BLOCK_ID_MCD2_BY2 = 0x71,
+ DBG_BLOCK_ID_MCD4_BY2 = 0x72,
+ DBG_BLOCK_ID_UNUSED51_BY2 = 0x73,
+} DebugBlockId_BY2;
+typedef enum DebugBlockId_BY4 {
+ DBG_BLOCK_ID_RESERVED_BY4 = 0x0,
+ DBG_BLOCK_ID_CG_BY4 = 0x1,
+ DBG_BLOCK_ID_CSC_BY4 = 0x2,
+ DBG_BLOCK_ID_SQ_BY4 = 0x3,
+ DBG_BLOCK_ID_DMA0_BY4 = 0x4,
+ DBG_BLOCK_ID_SPIS_BY4 = 0x5,
+ DBG_BLOCK_ID_CP0_BY4 = 0x6,
+ DBG_BLOCK_ID_UVDU_BY4 = 0x7,
+ DBG_BLOCK_ID_VGT0_BY4 = 0x8,
+ DBG_BLOCK_ID_SCT0_BY4 = 0x9,
+ DBG_BLOCK_ID_TCAA_BY4 = 0xa,
+ DBG_BLOCK_ID_MCC0_BY4 = 0xb,
+ DBG_BLOCK_ID_SX0_BY4 = 0xc,
+ DBG_BLOCK_ID_UNUSED4_BY4 = 0xd,
+ DBG_BLOCK_ID_PC0_BY4 = 0xe,
+ DBG_BLOCK_ID_UNUSED10_BY4 = 0xf,
+ DBG_BLOCK_ID_SCB0_BY4 = 0x10,
+ DBG_BLOCK_ID_SCF0_BY4 = 0x11,
+ DBG_BLOCK_ID_BCI0_BY4 = 0x12,
+ DBG_BLOCK_ID_UNUSED17_BY4 = 0x13,
+ DBG_BLOCK_ID_CB00_BY4 = 0x14,
+ DBG_BLOCK_ID_CB04_BY4 = 0x15,
+ DBG_BLOCK_ID_CB10_BY4 = 0x16,
+ DBG_BLOCK_ID_CB14_BY4 = 0x17,
+ DBG_BLOCK_ID_TCP0_BY4 = 0x18,
+ DBG_BLOCK_ID_TCP4_BY4 = 0x19,
+ DBG_BLOCK_ID_TCP8_BY4 = 0x1a,
+ DBG_BLOCK_ID_TCP12_BY4 = 0x1b,
+ DBG_BLOCK_ID_TCP16_BY4 = 0x1c,
+ DBG_BLOCK_ID_TCP20_BY4 = 0x1d,
+ DBG_BLOCK_ID_TCP_RESERVED0_BY4 = 0x1e,
+ DBG_BLOCK_ID_TCP_RESERVED4_BY4 = 0x1f,
+ DBG_BLOCK_ID_DB_BY4 = 0x20,
+ DBG_BLOCK_ID_DB04_BY4 = 0x21,
+ DBG_BLOCK_ID_DB10_BY4 = 0x22,
+ DBG_BLOCK_ID_DB14_BY4 = 0x23,
+ DBG_BLOCK_ID_TCC0_BY4 = 0x24,
+ DBG_BLOCK_ID_TCC4_BY4 = 0x25,
+ DBG_BLOCK_ID_SPS00_BY4 = 0x26,
+ DBG_BLOCK_ID_SPS11_BY4 = 0x27,
+ DBG_BLOCK_ID_TA00_BY4 = 0x28,
+ DBG_BLOCK_ID_TA04_BY4 = 0x29,
+ DBG_BLOCK_ID_TA08_BY4 = 0x2a,
+ DBG_BLOCK_ID_UNUSED35_BY4 = 0x2b,
+ DBG_BLOCK_ID_TA10_BY4 = 0x2c,
+ DBG_BLOCK_ID_TA14_BY4 = 0x2d,
+ DBG_BLOCK_ID_TA18_BY4 = 0x2e,
+ DBG_BLOCK_ID_UNUSED39_BY4 = 0x2f,
+ DBG_BLOCK_ID_TD00_BY4 = 0x30,
+ DBG_BLOCK_ID_TD04_BY4 = 0x31,
+ DBG_BLOCK_ID_TD08_BY4 = 0x32,
+ DBG_BLOCK_ID_UNUSED43_BY4 = 0x33,
+ DBG_BLOCK_ID_TD10_BY4 = 0x34,
+ DBG_BLOCK_ID_TD14_BY4 = 0x35,
+ DBG_BLOCK_ID_TD18_BY4 = 0x36,
+ DBG_BLOCK_ID_UNUSED47_BY4 = 0x37,
+ DBG_BLOCK_ID_MCD0_BY4 = 0x38,
+ DBG_BLOCK_ID_MCD4_BY4 = 0x39,
+} DebugBlockId_BY4;
+typedef enum DebugBlockId_BY8 {
+ DBG_BLOCK_ID_RESERVED_BY8 = 0x0,
+ DBG_BLOCK_ID_CSC_BY8 = 0x1,
+ DBG_BLOCK_ID_DMA0_BY8 = 0x2,
+ DBG_BLOCK_ID_CP0_BY8 = 0x3,
+ DBG_BLOCK_ID_VGT0_BY8 = 0x4,
+ DBG_BLOCK_ID_TCAA_BY8 = 0x5,
+ DBG_BLOCK_ID_SX0_BY8 = 0x6,
+ DBG_BLOCK_ID_PC0_BY8 = 0x7,
+ DBG_BLOCK_ID_SCB0_BY8 = 0x8,
+ DBG_BLOCK_ID_BCI0_BY8 = 0x9,
+ DBG_BLOCK_ID_CB00_BY8 = 0xa,
+ DBG_BLOCK_ID_CB10_BY8 = 0xb,
+ DBG_BLOCK_ID_TCP0_BY8 = 0xc,
+ DBG_BLOCK_ID_TCP8_BY8 = 0xd,
+ DBG_BLOCK_ID_TCP16_BY8 = 0xe,
+ DBG_BLOCK_ID_TCP_RESERVED0_BY8 = 0xf,
+ DBG_BLOCK_ID_DB00_BY8 = 0x10,
+ DBG_BLOCK_ID_DB10_BY8 = 0x11,
+ DBG_BLOCK_ID_TCC0_BY8 = 0x12,
+ DBG_BLOCK_ID_SPS00_BY8 = 0x13,
+ DBG_BLOCK_ID_TA00_BY8 = 0x14,
+ DBG_BLOCK_ID_TA08_BY8 = 0x15,
+ DBG_BLOCK_ID_TA10_BY8 = 0x16,
+ DBG_BLOCK_ID_TA18_BY8 = 0x17,
+ DBG_BLOCK_ID_TD00_BY8 = 0x18,
+ DBG_BLOCK_ID_TD08_BY8 = 0x19,
+ DBG_BLOCK_ID_TD10_BY8 = 0x1a,
+ DBG_BLOCK_ID_TD18_BY8 = 0x1b,
+ DBG_BLOCK_ID_MCD0_BY8 = 0x1c,
+} DebugBlockId_BY8;
+typedef enum DebugBlockId_BY16 {
+ DBG_BLOCK_ID_RESERVED_BY16 = 0x0,
+ DBG_BLOCK_ID_DMA0_BY16 = 0x1,
+ DBG_BLOCK_ID_VGT0_BY16 = 0x2,
+ DBG_BLOCK_ID_SX0_BY16 = 0x3,
+ DBG_BLOCK_ID_SCB0_BY16 = 0x4,
+ DBG_BLOCK_ID_CB00_BY16 = 0x5,
+ DBG_BLOCK_ID_TCP0_BY16 = 0x6,
+ DBG_BLOCK_ID_TCP16_BY16 = 0x7,
+ DBG_BLOCK_ID_DB00_BY16 = 0x8,
+ DBG_BLOCK_ID_TCC0_BY16 = 0x9,
+ DBG_BLOCK_ID_TA00_BY16 = 0xa,
+ DBG_BLOCK_ID_TA10_BY16 = 0xb,
+ DBG_BLOCK_ID_TD00_BY16 = 0xc,
+ DBG_BLOCK_ID_TD10_BY16 = 0xd,
+ DBG_BLOCK_ID_MCD0_BY16 = 0xe,
+} DebugBlockId_BY16;
+typedef enum ColorTransform {
+ DCC_CT_AUTO = 0x0,
+ DCC_CT_NONE = 0x1,
+ ABGR_TO_A_BG_G_RB = 0x2,
+ BGRA_TO_BG_G_RB_A = 0x3,
+} ColorTransform;
+typedef enum CompareRef {
+ REF_NEVER = 0x0,
+ REF_LESS = 0x1,
+ REF_EQUAL = 0x2,
+ REF_LEQUAL = 0x3,
+ REF_GREATER = 0x4,
+ REF_NOTEQUAL = 0x5,
+ REF_GEQUAL = 0x6,
+ REF_ALWAYS = 0x7,
+} CompareRef;
+typedef enum ReadSize {
+ READ_256_BITS = 0x0,
+ READ_512_BITS = 0x1,
+} ReadSize;
+typedef enum DepthFormat {
+ DEPTH_INVALID = 0x0,
+ DEPTH_16 = 0x1,
+ DEPTH_X8_24 = 0x2,
+ DEPTH_8_24 = 0x3,
+ DEPTH_X8_24_FLOAT = 0x4,
+ DEPTH_8_24_FLOAT = 0x5,
+ DEPTH_32_FLOAT = 0x6,
+ DEPTH_X24_8_32_FLOAT = 0x7,
+} DepthFormat;
+typedef enum ZFormat {
+ Z_INVALID = 0x0,
+ Z_16 = 0x1,
+ Z_24 = 0x2,
+ Z_32_FLOAT = 0x3,
+} ZFormat;
+typedef enum StencilFormat {
+ STENCIL_INVALID = 0x0,
+ STENCIL_8 = 0x1,
+} StencilFormat;
+typedef enum CmaskMode {
+ CMASK_CLEAR_NONE = 0x0,
+ CMASK_CLEAR_ONE = 0x1,
+ CMASK_CLEAR_ALL = 0x2,
+ CMASK_ANY_EXPANDED = 0x3,
+ CMASK_ALPHA0_FRAG1 = 0x4,
+ CMASK_ALPHA0_FRAG2 = 0x5,
+ CMASK_ALPHA0_FRAG4 = 0x6,
+ CMASK_ALPHA0_FRAGS = 0x7,
+ CMASK_ALPHA1_FRAG1 = 0x8,
+ CMASK_ALPHA1_FRAG2 = 0x9,
+ CMASK_ALPHA1_FRAG4 = 0xa,
+ CMASK_ALPHA1_FRAGS = 0xb,
+ CMASK_ALPHAX_FRAG1 = 0xc,
+ CMASK_ALPHAX_FRAG2 = 0xd,
+ CMASK_ALPHAX_FRAG4 = 0xe,
+ CMASK_ALPHAX_FRAGS = 0xf,
+} CmaskMode;
+typedef enum QuadExportFormat {
+ EXPORT_UNUSED = 0x0,
+ EXPORT_32_R = 0x1,
+ EXPORT_32_GR = 0x2,
+ EXPORT_32_AR = 0x3,
+ EXPORT_FP16_ABGR = 0x4,
+ EXPORT_UNSIGNED16_ABGR = 0x5,
+ EXPORT_SIGNED16_ABGR = 0x6,
+ EXPORT_32_ABGR = 0x7,
+} QuadExportFormat;
+typedef enum QuadExportFormatOld {
+ EXPORT_4P_32BPC_ABGR = 0x0,
+ EXPORT_4P_16BPC_ABGR = 0x1,
+ EXPORT_4P_32BPC_GR = 0x2,
+ EXPORT_4P_32BPC_AR = 0x3,
+ EXPORT_2P_32BPC_ABGR = 0x4,
+ EXPORT_8P_32BPC_R = 0x5,
+} QuadExportFormatOld;
+typedef enum ColorFormat {
+ COLOR_INVALID = 0x0,
+ COLOR_8 = 0x1,
+ COLOR_16 = 0x2,
+ COLOR_8_8 = 0x3,
+ COLOR_32 = 0x4,
+ COLOR_16_16 = 0x5,
+ COLOR_10_11_11 = 0x6,
+ COLOR_11_11_10 = 0x7,
+ COLOR_10_10_10_2 = 0x8,
+ COLOR_2_10_10_10 = 0x9,
+ COLOR_8_8_8_8 = 0xa,
+ COLOR_32_32 = 0xb,
+ COLOR_16_16_16_16 = 0xc,
+ COLOR_RESERVED_13 = 0xd,
+ COLOR_32_32_32_32 = 0xe,
+ COLOR_RESERVED_15 = 0xf,
+ COLOR_5_6_5 = 0x10,
+ COLOR_1_5_5_5 = 0x11,
+ COLOR_5_5_5_1 = 0x12,
+ COLOR_4_4_4_4 = 0x13,
+ COLOR_8_24 = 0x14,
+ COLOR_24_8 = 0x15,
+ COLOR_X24_8_32_FLOAT = 0x16,
+ COLOR_RESERVED_23 = 0x17,
+} ColorFormat;
+typedef enum SurfaceFormat {
+ FMT_INVALID = 0x0,
+ FMT_8 = 0x1,
+ FMT_16 = 0x2,
+ FMT_8_8 = 0x3,
+ FMT_32 = 0x4,
+ FMT_16_16 = 0x5,
+ FMT_10_11_11 = 0x6,
+ FMT_11_11_10 = 0x7,
+ FMT_10_10_10_2 = 0x8,
+ FMT_2_10_10_10 = 0x9,
+ FMT_8_8_8_8 = 0xa,
+ FMT_32_32 = 0xb,
+ FMT_16_16_16_16 = 0xc,
+ FMT_32_32_32 = 0xd,
+ FMT_32_32_32_32 = 0xe,
+ FMT_RESERVED_4 = 0xf,
+ FMT_5_6_5 = 0x10,
+ FMT_1_5_5_5 = 0x11,
+ FMT_5_5_5_1 = 0x12,
+ FMT_4_4_4_4 = 0x13,
+ FMT_8_24 = 0x14,
+ FMT_24_8 = 0x15,
+ FMT_X24_8_32_FLOAT = 0x16,
+ FMT_RESERVED_33 = 0x17,
+ FMT_11_11_10_FLOAT = 0x18,
+ FMT_16_FLOAT = 0x19,
+ FMT_32_FLOAT = 0x1a,
+ FMT_16_16_FLOAT = 0x1b,
+ FMT_8_24_FLOAT = 0x1c,
+ FMT_24_8_FLOAT = 0x1d,
+ FMT_32_32_FLOAT = 0x1e,
+ FMT_10_11_11_FLOAT = 0x1f,
+ FMT_16_16_16_16_FLOAT = 0x20,
+ FMT_3_3_2 = 0x21,
+ FMT_6_5_5 = 0x22,
+ FMT_32_32_32_32_FLOAT = 0x23,
+ FMT_RESERVED_36 = 0x24,
+ FMT_1 = 0x25,
+ FMT_1_REVERSED = 0x26,
+ FMT_GB_GR = 0x27,
+ FMT_BG_RG = 0x28,
+ FMT_32_AS_8 = 0x29,
+ FMT_32_AS_8_8 = 0x2a,
+ FMT_5_9_9_9_SHAREDEXP = 0x2b,
+ FMT_8_8_8 = 0x2c,
+ FMT_16_16_16 = 0x2d,
+ FMT_16_16_16_FLOAT = 0x2e,
+ FMT_4_4 = 0x2f,
+ FMT_32_32_32_FLOAT = 0x30,
+ FMT_BC1 = 0x31,
+ FMT_BC2 = 0x32,
+ FMT_BC3 = 0x33,
+ FMT_BC4 = 0x34,
+ FMT_BC5 = 0x35,
+ FMT_BC6 = 0x36,
+ FMT_BC7 = 0x37,
+ FMT_32_AS_32_32_32_32 = 0x38,
+ FMT_APC3 = 0x39,
+ FMT_APC4 = 0x3a,
+ FMT_APC5 = 0x3b,
+ FMT_APC6 = 0x3c,
+ FMT_APC7 = 0x3d,
+ FMT_CTX1 = 0x3e,
+ FMT_RESERVED_63 = 0x3f,
+} SurfaceFormat;
+typedef enum BUF_DATA_FORMAT {
+ BUF_DATA_FORMAT_INVALID = 0x0,
+ BUF_DATA_FORMAT_8 = 0x1,
+ BUF_DATA_FORMAT_16 = 0x2,
+ BUF_DATA_FORMAT_8_8 = 0x3,
+ BUF_DATA_FORMAT_32 = 0x4,
+ BUF_DATA_FORMAT_16_16 = 0x5,
+ BUF_DATA_FORMAT_10_11_11 = 0x6,
+ BUF_DATA_FORMAT_11_11_10 = 0x7,
+ BUF_DATA_FORMAT_10_10_10_2 = 0x8,
+ BUF_DATA_FORMAT_2_10_10_10 = 0x9,
+ BUF_DATA_FORMAT_8_8_8_8 = 0xa,
+ BUF_DATA_FORMAT_32_32 = 0xb,
+ BUF_DATA_FORMAT_16_16_16_16 = 0xc,
+ BUF_DATA_FORMAT_32_32_32 = 0xd,
+ BUF_DATA_FORMAT_32_32_32_32 = 0xe,
+ BUF_DATA_FORMAT_RESERVED_15 = 0xf,
+} BUF_DATA_FORMAT;
+typedef enum IMG_DATA_FORMAT {
+ IMG_DATA_FORMAT_INVALID = 0x0,
+ IMG_DATA_FORMAT_8 = 0x1,
+ IMG_DATA_FORMAT_16 = 0x2,
+ IMG_DATA_FORMAT_8_8 = 0x3,
+ IMG_DATA_FORMAT_32 = 0x4,
+ IMG_DATA_FORMAT_16_16 = 0x5,
+ IMG_DATA_FORMAT_10_11_11 = 0x6,
+ IMG_DATA_FORMAT_11_11_10 = 0x7,
+ IMG_DATA_FORMAT_10_10_10_2 = 0x8,
+ IMG_DATA_FORMAT_2_10_10_10 = 0x9,
+ IMG_DATA_FORMAT_8_8_8_8 = 0xa,
+ IMG_DATA_FORMAT_32_32 = 0xb,
+ IMG_DATA_FORMAT_16_16_16_16 = 0xc,
+ IMG_DATA_FORMAT_32_32_32 = 0xd,
+ IMG_DATA_FORMAT_32_32_32_32 = 0xe,
+ IMG_DATA_FORMAT_RESERVED_15 = 0xf,
+ IMG_DATA_FORMAT_5_6_5 = 0x10,
+ IMG_DATA_FORMAT_1_5_5_5 = 0x11,
+ IMG_DATA_FORMAT_5_5_5_1 = 0x12,
+ IMG_DATA_FORMAT_4_4_4_4 = 0x13,
+ IMG_DATA_FORMAT_8_24 = 0x14,
+ IMG_DATA_FORMAT_24_8 = 0x15,
+ IMG_DATA_FORMAT_X24_8_32 = 0x16,
+ IMG_DATA_FORMAT_RESERVED_23 = 0x17,
+ IMG_DATA_FORMAT_RESERVED_24 = 0x18,
+ IMG_DATA_FORMAT_RESERVED_25 = 0x19,
+ IMG_DATA_FORMAT_RESERVED_26 = 0x1a,
+ IMG_DATA_FORMAT_RESERVED_27 = 0x1b,
+ IMG_DATA_FORMAT_RESERVED_28 = 0x1c,
+ IMG_DATA_FORMAT_RESERVED_29 = 0x1d,
+ IMG_DATA_FORMAT_RESERVED_30 = 0x1e,
+ IMG_DATA_FORMAT_RESERVED_31 = 0x1f,
+ IMG_DATA_FORMAT_GB_GR = 0x20,
+ IMG_DATA_FORMAT_BG_RG = 0x21,
+ IMG_DATA_FORMAT_5_9_9_9 = 0x22,
+ IMG_DATA_FORMAT_BC1 = 0x23,
+ IMG_DATA_FORMAT_BC2 = 0x24,
+ IMG_DATA_FORMAT_BC3 = 0x25,
+ IMG_DATA_FORMAT_BC4 = 0x26,
+ IMG_DATA_FORMAT_BC5 = 0x27,
+ IMG_DATA_FORMAT_BC6 = 0x28,
+ IMG_DATA_FORMAT_BC7 = 0x29,
+ IMG_DATA_FORMAT_RESERVED_42 = 0x2a,
+ IMG_DATA_FORMAT_RESERVED_43 = 0x2b,
+ IMG_DATA_FORMAT_FMASK8_S2_F1 = 0x2c,
+ IMG_DATA_FORMAT_FMASK8_S4_F1 = 0x2d,
+ IMG_DATA_FORMAT_FMASK8_S8_F1 = 0x2e,
+ IMG_DATA_FORMAT_FMASK8_S2_F2 = 0x2f,
+ IMG_DATA_FORMAT_FMASK8_S4_F2 = 0x30,
+ IMG_DATA_FORMAT_FMASK8_S4_F4 = 0x31,
+ IMG_DATA_FORMAT_FMASK16_S16_F1 = 0x32,
+ IMG_DATA_FORMAT_FMASK16_S8_F2 = 0x33,
+ IMG_DATA_FORMAT_FMASK32_S16_F2 = 0x34,
+ IMG_DATA_FORMAT_FMASK32_S8_F4 = 0x35,
+ IMG_DATA_FORMAT_FMASK32_S8_F8 = 0x36,
+ IMG_DATA_FORMAT_FMASK64_S16_F4 = 0x37,
+ IMG_DATA_FORMAT_FMASK64_S16_F8 = 0x38,
+ IMG_DATA_FORMAT_4_4 = 0x39,
+ IMG_DATA_FORMAT_6_5_5 = 0x3a,
+ IMG_DATA_FORMAT_1 = 0x3b,
+ IMG_DATA_FORMAT_1_REVERSED = 0x3c,
+ IMG_DATA_FORMAT_32_AS_8 = 0x3d,
+ IMG_DATA_FORMAT_32_AS_8_8 = 0x3e,
+ IMG_DATA_FORMAT_32_AS_32_32_32_32 = 0x3f,
+} IMG_DATA_FORMAT;
+typedef enum BUF_NUM_FORMAT {
+ BUF_NUM_FORMAT_UNORM = 0x0,
+ BUF_NUM_FORMAT_SNORM = 0x1,
+ BUF_NUM_FORMAT_USCALED = 0x2,
+ BUF_NUM_FORMAT_SSCALED = 0x3,
+ BUF_NUM_FORMAT_UINT = 0x4,
+ BUF_NUM_FORMAT_SINT = 0x5,
+ BUF_NUM_FORMAT_RESERVED_6 = 0x6,
+ BUF_NUM_FORMAT_FLOAT = 0x7,
+} BUF_NUM_FORMAT;
+typedef enum IMG_NUM_FORMAT {
+ IMG_NUM_FORMAT_UNORM = 0x0,
+ IMG_NUM_FORMAT_SNORM = 0x1,
+ IMG_NUM_FORMAT_USCALED = 0x2,
+ IMG_NUM_FORMAT_SSCALED = 0x3,
+ IMG_NUM_FORMAT_UINT = 0x4,
+ IMG_NUM_FORMAT_SINT = 0x5,
+ IMG_NUM_FORMAT_RESERVED_6 = 0x6,
+ IMG_NUM_FORMAT_FLOAT = 0x7,
+ IMG_NUM_FORMAT_RESERVED_8 = 0x8,
+ IMG_NUM_FORMAT_SRGB = 0x9,
+ IMG_NUM_FORMAT_RESERVED_10 = 0xa,
+ IMG_NUM_FORMAT_RESERVED_11 = 0xb,
+ IMG_NUM_FORMAT_RESERVED_12 = 0xc,
+ IMG_NUM_FORMAT_RESERVED_13 = 0xd,
+ IMG_NUM_FORMAT_RESERVED_14 = 0xe,
+ IMG_NUM_FORMAT_RESERVED_15 = 0xf,
+} IMG_NUM_FORMAT;
+typedef enum TileType {
+ ARRAY_COLOR_TILE = 0x0,
+ ARRAY_DEPTH_TILE = 0x1,
+} TileType;
+typedef enum NonDispTilingOrder {
+ ADDR_SURF_MICRO_TILING_DISPLAY = 0x0,
+ ADDR_SURF_MICRO_TILING_NON_DISPLAY = 0x1,
+} NonDispTilingOrder;
+typedef enum MicroTileMode {
+ ADDR_SURF_DISPLAY_MICRO_TILING = 0x0,
+ ADDR_SURF_THIN_MICRO_TILING = 0x1,
+ ADDR_SURF_DEPTH_MICRO_TILING = 0x2,
+ ADDR_SURF_ROTATED_MICRO_TILING = 0x3,
+ ADDR_SURF_THICK_MICRO_TILING = 0x4,
+} MicroTileMode;
+typedef enum TileSplit {
+ ADDR_SURF_TILE_SPLIT_64B = 0x0,
+ ADDR_SURF_TILE_SPLIT_128B = 0x1,
+ ADDR_SURF_TILE_SPLIT_256B = 0x2,
+ ADDR_SURF_TILE_SPLIT_512B = 0x3,
+ ADDR_SURF_TILE_SPLIT_1KB = 0x4,
+ ADDR_SURF_TILE_SPLIT_2KB = 0x5,
+ ADDR_SURF_TILE_SPLIT_4KB = 0x6,
+} TileSplit;
+typedef enum SampleSplit {
+ ADDR_SURF_SAMPLE_SPLIT_1 = 0x0,
+ ADDR_SURF_SAMPLE_SPLIT_2 = 0x1,
+ ADDR_SURF_SAMPLE_SPLIT_4 = 0x2,
+ ADDR_SURF_SAMPLE_SPLIT_8 = 0x3,
+} SampleSplit;
+typedef enum PipeConfig {
+ ADDR_SURF_P2 = 0x0,
+ ADDR_SURF_P2_RESERVED0 = 0x1,
+ ADDR_SURF_P2_RESERVED1 = 0x2,
+ ADDR_SURF_P2_RESERVED2 = 0x3,
+ ADDR_SURF_P4_8x16 = 0x4,
+ ADDR_SURF_P4_16x16 = 0x5,
+ ADDR_SURF_P4_16x32 = 0x6,
+ ADDR_SURF_P4_32x32 = 0x7,
+ ADDR_SURF_P8_16x16_8x16 = 0x8,
+ ADDR_SURF_P8_16x32_8x16 = 0x9,
+ ADDR_SURF_P8_32x32_8x16 = 0xa,
+ ADDR_SURF_P8_16x32_16x16 = 0xb,
+ ADDR_SURF_P8_32x32_16x16 = 0xc,
+ ADDR_SURF_P8_32x32_16x32 = 0xd,
+ ADDR_SURF_P8_32x64_32x32 = 0xe,
+ ADDR_SURF_P8_RESERVED0 = 0xf,
+ ADDR_SURF_P16_32x32_8x16 = 0x10,
+ ADDR_SURF_P16_32x32_16x16 = 0x11,
+} PipeConfig;
+typedef enum NumBanks {
+ ADDR_SURF_2_BANK = 0x0,
+ ADDR_SURF_4_BANK = 0x1,
+ ADDR_SURF_8_BANK = 0x2,
+ ADDR_SURF_16_BANK = 0x3,
+} NumBanks;
+typedef enum BankWidth {
+ ADDR_SURF_BANK_WIDTH_1 = 0x0,
+ ADDR_SURF_BANK_WIDTH_2 = 0x1,
+ ADDR_SURF_BANK_WIDTH_4 = 0x2,
+ ADDR_SURF_BANK_WIDTH_8 = 0x3,
+} BankWidth;
+typedef enum BankHeight {
+ ADDR_SURF_BANK_HEIGHT_1 = 0x0,
+ ADDR_SURF_BANK_HEIGHT_2 = 0x1,
+ ADDR_SURF_BANK_HEIGHT_4 = 0x2,
+ ADDR_SURF_BANK_HEIGHT_8 = 0x3,
+} BankHeight;
+typedef enum BankWidthHeight {
+ ADDR_SURF_BANK_WH_1 = 0x0,
+ ADDR_SURF_BANK_WH_2 = 0x1,
+ ADDR_SURF_BANK_WH_4 = 0x2,
+ ADDR_SURF_BANK_WH_8 = 0x3,
+} BankWidthHeight;
+typedef enum MacroTileAspect {
+ ADDR_SURF_MACRO_ASPECT_1 = 0x0,
+ ADDR_SURF_MACRO_ASPECT_2 = 0x1,
+ ADDR_SURF_MACRO_ASPECT_4 = 0x2,
+ ADDR_SURF_MACRO_ASPECT_8 = 0x3,
+} MacroTileAspect;
+typedef enum GATCL1RequestType {
+ GATCL1_TYPE_NORMAL = 0x0,
+ GATCL1_TYPE_SHOOTDOWN = 0x1,
+ GATCL1_TYPE_BYPASS = 0x2,
+} GATCL1RequestType;
+typedef enum TCC_CACHE_POLICIES {
+ TCC_CACHE_POLICY_LRU = 0x0,
+ TCC_CACHE_POLICY_STREAM = 0x1,
+} TCC_CACHE_POLICIES;
+typedef enum MTYPE {
+ MTYPE_NC_NV = 0x0,
+ MTYPE_NC = 0x1,
+ MTYPE_CC = 0x2,
+ MTYPE_UC = 0x3,
+} MTYPE;
+typedef enum PERFMON_COUNTER_MODE {
+ PERFMON_COUNTER_MODE_ACCUM = 0x0,
+ PERFMON_COUNTER_MODE_ACTIVE_CYCLES = 0x1,
+ PERFMON_COUNTER_MODE_MAX = 0x2,
+ PERFMON_COUNTER_MODE_DIRTY = 0x3,
+ PERFMON_COUNTER_MODE_SAMPLE = 0x4,
+ PERFMON_COUNTER_MODE_CYCLES_SINCE_FIRST_EVENT = 0x5,
+ PERFMON_COUNTER_MODE_CYCLES_SINCE_LAST_EVENT = 0x6,
+ PERFMON_COUNTER_MODE_CYCLES_GE_HI = 0x7,
+ PERFMON_COUNTER_MODE_CYCLES_EQ_HI = 0x8,
+ PERFMON_COUNTER_MODE_INACTIVE_CYCLES = 0x9,
+ PERFMON_COUNTER_MODE_RESERVED = 0xf,
+} PERFMON_COUNTER_MODE;
+typedef enum PERFMON_SPM_MODE {
+ PERFMON_SPM_MODE_OFF = 0x0,
+ PERFMON_SPM_MODE_16BIT_CLAMP = 0x1,
+ PERFMON_SPM_MODE_16BIT_NO_CLAMP = 0x2,
+ PERFMON_SPM_MODE_32BIT_CLAMP = 0x3,
+ PERFMON_SPM_MODE_32BIT_NO_CLAMP = 0x4,
+ PERFMON_SPM_MODE_RESERVED_5 = 0x5,
+ PERFMON_SPM_MODE_RESERVED_6 = 0x6,
+ PERFMON_SPM_MODE_RESERVED_7 = 0x7,
+ PERFMON_SPM_MODE_TEST_MODE_0 = 0x8,
+ PERFMON_SPM_MODE_TEST_MODE_1 = 0x9,
+ PERFMON_SPM_MODE_TEST_MODE_2 = 0xa,
+} PERFMON_SPM_MODE;
+typedef enum SurfaceTiling {
+ ARRAY_LINEAR = 0x0,
+ ARRAY_TILED = 0x1,
+} SurfaceTiling;
+typedef enum SurfaceArray {
+ ARRAY_1D = 0x0,
+ ARRAY_2D = 0x1,
+ ARRAY_3D = 0x2,
+ ARRAY_3D_SLICE = 0x3,
+} SurfaceArray;
+typedef enum ColorArray {
+ ARRAY_2D_ALT_COLOR = 0x0,
+ ARRAY_2D_COLOR = 0x1,
+ ARRAY_3D_SLICE_COLOR = 0x3,
+} ColorArray;
+typedef enum DepthArray {
+ ARRAY_2D_ALT_DEPTH = 0x0,
+ ARRAY_2D_DEPTH = 0x1,
+} DepthArray;
+typedef enum ENUM_NUM_SIMD_PER_CU {
+ NUM_SIMD_PER_CU = 0x4,
+} ENUM_NUM_SIMD_PER_CU;
+typedef enum MEM_PWR_FORCE_CTRL {
+ NO_FORCE_REQUEST = 0x0,
+ FORCE_LIGHT_SLEEP_REQUEST = 0x1,
+ FORCE_DEEP_SLEEP_REQUEST = 0x2,
+ FORCE_SHUT_DOWN_REQUEST = 0x3,
+} MEM_PWR_FORCE_CTRL;
+typedef enum MEM_PWR_FORCE_CTRL2 {
+ NO_FORCE_REQ = 0x0,
+ FORCE_LIGHT_SLEEP_REQ = 0x1,
+} MEM_PWR_FORCE_CTRL2;
+typedef enum MEM_PWR_DIS_CTRL {
+ ENABLE_MEM_PWR_CTRL = 0x0,
+ DISABLE_MEM_PWR_CTRL = 0x1,
+} MEM_PWR_DIS_CTRL;
+typedef enum MEM_PWR_SEL_CTRL {
+ DYNAMIC_SHUT_DOWN_ENABLE = 0x0,
+ DYNAMIC_DEEP_SLEEP_ENABLE = 0x1,
+ DYNAMIC_LIGHT_SLEEP_ENABLE = 0x2,
+} MEM_PWR_SEL_CTRL;
+typedef enum MEM_PWR_SEL_CTRL2 {
+ DYNAMIC_DEEP_SLEEP_EN = 0x0,
+ DYNAMIC_LIGHT_SLEEP_EN = 0x1,
+} MEM_PWR_SEL_CTRL2;
+typedef enum HPD_INT_CONTROL_ACK {
+ HPD_INT_CONTROL_ACK_0 = 0x0,
+ HPD_INT_CONTROL_ACK_1 = 0x1,
+} HPD_INT_CONTROL_ACK;
+typedef enum HPD_INT_CONTROL_POLARITY {
+ HPD_INT_CONTROL_GEN_INT_ON_DISCON = 0x0,
+ HPD_INT_CONTROL_GEN_INT_ON_CON = 0x1,
+} HPD_INT_CONTROL_POLARITY;
+typedef enum HPD_INT_CONTROL_RX_INT_ACK {
+ HPD_INT_CONTROL_RX_INT_ACK_0 = 0x0,
+ HPD_INT_CONTROL_RX_INT_ACK_1 = 0x1,
+} HPD_INT_CONTROL_RX_INT_ACK;
+typedef enum DPDBG_EN {
+ DPDBG_DISABLE = 0x0,
+ DPDBG_ENABLE = 0x1,
+} DPDBG_EN;
+typedef enum DPDBG_INPUT_EN {
+ DPDBG_INPUT_DISABLE = 0x0,
+ DPDBG_INPUT_ENABLE = 0x1,
+} DPDBG_INPUT_EN;
+typedef enum DPDBG_ERROR_DETECTION_MODE {
+ DPDBG_ERROR_DETECTION_MODE_CSC = 0x0,
+ DPDBG_ERROR_DETECTION_MODE_RS_ENCODING = 0x1,
+} DPDBG_ERROR_DETECTION_MODE;
+typedef enum DPDBG_FIFO_OVERFLOW_INTERRUPT_MASK {
+ DPDBG_FIFO_OVERFLOW_INT_DISABLE = 0x0,
+ DPDBG_FIFO_OVERFLOW_INT_ENABLE = 0x1,
+} DPDBG_FIFO_OVERFLOW_INTERRUPT_MASK;
+typedef enum DPDBG_FIFO_OVERFLOW_INTERRUPT_TYPE {
+ DPDBG_FIFO_OVERFLOW_INT_LEVEL_BASED = 0x0,
+ DPDBG_FIFO_OVERFLOW_INT_PULSE_BASED = 0x1,
+} DPDBG_FIFO_OVERFLOW_INTERRUPT_TYPE;
+typedef enum DPDBG_FIFO_OVERFLOW_INTERRUPT_ACK {
+ DPDBG_FIFO_OVERFLOW_INT_NO_ACK = 0x0,
+ DPDBG_FIFO_OVERFLOW_INT_CLEAR = 0x1,
+} DPDBG_FIFO_OVERFLOW_INTERRUPT_ACK;
+typedef enum PM_ASSERT_RESET {
+ PM_ASSERT_RESET_0 = 0x0,
+ PM_ASSERT_RESET_1 = 0x1,
+} PM_ASSERT_RESET;
+typedef enum DAC_MUX_SELECT {
+ DAC_MUX_SELECT_DACA = 0x0,
+ DAC_MUX_SELECT_DACB = 0x1,
+} DAC_MUX_SELECT;
+typedef enum TMDS_DVO_MUX_SELECT {
+ TMDS_DVO_MUX_SELECT_B = 0x0,
+ TMDS_DVO_MUX_SELECT_G = 0x1,
+ TMDS_DVO_MUX_SELECT_R = 0x2,
+ TMDS_DVO_MUX_SELECT_RESERVED = 0x3,
+} TMDS_DVO_MUX_SELECT;
+typedef enum DACA_SOFT_RESET {
+ DACA_SOFT_RESET_0 = 0x0,
+ DACA_SOFT_RESET_1 = 0x1,
+} DACA_SOFT_RESET;
+typedef enum I2S0_SPDIF0_SOFT_RESET {
+ I2S0_SPDIF0_SOFT_RESET_0 = 0x0,
+ I2S0_SPDIF0_SOFT_RESET_1 = 0x1,
+} I2S0_SPDIF0_SOFT_RESET;
+typedef enum I2S1_SOFT_RESET {
+ I2S1_SOFT_RESET_0 = 0x0,
+ I2S1_SOFT_RESET_1 = 0x1,
+} I2S1_SOFT_RESET;
+typedef enum SPDIF1_SOFT_RESET {
+ SPDIF1_SOFT_RESET_0 = 0x0,
+ SPDIF1_SOFT_RESET_1 = 0x1,
+} SPDIF1_SOFT_RESET;
+typedef enum DB_CLK_SOFT_RESET {
+ DB_CLK_SOFT_RESET_0 = 0x0,
+ DB_CLK_SOFT_RESET_1 = 0x1,
+} DB_CLK_SOFT_RESET;
+typedef enum FMT0_SOFT_RESET {
+ FMT0_SOFT_RESET_0 = 0x0,
+ FMT0_SOFT_RESET_1 = 0x1,
+} FMT0_SOFT_RESET;
+typedef enum FMT1_SOFT_RESET {
+ FMT1_SOFT_RESET_0 = 0x0,
+ FMT1_SOFT_RESET_1 = 0x1,
+} FMT1_SOFT_RESET;
+typedef enum FMT2_SOFT_RESET {
+ FMT2_SOFT_RESET_0 = 0x0,
+ FMT2_SOFT_RESET_1 = 0x1,
+} FMT2_SOFT_RESET;
+typedef enum FMT3_SOFT_RESET {
+ FMT3_SOFT_RESET_0 = 0x0,
+ FMT3_SOFT_RESET_1 = 0x1,
+} FMT3_SOFT_RESET;
+typedef enum FMT4_SOFT_RESET {
+ FMT4_SOFT_RESET_0 = 0x0,
+ FMT4_SOFT_RESET_1 = 0x1,
+} FMT4_SOFT_RESET;
+typedef enum FMT5_SOFT_RESET {
+ FMT5_SOFT_RESET_0 = 0x0,
+ FMT5_SOFT_RESET_1 = 0x1,
+} FMT5_SOFT_RESET;
+typedef enum MVP_SOFT_RESET {
+ MVP_SOFT_RESET_0 = 0x0,
+ MVP_SOFT_RESET_1 = 0x1,
+} MVP_SOFT_RESET;
+typedef enum ABM_SOFT_RESET {
+ ABM_SOFT_RESET_0 = 0x0,
+ ABM_SOFT_RESET_1 = 0x1,
+} ABM_SOFT_RESET;
+typedef enum DVO_SOFT_RESET {
+ DVO_SOFT_RESET_0 = 0x0,
+ DVO_SOFT_RESET_1 = 0x1,
+} DVO_SOFT_RESET;
+typedef enum DIGA_FE_SOFT_RESET {
+ DIGA_FE_SOFT_RESET_0 = 0x0,
+ DIGA_FE_SOFT_RESET_1 = 0x1,
+} DIGA_FE_SOFT_RESET;
+typedef enum DIGA_BE_SOFT_RESET {
+ DIGA_BE_SOFT_RESET_0 = 0x0,
+ DIGA_BE_SOFT_RESET_1 = 0x1,
+} DIGA_BE_SOFT_RESET;
+typedef enum DIGB_FE_SOFT_RESET {
+ DIGB_FE_SOFT_RESET_0 = 0x0,
+ DIGB_FE_SOFT_RESET_1 = 0x1,
+} DIGB_FE_SOFT_RESET;
+typedef enum DIGB_BE_SOFT_RESET {
+ DIGB_BE_SOFT_RESET_0 = 0x0,
+ DIGB_BE_SOFT_RESET_1 = 0x1,
+} DIGB_BE_SOFT_RESET;
+typedef enum DIGC_FE_SOFT_RESET {
+ DIGC_FE_SOFT_RESET_0 = 0x0,
+ DIGC_FE_SOFT_RESET_1 = 0x1,
+} DIGC_FE_SOFT_RESET;
+typedef enum DIGC_BE_SOFT_RESET {
+ DIGC_BE_SOFT_RESET_0 = 0x0,
+ DIGC_BE_SOFT_RESET_1 = 0x1,
+} DIGC_BE_SOFT_RESET;
+typedef enum DIGD_FE_SOFT_RESET {
+ DIGD_FE_SOFT_RESET_0 = 0x0,
+ DIGD_FE_SOFT_RESET_1 = 0x1,
+} DIGD_FE_SOFT_RESET;
+typedef enum DIGD_BE_SOFT_RESET {
+ DIGD_BE_SOFT_RESET_0 = 0x0,
+ DIGD_BE_SOFT_RESET_1 = 0x1,
+} DIGD_BE_SOFT_RESET;
+typedef enum DIGE_FE_SOFT_RESET {
+ DIGE_FE_SOFT_RESET_0 = 0x0,
+ DIGE_FE_SOFT_RESET_1 = 0x1,
+} DIGE_FE_SOFT_RESET;
+typedef enum DIGE_BE_SOFT_RESET {
+ DIGE_BE_SOFT_RESET_0 = 0x0,
+ DIGE_BE_SOFT_RESET_1 = 0x1,
+} DIGE_BE_SOFT_RESET;
+typedef enum DIGF_FE_SOFT_RESET {
+ DIGF_FE_SOFT_RESET_0 = 0x0,
+ DIGF_FE_SOFT_RESET_1 = 0x1,
+} DIGF_FE_SOFT_RESET;
+typedef enum DIGF_BE_SOFT_RESET {
+ DIGF_BE_SOFT_RESET_0 = 0x0,
+ DIGF_BE_SOFT_RESET_1 = 0x1,
+} DIGF_BE_SOFT_RESET;
+typedef enum DIGG_FE_SOFT_RESET {
+ DIGG_FE_SOFT_RESET_0 = 0x0,
+ DIGG_FE_SOFT_RESET_1 = 0x1,
+} DIGG_FE_SOFT_RESET;
+typedef enum DIGG_BE_SOFT_RESET {
+ DIGG_BE_SOFT_RESET_0 = 0x0,
+ DIGG_BE_SOFT_RESET_1 = 0x1,
+} DIGG_BE_SOFT_RESET;
+typedef enum DPDBG_SOFT_RESET {
+ DPDBG_SOFT_RESET_0 = 0x0,
+ DPDBG_SOFT_RESET_1 = 0x1,
+} DPDBG_SOFT_RESET;
+typedef enum DIGLPA_FE_SOFT_RESET {
+ DIGLPA_FE_SOFT_RESET_0 = 0x0,
+ DIGLPA_FE_SOFT_RESET_1 = 0x1,
+} DIGLPA_FE_SOFT_RESET;
+typedef enum DIGLPA_BE_SOFT_RESET {
+ DIGLPA_BE_SOFT_RESET_0 = 0x0,
+ DIGLPA_BE_SOFT_RESET_1 = 0x1,
+} DIGLPA_BE_SOFT_RESET;
+typedef enum DIGLPB_FE_SOFT_RESET {
+ DIGLPB_FE_SOFT_RESET_0 = 0x0,
+ DIGLPB_FE_SOFT_RESET_1 = 0x1,
+} DIGLPB_FE_SOFT_RESET;
+typedef enum DIGLPB_BE_SOFT_RESET {
+ DIGLPB_BE_SOFT_RESET_0 = 0x0,
+ DIGLPB_BE_SOFT_RESET_1 = 0x1,
+} DIGLPB_BE_SOFT_RESET;
+typedef enum GENERICA_STEREOSYNC_SEL {
+ GENERICA_STEREOSYNC_SEL_D1 = 0x0,
+ GENERICA_STEREOSYNC_SEL_D2 = 0x1,
+ GENERICA_STEREOSYNC_SEL_D3 = 0x2,
+ GENERICA_STEREOSYNC_SEL_D4 = 0x3,
+ GENERICA_STEREOSYNC_SEL_D5 = 0x4,
+ GENERICA_STEREOSYNC_SEL_D6 = 0x5,
+ GENERICA_STEREOSYNC_SEL_RESERVED = 0x6,
+} GENERICA_STEREOSYNC_SEL;
+typedef enum GENERICB_STEREOSYNC_SEL {
+ GENERICB_STEREOSYNC_SEL_D1 = 0x0,
+ GENERICB_STEREOSYNC_SEL_D2 = 0x1,
+ GENERICB_STEREOSYNC_SEL_D3 = 0x2,
+ GENERICB_STEREOSYNC_SEL_D4 = 0x3,
+ GENERICB_STEREOSYNC_SEL_D5 = 0x4,
+ GENERICB_STEREOSYNC_SEL_D6 = 0x5,
+ GENERICB_STEREOSYNC_SEL_RESERVED = 0x6,
+} GENERICB_STEREOSYNC_SEL;
+typedef enum DCO_DBG_BLOCK_SEL {
+ DCO_DBG_BLOCK_SEL_DCO = 0x0,
+ DCO_DBG_BLOCK_SEL_ABM = 0x1,
+ DCO_DBG_BLOCK_SEL_DVO = 0x2,
+ DCO_DBG_BLOCK_SEL_DAC = 0x3,
+ DCO_DBG_BLOCK_SEL_MVP = 0x4,
+ DCO_DBG_BLOCK_SEL_FMT0 = 0x5,
+ DCO_DBG_BLOCK_SEL_FMT1 = 0x6,
+ DCO_DBG_BLOCK_SEL_FMT2 = 0x7,
+ DCO_DBG_BLOCK_SEL_FMT3 = 0x8,
+ DCO_DBG_BLOCK_SEL_FMT4 = 0x9,
+ DCO_DBG_BLOCK_SEL_FMT5 = 0xa,
+ DCO_DBG_BLOCK_SEL_DIGFE_A = 0xb,
+ DCO_DBG_BLOCK_SEL_DIGFE_B = 0xc,
+ DCO_DBG_BLOCK_SEL_DIGFE_C = 0xd,
+ DCO_DBG_BLOCK_SEL_DIGFE_D = 0xe,
+ DCO_DBG_BLOCK_SEL_DIGFE_E = 0xf,
+ DCO_DBG_BLOCK_SEL_DIGFE_F = 0x10,
+ DCO_DBG_BLOCK_SEL_DIGFE_G = 0x11,
+ DCO_DBG_BLOCK_SEL_DIGA = 0x12,
+ DCO_DBG_BLOCK_SEL_DIGB = 0x13,
+ DCO_DBG_BLOCK_SEL_DIGC = 0x14,
+ DCO_DBG_BLOCK_SEL_DIGD = 0x15,
+ DCO_DBG_BLOCK_SEL_DIGE = 0x16,
+ DCO_DBG_BLOCK_SEL_DIGF = 0x17,
+ DCO_DBG_BLOCK_SEL_DIGG = 0x18,
+ DCO_DBG_BLOCK_SEL_DPFE_A = 0x19,
+ DCO_DBG_BLOCK_SEL_DPFE_B = 0x1a,
+ DCO_DBG_BLOCK_SEL_DPFE_C = 0x1b,
+ DCO_DBG_BLOCK_SEL_DPFE_D = 0x1c,
+ DCO_DBG_BLOCK_SEL_DPFE_E = 0x1d,
+ DCO_DBG_BLOCK_SEL_DPFE_F = 0x1e,
+ DCO_DBG_BLOCK_SEL_DPFE_G = 0x1f,
+ DCO_DBG_BLOCK_SEL_DPA = 0x20,
+ DCO_DBG_BLOCK_SEL_DPB = 0x21,
+ DCO_DBG_BLOCK_SEL_DPC = 0x22,
+ DCO_DBG_BLOCK_SEL_DPD = 0x23,
+ DCO_DBG_BLOCK_SEL_DPE = 0x24,
+ DCO_DBG_BLOCK_SEL_DPF = 0x25,
+ DCO_DBG_BLOCK_SEL_DPG = 0x26,
+ DCO_DBG_BLOCK_SEL_AUX0 = 0x27,
+ DCO_DBG_BLOCK_SEL_AUX1 = 0x28,
+ DCO_DBG_BLOCK_SEL_AUX2 = 0x29,
+ DCO_DBG_BLOCK_SEL_AUX3 = 0x2a,
+ DCO_DBG_BLOCK_SEL_AUX4 = 0x2b,
+ DCO_DBG_BLOCK_SEL_AUX5 = 0x2c,
+ DCO_DBG_BLOCK_SEL_PERFMON_DCO = 0x2d,
+ DCO_DBG_BLOCK_SEL_AUDIO_OUT = 0x2e,
+ DCO_DBG_BLOCK_SEL_DIGLPFEA = 0x2f,
+ DCO_DBG_BLOCK_SEL_DIGLPFEB = 0x30,
+ DCO_DBG_BLOCK_SEL_DIGLPA = 0x31,
+ DCO_DBG_BLOCK_SEL_DIGLPB = 0x32,
+ DCO_DBG_BLOCK_SEL_DPLPFEA = 0x33,
+ DCO_DBG_BLOCK_SEL_DPLPFEB = 0x34,
+ DCO_DBG_BLOCK_SEL_DPLPA = 0x35,
+ DCO_DBG_BLOCK_SEL_DPLPB = 0x36,
+} DCO_DBG_BLOCK_SEL;
+typedef enum DCO_DBG_CLOCK_SEL {
+ DCO_DBG_CLOCK_SEL_DISPCLK = 0x0,
+ DCO_DBG_CLOCK_SEL_SCLK = 0x1,
+ DCO_DBG_CLOCK_SEL_MVPCLK = 0x2,
+ DCO_DBG_CLOCK_SEL_DVOCLK = 0x3,
+ DCO_DBG_CLOCK_SEL_DACCLK = 0x4,
+ DCO_DBG_CLOCK_SEL_REFCLK = 0x5,
+ DCO_DBG_CLOCK_SEL_SYMCLKA = 0x6,
+ DCO_DBG_CLOCK_SEL_SYMCLKB = 0x7,
+ DCO_DBG_CLOCK_SEL_SYMCLKC = 0x8,
+ DCO_DBG_CLOCK_SEL_SYMCLKD = 0x9,
+ DCO_DBG_CLOCK_SEL_SYMCLKE = 0xa,
+ DCO_DBG_CLOCK_SEL_SYMCLKF = 0xb,
+ DCO_DBG_CLOCK_SEL_SYMCLKG = 0xc,
+ DCO_DBG_CLOCK_SEL_RESERVED = 0xd,
+ DCO_DBG_CLOCK_SEL_AM0CLK = 0xe,
+ DCO_DBG_CLOCK_SEL_AM1CLK = 0xf,
+ DCO_DBG_CLOCK_SEL_AM2CLK = 0x10,
+ DCO_DBG_CLOCK_SEL_SYMCLKLPA = 0x11,
+ DCO_DBG_CLOCK_SEL_SYMCLKLPB = 0x12,
+} DCO_DBG_CLOCK_SEL;
+typedef enum DCO_HDMI_RXSTATUS_TIMER_CONTROL_DCO_HDMI_RXSTATUS_TIMER_TYPE {
+ DCO_HDMI_RXSTATUS_TIMER_TYPE_LEVEL = 0x0,
+ DCO_HDMI_RXSTATUS_TIMER_TYPE_PULSE = 0x1,
+} DCO_HDMI_RXSTATUS_TIMER_CONTROL_DCO_HDMI_RXSTATUS_TIMER_TYPE;
+typedef enum FMT420_MEMORY_SOURCE_SEL {
+ FMT420_MEMORY_SOURCE_SEL_FMT0 = 0x0,
+ FMT420_MEMORY_SOURCE_SEL_FMT1 = 0x1,
+ FMT420_MEMORY_SOURCE_SEL_FMT2 = 0x2,
+ FMT420_MEMORY_SOURCE_SEL_FMT3 = 0x3,
+ FMT420_MEMORY_SOURCE_SEL_FMT4 = 0x4,
+ FMT420_MEMORY_SOURCE_SEL_FMT5 = 0x5,
+ FMT420_MEMORY_SOURCE_SEL_FMT_RESERVED = 0x6,
+} FMT420_MEMORY_SOURCE_SEL;
+typedef enum DOUT_I2C_CONTROL_GO {
+ DOUT_I2C_CONTROL_STOP_TRANSFER = 0x0,
+ DOUT_I2C_CONTROL_START_TRANSFER = 0x1,
+} DOUT_I2C_CONTROL_GO;
+typedef enum DOUT_I2C_CONTROL_SOFT_RESET {
+ DOUT_I2C_CONTROL_NOT_RESET_I2C_CONTROLLER = 0x0,
+ DOUT_I2C_CONTROL_RESET_I2C_CONTROLLER = 0x1,
+} DOUT_I2C_CONTROL_SOFT_RESET;
+typedef enum DOUT_I2C_CONTROL_SEND_RESET {
+ DOUT_I2C_CONTROL__NOT_SEND_RESET = 0x0,
+ DOUT_I2C_CONTROL__SEND_RESET = 0x1,
+} DOUT_I2C_CONTROL_SEND_RESET;
+typedef enum DOUT_I2C_CONTROL_SW_STATUS_RESET {
+ DOUT_I2C_CONTROL_NOT_RESET_SW_STATUS = 0x0,
+ DOUT_I2C_CONTROL_RESET_SW_STATUS = 0x1,
+} DOUT_I2C_CONTROL_SW_STATUS_RESET;
+typedef enum DOUT_I2C_CONTROL_DDC_SELECT {
+ DOUT_I2C_CONTROL_SELECT_DDC1 = 0x0,
+ DOUT_I2C_CONTROL_SELECT_DDC2 = 0x1,
+ DOUT_I2C_CONTROL_SELECT_DDC3 = 0x2,
+ DOUT_I2C_CONTROL_SELECT_DDC4 = 0x3,
+ DOUT_I2C_CONTROL_SELECT_DDC5 = 0x4,
+ DOUT_I2C_CONTROL_SELECT_DDC6 = 0x5,
+ DOUT_I2C_CONTROL_SELECT_DDCVGA = 0x6,
+} DOUT_I2C_CONTROL_DDC_SELECT;
+typedef enum DOUT_I2C_CONTROL_TRANSACTION_COUNT {
+ DOUT_I2C_CONTROL_TRANS0 = 0x0,
+ DOUT_I2C_CONTROL_TRANS0_TRANS1 = 0x1,
+ DOUT_I2C_CONTROL_TRANS0_TRANS1_TRANS2 = 0x2,
+ DOUT_I2C_CONTROL_TRANS0_TRANS1_TRANS2_TRANS3 = 0x3,
+} DOUT_I2C_CONTROL_TRANSACTION_COUNT;
+typedef enum DOUT_I2C_CONTROL_DBG_REF_SEL {
+ DOUT_I2C_CONTROL_NORMAL_DEBUG = 0x0,
+ DOUT_I2C_CONTROL_FAST_REFERENCE_DEBUG = 0x1,
+} DOUT_I2C_CONTROL_DBG_REF_SEL;
+typedef enum DOUT_I2C_ARBITRATION_SW_PRIORITY {
+ DOUT_I2C_ARBITRATION_SW_PRIORITY_NORMAL = 0x0,
+ DOUT_I2C_ARBITRATION_SW_PRIORITY_HIGH = 0x1,
+ DOUT_I2C_ARBITRATION_SW_PRIORITY_0_RESERVED = 0x2,
+ DOUT_I2C_ARBITRATION_SW_PRIORITY_1_RESERVED = 0x3,
+} DOUT_I2C_ARBITRATION_SW_PRIORITY;
+typedef enum DOUT_I2C_ARBITRATION_NO_QUEUED_SW_GO {
+ DOUT_I2C_ARBITRATION_SW_QUEUE_ENABLED = 0x0,
+ DOUT_I2C_ARBITRATION_SW_QUEUE_DISABLED = 0x1,
+} DOUT_I2C_ARBITRATION_NO_QUEUED_SW_GO;
+typedef enum DOUT_I2C_ARBITRATION_ABORT_XFER {
+ DOUT_I2C_ARBITRATION_NOT_ABORT_CURRENT_TRANSFER = 0x0,
+ DOUT_I2C_ARBITRATION_ABORT_CURRENT_TRANSFER = 0x1,
+} DOUT_I2C_ARBITRATION_ABORT_XFER;
+typedef enum DOUT_I2C_ARBITRATION_USE_I2C_REG_REQ {
+ DOUT_I2C_ARBITRATION__NOT_USE_I2C_REG_REQ = 0x0,
+ DOUT_I2C_ARBITRATION__USE_I2C_REG_REQ = 0x1,
+} DOUT_I2C_ARBITRATION_USE_I2C_REG_REQ;
+typedef enum DOUT_I2C_ARBITRATION_DONE_USING_I2C_REG {
+ DOUT_I2C_ARBITRATION_DONE__NOT_USING_I2C_REG = 0x0,
+ DOUT_I2C_ARBITRATION_DONE__USING_I2C_REG = 0x1,
+} DOUT_I2C_ARBITRATION_DONE_USING_I2C_REG;
+typedef enum DOUT_I2C_ACK {
+ DOUT_I2C_NO_ACK = 0x0,
+ DOUT_I2C_ACK_TO_CLEAN = 0x1,
+} DOUT_I2C_ACK;
+typedef enum DOUT_I2C_DDC_SPEED_THRESHOLD {
+ DOUT_I2C_DDC_SPEED_THRESHOLD_BIG_THAN_ZERO = 0x0,
+ DOUT_I2C_DDC_SPEED_THRESHOLD_QUATER_OF_TOTAL_SAMPLE= 0x1,
+ DOUT_I2C_DDC_SPEED_THRESHOLD_HALF_OF_TOTAL_SAMPLE= 0x2,
+ DOUT_I2C_DDC_SPEED_THRESHOLD_THREE_QUATERS_OF_TOTAL_SAMPLE= 0x3,
+} DOUT_I2C_DDC_SPEED_THRESHOLD;
+typedef enum DOUT_I2C_DDC_SETUP_DATA_DRIVE_EN {
+ DOUT_I2C_DDC_SETUP_DATA_DRIVE_BY_EXTERNAL_RESISTOR= 0x0,
+ DOUT_I2C_DDC_SETUP_I2C_PAD_DRIVE_SDA = 0x1,
+} DOUT_I2C_DDC_SETUP_DATA_DRIVE_EN;
+typedef enum DOUT_I2C_DDC_SETUP_DATA_DRIVE_SEL {
+ DOUT_I2C_DDC_SETUP_DATA_DRIVE_FOR_10MCLKS = 0x0,
+ DOUT_I2C_DDC_SETUP_DATA_DRIVE_FOR_20MCLKS = 0x1,
+} DOUT_I2C_DDC_SETUP_DATA_DRIVE_SEL;
+typedef enum DOUT_I2C_DDC_SETUP_EDID_DETECT_MODE {
+ DOUT_I2C_DDC_SETUP_EDID_DETECT_CONNECT = 0x0,
+ DOUT_I2C_DDC_SETUP_EDID_DETECT_DISCONNECT = 0x1,
+} DOUT_I2C_DDC_SETUP_EDID_DETECT_MODE;
+typedef enum DOUT_I2C_DDC_SETUP_CLK_DRIVE_EN {
+ DOUT_I2C_DDC_SETUP_CLK_DRIVE_BY_EXTERNAL_RESISTOR= 0x0,
+ DOUT_I2C_DDC_SETUP_I2C_PAD_DRIVE_SCL = 0x1,
+} DOUT_I2C_DDC_SETUP_CLK_DRIVE_EN;
+typedef enum DOUT_I2C_TRANSACTION_STOP_ON_NACK {
+ DOUT_I2C_TRANSACTION_STOP_CURRENT_TRANS = 0x0,
+ DOUT_I2C_TRANSACTION_STOP_ALL_TRANS = 0x1,
+} DOUT_I2C_TRANSACTION_STOP_ON_NACK;
+typedef enum DOUT_I2C_DATA_INDEX_WRITE {
+ DOUT_I2C_DATA__NOT_INDEX_WRITE = 0x0,
+ DOUT_I2C_DATA__INDEX_WRITE = 0x1,
+} DOUT_I2C_DATA_INDEX_WRITE;
+typedef enum DOUT_I2C_EDID_DETECT_CTRL_SEND_RESET {
+ DOUT_I2C_EDID_NOT_SEND_RESET_BEFORE_EDID_READ_TRACTION= 0x0,
+ DOUT_I2C_EDID_SEND_RESET_BEFORE_EDID_READ_TRACTION= 0x1,
+} DOUT_I2C_EDID_DETECT_CTRL_SEND_RESET;
+typedef enum DOUT_I2C_READ_REQUEST_INTERRUPT_TYPE {
+ DOUT_I2C_READ_REQUEST_INTERRUPT_TYPE__LEVEL = 0x0,
+ DOUT_I2C_READ_REQUEST_INTERRUPT_TYPE__PULSE = 0x1,
+} DOUT_I2C_READ_REQUEST_INTERRUPT_TYPE;
+typedef enum BLNDV_CONTROL_BLND_MODE {
+ BLNDV_CONTROL_BLND_MODE_CURRENT_PIPE_ONLY = 0x0,
+ BLNDV_CONTROL_BLND_MODE_OTHER_PIPE_ONLY = 0x1,
+ BLNDV_CONTROL_BLND_MODE_ALPHA_BLENDING_MODE = 0x2,
+ BLNDV_CONTROL_BLND_MODE_OTHER_STEREO_TYPE = 0x3,
+} BLNDV_CONTROL_BLND_MODE;
+typedef enum BLNDV_CONTROL_BLND_STEREO_TYPE {
+ BLNDV_CONTROL_BLND_STEREO_TYPE_NON_SINGLE_PIPE_STEREO= 0x0,
+ BLNDV_CONTROL_BLND_STEREO_TYPE_SIDE_BY_SIDE_SINGLE_PIPE_STEREO= 0x1,
+ BLNDV_CONTROL_BLND_STEREO_TYPE_TOP_BOTTOM_SINGLE_PIPE_STEREO= 0x2,
+ BLNDV_CONTROL_BLND_STEREO_TYPE_UNUSED = 0x3,
+} BLNDV_CONTROL_BLND_STEREO_TYPE;
+typedef enum BLNDV_CONTROL_BLND_STEREO_POLARITY {
+ BLNDV_CONTROL_BLND_STEREO_POLARITY_LOW = 0x0,
+ BLNDV_CONTROL_BLND_STEREO_POLARITY_HIGH = 0x1,
+} BLNDV_CONTROL_BLND_STEREO_POLARITY;
+typedef enum BLNDV_CONTROL_BLND_FEEDTHROUGH_EN {
+ BLNDV_CONTROL_BLND_FEEDTHROUGH_EN_FALSE = 0x0,
+ BLNDV_CONTROL_BLND_FEEDTHROUGH_EN_TRUE = 0x1,
+} BLNDV_CONTROL_BLND_FEEDTHROUGH_EN;
+typedef enum BLNDV_CONTROL_BLND_ALPHA_MODE {
+ BLNDV_CONTROL_BLND_ALPHA_MODE_CURRENT_PIXEL_ALPHA= 0x0,
+ BLNDV_CONTROL_BLND_ALPHA_MODE_PIXEL_ALPHA_COMBINED_GLOBAL_GAIN= 0x1,
+ BLNDV_CONTROL_BLND_ALPHA_MODE_GLOBAL_ALPHA_ONLY = 0x2,
+ BLNDV_CONTROL_BLND_ALPHA_MODE_UNUSED = 0x3,
+} BLNDV_CONTROL_BLND_ALPHA_MODE;
+typedef enum BLNDV_CONTROL_BLND_ACTIVE_OVERLAP_ONLY {
+ BLNDV_CONTROL_BLND_ACTIVE_OVERLAP_ONLY_FALSE = 0x0,
+ BLNDV_CONTROL_BLND_ACTIVE_OVERLAP_ONLY_TRUE = 0x1,
+} BLNDV_CONTROL_BLND_ACTIVE_OVERLAP_ONLY;
+typedef enum BLNDV_CONTROL_BLND_MULTIPLIED_MODE {
+ BLNDV_CONTROL_BLND_MULTIPLIED_MODE_FALSE = 0x0,
+ BLNDV_CONTROL_BLND_MULTIPLIED_MODE_TRUE = 0x1,
+} BLNDV_CONTROL_BLND_MULTIPLIED_MODE;
+typedef enum BLNDV_SM_CONTROL2_SM_MODE {
+ BLNDV_SM_CONTROL2_SM_MODE_SINGLE_PLANE = 0x0,
+ BLNDV_SM_CONTROL2_SM_MODE_ROW_SUBSAMPLING = 0x2,
+ BLNDV_SM_CONTROL2_SM_MODE_COLUMN_SUBSAMPLING = 0x4,
+ BLNDV_SM_CONTROL2_SM_MODE_CHECKERBOARD_SUBSAMPLING= 0x6,
+} BLNDV_SM_CONTROL2_SM_MODE;
+typedef enum BLNDV_SM_CONTROL2_SM_FRAME_ALTERNATE {
+ BLNDV_SM_CONTROL2_SM_FRAME_ALTERNATE_FALSE = 0x0,
+ BLNDV_SM_CONTROL2_SM_FRAME_ALTERNATE_TRUE = 0x1,
+} BLNDV_SM_CONTROL2_SM_FRAME_ALTERNATE;
+typedef enum BLNDV_SM_CONTROL2_SM_FIELD_ALTERNATE {
+ BLNDV_SM_CONTROL2_SM_FIELD_ALTERNATE_FALSE = 0x0,
+ BLNDV_SM_CONTROL2_SM_FIELD_ALTERNATE_TRUE = 0x1,
+} BLNDV_SM_CONTROL2_SM_FIELD_ALTERNATE;
+typedef enum BLNDV_SM_CONTROL2_SM_FORCE_NEXT_FRAME_POL {
+ BLNDV_SM_CONTROL2_SM_FORCE_NEXT_FRAME_POL_NO_FORCE= 0x0,
+ BLNDV_SM_CONTROL2_SM_FORCE_NEXT_FRAME_POL_RESERVED= 0x1,
+ BLNDV_SM_CONTROL2_SM_FORCE_NEXT_FRAME_POL_FORCE_LOW= 0x2,
+ BLNDV_SM_CONTROL2_SM_FORCE_NEXT_FRAME_POL_FORCE_HIGH= 0x3,
+} BLNDV_SM_CONTROL2_SM_FORCE_NEXT_FRAME_POL;
+typedef enum BLNDV_SM_CONTROL2_SM_FORCE_NEXT_TOP_POL {
+ BLNDV_SM_CONTROL2_SM_FORCE_NEXT_TOP_POL_NO_FORCE = 0x0,
+ BLNDV_SM_CONTROL2_SM_FORCE_NEXT_TOP_POL_RESERVED = 0x1,
+ BLNDV_SM_CONTROL2_SM_FORCE_NEXT_TOP_POL_FORCE_LOW= 0x2,
+ BLNDV_SM_CONTROL2_SM_FORCE_NEXT_TOP_POL_FORCE_HIGH= 0x3,
+} BLNDV_SM_CONTROL2_SM_FORCE_NEXT_TOP_POL;
+typedef enum BLNDV_CONTROL2_PTI_ENABLE {
+ BLNDV_CONTROL2_PTI_ENABLE_FALSE = 0x0,
+ BLNDV_CONTROL2_PTI_ENABLE_TRUE = 0x1,
+} BLNDV_CONTROL2_PTI_ENABLE;
+typedef enum BLNDV_CONTROL2_BLND_SUPERAA_DEGAMMA_EN {
+ BLNDV_CONTROL2_BLND_SUPERAA_DEGAMMA_EN_FALSE = 0x0,
+ BLNDV_CONTROL2_BLND_SUPERAA_DEGAMMA_EN_TRUE = 0x1,
+} BLNDV_CONTROL2_BLND_SUPERAA_DEGAMMA_EN;
+typedef enum BLNDV_CONTROL2_BLND_SUPERAA_REGAMMA_EN {
+ BLNDV_CONTROL2_BLND_SUPERAA_REGAMMA_EN_FALSE = 0x0,
+ BLNDV_CONTROL2_BLND_SUPERAA_REGAMMA_EN_TRUE = 0x1,
+} BLNDV_CONTROL2_BLND_SUPERAA_REGAMMA_EN;
+typedef enum BLNDV_UNDERFLOW_INTERRUPT_BLND_UNDERFLOW_INT_ACK {
+ BLNDV_UNDERFLOW_INTERRUPT_BLND_UNDERFLOW_INT_ACK_FALSE= 0x0,
+ BLNDV_UNDERFLOW_INTERRUPT_BLND_UNDERFLOW_INT_ACK_TRUE= 0x1,
+} BLNDV_UNDERFLOW_INTERRUPT_BLND_UNDERFLOW_INT_ACK;
+typedef enum BLNDV_UNDERFLOW_INTERRUPT_BLND_UNDERFLOW_INT_MASK {
+ BLNDV_UNDERFLOW_INTERRUPT_BLND_UNDERFLOW_INT_MASK_FALSE= 0x0,
+ BLNDV_UNDERFLOW_INTERRUPT_BLND_UNDERFLOW_INT_MASK_TRUE= 0x1,
+} BLNDV_UNDERFLOW_INTERRUPT_BLND_UNDERFLOW_INT_MASK;
+typedef enum BLNDV_V_UPDATE_LOCK_BLND_DCP_GRPH_V_UPDATE_LOCK {
+ BLNDV_V_UPDATE_LOCK_BLND_DCP_GRPH_V_UPDATE_LOCK_FALSE= 0x0,
+ BLNDV_V_UPDATE_LOCK_BLND_DCP_GRPH_V_UPDATE_LOCK_TRUE= 0x1,
+} BLNDV_V_UPDATE_LOCK_BLND_DCP_GRPH_V_UPDATE_LOCK;
+typedef enum BLNDV_V_UPDATE_LOCK_BLND_DCP_GRPH_SURF_V_UPDATE_LOCK {
+ BLNDV_V_UPDATE_LOCK_BLND_DCP_GRPH_SURF_V_UPDATE_LOCK_FALSE= 0x0,
+ BLNDV_V_UPDATE_LOCK_BLND_DCP_GRPH_SURF_V_UPDATE_LOCK_TRUE= 0x1,
+} BLNDV_V_UPDATE_LOCK_BLND_DCP_GRPH_SURF_V_UPDATE_LOCK;
+typedef enum BLNDV_V_UPDATE_LOCK_BLND_DCP_CUR_V_UPDATE_LOCK {
+ BLNDV_V_UPDATE_LOCK_BLND_DCP_CUR_V_UPDATE_LOCK_FALSE= 0x0,
+ BLNDV_V_UPDATE_LOCK_BLND_DCP_CUR_V_UPDATE_LOCK_TRUE= 0x1,
+} BLNDV_V_UPDATE_LOCK_BLND_DCP_CUR_V_UPDATE_LOCK;
+typedef enum BLNDV_V_UPDATE_LOCK_BLND_DCP_CUR2_V_UPDATE_LOCK {
+ BLNDV_V_UPDATE_LOCK_BLND_DCP_CUR2_V_UPDATE_LOCK_FALSE= 0x0,
+ BLNDV_V_UPDATE_LOCK_BLND_DCP_CUR2_V_UPDATE_LOCK_TRUE= 0x1,
+} BLNDV_V_UPDATE_LOCK_BLND_DCP_CUR2_V_UPDATE_LOCK;
+typedef enum BLNDV_V_UPDATE_LOCK_BLND_SCL_V_UPDATE_LOCK {
+ BLNDV_V_UPDATE_LOCK_BLND_SCL_V_UPDATE_LOCK_FALSE = 0x0,
+ BLNDV_V_UPDATE_LOCK_BLND_SCL_V_UPDATE_LOCK_TRUE = 0x1,
+} BLNDV_V_UPDATE_LOCK_BLND_SCL_V_UPDATE_LOCK;
+typedef enum BLNDV_V_UPDATE_LOCK_BLND_BLND_V_UPDATE_LOCK {
+ BLNDV_V_UPDATE_LOCK_BLND_BLND_V_UPDATE_LOCK_FALSE= 0x0,
+ BLNDV_V_UPDATE_LOCK_BLND_BLND_V_UPDATE_LOCK_TRUE = 0x1,
+} BLNDV_V_UPDATE_LOCK_BLND_BLND_V_UPDATE_LOCK;
+typedef enum BLNDV_V_UPDATE_LOCK_BLND_V_UPDATE_LOCK_MODE {
+ BLNDV_V_UPDATE_LOCK_BLND_V_UPDATE_LOCK_MODE_FALSE= 0x0,
+ BLNDV_V_UPDATE_LOCK_BLND_V_UPDATE_LOCK_MODE_TRUE = 0x1,
+} BLNDV_V_UPDATE_LOCK_BLND_V_UPDATE_LOCK_MODE;
+typedef enum BLNDV_DEBUG_BLND_CNV_MUX_SELECT {
+ BLNDV_DEBUG_BLND_CNV_MUX_SELECT_LOW = 0x0,
+ BLNDV_DEBUG_BLND_CNV_MUX_SELECT_HIGH = 0x1,
+} BLNDV_DEBUG_BLND_CNV_MUX_SELECT;
+typedef enum BLNDV_TEST_DEBUG_INDEX_BLND_TEST_DEBUG_WRITE_EN {
+ BLNDV_TEST_DEBUG_INDEX_BLND_TEST_DEBUG_WRITE_EN_FALSE= 0x0,
+ BLNDV_TEST_DEBUG_INDEX_BLND_TEST_DEBUG_WRITE_EN_TRUE= 0x1,
+} BLNDV_TEST_DEBUG_INDEX_BLND_TEST_DEBUG_WRITE_EN;
+typedef enum DPCSTX_DBG_CFGCLK_SEL {
+ DPCSTX_DBG_CFGCLK_SEL_DC_DPCS_INF = 0x0,
+ DPCSTX_DBG_CFGCLK_SEL_DPCS_BPHY_INF = 0x1,
+ DPCSTX_DBG_CFGCLK_SEL_CBUS_SLAVE = 0x2,
+ DPCSTX_DBG_CFGCLK_SEL_CBUS_MASTER = 0x3,
+} DPCSTX_DBG_CFGCLK_SEL;
+typedef enum DPCSTX_TX_SYMCLK_SEL {
+ DPCSTX_DBG_TX_SYMCLK_SEL_IN0 = 0x0,
+ DPCSTX_DBG_TX_SYMCLK_SEL_IN1 = 0x1,
+ DPCSTX_DBG_TX_SYMCLK_SEL_FIFO_WR = 0x2,
+} DPCSTX_TX_SYMCLK_SEL;
+typedef enum DPCSTX_TX_SYMCLK_DIV2_SEL {
+ DPCSTX_DBG_TX_SYMCLK_DIV2_SEL_OUT0 = 0x0,
+ DPCSTX_DBG_TX_SYMCLK_DIV2_SEL_OUT1 = 0x1,
+ DPCSTX_DBG_TX_SYMCLK_DIV2_SEL_OUT2 = 0x2,
+ DPCSTX_DBG_TX_SYMCLK_DIV2_SEL_OUT3 = 0x3,
+ DPCSTX_DBG_TX_SYMCLK_DIV2_SEL_FIFO_RD = 0x4,
+ DPCSTX_DBG_TX_SYMCLK_DIV2_SEL_INT = 0x5,
+} DPCSTX_TX_SYMCLK_DIV2_SEL;
+typedef enum DPCSTX_DBG_CLOCK_SEL {
+ DPCSTX_DBG_CLOCK_SEL_DC_CFGCLK = 0x0,
+ DPCSTX_DBG_CLOCK_SEL_PHY_CFGCLK = 0x1,
+ DPCSTX_DBG_CLOCK_SEL_TXSYMCLK = 0x2,
+} DPCSTX_DBG_CLOCK_SEL;
+typedef enum DPCSTX_DVI_LINK_MODE {
+ DPCSTX_DVI_LINK_MODE_NORMAL = 0x0,
+ DPCSTX_DVI_LINK_MODE_DUAL_LINK_MASTER = 0x1,
+ DPCSTX_DVI_LINK_MODE_DUAL_LINK_SLAVER = 0x2,
+} DPCSTX_DVI_LINK_MODE;
+
+#endif /* DCE_11_2_ENUM_H */
diff --git a/drivers/gpu/drm/amd/include/asic_reg/dce/dce_11_2_sh_mask.h b/drivers/gpu/drm/amd/include/asic_reg/dce/dce_11_2_sh_mask.h
new file mode 100755
index 0000000..1ddc418
--- /dev/null
+++ b/drivers/gpu/drm/amd/include/asic_reg/dce/dce_11_2_sh_mask.h
@@ -0,0 +1,18687 @@
+/*
+ * DCE_11_2 Register documentation
+ *
+ * Copyright (C) 2016 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included
+ * in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+ * OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
+ * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+
+#ifndef DCE_11_2_SH_MASK_H
+#define DCE_11_2_SH_MASK_H
+
+#define PIPE0_PG_CONFIG__PIPE0_POWER_FORCEON_MASK 0x1
+#define PIPE0_PG_CONFIG__PIPE0_POWER_FORCEON__SHIFT 0x0
+#define PIPE0_PG_ENABLE__PIPE0_POWER_GATE_MASK 0x1
+#define PIPE0_PG_ENABLE__PIPE0_POWER_GATE__SHIFT 0x0
+#define PIPE0_PG_STATUS__PIPE0_PGFSM_READ_DATA_MASK 0xffffff
+#define PIPE0_PG_STATUS__PIPE0_PGFSM_READ_DATA__SHIFT 0x0
+#define PIPE0_PG_STATUS__PIPE0_DEBUG_PWR_STATUS_MASK 0x3000000
+#define PIPE0_PG_STATUS__PIPE0_DEBUG_PWR_STATUS__SHIFT 0x18
+#define PIPE0_PG_STATUS__PIPE0_DESIRED_PWR_STATE_MASK 0x10000000
+#define PIPE0_PG_STATUS__PIPE0_DESIRED_PWR_STATE__SHIFT 0x1c
+#define PIPE0_PG_STATUS__PIPE0_REQUESTED_PWR_STATE_MASK 0x20000000
+#define PIPE0_PG_STATUS__PIPE0_REQUESTED_PWR_STATE__SHIFT 0x1d
+#define PIPE0_PG_STATUS__PIPE0_PGFSM_PWR_STATUS_MASK 0xc0000000
+#define PIPE0_PG_STATUS__PIPE0_PGFSM_PWR_STATUS__SHIFT 0x1e
+#define PIPE1_PG_CONFIG__PIPE1_POWER_FORCEON_MASK 0x1
+#define PIPE1_PG_CONFIG__PIPE1_POWER_FORCEON__SHIFT 0x0
+#define PIPE1_PG_ENABLE__PIPE1_POWER_GATE_MASK 0x1
+#define PIPE1_PG_ENABLE__PIPE1_POWER_GATE__SHIFT 0x0
+#define PIPE1_PG_STATUS__PIPE1_PGFSM_READ_DATA_MASK 0xffffff
+#define PIPE1_PG_STATUS__PIPE1_PGFSM_READ_DATA__SHIFT 0x0
+#define PIPE1_PG_STATUS__PIPE1_DEBUG_PWR_STATUS_MASK 0x3000000
+#define PIPE1_PG_STATUS__PIPE1_DEBUG_PWR_STATUS__SHIFT 0x18
+#define PIPE1_PG_STATUS__PIPE1_DESIRED_PWR_STATE_MASK 0x10000000
+#define PIPE1_PG_STATUS__PIPE1_DESIRED_PWR_STATE__SHIFT 0x1c
+#define PIPE1_PG_STATUS__PIPE1_REQUESTED_PWR_STATE_MASK 0x20000000
+#define PIPE1_PG_STATUS__PIPE1_REQUESTED_PWR_STATE__SHIFT 0x1d
+#define PIPE1_PG_STATUS__PIPE1_PGFSM_PWR_STATUS_MASK 0xc0000000
+#define PIPE1_PG_STATUS__PIPE1_PGFSM_PWR_STATUS__SHIFT 0x1e
+#define PIPE2_PG_CONFIG__PIPE2_POWER_FORCEON_MASK 0x1
+#define PIPE2_PG_CONFIG__PIPE2_POWER_FORCEON__SHIFT 0x0
+#define PIPE2_PG_ENABLE__PIPE2_POWER_GATE_MASK 0x1
+#define PIPE2_PG_ENABLE__PIPE2_POWER_GATE__SHIFT 0x0
+#define PIPE2_PG_STATUS__PIPE2_PGFSM_READ_DATA_MASK 0xffffff
+#define PIPE2_PG_STATUS__PIPE2_PGFSM_READ_DATA__SHIFT 0x0
+#define PIPE2_PG_STATUS__PIPE2_DEBUG_PWR_STATUS_MASK 0x3000000
+#define PIPE2_PG_STATUS__PIPE2_DEBUG_PWR_STATUS__SHIFT 0x18
+#define PIPE2_PG_STATUS__PIPE2_DESIRED_PWR_STATE_MASK 0x10000000
+#define PIPE2_PG_STATUS__PIPE2_DESIRED_PWR_STATE__SHIFT 0x1c
+#define PIPE2_PG_STATUS__PIPE2_REQUESTED_PWR_STATE_MASK 0x20000000
+#define PIPE2_PG_STATUS__PIPE2_REQUESTED_PWR_STATE__SHIFT 0x1d
+#define PIPE2_PG_STATUS__PIPE2_PGFSM_PWR_STATUS_MASK 0xc0000000
+#define PIPE2_PG_STATUS__PIPE2_PGFSM_PWR_STATUS__SHIFT 0x1e
+#define PIPE3_PG_CONFIG__PIPE3_POWER_FORCEON_MASK 0x1
+#define PIPE3_PG_CONFIG__PIPE3_POWER_FORCEON__SHIFT 0x0
+#define PIPE3_PG_ENABLE__PIPE3_POWER_GATE_MASK 0x1
+#define PIPE3_PG_ENABLE__PIPE3_POWER_GATE__SHIFT 0x0
+#define PIPE3_PG_STATUS__PIPE3_PGFSM_READ_DATA_MASK 0xffffff
+#define PIPE3_PG_STATUS__PIPE3_PGFSM_READ_DATA__SHIFT 0x0
+#define PIPE3_PG_STATUS__PIPE3_DEBUG_PWR_STATUS_MASK 0x3000000
+#define PIPE3_PG_STATUS__PIPE3_DEBUG_PWR_STATUS__SHIFT 0x18
+#define PIPE3_PG_STATUS__PIPE3_DESIRED_PWR_STATE_MASK 0x10000000
+#define PIPE3_PG_STATUS__PIPE3_DESIRED_PWR_STATE__SHIFT 0x1c
+#define PIPE3_PG_STATUS__PIPE3_REQUESTED_PWR_STATE_MASK 0x20000000
+#define PIPE3_PG_STATUS__PIPE3_REQUESTED_PWR_STATE__SHIFT 0x1d
+#define PIPE3_PG_STATUS__PIPE3_PGFSM_PWR_STATUS_MASK 0xc0000000
+#define PIPE3_PG_STATUS__PIPE3_PGFSM_PWR_STATUS__SHIFT 0x1e
+#define PIPE4_PG_CONFIG__PIPE4_POWER_FORCEON_MASK 0x1
+#define PIPE4_PG_CONFIG__PIPE4_POWER_FORCEON__SHIFT 0x0
+#define PIPE4_PG_ENABLE__PIPE4_POWER_GATE_MASK 0x1
+#define PIPE4_PG_ENABLE__PIPE4_POWER_GATE__SHIFT 0x0
+#define PIPE4_PG_STATUS__PIPE4_PGFSM_READ_DATA_MASK 0xffffff
+#define PIPE4_PG_STATUS__PIPE4_PGFSM_READ_DATA__SHIFT 0x0
+#define PIPE4_PG_STATUS__PIPE4_DEBUG_PWR_STATUS_MASK 0x3000000
+#define PIPE4_PG_STATUS__PIPE4_DEBUG_PWR_STATUS__SHIFT 0x18
+#define PIPE4_PG_STATUS__PIPE4_DESIRED_PWR_STATE_MASK 0x10000000
+#define PIPE4_PG_STATUS__PIPE4_DESIRED_PWR_STATE__SHIFT 0x1c
+#define PIPE4_PG_STATUS__PIPE4_REQUESTED_PWR_STATE_MASK 0x20000000
+#define PIPE4_PG_STATUS__PIPE4_REQUESTED_PWR_STATE__SHIFT 0x1d
+#define PIPE4_PG_STATUS__PIPE4_PGFSM_PWR_STATUS_MASK 0xc0000000
+#define PIPE4_PG_STATUS__PIPE4_PGFSM_PWR_STATUS__SHIFT 0x1e
+#define PIPE5_PG_CONFIG__PIPE5_POWER_FORCEON_MASK 0x1
+#define PIPE5_PG_CONFIG__PIPE5_POWER_FORCEON__SHIFT 0x0
+#define PIPE5_PG_ENABLE__PIPE5_POWER_GATE_MASK 0x1
+#define PIPE5_PG_ENABLE__PIPE5_POWER_GATE__SHIFT 0x0
+#define PIPE5_PG_STATUS__PIPE5_PGFSM_READ_DATA_MASK 0xffffff
+#define PIPE5_PG_STATUS__PIPE5_PGFSM_READ_DATA__SHIFT 0x0
+#define PIPE5_PG_STATUS__PIPE5_DEBUG_PWR_STATUS_MASK 0x3000000
+#define PIPE5_PG_STATUS__PIPE5_DEBUG_PWR_STATUS__SHIFT 0x18
+#define PIPE5_PG_STATUS__PIPE5_DESIRED_PWR_STATE_MASK 0x10000000
+#define PIPE5_PG_STATUS__PIPE5_DESIRED_PWR_STATE__SHIFT 0x1c
+#define PIPE5_PG_STATUS__PIPE5_REQUESTED_PWR_STATE_MASK 0x20000000
+#define PIPE5_PG_STATUS__PIPE5_REQUESTED_PWR_STATE__SHIFT 0x1d
+#define PIPE5_PG_STATUS__PIPE5_PGFSM_PWR_STATUS_MASK 0xc0000000
+#define PIPE5_PG_STATUS__PIPE5_PGFSM_PWR_STATUS__SHIFT 0x1e
+#define DCPG_INTERRUPT_STATUS__DCFE0_POWER_UP_INT_OCCURRED_MASK 0x1
+#define DCPG_INTERRUPT_STATUS__DCFE0_POWER_UP_INT_OCCURRED__SHIFT 0x0
+#define DCPG_INTERRUPT_STATUS__DCFE0_POWER_DOWN_INT_OCCURRED_MASK 0x2
+#define DCPG_INTERRUPT_STATUS__DCFE0_POWER_DOWN_INT_OCCURRED__SHIFT 0x1
+#define DCPG_INTERRUPT_STATUS__DCFE1_POWER_UP_INT_OCCURRED_MASK 0x4
+#define DCPG_INTERRUPT_STATUS__DCFE1_POWER_UP_INT_OCCURRED__SHIFT 0x2
+#define DCPG_INTERRUPT_STATUS__DCFE1_POWER_DOWN_INT_OCCURRED_MASK 0x8
+#define DCPG_INTERRUPT_STATUS__DCFE1_POWER_DOWN_INT_OCCURRED__SHIFT 0x3
+#define DCPG_INTERRUPT_STATUS__DCFE2_POWER_UP_INT_OCCURRED_MASK 0x10
+#define DCPG_INTERRUPT_STATUS__DCFE2_POWER_UP_INT_OCCURRED__SHIFT 0x4
+#define DCPG_INTERRUPT_STATUS__DCFE2_POWER_DOWN_INT_OCCURRED_MASK 0x20
+#define DCPG_INTERRUPT_STATUS__DCFE2_POWER_DOWN_INT_OCCURRED__SHIFT 0x5
+#define DCPG_INTERRUPT_STATUS__DCFE3_POWER_UP_INT_OCCURRED_MASK 0x40
+#define DCPG_INTERRUPT_STATUS__DCFE3_POWER_UP_INT_OCCURRED__SHIFT 0x6
+#define DCPG_INTERRUPT_STATUS__DCFE3_POWER_DOWN_INT_OCCURRED_MASK 0x80
+#define DCPG_INTERRUPT_STATUS__DCFE3_POWER_DOWN_INT_OCCURRED__SHIFT 0x7
+#define DCPG_INTERRUPT_STATUS__DCFE4_POWER_UP_INT_OCCURRED_MASK 0x100
+#define DCPG_INTERRUPT_STATUS__DCFE4_POWER_UP_INT_OCCURRED__SHIFT 0x8
+#define DCPG_INTERRUPT_STATUS__DCFE4_POWER_DOWN_INT_OCCURRED_MASK 0x200
+#define DCPG_INTERRUPT_STATUS__DCFE4_POWER_DOWN_INT_OCCURRED__SHIFT 0x9
+#define DCPG_INTERRUPT_STATUS__DCFE5_POWER_UP_INT_OCCURRED_MASK 0x400
+#define DCPG_INTERRUPT_STATUS__DCFE5_POWER_UP_INT_OCCURRED__SHIFT 0xa
+#define DCPG_INTERRUPT_STATUS__DCFE5_POWER_DOWN_INT_OCCURRED_MASK 0x800
+#define DCPG_INTERRUPT_STATUS__DCFE5_POWER_DOWN_INT_OCCURRED__SHIFT 0xb
+#define DCPG_INTERRUPT_STATUS__DCFEV0_POWER_UP_INT_OCCURRED_MASK 0x1000
+#define DCPG_INTERRUPT_STATUS__DCFEV0_POWER_UP_INT_OCCURRED__SHIFT 0xc
+#define DCPG_INTERRUPT_STATUS__DCFEV0_POWER_DOWN_INT_OCCURRED_MASK 0x2000
+#define DCPG_INTERRUPT_STATUS__DCFEV0_POWER_DOWN_INT_OCCURRED__SHIFT 0xd
+#define DCPG_INTERRUPT_STATUS__DSI_POWER_UP_INT_OCCURRED_MASK 0x4000
+#define DCPG_INTERRUPT_STATUS__DSI_POWER_UP_INT_OCCURRED__SHIFT 0xe
+#define DCPG_INTERRUPT_STATUS__DSI_POWER_DOWN_INT_OCCURRED_MASK 0x8000
+#define DCPG_INTERRUPT_STATUS__DSI_POWER_DOWN_INT_OCCURRED__SHIFT 0xf
+#define DCPG_INTERRUPT_STATUS__DCFEV1_POWER_UP_INT_OCCURRED_MASK 0x10000
+#define DCPG_INTERRUPT_STATUS__DCFEV1_POWER_UP_INT_OCCURRED__SHIFT 0x10
+#define DCPG_INTERRUPT_STATUS__DCFEV1_POWER_DOWN_INT_OCCURRED_MASK 0x20000
+#define DCPG_INTERRUPT_STATUS__DCFEV1_POWER_DOWN_INT_OCCURRED__SHIFT 0x11
+#define DCPG_INTERRUPT_CONTROL__DCFE0_POWER_UP_INT_MASK_MASK 0x1
+#define DCPG_INTERRUPT_CONTROL__DCFE0_POWER_UP_INT_MASK__SHIFT 0x0
+#define DCPG_INTERRUPT_CONTROL__DCFE0_POWER_UP_INT_CLEAR_MASK 0x2
+#define DCPG_INTERRUPT_CONTROL__DCFE0_POWER_UP_INT_CLEAR__SHIFT 0x1
+#define DCPG_INTERRUPT_CONTROL__DCFE0_POWER_DOWN_INT_MASK_MASK 0x4
+#define DCPG_INTERRUPT_CONTROL__DCFE0_POWER_DOWN_INT_MASK__SHIFT 0x2
+#define DCPG_INTERRUPT_CONTROL__DCFE0_POWER_DOWN_INT_CLEAR_MASK 0x8
+#define DCPG_INTERRUPT_CONTROL__DCFE0_POWER_DOWN_INT_CLEAR__SHIFT 0x3
+#define DCPG_INTERRUPT_CONTROL__DCFE1_POWER_UP_INT_MASK_MASK 0x10
+#define DCPG_INTERRUPT_CONTROL__DCFE1_POWER_UP_INT_MASK__SHIFT 0x4
+#define DCPG_INTERRUPT_CONTROL__DCFE1_POWER_UP_INT_CLEAR_MASK 0x20
+#define DCPG_INTERRUPT_CONTROL__DCFE1_POWER_UP_INT_CLEAR__SHIFT 0x5
+#define DCPG_INTERRUPT_CONTROL__DCFE1_POWER_DOWN_INT_MASK_MASK 0x40
+#define DCPG_INTERRUPT_CONTROL__DCFE1_POWER_DOWN_INT_MASK__SHIFT 0x6
+#define DCPG_INTERRUPT_CONTROL__DCFE1_POWER_DOWN_INT_CLEAR_MASK 0x80
+#define DCPG_INTERRUPT_CONTROL__DCFE1_POWER_DOWN_INT_CLEAR__SHIFT 0x7
+#define DCPG_INTERRUPT_CONTROL__DCFE2_POWER_UP_INT_MASK_MASK 0x100
+#define DCPG_INTERRUPT_CONTROL__DCFE2_POWER_UP_INT_MASK__SHIFT 0x8
+#define DCPG_INTERRUPT_CONTROL__DCFE2_POWER_UP_INT_CLEAR_MASK 0x200
+#define DCPG_INTERRUPT_CONTROL__DCFE2_POWER_UP_INT_CLEAR__SHIFT 0x9
+#define DCPG_INTERRUPT_CONTROL__DCFE2_POWER_DOWN_INT_MASK_MASK 0x400
+#define DCPG_INTERRUPT_CONTROL__DCFE2_POWER_DOWN_INT_MASK__SHIFT 0xa
+#define DCPG_INTERRUPT_CONTROL__DCFE2_POWER_DOWN_INT_CLEAR_MASK 0x800
+#define DCPG_INTERRUPT_CONTROL__DCFE2_POWER_DOWN_INT_CLEAR__SHIFT 0xb
+#define DCPG_INTERRUPT_CONTROL__DCFE3_POWER_UP_INT_MASK_MASK 0x1000
+#define DCPG_INTERRUPT_CONTROL__DCFE3_POWER_UP_INT_MASK__SHIFT 0xc
+#define DCPG_INTERRUPT_CONTROL__DCFE3_POWER_UP_INT_CLEAR_MASK 0x2000
+#define DCPG_INTERRUPT_CONTROL__DCFE3_POWER_UP_INT_CLEAR__SHIFT 0xd
+#define DCPG_INTERRUPT_CONTROL__DCFE3_POWER_DOWN_INT_MASK_MASK 0x4000
+#define DCPG_INTERRUPT_CONTROL__DCFE3_POWER_DOWN_INT_MASK__SHIFT 0xe
+#define DCPG_INTERRUPT_CONTROL__DCFE3_POWER_DOWN_INT_CLEAR_MASK 0x8000
+#define DCPG_INTERRUPT_CONTROL__DCFE3_POWER_DOWN_INT_CLEAR__SHIFT 0xf
+#define DCPG_INTERRUPT_CONTROL__DCFE4_POWER_UP_INT_MASK_MASK 0x10000
+#define DCPG_INTERRUPT_CONTROL__DCFE4_POWER_UP_INT_MASK__SHIFT 0x10
+#define DCPG_INTERRUPT_CONTROL__DCFE4_POWER_UP_INT_CLEAR_MASK 0x20000
+#define DCPG_INTERRUPT_CONTROL__DCFE4_POWER_UP_INT_CLEAR__SHIFT 0x11
+#define DCPG_INTERRUPT_CONTROL__DCFE4_POWER_DOWN_INT_MASK_MASK 0x40000
+#define DCPG_INTERRUPT_CONTROL__DCFE4_POWER_DOWN_INT_MASK__SHIFT 0x12
+#define DCPG_INTERRUPT_CONTROL__DCFE4_POWER_DOWN_INT_CLEAR_MASK 0x80000
+#define DCPG_INTERRUPT_CONTROL__DCFE4_POWER_DOWN_INT_CLEAR__SHIFT 0x13
+#define DCPG_INTERRUPT_CONTROL__DCFE5_POWER_UP_INT_MASK_MASK 0x100000
+#define DCPG_INTERRUPT_CONTROL__DCFE5_POWER_UP_INT_MASK__SHIFT 0x14
+#define DCPG_INTERRUPT_CONTROL__DCFE5_POWER_UP_INT_CLEAR_MASK 0x200000
+#define DCPG_INTERRUPT_CONTROL__DCFE5_POWER_UP_INT_CLEAR__SHIFT 0x15
+#define DCPG_INTERRUPT_CONTROL__DCFE5_POWER_DOWN_INT_MASK_MASK 0x400000
+#define DCPG_INTERRUPT_CONTROL__DCFE5_POWER_DOWN_INT_MASK__SHIFT 0x16
+#define DCPG_INTERRUPT_CONTROL__DCFE5_POWER_DOWN_INT_CLEAR_MASK 0x800000
+#define DCPG_INTERRUPT_CONTROL__DCFE5_POWER_DOWN_INT_CLEAR__SHIFT 0x17
+#define DCPG_INTERRUPT_CONTROL__DCFEV0_POWER_UP_INT_MASK_MASK 0x1000000
+#define DCPG_INTERRUPT_CONTROL__DCFEV0_POWER_UP_INT_MASK__SHIFT 0x18
+#define DCPG_INTERRUPT_CONTROL__DCFEV0_POWER_UP_INT_CLEAR_MASK 0x2000000
+#define DCPG_INTERRUPT_CONTROL__DCFEV0_POWER_UP_INT_CLEAR__SHIFT 0x19
+#define DCPG_INTERRUPT_CONTROL__DCFEV0_POWER_DOWN_INT_MASK_MASK 0x4000000
+#define DCPG_INTERRUPT_CONTROL__DCFEV0_POWER_DOWN_INT_MASK__SHIFT 0x1a
+#define DCPG_INTERRUPT_CONTROL__DCFEV0_POWER_DOWN_INT_CLEAR_MASK 0x8000000
+#define DCPG_INTERRUPT_CONTROL__DCFEV0_POWER_DOWN_INT_CLEAR__SHIFT 0x1b
+#define DCPG_INTERRUPT_CONTROL__DSI_POWER_UP_INT_MASK_MASK 0x10000000
+#define DCPG_INTERRUPT_CONTROL__DSI_POWER_UP_INT_MASK__SHIFT 0x1c
+#define DCPG_INTERRUPT_CONTROL__DSI_POWER_UP_INT_CLEAR_MASK 0x20000000
+#define DCPG_INTERRUPT_CONTROL__DSI_POWER_UP_INT_CLEAR__SHIFT 0x1d
+#define DCPG_INTERRUPT_CONTROL__DSI_POWER_DOWN_INT_MASK_MASK 0x40000000
+#define DCPG_INTERRUPT_CONTROL__DSI_POWER_DOWN_INT_MASK__SHIFT 0x1e
+#define DCPG_INTERRUPT_CONTROL__DSI_POWER_DOWN_INT_CLEAR_MASK 0x80000000
+#define DCPG_INTERRUPT_CONTROL__DSI_POWER_DOWN_INT_CLEAR__SHIFT 0x1f
+#define DCPG_INTERRUPT_CONTROL2__DCFEV1_POWER_UP_INT_MASK_MASK 0x1000000
+#define DCPG_INTERRUPT_CONTROL2__DCFEV1_POWER_UP_INT_MASK__SHIFT 0x18
+#define DCPG_INTERRUPT_CONTROL2__DCFEV1_POWER_UP_INT_CLEAR_MASK 0x2000000
+#define DCPG_INTERRUPT_CONTROL2__DCFEV1_POWER_UP_INT_CLEAR__SHIFT 0x19
+#define DCPG_INTERRUPT_CONTROL2__DCFEV1_POWER_DOWN_INT_MASK_MASK 0x4000000
+#define DCPG_INTERRUPT_CONTROL2__DCFEV1_POWER_DOWN_INT_MASK__SHIFT 0x1a
+#define DCPG_INTERRUPT_CONTROL2__DCFEV1_POWER_DOWN_INT_CLEAR_MASK 0x8000000
+#define DCPG_INTERRUPT_CONTROL2__DCFEV1_POWER_DOWN_INT_CLEAR__SHIFT 0x1b
+#define DC_IP_REQUEST_CNTL__IP_REQUEST_EN_MASK 0x1
+#define DC_IP_REQUEST_CNTL__IP_REQUEST_EN__SHIFT 0x0
+#define DC_PGFSM_CONFIG_REG__PGFSM_CONFIG_REG_MASK 0xffffffff
+#define DC_PGFSM_CONFIG_REG__PGFSM_CONFIG_REG__SHIFT 0x0
+#define DC_PGFSM_WRITE_REG__PGFSM_WRITE_REG_MASK 0xffffffff
+#define DC_PGFSM_WRITE_REG__PGFSM_WRITE_REG__SHIFT 0x0
+#define DC_PGCNTL_STATUS_REG__SWREQ_RWOP_BUSY_MASK 0x1
+#define DC_PGCNTL_STATUS_REG__SWREQ_RWOP_BUSY__SHIFT 0x0
+#define DC_PGCNTL_STATUS_REG__SWREQ_RWOP_FORCE_MASK 0x2
+#define DC_PGCNTL_STATUS_REG__SWREQ_RWOP_FORCE__SHIFT 0x1
+#define DC_PGCNTL_STATUS_REG__IPREQ_IGNORE_STATUS_MASK 0x4
+#define DC_PGCNTL_STATUS_REG__IPREQ_IGNORE_STATUS__SHIFT 0x2
+#define DC_PGCNTL_STATUS_REG__DCPG_ECO_DEBUG_MASK 0xffff0000
+#define DC_PGCNTL_STATUS_REG__DCPG_ECO_DEBUG__SHIFT 0x10
+#define DCPG_TEST_DEBUG_INDEX__DCPG_TEST_DEBUG_INDEX_MASK 0xff
+#define DCPG_TEST_DEBUG_INDEX__DCPG_TEST_DEBUG_INDEX__SHIFT 0x0
+#define DCPG_TEST_DEBUG_INDEX__DCPG_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define DCPG_TEST_DEBUG_INDEX__DCPG_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define DCPG_TEST_DEBUG_DATA__DCPG_TEST_DEBUG_DATA_MASK 0xffffffff
+#define DCPG_TEST_DEBUG_DATA__DCPG_TEST_DEBUG_DATA__SHIFT 0x0
+#define BL1_PWM_AMBIENT_LIGHT_LEVEL__BL1_PWM_AMBIENT_LIGHT_LEVEL_MASK 0x1ffff
+#define BL1_PWM_AMBIENT_LIGHT_LEVEL__BL1_PWM_AMBIENT_LIGHT_LEVEL__SHIFT 0x0
+#define BL1_PWM_USER_LEVEL__BL1_PWM_USER_LEVEL_MASK 0x1ffff
+#define BL1_PWM_USER_LEVEL__BL1_PWM_USER_LEVEL__SHIFT 0x0
+#define BL1_PWM_TARGET_ABM_LEVEL__BL1_PWM_TARGET_ABM_LEVEL_MASK 0x1ffff
+#define BL1_PWM_TARGET_ABM_LEVEL__BL1_PWM_TARGET_ABM_LEVEL__SHIFT 0x0
+#define BL1_PWM_CURRENT_ABM_LEVEL__BL1_PWM_CURRENT_ABM_LEVEL_MASK 0x1ffff
+#define BL1_PWM_CURRENT_ABM_LEVEL__BL1_PWM_CURRENT_ABM_LEVEL__SHIFT 0x0
+#define BL1_PWM_FINAL_DUTY_CYCLE__BL1_PWM_FINAL_DUTY_CYCLE_MASK 0x1ffff
+#define BL1_PWM_FINAL_DUTY_CYCLE__BL1_PWM_FINAL_DUTY_CYCLE__SHIFT 0x0
+#define BL1_PWM_MINIMUM_DUTY_CYCLE__BL1_PWM_MINIMUM_DUTY_CYCLE_MASK 0x1ffff
+#define BL1_PWM_MINIMUM_DUTY_CYCLE__BL1_PWM_MINIMUM_DUTY_CYCLE__SHIFT 0x0
+#define BL1_PWM_ABM_CNTL__BL1_PWM_USE_ABM_EN_MASK 0x1
+#define BL1_PWM_ABM_CNTL__BL1_PWM_USE_ABM_EN__SHIFT 0x0
+#define BL1_PWM_ABM_CNTL__BL1_PWM_USE_AMBIENT_LEVEL_EN_MASK 0x2
+#define BL1_PWM_ABM_CNTL__BL1_PWM_USE_AMBIENT_LEVEL_EN__SHIFT 0x1
+#define BL1_PWM_ABM_CNTL__BL1_PWM_AUTO_UPDATE_CURRENT_ABM_LEVEL_EN_MASK 0x4
+#define BL1_PWM_ABM_CNTL__BL1_PWM_AUTO_UPDATE_CURRENT_ABM_LEVEL_EN__SHIFT 0x2
+#define BL1_PWM_ABM_CNTL__BL1_PWM_AUTO_CALC_FINAL_DUTY_CYCLE_EN_MASK 0x8
+#define BL1_PWM_ABM_CNTL__BL1_PWM_AUTO_CALC_FINAL_DUTY_CYCLE_EN__SHIFT 0x3
+#define BL1_PWM_ABM_CNTL__BL1_PWM_AUTO_UPDATE_CURRENT_ABM_STEP_SIZE_MASK 0xffff0000
+#define BL1_PWM_ABM_CNTL__BL1_PWM_AUTO_UPDATE_CURRENT_ABM_STEP_SIZE__SHIFT 0x10
+#define BL1_PWM_BL_UPDATE_SAMPLE_RATE__BL1_PWM_BL_UPDATE_SAMPLE_RATE_COUNT_EN_MASK 0x1
+#define BL1_PWM_BL_UPDATE_SAMPLE_RATE__BL1_PWM_BL_UPDATE_SAMPLE_RATE_COUNT_EN__SHIFT 0x0
+#define BL1_PWM_BL_UPDATE_SAMPLE_RATE__BL1_PWM_BL_UPDATE_RESET_SAMPLE_RATE_FRAME_COUNTER_MASK 0x2
+#define BL1_PWM_BL_UPDATE_SAMPLE_RATE__BL1_PWM_BL_UPDATE_RESET_SAMPLE_RATE_FRAME_COUNTER__SHIFT 0x1
+#define BL1_PWM_BL_UPDATE_SAMPLE_RATE__BL1_PWM_BL_UPDATE_SAMPLE_RATE_FRAME_COUNT_MASK 0xff00
+#define BL1_PWM_BL_UPDATE_SAMPLE_RATE__BL1_PWM_BL_UPDATE_SAMPLE_RATE_FRAME_COUNT__SHIFT 0x8
+#define BL1_PWM_BL_UPDATE_SAMPLE_RATE__BL1_PWM_BL_UPDATE_INITIAL_SAMPLE_RATE_COUNT_VALUE_WHEN_RESET_MASK 0xff0000
+#define BL1_PWM_BL_UPDATE_SAMPLE_RATE__BL1_PWM_BL_UPDATE_INITIAL_SAMPLE_RATE_COUNT_VALUE_WHEN_RESET__SHIFT 0x10
+#define BL1_PWM_BL_UPDATE_SAMPLE_RATE__ABM1_HGLS_REG_LOCK_MASK 0x80000000
+#define BL1_PWM_BL_UPDATE_SAMPLE_RATE__ABM1_HGLS_REG_LOCK__SHIFT 0x1f
+#define BL1_PWM_GRP2_REG_LOCK__BL1_PWM_GRP2_REG_LOCK_MASK 0x1
+#define BL1_PWM_GRP2_REG_LOCK__BL1_PWM_GRP2_REG_LOCK__SHIFT 0x0
+#define BL1_PWM_GRP2_REG_LOCK__BL1_PWM_GRP2_REG_UPDATE_PENDING_MASK 0x100
+#define BL1_PWM_GRP2_REG_LOCK__BL1_PWM_GRP2_REG_UPDATE_PENDING__SHIFT 0x8
+#define BL1_PWM_GRP2_REG_LOCK__BL1_PWM_GRP2_UPDATE_AT_FRAME_START_MASK 0x10000
+#define BL1_PWM_GRP2_REG_LOCK__BL1_PWM_GRP2_UPDATE_AT_FRAME_START__SHIFT 0x10
+#define BL1_PWM_GRP2_REG_LOCK__BL1_PWM_GRP2_FRAME_START_DISP_SEL_MASK 0xe0000
+#define BL1_PWM_GRP2_REG_LOCK__BL1_PWM_GRP2_FRAME_START_DISP_SEL__SHIFT 0x11
+#define BL1_PWM_GRP2_REG_LOCK__BL1_PWM_GRP2_READBACK_DB_REG_VALUE_EN_MASK 0x1000000
+#define BL1_PWM_GRP2_REG_LOCK__BL1_PWM_GRP2_READBACK_DB_REG_VALUE_EN__SHIFT 0x18
+#define BL1_PWM_GRP2_REG_LOCK__BL1_PWM_GRP2_IGNORE_MASTER_LOCK_EN_MASK 0x80000000
+#define BL1_PWM_GRP2_REG_LOCK__BL1_PWM_GRP2_IGNORE_MASTER_LOCK_EN__SHIFT 0x1f
+#define DC_ABM1_CNTL__ABM1_EN_MASK 0x1
+#define DC_ABM1_CNTL__ABM1_EN__SHIFT 0x0
+#define DC_ABM1_CNTL__ABM1_SOURCE_SELECT_MASK 0x700
+#define DC_ABM1_CNTL__ABM1_SOURCE_SELECT__SHIFT 0x8
+#define DC_ABM1_CNTL__ABM1_BLANK_MODE_SUPPORT_ENABLE_MASK 0x80000000
+#define DC_ABM1_CNTL__ABM1_BLANK_MODE_SUPPORT_ENABLE__SHIFT 0x1f
+#define DC_ABM1_IPCSC_COEFF_SEL__ABM1_IPCSC_COEFF_SEL_B_MASK 0xf
+#define DC_ABM1_IPCSC_COEFF_SEL__ABM1_IPCSC_COEFF_SEL_B__SHIFT 0x0
+#define DC_ABM1_IPCSC_COEFF_SEL__ABM1_IPCSC_COEFF_SEL_G_MASK 0xf00
+#define DC_ABM1_IPCSC_COEFF_SEL__ABM1_IPCSC_COEFF_SEL_G__SHIFT 0x8
+#define DC_ABM1_IPCSC_COEFF_SEL__ABM1_IPCSC_COEFF_SEL_R_MASK 0xf0000
+#define DC_ABM1_IPCSC_COEFF_SEL__ABM1_IPCSC_COEFF_SEL_R__SHIFT 0x10
+#define DC_ABM1_IPCSC_COEFF_SEL__ABM1_HGLS_REG_LOCK_MASK 0x80000000
+#define DC_ABM1_IPCSC_COEFF_SEL__ABM1_HGLS_REG_LOCK__SHIFT 0x1f
+#define DC_ABM1_ACE_OFFSET_SLOPE_0__ABM1_ACE_SLOPE_0_MASK 0x7fff
+#define DC_ABM1_ACE_OFFSET_SLOPE_0__ABM1_ACE_SLOPE_0__SHIFT 0x0
+#define DC_ABM1_ACE_OFFSET_SLOPE_0__ABM1_ACE_OFFSET_0_MASK 0x7ff0000
+#define DC_ABM1_ACE_OFFSET_SLOPE_0__ABM1_ACE_OFFSET_0__SHIFT 0x10
+#define DC_ABM1_ACE_OFFSET_SLOPE_0__ABM1_ACE_LOCK_MASK 0x80000000
+#define DC_ABM1_ACE_OFFSET_SLOPE_0__ABM1_ACE_LOCK__SHIFT 0x1f
+#define DC_ABM1_ACE_OFFSET_SLOPE_1__ABM1_ACE_SLOPE_1_MASK 0x7fff
+#define DC_ABM1_ACE_OFFSET_SLOPE_1__ABM1_ACE_SLOPE_1__SHIFT 0x0
+#define DC_ABM1_ACE_OFFSET_SLOPE_1__ABM1_ACE_OFFSET_1_MASK 0x7ff0000
+#define DC_ABM1_ACE_OFFSET_SLOPE_1__ABM1_ACE_OFFSET_1__SHIFT 0x10
+#define DC_ABM1_ACE_OFFSET_SLOPE_1__ABM1_ACE_LOCK_MASK 0x80000000
+#define DC_ABM1_ACE_OFFSET_SLOPE_1__ABM1_ACE_LOCK__SHIFT 0x1f
+#define DC_ABM1_ACE_OFFSET_SLOPE_2__ABM1_ACE_SLOPE_2_MASK 0x7fff
+#define DC_ABM1_ACE_OFFSET_SLOPE_2__ABM1_ACE_SLOPE_2__SHIFT 0x0
+#define DC_ABM1_ACE_OFFSET_SLOPE_2__ABM1_ACE_OFFSET_2_MASK 0x7ff0000
+#define DC_ABM1_ACE_OFFSET_SLOPE_2__ABM1_ACE_OFFSET_2__SHIFT 0x10
+#define DC_ABM1_ACE_OFFSET_SLOPE_2__ABM1_ACE_LOCK_MASK 0x80000000
+#define DC_ABM1_ACE_OFFSET_SLOPE_2__ABM1_ACE_LOCK__SHIFT 0x1f
+#define DC_ABM1_ACE_OFFSET_SLOPE_3__ABM1_ACE_SLOPE_3_MASK 0x7fff
+#define DC_ABM1_ACE_OFFSET_SLOPE_3__ABM1_ACE_SLOPE_3__SHIFT 0x0
+#define DC_ABM1_ACE_OFFSET_SLOPE_3__ABM1_ACE_OFFSET_3_MASK 0x7ff0000
+#define DC_ABM1_ACE_OFFSET_SLOPE_3__ABM1_ACE_OFFSET_3__SHIFT 0x10
+#define DC_ABM1_ACE_OFFSET_SLOPE_3__ABM1_ACE_LOCK_MASK 0x80000000
+#define DC_ABM1_ACE_OFFSET_SLOPE_3__ABM1_ACE_LOCK__SHIFT 0x1f
+#define DC_ABM1_ACE_OFFSET_SLOPE_4__ABM1_ACE_SLOPE_4_MASK 0x7fff
+#define DC_ABM1_ACE_OFFSET_SLOPE_4__ABM1_ACE_SLOPE_4__SHIFT 0x0
+#define DC_ABM1_ACE_OFFSET_SLOPE_4__ABM1_ACE_OFFSET_4_MASK 0x7ff0000
+#define DC_ABM1_ACE_OFFSET_SLOPE_4__ABM1_ACE_OFFSET_4__SHIFT 0x10
+#define DC_ABM1_ACE_OFFSET_SLOPE_4__ABM1_ACE_LOCK_MASK 0x80000000
+#define DC_ABM1_ACE_OFFSET_SLOPE_4__ABM1_ACE_LOCK__SHIFT 0x1f
+#define DC_ABM1_ACE_THRES_12__ABM1_ACE_THRES_1_MASK 0x3ff
+#define DC_ABM1_ACE_THRES_12__ABM1_ACE_THRES_1__SHIFT 0x0
+#define DC_ABM1_ACE_THRES_12__ABM1_ACE_THRES_2_MASK 0x3ff0000
+#define DC_ABM1_ACE_THRES_12__ABM1_ACE_THRES_2__SHIFT 0x10
+#define DC_ABM1_ACE_THRES_12__ABM1_ACE_LOCK_MASK 0x80000000
+#define DC_ABM1_ACE_THRES_12__ABM1_ACE_LOCK__SHIFT 0x1f
+#define DC_ABM1_ACE_THRES_34__ABM1_ACE_THRES_3_MASK 0x3ff
+#define DC_ABM1_ACE_THRES_34__ABM1_ACE_THRES_3__SHIFT 0x0
+#define DC_ABM1_ACE_THRES_34__ABM1_ACE_THRES_4_MASK 0x3ff0000
+#define DC_ABM1_ACE_THRES_34__ABM1_ACE_THRES_4__SHIFT 0x10
+#define DC_ABM1_ACE_THRES_34__ABM1_ACE_IGNORE_MASTER_LOCK_EN_MASK 0x10000000
+#define DC_ABM1_ACE_THRES_34__ABM1_ACE_IGNORE_MASTER_LOCK_EN__SHIFT 0x1c
+#define DC_ABM1_ACE_THRES_34__ABM1_ACE_READBACK_DB_REG_VALUE_EN_MASK 0x20000000
+#define DC_ABM1_ACE_THRES_34__ABM1_ACE_READBACK_DB_REG_VALUE_EN__SHIFT 0x1d
+#define DC_ABM1_ACE_THRES_34__ABM1_ACE_DBUF_REG_UPDATE_PENDING_MASK 0x40000000
+#define DC_ABM1_ACE_THRES_34__ABM1_ACE_DBUF_REG_UPDATE_PENDING__SHIFT 0x1e
+#define DC_ABM1_ACE_THRES_34__ABM1_ACE_LOCK_MASK 0x80000000
+#define DC_ABM1_ACE_THRES_34__ABM1_ACE_LOCK__SHIFT 0x1f
+#define DC_ABM1_ACE_CNTL_MISC__ABM1_ACE_REG_WR_MISSED_FRAME_MASK 0x1
+#define DC_ABM1_ACE_CNTL_MISC__ABM1_ACE_REG_WR_MISSED_FRAME__SHIFT 0x0
+#define DC_ABM1_ACE_CNTL_MISC__ABM1_ACE_REG_WR_MISSED_FRAME_CLEAR_MASK 0x100
+#define DC_ABM1_ACE_CNTL_MISC__ABM1_ACE_REG_WR_MISSED_FRAME_CLEAR__SHIFT 0x8
+#define DC_ABM1_DEBUG_MISC__ABM1_HG_FORCE_INTERRUPT_MASK 0x1
+#define DC_ABM1_DEBUG_MISC__ABM1_HG_FORCE_INTERRUPT__SHIFT 0x0
+#define DC_ABM1_DEBUG_MISC__ABM1_LS_FORCE_INTERRUPT_MASK 0x100
+#define DC_ABM1_DEBUG_MISC__ABM1_LS_FORCE_INTERRUPT__SHIFT 0x8
+#define DC_ABM1_DEBUG_MISC__ABM1_BL_FORCE_INTERRUPT_MASK 0x10000
+#define DC_ABM1_DEBUG_MISC__ABM1_BL_FORCE_INTERRUPT__SHIFT 0x10
+#define DC_ABM1_HGLS_REG_READ_PROGRESS__ABM1_HG_REG_READ_IN_PROGRESS_MASK 0x1
+#define DC_ABM1_HGLS_REG_READ_PROGRESS__ABM1_HG_REG_READ_IN_PROGRESS__SHIFT 0x0
+#define DC_ABM1_HGLS_REG_READ_PROGRESS__ABM1_LS_REG_READ_IN_PROGRESS_MASK 0x2
+#define DC_ABM1_HGLS_REG_READ_PROGRESS__ABM1_LS_REG_READ_IN_PROGRESS__SHIFT 0x1
+#define DC_ABM1_HGLS_REG_READ_PROGRESS__ABM1_BL_REG_READ_IN_PROGRESS_MASK 0x4
+#define DC_ABM1_HGLS_REG_READ_PROGRESS__ABM1_BL_REG_READ_IN_PROGRESS__SHIFT 0x2
+#define DC_ABM1_HGLS_REG_READ_PROGRESS__ABM1_HG_REG_READ_MISSED_FRAME_MASK 0x100
+#define DC_ABM1_HGLS_REG_READ_PROGRESS__ABM1_HG_REG_READ_MISSED_FRAME__SHIFT 0x8
+#define DC_ABM1_HGLS_REG_READ_PROGRESS__ABM1_LS_REG_READ_MISSED_FRAME_MASK 0x200
+#define DC_ABM1_HGLS_REG_READ_PROGRESS__ABM1_LS_REG_READ_MISSED_FRAME__SHIFT 0x9
+#define DC_ABM1_HGLS_REG_READ_PROGRESS__ABM1_BL_REG_READ_MISSED_FRAME_MASK 0x400
+#define DC_ABM1_HGLS_REG_READ_PROGRESS__ABM1_BL_REG_READ_MISSED_FRAME__SHIFT 0xa
+#define DC_ABM1_HGLS_REG_READ_PROGRESS__ABM1_HG_REG_READ_MISSED_FRAME_CLEAR_MASK 0x10000
+#define DC_ABM1_HGLS_REG_READ_PROGRESS__ABM1_HG_REG_READ_MISSED_FRAME_CLEAR__SHIFT 0x10
+#define DC_ABM1_HGLS_REG_READ_PROGRESS__ABM1_LS_REG_READ_MISSED_FRAME_CLEAR_MASK 0x1000000
+#define DC_ABM1_HGLS_REG_READ_PROGRESS__ABM1_LS_REG_READ_MISSED_FRAME_CLEAR__SHIFT 0x18
+#define DC_ABM1_HGLS_REG_READ_PROGRESS__ABM1_BL_REG_READ_MISSED_FRAME_CLEAR_MASK 0x80000000
+#define DC_ABM1_HGLS_REG_READ_PROGRESS__ABM1_BL_REG_READ_MISSED_FRAME_CLEAR__SHIFT 0x1f
+#define DC_ABM1_HG_MISC_CTRL__ABM1_HG_NUM_OF_BINS_SEL_MASK 0x3
+#define DC_ABM1_HG_MISC_CTRL__ABM1_HG_NUM_OF_BINS_SEL__SHIFT 0x0
+#define DC_ABM1_HG_MISC_CTRL__ABM1_HG_VMAX_SEL_MASK 0x100
+#define DC_ABM1_HG_MISC_CTRL__ABM1_HG_VMAX_SEL__SHIFT 0x8
+#define DC_ABM1_HG_MISC_CTRL__ABM1_HG_FINE_MODE_BIN_SEL_MASK 0x1000
+#define DC_ABM1_HG_MISC_CTRL__ABM1_HG_FINE_MODE_BIN_SEL__SHIFT 0xc
+#define DC_ABM1_HG_MISC_CTRL__ABM1_HG_BIN_BITWIDTH_SIZE_SEL_MASK 0x30000
+#define DC_ABM1_HG_MISC_CTRL__ABM1_HG_BIN_BITWIDTH_SIZE_SEL__SHIFT 0x10
+#define DC_ABM1_HG_MISC_CTRL__ABM1_OVR_SCAN_PIXEL_PROCESS_EN_MASK 0x100000
+#define DC_ABM1_HG_MISC_CTRL__ABM1_OVR_SCAN_PIXEL_PROCESS_EN__SHIFT 0x14
+#define DC_ABM1_HG_MISC_CTRL__ABM1_DBUF_HGLS_READBACK_DB_REG_VALUE_EN_MASK 0x800000
+#define DC_ABM1_HG_MISC_CTRL__ABM1_DBUF_HGLS_READBACK_DB_REG_VALUE_EN__SHIFT 0x17
+#define DC_ABM1_HG_MISC_CTRL__ABM1_DBUF_HGLS_REG_FRAME_START_DISP_SEL_MASK 0x7000000
+#define DC_ABM1_HG_MISC_CTRL__ABM1_DBUF_HGLS_REG_FRAME_START_DISP_SEL__SHIFT 0x18
+#define DC_ABM1_HG_MISC_CTRL__ABM1_DBUF_HGLS_REG_UPDATE_AT_FRAME_START_MASK 0x10000000
+#define DC_ABM1_HG_MISC_CTRL__ABM1_DBUF_HGLS_REG_UPDATE_AT_FRAME_START__SHIFT 0x1c
+#define DC_ABM1_HG_MISC_CTRL__ABM1_HGLS_IGNORE_MASTER_LOCK_EN_MASK 0x20000000
+#define DC_ABM1_HG_MISC_CTRL__ABM1_HGLS_IGNORE_MASTER_LOCK_EN__SHIFT 0x1d
+#define DC_ABM1_HG_MISC_CTRL__ABM1_DBUF_HGLS_REG_UPDATE_PENDING_MASK 0x40000000
+#define DC_ABM1_HG_MISC_CTRL__ABM1_DBUF_HGLS_REG_UPDATE_PENDING__SHIFT 0x1e
+#define DC_ABM1_HG_MISC_CTRL__ABM1_HGLS_REG_LOCK_MASK 0x80000000
+#define DC_ABM1_HG_MISC_CTRL__ABM1_HGLS_REG_LOCK__SHIFT 0x1f
+#define DC_ABM1_LS_SUM_OF_LUMA__ABM1_LS_SUM_OF_LUMA_MASK 0xffffffff
+#define DC_ABM1_LS_SUM_OF_LUMA__ABM1_LS_SUM_OF_LUMA__SHIFT 0x0
+#define DC_ABM1_LS_MIN_MAX_LUMA__ABM1_LS_MIN_LUMA_MASK 0x3ff
+#define DC_ABM1_LS_MIN_MAX_LUMA__ABM1_LS_MIN_LUMA__SHIFT 0x0
+#define DC_ABM1_LS_MIN_MAX_LUMA__ABM1_LS_MAX_LUMA_MASK 0x3ff0000
+#define DC_ABM1_LS_MIN_MAX_LUMA__ABM1_LS_MAX_LUMA__SHIFT 0x10
+#define DC_ABM1_LS_FILTERED_MIN_MAX_LUMA__ABM1_LS_FILTERED_MIN_LUMA_MASK 0x3ff
+#define DC_ABM1_LS_FILTERED_MIN_MAX_LUMA__ABM1_LS_FILTERED_MIN_LUMA__SHIFT 0x0
+#define DC_ABM1_LS_FILTERED_MIN_MAX_LUMA__ABM1_LS_FILTERED_MAX_LUMA_MASK 0x3ff0000
+#define DC_ABM1_LS_FILTERED_MIN_MAX_LUMA__ABM1_LS_FILTERED_MAX_LUMA__SHIFT 0x10
+#define DC_ABM1_LS_PIXEL_COUNT__ABM1_LS_PIXEL_COUNT_MASK 0xffffff
+#define DC_ABM1_LS_PIXEL_COUNT__ABM1_LS_PIXEL_COUNT__SHIFT 0x0
+#define DC_ABM1_LS_OVR_SCAN_BIN__ABM1_LS_OVR_SCAN_BIN_MASK 0xffffff
+#define DC_ABM1_LS_OVR_SCAN_BIN__ABM1_LS_OVR_SCAN_BIN__SHIFT 0x0
+#define DC_ABM1_LS_MIN_MAX_PIXEL_VALUE_THRES__ABM1_LS_MIN_PIXEL_VALUE_THRES_MASK 0x3ff
+#define DC_ABM1_LS_MIN_MAX_PIXEL_VALUE_THRES__ABM1_LS_MIN_PIXEL_VALUE_THRES__SHIFT 0x0
+#define DC_ABM1_LS_MIN_MAX_PIXEL_VALUE_THRES__ABM1_LS_MAX_PIXEL_VALUE_THRES_MASK 0x3ff0000
+#define DC_ABM1_LS_MIN_MAX_PIXEL_VALUE_THRES__ABM1_LS_MAX_PIXEL_VALUE_THRES__SHIFT 0x10
+#define DC_ABM1_LS_MIN_MAX_PIXEL_VALUE_THRES__ABM1_HGLS_REG_LOCK_MASK 0x80000000
+#define DC_ABM1_LS_MIN_MAX_PIXEL_VALUE_THRES__ABM1_HGLS_REG_LOCK__SHIFT 0x1f
+#define DC_ABM1_LS_MIN_PIXEL_VALUE_COUNT__ABM1_LS_MIN_PIXEL_VALUE_COUNT_MASK 0xffffff
+#define DC_ABM1_LS_MIN_PIXEL_VALUE_COUNT__ABM1_LS_MIN_PIXEL_VALUE_COUNT__SHIFT 0x0
+#define DC_ABM1_LS_MAX_PIXEL_VALUE_COUNT__ABM1_LS_MAX_PIXEL_VALUE_COUNT_MASK 0xffffff
+#define DC_ABM1_LS_MAX_PIXEL_VALUE_COUNT__ABM1_LS_MAX_PIXEL_VALUE_COUNT__SHIFT 0x0
+#define DC_ABM1_HG_SAMPLE_RATE__ABM1_HG_SAMPLE_RATE_COUNT_EN_MASK 0x1
+#define DC_ABM1_HG_SAMPLE_RATE__ABM1_HG_SAMPLE_RATE_COUNT_EN__SHIFT 0x0
+#define DC_ABM1_HG_SAMPLE_RATE__ABM1_HG_RESET_SAMPLE_RATE_FRAME_COUNTER_MASK 0x2
+#define DC_ABM1_HG_SAMPLE_RATE__ABM1_HG_RESET_SAMPLE_RATE_FRAME_COUNTER__SHIFT 0x1
+#define DC_ABM1_HG_SAMPLE_RATE__ABM1_HG_SAMPLE_RATE_FRAME_COUNT_MASK 0xff00
+#define DC_ABM1_HG_SAMPLE_RATE__ABM1_HG_SAMPLE_RATE_FRAME_COUNT__SHIFT 0x8
+#define DC_ABM1_HG_SAMPLE_RATE__ABM1_HG_INITIAL_SAMPLE_RATE_COUNT_VALUE_WHEN_RESET_MASK 0xff0000
+#define DC_ABM1_HG_SAMPLE_RATE__ABM1_HG_INITIAL_SAMPLE_RATE_COUNT_VALUE_WHEN_RESET__SHIFT 0x10
+#define DC_ABM1_HG_SAMPLE_RATE__ABM1_HGLS_REG_LOCK_MASK 0x80000000
+#define DC_ABM1_HG_SAMPLE_RATE__ABM1_HGLS_REG_LOCK__SHIFT 0x1f
+#define DC_ABM1_LS_SAMPLE_RATE__ABM1_LS_SAMPLE_RATE_COUNT_EN_MASK 0x1
+#define DC_ABM1_LS_SAMPLE_RATE__ABM1_LS_SAMPLE_RATE_COUNT_EN__SHIFT 0x0
+#define DC_ABM1_LS_SAMPLE_RATE__ABM1_LS_RESET_SAMPLE_RATE_FRAME_COUNTER_MASK 0x2
+#define DC_ABM1_LS_SAMPLE_RATE__ABM1_LS_RESET_SAMPLE_RATE_FRAME_COUNTER__SHIFT 0x1
+#define DC_ABM1_LS_SAMPLE_RATE__ABM1_LS_SAMPLE_RATE_FRAME_COUNT_MASK 0xff00
+#define DC_ABM1_LS_SAMPLE_RATE__ABM1_LS_SAMPLE_RATE_FRAME_COUNT__SHIFT 0x8
+#define DC_ABM1_LS_SAMPLE_RATE__ABM1_LS_INITIAL_SAMPLE_RATE_COUNT_VALUE_WHEN_RESET_MASK 0xff0000
+#define DC_ABM1_LS_SAMPLE_RATE__ABM1_LS_INITIAL_SAMPLE_RATE_COUNT_VALUE_WHEN_RESET__SHIFT 0x10
+#define DC_ABM1_LS_SAMPLE_RATE__ABM1_HGLS_REG_LOCK_MASK 0x80000000
+#define DC_ABM1_LS_SAMPLE_RATE__ABM1_HGLS_REG_LOCK__SHIFT 0x1f
+#define DC_ABM1_HG_BIN_1_32_SHIFT_FLAG__ABM1_HG_BIN_1_32_SHIFT_FLAG_MASK 0xffffffff
+#define DC_ABM1_HG_BIN_1_32_SHIFT_FLAG__ABM1_HG_BIN_1_32_SHIFT_FLAG__SHIFT 0x0
+#define DC_ABM1_HG_BIN_1_8_SHIFT_INDEX__ABM1_HG_BIN_1_8_SHIFT_INDEX_MASK 0xffffffff
+#define DC_ABM1_HG_BIN_1_8_SHIFT_INDEX__ABM1_HG_BIN_1_8_SHIFT_INDEX__SHIFT 0x0
+#define DC_ABM1_HG_BIN_9_16_SHIFT_INDEX__ABM1_HG_BIN_9_16_SHIFT_INDEX_MASK 0xffffffff
+#define DC_ABM1_HG_BIN_9_16_SHIFT_INDEX__ABM1_HG_BIN_9_16_SHIFT_INDEX__SHIFT 0x0
+#define DC_ABM1_HG_BIN_17_24_SHIFT_INDEX__ABM1_HG_BIN_17_24_SHIFT_INDEX_MASK 0xffffffff
+#define DC_ABM1_HG_BIN_17_24_SHIFT_INDEX__ABM1_HG_BIN_17_24_SHIFT_INDEX__SHIFT 0x0
+#define DC_ABM1_HG_BIN_25_32_SHIFT_INDEX__ABM1_HG_BIN_25_32_SHIFT_INDEX_MASK 0xffffffff
+#define DC_ABM1_HG_BIN_25_32_SHIFT_INDEX__ABM1_HG_BIN_25_32_SHIFT_INDEX__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_1__ABM1_HG_RESULT_1_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_1__ABM1_HG_RESULT_1__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_2__ABM1_HG_RESULT_2_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_2__ABM1_HG_RESULT_2__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_3__ABM1_HG_RESULT_3_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_3__ABM1_HG_RESULT_3__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_4__ABM1_HG_RESULT_4_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_4__ABM1_HG_RESULT_4__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_5__ABM1_HG_RESULT_5_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_5__ABM1_HG_RESULT_5__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_6__ABM1_HG_RESULT_6_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_6__ABM1_HG_RESULT_6__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_7__ABM1_HG_RESULT_7_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_7__ABM1_HG_RESULT_7__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_8__ABM1_HG_RESULT_8_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_8__ABM1_HG_RESULT_8__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_9__ABM1_HG_RESULT_9_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_9__ABM1_HG_RESULT_9__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_10__ABM1_HG_RESULT_10_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_10__ABM1_HG_RESULT_10__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_11__ABM1_HG_RESULT_11_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_11__ABM1_HG_RESULT_11__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_12__ABM1_HG_RESULT_12_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_12__ABM1_HG_RESULT_12__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_13__ABM1_HG_RESULT_13_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_13__ABM1_HG_RESULT_13__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_14__ABM1_HG_RESULT_14_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_14__ABM1_HG_RESULT_14__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_15__ABM1_HG_RESULT_15_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_15__ABM1_HG_RESULT_15__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_16__ABM1_HG_RESULT_16_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_16__ABM1_HG_RESULT_16__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_17__ABM1_HG_RESULT_17_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_17__ABM1_HG_RESULT_17__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_18__ABM1_HG_RESULT_18_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_18__ABM1_HG_RESULT_18__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_19__ABM1_HG_RESULT_19_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_19__ABM1_HG_RESULT_19__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_20__ABM1_HG_RESULT_20_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_20__ABM1_HG_RESULT_20__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_21__ABM1_HG_RESULT_21_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_21__ABM1_HG_RESULT_21__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_22__ABM1_HG_RESULT_22_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_22__ABM1_HG_RESULT_22__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_23__ABM1_HG_RESULT_23_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_23__ABM1_HG_RESULT_23__SHIFT 0x0
+#define DC_ABM1_HG_RESULT_24__ABM1_HG_RESULT_24_MASK 0xffffffff
+#define DC_ABM1_HG_RESULT_24__ABM1_HG_RESULT_24__SHIFT 0x0
+#define DC_ABM1_OVERSCAN_PIXEL_VALUE__ABM1_OVERSCAN_R_PIXEL_VALUE_MASK 0x3ff
+#define DC_ABM1_OVERSCAN_PIXEL_VALUE__ABM1_OVERSCAN_R_PIXEL_VALUE__SHIFT 0x0
+#define DC_ABM1_OVERSCAN_PIXEL_VALUE__ABM1_OVERSCAN_G_PIXEL_VALUE_MASK 0xffc00
+#define DC_ABM1_OVERSCAN_PIXEL_VALUE__ABM1_OVERSCAN_G_PIXEL_VALUE__SHIFT 0xa
+#define DC_ABM1_OVERSCAN_PIXEL_VALUE__ABM1_OVERSCAN_B_PIXEL_VALUE_MASK 0x3ff00000
+#define DC_ABM1_OVERSCAN_PIXEL_VALUE__ABM1_OVERSCAN_B_PIXEL_VALUE__SHIFT 0x14
+#define DC_ABM1_BL_MASTER_LOCK__ABM1_BL_MASTER_LOCK_MASK 0x80000000
+#define DC_ABM1_BL_MASTER_LOCK__ABM1_BL_MASTER_LOCK__SHIFT 0x1f
+#define ABM_TEST_DEBUG_INDEX__ABM_TEST_DEBUG_INDEX_MASK 0xff
+#define ABM_TEST_DEBUG_INDEX__ABM_TEST_DEBUG_INDEX__SHIFT 0x0
+#define ABM_TEST_DEBUG_INDEX__ABM_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define ABM_TEST_DEBUG_INDEX__ABM_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define ABM_TEST_DEBUG_DATA__ABM_TEST_DEBUG_DATA_MASK 0xffffffff
+#define ABM_TEST_DEBUG_DATA__ABM_TEST_DEBUG_DATA__SHIFT 0x0
+#define CRTC_H_BLANK_EARLY_NUM__CRTC_H_BLANK_EARLY_NUM_MASK 0x3ff
+#define CRTC_H_BLANK_EARLY_NUM__CRTC_H_BLANK_EARLY_NUM__SHIFT 0x0
+#define CRTC_H_BLANK_EARLY_NUM__CRTC_H_BLANK_EARLY_NUM_DIS_MASK 0x10000
+#define CRTC_H_BLANK_EARLY_NUM__CRTC_H_BLANK_EARLY_NUM_DIS__SHIFT 0x10
+#define CRTC_H_TOTAL__CRTC_H_TOTAL_MASK 0x3fff
+#define CRTC_H_TOTAL__CRTC_H_TOTAL__SHIFT 0x0
+#define CRTC_H_BLANK_START_END__CRTC_H_BLANK_START_MASK 0x3fff
+#define CRTC_H_BLANK_START_END__CRTC_H_BLANK_START__SHIFT 0x0
+#define CRTC_H_BLANK_START_END__CRTC_H_BLANK_END_MASK 0x3fff0000
+#define CRTC_H_BLANK_START_END__CRTC_H_BLANK_END__SHIFT 0x10
+#define CRTC_H_SYNC_A__CRTC_H_SYNC_A_START_MASK 0x3fff
+#define CRTC_H_SYNC_A__CRTC_H_SYNC_A_START__SHIFT 0x0
+#define CRTC_H_SYNC_A__CRTC_H_SYNC_A_END_MASK 0x3fff0000
+#define CRTC_H_SYNC_A__CRTC_H_SYNC_A_END__SHIFT 0x10
+#define CRTC_H_SYNC_A_CNTL__CRTC_H_SYNC_A_POL_MASK 0x1
+#define CRTC_H_SYNC_A_CNTL__CRTC_H_SYNC_A_POL__SHIFT 0x0
+#define CRTC_H_SYNC_A_CNTL__CRTC_COMP_SYNC_A_EN_MASK 0x10000
+#define CRTC_H_SYNC_A_CNTL__CRTC_COMP_SYNC_A_EN__SHIFT 0x10
+#define CRTC_H_SYNC_A_CNTL__CRTC_H_SYNC_A_CUTOFF_MASK 0x20000
+#define CRTC_H_SYNC_A_CNTL__CRTC_H_SYNC_A_CUTOFF__SHIFT 0x11
+#define CRTC_H_SYNC_B__CRTC_H_SYNC_B_START_MASK 0x3fff
+#define CRTC_H_SYNC_B__CRTC_H_SYNC_B_START__SHIFT 0x0
+#define CRTC_H_SYNC_B__CRTC_H_SYNC_B_END_MASK 0x3fff0000
+#define CRTC_H_SYNC_B__CRTC_H_SYNC_B_END__SHIFT 0x10
+#define CRTC_H_SYNC_B_CNTL__CRTC_H_SYNC_B_POL_MASK 0x1
+#define CRTC_H_SYNC_B_CNTL__CRTC_H_SYNC_B_POL__SHIFT 0x0
+#define CRTC_H_SYNC_B_CNTL__CRTC_COMP_SYNC_B_EN_MASK 0x10000
+#define CRTC_H_SYNC_B_CNTL__CRTC_COMP_SYNC_B_EN__SHIFT 0x10
+#define CRTC_H_SYNC_B_CNTL__CRTC_H_SYNC_B_CUTOFF_MASK 0x20000
+#define CRTC_H_SYNC_B_CNTL__CRTC_H_SYNC_B_CUTOFF__SHIFT 0x11
+#define CRTC_VBI_END__CRTC_VBI_V_END_MASK 0x3fff
+#define CRTC_VBI_END__CRTC_VBI_V_END__SHIFT 0x0
+#define CRTC_VBI_END__CRTC_VBI_H_END_MASK 0x3fff0000
+#define CRTC_VBI_END__CRTC_VBI_H_END__SHIFT 0x10
+#define CRTC_V_TOTAL__CRTC_V_TOTAL_MASK 0x3fff
+#define CRTC_V_TOTAL__CRTC_V_TOTAL__SHIFT 0x0
+#define CRTC_V_TOTAL_MIN__CRTC_V_TOTAL_MIN_MASK 0x3fff
+#define CRTC_V_TOTAL_MIN__CRTC_V_TOTAL_MIN__SHIFT 0x0
+#define CRTC_V_TOTAL_MAX__CRTC_V_TOTAL_MAX_MASK 0x3fff
+#define CRTC_V_TOTAL_MAX__CRTC_V_TOTAL_MAX__SHIFT 0x0
+#define CRTC_V_TOTAL_MAX__CRTC_ALLOW_VBLANK_EXTENSION_FOR_MC_TRAINING_MASK 0x10000
+#define CRTC_V_TOTAL_MAX__CRTC_ALLOW_VBLANK_EXTENSION_FOR_MC_TRAINING__SHIFT 0x10
+#define CRTC_V_TOTAL_CONTROL__CRTC_V_TOTAL_MIN_SEL_MASK 0x1
+#define CRTC_V_TOTAL_CONTROL__CRTC_V_TOTAL_MIN_SEL__SHIFT 0x0
+#define CRTC_V_TOTAL_CONTROL__CRTC_V_TOTAL_MAX_SEL_MASK 0x10
+#define CRTC_V_TOTAL_CONTROL__CRTC_V_TOTAL_MAX_SEL__SHIFT 0x4
+#define CRTC_V_TOTAL_CONTROL__CRTC_FORCE_LOCK_ON_EVENT_MASK 0x100
+#define CRTC_V_TOTAL_CONTROL__CRTC_FORCE_LOCK_ON_EVENT__SHIFT 0x8
+#define CRTC_V_TOTAL_CONTROL__CRTC_FORCE_LOCK_TO_MASTER_VSYNC_MASK 0x1000
+#define CRTC_V_TOTAL_CONTROL__CRTC_FORCE_LOCK_TO_MASTER_VSYNC__SHIFT 0xc
+#define CRTC_V_TOTAL_CONTROL__CRTC_SET_V_TOTAL_MIN_MASK_EN_MASK 0x8000
+#define CRTC_V_TOTAL_CONTROL__CRTC_SET_V_TOTAL_MIN_MASK_EN__SHIFT 0xf
+#define CRTC_V_TOTAL_CONTROL__CRTC_SET_V_TOTAL_MIN_MASK_MASK 0xffff0000
+#define CRTC_V_TOTAL_CONTROL__CRTC_SET_V_TOTAL_MIN_MASK__SHIFT 0x10
+#define CRTC_V_TOTAL_INT_STATUS__CRTC_SET_V_TOTAL_MIN_EVENT_OCCURED_MASK 0x1
+#define CRTC_V_TOTAL_INT_STATUS__CRTC_SET_V_TOTAL_MIN_EVENT_OCCURED__SHIFT 0x0
+#define CRTC_V_TOTAL_INT_STATUS__CRTC_SET_V_TOTAL_MIN_EVENT_OCCURED_INT_MASK 0x10
+#define CRTC_V_TOTAL_INT_STATUS__CRTC_SET_V_TOTAL_MIN_EVENT_OCCURED_INT__SHIFT 0x4
+#define CRTC_V_TOTAL_INT_STATUS__CRTC_SET_V_TOTAL_MIN_EVENT_OCCURED_ACK_MASK 0x100
+#define CRTC_V_TOTAL_INT_STATUS__CRTC_SET_V_TOTAL_MIN_EVENT_OCCURED_ACK__SHIFT 0x8
+#define CRTC_V_TOTAL_INT_STATUS__CRTC_SET_V_TOTAL_MIN_EVENT_OCCURED_MSK_MASK 0x1000
+#define CRTC_V_TOTAL_INT_STATUS__CRTC_SET_V_TOTAL_MIN_EVENT_OCCURED_MSK__SHIFT 0xc
+#define CRTC_VSYNC_NOM_INT_STATUS__CRTC_VSYNC_NOM_MASK 0x1
+#define CRTC_VSYNC_NOM_INT_STATUS__CRTC_VSYNC_NOM__SHIFT 0x0
+#define CRTC_VSYNC_NOM_INT_STATUS__CRTC_VSYNC_NOM_INT_CLEAR_MASK 0x10
+#define CRTC_VSYNC_NOM_INT_STATUS__CRTC_VSYNC_NOM_INT_CLEAR__SHIFT 0x4
+#define CRTC_V_BLANK_START_END__CRTC_V_BLANK_START_MASK 0x3fff
+#define CRTC_V_BLANK_START_END__CRTC_V_BLANK_START__SHIFT 0x0
+#define CRTC_V_BLANK_START_END__CRTC_V_BLANK_END_MASK 0x3fff0000
+#define CRTC_V_BLANK_START_END__CRTC_V_BLANK_END__SHIFT 0x10
+#define CRTC_V_SYNC_A__CRTC_V_SYNC_A_START_MASK 0x3fff
+#define CRTC_V_SYNC_A__CRTC_V_SYNC_A_START__SHIFT 0x0
+#define CRTC_V_SYNC_A__CRTC_V_SYNC_A_END_MASK 0x3fff0000
+#define CRTC_V_SYNC_A__CRTC_V_SYNC_A_END__SHIFT 0x10
+#define CRTC_V_SYNC_A_CNTL__CRTC_V_SYNC_A_POL_MASK 0x1
+#define CRTC_V_SYNC_A_CNTL__CRTC_V_SYNC_A_POL__SHIFT 0x0
+#define CRTC_V_SYNC_B__CRTC_V_SYNC_B_START_MASK 0x3fff
+#define CRTC_V_SYNC_B__CRTC_V_SYNC_B_START__SHIFT 0x0
+#define CRTC_V_SYNC_B__CRTC_V_SYNC_B_END_MASK 0x3fff0000
+#define CRTC_V_SYNC_B__CRTC_V_SYNC_B_END__SHIFT 0x10
+#define CRTC_V_SYNC_B_CNTL__CRTC_V_SYNC_B_POL_MASK 0x1
+#define CRTC_V_SYNC_B_CNTL__CRTC_V_SYNC_B_POL__SHIFT 0x0
+#define CRTC_DTMTEST_CNTL__CRTC_DTMTEST_CRTC_EN_MASK 0x1
+#define CRTC_DTMTEST_CNTL__CRTC_DTMTEST_CRTC_EN__SHIFT 0x0
+#define CRTC_DTMTEST_CNTL__CRTC_DTMTEST_CLK_DIV_MASK 0x1e
+#define CRTC_DTMTEST_CNTL__CRTC_DTMTEST_CLK_DIV__SHIFT 0x1
+#define CRTC_DTMTEST_STATUS_POSITION__CRTC_DTMTEST_VERT_COUNT_MASK 0x3fff
+#define CRTC_DTMTEST_STATUS_POSITION__CRTC_DTMTEST_VERT_COUNT__SHIFT 0x0
+#define CRTC_DTMTEST_STATUS_POSITION__CRTC_DTMTEST_HORZ_COUNT_MASK 0x3fff0000
+#define CRTC_DTMTEST_STATUS_POSITION__CRTC_DTMTEST_HORZ_COUNT__SHIFT 0x10
+#define CRTC_TRIGA_CNTL__CRTC_TRIGA_SOURCE_SELECT_MASK 0x1f
+#define CRTC_TRIGA_CNTL__CRTC_TRIGA_SOURCE_SELECT__SHIFT 0x0
+#define CRTC_TRIGA_CNTL__CRTC_TRIGA_POLARITY_SELECT_MASK 0xe0
+#define CRTC_TRIGA_CNTL__CRTC_TRIGA_POLARITY_SELECT__SHIFT 0x5
+#define CRTC_TRIGA_CNTL__CRTC_TRIGA_RESYNC_BYPASS_EN_MASK 0x100
+#define CRTC_TRIGA_CNTL__CRTC_TRIGA_RESYNC_BYPASS_EN__SHIFT 0x8
+#define CRTC_TRIGA_CNTL__CRTC_TRIGA_INPUT_STATUS_MASK 0x200
+#define CRTC_TRIGA_CNTL__CRTC_TRIGA_INPUT_STATUS__SHIFT 0x9
+#define CRTC_TRIGA_CNTL__CRTC_TRIGA_POLARITY_STATUS_MASK 0x400
+#define CRTC_TRIGA_CNTL__CRTC_TRIGA_POLARITY_STATUS__SHIFT 0xa
+#define CRTC_TRIGA_CNTL__CRTC_TRIGA_OCCURRED_MASK 0x800
+#define CRTC_TRIGA_CNTL__CRTC_TRIGA_OCCURRED__SHIFT 0xb
+#define CRTC_TRIGA_CNTL__CRTC_TRIGA_RISING_EDGE_DETECT_CNTL_MASK 0x3000
+#define CRTC_TRIGA_CNTL__CRTC_TRIGA_RISING_EDGE_DETECT_CNTL__SHIFT 0xc
+#define CRTC_TRIGA_CNTL__CRTC_TRIGA_FALLING_EDGE_DETECT_CNTL_MASK 0x30000
+#define CRTC_TRIGA_CNTL__CRTC_TRIGA_FALLING_EDGE_DETECT_CNTL__SHIFT 0x10
+#define CRTC_TRIGA_CNTL__CRTC_TRIGA_FREQUENCY_SELECT_MASK 0x300000
+#define CRTC_TRIGA_CNTL__CRTC_TRIGA_FREQUENCY_SELECT__SHIFT 0x14
+#define CRTC_TRIGA_CNTL__CRTC_TRIGA_DELAY_MASK 0x1f000000
+#define CRTC_TRIGA_CNTL__CRTC_TRIGA_DELAY__SHIFT 0x18
+#define CRTC_TRIGA_CNTL__CRTC_TRIGA_CLEAR_MASK 0x80000000
+#define CRTC_TRIGA_CNTL__CRTC_TRIGA_CLEAR__SHIFT 0x1f
+#define CRTC_TRIGA_MANUAL_TRIG__CRTC_TRIGA_MANUAL_TRIG_MASK 0x1
+#define CRTC_TRIGA_MANUAL_TRIG__CRTC_TRIGA_MANUAL_TRIG__SHIFT 0x0
+#define CRTC_TRIGB_CNTL__CRTC_TRIGB_SOURCE_SELECT_MASK 0x1f
+#define CRTC_TRIGB_CNTL__CRTC_TRIGB_SOURCE_SELECT__SHIFT 0x0
+#define CRTC_TRIGB_CNTL__CRTC_TRIGB_POLARITY_SELECT_MASK 0xe0
+#define CRTC_TRIGB_CNTL__CRTC_TRIGB_POLARITY_SELECT__SHIFT 0x5
+#define CRTC_TRIGB_CNTL__CRTC_TRIGB_RESYNC_BYPASS_EN_MASK 0x100
+#define CRTC_TRIGB_CNTL__CRTC_TRIGB_RESYNC_BYPASS_EN__SHIFT 0x8
+#define CRTC_TRIGB_CNTL__CRTC_TRIGB_INPUT_STATUS_MASK 0x200
+#define CRTC_TRIGB_CNTL__CRTC_TRIGB_INPUT_STATUS__SHIFT 0x9
+#define CRTC_TRIGB_CNTL__CRTC_TRIGB_POLARITY_STATUS_MASK 0x400
+#define CRTC_TRIGB_CNTL__CRTC_TRIGB_POLARITY_STATUS__SHIFT 0xa
+#define CRTC_TRIGB_CNTL__CRTC_TRIGB_OCCURRED_MASK 0x800
+#define CRTC_TRIGB_CNTL__CRTC_TRIGB_OCCURRED__SHIFT 0xb
+#define CRTC_TRIGB_CNTL__CRTC_TRIGB_RISING_EDGE_DETECT_CNTL_MASK 0x3000
+#define CRTC_TRIGB_CNTL__CRTC_TRIGB_RISING_EDGE_DETECT_CNTL__SHIFT 0xc
+#define CRTC_TRIGB_CNTL__CRTC_TRIGB_FALLING_EDGE_DETECT_CNTL_MASK 0x30000
+#define CRTC_TRIGB_CNTL__CRTC_TRIGB_FALLING_EDGE_DETECT_CNTL__SHIFT 0x10
+#define CRTC_TRIGB_CNTL__CRTC_TRIGB_FREQUENCY_SELECT_MASK 0x300000
+#define CRTC_TRIGB_CNTL__CRTC_TRIGB_FREQUENCY_SELECT__SHIFT 0x14
+#define CRTC_TRIGB_CNTL__CRTC_TRIGB_DELAY_MASK 0x1f000000
+#define CRTC_TRIGB_CNTL__CRTC_TRIGB_DELAY__SHIFT 0x18
+#define CRTC_TRIGB_CNTL__CRTC_TRIGB_CLEAR_MASK 0x80000000
+#define CRTC_TRIGB_CNTL__CRTC_TRIGB_CLEAR__SHIFT 0x1f
+#define CRTC_TRIGB_MANUAL_TRIG__CRTC_TRIGB_MANUAL_TRIG_MASK 0x1
+#define CRTC_TRIGB_MANUAL_TRIG__CRTC_TRIGB_MANUAL_TRIG__SHIFT 0x0
+#define CRTC_FORCE_COUNT_NOW_CNTL__CRTC_FORCE_COUNT_NOW_MODE_MASK 0x3
+#define CRTC_FORCE_COUNT_NOW_CNTL__CRTC_FORCE_COUNT_NOW_MODE__SHIFT 0x0
+#define CRTC_FORCE_COUNT_NOW_CNTL__CRTC_FORCE_COUNT_NOW_CHECK_MASK 0x10
+#define CRTC_FORCE_COUNT_NOW_CNTL__CRTC_FORCE_COUNT_NOW_CHECK__SHIFT 0x4
+#define CRTC_FORCE_COUNT_NOW_CNTL__CRTC_FORCE_COUNT_NOW_TRIG_SEL_MASK 0x100
+#define CRTC_FORCE_COUNT_NOW_CNTL__CRTC_FORCE_COUNT_NOW_TRIG_SEL__SHIFT 0x8
+#define CRTC_FORCE_COUNT_NOW_CNTL__CRTC_FORCE_COUNT_NOW_OCCURRED_MASK 0x10000
+#define CRTC_FORCE_COUNT_NOW_CNTL__CRTC_FORCE_COUNT_NOW_OCCURRED__SHIFT 0x10
+#define CRTC_FORCE_COUNT_NOW_CNTL__CRTC_FORCE_COUNT_NOW_CLEAR_MASK 0x1000000
+#define CRTC_FORCE_COUNT_NOW_CNTL__CRTC_FORCE_COUNT_NOW_CLEAR__SHIFT 0x18
+#define CRTC_FLOW_CONTROL__CRTC_FLOW_CONTROL_SOURCE_SELECT_MASK 0x1f
+#define CRTC_FLOW_CONTROL__CRTC_FLOW_CONTROL_SOURCE_SELECT__SHIFT 0x0
+#define CRTC_FLOW_CONTROL__CRTC_FLOW_CONTROL_POLARITY_MASK 0x100
+#define CRTC_FLOW_CONTROL__CRTC_FLOW_CONTROL_POLARITY__SHIFT 0x8
+#define CRTC_FLOW_CONTROL__CRTC_FLOW_CONTROL_GRANULARITY_MASK 0x10000
+#define CRTC_FLOW_CONTROL__CRTC_FLOW_CONTROL_GRANULARITY__SHIFT 0x10
+#define CRTC_FLOW_CONTROL__CRTC_FLOW_CONTROL_INPUT_STATUS_MASK 0x1000000
+#define CRTC_FLOW_CONTROL__CRTC_FLOW_CONTROL_INPUT_STATUS__SHIFT 0x18
+#define CRTC_STEREO_FORCE_NEXT_EYE__CRTC_STEREO_FORCE_NEXT_EYE_MASK 0x3
+#define CRTC_STEREO_FORCE_NEXT_EYE__CRTC_STEREO_FORCE_NEXT_EYE__SHIFT 0x0
+#define CRTC_STEREO_FORCE_NEXT_EYE__CRTC_AVSYNC_FRAME_COUNTER_MASK 0xff00
+#define CRTC_STEREO_FORCE_NEXT_EYE__CRTC_AVSYNC_FRAME_COUNTER__SHIFT 0x8
+#define CRTC_STEREO_FORCE_NEXT_EYE__CRTC_AVSYNC_LINE_COUNTER_MASK 0x1fff0000
+#define CRTC_STEREO_FORCE_NEXT_EYE__CRTC_AVSYNC_LINE_COUNTER__SHIFT 0x10
+#define CRTC_AVSYNC_COUNTER__CRTC_AVSYNC_COUNTER_MASK 0xffffffff
+#define CRTC_AVSYNC_COUNTER__CRTC_AVSYNC_COUNTER__SHIFT 0x0
+#define CRTC_CONTROL__CRTC_MASTER_EN_MASK 0x1
+#define CRTC_CONTROL__CRTC_MASTER_EN__SHIFT 0x0
+#define CRTC_CONTROL__CRTC_SYNC_RESET_SEL_MASK 0x10
+#define CRTC_CONTROL__CRTC_SYNC_RESET_SEL__SHIFT 0x4
+#define CRTC_CONTROL__CRTC_DISABLE_POINT_CNTL_MASK 0x300
+#define CRTC_CONTROL__CRTC_DISABLE_POINT_CNTL__SHIFT 0x8
+#define CRTC_CONTROL__CRTC_START_POINT_CNTL_MASK 0x1000
+#define CRTC_CONTROL__CRTC_START_POINT_CNTL__SHIFT 0xc
+#define CRTC_CONTROL__CRTC_FIELD_NUMBER_CNTL_MASK 0x2000
+#define CRTC_CONTROL__CRTC_FIELD_NUMBER_CNTL__SHIFT 0xd
+#define CRTC_CONTROL__CRTC_FIELD_NUMBER_POLARITY_MASK 0x4000
+#define CRTC_CONTROL__CRTC_FIELD_NUMBER_POLARITY__SHIFT 0xe
+#define CRTC_CONTROL__CRTC_CURRENT_MASTER_EN_STATE_MASK 0x10000
+#define CRTC_CONTROL__CRTC_CURRENT_MASTER_EN_STATE__SHIFT 0x10
+#define CRTC_CONTROL__CRTC_HBLANK_EARLY_CONTROL_MASK 0x700000
+#define CRTC_CONTROL__CRTC_HBLANK_EARLY_CONTROL__SHIFT 0x14
+#define CRTC_CONTROL__CRTC_DISP_READ_REQUEST_DISABLE_MASK 0x1000000
+#define CRTC_CONTROL__CRTC_DISP_READ_REQUEST_DISABLE__SHIFT 0x18
+#define CRTC_CONTROL__CRTC_SOF_PULL_EN_MASK 0x20000000
+#define CRTC_CONTROL__CRTC_SOF_PULL_EN__SHIFT 0x1d
+#define CRTC_CONTROL__CRTC_AVSYNC_LOCK_SNAPSHOT_MASK 0x40000000
+#define CRTC_CONTROL__CRTC_AVSYNC_LOCK_SNAPSHOT__SHIFT 0x1e
+#define CRTC_CONTROL__CRTC_AVSYNC_VSYNC_N_HSYNC_MODE_MASK 0x80000000
+#define CRTC_CONTROL__CRTC_AVSYNC_VSYNC_N_HSYNC_MODE__SHIFT 0x1f
+#define CRTC_BLANK_CONTROL__CRTC_CURRENT_BLANK_STATE_MASK 0x1
+#define CRTC_BLANK_CONTROL__CRTC_CURRENT_BLANK_STATE__SHIFT 0x0
+#define CRTC_BLANK_CONTROL__CRTC_BLANK_DATA_EN_MASK 0x100
+#define CRTC_BLANK_CONTROL__CRTC_BLANK_DATA_EN__SHIFT 0x8
+#define CRTC_BLANK_CONTROL__CRTC_BLANK_DE_MODE_MASK 0x10000
+#define CRTC_BLANK_CONTROL__CRTC_BLANK_DE_MODE__SHIFT 0x10
+#define CRTC_INTERLACE_CONTROL__CRTC_INTERLACE_ENABLE_MASK 0x1
+#define CRTC_INTERLACE_CONTROL__CRTC_INTERLACE_ENABLE__SHIFT 0x0
+#define CRTC_INTERLACE_CONTROL__CRTC_INTERLACE_FORCE_NEXT_FIELD_MASK 0x30000
+#define CRTC_INTERLACE_CONTROL__CRTC_INTERLACE_FORCE_NEXT_FIELD__SHIFT 0x10
+#define CRTC_INTERLACE_STATUS__CRTC_INTERLACE_CURRENT_FIELD_MASK 0x1
+#define CRTC_INTERLACE_STATUS__CRTC_INTERLACE_CURRENT_FIELD__SHIFT 0x0
+#define CRTC_INTERLACE_STATUS__CRTC_INTERLACE_NEXT_FIELD_MASK 0x2
+#define CRTC_INTERLACE_STATUS__CRTC_INTERLACE_NEXT_FIELD__SHIFT 0x1
+#define CRTC_FIELD_INDICATION_CONTROL__CRTC_FIELD_INDICATION_OUTPUT_POLARITY_MASK 0x1
+#define CRTC_FIELD_INDICATION_CONTROL__CRTC_FIELD_INDICATION_OUTPUT_POLARITY__SHIFT 0x0
+#define CRTC_FIELD_INDICATION_CONTROL__CRTC_FIELD_ALIGNMENT_MASK 0x2
+#define CRTC_FIELD_INDICATION_CONTROL__CRTC_FIELD_ALIGNMENT__SHIFT 0x1
+#define CRTC_PIXEL_DATA_READBACK0__CRTC_PIXEL_DATA_BLUE_CB_MASK 0xfff
+#define CRTC_PIXEL_DATA_READBACK0__CRTC_PIXEL_DATA_BLUE_CB__SHIFT 0x0
+#define CRTC_PIXEL_DATA_READBACK0__CRTC_PIXEL_DATA_GREEN_Y_MASK 0xfff0000
+#define CRTC_PIXEL_DATA_READBACK0__CRTC_PIXEL_DATA_GREEN_Y__SHIFT 0x10
+#define CRTC_PIXEL_DATA_READBACK1__CRTC_PIXEL_DATA_RED_CR_MASK 0xfff
+#define CRTC_PIXEL_DATA_READBACK1__CRTC_PIXEL_DATA_RED_CR__SHIFT 0x0
+#define CRTC_STATUS__CRTC_V_BLANK_MASK 0x1
+#define CRTC_STATUS__CRTC_V_BLANK__SHIFT 0x0
+#define CRTC_STATUS__CRTC_V_ACTIVE_DISP_MASK 0x2
+#define CRTC_STATUS__CRTC_V_ACTIVE_DISP__SHIFT 0x1
+#define CRTC_STATUS__CRTC_V_SYNC_A_MASK 0x4
+#define CRTC_STATUS__CRTC_V_SYNC_A__SHIFT 0x2
+#define CRTC_STATUS__CRTC_V_UPDATE_MASK 0x8
+#define CRTC_STATUS__CRTC_V_UPDATE__SHIFT 0x3
+#define CRTC_STATUS__CRTC_V_START_LINE_MASK 0x10
+#define CRTC_STATUS__CRTC_V_START_LINE__SHIFT 0x4
+#define CRTC_STATUS__CRTC_V_BLANK_3D_STRUCTURE_MASK 0x20
+#define CRTC_STATUS__CRTC_V_BLANK_3D_STRUCTURE__SHIFT 0x5
+#define CRTC_STATUS__CRTC_H_BLANK_MASK 0x10000
+#define CRTC_STATUS__CRTC_H_BLANK__SHIFT 0x10
+#define CRTC_STATUS__CRTC_H_ACTIVE_DISP_MASK 0x20000
+#define CRTC_STATUS__CRTC_H_ACTIVE_DISP__SHIFT 0x11
+#define CRTC_STATUS__CRTC_H_SYNC_A_MASK 0x40000
+#define CRTC_STATUS__CRTC_H_SYNC_A__SHIFT 0x12
+#define CRTC_STATUS_POSITION__CRTC_VERT_COUNT_MASK 0x3fff
+#define CRTC_STATUS_POSITION__CRTC_VERT_COUNT__SHIFT 0x0
+#define CRTC_STATUS_POSITION__CRTC_HORZ_COUNT_MASK 0x3fff0000
+#define CRTC_STATUS_POSITION__CRTC_HORZ_COUNT__SHIFT 0x10
+#define CRTC_NOM_VERT_POSITION__CRTC_VERT_COUNT_NOM_MASK 0x3fff
+#define CRTC_NOM_VERT_POSITION__CRTC_VERT_COUNT_NOM__SHIFT 0x0
+#define CRTC_STATUS_FRAME_COUNT__CRTC_FRAME_COUNT_MASK 0xffffff
+#define CRTC_STATUS_FRAME_COUNT__CRTC_FRAME_COUNT__SHIFT 0x0
+#define CRTC_STATUS_VF_COUNT__CRTC_VF_COUNT_MASK 0x3fffffff
+#define CRTC_STATUS_VF_COUNT__CRTC_VF_COUNT__SHIFT 0x0
+#define CRTC_STATUS_HV_COUNT__CRTC_HV_COUNT_MASK 0x3fffffff
+#define CRTC_STATUS_HV_COUNT__CRTC_HV_COUNT__SHIFT 0x0
+#define CRTC_COUNT_CONTROL__CRTC_HORZ_COUNT_BY2_EN_MASK 0x1
+#define CRTC_COUNT_CONTROL__CRTC_HORZ_COUNT_BY2_EN__SHIFT 0x0
+#define CRTC_COUNT_CONTROL__CRTC_HORZ_REPETITION_COUNT_MASK 0x1e
+#define CRTC_COUNT_CONTROL__CRTC_HORZ_REPETITION_COUNT__SHIFT 0x1
+#define CRTC_COUNT_RESET__CRTC_RESET_FRAME_COUNT_MASK 0x1
+#define CRTC_COUNT_RESET__CRTC_RESET_FRAME_COUNT__SHIFT 0x0
+#define CRTC_MANUAL_FORCE_VSYNC_NEXT_LINE__CRTC_MANUAL_FORCE_VSYNC_NEXT_LINE_MASK 0x1
+#define CRTC_MANUAL_FORCE_VSYNC_NEXT_LINE__CRTC_MANUAL_FORCE_VSYNC_NEXT_LINE__SHIFT 0x0
+#define CRTC_VERT_SYNC_CONTROL__CRTC_FORCE_VSYNC_NEXT_LINE_OCCURRED_MASK 0x1
+#define CRTC_VERT_SYNC_CONTROL__CRTC_FORCE_VSYNC_NEXT_LINE_OCCURRED__SHIFT 0x0
+#define CRTC_VERT_SYNC_CONTROL__CRTC_FORCE_VSYNC_NEXT_LINE_CLEAR_MASK 0x100
+#define CRTC_VERT_SYNC_CONTROL__CRTC_FORCE_VSYNC_NEXT_LINE_CLEAR__SHIFT 0x8
+#define CRTC_VERT_SYNC_CONTROL__CRTC_AUTO_FORCE_VSYNC_MODE_MASK 0x30000
+#define CRTC_VERT_SYNC_CONTROL__CRTC_AUTO_FORCE_VSYNC_MODE__SHIFT 0x10
+#define CRTC_STEREO_STATUS__CRTC_STEREO_CURRENT_EYE_MASK 0x1
+#define CRTC_STEREO_STATUS__CRTC_STEREO_CURRENT_EYE__SHIFT 0x0
+#define CRTC_STEREO_STATUS__CRTC_STEREO_SYNC_OUTPUT_MASK 0x100
+#define CRTC_STEREO_STATUS__CRTC_STEREO_SYNC_OUTPUT__SHIFT 0x8
+#define CRTC_STEREO_STATUS__CRTC_STEREO_SYNC_SELECT_MASK 0x10000
+#define CRTC_STEREO_STATUS__CRTC_STEREO_SYNC_SELECT__SHIFT 0x10
+#define CRTC_STEREO_STATUS__CRTC_STEREO_EYE_FLAG_MASK 0x100000
+#define CRTC_STEREO_STATUS__CRTC_STEREO_EYE_FLAG__SHIFT 0x14
+#define CRTC_STEREO_STATUS__CRTC_STEREO_FORCE_NEXT_EYE_PENDING_MASK 0x3000000
+#define CRTC_STEREO_STATUS__CRTC_STEREO_FORCE_NEXT_EYE_PENDING__SHIFT 0x18
+#define CRTC_STEREO_CONTROL__CRTC_STEREO_SYNC_OUTPUT_LINE_NUM_MASK 0x3fff
+#define CRTC_STEREO_CONTROL__CRTC_STEREO_SYNC_OUTPUT_LINE_NUM__SHIFT 0x0
+#define CRTC_STEREO_CONTROL__CRTC_STEREO_SYNC_OUTPUT_POLARITY_MASK 0x8000
+#define CRTC_STEREO_CONTROL__CRTC_STEREO_SYNC_OUTPUT_POLARITY__SHIFT 0xf
+#define CRTC_STEREO_CONTROL__CRTC_STEREO_SYNC_SELECT_POLARITY_MASK 0x10000
+#define CRTC_STEREO_CONTROL__CRTC_STEREO_SYNC_SELECT_POLARITY__SHIFT 0x10
+#define CRTC_STEREO_CONTROL__CRTC_STEREO_EYE_FLAG_POLARITY_MASK 0x20000
+#define CRTC_STEREO_CONTROL__CRTC_STEREO_EYE_FLAG_POLARITY__SHIFT 0x11
+#define CRTC_STEREO_CONTROL__CRTC_DISABLE_STEREOSYNC_OUTPUT_FOR_DP_MASK 0x40000
+#define CRTC_STEREO_CONTROL__CRTC_DISABLE_STEREOSYNC_OUTPUT_FOR_DP__SHIFT 0x12
+#define CRTC_STEREO_CONTROL__CRTC_DISABLE_FIELD_NUM_MASK 0x80000
+#define CRTC_STEREO_CONTROL__CRTC_DISABLE_FIELD_NUM__SHIFT 0x13
+#define CRTC_STEREO_CONTROL__CRTC_DISABLE_V_BLANK_FOR_DP_FIX_MASK 0x100000
+#define CRTC_STEREO_CONTROL__CRTC_DISABLE_V_BLANK_FOR_DP_FIX__SHIFT 0x14
+#define CRTC_STEREO_CONTROL__CRTC_STEREO_EN_MASK 0x1000000
+#define CRTC_STEREO_CONTROL__CRTC_STEREO_EN__SHIFT 0x18
+#define CRTC_SNAPSHOT_STATUS__CRTC_SNAPSHOT_OCCURRED_MASK 0x1
+#define CRTC_SNAPSHOT_STATUS__CRTC_SNAPSHOT_OCCURRED__SHIFT 0x0
+#define CRTC_SNAPSHOT_STATUS__CRTC_SNAPSHOT_CLEAR_MASK 0x2
+#define CRTC_SNAPSHOT_STATUS__CRTC_SNAPSHOT_CLEAR__SHIFT 0x1
+#define CRTC_SNAPSHOT_STATUS__CRTC_SNAPSHOT_MANUAL_TRIGGER_MASK 0x4
+#define CRTC_SNAPSHOT_STATUS__CRTC_SNAPSHOT_MANUAL_TRIGGER__SHIFT 0x2
+#define CRTC_SNAPSHOT_CONTROL__CRTC_AUTO_SNAPSHOT_TRIG_SEL_MASK 0x3
+#define CRTC_SNAPSHOT_CONTROL__CRTC_AUTO_SNAPSHOT_TRIG_SEL__SHIFT 0x0
+#define CRTC_SNAPSHOT_POSITION__CRTC_SNAPSHOT_VERT_COUNT_MASK 0x3fff
+#define CRTC_SNAPSHOT_POSITION__CRTC_SNAPSHOT_VERT_COUNT__SHIFT 0x0
+#define CRTC_SNAPSHOT_POSITION__CRTC_SNAPSHOT_HORZ_COUNT_MASK 0x3fff0000
+#define CRTC_SNAPSHOT_POSITION__CRTC_SNAPSHOT_HORZ_COUNT__SHIFT 0x10
+#define CRTC_SNAPSHOT_FRAME__CRTC_SNAPSHOT_FRAME_COUNT_MASK 0xffffff
+#define CRTC_SNAPSHOT_FRAME__CRTC_SNAPSHOT_FRAME_COUNT__SHIFT 0x0
+#define CRTC_START_LINE_CONTROL__CRTC_PROGRESSIVE_START_LINE_EARLY_MASK 0x1
+#define CRTC_START_LINE_CONTROL__CRTC_PROGRESSIVE_START_LINE_EARLY__SHIFT 0x0
+#define CRTC_START_LINE_CONTROL__CRTC_INTERLACE_START_LINE_EARLY_MASK 0x2
+#define CRTC_START_LINE_CONTROL__CRTC_INTERLACE_START_LINE_EARLY__SHIFT 0x1
+#define CRTC_START_LINE_CONTROL__CRTC_PREFETCH_EN_MASK 0x4
+#define CRTC_START_LINE_CONTROL__CRTC_PREFETCH_EN__SHIFT 0x2
+#define CRTC_START_LINE_CONTROL__CRTC_LEGACY_REQUESTOR_EN_MASK 0x100
+#define CRTC_START_LINE_CONTROL__CRTC_LEGACY_REQUESTOR_EN__SHIFT 0x8
+#define CRTC_START_LINE_CONTROL__CRTC_ADVANCED_START_LINE_POSITION_MASK 0xff000
+#define CRTC_START_LINE_CONTROL__CRTC_ADVANCED_START_LINE_POSITION__SHIFT 0xc
+#define CRTC_INTERRUPT_CONTROL__CRTC_SNAPSHOT_INT_MSK_MASK 0x1
+#define CRTC_INTERRUPT_CONTROL__CRTC_SNAPSHOT_INT_MSK__SHIFT 0x0
+#define CRTC_INTERRUPT_CONTROL__CRTC_SNAPSHOT_INT_TYPE_MASK 0x2
+#define CRTC_INTERRUPT_CONTROL__CRTC_SNAPSHOT_INT_TYPE__SHIFT 0x1
+#define CRTC_INTERRUPT_CONTROL__CRTC_V_UPDATE_INT_MSK_MASK 0x10
+#define CRTC_INTERRUPT_CONTROL__CRTC_V_UPDATE_INT_MSK__SHIFT 0x4
+#define CRTC_INTERRUPT_CONTROL__CRTC_V_UPDATE_INT_TYPE_MASK 0x20
+#define CRTC_INTERRUPT_CONTROL__CRTC_V_UPDATE_INT_TYPE__SHIFT 0x5
+#define CRTC_INTERRUPT_CONTROL__CRTC_FORCE_COUNT_NOW_INT_MSK_MASK 0x100
+#define CRTC_INTERRUPT_CONTROL__CRTC_FORCE_COUNT_NOW_INT_MSK__SHIFT 0x8
+#define CRTC_INTERRUPT_CONTROL__CRTC_FORCE_COUNT_NOW_INT_TYPE_MASK 0x200
+#define CRTC_INTERRUPT_CONTROL__CRTC_FORCE_COUNT_NOW_INT_TYPE__SHIFT 0x9
+#define CRTC_INTERRUPT_CONTROL__CRTC_FORCE_VSYNC_NEXT_LINE_INT_MSK_MASK 0x10000
+#define CRTC_INTERRUPT_CONTROL__CRTC_FORCE_VSYNC_NEXT_LINE_INT_MSK__SHIFT 0x10
+#define CRTC_INTERRUPT_CONTROL__CRTC_FORCE_VSYNC_NEXT_LINE_INT_TYPE_MASK 0x20000
+#define CRTC_INTERRUPT_CONTROL__CRTC_FORCE_VSYNC_NEXT_LINE_INT_TYPE__SHIFT 0x11
+#define CRTC_INTERRUPT_CONTROL__CRTC_TRIGA_INT_MSK_MASK 0x1000000
+#define CRTC_INTERRUPT_CONTROL__CRTC_TRIGA_INT_MSK__SHIFT 0x18
+#define CRTC_INTERRUPT_CONTROL__CRTC_TRIGB_INT_MSK_MASK 0x2000000
+#define CRTC_INTERRUPT_CONTROL__CRTC_TRIGB_INT_MSK__SHIFT 0x19
+#define CRTC_INTERRUPT_CONTROL__CRTC_TRIGA_INT_TYPE_MASK 0x4000000
+#define CRTC_INTERRUPT_CONTROL__CRTC_TRIGA_INT_TYPE__SHIFT 0x1a
+#define CRTC_INTERRUPT_CONTROL__CRTC_TRIGB_INT_TYPE_MASK 0x8000000
+#define CRTC_INTERRUPT_CONTROL__CRTC_TRIGB_INT_TYPE__SHIFT 0x1b
+#define CRTC_INTERRUPT_CONTROL__CRTC_VSYNC_NOM_INT_MSK_MASK 0x10000000
+#define CRTC_INTERRUPT_CONTROL__CRTC_VSYNC_NOM_INT_MSK__SHIFT 0x1c
+#define CRTC_INTERRUPT_CONTROL__CRTC_VSYNC_NOM_INT_TYPE_MASK 0x20000000
+#define CRTC_INTERRUPT_CONTROL__CRTC_VSYNC_NOM_INT_TYPE__SHIFT 0x1d
+#define CRTC_INTERRUPT_CONTROL__CRTC_GSL_VSYNC_GAP_INT_MSK_MASK 0x40000000
+#define CRTC_INTERRUPT_CONTROL__CRTC_GSL_VSYNC_GAP_INT_MSK__SHIFT 0x1e
+#define CRTC_INTERRUPT_CONTROL__CRTC_GSL_VSYNC_GAP_INT_TYPE_MASK 0x80000000
+#define CRTC_INTERRUPT_CONTROL__CRTC_GSL_VSYNC_GAP_INT_TYPE__SHIFT 0x1f
+#define CRTC_UPDATE_LOCK__CRTC_UPDATE_LOCK_MASK 0x1
+#define CRTC_UPDATE_LOCK__CRTC_UPDATE_LOCK__SHIFT 0x0
+#define CRTC_DOUBLE_BUFFER_CONTROL__CRTC_UPDATE_PENDING_MASK 0x1
+#define CRTC_DOUBLE_BUFFER_CONTROL__CRTC_UPDATE_PENDING__SHIFT 0x0
+#define CRTC_DOUBLE_BUFFER_CONTROL__CRTC_UPDATE_INSTANTLY_MASK 0x100
+#define CRTC_DOUBLE_BUFFER_CONTROL__CRTC_UPDATE_INSTANTLY__SHIFT 0x8
+#define CRTC_DOUBLE_BUFFER_CONTROL__CRTC_BLANK_DATA_DOUBLE_BUFFER_EN_MASK 0x10000
+#define CRTC_DOUBLE_BUFFER_CONTROL__CRTC_BLANK_DATA_DOUBLE_BUFFER_EN__SHIFT 0x10
+#define CRTC_VGA_PARAMETER_CAPTURE_MODE__CRTC_VGA_PARAMETER_CAPTURE_MODE_MASK 0x1
+#define CRTC_VGA_PARAMETER_CAPTURE_MODE__CRTC_VGA_PARAMETER_CAPTURE_MODE__SHIFT 0x0
+#define CRTC_TEST_PATTERN_CONTROL__CRTC_TEST_PATTERN_EN_MASK 0x1
+#define CRTC_TEST_PATTERN_CONTROL__CRTC_TEST_PATTERN_EN__SHIFT 0x0
+#define CRTC_TEST_PATTERN_CONTROL__CRTC_TEST_PATTERN_MODE_MASK 0x700
+#define CRTC_TEST_PATTERN_CONTROL__CRTC_TEST_PATTERN_MODE__SHIFT 0x8
+#define CRTC_TEST_PATTERN_CONTROL__CRTC_TEST_PATTERN_DYNAMIC_RANGE_MASK 0x10000
+#define CRTC_TEST_PATTERN_CONTROL__CRTC_TEST_PATTERN_DYNAMIC_RANGE__SHIFT 0x10
+#define CRTC_TEST_PATTERN_CONTROL__CRTC_TEST_PATTERN_COLOR_FORMAT_MASK 0xff000000
+#define CRTC_TEST_PATTERN_CONTROL__CRTC_TEST_PATTERN_COLOR_FORMAT__SHIFT 0x18
+#define CRTC_TEST_PATTERN_PARAMETERS__CRTC_TEST_PATTERN_INC0_MASK 0xf
+#define CRTC_TEST_PATTERN_PARAMETERS__CRTC_TEST_PATTERN_INC0__SHIFT 0x0
+#define CRTC_TEST_PATTERN_PARAMETERS__CRTC_TEST_PATTERN_INC1_MASK 0xf0
+#define CRTC_TEST_PATTERN_PARAMETERS__CRTC_TEST_PATTERN_INC1__SHIFT 0x4
+#define CRTC_TEST_PATTERN_PARAMETERS__CRTC_TEST_PATTERN_VRES_MASK 0xf00
+#define CRTC_TEST_PATTERN_PARAMETERS__CRTC_TEST_PATTERN_VRES__SHIFT 0x8
+#define CRTC_TEST_PATTERN_PARAMETERS__CRTC_TEST_PATTERN_HRES_MASK 0xf000
+#define CRTC_TEST_PATTERN_PARAMETERS__CRTC_TEST_PATTERN_HRES__SHIFT 0xc
+#define CRTC_TEST_PATTERN_PARAMETERS__CRTC_TEST_PATTERN_RAMP0_OFFSET_MASK 0xffff0000
+#define CRTC_TEST_PATTERN_PARAMETERS__CRTC_TEST_PATTERN_RAMP0_OFFSET__SHIFT 0x10
+#define CRTC_TEST_PATTERN_COLOR__CRTC_TEST_PATTERN_DATA_MASK 0xffff
+#define CRTC_TEST_PATTERN_COLOR__CRTC_TEST_PATTERN_DATA__SHIFT 0x0
+#define CRTC_TEST_PATTERN_COLOR__CRTC_TEST_PATTERN_MASK_MASK 0x3f0000
+#define CRTC_TEST_PATTERN_COLOR__CRTC_TEST_PATTERN_MASK__SHIFT 0x10
+#define CRTC_MASTER_UPDATE_LOCK__MASTER_UPDATE_LOCK_MASK 0x1
+#define CRTC_MASTER_UPDATE_LOCK__MASTER_UPDATE_LOCK__SHIFT 0x0
+#define CRTC_MASTER_UPDATE_LOCK__GSL_CONTROL_MASTER_UPDATE_LOCK_MASK 0x100
+#define CRTC_MASTER_UPDATE_LOCK__GSL_CONTROL_MASTER_UPDATE_LOCK__SHIFT 0x8
+#define CRTC_MASTER_UPDATE_LOCK__UNDERFLOW_UPDATE_LOCK_MASK 0x10000
+#define CRTC_MASTER_UPDATE_LOCK__UNDERFLOW_UPDATE_LOCK__SHIFT 0x10
+#define CRTC_MASTER_UPDATE_MODE__MASTER_UPDATE_MODE_MASK 0x7
+#define CRTC_MASTER_UPDATE_MODE__MASTER_UPDATE_MODE__SHIFT 0x0
+#define CRTC_MASTER_UPDATE_MODE__MASTER_UPDATE_INTERLACED_MODE_MASK 0x30000
+#define CRTC_MASTER_UPDATE_MODE__MASTER_UPDATE_INTERLACED_MODE__SHIFT 0x10
+#define CRTC_MVP_INBAND_CNTL_INSERT__CRTC_MVP_INBAND_OUT_MODE_MASK 0x3
+#define CRTC_MVP_INBAND_CNTL_INSERT__CRTC_MVP_INBAND_OUT_MODE__SHIFT 0x0
+#define CRTC_MVP_INBAND_CNTL_INSERT__CRTC_MVP_INBAND_CNTL_CHAR_INSERT_MASK 0xffffff00
+#define CRTC_MVP_INBAND_CNTL_INSERT__CRTC_MVP_INBAND_CNTL_CHAR_INSERT__SHIFT 0x8
+#define CRTC_MVP_INBAND_CNTL_INSERT_TIMER__CRTC_MVP_INBAND_CNTL_CHAR_INSERT_TIMER_MASK 0xff
+#define CRTC_MVP_INBAND_CNTL_INSERT_TIMER__CRTC_MVP_INBAND_CNTL_CHAR_INSERT_TIMER__SHIFT 0x0
+#define CRTC_MVP_STATUS__CRTC_FLIP_NOW_OCCURRED_MASK 0x1
+#define CRTC_MVP_STATUS__CRTC_FLIP_NOW_OCCURRED__SHIFT 0x0
+#define CRTC_MVP_STATUS__CRTC_AFR_HSYNC_SWITCH_DONE_OCCURRED_MASK 0x10
+#define CRTC_MVP_STATUS__CRTC_AFR_HSYNC_SWITCH_DONE_OCCURRED__SHIFT 0x4
+#define CRTC_MVP_STATUS__CRTC_FLIP_NOW_CLEAR_MASK 0x10000
+#define CRTC_MVP_STATUS__CRTC_FLIP_NOW_CLEAR__SHIFT 0x10
+#define CRTC_MVP_STATUS__CRTC_AFR_HSYNC_SWITCH_DONE_CLEAR_MASK 0x100000
+#define CRTC_MVP_STATUS__CRTC_AFR_HSYNC_SWITCH_DONE_CLEAR__SHIFT 0x14
+#define CRTC_MASTER_EN__CRTC_MASTER_EN_MASK 0x1
+#define CRTC_MASTER_EN__CRTC_MASTER_EN__SHIFT 0x0
+#define CRTC_ALLOW_STOP_OFF_V_CNT__CRTC_ALLOW_STOP_OFF_V_CNT_MASK 0xff
+#define CRTC_ALLOW_STOP_OFF_V_CNT__CRTC_ALLOW_STOP_OFF_V_CNT__SHIFT 0x0
+#define CRTC_ALLOW_STOP_OFF_V_CNT__CRTC_DISABLE_ALLOW_STOP_OFF_V_CNT_MASK 0x10000
+#define CRTC_ALLOW_STOP_OFF_V_CNT__CRTC_DISABLE_ALLOW_STOP_OFF_V_CNT__SHIFT 0x10
+#define CRTC_V_UPDATE_INT_STATUS__CRTC_V_UPDATE_INT_OCCURRED_MASK 0x1
+#define CRTC_V_UPDATE_INT_STATUS__CRTC_V_UPDATE_INT_OCCURRED__SHIFT 0x0
+#define CRTC_V_UPDATE_INT_STATUS__CRTC_V_UPDATE_INT_CLEAR_MASK 0x100
+#define CRTC_V_UPDATE_INT_STATUS__CRTC_V_UPDATE_INT_CLEAR__SHIFT 0x8
+#define CRTC_OVERSCAN_COLOR__CRTC_OVERSCAN_COLOR_BLUE_MASK 0x3ff
+#define CRTC_OVERSCAN_COLOR__CRTC_OVERSCAN_COLOR_BLUE__SHIFT 0x0
+#define CRTC_OVERSCAN_COLOR__CRTC_OVERSCAN_COLOR_GREEN_MASK 0xffc00
+#define CRTC_OVERSCAN_COLOR__CRTC_OVERSCAN_COLOR_GREEN__SHIFT 0xa
+#define CRTC_OVERSCAN_COLOR__CRTC_OVERSCAN_COLOR_RED_MASK 0x3ff00000
+#define CRTC_OVERSCAN_COLOR__CRTC_OVERSCAN_COLOR_RED__SHIFT 0x14
+#define CRTC_OVERSCAN_COLOR_EXT__CRTC_OVERSCAN_COLOR_BLUE_EXT_MASK 0x3
+#define CRTC_OVERSCAN_COLOR_EXT__CRTC_OVERSCAN_COLOR_BLUE_EXT__SHIFT 0x0
+#define CRTC_OVERSCAN_COLOR_EXT__CRTC_OVERSCAN_COLOR_GREEN_EXT_MASK 0x300
+#define CRTC_OVERSCAN_COLOR_EXT__CRTC_OVERSCAN_COLOR_GREEN_EXT__SHIFT 0x8
+#define CRTC_OVERSCAN_COLOR_EXT__CRTC_OVERSCAN_COLOR_RED_EXT_MASK 0x30000
+#define CRTC_OVERSCAN_COLOR_EXT__CRTC_OVERSCAN_COLOR_RED_EXT__SHIFT 0x10
+#define CRTC_BLANK_DATA_COLOR__CRTC_BLANK_DATA_COLOR_BLUE_CB_MASK 0x3ff
+#define CRTC_BLANK_DATA_COLOR__CRTC_BLANK_DATA_COLOR_BLUE_CB__SHIFT 0x0
+#define CRTC_BLANK_DATA_COLOR__CRTC_BLANK_DATA_COLOR_GREEN_Y_MASK 0xffc00
+#define CRTC_BLANK_DATA_COLOR__CRTC_BLANK_DATA_COLOR_GREEN_Y__SHIFT 0xa
+#define CRTC_BLANK_DATA_COLOR__CRTC_BLANK_DATA_COLOR_RED_CR_MASK 0x3ff00000
+#define CRTC_BLANK_DATA_COLOR__CRTC_BLANK_DATA_COLOR_RED_CR__SHIFT 0x14
+#define CRTC_BLANK_DATA_COLOR_EXT__CRTC_BLANK_DATA_COLOR_BLUE_CB_EXT_MASK 0x3
+#define CRTC_BLANK_DATA_COLOR_EXT__CRTC_BLANK_DATA_COLOR_BLUE_CB_EXT__SHIFT 0x0
+#define CRTC_BLANK_DATA_COLOR_EXT__CRTC_BLANK_DATA_COLOR_GREEN_Y_EXT_MASK 0x300
+#define CRTC_BLANK_DATA_COLOR_EXT__CRTC_BLANK_DATA_COLOR_GREEN_Y_EXT__SHIFT 0x8
+#define CRTC_BLANK_DATA_COLOR_EXT__CRTC_BLANK_DATA_COLOR_RED_CR_EXT_MASK 0x30000
+#define CRTC_BLANK_DATA_COLOR_EXT__CRTC_BLANK_DATA_COLOR_RED_CR_EXT__SHIFT 0x10
+#define CRTC_BLACK_COLOR__CRTC_BLACK_COLOR_B_CB_MASK 0x3ff
+#define CRTC_BLACK_COLOR__CRTC_BLACK_COLOR_B_CB__SHIFT 0x0
+#define CRTC_BLACK_COLOR__CRTC_BLACK_COLOR_G_Y_MASK 0xffc00
+#define CRTC_BLACK_COLOR__CRTC_BLACK_COLOR_G_Y__SHIFT 0xa
+#define CRTC_BLACK_COLOR__CRTC_BLACK_COLOR_R_CR_MASK 0x3ff00000
+#define CRTC_BLACK_COLOR__CRTC_BLACK_COLOR_R_CR__SHIFT 0x14
+#define CRTC_BLACK_COLOR_EXT__CRTC_BLACK_COLOR_B_CB_EXT_MASK 0x3
+#define CRTC_BLACK_COLOR_EXT__CRTC_BLACK_COLOR_B_CB_EXT__SHIFT 0x0
+#define CRTC_BLACK_COLOR_EXT__CRTC_BLACK_COLOR_G_Y_EXT_MASK 0x300
+#define CRTC_BLACK_COLOR_EXT__CRTC_BLACK_COLOR_G_Y_EXT__SHIFT 0x8
+#define CRTC_BLACK_COLOR_EXT__CRTC_BLACK_COLOR_R_CR_EXT_MASK 0x30000
+#define CRTC_BLACK_COLOR_EXT__CRTC_BLACK_COLOR_R_CR_EXT__SHIFT 0x10
+#define CRTC_VERTICAL_INTERRUPT0_POSITION__CRTC_VERTICAL_INTERRUPT0_LINE_START_MASK 0x3fff
+#define CRTC_VERTICAL_INTERRUPT0_POSITION__CRTC_VERTICAL_INTERRUPT0_LINE_START__SHIFT 0x0
+#define CRTC_VERTICAL_INTERRUPT0_POSITION__CRTC_VERTICAL_INTERRUPT0_LINE_END_MASK 0x3fff0000
+#define CRTC_VERTICAL_INTERRUPT0_POSITION__CRTC_VERTICAL_INTERRUPT0_LINE_END__SHIFT 0x10
+#define CRTC_VERTICAL_INTERRUPT0_CONTROL__CRTC_VERTICAL_INTERRUPT0_OUTPUT_POLARITY_MASK 0x10
+#define CRTC_VERTICAL_INTERRUPT0_CONTROL__CRTC_VERTICAL_INTERRUPT0_OUTPUT_POLARITY__SHIFT 0x4
+#define CRTC_VERTICAL_INTERRUPT0_CONTROL__CRTC_VERTICAL_INTERRUPT0_INT_ENABLE_MASK 0x100
+#define CRTC_VERTICAL_INTERRUPT0_CONTROL__CRTC_VERTICAL_INTERRUPT0_INT_ENABLE__SHIFT 0x8
+#define CRTC_VERTICAL_INTERRUPT0_CONTROL__CRTC_VERTICAL_INTERRUPT0_STATUS_MASK 0x1000
+#define CRTC_VERTICAL_INTERRUPT0_CONTROL__CRTC_VERTICAL_INTERRUPT0_STATUS__SHIFT 0xc
+#define CRTC_VERTICAL_INTERRUPT0_CONTROL__CRTC_VERTICAL_INTERRUPT0_INT_STATUS_MASK 0x10000
+#define CRTC_VERTICAL_INTERRUPT0_CONTROL__CRTC_VERTICAL_INTERRUPT0_INT_STATUS__SHIFT 0x10
+#define CRTC_VERTICAL_INTERRUPT0_CONTROL__CRTC_VERTICAL_INTERRUPT0_CLEAR_MASK 0x100000
+#define CRTC_VERTICAL_INTERRUPT0_CONTROL__CRTC_VERTICAL_INTERRUPT0_CLEAR__SHIFT 0x14
+#define CRTC_VERTICAL_INTERRUPT0_CONTROL__CRTC_VERTICAL_INTERRUPT0_INT_TYPE_MASK 0x1000000
+#define CRTC_VERTICAL_INTERRUPT0_CONTROL__CRTC_VERTICAL_INTERRUPT0_INT_TYPE__SHIFT 0x18
+#define CRTC_VERTICAL_INTERRUPT1_POSITION__CRTC_VERTICAL_INTERRUPT1_LINE_START_MASK 0x3fff
+#define CRTC_VERTICAL_INTERRUPT1_POSITION__CRTC_VERTICAL_INTERRUPT1_LINE_START__SHIFT 0x0
+#define CRTC_VERTICAL_INTERRUPT1_CONTROL__CRTC_VERTICAL_INTERRUPT1_INT_ENABLE_MASK 0x100
+#define CRTC_VERTICAL_INTERRUPT1_CONTROL__CRTC_VERTICAL_INTERRUPT1_INT_ENABLE__SHIFT 0x8
+#define CRTC_VERTICAL_INTERRUPT1_CONTROL__CRTC_VERTICAL_INTERRUPT1_STATUS_MASK 0x1000
+#define CRTC_VERTICAL_INTERRUPT1_CONTROL__CRTC_VERTICAL_INTERRUPT1_STATUS__SHIFT 0xc
+#define CRTC_VERTICAL_INTERRUPT1_CONTROL__CRTC_VERTICAL_INTERRUPT1_INT_STATUS_MASK 0x10000
+#define CRTC_VERTICAL_INTERRUPT1_CONTROL__CRTC_VERTICAL_INTERRUPT1_INT_STATUS__SHIFT 0x10
+#define CRTC_VERTICAL_INTERRUPT1_CONTROL__CRTC_VERTICAL_INTERRUPT1_CLEAR_MASK 0x100000
+#define CRTC_VERTICAL_INTERRUPT1_CONTROL__CRTC_VERTICAL_INTERRUPT1_CLEAR__SHIFT 0x14
+#define CRTC_VERTICAL_INTERRUPT1_CONTROL__CRTC_VERTICAL_INTERRUPT1_INT_TYPE_MASK 0x1000000
+#define CRTC_VERTICAL_INTERRUPT1_CONTROL__CRTC_VERTICAL_INTERRUPT1_INT_TYPE__SHIFT 0x18
+#define CRTC_VERTICAL_INTERRUPT2_POSITION__CRTC_VERTICAL_INTERRUPT2_LINE_START_MASK 0x3fff
+#define CRTC_VERTICAL_INTERRUPT2_POSITION__CRTC_VERTICAL_INTERRUPT2_LINE_START__SHIFT 0x0
+#define CRTC_VERTICAL_INTERRUPT2_CONTROL__CRTC_VERTICAL_INTERRUPT2_INT_ENABLE_MASK 0x100
+#define CRTC_VERTICAL_INTERRUPT2_CONTROL__CRTC_VERTICAL_INTERRUPT2_INT_ENABLE__SHIFT 0x8
+#define CRTC_VERTICAL_INTERRUPT2_CONTROL__CRTC_VERTICAL_INTERRUPT2_STATUS_MASK 0x1000
+#define CRTC_VERTICAL_INTERRUPT2_CONTROL__CRTC_VERTICAL_INTERRUPT2_STATUS__SHIFT 0xc
+#define CRTC_VERTICAL_INTERRUPT2_CONTROL__CRTC_VERTICAL_INTERRUPT2_INT_STATUS_MASK 0x10000
+#define CRTC_VERTICAL_INTERRUPT2_CONTROL__CRTC_VERTICAL_INTERRUPT2_INT_STATUS__SHIFT 0x10
+#define CRTC_VERTICAL_INTERRUPT2_CONTROL__CRTC_VERTICAL_INTERRUPT2_CLEAR_MASK 0x100000
+#define CRTC_VERTICAL_INTERRUPT2_CONTROL__CRTC_VERTICAL_INTERRUPT2_CLEAR__SHIFT 0x14
+#define CRTC_VERTICAL_INTERRUPT2_CONTROL__CRTC_VERTICAL_INTERRUPT2_INT_TYPE_MASK 0x1000000
+#define CRTC_VERTICAL_INTERRUPT2_CONTROL__CRTC_VERTICAL_INTERRUPT2_INT_TYPE__SHIFT 0x18
+#define CRTC_CRC_CNTL__CRTC_CRC_EN_MASK 0x1
+#define CRTC_CRC_CNTL__CRTC_CRC_EN__SHIFT 0x0
+#define CRTC_CRC_CNTL__CRTC_CRC_CONT_EN_MASK 0x10
+#define CRTC_CRC_CNTL__CRTC_CRC_CONT_EN__SHIFT 0x4
+#define CRTC_CRC_CNTL__CRTC_CRC_STEREO_MODE_MASK 0x300
+#define CRTC_CRC_CNTL__CRTC_CRC_STEREO_MODE__SHIFT 0x8
+#define CRTC_CRC_CNTL__CRTC_CRC_INTERLACE_MODE_MASK 0x3000
+#define CRTC_CRC_CNTL__CRTC_CRC_INTERLACE_MODE__SHIFT 0xc
+#define CRTC_CRC_CNTL__CRTC_CRC_USE_NEW_AND_REPEATED_PIXELS_MASK 0x10000
+#define CRTC_CRC_CNTL__CRTC_CRC_USE_NEW_AND_REPEATED_PIXELS__SHIFT 0x10
+#define CRTC_CRC_CNTL__CRTC_CRC0_SELECT_MASK 0x700000
+#define CRTC_CRC_CNTL__CRTC_CRC0_SELECT__SHIFT 0x14
+#define CRTC_CRC_CNTL__CRTC_CRC1_SELECT_MASK 0x7000000
+#define CRTC_CRC_CNTL__CRTC_CRC1_SELECT__SHIFT 0x18
+#define CRTC_CRC0_WINDOWA_X_CONTROL__CRTC_CRC0_WINDOWA_X_START_MASK 0x3fff
+#define CRTC_CRC0_WINDOWA_X_CONTROL__CRTC_CRC0_WINDOWA_X_START__SHIFT 0x0
+#define CRTC_CRC0_WINDOWA_X_CONTROL__CRTC_CRC0_WINDOWA_X_END_MASK 0x3fff0000
+#define CRTC_CRC0_WINDOWA_X_CONTROL__CRTC_CRC0_WINDOWA_X_END__SHIFT 0x10
+#define CRTC_CRC0_WINDOWA_Y_CONTROL__CRTC_CRC0_WINDOWA_Y_START_MASK 0x3fff
+#define CRTC_CRC0_WINDOWA_Y_CONTROL__CRTC_CRC0_WINDOWA_Y_START__SHIFT 0x0
+#define CRTC_CRC0_WINDOWA_Y_CONTROL__CRTC_CRC0_WINDOWA_Y_END_MASK 0x3fff0000
+#define CRTC_CRC0_WINDOWA_Y_CONTROL__CRTC_CRC0_WINDOWA_Y_END__SHIFT 0x10
+#define CRTC_CRC0_WINDOWB_X_CONTROL__CRTC_CRC0_WINDOWB_X_START_MASK 0x3fff
+#define CRTC_CRC0_WINDOWB_X_CONTROL__CRTC_CRC0_WINDOWB_X_START__SHIFT 0x0
+#define CRTC_CRC0_WINDOWB_X_CONTROL__CRTC_CRC0_WINDOWB_X_END_MASK 0x3fff0000
+#define CRTC_CRC0_WINDOWB_X_CONTROL__CRTC_CRC0_WINDOWB_X_END__SHIFT 0x10
+#define CRTC_CRC0_WINDOWB_Y_CONTROL__CRTC_CRC0_WINDOWB_Y_START_MASK 0x3fff
+#define CRTC_CRC0_WINDOWB_Y_CONTROL__CRTC_CRC0_WINDOWB_Y_START__SHIFT 0x0
+#define CRTC_CRC0_WINDOWB_Y_CONTROL__CRTC_CRC0_WINDOWB_Y_END_MASK 0x3fff0000
+#define CRTC_CRC0_WINDOWB_Y_CONTROL__CRTC_CRC0_WINDOWB_Y_END__SHIFT 0x10
+#define CRTC_CRC0_DATA_RG__CRC0_R_CR_MASK 0xffff
+#define CRTC_CRC0_DATA_RG__CRC0_R_CR__SHIFT 0x0
+#define CRTC_CRC0_DATA_RG__CRC0_G_Y_MASK 0xffff0000
+#define CRTC_CRC0_DATA_RG__CRC0_G_Y__SHIFT 0x10
+#define CRTC_CRC0_DATA_B__CRC0_B_CB_MASK 0xffff
+#define CRTC_CRC0_DATA_B__CRC0_B_CB__SHIFT 0x0
+#define CRTC_CRC1_WINDOWA_X_CONTROL__CRTC_CRC1_WINDOWA_X_START_MASK 0x3fff
+#define CRTC_CRC1_WINDOWA_X_CONTROL__CRTC_CRC1_WINDOWA_X_START__SHIFT 0x0
+#define CRTC_CRC1_WINDOWA_X_CONTROL__CRTC_CRC1_WINDOWA_X_END_MASK 0x3fff0000
+#define CRTC_CRC1_WINDOWA_X_CONTROL__CRTC_CRC1_WINDOWA_X_END__SHIFT 0x10
+#define CRTC_CRC1_WINDOWA_Y_CONTROL__CRTC_CRC1_WINDOWA_Y_START_MASK 0x3fff
+#define CRTC_CRC1_WINDOWA_Y_CONTROL__CRTC_CRC1_WINDOWA_Y_START__SHIFT 0x0
+#define CRTC_CRC1_WINDOWA_Y_CONTROL__CRTC_CRC1_WINDOWA_Y_END_MASK 0x3fff0000
+#define CRTC_CRC1_WINDOWA_Y_CONTROL__CRTC_CRC1_WINDOWA_Y_END__SHIFT 0x10
+#define CRTC_CRC1_WINDOWB_X_CONTROL__CRTC_CRC1_WINDOWB_X_START_MASK 0x3fff
+#define CRTC_CRC1_WINDOWB_X_CONTROL__CRTC_CRC1_WINDOWB_X_START__SHIFT 0x0
+#define CRTC_CRC1_WINDOWB_X_CONTROL__CRTC_CRC1_WINDOWB_X_END_MASK 0x3fff0000
+#define CRTC_CRC1_WINDOWB_X_CONTROL__CRTC_CRC1_WINDOWB_X_END__SHIFT 0x10
+#define CRTC_CRC1_WINDOWB_Y_CONTROL__CRTC_CRC1_WINDOWB_Y_START_MASK 0x3fff
+#define CRTC_CRC1_WINDOWB_Y_CONTROL__CRTC_CRC1_WINDOWB_Y_START__SHIFT 0x0
+#define CRTC_CRC1_WINDOWB_Y_CONTROL__CRTC_CRC1_WINDOWB_Y_END_MASK 0x3fff0000
+#define CRTC_CRC1_WINDOWB_Y_CONTROL__CRTC_CRC1_WINDOWB_Y_END__SHIFT 0x10
+#define CRTC_CRC1_DATA_RG__CRC1_R_CR_MASK 0xffff
+#define CRTC_CRC1_DATA_RG__CRC1_R_CR__SHIFT 0x0
+#define CRTC_CRC1_DATA_RG__CRC1_G_Y_MASK 0xffff0000
+#define CRTC_CRC1_DATA_RG__CRC1_G_Y__SHIFT 0x10
+#define CRTC_CRC1_DATA_B__CRC1_B_CB_MASK 0xffff
+#define CRTC_CRC1_DATA_B__CRC1_B_CB__SHIFT 0x0
+#define CRTC_EXT_TIMING_SYNC_CONTROL__CRTC_EXT_TIMING_SYNC_ENABLE_MASK 0x3
+#define CRTC_EXT_TIMING_SYNC_CONTROL__CRTC_EXT_TIMING_SYNC_ENABLE__SHIFT 0x0
+#define CRTC_EXT_TIMING_SYNC_CONTROL__CRTC_EXT_TIMING_SYNC_HCOUNT_MODE_ENABLE_MASK 0x8
+#define CRTC_EXT_TIMING_SYNC_CONTROL__CRTC_EXT_TIMING_SYNC_HCOUNT_MODE_ENABLE__SHIFT 0x3
+#define CRTC_EXT_TIMING_SYNC_CONTROL__CRTC_EXT_TIMING_SYNC_JITTER_FILTERING_ENABLE_MASK 0x10
+#define CRTC_EXT_TIMING_SYNC_CONTROL__CRTC_EXT_TIMING_SYNC_JITTER_FILTERING_ENABLE__SHIFT 0x4
+#define CRTC_EXT_TIMING_SYNC_CONTROL__CRTC_EXT_TIMING_SYNC_JITTER_FILTERING_WINDOW_MASK 0x60
+#define CRTC_EXT_TIMING_SYNC_CONTROL__CRTC_EXT_TIMING_SYNC_JITTER_FILTERING_WINDOW__SHIFT 0x5
+#define CRTC_EXT_TIMING_SYNC_CONTROL__CRTC_EXT_TIMING_SYNC_WINDOW_ENABLE_MASK 0x100
+#define CRTC_EXT_TIMING_SYNC_CONTROL__CRTC_EXT_TIMING_SYNC_WINDOW_ENABLE__SHIFT 0x8
+#define CRTC_EXT_TIMING_SYNC_CONTROL__CRTC_EXT_TIMING_SYNC_WINDOW_UPDATE_MASK 0x200
+#define CRTC_EXT_TIMING_SYNC_CONTROL__CRTC_EXT_TIMING_SYNC_WINDOW_UPDATE__SHIFT 0x9
+#define CRTC_EXT_TIMING_SYNC_CONTROL__CRTC_EXT_TIMING_SYNC_VSYNC_POLARITY_MASK 0x1000
+#define CRTC_EXT_TIMING_SYNC_CONTROL__CRTC_EXT_TIMING_SYNC_VSYNC_POLARITY__SHIFT 0xc
+#define CRTC_EXT_TIMING_SYNC_CONTROL__CRTC_EXT_TIMING_SYNC_HSYNC_POLARITY_MASK 0x2000
+#define CRTC_EXT_TIMING_SYNC_CONTROL__CRTC_EXT_TIMING_SYNC_HSYNC_POLARITY__SHIFT 0xd
+#define CRTC_EXT_TIMING_SYNC_CONTROL__CRTC_EXT_TIMING_SYNC_INTERLACE_MODE_MASK 0x4000
+#define CRTC_EXT_TIMING_SYNC_CONTROL__CRTC_EXT_TIMING_SYNC_INTERLACE_MODE__SHIFT 0xe
+#define CRTC_EXT_TIMING_SYNC_CONTROL__CRTC_EXT_TIMING_SYNC_MASTER_FRAME_RATE_MASK 0x7000000
+#define CRTC_EXT_TIMING_SYNC_CONTROL__CRTC_EXT_TIMING_SYNC_MASTER_FRAME_RATE__SHIFT 0x18
+#define CRTC_EXT_TIMING_SYNC_CONTROL__CRTC_EXT_TIMING_SYNC_SLAVE_FRAME_RATE_MASK 0x70000000
+#define CRTC_EXT_TIMING_SYNC_CONTROL__CRTC_EXT_TIMING_SYNC_SLAVE_FRAME_RATE__SHIFT 0x1c
+#define CRTC_EXT_TIMING_SYNC_WINDOW_START__CRTC_EXT_TIMING_SYNC_WINDOW_START_X_MASK 0x3fff
+#define CRTC_EXT_TIMING_SYNC_WINDOW_START__CRTC_EXT_TIMING_SYNC_WINDOW_START_X__SHIFT 0x0
+#define CRTC_EXT_TIMING_SYNC_WINDOW_START__CRTC_EXT_TIMING_SYNC_WINDOW_START_Y_MASK 0x3fff0000
+#define CRTC_EXT_TIMING_SYNC_WINDOW_START__CRTC_EXT_TIMING_SYNC_WINDOW_START_Y__SHIFT 0x10
+#define CRTC_EXT_TIMING_SYNC_WINDOW_END__CRTC_EXT_TIMING_SYNC_WINDOW_END_X_MASK 0x3fff
+#define CRTC_EXT_TIMING_SYNC_WINDOW_END__CRTC_EXT_TIMING_SYNC_WINDOW_END_X__SHIFT 0x0
+#define CRTC_EXT_TIMING_SYNC_WINDOW_END__CRTC_EXT_TIMING_SYNC_WINDOW_END_Y_MASK 0x3fff0000
+#define CRTC_EXT_TIMING_SYNC_WINDOW_END__CRTC_EXT_TIMING_SYNC_WINDOW_END_Y__SHIFT 0x10
+#define CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_LOSS_INT_ENABLE_MASK 0x1
+#define CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_LOSS_INT_ENABLE__SHIFT 0x0
+#define CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_LOSS_STATUS_MASK 0x10
+#define CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_LOSS_STATUS__SHIFT 0x4
+#define CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_LOSS_INT_STATUS_MASK 0x100
+#define CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_LOSS_INT_STATUS__SHIFT 0x8
+#define CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_LOSS_CLEAR_MASK 0x10000
+#define CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_LOSS_CLEAR__SHIFT 0x10
+#define CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_LOSS_INT_TYPE_MASK 0x100000
+#define CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_LOSS_INT_TYPE__SHIFT 0x14
+#define CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_LOSS_FRAME_COUNT_MASK 0xe0000000
+#define CRTC_EXT_TIMING_SYNC_LOSS_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_LOSS_FRAME_COUNT__SHIFT 0x1d
+#define CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_INT_ENABLE_MASK 0x1
+#define CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_INT_ENABLE__SHIFT 0x0
+#define CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_STATUS_MASK 0x10
+#define CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_STATUS__SHIFT 0x4
+#define CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_INT_STATUS_MASK 0x100
+#define CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_INT_STATUS__SHIFT 0x8
+#define CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_CLEAR_MASK 0x10000
+#define CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_CLEAR__SHIFT 0x10
+#define CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_INT_TYPE_MASK 0x100000
+#define CRTC_EXT_TIMING_SYNC_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_INT_TYPE__SHIFT 0x14
+#define CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_SIGNAL_INT_ENABLE_MASK 0x1
+#define CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_SIGNAL_INT_ENABLE__SHIFT 0x0
+#define CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_SIGNAL_STATUS_MASK 0x10
+#define CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_SIGNAL_STATUS__SHIFT 0x4
+#define CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_SIGNAL_INT_STATUS_MASK 0x100
+#define CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_SIGNAL_INT_STATUS__SHIFT 0x8
+#define CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_SIGNAL_CLEAR_MASK 0x10000
+#define CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_SIGNAL_CLEAR__SHIFT 0x10
+#define CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_SIGNAL_INT_TYPE_MASK 0x100000
+#define CRTC_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_CONTROL__CRTC_EXT_TIMING_SYNC_SIGNAL_INT_TYPE__SHIFT 0x14
+#define CRTC_STATIC_SCREEN_CONTROL__CRTC_STATIC_SCREEN_EVENT_MASK_MASK 0xffff
+#define CRTC_STATIC_SCREEN_CONTROL__CRTC_STATIC_SCREEN_EVENT_MASK__SHIFT 0x0
+#define CRTC_STATIC_SCREEN_CONTROL__CRTC_STATIC_SCREEN_FRAME_COUNT_MASK 0xff0000
+#define CRTC_STATIC_SCREEN_CONTROL__CRTC_STATIC_SCREEN_FRAME_COUNT__SHIFT 0x10
+#define CRTC_STATIC_SCREEN_CONTROL__CRTC_CPU_SS_INT_ENABLE_MASK 0x1000000
+#define CRTC_STATIC_SCREEN_CONTROL__CRTC_CPU_SS_INT_ENABLE__SHIFT 0x18
+#define CRTC_STATIC_SCREEN_CONTROL__CRTC_SS_STATUS_MASK 0x2000000
+#define CRTC_STATIC_SCREEN_CONTROL__CRTC_SS_STATUS__SHIFT 0x19
+#define CRTC_STATIC_SCREEN_CONTROL__CRTC_CPU_SS_INT_STATUS_MASK 0x4000000
+#define CRTC_STATIC_SCREEN_CONTROL__CRTC_CPU_SS_INT_STATUS__SHIFT 0x1a
+#define CRTC_STATIC_SCREEN_CONTROL__CRTC_CPU_SS_INT_CLEAR_MASK 0x8000000
+#define CRTC_STATIC_SCREEN_CONTROL__CRTC_CPU_SS_INT_CLEAR__SHIFT 0x1b
+#define CRTC_STATIC_SCREEN_CONTROL__CRTC_CPU_SS_INT_TYPE_MASK 0x10000000
+#define CRTC_STATIC_SCREEN_CONTROL__CRTC_CPU_SS_INT_TYPE__SHIFT 0x1c
+#define CRTC_STATIC_SCREEN_CONTROL__CRTC_STATIC_SCREEN_OVERRIDE_MASK 0x40000000
+#define CRTC_STATIC_SCREEN_CONTROL__CRTC_STATIC_SCREEN_OVERRIDE__SHIFT 0x1e
+#define CRTC_STATIC_SCREEN_CONTROL__CRTC_STATIC_SCREEN_OVERRIDE_VALUE_MASK 0x80000000
+#define CRTC_STATIC_SCREEN_CONTROL__CRTC_STATIC_SCREEN_OVERRIDE_VALUE__SHIFT 0x1f
+#define CRTC_3D_STRUCTURE_CONTROL__CRTC_3D_STRUCTURE_EN_MASK 0x1
+#define CRTC_3D_STRUCTURE_CONTROL__CRTC_3D_STRUCTURE_EN__SHIFT 0x0
+#define CRTC_3D_STRUCTURE_CONTROL__CRTC_3D_STRUCTURE_EN_DB_MASK 0x10
+#define CRTC_3D_STRUCTURE_CONTROL__CRTC_3D_STRUCTURE_EN_DB__SHIFT 0x4
+#define CRTC_3D_STRUCTURE_CONTROL__CRTC_3D_STRUCTURE_V_UPDATE_MODE_MASK 0x300
+#define CRTC_3D_STRUCTURE_CONTROL__CRTC_3D_STRUCTURE_V_UPDATE_MODE__SHIFT 0x8
+#define CRTC_3D_STRUCTURE_CONTROL__CRTC_3D_STRUCTURE_STEREO_SEL_OVR_MASK 0x1000
+#define CRTC_3D_STRUCTURE_CONTROL__CRTC_3D_STRUCTURE_STEREO_SEL_OVR__SHIFT 0xc
+#define CRTC_3D_STRUCTURE_CONTROL__CRTC_3D_STRUCTURE_F_COUNT_RESET_MASK 0x10000
+#define CRTC_3D_STRUCTURE_CONTROL__CRTC_3D_STRUCTURE_F_COUNT_RESET__SHIFT 0x10
+#define CRTC_3D_STRUCTURE_CONTROL__CRTC_3D_STRUCTURE_F_COUNT_RESET_PENDING_MASK 0x20000
+#define CRTC_3D_STRUCTURE_CONTROL__CRTC_3D_STRUCTURE_F_COUNT_RESET_PENDING__SHIFT 0x11
+#define CRTC_3D_STRUCTURE_CONTROL__CRTC_3D_STRUCTURE_F_COUNT_MASK 0xc0000
+#define CRTC_3D_STRUCTURE_CONTROL__CRTC_3D_STRUCTURE_F_COUNT__SHIFT 0x12
+#define CRTC_GSL_VSYNC_GAP__CRTC_GSL_VSYNC_GAP_LIMIT_MASK 0xff
+#define CRTC_GSL_VSYNC_GAP__CRTC_GSL_VSYNC_GAP_LIMIT__SHIFT 0x0
+#define CRTC_GSL_VSYNC_GAP__CRTC_GSL_VSYNC_GAP_DELAY_MASK 0xff00
+#define CRTC_GSL_VSYNC_GAP__CRTC_GSL_VSYNC_GAP_DELAY__SHIFT 0x8
+#define CRTC_GSL_VSYNC_GAP__CRTC_GSL_VSYNC_GAP_SOURCE_SEL_MASK 0x10000
+#define CRTC_GSL_VSYNC_GAP__CRTC_GSL_VSYNC_GAP_SOURCE_SEL__SHIFT 0x10
+#define CRTC_GSL_VSYNC_GAP__CRTC_GSL_VSYNC_GAP_MODE_MASK 0x60000
+#define CRTC_GSL_VSYNC_GAP__CRTC_GSL_VSYNC_GAP_MODE__SHIFT 0x11
+#define CRTC_GSL_VSYNC_GAP__CRTC_GSL_VSYNC_GAP_CLEAR_MASK 0x80000
+#define CRTC_GSL_VSYNC_GAP__CRTC_GSL_VSYNC_GAP_CLEAR__SHIFT 0x13
+#define CRTC_GSL_VSYNC_GAP__CRTC_GSL_VSYNC_GAP_OCCURRED_MASK 0x100000
+#define CRTC_GSL_VSYNC_GAP__CRTC_GSL_VSYNC_GAP_OCCURRED__SHIFT 0x14
+#define CRTC_GSL_VSYNC_GAP__CRTC_GSL_VSYNC_GAP_MASTER_FASTER_MASK 0x800000
+#define CRTC_GSL_VSYNC_GAP__CRTC_GSL_VSYNC_GAP_MASTER_FASTER__SHIFT 0x17
+#define CRTC_GSL_VSYNC_GAP__CRTC_GSL_VSYNC_GAP_MASK 0xff000000
+#define CRTC_GSL_VSYNC_GAP__CRTC_GSL_VSYNC_GAP__SHIFT 0x18
+#define CRTC_GSL_WINDOW__CRTC_GSL_WINDOW_START_MASK 0x3fff
+#define CRTC_GSL_WINDOW__CRTC_GSL_WINDOW_START__SHIFT 0x0
+#define CRTC_GSL_WINDOW__CRTC_GSL_WINDOW_END_MASK 0x3fff0000
+#define CRTC_GSL_WINDOW__CRTC_GSL_WINDOW_END__SHIFT 0x10
+#define CRTC_GSL_CONTROL__CRTC_GSL_CHECK_LINE_NUM_MASK 0x3fff
+#define CRTC_GSL_CONTROL__CRTC_GSL_CHECK_LINE_NUM__SHIFT 0x0
+#define CRTC_GSL_CONTROL__CRTC_GSL_FORCE_DELAY_MASK 0x1f0000
+#define CRTC_GSL_CONTROL__CRTC_GSL_FORCE_DELAY__SHIFT 0x10
+#define CRTC_GSL_CONTROL__CRTC_GSL_CHECK_ALL_FIELDS_MASK 0x10000000
+#define CRTC_GSL_CONTROL__CRTC_GSL_CHECK_ALL_FIELDS__SHIFT 0x1c
+#define CRTC_TEST_DEBUG_INDEX__CRTC_TEST_DEBUG_INDEX_MASK 0xff
+#define CRTC_TEST_DEBUG_INDEX__CRTC_TEST_DEBUG_INDEX__SHIFT 0x0
+#define CRTC_TEST_DEBUG_INDEX__CRTC_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define CRTC_TEST_DEBUG_INDEX__CRTC_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define CRTC_TEST_DEBUG_DATA__CRTC_TEST_DEBUG_DATA_MASK 0xffffffff
+#define CRTC_TEST_DEBUG_DATA__CRTC_TEST_DEBUG_DATA__SHIFT 0x0
+#define DAC_ENABLE__DAC_ENABLE_MASK 0x1
+#define DAC_ENABLE__DAC_ENABLE__SHIFT 0x0
+#define DAC_ENABLE__DAC_RESYNC_FIFO_ENABLE_MASK 0x2
+#define DAC_ENABLE__DAC_RESYNC_FIFO_ENABLE__SHIFT 0x1
+#define DAC_ENABLE__DAC_RESYNC_FIFO_POINTER_SKEW_MASK 0xc
+#define DAC_ENABLE__DAC_RESYNC_FIFO_POINTER_SKEW__SHIFT 0x2
+#define DAC_ENABLE__DAC_RESYNC_FIFO_ERROR_MASK 0x10
+#define DAC_ENABLE__DAC_RESYNC_FIFO_ERROR__SHIFT 0x4
+#define DAC_ENABLE__DAC_RESYNC_FIFO_ERROR_ACK_MASK 0x20
+#define DAC_ENABLE__DAC_RESYNC_FIFO_ERROR_ACK__SHIFT 0x5
+#define DAC_ENABLE__DAC_RESYNC_FIFO_TVOUT_SIM_MASK 0x100
+#define DAC_ENABLE__DAC_RESYNC_FIFO_TVOUT_SIM__SHIFT 0x8
+#define DAC_SOURCE_SELECT__DAC_SOURCE_SELECT_MASK 0x7
+#define DAC_SOURCE_SELECT__DAC_SOURCE_SELECT__SHIFT 0x0
+#define DAC_SOURCE_SELECT__DAC_TV_SELECT_MASK 0x8
+#define DAC_SOURCE_SELECT__DAC_TV_SELECT__SHIFT 0x3
+#define DAC_CRC_EN__DAC_CRC_EN_MASK 0x1
+#define DAC_CRC_EN__DAC_CRC_EN__SHIFT 0x0
+#define DAC_CRC_EN__DAC_CRC_CONT_EN_MASK 0x10000
+#define DAC_CRC_EN__DAC_CRC_CONT_EN__SHIFT 0x10
+#define DAC_CRC_CONTROL__DAC_CRC_FIELD_MASK 0x1
+#define DAC_CRC_CONTROL__DAC_CRC_FIELD__SHIFT 0x0
+#define DAC_CRC_CONTROL__DAC_CRC_ONLY_BLANKB_MASK 0x100
+#define DAC_CRC_CONTROL__DAC_CRC_ONLY_BLANKB__SHIFT 0x8
+#define DAC_CRC_SIG_RGB_MASK__DAC_CRC_SIG_BLUE_MASK_MASK 0x3ff
+#define DAC_CRC_SIG_RGB_MASK__DAC_CRC_SIG_BLUE_MASK__SHIFT 0x0
+#define DAC_CRC_SIG_RGB_MASK__DAC_CRC_SIG_GREEN_MASK_MASK 0xffc00
+#define DAC_CRC_SIG_RGB_MASK__DAC_CRC_SIG_GREEN_MASK__SHIFT 0xa
+#define DAC_CRC_SIG_RGB_MASK__DAC_CRC_SIG_RED_MASK_MASK 0x3ff00000
+#define DAC_CRC_SIG_RGB_MASK__DAC_CRC_SIG_RED_MASK__SHIFT 0x14
+#define DAC_CRC_SIG_CONTROL_MASK__DAC_CRC_SIG_CONTROL_MASK_MASK 0x3f
+#define DAC_CRC_SIG_CONTROL_MASK__DAC_CRC_SIG_CONTROL_MASK__SHIFT 0x0
+#define DAC_CRC_SIG_RGB__DAC_CRC_SIG_BLUE_MASK 0x3ff
+#define DAC_CRC_SIG_RGB__DAC_CRC_SIG_BLUE__SHIFT 0x0
+#define DAC_CRC_SIG_RGB__DAC_CRC_SIG_GREEN_MASK 0xffc00
+#define DAC_CRC_SIG_RGB__DAC_CRC_SIG_GREEN__SHIFT 0xa
+#define DAC_CRC_SIG_RGB__DAC_CRC_SIG_RED_MASK 0x3ff00000
+#define DAC_CRC_SIG_RGB__DAC_CRC_SIG_RED__SHIFT 0x14
+#define DAC_CRC_SIG_CONTROL__DAC_CRC_SIG_CONTROL_MASK 0x3f
+#define DAC_CRC_SIG_CONTROL__DAC_CRC_SIG_CONTROL__SHIFT 0x0
+#define DAC_SYNC_TRISTATE_CONTROL__DAC_HSYNCA_TRISTATE_MASK 0x1
+#define DAC_SYNC_TRISTATE_CONTROL__DAC_HSYNCA_TRISTATE__SHIFT 0x0
+#define DAC_SYNC_TRISTATE_CONTROL__DAC_VSYNCA_TRISTATE_MASK 0x100
+#define DAC_SYNC_TRISTATE_CONTROL__DAC_VSYNCA_TRISTATE__SHIFT 0x8
+#define DAC_SYNC_TRISTATE_CONTROL__DAC_SYNCA_TRISTATE_MASK 0x10000
+#define DAC_SYNC_TRISTATE_CONTROL__DAC_SYNCA_TRISTATE__SHIFT 0x10
+#define DAC_STEREOSYNC_SELECT__DAC_STEREOSYNC_SELECT_MASK 0x7
+#define DAC_STEREOSYNC_SELECT__DAC_STEREOSYNC_SELECT__SHIFT 0x0
+#define DAC_AUTODETECT_CONTROL__DAC_AUTODETECT_MODE_MASK 0x3
+#define DAC_AUTODETECT_CONTROL__DAC_AUTODETECT_MODE__SHIFT 0x0
+#define DAC_AUTODETECT_CONTROL__DAC_AUTODETECT_FRAME_TIME_COUNTER_MASK 0xff00
+#define DAC_AUTODETECT_CONTROL__DAC_AUTODETECT_FRAME_TIME_COUNTER__SHIFT 0x8
+#define DAC_AUTODETECT_CONTROL__DAC_AUTODETECT_CHECK_MASK_MASK 0x70000
+#define DAC_AUTODETECT_CONTROL__DAC_AUTODETECT_CHECK_MASK__SHIFT 0x10
+#define DAC_AUTODETECT_CONTROL2__DAC_AUTODETECT_POWERUP_COUNTER_MASK 0xff
+#define DAC_AUTODETECT_CONTROL2__DAC_AUTODETECT_POWERUP_COUNTER__SHIFT 0x0
+#define DAC_AUTODETECT_CONTROL2__DAC_AUTODETECT_TESTMODE_MASK 0x100
+#define DAC_AUTODETECT_CONTROL2__DAC_AUTODETECT_TESTMODE__SHIFT 0x8
+#define DAC_AUTODETECT_CONTROL3__DAC_AUTODET_COMPARATOR_IN_DELAY_MASK 0xff
+#define DAC_AUTODETECT_CONTROL3__DAC_AUTODET_COMPARATOR_IN_DELAY__SHIFT 0x0
+#define DAC_AUTODETECT_CONTROL3__DAC_AUTODET_COMPARATOR_OUT_DELAY_MASK 0xff00
+#define DAC_AUTODETECT_CONTROL3__DAC_AUTODET_COMPARATOR_OUT_DELAY__SHIFT 0x8
+#define DAC_AUTODETECT_STATUS__DAC_AUTODETECT_STATUS_MASK 0x1
+#define DAC_AUTODETECT_STATUS__DAC_AUTODETECT_STATUS__SHIFT 0x0
+#define DAC_AUTODETECT_STATUS__DAC_AUTODETECT_CONNECT_MASK 0x10
+#define DAC_AUTODETECT_STATUS__DAC_AUTODETECT_CONNECT__SHIFT 0x4
+#define DAC_AUTODETECT_STATUS__DAC_AUTODETECT_RED_SENSE_MASK 0x300
+#define DAC_AUTODETECT_STATUS__DAC_AUTODETECT_RED_SENSE__SHIFT 0x8
+#define DAC_AUTODETECT_STATUS__DAC_AUTODETECT_GREEN_SENSE_MASK 0x30000
+#define DAC_AUTODETECT_STATUS__DAC_AUTODETECT_GREEN_SENSE__SHIFT 0x10
+#define DAC_AUTODETECT_STATUS__DAC_AUTODETECT_BLUE_SENSE_MASK 0x3000000
+#define DAC_AUTODETECT_STATUS__DAC_AUTODETECT_BLUE_SENSE__SHIFT 0x18
+#define DAC_AUTODETECT_INT_CONTROL__DAC_AUTODETECT_ACK_MASK 0x1
+#define DAC_AUTODETECT_INT_CONTROL__DAC_AUTODETECT_ACK__SHIFT 0x0
+#define DAC_AUTODETECT_INT_CONTROL__DAC_AUTODETECT_INT_ENABLE_MASK 0x10000
+#define DAC_AUTODETECT_INT_CONTROL__DAC_AUTODETECT_INT_ENABLE__SHIFT 0x10
+#define DAC_FORCE_OUTPUT_CNTL__DAC_FORCE_DATA_EN_MASK 0x1
+#define DAC_FORCE_OUTPUT_CNTL__DAC_FORCE_DATA_EN__SHIFT 0x0
+#define DAC_FORCE_OUTPUT_CNTL__DAC_FORCE_DATA_SEL_MASK 0x700
+#define DAC_FORCE_OUTPUT_CNTL__DAC_FORCE_DATA_SEL__SHIFT 0x8
+#define DAC_FORCE_OUTPUT_CNTL__DAC_FORCE_DATA_ON_BLANKB_ONLY_MASK 0x1000000
+#define DAC_FORCE_OUTPUT_CNTL__DAC_FORCE_DATA_ON_BLANKB_ONLY__SHIFT 0x18
+#define DAC_FORCE_DATA__DAC_FORCE_DATA_MASK 0x3ff
+#define DAC_FORCE_DATA__DAC_FORCE_DATA__SHIFT 0x0
+#define DAC_POWERDOWN__DAC_POWERDOWN_MASK 0x1
+#define DAC_POWERDOWN__DAC_POWERDOWN__SHIFT 0x0
+#define DAC_POWERDOWN__DAC_POWERDOWN_BLUE_MASK 0x100
+#define DAC_POWERDOWN__DAC_POWERDOWN_BLUE__SHIFT 0x8
+#define DAC_POWERDOWN__DAC_POWERDOWN_GREEN_MASK 0x10000
+#define DAC_POWERDOWN__DAC_POWERDOWN_GREEN__SHIFT 0x10
+#define DAC_POWERDOWN__DAC_POWERDOWN_RED_MASK 0x1000000
+#define DAC_POWERDOWN__DAC_POWERDOWN_RED__SHIFT 0x18
+#define DAC_CONTROL__DAC_DFORCE_EN_MASK 0x1
+#define DAC_CONTROL__DAC_DFORCE_EN__SHIFT 0x0
+#define DAC_CONTROL__DAC_TV_ENABLE_MASK 0x100
+#define DAC_CONTROL__DAC_TV_ENABLE__SHIFT 0x8
+#define DAC_CONTROL__DAC_ZSCALE_SHIFT_MASK 0x10000
+#define DAC_CONTROL__DAC_ZSCALE_SHIFT__SHIFT 0x10
+#define DAC_COMPARATOR_ENABLE__DAC_COMP_DDET_REF_EN_MASK 0x1
+#define DAC_COMPARATOR_ENABLE__DAC_COMP_DDET_REF_EN__SHIFT 0x0
+#define DAC_COMPARATOR_ENABLE__DAC_COMP_SDET_REF_EN_MASK 0x100
+#define DAC_COMPARATOR_ENABLE__DAC_COMP_SDET_REF_EN__SHIFT 0x8
+#define DAC_COMPARATOR_ENABLE__DAC_R_ASYNC_ENABLE_MASK 0x10000
+#define DAC_COMPARATOR_ENABLE__DAC_R_ASYNC_ENABLE__SHIFT 0x10
+#define DAC_COMPARATOR_ENABLE__DAC_G_ASYNC_ENABLE_MASK 0x20000
+#define DAC_COMPARATOR_ENABLE__DAC_G_ASYNC_ENABLE__SHIFT 0x11
+#define DAC_COMPARATOR_ENABLE__DAC_B_ASYNC_ENABLE_MASK 0x40000
+#define DAC_COMPARATOR_ENABLE__DAC_B_ASYNC_ENABLE__SHIFT 0x12
+#define DAC_COMPARATOR_OUTPUT__DAC_COMPARATOR_OUTPUT_MASK 0x1
+#define DAC_COMPARATOR_OUTPUT__DAC_COMPARATOR_OUTPUT__SHIFT 0x0
+#define DAC_COMPARATOR_OUTPUT__DAC_COMPARATOR_OUTPUT_BLUE_MASK 0x2
+#define DAC_COMPARATOR_OUTPUT__DAC_COMPARATOR_OUTPUT_BLUE__SHIFT 0x1
+#define DAC_COMPARATOR_OUTPUT__DAC_COMPARATOR_OUTPUT_GREEN_MASK 0x4
+#define DAC_COMPARATOR_OUTPUT__DAC_COMPARATOR_OUTPUT_GREEN__SHIFT 0x2
+#define DAC_COMPARATOR_OUTPUT__DAC_COMPARATOR_OUTPUT_RED_MASK 0x8
+#define DAC_COMPARATOR_OUTPUT__DAC_COMPARATOR_OUTPUT_RED__SHIFT 0x3
+#define DAC_PWR_CNTL__DAC_BG_MODE_MASK 0x3
+#define DAC_PWR_CNTL__DAC_BG_MODE__SHIFT 0x0
+#define DAC_PWR_CNTL__DAC_PWRCNTL_MASK 0x30000
+#define DAC_PWR_CNTL__DAC_PWRCNTL__SHIFT 0x10
+#define DAC_DFT_CONFIG__DAC_DFT_CONFIG_MASK 0xffffffff
+#define DAC_DFT_CONFIG__DAC_DFT_CONFIG__SHIFT 0x0
+#define DAC_FIFO_STATUS__DAC_FIFO_USE_OVERWRITE_LEVEL_MASK 0x2
+#define DAC_FIFO_STATUS__DAC_FIFO_USE_OVERWRITE_LEVEL__SHIFT 0x1
+#define DAC_FIFO_STATUS__DAC_FIFO_OVERWRITE_LEVEL_MASK 0xfc
+#define DAC_FIFO_STATUS__DAC_FIFO_OVERWRITE_LEVEL__SHIFT 0x2
+#define DAC_FIFO_STATUS__DAC_FIFO_CAL_AVERAGE_LEVEL_MASK 0xfc00
+#define DAC_FIFO_STATUS__DAC_FIFO_CAL_AVERAGE_LEVEL__SHIFT 0xa
+#define DAC_FIFO_STATUS__DAC_FIFO_MAXIMUM_LEVEL_MASK 0xf0000
+#define DAC_FIFO_STATUS__DAC_FIFO_MAXIMUM_LEVEL__SHIFT 0x10
+#define DAC_FIFO_STATUS__DAC_FIFO_MINIMUM_LEVEL_MASK 0x3c00000
+#define DAC_FIFO_STATUS__DAC_FIFO_MINIMUM_LEVEL__SHIFT 0x16
+#define DAC_FIFO_STATUS__DAC_FIFO_CALIBRATED_MASK 0x20000000
+#define DAC_FIFO_STATUS__DAC_FIFO_CALIBRATED__SHIFT 0x1d
+#define DAC_FIFO_STATUS__DAC_FIFO_FORCE_RECAL_AVERAGE_MASK 0x40000000
+#define DAC_FIFO_STATUS__DAC_FIFO_FORCE_RECAL_AVERAGE__SHIFT 0x1e
+#define DAC_FIFO_STATUS__DAC_FIFO_FORCE_RECOMP_MINMAX_MASK 0x80000000
+#define DAC_FIFO_STATUS__DAC_FIFO_FORCE_RECOMP_MINMAX__SHIFT 0x1f
+#define DAC_TEST_DEBUG_INDEX__DAC_TEST_DEBUG_INDEX_MASK 0xff
+#define DAC_TEST_DEBUG_INDEX__DAC_TEST_DEBUG_INDEX__SHIFT 0x0
+#define DAC_TEST_DEBUG_INDEX__DAC_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define DAC_TEST_DEBUG_INDEX__DAC_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define DAC_TEST_DEBUG_DATA__DAC_TEST_DEBUG_DATA_MASK 0xffffffff
+#define DAC_TEST_DEBUG_DATA__DAC_TEST_DEBUG_DATA__SHIFT 0x0
+#define PERFCOUNTER_CNTL__PERFCOUNTER_EVENT_SEL_MASK 0x1ff
+#define PERFCOUNTER_CNTL__PERFCOUNTER_EVENT_SEL__SHIFT 0x0
+#define PERFCOUNTER_CNTL__PERFCOUNTER_CVALUE_SEL_MASK 0xe00
+#define PERFCOUNTER_CNTL__PERFCOUNTER_CVALUE_SEL__SHIFT 0x9
+#define PERFCOUNTER_CNTL__PERFCOUNTER_INC_MODE_MASK 0x3000
+#define PERFCOUNTER_CNTL__PERFCOUNTER_INC_MODE__SHIFT 0xc
+#define PERFCOUNTER_CNTL__PERFCOUNTER_HW_CNTL_SEL_MASK 0x4000
+#define PERFCOUNTER_CNTL__PERFCOUNTER_HW_CNTL_SEL__SHIFT 0xe
+#define PERFCOUNTER_CNTL__PERFCOUNTER_RUNEN_MODE_MASK 0x8000
+#define PERFCOUNTER_CNTL__PERFCOUNTER_RUNEN_MODE__SHIFT 0xf
+#define PERFCOUNTER_CNTL__PERFCOUNTER_CNTOFF_SEL_MASK 0x1f0000
+#define PERFCOUNTER_CNTL__PERFCOUNTER_CNTOFF_SEL__SHIFT 0x10
+#define PERFCOUNTER_CNTL__PERFCOUNTER_CNTOFF_START_DIS_MASK 0x200000
+#define PERFCOUNTER_CNTL__PERFCOUNTER_CNTOFF_START_DIS__SHIFT 0x15
+#define PERFCOUNTER_CNTL__PERFCOUNTER_RESTART_EN_MASK 0x400000
+#define PERFCOUNTER_CNTL__PERFCOUNTER_RESTART_EN__SHIFT 0x16
+#define PERFCOUNTER_CNTL__PERFCOUNTER_INT_EN_MASK 0x800000
+#define PERFCOUNTER_CNTL__PERFCOUNTER_INT_EN__SHIFT 0x17
+#define PERFCOUNTER_CNTL__PERFCOUNTER_OFF_MASK_MASK 0x1000000
+#define PERFCOUNTER_CNTL__PERFCOUNTER_OFF_MASK__SHIFT 0x18
+#define PERFCOUNTER_CNTL__PERFCOUNTER_ACTIVE_MASK 0x2000000
+#define PERFCOUNTER_CNTL__PERFCOUNTER_ACTIVE__SHIFT 0x19
+#define PERFCOUNTER_CNTL__PERFCOUNTER_INT_TYPE_MASK 0x4000000
+#define PERFCOUNTER_CNTL__PERFCOUNTER_INT_TYPE__SHIFT 0x1a
+#define PERFCOUNTER_CNTL__PERFCOUNTER_COUNTED_VALUE_TYPE_MASK 0x8000000
+#define PERFCOUNTER_CNTL__PERFCOUNTER_COUNTED_VALUE_TYPE__SHIFT 0x1b
+#define PERFCOUNTER_CNTL__PERFCOUNTER_CNTL_SEL_MASK 0xe0000000
+#define PERFCOUNTER_CNTL__PERFCOUNTER_CNTL_SEL__SHIFT 0x1d
+#define PERFCOUNTER_STATE__PERFCOUNTER_CNT0_STATE_MASK 0x3
+#define PERFCOUNTER_STATE__PERFCOUNTER_CNT0_STATE__SHIFT 0x0
+#define PERFCOUNTER_STATE__PERFCOUNTER_STATE_SEL0_MASK 0x4
+#define PERFCOUNTER_STATE__PERFCOUNTER_STATE_SEL0__SHIFT 0x2
+#define PERFCOUNTER_STATE__PERFCOUNTER_CNT1_STATE_MASK 0x30
+#define PERFCOUNTER_STATE__PERFCOUNTER_CNT1_STATE__SHIFT 0x4
+#define PERFCOUNTER_STATE__PERFCOUNTER_STATE_SEL1_MASK 0x40
+#define PERFCOUNTER_STATE__PERFCOUNTER_STATE_SEL1__SHIFT 0x6
+#define PERFCOUNTER_STATE__PERFCOUNTER_CNT2_STATE_MASK 0x300
+#define PERFCOUNTER_STATE__PERFCOUNTER_CNT2_STATE__SHIFT 0x8
+#define PERFCOUNTER_STATE__PERFCOUNTER_STATE_SEL2_MASK 0x400
+#define PERFCOUNTER_STATE__PERFCOUNTER_STATE_SEL2__SHIFT 0xa
+#define PERFCOUNTER_STATE__PERFCOUNTER_CNT3_STATE_MASK 0x3000
+#define PERFCOUNTER_STATE__PERFCOUNTER_CNT3_STATE__SHIFT 0xc
+#define PERFCOUNTER_STATE__PERFCOUNTER_STATE_SEL3_MASK 0x4000
+#define PERFCOUNTER_STATE__PERFCOUNTER_STATE_SEL3__SHIFT 0xe
+#define PERFCOUNTER_STATE__PERFCOUNTER_CNT4_STATE_MASK 0x30000
+#define PERFCOUNTER_STATE__PERFCOUNTER_CNT4_STATE__SHIFT 0x10
+#define PERFCOUNTER_STATE__PERFCOUNTER_STATE_SEL4_MASK 0x40000
+#define PERFCOUNTER_STATE__PERFCOUNTER_STATE_SEL4__SHIFT 0x12
+#define PERFCOUNTER_STATE__PERFCOUNTER_CNT5_STATE_MASK 0x300000
+#define PERFCOUNTER_STATE__PERFCOUNTER_CNT5_STATE__SHIFT 0x14
+#define PERFCOUNTER_STATE__PERFCOUNTER_STATE_SEL5_MASK 0x400000
+#define PERFCOUNTER_STATE__PERFCOUNTER_STATE_SEL5__SHIFT 0x16
+#define PERFCOUNTER_STATE__PERFCOUNTER_CNT6_STATE_MASK 0x3000000
+#define PERFCOUNTER_STATE__PERFCOUNTER_CNT6_STATE__SHIFT 0x18
+#define PERFCOUNTER_STATE__PERFCOUNTER_STATE_SEL6_MASK 0x4000000
+#define PERFCOUNTER_STATE__PERFCOUNTER_STATE_SEL6__SHIFT 0x1a
+#define PERFCOUNTER_STATE__PERFCOUNTER_CNT7_STATE_MASK 0x30000000
+#define PERFCOUNTER_STATE__PERFCOUNTER_CNT7_STATE__SHIFT 0x1c
+#define PERFCOUNTER_STATE__PERFCOUNTER_STATE_SEL7_MASK 0x40000000
+#define PERFCOUNTER_STATE__PERFCOUNTER_STATE_SEL7__SHIFT 0x1e
+#define PERFMON_CNTL__PERFMON_STATE_MASK 0x3
+#define PERFMON_CNTL__PERFMON_STATE__SHIFT 0x0
+#define PERFMON_CNTL__PERFMON_RUN_ENABLE_SEL_MASK 0xfc
+#define PERFMON_CNTL__PERFMON_RUN_ENABLE_SEL__SHIFT 0x2
+#define PERFMON_CNTL__PERFMON_RPT_COUNT_MASK 0xfffff00
+#define PERFMON_CNTL__PERFMON_RPT_COUNT__SHIFT 0x8
+#define PERFMON_CNTL__PERFMON_CNTOFF_AND_OR_MASK 0x10000000
+#define PERFMON_CNTL__PERFMON_CNTOFF_AND_OR__SHIFT 0x1c
+#define PERFMON_CNTL__PERFMON_CNTOFF_INT_EN_MASK 0x20000000
+#define PERFMON_CNTL__PERFMON_CNTOFF_INT_EN__SHIFT 0x1d
+#define PERFMON_CNTL__PERFMON_CNTOFF_INT_STATUS_MASK 0x40000000
+#define PERFMON_CNTL__PERFMON_CNTOFF_INT_STATUS__SHIFT 0x1e
+#define PERFMON_CNTL__PERFMON_CNTOFF_INT_ACK_MASK 0x80000000
+#define PERFMON_CNTL__PERFMON_CNTOFF_INT_ACK__SHIFT 0x1f
+#define PERFMON_CNTL2__PERFMON_CNTOFF_INT_TYPE_MASK 0x1
+#define PERFMON_CNTL2__PERFMON_CNTOFF_INT_TYPE__SHIFT 0x0
+#define PERFMON_CNTL2__PERFMON_CLK_ENABLE_MASK 0x2
+#define PERFMON_CNTL2__PERFMON_CLK_ENABLE__SHIFT 0x1
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT0_STATUS_MASK 0x1
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT0_STATUS__SHIFT 0x0
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT1_STATUS_MASK 0x2
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT1_STATUS__SHIFT 0x1
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT2_STATUS_MASK 0x4
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT2_STATUS__SHIFT 0x2
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT3_STATUS_MASK 0x8
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT3_STATUS__SHIFT 0x3
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT4_STATUS_MASK 0x10
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT4_STATUS__SHIFT 0x4
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT5_STATUS_MASK 0x20
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT5_STATUS__SHIFT 0x5
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT6_STATUS_MASK 0x40
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT6_STATUS__SHIFT 0x6
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT7_STATUS_MASK 0x80
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT7_STATUS__SHIFT 0x7
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT0_ACK_MASK 0x100
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT0_ACK__SHIFT 0x8
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT1_ACK_MASK 0x200
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT1_ACK__SHIFT 0x9
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT2_ACK_MASK 0x400
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT2_ACK__SHIFT 0xa
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT3_ACK_MASK 0x800
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT3_ACK__SHIFT 0xb
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT4_ACK_MASK 0x1000
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT4_ACK__SHIFT 0xc
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT5_ACK_MASK 0x2000
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT5_ACK__SHIFT 0xd
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT6_ACK_MASK 0x4000
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT6_ACK__SHIFT 0xe
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT7_ACK_MASK 0x8000
+#define PERFMON_CVALUE_INT_MISC__PERFCOUNTER_INT7_ACK__SHIFT 0xf
+#define PERFMON_CVALUE_INT_MISC__PERFMON_CVALUE_HI_MASK 0xffff0000
+#define PERFMON_CVALUE_INT_MISC__PERFMON_CVALUE_HI__SHIFT 0x10
+#define PERFMON_CVALUE_LOW__PERFMON_CVALUE_LOW_MASK 0xffffffff
+#define PERFMON_CVALUE_LOW__PERFMON_CVALUE_LOW__SHIFT 0x0
+#define PERFMON_HI__PERFMON_HI_MASK 0xffff
+#define PERFMON_HI__PERFMON_HI__SHIFT 0x0
+#define PERFMON_HI__PERFMON_READ_SEL_MASK 0xe0000000
+#define PERFMON_HI__PERFMON_READ_SEL__SHIFT 0x1d
+#define PERFMON_LOW__PERFMON_LOW_MASK 0xffffffff
+#define PERFMON_LOW__PERFMON_LOW__SHIFT 0x0
+#define PERFMON_TEST_DEBUG_INDEX__PERFMON_TEST_DEBUG_INDEX_MASK 0xff
+#define PERFMON_TEST_DEBUG_INDEX__PERFMON_TEST_DEBUG_INDEX__SHIFT 0x0
+#define PERFMON_TEST_DEBUG_INDEX__PERFMON_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define PERFMON_TEST_DEBUG_INDEX__PERFMON_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define PERFMON_TEST_DEBUG_DATA__PERFMON_TEST_DEBUG_DATA_MASK 0xffffffff
+#define PERFMON_TEST_DEBUG_DATA__PERFMON_TEST_DEBUG_DATA__SHIFT 0x0
+#define REFCLK_CNTL__REFCLK_CLOCK_EN_MASK 0x1
+#define REFCLK_CNTL__REFCLK_CLOCK_EN__SHIFT 0x0
+#define REFCLK_CNTL__REFCLK_SRC_SEL_MASK 0x2
+#define REFCLK_CNTL__REFCLK_SRC_SEL__SHIFT 0x1
+#define DCCG_CBUS_ANTIGLITCH_RESETB__P0PLL_CBUS_ANTIGLITCH_RESETB_MASK 0x1
+#define DCCG_CBUS_ANTIGLITCH_RESETB__P0PLL_CBUS_ANTIGLITCH_RESETB__SHIFT 0x0
+#define DCCG_CBUS_ANTIGLITCH_RESETB__P1PLL_CBUS_ANTIGLITCH_RESETB_MASK 0x2
+#define DCCG_CBUS_ANTIGLITCH_RESETB__P1PLL_CBUS_ANTIGLITCH_RESETB__SHIFT 0x1
+#define DCCG_CBUS_ANTIGLITCH_RESETB__P2PLL_CBUS_ANTIGLITCH_RESETB_MASK 0x4
+#define DCCG_CBUS_ANTIGLITCH_RESETB__P2PLL_CBUS_ANTIGLITCH_RESETB__SHIFT 0x2
+#define DCCG_CBUS_ANTIGLITCH_RESETB__P3PLL_CBUS_ANTIGLITCH_RESETB_MASK 0x8
+#define DCCG_CBUS_ANTIGLITCH_RESETB__P3PLL_CBUS_ANTIGLITCH_RESETB__SHIFT 0x3
+#define DCCG_CBUS_SPARE__P0PLL_CBUS_SPARE_MASK 0xff
+#define DCCG_CBUS_SPARE__P0PLL_CBUS_SPARE__SHIFT 0x0
+#define DCCG_CBUS_SPARE__P1PLL_CBUS_SPARE_MASK 0xff00
+#define DCCG_CBUS_SPARE__P1PLL_CBUS_SPARE__SHIFT 0x8
+#define DCCG_CBUS_SPARE__P2PLL_CBUS_SPARE_MASK 0xff0000
+#define DCCG_CBUS_SPARE__P2PLL_CBUS_SPARE__SHIFT 0x10
+#define DCCG_CBUS_SPARE__P3PLL_CBUS_SPARE_MASK 0xff000000
+#define DCCG_CBUS_SPARE__P3PLL_CBUS_SPARE__SHIFT 0x18
+#define DCCG_CBUS_WRCMD_DELAY__CBUS_PLL_WRCMD_DELAY_MASK 0xf
+#define DCCG_CBUS_WRCMD_DELAY__CBUS_PLL_WRCMD_DELAY__SHIFT 0x0
+#define DPREFCLK_CNTL__DPREFCLK_SRC_SEL_MASK 0x7
+#define DPREFCLK_CNTL__DPREFCLK_SRC_SEL__SHIFT 0x0
+#define DPREFCLK_CNTL__UNB_DB_CLK_ENABLE_MASK 0x100
+#define DPREFCLK_CNTL__UNB_DB_CLK_ENABLE__SHIFT 0x8
+#define DCE_VERSION__MAJOR_VERSION_MASK 0xff
+#define DCE_VERSION__MAJOR_VERSION__SHIFT 0x0
+#define DCE_VERSION__MINOR_VERSION_MASK 0xff00
+#define DCE_VERSION__MINOR_VERSION__SHIFT 0x8
+#define AVSYNC_COUNTER_WRITE__AVSYNC_COUNTER_WRVALUE_MASK 0xffffffff
+#define AVSYNC_COUNTER_WRITE__AVSYNC_COUNTER_WRVALUE__SHIFT 0x0
+#define AVSYNC_COUNTER_CONTROL__AVSYNC_COUNTER_ENABLE_MASK 0x1
+#define AVSYNC_COUNTER_CONTROL__AVSYNC_COUNTER_ENABLE__SHIFT 0x0
+#define AVSYNC_COUNTER_READ__AVSYNC_COUNTER_RDVALUE_MASK 0xffffffff
+#define AVSYNC_COUNTER_READ__AVSYNC_COUNTER_RDVALUE__SHIFT 0x0
+#define DCCG_GTC_CNTL__DCCG_GTC_ENABLE_MASK 0x1
+#define DCCG_GTC_CNTL__DCCG_GTC_ENABLE__SHIFT 0x0
+#define DCCG_GTC_DTO_INCR__DCCG_GTC_DTO_INCR_MASK 0xffffffff
+#define DCCG_GTC_DTO_INCR__DCCG_GTC_DTO_INCR__SHIFT 0x0
+#define DCCG_GTC_DTO_MODULO__DCCG_GTC_DTO_MODULO_MASK 0xffffffff
+#define DCCG_GTC_DTO_MODULO__DCCG_GTC_DTO_MODULO__SHIFT 0x0
+#define DCCG_GTC_CURRENT__DCCG_GTC_CURRENT_MASK 0xffffffff
+#define DCCG_GTC_CURRENT__DCCG_GTC_CURRENT__SHIFT 0x0
+#define DCCG_DS_DTO_INCR__DCCG_DS_DTO_INCR_MASK 0xffffffff
+#define DCCG_DS_DTO_INCR__DCCG_DS_DTO_INCR__SHIFT 0x0
+#define DCCG_DS_DTO_MODULO__DCCG_DS_DTO_MODULO_MASK 0xffffffff
+#define DCCG_DS_DTO_MODULO__DCCG_DS_DTO_MODULO__SHIFT 0x0
+#define DCCG_DS_CNTL__DCCG_DS_ENABLE_MASK 0x1
+#define DCCG_DS_CNTL__DCCG_DS_ENABLE__SHIFT 0x0
+#define DCCG_DS_CNTL__DCCG_DS_REF_SRC_MASK 0x30
+#define DCCG_DS_CNTL__DCCG_DS_REF_SRC__SHIFT 0x4
+#define DCCG_DS_CNTL__DCCG_DS_HW_CAL_ENABLE_MASK 0x100
+#define DCCG_DS_CNTL__DCCG_DS_HW_CAL_ENABLE__SHIFT 0x8
+#define DCCG_DS_CNTL__DCCG_DS_ENABLED_STATUS_MASK 0x200
+#define DCCG_DS_CNTL__DCCG_DS_ENABLED_STATUS__SHIFT 0x9
+#define DCCG_DS_CNTL__DCCG_DS_XTALIN_RATE_DIV_MASK 0x30000
+#define DCCG_DS_CNTL__DCCG_DS_XTALIN_RATE_DIV__SHIFT 0x10
+#define DCCG_DS_CNTL__DCCG_DS_JITTER_REMOVE_DIS_MASK 0x1000000
+#define DCCG_DS_CNTL__DCCG_DS_JITTER_REMOVE_DIS__SHIFT 0x18
+#define DCCG_DS_CNTL__DCCG_DS_DELAY_XTAL_SEL_MASK 0x2000000
+#define DCCG_DS_CNTL__DCCG_DS_DELAY_XTAL_SEL__SHIFT 0x19
+#define DCCG_DS_HW_CAL_INTERVAL__DCCG_DS_HW_CAL_INTERVAL_MASK 0xffffffff
+#define DCCG_DS_HW_CAL_INTERVAL__DCCG_DS_HW_CAL_INTERVAL__SHIFT 0x0
+#define DCCG_DS_DEBUG_CNTL__DCCG_DS_DEBUG_COUNT_ENABLE_MASK 0x1
+#define DCCG_DS_DEBUG_CNTL__DCCG_DS_DEBUG_COUNT_ENABLE__SHIFT 0x0
+#define DCCG_DS_DEBUG_CNTL__DCCG_DS_DEBUG_COUNT_TRIG_VALUE_MASK 0x1ff0
+#define DCCG_DS_DEBUG_CNTL__DCCG_DS_DEBUG_COUNT_TRIG_VALUE__SHIFT 0x4
+#define DCCG_DS_DEBUG_CNTL__DCCG_DS_DEBUG_COUNT_TRIG_OCCURRED_MASK 0x10000
+#define DCCG_DS_DEBUG_CNTL__DCCG_DS_DEBUG_COUNT_TRIG_OCCURRED__SHIFT 0x10
+#define DCCG_DS_DEBUG_CNTL__DCCG_DS_DEBUG_COUNT_TRIG_CLEAR_MASK 0x20000
+#define DCCG_DS_DEBUG_CNTL__DCCG_DS_DEBUG_COUNT_TRIG_CLEAR__SHIFT 0x11
+#define DCCG_DS_DEBUG_CNTL__DCCG_DS_JITTER_COUNT_ENABLE_MASK 0x100000
+#define DCCG_DS_DEBUG_CNTL__DCCG_DS_JITTER_COUNT_ENABLE__SHIFT 0x14
+#define DCCG_DS_DEBUG_CNTL__DCCG_DS_JITTER_COUNT_SRC_SEL_MASK 0x200000
+#define DCCG_DS_DEBUG_CNTL__DCCG_DS_JITTER_COUNT_SRC_SEL__SHIFT 0x15
+#define DCCG_DS_DEBUG_CNTL__DCCG_DS_JITTER_COUNT_MASK 0xff000000
+#define DCCG_DS_DEBUG_CNTL__DCCG_DS_JITTER_COUNT__SHIFT 0x18
+#define DMCU_SMU_INTERRUPT_CNTL__DMCU_SMU_STATIC_SCREEN_INT_MASK 0x1
+#define DMCU_SMU_INTERRUPT_CNTL__DMCU_SMU_STATIC_SCREEN_INT__SHIFT 0x0
+#define DMCU_SMU_INTERRUPT_CNTL__DMCU_SMU_STATIC_SCREEN_STATUS_MASK 0xffff0000
+#define DMCU_SMU_INTERRUPT_CNTL__DMCU_SMU_STATIC_SCREEN_STATUS__SHIFT 0x10
+#define SMU_CONTROL__DISPLAY0_FORCE_VBI_MASK 0x1
+#define SMU_CONTROL__DISPLAY0_FORCE_VBI__SHIFT 0x0
+#define SMU_CONTROL__DISPLAY1_FORCE_VBI_MASK 0x2
+#define SMU_CONTROL__DISPLAY1_FORCE_VBI__SHIFT 0x1
+#define SMU_CONTROL__DISPLAY2_FORCE_VBI_MASK 0x4
+#define SMU_CONTROL__DISPLAY2_FORCE_VBI__SHIFT 0x2
+#define SMU_CONTROL__DISPLAY3_FORCE_VBI_MASK 0x8
+#define SMU_CONTROL__DISPLAY3_FORCE_VBI__SHIFT 0x3
+#define SMU_CONTROL__DISPLAY4_FORCE_VBI_MASK 0x10
+#define SMU_CONTROL__DISPLAY4_FORCE_VBI__SHIFT 0x4
+#define SMU_CONTROL__DISPLAY5_FORCE_VBI_MASK 0x20
+#define SMU_CONTROL__DISPLAY5_FORCE_VBI__SHIFT 0x5
+#define SMU_CONTROL__DISPLAY_V0_FORCE_VBI_MASK 0x40
+#define SMU_CONTROL__DISPLAY_V0_FORCE_VBI__SHIFT 0x6
+#define SMU_CONTROL__DISPLAY_V1_FORCE_VBI_MASK 0x80
+#define SMU_CONTROL__DISPLAY_V1_FORCE_VBI__SHIFT 0x7
+#define SMU_CONTROL__MCIF_WB_FORCE_VBI_MASK 0x100
+#define SMU_CONTROL__MCIF_WB_FORCE_VBI__SHIFT 0x8
+#define SMU_CONTROL__SMU_DC_INT_CLEAR_MASK 0x10000
+#define SMU_CONTROL__SMU_DC_INT_CLEAR__SHIFT 0x10
+#define SMU_INTERRUPT_CONTROL__DC_SMU_INT_ENABLE_MASK 0x1
+#define SMU_INTERRUPT_CONTROL__DC_SMU_INT_ENABLE__SHIFT 0x0
+#define SMU_INTERRUPT_CONTROL__DC_SMU_INT_STATUS_MASK 0x10
+#define SMU_INTERRUPT_CONTROL__DC_SMU_INT_STATUS__SHIFT 0x4
+#define SMU_INTERRUPT_CONTROL__DC_SMU_INT_EVENT_MASK 0xffff0000
+#define SMU_INTERRUPT_CONTROL__DC_SMU_INT_EVENT__SHIFT 0x10
+#define DAC_CLK_ENABLE__DACA_CLK_ENABLE_MASK 0x1
+#define DAC_CLK_ENABLE__DACA_CLK_ENABLE__SHIFT 0x0
+#define DAC_CLK_ENABLE__DACB_CLK_ENABLE_MASK 0x10
+#define DAC_CLK_ENABLE__DACB_CLK_ENABLE__SHIFT 0x4
+#define DVO_CLK_ENABLE__DVO_CLK_ENABLE_MASK 0x1
+#define DVO_CLK_ENABLE__DVO_CLK_ENABLE__SHIFT 0x0
+#define DCCG_GATE_DISABLE_CNTL__DISPCLK_DCCG_GATE_DISABLE_MASK 0x1
+#define DCCG_GATE_DISABLE_CNTL__DISPCLK_DCCG_GATE_DISABLE__SHIFT 0x0
+#define DCCG_GATE_DISABLE_CNTL__DISPCLK_R_DCCG_GATE_DISABLE_MASK 0x2
+#define DCCG_GATE_DISABLE_CNTL__DISPCLK_R_DCCG_GATE_DISABLE__SHIFT 0x1
+#define DCCG_GATE_DISABLE_CNTL__SCLK_GATE_DISABLE_MASK 0x4
+#define DCCG_GATE_DISABLE_CNTL__SCLK_GATE_DISABLE__SHIFT 0x2
+#define DCCG_GATE_DISABLE_CNTL__DPREFCLK_GATE_DISABLE_MASK 0x8
+#define DCCG_GATE_DISABLE_CNTL__DPREFCLK_GATE_DISABLE__SHIFT 0x3
+#define DCCG_GATE_DISABLE_CNTL__DACACLK_GATE_DISABLE_MASK 0x10
+#define DCCG_GATE_DISABLE_CNTL__DACACLK_GATE_DISABLE__SHIFT 0x4
+#define DCCG_GATE_DISABLE_CNTL__DACBCLK_GATE_DISABLE_MASK 0x20
+#define DCCG_GATE_DISABLE_CNTL__DACBCLK_GATE_DISABLE__SHIFT 0x5
+#define DCCG_GATE_DISABLE_CNTL__DVOACLK_GATE_DISABLE_MASK 0x40
+#define DCCG_GATE_DISABLE_CNTL__DVOACLK_GATE_DISABLE__SHIFT 0x6
+#define DCCG_GATE_DISABLE_CNTL__DPDBG_CLK_GATE_DISABLE_MASK 0x80
+#define DCCG_GATE_DISABLE_CNTL__DPDBG_CLK_GATE_DISABLE__SHIFT 0x7
+#define DCCG_GATE_DISABLE_CNTL__DPREFCLK_R_DCCG_GATE_DISABLE_MASK 0x100
+#define DCCG_GATE_DISABLE_CNTL__DPREFCLK_R_DCCG_GATE_DISABLE__SHIFT 0x8
+#define DCCG_GATE_DISABLE_CNTL__AOMCLK0_GATE_DISABLE_MASK 0x20000
+#define DCCG_GATE_DISABLE_CNTL__AOMCLK0_GATE_DISABLE__SHIFT 0x11
+#define DCCG_GATE_DISABLE_CNTL__AOMCLK1_GATE_DISABLE_MASK 0x40000
+#define DCCG_GATE_DISABLE_CNTL__AOMCLK1_GATE_DISABLE__SHIFT 0x12
+#define DCCG_GATE_DISABLE_CNTL__AOMCLK2_GATE_DISABLE_MASK 0x80000
+#define DCCG_GATE_DISABLE_CNTL__AOMCLK2_GATE_DISABLE__SHIFT 0x13
+#define DCCG_GATE_DISABLE_CNTL__AUDIO_DTO2_CLK_GATE_DISABLE_MASK 0x200000
+#define DCCG_GATE_DISABLE_CNTL__AUDIO_DTO2_CLK_GATE_DISABLE__SHIFT 0x15
+#define DCCG_GATE_DISABLE_CNTL__DPREFCLK_GTC_GATE_DISABLE_MASK 0x400000
+#define DCCG_GATE_DISABLE_CNTL__DPREFCLK_GTC_GATE_DISABLE__SHIFT 0x16
+#define DCCG_GATE_DISABLE_CNTL__UNB_DB_CLK_GATE_DISABLE_MASK 0x800000
+#define DCCG_GATE_DISABLE_CNTL__UNB_DB_CLK_GATE_DISABLE__SHIFT 0x17
+#define DCCG_GATE_DISABLE_CNTL__REFCLK_GATE_DISABLE_MASK 0x4000000
+#define DCCG_GATE_DISABLE_CNTL__REFCLK_GATE_DISABLE__SHIFT 0x1a
+#define DCCG_GATE_DISABLE_CNTL__REFCLK_R_DIG_GATE_DISABLE_MASK 0x8000000
+#define DCCG_GATE_DISABLE_CNTL__REFCLK_R_DIG_GATE_DISABLE__SHIFT 0x1b
+#define DCCG_GATE_DISABLE_CNTL__DSICLK_GATE_DISABLE_MASK 0x10000000
+#define DCCG_GATE_DISABLE_CNTL__DSICLK_GATE_DISABLE__SHIFT 0x1c
+#define DCCG_GATE_DISABLE_CNTL__BYTECLK_GATE_DISABLE_MASK 0x20000000
+#define DCCG_GATE_DISABLE_CNTL__BYTECLK_GATE_DISABLE__SHIFT 0x1d
+#define DCCG_GATE_DISABLE_CNTL__ESCCLK_GATE_DISABLE_MASK 0x40000000
+#define DCCG_GATE_DISABLE_CNTL__ESCCLK_GATE_DISABLE__SHIFT 0x1e
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKA_FE_GATE_DISABLE_MASK 0x1
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKA_FE_GATE_DISABLE__SHIFT 0x0
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKB_FE_GATE_DISABLE_MASK 0x2
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKB_FE_GATE_DISABLE__SHIFT 0x1
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKC_FE_GATE_DISABLE_MASK 0x4
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKC_FE_GATE_DISABLE__SHIFT 0x2
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKD_FE_GATE_DISABLE_MASK 0x8
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKD_FE_GATE_DISABLE__SHIFT 0x3
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKE_FE_GATE_DISABLE_MASK 0x10
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKE_FE_GATE_DISABLE__SHIFT 0x4
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKF_FE_GATE_DISABLE_MASK 0x20
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKF_FE_GATE_DISABLE__SHIFT 0x5
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKG_FE_GATE_DISABLE_MASK 0x40
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKG_FE_GATE_DISABLE__SHIFT 0x6
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKLPA_FE_GATE_DISABLE_MASK 0x100
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKLPA_FE_GATE_DISABLE__SHIFT 0x8
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKLPB_FE_GATE_DISABLE_MASK 0x200
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKLPB_FE_GATE_DISABLE__SHIFT 0x9
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKA_GATE_DISABLE_MASK 0x10000
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKA_GATE_DISABLE__SHIFT 0x10
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKB_GATE_DISABLE_MASK 0x20000
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKB_GATE_DISABLE__SHIFT 0x11
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKC_GATE_DISABLE_MASK 0x40000
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKC_GATE_DISABLE__SHIFT 0x12
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKD_GATE_DISABLE_MASK 0x80000
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKD_GATE_DISABLE__SHIFT 0x13
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKE_GATE_DISABLE_MASK 0x100000
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKE_GATE_DISABLE__SHIFT 0x14
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKF_GATE_DISABLE_MASK 0x200000
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKF_GATE_DISABLE__SHIFT 0x15
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKG_GATE_DISABLE_MASK 0x400000
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKG_GATE_DISABLE__SHIFT 0x16
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKLPA_GATE_DISABLE_MASK 0x1000000
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKLPA_GATE_DISABLE__SHIFT 0x18
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKLPB_GATE_DISABLE_MASK 0x2000000
+#define DCCG_GATE_DISABLE_CNTL2__SYMCLKLPB_GATE_DISABLE__SHIFT 0x19
+#define DISPCLK_CGTT_BLK_CTRL_REG__DISPCLK_TURN_ON_DELAY_MASK 0xf
+#define DISPCLK_CGTT_BLK_CTRL_REG__DISPCLK_TURN_ON_DELAY__SHIFT 0x0
+#define DISPCLK_CGTT_BLK_CTRL_REG__DISPCLK_TURN_OFF_DELAY_MASK 0xff0
+#define DISPCLK_CGTT_BLK_CTRL_REG__DISPCLK_TURN_OFF_DELAY__SHIFT 0x4
+#define SCLK_CGTT_BLK_CTRL_REG__SCLK_TURN_ON_DELAY_MASK 0xf
+#define SCLK_CGTT_BLK_CTRL_REG__SCLK_TURN_ON_DELAY__SHIFT 0x0
+#define SCLK_CGTT_BLK_CTRL_REG__SCLK_TURN_OFF_DELAY_MASK 0xff0
+#define SCLK_CGTT_BLK_CTRL_REG__SCLK_TURN_OFF_DELAY__SHIFT 0x4
+#define SCLK_CGTT_BLK_CTRL_REG__CGTT_SCLK_OVERRIDE_MASK 0x1000
+#define SCLK_CGTT_BLK_CTRL_REG__CGTT_SCLK_OVERRIDE__SHIFT 0xc
+#define DPREFCLK_CGTT_BLK_CTRL_REG__DPREFCLK_TURN_ON_DELAY_MASK 0xf
+#define DPREFCLK_CGTT_BLK_CTRL_REG__DPREFCLK_TURN_ON_DELAY__SHIFT 0x0
+#define DPREFCLK_CGTT_BLK_CTRL_REG__DPREFCLK_TURN_OFF_DELAY_MASK 0xff0
+#define DPREFCLK_CGTT_BLK_CTRL_REG__DPREFCLK_TURN_OFF_DELAY__SHIFT 0x4
+#define REFCLK_CGTT_BLK_CTRL_REG__REFCLK_TURN_ON_DELAY_MASK 0xf
+#define REFCLK_CGTT_BLK_CTRL_REG__REFCLK_TURN_ON_DELAY__SHIFT 0x0
+#define REFCLK_CGTT_BLK_CTRL_REG__REFCLK_TURN_OFF_DELAY_MASK 0xff0
+#define REFCLK_CGTT_BLK_CTRL_REG__REFCLK_TURN_OFF_DELAY__SHIFT 0x4
+#define SYMCLK_CGTT_BLK_CTRL_REG__SYMCLK_TURN_ON_DELAY_MASK 0xf
+#define SYMCLK_CGTT_BLK_CTRL_REG__SYMCLK_TURN_ON_DELAY__SHIFT 0x0
+#define SYMCLK_CGTT_BLK_CTRL_REG__SYMCLK_TURN_OFF_DELAY_MASK 0xff0
+#define SYMCLK_CGTT_BLK_CTRL_REG__SYMCLK_TURN_OFF_DELAY__SHIFT 0x4
+#define DCCG_CAC_STATUS__CAC_STATUS_RDDATA_MASK 0xffffffff
+#define DCCG_CAC_STATUS__CAC_STATUS_RDDATA__SHIFT 0x0
+#define PIXCLK0_RESYNC_CNTL__PIXCLK0_RESYNC_ENABLE_MASK 0x1
+#define PIXCLK0_RESYNC_CNTL__PIXCLK0_RESYNC_ENABLE__SHIFT 0x0
+#define PIXCLK0_RESYNC_CNTL__DCCG_DEEP_COLOR_CNTL0_MASK 0x30
+#define PIXCLK0_RESYNC_CNTL__DCCG_DEEP_COLOR_CNTL0__SHIFT 0x4
+#define PHYPLLA_PIXCLK_RESYNC_CNTL__PHYPLLA_PIXCLK_RESYNC_ENABLE_MASK 0x1
+#define PHYPLLA_PIXCLK_RESYNC_CNTL__PHYPLLA_PIXCLK_RESYNC_ENABLE__SHIFT 0x0
+#define PHYPLLA_PIXCLK_RESYNC_CNTL__PHYPLLA_DCCG_DEEP_COLOR_CNTL_MASK 0x30
+#define PHYPLLA_PIXCLK_RESYNC_CNTL__PHYPLLA_DCCG_DEEP_COLOR_CNTL__SHIFT 0x4
+#define PHYPLLA_PIXCLK_RESYNC_CNTL__PHYPLLA_PIXCLK_ENABLE_MASK 0x100
+#define PHYPLLA_PIXCLK_RESYNC_CNTL__PHYPLLA_PIXCLK_ENABLE__SHIFT 0x8
+#define PHYPLLA_PIXCLK_RESYNC_CNTL__PHYPLLA_PIXCLK_DOUBLE_RATE_ENABLE_MASK 0x200
+#define PHYPLLA_PIXCLK_RESYNC_CNTL__PHYPLLA_PIXCLK_DOUBLE_RATE_ENABLE__SHIFT 0x9
+#define PHYPLLB_PIXCLK_RESYNC_CNTL__PHYPLLB_PIXCLK_RESYNC_ENABLE_MASK 0x1
+#define PHYPLLB_PIXCLK_RESYNC_CNTL__PHYPLLB_PIXCLK_RESYNC_ENABLE__SHIFT 0x0
+#define PHYPLLB_PIXCLK_RESYNC_CNTL__PHYPLLB_DCCG_DEEP_COLOR_CNTL_MASK 0x30
+#define PHYPLLB_PIXCLK_RESYNC_CNTL__PHYPLLB_DCCG_DEEP_COLOR_CNTL__SHIFT 0x4
+#define PHYPLLB_PIXCLK_RESYNC_CNTL__PHYPLLB_PIXCLK_ENABLE_MASK 0x100
+#define PHYPLLB_PIXCLK_RESYNC_CNTL__PHYPLLB_PIXCLK_ENABLE__SHIFT 0x8
+#define PHYPLLB_PIXCLK_RESYNC_CNTL__PHYPLLB_PIXCLK_DOUBLE_RATE_ENABLE_MASK 0x200
+#define PHYPLLB_PIXCLK_RESYNC_CNTL__PHYPLLB_PIXCLK_DOUBLE_RATE_ENABLE__SHIFT 0x9
+#define PHYPLLC_PIXCLK_RESYNC_CNTL__PHYPLLC_PIXCLK_RESYNC_ENABLE_MASK 0x1
+#define PHYPLLC_PIXCLK_RESYNC_CNTL__PHYPLLC_PIXCLK_RESYNC_ENABLE__SHIFT 0x0
+#define PHYPLLC_PIXCLK_RESYNC_CNTL__PHYPLLC_DCCG_DEEP_COLOR_CNTL_MASK 0x30
+#define PHYPLLC_PIXCLK_RESYNC_CNTL__PHYPLLC_DCCG_DEEP_COLOR_CNTL__SHIFT 0x4
+#define PHYPLLC_PIXCLK_RESYNC_CNTL__PHYPLLC_PIXCLK_ENABLE_MASK 0x100
+#define PHYPLLC_PIXCLK_RESYNC_CNTL__PHYPLLC_PIXCLK_ENABLE__SHIFT 0x8
+#define PHYPLLC_PIXCLK_RESYNC_CNTL__PHYPLLC_PIXCLK_DOUBLE_RATE_ENABLE_MASK 0x200
+#define PHYPLLC_PIXCLK_RESYNC_CNTL__PHYPLLC_PIXCLK_DOUBLE_RATE_ENABLE__SHIFT 0x9
+#define PHYPLLD_PIXCLK_RESYNC_CNTL__PHYPLLD_PIXCLK_RESYNC_ENABLE_MASK 0x1
+#define PHYPLLD_PIXCLK_RESYNC_CNTL__PHYPLLD_PIXCLK_RESYNC_ENABLE__SHIFT 0x0
+#define PHYPLLD_PIXCLK_RESYNC_CNTL__PHYPLLD_DCCG_DEEP_COLOR_CNTL_MASK 0x30
+#define PHYPLLD_PIXCLK_RESYNC_CNTL__PHYPLLD_DCCG_DEEP_COLOR_CNTL__SHIFT 0x4
+#define PHYPLLD_PIXCLK_RESYNC_CNTL__PHYPLLD_PIXCLK_ENABLE_MASK 0x100
+#define PHYPLLD_PIXCLK_RESYNC_CNTL__PHYPLLD_PIXCLK_ENABLE__SHIFT 0x8
+#define PHYPLLD_PIXCLK_RESYNC_CNTL__PHYPLLD_PIXCLK_DOUBLE_RATE_ENABLE_MASK 0x200
+#define PHYPLLD_PIXCLK_RESYNC_CNTL__PHYPLLD_PIXCLK_DOUBLE_RATE_ENABLE__SHIFT 0x9
+#define PHYPLLE_PIXCLK_RESYNC_CNTL__PHYPLLE_PIXCLK_RESYNC_ENABLE_MASK 0x1
+#define PHYPLLE_PIXCLK_RESYNC_CNTL__PHYPLLE_PIXCLK_RESYNC_ENABLE__SHIFT 0x0
+#define PHYPLLE_PIXCLK_RESYNC_CNTL__PHYPLLE_DCCG_DEEP_COLOR_CNTL_MASK 0x30
+#define PHYPLLE_PIXCLK_RESYNC_CNTL__PHYPLLE_DCCG_DEEP_COLOR_CNTL__SHIFT 0x4
+#define PHYPLLE_PIXCLK_RESYNC_CNTL__PHYPLLE_PIXCLK_ENABLE_MASK 0x100
+#define PHYPLLE_PIXCLK_RESYNC_CNTL__PHYPLLE_PIXCLK_ENABLE__SHIFT 0x8
+#define PHYPLLE_PIXCLK_RESYNC_CNTL__PHYPLLE_PIXCLK_DOUBLE_RATE_ENABLE_MASK 0x200
+#define PHYPLLE_PIXCLK_RESYNC_CNTL__PHYPLLE_PIXCLK_DOUBLE_RATE_ENABLE__SHIFT 0x9
+#define PHYPLLF_PIXCLK_RESYNC_CNTL__PHYPLLF_PIXCLK_RESYNC_ENABLE_MASK 0x1
+#define PHYPLLF_PIXCLK_RESYNC_CNTL__PHYPLLF_PIXCLK_RESYNC_ENABLE__SHIFT 0x0
+#define PHYPLLF_PIXCLK_RESYNC_CNTL__PHYPLLF_DCCG_DEEP_COLOR_CNTL_MASK 0x30
+#define PHYPLLF_PIXCLK_RESYNC_CNTL__PHYPLLF_DCCG_DEEP_COLOR_CNTL__SHIFT 0x4
+#define PHYPLLF_PIXCLK_RESYNC_CNTL__PHYPLLF_PIXCLK_ENABLE_MASK 0x100
+#define PHYPLLF_PIXCLK_RESYNC_CNTL__PHYPLLF_PIXCLK_ENABLE__SHIFT 0x8
+#define PHYPLLF_PIXCLK_RESYNC_CNTL__PHYPLLF_PIXCLK_DOUBLE_RATE_ENABLE_MASK 0x200
+#define PHYPLLF_PIXCLK_RESYNC_CNTL__PHYPLLF_PIXCLK_DOUBLE_RATE_ENABLE__SHIFT 0x9
+#define MICROSECOND_TIME_BASE_DIV__MICROSECOND_TIME_BASE_DIV_MASK 0x7f
+#define MICROSECOND_TIME_BASE_DIV__MICROSECOND_TIME_BASE_DIV__SHIFT 0x0
+#define MICROSECOND_TIME_BASE_DIV__XTAL_REF_DIV_MASK 0x7f00
+#define MICROSECOND_TIME_BASE_DIV__XTAL_REF_DIV__SHIFT 0x8
+#define MICROSECOND_TIME_BASE_DIV__XTAL_REF_SEL_MASK 0x10000
+#define MICROSECOND_TIME_BASE_DIV__XTAL_REF_SEL__SHIFT 0x10
+#define MICROSECOND_TIME_BASE_DIV__XTAL_REF_CLOCK_SOURCE_SEL_MASK 0x20000
+#define MICROSECOND_TIME_BASE_DIV__XTAL_REF_CLOCK_SOURCE_SEL__SHIFT 0x11
+#define MICROSECOND_TIME_BASE_DIV__MICROSECOND_TIME_BASE_CLOCK_SOURCE_SEL_MASK 0x100000
+#define MICROSECOND_TIME_BASE_DIV__MICROSECOND_TIME_BASE_CLOCK_SOURCE_SEL__SHIFT 0x14
+#define DCCG_DISP_CNTL_REG__ALLOW_SR_ON_TRANS_REQ_MASK 0x100
+#define DCCG_DISP_CNTL_REG__ALLOW_SR_ON_TRANS_REQ__SHIFT 0x8
+#define MILLISECOND_TIME_BASE_DIV__MILLISECOND_TIME_BASE_DIV_MASK 0x1ffff
+#define MILLISECOND_TIME_BASE_DIV__MILLISECOND_TIME_BASE_DIV__SHIFT 0x0
+#define MILLISECOND_TIME_BASE_DIV__MILLISECOND_TIME_BASE_CLOCK_SOURCE_SEL_MASK 0x100000
+#define MILLISECOND_TIME_BASE_DIV__MILLISECOND_TIME_BASE_CLOCK_SOURCE_SEL__SHIFT 0x14
+#define DISPCLK_FREQ_CHANGE_CNTL__DISPCLK_STEP_DELAY_MASK 0x3fff
+#define DISPCLK_FREQ_CHANGE_CNTL__DISPCLK_STEP_DELAY__SHIFT 0x0
+#define DISPCLK_FREQ_CHANGE_CNTL__DISPCLK_STEP_SIZE_MASK 0xf0000
+#define DISPCLK_FREQ_CHANGE_CNTL__DISPCLK_STEP_SIZE__SHIFT 0x10
+#define DISPCLK_FREQ_CHANGE_CNTL__DISPCLK_FREQ_RAMP_DONE_MASK 0x100000
+#define DISPCLK_FREQ_CHANGE_CNTL__DISPCLK_FREQ_RAMP_DONE__SHIFT 0x14
+#define DISPCLK_FREQ_CHANGE_CNTL__DISPCLK_MAX_ERRDET_CYCLES_MASK 0xe000000
+#define DISPCLK_FREQ_CHANGE_CNTL__DISPCLK_MAX_ERRDET_CYCLES__SHIFT 0x19
+#define DISPCLK_FREQ_CHANGE_CNTL__DCCG_FIFO_ERRDET_RESET_MASK 0x10000000
+#define DISPCLK_FREQ_CHANGE_CNTL__DCCG_FIFO_ERRDET_RESET__SHIFT 0x1c
+#define DISPCLK_FREQ_CHANGE_CNTL__DCCG_FIFO_ERRDET_STATE_MASK 0x20000000
+#define DISPCLK_FREQ_CHANGE_CNTL__DCCG_FIFO_ERRDET_STATE__SHIFT 0x1d
+#define DISPCLK_FREQ_CHANGE_CNTL__DCCG_FIFO_ERRDET_OVR_EN_MASK 0x40000000
+#define DISPCLK_FREQ_CHANGE_CNTL__DCCG_FIFO_ERRDET_OVR_EN__SHIFT 0x1e
+#define DISPCLK_FREQ_CHANGE_CNTL__DISPCLK_CHG_FWD_CORR_DISABLE_MASK 0x80000000
+#define DISPCLK_FREQ_CHANGE_CNTL__DISPCLK_CHG_FWD_CORR_DISABLE__SHIFT 0x1f
+#define DC_MEM_GLOBAL_PWR_REQ_CNTL__DC_MEM_GLOBAL_PWR_REQ_DIS_MASK 0x1
+#define DC_MEM_GLOBAL_PWR_REQ_CNTL__DC_MEM_GLOBAL_PWR_REQ_DIS__SHIFT 0x0
+#define DCCG_PERFMON_CNTL__DCCG_PERF_DISPCLK_ENABLE_MASK 0x1
+#define DCCG_PERFMON_CNTL__DCCG_PERF_DISPCLK_ENABLE__SHIFT 0x0
+#define DCCG_PERFMON_CNTL__DCCG_PERF_DPREFCLK_ENABLE_MASK 0x2
+#define DCCG_PERFMON_CNTL__DCCG_PERF_DPREFCLK_ENABLE__SHIFT 0x1
+#define DCCG_PERFMON_CNTL__DCCG_PERF_UNIPHYA_PIXCLK_ENABLE_MASK 0x4
+#define DCCG_PERFMON_CNTL__DCCG_PERF_UNIPHYA_PIXCLK_ENABLE__SHIFT 0x2
+#define DCCG_PERFMON_CNTL__DCCG_PERF_UNIPHYB_PIXCLK_ENABLE_MASK 0x8
+#define DCCG_PERFMON_CNTL__DCCG_PERF_UNIPHYB_PIXCLK_ENABLE__SHIFT 0x3
+#define DCCG_PERFMON_CNTL__DCCG_PERF_PIXCLK0_ENABLE_MASK 0x10
+#define DCCG_PERFMON_CNTL__DCCG_PERF_PIXCLK0_ENABLE__SHIFT 0x4
+#define DCCG_PERFMON_CNTL__DCCG_PERF_RUN_MASK 0x20
+#define DCCG_PERFMON_CNTL__DCCG_PERF_RUN__SHIFT 0x5
+#define DCCG_PERFMON_CNTL__DCCG_PERF_MODE_VSYNC_MASK 0x40
+#define DCCG_PERFMON_CNTL__DCCG_PERF_MODE_VSYNC__SHIFT 0x6
+#define DCCG_PERFMON_CNTL__DCCG_PERF_MODE_HSYNC_MASK 0x80
+#define DCCG_PERFMON_CNTL__DCCG_PERF_MODE_HSYNC__SHIFT 0x7
+#define DCCG_PERFMON_CNTL__DCCG_PERF_CRTC_SEL_MASK 0x700
+#define DCCG_PERFMON_CNTL__DCCG_PERF_CRTC_SEL__SHIFT 0x8
+#define DCCG_PERFMON_CNTL__DCCG_PERF_XTALIN_PULSE_DIV_MASK 0xfffff800
+#define DCCG_PERFMON_CNTL__DCCG_PERF_XTALIN_PULSE_DIV__SHIFT 0xb
+#define DCCG_PERFMON_CNTL2__DCCG_PERF_DSICLK_ENABLE_MASK 0x1
+#define DCCG_PERFMON_CNTL2__DCCG_PERF_DSICLK_ENABLE__SHIFT 0x0
+#define DCCG_PERFMON_CNTL2__DCCG_PERF_REFCLK_ENABLE_MASK 0x2
+#define DCCG_PERFMON_CNTL2__DCCG_PERF_REFCLK_ENABLE__SHIFT 0x1
+#define DCCG_PERFMON_CNTL2__DCCG_PERF_PIXCLK1_ENABLE_MASK 0x4
+#define DCCG_PERFMON_CNTL2__DCCG_PERF_PIXCLK1_ENABLE__SHIFT 0x2
+#define DCCG_PERFMON_CNTL2__DCCG_PERF_PIXCLK2_ENABLE_MASK 0x8
+#define DCCG_PERFMON_CNTL2__DCCG_PERF_PIXCLK2_ENABLE__SHIFT 0x3
+#define DCCG_PERFMON_CNTL2__DCCG_PERF_UNIPHYC_PIXCLK_ENABLE_MASK 0x10
+#define DCCG_PERFMON_CNTL2__DCCG_PERF_UNIPHYC_PIXCLK_ENABLE__SHIFT 0x4
+#define DCCG_PERFMON_CNTL2__DCCG_PERF_UNIPHYD_PIXCLK_ENABLE_MASK 0x20
+#define DCCG_PERFMON_CNTL2__DCCG_PERF_UNIPHYD_PIXCLK_ENABLE__SHIFT 0x5
+#define DCCG_PERFMON_CNTL2__DCCG_PERF_UNIPHYE_PIXCLK_ENABLE_MASK 0x40
+#define DCCG_PERFMON_CNTL2__DCCG_PERF_UNIPHYE_PIXCLK_ENABLE__SHIFT 0x6
+#define DCCG_PERFMON_CNTL2__DCCG_PERF_UNIPHYF_PIXCLK_ENABLE_MASK 0x80
+#define DCCG_PERFMON_CNTL2__DCCG_PERF_UNIPHYF_PIXCLK_ENABLE__SHIFT 0x7
+#define DCCG_PERFMON_CNTL2__DCCG_PERF_UNIPHYG_PIXCLK_ENABLE_MASK 0x100
+#define DCCG_PERFMON_CNTL2__DCCG_PERF_UNIPHYG_PIXCLK_ENABLE__SHIFT 0x8
+#define CRTC0_PIXEL_RATE_CNTL__CRTC0_PIXEL_RATE_SOURCE_MASK 0x3
+#define CRTC0_PIXEL_RATE_CNTL__CRTC0_PIXEL_RATE_SOURCE__SHIFT 0x0
+#define CRTC0_PIXEL_RATE_CNTL__DP_DTO0_ENABLE_MASK 0x10
+#define CRTC0_PIXEL_RATE_CNTL__DP_DTO0_ENABLE__SHIFT 0x4
+#define CRTC0_PIXEL_RATE_CNTL__DP_DTO0_DS_DISABLE_MASK 0x20
+#define CRTC0_PIXEL_RATE_CNTL__DP_DTO0_DS_DISABLE__SHIFT 0x5
+#define CRTC0_PIXEL_RATE_CNTL__CRTC0_ADD_PIXEL_MASK 0x100
+#define CRTC0_PIXEL_RATE_CNTL__CRTC0_ADD_PIXEL__SHIFT 0x8
+#define CRTC0_PIXEL_RATE_CNTL__CRTC0_DROP_PIXEL_MASK 0x200
+#define CRTC0_PIXEL_RATE_CNTL__CRTC0_DROP_PIXEL__SHIFT 0x9
+#define CRTC0_PIXEL_RATE_CNTL__CRTC0_DISPOUT_HALF_RATE_EN_MASK 0x800
+#define CRTC0_PIXEL_RATE_CNTL__CRTC0_DISPOUT_HALF_RATE_EN__SHIFT 0xb
+#define CRTC0_PIXEL_RATE_CNTL__CRTC0_DISPOUT_FIFO_ERROR_MASK 0xc000
+#define CRTC0_PIXEL_RATE_CNTL__CRTC0_DISPOUT_FIFO_ERROR__SHIFT 0xe
+#define CRTC0_PIXEL_RATE_CNTL__CRTC0_DISPOUT_ERROR_COUNT_MASK 0xfff0000
+#define CRTC0_PIXEL_RATE_CNTL__CRTC0_DISPOUT_ERROR_COUNT__SHIFT 0x10
+#define DP_DTO0_PHASE__DP_DTO0_PHASE_MASK 0xffffffff
+#define DP_DTO0_PHASE__DP_DTO0_PHASE__SHIFT 0x0
+#define DP_DTO0_MODULO__DP_DTO0_MODULO_MASK 0xffffffff
+#define DP_DTO0_MODULO__DP_DTO0_MODULO__SHIFT 0x0
+#define CRTC0_PHYPLL_PIXEL_RATE_CNTL__CRTC0_PHYPLL_PIXEL_RATE_SOURCE_MASK 0x7
+#define CRTC0_PHYPLL_PIXEL_RATE_CNTL__CRTC0_PHYPLL_PIXEL_RATE_SOURCE__SHIFT 0x0
+#define CRTC0_PHYPLL_PIXEL_RATE_CNTL__CRTC0_PIXEL_RATE_PLL_SOURCE_MASK 0x10
+#define CRTC0_PHYPLL_PIXEL_RATE_CNTL__CRTC0_PIXEL_RATE_PLL_SOURCE__SHIFT 0x4
+#define CRTC1_PIXEL_RATE_CNTL__CRTC1_PIXEL_RATE_SOURCE_MASK 0x3
+#define CRTC1_PIXEL_RATE_CNTL__CRTC1_PIXEL_RATE_SOURCE__SHIFT 0x0
+#define CRTC1_PIXEL_RATE_CNTL__DP_DTO1_ENABLE_MASK 0x10
+#define CRTC1_PIXEL_RATE_CNTL__DP_DTO1_ENABLE__SHIFT 0x4
+#define CRTC1_PIXEL_RATE_CNTL__DP_DTO1_DS_DISABLE_MASK 0x20
+#define CRTC1_PIXEL_RATE_CNTL__DP_DTO1_DS_DISABLE__SHIFT 0x5
+#define CRTC1_PIXEL_RATE_CNTL__CRTC1_ADD_PIXEL_MASK 0x100
+#define CRTC1_PIXEL_RATE_CNTL__CRTC1_ADD_PIXEL__SHIFT 0x8
+#define CRTC1_PIXEL_RATE_CNTL__CRTC1_DROP_PIXEL_MASK 0x200
+#define CRTC1_PIXEL_RATE_CNTL__CRTC1_DROP_PIXEL__SHIFT 0x9
+#define CRTC1_PIXEL_RATE_CNTL__CRTC1_DISPOUT_HALF_RATE_EN_MASK 0x800
+#define CRTC1_PIXEL_RATE_CNTL__CRTC1_DISPOUT_HALF_RATE_EN__SHIFT 0xb
+#define CRTC1_PIXEL_RATE_CNTL__CRTC1_DISPOUT_FIFO_ERROR_MASK 0xc000
+#define CRTC1_PIXEL_RATE_CNTL__CRTC1_DISPOUT_FIFO_ERROR__SHIFT 0xe
+#define CRTC1_PIXEL_RATE_CNTL__CRTC1_DISPOUT_ERROR_COUNT_MASK 0xfff0000
+#define CRTC1_PIXEL_RATE_CNTL__CRTC1_DISPOUT_ERROR_COUNT__SHIFT 0x10
+#define DP_DTO1_PHASE__DP_DTO1_PHASE_MASK 0xffffffff
+#define DP_DTO1_PHASE__DP_DTO1_PHASE__SHIFT 0x0
+#define DP_DTO1_MODULO__DP_DTO1_MODULO_MASK 0xffffffff
+#define DP_DTO1_MODULO__DP_DTO1_MODULO__SHIFT 0x0
+#define CRTC1_PHYPLL_PIXEL_RATE_CNTL__CRTC1_PHYPLL_PIXEL_RATE_SOURCE_MASK 0x7
+#define CRTC1_PHYPLL_PIXEL_RATE_CNTL__CRTC1_PHYPLL_PIXEL_RATE_SOURCE__SHIFT 0x0
+#define CRTC1_PHYPLL_PIXEL_RATE_CNTL__CRTC1_PIXEL_RATE_PLL_SOURCE_MASK 0x10
+#define CRTC1_PHYPLL_PIXEL_RATE_CNTL__CRTC1_PIXEL_RATE_PLL_SOURCE__SHIFT 0x4
+#define CRTC2_PIXEL_RATE_CNTL__CRTC2_PIXEL_RATE_SOURCE_MASK 0x3
+#define CRTC2_PIXEL_RATE_CNTL__CRTC2_PIXEL_RATE_SOURCE__SHIFT 0x0
+#define CRTC2_PIXEL_RATE_CNTL__DP_DTO2_ENABLE_MASK 0x10
+#define CRTC2_PIXEL_RATE_CNTL__DP_DTO2_ENABLE__SHIFT 0x4
+#define CRTC2_PIXEL_RATE_CNTL__DP_DTO2_DS_DISABLE_MASK 0x20
+#define CRTC2_PIXEL_RATE_CNTL__DP_DTO2_DS_DISABLE__SHIFT 0x5
+#define CRTC2_PIXEL_RATE_CNTL__CRTC2_ADD_PIXEL_MASK 0x100
+#define CRTC2_PIXEL_RATE_CNTL__CRTC2_ADD_PIXEL__SHIFT 0x8
+#define CRTC2_PIXEL_RATE_CNTL__CRTC2_DROP_PIXEL_MASK 0x200
+#define CRTC2_PIXEL_RATE_CNTL__CRTC2_DROP_PIXEL__SHIFT 0x9
+#define CRTC2_PIXEL_RATE_CNTL__CRTC2_DISPOUT_HALF_RATE_EN_MASK 0x800
+#define CRTC2_PIXEL_RATE_CNTL__CRTC2_DISPOUT_HALF_RATE_EN__SHIFT 0xb
+#define CRTC2_PIXEL_RATE_CNTL__CRTC2_DISPOUT_FIFO_ERROR_MASK 0xc000
+#define CRTC2_PIXEL_RATE_CNTL__CRTC2_DISPOUT_FIFO_ERROR__SHIFT 0xe
+#define CRTC2_PIXEL_RATE_CNTL__CRTC2_DISPOUT_ERROR_COUNT_MASK 0xfff0000
+#define CRTC2_PIXEL_RATE_CNTL__CRTC2_DISPOUT_ERROR_COUNT__SHIFT 0x10
+#define DP_DTO2_PHASE__DP_DTO2_PHASE_MASK 0xffffffff
+#define DP_DTO2_PHASE__DP_DTO2_PHASE__SHIFT 0x0
+#define DP_DTO2_MODULO__DP_DTO2_MODULO_MASK 0xffffffff
+#define DP_DTO2_MODULO__DP_DTO2_MODULO__SHIFT 0x0
+#define CRTC2_PHYPLL_PIXEL_RATE_CNTL__CRTC2_PHYPLL_PIXEL_RATE_SOURCE_MASK 0x7
+#define CRTC2_PHYPLL_PIXEL_RATE_CNTL__CRTC2_PHYPLL_PIXEL_RATE_SOURCE__SHIFT 0x0
+#define CRTC2_PHYPLL_PIXEL_RATE_CNTL__CRTC2_PIXEL_RATE_PLL_SOURCE_MASK 0x10
+#define CRTC2_PHYPLL_PIXEL_RATE_CNTL__CRTC2_PIXEL_RATE_PLL_SOURCE__SHIFT 0x4
+#define CRTC3_PIXEL_RATE_CNTL__CRTC3_PIXEL_RATE_SOURCE_MASK 0x3
+#define CRTC3_PIXEL_RATE_CNTL__CRTC3_PIXEL_RATE_SOURCE__SHIFT 0x0
+#define CRTC3_PIXEL_RATE_CNTL__DP_DTO3_ENABLE_MASK 0x10
+#define CRTC3_PIXEL_RATE_CNTL__DP_DTO3_ENABLE__SHIFT 0x4
+#define CRTC3_PIXEL_RATE_CNTL__DP_DTO3_DS_DISABLE_MASK 0x20
+#define CRTC3_PIXEL_RATE_CNTL__DP_DTO3_DS_DISABLE__SHIFT 0x5
+#define CRTC3_PIXEL_RATE_CNTL__CRTC3_ADD_PIXEL_MASK 0x100
+#define CRTC3_PIXEL_RATE_CNTL__CRTC3_ADD_PIXEL__SHIFT 0x8
+#define CRTC3_PIXEL_RATE_CNTL__CRTC3_DROP_PIXEL_MASK 0x200
+#define CRTC3_PIXEL_RATE_CNTL__CRTC3_DROP_PIXEL__SHIFT 0x9
+#define CRTC3_PIXEL_RATE_CNTL__CRTC3_DISPOUT_HALF_RATE_EN_MASK 0x800
+#define CRTC3_PIXEL_RATE_CNTL__CRTC3_DISPOUT_HALF_RATE_EN__SHIFT 0xb
+#define CRTC3_PIXEL_RATE_CNTL__CRTC3_DISPOUT_FIFO_ERROR_MASK 0xc000
+#define CRTC3_PIXEL_RATE_CNTL__CRTC3_DISPOUT_FIFO_ERROR__SHIFT 0xe
+#define CRTC3_PIXEL_RATE_CNTL__CRTC3_DISPOUT_ERROR_COUNT_MASK 0xfff0000
+#define CRTC3_PIXEL_RATE_CNTL__CRTC3_DISPOUT_ERROR_COUNT__SHIFT 0x10
+#define DP_DTO3_PHASE__DP_DTO3_PHASE_MASK 0xffffffff
+#define DP_DTO3_PHASE__DP_DTO3_PHASE__SHIFT 0x0
+#define DP_DTO3_MODULO__DP_DTO3_MODULO_MASK 0xffffffff
+#define DP_DTO3_MODULO__DP_DTO3_MODULO__SHIFT 0x0
+#define CRTC3_PHYPLL_PIXEL_RATE_CNTL__CRTC3_PHYPLL_PIXEL_RATE_SOURCE_MASK 0x7
+#define CRTC3_PHYPLL_PIXEL_RATE_CNTL__CRTC3_PHYPLL_PIXEL_RATE_SOURCE__SHIFT 0x0
+#define CRTC3_PHYPLL_PIXEL_RATE_CNTL__CRTC3_PIXEL_RATE_PLL_SOURCE_MASK 0x10
+#define CRTC3_PHYPLL_PIXEL_RATE_CNTL__CRTC3_PIXEL_RATE_PLL_SOURCE__SHIFT 0x4
+#define CRTC4_PIXEL_RATE_CNTL__CRTC4_PIXEL_RATE_SOURCE_MASK 0x3
+#define CRTC4_PIXEL_RATE_CNTL__CRTC4_PIXEL_RATE_SOURCE__SHIFT 0x0
+#define CRTC4_PIXEL_RATE_CNTL__DP_DTO4_ENABLE_MASK 0x10
+#define CRTC4_PIXEL_RATE_CNTL__DP_DTO4_ENABLE__SHIFT 0x4
+#define CRTC4_PIXEL_RATE_CNTL__DP_DTO4_DS_DISABLE_MASK 0x20
+#define CRTC4_PIXEL_RATE_CNTL__DP_DTO4_DS_DISABLE__SHIFT 0x5
+#define CRTC4_PIXEL_RATE_CNTL__CRTC4_ADD_PIXEL_MASK 0x100
+#define CRTC4_PIXEL_RATE_CNTL__CRTC4_ADD_PIXEL__SHIFT 0x8
+#define CRTC4_PIXEL_RATE_CNTL__CRTC4_DROP_PIXEL_MASK 0x200
+#define CRTC4_PIXEL_RATE_CNTL__CRTC4_DROP_PIXEL__SHIFT 0x9
+#define CRTC4_PIXEL_RATE_CNTL__CRTC4_DISPOUT_HALF_RATE_EN_MASK 0x800
+#define CRTC4_PIXEL_RATE_CNTL__CRTC4_DISPOUT_HALF_RATE_EN__SHIFT 0xb
+#define CRTC4_PIXEL_RATE_CNTL__CRTC4_DISPOUT_FIFO_ERROR_MASK 0xc000
+#define CRTC4_PIXEL_RATE_CNTL__CRTC4_DISPOUT_FIFO_ERROR__SHIFT 0xe
+#define CRTC4_PIXEL_RATE_CNTL__CRTC4_DISPOUT_ERROR_COUNT_MASK 0xfff0000
+#define CRTC4_PIXEL_RATE_CNTL__CRTC4_DISPOUT_ERROR_COUNT__SHIFT 0x10
+#define DP_DTO4_PHASE__DP_DTO4_PHASE_MASK 0xffffffff
+#define DP_DTO4_PHASE__DP_DTO4_PHASE__SHIFT 0x0
+#define DP_DTO4_MODULO__DP_DTO4_MODULO_MASK 0xffffffff
+#define DP_DTO4_MODULO__DP_DTO4_MODULO__SHIFT 0x0
+#define CRTC4_PHYPLL_PIXEL_RATE_CNTL__CRTC4_PHYPLL_PIXEL_RATE_SOURCE_MASK 0x7
+#define CRTC4_PHYPLL_PIXEL_RATE_CNTL__CRTC4_PHYPLL_PIXEL_RATE_SOURCE__SHIFT 0x0
+#define CRTC4_PHYPLL_PIXEL_RATE_CNTL__CRTC4_PIXEL_RATE_PLL_SOURCE_MASK 0x10
+#define CRTC4_PHYPLL_PIXEL_RATE_CNTL__CRTC4_PIXEL_RATE_PLL_SOURCE__SHIFT 0x4
+#define CRTC5_PIXEL_RATE_CNTL__CRTC5_PIXEL_RATE_SOURCE_MASK 0x3
+#define CRTC5_PIXEL_RATE_CNTL__CRTC5_PIXEL_RATE_SOURCE__SHIFT 0x0
+#define CRTC5_PIXEL_RATE_CNTL__DP_DTO5_ENABLE_MASK 0x10
+#define CRTC5_PIXEL_RATE_CNTL__DP_DTO5_ENABLE__SHIFT 0x4
+#define CRTC5_PIXEL_RATE_CNTL__DP_DTO5_DS_DISABLE_MASK 0x20
+#define CRTC5_PIXEL_RATE_CNTL__DP_DTO5_DS_DISABLE__SHIFT 0x5
+#define CRTC5_PIXEL_RATE_CNTL__CRTC5_ADD_PIXEL_MASK 0x100
+#define CRTC5_PIXEL_RATE_CNTL__CRTC5_ADD_PIXEL__SHIFT 0x8
+#define CRTC5_PIXEL_RATE_CNTL__CRTC5_DROP_PIXEL_MASK 0x200
+#define CRTC5_PIXEL_RATE_CNTL__CRTC5_DROP_PIXEL__SHIFT 0x9
+#define CRTC5_PIXEL_RATE_CNTL__CRTC5_DISPOUT_HALF_RATE_EN_MASK 0x800
+#define CRTC5_PIXEL_RATE_CNTL__CRTC5_DISPOUT_HALF_RATE_EN__SHIFT 0xb
+#define CRTC5_PIXEL_RATE_CNTL__CRTC5_DISPOUT_FIFO_ERROR_MASK 0xc000
+#define CRTC5_PIXEL_RATE_CNTL__CRTC5_DISPOUT_FIFO_ERROR__SHIFT 0xe
+#define CRTC5_PIXEL_RATE_CNTL__CRTC5_DISPOUT_ERROR_COUNT_MASK 0xfff0000
+#define CRTC5_PIXEL_RATE_CNTL__CRTC5_DISPOUT_ERROR_COUNT__SHIFT 0x10
+#define DP_DTO5_PHASE__DP_DTO5_PHASE_MASK 0xffffffff
+#define DP_DTO5_PHASE__DP_DTO5_PHASE__SHIFT 0x0
+#define DP_DTO5_MODULO__DP_DTO5_MODULO_MASK 0xffffffff
+#define DP_DTO5_MODULO__DP_DTO5_MODULO__SHIFT 0x0
+#define CRTC5_PHYPLL_PIXEL_RATE_CNTL__CRTC5_PHYPLL_PIXEL_RATE_SOURCE_MASK 0x7
+#define CRTC5_PHYPLL_PIXEL_RATE_CNTL__CRTC5_PHYPLL_PIXEL_RATE_SOURCE__SHIFT 0x0
+#define CRTC5_PHYPLL_PIXEL_RATE_CNTL__CRTC5_PIXEL_RATE_PLL_SOURCE_MASK 0x10
+#define CRTC5_PHYPLL_PIXEL_RATE_CNTL__CRTC5_PIXEL_RATE_PLL_SOURCE__SHIFT 0x4
+#define DCCG_SOFT_RESET__REFCLK_SOFT_RESET_MASK 0x1
+#define DCCG_SOFT_RESET__REFCLK_SOFT_RESET__SHIFT 0x0
+#define DCCG_SOFT_RESET__PCIE_REFCLK_SOFT_RESET_MASK 0x2
+#define DCCG_SOFT_RESET__PCIE_REFCLK_SOFT_RESET__SHIFT 0x1
+#define DCCG_SOFT_RESET__SOFT_RESET_DVO_MASK 0x4
+#define DCCG_SOFT_RESET__SOFT_RESET_DVO__SHIFT 0x2
+#define DCCG_SOFT_RESET__DVO_ENABLE_RST_MASK 0x8
+#define DCCG_SOFT_RESET__DVO_ENABLE_RST__SHIFT 0x3
+#define DCCG_SOFT_RESET__AUDIO_DTO2_CLK_SOFT_RESET_MASK 0x10
+#define DCCG_SOFT_RESET__AUDIO_DTO2_CLK_SOFT_RESET__SHIFT 0x4
+#define DCCG_SOFT_RESET__DPREFCLK_SOFT_RESET_MASK 0x100
+#define DCCG_SOFT_RESET__DPREFCLK_SOFT_RESET__SHIFT 0x8
+#define DCCG_SOFT_RESET__AMCLK0_SOFT_RESET_MASK 0x1000
+#define DCCG_SOFT_RESET__AMCLK0_SOFT_RESET__SHIFT 0xc
+#define DCCG_SOFT_RESET__AMCLK1_SOFT_RESET_MASK 0x2000
+#define DCCG_SOFT_RESET__AMCLK1_SOFT_RESET__SHIFT 0xd
+#define DCCG_SOFT_RESET__P0PLL_CFG_IF_SOFT_RESET_MASK 0x4000
+#define DCCG_SOFT_RESET__P0PLL_CFG_IF_SOFT_RESET__SHIFT 0xe
+#define DCCG_SOFT_RESET__P1PLL_CFG_IF_SOFT_RESET_MASK 0x8000
+#define DCCG_SOFT_RESET__P1PLL_CFG_IF_SOFT_RESET__SHIFT 0xf
+#define DCCG_SOFT_RESET__P2PLL_CFG_IF_SOFT_RESET_MASK 0x10000
+#define DCCG_SOFT_RESET__P2PLL_CFG_IF_SOFT_RESET__SHIFT 0x10
+#define DCCG_SOFT_RESET__A0PLL_CFG_IF_SOFT_RESET_MASK 0x20000
+#define DCCG_SOFT_RESET__A0PLL_CFG_IF_SOFT_RESET__SHIFT 0x11
+#define DCCG_SOFT_RESET__A1PLL_CFG_IF_SOFT_RESET_MASK 0x40000
+#define DCCG_SOFT_RESET__A1PLL_CFG_IF_SOFT_RESET__SHIFT 0x12
+#define DCCG_SOFT_RESET__C0PLL_CFG_IF_SOFT_RESET_MASK 0x80000
+#define DCCG_SOFT_RESET__C0PLL_CFG_IF_SOFT_RESET__SHIFT 0x13
+#define DCCG_SOFT_RESET__C1PLL_CFG_IF_SOFT_RESET_MASK 0x100000
+#define DCCG_SOFT_RESET__C1PLL_CFG_IF_SOFT_RESET__SHIFT 0x14
+#define DCCG_SOFT_RESET__C2PLL_CFG_IF_SOFT_RESET_MASK 0x200000
+#define DCCG_SOFT_RESET__C2PLL_CFG_IF_SOFT_RESET__SHIFT 0x15
+#define SYMCLKA_CLOCK_ENABLE__SYMCLKA_CLOCK_ENABLE_MASK 0x1
+#define SYMCLKA_CLOCK_ENABLE__SYMCLKA_CLOCK_ENABLE__SHIFT 0x0
+#define SYMCLKA_CLOCK_ENABLE__SYMCLKA_FE_FORCE_EN_MASK 0x10
+#define SYMCLKA_CLOCK_ENABLE__SYMCLKA_FE_FORCE_EN__SHIFT 0x4
+#define SYMCLKA_CLOCK_ENABLE__SYMCLKA_FE_FORCE_SRC_MASK 0x700
+#define SYMCLKA_CLOCK_ENABLE__SYMCLKA_FE_FORCE_SRC__SHIFT 0x8
+#define SYMCLKB_CLOCK_ENABLE__SYMCLKB_CLOCK_ENABLE_MASK 0x1
+#define SYMCLKB_CLOCK_ENABLE__SYMCLKB_CLOCK_ENABLE__SHIFT 0x0
+#define SYMCLKB_CLOCK_ENABLE__SYMCLKB_FE_FORCE_EN_MASK 0x10
+#define SYMCLKB_CLOCK_ENABLE__SYMCLKB_FE_FORCE_EN__SHIFT 0x4
+#define SYMCLKB_CLOCK_ENABLE__SYMCLKB_FE_FORCE_SRC_MASK 0x700
+#define SYMCLKB_CLOCK_ENABLE__SYMCLKB_FE_FORCE_SRC__SHIFT 0x8
+#define SYMCLKC_CLOCK_ENABLE__SYMCLKC_CLOCK_ENABLE_MASK 0x1
+#define SYMCLKC_CLOCK_ENABLE__SYMCLKC_CLOCK_ENABLE__SHIFT 0x0
+#define SYMCLKC_CLOCK_ENABLE__SYMCLKC_FE_FORCE_EN_MASK 0x10
+#define SYMCLKC_CLOCK_ENABLE__SYMCLKC_FE_FORCE_EN__SHIFT 0x4
+#define SYMCLKC_CLOCK_ENABLE__SYMCLKC_FE_FORCE_SRC_MASK 0x700
+#define SYMCLKC_CLOCK_ENABLE__SYMCLKC_FE_FORCE_SRC__SHIFT 0x8
+#define SYMCLKD_CLOCK_ENABLE__SYMCLKD_CLOCK_ENABLE_MASK 0x1
+#define SYMCLKD_CLOCK_ENABLE__SYMCLKD_CLOCK_ENABLE__SHIFT 0x0
+#define SYMCLKD_CLOCK_ENABLE__SYMCLKD_FE_FORCE_EN_MASK 0x10
+#define SYMCLKD_CLOCK_ENABLE__SYMCLKD_FE_FORCE_EN__SHIFT 0x4
+#define SYMCLKD_CLOCK_ENABLE__SYMCLKD_FE_FORCE_SRC_MASK 0x700
+#define SYMCLKD_CLOCK_ENABLE__SYMCLKD_FE_FORCE_SRC__SHIFT 0x8
+#define SYMCLKE_CLOCK_ENABLE__SYMCLKE_CLOCK_ENABLE_MASK 0x1
+#define SYMCLKE_CLOCK_ENABLE__SYMCLKE_CLOCK_ENABLE__SHIFT 0x0
+#define SYMCLKE_CLOCK_ENABLE__SYMCLKE_FE_FORCE_EN_MASK 0x10
+#define SYMCLKE_CLOCK_ENABLE__SYMCLKE_FE_FORCE_EN__SHIFT 0x4
+#define SYMCLKE_CLOCK_ENABLE__SYMCLKE_FE_FORCE_SRC_MASK 0x700
+#define SYMCLKE_CLOCK_ENABLE__SYMCLKE_FE_FORCE_SRC__SHIFT 0x8
+#define SYMCLKF_CLOCK_ENABLE__SYMCLKF_CLOCK_ENABLE_MASK 0x1
+#define SYMCLKF_CLOCK_ENABLE__SYMCLKF_CLOCK_ENABLE__SHIFT 0x0
+#define SYMCLKF_CLOCK_ENABLE__SYMCLKF_FE_FORCE_EN_MASK 0x10
+#define SYMCLKF_CLOCK_ENABLE__SYMCLKF_FE_FORCE_EN__SHIFT 0x4
+#define SYMCLKF_CLOCK_ENABLE__SYMCLKF_FE_FORCE_SRC_MASK 0x700
+#define SYMCLKF_CLOCK_ENABLE__SYMCLKF_FE_FORCE_SRC__SHIFT 0x8
+#define DPDBG_CLK_FORCE_CONTROL__DPDBG_CLK_FORCE_EN_MASK 0x10
+#define DPDBG_CLK_FORCE_CONTROL__DPDBG_CLK_FORCE_EN__SHIFT 0x4
+#define DPDBG_CLK_FORCE_CONTROL__DPDBG_CLK_FORCE_SRC_MASK 0x700
+#define DPDBG_CLK_FORCE_CONTROL__DPDBG_CLK_FORCE_SRC__SHIFT 0x8
+#define DCCG_AUDIO_DTO_SOURCE__DCCG_AUDIO_DTO0_SOURCE_SEL_MASK 0x7
+#define DCCG_AUDIO_DTO_SOURCE__DCCG_AUDIO_DTO0_SOURCE_SEL__SHIFT 0x0
+#define DCCG_AUDIO_DTO_SOURCE__DCCG_AUDIO_DTO_SEL_MASK 0x30
+#define DCCG_AUDIO_DTO_SOURCE__DCCG_AUDIO_DTO_SEL__SHIFT 0x4
+#define DCCG_AUDIO_DTO_SOURCE__DCCG_AUDIO_DTO2_SOURCE_SEL_MASK 0x3000
+#define DCCG_AUDIO_DTO_SOURCE__DCCG_AUDIO_DTO2_SOURCE_SEL__SHIFT 0xc
+#define DCCG_AUDIO_DTO_SOURCE__DCCG_AUDIO_DTO2_CLOCK_EN_MASK 0x10000
+#define DCCG_AUDIO_DTO_SOURCE__DCCG_AUDIO_DTO2_CLOCK_EN__SHIFT 0x10
+#define DCCG_AUDIO_DTO_SOURCE__DCCG_AUDIO_DTO2_USE_512FBR_DTO_MASK 0x100000
+#define DCCG_AUDIO_DTO_SOURCE__DCCG_AUDIO_DTO2_USE_512FBR_DTO__SHIFT 0x14
+#define DCCG_AUDIO_DTO_SOURCE__DCCG_AUDIO_DTO0_USE_512FBR_DTO_MASK 0x1000000
+#define DCCG_AUDIO_DTO_SOURCE__DCCG_AUDIO_DTO0_USE_512FBR_DTO__SHIFT 0x18
+#define DCCG_AUDIO_DTO_SOURCE__DCCG_AUDIO_DTO1_USE_512FBR_DTO_MASK 0x10000000
+#define DCCG_AUDIO_DTO_SOURCE__DCCG_AUDIO_DTO1_USE_512FBR_DTO__SHIFT 0x1c
+#define DCCG_AUDIO_DTO0_PHASE__DCCG_AUDIO_DTO0_PHASE_MASK 0xffffffff
+#define DCCG_AUDIO_DTO0_PHASE__DCCG_AUDIO_DTO0_PHASE__SHIFT 0x0
+#define DCCG_AUDIO_DTO0_MODULE__DCCG_AUDIO_DTO0_MODULE_MASK 0xffffffff
+#define DCCG_AUDIO_DTO0_MODULE__DCCG_AUDIO_DTO0_MODULE__SHIFT 0x0
+#define DCCG_AUDIO_DTO1_PHASE__DCCG_AUDIO_DTO1_PHASE_MASK 0xffffffff
+#define DCCG_AUDIO_DTO1_PHASE__DCCG_AUDIO_DTO1_PHASE__SHIFT 0x0
+#define DCCG_AUDIO_DTO1_MODULE__DCCG_AUDIO_DTO1_MODULE_MASK 0xffffffff
+#define DCCG_AUDIO_DTO1_MODULE__DCCG_AUDIO_DTO1_MODULE__SHIFT 0x0
+#define DCCG_TEST_DEBUG_INDEX__DCCG_TEST_DEBUG_INDEX_MASK 0xff
+#define DCCG_TEST_DEBUG_INDEX__DCCG_TEST_DEBUG_INDEX__SHIFT 0x0
+#define DCCG_TEST_DEBUG_INDEX__DCCG_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define DCCG_TEST_DEBUG_INDEX__DCCG_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define DCCG_TEST_DEBUG_DATA__DCCG_TEST_DEBUG_DATA_MASK 0xffffffff
+#define DCCG_TEST_DEBUG_DATA__DCCG_TEST_DEBUG_DATA__SHIFT 0x0
+#define DCCG_TEST_CLK_SEL__DCCG_TEST_CLK_GENERICA_SEL_MASK 0x1ff
+#define DCCG_TEST_CLK_SEL__DCCG_TEST_CLK_GENERICA_SEL__SHIFT 0x0
+#define DCCG_TEST_CLK_SEL__DCCG_TEST_CLK_GENERICA_INV_MASK 0x1000
+#define DCCG_TEST_CLK_SEL__DCCG_TEST_CLK_GENERICA_INV__SHIFT 0xc
+#define DCCG_TEST_CLK_SEL__DCCG_TEST_CLK_GENERICB_SEL_MASK 0x1ff0000
+#define DCCG_TEST_CLK_SEL__DCCG_TEST_CLK_GENERICB_SEL__SHIFT 0x10
+#define DCCG_TEST_CLK_SEL__DCCG_TEST_CLK_GENERICB_INV_MASK 0x10000000
+#define DCCG_TEST_CLK_SEL__DCCG_TEST_CLK_GENERICB_INV__SHIFT 0x1c
+#define CPLL_MACRO_CNTL_RESERVED0__CPLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define CPLL_MACRO_CNTL_RESERVED0__CPLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define CPLL_MACRO_CNTL_RESERVED1__CPLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define CPLL_MACRO_CNTL_RESERVED1__CPLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define CPLL_MACRO_CNTL_RESERVED2__CPLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define CPLL_MACRO_CNTL_RESERVED2__CPLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define CPLL_MACRO_CNTL_RESERVED3__CPLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define CPLL_MACRO_CNTL_RESERVED3__CPLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define CPLL_MACRO_CNTL_RESERVED4__CPLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define CPLL_MACRO_CNTL_RESERVED4__CPLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define CPLL_MACRO_CNTL_RESERVED5__CPLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define CPLL_MACRO_CNTL_RESERVED5__CPLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define CPLL_MACRO_CNTL_RESERVED6__CPLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define CPLL_MACRO_CNTL_RESERVED6__CPLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define CPLL_MACRO_CNTL_RESERVED7__CPLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define CPLL_MACRO_CNTL_RESERVED7__CPLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define CPLL_MACRO_CNTL_RESERVED8__CPLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define CPLL_MACRO_CNTL_RESERVED8__CPLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define CPLL_MACRO_CNTL_RESERVED9__CPLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define CPLL_MACRO_CNTL_RESERVED9__CPLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define CPLL_MACRO_CNTL_RESERVED10__CPLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define CPLL_MACRO_CNTL_RESERVED10__CPLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define CPLL_MACRO_CNTL_RESERVED11__CPLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define CPLL_MACRO_CNTL_RESERVED11__CPLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED0__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED0__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED1__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED1__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED2__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED2__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED3__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED3__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED4__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED4__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED5__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED5__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED6__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED6__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED7__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED7__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED8__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED8__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED9__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED9__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED10__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED10__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED11__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED11__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED12__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED12__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED13__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED13__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED14__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED14__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED15__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED15__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED16__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED16__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED17__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED17__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED18__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED18__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED19__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED19__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED20__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED20__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED21__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED21__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED22__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED22__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED23__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED23__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED24__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED24__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED25__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED25__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED26__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED26__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED27__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED27__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED28__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED28__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED29__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED29__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED30__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED30__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED31__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED31__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED32__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED32__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED33__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED33__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED34__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED34__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED35__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED35__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED36__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED36__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED37__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED37__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED38__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED38__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED39__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED39__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED40__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED40__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define PLL_MACRO_CNTL_RESERVED41__PLL_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define PLL_MACRO_CNTL_RESERVED41__PLL_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DENTIST_DISPCLK_CNTL__DENTIST_DISPCLK_WDIVIDER_MASK 0x7f
+#define DENTIST_DISPCLK_CNTL__DENTIST_DISPCLK_WDIVIDER__SHIFT 0x0
+#define DENTIST_DISPCLK_CNTL__DENTIST_DISPCLK_RDIVIDER_MASK 0x7f00
+#define DENTIST_DISPCLK_CNTL__DENTIST_DISPCLK_RDIVIDER__SHIFT 0x8
+#define DENTIST_DISPCLK_CNTL__DENTIST_DISPCLK_CHG_MODE_MASK 0x18000
+#define DENTIST_DISPCLK_CNTL__DENTIST_DISPCLK_CHG_MODE__SHIFT 0xf
+#define DENTIST_DISPCLK_CNTL__DENTIST_DISPCLK_CHGTOG_MASK 0x20000
+#define DENTIST_DISPCLK_CNTL__DENTIST_DISPCLK_CHGTOG__SHIFT 0x11
+#define DENTIST_DISPCLK_CNTL__DENTIST_DISPCLK_DONETOG_MASK 0x40000
+#define DENTIST_DISPCLK_CNTL__DENTIST_DISPCLK_DONETOG__SHIFT 0x12
+#define DENTIST_DISPCLK_CNTL__DENTIST_DISPCLK_CHG_DONE_MASK 0x80000
+#define DENTIST_DISPCLK_CNTL__DENTIST_DISPCLK_CHG_DONE__SHIFT 0x13
+#define DENTIST_DISPCLK_CNTL__DENTIST_DPREFCLK_CHG_DONE_MASK 0x100000
+#define DENTIST_DISPCLK_CNTL__DENTIST_DPREFCLK_CHG_DONE__SHIFT 0x14
+#define DENTIST_DISPCLK_CNTL__DENTIST_DPREFCLK_CHGTOG_MASK 0x200000
+#define DENTIST_DISPCLK_CNTL__DENTIST_DPREFCLK_CHGTOG__SHIFT 0x15
+#define DENTIST_DISPCLK_CNTL__DENTIST_DPREFCLK_DONETOG_MASK 0x400000
+#define DENTIST_DISPCLK_CNTL__DENTIST_DPREFCLK_DONETOG__SHIFT 0x16
+#define DENTIST_DISPCLK_CNTL__DENTIST_DPREFCLK_WDIVIDER_MASK 0x7f000000
+#define DENTIST_DISPCLK_CNTL__DENTIST_DPREFCLK_WDIVIDER__SHIFT 0x18
+#define DCDEBUG_BUS_CLK1_SEL__DCDEBUG_BUS_CLK1_SEL_MASK 0xffffffff
+#define DCDEBUG_BUS_CLK1_SEL__DCDEBUG_BUS_CLK1_SEL__SHIFT 0x0
+#define DCDEBUG_BUS_CLK2_SEL__DCDEBUG_BUS_CLK2_SEL_MASK 0xffffffff
+#define DCDEBUG_BUS_CLK2_SEL__DCDEBUG_BUS_CLK2_SEL__SHIFT 0x0
+#define DCDEBUG_BUS_CLK3_SEL__DCDEBUG_BUS_CLK3_SEL_MASK 0xffffffff
+#define DCDEBUG_BUS_CLK3_SEL__DCDEBUG_BUS_CLK3_SEL__SHIFT 0x0
+#define DCDEBUG_BUS_CLK4_SEL__DCDEBUG_BUS_CLK4_SEL_MASK 0xffffffff
+#define DCDEBUG_BUS_CLK4_SEL__DCDEBUG_BUS_CLK4_SEL__SHIFT 0x0
+#define DCDEBUG_BUS_CLK5_SEL__DCDEBUG_BUS_CLK5_SEL_MASK 0xffffffff
+#define DCDEBUG_BUS_CLK5_SEL__DCDEBUG_BUS_CLK5_SEL__SHIFT 0x0
+#define DCDEBUG_OUT_PIN_OVERRIDE__DCDEBUG_OUT_OVERRIDE1_PIN_SEL_MASK 0x1f
+#define DCDEBUG_OUT_PIN_OVERRIDE__DCDEBUG_OUT_OVERRIDE1_PIN_SEL__SHIFT 0x0
+#define DCDEBUG_OUT_PIN_OVERRIDE__DCDEBUG_OUT_OVERRIDE1_REGBIT_SEL_MASK 0x3e0
+#define DCDEBUG_OUT_PIN_OVERRIDE__DCDEBUG_OUT_OVERRIDE1_REGBIT_SEL__SHIFT 0x5
+#define DCDEBUG_OUT_PIN_OVERRIDE__DCDEBUG_OUT_OVERRIDE1_EN_MASK 0x1000
+#define DCDEBUG_OUT_PIN_OVERRIDE__DCDEBUG_OUT_OVERRIDE1_EN__SHIFT 0xc
+#define DCDEBUG_OUT_PIN_OVERRIDE__DCDEBUG_OUT_OVERRIDE2_PIN_SEL_MASK 0xf8000
+#define DCDEBUG_OUT_PIN_OVERRIDE__DCDEBUG_OUT_OVERRIDE2_PIN_SEL__SHIFT 0xf
+#define DCDEBUG_OUT_PIN_OVERRIDE__DCDEBUG_OUT_OVERRIDE2_REGBIT_SEL_MASK 0x1f00000
+#define DCDEBUG_OUT_PIN_OVERRIDE__DCDEBUG_OUT_OVERRIDE2_REGBIT_SEL__SHIFT 0x14
+#define DCDEBUG_OUT_PIN_OVERRIDE__DCDEBUG_OUT_OVERRIDE2_EN_MASK 0x10000000
+#define DCDEBUG_OUT_PIN_OVERRIDE__DCDEBUG_OUT_OVERRIDE2_EN__SHIFT 0x1c
+#define DCDEBUG_OUT_CNTL__DCDEBUG_BLOCK_SEL_MASK 0x1f
+#define DCDEBUG_OUT_CNTL__DCDEBUG_BLOCK_SEL__SHIFT 0x0
+#define DCDEBUG_OUT_CNTL__DCDEBUG_OUT_24BIT_SEL_MASK 0x800000
+#define DCDEBUG_OUT_CNTL__DCDEBUG_OUT_24BIT_SEL__SHIFT 0x17
+#define DCDEBUG_OUT_CNTL__DCDEBUG_CLK_SEL_MASK 0x1f000000
+#define DCDEBUG_OUT_CNTL__DCDEBUG_CLK_SEL__SHIFT 0x18
+#define DCDEBUG_OUT_DATA__DCDEBUG_OUT_DATA_MASK 0xffffffff
+#define DCDEBUG_OUT_DATA__DCDEBUG_OUT_DATA__SHIFT 0x0
+#define DMIF_CONTROL__DMIF_BUFF_SIZE_MASK 0x3
+#define DMIF_CONTROL__DMIF_BUFF_SIZE__SHIFT 0x0
+#define DMIF_CONTROL__DMIF_GROUP_REQUESTS_IN_CHUNK_MASK 0x4
+#define DMIF_CONTROL__DMIF_GROUP_REQUESTS_IN_CHUNK__SHIFT 0x2
+#define DMIF_CONTROL__DMIF_DISABLE_EARLY_RECEIVED_LEVEL_COUNT_MASK 0x10
+#define DMIF_CONTROL__DMIF_DISABLE_EARLY_RECEIVED_LEVEL_COUNT__SHIFT 0x4
+#define DMIF_CONTROL__DMIF_REQ_BURST_SIZE_MASK 0x700
+#define DMIF_CONTROL__DMIF_REQ_BURST_SIZE__SHIFT 0x8
+#define DMIF_CONTROL__DMIF_UNDERFLOW_RECOVERY_EN_MASK 0x800
+#define DMIF_CONTROL__DMIF_UNDERFLOW_RECOVERY_EN__SHIFT 0xb
+#define DMIF_CONTROL__DMIF_FORCE_TOTAL_REQ_BURST_SIZE_MASK 0x1f000
+#define DMIF_CONTROL__DMIF_FORCE_TOTAL_REQ_BURST_SIZE__SHIFT 0xc
+#define DMIF_CONTROL__DMIF_MAX_TOTAL_OUTSTANDING_CHUNK_REQUESTS_MASK 0x7e0000
+#define DMIF_CONTROL__DMIF_MAX_TOTAL_OUTSTANDING_CHUNK_REQUESTS__SHIFT 0x11
+#define DMIF_CONTROL__DMIF_DELAY_ARBITRATION_MASK 0x1f000000
+#define DMIF_CONTROL__DMIF_DELAY_ARBITRATION__SHIFT 0x18
+#define DMIF_CONTROL__DMIF_CHUNK_BUFF_MARGIN_MASK 0x60000000
+#define DMIF_CONTROL__DMIF_CHUNK_BUFF_MARGIN__SHIFT 0x1d
+#define DMIF_CONTROL__DMIF_PSTATE_URGENT_DISABLE_MASK 0x80000000
+#define DMIF_CONTROL__DMIF_PSTATE_URGENT_DISABLE__SHIFT 0x1f
+#define DMIF_STATUS__DMIF_MC_SEND_ON_IDLE_MASK 0x3f
+#define DMIF_STATUS__DMIF_MC_SEND_ON_IDLE__SHIFT 0x0
+#define DMIF_STATUS__DMIF_CLEAR_MC_SEND_ON_IDLE_MASK 0x3f00
+#define DMIF_STATUS__DMIF_CLEAR_MC_SEND_ON_IDLE__SHIFT 0x8
+#define DMIF_STATUS__DMIF_MC_LATENCY_COUNTER_ENABLE_MASK 0x10000
+#define DMIF_STATUS__DMIF_MC_LATENCY_COUNTER_ENABLE__SHIFT 0x10
+#define DMIF_STATUS__DMIF_MC_LATENCY_COUNTER_URGENT_ONLY_MASK 0x20000
+#define DMIF_STATUS__DMIF_MC_LATENCY_COUNTER_URGENT_ONLY__SHIFT 0x11
+#define DMIF_STATUS__DMIF_MC_LATENCY_COUNTER_SOURCE_SELECT_MASK 0xf00000
+#define DMIF_STATUS__DMIF_MC_LATENCY_COUNTER_SOURCE_SELECT__SHIFT 0x14
+#define DMIF_STATUS__DMIF_PERFORMANCE_COUNTER_SOURCE_SELECT_MASK 0xf000000
+#define DMIF_STATUS__DMIF_PERFORMANCE_COUNTER_SOURCE_SELECT__SHIFT 0x18
+#define DMIF_STATUS__DMIF_UNDERFLOW_MASK 0x10000000
+#define DMIF_STATUS__DMIF_UNDERFLOW__SHIFT 0x1c
+#define DMIF_STATUS__DMIF_MC_LATENCY_TAP_POINT_MASK 0x60000000
+#define DMIF_STATUS__DMIF_MC_LATENCY_TAP_POINT__SHIFT 0x1d
+#define DMIF_STATUS__DMIF_MC_LATENCY_REQ_TYPE_MASK 0x80000000
+#define DMIF_STATUS__DMIF_MC_LATENCY_REQ_TYPE__SHIFT 0x1f
+#define DMIFV_STATUS__DMIFV_MC_SEND_ON_IDLE_MASK 0xf
+#define DMIFV_STATUS__DMIFV_MC_SEND_ON_IDLE__SHIFT 0x0
+#define DMIFV_STATUS__DMIFV_CLEAR_MC_SEND_ON_IDLE_MASK 0xf00
+#define DMIFV_STATUS__DMIFV_CLEAR_MC_SEND_ON_IDLE__SHIFT 0x8
+#define DMIF_HW_DEBUG__DMIF_HW_DEBUG_MASK 0xffffffff
+#define DMIF_HW_DEBUG__DMIF_HW_DEBUG__SHIFT 0x0
+#define DMIF_ARBITRATION_CONTROL__DMIF_ARBITRATION_REFERENCE_CLOCK_PERIOD_MASK 0xffff
+#define DMIF_ARBITRATION_CONTROL__DMIF_ARBITRATION_REFERENCE_CLOCK_PERIOD__SHIFT 0x0
+#define DMIF_ARBITRATION_CONTROL__PIPE_SWITCH_EFFICIENCY_WEIGHT_MASK 0xffff0000
+#define DMIF_ARBITRATION_CONTROL__PIPE_SWITCH_EFFICIENCY_WEIGHT__SHIFT 0x10
+#define PIPE0_ARBITRATION_CONTROL3__EFFICIENCY_WEIGHT_MASK 0xffff
+#define PIPE0_ARBITRATION_CONTROL3__EFFICIENCY_WEIGHT__SHIFT 0x0
+#define PIPE1_ARBITRATION_CONTROL3__EFFICIENCY_WEIGHT_MASK 0xffff
+#define PIPE1_ARBITRATION_CONTROL3__EFFICIENCY_WEIGHT__SHIFT 0x0
+#define PIPE2_ARBITRATION_CONTROL3__EFFICIENCY_WEIGHT_MASK 0xffff
+#define PIPE2_ARBITRATION_CONTROL3__EFFICIENCY_WEIGHT__SHIFT 0x0
+#define PIPE3_ARBITRATION_CONTROL3__EFFICIENCY_WEIGHT_MASK 0xffff
+#define PIPE3_ARBITRATION_CONTROL3__EFFICIENCY_WEIGHT__SHIFT 0x0
+#define PIPE4_ARBITRATION_CONTROL3__EFFICIENCY_WEIGHT_MASK 0xffff
+#define PIPE4_ARBITRATION_CONTROL3__EFFICIENCY_WEIGHT__SHIFT 0x0
+#define PIPE5_ARBITRATION_CONTROL3__EFFICIENCY_WEIGHT_MASK 0xffff
+#define PIPE5_ARBITRATION_CONTROL3__EFFICIENCY_WEIGHT__SHIFT 0x0
+#define PIPE6_ARBITRATION_CONTROL3__EFFICIENCY_WEIGHT_MASK 0xffff
+#define PIPE6_ARBITRATION_CONTROL3__EFFICIENCY_WEIGHT__SHIFT 0x0
+#define PIPE7_ARBITRATION_CONTROL3__EFFICIENCY_WEIGHT_MASK 0xffff
+#define PIPE7_ARBITRATION_CONTROL3__EFFICIENCY_WEIGHT__SHIFT 0x0
+#define DMIF_P_VMID__P_VMID_PIPE0_MASK 0xf
+#define DMIF_P_VMID__P_VMID_PIPE0__SHIFT 0x0
+#define DMIF_P_VMID__P_VMID_PIPE1_MASK 0xf0
+#define DMIF_P_VMID__P_VMID_PIPE1__SHIFT 0x4
+#define DMIF_P_VMID__P_VMID_PIPE2_MASK 0xf00
+#define DMIF_P_VMID__P_VMID_PIPE2__SHIFT 0x8
+#define DMIF_P_VMID__P_VMID_PIPE3_MASK 0xf000
+#define DMIF_P_VMID__P_VMID_PIPE3__SHIFT 0xc
+#define DMIF_P_VMID__P_VMID_PIPE4_MASK 0xf0000
+#define DMIF_P_VMID__P_VMID_PIPE4__SHIFT 0x10
+#define DMIF_P_VMID__P_VMID_PIPE5_MASK 0xf00000
+#define DMIF_P_VMID__P_VMID_PIPE5__SHIFT 0x14
+#define DMIF_P_VMID__P_VMID_PIPE6_MASK 0xf000000
+#define DMIF_P_VMID__P_VMID_PIPE6__SHIFT 0x18
+#define DMIF_P_VMID__P_VMID_PIPE7_MASK 0xf0000000
+#define DMIF_P_VMID__P_VMID_PIPE7__SHIFT 0x1c
+#define DMIF_URG_OVERRIDE__DMIF_URG_OVERRIDE_EN_MASK 0x1
+#define DMIF_URG_OVERRIDE__DMIF_URG_OVERRIDE_EN__SHIFT 0x0
+#define DMIF_URG_OVERRIDE__DMIF_URG_OVERRIDE_LEVEL_MASK 0xf0
+#define DMIF_URG_OVERRIDE__DMIF_URG_OVERRIDE_LEVEL__SHIFT 0x4
+#define DMIF_TEST_DEBUG_INDEX__DMIF_TEST_DEBUG_INDEX_MASK 0xff
+#define DMIF_TEST_DEBUG_INDEX__DMIF_TEST_DEBUG_INDEX__SHIFT 0x0
+#define DMIF_TEST_DEBUG_INDEX__DMIF_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define DMIF_TEST_DEBUG_INDEX__DMIF_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define DMIF_TEST_DEBUG_DATA__DMIF_TEST_DEBUG_DATA_MASK 0xffffffff
+#define DMIF_TEST_DEBUG_DATA__DMIF_TEST_DEBUG_DATA__SHIFT 0x0
+#define DMIF_DEBUG02_CORE0__DB_DATA_MASK 0xffff
+#define DMIF_DEBUG02_CORE0__DB_DATA__SHIFT 0x0
+#define DMIF_DEBUG02_CORE0__MC_RDRET_COUNT_EN_MASK 0x10000
+#define DMIF_DEBUG02_CORE0__MC_RDRET_COUNT_EN__SHIFT 0x10
+#define DMIF_DEBUG02_CORE0__MC_RDRET_COUNTER_MASK 0xffe0000
+#define DMIF_DEBUG02_CORE0__MC_RDRET_COUNTER__SHIFT 0x11
+#define DMIF_DEBUG02_CORE1__DB_DATA_MASK 0xffff
+#define DMIF_DEBUG02_CORE1__DB_DATA__SHIFT 0x0
+#define DMIF_DEBUG02_CORE1__MC_RDRET_COUNT_EN_MASK 0x10000
+#define DMIF_DEBUG02_CORE1__MC_RDRET_COUNT_EN__SHIFT 0x10
+#define DMIF_DEBUG02_CORE1__MC_RDRET_COUNTER_MASK 0xffe0000
+#define DMIF_DEBUG02_CORE1__MC_RDRET_COUNTER__SHIFT 0x11
+#define DMIF_ADDR_CALC__ADDR_CONFIG_PIPE_INTERLEAVE_SIZE_MASK 0x70
+#define DMIF_ADDR_CALC__ADDR_CONFIG_PIPE_INTERLEAVE_SIZE__SHIFT 0x4
+#define DMIF_ADDR_CALC__ADDR_CONFIG_ROW_SIZE_MASK 0x30000000
+#define DMIF_ADDR_CALC__ADDR_CONFIG_ROW_SIZE__SHIFT 0x1c
+#define DMIF_STATUS2__DMIF_PIPE0_DISPCLK_STATUS_MASK 0x1
+#define DMIF_STATUS2__DMIF_PIPE0_DISPCLK_STATUS__SHIFT 0x0
+#define DMIF_STATUS2__DMIF_PIPE1_DISPCLK_STATUS_MASK 0x2
+#define DMIF_STATUS2__DMIF_PIPE1_DISPCLK_STATUS__SHIFT 0x1
+#define DMIF_STATUS2__DMIF_PIPE2_DISPCLK_STATUS_MASK 0x4
+#define DMIF_STATUS2__DMIF_PIPE2_DISPCLK_STATUS__SHIFT 0x2
+#define DMIF_STATUS2__DMIF_PIPE3_DISPCLK_STATUS_MASK 0x8
+#define DMIF_STATUS2__DMIF_PIPE3_DISPCLK_STATUS__SHIFT 0x3
+#define DMIF_STATUS2__DMIF_PIPE4_DISPCLK_STATUS_MASK 0x10
+#define DMIF_STATUS2__DMIF_PIPE4_DISPCLK_STATUS__SHIFT 0x4
+#define DMIF_STATUS2__DMIF_PIPE5_DISPCLK_STATUS_MASK 0x20
+#define DMIF_STATUS2__DMIF_PIPE5_DISPCLK_STATUS__SHIFT 0x5
+#define DMIF_STATUS2__DMIF_CHUNK_TRACKER_SCLK_STATUS_MASK 0x100
+#define DMIF_STATUS2__DMIF_CHUNK_TRACKER_SCLK_STATUS__SHIFT 0x8
+#define DMIF_STATUS2__DMIF_FBC_TRACKER_SCLK_STATUS_MASK 0x200
+#define DMIF_STATUS2__DMIF_FBC_TRACKER_SCLK_STATUS__SHIFT 0x9
+#define PIPE0_MAX_REQUESTS__MAX_REQUESTS_MASK 0x3ff
+#define PIPE0_MAX_REQUESTS__MAX_REQUESTS__SHIFT 0x0
+#define PIPE1_MAX_REQUESTS__MAX_REQUESTS_MASK 0x3ff
+#define PIPE1_MAX_REQUESTS__MAX_REQUESTS__SHIFT 0x0
+#define PIPE2_MAX_REQUESTS__MAX_REQUESTS_MASK 0x3ff
+#define PIPE2_MAX_REQUESTS__MAX_REQUESTS__SHIFT 0x0
+#define PIPE3_MAX_REQUESTS__MAX_REQUESTS_MASK 0x3ff
+#define PIPE3_MAX_REQUESTS__MAX_REQUESTS__SHIFT 0x0
+#define PIPE4_MAX_REQUESTS__MAX_REQUESTS_MASK 0x3ff
+#define PIPE4_MAX_REQUESTS__MAX_REQUESTS__SHIFT 0x0
+#define PIPE5_MAX_REQUESTS__MAX_REQUESTS_MASK 0x3ff
+#define PIPE5_MAX_REQUESTS__MAX_REQUESTS__SHIFT 0x0
+#define PIPE6_MAX_REQUESTS__MAX_REQUESTS_MASK 0x3ff
+#define PIPE6_MAX_REQUESTS__MAX_REQUESTS__SHIFT 0x0
+#define PIPE7_MAX_REQUESTS__MAX_REQUESTS_MASK 0x3ff
+#define PIPE7_MAX_REQUESTS__MAX_REQUESTS__SHIFT 0x0
+#define DVMM_REG_RD_STATUS__DVMM_REG_RD_STATUS_MASK 0x1
+#define DVMM_REG_RD_STATUS__DVMM_REG_RD_STATUS__SHIFT 0x0
+#define DVMM_REG_RD_DATA__DVMM_REG_RD_DATA_MASK 0xffffffff
+#define DVMM_REG_RD_DATA__DVMM_REG_RD_DATA__SHIFT 0x0
+#define DVMM_PTE_REQ__MAX_PTEREQ_TO_ISSUE_MASK 0xff
+#define DVMM_PTE_REQ__MAX_PTEREQ_TO_ISSUE__SHIFT 0x0
+#define DVMM_PTE_REQ__HFLIP_PTEREQ_PER_CHUNK_INT_MASK 0xff00
+#define DVMM_PTE_REQ__HFLIP_PTEREQ_PER_CHUNK_INT__SHIFT 0x8
+#define DVMM_PTE_REQ__HFLIP_PTEREQ_PER_CHUNK_MULTIPLIER_MASK 0x3f0000
+#define DVMM_PTE_REQ__HFLIP_PTEREQ_PER_CHUNK_MULTIPLIER__SHIFT 0x10
+#define DVMM_CNTL__PDE_CACHE_INVALIDATE_CNTL_MASK 0x3
+#define DVMM_CNTL__PDE_CACHE_INVALIDATE_CNTL__SHIFT 0x0
+#define DVMM_CNTL__DEBUG_SYSTEM_ACCESS_MODE_MASK 0x30
+#define DVMM_CNTL__DEBUG_SYSTEM_ACCESS_MODE__SHIFT 0x4
+#define DVMM_CNTL__FORCE_SYSTEM_ACCESS_MODE_MASK 0x80
+#define DVMM_CNTL__FORCE_SYSTEM_ACCESS_MODE__SHIFT 0x7
+#define DVMM_CNTL__DBG_DCE_VMID_MASK 0xf00
+#define DVMM_CNTL__DBG_DCE_VMID__SHIFT 0x8
+#define DVMM_CNTL__FORCE_DBG_DCE_VMID_MASK 0x8000
+#define DVMM_CNTL__FORCE_DBG_DCE_VMID__SHIFT 0xf
+#define DVMM_CNTL__OVERRIDE_SNOOP_MASK 0x20000
+#define DVMM_CNTL__OVERRIDE_SNOOP__SHIFT 0x11
+#define DVMM_CNTL__ENABLE_PDE_INVALIDATE_MASK 0x40000
+#define DVMM_CNTL__ENABLE_PDE_INVALIDATE__SHIFT 0x12
+#define DVMM_FAULT_STATUS__DVMM_FAULT_STATUS_MASK 0xffffffff
+#define DVMM_FAULT_STATUS__DVMM_FAULT_STATUS__SHIFT 0x0
+#define DVMM_FAULT_ADDR__DVMM_FAULT_ADDR_MASK 0xffffffff
+#define DVMM_FAULT_ADDR__DVMM_FAULT_ADDR__SHIFT 0x0
+#define LOW_POWER_TILING_CONTROL__LOW_POWER_TILING_ENABLE_MASK 0x1
+#define LOW_POWER_TILING_CONTROL__LOW_POWER_TILING_ENABLE__SHIFT 0x0
+#define LOW_POWER_TILING_CONTROL__LOW_POWER_TILING_MODE_MASK 0x18
+#define LOW_POWER_TILING_CONTROL__LOW_POWER_TILING_MODE__SHIFT 0x3
+#define LOW_POWER_TILING_CONTROL__LOW_POWER_TILING_NUM_PIPES_MASK 0xe0
+#define LOW_POWER_TILING_CONTROL__LOW_POWER_TILING_NUM_PIPES__SHIFT 0x5
+#define LOW_POWER_TILING_CONTROL__LOW_POWER_TILING_NUM_BANKS_MASK 0x700
+#define LOW_POWER_TILING_CONTROL__LOW_POWER_TILING_NUM_BANKS__SHIFT 0x8
+#define LOW_POWER_TILING_CONTROL__LOW_POWER_TILING_PIPE_INTERLEAVE_SIZE_MASK 0x800
+#define LOW_POWER_TILING_CONTROL__LOW_POWER_TILING_PIPE_INTERLEAVE_SIZE__SHIFT 0xb
+#define LOW_POWER_TILING_CONTROL__LOW_POWER_TILING_ROW_SIZE_MASK 0x7000
+#define LOW_POWER_TILING_CONTROL__LOW_POWER_TILING_ROW_SIZE__SHIFT 0xc
+#define LOW_POWER_TILING_CONTROL__LOW_POWER_TILING_ROWS_PER_CHAN_MASK 0xfff0000
+#define LOW_POWER_TILING_CONTROL__LOW_POWER_TILING_ROWS_PER_CHAN__SHIFT 0x10
+#define MCIF_CONTROL__MCIF_BUFF_SIZE_MASK 0x3
+#define MCIF_CONTROL__MCIF_BUFF_SIZE__SHIFT 0x0
+#define MCIF_CONTROL__ADDRESS_TRANSLATION_ENABLE_MASK 0x10
+#define MCIF_CONTROL__ADDRESS_TRANSLATION_ENABLE__SHIFT 0x4
+#define MCIF_CONTROL__PRIVILEGED_ACCESS_ENABLE_MASK 0x100
+#define MCIF_CONTROL__PRIVILEGED_ACCESS_ENABLE__SHIFT 0x8
+#define MCIF_CONTROL__MCIF_SLOW_REQ_INTERVAL_MASK 0xf000
+#define MCIF_CONTROL__MCIF_SLOW_REQ_INTERVAL__SHIFT 0xc
+#define MCIF_CONTROL__LOW_READ_URG_LEVEL_MASK 0xff0000
+#define MCIF_CONTROL__LOW_READ_URG_LEVEL__SHIFT 0x10
+#define MCIF_CONTROL__MC_CLEAN_DEASSERT_LATENCY_MASK 0x3f000000
+#define MCIF_CONTROL__MC_CLEAN_DEASSERT_LATENCY__SHIFT 0x18
+#define MCIF_CONTROL__MCIF_MC_LATENCY_COUNTER_ENABLE_MASK 0x40000000
+#define MCIF_CONTROL__MCIF_MC_LATENCY_COUNTER_ENABLE__SHIFT 0x1e
+#define MCIF_CONTROL__MCIF_MC_LATENCY_COUNTER_URGENT_ONLY_MASK 0x80000000
+#define MCIF_CONTROL__MCIF_MC_LATENCY_COUNTER_URGENT_ONLY__SHIFT 0x1f
+#define MCIF_WRITE_COMBINE_CONTROL__MCIF_WRITE_COMBINE_TIMEOUT_MASK 0xff
+#define MCIF_WRITE_COMBINE_CONTROL__MCIF_WRITE_COMBINE_TIMEOUT__SHIFT 0x0
+#define MCIF_WRITE_COMBINE_CONTROL__VIP_WRITE_COMBINE_TIMEOUT_MASK 0xff00
+#define MCIF_WRITE_COMBINE_CONTROL__VIP_WRITE_COMBINE_TIMEOUT__SHIFT 0x8
+#define MCIF_TEST_DEBUG_INDEX__MCIF_TEST_DEBUG_INDEX_MASK 0xff
+#define MCIF_TEST_DEBUG_INDEX__MCIF_TEST_DEBUG_INDEX__SHIFT 0x0
+#define MCIF_TEST_DEBUG_INDEX__MCIF_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define MCIF_TEST_DEBUG_INDEX__MCIF_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define MCIF_TEST_DEBUG_DATA__MCIF_TEST_DEBUG_DATA_MASK 0xffffffff
+#define MCIF_TEST_DEBUG_DATA__MCIF_TEST_DEBUG_DATA__SHIFT 0x0
+#define IDDCCIF02_DBG_DCCIF_C__DBG_DCCIF_C_MASK 0xffffffff
+#define IDDCCIF02_DBG_DCCIF_C__DBG_DCCIF_C__SHIFT 0x0
+#define IDDCCIF04_DBG_DCCIF_E__DBG_DCCIF_E_MASK 0xffffffff
+#define IDDCCIF04_DBG_DCCIF_E__DBG_DCCIF_E__SHIFT 0x0
+#define IDDCCIF05_DBG_DCCIF_F__DBG_DCCIF_F_MASK 0xffffffff
+#define IDDCCIF05_DBG_DCCIF_F__DBG_DCCIF_F__SHIFT 0x0
+#define MCIF_VMID__MCIF_WR_VMID_MASK 0xf
+#define MCIF_VMID__MCIF_WR_VMID__SHIFT 0x0
+#define MCIF_VMID__VIP_WR_VMID_MASK 0xf0
+#define MCIF_VMID__VIP_WR_VMID__SHIFT 0x4
+#define MCIF_MEM_CONTROL__MCIFMEM_CACHE_MODE_DIS_MASK 0x1
+#define MCIF_MEM_CONTROL__MCIFMEM_CACHE_MODE_DIS__SHIFT 0x0
+#define MCIF_MEM_CONTROL__MCIFMEM_CACHE_MODE_MASK 0x30
+#define MCIF_MEM_CONTROL__MCIFMEM_CACHE_MODE__SHIFT 0x4
+#define MCIF_MEM_CONTROL__MCIFMEM_CACHE_SIZE_MASK 0xff00
+#define MCIF_MEM_CONTROL__MCIFMEM_CACHE_SIZE__SHIFT 0x8
+#define MCIF_MEM_CONTROL__MCIFMEM_CACHE_PIPE_MASK 0x70000
+#define MCIF_MEM_CONTROL__MCIFMEM_CACHE_PIPE__SHIFT 0x10
+#define MCIF_MEM_CONTROL__MCIFMEM_CACHE_TYPE_MASK 0x180000
+#define MCIF_MEM_CONTROL__MCIFMEM_CACHE_TYPE__SHIFT 0x13
+#define CC_DC_PIPE_DIS__DC_PIPE_DIS_MASK 0x7e
+#define CC_DC_PIPE_DIS__DC_PIPE_DIS__SHIFT 0x1
+#define CC_DC_PIPE_DIS__DC_UNDERLAY_PIPE_DIS_MASK 0x3f0000
+#define CC_DC_PIPE_DIS__DC_UNDERLAY_PIPE_DIS__SHIFT 0x10
+#define MC_DC_INTERFACE_NACK_STATUS__DMIF_RDRET_NACK_OCCURRED_MASK 0x1
+#define MC_DC_INTERFACE_NACK_STATUS__DMIF_RDRET_NACK_OCCURRED__SHIFT 0x0
+#define MC_DC_INTERFACE_NACK_STATUS__DMIF_RDRET_NACK_CLEAR_MASK 0x10
+#define MC_DC_INTERFACE_NACK_STATUS__DMIF_RDRET_NACK_CLEAR__SHIFT 0x4
+#define MC_DC_INTERFACE_NACK_STATUS__VIP_WRRET_NACK_OCCURRED_MASK 0x100
+#define MC_DC_INTERFACE_NACK_STATUS__VIP_WRRET_NACK_OCCURRED__SHIFT 0x8
+#define MC_DC_INTERFACE_NACK_STATUS__VIP_WRRET_NACK_CLEAR_MASK 0x1000
+#define MC_DC_INTERFACE_NACK_STATUS__VIP_WRRET_NACK_CLEAR__SHIFT 0xc
+#define MC_DC_INTERFACE_NACK_STATUS__MCIF_RDRET_NACK_OCCURRED_MASK 0x10000
+#define MC_DC_INTERFACE_NACK_STATUS__MCIF_RDRET_NACK_OCCURRED__SHIFT 0x10
+#define MC_DC_INTERFACE_NACK_STATUS__MCIF_RDRET_NACK_CLEAR_MASK 0x100000
+#define MC_DC_INTERFACE_NACK_STATUS__MCIF_RDRET_NACK_CLEAR__SHIFT 0x14
+#define MC_DC_INTERFACE_NACK_STATUS__MCIF_WRRET_NACK_OCCURRED_MASK 0x1000000
+#define MC_DC_INTERFACE_NACK_STATUS__MCIF_WRRET_NACK_OCCURRED__SHIFT 0x18
+#define MC_DC_INTERFACE_NACK_STATUS__MCIF_WRRET_NACK_CLEAR_MASK 0x10000000
+#define MC_DC_INTERFACE_NACK_STATUS__MCIF_WRRET_NACK_CLEAR__SHIFT 0x1c
+#define RBBMIF_TIMEOUT__RBBMIF_TIMEOUT_DELAY_MASK 0xfffff
+#define RBBMIF_TIMEOUT__RBBMIF_TIMEOUT_DELAY__SHIFT 0x0
+#define RBBMIF_TIMEOUT__RBBMIF_TIMEOUT_TO_REQ_HOLD_MASK 0xfff00000
+#define RBBMIF_TIMEOUT__RBBMIF_TIMEOUT_TO_REQ_HOLD__SHIFT 0x14
+#define RBBMIF_STATUS__RBBMIF_TIMEOUT_CLIENTS_DEC_MASK 0xffff
+#define RBBMIF_STATUS__RBBMIF_TIMEOUT_CLIENTS_DEC__SHIFT 0x0
+#define RBBMIF_STATUS__RBBMIF_TIMEOUT_OP_MASK 0x10000000
+#define RBBMIF_STATUS__RBBMIF_TIMEOUT_OP__SHIFT 0x1c
+#define RBBMIF_STATUS__RBBMIF_TIMEOUT_RDWR_STATUS_MASK 0x20000000
+#define RBBMIF_STATUS__RBBMIF_TIMEOUT_RDWR_STATUS__SHIFT 0x1d
+#define RBBMIF_STATUS__RBBMIF_TIMEOUT_ACK_MASK 0x40000000
+#define RBBMIF_STATUS__RBBMIF_TIMEOUT_ACK__SHIFT 0x1e
+#define RBBMIF_STATUS__RBBMIF_TIMEOUT_MASK_MASK 0x80000000
+#define RBBMIF_STATUS__RBBMIF_TIMEOUT_MASK__SHIFT 0x1f
+#define RBBMIF_TIMEOUT_DIS__CLIENT0_TIMEOUT_DIS_MASK 0x1
+#define RBBMIF_TIMEOUT_DIS__CLIENT0_TIMEOUT_DIS__SHIFT 0x0
+#define RBBMIF_TIMEOUT_DIS__CLIENT1_TIMEOUT_DIS_MASK 0x2
+#define RBBMIF_TIMEOUT_DIS__CLIENT1_TIMEOUT_DIS__SHIFT 0x1
+#define RBBMIF_TIMEOUT_DIS__CLIENT2_TIMEOUT_DIS_MASK 0x4
+#define RBBMIF_TIMEOUT_DIS__CLIENT2_TIMEOUT_DIS__SHIFT 0x2
+#define RBBMIF_TIMEOUT_DIS__CLIENT3_TIMEOUT_DIS_MASK 0x8
+#define RBBMIF_TIMEOUT_DIS__CLIENT3_TIMEOUT_DIS__SHIFT 0x3
+#define RBBMIF_TIMEOUT_DIS__CLIENT4_TIMEOUT_DIS_MASK 0x10
+#define RBBMIF_TIMEOUT_DIS__CLIENT4_TIMEOUT_DIS__SHIFT 0x4
+#define RBBMIF_TIMEOUT_DIS__CLIENT5_TIMEOUT_DIS_MASK 0x20
+#define RBBMIF_TIMEOUT_DIS__CLIENT5_TIMEOUT_DIS__SHIFT 0x5
+#define RBBMIF_TIMEOUT_DIS__CLIENT6_TIMEOUT_DIS_MASK 0x40
+#define RBBMIF_TIMEOUT_DIS__CLIENT6_TIMEOUT_DIS__SHIFT 0x6
+#define RBBMIF_TIMEOUT_DIS__CLIENT7_TIMEOUT_DIS_MASK 0x80
+#define RBBMIF_TIMEOUT_DIS__CLIENT7_TIMEOUT_DIS__SHIFT 0x7
+#define RBBMIF_TIMEOUT_DIS__CLIENT8_TIMEOUT_DIS_MASK 0x100
+#define RBBMIF_TIMEOUT_DIS__CLIENT8_TIMEOUT_DIS__SHIFT 0x8
+#define RBBMIF_TIMEOUT_DIS__CLIENT9_TIMEOUT_DIS_MASK 0x200
+#define RBBMIF_TIMEOUT_DIS__CLIENT9_TIMEOUT_DIS__SHIFT 0x9
+#define RBBMIF_TIMEOUT_DIS__CLIENT10_TIMEOUT_DIS_MASK 0x400
+#define RBBMIF_TIMEOUT_DIS__CLIENT10_TIMEOUT_DIS__SHIFT 0xa
+#define RBBMIF_TIMEOUT_DIS__CLIENT11_TIMEOUT_DIS_MASK 0x800
+#define RBBMIF_TIMEOUT_DIS__CLIENT11_TIMEOUT_DIS__SHIFT 0xb
+#define RBBMIF_TIMEOUT_DIS__CLIENT12_TIMEOUT_DIS_MASK 0x1000
+#define RBBMIF_TIMEOUT_DIS__CLIENT12_TIMEOUT_DIS__SHIFT 0xc
+#define RBBMIF_TIMEOUT_DIS__CLIENT13_TIMEOUT_DIS_MASK 0x2000
+#define RBBMIF_TIMEOUT_DIS__CLIENT13_TIMEOUT_DIS__SHIFT 0xd
+#define RBBMIF_TIMEOUT_DIS__CLIENT14_TIMEOUT_DIS_MASK 0x4000
+#define RBBMIF_TIMEOUT_DIS__CLIENT14_TIMEOUT_DIS__SHIFT 0xe
+#define RBBMIF_TIMEOUT_DIS__CLIENT15_TIMEOUT_DIS_MASK 0x8000
+#define RBBMIF_TIMEOUT_DIS__CLIENT15_TIMEOUT_DIS__SHIFT 0xf
+#define RBBMIF_STATUS_FLAG__RBBMIF_STATE_MASK 0x3
+#define RBBMIF_STATUS_FLAG__RBBMIF_STATE__SHIFT 0x0
+#define RBBMIF_STATUS_FLAG__RBBMIF_READ_TIMEOUT_MASK 0x10
+#define RBBMIF_STATUS_FLAG__RBBMIF_READ_TIMEOUT__SHIFT 0x4
+#define RBBMIF_STATUS_FLAG__RBBMIF_FIFO_EMPTY_MASK 0x20
+#define RBBMIF_STATUS_FLAG__RBBMIF_FIFO_EMPTY__SHIFT 0x5
+#define RBBMIF_STATUS_FLAG__RBBMIF_FIFO_FULL_MASK 0x40
+#define RBBMIF_STATUS_FLAG__RBBMIF_FIFO_FULL__SHIFT 0x6
+#define DCI_MEM_PWR_STATUS__DMIF_RDREQ_MEM1_PWR_STATE_MASK 0x3
+#define DCI_MEM_PWR_STATUS__DMIF_RDREQ_MEM1_PWR_STATE__SHIFT 0x0
+#define DCI_MEM_PWR_STATUS__DMIF_RDREQ_MEM2_PWR_STATE_MASK 0xc
+#define DCI_MEM_PWR_STATUS__DMIF_RDREQ_MEM2_PWR_STATE__SHIFT 0x2
+#define DCI_MEM_PWR_STATUS__MCIF_RDREQ_MEM_PWR_STATE_MASK 0x10
+#define DCI_MEM_PWR_STATUS__MCIF_RDREQ_MEM_PWR_STATE__SHIFT 0x4
+#define DCI_MEM_PWR_STATUS__MCIF_WRREQ_MEM_PWR_STATE_MASK 0x40
+#define DCI_MEM_PWR_STATUS__MCIF_WRREQ_MEM_PWR_STATE__SHIFT 0x6
+#define DCI_MEM_PWR_STATUS__VGA_MEM_PWR_STATE_MASK 0x100
+#define DCI_MEM_PWR_STATUS__VGA_MEM_PWR_STATE__SHIFT 0x8
+#define DCI_MEM_PWR_STATUS__DMCU_ERAM_MEM_PWR_STATE_MASK 0x600
+#define DCI_MEM_PWR_STATUS__DMCU_ERAM_MEM_PWR_STATE__SHIFT 0x9
+#define DCI_MEM_PWR_STATUS__DMCU_IRAM_MEM_PWR_STATE_MASK 0x800
+#define DCI_MEM_PWR_STATUS__DMCU_IRAM_MEM_PWR_STATE__SHIFT 0xb
+#define DCI_MEM_PWR_STATUS__FBC_MEM_PWR_STATE_MASK 0x3000
+#define DCI_MEM_PWR_STATUS__FBC_MEM_PWR_STATE__SHIFT 0xc
+#define DCI_MEM_PWR_STATUS__MCIF_MEM_PWR_STATE_MASK 0xc000
+#define DCI_MEM_PWR_STATUS__MCIF_MEM_PWR_STATE__SHIFT 0xe
+#define DCI_MEM_PWR_STATUS__VIP_MEM_PWR_STATE_MASK 0x400000
+#define DCI_MEM_PWR_STATUS__VIP_MEM_PWR_STATE__SHIFT 0x16
+#define DCI_MEM_PWR_STATUS__DMIF0_ASYNC_MEM_PWR_STATE_MASK 0x3000000
+#define DCI_MEM_PWR_STATUS__DMIF0_ASYNC_MEM_PWR_STATE__SHIFT 0x18
+#define DCI_MEM_PWR_STATUS__DMIF0_DATA_MEM_PWR_STATE_MASK 0xc000000
+#define DCI_MEM_PWR_STATUS__DMIF0_DATA_MEM_PWR_STATE__SHIFT 0x1a
+#define DCI_MEM_PWR_STATUS__DMIF0_CHUNK_MEM_PWR_STATE_MASK 0x10000000
+#define DCI_MEM_PWR_STATUS__DMIF0_CHUNK_MEM_PWR_STATE__SHIFT 0x1c
+#define DCI_MEM_PWR_STATUS__DMIF_RDREQ_MEM3_PWR_STATE_MASK 0xc0000000
+#define DCI_MEM_PWR_STATUS__DMIF_RDREQ_MEM3_PWR_STATE__SHIFT 0x1e
+#define DCI_MEM_PWR_STATUS2__DMIF1_ASYNC_MEM_PWR_STATE_MASK 0x3
+#define DCI_MEM_PWR_STATUS2__DMIF1_ASYNC_MEM_PWR_STATE__SHIFT 0x0
+#define DCI_MEM_PWR_STATUS2__DMIF1_DATA_MEM_PWR_STATE_MASK 0xc
+#define DCI_MEM_PWR_STATUS2__DMIF1_DATA_MEM_PWR_STATE__SHIFT 0x2
+#define DCI_MEM_PWR_STATUS2__DMIF1_CHUNK_MEM_PWR_STATE_MASK 0x10
+#define DCI_MEM_PWR_STATUS2__DMIF1_CHUNK_MEM_PWR_STATE__SHIFT 0x4
+#define DCI_MEM_PWR_STATUS2__DMIF2_ASYNC_MEM_PWR_STATE_MASK 0x60
+#define DCI_MEM_PWR_STATUS2__DMIF2_ASYNC_MEM_PWR_STATE__SHIFT 0x5
+#define DCI_MEM_PWR_STATUS2__DMIF2_DATA_MEM_PWR_STATE_MASK 0x180
+#define DCI_MEM_PWR_STATUS2__DMIF2_DATA_MEM_PWR_STATE__SHIFT 0x7
+#define DCI_MEM_PWR_STATUS2__DMIF2_CHUNK_MEM_PWR_STATE_MASK 0x200
+#define DCI_MEM_PWR_STATUS2__DMIF2_CHUNK_MEM_PWR_STATE__SHIFT 0x9
+#define DCI_MEM_PWR_STATUS2__DMIF3_ASYNC_MEM_PWR_STATE_MASK 0xc00
+#define DCI_MEM_PWR_STATUS2__DMIF3_ASYNC_MEM_PWR_STATE__SHIFT 0xa
+#define DCI_MEM_PWR_STATUS2__DMIF3_DATA_MEM_PWR_STATE_MASK 0x3000
+#define DCI_MEM_PWR_STATUS2__DMIF3_DATA_MEM_PWR_STATE__SHIFT 0xc
+#define DCI_MEM_PWR_STATUS2__DMIF3_CHUNK_MEM_PWR_STATE_MASK 0x4000
+#define DCI_MEM_PWR_STATUS2__DMIF3_CHUNK_MEM_PWR_STATE__SHIFT 0xe
+#define DCI_MEM_PWR_STATUS2__DMIF4_ASYNC_MEM_PWR_STATE_MASK 0x18000
+#define DCI_MEM_PWR_STATUS2__DMIF4_ASYNC_MEM_PWR_STATE__SHIFT 0xf
+#define DCI_MEM_PWR_STATUS2__DMIF4_DATA_MEM_PWR_STATE_MASK 0x60000
+#define DCI_MEM_PWR_STATUS2__DMIF4_DATA_MEM_PWR_STATE__SHIFT 0x11
+#define DCI_MEM_PWR_STATUS2__DMIF4_CHUNK_MEM_PWR_STATE_MASK 0x80000
+#define DCI_MEM_PWR_STATUS2__DMIF4_CHUNK_MEM_PWR_STATE__SHIFT 0x13
+#define DCI_MEM_PWR_STATUS2__DMIF5_ASYNC_MEM_PWR_STATE_MASK 0x300000
+#define DCI_MEM_PWR_STATUS2__DMIF5_ASYNC_MEM_PWR_STATE__SHIFT 0x14
+#define DCI_MEM_PWR_STATUS2__DMIF5_DATA_MEM_PWR_STATE_MASK 0xc00000
+#define DCI_MEM_PWR_STATUS2__DMIF5_DATA_MEM_PWR_STATE__SHIFT 0x16
+#define DCI_MEM_PWR_STATUS2__DMIF5_CHUNK_MEM_PWR_STATE_MASK 0x1000000
+#define DCI_MEM_PWR_STATUS2__DMIF5_CHUNK_MEM_PWR_STATE__SHIFT 0x18
+#define DCI_MEM_PWR_STATUS3__MCIF_DWB_LUMA_MEM0_PWR_STATE_MASK 0x3
+#define DCI_MEM_PWR_STATUS3__MCIF_DWB_LUMA_MEM0_PWR_STATE__SHIFT 0x0
+#define DCI_MEM_PWR_STATUS3__MCIF_DWB_LUMA_MEM1_PWR_STATE_MASK 0xc
+#define DCI_MEM_PWR_STATUS3__MCIF_DWB_LUMA_MEM1_PWR_STATE__SHIFT 0x2
+#define DCI_MEM_PWR_STATUS3__MCIF_DWB_CHROMA_MEM0_PWR_STATE_MASK 0x30
+#define DCI_MEM_PWR_STATUS3__MCIF_DWB_CHROMA_MEM0_PWR_STATE__SHIFT 0x4
+#define DCI_MEM_PWR_STATUS3__MCIF_DWB_CHROMA_MEM1_PWR_STATE_MASK 0xc0
+#define DCI_MEM_PWR_STATUS3__MCIF_DWB_CHROMA_MEM1_PWR_STATE__SHIFT 0x6
+#define DCI_MEM_PWR_STATUS3__MCIF_CWB0_LUMA_MEM0_PWR_STATE_MASK 0x300
+#define DCI_MEM_PWR_STATUS3__MCIF_CWB0_LUMA_MEM0_PWR_STATE__SHIFT 0x8
+#define DCI_MEM_PWR_STATUS3__MCIF_CWB0_LUMA_MEM1_PWR_STATE_MASK 0xc00
+#define DCI_MEM_PWR_STATUS3__MCIF_CWB0_LUMA_MEM1_PWR_STATE__SHIFT 0xa
+#define DCI_MEM_PWR_STATUS3__MCIF_CWB0_CHROMA_MEM0_PWR_STATE_MASK 0x3000
+#define DCI_MEM_PWR_STATUS3__MCIF_CWB0_CHROMA_MEM0_PWR_STATE__SHIFT 0xc
+#define DCI_MEM_PWR_STATUS3__MCIF_CWB0_CHROMA_MEM1_PWR_STATE_MASK 0xc000
+#define DCI_MEM_PWR_STATUS3__MCIF_CWB0_CHROMA_MEM1_PWR_STATE__SHIFT 0xe
+#define DCI_MEM_PWR_STATUS3__MCIF_CWB1_LUMA_MEM0_PWR_STATE_MASK 0x30000
+#define DCI_MEM_PWR_STATUS3__MCIF_CWB1_LUMA_MEM0_PWR_STATE__SHIFT 0x10
+#define DCI_MEM_PWR_STATUS3__MCIF_CWB1_LUMA_MEM1_PWR_STATE_MASK 0xc0000
+#define DCI_MEM_PWR_STATUS3__MCIF_CWB1_LUMA_MEM1_PWR_STATE__SHIFT 0x12
+#define DCI_MEM_PWR_STATUS3__MCIF_CWB1_CHROMA_MEM0_PWR_STATE_MASK 0x300000
+#define DCI_MEM_PWR_STATUS3__MCIF_CWB1_CHROMA_MEM0_PWR_STATE__SHIFT 0x14
+#define DCI_MEM_PWR_STATUS3__MCIF_CWB1_CHROMA_MEM1_PWR_STATE_MASK 0xc00000
+#define DCI_MEM_PWR_STATUS3__MCIF_CWB1_CHROMA_MEM1_PWR_STATE__SHIFT 0x16
+#define DCI_CLK_CNTL__DCI_TEST_CLK_SEL_MASK 0x1f
+#define DCI_CLK_CNTL__DCI_TEST_CLK_SEL__SHIFT 0x0
+#define DCI_CLK_CNTL__DISPCLK_R_DCI_GATE_DIS_MASK 0x20
+#define DCI_CLK_CNTL__DISPCLK_R_DCI_GATE_DIS__SHIFT 0x5
+#define DCI_CLK_CNTL__DISPCLK_M_GATE_DIS_MASK 0x40
+#define DCI_CLK_CNTL__DISPCLK_M_GATE_DIS__SHIFT 0x6
+#define DCI_CLK_CNTL__SCLK_G_STREAM_AZ_GATE_DIS_MASK 0x80
+#define DCI_CLK_CNTL__SCLK_G_STREAM_AZ_GATE_DIS__SHIFT 0x7
+#define DCI_CLK_CNTL__SCLK_R_AZ_GATE_DIS_MASK 0x100
+#define DCI_CLK_CNTL__SCLK_R_AZ_GATE_DIS__SHIFT 0x8
+#define DCI_CLK_CNTL__DISPCLK_G_FBC_GATE_DIS_MASK 0x200
+#define DCI_CLK_CNTL__DISPCLK_G_FBC_GATE_DIS__SHIFT 0x9
+#define DCI_CLK_CNTL__DISPCLK_G_DMIFV1_L_GATE_DIS_MASK 0x400
+#define DCI_CLK_CNTL__DISPCLK_G_DMIFV1_L_GATE_DIS__SHIFT 0xa
+#define DCI_CLK_CNTL__DISPCLK_G_VGA_GATE_DIS_MASK 0x800
+#define DCI_CLK_CNTL__DISPCLK_G_VGA_GATE_DIS__SHIFT 0xb
+#define DCI_CLK_CNTL__DISPCLK_G_DMIFV1_C_GATE_DIS_MASK 0x1000
+#define DCI_CLK_CNTL__DISPCLK_G_DMIFV1_C_GATE_DIS__SHIFT 0xc
+#define DCI_CLK_CNTL__DISPCLK_G_VIP_GATE_DIS_MASK 0x2000
+#define DCI_CLK_CNTL__DISPCLK_G_VIP_GATE_DIS__SHIFT 0xd
+#define DCI_CLK_CNTL__VPCLK_POL_MASK 0x4000
+#define DCI_CLK_CNTL__VPCLK_POL__SHIFT 0xe
+#define DCI_CLK_CNTL__DISPCLK_G_DMCU_GATE_DIS_MASK 0x8000
+#define DCI_CLK_CNTL__DISPCLK_G_DMCU_GATE_DIS__SHIFT 0xf
+#define DCI_CLK_CNTL__DISPCLK_G_DMIF0_GATE_DIS_MASK 0x10000
+#define DCI_CLK_CNTL__DISPCLK_G_DMIF0_GATE_DIS__SHIFT 0x10
+#define DCI_CLK_CNTL__DISPCLK_G_DMIF1_GATE_DIS_MASK 0x20000
+#define DCI_CLK_CNTL__DISPCLK_G_DMIF1_GATE_DIS__SHIFT 0x11
+#define DCI_CLK_CNTL__DISPCLK_G_DMIF2_GATE_DIS_MASK 0x40000
+#define DCI_CLK_CNTL__DISPCLK_G_DMIF2_GATE_DIS__SHIFT 0x12
+#define DCI_CLK_CNTL__DISPCLK_G_DMIF3_GATE_DIS_MASK 0x80000
+#define DCI_CLK_CNTL__DISPCLK_G_DMIF3_GATE_DIS__SHIFT 0x13
+#define DCI_CLK_CNTL__DISPCLK_G_DMIF4_GATE_DIS_MASK 0x100000
+#define DCI_CLK_CNTL__DISPCLK_G_DMIF4_GATE_DIS__SHIFT 0x14
+#define DCI_CLK_CNTL__DISPCLK_G_DMIF5_GATE_DIS_MASK 0x200000
+#define DCI_CLK_CNTL__DISPCLK_G_DMIF5_GATE_DIS__SHIFT 0x15
+#define DCI_CLK_CNTL__SCLK_G_DMIF_GATE_DIS_MASK 0x400000
+#define DCI_CLK_CNTL__SCLK_G_DMIF_GATE_DIS__SHIFT 0x16
+#define DCI_CLK_CNTL__SCLK_G_DMIFTRK_GATE_DIS_MASK 0x800000
+#define DCI_CLK_CNTL__SCLK_G_DMIFTRK_GATE_DIS__SHIFT 0x17
+#define DCI_CLK_CNTL__SCLK_G_CNTL_AZ_GATE_DIS_MASK 0x1000000
+#define DCI_CLK_CNTL__SCLK_G_CNTL_AZ_GATE_DIS__SHIFT 0x18
+#define DCI_CLK_CNTL__DISPCLK_G_DMIFV0_L_GATE_DIS_MASK 0x2000000
+#define DCI_CLK_CNTL__DISPCLK_G_DMIFV0_L_GATE_DIS__SHIFT 0x19
+#define DCI_CLK_CNTL__DISPCLK_G_DMIFV0_C_GATE_DIS_MASK 0x4000000
+#define DCI_CLK_CNTL__DISPCLK_G_DMIFV0_C_GATE_DIS__SHIFT 0x1a
+#define DCI_CLK_CNTL__DCI_PG_TEST_CLK_SEL_MASK 0xf8000000
+#define DCI_CLK_CNTL__DCI_PG_TEST_CLK_SEL__SHIFT 0x1b
+#define DCI_CLK_RAMP_CNTL__DISPCLK_G_MCIF_DWB_GATE_DIS_MASK 0x1
+#define DCI_CLK_RAMP_CNTL__DISPCLK_G_MCIF_DWB_GATE_DIS__SHIFT 0x0
+#define DCI_CLK_RAMP_CNTL__SCLK_G_MCIF_DWB_GATE_DIS_MASK 0x2
+#define DCI_CLK_RAMP_CNTL__SCLK_G_MCIF_DWB_GATE_DIS__SHIFT 0x1
+#define DCI_CLK_RAMP_CNTL__DISPCLK_G_MCIF_CWB0_GATE_DIS_MASK 0x4
+#define DCI_CLK_RAMP_CNTL__DISPCLK_G_MCIF_CWB0_GATE_DIS__SHIFT 0x2
+#define DCI_CLK_RAMP_CNTL__SCLK_G_MCIF_CWB0_GATE_DIS_MASK 0x8
+#define DCI_CLK_RAMP_CNTL__SCLK_G_MCIF_CWB0_GATE_DIS__SHIFT 0x3
+#define DCI_CLK_RAMP_CNTL__DISPCLK_G_MCIF_CWB1_GATE_DIS_MASK 0x10
+#define DCI_CLK_RAMP_CNTL__DISPCLK_G_MCIF_CWB1_GATE_DIS__SHIFT 0x4
+#define DCI_CLK_RAMP_CNTL__SCLK_G_MCIF_CWB1_GATE_DIS_MASK 0x80000000
+#define DCI_CLK_RAMP_CNTL__SCLK_G_MCIF_CWB1_GATE_DIS__SHIFT 0x1f
+#define DCI_MEM_PWR_CNTL__DMIF_RDREQ_MEM_PWR_FORCE_MASK 0x3
+#define DCI_MEM_PWR_CNTL__DMIF_RDREQ_MEM_PWR_FORCE__SHIFT 0x0
+#define DCI_MEM_PWR_CNTL__DMIF_RDREQ_MEM_PWR_DIS_MASK 0x4
+#define DCI_MEM_PWR_CNTL__DMIF_RDREQ_MEM_PWR_DIS__SHIFT 0x2
+#define DCI_MEM_PWR_CNTL__MCIF_RDREQ_MEM_PWR_FORCE_MASK 0x8
+#define DCI_MEM_PWR_CNTL__MCIF_RDREQ_MEM_PWR_FORCE__SHIFT 0x3
+#define DCI_MEM_PWR_CNTL__MCIF_RDREQ_MEM_PWR_DIS_MASK 0x10
+#define DCI_MEM_PWR_CNTL__MCIF_RDREQ_MEM_PWR_DIS__SHIFT 0x4
+#define DCI_MEM_PWR_CNTL__MCIF_WRREQ_MEM_PWR_FORCE_MASK 0x20
+#define DCI_MEM_PWR_CNTL__MCIF_WRREQ_MEM_PWR_FORCE__SHIFT 0x5
+#define DCI_MEM_PWR_CNTL__MCIF_WRREQ_MEM_PWR_DIS_MASK 0x40
+#define DCI_MEM_PWR_CNTL__MCIF_WRREQ_MEM_PWR_DIS__SHIFT 0x6
+#define DCI_MEM_PWR_CNTL__VGA_MEM_PWR_FORCE_MASK 0x80
+#define DCI_MEM_PWR_CNTL__VGA_MEM_PWR_FORCE__SHIFT 0x7
+#define DCI_MEM_PWR_CNTL__VGA_MEM_PWR_DIS_MASK 0x100
+#define DCI_MEM_PWR_CNTL__VGA_MEM_PWR_DIS__SHIFT 0x8
+#define DCI_MEM_PWR_CNTL__DMCU_ERAM_MEM_PWR_FORCE_MASK 0x600
+#define DCI_MEM_PWR_CNTL__DMCU_ERAM_MEM_PWR_FORCE__SHIFT 0x9
+#define DCI_MEM_PWR_CNTL__DMCU_ERAM_MEM_PWR_DIS_MASK 0x800
+#define DCI_MEM_PWR_CNTL__DMCU_ERAM_MEM_PWR_DIS__SHIFT 0xb
+#define DCI_MEM_PWR_CNTL__DMCU_IRAM_MEM_PWR_FORCE_MASK 0x1000
+#define DCI_MEM_PWR_CNTL__DMCU_IRAM_MEM_PWR_FORCE__SHIFT 0xc
+#define DCI_MEM_PWR_CNTL__DMCU_IRAM_MEM_PWR_DIS_MASK 0x2000
+#define DCI_MEM_PWR_CNTL__DMCU_IRAM_MEM_PWR_DIS__SHIFT 0xd
+#define DCI_MEM_PWR_CNTL__FBC_MEM_PWR_FORCE_MASK 0xc000
+#define DCI_MEM_PWR_CNTL__FBC_MEM_PWR_FORCE__SHIFT 0xe
+#define DCI_MEM_PWR_CNTL__FBC_MEM_PWR_DIS_MASK 0x10000
+#define DCI_MEM_PWR_CNTL__FBC_MEM_PWR_DIS__SHIFT 0x10
+#define DCI_MEM_PWR_CNTL__MCIF_MEM_PWR_FORCE_MASK 0x60000
+#define DCI_MEM_PWR_CNTL__MCIF_MEM_PWR_FORCE__SHIFT 0x11
+#define DCI_MEM_PWR_CNTL__MCIF_MEM_PWR_DIS_MASK 0x80000
+#define DCI_MEM_PWR_CNTL__MCIF_MEM_PWR_DIS__SHIFT 0x13
+#define DCI_MEM_PWR_CNTL__MCIF_DWB_MEM_PWR_FORCE_MASK 0x300000
+#define DCI_MEM_PWR_CNTL__MCIF_DWB_MEM_PWR_FORCE__SHIFT 0x14
+#define DCI_MEM_PWR_CNTL__MCIF_DWB_MEM_PWR_DIS_MASK 0x400000
+#define DCI_MEM_PWR_CNTL__MCIF_DWB_MEM_PWR_DIS__SHIFT 0x16
+#define DCI_MEM_PWR_CNTL__MCIF_CWB0_MEM_PWR_FORCE_MASK 0x1800000
+#define DCI_MEM_PWR_CNTL__MCIF_CWB0_MEM_PWR_FORCE__SHIFT 0x17
+#define DCI_MEM_PWR_CNTL__MCIF_CWB0_MEM_PWR_DIS_MASK 0x2000000
+#define DCI_MEM_PWR_CNTL__MCIF_CWB0_MEM_PWR_DIS__SHIFT 0x19
+#define DCI_MEM_PWR_CNTL__MCIF_CWB1_MEM_PWR_FORCE_MASK 0xc000000
+#define DCI_MEM_PWR_CNTL__MCIF_CWB1_MEM_PWR_FORCE__SHIFT 0x1a
+#define DCI_MEM_PWR_CNTL__MCIF_CWB1_MEM_PWR_DIS_MASK 0x10000000
+#define DCI_MEM_PWR_CNTL__MCIF_CWB1_MEM_PWR_DIS__SHIFT 0x1c
+#define DCI_MEM_PWR_CNTL__VIP_MEM_PWR_FORCE_MASK 0x20000000
+#define DCI_MEM_PWR_CNTL__VIP_MEM_PWR_FORCE__SHIFT 0x1d
+#define DCI_MEM_PWR_CNTL__VIP_MEM_PWR_DIS_MASK 0x40000000
+#define DCI_MEM_PWR_CNTL__VIP_MEM_PWR_DIS__SHIFT 0x1e
+#define DCI_MEM_PWR_CNTL2__DMIF0_ASYNC_MEM_PWR_FORCE_MASK 0x3
+#define DCI_MEM_PWR_CNTL2__DMIF0_ASYNC_MEM_PWR_FORCE__SHIFT 0x0
+#define DCI_MEM_PWR_CNTL2__DMIF0_ASYNC_MEM_PWR_DIS_MASK 0x4
+#define DCI_MEM_PWR_CNTL2__DMIF0_ASYNC_MEM_PWR_DIS__SHIFT 0x2
+#define DCI_MEM_PWR_CNTL2__DMIF0_DATA_MEM_PWR_FORCE_MASK 0x18
+#define DCI_MEM_PWR_CNTL2__DMIF0_DATA_MEM_PWR_FORCE__SHIFT 0x3
+#define DCI_MEM_PWR_CNTL2__DMIF0_DATA_MEM_PWR_DIS_MASK 0x20
+#define DCI_MEM_PWR_CNTL2__DMIF0_DATA_MEM_PWR_DIS__SHIFT 0x5
+#define DCI_MEM_PWR_CNTL2__DMIF0_CHUNK_MEM_PWR_FORCE_MASK 0x40
+#define DCI_MEM_PWR_CNTL2__DMIF0_CHUNK_MEM_PWR_FORCE__SHIFT 0x6
+#define DCI_MEM_PWR_CNTL2__DMIF0_CHUNK_MEM_PWR_DIS_MASK 0x80
+#define DCI_MEM_PWR_CNTL2__DMIF0_CHUNK_MEM_PWR_DIS__SHIFT 0x7
+#define DCI_MEM_PWR_CNTL2__DMIF1_ASYNC_MEM_PWR_FORCE_MASK 0x300
+#define DCI_MEM_PWR_CNTL2__DMIF1_ASYNC_MEM_PWR_FORCE__SHIFT 0x8
+#define DCI_MEM_PWR_CNTL2__DMIF1_ASYNC_MEM_PWR_DIS_MASK 0x400
+#define DCI_MEM_PWR_CNTL2__DMIF1_ASYNC_MEM_PWR_DIS__SHIFT 0xa
+#define DCI_MEM_PWR_CNTL2__DMIF1_DATA_MEM_PWR_FORCE_MASK 0x1800
+#define DCI_MEM_PWR_CNTL2__DMIF1_DATA_MEM_PWR_FORCE__SHIFT 0xb
+#define DCI_MEM_PWR_CNTL2__DMIF1_DATA_MEM_PWR_DIS_MASK 0x2000
+#define DCI_MEM_PWR_CNTL2__DMIF1_DATA_MEM_PWR_DIS__SHIFT 0xd
+#define DCI_MEM_PWR_CNTL2__DMIF1_CHUNK_MEM_PWR_FORCE_MASK 0x4000
+#define DCI_MEM_PWR_CNTL2__DMIF1_CHUNK_MEM_PWR_FORCE__SHIFT 0xe
+#define DCI_MEM_PWR_CNTL2__DMIF1_CHUNK_MEM_PWR_DIS_MASK 0x8000
+#define DCI_MEM_PWR_CNTL2__DMIF1_CHUNK_MEM_PWR_DIS__SHIFT 0xf
+#define DCI_MEM_PWR_CNTL2__DMIF2_ASYNC_MEM_PWR_FORCE_MASK 0x30000
+#define DCI_MEM_PWR_CNTL2__DMIF2_ASYNC_MEM_PWR_FORCE__SHIFT 0x10
+#define DCI_MEM_PWR_CNTL2__DMIF2_ASYNC_MEM_PWR_DIS_MASK 0x40000
+#define DCI_MEM_PWR_CNTL2__DMIF2_ASYNC_MEM_PWR_DIS__SHIFT 0x12
+#define DCI_MEM_PWR_CNTL2__DMIF2_DATA_MEM_PWR_FORCE_MASK 0x180000
+#define DCI_MEM_PWR_CNTL2__DMIF2_DATA_MEM_PWR_FORCE__SHIFT 0x13
+#define DCI_MEM_PWR_CNTL2__DMIF2_DATA_MEM_PWR_DIS_MASK 0x200000
+#define DCI_MEM_PWR_CNTL2__DMIF2_DATA_MEM_PWR_DIS__SHIFT 0x15
+#define DCI_MEM_PWR_CNTL2__DMIF2_CHUNK_MEM_PWR_FORCE_MASK 0x400000
+#define DCI_MEM_PWR_CNTL2__DMIF2_CHUNK_MEM_PWR_FORCE__SHIFT 0x16
+#define DCI_MEM_PWR_CNTL2__DMIF2_CHUNK_MEM_PWR_DIS_MASK 0x800000
+#define DCI_MEM_PWR_CNTL2__DMIF2_CHUNK_MEM_PWR_DIS__SHIFT 0x17
+#define DCI_MEM_PWR_CNTL2__DMIF3_ASYNC_MEM_PWR_FORCE_MASK 0x3000000
+#define DCI_MEM_PWR_CNTL2__DMIF3_ASYNC_MEM_PWR_FORCE__SHIFT 0x18
+#define DCI_MEM_PWR_CNTL2__DMIF3_ASYNC_MEM_PWR_DIS_MASK 0x4000000
+#define DCI_MEM_PWR_CNTL2__DMIF3_ASYNC_MEM_PWR_DIS__SHIFT 0x1a
+#define DCI_MEM_PWR_CNTL2__DMIF3_DATA_MEM_PWR_FORCE_MASK 0x18000000
+#define DCI_MEM_PWR_CNTL2__DMIF3_DATA_MEM_PWR_FORCE__SHIFT 0x1b
+#define DCI_MEM_PWR_CNTL2__DMIF3_DATA_MEM_PWR_DIS_MASK 0x20000000
+#define DCI_MEM_PWR_CNTL2__DMIF3_DATA_MEM_PWR_DIS__SHIFT 0x1d
+#define DCI_MEM_PWR_CNTL2__DMIF3_CHUNK_MEM_PWR_FORCE_MASK 0x40000000
+#define DCI_MEM_PWR_CNTL2__DMIF3_CHUNK_MEM_PWR_FORCE__SHIFT 0x1e
+#define DCI_MEM_PWR_CNTL2__DMIF3_CHUNK_MEM_PWR_DIS_MASK 0x80000000
+#define DCI_MEM_PWR_CNTL2__DMIF3_CHUNK_MEM_PWR_DIS__SHIFT 0x1f
+#define DCI_MEM_PWR_CNTL3__DMIF4_ASYNC_MEM_PWR_FORCE_MASK 0x3
+#define DCI_MEM_PWR_CNTL3__DMIF4_ASYNC_MEM_PWR_FORCE__SHIFT 0x0
+#define DCI_MEM_PWR_CNTL3__DMIF4_ASYNC_MEM_PWR_DIS_MASK 0x4
+#define DCI_MEM_PWR_CNTL3__DMIF4_ASYNC_MEM_PWR_DIS__SHIFT 0x2
+#define DCI_MEM_PWR_CNTL3__DMIF4_DATA_MEM_PWR_FORCE_MASK 0x18
+#define DCI_MEM_PWR_CNTL3__DMIF4_DATA_MEM_PWR_FORCE__SHIFT 0x3
+#define DCI_MEM_PWR_CNTL3__DMIF4_DATA_MEM_PWR_DIS_MASK 0x20
+#define DCI_MEM_PWR_CNTL3__DMIF4_DATA_MEM_PWR_DIS__SHIFT 0x5
+#define DCI_MEM_PWR_CNTL3__DMIF4_CHUNK_MEM_PWR_FORCE_MASK 0x40
+#define DCI_MEM_PWR_CNTL3__DMIF4_CHUNK_MEM_PWR_FORCE__SHIFT 0x6
+#define DCI_MEM_PWR_CNTL3__DMIF4_CHUNK_MEM_PWR_DIS_MASK 0x80
+#define DCI_MEM_PWR_CNTL3__DMIF4_CHUNK_MEM_PWR_DIS__SHIFT 0x7
+#define DCI_MEM_PWR_CNTL3__DMIF5_ASYNC_MEM_PWR_FORCE_MASK 0x300
+#define DCI_MEM_PWR_CNTL3__DMIF5_ASYNC_MEM_PWR_FORCE__SHIFT 0x8
+#define DCI_MEM_PWR_CNTL3__DMIF5_ASYNC_MEM_PWR_DIS_MASK 0x400
+#define DCI_MEM_PWR_CNTL3__DMIF5_ASYNC_MEM_PWR_DIS__SHIFT 0xa
+#define DCI_MEM_PWR_CNTL3__DMIF5_DATA_MEM_PWR_FORCE_MASK 0x1800
+#define DCI_MEM_PWR_CNTL3__DMIF5_DATA_MEM_PWR_FORCE__SHIFT 0xb
+#define DCI_MEM_PWR_CNTL3__DMIF5_DATA_MEM_PWR_DIS_MASK 0x2000
+#define DCI_MEM_PWR_CNTL3__DMIF5_DATA_MEM_PWR_DIS__SHIFT 0xd
+#define DCI_MEM_PWR_CNTL3__DMIF5_CHUNK_MEM_PWR_FORCE_MASK 0x4000
+#define DCI_MEM_PWR_CNTL3__DMIF5_CHUNK_MEM_PWR_FORCE__SHIFT 0xe
+#define DCI_MEM_PWR_CNTL3__DMIF5_CHUNK_MEM_PWR_DIS_MASK 0x8000
+#define DCI_MEM_PWR_CNTL3__DMIF5_CHUNK_MEM_PWR_DIS__SHIFT 0xf
+#define DCI_MEM_PWR_CNTL3__DMIF_RDREQ_MEM_PWR_MODE_SEL_MASK 0x30000
+#define DCI_MEM_PWR_CNTL3__DMIF_RDREQ_MEM_PWR_MODE_SEL__SHIFT 0x10
+#define DCI_MEM_PWR_CNTL3__DMIF_ASYNC_MEM_PWR_MODE_SEL_MASK 0xc0000
+#define DCI_MEM_PWR_CNTL3__DMIF_ASYNC_MEM_PWR_MODE_SEL__SHIFT 0x12
+#define DCI_MEM_PWR_CNTL3__DMIF_DATA_MEM_PWR_MODE_SEL_MASK 0x300000
+#define DCI_MEM_PWR_CNTL3__DMIF_DATA_MEM_PWR_MODE_SEL__SHIFT 0x14
+#define DCI_MEM_PWR_CNTL3__DMCU_ERAM_MEM_PWR_MODE_SEL_MASK 0x400000
+#define DCI_MEM_PWR_CNTL3__DMCU_ERAM_MEM_PWR_MODE_SEL__SHIFT 0x16
+#define DCI_MEM_PWR_CNTL3__FBC_MEM_PWR_MODE_SEL_MASK 0x1800000
+#define DCI_MEM_PWR_CNTL3__FBC_MEM_PWR_MODE_SEL__SHIFT 0x17
+#define DCI_MEM_PWR_CNTL3__MCIF_CWB0_MEM_PWR_MODE_SEL_MASK 0x6000000
+#define DCI_MEM_PWR_CNTL3__MCIF_CWB0_MEM_PWR_MODE_SEL__SHIFT 0x19
+#define DCI_MEM_PWR_CNTL3__MCIF_CWB1_MEM_PWR_MODE_SEL_MASK 0x18000000
+#define DCI_MEM_PWR_CNTL3__MCIF_CWB1_MEM_PWR_MODE_SEL__SHIFT 0x1b
+#define DCI_MEM_PWR_CNTL3__MCIF_DWB_MEM_PWR_MODE_SEL_MASK 0x60000000
+#define DCI_MEM_PWR_CNTL3__MCIF_DWB_MEM_PWR_MODE_SEL__SHIFT 0x1d
+#define DCI_MEM_PWR_CNTL4__MCIF_DWB_LUMA_MEM_EN_NUM_MASK 0x1
+#define DCI_MEM_PWR_CNTL4__MCIF_DWB_LUMA_MEM_EN_NUM__SHIFT 0x0
+#define DCI_MEM_PWR_CNTL4__MCIF_DWB_CHROMA_MEM_EN_NUM_MASK 0x2
+#define DCI_MEM_PWR_CNTL4__MCIF_DWB_CHROMA_MEM_EN_NUM__SHIFT 0x1
+#define DCI_MEM_PWR_CNTL4__MCIF_CWB0_LUMA_MEM_EN_NUM_MASK 0x4
+#define DCI_MEM_PWR_CNTL4__MCIF_CWB0_LUMA_MEM_EN_NUM__SHIFT 0x2
+#define DCI_MEM_PWR_CNTL4__MCIF_CWB0_CHROMA_MEM_EN_NUM_MASK 0x8
+#define DCI_MEM_PWR_CNTL4__MCIF_CWB0_CHROMA_MEM_EN_NUM__SHIFT 0x3
+#define DCI_MEM_PWR_CNTL4__MCIF_CWB1_LUMA_MEM_EN_NUM_MASK 0x10
+#define DCI_MEM_PWR_CNTL4__MCIF_CWB1_LUMA_MEM_EN_NUM__SHIFT 0x4
+#define DCI_MEM_PWR_CNTL4__MCIF_CWB1_CHROMA_MEM_EN_NUM_MASK 0x20
+#define DCI_MEM_PWR_CNTL4__MCIF_CWB1_CHROMA_MEM_EN_NUM__SHIFT 0x5
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE0_MEM_PWR_FORCE_MASK 0x3
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE0_MEM_PWR_FORCE__SHIFT 0x0
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE0_MEM_PWR_DIS_MASK 0x4
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE0_MEM_PWR_DIS__SHIFT 0x2
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE1_MEM_PWR_FORCE_MASK 0x18
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE1_MEM_PWR_FORCE__SHIFT 0x3
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE1_MEM_PWR_DIS_MASK 0x20
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE1_MEM_PWR_DIS__SHIFT 0x5
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE2_MEM_PWR_FORCE_MASK 0xc0
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE2_MEM_PWR_FORCE__SHIFT 0x6
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE2_MEM_PWR_DIS_MASK 0x100
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE2_MEM_PWR_DIS__SHIFT 0x8
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE3_MEM_PWR_FORCE_MASK 0x600
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE3_MEM_PWR_FORCE__SHIFT 0x9
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE3_MEM_PWR_DIS_MASK 0x800
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE3_MEM_PWR_DIS__SHIFT 0xb
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE4_MEM_PWR_FORCE_MASK 0x3000
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE4_MEM_PWR_FORCE__SHIFT 0xc
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE4_MEM_PWR_DIS_MASK 0x4000
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE4_MEM_PWR_DIS__SHIFT 0xe
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE5_MEM_PWR_FORCE_MASK 0x18000
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE5_MEM_PWR_FORCE__SHIFT 0xf
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE5_MEM_PWR_DIS_MASK 0x20000
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE5_MEM_PWR_DIS__SHIFT 0x11
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE6_MEM_PWR_FORCE_MASK 0xc0000
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE6_MEM_PWR_FORCE__SHIFT 0x12
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE6_MEM_PWR_DIS_MASK 0x100000
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE6_MEM_PWR_DIS__SHIFT 0x14
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE7_MEM_PWR_FORCE_MASK 0x600000
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE7_MEM_PWR_FORCE__SHIFT 0x15
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE7_MEM_PWR_DIS_MASK 0x800000
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE7_MEM_PWR_DIS__SHIFT 0x17
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE_MEM_PWR_MODE_SEL_MASK 0x3000000
+#define DVMM_PTE_PGMEM_CONTROL__DVMM_PTE_MEM_PWR_MODE_SEL__SHIFT 0x18
+#define DVMM_PTE_PGMEM_STATE__DVMM_PIPE0_PTE_PGMEM_STATE_MASK 0x3
+#define DVMM_PTE_PGMEM_STATE__DVMM_PIPE0_PTE_PGMEM_STATE__SHIFT 0x0
+#define DVMM_PTE_PGMEM_STATE__DVMM_PIPE1_PTE_PGMEM_STATE_MASK 0xc
+#define DVMM_PTE_PGMEM_STATE__DVMM_PIPE1_PTE_PGMEM_STATE__SHIFT 0x2
+#define DVMM_PTE_PGMEM_STATE__DVMM_PIPE2_PTE_PGMEM_STATE_MASK 0x30
+#define DVMM_PTE_PGMEM_STATE__DVMM_PIPE2_PTE_PGMEM_STATE__SHIFT 0x4
+#define DVMM_PTE_PGMEM_STATE__DVMM_PIPE3_PTE_PGMEM_STATE_MASK 0xc0
+#define DVMM_PTE_PGMEM_STATE__DVMM_PIPE3_PTE_PGMEM_STATE__SHIFT 0x6
+#define DVMM_PTE_PGMEM_STATE__DVMM_PIPE4_PTE_PGMEM_STATE_MASK 0x300
+#define DVMM_PTE_PGMEM_STATE__DVMM_PIPE4_PTE_PGMEM_STATE__SHIFT 0x8
+#define DVMM_PTE_PGMEM_STATE__DVMM_PIPE5_PTE_PGMEM_STATE_MASK 0xc00
+#define DVMM_PTE_PGMEM_STATE__DVMM_PIPE5_PTE_PGMEM_STATE__SHIFT 0xa
+#define DVMM_PTE_PGMEM_STATE__DVMM_PIPE6_PTE_PGMEM_STATE_MASK 0x3000
+#define DVMM_PTE_PGMEM_STATE__DVMM_PIPE6_PTE_PGMEM_STATE__SHIFT 0xc
+#define DVMM_PTE_PGMEM_STATE__DVMM_PIPE7_PTE_PGMEM_STATE_MASK 0xc000
+#define DVMM_PTE_PGMEM_STATE__DVMM_PIPE7_PTE_PGMEM_STATE__SHIFT 0xe
+#define DCI_SOFT_RESET__VGA_SOFT_RESET_MASK 0x1
+#define DCI_SOFT_RESET__VGA_SOFT_RESET__SHIFT 0x0
+#define DCI_SOFT_RESET__VIP_SOFT_RESET_MASK 0x2
+#define DCI_SOFT_RESET__VIP_SOFT_RESET__SHIFT 0x1
+#define DCI_SOFT_RESET__MCIF_SOFT_RESET_MASK 0x4
+#define DCI_SOFT_RESET__MCIF_SOFT_RESET__SHIFT 0x2
+#define DCI_SOFT_RESET__FBC_SOFT_RESET_MASK 0x8
+#define DCI_SOFT_RESET__FBC_SOFT_RESET__SHIFT 0x3
+#define DCI_SOFT_RESET__DMIF0_SOFT_RESET_MASK 0x10
+#define DCI_SOFT_RESET__DMIF0_SOFT_RESET__SHIFT 0x4
+#define DCI_SOFT_RESET__DMIF1_SOFT_RESET_MASK 0x20
+#define DCI_SOFT_RESET__DMIF1_SOFT_RESET__SHIFT 0x5
+#define DCI_SOFT_RESET__DMIF2_SOFT_RESET_MASK 0x40
+#define DCI_SOFT_RESET__DMIF2_SOFT_RESET__SHIFT 0x6
+#define DCI_SOFT_RESET__DMIF3_SOFT_RESET_MASK 0x80
+#define DCI_SOFT_RESET__DMIF3_SOFT_RESET__SHIFT 0x7
+#define DCI_SOFT_RESET__DMIF4_SOFT_RESET_MASK 0x100
+#define DCI_SOFT_RESET__DMIF4_SOFT_RESET__SHIFT 0x8
+#define DCI_SOFT_RESET__DMIF5_SOFT_RESET_MASK 0x200
+#define DCI_SOFT_RESET__DMIF5_SOFT_RESET__SHIFT 0x9
+#define DCI_SOFT_RESET__DCFEV0_L_SOFT_RESET_MASK 0x400
+#define DCI_SOFT_RESET__DCFEV0_L_SOFT_RESET__SHIFT 0xa
+#define DCI_SOFT_RESET__DCFEV0_C_SOFT_RESET_MASK 0x800
+#define DCI_SOFT_RESET__DCFEV0_C_SOFT_RESET__SHIFT 0xb
+#define DCI_SOFT_RESET__DCFEV1_L_SOFT_RESET_MASK 0x1000
+#define DCI_SOFT_RESET__DCFEV1_L_SOFT_RESET__SHIFT 0xc
+#define DCI_SOFT_RESET__DCFEV1_C_SOFT_RESET_MASK 0x2000
+#define DCI_SOFT_RESET__DCFEV1_C_SOFT_RESET__SHIFT 0xd
+#define DCI_SOFT_RESET__DMIFARB_SOFT_RESET_MASK 0x4000
+#define DCI_SOFT_RESET__DMIFARB_SOFT_RESET__SHIFT 0xe
+#define DCI_SOFT_RESET__MCIF_DWB_SOFT_RESET_MASK 0x10000
+#define DCI_SOFT_RESET__MCIF_DWB_SOFT_RESET__SHIFT 0x10
+#define DCI_SOFT_RESET__MCIF_CWB0_SOFT_RESET_MASK 0x20000
+#define DCI_SOFT_RESET__MCIF_CWB0_SOFT_RESET__SHIFT 0x11
+#define DCI_SOFT_RESET__MCIF_CWB1_SOFT_RESET_MASK 0x40000
+#define DCI_SOFT_RESET__MCIF_CWB1_SOFT_RESET__SHIFT 0x12
+#define DCI_SOFT_RESET__MCIF_WB_SOFT_RESET_MASK 0x80000
+#define DCI_SOFT_RESET__MCIF_WB_SOFT_RESET__SHIFT 0x13
+#define DCI_MISC__MCIF_WB_URG_OVRD_MASK 0x1
+#define DCI_MISC__MCIF_WB_URG_OVRD__SHIFT 0x0
+#define DCI_MISC__MCIF_WB_URG_LVL_MASK 0x1e
+#define DCI_MISC__MCIF_WB_URG_LVL__SHIFT 0x1
+#define DCI_TEST_DEBUG_INDEX__DCI_TEST_DEBUG_INDEX_MASK 0xff
+#define DCI_TEST_DEBUG_INDEX__DCI_TEST_DEBUG_INDEX__SHIFT 0x0
+#define DCI_TEST_DEBUG_INDEX__DCI_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define DCI_TEST_DEBUG_INDEX__DCI_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define DCI_TEST_DEBUG_DATA__DCI_TEST_DEBUG_DATA_MASK 0xffffffff
+#define DCI_TEST_DEBUG_DATA__DCI_TEST_DEBUG_DATA__SHIFT 0x0
+#define DCI_DEBUG_CONFIG__DCI_DBG_EN_MASK 0x1
+#define DCI_DEBUG_CONFIG__DCI_DBG_EN__SHIFT 0x0
+#define DCI_DEBUG_CONFIG__DCI_DBG_BLOCK_SEL_MASK 0xf0
+#define DCI_DEBUG_CONFIG__DCI_DBG_BLOCK_SEL__SHIFT 0x4
+#define DCI_DEBUG_CONFIG__DCI_DBG_CLOCK_SEL_MASK 0xf00
+#define DCI_DEBUG_CONFIG__DCI_DBG_CLOCK_SEL__SHIFT 0x8
+#define PIPE0_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATED_MASK 0x7
+#define PIPE0_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATED__SHIFT 0x0
+#define PIPE0_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATION_COMPLETED_MASK 0x10
+#define PIPE0_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATION_COMPLETED__SHIFT 0x4
+#define PIPE1_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATED_MASK 0x7
+#define PIPE1_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATED__SHIFT 0x0
+#define PIPE1_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATION_COMPLETED_MASK 0x10
+#define PIPE1_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATION_COMPLETED__SHIFT 0x4
+#define PIPE2_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATED_MASK 0x7
+#define PIPE2_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATED__SHIFT 0x0
+#define PIPE2_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATION_COMPLETED_MASK 0x10
+#define PIPE2_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATION_COMPLETED__SHIFT 0x4
+#define PIPE3_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATED_MASK 0x7
+#define PIPE3_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATED__SHIFT 0x0
+#define PIPE3_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATION_COMPLETED_MASK 0x10
+#define PIPE3_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATION_COMPLETED__SHIFT 0x4
+#define PIPE4_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATED_MASK 0x7
+#define PIPE4_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATED__SHIFT 0x0
+#define PIPE4_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATION_COMPLETED_MASK 0x10
+#define PIPE4_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATION_COMPLETED__SHIFT 0x4
+#define PIPE5_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATED_MASK 0x7
+#define PIPE5_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATED__SHIFT 0x0
+#define PIPE5_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATION_COMPLETED_MASK 0x10
+#define PIPE5_DMIF_BUFFER_CONTROL__DMIF_BUFFERS_ALLOCATION_COMPLETED__SHIFT 0x4
+#define DC_GENERICA__GENERICA_EN_MASK 0x1
+#define DC_GENERICA__GENERICA_EN__SHIFT 0x0
+#define DC_GENERICA__GENERICA_SEL_MASK 0xf80
+#define DC_GENERICA__GENERICA_SEL__SHIFT 0x7
+#define DC_GENERICA__GENERICA_UNIPHY_REFDIV_CLK_SEL_MASK 0xf000
+#define DC_GENERICA__GENERICA_UNIPHY_REFDIV_CLK_SEL__SHIFT 0xc
+#define DC_GENERICA__GENERICA_UNIPHY_FBDIV_CLK_SEL_MASK 0xf0000
+#define DC_GENERICA__GENERICA_UNIPHY_FBDIV_CLK_SEL__SHIFT 0x10
+#define DC_GENERICA__GENERICA_UNIPHY_FBDIV_SSC_CLK_SEL_MASK 0xf00000
+#define DC_GENERICA__GENERICA_UNIPHY_FBDIV_SSC_CLK_SEL__SHIFT 0x14
+#define DC_GENERICA__GENERICA_UNIPHY_FBDIV_CLK_DIV2_SEL_MASK 0xf000000
+#define DC_GENERICA__GENERICA_UNIPHY_FBDIV_CLK_DIV2_SEL__SHIFT 0x18
+#define DC_GENERICB__GENERICB_EN_MASK 0x1
+#define DC_GENERICB__GENERICB_EN__SHIFT 0x0
+#define DC_GENERICB__GENERICB_SEL_MASK 0xf00
+#define DC_GENERICB__GENERICB_SEL__SHIFT 0x8
+#define DC_GENERICB__GENERICB_UNIPHY_REFDIV_CLK_SEL_MASK 0xf000
+#define DC_GENERICB__GENERICB_UNIPHY_REFDIV_CLK_SEL__SHIFT 0xc
+#define DC_GENERICB__GENERICB_UNIPHY_FBDIV_CLK_SEL_MASK 0xf0000
+#define DC_GENERICB__GENERICB_UNIPHY_FBDIV_CLK_SEL__SHIFT 0x10
+#define DC_GENERICB__GENERICB_UNIPHY_FBDIV_SSC_CLK_SEL_MASK 0xf00000
+#define DC_GENERICB__GENERICB_UNIPHY_FBDIV_SSC_CLK_SEL__SHIFT 0x14
+#define DC_GENERICB__GENERICB_UNIPHY_FBDIV_CLK_DIV2_SEL_MASK 0xf000000
+#define DC_GENERICB__GENERICB_UNIPHY_FBDIV_CLK_DIV2_SEL__SHIFT 0x18
+#define DC_PAD_EXTERN_SIG__DC_PAD_EXTERN_SIG_SEL_MASK 0xf
+#define DC_PAD_EXTERN_SIG__DC_PAD_EXTERN_SIG_SEL__SHIFT 0x0
+#define DC_PAD_EXTERN_SIG__MVP_PIXEL_SRC_STATUS_MASK 0x30
+#define DC_PAD_EXTERN_SIG__MVP_PIXEL_SRC_STATUS__SHIFT 0x4
+#define DC_REF_CLK_CNTL__HSYNCA_OUTPUT_SEL_MASK 0x3
+#define DC_REF_CLK_CNTL__HSYNCA_OUTPUT_SEL__SHIFT 0x0
+#define DC_REF_CLK_CNTL__GENLK_CLK_OUTPUT_SEL_MASK 0x300
+#define DC_REF_CLK_CNTL__GENLK_CLK_OUTPUT_SEL__SHIFT 0x8
+#define DC_GPIO_DEBUG__DC_GPIO_VIP_DEBUG_MASK 0x1
+#define DC_GPIO_DEBUG__DC_GPIO_VIP_DEBUG__SHIFT 0x0
+#define DC_GPIO_DEBUG__DC_GPIO_MACRO_DEBUG_MASK 0x300
+#define DC_GPIO_DEBUG__DC_GPIO_MACRO_DEBUG__SHIFT 0x8
+#define DC_GPIO_DEBUG__DC_GPIO_CHIP_DEBUG_OUT_PIN_SEL_MASK 0x10000
+#define DC_GPIO_DEBUG__DC_GPIO_CHIP_DEBUG_OUT_PIN_SEL__SHIFT 0x10
+#define DC_GPIO_DEBUG__DC_GPIO_DEBUG_BUS_FLOP_EN_MASK 0x20000
+#define DC_GPIO_DEBUG__DC_GPIO_DEBUG_BUS_FLOP_EN__SHIFT 0x11
+#define DC_GPIO_DEBUG__DPRX_LOOPBACK_ENABLE_MASK 0x80000000
+#define DC_GPIO_DEBUG__DPRX_LOOPBACK_ENABLE__SHIFT 0x1f
+#define UNIPHYA_LINK_CNTL__UNIPHY_PFREQCHG_MASK 0x1
+#define UNIPHYA_LINK_CNTL__UNIPHY_PFREQCHG__SHIFT 0x0
+#define UNIPHYA_LINK_CNTL__UNIPHY_PIXVLD_RESET_MASK 0x10
+#define UNIPHYA_LINK_CNTL__UNIPHY_PIXVLD_RESET__SHIFT 0x4
+#define UNIPHYA_LINK_CNTL__UNIPHY_MINIMUM_PIXVLD_LOW_DURATION_MASK 0x700
+#define UNIPHYA_LINK_CNTL__UNIPHY_MINIMUM_PIXVLD_LOW_DURATION__SHIFT 0x8
+#define UNIPHYA_LINK_CNTL__UNIPHY_CHANNEL0_INVERT_MASK 0x1000
+#define UNIPHYA_LINK_CNTL__UNIPHY_CHANNEL0_INVERT__SHIFT 0xc
+#define UNIPHYA_LINK_CNTL__UNIPHY_CHANNEL1_INVERT_MASK 0x2000
+#define UNIPHYA_LINK_CNTL__UNIPHY_CHANNEL1_INVERT__SHIFT 0xd
+#define UNIPHYA_LINK_CNTL__UNIPHY_CHANNEL2_INVERT_MASK 0x4000
+#define UNIPHYA_LINK_CNTL__UNIPHY_CHANNEL2_INVERT__SHIFT 0xe
+#define UNIPHYA_LINK_CNTL__UNIPHY_CHANNEL3_INVERT_MASK 0x8000
+#define UNIPHYA_LINK_CNTL__UNIPHY_CHANNEL3_INVERT__SHIFT 0xf
+#define UNIPHYA_LINK_CNTL__UNIPHY_LANE_STAGGER_DELAY_MASK 0x700000
+#define UNIPHYA_LINK_CNTL__UNIPHY_LANE_STAGGER_DELAY__SHIFT 0x14
+#define UNIPHYA_LINK_CNTL__UNIPHY_LINK_ENABLE_HPD_MASK_MASK 0x3000000
+#define UNIPHYA_LINK_CNTL__UNIPHY_LINK_ENABLE_HPD_MASK__SHIFT 0x18
+#define UNIPHYB_LINK_CNTL__UNIPHY_PFREQCHG_MASK 0x1
+#define UNIPHYB_LINK_CNTL__UNIPHY_PFREQCHG__SHIFT 0x0
+#define UNIPHYB_LINK_CNTL__UNIPHY_PIXVLD_RESET_MASK 0x10
+#define UNIPHYB_LINK_CNTL__UNIPHY_PIXVLD_RESET__SHIFT 0x4
+#define UNIPHYB_LINK_CNTL__UNIPHY_MINIMUM_PIXVLD_LOW_DURATION_MASK 0x700
+#define UNIPHYB_LINK_CNTL__UNIPHY_MINIMUM_PIXVLD_LOW_DURATION__SHIFT 0x8
+#define UNIPHYB_LINK_CNTL__UNIPHY_CHANNEL0_INVERT_MASK 0x1000
+#define UNIPHYB_LINK_CNTL__UNIPHY_CHANNEL0_INVERT__SHIFT 0xc
+#define UNIPHYB_LINK_CNTL__UNIPHY_CHANNEL1_INVERT_MASK 0x2000
+#define UNIPHYB_LINK_CNTL__UNIPHY_CHANNEL1_INVERT__SHIFT 0xd
+#define UNIPHYB_LINK_CNTL__UNIPHY_CHANNEL2_INVERT_MASK 0x4000
+#define UNIPHYB_LINK_CNTL__UNIPHY_CHANNEL2_INVERT__SHIFT 0xe
+#define UNIPHYB_LINK_CNTL__UNIPHY_CHANNEL3_INVERT_MASK 0x8000
+#define UNIPHYB_LINK_CNTL__UNIPHY_CHANNEL3_INVERT__SHIFT 0xf
+#define UNIPHYB_LINK_CNTL__UNIPHY_LANE_STAGGER_DELAY_MASK 0x700000
+#define UNIPHYB_LINK_CNTL__UNIPHY_LANE_STAGGER_DELAY__SHIFT 0x14
+#define UNIPHYB_LINK_CNTL__UNIPHY_LINK_ENABLE_HPD_MASK_MASK 0x3000000
+#define UNIPHYB_LINK_CNTL__UNIPHY_LINK_ENABLE_HPD_MASK__SHIFT 0x18
+#define UNIPHYC_LINK_CNTL__UNIPHY_PFREQCHG_MASK 0x1
+#define UNIPHYC_LINK_CNTL__UNIPHY_PFREQCHG__SHIFT 0x0
+#define UNIPHYC_LINK_CNTL__UNIPHY_PIXVLD_RESET_MASK 0x10
+#define UNIPHYC_LINK_CNTL__UNIPHY_PIXVLD_RESET__SHIFT 0x4
+#define UNIPHYC_LINK_CNTL__UNIPHY_MINIMUM_PIXVLD_LOW_DURATION_MASK 0x700
+#define UNIPHYC_LINK_CNTL__UNIPHY_MINIMUM_PIXVLD_LOW_DURATION__SHIFT 0x8
+#define UNIPHYC_LINK_CNTL__UNIPHY_CHANNEL0_INVERT_MASK 0x1000
+#define UNIPHYC_LINK_CNTL__UNIPHY_CHANNEL0_INVERT__SHIFT 0xc
+#define UNIPHYC_LINK_CNTL__UNIPHY_CHANNEL1_INVERT_MASK 0x2000
+#define UNIPHYC_LINK_CNTL__UNIPHY_CHANNEL1_INVERT__SHIFT 0xd
+#define UNIPHYC_LINK_CNTL__UNIPHY_CHANNEL2_INVERT_MASK 0x4000
+#define UNIPHYC_LINK_CNTL__UNIPHY_CHANNEL2_INVERT__SHIFT 0xe
+#define UNIPHYC_LINK_CNTL__UNIPHY_CHANNEL3_INVERT_MASK 0x8000
+#define UNIPHYC_LINK_CNTL__UNIPHY_CHANNEL3_INVERT__SHIFT 0xf
+#define UNIPHYC_LINK_CNTL__UNIPHY_LANE_STAGGER_DELAY_MASK 0x700000
+#define UNIPHYC_LINK_CNTL__UNIPHY_LANE_STAGGER_DELAY__SHIFT 0x14
+#define UNIPHYC_LINK_CNTL__UNIPHY_LINK_ENABLE_HPD_MASK_MASK 0x3000000
+#define UNIPHYC_LINK_CNTL__UNIPHY_LINK_ENABLE_HPD_MASK__SHIFT 0x18
+#define UNIPHYD_LINK_CNTL__UNIPHY_PFREQCHG_MASK 0x1
+#define UNIPHYD_LINK_CNTL__UNIPHY_PFREQCHG__SHIFT 0x0
+#define UNIPHYD_LINK_CNTL__UNIPHY_PIXVLD_RESET_MASK 0x10
+#define UNIPHYD_LINK_CNTL__UNIPHY_PIXVLD_RESET__SHIFT 0x4
+#define UNIPHYD_LINK_CNTL__UNIPHY_MINIMUM_PIXVLD_LOW_DURATION_MASK 0x700
+#define UNIPHYD_LINK_CNTL__UNIPHY_MINIMUM_PIXVLD_LOW_DURATION__SHIFT 0x8
+#define UNIPHYD_LINK_CNTL__UNIPHY_CHANNEL0_INVERT_MASK 0x1000
+#define UNIPHYD_LINK_CNTL__UNIPHY_CHANNEL0_INVERT__SHIFT 0xc
+#define UNIPHYD_LINK_CNTL__UNIPHY_CHANNEL1_INVERT_MASK 0x2000
+#define UNIPHYD_LINK_CNTL__UNIPHY_CHANNEL1_INVERT__SHIFT 0xd
+#define UNIPHYD_LINK_CNTL__UNIPHY_CHANNEL2_INVERT_MASK 0x4000
+#define UNIPHYD_LINK_CNTL__UNIPHY_CHANNEL2_INVERT__SHIFT 0xe
+#define UNIPHYD_LINK_CNTL__UNIPHY_CHANNEL3_INVERT_MASK 0x8000
+#define UNIPHYD_LINK_CNTL__UNIPHY_CHANNEL3_INVERT__SHIFT 0xf
+#define UNIPHYD_LINK_CNTL__UNIPHY_LANE_STAGGER_DELAY_MASK 0x700000
+#define UNIPHYD_LINK_CNTL__UNIPHY_LANE_STAGGER_DELAY__SHIFT 0x14
+#define UNIPHYD_LINK_CNTL__UNIPHY_LINK_ENABLE_HPD_MASK_MASK 0x3000000
+#define UNIPHYD_LINK_CNTL__UNIPHY_LINK_ENABLE_HPD_MASK__SHIFT 0x18
+#define UNIPHYE_LINK_CNTL__UNIPHY_PFREQCHG_MASK 0x1
+#define UNIPHYE_LINK_CNTL__UNIPHY_PFREQCHG__SHIFT 0x0
+#define UNIPHYE_LINK_CNTL__UNIPHY_PIXVLD_RESET_MASK 0x10
+#define UNIPHYE_LINK_CNTL__UNIPHY_PIXVLD_RESET__SHIFT 0x4
+#define UNIPHYE_LINK_CNTL__UNIPHY_MINIMUM_PIXVLD_LOW_DURATION_MASK 0x700
+#define UNIPHYE_LINK_CNTL__UNIPHY_MINIMUM_PIXVLD_LOW_DURATION__SHIFT 0x8
+#define UNIPHYE_LINK_CNTL__UNIPHY_CHANNEL0_INVERT_MASK 0x1000
+#define UNIPHYE_LINK_CNTL__UNIPHY_CHANNEL0_INVERT__SHIFT 0xc
+#define UNIPHYE_LINK_CNTL__UNIPHY_CHANNEL1_INVERT_MASK 0x2000
+#define UNIPHYE_LINK_CNTL__UNIPHY_CHANNEL1_INVERT__SHIFT 0xd
+#define UNIPHYE_LINK_CNTL__UNIPHY_CHANNEL2_INVERT_MASK 0x4000
+#define UNIPHYE_LINK_CNTL__UNIPHY_CHANNEL2_INVERT__SHIFT 0xe
+#define UNIPHYE_LINK_CNTL__UNIPHY_CHANNEL3_INVERT_MASK 0x8000
+#define UNIPHYE_LINK_CNTL__UNIPHY_CHANNEL3_INVERT__SHIFT 0xf
+#define UNIPHYE_LINK_CNTL__UNIPHY_LANE_STAGGER_DELAY_MASK 0x700000
+#define UNIPHYE_LINK_CNTL__UNIPHY_LANE_STAGGER_DELAY__SHIFT 0x14
+#define UNIPHYE_LINK_CNTL__UNIPHY_LINK_ENABLE_HPD_MASK_MASK 0x3000000
+#define UNIPHYE_LINK_CNTL__UNIPHY_LINK_ENABLE_HPD_MASK__SHIFT 0x18
+#define UNIPHYF_LINK_CNTL__UNIPHY_PFREQCHG_MASK 0x1
+#define UNIPHYF_LINK_CNTL__UNIPHY_PFREQCHG__SHIFT 0x0
+#define UNIPHYF_LINK_CNTL__UNIPHY_PIXVLD_RESET_MASK 0x10
+#define UNIPHYF_LINK_CNTL__UNIPHY_PIXVLD_RESET__SHIFT 0x4
+#define UNIPHYF_LINK_CNTL__UNIPHY_MINIMUM_PIXVLD_LOW_DURATION_MASK 0x700
+#define UNIPHYF_LINK_CNTL__UNIPHY_MINIMUM_PIXVLD_LOW_DURATION__SHIFT 0x8
+#define UNIPHYF_LINK_CNTL__UNIPHY_CHANNEL0_INVERT_MASK 0x1000
+#define UNIPHYF_LINK_CNTL__UNIPHY_CHANNEL0_INVERT__SHIFT 0xc
+#define UNIPHYF_LINK_CNTL__UNIPHY_CHANNEL1_INVERT_MASK 0x2000
+#define UNIPHYF_LINK_CNTL__UNIPHY_CHANNEL1_INVERT__SHIFT 0xd
+#define UNIPHYF_LINK_CNTL__UNIPHY_CHANNEL2_INVERT_MASK 0x4000
+#define UNIPHYF_LINK_CNTL__UNIPHY_CHANNEL2_INVERT__SHIFT 0xe
+#define UNIPHYF_LINK_CNTL__UNIPHY_CHANNEL3_INVERT_MASK 0x8000
+#define UNIPHYF_LINK_CNTL__UNIPHY_CHANNEL3_INVERT__SHIFT 0xf
+#define UNIPHYF_LINK_CNTL__UNIPHY_LANE_STAGGER_DELAY_MASK 0x700000
+#define UNIPHYF_LINK_CNTL__UNIPHY_LANE_STAGGER_DELAY__SHIFT 0x14
+#define UNIPHYF_LINK_CNTL__UNIPHY_LINK_ENABLE_HPD_MASK_MASK 0x3000000
+#define UNIPHYF_LINK_CNTL__UNIPHY_LINK_ENABLE_HPD_MASK__SHIFT 0x18
+#define UNIPHYG_LINK_CNTL__UNIPHY_PFREQCHG_MASK 0x1
+#define UNIPHYG_LINK_CNTL__UNIPHY_PFREQCHG__SHIFT 0x0
+#define UNIPHYG_LINK_CNTL__UNIPHY_PIXVLD_RESET_MASK 0x10
+#define UNIPHYG_LINK_CNTL__UNIPHY_PIXVLD_RESET__SHIFT 0x4
+#define UNIPHYG_LINK_CNTL__UNIPHY_MINIMUM_PIXVLD_LOW_DURATION_MASK 0x700
+#define UNIPHYG_LINK_CNTL__UNIPHY_MINIMUM_PIXVLD_LOW_DURATION__SHIFT 0x8
+#define UNIPHYG_LINK_CNTL__UNIPHY_CHANNEL0_INVERT_MASK 0x1000
+#define UNIPHYG_LINK_CNTL__UNIPHY_CHANNEL0_INVERT__SHIFT 0xc
+#define UNIPHYG_LINK_CNTL__UNIPHY_CHANNEL1_INVERT_MASK 0x2000
+#define UNIPHYG_LINK_CNTL__UNIPHY_CHANNEL1_INVERT__SHIFT 0xd
+#define UNIPHYG_LINK_CNTL__UNIPHY_CHANNEL2_INVERT_MASK 0x4000
+#define UNIPHYG_LINK_CNTL__UNIPHY_CHANNEL2_INVERT__SHIFT 0xe
+#define UNIPHYG_LINK_CNTL__UNIPHY_CHANNEL3_INVERT_MASK 0x8000
+#define UNIPHYG_LINK_CNTL__UNIPHY_CHANNEL3_INVERT__SHIFT 0xf
+#define UNIPHYG_LINK_CNTL__UNIPHY_LANE_STAGGER_DELAY_MASK 0x700000
+#define UNIPHYG_LINK_CNTL__UNIPHY_LANE_STAGGER_DELAY__SHIFT 0x14
+#define UNIPHYG_LINK_CNTL__UNIPHY_LINK_ENABLE_HPD_MASK_MASK 0x3000000
+#define UNIPHYG_LINK_CNTL__UNIPHY_LINK_ENABLE_HPD_MASK__SHIFT 0x18
+#define UNIPHYA_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL0_XBAR_SOURCE_MASK 0x3
+#define UNIPHYA_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL0_XBAR_SOURCE__SHIFT 0x0
+#define UNIPHYA_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL1_XBAR_SOURCE_MASK 0x300
+#define UNIPHYA_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL1_XBAR_SOURCE__SHIFT 0x8
+#define UNIPHYA_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL2_XBAR_SOURCE_MASK 0x30000
+#define UNIPHYA_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL2_XBAR_SOURCE__SHIFT 0x10
+#define UNIPHYA_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL3_XBAR_SOURCE_MASK 0x3000000
+#define UNIPHYA_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL3_XBAR_SOURCE__SHIFT 0x18
+#define UNIPHYA_CHANNEL_XBAR_CNTL__UNIPHY_LINK_ENABLE_MASK 0x10000000
+#define UNIPHYA_CHANNEL_XBAR_CNTL__UNIPHY_LINK_ENABLE__SHIFT 0x1c
+#define UNIPHYB_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL0_XBAR_SOURCE_MASK 0x3
+#define UNIPHYB_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL0_XBAR_SOURCE__SHIFT 0x0
+#define UNIPHYB_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL1_XBAR_SOURCE_MASK 0x300
+#define UNIPHYB_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL1_XBAR_SOURCE__SHIFT 0x8
+#define UNIPHYB_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL2_XBAR_SOURCE_MASK 0x30000
+#define UNIPHYB_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL2_XBAR_SOURCE__SHIFT 0x10
+#define UNIPHYB_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL3_XBAR_SOURCE_MASK 0x3000000
+#define UNIPHYB_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL3_XBAR_SOURCE__SHIFT 0x18
+#define UNIPHYB_CHANNEL_XBAR_CNTL__UNIPHY_LINK_ENABLE_MASK 0x10000000
+#define UNIPHYB_CHANNEL_XBAR_CNTL__UNIPHY_LINK_ENABLE__SHIFT 0x1c
+#define UNIPHYC_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL0_XBAR_SOURCE_MASK 0x3
+#define UNIPHYC_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL0_XBAR_SOURCE__SHIFT 0x0
+#define UNIPHYC_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL1_XBAR_SOURCE_MASK 0x300
+#define UNIPHYC_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL1_XBAR_SOURCE__SHIFT 0x8
+#define UNIPHYC_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL2_XBAR_SOURCE_MASK 0x30000
+#define UNIPHYC_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL2_XBAR_SOURCE__SHIFT 0x10
+#define UNIPHYC_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL3_XBAR_SOURCE_MASK 0x3000000
+#define UNIPHYC_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL3_XBAR_SOURCE__SHIFT 0x18
+#define UNIPHYC_CHANNEL_XBAR_CNTL__UNIPHY_LINK_ENABLE_MASK 0x10000000
+#define UNIPHYC_CHANNEL_XBAR_CNTL__UNIPHY_LINK_ENABLE__SHIFT 0x1c
+#define UNIPHYD_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL0_XBAR_SOURCE_MASK 0x3
+#define UNIPHYD_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL0_XBAR_SOURCE__SHIFT 0x0
+#define UNIPHYD_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL1_XBAR_SOURCE_MASK 0x300
+#define UNIPHYD_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL1_XBAR_SOURCE__SHIFT 0x8
+#define UNIPHYD_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL2_XBAR_SOURCE_MASK 0x30000
+#define UNIPHYD_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL2_XBAR_SOURCE__SHIFT 0x10
+#define UNIPHYD_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL3_XBAR_SOURCE_MASK 0x3000000
+#define UNIPHYD_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL3_XBAR_SOURCE__SHIFT 0x18
+#define UNIPHYD_CHANNEL_XBAR_CNTL__UNIPHY_LINK_ENABLE_MASK 0x10000000
+#define UNIPHYD_CHANNEL_XBAR_CNTL__UNIPHY_LINK_ENABLE__SHIFT 0x1c
+#define UNIPHYE_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL0_XBAR_SOURCE_MASK 0x3
+#define UNIPHYE_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL0_XBAR_SOURCE__SHIFT 0x0
+#define UNIPHYE_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL1_XBAR_SOURCE_MASK 0x300
+#define UNIPHYE_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL1_XBAR_SOURCE__SHIFT 0x8
+#define UNIPHYE_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL2_XBAR_SOURCE_MASK 0x30000
+#define UNIPHYE_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL2_XBAR_SOURCE__SHIFT 0x10
+#define UNIPHYE_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL3_XBAR_SOURCE_MASK 0x3000000
+#define UNIPHYE_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL3_XBAR_SOURCE__SHIFT 0x18
+#define UNIPHYE_CHANNEL_XBAR_CNTL__UNIPHY_LINK_ENABLE_MASK 0x10000000
+#define UNIPHYE_CHANNEL_XBAR_CNTL__UNIPHY_LINK_ENABLE__SHIFT 0x1c
+#define UNIPHYF_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL0_XBAR_SOURCE_MASK 0x3
+#define UNIPHYF_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL0_XBAR_SOURCE__SHIFT 0x0
+#define UNIPHYF_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL1_XBAR_SOURCE_MASK 0x300
+#define UNIPHYF_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL1_XBAR_SOURCE__SHIFT 0x8
+#define UNIPHYF_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL2_XBAR_SOURCE_MASK 0x30000
+#define UNIPHYF_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL2_XBAR_SOURCE__SHIFT 0x10
+#define UNIPHYF_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL3_XBAR_SOURCE_MASK 0x3000000
+#define UNIPHYF_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL3_XBAR_SOURCE__SHIFT 0x18
+#define UNIPHYF_CHANNEL_XBAR_CNTL__UNIPHY_LINK_ENABLE_MASK 0x10000000
+#define UNIPHYF_CHANNEL_XBAR_CNTL__UNIPHY_LINK_ENABLE__SHIFT 0x1c
+#define UNIPHYG_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL0_XBAR_SOURCE_MASK 0x3
+#define UNIPHYG_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL0_XBAR_SOURCE__SHIFT 0x0
+#define UNIPHYG_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL1_XBAR_SOURCE_MASK 0x300
+#define UNIPHYG_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL1_XBAR_SOURCE__SHIFT 0x8
+#define UNIPHYG_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL2_XBAR_SOURCE_MASK 0x30000
+#define UNIPHYG_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL2_XBAR_SOURCE__SHIFT 0x10
+#define UNIPHYG_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL3_XBAR_SOURCE_MASK 0x3000000
+#define UNIPHYG_CHANNEL_XBAR_CNTL__UNIPHY_CHANNEL3_XBAR_SOURCE__SHIFT 0x18
+#define UNIPHYG_CHANNEL_XBAR_CNTL__UNIPHY_LINK_ENABLE_MASK 0x10000000
+#define UNIPHYG_CHANNEL_XBAR_CNTL__UNIPHY_LINK_ENABLE__SHIFT 0x1c
+#define UNIPHYLPA_LINK_CNTL__UNIPHYLP_PFREQCHG_MASK 0x1
+#define UNIPHYLPA_LINK_CNTL__UNIPHYLP_PFREQCHG__SHIFT 0x0
+#define UNIPHYLPA_LINK_CNTL__UNIPHYLP_PIXVLD_RESET_MASK 0x10
+#define UNIPHYLPA_LINK_CNTL__UNIPHYLP_PIXVLD_RESET__SHIFT 0x4
+#define UNIPHYLPA_LINK_CNTL__UNIPHYLP_MINIMUM_PIXVLD_LOW_DURATION_MASK 0x700
+#define UNIPHYLPA_LINK_CNTL__UNIPHYLP_MINIMUM_PIXVLD_LOW_DURATION__SHIFT 0x8
+#define UNIPHYLPA_LINK_CNTL__UNIPHYLP_CHANNEL0_INVERT_MASK 0x1000
+#define UNIPHYLPA_LINK_CNTL__UNIPHYLP_CHANNEL0_INVERT__SHIFT 0xc
+#define UNIPHYLPA_LINK_CNTL__UNIPHYLP_CHANNEL1_INVERT_MASK 0x2000
+#define UNIPHYLPA_LINK_CNTL__UNIPHYLP_CHANNEL1_INVERT__SHIFT 0xd
+#define UNIPHYLPA_LINK_CNTL__UNIPHYLP_CHANNEL2_INVERT_MASK 0x4000
+#define UNIPHYLPA_LINK_CNTL__UNIPHYLP_CHANNEL2_INVERT__SHIFT 0xe
+#define UNIPHYLPA_LINK_CNTL__UNIPHYLP_CHANNEL3_INVERT_MASK 0x8000
+#define UNIPHYLPA_LINK_CNTL__UNIPHYLP_CHANNEL3_INVERT__SHIFT 0xf
+#define UNIPHYLPA_LINK_CNTL__UNIPHYLP_LANE_STAGGER_DELAY_MASK 0x700000
+#define UNIPHYLPA_LINK_CNTL__UNIPHYLP_LANE_STAGGER_DELAY__SHIFT 0x14
+#define UNIPHYLPA_LINK_CNTL__UNIPHYLP_LINK_ENABLE_HPD_MASK_MASK 0x3000000
+#define UNIPHYLPA_LINK_CNTL__UNIPHYLP_LINK_ENABLE_HPD_MASK__SHIFT 0x18
+#define UNIPHYLPB_LINK_CNTL__UNIPHYLP_PFREQCHG_MASK 0x1
+#define UNIPHYLPB_LINK_CNTL__UNIPHYLP_PFREQCHG__SHIFT 0x0
+#define UNIPHYLPB_LINK_CNTL__UNIPHYLP_PIXVLD_RESET_MASK 0x10
+#define UNIPHYLPB_LINK_CNTL__UNIPHYLP_PIXVLD_RESET__SHIFT 0x4
+#define UNIPHYLPB_LINK_CNTL__UNIPHYLP_MINIMUM_PIXVLD_LOW_DURATION_MASK 0x700
+#define UNIPHYLPB_LINK_CNTL__UNIPHYLP_MINIMUM_PIXVLD_LOW_DURATION__SHIFT 0x8
+#define UNIPHYLPB_LINK_CNTL__UNIPHYLP_CHANNEL0_INVERT_MASK 0x1000
+#define UNIPHYLPB_LINK_CNTL__UNIPHYLP_CHANNEL0_INVERT__SHIFT 0xc
+#define UNIPHYLPB_LINK_CNTL__UNIPHYLP_CHANNEL1_INVERT_MASK 0x2000
+#define UNIPHYLPB_LINK_CNTL__UNIPHYLP_CHANNEL1_INVERT__SHIFT 0xd
+#define UNIPHYLPB_LINK_CNTL__UNIPHYLP_CHANNEL2_INVERT_MASK 0x4000
+#define UNIPHYLPB_LINK_CNTL__UNIPHYLP_CHANNEL2_INVERT__SHIFT 0xe
+#define UNIPHYLPB_LINK_CNTL__UNIPHYLP_CHANNEL3_INVERT_MASK 0x8000
+#define UNIPHYLPB_LINK_CNTL__UNIPHYLP_CHANNEL3_INVERT__SHIFT 0xf
+#define UNIPHYLPB_LINK_CNTL__UNIPHYLP_LANE_STAGGER_DELAY_MASK 0x700000
+#define UNIPHYLPB_LINK_CNTL__UNIPHYLP_LANE_STAGGER_DELAY__SHIFT 0x14
+#define UNIPHYLPB_LINK_CNTL__UNIPHYLP_LINK_ENABLE_HPD_MASK_MASK 0x3000000
+#define UNIPHYLPB_LINK_CNTL__UNIPHYLP_LINK_ENABLE_HPD_MASK__SHIFT 0x18
+#define UNIPHYLPA_CHANNEL_XBAR_CNTL__UNIPHYLP_CHANNEL0_XBAR_SOURCE_MASK 0x3
+#define UNIPHYLPA_CHANNEL_XBAR_CNTL__UNIPHYLP_CHANNEL0_XBAR_SOURCE__SHIFT 0x0
+#define UNIPHYLPA_CHANNEL_XBAR_CNTL__UNIPHYLP_CHANNEL1_XBAR_SOURCE_MASK 0x300
+#define UNIPHYLPA_CHANNEL_XBAR_CNTL__UNIPHYLP_CHANNEL1_XBAR_SOURCE__SHIFT 0x8
+#define UNIPHYLPA_CHANNEL_XBAR_CNTL__UNIPHYLP_CHANNEL2_XBAR_SOURCE_MASK 0x30000
+#define UNIPHYLPA_CHANNEL_XBAR_CNTL__UNIPHYLP_CHANNEL2_XBAR_SOURCE__SHIFT 0x10
+#define UNIPHYLPA_CHANNEL_XBAR_CNTL__UNIPHYLP_CHANNEL3_XBAR_SOURCE_MASK 0x3000000
+#define UNIPHYLPA_CHANNEL_XBAR_CNTL__UNIPHYLP_CHANNEL3_XBAR_SOURCE__SHIFT 0x18
+#define UNIPHYLPA_CHANNEL_XBAR_CNTL__UNIPHYLP_LINK_ENABLE_MASK 0x10000000
+#define UNIPHYLPA_CHANNEL_XBAR_CNTL__UNIPHYLP_LINK_ENABLE__SHIFT 0x1c
+#define UNIPHYLPB_CHANNEL_XBAR_CNTL__UNIPHYLP_CHANNEL0_XBAR_SOURCE_MASK 0x3
+#define UNIPHYLPB_CHANNEL_XBAR_CNTL__UNIPHYLP_CHANNEL0_XBAR_SOURCE__SHIFT 0x0
+#define UNIPHYLPB_CHANNEL_XBAR_CNTL__UNIPHYLP_CHANNEL1_XBAR_SOURCE_MASK 0x300
+#define UNIPHYLPB_CHANNEL_XBAR_CNTL__UNIPHYLP_CHANNEL1_XBAR_SOURCE__SHIFT 0x8
+#define UNIPHYLPB_CHANNEL_XBAR_CNTL__UNIPHYLP_CHANNEL2_XBAR_SOURCE_MASK 0x30000
+#define UNIPHYLPB_CHANNEL_XBAR_CNTL__UNIPHYLP_CHANNEL2_XBAR_SOURCE__SHIFT 0x10
+#define UNIPHYLPB_CHANNEL_XBAR_CNTL__UNIPHYLP_CHANNEL3_XBAR_SOURCE_MASK 0x3000000
+#define UNIPHYLPB_CHANNEL_XBAR_CNTL__UNIPHYLP_CHANNEL3_XBAR_SOURCE__SHIFT 0x18
+#define UNIPHYLPB_CHANNEL_XBAR_CNTL__UNIPHYLP_LINK_ENABLE_MASK 0x10000000
+#define UNIPHYLPB_CHANNEL_XBAR_CNTL__UNIPHYLP_LINK_ENABLE__SHIFT 0x1c
+#define UNIPHY_IMPCAL_LINKA__UNIPHY_IMPCAL_ENABLE_LINKA_MASK 0x1
+#define UNIPHY_IMPCAL_LINKA__UNIPHY_IMPCAL_ENABLE_LINKA__SHIFT 0x0
+#define UNIPHY_IMPCAL_LINKA__UNIPHY_IMPCAL_CALOUT_LINKA_MASK 0x100
+#define UNIPHY_IMPCAL_LINKA__UNIPHY_IMPCAL_CALOUT_LINKA__SHIFT 0x8
+#define UNIPHY_IMPCAL_LINKA__UNIPHY_CALOUT_ERROR_LINKA_MASK 0x200
+#define UNIPHY_IMPCAL_LINKA__UNIPHY_CALOUT_ERROR_LINKA__SHIFT 0x9
+#define UNIPHY_IMPCAL_LINKA__UNIPHY_CALOUT_ERROR_LINKA_AK_MASK 0x400
+#define UNIPHY_IMPCAL_LINKA__UNIPHY_CALOUT_ERROR_LINKA_AK__SHIFT 0xa
+#define UNIPHY_IMPCAL_LINKA__UNIPHY_IMPCAL_VALUE_LINKA_MASK 0xf0000
+#define UNIPHY_IMPCAL_LINKA__UNIPHY_IMPCAL_VALUE_LINKA__SHIFT 0x10
+#define UNIPHY_IMPCAL_LINKA__UNIPHY_IMPCAL_STEP_DELAY_LINKA_MASK 0xf00000
+#define UNIPHY_IMPCAL_LINKA__UNIPHY_IMPCAL_STEP_DELAY_LINKA__SHIFT 0x14
+#define UNIPHY_IMPCAL_LINKA__UNIPHY_IMPCAL_OVERRIDE_LINKA_MASK 0xf000000
+#define UNIPHY_IMPCAL_LINKA__UNIPHY_IMPCAL_OVERRIDE_LINKA__SHIFT 0x18
+#define UNIPHY_IMPCAL_LINKA__UNIPHY_IMPCAL_OVERRIDE_ENABLE_LINKA_MASK 0x10000000
+#define UNIPHY_IMPCAL_LINKA__UNIPHY_IMPCAL_OVERRIDE_ENABLE_LINKA__SHIFT 0x1c
+#define UNIPHY_IMPCAL_LINKA__UNIPHY_IMPCAL_SEL_LINKA_MASK 0x40000000
+#define UNIPHY_IMPCAL_LINKA__UNIPHY_IMPCAL_SEL_LINKA__SHIFT 0x1e
+#define UNIPHY_IMPCAL_LINKB__UNIPHY_IMPCAL_ENABLE_LINKB_MASK 0x1
+#define UNIPHY_IMPCAL_LINKB__UNIPHY_IMPCAL_ENABLE_LINKB__SHIFT 0x0
+#define UNIPHY_IMPCAL_LINKB__UNIPHY_IMPCAL_CALOUT_LINKB_MASK 0x100
+#define UNIPHY_IMPCAL_LINKB__UNIPHY_IMPCAL_CALOUT_LINKB__SHIFT 0x8
+#define UNIPHY_IMPCAL_LINKB__UNIPHY_CALOUT_ERROR_LINKB_MASK 0x200
+#define UNIPHY_IMPCAL_LINKB__UNIPHY_CALOUT_ERROR_LINKB__SHIFT 0x9
+#define UNIPHY_IMPCAL_LINKB__UNIPHY_CALOUT_ERROR_LINKB_AK_MASK 0x400
+#define UNIPHY_IMPCAL_LINKB__UNIPHY_CALOUT_ERROR_LINKB_AK__SHIFT 0xa
+#define UNIPHY_IMPCAL_LINKB__UNIPHY_IMPCAL_VALUE_LINKB_MASK 0xf0000
+#define UNIPHY_IMPCAL_LINKB__UNIPHY_IMPCAL_VALUE_LINKB__SHIFT 0x10
+#define UNIPHY_IMPCAL_LINKB__UNIPHY_IMPCAL_STEP_DELAY_LINKB_MASK 0xf00000
+#define UNIPHY_IMPCAL_LINKB__UNIPHY_IMPCAL_STEP_DELAY_LINKB__SHIFT 0x14
+#define UNIPHY_IMPCAL_LINKB__UNIPHY_IMPCAL_OVERRIDE_LINKB_MASK 0xf000000
+#define UNIPHY_IMPCAL_LINKB__UNIPHY_IMPCAL_OVERRIDE_LINKB__SHIFT 0x18
+#define UNIPHY_IMPCAL_LINKB__UNIPHY_IMPCAL_OVERRIDE_ENABLE_LINKB_MASK 0x10000000
+#define UNIPHY_IMPCAL_LINKB__UNIPHY_IMPCAL_OVERRIDE_ENABLE_LINKB__SHIFT 0x1c
+#define UNIPHY_IMPCAL_LINKB__UNIPHY_IMPCAL_SEL_LINKB_MASK 0x40000000
+#define UNIPHY_IMPCAL_LINKB__UNIPHY_IMPCAL_SEL_LINKB__SHIFT 0x1e
+#define UNIPHY_IMPCAL_LINKC__UNIPHY_IMPCAL_ENABLE_LINKC_MASK 0x1
+#define UNIPHY_IMPCAL_LINKC__UNIPHY_IMPCAL_ENABLE_LINKC__SHIFT 0x0
+#define UNIPHY_IMPCAL_LINKC__UNIPHY_IMPCAL_CALOUT_LINKC_MASK 0x100
+#define UNIPHY_IMPCAL_LINKC__UNIPHY_IMPCAL_CALOUT_LINKC__SHIFT 0x8
+#define UNIPHY_IMPCAL_LINKC__UNIPHY_CALOUT_ERROR_LINKC_MASK 0x200
+#define UNIPHY_IMPCAL_LINKC__UNIPHY_CALOUT_ERROR_LINKC__SHIFT 0x9
+#define UNIPHY_IMPCAL_LINKC__UNIPHY_CALOUT_ERROR_LINKC_AK_MASK 0x400
+#define UNIPHY_IMPCAL_LINKC__UNIPHY_CALOUT_ERROR_LINKC_AK__SHIFT 0xa
+#define UNIPHY_IMPCAL_LINKC__UNIPHY_IMPCAL_VALUE_LINKC_MASK 0xf0000
+#define UNIPHY_IMPCAL_LINKC__UNIPHY_IMPCAL_VALUE_LINKC__SHIFT 0x10
+#define UNIPHY_IMPCAL_LINKC__UNIPHY_IMPCAL_STEP_DELAY_LINKC_MASK 0xf00000
+#define UNIPHY_IMPCAL_LINKC__UNIPHY_IMPCAL_STEP_DELAY_LINKC__SHIFT 0x14
+#define UNIPHY_IMPCAL_LINKC__UNIPHY_IMPCAL_OVERRIDE_LINKC_MASK 0xf000000
+#define UNIPHY_IMPCAL_LINKC__UNIPHY_IMPCAL_OVERRIDE_LINKC__SHIFT 0x18
+#define UNIPHY_IMPCAL_LINKC__UNIPHY_IMPCAL_OVERRIDE_ENABLE_LINKC_MASK 0x10000000
+#define UNIPHY_IMPCAL_LINKC__UNIPHY_IMPCAL_OVERRIDE_ENABLE_LINKC__SHIFT 0x1c
+#define UNIPHY_IMPCAL_LINKC__UNIPHY_IMPCAL_SEL_LINKC_MASK 0x40000000
+#define UNIPHY_IMPCAL_LINKC__UNIPHY_IMPCAL_SEL_LINKC__SHIFT 0x1e
+#define UNIPHY_IMPCAL_LINKD__UNIPHY_IMPCAL_ENABLE_LINKD_MASK 0x1
+#define UNIPHY_IMPCAL_LINKD__UNIPHY_IMPCAL_ENABLE_LINKD__SHIFT 0x0
+#define UNIPHY_IMPCAL_LINKD__UNIPHY_IMPCAL_CALOUT_LINKD_MASK 0x100
+#define UNIPHY_IMPCAL_LINKD__UNIPHY_IMPCAL_CALOUT_LINKD__SHIFT 0x8
+#define UNIPHY_IMPCAL_LINKD__UNIPHY_CALOUT_ERROR_LINKD_MASK 0x200
+#define UNIPHY_IMPCAL_LINKD__UNIPHY_CALOUT_ERROR_LINKD__SHIFT 0x9
+#define UNIPHY_IMPCAL_LINKD__UNIPHY_CALOUT_ERROR_LINKD_AK_MASK 0x400
+#define UNIPHY_IMPCAL_LINKD__UNIPHY_CALOUT_ERROR_LINKD_AK__SHIFT 0xa
+#define UNIPHY_IMPCAL_LINKD__UNIPHY_IMPCAL_VALUE_LINKD_MASK 0xf0000
+#define UNIPHY_IMPCAL_LINKD__UNIPHY_IMPCAL_VALUE_LINKD__SHIFT 0x10
+#define UNIPHY_IMPCAL_LINKD__UNIPHY_IMPCAL_STEP_DELAY_LINKD_MASK 0xf00000
+#define UNIPHY_IMPCAL_LINKD__UNIPHY_IMPCAL_STEP_DELAY_LINKD__SHIFT 0x14
+#define UNIPHY_IMPCAL_LINKD__UNIPHY_IMPCAL_OVERRIDE_LINKD_MASK 0xf000000
+#define UNIPHY_IMPCAL_LINKD__UNIPHY_IMPCAL_OVERRIDE_LINKD__SHIFT 0x18
+#define UNIPHY_IMPCAL_LINKD__UNIPHY_IMPCAL_OVERRIDE_ENABLE_LINKD_MASK 0x10000000
+#define UNIPHY_IMPCAL_LINKD__UNIPHY_IMPCAL_OVERRIDE_ENABLE_LINKD__SHIFT 0x1c
+#define UNIPHY_IMPCAL_LINKD__UNIPHY_IMPCAL_SEL_LINKD_MASK 0x40000000
+#define UNIPHY_IMPCAL_LINKD__UNIPHY_IMPCAL_SEL_LINKD__SHIFT 0x1e
+#define UNIPHY_IMPCAL_LINKE__UNIPHY_IMPCAL_ENABLE_LINKE_MASK 0x1
+#define UNIPHY_IMPCAL_LINKE__UNIPHY_IMPCAL_ENABLE_LINKE__SHIFT 0x0
+#define UNIPHY_IMPCAL_LINKE__UNIPHY_IMPCAL_CALOUT_LINKE_MASK 0x100
+#define UNIPHY_IMPCAL_LINKE__UNIPHY_IMPCAL_CALOUT_LINKE__SHIFT 0x8
+#define UNIPHY_IMPCAL_LINKE__UNIPHY_CALOUT_ERROR_LINKE_MASK 0x200
+#define UNIPHY_IMPCAL_LINKE__UNIPHY_CALOUT_ERROR_LINKE__SHIFT 0x9
+#define UNIPHY_IMPCAL_LINKE__UNIPHY_CALOUT_ERROR_LINKE_AK_MASK 0x400
+#define UNIPHY_IMPCAL_LINKE__UNIPHY_CALOUT_ERROR_LINKE_AK__SHIFT 0xa
+#define UNIPHY_IMPCAL_LINKE__UNIPHY_IMPCAL_VALUE_LINKE_MASK 0xf0000
+#define UNIPHY_IMPCAL_LINKE__UNIPHY_IMPCAL_VALUE_LINKE__SHIFT 0x10
+#define UNIPHY_IMPCAL_LINKE__UNIPHY_IMPCAL_STEP_DELAY_LINKE_MASK 0xf00000
+#define UNIPHY_IMPCAL_LINKE__UNIPHY_IMPCAL_STEP_DELAY_LINKE__SHIFT 0x14
+#define UNIPHY_IMPCAL_LINKE__UNIPHY_IMPCAL_OVERRIDE_LINKE_MASK 0xf000000
+#define UNIPHY_IMPCAL_LINKE__UNIPHY_IMPCAL_OVERRIDE_LINKE__SHIFT 0x18
+#define UNIPHY_IMPCAL_LINKE__UNIPHY_IMPCAL_OVERRIDE_ENABLE_LINKE_MASK 0x10000000
+#define UNIPHY_IMPCAL_LINKE__UNIPHY_IMPCAL_OVERRIDE_ENABLE_LINKE__SHIFT 0x1c
+#define UNIPHY_IMPCAL_LINKE__UNIPHY_IMPCAL_SEL_LINKE_MASK 0x40000000
+#define UNIPHY_IMPCAL_LINKE__UNIPHY_IMPCAL_SEL_LINKE__SHIFT 0x1e
+#define UNIPHY_IMPCAL_LINKF__UNIPHY_IMPCAL_ENABLE_LINKF_MASK 0x1
+#define UNIPHY_IMPCAL_LINKF__UNIPHY_IMPCAL_ENABLE_LINKF__SHIFT 0x0
+#define UNIPHY_IMPCAL_LINKF__UNIPHY_IMPCAL_CALOUT_LINKF_MASK 0x100
+#define UNIPHY_IMPCAL_LINKF__UNIPHY_IMPCAL_CALOUT_LINKF__SHIFT 0x8
+#define UNIPHY_IMPCAL_LINKF__UNIPHY_CALOUT_ERROR_LINKF_MASK 0x200
+#define UNIPHY_IMPCAL_LINKF__UNIPHY_CALOUT_ERROR_LINKF__SHIFT 0x9
+#define UNIPHY_IMPCAL_LINKF__UNIPHY_CALOUT_ERROR_LINKF_AK_MASK 0x400
+#define UNIPHY_IMPCAL_LINKF__UNIPHY_CALOUT_ERROR_LINKF_AK__SHIFT 0xa
+#define UNIPHY_IMPCAL_LINKF__UNIPHY_IMPCAL_VALUE_LINKF_MASK 0xf0000
+#define UNIPHY_IMPCAL_LINKF__UNIPHY_IMPCAL_VALUE_LINKF__SHIFT 0x10
+#define UNIPHY_IMPCAL_LINKF__UNIPHY_IMPCAL_STEP_DELAY_LINKF_MASK 0xf00000
+#define UNIPHY_IMPCAL_LINKF__UNIPHY_IMPCAL_STEP_DELAY_LINKF__SHIFT 0x14
+#define UNIPHY_IMPCAL_LINKF__UNIPHY_IMPCAL_OVERRIDE_LINKF_MASK 0xf000000
+#define UNIPHY_IMPCAL_LINKF__UNIPHY_IMPCAL_OVERRIDE_LINKF__SHIFT 0x18
+#define UNIPHY_IMPCAL_LINKF__UNIPHY_IMPCAL_OVERRIDE_ENABLE_LINKF_MASK 0x10000000
+#define UNIPHY_IMPCAL_LINKF__UNIPHY_IMPCAL_OVERRIDE_ENABLE_LINKF__SHIFT 0x1c
+#define UNIPHY_IMPCAL_LINKF__UNIPHY_IMPCAL_SEL_LINKF_MASK 0x40000000
+#define UNIPHY_IMPCAL_LINKF__UNIPHY_IMPCAL_SEL_LINKF__SHIFT 0x1e
+#define UNIPHY_IMPCAL_PERIOD__UNIPHY_IMPCAL_PERIOD_MASK 0xffffffff
+#define UNIPHY_IMPCAL_PERIOD__UNIPHY_IMPCAL_PERIOD__SHIFT 0x0
+#define AUXP_IMPCAL__AUXP_IMPCAL_ENABLE_MASK 0x1
+#define AUXP_IMPCAL__AUXP_IMPCAL_ENABLE__SHIFT 0x0
+#define AUXP_IMPCAL__AUXP_IMPCAL_CALOUT_MASK 0x100
+#define AUXP_IMPCAL__AUXP_IMPCAL_CALOUT__SHIFT 0x8
+#define AUXP_IMPCAL__AUXP_CALOUT_ERROR_MASK 0x200
+#define AUXP_IMPCAL__AUXP_CALOUT_ERROR__SHIFT 0x9
+#define AUXP_IMPCAL__AUXP_CALOUT_ERROR_AK_MASK 0x400
+#define AUXP_IMPCAL__AUXP_CALOUT_ERROR_AK__SHIFT 0xa
+#define AUXP_IMPCAL__AUXP_IMPCAL_VALUE_MASK 0xf0000
+#define AUXP_IMPCAL__AUXP_IMPCAL_VALUE__SHIFT 0x10
+#define AUXP_IMPCAL__AUXP_IMPCAL_STEP_DELAY_MASK 0xf00000
+#define AUXP_IMPCAL__AUXP_IMPCAL_STEP_DELAY__SHIFT 0x14
+#define AUXP_IMPCAL__AUXP_IMPCAL_OVERRIDE_MASK 0xf000000
+#define AUXP_IMPCAL__AUXP_IMPCAL_OVERRIDE__SHIFT 0x18
+#define AUXP_IMPCAL__AUXP_IMPCAL_OVERRIDE_ENABLE_MASK 0x10000000
+#define AUXP_IMPCAL__AUXP_IMPCAL_OVERRIDE_ENABLE__SHIFT 0x1c
+#define AUXN_IMPCAL__AUXN_IMPCAL_ENABLE_MASK 0x1
+#define AUXN_IMPCAL__AUXN_IMPCAL_ENABLE__SHIFT 0x0
+#define AUXN_IMPCAL__AUXN_IMPCAL_CALOUT_MASK 0x100
+#define AUXN_IMPCAL__AUXN_IMPCAL_CALOUT__SHIFT 0x8
+#define AUXN_IMPCAL__AUXN_CALOUT_ERROR_MASK 0x200
+#define AUXN_IMPCAL__AUXN_CALOUT_ERROR__SHIFT 0x9
+#define AUXN_IMPCAL__AUXN_CALOUT_ERROR_AK_MASK 0x400
+#define AUXN_IMPCAL__AUXN_CALOUT_ERROR_AK__SHIFT 0xa
+#define AUXN_IMPCAL__AUXN_IMPCAL_VALUE_MASK 0xf0000
+#define AUXN_IMPCAL__AUXN_IMPCAL_VALUE__SHIFT 0x10
+#define AUXN_IMPCAL__AUXN_IMPCAL_STEP_DELAY_MASK 0xf00000
+#define AUXN_IMPCAL__AUXN_IMPCAL_STEP_DELAY__SHIFT 0x14
+#define AUXN_IMPCAL__AUXN_IMPCAL_OVERRIDE_MASK 0xf000000
+#define AUXN_IMPCAL__AUXN_IMPCAL_OVERRIDE__SHIFT 0x18
+#define AUXN_IMPCAL__AUXN_IMPCAL_OVERRIDE_ENABLE_MASK 0x10000000
+#define AUXN_IMPCAL__AUXN_IMPCAL_OVERRIDE_ENABLE__SHIFT 0x1c
+#define DCIO_IMPCAL_CNTL__CALR_CNTL_OVERRIDE_MASK 0xf
+#define DCIO_IMPCAL_CNTL__CALR_CNTL_OVERRIDE__SHIFT 0x0
+#define DCIO_IMPCAL_CNTL__IMPCAL_SOFT_RESET_MASK 0x20
+#define DCIO_IMPCAL_CNTL__IMPCAL_SOFT_RESET__SHIFT 0x5
+#define DCIO_IMPCAL_CNTL__IMPCAL_STATUS_MASK 0x300
+#define DCIO_IMPCAL_CNTL__IMPCAL_STATUS__SHIFT 0x8
+#define DCIO_IMPCAL_CNTL__IMPCAL_ARB_STATE_MASK 0x7000
+#define DCIO_IMPCAL_CNTL__IMPCAL_ARB_STATE__SHIFT 0xc
+#define DCIO_IMPCAL_CNTL__AUX_IMPCAL_INTERVAL_MASK 0x78000
+#define DCIO_IMPCAL_CNTL__AUX_IMPCAL_INTERVAL__SHIFT 0xf
+#define DCIO_IMPCAL_CNTL__AUX_IMPCAL_BIASENTST_MASK 0x380000
+#define DCIO_IMPCAL_CNTL__AUX_IMPCAL_BIASENTST__SHIFT 0x13
+#define DCIO_IMPCAL_CNTL__AUX_IMPCAL_RESBIASEN_MASK 0x400000
+#define DCIO_IMPCAL_CNTL__AUX_IMPCAL_RESBIASEN__SHIFT 0x16
+#define DCIO_IMPCAL_CNTL__AUX_IMPCAL_SPARE_CONTROL_MASK 0x1800000
+#define DCIO_IMPCAL_CNTL__AUX_IMPCAL_SPARE_CONTROL__SHIFT 0x17
+#define UNIPHY_IMPCAL_PSW_AB__UNIPHY_IMPCAL_PSW_LINKA_MASK 0x7fff
+#define UNIPHY_IMPCAL_PSW_AB__UNIPHY_IMPCAL_PSW_LINKA__SHIFT 0x0
+#define UNIPHY_IMPCAL_PSW_AB__UNIPHY_IMPCAL_PSW_LINKB_MASK 0x7fff0000
+#define UNIPHY_IMPCAL_PSW_AB__UNIPHY_IMPCAL_PSW_LINKB__SHIFT 0x10
+#define DCIO_IMPCAL_CNTL_CD__CALR_CNTL_OVERRIDE_MASK 0xf
+#define DCIO_IMPCAL_CNTL_CD__CALR_CNTL_OVERRIDE__SHIFT 0x0
+#define DCIO_IMPCAL_CNTL_CD__IMPCAL_SOFT_RESET_MASK 0x20
+#define DCIO_IMPCAL_CNTL_CD__IMPCAL_SOFT_RESET__SHIFT 0x5
+#define DCIO_IMPCAL_CNTL_CD__IMPCAL_STATUS_MASK 0x300
+#define DCIO_IMPCAL_CNTL_CD__IMPCAL_STATUS__SHIFT 0x8
+#define DCIO_IMPCAL_CNTL_CD__IMPCAL_ARB_STATE_MASK 0x7000
+#define DCIO_IMPCAL_CNTL_CD__IMPCAL_ARB_STATE__SHIFT 0xc
+#define UNIPHY_IMPCAL_PSW_CD__UNIPHY_IMPCAL_PSW_LINKC_MASK 0x7fff
+#define UNIPHY_IMPCAL_PSW_CD__UNIPHY_IMPCAL_PSW_LINKC__SHIFT 0x0
+#define UNIPHY_IMPCAL_PSW_CD__UNIPHY_IMPCAL_PSW_LINKD_MASK 0x7fff0000
+#define UNIPHY_IMPCAL_PSW_CD__UNIPHY_IMPCAL_PSW_LINKD__SHIFT 0x10
+#define DCIO_IMPCAL_CNTL_EF__CALR_CNTL_OVERRIDE_MASK 0xf
+#define DCIO_IMPCAL_CNTL_EF__CALR_CNTL_OVERRIDE__SHIFT 0x0
+#define DCIO_IMPCAL_CNTL_EF__IMPCAL_SOFT_RESET_MASK 0x20
+#define DCIO_IMPCAL_CNTL_EF__IMPCAL_SOFT_RESET__SHIFT 0x5
+#define DCIO_IMPCAL_CNTL_EF__IMPCAL_STATUS_MASK 0x300
+#define DCIO_IMPCAL_CNTL_EF__IMPCAL_STATUS__SHIFT 0x8
+#define DCIO_IMPCAL_CNTL_EF__IMPCAL_ARB_STATE_MASK 0x7000
+#define DCIO_IMPCAL_CNTL_EF__IMPCAL_ARB_STATE__SHIFT 0xc
+#define UNIPHY_IMPCAL_PSW_EF__UNIPHY_IMPCAL_PSW_LINKE_MASK 0x7fff
+#define UNIPHY_IMPCAL_PSW_EF__UNIPHY_IMPCAL_PSW_LINKE__SHIFT 0x0
+#define UNIPHY_IMPCAL_PSW_EF__UNIPHY_IMPCAL_PSW_LINKF_MASK 0x7fff0000
+#define UNIPHY_IMPCAL_PSW_EF__UNIPHY_IMPCAL_PSW_LINKF__SHIFT 0x10
+#define DCIO_WRCMD_DELAY__UNIPHY_DELAY_MASK 0xf
+#define DCIO_WRCMD_DELAY__UNIPHY_DELAY__SHIFT 0x0
+#define DCIO_WRCMD_DELAY__DAC_DELAY_MASK 0xf0
+#define DCIO_WRCMD_DELAY__DAC_DELAY__SHIFT 0x4
+#define DCIO_WRCMD_DELAY__DPHY_DELAY_MASK 0xf00
+#define DCIO_WRCMD_DELAY__DPHY_DELAY__SHIFT 0x8
+#define DCIO_WRCMD_DELAY__DCRXPHY_DELAY_MASK 0xf000
+#define DCIO_WRCMD_DELAY__DCRXPHY_DELAY__SHIFT 0xc
+#define DCIO_WRCMD_DELAY__ZCAL_DELAY_MASK 0xf0000
+#define DCIO_WRCMD_DELAY__ZCAL_DELAY__SHIFT 0x10
+#define DC_PINSTRAPS__DC_PINSTRAPS_BIF_CEC_DIS_MASK 0x400
+#define DC_PINSTRAPS__DC_PINSTRAPS_BIF_CEC_DIS__SHIFT 0xa
+#define DC_PINSTRAPS__DC_PINSTRAPS_SMS_EN_HARD_MASK 0x2000
+#define DC_PINSTRAPS__DC_PINSTRAPS_SMS_EN_HARD__SHIFT 0xd
+#define DC_PINSTRAPS__DC_PINSTRAPS_AUDIO_MASK 0xc000
+#define DC_PINSTRAPS__DC_PINSTRAPS_AUDIO__SHIFT 0xe
+#define DC_PINSTRAPS__DC_PINSTRAPS_CCBYPASS_MASK 0x10000
+#define DC_PINSTRAPS__DC_PINSTRAPS_CCBYPASS__SHIFT 0x10
+#define DC_PINSTRAPS__DC_PINSTRAPS_CONNECTIVITY_MASK 0xe0000
+#define DC_PINSTRAPS__DC_PINSTRAPS_CONNECTIVITY__SHIFT 0x11
+#define DC_DVODATA_CONFIG__VIP_MUX_EN_MASK 0x80000
+#define DC_DVODATA_CONFIG__VIP_MUX_EN__SHIFT 0x13
+#define DC_DVODATA_CONFIG__VIP_ALTER_MAPPING_EN_MASK 0x100000
+#define DC_DVODATA_CONFIG__VIP_ALTER_MAPPING_EN__SHIFT 0x14
+#define DC_DVODATA_CONFIG__DVO_ALTER_MAPPING_EN_MASK 0x200000
+#define DC_DVODATA_CONFIG__DVO_ALTER_MAPPING_EN__SHIFT 0x15
+#define LVTMA_PWRSEQ_CNTL__LVTMA_PWRSEQ_EN_MASK 0x1
+#define LVTMA_PWRSEQ_CNTL__LVTMA_PWRSEQ_EN__SHIFT 0x0
+#define LVTMA_PWRSEQ_CNTL__LVTMA_PWRSEQ_DISABLE_SYNCEN_CONTROL_OF_TX_EN_MASK 0x2
+#define LVTMA_PWRSEQ_CNTL__LVTMA_PWRSEQ_DISABLE_SYNCEN_CONTROL_OF_TX_EN__SHIFT 0x1
+#define LVTMA_PWRSEQ_CNTL__LVTMA_PWRSEQ_TARGET_STATE_MASK 0x10
+#define LVTMA_PWRSEQ_CNTL__LVTMA_PWRSEQ_TARGET_STATE__SHIFT 0x4
+#define LVTMA_PWRSEQ_CNTL__LVTMA_SYNCEN_MASK 0x100
+#define LVTMA_PWRSEQ_CNTL__LVTMA_SYNCEN__SHIFT 0x8
+#define LVTMA_PWRSEQ_CNTL__LVTMA_SYNCEN_OVRD_MASK 0x200
+#define LVTMA_PWRSEQ_CNTL__LVTMA_SYNCEN_OVRD__SHIFT 0x9
+#define LVTMA_PWRSEQ_CNTL__LVTMA_SYNCEN_POL_MASK 0x400
+#define LVTMA_PWRSEQ_CNTL__LVTMA_SYNCEN_POL__SHIFT 0xa
+#define LVTMA_PWRSEQ_CNTL__LVTMA_DIGON_MASK 0x10000
+#define LVTMA_PWRSEQ_CNTL__LVTMA_DIGON__SHIFT 0x10
+#define LVTMA_PWRSEQ_CNTL__LVTMA_DIGON_OVRD_MASK 0x20000
+#define LVTMA_PWRSEQ_CNTL__LVTMA_DIGON_OVRD__SHIFT 0x11
+#define LVTMA_PWRSEQ_CNTL__LVTMA_DIGON_POL_MASK 0x40000
+#define LVTMA_PWRSEQ_CNTL__LVTMA_DIGON_POL__SHIFT 0x12
+#define LVTMA_PWRSEQ_CNTL__LVTMA_BLON_MASK 0x1000000
+#define LVTMA_PWRSEQ_CNTL__LVTMA_BLON__SHIFT 0x18
+#define LVTMA_PWRSEQ_CNTL__LVTMA_BLON_OVRD_MASK 0x2000000
+#define LVTMA_PWRSEQ_CNTL__LVTMA_BLON_OVRD__SHIFT 0x19
+#define LVTMA_PWRSEQ_CNTL__LVTMA_BLON_POL_MASK 0x4000000
+#define LVTMA_PWRSEQ_CNTL__LVTMA_BLON_POL__SHIFT 0x1a
+#define LVTMA_PWRSEQ_STATE__LVTMA_PWRSEQ_TARGET_STATE_R_MASK 0x1
+#define LVTMA_PWRSEQ_STATE__LVTMA_PWRSEQ_TARGET_STATE_R__SHIFT 0x0
+#define LVTMA_PWRSEQ_STATE__LVTMA_PWRSEQ_DIGON_MASK 0x2
+#define LVTMA_PWRSEQ_STATE__LVTMA_PWRSEQ_DIGON__SHIFT 0x1
+#define LVTMA_PWRSEQ_STATE__LVTMA_PWRSEQ_SYNCEN_MASK 0x4
+#define LVTMA_PWRSEQ_STATE__LVTMA_PWRSEQ_SYNCEN__SHIFT 0x2
+#define LVTMA_PWRSEQ_STATE__LVTMA_PWRSEQ_BLON_MASK 0x8
+#define LVTMA_PWRSEQ_STATE__LVTMA_PWRSEQ_BLON__SHIFT 0x3
+#define LVTMA_PWRSEQ_STATE__LVTMA_PWRSEQ_DONE_MASK 0x10
+#define LVTMA_PWRSEQ_STATE__LVTMA_PWRSEQ_DONE__SHIFT 0x4
+#define LVTMA_PWRSEQ_STATE__LVTMA_PWRSEQ_STATE_MASK 0xf00
+#define LVTMA_PWRSEQ_STATE__LVTMA_PWRSEQ_STATE__SHIFT 0x8
+#define LVTMA_PWRSEQ_REF_DIV__LVTMA_PWRSEQ_REF_DIV_MASK 0xfff
+#define LVTMA_PWRSEQ_REF_DIV__LVTMA_PWRSEQ_REF_DIV__SHIFT 0x0
+#define LVTMA_PWRSEQ_REF_DIV__BL_PWM_REF_DIV_MASK 0xffff0000
+#define LVTMA_PWRSEQ_REF_DIV__BL_PWM_REF_DIV__SHIFT 0x10
+#define LVTMA_PWRSEQ_DELAY1__LVTMA_PWRUP_DELAY1_MASK 0xff
+#define LVTMA_PWRSEQ_DELAY1__LVTMA_PWRUP_DELAY1__SHIFT 0x0
+#define LVTMA_PWRSEQ_DELAY1__LVTMA_PWRUP_DELAY2_MASK 0xff00
+#define LVTMA_PWRSEQ_DELAY1__LVTMA_PWRUP_DELAY2__SHIFT 0x8
+#define LVTMA_PWRSEQ_DELAY1__LVTMA_PWRDN_DELAY1_MASK 0xff0000
+#define LVTMA_PWRSEQ_DELAY1__LVTMA_PWRDN_DELAY1__SHIFT 0x10
+#define LVTMA_PWRSEQ_DELAY1__LVTMA_PWRDN_DELAY2_MASK 0xff000000
+#define LVTMA_PWRSEQ_DELAY1__LVTMA_PWRDN_DELAY2__SHIFT 0x18
+#define LVTMA_PWRSEQ_DELAY2__LVTMA_PWRDN_MIN_LENGTH_MASK 0xff
+#define LVTMA_PWRSEQ_DELAY2__LVTMA_PWRDN_MIN_LENGTH__SHIFT 0x0
+#define LVTMA_PWRSEQ_DELAY2__LVTMA_PWRUP_DELAY3_MASK 0xff00
+#define LVTMA_PWRSEQ_DELAY2__LVTMA_PWRUP_DELAY3__SHIFT 0x8
+#define LVTMA_PWRSEQ_DELAY2__LVTMA_PWRDN_DELAY3_MASK 0xff0000
+#define LVTMA_PWRSEQ_DELAY2__LVTMA_PWRDN_DELAY3__SHIFT 0x10
+#define LVTMA_PWRSEQ_DELAY2__LVTMA_VARY_BL_OVERRIDE_EN_MASK 0x1000000
+#define LVTMA_PWRSEQ_DELAY2__LVTMA_VARY_BL_OVERRIDE_EN__SHIFT 0x18
+#define BL_PWM_CNTL__BL_ACTIVE_INT_FRAC_CNT_MASK 0xffff
+#define BL_PWM_CNTL__BL_ACTIVE_INT_FRAC_CNT__SHIFT 0x0
+#define BL_PWM_CNTL__BL_PWM_FRACTIONAL_EN_MASK 0x40000000
+#define BL_PWM_CNTL__BL_PWM_FRACTIONAL_EN__SHIFT 0x1e
+#define BL_PWM_CNTL__BL_PWM_EN_MASK 0x80000000
+#define BL_PWM_CNTL__BL_PWM_EN__SHIFT 0x1f
+#define BL_PWM_CNTL2__BL_PWM_POST_FRAME_START_DELAY_BEFORE_UPDATE_MASK 0xffff
+#define BL_PWM_CNTL2__BL_PWM_POST_FRAME_START_DELAY_BEFORE_UPDATE__SHIFT 0x0
+#define BL_PWM_CNTL2__DBG_BL_PWM_INPUT_REFCLK_SELECT_MASK 0x30000000
+#define BL_PWM_CNTL2__DBG_BL_PWM_INPUT_REFCLK_SELECT__SHIFT 0x1c
+#define BL_PWM_CNTL2__BL_PWM_OVERRIDE_BL_OUT_ENABLE_MASK 0x40000000
+#define BL_PWM_CNTL2__BL_PWM_OVERRIDE_BL_OUT_ENABLE__SHIFT 0x1e
+#define BL_PWM_CNTL2__BL_PWM_OVERRIDE_LVTMA_PWRSEQ_EN_MASK 0x80000000
+#define BL_PWM_CNTL2__BL_PWM_OVERRIDE_LVTMA_PWRSEQ_EN__SHIFT 0x1f
+#define BL_PWM_PERIOD_CNTL__BL_PWM_PERIOD_MASK 0xffff
+#define BL_PWM_PERIOD_CNTL__BL_PWM_PERIOD__SHIFT 0x0
+#define BL_PWM_PERIOD_CNTL__BL_PWM_PERIOD_BITCNT_MASK 0xf0000
+#define BL_PWM_PERIOD_CNTL__BL_PWM_PERIOD_BITCNT__SHIFT 0x10
+#define BL_PWM_GRP1_REG_LOCK__BL_PWM_GRP1_REG_LOCK_MASK 0x1
+#define BL_PWM_GRP1_REG_LOCK__BL_PWM_GRP1_REG_LOCK__SHIFT 0x0
+#define BL_PWM_GRP1_REG_LOCK__BL_PWM_GRP1_REG_UPDATE_PENDING_MASK 0x100
+#define BL_PWM_GRP1_REG_LOCK__BL_PWM_GRP1_REG_UPDATE_PENDING__SHIFT 0x8
+#define BL_PWM_GRP1_REG_LOCK__BL_PWM_GRP1_UPDATE_AT_FRAME_START_MASK 0x10000
+#define BL_PWM_GRP1_REG_LOCK__BL_PWM_GRP1_UPDATE_AT_FRAME_START__SHIFT 0x10
+#define BL_PWM_GRP1_REG_LOCK__BL_PWM_GRP1_FRAME_START_DISP_SEL_MASK 0xe0000
+#define BL_PWM_GRP1_REG_LOCK__BL_PWM_GRP1_FRAME_START_DISP_SEL__SHIFT 0x11
+#define BL_PWM_GRP1_REG_LOCK__BL_PWM_GRP1_READBACK_DB_REG_VALUE_EN_MASK 0x1000000
+#define BL_PWM_GRP1_REG_LOCK__BL_PWM_GRP1_READBACK_DB_REG_VALUE_EN__SHIFT 0x18
+#define BL_PWM_GRP1_REG_LOCK__BL_PWM_GRP1_IGNORE_MASTER_LOCK_EN_MASK 0x80000000
+#define BL_PWM_GRP1_REG_LOCK__BL_PWM_GRP1_IGNORE_MASTER_LOCK_EN__SHIFT 0x1f
+#define DCIO_GSL_GENLK_PAD_CNTL__DCIO_GENLK_CLK_GSL_TIMING_SYNC_SEL_MASK 0x3
+#define DCIO_GSL_GENLK_PAD_CNTL__DCIO_GENLK_CLK_GSL_TIMING_SYNC_SEL__SHIFT 0x0
+#define DCIO_GSL_GENLK_PAD_CNTL__DCIO_GENLK_CLK_GSL_FLIP_LOCK_SEL_MASK 0x30
+#define DCIO_GSL_GENLK_PAD_CNTL__DCIO_GENLK_CLK_GSL_FLIP_LOCK_SEL__SHIFT 0x4
+#define DCIO_GSL_GENLK_PAD_CNTL__DCIO_GENLK_CLK_GSL_MASK_MASK 0x300
+#define DCIO_GSL_GENLK_PAD_CNTL__DCIO_GENLK_CLK_GSL_MASK__SHIFT 0x8
+#define DCIO_GSL_GENLK_PAD_CNTL__DCIO_GENLK_VSYNC_GSL_TIMING_SYNC_SEL_MASK 0x30000
+#define DCIO_GSL_GENLK_PAD_CNTL__DCIO_GENLK_VSYNC_GSL_TIMING_SYNC_SEL__SHIFT 0x10
+#define DCIO_GSL_GENLK_PAD_CNTL__DCIO_GENLK_VSYNC_GSL_FLIP_LOCK_SEL_MASK 0x300000
+#define DCIO_GSL_GENLK_PAD_CNTL__DCIO_GENLK_VSYNC_GSL_FLIP_LOCK_SEL__SHIFT 0x14
+#define DCIO_GSL_GENLK_PAD_CNTL__DCIO_GENLK_VSYNC_GSL_MASK_MASK 0x3000000
+#define DCIO_GSL_GENLK_PAD_CNTL__DCIO_GENLK_VSYNC_GSL_MASK__SHIFT 0x18
+#define DCIO_GSL_SWAPLOCK_PAD_CNTL__DCIO_SWAPLOCK_A_GSL_TIMING_SYNC_SEL_MASK 0x3
+#define DCIO_GSL_SWAPLOCK_PAD_CNTL__DCIO_SWAPLOCK_A_GSL_TIMING_SYNC_SEL__SHIFT 0x0
+#define DCIO_GSL_SWAPLOCK_PAD_CNTL__DCIO_SWAPLOCK_A_GSL_FLIP_LOCK_SEL_MASK 0x30
+#define DCIO_GSL_SWAPLOCK_PAD_CNTL__DCIO_SWAPLOCK_A_GSL_FLIP_LOCK_SEL__SHIFT 0x4
+#define DCIO_GSL_SWAPLOCK_PAD_CNTL__DCIO_SWAPLOCK_A_GSL_MASK_MASK 0x300
+#define DCIO_GSL_SWAPLOCK_PAD_CNTL__DCIO_SWAPLOCK_A_GSL_MASK__SHIFT 0x8
+#define DCIO_GSL_SWAPLOCK_PAD_CNTL__DCIO_SWAPLOCK_B_GSL_TIMING_SYNC_SEL_MASK 0x30000
+#define DCIO_GSL_SWAPLOCK_PAD_CNTL__DCIO_SWAPLOCK_B_GSL_TIMING_SYNC_SEL__SHIFT 0x10
+#define DCIO_GSL_SWAPLOCK_PAD_CNTL__DCIO_SWAPLOCK_B_GSL_FLIP_LOCK_SEL_MASK 0x300000
+#define DCIO_GSL_SWAPLOCK_PAD_CNTL__DCIO_SWAPLOCK_B_GSL_FLIP_LOCK_SEL__SHIFT 0x14
+#define DCIO_GSL_SWAPLOCK_PAD_CNTL__DCIO_SWAPLOCK_B_GSL_MASK_MASK 0x3000000
+#define DCIO_GSL_SWAPLOCK_PAD_CNTL__DCIO_SWAPLOCK_B_GSL_MASK__SHIFT 0x18
+#define DCIO_GSL0_CNTL__DCIO_GSL0_VSYNC_SEL_MASK 0x7
+#define DCIO_GSL0_CNTL__DCIO_GSL0_VSYNC_SEL__SHIFT 0x0
+#define DCIO_GSL0_CNTL__DCIO_GSL0_TIMING_SYNC_SEL_MASK 0x700
+#define DCIO_GSL0_CNTL__DCIO_GSL0_TIMING_SYNC_SEL__SHIFT 0x8
+#define DCIO_GSL0_CNTL__DCIO_GSL0_GLOBAL_UNLOCK_SEL_MASK 0x70000
+#define DCIO_GSL0_CNTL__DCIO_GSL0_GLOBAL_UNLOCK_SEL__SHIFT 0x10
+#define DCIO_GSL1_CNTL__DCIO_GSL1_VSYNC_SEL_MASK 0x7
+#define DCIO_GSL1_CNTL__DCIO_GSL1_VSYNC_SEL__SHIFT 0x0
+#define DCIO_GSL1_CNTL__DCIO_GSL1_TIMING_SYNC_SEL_MASK 0x700
+#define DCIO_GSL1_CNTL__DCIO_GSL1_TIMING_SYNC_SEL__SHIFT 0x8
+#define DCIO_GSL1_CNTL__DCIO_GSL1_GLOBAL_UNLOCK_SEL_MASK 0x70000
+#define DCIO_GSL1_CNTL__DCIO_GSL1_GLOBAL_UNLOCK_SEL__SHIFT 0x10
+#define DCIO_GSL2_CNTL__DCIO_GSL2_VSYNC_SEL_MASK 0x7
+#define DCIO_GSL2_CNTL__DCIO_GSL2_VSYNC_SEL__SHIFT 0x0
+#define DCIO_GSL2_CNTL__DCIO_GSL2_TIMING_SYNC_SEL_MASK 0x700
+#define DCIO_GSL2_CNTL__DCIO_GSL2_TIMING_SYNC_SEL__SHIFT 0x8
+#define DCIO_GSL2_CNTL__DCIO_GSL2_GLOBAL_UNLOCK_SEL_MASK 0x70000
+#define DCIO_GSL2_CNTL__DCIO_GSL2_GLOBAL_UNLOCK_SEL__SHIFT 0x10
+#define DC_GPU_TIMER_START_POSITION_V_UPDATE__DC_GPU_TIMER_START_POSITION_D1_V_UPDATE_MASK 0x7
+#define DC_GPU_TIMER_START_POSITION_V_UPDATE__DC_GPU_TIMER_START_POSITION_D1_V_UPDATE__SHIFT 0x0
+#define DC_GPU_TIMER_START_POSITION_V_UPDATE__DC_GPU_TIMER_START_POSITION_D2_V_UPDATE_MASK 0x70
+#define DC_GPU_TIMER_START_POSITION_V_UPDATE__DC_GPU_TIMER_START_POSITION_D2_V_UPDATE__SHIFT 0x4
+#define DC_GPU_TIMER_START_POSITION_V_UPDATE__DC_GPU_TIMER_START_POSITION_D3_V_UPDATE_MASK 0x700
+#define DC_GPU_TIMER_START_POSITION_V_UPDATE__DC_GPU_TIMER_START_POSITION_D3_V_UPDATE__SHIFT 0x8
+#define DC_GPU_TIMER_START_POSITION_V_UPDATE__DC_GPU_TIMER_START_POSITION_D4_V_UPDATE_MASK 0x7000
+#define DC_GPU_TIMER_START_POSITION_V_UPDATE__DC_GPU_TIMER_START_POSITION_D4_V_UPDATE__SHIFT 0xc
+#define DC_GPU_TIMER_START_POSITION_V_UPDATE__DC_GPU_TIMER_START_POSITION_D5_V_UPDATE_MASK 0x70000
+#define DC_GPU_TIMER_START_POSITION_V_UPDATE__DC_GPU_TIMER_START_POSITION_D5_V_UPDATE__SHIFT 0x10
+#define DC_GPU_TIMER_START_POSITION_V_UPDATE__DC_GPU_TIMER_START_POSITION_D6_V_UPDATE_MASK 0x700000
+#define DC_GPU_TIMER_START_POSITION_V_UPDATE__DC_GPU_TIMER_START_POSITION_D6_V_UPDATE__SHIFT 0x14
+#define DC_GPU_TIMER_START_POSITION_P_FLIP__DC_GPU_TIMER_START_POSITION_D1_P_FLIP_MASK 0x7
+#define DC_GPU_TIMER_START_POSITION_P_FLIP__DC_GPU_TIMER_START_POSITION_D1_P_FLIP__SHIFT 0x0
+#define DC_GPU_TIMER_START_POSITION_P_FLIP__DC_GPU_TIMER_START_POSITION_D2_P_FLIP_MASK 0x70
+#define DC_GPU_TIMER_START_POSITION_P_FLIP__DC_GPU_TIMER_START_POSITION_D2_P_FLIP__SHIFT 0x4
+#define DC_GPU_TIMER_START_POSITION_P_FLIP__DC_GPU_TIMER_START_POSITION_D3_P_FLIP_MASK 0x700
+#define DC_GPU_TIMER_START_POSITION_P_FLIP__DC_GPU_TIMER_START_POSITION_D3_P_FLIP__SHIFT 0x8
+#define DC_GPU_TIMER_START_POSITION_P_FLIP__DC_GPU_TIMER_START_POSITION_D4_P_FLIP_MASK 0x7000
+#define DC_GPU_TIMER_START_POSITION_P_FLIP__DC_GPU_TIMER_START_POSITION_D4_P_FLIP__SHIFT 0xc
+#define DC_GPU_TIMER_START_POSITION_P_FLIP__DC_GPU_TIMER_START_POSITION_D5_P_FLIP_MASK 0x70000
+#define DC_GPU_TIMER_START_POSITION_P_FLIP__DC_GPU_TIMER_START_POSITION_D5_P_FLIP__SHIFT 0x10
+#define DC_GPU_TIMER_START_POSITION_P_FLIP__DC_GPU_TIMER_START_POSITION_D6_P_FLIP_MASK 0x700000
+#define DC_GPU_TIMER_START_POSITION_P_FLIP__DC_GPU_TIMER_START_POSITION_D6_P_FLIP__SHIFT 0x14
+#define DC_GPU_TIMER_START_POSITION_P_FLIP__DC_GPU_TIMER_START_POSITION_DCFEV0_P_FLIP_MASK 0x3800000
+#define DC_GPU_TIMER_START_POSITION_P_FLIP__DC_GPU_TIMER_START_POSITION_DCFEV0_P_FLIP__SHIFT 0x17
+#define DC_GPU_TIMER_START_POSITION_P_FLIP__DC_GPU_TIMER_START_POSITION_DCFEV1_P_FLIP_MASK 0x1c000000
+#define DC_GPU_TIMER_START_POSITION_P_FLIP__DC_GPU_TIMER_START_POSITION_DCFEV1_P_FLIP__SHIFT 0x1a
+#define DC_GPU_TIMER_READ__DC_GPU_TIMER_READ_MASK 0xffffffff
+#define DC_GPU_TIMER_READ__DC_GPU_TIMER_READ__SHIFT 0x0
+#define DC_GPU_TIMER_READ_CNTL__DC_GPU_TIMER_READ_SELECT_MASK 0x3f
+#define DC_GPU_TIMER_READ_CNTL__DC_GPU_TIMER_READ_SELECT__SHIFT 0x0
+#define DC_GPU_TIMER_READ_CNTL__DC_GPU_TIMER_START_POSITION_D1_VSYNC_NOM_MASK 0x700
+#define DC_GPU_TIMER_READ_CNTL__DC_GPU_TIMER_START_POSITION_D1_VSYNC_NOM__SHIFT 0x8
+#define DC_GPU_TIMER_READ_CNTL__DC_GPU_TIMER_START_POSITION_D2_VSYNC_NOM_MASK 0x3800
+#define DC_GPU_TIMER_READ_CNTL__DC_GPU_TIMER_START_POSITION_D2_VSYNC_NOM__SHIFT 0xb
+#define DC_GPU_TIMER_READ_CNTL__DC_GPU_TIMER_START_POSITION_D3_VSYNC_NOM_MASK 0x1c000
+#define DC_GPU_TIMER_READ_CNTL__DC_GPU_TIMER_START_POSITION_D3_VSYNC_NOM__SHIFT 0xe
+#define DC_GPU_TIMER_READ_CNTL__DC_GPU_TIMER_START_POSITION_D4_VSYNC_NOM_MASK 0xe0000
+#define DC_GPU_TIMER_READ_CNTL__DC_GPU_TIMER_START_POSITION_D4_VSYNC_NOM__SHIFT 0x11
+#define DC_GPU_TIMER_READ_CNTL__DC_GPU_TIMER_START_POSITION_D5_VSYNC_NOM_MASK 0x700000
+#define DC_GPU_TIMER_READ_CNTL__DC_GPU_TIMER_START_POSITION_D5_VSYNC_NOM__SHIFT 0x14
+#define DC_GPU_TIMER_READ_CNTL__DC_GPU_TIMER_START_POSITION_D6_VSYNC_NOM_MASK 0x3800000
+#define DC_GPU_TIMER_READ_CNTL__DC_GPU_TIMER_START_POSITION_D6_VSYNC_NOM__SHIFT 0x17
+#define DCIO_CLOCK_CNTL__DCIO_TEST_CLK_SEL_MASK 0x1f
+#define DCIO_CLOCK_CNTL__DCIO_TEST_CLK_SEL__SHIFT 0x0
+#define DCIO_CLOCK_CNTL__DISPCLK_R_DCIO_GATE_DIS_MASK 0x20
+#define DCIO_CLOCK_CNTL__DISPCLK_R_DCIO_GATE_DIS__SHIFT 0x5
+#define DCIO_DEBUG__DCIO_DEBUG_MASK 0xffffffff
+#define DCIO_DEBUG__DCIO_DEBUG__SHIFT 0x0
+#define DCO_DCFE_EXT_VSYNC_CNTL__DCO_DCFE0_EXT_VSYNC_MUX_MASK 0x7
+#define DCO_DCFE_EXT_VSYNC_CNTL__DCO_DCFE0_EXT_VSYNC_MUX__SHIFT 0x0
+#define DCO_DCFE_EXT_VSYNC_CNTL__DCO_DCFE1_EXT_VSYNC_MUX_MASK 0x70
+#define DCO_DCFE_EXT_VSYNC_CNTL__DCO_DCFE1_EXT_VSYNC_MUX__SHIFT 0x4
+#define DCO_DCFE_EXT_VSYNC_CNTL__DCO_DCFE2_EXT_VSYNC_MUX_MASK 0x700
+#define DCO_DCFE_EXT_VSYNC_CNTL__DCO_DCFE2_EXT_VSYNC_MUX__SHIFT 0x8
+#define DCO_DCFE_EXT_VSYNC_CNTL__DCO_DCFE3_EXT_VSYNC_MUX_MASK 0x7000
+#define DCO_DCFE_EXT_VSYNC_CNTL__DCO_DCFE3_EXT_VSYNC_MUX__SHIFT 0xc
+#define DCO_DCFE_EXT_VSYNC_CNTL__DCO_DCFE4_EXT_VSYNC_MUX_MASK 0x70000
+#define DCO_DCFE_EXT_VSYNC_CNTL__DCO_DCFE4_EXT_VSYNC_MUX__SHIFT 0x10
+#define DCO_DCFE_EXT_VSYNC_CNTL__DCO_DCFE5_EXT_VSYNC_MUX_MASK 0x700000
+#define DCO_DCFE_EXT_VSYNC_CNTL__DCO_DCFE5_EXT_VSYNC_MUX__SHIFT 0x14
+#define DCO_DCFE_EXT_VSYNC_CNTL__DCO_SWAPLOCKB_EXT_VSYNC_MASK_MASK 0x7000000
+#define DCO_DCFE_EXT_VSYNC_CNTL__DCO_SWAPLOCKB_EXT_VSYNC_MASK__SHIFT 0x18
+#define DCO_DCFE_EXT_VSYNC_CNTL__DCO_GENERICB_EXT_VSYNC_MASK_MASK 0x70000000
+#define DCO_DCFE_EXT_VSYNC_CNTL__DCO_GENERICB_EXT_VSYNC_MASK__SHIFT 0x1c
+#define DCO_DCFE_EXT_VSYNC_CNTL__DCO_CRTC_MANUAL_FLOW_CONTROL_MASK 0x80000000
+#define DCO_DCFE_EXT_VSYNC_CNTL__DCO_CRTC_MANUAL_FLOW_CONTROL__SHIFT 0x1f
+#define DBG_OUT_CNTL__DBG_OUT_PIN_EN_MASK 0x1
+#define DBG_OUT_CNTL__DBG_OUT_PIN_EN__SHIFT 0x0
+#define DBG_OUT_CNTL__DBG_OUT_PIN_SEL_MASK 0x10
+#define DBG_OUT_CNTL__DBG_OUT_PIN_SEL__SHIFT 0x4
+#define DBG_OUT_CNTL__DBG_OUT_12BIT_SEL_MASK 0x300
+#define DBG_OUT_CNTL__DBG_OUT_12BIT_SEL__SHIFT 0x8
+#define DBG_OUT_CNTL__DBG_OUT_TEST_DATA_MASK 0xfff000
+#define DBG_OUT_CNTL__DBG_OUT_TEST_DATA__SHIFT 0xc
+#define DCIO_DEBUG_CONFIG__DCIO_DBG_EN_MASK 0x1
+#define DCIO_DEBUG_CONFIG__DCIO_DBG_EN__SHIFT 0x0
+#define DCIO_DEBUG_CONFIG__DCIO_DBG_SEL_MASK 0xf00
+#define DCIO_DEBUG_CONFIG__DCIO_DBG_SEL__SHIFT 0x8
+#define DCIO_SOFT_RESET__UNIPHYA_SOFT_RESET_MASK 0x1
+#define DCIO_SOFT_RESET__UNIPHYA_SOFT_RESET__SHIFT 0x0
+#define DCIO_SOFT_RESET__DSYNCA_SOFT_RESET_MASK 0x2
+#define DCIO_SOFT_RESET__DSYNCA_SOFT_RESET__SHIFT 0x1
+#define DCIO_SOFT_RESET__UNIPHYB_SOFT_RESET_MASK 0x4
+#define DCIO_SOFT_RESET__UNIPHYB_SOFT_RESET__SHIFT 0x2
+#define DCIO_SOFT_RESET__DSYNCB_SOFT_RESET_MASK 0x8
+#define DCIO_SOFT_RESET__DSYNCB_SOFT_RESET__SHIFT 0x3
+#define DCIO_SOFT_RESET__UNIPHYC_SOFT_RESET_MASK 0x10
+#define DCIO_SOFT_RESET__UNIPHYC_SOFT_RESET__SHIFT 0x4
+#define DCIO_SOFT_RESET__DSYNCC_SOFT_RESET_MASK 0x20
+#define DCIO_SOFT_RESET__DSYNCC_SOFT_RESET__SHIFT 0x5
+#define DCIO_SOFT_RESET__UNIPHYD_SOFT_RESET_MASK 0x40
+#define DCIO_SOFT_RESET__UNIPHYD_SOFT_RESET__SHIFT 0x6
+#define DCIO_SOFT_RESET__DSYNCD_SOFT_RESET_MASK 0x80
+#define DCIO_SOFT_RESET__DSYNCD_SOFT_RESET__SHIFT 0x7
+#define DCIO_SOFT_RESET__UNIPHYE_SOFT_RESET_MASK 0x100
+#define DCIO_SOFT_RESET__UNIPHYE_SOFT_RESET__SHIFT 0x8
+#define DCIO_SOFT_RESET__DSYNCE_SOFT_RESET_MASK 0x200
+#define DCIO_SOFT_RESET__DSYNCE_SOFT_RESET__SHIFT 0x9
+#define DCIO_SOFT_RESET__UNIPHYF_SOFT_RESET_MASK 0x400
+#define DCIO_SOFT_RESET__UNIPHYF_SOFT_RESET__SHIFT 0xa
+#define DCIO_SOFT_RESET__DSYNCF_SOFT_RESET_MASK 0x800
+#define DCIO_SOFT_RESET__DSYNCF_SOFT_RESET__SHIFT 0xb
+#define DCIO_SOFT_RESET__UNIPHYG_SOFT_RESET_MASK 0x1000
+#define DCIO_SOFT_RESET__UNIPHYG_SOFT_RESET__SHIFT 0xc
+#define DCIO_SOFT_RESET__DSYNCG_SOFT_RESET_MASK 0x2000
+#define DCIO_SOFT_RESET__DSYNCG_SOFT_RESET__SHIFT 0xd
+#define DCIO_SOFT_RESET__DACA_SOFT_RESET_MASK 0x10000
+#define DCIO_SOFT_RESET__DACA_SOFT_RESET__SHIFT 0x10
+#define DCIO_SOFT_RESET__DCRXPHY_SOFT_RESET_MASK 0x100000
+#define DCIO_SOFT_RESET__DCRXPHY_SOFT_RESET__SHIFT 0x14
+#define DCIO_SOFT_RESET__DPHY_SOFT_RESET_MASK 0x1000000
+#define DCIO_SOFT_RESET__DPHY_SOFT_RESET__SHIFT 0x18
+#define DCIO_SOFT_RESET__ZCAL_SOFT_RESET_MASK 0x4000000
+#define DCIO_SOFT_RESET__ZCAL_SOFT_RESET__SHIFT 0x1a
+#define DCIO_SOFT_RESET__UNIPHYLPA_SOFT_RESET_MASK 0x10000000
+#define DCIO_SOFT_RESET__UNIPHYLPA_SOFT_RESET__SHIFT 0x1c
+#define DCIO_SOFT_RESET__DSYNCLPA_SOFT_RESET_MASK 0x20000000
+#define DCIO_SOFT_RESET__DSYNCLPA_SOFT_RESET__SHIFT 0x1d
+#define DCIO_SOFT_RESET__UNIPHYLPB_SOFT_RESET_MASK 0x40000000
+#define DCIO_SOFT_RESET__UNIPHYLPB_SOFT_RESET__SHIFT 0x1e
+#define DCIO_SOFT_RESET__DSYNCLPB_SOFT_RESET_MASK 0x80000000
+#define DCIO_SOFT_RESET__DSYNCLPB_SOFT_RESET__SHIFT 0x1f
+#define DCIO_DPHY_SEL__DPHY_LANE0_SEL_MASK 0x3
+#define DCIO_DPHY_SEL__DPHY_LANE0_SEL__SHIFT 0x0
+#define DCIO_DPHY_SEL__DPHY_LANE1_SEL_MASK 0xc
+#define DCIO_DPHY_SEL__DPHY_LANE1_SEL__SHIFT 0x2
+#define DCIO_DPHY_SEL__DPHY_LANE2_SEL_MASK 0x30
+#define DCIO_DPHY_SEL__DPHY_LANE2_SEL__SHIFT 0x4
+#define DCIO_DPHY_SEL__DPHY_LANE3_SEL_MASK 0xc0
+#define DCIO_DPHY_SEL__DPHY_LANE3_SEL__SHIFT 0x6
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXA_INT_TYPE_MASK 0x1
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXA_INT_TYPE__SHIFT 0x0
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXA_INT_MASK_MASK 0x2
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXA_INT_MASK__SHIFT 0x1
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXA_INT_OCCUR_MASK 0x4
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXA_INT_OCCUR__SHIFT 0x2
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXB_INT_TYPE_MASK 0x8
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXB_INT_TYPE__SHIFT 0x3
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXB_INT_MASK_MASK 0x10
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXB_INT_MASK__SHIFT 0x4
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXB_INT_OCCUR_MASK 0x20
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXB_INT_OCCUR__SHIFT 0x5
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXC_INT_TYPE_MASK 0x40
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXC_INT_TYPE__SHIFT 0x6
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXC_INT_MASK_MASK 0x80
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXC_INT_MASK__SHIFT 0x7
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXC_INT_OCCUR_MASK 0x100
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXC_INT_OCCUR__SHIFT 0x8
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXD_INT_TYPE_MASK 0x200
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXD_INT_TYPE__SHIFT 0x9
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXD_INT_MASK_MASK 0x400
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXD_INT_MASK__SHIFT 0xa
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXD_INT_OCCUR_MASK 0x800
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXD_INT_OCCUR__SHIFT 0xb
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXE_INT_TYPE_MASK 0x1000
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXE_INT_TYPE__SHIFT 0xc
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXE_INT_MASK_MASK 0x2000
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXE_INT_MASK__SHIFT 0xd
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXE_INT_OCCUR_MASK 0x4000
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXE_INT_OCCUR__SHIFT 0xe
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXF_INT_TYPE_MASK 0x8000
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXF_INT_TYPE__SHIFT 0xf
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXF_INT_MASK_MASK 0x10000
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXF_INT_MASK__SHIFT 0x10
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXF_INT_OCCUR_MASK 0x20000
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXF_INT_OCCUR__SHIFT 0x11
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXG_INT_TYPE_MASK 0x40000
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXG_INT_TYPE__SHIFT 0x12
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXG_INT_MASK_MASK 0x80000
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXG_INT_MASK__SHIFT 0x13
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXG_INT_OCCUR_MASK 0x100000
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXG_INT_OCCUR__SHIFT 0x14
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXLPA_INT_TYPE_MASK 0x1000000
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXLPA_INT_TYPE__SHIFT 0x18
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXLPA_INT_MASK_MASK 0x2000000
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXLPA_INT_MASK__SHIFT 0x19
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXLPA_INT_OCCUR_MASK 0x4000000
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXLPA_INT_OCCUR__SHIFT 0x1a
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXLPB_INT_TYPE_MASK 0x8000000
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXLPB_INT_TYPE__SHIFT 0x1b
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXLPB_INT_MASK_MASK 0x10000000
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXLPB_INT_MASK__SHIFT 0x1c
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXLPB_INT_OCCUR_MASK 0x20000000
+#define DCIO_DPCS_TX_INTERRUPT__DCIO_DPCS_TXLPB_INT_OCCUR__SHIFT 0x1d
+#define DCIO_DPCS_RX_INTERRUPT__DCIO_DPCS_RXA_INT_TYPE_MASK 0x1
+#define DCIO_DPCS_RX_INTERRUPT__DCIO_DPCS_RXA_INT_TYPE__SHIFT 0x0
+#define DCIO_DPCS_RX_INTERRUPT__DCIO_DPCS_RXA_INT_MASK_MASK 0x2
+#define DCIO_DPCS_RX_INTERRUPT__DCIO_DPCS_RXA_INT_MASK__SHIFT 0x1
+#define DCIO_DPCS_RX_INTERRUPT__DCIO_DPCS_RXA_INT_OCCUR_MASK 0x4
+#define DCIO_DPCS_RX_INTERRUPT__DCIO_DPCS_RXA_INT_OCCUR__SHIFT 0x2
+#define DCIO_SEMAPHORE0__DCIO_SEMAPHORE0_REQ_MASK 0xffff
+#define DCIO_SEMAPHORE0__DCIO_SEMAPHORE0_REQ__SHIFT 0x0
+#define DCIO_SEMAPHORE0__DCIO_SEMAPHORE0_GNT_MASK 0xffff0000
+#define DCIO_SEMAPHORE0__DCIO_SEMAPHORE0_GNT__SHIFT 0x10
+#define DCIO_SEMAPHORE1__DCIO_SEMAPHORE1_REQ_MASK 0xffff
+#define DCIO_SEMAPHORE1__DCIO_SEMAPHORE1_REQ__SHIFT 0x0
+#define DCIO_SEMAPHORE1__DCIO_SEMAPHORE1_GNT_MASK 0xffff0000
+#define DCIO_SEMAPHORE1__DCIO_SEMAPHORE1_GNT__SHIFT 0x10
+#define DCIO_SEMAPHORE2__DCIO_SEMAPHORE2_REQ_MASK 0xffff
+#define DCIO_SEMAPHORE2__DCIO_SEMAPHORE2_REQ__SHIFT 0x0
+#define DCIO_SEMAPHORE2__DCIO_SEMAPHORE2_GNT_MASK 0xffff0000
+#define DCIO_SEMAPHORE2__DCIO_SEMAPHORE2_GNT__SHIFT 0x10
+#define DCIO_SEMAPHORE3__DCIO_SEMAPHORE3_REQ_MASK 0xffff
+#define DCIO_SEMAPHORE3__DCIO_SEMAPHORE3_REQ__SHIFT 0x0
+#define DCIO_SEMAPHORE3__DCIO_SEMAPHORE3_GNT_MASK 0xffff0000
+#define DCIO_SEMAPHORE3__DCIO_SEMAPHORE3_GNT__SHIFT 0x10
+#define DCIO_SEMAPHORE4__DCIO_SEMAPHORE4_REQ_MASK 0xffff
+#define DCIO_SEMAPHORE4__DCIO_SEMAPHORE4_REQ__SHIFT 0x0
+#define DCIO_SEMAPHORE4__DCIO_SEMAPHORE4_GNT_MASK 0xffff0000
+#define DCIO_SEMAPHORE4__DCIO_SEMAPHORE4_GNT__SHIFT 0x10
+#define DCIO_SEMAPHORE5__DCIO_SEMAPHORE5_REQ_MASK 0xffff
+#define DCIO_SEMAPHORE5__DCIO_SEMAPHORE5_REQ__SHIFT 0x0
+#define DCIO_SEMAPHORE5__DCIO_SEMAPHORE5_GNT_MASK 0xffff0000
+#define DCIO_SEMAPHORE5__DCIO_SEMAPHORE5_GNT__SHIFT 0x10
+#define DCIO_SEMAPHORE6__DCIO_SEMAPHORE6_REQ_MASK 0xffff
+#define DCIO_SEMAPHORE6__DCIO_SEMAPHORE6_REQ__SHIFT 0x0
+#define DCIO_SEMAPHORE6__DCIO_SEMAPHORE6_GNT_MASK 0xffff0000
+#define DCIO_SEMAPHORE6__DCIO_SEMAPHORE6_GNT__SHIFT 0x10
+#define DCIO_SEMAPHORE7__DCIO_SEMAPHORE7_REQ_MASK 0xffff
+#define DCIO_SEMAPHORE7__DCIO_SEMAPHORE7_REQ__SHIFT 0x0
+#define DCIO_SEMAPHORE7__DCIO_SEMAPHORE7_GNT_MASK 0xffff0000
+#define DCIO_SEMAPHORE7__DCIO_SEMAPHORE7_GNT__SHIFT 0x10
+#define DCIO_TEST_DEBUG_INDEX__DCIO_TEST_DEBUG_INDEX_MASK 0xff
+#define DCIO_TEST_DEBUG_INDEX__DCIO_TEST_DEBUG_INDEX__SHIFT 0x0
+#define DCIO_TEST_DEBUG_INDEX__DCIO_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define DCIO_TEST_DEBUG_INDEX__DCIO_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define DCIO_TEST_DEBUG_DATA__DCIO_TEST_DEBUG_DATA_MASK 0xffffffff
+#define DCIO_TEST_DEBUG_DATA__DCIO_TEST_DEBUG_DATA__SHIFT 0x0
+#define DCIO_DEBUG1__DCO_DCIO_MVP_DVOCNTL_A0_REG_MASK 0x3
+#define DCIO_DEBUG1__DCO_DCIO_MVP_DVOCNTL_A0_REG__SHIFT 0x0
+#define DCIO_DEBUG1__DCO_DCIO_MVP_DVOCNTL_MASK_REG_MASK 0xc
+#define DCIO_DEBUG1__DCO_DCIO_MVP_DVOCNTL_MASK_REG__SHIFT 0x2
+#define DCIO_DEBUG1__DCO_DCIO_MVP_DVOCNTL_EN_REG_MASK 0x30
+#define DCIO_DEBUG1__DCO_DCIO_MVP_DVOCNTL_EN_REG__SHIFT 0x4
+#define DCIO_DEBUG1__DCO_DCIO_MVP_DVOCNTL_A0_MASK 0xc0
+#define DCIO_DEBUG1__DCO_DCIO_MVP_DVOCNTL_A0__SHIFT 0x6
+#define DCIO_DEBUG1__DCO_DCIO_MVP_DVOCNTL_SEL0_MASK 0x300
+#define DCIO_DEBUG1__DCO_DCIO_MVP_DVOCNTL_SEL0__SHIFT 0x8
+#define DCIO_DEBUG1__DCO_DCIO_MVP_DVOCNTL_EN_MASK 0xc00
+#define DCIO_DEBUG1__DCO_DCIO_MVP_DVOCNTL_EN__SHIFT 0xa
+#define DCIO_DEBUG1__DCO_DCIO_MVP_DVOCLK_C_MASK 0x1000
+#define DCIO_DEBUG1__DCO_DCIO_MVP_DVOCLK_C__SHIFT 0xc
+#define DCIO_DEBUG1__DCO_DCIO_DVOCNTL1_A0_REG_MASK 0x2000
+#define DCIO_DEBUG1__DCO_DCIO_DVOCNTL1_A0_REG__SHIFT 0xd
+#define DCIO_DEBUG1__DCO_DCIO_DVOCNTL1_A0_PREMUX_MASK 0x4000
+#define DCIO_DEBUG1__DCO_DCIO_DVOCNTL1_A0_PREMUX__SHIFT 0xe
+#define DCIO_DEBUG1__DCO_DCIO_DVOCNTL1_A0_MASK 0x8000
+#define DCIO_DEBUG1__DCO_DCIO_DVOCNTL1_A0__SHIFT 0xf
+#define DCIO_DEBUG1__DCO_DCIO_DVOCNTL1_EN_REG_MASK 0x10000
+#define DCIO_DEBUG1__DCO_DCIO_DVOCNTL1_EN_REG__SHIFT 0x10
+#define DCIO_DEBUG1__DCO_DCIO_DVO_HSYNC_TRISTATE_MASK 0x20000
+#define DCIO_DEBUG1__DCO_DCIO_DVO_HSYNC_TRISTATE__SHIFT 0x11
+#define DCIO_DEBUG1__DCO_DCIO_DVO_CLK_TRISTATE_MASK 0x40000
+#define DCIO_DEBUG1__DCO_DCIO_DVO_CLK_TRISTATE__SHIFT 0x12
+#define DCIO_DEBUG1__DCO_DCIO_DVOCNTL1_EN_PREMUX_MASK 0x80000
+#define DCIO_DEBUG1__DCO_DCIO_DVOCNTL1_EN_PREMUX__SHIFT 0x13
+#define DCIO_DEBUG1__DCO_DCIO_DVOCNTL1_EN_MASK 0x100000
+#define DCIO_DEBUG1__DCO_DCIO_DVOCNTL1_EN__SHIFT 0x14
+#define DCIO_DEBUG1__DCO_DCIO_DVOCNTL1_MUX_MASK 0x200000
+#define DCIO_DEBUG1__DCO_DCIO_DVOCNTL1_MUX__SHIFT 0x15
+#define DCIO_DEBUG1__DCO_DCIO_DVOCNTL1_MASK_REG_MASK 0x400000
+#define DCIO_DEBUG1__DCO_DCIO_DVOCNTL1_MASK_REG__SHIFT 0x16
+#define DCIO_DEBUG1__DCO_DCIO_DVO_ENABLE_MASK 0x800000
+#define DCIO_DEBUG1__DCO_DCIO_DVO_ENABLE__SHIFT 0x17
+#define DCIO_DEBUG1__DCO_DCIO_DVO_VSYNC_TRISTATE_MASK 0x1000000
+#define DCIO_DEBUG1__DCO_DCIO_DVO_VSYNC_TRISTATE__SHIFT 0x18
+#define DCIO_DEBUG1__DCO_DCIO_DVO_RATE_SEL_MASK 0x2000000
+#define DCIO_DEBUG1__DCO_DCIO_DVO_RATE_SEL__SHIFT 0x19
+#define DCIO_DEBUG1__DCO_DCIO_DVOCNTL1_SEL0_PREMUX_MASK 0x4000000
+#define DCIO_DEBUG1__DCO_DCIO_DVOCNTL1_SEL0_PREMUX__SHIFT 0x1a
+#define DCIO_DEBUG1__DCO_DCIO_DVOCNTL1_SEL0_MASK 0x8000000
+#define DCIO_DEBUG1__DCO_DCIO_DVOCNTL1_SEL0__SHIFT 0x1b
+#define DCIO_DEBUG2__DCIO_DEBUG2_MASK 0xffffffff
+#define DCIO_DEBUG2__DCIO_DEBUG2__SHIFT 0x0
+#define DCIO_DEBUG3__DCIO_DEBUG3_MASK 0xffffffff
+#define DCIO_DEBUG3__DCIO_DEBUG3__SHIFT 0x0
+#define DCIO_DEBUG4__DCIO_DEBUG4_MASK 0xffffffff
+#define DCIO_DEBUG4__DCIO_DEBUG4__SHIFT 0x0
+#define DCIO_DEBUG5__DCIO_DEBUG5_MASK 0xffffffff
+#define DCIO_DEBUG5__DCIO_DEBUG5__SHIFT 0x0
+#define DCIO_DEBUG6__DCIO_DEBUG6_MASK 0xffffffff
+#define DCIO_DEBUG6__DCIO_DEBUG6__SHIFT 0x0
+#define DCIO_DEBUG7__DCIO_DEBUG7_MASK 0xffffffff
+#define DCIO_DEBUG7__DCIO_DEBUG7__SHIFT 0x0
+#define DCIO_DEBUG8__DCIO_DEBUG8_MASK 0xffffffff
+#define DCIO_DEBUG8__DCIO_DEBUG8__SHIFT 0x0
+#define DCIO_DEBUG9__DCIO_DEBUG9_MASK 0xffffffff
+#define DCIO_DEBUG9__DCIO_DEBUG9__SHIFT 0x0
+#define DCIO_DEBUGA__DCIO_DEBUGA_MASK 0xffffffff
+#define DCIO_DEBUGA__DCIO_DEBUGA__SHIFT 0x0
+#define DCIO_DEBUGB__DCIO_DEBUGB_MASK 0xffffffff
+#define DCIO_DEBUGB__DCIO_DEBUGB__SHIFT 0x0
+#define DCIO_DEBUGC__DCIO_DEBUGC_MASK 0xffffffff
+#define DCIO_DEBUGC__DCIO_DEBUGC__SHIFT 0x0
+#define DCIO_DEBUGD__DCIO_DEBUGD_MASK 0xffffffff
+#define DCIO_DEBUGD__DCIO_DEBUGD__SHIFT 0x0
+#define DCIO_DEBUGE__DCIO_DIGA_DEBUG_MASK 0xffffffff
+#define DCIO_DEBUGE__DCIO_DIGA_DEBUG__SHIFT 0x0
+#define DCIO_DEBUGF__DCIO_DIGB_DEBUG_MASK 0xffffffff
+#define DCIO_DEBUGF__DCIO_DIGB_DEBUG__SHIFT 0x0
+#define DCIO_DEBUG10__DCIO_DIGC_DEBUG_MASK 0xffffffff
+#define DCIO_DEBUG10__DCIO_DIGC_DEBUG__SHIFT 0x0
+#define DCIO_DEBUG11__DCIO_DIGD_DEBUG_MASK 0xffffffff
+#define DCIO_DEBUG11__DCIO_DIGD_DEBUG__SHIFT 0x0
+#define DCIO_DEBUG12__DCIO_DIGE_DEBUG_MASK 0xffffffff
+#define DCIO_DEBUG12__DCIO_DIGE_DEBUG__SHIFT 0x0
+#define DCIO_DEBUG13__DCIO_DIGF_DEBUG_MASK 0xffffffff
+#define DCIO_DEBUG13__DCIO_DIGF_DEBUG__SHIFT 0x0
+#define DCIO_DEBUG14__DCIO_DIGG_DEBUG_MASK 0xffffffff
+#define DCIO_DEBUG14__DCIO_DIGG_DEBUG__SHIFT 0x0
+#define DCIO_DEBUG15__DCIO_DEBUG15_MASK 0xffffffff
+#define DCIO_DEBUG15__DCIO_DEBUG15__SHIFT 0x0
+#define DCIO_DEBUG16__DCIO_DEBUG16_MASK 0xffffffff
+#define DCIO_DEBUG16__DCIO_DEBUG16__SHIFT 0x0
+#define DCIO_DEBUG17__DCIO_DEBUG17_MASK 0xffffffff
+#define DCIO_DEBUG17__DCIO_DEBUG17__SHIFT 0x0
+#define DCIO_DEBUG18__DCIO_DEBUG18_MASK 0xffffffff
+#define DCIO_DEBUG18__DCIO_DEBUG18__SHIFT 0x0
+#define DCIO_DEBUG19__DCIO_DIGLPA_DEBUG_MASK 0xffffffff
+#define DCIO_DEBUG19__DCIO_DIGLPA_DEBUG__SHIFT 0x0
+#define DCIO_DEBUG1A__DCIO_DIGLPB_DEBUG_MASK 0xffffffff
+#define DCIO_DEBUG1A__DCIO_DIGLPB_DEBUG__SHIFT 0x0
+#define DCIO_DEBUG1B__DCIO_DEBUGHPD_MASK 0xffffffff
+#define DCIO_DEBUG1B__DCIO_DEBUGHPD__SHIFT 0x0
+#define DCIO_DEBUG1C__DCIO_DEBUG_UNIPHYA_CFG_MASK 0xffffffff
+#define DCIO_DEBUG1C__DCIO_DEBUG_UNIPHYA_CFG__SHIFT 0x0
+#define DCIO_DEBUG1D__DCIO_DEBUG_UNIPHYB_CFG_MASK 0xffffffff
+#define DCIO_DEBUG1D__DCIO_DEBUG_UNIPHYB_CFG__SHIFT 0x0
+#define DCIO_DEBUG1E__DCIO_DEBUG_UNIPHYC_CFG_MASK 0xffffffff
+#define DCIO_DEBUG1E__DCIO_DEBUG_UNIPHYC_CFG__SHIFT 0x0
+#define DCIO_DEBUG1F__DCIO_DEBUG_UNIPHYD_CFG_MASK 0xffffffff
+#define DCIO_DEBUG1F__DCIO_DEBUG_UNIPHYD_CFG__SHIFT 0x0
+#define DCIO_DEBUG20__DCIO_DEBUG_UNIPHYE_CFG_MASK 0xffffffff
+#define DCIO_DEBUG20__DCIO_DEBUG_UNIPHYE_CFG__SHIFT 0x0
+#define DCIO_DEBUG21__DCIO_DEBUG_UNIPHYF_CFG_MASK 0xffffffff
+#define DCIO_DEBUG21__DCIO_DEBUG_UNIPHYF_CFG__SHIFT 0x0
+#define DCIO_DEBUG22__DCIO_DEBUG_UNIPHYG_CFG_MASK 0xffffffff
+#define DCIO_DEBUG22__DCIO_DEBUG_UNIPHYG_CFG__SHIFT 0x0
+#define DCIO_DEBUG23__DCIO_DEBUG_UNIPHYLPA_CFG_MASK 0xffffffff
+#define DCIO_DEBUG23__DCIO_DEBUG_UNIPHYLPA_CFG__SHIFT 0x0
+#define DCIO_DEBUG24__DCIO_DEBUG_UNIPHYLPB_CFG_MASK 0xffffffff
+#define DCIO_DEBUG24__DCIO_DEBUG_UNIPHYLPB_CFG__SHIFT 0x0
+#define DCIO_DEBUG25__DCIO_DEBUG_DCRXPHY_CFG_MASK 0xffffffff
+#define DCIO_DEBUG25__DCIO_DEBUG_DCRXPHY_CFG__SHIFT 0x0
+#define DCIO_DEBUG26__DCIO_DEBUG_DPHY_CFG_MASK 0xffffffff
+#define DCIO_DEBUG26__DCIO_DEBUG_DPHY_CFG__SHIFT 0x0
+#define DCIO_DEBUG27__DCIO_DEBUG_DACA_CFG_MASK 0xffffffff
+#define DCIO_DEBUG27__DCIO_DEBUG_DACA_CFG__SHIFT 0x0
+#define DCIO_DEBUG28__DCIO_DEBUG_ZCAL_CFG_MASK 0xffffffff
+#define DCIO_DEBUG28__DCIO_DEBUG_ZCAL_CFG__SHIFT 0x0
+#define DCIO_DEBUG_ID__DCIO_DEBUG_ID_MASK 0xffffffff
+#define DCIO_DEBUG_ID__DCIO_DEBUG_ID__SHIFT 0x0
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICA_MASK_MASK 0x1
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICA_MASK__SHIFT 0x0
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICA_PD_DIS_MASK 0x2
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICA_PD_DIS__SHIFT 0x1
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICA_RECV_MASK 0x4
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICA_RECV__SHIFT 0x2
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICA_RECV1_MASK 0x8
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICA_RECV1__SHIFT 0x3
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICB_MASK_MASK 0x10
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICB_MASK__SHIFT 0x4
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICB_PD_DIS_MASK 0x20
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICB_PD_DIS__SHIFT 0x5
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICB_RECV_MASK 0x40
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICB_RECV__SHIFT 0x6
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICB_RECV1_MASK 0x80
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICB_RECV1__SHIFT 0x7
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICC_MASK_MASK 0x100
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICC_MASK__SHIFT 0x8
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICC_PD_DIS_MASK 0x200
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICC_PD_DIS__SHIFT 0x9
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICC_RECV_MASK 0x400
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICC_RECV__SHIFT 0xa
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICC_RECV1_MASK 0x800
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICC_RECV1__SHIFT 0xb
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICD_MASK_MASK 0x1000
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICD_MASK__SHIFT 0xc
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICD_PD_DIS_MASK 0x2000
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICD_PD_DIS__SHIFT 0xd
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICD_RECV_MASK 0x4000
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICD_RECV__SHIFT 0xe
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICD_RECV1_MASK 0x8000
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICD_RECV1__SHIFT 0xf
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICE_MASK_MASK 0x10000
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICE_MASK__SHIFT 0x10
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICE_PD_DIS_MASK 0x20000
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICE_PD_DIS__SHIFT 0x11
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICE_RECV_MASK 0x40000
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICE_RECV__SHIFT 0x12
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICE_RECV1_MASK 0x80000
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICE_RECV1__SHIFT 0x13
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICF_MASK_MASK 0x100000
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICF_MASK__SHIFT 0x14
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICF_PD_DIS_MASK 0x200000
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICF_PD_DIS__SHIFT 0x15
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICF_RECV_MASK 0x400000
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICF_RECV__SHIFT 0x16
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICF_RECV1_MASK 0x800000
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICF_RECV1__SHIFT 0x17
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICG_MASK_MASK 0x1000000
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICG_MASK__SHIFT 0x18
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICG_PD_DIS_MASK 0x2000000
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICG_PD_DIS__SHIFT 0x19
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICG_RECV_MASK 0x4000000
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICG_RECV__SHIFT 0x1a
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICG_RECV1_MASK 0x8000000
+#define DC_GPIO_GENERIC_MASK__DC_GPIO_GENERICG_RECV1__SHIFT 0x1b
+#define DC_GPIO_GENERIC_A__DC_GPIO_GENERICA_A_MASK 0x1
+#define DC_GPIO_GENERIC_A__DC_GPIO_GENERICA_A__SHIFT 0x0
+#define DC_GPIO_GENERIC_A__DC_GPIO_GENERICB_A_MASK 0x100
+#define DC_GPIO_GENERIC_A__DC_GPIO_GENERICB_A__SHIFT 0x8
+#define DC_GPIO_GENERIC_A__DC_GPIO_GENERICC_A_MASK 0x10000
+#define DC_GPIO_GENERIC_A__DC_GPIO_GENERICC_A__SHIFT 0x10
+#define DC_GPIO_GENERIC_A__DC_GPIO_GENERICD_A_MASK 0x100000
+#define DC_GPIO_GENERIC_A__DC_GPIO_GENERICD_A__SHIFT 0x14
+#define DC_GPIO_GENERIC_A__DC_GPIO_GENERICE_A_MASK 0x200000
+#define DC_GPIO_GENERIC_A__DC_GPIO_GENERICE_A__SHIFT 0x15
+#define DC_GPIO_GENERIC_A__DC_GPIO_GENERICF_A_MASK 0x400000
+#define DC_GPIO_GENERIC_A__DC_GPIO_GENERICF_A__SHIFT 0x16
+#define DC_GPIO_GENERIC_A__DC_GPIO_GENERICG_A_MASK 0x800000
+#define DC_GPIO_GENERIC_A__DC_GPIO_GENERICG_A__SHIFT 0x17
+#define DC_GPIO_GENERIC_EN__DC_GPIO_GENERICA_EN_MASK 0x1
+#define DC_GPIO_GENERIC_EN__DC_GPIO_GENERICA_EN__SHIFT 0x0
+#define DC_GPIO_GENERIC_EN__DC_GPIO_GENERICB_EN_MASK 0x100
+#define DC_GPIO_GENERIC_EN__DC_GPIO_GENERICB_EN__SHIFT 0x8
+#define DC_GPIO_GENERIC_EN__DC_GPIO_GENERICC_EN_MASK 0x10000
+#define DC_GPIO_GENERIC_EN__DC_GPIO_GENERICC_EN__SHIFT 0x10
+#define DC_GPIO_GENERIC_EN__DC_GPIO_GENERICD_EN_MASK 0x100000
+#define DC_GPIO_GENERIC_EN__DC_GPIO_GENERICD_EN__SHIFT 0x14
+#define DC_GPIO_GENERIC_EN__DC_GPIO_GENERICE_EN_MASK 0x200000
+#define DC_GPIO_GENERIC_EN__DC_GPIO_GENERICE_EN__SHIFT 0x15
+#define DC_GPIO_GENERIC_EN__DC_GPIO_GENERICF_EN_MASK 0x400000
+#define DC_GPIO_GENERIC_EN__DC_GPIO_GENERICF_EN__SHIFT 0x16
+#define DC_GPIO_GENERIC_EN__DC_GPIO_GENERICG_EN_MASK 0x800000
+#define DC_GPIO_GENERIC_EN__DC_GPIO_GENERICG_EN__SHIFT 0x17
+#define DC_GPIO_GENERIC_Y__DC_GPIO_GENERICA_Y_MASK 0x1
+#define DC_GPIO_GENERIC_Y__DC_GPIO_GENERICA_Y__SHIFT 0x0
+#define DC_GPIO_GENERIC_Y__DC_GPIO_GENERICB_Y_MASK 0x100
+#define DC_GPIO_GENERIC_Y__DC_GPIO_GENERICB_Y__SHIFT 0x8
+#define DC_GPIO_GENERIC_Y__DC_GPIO_GENERICC_Y_MASK 0x10000
+#define DC_GPIO_GENERIC_Y__DC_GPIO_GENERICC_Y__SHIFT 0x10
+#define DC_GPIO_GENERIC_Y__DC_GPIO_GENERICD_Y_MASK 0x100000
+#define DC_GPIO_GENERIC_Y__DC_GPIO_GENERICD_Y__SHIFT 0x14
+#define DC_GPIO_GENERIC_Y__DC_GPIO_GENERICE_Y_MASK 0x200000
+#define DC_GPIO_GENERIC_Y__DC_GPIO_GENERICE_Y__SHIFT 0x15
+#define DC_GPIO_GENERIC_Y__DC_GPIO_GENERICF_Y_MASK 0x400000
+#define DC_GPIO_GENERIC_Y__DC_GPIO_GENERICF_Y__SHIFT 0x16
+#define DC_GPIO_GENERIC_Y__DC_GPIO_GENERICG_Y_MASK 0x800000
+#define DC_GPIO_GENERIC_Y__DC_GPIO_GENERICG_Y__SHIFT 0x17
+#define DC_GPIO_DDC1_MASK__DC_GPIO_DDC1CLK_MASK_MASK 0x1
+#define DC_GPIO_DDC1_MASK__DC_GPIO_DDC1CLK_MASK__SHIFT 0x0
+#define DC_GPIO_DDC1_MASK__DC_GPIO_DDC1CLK_PD_EN_MASK 0x10
+#define DC_GPIO_DDC1_MASK__DC_GPIO_DDC1CLK_PD_EN__SHIFT 0x4
+#define DC_GPIO_DDC1_MASK__DC_GPIO_DDC1CLK_RECV_MASK 0x40
+#define DC_GPIO_DDC1_MASK__DC_GPIO_DDC1CLK_RECV__SHIFT 0x6
+#define DC_GPIO_DDC1_MASK__DC_GPIO_DDC1CLK_RECV1_MASK 0x80
+#define DC_GPIO_DDC1_MASK__DC_GPIO_DDC1CLK_RECV1__SHIFT 0x7
+#define DC_GPIO_DDC1_MASK__DC_GPIO_DDC1DATA_MASK_MASK 0x100
+#define DC_GPIO_DDC1_MASK__DC_GPIO_DDC1DATA_MASK__SHIFT 0x8
+#define DC_GPIO_DDC1_MASK__DC_GPIO_DDC1DATA_PD_EN_MASK 0x1000
+#define DC_GPIO_DDC1_MASK__DC_GPIO_DDC1DATA_PD_EN__SHIFT 0xc
+#define DC_GPIO_DDC1_MASK__DC_GPIO_DDC1DATA_RECV_MASK 0x4000
+#define DC_GPIO_DDC1_MASK__DC_GPIO_DDC1DATA_RECV__SHIFT 0xe
+#define DC_GPIO_DDC1_MASK__DC_GPIO_DDC1DATA_RECV1_MASK 0x8000
+#define DC_GPIO_DDC1_MASK__DC_GPIO_DDC1DATA_RECV1__SHIFT 0xf
+#define DC_GPIO_DDC1_MASK__AUX_PAD1_MODE_MASK 0x10000
+#define DC_GPIO_DDC1_MASK__AUX_PAD1_MODE__SHIFT 0x10
+#define DC_GPIO_DDC1_MASK__AUX1_POL_MASK 0x100000
+#define DC_GPIO_DDC1_MASK__AUX1_POL__SHIFT 0x14
+#define DC_GPIO_DDC1_MASK__ALLOW_HW_DDC1_PD_EN_MASK 0x400000
+#define DC_GPIO_DDC1_MASK__ALLOW_HW_DDC1_PD_EN__SHIFT 0x16
+#define DC_GPIO_DDC1_MASK__DC_GPIO_DDC1CLK_STR_MASK 0xf000000
+#define DC_GPIO_DDC1_MASK__DC_GPIO_DDC1CLK_STR__SHIFT 0x18
+#define DC_GPIO_DDC1_MASK__DC_GPIO_DDC1DATA_STR_MASK 0xf0000000
+#define DC_GPIO_DDC1_MASK__DC_GPIO_DDC1DATA_STR__SHIFT 0x1c
+#define DC_GPIO_DDC1_A__DC_GPIO_DDC1CLK_A_MASK 0x1
+#define DC_GPIO_DDC1_A__DC_GPIO_DDC1CLK_A__SHIFT 0x0
+#define DC_GPIO_DDC1_A__DC_GPIO_DDC1DATA_A_MASK 0x100
+#define DC_GPIO_DDC1_A__DC_GPIO_DDC1DATA_A__SHIFT 0x8
+#define DC_GPIO_DDC1_EN__DC_GPIO_DDC1CLK_EN_MASK 0x1
+#define DC_GPIO_DDC1_EN__DC_GPIO_DDC1CLK_EN__SHIFT 0x0
+#define DC_GPIO_DDC1_EN__DC_GPIO_DDC1DATA_EN_MASK 0x100
+#define DC_GPIO_DDC1_EN__DC_GPIO_DDC1DATA_EN__SHIFT 0x8
+#define DC_GPIO_DDC1_Y__DC_GPIO_DDC1CLK_Y_MASK 0x1
+#define DC_GPIO_DDC1_Y__DC_GPIO_DDC1CLK_Y__SHIFT 0x0
+#define DC_GPIO_DDC1_Y__DC_GPIO_DDC1DATA_Y_MASK 0x100
+#define DC_GPIO_DDC1_Y__DC_GPIO_DDC1DATA_Y__SHIFT 0x8
+#define DC_GPIO_DDC2_MASK__DC_GPIO_DDC2CLK_MASK_MASK 0x1
+#define DC_GPIO_DDC2_MASK__DC_GPIO_DDC2CLK_MASK__SHIFT 0x0
+#define DC_GPIO_DDC2_MASK__DC_GPIO_DDC2CLK_PD_EN_MASK 0x10
+#define DC_GPIO_DDC2_MASK__DC_GPIO_DDC2CLK_PD_EN__SHIFT 0x4
+#define DC_GPIO_DDC2_MASK__DC_GPIO_DDC2CLK_RECV_MASK 0x40
+#define DC_GPIO_DDC2_MASK__DC_GPIO_DDC2CLK_RECV__SHIFT 0x6
+#define DC_GPIO_DDC2_MASK__DC_GPIO_DDC2CLK_RECV1_MASK 0x80
+#define DC_GPIO_DDC2_MASK__DC_GPIO_DDC2CLK_RECV1__SHIFT 0x7
+#define DC_GPIO_DDC2_MASK__DC_GPIO_DDC2DATA_MASK_MASK 0x100
+#define DC_GPIO_DDC2_MASK__DC_GPIO_DDC2DATA_MASK__SHIFT 0x8
+#define DC_GPIO_DDC2_MASK__DC_GPIO_DDC2DATA_PD_EN_MASK 0x1000
+#define DC_GPIO_DDC2_MASK__DC_GPIO_DDC2DATA_PD_EN__SHIFT 0xc
+#define DC_GPIO_DDC2_MASK__DC_GPIO_DDC2DATA_RECV_MASK 0x4000
+#define DC_GPIO_DDC2_MASK__DC_GPIO_DDC2DATA_RECV__SHIFT 0xe
+#define DC_GPIO_DDC2_MASK__DC_GPIO_DDC2DATA_RECV1_MASK 0x8000
+#define DC_GPIO_DDC2_MASK__DC_GPIO_DDC2DATA_RECV1__SHIFT 0xf
+#define DC_GPIO_DDC2_MASK__AUX_PAD2_MODE_MASK 0x10000
+#define DC_GPIO_DDC2_MASK__AUX_PAD2_MODE__SHIFT 0x10
+#define DC_GPIO_DDC2_MASK__AUX2_POL_MASK 0x100000
+#define DC_GPIO_DDC2_MASK__AUX2_POL__SHIFT 0x14
+#define DC_GPIO_DDC2_MASK__ALLOW_HW_DDC2_PD_EN_MASK 0x400000
+#define DC_GPIO_DDC2_MASK__ALLOW_HW_DDC2_PD_EN__SHIFT 0x16
+#define DC_GPIO_DDC2_MASK__DC_GPIO_DDC2CLK_STR_MASK 0xf000000
+#define DC_GPIO_DDC2_MASK__DC_GPIO_DDC2CLK_STR__SHIFT 0x18
+#define DC_GPIO_DDC2_MASK__DC_GPIO_DDC2DATA_STR_MASK 0xf0000000
+#define DC_GPIO_DDC2_MASK__DC_GPIO_DDC2DATA_STR__SHIFT 0x1c
+#define DC_GPIO_DDC2_A__DC_GPIO_DDC2CLK_A_MASK 0x1
+#define DC_GPIO_DDC2_A__DC_GPIO_DDC2CLK_A__SHIFT 0x0
+#define DC_GPIO_DDC2_A__DC_GPIO_DDC2DATA_A_MASK 0x100
+#define DC_GPIO_DDC2_A__DC_GPIO_DDC2DATA_A__SHIFT 0x8
+#define DC_GPIO_DDC2_EN__DC_GPIO_DDC2CLK_EN_MASK 0x1
+#define DC_GPIO_DDC2_EN__DC_GPIO_DDC2CLK_EN__SHIFT 0x0
+#define DC_GPIO_DDC2_EN__DC_GPIO_DDC2DATA_EN_MASK 0x100
+#define DC_GPIO_DDC2_EN__DC_GPIO_DDC2DATA_EN__SHIFT 0x8
+#define DC_GPIO_DDC2_Y__DC_GPIO_DDC2CLK_Y_MASK 0x1
+#define DC_GPIO_DDC2_Y__DC_GPIO_DDC2CLK_Y__SHIFT 0x0
+#define DC_GPIO_DDC2_Y__DC_GPIO_DDC2DATA_Y_MASK 0x100
+#define DC_GPIO_DDC2_Y__DC_GPIO_DDC2DATA_Y__SHIFT 0x8
+#define DC_GPIO_DDC3_MASK__DC_GPIO_DDC3CLK_MASK_MASK 0x1
+#define DC_GPIO_DDC3_MASK__DC_GPIO_DDC3CLK_MASK__SHIFT 0x0
+#define DC_GPIO_DDC3_MASK__DC_GPIO_DDC3CLK_PD_EN_MASK 0x10
+#define DC_GPIO_DDC3_MASK__DC_GPIO_DDC3CLK_PD_EN__SHIFT 0x4
+#define DC_GPIO_DDC3_MASK__DC_GPIO_DDC3CLK_RECV_MASK 0x40
+#define DC_GPIO_DDC3_MASK__DC_GPIO_DDC3CLK_RECV__SHIFT 0x6
+#define DC_GPIO_DDC3_MASK__DC_GPIO_DDC3CLK_RECV1_MASK 0x80
+#define DC_GPIO_DDC3_MASK__DC_GPIO_DDC3CLK_RECV1__SHIFT 0x7
+#define DC_GPIO_DDC3_MASK__DC_GPIO_DDC3DATA_MASK_MASK 0x100
+#define DC_GPIO_DDC3_MASK__DC_GPIO_DDC3DATA_MASK__SHIFT 0x8
+#define DC_GPIO_DDC3_MASK__DC_GPIO_DDC3DATA_PD_EN_MASK 0x1000
+#define DC_GPIO_DDC3_MASK__DC_GPIO_DDC3DATA_PD_EN__SHIFT 0xc
+#define DC_GPIO_DDC3_MASK__DC_GPIO_DDC3DATA_RECV_MASK 0x4000
+#define DC_GPIO_DDC3_MASK__DC_GPIO_DDC3DATA_RECV__SHIFT 0xe
+#define DC_GPIO_DDC3_MASK__DC_GPIO_DDC3DATA_RECV1_MASK 0x8000
+#define DC_GPIO_DDC3_MASK__DC_GPIO_DDC3DATA_RECV1__SHIFT 0xf
+#define DC_GPIO_DDC3_MASK__AUX_PAD3_MODE_MASK 0x10000
+#define DC_GPIO_DDC3_MASK__AUX_PAD3_MODE__SHIFT 0x10
+#define DC_GPIO_DDC3_MASK__AUX3_POL_MASK 0x100000
+#define DC_GPIO_DDC3_MASK__AUX3_POL__SHIFT 0x14
+#define DC_GPIO_DDC3_MASK__ALLOW_HW_DDC3_PD_EN_MASK 0x400000
+#define DC_GPIO_DDC3_MASK__ALLOW_HW_DDC3_PD_EN__SHIFT 0x16
+#define DC_GPIO_DDC3_MASK__DC_GPIO_DDC3CLK_STR_MASK 0xf000000
+#define DC_GPIO_DDC3_MASK__DC_GPIO_DDC3CLK_STR__SHIFT 0x18
+#define DC_GPIO_DDC3_MASK__DC_GPIO_DDC3DATA_STR_MASK 0xf0000000
+#define DC_GPIO_DDC3_MASK__DC_GPIO_DDC3DATA_STR__SHIFT 0x1c
+#define DC_GPIO_DDC3_A__DC_GPIO_DDC3CLK_A_MASK 0x1
+#define DC_GPIO_DDC3_A__DC_GPIO_DDC3CLK_A__SHIFT 0x0
+#define DC_GPIO_DDC3_A__DC_GPIO_DDC3DATA_A_MASK 0x100
+#define DC_GPIO_DDC3_A__DC_GPIO_DDC3DATA_A__SHIFT 0x8
+#define DC_GPIO_DDC3_EN__DC_GPIO_DDC3CLK_EN_MASK 0x1
+#define DC_GPIO_DDC3_EN__DC_GPIO_DDC3CLK_EN__SHIFT 0x0
+#define DC_GPIO_DDC3_EN__DC_GPIO_DDC3DATA_EN_MASK 0x100
+#define DC_GPIO_DDC3_EN__DC_GPIO_DDC3DATA_EN__SHIFT 0x8
+#define DC_GPIO_DDC3_Y__DC_GPIO_DDC3CLK_Y_MASK 0x1
+#define DC_GPIO_DDC3_Y__DC_GPIO_DDC3CLK_Y__SHIFT 0x0
+#define DC_GPIO_DDC3_Y__DC_GPIO_DDC3DATA_Y_MASK 0x100
+#define DC_GPIO_DDC3_Y__DC_GPIO_DDC3DATA_Y__SHIFT 0x8
+#define DC_GPIO_DDC4_MASK__DC_GPIO_DDC4CLK_MASK_MASK 0x1
+#define DC_GPIO_DDC4_MASK__DC_GPIO_DDC4CLK_MASK__SHIFT 0x0
+#define DC_GPIO_DDC4_MASK__DC_GPIO_DDC4CLK_PD_EN_MASK 0x10
+#define DC_GPIO_DDC4_MASK__DC_GPIO_DDC4CLK_PD_EN__SHIFT 0x4
+#define DC_GPIO_DDC4_MASK__DC_GPIO_DDC4CLK_RECV_MASK 0x40
+#define DC_GPIO_DDC4_MASK__DC_GPIO_DDC4CLK_RECV__SHIFT 0x6
+#define DC_GPIO_DDC4_MASK__DC_GPIO_DDC4CLK_RECV1_MASK 0x80
+#define DC_GPIO_DDC4_MASK__DC_GPIO_DDC4CLK_RECV1__SHIFT 0x7
+#define DC_GPIO_DDC4_MASK__DC_GPIO_DDC4DATA_MASK_MASK 0x100
+#define DC_GPIO_DDC4_MASK__DC_GPIO_DDC4DATA_MASK__SHIFT 0x8
+#define DC_GPIO_DDC4_MASK__DC_GPIO_DDC4DATA_PD_EN_MASK 0x1000
+#define DC_GPIO_DDC4_MASK__DC_GPIO_DDC4DATA_PD_EN__SHIFT 0xc
+#define DC_GPIO_DDC4_MASK__DC_GPIO_DDC4DATA_RECV_MASK 0x4000
+#define DC_GPIO_DDC4_MASK__DC_GPIO_DDC4DATA_RECV__SHIFT 0xe
+#define DC_GPIO_DDC4_MASK__DC_GPIO_DDC4DATA_RECV1_MASK 0x8000
+#define DC_GPIO_DDC4_MASK__DC_GPIO_DDC4DATA_RECV1__SHIFT 0xf
+#define DC_GPIO_DDC4_MASK__AUX_PAD4_MODE_MASK 0x10000
+#define DC_GPIO_DDC4_MASK__AUX_PAD4_MODE__SHIFT 0x10
+#define DC_GPIO_DDC4_MASK__AUX4_POL_MASK 0x100000
+#define DC_GPIO_DDC4_MASK__AUX4_POL__SHIFT 0x14
+#define DC_GPIO_DDC4_MASK__ALLOW_HW_DDC4_PD_EN_MASK 0x400000
+#define DC_GPIO_DDC4_MASK__ALLOW_HW_DDC4_PD_EN__SHIFT 0x16
+#define DC_GPIO_DDC4_MASK__DC_GPIO_DDC4CLK_STR_MASK 0xf000000
+#define DC_GPIO_DDC4_MASK__DC_GPIO_DDC4CLK_STR__SHIFT 0x18
+#define DC_GPIO_DDC4_MASK__DC_GPIO_DDC4DATA_STR_MASK 0xf0000000
+#define DC_GPIO_DDC4_MASK__DC_GPIO_DDC4DATA_STR__SHIFT 0x1c
+#define DC_GPIO_DDC4_A__DC_GPIO_DDC4CLK_A_MASK 0x1
+#define DC_GPIO_DDC4_A__DC_GPIO_DDC4CLK_A__SHIFT 0x0
+#define DC_GPIO_DDC4_A__DC_GPIO_DDC4DATA_A_MASK 0x100
+#define DC_GPIO_DDC4_A__DC_GPIO_DDC4DATA_A__SHIFT 0x8
+#define DC_GPIO_DDC4_EN__DC_GPIO_DDC4CLK_EN_MASK 0x1
+#define DC_GPIO_DDC4_EN__DC_GPIO_DDC4CLK_EN__SHIFT 0x0
+#define DC_GPIO_DDC4_EN__DC_GPIO_DDC4DATA_EN_MASK 0x100
+#define DC_GPIO_DDC4_EN__DC_GPIO_DDC4DATA_EN__SHIFT 0x8
+#define DC_GPIO_DDC4_Y__DC_GPIO_DDC4CLK_Y_MASK 0x1
+#define DC_GPIO_DDC4_Y__DC_GPIO_DDC4CLK_Y__SHIFT 0x0
+#define DC_GPIO_DDC4_Y__DC_GPIO_DDC4DATA_Y_MASK 0x100
+#define DC_GPIO_DDC4_Y__DC_GPIO_DDC4DATA_Y__SHIFT 0x8
+#define DC_GPIO_DDC5_MASK__DC_GPIO_DDC5CLK_MASK_MASK 0x1
+#define DC_GPIO_DDC5_MASK__DC_GPIO_DDC5CLK_MASK__SHIFT 0x0
+#define DC_GPIO_DDC5_MASK__DC_GPIO_DDC5CLK_PD_EN_MASK 0x10
+#define DC_GPIO_DDC5_MASK__DC_GPIO_DDC5CLK_PD_EN__SHIFT 0x4
+#define DC_GPIO_DDC5_MASK__DC_GPIO_DDC5CLK_RECV_MASK 0x40
+#define DC_GPIO_DDC5_MASK__DC_GPIO_DDC5CLK_RECV__SHIFT 0x6
+#define DC_GPIO_DDC5_MASK__DC_GPIO_DDC5CLK_RECV1_MASK 0x80
+#define DC_GPIO_DDC5_MASK__DC_GPIO_DDC5CLK_RECV1__SHIFT 0x7
+#define DC_GPIO_DDC5_MASK__DC_GPIO_DDC5DATA_MASK_MASK 0x100
+#define DC_GPIO_DDC5_MASK__DC_GPIO_DDC5DATA_MASK__SHIFT 0x8
+#define DC_GPIO_DDC5_MASK__DC_GPIO_DDC5DATA_PD_EN_MASK 0x1000
+#define DC_GPIO_DDC5_MASK__DC_GPIO_DDC5DATA_PD_EN__SHIFT 0xc
+#define DC_GPIO_DDC5_MASK__DC_GPIO_DDC5DATA_RECV_MASK 0x4000
+#define DC_GPIO_DDC5_MASK__DC_GPIO_DDC5DATA_RECV__SHIFT 0xe
+#define DC_GPIO_DDC5_MASK__DC_GPIO_DDC5DATA_RECV1_MASK 0x8000
+#define DC_GPIO_DDC5_MASK__DC_GPIO_DDC5DATA_RECV1__SHIFT 0xf
+#define DC_GPIO_DDC5_MASK__AUX_PAD5_MODE_MASK 0x10000
+#define DC_GPIO_DDC5_MASK__AUX_PAD5_MODE__SHIFT 0x10
+#define DC_GPIO_DDC5_MASK__AUX5_POL_MASK 0x100000
+#define DC_GPIO_DDC5_MASK__AUX5_POL__SHIFT 0x14
+#define DC_GPIO_DDC5_MASK__ALLOW_HW_DDC5_PD_EN_MASK 0x400000
+#define DC_GPIO_DDC5_MASK__ALLOW_HW_DDC5_PD_EN__SHIFT 0x16
+#define DC_GPIO_DDC5_MASK__DC_GPIO_DDC5CLK_STR_MASK 0xf000000
+#define DC_GPIO_DDC5_MASK__DC_GPIO_DDC5CLK_STR__SHIFT 0x18
+#define DC_GPIO_DDC5_MASK__DC_GPIO_DDC5DATA_STR_MASK 0xf0000000
+#define DC_GPIO_DDC5_MASK__DC_GPIO_DDC5DATA_STR__SHIFT 0x1c
+#define DC_GPIO_DDC5_A__DC_GPIO_DDC5CLK_A_MASK 0x1
+#define DC_GPIO_DDC5_A__DC_GPIO_DDC5CLK_A__SHIFT 0x0
+#define DC_GPIO_DDC5_A__DC_GPIO_DDC5DATA_A_MASK 0x100
+#define DC_GPIO_DDC5_A__DC_GPIO_DDC5DATA_A__SHIFT 0x8
+#define DC_GPIO_DDC5_EN__DC_GPIO_DDC5CLK_EN_MASK 0x1
+#define DC_GPIO_DDC5_EN__DC_GPIO_DDC5CLK_EN__SHIFT 0x0
+#define DC_GPIO_DDC5_EN__DC_GPIO_DDC5DATA_EN_MASK 0x100
+#define DC_GPIO_DDC5_EN__DC_GPIO_DDC5DATA_EN__SHIFT 0x8
+#define DC_GPIO_DDC5_Y__DC_GPIO_DDC5CLK_Y_MASK 0x1
+#define DC_GPIO_DDC5_Y__DC_GPIO_DDC5CLK_Y__SHIFT 0x0
+#define DC_GPIO_DDC5_Y__DC_GPIO_DDC5DATA_Y_MASK 0x100
+#define DC_GPIO_DDC5_Y__DC_GPIO_DDC5DATA_Y__SHIFT 0x8
+#define DC_GPIO_DDC6_MASK__DC_GPIO_DDC6CLK_MASK_MASK 0x1
+#define DC_GPIO_DDC6_MASK__DC_GPIO_DDC6CLK_MASK__SHIFT 0x0
+#define DC_GPIO_DDC6_MASK__DC_GPIO_DDC6CLK_PD_EN_MASK 0x10
+#define DC_GPIO_DDC6_MASK__DC_GPIO_DDC6CLK_PD_EN__SHIFT 0x4
+#define DC_GPIO_DDC6_MASK__DC_GPIO_DDC6CLK_RECV_MASK 0x40
+#define DC_GPIO_DDC6_MASK__DC_GPIO_DDC6CLK_RECV__SHIFT 0x6
+#define DC_GPIO_DDC6_MASK__DC_GPIO_DDC6CLK_RECV1_MASK 0x80
+#define DC_GPIO_DDC6_MASK__DC_GPIO_DDC6CLK_RECV1__SHIFT 0x7
+#define DC_GPIO_DDC6_MASK__DC_GPIO_DDC6DATA_MASK_MASK 0x100
+#define DC_GPIO_DDC6_MASK__DC_GPIO_DDC6DATA_MASK__SHIFT 0x8
+#define DC_GPIO_DDC6_MASK__DC_GPIO_DDC6DATA_PD_EN_MASK 0x1000
+#define DC_GPIO_DDC6_MASK__DC_GPIO_DDC6DATA_PD_EN__SHIFT 0xc
+#define DC_GPIO_DDC6_MASK__DC_GPIO_DDC6DATA_RECV_MASK 0x4000
+#define DC_GPIO_DDC6_MASK__DC_GPIO_DDC6DATA_RECV__SHIFT 0xe
+#define DC_GPIO_DDC6_MASK__DC_GPIO_DDC6DATA_RECV1_MASK 0x8000
+#define DC_GPIO_DDC6_MASK__DC_GPIO_DDC6DATA_RECV1__SHIFT 0xf
+#define DC_GPIO_DDC6_MASK__AUX_PAD6_MODE_MASK 0x10000
+#define DC_GPIO_DDC6_MASK__AUX_PAD6_MODE__SHIFT 0x10
+#define DC_GPIO_DDC6_MASK__AUX6_POL_MASK 0x100000
+#define DC_GPIO_DDC6_MASK__AUX6_POL__SHIFT 0x14
+#define DC_GPIO_DDC6_MASK__ALLOW_HW_DDC6_PD_EN_MASK 0x400000
+#define DC_GPIO_DDC6_MASK__ALLOW_HW_DDC6_PD_EN__SHIFT 0x16
+#define DC_GPIO_DDC6_MASK__DC_GPIO_DDC6CLK_STR_MASK 0xf000000
+#define DC_GPIO_DDC6_MASK__DC_GPIO_DDC6CLK_STR__SHIFT 0x18
+#define DC_GPIO_DDC6_MASK__DC_GPIO_DDC6DATA_STR_MASK 0xf0000000
+#define DC_GPIO_DDC6_MASK__DC_GPIO_DDC6DATA_STR__SHIFT 0x1c
+#define DC_GPIO_DDC6_A__DC_GPIO_DDC6CLK_A_MASK 0x1
+#define DC_GPIO_DDC6_A__DC_GPIO_DDC6CLK_A__SHIFT 0x0
+#define DC_GPIO_DDC6_A__DC_GPIO_DDC6DATA_A_MASK 0x100
+#define DC_GPIO_DDC6_A__DC_GPIO_DDC6DATA_A__SHIFT 0x8
+#define DC_GPIO_DDC6_EN__DC_GPIO_DDC6CLK_EN_MASK 0x1
+#define DC_GPIO_DDC6_EN__DC_GPIO_DDC6CLK_EN__SHIFT 0x0
+#define DC_GPIO_DDC6_EN__DC_GPIO_DDC6DATA_EN_MASK 0x100
+#define DC_GPIO_DDC6_EN__DC_GPIO_DDC6DATA_EN__SHIFT 0x8
+#define DC_GPIO_DDC6_Y__DC_GPIO_DDC6CLK_Y_MASK 0x1
+#define DC_GPIO_DDC6_Y__DC_GPIO_DDC6CLK_Y__SHIFT 0x0
+#define DC_GPIO_DDC6_Y__DC_GPIO_DDC6DATA_Y_MASK 0x100
+#define DC_GPIO_DDC6_Y__DC_GPIO_DDC6DATA_Y__SHIFT 0x8
+#define DC_GPIO_DDCVGA_MASK__DC_GPIO_DDCVGACLK_MASK_MASK 0x1
+#define DC_GPIO_DDCVGA_MASK__DC_GPIO_DDCVGACLK_MASK__SHIFT 0x0
+#define DC_GPIO_DDCVGA_MASK__DC_GPIO_DDCVGACLK_RECV_MASK 0x40
+#define DC_GPIO_DDCVGA_MASK__DC_GPIO_DDCVGACLK_RECV__SHIFT 0x6
+#define DC_GPIO_DDCVGA_MASK__DC_GPIO_DDCVGACLK_RECV1_MASK 0x80
+#define DC_GPIO_DDCVGA_MASK__DC_GPIO_DDCVGACLK_RECV1__SHIFT 0x7
+#define DC_GPIO_DDCVGA_MASK__DC_GPIO_DDCVGADATA_MASK_MASK 0x100
+#define DC_GPIO_DDCVGA_MASK__DC_GPIO_DDCVGADATA_MASK__SHIFT 0x8
+#define DC_GPIO_DDCVGA_MASK__DC_GPIO_DDCVGADATA_PD_EN_MASK 0x1000
+#define DC_GPIO_DDCVGA_MASK__DC_GPIO_DDCVGADATA_PD_EN__SHIFT 0xc
+#define DC_GPIO_DDCVGA_MASK__DC_GPIO_DDCVGADATA_RECV_MASK 0x4000
+#define DC_GPIO_DDCVGA_MASK__DC_GPIO_DDCVGADATA_RECV__SHIFT 0xe
+#define DC_GPIO_DDCVGA_MASK__DC_GPIO_DDCVGADATA_RECV1_MASK 0x8000
+#define DC_GPIO_DDCVGA_MASK__DC_GPIO_DDCVGADATA_RECV1__SHIFT 0xf
+#define DC_GPIO_DDCVGA_MASK__AUX_PADVGA_MODE_MASK 0x10000
+#define DC_GPIO_DDCVGA_MASK__AUX_PADVGA_MODE__SHIFT 0x10
+#define DC_GPIO_DDCVGA_MASK__AUXVGA_POL_MASK 0x100000
+#define DC_GPIO_DDCVGA_MASK__AUXVGA_POL__SHIFT 0x14
+#define DC_GPIO_DDCVGA_MASK__ALLOW_HW_DDCVGA_PD_EN_MASK 0x400000
+#define DC_GPIO_DDCVGA_MASK__ALLOW_HW_DDCVGA_PD_EN__SHIFT 0x16
+#define DC_GPIO_DDCVGA_MASK__DC_GPIO_DDCVGACLK_STR_MASK 0xf000000
+#define DC_GPIO_DDCVGA_MASK__DC_GPIO_DDCVGACLK_STR__SHIFT 0x18
+#define DC_GPIO_DDCVGA_MASK__DC_GPIO_DDCVGADATA_STR_MASK 0xf0000000
+#define DC_GPIO_DDCVGA_MASK__DC_GPIO_DDCVGADATA_STR__SHIFT 0x1c
+#define DC_GPIO_DDCVGA_A__DC_GPIO_DDCVGACLK_A_MASK 0x1
+#define DC_GPIO_DDCVGA_A__DC_GPIO_DDCVGACLK_A__SHIFT 0x0
+#define DC_GPIO_DDCVGA_A__DC_GPIO_DDCVGADATA_A_MASK 0x100
+#define DC_GPIO_DDCVGA_A__DC_GPIO_DDCVGADATA_A__SHIFT 0x8
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGACLK_EN_MASK 0x1
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGACLK_EN__SHIFT 0x0
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_EN_MASK 0x100
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_EN__SHIFT 0x8
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_RXSEL_MASK 0x30000
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_RXSEL__SHIFT 0x10
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_SPARE_MASK 0xc0000
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_SPARE__SHIFT 0x12
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_BIASCRTEN_MASK 0x100000
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_BIASCRTEN__SHIFT 0x14
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_CSEL0P9_MASK 0x200000
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_CSEL0P9__SHIFT 0x15
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_CSEL1P1_MASK 0x400000
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_CSEL1P1__SHIFT 0x16
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_COMPSEL_MASK 0x800000
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_COMPSEL__SHIFT 0x17
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_RSEL0P9_MASK 0x1000000
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_RSEL0P9__SHIFT 0x18
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_RSEL1P1_MASK 0x2000000
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_RSEL1P1__SHIFT 0x19
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_SPIKERCEN_MASK 0x4000000
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_SPIKERCEN__SHIFT 0x1a
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_SPIKERCSEL_MASK 0x8000000
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_SPIKERCSEL__SHIFT 0x1b
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_FALLSLEWSEL_MASK 0x30000000
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_FALLSLEWSEL__SHIFT 0x1c
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_RESBIASEN_MASK 0x40000000
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_RESBIASEN__SHIFT 0x1e
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_SLEWN_MASK 0x80000000
+#define DC_GPIO_DDCVGA_EN__DC_GPIO_DDCVGADATA_PAD_SLEWN__SHIFT 0x1f
+#define DC_GPIO_DDCVGA_Y__DC_GPIO_DDCVGACLK_Y_MASK 0x1
+#define DC_GPIO_DDCVGA_Y__DC_GPIO_DDCVGACLK_Y__SHIFT 0x0
+#define DC_GPIO_DDCVGA_Y__DC_GPIO_DDCVGADATA_Y_MASK 0x100
+#define DC_GPIO_DDCVGA_Y__DC_GPIO_DDCVGADATA_Y__SHIFT 0x8
+#define DC_GPIO_SYNCA_MASK__DC_GPIO_HSYNCA_MASK_MASK 0x1
+#define DC_GPIO_SYNCA_MASK__DC_GPIO_HSYNCA_MASK__SHIFT 0x0
+#define DC_GPIO_SYNCA_MASK__DC_GPIO_HSYNCA_PD_DIS_MASK 0x10
+#define DC_GPIO_SYNCA_MASK__DC_GPIO_HSYNCA_PD_DIS__SHIFT 0x4
+#define DC_GPIO_SYNCA_MASK__DC_GPIO_HSYNCA_RECV_MASK 0x40
+#define DC_GPIO_SYNCA_MASK__DC_GPIO_HSYNCA_RECV__SHIFT 0x6
+#define DC_GPIO_SYNCA_MASK__DC_GPIO_HSYNCA_RECV1_MASK 0x80
+#define DC_GPIO_SYNCA_MASK__DC_GPIO_HSYNCA_RECV1__SHIFT 0x7
+#define DC_GPIO_SYNCA_MASK__DC_GPIO_VSYNCA_MASK_MASK 0x100
+#define DC_GPIO_SYNCA_MASK__DC_GPIO_VSYNCA_MASK__SHIFT 0x8
+#define DC_GPIO_SYNCA_MASK__DC_GPIO_VSYNCA_PD_DIS_MASK 0x1000
+#define DC_GPIO_SYNCA_MASK__DC_GPIO_VSYNCA_PD_DIS__SHIFT 0xc
+#define DC_GPIO_SYNCA_MASK__DC_GPIO_VSYNCA_RECV_MASK 0x4000
+#define DC_GPIO_SYNCA_MASK__DC_GPIO_VSYNCA_RECV__SHIFT 0xe
+#define DC_GPIO_SYNCA_MASK__DC_GPIO_VSYNCA_RECV1_MASK 0x8000
+#define DC_GPIO_SYNCA_MASK__DC_GPIO_VSYNCA_RECV1__SHIFT 0xf
+#define DC_GPIO_SYNCA_MASK__DC_GPIO_HSYNCA_CRTC_HSYNC_MASK_MASK 0x7000000
+#define DC_GPIO_SYNCA_MASK__DC_GPIO_HSYNCA_CRTC_HSYNC_MASK__SHIFT 0x18
+#define DC_GPIO_SYNCA_MASK__DC_GPIO_VSYNCA_CRTC_VSYNC_MASK_MASK 0x70000000
+#define DC_GPIO_SYNCA_MASK__DC_GPIO_VSYNCA_CRTC_VSYNC_MASK__SHIFT 0x1c
+#define DC_GPIO_SYNCA_A__DC_GPIO_HSYNCA_A_MASK 0x1
+#define DC_GPIO_SYNCA_A__DC_GPIO_HSYNCA_A__SHIFT 0x0
+#define DC_GPIO_SYNCA_A__DC_GPIO_VSYNCA_A_MASK 0x100
+#define DC_GPIO_SYNCA_A__DC_GPIO_VSYNCA_A__SHIFT 0x8
+#define DC_GPIO_SYNCA_EN__DC_GPIO_HSYNCA_EN_MASK 0x1
+#define DC_GPIO_SYNCA_EN__DC_GPIO_HSYNCA_EN__SHIFT 0x0
+#define DC_GPIO_SYNCA_EN__DC_GPIO_VSYNCA_EN_MASK 0x100
+#define DC_GPIO_SYNCA_EN__DC_GPIO_VSYNCA_EN__SHIFT 0x8
+#define DC_GPIO_SYNCA_Y__DC_GPIO_HSYNCA_Y_MASK 0x1
+#define DC_GPIO_SYNCA_Y__DC_GPIO_HSYNCA_Y__SHIFT 0x0
+#define DC_GPIO_SYNCA_Y__DC_GPIO_VSYNCA_Y_MASK 0x100
+#define DC_GPIO_SYNCA_Y__DC_GPIO_VSYNCA_Y__SHIFT 0x8
+#define DC_GPIO_GENLK_MASK__DC_GPIO_GENLK_CLK_MASK_MASK 0x1
+#define DC_GPIO_GENLK_MASK__DC_GPIO_GENLK_CLK_MASK__SHIFT 0x0
+#define DC_GPIO_GENLK_MASK__DC_GPIO_GENLK_CLK_PD_DIS_MASK 0x2
+#define DC_GPIO_GENLK_MASK__DC_GPIO_GENLK_CLK_PD_DIS__SHIFT 0x1
+#define DC_GPIO_GENLK_MASK__DC_GPIO_GENLK_CLK_RECV_MASK 0x4
+#define DC_GPIO_GENLK_MASK__DC_GPIO_GENLK_CLK_RECV__SHIFT 0x2
+#define DC_GPIO_GENLK_MASK__DC_GPIO_GENLK_CLK_PU_EN_MASK 0x8
+#define DC_GPIO_GENLK_MASK__DC_GPIO_GENLK_CLK_PU_EN__SHIFT 0x3
+#define DC_GPIO_GENLK_MASK__DC_GPIO_GENLK_CLK_RECV1_MASK 0x10
+#define DC_GPIO_GENLK_MASK__DC_GPIO_GENLK_CLK_RECV1__SHIFT 0x4
+#define DC_GPIO_GENLK_MASK__DC_GPIO_GENLK_VSYNC_RECV1_MASK 0x20
+#define DC_GPIO_GENLK_MASK__DC_GPIO_GENLK_VSYNC_RECV1__SHIFT 0x5
+#define DC_GPIO_GENLK_MASK__DC_GPIO_GENLK_VSYNC_MASK_MASK 0x100
+#define DC_GPIO_GENLK_MASK__DC_GPIO_GENLK_VSYNC_MASK__SHIFT 0x8
+#define DC_GPIO_GENLK_MASK__DC_GPIO_GENLK_VSYNC_PD_DIS_MASK 0x200
+#define DC_GPIO_GENLK_MASK__DC_GPIO_GENLK_VSYNC_PD_DIS__SHIFT 0x9
+#define DC_GPIO_GENLK_MASK__DC_GPIO_GENLK_VSYNC_RECV_MASK 0x400
+#define DC_GPIO_GENLK_MASK__DC_GPIO_GENLK_VSYNC_RECV__SHIFT 0xa
+#define DC_GPIO_GENLK_MASK__DC_GPIO_GENLK_VSYNC_PU_EN_MASK 0x800
+#define DC_GPIO_GENLK_MASK__DC_GPIO_GENLK_VSYNC_PU_EN__SHIFT 0xb
+#define DC_GPIO_GENLK_MASK__DC_GPIO_SWAPLOCK_A_MASK_MASK 0x10000
+#define DC_GPIO_GENLK_MASK__DC_GPIO_SWAPLOCK_A_MASK__SHIFT 0x10
+#define DC_GPIO_GENLK_MASK__DC_GPIO_SWAPLOCK_A_PD_DIS_MASK 0x20000
+#define DC_GPIO_GENLK_MASK__DC_GPIO_SWAPLOCK_A_PD_DIS__SHIFT 0x11
+#define DC_GPIO_GENLK_MASK__DC_GPIO_SWAPLOCK_A_RECV_MASK 0x40000
+#define DC_GPIO_GENLK_MASK__DC_GPIO_SWAPLOCK_A_RECV__SHIFT 0x12
+#define DC_GPIO_GENLK_MASK__DC_GPIO_SWAPLOCK_A_PU_EN_MASK 0x80000
+#define DC_GPIO_GENLK_MASK__DC_GPIO_SWAPLOCK_A_PU_EN__SHIFT 0x13
+#define DC_GPIO_GENLK_MASK__DC_GPIO_SWAPLOCK_A_RECV1_MASK 0x100000
+#define DC_GPIO_GENLK_MASK__DC_GPIO_SWAPLOCK_A_RECV1__SHIFT 0x14
+#define DC_GPIO_GENLK_MASK__DC_GPIO_SWAPLOCK_B_RECV1_MASK 0x800000
+#define DC_GPIO_GENLK_MASK__DC_GPIO_SWAPLOCK_B_RECV1__SHIFT 0x17
+#define DC_GPIO_GENLK_MASK__DC_GPIO_SWAPLOCK_B_MASK_MASK 0x1000000
+#define DC_GPIO_GENLK_MASK__DC_GPIO_SWAPLOCK_B_MASK__SHIFT 0x18
+#define DC_GPIO_GENLK_MASK__DC_GPIO_SWAPLOCK_B_PD_DIS_MASK 0x2000000
+#define DC_GPIO_GENLK_MASK__DC_GPIO_SWAPLOCK_B_PD_DIS__SHIFT 0x19
+#define DC_GPIO_GENLK_MASK__DC_GPIO_SWAPLOCK_B_RECV_MASK 0x4000000
+#define DC_GPIO_GENLK_MASK__DC_GPIO_SWAPLOCK_B_RECV__SHIFT 0x1a
+#define DC_GPIO_GENLK_MASK__DC_GPIO_SWAPLOCK_B_PU_EN_MASK 0x8000000
+#define DC_GPIO_GENLK_MASK__DC_GPIO_SWAPLOCK_B_PU_EN__SHIFT 0x1b
+#define DC_GPIO_GENLK_A__DC_GPIO_GENLK_CLK_A_MASK 0x1
+#define DC_GPIO_GENLK_A__DC_GPIO_GENLK_CLK_A__SHIFT 0x0
+#define DC_GPIO_GENLK_A__DC_GPIO_GENLK_VSYNC_A_MASK 0x100
+#define DC_GPIO_GENLK_A__DC_GPIO_GENLK_VSYNC_A__SHIFT 0x8
+#define DC_GPIO_GENLK_A__DC_GPIO_SWAPLOCK_A_A_MASK 0x10000
+#define DC_GPIO_GENLK_A__DC_GPIO_SWAPLOCK_A_A__SHIFT 0x10
+#define DC_GPIO_GENLK_A__DC_GPIO_SWAPLOCK_B_A_MASK 0x1000000
+#define DC_GPIO_GENLK_A__DC_GPIO_SWAPLOCK_B_A__SHIFT 0x18
+#define DC_GPIO_GENLK_EN__DC_GPIO_GENLK_CLK_EN_MASK 0x1
+#define DC_GPIO_GENLK_EN__DC_GPIO_GENLK_CLK_EN__SHIFT 0x0
+#define DC_GPIO_GENLK_EN__DC_GPIO_GENLK_VSYNC_EN_MASK 0x100
+#define DC_GPIO_GENLK_EN__DC_GPIO_GENLK_VSYNC_EN__SHIFT 0x8
+#define DC_GPIO_GENLK_EN__DC_GPIO_SWAPLOCK_A_EN_MASK 0x10000
+#define DC_GPIO_GENLK_EN__DC_GPIO_SWAPLOCK_A_EN__SHIFT 0x10
+#define DC_GPIO_GENLK_EN__DC_GPIO_SWAPLOCK_B_EN_MASK 0x1000000
+#define DC_GPIO_GENLK_EN__DC_GPIO_SWAPLOCK_B_EN__SHIFT 0x18
+#define DC_GPIO_GENLK_Y__DC_GPIO_GENLK_CLK_Y_MASK 0x1
+#define DC_GPIO_GENLK_Y__DC_GPIO_GENLK_CLK_Y__SHIFT 0x0
+#define DC_GPIO_GENLK_Y__DC_GPIO_GENLK_VSYNC_Y_MASK 0x100
+#define DC_GPIO_GENLK_Y__DC_GPIO_GENLK_VSYNC_Y__SHIFT 0x8
+#define DC_GPIO_GENLK_Y__DC_GPIO_SWAPLOCK_A_Y_MASK 0x10000
+#define DC_GPIO_GENLK_Y__DC_GPIO_SWAPLOCK_A_Y__SHIFT 0x10
+#define DC_GPIO_GENLK_Y__DC_GPIO_SWAPLOCK_B_Y_MASK 0x1000000
+#define DC_GPIO_GENLK_Y__DC_GPIO_SWAPLOCK_B_Y__SHIFT 0x18
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD1_MASK_MASK 0x1
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD1_MASK__SHIFT 0x0
+#define DC_GPIO_HPD_MASK__DC_GPIO_RX_HPD_MASK_MASK 0x2
+#define DC_GPIO_HPD_MASK__DC_GPIO_RX_HPD_MASK__SHIFT 0x1
+#define DC_GPIO_HPD_MASK__DC_GPIO_RX_HPD_PD_DIS_MASK 0x4
+#define DC_GPIO_HPD_MASK__DC_GPIO_RX_HPD_PD_DIS__SHIFT 0x2
+#define DC_GPIO_HPD_MASK__DC_GPIO_RX_HPD_RECV_MASK 0x8
+#define DC_GPIO_HPD_MASK__DC_GPIO_RX_HPD_RECV__SHIFT 0x3
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD1_PD_DIS_MASK 0x10
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD1_PD_DIS__SHIFT 0x4
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD1_RECV1_MASK 0x20
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD1_RECV1__SHIFT 0x5
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD1_RECV_MASK 0x40
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD1_RECV__SHIFT 0x6
+#define DC_GPIO_HPD_MASK__DC_GPIO_RX_HPD_RECV1_MASK 0x80
+#define DC_GPIO_HPD_MASK__DC_GPIO_RX_HPD_RECV1__SHIFT 0x7
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD2_MASK_MASK 0x100
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD2_MASK__SHIFT 0x8
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD2_PD_DIS_MASK 0x200
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD2_PD_DIS__SHIFT 0x9
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD2_RECV_MASK 0x400
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD2_RECV__SHIFT 0xa
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD2_RECV1_MASK 0x800
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD2_RECV1__SHIFT 0xb
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD3_MASK_MASK 0x10000
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD3_MASK__SHIFT 0x10
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD3_PD_DIS_MASK 0x20000
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD3_PD_DIS__SHIFT 0x11
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD3_RECV_MASK 0x40000
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD3_RECV__SHIFT 0x12
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD3_RECV1_MASK 0x80000
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD3_RECV1__SHIFT 0x13
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD4_MASK_MASK 0x100000
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD4_MASK__SHIFT 0x14
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD4_PD_DIS_MASK 0x200000
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD4_PD_DIS__SHIFT 0x15
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD4_RECV_MASK 0x400000
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD4_RECV__SHIFT 0x16
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD4_RECV1_MASK 0x800000
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD4_RECV1__SHIFT 0x17
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD5_MASK_MASK 0x1000000
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD5_MASK__SHIFT 0x18
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD5_PD_DIS_MASK 0x2000000
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD5_PD_DIS__SHIFT 0x19
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD5_RECV_MASK 0x4000000
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD5_RECV__SHIFT 0x1a
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD5_RECV1_MASK 0x8000000
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD5_RECV1__SHIFT 0x1b
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD6_MASK_MASK 0x10000000
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD6_MASK__SHIFT 0x1c
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD6_PD_DIS_MASK 0x20000000
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD6_PD_DIS__SHIFT 0x1d
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD6_RECV_MASK 0x40000000
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD6_RECV__SHIFT 0x1e
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD6_RECV1_MASK 0x80000000
+#define DC_GPIO_HPD_MASK__DC_GPIO_HPD6_RECV1__SHIFT 0x1f
+#define DC_GPIO_HPD_A__DC_GPIO_HPD1_A_MASK 0x1
+#define DC_GPIO_HPD_A__DC_GPIO_HPD1_A__SHIFT 0x0
+#define DC_GPIO_HPD_A__DC_GPIO_HPD2_A_MASK 0x100
+#define DC_GPIO_HPD_A__DC_GPIO_HPD2_A__SHIFT 0x8
+#define DC_GPIO_HPD_A__DC_GPIO_HPD3_A_MASK 0x10000
+#define DC_GPIO_HPD_A__DC_GPIO_HPD3_A__SHIFT 0x10
+#define DC_GPIO_HPD_A__DC_GPIO_HPD4_A_MASK 0x1000000
+#define DC_GPIO_HPD_A__DC_GPIO_HPD4_A__SHIFT 0x18
+#define DC_GPIO_HPD_A__DC_GPIO_HPD5_A_MASK 0x4000000
+#define DC_GPIO_HPD_A__DC_GPIO_HPD5_A__SHIFT 0x1a
+#define DC_GPIO_HPD_A__DC_GPIO_HPD6_A_MASK 0x10000000
+#define DC_GPIO_HPD_A__DC_GPIO_HPD6_A__SHIFT 0x1c
+#define DC_GPIO_HPD_EN__DC_GPIO_HPD1_EN_MASK 0x1
+#define DC_GPIO_HPD_EN__DC_GPIO_HPD1_EN__SHIFT 0x0
+#define DC_GPIO_HPD_EN__HPD1_SCHMEN_PI_MASK 0x2
+#define DC_GPIO_HPD_EN__HPD1_SCHMEN_PI__SHIFT 0x1
+#define DC_GPIO_HPD_EN__HPD1_SLEWNCORE_MASK 0x4
+#define DC_GPIO_HPD_EN__HPD1_SLEWNCORE__SHIFT 0x2
+#define DC_GPIO_HPD_EN__RX_HPD_SCHMEN_PI_MASK 0x8
+#define DC_GPIO_HPD_EN__RX_HPD_SCHMEN_PI__SHIFT 0x3
+#define DC_GPIO_HPD_EN__RX_HPD_SLEWNCORE_MASK 0x10
+#define DC_GPIO_HPD_EN__RX_HPD_SLEWNCORE__SHIFT 0x4
+#define DC_GPIO_HPD_EN__HPD12_SPARE0_MASK 0x20
+#define DC_GPIO_HPD_EN__HPD12_SPARE0__SHIFT 0x5
+#define DC_GPIO_HPD_EN__HPD1_SEL0_MASK 0x40
+#define DC_GPIO_HPD_EN__HPD1_SEL0__SHIFT 0x6
+#define DC_GPIO_HPD_EN__RX_HPD_SEL0_MASK 0x80
+#define DC_GPIO_HPD_EN__RX_HPD_SEL0__SHIFT 0x7
+#define DC_GPIO_HPD_EN__DC_GPIO_HPD2_EN_MASK 0x100
+#define DC_GPIO_HPD_EN__DC_GPIO_HPD2_EN__SHIFT 0x8
+#define DC_GPIO_HPD_EN__HPD2_SCHMEN_PI_MASK 0x200
+#define DC_GPIO_HPD_EN__HPD2_SCHMEN_PI__SHIFT 0x9
+#define DC_GPIO_HPD_EN__HPD12_SPARE1_MASK 0x400
+#define DC_GPIO_HPD_EN__HPD12_SPARE1__SHIFT 0xa
+#define DC_GPIO_HPD_EN__DC_GPIO_HPD3_EN_MASK 0x10000
+#define DC_GPIO_HPD_EN__DC_GPIO_HPD3_EN__SHIFT 0x10
+#define DC_GPIO_HPD_EN__HPD3_SCHMEN_PI_MASK 0x20000
+#define DC_GPIO_HPD_EN__HPD3_SCHMEN_PI__SHIFT 0x11
+#define DC_GPIO_HPD_EN__HPD34_SPARE0_MASK 0x40000
+#define DC_GPIO_HPD_EN__HPD34_SPARE0__SHIFT 0x12
+#define DC_GPIO_HPD_EN__DC_GPIO_HPD4_EN_MASK 0x100000
+#define DC_GPIO_HPD_EN__DC_GPIO_HPD4_EN__SHIFT 0x14
+#define DC_GPIO_HPD_EN__HPD4_SCHMEN_PI_MASK 0x200000
+#define DC_GPIO_HPD_EN__HPD4_SCHMEN_PI__SHIFT 0x15
+#define DC_GPIO_HPD_EN__HPD34_SPARE1_MASK 0x400000
+#define DC_GPIO_HPD_EN__HPD34_SPARE1__SHIFT 0x16
+#define DC_GPIO_HPD_EN__DC_GPIO_HPD5_EN_MASK 0x1000000
+#define DC_GPIO_HPD_EN__DC_GPIO_HPD5_EN__SHIFT 0x18
+#define DC_GPIO_HPD_EN__HPD5_SCHMEN_PI_MASK 0x2000000
+#define DC_GPIO_HPD_EN__HPD5_SCHMEN_PI__SHIFT 0x19
+#define DC_GPIO_HPD_EN__HPD56_SPARE0_MASK 0x4000000
+#define DC_GPIO_HPD_EN__HPD56_SPARE0__SHIFT 0x1a
+#define DC_GPIO_HPD_EN__DC_GPIO_HPD6_EN_MASK 0x10000000
+#define DC_GPIO_HPD_EN__DC_GPIO_HPD6_EN__SHIFT 0x1c
+#define DC_GPIO_HPD_EN__HPD6_SCHMEN_PI_MASK 0x20000000
+#define DC_GPIO_HPD_EN__HPD6_SCHMEN_PI__SHIFT 0x1d
+#define DC_GPIO_HPD_EN__HPD56_SPARE1_MASK 0x40000000
+#define DC_GPIO_HPD_EN__HPD56_SPARE1__SHIFT 0x1e
+#define DC_GPIO_HPD_Y__DC_GPIO_HPD1_Y_MASK 0x1
+#define DC_GPIO_HPD_Y__DC_GPIO_HPD1_Y__SHIFT 0x0
+#define DC_GPIO_HPD_Y__DC_GPIO_HPD2_Y_MASK 0x100
+#define DC_GPIO_HPD_Y__DC_GPIO_HPD2_Y__SHIFT 0x8
+#define DC_GPIO_HPD_Y__DC_GPIO_HPD3_Y_MASK 0x10000
+#define DC_GPIO_HPD_Y__DC_GPIO_HPD3_Y__SHIFT 0x10
+#define DC_GPIO_HPD_Y__DC_GPIO_HPD4_Y_MASK 0x1000000
+#define DC_GPIO_HPD_Y__DC_GPIO_HPD4_Y__SHIFT 0x18
+#define DC_GPIO_HPD_Y__DC_GPIO_HPD5_Y_MASK 0x4000000
+#define DC_GPIO_HPD_Y__DC_GPIO_HPD5_Y__SHIFT 0x1a
+#define DC_GPIO_HPD_Y__DC_GPIO_HPD6_Y_MASK 0x10000000
+#define DC_GPIO_HPD_Y__DC_GPIO_HPD6_Y__SHIFT 0x1c
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_BLON_MASK_MASK 0x1
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_BLON_MASK__SHIFT 0x0
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_BLON_PD_DIS_MASK 0x10
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_BLON_PD_DIS__SHIFT 0x4
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_BLON_RECV_MASK 0x40
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_BLON_RECV__SHIFT 0x6
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_BLON_RECV1_MASK 0x80
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_BLON_RECV1__SHIFT 0x7
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_DIGON_MASK_MASK 0x100
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_DIGON_MASK__SHIFT 0x8
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_DIGON_PD_DIS_MASK 0x1000
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_DIGON_PD_DIS__SHIFT 0xc
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_DIGON_RECV_MASK 0x4000
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_DIGON_RECV__SHIFT 0xe
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_DIGON_RECV1_MASK 0x8000
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_DIGON_RECV1__SHIFT 0xf
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_ENA_BL_MASK_MASK 0x10000
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_ENA_BL_MASK__SHIFT 0x10
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_ENA_BL_PD_DIS_MASK 0x100000
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_ENA_BL_PD_DIS__SHIFT 0x14
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_ENA_BL_RECV_MASK 0x400000
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_ENA_BL_RECV__SHIFT 0x16
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_ENA_BL_RECV1_MASK 0x800000
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_ENA_BL_RECV1__SHIFT 0x17
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_VSYNC_IN_MASK_MASK 0x1000000
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_VSYNC_IN_MASK__SHIFT 0x18
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_VSYNC_IN_PD_DIS_MASK 0x2000000
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_VSYNC_IN_PD_DIS__SHIFT 0x19
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_VSYNC_IN_RECV_MASK 0x4000000
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_VSYNC_IN_RECV__SHIFT 0x1a
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_VSYNC_IN_RECV1_MASK 0x8000000
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_VSYNC_IN_RECV1__SHIFT 0x1b
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_HSYNC_IN_MASK_MASK 0x10000000
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_HSYNC_IN_MASK__SHIFT 0x1c
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_HSYNC_IN_PD_DIS_MASK 0x20000000
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_HSYNC_IN_PD_DIS__SHIFT 0x1d
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_HSYNC_IN_RECV_MASK 0x40000000
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_HSYNC_IN_RECV__SHIFT 0x1e
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_HSYNC_IN_RECV1_MASK 0x80000000
+#define DC_GPIO_PWRSEQ_MASK__DC_GPIO_HSYNC_IN_RECV1__SHIFT 0x1f
+#define DC_GPIO_PWRSEQ_A__DC_GPIO_BLON_A_MASK 0x1
+#define DC_GPIO_PWRSEQ_A__DC_GPIO_BLON_A__SHIFT 0x0
+#define DC_GPIO_PWRSEQ_A__DC_GPIO_DIGON_A_MASK 0x100
+#define DC_GPIO_PWRSEQ_A__DC_GPIO_DIGON_A__SHIFT 0x8
+#define DC_GPIO_PWRSEQ_A__DC_GPIO_ENA_BL_A_MASK 0x10000
+#define DC_GPIO_PWRSEQ_A__DC_GPIO_ENA_BL_A__SHIFT 0x10
+#define DC_GPIO_PWRSEQ_A__DC_GPIO_VSYNC_IN_A_MASK 0x1000000
+#define DC_GPIO_PWRSEQ_A__DC_GPIO_VSYNC_IN_A__SHIFT 0x18
+#define DC_GPIO_PWRSEQ_A__DC_GPIO_HSYNC_IN_A_MASK 0x80000000
+#define DC_GPIO_PWRSEQ_A__DC_GPIO_HSYNC_IN_A__SHIFT 0x1f
+#define DC_GPIO_PWRSEQ_EN__DC_GPIO_BLON_EN_MASK 0x1
+#define DC_GPIO_PWRSEQ_EN__DC_GPIO_BLON_EN__SHIFT 0x0
+#define DC_GPIO_PWRSEQ_EN__DC_GPIO_VARY_BL_GENERICA_EN_MASK 0x2
+#define DC_GPIO_PWRSEQ_EN__DC_GPIO_VARY_BL_GENERICA_EN__SHIFT 0x1
+#define DC_GPIO_PWRSEQ_EN__DC_GPIO_DIGON_EN_MASK 0x100
+#define DC_GPIO_PWRSEQ_EN__DC_GPIO_DIGON_EN__SHIFT 0x8
+#define DC_GPIO_PWRSEQ_EN__DC_GPIO_ENA_BL_EN_MASK 0x10000
+#define DC_GPIO_PWRSEQ_EN__DC_GPIO_ENA_BL_EN__SHIFT 0x10
+#define DC_GPIO_PWRSEQ_EN__DC_GPIO_VSYNC_IN_EN_MASK 0x1000000
+#define DC_GPIO_PWRSEQ_EN__DC_GPIO_VSYNC_IN_EN__SHIFT 0x18
+#define DC_GPIO_PWRSEQ_EN__DC_GPIO_HSYNC_IN_EN_MASK 0x80000000
+#define DC_GPIO_PWRSEQ_EN__DC_GPIO_HSYNC_IN_EN__SHIFT 0x1f
+#define DC_GPIO_PWRSEQ_Y__DC_GPIO_BLON_Y_MASK 0x1
+#define DC_GPIO_PWRSEQ_Y__DC_GPIO_BLON_Y__SHIFT 0x0
+#define DC_GPIO_PWRSEQ_Y__DC_GPIO_DIGON_Y_MASK 0x100
+#define DC_GPIO_PWRSEQ_Y__DC_GPIO_DIGON_Y__SHIFT 0x8
+#define DC_GPIO_PWRSEQ_Y__DC_GPIO_ENA_BL_Y_MASK 0x10000
+#define DC_GPIO_PWRSEQ_Y__DC_GPIO_ENA_BL_Y__SHIFT 0x10
+#define DC_GPIO_PWRSEQ_Y__DC_GPIO_VSYNC_IN_MASK 0x1000000
+#define DC_GPIO_PWRSEQ_Y__DC_GPIO_VSYNC_IN__SHIFT 0x18
+#define DC_GPIO_PWRSEQ_Y__DC_GPIO_HSYNC_IN_MASK 0x80000000
+#define DC_GPIO_PWRSEQ_Y__DC_GPIO_HSYNC_IN__SHIFT 0x1f
+#define DC_GPIO_PAD_STRENGTH_1__GENLK_STRENGTH_SN_MASK 0xf
+#define DC_GPIO_PAD_STRENGTH_1__GENLK_STRENGTH_SN__SHIFT 0x0
+#define DC_GPIO_PAD_STRENGTH_1__GENLK_STRENGTH_SP_MASK 0xf0
+#define DC_GPIO_PAD_STRENGTH_1__GENLK_STRENGTH_SP__SHIFT 0x4
+#define DC_GPIO_PAD_STRENGTH_1__RX_HPD_STRENGTH_SN_MASK 0xf00
+#define DC_GPIO_PAD_STRENGTH_1__RX_HPD_STRENGTH_SN__SHIFT 0x8
+#define DC_GPIO_PAD_STRENGTH_1__RX_HPD_STRENGTH_SP_MASK 0xf000
+#define DC_GPIO_PAD_STRENGTH_1__RX_HPD_STRENGTH_SP__SHIFT 0xc
+#define DC_GPIO_PAD_STRENGTH_1__TX_HPD_STRENGTH_SN_MASK 0xf0000
+#define DC_GPIO_PAD_STRENGTH_1__TX_HPD_STRENGTH_SN__SHIFT 0x10
+#define DC_GPIO_PAD_STRENGTH_1__TX_HPD_STRENGTH_SP_MASK 0xf00000
+#define DC_GPIO_PAD_STRENGTH_1__TX_HPD_STRENGTH_SP__SHIFT 0x14
+#define DC_GPIO_PAD_STRENGTH_1__SYNC_STRENGTH_SN_MASK 0xf000000
+#define DC_GPIO_PAD_STRENGTH_1__SYNC_STRENGTH_SN__SHIFT 0x18
+#define DC_GPIO_PAD_STRENGTH_1__SYNC_STRENGTH_SP_MASK 0xf0000000
+#define DC_GPIO_PAD_STRENGTH_1__SYNC_STRENGTH_SP__SHIFT 0x1c
+#define DC_GPIO_PAD_STRENGTH_2__STRENGTH_SN_MASK 0xf
+#define DC_GPIO_PAD_STRENGTH_2__STRENGTH_SN__SHIFT 0x0
+#define DC_GPIO_PAD_STRENGTH_2__STRENGTH_SP_MASK 0xf0
+#define DC_GPIO_PAD_STRENGTH_2__STRENGTH_SP__SHIFT 0x4
+#define DC_GPIO_PAD_STRENGTH_2__EXT_RESET_DRVSTRENGTH_MASK 0x700
+#define DC_GPIO_PAD_STRENGTH_2__EXT_RESET_DRVSTRENGTH__SHIFT 0x8
+#define DC_GPIO_PAD_STRENGTH_2__REF_27_DRVSTRENGTH_MASK 0x7000
+#define DC_GPIO_PAD_STRENGTH_2__REF_27_DRVSTRENGTH__SHIFT 0xc
+#define DC_GPIO_PAD_STRENGTH_2__PWRSEQ_STRENGTH_SN_MASK 0xf0000
+#define DC_GPIO_PAD_STRENGTH_2__PWRSEQ_STRENGTH_SN__SHIFT 0x10
+#define DC_GPIO_PAD_STRENGTH_2__PWRSEQ_STRENGTH_SP_MASK 0xf00000
+#define DC_GPIO_PAD_STRENGTH_2__PWRSEQ_STRENGTH_SP__SHIFT 0x14
+#define DC_GPIO_PAD_STRENGTH_2__REF_27_SRC_SEL_MASK 0xc0000000
+#define DC_GPIO_PAD_STRENGTH_2__REF_27_SRC_SEL__SHIFT 0x1e
+#define PHY_AUX_CNTL__AUXSLAVE_PAD_SLEWN_MASK 0x1
+#define PHY_AUX_CNTL__AUXSLAVE_PAD_SLEWN__SHIFT 0x0
+#define PHY_AUX_CNTL__AUXSLAVE_PAD_WAKE_MASK 0x2
+#define PHY_AUX_CNTL__AUXSLAVE_PAD_WAKE__SHIFT 0x1
+#define PHY_AUX_CNTL__AUXSLAVE_PAD_RXSEL_MASK 0x4
+#define PHY_AUX_CNTL__AUXSLAVE_PAD_RXSEL__SHIFT 0x2
+#define PHY_AUX_CNTL__AUXSLAVE_PAD_MODE_MASK 0x8
+#define PHY_AUX_CNTL__AUXSLAVE_PAD_MODE__SHIFT 0x3
+#define PHY_AUX_CNTL__DDCSLAVE_DATA_PD_EN_MASK 0x10
+#define PHY_AUX_CNTL__DDCSLAVE_DATA_PD_EN__SHIFT 0x4
+#define PHY_AUX_CNTL__DDCSLAVE_DATA_EN_MASK 0x20
+#define PHY_AUX_CNTL__DDCSLAVE_DATA_EN__SHIFT 0x5
+#define PHY_AUX_CNTL__DDCSLAVE_CLK_PD_EN_MASK 0x40
+#define PHY_AUX_CNTL__DDCSLAVE_CLK_PD_EN__SHIFT 0x6
+#define PHY_AUX_CNTL__DDCSLAVE_CLK_EN_MASK 0x80
+#define PHY_AUX_CNTL__DDCSLAVE_CLK_EN__SHIFT 0x7
+#define PHY_AUX_CNTL__AUX_PAD_SLEWN_MASK 0x1000
+#define PHY_AUX_CNTL__AUX_PAD_SLEWN__SHIFT 0xc
+#define PHY_AUX_CNTL__AUXSLAVE_CLK_PD_EN_MASK 0x2000
+#define PHY_AUX_CNTL__AUXSLAVE_CLK_PD_EN__SHIFT 0xd
+#define PHY_AUX_CNTL__AUX_PAD_WAKE_MASK 0x4000
+#define PHY_AUX_CNTL__AUX_PAD_WAKE__SHIFT 0xe
+#define PHY_AUX_CNTL__AUX_PAD_RXSEL_MASK 0x30000
+#define PHY_AUX_CNTL__AUX_PAD_RXSEL__SHIFT 0x10
+#define PHY_AUX_CNTL__AUX_PAD_RESBIASEN_MASK 0x40000
+#define PHY_AUX_CNTL__AUX_PAD_RESBIASEN__SHIFT 0x12
+#define PHY_AUX_CNTL__AUX_PAD_COMPSEL_MASK 0x80000
+#define PHY_AUX_CNTL__AUX_PAD_COMPSEL__SHIFT 0x13
+#define DC_GPIO_I2CPAD_A__DC_GPIO_SCL_A_MASK 0x1
+#define DC_GPIO_I2CPAD_A__DC_GPIO_SCL_A__SHIFT 0x0
+#define DC_GPIO_I2CPAD_A__DC_GPIO_SDA_A_MASK 0x2
+#define DC_GPIO_I2CPAD_A__DC_GPIO_SDA_A__SHIFT 0x1
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_SCL_EN_MASK 0x1
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_SCL_EN__SHIFT 0x0
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_SDA_EN_MASK 0x2
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_SDA_EN__SHIFT 0x1
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_DATA_PD_EN_MASK 0x4
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_DATA_PD_EN__SHIFT 0x2
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_RXSEL_MASK 0x30000
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_RXSEL__SHIFT 0x10
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_SPARE_MASK 0xc0000
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_SPARE__SHIFT 0x12
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_BIASCRTEN_MASK 0x100000
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_BIASCRTEN__SHIFT 0x14
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_CSEL0P9_MASK 0x200000
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_CSEL0P9__SHIFT 0x15
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_CSEL1P1_MASK 0x400000
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_CSEL1P1__SHIFT 0x16
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_COMPSEL_MASK 0x800000
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_COMPSEL__SHIFT 0x17
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_RSEL0P9_MASK 0x1000000
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_RSEL0P9__SHIFT 0x18
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_RSEL1P1_MASK 0x2000000
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_RSEL1P1__SHIFT 0x19
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_SPIKERCEN_MASK 0x4000000
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_SPIKERCEN__SHIFT 0x1a
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_SPIKERCSEL_MASK 0x8000000
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_SPIKERCSEL__SHIFT 0x1b
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_FALLSLEWSEL_MASK 0x30000000
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_FALLSLEWSEL__SHIFT 0x1c
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_RESBIASEN_MASK 0x40000000
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_RESBIASEN__SHIFT 0x1e
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_SLEWN_MASK 0x80000000
+#define DC_GPIO_I2CPAD_EN__DC_GPIO_I2C_PAD_SLEWN__SHIFT 0x1f
+#define DC_GPIO_I2CPAD_Y__DC_GPIO_SCL_Y_MASK 0x1
+#define DC_GPIO_I2CPAD_Y__DC_GPIO_SCL_Y__SHIFT 0x0
+#define DC_GPIO_I2CPAD_Y__DC_GPIO_SDA_Y_MASK 0x2
+#define DC_GPIO_I2CPAD_Y__DC_GPIO_SDA_Y__SHIFT 0x1
+#define DC_GPIO_I2CPAD_STRENGTH__I2C_STRENGTH_SN_MASK 0xf
+#define DC_GPIO_I2CPAD_STRENGTH__I2C_STRENGTH_SN__SHIFT 0x0
+#define DC_GPIO_I2CPAD_STRENGTH__I2C_STRENGTH_SP_MASK 0xf0
+#define DC_GPIO_I2CPAD_STRENGTH__I2C_STRENGTH_SP__SHIFT 0x4
+#define DVO_VREF_CONTROL__DVO_VREFPON_MASK 0x1
+#define DVO_VREF_CONTROL__DVO_VREFPON__SHIFT 0x0
+#define DVO_VREF_CONTROL__DVO_VREFSEL_MASK 0x2
+#define DVO_VREF_CONTROL__DVO_VREFSEL__SHIFT 0x1
+#define DVO_VREF_CONTROL__DVO_VREFCAL_MASK 0xf0
+#define DVO_VREF_CONTROL__DVO_VREFCAL__SHIFT 0x4
+#define DVO_SKEW_ADJUST__DVO_SKEW_ADJUST_MASK 0xffffffff
+#define DVO_SKEW_ADJUST__DVO_SKEW_ADJUST__SHIFT 0x0
+#define DC_GPIO_RECEIVER_EN0__VIPPAD_SCL_RECEN_MASK 0x1
+#define DC_GPIO_RECEIVER_EN0__VIPPAD_SCL_RECEN__SHIFT 0x0
+#define DC_GPIO_RECEIVER_EN0__VIPPAD_SDA_RECEN_MASK 0x2
+#define DC_GPIO_RECEIVER_EN0__VIPPAD_SDA_RECEN__SHIFT 0x1
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_RX_HPD_RECEN_MASK 0x10000
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_RX_HPD_RECEN__SHIFT 0x10
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_HPD1_RECEN_MASK 0x20000
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_HPD1_RECEN__SHIFT 0x11
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_GENLK_VSYNC_RECEN_MASK 0x40000
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_GENLK_VSYNC_RECEN__SHIFT 0x12
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_GENLK_CLK_RECEN_MASK 0x80000
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_GENLK_CLK_RECEN__SHIFT 0x13
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_VSYNCA_RECEN_MASK 0x100000
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_VSYNCA_RECEN__SHIFT 0x14
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_HSYNCA_RECEN_MASK 0x200000
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_HSYNCA_RECEN__SHIFT 0x15
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_GENERICG_RECEN_MASK 0x400000
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_GENERICG_RECEN__SHIFT 0x16
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_GENERICF_RECEN_MASK 0x800000
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_GENERICF_RECEN__SHIFT 0x17
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_GENERICE_RECEN_MASK 0x1000000
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_GENERICE_RECEN__SHIFT 0x18
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_GENERICD_RECEN_MASK 0x2000000
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_GENERICD_RECEN__SHIFT 0x19
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_GENERICC_RECEN_MASK 0x4000000
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_GENERICC_RECEN__SHIFT 0x1a
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_GENERICB_RECEN_MASK 0x8000000
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_GENERICB_RECEN__SHIFT 0x1b
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_GENERICA_RECEN_MASK 0x10000000
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_GENERICA_RECEN__SHIFT 0x1c
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_BLON_RECEN_MASK 0x20000000
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_BLON_RECEN__SHIFT 0x1d
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_DIGON_RECEN_MASK 0x40000000
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_DIGON_RECEN__SHIFT 0x1e
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_DDC2DATA_RECEN_MASK 0x80000000
+#define DC_GPIO_RECEIVER_EN0__DC_GPIO_DDC2DATA_RECEN__SHIFT 0x1f
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_DDC2CLK_RECEN_MASK 0x1
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_DDC2CLK_RECEN__SHIFT 0x0
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_DDC1DATA_RECEN_MASK 0x2
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_DDC1DATA_RECEN__SHIFT 0x1
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_DDC1CLK_RECEN_MASK 0x4
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_DDC1CLK_RECEN__SHIFT 0x2
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_DDC3DATA_RECEN_MASK 0x8
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_DDC3DATA_RECEN__SHIFT 0x3
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_DDC3CLK_RECEN_MASK 0x10
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_DDC3CLK_RECEN__SHIFT 0x4
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_DDC4DATA_RECEN_MASK 0x20
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_DDC4DATA_RECEN__SHIFT 0x5
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_DDC4CLK_RECEN_MASK 0x40
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_DDC4CLK_RECEN__SHIFT 0x6
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_DDC5DATA_RECEN_MASK 0x80
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_DDC5DATA_RECEN__SHIFT 0x7
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_DDC5CLK_RECEN_MASK 0x100
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_DDC5CLK_RECEN__SHIFT 0x8
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_DDC6DATA_RECEN_MASK 0x200
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_DDC6DATA_RECEN__SHIFT 0x9
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_DDC6CLK_RECEN_MASK 0x400
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_DDC6CLK_RECEN__SHIFT 0xa
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_HPD2_RECEN_MASK 0x800
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_HPD2_RECEN__SHIFT 0xb
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_HPD3_RECEN_MASK 0x1000
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_HPD3_RECEN__SHIFT 0xc
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_HPD4_RECEN_MASK 0x2000
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_HPD4_RECEN__SHIFT 0xd
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_HPD5_RECEN_MASK 0x4000
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_HPD5_RECEN__SHIFT 0xe
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_HPD6_RECEN_MASK 0x8000
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_HPD6_RECEN__SHIFT 0xf
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_ENA_BL_RECEN_MASK 0x10000
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_ENA_BL_RECEN__SHIFT 0x10
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_SWAPLOCK_A_RECEN_MASK 0x20000
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_SWAPLOCK_A_RECEN__SHIFT 0x11
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_SWAPLOCK_B_RECEN_MASK 0x40000
+#define DC_GPIO_RECEIVER_EN1__DC_GPIO_SWAPLOCK_B_RECEN__SHIFT 0x12
+#define DC_GPIO_I2S_SPDIF_MASK__DC_GPIO_I2SDATA0_MASK_MASK 0xf
+#define DC_GPIO_I2S_SPDIF_MASK__DC_GPIO_I2SDATA0_MASK__SHIFT 0x0
+#define DC_GPIO_I2S_SPDIF_MASK__DC_GPIO_MCLK0_MASK_MASK 0x10
+#define DC_GPIO_I2S_SPDIF_MASK__DC_GPIO_MCLK0_MASK__SHIFT 0x4
+#define DC_GPIO_I2S_SPDIF_MASK__DC_GPIO_BCLK0_MASK_MASK 0x20
+#define DC_GPIO_I2S_SPDIF_MASK__DC_GPIO_BCLK0_MASK__SHIFT 0x5
+#define DC_GPIO_I2S_SPDIF_MASK__DC_GPIO_LRCK0_MASK_MASK 0x40
+#define DC_GPIO_I2S_SPDIF_MASK__DC_GPIO_LRCK0_MASK__SHIFT 0x6
+#define DC_GPIO_I2S_SPDIF_MASK__DC_GPIO_SPDIF0_MASK_MASK 0x80
+#define DC_GPIO_I2S_SPDIF_MASK__DC_GPIO_SPDIF0_MASK__SHIFT 0x7
+#define DC_GPIO_I2S_SPDIF_MASK__DC_GPIO_I2SDATA1_MASK_MASK 0x100
+#define DC_GPIO_I2S_SPDIF_MASK__DC_GPIO_I2SDATA1_MASK__SHIFT 0x8
+#define DC_GPIO_I2S_SPDIF_MASK__DC_GPIO_MCLK1_MASK_MASK 0x200
+#define DC_GPIO_I2S_SPDIF_MASK__DC_GPIO_MCLK1_MASK__SHIFT 0x9
+#define DC_GPIO_I2S_SPDIF_MASK__DC_GPIO_BCLK1_MASK_MASK 0x400
+#define DC_GPIO_I2S_SPDIF_MASK__DC_GPIO_BCLK1_MASK__SHIFT 0xa
+#define DC_GPIO_I2S_SPDIF_MASK__DC_GPIO_LRCK1_MASK_MASK 0x800
+#define DC_GPIO_I2S_SPDIF_MASK__DC_GPIO_LRCK1_MASK__SHIFT 0xb
+#define DC_GPIO_I2S_SPDIF_MASK__DC_GPIO_SPDIF1_MASK_MASK 0x1000
+#define DC_GPIO_I2S_SPDIF_MASK__DC_GPIO_SPDIF1_MASK__SHIFT 0xc
+#define DC_GPIO_I2S_SPDIF_A__DC_GPIO_I2SDATA0_A_MASK 0xf
+#define DC_GPIO_I2S_SPDIF_A__DC_GPIO_I2SDATA0_A__SHIFT 0x0
+#define DC_GPIO_I2S_SPDIF_A__DC_GPIO_MCLK0_A_MASK 0x10
+#define DC_GPIO_I2S_SPDIF_A__DC_GPIO_MCLK0_A__SHIFT 0x4
+#define DC_GPIO_I2S_SPDIF_A__DC_GPIO_BCLK0_A_MASK 0x20
+#define DC_GPIO_I2S_SPDIF_A__DC_GPIO_BCLK0_A__SHIFT 0x5
+#define DC_GPIO_I2S_SPDIF_A__DC_GPIO_LRCK0_A_MASK 0x40
+#define DC_GPIO_I2S_SPDIF_A__DC_GPIO_LRCK0_A__SHIFT 0x6
+#define DC_GPIO_I2S_SPDIF_A__DC_GPIO_SPDIF0_A_MASK 0x80
+#define DC_GPIO_I2S_SPDIF_A__DC_GPIO_SPDIF0_A__SHIFT 0x7
+#define DC_GPIO_I2S_SPDIF_A__DC_GPIO_I2SDATA1_A_MASK 0x100
+#define DC_GPIO_I2S_SPDIF_A__DC_GPIO_I2SDATA1_A__SHIFT 0x8
+#define DC_GPIO_I2S_SPDIF_A__DC_GPIO_MCLK1_A_MASK 0x200
+#define DC_GPIO_I2S_SPDIF_A__DC_GPIO_MCLK1_A__SHIFT 0x9
+#define DC_GPIO_I2S_SPDIF_A__DC_GPIO_BCLK1_A_MASK 0x400
+#define DC_GPIO_I2S_SPDIF_A__DC_GPIO_BCLK1_A__SHIFT 0xa
+#define DC_GPIO_I2S_SPDIF_A__DC_GPIO_LRCK1_A_MASK 0x800
+#define DC_GPIO_I2S_SPDIF_A__DC_GPIO_LRCK1_A__SHIFT 0xb
+#define DC_GPIO_I2S_SPDIF_A__DC_GPIO_SPDIF1_A_MASK 0x1000
+#define DC_GPIO_I2S_SPDIF_A__DC_GPIO_SPDIF1_A__SHIFT 0xc
+#define DC_GPIO_I2S_SPDIF_EN__DC_GPIO_I2SDATA0_EN_MASK 0xf
+#define DC_GPIO_I2S_SPDIF_EN__DC_GPIO_I2SDATA0_EN__SHIFT 0x0
+#define DC_GPIO_I2S_SPDIF_EN__DC_GPIO_MCLK0_EN_MASK 0x10
+#define DC_GPIO_I2S_SPDIF_EN__DC_GPIO_MCLK0_EN__SHIFT 0x4
+#define DC_GPIO_I2S_SPDIF_EN__DC_GPIO_BCLK0_EN_MASK 0x20
+#define DC_GPIO_I2S_SPDIF_EN__DC_GPIO_BCLK0_EN__SHIFT 0x5
+#define DC_GPIO_I2S_SPDIF_EN__DC_GPIO_LRCK0_EN_MASK 0x40
+#define DC_GPIO_I2S_SPDIF_EN__DC_GPIO_LRCK0_EN__SHIFT 0x6
+#define DC_GPIO_I2S_SPDIF_EN__DC_GPIO_SPDIF0_EN_MASK 0x80
+#define DC_GPIO_I2S_SPDIF_EN__DC_GPIO_SPDIF0_EN__SHIFT 0x7
+#define DC_GPIO_I2S_SPDIF_EN__DC_GPIO_I2SDATA1_EN_MASK 0x100
+#define DC_GPIO_I2S_SPDIF_EN__DC_GPIO_I2SDATA1_EN__SHIFT 0x8
+#define DC_GPIO_I2S_SPDIF_EN__DC_GPIO_MCLK1_EN_MASK 0x200
+#define DC_GPIO_I2S_SPDIF_EN__DC_GPIO_MCLK1_EN__SHIFT 0x9
+#define DC_GPIO_I2S_SPDIF_EN__DC_GPIO_BCLK1_EN_MASK 0x400
+#define DC_GPIO_I2S_SPDIF_EN__DC_GPIO_BCLK1_EN__SHIFT 0xa
+#define DC_GPIO_I2S_SPDIF_EN__DC_GPIO_LRCK1_EN_MASK 0x800
+#define DC_GPIO_I2S_SPDIF_EN__DC_GPIO_LRCK1_EN__SHIFT 0xb
+#define DC_GPIO_I2S_SPDIF_EN__DC_GPIO_SPDIF1_EN_MASK 0x1000
+#define DC_GPIO_I2S_SPDIF_EN__DC_GPIO_SPDIF1_EN__SHIFT 0xc
+#define DC_GPIO_I2S_SPDIF_EN__SPDIF1_APORT_MASK 0x2000
+#define DC_GPIO_I2S_SPDIF_EN__SPDIF1_APORT__SHIFT 0xd
+#define DC_GPIO_I2S_SPDIF_EN__SPDIF1_PU_MASK 0x4000
+#define DC_GPIO_I2S_SPDIF_EN__SPDIF1_PU__SHIFT 0xe
+#define DC_GPIO_I2S_SPDIF_EN__SPDIF1_RXSEL_MASK 0x8000
+#define DC_GPIO_I2S_SPDIF_EN__SPDIF1_RXSEL__SHIFT 0xf
+#define DC_GPIO_I2S_SPDIF_EN__SPDIF1_SCHMEN_MASK 0x10000
+#define DC_GPIO_I2S_SPDIF_EN__SPDIF1_SCHMEN__SHIFT 0x10
+#define DC_GPIO_I2S_SPDIF_EN__SPDIF1_SMODE_EN_MASK 0x20000
+#define DC_GPIO_I2S_SPDIF_EN__SPDIF1_SMODE_EN__SHIFT 0x11
+#define DC_GPIO_I2S_SPDIF_EN__SPDIF1_IMODE_MASK 0x40000
+#define DC_GPIO_I2S_SPDIF_EN__SPDIF1_IMODE__SHIFT 0x12
+#define DC_GPIO_I2S_SPDIF_Y__DC_GPIO_I2SDATA0_Y_MASK 0xf
+#define DC_GPIO_I2S_SPDIF_Y__DC_GPIO_I2SDATA0_Y__SHIFT 0x0
+#define DC_GPIO_I2S_SPDIF_Y__DC_GPIO_MCLK0_Y_MASK 0x10
+#define DC_GPIO_I2S_SPDIF_Y__DC_GPIO_MCLK0_Y__SHIFT 0x4
+#define DC_GPIO_I2S_SPDIF_Y__DC_GPIO_BCLK0_Y_MASK 0x20
+#define DC_GPIO_I2S_SPDIF_Y__DC_GPIO_BCLK0_Y__SHIFT 0x5
+#define DC_GPIO_I2S_SPDIF_Y__DC_GPIO_LRCK0_Y_MASK 0x40
+#define DC_GPIO_I2S_SPDIF_Y__DC_GPIO_LRCK0_Y__SHIFT 0x6
+#define DC_GPIO_I2S_SPDIF_Y__DC_GPIO_SPDIF0_Y_MASK 0x80
+#define DC_GPIO_I2S_SPDIF_Y__DC_GPIO_SPDIF0_Y__SHIFT 0x7
+#define DC_GPIO_I2S_SPDIF_Y__DC_GPIO_I2SDATA1_Y_MASK 0x100
+#define DC_GPIO_I2S_SPDIF_Y__DC_GPIO_I2SDATA1_Y__SHIFT 0x8
+#define DC_GPIO_I2S_SPDIF_Y__DC_GPIO_MCLK1_Y_MASK 0x200
+#define DC_GPIO_I2S_SPDIF_Y__DC_GPIO_MCLK1_Y__SHIFT 0x9
+#define DC_GPIO_I2S_SPDIF_Y__DC_GPIO_BCLK1_Y_MASK 0x400
+#define DC_GPIO_I2S_SPDIF_Y__DC_GPIO_BCLK1_Y__SHIFT 0xa
+#define DC_GPIO_I2S_SPDIF_Y__DC_GPIO_LRCK1_Y_MASK 0x800
+#define DC_GPIO_I2S_SPDIF_Y__DC_GPIO_LRCK1_Y__SHIFT 0xb
+#define DC_GPIO_I2S_SPDIF_Y__DC_GPIO_SPDIF1_Y_MASK 0x1000
+#define DC_GPIO_I2S_SPDIF_Y__DC_GPIO_SPDIF1_Y__SHIFT 0xc
+#define DC_GPIO_I2S_SPDIF_STRENGTH__I2S0_DRVSTRENGTH_MASK 0x7
+#define DC_GPIO_I2S_SPDIF_STRENGTH__I2S0_DRVSTRENGTH__SHIFT 0x0
+#define DC_GPIO_I2S_SPDIF_STRENGTH__SPDIF0_DRVSTRENGTH_SN_MASK 0x700
+#define DC_GPIO_I2S_SPDIF_STRENGTH__SPDIF0_DRVSTRENGTH_SN__SHIFT 0x8
+#define DC_GPIO_I2S_SPDIF_STRENGTH__SPDIF0_DRVSTRENGTH_SP_MASK 0x3800
+#define DC_GPIO_I2S_SPDIF_STRENGTH__SPDIF0_DRVSTRENGTH_SP__SHIFT 0xb
+#define DC_GPIO_I2S_SPDIF_STRENGTH__I2S1_DRVSTRENGTH_MASK 0x70000
+#define DC_GPIO_I2S_SPDIF_STRENGTH__I2S1_DRVSTRENGTH__SHIFT 0x10
+#define DC_GPIO_I2S_SPDIF_STRENGTH__SPDIF1_DRVSTRENGTH_SN_MASK 0x7000000
+#define DC_GPIO_I2S_SPDIF_STRENGTH__SPDIF1_DRVSTRENGTH_SN__SHIFT 0x18
+#define DC_GPIO_I2S_SPDIF_STRENGTH__SPDIF1_DRVSTRENGTH_SP_MASK 0x38000000
+#define DC_GPIO_I2S_SPDIF_STRENGTH__SPDIF1_DRVSTRENGTH_SP__SHIFT 0x1b
+#define DC_GPIO_TX12_EN__DC_GPIO_BLON_TX12_EN_MASK 0x1
+#define DC_GPIO_TX12_EN__DC_GPIO_BLON_TX12_EN__SHIFT 0x0
+#define DC_GPIO_TX12_EN__DC_GPIO_DIGON_TX12_EN_MASK 0x2
+#define DC_GPIO_TX12_EN__DC_GPIO_DIGON_TX12_EN__SHIFT 0x1
+#define DC_GPIO_TX12_EN__DC_GPIO_ENA_BL_TX12_EN_MASK 0x4
+#define DC_GPIO_TX12_EN__DC_GPIO_ENA_BL_TX12_EN__SHIFT 0x2
+#define DC_GPIO_TX12_EN__DC_GPIO_GENERICA_TX12_EN_MASK 0x8
+#define DC_GPIO_TX12_EN__DC_GPIO_GENERICA_TX12_EN__SHIFT 0x3
+#define DC_GPIO_TX12_EN__DC_GPIO_GENERICB_TX12_EN_MASK 0x10
+#define DC_GPIO_TX12_EN__DC_GPIO_GENERICB_TX12_EN__SHIFT 0x4
+#define DC_GPIO_TX12_EN__DC_GPIO_GENERICC_TX12_EN_MASK 0x20
+#define DC_GPIO_TX12_EN__DC_GPIO_GENERICC_TX12_EN__SHIFT 0x5
+#define DC_GPIO_TX12_EN__DC_GPIO_GENERICD_TX12_EN_MASK 0x40
+#define DC_GPIO_TX12_EN__DC_GPIO_GENERICD_TX12_EN__SHIFT 0x6
+#define DC_GPIO_TX12_EN__DC_GPIO_GENERICE_TX12_EN_MASK 0x80
+#define DC_GPIO_TX12_EN__DC_GPIO_GENERICE_TX12_EN__SHIFT 0x7
+#define DC_GPIO_TX12_EN__DC_GPIO_GENERICF_TX12_EN_MASK 0x100
+#define DC_GPIO_TX12_EN__DC_GPIO_GENERICF_TX12_EN__SHIFT 0x8
+#define DC_GPIO_TX12_EN__DC_GPIO_GENERICG_TX12_EN_MASK 0x200
+#define DC_GPIO_TX12_EN__DC_GPIO_GENERICG_TX12_EN__SHIFT 0x9
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX1_FALLSLEWSEL_MASK 0x3
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX1_FALLSLEWSEL__SHIFT 0x0
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX2_FALLSLEWSEL_MASK 0xc
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX2_FALLSLEWSEL__SHIFT 0x2
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX3_FALLSLEWSEL_MASK 0x30
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX3_FALLSLEWSEL__SHIFT 0x4
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX4_FALLSLEWSEL_MASK 0xc0
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX4_FALLSLEWSEL__SHIFT 0x6
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX5_FALLSLEWSEL_MASK 0x300
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX5_FALLSLEWSEL__SHIFT 0x8
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX6_FALLSLEWSEL_MASK 0xc00
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX6_FALLSLEWSEL__SHIFT 0xa
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX1_SPIKERCEN_MASK 0x10000
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX1_SPIKERCEN__SHIFT 0x10
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX2_SPIKERCEN_MASK 0x20000
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX2_SPIKERCEN__SHIFT 0x11
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX3_SPIKERCEN_MASK 0x40000
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX3_SPIKERCEN__SHIFT 0x12
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX4_SPIKERCEN_MASK 0x80000
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX4_SPIKERCEN__SHIFT 0x13
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX5_SPIKERCEN_MASK 0x100000
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX5_SPIKERCEN__SHIFT 0x14
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX6_SPIKERCEN_MASK 0x200000
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX6_SPIKERCEN__SHIFT 0x15
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX1_SPIKERCSEL_MASK 0x1000000
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX1_SPIKERCSEL__SHIFT 0x18
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX2_SPIKERCSEL_MASK 0x2000000
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX2_SPIKERCSEL__SHIFT 0x19
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX3_SPIKERCSEL_MASK 0x4000000
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX3_SPIKERCSEL__SHIFT 0x1a
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX4_SPIKERCSEL_MASK 0x8000000
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX4_SPIKERCSEL__SHIFT 0x1b
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX5_SPIKERCSEL_MASK 0x10000000
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX5_SPIKERCSEL__SHIFT 0x1c
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX6_SPIKERCSEL_MASK 0x20000000
+#define DC_GPIO_AUX_CTRL_0__DC_GPIO_AUX6_SPIKERCSEL__SHIFT 0x1d
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX1_CSEL_0P9_MASK 0x1
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX1_CSEL_0P9__SHIFT 0x0
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX2_CSEL_0P9_MASK 0x2
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX2_CSEL_0P9__SHIFT 0x1
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX3_CSEL_0P9_MASK 0x4
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX3_CSEL_0P9__SHIFT 0x2
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX4_CSEL_0P9_MASK 0x8
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX4_CSEL_0P9__SHIFT 0x3
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX5_CSEL_0P9_MASK 0x10
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX5_CSEL_0P9__SHIFT 0x4
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX6_CSEL_0P9_MASK 0x20
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX6_CSEL_0P9__SHIFT 0x5
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX1_CSEL_1P1_MASK 0x100
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX1_CSEL_1P1__SHIFT 0x8
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX2_CSEL_1P1_MASK 0x200
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX2_CSEL_1P1__SHIFT 0x9
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX3_CSEL_1P1_MASK 0x400
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX3_CSEL_1P1__SHIFT 0xa
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX4_CSEL_1P1_MASK 0x800
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX4_CSEL_1P1__SHIFT 0xb
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX5_CSEL_1P1_MASK 0x1000
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX5_CSEL_1P1__SHIFT 0xc
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX6_CSEL_1P1_MASK 0x2000
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX6_CSEL_1P1__SHIFT 0xd
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX1_RSEL_0P9_MASK 0x10000
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX1_RSEL_0P9__SHIFT 0x10
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX2_RSEL_0P9_MASK 0x20000
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX2_RSEL_0P9__SHIFT 0x11
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX3_RSEL_0P9_MASK 0x40000
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX3_RSEL_0P9__SHIFT 0x12
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX4_RSEL_0P9_MASK 0x80000
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX4_RSEL_0P9__SHIFT 0x13
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX5_RSEL_0P9_MASK 0x100000
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX5_RSEL_0P9__SHIFT 0x14
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX6_RSEL_0P9_MASK 0x200000
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX6_RSEL_0P9__SHIFT 0x15
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX1_RSEL_1P1_MASK 0x1000000
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX1_RSEL_1P1__SHIFT 0x18
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX2_RSEL_1P1_MASK 0x2000000
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX2_RSEL_1P1__SHIFT 0x19
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX3_RSEL_1P1_MASK 0x4000000
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX3_RSEL_1P1__SHIFT 0x1a
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX4_RSEL_1P1_MASK 0x8000000
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX4_RSEL_1P1__SHIFT 0x1b
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX5_RSEL_1P1_MASK 0x10000000
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX5_RSEL_1P1__SHIFT 0x1c
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX6_RSEL_1P1_MASK 0x20000000
+#define DC_GPIO_AUX_CTRL_1__DC_GPIO_AUX6_RSEL_1P1__SHIFT 0x1d
+#define DC_GPIO_AUX_CTRL_2__DC_GPIO_AUX1_BIASCRTEN_MASK 0x1
+#define DC_GPIO_AUX_CTRL_2__DC_GPIO_AUX1_BIASCRTEN__SHIFT 0x0
+#define DC_GPIO_AUX_CTRL_2__DC_GPIO_AUX2_BIASCRTEN_MASK 0x2
+#define DC_GPIO_AUX_CTRL_2__DC_GPIO_AUX2_BIASCRTEN__SHIFT 0x1
+#define DC_GPIO_AUX_CTRL_2__DC_GPIO_AUX3_BIASCRTEN_MASK 0x4
+#define DC_GPIO_AUX_CTRL_2__DC_GPIO_AUX3_BIASCRTEN__SHIFT 0x2
+#define DC_GPIO_AUX_CTRL_2__DC_GPIO_AUX4_BIASCRTEN_MASK 0x8
+#define DC_GPIO_AUX_CTRL_2__DC_GPIO_AUX4_BIASCRTEN__SHIFT 0x3
+#define DC_GPIO_AUX_CTRL_2__DC_GPIO_AUX5_BIASCRTEN_MASK 0x10
+#define DC_GPIO_AUX_CTRL_2__DC_GPIO_AUX5_BIASCRTEN__SHIFT 0x4
+#define DC_GPIO_AUX_CTRL_2__DC_GPIO_AUX6_BIASCRTEN_MASK 0x20
+#define DC_GPIO_AUX_CTRL_2__DC_GPIO_AUX6_BIASCRTEN__SHIFT 0x5
+#define DC_GPIO_AUX_CTRL_2__DC_IO_AUX1_SPARE_MASK 0xc0
+#define DC_GPIO_AUX_CTRL_2__DC_IO_AUX1_SPARE__SHIFT 0x6
+#define DC_GPIO_AUX_CTRL_2__DC_IO_AUX2_SPARE_MASK 0x300
+#define DC_GPIO_AUX_CTRL_2__DC_IO_AUX2_SPARE__SHIFT 0x8
+#define DC_GPIO_AUX_CTRL_2__DC_IO_AUX3_SPARE_MASK 0xc00
+#define DC_GPIO_AUX_CTRL_2__DC_IO_AUX3_SPARE__SHIFT 0xa
+#define DC_GPIO_AUX_CTRL_2__DC_IO_AUX4_SPARE_MASK 0x3000
+#define DC_GPIO_AUX_CTRL_2__DC_IO_AUX4_SPARE__SHIFT 0xc
+#define DC_GPIO_AUX_CTRL_2__DC_IO_AUX5_SPARE_MASK 0xc000
+#define DC_GPIO_AUX_CTRL_2__DC_IO_AUX5_SPARE__SHIFT 0xe
+#define DC_GPIO_AUX_CTRL_2__DC_IO_AUX6_SPARE_MASK 0x30000
+#define DC_GPIO_AUX_CTRL_2__DC_IO_AUX6_SPARE__SHIFT 0x10
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD12_FALLSLEWSEL_MASK 0x3
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD12_FALLSLEWSEL__SHIFT 0x0
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD34_FALLSLEWSEL_MASK 0xc
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD34_FALLSLEWSEL__SHIFT 0x2
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD56_FALLSLEWSEL_MASK 0x30
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD56_FALLSLEWSEL__SHIFT 0x4
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD12_SPIKERCEN_MASK 0x100
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD12_SPIKERCEN__SHIFT 0x8
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD34_SPIKERCEN_MASK 0x200
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD34_SPIKERCEN__SHIFT 0x9
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD56_SPIKERCEN_MASK 0x400
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD56_SPIKERCEN__SHIFT 0xa
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD12_SPIKERCSEL_MASK 0x1000
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD12_SPIKERCSEL__SHIFT 0xc
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD34_SPIKERCSEL_MASK 0x2000
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD34_SPIKERCSEL__SHIFT 0xd
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD56_SPIKERCSEL_MASK 0x4000
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD56_SPIKERCSEL__SHIFT 0xe
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD12_CSEL_0P9_MASK 0x10000
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD12_CSEL_0P9__SHIFT 0x10
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD34_CSEL_0P9_MASK 0x20000
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD34_CSEL_0P9__SHIFT 0x11
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD56_CSEL_0P9_MASK 0x40000
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD56_CSEL_0P9__SHIFT 0x12
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD12_CSEL_1P1_MASK 0x100000
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD12_CSEL_1P1__SHIFT 0x14
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD34_CSEL_1P1_MASK 0x200000
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD34_CSEL_1P1__SHIFT 0x15
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD56_CSEL_1P1_MASK 0x400000
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD56_CSEL_1P1__SHIFT 0x16
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD12_RSEL_0P9_MASK 0x1000000
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD12_RSEL_0P9__SHIFT 0x18
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD34_RSEL_0P9_MASK 0x2000000
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD34_RSEL_0P9__SHIFT 0x19
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD56_RSEL_0P9_MASK 0x4000000
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD56_RSEL_0P9__SHIFT 0x1a
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD12_RSEL_1P1_MASK 0x10000000
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD12_RSEL_1P1__SHIFT 0x1c
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD34_RSEL_1P1_MASK 0x20000000
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD34_RSEL_1P1__SHIFT 0x1d
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD56_RSEL_1P1_MASK 0x40000000
+#define DC_GPIO_HPD_CTRL_0__DC_GPIO_HPD56_RSEL_1P1__SHIFT 0x1e
+#define DC_GPIO_HPD_CTRL_1__DC_GPIO_HPD12_BIASCRTEN_MASK 0x1
+#define DC_GPIO_HPD_CTRL_1__DC_GPIO_HPD12_BIASCRTEN__SHIFT 0x0
+#define DC_GPIO_HPD_CTRL_1__DC_GPIO_HPD34_BIASCRTEN_MASK 0x2
+#define DC_GPIO_HPD_CTRL_1__DC_GPIO_HPD34_BIASCRTEN__SHIFT 0x1
+#define DC_GPIO_HPD_CTRL_1__DC_GPIO_HPD56_BIASCRTEN_MASK 0x4
+#define DC_GPIO_HPD_CTRL_1__DC_GPIO_HPD56_BIASCRTEN__SHIFT 0x2
+#define DC_GPIO_HPD_CTRL_1__DC_GPIO_HPD12_SLEWN_MASK 0x10
+#define DC_GPIO_HPD_CTRL_1__DC_GPIO_HPD12_SLEWN__SHIFT 0x4
+#define DC_GPIO_HPD_CTRL_1__DC_GPIO_HPD34_SLEWN_MASK 0x20
+#define DC_GPIO_HPD_CTRL_1__DC_GPIO_HPD34_SLEWN__SHIFT 0x5
+#define DC_GPIO_HPD_CTRL_1__DC_GPIO_HPD56_SLEWN_MASK 0x40
+#define DC_GPIO_HPD_CTRL_1__DC_GPIO_HPD56_SLEWN__SHIFT 0x6
+#define DAC_MACRO_CNTL_RESERVED0__DAC_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DAC_MACRO_CNTL_RESERVED0__DAC_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DAC_MACRO_CNTL_RESERVED1__DAC_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DAC_MACRO_CNTL_RESERVED1__DAC_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DAC_MACRO_CNTL_RESERVED2__DAC_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DAC_MACRO_CNTL_RESERVED2__DAC_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DAC_MACRO_CNTL_RESERVED3__DAC_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DAC_MACRO_CNTL_RESERVED3__DAC_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED0__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED0__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED1__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED1__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED2__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED2__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED3__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED3__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED4__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED4__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED5__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED5__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED6__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED6__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED7__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED7__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED8__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED8__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED9__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED9__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED10__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED10__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED11__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED11__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED12__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED12__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED13__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED13__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED14__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED14__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED15__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED15__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED16__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED16__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED17__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED17__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED18__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED18__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED19__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED19__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED20__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED20__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED21__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED21__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED22__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED22__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED23__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED23__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED24__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED24__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED25__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED25__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED26__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED26__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED27__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED27__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED28__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED28__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED29__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED29__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED30__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED30__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED31__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED31__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED32__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED32__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED33__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED33__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED34__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED34__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED35__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED35__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED36__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED36__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED37__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED37__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED38__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED38__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED39__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED39__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED40__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED40__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED41__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED41__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED42__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED42__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED43__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED43__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED44__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED44__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED45__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED45__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED46__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED46__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED47__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED47__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED48__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED48__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED49__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED49__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED50__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED50__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED51__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED51__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED52__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED52__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED53__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED53__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED54__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED54__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED55__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED55__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED56__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED56__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED57__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED57__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED58__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED58__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED59__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED59__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED60__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED60__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED61__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED61__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED62__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED62__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED63__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED63__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED64__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED64__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED65__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED65__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED66__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED66__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED67__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED67__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED68__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED68__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED69__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED69__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED70__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED70__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED71__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED71__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED72__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED72__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED73__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED73__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED74__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED74__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED75__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED75__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED76__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED76__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED77__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED77__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED78__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED78__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED79__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED79__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED80__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED80__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED81__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED81__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED82__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED82__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED83__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED83__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED84__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED84__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED85__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED85__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED86__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED86__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED87__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED87__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED88__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED88__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED89__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED89__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED90__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED90__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED91__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED91__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED92__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED92__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED93__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED93__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED94__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED94__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED95__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED95__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED96__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED96__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED97__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED97__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED98__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED98__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED99__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED99__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED100__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED100__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED101__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED101__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED102__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED102__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED103__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED103__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED104__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED104__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED105__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED105__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED106__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED106__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED107__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED107__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED108__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED108__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED109__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED109__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED110__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED110__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED111__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED111__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED112__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED112__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED113__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED113__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED114__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED114__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED115__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED115__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED116__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED116__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED117__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED117__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED118__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED118__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED119__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED119__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED120__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED120__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED121__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED121__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED122__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED122__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED123__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED123__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED124__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED124__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED125__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED125__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED126__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED126__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED127__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED127__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED128__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED128__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED129__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED129__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED130__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED130__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED131__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED131__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED132__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED132__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED133__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED133__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED134__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED134__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED135__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED135__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED136__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED136__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED137__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED137__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED138__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED138__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED139__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED139__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED140__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED140__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED141__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED141__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED142__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED142__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED143__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED143__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED144__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED144__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED145__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED145__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED146__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED146__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED147__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED147__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED148__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED148__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED149__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED149__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED150__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED150__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED151__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED151__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED152__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED152__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED153__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED153__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED154__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED154__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED155__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED155__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED156__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED156__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED157__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED157__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED158__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED158__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define UNIPHY_MACRO_CNTL_RESERVED159__UNIPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define UNIPHY_MACRO_CNTL_RESERVED159__UNIPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED0__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED0__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED1__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED1__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED2__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED2__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED3__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED3__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED4__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED4__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED5__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED5__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED6__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED6__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED7__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED7__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED8__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED8__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED9__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED9__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED10__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED10__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED11__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED11__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED12__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED12__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED13__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED13__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED14__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED14__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED15__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED15__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED16__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED16__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED17__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED17__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED18__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED18__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED19__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED19__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED20__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED20__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED21__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED21__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED22__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED22__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED23__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED23__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED24__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED24__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED25__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED25__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED26__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED26__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED27__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED27__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED28__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED28__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED29__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED29__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED30__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED30__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED31__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED31__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED32__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED32__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED33__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED33__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED34__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED34__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED35__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED35__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED36__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED36__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED37__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED37__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED38__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED38__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED39__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED39__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED40__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED40__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED41__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED41__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED42__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED42__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED43__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED43__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED44__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED44__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED45__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED45__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED46__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED46__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED47__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED47__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED48__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED48__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED49__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED49__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED50__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED50__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED51__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED51__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED52__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED52__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED53__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED53__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED54__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED54__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED55__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED55__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED56__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED56__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED57__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED57__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED58__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED58__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED59__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED59__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED60__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED60__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED61__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED61__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED62__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED62__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED63__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED63__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED64__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED64__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED65__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED65__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED66__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED66__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED67__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED67__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED68__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED68__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED69__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED69__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED70__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED70__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED71__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED71__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED72__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED72__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED73__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED73__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED74__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED74__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED75__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED75__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED76__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED76__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED77__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED77__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED78__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED78__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED79__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED79__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED80__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED80__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED81__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED81__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED82__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED82__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED83__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED83__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED84__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED84__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED85__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED85__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED86__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED86__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED87__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED87__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED88__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED88__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED89__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED89__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED90__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED90__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED91__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED91__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED92__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED92__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED93__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED93__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED94__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED94__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED95__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED95__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED96__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED96__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED97__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED97__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED98__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED98__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED99__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED99__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED100__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED100__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED101__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED101__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED102__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED102__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED103__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED103__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED104__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED104__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED105__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED105__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED106__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED106__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED107__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED107__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED108__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED108__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED109__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED109__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED110__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED110__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED111__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED111__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED112__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED112__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED113__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED113__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED114__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED114__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED115__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED115__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED116__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED116__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED117__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED117__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED118__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED118__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED119__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED119__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED120__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED120__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED121__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED121__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED122__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED122__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED123__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED123__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED124__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED124__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED125__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED125__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED126__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED126__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED127__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED127__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED128__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED128__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED129__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED129__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED130__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED130__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED131__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED131__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED132__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED132__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED133__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED133__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED134__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED134__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED135__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED135__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED136__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED136__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED137__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED137__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED138__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED138__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED139__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED139__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED140__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED140__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED141__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED141__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED142__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED142__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED143__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED143__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED144__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED144__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED145__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED145__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED146__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED146__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED147__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED147__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED148__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED148__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED149__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED149__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED150__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED150__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED151__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED151__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED152__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED152__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED153__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED153__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED154__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED154__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED155__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED155__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED156__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED156__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED157__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED157__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED158__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED158__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED159__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED159__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED160__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED160__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED161__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED161__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED162__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED162__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED163__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED163__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED164__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED164__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED165__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED165__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED166__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED166__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED167__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED167__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED168__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED168__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED169__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED169__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED170__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED170__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED171__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED171__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED172__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED172__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED173__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED173__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED174__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED174__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED175__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED175__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED176__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED176__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED177__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED177__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED178__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED178__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED179__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED179__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED180__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED180__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED181__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED181__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED182__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED182__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED183__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED183__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED184__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED184__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED185__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED185__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED186__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED186__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED187__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED187__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED188__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED188__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED189__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED189__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED190__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED190__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED191__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED191__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED192__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED192__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED193__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED193__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED194__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED194__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED195__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED195__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED196__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED196__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED197__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED197__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED198__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED198__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED199__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED199__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED200__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED200__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED201__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED201__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED202__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED202__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED203__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED203__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED204__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED204__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED205__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED205__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED206__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED206__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED207__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED207__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED208__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED208__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED209__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED209__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED210__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED210__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED211__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED211__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED212__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED212__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED213__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED213__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED214__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED214__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED215__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED215__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED216__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED216__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED217__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED217__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED218__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED218__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED219__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED219__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED220__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED220__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED221__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED221__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED222__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED222__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED223__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED223__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED224__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED224__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED225__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED225__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED226__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED226__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED227__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED227__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED228__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED228__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED229__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED229__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED230__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED230__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED231__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED231__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED232__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED232__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED233__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED233__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED234__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED234__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED235__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED235__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED236__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED236__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED237__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED237__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED238__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED238__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED239__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED239__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED240__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED240__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED241__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED241__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED242__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED242__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED243__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED243__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED244__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED244__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED245__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED245__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED246__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED246__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED247__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED247__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED248__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED248__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED249__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED249__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED250__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED250__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED251__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED251__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED252__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED252__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED253__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED253__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED254__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED254__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED255__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED255__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED256__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED256__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED257__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED257__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED258__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED258__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED259__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED259__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED260__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED260__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED261__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED261__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED262__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED262__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED263__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED263__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED264__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED264__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED265__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED265__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED266__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED266__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED267__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED267__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED268__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED268__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED269__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED269__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED270__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED270__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED271__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED271__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED272__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED272__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED273__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED273__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED274__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED274__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED275__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED275__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED276__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED276__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED277__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED277__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED278__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED278__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED279__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED279__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED280__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED280__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED281__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED281__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED282__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED282__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED283__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED283__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED284__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED284__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED285__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED285__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED286__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED286__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED287__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED287__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED288__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED288__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED289__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED289__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED290__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED290__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED291__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED291__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED292__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED292__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED293__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED293__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED294__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED294__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED295__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED295__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED296__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED296__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED297__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED297__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED298__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED298__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED299__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED299__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED300__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED300__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED301__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED301__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED302__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED302__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED303__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED303__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED304__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED304__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED305__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED305__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED306__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED306__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED307__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED307__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED308__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED308__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED309__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED309__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED310__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED310__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED311__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED311__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED312__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED312__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED313__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED313__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED314__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED314__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED315__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED315__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED316__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED316__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED317__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED317__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED318__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED318__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED319__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED319__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED320__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED320__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED321__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED321__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED322__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED322__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED323__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED323__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED324__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED324__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED325__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED325__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED326__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED326__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED327__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED327__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED328__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED328__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED329__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED329__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED330__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED330__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED331__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED331__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED332__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED332__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED333__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED333__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED334__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED334__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED335__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED335__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED336__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED336__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED337__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED337__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED338__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED338__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED339__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED339__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED340__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED340__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED341__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED341__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED342__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED342__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED343__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED343__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED344__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED344__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED345__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED345__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED346__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED346__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED347__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED347__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED348__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED348__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED349__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED349__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED350__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED350__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED351__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED351__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED352__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED352__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED353__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED353__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED354__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED354__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED355__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED355__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED356__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED356__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED357__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED357__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED358__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED358__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED359__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED359__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED360__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED360__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED361__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED361__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED362__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED362__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED363__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED363__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED364__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED364__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED365__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED365__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED366__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED366__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED367__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED367__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED368__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED368__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED369__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED369__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED370__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED370__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED371__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED371__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED372__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED372__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED373__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED373__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED374__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED374__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED375__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED375__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED376__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED376__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED377__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED377__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED378__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED378__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DCRX_PHY_MACRO_CNTL_RESERVED379__DCRX_PHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DCRX_PHY_MACRO_CNTL_RESERVED379__DCRX_PHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED0__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED0__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED1__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED1__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED2__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED2__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED3__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED3__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED4__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED4__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED5__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED5__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED6__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED6__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED7__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED7__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED8__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED8__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED9__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED9__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED10__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED10__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED11__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED11__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED12__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED12__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED13__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED13__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED14__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED14__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED15__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED15__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED16__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED16__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED17__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED17__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED18__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED18__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED19__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED19__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED20__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED20__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED21__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED21__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED22__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED22__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED23__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED23__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED24__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED24__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED25__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED25__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED26__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED26__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED27__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED27__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED28__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED28__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED29__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED29__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED30__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED30__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED31__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED31__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED32__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED32__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED33__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED33__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED34__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED34__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED35__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED35__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED36__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED36__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED37__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED37__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED38__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED38__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED39__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED39__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED40__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED40__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED41__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED41__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED42__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED42__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED43__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED43__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED44__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED44__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED45__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED45__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED46__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED46__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED47__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED47__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED48__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED48__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED49__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED49__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED50__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED50__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED51__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED51__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED52__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED52__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED53__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED53__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED54__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED54__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED55__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED55__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED56__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED56__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED57__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED57__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED58__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED58__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED59__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED59__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED60__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED60__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED61__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED61__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED62__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED62__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define DPHY_MACRO_CNTL_RESERVED63__DPHY_MACRO_CNTL_RESERVED_MASK 0xffffffff
+#define DPHY_MACRO_CNTL_RESERVED63__DPHY_MACRO_CNTL_RESERVED__SHIFT 0x0
+#define GRPH_ENABLE__GRPH_ENABLE_MASK 0x1
+#define GRPH_ENABLE__GRPH_ENABLE__SHIFT 0x0
+#define GRPH_ENABLE__GRPH_KEYER_ALPHA_SEL_MASK 0x2
+#define GRPH_ENABLE__GRPH_KEYER_ALPHA_SEL__SHIFT 0x1
+#define GRPH_CONTROL__GRPH_DEPTH_MASK 0x3
+#define GRPH_CONTROL__GRPH_DEPTH__SHIFT 0x0
+#define GRPH_CONTROL__GRPH_NUM_BANKS_MASK 0xc
+#define GRPH_CONTROL__GRPH_NUM_BANKS__SHIFT 0x2
+#define GRPH_CONTROL__GRPH_Z_MASK 0x30
+#define GRPH_CONTROL__GRPH_Z__SHIFT 0x4
+#define GRPH_CONTROL__GRPH_BANK_WIDTH_MASK 0xc0
+#define GRPH_CONTROL__GRPH_BANK_WIDTH__SHIFT 0x6
+#define GRPH_CONTROL__GRPH_FORMAT_MASK 0x700
+#define GRPH_CONTROL__GRPH_FORMAT__SHIFT 0x8
+#define GRPH_CONTROL__GRPH_BANK_HEIGHT_MASK 0x1800
+#define GRPH_CONTROL__GRPH_BANK_HEIGHT__SHIFT 0xb
+#define GRPH_CONTROL__GRPH_TILE_SPLIT_MASK 0xe000
+#define GRPH_CONTROL__GRPH_TILE_SPLIT__SHIFT 0xd
+#define GRPH_CONTROL__GRPH_ADDRESS_TRANSLATION_ENABLE_MASK 0x10000
+#define GRPH_CONTROL__GRPH_ADDRESS_TRANSLATION_ENABLE__SHIFT 0x10
+#define GRPH_CONTROL__GRPH_PRIVILEGED_ACCESS_ENABLE_MASK 0x20000
+#define GRPH_CONTROL__GRPH_PRIVILEGED_ACCESS_ENABLE__SHIFT 0x11
+#define GRPH_CONTROL__GRPH_MACRO_TILE_ASPECT_MASK 0xc0000
+#define GRPH_CONTROL__GRPH_MACRO_TILE_ASPECT__SHIFT 0x12
+#define GRPH_CONTROL__GRPH_ARRAY_MODE_MASK 0xf00000
+#define GRPH_CONTROL__GRPH_ARRAY_MODE__SHIFT 0x14
+#define GRPH_CONTROL__GRPH_PIPE_CONFIG_MASK 0x1f000000
+#define GRPH_CONTROL__GRPH_PIPE_CONFIG__SHIFT 0x18
+#define GRPH_CONTROL__GRPH_MICRO_TILE_MODE_MASK 0x60000000
+#define GRPH_CONTROL__GRPH_MICRO_TILE_MODE__SHIFT 0x1d
+#define GRPH_CONTROL__GRPH_COLOR_EXPANSION_MODE_MASK 0x80000000
+#define GRPH_CONTROL__GRPH_COLOR_EXPANSION_MODE__SHIFT 0x1f
+#define GRPH_LUT_10BIT_BYPASS__GRPH_LUT_10BIT_BYPASS_EN_MASK 0x100
+#define GRPH_LUT_10BIT_BYPASS__GRPH_LUT_10BIT_BYPASS_EN__SHIFT 0x8
+#define GRPH_LUT_10BIT_BYPASS__GRPH_LUT_10BIT_BYPASS_DBL_BUF_EN_MASK 0x10000
+#define GRPH_LUT_10BIT_BYPASS__GRPH_LUT_10BIT_BYPASS_DBL_BUF_EN__SHIFT 0x10
+#define GRPH_SWAP_CNTL__GRPH_ENDIAN_SWAP_MASK 0x3
+#define GRPH_SWAP_CNTL__GRPH_ENDIAN_SWAP__SHIFT 0x0
+#define GRPH_SWAP_CNTL__GRPH_RED_CROSSBAR_MASK 0x30
+#define GRPH_SWAP_CNTL__GRPH_RED_CROSSBAR__SHIFT 0x4
+#define GRPH_SWAP_CNTL__GRPH_GREEN_CROSSBAR_MASK 0xc0
+#define GRPH_SWAP_CNTL__GRPH_GREEN_CROSSBAR__SHIFT 0x6
+#define GRPH_SWAP_CNTL__GRPH_BLUE_CROSSBAR_MASK 0x300
+#define GRPH_SWAP_CNTL__GRPH_BLUE_CROSSBAR__SHIFT 0x8
+#define GRPH_SWAP_CNTL__GRPH_ALPHA_CROSSBAR_MASK 0xc00
+#define GRPH_SWAP_CNTL__GRPH_ALPHA_CROSSBAR__SHIFT 0xa
+#define GRPH_PRIMARY_SURFACE_ADDRESS__GRPH_PRIMARY_DFQ_ENABLE_MASK 0x1
+#define GRPH_PRIMARY_SURFACE_ADDRESS__GRPH_PRIMARY_DFQ_ENABLE__SHIFT 0x0
+#define GRPH_PRIMARY_SURFACE_ADDRESS__GRPH_PRIMARY_SURFACE_ADDRESS_MASK 0xffffff00
+#define GRPH_PRIMARY_SURFACE_ADDRESS__GRPH_PRIMARY_SURFACE_ADDRESS__SHIFT 0x8
+#define GRPH_SECONDARY_SURFACE_ADDRESS__GRPH_SECONDARY_DFQ_ENABLE_MASK 0x1
+#define GRPH_SECONDARY_SURFACE_ADDRESS__GRPH_SECONDARY_DFQ_ENABLE__SHIFT 0x0
+#define GRPH_SECONDARY_SURFACE_ADDRESS__GRPH_SECONDARY_SURFACE_ADDRESS_MASK 0xffffff00
+#define GRPH_SECONDARY_SURFACE_ADDRESS__GRPH_SECONDARY_SURFACE_ADDRESS__SHIFT 0x8
+#define GRPH_PITCH__GRPH_PITCH_MASK 0x7fff
+#define GRPH_PITCH__GRPH_PITCH__SHIFT 0x0
+#define GRPH_PRIMARY_SURFACE_ADDRESS_HIGH__GRPH_PRIMARY_SURFACE_ADDRESS_HIGH_MASK 0xff
+#define GRPH_PRIMARY_SURFACE_ADDRESS_HIGH__GRPH_PRIMARY_SURFACE_ADDRESS_HIGH__SHIFT 0x0
+#define GRPH_SECONDARY_SURFACE_ADDRESS_HIGH__GRPH_SECONDARY_SURFACE_ADDRESS_HIGH_MASK 0xff
+#define GRPH_SECONDARY_SURFACE_ADDRESS_HIGH__GRPH_SECONDARY_SURFACE_ADDRESS_HIGH__SHIFT 0x0
+#define GRPH_SURFACE_OFFSET_X__GRPH_SURFACE_OFFSET_X_MASK 0x3fff
+#define GRPH_SURFACE_OFFSET_X__GRPH_SURFACE_OFFSET_X__SHIFT 0x0
+#define GRPH_SURFACE_OFFSET_Y__GRPH_SURFACE_OFFSET_Y_MASK 0x3fff
+#define GRPH_SURFACE_OFFSET_Y__GRPH_SURFACE_OFFSET_Y__SHIFT 0x0
+#define GRPH_X_START__GRPH_X_START_MASK 0x3fff
+#define GRPH_X_START__GRPH_X_START__SHIFT 0x0
+#define GRPH_Y_START__GRPH_Y_START_MASK 0x3fff
+#define GRPH_Y_START__GRPH_Y_START__SHIFT 0x0
+#define GRPH_X_END__GRPH_X_END_MASK 0x7fff
+#define GRPH_X_END__GRPH_X_END__SHIFT 0x0
+#define GRPH_Y_END__GRPH_Y_END_MASK 0x7fff
+#define GRPH_Y_END__GRPH_Y_END__SHIFT 0x0
+#define INPUT_GAMMA_CONTROL__GRPH_INPUT_GAMMA_MODE_MASK 0x1
+#define INPUT_GAMMA_CONTROL__GRPH_INPUT_GAMMA_MODE__SHIFT 0x0
+#define GRPH_UPDATE__GRPH_MODE_UPDATE_PENDING_MASK 0x1
+#define GRPH_UPDATE__GRPH_MODE_UPDATE_PENDING__SHIFT 0x0
+#define GRPH_UPDATE__GRPH_MODE_UPDATE_TAKEN_MASK 0x2
+#define GRPH_UPDATE__GRPH_MODE_UPDATE_TAKEN__SHIFT 0x1
+#define GRPH_UPDATE__GRPH_SURFACE_UPDATE_PENDING_MASK 0x4
+#define GRPH_UPDATE__GRPH_SURFACE_UPDATE_PENDING__SHIFT 0x2
+#define GRPH_UPDATE__GRPH_SURFACE_UPDATE_TAKEN_MASK 0x8
+#define GRPH_UPDATE__GRPH_SURFACE_UPDATE_TAKEN__SHIFT 0x3
+#define GRPH_UPDATE__GRPH_SURFACE_XDMA_PENDING_ENABLE_MASK 0x100
+#define GRPH_UPDATE__GRPH_SURFACE_XDMA_PENDING_ENABLE__SHIFT 0x8
+#define GRPH_UPDATE__GRPH_UPDATE_LOCK_MASK 0x10000
+#define GRPH_UPDATE__GRPH_UPDATE_LOCK__SHIFT 0x10
+#define GRPH_UPDATE__GRPH_SURFACE_IGNORE_UPDATE_LOCK_MASK 0x100000
+#define GRPH_UPDATE__GRPH_SURFACE_IGNORE_UPDATE_LOCK__SHIFT 0x14
+#define GRPH_UPDATE__GRPH_MODE_DISABLE_MULTIPLE_UPDATE_MASK 0x1000000
+#define GRPH_UPDATE__GRPH_MODE_DISABLE_MULTIPLE_UPDATE__SHIFT 0x18
+#define GRPH_UPDATE__GRPH_SURFACE_DISABLE_MULTIPLE_UPDATE_MASK 0x10000000
+#define GRPH_UPDATE__GRPH_SURFACE_DISABLE_MULTIPLE_UPDATE__SHIFT 0x1c
+#define GRPH_FLIP_CONTROL__GRPH_SURFACE_UPDATE_H_RETRACE_EN_MASK 0x1
+#define GRPH_FLIP_CONTROL__GRPH_SURFACE_UPDATE_H_RETRACE_EN__SHIFT 0x0
+#define GRPH_FLIP_CONTROL__GRPH_XDMA_SUPER_AA_EN_MASK 0x2
+#define GRPH_FLIP_CONTROL__GRPH_XDMA_SUPER_AA_EN__SHIFT 0x1
+#define GRPH_FLIP_CONTROL__GRPH_SURFACE_UPDATE_IMMEDIATE_EN_MASK 0x10
+#define GRPH_FLIP_CONTROL__GRPH_SURFACE_UPDATE_IMMEDIATE_EN__SHIFT 0x4
+#define GRPH_FLIP_CONTROL__GRPH_SURFACE_UPDATE_PENDING_MODE_MASK 0x20
+#define GRPH_FLIP_CONTROL__GRPH_SURFACE_UPDATE_PENDING_MODE__SHIFT 0x5
+#define GRPH_SURFACE_ADDRESS_INUSE__GRPH_SURFACE_ADDRESS_INUSE_MASK 0xffffff00
+#define GRPH_SURFACE_ADDRESS_INUSE__GRPH_SURFACE_ADDRESS_INUSE__SHIFT 0x8
+#define GRPH_DFQ_CONTROL__GRPH_DFQ_RESET_MASK 0x1
+#define GRPH_DFQ_CONTROL__GRPH_DFQ_RESET__SHIFT 0x0
+#define GRPH_DFQ_CONTROL__GRPH_DFQ_SIZE_MASK 0x70
+#define GRPH_DFQ_CONTROL__GRPH_DFQ_SIZE__SHIFT 0x4
+#define GRPH_DFQ_CONTROL__GRPH_DFQ_MIN_FREE_ENTRIES_MASK 0x700
+#define GRPH_DFQ_CONTROL__GRPH_DFQ_MIN_FREE_ENTRIES__SHIFT 0x8
+#define GRPH_DFQ_STATUS__GRPH_PRIMARY_DFQ_NUM_ENTRIES_MASK 0xf
+#define GRPH_DFQ_STATUS__GRPH_PRIMARY_DFQ_NUM_ENTRIES__SHIFT 0x0
+#define GRPH_DFQ_STATUS__GRPH_SECONDARY_DFQ_NUM_ENTRIES_MASK 0xf0
+#define GRPH_DFQ_STATUS__GRPH_SECONDARY_DFQ_NUM_ENTRIES__SHIFT 0x4
+#define GRPH_DFQ_STATUS__GRPH_DFQ_RESET_FLAG_MASK 0x100
+#define GRPH_DFQ_STATUS__GRPH_DFQ_RESET_FLAG__SHIFT 0x8
+#define GRPH_DFQ_STATUS__GRPH_DFQ_RESET_ACK_MASK 0x200
+#define GRPH_DFQ_STATUS__GRPH_DFQ_RESET_ACK__SHIFT 0x9
+#define GRPH_INTERRUPT_STATUS__GRPH_PFLIP_INT_OCCURRED_MASK 0x1
+#define GRPH_INTERRUPT_STATUS__GRPH_PFLIP_INT_OCCURRED__SHIFT 0x0
+#define GRPH_INTERRUPT_STATUS__GRPH_PFLIP_INT_CLEAR_MASK 0x100
+#define GRPH_INTERRUPT_STATUS__GRPH_PFLIP_INT_CLEAR__SHIFT 0x8
+#define GRPH_INTERRUPT_CONTROL__GRPH_PFLIP_INT_MASK_MASK 0x1
+#define GRPH_INTERRUPT_CONTROL__GRPH_PFLIP_INT_MASK__SHIFT 0x0
+#define GRPH_INTERRUPT_CONTROL__GRPH_PFLIP_INT_TYPE_MASK 0x100
+#define GRPH_INTERRUPT_CONTROL__GRPH_PFLIP_INT_TYPE__SHIFT 0x8
+#define GRPH_SURFACE_ADDRESS_HIGH_INUSE__GRPH_SURFACE_ADDRESS_HIGH_INUSE_MASK 0xff
+#define GRPH_SURFACE_ADDRESS_HIGH_INUSE__GRPH_SURFACE_ADDRESS_HIGH_INUSE__SHIFT 0x0
+#define GRPH_COMPRESS_SURFACE_ADDRESS__GRPH_COMPRESS_SURFACE_ADDRESS_MASK 0xffffff00
+#define GRPH_COMPRESS_SURFACE_ADDRESS__GRPH_COMPRESS_SURFACE_ADDRESS__SHIFT 0x8
+#define GRPH_COMPRESS_PITCH__GRPH_COMPRESS_PITCH_MASK 0x1ffc0
+#define GRPH_COMPRESS_PITCH__GRPH_COMPRESS_PITCH__SHIFT 0x6
+#define GRPH_COMPRESS_SURFACE_ADDRESS_HIGH__GRPH_COMPRESS_SURFACE_ADDRESS_HIGH_MASK 0xff
+#define GRPH_COMPRESS_SURFACE_ADDRESS_HIGH__GRPH_COMPRESS_SURFACE_ADDRESS_HIGH__SHIFT 0x0
+#define GRPH_PIPE_OUTSTANDING_REQUEST_LIMIT__GRPH_PIPE_OUTSTANDING_REQUEST_LIMIT_MASK 0xff
+#define GRPH_PIPE_OUTSTANDING_REQUEST_LIMIT__GRPH_PIPE_OUTSTANDING_REQUEST_LIMIT__SHIFT 0x0
+#define PRESCALE_GRPH_CONTROL__GRPH_PRESCALE_SELECT_MASK 0x1
+#define PRESCALE_GRPH_CONTROL__GRPH_PRESCALE_SELECT__SHIFT 0x0
+#define PRESCALE_GRPH_CONTROL__GRPH_PRESCALE_R_SIGN_MASK 0x2
+#define PRESCALE_GRPH_CONTROL__GRPH_PRESCALE_R_SIGN__SHIFT 0x1
+#define PRESCALE_GRPH_CONTROL__GRPH_PRESCALE_G_SIGN_MASK 0x4
+#define PRESCALE_GRPH_CONTROL__GRPH_PRESCALE_G_SIGN__SHIFT 0x2
+#define PRESCALE_GRPH_CONTROL__GRPH_PRESCALE_B_SIGN_MASK 0x8
+#define PRESCALE_GRPH_CONTROL__GRPH_PRESCALE_B_SIGN__SHIFT 0x3
+#define PRESCALE_GRPH_CONTROL__GRPH_PRESCALE_BYPASS_MASK 0x10
+#define PRESCALE_GRPH_CONTROL__GRPH_PRESCALE_BYPASS__SHIFT 0x4
+#define PRESCALE_VALUES_GRPH_R__GRPH_PRESCALE_BIAS_R_MASK 0xffff
+#define PRESCALE_VALUES_GRPH_R__GRPH_PRESCALE_BIAS_R__SHIFT 0x0
+#define PRESCALE_VALUES_GRPH_R__GRPH_PRESCALE_SCALE_R_MASK 0xffff0000
+#define PRESCALE_VALUES_GRPH_R__GRPH_PRESCALE_SCALE_R__SHIFT 0x10
+#define PRESCALE_VALUES_GRPH_G__GRPH_PRESCALE_BIAS_G_MASK 0xffff
+#define PRESCALE_VALUES_GRPH_G__GRPH_PRESCALE_BIAS_G__SHIFT 0x0
+#define PRESCALE_VALUES_GRPH_G__GRPH_PRESCALE_SCALE_G_MASK 0xffff0000
+#define PRESCALE_VALUES_GRPH_G__GRPH_PRESCALE_SCALE_G__SHIFT 0x10
+#define PRESCALE_VALUES_GRPH_B__GRPH_PRESCALE_BIAS_B_MASK 0xffff
+#define PRESCALE_VALUES_GRPH_B__GRPH_PRESCALE_BIAS_B__SHIFT 0x0
+#define PRESCALE_VALUES_GRPH_B__GRPH_PRESCALE_SCALE_B_MASK 0xffff0000
+#define PRESCALE_VALUES_GRPH_B__GRPH_PRESCALE_SCALE_B__SHIFT 0x10
+#define INPUT_CSC_CONTROL__INPUT_CSC_GRPH_MODE_MASK 0x3
+#define INPUT_CSC_CONTROL__INPUT_CSC_GRPH_MODE__SHIFT 0x0
+#define INPUT_CSC_C11_C12__INPUT_CSC_C11_MASK 0xffff
+#define INPUT_CSC_C11_C12__INPUT_CSC_C11__SHIFT 0x0
+#define INPUT_CSC_C11_C12__INPUT_CSC_C12_MASK 0xffff0000
+#define INPUT_CSC_C11_C12__INPUT_CSC_C12__SHIFT 0x10
+#define INPUT_CSC_C13_C14__INPUT_CSC_C13_MASK 0xffff
+#define INPUT_CSC_C13_C14__INPUT_CSC_C13__SHIFT 0x0
+#define INPUT_CSC_C13_C14__INPUT_CSC_C14_MASK 0xffff0000
+#define INPUT_CSC_C13_C14__INPUT_CSC_C14__SHIFT 0x10
+#define INPUT_CSC_C21_C22__INPUT_CSC_C21_MASK 0xffff
+#define INPUT_CSC_C21_C22__INPUT_CSC_C21__SHIFT 0x0
+#define INPUT_CSC_C21_C22__INPUT_CSC_C22_MASK 0xffff0000
+#define INPUT_CSC_C21_C22__INPUT_CSC_C22__SHIFT 0x10
+#define INPUT_CSC_C23_C24__INPUT_CSC_C23_MASK 0xffff
+#define INPUT_CSC_C23_C24__INPUT_CSC_C23__SHIFT 0x0
+#define INPUT_CSC_C23_C24__INPUT_CSC_C24_MASK 0xffff0000
+#define INPUT_CSC_C23_C24__INPUT_CSC_C24__SHIFT 0x10
+#define INPUT_CSC_C31_C32__INPUT_CSC_C31_MASK 0xffff
+#define INPUT_CSC_C31_C32__INPUT_CSC_C31__SHIFT 0x0
+#define INPUT_CSC_C31_C32__INPUT_CSC_C32_MASK 0xffff0000
+#define INPUT_CSC_C31_C32__INPUT_CSC_C32__SHIFT 0x10
+#define INPUT_CSC_C33_C34__INPUT_CSC_C33_MASK 0xffff
+#define INPUT_CSC_C33_C34__INPUT_CSC_C33__SHIFT 0x0
+#define INPUT_CSC_C33_C34__INPUT_CSC_C34_MASK 0xffff0000
+#define INPUT_CSC_C33_C34__INPUT_CSC_C34__SHIFT 0x10
+#define OUTPUT_CSC_CONTROL__OUTPUT_CSC_GRPH_MODE_MASK 0x7
+#define OUTPUT_CSC_CONTROL__OUTPUT_CSC_GRPH_MODE__SHIFT 0x0
+#define OUTPUT_CSC_C11_C12__OUTPUT_CSC_C11_MASK 0xffff
+#define OUTPUT_CSC_C11_C12__OUTPUT_CSC_C11__SHIFT 0x0
+#define OUTPUT_CSC_C11_C12__OUTPUT_CSC_C12_MASK 0xffff0000
+#define OUTPUT_CSC_C11_C12__OUTPUT_CSC_C12__SHIFT 0x10
+#define OUTPUT_CSC_C13_C14__OUTPUT_CSC_C13_MASK 0xffff
+#define OUTPUT_CSC_C13_C14__OUTPUT_CSC_C13__SHIFT 0x0
+#define OUTPUT_CSC_C13_C14__OUTPUT_CSC_C14_MASK 0xffff0000
+#define OUTPUT_CSC_C13_C14__OUTPUT_CSC_C14__SHIFT 0x10
+#define OUTPUT_CSC_C21_C22__OUTPUT_CSC_C21_MASK 0xffff
+#define OUTPUT_CSC_C21_C22__OUTPUT_CSC_C21__SHIFT 0x0
+#define OUTPUT_CSC_C21_C22__OUTPUT_CSC_C22_MASK 0xffff0000
+#define OUTPUT_CSC_C21_C22__OUTPUT_CSC_C22__SHIFT 0x10
+#define OUTPUT_CSC_C23_C24__OUTPUT_CSC_C23_MASK 0xffff
+#define OUTPUT_CSC_C23_C24__OUTPUT_CSC_C23__SHIFT 0x0
+#define OUTPUT_CSC_C23_C24__OUTPUT_CSC_C24_MASK 0xffff0000
+#define OUTPUT_CSC_C23_C24__OUTPUT_CSC_C24__SHIFT 0x10
+#define OUTPUT_CSC_C31_C32__OUTPUT_CSC_C31_MASK 0xffff
+#define OUTPUT_CSC_C31_C32__OUTPUT_CSC_C31__SHIFT 0x0
+#define OUTPUT_CSC_C31_C32__OUTPUT_CSC_C32_MASK 0xffff0000
+#define OUTPUT_CSC_C31_C32__OUTPUT_CSC_C32__SHIFT 0x10
+#define OUTPUT_CSC_C33_C34__OUTPUT_CSC_C33_MASK 0xffff
+#define OUTPUT_CSC_C33_C34__OUTPUT_CSC_C33__SHIFT 0x0
+#define OUTPUT_CSC_C33_C34__OUTPUT_CSC_C34_MASK 0xffff0000
+#define OUTPUT_CSC_C33_C34__OUTPUT_CSC_C34__SHIFT 0x10
+#define COMM_MATRIXA_TRANS_C11_C12__COMM_MATRIXA_TRANS_C11_MASK 0xffff
+#define COMM_MATRIXA_TRANS_C11_C12__COMM_MATRIXA_TRANS_C11__SHIFT 0x0
+#define COMM_MATRIXA_TRANS_C11_C12__COMM_MATRIXA_TRANS_C12_MASK 0xffff0000
+#define COMM_MATRIXA_TRANS_C11_C12__COMM_MATRIXA_TRANS_C12__SHIFT 0x10
+#define COMM_MATRIXA_TRANS_C13_C14__COMM_MATRIXA_TRANS_C13_MASK 0xffff
+#define COMM_MATRIXA_TRANS_C13_C14__COMM_MATRIXA_TRANS_C13__SHIFT 0x0
+#define COMM_MATRIXA_TRANS_C13_C14__COMM_MATRIXA_TRANS_C14_MASK 0xffff0000
+#define COMM_MATRIXA_TRANS_C13_C14__COMM_MATRIXA_TRANS_C14__SHIFT 0x10
+#define COMM_MATRIXA_TRANS_C21_C22__COMM_MATRIXA_TRANS_C21_MASK 0xffff
+#define COMM_MATRIXA_TRANS_C21_C22__COMM_MATRIXA_TRANS_C21__SHIFT 0x0
+#define COMM_MATRIXA_TRANS_C21_C22__COMM_MATRIXA_TRANS_C22_MASK 0xffff0000
+#define COMM_MATRIXA_TRANS_C21_C22__COMM_MATRIXA_TRANS_C22__SHIFT 0x10
+#define COMM_MATRIXA_TRANS_C23_C24__COMM_MATRIXA_TRANS_C23_MASK 0xffff
+#define COMM_MATRIXA_TRANS_C23_C24__COMM_MATRIXA_TRANS_C23__SHIFT 0x0
+#define COMM_MATRIXA_TRANS_C23_C24__COMM_MATRIXA_TRANS_C24_MASK 0xffff0000
+#define COMM_MATRIXA_TRANS_C23_C24__COMM_MATRIXA_TRANS_C24__SHIFT 0x10
+#define COMM_MATRIXA_TRANS_C31_C32__COMM_MATRIXA_TRANS_C31_MASK 0xffff
+#define COMM_MATRIXA_TRANS_C31_C32__COMM_MATRIXA_TRANS_C31__SHIFT 0x0
+#define COMM_MATRIXA_TRANS_C31_C32__COMM_MATRIXA_TRANS_C32_MASK 0xffff0000
+#define COMM_MATRIXA_TRANS_C31_C32__COMM_MATRIXA_TRANS_C32__SHIFT 0x10
+#define COMM_MATRIXA_TRANS_C33_C34__COMM_MATRIXA_TRANS_C33_MASK 0xffff
+#define COMM_MATRIXA_TRANS_C33_C34__COMM_MATRIXA_TRANS_C33__SHIFT 0x0
+#define COMM_MATRIXA_TRANS_C33_C34__COMM_MATRIXA_TRANS_C34_MASK 0xffff0000
+#define COMM_MATRIXA_TRANS_C33_C34__COMM_MATRIXA_TRANS_C34__SHIFT 0x10
+#define COMM_MATRIXB_TRANS_C11_C12__COMM_MATRIXB_TRANS_C11_MASK 0xffff
+#define COMM_MATRIXB_TRANS_C11_C12__COMM_MATRIXB_TRANS_C11__SHIFT 0x0
+#define COMM_MATRIXB_TRANS_C11_C12__COMM_MATRIXB_TRANS_C12_MASK 0xffff0000
+#define COMM_MATRIXB_TRANS_C11_C12__COMM_MATRIXB_TRANS_C12__SHIFT 0x10
+#define COMM_MATRIXB_TRANS_C13_C14__COMM_MATRIXB_TRANS_C13_MASK 0xffff
+#define COMM_MATRIXB_TRANS_C13_C14__COMM_MATRIXB_TRANS_C13__SHIFT 0x0
+#define COMM_MATRIXB_TRANS_C13_C14__COMM_MATRIXB_TRANS_C14_MASK 0xffff0000
+#define COMM_MATRIXB_TRANS_C13_C14__COMM_MATRIXB_TRANS_C14__SHIFT 0x10
+#define COMM_MATRIXB_TRANS_C21_C22__COMM_MATRIXB_TRANS_C21_MASK 0xffff
+#define COMM_MATRIXB_TRANS_C21_C22__COMM_MATRIXB_TRANS_C21__SHIFT 0x0
+#define COMM_MATRIXB_TRANS_C21_C22__COMM_MATRIXB_TRANS_C22_MASK 0xffff0000
+#define COMM_MATRIXB_TRANS_C21_C22__COMM_MATRIXB_TRANS_C22__SHIFT 0x10
+#define COMM_MATRIXB_TRANS_C23_C24__COMM_MATRIXB_TRANS_C23_MASK 0xffff
+#define COMM_MATRIXB_TRANS_C23_C24__COMM_MATRIXB_TRANS_C23__SHIFT 0x0
+#define COMM_MATRIXB_TRANS_C23_C24__COMM_MATRIXB_TRANS_C24_MASK 0xffff0000
+#define COMM_MATRIXB_TRANS_C23_C24__COMM_MATRIXB_TRANS_C24__SHIFT 0x10
+#define COMM_MATRIXB_TRANS_C31_C32__COMM_MATRIXB_TRANS_C31_MASK 0xffff
+#define COMM_MATRIXB_TRANS_C31_C32__COMM_MATRIXB_TRANS_C31__SHIFT 0x0
+#define COMM_MATRIXB_TRANS_C31_C32__COMM_MATRIXB_TRANS_C32_MASK 0xffff0000
+#define COMM_MATRIXB_TRANS_C31_C32__COMM_MATRIXB_TRANS_C32__SHIFT 0x10
+#define COMM_MATRIXB_TRANS_C33_C34__COMM_MATRIXB_TRANS_C33_MASK 0xffff
+#define COMM_MATRIXB_TRANS_C33_C34__COMM_MATRIXB_TRANS_C33__SHIFT 0x0
+#define COMM_MATRIXB_TRANS_C33_C34__COMM_MATRIXB_TRANS_C34_MASK 0xffff0000
+#define COMM_MATRIXB_TRANS_C33_C34__COMM_MATRIXB_TRANS_C34__SHIFT 0x10
+#define DENORM_CONTROL__DENORM_MODE_MASK 0x7
+#define DENORM_CONTROL__DENORM_MODE__SHIFT 0x0
+#define DENORM_CONTROL__DENORM_14BIT_OUT_MASK 0x10
+#define DENORM_CONTROL__DENORM_14BIT_OUT__SHIFT 0x4
+#define OUT_ROUND_CONTROL__OUT_ROUND_TRUNC_MODE_MASK 0xf
+#define OUT_ROUND_CONTROL__OUT_ROUND_TRUNC_MODE__SHIFT 0x0
+#define OUT_CLAMP_CONTROL_R_CR__OUT_CLAMP_MAX_R_CR_MASK 0x3fff
+#define OUT_CLAMP_CONTROL_R_CR__OUT_CLAMP_MAX_R_CR__SHIFT 0x0
+#define OUT_CLAMP_CONTROL_R_CR__OUT_CLAMP_MIN_R_CR_MASK 0x3fff0000
+#define OUT_CLAMP_CONTROL_R_CR__OUT_CLAMP_MIN_R_CR__SHIFT 0x10
+#define OUT_CLAMP_CONTROL_G_Y__OUT_CLAMP_MAX_G_Y_MASK 0x3fff
+#define OUT_CLAMP_CONTROL_G_Y__OUT_CLAMP_MAX_G_Y__SHIFT 0x0
+#define OUT_CLAMP_CONTROL_G_Y__OUT_CLAMP_MIN_G_Y_MASK 0x3fff0000
+#define OUT_CLAMP_CONTROL_G_Y__OUT_CLAMP_MIN_G_Y__SHIFT 0x10
+#define OUT_CLAMP_CONTROL_B_CB__OUT_CLAMP_MAX_B_CB_MASK 0x3fff
+#define OUT_CLAMP_CONTROL_B_CB__OUT_CLAMP_MAX_B_CB__SHIFT 0x0
+#define OUT_CLAMP_CONTROL_B_CB__OUT_CLAMP_MIN_B_CB_MASK 0x3fff0000
+#define OUT_CLAMP_CONTROL_B_CB__OUT_CLAMP_MIN_B_CB__SHIFT 0x10
+#define KEY_CONTROL__KEY_MODE_MASK 0x6
+#define KEY_CONTROL__KEY_MODE__SHIFT 0x1
+#define KEY_RANGE_ALPHA__KEY_ALPHA_LOW_MASK 0xffff
+#define KEY_RANGE_ALPHA__KEY_ALPHA_LOW__SHIFT 0x0
+#define KEY_RANGE_ALPHA__KEY_ALPHA_HIGH_MASK 0xffff0000
+#define KEY_RANGE_ALPHA__KEY_ALPHA_HIGH__SHIFT 0x10
+#define KEY_RANGE_RED__KEY_RED_LOW_MASK 0xffff
+#define KEY_RANGE_RED__KEY_RED_LOW__SHIFT 0x0
+#define KEY_RANGE_RED__KEY_RED_HIGH_MASK 0xffff0000
+#define KEY_RANGE_RED__KEY_RED_HIGH__SHIFT 0x10
+#define KEY_RANGE_GREEN__KEY_GREEN_LOW_MASK 0xffff
+#define KEY_RANGE_GREEN__KEY_GREEN_LOW__SHIFT 0x0
+#define KEY_RANGE_GREEN__KEY_GREEN_HIGH_MASK 0xffff0000
+#define KEY_RANGE_GREEN__KEY_GREEN_HIGH__SHIFT 0x10
+#define KEY_RANGE_BLUE__KEY_BLUE_LOW_MASK 0xffff
+#define KEY_RANGE_BLUE__KEY_BLUE_LOW__SHIFT 0x0
+#define KEY_RANGE_BLUE__KEY_BLUE_HIGH_MASK 0xffff0000
+#define KEY_RANGE_BLUE__KEY_BLUE_HIGH__SHIFT 0x10
+#define DEGAMMA_CONTROL__GRPH_DEGAMMA_MODE_MASK 0x3
+#define DEGAMMA_CONTROL__GRPH_DEGAMMA_MODE__SHIFT 0x0
+#define DEGAMMA_CONTROL__CURSOR2_DEGAMMA_MODE_MASK 0x300
+#define DEGAMMA_CONTROL__CURSOR2_DEGAMMA_MODE__SHIFT 0x8
+#define DEGAMMA_CONTROL__CURSOR_DEGAMMA_MODE_MASK 0x3000
+#define DEGAMMA_CONTROL__CURSOR_DEGAMMA_MODE__SHIFT 0xc
+#define GAMUT_REMAP_CONTROL__GRPH_GAMUT_REMAP_MODE_MASK 0x3
+#define GAMUT_REMAP_CONTROL__GRPH_GAMUT_REMAP_MODE__SHIFT 0x0
+#define GAMUT_REMAP_C11_C12__GAMUT_REMAP_C11_MASK 0xffff
+#define GAMUT_REMAP_C11_C12__GAMUT_REMAP_C11__SHIFT 0x0
+#define GAMUT_REMAP_C11_C12__GAMUT_REMAP_C12_MASK 0xffff0000
+#define GAMUT_REMAP_C11_C12__GAMUT_REMAP_C12__SHIFT 0x10
+#define GAMUT_REMAP_C13_C14__GAMUT_REMAP_C13_MASK 0xffff
+#define GAMUT_REMAP_C13_C14__GAMUT_REMAP_C13__SHIFT 0x0
+#define GAMUT_REMAP_C13_C14__GAMUT_REMAP_C14_MASK 0xffff0000
+#define GAMUT_REMAP_C13_C14__GAMUT_REMAP_C14__SHIFT 0x10
+#define GAMUT_REMAP_C21_C22__GAMUT_REMAP_C21_MASK 0xffff
+#define GAMUT_REMAP_C21_C22__GAMUT_REMAP_C21__SHIFT 0x0
+#define GAMUT_REMAP_C21_C22__GAMUT_REMAP_C22_MASK 0xffff0000
+#define GAMUT_REMAP_C21_C22__GAMUT_REMAP_C22__SHIFT 0x10
+#define GAMUT_REMAP_C23_C24__GAMUT_REMAP_C23_MASK 0xffff
+#define GAMUT_REMAP_C23_C24__GAMUT_REMAP_C23__SHIFT 0x0
+#define GAMUT_REMAP_C23_C24__GAMUT_REMAP_C24_MASK 0xffff0000
+#define GAMUT_REMAP_C23_C24__GAMUT_REMAP_C24__SHIFT 0x10
+#define GAMUT_REMAP_C31_C32__GAMUT_REMAP_C31_MASK 0xffff
+#define GAMUT_REMAP_C31_C32__GAMUT_REMAP_C31__SHIFT 0x0
+#define GAMUT_REMAP_C31_C32__GAMUT_REMAP_C32_MASK 0xffff0000
+#define GAMUT_REMAP_C31_C32__GAMUT_REMAP_C32__SHIFT 0x10
+#define GAMUT_REMAP_C33_C34__GAMUT_REMAP_C33_MASK 0xffff
+#define GAMUT_REMAP_C33_C34__GAMUT_REMAP_C33__SHIFT 0x0
+#define GAMUT_REMAP_C33_C34__GAMUT_REMAP_C34_MASK 0xffff0000
+#define GAMUT_REMAP_C33_C34__GAMUT_REMAP_C34__SHIFT 0x10
+#define DCP_SPATIAL_DITHER_CNTL__DCP_SPATIAL_DITHER_EN_MASK 0x1
+#define DCP_SPATIAL_DITHER_CNTL__DCP_SPATIAL_DITHER_EN__SHIFT 0x0
+#define DCP_SPATIAL_DITHER_CNTL__DCP_SPATIAL_DITHER_MODE_MASK 0x30
+#define DCP_SPATIAL_DITHER_CNTL__DCP_SPATIAL_DITHER_MODE__SHIFT 0x4
+#define DCP_SPATIAL_DITHER_CNTL__DCP_SPATIAL_DITHER_DEPTH_MASK 0xc0
+#define DCP_SPATIAL_DITHER_CNTL__DCP_SPATIAL_DITHER_DEPTH__SHIFT 0x6
+#define DCP_SPATIAL_DITHER_CNTL__DCP_FRAME_RANDOM_ENABLE_MASK 0x100
+#define DCP_SPATIAL_DITHER_CNTL__DCP_FRAME_RANDOM_ENABLE__SHIFT 0x8
+#define DCP_SPATIAL_DITHER_CNTL__DCP_RGB_RANDOM_ENABLE_MASK 0x200
+#define DCP_SPATIAL_DITHER_CNTL__DCP_RGB_RANDOM_ENABLE__SHIFT 0x9
+#define DCP_SPATIAL_DITHER_CNTL__DCP_HIGHPASS_RANDOM_ENABLE_MASK 0x400
+#define DCP_SPATIAL_DITHER_CNTL__DCP_HIGHPASS_RANDOM_ENABLE__SHIFT 0xa
+#define DCP_RANDOM_SEEDS__DCP_RAND_R_SEED_MASK 0xff
+#define DCP_RANDOM_SEEDS__DCP_RAND_R_SEED__SHIFT 0x0
+#define DCP_RANDOM_SEEDS__DCP_RAND_G_SEED_MASK 0xff00
+#define DCP_RANDOM_SEEDS__DCP_RAND_G_SEED__SHIFT 0x8
+#define DCP_RANDOM_SEEDS__DCP_RAND_B_SEED_MASK 0xff0000
+#define DCP_RANDOM_SEEDS__DCP_RAND_B_SEED__SHIFT 0x10
+#define DCP_FP_CONVERTED_FIELD__DCP_FP_CONVERTED_FIELD_DATA_MASK 0x3ffff
+#define DCP_FP_CONVERTED_FIELD__DCP_FP_CONVERTED_FIELD_DATA__SHIFT 0x0
+#define DCP_FP_CONVERTED_FIELD__DCP_FP_CONVERTED_FIELD_INDEX_MASK 0x7f00000
+#define DCP_FP_CONVERTED_FIELD__DCP_FP_CONVERTED_FIELD_INDEX__SHIFT 0x14
+#define CUR_CONTROL__CURSOR_EN_MASK 0x1
+#define CUR_CONTROL__CURSOR_EN__SHIFT 0x0
+#define CUR_CONTROL__CUR_INV_TRANS_CLAMP_MASK 0x10
+#define CUR_CONTROL__CUR_INV_TRANS_CLAMP__SHIFT 0x4
+#define CUR_CONTROL__CURSOR_MODE_MASK 0x300
+#define CUR_CONTROL__CURSOR_MODE__SHIFT 0x8
+#define CUR_CONTROL__CURSOR_BUSY_START_LINE_POSITION_MASK 0xf000
+#define CUR_CONTROL__CURSOR_BUSY_START_LINE_POSITION__SHIFT 0xc
+#define CUR_CONTROL__CURSOR_2X_MAGNIFY_MASK 0x10000
+#define CUR_CONTROL__CURSOR_2X_MAGNIFY__SHIFT 0x10
+#define CUR_CONTROL__CURSOR_FORCE_MC_ON_MASK 0x100000
+#define CUR_CONTROL__CURSOR_FORCE_MC_ON__SHIFT 0x14
+#define CUR_CONTROL__CURSOR_URGENT_CONTROL_MASK 0x7000000
+#define CUR_CONTROL__CURSOR_URGENT_CONTROL__SHIFT 0x18
+#define CUR_SURFACE_ADDRESS__CURSOR_SURFACE_ADDRESS_MASK 0xffffffff
+#define CUR_SURFACE_ADDRESS__CURSOR_SURFACE_ADDRESS__SHIFT 0x0
+#define CUR_SIZE__CURSOR_HEIGHT_MASK 0x7f
+#define CUR_SIZE__CURSOR_HEIGHT__SHIFT 0x0
+#define CUR_SIZE__CURSOR_WIDTH_MASK 0x7f0000
+#define CUR_SIZE__CURSOR_WIDTH__SHIFT 0x10
+#define CUR_SURFACE_ADDRESS_HIGH__CURSOR_SURFACE_ADDRESS_HIGH_MASK 0xff
+#define CUR_SURFACE_ADDRESS_HIGH__CURSOR_SURFACE_ADDRESS_HIGH__SHIFT 0x0
+#define CUR_POSITION__CURSOR_Y_POSITION_MASK 0x3fff
+#define CUR_POSITION__CURSOR_Y_POSITION__SHIFT 0x0
+#define CUR_POSITION__CURSOR_X_POSITION_MASK 0x3fff0000
+#define CUR_POSITION__CURSOR_X_POSITION__SHIFT 0x10
+#define CUR_HOT_SPOT__CURSOR_HOT_SPOT_Y_MASK 0x7f
+#define CUR_HOT_SPOT__CURSOR_HOT_SPOT_Y__SHIFT 0x0
+#define CUR_HOT_SPOT__CURSOR_HOT_SPOT_X_MASK 0x7f0000
+#define CUR_HOT_SPOT__CURSOR_HOT_SPOT_X__SHIFT 0x10
+#define CUR_COLOR1__CUR_COLOR1_BLUE_MASK 0xff
+#define CUR_COLOR1__CUR_COLOR1_BLUE__SHIFT 0x0
+#define CUR_COLOR1__CUR_COLOR1_GREEN_MASK 0xff00
+#define CUR_COLOR1__CUR_COLOR1_GREEN__SHIFT 0x8
+#define CUR_COLOR1__CUR_COLOR1_RED_MASK 0xff0000
+#define CUR_COLOR1__CUR_COLOR1_RED__SHIFT 0x10
+#define CUR_COLOR2__CUR_COLOR2_BLUE_MASK 0xff
+#define CUR_COLOR2__CUR_COLOR2_BLUE__SHIFT 0x0
+#define CUR_COLOR2__CUR_COLOR2_GREEN_MASK 0xff00
+#define CUR_COLOR2__CUR_COLOR2_GREEN__SHIFT 0x8
+#define CUR_COLOR2__CUR_COLOR2_RED_MASK 0xff0000
+#define CUR_COLOR2__CUR_COLOR2_RED__SHIFT 0x10
+#define CUR_UPDATE__CURSOR_UPDATE_PENDING_MASK 0x1
+#define CUR_UPDATE__CURSOR_UPDATE_PENDING__SHIFT 0x0
+#define CUR_UPDATE__CURSOR_UPDATE_TAKEN_MASK 0x2
+#define CUR_UPDATE__CURSOR_UPDATE_TAKEN__SHIFT 0x1
+#define CUR_UPDATE__CURSOR_UPDATE_LOCK_MASK 0x10000
+#define CUR_UPDATE__CURSOR_UPDATE_LOCK__SHIFT 0x10
+#define CUR_UPDATE__CURSOR_DISABLE_MULTIPLE_UPDATE_MASK 0x1000000
+#define CUR_UPDATE__CURSOR_DISABLE_MULTIPLE_UPDATE__SHIFT 0x18
+#define CUR_UPDATE__CURSOR_UPDATE_STEREO_MODE_MASK 0x6000000
+#define CUR_UPDATE__CURSOR_UPDATE_STEREO_MODE__SHIFT 0x19
+#define CUR_REQUEST_FILTER_CNTL__CUR_REQUEST_FILTER_DIS_MASK 0x1
+#define CUR_REQUEST_FILTER_CNTL__CUR_REQUEST_FILTER_DIS__SHIFT 0x0
+#define CUR_STEREO_CONTROL__CURSOR_STEREO_EN_MASK 0x1
+#define CUR_STEREO_CONTROL__CURSOR_STEREO_EN__SHIFT 0x0
+#define CUR_STEREO_CONTROL__CURSOR_STEREO_OFFSET_YNX_MASK 0x2
+#define CUR_STEREO_CONTROL__CURSOR_STEREO_OFFSET_YNX__SHIFT 0x1
+#define CUR_STEREO_CONTROL__CURSOR_PRIMARY_OFFSET_MASK 0x3ff0
+#define CUR_STEREO_CONTROL__CURSOR_PRIMARY_OFFSET__SHIFT 0x4
+#define CUR_STEREO_CONTROL__CURSOR_SECONDARY_OFFSET_MASK 0x3ff0000
+#define CUR_STEREO_CONTROL__CURSOR_SECONDARY_OFFSET__SHIFT 0x10
+#define DC_LUT_RW_MODE__DC_LUT_RW_MODE_MASK 0x1
+#define DC_LUT_RW_MODE__DC_LUT_RW_MODE__SHIFT 0x0
+#define DC_LUT_RW_MODE__DC_LUT_ERROR_MASK 0x10000
+#define DC_LUT_RW_MODE__DC_LUT_ERROR__SHIFT 0x10
+#define DC_LUT_RW_MODE__DC_LUT_ERROR_RST_MASK 0x20000
+#define DC_LUT_RW_MODE__DC_LUT_ERROR_RST__SHIFT 0x11
+#define DC_LUT_RW_INDEX__DC_LUT_RW_INDEX_MASK 0xff
+#define DC_LUT_RW_INDEX__DC_LUT_RW_INDEX__SHIFT 0x0
+#define DC_LUT_SEQ_COLOR__DC_LUT_SEQ_COLOR_MASK 0xffff
+#define DC_LUT_SEQ_COLOR__DC_LUT_SEQ_COLOR__SHIFT 0x0
+#define DC_LUT_PWL_DATA__DC_LUT_BASE_MASK 0xffff
+#define DC_LUT_PWL_DATA__DC_LUT_BASE__SHIFT 0x0
+#define DC_LUT_PWL_DATA__DC_LUT_DELTA_MASK 0xffff0000
+#define DC_LUT_PWL_DATA__DC_LUT_DELTA__SHIFT 0x10
+#define DC_LUT_30_COLOR__DC_LUT_COLOR_10_BLUE_MASK 0x3ff
+#define DC_LUT_30_COLOR__DC_LUT_COLOR_10_BLUE__SHIFT 0x0
+#define DC_LUT_30_COLOR__DC_LUT_COLOR_10_GREEN_MASK 0xffc00
+#define DC_LUT_30_COLOR__DC_LUT_COLOR_10_GREEN__SHIFT 0xa
+#define DC_LUT_30_COLOR__DC_LUT_COLOR_10_RED_MASK 0x3ff00000
+#define DC_LUT_30_COLOR__DC_LUT_COLOR_10_RED__SHIFT 0x14
+#define DC_LUT_VGA_ACCESS_ENABLE__DC_LUT_VGA_ACCESS_ENABLE_MASK 0x1
+#define DC_LUT_VGA_ACCESS_ENABLE__DC_LUT_VGA_ACCESS_ENABLE__SHIFT 0x0
+#define DC_LUT_WRITE_EN_MASK__DC_LUT_WRITE_EN_MASK_MASK 0x7
+#define DC_LUT_WRITE_EN_MASK__DC_LUT_WRITE_EN_MASK__SHIFT 0x0
+#define DC_LUT_AUTOFILL__DC_LUT_AUTOFILL_MASK 0x1
+#define DC_LUT_AUTOFILL__DC_LUT_AUTOFILL__SHIFT 0x0
+#define DC_LUT_AUTOFILL__DC_LUT_AUTOFILL_DONE_MASK 0x2
+#define DC_LUT_AUTOFILL__DC_LUT_AUTOFILL_DONE__SHIFT 0x1
+#define DC_LUT_CONTROL__DC_LUT_INC_B_MASK 0xf
+#define DC_LUT_CONTROL__DC_LUT_INC_B__SHIFT 0x0
+#define DC_LUT_CONTROL__DC_LUT_DATA_B_SIGNED_EN_MASK 0x10
+#define DC_LUT_CONTROL__DC_LUT_DATA_B_SIGNED_EN__SHIFT 0x4
+#define DC_LUT_CONTROL__DC_LUT_DATA_B_FLOAT_POINT_EN_MASK 0x20
+#define DC_LUT_CONTROL__DC_LUT_DATA_B_FLOAT_POINT_EN__SHIFT 0x5
+#define DC_LUT_CONTROL__DC_LUT_DATA_B_FORMAT_MASK 0xc0
+#define DC_LUT_CONTROL__DC_LUT_DATA_B_FORMAT__SHIFT 0x6
+#define DC_LUT_CONTROL__DC_LUT_INC_G_MASK 0xf00
+#define DC_LUT_CONTROL__DC_LUT_INC_G__SHIFT 0x8
+#define DC_LUT_CONTROL__DC_LUT_DATA_G_SIGNED_EN_MASK 0x1000
+#define DC_LUT_CONTROL__DC_LUT_DATA_G_SIGNED_EN__SHIFT 0xc
+#define DC_LUT_CONTROL__DC_LUT_DATA_G_FLOAT_POINT_EN_MASK 0x2000
+#define DC_LUT_CONTROL__DC_LUT_DATA_G_FLOAT_POINT_EN__SHIFT 0xd
+#define DC_LUT_CONTROL__DC_LUT_DATA_G_FORMAT_MASK 0xc000
+#define DC_LUT_CONTROL__DC_LUT_DATA_G_FORMAT__SHIFT 0xe
+#define DC_LUT_CONTROL__DC_LUT_INC_R_MASK 0xf0000
+#define DC_LUT_CONTROL__DC_LUT_INC_R__SHIFT 0x10
+#define DC_LUT_CONTROL__DC_LUT_DATA_R_SIGNED_EN_MASK 0x100000
+#define DC_LUT_CONTROL__DC_LUT_DATA_R_SIGNED_EN__SHIFT 0x14
+#define DC_LUT_CONTROL__DC_LUT_DATA_R_FLOAT_POINT_EN_MASK 0x200000
+#define DC_LUT_CONTROL__DC_LUT_DATA_R_FLOAT_POINT_EN__SHIFT 0x15
+#define DC_LUT_CONTROL__DC_LUT_DATA_R_FORMAT_MASK 0xc00000
+#define DC_LUT_CONTROL__DC_LUT_DATA_R_FORMAT__SHIFT 0x16
+#define DC_LUT_BLACK_OFFSET_BLUE__DC_LUT_BLACK_OFFSET_BLUE_MASK 0xffff
+#define DC_LUT_BLACK_OFFSET_BLUE__DC_LUT_BLACK_OFFSET_BLUE__SHIFT 0x0
+#define DC_LUT_BLACK_OFFSET_GREEN__DC_LUT_BLACK_OFFSET_GREEN_MASK 0xffff
+#define DC_LUT_BLACK_OFFSET_GREEN__DC_LUT_BLACK_OFFSET_GREEN__SHIFT 0x0
+#define DC_LUT_BLACK_OFFSET_RED__DC_LUT_BLACK_OFFSET_RED_MASK 0xffff
+#define DC_LUT_BLACK_OFFSET_RED__DC_LUT_BLACK_OFFSET_RED__SHIFT 0x0
+#define DC_LUT_WHITE_OFFSET_BLUE__DC_LUT_WHITE_OFFSET_BLUE_MASK 0xffff
+#define DC_LUT_WHITE_OFFSET_BLUE__DC_LUT_WHITE_OFFSET_BLUE__SHIFT 0x0
+#define DC_LUT_WHITE_OFFSET_GREEN__DC_LUT_WHITE_OFFSET_GREEN_MASK 0xffff
+#define DC_LUT_WHITE_OFFSET_GREEN__DC_LUT_WHITE_OFFSET_GREEN__SHIFT 0x0
+#define DC_LUT_WHITE_OFFSET_RED__DC_LUT_WHITE_OFFSET_RED_MASK 0xffff
+#define DC_LUT_WHITE_OFFSET_RED__DC_LUT_WHITE_OFFSET_RED__SHIFT 0x0
+#define DCP_CRC_CONTROL__DCP_CRC_ENABLE_MASK 0x1
+#define DCP_CRC_CONTROL__DCP_CRC_ENABLE__SHIFT 0x0
+#define DCP_CRC_CONTROL__DCP_CRC_SOURCE_SEL_MASK 0x1c
+#define DCP_CRC_CONTROL__DCP_CRC_SOURCE_SEL__SHIFT 0x2
+#define DCP_CRC_CONTROL__DCP_CRC_LINE_SEL_MASK 0x300
+#define DCP_CRC_CONTROL__DCP_CRC_LINE_SEL__SHIFT 0x8
+#define DCP_CRC_MASK__DCP_CRC_MASK_MASK 0xffffffff
+#define DCP_CRC_MASK__DCP_CRC_MASK__SHIFT 0x0
+#define DCP_CRC_CURRENT__DCP_CRC_CURRENT_MASK 0xffffffff
+#define DCP_CRC_CURRENT__DCP_CRC_CURRENT__SHIFT 0x0
+#define DVMM_PTE_CONTROL__DVMM_USE_SINGLE_PTE_MASK 0x1
+#define DVMM_PTE_CONTROL__DVMM_USE_SINGLE_PTE__SHIFT 0x0
+#define DVMM_PTE_CONTROL__DVMM_PAGE_WIDTH_MASK 0x1e
+#define DVMM_PTE_CONTROL__DVMM_PAGE_WIDTH__SHIFT 0x1
+#define DVMM_PTE_CONTROL__DVMM_PAGE_HEIGHT_MASK 0x1e0
+#define DVMM_PTE_CONTROL__DVMM_PAGE_HEIGHT__SHIFT 0x5
+#define DVMM_PTE_CONTROL__DVMM_MIN_PTE_BEFORE_FLIP_MASK 0x7fe00
+#define DVMM_PTE_CONTROL__DVMM_MIN_PTE_BEFORE_FLIP__SHIFT 0x9
+#define DVMM_PTE_CONTROL__DVMM_PTE_BUFFER_MODE0_MASK 0x100000
+#define DVMM_PTE_CONTROL__DVMM_PTE_BUFFER_MODE0__SHIFT 0x14
+#define DVMM_PTE_CONTROL__DVMM_PTE_BUFFER_MODE1_MASK 0x200000
+#define DVMM_PTE_CONTROL__DVMM_PTE_BUFFER_MODE1__SHIFT 0x15
+#define DCP_CRC_LAST__DCP_CRC_LAST_MASK 0xffffffff
+#define DCP_CRC_LAST__DCP_CRC_LAST__SHIFT 0x0
+#define DCP_DEBUG__DCP_DEBUG_MASK 0xffffffff
+#define DCP_DEBUG__DCP_DEBUG__SHIFT 0x0
+#define GRPH_FLIP_RATE_CNTL__GRPH_FLIP_RATE_MASK 0x7
+#define GRPH_FLIP_RATE_CNTL__GRPH_FLIP_RATE__SHIFT 0x0
+#define GRPH_FLIP_RATE_CNTL__GRPH_FLIP_RATE_ENABLE_MASK 0x8
+#define GRPH_FLIP_RATE_CNTL__GRPH_FLIP_RATE_ENABLE__SHIFT 0x3
+#define DCP_GSL_CONTROL__DCP_GSL0_EN_MASK 0x1
+#define DCP_GSL_CONTROL__DCP_GSL0_EN__SHIFT 0x0
+#define DCP_GSL_CONTROL__DCP_GSL1_EN_MASK 0x2
+#define DCP_GSL_CONTROL__DCP_GSL1_EN__SHIFT 0x1
+#define DCP_GSL_CONTROL__DCP_GSL2_EN_MASK 0x4
+#define DCP_GSL_CONTROL__DCP_GSL2_EN__SHIFT 0x2
+#define DCP_GSL_CONTROL__DCP_GSL_HSYNC_FLIP_FORCE_DELAY_MASK 0xf000
+#define DCP_GSL_CONTROL__DCP_GSL_HSYNC_FLIP_FORCE_DELAY__SHIFT 0xc
+#define DCP_GSL_CONTROL__DCP_GSL_MASTER_EN_MASK 0x10000
+#define DCP_GSL_CONTROL__DCP_GSL_MASTER_EN__SHIFT 0x10
+#define DCP_GSL_CONTROL__DCP_GSL_XDMA_GROUP_MASK 0x60000
+#define DCP_GSL_CONTROL__DCP_GSL_XDMA_GROUP__SHIFT 0x11
+#define DCP_GSL_CONTROL__DCP_GSL_XDMA_GROUP_UNDERFLOW_EN_MASK 0x80000
+#define DCP_GSL_CONTROL__DCP_GSL_XDMA_GROUP_UNDERFLOW_EN__SHIFT 0x13
+#define DCP_GSL_CONTROL__DCP_GSL_SYNC_SOURCE_MASK 0x3000000
+#define DCP_GSL_CONTROL__DCP_GSL_SYNC_SOURCE__SHIFT 0x18
+#define DCP_GSL_CONTROL__DCP_GSL_DELAY_SURFACE_UPDATE_PENDING_MASK 0x8000000
+#define DCP_GSL_CONTROL__DCP_GSL_DELAY_SURFACE_UPDATE_PENDING__SHIFT 0x1b
+#define DCP_GSL_CONTROL__DCP_GSL_HSYNC_FLIP_CHECK_DELAY_MASK 0xf0000000
+#define DCP_GSL_CONTROL__DCP_GSL_HSYNC_FLIP_CHECK_DELAY__SHIFT 0x1c
+#define DCP_LB_DATA_GAP_BETWEEN_CHUNK__DCP_LB_GAP_BETWEEN_CHUNK_20BPP_MASK 0xf
+#define DCP_LB_DATA_GAP_BETWEEN_CHUNK__DCP_LB_GAP_BETWEEN_CHUNK_20BPP__SHIFT 0x0
+#define DCP_LB_DATA_GAP_BETWEEN_CHUNK__DCP_LB_GAP_BETWEEN_CHUNK_30BPP_MASK 0x1f0
+#define DCP_LB_DATA_GAP_BETWEEN_CHUNK__DCP_LB_GAP_BETWEEN_CHUNK_30BPP__SHIFT 0x4
+#define DCP_DEBUG_SG__DCP_DEBUG_SG_MASK 0xffffffff
+#define DCP_DEBUG_SG__DCP_DEBUG_SG__SHIFT 0x0
+#define DCP_DEBUG_SG2__DCP_DEBUG_SG2_MASK 0xffffffff
+#define DCP_DEBUG_SG2__DCP_DEBUG_SG2__SHIFT 0x0
+#define DCP_DVMM_DEBUG__DCP_DVMM_DEBUG_MASK 0xffffffff
+#define DCP_DVMM_DEBUG__DCP_DVMM_DEBUG__SHIFT 0x0
+#define DCP_TEST_DEBUG_INDEX__DCP_TEST_DEBUG_INDEX_MASK 0xff
+#define DCP_TEST_DEBUG_INDEX__DCP_TEST_DEBUG_INDEX__SHIFT 0x0
+#define DCP_TEST_DEBUG_INDEX__DCP_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define DCP_TEST_DEBUG_INDEX__DCP_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define DCP_TEST_DEBUG_DATA__DCP_TEST_DEBUG_DATA_MASK 0xffffffff
+#define DCP_TEST_DEBUG_DATA__DCP_TEST_DEBUG_DATA__SHIFT 0x0
+#define GRPH_STEREOSYNC_FLIP__GRPH_STEREOSYNC_FLIP_EN_MASK 0x1
+#define GRPH_STEREOSYNC_FLIP__GRPH_STEREOSYNC_FLIP_EN__SHIFT 0x0
+#define GRPH_STEREOSYNC_FLIP__GRPH_STEREOSYNC_FLIP_MODE_MASK 0x300
+#define GRPH_STEREOSYNC_FLIP__GRPH_STEREOSYNC_FLIP_MODE__SHIFT 0x8
+#define GRPH_STEREOSYNC_FLIP__GRPH_PRIMARY_SURFACE_PENDING_MASK 0x10000
+#define GRPH_STEREOSYNC_FLIP__GRPH_PRIMARY_SURFACE_PENDING__SHIFT 0x10
+#define GRPH_STEREOSYNC_FLIP__GRPH_SECONDARY_SURFACE_PENDING_MASK 0x20000
+#define GRPH_STEREOSYNC_FLIP__GRPH_SECONDARY_SURFACE_PENDING__SHIFT 0x11
+#define GRPH_STEREOSYNC_FLIP__GRPH_STEREOSYNC_SELECT_DISABLE_MASK 0x10000000
+#define GRPH_STEREOSYNC_FLIP__GRPH_STEREOSYNC_SELECT_DISABLE__SHIFT 0x1c
+#define DCP_DEBUG2__DCP_DEBUG2_MASK 0xffffffff
+#define DCP_DEBUG2__DCP_DEBUG2__SHIFT 0x0
+#define HW_ROTATION__GRPH_ROTATION_ANGLE_MASK 0x7
+#define HW_ROTATION__GRPH_ROTATION_ANGLE__SHIFT 0x0
+#define GRPH_XDMA_CACHE_UNDERFLOW_DET_CNTL__GRPH_XDMA_CACHE_UNDERFLOW_CNT_EN_MASK 0x1
+#define GRPH_XDMA_CACHE_UNDERFLOW_DET_CNTL__GRPH_XDMA_CACHE_UNDERFLOW_CNT_EN__SHIFT 0x0
+#define GRPH_XDMA_CACHE_UNDERFLOW_DET_CNTL__GRPH_XDMA_CACHE_UNDERFLOW_CNT_MODE_MASK 0x2
+#define GRPH_XDMA_CACHE_UNDERFLOW_DET_CNTL__GRPH_XDMA_CACHE_UNDERFLOW_CNT_MODE__SHIFT 0x1
+#define GRPH_XDMA_CACHE_UNDERFLOW_DET_CNTL__GRPH_XDMA_CACHE_UNDERFLOW_FRAME_CNT_MASK 0x1fff0
+#define GRPH_XDMA_CACHE_UNDERFLOW_DET_CNTL__GRPH_XDMA_CACHE_UNDERFLOW_FRAME_CNT__SHIFT 0x4
+#define REGAMMA_CONTROL__GRPH_REGAMMA_MODE_MASK 0x7
+#define REGAMMA_CONTROL__GRPH_REGAMMA_MODE__SHIFT 0x0
+#define REGAMMA_LUT_INDEX__REGAMMA_LUT_INDEX_MASK 0x1ff
+#define REGAMMA_LUT_INDEX__REGAMMA_LUT_INDEX__SHIFT 0x0
+#define REGAMMA_LUT_DATA__REGAMMA_LUT_DATA_MASK 0x7ffff
+#define REGAMMA_LUT_DATA__REGAMMA_LUT_DATA__SHIFT 0x0
+#define REGAMMA_LUT_WRITE_EN_MASK__REGAMMA_LUT_WRITE_EN_MASK_MASK 0x7
+#define REGAMMA_LUT_WRITE_EN_MASK__REGAMMA_LUT_WRITE_EN_MASK__SHIFT 0x0
+#define REGAMMA_CNTLA_START_CNTL__REGAMMA_CNTLA_EXP_REGION_START_MASK 0x3ffff
+#define REGAMMA_CNTLA_START_CNTL__REGAMMA_CNTLA_EXP_REGION_START__SHIFT 0x0
+#define REGAMMA_CNTLA_START_CNTL__REGAMMA_CNTLA_EXP_REGION_START_SEGMENT_MASK 0x7f00000
+#define REGAMMA_CNTLA_START_CNTL__REGAMMA_CNTLA_EXP_REGION_START_SEGMENT__SHIFT 0x14
+#define REGAMMA_CNTLA_SLOPE_CNTL__REGAMMA_CNTLA_EXP_REGION_LINEAR_SLOPE_MASK 0x3ffff
+#define REGAMMA_CNTLA_SLOPE_CNTL__REGAMMA_CNTLA_EXP_REGION_LINEAR_SLOPE__SHIFT 0x0
+#define REGAMMA_CNTLA_END_CNTL1__REGAMMA_CNTLA_EXP_REGION_END_MASK 0xffff
+#define REGAMMA_CNTLA_END_CNTL1__REGAMMA_CNTLA_EXP_REGION_END__SHIFT 0x0
+#define REGAMMA_CNTLA_END_CNTL2__REGAMMA_CNTLA_EXP_REGION_END_SLOPE_MASK 0xffff
+#define REGAMMA_CNTLA_END_CNTL2__REGAMMA_CNTLA_EXP_REGION_END_SLOPE__SHIFT 0x0
+#define REGAMMA_CNTLA_END_CNTL2__REGAMMA_CNTLA_EXP_REGION_END_BASE_MASK 0xffff0000
+#define REGAMMA_CNTLA_END_CNTL2__REGAMMA_CNTLA_EXP_REGION_END_BASE__SHIFT 0x10
+#define REGAMMA_CNTLA_REGION_0_1__REGAMMA_CNTLA_EXP_REGION0_LUT_OFFSET_MASK 0x1ff
+#define REGAMMA_CNTLA_REGION_0_1__REGAMMA_CNTLA_EXP_REGION0_LUT_OFFSET__SHIFT 0x0
+#define REGAMMA_CNTLA_REGION_0_1__REGAMMA_CNTLA_EXP_REGION0_NUM_SEGMENTS_MASK 0x7000
+#define REGAMMA_CNTLA_REGION_0_1__REGAMMA_CNTLA_EXP_REGION0_NUM_SEGMENTS__SHIFT 0xc
+#define REGAMMA_CNTLA_REGION_0_1__REGAMMA_CNTLA_EXP_REGION1_LUT_OFFSET_MASK 0x1ff0000
+#define REGAMMA_CNTLA_REGION_0_1__REGAMMA_CNTLA_EXP_REGION1_LUT_OFFSET__SHIFT 0x10
+#define REGAMMA_CNTLA_REGION_0_1__REGAMMA_CNTLA_EXP_REGION1_NUM_SEGMENTS_MASK 0x70000000
+#define REGAMMA_CNTLA_REGION_0_1__REGAMMA_CNTLA_EXP_REGION1_NUM_SEGMENTS__SHIFT 0x1c
+#define REGAMMA_CNTLA_REGION_2_3__REGAMMA_CNTLA_EXP_REGION2_LUT_OFFSET_MASK 0x1ff
+#define REGAMMA_CNTLA_REGION_2_3__REGAMMA_CNTLA_EXP_REGION2_LUT_OFFSET__SHIFT 0x0
+#define REGAMMA_CNTLA_REGION_2_3__REGAMMA_CNTLA_EXP_REGION2_NUM_SEGMENTS_MASK 0x7000
+#define REGAMMA_CNTLA_REGION_2_3__REGAMMA_CNTLA_EXP_REGION2_NUM_SEGMENTS__SHIFT 0xc
+#define REGAMMA_CNTLA_REGION_2_3__REGAMMA_CNTLA_EXP_REGION3_LUT_OFFSET_MASK 0x1ff0000
+#define REGAMMA_CNTLA_REGION_2_3__REGAMMA_CNTLA_EXP_REGION3_LUT_OFFSET__SHIFT 0x10
+#define REGAMMA_CNTLA_REGION_2_3__REGAMMA_CNTLA_EXP_REGION3_NUM_SEGMENTS_MASK 0x70000000
+#define REGAMMA_CNTLA_REGION_2_3__REGAMMA_CNTLA_EXP_REGION3_NUM_SEGMENTS__SHIFT 0x1c
+#define REGAMMA_CNTLA_REGION_4_5__REGAMMA_CNTLA_EXP_REGION4_LUT_OFFSET_MASK 0x1ff
+#define REGAMMA_CNTLA_REGION_4_5__REGAMMA_CNTLA_EXP_REGION4_LUT_OFFSET__SHIFT 0x0
+#define REGAMMA_CNTLA_REGION_4_5__REGAMMA_CNTLA_EXP_REGION4_NUM_SEGMENTS_MASK 0x7000
+#define REGAMMA_CNTLA_REGION_4_5__REGAMMA_CNTLA_EXP_REGION4_NUM_SEGMENTS__SHIFT 0xc
+#define REGAMMA_CNTLA_REGION_4_5__REGAMMA_CNTLA_EXP_REGION5_LUT_OFFSET_MASK 0x1ff0000
+#define REGAMMA_CNTLA_REGION_4_5__REGAMMA_CNTLA_EXP_REGION5_LUT_OFFSET__SHIFT 0x10
+#define REGAMMA_CNTLA_REGION_4_5__REGAMMA_CNTLA_EXP_REGION5_NUM_SEGMENTS_MASK 0x70000000
+#define REGAMMA_CNTLA_REGION_4_5__REGAMMA_CNTLA_EXP_REGION5_NUM_SEGMENTS__SHIFT 0x1c
+#define REGAMMA_CNTLA_REGION_6_7__REGAMMA_CNTLA_EXP_REGION6_LUT_OFFSET_MASK 0x1ff
+#define REGAMMA_CNTLA_REGION_6_7__REGAMMA_CNTLA_EXP_REGION6_LUT_OFFSET__SHIFT 0x0
+#define REGAMMA_CNTLA_REGION_6_7__REGAMMA_CNTLA_EXP_REGION6_NUM_SEGMENTS_MASK 0x7000
+#define REGAMMA_CNTLA_REGION_6_7__REGAMMA_CNTLA_EXP_REGION6_NUM_SEGMENTS__SHIFT 0xc
+#define REGAMMA_CNTLA_REGION_6_7__REGAMMA_CNTLA_EXP_REGION7_LUT_OFFSET_MASK 0x1ff0000
+#define REGAMMA_CNTLA_REGION_6_7__REGAMMA_CNTLA_EXP_REGION7_LUT_OFFSET__SHIFT 0x10
+#define REGAMMA_CNTLA_REGION_6_7__REGAMMA_CNTLA_EXP_REGION7_NUM_SEGMENTS_MASK 0x70000000
+#define REGAMMA_CNTLA_REGION_6_7__REGAMMA_CNTLA_EXP_REGION7_NUM_SEGMENTS__SHIFT 0x1c
+#define REGAMMA_CNTLA_REGION_8_9__REGAMMA_CNTLA_EXP_REGION8_LUT_OFFSET_MASK 0x1ff
+#define REGAMMA_CNTLA_REGION_8_9__REGAMMA_CNTLA_EXP_REGION8_LUT_OFFSET__SHIFT 0x0
+#define REGAMMA_CNTLA_REGION_8_9__REGAMMA_CNTLA_EXP_REGION8_NUM_SEGMENTS_MASK 0x7000
+#define REGAMMA_CNTLA_REGION_8_9__REGAMMA_CNTLA_EXP_REGION8_NUM_SEGMENTS__SHIFT 0xc
+#define REGAMMA_CNTLA_REGION_8_9__REGAMMA_CNTLA_EXP_REGION9_LUT_OFFSET_MASK 0x1ff0000
+#define REGAMMA_CNTLA_REGION_8_9__REGAMMA_CNTLA_EXP_REGION9_LUT_OFFSET__SHIFT 0x10
+#define REGAMMA_CNTLA_REGION_8_9__REGAMMA_CNTLA_EXP_REGION9_NUM_SEGMENTS_MASK 0x70000000
+#define REGAMMA_CNTLA_REGION_8_9__REGAMMA_CNTLA_EXP_REGION9_NUM_SEGMENTS__SHIFT 0x1c
+#define REGAMMA_CNTLA_REGION_10_11__REGAMMA_CNTLA_EXP_REGION10_LUT_OFFSET_MASK 0x1ff
+#define REGAMMA_CNTLA_REGION_10_11__REGAMMA_CNTLA_EXP_REGION10_LUT_OFFSET__SHIFT 0x0
+#define REGAMMA_CNTLA_REGION_10_11__REGAMMA_CNTLA_EXP_REGION10_NUM_SEGMENTS_MASK 0x7000
+#define REGAMMA_CNTLA_REGION_10_11__REGAMMA_CNTLA_EXP_REGION10_NUM_SEGMENTS__SHIFT 0xc
+#define REGAMMA_CNTLA_REGION_10_11__REGAMMA_CNTLA_EXP_REGION11_LUT_OFFSET_MASK 0x1ff0000
+#define REGAMMA_CNTLA_REGION_10_11__REGAMMA_CNTLA_EXP_REGION11_LUT_OFFSET__SHIFT 0x10
+#define REGAMMA_CNTLA_REGION_10_11__REGAMMA_CNTLA_EXP_REGION11_NUM_SEGMENTS_MASK 0x70000000
+#define REGAMMA_CNTLA_REGION_10_11__REGAMMA_CNTLA_EXP_REGION11_NUM_SEGMENTS__SHIFT 0x1c
+#define REGAMMA_CNTLA_REGION_12_13__REGAMMA_CNTLA_EXP_REGION12_LUT_OFFSET_MASK 0x1ff
+#define REGAMMA_CNTLA_REGION_12_13__REGAMMA_CNTLA_EXP_REGION12_LUT_OFFSET__SHIFT 0x0
+#define REGAMMA_CNTLA_REGION_12_13__REGAMMA_CNTLA_EXP_REGION12_NUM_SEGMENTS_MASK 0x7000
+#define REGAMMA_CNTLA_REGION_12_13__REGAMMA_CNTLA_EXP_REGION12_NUM_SEGMENTS__SHIFT 0xc
+#define REGAMMA_CNTLA_REGION_12_13__REGAMMA_CNTLA_EXP_REGION13_LUT_OFFSET_MASK 0x1ff0000
+#define REGAMMA_CNTLA_REGION_12_13__REGAMMA_CNTLA_EXP_REGION13_LUT_OFFSET__SHIFT 0x10
+#define REGAMMA_CNTLA_REGION_12_13__REGAMMA_CNTLA_EXP_REGION13_NUM_SEGMENTS_MASK 0x70000000
+#define REGAMMA_CNTLA_REGION_12_13__REGAMMA_CNTLA_EXP_REGION13_NUM_SEGMENTS__SHIFT 0x1c
+#define REGAMMA_CNTLA_REGION_14_15__REGAMMA_CNTLA_EXP_REGION14_LUT_OFFSET_MASK 0x1ff
+#define REGAMMA_CNTLA_REGION_14_15__REGAMMA_CNTLA_EXP_REGION14_LUT_OFFSET__SHIFT 0x0
+#define REGAMMA_CNTLA_REGION_14_15__REGAMMA_CNTLA_EXP_REGION14_NUM_SEGMENTS_MASK 0x7000
+#define REGAMMA_CNTLA_REGION_14_15__REGAMMA_CNTLA_EXP_REGION14_NUM_SEGMENTS__SHIFT 0xc
+#define REGAMMA_CNTLA_REGION_14_15__REGAMMA_CNTLA_EXP_REGION15_LUT_OFFSET_MASK 0x1ff0000
+#define REGAMMA_CNTLA_REGION_14_15__REGAMMA_CNTLA_EXP_REGION15_LUT_OFFSET__SHIFT 0x10
+#define REGAMMA_CNTLA_REGION_14_15__REGAMMA_CNTLA_EXP_REGION15_NUM_SEGMENTS_MASK 0x70000000
+#define REGAMMA_CNTLA_REGION_14_15__REGAMMA_CNTLA_EXP_REGION15_NUM_SEGMENTS__SHIFT 0x1c
+#define REGAMMA_CNTLB_START_CNTL__REGAMMA_CNTLB_EXP_REGION_START_MASK 0x3ffff
+#define REGAMMA_CNTLB_START_CNTL__REGAMMA_CNTLB_EXP_REGION_START__SHIFT 0x0
+#define REGAMMA_CNTLB_START_CNTL__REGAMMA_CNTLB_EXP_REGION_START_SEGMENT_MASK 0x7f00000
+#define REGAMMA_CNTLB_START_CNTL__REGAMMA_CNTLB_EXP_REGION_START_SEGMENT__SHIFT 0x14
+#define REGAMMA_CNTLB_SLOPE_CNTL__REGAMMA_CNTLB_EXP_REGION_LINEAR_SLOPE_MASK 0x3ffff
+#define REGAMMA_CNTLB_SLOPE_CNTL__REGAMMA_CNTLB_EXP_REGION_LINEAR_SLOPE__SHIFT 0x0
+#define REGAMMA_CNTLB_END_CNTL1__REGAMMA_CNTLB_EXP_REGION_END_MASK 0xffff
+#define REGAMMA_CNTLB_END_CNTL1__REGAMMA_CNTLB_EXP_REGION_END__SHIFT 0x0
+#define REGAMMA_CNTLB_END_CNTL2__REGAMMA_CNTLB_EXP_REGION_END_SLOPE_MASK 0xffff
+#define REGAMMA_CNTLB_END_CNTL2__REGAMMA_CNTLB_EXP_REGION_END_SLOPE__SHIFT 0x0
+#define REGAMMA_CNTLB_END_CNTL2__REGAMMA_CNTLB_EXP_REGION_END_BASE_MASK 0xffff0000
+#define REGAMMA_CNTLB_END_CNTL2__REGAMMA_CNTLB_EXP_REGION_END_BASE__SHIFT 0x10
+#define REGAMMA_CNTLB_REGION_0_1__REGAMMA_CNTLB_EXP_REGION0_LUT_OFFSET_MASK 0x1ff
+#define REGAMMA_CNTLB_REGION_0_1__REGAMMA_CNTLB_EXP_REGION0_LUT_OFFSET__SHIFT 0x0
+#define REGAMMA_CNTLB_REGION_0_1__REGAMMA_CNTLB_EXP_REGION0_NUM_SEGMENTS_MASK 0x7000
+#define REGAMMA_CNTLB_REGION_0_1__REGAMMA_CNTLB_EXP_REGION0_NUM_SEGMENTS__SHIFT 0xc
+#define REGAMMA_CNTLB_REGION_0_1__REGAMMA_CNTLB_EXP_REGION1_LUT_OFFSET_MASK 0x1ff0000
+#define REGAMMA_CNTLB_REGION_0_1__REGAMMA_CNTLB_EXP_REGION1_LUT_OFFSET__SHIFT 0x10
+#define REGAMMA_CNTLB_REGION_0_1__REGAMMA_CNTLB_EXP_REGION1_NUM_SEGMENTS_MASK 0x70000000
+#define REGAMMA_CNTLB_REGION_0_1__REGAMMA_CNTLB_EXP_REGION1_NUM_SEGMENTS__SHIFT 0x1c
+#define REGAMMA_CNTLB_REGION_2_3__REGAMMA_CNTLB_EXP_REGION2_LUT_OFFSET_MASK 0x1ff
+#define REGAMMA_CNTLB_REGION_2_3__REGAMMA_CNTLB_EXP_REGION2_LUT_OFFSET__SHIFT 0x0
+#define REGAMMA_CNTLB_REGION_2_3__REGAMMA_CNTLB_EXP_REGION2_NUM_SEGMENTS_MASK 0x7000
+#define REGAMMA_CNTLB_REGION_2_3__REGAMMA_CNTLB_EXP_REGION2_NUM_SEGMENTS__SHIFT 0xc
+#define REGAMMA_CNTLB_REGION_2_3__REGAMMA_CNTLB_EXP_REGION3_LUT_OFFSET_MASK 0x1ff0000
+#define REGAMMA_CNTLB_REGION_2_3__REGAMMA_CNTLB_EXP_REGION3_LUT_OFFSET__SHIFT 0x10
+#define REGAMMA_CNTLB_REGION_2_3__REGAMMA_CNTLB_EXP_REGION3_NUM_SEGMENTS_MASK 0x70000000
+#define REGAMMA_CNTLB_REGION_2_3__REGAMMA_CNTLB_EXP_REGION3_NUM_SEGMENTS__SHIFT 0x1c
+#define REGAMMA_CNTLB_REGION_4_5__REGAMMA_CNTLB_EXP_REGION4_LUT_OFFSET_MASK 0x1ff
+#define REGAMMA_CNTLB_REGION_4_5__REGAMMA_CNTLB_EXP_REGION4_LUT_OFFSET__SHIFT 0x0
+#define REGAMMA_CNTLB_REGION_4_5__REGAMMA_CNTLB_EXP_REGION4_NUM_SEGMENTS_MASK 0x7000
+#define REGAMMA_CNTLB_REGION_4_5__REGAMMA_CNTLB_EXP_REGION4_NUM_SEGMENTS__SHIFT 0xc
+#define REGAMMA_CNTLB_REGION_4_5__REGAMMA_CNTLB_EXP_REGION5_LUT_OFFSET_MASK 0x1ff0000
+#define REGAMMA_CNTLB_REGION_4_5__REGAMMA_CNTLB_EXP_REGION5_LUT_OFFSET__SHIFT 0x10
+#define REGAMMA_CNTLB_REGION_4_5__REGAMMA_CNTLB_EXP_REGION5_NUM_SEGMENTS_MASK 0x70000000
+#define REGAMMA_CNTLB_REGION_4_5__REGAMMA_CNTLB_EXP_REGION5_NUM_SEGMENTS__SHIFT 0x1c
+#define REGAMMA_CNTLB_REGION_6_7__REGAMMA_CNTLB_EXP_REGION6_LUT_OFFSET_MASK 0x1ff
+#define REGAMMA_CNTLB_REGION_6_7__REGAMMA_CNTLB_EXP_REGION6_LUT_OFFSET__SHIFT 0x0
+#define REGAMMA_CNTLB_REGION_6_7__REGAMMA_CNTLB_EXP_REGION6_NUM_SEGMENTS_MASK 0x7000
+#define REGAMMA_CNTLB_REGION_6_7__REGAMMA_CNTLB_EXP_REGION6_NUM_SEGMENTS__SHIFT 0xc
+#define REGAMMA_CNTLB_REGION_6_7__REGAMMA_CNTLB_EXP_REGION7_LUT_OFFSET_MASK 0x1ff0000
+#define REGAMMA_CNTLB_REGION_6_7__REGAMMA_CNTLB_EXP_REGION7_LUT_OFFSET__SHIFT 0x10
+#define REGAMMA_CNTLB_REGION_6_7__REGAMMA_CNTLB_EXP_REGION7_NUM_SEGMENTS_MASK 0x70000000
+#define REGAMMA_CNTLB_REGION_6_7__REGAMMA_CNTLB_EXP_REGION7_NUM_SEGMENTS__SHIFT 0x1c
+#define REGAMMA_CNTLB_REGION_8_9__REGAMMA_CNTLB_EXP_REGION8_LUT_OFFSET_MASK 0x1ff
+#define REGAMMA_CNTLB_REGION_8_9__REGAMMA_CNTLB_EXP_REGION8_LUT_OFFSET__SHIFT 0x0
+#define REGAMMA_CNTLB_REGION_8_9__REGAMMA_CNTLB_EXP_REGION8_NUM_SEGMENTS_MASK 0x7000
+#define REGAMMA_CNTLB_REGION_8_9__REGAMMA_CNTLB_EXP_REGION8_NUM_SEGMENTS__SHIFT 0xc
+#define REGAMMA_CNTLB_REGION_8_9__REGAMMA_CNTLB_EXP_REGION9_LUT_OFFSET_MASK 0x1ff0000
+#define REGAMMA_CNTLB_REGION_8_9__REGAMMA_CNTLB_EXP_REGION9_LUT_OFFSET__SHIFT 0x10
+#define REGAMMA_CNTLB_REGION_8_9__REGAMMA_CNTLB_EXP_REGION9_NUM_SEGMENTS_MASK 0x70000000
+#define REGAMMA_CNTLB_REGION_8_9__REGAMMA_CNTLB_EXP_REGION9_NUM_SEGMENTS__SHIFT 0x1c
+#define REGAMMA_CNTLB_REGION_10_11__REGAMMA_CNTLB_EXP_REGION10_LUT_OFFSET_MASK 0x1ff
+#define REGAMMA_CNTLB_REGION_10_11__REGAMMA_CNTLB_EXP_REGION10_LUT_OFFSET__SHIFT 0x0
+#define REGAMMA_CNTLB_REGION_10_11__REGAMMA_CNTLB_EXP_REGION10_NUM_SEGMENTS_MASK 0x7000
+#define REGAMMA_CNTLB_REGION_10_11__REGAMMA_CNTLB_EXP_REGION10_NUM_SEGMENTS__SHIFT 0xc
+#define REGAMMA_CNTLB_REGION_10_11__REGAMMA_CNTLB_EXP_REGION11_LUT_OFFSET_MASK 0x1ff0000
+#define REGAMMA_CNTLB_REGION_10_11__REGAMMA_CNTLB_EXP_REGION11_LUT_OFFSET__SHIFT 0x10
+#define REGAMMA_CNTLB_REGION_10_11__REGAMMA_CNTLB_EXP_REGION11_NUM_SEGMENTS_MASK 0x70000000
+#define REGAMMA_CNTLB_REGION_10_11__REGAMMA_CNTLB_EXP_REGION11_NUM_SEGMENTS__SHIFT 0x1c
+#define REGAMMA_CNTLB_REGION_12_13__REGAMMA_CNTLB_EXP_REGION12_LUT_OFFSET_MASK 0x1ff
+#define REGAMMA_CNTLB_REGION_12_13__REGAMMA_CNTLB_EXP_REGION12_LUT_OFFSET__SHIFT 0x0
+#define REGAMMA_CNTLB_REGION_12_13__REGAMMA_CNTLB_EXP_REGION12_NUM_SEGMENTS_MASK 0x7000
+#define REGAMMA_CNTLB_REGION_12_13__REGAMMA_CNTLB_EXP_REGION12_NUM_SEGMENTS__SHIFT 0xc
+#define REGAMMA_CNTLB_REGION_12_13__REGAMMA_CNTLB_EXP_REGION13_LUT_OFFSET_MASK 0x1ff0000
+#define REGAMMA_CNTLB_REGION_12_13__REGAMMA_CNTLB_EXP_REGION13_LUT_OFFSET__SHIFT 0x10
+#define REGAMMA_CNTLB_REGION_12_13__REGAMMA_CNTLB_EXP_REGION13_NUM_SEGMENTS_MASK 0x70000000
+#define REGAMMA_CNTLB_REGION_12_13__REGAMMA_CNTLB_EXP_REGION13_NUM_SEGMENTS__SHIFT 0x1c
+#define REGAMMA_CNTLB_REGION_14_15__REGAMMA_CNTLB_EXP_REGION14_LUT_OFFSET_MASK 0x1ff
+#define REGAMMA_CNTLB_REGION_14_15__REGAMMA_CNTLB_EXP_REGION14_LUT_OFFSET__SHIFT 0x0
+#define REGAMMA_CNTLB_REGION_14_15__REGAMMA_CNTLB_EXP_REGION14_NUM_SEGMENTS_MASK 0x7000
+#define REGAMMA_CNTLB_REGION_14_15__REGAMMA_CNTLB_EXP_REGION14_NUM_SEGMENTS__SHIFT 0xc
+#define REGAMMA_CNTLB_REGION_14_15__REGAMMA_CNTLB_EXP_REGION15_LUT_OFFSET_MASK 0x1ff0000
+#define REGAMMA_CNTLB_REGION_14_15__REGAMMA_CNTLB_EXP_REGION15_LUT_OFFSET__SHIFT 0x10
+#define REGAMMA_CNTLB_REGION_14_15__REGAMMA_CNTLB_EXP_REGION15_NUM_SEGMENTS_MASK 0x70000000
+#define REGAMMA_CNTLB_REGION_14_15__REGAMMA_CNTLB_EXP_REGION15_NUM_SEGMENTS__SHIFT 0x1c
+#define ALPHA_CONTROL__ALPHA_ROUND_TRUNC_MODE_MASK 0x1
+#define ALPHA_CONTROL__ALPHA_ROUND_TRUNC_MODE__SHIFT 0x0
+#define ALPHA_CONTROL__CURSOR_ALPHA_BLND_ENA_MASK 0x2
+#define ALPHA_CONTROL__CURSOR_ALPHA_BLND_ENA__SHIFT 0x1
+#define GRPH_XDMA_RECOVERY_SURFACE_ADDRESS__GRPH_XDMA_RECOVERY_SURFACE_ADDRESS_MASK 0xffffff00
+#define GRPH_XDMA_RECOVERY_SURFACE_ADDRESS__GRPH_XDMA_RECOVERY_SURFACE_ADDRESS__SHIFT 0x8
+#define GRPH_XDMA_RECOVERY_SURFACE_ADDRESS_HIGH__GRPH_XDMA_RECOVERY_SURFACE_ADDRESS_HIGH_MASK 0xff
+#define GRPH_XDMA_RECOVERY_SURFACE_ADDRESS_HIGH__GRPH_XDMA_RECOVERY_SURFACE_ADDRESS_HIGH__SHIFT 0x0
+#define GRPH_XDMA_CACHE_UNDERFLOW_DET_STATUS__GRPH_XDMA_CACHE_UNDERFLOW_CNT_MASK 0xfffff
+#define GRPH_XDMA_CACHE_UNDERFLOW_DET_STATUS__GRPH_XDMA_CACHE_UNDERFLOW_CNT__SHIFT 0x0
+#define GRPH_XDMA_CACHE_UNDERFLOW_DET_STATUS__GRPH_XDMA_CACHE_UNDERFLOW_CNT_STATUS_MASK 0x1000000
+#define GRPH_XDMA_CACHE_UNDERFLOW_DET_STATUS__GRPH_XDMA_CACHE_UNDERFLOW_CNT_STATUS__SHIFT 0x18
+#define GRPH_XDMA_CACHE_UNDERFLOW_DET_STATUS__GRPH_XDMA_CACHE_UNDERFLOW_FRAME_MASK_MASK 0x2000000
+#define GRPH_XDMA_CACHE_UNDERFLOW_DET_STATUS__GRPH_XDMA_CACHE_UNDERFLOW_FRAME_MASK__SHIFT 0x19
+#define GRPH_XDMA_CACHE_UNDERFLOW_DET_STATUS__GRPH_XDMA_CACHE_UNDERFLOW_FRAME_ACK_MASK 0x4000000
+#define GRPH_XDMA_CACHE_UNDERFLOW_DET_STATUS__GRPH_XDMA_CACHE_UNDERFLOW_FRAME_ACK__SHIFT 0x1a
+#define GRPH_XDMA_CACHE_UNDERFLOW_DET_STATUS__GRPH_XDMA_CACHE_UNDERFLOW_INT_MASK 0x10000000
+#define GRPH_XDMA_CACHE_UNDERFLOW_DET_STATUS__GRPH_XDMA_CACHE_UNDERFLOW_INT__SHIFT 0x1c
+#define GRPH_XDMA_CACHE_UNDERFLOW_DET_STATUS__GRPH_XDMA_CACHE_UNDERFLOW_INT_MASK_MASK 0x20000000
+#define GRPH_XDMA_CACHE_UNDERFLOW_DET_STATUS__GRPH_XDMA_CACHE_UNDERFLOW_INT_MASK__SHIFT 0x1d
+#define GRPH_XDMA_CACHE_UNDERFLOW_DET_STATUS__GRPH_XDMA_CACHE_UNDERFLOW_INT_ACK_MASK 0x40000000
+#define GRPH_XDMA_CACHE_UNDERFLOW_DET_STATUS__GRPH_XDMA_CACHE_UNDERFLOW_INT_ACK__SHIFT 0x1e
+#define GRPH_SURFACE_COUNTER_CONTROL__GRPH_SURFACE_COUNTER_EN_MASK 0x1
+#define GRPH_SURFACE_COUNTER_CONTROL__GRPH_SURFACE_COUNTER_EN__SHIFT 0x0
+#define GRPH_SURFACE_COUNTER_CONTROL__GRPH_SURFACE_COUNTER_EVENT_SELECT_MASK 0x1e
+#define GRPH_SURFACE_COUNTER_CONTROL__GRPH_SURFACE_COUNTER_EVENT_SELECT__SHIFT 0x1
+#define GRPH_SURFACE_COUNTER_CONTROL__GRPH_SURFACE_COUNTER_ERR_WRAP_OCCURED_MASK 0x200
+#define GRPH_SURFACE_COUNTER_CONTROL__GRPH_SURFACE_COUNTER_ERR_WRAP_OCCURED__SHIFT 0x9
+#define GRPH_SURFACE_COUNTER_OUTPUT__GRPH_SURFACE_COUNTER_MIN_MASK 0xffff
+#define GRPH_SURFACE_COUNTER_OUTPUT__GRPH_SURFACE_COUNTER_MIN__SHIFT 0x0
+#define GRPH_SURFACE_COUNTER_OUTPUT__GRPH_SURFACE_COUNTER_MAX_MASK 0xffff0000
+#define GRPH_SURFACE_COUNTER_OUTPUT__GRPH_SURFACE_COUNTER_MAX__SHIFT 0x10
+#define DIG_FE_CNTL__DIG_SOURCE_SELECT_MASK 0x7
+#define DIG_FE_CNTL__DIG_SOURCE_SELECT__SHIFT 0x0
+#define DIG_FE_CNTL__DIG_STEREOSYNC_SELECT_MASK 0x70
+#define DIG_FE_CNTL__DIG_STEREOSYNC_SELECT__SHIFT 0x4
+#define DIG_FE_CNTL__DIG_STEREOSYNC_GATE_EN_MASK 0x100
+#define DIG_FE_CNTL__DIG_STEREOSYNC_GATE_EN__SHIFT 0x8
+#define DIG_FE_CNTL__DIG_START_MASK 0x400
+#define DIG_FE_CNTL__DIG_START__SHIFT 0xa
+#define DIG_FE_CNTL__DIG_SYMCLK_FE_ON_MASK 0x1000000
+#define DIG_FE_CNTL__DIG_SYMCLK_FE_ON__SHIFT 0x18
+#define DIG_FE_CNTL__TMDS_PIXEL_ENCODING_MASK 0x10000000
+#define DIG_FE_CNTL__TMDS_PIXEL_ENCODING__SHIFT 0x1c
+#define DIG_FE_CNTL__TMDS_COLOR_FORMAT_MASK 0xc0000000
+#define DIG_FE_CNTL__TMDS_COLOR_FORMAT__SHIFT 0x1e
+#define DIG_OUTPUT_CRC_CNTL__DIG_OUTPUT_CRC_EN_MASK 0x1
+#define DIG_OUTPUT_CRC_CNTL__DIG_OUTPUT_CRC_EN__SHIFT 0x0
+#define DIG_OUTPUT_CRC_CNTL__DIG_OUTPUT_CRC_LINK_SEL_MASK 0x10
+#define DIG_OUTPUT_CRC_CNTL__DIG_OUTPUT_CRC_LINK_SEL__SHIFT 0x4
+#define DIG_OUTPUT_CRC_CNTL__DIG_OUTPUT_CRC_DATA_SEL_MASK 0x300
+#define DIG_OUTPUT_CRC_CNTL__DIG_OUTPUT_CRC_DATA_SEL__SHIFT 0x8
+#define DIG_OUTPUT_CRC_RESULT__DIG_OUTPUT_CRC_RESULT_MASK 0x3fffffff
+#define DIG_OUTPUT_CRC_RESULT__DIG_OUTPUT_CRC_RESULT__SHIFT 0x0
+#define DIG_CLOCK_PATTERN__DIG_CLOCK_PATTERN_MASK 0x3ff
+#define DIG_CLOCK_PATTERN__DIG_CLOCK_PATTERN__SHIFT 0x0
+#define DIG_TEST_PATTERN__DIG_TEST_PATTERN_OUT_EN_MASK 0x1
+#define DIG_TEST_PATTERN__DIG_TEST_PATTERN_OUT_EN__SHIFT 0x0
+#define DIG_TEST_PATTERN__DIG_HALF_CLOCK_PATTERN_SEL_MASK 0x2
+#define DIG_TEST_PATTERN__DIG_HALF_CLOCK_PATTERN_SEL__SHIFT 0x1
+#define DIG_TEST_PATTERN__DIG_RANDOM_PATTERN_OUT_EN_MASK 0x10
+#define DIG_TEST_PATTERN__DIG_RANDOM_PATTERN_OUT_EN__SHIFT 0x4
+#define DIG_TEST_PATTERN__DIG_RANDOM_PATTERN_RESET_MASK 0x20
+#define DIG_TEST_PATTERN__DIG_RANDOM_PATTERN_RESET__SHIFT 0x5
+#define DIG_TEST_PATTERN__DIG_TEST_PATTERN_EXTERNAL_RESET_EN_MASK 0x40
+#define DIG_TEST_PATTERN__DIG_TEST_PATTERN_EXTERNAL_RESET_EN__SHIFT 0x6
+#define DIG_TEST_PATTERN__DIG_STATIC_TEST_PATTERN_MASK 0x3ff0000
+#define DIG_TEST_PATTERN__DIG_STATIC_TEST_PATTERN__SHIFT 0x10
+#define DIG_RANDOM_PATTERN_SEED__DIG_RANDOM_PATTERN_SEED_MASK 0xffffff
+#define DIG_RANDOM_PATTERN_SEED__DIG_RANDOM_PATTERN_SEED__SHIFT 0x0
+#define DIG_RANDOM_PATTERN_SEED__DIG_RAN_PAT_DURING_DE_ONLY_MASK 0x1000000
+#define DIG_RANDOM_PATTERN_SEED__DIG_RAN_PAT_DURING_DE_ONLY__SHIFT 0x18
+#define DIG_FIFO_STATUS__DIG_FIFO_LEVEL_ERROR_MASK 0x1
+#define DIG_FIFO_STATUS__DIG_FIFO_LEVEL_ERROR__SHIFT 0x0
+#define DIG_FIFO_STATUS__DIG_FIFO_USE_OVERWRITE_LEVEL_MASK 0x2
+#define DIG_FIFO_STATUS__DIG_FIFO_USE_OVERWRITE_LEVEL__SHIFT 0x1
+#define DIG_FIFO_STATUS__DIG_FIFO_OVERWRITE_LEVEL_MASK 0xfc
+#define DIG_FIFO_STATUS__DIG_FIFO_OVERWRITE_LEVEL__SHIFT 0x2
+#define DIG_FIFO_STATUS__DIG_FIFO_ERROR_ACK_MASK 0x100
+#define DIG_FIFO_STATUS__DIG_FIFO_ERROR_ACK__SHIFT 0x8
+#define DIG_FIFO_STATUS__DIG_FIFO_CAL_AVERAGE_LEVEL_MASK 0xfc00
+#define DIG_FIFO_STATUS__DIG_FIFO_CAL_AVERAGE_LEVEL__SHIFT 0xa
+#define DIG_FIFO_STATUS__DIG_FIFO_MAXIMUM_LEVEL_MASK 0x1f0000
+#define DIG_FIFO_STATUS__DIG_FIFO_MAXIMUM_LEVEL__SHIFT 0x10
+#define DIG_FIFO_STATUS__DIG_FIFO_MINIMUM_LEVEL_MASK 0x3c00000
+#define DIG_FIFO_STATUS__DIG_FIFO_MINIMUM_LEVEL__SHIFT 0x16
+#define DIG_FIFO_STATUS__DIG_FIFO_READ_CLOCK_SRC_MASK 0x4000000
+#define DIG_FIFO_STATUS__DIG_FIFO_READ_CLOCK_SRC__SHIFT 0x1a
+#define DIG_FIFO_STATUS__DIG_FIFO_CALIBRATED_MASK 0x20000000
+#define DIG_FIFO_STATUS__DIG_FIFO_CALIBRATED__SHIFT 0x1d
+#define DIG_FIFO_STATUS__DIG_FIFO_FORCE_RECAL_AVERAGE_MASK 0x40000000
+#define DIG_FIFO_STATUS__DIG_FIFO_FORCE_RECAL_AVERAGE__SHIFT 0x1e
+#define DIG_FIFO_STATUS__DIG_FIFO_FORCE_RECOMP_MINMAX_MASK 0x80000000
+#define DIG_FIFO_STATUS__DIG_FIFO_FORCE_RECOMP_MINMAX__SHIFT 0x1f
+#define DIG_DISPCLK_SWITCH_CNTL__DIG_DISPCLK_SWITCH_POINT_MASK 0x1
+#define DIG_DISPCLK_SWITCH_CNTL__DIG_DISPCLK_SWITCH_POINT__SHIFT 0x0
+#define DIG_DISPCLK_SWITCH_STATUS__DIG_DISPCLK_SWITCH_ALLOWED_MASK 0x1
+#define DIG_DISPCLK_SWITCH_STATUS__DIG_DISPCLK_SWITCH_ALLOWED__SHIFT 0x0
+#define DIG_DISPCLK_SWITCH_STATUS__DIG_DISPCLK_SWITCH_ALLOWED_INT_MASK 0x10
+#define DIG_DISPCLK_SWITCH_STATUS__DIG_DISPCLK_SWITCH_ALLOWED_INT__SHIFT 0x4
+#define DIG_DISPCLK_SWITCH_STATUS__DIG_DISPCLK_SWITCH_ALLOWED_INT_ACK_MASK 0x100
+#define DIG_DISPCLK_SWITCH_STATUS__DIG_DISPCLK_SWITCH_ALLOWED_INT_ACK__SHIFT 0x8
+#define DIG_DISPCLK_SWITCH_STATUS__DIG_DISPCLK_SWITCH_ALLOWED_INT_MASK_MASK 0x1000
+#define DIG_DISPCLK_SWITCH_STATUS__DIG_DISPCLK_SWITCH_ALLOWED_INT_MASK__SHIFT 0xc
+#define HDMI_CONTROL__HDMI_KEEPOUT_MODE_MASK 0x1
+#define HDMI_CONTROL__HDMI_KEEPOUT_MODE__SHIFT 0x0
+#define HDMI_CONTROL__HDMI_CLOCK_CHANNEL_RATE_MASK 0x4
+#define HDMI_CONTROL__HDMI_CLOCK_CHANNEL_RATE__SHIFT 0x2
+#define HDMI_CONTROL__HDMI_NO_EXTRA_NULL_PACKET_FILLED_MASK 0x8
+#define HDMI_CONTROL__HDMI_NO_EXTRA_NULL_PACKET_FILLED__SHIFT 0x3
+#define HDMI_CONTROL__HDMI_PACKET_GEN_VERSION_MASK 0x10
+#define HDMI_CONTROL__HDMI_PACKET_GEN_VERSION__SHIFT 0x4
+#define HDMI_CONTROL__HDMI_ERROR_ACK_MASK 0x100
+#define HDMI_CONTROL__HDMI_ERROR_ACK__SHIFT 0x8
+#define HDMI_CONTROL__HDMI_ERROR_MASK_MASK 0x200
+#define HDMI_CONTROL__HDMI_ERROR_MASK__SHIFT 0x9
+#define HDMI_CONTROL__HDMI_DEEP_COLOR_ENABLE_MASK 0x1000000
+#define HDMI_CONTROL__HDMI_DEEP_COLOR_ENABLE__SHIFT 0x18
+#define HDMI_CONTROL__HDMI_DEEP_COLOR_DEPTH_MASK 0x30000000
+#define HDMI_CONTROL__HDMI_DEEP_COLOR_DEPTH__SHIFT 0x1c
+#define HDMI_STATUS__HDMI_ACTIVE_AVMUTE_MASK 0x1
+#define HDMI_STATUS__HDMI_ACTIVE_AVMUTE__SHIFT 0x0
+#define HDMI_STATUS__HDMI_AUDIO_PACKET_ERROR_MASK 0x10000
+#define HDMI_STATUS__HDMI_AUDIO_PACKET_ERROR__SHIFT 0x10
+#define HDMI_STATUS__HDMI_VBI_PACKET_ERROR_MASK 0x100000
+#define HDMI_STATUS__HDMI_VBI_PACKET_ERROR__SHIFT 0x14
+#define HDMI_STATUS__HDMI_ERROR_INT_MASK 0x8000000
+#define HDMI_STATUS__HDMI_ERROR_INT__SHIFT 0x1b
+#define HDMI_AUDIO_PACKET_CONTROL__HDMI_AUDIO_DELAY_EN_MASK 0x30
+#define HDMI_AUDIO_PACKET_CONTROL__HDMI_AUDIO_DELAY_EN__SHIFT 0x4
+#define HDMI_AUDIO_PACKET_CONTROL__HDMI_AUDIO_SEND_MAX_PACKETS_MASK 0x100
+#define HDMI_AUDIO_PACKET_CONTROL__HDMI_AUDIO_SEND_MAX_PACKETS__SHIFT 0x8
+#define HDMI_AUDIO_PACKET_CONTROL__HDMI_AUDIO_PACKETS_PER_LINE_MASK 0x1f0000
+#define HDMI_AUDIO_PACKET_CONTROL__HDMI_AUDIO_PACKETS_PER_LINE__SHIFT 0x10
+#define HDMI_ACR_PACKET_CONTROL__HDMI_ACR_SEND_MASK 0x1
+#define HDMI_ACR_PACKET_CONTROL__HDMI_ACR_SEND__SHIFT 0x0
+#define HDMI_ACR_PACKET_CONTROL__HDMI_ACR_CONT_MASK 0x2
+#define HDMI_ACR_PACKET_CONTROL__HDMI_ACR_CONT__SHIFT 0x1
+#define HDMI_ACR_PACKET_CONTROL__HDMI_ACR_SELECT_MASK 0x30
+#define HDMI_ACR_PACKET_CONTROL__HDMI_ACR_SELECT__SHIFT 0x4
+#define HDMI_ACR_PACKET_CONTROL__HDMI_ACR_SOURCE_MASK 0x100
+#define HDMI_ACR_PACKET_CONTROL__HDMI_ACR_SOURCE__SHIFT 0x8
+#define HDMI_ACR_PACKET_CONTROL__HDMI_ACR_AUTO_SEND_MASK 0x1000
+#define HDMI_ACR_PACKET_CONTROL__HDMI_ACR_AUTO_SEND__SHIFT 0xc
+#define HDMI_ACR_PACKET_CONTROL__HDMI_ACR_N_MULTIPLE_MASK 0x70000
+#define HDMI_ACR_PACKET_CONTROL__HDMI_ACR_N_MULTIPLE__SHIFT 0x10
+#define HDMI_ACR_PACKET_CONTROL__HDMI_ACR_AUDIO_PRIORITY_MASK 0x80000000
+#define HDMI_ACR_PACKET_CONTROL__HDMI_ACR_AUDIO_PRIORITY__SHIFT 0x1f
+#define HDMI_VBI_PACKET_CONTROL__HDMI_NULL_SEND_MASK 0x1
+#define HDMI_VBI_PACKET_CONTROL__HDMI_NULL_SEND__SHIFT 0x0
+#define HDMI_VBI_PACKET_CONTROL__HDMI_GC_SEND_MASK 0x10
+#define HDMI_VBI_PACKET_CONTROL__HDMI_GC_SEND__SHIFT 0x4
+#define HDMI_VBI_PACKET_CONTROL__HDMI_GC_CONT_MASK 0x20
+#define HDMI_VBI_PACKET_CONTROL__HDMI_GC_CONT__SHIFT 0x5
+#define HDMI_VBI_PACKET_CONTROL__HDMI_ISRC_SEND_MASK 0x100
+#define HDMI_VBI_PACKET_CONTROL__HDMI_ISRC_SEND__SHIFT 0x8
+#define HDMI_VBI_PACKET_CONTROL__HDMI_ISRC_CONT_MASK 0x200
+#define HDMI_VBI_PACKET_CONTROL__HDMI_ISRC_CONT__SHIFT 0x9
+#define HDMI_VBI_PACKET_CONTROL__HDMI_ISRC_LINE_MASK 0x3f0000
+#define HDMI_VBI_PACKET_CONTROL__HDMI_ISRC_LINE__SHIFT 0x10
+#define HDMI_INFOFRAME_CONTROL0__HDMI_AVI_INFO_SEND_MASK 0x1
+#define HDMI_INFOFRAME_CONTROL0__HDMI_AVI_INFO_SEND__SHIFT 0x0
+#define HDMI_INFOFRAME_CONTROL0__HDMI_AVI_INFO_CONT_MASK 0x2
+#define HDMI_INFOFRAME_CONTROL0__HDMI_AVI_INFO_CONT__SHIFT 0x1
+#define HDMI_INFOFRAME_CONTROL0__HDMI_AUDIO_INFO_SEND_MASK 0x10
+#define HDMI_INFOFRAME_CONTROL0__HDMI_AUDIO_INFO_SEND__SHIFT 0x4
+#define HDMI_INFOFRAME_CONTROL0__HDMI_AUDIO_INFO_CONT_MASK 0x20
+#define HDMI_INFOFRAME_CONTROL0__HDMI_AUDIO_INFO_CONT__SHIFT 0x5
+#define HDMI_INFOFRAME_CONTROL0__HDMI_MPEG_INFO_SEND_MASK 0x100
+#define HDMI_INFOFRAME_CONTROL0__HDMI_MPEG_INFO_SEND__SHIFT 0x8
+#define HDMI_INFOFRAME_CONTROL0__HDMI_MPEG_INFO_CONT_MASK 0x200
+#define HDMI_INFOFRAME_CONTROL0__HDMI_MPEG_INFO_CONT__SHIFT 0x9
+#define HDMI_INFOFRAME_CONTROL1__HDMI_AVI_INFO_LINE_MASK 0x3f
+#define HDMI_INFOFRAME_CONTROL1__HDMI_AVI_INFO_LINE__SHIFT 0x0
+#define HDMI_INFOFRAME_CONTROL1__HDMI_AUDIO_INFO_LINE_MASK 0x3f00
+#define HDMI_INFOFRAME_CONTROL1__HDMI_AUDIO_INFO_LINE__SHIFT 0x8
+#define HDMI_INFOFRAME_CONTROL1__HDMI_MPEG_INFO_LINE_MASK 0x3f0000
+#define HDMI_INFOFRAME_CONTROL1__HDMI_MPEG_INFO_LINE__SHIFT 0x10
+#define HDMI_GENERIC_PACKET_CONTROL0__HDMI_GENERIC0_SEND_MASK 0x1
+#define HDMI_GENERIC_PACKET_CONTROL0__HDMI_GENERIC0_SEND__SHIFT 0x0
+#define HDMI_GENERIC_PACKET_CONTROL0__HDMI_GENERIC0_CONT_MASK 0x2
+#define HDMI_GENERIC_PACKET_CONTROL0__HDMI_GENERIC0_CONT__SHIFT 0x1
+#define HDMI_GENERIC_PACKET_CONTROL0__HDMI_GENERIC1_SEND_MASK 0x10
+#define HDMI_GENERIC_PACKET_CONTROL0__HDMI_GENERIC1_SEND__SHIFT 0x4
+#define HDMI_GENERIC_PACKET_CONTROL0__HDMI_GENERIC1_CONT_MASK 0x20
+#define HDMI_GENERIC_PACKET_CONTROL0__HDMI_GENERIC1_CONT__SHIFT 0x5
+#define HDMI_GENERIC_PACKET_CONTROL0__HDMI_GENERIC0_LINE_MASK 0x3f0000
+#define HDMI_GENERIC_PACKET_CONTROL0__HDMI_GENERIC0_LINE__SHIFT 0x10
+#define HDMI_GENERIC_PACKET_CONTROL0__HDMI_GENERIC1_LINE_MASK 0x3f000000
+#define HDMI_GENERIC_PACKET_CONTROL0__HDMI_GENERIC1_LINE__SHIFT 0x18
+#define HDMI_GC__HDMI_GC_AVMUTE_MASK 0x1
+#define HDMI_GC__HDMI_GC_AVMUTE__SHIFT 0x0
+#define HDMI_GC__HDMI_GC_AVMUTE_CONT_MASK 0x4
+#define HDMI_GC__HDMI_GC_AVMUTE_CONT__SHIFT 0x2
+#define HDMI_GC__HDMI_DEFAULT_PHASE_MASK 0x10
+#define HDMI_GC__HDMI_DEFAULT_PHASE__SHIFT 0x4
+#define HDMI_GC__HDMI_PACKING_PHASE_MASK 0xf00
+#define HDMI_GC__HDMI_PACKING_PHASE__SHIFT 0x8
+#define HDMI_GC__HDMI_PACKING_PHASE_OVERRIDE_MASK 0x1000
+#define HDMI_GC__HDMI_PACKING_PHASE_OVERRIDE__SHIFT 0xc
+#define AFMT_AUDIO_PACKET_CONTROL2__AFMT_AUDIO_LAYOUT_OVRD_MASK 0x1
+#define AFMT_AUDIO_PACKET_CONTROL2__AFMT_AUDIO_LAYOUT_OVRD__SHIFT 0x0
+#define AFMT_AUDIO_PACKET_CONTROL2__AFMT_AUDIO_LAYOUT_SELECT_MASK 0x2
+#define AFMT_AUDIO_PACKET_CONTROL2__AFMT_AUDIO_LAYOUT_SELECT__SHIFT 0x1
+#define AFMT_AUDIO_PACKET_CONTROL2__AFMT_AUDIO_CHANNEL_ENABLE_MASK 0xff00
+#define AFMT_AUDIO_PACKET_CONTROL2__AFMT_AUDIO_CHANNEL_ENABLE__SHIFT 0x8
+#define AFMT_AUDIO_PACKET_CONTROL2__AFMT_DP_AUDIO_STREAM_ID_MASK 0xff0000
+#define AFMT_AUDIO_PACKET_CONTROL2__AFMT_DP_AUDIO_STREAM_ID__SHIFT 0x10
+#define AFMT_AUDIO_PACKET_CONTROL2__AFMT_HBR_ENABLE_OVRD_MASK 0x1000000
+#define AFMT_AUDIO_PACKET_CONTROL2__AFMT_HBR_ENABLE_OVRD__SHIFT 0x18
+#define AFMT_AUDIO_PACKET_CONTROL2__AFMT_60958_OSF_OVRD_MASK 0x10000000
+#define AFMT_AUDIO_PACKET_CONTROL2__AFMT_60958_OSF_OVRD__SHIFT 0x1c
+#define AFMT_ISRC1_0__AFMT_ISRC_STATUS_MASK 0x7
+#define AFMT_ISRC1_0__AFMT_ISRC_STATUS__SHIFT 0x0
+#define AFMT_ISRC1_0__AFMT_ISRC_CONTINUE_MASK 0x40
+#define AFMT_ISRC1_0__AFMT_ISRC_CONTINUE__SHIFT 0x6
+#define AFMT_ISRC1_0__AFMT_ISRC_VALID_MASK 0x80
+#define AFMT_ISRC1_0__AFMT_ISRC_VALID__SHIFT 0x7
+#define AFMT_ISRC1_1__AFMT_UPC_EAN_ISRC0_MASK 0xff
+#define AFMT_ISRC1_1__AFMT_UPC_EAN_ISRC0__SHIFT 0x0
+#define AFMT_ISRC1_1__AFMT_UPC_EAN_ISRC1_MASK 0xff00
+#define AFMT_ISRC1_1__AFMT_UPC_EAN_ISRC1__SHIFT 0x8
+#define AFMT_ISRC1_1__AFMT_UPC_EAN_ISRC2_MASK 0xff0000
+#define AFMT_ISRC1_1__AFMT_UPC_EAN_ISRC2__SHIFT 0x10
+#define AFMT_ISRC1_1__AFMT_UPC_EAN_ISRC3_MASK 0xff000000
+#define AFMT_ISRC1_1__AFMT_UPC_EAN_ISRC3__SHIFT 0x18
+#define AFMT_ISRC1_2__AFMT_UPC_EAN_ISRC4_MASK 0xff
+#define AFMT_ISRC1_2__AFMT_UPC_EAN_ISRC4__SHIFT 0x0
+#define AFMT_ISRC1_2__AFMT_UPC_EAN_ISRC5_MASK 0xff00
+#define AFMT_ISRC1_2__AFMT_UPC_EAN_ISRC5__SHIFT 0x8
+#define AFMT_ISRC1_2__AFMT_UPC_EAN_ISRC6_MASK 0xff0000
+#define AFMT_ISRC1_2__AFMT_UPC_EAN_ISRC6__SHIFT 0x10
+#define AFMT_ISRC1_2__AFMT_UPC_EAN_ISRC7_MASK 0xff000000
+#define AFMT_ISRC1_2__AFMT_UPC_EAN_ISRC7__SHIFT 0x18
+#define AFMT_ISRC1_3__AFMT_UPC_EAN_ISRC8_MASK 0xff
+#define AFMT_ISRC1_3__AFMT_UPC_EAN_ISRC8__SHIFT 0x0
+#define AFMT_ISRC1_3__AFMT_UPC_EAN_ISRC9_MASK 0xff00
+#define AFMT_ISRC1_3__AFMT_UPC_EAN_ISRC9__SHIFT 0x8
+#define AFMT_ISRC1_3__AFMT_UPC_EAN_ISRC10_MASK 0xff0000
+#define AFMT_ISRC1_3__AFMT_UPC_EAN_ISRC10__SHIFT 0x10
+#define AFMT_ISRC1_3__AFMT_UPC_EAN_ISRC11_MASK 0xff000000
+#define AFMT_ISRC1_3__AFMT_UPC_EAN_ISRC11__SHIFT 0x18
+#define AFMT_ISRC1_4__AFMT_UPC_EAN_ISRC12_MASK 0xff
+#define AFMT_ISRC1_4__AFMT_UPC_EAN_ISRC12__SHIFT 0x0
+#define AFMT_ISRC1_4__AFMT_UPC_EAN_ISRC13_MASK 0xff00
+#define AFMT_ISRC1_4__AFMT_UPC_EAN_ISRC13__SHIFT 0x8
+#define AFMT_ISRC1_4__AFMT_UPC_EAN_ISRC14_MASK 0xff0000
+#define AFMT_ISRC1_4__AFMT_UPC_EAN_ISRC14__SHIFT 0x10
+#define AFMT_ISRC1_4__AFMT_UPC_EAN_ISRC15_MASK 0xff000000
+#define AFMT_ISRC1_4__AFMT_UPC_EAN_ISRC15__SHIFT 0x18
+#define AFMT_ISRC2_0__AFMT_UPC_EAN_ISRC16_MASK 0xff
+#define AFMT_ISRC2_0__AFMT_UPC_EAN_ISRC16__SHIFT 0x0
+#define AFMT_ISRC2_0__AFMT_UPC_EAN_ISRC17_MASK 0xff00
+#define AFMT_ISRC2_0__AFMT_UPC_EAN_ISRC17__SHIFT 0x8
+#define AFMT_ISRC2_0__AFMT_UPC_EAN_ISRC18_MASK 0xff0000
+#define AFMT_ISRC2_0__AFMT_UPC_EAN_ISRC18__SHIFT 0x10
+#define AFMT_ISRC2_0__AFMT_UPC_EAN_ISRC19_MASK 0xff000000
+#define AFMT_ISRC2_0__AFMT_UPC_EAN_ISRC19__SHIFT 0x18
+#define AFMT_ISRC2_1__AFMT_UPC_EAN_ISRC20_MASK 0xff
+#define AFMT_ISRC2_1__AFMT_UPC_EAN_ISRC20__SHIFT 0x0
+#define AFMT_ISRC2_1__AFMT_UPC_EAN_ISRC21_MASK 0xff00
+#define AFMT_ISRC2_1__AFMT_UPC_EAN_ISRC21__SHIFT 0x8
+#define AFMT_ISRC2_1__AFMT_UPC_EAN_ISRC22_MASK 0xff0000
+#define AFMT_ISRC2_1__AFMT_UPC_EAN_ISRC22__SHIFT 0x10
+#define AFMT_ISRC2_1__AFMT_UPC_EAN_ISRC23_MASK 0xff000000
+#define AFMT_ISRC2_1__AFMT_UPC_EAN_ISRC23__SHIFT 0x18
+#define AFMT_ISRC2_2__AFMT_UPC_EAN_ISRC24_MASK 0xff
+#define AFMT_ISRC2_2__AFMT_UPC_EAN_ISRC24__SHIFT 0x0
+#define AFMT_ISRC2_2__AFMT_UPC_EAN_ISRC25_MASK 0xff00
+#define AFMT_ISRC2_2__AFMT_UPC_EAN_ISRC25__SHIFT 0x8
+#define AFMT_ISRC2_2__AFMT_UPC_EAN_ISRC26_MASK 0xff0000
+#define AFMT_ISRC2_2__AFMT_UPC_EAN_ISRC26__SHIFT 0x10
+#define AFMT_ISRC2_2__AFMT_UPC_EAN_ISRC27_MASK 0xff000000
+#define AFMT_ISRC2_2__AFMT_UPC_EAN_ISRC27__SHIFT 0x18
+#define AFMT_ISRC2_3__AFMT_UPC_EAN_ISRC28_MASK 0xff
+#define AFMT_ISRC2_3__AFMT_UPC_EAN_ISRC28__SHIFT 0x0
+#define AFMT_ISRC2_3__AFMT_UPC_EAN_ISRC29_MASK 0xff00
+#define AFMT_ISRC2_3__AFMT_UPC_EAN_ISRC29__SHIFT 0x8
+#define AFMT_ISRC2_3__AFMT_UPC_EAN_ISRC30_MASK 0xff0000
+#define AFMT_ISRC2_3__AFMT_UPC_EAN_ISRC30__SHIFT 0x10
+#define AFMT_ISRC2_3__AFMT_UPC_EAN_ISRC31_MASK 0xff000000
+#define AFMT_ISRC2_3__AFMT_UPC_EAN_ISRC31__SHIFT 0x18
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_CHECKSUM_MASK 0xff
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_CHECKSUM__SHIFT 0x0
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_S_MASK 0x300
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_S__SHIFT 0x8
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_B_MASK 0xc00
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_B__SHIFT 0xa
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_A_MASK 0x1000
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_A__SHIFT 0xc
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_Y_MASK 0xe000
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_Y__SHIFT 0xd
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_R_MASK 0xf0000
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_R__SHIFT 0x10
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_M_MASK 0x300000
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_M__SHIFT 0x14
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_C_MASK 0xc00000
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_C__SHIFT 0x16
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_SC_MASK 0x3000000
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_SC__SHIFT 0x18
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_Q_MASK 0xc000000
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_Q__SHIFT 0x1a
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_EC_MASK 0x70000000
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_EC__SHIFT 0x1c
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_ITC_MASK 0x80000000
+#define AFMT_AVI_INFO0__AFMT_AVI_INFO_ITC__SHIFT 0x1f
+#define AFMT_AVI_INFO1__AFMT_AVI_INFO_VIC_MASK 0xff
+#define AFMT_AVI_INFO1__AFMT_AVI_INFO_VIC__SHIFT 0x0
+#define AFMT_AVI_INFO1__AFMT_AVI_INFO_PR_MASK 0xf00
+#define AFMT_AVI_INFO1__AFMT_AVI_INFO_PR__SHIFT 0x8
+#define AFMT_AVI_INFO1__AFMT_AVI_INFO_CN_MASK 0x3000
+#define AFMT_AVI_INFO1__AFMT_AVI_INFO_CN__SHIFT 0xc
+#define AFMT_AVI_INFO1__AFMT_AVI_INFO_YQ_MASK 0xc000
+#define AFMT_AVI_INFO1__AFMT_AVI_INFO_YQ__SHIFT 0xe
+#define AFMT_AVI_INFO1__AFMT_AVI_INFO_TOP_MASK 0xffff0000
+#define AFMT_AVI_INFO1__AFMT_AVI_INFO_TOP__SHIFT 0x10
+#define AFMT_AVI_INFO2__AFMT_AVI_INFO_BOTTOM_MASK 0xffff
+#define AFMT_AVI_INFO2__AFMT_AVI_INFO_BOTTOM__SHIFT 0x0
+#define AFMT_AVI_INFO2__AFMT_AVI_INFO_LEFT_MASK 0xffff0000
+#define AFMT_AVI_INFO2__AFMT_AVI_INFO_LEFT__SHIFT 0x10
+#define AFMT_AVI_INFO3__AFMT_AVI_INFO_RIGHT_MASK 0xffff
+#define AFMT_AVI_INFO3__AFMT_AVI_INFO_RIGHT__SHIFT 0x0
+#define AFMT_AVI_INFO3__AFMT_AVI_INFO_VERSION_MASK 0xff000000
+#define AFMT_AVI_INFO3__AFMT_AVI_INFO_VERSION__SHIFT 0x18
+#define AFMT_MPEG_INFO0__AFMT_MPEG_INFO_CHECKSUM_MASK 0xff
+#define AFMT_MPEG_INFO0__AFMT_MPEG_INFO_CHECKSUM__SHIFT 0x0
+#define AFMT_MPEG_INFO0__AFMT_MPEG_INFO_MB0_MASK 0xff00
+#define AFMT_MPEG_INFO0__AFMT_MPEG_INFO_MB0__SHIFT 0x8
+#define AFMT_MPEG_INFO0__AFMT_MPEG_INFO_MB1_MASK 0xff0000
+#define AFMT_MPEG_INFO0__AFMT_MPEG_INFO_MB1__SHIFT 0x10
+#define AFMT_MPEG_INFO0__AFMT_MPEG_INFO_MB2_MASK 0xff000000
+#define AFMT_MPEG_INFO0__AFMT_MPEG_INFO_MB2__SHIFT 0x18
+#define AFMT_MPEG_INFO1__AFMT_MPEG_INFO_MB3_MASK 0xff
+#define AFMT_MPEG_INFO1__AFMT_MPEG_INFO_MB3__SHIFT 0x0
+#define AFMT_MPEG_INFO1__AFMT_MPEG_INFO_MF_MASK 0x300
+#define AFMT_MPEG_INFO1__AFMT_MPEG_INFO_MF__SHIFT 0x8
+#define AFMT_MPEG_INFO1__AFMT_MPEG_INFO_FR_MASK 0x1000
+#define AFMT_MPEG_INFO1__AFMT_MPEG_INFO_FR__SHIFT 0xc
+#define AFMT_GENERIC_HDR__AFMT_GENERIC_HB0_MASK 0xff
+#define AFMT_GENERIC_HDR__AFMT_GENERIC_HB0__SHIFT 0x0
+#define AFMT_GENERIC_HDR__AFMT_GENERIC_HB1_MASK 0xff00
+#define AFMT_GENERIC_HDR__AFMT_GENERIC_HB1__SHIFT 0x8
+#define AFMT_GENERIC_HDR__AFMT_GENERIC_HB2_MASK 0xff0000
+#define AFMT_GENERIC_HDR__AFMT_GENERIC_HB2__SHIFT 0x10
+#define AFMT_GENERIC_HDR__AFMT_GENERIC_HB3_MASK 0xff000000
+#define AFMT_GENERIC_HDR__AFMT_GENERIC_HB3__SHIFT 0x18
+#define AFMT_GENERIC_0__AFMT_GENERIC_BYTE0_MASK 0xff
+#define AFMT_GENERIC_0__AFMT_GENERIC_BYTE0__SHIFT 0x0
+#define AFMT_GENERIC_0__AFMT_GENERIC_BYTE1_MASK 0xff00
+#define AFMT_GENERIC_0__AFMT_GENERIC_BYTE1__SHIFT 0x8
+#define AFMT_GENERIC_0__AFMT_GENERIC_BYTE2_MASK 0xff0000
+#define AFMT_GENERIC_0__AFMT_GENERIC_BYTE2__SHIFT 0x10
+#define AFMT_GENERIC_0__AFMT_GENERIC_BYTE3_MASK 0xff000000
+#define AFMT_GENERIC_0__AFMT_GENERIC_BYTE3__SHIFT 0x18
+#define AFMT_GENERIC_1__AFMT_GENERIC_BYTE4_MASK 0xff
+#define AFMT_GENERIC_1__AFMT_GENERIC_BYTE4__SHIFT 0x0
+#define AFMT_GENERIC_1__AFMT_GENERIC_BYTE5_MASK 0xff00
+#define AFMT_GENERIC_1__AFMT_GENERIC_BYTE5__SHIFT 0x8
+#define AFMT_GENERIC_1__AFMT_GENERIC_BYTE6_MASK 0xff0000
+#define AFMT_GENERIC_1__AFMT_GENERIC_BYTE6__SHIFT 0x10
+#define AFMT_GENERIC_1__AFMT_GENERIC_BYTE7_MASK 0xff000000
+#define AFMT_GENERIC_1__AFMT_GENERIC_BYTE7__SHIFT 0x18
+#define AFMT_GENERIC_2__AFMT_GENERIC_BYTE8_MASK 0xff
+#define AFMT_GENERIC_2__AFMT_GENERIC_BYTE8__SHIFT 0x0
+#define AFMT_GENERIC_2__AFMT_GENERIC_BYTE9_MASK 0xff00
+#define AFMT_GENERIC_2__AFMT_GENERIC_BYTE9__SHIFT 0x8
+#define AFMT_GENERIC_2__AFMT_GENERIC_BYTE10_MASK 0xff0000
+#define AFMT_GENERIC_2__AFMT_GENERIC_BYTE10__SHIFT 0x10
+#define AFMT_GENERIC_2__AFMT_GENERIC_BYTE11_MASK 0xff000000
+#define AFMT_GENERIC_2__AFMT_GENERIC_BYTE11__SHIFT 0x18
+#define AFMT_GENERIC_3__AFMT_GENERIC_BYTE12_MASK 0xff
+#define AFMT_GENERIC_3__AFMT_GENERIC_BYTE12__SHIFT 0x0
+#define AFMT_GENERIC_3__AFMT_GENERIC_BYTE13_MASK 0xff00
+#define AFMT_GENERIC_3__AFMT_GENERIC_BYTE13__SHIFT 0x8
+#define AFMT_GENERIC_3__AFMT_GENERIC_BYTE14_MASK 0xff0000
+#define AFMT_GENERIC_3__AFMT_GENERIC_BYTE14__SHIFT 0x10
+#define AFMT_GENERIC_3__AFMT_GENERIC_BYTE15_MASK 0xff000000
+#define AFMT_GENERIC_3__AFMT_GENERIC_BYTE15__SHIFT 0x18
+#define AFMT_GENERIC_4__AFMT_GENERIC_BYTE16_MASK 0xff
+#define AFMT_GENERIC_4__AFMT_GENERIC_BYTE16__SHIFT 0x0
+#define AFMT_GENERIC_4__AFMT_GENERIC_BYTE17_MASK 0xff00
+#define AFMT_GENERIC_4__AFMT_GENERIC_BYTE17__SHIFT 0x8
+#define AFMT_GENERIC_4__AFMT_GENERIC_BYTE18_MASK 0xff0000
+#define AFMT_GENERIC_4__AFMT_GENERIC_BYTE18__SHIFT 0x10
+#define AFMT_GENERIC_4__AFMT_GENERIC_BYTE19_MASK 0xff000000
+#define AFMT_GENERIC_4__AFMT_GENERIC_BYTE19__SHIFT 0x18
+#define AFMT_GENERIC_5__AFMT_GENERIC_BYTE20_MASK 0xff
+#define AFMT_GENERIC_5__AFMT_GENERIC_BYTE20__SHIFT 0x0
+#define AFMT_GENERIC_5__AFMT_GENERIC_BYTE21_MASK 0xff00
+#define AFMT_GENERIC_5__AFMT_GENERIC_BYTE21__SHIFT 0x8
+#define AFMT_GENERIC_5__AFMT_GENERIC_BYTE22_MASK 0xff0000
+#define AFMT_GENERIC_5__AFMT_GENERIC_BYTE22__SHIFT 0x10
+#define AFMT_GENERIC_5__AFMT_GENERIC_BYTE23_MASK 0xff000000
+#define AFMT_GENERIC_5__AFMT_GENERIC_BYTE23__SHIFT 0x18
+#define AFMT_GENERIC_6__AFMT_GENERIC_BYTE24_MASK 0xff
+#define AFMT_GENERIC_6__AFMT_GENERIC_BYTE24__SHIFT 0x0
+#define AFMT_GENERIC_6__AFMT_GENERIC_BYTE25_MASK 0xff00
+#define AFMT_GENERIC_6__AFMT_GENERIC_BYTE25__SHIFT 0x8
+#define AFMT_GENERIC_6__AFMT_GENERIC_BYTE26_MASK 0xff0000
+#define AFMT_GENERIC_6__AFMT_GENERIC_BYTE26__SHIFT 0x10
+#define AFMT_GENERIC_6__AFMT_GENERIC_BYTE27_MASK 0xff000000
+#define AFMT_GENERIC_6__AFMT_GENERIC_BYTE27__SHIFT 0x18
+#define AFMT_GENERIC_7__AFMT_GENERIC_BYTE28_MASK 0xff
+#define AFMT_GENERIC_7__AFMT_GENERIC_BYTE28__SHIFT 0x0
+#define AFMT_GENERIC_7__AFMT_GENERIC_BYTE29_MASK 0xff00
+#define AFMT_GENERIC_7__AFMT_GENERIC_BYTE29__SHIFT 0x8
+#define AFMT_GENERIC_7__AFMT_GENERIC_BYTE30_MASK 0xff0000
+#define AFMT_GENERIC_7__AFMT_GENERIC_BYTE30__SHIFT 0x10
+#define AFMT_GENERIC_7__AFMT_GENERIC_BYTE31_MASK 0xff000000
+#define AFMT_GENERIC_7__AFMT_GENERIC_BYTE31__SHIFT 0x18
+#define HDMI_GENERIC_PACKET_CONTROL1__HDMI_GENERIC2_SEND_MASK 0x1
+#define HDMI_GENERIC_PACKET_CONTROL1__HDMI_GENERIC2_SEND__SHIFT 0x0
+#define HDMI_GENERIC_PACKET_CONTROL1__HDMI_GENERIC2_CONT_MASK 0x2
+#define HDMI_GENERIC_PACKET_CONTROL1__HDMI_GENERIC2_CONT__SHIFT 0x1
+#define HDMI_GENERIC_PACKET_CONTROL1__HDMI_GENERIC3_SEND_MASK 0x10
+#define HDMI_GENERIC_PACKET_CONTROL1__HDMI_GENERIC3_SEND__SHIFT 0x4
+#define HDMI_GENERIC_PACKET_CONTROL1__HDMI_GENERIC3_CONT_MASK 0x20
+#define HDMI_GENERIC_PACKET_CONTROL1__HDMI_GENERIC3_CONT__SHIFT 0x5
+#define HDMI_GENERIC_PACKET_CONTROL1__HDMI_GENERIC2_LINE_MASK 0x3f0000
+#define HDMI_GENERIC_PACKET_CONTROL1__HDMI_GENERIC2_LINE__SHIFT 0x10
+#define HDMI_GENERIC_PACKET_CONTROL1__HDMI_GENERIC3_LINE_MASK 0x3f000000
+#define HDMI_GENERIC_PACKET_CONTROL1__HDMI_GENERIC3_LINE__SHIFT 0x18
+#define HDMI_ACR_32_0__HDMI_ACR_CTS_32_MASK 0xfffff000
+#define HDMI_ACR_32_0__HDMI_ACR_CTS_32__SHIFT 0xc
+#define HDMI_ACR_32_1__HDMI_ACR_N_32_MASK 0xfffff
+#define HDMI_ACR_32_1__HDMI_ACR_N_32__SHIFT 0x0
+#define HDMI_ACR_44_0__HDMI_ACR_CTS_44_MASK 0xfffff000
+#define HDMI_ACR_44_0__HDMI_ACR_CTS_44__SHIFT 0xc
+#define HDMI_ACR_44_1__HDMI_ACR_N_44_MASK 0xfffff
+#define HDMI_ACR_44_1__HDMI_ACR_N_44__SHIFT 0x0
+#define HDMI_ACR_48_0__HDMI_ACR_CTS_48_MASK 0xfffff000
+#define HDMI_ACR_48_0__HDMI_ACR_CTS_48__SHIFT 0xc
+#define HDMI_ACR_48_1__HDMI_ACR_N_48_MASK 0xfffff
+#define HDMI_ACR_48_1__HDMI_ACR_N_48__SHIFT 0x0
+#define HDMI_ACR_STATUS_0__HDMI_ACR_CTS_MASK 0xfffff000
+#define HDMI_ACR_STATUS_0__HDMI_ACR_CTS__SHIFT 0xc
+#define HDMI_ACR_STATUS_1__HDMI_ACR_N_MASK 0xfffff
+#define HDMI_ACR_STATUS_1__HDMI_ACR_N__SHIFT 0x0
+#define AFMT_AUDIO_INFO0__AFMT_AUDIO_INFO_CHECKSUM_MASK 0xff
+#define AFMT_AUDIO_INFO0__AFMT_AUDIO_INFO_CHECKSUM__SHIFT 0x0
+#define AFMT_AUDIO_INFO0__AFMT_AUDIO_INFO_CC_MASK 0x700
+#define AFMT_AUDIO_INFO0__AFMT_AUDIO_INFO_CC__SHIFT 0x8
+#define AFMT_AUDIO_INFO0__AFMT_AUDIO_INFO_CT_MASK 0x7800
+#define AFMT_AUDIO_INFO0__AFMT_AUDIO_INFO_CT__SHIFT 0xb
+#define AFMT_AUDIO_INFO0__AFMT_AUDIO_INFO_CHECKSUM_OFFSET_MASK 0xff0000
+#define AFMT_AUDIO_INFO0__AFMT_AUDIO_INFO_CHECKSUM_OFFSET__SHIFT 0x10
+#define AFMT_AUDIO_INFO0__AFMT_AUDIO_INFO_CXT_MASK 0x1f000000
+#define AFMT_AUDIO_INFO0__AFMT_AUDIO_INFO_CXT__SHIFT 0x18
+#define AFMT_AUDIO_INFO1__AFMT_AUDIO_INFO_CA_MASK 0xff
+#define AFMT_AUDIO_INFO1__AFMT_AUDIO_INFO_CA__SHIFT 0x0
+#define AFMT_AUDIO_INFO1__AFMT_AUDIO_INFO_LSV_MASK 0x7800
+#define AFMT_AUDIO_INFO1__AFMT_AUDIO_INFO_LSV__SHIFT 0xb
+#define AFMT_AUDIO_INFO1__AFMT_AUDIO_INFO_DM_INH_MASK 0x8000
+#define AFMT_AUDIO_INFO1__AFMT_AUDIO_INFO_DM_INH__SHIFT 0xf
+#define AFMT_AUDIO_INFO1__AFMT_AUDIO_INFO_LFEPBL_MASK 0x30000
+#define AFMT_AUDIO_INFO1__AFMT_AUDIO_INFO_LFEPBL__SHIFT 0x10
+#define AFMT_60958_0__AFMT_60958_CS_A_MASK 0x1
+#define AFMT_60958_0__AFMT_60958_CS_A__SHIFT 0x0
+#define AFMT_60958_0__AFMT_60958_CS_B_MASK 0x2
+#define AFMT_60958_0__AFMT_60958_CS_B__SHIFT 0x1
+#define AFMT_60958_0__AFMT_60958_CS_C_MASK 0x4
+#define AFMT_60958_0__AFMT_60958_CS_C__SHIFT 0x2
+#define AFMT_60958_0__AFMT_60958_CS_D_MASK 0x38
+#define AFMT_60958_0__AFMT_60958_CS_D__SHIFT 0x3
+#define AFMT_60958_0__AFMT_60958_CS_MODE_MASK 0xc0
+#define AFMT_60958_0__AFMT_60958_CS_MODE__SHIFT 0x6
+#define AFMT_60958_0__AFMT_60958_CS_CATEGORY_CODE_MASK 0xff00
+#define AFMT_60958_0__AFMT_60958_CS_CATEGORY_CODE__SHIFT 0x8
+#define AFMT_60958_0__AFMT_60958_CS_SOURCE_NUMBER_MASK 0xf0000
+#define AFMT_60958_0__AFMT_60958_CS_SOURCE_NUMBER__SHIFT 0x10
+#define AFMT_60958_0__AFMT_60958_CS_CHANNEL_NUMBER_L_MASK 0xf00000
+#define AFMT_60958_0__AFMT_60958_CS_CHANNEL_NUMBER_L__SHIFT 0x14
+#define AFMT_60958_0__AFMT_60958_CS_SAMPLING_FREQUENCY_MASK 0xf000000
+#define AFMT_60958_0__AFMT_60958_CS_SAMPLING_FREQUENCY__SHIFT 0x18
+#define AFMT_60958_0__AFMT_60958_CS_CLOCK_ACCURACY_MASK 0x30000000
+#define AFMT_60958_0__AFMT_60958_CS_CLOCK_ACCURACY__SHIFT 0x1c
+#define AFMT_60958_1__AFMT_60958_CS_WORD_LENGTH_MASK 0xf
+#define AFMT_60958_1__AFMT_60958_CS_WORD_LENGTH__SHIFT 0x0
+#define AFMT_60958_1__AFMT_60958_CS_ORIGINAL_SAMPLING_FREQUENCY_MASK 0xf0
+#define AFMT_60958_1__AFMT_60958_CS_ORIGINAL_SAMPLING_FREQUENCY__SHIFT 0x4
+#define AFMT_60958_1__AFMT_60958_VALID_L_MASK 0x10000
+#define AFMT_60958_1__AFMT_60958_VALID_L__SHIFT 0x10
+#define AFMT_60958_1__AFMT_60958_VALID_R_MASK 0x40000
+#define AFMT_60958_1__AFMT_60958_VALID_R__SHIFT 0x12
+#define AFMT_60958_1__AFMT_60958_CS_CHANNEL_NUMBER_R_MASK 0xf00000
+#define AFMT_60958_1__AFMT_60958_CS_CHANNEL_NUMBER_R__SHIFT 0x14
+#define AFMT_AUDIO_CRC_CONTROL__AFMT_AUDIO_CRC_EN_MASK 0x1
+#define AFMT_AUDIO_CRC_CONTROL__AFMT_AUDIO_CRC_EN__SHIFT 0x0
+#define AFMT_AUDIO_CRC_CONTROL__AFMT_AUDIO_CRC_CONT_MASK 0x10
+#define AFMT_AUDIO_CRC_CONTROL__AFMT_AUDIO_CRC_CONT__SHIFT 0x4
+#define AFMT_AUDIO_CRC_CONTROL__AFMT_AUDIO_CRC_SOURCE_MASK 0x100
+#define AFMT_AUDIO_CRC_CONTROL__AFMT_AUDIO_CRC_SOURCE__SHIFT 0x8
+#define AFMT_AUDIO_CRC_CONTROL__AFMT_AUDIO_CRC_CH_SEL_MASK 0xf000
+#define AFMT_AUDIO_CRC_CONTROL__AFMT_AUDIO_CRC_CH_SEL__SHIFT 0xc
+#define AFMT_AUDIO_CRC_CONTROL__AFMT_AUDIO_CRC_COUNT_MASK 0xffff0000
+#define AFMT_AUDIO_CRC_CONTROL__AFMT_AUDIO_CRC_COUNT__SHIFT 0x10
+#define AFMT_RAMP_CONTROL0__AFMT_RAMP_MAX_COUNT_MASK 0xffffff
+#define AFMT_RAMP_CONTROL0__AFMT_RAMP_MAX_COUNT__SHIFT 0x0
+#define AFMT_RAMP_CONTROL0__AFMT_RAMP_DATA_SIGN_MASK 0x80000000
+#define AFMT_RAMP_CONTROL0__AFMT_RAMP_DATA_SIGN__SHIFT 0x1f
+#define AFMT_RAMP_CONTROL1__AFMT_RAMP_MIN_COUNT_MASK 0xffffff
+#define AFMT_RAMP_CONTROL1__AFMT_RAMP_MIN_COUNT__SHIFT 0x0
+#define AFMT_RAMP_CONTROL1__AFMT_AUDIO_TEST_CH_DISABLE_MASK 0xff000000
+#define AFMT_RAMP_CONTROL1__AFMT_AUDIO_TEST_CH_DISABLE__SHIFT 0x18
+#define AFMT_RAMP_CONTROL2__AFMT_RAMP_INC_COUNT_MASK 0xffffff
+#define AFMT_RAMP_CONTROL2__AFMT_RAMP_INC_COUNT__SHIFT 0x0
+#define AFMT_RAMP_CONTROL3__AFMT_RAMP_DEC_COUNT_MASK 0xffffff
+#define AFMT_RAMP_CONTROL3__AFMT_RAMP_DEC_COUNT__SHIFT 0x0
+#define AFMT_60958_2__AFMT_60958_CS_CHANNEL_NUMBER_2_MASK 0xf
+#define AFMT_60958_2__AFMT_60958_CS_CHANNEL_NUMBER_2__SHIFT 0x0
+#define AFMT_60958_2__AFMT_60958_CS_CHANNEL_NUMBER_3_MASK 0xf0
+#define AFMT_60958_2__AFMT_60958_CS_CHANNEL_NUMBER_3__SHIFT 0x4
+#define AFMT_60958_2__AFMT_60958_CS_CHANNEL_NUMBER_4_MASK 0xf00
+#define AFMT_60958_2__AFMT_60958_CS_CHANNEL_NUMBER_4__SHIFT 0x8
+#define AFMT_60958_2__AFMT_60958_CS_CHANNEL_NUMBER_5_MASK 0xf000
+#define AFMT_60958_2__AFMT_60958_CS_CHANNEL_NUMBER_5__SHIFT 0xc
+#define AFMT_60958_2__AFMT_60958_CS_CHANNEL_NUMBER_6_MASK 0xf0000
+#define AFMT_60958_2__AFMT_60958_CS_CHANNEL_NUMBER_6__SHIFT 0x10
+#define AFMT_60958_2__AFMT_60958_CS_CHANNEL_NUMBER_7_MASK 0xf00000
+#define AFMT_60958_2__AFMT_60958_CS_CHANNEL_NUMBER_7__SHIFT 0x14
+#define AFMT_AUDIO_CRC_RESULT__AFMT_AUDIO_CRC_DONE_MASK 0x1
+#define AFMT_AUDIO_CRC_RESULT__AFMT_AUDIO_CRC_DONE__SHIFT 0x0
+#define AFMT_AUDIO_CRC_RESULT__AFMT_AUDIO_CRC_MASK 0xffffff00
+#define AFMT_AUDIO_CRC_RESULT__AFMT_AUDIO_CRC__SHIFT 0x8
+#define AFMT_STATUS__AFMT_AUDIO_ENABLE_MASK 0x10
+#define AFMT_STATUS__AFMT_AUDIO_ENABLE__SHIFT 0x4
+#define AFMT_STATUS__AFMT_AZ_HBR_ENABLE_MASK 0x100
+#define AFMT_STATUS__AFMT_AZ_HBR_ENABLE__SHIFT 0x8
+#define AFMT_STATUS__AFMT_AUDIO_FIFO_OVERFLOW_MASK 0x1000000
+#define AFMT_STATUS__AFMT_AUDIO_FIFO_OVERFLOW__SHIFT 0x18
+#define AFMT_STATUS__AFMT_AZ_AUDIO_ENABLE_CHG_MASK 0x40000000
+#define AFMT_STATUS__AFMT_AZ_AUDIO_ENABLE_CHG__SHIFT 0x1e
+#define AFMT_AUDIO_PACKET_CONTROL__AFMT_AUDIO_SAMPLE_SEND_MASK 0x1
+#define AFMT_AUDIO_PACKET_CONTROL__AFMT_AUDIO_SAMPLE_SEND__SHIFT 0x0
+#define AFMT_AUDIO_PACKET_CONTROL__AFMT_RESET_FIFO_WHEN_AUDIO_DIS_MASK 0x800
+#define AFMT_AUDIO_PACKET_CONTROL__AFMT_RESET_FIFO_WHEN_AUDIO_DIS__SHIFT 0xb
+#define AFMT_AUDIO_PACKET_CONTROL__AFMT_AUDIO_TEST_EN_MASK 0x1000
+#define AFMT_AUDIO_PACKET_CONTROL__AFMT_AUDIO_TEST_EN__SHIFT 0xc
+#define AFMT_AUDIO_PACKET_CONTROL__AFMT_AUDIO_TEST_MODE_MASK 0x4000
+#define AFMT_AUDIO_PACKET_CONTROL__AFMT_AUDIO_TEST_MODE__SHIFT 0xe
+#define AFMT_AUDIO_PACKET_CONTROL__AFMT_AUDIO_FIFO_OVERFLOW_ACK_MASK 0x800000
+#define AFMT_AUDIO_PACKET_CONTROL__AFMT_AUDIO_FIFO_OVERFLOW_ACK__SHIFT 0x17
+#define AFMT_AUDIO_PACKET_CONTROL__AFMT_AUDIO_CHANNEL_SWAP_MASK 0x1000000
+#define AFMT_AUDIO_PACKET_CONTROL__AFMT_AUDIO_CHANNEL_SWAP__SHIFT 0x18
+#define AFMT_AUDIO_PACKET_CONTROL__AFMT_60958_CS_UPDATE_MASK 0x4000000
+#define AFMT_AUDIO_PACKET_CONTROL__AFMT_60958_CS_UPDATE__SHIFT 0x1a
+#define AFMT_AUDIO_PACKET_CONTROL__AFMT_AZ_AUDIO_ENABLE_CHG_ACK_MASK 0x40000000
+#define AFMT_AUDIO_PACKET_CONTROL__AFMT_AZ_AUDIO_ENABLE_CHG_ACK__SHIFT 0x1e
+#define AFMT_AUDIO_PACKET_CONTROL__AFMT_BLANK_TEST_DATA_ON_ENC_ENB_MASK 0x80000000
+#define AFMT_AUDIO_PACKET_CONTROL__AFMT_BLANK_TEST_DATA_ON_ENC_ENB__SHIFT 0x1f
+#define AFMT_VBI_PACKET_CONTROL__AFMT_GENERIC0_UPDATE_MASK 0x4
+#define AFMT_VBI_PACKET_CONTROL__AFMT_GENERIC0_UPDATE__SHIFT 0x2
+#define AFMT_VBI_PACKET_CONTROL__AFMT_GENERIC2_UPDATE_MASK 0x8
+#define AFMT_VBI_PACKET_CONTROL__AFMT_GENERIC2_UPDATE__SHIFT 0x3
+#define AFMT_VBI_PACKET_CONTROL__AFMT_GENERIC_INDEX_MASK 0xc0000000
+#define AFMT_VBI_PACKET_CONTROL__AFMT_GENERIC_INDEX__SHIFT 0x1e
+#define AFMT_INFOFRAME_CONTROL0__AFMT_AUDIO_INFO_SOURCE_MASK 0x40
+#define AFMT_INFOFRAME_CONTROL0__AFMT_AUDIO_INFO_SOURCE__SHIFT 0x6
+#define AFMT_INFOFRAME_CONTROL0__AFMT_AUDIO_INFO_UPDATE_MASK 0x80
+#define AFMT_INFOFRAME_CONTROL0__AFMT_AUDIO_INFO_UPDATE__SHIFT 0x7
+#define AFMT_INFOFRAME_CONTROL0__AFMT_MPEG_INFO_UPDATE_MASK 0x400
+#define AFMT_INFOFRAME_CONTROL0__AFMT_MPEG_INFO_UPDATE__SHIFT 0xa
+#define AFMT_AUDIO_SRC_CONTROL__AFMT_AUDIO_SRC_SELECT_MASK 0x7
+#define AFMT_AUDIO_SRC_CONTROL__AFMT_AUDIO_SRC_SELECT__SHIFT 0x0
+#define AFMT_AUDIO_DBG_DTO_CNTL__AFMT_AUDIO_DTO_FS_DIV_SEL_MASK 0x7
+#define AFMT_AUDIO_DBG_DTO_CNTL__AFMT_AUDIO_DTO_FS_DIV_SEL__SHIFT 0x0
+#define AFMT_AUDIO_DBG_DTO_CNTL__AFMT_AUDIO_DTO_DBG_BASE_MASK 0x100
+#define AFMT_AUDIO_DBG_DTO_CNTL__AFMT_AUDIO_DTO_DBG_BASE__SHIFT 0x8
+#define AFMT_AUDIO_DBG_DTO_CNTL__AFMT_AUDIO_DTO_DBG_MULTI_MASK 0x7000
+#define AFMT_AUDIO_DBG_DTO_CNTL__AFMT_AUDIO_DTO_DBG_MULTI__SHIFT 0xc
+#define AFMT_AUDIO_DBG_DTO_CNTL__AFMT_AUDIO_DTO_DBG_DIV_MASK 0x70000
+#define AFMT_AUDIO_DBG_DTO_CNTL__AFMT_AUDIO_DTO_DBG_DIV__SHIFT 0x10
+#define AFMT_CNTL__AFMT_AUDIO_CLOCK_EN_MASK 0x1
+#define AFMT_CNTL__AFMT_AUDIO_CLOCK_EN__SHIFT 0x0
+#define AFMT_CNTL__AFMT_AUDIO_CLOCK_ON_MASK 0x100
+#define AFMT_CNTL__AFMT_AUDIO_CLOCK_ON__SHIFT 0x8
+#define DIG_BE_CNTL__DIG_DUAL_LINK_ENABLE_MASK 0x1
+#define DIG_BE_CNTL__DIG_DUAL_LINK_ENABLE__SHIFT 0x0
+#define DIG_BE_CNTL__DIG_SWAP_MASK 0x2
+#define DIG_BE_CNTL__DIG_SWAP__SHIFT 0x1
+#define DIG_BE_CNTL__DIG_RB_SWITCH_EN_MASK 0x4
+#define DIG_BE_CNTL__DIG_RB_SWITCH_EN__SHIFT 0x2
+#define DIG_BE_CNTL__DIG_FE_SOURCE_SELECT_MASK 0x7f00
+#define DIG_BE_CNTL__DIG_FE_SOURCE_SELECT__SHIFT 0x8
+#define DIG_BE_CNTL__DIG_MODE_MASK 0x70000
+#define DIG_BE_CNTL__DIG_MODE__SHIFT 0x10
+#define DIG_BE_CNTL__DIG_HPD_SELECT_MASK 0x70000000
+#define DIG_BE_CNTL__DIG_HPD_SELECT__SHIFT 0x1c
+#define DIG_BE_EN_CNTL__DIG_ENABLE_MASK 0x1
+#define DIG_BE_EN_CNTL__DIG_ENABLE__SHIFT 0x0
+#define DIG_BE_EN_CNTL__DIG_SYMCLK_BE_ON_MASK 0x100
+#define DIG_BE_EN_CNTL__DIG_SYMCLK_BE_ON__SHIFT 0x8
+#define TMDS_CNTL__TMDS_SYNC_PHASE_MASK 0x1
+#define TMDS_CNTL__TMDS_SYNC_PHASE__SHIFT 0x0
+#define TMDS_CONTROL_CHAR__TMDS_CONTROL_CHAR0_OUT_EN_MASK 0x1
+#define TMDS_CONTROL_CHAR__TMDS_CONTROL_CHAR0_OUT_EN__SHIFT 0x0
+#define TMDS_CONTROL_CHAR__TMDS_CONTROL_CHAR1_OUT_EN_MASK 0x2
+#define TMDS_CONTROL_CHAR__TMDS_CONTROL_CHAR1_OUT_EN__SHIFT 0x1
+#define TMDS_CONTROL_CHAR__TMDS_CONTROL_CHAR2_OUT_EN_MASK 0x4
+#define TMDS_CONTROL_CHAR__TMDS_CONTROL_CHAR2_OUT_EN__SHIFT 0x2
+#define TMDS_CONTROL_CHAR__TMDS_CONTROL_CHAR3_OUT_EN_MASK 0x8
+#define TMDS_CONTROL_CHAR__TMDS_CONTROL_CHAR3_OUT_EN__SHIFT 0x3
+#define TMDS_CONTROL0_FEEDBACK__TMDS_CONTROL0_FEEDBACK_SELECT_MASK 0x3
+#define TMDS_CONTROL0_FEEDBACK__TMDS_CONTROL0_FEEDBACK_SELECT__SHIFT 0x0
+#define TMDS_CONTROL0_FEEDBACK__TMDS_CONTROL0_FEEDBACK_DELAY_MASK 0x300
+#define TMDS_CONTROL0_FEEDBACK__TMDS_CONTROL0_FEEDBACK_DELAY__SHIFT 0x8
+#define TMDS_STEREOSYNC_CTL_SEL__TMDS_STEREOSYNC_CTL_SEL_MASK 0x3
+#define TMDS_STEREOSYNC_CTL_SEL__TMDS_STEREOSYNC_CTL_SEL__SHIFT 0x0
+#define TMDS_SYNC_CHAR_PATTERN_0_1__TMDS_SYNC_CHAR_PATTERN0_MASK 0x3ff
+#define TMDS_SYNC_CHAR_PATTERN_0_1__TMDS_SYNC_CHAR_PATTERN0__SHIFT 0x0
+#define TMDS_SYNC_CHAR_PATTERN_0_1__TMDS_SYNC_CHAR_PATTERN1_MASK 0x3ff0000
+#define TMDS_SYNC_CHAR_PATTERN_0_1__TMDS_SYNC_CHAR_PATTERN1__SHIFT 0x10
+#define TMDS_SYNC_CHAR_PATTERN_2_3__TMDS_SYNC_CHAR_PATTERN2_MASK 0x3ff
+#define TMDS_SYNC_CHAR_PATTERN_2_3__TMDS_SYNC_CHAR_PATTERN2__SHIFT 0x0
+#define TMDS_SYNC_CHAR_PATTERN_2_3__TMDS_SYNC_CHAR_PATTERN3_MASK 0x3ff0000
+#define TMDS_SYNC_CHAR_PATTERN_2_3__TMDS_SYNC_CHAR_PATTERN3__SHIFT 0x10
+#define TMDS_DEBUG__TMDS_DEBUG_EN_MASK 0x1
+#define TMDS_DEBUG__TMDS_DEBUG_EN__SHIFT 0x0
+#define TMDS_DEBUG__TMDS_DEBUG_HSYNC_MASK 0x100
+#define TMDS_DEBUG__TMDS_DEBUG_HSYNC__SHIFT 0x8
+#define TMDS_DEBUG__TMDS_DEBUG_HSYNC_EN_MASK 0x200
+#define TMDS_DEBUG__TMDS_DEBUG_HSYNC_EN__SHIFT 0x9
+#define TMDS_DEBUG__TMDS_DEBUG_VSYNC_MASK 0x10000
+#define TMDS_DEBUG__TMDS_DEBUG_VSYNC__SHIFT 0x10
+#define TMDS_DEBUG__TMDS_DEBUG_VSYNC_EN_MASK 0x20000
+#define TMDS_DEBUG__TMDS_DEBUG_VSYNC_EN__SHIFT 0x11
+#define TMDS_DEBUG__TMDS_DEBUG_DE_MASK 0x1000000
+#define TMDS_DEBUG__TMDS_DEBUG_DE__SHIFT 0x18
+#define TMDS_DEBUG__TMDS_DEBUG_DE_EN_MASK 0x2000000
+#define TMDS_DEBUG__TMDS_DEBUG_DE_EN__SHIFT 0x19
+#define TMDS_CTL_BITS__TMDS_CTL0_MASK 0x1
+#define TMDS_CTL_BITS__TMDS_CTL0__SHIFT 0x0
+#define TMDS_CTL_BITS__TMDS_CTL1_MASK 0x100
+#define TMDS_CTL_BITS__TMDS_CTL1__SHIFT 0x8
+#define TMDS_CTL_BITS__TMDS_CTL2_MASK 0x10000
+#define TMDS_CTL_BITS__TMDS_CTL2__SHIFT 0x10
+#define TMDS_CTL_BITS__TMDS_CTL3_MASK 0x1000000
+#define TMDS_CTL_BITS__TMDS_CTL3__SHIFT 0x18
+#define TMDS_DCBALANCER_CONTROL__TMDS_DCBALANCER_EN_MASK 0x1
+#define TMDS_DCBALANCER_CONTROL__TMDS_DCBALANCER_EN__SHIFT 0x0
+#define TMDS_DCBALANCER_CONTROL__TMDS_SYNC_DCBAL_EN_MASK 0x70
+#define TMDS_DCBALANCER_CONTROL__TMDS_SYNC_DCBAL_EN__SHIFT 0x4
+#define TMDS_DCBALANCER_CONTROL__TMDS_DCBALANCER_TEST_EN_MASK 0x100
+#define TMDS_DCBALANCER_CONTROL__TMDS_DCBALANCER_TEST_EN__SHIFT 0x8
+#define TMDS_DCBALANCER_CONTROL__TMDS_DCBALANCER_TEST_IN_MASK 0xf0000
+#define TMDS_DCBALANCER_CONTROL__TMDS_DCBALANCER_TEST_IN__SHIFT 0x10
+#define TMDS_DCBALANCER_CONTROL__TMDS_DCBALANCER_FORCE_MASK 0x1000000
+#define TMDS_DCBALANCER_CONTROL__TMDS_DCBALANCER_FORCE__SHIFT 0x18
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL0_DATA_SEL_MASK 0xf
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL0_DATA_SEL__SHIFT 0x0
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL0_DATA_DELAY_MASK 0x70
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL0_DATA_DELAY__SHIFT 0x4
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL0_DATA_INVERT_MASK 0x80
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL0_DATA_INVERT__SHIFT 0x7
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL0_DATA_MODULATION_MASK 0x300
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL0_DATA_MODULATION__SHIFT 0x8
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL0_USE_FEEDBACK_PATH_MASK 0x400
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL0_USE_FEEDBACK_PATH__SHIFT 0xa
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL0_FB_SYNC_CONT_MASK 0x800
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL0_FB_SYNC_CONT__SHIFT 0xb
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL0_PATTERN_OUT_EN_MASK 0x1000
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL0_PATTERN_OUT_EN__SHIFT 0xc
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL1_DATA_SEL_MASK 0xf0000
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL1_DATA_SEL__SHIFT 0x10
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL1_DATA_DELAY_MASK 0x700000
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL1_DATA_DELAY__SHIFT 0x14
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL1_DATA_INVERT_MASK 0x800000
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL1_DATA_INVERT__SHIFT 0x17
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL1_DATA_MODULATION_MASK 0x3000000
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL1_DATA_MODULATION__SHIFT 0x18
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL1_USE_FEEDBACK_PATH_MASK 0x4000000
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL1_USE_FEEDBACK_PATH__SHIFT 0x1a
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL1_FB_SYNC_CONT_MASK 0x8000000
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL1_FB_SYNC_CONT__SHIFT 0x1b
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL1_PATTERN_OUT_EN_MASK 0x10000000
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_CTL1_PATTERN_OUT_EN__SHIFT 0x1c
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_2BIT_COUNTER_EN_MASK 0x80000000
+#define TMDS_CTL0_1_GEN_CNTL__TMDS_2BIT_COUNTER_EN__SHIFT 0x1f
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL2_DATA_SEL_MASK 0xf
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL2_DATA_SEL__SHIFT 0x0
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL2_DATA_DELAY_MASK 0x70
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL2_DATA_DELAY__SHIFT 0x4
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL2_DATA_INVERT_MASK 0x80
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL2_DATA_INVERT__SHIFT 0x7
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL2_DATA_MODULATION_MASK 0x300
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL2_DATA_MODULATION__SHIFT 0x8
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL2_USE_FEEDBACK_PATH_MASK 0x400
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL2_USE_FEEDBACK_PATH__SHIFT 0xa
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL2_FB_SYNC_CONT_MASK 0x800
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL2_FB_SYNC_CONT__SHIFT 0xb
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL2_PATTERN_OUT_EN_MASK 0x1000
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL2_PATTERN_OUT_EN__SHIFT 0xc
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL3_DATA_SEL_MASK 0xf0000
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL3_DATA_SEL__SHIFT 0x10
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL3_DATA_DELAY_MASK 0x700000
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL3_DATA_DELAY__SHIFT 0x14
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL3_DATA_INVERT_MASK 0x800000
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL3_DATA_INVERT__SHIFT 0x17
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL3_DATA_MODULATION_MASK 0x3000000
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL3_DATA_MODULATION__SHIFT 0x18
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL3_USE_FEEDBACK_PATH_MASK 0x4000000
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL3_USE_FEEDBACK_PATH__SHIFT 0x1a
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL3_FB_SYNC_CONT_MASK 0x8000000
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL3_FB_SYNC_CONT__SHIFT 0x1b
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL3_PATTERN_OUT_EN_MASK 0x10000000
+#define TMDS_CTL2_3_GEN_CNTL__TMDS_CTL3_PATTERN_OUT_EN__SHIFT 0x1c
+#define DIG_VERSION__DIG_TYPE_MASK 0x1
+#define DIG_VERSION__DIG_TYPE__SHIFT 0x0
+#define DIG_LANE_ENABLE__DIG_LANE0EN_MASK 0x1
+#define DIG_LANE_ENABLE__DIG_LANE0EN__SHIFT 0x0
+#define DIG_LANE_ENABLE__DIG_LANE1EN_MASK 0x2
+#define DIG_LANE_ENABLE__DIG_LANE1EN__SHIFT 0x1
+#define DIG_LANE_ENABLE__DIG_LANE2EN_MASK 0x4
+#define DIG_LANE_ENABLE__DIG_LANE2EN__SHIFT 0x2
+#define DIG_LANE_ENABLE__DIG_LANE3EN_MASK 0x8
+#define DIG_LANE_ENABLE__DIG_LANE3EN__SHIFT 0x3
+#define DIG_LANE_ENABLE__DIG_CLK_EN_MASK 0x100
+#define DIG_LANE_ENABLE__DIG_CLK_EN__SHIFT 0x8
+#define DIG_TEST_DEBUG_INDEX__DIG_TEST_DEBUG_INDEX_MASK 0xff
+#define DIG_TEST_DEBUG_INDEX__DIG_TEST_DEBUG_INDEX__SHIFT 0x0
+#define DIG_TEST_DEBUG_INDEX__DIG_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define DIG_TEST_DEBUG_INDEX__DIG_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define DIG_TEST_DEBUG_DATA__DIG_TEST_DEBUG_DATA_MASK 0xffffffff
+#define DIG_TEST_DEBUG_DATA__DIG_TEST_DEBUG_DATA__SHIFT 0x0
+#define DIG_FE_TEST_DEBUG_INDEX__DIG_FE_TEST_DEBUG_INDEX_MASK 0xff
+#define DIG_FE_TEST_DEBUG_INDEX__DIG_FE_TEST_DEBUG_INDEX__SHIFT 0x0
+#define DIG_FE_TEST_DEBUG_INDEX__DIG_FE_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define DIG_FE_TEST_DEBUG_INDEX__DIG_FE_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define DIG_FE_TEST_DEBUG_DATA__DIG_FE_TEST_DEBUG_DATA_MASK 0xffffffff
+#define DIG_FE_TEST_DEBUG_DATA__DIG_FE_TEST_DEBUG_DATA__SHIFT 0x0
+#define DMCU_CTRL__RESET_UC_MASK 0x1
+#define DMCU_CTRL__RESET_UC__SHIFT 0x0
+#define DMCU_CTRL__IGNORE_PWRMGT_MASK 0x2
+#define DMCU_CTRL__IGNORE_PWRMGT__SHIFT 0x1
+#define DMCU_CTRL__DISABLE_IRQ_TO_UC_MASK 0x4
+#define DMCU_CTRL__DISABLE_IRQ_TO_UC__SHIFT 0x2
+#define DMCU_CTRL__DISABLE_XIRQ_TO_UC_MASK 0x8
+#define DMCU_CTRL__DISABLE_XIRQ_TO_UC__SHIFT 0x3
+#define DMCU_CTRL__DMCU_ENABLE_MASK 0x10
+#define DMCU_CTRL__DMCU_ENABLE__SHIFT 0x4
+#define DMCU_CTRL__DMCU_DYN_CLK_GATING_EN_MASK 0x100
+#define DMCU_CTRL__DMCU_DYN_CLK_GATING_EN__SHIFT 0x8
+#define DMCU_CTRL__UC_REG_RD_TIMEOUT_MASK 0xffff0000
+#define DMCU_CTRL__UC_REG_RD_TIMEOUT__SHIFT 0x10
+#define DMCU_STATUS__UC_IN_RESET_MASK 0x1
+#define DMCU_STATUS__UC_IN_RESET__SHIFT 0x0
+#define DMCU_STATUS__UC_IN_WAIT_MODE_MASK 0x2
+#define DMCU_STATUS__UC_IN_WAIT_MODE__SHIFT 0x1
+#define DMCU_STATUS__UC_IN_STOP_MODE_MASK 0x4
+#define DMCU_STATUS__UC_IN_STOP_MODE__SHIFT 0x2
+#define DMCU_PC_START_ADDR__PC_START_ADDR_LSB_MASK 0xff
+#define DMCU_PC_START_ADDR__PC_START_ADDR_LSB__SHIFT 0x0
+#define DMCU_PC_START_ADDR__PC_START_ADDR_MSB_MASK 0xff00
+#define DMCU_PC_START_ADDR__PC_START_ADDR_MSB__SHIFT 0x8
+#define DMCU_FW_START_ADDR__FW_START_ADDR_LSB_MASK 0xff
+#define DMCU_FW_START_ADDR__FW_START_ADDR_LSB__SHIFT 0x0
+#define DMCU_FW_START_ADDR__FW_START_ADDR_MSB_MASK 0xff00
+#define DMCU_FW_START_ADDR__FW_START_ADDR_MSB__SHIFT 0x8
+#define DMCU_FW_END_ADDR__FW_END_ADDR_LSB_MASK 0xff
+#define DMCU_FW_END_ADDR__FW_END_ADDR_LSB__SHIFT 0x0
+#define DMCU_FW_END_ADDR__FW_END_ADDR_MSB_MASK 0xff00
+#define DMCU_FW_END_ADDR__FW_END_ADDR_MSB__SHIFT 0x8
+#define DMCU_FW_ISR_START_ADDR__FW_ISR_START_ADDR_LSB_MASK 0xff
+#define DMCU_FW_ISR_START_ADDR__FW_ISR_START_ADDR_LSB__SHIFT 0x0
+#define DMCU_FW_ISR_START_ADDR__FW_ISR_START_ADDR_MSB_MASK 0xff00
+#define DMCU_FW_ISR_START_ADDR__FW_ISR_START_ADDR_MSB__SHIFT 0x8
+#define DMCU_FW_CS_HI__FW_CHECKSUM_HI_MASK 0xffffffff
+#define DMCU_FW_CS_HI__FW_CHECKSUM_HI__SHIFT 0x0
+#define DMCU_FW_CS_LO__FW_CHECKSUM_LO_MASK 0xffffffff
+#define DMCU_FW_CS_LO__FW_CHECKSUM_LO__SHIFT 0x0
+#define DMCU_RAM_ACCESS_CTRL__ERAM_WR_ADDR_AUTO_INC_MASK 0x1
+#define DMCU_RAM_ACCESS_CTRL__ERAM_WR_ADDR_AUTO_INC__SHIFT 0x0
+#define DMCU_RAM_ACCESS_CTRL__ERAM_RD_ADDR_AUTO_INC_MASK 0x2
+#define DMCU_RAM_ACCESS_CTRL__ERAM_RD_ADDR_AUTO_INC__SHIFT 0x1
+#define DMCU_RAM_ACCESS_CTRL__IRAM_WR_ADDR_AUTO_INC_MASK 0x4
+#define DMCU_RAM_ACCESS_CTRL__IRAM_WR_ADDR_AUTO_INC__SHIFT 0x2
+#define DMCU_RAM_ACCESS_CTRL__IRAM_RD_ADDR_AUTO_INC_MASK 0x8
+#define DMCU_RAM_ACCESS_CTRL__IRAM_RD_ADDR_AUTO_INC__SHIFT 0x3
+#define DMCU_RAM_ACCESS_CTRL__ERAM_HOST_ACCESS_EN_MASK 0x10
+#define DMCU_RAM_ACCESS_CTRL__ERAM_HOST_ACCESS_EN__SHIFT 0x4
+#define DMCU_RAM_ACCESS_CTRL__IRAM_HOST_ACCESS_EN_MASK 0x20
+#define DMCU_RAM_ACCESS_CTRL__IRAM_HOST_ACCESS_EN__SHIFT 0x5
+#define DMCU_ERAM_WR_CTRL__ERAM_WR_ADDR_MASK 0xffff
+#define DMCU_ERAM_WR_CTRL__ERAM_WR_ADDR__SHIFT 0x0
+#define DMCU_ERAM_WR_CTRL__ERAM_WR_BE_MASK 0xf0000
+#define DMCU_ERAM_WR_CTRL__ERAM_WR_BE__SHIFT 0x10
+#define DMCU_ERAM_WR_CTRL__ERAM_WR_BYTE_MODE_MASK 0x100000
+#define DMCU_ERAM_WR_CTRL__ERAM_WR_BYTE_MODE__SHIFT 0x14
+#define DMCU_ERAM_WR_DATA__ERAM_WR_DATA_MASK 0xffffffff
+#define DMCU_ERAM_WR_DATA__ERAM_WR_DATA__SHIFT 0x0
+#define DMCU_ERAM_RD_CTRL__ERAM_RD_ADDR_MASK 0xffff
+#define DMCU_ERAM_RD_CTRL__ERAM_RD_ADDR__SHIFT 0x0
+#define DMCU_ERAM_RD_CTRL__ERAM_RD_BE_MASK 0xf0000
+#define DMCU_ERAM_RD_CTRL__ERAM_RD_BE__SHIFT 0x10
+#define DMCU_ERAM_RD_CTRL__ERAM_RD_BYTE_MODE_MASK 0x100000
+#define DMCU_ERAM_RD_CTRL__ERAM_RD_BYTE_MODE__SHIFT 0x14
+#define DMCU_ERAM_RD_DATA__ERAM_RD_DATA_MASK 0xffffffff
+#define DMCU_ERAM_RD_DATA__ERAM_RD_DATA__SHIFT 0x0
+#define DMCU_IRAM_WR_CTRL__IRAM_WR_ADDR_MASK 0x3ff
+#define DMCU_IRAM_WR_CTRL__IRAM_WR_ADDR__SHIFT 0x0
+#define DMCU_IRAM_WR_DATA__IRAM_WR_DATA_MASK 0xff
+#define DMCU_IRAM_WR_DATA__IRAM_WR_DATA__SHIFT 0x0
+#define DMCU_IRAM_RD_CTRL__IRAM_RD_ADDR_MASK 0x3ff
+#define DMCU_IRAM_RD_CTRL__IRAM_RD_ADDR__SHIFT 0x0
+#define DMCU_IRAM_RD_DATA__IRAM_RD_DATA_MASK 0xff
+#define DMCU_IRAM_RD_DATA__IRAM_RD_DATA__SHIFT 0x0
+#define DMCU_EVENT_TRIGGER__GEN_SW_INT_TO_UC_MASK 0x1
+#define DMCU_EVENT_TRIGGER__GEN_SW_INT_TO_UC__SHIFT 0x0
+#define DMCU_EVENT_TRIGGER__UC_INTERNAL_INT_CODE_MASK 0x7f0000
+#define DMCU_EVENT_TRIGGER__UC_INTERNAL_INT_CODE__SHIFT 0x10
+#define DMCU_EVENT_TRIGGER__GEN_UC_INTERNAL_INT_TO_HOST_MASK 0x800000
+#define DMCU_EVENT_TRIGGER__GEN_UC_INTERNAL_INT_TO_HOST__SHIFT 0x17
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_IRQ_N_PIN_MASK 0x1
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_IRQ_N_PIN__SHIFT 0x0
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_XIRQ_N_PIN_MASK 0x2
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_XIRQ_N_PIN__SHIFT 0x1
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_SOFTWARE_INTERRUPT_MASK 0x4
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_SOFTWARE_INTERRUPT__SHIFT 0x2
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_ILLEGAL_OPCODE_TRAP_MASK 0x8
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_ILLEGAL_OPCODE_TRAP__SHIFT 0x3
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_TIMER_OUTPUT_COMPARE_4_MASK 0x10
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_TIMER_OUTPUT_COMPARE_4__SHIFT 0x4
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_TIMER_OUTPUT_COMPARE_3_MASK 0x20
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_TIMER_OUTPUT_COMPARE_3__SHIFT 0x5
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_TIMER_OUTPUT_COMPARE_2_MASK 0x40
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_TIMER_OUTPUT_COMPARE_2__SHIFT 0x6
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_TIMER_OUTPUT_COMPARE_1_MASK 0x80
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_TIMER_OUTPUT_COMPARE_1__SHIFT 0x7
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_TIMER_OVERFLOW_MASK 0x100
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_TIMER_OVERFLOW__SHIFT 0x8
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_REAL_TIME_INTERRUPT_MASK 0x200
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_REAL_TIME_INTERRUPT__SHIFT 0x9
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_TIMER_INPUT_CAPTURE_4_OUTPUT_COMPARE_5_MASK 0x400
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_TIMER_INPUT_CAPTURE_4_OUTPUT_COMPARE_5__SHIFT 0xa
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_TIMER_INPUT_CAPTURE_3_MASK 0x800
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_TIMER_INPUT_CAPTURE_3__SHIFT 0xb
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_TIMER_INPUT_CAPTURE_2_MASK 0x1000
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_TIMER_INPUT_CAPTURE_2__SHIFT 0xc
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_TIMER_INPUT_CAPTURE_1_MASK 0x2000
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_TIMER_INPUT_CAPTURE_1__SHIFT 0xd
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_PULSE_ACCUMULATOR_INPUT_EDGE_MASK 0x4000
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_PULSE_ACCUMULATOR_INPUT_EDGE__SHIFT 0xe
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_PULSE_ACCUMULATOR_OVERFLOW_MASK 0x8000
+#define DMCU_UC_INTERNAL_INT_STATUS__UC_INT_PULSE_ACCUMULATOR_OVERFLOW__SHIFT 0xf
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN1_INT_STATUS_MASK 0x2000
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN1_INT_STATUS__SHIFT 0xd
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN1_INT_OCCURRED_MASK 0x4000
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN1_INT_OCCURRED__SHIFT 0xe
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN1_INT_CLEAR_MASK 0x4000
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN1_INT_CLEAR__SHIFT 0xe
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN2_INT_STATUS_MASK 0x8000
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN2_INT_STATUS__SHIFT 0xf
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN2_INT_OCCURRED_MASK 0x10000
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN2_INT_OCCURRED__SHIFT 0x10
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN2_INT_CLEAR_MASK 0x10000
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN2_INT_CLEAR__SHIFT 0x10
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN3_INT_STATUS_MASK 0x20000
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN3_INT_STATUS__SHIFT 0x11
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN3_INT_OCCURRED_MASK 0x40000
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN3_INT_OCCURRED__SHIFT 0x12
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN3_INT_CLEAR_MASK 0x40000
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN3_INT_CLEAR__SHIFT 0x12
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN4_INT_STATUS_MASK 0x80000
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN4_INT_STATUS__SHIFT 0x13
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN4_INT_OCCURRED_MASK 0x100000
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN4_INT_OCCURRED__SHIFT 0x14
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN4_INT_CLEAR_MASK 0x100000
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN4_INT_CLEAR__SHIFT 0x14
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN5_INT_STATUS_MASK 0x200000
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN5_INT_STATUS__SHIFT 0x15
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN5_INT_OCCURRED_MASK 0x400000
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN5_INT_OCCURRED__SHIFT 0x16
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN5_INT_CLEAR_MASK 0x400000
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN5_INT_CLEAR__SHIFT 0x16
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN6_INT_STATUS_MASK 0x800000
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN6_INT_STATUS__SHIFT 0x17
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN6_INT_OCCURRED_MASK 0x1000000
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN6_INT_OCCURRED__SHIFT 0x18
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN6_INT_CLEAR_MASK 0x1000000
+#define DMCU_SS_INTERRUPT_CNTL_STATUS__STATIC_SCREEN6_INT_CLEAR__SHIFT 0x18
+#define DMCU_INTERRUPT_STATUS__ABM1_HG_READY_INT_OCCURRED_MASK 0x1
+#define DMCU_INTERRUPT_STATUS__ABM1_HG_READY_INT_OCCURRED__SHIFT 0x0
+#define DMCU_INTERRUPT_STATUS__ABM1_HG_READY_INT_CLEAR_MASK 0x1
+#define DMCU_INTERRUPT_STATUS__ABM1_HG_READY_INT_CLEAR__SHIFT 0x0
+#define DMCU_INTERRUPT_STATUS__ABM1_LS_READY_INT_OCCURRED_MASK 0x2
+#define DMCU_INTERRUPT_STATUS__ABM1_LS_READY_INT_OCCURRED__SHIFT 0x1
+#define DMCU_INTERRUPT_STATUS__ABM1_LS_READY_INT_CLEAR_MASK 0x2
+#define DMCU_INTERRUPT_STATUS__ABM1_LS_READY_INT_CLEAR__SHIFT 0x1
+#define DMCU_INTERRUPT_STATUS__ABM1_BL_UPDATE_INT_OCCURRED_MASK 0x4
+#define DMCU_INTERRUPT_STATUS__ABM1_BL_UPDATE_INT_OCCURRED__SHIFT 0x2
+#define DMCU_INTERRUPT_STATUS__ABM1_BL_UPDATE_INT_CLEAR_MASK 0x4
+#define DMCU_INTERRUPT_STATUS__ABM1_BL_UPDATE_INT_CLEAR__SHIFT 0x2
+#define DMCU_INTERRUPT_STATUS__MCP_INT_OCCURRED_MASK 0x8
+#define DMCU_INTERRUPT_STATUS__MCP_INT_OCCURRED__SHIFT 0x3
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DSI_POWER_UP_INT_OCCURRED_MASK 0x10
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DSI_POWER_UP_INT_OCCURRED__SHIFT 0x4
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DSI_POWER_UP_INT_CLEAR_MASK 0x10
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DSI_POWER_UP_INT_CLEAR__SHIFT 0x4
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DSI_POWER_DOWN_INT_OCCURRED_MASK 0x20
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DSI_POWER_DOWN_INT_OCCURRED__SHIFT 0x5
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DSI_POWER_DOWN_INT_CLEAR_MASK 0x20
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DSI_POWER_DOWN_INT_CLEAR__SHIFT 0x5
+#define DMCU_INTERRUPT_STATUS__EXTERNAL_SW_INT_OCCURRED_MASK 0x100
+#define DMCU_INTERRUPT_STATUS__EXTERNAL_SW_INT_OCCURRED__SHIFT 0x8
+#define DMCU_INTERRUPT_STATUS__EXTERNAL_SW_INT_CLEAR_MASK 0x100
+#define DMCU_INTERRUPT_STATUS__EXTERNAL_SW_INT_CLEAR__SHIFT 0x8
+#define DMCU_INTERRUPT_STATUS__SCP_INT_OCCURRED_MASK 0x200
+#define DMCU_INTERRUPT_STATUS__SCP_INT_OCCURRED__SHIFT 0x9
+#define DMCU_INTERRUPT_STATUS__UC_INTERNAL_INT_OCCURRED_MASK 0x400
+#define DMCU_INTERRUPT_STATUS__UC_INTERNAL_INT_OCCURRED__SHIFT 0xa
+#define DMCU_INTERRUPT_STATUS__UC_INTERNAL_INT_CLEAR_MASK 0x400
+#define DMCU_INTERRUPT_STATUS__UC_INTERNAL_INT_CLEAR__SHIFT 0xa
+#define DMCU_INTERRUPT_STATUS__UC_REG_RD_TIMEOUT_INT_OCCURRED_MASK 0x800
+#define DMCU_INTERRUPT_STATUS__UC_REG_RD_TIMEOUT_INT_OCCURRED__SHIFT 0xb
+#define DMCU_INTERRUPT_STATUS__UC_REG_RD_TIMEOUT_INT_CLEAR_MASK 0x800
+#define DMCU_INTERRUPT_STATUS__UC_REG_RD_TIMEOUT_INT_CLEAR__SHIFT 0xb
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE0_POWER_UP_INT_OCCURRED_MASK 0x1000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE0_POWER_UP_INT_OCCURRED__SHIFT 0xc
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE0_POWER_UP_INT_CLEAR_MASK 0x1000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE0_POWER_UP_INT_CLEAR__SHIFT 0xc
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE1_POWER_UP_INT_OCCURRED_MASK 0x2000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE1_POWER_UP_INT_OCCURRED__SHIFT 0xd
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE1_POWER_UP_INT_CLEAR_MASK 0x2000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE1_POWER_UP_INT_CLEAR__SHIFT 0xd
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE2_POWER_UP_INT_OCCURRED_MASK 0x4000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE2_POWER_UP_INT_OCCURRED__SHIFT 0xe
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE2_POWER_UP_INT_CLEAR_MASK 0x4000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE2_POWER_UP_INT_CLEAR__SHIFT 0xe
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE3_POWER_UP_INT_OCCURRED_MASK 0x8000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE3_POWER_UP_INT_OCCURRED__SHIFT 0xf
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE3_POWER_UP_INT_CLEAR_MASK 0x8000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE3_POWER_UP_INT_CLEAR__SHIFT 0xf
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE4_POWER_UP_INT_OCCURRED_MASK 0x10000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE4_POWER_UP_INT_OCCURRED__SHIFT 0x10
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE4_POWER_UP_INT_CLEAR_MASK 0x10000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE4_POWER_UP_INT_CLEAR__SHIFT 0x10
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE5_POWER_UP_INT_OCCURRED_MASK 0x20000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE5_POWER_UP_INT_OCCURRED__SHIFT 0x11
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE5_POWER_UP_INT_CLEAR_MASK 0x20000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE5_POWER_UP_INT_CLEAR__SHIFT 0x11
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE0_POWER_DOWN_INT_OCCURRED_MASK 0x40000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE0_POWER_DOWN_INT_OCCURRED__SHIFT 0x12
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE0_POWER_DOWN_INT_CLEAR_MASK 0x40000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE0_POWER_DOWN_INT_CLEAR__SHIFT 0x12
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE1_POWER_DOWN_INT_OCCURRED_MASK 0x80000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE1_POWER_DOWN_INT_OCCURRED__SHIFT 0x13
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE1_POWER_DOWN_INT_CLEAR_MASK 0x80000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE1_POWER_DOWN_INT_CLEAR__SHIFT 0x13
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE2_POWER_DOWN_INT_OCCURRED_MASK 0x100000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE2_POWER_DOWN_INT_OCCURRED__SHIFT 0x14
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE2_POWER_DOWN_INT_CLEAR_MASK 0x100000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE2_POWER_DOWN_INT_CLEAR__SHIFT 0x14
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE3_POWER_DOWN_INT_OCCURRED_MASK 0x200000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE3_POWER_DOWN_INT_OCCURRED__SHIFT 0x15
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE3_POWER_DOWN_INT_CLEAR_MASK 0x200000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE3_POWER_DOWN_INT_CLEAR__SHIFT 0x15
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE4_POWER_DOWN_INT_OCCURRED_MASK 0x400000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE4_POWER_DOWN_INT_OCCURRED__SHIFT 0x16
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE4_POWER_DOWN_INT_CLEAR_MASK 0x400000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE4_POWER_DOWN_INT_CLEAR__SHIFT 0x16
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE5_POWER_DOWN_INT_OCCURRED_MASK 0x800000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE5_POWER_DOWN_INT_OCCURRED__SHIFT 0x17
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE5_POWER_DOWN_INT_CLEAR_MASK 0x800000
+#define DMCU_INTERRUPT_STATUS__DCPG_IHC_DCFE5_POWER_DOWN_INT_CLEAR__SHIFT 0x17
+#define DMCU_INTERRUPT_STATUS__VBLANK1_INT_OCCURRED_MASK 0x1000000
+#define DMCU_INTERRUPT_STATUS__VBLANK1_INT_OCCURRED__SHIFT 0x18
+#define DMCU_INTERRUPT_STATUS__VBLANK1_INT_CLEAR_MASK 0x1000000
+#define DMCU_INTERRUPT_STATUS__VBLANK1_INT_CLEAR__SHIFT 0x18
+#define DMCU_INTERRUPT_STATUS__VBLANK2_INT_OCCURRED_MASK 0x2000000
+#define DMCU_INTERRUPT_STATUS__VBLANK2_INT_OCCURRED__SHIFT 0x19
+#define DMCU_INTERRUPT_STATUS__VBLANK2_INT_CLEAR_MASK 0x2000000
+#define DMCU_INTERRUPT_STATUS__VBLANK2_INT_CLEAR__SHIFT 0x19
+#define DMCU_INTERRUPT_STATUS__VBLANK3_INT_OCCURRED_MASK 0x4000000
+#define DMCU_INTERRUPT_STATUS__VBLANK3_INT_OCCURRED__SHIFT 0x1a
+#define DMCU_INTERRUPT_STATUS__VBLANK3_INT_CLEAR_MASK 0x4000000
+#define DMCU_INTERRUPT_STATUS__VBLANK3_INT_CLEAR__SHIFT 0x1a
+#define DMCU_INTERRUPT_STATUS__VBLANK4_INT_OCCURRED_MASK 0x8000000
+#define DMCU_INTERRUPT_STATUS__VBLANK4_INT_OCCURRED__SHIFT 0x1b
+#define DMCU_INTERRUPT_STATUS__VBLANK4_INT_CLEAR_MASK 0x8000000
+#define DMCU_INTERRUPT_STATUS__VBLANK4_INT_CLEAR__SHIFT 0x1b
+#define DMCU_INTERRUPT_STATUS__VBLANK5_INT_OCCURRED_MASK 0x10000000
+#define DMCU_INTERRUPT_STATUS__VBLANK5_INT_OCCURRED__SHIFT 0x1c
+#define DMCU_INTERRUPT_STATUS__VBLANK5_INT_CLEAR_MASK 0x10000000
+#define DMCU_INTERRUPT_STATUS__VBLANK5_INT_CLEAR__SHIFT 0x1c
+#define DMCU_INTERRUPT_STATUS__VBLANK6_INT_OCCURRED_MASK 0x20000000
+#define DMCU_INTERRUPT_STATUS__VBLANK6_INT_OCCURRED__SHIFT 0x1d
+#define DMCU_INTERRUPT_STATUS__VBLANK6_INT_CLEAR_MASK 0x20000000
+#define DMCU_INTERRUPT_STATUS__VBLANK6_INT_CLEAR__SHIFT 0x1d
+#define DMCU_INTERRUPT_STATUS_1__DCPG_IHC_DCFEV0_POWER_UP_INT_OCCURRED_MASK 0x1
+#define DMCU_INTERRUPT_STATUS_1__DCPG_IHC_DCFEV0_POWER_UP_INT_OCCURRED__SHIFT 0x0
+#define DMCU_INTERRUPT_STATUS_1__DCPG_IHC_DCFEV0_POWER_UP_INT_CLEAR_MASK 0x1
+#define DMCU_INTERRUPT_STATUS_1__DCPG_IHC_DCFEV0_POWER_UP_INT_CLEAR__SHIFT 0x0
+#define DMCU_INTERRUPT_STATUS_1__DCPG_IHC_DCFEV1_POWER_UP_INT_OCCURRED_MASK 0x2
+#define DMCU_INTERRUPT_STATUS_1__DCPG_IHC_DCFEV1_POWER_UP_INT_OCCURRED__SHIFT 0x1
+#define DMCU_INTERRUPT_STATUS_1__DCPG_IHC_DCFEV1_POWER_UP_INT_CLEAR_MASK 0x2
+#define DMCU_INTERRUPT_STATUS_1__DCPG_IHC_DCFEV1_POWER_UP_INT_CLEAR__SHIFT 0x1
+#define DMCU_INTERRUPT_STATUS_1__DCPG_IHC_DCFEV0_POWER_DOWN_INT_OCCURRED_MASK 0x4
+#define DMCU_INTERRUPT_STATUS_1__DCPG_IHC_DCFEV0_POWER_DOWN_INT_OCCURRED__SHIFT 0x2
+#define DMCU_INTERRUPT_STATUS_1__DCPG_IHC_DCFEV0_POWER_DOWN_INT_CLEAR_MASK 0x4
+#define DMCU_INTERRUPT_STATUS_1__DCPG_IHC_DCFEV0_POWER_DOWN_INT_CLEAR__SHIFT 0x2
+#define DMCU_INTERRUPT_STATUS_1__DCPG_IHC_DCFEV1_POWER_DOWN_INT_OCCURRED_MASK 0x8
+#define DMCU_INTERRUPT_STATUS_1__DCPG_IHC_DCFEV1_POWER_DOWN_INT_OCCURRED__SHIFT 0x3
+#define DMCU_INTERRUPT_STATUS_1__DCPG_IHC_DCFEV1_POWER_DOWN_INT_CLEAR_MASK 0x8
+#define DMCU_INTERRUPT_STATUS_1__DCPG_IHC_DCFEV1_POWER_DOWN_INT_CLEAR__SHIFT 0x3
+#define DMCU_INTERRUPT_STATUS_1__DCFEV0_VBLANK_INT_OCCURRED_MASK 0x10
+#define DMCU_INTERRUPT_STATUS_1__DCFEV0_VBLANK_INT_OCCURRED__SHIFT 0x4
+#define DMCU_INTERRUPT_STATUS_1__DCFEV0_VBLANK_INT_CLEAR_MASK 0x10
+#define DMCU_INTERRUPT_STATUS_1__DCFEV0_VBLANK_INT_CLEAR__SHIFT 0x4
+#define DMCU_INTERRUPT_STATUS_1__DCFEV1_VBLANK_INT_OCCURRED_MASK 0x20
+#define DMCU_INTERRUPT_STATUS_1__DCFEV1_VBLANK_INT_OCCURRED__SHIFT 0x5
+#define DMCU_INTERRUPT_STATUS_1__DCFEV1_VBLANK_INT_CLEAR_MASK 0x20
+#define DMCU_INTERRUPT_STATUS_1__DCFEV1_VBLANK_INT_CLEAR__SHIFT 0x5
+#define DMCU_INTERRUPT_STATUS_1__DMCU_GENERIC_INTERRUPT_OCCURRED_MASK 0x2000
+#define DMCU_INTERRUPT_STATUS_1__DMCU_GENERIC_INTERRUPT_OCCURRED__SHIFT 0xd
+#define DMCU_INTERRUPT_STATUS_1__DMCU_GENERIC_INTERRUPT_CLEAR_MASK 0x2000
+#define DMCU_INTERRUPT_STATUS_1__DMCU_GENERIC_INTERRUPT_CLEAR__SHIFT 0xd
+#define DMCU_INTERRUPT_TO_HOST_EN_MASK__ABM1_HG_READY_INT_MASK_MASK 0x1
+#define DMCU_INTERRUPT_TO_HOST_EN_MASK__ABM1_HG_READY_INT_MASK__SHIFT 0x0
+#define DMCU_INTERRUPT_TO_HOST_EN_MASK__ABM1_LS_READY_INT_MASK_MASK 0x2
+#define DMCU_INTERRUPT_TO_HOST_EN_MASK__ABM1_LS_READY_INT_MASK__SHIFT 0x1
+#define DMCU_INTERRUPT_TO_HOST_EN_MASK__ABM1_BL_UPDATE_INT_MASK_MASK 0x4
+#define DMCU_INTERRUPT_TO_HOST_EN_MASK__ABM1_BL_UPDATE_INT_MASK__SHIFT 0x2
+#define DMCU_INTERRUPT_TO_HOST_EN_MASK__SCP_INT_MASK_MASK 0x200
+#define DMCU_INTERRUPT_TO_HOST_EN_MASK__SCP_INT_MASK__SHIFT 0x9
+#define DMCU_INTERRUPT_TO_HOST_EN_MASK__UC_INTERNAL_INT_MASK_MASK 0x400
+#define DMCU_INTERRUPT_TO_HOST_EN_MASK__UC_INTERNAL_INT_MASK__SHIFT 0xa
+#define DMCU_INTERRUPT_TO_HOST_EN_MASK__UC_REG_RD_TIMEOUT_INT_MASK_MASK 0x800
+#define DMCU_INTERRUPT_TO_HOST_EN_MASK__UC_REG_RD_TIMEOUT_INT_MASK__SHIFT 0xb
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__ABM1_HG_READY_INT_TO_UC_EN_MASK 0x1
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__ABM1_HG_READY_INT_TO_UC_EN__SHIFT 0x0
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__ABM1_LS_READY_INT_TO_UC_EN_MASK 0x2
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__ABM1_LS_READY_INT_TO_UC_EN__SHIFT 0x1
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__ABM1_BL_UPDATE_INT_TO_UC_EN_MASK 0x4
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__ABM1_BL_UPDATE_INT_TO_UC_EN__SHIFT 0x2
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__MCP_INT_TO_UC_EN_MASK 0x8
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__MCP_INT_TO_UC_EN__SHIFT 0x3
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DSI_POWER_UP_INT_TO_UC_EN_MASK 0x10
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DSI_POWER_UP_INT_TO_UC_EN__SHIFT 0x4
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DSI_POWER_DOWN_INT_TO_UC_EN_MASK 0x20
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DSI_POWER_DOWN_INT_TO_UC_EN__SHIFT 0x5
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__STATIC_SCREEN1_INT_TO_UC_EN_MASK 0x40
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__STATIC_SCREEN1_INT_TO_UC_EN__SHIFT 0x6
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__STATIC_SCREEN2_INT_TO_UC_EN_MASK 0x80
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__STATIC_SCREEN2_INT_TO_UC_EN__SHIFT 0x7
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__EXTERNAL_SW_INT_TO_UC_EN_MASK 0x100
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__EXTERNAL_SW_INT_TO_UC_EN__SHIFT 0x8
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__STATIC_SCREEN3_INT_TO_UC_EN_MASK 0x200
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__STATIC_SCREEN3_INT_TO_UC_EN__SHIFT 0x9
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__STATIC_SCREEN4_INT_TO_UC_EN_MASK 0x400
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__STATIC_SCREEN4_INT_TO_UC_EN__SHIFT 0xa
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__STATIC_SCREEN5_INT_TO_UC_EN_MASK 0x800
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__STATIC_SCREEN5_INT_TO_UC_EN__SHIFT 0xb
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE0_POWER_UP_INT_TO_UC_EN_MASK 0x1000
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE0_POWER_UP_INT_TO_UC_EN__SHIFT 0xc
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE1_POWER_UP_INT_TO_UC_EN_MASK 0x2000
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE1_POWER_UP_INT_TO_UC_EN__SHIFT 0xd
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE2_POWER_UP_INT_TO_UC_EN_MASK 0x4000
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE2_POWER_UP_INT_TO_UC_EN__SHIFT 0xe
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE3_POWER_UP_INT_TO_UC_EN_MASK 0x8000
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE3_POWER_UP_INT_TO_UC_EN__SHIFT 0xf
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE4_POWER_UP_INT_TO_UC_EN_MASK 0x10000
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE4_POWER_UP_INT_TO_UC_EN__SHIFT 0x10
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE5_POWER_UP_INT_TO_UC_EN_MASK 0x20000
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE5_POWER_UP_INT_TO_UC_EN__SHIFT 0x11
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE0_POWER_DOWN_INT_TO_UC_EN_MASK 0x40000
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE0_POWER_DOWN_INT_TO_UC_EN__SHIFT 0x12
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE1_POWER_DOWN_INT_TO_UC_EN_MASK 0x80000
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE1_POWER_DOWN_INT_TO_UC_EN__SHIFT 0x13
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE2_POWER_DOWN_INT_TO_UC_EN_MASK 0x100000
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE2_POWER_DOWN_INT_TO_UC_EN__SHIFT 0x14
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE3_POWER_DOWN_INT_TO_UC_EN_MASK 0x200000
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE3_POWER_DOWN_INT_TO_UC_EN__SHIFT 0x15
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE4_POWER_DOWN_INT_TO_UC_EN_MASK 0x400000
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE4_POWER_DOWN_INT_TO_UC_EN__SHIFT 0x16
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE5_POWER_DOWN_INT_TO_UC_EN_MASK 0x800000
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__DCPG_IHC_DCFE5_POWER_DOWN_INT_TO_UC_EN__SHIFT 0x17
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__VBLANK1_INT_TO_UC_EN_MASK 0x1000000
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__VBLANK1_INT_TO_UC_EN__SHIFT 0x18
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__VBLANK2_INT_TO_UC_EN_MASK 0x2000000
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__VBLANK2_INT_TO_UC_EN__SHIFT 0x19
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__VBLANK3_INT_TO_UC_EN_MASK 0x4000000
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__VBLANK3_INT_TO_UC_EN__SHIFT 0x1a
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__VBLANK4_INT_TO_UC_EN_MASK 0x8000000
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__VBLANK4_INT_TO_UC_EN__SHIFT 0x1b
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__VBLANK5_INT_TO_UC_EN_MASK 0x10000000
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__VBLANK5_INT_TO_UC_EN__SHIFT 0x1c
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__VBLANK6_INT_TO_UC_EN_MASK 0x20000000
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__VBLANK6_INT_TO_UC_EN__SHIFT 0x1d
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__STATIC_SCREEN6_INT_TO_UC_EN_MASK 0x40000000
+#define DMCU_INTERRUPT_TO_UC_EN_MASK__STATIC_SCREEN6_INT_TO_UC_EN__SHIFT 0x1e
+#define DMCU_INTERRUPT_TO_UC_EN_MASK_1__DCPG_IHC_DCFEV0_POWER_UP_INT_TO_UC_EN_MASK 0x1
+#define DMCU_INTERRUPT_TO_UC_EN_MASK_1__DCPG_IHC_DCFEV0_POWER_UP_INT_TO_UC_EN__SHIFT 0x0
+#define DMCU_INTERRUPT_TO_UC_EN_MASK_1__DCPG_IHC_DCFEV0_POWER_DOWN_INT_TO_UC_EN_MASK 0x2
+#define DMCU_INTERRUPT_TO_UC_EN_MASK_1__DCPG_IHC_DCFEV0_POWER_DOWN_INT_TO_UC_EN__SHIFT 0x1
+#define DMCU_INTERRUPT_TO_UC_EN_MASK_1__DCFEV0_VBLANK_INT_TO_UC_EN_MASK 0x4
+#define DMCU_INTERRUPT_TO_UC_EN_MASK_1__DCFEV0_VBLANK_INT_TO_UC_EN__SHIFT 0x2
+#define DMCU_INTERRUPT_TO_UC_EN_MASK_1__DCPG_IHC_DCFEV1_POWER_UP_INT_TO_UC_EN_MASK 0x8
+#define DMCU_INTERRUPT_TO_UC_EN_MASK_1__DCPG_IHC_DCFEV1_POWER_UP_INT_TO_UC_EN__SHIFT 0x3
+#define DMCU_INTERRUPT_TO_UC_EN_MASK_1__DCPG_IHC_DCFEV1_POWER_DOWN_INT_TO_UC_EN_MASK 0x10
+#define DMCU_INTERRUPT_TO_UC_EN_MASK_1__DCPG_IHC_DCFEV1_POWER_DOWN_INT_TO_UC_EN__SHIFT 0x4
+#define DMCU_INTERRUPT_TO_UC_EN_MASK_1__DCFEV1_VBLANK_INT_TO_UC_EN_MASK 0x20
+#define DMCU_INTERRUPT_TO_UC_EN_MASK_1__DCFEV1_VBLANK_INT_TO_UC_EN__SHIFT 0x5
+#define DMCU_INTERRUPT_TO_UC_EN_MASK_1__DMCU_GENERIC_INT_TO_UC_EN_MASK 0x2000
+#define DMCU_INTERRUPT_TO_UC_EN_MASK_1__DMCU_GENERIC_INT_TO_UC_EN__SHIFT 0xd
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__ABM1_HG_READY_INT_XIRQ_IRQ_SEL_MASK 0x1
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__ABM1_HG_READY_INT_XIRQ_IRQ_SEL__SHIFT 0x0
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__ABM1_LS_READY_INT_XIRQ_IRQ_SEL_MASK 0x2
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__ABM1_LS_READY_INT_XIRQ_IRQ_SEL__SHIFT 0x1
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__ABM1_BL_UPDATE_INT_XIRQ_IRQ_SEL_MASK 0x4
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__ABM1_BL_UPDATE_INT_XIRQ_IRQ_SEL__SHIFT 0x2
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__MCP_INT_XIRQ_IRQ_SEL_MASK 0x8
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__MCP_INT_XIRQ_IRQ_SEL__SHIFT 0x3
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DSI_POWER_UP_INT_XIRQ_IRQ_SEL_MASK 0x10
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DSI_POWER_UP_INT_XIRQ_IRQ_SEL__SHIFT 0x4
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DSI_POWER_DOWN_INT_XIRQ_IRQ_SEL_MASK 0x20
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DSI_POWER_DOWN_INT_XIRQ_IRQ_SEL__SHIFT 0x5
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__STATIC_SCREEN1_INT_XIRQ_IRQ_SEL_MASK 0x40
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__STATIC_SCREEN1_INT_XIRQ_IRQ_SEL__SHIFT 0x6
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__STATIC_SCREEN2_INT_XIRQ_IRQ_SEL_MASK 0x80
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__STATIC_SCREEN2_INT_XIRQ_IRQ_SEL__SHIFT 0x7
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__EXTERNAL_SW_INT_XIRQ_IRQ_SEL_MASK 0x100
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__EXTERNAL_SW_INT_XIRQ_IRQ_SEL__SHIFT 0x8
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__STATIC_SCREEN3_INT_XIRQ_IRQ_SEL_MASK 0x200
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__STATIC_SCREEN3_INT_XIRQ_IRQ_SEL__SHIFT 0x9
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__STATIC_SCREEN4_INT_XIRQ_IRQ_SEL_MASK 0x400
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__STATIC_SCREEN4_INT_XIRQ_IRQ_SEL__SHIFT 0xa
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__STATIC_SCREEN5_INT_XIRQ_IRQ_SEL_MASK 0x800
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__STATIC_SCREEN5_INT_XIRQ_IRQ_SEL__SHIFT 0xb
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE0_POWER_UP_INT_XIRQ_IRQ_SEL_MASK 0x1000
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE0_POWER_UP_INT_XIRQ_IRQ_SEL__SHIFT 0xc
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE1_POWER_UP_INT_XIRQ_IRQ_SEL_MASK 0x2000
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE1_POWER_UP_INT_XIRQ_IRQ_SEL__SHIFT 0xd
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE2_POWER_UP_INT_XIRQ_IRQ_SEL_MASK 0x4000
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE2_POWER_UP_INT_XIRQ_IRQ_SEL__SHIFT 0xe
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE3_POWER_UP_INT_XIRQ_IRQ_SEL_MASK 0x8000
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE3_POWER_UP_INT_XIRQ_IRQ_SEL__SHIFT 0xf
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE4_POWER_UP_INT_XIRQ_IRQ_SEL_MASK 0x10000
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE4_POWER_UP_INT_XIRQ_IRQ_SEL__SHIFT 0x10
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE5_POWER_UP_INT_XIRQ_IRQ_SEL_MASK 0x20000
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE5_POWER_UP_INT_XIRQ_IRQ_SEL__SHIFT 0x11
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE0_POWER_DOWN_INT_XIRQ_IRQ_SEL_MASK 0x40000
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE0_POWER_DOWN_INT_XIRQ_IRQ_SEL__SHIFT 0x12
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE1_POWER_DOWN_INT_XIRQ_IRQ_SEL_MASK 0x80000
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE1_POWER_DOWN_INT_XIRQ_IRQ_SEL__SHIFT 0x13
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE2_POWER_DOWN_INT_XIRQ_IRQ_SEL_MASK 0x100000
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE2_POWER_DOWN_INT_XIRQ_IRQ_SEL__SHIFT 0x14
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE3_POWER_DOWN_INT_XIRQ_IRQ_SEL_MASK 0x200000
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE3_POWER_DOWN_INT_XIRQ_IRQ_SEL__SHIFT 0x15
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE4_POWER_DOWN_INT_XIRQ_IRQ_SEL_MASK 0x400000
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE4_POWER_DOWN_INT_XIRQ_IRQ_SEL__SHIFT 0x16
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE5_POWER_DOWN_INT_XIRQ_IRQ_SEL_MASK 0x800000
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__DCPG_IHC_DCFE5_POWER_DOWN_INT_XIRQ_IRQ_SEL__SHIFT 0x17
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__VBLANK1_INT_XIRQ_IRQ_SEL_MASK 0x1000000
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__VBLANK1_INT_XIRQ_IRQ_SEL__SHIFT 0x18
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__VBLANK2_INT_XIRQ_IRQ_SEL_MASK 0x2000000
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__VBLANK2_INT_XIRQ_IRQ_SEL__SHIFT 0x19
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__VBLANK3_INT_XIRQ_IRQ_SEL_MASK 0x4000000
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__VBLANK3_INT_XIRQ_IRQ_SEL__SHIFT 0x1a
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__VBLANK4_INT_XIRQ_IRQ_SEL_MASK 0x8000000
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__VBLANK4_INT_XIRQ_IRQ_SEL__SHIFT 0x1b
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__VBLANK5_INT_XIRQ_IRQ_SEL_MASK 0x10000000
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__VBLANK5_INT_XIRQ_IRQ_SEL__SHIFT 0x1c
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__VBLANK6_INT_XIRQ_IRQ_SEL_MASK 0x20000000
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__VBLANK6_INT_XIRQ_IRQ_SEL__SHIFT 0x1d
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__STATIC_SCREEN6_INT_XIRQ_IRQ_SEL_MASK 0x40000000
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL__STATIC_SCREEN6_INT_XIRQ_IRQ_SEL__SHIFT 0x1e
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL_1__DCPG_IHC_DCFEV0_POWER_UP_INT_XIRQ_IRQ_SEL_MASK 0x1
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL_1__DCPG_IHC_DCFEV0_POWER_UP_INT_XIRQ_IRQ_SEL__SHIFT 0x0
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL_1__DCPG_IHC_DCFEV0_POWER_DOWN_INT_XIRQ_IRQ_SEL_MASK 0x2
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL_1__DCPG_IHC_DCFEV0_POWER_DOWN_INT_XIRQ_IRQ_SEL__SHIFT 0x1
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL_1__DCFEV0_VBLANK_INT_XIRQ_IRQ_SEL_MASK 0x4
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL_1__DCFEV0_VBLANK_INT_XIRQ_IRQ_SEL__SHIFT 0x2
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL_1__DCPG_IHC_DCFEV1_POWER_UP_INT_XIRQ_IRQ_SEL_MASK 0x8
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL_1__DCPG_IHC_DCFEV1_POWER_UP_INT_XIRQ_IRQ_SEL__SHIFT 0x3
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL_1__DCPG_IHC_DCFEV1_POWER_DOWN_INT_XIRQ_IRQ_SEL_MASK 0x10
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL_1__DCPG_IHC_DCFEV1_POWER_DOWN_INT_XIRQ_IRQ_SEL__SHIFT 0x4
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL_1__DCFEV1_VBLANK_INT_XIRQ_IRQ_SEL_MASK 0x20
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL_1__DCFEV1_VBLANK_INT_XIRQ_IRQ_SEL__SHIFT 0x5
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL_1__DMCU_GENERIC_INT_XIRQ_IRQ_SEL_MASK 0x2000
+#define DMCU_INTERRUPT_TO_UC_XIRQ_IRQ_SEL_1__DMCU_GENERIC_INT_XIRQ_IRQ_SEL__SHIFT 0xd
+#define DC_DMCU_SCRATCH__DMCU_SCRATCH_MASK 0xffffffff
+#define DC_DMCU_SCRATCH__DMCU_SCRATCH__SHIFT 0x0
+#define DMCU_INT_CNT__DMCU_ABM1_HG_READY_INT_CNT_MASK 0xff
+#define DMCU_INT_CNT__DMCU_ABM1_HG_READY_INT_CNT__SHIFT 0x0
+#define DMCU_INT_CNT__DMCU_ABM1_LS_READY_INT_CNT_MASK 0xff00
+#define DMCU_INT_CNT__DMCU_ABM1_LS_READY_INT_CNT__SHIFT 0x8
+#define DMCU_INT_CNT__DMCU_ABM1_BL_UPDATE_INT_CNT_MASK 0xff0000
+#define DMCU_INT_CNT__DMCU_ABM1_BL_UPDATE_INT_CNT__SHIFT 0x10
+#define DMCU_FW_CHECKSUM_SMPL_BYTE_POS__DMCU_FW_CHECKSUM_LO_SMPL_BYTE_POS_MASK 0x3
+#define DMCU_FW_CHECKSUM_SMPL_BYTE_POS__DMCU_FW_CHECKSUM_LO_SMPL_BYTE_POS__SHIFT 0x0
+#define DMCU_FW_CHECKSUM_SMPL_BYTE_POS__DMCU_FW_CHECKSUM_HI_SMPL_BYTE_POS_MASK 0xc
+#define DMCU_FW_CHECKSUM_SMPL_BYTE_POS__DMCU_FW_CHECKSUM_HI_SMPL_BYTE_POS__SHIFT 0x2
+#define DMCU_UC_CLK_GATING_CNTL__UC_IRAM_RD_DELAY_MASK 0x7
+#define DMCU_UC_CLK_GATING_CNTL__UC_IRAM_RD_DELAY__SHIFT 0x0
+#define DMCU_UC_CLK_GATING_CNTL__UC_ERAM_RD_DELAY_MASK 0x700
+#define DMCU_UC_CLK_GATING_CNTL__UC_ERAM_RD_DELAY__SHIFT 0x8
+#define DMCU_UC_CLK_GATING_CNTL__UC_RBBM_RD_CLK_GATING_EN_MASK 0x10000
+#define DMCU_UC_CLK_GATING_CNTL__UC_RBBM_RD_CLK_GATING_EN__SHIFT 0x10
+#define MASTER_COMM_DATA_REG1__MASTER_COMM_DATA_REG1_BYTE0_MASK 0xff
+#define MASTER_COMM_DATA_REG1__MASTER_COMM_DATA_REG1_BYTE0__SHIFT 0x0
+#define MASTER_COMM_DATA_REG1__MASTER_COMM_DATA_REG1_BYTE1_MASK 0xff00
+#define MASTER_COMM_DATA_REG1__MASTER_COMM_DATA_REG1_BYTE1__SHIFT 0x8
+#define MASTER_COMM_DATA_REG1__MASTER_COMM_DATA_REG1_BYTE2_MASK 0xff0000
+#define MASTER_COMM_DATA_REG1__MASTER_COMM_DATA_REG1_BYTE2__SHIFT 0x10
+#define MASTER_COMM_DATA_REG1__MASTER_COMM_DATA_REG1_BYTE3_MASK 0xff000000
+#define MASTER_COMM_DATA_REG1__MASTER_COMM_DATA_REG1_BYTE3__SHIFT 0x18
+#define MASTER_COMM_DATA_REG2__MASTER_COMM_DATA_REG2_BYTE0_MASK 0xff
+#define MASTER_COMM_DATA_REG2__MASTER_COMM_DATA_REG2_BYTE0__SHIFT 0x0
+#define MASTER_COMM_DATA_REG2__MASTER_COMM_DATA_REG2_BYTE1_MASK 0xff00
+#define MASTER_COMM_DATA_REG2__MASTER_COMM_DATA_REG2_BYTE1__SHIFT 0x8
+#define MASTER_COMM_DATA_REG2__MASTER_COMM_DATA_REG2_BYTE2_MASK 0xff0000
+#define MASTER_COMM_DATA_REG2__MASTER_COMM_DATA_REG2_BYTE2__SHIFT 0x10
+#define MASTER_COMM_DATA_REG2__MASTER_COMM_DATA_REG2_BYTE3_MASK 0xff000000
+#define MASTER_COMM_DATA_REG2__MASTER_COMM_DATA_REG2_BYTE3__SHIFT 0x18
+#define MASTER_COMM_DATA_REG3__MASTER_COMM_DATA_REG3_BYTE0_MASK 0xff
+#define MASTER_COMM_DATA_REG3__MASTER_COMM_DATA_REG3_BYTE0__SHIFT 0x0
+#define MASTER_COMM_DATA_REG3__MASTER_COMM_DATA_REG3_BYTE1_MASK 0xff00
+#define MASTER_COMM_DATA_REG3__MASTER_COMM_DATA_REG3_BYTE1__SHIFT 0x8
+#define MASTER_COMM_DATA_REG3__MASTER_COMM_DATA_REG3_BYTE2_MASK 0xff0000
+#define MASTER_COMM_DATA_REG3__MASTER_COMM_DATA_REG3_BYTE2__SHIFT 0x10
+#define MASTER_COMM_DATA_REG3__MASTER_COMM_DATA_REG3_BYTE3_MASK 0xff000000
+#define MASTER_COMM_DATA_REG3__MASTER_COMM_DATA_REG3_BYTE3__SHIFT 0x18
+#define MASTER_COMM_CMD_REG__MASTER_COMM_CMD_REG_BYTE0_MASK 0xff
+#define MASTER_COMM_CMD_REG__MASTER_COMM_CMD_REG_BYTE0__SHIFT 0x0
+#define MASTER_COMM_CMD_REG__MASTER_COMM_CMD_REG_BYTE1_MASK 0xff00
+#define MASTER_COMM_CMD_REG__MASTER_COMM_CMD_REG_BYTE1__SHIFT 0x8
+#define MASTER_COMM_CMD_REG__MASTER_COMM_CMD_REG_BYTE2_MASK 0xff0000
+#define MASTER_COMM_CMD_REG__MASTER_COMM_CMD_REG_BYTE2__SHIFT 0x10
+#define MASTER_COMM_CMD_REG__MASTER_COMM_CMD_REG_BYTE3_MASK 0xff000000
+#define MASTER_COMM_CMD_REG__MASTER_COMM_CMD_REG_BYTE3__SHIFT 0x18
+#define MASTER_COMM_CNTL_REG__MASTER_COMM_INTERRUPT_MASK 0x1
+#define MASTER_COMM_CNTL_REG__MASTER_COMM_INTERRUPT__SHIFT 0x0
+#define SLAVE_COMM_DATA_REG1__SLAVE_COMM_DATA_REG1_BYTE0_MASK 0xff
+#define SLAVE_COMM_DATA_REG1__SLAVE_COMM_DATA_REG1_BYTE0__SHIFT 0x0
+#define SLAVE_COMM_DATA_REG1__SLAVE_COMM_DATA_REG1_BYTE1_MASK 0xff00
+#define SLAVE_COMM_DATA_REG1__SLAVE_COMM_DATA_REG1_BYTE1__SHIFT 0x8
+#define SLAVE_COMM_DATA_REG1__SLAVE_COMM_DATA_REG1_BYTE2_MASK 0xff0000
+#define SLAVE_COMM_DATA_REG1__SLAVE_COMM_DATA_REG1_BYTE2__SHIFT 0x10
+#define SLAVE_COMM_DATA_REG1__SLAVE_COMM_DATA_REG1_BYTE3_MASK 0xff000000
+#define SLAVE_COMM_DATA_REG1__SLAVE_COMM_DATA_REG1_BYTE3__SHIFT 0x18
+#define SLAVE_COMM_DATA_REG2__SLAVE_COMM_DATA_REG2_BYTE0_MASK 0xff
+#define SLAVE_COMM_DATA_REG2__SLAVE_COMM_DATA_REG2_BYTE0__SHIFT 0x0
+#define SLAVE_COMM_DATA_REG2__SLAVE_COMM_DATA_REG2_BYTE1_MASK 0xff00
+#define SLAVE_COMM_DATA_REG2__SLAVE_COMM_DATA_REG2_BYTE1__SHIFT 0x8
+#define SLAVE_COMM_DATA_REG2__SLAVE_COMM_DATA_REG2_BYTE2_MASK 0xff0000
+#define SLAVE_COMM_DATA_REG2__SLAVE_COMM_DATA_REG2_BYTE2__SHIFT 0x10
+#define SLAVE_COMM_DATA_REG2__SLAVE_COMM_DATA_REG2_BYTE3_MASK 0xff000000
+#define SLAVE_COMM_DATA_REG2__SLAVE_COMM_DATA_REG2_BYTE3__SHIFT 0x18
+#define SLAVE_COMM_DATA_REG3__SLAVE_COMM_DATA_REG3_BYTE0_MASK 0xff
+#define SLAVE_COMM_DATA_REG3__SLAVE_COMM_DATA_REG3_BYTE0__SHIFT 0x0
+#define SLAVE_COMM_DATA_REG3__SLAVE_COMM_DATA_REG3_BYTE1_MASK 0xff00
+#define SLAVE_COMM_DATA_REG3__SLAVE_COMM_DATA_REG3_BYTE1__SHIFT 0x8
+#define SLAVE_COMM_DATA_REG3__SLAVE_COMM_DATA_REG3_BYTE2_MASK 0xff0000
+#define SLAVE_COMM_DATA_REG3__SLAVE_COMM_DATA_REG3_BYTE2__SHIFT 0x10
+#define SLAVE_COMM_DATA_REG3__SLAVE_COMM_DATA_REG3_BYTE3_MASK 0xff000000
+#define SLAVE_COMM_DATA_REG3__SLAVE_COMM_DATA_REG3_BYTE3__SHIFT 0x18
+#define SLAVE_COMM_CMD_REG__SLAVE_COMM_CMD_REG_BYTE0_MASK 0xff
+#define SLAVE_COMM_CMD_REG__SLAVE_COMM_CMD_REG_BYTE0__SHIFT 0x0
+#define SLAVE_COMM_CMD_REG__SLAVE_COMM_CMD_REG_BYTE1_MASK 0xff00
+#define SLAVE_COMM_CMD_REG__SLAVE_COMM_CMD_REG_BYTE1__SHIFT 0x8
+#define SLAVE_COMM_CMD_REG__SLAVE_COMM_CMD_REG_BYTE2_MASK 0xff0000
+#define SLAVE_COMM_CMD_REG__SLAVE_COMM_CMD_REG_BYTE2__SHIFT 0x10
+#define SLAVE_COMM_CMD_REG__SLAVE_COMM_CMD_REG_BYTE3_MASK 0xff000000
+#define SLAVE_COMM_CMD_REG__SLAVE_COMM_CMD_REG_BYTE3__SHIFT 0x18
+#define SLAVE_COMM_CNTL_REG__SLAVE_COMM_INTERRUPT_MASK 0x1
+#define SLAVE_COMM_CNTL_REG__SLAVE_COMM_INTERRUPT__SHIFT 0x0
+#define SLAVE_COMM_CNTL_REG__COMM_PORT_MSG_TO_HOST_IN_PROGRESS_MASK 0x100
+#define SLAVE_COMM_CNTL_REG__COMM_PORT_MSG_TO_HOST_IN_PROGRESS__SHIFT 0x8
+#define DMCU_TEST_DEBUG_INDEX__DMCU_TEST_DEBUG_INDEX_MASK 0xff
+#define DMCU_TEST_DEBUG_INDEX__DMCU_TEST_DEBUG_INDEX__SHIFT 0x0
+#define DMCU_TEST_DEBUG_INDEX__DMCU_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define DMCU_TEST_DEBUG_INDEX__DMCU_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define DMCU_TEST_DEBUG_DATA__DMCU_TEST_DEBUG_DATA_MASK 0xffffffff
+#define DMCU_TEST_DEBUG_DATA__DMCU_TEST_DEBUG_DATA__SHIFT 0x0
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER0_INT_OCCURRED_MASK 0x1
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER0_INT_OCCURRED__SHIFT 0x0
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER0_INT_CLEAR_MASK 0x1
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER0_INT_CLEAR__SHIFT 0x0
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER1_INT_OCCURRED_MASK 0x2
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER1_INT_OCCURRED__SHIFT 0x1
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER1_INT_CLEAR_MASK 0x2
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER1_INT_CLEAR__SHIFT 0x1
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER2_INT_OCCURRED_MASK 0x4
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER2_INT_OCCURRED__SHIFT 0x2
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER2_INT_CLEAR_MASK 0x4
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER2_INT_CLEAR__SHIFT 0x2
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER3_INT_OCCURRED_MASK 0x8
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER3_INT_OCCURRED__SHIFT 0x3
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER3_INT_CLEAR_MASK 0x8
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER3_INT_CLEAR__SHIFT 0x3
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER4_INT_OCCURRED_MASK 0x10
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER4_INT_OCCURRED__SHIFT 0x4
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER4_INT_CLEAR_MASK 0x10
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER4_INT_CLEAR__SHIFT 0x4
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER5_INT_OCCURRED_MASK 0x20
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER5_INT_OCCURRED__SHIFT 0x5
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER5_INT_CLEAR_MASK 0x20
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER5_INT_CLEAR__SHIFT 0x5
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER6_INT_OCCURRED_MASK 0x40
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER6_INT_OCCURRED__SHIFT 0x6
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER6_INT_CLEAR_MASK 0x40
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER6_INT_CLEAR__SHIFT 0x6
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER7_INT_OCCURRED_MASK 0x80
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER7_INT_OCCURRED__SHIFT 0x7
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER7_INT_CLEAR_MASK 0x80
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER7_INT_CLEAR__SHIFT 0x7
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER0_INT_OCCURRED_MASK 0x100
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER0_INT_OCCURRED__SHIFT 0x8
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER0_INT_CLEAR_MASK 0x100
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER0_INT_CLEAR__SHIFT 0x8
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER1_INT_OCCURRED_MASK 0x200
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER1_INT_OCCURRED__SHIFT 0x9
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER1_INT_CLEAR_MASK 0x200
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER1_INT_CLEAR__SHIFT 0x9
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER2_INT_OCCURRED_MASK 0x400
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER2_INT_OCCURRED__SHIFT 0xa
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER2_INT_CLEAR_MASK 0x400
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER2_INT_CLEAR__SHIFT 0xa
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER3_INT_OCCURRED_MASK 0x800
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER3_INT_OCCURRED__SHIFT 0xb
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER3_INT_CLEAR_MASK 0x800
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER3_INT_CLEAR__SHIFT 0xb
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER4_INT_OCCURRED_MASK 0x1000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER4_INT_OCCURRED__SHIFT 0xc
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER4_INT_CLEAR_MASK 0x1000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER4_INT_CLEAR__SHIFT 0xc
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER5_INT_OCCURRED_MASK 0x2000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER5_INT_OCCURRED__SHIFT 0xd
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER5_INT_CLEAR_MASK 0x2000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER5_INT_CLEAR__SHIFT 0xd
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER6_INT_OCCURRED_MASK 0x4000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER6_INT_OCCURRED__SHIFT 0xe
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER6_INT_CLEAR_MASK 0x4000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER6_INT_CLEAR__SHIFT 0xe
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER7_INT_OCCURRED_MASK 0x8000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER7_INT_OCCURRED__SHIFT 0xf
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER7_INT_CLEAR_MASK 0x8000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER7_INT_CLEAR__SHIFT 0xf
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER0_INT_OCCURRED_MASK 0x10000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER0_INT_OCCURRED__SHIFT 0x10
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER0_INT_CLEAR_MASK 0x10000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER0_INT_CLEAR__SHIFT 0x10
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER1_INT_OCCURRED_MASK 0x20000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER1_INT_OCCURRED__SHIFT 0x11
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER1_INT_CLEAR_MASK 0x20000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER1_INT_CLEAR__SHIFT 0x11
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER2_INT_OCCURRED_MASK 0x40000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER2_INT_OCCURRED__SHIFT 0x12
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER2_INT_CLEAR_MASK 0x40000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER2_INT_CLEAR__SHIFT 0x12
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER3_INT_OCCURRED_MASK 0x80000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER3_INT_OCCURRED__SHIFT 0x13
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER3_INT_CLEAR_MASK 0x80000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER3_INT_CLEAR__SHIFT 0x13
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER4_INT_OCCURRED_MASK 0x100000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER4_INT_OCCURRED__SHIFT 0x14
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER4_INT_CLEAR_MASK 0x100000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER4_INT_CLEAR__SHIFT 0x14
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER5_INT_OCCURRED_MASK 0x200000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER5_INT_OCCURRED__SHIFT 0x15
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER5_INT_CLEAR_MASK 0x200000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER5_INT_CLEAR__SHIFT 0x15
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER6_INT_OCCURRED_MASK 0x400000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER6_INT_OCCURRED__SHIFT 0x16
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER6_INT_CLEAR_MASK 0x400000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER6_INT_CLEAR__SHIFT 0x16
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER7_INT_OCCURRED_MASK 0x800000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER7_INT_OCCURRED__SHIFT 0x17
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER7_INT_CLEAR_MASK 0x800000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER7_INT_CLEAR__SHIFT 0x17
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER_OFF_INT_OCCURRED_MASK 0x1000000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER_OFF_INT_OCCURRED__SHIFT 0x18
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER_OFF_INT_CLEAR_MASK 0x1000000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCI_PERFMON_COUNTER_OFF_INT_CLEAR__SHIFT 0x18
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER_OFF_INT_OCCURRED_MASK 0x2000000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER_OFF_INT_OCCURRED__SHIFT 0x19
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER_OFF_INT_CLEAR_MASK 0x2000000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCO_PERFMON_COUNTER_OFF_INT_CLEAR__SHIFT 0x19
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER_OFF_INT_OCCURRED_MASK 0x4000000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER_OFF_INT_OCCURRED__SHIFT 0x1a
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER_OFF_INT_CLEAR_MASK 0x4000000
+#define DMCU_PERFMON_INTERRUPT_STATUS1__DCCG_PERFMON_COUNTER_OFF_INT_CLEAR__SHIFT 0x1a
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER0_INT_OCCURRED_MASK 0x1
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER0_INT_OCCURRED__SHIFT 0x0
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER0_INT_CLEAR_MASK 0x1
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER0_INT_CLEAR__SHIFT 0x0
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER1_INT_OCCURRED_MASK 0x2
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER1_INT_OCCURRED__SHIFT 0x1
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER1_INT_CLEAR_MASK 0x2
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER1_INT_CLEAR__SHIFT 0x1
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER2_INT_OCCURRED_MASK 0x4
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER2_INT_OCCURRED__SHIFT 0x2
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER2_INT_CLEAR_MASK 0x4
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER2_INT_CLEAR__SHIFT 0x2
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER3_INT_OCCURRED_MASK 0x8
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER3_INT_OCCURRED__SHIFT 0x3
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER3_INT_CLEAR_MASK 0x8
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER3_INT_CLEAR__SHIFT 0x3
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER4_INT_OCCURRED_MASK 0x10
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER4_INT_OCCURRED__SHIFT 0x4
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER4_INT_CLEAR_MASK 0x10
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER4_INT_CLEAR__SHIFT 0x4
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER5_INT_OCCURRED_MASK 0x20
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER5_INT_OCCURRED__SHIFT 0x5
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER5_INT_CLEAR_MASK 0x20
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER5_INT_CLEAR__SHIFT 0x5
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER6_INT_OCCURRED_MASK 0x40
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER6_INT_OCCURRED__SHIFT 0x6
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER6_INT_CLEAR_MASK 0x40
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER6_INT_CLEAR__SHIFT 0x6
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER7_INT_OCCURRED_MASK 0x80
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER7_INT_OCCURRED__SHIFT 0x7
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER7_INT_CLEAR_MASK 0x80
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER7_INT_CLEAR__SHIFT 0x7
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER0_INT_OCCURRED_MASK 0x100
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER0_INT_OCCURRED__SHIFT 0x8
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER0_INT_CLEAR_MASK 0x100
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER0_INT_CLEAR__SHIFT 0x8
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER1_INT_OCCURRED_MASK 0x200
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER1_INT_OCCURRED__SHIFT 0x9
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER1_INT_CLEAR_MASK 0x200
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER1_INT_CLEAR__SHIFT 0x9
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER2_INT_OCCURRED_MASK 0x400
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER2_INT_OCCURRED__SHIFT 0xa
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER2_INT_CLEAR_MASK 0x400
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER2_INT_CLEAR__SHIFT 0xa
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER3_INT_OCCURRED_MASK 0x800
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER3_INT_OCCURRED__SHIFT 0xb
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER3_INT_CLEAR_MASK 0x800
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER3_INT_CLEAR__SHIFT 0xb
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER4_INT_OCCURRED_MASK 0x1000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER4_INT_OCCURRED__SHIFT 0xc
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER4_INT_CLEAR_MASK 0x1000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER4_INT_CLEAR__SHIFT 0xc
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER5_INT_OCCURRED_MASK 0x2000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER5_INT_OCCURRED__SHIFT 0xd
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER5_INT_CLEAR_MASK 0x2000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER5_INT_CLEAR__SHIFT 0xd
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER6_INT_OCCURRED_MASK 0x4000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER6_INT_OCCURRED__SHIFT 0xe
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER6_INT_CLEAR_MASK 0x4000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER6_INT_CLEAR__SHIFT 0xe
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER7_INT_OCCURRED_MASK 0x8000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER7_INT_OCCURRED__SHIFT 0xf
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER7_INT_CLEAR_MASK 0x8000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER7_INT_CLEAR__SHIFT 0xf
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER0_INT_OCCURRED_MASK 0x10000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER0_INT_OCCURRED__SHIFT 0x10
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER0_INT_CLEAR_MASK 0x10000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER0_INT_CLEAR__SHIFT 0x10
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER1_INT_OCCURRED_MASK 0x20000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER1_INT_OCCURRED__SHIFT 0x11
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER1_INT_CLEAR_MASK 0x20000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER1_INT_CLEAR__SHIFT 0x11
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER2_INT_OCCURRED_MASK 0x40000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER2_INT_OCCURRED__SHIFT 0x12
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER2_INT_CLEAR_MASK 0x40000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER2_INT_CLEAR__SHIFT 0x12
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER3_INT_OCCURRED_MASK 0x80000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER3_INT_OCCURRED__SHIFT 0x13
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER3_INT_CLEAR_MASK 0x80000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER3_INT_CLEAR__SHIFT 0x13
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER4_INT_OCCURRED_MASK 0x100000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER4_INT_OCCURRED__SHIFT 0x14
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER4_INT_CLEAR_MASK 0x100000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER4_INT_CLEAR__SHIFT 0x14
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER5_INT_OCCURRED_MASK 0x200000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER5_INT_OCCURRED__SHIFT 0x15
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER5_INT_CLEAR_MASK 0x200000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER5_INT_CLEAR__SHIFT 0x15
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER6_INT_OCCURRED_MASK 0x400000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER6_INT_OCCURRED__SHIFT 0x16
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER6_INT_CLEAR_MASK 0x400000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER6_INT_CLEAR__SHIFT 0x16
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER7_INT_OCCURRED_MASK 0x800000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER7_INT_OCCURRED__SHIFT 0x17
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER7_INT_CLEAR_MASK 0x800000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER7_INT_CLEAR__SHIFT 0x17
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER_OFF_INT_OCCURRED_MASK 0x1000000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER_OFF_INT_OCCURRED__SHIFT 0x18
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER_OFF_INT_CLEAR_MASK 0x1000000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE0_PERFMON_COUNTER_OFF_INT_CLEAR__SHIFT 0x18
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER_OFF_INT_OCCURRED_MASK 0x2000000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER_OFF_INT_OCCURRED__SHIFT 0x19
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER_OFF_INT_CLEAR_MASK 0x2000000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE1_PERFMON_COUNTER_OFF_INT_CLEAR__SHIFT 0x19
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER_OFF_INT_OCCURRED_MASK 0x4000000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER_OFF_INT_OCCURRED__SHIFT 0x1a
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER_OFF_INT_CLEAR_MASK 0x4000000
+#define DMCU_PERFMON_INTERRUPT_STATUS2__DCFE2_PERFMON_COUNTER_OFF_INT_CLEAR__SHIFT 0x1a
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER0_INT_OCCURRED_MASK 0x1
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER0_INT_OCCURRED__SHIFT 0x0
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER0_INT_CLEAR_MASK 0x1
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER0_INT_CLEAR__SHIFT 0x0
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER1_INT_OCCURRED_MASK 0x2
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER1_INT_OCCURRED__SHIFT 0x1
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER1_INT_CLEAR_MASK 0x2
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER1_INT_CLEAR__SHIFT 0x1
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER2_INT_OCCURRED_MASK 0x4
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER2_INT_OCCURRED__SHIFT 0x2
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER2_INT_CLEAR_MASK 0x4
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER2_INT_CLEAR__SHIFT 0x2
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER3_INT_OCCURRED_MASK 0x8
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER3_INT_OCCURRED__SHIFT 0x3
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER3_INT_CLEAR_MASK 0x8
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER3_INT_CLEAR__SHIFT 0x3
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER4_INT_OCCURRED_MASK 0x10
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER4_INT_OCCURRED__SHIFT 0x4
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER4_INT_CLEAR_MASK 0x10
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER4_INT_CLEAR__SHIFT 0x4
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER5_INT_OCCURRED_MASK 0x20
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER5_INT_OCCURRED__SHIFT 0x5
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER5_INT_CLEAR_MASK 0x20
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER5_INT_CLEAR__SHIFT 0x5
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER6_INT_OCCURRED_MASK 0x40
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER6_INT_OCCURRED__SHIFT 0x6
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER6_INT_CLEAR_MASK 0x40
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER6_INT_CLEAR__SHIFT 0x6
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER7_INT_OCCURRED_MASK 0x80
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER7_INT_OCCURRED__SHIFT 0x7
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER7_INT_CLEAR_MASK 0x80
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER7_INT_CLEAR__SHIFT 0x7
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER0_INT_OCCURRED_MASK 0x100
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER0_INT_OCCURRED__SHIFT 0x8
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER0_INT_CLEAR_MASK 0x100
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER0_INT_CLEAR__SHIFT 0x8
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER1_INT_OCCURRED_MASK 0x200
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER1_INT_OCCURRED__SHIFT 0x9
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER1_INT_CLEAR_MASK 0x200
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER1_INT_CLEAR__SHIFT 0x9
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER2_INT_OCCURRED_MASK 0x400
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER2_INT_OCCURRED__SHIFT 0xa
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER2_INT_CLEAR_MASK 0x400
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER2_INT_CLEAR__SHIFT 0xa
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER3_INT_OCCURRED_MASK 0x800
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER3_INT_OCCURRED__SHIFT 0xb
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER3_INT_CLEAR_MASK 0x800
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER3_INT_CLEAR__SHIFT 0xb
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER4_INT_OCCURRED_MASK 0x1000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER4_INT_OCCURRED__SHIFT 0xc
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER4_INT_CLEAR_MASK 0x1000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER4_INT_CLEAR__SHIFT 0xc
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER5_INT_OCCURRED_MASK 0x2000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER5_INT_OCCURRED__SHIFT 0xd
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER5_INT_CLEAR_MASK 0x2000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER5_INT_CLEAR__SHIFT 0xd
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER6_INT_OCCURRED_MASK 0x4000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER6_INT_OCCURRED__SHIFT 0xe
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER6_INT_CLEAR_MASK 0x4000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER6_INT_CLEAR__SHIFT 0xe
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER7_INT_OCCURRED_MASK 0x8000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER7_INT_OCCURRED__SHIFT 0xf
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER7_INT_CLEAR_MASK 0x8000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER7_INT_CLEAR__SHIFT 0xf
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER0_INT_OCCURRED_MASK 0x10000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER0_INT_OCCURRED__SHIFT 0x10
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER0_INT_CLEAR_MASK 0x10000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER0_INT_CLEAR__SHIFT 0x10
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER1_INT_OCCURRED_MASK 0x20000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER1_INT_OCCURRED__SHIFT 0x11
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER1_INT_CLEAR_MASK 0x20000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER1_INT_CLEAR__SHIFT 0x11
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER2_INT_OCCURRED_MASK 0x40000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER2_INT_OCCURRED__SHIFT 0x12
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER2_INT_CLEAR_MASK 0x40000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER2_INT_CLEAR__SHIFT 0x12
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER3_INT_OCCURRED_MASK 0x80000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER3_INT_OCCURRED__SHIFT 0x13
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER3_INT_CLEAR_MASK 0x80000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER3_INT_CLEAR__SHIFT 0x13
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER4_INT_OCCURRED_MASK 0x100000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER4_INT_OCCURRED__SHIFT 0x14
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER4_INT_CLEAR_MASK 0x100000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER4_INT_CLEAR__SHIFT 0x14
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER5_INT_OCCURRED_MASK 0x200000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER5_INT_OCCURRED__SHIFT 0x15
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER5_INT_CLEAR_MASK 0x200000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER5_INT_CLEAR__SHIFT 0x15
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER6_INT_OCCURRED_MASK 0x400000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER6_INT_OCCURRED__SHIFT 0x16
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER6_INT_CLEAR_MASK 0x400000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER6_INT_CLEAR__SHIFT 0x16
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER7_INT_OCCURRED_MASK 0x800000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER7_INT_OCCURRED__SHIFT 0x17
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER7_INT_CLEAR_MASK 0x800000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER7_INT_CLEAR__SHIFT 0x17
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER_OFF_INT_OCCURRED_MASK 0x1000000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER_OFF_INT_OCCURRED__SHIFT 0x18
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER_OFF_INT_CLEAR_MASK 0x1000000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE3_PERFMON_COUNTER_OFF_INT_CLEAR__SHIFT 0x18
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER_OFF_INT_OCCURRED_MASK 0x2000000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER_OFF_INT_OCCURRED__SHIFT 0x19
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER_OFF_INT_CLEAR_MASK 0x2000000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE4_PERFMON_COUNTER_OFF_INT_CLEAR__SHIFT 0x19
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER_OFF_INT_OCCURRED_MASK 0x4000000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER_OFF_INT_OCCURRED__SHIFT 0x1a
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER_OFF_INT_CLEAR_MASK 0x4000000
+#define DMCU_PERFMON_INTERRUPT_STATUS3__DCFE5_PERFMON_COUNTER_OFF_INT_CLEAR__SHIFT 0x1a
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER0_INT_OCCURRED_MASK 0x1
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER0_INT_OCCURRED__SHIFT 0x0
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER0_INT_CLEAR_MASK 0x1
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER0_INT_CLEAR__SHIFT 0x0
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER1_INT_OCCURRED_MASK 0x2
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER1_INT_OCCURRED__SHIFT 0x1
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER1_INT_CLEAR_MASK 0x2
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER1_INT_CLEAR__SHIFT 0x1
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER2_INT_OCCURRED_MASK 0x4
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER2_INT_OCCURRED__SHIFT 0x2
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER2_INT_CLEAR_MASK 0x4
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER2_INT_CLEAR__SHIFT 0x2
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER3_INT_OCCURRED_MASK 0x8
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER3_INT_OCCURRED__SHIFT 0x3
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER3_INT_CLEAR_MASK 0x8
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER3_INT_CLEAR__SHIFT 0x3
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER4_INT_OCCURRED_MASK 0x10
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER4_INT_OCCURRED__SHIFT 0x4
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER4_INT_CLEAR_MASK 0x10
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER4_INT_CLEAR__SHIFT 0x4
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER5_INT_OCCURRED_MASK 0x20
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER5_INT_OCCURRED__SHIFT 0x5
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER5_INT_CLEAR_MASK 0x20
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER5_INT_CLEAR__SHIFT 0x5
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER6_INT_OCCURRED_MASK 0x40
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER6_INT_OCCURRED__SHIFT 0x6
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER6_INT_CLEAR_MASK 0x40
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER6_INT_CLEAR__SHIFT 0x6
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER7_INT_OCCURRED_MASK 0x80
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER7_INT_OCCURRED__SHIFT 0x7
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER7_INT_CLEAR_MASK 0x80
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER7_INT_CLEAR__SHIFT 0x7
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER0_INT_OCCURRED_MASK 0x100
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER0_INT_OCCURRED__SHIFT 0x8
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER0_INT_CLEAR_MASK 0x100
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER0_INT_CLEAR__SHIFT 0x8
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER1_INT_OCCURRED_MASK 0x200
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER1_INT_OCCURRED__SHIFT 0x9
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER1_INT_CLEAR_MASK 0x200
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER1_INT_CLEAR__SHIFT 0x9
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER2_INT_OCCURRED_MASK 0x400
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER2_INT_OCCURRED__SHIFT 0xa
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER2_INT_CLEAR_MASK 0x400
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER2_INT_CLEAR__SHIFT 0xa
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER3_INT_OCCURRED_MASK 0x800
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER3_INT_OCCURRED__SHIFT 0xb
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER3_INT_CLEAR_MASK 0x800
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER3_INT_CLEAR__SHIFT 0xb
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER4_INT_OCCURRED_MASK 0x1000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER4_INT_OCCURRED__SHIFT 0xc
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER4_INT_CLEAR_MASK 0x1000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER4_INT_CLEAR__SHIFT 0xc
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER5_INT_OCCURRED_MASK 0x2000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER5_INT_OCCURRED__SHIFT 0xd
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER5_INT_CLEAR_MASK 0x2000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER5_INT_CLEAR__SHIFT 0xd
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER6_INT_OCCURRED_MASK 0x4000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER6_INT_OCCURRED__SHIFT 0xe
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER6_INT_CLEAR_MASK 0x4000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER6_INT_CLEAR__SHIFT 0xe
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER7_INT_OCCURRED_MASK 0x8000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER7_INT_OCCURRED__SHIFT 0xf
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER7_INT_CLEAR_MASK 0x8000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER7_INT_CLEAR__SHIFT 0xf
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER0_INT_OCCURRED_MASK 0x10000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER0_INT_OCCURRED__SHIFT 0x10
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER0_INT_CLEAR_MASK 0x10000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER0_INT_CLEAR__SHIFT 0x10
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER1_INT_OCCURRED_MASK 0x20000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER1_INT_OCCURRED__SHIFT 0x11
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER1_INT_CLEAR_MASK 0x20000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER1_INT_CLEAR__SHIFT 0x11
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER2_INT_OCCURRED_MASK 0x40000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER2_INT_OCCURRED__SHIFT 0x12
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER2_INT_CLEAR_MASK 0x40000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER2_INT_CLEAR__SHIFT 0x12
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER3_INT_OCCURRED_MASK 0x80000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER3_INT_OCCURRED__SHIFT 0x13
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER3_INT_CLEAR_MASK 0x80000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER3_INT_CLEAR__SHIFT 0x13
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER4_INT_OCCURRED_MASK 0x100000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER4_INT_OCCURRED__SHIFT 0x14
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER4_INT_CLEAR_MASK 0x100000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER4_INT_CLEAR__SHIFT 0x14
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER5_INT_OCCURRED_MASK 0x200000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER5_INT_OCCURRED__SHIFT 0x15
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER5_INT_CLEAR_MASK 0x200000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER5_INT_CLEAR__SHIFT 0x15
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER6_INT_OCCURRED_MASK 0x400000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER6_INT_OCCURRED__SHIFT 0x16
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER6_INT_CLEAR_MASK 0x400000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER6_INT_CLEAR__SHIFT 0x16
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER7_INT_OCCURRED_MASK 0x800000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER7_INT_OCCURRED__SHIFT 0x17
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER7_INT_CLEAR_MASK 0x800000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER7_INT_CLEAR__SHIFT 0x17
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER_OFF_INT_OCCURRED_MASK 0x1000000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER_OFF_INT_OCCURRED__SHIFT 0x18
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER_OFF_INT_CLEAR_MASK 0x1000000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__WB_PERFMON_COUNTER_OFF_INT_CLEAR__SHIFT 0x18
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER_OFF_INT_OCCURRED_MASK 0x2000000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER_OFF_INT_OCCURRED__SHIFT 0x19
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER_OFF_INT_CLEAR_MASK 0x2000000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCRX_PERFMON_COUNTER_OFF_INT_CLEAR__SHIFT 0x19
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER_OFF_INT_OCCURRED_MASK 0x4000000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER_OFF_INT_OCCURRED__SHIFT 0x1a
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER_OFF_INT_CLEAR_MASK 0x4000000
+#define DMCU_PERFMON_INTERRUPT_STATUS4__DCCG_PERFMON2_COUNTER_OFF_INT_CLEAR__SHIFT 0x1a
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER0_INT_OCCURRED_MASK 0x1
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER0_INT_OCCURRED__SHIFT 0x0
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER0_INT_CLEAR_MASK 0x1
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER0_INT_CLEAR__SHIFT 0x0
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER1_INT_OCCURRED_MASK 0x2
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER1_INT_OCCURRED__SHIFT 0x1
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER1_INT_CLEAR_MASK 0x2
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER1_INT_CLEAR__SHIFT 0x1
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER2_INT_OCCURRED_MASK 0x4
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER2_INT_OCCURRED__SHIFT 0x2
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER2_INT_CLEAR_MASK 0x4
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER2_INT_CLEAR__SHIFT 0x2
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER3_INT_OCCURRED_MASK 0x8
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER3_INT_OCCURRED__SHIFT 0x3
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER3_INT_CLEAR_MASK 0x8
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER3_INT_CLEAR__SHIFT 0x3
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER4_INT_OCCURRED_MASK 0x10
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER4_INT_OCCURRED__SHIFT 0x4
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER4_INT_CLEAR_MASK 0x10
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER4_INT_CLEAR__SHIFT 0x4
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER5_INT_OCCURRED_MASK 0x20
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER5_INT_OCCURRED__SHIFT 0x5
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER5_INT_CLEAR_MASK 0x20
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER5_INT_CLEAR__SHIFT 0x5
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER6_INT_OCCURRED_MASK 0x40
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER6_INT_OCCURRED__SHIFT 0x6
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER6_INT_CLEAR_MASK 0x40
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER6_INT_CLEAR__SHIFT 0x6
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER7_INT_OCCURRED_MASK 0x80
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER7_INT_OCCURRED__SHIFT 0x7
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER7_INT_CLEAR_MASK 0x80
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER7_INT_CLEAR__SHIFT 0x7
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER0_INT_OCCURRED_MASK 0x100
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER0_INT_OCCURRED__SHIFT 0x8
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER0_INT_CLEAR_MASK 0x100
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER0_INT_CLEAR__SHIFT 0x8
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER1_INT_OCCURRED_MASK 0x200
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER1_INT_OCCURRED__SHIFT 0x9
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER1_INT_CLEAR_MASK 0x200
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER1_INT_CLEAR__SHIFT 0x9
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER2_INT_OCCURRED_MASK 0x400
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER2_INT_OCCURRED__SHIFT 0xa
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER2_INT_CLEAR_MASK 0x400
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER2_INT_CLEAR__SHIFT 0xa
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER3_INT_OCCURRED_MASK 0x800
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER3_INT_OCCURRED__SHIFT 0xb
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER3_INT_CLEAR_MASK 0x800
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER3_INT_CLEAR__SHIFT 0xb
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER4_INT_OCCURRED_MASK 0x1000
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER4_INT_OCCURRED__SHIFT 0xc
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER4_INT_CLEAR_MASK 0x1000
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER4_INT_CLEAR__SHIFT 0xc
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER5_INT_OCCURRED_MASK 0x2000
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER5_INT_OCCURRED__SHIFT 0xd
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER5_INT_CLEAR_MASK 0x2000
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER5_INT_CLEAR__SHIFT 0xd
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER6_INT_OCCURRED_MASK 0x4000
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER6_INT_OCCURRED__SHIFT 0xe
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER6_INT_CLEAR_MASK 0x4000
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER6_INT_CLEAR__SHIFT 0xe
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER7_INT_OCCURRED_MASK 0x8000
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER7_INT_OCCURRED__SHIFT 0xf
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER7_INT_CLEAR_MASK 0x8000
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER7_INT_CLEAR__SHIFT 0xf
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER_OFF_INT_OCCURRED_MASK 0x10000
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER_OFF_INT_OCCURRED__SHIFT 0x10
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER_OFF_INT_CLEAR_MASK 0x10000
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV0_PERFMON_COUNTER_OFF_INT_CLEAR__SHIFT 0x10
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER_OFF_INT_OCCURRED_MASK 0x20000
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER_OFF_INT_OCCURRED__SHIFT 0x11
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER_OFF_INT_CLEAR_MASK 0x20000
+#define DMCU_PERFMON_INTERRUPT_STATUS5__DCFEV1_PERFMON_COUNTER_OFF_INT_CLEAR__SHIFT 0x11
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCI_PERFMON_COUNTER0_INT_TO_UC_EN_MASK 0x1
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCI_PERFMON_COUNTER0_INT_TO_UC_EN__SHIFT 0x0
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCI_PERFMON_COUNTER1_INT_TO_UC_EN_MASK 0x2
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCI_PERFMON_COUNTER1_INT_TO_UC_EN__SHIFT 0x1
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCI_PERFMON_COUNTER2_INT_TO_UC_EN_MASK 0x4
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCI_PERFMON_COUNTER2_INT_TO_UC_EN__SHIFT 0x2
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCI_PERFMON_COUNTER3_INT_TO_UC_EN_MASK 0x8
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCI_PERFMON_COUNTER3_INT_TO_UC_EN__SHIFT 0x3
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCI_PERFMON_COUNTER4_INT_TO_UC_EN_MASK 0x10
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCI_PERFMON_COUNTER4_INT_TO_UC_EN__SHIFT 0x4
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCI_PERFMON_COUNTER5_INT_TO_UC_EN_MASK 0x20
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCI_PERFMON_COUNTER5_INT_TO_UC_EN__SHIFT 0x5
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCI_PERFMON_COUNTER6_INT_TO_UC_EN_MASK 0x40
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCI_PERFMON_COUNTER6_INT_TO_UC_EN__SHIFT 0x6
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCI_PERFMON_COUNTER7_INT_TO_UC_EN_MASK 0x80
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCI_PERFMON_COUNTER7_INT_TO_UC_EN__SHIFT 0x7
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCO_PERFMON_COUNTER0_INT_TO_UC_EN_MASK 0x100
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCO_PERFMON_COUNTER0_INT_TO_UC_EN__SHIFT 0x8
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCO_PERFMON_COUNTER1_INT_TO_UC_EN_MASK 0x200
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCO_PERFMON_COUNTER1_INT_TO_UC_EN__SHIFT 0x9
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCO_PERFMON_COUNTER2_INT_TO_UC_EN_MASK 0x400
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCO_PERFMON_COUNTER2_INT_TO_UC_EN__SHIFT 0xa
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCO_PERFMON_COUNTER3_INT_TO_UC_EN_MASK 0x800
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCO_PERFMON_COUNTER3_INT_TO_UC_EN__SHIFT 0xb
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCO_PERFMON_COUNTER4_INT_TO_UC_EN_MASK 0x1000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCO_PERFMON_COUNTER4_INT_TO_UC_EN__SHIFT 0xc
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCO_PERFMON_COUNTER5_INT_TO_UC_EN_MASK 0x2000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCO_PERFMON_COUNTER5_INT_TO_UC_EN__SHIFT 0xd
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCO_PERFMON_COUNTER6_INT_TO_UC_EN_MASK 0x4000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCO_PERFMON_COUNTER6_INT_TO_UC_EN__SHIFT 0xe
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCO_PERFMON_COUNTER7_INT_TO_UC_EN_MASK 0x8000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCO_PERFMON_COUNTER7_INT_TO_UC_EN__SHIFT 0xf
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCCG_PERFMON_COUNTER0_INT_TO_UC_EN_MASK 0x10000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCCG_PERFMON_COUNTER0_INT_TO_UC_EN__SHIFT 0x10
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCCG_PERFMON_COUNTER1_INT_TO_UC_EN_MASK 0x20000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCCG_PERFMON_COUNTER1_INT_TO_UC_EN__SHIFT 0x11
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCCG_PERFMON_COUNTER2_INT_TO_UC_EN_MASK 0x40000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCCG_PERFMON_COUNTER2_INT_TO_UC_EN__SHIFT 0x12
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCCG_PERFMON_COUNTER3_INT_TO_UC_EN_MASK 0x80000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCCG_PERFMON_COUNTER3_INT_TO_UC_EN__SHIFT 0x13
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCCG_PERFMON_COUNTER4_INT_TO_UC_EN_MASK 0x100000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCCG_PERFMON_COUNTER4_INT_TO_UC_EN__SHIFT 0x14
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCCG_PERFMON_COUNTER5_INT_TO_UC_EN_MASK 0x200000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCCG_PERFMON_COUNTER5_INT_TO_UC_EN__SHIFT 0x15
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCCG_PERFMON_COUNTER6_INT_TO_UC_EN_MASK 0x400000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCCG_PERFMON_COUNTER6_INT_TO_UC_EN__SHIFT 0x16
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCCG_PERFMON_COUNTER7_INT_TO_UC_EN_MASK 0x800000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCCG_PERFMON_COUNTER7_INT_TO_UC_EN__SHIFT 0x17
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCI_PERFMON_COUNTER_OFF_INT_TO_UC_EN_MASK 0x1000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCI_PERFMON_COUNTER_OFF_INT_TO_UC_EN__SHIFT 0x18
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCO_PERFMON_COUNTER_OFF_INT_TO_UC_EN_MASK 0x2000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCO_PERFMON_COUNTER_OFF_INT_TO_UC_EN__SHIFT 0x19
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCCG_PERFMON_COUNTER_OFF_INT_TO_UC_EN_MASK 0x4000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK1__DCCG_PERFMON_COUNTER_OFF_INT_TO_UC_EN__SHIFT 0x1a
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE0_PERFMON_COUNTER0_INT_TO_UC_EN_MASK 0x1
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE0_PERFMON_COUNTER0_INT_TO_UC_EN__SHIFT 0x0
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE0_PERFMON_COUNTER1_INT_TO_UC_EN_MASK 0x2
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE0_PERFMON_COUNTER1_INT_TO_UC_EN__SHIFT 0x1
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE0_PERFMON_COUNTER2_INT_TO_UC_EN_MASK 0x4
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE0_PERFMON_COUNTER2_INT_TO_UC_EN__SHIFT 0x2
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE0_PERFMON_COUNTER3_INT_TO_UC_EN_MASK 0x8
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE0_PERFMON_COUNTER3_INT_TO_UC_EN__SHIFT 0x3
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE0_PERFMON_COUNTER4_INT_TO_UC_EN_MASK 0x10
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE0_PERFMON_COUNTER4_INT_TO_UC_EN__SHIFT 0x4
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE0_PERFMON_COUNTER5_INT_TO_UC_EN_MASK 0x20
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE0_PERFMON_COUNTER5_INT_TO_UC_EN__SHIFT 0x5
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE0_PERFMON_COUNTER6_INT_TO_UC_EN_MASK 0x40
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE0_PERFMON_COUNTER6_INT_TO_UC_EN__SHIFT 0x6
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE0_PERFMON_COUNTER7_INT_TO_UC_EN_MASK 0x80
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE0_PERFMON_COUNTER7_INT_TO_UC_EN__SHIFT 0x7
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE1_PERFMON_COUNTER0_INT_TO_UC_EN_MASK 0x100
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE1_PERFMON_COUNTER0_INT_TO_UC_EN__SHIFT 0x8
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE1_PERFMON_COUNTER1_INT_TO_UC_EN_MASK 0x200
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE1_PERFMON_COUNTER1_INT_TO_UC_EN__SHIFT 0x9
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE1_PERFMON_COUNTER2_INT_TO_UC_EN_MASK 0x400
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE1_PERFMON_COUNTER2_INT_TO_UC_EN__SHIFT 0xa
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE1_PERFMON_COUNTER3_INT_TO_UC_EN_MASK 0x800
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE1_PERFMON_COUNTER3_INT_TO_UC_EN__SHIFT 0xb
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE1_PERFMON_COUNTER4_INT_TO_UC_EN_MASK 0x1000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE1_PERFMON_COUNTER4_INT_TO_UC_EN__SHIFT 0xc
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE1_PERFMON_COUNTER5_INT_TO_UC_EN_MASK 0x2000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE1_PERFMON_COUNTER5_INT_TO_UC_EN__SHIFT 0xd
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE1_PERFMON_COUNTER6_INT_TO_UC_EN_MASK 0x4000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE1_PERFMON_COUNTER6_INT_TO_UC_EN__SHIFT 0xe
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE1_PERFMON_COUNTER7_INT_TO_UC_EN_MASK 0x8000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE1_PERFMON_COUNTER7_INT_TO_UC_EN__SHIFT 0xf
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE2_PERFMON_COUNTER0_INT_TO_UC_EN_MASK 0x10000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE2_PERFMON_COUNTER0_INT_TO_UC_EN__SHIFT 0x10
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE2_PERFMON_COUNTER1_INT_TO_UC_EN_MASK 0x20000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE2_PERFMON_COUNTER1_INT_TO_UC_EN__SHIFT 0x11
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE2_PERFMON_COUNTER2_INT_TO_UC_EN_MASK 0x40000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE2_PERFMON_COUNTER2_INT_TO_UC_EN__SHIFT 0x12
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE2_PERFMON_COUNTER3_INT_TO_UC_EN_MASK 0x80000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE2_PERFMON_COUNTER3_INT_TO_UC_EN__SHIFT 0x13
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE2_PERFMON_COUNTER4_INT_TO_UC_EN_MASK 0x100000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE2_PERFMON_COUNTER4_INT_TO_UC_EN__SHIFT 0x14
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE2_PERFMON_COUNTER5_INT_TO_UC_EN_MASK 0x200000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE2_PERFMON_COUNTER5_INT_TO_UC_EN__SHIFT 0x15
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE2_PERFMON_COUNTER6_INT_TO_UC_EN_MASK 0x400000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE2_PERFMON_COUNTER6_INT_TO_UC_EN__SHIFT 0x16
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE2_PERFMON_COUNTER7_INT_TO_UC_EN_MASK 0x800000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE2_PERFMON_COUNTER7_INT_TO_UC_EN__SHIFT 0x17
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE0_PERFMON_COUNTER_OFF_INT_TO_UC_EN_MASK 0x1000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE0_PERFMON_COUNTER_OFF_INT_TO_UC_EN__SHIFT 0x18
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE1_PERFMON_COUNTER_OFF_INT_TO_UC_EN_MASK 0x2000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE1_PERFMON_COUNTER_OFF_INT_TO_UC_EN__SHIFT 0x19
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE2_PERFMON_COUNTER_OFF_INT_TO_UC_EN_MASK 0x4000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK2__DCFE2_PERFMON_COUNTER_OFF_INT_TO_UC_EN__SHIFT 0x1a
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE3_PERFMON_COUNTER0_INT_TO_UC_EN_MASK 0x1
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE3_PERFMON_COUNTER0_INT_TO_UC_EN__SHIFT 0x0
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE3_PERFMON_COUNTER1_INT_TO_UC_EN_MASK 0x2
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE3_PERFMON_COUNTER1_INT_TO_UC_EN__SHIFT 0x1
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE3_PERFMON_COUNTER2_INT_TO_UC_EN_MASK 0x4
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE3_PERFMON_COUNTER2_INT_TO_UC_EN__SHIFT 0x2
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE3_PERFMON_COUNTER3_INT_TO_UC_EN_MASK 0x8
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE3_PERFMON_COUNTER3_INT_TO_UC_EN__SHIFT 0x3
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE3_PERFMON_COUNTER4_INT_TO_UC_EN_MASK 0x10
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE3_PERFMON_COUNTER4_INT_TO_UC_EN__SHIFT 0x4
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE3_PERFMON_COUNTER5_INT_TO_UC_EN_MASK 0x20
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE3_PERFMON_COUNTER5_INT_TO_UC_EN__SHIFT 0x5
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE3_PERFMON_COUNTER6_INT_TO_UC_EN_MASK 0x40
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE3_PERFMON_COUNTER6_INT_TO_UC_EN__SHIFT 0x6
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE3_PERFMON_COUNTER7_INT_TO_UC_EN_MASK 0x80
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE3_PERFMON_COUNTER7_INT_TO_UC_EN__SHIFT 0x7
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE4_PERFMON_COUNTER0_INT_TO_UC_EN_MASK 0x100
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE4_PERFMON_COUNTER0_INT_TO_UC_EN__SHIFT 0x8
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE4_PERFMON_COUNTER1_INT_TO_UC_EN_MASK 0x200
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE4_PERFMON_COUNTER1_INT_TO_UC_EN__SHIFT 0x9
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE4_PERFMON_COUNTER2_INT_TO_UC_EN_MASK 0x400
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE4_PERFMON_COUNTER2_INT_TO_UC_EN__SHIFT 0xa
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE4_PERFMON_COUNTER3_INT_TO_UC_EN_MASK 0x800
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE4_PERFMON_COUNTER3_INT_TO_UC_EN__SHIFT 0xb
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE4_PERFMON_COUNTER4_INT_TO_UC_EN_MASK 0x1000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE4_PERFMON_COUNTER4_INT_TO_UC_EN__SHIFT 0xc
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE4_PERFMON_COUNTER5_INT_TO_UC_EN_MASK 0x2000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE4_PERFMON_COUNTER5_INT_TO_UC_EN__SHIFT 0xd
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE4_PERFMON_COUNTER6_INT_TO_UC_EN_MASK 0x4000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE4_PERFMON_COUNTER6_INT_TO_UC_EN__SHIFT 0xe
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE4_PERFMON_COUNTER7_INT_TO_UC_EN_MASK 0x8000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE4_PERFMON_COUNTER7_INT_TO_UC_EN__SHIFT 0xf
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE5_PERFMON_COUNTER0_INT_TO_UC_EN_MASK 0x10000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE5_PERFMON_COUNTER0_INT_TO_UC_EN__SHIFT 0x10
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE5_PERFMON_COUNTER1_INT_TO_UC_EN_MASK 0x20000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE5_PERFMON_COUNTER1_INT_TO_UC_EN__SHIFT 0x11
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE5_PERFMON_COUNTER2_INT_TO_UC_EN_MASK 0x40000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE5_PERFMON_COUNTER2_INT_TO_UC_EN__SHIFT 0x12
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE5_PERFMON_COUNTER3_INT_TO_UC_EN_MASK 0x80000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE5_PERFMON_COUNTER3_INT_TO_UC_EN__SHIFT 0x13
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE5_PERFMON_COUNTER4_INT_TO_UC_EN_MASK 0x100000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE5_PERFMON_COUNTER4_INT_TO_UC_EN__SHIFT 0x14
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE5_PERFMON_COUNTER5_INT_TO_UC_EN_MASK 0x200000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE5_PERFMON_COUNTER5_INT_TO_UC_EN__SHIFT 0x15
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE5_PERFMON_COUNTER6_INT_TO_UC_EN_MASK 0x400000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE5_PERFMON_COUNTER6_INT_TO_UC_EN__SHIFT 0x16
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE5_PERFMON_COUNTER7_INT_TO_UC_EN_MASK 0x800000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE5_PERFMON_COUNTER7_INT_TO_UC_EN__SHIFT 0x17
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE3_PERFMON_COUNTER_OFF_INT_TO_UC_EN_MASK 0x1000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE3_PERFMON_COUNTER_OFF_INT_TO_UC_EN__SHIFT 0x18
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE4_PERFMON_COUNTER_OFF_INT_TO_UC_EN_MASK 0x2000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE4_PERFMON_COUNTER_OFF_INT_TO_UC_EN__SHIFT 0x19
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE5_PERFMON_COUNTER_OFF_INT_TO_UC_EN_MASK 0x4000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK3__DCFE5_PERFMON_COUNTER_OFF_INT_TO_UC_EN__SHIFT 0x1a
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__WB_PERFMON_COUNTER0_INT_TO_UC_EN_MASK 0x1
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__WB_PERFMON_COUNTER0_INT_TO_UC_EN__SHIFT 0x0
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__WB_PERFMON_COUNTER1_INT_TO_UC_EN_MASK 0x2
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__WB_PERFMON_COUNTER1_INT_TO_UC_EN__SHIFT 0x1
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__WB_PERFMON_COUNTER2_INT_TO_UC_EN_MASK 0x4
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__WB_PERFMON_COUNTER2_INT_TO_UC_EN__SHIFT 0x2
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__WB_PERFMON_COUNTER3_INT_TO_UC_EN_MASK 0x8
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__WB_PERFMON_COUNTER3_INT_TO_UC_EN__SHIFT 0x3
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__WB_PERFMON_COUNTER4_INT_TO_UC_EN_MASK 0x10
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__WB_PERFMON_COUNTER4_INT_TO_UC_EN__SHIFT 0x4
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__WB_PERFMON_COUNTER5_INT_TO_UC_EN_MASK 0x20
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__WB_PERFMON_COUNTER5_INT_TO_UC_EN__SHIFT 0x5
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__WB_PERFMON_COUNTER6_INT_TO_UC_EN_MASK 0x40
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__WB_PERFMON_COUNTER6_INT_TO_UC_EN__SHIFT 0x6
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__WB_PERFMON_COUNTER7_INT_TO_UC_EN_MASK 0x80
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__WB_PERFMON_COUNTER7_INT_TO_UC_EN__SHIFT 0x7
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCRX_PERFMON_COUNTER0_INT_TO_UC_EN_MASK 0x100
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCRX_PERFMON_COUNTER0_INT_TO_UC_EN__SHIFT 0x8
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCRX_PERFMON_COUNTER1_INT_TO_UC_EN_MASK 0x200
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCRX_PERFMON_COUNTER1_INT_TO_UC_EN__SHIFT 0x9
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCRX_PERFMON_COUNTER2_INT_TO_UC_EN_MASK 0x400
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCRX_PERFMON_COUNTER2_INT_TO_UC_EN__SHIFT 0xa
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCRX_PERFMON_COUNTER3_INT_TO_UC_EN_MASK 0x800
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCRX_PERFMON_COUNTER3_INT_TO_UC_EN__SHIFT 0xb
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCRX_PERFMON_COUNTER4_INT_TO_UC_EN_MASK 0x1000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCRX_PERFMON_COUNTER4_INT_TO_UC_EN__SHIFT 0xc
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCRX_PERFMON_COUNTER5_INT_TO_UC_EN_MASK 0x2000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCRX_PERFMON_COUNTER5_INT_TO_UC_EN__SHIFT 0xd
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCRX_PERFMON_COUNTER6_INT_TO_UC_EN_MASK 0x4000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCRX_PERFMON_COUNTER6_INT_TO_UC_EN__SHIFT 0xe
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCRX_PERFMON_COUNTER7_INT_TO_UC_EN_MASK 0x8000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCRX_PERFMON_COUNTER7_INT_TO_UC_EN__SHIFT 0xf
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCCG_PERFMON2_COUNTER0_INT_TO_UC_EN_MASK 0x10000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCCG_PERFMON2_COUNTER0_INT_TO_UC_EN__SHIFT 0x10
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCCG_PERFMON2_COUNTER1_INT_TO_UC_EN_MASK 0x20000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCCG_PERFMON2_COUNTER1_INT_TO_UC_EN__SHIFT 0x11
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCCG_PERFMON2_COUNTER2_INT_TO_UC_EN_MASK 0x40000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCCG_PERFMON2_COUNTER2_INT_TO_UC_EN__SHIFT 0x12
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCCG_PERFMON2_COUNTER3_INT_TO_UC_EN_MASK 0x80000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCCG_PERFMON2_COUNTER3_INT_TO_UC_EN__SHIFT 0x13
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCCG_PERFMON2_COUNTER4_INT_TO_UC_EN_MASK 0x100000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCCG_PERFMON2_COUNTER4_INT_TO_UC_EN__SHIFT 0x14
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCCG_PERFMON2_COUNTER5_INT_TO_UC_EN_MASK 0x200000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCCG_PERFMON2_COUNTER5_INT_TO_UC_EN__SHIFT 0x15
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCCG_PERFMON2_COUNTER6_INT_TO_UC_EN_MASK 0x400000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCCG_PERFMON2_COUNTER6_INT_TO_UC_EN__SHIFT 0x16
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCCG_PERFMON2_COUNTER7_INT_TO_UC_EN_MASK 0x800000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCCG_PERFMON2_COUNTER7_INT_TO_UC_EN__SHIFT 0x17
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__WB_PERFMON_COUNTER_OFF_INT_TO_UC_EN_MASK 0x1000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__WB_PERFMON_COUNTER_OFF_INT_TO_UC_EN__SHIFT 0x18
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCRX_PERFMON_COUNTER_OFF_INT_TO_UC_EN_MASK 0x2000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCRX_PERFMON_COUNTER_OFF_INT_TO_UC_EN__SHIFT 0x19
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCCG_PERFMON2_COUNTER_OFF_INT_TO_UC_EN_MASK 0x4000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK4__DCCG_PERFMON2_COUNTER_OFF_INT_TO_UC_EN__SHIFT 0x1a
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV0_PERFMON_COUNTER0_INT_TO_UC_EN_MASK 0x1
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV0_PERFMON_COUNTER0_INT_TO_UC_EN__SHIFT 0x0
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV0_PERFMON_COUNTER1_INT_TO_UC_EN_MASK 0x2
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV0_PERFMON_COUNTER1_INT_TO_UC_EN__SHIFT 0x1
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV0_PERFMON_COUNTER2_INT_TO_UC_EN_MASK 0x4
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV0_PERFMON_COUNTER2_INT_TO_UC_EN__SHIFT 0x2
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV0_PERFMON_COUNTER3_INT_TO_UC_EN_MASK 0x8
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV0_PERFMON_COUNTER3_INT_TO_UC_EN__SHIFT 0x3
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV0_PERFMON_COUNTER4_INT_TO_UC_EN_MASK 0x10
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV0_PERFMON_COUNTER4_INT_TO_UC_EN__SHIFT 0x4
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV0_PERFMON_COUNTER5_INT_TO_UC_EN_MASK 0x20
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV0_PERFMON_COUNTER5_INT_TO_UC_EN__SHIFT 0x5
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV0_PERFMON_COUNTER6_INT_TO_UC_EN_MASK 0x40
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV0_PERFMON_COUNTER6_INT_TO_UC_EN__SHIFT 0x6
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV0_PERFMON_COUNTER7_INT_TO_UC_EN_MASK 0x80
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV0_PERFMON_COUNTER7_INT_TO_UC_EN__SHIFT 0x7
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV0_PERFMON_COUNTER_OFF_INT_TO_UC_EN_MASK 0x100
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV0_PERFMON_COUNTER_OFF_INT_TO_UC_EN__SHIFT 0x8
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV1_PERFMON_COUNTER0_INT_TO_UC_EN_MASK 0x200
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV1_PERFMON_COUNTER0_INT_TO_UC_EN__SHIFT 0x9
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV1_PERFMON_COUNTER1_INT_TO_UC_EN_MASK 0x400
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV1_PERFMON_COUNTER1_INT_TO_UC_EN__SHIFT 0xa
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV1_PERFMON_COUNTER2_INT_TO_UC_EN_MASK 0x800
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV1_PERFMON_COUNTER2_INT_TO_UC_EN__SHIFT 0xb
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV1_PERFMON_COUNTER3_INT_TO_UC_EN_MASK 0x1000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV1_PERFMON_COUNTER3_INT_TO_UC_EN__SHIFT 0xc
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV1_PERFMON_COUNTER4_INT_TO_UC_EN_MASK 0x2000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV1_PERFMON_COUNTER4_INT_TO_UC_EN__SHIFT 0xd
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV1_PERFMON_COUNTER5_INT_TO_UC_EN_MASK 0x4000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV1_PERFMON_COUNTER5_INT_TO_UC_EN__SHIFT 0xe
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV1_PERFMON_COUNTER6_INT_TO_UC_EN_MASK 0x8000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV1_PERFMON_COUNTER6_INT_TO_UC_EN__SHIFT 0xf
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV1_PERFMON_COUNTER7_INT_TO_UC_EN_MASK 0x10000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV1_PERFMON_COUNTER7_INT_TO_UC_EN__SHIFT 0x10
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV1_PERFMON_COUNTER_OFF_INT_TO_UC_EN_MASK 0x20000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_EN_MASK5__DCFEV1_PERFMON_COUNTER_OFF_INT_TO_UC_EN__SHIFT 0x11
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCI_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL_MASK 0x1
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCI_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL__SHIFT 0x0
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCI_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL_MASK 0x2
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCI_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL__SHIFT 0x1
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCI_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL_MASK 0x4
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCI_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL__SHIFT 0x2
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCI_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL_MASK 0x8
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCI_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL__SHIFT 0x3
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCI_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL_MASK 0x10
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCI_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL__SHIFT 0x4
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCI_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL_MASK 0x20
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCI_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL__SHIFT 0x5
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCI_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL_MASK 0x40
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCI_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL__SHIFT 0x6
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCI_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL_MASK 0x80
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCI_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL__SHIFT 0x7
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCO_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL_MASK 0x100
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCO_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL__SHIFT 0x8
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCO_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL_MASK 0x200
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCO_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL__SHIFT 0x9
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCO_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL_MASK 0x400
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCO_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL__SHIFT 0xa
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCO_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL_MASK 0x800
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCO_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL__SHIFT 0xb
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCO_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL_MASK 0x1000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCO_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL__SHIFT 0xc
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCO_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL_MASK 0x2000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCO_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL__SHIFT 0xd
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCO_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL_MASK 0x4000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCO_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL__SHIFT 0xe
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCO_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL_MASK 0x8000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCO_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL__SHIFT 0xf
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCCG_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL_MASK 0x10000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCCG_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL__SHIFT 0x10
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCCG_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL_MASK 0x20000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCCG_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL__SHIFT 0x11
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCCG_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL_MASK 0x40000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCCG_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL__SHIFT 0x12
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCCG_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL_MASK 0x80000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCCG_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL__SHIFT 0x13
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCCG_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL_MASK 0x100000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCCG_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL__SHIFT 0x14
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCCG_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL_MASK 0x200000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCCG_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL__SHIFT 0x15
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCCG_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL_MASK 0x400000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCCG_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL__SHIFT 0x16
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCCG_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL_MASK 0x800000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCCG_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL__SHIFT 0x17
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCI_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL_MASK 0x1000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCI_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL__SHIFT 0x18
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCO_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL_MASK 0x2000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCO_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL__SHIFT 0x19
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCCG_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL_MASK 0x4000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DCCG_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL__SHIFT 0x1a
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE0_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL_MASK 0x1
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE0_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL__SHIFT 0x0
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE0_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL_MASK 0x2
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE0_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL__SHIFT 0x1
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE0_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL_MASK 0x4
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE0_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL__SHIFT 0x2
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE0_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL_MASK 0x8
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE0_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL__SHIFT 0x3
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE0_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL_MASK 0x10
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE0_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL__SHIFT 0x4
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE0_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL_MASK 0x20
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE0_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL__SHIFT 0x5
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE0_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL_MASK 0x40
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE0_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL__SHIFT 0x6
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE0_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL_MASK 0x80
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE0_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL__SHIFT 0x7
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE1_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL_MASK 0x100
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE1_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL__SHIFT 0x8
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE1_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL_MASK 0x200
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE1_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL__SHIFT 0x9
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE1_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL_MASK 0x400
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE1_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL__SHIFT 0xa
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE1_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL_MASK 0x800
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE1_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL__SHIFT 0xb
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE1_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL_MASK 0x1000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE1_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL__SHIFT 0xc
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE1_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL_MASK 0x2000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE1_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL__SHIFT 0xd
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE1_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL_MASK 0x4000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE1_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL__SHIFT 0xe
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE1_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL_MASK 0x8000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE1_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL__SHIFT 0xf
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE2_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL_MASK 0x10000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE2_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL__SHIFT 0x10
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE2_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL_MASK 0x20000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE2_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL__SHIFT 0x11
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE2_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL_MASK 0x40000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE2_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL__SHIFT 0x12
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE2_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL_MASK 0x80000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE2_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL__SHIFT 0x13
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE2_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL_MASK 0x100000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE2_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL__SHIFT 0x14
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE2_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL_MASK 0x200000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE2_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL__SHIFT 0x15
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE2_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL_MASK 0x400000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE2_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL__SHIFT 0x16
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE2_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL_MASK 0x800000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE2_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL__SHIFT 0x17
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE0_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL_MASK 0x1000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE0_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL__SHIFT 0x18
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE1_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL_MASK 0x2000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE1_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL__SHIFT 0x19
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE2_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL_MASK 0x4000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL2__DCFE2_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL__SHIFT 0x1a
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE3_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL_MASK 0x1
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE3_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL__SHIFT 0x0
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE3_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL_MASK 0x2
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE3_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL__SHIFT 0x1
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE3_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL_MASK 0x4
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE3_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL__SHIFT 0x2
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE3_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL_MASK 0x8
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE3_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL__SHIFT 0x3
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE3_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL_MASK 0x10
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE3_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL__SHIFT 0x4
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE3_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL_MASK 0x20
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE3_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL__SHIFT 0x5
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE3_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL_MASK 0x40
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE3_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL__SHIFT 0x6
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE3_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL_MASK 0x80
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE3_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL__SHIFT 0x7
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE4_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL_MASK 0x100
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE4_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL__SHIFT 0x8
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE4_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL_MASK 0x200
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE4_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL__SHIFT 0x9
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE4_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL_MASK 0x400
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE4_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL__SHIFT 0xa
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE4_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL_MASK 0x800
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE4_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL__SHIFT 0xb
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE4_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL_MASK 0x1000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE4_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL__SHIFT 0xc
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE4_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL_MASK 0x2000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE4_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL__SHIFT 0xd
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE4_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL_MASK 0x4000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE4_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL__SHIFT 0xe
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE4_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL_MASK 0x8000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE4_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL__SHIFT 0xf
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE5_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL_MASK 0x10000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE5_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL__SHIFT 0x10
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE5_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL_MASK 0x20000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE5_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL__SHIFT 0x11
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE5_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL_MASK 0x40000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE5_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL__SHIFT 0x12
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE5_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL_MASK 0x80000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE5_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL__SHIFT 0x13
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE5_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL_MASK 0x100000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE5_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL__SHIFT 0x14
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE5_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL_MASK 0x200000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE5_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL__SHIFT 0x15
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE5_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL_MASK 0x400000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE5_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL__SHIFT 0x16
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE5_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL_MASK 0x800000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE5_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL__SHIFT 0x17
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE3_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL_MASK 0x1000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE3_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL__SHIFT 0x18
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE4_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL_MASK 0x2000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE4_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL__SHIFT 0x19
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE5_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL_MASK 0x4000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL3__DCFE5_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL__SHIFT 0x1a
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__WB_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL_MASK 0x1
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__WB_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL__SHIFT 0x0
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__WB_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL_MASK 0x2
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__WB_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL__SHIFT 0x1
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__WB_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL_MASK 0x4
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__WB_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL__SHIFT 0x2
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__WB_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL_MASK 0x8
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__WB_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL__SHIFT 0x3
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__WB_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL_MASK 0x10
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__WB_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL__SHIFT 0x4
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__WB_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL_MASK 0x20
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__WB_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL__SHIFT 0x5
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__WB_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL_MASK 0x40
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__WB_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL__SHIFT 0x6
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__WB_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL_MASK 0x80
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__WB_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL__SHIFT 0x7
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCRX_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL_MASK 0x100
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCRX_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL__SHIFT 0x8
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCRX_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL_MASK 0x200
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCRX_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL__SHIFT 0x9
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCRX_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL_MASK 0x400
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCRX_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL__SHIFT 0xa
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCRX_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL_MASK 0x800
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCRX_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL__SHIFT 0xb
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCRX_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL_MASK 0x1000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCRX_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL__SHIFT 0xc
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCRX_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL_MASK 0x2000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCRX_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL__SHIFT 0xd
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCRX_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL_MASK 0x4000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCRX_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL__SHIFT 0xe
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCRX_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL_MASK 0x8000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCRX_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL__SHIFT 0xf
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCCG_PERFMON2_COUNTER0_INT_XIRQ_IRQ_SEL_MASK 0x10000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCCG_PERFMON2_COUNTER0_INT_XIRQ_IRQ_SEL__SHIFT 0x10
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCCG_PERFMON2_COUNTER1_INT_XIRQ_IRQ_SEL_MASK 0x20000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCCG_PERFMON2_COUNTER1_INT_XIRQ_IRQ_SEL__SHIFT 0x11
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCCG_PERFMON2_COUNTER2_INT_XIRQ_IRQ_SEL_MASK 0x40000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCCG_PERFMON2_COUNTER2_INT_XIRQ_IRQ_SEL__SHIFT 0x12
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCCG_PERFMON2_COUNTER3_INT_XIRQ_IRQ_SEL_MASK 0x80000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCCG_PERFMON2_COUNTER3_INT_XIRQ_IRQ_SEL__SHIFT 0x13
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCCG_PERFMON2_COUNTER4_INT_XIRQ_IRQ_SEL_MASK 0x100000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCCG_PERFMON2_COUNTER4_INT_XIRQ_IRQ_SEL__SHIFT 0x14
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCCG_PERFMON2_COUNTER5_INT_XIRQ_IRQ_SEL_MASK 0x200000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCCG_PERFMON2_COUNTER5_INT_XIRQ_IRQ_SEL__SHIFT 0x15
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCCG_PERFMON2_COUNTER6_INT_XIRQ_IRQ_SEL_MASK 0x400000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCCG_PERFMON2_COUNTER6_INT_XIRQ_IRQ_SEL__SHIFT 0x16
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCCG_PERFMON2_COUNTER7_INT_XIRQ_IRQ_SEL_MASK 0x800000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCCG_PERFMON2_COUNTER7_INT_XIRQ_IRQ_SEL__SHIFT 0x17
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__WB_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL_MASK 0x1000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__WB_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL__SHIFT 0x18
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCRX_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL_MASK 0x2000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCRX_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL__SHIFT 0x19
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCCG_PERFMON2_COUNTER_OFF_INT_XIRQ_IRQ_SEL_MASK 0x4000000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL4__DCCG_PERFMON2_COUNTER_OFF_INT_XIRQ_IRQ_SEL__SHIFT 0x1a
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV0_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL_MASK 0x1
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV0_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL__SHIFT 0x0
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV0_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL_MASK 0x2
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV0_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL__SHIFT 0x1
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV0_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL_MASK 0x4
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV0_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL__SHIFT 0x2
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV0_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL_MASK 0x8
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV0_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL__SHIFT 0x3
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV0_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL_MASK 0x10
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV0_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL__SHIFT 0x4
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV0_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL_MASK 0x20
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV0_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL__SHIFT 0x5
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV0_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL_MASK 0x40
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV0_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL__SHIFT 0x6
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV0_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL_MASK 0x80
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV0_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL__SHIFT 0x7
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV0_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL_MASK 0x100
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV0_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL__SHIFT 0x8
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV1_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL_MASK 0x200
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV1_PERFMON_COUNTER0_INT_XIRQ_IRQ_SEL__SHIFT 0x9
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV1_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL_MASK 0x400
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV1_PERFMON_COUNTER1_INT_XIRQ_IRQ_SEL__SHIFT 0xa
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV1_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL_MASK 0x800
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV1_PERFMON_COUNTER2_INT_XIRQ_IRQ_SEL__SHIFT 0xb
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV1_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL_MASK 0x1000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV1_PERFMON_COUNTER3_INT_XIRQ_IRQ_SEL__SHIFT 0xc
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV1_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL_MASK 0x2000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV1_PERFMON_COUNTER4_INT_XIRQ_IRQ_SEL__SHIFT 0xd
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV1_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL_MASK 0x4000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV1_PERFMON_COUNTER5_INT_XIRQ_IRQ_SEL__SHIFT 0xe
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV1_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL_MASK 0x8000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV1_PERFMON_COUNTER6_INT_XIRQ_IRQ_SEL__SHIFT 0xf
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV1_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL_MASK 0x10000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV1_PERFMON_COUNTER7_INT_XIRQ_IRQ_SEL__SHIFT 0x10
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV1_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL_MASK 0x20000
+#define DMCU_PERFMON_INTERRUPT_TO_UC_XIRQ_IRQ_SEL5__DCFEV1_PERFMON_COUNTER_OFF_INT_XIRQ_IRQ_SEL__SHIFT 0x11
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD0P0_MSA_RECEIVED_INT_OCCURRED_MASK 0x1
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD0P0_MSA_RECEIVED_INT_OCCURRED__SHIFT 0x0
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD0P0_MSA_RECEIVED_INT_CLEAR_MASK 0x1
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD0P0_MSA_RECEIVED_INT_CLEAR__SHIFT 0x0
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD0P0_VBID_VID_STREAM_STATUS_TOGGLED_INT_OCCURRED_MASK 0x2
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD0P0_VBID_VID_STREAM_STATUS_TOGGLED_INT_OCCURRED__SHIFT 0x1
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD0P0_VBID_VID_STREAM_STATUS_TOGGLED_INT_CLEAR_MASK 0x2
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD0P0_VBID_VID_STREAM_STATUS_TOGGLED_INT_CLEAR__SHIFT 0x1
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD0P0_VERTICAL_INT0_OCCURRED_MASK 0x4
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD0P0_VERTICAL_INT0_OCCURRED__SHIFT 0x2
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD0P0_VERTICAL_INT0_CLEAR_MASK 0x4
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD0P0_VERTICAL_INT0_CLEAR__SHIFT 0x2
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD0P0_VERTICAL_INT1_OCCURRED_MASK 0x8
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD0P0_VERTICAL_INT1_OCCURRED__SHIFT 0x3
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD0P0_VERTICAL_INT1_CLEAR_MASK 0x8
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD0P0_VERTICAL_INT1_CLEAR__SHIFT 0x3
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD0P0_SDP_RECEIVED_INT_OCCURRED_MASK 0x10
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD0P0_SDP_RECEIVED_INT_OCCURRED__SHIFT 0x4
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD0P0_SDP_RECEIVED_INT_CLEAR_MASK 0x10
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD0P0_SDP_RECEIVED_INT_CLEAR__SHIFT 0x4
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD1P0_MSA_RECEIVED_INT_OCCURRED_MASK 0x20
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD1P0_MSA_RECEIVED_INT_OCCURRED__SHIFT 0x5
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD1P0_MSA_RECEIVED_INT_CLEAR_MASK 0x20
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD1P0_MSA_RECEIVED_INT_CLEAR__SHIFT 0x5
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD1P0_VBID_VID_STREAM_STATUS_TOGGLED_INT_OCCURRED_MASK 0x40
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD1P0_VBID_VID_STREAM_STATUS_TOGGLED_INT_OCCURRED__SHIFT 0x6
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD1P0_VBID_VID_STREAM_STATUS_TOGGLED_INT_CLEAR_MASK 0x40
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD1P0_VBID_VID_STREAM_STATUS_TOGGLED_INT_CLEAR__SHIFT 0x6
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD1P0_VERTICAL_INT0_OCCURRED_MASK 0x80
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD1P0_VERTICAL_INT0_OCCURRED__SHIFT 0x7
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD1P0_VERTICAL_INT0_CLEAR_MASK 0x80
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD1P0_VERTICAL_INT0_CLEAR__SHIFT 0x7
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD1P0_VERTICAL_INT1_OCCURRED_MASK 0x100
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD1P0_VERTICAL_INT1_OCCURRED__SHIFT 0x8
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD1P0_VERTICAL_INT1_CLEAR_MASK 0x100
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD1P0_VERTICAL_INT1_CLEAR__SHIFT 0x8
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD1P0_SDP_RECEIVED_INT_OCCURRED_MASK 0x200
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD1P0_SDP_RECEIVED_INT_OCCURRED__SHIFT 0x9
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD1P0_SDP_RECEIVED_INT_CLEAR_MASK 0x200
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_SD1P0_SDP_RECEIVED_INT_CLEAR__SHIFT 0x9
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_BS_INTERVAL_ERROR_THRESH_EXCEEDED_INT_OCCURRED_MASK 0x400
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_BS_INTERVAL_ERROR_THRESH_EXCEEDED_INT_OCCURRED__SHIFT 0xa
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_BS_INTERVAL_ERROR_THRESH_EXCEEDED_INT_CLEAR_MASK 0x400
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_BS_INTERVAL_ERROR_THRESH_EXCEEDED_INT_CLEAR__SHIFT 0xa
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_SR_INTERVAL_ERROR_THRESH_EXCEEDED_INT_OCCURRED_MASK 0x800
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_SR_INTERVAL_ERROR_THRESH_EXCEEDED_INT_OCCURRED__SHIFT 0xb
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_SR_INTERVAL_ERROR_THRESH_EXCEEDED_INT_CLEAR_MASK 0x800
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_SR_INTERVAL_ERROR_THRESH_EXCEEDED_INT_CLEAR__SHIFT 0xb
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_SYMBOL_ERROR_THRESH_EXCEEDED_INT_OCCURRED_MASK 0x1000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_SYMBOL_ERROR_THRESH_EXCEEDED_INT_OCCURRED__SHIFT 0xc
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_SYMBOL_ERROR_THRESH_EXCEEDED_INT_CLEAR_MASK 0x1000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_SYMBOL_ERROR_THRESH_EXCEEDED_INT_CLEAR__SHIFT 0xc
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_DISPARITY_ERROR_THRESH_EXCEEDED_INT_OCCURRED_MASK 0x2000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_DISPARITY_ERROR_THRESH_EXCEEDED_INT_OCCURRED__SHIFT 0xd
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_DISPARITY_ERROR_THRESH_EXCEEDED_INT_CLEAR_MASK 0x2000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_DISPARITY_ERROR_THRESH_EXCEEDED_INT_CLEAR__SHIFT 0xd
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_TRAINING_ERROR_THRESH_EXCEEDED_INT_OCCURRED_MASK 0x4000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_TRAINING_ERROR_THRESH_EXCEEDED_INT_OCCURRED__SHIFT 0xe
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_TRAINING_ERROR_THRESH_EXCEEDED_INT_CLEAR_MASK 0x4000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_TRAINING_ERROR_THRESH_EXCEEDED_INT_CLEAR__SHIFT 0xe
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_TEST_PATTERN_ERROR_THRESH_EXCEEDED_INT_OCCURRED_MASK 0x8000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_TEST_PATTERN_ERROR_THRESH_EXCEEDED_INT_OCCURRED__SHIFT 0xf
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_TEST_PATTERN_ERROR_THRESH_EXCEEDED_INT_CLEAR_MASK 0x8000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_TEST_PATTERN_ERROR_THRESH_EXCEEDED_INT_CLEAR__SHIFT 0xf
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_ECF_ERROR_THRESH_EXCEEDED_INT_OCCURRED_MASK 0x10000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_ECF_ERROR_THRESH_EXCEEDED_INT_OCCURRED__SHIFT 0x10
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_ECF_ERROR_THRESH_EXCEEDED_INT_CLEAR_MASK 0x10000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_ECF_ERROR_THRESH_EXCEEDED_INT_CLEAR__SHIFT 0x10
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_DETECT_SR_LOCK_INT_OCCURRED_MASK 0x20000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_DETECT_SR_LOCK_INT_OCCURRED__SHIFT 0x11
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_DETECT_SR_LOCK_INT_CLEAR_MASK 0x20000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_DETECT_SR_LOCK_INT_CLEAR__SHIFT 0x11
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_LOSS_OF_ALIGN_INT_OCCURRED_MASK 0x40000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_LOSS_OF_ALIGN_INT_OCCURRED__SHIFT 0x12
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_LOSS_OF_ALIGN_INT_CLEAR_MASK 0x40000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_LOSS_OF_ALIGN_INT_CLEAR__SHIFT 0x12
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_LOSS_OF_DESKEW_INT_OCCURRED_MASK 0x80000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_LOSS_OF_DESKEW_INT_OCCURRED__SHIFT 0x13
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_LOSS_OF_DESKEW_INT_CLEAR_MASK 0x80000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_LOSS_OF_DESKEW_INT_CLEAR__SHIFT 0x13
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_EXCESSIVE_ERROR_INT_OCCURRED_MASK 0x100000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_EXCESSIVE_ERROR_INT_OCCURRED__SHIFT 0x14
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_EXCESSIVE_ERROR_INT_CLEAR_MASK 0x100000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_EXCESSIVE_ERROR_INT_CLEAR__SHIFT 0x14
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_DESKEW_FIFO_OVERFLOW_INT_OCCURRED_MASK 0x200000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_DESKEW_FIFO_OVERFLOW_INT_OCCURRED__SHIFT 0x15
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_DESKEW_FIFO_OVERFLOW_INT_CLEAR_MASK 0x200000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_DPHY_P0_DESKEW_FIFO_OVERFLOW_INT_CLEAR__SHIFT 0x15
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_AUX_INT_OCCURRED_MASK 0x400000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_AUX_INT_OCCURRED__SHIFT 0x16
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_AUX_INT_CLEAR_MASK 0x400000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_AUX_INT_CLEAR__SHIFT 0x16
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_I2C_INT_OCCURRED_MASK 0x800000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_I2C_INT_OCCURRED__SHIFT 0x17
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_I2C_INT_CLEAR_MASK 0x800000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_I2C_INT_CLEAR__SHIFT 0x17
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_CPU_INT_OCCURRED_MASK 0x1000000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_CPU_INT_OCCURRED__SHIFT 0x18
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_CPU_INT_CLEAR_MASK 0x1000000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_CPU_INT_CLEAR__SHIFT 0x18
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_MSG1_TIMEOUT_INT_OCCURRED_MASK 0x2000000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_MSG1_TIMEOUT_INT_OCCURRED__SHIFT 0x19
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_MSG1_TIMEOUT_INT_CLEAR_MASK 0x2000000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_MSG1_TIMEOUT_INT_CLEAR__SHIFT 0x19
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_MSG2_TIMEOUT_INT_OCCURRED_MASK 0x4000000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_MSG2_TIMEOUT_INT_OCCURRED__SHIFT 0x1a
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_MSG2_TIMEOUT_INT_CLEAR_MASK 0x4000000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_MSG2_TIMEOUT_INT_CLEAR__SHIFT 0x1a
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_MSG3_TIMEOUT_INT_OCCURRED_MASK 0x8000000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_MSG3_TIMEOUT_INT_OCCURRED__SHIFT 0x1b
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_MSG3_TIMEOUT_INT_CLEAR_MASK 0x8000000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_MSG3_TIMEOUT_INT_CLEAR__SHIFT 0x1b
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_MSG4_TIMEOUT_INT_OCCURRED_MASK 0x10000000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_MSG4_TIMEOUT_INT_OCCURRED__SHIFT 0x1c
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_MSG4_TIMEOUT_INT_CLEAR_MASK 0x10000000
+#define DMCU_DPRX_INTERRUPT_STATUS1__DPRX_AUX_P0_MSG4_TIMEOUT_INT_CLEAR__SHIFT 0x1c
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_SD0P0_MSA_RECEIVED_INT_TO_UC_EN_MASK 0x1
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_SD0P0_MSA_RECEIVED_INT_TO_UC_EN__SHIFT 0x0
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_SD0P0_VBID_VID_STREAM_STATUS_TOGGLED_INT_TO_UC_EN_MASK 0x2
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_SD0P0_VBID_VID_STREAM_STATUS_TOGGLED_INT_TO_UC_EN__SHIFT 0x1
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_SD0P0_VERTICAL_INT0_TO_UC_EN_MASK 0x4
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_SD0P0_VERTICAL_INT0_TO_UC_EN__SHIFT 0x2
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_SD0P0_VERTICAL_INT1_TO_UC_EN_MASK 0x8
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_SD0P0_VERTICAL_INT1_TO_UC_EN__SHIFT 0x3
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_SD0P0_SDP_RECEIVED_INT_TO_UC_EN_MASK 0x10
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_SD0P0_SDP_RECEIVED_INT_TO_UC_EN__SHIFT 0x4
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_SD1P0_MSA_RECEIVED_INT_TO_UC_EN_MASK 0x20
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_SD1P0_MSA_RECEIVED_INT_TO_UC_EN__SHIFT 0x5
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_SD1P0_VBID_VID_STREAM_STATUS_TOGGLED_INT_TO_UC_EN_MASK 0x40
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_SD1P0_VBID_VID_STREAM_STATUS_TOGGLED_INT_TO_UC_EN__SHIFT 0x6
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_SD1P0_VERTICAL_INT0_TO_UC_EN_MASK 0x80
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_SD1P0_VERTICAL_INT0_TO_UC_EN__SHIFT 0x7
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_SD1P0_VERTICAL_INT1_TO_UC_EN_MASK 0x100
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_SD1P0_VERTICAL_INT1_TO_UC_EN__SHIFT 0x8
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_SD1P0_SDP_RECEIVED_INT_TO_UC_EN_MASK 0x200
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_SD1P0_SDP_RECEIVED_INT_TO_UC_EN__SHIFT 0x9
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_BS_INTERVAL_ERROR_THRESH_EXCEEDED_INT_TO_UC_EN_MASK 0x400
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_BS_INTERVAL_ERROR_THRESH_EXCEEDED_INT_TO_UC_EN__SHIFT 0xa
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_SR_INTERVAL_ERROR_THRESH_EXCEEDED_INT_TO_UC_EN_MASK 0x800
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_SR_INTERVAL_ERROR_THRESH_EXCEEDED_INT_TO_UC_EN__SHIFT 0xb
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_SYMBOL_ERROR_THRESH_EXCEEDED_INT_TO_UC_EN_MASK 0x1000
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_SYMBOL_ERROR_THRESH_EXCEEDED_INT_TO_UC_EN__SHIFT 0xc
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_DISPARITY_ERROR_THRESH_EXCEEDED_INT_TO_UC_EN_MASK 0x2000
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_DISPARITY_ERROR_THRESH_EXCEEDED_INT_TO_UC_EN__SHIFT 0xd
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_TRAINING_ERROR_THRESH_EXCEEDED_INT_TO_UC_EN_MASK 0x4000
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_TRAINING_ERROR_THRESH_EXCEEDED_INT_TO_UC_EN__SHIFT 0xe
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_TEST_PATTERN_ERROR_THRESH_EXCEEDED_INT_TO_UC_EN_MASK 0x8000
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_TEST_PATTERN_ERROR_THRESH_EXCEEDED_INT_TO_UC_EN__SHIFT 0xf
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_ECF_ERROR_THRESH_EXCEEDED_INT_TO_UC_EN_MASK 0x10000
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_ECF_ERROR_THRESH_EXCEEDED_INT_TO_UC_EN__SHIFT 0x10
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_DETECT_SR_LOCK_INT_TO_UC_EN_MASK 0x20000
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_DETECT_SR_LOCK_INT_TO_UC_EN__SHIFT 0x11
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_LOSS_OF_ALIGN_INT_TO_UC_EN_MASK 0x40000
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_LOSS_OF_ALIGN_INT_TO_UC_EN__SHIFT 0x12
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_LOSS_OF_DESKEW_INT_TO_UC_EN_MASK 0x80000
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_LOSS_OF_DESKEW_INT_TO_UC_EN__SHIFT 0x13
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_EXCESSIVE_ERROR_INT_TO_UC_EN_MASK 0x100000
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_EXCESSIVE_ERROR_INT_TO_UC_EN__SHIFT 0x14
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_DESKEW_FIFO_OVERFLOW_INT_TO_UC_EN_MASK 0x200000
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_DPHY_P0_DESKEW_FIFO_OVERFLOW_INT_TO_UC_EN__SHIFT 0x15
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_AUX_P0_AUX_INT_TO_UC_EN_MASK 0x400000
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_AUX_P0_AUX_INT_TO_UC_EN__SHIFT 0x16
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_AUX_P0_I2C_INT_TO_UC_EN_MASK 0x800000
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_AUX_P0_I2C_INT_TO_UC_EN__SHIFT 0x17
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_AUX_P0_CPU_INT_TO_UC_EN_MASK 0x1000000
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_AUX_P0_CPU_INT_TO_UC_EN__SHIFT 0x18
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_AUX_P0_MSG1_TIMEOUT_INT_TO_UC_EN_MASK 0x2000000
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_AUX_P0_MSG1_TIMEOUT_INT_TO_UC_EN__SHIFT 0x19
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_AUX_P0_MSG2_TIMEOUT_INT_TO_UC_EN_MASK 0x4000000
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_AUX_P0_MSG2_TIMEOUT_INT_TO_UC_EN__SHIFT 0x1a
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_AUX_P0_MSG3_TIMEOUT_INT_TO_UC_EN_MASK 0x8000000
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_AUX_P0_MSG3_TIMEOUT_INT_TO_UC_EN__SHIFT 0x1b
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_AUX_P0_MSG4_TIMEOUT_INT_TO_UC_EN_MASK 0x10000000
+#define DMCU_DPRX_INTERRUPT_TO_UC_EN_MASK1__DPRX_AUX_P0_MSG4_TIMEOUT_INT_TO_UC_EN__SHIFT 0x1c
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_SD0P0_MSA_RECEIVED_INT_XIRQ_IRQ_SEL_MASK 0x1
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_SD0P0_MSA_RECEIVED_INT_XIRQ_IRQ_SEL__SHIFT 0x0
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_SD0P0_VBID_VID_STREAM_STATUS_TOGGLED_INT_XIRQ_IRQ_SEL_MASK 0x2
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_SD0P0_VBID_VID_STREAM_STATUS_TOGGLED_INT_XIRQ_IRQ_SEL__SHIFT 0x1
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_SD0P0_VERTICAL_INT0_XIRQ_IRQ_SEL_MASK 0x4
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_SD0P0_VERTICAL_INT0_XIRQ_IRQ_SEL__SHIFT 0x2
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_SD0P0_VERTICAL_INT1_XIRQ_IRQ_SEL_MASK 0x8
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_SD0P0_VERTICAL_INT1_XIRQ_IRQ_SEL__SHIFT 0x3
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_SD0P0_SDP_RECEIVED_INT_XIRQ_IRQ_SEL_MASK 0x10
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_SD0P0_SDP_RECEIVED_INT_XIRQ_IRQ_SEL__SHIFT 0x4
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_SD1P0_MSA_RECEIVED_INT_XIRQ_IRQ_SEL_MASK 0x20
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_SD1P0_MSA_RECEIVED_INT_XIRQ_IRQ_SEL__SHIFT 0x5
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_SD1P0_VBID_VID_STREAM_STATUS_TOGGLED_INT_XIRQ_IRQ_SEL_MASK 0x40
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_SD1P0_VBID_VID_STREAM_STATUS_TOGGLED_INT_XIRQ_IRQ_SEL__SHIFT 0x6
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_SD1P0_VERTICAL_INT0_XIRQ_IRQ_SEL_MASK 0x80
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_SD1P0_VERTICAL_INT0_XIRQ_IRQ_SEL__SHIFT 0x7
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_SD1P0_VERTICAL_INT1_XIRQ_IRQ_SEL_MASK 0x100
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_SD1P0_VERTICAL_INT1_XIRQ_IRQ_SEL__SHIFT 0x8
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_SD1P0_SDP_RECEIVED_INT_XIRQ_IRQ_SEL_MASK 0x200
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_SD1P0_SDP_RECEIVED_INT_XIRQ_IRQ_SEL__SHIFT 0x9
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_BS_INTERVAL_ERROR_THRESH_EXCEEDED_INT_XIRQ_IRQ_SEL_MASK 0x400
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_BS_INTERVAL_ERROR_THRESH_EXCEEDED_INT_XIRQ_IRQ_SEL__SHIFT 0xa
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_SR_INTERVAL_ERROR_THRESH_EXCEEDED_INT_XIRQ_IRQ_SEL_MASK 0x800
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_SR_INTERVAL_ERROR_THRESH_EXCEEDED_INT_XIRQ_IRQ_SEL__SHIFT 0xb
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_SYMBOL_ERROR_THRESH_EXCEEDED_INT_XIRQ_IRQ_SEL_MASK 0x1000
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_SYMBOL_ERROR_THRESH_EXCEEDED_INT_XIRQ_IRQ_SEL__SHIFT 0xc
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_DISPARITY_ERROR_THRESH_EXCEEDED_INT_XIRQ_IRQ_SEL_MASK 0x2000
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_DISPARITY_ERROR_THRESH_EXCEEDED_INT_XIRQ_IRQ_SEL__SHIFT 0xd
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_TRAINING_ERROR_THRESH_EXCEEDED_INT_XIRQ_IRQ_SEL_MASK 0x4000
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_TRAINING_ERROR_THRESH_EXCEEDED_INT_XIRQ_IRQ_SEL__SHIFT 0xe
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_TEST_PATTERN_ERROR_THRESH_EXCEEDED_INT_XIRQ_IRQ_SEL_MASK 0x8000
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_TEST_PATTERN_ERROR_THRESH_EXCEEDED_INT_XIRQ_IRQ_SEL__SHIFT 0xf
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_ECF_ERROR_THRESH_EXCEEDED_INT_XIRQ_IRQ_SEL_MASK 0x10000
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_ECF_ERROR_THRESH_EXCEEDED_INT_XIRQ_IRQ_SEL__SHIFT 0x10
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_DETECT_SR_LOCK_INT_XIRQ_IRQ_SEL_MASK 0x20000
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_DETECT_SR_LOCK_INT_XIRQ_IRQ_SEL__SHIFT 0x11
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_LOSS_OF_ALIGN_INT_XIRQ_IRQ_SEL_MASK 0x40000
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_LOSS_OF_ALIGN_INT_XIRQ_IRQ_SEL__SHIFT 0x12
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_LOSS_OF_DESKEW_INT_XIRQ_IRQ_SEL_MASK 0x80000
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_LOSS_OF_DESKEW_INT_XIRQ_IRQ_SEL__SHIFT 0x13
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_EXCESSIVE_ERROR_INT_XIRQ_IRQ_SEL_MASK 0x100000
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_EXCESSIVE_ERROR_INT_XIRQ_IRQ_SEL__SHIFT 0x14
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_DESKEW_FIFO_OVERFLOW_INT_XIRQ_IRQ_SEL_MASK 0x200000
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_DPHY_P0_DESKEW_FIFO_OVERFLOW_INT_XIRQ_IRQ_SEL__SHIFT 0x15
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_AUX_P0_AUX_INT_XIRQ_IRQ_SEL_MASK 0x400000
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_AUX_P0_AUX_INT_XIRQ_IRQ_SEL__SHIFT 0x16
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_AUX_P0_I2C_INT_XIRQ_IRQ_SEL_MASK 0x800000
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_AUX_P0_I2C_INT_XIRQ_IRQ_SEL__SHIFT 0x17
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_AUX_P0_CPU_INT_XIRQ_IRQ_SEL_MASK 0x1000000
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_AUX_P0_CPU_INT_XIRQ_IRQ_SEL__SHIFT 0x18
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_AUX_P0_MSG1_TIMEOUT_INT_XIRQ_IRQ_SEL_MASK 0x2000000
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_AUX_P0_MSG1_TIMEOUT_INT_XIRQ_IRQ_SEL__SHIFT 0x19
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_AUX_P0_MSG2_TIMEOUT_INT_XIRQ_IRQ_SEL_MASK 0x4000000
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_AUX_P0_MSG2_TIMEOUT_INT_XIRQ_IRQ_SEL__SHIFT 0x1a
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_AUX_P0_MSG3_TIMEOUT_INT_XIRQ_IRQ_SEL_MASK 0x8000000
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_AUX_P0_MSG3_TIMEOUT_INT_XIRQ_IRQ_SEL__SHIFT 0x1b
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_AUX_P0_MSG4_TIMEOUT_INT_XIRQ_IRQ_SEL_MASK 0x10000000
+#define DMCU_DPRX_INTERRUPT_TO_UC_XIRQ_IRQ_SEL1__DPRX_AUX_P0_MSG4_TIMEOUT_INT_XIRQ_IRQ_SEL__SHIFT 0x1c
+#define DP_LINK_CNTL__DP_LINK_TRAINING_COMPLETE_MASK 0x10
+#define DP_LINK_CNTL__DP_LINK_TRAINING_COMPLETE__SHIFT 0x4
+#define DP_LINK_CNTL__DP_LINK_STATUS_MASK 0x100
+#define DP_LINK_CNTL__DP_LINK_STATUS__SHIFT 0x8
+#define DP_LINK_CNTL__DP_EMBEDDED_PANEL_MODE_MASK 0x20000
+#define DP_LINK_CNTL__DP_EMBEDDED_PANEL_MODE__SHIFT 0x11
+#define DP_PIXEL_FORMAT__DP_PIXEL_ENCODING_MASK 0x7
+#define DP_PIXEL_FORMAT__DP_PIXEL_ENCODING__SHIFT 0x0
+#define DP_PIXEL_FORMAT__DP_DYN_RANGE_MASK 0x100
+#define DP_PIXEL_FORMAT__DP_DYN_RANGE__SHIFT 0x8
+#define DP_PIXEL_FORMAT__DP_YCBCR_RANGE_MASK 0x10000
+#define DP_PIXEL_FORMAT__DP_YCBCR_RANGE__SHIFT 0x10
+#define DP_PIXEL_FORMAT__DP_COMPONENT_DEPTH_MASK 0x7000000
+#define DP_PIXEL_FORMAT__DP_COMPONENT_DEPTH__SHIFT 0x18
+#define DP_MSA_COLORIMETRY__DP_MSA_MISC0_OVERRIDE_MASK 0xff
+#define DP_MSA_COLORIMETRY__DP_MSA_MISC0_OVERRIDE__SHIFT 0x0
+#define DP_MSA_COLORIMETRY__DP_MSA_MISC0_OVERRIDE_ENABLE_MASK 0x100
+#define DP_MSA_COLORIMETRY__DP_MSA_MISC0_OVERRIDE_ENABLE__SHIFT 0x8
+#define DP_MSA_COLORIMETRY__DP_MSA_MISC1_BIT7_OVERRIDE_MASK 0x200
+#define DP_MSA_COLORIMETRY__DP_MSA_MISC1_BIT7_OVERRIDE__SHIFT 0x9
+#define DP_MSA_COLORIMETRY__DP_MSA_MISC1_BIT7_OVERRIDE_ENABLE_MASK 0x20000
+#define DP_MSA_COLORIMETRY__DP_MSA_MISC1_BIT7_OVERRIDE_ENABLE__SHIFT 0x11
+#define DP_CONFIG__DP_UDI_LANES_MASK 0x3
+#define DP_CONFIG__DP_UDI_LANES__SHIFT 0x0
+#define DP_VID_STREAM_CNTL__DP_VID_STREAM_ENABLE_MASK 0x1
+#define DP_VID_STREAM_CNTL__DP_VID_STREAM_ENABLE__SHIFT 0x0
+#define DP_VID_STREAM_CNTL__DP_VID_STREAM_DIS_DEFER_MASK 0x300
+#define DP_VID_STREAM_CNTL__DP_VID_STREAM_DIS_DEFER__SHIFT 0x8
+#define DP_VID_STREAM_CNTL__DP_VID_STREAM_STATUS_MASK 0x10000
+#define DP_VID_STREAM_CNTL__DP_VID_STREAM_STATUS__SHIFT 0x10
+#define DP_VID_STREAM_CNTL__DP_VID_STREAM_CHANGE_KEEPOUT_MASK 0x100000
+#define DP_VID_STREAM_CNTL__DP_VID_STREAM_CHANGE_KEEPOUT__SHIFT 0x14
+#define DP_STEER_FIFO__DP_STEER_FIFO_RESET_MASK 0x1
+#define DP_STEER_FIFO__DP_STEER_FIFO_RESET__SHIFT 0x0
+#define DP_STEER_FIFO__DP_STEER_OVERFLOW_FLAG_MASK 0x10
+#define DP_STEER_FIFO__DP_STEER_OVERFLOW_FLAG__SHIFT 0x4
+#define DP_STEER_FIFO__DP_STEER_OVERFLOW_INT_MASK 0x20
+#define DP_STEER_FIFO__DP_STEER_OVERFLOW_INT__SHIFT 0x5
+#define DP_STEER_FIFO__DP_STEER_OVERFLOW_ACK_MASK 0x40
+#define DP_STEER_FIFO__DP_STEER_OVERFLOW_ACK__SHIFT 0x6
+#define DP_STEER_FIFO__DP_STEER_OVERFLOW_MASK_MASK 0x80
+#define DP_STEER_FIFO__DP_STEER_OVERFLOW_MASK__SHIFT 0x7
+#define DP_STEER_FIFO__DP_TU_OVERFLOW_FLAG_MASK 0x100
+#define DP_STEER_FIFO__DP_TU_OVERFLOW_FLAG__SHIFT 0x8
+#define DP_STEER_FIFO__DP_TU_OVERFLOW_ACK_MASK 0x1000
+#define DP_STEER_FIFO__DP_TU_OVERFLOW_ACK__SHIFT 0xc
+#define DP_MSA_MISC__DP_MSA_MISC1_MASK 0x78
+#define DP_MSA_MISC__DP_MSA_MISC1__SHIFT 0x3
+#define DP_MSA_MISC__DP_MSA_MISC2_MASK 0xff00
+#define DP_MSA_MISC__DP_MSA_MISC2__SHIFT 0x8
+#define DP_MSA_MISC__DP_MSA_MISC3_MASK 0xff0000
+#define DP_MSA_MISC__DP_MSA_MISC3__SHIFT 0x10
+#define DP_MSA_MISC__DP_MSA_MISC4_MASK 0xff000000
+#define DP_MSA_MISC__DP_MSA_MISC4__SHIFT 0x18
+#define DP_VID_TIMING__DP_VID_TIMING_MODE_MASK 0x1
+#define DP_VID_TIMING__DP_VID_TIMING_MODE__SHIFT 0x0
+#define DP_VID_TIMING__DP_VID_M_N_DOUBLE_BUFFER_MODE_MASK 0x10
+#define DP_VID_TIMING__DP_VID_M_N_DOUBLE_BUFFER_MODE__SHIFT 0x4
+#define DP_VID_TIMING__DP_VID_M_N_GEN_EN_MASK 0x100
+#define DP_VID_TIMING__DP_VID_M_N_GEN_EN__SHIFT 0x8
+#define DP_VID_TIMING__DP_VID_M_DOUBLE_VALUE_EN_MASK 0x200
+#define DP_VID_TIMING__DP_VID_M_DOUBLE_VALUE_EN__SHIFT 0x9
+#define DP_VID_TIMING__DP_VID_N_DIV_MASK 0xff000000
+#define DP_VID_TIMING__DP_VID_N_DIV__SHIFT 0x18
+#define DP_VID_N__DP_VID_N_MASK 0xffffff
+#define DP_VID_N__DP_VID_N__SHIFT 0x0
+#define DP_VID_M__DP_VID_M_MASK 0xffffff
+#define DP_VID_M__DP_VID_M__SHIFT 0x0
+#define DP_LINK_FRAMING_CNTL__DP_IDLE_BS_INTERVAL_MASK 0x3ffff
+#define DP_LINK_FRAMING_CNTL__DP_IDLE_BS_INTERVAL__SHIFT 0x0
+#define DP_LINK_FRAMING_CNTL__DP_VBID_DISABLE_MASK 0x1000000
+#define DP_LINK_FRAMING_CNTL__DP_VBID_DISABLE__SHIFT 0x18
+#define DP_LINK_FRAMING_CNTL__DP_VID_ENHANCED_FRAME_MODE_MASK 0x10000000
+#define DP_LINK_FRAMING_CNTL__DP_VID_ENHANCED_FRAME_MODE__SHIFT 0x1c
+#define DP_HBR2_EYE_PATTERN__DP_HBR2_EYE_PATTERN_ENABLE_MASK 0x1
+#define DP_HBR2_EYE_PATTERN__DP_HBR2_EYE_PATTERN_ENABLE__SHIFT 0x0
+#define DP_VID_MSA_VBID__DP_VID_MSA_LOCATION_MASK 0xfff
+#define DP_VID_MSA_VBID__DP_VID_MSA_LOCATION__SHIFT 0x0
+#define DP_VID_MSA_VBID__DP_VID_MSA_TOP_FIELD_MODE_MASK 0x10000
+#define DP_VID_MSA_VBID__DP_VID_MSA_TOP_FIELD_MODE__SHIFT 0x10
+#define DP_VID_MSA_VBID__DP_VID_VBID_FIELD_POL_MASK 0x1000000
+#define DP_VID_MSA_VBID__DP_VID_VBID_FIELD_POL__SHIFT 0x18
+#define DP_VID_INTERRUPT_CNTL__DP_VID_STREAM_DISABLE_INT_MASK 0x1
+#define DP_VID_INTERRUPT_CNTL__DP_VID_STREAM_DISABLE_INT__SHIFT 0x0
+#define DP_VID_INTERRUPT_CNTL__DP_VID_STREAM_DISABLE_ACK_MASK 0x2
+#define DP_VID_INTERRUPT_CNTL__DP_VID_STREAM_DISABLE_ACK__SHIFT 0x1
+#define DP_VID_INTERRUPT_CNTL__DP_VID_STREAM_DISABLE_MASK_MASK 0x4
+#define DP_VID_INTERRUPT_CNTL__DP_VID_STREAM_DISABLE_MASK__SHIFT 0x2
+#define DP_DPHY_CNTL__DPHY_ATEST_SEL_LANE0_MASK 0x1
+#define DP_DPHY_CNTL__DPHY_ATEST_SEL_LANE0__SHIFT 0x0
+#define DP_DPHY_CNTL__DPHY_ATEST_SEL_LANE1_MASK 0x2
+#define DP_DPHY_CNTL__DPHY_ATEST_SEL_LANE1__SHIFT 0x1
+#define DP_DPHY_CNTL__DPHY_ATEST_SEL_LANE2_MASK 0x4
+#define DP_DPHY_CNTL__DPHY_ATEST_SEL_LANE2__SHIFT 0x2
+#define DP_DPHY_CNTL__DPHY_ATEST_SEL_LANE3_MASK 0x8
+#define DP_DPHY_CNTL__DPHY_ATEST_SEL_LANE3__SHIFT 0x3
+#define DP_DPHY_CNTL__DPHY_BYPASS_MASK 0x10000
+#define DP_DPHY_CNTL__DPHY_BYPASS__SHIFT 0x10
+#define DP_DPHY_CNTL__DPHY_SKEW_BYPASS_MASK 0x1000000
+#define DP_DPHY_CNTL__DPHY_SKEW_BYPASS__SHIFT 0x18
+#define DP_DPHY_TRAINING_PATTERN_SEL__DPHY_TRAINING_PATTERN_SEL_MASK 0x3
+#define DP_DPHY_TRAINING_PATTERN_SEL__DPHY_TRAINING_PATTERN_SEL__SHIFT 0x0
+#define DP_DPHY_SYM0__DPHY_SYM1_MASK 0x3ff
+#define DP_DPHY_SYM0__DPHY_SYM1__SHIFT 0x0
+#define DP_DPHY_SYM0__DPHY_SYM2_MASK 0xffc00
+#define DP_DPHY_SYM0__DPHY_SYM2__SHIFT 0xa
+#define DP_DPHY_SYM0__DPHY_SYM3_MASK 0x3ff00000
+#define DP_DPHY_SYM0__DPHY_SYM3__SHIFT 0x14
+#define DP_DPHY_SYM1__DPHY_SYM4_MASK 0x3ff
+#define DP_DPHY_SYM1__DPHY_SYM4__SHIFT 0x0
+#define DP_DPHY_SYM1__DPHY_SYM5_MASK 0xffc00
+#define DP_DPHY_SYM1__DPHY_SYM5__SHIFT 0xa
+#define DP_DPHY_SYM1__DPHY_SYM6_MASK 0x3ff00000
+#define DP_DPHY_SYM1__DPHY_SYM6__SHIFT 0x14
+#define DP_DPHY_SYM2__DPHY_SYM7_MASK 0x3ff
+#define DP_DPHY_SYM2__DPHY_SYM7__SHIFT 0x0
+#define DP_DPHY_SYM2__DPHY_SYM8_MASK 0xffc00
+#define DP_DPHY_SYM2__DPHY_SYM8__SHIFT 0xa
+#define DP_DPHY_8B10B_CNTL__DPHY_8B10B_RESET_MASK 0x100
+#define DP_DPHY_8B10B_CNTL__DPHY_8B10B_RESET__SHIFT 0x8
+#define DP_DPHY_8B10B_CNTL__DPHY_8B10B_EXT_DISP_MASK 0x10000
+#define DP_DPHY_8B10B_CNTL__DPHY_8B10B_EXT_DISP__SHIFT 0x10
+#define DP_DPHY_8B10B_CNTL__DPHY_8B10B_CUR_DISP_MASK 0x1000000
+#define DP_DPHY_8B10B_CNTL__DPHY_8B10B_CUR_DISP__SHIFT 0x18
+#define DP_DPHY_PRBS_CNTL__DPHY_PRBS_EN_MASK 0x1
+#define DP_DPHY_PRBS_CNTL__DPHY_PRBS_EN__SHIFT 0x0
+#define DP_DPHY_PRBS_CNTL__DPHY_PRBS_SEL_MASK 0x30
+#define DP_DPHY_PRBS_CNTL__DPHY_PRBS_SEL__SHIFT 0x4
+#define DP_DPHY_PRBS_CNTL__DPHY_PRBS_SEED_MASK 0x7fffff00
+#define DP_DPHY_PRBS_CNTL__DPHY_PRBS_SEED__SHIFT 0x8
+#define DP_DPHY_BS_SR_SWAP_CNTL__DPHY_LOAD_BS_COUNT_MASK 0x3ff
+#define DP_DPHY_BS_SR_SWAP_CNTL__DPHY_LOAD_BS_COUNT__SHIFT 0x0
+#define DP_DPHY_BS_SR_SWAP_CNTL__DPHY_BS_SR_SWAP_DONE_MASK 0x8000
+#define DP_DPHY_BS_SR_SWAP_CNTL__DPHY_BS_SR_SWAP_DONE__SHIFT 0xf
+#define DP_DPHY_BS_SR_SWAP_CNTL__DPHY_LOAD_BS_COUNT_START_MASK 0x10000
+#define DP_DPHY_BS_SR_SWAP_CNTL__DPHY_LOAD_BS_COUNT_START__SHIFT 0x10
+#define DP_DPHY_CRC_EN__DPHY_CRC_EN_MASK 0x1
+#define DP_DPHY_CRC_EN__DPHY_CRC_EN__SHIFT 0x0
+#define DP_DPHY_CRC_EN__DPHY_CRC_CONT_EN_MASK 0x10
+#define DP_DPHY_CRC_EN__DPHY_CRC_CONT_EN__SHIFT 0x4
+#define DP_DPHY_CRC_EN__DPHY_CRC_RESULT_VALID_MASK 0x100
+#define DP_DPHY_CRC_EN__DPHY_CRC_RESULT_VALID__SHIFT 0x8
+#define DP_DPHY_CRC_CNTL__DPHY_CRC_FIELD_MASK 0x1
+#define DP_DPHY_CRC_CNTL__DPHY_CRC_FIELD__SHIFT 0x0
+#define DP_DPHY_CRC_CNTL__DPHY_CRC_SEL_MASK 0x30
+#define DP_DPHY_CRC_CNTL__DPHY_CRC_SEL__SHIFT 0x4
+#define DP_DPHY_CRC_CNTL__DPHY_CRC_MASK_MASK 0xff0000
+#define DP_DPHY_CRC_CNTL__DPHY_CRC_MASK__SHIFT 0x10
+#define DP_DPHY_CRC_RESULT__DPHY_CRC_RESULT_MASK 0xff
+#define DP_DPHY_CRC_RESULT__DPHY_CRC_RESULT__SHIFT 0x0
+#define DP_DPHY_CRC_RESULT__DPHY_CRC_RESULT1_MASK 0xff00
+#define DP_DPHY_CRC_RESULT__DPHY_CRC_RESULT1__SHIFT 0x8
+#define DP_DPHY_CRC_RESULT__DPHY_CRC_RESULT2_MASK 0xff0000
+#define DP_DPHY_CRC_RESULT__DPHY_CRC_RESULT2__SHIFT 0x10
+#define DP_DPHY_CRC_RESULT__DPHY_CRC_RESULT3_MASK 0xff000000
+#define DP_DPHY_CRC_RESULT__DPHY_CRC_RESULT3__SHIFT 0x18
+#define DP_DPHY_CRC_MST_CNTL__DPHY_CRC_MST_FIRST_SLOT_MASK 0x3f
+#define DP_DPHY_CRC_MST_CNTL__DPHY_CRC_MST_FIRST_SLOT__SHIFT 0x0
+#define DP_DPHY_CRC_MST_CNTL__DPHY_CRC_MST_LAST_SLOT_MASK 0x3f00
+#define DP_DPHY_CRC_MST_CNTL__DPHY_CRC_MST_LAST_SLOT__SHIFT 0x8
+#define DP_DPHY_CRC_MST_STATUS__DPHY_CRC_MST_PHASE_LOCK_MASK 0x1
+#define DP_DPHY_CRC_MST_STATUS__DPHY_CRC_MST_PHASE_LOCK__SHIFT 0x0
+#define DP_DPHY_CRC_MST_STATUS__DPHY_CRC_MST_PHASE_ERROR_MASK 0x100
+#define DP_DPHY_CRC_MST_STATUS__DPHY_CRC_MST_PHASE_ERROR__SHIFT 0x8
+#define DP_DPHY_CRC_MST_STATUS__DPHY_CRC_MST_PHASE_ERROR_ACK_MASK 0x10000
+#define DP_DPHY_CRC_MST_STATUS__DPHY_CRC_MST_PHASE_ERROR_ACK__SHIFT 0x10
+#define DP_DPHY_FAST_TRAINING__DPHY_RX_FAST_TRAINING_CAPABLE_MASK 0x1
+#define DP_DPHY_FAST_TRAINING__DPHY_RX_FAST_TRAINING_CAPABLE__SHIFT 0x0
+#define DP_DPHY_FAST_TRAINING__DPHY_SW_FAST_TRAINING_START_MASK 0x2
+#define DP_DPHY_FAST_TRAINING__DPHY_SW_FAST_TRAINING_START__SHIFT 0x1
+#define DP_DPHY_FAST_TRAINING__DPHY_FAST_TRAINING_VBLANK_EDGE_DETECT_EN_MASK 0x4
+#define DP_DPHY_FAST_TRAINING__DPHY_FAST_TRAINING_VBLANK_EDGE_DETECT_EN__SHIFT 0x2
+#define DP_DPHY_FAST_TRAINING__DPHY_FAST_TRAINING_TP1_TIME_MASK 0xfff00
+#define DP_DPHY_FAST_TRAINING__DPHY_FAST_TRAINING_TP1_TIME__SHIFT 0x8
+#define DP_DPHY_FAST_TRAINING__DPHY_FAST_TRAINING_TP2_TIME_MASK 0xfff00000
+#define DP_DPHY_FAST_TRAINING__DPHY_FAST_TRAINING_TP2_TIME__SHIFT 0x14
+#define DP_DPHY_FAST_TRAINING_STATUS__DPHY_FAST_TRAINING_STATE_MASK 0x7
+#define DP_DPHY_FAST_TRAINING_STATUS__DPHY_FAST_TRAINING_STATE__SHIFT 0x0
+#define DP_DPHY_FAST_TRAINING_STATUS__DPHY_FAST_TRAINING_COMPLETE_OCCURRED_MASK 0x10
+#define DP_DPHY_FAST_TRAINING_STATUS__DPHY_FAST_TRAINING_COMPLETE_OCCURRED__SHIFT 0x4
+#define DP_DPHY_FAST_TRAINING_STATUS__DPHY_FAST_TRAINING_COMPLETE_MASK_MASK 0x100
+#define DP_DPHY_FAST_TRAINING_STATUS__DPHY_FAST_TRAINING_COMPLETE_MASK__SHIFT 0x8
+#define DP_DPHY_FAST_TRAINING_STATUS__DPHY_FAST_TRAINING_COMPLETE_ACK_MASK 0x1000
+#define DP_DPHY_FAST_TRAINING_STATUS__DPHY_FAST_TRAINING_COMPLETE_ACK__SHIFT 0xc
+#define DP_DPHY_HBR2_PATTERN_CONTROL__DP_DPHY_HBR2_PATTERN_CONTROL_MASK 0x7
+#define DP_DPHY_HBR2_PATTERN_CONTROL__DP_DPHY_HBR2_PATTERN_CONTROL__SHIFT 0x0
+#define DP_MSA_V_TIMING_OVERRIDE1__DP_MSA_V_TIMING_OVERRIDE_EN_MASK 0x1
+#define DP_MSA_V_TIMING_OVERRIDE1__DP_MSA_V_TIMING_OVERRIDE_EN__SHIFT 0x0
+#define DP_MSA_V_TIMING_OVERRIDE1__DP_MSA_V_TOTAL_OVERRIDE_MASK 0x3fff0
+#define DP_MSA_V_TIMING_OVERRIDE1__DP_MSA_V_TOTAL_OVERRIDE__SHIFT 0x4
+#define DP_MSA_V_TIMING_OVERRIDE2__DP_MSA_V_BLANK_START_OVERRIDE_MASK 0x3fff
+#define DP_MSA_V_TIMING_OVERRIDE2__DP_MSA_V_BLANK_START_OVERRIDE__SHIFT 0x0
+#define DP_MSA_V_TIMING_OVERRIDE2__DP_MSA_V_BLANK_END_OVERRIDE_MASK 0x3fff0000
+#define DP_MSA_V_TIMING_OVERRIDE2__DP_MSA_V_BLANK_END_OVERRIDE__SHIFT 0x10
+#define DP_SEC_CNTL__DP_SEC_STREAM_ENABLE_MASK 0x1
+#define DP_SEC_CNTL__DP_SEC_STREAM_ENABLE__SHIFT 0x0
+#define DP_SEC_CNTL__DP_SEC_ASP_ENABLE_MASK 0x10
+#define DP_SEC_CNTL__DP_SEC_ASP_ENABLE__SHIFT 0x4
+#define DP_SEC_CNTL__DP_SEC_ATP_ENABLE_MASK 0x100
+#define DP_SEC_CNTL__DP_SEC_ATP_ENABLE__SHIFT 0x8
+#define DP_SEC_CNTL__DP_SEC_AIP_ENABLE_MASK 0x1000
+#define DP_SEC_CNTL__DP_SEC_AIP_ENABLE__SHIFT 0xc
+#define DP_SEC_CNTL__DP_SEC_ACM_ENABLE_MASK 0x10000
+#define DP_SEC_CNTL__DP_SEC_ACM_ENABLE__SHIFT 0x10
+#define DP_SEC_CNTL__DP_SEC_GSP0_ENABLE_MASK 0x100000
+#define DP_SEC_CNTL__DP_SEC_GSP0_ENABLE__SHIFT 0x14
+#define DP_SEC_CNTL__DP_SEC_GSP1_ENABLE_MASK 0x200000
+#define DP_SEC_CNTL__DP_SEC_GSP1_ENABLE__SHIFT 0x15
+#define DP_SEC_CNTL__DP_SEC_GSP2_ENABLE_MASK 0x400000
+#define DP_SEC_CNTL__DP_SEC_GSP2_ENABLE__SHIFT 0x16
+#define DP_SEC_CNTL__DP_SEC_GSP3_ENABLE_MASK 0x800000
+#define DP_SEC_CNTL__DP_SEC_GSP3_ENABLE__SHIFT 0x17
+#define DP_SEC_CNTL__DP_SEC_AVI_ENABLE_MASK 0x1000000
+#define DP_SEC_CNTL__DP_SEC_AVI_ENABLE__SHIFT 0x18
+#define DP_SEC_CNTL__DP_SEC_MPG_ENABLE_MASK 0x10000000
+#define DP_SEC_CNTL__DP_SEC_MPG_ENABLE__SHIFT 0x1c
+#define DP_SEC_CNTL1__DP_SEC_ISRC_ENABLE_MASK 0x1
+#define DP_SEC_CNTL1__DP_SEC_ISRC_ENABLE__SHIFT 0x0
+#define DP_SEC_CNTL1__DP_SEC_GSP0_PRIORITY_MASK 0x10
+#define DP_SEC_CNTL1__DP_SEC_GSP0_PRIORITY__SHIFT 0x4
+#define DP_SEC_CNTL1__DP_SEC_GSP0_SEND_MASK 0x20
+#define DP_SEC_CNTL1__DP_SEC_GSP0_SEND__SHIFT 0x5
+#define DP_SEC_CNTL1__DP_SEC_GSP0_SEND_PENDING_MASK 0x40
+#define DP_SEC_CNTL1__DP_SEC_GSP0_SEND_PENDING__SHIFT 0x6
+#define DP_SEC_CNTL1__DP_SEC_GSP0_SEND_DEADLINE_MISSED_MASK 0x80
+#define DP_SEC_CNTL1__DP_SEC_GSP0_SEND_DEADLINE_MISSED__SHIFT 0x7
+#define DP_SEC_CNTL1__DP_SEC_GSP0_LINE_NUM_MASK 0xffff0000
+#define DP_SEC_CNTL1__DP_SEC_GSP0_LINE_NUM__SHIFT 0x10
+#define DP_SEC_FRAMING1__DP_SEC_FRAME_START_LOCATION_MASK 0xfff
+#define DP_SEC_FRAMING1__DP_SEC_FRAME_START_LOCATION__SHIFT 0x0
+#define DP_SEC_FRAMING1__DP_SEC_VBLANK_TRANSMIT_WIDTH_MASK 0xffff0000
+#define DP_SEC_FRAMING1__DP_SEC_VBLANK_TRANSMIT_WIDTH__SHIFT 0x10
+#define DP_SEC_FRAMING2__DP_SEC_START_POSITION_MASK 0xffff
+#define DP_SEC_FRAMING2__DP_SEC_START_POSITION__SHIFT 0x0
+#define DP_SEC_FRAMING2__DP_SEC_HBLANK_TRANSMIT_WIDTH_MASK 0xffff0000
+#define DP_SEC_FRAMING2__DP_SEC_HBLANK_TRANSMIT_WIDTH__SHIFT 0x10
+#define DP_SEC_FRAMING3__DP_SEC_IDLE_FRAME_SIZE_MASK 0x3fff
+#define DP_SEC_FRAMING3__DP_SEC_IDLE_FRAME_SIZE__SHIFT 0x0
+#define DP_SEC_FRAMING3__DP_SEC_IDLE_TRANSMIT_WIDTH_MASK 0xffff0000
+#define DP_SEC_FRAMING3__DP_SEC_IDLE_TRANSMIT_WIDTH__SHIFT 0x10
+#define DP_SEC_FRAMING4__DP_SEC_COLLISION_STATUS_MASK 0x100000
+#define DP_SEC_FRAMING4__DP_SEC_COLLISION_STATUS__SHIFT 0x14
+#define DP_SEC_FRAMING4__DP_SEC_COLLISION_ACK_MASK 0x1000000
+#define DP_SEC_FRAMING4__DP_SEC_COLLISION_ACK__SHIFT 0x18
+#define DP_SEC_FRAMING4__DP_SEC_AUDIO_MUTE_MASK 0x10000000
+#define DP_SEC_FRAMING4__DP_SEC_AUDIO_MUTE__SHIFT 0x1c
+#define DP_SEC_FRAMING4__DP_SEC_AUDIO_MUTE_STATUS_MASK 0x20000000
+#define DP_SEC_FRAMING4__DP_SEC_AUDIO_MUTE_STATUS__SHIFT 0x1d
+#define DP_SEC_AUD_N__DP_SEC_AUD_N_MASK 0xffffff
+#define DP_SEC_AUD_N__DP_SEC_AUD_N__SHIFT 0x0
+#define DP_SEC_AUD_N_READBACK__DP_SEC_AUD_N_READBACK_MASK 0xffffff
+#define DP_SEC_AUD_N_READBACK__DP_SEC_AUD_N_READBACK__SHIFT 0x0
+#define DP_SEC_AUD_M__DP_SEC_AUD_M_MASK 0xffffff
+#define DP_SEC_AUD_M__DP_SEC_AUD_M__SHIFT 0x0
+#define DP_SEC_AUD_M_READBACK__DP_SEC_AUD_M_READBACK_MASK 0xffffff
+#define DP_SEC_AUD_M_READBACK__DP_SEC_AUD_M_READBACK__SHIFT 0x0
+#define DP_SEC_TIMESTAMP__DP_SEC_TIMESTAMP_MODE_MASK 0x1
+#define DP_SEC_TIMESTAMP__DP_SEC_TIMESTAMP_MODE__SHIFT 0x0
+#define DP_SEC_PACKET_CNTL__DP_SEC_ASP_CODING_TYPE_MASK 0xe
+#define DP_SEC_PACKET_CNTL__DP_SEC_ASP_CODING_TYPE__SHIFT 0x1
+#define DP_SEC_PACKET_CNTL__DP_SEC_ASP_PRIORITY_MASK 0x10
+#define DP_SEC_PACKET_CNTL__DP_SEC_ASP_PRIORITY__SHIFT 0x4
+#define DP_SEC_PACKET_CNTL__DP_SEC_VERSION_MASK 0x3f00
+#define DP_SEC_PACKET_CNTL__DP_SEC_VERSION__SHIFT 0x8
+#define DP_SEC_PACKET_CNTL__DP_SEC_ASP_CHANNEL_COUNT_OVERRIDE_MASK 0x10000
+#define DP_SEC_PACKET_CNTL__DP_SEC_ASP_CHANNEL_COUNT_OVERRIDE__SHIFT 0x10
+#define DP_MSE_RATE_CNTL__DP_MSE_RATE_Y_MASK 0x3ffffff
+#define DP_MSE_RATE_CNTL__DP_MSE_RATE_Y__SHIFT 0x0
+#define DP_MSE_RATE_CNTL__DP_MSE_RATE_X_MASK 0xfc000000
+#define DP_MSE_RATE_CNTL__DP_MSE_RATE_X__SHIFT 0x1a
+#define DP_MSE_RATE_UPDATE__DP_MSE_RATE_UPDATE_PENDING_MASK 0x1
+#define DP_MSE_RATE_UPDATE__DP_MSE_RATE_UPDATE_PENDING__SHIFT 0x0
+#define DP_MSE_SAT0__DP_MSE_SAT_SRC0_MASK 0x7
+#define DP_MSE_SAT0__DP_MSE_SAT_SRC0__SHIFT 0x0
+#define DP_MSE_SAT0__DP_MSE_SAT_SLOT_COUNT0_MASK 0x3f00
+#define DP_MSE_SAT0__DP_MSE_SAT_SLOT_COUNT0__SHIFT 0x8
+#define DP_MSE_SAT0__DP_MSE_SAT_SRC1_MASK 0x70000
+#define DP_MSE_SAT0__DP_MSE_SAT_SRC1__SHIFT 0x10
+#define DP_MSE_SAT0__DP_MSE_SAT_SLOT_COUNT1_MASK 0x3f000000
+#define DP_MSE_SAT0__DP_MSE_SAT_SLOT_COUNT1__SHIFT 0x18
+#define DP_MSE_SAT1__DP_MSE_SAT_SRC2_MASK 0x7
+#define DP_MSE_SAT1__DP_MSE_SAT_SRC2__SHIFT 0x0
+#define DP_MSE_SAT1__DP_MSE_SAT_SLOT_COUNT2_MASK 0x3f00
+#define DP_MSE_SAT1__DP_MSE_SAT_SLOT_COUNT2__SHIFT 0x8
+#define DP_MSE_SAT1__DP_MSE_SAT_SRC3_MASK 0x70000
+#define DP_MSE_SAT1__DP_MSE_SAT_SRC3__SHIFT 0x10
+#define DP_MSE_SAT1__DP_MSE_SAT_SLOT_COUNT3_MASK 0x3f000000
+#define DP_MSE_SAT1__DP_MSE_SAT_SLOT_COUNT3__SHIFT 0x18
+#define DP_MSE_SAT2__DP_MSE_SAT_SRC4_MASK 0x7
+#define DP_MSE_SAT2__DP_MSE_SAT_SRC4__SHIFT 0x0
+#define DP_MSE_SAT2__DP_MSE_SAT_SLOT_COUNT4_MASK 0x3f00
+#define DP_MSE_SAT2__DP_MSE_SAT_SLOT_COUNT4__SHIFT 0x8
+#define DP_MSE_SAT2__DP_MSE_SAT_SRC5_MASK 0x70000
+#define DP_MSE_SAT2__DP_MSE_SAT_SRC5__SHIFT 0x10
+#define DP_MSE_SAT2__DP_MSE_SAT_SLOT_COUNT5_MASK 0x3f000000
+#define DP_MSE_SAT2__DP_MSE_SAT_SLOT_COUNT5__SHIFT 0x18
+#define DP_MSE_SAT_UPDATE__DP_MSE_SAT_UPDATE_MASK 0x3
+#define DP_MSE_SAT_UPDATE__DP_MSE_SAT_UPDATE__SHIFT 0x0
+#define DP_MSE_SAT_UPDATE__DP_MSE_16_MTP_KEEPOUT_MASK 0x100
+#define DP_MSE_SAT_UPDATE__DP_MSE_16_MTP_KEEPOUT__SHIFT 0x8
+#define DP_MSE_LINK_TIMING__DP_MSE_LINK_FRAME_MASK 0x3ff
+#define DP_MSE_LINK_TIMING__DP_MSE_LINK_FRAME__SHIFT 0x0
+#define DP_MSE_LINK_TIMING__DP_MSE_LINK_LINE_MASK 0x30000
+#define DP_MSE_LINK_TIMING__DP_MSE_LINK_LINE__SHIFT 0x10
+#define DP_MSE_MISC_CNTL__DP_MSE_BLANK_CODE_MASK 0x1
+#define DP_MSE_MISC_CNTL__DP_MSE_BLANK_CODE__SHIFT 0x0
+#define DP_MSE_MISC_CNTL__DP_MSE_TIMESTAMP_MODE_MASK 0x10
+#define DP_MSE_MISC_CNTL__DP_MSE_TIMESTAMP_MODE__SHIFT 0x4
+#define DP_MSE_MISC_CNTL__DP_MSE_ZERO_ENCODER_MASK 0x100
+#define DP_MSE_MISC_CNTL__DP_MSE_ZERO_ENCODER__SHIFT 0x8
+#define DP_MSE_MISC_CNTL__DP_MSE_OUTPUT_DPDBG_DATA_MASK 0x10000
+#define DP_MSE_MISC_CNTL__DP_MSE_OUTPUT_DPDBG_DATA__SHIFT 0x10
+#define DP_MSE_SAT0_STATUS__DP_MSE_SAT_SRC0_STATUS_MASK 0x7
+#define DP_MSE_SAT0_STATUS__DP_MSE_SAT_SRC0_STATUS__SHIFT 0x0
+#define DP_MSE_SAT0_STATUS__DP_MSE_SAT_SLOT_COUNT0_STATUS_MASK 0x3f00
+#define DP_MSE_SAT0_STATUS__DP_MSE_SAT_SLOT_COUNT0_STATUS__SHIFT 0x8
+#define DP_MSE_SAT0_STATUS__DP_MSE_SAT_SRC1_STATUS_MASK 0x70000
+#define DP_MSE_SAT0_STATUS__DP_MSE_SAT_SRC1_STATUS__SHIFT 0x10
+#define DP_MSE_SAT0_STATUS__DP_MSE_SAT_SLOT_COUNT1_STATUS_MASK 0x3f000000
+#define DP_MSE_SAT0_STATUS__DP_MSE_SAT_SLOT_COUNT1_STATUS__SHIFT 0x18
+#define DP_MSE_SAT1_STATUS__DP_MSE_SAT_SRC2_STATUS_MASK 0x7
+#define DP_MSE_SAT1_STATUS__DP_MSE_SAT_SRC2_STATUS__SHIFT 0x0
+#define DP_MSE_SAT1_STATUS__DP_MSE_SAT_SLOT_COUNT2_STATUS_MASK 0x3f00
+#define DP_MSE_SAT1_STATUS__DP_MSE_SAT_SLOT_COUNT2_STATUS__SHIFT 0x8
+#define DP_MSE_SAT1_STATUS__DP_MSE_SAT_SRC3_STATUS_MASK 0x70000
+#define DP_MSE_SAT1_STATUS__DP_MSE_SAT_SRC3_STATUS__SHIFT 0x10
+#define DP_MSE_SAT1_STATUS__DP_MSE_SAT_SLOT_COUNT3_STATUS_MASK 0x3f000000
+#define DP_MSE_SAT1_STATUS__DP_MSE_SAT_SLOT_COUNT3_STATUS__SHIFT 0x18
+#define DP_MSE_SAT2_STATUS__DP_MSE_SAT_SRC4_STATUS_MASK 0x7
+#define DP_MSE_SAT2_STATUS__DP_MSE_SAT_SRC4_STATUS__SHIFT 0x0
+#define DP_MSE_SAT2_STATUS__DP_MSE_SAT_SLOT_COUNT4_STATUS_MASK 0x3f00
+#define DP_MSE_SAT2_STATUS__DP_MSE_SAT_SLOT_COUNT4_STATUS__SHIFT 0x8
+#define DP_MSE_SAT2_STATUS__DP_MSE_SAT_SRC5_STATUS_MASK 0x70000
+#define DP_MSE_SAT2_STATUS__DP_MSE_SAT_SRC5_STATUS__SHIFT 0x10
+#define DP_MSE_SAT2_STATUS__DP_MSE_SAT_SLOT_COUNT5_STATUS_MASK 0x3f000000
+#define DP_MSE_SAT2_STATUS__DP_MSE_SAT_SLOT_COUNT5_STATUS__SHIFT 0x18
+#define DP_TEST_DEBUG_INDEX__DP_TEST_DEBUG_INDEX_MASK 0xff
+#define DP_TEST_DEBUG_INDEX__DP_TEST_DEBUG_INDEX__SHIFT 0x0
+#define DP_TEST_DEBUG_INDEX__DP_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define DP_TEST_DEBUG_INDEX__DP_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define DP_TEST_DEBUG_DATA__DP_TEST_DEBUG_DATA_MASK 0xffffffff
+#define DP_TEST_DEBUG_DATA__DP_TEST_DEBUG_DATA__SHIFT 0x0
+#define DP_FE_TEST_DEBUG_INDEX__DP_FE_TEST_DEBUG_INDEX_MASK 0xff
+#define DP_FE_TEST_DEBUG_INDEX__DP_FE_TEST_DEBUG_INDEX__SHIFT 0x0
+#define DP_FE_TEST_DEBUG_INDEX__DP_FE_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define DP_FE_TEST_DEBUG_INDEX__DP_FE_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define DP_FE_TEST_DEBUG_DATA__DP_FE_TEST_DEBUG_DATA_MASK 0xffffffff
+#define DP_FE_TEST_DEBUG_DATA__DP_FE_TEST_DEBUG_DATA__SHIFT 0x0
+#define AUX_CONTROL__AUX_EN_MASK 0x1
+#define AUX_CONTROL__AUX_EN__SHIFT 0x0
+#define AUX_CONTROL__AUX_RESET_MASK 0x10
+#define AUX_CONTROL__AUX_RESET__SHIFT 0x4
+#define AUX_CONTROL__AUX_RESET_DONE_MASK 0x20
+#define AUX_CONTROL__AUX_RESET_DONE__SHIFT 0x5
+#define AUX_CONTROL__AUX_LS_READ_EN_MASK 0x100
+#define AUX_CONTROL__AUX_LS_READ_EN__SHIFT 0x8
+#define AUX_CONTROL__AUX_LS_UPDATE_DISABLE_MASK 0x1000
+#define AUX_CONTROL__AUX_LS_UPDATE_DISABLE__SHIFT 0xc
+#define AUX_CONTROL__AUX_IGNORE_HPD_DISCON_MASK 0x10000
+#define AUX_CONTROL__AUX_IGNORE_HPD_DISCON__SHIFT 0x10
+#define AUX_CONTROL__AUX_MODE_DET_EN_MASK 0x40000
+#define AUX_CONTROL__AUX_MODE_DET_EN__SHIFT 0x12
+#define AUX_CONTROL__AUX_HPD_SEL_MASK 0x700000
+#define AUX_CONTROL__AUX_HPD_SEL__SHIFT 0x14
+#define AUX_CONTROL__AUX_IMPCAL_REQ_EN_MASK 0x1000000
+#define AUX_CONTROL__AUX_IMPCAL_REQ_EN__SHIFT 0x18
+#define AUX_CONTROL__AUX_TEST_MODE_MASK 0x10000000
+#define AUX_CONTROL__AUX_TEST_MODE__SHIFT 0x1c
+#define AUX_CONTROL__AUX_DEGLITCH_EN_MASK 0x20000000
+#define AUX_CONTROL__AUX_DEGLITCH_EN__SHIFT 0x1d
+#define AUX_CONTROL__SPARE_0_MASK 0x40000000
+#define AUX_CONTROL__SPARE_0__SHIFT 0x1e
+#define AUX_CONTROL__SPARE_1_MASK 0x80000000
+#define AUX_CONTROL__SPARE_1__SHIFT 0x1f
+#define AUX_SW_CONTROL__AUX_SW_GO_MASK 0x1
+#define AUX_SW_CONTROL__AUX_SW_GO__SHIFT 0x0
+#define AUX_SW_CONTROL__AUX_LS_READ_TRIG_MASK 0x4
+#define AUX_SW_CONTROL__AUX_LS_READ_TRIG__SHIFT 0x2
+#define AUX_SW_CONTROL__AUX_SW_START_DELAY_MASK 0xf0
+#define AUX_SW_CONTROL__AUX_SW_START_DELAY__SHIFT 0x4
+#define AUX_SW_CONTROL__AUX_SW_WR_BYTES_MASK 0x1f0000
+#define AUX_SW_CONTROL__AUX_SW_WR_BYTES__SHIFT 0x10
+#define AUX_ARB_CONTROL__AUX_ARB_PRIORITY_MASK 0x3
+#define AUX_ARB_CONTROL__AUX_ARB_PRIORITY__SHIFT 0x0
+#define AUX_ARB_CONTROL__AUX_REG_RW_CNTL_STATUS_MASK 0xc
+#define AUX_ARB_CONTROL__AUX_REG_RW_CNTL_STATUS__SHIFT 0x2
+#define AUX_ARB_CONTROL__AUX_NO_QUEUED_SW_GO_MASK 0x100
+#define AUX_ARB_CONTROL__AUX_NO_QUEUED_SW_GO__SHIFT 0x8
+#define AUX_ARB_CONTROL__AUX_NO_QUEUED_LS_GO_MASK 0x400
+#define AUX_ARB_CONTROL__AUX_NO_QUEUED_LS_GO__SHIFT 0xa
+#define AUX_ARB_CONTROL__AUX_SW_USE_AUX_REG_REQ_MASK 0x10000
+#define AUX_ARB_CONTROL__AUX_SW_USE_AUX_REG_REQ__SHIFT 0x10
+#define AUX_ARB_CONTROL__AUX_SW_PENDING_USE_AUX_REG_REQ_MASK 0x10000
+#define AUX_ARB_CONTROL__AUX_SW_PENDING_USE_AUX_REG_REQ__SHIFT 0x10
+#define AUX_ARB_CONTROL__AUX_SW_DONE_USING_AUX_REG_MASK 0x20000
+#define AUX_ARB_CONTROL__AUX_SW_DONE_USING_AUX_REG__SHIFT 0x11
+#define AUX_ARB_CONTROL__AUX_DMCU_USE_AUX_REG_REQ_MASK 0x1000000
+#define AUX_ARB_CONTROL__AUX_DMCU_USE_AUX_REG_REQ__SHIFT 0x18
+#define AUX_ARB_CONTROL__AUX_DMCU_PENDING_USE_AUX_REG_REQ_MASK 0x1000000
+#define AUX_ARB_CONTROL__AUX_DMCU_PENDING_USE_AUX_REG_REQ__SHIFT 0x18
+#define AUX_ARB_CONTROL__AUX_DMCU_DONE_USING_AUX_REG_MASK 0x2000000
+#define AUX_ARB_CONTROL__AUX_DMCU_DONE_USING_AUX_REG__SHIFT 0x19
+#define AUX_INTERRUPT_CONTROL__AUX_SW_DONE_INT_MASK 0x1
+#define AUX_INTERRUPT_CONTROL__AUX_SW_DONE_INT__SHIFT 0x0
+#define AUX_INTERRUPT_CONTROL__AUX_SW_DONE_ACK_MASK 0x2
+#define AUX_INTERRUPT_CONTROL__AUX_SW_DONE_ACK__SHIFT 0x1
+#define AUX_INTERRUPT_CONTROL__AUX_SW_DONE_MASK_MASK 0x4
+#define AUX_INTERRUPT_CONTROL__AUX_SW_DONE_MASK__SHIFT 0x2
+#define AUX_INTERRUPT_CONTROL__AUX_LS_DONE_INT_MASK 0x10
+#define AUX_INTERRUPT_CONTROL__AUX_LS_DONE_INT__SHIFT 0x4
+#define AUX_INTERRUPT_CONTROL__AUX_LS_DONE_ACK_MASK 0x20
+#define AUX_INTERRUPT_CONTROL__AUX_LS_DONE_ACK__SHIFT 0x5
+#define AUX_INTERRUPT_CONTROL__AUX_LS_DONE_MASK_MASK 0x40
+#define AUX_INTERRUPT_CONTROL__AUX_LS_DONE_MASK__SHIFT 0x6
+#define AUX_INTERRUPT_CONTROL__AUX_GTC_SYNC_LOCK_DONE_INT_MASK 0x100
+#define AUX_INTERRUPT_CONTROL__AUX_GTC_SYNC_LOCK_DONE_INT__SHIFT 0x8
+#define AUX_INTERRUPT_CONTROL__AUX_GTC_SYNC_LOCK_DONE_ACK_MASK 0x200
+#define AUX_INTERRUPT_CONTROL__AUX_GTC_SYNC_LOCK_DONE_ACK__SHIFT 0x9
+#define AUX_INTERRUPT_CONTROL__AUX_GTC_SYNC_LOCK_DONE_INT_MASK_MASK 0x400
+#define AUX_INTERRUPT_CONTROL__AUX_GTC_SYNC_LOCK_DONE_INT_MASK__SHIFT 0xa
+#define AUX_INTERRUPT_CONTROL__AUX_GTC_SYNC_ERROR_INT_MASK 0x1000
+#define AUX_INTERRUPT_CONTROL__AUX_GTC_SYNC_ERROR_INT__SHIFT 0xc
+#define AUX_INTERRUPT_CONTROL__AUX_GTC_SYNC_ERROR_ACK_MASK 0x2000
+#define AUX_INTERRUPT_CONTROL__AUX_GTC_SYNC_ERROR_ACK__SHIFT 0xd
+#define AUX_INTERRUPT_CONTROL__AUX_GTC_SYNC_ERROR_INT_MASK_MASK 0x4000
+#define AUX_INTERRUPT_CONTROL__AUX_GTC_SYNC_ERROR_INT_MASK__SHIFT 0xe
+#define AUX_SW_STATUS__AUX_SW_DONE_MASK 0x1
+#define AUX_SW_STATUS__AUX_SW_DONE__SHIFT 0x0
+#define AUX_SW_STATUS__AUX_SW_REQ_MASK 0x2
+#define AUX_SW_STATUS__AUX_SW_REQ__SHIFT 0x1
+#define AUX_SW_STATUS__AUX_SW_RX_TIMEOUT_STATE_MASK 0x70
+#define AUX_SW_STATUS__AUX_SW_RX_TIMEOUT_STATE__SHIFT 0x4
+#define AUX_SW_STATUS__AUX_SW_RX_TIMEOUT_MASK 0x80
+#define AUX_SW_STATUS__AUX_SW_RX_TIMEOUT__SHIFT 0x7
+#define AUX_SW_STATUS__AUX_SW_RX_OVERFLOW_MASK 0x100
+#define AUX_SW_STATUS__AUX_SW_RX_OVERFLOW__SHIFT 0x8
+#define AUX_SW_STATUS__AUX_SW_HPD_DISCON_MASK 0x200
+#define AUX_SW_STATUS__AUX_SW_HPD_DISCON__SHIFT 0x9
+#define AUX_SW_STATUS__AUX_SW_RX_PARTIAL_BYTE_MASK 0x400
+#define AUX_SW_STATUS__AUX_SW_RX_PARTIAL_BYTE__SHIFT 0xa
+#define AUX_SW_STATUS__AUX_SW_NON_AUX_MODE_MASK 0x800
+#define AUX_SW_STATUS__AUX_SW_NON_AUX_MODE__SHIFT 0xb
+#define AUX_SW_STATUS__AUX_SW_RX_MIN_COUNT_VIOL_MASK 0x1000
+#define AUX_SW_STATUS__AUX_SW_RX_MIN_COUNT_VIOL__SHIFT 0xc
+#define AUX_SW_STATUS__AUX_SW_RX_INVALID_STOP_MASK 0x4000
+#define AUX_SW_STATUS__AUX_SW_RX_INVALID_STOP__SHIFT 0xe
+#define AUX_SW_STATUS__AUX_SW_RX_SYNC_INVALID_L_MASK 0x20000
+#define AUX_SW_STATUS__AUX_SW_RX_SYNC_INVALID_L__SHIFT 0x11
+#define AUX_SW_STATUS__AUX_SW_RX_SYNC_INVALID_H_MASK 0x40000
+#define AUX_SW_STATUS__AUX_SW_RX_SYNC_INVALID_H__SHIFT 0x12
+#define AUX_SW_STATUS__AUX_SW_RX_INVALID_START_MASK 0x80000
+#define AUX_SW_STATUS__AUX_SW_RX_INVALID_START__SHIFT 0x13
+#define AUX_SW_STATUS__AUX_SW_RX_RECV_NO_DET_MASK 0x100000
+#define AUX_SW_STATUS__AUX_SW_RX_RECV_NO_DET__SHIFT 0x14
+#define AUX_SW_STATUS__AUX_SW_RX_RECV_INVALID_H_MASK 0x400000
+#define AUX_SW_STATUS__AUX_SW_RX_RECV_INVALID_H__SHIFT 0x16
+#define AUX_SW_STATUS__AUX_SW_RX_RECV_INVALID_L_MASK 0x800000
+#define AUX_SW_STATUS__AUX_SW_RX_RECV_INVALID_L__SHIFT 0x17
+#define AUX_SW_STATUS__AUX_SW_REPLY_BYTE_COUNT_MASK 0x1f000000
+#define AUX_SW_STATUS__AUX_SW_REPLY_BYTE_COUNT__SHIFT 0x18
+#define AUX_SW_STATUS__AUX_ARB_STATUS_MASK 0xc0000000
+#define AUX_SW_STATUS__AUX_ARB_STATUS__SHIFT 0x1e
+#define AUX_LS_STATUS__AUX_LS_DONE_MASK 0x1
+#define AUX_LS_STATUS__AUX_LS_DONE__SHIFT 0x0
+#define AUX_LS_STATUS__AUX_LS_REQ_MASK 0x2
+#define AUX_LS_STATUS__AUX_LS_REQ__SHIFT 0x1
+#define AUX_LS_STATUS__AUX_LS_RX_TIMEOUT_STATE_MASK 0x70
+#define AUX_LS_STATUS__AUX_LS_RX_TIMEOUT_STATE__SHIFT 0x4
+#define AUX_LS_STATUS__AUX_LS_RX_TIMEOUT_MASK 0x80
+#define AUX_LS_STATUS__AUX_LS_RX_TIMEOUT__SHIFT 0x7
+#define AUX_LS_STATUS__AUX_LS_RX_OVERFLOW_MASK 0x100
+#define AUX_LS_STATUS__AUX_LS_RX_OVERFLOW__SHIFT 0x8
+#define AUX_LS_STATUS__AUX_LS_HPD_DISCON_MASK 0x200
+#define AUX_LS_STATUS__AUX_LS_HPD_DISCON__SHIFT 0x9
+#define AUX_LS_STATUS__AUX_LS_RX_PARTIAL_BYTE_MASK 0x400
+#define AUX_LS_STATUS__AUX_LS_RX_PARTIAL_BYTE__SHIFT 0xa
+#define AUX_LS_STATUS__AUX_LS_NON_AUX_MODE_MASK 0x800
+#define AUX_LS_STATUS__AUX_LS_NON_AUX_MODE__SHIFT 0xb
+#define AUX_LS_STATUS__AUX_LS_RX_MIN_COUNT_VIOL_MASK 0x1000
+#define AUX_LS_STATUS__AUX_LS_RX_MIN_COUNT_VIOL__SHIFT 0xc
+#define AUX_LS_STATUS__AUX_LS_RX_INVALID_STOP_MASK 0x4000
+#define AUX_LS_STATUS__AUX_LS_RX_INVALID_STOP__SHIFT 0xe
+#define AUX_LS_STATUS__AUX_LS_RX_SYNC_INVALID_L_MASK 0x20000
+#define AUX_LS_STATUS__AUX_LS_RX_SYNC_INVALID_L__SHIFT 0x11
+#define AUX_LS_STATUS__AUX_LS_RX_SYNC_INVALID_H_MASK 0x40000
+#define AUX_LS_STATUS__AUX_LS_RX_SYNC_INVALID_H__SHIFT 0x12
+#define AUX_LS_STATUS__AUX_LS_RX_INVALID_START_MASK 0x80000
+#define AUX_LS_STATUS__AUX_LS_RX_INVALID_START__SHIFT 0x13
+#define AUX_LS_STATUS__AUX_LS_RX_RECV_NO_DET_MASK 0x100000
+#define AUX_LS_STATUS__AUX_LS_RX_RECV_NO_DET__SHIFT 0x14
+#define AUX_LS_STATUS__AUX_LS_RX_RECV_INVALID_H_MASK 0x400000
+#define AUX_LS_STATUS__AUX_LS_RX_RECV_INVALID_H__SHIFT 0x16
+#define AUX_LS_STATUS__AUX_LS_RX_RECV_INVALID_L_MASK 0x800000
+#define AUX_LS_STATUS__AUX_LS_RX_RECV_INVALID_L__SHIFT 0x17
+#define AUX_LS_STATUS__AUX_LS_REPLY_BYTE_COUNT_MASK 0x1f000000
+#define AUX_LS_STATUS__AUX_LS_REPLY_BYTE_COUNT__SHIFT 0x18
+#define AUX_LS_STATUS__AUX_LS_CP_IRQ_MASK 0x20000000
+#define AUX_LS_STATUS__AUX_LS_CP_IRQ__SHIFT 0x1d
+#define AUX_LS_STATUS__AUX_LS_UPDATED_MASK 0x40000000
+#define AUX_LS_STATUS__AUX_LS_UPDATED__SHIFT 0x1e
+#define AUX_LS_STATUS__AUX_LS_UPDATED_ACK_MASK 0x80000000
+#define AUX_LS_STATUS__AUX_LS_UPDATED_ACK__SHIFT 0x1f
+#define AUX_SW_DATA__AUX_SW_DATA_RW_MASK 0x1
+#define AUX_SW_DATA__AUX_SW_DATA_RW__SHIFT 0x0
+#define AUX_SW_DATA__AUX_SW_DATA_MASK 0xff00
+#define AUX_SW_DATA__AUX_SW_DATA__SHIFT 0x8
+#define AUX_SW_DATA__AUX_SW_INDEX_MASK 0x1f0000
+#define AUX_SW_DATA__AUX_SW_INDEX__SHIFT 0x10
+#define AUX_SW_DATA__AUX_SW_AUTOINCREMENT_DISABLE_MASK 0x80000000
+#define AUX_SW_DATA__AUX_SW_AUTOINCREMENT_DISABLE__SHIFT 0x1f
+#define AUX_LS_DATA__AUX_LS_DATA_MASK 0xff00
+#define AUX_LS_DATA__AUX_LS_DATA__SHIFT 0x8
+#define AUX_LS_DATA__AUX_LS_INDEX_MASK 0x1f0000
+#define AUX_LS_DATA__AUX_LS_INDEX__SHIFT 0x10
+#define AUX_DPHY_TX_REF_CONTROL__AUX_TX_REF_SEL_MASK 0x1
+#define AUX_DPHY_TX_REF_CONTROL__AUX_TX_REF_SEL__SHIFT 0x0
+#define AUX_DPHY_TX_REF_CONTROL__AUX_TX_RATE_MASK 0x30
+#define AUX_DPHY_TX_REF_CONTROL__AUX_TX_RATE__SHIFT 0x4
+#define AUX_DPHY_TX_REF_CONTROL__AUX_TX_REF_DIV_MASK 0x1ff0000
+#define AUX_DPHY_TX_REF_CONTROL__AUX_TX_REF_DIV__SHIFT 0x10
+#define AUX_DPHY_TX_CONTROL__AUX_TX_PRECHARGE_LEN_MASK 0x7
+#define AUX_DPHY_TX_CONTROL__AUX_TX_PRECHARGE_LEN__SHIFT 0x0
+#define AUX_DPHY_TX_CONTROL__AUX_TX_PRECHARGE_SYMBOLS_MASK 0x3f00
+#define AUX_DPHY_TX_CONTROL__AUX_TX_PRECHARGE_SYMBOLS__SHIFT 0x8
+#define AUX_DPHY_TX_CONTROL__AUX_MODE_DET_CHECK_DELAY_MASK 0x70000
+#define AUX_DPHY_TX_CONTROL__AUX_MODE_DET_CHECK_DELAY__SHIFT 0x10
+#define AUX_DPHY_RX_CONTROL0__AUX_RX_START_WINDOW_MASK 0x70
+#define AUX_DPHY_RX_CONTROL0__AUX_RX_START_WINDOW__SHIFT 0x4
+#define AUX_DPHY_RX_CONTROL0__AUX_RX_RECEIVE_WINDOW_MASK 0x700
+#define AUX_DPHY_RX_CONTROL0__AUX_RX_RECEIVE_WINDOW__SHIFT 0x8
+#define AUX_DPHY_RX_CONTROL0__AUX_RX_HALF_SYM_DETECT_LEN_MASK 0x3000
+#define AUX_DPHY_RX_CONTROL0__AUX_RX_HALF_SYM_DETECT_LEN__SHIFT 0xc
+#define AUX_DPHY_RX_CONTROL0__AUX_RX_TRANSITION_FILTER_EN_MASK 0x10000
+#define AUX_DPHY_RX_CONTROL0__AUX_RX_TRANSITION_FILTER_EN__SHIFT 0x10
+#define AUX_DPHY_RX_CONTROL0__AUX_RX_ALLOW_BELOW_THRESHOLD_PHASE_DETECT_MASK 0x20000
+#define AUX_DPHY_RX_CONTROL0__AUX_RX_ALLOW_BELOW_THRESHOLD_PHASE_DETECT__SHIFT 0x11
+#define AUX_DPHY_RX_CONTROL0__AUX_RX_ALLOW_BELOW_THRESHOLD_START_MASK 0x40000
+#define AUX_DPHY_RX_CONTROL0__AUX_RX_ALLOW_BELOW_THRESHOLD_START__SHIFT 0x12
+#define AUX_DPHY_RX_CONTROL0__AUX_RX_ALLOW_BELOW_THRESHOLD_STOP_MASK 0x80000
+#define AUX_DPHY_RX_CONTROL0__AUX_RX_ALLOW_BELOW_THRESHOLD_STOP__SHIFT 0x13
+#define AUX_DPHY_RX_CONTROL0__AUX_RX_PHASE_DETECT_LEN_MASK 0x300000
+#define AUX_DPHY_RX_CONTROL0__AUX_RX_PHASE_DETECT_LEN__SHIFT 0x14
+#define AUX_DPHY_RX_CONTROL0__AUX_RX_TIMEOUT_LEN_MASK 0x7000000
+#define AUX_DPHY_RX_CONTROL0__AUX_RX_TIMEOUT_LEN__SHIFT 0x18
+#define AUX_DPHY_RX_CONTROL0__AUX_RX_DETECTION_THRESHOLD_MASK 0x70000000
+#define AUX_DPHY_RX_CONTROL0__AUX_RX_DETECTION_THRESHOLD__SHIFT 0x1c
+#define AUX_DPHY_RX_CONTROL1__AUX_RX_PRECHARGE_SKIP_MASK 0xff
+#define AUX_DPHY_RX_CONTROL1__AUX_RX_PRECHARGE_SKIP__SHIFT 0x0
+#define AUX_DPHY_TX_STATUS__AUX_TX_ACTIVE_MASK 0x1
+#define AUX_DPHY_TX_STATUS__AUX_TX_ACTIVE__SHIFT 0x0
+#define AUX_DPHY_TX_STATUS__AUX_TX_STATE_MASK 0x70
+#define AUX_DPHY_TX_STATUS__AUX_TX_STATE__SHIFT 0x4
+#define AUX_DPHY_TX_STATUS__AUX_TX_HALF_SYM_PERIOD_MASK 0x1ff0000
+#define AUX_DPHY_TX_STATUS__AUX_TX_HALF_SYM_PERIOD__SHIFT 0x10
+#define AUX_DPHY_RX_STATUS__AUX_RX_STATE_MASK 0x7
+#define AUX_DPHY_RX_STATUS__AUX_RX_STATE__SHIFT 0x0
+#define AUX_DPHY_RX_STATUS__AUX_RX_SYNC_VALID_COUNT_MASK 0x1f00
+#define AUX_DPHY_RX_STATUS__AUX_RX_SYNC_VALID_COUNT__SHIFT 0x8
+#define AUX_DPHY_RX_STATUS__AUX_RX_HALF_SYM_PERIOD_FRACT_MASK 0x1f0000
+#define AUX_DPHY_RX_STATUS__AUX_RX_HALF_SYM_PERIOD_FRACT__SHIFT 0x10
+#define AUX_DPHY_RX_STATUS__AUX_RX_HALF_SYM_PERIOD_MASK 0x3fe00000
+#define AUX_DPHY_RX_STATUS__AUX_RX_HALF_SYM_PERIOD__SHIFT 0x15
+#define AUX_GTC_SYNC_ERROR_CONTROL__AUX_GTC_POTENTIAL_ERROR_THRESHOLD_MASK 0x1f
+#define AUX_GTC_SYNC_ERROR_CONTROL__AUX_GTC_POTENTIAL_ERROR_THRESHOLD__SHIFT 0x0
+#define AUX_GTC_SYNC_ERROR_CONTROL__AUX_GTC_DEFINITE_ERROR_THRESHOLD_MASK 0x1f00
+#define AUX_GTC_SYNC_ERROR_CONTROL__AUX_GTC_DEFINITE_ERROR_THRESHOLD__SHIFT 0x8
+#define AUX_GTC_SYNC_ERROR_CONTROL__AUX_GTC_SYNC_LOCK_ACQ_TIMEOUT_LEN_MASK 0x30000
+#define AUX_GTC_SYNC_ERROR_CONTROL__AUX_GTC_SYNC_LOCK_ACQ_TIMEOUT_LEN__SHIFT 0x10
+#define AUX_GTC_SYNC_ERROR_CONTROL__AUX_GTC_SYNC_NUM_RETRY_FOR_LOCK_MAINT_MASK 0x300000
+#define AUX_GTC_SYNC_ERROR_CONTROL__AUX_GTC_SYNC_NUM_RETRY_FOR_LOCK_MAINT__SHIFT 0x14
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_LOCK_ACQ_COMPLETE_MASK 0x1
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_LOCK_ACQ_COMPLETE__SHIFT 0x0
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_LOCK_LOST_MASK 0x10
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_LOCK_LOST__SHIFT 0x4
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_LOCK_ACQ_TIMEOUT_OCCURRED_MASK 0x100
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_LOCK_ACQ_TIMEOUT_OCCURRED__SHIFT 0x8
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_LOCK_ACQ_TIMEOUT_STATE_MASK 0x1e00
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_LOCK_ACQ_TIMEOUT_STATE__SHIFT 0x9
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_PHASE_ADJUST_TIME_VIOL_MASK 0x10000
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_PHASE_ADJUST_TIME_VIOL__SHIFT 0x10
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_CRITICAL_ERR_OCCURRED_MASK 0x100000
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_CRITICAL_ERR_OCCURRED__SHIFT 0x14
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_CRITICAL_ERR_OCCURRED_ACK_MASK 0x200000
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_CRITICAL_ERR_OCCURRED_ACK__SHIFT 0x15
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_MAX_POTENTIAL_ERR_REACHED_MASK 0x400000
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_MAX_POTENTIAL_ERR_REACHED__SHIFT 0x16
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_MAX_POTENTIAL_ERR_REACHED_ACK_MASK 0x800000
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_MAX_POTENTIAL_ERR_REACHED_ACK__SHIFT 0x17
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_MAX_DEFINITE_ERR_REACHED_MASK 0x1000000
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_MAX_DEFINITE_ERR_REACHED__SHIFT 0x18
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_MAX_DEFINITE_ERR_REACHED_ACK_MASK 0x2000000
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_MAX_DEFINITE_ERR_REACHED_ACK__SHIFT 0x19
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_CTRL_STATE_MASK 0xf0000000
+#define AUX_GTC_SYNC_CONTROLLER_STATUS__AUX_GTC_SYNC_CTRL_STATE__SHIFT 0x1c
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_DONE_MASK 0x1
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_DONE__SHIFT 0x0
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_REQ_MASK 0x2
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_REQ__SHIFT 0x1
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_RX_TIMEOUT_STATE_MASK 0x70
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_RX_TIMEOUT_STATE__SHIFT 0x4
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_TIMEOUT_MASK 0x80
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_TIMEOUT__SHIFT 0x7
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_RX_OVERFLOW_MASK 0x100
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_RX_OVERFLOW__SHIFT 0x8
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_HPD_DISCON_MASK 0x200
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_HPD_DISCON__SHIFT 0x9
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_RX_PARTIAL_BYTE_MASK 0x400
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_RX_PARTIAL_BYTE__SHIFT 0xa
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_NON_AUX_MODE_MASK 0x800
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_NON_AUX_MODE__SHIFT 0xb
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_RX_MIN_COUNT_VIOL_MASK 0x1000
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_RX_MIN_COUNT_VIOL__SHIFT 0xc
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_RX_INVALID_STOP_MASK 0x4000
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_RX_INVALID_STOP__SHIFT 0xe
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_RX_SYNC_INVALID_L_MASK 0x20000
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_RX_SYNC_INVALID_L__SHIFT 0x11
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_RX_SYNC_INVALID_H_MASK 0x40000
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_RX_SYNC_INVALID_H__SHIFT 0x12
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_RX_INVALID_START_MASK 0x80000
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_RX_INVALID_START__SHIFT 0x13
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_RX_RECV_NO_DET_MASK 0x100000
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_RX_RECV_NO_DET__SHIFT 0x14
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_RX_RECV_INVALID_H_MASK 0x400000
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_RX_RECV_INVALID_H__SHIFT 0x16
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_RX_RECV_INVALID_L_MASK 0x800000
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_RX_RECV_INVALID_L__SHIFT 0x17
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_REPLY_BYTE_COUNT_MASK 0x1f000000
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_REPLY_BYTE_COUNT__SHIFT 0x18
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_NACKED_MASK 0x20000000
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_SYNC_NACKED__SHIFT 0x1d
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_MASTER_REQ_BY_RX_MASK 0x40000000
+#define AUX_GTC_SYNC_STATUS__AUX_GTC_MASTER_REQ_BY_RX__SHIFT 0x1e
+#define AUX_TEST_DEBUG_INDEX__AUX_TEST_DEBUG_INDEX_MASK 0xff
+#define AUX_TEST_DEBUG_INDEX__AUX_TEST_DEBUG_INDEX__SHIFT 0x0
+#define AUX_TEST_DEBUG_INDEX__AUX_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define AUX_TEST_DEBUG_INDEX__AUX_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define AUX_TEST_DEBUG_DATA__AUX_TEST_DEBUG_DATA_MASK 0xffffffff
+#define AUX_TEST_DEBUG_DATA__AUX_TEST_DEBUG_DATA__SHIFT 0x0
+#define DP_AUX_DEBUG_A__DP_AUX_DEBUG_A_MASK 0xffffffff
+#define DP_AUX_DEBUG_A__DP_AUX_DEBUG_A__SHIFT 0x0
+#define DP_AUX_DEBUG_B__DP_AUX_DEBUG_B_MASK 0xffffffff
+#define DP_AUX_DEBUG_B__DP_AUX_DEBUG_B__SHIFT 0x0
+#define DP_AUX_DEBUG_C__DP_AUX_DEBUG_C_MASK 0xffffffff
+#define DP_AUX_DEBUG_C__DP_AUX_DEBUG_C__SHIFT 0x0
+#define DP_AUX_DEBUG_D__DP_AUX_DEBUG_D_MASK 0xffffffff
+#define DP_AUX_DEBUG_D__DP_AUX_DEBUG_D__SHIFT 0x0
+#define DP_AUX_DEBUG_E__DP_AUX_DEBUG_E_MASK 0xffffffff
+#define DP_AUX_DEBUG_E__DP_AUX_DEBUG_E__SHIFT 0x0
+#define DP_AUX_DEBUG_F__DP_AUX_DEBUG_F_MASK 0xffffffff
+#define DP_AUX_DEBUG_F__DP_AUX_DEBUG_F__SHIFT 0x0
+#define DP_AUX_DEBUG_G__DP_AUX_DEBUG_G_MASK 0xffffffff
+#define DP_AUX_DEBUG_G__DP_AUX_DEBUG_G__SHIFT 0x0
+#define DP_AUX_DEBUG_H__DP_AUX_DEBUG_H_MASK 0xffffffff
+#define DP_AUX_DEBUG_H__DP_AUX_DEBUG_H__SHIFT 0x0
+#define DP_AUX_DEBUG_I__DP_AUX_DEBUG_I_MASK 0xffffffff
+#define DP_AUX_DEBUG_I__DP_AUX_DEBUG_I__SHIFT 0x0
+#define DP_AUX_DEBUG_J__DP_AUX_DEBUG_J_MASK 0xffffffff
+#define DP_AUX_DEBUG_J__DP_AUX_DEBUG_J__SHIFT 0x0
+#define DP_AUX_DEBUG_K__DP_AUX_DEBUG_K_MASK 0xffffffff
+#define DP_AUX_DEBUG_K__DP_AUX_DEBUG_K__SHIFT 0x0
+#define DP_AUX_DEBUG_L__DP_AUX_DEBUG_L_MASK 0xffffffff
+#define DP_AUX_DEBUG_L__DP_AUX_DEBUG_L__SHIFT 0x0
+#define DP_AUX_DEBUG_M__DP_AUX_DEBUG_M_MASK 0xffffffff
+#define DP_AUX_DEBUG_M__DP_AUX_DEBUG_M__SHIFT 0x0
+#define DP_AUX_DEBUG_N__DP_AUX_DEBUG_N_MASK 0xffffffff
+#define DP_AUX_DEBUG_N__DP_AUX_DEBUG_N__SHIFT 0x0
+#define DP_AUX_DEBUG_O__DP_AUX_DEBUG_O_MASK 0xffffffff
+#define DP_AUX_DEBUG_O__DP_AUX_DEBUG_O__SHIFT 0x0
+#define DP_AUX_DEBUG_P__DP_AUX_DEBUG_P_MASK 0xffffffff
+#define DP_AUX_DEBUG_P__DP_AUX_DEBUG_P__SHIFT 0x0
+#define DP_AUX_DEBUG_Q__DP_AUX_DEBUG_Q_MASK 0xffffffff
+#define DP_AUX_DEBUG_Q__DP_AUX_DEBUG_Q__SHIFT 0x0
+#define DVO_ENABLE__DVO_ENABLE_MASK 0x1
+#define DVO_ENABLE__DVO_ENABLE__SHIFT 0x0
+#define DVO_ENABLE__DVO_PIXEL_WIDTH_MASK 0x30
+#define DVO_ENABLE__DVO_PIXEL_WIDTH__SHIFT 0x4
+#define DVO_SOURCE_SELECT__DVO_SOURCE_SELECT_MASK 0x7
+#define DVO_SOURCE_SELECT__DVO_SOURCE_SELECT__SHIFT 0x0
+#define DVO_SOURCE_SELECT__DVO_STEREOSYNC_SELECT_MASK 0x70000
+#define DVO_SOURCE_SELECT__DVO_STEREOSYNC_SELECT__SHIFT 0x10
+#define DVO_OUTPUT__DVO_OUTPUT_ENABLE_MODE_MASK 0x3
+#define DVO_OUTPUT__DVO_OUTPUT_ENABLE_MODE__SHIFT 0x0
+#define DVO_OUTPUT__DVO_CLOCK_MODE_MASK 0x100
+#define DVO_OUTPUT__DVO_CLOCK_MODE__SHIFT 0x8
+#define DVO_CONTROL__DVO_RATE_SELECT_MASK 0x1
+#define DVO_CONTROL__DVO_RATE_SELECT__SHIFT 0x0
+#define DVO_CONTROL__DVO_SDRCLK_SEL_MASK 0x2
+#define DVO_CONTROL__DVO_SDRCLK_SEL__SHIFT 0x1
+#define DVO_CONTROL__DVO_DVPDATA_WIDTH_MASK 0x30
+#define DVO_CONTROL__DVO_DVPDATA_WIDTH__SHIFT 0x4
+#define DVO_CONTROL__DVO_DUAL_CHANNEL_EN_MASK 0x100
+#define DVO_CONTROL__DVO_DUAL_CHANNEL_EN__SHIFT 0x8
+#define DVO_CONTROL__DVO_RESET_FIFO_MASK 0x10000
+#define DVO_CONTROL__DVO_RESET_FIFO__SHIFT 0x10
+#define DVO_CONTROL__DVO_SYNC_PHASE_MASK 0x20000
+#define DVO_CONTROL__DVO_SYNC_PHASE__SHIFT 0x11
+#define DVO_CONTROL__DVO_INVERT_DVOCLK_MASK 0x40000
+#define DVO_CONTROL__DVO_INVERT_DVOCLK__SHIFT 0x12
+#define DVO_CONTROL__DVO_HSYNC_POLARITY_MASK 0x100000
+#define DVO_CONTROL__DVO_HSYNC_POLARITY__SHIFT 0x14
+#define DVO_CONTROL__DVO_VSYNC_POLARITY_MASK 0x200000
+#define DVO_CONTROL__DVO_VSYNC_POLARITY__SHIFT 0x15
+#define DVO_CONTROL__DVO_DE_POLARITY_MASK 0x400000
+#define DVO_CONTROL__DVO_DE_POLARITY__SHIFT 0x16
+#define DVO_CONTROL__DVO_COLOR_FORMAT_MASK 0x3000000
+#define DVO_CONTROL__DVO_COLOR_FORMAT__SHIFT 0x18
+#define DVO_CONTROL__DVO_CTL3_MASK 0x80000000
+#define DVO_CONTROL__DVO_CTL3__SHIFT 0x1f
+#define DVO_CRC_EN__DVO_CRC2_EN_MASK 0x10000
+#define DVO_CRC_EN__DVO_CRC2_EN__SHIFT 0x10
+#define DVO_CRC2_SIG_MASK__DVO_CRC2_SIG_MASK_MASK 0x7ffffff
+#define DVO_CRC2_SIG_MASK__DVO_CRC2_SIG_MASK__SHIFT 0x0
+#define DVO_CRC2_SIG_RESULT__DVO_CRC2_SIG_RESULT_MASK 0x7ffffff
+#define DVO_CRC2_SIG_RESULT__DVO_CRC2_SIG_RESULT__SHIFT 0x0
+#define DVO_FIFO_ERROR_STATUS__DVO_FIFO_LEVEL_ERROR_MASK 0x1
+#define DVO_FIFO_ERROR_STATUS__DVO_FIFO_LEVEL_ERROR__SHIFT 0x0
+#define DVO_FIFO_ERROR_STATUS__DVO_FIFO_USE_OVERWRITE_LEVEL_MASK 0x2
+#define DVO_FIFO_ERROR_STATUS__DVO_FIFO_USE_OVERWRITE_LEVEL__SHIFT 0x1
+#define DVO_FIFO_ERROR_STATUS__DVO_FIFO_OVERWRITE_LEVEL_MASK 0xfc
+#define DVO_FIFO_ERROR_STATUS__DVO_FIFO_OVERWRITE_LEVEL__SHIFT 0x2
+#define DVO_FIFO_ERROR_STATUS__DVO_FIFO_ERROR_ACK_MASK 0x100
+#define DVO_FIFO_ERROR_STATUS__DVO_FIFO_ERROR_ACK__SHIFT 0x8
+#define DVO_FIFO_ERROR_STATUS__DVO_FIFO_CAL_AVERAGE_LEVEL_MASK 0xfc00
+#define DVO_FIFO_ERROR_STATUS__DVO_FIFO_CAL_AVERAGE_LEVEL__SHIFT 0xa
+#define DVO_FIFO_ERROR_STATUS__DVO_FIFO_MAXIMUM_LEVEL_MASK 0xf0000
+#define DVO_FIFO_ERROR_STATUS__DVO_FIFO_MAXIMUM_LEVEL__SHIFT 0x10
+#define DVO_FIFO_ERROR_STATUS__DVO_FIFO_MINIMUM_LEVEL_MASK 0x3c00000
+#define DVO_FIFO_ERROR_STATUS__DVO_FIFO_MINIMUM_LEVEL__SHIFT 0x16
+#define DVO_FIFO_ERROR_STATUS__DVO_FIFO_CALIBRATED_MASK 0x20000000
+#define DVO_FIFO_ERROR_STATUS__DVO_FIFO_CALIBRATED__SHIFT 0x1d
+#define DVO_FIFO_ERROR_STATUS__DVO_FIFO_FORCE_RECAL_AVERAGE_MASK 0x40000000
+#define DVO_FIFO_ERROR_STATUS__DVO_FIFO_FORCE_RECAL_AVERAGE__SHIFT 0x1e
+#define DVO_FIFO_ERROR_STATUS__DVO_FIFO_FORCE_RECOMP_MINMAX_MASK 0x80000000
+#define DVO_FIFO_ERROR_STATUS__DVO_FIFO_FORCE_RECOMP_MINMAX__SHIFT 0x1f
+#define DVO_TEST_DEBUG_INDEX__DVO_TEST_DEBUG_INDEX_MASK 0xff
+#define DVO_TEST_DEBUG_INDEX__DVO_TEST_DEBUG_INDEX__SHIFT 0x0
+#define DVO_TEST_DEBUG_INDEX__DVO_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define DVO_TEST_DEBUG_INDEX__DVO_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define DVO_TEST_DEBUG_DATA__DVO_TEST_DEBUG_DATA_MASK 0xffffffff
+#define DVO_TEST_DEBUG_DATA__DVO_TEST_DEBUG_DATA__SHIFT 0x0
+#define FBC_CNTL__FBC_GRPH_COMP_EN_MASK 0x1
+#define FBC_CNTL__FBC_GRPH_COMP_EN__SHIFT 0x0
+#define FBC_CNTL__FBC_SRC_SEL_MASK 0xe
+#define FBC_CNTL__FBC_SRC_SEL__SHIFT 0x1
+#define FBC_CNTL__FBC_COMP_CLK_GATE_EN_MASK 0x100
+#define FBC_CNTL__FBC_COMP_CLK_GATE_EN__SHIFT 0x8
+#define FBC_CNTL__FBC_DECOMP_CLK_GATE_EN_MASK 0x400
+#define FBC_CNTL__FBC_DECOMP_CLK_GATE_EN__SHIFT 0xa
+#define FBC_CNTL__FBC_COHERENCY_MODE_MASK 0x30000
+#define FBC_CNTL__FBC_COHERENCY_MODE__SHIFT 0x10
+#define FBC_CNTL__FBC_SOFT_COMPRESS_EN_MASK 0x2000000
+#define FBC_CNTL__FBC_SOFT_COMPRESS_EN__SHIFT 0x19
+#define FBC_CNTL__FBC_EN_MASK 0x80000000
+#define FBC_CNTL__FBC_EN__SHIFT 0x1f
+#define FBC_IDLE_FORCE_CLEAR_MASK__FBC_IDLE_FORCE_CLEAR_MASK_MASK 0xffffffff
+#define FBC_IDLE_FORCE_CLEAR_MASK__FBC_IDLE_FORCE_CLEAR_MASK__SHIFT 0x0
+#define FBC_START_STOP_DELAY__FBC_DECOMP_START_DELAY_MASK 0x1f
+#define FBC_START_STOP_DELAY__FBC_DECOMP_START_DELAY__SHIFT 0x0
+#define FBC_START_STOP_DELAY__FBC_DECOMP_STOP_DELAY_MASK 0x80
+#define FBC_START_STOP_DELAY__FBC_DECOMP_STOP_DELAY__SHIFT 0x7
+#define FBC_START_STOP_DELAY__FBC_COMP_START_DELAY_MASK 0x1f00
+#define FBC_START_STOP_DELAY__FBC_COMP_START_DELAY__SHIFT 0x8
+#define FBC_COMP_CNTL__FBC_MIN_COMPRESSION_MASK 0xf
+#define FBC_COMP_CNTL__FBC_MIN_COMPRESSION__SHIFT 0x0
+#define FBC_COMP_CNTL__FBC_DEPTH_MONO08_EN_MASK 0x10000
+#define FBC_COMP_CNTL__FBC_DEPTH_MONO08_EN__SHIFT 0x10
+#define FBC_COMP_CNTL__FBC_DEPTH_MONO16_EN_MASK 0x20000
+#define FBC_COMP_CNTL__FBC_DEPTH_MONO16_EN__SHIFT 0x11
+#define FBC_COMP_CNTL__FBC_DEPTH_RGB04_EN_MASK 0x40000
+#define FBC_COMP_CNTL__FBC_DEPTH_RGB04_EN__SHIFT 0x12
+#define FBC_COMP_CNTL__FBC_DEPTH_RGB08_EN_MASK 0x80000
+#define FBC_COMP_CNTL__FBC_DEPTH_RGB08_EN__SHIFT 0x13
+#define FBC_COMP_CNTL__FBC_DEPTH_RGB16_EN_MASK 0x100000
+#define FBC_COMP_CNTL__FBC_DEPTH_RGB16_EN__SHIFT 0x14
+#define FBC_COMP_MODE__FBC_RLE_EN_MASK 0x1
+#define FBC_COMP_MODE__FBC_RLE_EN__SHIFT 0x0
+#define FBC_COMP_MODE__FBC_DPCM4_RGB_EN_MASK 0x100
+#define FBC_COMP_MODE__FBC_DPCM4_RGB_EN__SHIFT 0x8
+#define FBC_COMP_MODE__FBC_DPCM8_RGB_EN_MASK 0x200
+#define FBC_COMP_MODE__FBC_DPCM8_RGB_EN__SHIFT 0x9
+#define FBC_COMP_MODE__FBC_DPCM4_YUV_EN_MASK 0x400
+#define FBC_COMP_MODE__FBC_DPCM4_YUV_EN__SHIFT 0xa
+#define FBC_COMP_MODE__FBC_DPCM8_YUV_EN_MASK 0x800
+#define FBC_COMP_MODE__FBC_DPCM8_YUV_EN__SHIFT 0xb
+#define FBC_COMP_MODE__FBC_IND_EN_MASK 0x10000
+#define FBC_COMP_MODE__FBC_IND_EN__SHIFT 0x10
+#define FBC_DEBUG0__FBC_PERF_MUX0_MASK 0xff
+#define FBC_DEBUG0__FBC_PERF_MUX0__SHIFT 0x0
+#define FBC_DEBUG0__FBC_PERF_MUX1_MASK 0xff00
+#define FBC_DEBUG0__FBC_PERF_MUX1__SHIFT 0x8
+#define FBC_DEBUG0__FBC_COMP_WAKE_DIS_MASK 0x10000
+#define FBC_DEBUG0__FBC_COMP_WAKE_DIS__SHIFT 0x10
+#define FBC_DEBUG0__FBC_DEBUG0_MASK 0xfe0000
+#define FBC_DEBUG0__FBC_DEBUG0__SHIFT 0x11
+#define FBC_DEBUG0__FBC_DEBUG_MUX_MASK 0xff000000
+#define FBC_DEBUG0__FBC_DEBUG_MUX__SHIFT 0x18
+#define FBC_DEBUG1__FBC_DEBUG1_MASK 0xffffffff
+#define FBC_DEBUG1__FBC_DEBUG1__SHIFT 0x0
+#define FBC_DEBUG2__FBC_DEBUG2_MASK 0xffffffff
+#define FBC_DEBUG2__FBC_DEBUG2__SHIFT 0x0
+#define FBC_IND_LUT0__FBC_IND_LUT0_MASK 0xffffffff
+#define FBC_IND_LUT0__FBC_IND_LUT0__SHIFT 0x0
+#define FBC_IND_LUT1__FBC_IND_LUT1_MASK 0xffffffff
+#define FBC_IND_LUT1__FBC_IND_LUT1__SHIFT 0x0
+#define FBC_IND_LUT2__FBC_IND_LUT2_MASK 0xffffffff
+#define FBC_IND_LUT2__FBC_IND_LUT2__SHIFT 0x0
+#define FBC_IND_LUT3__FBC_IND_LUT3_MASK 0xffffffff
+#define FBC_IND_LUT3__FBC_IND_LUT3__SHIFT 0x0
+#define FBC_IND_LUT4__FBC_IND_LUT4_MASK 0xffffffff
+#define FBC_IND_LUT4__FBC_IND_LUT4__SHIFT 0x0
+#define FBC_IND_LUT5__FBC_IND_LUT5_MASK 0xffffffff
+#define FBC_IND_LUT5__FBC_IND_LUT5__SHIFT 0x0
+#define FBC_IND_LUT6__FBC_IND_LUT6_MASK 0xffffffff
+#define FBC_IND_LUT6__FBC_IND_LUT6__SHIFT 0x0
+#define FBC_IND_LUT7__FBC_IND_LUT7_MASK 0xffffffff
+#define FBC_IND_LUT7__FBC_IND_LUT7__SHIFT 0x0
+#define FBC_IND_LUT8__FBC_IND_LUT8_MASK 0xffffffff
+#define FBC_IND_LUT8__FBC_IND_LUT8__SHIFT 0x0
+#define FBC_IND_LUT9__FBC_IND_LUT9_MASK 0xffffffff
+#define FBC_IND_LUT9__FBC_IND_LUT9__SHIFT 0x0
+#define FBC_IND_LUT10__FBC_IND_LUT10_MASK 0xffffffff
+#define FBC_IND_LUT10__FBC_IND_LUT10__SHIFT 0x0
+#define FBC_IND_LUT11__FBC_IND_LUT11_MASK 0xffffffff
+#define FBC_IND_LUT11__FBC_IND_LUT11__SHIFT 0x0
+#define FBC_IND_LUT12__FBC_IND_LUT12_MASK 0xffffffff
+#define FBC_IND_LUT12__FBC_IND_LUT12__SHIFT 0x0
+#define FBC_IND_LUT13__FBC_IND_LUT13_MASK 0xffffffff
+#define FBC_IND_LUT13__FBC_IND_LUT13__SHIFT 0x0
+#define FBC_IND_LUT14__FBC_IND_LUT14_MASK 0xffffffff
+#define FBC_IND_LUT14__FBC_IND_LUT14__SHIFT 0x0
+#define FBC_IND_LUT15__FBC_IND_LUT15_MASK 0xffffffff
+#define FBC_IND_LUT15__FBC_IND_LUT15__SHIFT 0x0
+#define FBC_CSM_REGION_OFFSET_01__FBC_CSM_REGION_OFFSET_0_MASK 0xfff
+#define FBC_CSM_REGION_OFFSET_01__FBC_CSM_REGION_OFFSET_0__SHIFT 0x0
+#define FBC_CSM_REGION_OFFSET_01__FBC_CSM_REGION_OFFSET_1_MASK 0xfff0000
+#define FBC_CSM_REGION_OFFSET_01__FBC_CSM_REGION_OFFSET_1__SHIFT 0x10
+#define FBC_CSM_REGION_OFFSET_23__FBC_CSM_REGION_OFFSET_2_MASK 0xfff
+#define FBC_CSM_REGION_OFFSET_23__FBC_CSM_REGION_OFFSET_2__SHIFT 0x0
+#define FBC_CSM_REGION_OFFSET_23__FBC_CSM_REGION_OFFSET_3_MASK 0xfff0000
+#define FBC_CSM_REGION_OFFSET_23__FBC_CSM_REGION_OFFSET_3__SHIFT 0x10
+#define FBC_CLIENT_REGION_MASK__FBC_MEMORY_REGION_MASK_MASK 0xf0000
+#define FBC_CLIENT_REGION_MASK__FBC_MEMORY_REGION_MASK__SHIFT 0x10
+#define FBC_DEBUG_COMP__FBC_COMP_SWAP_MASK 0x3
+#define FBC_DEBUG_COMP__FBC_COMP_SWAP__SHIFT 0x0
+#define FBC_DEBUG_COMP__FBC_COMP_RSIZE_MASK 0x8
+#define FBC_DEBUG_COMP__FBC_COMP_RSIZE__SHIFT 0x3
+#define FBC_DEBUG_COMP__FBC_COMP_BUSY_HYSTERESIS_MASK 0xf0
+#define FBC_DEBUG_COMP__FBC_COMP_BUSY_HYSTERESIS__SHIFT 0x4
+#define FBC_DEBUG_COMP__FBC_COMP_CLK_CNTL_MASK 0x300
+#define FBC_DEBUG_COMP__FBC_COMP_CLK_CNTL__SHIFT 0x8
+#define FBC_DEBUG_COMP__FBC_COMP_PRIVILEGED_ACCESS_ENABLE_MASK 0x400
+#define FBC_DEBUG_COMP__FBC_COMP_PRIVILEGED_ACCESS_ENABLE__SHIFT 0xa
+#define FBC_DEBUG_COMP__FBC_COMP_ADDRESS_TRANSLATION_ENABLE_MASK 0x800
+#define FBC_DEBUG_COMP__FBC_COMP_ADDRESS_TRANSLATION_ENABLE__SHIFT 0xb
+#define FBC_DEBUG_CSR__FBC_DEBUG_CSR_ADDR_MASK 0xfff
+#define FBC_DEBUG_CSR__FBC_DEBUG_CSR_ADDR__SHIFT 0x0
+#define FBC_DEBUG_CSR__FBC_DEBUG_CSR_WR_DATA_MASK 0x10000
+#define FBC_DEBUG_CSR__FBC_DEBUG_CSR_WR_DATA__SHIFT 0x10
+#define FBC_DEBUG_CSR__FBC_DEBUG_CSR_RD_DATA_MASK 0x20000
+#define FBC_DEBUG_CSR__FBC_DEBUG_CSR_RD_DATA__SHIFT 0x11
+#define FBC_DEBUG_CSR__FBC_DEBUG_CSR_EN_MASK 0x80000000
+#define FBC_DEBUG_CSR__FBC_DEBUG_CSR_EN__SHIFT 0x1f
+#define FBC_DEBUG_CSR_RDATA__FBC_DEBUG_CSR_RDATA_MASK 0xffffffff
+#define FBC_DEBUG_CSR_RDATA__FBC_DEBUG_CSR_RDATA__SHIFT 0x0
+#define FBC_DEBUG_CSR_WDATA__FBC_DEBUG_CSR_WDATA_MASK 0xffffffff
+#define FBC_DEBUG_CSR_WDATA__FBC_DEBUG_CSR_WDATA__SHIFT 0x0
+#define FBC_DEBUG_CSR_RDATA_HI__FBC_DEBUG_CSR_RDATA_HI_MASK 0xff
+#define FBC_DEBUG_CSR_RDATA_HI__FBC_DEBUG_CSR_RDATA_HI__SHIFT 0x0
+#define FBC_DEBUG_CSR_WDATA_HI__FBC_DEBUG_CSR_WDATA_HI_MASK 0xff
+#define FBC_DEBUG_CSR_WDATA_HI__FBC_DEBUG_CSR_WDATA_HI__SHIFT 0x0
+#define FBC_MISC__FBC_DECOMPRESS_ERROR_MASK 0x3
+#define FBC_MISC__FBC_DECOMPRESS_ERROR__SHIFT 0x0
+#define FBC_MISC__FBC_STOP_ON_ERROR_MASK 0x4
+#define FBC_MISC__FBC_STOP_ON_ERROR__SHIFT 0x2
+#define FBC_MISC__FBC_INVALIDATE_ON_ERROR_MASK 0x8
+#define FBC_MISC__FBC_INVALIDATE_ON_ERROR__SHIFT 0x3
+#define FBC_MISC__FBC_ERROR_PIXEL_MASK 0xf0
+#define FBC_MISC__FBC_ERROR_PIXEL__SHIFT 0x4
+#define FBC_MISC__FBC_DIVIDE_X_MASK 0x300
+#define FBC_MISC__FBC_DIVIDE_X__SHIFT 0x8
+#define FBC_MISC__FBC_DIVIDE_Y_MASK 0x400
+#define FBC_MISC__FBC_DIVIDE_Y__SHIFT 0xa
+#define FBC_MISC__FBC_RSM_WRITE_VALUE_MASK 0x800
+#define FBC_MISC__FBC_RSM_WRITE_VALUE__SHIFT 0xb
+#define FBC_MISC__FBC_RSM_UNCOMP_DATA_IMMEDIATELY_MASK 0x1000
+#define FBC_MISC__FBC_RSM_UNCOMP_DATA_IMMEDIATELY__SHIFT 0xc
+#define FBC_MISC__FBC_STOP_ON_HFLIP_EVENT_MASK 0x2000
+#define FBC_MISC__FBC_STOP_ON_HFLIP_EVENT__SHIFT 0xd
+#define FBC_MISC__FBC_DECOMPRESS_ERROR_CLEAR_MASK 0x10000
+#define FBC_MISC__FBC_DECOMPRESS_ERROR_CLEAR__SHIFT 0x10
+#define FBC_MISC__FBC_RESET_AT_ENABLE_MASK 0x100000
+#define FBC_MISC__FBC_RESET_AT_ENABLE__SHIFT 0x14
+#define FBC_MISC__FBC_RESET_AT_DISABLE_MASK 0x200000
+#define FBC_MISC__FBC_RESET_AT_DISABLE__SHIFT 0x15
+#define FBC_MISC__FBC_SLOW_REQ_INTERVAL_MASK 0x1f000000
+#define FBC_MISC__FBC_SLOW_REQ_INTERVAL__SHIFT 0x18
+#define FBC_MISC__FBC_FORCE_DECOMPRESSOR_EN_MASK 0x80000000
+#define FBC_MISC__FBC_FORCE_DECOMPRESSOR_EN__SHIFT 0x1f
+#define FBC_STATUS__FBC_ENABLE_STATUS_MASK 0x1
+#define FBC_STATUS__FBC_ENABLE_STATUS__SHIFT 0x0
+#define FBC_ALPHA_CNTL__FBC_ALPHA_COMP_EN_MASK 0x1
+#define FBC_ALPHA_CNTL__FBC_ALPHA_COMP_EN__SHIFT 0x0
+#define FBC_ALPHA_CNTL__FBC_FORCE_COPY_TO_COMP_BUF_MASK 0x10
+#define FBC_ALPHA_CNTL__FBC_FORCE_COPY_TO_COMP_BUF__SHIFT 0x4
+#define FBC_ALPHA_CNTL__FBC_ZERO_ALPHA_CHUNK_SKIP_EN_MASK 0x100
+#define FBC_ALPHA_CNTL__FBC_ZERO_ALPHA_CHUNK_SKIP_EN__SHIFT 0x8
+#define FBC_ALPHA_RGB_OVERRIDE__FBC_ZERO_ALPHA_R_VAL_MASK 0xff
+#define FBC_ALPHA_RGB_OVERRIDE__FBC_ZERO_ALPHA_R_VAL__SHIFT 0x0
+#define FBC_ALPHA_RGB_OVERRIDE__FBC_ZERO_ALPHA_G_VAL_MASK 0xff000
+#define FBC_ALPHA_RGB_OVERRIDE__FBC_ZERO_ALPHA_G_VAL__SHIFT 0xc
+#define FBC_ALPHA_RGB_OVERRIDE__FBC_ZERO_ALPHA_B_VAL_MASK 0xff000000
+#define FBC_ALPHA_RGB_OVERRIDE__FBC_ZERO_ALPHA_B_VAL__SHIFT 0x18
+#define FBC_TEST_DEBUG_INDEX__FBC_TEST_DEBUG_INDEX_MASK 0xff
+#define FBC_TEST_DEBUG_INDEX__FBC_TEST_DEBUG_INDEX__SHIFT 0x0
+#define FBC_TEST_DEBUG_INDEX__FBC_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define FBC_TEST_DEBUG_INDEX__FBC_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define FBC_TEST_DEBUG_DATA__FBC_TEST_DEBUG_DATA_MASK 0xffffffff
+#define FBC_TEST_DEBUG_DATA__FBC_TEST_DEBUG_DATA__SHIFT 0x0
+#define FMT_CLAMP_COMPONENT_R__FMT_CLAMP_LOWER_R_MASK 0xffff
+#define FMT_CLAMP_COMPONENT_R__FMT_CLAMP_LOWER_R__SHIFT 0x0
+#define FMT_CLAMP_COMPONENT_R__FMT_CLAMP_UPPER_R_MASK 0xffff0000
+#define FMT_CLAMP_COMPONENT_R__FMT_CLAMP_UPPER_R__SHIFT 0x10
+#define FMT_CLAMP_COMPONENT_G__FMT_CLAMP_LOWER_G_MASK 0xffff
+#define FMT_CLAMP_COMPONENT_G__FMT_CLAMP_LOWER_G__SHIFT 0x0
+#define FMT_CLAMP_COMPONENT_G__FMT_CLAMP_UPPER_G_MASK 0xffff0000
+#define FMT_CLAMP_COMPONENT_G__FMT_CLAMP_UPPER_G__SHIFT 0x10
+#define FMT_CLAMP_COMPONENT_B__FMT_CLAMP_LOWER_B_MASK 0xffff
+#define FMT_CLAMP_COMPONENT_B__FMT_CLAMP_LOWER_B__SHIFT 0x0
+#define FMT_CLAMP_COMPONENT_B__FMT_CLAMP_UPPER_B_MASK 0xffff0000
+#define FMT_CLAMP_COMPONENT_B__FMT_CLAMP_UPPER_B__SHIFT 0x10
+#define FMT_DYNAMIC_EXP_CNTL__FMT_DYNAMIC_EXP_EN_MASK 0x1
+#define FMT_DYNAMIC_EXP_CNTL__FMT_DYNAMIC_EXP_EN__SHIFT 0x0
+#define FMT_DYNAMIC_EXP_CNTL__FMT_DYNAMIC_EXP_MODE_MASK 0x10
+#define FMT_DYNAMIC_EXP_CNTL__FMT_DYNAMIC_EXP_MODE__SHIFT 0x4
+#define FMT_CONTROL__FMT_STEREOSYNC_OVERRIDE_MASK 0x1
+#define FMT_CONTROL__FMT_STEREOSYNC_OVERRIDE__SHIFT 0x0
+#define FMT_CONTROL__FMT_STEREOSYNC_OVR_POL_MASK 0x10
+#define FMT_CONTROL__FMT_STEREOSYNC_OVR_POL__SHIFT 0x4
+#define FMT_CONTROL__FMT_SPATIAL_DITHER_FRAME_COUNTER_MAX_MASK 0xf00
+#define FMT_CONTROL__FMT_SPATIAL_DITHER_FRAME_COUNTER_MAX__SHIFT 0x8
+#define FMT_CONTROL__FMT_SPATIAL_DITHER_FRAME_COUNTER_BIT_SWAP_MASK 0x3000
+#define FMT_CONTROL__FMT_SPATIAL_DITHER_FRAME_COUNTER_BIT_SWAP__SHIFT 0xc
+#define FMT_CONTROL__FMT_PIXEL_ENCODING_MASK 0x30000
+#define FMT_CONTROL__FMT_PIXEL_ENCODING__SHIFT 0x10
+#define FMT_CONTROL__FMT_SUBSAMPLING_MODE_MASK 0xc0000
+#define FMT_CONTROL__FMT_SUBSAMPLING_MODE__SHIFT 0x12
+#define FMT_CONTROL__FMT_SUBSAMPLING_ORDER_MASK 0x100000
+#define FMT_CONTROL__FMT_SUBSAMPLING_ORDER__SHIFT 0x14
+#define FMT_CONTROL__FMT_CBCR_BIT_REDUCTION_BYPASS_MASK 0x200000
+#define FMT_CONTROL__FMT_CBCR_BIT_REDUCTION_BYPASS__SHIFT 0x15
+#define FMT_CONTROL__FMT_SRC_SELECT_MASK 0x7000000
+#define FMT_CONTROL__FMT_SRC_SELECT__SHIFT 0x18
+#define FMT_CONTROL__FMT_420_PIXEL_PHASE_LOCKED_MASK 0x40000000
+#define FMT_CONTROL__FMT_420_PIXEL_PHASE_LOCKED__SHIFT 0x1e
+#define FMT_CONTROL__FMT_420_PIXEL_PHASE_LOCKED_CLEAR_MASK 0x80000000
+#define FMT_CONTROL__FMT_420_PIXEL_PHASE_LOCKED_CLEAR__SHIFT 0x1f
+#define FMT_BIT_DEPTH_CONTROL__FMT_TRUNCATE_EN_MASK 0x1
+#define FMT_BIT_DEPTH_CONTROL__FMT_TRUNCATE_EN__SHIFT 0x0
+#define FMT_BIT_DEPTH_CONTROL__FMT_TRUNCATE_MODE_MASK 0x2
+#define FMT_BIT_DEPTH_CONTROL__FMT_TRUNCATE_MODE__SHIFT 0x1
+#define FMT_BIT_DEPTH_CONTROL__FMT_TRUNCATE_DEPTH_MASK 0x30
+#define FMT_BIT_DEPTH_CONTROL__FMT_TRUNCATE_DEPTH__SHIFT 0x4
+#define FMT_BIT_DEPTH_CONTROL__FMT_SPATIAL_DITHER_EN_MASK 0x100
+#define FMT_BIT_DEPTH_CONTROL__FMT_SPATIAL_DITHER_EN__SHIFT 0x8
+#define FMT_BIT_DEPTH_CONTROL__FMT_SPATIAL_DITHER_MODE_MASK 0x600
+#define FMT_BIT_DEPTH_CONTROL__FMT_SPATIAL_DITHER_MODE__SHIFT 0x9
+#define FMT_BIT_DEPTH_CONTROL__FMT_SPATIAL_DITHER_DEPTH_MASK 0x1800
+#define FMT_BIT_DEPTH_CONTROL__FMT_SPATIAL_DITHER_DEPTH__SHIFT 0xb
+#define FMT_BIT_DEPTH_CONTROL__FMT_FRAME_RANDOM_ENABLE_MASK 0x2000
+#define FMT_BIT_DEPTH_CONTROL__FMT_FRAME_RANDOM_ENABLE__SHIFT 0xd
+#define FMT_BIT_DEPTH_CONTROL__FMT_RGB_RANDOM_ENABLE_MASK 0x4000
+#define FMT_BIT_DEPTH_CONTROL__FMT_RGB_RANDOM_ENABLE__SHIFT 0xe
+#define FMT_BIT_DEPTH_CONTROL__FMT_HIGHPASS_RANDOM_ENABLE_MASK 0x8000
+#define FMT_BIT_DEPTH_CONTROL__FMT_HIGHPASS_RANDOM_ENABLE__SHIFT 0xf
+#define FMT_BIT_DEPTH_CONTROL__FMT_TEMPORAL_DITHER_EN_MASK 0x10000
+#define FMT_BIT_DEPTH_CONTROL__FMT_TEMPORAL_DITHER_EN__SHIFT 0x10
+#define FMT_BIT_DEPTH_CONTROL__FMT_TEMPORAL_DITHER_DEPTH_MASK 0x60000
+#define FMT_BIT_DEPTH_CONTROL__FMT_TEMPORAL_DITHER_DEPTH__SHIFT 0x11
+#define FMT_BIT_DEPTH_CONTROL__FMT_TEMPORAL_DITHER_OFFSET_MASK 0x600000
+#define FMT_BIT_DEPTH_CONTROL__FMT_TEMPORAL_DITHER_OFFSET__SHIFT 0x15
+#define FMT_BIT_DEPTH_CONTROL__FMT_TEMPORAL_LEVEL_MASK 0x1000000
+#define FMT_BIT_DEPTH_CONTROL__FMT_TEMPORAL_LEVEL__SHIFT 0x18
+#define FMT_BIT_DEPTH_CONTROL__FMT_TEMPORAL_DITHER_RESET_MASK 0x2000000
+#define FMT_BIT_DEPTH_CONTROL__FMT_TEMPORAL_DITHER_RESET__SHIFT 0x19
+#define FMT_BIT_DEPTH_CONTROL__FMT_25FRC_SEL_MASK 0xc000000
+#define FMT_BIT_DEPTH_CONTROL__FMT_25FRC_SEL__SHIFT 0x1a
+#define FMT_BIT_DEPTH_CONTROL__FMT_50FRC_SEL_MASK 0x30000000
+#define FMT_BIT_DEPTH_CONTROL__FMT_50FRC_SEL__SHIFT 0x1c
+#define FMT_BIT_DEPTH_CONTROL__FMT_75FRC_SEL_MASK 0xc0000000
+#define FMT_BIT_DEPTH_CONTROL__FMT_75FRC_SEL__SHIFT 0x1e
+#define FMT_DITHER_RAND_R_SEED__FMT_RAND_R_SEED_MASK 0xff
+#define FMT_DITHER_RAND_R_SEED__FMT_RAND_R_SEED__SHIFT 0x0
+#define FMT_DITHER_RAND_R_SEED__FMT_OFFSET_R_CR_MASK 0xffff0000
+#define FMT_DITHER_RAND_R_SEED__FMT_OFFSET_R_CR__SHIFT 0x10
+#define FMT_DITHER_RAND_G_SEED__FMT_RAND_G_SEED_MASK 0xff
+#define FMT_DITHER_RAND_G_SEED__FMT_RAND_G_SEED__SHIFT 0x0
+#define FMT_DITHER_RAND_G_SEED__FMT_OFFSET_G_Y_MASK 0xffff0000
+#define FMT_DITHER_RAND_G_SEED__FMT_OFFSET_G_Y__SHIFT 0x10
+#define FMT_DITHER_RAND_B_SEED__FMT_RAND_B_SEED_MASK 0xff
+#define FMT_DITHER_RAND_B_SEED__FMT_RAND_B_SEED__SHIFT 0x0
+#define FMT_DITHER_RAND_B_SEED__FMT_OFFSET_B_CB_MASK 0xffff0000
+#define FMT_DITHER_RAND_B_SEED__FMT_OFFSET_B_CB__SHIFT 0x10
+#define FMT_CLAMP_CNTL__FMT_CLAMP_DATA_EN_MASK 0x1
+#define FMT_CLAMP_CNTL__FMT_CLAMP_DATA_EN__SHIFT 0x0
+#define FMT_CLAMP_CNTL__FMT_CLAMP_COLOR_FORMAT_MASK 0x70000
+#define FMT_CLAMP_CNTL__FMT_CLAMP_COLOR_FORMAT__SHIFT 0x10
+#define FMT_CRC_CNTL__FMT_CRC_EN_MASK 0x1
+#define FMT_CRC_CNTL__FMT_CRC_EN__SHIFT 0x0
+#define FMT_CRC_CNTL__FMT_DTMTEST_CRC_EN_MASK 0x2
+#define FMT_CRC_CNTL__FMT_DTMTEST_CRC_EN__SHIFT 0x1
+#define FMT_CRC_CNTL__FMT_CRC_CONT_EN_MASK 0x10
+#define FMT_CRC_CNTL__FMT_CRC_CONT_EN__SHIFT 0x4
+#define FMT_CRC_CNTL__FMT_ONE_SHOT_CRC_PENDING_MASK 0x20
+#define FMT_CRC_CNTL__FMT_ONE_SHOT_CRC_PENDING__SHIFT 0x5
+#define FMT_CRC_CNTL__FMT_CRC_INCLUDE_OVERSCAN_MASK 0x40
+#define FMT_CRC_CNTL__FMT_CRC_INCLUDE_OVERSCAN__SHIFT 0x6
+#define FMT_CRC_CNTL__FMT_CRC_ONLY_BLANKB_MASK 0x100
+#define FMT_CRC_CNTL__FMT_CRC_ONLY_BLANKB__SHIFT 0x8
+#define FMT_CRC_CNTL__FMT_CRC_PSR_MODE_ENABLE_MASK 0x200
+#define FMT_CRC_CNTL__FMT_CRC_PSR_MODE_ENABLE__SHIFT 0x9
+#define FMT_CRC_CNTL__FMT_CRC_INTERLACE_MODE_MASK 0x3000
+#define FMT_CRC_CNTL__FMT_CRC_INTERLACE_MODE__SHIFT 0xc
+#define FMT_CRC_CNTL__FMT_CRC_USE_NEW_AND_REPEATED_PIXELS_MASK 0x10000
+#define FMT_CRC_CNTL__FMT_CRC_USE_NEW_AND_REPEATED_PIXELS__SHIFT 0x10
+#define FMT_CRC_CNTL__FMT_CRC_EVEN_ODD_PIX_ENABLE_MASK 0x100000
+#define FMT_CRC_CNTL__FMT_CRC_EVEN_ODD_PIX_ENABLE__SHIFT 0x14
+#define FMT_CRC_CNTL__FMT_CRC_EVEN_ODD_PIX_SELECT_MASK 0x1000000
+#define FMT_CRC_CNTL__FMT_CRC_EVEN_ODD_PIX_SELECT__SHIFT 0x18
+#define FMT_CRC_SIG_RED_GREEN_MASK__FMT_CRC_SIG_RED_MASK_MASK 0xffff
+#define FMT_CRC_SIG_RED_GREEN_MASK__FMT_CRC_SIG_RED_MASK__SHIFT 0x0
+#define FMT_CRC_SIG_RED_GREEN_MASK__FMT_CRC_SIG_GREEN_MASK_MASK 0xffff0000
+#define FMT_CRC_SIG_RED_GREEN_MASK__FMT_CRC_SIG_GREEN_MASK__SHIFT 0x10
+#define FMT_CRC_SIG_BLUE_CONTROL_MASK__FMT_CRC_SIG_BLUE_MASK_MASK 0xffff
+#define FMT_CRC_SIG_BLUE_CONTROL_MASK__FMT_CRC_SIG_BLUE_MASK__SHIFT 0x0
+#define FMT_CRC_SIG_BLUE_CONTROL_MASK__FMT_CRC_SIG_CONTROL_MASK_MASK 0xffff0000
+#define FMT_CRC_SIG_BLUE_CONTROL_MASK__FMT_CRC_SIG_CONTROL_MASK__SHIFT 0x10
+#define FMT_CRC_SIG_RED_GREEN__FMT_CRC_SIG_RED_MASK 0xffff
+#define FMT_CRC_SIG_RED_GREEN__FMT_CRC_SIG_RED__SHIFT 0x0
+#define FMT_CRC_SIG_RED_GREEN__FMT_CRC_SIG_GREEN_MASK 0xffff0000
+#define FMT_CRC_SIG_RED_GREEN__FMT_CRC_SIG_GREEN__SHIFT 0x10
+#define FMT_CRC_SIG_BLUE_CONTROL__FMT_CRC_SIG_BLUE_MASK 0xffff
+#define FMT_CRC_SIG_BLUE_CONTROL__FMT_CRC_SIG_BLUE__SHIFT 0x0
+#define FMT_CRC_SIG_BLUE_CONTROL__FMT_CRC_SIG_CONTROL_MASK 0xffff0000
+#define FMT_CRC_SIG_BLUE_CONTROL__FMT_CRC_SIG_CONTROL__SHIFT 0x10
+#define FMT_DEBUG_CNTL__FMT_DEBUG_COLOR_SELECT_MASK 0x3
+#define FMT_DEBUG_CNTL__FMT_DEBUG_COLOR_SELECT__SHIFT 0x0
+#define FMT_SIDE_BY_SIDE_STEREO_CONTROL__FMT_SIDE_BY_SIDE_STEREO_ACTIVE_WIDTH_MASK 0x1fff
+#define FMT_SIDE_BY_SIDE_STEREO_CONTROL__FMT_SIDE_BY_SIDE_STEREO_ACTIVE_WIDTH__SHIFT 0x0
+#define FMT_420_HBLANK_EARLY_START__FMT_420_HBLANK_EARLY_START_MASK 0xfff
+#define FMT_420_HBLANK_EARLY_START__FMT_420_HBLANK_EARLY_START__SHIFT 0x0
+#define FMT_TEST_DEBUG_INDEX__FMT_TEST_DEBUG_INDEX_MASK 0xff
+#define FMT_TEST_DEBUG_INDEX__FMT_TEST_DEBUG_INDEX__SHIFT 0x0
+#define FMT_TEST_DEBUG_INDEX__FMT_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define FMT_TEST_DEBUG_INDEX__FMT_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define FMT_TEST_DEBUG_DATA__FMT_TEST_DEBUG_DATA_MASK 0xffffffff
+#define FMT_TEST_DEBUG_DATA__FMT_TEST_DEBUG_DATA__SHIFT 0x0
+#define FMT_DEBUG0__FMT_DEBUG0_MASK 0xffffffff
+#define FMT_DEBUG0__FMT_DEBUG0__SHIFT 0x0
+#define FMT_DEBUG1__FMT_DEBUG1_MASK 0xffffffff
+#define FMT_DEBUG1__FMT_DEBUG1__SHIFT 0x0
+#define FMT_DEBUG2__FMT_DEBUG2_MASK 0xffffffff
+#define FMT_DEBUG2__FMT_DEBUG2__SHIFT 0x0
+#define FMT_DEBUG3__FMT_DEBUG3_MASK 0xffffffff
+#define FMT_DEBUG3__FMT_DEBUG3__SHIFT 0x0
+#define FMT_DEBUG_ID__FMT_DEBUG_ID_MASK 0xffffffff
+#define FMT_DEBUG_ID__FMT_DEBUG_ID__SHIFT 0x0
+#define LB_DATA_FORMAT__PIXEL_DEPTH_MASK 0x3
+#define LB_DATA_FORMAT__PIXEL_DEPTH__SHIFT 0x0
+#define LB_DATA_FORMAT__PIXEL_EXPAN_MODE_MASK 0x4
+#define LB_DATA_FORMAT__PIXEL_EXPAN_MODE__SHIFT 0x2
+#define LB_DATA_FORMAT__INTERLEAVE_EN_MASK 0x8
+#define LB_DATA_FORMAT__INTERLEAVE_EN__SHIFT 0x3
+#define LB_DATA_FORMAT__PIXEL_REDUCE_MODE_MASK 0x10
+#define LB_DATA_FORMAT__PIXEL_REDUCE_MODE__SHIFT 0x4
+#define LB_DATA_FORMAT__DYNAMIC_PIXEL_DEPTH_MASK 0x20
+#define LB_DATA_FORMAT__DYNAMIC_PIXEL_DEPTH__SHIFT 0x5
+#define LB_DATA_FORMAT__PREFILL_EN_MASK 0x100
+#define LB_DATA_FORMAT__PREFILL_EN__SHIFT 0x8
+#define LB_DATA_FORMAT__PREFETCH_MASK 0x1000
+#define LB_DATA_FORMAT__PREFETCH__SHIFT 0xc
+#define LB_DATA_FORMAT__REQUEST_MODE_MASK 0x1000000
+#define LB_DATA_FORMAT__REQUEST_MODE__SHIFT 0x18
+#define LB_DATA_FORMAT__ALPHA_EN_MASK 0x80000000
+#define LB_DATA_FORMAT__ALPHA_EN__SHIFT 0x1f
+#define LB_MEMORY_CTRL__LB_MEMORY_SIZE_MASK 0x1fff
+#define LB_MEMORY_CTRL__LB_MEMORY_SIZE__SHIFT 0x0
+#define LB_MEMORY_CTRL__LB_NUM_PARTITIONS_MASK 0xf0000
+#define LB_MEMORY_CTRL__LB_NUM_PARTITIONS__SHIFT 0x10
+#define LB_MEMORY_CTRL__LB_MEMORY_CONFIG_MASK 0x300000
+#define LB_MEMORY_CTRL__LB_MEMORY_CONFIG__SHIFT 0x14
+#define LB_MEMORY_SIZE_STATUS__LB_MEMORY_SIZE_STATUS_MASK 0x1fff
+#define LB_MEMORY_SIZE_STATUS__LB_MEMORY_SIZE_STATUS__SHIFT 0x0
+#define LB_DESKTOP_HEIGHT__DESKTOP_HEIGHT_MASK 0x7fff
+#define LB_DESKTOP_HEIGHT__DESKTOP_HEIGHT__SHIFT 0x0
+#define LB_VLINE_START_END__VLINE_START_MASK 0x3fff
+#define LB_VLINE_START_END__VLINE_START__SHIFT 0x0
+#define LB_VLINE_START_END__VLINE_END_MASK 0x7fff0000
+#define LB_VLINE_START_END__VLINE_END__SHIFT 0x10
+#define LB_VLINE_START_END__VLINE_INV_MASK 0x80000000
+#define LB_VLINE_START_END__VLINE_INV__SHIFT 0x1f
+#define LB_VLINE2_START_END__VLINE2_START_MASK 0x3fff
+#define LB_VLINE2_START_END__VLINE2_START__SHIFT 0x0
+#define LB_VLINE2_START_END__VLINE2_END_MASK 0x7fff0000
+#define LB_VLINE2_START_END__VLINE2_END__SHIFT 0x10
+#define LB_VLINE2_START_END__VLINE2_INV_MASK 0x80000000
+#define LB_VLINE2_START_END__VLINE2_INV__SHIFT 0x1f
+#define LB_V_COUNTER__V_COUNTER_MASK 0x7fff
+#define LB_V_COUNTER__V_COUNTER__SHIFT 0x0
+#define LB_SNAPSHOT_V_COUNTER__SNAPSHOT_V_COUNTER_MASK 0x7fff
+#define LB_SNAPSHOT_V_COUNTER__SNAPSHOT_V_COUNTER__SHIFT 0x0
+#define LB_INTERRUPT_MASK__VBLANK_INTERRUPT_MASK_MASK 0x1
+#define LB_INTERRUPT_MASK__VBLANK_INTERRUPT_MASK__SHIFT 0x0
+#define LB_INTERRUPT_MASK__VLINE_INTERRUPT_MASK_MASK 0x10
+#define LB_INTERRUPT_MASK__VLINE_INTERRUPT_MASK__SHIFT 0x4
+#define LB_INTERRUPT_MASK__VLINE2_INTERRUPT_MASK_MASK 0x100
+#define LB_INTERRUPT_MASK__VLINE2_INTERRUPT_MASK__SHIFT 0x8
+#define LB_VLINE_STATUS__VLINE_OCCURRED_MASK 0x1
+#define LB_VLINE_STATUS__VLINE_OCCURRED__SHIFT 0x0
+#define LB_VLINE_STATUS__VLINE_ACK_MASK 0x10
+#define LB_VLINE_STATUS__VLINE_ACK__SHIFT 0x4
+#define LB_VLINE_STATUS__VLINE_STAT_MASK 0x1000
+#define LB_VLINE_STATUS__VLINE_STAT__SHIFT 0xc
+#define LB_VLINE_STATUS__VLINE_INTERRUPT_MASK 0x10000
+#define LB_VLINE_STATUS__VLINE_INTERRUPT__SHIFT 0x10
+#define LB_VLINE_STATUS__VLINE_INTERRUPT_TYPE_MASK 0x20000
+#define LB_VLINE_STATUS__VLINE_INTERRUPT_TYPE__SHIFT 0x11
+#define LB_VLINE2_STATUS__VLINE2_OCCURRED_MASK 0x1
+#define LB_VLINE2_STATUS__VLINE2_OCCURRED__SHIFT 0x0
+#define LB_VLINE2_STATUS__VLINE2_ACK_MASK 0x10
+#define LB_VLINE2_STATUS__VLINE2_ACK__SHIFT 0x4
+#define LB_VLINE2_STATUS__VLINE2_STAT_MASK 0x1000
+#define LB_VLINE2_STATUS__VLINE2_STAT__SHIFT 0xc
+#define LB_VLINE2_STATUS__VLINE2_INTERRUPT_MASK 0x10000
+#define LB_VLINE2_STATUS__VLINE2_INTERRUPT__SHIFT 0x10
+#define LB_VLINE2_STATUS__VLINE2_INTERRUPT_TYPE_MASK 0x20000
+#define LB_VLINE2_STATUS__VLINE2_INTERRUPT_TYPE__SHIFT 0x11
+#define LB_VBLANK_STATUS__VBLANK_OCCURRED_MASK 0x1
+#define LB_VBLANK_STATUS__VBLANK_OCCURRED__SHIFT 0x0
+#define LB_VBLANK_STATUS__VBLANK_ACK_MASK 0x10
+#define LB_VBLANK_STATUS__VBLANK_ACK__SHIFT 0x4
+#define LB_VBLANK_STATUS__VBLANK_STAT_MASK 0x1000
+#define LB_VBLANK_STATUS__VBLANK_STAT__SHIFT 0xc
+#define LB_VBLANK_STATUS__VBLANK_INTERRUPT_MASK 0x10000
+#define LB_VBLANK_STATUS__VBLANK_INTERRUPT__SHIFT 0x10
+#define LB_VBLANK_STATUS__VBLANK_INTERRUPT_TYPE_MASK 0x20000
+#define LB_VBLANK_STATUS__VBLANK_INTERRUPT_TYPE__SHIFT 0x11
+#define LB_SYNC_RESET_SEL__LB_SYNC_RESET_SEL_MASK 0x3
+#define LB_SYNC_RESET_SEL__LB_SYNC_RESET_SEL__SHIFT 0x0
+#define LB_SYNC_RESET_SEL__LB_SYNC_RESET_SEL2_MASK 0x10
+#define LB_SYNC_RESET_SEL__LB_SYNC_RESET_SEL2__SHIFT 0x4
+#define LB_SYNC_RESET_SEL__LB_SYNC_RESET_DELAY_MASK 0xff00
+#define LB_SYNC_RESET_SEL__LB_SYNC_RESET_DELAY__SHIFT 0x8
+#define LB_SYNC_RESET_SEL__LB_SYNC_DURATION_MASK 0xc00000
+#define LB_SYNC_RESET_SEL__LB_SYNC_DURATION__SHIFT 0x16
+#define LB_BLACK_KEYER_R_CR__LB_BLACK_KEYER_R_CR_MASK 0xfff0
+#define LB_BLACK_KEYER_R_CR__LB_BLACK_KEYER_R_CR__SHIFT 0x4
+#define LB_BLACK_KEYER_G_Y__LB_BLACK_KEYER_G_Y_MASK 0xfff0
+#define LB_BLACK_KEYER_G_Y__LB_BLACK_KEYER_G_Y__SHIFT 0x4
+#define LB_BLACK_KEYER_B_CB__LB_BLACK_KEYER_B_CB_MASK 0xfff0
+#define LB_BLACK_KEYER_B_CB__LB_BLACK_KEYER_B_CB__SHIFT 0x4
+#define LB_KEYER_COLOR_CTRL__LB_KEYER_COLOR_EN_MASK 0x1
+#define LB_KEYER_COLOR_CTRL__LB_KEYER_COLOR_EN__SHIFT 0x0
+#define LB_KEYER_COLOR_CTRL__LB_KEYER_COLOR_REP_EN_MASK 0x100
+#define LB_KEYER_COLOR_CTRL__LB_KEYER_COLOR_REP_EN__SHIFT 0x8
+#define LB_KEYER_COLOR_R_CR__LB_KEYER_COLOR_R_CR_MASK 0xfff0
+#define LB_KEYER_COLOR_R_CR__LB_KEYER_COLOR_R_CR__SHIFT 0x4
+#define LB_KEYER_COLOR_G_Y__LB_KEYER_COLOR_G_Y_MASK 0xfff0
+#define LB_KEYER_COLOR_G_Y__LB_KEYER_COLOR_G_Y__SHIFT 0x4
+#define LB_KEYER_COLOR_B_CB__LB_KEYER_COLOR_B_CB_MASK 0xfff0
+#define LB_KEYER_COLOR_B_CB__LB_KEYER_COLOR_B_CB__SHIFT 0x4
+#define LB_KEYER_COLOR_REP_R_CR__LB_KEYER_COLOR_REP_R_CR_MASK 0xfff0
+#define LB_KEYER_COLOR_REP_R_CR__LB_KEYER_COLOR_REP_R_CR__SHIFT 0x4
+#define LB_KEYER_COLOR_REP_G_Y__LB_KEYER_COLOR_REP_G_Y_MASK 0xfff0
+#define LB_KEYER_COLOR_REP_G_Y__LB_KEYER_COLOR_REP_G_Y__SHIFT 0x4
+#define LB_KEYER_COLOR_REP_B_CB__LB_KEYER_COLOR_REP_B_CB_MASK 0xfff0
+#define LB_KEYER_COLOR_REP_B_CB__LB_KEYER_COLOR_REP_B_CB__SHIFT 0x4
+#define LB_BUFFER_LEVEL_STATUS__REQ_FIFO_LEVEL_MASK 0x3f
+#define LB_BUFFER_LEVEL_STATUS__REQ_FIFO_LEVEL__SHIFT 0x0
+#define LB_BUFFER_LEVEL_STATUS__REQ_FIFO_FULL_CNTL_MASK 0xfc00
+#define LB_BUFFER_LEVEL_STATUS__REQ_FIFO_FULL_CNTL__SHIFT 0xa
+#define LB_BUFFER_LEVEL_STATUS__DATA_BUFFER_LEVEL_MASK 0xfff0000
+#define LB_BUFFER_LEVEL_STATUS__DATA_BUFFER_LEVEL__SHIFT 0x10
+#define LB_BUFFER_LEVEL_STATUS__DATA_FIFO_FULL_CNTL_MASK 0xf0000000
+#define LB_BUFFER_LEVEL_STATUS__DATA_FIFO_FULL_CNTL__SHIFT 0x1c
+#define LB_BUFFER_URGENCY_CTRL__LB_BUFFER_URGENCY_MARK_ON_MASK 0xfff
+#define LB_BUFFER_URGENCY_CTRL__LB_BUFFER_URGENCY_MARK_ON__SHIFT 0x0
+#define LB_BUFFER_URGENCY_CTRL__LB_BUFFER_URGENCY_MARK_OFF_MASK 0xfff0000
+#define LB_BUFFER_URGENCY_CTRL__LB_BUFFER_URGENCY_MARK_OFF__SHIFT 0x10
+#define LB_BUFFER_URGENCY_STATUS__LB_BUFFER_URGENCY_LEVEL_MASK 0xfff
+#define LB_BUFFER_URGENCY_STATUS__LB_BUFFER_URGENCY_LEVEL__SHIFT 0x0
+#define LB_BUFFER_URGENCY_STATUS__LB_BUFFER_URGENCY_STAT_MASK 0x10000
+#define LB_BUFFER_URGENCY_STATUS__LB_BUFFER_URGENCY_STAT__SHIFT 0x10
+#define LB_BUFFER_STATUS__LB_BUFFER_EMPTY_MARGIN_MASK 0xf
+#define LB_BUFFER_STATUS__LB_BUFFER_EMPTY_MARGIN__SHIFT 0x0
+#define LB_BUFFER_STATUS__LB_BUFFER_EMPTY_STAT_MASK 0x10
+#define LB_BUFFER_STATUS__LB_BUFFER_EMPTY_STAT__SHIFT 0x4
+#define LB_BUFFER_STATUS__LB_BUFFER_EMPTY_OCCURRED_MASK 0x100
+#define LB_BUFFER_STATUS__LB_BUFFER_EMPTY_OCCURRED__SHIFT 0x8
+#define LB_BUFFER_STATUS__LB_BUFFER_EMPTY_ACK_MASK 0x1000
+#define LB_BUFFER_STATUS__LB_BUFFER_EMPTY_ACK__SHIFT 0xc
+#define LB_BUFFER_STATUS__LB_BUFFER_FULL_STAT_MASK 0x10000
+#define LB_BUFFER_STATUS__LB_BUFFER_FULL_STAT__SHIFT 0x10
+#define LB_BUFFER_STATUS__LB_BUFFER_FULL_OCCURRED_MASK 0x100000
+#define LB_BUFFER_STATUS__LB_BUFFER_FULL_OCCURRED__SHIFT 0x14
+#define LB_BUFFER_STATUS__LB_BUFFER_FULL_ACK_MASK 0x1000000
+#define LB_BUFFER_STATUS__LB_BUFFER_FULL_ACK__SHIFT 0x18
+#define LB_NO_OUTSTANDING_REQ_STATUS__LB_NO_OUTSTANDING_REQ_STAT_MASK 0x1
+#define LB_NO_OUTSTANDING_REQ_STATUS__LB_NO_OUTSTANDING_REQ_STAT__SHIFT 0x0
+#define MVP_AFR_FLIP_MODE__MVP_AFR_FLIP_MODE_MASK 0x3
+#define MVP_AFR_FLIP_MODE__MVP_AFR_FLIP_MODE__SHIFT 0x0
+#define MVP_AFR_FLIP_FIFO_CNTL__MVP_AFR_FLIP_FIFO_NUM_ENTRIES_MASK 0xf
+#define MVP_AFR_FLIP_FIFO_CNTL__MVP_AFR_FLIP_FIFO_NUM_ENTRIES__SHIFT 0x0
+#define MVP_AFR_FLIP_FIFO_CNTL__MVP_AFR_FLIP_FIFO_RESET_MASK 0x10
+#define MVP_AFR_FLIP_FIFO_CNTL__MVP_AFR_FLIP_FIFO_RESET__SHIFT 0x4
+#define MVP_AFR_FLIP_FIFO_CNTL__MVP_AFR_FLIP_FIFO_RESET_FLAG_MASK 0x100
+#define MVP_AFR_FLIP_FIFO_CNTL__MVP_AFR_FLIP_FIFO_RESET_FLAG__SHIFT 0x8
+#define MVP_AFR_FLIP_FIFO_CNTL__MVP_AFR_FLIP_FIFO_RESET_ACK_MASK 0x1000
+#define MVP_AFR_FLIP_FIFO_CNTL__MVP_AFR_FLIP_FIFO_RESET_ACK__SHIFT 0xc
+#define MVP_FLIP_LINE_NUM_INSERT__MVP_FLIP_LINE_NUM_INSERT_MODE_MASK 0x3
+#define MVP_FLIP_LINE_NUM_INSERT__MVP_FLIP_LINE_NUM_INSERT_MODE__SHIFT 0x0
+#define MVP_FLIP_LINE_NUM_INSERT__MVP_FLIP_LINE_NUM_INSERT_MASK 0x7fff00
+#define MVP_FLIP_LINE_NUM_INSERT__MVP_FLIP_LINE_NUM_INSERT__SHIFT 0x8
+#define MVP_FLIP_LINE_NUM_INSERT__MVP_FLIP_LINE_NUM_OFFSET_MASK 0x3f000000
+#define MVP_FLIP_LINE_NUM_INSERT__MVP_FLIP_LINE_NUM_OFFSET__SHIFT 0x18
+#define MVP_FLIP_LINE_NUM_INSERT__MVP_FLIP_AUTO_ENABLE_MASK 0x40000000
+#define MVP_FLIP_LINE_NUM_INSERT__MVP_FLIP_AUTO_ENABLE__SHIFT 0x1e
+#define DC_MVP_LB_CONTROL__MVP_SWAP_LOCK_IN_MODE_MASK 0x3
+#define DC_MVP_LB_CONTROL__MVP_SWAP_LOCK_IN_MODE__SHIFT 0x0
+#define DC_MVP_LB_CONTROL__DC_MVP_SWAP_LOCK_OUT_SEL_MASK 0x100
+#define DC_MVP_LB_CONTROL__DC_MVP_SWAP_LOCK_OUT_SEL__SHIFT 0x8
+#define DC_MVP_LB_CONTROL__DC_MVP_SWAP_LOCK_OUT_FORCE_ONE_MASK 0x1000
+#define DC_MVP_LB_CONTROL__DC_MVP_SWAP_LOCK_OUT_FORCE_ONE__SHIFT 0xc
+#define DC_MVP_LB_CONTROL__DC_MVP_SWAP_LOCK_OUT_FORCE_ZERO_MASK 0x10000
+#define DC_MVP_LB_CONTROL__DC_MVP_SWAP_LOCK_OUT_FORCE_ZERO__SHIFT 0x10
+#define DC_MVP_LB_CONTROL__DC_MVP_SWAP_LOCK_STATUS_MASK 0x100000
+#define DC_MVP_LB_CONTROL__DC_MVP_SWAP_LOCK_STATUS__SHIFT 0x14
+#define DC_MVP_LB_CONTROL__DC_MVP_SWAP_LOCK_IN_CAP_MASK 0x10000000
+#define DC_MVP_LB_CONTROL__DC_MVP_SWAP_LOCK_IN_CAP__SHIFT 0x1c
+#define DC_MVP_LB_CONTROL__DC_MVP_SPARE_FLOPS_MASK 0x80000000
+#define DC_MVP_LB_CONTROL__DC_MVP_SPARE_FLOPS__SHIFT 0x1f
+#define LB_DEBUG__LB_DEBUG_MASK 0xffffffff
+#define LB_DEBUG__LB_DEBUG__SHIFT 0x0
+#define LB_DEBUG2__LB_DEBUG2_MASK 0xffffffff
+#define LB_DEBUG2__LB_DEBUG2__SHIFT 0x0
+#define LB_DEBUG3__LB_DEBUG3_MASK 0xffffffff
+#define LB_DEBUG3__LB_DEBUG3__SHIFT 0x0
+#define LB_TEST_DEBUG_INDEX__LB_TEST_DEBUG_INDEX_MASK 0xff
+#define LB_TEST_DEBUG_INDEX__LB_TEST_DEBUG_INDEX__SHIFT 0x0
+#define LB_TEST_DEBUG_INDEX__LB_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define LB_TEST_DEBUG_INDEX__LB_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define LB_TEST_DEBUG_DATA__LB_TEST_DEBUG_DATA_MASK 0xffffffff
+#define LB_TEST_DEBUG_DATA__LB_TEST_DEBUG_DATA__SHIFT 0x0
+#define LBV_DATA_FORMAT__PIXEL_DEPTH_MASK 0x3
+#define LBV_DATA_FORMAT__PIXEL_DEPTH__SHIFT 0x0
+#define LBV_DATA_FORMAT__PIXEL_EXPAN_MODE_MASK 0x4
+#define LBV_DATA_FORMAT__PIXEL_EXPAN_MODE__SHIFT 0x2
+#define LBV_DATA_FORMAT__INTERLEAVE_EN_MASK 0x8
+#define LBV_DATA_FORMAT__INTERLEAVE_EN__SHIFT 0x3
+#define LBV_DATA_FORMAT__PIXEL_REDUCE_MODE_MASK 0x10
+#define LBV_DATA_FORMAT__PIXEL_REDUCE_MODE__SHIFT 0x4
+#define LBV_DATA_FORMAT__DYNAMIC_PIXEL_DEPTH_MASK 0x20
+#define LBV_DATA_FORMAT__DYNAMIC_PIXEL_DEPTH__SHIFT 0x5
+#define LBV_DATA_FORMAT__DITHER_EN_MASK 0x40
+#define LBV_DATA_FORMAT__DITHER_EN__SHIFT 0x6
+#define LBV_DATA_FORMAT__DOWNSCALE_PREFETCH_EN_MASK 0x80
+#define LBV_DATA_FORMAT__DOWNSCALE_PREFETCH_EN__SHIFT 0x7
+#define LBV_DATA_FORMAT__PREFETCH_MASK 0x1000
+#define LBV_DATA_FORMAT__PREFETCH__SHIFT 0xc
+#define LBV_DATA_FORMAT__REQUEST_MODE_MASK 0x1000000
+#define LBV_DATA_FORMAT__REQUEST_MODE__SHIFT 0x18
+#define LBV_DATA_FORMAT__ALPHA_EN_MASK 0x80000000
+#define LBV_DATA_FORMAT__ALPHA_EN__SHIFT 0x1f
+#define LBV_MEMORY_CTRL__LB_MEMORY_SIZE_MASK 0xfff
+#define LBV_MEMORY_CTRL__LB_MEMORY_SIZE__SHIFT 0x0
+#define LBV_MEMORY_CTRL__LB_NUM_PARTITIONS_MASK 0xf0000
+#define LBV_MEMORY_CTRL__LB_NUM_PARTITIONS__SHIFT 0x10
+#define LBV_MEMORY_CTRL__LB_MEMORY_CONFIG_MASK 0x300000
+#define LBV_MEMORY_CTRL__LB_MEMORY_CONFIG__SHIFT 0x14
+#define LBV_MEMORY_SIZE_STATUS__LB_MEMORY_SIZE_STATUS_MASK 0xfff
+#define LBV_MEMORY_SIZE_STATUS__LB_MEMORY_SIZE_STATUS__SHIFT 0x0
+#define LBV_DESKTOP_HEIGHT__DESKTOP_HEIGHT_MASK 0x7fff
+#define LBV_DESKTOP_HEIGHT__DESKTOP_HEIGHT__SHIFT 0x0
+#define LBV_VLINE_START_END__VLINE_START_MASK 0x3fff
+#define LBV_VLINE_START_END__VLINE_START__SHIFT 0x0
+#define LBV_VLINE_START_END__VLINE_END_MASK 0x7fff0000
+#define LBV_VLINE_START_END__VLINE_END__SHIFT 0x10
+#define LBV_VLINE_START_END__VLINE_INV_MASK 0x80000000
+#define LBV_VLINE_START_END__VLINE_INV__SHIFT 0x1f
+#define LBV_VLINE2_START_END__VLINE2_START_MASK 0x3fff
+#define LBV_VLINE2_START_END__VLINE2_START__SHIFT 0x0
+#define LBV_VLINE2_START_END__VLINE2_END_MASK 0x7fff0000
+#define LBV_VLINE2_START_END__VLINE2_END__SHIFT 0x10
+#define LBV_VLINE2_START_END__VLINE2_INV_MASK 0x80000000
+#define LBV_VLINE2_START_END__VLINE2_INV__SHIFT 0x1f
+#define LBV_V_COUNTER__V_COUNTER_MASK 0x7fff
+#define LBV_V_COUNTER__V_COUNTER__SHIFT 0x0
+#define LBV_SNAPSHOT_V_COUNTER__SNAPSHOT_V_COUNTER_MASK 0x7fff
+#define LBV_SNAPSHOT_V_COUNTER__SNAPSHOT_V_COUNTER__SHIFT 0x0
+#define LBV_V_COUNTER_CHROMA__V_COUNTER_CHROMA_MASK 0x7fff
+#define LBV_V_COUNTER_CHROMA__V_COUNTER_CHROMA__SHIFT 0x0
+#define LBV_SNAPSHOT_V_COUNTER_CHROMA__SNAPSHOT_V_COUNTER_CHROMA_MASK 0x7fff
+#define LBV_SNAPSHOT_V_COUNTER_CHROMA__SNAPSHOT_V_COUNTER_CHROMA__SHIFT 0x0
+#define LBV_INTERRUPT_MASK__VBLANK_INTERRUPT_MASK_MASK 0x1
+#define LBV_INTERRUPT_MASK__VBLANK_INTERRUPT_MASK__SHIFT 0x0
+#define LBV_INTERRUPT_MASK__VLINE_INTERRUPT_MASK_MASK 0x10
+#define LBV_INTERRUPT_MASK__VLINE_INTERRUPT_MASK__SHIFT 0x4
+#define LBV_INTERRUPT_MASK__VLINE2_INTERRUPT_MASK_MASK 0x100
+#define LBV_INTERRUPT_MASK__VLINE2_INTERRUPT_MASK__SHIFT 0x8
+#define LBV_VLINE_STATUS__VLINE_OCCURRED_MASK 0x1
+#define LBV_VLINE_STATUS__VLINE_OCCURRED__SHIFT 0x0
+#define LBV_VLINE_STATUS__VLINE_ACK_MASK 0x10
+#define LBV_VLINE_STATUS__VLINE_ACK__SHIFT 0x4
+#define LBV_VLINE_STATUS__VLINE_STAT_MASK 0x1000
+#define LBV_VLINE_STATUS__VLINE_STAT__SHIFT 0xc
+#define LBV_VLINE_STATUS__VLINE_INTERRUPT_MASK 0x10000
+#define LBV_VLINE_STATUS__VLINE_INTERRUPT__SHIFT 0x10
+#define LBV_VLINE_STATUS__VLINE_INTERRUPT_TYPE_MASK 0x20000
+#define LBV_VLINE_STATUS__VLINE_INTERRUPT_TYPE__SHIFT 0x11
+#define LBV_VLINE2_STATUS__VLINE2_OCCURRED_MASK 0x1
+#define LBV_VLINE2_STATUS__VLINE2_OCCURRED__SHIFT 0x0
+#define LBV_VLINE2_STATUS__VLINE2_ACK_MASK 0x10
+#define LBV_VLINE2_STATUS__VLINE2_ACK__SHIFT 0x4
+#define LBV_VLINE2_STATUS__VLINE2_STAT_MASK 0x1000
+#define LBV_VLINE2_STATUS__VLINE2_STAT__SHIFT 0xc
+#define LBV_VLINE2_STATUS__VLINE2_INTERRUPT_MASK 0x10000
+#define LBV_VLINE2_STATUS__VLINE2_INTERRUPT__SHIFT 0x10
+#define LBV_VLINE2_STATUS__VLINE2_INTERRUPT_TYPE_MASK 0x20000
+#define LBV_VLINE2_STATUS__VLINE2_INTERRUPT_TYPE__SHIFT 0x11
+#define LBV_VBLANK_STATUS__VBLANK_OCCURRED_MASK 0x1
+#define LBV_VBLANK_STATUS__VBLANK_OCCURRED__SHIFT 0x0
+#define LBV_VBLANK_STATUS__VBLANK_ACK_MASK 0x10
+#define LBV_VBLANK_STATUS__VBLANK_ACK__SHIFT 0x4
+#define LBV_VBLANK_STATUS__VBLANK_STAT_MASK 0x1000
+#define LBV_VBLANK_STATUS__VBLANK_STAT__SHIFT 0xc
+#define LBV_VBLANK_STATUS__VBLANK_INTERRUPT_MASK 0x10000
+#define LBV_VBLANK_STATUS__VBLANK_INTERRUPT__SHIFT 0x10
+#define LBV_VBLANK_STATUS__VBLANK_INTERRUPT_TYPE_MASK 0x20000
+#define LBV_VBLANK_STATUS__VBLANK_INTERRUPT_TYPE__SHIFT 0x11
+#define LBV_SYNC_RESET_SEL__LB_SYNC_RESET_SEL_MASK 0x3
+#define LBV_SYNC_RESET_SEL__LB_SYNC_RESET_SEL__SHIFT 0x0
+#define LBV_SYNC_RESET_SEL__LB_SYNC_RESET_SEL2_MASK 0x10
+#define LBV_SYNC_RESET_SEL__LB_SYNC_RESET_SEL2__SHIFT 0x4
+#define LBV_SYNC_RESET_SEL__LB_SYNC_RESET_DELAY_MASK 0xff00
+#define LBV_SYNC_RESET_SEL__LB_SYNC_RESET_DELAY__SHIFT 0x8
+#define LBV_SYNC_RESET_SEL__LB_SYNC_DURATION_MASK 0xc00000
+#define LBV_SYNC_RESET_SEL__LB_SYNC_DURATION__SHIFT 0x16
+#define LBV_BLACK_KEYER_R_CR__LB_BLACK_KEYER_R_CR_MASK 0xfff0
+#define LBV_BLACK_KEYER_R_CR__LB_BLACK_KEYER_R_CR__SHIFT 0x4
+#define LBV_BLACK_KEYER_G_Y__LB_BLACK_KEYER_G_Y_MASK 0xfff0
+#define LBV_BLACK_KEYER_G_Y__LB_BLACK_KEYER_G_Y__SHIFT 0x4
+#define LBV_BLACK_KEYER_B_CB__LB_BLACK_KEYER_B_CB_MASK 0xfff0
+#define LBV_BLACK_KEYER_B_CB__LB_BLACK_KEYER_B_CB__SHIFT 0x4
+#define LBV_KEYER_COLOR_CTRL__LB_KEYER_COLOR_EN_MASK 0x1
+#define LBV_KEYER_COLOR_CTRL__LB_KEYER_COLOR_EN__SHIFT 0x0
+#define LBV_KEYER_COLOR_CTRL__LB_KEYER_COLOR_REP_EN_MASK 0x100
+#define LBV_KEYER_COLOR_CTRL__LB_KEYER_COLOR_REP_EN__SHIFT 0x8
+#define LBV_KEYER_COLOR_R_CR__LB_KEYER_COLOR_R_CR_MASK 0xfff0
+#define LBV_KEYER_COLOR_R_CR__LB_KEYER_COLOR_R_CR__SHIFT 0x4
+#define LBV_KEYER_COLOR_G_Y__LB_KEYER_COLOR_G_Y_MASK 0xfff0
+#define LBV_KEYER_COLOR_G_Y__LB_KEYER_COLOR_G_Y__SHIFT 0x4
+#define LBV_KEYER_COLOR_B_CB__LB_KEYER_COLOR_B_CB_MASK 0xfff0
+#define LBV_KEYER_COLOR_B_CB__LB_KEYER_COLOR_B_CB__SHIFT 0x4
+#define LBV_KEYER_COLOR_REP_R_CR__LB_KEYER_COLOR_REP_R_CR_MASK 0xfff0
+#define LBV_KEYER_COLOR_REP_R_CR__LB_KEYER_COLOR_REP_R_CR__SHIFT 0x4
+#define LBV_KEYER_COLOR_REP_G_Y__LB_KEYER_COLOR_REP_G_Y_MASK 0xfff0
+#define LBV_KEYER_COLOR_REP_G_Y__LB_KEYER_COLOR_REP_G_Y__SHIFT 0x4
+#define LBV_KEYER_COLOR_REP_B_CB__LB_KEYER_COLOR_REP_B_CB_MASK 0xfff0
+#define LBV_KEYER_COLOR_REP_B_CB__LB_KEYER_COLOR_REP_B_CB__SHIFT 0x4
+#define LBV_BUFFER_LEVEL_STATUS__REQ_FIFO_LEVEL_MASK 0x3f
+#define LBV_BUFFER_LEVEL_STATUS__REQ_FIFO_LEVEL__SHIFT 0x0
+#define LBV_BUFFER_LEVEL_STATUS__REQ_FIFO_FULL_CNTL_MASK 0xfc00
+#define LBV_BUFFER_LEVEL_STATUS__REQ_FIFO_FULL_CNTL__SHIFT 0xa
+#define LBV_BUFFER_LEVEL_STATUS__DATA_BUFFER_LEVEL_MASK 0xfff0000
+#define LBV_BUFFER_LEVEL_STATUS__DATA_BUFFER_LEVEL__SHIFT 0x10
+#define LBV_BUFFER_LEVEL_STATUS__DATA_FIFO_FULL_CNTL_MASK 0xf0000000
+#define LBV_BUFFER_LEVEL_STATUS__DATA_FIFO_FULL_CNTL__SHIFT 0x1c
+#define LBV_BUFFER_URGENCY_CTRL__LB_BUFFER_URGENCY_MARK_ON_MASK 0xfff
+#define LBV_BUFFER_URGENCY_CTRL__LB_BUFFER_URGENCY_MARK_ON__SHIFT 0x0
+#define LBV_BUFFER_URGENCY_CTRL__LB_BUFFER_URGENCY_MARK_OFF_MASK 0xfff0000
+#define LBV_BUFFER_URGENCY_CTRL__LB_BUFFER_URGENCY_MARK_OFF__SHIFT 0x10
+#define LBV_BUFFER_URGENCY_STATUS__LB_BUFFER_URGENCY_LEVEL_MASK 0xfff
+#define LBV_BUFFER_URGENCY_STATUS__LB_BUFFER_URGENCY_LEVEL__SHIFT 0x0
+#define LBV_BUFFER_URGENCY_STATUS__LB_BUFFER_URGENCY_STAT_MASK 0x10000
+#define LBV_BUFFER_URGENCY_STATUS__LB_BUFFER_URGENCY_STAT__SHIFT 0x10
+#define LBV_BUFFER_STATUS__LB_BUFFER_EMPTY_MARGIN_MASK 0xf
+#define LBV_BUFFER_STATUS__LB_BUFFER_EMPTY_MARGIN__SHIFT 0x0
+#define LBV_BUFFER_STATUS__LB_BUFFER_EMPTY_STAT_MASK 0x10
+#define LBV_BUFFER_STATUS__LB_BUFFER_EMPTY_STAT__SHIFT 0x4
+#define LBV_BUFFER_STATUS__LB_BUFFER_EMPTY_OCCURRED_MASK 0x100
+#define LBV_BUFFER_STATUS__LB_BUFFER_EMPTY_OCCURRED__SHIFT 0x8
+#define LBV_BUFFER_STATUS__LB_BUFFER_EMPTY_ACK_MASK 0x1000
+#define LBV_BUFFER_STATUS__LB_BUFFER_EMPTY_ACK__SHIFT 0xc
+#define LBV_BUFFER_STATUS__LB_BUFFER_FULL_STAT_MASK 0x10000
+#define LBV_BUFFER_STATUS__LB_BUFFER_FULL_STAT__SHIFT 0x10
+#define LBV_BUFFER_STATUS__LB_BUFFER_FULL_OCCURRED_MASK 0x100000
+#define LBV_BUFFER_STATUS__LB_BUFFER_FULL_OCCURRED__SHIFT 0x14
+#define LBV_BUFFER_STATUS__LB_BUFFER_FULL_ACK_MASK 0x1000000
+#define LBV_BUFFER_STATUS__LB_BUFFER_FULL_ACK__SHIFT 0x18
+#define LBV_BUFFER_STATUS__LB_ENABLE_HIGH_THROUGHPUT_MASK 0x2000000
+#define LBV_BUFFER_STATUS__LB_ENABLE_HIGH_THROUGHPUT__SHIFT 0x19
+#define LBV_BUFFER_STATUS__LB_HIGH_THROUGHPUT_CNTL_MASK 0x1c000000
+#define LBV_BUFFER_STATUS__LB_HIGH_THROUGHPUT_CNTL__SHIFT 0x1a
+#define LBV_NO_OUTSTANDING_REQ_STATUS__LB_NO_OUTSTANDING_REQ_STAT_MASK 0x1
+#define LBV_NO_OUTSTANDING_REQ_STATUS__LB_NO_OUTSTANDING_REQ_STAT__SHIFT 0x0
+#define LBV_DEBUG__LB_DEBUG_MASK 0xffffffff
+#define LBV_DEBUG__LB_DEBUG__SHIFT 0x0
+#define LBV_DEBUG2__LB_DEBUG2_MASK 0xffffffff
+#define LBV_DEBUG2__LB_DEBUG2__SHIFT 0x0
+#define LBV_DEBUG3__LB_DEBUG3_MASK 0xffffffff
+#define LBV_DEBUG3__LB_DEBUG3__SHIFT 0x0
+#define LBV_TEST_DEBUG_INDEX__LB_TEST_DEBUG_INDEX_MASK 0xff
+#define LBV_TEST_DEBUG_INDEX__LB_TEST_DEBUG_INDEX__SHIFT 0x0
+#define LBV_TEST_DEBUG_INDEX__LB_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define LBV_TEST_DEBUG_INDEX__LB_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define LBV_TEST_DEBUG_DATA__LB_TEST_DEBUG_DATA_MASK 0xffffffff
+#define LBV_TEST_DEBUG_DATA__LB_TEST_DEBUG_DATA__SHIFT 0x0
+#define MVP_CONTROL1__MVP_EN_MASK 0x1
+#define MVP_CONTROL1__MVP_EN__SHIFT 0x0
+#define MVP_CONTROL1__MVP_MIXER_MODE_MASK 0x70
+#define MVP_CONTROL1__MVP_MIXER_MODE__SHIFT 0x4
+#define MVP_CONTROL1__MVP_MIXER_SLAVE_SEL_MASK 0x100
+#define MVP_CONTROL1__MVP_MIXER_SLAVE_SEL__SHIFT 0x8
+#define MVP_CONTROL1__MVP_MIXER_SLAVE_SEL_DELAY_UNTIL_END_OF_BLANK_MASK 0x200
+#define MVP_CONTROL1__MVP_MIXER_SLAVE_SEL_DELAY_UNTIL_END_OF_BLANK__SHIFT 0x9
+#define MVP_CONTROL1__MVP_ARBITRATION_MODE_FOR_AFR_MANUAL_SWITCH_MODE_MASK 0x400
+#define MVP_CONTROL1__MVP_ARBITRATION_MODE_FOR_AFR_MANUAL_SWITCH_MODE__SHIFT 0xa
+#define MVP_CONTROL1__MVP_RATE_CONTROL_MASK 0x1000
+#define MVP_CONTROL1__MVP_RATE_CONTROL__SHIFT 0xc
+#define MVP_CONTROL1__MVP_CHANNEL_CONTROL_MASK 0x10000
+#define MVP_CONTROL1__MVP_CHANNEL_CONTROL__SHIFT 0x10
+#define MVP_CONTROL1__MVP_GPU_CHAIN_LOCATION_MASK 0x300000
+#define MVP_CONTROL1__MVP_GPU_CHAIN_LOCATION__SHIFT 0x14
+#define MVP_CONTROL1__MVP_DISABLE_MSB_EXPAND_MASK 0x1000000
+#define MVP_CONTROL1__MVP_DISABLE_MSB_EXPAND__SHIFT 0x18
+#define MVP_CONTROL1__MVP_30BPP_EN_MASK 0x10000000
+#define MVP_CONTROL1__MVP_30BPP_EN__SHIFT 0x1c
+#define MVP_CONTROL1__MVP_TERMINATION_CNTL_A_MASK 0x40000000
+#define MVP_CONTROL1__MVP_TERMINATION_CNTL_A__SHIFT 0x1e
+#define MVP_CONTROL1__MVP_TERMINATION_CNTL_B_MASK 0x80000000
+#define MVP_CONTROL1__MVP_TERMINATION_CNTL_B__SHIFT 0x1f
+#define MVP_CONTROL2__MVP_MUX_DE_DVOCNTL0_SEL_MASK 0x1
+#define MVP_CONTROL2__MVP_MUX_DE_DVOCNTL0_SEL__SHIFT 0x0
+#define MVP_CONTROL2__MVP_MUX_DE_DVOCNTL2_SEL_MASK 0x10
+#define MVP_CONTROL2__MVP_MUX_DE_DVOCNTL2_SEL__SHIFT 0x4
+#define MVP_CONTROL2__MVP_MUXA_CLK_SEL_MASK 0x100
+#define MVP_CONTROL2__MVP_MUXA_CLK_SEL__SHIFT 0x8
+#define MVP_CONTROL2__MVP_MUXB_CLK_SEL_MASK 0x1000
+#define MVP_CONTROL2__MVP_MUXB_CLK_SEL__SHIFT 0xc
+#define MVP_CONTROL2__MVP_DVOCNTL_MUX_MASK 0x10000
+#define MVP_CONTROL2__MVP_DVOCNTL_MUX__SHIFT 0x10
+#define MVP_CONTROL2__MVP_FLOW_CONTROL_OUT_EN_MASK 0x100000
+#define MVP_CONTROL2__MVP_FLOW_CONTROL_OUT_EN__SHIFT 0x14
+#define MVP_CONTROL2__MVP_SWAP_LOCK_OUT_EN_MASK 0x1000000
+#define MVP_CONTROL2__MVP_SWAP_LOCK_OUT_EN__SHIFT 0x18
+#define MVP_CONTROL2__MVP_SWAP_AB_IN_DC_DDR_MASK 0x10000000
+#define MVP_CONTROL2__MVP_SWAP_AB_IN_DC_DDR__SHIFT 0x1c
+#define MVP_FIFO_CONTROL__MVP_STOP_SLAVE_WM_MASK 0xff
+#define MVP_FIFO_CONTROL__MVP_STOP_SLAVE_WM__SHIFT 0x0
+#define MVP_FIFO_CONTROL__MVP_PAUSE_SLAVE_WM_MASK 0xff00
+#define MVP_FIFO_CONTROL__MVP_PAUSE_SLAVE_WM__SHIFT 0x8
+#define MVP_FIFO_CONTROL__MVP_PAUSE_SLAVE_CNT_MASK 0xff0000
+#define MVP_FIFO_CONTROL__MVP_PAUSE_SLAVE_CNT__SHIFT 0x10
+#define MVP_FIFO_STATUS__MVP_FIFO_LEVEL_MASK 0xff
+#define MVP_FIFO_STATUS__MVP_FIFO_LEVEL__SHIFT 0x0
+#define MVP_FIFO_STATUS__MVP_FIFO_OVERFLOW_MASK 0x100
+#define MVP_FIFO_STATUS__MVP_FIFO_OVERFLOW__SHIFT 0x8
+#define MVP_FIFO_STATUS__MVP_FIFO_OVERFLOW_OCCURRED_MASK 0x1000
+#define MVP_FIFO_STATUS__MVP_FIFO_OVERFLOW_OCCURRED__SHIFT 0xc
+#define MVP_FIFO_STATUS__MVP_FIFO_OVERFLOW_ACK_MASK 0x10000
+#define MVP_FIFO_STATUS__MVP_FIFO_OVERFLOW_ACK__SHIFT 0x10
+#define MVP_FIFO_STATUS__MVP_FIFO_UNDERFLOW_MASK 0x100000
+#define MVP_FIFO_STATUS__MVP_FIFO_UNDERFLOW__SHIFT 0x14
+#define MVP_FIFO_STATUS__MVP_FIFO_UNDERFLOW_OCCURRED_MASK 0x1000000
+#define MVP_FIFO_STATUS__MVP_FIFO_UNDERFLOW_OCCURRED__SHIFT 0x18
+#define MVP_FIFO_STATUS__MVP_FIFO_UNDERFLOW_ACK_MASK 0x10000000
+#define MVP_FIFO_STATUS__MVP_FIFO_UNDERFLOW_ACK__SHIFT 0x1c
+#define MVP_FIFO_STATUS__MVP_FIFO_ERROR_MASK_MASK 0x40000000
+#define MVP_FIFO_STATUS__MVP_FIFO_ERROR_MASK__SHIFT 0x1e
+#define MVP_FIFO_STATUS__MVP_FIFO_ERROR_INT_STATUS_MASK 0x80000000
+#define MVP_FIFO_STATUS__MVP_FIFO_ERROR_INT_STATUS__SHIFT 0x1f
+#define MVP_SLAVE_STATUS__MVP_SLAVE_PIXELS_PER_LINE_RCVED_MASK 0x1fff
+#define MVP_SLAVE_STATUS__MVP_SLAVE_PIXELS_PER_LINE_RCVED__SHIFT 0x0
+#define MVP_SLAVE_STATUS__MVP_SLAVE_LINES_PER_FRAME_RCVED_MASK 0x1fff0000
+#define MVP_SLAVE_STATUS__MVP_SLAVE_LINES_PER_FRAME_RCVED__SHIFT 0x10
+#define MVP_INBAND_CNTL_CAP__MVP_IGNOR_INBAND_CNTL_MASK 0x1
+#define MVP_INBAND_CNTL_CAP__MVP_IGNOR_INBAND_CNTL__SHIFT 0x0
+#define MVP_INBAND_CNTL_CAP__MVP_PASSING_INBAND_CNTL_EN_MASK 0x10
+#define MVP_INBAND_CNTL_CAP__MVP_PASSING_INBAND_CNTL_EN__SHIFT 0x4
+#define MVP_INBAND_CNTL_CAP__MVP_INBAND_CNTL_CHAR_CAP_MASK 0xffffff00
+#define MVP_INBAND_CNTL_CAP__MVP_INBAND_CNTL_CHAR_CAP__SHIFT 0x8
+#define MVP_BLACK_KEYER__MVP_BLACK_KEYER_R_MASK 0x3ff
+#define MVP_BLACK_KEYER__MVP_BLACK_KEYER_R__SHIFT 0x0
+#define MVP_BLACK_KEYER__MVP_BLACK_KEYER_G_MASK 0xffc00
+#define MVP_BLACK_KEYER__MVP_BLACK_KEYER_G__SHIFT 0xa
+#define MVP_BLACK_KEYER__MVP_BLACK_KEYER_B_MASK 0x3ff00000
+#define MVP_BLACK_KEYER__MVP_BLACK_KEYER_B__SHIFT 0x14
+#define MVP_CRC_CNTL__MVP_CRC_BLUE_MASK_MASK 0xff
+#define MVP_CRC_CNTL__MVP_CRC_BLUE_MASK__SHIFT 0x0
+#define MVP_CRC_CNTL__MVP_CRC_GREEN_MASK_MASK 0xff00
+#define MVP_CRC_CNTL__MVP_CRC_GREEN_MASK__SHIFT 0x8
+#define MVP_CRC_CNTL__MVP_CRC_RED_MASK_MASK 0xff0000
+#define MVP_CRC_CNTL__MVP_CRC_RED_MASK__SHIFT 0x10
+#define MVP_CRC_CNTL__MVP_CRC_EN_MASK 0x10000000
+#define MVP_CRC_CNTL__MVP_CRC_EN__SHIFT 0x1c
+#define MVP_CRC_CNTL__MVP_CRC_CONT_EN_MASK 0x20000000
+#define MVP_CRC_CNTL__MVP_CRC_CONT_EN__SHIFT 0x1d
+#define MVP_CRC_CNTL__MVP_DC_DDR_CRC_EVEN_ODD_PIX_SEL_MASK 0x40000000
+#define MVP_CRC_CNTL__MVP_DC_DDR_CRC_EVEN_ODD_PIX_SEL__SHIFT 0x1e
+#define MVP_CRC_RESULT_BLUE_GREEN__MVP_CRC_BLUE_RESULT_MASK 0xffff
+#define MVP_CRC_RESULT_BLUE_GREEN__MVP_CRC_BLUE_RESULT__SHIFT 0x0
+#define MVP_CRC_RESULT_BLUE_GREEN__MVP_CRC_GREEN_RESULT_MASK 0xffff0000
+#define MVP_CRC_RESULT_BLUE_GREEN__MVP_CRC_GREEN_RESULT__SHIFT 0x10
+#define MVP_CRC_RESULT_RED__MVP_CRC_RED_RESULT_MASK 0xffff
+#define MVP_CRC_RESULT_RED__MVP_CRC_RED_RESULT__SHIFT 0x0
+#define MVP_CONTROL3__MVP_RESET_IN_BETWEEN_FRAMES_MASK 0x1
+#define MVP_CONTROL3__MVP_RESET_IN_BETWEEN_FRAMES__SHIFT 0x0
+#define MVP_CONTROL3__MVP_DDR_SC_AB_SEL_MASK 0x10
+#define MVP_CONTROL3__MVP_DDR_SC_AB_SEL__SHIFT 0x4
+#define MVP_CONTROL3__MVP_DDR_SC_B_START_MODE_MASK 0x100
+#define MVP_CONTROL3__MVP_DDR_SC_B_START_MODE__SHIFT 0x8
+#define MVP_CONTROL3__MVP_FLOW_CONTROL_OUT_FORCE_ONE_MASK 0x1000
+#define MVP_CONTROL3__MVP_FLOW_CONTROL_OUT_FORCE_ONE__SHIFT 0xc
+#define MVP_CONTROL3__MVP_FLOW_CONTROL_OUT_FORCE_ZERO_MASK 0x10000
+#define MVP_CONTROL3__MVP_FLOW_CONTROL_OUT_FORCE_ZERO__SHIFT 0x10
+#define MVP_CONTROL3__MVP_FLOW_CONTROL_CASCADE_EN_MASK 0x100000
+#define MVP_CONTROL3__MVP_FLOW_CONTROL_CASCADE_EN__SHIFT 0x14
+#define MVP_CONTROL3__MVP_SWAP_48BIT_EN_MASK 0x1000000
+#define MVP_CONTROL3__MVP_SWAP_48BIT_EN__SHIFT 0x18
+#define MVP_CONTROL3__MVP_FLOW_CONTROL_IN_CAP_MASK 0x10000000
+#define MVP_CONTROL3__MVP_FLOW_CONTROL_IN_CAP__SHIFT 0x1c
+#define MVP_RECEIVE_CNT_CNTL1__MVP_SLAVE_PIXEL_ERROR_CNT_MASK 0x1fff
+#define MVP_RECEIVE_CNT_CNTL1__MVP_SLAVE_PIXEL_ERROR_CNT__SHIFT 0x0
+#define MVP_RECEIVE_CNT_CNTL1__MVP_SLAVE_LINE_ERROR_CNT_MASK 0x1fff0000
+#define MVP_RECEIVE_CNT_CNTL1__MVP_SLAVE_LINE_ERROR_CNT__SHIFT 0x10
+#define MVP_RECEIVE_CNT_CNTL1__MVP_SLAVE_DATA_CHK_EN_MASK 0x80000000
+#define MVP_RECEIVE_CNT_CNTL1__MVP_SLAVE_DATA_CHK_EN__SHIFT 0x1f
+#define MVP_RECEIVE_CNT_CNTL2__MVP_SLAVE_FRAME_ERROR_CNT_MASK 0x1fff
+#define MVP_RECEIVE_CNT_CNTL2__MVP_SLAVE_FRAME_ERROR_CNT__SHIFT 0x0
+#define MVP_RECEIVE_CNT_CNTL2__MVP_SLAVE_FRAME_ERROR_CNT_RESET_MASK 0x80000000
+#define MVP_RECEIVE_CNT_CNTL2__MVP_SLAVE_FRAME_ERROR_CNT_RESET__SHIFT 0x1f
+#define MVP_DEBUG__MVP_SWAP_LOCK_IN_EN_MASK 0x1
+#define MVP_DEBUG__MVP_SWAP_LOCK_IN_EN__SHIFT 0x0
+#define MVP_DEBUG__MVP_FLOW_CONTROL_IN_EN_MASK 0x2
+#define MVP_DEBUG__MVP_FLOW_CONTROL_IN_EN__SHIFT 0x1
+#define MVP_DEBUG__MVP_SWAP_LOCK_IN_SEL_MASK 0x4
+#define MVP_DEBUG__MVP_SWAP_LOCK_IN_SEL__SHIFT 0x2
+#define MVP_DEBUG__MVP_FLOW_CONTROL_IN_SEL_MASK 0x8
+#define MVP_DEBUG__MVP_FLOW_CONTROL_IN_SEL__SHIFT 0x3
+#define MVP_DEBUG__MVP_DIS_FIX_AFR_MANUAL_HSYNC_FLIP_MASK 0x10
+#define MVP_DEBUG__MVP_DIS_FIX_AFR_MANUAL_HSYNC_FLIP__SHIFT 0x4
+#define MVP_DEBUG__MVP_DIS_FIX_AFR_AUTO_VSYNC_FLIP_MASK 0x20
+#define MVP_DEBUG__MVP_DIS_FIX_AFR_AUTO_VSYNC_FLIP__SHIFT 0x5
+#define MVP_DEBUG__MVP_EN_FIX_AFR_MANUAL_SWITCH_IN_SFR_MASK 0x40
+#define MVP_DEBUG__MVP_EN_FIX_AFR_MANUAL_SWITCH_IN_SFR__SHIFT 0x6
+#define MVP_DEBUG__MVP_DIS_READ_POINTER_RESET_DELAY_MASK 0x80
+#define MVP_DEBUG__MVP_DIS_READ_POINTER_RESET_DELAY__SHIFT 0x7
+#define MVP_DEBUG__MVP_DEBUG_BITS_MASK 0xffffff00
+#define MVP_DEBUG__MVP_DEBUG_BITS__SHIFT 0x8
+#define MVP_TEST_DEBUG_INDEX__MVP_TEST_DEBUG_INDEX_MASK 0xff
+#define MVP_TEST_DEBUG_INDEX__MVP_TEST_DEBUG_INDEX__SHIFT 0x0
+#define MVP_TEST_DEBUG_INDEX__MVP_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define MVP_TEST_DEBUG_INDEX__MVP_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define MVP_TEST_DEBUG_DATA__MVP_TEST_DEBUG_DATA_MASK 0xffffffff
+#define MVP_TEST_DEBUG_DATA__MVP_TEST_DEBUG_DATA__SHIFT 0x0
+#define MVP_DEBUG_12__IDEC_MVP_DATA_A_H_MASK 0x1
+#define MVP_DEBUG_12__IDEC_MVP_DATA_A_H__SHIFT 0x0
+#define MVP_DEBUG_12__IDEC_MVP_DATA_A_MASK 0x1fffffe
+#define MVP_DEBUG_12__IDEC_MVP_DATA_A__SHIFT 0x1
+#define MVP_DEBUG_13__IDED_MVP_DATA_B_H_MASK 0x1
+#define MVP_DEBUG_13__IDED_MVP_DATA_B_H__SHIFT 0x0
+#define MVP_DEBUG_13__IDED_MVP_DATA_B_MASK 0x1fffffe
+#define MVP_DEBUG_13__IDED_MVP_DATA_B__SHIFT 0x1
+#define MVP_DEBUG_13__IDED_START_READ_B_MASK 0x2000000
+#define MVP_DEBUG_13__IDED_START_READ_B__SHIFT 0x19
+#define MVP_DEBUG_13__IDED_READ_FIFO_ENTRY_DE_B_MASK 0x4000000
+#define MVP_DEBUG_13__IDED_READ_FIFO_ENTRY_DE_B__SHIFT 0x1a
+#define MVP_DEBUG_13__IDED_WRITE_ADD_B_MASK 0x38000000
+#define MVP_DEBUG_13__IDED_WRITE_ADD_B__SHIFT 0x1b
+#define MVP_DEBUG_14__IDEE_READ_ADD_MASK 0x7
+#define MVP_DEBUG_14__IDEE_READ_ADD__SHIFT 0x0
+#define MVP_DEBUG_14__IDEE_WRITE_ADD_A_MASK 0x38
+#define MVP_DEBUG_14__IDEE_WRITE_ADD_A__SHIFT 0x3
+#define MVP_DEBUG_14__IDEE_WRITE_ADD_B_MASK 0x1c0
+#define MVP_DEBUG_14__IDEE_WRITE_ADD_B__SHIFT 0x6
+#define MVP_DEBUG_14__IDEE_START_READ_MASK 0x200
+#define MVP_DEBUG_14__IDEE_START_READ__SHIFT 0x9
+#define MVP_DEBUG_14__IDEE_START_READ_B_MASK 0x400
+#define MVP_DEBUG_14__IDEE_START_READ_B__SHIFT 0xa
+#define MVP_DEBUG_14__IDEE_START_INCR_WR_A_MASK 0x800
+#define MVP_DEBUG_14__IDEE_START_INCR_WR_A__SHIFT 0xb
+#define MVP_DEBUG_14__IDEE_START_INCR_WR_B_MASK 0x1000
+#define MVP_DEBUG_14__IDEE_START_INCR_WR_B__SHIFT 0xc
+#define MVP_DEBUG_14__IDEE_WRITE2FIFO_MASK 0x2000
+#define MVP_DEBUG_14__IDEE_WRITE2FIFO__SHIFT 0xd
+#define MVP_DEBUG_14__IDEE_READ_FIFO_ENTRY_DE_MASK 0x4000
+#define MVP_DEBUG_14__IDEE_READ_FIFO_ENTRY_DE__SHIFT 0xe
+#define MVP_DEBUG_14__IDEE_READ_FIFO_ENTRY_DE_B_MASK 0x8000
+#define MVP_DEBUG_14__IDEE_READ_FIFO_ENTRY_DE_B__SHIFT 0xf
+#define MVP_DEBUG_14__IDEE_READ_FIFO_DE_MASK 0x10000
+#define MVP_DEBUG_14__IDEE_READ_FIFO_DE__SHIFT 0x10
+#define MVP_DEBUG_14__IDEE_READ_FIFO_DE_B_MASK 0x20000
+#define MVP_DEBUG_14__IDEE_READ_FIFO_DE_B__SHIFT 0x11
+#define MVP_DEBUG_14__IDEE_READ_FIFO_ENABLE_MASK 0x40000
+#define MVP_DEBUG_14__IDEE_READ_FIFO_ENABLE__SHIFT 0x12
+#define MVP_DEBUG_14__IDEE_CRTC1_CNTL_CAPTURE_START_A_MASK 0x80000
+#define MVP_DEBUG_14__IDEE_CRTC1_CNTL_CAPTURE_START_A__SHIFT 0x13
+#define MVP_DEBUG_14__IDEE_CRC_PHASE_MASK 0x100000
+#define MVP_DEBUG_14__IDEE_CRC_PHASE__SHIFT 0x14
+#define MVP_DEBUG_15__IDEF_MVP_ASYNC_FIFO_WEN_MASK 0x1
+#define MVP_DEBUG_15__IDEF_MVP_ASYNC_FIFO_WEN__SHIFT 0x0
+#define MVP_DEBUG_15__IDEF_MVP_ASYNC_FIFO_WDATA_MASK 0xfffffff0
+#define MVP_DEBUG_15__IDEF_MVP_ASYNC_FIFO_WDATA__SHIFT 0x4
+#define MVP_DEBUG_16__IDCC_MVP_ASYNC_FIFO_READ_MASK 0x1
+#define MVP_DEBUG_16__IDCC_MVP_ASYNC_FIFO_READ__SHIFT 0x0
+#define MVP_DEBUG_16__IDCC_MVP_ASYNC_FIFO_EXCEED_STOP_LEVEL_MASK 0x2
+#define MVP_DEBUG_16__IDCC_MVP_ASYNC_FIFO_EXCEED_STOP_LEVEL__SHIFT 0x1
+#define MVP_DEBUG_16__IDCC_MVP_ASYNC_FIFO_EXCEED_PAUSE_LEVEL_MASK 0x4
+#define MVP_DEBUG_16__IDCC_MVP_ASYNC_FIFO_EXCEED_PAUSE_LEVEL__SHIFT 0x2
+#define MVP_DEBUG_16__IDCC_FLOW_CONTROL_OUT_MASK 0x8
+#define MVP_DEBUG_16__IDCC_FLOW_CONTROL_OUT__SHIFT 0x3
+#define MVP_DEBUG_16__IDCC_MVP_ASYNC_FIFO_NUM_ENTRIES_MASK 0xff0
+#define MVP_DEBUG_16__IDCC_MVP_ASYNC_FIFO_NUM_ENTRIES__SHIFT 0x4
+#define MVP_DEBUG_16__IDCC_MVP_ASYNC_FIFO_OVERFLOW_MASK 0x1000
+#define MVP_DEBUG_16__IDCC_MVP_ASYNC_FIFO_OVERFLOW__SHIFT 0xc
+#define MVP_DEBUG_16__IDCC_MVP_ASYNC_FIFO_UNDERFLOW_MASK 0x2000
+#define MVP_DEBUG_16__IDCC_MVP_ASYNC_FIFO_UNDERFLOW__SHIFT 0xd
+#define MVP_DEBUG_16__IDCC_MVP_ASYNC_READ_ADDR_MASK 0xff0000
+#define MVP_DEBUG_16__IDCC_MVP_ASYNC_READ_ADDR__SHIFT 0x10
+#define MVP_DEBUG_16__IDCC_MVP_ASYNC_WRITE_ADDR_MASK 0xff000000
+#define MVP_DEBUG_16__IDCC_MVP_ASYNC_WRITE_ADDR__SHIFT 0x18
+#define MVP_DEBUG_17__IDCD_MVP_ASYNC_FIFO_READ_MASK 0x1
+#define MVP_DEBUG_17__IDCD_MVP_ASYNC_FIFO_READ__SHIFT 0x0
+#define MVP_DEBUG_17__IDCD_MVP_ASYNC_FIFO_PHASE_MASK 0x2
+#define MVP_DEBUG_17__IDCD_MVP_ASYNC_FIFO_PHASE__SHIFT 0x1
+#define MVP_DEBUG_17__IDCD_MVP_ASYNC_FIFO_READ_DATA_MASK 0xfffffffc
+#define MVP_DEBUG_17__IDCD_MVP_ASYNC_FIFO_READ_DATA__SHIFT 0x2
+#define SCL_COEF_RAM_SELECT__SCL_C_RAM_TAP_PAIR_IDX_MASK 0xf
+#define SCL_COEF_RAM_SELECT__SCL_C_RAM_TAP_PAIR_IDX__SHIFT 0x0
+#define SCL_COEF_RAM_SELECT__SCL_C_RAM_PHASE_MASK 0xf00
+#define SCL_COEF_RAM_SELECT__SCL_C_RAM_PHASE__SHIFT 0x8
+#define SCL_COEF_RAM_SELECT__SCL_C_RAM_FILTER_TYPE_MASK 0x70000
+#define SCL_COEF_RAM_SELECT__SCL_C_RAM_FILTER_TYPE__SHIFT 0x10
+#define SCL_COEF_RAM_TAP_DATA__SCL_C_RAM_EVEN_TAP_COEF_MASK 0x3fff
+#define SCL_COEF_RAM_TAP_DATA__SCL_C_RAM_EVEN_TAP_COEF__SHIFT 0x0
+#define SCL_COEF_RAM_TAP_DATA__SCL_C_RAM_EVEN_TAP_COEF_EN_MASK 0x8000
+#define SCL_COEF_RAM_TAP_DATA__SCL_C_RAM_EVEN_TAP_COEF_EN__SHIFT 0xf
+#define SCL_COEF_RAM_TAP_DATA__SCL_C_RAM_ODD_TAP_COEF_MASK 0x3fff0000
+#define SCL_COEF_RAM_TAP_DATA__SCL_C_RAM_ODD_TAP_COEF__SHIFT 0x10
+#define SCL_COEF_RAM_TAP_DATA__SCL_C_RAM_ODD_TAP_COEF_EN_MASK 0x80000000
+#define SCL_COEF_RAM_TAP_DATA__SCL_C_RAM_ODD_TAP_COEF_EN__SHIFT 0x1f
+#define SCL_MODE__SCL_MODE_MASK 0x3
+#define SCL_MODE__SCL_MODE__SHIFT 0x0
+#define SCL_MODE__SCL_PSCL_EN_MASK 0x10
+#define SCL_MODE__SCL_PSCL_EN__SHIFT 0x4
+#define SCL_TAP_CONTROL__SCL_V_NUM_OF_TAPS_MASK 0x7
+#define SCL_TAP_CONTROL__SCL_V_NUM_OF_TAPS__SHIFT 0x0
+#define SCL_TAP_CONTROL__SCL_H_NUM_OF_TAPS_MASK 0xf00
+#define SCL_TAP_CONTROL__SCL_H_NUM_OF_TAPS__SHIFT 0x8
+#define SCL_CONTROL__SCL_BOUNDARY_MODE_MASK 0x1
+#define SCL_CONTROL__SCL_BOUNDARY_MODE__SHIFT 0x0
+#define SCL_CONTROL__SCL_EARLY_EOL_MODE_MASK 0x10
+#define SCL_CONTROL__SCL_EARLY_EOL_MODE__SHIFT 0x4
+#define SCL_BYPASS_CONTROL__SCL_BYPASS_MODE_MASK 0x3
+#define SCL_BYPASS_CONTROL__SCL_BYPASS_MODE__SHIFT 0x0
+#define SCL_MANUAL_REPLICATE_CONTROL__SCL_V_MANUAL_REPLICATE_FACTOR_MASK 0xf
+#define SCL_MANUAL_REPLICATE_CONTROL__SCL_V_MANUAL_REPLICATE_FACTOR__SHIFT 0x0
+#define SCL_MANUAL_REPLICATE_CONTROL__SCL_H_MANUAL_REPLICATE_FACTOR_MASK 0xf00
+#define SCL_MANUAL_REPLICATE_CONTROL__SCL_H_MANUAL_REPLICATE_FACTOR__SHIFT 0x8
+#define SCL_AUTOMATIC_MODE_CONTROL__SCL_V_CALC_AUTO_RATIO_EN_MASK 0x1
+#define SCL_AUTOMATIC_MODE_CONTROL__SCL_V_CALC_AUTO_RATIO_EN__SHIFT 0x0
+#define SCL_AUTOMATIC_MODE_CONTROL__SCL_H_CALC_AUTO_RATIO_EN_MASK 0x10000
+#define SCL_AUTOMATIC_MODE_CONTROL__SCL_H_CALC_AUTO_RATIO_EN__SHIFT 0x10
+#define SCL_HORZ_FILTER_CONTROL__SCL_H_FILTER_PICK_NEAREST_MASK 0x1
+#define SCL_HORZ_FILTER_CONTROL__SCL_H_FILTER_PICK_NEAREST__SHIFT 0x0
+#define SCL_HORZ_FILTER_CONTROL__SCL_H_2TAP_HARDCODE_COEF_EN_MASK 0x100
+#define SCL_HORZ_FILTER_CONTROL__SCL_H_2TAP_HARDCODE_COEF_EN__SHIFT 0x8
+#define SCL_HORZ_FILTER_SCALE_RATIO__SCL_H_SCALE_RATIO_MASK 0x3ffffff
+#define SCL_HORZ_FILTER_SCALE_RATIO__SCL_H_SCALE_RATIO__SHIFT 0x0
+#define SCL_HORZ_FILTER_INIT__SCL_H_INIT_FRAC_MASK 0xffffff
+#define SCL_HORZ_FILTER_INIT__SCL_H_INIT_FRAC__SHIFT 0x0
+#define SCL_HORZ_FILTER_INIT__SCL_H_INIT_INT_MASK 0xf000000
+#define SCL_HORZ_FILTER_INIT__SCL_H_INIT_INT__SHIFT 0x18
+#define SCL_VERT_FILTER_CONTROL__SCL_V_FILTER_PICK_NEAREST_MASK 0x1
+#define SCL_VERT_FILTER_CONTROL__SCL_V_FILTER_PICK_NEAREST__SHIFT 0x0
+#define SCL_VERT_FILTER_CONTROL__SCL_V_2TAP_HARDCODE_COEF_EN_MASK 0x100
+#define SCL_VERT_FILTER_CONTROL__SCL_V_2TAP_HARDCODE_COEF_EN__SHIFT 0x8
+#define SCL_VERT_FILTER_SCALE_RATIO__SCL_V_SCALE_RATIO_MASK 0x3ffffff
+#define SCL_VERT_FILTER_SCALE_RATIO__SCL_V_SCALE_RATIO__SHIFT 0x0
+#define SCL_VERT_FILTER_INIT__SCL_V_INIT_FRAC_MASK 0xffffff
+#define SCL_VERT_FILTER_INIT__SCL_V_INIT_FRAC__SHIFT 0x0
+#define SCL_VERT_FILTER_INIT__SCL_V_INIT_INT_MASK 0x7000000
+#define SCL_VERT_FILTER_INIT__SCL_V_INIT_INT__SHIFT 0x18
+#define SCL_VERT_FILTER_INIT_BOT__SCL_V_INIT_FRAC_BOT_MASK 0xffffff
+#define SCL_VERT_FILTER_INIT_BOT__SCL_V_INIT_FRAC_BOT__SHIFT 0x0
+#define SCL_VERT_FILTER_INIT_BOT__SCL_V_INIT_INT_BOT_MASK 0x7000000
+#define SCL_VERT_FILTER_INIT_BOT__SCL_V_INIT_INT_BOT__SHIFT 0x18
+#define SCL_ROUND_OFFSET__SCL_ROUND_OFFSET_RGB_Y_MASK 0xffff
+#define SCL_ROUND_OFFSET__SCL_ROUND_OFFSET_RGB_Y__SHIFT 0x0
+#define SCL_ROUND_OFFSET__SCL_ROUND_OFFSET_CBCR_MASK 0xffff0000
+#define SCL_ROUND_OFFSET__SCL_ROUND_OFFSET_CBCR__SHIFT 0x10
+#define SCL_UPDATE__SCL_UPDATE_PENDING_MASK 0x1
+#define SCL_UPDATE__SCL_UPDATE_PENDING__SHIFT 0x0
+#define SCL_UPDATE__SCL_UPDATE_TAKEN_MASK 0x100
+#define SCL_UPDATE__SCL_UPDATE_TAKEN__SHIFT 0x8
+#define SCL_UPDATE__SCL_UPDATE_LOCK_MASK 0x10000
+#define SCL_UPDATE__SCL_UPDATE_LOCK__SHIFT 0x10
+#define SCL_UPDATE__SCL_COEF_UPDATE_COMPLETE_MASK 0x1000000
+#define SCL_UPDATE__SCL_COEF_UPDATE_COMPLETE__SHIFT 0x18
+#define SCL_F_SHARP_CONTROL__SCL_HF_SHARP_SCALE_FACTOR_MASK 0x7
+#define SCL_F_SHARP_CONTROL__SCL_HF_SHARP_SCALE_FACTOR__SHIFT 0x0
+#define SCL_F_SHARP_CONTROL__SCL_HF_SHARP_EN_MASK 0x10
+#define SCL_F_SHARP_CONTROL__SCL_HF_SHARP_EN__SHIFT 0x4
+#define SCL_F_SHARP_CONTROL__SCL_VF_SHARP_SCALE_FACTOR_MASK 0x700
+#define SCL_F_SHARP_CONTROL__SCL_VF_SHARP_SCALE_FACTOR__SHIFT 0x8
+#define SCL_F_SHARP_CONTROL__SCL_VF_SHARP_EN_MASK 0x1000
+#define SCL_F_SHARP_CONTROL__SCL_VF_SHARP_EN__SHIFT 0xc
+#define SCL_ALU_CONTROL__SCL_ALU_DISABLE_MASK 0x1
+#define SCL_ALU_CONTROL__SCL_ALU_DISABLE__SHIFT 0x0
+#define SCL_COEF_RAM_CONFLICT_STATUS__SCL_HOST_CONFLICT_FLAG_MASK 0x1
+#define SCL_COEF_RAM_CONFLICT_STATUS__SCL_HOST_CONFLICT_FLAG__SHIFT 0x0
+#define SCL_COEF_RAM_CONFLICT_STATUS__SCL_HOST_CONFLICT_ACK_MASK 0x100
+#define SCL_COEF_RAM_CONFLICT_STATUS__SCL_HOST_CONFLICT_ACK__SHIFT 0x8
+#define SCL_COEF_RAM_CONFLICT_STATUS__SCL_HOST_CONFLICT_MASK_MASK 0x1000
+#define SCL_COEF_RAM_CONFLICT_STATUS__SCL_HOST_CONFLICT_MASK__SHIFT 0xc
+#define SCL_COEF_RAM_CONFLICT_STATUS__SCL_HOST_CONFLICT_INT_STATUS_MASK 0x10000
+#define SCL_COEF_RAM_CONFLICT_STATUS__SCL_HOST_CONFLICT_INT_STATUS__SHIFT 0x10
+#define VIEWPORT_START_SECONDARY__VIEWPORT_Y_START_SECONDARY_MASK 0x3fff
+#define VIEWPORT_START_SECONDARY__VIEWPORT_Y_START_SECONDARY__SHIFT 0x0
+#define VIEWPORT_START_SECONDARY__VIEWPORT_X_START_SECONDARY_MASK 0x3fff0000
+#define VIEWPORT_START_SECONDARY__VIEWPORT_X_START_SECONDARY__SHIFT 0x10
+#define VIEWPORT_START__VIEWPORT_Y_START_MASK 0x3fff
+#define VIEWPORT_START__VIEWPORT_Y_START__SHIFT 0x0
+#define VIEWPORT_START__VIEWPORT_X_START_MASK 0x3fff0000
+#define VIEWPORT_START__VIEWPORT_X_START__SHIFT 0x10
+#define VIEWPORT_SIZE__VIEWPORT_HEIGHT_MASK 0x3fff
+#define VIEWPORT_SIZE__VIEWPORT_HEIGHT__SHIFT 0x0
+#define VIEWPORT_SIZE__VIEWPORT_WIDTH_MASK 0x3fff0000
+#define VIEWPORT_SIZE__VIEWPORT_WIDTH__SHIFT 0x10
+#define EXT_OVERSCAN_LEFT_RIGHT__EXT_OVERSCAN_RIGHT_MASK 0x1fff
+#define EXT_OVERSCAN_LEFT_RIGHT__EXT_OVERSCAN_RIGHT__SHIFT 0x0
+#define EXT_OVERSCAN_LEFT_RIGHT__EXT_OVERSCAN_LEFT_MASK 0x1fff0000
+#define EXT_OVERSCAN_LEFT_RIGHT__EXT_OVERSCAN_LEFT__SHIFT 0x10
+#define EXT_OVERSCAN_TOP_BOTTOM__EXT_OVERSCAN_BOTTOM_MASK 0x1fff
+#define EXT_OVERSCAN_TOP_BOTTOM__EXT_OVERSCAN_BOTTOM__SHIFT 0x0
+#define EXT_OVERSCAN_TOP_BOTTOM__EXT_OVERSCAN_TOP_MASK 0x1fff0000
+#define EXT_OVERSCAN_TOP_BOTTOM__EXT_OVERSCAN_TOP__SHIFT 0x10
+#define SCL_MODE_CHANGE_DET1__SCL_MODE_CHANGE_MASK 0x1
+#define SCL_MODE_CHANGE_DET1__SCL_MODE_CHANGE__SHIFT 0x0
+#define SCL_MODE_CHANGE_DET1__SCL_MODE_CHANGE_ACK_MASK 0x10
+#define SCL_MODE_CHANGE_DET1__SCL_MODE_CHANGE_ACK__SHIFT 0x4
+#define SCL_MODE_CHANGE_DET1__SCL_ALU_H_SCALE_RATIO_MASK 0xfffff80
+#define SCL_MODE_CHANGE_DET1__SCL_ALU_H_SCALE_RATIO__SHIFT 0x7
+#define SCL_MODE_CHANGE_DET2__SCL_ALU_V_SCALE_RATIO_MASK 0x1fffff
+#define SCL_MODE_CHANGE_DET2__SCL_ALU_V_SCALE_RATIO__SHIFT 0x0
+#define SCL_MODE_CHANGE_DET3__SCL_ALU_SOURCE_HEIGHT_MASK 0x3fff
+#define SCL_MODE_CHANGE_DET3__SCL_ALU_SOURCE_HEIGHT__SHIFT 0x0
+#define SCL_MODE_CHANGE_DET3__SCL_ALU_SOURCE_WIDTH_MASK 0x3fff0000
+#define SCL_MODE_CHANGE_DET3__SCL_ALU_SOURCE_WIDTH__SHIFT 0x10
+#define SCL_MODE_CHANGE_MASK__SCL_MODE_CHANGE_MASK_MASK 0x1
+#define SCL_MODE_CHANGE_MASK__SCL_MODE_CHANGE_MASK__SHIFT 0x0
+#define SCL_DEBUG2__SCL_DEBUG_REQ_MODE_MASK 0x1
+#define SCL_DEBUG2__SCL_DEBUG_REQ_MODE__SHIFT 0x0
+#define SCL_DEBUG2__SCL_DEBUG_EOF_MODE_MASK 0x6
+#define SCL_DEBUG2__SCL_DEBUG_EOF_MODE__SHIFT 0x1
+#define SCL_DEBUG2__SCL_DEBUG2_MASK 0xfffffff8
+#define SCL_DEBUG2__SCL_DEBUG2__SHIFT 0x3
+#define SCL_DEBUG__SCL_DEBUG_MASK 0xffffffff
+#define SCL_DEBUG__SCL_DEBUG__SHIFT 0x0
+#define SCL_TEST_DEBUG_INDEX__SCL_TEST_DEBUG_INDEX_MASK 0xff
+#define SCL_TEST_DEBUG_INDEX__SCL_TEST_DEBUG_INDEX__SHIFT 0x0
+#define SCL_TEST_DEBUG_INDEX__SCL_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define SCL_TEST_DEBUG_INDEX__SCL_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define SCL_TEST_DEBUG_DATA__SCL_TEST_DEBUG_DATA_MASK 0xffffffff
+#define SCL_TEST_DEBUG_DATA__SCL_TEST_DEBUG_DATA__SHIFT 0x0
+#define SCLV_COEF_RAM_SELECT__SCL_C_RAM_TAP_PAIR_IDX_MASK 0x3
+#define SCLV_COEF_RAM_SELECT__SCL_C_RAM_TAP_PAIR_IDX__SHIFT 0x0
+#define SCLV_COEF_RAM_SELECT__SCL_C_RAM_PHASE_MASK 0x7f00
+#define SCLV_COEF_RAM_SELECT__SCL_C_RAM_PHASE__SHIFT 0x8
+#define SCLV_COEF_RAM_SELECT__SCL_C_RAM_FILTER_TYPE_MASK 0x30000
+#define SCLV_COEF_RAM_SELECT__SCL_C_RAM_FILTER_TYPE__SHIFT 0x10
+#define SCLV_COEF_RAM_TAP_DATA__SCL_C_RAM_EVEN_TAP_COEF_MASK 0x3fff
+#define SCLV_COEF_RAM_TAP_DATA__SCL_C_RAM_EVEN_TAP_COEF__SHIFT 0x0
+#define SCLV_COEF_RAM_TAP_DATA__SCL_C_RAM_EVEN_TAP_COEF_EN_MASK 0x8000
+#define SCLV_COEF_RAM_TAP_DATA__SCL_C_RAM_EVEN_TAP_COEF_EN__SHIFT 0xf
+#define SCLV_COEF_RAM_TAP_DATA__SCL_C_RAM_ODD_TAP_COEF_MASK 0x3fff0000
+#define SCLV_COEF_RAM_TAP_DATA__SCL_C_RAM_ODD_TAP_COEF__SHIFT 0x10
+#define SCLV_COEF_RAM_TAP_DATA__SCL_C_RAM_ODD_TAP_COEF_EN_MASK 0x80000000
+#define SCLV_COEF_RAM_TAP_DATA__SCL_C_RAM_ODD_TAP_COEF_EN__SHIFT 0x1f
+#define SCLV_MODE__SCL_MODE_MASK 0x3
+#define SCLV_MODE__SCL_MODE__SHIFT 0x0
+#define SCLV_MODE__SCL_MODE_C_MASK 0xc
+#define SCLV_MODE__SCL_MODE_C__SHIFT 0x2
+#define SCLV_MODE__SCL_PSCL_EN_MASK 0x10
+#define SCLV_MODE__SCL_PSCL_EN__SHIFT 0x4
+#define SCLV_MODE__SCL_PSCL_EN_C_MASK 0x20
+#define SCLV_MODE__SCL_PSCL_EN_C__SHIFT 0x5
+#define SCLV_MODE__SCL_INTERLACE_SOURCE_MASK 0x300
+#define SCLV_MODE__SCL_INTERLACE_SOURCE__SHIFT 0x8
+#define SCLV_TAP_CONTROL__SCL_V_NUM_OF_TAPS_MASK 0x7
+#define SCLV_TAP_CONTROL__SCL_V_NUM_OF_TAPS__SHIFT 0x0
+#define SCLV_TAP_CONTROL__SCL_H_NUM_OF_TAPS_MASK 0x70
+#define SCLV_TAP_CONTROL__SCL_H_NUM_OF_TAPS__SHIFT 0x4
+#define SCLV_TAP_CONTROL__SCL_V_NUM_OF_TAPS_C_MASK 0x700
+#define SCLV_TAP_CONTROL__SCL_V_NUM_OF_TAPS_C__SHIFT 0x8
+#define SCLV_TAP_CONTROL__SCL_H_NUM_OF_TAPS_C_MASK 0x7000
+#define SCLV_TAP_CONTROL__SCL_H_NUM_OF_TAPS_C__SHIFT 0xc
+#define SCLV_CONTROL__SCL_BOUNDARY_MODE_MASK 0x1
+#define SCLV_CONTROL__SCL_BOUNDARY_MODE__SHIFT 0x0
+#define SCLV_CONTROL__SCL_EARLY_EOL_MODE_MASK 0x10
+#define SCLV_CONTROL__SCL_EARLY_EOL_MODE__SHIFT 0x4
+#define SCLV_CONTROL__SCL_TOTAL_PHASE_MASK 0x100
+#define SCLV_CONTROL__SCL_TOTAL_PHASE__SHIFT 0x8
+#define SCLV_MANUAL_REPLICATE_CONTROL__SCL_V_MANUAL_REPLICATE_FACTOR_MASK 0xf
+#define SCLV_MANUAL_REPLICATE_CONTROL__SCL_V_MANUAL_REPLICATE_FACTOR__SHIFT 0x0
+#define SCLV_MANUAL_REPLICATE_CONTROL__SCL_H_MANUAL_REPLICATE_FACTOR_MASK 0xf00
+#define SCLV_MANUAL_REPLICATE_CONTROL__SCL_H_MANUAL_REPLICATE_FACTOR__SHIFT 0x8
+#define SCLV_AUTOMATIC_MODE_CONTROL__SCL_V_CALC_AUTO_RATIO_EN_MASK 0x1
+#define SCLV_AUTOMATIC_MODE_CONTROL__SCL_V_CALC_AUTO_RATIO_EN__SHIFT 0x0
+#define SCLV_AUTOMATIC_MODE_CONTROL__SCL_H_CALC_AUTO_RATIO_EN_MASK 0x10000
+#define SCLV_AUTOMATIC_MODE_CONTROL__SCL_H_CALC_AUTO_RATIO_EN__SHIFT 0x10
+#define SCLV_HORZ_FILTER_CONTROL__SCL_H_2TAP_HARDCODE_COEF_EN_MASK 0x100
+#define SCLV_HORZ_FILTER_CONTROL__SCL_H_2TAP_HARDCODE_COEF_EN__SHIFT 0x8
+#define SCLV_HORZ_FILTER_SCALE_RATIO__SCL_H_SCALE_RATIO_MASK 0x3ffffff
+#define SCLV_HORZ_FILTER_SCALE_RATIO__SCL_H_SCALE_RATIO__SHIFT 0x0
+#define SCLV_HORZ_FILTER_INIT__SCL_H_INIT_FRAC_MASK 0xffffff
+#define SCLV_HORZ_FILTER_INIT__SCL_H_INIT_FRAC__SHIFT 0x0
+#define SCLV_HORZ_FILTER_INIT__SCL_H_INIT_INT_MASK 0xf000000
+#define SCLV_HORZ_FILTER_INIT__SCL_H_INIT_INT__SHIFT 0x18
+#define SCLV_HORZ_FILTER_SCALE_RATIO_C__SCL_H_SCALE_RATIO_C_MASK 0x3ffffff
+#define SCLV_HORZ_FILTER_SCALE_RATIO_C__SCL_H_SCALE_RATIO_C__SHIFT 0x0
+#define SCLV_HORZ_FILTER_INIT_C__SCL_H_INIT_FRAC_C_MASK 0xffffff
+#define SCLV_HORZ_FILTER_INIT_C__SCL_H_INIT_FRAC_C__SHIFT 0x0
+#define SCLV_HORZ_FILTER_INIT_C__SCL_H_INIT_INT_C_MASK 0xf000000
+#define SCLV_HORZ_FILTER_INIT_C__SCL_H_INIT_INT_C__SHIFT 0x18
+#define SCLV_VERT_FILTER_CONTROL__SCL_V_2TAP_HARDCODE_COEF_EN_MASK 0x100
+#define SCLV_VERT_FILTER_CONTROL__SCL_V_2TAP_HARDCODE_COEF_EN__SHIFT 0x8
+#define SCLV_VERT_FILTER_SCALE_RATIO__SCL_V_SCALE_RATIO_MASK 0x3ffffff
+#define SCLV_VERT_FILTER_SCALE_RATIO__SCL_V_SCALE_RATIO__SHIFT 0x0
+#define SCLV_VERT_FILTER_INIT__SCL_V_INIT_FRAC_MASK 0xffffff
+#define SCLV_VERT_FILTER_INIT__SCL_V_INIT_FRAC__SHIFT 0x0
+#define SCLV_VERT_FILTER_INIT__SCL_V_INIT_INT_MASK 0x7000000
+#define SCLV_VERT_FILTER_INIT__SCL_V_INIT_INT__SHIFT 0x18
+#define SCLV_VERT_FILTER_INIT_BOT__SCL_V_INIT_FRAC_BOT_MASK 0xffffff
+#define SCLV_VERT_FILTER_INIT_BOT__SCL_V_INIT_FRAC_BOT__SHIFT 0x0
+#define SCLV_VERT_FILTER_INIT_BOT__SCL_V_INIT_INT_BOT_MASK 0x7000000
+#define SCLV_VERT_FILTER_INIT_BOT__SCL_V_INIT_INT_BOT__SHIFT 0x18
+#define SCLV_VERT_FILTER_SCALE_RATIO_C__SCL_V_SCALE_RATIO_C_MASK 0x3ffffff
+#define SCLV_VERT_FILTER_SCALE_RATIO_C__SCL_V_SCALE_RATIO_C__SHIFT 0x0
+#define SCLV_VERT_FILTER_INIT_C__SCL_V_INIT_FRAC_C_MASK 0xffffff
+#define SCLV_VERT_FILTER_INIT_C__SCL_V_INIT_FRAC_C__SHIFT 0x0
+#define SCLV_VERT_FILTER_INIT_C__SCL_V_INIT_INT_C_MASK 0x7000000
+#define SCLV_VERT_FILTER_INIT_C__SCL_V_INIT_INT_C__SHIFT 0x18
+#define SCLV_VERT_FILTER_INIT_BOT_C__SCL_V_INIT_FRAC_BOT_C_MASK 0xffffff
+#define SCLV_VERT_FILTER_INIT_BOT_C__SCL_V_INIT_FRAC_BOT_C__SHIFT 0x0
+#define SCLV_VERT_FILTER_INIT_BOT_C__SCL_V_INIT_INT_BOT_C_MASK 0x7000000
+#define SCLV_VERT_FILTER_INIT_BOT_C__SCL_V_INIT_INT_BOT_C__SHIFT 0x18
+#define SCLV_ROUND_OFFSET__SCL_ROUND_OFFSET_RGB_Y_MASK 0xffff
+#define SCLV_ROUND_OFFSET__SCL_ROUND_OFFSET_RGB_Y__SHIFT 0x0
+#define SCLV_ROUND_OFFSET__SCL_ROUND_OFFSET_CBCR_MASK 0xffff0000
+#define SCLV_ROUND_OFFSET__SCL_ROUND_OFFSET_CBCR__SHIFT 0x10
+#define SCLV_UPDATE__SCL_UPDATE_PENDING_MASK 0x1
+#define SCLV_UPDATE__SCL_UPDATE_PENDING__SHIFT 0x0
+#define SCLV_UPDATE__SCL_UPDATE_TAKEN_MASK 0x100
+#define SCLV_UPDATE__SCL_UPDATE_TAKEN__SHIFT 0x8
+#define SCLV_UPDATE__SCL_UPDATE_LOCK_MASK 0x10000
+#define SCLV_UPDATE__SCL_UPDATE_LOCK__SHIFT 0x10
+#define SCLV_UPDATE__SCL_COEF_UPDATE_COMPLETE_MASK 0x1000000
+#define SCLV_UPDATE__SCL_COEF_UPDATE_COMPLETE__SHIFT 0x18
+#define SCLV_ALU_CONTROL__SCL_ALU_DISABLE_MASK 0x1
+#define SCLV_ALU_CONTROL__SCL_ALU_DISABLE__SHIFT 0x0
+#define SCLV_VIEWPORT_START__VIEWPORT_Y_START_MASK 0x3fff
+#define SCLV_VIEWPORT_START__VIEWPORT_Y_START__SHIFT 0x0
+#define SCLV_VIEWPORT_START__VIEWPORT_X_START_MASK 0x3fff0000
+#define SCLV_VIEWPORT_START__VIEWPORT_X_START__SHIFT 0x10
+#define SCLV_VIEWPORT_START_SECONDARY__VIEWPORT_Y_START_SECONDARY_MASK 0x3fff
+#define SCLV_VIEWPORT_START_SECONDARY__VIEWPORT_Y_START_SECONDARY__SHIFT 0x0
+#define SCLV_VIEWPORT_START_SECONDARY__VIEWPORT_X_START_SECONDARY_MASK 0x3fff0000
+#define SCLV_VIEWPORT_START_SECONDARY__VIEWPORT_X_START_SECONDARY__SHIFT 0x10
+#define SCLV_VIEWPORT_SIZE__VIEWPORT_HEIGHT_MASK 0x1fff
+#define SCLV_VIEWPORT_SIZE__VIEWPORT_HEIGHT__SHIFT 0x0
+#define SCLV_VIEWPORT_SIZE__VIEWPORT_WIDTH_MASK 0x1fff0000
+#define SCLV_VIEWPORT_SIZE__VIEWPORT_WIDTH__SHIFT 0x10
+#define SCLV_VIEWPORT_START_C__VIEWPORT_Y_START_C_MASK 0x3fff
+#define SCLV_VIEWPORT_START_C__VIEWPORT_Y_START_C__SHIFT 0x0
+#define SCLV_VIEWPORT_START_C__VIEWPORT_X_START_C_MASK 0x3fff0000
+#define SCLV_VIEWPORT_START_C__VIEWPORT_X_START_C__SHIFT 0x10
+#define SCLV_VIEWPORT_START_SECONDARY_C__VIEWPORT_Y_START_SECONDARY_C_MASK 0x3fff
+#define SCLV_VIEWPORT_START_SECONDARY_C__VIEWPORT_Y_START_SECONDARY_C__SHIFT 0x0
+#define SCLV_VIEWPORT_START_SECONDARY_C__VIEWPORT_X_START_SECONDARY_C_MASK 0x3fff0000
+#define SCLV_VIEWPORT_START_SECONDARY_C__VIEWPORT_X_START_SECONDARY_C__SHIFT 0x10
+#define SCLV_VIEWPORT_SIZE_C__VIEWPORT_HEIGHT_C_MASK 0x1fff
+#define SCLV_VIEWPORT_SIZE_C__VIEWPORT_HEIGHT_C__SHIFT 0x0
+#define SCLV_VIEWPORT_SIZE_C__VIEWPORT_WIDTH_C_MASK 0x1fff0000
+#define SCLV_VIEWPORT_SIZE_C__VIEWPORT_WIDTH_C__SHIFT 0x10
+#define SCLV_EXT_OVERSCAN_LEFT_RIGHT__EXT_OVERSCAN_RIGHT_MASK 0x1fff
+#define SCLV_EXT_OVERSCAN_LEFT_RIGHT__EXT_OVERSCAN_RIGHT__SHIFT 0x0
+#define SCLV_EXT_OVERSCAN_LEFT_RIGHT__EXT_OVERSCAN_LEFT_MASK 0x1fff0000
+#define SCLV_EXT_OVERSCAN_LEFT_RIGHT__EXT_OVERSCAN_LEFT__SHIFT 0x10
+#define SCLV_EXT_OVERSCAN_TOP_BOTTOM__EXT_OVERSCAN_BOTTOM_MASK 0x1fff
+#define SCLV_EXT_OVERSCAN_TOP_BOTTOM__EXT_OVERSCAN_BOTTOM__SHIFT 0x0
+#define SCLV_EXT_OVERSCAN_TOP_BOTTOM__EXT_OVERSCAN_TOP_MASK 0x1fff0000
+#define SCLV_EXT_OVERSCAN_TOP_BOTTOM__EXT_OVERSCAN_TOP__SHIFT 0x10
+#define SCLV_MODE_CHANGE_DET1__SCL_MODE_CHANGE_MASK 0x1
+#define SCLV_MODE_CHANGE_DET1__SCL_MODE_CHANGE__SHIFT 0x0
+#define SCLV_MODE_CHANGE_DET1__SCL_MODE_CHANGE_ACK_MASK 0x10
+#define SCLV_MODE_CHANGE_DET1__SCL_MODE_CHANGE_ACK__SHIFT 0x4
+#define SCLV_MODE_CHANGE_DET1__SCL_ALU_H_SCALE_RATIO_MASK 0xfffff80
+#define SCLV_MODE_CHANGE_DET1__SCL_ALU_H_SCALE_RATIO__SHIFT 0x7
+#define SCLV_MODE_CHANGE_DET2__SCL_ALU_V_SCALE_RATIO_MASK 0x1fffff
+#define SCLV_MODE_CHANGE_DET2__SCL_ALU_V_SCALE_RATIO__SHIFT 0x0
+#define SCLV_MODE_CHANGE_DET3__SCL_ALU_SOURCE_HEIGHT_MASK 0x3fff
+#define SCLV_MODE_CHANGE_DET3__SCL_ALU_SOURCE_HEIGHT__SHIFT 0x0
+#define SCLV_MODE_CHANGE_DET3__SCL_ALU_SOURCE_WIDTH_MASK 0x3fff0000
+#define SCLV_MODE_CHANGE_DET3__SCL_ALU_SOURCE_WIDTH__SHIFT 0x10
+#define SCLV_MODE_CHANGE_MASK__SCL_MODE_CHANGE_MASK_MASK 0x1
+#define SCLV_MODE_CHANGE_MASK__SCL_MODE_CHANGE_MASK__SHIFT 0x0
+#define SCLV_HORZ_FILTER_INIT_BOT__SCL_H_INIT_FRAC_BOT_MASK 0xffffff
+#define SCLV_HORZ_FILTER_INIT_BOT__SCL_H_INIT_FRAC_BOT__SHIFT 0x0
+#define SCLV_HORZ_FILTER_INIT_BOT__SCL_H_INIT_INT_BOT_MASK 0xf000000
+#define SCLV_HORZ_FILTER_INIT_BOT__SCL_H_INIT_INT_BOT__SHIFT 0x18
+#define SCLV_HORZ_FILTER_INIT_BOT_C__SCL_H_INIT_FRAC_BOT_C_MASK 0xffffff
+#define SCLV_HORZ_FILTER_INIT_BOT_C__SCL_H_INIT_FRAC_BOT_C__SHIFT 0x0
+#define SCLV_HORZ_FILTER_INIT_BOT_C__SCL_H_INIT_INT_BOT_C_MASK 0xf000000
+#define SCLV_HORZ_FILTER_INIT_BOT_C__SCL_H_INIT_INT_BOT_C__SHIFT 0x18
+#define SCLV_DEBUG2__SCL_DEBUG_REQ_MODE_MASK 0x1
+#define SCLV_DEBUG2__SCL_DEBUG_REQ_MODE__SHIFT 0x0
+#define SCLV_DEBUG2__SCL_DEBUG_EOF_MODE_MASK 0x6
+#define SCLV_DEBUG2__SCL_DEBUG_EOF_MODE__SHIFT 0x1
+#define SCLV_DEBUG2__SCL_DEBUG2_MASK 0xfffffff8
+#define SCLV_DEBUG2__SCL_DEBUG2__SHIFT 0x3
+#define SCLV_DEBUG__SCL_DEBUG_MASK 0xffffffff
+#define SCLV_DEBUG__SCL_DEBUG__SHIFT 0x0
+#define SCLV_TEST_DEBUG_INDEX__SCL_TEST_DEBUG_INDEX_MASK 0xff
+#define SCLV_TEST_DEBUG_INDEX__SCL_TEST_DEBUG_INDEX__SHIFT 0x0
+#define SCLV_TEST_DEBUG_INDEX__SCL_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define SCLV_TEST_DEBUG_INDEX__SCL_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define SCLV_TEST_DEBUG_DATA__SCL_TEST_DEBUG_DATA_MASK 0xffffffff
+#define SCLV_TEST_DEBUG_DATA__SCL_TEST_DEBUG_DATA__SHIFT 0x0
+#define COL_MAN_UPDATE__COL_MAN_UPDATE_PENDING_MASK 0x1
+#define COL_MAN_UPDATE__COL_MAN_UPDATE_PENDING__SHIFT 0x0
+#define COL_MAN_UPDATE__COL_MAN_UPDATE_TAKEN_MASK 0x2
+#define COL_MAN_UPDATE__COL_MAN_UPDATE_TAKEN__SHIFT 0x1
+#define COL_MAN_UPDATE__COL_MAN_UPDATE_LOCK_MASK 0x10000
+#define COL_MAN_UPDATE__COL_MAN_UPDATE_LOCK__SHIFT 0x10
+#define COL_MAN_UPDATE__COL_MAN_DISABLE_MULTIPLE_UPDATE_MASK 0x1000000
+#define COL_MAN_UPDATE__COL_MAN_DISABLE_MULTIPLE_UPDATE__SHIFT 0x18
+#define COL_MAN_INPUT_CSC_CONTROL__INPUT_CSC_MODE_MASK 0x3
+#define COL_MAN_INPUT_CSC_CONTROL__INPUT_CSC_MODE__SHIFT 0x0
+#define COL_MAN_INPUT_CSC_CONTROL__INPUT_CSC_INPUT_TYPE_MASK 0x300
+#define COL_MAN_INPUT_CSC_CONTROL__INPUT_CSC_INPUT_TYPE__SHIFT 0x8
+#define COL_MAN_INPUT_CSC_CONTROL__INPUT_CSC_CONVERSION_MODE_MASK 0x10000
+#define COL_MAN_INPUT_CSC_CONTROL__INPUT_CSC_CONVERSION_MODE__SHIFT 0x10
+#define INPUT_CSC_C11_C12_A__INPUT_CSC_C11_A_MASK 0xffff
+#define INPUT_CSC_C11_C12_A__INPUT_CSC_C11_A__SHIFT 0x0
+#define INPUT_CSC_C11_C12_A__INPUT_CSC_C12_A_MASK 0xffff0000
+#define INPUT_CSC_C11_C12_A__INPUT_CSC_C12_A__SHIFT 0x10
+#define INPUT_CSC_C13_C14_A__INPUT_CSC_C13_A_MASK 0xffff
+#define INPUT_CSC_C13_C14_A__INPUT_CSC_C13_A__SHIFT 0x0
+#define INPUT_CSC_C13_C14_A__INPUT_CSC_C14_A_MASK 0xffff0000
+#define INPUT_CSC_C13_C14_A__INPUT_CSC_C14_A__SHIFT 0x10
+#define INPUT_CSC_C21_C22_A__INPUT_CSC_C21_A_MASK 0xffff
+#define INPUT_CSC_C21_C22_A__INPUT_CSC_C21_A__SHIFT 0x0
+#define INPUT_CSC_C21_C22_A__INPUT_CSC_C22_A_MASK 0xffff0000
+#define INPUT_CSC_C21_C22_A__INPUT_CSC_C22_A__SHIFT 0x10
+#define INPUT_CSC_C23_C24_A__INPUT_CSC_C23_A_MASK 0xffff
+#define INPUT_CSC_C23_C24_A__INPUT_CSC_C23_A__SHIFT 0x0
+#define INPUT_CSC_C23_C24_A__INPUT_CSC_C24_A_MASK 0xffff0000
+#define INPUT_CSC_C23_C24_A__INPUT_CSC_C24_A__SHIFT 0x10
+#define INPUT_CSC_C31_C32_A__INPUT_CSC_C31_A_MASK 0xffff
+#define INPUT_CSC_C31_C32_A__INPUT_CSC_C31_A__SHIFT 0x0
+#define INPUT_CSC_C31_C32_A__INPUT_CSC_C32_A_MASK 0xffff0000
+#define INPUT_CSC_C31_C32_A__INPUT_CSC_C32_A__SHIFT 0x10
+#define INPUT_CSC_C33_C34_A__INPUT_CSC_C33_A_MASK 0xffff
+#define INPUT_CSC_C33_C34_A__INPUT_CSC_C33_A__SHIFT 0x0
+#define INPUT_CSC_C33_C34_A__INPUT_CSC_C34_A_MASK 0xffff0000
+#define INPUT_CSC_C33_C34_A__INPUT_CSC_C34_A__SHIFT 0x10
+#define INPUT_CSC_C11_C12_B__INPUT_CSC_C11_B_MASK 0xffff
+#define INPUT_CSC_C11_C12_B__INPUT_CSC_C11_B__SHIFT 0x0
+#define INPUT_CSC_C11_C12_B__INPUT_CSC_C12_B_MASK 0xffff0000
+#define INPUT_CSC_C11_C12_B__INPUT_CSC_C12_B__SHIFT 0x10
+#define INPUT_CSC_C13_C14_B__INPUT_CSC_C13_B_MASK 0xffff
+#define INPUT_CSC_C13_C14_B__INPUT_CSC_C13_B__SHIFT 0x0
+#define INPUT_CSC_C13_C14_B__INPUT_CSC_C14_B_MASK 0xffff0000
+#define INPUT_CSC_C13_C14_B__INPUT_CSC_C14_B__SHIFT 0x10
+#define INPUT_CSC_C21_C22_B__INPUT_CSC_C21_B_MASK 0xffff
+#define INPUT_CSC_C21_C22_B__INPUT_CSC_C21_B__SHIFT 0x0
+#define INPUT_CSC_C21_C22_B__INPUT_CSC_C22_B_MASK 0xffff0000
+#define INPUT_CSC_C21_C22_B__INPUT_CSC_C22_B__SHIFT 0x10
+#define INPUT_CSC_C23_C24_B__INPUT_CSC_C23_B_MASK 0xffff
+#define INPUT_CSC_C23_C24_B__INPUT_CSC_C23_B__SHIFT 0x0
+#define INPUT_CSC_C23_C24_B__INPUT_CSC_C24_B_MASK 0xffff0000
+#define INPUT_CSC_C23_C24_B__INPUT_CSC_C24_B__SHIFT 0x10
+#define INPUT_CSC_C31_C32_B__INPUT_CSC_C31_B_MASK 0xffff
+#define INPUT_CSC_C31_C32_B__INPUT_CSC_C31_B__SHIFT 0x0
+#define INPUT_CSC_C31_C32_B__INPUT_CSC_C32_B_MASK 0xffff0000
+#define INPUT_CSC_C31_C32_B__INPUT_CSC_C32_B__SHIFT 0x10
+#define INPUT_CSC_C33_C34_B__INPUT_CSC_C33_B_MASK 0xffff
+#define INPUT_CSC_C33_C34_B__INPUT_CSC_C33_B__SHIFT 0x0
+#define INPUT_CSC_C33_C34_B__INPUT_CSC_C34_B_MASK 0xffff0000
+#define INPUT_CSC_C33_C34_B__INPUT_CSC_C34_B__SHIFT 0x10
+#define PRESCALE_CONTROL__PRESCALE_MODE_MASK 0x3
+#define PRESCALE_CONTROL__PRESCALE_MODE__SHIFT 0x0
+#define PRESCALE_VALUES_R__PRESCALE_BIAS_R_MASK 0xffff
+#define PRESCALE_VALUES_R__PRESCALE_BIAS_R__SHIFT 0x0
+#define PRESCALE_VALUES_R__PRESCALE_SCALE_R_MASK 0xffff0000
+#define PRESCALE_VALUES_R__PRESCALE_SCALE_R__SHIFT 0x10
+#define PRESCALE_VALUES_G__PRESCALE_BIAS_G_MASK 0xffff
+#define PRESCALE_VALUES_G__PRESCALE_BIAS_G__SHIFT 0x0
+#define PRESCALE_VALUES_G__PRESCALE_SCALE_G_MASK 0xffff0000
+#define PRESCALE_VALUES_G__PRESCALE_SCALE_G__SHIFT 0x10
+#define PRESCALE_VALUES_B__PRESCALE_BIAS_B_MASK 0xffff
+#define PRESCALE_VALUES_B__PRESCALE_BIAS_B__SHIFT 0x0
+#define PRESCALE_VALUES_B__PRESCALE_SCALE_B_MASK 0xffff0000
+#define PRESCALE_VALUES_B__PRESCALE_SCALE_B__SHIFT 0x10
+#define COL_MAN_OUTPUT_CSC_CONTROL__OUTPUT_CSC_MODE_MASK 0x7
+#define COL_MAN_OUTPUT_CSC_CONTROL__OUTPUT_CSC_MODE__SHIFT 0x0
+#define OUTPUT_CSC_C11_C12_A__OUTPUT_CSC_C11_A_MASK 0xffff
+#define OUTPUT_CSC_C11_C12_A__OUTPUT_CSC_C11_A__SHIFT 0x0
+#define OUTPUT_CSC_C11_C12_A__OUTPUT_CSC_C12_A_MASK 0xffff0000
+#define OUTPUT_CSC_C11_C12_A__OUTPUT_CSC_C12_A__SHIFT 0x10
+#define OUTPUT_CSC_C13_C14_A__OUTPUT_CSC_C13_A_MASK 0xffff
+#define OUTPUT_CSC_C13_C14_A__OUTPUT_CSC_C13_A__SHIFT 0x0
+#define OUTPUT_CSC_C13_C14_A__OUTPUT_CSC_C14_A_MASK 0xffff0000
+#define OUTPUT_CSC_C13_C14_A__OUTPUT_CSC_C14_A__SHIFT 0x10
+#define OUTPUT_CSC_C21_C22_A__OUTPUT_CSC_C21_A_MASK 0xffff
+#define OUTPUT_CSC_C21_C22_A__OUTPUT_CSC_C21_A__SHIFT 0x0
+#define OUTPUT_CSC_C21_C22_A__OUTPUT_CSC_C22_A_MASK 0xffff0000
+#define OUTPUT_CSC_C21_C22_A__OUTPUT_CSC_C22_A__SHIFT 0x10
+#define OUTPUT_CSC_C23_C24_A__OUTPUT_CSC_C23_A_MASK 0xffff
+#define OUTPUT_CSC_C23_C24_A__OUTPUT_CSC_C23_A__SHIFT 0x0
+#define OUTPUT_CSC_C23_C24_A__OUTPUT_CSC_C24_A_MASK 0xffff0000
+#define OUTPUT_CSC_C23_C24_A__OUTPUT_CSC_C24_A__SHIFT 0x10
+#define OUTPUT_CSC_C31_C32_A__OUTPUT_CSC_C31_A_MASK 0xffff
+#define OUTPUT_CSC_C31_C32_A__OUTPUT_CSC_C31_A__SHIFT 0x0
+#define OUTPUT_CSC_C31_C32_A__OUTPUT_CSC_C32_A_MASK 0xffff0000
+#define OUTPUT_CSC_C31_C32_A__OUTPUT_CSC_C32_A__SHIFT 0x10
+#define OUTPUT_CSC_C33_C34_A__OUTPUT_CSC_C33_A_MASK 0xffff
+#define OUTPUT_CSC_C33_C34_A__OUTPUT_CSC_C33_A__SHIFT 0x0
+#define OUTPUT_CSC_C33_C34_A__OUTPUT_CSC_C34_A_MASK 0xffff0000
+#define OUTPUT_CSC_C33_C34_A__OUTPUT_CSC_C34_A__SHIFT 0x10
+#define OUTPUT_CSC_C11_C12_B__OUTPUT_CSC_C11_B_MASK 0xffff
+#define OUTPUT_CSC_C11_C12_B__OUTPUT_CSC_C11_B__SHIFT 0x0
+#define OUTPUT_CSC_C11_C12_B__OUTPUT_CSC_C12_B_MASK 0xffff0000
+#define OUTPUT_CSC_C11_C12_B__OUTPUT_CSC_C12_B__SHIFT 0x10
+#define OUTPUT_CSC_C13_C14_B__OUTPUT_CSC_C13_B_MASK 0xffff
+#define OUTPUT_CSC_C13_C14_B__OUTPUT_CSC_C13_B__SHIFT 0x0
+#define OUTPUT_CSC_C13_C14_B__OUTPUT_CSC_C14_B_MASK 0xffff0000
+#define OUTPUT_CSC_C13_C14_B__OUTPUT_CSC_C14_B__SHIFT 0x10
+#define OUTPUT_CSC_C21_C22_B__OUTPUT_CSC_C21_B_MASK 0xffff
+#define OUTPUT_CSC_C21_C22_B__OUTPUT_CSC_C21_B__SHIFT 0x0
+#define OUTPUT_CSC_C21_C22_B__OUTPUT_CSC_C22_B_MASK 0xffff0000
+#define OUTPUT_CSC_C21_C22_B__OUTPUT_CSC_C22_B__SHIFT 0x10
+#define OUTPUT_CSC_C23_C24_B__OUTPUT_CSC_C23_B_MASK 0xffff
+#define OUTPUT_CSC_C23_C24_B__OUTPUT_CSC_C23_B__SHIFT 0x0
+#define OUTPUT_CSC_C23_C24_B__OUTPUT_CSC_C24_B_MASK 0xffff0000
+#define OUTPUT_CSC_C23_C24_B__OUTPUT_CSC_C24_B__SHIFT 0x10
+#define OUTPUT_CSC_C31_C32_B__OUTPUT_CSC_C31_B_MASK 0xffff
+#define OUTPUT_CSC_C31_C32_B__OUTPUT_CSC_C31_B__SHIFT 0x0
+#define OUTPUT_CSC_C31_C32_B__OUTPUT_CSC_C32_B_MASK 0xffff0000
+#define OUTPUT_CSC_C31_C32_B__OUTPUT_CSC_C32_B__SHIFT 0x10
+#define OUTPUT_CSC_C33_C34_B__OUTPUT_CSC_C33_B_MASK 0xffff
+#define OUTPUT_CSC_C33_C34_B__OUTPUT_CSC_C33_B__SHIFT 0x0
+#define OUTPUT_CSC_C33_C34_B__OUTPUT_CSC_C34_B_MASK 0xffff0000
+#define OUTPUT_CSC_C33_C34_B__OUTPUT_CSC_C34_B__SHIFT 0x10
+#define DENORM_CLAMP_CONTROL__DENORM_MODE_MASK 0x3
+#define DENORM_CLAMP_CONTROL__DENORM_MODE__SHIFT 0x0
+#define DENORM_CLAMP_CONTROL__DENORM_10BIT_OUT_MASK 0x100
+#define DENORM_CLAMP_CONTROL__DENORM_10BIT_OUT__SHIFT 0x8
+#define DENORM_CLAMP_RANGE_R_CR__RANGE_CLAMP_MAX_R_CR_MASK 0xfff
+#define DENORM_CLAMP_RANGE_R_CR__RANGE_CLAMP_MAX_R_CR__SHIFT 0x0
+#define DENORM_CLAMP_RANGE_R_CR__RANGE_CLAMP_MIN_R_CR_MASK 0xfff000
+#define DENORM_CLAMP_RANGE_R_CR__RANGE_CLAMP_MIN_R_CR__SHIFT 0xc
+#define DENORM_CLAMP_RANGE_G_Y__RANGE_CLAMP_MAX_G_Y_MASK 0xfff
+#define DENORM_CLAMP_RANGE_G_Y__RANGE_CLAMP_MAX_G_Y__SHIFT 0x0
+#define DENORM_CLAMP_RANGE_G_Y__RANGE_CLAMP_MIN_G_Y_MASK 0xfff000
+#define DENORM_CLAMP_RANGE_G_Y__RANGE_CLAMP_MIN_G_Y__SHIFT 0xc
+#define DENORM_CLAMP_RANGE_B_CB__RANGE_CLAMP_MAX_B_CB_MASK 0xfff
+#define DENORM_CLAMP_RANGE_B_CB__RANGE_CLAMP_MAX_B_CB__SHIFT 0x0
+#define DENORM_CLAMP_RANGE_B_CB__RANGE_CLAMP_MIN_B_CB_MASK 0xfff000
+#define DENORM_CLAMP_RANGE_B_CB__RANGE_CLAMP_MIN_B_CB__SHIFT 0xc
+#define COL_MAN_FP_CONVERTED_FIELD__COL_MAN_FP_CONVERTED_FIELD_DATA_MASK 0x3ffff
+#define COL_MAN_FP_CONVERTED_FIELD__COL_MAN_FP_CONVERTED_FIELD_DATA__SHIFT 0x0
+#define COL_MAN_FP_CONVERTED_FIELD__COL_MAN_FP_CONVERTED_FIELD_INDEX_MASK 0x3f00000
+#define COL_MAN_FP_CONVERTED_FIELD__COL_MAN_FP_CONVERTED_FIELD_INDEX__SHIFT 0x14
+#define GAMMA_CORR_CONTROL__GAMMA_CORR_MODE_MASK 0x3
+#define GAMMA_CORR_CONTROL__GAMMA_CORR_MODE__SHIFT 0x0
+#define GAMMA_CORR_LUT_INDEX__GAMMA_CORR_LUT_INDEX_MASK 0xff
+#define GAMMA_CORR_LUT_INDEX__GAMMA_CORR_LUT_INDEX__SHIFT 0x0
+#define GAMMA_CORR_LUT_DATA__GAMMA_CORR_LUT_DATA_MASK 0x7ffff
+#define GAMMA_CORR_LUT_DATA__GAMMA_CORR_LUT_DATA__SHIFT 0x0
+#define GAMMA_CORR_LUT_WRITE_EN_MASK__GAMMA_CORR_LUT_WRITE_EN_MASK_MASK 0x7
+#define GAMMA_CORR_LUT_WRITE_EN_MASK__GAMMA_CORR_LUT_WRITE_EN_MASK__SHIFT 0x0
+#define GAMMA_CORR_CNTLA_START_CNTL__GAMMA_CORR_CNTLA_EXP_REGION_START_MASK 0x3ffff
+#define GAMMA_CORR_CNTLA_START_CNTL__GAMMA_CORR_CNTLA_EXP_REGION_START__SHIFT 0x0
+#define GAMMA_CORR_CNTLA_START_CNTL__GAMMA_CORR_CNTLA_EXP_REGION_START_SEGMENT_MASK 0x7f00000
+#define GAMMA_CORR_CNTLA_START_CNTL__GAMMA_CORR_CNTLA_EXP_REGION_START_SEGMENT__SHIFT 0x14
+#define GAMMA_CORR_CNTLA_SLOPE_CNTL__GAMMA_CORR_CNTLA_EXP_REGION_LINEAR_SLOPE_MASK 0x3ffff
+#define GAMMA_CORR_CNTLA_SLOPE_CNTL__GAMMA_CORR_CNTLA_EXP_REGION_LINEAR_SLOPE__SHIFT 0x0
+#define GAMMA_CORR_CNTLA_END_CNTL1__GAMMA_CORR_CNTLA_EXP_REGION_END_MASK 0xffff
+#define GAMMA_CORR_CNTLA_END_CNTL1__GAMMA_CORR_CNTLA_EXP_REGION_END__SHIFT 0x0
+#define GAMMA_CORR_CNTLA_END_CNTL2__GAMMA_CORR_CNTLA_EXP_REGION_END_SLOPE_MASK 0xffff
+#define GAMMA_CORR_CNTLA_END_CNTL2__GAMMA_CORR_CNTLA_EXP_REGION_END_SLOPE__SHIFT 0x0
+#define GAMMA_CORR_CNTLA_END_CNTL2__GAMMA_CORR_CNTLA_EXP_REGION_END_BASE_MASK 0xffff0000
+#define GAMMA_CORR_CNTLA_END_CNTL2__GAMMA_CORR_CNTLA_EXP_REGION_END_BASE__SHIFT 0x10
+#define GAMMA_CORR_CNTLA_REGION_0_1__GAMMA_CORR_CNTLA_EXP_REGION0_LUT_OFFSET_MASK 0xff
+#define GAMMA_CORR_CNTLA_REGION_0_1__GAMMA_CORR_CNTLA_EXP_REGION0_LUT_OFFSET__SHIFT 0x0
+#define GAMMA_CORR_CNTLA_REGION_0_1__GAMMA_CORR_CNTLA_EXP_REGION0_NUM_SEGMENTS_MASK 0x3800
+#define GAMMA_CORR_CNTLA_REGION_0_1__GAMMA_CORR_CNTLA_EXP_REGION0_NUM_SEGMENTS__SHIFT 0xb
+#define GAMMA_CORR_CNTLA_REGION_0_1__GAMMA_CORR_CNTLA_EXP_REGION1_LUT_OFFSET_MASK 0x7f8000
+#define GAMMA_CORR_CNTLA_REGION_0_1__GAMMA_CORR_CNTLA_EXP_REGION1_LUT_OFFSET__SHIFT 0xf
+#define GAMMA_CORR_CNTLA_REGION_0_1__GAMMA_CORR_CNTLA_EXP_REGION1_NUM_SEGMENTS_MASK 0x38000000
+#define GAMMA_CORR_CNTLA_REGION_0_1__GAMMA_CORR_CNTLA_EXP_REGION1_NUM_SEGMENTS__SHIFT 0x1b
+#define GAMMA_CORR_CNTLA_REGION_2_3__GAMMA_CORR_CNTLA_EXP_REGION2_LUT_OFFSET_MASK 0xff
+#define GAMMA_CORR_CNTLA_REGION_2_3__GAMMA_CORR_CNTLA_EXP_REGION2_LUT_OFFSET__SHIFT 0x0
+#define GAMMA_CORR_CNTLA_REGION_2_3__GAMMA_CORR_CNTLA_EXP_REGION2_NUM_SEGMENTS_MASK 0x3800
+#define GAMMA_CORR_CNTLA_REGION_2_3__GAMMA_CORR_CNTLA_EXP_REGION2_NUM_SEGMENTS__SHIFT 0xb
+#define GAMMA_CORR_CNTLA_REGION_2_3__GAMMA_CORR_CNTLA_EXP_REGION3_LUT_OFFSET_MASK 0x7f8000
+#define GAMMA_CORR_CNTLA_REGION_2_3__GAMMA_CORR_CNTLA_EXP_REGION3_LUT_OFFSET__SHIFT 0xf
+#define GAMMA_CORR_CNTLA_REGION_2_3__GAMMA_CORR_CNTLA_EXP_REGION3_NUM_SEGMENTS_MASK 0x38000000
+#define GAMMA_CORR_CNTLA_REGION_2_3__GAMMA_CORR_CNTLA_EXP_REGION3_NUM_SEGMENTS__SHIFT 0x1b
+#define GAMMA_CORR_CNTLA_REGION_4_5__GAMMA_CORR_CNTLA_EXP_REGION4_LUT_OFFSET_MASK 0xff
+#define GAMMA_CORR_CNTLA_REGION_4_5__GAMMA_CORR_CNTLA_EXP_REGION4_LUT_OFFSET__SHIFT 0x0
+#define GAMMA_CORR_CNTLA_REGION_4_5__GAMMA_CORR_CNTLA_EXP_REGION4_NUM_SEGMENTS_MASK 0x3800
+#define GAMMA_CORR_CNTLA_REGION_4_5__GAMMA_CORR_CNTLA_EXP_REGION4_NUM_SEGMENTS__SHIFT 0xb
+#define GAMMA_CORR_CNTLA_REGION_4_5__GAMMA_CORR_CNTLA_EXP_REGION5_LUT_OFFSET_MASK 0x7f8000
+#define GAMMA_CORR_CNTLA_REGION_4_5__GAMMA_CORR_CNTLA_EXP_REGION5_LUT_OFFSET__SHIFT 0xf
+#define GAMMA_CORR_CNTLA_REGION_4_5__GAMMA_CORR_CNTLA_EXP_REGION5_NUM_SEGMENTS_MASK 0x38000000
+#define GAMMA_CORR_CNTLA_REGION_4_5__GAMMA_CORR_CNTLA_EXP_REGION5_NUM_SEGMENTS__SHIFT 0x1b
+#define GAMMA_CORR_CNTLA_REGION_6_7__GAMMA_CORR_CNTLA_EXP_REGION6_LUT_OFFSET_MASK 0xff
+#define GAMMA_CORR_CNTLA_REGION_6_7__GAMMA_CORR_CNTLA_EXP_REGION6_LUT_OFFSET__SHIFT 0x0
+#define GAMMA_CORR_CNTLA_REGION_6_7__GAMMA_CORR_CNTLA_EXP_REGION6_NUM_SEGMENTS_MASK 0x3800
+#define GAMMA_CORR_CNTLA_REGION_6_7__GAMMA_CORR_CNTLA_EXP_REGION6_NUM_SEGMENTS__SHIFT 0xb
+#define GAMMA_CORR_CNTLA_REGION_6_7__GAMMA_CORR_CNTLA_EXP_REGION7_LUT_OFFSET_MASK 0x7f8000
+#define GAMMA_CORR_CNTLA_REGION_6_7__GAMMA_CORR_CNTLA_EXP_REGION7_LUT_OFFSET__SHIFT 0xf
+#define GAMMA_CORR_CNTLA_REGION_6_7__GAMMA_CORR_CNTLA_EXP_REGION7_NUM_SEGMENTS_MASK 0x38000000
+#define GAMMA_CORR_CNTLA_REGION_6_7__GAMMA_CORR_CNTLA_EXP_REGION7_NUM_SEGMENTS__SHIFT 0x1b
+#define GAMMA_CORR_CNTLA_REGION_8_9__GAMMA_CORR_CNTLA_EXP_REGION8_LUT_OFFSET_MASK 0xff
+#define GAMMA_CORR_CNTLA_REGION_8_9__GAMMA_CORR_CNTLA_EXP_REGION8_LUT_OFFSET__SHIFT 0x0
+#define GAMMA_CORR_CNTLA_REGION_8_9__GAMMA_CORR_CNTLA_EXP_REGION8_NUM_SEGMENTS_MASK 0x3800
+#define GAMMA_CORR_CNTLA_REGION_8_9__GAMMA_CORR_CNTLA_EXP_REGION8_NUM_SEGMENTS__SHIFT 0xb
+#define GAMMA_CORR_CNTLA_REGION_8_9__GAMMA_CORR_CNTLA_EXP_REGION9_LUT_OFFSET_MASK 0x7f8000
+#define GAMMA_CORR_CNTLA_REGION_8_9__GAMMA_CORR_CNTLA_EXP_REGION9_LUT_OFFSET__SHIFT 0xf
+#define GAMMA_CORR_CNTLA_REGION_8_9__GAMMA_CORR_CNTLA_EXP_REGION9_NUM_SEGMENTS_MASK 0x38000000
+#define GAMMA_CORR_CNTLA_REGION_8_9__GAMMA_CORR_CNTLA_EXP_REGION9_NUM_SEGMENTS__SHIFT 0x1b
+#define GAMMA_CORR_CNTLA_REGION_10_11__GAMMA_CORR_CNTLA_EXP_REGION10_LUT_OFFSET_MASK 0xff
+#define GAMMA_CORR_CNTLA_REGION_10_11__GAMMA_CORR_CNTLA_EXP_REGION10_LUT_OFFSET__SHIFT 0x0
+#define GAMMA_CORR_CNTLA_REGION_10_11__GAMMA_CORR_CNTLA_EXP_REGION10_NUM_SEGMENTS_MASK 0x3800
+#define GAMMA_CORR_CNTLA_REGION_10_11__GAMMA_CORR_CNTLA_EXP_REGION10_NUM_SEGMENTS__SHIFT 0xb
+#define GAMMA_CORR_CNTLA_REGION_10_11__GAMMA_CORR_CNTLA_EXP_REGION11_LUT_OFFSET_MASK 0x7f8000
+#define GAMMA_CORR_CNTLA_REGION_10_11__GAMMA_CORR_CNTLA_EXP_REGION11_LUT_OFFSET__SHIFT 0xf
+#define GAMMA_CORR_CNTLA_REGION_10_11__GAMMA_CORR_CNTLA_EXP_REGION11_NUM_SEGMENTS_MASK 0x38000000
+#define GAMMA_CORR_CNTLA_REGION_10_11__GAMMA_CORR_CNTLA_EXP_REGION11_NUM_SEGMENTS__SHIFT 0x1b
+#define GAMMA_CORR_CNTLA_REGION_12_13__GAMMA_CORR_CNTLA_EXP_REGION12_LUT_OFFSET_MASK 0xff
+#define GAMMA_CORR_CNTLA_REGION_12_13__GAMMA_CORR_CNTLA_EXP_REGION12_LUT_OFFSET__SHIFT 0x0
+#define GAMMA_CORR_CNTLA_REGION_12_13__GAMMA_CORR_CNTLA_EXP_REGION12_NUM_SEGMENTS_MASK 0x3800
+#define GAMMA_CORR_CNTLA_REGION_12_13__GAMMA_CORR_CNTLA_EXP_REGION12_NUM_SEGMENTS__SHIFT 0xb
+#define GAMMA_CORR_CNTLA_REGION_12_13__GAMMA_CORR_CNTLA_EXP_REGION13_LUT_OFFSET_MASK 0x7f8000
+#define GAMMA_CORR_CNTLA_REGION_12_13__GAMMA_CORR_CNTLA_EXP_REGION13_LUT_OFFSET__SHIFT 0xf
+#define GAMMA_CORR_CNTLA_REGION_12_13__GAMMA_CORR_CNTLA_EXP_REGION13_NUM_SEGMENTS_MASK 0x38000000
+#define GAMMA_CORR_CNTLA_REGION_12_13__GAMMA_CORR_CNTLA_EXP_REGION13_NUM_SEGMENTS__SHIFT 0x1b
+#define GAMMA_CORR_CNTLA_REGION_14_15__GAMMA_CORR_CNTLA_EXP_REGION14_LUT_OFFSET_MASK 0xff
+#define GAMMA_CORR_CNTLA_REGION_14_15__GAMMA_CORR_CNTLA_EXP_REGION14_LUT_OFFSET__SHIFT 0x0
+#define GAMMA_CORR_CNTLA_REGION_14_15__GAMMA_CORR_CNTLA_EXP_REGION14_NUM_SEGMENTS_MASK 0x3800
+#define GAMMA_CORR_CNTLA_REGION_14_15__GAMMA_CORR_CNTLA_EXP_REGION14_NUM_SEGMENTS__SHIFT 0xb
+#define GAMMA_CORR_CNTLA_REGION_14_15__GAMMA_CORR_CNTLA_EXP_REGION15_LUT_OFFSET_MASK 0x7f8000
+#define GAMMA_CORR_CNTLA_REGION_14_15__GAMMA_CORR_CNTLA_EXP_REGION15_LUT_OFFSET__SHIFT 0xf
+#define GAMMA_CORR_CNTLA_REGION_14_15__GAMMA_CORR_CNTLA_EXP_REGION15_NUM_SEGMENTS_MASK 0x38000000
+#define GAMMA_CORR_CNTLA_REGION_14_15__GAMMA_CORR_CNTLA_EXP_REGION15_NUM_SEGMENTS__SHIFT 0x1b
+#define GAMMA_CORR_CNTLB_START_CNTL__GAMMA_CORR_CNTLB_EXP_REGION_START_MASK 0x3ffff
+#define GAMMA_CORR_CNTLB_START_CNTL__GAMMA_CORR_CNTLB_EXP_REGION_START__SHIFT 0x0
+#define GAMMA_CORR_CNTLB_START_CNTL__GAMMA_CORR_CNTLB_EXP_REGION_START_SEGMENT_MASK 0x7f00000
+#define GAMMA_CORR_CNTLB_START_CNTL__GAMMA_CORR_CNTLB_EXP_REGION_START_SEGMENT__SHIFT 0x14
+#define GAMMA_CORR_CNTLB_SLOPE_CNTL__GAMMA_CORR_CNTLB_EXP_REGION_LINEAR_SLOPE_MASK 0x3ffff
+#define GAMMA_CORR_CNTLB_SLOPE_CNTL__GAMMA_CORR_CNTLB_EXP_REGION_LINEAR_SLOPE__SHIFT 0x0
+#define GAMMA_CORR_CNTLB_END_CNTL1__GAMMA_CORR_CNTLB_EXP_REGION_END_MASK 0xffff
+#define GAMMA_CORR_CNTLB_END_CNTL1__GAMMA_CORR_CNTLB_EXP_REGION_END__SHIFT 0x0
+#define GAMMA_CORR_CNTLB_END_CNTL2__GAMMA_CORR_CNTLB_EXP_REGION_END_SLOPE_MASK 0xffff
+#define GAMMA_CORR_CNTLB_END_CNTL2__GAMMA_CORR_CNTLB_EXP_REGION_END_SLOPE__SHIFT 0x0
+#define GAMMA_CORR_CNTLB_END_CNTL2__GAMMA_CORR_CNTLB_EXP_REGION_END_BASE_MASK 0xffff0000
+#define GAMMA_CORR_CNTLB_END_CNTL2__GAMMA_CORR_CNTLB_EXP_REGION_END_BASE__SHIFT 0x10
+#define GAMMA_CORR_CNTLB_REGION_0_1__GAMMA_CORR_CNTLB_EXP_REGION0_LUT_OFFSET_MASK 0xff
+#define GAMMA_CORR_CNTLB_REGION_0_1__GAMMA_CORR_CNTLB_EXP_REGION0_LUT_OFFSET__SHIFT 0x0
+#define GAMMA_CORR_CNTLB_REGION_0_1__GAMMA_CORR_CNTLB_EXP_REGION0_NUM_SEGMENTS_MASK 0x3800
+#define GAMMA_CORR_CNTLB_REGION_0_1__GAMMA_CORR_CNTLB_EXP_REGION0_NUM_SEGMENTS__SHIFT 0xb
+#define GAMMA_CORR_CNTLB_REGION_0_1__GAMMA_CORR_CNTLB_EXP_REGION1_LUT_OFFSET_MASK 0x7f8000
+#define GAMMA_CORR_CNTLB_REGION_0_1__GAMMA_CORR_CNTLB_EXP_REGION1_LUT_OFFSET__SHIFT 0xf
+#define GAMMA_CORR_CNTLB_REGION_0_1__GAMMA_CORR_CNTLB_EXP_REGION1_NUM_SEGMENTS_MASK 0x38000000
+#define GAMMA_CORR_CNTLB_REGION_0_1__GAMMA_CORR_CNTLB_EXP_REGION1_NUM_SEGMENTS__SHIFT 0x1b
+#define GAMMA_CORR_CNTLB_REGION_2_3__GAMMA_CORR_CNTLB_EXP_REGION2_LUT_OFFSET_MASK 0xff
+#define GAMMA_CORR_CNTLB_REGION_2_3__GAMMA_CORR_CNTLB_EXP_REGION2_LUT_OFFSET__SHIFT 0x0
+#define GAMMA_CORR_CNTLB_REGION_2_3__GAMMA_CORR_CNTLB_EXP_REGION2_NUM_SEGMENTS_MASK 0x3800
+#define GAMMA_CORR_CNTLB_REGION_2_3__GAMMA_CORR_CNTLB_EXP_REGION2_NUM_SEGMENTS__SHIFT 0xb
+#define GAMMA_CORR_CNTLB_REGION_2_3__GAMMA_CORR_CNTLB_EXP_REGION3_LUT_OFFSET_MASK 0x7f8000
+#define GAMMA_CORR_CNTLB_REGION_2_3__GAMMA_CORR_CNTLB_EXP_REGION3_LUT_OFFSET__SHIFT 0xf
+#define GAMMA_CORR_CNTLB_REGION_2_3__GAMMA_CORR_CNTLB_EXP_REGION3_NUM_SEGMENTS_MASK 0x38000000
+#define GAMMA_CORR_CNTLB_REGION_2_3__GAMMA_CORR_CNTLB_EXP_REGION3_NUM_SEGMENTS__SHIFT 0x1b
+#define GAMMA_CORR_CNTLB_REGION_4_5__GAMMA_CORR_CNTLB_EXP_REGION4_LUT_OFFSET_MASK 0xff
+#define GAMMA_CORR_CNTLB_REGION_4_5__GAMMA_CORR_CNTLB_EXP_REGION4_LUT_OFFSET__SHIFT 0x0
+#define GAMMA_CORR_CNTLB_REGION_4_5__GAMMA_CORR_CNTLB_EXP_REGION4_NUM_SEGMENTS_MASK 0x3800
+#define GAMMA_CORR_CNTLB_REGION_4_5__GAMMA_CORR_CNTLB_EXP_REGION4_NUM_SEGMENTS__SHIFT 0xb
+#define GAMMA_CORR_CNTLB_REGION_4_5__GAMMA_CORR_CNTLB_EXP_REGION5_LUT_OFFSET_MASK 0x7f8000
+#define GAMMA_CORR_CNTLB_REGION_4_5__GAMMA_CORR_CNTLB_EXP_REGION5_LUT_OFFSET__SHIFT 0xf
+#define GAMMA_CORR_CNTLB_REGION_4_5__GAMMA_CORR_CNTLB_EXP_REGION5_NUM_SEGMENTS_MASK 0x38000000
+#define GAMMA_CORR_CNTLB_REGION_4_5__GAMMA_CORR_CNTLB_EXP_REGION5_NUM_SEGMENTS__SHIFT 0x1b
+#define GAMMA_CORR_CNTLB_REGION_6_7__GAMMA_CORR_CNTLB_EXP_REGION6_LUT_OFFSET_MASK 0xff
+#define GAMMA_CORR_CNTLB_REGION_6_7__GAMMA_CORR_CNTLB_EXP_REGION6_LUT_OFFSET__SHIFT 0x0
+#define GAMMA_CORR_CNTLB_REGION_6_7__GAMMA_CORR_CNTLB_EXP_REGION6_NUM_SEGMENTS_MASK 0x3800
+#define GAMMA_CORR_CNTLB_REGION_6_7__GAMMA_CORR_CNTLB_EXP_REGION6_NUM_SEGMENTS__SHIFT 0xb
+#define GAMMA_CORR_CNTLB_REGION_6_7__GAMMA_CORR_CNTLB_EXP_REGION7_LUT_OFFSET_MASK 0x7f8000
+#define GAMMA_CORR_CNTLB_REGION_6_7__GAMMA_CORR_CNTLB_EXP_REGION7_LUT_OFFSET__SHIFT 0xf
+#define GAMMA_CORR_CNTLB_REGION_6_7__GAMMA_CORR_CNTLB_EXP_REGION7_NUM_SEGMENTS_MASK 0x38000000
+#define GAMMA_CORR_CNTLB_REGION_6_7__GAMMA_CORR_CNTLB_EXP_REGION7_NUM_SEGMENTS__SHIFT 0x1b
+#define GAMMA_CORR_CNTLB_REGION_8_9__GAMMA_CORR_CNTLB_EXP_REGION8_LUT_OFFSET_MASK 0xff
+#define GAMMA_CORR_CNTLB_REGION_8_9__GAMMA_CORR_CNTLB_EXP_REGION8_LUT_OFFSET__SHIFT 0x0
+#define GAMMA_CORR_CNTLB_REGION_8_9__GAMMA_CORR_CNTLB_EXP_REGION8_NUM_SEGMENTS_MASK 0x3800
+#define GAMMA_CORR_CNTLB_REGION_8_9__GAMMA_CORR_CNTLB_EXP_REGION8_NUM_SEGMENTS__SHIFT 0xb
+#define GAMMA_CORR_CNTLB_REGION_8_9__GAMMA_CORR_CNTLB_EXP_REGION9_LUT_OFFSET_MASK 0x7f8000
+#define GAMMA_CORR_CNTLB_REGION_8_9__GAMMA_CORR_CNTLB_EXP_REGION9_LUT_OFFSET__SHIFT 0xf
+#define GAMMA_CORR_CNTLB_REGION_8_9__GAMMA_CORR_CNTLB_EXP_REGION9_NUM_SEGMENTS_MASK 0x38000000
+#define GAMMA_CORR_CNTLB_REGION_8_9__GAMMA_CORR_CNTLB_EXP_REGION9_NUM_SEGMENTS__SHIFT 0x1b
+#define GAMMA_CORR_CNTLB_REGION_10_11__GAMMA_CORR_CNTLB_EXP_REGION10_LUT_OFFSET_MASK 0xff
+#define GAMMA_CORR_CNTLB_REGION_10_11__GAMMA_CORR_CNTLB_EXP_REGION10_LUT_OFFSET__SHIFT 0x0
+#define GAMMA_CORR_CNTLB_REGION_10_11__GAMMA_CORR_CNTLB_EXP_REGION10_NUM_SEGMENTS_MASK 0x3800
+#define GAMMA_CORR_CNTLB_REGION_10_11__GAMMA_CORR_CNTLB_EXP_REGION10_NUM_SEGMENTS__SHIFT 0xb
+#define GAMMA_CORR_CNTLB_REGION_10_11__GAMMA_CORR_CNTLB_EXP_REGION11_LUT_OFFSET_MASK 0x7f8000
+#define GAMMA_CORR_CNTLB_REGION_10_11__GAMMA_CORR_CNTLB_EXP_REGION11_LUT_OFFSET__SHIFT 0xf
+#define GAMMA_CORR_CNTLB_REGION_10_11__GAMMA_CORR_CNTLB_EXP_REGION11_NUM_SEGMENTS_MASK 0x38000000
+#define GAMMA_CORR_CNTLB_REGION_10_11__GAMMA_CORR_CNTLB_EXP_REGION11_NUM_SEGMENTS__SHIFT 0x1b
+#define GAMMA_CORR_CNTLB_REGION_12_13__GAMMA_CORR_CNTLB_EXP_REGION12_LUT_OFFSET_MASK 0xff
+#define GAMMA_CORR_CNTLB_REGION_12_13__GAMMA_CORR_CNTLB_EXP_REGION12_LUT_OFFSET__SHIFT 0x0
+#define GAMMA_CORR_CNTLB_REGION_12_13__GAMMA_CORR_CNTLB_EXP_REGION12_NUM_SEGMENTS_MASK 0x3800
+#define GAMMA_CORR_CNTLB_REGION_12_13__GAMMA_CORR_CNTLB_EXP_REGION12_NUM_SEGMENTS__SHIFT 0xb
+#define GAMMA_CORR_CNTLB_REGION_12_13__GAMMA_CORR_CNTLB_EXP_REGION13_LUT_OFFSET_MASK 0x7f8000
+#define GAMMA_CORR_CNTLB_REGION_12_13__GAMMA_CORR_CNTLB_EXP_REGION13_LUT_OFFSET__SHIFT 0xf
+#define GAMMA_CORR_CNTLB_REGION_12_13__GAMMA_CORR_CNTLB_EXP_REGION13_NUM_SEGMENTS_MASK 0x38000000
+#define GAMMA_CORR_CNTLB_REGION_12_13__GAMMA_CORR_CNTLB_EXP_REGION13_NUM_SEGMENTS__SHIFT 0x1b
+#define GAMMA_CORR_CNTLB_REGION_14_15__GAMMA_CORR_CNTLB_EXP_REGION14_LUT_OFFSET_MASK 0xff
+#define GAMMA_CORR_CNTLB_REGION_14_15__GAMMA_CORR_CNTLB_EXP_REGION14_LUT_OFFSET__SHIFT 0x0
+#define GAMMA_CORR_CNTLB_REGION_14_15__GAMMA_CORR_CNTLB_EXP_REGION14_NUM_SEGMENTS_MASK 0x3800
+#define GAMMA_CORR_CNTLB_REGION_14_15__GAMMA_CORR_CNTLB_EXP_REGION14_NUM_SEGMENTS__SHIFT 0xb
+#define GAMMA_CORR_CNTLB_REGION_14_15__GAMMA_CORR_CNTLB_EXP_REGION15_LUT_OFFSET_MASK 0x7f8000
+#define GAMMA_CORR_CNTLB_REGION_14_15__GAMMA_CORR_CNTLB_EXP_REGION15_LUT_OFFSET__SHIFT 0xf
+#define GAMMA_CORR_CNTLB_REGION_14_15__GAMMA_CORR_CNTLB_EXP_REGION15_NUM_SEGMENTS_MASK 0x38000000
+#define GAMMA_CORR_CNTLB_REGION_14_15__GAMMA_CORR_CNTLB_EXP_REGION15_NUM_SEGMENTS__SHIFT 0x1b
+#define PACK_FIFO_ERROR__PACK_FIFO_L_UNDERFLOW_OCCURED_MASK 0x1
+#define PACK_FIFO_ERROR__PACK_FIFO_L_UNDERFLOW_OCCURED__SHIFT 0x0
+#define PACK_FIFO_ERROR__PACK_FIFO_L_UNDERFLOW_ACK_MASK 0x2
+#define PACK_FIFO_ERROR__PACK_FIFO_L_UNDERFLOW_ACK__SHIFT 0x1
+#define PACK_FIFO_ERROR__PACK_FIFO_C_UNDERFLOW_OCCURED_MASK 0x100
+#define PACK_FIFO_ERROR__PACK_FIFO_C_UNDERFLOW_OCCURED__SHIFT 0x8
+#define PACK_FIFO_ERROR__PACK_FIFO_C_UNDERFLOW_ACK_MASK 0x200
+#define PACK_FIFO_ERROR__PACK_FIFO_C_UNDERFLOW_ACK__SHIFT 0x9
+#define PACK_FIFO_ERROR__PACK_FIFO_L_OVERFLOW_OCCURED_MASK 0x10000
+#define PACK_FIFO_ERROR__PACK_FIFO_L_OVERFLOW_OCCURED__SHIFT 0x10
+#define PACK_FIFO_ERROR__PACK_FIFO_L_OVERFLOW_ACK_MASK 0x20000
+#define PACK_FIFO_ERROR__PACK_FIFO_L_OVERFLOW_ACK__SHIFT 0x11
+#define PACK_FIFO_ERROR__PACK_FIFO_C_OVERFLOW_OCCURED_MASK 0x1000000
+#define PACK_FIFO_ERROR__PACK_FIFO_C_OVERFLOW_OCCURED__SHIFT 0x18
+#define PACK_FIFO_ERROR__PACK_FIFO_C_OVERFLOW_ACK_MASK 0x2000000
+#define PACK_FIFO_ERROR__PACK_FIFO_C_OVERFLOW_ACK__SHIFT 0x19
+#define OUTPUT_FIFO_ERROR__OUTPUT_FIFO_UNDERFLOW_OCCURED_MASK 0x1
+#define OUTPUT_FIFO_ERROR__OUTPUT_FIFO_UNDERFLOW_OCCURED__SHIFT 0x0
+#define OUTPUT_FIFO_ERROR__OUTPUT_FIFO_UNDERFLOW_ACK_MASK 0x2
+#define OUTPUT_FIFO_ERROR__OUTPUT_FIFO_UNDERFLOW_ACK__SHIFT 0x1
+#define OUTPUT_FIFO_ERROR__OUTPUT_FIFO_OVERFLOW_OCCURED_MASK 0x100
+#define OUTPUT_FIFO_ERROR__OUTPUT_FIFO_OVERFLOW_OCCURED__SHIFT 0x8
+#define OUTPUT_FIFO_ERROR__OUTPUT_FIFO_OVERFLOW_ACK_MASK 0x200
+#define OUTPUT_FIFO_ERROR__OUTPUT_FIFO_OVERFLOW_ACK__SHIFT 0x9
+#define INPUT_GAMMA_LUT_AUTOFILL__INPUT_GAMMA_LUT_AUTOFILL_MASK 0x1
+#define INPUT_GAMMA_LUT_AUTOFILL__INPUT_GAMMA_LUT_AUTOFILL__SHIFT 0x0
+#define INPUT_GAMMA_LUT_AUTOFILL__INPUT_GAMMA_LUT_AUTOFILL_DONE_MASK 0x2
+#define INPUT_GAMMA_LUT_AUTOFILL__INPUT_GAMMA_LUT_AUTOFILL_DONE__SHIFT 0x1
+#define INPUT_GAMMA_LUT_RW_INDEX__INPUT_GAMMA_LUT_RW_INDEX_MASK 0xff
+#define INPUT_GAMMA_LUT_RW_INDEX__INPUT_GAMMA_LUT_RW_INDEX__SHIFT 0x0
+#define INPUT_GAMMA_LUT_SEQ_COLOR__INPUT_GAMMA_LUT_SEQ_COLOR_MASK 0xffff
+#define INPUT_GAMMA_LUT_SEQ_COLOR__INPUT_GAMMA_LUT_SEQ_COLOR__SHIFT 0x0
+#define INPUT_GAMMA_LUT_PWL_DATA__INPUT_GAMMA_LUT_BASE_MASK 0xffff
+#define INPUT_GAMMA_LUT_PWL_DATA__INPUT_GAMMA_LUT_BASE__SHIFT 0x0
+#define INPUT_GAMMA_LUT_PWL_DATA__INPUT_GAMMA_LUT_DELTA_MASK 0xffff0000
+#define INPUT_GAMMA_LUT_PWL_DATA__INPUT_GAMMA_LUT_DELTA__SHIFT 0x10
+#define INPUT_GAMMA_LUT_30_COLOR__INPUT_GAMMA_LUT_COLOR_10_BLUE_MASK 0x3ff
+#define INPUT_GAMMA_LUT_30_COLOR__INPUT_GAMMA_LUT_COLOR_10_BLUE__SHIFT 0x0
+#define INPUT_GAMMA_LUT_30_COLOR__INPUT_GAMMA_LUT_COLOR_10_GREEN_MASK 0xffc00
+#define INPUT_GAMMA_LUT_30_COLOR__INPUT_GAMMA_LUT_COLOR_10_GREEN__SHIFT 0xa
+#define INPUT_GAMMA_LUT_30_COLOR__INPUT_GAMMA_LUT_COLOR_10_RED_MASK 0x3ff00000
+#define INPUT_GAMMA_LUT_30_COLOR__INPUT_GAMMA_LUT_COLOR_10_RED__SHIFT 0x14
+#define COL_MAN_INPUT_GAMMA_CONTROL1__INPUT_GAMMA_MODE_MASK 0x3
+#define COL_MAN_INPUT_GAMMA_CONTROL1__INPUT_GAMMA_MODE__SHIFT 0x0
+#define COL_MAN_INPUT_GAMMA_CONTROL1__INPUT_GAMMA_LUT_10BIT_BYPASS_EN_MASK 0x4000000
+#define COL_MAN_INPUT_GAMMA_CONTROL1__INPUT_GAMMA_LUT_10BIT_BYPASS_EN__SHIFT 0x1a
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_INC_B_MASK 0x1e
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_INC_B__SHIFT 0x1
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_DATA_B_SIGNED_EN_MASK 0x20
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_DATA_B_SIGNED_EN__SHIFT 0x5
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_DATA_B_FORMAT_MASK 0xc0
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_DATA_B_FORMAT__SHIFT 0x6
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_INC_G_MASK 0xf00
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_INC_G__SHIFT 0x8
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_DATA_G_SIGNED_EN_MASK 0x1000
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_DATA_G_SIGNED_EN__SHIFT 0xc
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_DATA_G_FORMAT_MASK 0x6000
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_DATA_G_FORMAT__SHIFT 0xd
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_INC_R_MASK 0x78000
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_INC_R__SHIFT 0xf
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_DATA_R_SIGNED_EN_MASK 0x80000
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_DATA_R_SIGNED_EN__SHIFT 0x13
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_DATA_R_FORMAT_MASK 0x300000
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_DATA_R_FORMAT__SHIFT 0x14
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_LUT_RW_MODE_MASK 0x400000
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_LUT_RW_MODE__SHIFT 0x16
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_LUT_WRITE_EN_MASK_MASK 0x3800000
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_LUT_WRITE_EN_MASK__SHIFT 0x17
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_LUT_VGA_ACCESS_ENABLE_MASK 0x4000000
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_LUT_VGA_ACCESS_ENABLE__SHIFT 0x1a
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_LUT_10BIT_BYPASS_DBL_BUF_EN_MASK 0x8000000
+#define COL_MAN_INPUT_GAMMA_CONTROL2__INPUT_GAMMA_LUT_10BIT_BYPASS_DBL_BUF_EN__SHIFT 0x1b
+#define INPUT_GAMMA_BW_OFFSETS_B__INPUT_GAMMA_BLACK_OFFSET_B_MASK 0xffff
+#define INPUT_GAMMA_BW_OFFSETS_B__INPUT_GAMMA_BLACK_OFFSET_B__SHIFT 0x0
+#define INPUT_GAMMA_BW_OFFSETS_B__INPUT_GAMMA_WHITE_OFFSET_B_MASK 0xffff0000
+#define INPUT_GAMMA_BW_OFFSETS_B__INPUT_GAMMA_WHITE_OFFSET_B__SHIFT 0x10
+#define INPUT_GAMMA_BW_OFFSETS_G__INPUT_GAMMA_BLACK_OFFSET_G_MASK 0xffff
+#define INPUT_GAMMA_BW_OFFSETS_G__INPUT_GAMMA_BLACK_OFFSET_G__SHIFT 0x0
+#define INPUT_GAMMA_BW_OFFSETS_G__INPUT_GAMMA_WHITE_OFFSET_G_MASK 0xffff0000
+#define INPUT_GAMMA_BW_OFFSETS_G__INPUT_GAMMA_WHITE_OFFSET_G__SHIFT 0x10
+#define INPUT_GAMMA_BW_OFFSETS_R__INPUT_GAMMA_BLACK_OFFSET_R_MASK 0xffff
+#define INPUT_GAMMA_BW_OFFSETS_R__INPUT_GAMMA_BLACK_OFFSET_R__SHIFT 0x0
+#define INPUT_GAMMA_BW_OFFSETS_R__INPUT_GAMMA_WHITE_OFFSET_R_MASK 0xffff0000
+#define INPUT_GAMMA_BW_OFFSETS_R__INPUT_GAMMA_WHITE_OFFSET_R__SHIFT 0x10
+#define COL_MAN_DEBUG_CONTROL__COL_MAN_GLOBAL_PASSTHROUGH_ENABLE_MASK 0x1
+#define COL_MAN_DEBUG_CONTROL__COL_MAN_GLOBAL_PASSTHROUGH_ENABLE__SHIFT 0x0
+#define COL_MAN_TEST_DEBUG_INDEX__COL_MAN_TEST_DEBUG_INDEX_MASK 0xff
+#define COL_MAN_TEST_DEBUG_INDEX__COL_MAN_TEST_DEBUG_INDEX__SHIFT 0x0
+#define COL_MAN_TEST_DEBUG_INDEX__COL_MAN_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define COL_MAN_TEST_DEBUG_INDEX__COL_MAN_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define COL_MAN_TEST_DEBUG_DATA__COL_MAN_TEST_DEBUG_DATA_MASK 0xffffffff
+#define COL_MAN_TEST_DEBUG_DATA__COL_MAN_TEST_DEBUG_DATA__SHIFT 0x0
+#define UNP_GRPH_ENABLE__GRPH_ENABLE_MASK 0x1
+#define UNP_GRPH_ENABLE__GRPH_ENABLE__SHIFT 0x0
+#define UNP_GRPH_CONTROL__GRPH_DEPTH_MASK 0x3
+#define UNP_GRPH_CONTROL__GRPH_DEPTH__SHIFT 0x0
+#define UNP_GRPH_CONTROL__GRPH_NUM_BANKS_MASK 0xc
+#define UNP_GRPH_CONTROL__GRPH_NUM_BANKS__SHIFT 0x2
+#define UNP_GRPH_CONTROL__GRPH_Z_MASK 0x30
+#define UNP_GRPH_CONTROL__GRPH_Z__SHIFT 0x4
+#define UNP_GRPH_CONTROL__GRPH_BANK_WIDTH_L_MASK 0xc0
+#define UNP_GRPH_CONTROL__GRPH_BANK_WIDTH_L__SHIFT 0x6
+#define UNP_GRPH_CONTROL__GRPH_FORMAT_MASK 0x700
+#define UNP_GRPH_CONTROL__GRPH_FORMAT__SHIFT 0x8
+#define UNP_GRPH_CONTROL__GRPH_BANK_HEIGHT_L_MASK 0x1800
+#define UNP_GRPH_CONTROL__GRPH_BANK_HEIGHT_L__SHIFT 0xb
+#define UNP_GRPH_CONTROL__GRPH_TILE_SPLIT_L_MASK 0xe000
+#define UNP_GRPH_CONTROL__GRPH_TILE_SPLIT_L__SHIFT 0xd
+#define UNP_GRPH_CONTROL__GRPH_ADDRESS_TRANSLATION_ENABLE_MASK 0x10000
+#define UNP_GRPH_CONTROL__GRPH_ADDRESS_TRANSLATION_ENABLE__SHIFT 0x10
+#define UNP_GRPH_CONTROL__GRPH_PRIVILEGED_ACCESS_ENABLE_MASK 0x20000
+#define UNP_GRPH_CONTROL__GRPH_PRIVILEGED_ACCESS_ENABLE__SHIFT 0x11
+#define UNP_GRPH_CONTROL__GRPH_MACRO_TILE_ASPECT_L_MASK 0xc0000
+#define UNP_GRPH_CONTROL__GRPH_MACRO_TILE_ASPECT_L__SHIFT 0x12
+#define UNP_GRPH_CONTROL__GRPH_ARRAY_MODE_MASK 0xf00000
+#define UNP_GRPH_CONTROL__GRPH_ARRAY_MODE__SHIFT 0x14
+#define UNP_GRPH_CONTROL__GRPH_PIPE_CONFIG_MASK 0x1f000000
+#define UNP_GRPH_CONTROL__GRPH_PIPE_CONFIG__SHIFT 0x18
+#define UNP_GRPH_CONTROL__GRPH_MICRO_TILE_MODE_L_MASK 0x60000000
+#define UNP_GRPH_CONTROL__GRPH_MICRO_TILE_MODE_L__SHIFT 0x1d
+#define UNP_GRPH_CONTROL__GRPH_COLOR_EXPANSION_MODE_MASK 0x80000000
+#define UNP_GRPH_CONTROL__GRPH_COLOR_EXPANSION_MODE__SHIFT 0x1f
+#define UNP_GRPH_CONTROL_C__GRPH_BANK_WIDTH_C_MASK 0xc0
+#define UNP_GRPH_CONTROL_C__GRPH_BANK_WIDTH_C__SHIFT 0x6
+#define UNP_GRPH_CONTROL_C__GRPH_BANK_HEIGHT_C_MASK 0x1800
+#define UNP_GRPH_CONTROL_C__GRPH_BANK_HEIGHT_C__SHIFT 0xb
+#define UNP_GRPH_CONTROL_C__GRPH_TILE_SPLIT_C_MASK 0xe000
+#define UNP_GRPH_CONTROL_C__GRPH_TILE_SPLIT_C__SHIFT 0xd
+#define UNP_GRPH_CONTROL_C__GRPH_MACRO_TILE_ASPECT_C_MASK 0xc0000
+#define UNP_GRPH_CONTROL_C__GRPH_MACRO_TILE_ASPECT_C__SHIFT 0x12
+#define UNP_GRPH_CONTROL_C__GRPH_MICRO_TILE_MODE_C_MASK 0x60000000
+#define UNP_GRPH_CONTROL_C__GRPH_MICRO_TILE_MODE_C__SHIFT 0x1d
+#define UNP_GRPH_CONTROL_EXP__VIDEO_FORMAT_MASK 0x7
+#define UNP_GRPH_CONTROL_EXP__VIDEO_FORMAT__SHIFT 0x0
+#define UNP_GRPH_SWAP_CNTL__GRPH_ENDIAN_SWAP_MASK 0x3
+#define UNP_GRPH_SWAP_CNTL__GRPH_ENDIAN_SWAP__SHIFT 0x0
+#define UNP_GRPH_SWAP_CNTL__GRPH_RED_CROSSBAR_MASK 0x30
+#define UNP_GRPH_SWAP_CNTL__GRPH_RED_CROSSBAR__SHIFT 0x4
+#define UNP_GRPH_SWAP_CNTL__GRPH_GREEN_CROSSBAR_MASK 0xc0
+#define UNP_GRPH_SWAP_CNTL__GRPH_GREEN_CROSSBAR__SHIFT 0x6
+#define UNP_GRPH_SWAP_CNTL__GRPH_BLUE_CROSSBAR_MASK 0x300
+#define UNP_GRPH_SWAP_CNTL__GRPH_BLUE_CROSSBAR__SHIFT 0x8
+#define UNP_GRPH_PRIMARY_SURFACE_ADDRESS_L__GRPH_PRIMARY_SURFACE_ADDRESS_L_MASK 0xffffff00
+#define UNP_GRPH_PRIMARY_SURFACE_ADDRESS_L__GRPH_PRIMARY_SURFACE_ADDRESS_L__SHIFT 0x8
+#define UNP_GRPH_PRIMARY_SURFACE_ADDRESS_C__GRPH_PRIMARY_SURFACE_ADDRESS_C_MASK 0xffffff00
+#define UNP_GRPH_PRIMARY_SURFACE_ADDRESS_C__GRPH_PRIMARY_SURFACE_ADDRESS_C__SHIFT 0x8
+#define UNP_GRPH_PRIMARY_SURFACE_ADDRESS_HIGH_L__GRPH_PRIMARY_SURFACE_ADDRESS_HIGH_L_MASK 0xff
+#define UNP_GRPH_PRIMARY_SURFACE_ADDRESS_HIGH_L__GRPH_PRIMARY_SURFACE_ADDRESS_HIGH_L__SHIFT 0x0
+#define UNP_GRPH_PRIMARY_SURFACE_ADDRESS_HIGH_C__GRPH_PRIMARY_SURFACE_ADDRESS_HIGH_C_MASK 0xff
+#define UNP_GRPH_PRIMARY_SURFACE_ADDRESS_HIGH_C__GRPH_PRIMARY_SURFACE_ADDRESS_HIGH_C__SHIFT 0x0
+#define UNP_GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_L__GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_L_MASK 0xffffff00
+#define UNP_GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_L__GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_L__SHIFT 0x8
+#define UNP_GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_C__GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_C_MASK 0xffffff00
+#define UNP_GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_C__GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_C__SHIFT 0x8
+#define UNP_GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_HIGH_L__GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_HIGH_L_MASK 0xff
+#define UNP_GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_HIGH_L__GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_HIGH_L__SHIFT 0x0
+#define UNP_GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_HIGH_C__GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_HIGH_C_MASK 0xff
+#define UNP_GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_HIGH_C__GRPH_PRIMARY_BOTTOM_SURFACE_ADDRESS_HIGH_C__SHIFT 0x0
+#define UNP_GRPH_SECONDARY_SURFACE_ADDRESS_L__GRPH_SECONDARY_SURFACE_ADDRESS_L_MASK 0xffffff00
+#define UNP_GRPH_SECONDARY_SURFACE_ADDRESS_L__GRPH_SECONDARY_SURFACE_ADDRESS_L__SHIFT 0x8
+#define UNP_GRPH_SECONDARY_SURFACE_ADDRESS_C__GRPH_SECONDARY_SURFACE_ADDRESS_C_MASK 0xffffff00
+#define UNP_GRPH_SECONDARY_SURFACE_ADDRESS_C__GRPH_SECONDARY_SURFACE_ADDRESS_C__SHIFT 0x8
+#define UNP_GRPH_SECONDARY_SURFACE_ADDRESS_HIGH_L__GRPH_SECONDARY_SURFACE_ADDRESS_HIGH_L_MASK 0xff
+#define UNP_GRPH_SECONDARY_SURFACE_ADDRESS_HIGH_L__GRPH_SECONDARY_SURFACE_ADDRESS_HIGH_L__SHIFT 0x0
+#define UNP_GRPH_SECONDARY_SURFACE_ADDRESS_HIGH_C__GRPH_SECONDARY_SURFACE_ADDRESS_HIGH_C_MASK 0xff
+#define UNP_GRPH_SECONDARY_SURFACE_ADDRESS_HIGH_C__GRPH_SECONDARY_SURFACE_ADDRESS_HIGH_C__SHIFT 0x0
+#define UNP_GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_L__GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_L_MASK 0xffffff00
+#define UNP_GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_L__GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_L__SHIFT 0x8
+#define UNP_GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_C__GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_C_MASK 0xffffff00
+#define UNP_GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_C__GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_C__SHIFT 0x8
+#define UNP_GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_HIGH_L__GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_HIGH_L_MASK 0xff
+#define UNP_GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_HIGH_L__GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_HIGH_L__SHIFT 0x0
+#define UNP_GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_HIGH_C__GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_HIGH_C_MASK 0xff
+#define UNP_GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_HIGH_C__GRPH_SECONDARY_BOTTOM_SURFACE_ADDRESS_HIGH_C__SHIFT 0x0
+#define UNP_GRPH_PITCH_L__GRPH_PITCH_L_MASK 0x7fff
+#define UNP_GRPH_PITCH_L__GRPH_PITCH_L__SHIFT 0x0
+#define UNP_GRPH_PITCH_C__GRPH_PITCH_C_MASK 0x7fff
+#define UNP_GRPH_PITCH_C__GRPH_PITCH_C__SHIFT 0x0
+#define UNP_GRPH_SURFACE_OFFSET_X_L__GRPH_SURFACE_OFFSET_X_L_MASK 0x3fff
+#define UNP_GRPH_SURFACE_OFFSET_X_L__GRPH_SURFACE_OFFSET_X_L__SHIFT 0x0
+#define UNP_GRPH_SURFACE_OFFSET_X_C__GRPH_SURFACE_OFFSET_X_C_MASK 0x3fff
+#define UNP_GRPH_SURFACE_OFFSET_X_C__GRPH_SURFACE_OFFSET_X_C__SHIFT 0x0
+#define UNP_GRPH_SURFACE_OFFSET_Y_L__GRPH_SURFACE_OFFSET_Y_L_MASK 0x3fff
+#define UNP_GRPH_SURFACE_OFFSET_Y_L__GRPH_SURFACE_OFFSET_Y_L__SHIFT 0x0
+#define UNP_GRPH_SURFACE_OFFSET_Y_C__GRPH_SURFACE_OFFSET_Y_C_MASK 0x3fff
+#define UNP_GRPH_SURFACE_OFFSET_Y_C__GRPH_SURFACE_OFFSET_Y_C__SHIFT 0x0
+#define UNP_GRPH_X_START_L__GRPH_X_START_L_MASK 0x3fff
+#define UNP_GRPH_X_START_L__GRPH_X_START_L__SHIFT 0x0
+#define UNP_GRPH_X_START_C__GRPH_X_START_C_MASK 0x3fff
+#define UNP_GRPH_X_START_C__GRPH_X_START_C__SHIFT 0x0
+#define UNP_GRPH_Y_START_L__GRPH_Y_START_L_MASK 0x3fff
+#define UNP_GRPH_Y_START_L__GRPH_Y_START_L__SHIFT 0x0
+#define UNP_GRPH_Y_START_C__GRPH_Y_START_C_MASK 0x3fff
+#define UNP_GRPH_Y_START_C__GRPH_Y_START_C__SHIFT 0x0
+#define UNP_GRPH_X_END_L__GRPH_X_END_L_MASK 0x7fff
+#define UNP_GRPH_X_END_L__GRPH_X_END_L__SHIFT 0x0
+#define UNP_GRPH_X_END_C__GRPH_X_END_C_MASK 0x7fff
+#define UNP_GRPH_X_END_C__GRPH_X_END_C__SHIFT 0x0
+#define UNP_GRPH_Y_END_L__GRPH_Y_END_L_MASK 0x7fff
+#define UNP_GRPH_Y_END_L__GRPH_Y_END_L__SHIFT 0x0
+#define UNP_GRPH_Y_END_C__GRPH_Y_END_C_MASK 0x7fff
+#define UNP_GRPH_Y_END_C__GRPH_Y_END_C__SHIFT 0x0
+#define UNP_GRPH_UPDATE__GRPH_MODE_UPDATE_PENDING_MASK 0x1
+#define UNP_GRPH_UPDATE__GRPH_MODE_UPDATE_PENDING__SHIFT 0x0
+#define UNP_GRPH_UPDATE__GRPH_MODE_UPDATE_TAKEN_MASK 0x2
+#define UNP_GRPH_UPDATE__GRPH_MODE_UPDATE_TAKEN__SHIFT 0x1
+#define UNP_GRPH_UPDATE__GRPH_SURFACE_UPDATE_PENDING_MASK 0x4
+#define UNP_GRPH_UPDATE__GRPH_SURFACE_UPDATE_PENDING__SHIFT 0x2
+#define UNP_GRPH_UPDATE__GRPH_SURFACE_UPDATE_TAKEN_MASK 0x8
+#define UNP_GRPH_UPDATE__GRPH_SURFACE_UPDATE_TAKEN__SHIFT 0x3
+#define UNP_GRPH_UPDATE__GRPH_UPDATE_LOCK_MASK 0x10000
+#define UNP_GRPH_UPDATE__GRPH_UPDATE_LOCK__SHIFT 0x10
+#define UNP_GRPH_UPDATE__GRPH_SURFACE_IGNORE_UPDATE_LOCK_MASK 0x100000
+#define UNP_GRPH_UPDATE__GRPH_SURFACE_IGNORE_UPDATE_LOCK__SHIFT 0x14
+#define UNP_GRPH_UPDATE__GRPH_MODE_DISABLE_MULTIPLE_UPDATE_MASK 0x1000000
+#define UNP_GRPH_UPDATE__GRPH_MODE_DISABLE_MULTIPLE_UPDATE__SHIFT 0x18
+#define UNP_GRPH_UPDATE__GRPH_SURFACE_DISABLE_MULTIPLE_UPDATE_MASK 0x10000000
+#define UNP_GRPH_UPDATE__GRPH_SURFACE_DISABLE_MULTIPLE_UPDATE__SHIFT 0x1c
+#define UNP_PIPE_OUTSTANDING_REQUEST_LIMIT__UNP_PIPE_OUTSTANDING_REQUEST_LIMIT_L_MASK 0xff
+#define UNP_PIPE_OUTSTANDING_REQUEST_LIMIT__UNP_PIPE_OUTSTANDING_REQUEST_LIMIT_L__SHIFT 0x0
+#define UNP_PIPE_OUTSTANDING_REQUEST_LIMIT__UNP_PIPE_OUTSTANDING_REQUEST_LIMIT_C_MASK 0xff00
+#define UNP_PIPE_OUTSTANDING_REQUEST_LIMIT__UNP_PIPE_OUTSTANDING_REQUEST_LIMIT_C__SHIFT 0x8
+#define UNP_GRPH_SURFACE_ADDRESS_INUSE_L__GRPH_SURFACE_ADDRESS_INUSE_L_MASK 0xffffff00
+#define UNP_GRPH_SURFACE_ADDRESS_INUSE_L__GRPH_SURFACE_ADDRESS_INUSE_L__SHIFT 0x8
+#define UNP_GRPH_SURFACE_ADDRESS_INUSE_C__GRPH_SURFACE_ADDRESS_INUSE_C_MASK 0xffffff00
+#define UNP_GRPH_SURFACE_ADDRESS_INUSE_C__GRPH_SURFACE_ADDRESS_INUSE_C__SHIFT 0x8
+#define UNP_GRPH_SURFACE_ADDRESS_HIGH_INUSE_L__GRPH_SURFACE_ADDRESS_HIGH_INUSE_L_MASK 0xff
+#define UNP_GRPH_SURFACE_ADDRESS_HIGH_INUSE_L__GRPH_SURFACE_ADDRESS_HIGH_INUSE_L__SHIFT 0x0
+#define UNP_GRPH_SURFACE_ADDRESS_HIGH_INUSE_C__GRPH_SURFACE_ADDRESS_HIGH_INUSE_C_MASK 0xff
+#define UNP_GRPH_SURFACE_ADDRESS_HIGH_INUSE_C__GRPH_SURFACE_ADDRESS_HIGH_INUSE_C__SHIFT 0x0
+#define UNP_DVMM_PTE_CONTROL__DVMM_USE_SINGLE_PTE_MASK 0x1
+#define UNP_DVMM_PTE_CONTROL__DVMM_USE_SINGLE_PTE__SHIFT 0x0
+#define UNP_DVMM_PTE_CONTROL__DVMM_PAGE_WIDTH_MASK 0x1e
+#define UNP_DVMM_PTE_CONTROL__DVMM_PAGE_WIDTH__SHIFT 0x1
+#define UNP_DVMM_PTE_CONTROL__DVMM_PAGE_HEIGHT_MASK 0x1e0
+#define UNP_DVMM_PTE_CONTROL__DVMM_PAGE_HEIGHT__SHIFT 0x5
+#define UNP_DVMM_PTE_CONTROL__DVMM_MIN_PTE_BEFORE_FLIP_MASK 0x7fe00
+#define UNP_DVMM_PTE_CONTROL__DVMM_MIN_PTE_BEFORE_FLIP__SHIFT 0x9
+#define UNP_DVMM_PTE_CONTROL__DVMM_PTE_BUFFER_MODE0_MASK 0x100000
+#define UNP_DVMM_PTE_CONTROL__DVMM_PTE_BUFFER_MODE0__SHIFT 0x14
+#define UNP_DVMM_PTE_CONTROL__DVMM_PTE_BUFFER_MODE1_MASK 0x200000
+#define UNP_DVMM_PTE_CONTROL__DVMM_PTE_BUFFER_MODE1__SHIFT 0x15
+#define UNP_GRPH_INTERRUPT_STATUS__GRPH_PFLIP_INT_OCCURRED_MASK 0x1
+#define UNP_GRPH_INTERRUPT_STATUS__GRPH_PFLIP_INT_OCCURRED__SHIFT 0x0
+#define UNP_GRPH_INTERRUPT_STATUS__GRPH_PFLIP_INT_CLEAR_MASK 0x100
+#define UNP_GRPH_INTERRUPT_STATUS__GRPH_PFLIP_INT_CLEAR__SHIFT 0x8
+#define UNP_GRPH_INTERRUPT_CONTROL__GRPH_PFLIP_INT_MASK_MASK 0x1
+#define UNP_GRPH_INTERRUPT_CONTROL__GRPH_PFLIP_INT_MASK__SHIFT 0x0
+#define UNP_GRPH_INTERRUPT_CONTROL__GRPH_PFLIP_INT_TYPE_MASK 0x100
+#define UNP_GRPH_INTERRUPT_CONTROL__GRPH_PFLIP_INT_TYPE__SHIFT 0x8
+#define UNP_GRPH_STEREOSYNC_FLIP__GRPH_STEREOSYNC_FLIP_EN_MASK 0x1
+#define UNP_GRPH_STEREOSYNC_FLIP__GRPH_STEREOSYNC_FLIP_EN__SHIFT 0x0
+#define UNP_GRPH_STEREOSYNC_FLIP__GRPH_STEREOSYNC_FLIP_MODE_MASK 0x30
+#define UNP_GRPH_STEREOSYNC_FLIP__GRPH_STEREOSYNC_FLIP_MODE__SHIFT 0x4
+#define UNP_GRPH_STEREOSYNC_FLIP__GRPH_STACK_INTERLACE_FLIP_EN_MASK 0x100
+#define UNP_GRPH_STEREOSYNC_FLIP__GRPH_STACK_INTERLACE_FLIP_EN__SHIFT 0x8
+#define UNP_GRPH_STEREOSYNC_FLIP__GRPH_STACK_INTERLACE_FLIP_MODE_MASK 0x3000
+#define UNP_GRPH_STEREOSYNC_FLIP__GRPH_STACK_INTERLACE_FLIP_MODE__SHIFT 0xc
+#define UNP_GRPH_STEREOSYNC_FLIP__GRPH_PRIMARY_SURFACE_PENDING_MASK 0x10000
+#define UNP_GRPH_STEREOSYNC_FLIP__GRPH_PRIMARY_SURFACE_PENDING__SHIFT 0x10
+#define UNP_GRPH_STEREOSYNC_FLIP__GRPH_SECONDARY_SURFACE_PENDING_MASK 0x20000
+#define UNP_GRPH_STEREOSYNC_FLIP__GRPH_SECONDARY_SURFACE_PENDING__SHIFT 0x11
+#define UNP_GRPH_STEREOSYNC_FLIP__GRPH_PRIMARY_BOTTOM_SURFACE_PENDING_MASK 0x40000
+#define UNP_GRPH_STEREOSYNC_FLIP__GRPH_PRIMARY_BOTTOM_SURFACE_PENDING__SHIFT 0x12
+#define UNP_GRPH_STEREOSYNC_FLIP__GRPH_SECONDARY_BOTTOM_SURFACE_PENDING_MASK 0x80000
+#define UNP_GRPH_STEREOSYNC_FLIP__GRPH_SECONDARY_BOTTOM_SURFACE_PENDING__SHIFT 0x13
+#define UNP_GRPH_STEREOSYNC_FLIP__GRPH_STEREOSYNC_SELECT_DISABLE_MASK 0x10000000
+#define UNP_GRPH_STEREOSYNC_FLIP__GRPH_STEREOSYNC_SELECT_DISABLE__SHIFT 0x1c
+#define UNP_FLIP_CONTROL__GRPH_SURFACE_UPDATE_PENDING_MODE_MASK 0x1
+#define UNP_FLIP_CONTROL__GRPH_SURFACE_UPDATE_PENDING_MODE__SHIFT 0x0
+#define UNP_FLIP_CONTROL__UNP_DEBUG_SG_MASK 0xfffffffc
+#define UNP_FLIP_CONTROL__UNP_DEBUG_SG__SHIFT 0x2
+#define UNP_CRC_CONTROL__UNP_CRC_ENABLE_MASK 0x1
+#define UNP_CRC_CONTROL__UNP_CRC_ENABLE__SHIFT 0x0
+#define UNP_CRC_CONTROL__UNP_CRC_SOURCE_SEL_MASK 0x1c
+#define UNP_CRC_CONTROL__UNP_CRC_SOURCE_SEL__SHIFT 0x2
+#define UNP_CRC_CONTROL__UNP_CRC_LINE_SEL_MASK 0x300
+#define UNP_CRC_CONTROL__UNP_CRC_LINE_SEL__SHIFT 0x8
+#define UNP_CRC_MASK__UNP_CRC_MASK_MASK 0xffffffff
+#define UNP_CRC_MASK__UNP_CRC_MASK__SHIFT 0x0
+#define UNP_CRC_CURRENT__UNP_CRC_CURRENT_MASK 0xffffffff
+#define UNP_CRC_CURRENT__UNP_CRC_CURRENT__SHIFT 0x0
+#define UNP_CRC_LAST__UNP_CRC_LAST_MASK 0xffffffff
+#define UNP_CRC_LAST__UNP_CRC_LAST__SHIFT 0x0
+#define UNP_LB_DATA_GAP_BETWEEN_CHUNK__UNP_LB_GAP_BETWEEN_CHUNK_MASK 0x1f0
+#define UNP_LB_DATA_GAP_BETWEEN_CHUNK__UNP_LB_GAP_BETWEEN_CHUNK__SHIFT 0x4
+#define UNP_HW_ROTATION__ROTATION_ANGLE_MASK 0x7
+#define UNP_HW_ROTATION__ROTATION_ANGLE__SHIFT 0x0
+#define UNP_HW_ROTATION__PIXEL_DROP_MASK 0x10
+#define UNP_HW_ROTATION__PIXEL_DROP__SHIFT 0x4
+#define UNP_HW_ROTATION__BUFFER_MODE_MASK 0x100
+#define UNP_HW_ROTATION__BUFFER_MODE__SHIFT 0x8
+#define UNP_DEBUG__UNP_DEBUG_MASK 0xffffffff
+#define UNP_DEBUG__UNP_DEBUG__SHIFT 0x0
+#define UNP_DEBUG2__UNP_DEBUG2_MASK 0xffffffff
+#define UNP_DEBUG2__UNP_DEBUG2__SHIFT 0x0
+#define UNP_DVMM_DEBUG__UNP_L_DVMM_DEBUG_MASK 0xffff
+#define UNP_DVMM_DEBUG__UNP_L_DVMM_DEBUG__SHIFT 0x0
+#define UNP_DVMM_DEBUG__UNP_C_DVMM_DEBUG_MASK 0xffff0000
+#define UNP_DVMM_DEBUG__UNP_C_DVMM_DEBUG__SHIFT 0x10
+#define UNP_TEST_DEBUG_INDEX__UNP_TEST_DEBUG_INDEX_MASK 0xff
+#define UNP_TEST_DEBUG_INDEX__UNP_TEST_DEBUG_INDEX__SHIFT 0x0
+#define UNP_TEST_DEBUG_INDEX__UNP_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define UNP_TEST_DEBUG_INDEX__UNP_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define UNP_TEST_DEBUG_DATA__UNP_TEST_DEBUG_DATA_MASK 0xffffffff
+#define UNP_TEST_DEBUG_DATA__UNP_TEST_DEBUG_DATA__SHIFT 0x0
+#define GENMO_WT__GENMO_MONO_ADDRESS_B_MASK 0x1
+#define GENMO_WT__GENMO_MONO_ADDRESS_B__SHIFT 0x0
+#define GENMO_WT__VGA_RAM_EN_MASK 0x2
+#define GENMO_WT__VGA_RAM_EN__SHIFT 0x1
+#define GENMO_WT__VGA_CKSEL_MASK 0xc
+#define GENMO_WT__VGA_CKSEL__SHIFT 0x2
+#define GENMO_WT__ODD_EVEN_MD_PGSEL_MASK 0x20
+#define GENMO_WT__ODD_EVEN_MD_PGSEL__SHIFT 0x5
+#define GENMO_WT__VGA_HSYNC_POL_MASK 0x40
+#define GENMO_WT__VGA_HSYNC_POL__SHIFT 0x6
+#define GENMO_WT__VGA_VSYNC_POL_MASK 0x80
+#define GENMO_WT__VGA_VSYNC_POL__SHIFT 0x7
+#define GENMO_RD__GENMO_MONO_ADDRESS_B_MASK 0x1
+#define GENMO_RD__GENMO_MONO_ADDRESS_B__SHIFT 0x0
+#define GENMO_RD__VGA_RAM_EN_MASK 0x2
+#define GENMO_RD__VGA_RAM_EN__SHIFT 0x1
+#define GENMO_RD__VGA_CKSEL_MASK 0xc
+#define GENMO_RD__VGA_CKSEL__SHIFT 0x2
+#define GENMO_RD__ODD_EVEN_MD_PGSEL_MASK 0x20
+#define GENMO_RD__ODD_EVEN_MD_PGSEL__SHIFT 0x5
+#define GENMO_RD__VGA_HSYNC_POL_MASK 0x40
+#define GENMO_RD__VGA_HSYNC_POL__SHIFT 0x6
+#define GENMO_RD__VGA_VSYNC_POL_MASK 0x80
+#define GENMO_RD__VGA_VSYNC_POL__SHIFT 0x7
+#define GENENB__BLK_IO_BASE_MASK 0xff
+#define GENENB__BLK_IO_BASE__SHIFT 0x0
+#define GENFC_WT__VSYNC_SEL_W_MASK 0x8
+#define GENFC_WT__VSYNC_SEL_W__SHIFT 0x3
+#define GENFC_RD__VSYNC_SEL_R_MASK 0x8
+#define GENFC_RD__VSYNC_SEL_R__SHIFT 0x3
+#define GENS0__SENSE_SWITCH_MASK 0x10
+#define GENS0__SENSE_SWITCH__SHIFT 0x4
+#define GENS0__CRT_INTR_MASK 0x80
+#define GENS0__CRT_INTR__SHIFT 0x7
+#define GENS1__NO_DISPLAY_MASK 0x1
+#define GENS1__NO_DISPLAY__SHIFT 0x0
+#define GENS1__VGA_VSTATUS_MASK 0x8
+#define GENS1__VGA_VSTATUS__SHIFT 0x3
+#define GENS1__PIXEL_READ_BACK_MASK 0x30
+#define GENS1__PIXEL_READ_BACK__SHIFT 0x4
+#define DAC_DATA__DAC_DATA_MASK 0x3f
+#define DAC_DATA__DAC_DATA__SHIFT 0x0
+#define DAC_MASK__DAC_MASK_MASK 0xff
+#define DAC_MASK__DAC_MASK__SHIFT 0x0
+#define DAC_R_INDEX__DAC_R_INDEX_MASK 0xff
+#define DAC_R_INDEX__DAC_R_INDEX__SHIFT 0x0
+#define DAC_W_INDEX__DAC_W_INDEX_MASK 0xff
+#define DAC_W_INDEX__DAC_W_INDEX__SHIFT 0x0
+#define SEQ8_IDX__SEQ_IDX_MASK 0x7
+#define SEQ8_IDX__SEQ_IDX__SHIFT 0x0
+#define SEQ8_DATA__SEQ_DATA_MASK 0xff
+#define SEQ8_DATA__SEQ_DATA__SHIFT 0x0
+#define SEQ00__SEQ_RST0B_MASK 0x1
+#define SEQ00__SEQ_RST0B__SHIFT 0x0
+#define SEQ00__SEQ_RST1B_MASK 0x2
+#define SEQ00__SEQ_RST1B__SHIFT 0x1
+#define SEQ01__SEQ_DOT8_MASK 0x1
+#define SEQ01__SEQ_DOT8__SHIFT 0x0
+#define SEQ01__SEQ_SHIFT2_MASK 0x4
+#define SEQ01__SEQ_SHIFT2__SHIFT 0x2
+#define SEQ01__SEQ_PCLKBY2_MASK 0x8
+#define SEQ01__SEQ_PCLKBY2__SHIFT 0x3
+#define SEQ01__SEQ_SHIFT4_MASK 0x10
+#define SEQ01__SEQ_SHIFT4__SHIFT 0x4
+#define SEQ01__SEQ_MAXBW_MASK 0x20
+#define SEQ01__SEQ_MAXBW__SHIFT 0x5
+#define SEQ02__SEQ_MAP0_EN_MASK 0x1
+#define SEQ02__SEQ_MAP0_EN__SHIFT 0x0
+#define SEQ02__SEQ_MAP1_EN_MASK 0x2
+#define SEQ02__SEQ_MAP1_EN__SHIFT 0x1
+#define SEQ02__SEQ_MAP2_EN_MASK 0x4
+#define SEQ02__SEQ_MAP2_EN__SHIFT 0x2
+#define SEQ02__SEQ_MAP3_EN_MASK 0x8
+#define SEQ02__SEQ_MAP3_EN__SHIFT 0x3
+#define SEQ03__SEQ_FONT_B1_MASK 0x1
+#define SEQ03__SEQ_FONT_B1__SHIFT 0x0
+#define SEQ03__SEQ_FONT_B2_MASK 0x2
+#define SEQ03__SEQ_FONT_B2__SHIFT 0x1
+#define SEQ03__SEQ_FONT_A1_MASK 0x4
+#define SEQ03__SEQ_FONT_A1__SHIFT 0x2
+#define SEQ03__SEQ_FONT_A2_MASK 0x8
+#define SEQ03__SEQ_FONT_A2__SHIFT 0x3
+#define SEQ03__SEQ_FONT_B0_MASK 0x10
+#define SEQ03__SEQ_FONT_B0__SHIFT 0x4
+#define SEQ03__SEQ_FONT_A0_MASK 0x20
+#define SEQ03__SEQ_FONT_A0__SHIFT 0x5
+#define SEQ04__SEQ_256K_MASK 0x2
+#define SEQ04__SEQ_256K__SHIFT 0x1
+#define SEQ04__SEQ_ODDEVEN_MASK 0x4
+#define SEQ04__SEQ_ODDEVEN__SHIFT 0x2
+#define SEQ04__SEQ_CHAIN_MASK 0x8
+#define SEQ04__SEQ_CHAIN__SHIFT 0x3
+#define CRTC8_IDX__VCRTC_IDX_MASK 0x3f
+#define CRTC8_IDX__VCRTC_IDX__SHIFT 0x0
+#define CRTC8_DATA__VCRTC_DATA_MASK 0xff
+#define CRTC8_DATA__VCRTC_DATA__SHIFT 0x0
+#define CRT00__H_TOTAL_MASK 0xff
+#define CRT00__H_TOTAL__SHIFT 0x0
+#define CRT01__H_DISP_END_MASK 0xff
+#define CRT01__H_DISP_END__SHIFT 0x0
+#define CRT02__H_BLANK_START_MASK 0xff
+#define CRT02__H_BLANK_START__SHIFT 0x0
+#define CRT03__H_BLANK_END_MASK 0x1f
+#define CRT03__H_BLANK_END__SHIFT 0x0
+#define CRT03__H_DE_SKEW_MASK 0x60
+#define CRT03__H_DE_SKEW__SHIFT 0x5
+#define CRT03__CR10CR11_R_DIS_B_MASK 0x80
+#define CRT03__CR10CR11_R_DIS_B__SHIFT 0x7
+#define CRT04__H_SYNC_START_MASK 0xff
+#define CRT04__H_SYNC_START__SHIFT 0x0
+#define CRT05__H_SYNC_END_MASK 0x1f
+#define CRT05__H_SYNC_END__SHIFT 0x0
+#define CRT05__H_SYNC_SKEW_MASK 0x60
+#define CRT05__H_SYNC_SKEW__SHIFT 0x5
+#define CRT05__H_BLANK_END_B5_MASK 0x80
+#define CRT05__H_BLANK_END_B5__SHIFT 0x7
+#define CRT06__V_TOTAL_MASK 0xff
+#define CRT06__V_TOTAL__SHIFT 0x0
+#define CRT07__V_TOTAL_B8_MASK 0x1
+#define CRT07__V_TOTAL_B8__SHIFT 0x0
+#define CRT07__V_DISP_END_B8_MASK 0x2
+#define CRT07__V_DISP_END_B8__SHIFT 0x1
+#define CRT07__V_SYNC_START_B8_MASK 0x4
+#define CRT07__V_SYNC_START_B8__SHIFT 0x2
+#define CRT07__V_BLANK_START_B8_MASK 0x8
+#define CRT07__V_BLANK_START_B8__SHIFT 0x3
+#define CRT07__LINE_CMP_B8_MASK 0x10
+#define CRT07__LINE_CMP_B8__SHIFT 0x4
+#define CRT07__V_TOTAL_B9_MASK 0x20
+#define CRT07__V_TOTAL_B9__SHIFT 0x5
+#define CRT07__V_DISP_END_B9_MASK 0x40
+#define CRT07__V_DISP_END_B9__SHIFT 0x6
+#define CRT07__V_SYNC_START_B9_MASK 0x80
+#define CRT07__V_SYNC_START_B9__SHIFT 0x7
+#define CRT08__ROW_SCAN_START_MASK 0x1f
+#define CRT08__ROW_SCAN_START__SHIFT 0x0
+#define CRT08__BYTE_PAN_MASK 0x60
+#define CRT08__BYTE_PAN__SHIFT 0x5
+#define CRT09__MAX_ROW_SCAN_MASK 0x1f
+#define CRT09__MAX_ROW_SCAN__SHIFT 0x0
+#define CRT09__V_BLANK_START_B9_MASK 0x20
+#define CRT09__V_BLANK_START_B9__SHIFT 0x5
+#define CRT09__LINE_CMP_B9_MASK 0x40
+#define CRT09__LINE_CMP_B9__SHIFT 0x6
+#define CRT09__DOUBLE_CHAR_HEIGHT_MASK 0x80
+#define CRT09__DOUBLE_CHAR_HEIGHT__SHIFT 0x7
+#define CRT0A__CURSOR_START_MASK 0x1f
+#define CRT0A__CURSOR_START__SHIFT 0x0
+#define CRT0A__CURSOR_DISABLE_MASK 0x20
+#define CRT0A__CURSOR_DISABLE__SHIFT 0x5
+#define CRT0B__CURSOR_END_MASK 0x1f
+#define CRT0B__CURSOR_END__SHIFT 0x0
+#define CRT0B__CURSOR_SKEW_MASK 0x60
+#define CRT0B__CURSOR_SKEW__SHIFT 0x5
+#define CRT0C__DISP_START_MASK 0xff
+#define CRT0C__DISP_START__SHIFT 0x0
+#define CRT0D__DISP_START_MASK 0xff
+#define CRT0D__DISP_START__SHIFT 0x0
+#define CRT0E__CURSOR_LOC_HI_MASK 0xff
+#define CRT0E__CURSOR_LOC_HI__SHIFT 0x0
+#define CRT0F__CURSOR_LOC_LO_MASK 0xff
+#define CRT0F__CURSOR_LOC_LO__SHIFT 0x0
+#define CRT10__V_SYNC_START_MASK 0xff
+#define CRT10__V_SYNC_START__SHIFT 0x0
+#define CRT11__V_SYNC_END_MASK 0xf
+#define CRT11__V_SYNC_END__SHIFT 0x0
+#define CRT11__V_INTR_CLR_MASK 0x10
+#define CRT11__V_INTR_CLR__SHIFT 0x4
+#define CRT11__V_INTR_EN_MASK 0x20
+#define CRT11__V_INTR_EN__SHIFT 0x5
+#define CRT11__SEL5_REFRESH_CYC_MASK 0x40
+#define CRT11__SEL5_REFRESH_CYC__SHIFT 0x6
+#define CRT11__C0T7_WR_ONLY_MASK 0x80
+#define CRT11__C0T7_WR_ONLY__SHIFT 0x7
+#define CRT12__V_DISP_END_MASK 0xff
+#define CRT12__V_DISP_END__SHIFT 0x0
+#define CRT13__DISP_PITCH_MASK 0xff
+#define CRT13__DISP_PITCH__SHIFT 0x0
+#define CRT14__UNDRLN_LOC_MASK 0x1f
+#define CRT14__UNDRLN_LOC__SHIFT 0x0
+#define CRT14__ADDR_CNT_BY4_MASK 0x20
+#define CRT14__ADDR_CNT_BY4__SHIFT 0x5
+#define CRT14__DOUBLE_WORD_MASK 0x40
+#define CRT14__DOUBLE_WORD__SHIFT 0x6
+#define CRT15__V_BLANK_START_MASK 0xff
+#define CRT15__V_BLANK_START__SHIFT 0x0
+#define CRT16__V_BLANK_END_MASK 0xff
+#define CRT16__V_BLANK_END__SHIFT 0x0
+#define CRT17__RA0_AS_A13B_MASK 0x1
+#define CRT17__RA0_AS_A13B__SHIFT 0x0
+#define CRT17__RA1_AS_A14B_MASK 0x2
+#define CRT17__RA1_AS_A14B__SHIFT 0x1
+#define CRT17__VCOUNT_BY2_MASK 0x4
+#define CRT17__VCOUNT_BY2__SHIFT 0x2
+#define CRT17__ADDR_CNT_BY2_MASK 0x8
+#define CRT17__ADDR_CNT_BY2__SHIFT 0x3
+#define CRT17__WRAP_A15TOA0_MASK 0x20
+#define CRT17__WRAP_A15TOA0__SHIFT 0x5
+#define CRT17__BYTE_MODE_MASK 0x40
+#define CRT17__BYTE_MODE__SHIFT 0x6
+#define CRT17__CRTC_SYNC_EN_MASK 0x80
+#define CRT17__CRTC_SYNC_EN__SHIFT 0x7
+#define CRT18__LINE_CMP_MASK 0xff
+#define CRT18__LINE_CMP__SHIFT 0x0
+#define CRT1E__GRPH_DEC_RD1_MASK 0x2
+#define CRT1E__GRPH_DEC_RD1__SHIFT 0x1
+#define CRT1F__GRPH_DEC_RD0_MASK 0xff
+#define CRT1F__GRPH_DEC_RD0__SHIFT 0x0
+#define CRT22__GRPH_LATCH_DATA_MASK 0xff
+#define CRT22__GRPH_LATCH_DATA__SHIFT 0x0
+#define GRPH8_IDX__GRPH_IDX_MASK 0xf
+#define GRPH8_IDX__GRPH_IDX__SHIFT 0x0
+#define GRPH8_DATA__GRPH_DATA_MASK 0xff
+#define GRPH8_DATA__GRPH_DATA__SHIFT 0x0
+#define GRA00__GRPH_SET_RESET0_MASK 0x1
+#define GRA00__GRPH_SET_RESET0__SHIFT 0x0
+#define GRA00__GRPH_SET_RESET1_MASK 0x2
+#define GRA00__GRPH_SET_RESET1__SHIFT 0x1
+#define GRA00__GRPH_SET_RESET2_MASK 0x4
+#define GRA00__GRPH_SET_RESET2__SHIFT 0x2
+#define GRA00__GRPH_SET_RESET3_MASK 0x8
+#define GRA00__GRPH_SET_RESET3__SHIFT 0x3
+#define GRA01__GRPH_SET_RESET_ENA0_MASK 0x1
+#define GRA01__GRPH_SET_RESET_ENA0__SHIFT 0x0
+#define GRA01__GRPH_SET_RESET_ENA1_MASK 0x2
+#define GRA01__GRPH_SET_RESET_ENA1__SHIFT 0x1
+#define GRA01__GRPH_SET_RESET_ENA2_MASK 0x4
+#define GRA01__GRPH_SET_RESET_ENA2__SHIFT 0x2
+#define GRA01__GRPH_SET_RESET_ENA3_MASK 0x8
+#define GRA01__GRPH_SET_RESET_ENA3__SHIFT 0x3
+#define GRA02__GRPH_CCOMP_MASK 0xf
+#define GRA02__GRPH_CCOMP__SHIFT 0x0
+#define GRA03__GRPH_ROTATE_MASK 0x7
+#define GRA03__GRPH_ROTATE__SHIFT 0x0
+#define GRA03__GRPH_FN_SEL_MASK 0x18
+#define GRA03__GRPH_FN_SEL__SHIFT 0x3
+#define GRA04__GRPH_RMAP_MASK 0x3
+#define GRA04__GRPH_RMAP__SHIFT 0x0
+#define GRA05__GRPH_WRITE_MODE_MASK 0x3
+#define GRA05__GRPH_WRITE_MODE__SHIFT 0x0
+#define GRA05__GRPH_READ1_MASK 0x8
+#define GRA05__GRPH_READ1__SHIFT 0x3
+#define GRA05__CGA_ODDEVEN_MASK 0x10
+#define GRA05__CGA_ODDEVEN__SHIFT 0x4
+#define GRA05__GRPH_OES_MASK 0x20
+#define GRA05__GRPH_OES__SHIFT 0x5
+#define GRA05__GRPH_PACK_MASK 0x40
+#define GRA05__GRPH_PACK__SHIFT 0x6
+#define GRA06__GRPH_GRAPHICS_MASK 0x1
+#define GRA06__GRPH_GRAPHICS__SHIFT 0x0
+#define GRA06__GRPH_ODDEVEN_MASK 0x2
+#define GRA06__GRPH_ODDEVEN__SHIFT 0x1
+#define GRA06__GRPH_ADRSEL_MASK 0xc
+#define GRA06__GRPH_ADRSEL__SHIFT 0x2
+#define GRA07__GRPH_XCARE0_MASK 0x1
+#define GRA07__GRPH_XCARE0__SHIFT 0x0
+#define GRA07__GRPH_XCARE1_MASK 0x2
+#define GRA07__GRPH_XCARE1__SHIFT 0x1
+#define GRA07__GRPH_XCARE2_MASK 0x4
+#define GRA07__GRPH_XCARE2__SHIFT 0x2
+#define GRA07__GRPH_XCARE3_MASK 0x8
+#define GRA07__GRPH_XCARE3__SHIFT 0x3
+#define GRA08__GRPH_BMSK_MASK 0xff
+#define GRA08__GRPH_BMSK__SHIFT 0x0
+#define ATTRX__ATTR_IDX_MASK 0x1f
+#define ATTRX__ATTR_IDX__SHIFT 0x0
+#define ATTRX__ATTR_PAL_RW_ENB_MASK 0x20
+#define ATTRX__ATTR_PAL_RW_ENB__SHIFT 0x5
+#define ATTRDW__ATTR_DATA_MASK 0xff
+#define ATTRDW__ATTR_DATA__SHIFT 0x0
+#define ATTRDR__ATTR_DATA_MASK 0xff
+#define ATTRDR__ATTR_DATA__SHIFT 0x0
+#define ATTR00__ATTR_PAL_MASK 0x3f
+#define ATTR00__ATTR_PAL__SHIFT 0x0
+#define ATTR01__ATTR_PAL_MASK 0x3f
+#define ATTR01__ATTR_PAL__SHIFT 0x0
+#define ATTR02__ATTR_PAL_MASK 0x3f
+#define ATTR02__ATTR_PAL__SHIFT 0x0
+#define ATTR03__ATTR_PAL_MASK 0x3f
+#define ATTR03__ATTR_PAL__SHIFT 0x0
+#define ATTR04__ATTR_PAL_MASK 0x3f
+#define ATTR04__ATTR_PAL__SHIFT 0x0
+#define ATTR05__ATTR_PAL_MASK 0x3f
+#define ATTR05__ATTR_PAL__SHIFT 0x0
+#define ATTR06__ATTR_PAL_MASK 0x3f
+#define ATTR06__ATTR_PAL__SHIFT 0x0
+#define ATTR07__ATTR_PAL_MASK 0x3f
+#define ATTR07__ATTR_PAL__SHIFT 0x0
+#define ATTR08__ATTR_PAL_MASK 0x3f
+#define ATTR08__ATTR_PAL__SHIFT 0x0
+#define ATTR09__ATTR_PAL_MASK 0x3f
+#define ATTR09__ATTR_PAL__SHIFT 0x0
+#define ATTR0A__ATTR_PAL_MASK 0x3f
+#define ATTR0A__ATTR_PAL__SHIFT 0x0
+#define ATTR0B__ATTR_PAL_MASK 0x3f
+#define ATTR0B__ATTR_PAL__SHIFT 0x0
+#define ATTR0C__ATTR_PAL_MASK 0x3f
+#define ATTR0C__ATTR_PAL__SHIFT 0x0
+#define ATTR0D__ATTR_PAL_MASK 0x3f
+#define ATTR0D__ATTR_PAL__SHIFT 0x0
+#define ATTR0E__ATTR_PAL_MASK 0x3f
+#define ATTR0E__ATTR_PAL__SHIFT 0x0
+#define ATTR0F__ATTR_PAL_MASK 0x3f
+#define ATTR0F__ATTR_PAL__SHIFT 0x0
+#define ATTR10__ATTR_GRPH_MODE_MASK 0x1
+#define ATTR10__ATTR_GRPH_MODE__SHIFT 0x0
+#define ATTR10__ATTR_MONO_EN_MASK 0x2
+#define ATTR10__ATTR_MONO_EN__SHIFT 0x1
+#define ATTR10__ATTR_LGRPH_EN_MASK 0x4
+#define ATTR10__ATTR_LGRPH_EN__SHIFT 0x2
+#define ATTR10__ATTR_BLINK_EN_MASK 0x8
+#define ATTR10__ATTR_BLINK_EN__SHIFT 0x3
+#define ATTR10__ATTR_PANTOPONLY_MASK 0x20
+#define ATTR10__ATTR_PANTOPONLY__SHIFT 0x5
+#define ATTR10__ATTR_PCLKBY2_MASK 0x40
+#define ATTR10__ATTR_PCLKBY2__SHIFT 0x6
+#define ATTR10__ATTR_CSEL_EN_MASK 0x80
+#define ATTR10__ATTR_CSEL_EN__SHIFT 0x7
+#define ATTR11__ATTR_OVSC_MASK 0xff
+#define ATTR11__ATTR_OVSC__SHIFT 0x0
+#define ATTR12__ATTR_MAP_EN_MASK 0xf
+#define ATTR12__ATTR_MAP_EN__SHIFT 0x0
+#define ATTR12__ATTR_VSMUX_MASK 0x30
+#define ATTR12__ATTR_VSMUX__SHIFT 0x4
+#define ATTR13__ATTR_PPAN_MASK 0xf
+#define ATTR13__ATTR_PPAN__SHIFT 0x0
+#define ATTR14__ATTR_CSEL1_MASK 0x3
+#define ATTR14__ATTR_CSEL1__SHIFT 0x0
+#define ATTR14__ATTR_CSEL2_MASK 0xc
+#define ATTR14__ATTR_CSEL2__SHIFT 0x2
+#define VGA_RENDER_CONTROL__VGA_BLINK_RATE_MASK 0x1f
+#define VGA_RENDER_CONTROL__VGA_BLINK_RATE__SHIFT 0x0
+#define VGA_RENDER_CONTROL__VGA_BLINK_MODE_MASK 0x60
+#define VGA_RENDER_CONTROL__VGA_BLINK_MODE__SHIFT 0x5
+#define VGA_RENDER_CONTROL__VGA_CURSOR_BLINK_INVERT_MASK 0x80
+#define VGA_RENDER_CONTROL__VGA_CURSOR_BLINK_INVERT__SHIFT 0x7
+#define VGA_RENDER_CONTROL__VGA_EXTD_ADDR_COUNT_ENABLE_MASK 0x100
+#define VGA_RENDER_CONTROL__VGA_EXTD_ADDR_COUNT_ENABLE__SHIFT 0x8
+#define VGA_RENDER_CONTROL__VGA_VSTATUS_CNTL_MASK 0x30000
+#define VGA_RENDER_CONTROL__VGA_VSTATUS_CNTL__SHIFT 0x10
+#define VGA_RENDER_CONTROL__VGA_LOCK_8DOT_MASK 0x1000000
+#define VGA_RENDER_CONTROL__VGA_LOCK_8DOT__SHIFT 0x18
+#define VGA_RENDER_CONTROL__VGAREG_LINECMP_COMPATIBILITY_SEL_MASK 0x2000000
+#define VGA_RENDER_CONTROL__VGAREG_LINECMP_COMPATIBILITY_SEL__SHIFT 0x19
+#define VGA_SOURCE_SELECT__VGA_SOURCE_SEL_A_MASK 0x7
+#define VGA_SOURCE_SELECT__VGA_SOURCE_SEL_A__SHIFT 0x0
+#define VGA_SOURCE_SELECT__VGA_SOURCE_SEL_B_MASK 0x700
+#define VGA_SOURCE_SELECT__VGA_SOURCE_SEL_B__SHIFT 0x8
+#define VGA_SEQUENCER_RESET_CONTROL__D1_BLANK_DISPLAY_WHEN_SEQUENCER_RESET_MASK 0x1
+#define VGA_SEQUENCER_RESET_CONTROL__D1_BLANK_DISPLAY_WHEN_SEQUENCER_RESET__SHIFT 0x0
+#define VGA_SEQUENCER_RESET_CONTROL__D2_BLANK_DISPLAY_WHEN_SEQUENCER_RESET_MASK 0x2
+#define VGA_SEQUENCER_RESET_CONTROL__D2_BLANK_DISPLAY_WHEN_SEQUENCER_RESET__SHIFT 0x1
+#define VGA_SEQUENCER_RESET_CONTROL__D3_BLANK_DISPLAY_WHEN_SEQUENCER_RESET_MASK 0x4
+#define VGA_SEQUENCER_RESET_CONTROL__D3_BLANK_DISPLAY_WHEN_SEQUENCER_RESET__SHIFT 0x2
+#define VGA_SEQUENCER_RESET_CONTROL__D4_BLANK_DISPLAY_WHEN_SEQUENCER_RESET_MASK 0x8
+#define VGA_SEQUENCER_RESET_CONTROL__D4_BLANK_DISPLAY_WHEN_SEQUENCER_RESET__SHIFT 0x3
+#define VGA_SEQUENCER_RESET_CONTROL__D5_BLANK_DISPLAY_WHEN_SEQUENCER_RESET_MASK 0x10
+#define VGA_SEQUENCER_RESET_CONTROL__D5_BLANK_DISPLAY_WHEN_SEQUENCER_RESET__SHIFT 0x4
+#define VGA_SEQUENCER_RESET_CONTROL__D6_BLANK_DISPLAY_WHEN_SEQUENCER_RESET_MASK 0x20
+#define VGA_SEQUENCER_RESET_CONTROL__D6_BLANK_DISPLAY_WHEN_SEQUENCER_RESET__SHIFT 0x5
+#define VGA_SEQUENCER_RESET_CONTROL__D1_DISABLE_SYNCS_AND_DE_WHEN_SEQUENCER_RESET_MASK 0x100
+#define VGA_SEQUENCER_RESET_CONTROL__D1_DISABLE_SYNCS_AND_DE_WHEN_SEQUENCER_RESET__SHIFT 0x8
+#define VGA_SEQUENCER_RESET_CONTROL__D2_DISABLE_SYNCS_AND_DE_WHEN_SEQUENCER_RESET_MASK 0x200
+#define VGA_SEQUENCER_RESET_CONTROL__D2_DISABLE_SYNCS_AND_DE_WHEN_SEQUENCER_RESET__SHIFT 0x9
+#define VGA_SEQUENCER_RESET_CONTROL__D3_DISABLE_SYNCS_AND_DE_WHEN_SEQUENCER_RESET_MASK 0x400
+#define VGA_SEQUENCER_RESET_CONTROL__D3_DISABLE_SYNCS_AND_DE_WHEN_SEQUENCER_RESET__SHIFT 0xa
+#define VGA_SEQUENCER_RESET_CONTROL__D4_DISABLE_SYNCS_AND_DE_WHEN_SEQUENCER_RESET_MASK 0x800
+#define VGA_SEQUENCER_RESET_CONTROL__D4_DISABLE_SYNCS_AND_DE_WHEN_SEQUENCER_RESET__SHIFT 0xb
+#define VGA_SEQUENCER_RESET_CONTROL__D5_DISABLE_SYNCS_AND_DE_WHEN_SEQUENCER_RESET_MASK 0x1000
+#define VGA_SEQUENCER_RESET_CONTROL__D5_DISABLE_SYNCS_AND_DE_WHEN_SEQUENCER_RESET__SHIFT 0xc
+#define VGA_SEQUENCER_RESET_CONTROL__D6_DISABLE_SYNCS_AND_DE_WHEN_SEQUENCER_RESET_MASK 0x2000
+#define VGA_SEQUENCER_RESET_CONTROL__D6_DISABLE_SYNCS_AND_DE_WHEN_SEQUENCER_RESET__SHIFT 0xd
+#define VGA_SEQUENCER_RESET_CONTROL__VGA_MODE_AUTO_TRIGGER_ENABLE_MASK 0x10000
+#define VGA_SEQUENCER_RESET_CONTROL__VGA_MODE_AUTO_TRIGGER_ENABLE__SHIFT 0x10
+#define VGA_SEQUENCER_RESET_CONTROL__VGA_MODE_AUTO_TRIGGER_REGISTER_SELECT_MASK 0x20000
+#define VGA_SEQUENCER_RESET_CONTROL__VGA_MODE_AUTO_TRIGGER_REGISTER_SELECT__SHIFT 0x11
+#define VGA_SEQUENCER_RESET_CONTROL__VGA_MODE_AUTO_TRIGGER_INDEX_SELECT_MASK 0xfc0000
+#define VGA_SEQUENCER_RESET_CONTROL__VGA_MODE_AUTO_TRIGGER_INDEX_SELECT__SHIFT 0x12
+#define VGA_MODE_CONTROL__VGA_ATI_LINEAR_MASK 0x1
+#define VGA_MODE_CONTROL__VGA_ATI_LINEAR__SHIFT 0x0
+#define VGA_MODE_CONTROL__VGA_LUT_PALETTE_UPDATE_MODE_MASK 0x30
+#define VGA_MODE_CONTROL__VGA_LUT_PALETTE_UPDATE_MODE__SHIFT 0x4
+#define VGA_MODE_CONTROL__VGA_128K_APERTURE_PAGING_MASK 0x100
+#define VGA_MODE_CONTROL__VGA_128K_APERTURE_PAGING__SHIFT 0x8
+#define VGA_MODE_CONTROL__VGA_TEXT_132_COLUMNS_EN_MASK 0x10000
+#define VGA_MODE_CONTROL__VGA_TEXT_132_COLUMNS_EN__SHIFT 0x10
+#define VGA_SURFACE_PITCH_SELECT__VGA_SURFACE_PITCH_SELECT_MASK 0x3
+#define VGA_SURFACE_PITCH_SELECT__VGA_SURFACE_PITCH_SELECT__SHIFT 0x0
+#define VGA_SURFACE_PITCH_SELECT__VGA_SURFACE_HEIGHT_SELECT_MASK 0x300
+#define VGA_SURFACE_PITCH_SELECT__VGA_SURFACE_HEIGHT_SELECT__SHIFT 0x8
+#define VGA_MEMORY_BASE_ADDRESS__VGA_MEMORY_BASE_ADDRESS_MASK 0xffffffff
+#define VGA_MEMORY_BASE_ADDRESS__VGA_MEMORY_BASE_ADDRESS__SHIFT 0x0
+#define VGA_MEMORY_BASE_ADDRESS_HIGH__VGA_MEMORY_BASE_ADDRESS_HIGH_MASK 0xff
+#define VGA_MEMORY_BASE_ADDRESS_HIGH__VGA_MEMORY_BASE_ADDRESS_HIGH__SHIFT 0x0
+#define VGA_DISPBUF1_SURFACE_ADDR__VGA_DISPBUF1_SURFACE_ADDR_MASK 0x1ffffff
+#define VGA_DISPBUF1_SURFACE_ADDR__VGA_DISPBUF1_SURFACE_ADDR__SHIFT 0x0
+#define VGA_DISPBUF2_SURFACE_ADDR__VGA_DISPBUF2_SURFACE_ADDR_MASK 0x1ffffff
+#define VGA_DISPBUF2_SURFACE_ADDR__VGA_DISPBUF2_SURFACE_ADDR__SHIFT 0x0
+#define VGA_HDP_CONTROL__VGA_MEM_PAGE_SELECT_EN_MASK 0x1
+#define VGA_HDP_CONTROL__VGA_MEM_PAGE_SELECT_EN__SHIFT 0x0
+#define VGA_HDP_CONTROL__VGA_MEMORY_DISABLE_MASK 0x10
+#define VGA_HDP_CONTROL__VGA_MEMORY_DISABLE__SHIFT 0x4
+#define VGA_HDP_CONTROL__VGA_RBBM_LOCK_DISABLE_MASK 0x100
+#define VGA_HDP_CONTROL__VGA_RBBM_LOCK_DISABLE__SHIFT 0x8
+#define VGA_HDP_CONTROL__VGA_SOFT_RESET_MASK 0x10000
+#define VGA_HDP_CONTROL__VGA_SOFT_RESET__SHIFT 0x10
+#define VGA_HDP_CONTROL__VGA_TEST_RESET_CONTROL_MASK 0x1000000
+#define VGA_HDP_CONTROL__VGA_TEST_RESET_CONTROL__SHIFT 0x18
+#define VGA_CACHE_CONTROL__VGA_WRITE_THROUGH_CACHE_DIS_MASK 0x1
+#define VGA_CACHE_CONTROL__VGA_WRITE_THROUGH_CACHE_DIS__SHIFT 0x0
+#define VGA_CACHE_CONTROL__VGA_READ_CACHE_DISABLE_MASK 0x100
+#define VGA_CACHE_CONTROL__VGA_READ_CACHE_DISABLE__SHIFT 0x8
+#define VGA_CACHE_CONTROL__VGA_READ_BUFFER_INVALIDATE_MASK 0x10000
+#define VGA_CACHE_CONTROL__VGA_READ_BUFFER_INVALIDATE__SHIFT 0x10
+#define VGA_CACHE_CONTROL__VGA_DCCIF_W256ONLY_MASK 0x100000
+#define VGA_CACHE_CONTROL__VGA_DCCIF_W256ONLY__SHIFT 0x14
+#define VGA_CACHE_CONTROL__VGA_DCCIF_WC_TIMEOUT_MASK 0x3f000000
+#define VGA_CACHE_CONTROL__VGA_DCCIF_WC_TIMEOUT__SHIFT 0x18
+#define D1VGA_CONTROL__D1VGA_MODE_ENABLE_MASK 0x1
+#define D1VGA_CONTROL__D1VGA_MODE_ENABLE__SHIFT 0x0
+#define D1VGA_CONTROL__D1VGA_TIMING_SELECT_MASK 0x100
+#define D1VGA_CONTROL__D1VGA_TIMING_SELECT__SHIFT 0x8
+#define D1VGA_CONTROL__D1VGA_SYNC_POLARITY_SELECT_MASK 0x200
+#define D1VGA_CONTROL__D1VGA_SYNC_POLARITY_SELECT__SHIFT 0x9
+#define D1VGA_CONTROL__D1VGA_OVERSCAN_COLOR_EN_MASK 0x10000
+#define D1VGA_CONTROL__D1VGA_OVERSCAN_COLOR_EN__SHIFT 0x10
+#define D1VGA_CONTROL__D1VGA_ROTATE_MASK 0x3000000
+#define D1VGA_CONTROL__D1VGA_ROTATE__SHIFT 0x18
+#define D2VGA_CONTROL__D2VGA_MODE_ENABLE_MASK 0x1
+#define D2VGA_CONTROL__D2VGA_MODE_ENABLE__SHIFT 0x0
+#define D2VGA_CONTROL__D2VGA_TIMING_SELECT_MASK 0x100
+#define D2VGA_CONTROL__D2VGA_TIMING_SELECT__SHIFT 0x8
+#define D2VGA_CONTROL__D2VGA_SYNC_POLARITY_SELECT_MASK 0x200
+#define D2VGA_CONTROL__D2VGA_SYNC_POLARITY_SELECT__SHIFT 0x9
+#define D2VGA_CONTROL__D2VGA_OVERSCAN_COLOR_EN_MASK 0x10000
+#define D2VGA_CONTROL__D2VGA_OVERSCAN_COLOR_EN__SHIFT 0x10
+#define D2VGA_CONTROL__D2VGA_ROTATE_MASK 0x3000000
+#define D2VGA_CONTROL__D2VGA_ROTATE__SHIFT 0x18
+#define D3VGA_CONTROL__D3VGA_MODE_ENABLE_MASK 0x1
+#define D3VGA_CONTROL__D3VGA_MODE_ENABLE__SHIFT 0x0
+#define D3VGA_CONTROL__D3VGA_TIMING_SELECT_MASK 0x100
+#define D3VGA_CONTROL__D3VGA_TIMING_SELECT__SHIFT 0x8
+#define D3VGA_CONTROL__D3VGA_SYNC_POLARITY_SELECT_MASK 0x200
+#define D3VGA_CONTROL__D3VGA_SYNC_POLARITY_SELECT__SHIFT 0x9
+#define D3VGA_CONTROL__D3VGA_OVERSCAN_COLOR_EN_MASK 0x10000
+#define D3VGA_CONTROL__D3VGA_OVERSCAN_COLOR_EN__SHIFT 0x10
+#define D3VGA_CONTROL__D3VGA_ROTATE_MASK 0x3000000
+#define D3VGA_CONTROL__D3VGA_ROTATE__SHIFT 0x18
+#define D4VGA_CONTROL__D4VGA_MODE_ENABLE_MASK 0x1
+#define D4VGA_CONTROL__D4VGA_MODE_ENABLE__SHIFT 0x0
+#define D4VGA_CONTROL__D4VGA_TIMING_SELECT_MASK 0x100
+#define D4VGA_CONTROL__D4VGA_TIMING_SELECT__SHIFT 0x8
+#define D4VGA_CONTROL__D4VGA_SYNC_POLARITY_SELECT_MASK 0x200
+#define D4VGA_CONTROL__D4VGA_SYNC_POLARITY_SELECT__SHIFT 0x9
+#define D4VGA_CONTROL__D4VGA_OVERSCAN_COLOR_EN_MASK 0x10000
+#define D4VGA_CONTROL__D4VGA_OVERSCAN_COLOR_EN__SHIFT 0x10
+#define D4VGA_CONTROL__D4VGA_ROTATE_MASK 0x3000000
+#define D4VGA_CONTROL__D4VGA_ROTATE__SHIFT 0x18
+#define D5VGA_CONTROL__D5VGA_MODE_ENABLE_MASK 0x1
+#define D5VGA_CONTROL__D5VGA_MODE_ENABLE__SHIFT 0x0
+#define D5VGA_CONTROL__D5VGA_TIMING_SELECT_MASK 0x100
+#define D5VGA_CONTROL__D5VGA_TIMING_SELECT__SHIFT 0x8
+#define D5VGA_CONTROL__D5VGA_SYNC_POLARITY_SELECT_MASK 0x200
+#define D5VGA_CONTROL__D5VGA_SYNC_POLARITY_SELECT__SHIFT 0x9
+#define D5VGA_CONTROL__D5VGA_OVERSCAN_COLOR_EN_MASK 0x10000
+#define D5VGA_CONTROL__D5VGA_OVERSCAN_COLOR_EN__SHIFT 0x10
+#define D5VGA_CONTROL__D5VGA_ROTATE_MASK 0x3000000
+#define D5VGA_CONTROL__D5VGA_ROTATE__SHIFT 0x18
+#define D6VGA_CONTROL__D6VGA_MODE_ENABLE_MASK 0x1
+#define D6VGA_CONTROL__D6VGA_MODE_ENABLE__SHIFT 0x0
+#define D6VGA_CONTROL__D6VGA_TIMING_SELECT_MASK 0x100
+#define D6VGA_CONTROL__D6VGA_TIMING_SELECT__SHIFT 0x8
+#define D6VGA_CONTROL__D6VGA_SYNC_POLARITY_SELECT_MASK 0x200
+#define D6VGA_CONTROL__D6VGA_SYNC_POLARITY_SELECT__SHIFT 0x9
+#define D6VGA_CONTROL__D6VGA_OVERSCAN_COLOR_EN_MASK 0x10000
+#define D6VGA_CONTROL__D6VGA_OVERSCAN_COLOR_EN__SHIFT 0x10
+#define D6VGA_CONTROL__D6VGA_ROTATE_MASK 0x3000000
+#define D6VGA_CONTROL__D6VGA_ROTATE__SHIFT 0x18
+#define VGA_HW_DEBUG__VGA_HW_DEBUG_MASK 0xffffffff
+#define VGA_HW_DEBUG__VGA_HW_DEBUG__SHIFT 0x0
+#define VGA_STATUS__VGA_MEM_ACCESS_STATUS_MASK 0x1
+#define VGA_STATUS__VGA_MEM_ACCESS_STATUS__SHIFT 0x0
+#define VGA_STATUS__VGA_REG_ACCESS_STATUS_MASK 0x2
+#define VGA_STATUS__VGA_REG_ACCESS_STATUS__SHIFT 0x1
+#define VGA_STATUS__VGA_DISPLAY_SWITCH_STATUS_MASK 0x4
+#define VGA_STATUS__VGA_DISPLAY_SWITCH_STATUS__SHIFT 0x2
+#define VGA_STATUS__VGA_MODE_AUTO_TRIGGER_STATUS_MASK 0x8
+#define VGA_STATUS__VGA_MODE_AUTO_TRIGGER_STATUS__SHIFT 0x3
+#define VGA_INTERRUPT_CONTROL__VGA_MEM_ACCESS_INT_MASK_MASK 0x1
+#define VGA_INTERRUPT_CONTROL__VGA_MEM_ACCESS_INT_MASK__SHIFT 0x0
+#define VGA_INTERRUPT_CONTROL__VGA_REG_ACCESS_INT_MASK_MASK 0x100
+#define VGA_INTERRUPT_CONTROL__VGA_REG_ACCESS_INT_MASK__SHIFT 0x8
+#define VGA_INTERRUPT_CONTROL__VGA_DISPLAY_SWITCH_INT_MASK_MASK 0x10000
+#define VGA_INTERRUPT_CONTROL__VGA_DISPLAY_SWITCH_INT_MASK__SHIFT 0x10
+#define VGA_INTERRUPT_CONTROL__VGA_MODE_AUTO_TRIGGER_INT_MASK_MASK 0x1000000
+#define VGA_INTERRUPT_CONTROL__VGA_MODE_AUTO_TRIGGER_INT_MASK__SHIFT 0x18
+#define VGA_STATUS_CLEAR__VGA_MEM_ACCESS_INT_CLEAR_MASK 0x1
+#define VGA_STATUS_CLEAR__VGA_MEM_ACCESS_INT_CLEAR__SHIFT 0x0
+#define VGA_STATUS_CLEAR__VGA_REG_ACCESS_INT_CLEAR_MASK 0x100
+#define VGA_STATUS_CLEAR__VGA_REG_ACCESS_INT_CLEAR__SHIFT 0x8
+#define VGA_STATUS_CLEAR__VGA_DISPLAY_SWITCH_INT_CLEAR_MASK 0x10000
+#define VGA_STATUS_CLEAR__VGA_DISPLAY_SWITCH_INT_CLEAR__SHIFT 0x10
+#define VGA_STATUS_CLEAR__VGA_MODE_AUTO_TRIGGER_INT_CLEAR_MASK 0x1000000
+#define VGA_STATUS_CLEAR__VGA_MODE_AUTO_TRIGGER_INT_CLEAR__SHIFT 0x18
+#define VGA_INTERRUPT_STATUS__VGA_MEM_ACCESS_INT_STATUS_MASK 0x1
+#define VGA_INTERRUPT_STATUS__VGA_MEM_ACCESS_INT_STATUS__SHIFT 0x0
+#define VGA_INTERRUPT_STATUS__VGA_REG_ACCESS_INT_STATUS_MASK 0x2
+#define VGA_INTERRUPT_STATUS__VGA_REG_ACCESS_INT_STATUS__SHIFT 0x1
+#define VGA_INTERRUPT_STATUS__VGA_DISPLAY_SWITCH_INT_STATUS_MASK 0x4
+#define VGA_INTERRUPT_STATUS__VGA_DISPLAY_SWITCH_INT_STATUS__SHIFT 0x2
+#define VGA_INTERRUPT_STATUS__VGA_MODE_AUTO_TRIGGER_INT_STATUS_MASK 0x8
+#define VGA_INTERRUPT_STATUS__VGA_MODE_AUTO_TRIGGER_INT_STATUS__SHIFT 0x3
+#define VGA_MAIN_CONTROL__VGA_CRTC_TIMEOUT_MASK 0x3
+#define VGA_MAIN_CONTROL__VGA_CRTC_TIMEOUT__SHIFT 0x0
+#define VGA_MAIN_CONTROL__VGA_RENDER_TIMEOUT_COUNT_MASK 0x18
+#define VGA_MAIN_CONTROL__VGA_RENDER_TIMEOUT_COUNT__SHIFT 0x3
+#define VGA_MAIN_CONTROL__VGA_VIRTUAL_VERTICAL_RETRACE_DURATION_MASK 0xe0
+#define VGA_MAIN_CONTROL__VGA_VIRTUAL_VERTICAL_RETRACE_DURATION__SHIFT 0x5
+#define VGA_MAIN_CONTROL__VGA_READBACK_VGA_VSTATUS_SOURCE_SELECT_MASK 0x300
+#define VGA_MAIN_CONTROL__VGA_READBACK_VGA_VSTATUS_SOURCE_SELECT__SHIFT 0x8
+#define VGA_MAIN_CONTROL__VGA_MC_WRITE_CLEAN_WAIT_DELAY_MASK 0xf000
+#define VGA_MAIN_CONTROL__VGA_MC_WRITE_CLEAN_WAIT_DELAY__SHIFT 0xc
+#define VGA_MAIN_CONTROL__VGA_READBACK_NO_DISPLAY_SOURCE_SELECT_MASK 0x30000
+#define VGA_MAIN_CONTROL__VGA_READBACK_NO_DISPLAY_SOURCE_SELECT__SHIFT 0x10
+#define VGA_MAIN_CONTROL__VGA_READBACK_CRT_INTR_SOURCE_SELECT_MASK 0x3000000
+#define VGA_MAIN_CONTROL__VGA_READBACK_CRT_INTR_SOURCE_SELECT__SHIFT 0x18
+#define VGA_MAIN_CONTROL__VGA_READBACK_SENSE_SWITCH_SELECT_MASK 0x4000000
+#define VGA_MAIN_CONTROL__VGA_READBACK_SENSE_SWITCH_SELECT__SHIFT 0x1a
+#define VGA_MAIN_CONTROL__VGA_READ_URGENT_ENABLE_MASK 0x8000000
+#define VGA_MAIN_CONTROL__VGA_READ_URGENT_ENABLE__SHIFT 0x1b
+#define VGA_MAIN_CONTROL__VGA_WRITES_URGENT_ENABLE_MASK 0x10000000
+#define VGA_MAIN_CONTROL__VGA_WRITES_URGENT_ENABLE__SHIFT 0x1c
+#define VGA_MAIN_CONTROL__VGA_EXTERNAL_DAC_SENSE_MASK 0x20000000
+#define VGA_MAIN_CONTROL__VGA_EXTERNAL_DAC_SENSE__SHIFT 0x1d
+#define VGA_MAIN_CONTROL__VGA_MAIN_TEST_VSTATUS_NO_DISPLAY_CRTC_TIMEOUT_MASK 0x80000000
+#define VGA_MAIN_CONTROL__VGA_MAIN_TEST_VSTATUS_NO_DISPLAY_CRTC_TIMEOUT__SHIFT 0x1f
+#define VGA_TEST_CONTROL__VGA_TEST_ENABLE_MASK 0x1
+#define VGA_TEST_CONTROL__VGA_TEST_ENABLE__SHIFT 0x0
+#define VGA_TEST_CONTROL__VGA_TEST_RENDER_START_MASK 0x100
+#define VGA_TEST_CONTROL__VGA_TEST_RENDER_START__SHIFT 0x8
+#define VGA_TEST_CONTROL__VGA_TEST_RENDER_DONE_MASK 0x10000
+#define VGA_TEST_CONTROL__VGA_TEST_RENDER_DONE__SHIFT 0x10
+#define VGA_TEST_CONTROL__VGA_TEST_RENDER_DISPBUF_SELECT_MASK 0x1000000
+#define VGA_TEST_CONTROL__VGA_TEST_RENDER_DISPBUF_SELECT__SHIFT 0x18
+#define VGA_DEBUG_READBACK_INDEX__VGA_DEBUG_READBACK_INDEX_MASK 0xff
+#define VGA_DEBUG_READBACK_INDEX__VGA_DEBUG_READBACK_INDEX__SHIFT 0x0
+#define VGA_DEBUG_READBACK_DATA__VGA_DEBUG_READBACK_DATA_MASK 0xffffffff
+#define VGA_DEBUG_READBACK_DATA__VGA_DEBUG_READBACK_DATA__SHIFT 0x0
+#define VGA_MEM_WRITE_PAGE_ADDR__VGA_MEM_WRITE_PAGE0_ADDR_MASK 0x3ff
+#define VGA_MEM_WRITE_PAGE_ADDR__VGA_MEM_WRITE_PAGE0_ADDR__SHIFT 0x0
+#define VGA_MEM_WRITE_PAGE_ADDR__VGA_MEM_WRITE_PAGE1_ADDR_MASK 0x3ff0000
+#define VGA_MEM_WRITE_PAGE_ADDR__VGA_MEM_WRITE_PAGE1_ADDR__SHIFT 0x10
+#define VGA_MEM_READ_PAGE_ADDR__VGA_MEM_READ_PAGE0_ADDR_MASK 0x3ff
+#define VGA_MEM_READ_PAGE_ADDR__VGA_MEM_READ_PAGE0_ADDR__SHIFT 0x0
+#define VGA_MEM_READ_PAGE_ADDR__VGA_MEM_READ_PAGE1_ADDR_MASK 0x3ff0000
+#define VGA_MEM_READ_PAGE_ADDR__VGA_MEM_READ_PAGE1_ADDR__SHIFT 0x10
+#define VGA_TEST_DEBUG_INDEX__VGA_TEST_DEBUG_INDEX_MASK 0xff
+#define VGA_TEST_DEBUG_INDEX__VGA_TEST_DEBUG_INDEX__SHIFT 0x0
+#define VGA_TEST_DEBUG_INDEX__VGA_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define VGA_TEST_DEBUG_INDEX__VGA_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define VGA_TEST_DEBUG_DATA__VGA_TEST_DEBUG_DATA_MASK 0xffffffff
+#define VGA_TEST_DEBUG_DATA__VGA_TEST_DEBUG_DATA__SHIFT 0x0
+#define VGADCC_DBG_DCCIF_C__DBG_DCCIF_C_MASK 0xffffffff
+#define VGADCC_DBG_DCCIF_C__DBG_DCCIF_C__SHIFT 0x0
+#define BPHYC_DAC_MACRO_CNTL__BPHYC_DAC_WHITE_LEVEL_MASK 0x3
+#define BPHYC_DAC_MACRO_CNTL__BPHYC_DAC_WHITE_LEVEL__SHIFT 0x0
+#define BPHYC_DAC_MACRO_CNTL__BPHYC_DAC_WHITE_FINE_CONTROL_MASK 0x3f00
+#define BPHYC_DAC_MACRO_CNTL__BPHYC_DAC_WHITE_FINE_CONTROL__SHIFT 0x8
+#define BPHYC_DAC_MACRO_CNTL__BPHYC_DAC_BANDGAP_ADJUSTMENT_MASK 0x3f0000
+#define BPHYC_DAC_MACRO_CNTL__BPHYC_DAC_BANDGAP_ADJUSTMENT__SHIFT 0x10
+#define BPHYC_DAC_MACRO_CNTL__BPHYC_DAC_ANALOG_MONITOR_MASK 0xf000000
+#define BPHYC_DAC_MACRO_CNTL__BPHYC_DAC_ANALOG_MONITOR__SHIFT 0x18
+#define BPHYC_DAC_MACRO_CNTL__BPHYC_DAC_COREMON_MASK 0x10000000
+#define BPHYC_DAC_MACRO_CNTL__BPHYC_DAC_COREMON__SHIFT 0x1c
+#define BPHYC_DAC_AUTO_CALIB_CONTROL__BPHYC_DAC_CAL_INITB_MASK 0x1
+#define BPHYC_DAC_AUTO_CALIB_CONTROL__BPHYC_DAC_CAL_INITB__SHIFT 0x0
+#define BPHYC_DAC_AUTO_CALIB_CONTROL__BPHYC_DAC_CAL_EN_MASK 0x2
+#define BPHYC_DAC_AUTO_CALIB_CONTROL__BPHYC_DAC_CAL_EN__SHIFT 0x1
+#define BPHYC_DAC_AUTO_CALIB_CONTROL__BPHYC_DAC_CAL_DACADJ_EN_MASK 0x4
+#define BPHYC_DAC_AUTO_CALIB_CONTROL__BPHYC_DAC_CAL_DACADJ_EN__SHIFT 0x2
+#define BPHYC_DAC_AUTO_CALIB_CONTROL__BPHYC_DAC_CAL_WAIT_ADJUST_MASK 0x3ff0
+#define BPHYC_DAC_AUTO_CALIB_CONTROL__BPHYC_DAC_CAL_WAIT_ADJUST__SHIFT 0x4
+#define BPHYC_DAC_AUTO_CALIB_CONTROL__BPHYC_DAC_CAL_MASK_MASK 0x700000
+#define BPHYC_DAC_AUTO_CALIB_CONTROL__BPHYC_DAC_CAL_MASK__SHIFT 0x14
+#define BPHYC_DAC_AUTO_CALIB_CONTROL__BPHYC_DAC_CAL_COMPLETE_MASK 0x10000000
+#define BPHYC_DAC_AUTO_CALIB_CONTROL__BPHYC_DAC_CAL_COMPLETE__SHIFT 0x1c
+#define DPG_PIPE_ARBITRATION_CONTROL1__PIXEL_DURATION_MASK 0xffff
+#define DPG_PIPE_ARBITRATION_CONTROL1__PIXEL_DURATION__SHIFT 0x0
+#define DPG_PIPE_ARBITRATION_CONTROL1__BASE_WEIGHT_MASK 0xffff0000
+#define DPG_PIPE_ARBITRATION_CONTROL1__BASE_WEIGHT__SHIFT 0x10
+#define DPG_PIPE_ARBITRATION_CONTROL2__TIME_WEIGHT_MASK 0xffff
+#define DPG_PIPE_ARBITRATION_CONTROL2__TIME_WEIGHT__SHIFT 0x0
+#define DPG_PIPE_ARBITRATION_CONTROL2__URGENCY_WEIGHT_MASK 0xffff0000
+#define DPG_PIPE_ARBITRATION_CONTROL2__URGENCY_WEIGHT__SHIFT 0x10
+#define DPG_WATERMARK_MASK_CONTROL__STUTTER_EXIT_SELF_REFRESH_WATERMARK_MASK_MASK 0x7
+#define DPG_WATERMARK_MASK_CONTROL__STUTTER_EXIT_SELF_REFRESH_WATERMARK_MASK__SHIFT 0x0
+#define DPG_WATERMARK_MASK_CONTROL__URGENCY_WATERMARK_MASK_MASK 0x700
+#define DPG_WATERMARK_MASK_CONTROL__URGENCY_WATERMARK_MASK__SHIFT 0x8
+#define DPG_WATERMARK_MASK_CONTROL__NB_PSTATE_CHANGE_WATERMARK_MASK_MASK 0x70000
+#define DPG_WATERMARK_MASK_CONTROL__NB_PSTATE_CHANGE_WATERMARK_MASK__SHIFT 0x10
+#define DPG_WATERMARK_MASK_CONTROL__DISABLE_FLIP_URGENT_MASK 0x1000000
+#define DPG_WATERMARK_MASK_CONTROL__DISABLE_FLIP_URGENT__SHIFT 0x18
+#define DPG_PIPE_URGENCY_CONTROL__URGENCY_LOW_WATERMARK_MASK 0xffff
+#define DPG_PIPE_URGENCY_CONTROL__URGENCY_LOW_WATERMARK__SHIFT 0x0
+#define DPG_PIPE_URGENCY_CONTROL__URGENCY_HIGH_WATERMARK_MASK 0xffff0000
+#define DPG_PIPE_URGENCY_CONTROL__URGENCY_HIGH_WATERMARK__SHIFT 0x10
+#define DPG_PIPE_DPM_CONTROL__DPM_ENABLE_MASK 0x1
+#define DPG_PIPE_DPM_CONTROL__DPM_ENABLE__SHIFT 0x0
+#define DPG_PIPE_DPM_CONTROL__MCLK_CHANGE_ENABLE_MASK 0x10
+#define DPG_PIPE_DPM_CONTROL__MCLK_CHANGE_ENABLE__SHIFT 0x4
+#define DPG_PIPE_DPM_CONTROL__MCLK_CHANGE_FORCE_ON_MASK 0x100
+#define DPG_PIPE_DPM_CONTROL__MCLK_CHANGE_FORCE_ON__SHIFT 0x8
+#define DPG_PIPE_DPM_CONTROL__MCLK_CHANGE_WATERMARK_MASK_MASK 0x3000
+#define DPG_PIPE_DPM_CONTROL__MCLK_CHANGE_WATERMARK_MASK__SHIFT 0xc
+#define DPG_PIPE_DPM_CONTROL__MCLK_CHANGE_WATERMARK_MASK 0xffff0000
+#define DPG_PIPE_DPM_CONTROL__MCLK_CHANGE_WATERMARK__SHIFT 0x10
+#define DPG_PIPE_STUTTER_CONTROL__STUTTER_ENABLE_MASK 0x1
+#define DPG_PIPE_STUTTER_CONTROL__STUTTER_ENABLE__SHIFT 0x0
+#define DPG_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_CURSOR_MASK 0x10
+#define DPG_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_CURSOR__SHIFT 0x4
+#define DPG_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_ICON_MASK 0x20
+#define DPG_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_ICON__SHIFT 0x5
+#define DPG_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_VGA_MASK 0x40
+#define DPG_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_VGA__SHIFT 0x6
+#define DPG_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_FBC_MASK 0x80
+#define DPG_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_FBC__SHIFT 0x7
+#define DPG_PIPE_STUTTER_CONTROL__STUTTER_WM_HIGH_FORCE_ON_MASK 0x100
+#define DPG_PIPE_STUTTER_CONTROL__STUTTER_WM_HIGH_FORCE_ON__SHIFT 0x8
+#define DPG_PIPE_STUTTER_CONTROL__STUTTER_WM_HIGH_EXCLUDES_VBLANK_MASK 0x200
+#define DPG_PIPE_STUTTER_CONTROL__STUTTER_WM_HIGH_EXCLUDES_VBLANK__SHIFT 0x9
+#define DPG_PIPE_STUTTER_CONTROL__STUTTER_URGENT_IN_NOT_SELF_REFRESH_MASK 0x400
+#define DPG_PIPE_STUTTER_CONTROL__STUTTER_URGENT_IN_NOT_SELF_REFRESH__SHIFT 0xa
+#define DPG_PIPE_STUTTER_CONTROL__STUTTER_SELF_REFRESH_FORCE_ON_MASK 0x800
+#define DPG_PIPE_STUTTER_CONTROL__STUTTER_SELF_REFRESH_FORCE_ON__SHIFT 0xb
+#define DPG_PIPE_STUTTER_CONTROL__STUTTER_EXIT_SELF_REFRESH_WATERMARK_MASK 0xffff0000
+#define DPG_PIPE_STUTTER_CONTROL__STUTTER_EXIT_SELF_REFRESH_WATERMARK__SHIFT 0x10
+#define DPG_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_ENABLE_MASK 0x1
+#define DPG_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_ENABLE__SHIFT 0x0
+#define DPG_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_URGENT_DURING_REQUEST_MASK 0x10
+#define DPG_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_URGENT_DURING_REQUEST__SHIFT 0x4
+#define DPG_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_NOT_SELF_REFRESH_DURING_REQUEST_MASK 0x100
+#define DPG_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_NOT_SELF_REFRESH_DURING_REQUEST__SHIFT 0x8
+#define DPG_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_FORCE_ON_MASK 0x200
+#define DPG_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_FORCE_ON__SHIFT 0x9
+#define DPG_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_ALLOW_FOR_URGENT_MASK 0x400
+#define DPG_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_ALLOW_FOR_URGENT__SHIFT 0xa
+#define DPG_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_WATERMARK_MASK 0xffff8000
+#define DPG_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_WATERMARK__SHIFT 0xf
+#define DPG_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_ENABLE_NONLPTCH_MASK 0x1
+#define DPG_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_ENABLE_NONLPTCH__SHIFT 0x0
+#define DPG_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_CURSOR_NONLPTCH_MASK 0x10
+#define DPG_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_CURSOR_NONLPTCH__SHIFT 0x4
+#define DPG_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_ICON_NONLPTCH_MASK 0x20
+#define DPG_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_ICON_NONLPTCH__SHIFT 0x5
+#define DPG_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_VGA_NONLPTCH_MASK 0x40
+#define DPG_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_VGA_NONLPTCH__SHIFT 0x6
+#define DPG_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_FBC_NONLPTCH_MASK 0x80
+#define DPG_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_FBC_NONLPTCH__SHIFT 0x7
+#define DPG_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_WM_HIGH_FORCE_ON_NONLPTCH_MASK 0x100
+#define DPG_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_WM_HIGH_FORCE_ON_NONLPTCH__SHIFT 0x8
+#define DPG_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_WM_HIGH_EXCLUDES_VBLANK_NONLPTCH_MASK 0x200
+#define DPG_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_WM_HIGH_EXCLUDES_VBLANK_NONLPTCH__SHIFT 0x9
+#define DPG_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_URGENT_IN_NOT_SELF_REFRESH_NONLPTCH_MASK 0x400
+#define DPG_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_URGENT_IN_NOT_SELF_REFRESH_NONLPTCH__SHIFT 0xa
+#define DPG_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_SELF_REFRESH_FORCE_ON_NONLPTCH_MASK 0x800
+#define DPG_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_SELF_REFRESH_FORCE_ON_NONLPTCH__SHIFT 0xb
+#define DPG_REPEATER_PROGRAM__REG_DPG_DMIFRC_REPEATER_MASK 0x7
+#define DPG_REPEATER_PROGRAM__REG_DPG_DMIFRC_REPEATER__SHIFT 0x0
+#define DPG_REPEATER_PROGRAM__REG_DMIFRC_DPG_REPEATER_MASK 0x70
+#define DPG_REPEATER_PROGRAM__REG_DMIFRC_DPG_REPEATER__SHIFT 0x4
+#define DPG_HW_DEBUG_A__DPG_HW_DEBUG_A_MASK 0xffffffff
+#define DPG_HW_DEBUG_A__DPG_HW_DEBUG_A__SHIFT 0x0
+#define DPG_HW_DEBUG_B__DPG_HW_DEBUG_B_MASK 0xffffffff
+#define DPG_HW_DEBUG_B__DPG_HW_DEBUG_B__SHIFT 0x0
+#define DPG_HW_DEBUG_11__DPG_HW_DEBUG_11_MASK 0x1
+#define DPG_HW_DEBUG_11__DPG_HW_DEBUG_11__SHIFT 0x0
+#define DPG_CHK_PRE_PROC_CNTL__DPG_DISABLE_DMIF_BUF_CHK_MASK 0x1
+#define DPG_CHK_PRE_PROC_CNTL__DPG_DISABLE_DMIF_BUF_CHK__SHIFT 0x0
+#define DPG_DVMM_STATUS__DPG_DVMM_FORCED_FLIP_TO_UNMAPPED_MASK 0x1
+#define DPG_DVMM_STATUS__DPG_DVMM_FORCED_FLIP_TO_UNMAPPED__SHIFT 0x0
+#define DPG_DVMM_STATUS__DPG_DVMM_FORCED_FLIP_TO_MAPPED_MASK 0x2
+#define DPG_DVMM_STATUS__DPG_DVMM_FORCED_FLIP_TO_MAPPED__SHIFT 0x1
+#define DPG_DVMM_STATUS__DPG_DVMM_FORCED_FLIP_TO_UNMAPPED_CLR_MASK 0x10
+#define DPG_DVMM_STATUS__DPG_DVMM_FORCED_FLIP_TO_UNMAPPED_CLR__SHIFT 0x4
+#define DPG_DVMM_STATUS__DPG_DVMM_FORCED_FLIP_TO_MAPPED_CLR_MASK 0x20
+#define DPG_DVMM_STATUS__DPG_DVMM_FORCED_FLIP_TO_MAPPED_CLR__SHIFT 0x5
+#define DPG_TEST_DEBUG_INDEX__DPG_TEST_DEBUG_INDEX_MASK 0xff
+#define DPG_TEST_DEBUG_INDEX__DPG_TEST_DEBUG_INDEX__SHIFT 0x0
+#define DPG_TEST_DEBUG_INDEX__DPG_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define DPG_TEST_DEBUG_INDEX__DPG_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define DPG_TEST_DEBUG_DATA__DPG_TEST_DEBUG_DATA_MASK 0xffffffff
+#define DPG_TEST_DEBUG_DATA__DPG_TEST_DEBUG_DATA__SHIFT 0x0
+#define DPGV0_PIPE_ARBITRATION_CONTROL1__PIXEL_DURATION_MASK 0xffff
+#define DPGV0_PIPE_ARBITRATION_CONTROL1__PIXEL_DURATION__SHIFT 0x0
+#define DPGV0_PIPE_ARBITRATION_CONTROL1__BASE_WEIGHT_MASK 0xffff0000
+#define DPGV0_PIPE_ARBITRATION_CONTROL1__BASE_WEIGHT__SHIFT 0x10
+#define DPGV1_PIPE_ARBITRATION_CONTROL1__PIXEL_DURATION_MASK 0xffff
+#define DPGV1_PIPE_ARBITRATION_CONTROL1__PIXEL_DURATION__SHIFT 0x0
+#define DPGV1_PIPE_ARBITRATION_CONTROL1__BASE_WEIGHT_MASK 0xffff0000
+#define DPGV1_PIPE_ARBITRATION_CONTROL1__BASE_WEIGHT__SHIFT 0x10
+#define DPGV0_PIPE_ARBITRATION_CONTROL2__TIME_WEIGHT_MASK 0xffff
+#define DPGV0_PIPE_ARBITRATION_CONTROL2__TIME_WEIGHT__SHIFT 0x0
+#define DPGV0_PIPE_ARBITRATION_CONTROL2__URGENCY_WEIGHT_MASK 0xffff0000
+#define DPGV0_PIPE_ARBITRATION_CONTROL2__URGENCY_WEIGHT__SHIFT 0x10
+#define DPGV1_PIPE_ARBITRATION_CONTROL2__TIME_WEIGHT_MASK 0xffff
+#define DPGV1_PIPE_ARBITRATION_CONTROL2__TIME_WEIGHT__SHIFT 0x0
+#define DPGV1_PIPE_ARBITRATION_CONTROL2__URGENCY_WEIGHT_MASK 0xffff0000
+#define DPGV1_PIPE_ARBITRATION_CONTROL2__URGENCY_WEIGHT__SHIFT 0x10
+#define DPGV0_WATERMARK_MASK_CONTROL__STUTTER_EXIT_SELF_REFRESH_WATERMARK_MASK_MASK 0x3
+#define DPGV0_WATERMARK_MASK_CONTROL__STUTTER_EXIT_SELF_REFRESH_WATERMARK_MASK__SHIFT 0x0
+#define DPGV0_WATERMARK_MASK_CONTROL__URGENCY_WATERMARK_MASK_MASK 0x300
+#define DPGV0_WATERMARK_MASK_CONTROL__URGENCY_WATERMARK_MASK__SHIFT 0x8
+#define DPGV0_WATERMARK_MASK_CONTROL__NB_PSTATE_CHANGE_WATERMARK_MASK_MASK 0x30000
+#define DPGV0_WATERMARK_MASK_CONTROL__NB_PSTATE_CHANGE_WATERMARK_MASK__SHIFT 0x10
+#define DPGV0_WATERMARK_MASK_CONTROL__DISABLE_FLIP_URGENT_MASK 0x1000000
+#define DPGV0_WATERMARK_MASK_CONTROL__DISABLE_FLIP_URGENT__SHIFT 0x18
+#define DPGV1_WATERMARK_MASK_CONTROL__STUTTER_EXIT_SELF_REFRESH_WATERMARK_MASK_MASK 0x3
+#define DPGV1_WATERMARK_MASK_CONTROL__STUTTER_EXIT_SELF_REFRESH_WATERMARK_MASK__SHIFT 0x0
+#define DPGV1_WATERMARK_MASK_CONTROL__URGENCY_WATERMARK_MASK_MASK 0x300
+#define DPGV1_WATERMARK_MASK_CONTROL__URGENCY_WATERMARK_MASK__SHIFT 0x8
+#define DPGV1_WATERMARK_MASK_CONTROL__NB_PSTATE_CHANGE_WATERMARK_MASK_MASK 0x30000
+#define DPGV1_WATERMARK_MASK_CONTROL__NB_PSTATE_CHANGE_WATERMARK_MASK__SHIFT 0x10
+#define DPGV1_WATERMARK_MASK_CONTROL__DISABLE_FLIP_URGENT_MASK 0x1000000
+#define DPGV1_WATERMARK_MASK_CONTROL__DISABLE_FLIP_URGENT__SHIFT 0x18
+#define DPGV0_PIPE_URGENCY_CONTROL__URGENCY_LOW_WATERMARK_MASK 0xffff
+#define DPGV0_PIPE_URGENCY_CONTROL__URGENCY_LOW_WATERMARK__SHIFT 0x0
+#define DPGV0_PIPE_URGENCY_CONTROL__URGENCY_HIGH_WATERMARK_MASK 0xffff0000
+#define DPGV0_PIPE_URGENCY_CONTROL__URGENCY_HIGH_WATERMARK__SHIFT 0x10
+#define DPGV1_PIPE_URGENCY_CONTROL__URGENCY_LOW_WATERMARK_MASK 0xffff
+#define DPGV1_PIPE_URGENCY_CONTROL__URGENCY_LOW_WATERMARK__SHIFT 0x0
+#define DPGV1_PIPE_URGENCY_CONTROL__URGENCY_HIGH_WATERMARK_MASK 0xffff0000
+#define DPGV1_PIPE_URGENCY_CONTROL__URGENCY_HIGH_WATERMARK__SHIFT 0x10
+#define DPGV0_PIPE_DPM_CONTROL__DPM_ENABLE_MASK 0x1
+#define DPGV0_PIPE_DPM_CONTROL__DPM_ENABLE__SHIFT 0x0
+#define DPGV0_PIPE_DPM_CONTROL__MCLK_CHANGE_ENABLE_MASK 0x10
+#define DPGV0_PIPE_DPM_CONTROL__MCLK_CHANGE_ENABLE__SHIFT 0x4
+#define DPGV0_PIPE_DPM_CONTROL__MCLK_CHANGE_FORCE_ON_MASK 0x100
+#define DPGV0_PIPE_DPM_CONTROL__MCLK_CHANGE_FORCE_ON__SHIFT 0x8
+#define DPGV0_PIPE_DPM_CONTROL__MCLK_CHANGE_WATERMARK_MASK_MASK 0x3000
+#define DPGV0_PIPE_DPM_CONTROL__MCLK_CHANGE_WATERMARK_MASK__SHIFT 0xc
+#define DPGV0_PIPE_DPM_CONTROL__MCLK_CHANGE_WATERMARK_MASK 0xffff0000
+#define DPGV0_PIPE_DPM_CONTROL__MCLK_CHANGE_WATERMARK__SHIFT 0x10
+#define DPGV1_PIPE_DPM_CONTROL__DPM_ENABLE_MASK 0x1
+#define DPGV1_PIPE_DPM_CONTROL__DPM_ENABLE__SHIFT 0x0
+#define DPGV1_PIPE_DPM_CONTROL__MCLK_CHANGE_ENABLE_MASK 0x10
+#define DPGV1_PIPE_DPM_CONTROL__MCLK_CHANGE_ENABLE__SHIFT 0x4
+#define DPGV1_PIPE_DPM_CONTROL__MCLK_CHANGE_FORCE_ON_MASK 0x100
+#define DPGV1_PIPE_DPM_CONTROL__MCLK_CHANGE_FORCE_ON__SHIFT 0x8
+#define DPGV1_PIPE_DPM_CONTROL__MCLK_CHANGE_WATERMARK_MASK_MASK 0x3000
+#define DPGV1_PIPE_DPM_CONTROL__MCLK_CHANGE_WATERMARK_MASK__SHIFT 0xc
+#define DPGV1_PIPE_DPM_CONTROL__MCLK_CHANGE_WATERMARK_MASK 0xffff0000
+#define DPGV1_PIPE_DPM_CONTROL__MCLK_CHANGE_WATERMARK__SHIFT 0x10
+#define DPGV0_PIPE_STUTTER_CONTROL__STUTTER_ENABLE_MASK 0x1
+#define DPGV0_PIPE_STUTTER_CONTROL__STUTTER_ENABLE__SHIFT 0x0
+#define DPGV0_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_CURSOR_MASK 0x10
+#define DPGV0_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_CURSOR__SHIFT 0x4
+#define DPGV0_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_ICON_MASK 0x20
+#define DPGV0_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_ICON__SHIFT 0x5
+#define DPGV0_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_VGA_MASK 0x40
+#define DPGV0_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_VGA__SHIFT 0x6
+#define DPGV0_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_FBC_MASK 0x80
+#define DPGV0_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_FBC__SHIFT 0x7
+#define DPGV0_PIPE_STUTTER_CONTROL__STUTTER_WM_HIGH_FORCE_ON_MASK 0x100
+#define DPGV0_PIPE_STUTTER_CONTROL__STUTTER_WM_HIGH_FORCE_ON__SHIFT 0x8
+#define DPGV0_PIPE_STUTTER_CONTROL__STUTTER_WM_HIGH_EXCLUDES_VBLANK_MASK 0x200
+#define DPGV0_PIPE_STUTTER_CONTROL__STUTTER_WM_HIGH_EXCLUDES_VBLANK__SHIFT 0x9
+#define DPGV0_PIPE_STUTTER_CONTROL__STUTTER_URGENT_IN_NOT_SELF_REFRESH_MASK 0x400
+#define DPGV0_PIPE_STUTTER_CONTROL__STUTTER_URGENT_IN_NOT_SELF_REFRESH__SHIFT 0xa
+#define DPGV0_PIPE_STUTTER_CONTROL__STUTTER_SELF_REFRESH_FORCE_ON_MASK 0x800
+#define DPGV0_PIPE_STUTTER_CONTROL__STUTTER_SELF_REFRESH_FORCE_ON__SHIFT 0xb
+#define DPGV0_PIPE_STUTTER_CONTROL__STUTTER_EXIT_SELF_REFRESH_WATERMARK_MASK 0xffff0000
+#define DPGV0_PIPE_STUTTER_CONTROL__STUTTER_EXIT_SELF_REFRESH_WATERMARK__SHIFT 0x10
+#define DPGV1_PIPE_STUTTER_CONTROL__STUTTER_ENABLE_MASK 0x1
+#define DPGV1_PIPE_STUTTER_CONTROL__STUTTER_ENABLE__SHIFT 0x0
+#define DPGV1_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_CURSOR_MASK 0x10
+#define DPGV1_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_CURSOR__SHIFT 0x4
+#define DPGV1_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_ICON_MASK 0x20
+#define DPGV1_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_ICON__SHIFT 0x5
+#define DPGV1_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_VGA_MASK 0x40
+#define DPGV1_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_VGA__SHIFT 0x6
+#define DPGV1_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_FBC_MASK 0x80
+#define DPGV1_PIPE_STUTTER_CONTROL__STUTTER_IGNORE_FBC__SHIFT 0x7
+#define DPGV1_PIPE_STUTTER_CONTROL__STUTTER_WM_HIGH_FORCE_ON_MASK 0x100
+#define DPGV1_PIPE_STUTTER_CONTROL__STUTTER_WM_HIGH_FORCE_ON__SHIFT 0x8
+#define DPGV1_PIPE_STUTTER_CONTROL__STUTTER_WM_HIGH_EXCLUDES_VBLANK_MASK 0x200
+#define DPGV1_PIPE_STUTTER_CONTROL__STUTTER_WM_HIGH_EXCLUDES_VBLANK__SHIFT 0x9
+#define DPGV1_PIPE_STUTTER_CONTROL__STUTTER_URGENT_IN_NOT_SELF_REFRESH_MASK 0x400
+#define DPGV1_PIPE_STUTTER_CONTROL__STUTTER_URGENT_IN_NOT_SELF_REFRESH__SHIFT 0xa
+#define DPGV1_PIPE_STUTTER_CONTROL__STUTTER_SELF_REFRESH_FORCE_ON_MASK 0x800
+#define DPGV1_PIPE_STUTTER_CONTROL__STUTTER_SELF_REFRESH_FORCE_ON__SHIFT 0xb
+#define DPGV1_PIPE_STUTTER_CONTROL__STUTTER_EXIT_SELF_REFRESH_WATERMARK_MASK 0xffff0000
+#define DPGV1_PIPE_STUTTER_CONTROL__STUTTER_EXIT_SELF_REFRESH_WATERMARK__SHIFT 0x10
+#define DPGV0_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_ENABLE_MASK 0x1
+#define DPGV0_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_ENABLE__SHIFT 0x0
+#define DPGV0_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_URGENT_DURING_REQUEST_MASK 0x10
+#define DPGV0_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_URGENT_DURING_REQUEST__SHIFT 0x4
+#define DPGV0_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_NOT_SELF_REFRESH_DURING_REQUEST_MASK 0x100
+#define DPGV0_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_NOT_SELF_REFRESH_DURING_REQUEST__SHIFT 0x8
+#define DPGV0_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_FORCE_ON_MASK 0x200
+#define DPGV0_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_FORCE_ON__SHIFT 0x9
+#define DPGV0_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_ALLOW_FOR_URGENT_MASK 0x400
+#define DPGV0_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_ALLOW_FOR_URGENT__SHIFT 0xa
+#define DPGV0_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_WATERMARK_MASK 0xffff0000
+#define DPGV0_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_WATERMARK__SHIFT 0x10
+#define DPGV1_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_ENABLE_MASK 0x1
+#define DPGV1_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_ENABLE__SHIFT 0x0
+#define DPGV1_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_URGENT_DURING_REQUEST_MASK 0x10
+#define DPGV1_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_URGENT_DURING_REQUEST__SHIFT 0x4
+#define DPGV1_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_NOT_SELF_REFRESH_DURING_REQUEST_MASK 0x100
+#define DPGV1_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_NOT_SELF_REFRESH_DURING_REQUEST__SHIFT 0x8
+#define DPGV1_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_FORCE_ON_MASK 0x200
+#define DPGV1_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_FORCE_ON__SHIFT 0x9
+#define DPGV1_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_ALLOW_FOR_URGENT_MASK 0x400
+#define DPGV1_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_ALLOW_FOR_URGENT__SHIFT 0xa
+#define DPGV1_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_WATERMARK_MASK 0xffff0000
+#define DPGV1_PIPE_NB_PSTATE_CHANGE_CONTROL__NB_PSTATE_CHANGE_WATERMARK__SHIFT 0x10
+#define DPGV0_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_ENABLE_NONLPTCH_MASK 0x1
+#define DPGV0_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_ENABLE_NONLPTCH__SHIFT 0x0
+#define DPGV0_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_CURSOR_NONLPTCH_MASK 0x10
+#define DPGV0_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_CURSOR_NONLPTCH__SHIFT 0x4
+#define DPGV0_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_ICON_NONLPTCH_MASK 0x20
+#define DPGV0_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_ICON_NONLPTCH__SHIFT 0x5
+#define DPGV0_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_VGA_NONLPTCH_MASK 0x40
+#define DPGV0_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_VGA_NONLPTCH__SHIFT 0x6
+#define DPGV0_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_FBC_NONLPTCH_MASK 0x80
+#define DPGV0_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_FBC_NONLPTCH__SHIFT 0x7
+#define DPGV0_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_WM_HIGH_FORCE_ON_NONLPTCH_MASK 0x100
+#define DPGV0_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_WM_HIGH_FORCE_ON_NONLPTCH__SHIFT 0x8
+#define DPGV0_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_WM_HIGH_EXCLUDES_VBLANK_NONLPTCH_MASK 0x200
+#define DPGV0_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_WM_HIGH_EXCLUDES_VBLANK_NONLPTCH__SHIFT 0x9
+#define DPGV0_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_URGENT_IN_NOT_SELF_REFRESH_NONLPTCH_MASK 0x400
+#define DPGV0_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_URGENT_IN_NOT_SELF_REFRESH_NONLPTCH__SHIFT 0xa
+#define DPGV0_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_SELF_REFRESH_FORCE_ON_NONLPTCH_MASK 0x800
+#define DPGV0_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_SELF_REFRESH_FORCE_ON_NONLPTCH__SHIFT 0xb
+#define DPGV1_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_ENABLE_NONLPTCH_MASK 0x1
+#define DPGV1_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_ENABLE_NONLPTCH__SHIFT 0x0
+#define DPGV1_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_CURSOR_NONLPTCH_MASK 0x10
+#define DPGV1_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_CURSOR_NONLPTCH__SHIFT 0x4
+#define DPGV1_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_ICON_NONLPTCH_MASK 0x20
+#define DPGV1_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_ICON_NONLPTCH__SHIFT 0x5
+#define DPGV1_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_VGA_NONLPTCH_MASK 0x40
+#define DPGV1_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_VGA_NONLPTCH__SHIFT 0x6
+#define DPGV1_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_FBC_NONLPTCH_MASK 0x80
+#define DPGV1_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_IGNORE_FBC_NONLPTCH__SHIFT 0x7
+#define DPGV1_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_WM_HIGH_FORCE_ON_NONLPTCH_MASK 0x100
+#define DPGV1_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_WM_HIGH_FORCE_ON_NONLPTCH__SHIFT 0x8
+#define DPGV1_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_WM_HIGH_EXCLUDES_VBLANK_NONLPTCH_MASK 0x200
+#define DPGV1_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_WM_HIGH_EXCLUDES_VBLANK_NONLPTCH__SHIFT 0x9
+#define DPGV1_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_URGENT_IN_NOT_SELF_REFRESH_NONLPTCH_MASK 0x400
+#define DPGV1_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_URGENT_IN_NOT_SELF_REFRESH_NONLPTCH__SHIFT 0xa
+#define DPGV1_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_SELF_REFRESH_FORCE_ON_NONLPTCH_MASK 0x800
+#define DPGV1_PIPE_STUTTER_CONTROL_NONLPTCH__STUTTER_SELF_REFRESH_FORCE_ON_NONLPTCH__SHIFT 0xb
+#define DPGV0_REPEATER_PROGRAM__REG_DPG_DMIFRC_REPEATER_MASK 0x7
+#define DPGV0_REPEATER_PROGRAM__REG_DPG_DMIFRC_REPEATER__SHIFT 0x0
+#define DPGV0_REPEATER_PROGRAM__REG_DMIFRC_DPG_REPEATER_MASK 0x70
+#define DPGV0_REPEATER_PROGRAM__REG_DMIFRC_DPG_REPEATER__SHIFT 0x4
+#define DPGV1_REPEATER_PROGRAM__REG_DPG_DMIFRC_REPEATER_MASK 0x7
+#define DPGV1_REPEATER_PROGRAM__REG_DPG_DMIFRC_REPEATER__SHIFT 0x0
+#define DPGV1_REPEATER_PROGRAM__REG_DMIFRC_DPG_REPEATER_MASK 0x70
+#define DPGV1_REPEATER_PROGRAM__REG_DMIFRC_DPG_REPEATER__SHIFT 0x4
+#define DPGV0_HW_DEBUG_A__DPG_HW_DEBUG_A_MASK 0xffffffff
+#define DPGV0_HW_DEBUG_A__DPG_HW_DEBUG_A__SHIFT 0x0
+#define DPGV1_HW_DEBUG_A__DPG_HW_DEBUG_A_MASK 0xffffffff
+#define DPGV1_HW_DEBUG_A__DPG_HW_DEBUG_A__SHIFT 0x0
+#define DPGV0_HW_DEBUG_B__DPG_HW_DEBUG_B_MASK 0xffffffff
+#define DPGV0_HW_DEBUG_B__DPG_HW_DEBUG_B__SHIFT 0x0
+#define DPGV1_HW_DEBUG_B__DPG_HW_DEBUG_B_MASK 0xffffffff
+#define DPGV1_HW_DEBUG_B__DPG_HW_DEBUG_B__SHIFT 0x0
+#define DPGV0_HW_DEBUG_11__DPG_HW_DEBUG_11_MASK 0x1
+#define DPGV0_HW_DEBUG_11__DPG_HW_DEBUG_11__SHIFT 0x0
+#define DPGV1_HW_DEBUG_11__DPG_HW_DEBUG_11_MASK 0x1
+#define DPGV1_HW_DEBUG_11__DPG_HW_DEBUG_11__SHIFT 0x0
+#define DPGV0_CHK_PRE_PROC_CNTL__DPG_DISABLE_DMIF_BUF_CHK_MASK 0x1
+#define DPGV0_CHK_PRE_PROC_CNTL__DPG_DISABLE_DMIF_BUF_CHK__SHIFT 0x0
+#define DPGV1_CHK_PRE_PROC_CNTL__DPG_DISABLE_DMIF_BUF_CHK_MASK 0x1
+#define DPGV1_CHK_PRE_PROC_CNTL__DPG_DISABLE_DMIF_BUF_CHK__SHIFT 0x0
+#define DPGV_TEST_DEBUG_INDEX__DPG_TEST_DEBUG_INDEX_MASK 0xff
+#define DPGV_TEST_DEBUG_INDEX__DPG_TEST_DEBUG_INDEX__SHIFT 0x0
+#define DPGV_TEST_DEBUG_INDEX__DPG_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define DPGV_TEST_DEBUG_INDEX__DPG_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define DPGV_TEST_DEBUG_DATA__DPG_TEST_DEBUG_DATA_MASK 0xffffffff
+#define DPGV_TEST_DEBUG_DATA__DPG_TEST_DEBUG_DATA__SHIFT 0x0
+#define AZROOT_IMMEDIATE_COMMAND_OUTPUT_INTERFACE_INDEX__IMMEDIATE_COMMAND_WRITE_MASK 0x1ffff
+#define AZROOT_IMMEDIATE_COMMAND_OUTPUT_INTERFACE_INDEX__IMMEDIATE_COMMAND_WRITE__SHIFT 0x0
+#define AZROOT_IMMEDIATE_COMMAND_OUTPUT_INTERFACE_DATA__IMMEDIATE_COMMAND_WRITE_MASK 0xffffffff
+#define AZROOT_IMMEDIATE_COMMAND_OUTPUT_INTERFACE_DATA__IMMEDIATE_COMMAND_WRITE__SHIFT 0x0
+#define AZALIA_F2_CODEC_ROOT_PARAMETER_VENDOR_AND_DEVICE_ID__AZALIA_CODEC_ROOT_PARAMETER_VENDOR_AND_DEVICE_ID_MASK 0xffffffff
+#define AZALIA_F2_CODEC_ROOT_PARAMETER_VENDOR_AND_DEVICE_ID__AZALIA_CODEC_ROOT_PARAMETER_VENDOR_AND_DEVICE_ID__SHIFT 0x0
+#define AZALIA_F2_CODEC_ROOT_PARAMETER_REVISION_ID__AZALIA_CODEC_ROOT_PARAMETER_REVISION_ID_MASK 0xffffffff
+#define AZALIA_F2_CODEC_ROOT_PARAMETER_REVISION_ID__AZALIA_CODEC_ROOT_PARAMETER_REVISION_ID__SHIFT 0x0
+#define AZALIA_F2_CODEC_ROOT_PARAMETER_SUBORDINATE_NODE_COUNT__AZALIA_CODEC_ROOT_PARAMETER_SUBORDINATE_NODE_COUNT_MASK 0xffffffff
+#define AZALIA_F2_CODEC_ROOT_PARAMETER_SUBORDINATE_NODE_COUNT__AZALIA_CODEC_ROOT_PARAMETER_SUBORDINATE_NODE_COUNT__SHIFT 0x0
+#define AZALIA_F2_CODEC_FUNCTION_PARAMETER_SUBORDINATE_NODE_COUNT__AZALIA_CODEC_FUNCTION_PARAMETER_SUBORDINATE_NODE_COUNT_MASK 0xffffffff
+#define AZALIA_F2_CODEC_FUNCTION_PARAMETER_SUBORDINATE_NODE_COUNT__AZALIA_CODEC_FUNCTION_PARAMETER_SUBORDINATE_NODE_COUNT__SHIFT 0x0
+#define AZALIA_F2_CODEC_FUNCTION_PARAMETER_GROUP_TYPE__AZALIA_CODEC_FUNCTION_PARAMETER_GROUP_TYPE_MASK 0xffffffff
+#define AZALIA_F2_CODEC_FUNCTION_PARAMETER_GROUP_TYPE__AZALIA_CODEC_FUNCTION_PARAMETER_GROUP_TYPE__SHIFT 0x0
+#define AZALIA_F2_CODEC_FUNCTION_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_RATE_CAPABILITIES_MASK 0xfff
+#define AZALIA_F2_CODEC_FUNCTION_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_RATE_CAPABILITIES__SHIFT 0x0
+#define AZALIA_F2_CODEC_FUNCTION_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_BIT_CAPABILITIES_MASK 0x1f0000
+#define AZALIA_F2_CODEC_FUNCTION_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_BIT_CAPABILITIES__SHIFT 0x10
+#define AZALIA_F2_CODEC_FUNCTION_PARAMETER_STREAM_FORMATS__AZALIA_CODEC_FUNCTION_PARAMETER_STREAM_FORMATS_MASK 0xffffffff
+#define AZALIA_F2_CODEC_FUNCTION_PARAMETER_STREAM_FORMATS__AZALIA_CODEC_FUNCTION_PARAMETER_STREAM_FORMATS__SHIFT 0x0
+#define AZALIA_F2_CODEC_FUNCTION_PARAMETER_POWER_STATES__AZALIA_CODEC_FUNCTION_PARAMETER_POWER_STATES_MASK 0x3fffffff
+#define AZALIA_F2_CODEC_FUNCTION_PARAMETER_POWER_STATES__AZALIA_CODEC_FUNCTION_PARAMETER_POWER_STATES__SHIFT 0x0
+#define AZALIA_F2_CODEC_FUNCTION_PARAMETER_POWER_STATES__CLKSTOP_MASK 0x40000000
+#define AZALIA_F2_CODEC_FUNCTION_PARAMETER_POWER_STATES__CLKSTOP__SHIFT 0x1e
+#define AZALIA_F2_CODEC_FUNCTION_PARAMETER_POWER_STATES__EPSS_MASK 0x80000000
+#define AZALIA_F2_CODEC_FUNCTION_PARAMETER_POWER_STATES__EPSS__SHIFT 0x1f
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_POWER_STATE__POWER_STATE_SET_MASK 0xf
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_POWER_STATE__POWER_STATE_SET__SHIFT 0x0
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_POWER_STATE__POWER_STATE_ACT_MASK 0xf0
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_POWER_STATE__POWER_STATE_ACT__SHIFT 0x4
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_POWER_STATE__CLKSTOPOK_MASK 0x200
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_POWER_STATE__CLKSTOPOK__SHIFT 0x9
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_POWER_STATE__POWER_STATE_SETTINGS_RESET_MASK 0x400
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_POWER_STATE__POWER_STATE_SETTINGS_RESET__SHIFT 0xa
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_RESET__CODEC_RESET_MASK 0x1
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_RESET__CODEC_RESET__SHIFT 0x0
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID__SUBSYSTEM_ID_BYTE0_MASK 0xff
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID__SUBSYSTEM_ID_BYTE0__SHIFT 0x0
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID__SUBSYSTEM_ID_BYTE1_MASK 0xff00
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID__SUBSYSTEM_ID_BYTE1__SHIFT 0x8
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID__SUBSYSTEM_ID_BYTE2_MASK 0xff0000
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID__SUBSYSTEM_ID_BYTE2__SHIFT 0x10
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID__SUBSYSTEM_ID_BYTE3_MASK 0xff000000
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID__SUBSYSTEM_ID_BYTE3__SHIFT 0x18
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID_2__SUBSYSTEM_ID_BYTE1_MASK 0xff
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID_2__SUBSYSTEM_ID_BYTE1__SHIFT 0x0
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID_3__SUBSYSTEM_ID_BYTE2_MASK 0xff
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID_3__SUBSYSTEM_ID_BYTE2__SHIFT 0x0
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID_4__SUBSYSTEM_ID_BYTE3_MASK 0xff
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID_4__SUBSYSTEM_ID_BYTE3__SHIFT 0x0
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_CONVERTER_SYNCHRONIZATION__CONVERTER_SYNCHRONIZATION_MASK 0x7f
+#define AZALIA_F2_CODEC_FUNCTION_CONTROL_CONVERTER_SYNCHRONIZATION__CONVERTER_SYNCHRONIZATION__SHIFT 0x0
+#define AZALIA_F0_CODEC_ROOT_PARAMETER_VENDOR_AND_DEVICE_ID__AZALIA_CODEC_ROOT_PARAMETER_VENDOR_AND_DEVICE_ID_MASK 0xffffffff
+#define AZALIA_F0_CODEC_ROOT_PARAMETER_VENDOR_AND_DEVICE_ID__AZALIA_CODEC_ROOT_PARAMETER_VENDOR_AND_DEVICE_ID__SHIFT 0x0
+#define AZALIA_F0_CODEC_ROOT_PARAMETER_REVISION_ID__AZALIA_CODEC_ROOT_PARAMETER_REVISION_ID_MASK 0xffffffff
+#define AZALIA_F0_CODEC_ROOT_PARAMETER_REVISION_ID__AZALIA_CODEC_ROOT_PARAMETER_REVISION_ID__SHIFT 0x0
+#define AZALIA_F0_CODEC_CHANNEL_COUNT_CONTROL__HBR_CHANNEL_COUNT_MASK 0x7
+#define AZALIA_F0_CODEC_CHANNEL_COUNT_CONTROL__HBR_CHANNEL_COUNT__SHIFT 0x0
+#define AZALIA_F0_CODEC_CHANNEL_COUNT_CONTROL__COMPRESSED_CHANNEL_COUNT_MASK 0x70
+#define AZALIA_F0_CODEC_CHANNEL_COUNT_CONTROL__COMPRESSED_CHANNEL_COUNT__SHIFT 0x4
+#define AZALIA_F0_CODEC_RESYNC_FIFO_CONTROL__RESYNC_FIFO_STARTUP_KEEPOUT_WINDOW_MASK 0x3f
+#define AZALIA_F0_CODEC_RESYNC_FIFO_CONTROL__RESYNC_FIFO_STARTUP_KEEPOUT_WINDOW__SHIFT 0x0
+#define AZALIA_F0_CODEC_FUNCTION_PARAMETER_GROUP_TYPE__AZALIA_CODEC_FUNCTION_PARAMETER_GROUP_TYPE_MASK 0xffffffff
+#define AZALIA_F0_CODEC_FUNCTION_PARAMETER_GROUP_TYPE__AZALIA_CODEC_FUNCTION_PARAMETER_GROUP_TYPE__SHIFT 0x0
+#define AZALIA_F0_CODEC_FUNCTION_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_RATE_CAPABILITIES_MASK 0xfff
+#define AZALIA_F0_CODEC_FUNCTION_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_RATE_CAPABILITIES__SHIFT 0x0
+#define AZALIA_F0_CODEC_FUNCTION_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_BIT_CAPABILITIES_MASK 0x1f0000
+#define AZALIA_F0_CODEC_FUNCTION_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_BIT_CAPABILITIES__SHIFT 0x10
+#define AZALIA_F0_CODEC_FUNCTION_PARAMETER_STREAM_FORMATS__AZALIA_CODEC_FUNCTION_PARAMETER_STREAM_FORMATS_MASK 0xffffffff
+#define AZALIA_F0_CODEC_FUNCTION_PARAMETER_STREAM_FORMATS__AZALIA_CODEC_FUNCTION_PARAMETER_STREAM_FORMATS__SHIFT 0x0
+#define AZALIA_F0_CODEC_FUNCTION_PARAMETER_POWER_STATES__AZALIA_CODEC_FUNCTION_PARAMETER_POWER_STATES_MASK 0x3fffffff
+#define AZALIA_F0_CODEC_FUNCTION_PARAMETER_POWER_STATES__AZALIA_CODEC_FUNCTION_PARAMETER_POWER_STATES__SHIFT 0x0
+#define AZALIA_F0_CODEC_FUNCTION_PARAMETER_POWER_STATES__CLKSTOP_MASK 0x40000000
+#define AZALIA_F0_CODEC_FUNCTION_PARAMETER_POWER_STATES__CLKSTOP__SHIFT 0x1e
+#define AZALIA_F0_CODEC_FUNCTION_PARAMETER_POWER_STATES__EPSS_MASK 0x80000000
+#define AZALIA_F0_CODEC_FUNCTION_PARAMETER_POWER_STATES__EPSS__SHIFT 0x1f
+#define AZALIA_F0_CODEC_FUNCTION_CONTROL_POWER_STATE__POWER_STATE_SET_MASK 0xf
+#define AZALIA_F0_CODEC_FUNCTION_CONTROL_POWER_STATE__POWER_STATE_SET__SHIFT 0x0
+#define AZALIA_F0_CODEC_FUNCTION_CONTROL_POWER_STATE__POWER_STATE_ACT_MASK 0xf0
+#define AZALIA_F0_CODEC_FUNCTION_CONTROL_POWER_STATE__POWER_STATE_ACT__SHIFT 0x4
+#define AZALIA_F0_CODEC_FUNCTION_CONTROL_POWER_STATE__CLKSTOPOK_MASK 0x200
+#define AZALIA_F0_CODEC_FUNCTION_CONTROL_POWER_STATE__CLKSTOPOK__SHIFT 0x9
+#define AZALIA_F0_CODEC_FUNCTION_CONTROL_POWER_STATE__POWER_STATE_SETTINGS_RESET_MASK 0x400
+#define AZALIA_F0_CODEC_FUNCTION_CONTROL_POWER_STATE__POWER_STATE_SETTINGS_RESET__SHIFT 0xa
+#define AZALIA_F0_CODEC_FUNCTION_CONTROL_RESET__CODEC_RESET_MASK 0x1
+#define AZALIA_F0_CODEC_FUNCTION_CONTROL_RESET__CODEC_RESET__SHIFT 0x0
+#define AZALIA_F0_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID__SUBSYSTEM_ID_BYTE0_MASK 0xff
+#define AZALIA_F0_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID__SUBSYSTEM_ID_BYTE0__SHIFT 0x0
+#define AZALIA_F0_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID__SUBSYSTEM_ID_BYTE1_MASK 0xff00
+#define AZALIA_F0_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID__SUBSYSTEM_ID_BYTE1__SHIFT 0x8
+#define AZALIA_F0_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID__SUBSYSTEM_ID_BYTE2_MASK 0xff0000
+#define AZALIA_F0_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID__SUBSYSTEM_ID_BYTE2__SHIFT 0x10
+#define AZALIA_F0_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID__SUBSYSTEM_ID_BYTE3_MASK 0xff000000
+#define AZALIA_F0_CODEC_FUNCTION_CONTROL_RESPONSE_SUBSYSTEM_ID__SUBSYSTEM_ID_BYTE3__SHIFT 0x18
+#define AZALIA_F0_CODEC_FUNCTION_CONTROL_CONVERTER_SYNCHRONIZATION__CONVERTER_SYNCHRONIZATION_MASK 0x7f
+#define AZALIA_F0_CODEC_FUNCTION_CONTROL_CONVERTER_SYNCHRONIZATION__CONVERTER_SYNCHRONIZATION__SHIFT 0x0
+#define CC_RCU_DC_AUDIO_PORT_CONNECTIVITY__PORT_CONNECTIVITY_MASK 0x7
+#define CC_RCU_DC_AUDIO_PORT_CONNECTIVITY__PORT_CONNECTIVITY__SHIFT 0x0
+#define CC_RCU_DC_AUDIO_PORT_CONNECTIVITY__PORT_CONNECTIVITY_OVERRIDE_ENABLE_MASK 0x10
+#define CC_RCU_DC_AUDIO_PORT_CONNECTIVITY__PORT_CONNECTIVITY_OVERRIDE_ENABLE__SHIFT 0x4
+#define CC_RCU_DC_AUDIO_INPUT_PORT_CONNECTIVITY__INPUT_PORT_CONNECTIVITY_MASK 0x7
+#define CC_RCU_DC_AUDIO_INPUT_PORT_CONNECTIVITY__INPUT_PORT_CONNECTIVITY__SHIFT 0x0
+#define CC_RCU_DC_AUDIO_INPUT_PORT_CONNECTIVITY__INPUT_PORT_CONNECTIVITY_OVERRIDE_ENABLE_MASK 0x10
+#define CC_RCU_DC_AUDIO_INPUT_PORT_CONNECTIVITY__INPUT_PORT_CONNECTIVITY_OVERRIDE_ENABLE__SHIFT 0x4
+#define AZALIA_F0_CODEC_DEBUG__DISABLE_FORMAT_COMPARISON_MASK 0x3f
+#define AZALIA_F0_CODEC_DEBUG__DISABLE_FORMAT_COMPARISON__SHIFT 0x0
+#define AZALIA_F0_CODEC_DEBUG__CODEC_DEBUG_MASK 0xffffffc0
+#define AZALIA_F0_CODEC_DEBUG__CODEC_DEBUG__SHIFT 0x6
+#define AZALIA_F0_GTC_GROUP_OFFSET0__GTC_GROUP_OFFSET0_MASK 0xffffffff
+#define AZALIA_F0_GTC_GROUP_OFFSET0__GTC_GROUP_OFFSET0__SHIFT 0x0
+#define AZALIA_F0_GTC_GROUP_OFFSET1__GTC_GROUP_OFFSET1_MASK 0xffffffff
+#define AZALIA_F0_GTC_GROUP_OFFSET1__GTC_GROUP_OFFSET1__SHIFT 0x0
+#define AZALIA_F0_GTC_GROUP_OFFSET2__GTC_GROUP_OFFSET2_MASK 0xffffffff
+#define AZALIA_F0_GTC_GROUP_OFFSET2__GTC_GROUP_OFFSET2__SHIFT 0x0
+#define AZALIA_F0_GTC_GROUP_OFFSET3__GTC_GROUP_OFFSET3_MASK 0xffffffff
+#define AZALIA_F0_GTC_GROUP_OFFSET3__GTC_GROUP_OFFSET3__SHIFT 0x0
+#define AZALIA_F0_GTC_GROUP_OFFSET4__GTC_GROUP_OFFSET4_MASK 0xffffffff
+#define AZALIA_F0_GTC_GROUP_OFFSET4__GTC_GROUP_OFFSET4__SHIFT 0x0
+#define AZALIA_F0_GTC_GROUP_OFFSET5__GTC_GROUP_OFFSET5_MASK 0xffffffff
+#define AZALIA_F0_GTC_GROUP_OFFSET5__GTC_GROUP_OFFSET5__SHIFT 0x0
+#define AZALIA_F0_GTC_GROUP_OFFSET6__GTC_GROUP_OFFSET6_MASK 0xffffffff
+#define AZALIA_F0_GTC_GROUP_OFFSET6__GTC_GROUP_OFFSET6__SHIFT 0x0
+#define GLOBAL_CAPABILITIES__SIXTY_FOUR_BIT_ADDRESS_SUPPORTED_MASK 0x1
+#define GLOBAL_CAPABILITIES__SIXTY_FOUR_BIT_ADDRESS_SUPPORTED__SHIFT 0x0
+#define GLOBAL_CAPABILITIES__NUMBER_OF_SERIAL_DATA_OUTPUT_SIGNALS_MASK 0x6
+#define GLOBAL_CAPABILITIES__NUMBER_OF_SERIAL_DATA_OUTPUT_SIGNALS__SHIFT 0x1
+#define GLOBAL_CAPABILITIES__NUMBER_OF_BIDIRECTIONAL_STREAMS_SUPPORTED_MASK 0xf8
+#define GLOBAL_CAPABILITIES__NUMBER_OF_BIDIRECTIONAL_STREAMS_SUPPORTED__SHIFT 0x3
+#define GLOBAL_CAPABILITIES__NUMBER_OF_INPUT_STREAMS_SUPPORTED_MASK 0xf00
+#define GLOBAL_CAPABILITIES__NUMBER_OF_INPUT_STREAMS_SUPPORTED__SHIFT 0x8
+#define GLOBAL_CAPABILITIES__NUMBER_OF_OUTPUT_STREAMS_SUPPORTED_MASK 0xf000
+#define GLOBAL_CAPABILITIES__NUMBER_OF_OUTPUT_STREAMS_SUPPORTED__SHIFT 0xc
+#define MINOR_VERSION__MINOR_VERSION_MASK 0xff
+#define MINOR_VERSION__MINOR_VERSION__SHIFT 0x0
+#define MAJOR_VERSION__MAJOR_VERSION_MASK 0xff
+#define MAJOR_VERSION__MAJOR_VERSION__SHIFT 0x0
+#define OUTPUT_PAYLOAD_CAPABILITY__OUTPUT_PAYLOAD_CAPABILITY_MASK 0xffff
+#define OUTPUT_PAYLOAD_CAPABILITY__OUTPUT_PAYLOAD_CAPABILITY__SHIFT 0x0
+#define INPUT_PAYLOAD_CAPABILITY__INPUT_PAYLOAD_CAPABILITY_MASK 0xffff
+#define INPUT_PAYLOAD_CAPABILITY__INPUT_PAYLOAD_CAPABILITY__SHIFT 0x0
+#define GLOBAL_CONTROL__CONTROLLER_RESET_MASK 0x1
+#define GLOBAL_CONTROL__CONTROLLER_RESET__SHIFT 0x0
+#define GLOBAL_CONTROL__FLUSH_CONTROL_MASK 0x2
+#define GLOBAL_CONTROL__FLUSH_CONTROL__SHIFT 0x1
+#define GLOBAL_CONTROL__ACCEPT_UNSOLICITED_RESPONSE_ENABLE_MASK 0x100
+#define GLOBAL_CONTROL__ACCEPT_UNSOLICITED_RESPONSE_ENABLE__SHIFT 0x8
+#define WAKE_ENABLE__SDIN_WAKE_ENABLE_FLAG_MASK 0x1
+#define WAKE_ENABLE__SDIN_WAKE_ENABLE_FLAG__SHIFT 0x0
+#define STATE_CHANGE_STATUS__STATE_CHANGE_STATUS_MASK 0x1
+#define STATE_CHANGE_STATUS__STATE_CHANGE_STATUS__SHIFT 0x0
+#define GLOBAL_STATUS__FLUSH_STATUS_MASK 0x2
+#define GLOBAL_STATUS__FLUSH_STATUS__SHIFT 0x1
+#define OUTPUT_STREAM_PAYLOAD_CAPABILITY__OUTSTRMPAY_MASK 0xffff
+#define OUTPUT_STREAM_PAYLOAD_CAPABILITY__OUTSTRMPAY__SHIFT 0x0
+#define INPUT_STREAM_PAYLOAD_CAPABILITY__INSTRMPAY_MASK 0xffff
+#define INPUT_STREAM_PAYLOAD_CAPABILITY__INSTRMPAY__SHIFT 0x0
+#define INTERRUPT_CONTROL__STREAM_0_INTERRUPT_ENABLE_MASK 0x1
+#define INTERRUPT_CONTROL__STREAM_0_INTERRUPT_ENABLE__SHIFT 0x0
+#define INTERRUPT_CONTROL__STREAM_1_INTERRUPT_ENABLE_MASK 0x2
+#define INTERRUPT_CONTROL__STREAM_1_INTERRUPT_ENABLE__SHIFT 0x1
+#define INTERRUPT_CONTROL__STREAM_2_INTERRUPT_ENABLE_MASK 0x4
+#define INTERRUPT_CONTROL__STREAM_2_INTERRUPT_ENABLE__SHIFT 0x2
+#define INTERRUPT_CONTROL__STREAM_3_INTERRUPT_ENABLE_MASK 0x8
+#define INTERRUPT_CONTROL__STREAM_3_INTERRUPT_ENABLE__SHIFT 0x3
+#define INTERRUPT_CONTROL__STREAM_4_INTERRUPT_ENABLE_MASK 0x10
+#define INTERRUPT_CONTROL__STREAM_4_INTERRUPT_ENABLE__SHIFT 0x4
+#define INTERRUPT_CONTROL__STREAM_5_INTERRUPT_ENABLE_MASK 0x20
+#define INTERRUPT_CONTROL__STREAM_5_INTERRUPT_ENABLE__SHIFT 0x5
+#define INTERRUPT_CONTROL__STREAM_6_INTERRUPT_ENABLE_MASK 0x40
+#define INTERRUPT_CONTROL__STREAM_6_INTERRUPT_ENABLE__SHIFT 0x6
+#define INTERRUPT_CONTROL__STREAM_7_INTERRUPT_ENABLE_MASK 0x80
+#define INTERRUPT_CONTROL__STREAM_7_INTERRUPT_ENABLE__SHIFT 0x7
+#define INTERRUPT_CONTROL__STREAM_8_INTERRUPT_ENABLE_MASK 0x100
+#define INTERRUPT_CONTROL__STREAM_8_INTERRUPT_ENABLE__SHIFT 0x8
+#define INTERRUPT_CONTROL__STREAM_9_INTERRUPT_ENABLE_MASK 0x200
+#define INTERRUPT_CONTROL__STREAM_9_INTERRUPT_ENABLE__SHIFT 0x9
+#define INTERRUPT_CONTROL__STREAM_10_INTERRUPT_ENABLE_MASK 0x400
+#define INTERRUPT_CONTROL__STREAM_10_INTERRUPT_ENABLE__SHIFT 0xa
+#define INTERRUPT_CONTROL__STREAM_11_INTERRUPT_ENABLE_MASK 0x800
+#define INTERRUPT_CONTROL__STREAM_11_INTERRUPT_ENABLE__SHIFT 0xb
+#define INTERRUPT_CONTROL__STREAM_12_INTERRUPT_ENABLE_MASK 0x1000
+#define INTERRUPT_CONTROL__STREAM_12_INTERRUPT_ENABLE__SHIFT 0xc
+#define INTERRUPT_CONTROL__STREAM_13_INTERRUPT_ENABLE_MASK 0x2000
+#define INTERRUPT_CONTROL__STREAM_13_INTERRUPT_ENABLE__SHIFT 0xd
+#define INTERRUPT_CONTROL__STREAM_14_INTERRUPT_ENABLE_MASK 0x4000
+#define INTERRUPT_CONTROL__STREAM_14_INTERRUPT_ENABLE__SHIFT 0xe
+#define INTERRUPT_CONTROL__STREAM_15_INTERRUPT_ENABLE_MASK 0x8000
+#define INTERRUPT_CONTROL__STREAM_15_INTERRUPT_ENABLE__SHIFT 0xf
+#define INTERRUPT_CONTROL__CONTROLLER_INTERRUPT_ENABLE_MASK 0x40000000
+#define INTERRUPT_CONTROL__CONTROLLER_INTERRUPT_ENABLE__SHIFT 0x1e
+#define INTERRUPT_CONTROL__GLOBAL_INTERRUPT_ENABLE_MASK 0x80000000
+#define INTERRUPT_CONTROL__GLOBAL_INTERRUPT_ENABLE__SHIFT 0x1f
+#define INTERRUPT_STATUS__STREAM_0_INTERRUPT_STATUS_MASK 0x1
+#define INTERRUPT_STATUS__STREAM_0_INTERRUPT_STATUS__SHIFT 0x0
+#define INTERRUPT_STATUS__STREAM_1_INTERRUPT_STATUS_MASK 0x2
+#define INTERRUPT_STATUS__STREAM_1_INTERRUPT_STATUS__SHIFT 0x1
+#define INTERRUPT_STATUS__STREAM_2_INTERRUPT_STATUS_MASK 0x4
+#define INTERRUPT_STATUS__STREAM_2_INTERRUPT_STATUS__SHIFT 0x2
+#define INTERRUPT_STATUS__STREAM_3_INTERRUPT_STATUS_MASK 0x8
+#define INTERRUPT_STATUS__STREAM_3_INTERRUPT_STATUS__SHIFT 0x3
+#define INTERRUPT_STATUS__STREAM_4_INTERRUPT_STATUS_MASK 0x10
+#define INTERRUPT_STATUS__STREAM_4_INTERRUPT_STATUS__SHIFT 0x4
+#define INTERRUPT_STATUS__STREAM_5_INTERRUPT_STATUS_MASK 0x20
+#define INTERRUPT_STATUS__STREAM_5_INTERRUPT_STATUS__SHIFT 0x5
+#define INTERRUPT_STATUS__STREAM_6_INTERRUPT_STATUS_MASK 0x40
+#define INTERRUPT_STATUS__STREAM_6_INTERRUPT_STATUS__SHIFT 0x6
+#define INTERRUPT_STATUS__STREAM_7_INTERRUPT_STATUS_MASK 0x80
+#define INTERRUPT_STATUS__STREAM_7_INTERRUPT_STATUS__SHIFT 0x7
+#define INTERRUPT_STATUS__STREAM_8_INTERRUPT_STATUS_MASK 0x100
+#define INTERRUPT_STATUS__STREAM_8_INTERRUPT_STATUS__SHIFT 0x8
+#define INTERRUPT_STATUS__STREAM_9_INTERRUPT_STATUS_MASK 0x200
+#define INTERRUPT_STATUS__STREAM_9_INTERRUPT_STATUS__SHIFT 0x9
+#define INTERRUPT_STATUS__STREAM_10_INTERRUPT_STATUS_MASK 0x400
+#define INTERRUPT_STATUS__STREAM_10_INTERRUPT_STATUS__SHIFT 0xa
+#define INTERRUPT_STATUS__STREAM_11_INTERRUPT_STATUS_MASK 0x800
+#define INTERRUPT_STATUS__STREAM_11_INTERRUPT_STATUS__SHIFT 0xb
+#define INTERRUPT_STATUS__STREAM_12_INTERRUPT_STATUS_MASK 0x1000
+#define INTERRUPT_STATUS__STREAM_12_INTERRUPT_STATUS__SHIFT 0xc
+#define INTERRUPT_STATUS__STREAM_13_INTERRUPT_STATUS_MASK 0x2000
+#define INTERRUPT_STATUS__STREAM_13_INTERRUPT_STATUS__SHIFT 0xd
+#define INTERRUPT_STATUS__STREAM_14_INTERRUPT_STATUS_MASK 0x4000
+#define INTERRUPT_STATUS__STREAM_14_INTERRUPT_STATUS__SHIFT 0xe
+#define INTERRUPT_STATUS__STREAM_15_INTERRUPT_STATUS_MASK 0x8000
+#define INTERRUPT_STATUS__STREAM_15_INTERRUPT_STATUS__SHIFT 0xf
+#define INTERRUPT_STATUS__CONTROLLER_INTERRUPT_STATUS_MASK 0x40000000
+#define INTERRUPT_STATUS__CONTROLLER_INTERRUPT_STATUS__SHIFT 0x1e
+#define INTERRUPT_STATUS__GLOBAL_INTERRUPT_STATUS_MASK 0x80000000
+#define INTERRUPT_STATUS__GLOBAL_INTERRUPT_STATUS__SHIFT 0x1f
+#define WALL_CLOCK_COUNTER__WALL_CLOCK_COUNTER_MASK 0xffffffff
+#define WALL_CLOCK_COUNTER__WALL_CLOCK_COUNTER__SHIFT 0x0
+#define STREAM_SYNCHRONIZATION__STREAM_0_SYNCHRONIZATION_MASK 0x1
+#define STREAM_SYNCHRONIZATION__STREAM_0_SYNCHRONIZATION__SHIFT 0x0
+#define STREAM_SYNCHRONIZATION__STREAM_1_SYNCHRONIZATION_MASK 0x2
+#define STREAM_SYNCHRONIZATION__STREAM_1_SYNCHRONIZATION__SHIFT 0x1
+#define STREAM_SYNCHRONIZATION__STREAM_2_SYNCHRONIZATION_MASK 0x4
+#define STREAM_SYNCHRONIZATION__STREAM_2_SYNCHRONIZATION__SHIFT 0x2
+#define STREAM_SYNCHRONIZATION__STREAM_3_SYNCHRONIZATION_MASK 0x8
+#define STREAM_SYNCHRONIZATION__STREAM_3_SYNCHRONIZATION__SHIFT 0x3
+#define STREAM_SYNCHRONIZATION__STREAM_4_SYNCHRONIZATION_MASK 0x10
+#define STREAM_SYNCHRONIZATION__STREAM_4_SYNCHRONIZATION__SHIFT 0x4
+#define STREAM_SYNCHRONIZATION__STREAM_5_SYNCHRONIZATION_MASK 0x20
+#define STREAM_SYNCHRONIZATION__STREAM_5_SYNCHRONIZATION__SHIFT 0x5
+#define STREAM_SYNCHRONIZATION__STREAM_6_SYNCHRONIZATION_MASK 0x40
+#define STREAM_SYNCHRONIZATION__STREAM_6_SYNCHRONIZATION__SHIFT 0x6
+#define STREAM_SYNCHRONIZATION__STREAM_7_SYNCHRONIZATION_MASK 0x80
+#define STREAM_SYNCHRONIZATION__STREAM_7_SYNCHRONIZATION__SHIFT 0x7
+#define STREAM_SYNCHRONIZATION__STREAM_8_SYNCHRONIZATION_MASK 0x100
+#define STREAM_SYNCHRONIZATION__STREAM_8_SYNCHRONIZATION__SHIFT 0x8
+#define STREAM_SYNCHRONIZATION__STREAM_9_SYNCHRONIZATION_MASK 0x200
+#define STREAM_SYNCHRONIZATION__STREAM_9_SYNCHRONIZATION__SHIFT 0x9
+#define STREAM_SYNCHRONIZATION__STREAM_10_SYNCHRONIZATION_MASK 0x400
+#define STREAM_SYNCHRONIZATION__STREAM_10_SYNCHRONIZATION__SHIFT 0xa
+#define STREAM_SYNCHRONIZATION__STREAM_11_SYNCHRONIZATION_MASK 0x800
+#define STREAM_SYNCHRONIZATION__STREAM_11_SYNCHRONIZATION__SHIFT 0xb
+#define STREAM_SYNCHRONIZATION__STREAM_12_SYNCHRONIZATION_MASK 0x1000
+#define STREAM_SYNCHRONIZATION__STREAM_12_SYNCHRONIZATION__SHIFT 0xc
+#define STREAM_SYNCHRONIZATION__STREAM_13_SYNCHRONIZATION_MASK 0x2000
+#define STREAM_SYNCHRONIZATION__STREAM_13_SYNCHRONIZATION__SHIFT 0xd
+#define STREAM_SYNCHRONIZATION__STREAM_14_SYNCHRONIZATION_MASK 0x4000
+#define STREAM_SYNCHRONIZATION__STREAM_14_SYNCHRONIZATION__SHIFT 0xe
+#define STREAM_SYNCHRONIZATION__STREAM_15_SYNCHRONIZATION_MASK 0x8000
+#define STREAM_SYNCHRONIZATION__STREAM_15_SYNCHRONIZATION__SHIFT 0xf
+#define CORB_LOWER_BASE_ADDRESS__CORB_LOWER_BASE_UNIMPLEMENTED_BITS_MASK 0x7f
+#define CORB_LOWER_BASE_ADDRESS__CORB_LOWER_BASE_UNIMPLEMENTED_BITS__SHIFT 0x0
+#define CORB_LOWER_BASE_ADDRESS__CORB_LOWER_BASE_ADDRESS_MASK 0xffffff80
+#define CORB_LOWER_BASE_ADDRESS__CORB_LOWER_BASE_ADDRESS__SHIFT 0x7
+#define CORB_UPPER_BASE_ADDRESS__CORB_UPPER_BASE_ADDRESS_MASK 0xffffffff
+#define CORB_UPPER_BASE_ADDRESS__CORB_UPPER_BASE_ADDRESS__SHIFT 0x0
+#define CORB_WRITE_POINTER__CORB_WRITE_POINTER_MASK 0xff
+#define CORB_WRITE_POINTER__CORB_WRITE_POINTER__SHIFT 0x0
+#define CORB_READ_POINTER__CORB_READ_POINTER_MASK 0xff
+#define CORB_READ_POINTER__CORB_READ_POINTER__SHIFT 0x0
+#define CORB_READ_POINTER__CORB_READ_POINTER_RESET_MASK 0x8000
+#define CORB_READ_POINTER__CORB_READ_POINTER_RESET__SHIFT 0xf
+#define CORB_CONTROL__CORB_MEMORY_ERROR_INTERRUPT_ENABLE_MASK 0x1
+#define CORB_CONTROL__CORB_MEMORY_ERROR_INTERRUPT_ENABLE__SHIFT 0x0
+#define CORB_CONTROL__ENABLE_CORB_DMA_ENGINE_MASK 0x2
+#define CORB_CONTROL__ENABLE_CORB_DMA_ENGINE__SHIFT 0x1
+#define CORB_STATUS__CORB_MEMORY_ERROR_INDICATION_MASK 0x1
+#define CORB_STATUS__CORB_MEMORY_ERROR_INDICATION__SHIFT 0x0
+#define CORB_SIZE__CORB_SIZE_MASK 0x3
+#define CORB_SIZE__CORB_SIZE__SHIFT 0x0
+#define CORB_SIZE__CORB_SIZE_CAPABILITY_MASK 0xf0
+#define CORB_SIZE__CORB_SIZE_CAPABILITY__SHIFT 0x4
+#define RIRB_LOWER_BASE_ADDRESS__RIRB_LOWER_BASE_UNIMPLEMENTED_BITS_MASK 0x7f
+#define RIRB_LOWER_BASE_ADDRESS__RIRB_LOWER_BASE_UNIMPLEMENTED_BITS__SHIFT 0x0
+#define RIRB_LOWER_BASE_ADDRESS__RIRB_LOWER_BASE_ADDRESS_MASK 0xffffff80
+#define RIRB_LOWER_BASE_ADDRESS__RIRB_LOWER_BASE_ADDRESS__SHIFT 0x7
+#define RIRB_UPPER_BASE_ADDRESS__RIRB_UPPER_BASE_ADDRESS_MASK 0xffffffff
+#define RIRB_UPPER_BASE_ADDRESS__RIRB_UPPER_BASE_ADDRESS__SHIFT 0x0
+#define RIRB_WRITE_POINTER__RIRB_WRITE_POINTER_MASK 0xff
+#define RIRB_WRITE_POINTER__RIRB_WRITE_POINTER__SHIFT 0x0
+#define RIRB_WRITE_POINTER__RIRB_WRITE_POINTER_RESET_MASK 0x8000
+#define RIRB_WRITE_POINTER__RIRB_WRITE_POINTER_RESET__SHIFT 0xf
+#define RESPONSE_INTERRUPT_COUNT__N_RESPONSE_INTERRUPT_COUNT_MASK 0xff
+#define RESPONSE_INTERRUPT_COUNT__N_RESPONSE_INTERRUPT_COUNT__SHIFT 0x0
+#define RIRB_CONTROL__RESPONSE_INTERRUPT_CONTROL_MASK 0x1
+#define RIRB_CONTROL__RESPONSE_INTERRUPT_CONTROL__SHIFT 0x0
+#define RIRB_CONTROL__RIRB_DMA_ENABLE_MASK 0x2
+#define RIRB_CONTROL__RIRB_DMA_ENABLE__SHIFT 0x1
+#define RIRB_CONTROL__RESPONSE_OVERRUN_INTERRUPT_CONTROL_MASK 0x4
+#define RIRB_CONTROL__RESPONSE_OVERRUN_INTERRUPT_CONTROL__SHIFT 0x2
+#define RIRB_STATUS__RESPONSE_INTERRUPT_MASK 0x1
+#define RIRB_STATUS__RESPONSE_INTERRUPT__SHIFT 0x0
+#define RIRB_STATUS__RESPONSE_OVERRUN_INTERRUPT_STATUS_MASK 0x4
+#define RIRB_STATUS__RESPONSE_OVERRUN_INTERRUPT_STATUS__SHIFT 0x2
+#define RIRB_SIZE__RIRB_SIZE_MASK 0x3
+#define RIRB_SIZE__RIRB_SIZE__SHIFT 0x0
+#define RIRB_SIZE__RIRB_SIZE_CAPABILITY_MASK 0xf0
+#define RIRB_SIZE__RIRB_SIZE_CAPABILITY__SHIFT 0x4
+#define IMMEDIATE_COMMAND_OUTPUT_INTERFACE__IMMEDIATE_COMMAND_WRITE_VERB_AND_PAYLOAD_MASK 0xfffffff
+#define IMMEDIATE_COMMAND_OUTPUT_INTERFACE__IMMEDIATE_COMMAND_WRITE_VERB_AND_PAYLOAD__SHIFT 0x0
+#define IMMEDIATE_COMMAND_OUTPUT_INTERFACE__IMMEDIATE_COMMAND_WRITE_CODEC_ADDRESS_MASK 0xf0000000
+#define IMMEDIATE_COMMAND_OUTPUT_INTERFACE__IMMEDIATE_COMMAND_WRITE_CODEC_ADDRESS__SHIFT 0x1c
+#define IMMEDIATE_COMMAND_OUTPUT_INTERFACE_INDEX__IMMEDIATE_COMMAND_WRITE_MASK 0xffff
+#define IMMEDIATE_COMMAND_OUTPUT_INTERFACE_INDEX__IMMEDIATE_COMMAND_WRITE__SHIFT 0x0
+#define IMMEDIATE_COMMAND_OUTPUT_INTERFACE_DATA__IMMEDIATE_COMMAND_WRITE_MASK 0xffffffff
+#define IMMEDIATE_COMMAND_OUTPUT_INTERFACE_DATA__IMMEDIATE_COMMAND_WRITE__SHIFT 0x0
+#define IMMEDIATE_RESPONSE_INPUT_INTERFACE__IMMEDIATE_RESPONSE_READ_MASK 0xffffffff
+#define IMMEDIATE_RESPONSE_INPUT_INTERFACE__IMMEDIATE_RESPONSE_READ__SHIFT 0x0
+#define IMMEDIATE_COMMAND_STATUS__IMMEDIATE_COMMAND_BUSY_MASK 0x1
+#define IMMEDIATE_COMMAND_STATUS__IMMEDIATE_COMMAND_BUSY__SHIFT 0x0
+#define IMMEDIATE_COMMAND_STATUS__IMMEDIATE_RESULT_VALID_MASK 0x2
+#define IMMEDIATE_COMMAND_STATUS__IMMEDIATE_RESULT_VALID__SHIFT 0x1
+#define DMA_POSITION_LOWER_BASE_ADDRESS__DMA_POSITION_BUFFER_ENABLE_MASK 0x1
+#define DMA_POSITION_LOWER_BASE_ADDRESS__DMA_POSITION_BUFFER_ENABLE__SHIFT 0x0
+#define DMA_POSITION_LOWER_BASE_ADDRESS__DMA_POSITION_LOWER_BASE_UNIMPLEMENTED_BITS_MASK 0x7e
+#define DMA_POSITION_LOWER_BASE_ADDRESS__DMA_POSITION_LOWER_BASE_UNIMPLEMENTED_BITS__SHIFT 0x1
+#define DMA_POSITION_LOWER_BASE_ADDRESS__DMA_POSITION_LOWER_BASE_ADDRESS_MASK 0xffffff80
+#define DMA_POSITION_LOWER_BASE_ADDRESS__DMA_POSITION_LOWER_BASE_ADDRESS__SHIFT 0x7
+#define DMA_POSITION_UPPER_BASE_ADDRESS__DMA_POSITION_UPPER_BASE_ADDRESS_MASK 0xffffffff
+#define DMA_POSITION_UPPER_BASE_ADDRESS__DMA_POSITION_UPPER_BASE_ADDRESS__SHIFT 0x0
+#define WALL_CLOCK_COUNTER_ALIAS__WALL_CLOCK_COUNTER_ALIAS_MASK 0xffffffff
+#define WALL_CLOCK_COUNTER_ALIAS__WALL_CLOCK_COUNTER_ALIAS__SHIFT 0x0
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__STREAM_RESET_MASK 0x1
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__STREAM_RESET__SHIFT 0x0
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__STREAM_RUN_MASK 0x2
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__STREAM_RUN__SHIFT 0x1
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__INTERRUPT_ON_COMPLETION_ENABLE_MASK 0x4
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__INTERRUPT_ON_COMPLETION_ENABLE__SHIFT 0x2
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__FIFO_ERROR_INTERRUPT_ENABLE_MASK 0x8
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__FIFO_ERROR_INTERRUPT_ENABLE__SHIFT 0x3
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__DESCRIPTOR_ERROR_INTERRUPT_ENABLE_MASK 0x10
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__DESCRIPTOR_ERROR_INTERRUPT_ENABLE__SHIFT 0x4
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__STRIPE_CONTROL_MASK 0x30000
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__STRIPE_CONTROL__SHIFT 0x10
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__TRAFFIC_PRIORITY_MASK 0x40000
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__TRAFFIC_PRIORITY__SHIFT 0x12
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__STREAM_NUMBER_MASK 0xf00000
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__STREAM_NUMBER__SHIFT 0x14
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__BUFFER_COMPLETION_INTERRUPT_STATUS_MASK 0x4000000
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__BUFFER_COMPLETION_INTERRUPT_STATUS__SHIFT 0x1a
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__FIFO_ERROR_MASK 0x8000000
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__FIFO_ERROR__SHIFT 0x1b
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__DESCRIPTOR_ERROR_MASK 0x10000000
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__DESCRIPTOR_ERROR__SHIFT 0x1c
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__FIFO_READY_MASK 0x20000000
+#define OUTPUT_STREAM_DESCRIPTOR_CONTROL_AND_STATUS__FIFO_READY__SHIFT 0x1d
+#define OUTPUT_STREAM_DESCRIPTOR_LINK_POSITION_IN_CURRENT_BUFFER__LINK_POSITION_IN_BUFFER_MASK 0xffffffff
+#define OUTPUT_STREAM_DESCRIPTOR_LINK_POSITION_IN_CURRENT_BUFFER__LINK_POSITION_IN_BUFFER__SHIFT 0x0
+#define OUTPUT_STREAM_DESCRIPTOR_CYCLIC_BUFFER_LENGTH__CYCLIC_BUFFER_LENGTH_MASK 0xffffffff
+#define OUTPUT_STREAM_DESCRIPTOR_CYCLIC_BUFFER_LENGTH__CYCLIC_BUFFER_LENGTH__SHIFT 0x0
+#define OUTPUT_STREAM_DESCRIPTOR_LAST_VALID_INDEX__LAST_VALID_INDEX_MASK 0xff
+#define OUTPUT_STREAM_DESCRIPTOR_LAST_VALID_INDEX__LAST_VALID_INDEX__SHIFT 0x0
+#define OUTPUT_STREAM_DESCRIPTOR_FIFO_SIZE__FIFO_SIZE_MASK 0xffff
+#define OUTPUT_STREAM_DESCRIPTOR_FIFO_SIZE__FIFO_SIZE__SHIFT 0x0
+#define OUTPUT_STREAM_DESCRIPTOR_FORMAT__NUMBER_OF_CHANNELS_MASK 0xf
+#define OUTPUT_STREAM_DESCRIPTOR_FORMAT__NUMBER_OF_CHANNELS__SHIFT 0x0
+#define OUTPUT_STREAM_DESCRIPTOR_FORMAT__BITS_PER_SAMPLE_MASK 0x70
+#define OUTPUT_STREAM_DESCRIPTOR_FORMAT__BITS_PER_SAMPLE__SHIFT 0x4
+#define OUTPUT_STREAM_DESCRIPTOR_FORMAT__SAMPLE_BASE_DIVISOR_MASK 0x700
+#define OUTPUT_STREAM_DESCRIPTOR_FORMAT__SAMPLE_BASE_DIVISOR__SHIFT 0x8
+#define OUTPUT_STREAM_DESCRIPTOR_FORMAT__SAMPLE_BASE_MULTIPLE_MASK 0x3800
+#define OUTPUT_STREAM_DESCRIPTOR_FORMAT__SAMPLE_BASE_MULTIPLE__SHIFT 0xb
+#define OUTPUT_STREAM_DESCRIPTOR_FORMAT__SAMPLE_BASE_RATE_MASK 0x4000
+#define OUTPUT_STREAM_DESCRIPTOR_FORMAT__SAMPLE_BASE_RATE__SHIFT 0xe
+#define OUTPUT_STREAM_DESCRIPTOR_BDL_POINTER_LOWER_BASE_ADDRESS__BUFFER_DESCRIPTOR_LIST_LOWER_BASE_ADDRESS_UNIMPLEMENTED_BITS_MASK 0x7f
+#define OUTPUT_STREAM_DESCRIPTOR_BDL_POINTER_LOWER_BASE_ADDRESS__BUFFER_DESCRIPTOR_LIST_LOWER_BASE_ADDRESS_UNIMPLEMENTED_BITS__SHIFT 0x0
+#define OUTPUT_STREAM_DESCRIPTOR_BDL_POINTER_LOWER_BASE_ADDRESS__BUFFER_DESCRIPTOR_LIST_LOWER_BASE_ADDRESS_MASK 0xffffff80
+#define OUTPUT_STREAM_DESCRIPTOR_BDL_POINTER_LOWER_BASE_ADDRESS__BUFFER_DESCRIPTOR_LIST_LOWER_BASE_ADDRESS__SHIFT 0x7
+#define OUTPUT_STREAM_DESCRIPTOR_BDL_POINTER_UPPER_BASE_ADDRESS__BUFFER_DESCRIPTOR_LIST_UPPER_BASE_ADDRESS_MASK 0xffffffff
+#define OUTPUT_STREAM_DESCRIPTOR_BDL_POINTER_UPPER_BASE_ADDRESS__BUFFER_DESCRIPTOR_LIST_UPPER_BASE_ADDRESS__SHIFT 0x0
+#define OUTPUT_STREAM_DESCRIPTOR_LINK_POSITION_IN_CURRENT_BUFFER_ALIAS__LINK_POSITION_IN_BUFFER_ALIAS_MASK 0xffffffff
+#define OUTPUT_STREAM_DESCRIPTOR_LINK_POSITION_IN_CURRENT_BUFFER_ALIAS__LINK_POSITION_IN_BUFFER_ALIAS__SHIFT 0x0
+#define AZENDPOINT_IMMEDIATE_COMMAND_OUTPUT_INTERFACE_INDEX__IMMEDIATE_COMMAND_WRITE_MASK 0x1ffff
+#define AZENDPOINT_IMMEDIATE_COMMAND_OUTPUT_INTERFACE_INDEX__IMMEDIATE_COMMAND_WRITE__SHIFT 0x0
+#define AZENDPOINT_IMMEDIATE_COMMAND_OUTPUT_INTERFACE_DATA__IMMEDIATE_COMMAND_WRITE_MASK 0xffffffff
+#define AZENDPOINT_IMMEDIATE_COMMAND_OUTPUT_INTERFACE_DATA__IMMEDIATE_COMMAND_WRITE__SHIFT 0x0
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_CHANNEL_CAPABILITIES_MASK 0x1
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_CHANNEL_CAPABILITIES__SHIFT 0x0
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__INPUT_AMPLIFIER_PRESENT_MASK 0x2
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__INPUT_AMPLIFIER_PRESENT__SHIFT 0x1
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__OUTPUT_AMPLIFIER_PRESENT_MASK 0x4
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__OUTPUT_AMPLIFIER_PRESENT__SHIFT 0x2
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AMPLIFIER_PARAMETER_OVERRIDE_MASK 0x8
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AMPLIFIER_PARAMETER_OVERRIDE__SHIFT 0x3
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__FORMAT_OVERRIDE_MASK 0x10
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__FORMAT_OVERRIDE__SHIFT 0x4
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__STRIPE_MASK 0x20
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__STRIPE__SHIFT 0x5
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__PROCESSING_WIDGET_MASK 0x40
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__PROCESSING_WIDGET__SHIFT 0x6
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__UNSOLICITED_RESPONSE_CAPABILITY_MASK 0x80
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__UNSOLICITED_RESPONSE_CAPABILITY__SHIFT 0x7
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__CONNECTION_LIST_MASK 0x100
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__CONNECTION_LIST__SHIFT 0x8
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__DIGITAL_MASK 0x200
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__DIGITAL__SHIFT 0x9
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__POWER_CONTROL_MASK 0x400
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__POWER_CONTROL__SHIFT 0xa
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__LR_SWAP_MASK 0x800
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__LR_SWAP__SHIFT 0xb
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_WIDGET_CAPABILITIES_DELAY_MASK 0xf0000
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_WIDGET_CAPABILITIES_DELAY__SHIFT 0x10
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__TYPE_MASK 0xf00000
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__TYPE__SHIFT 0x14
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_RATE_CAPABILITIES_MASK 0xfff
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_RATE_CAPABILITIES__SHIFT 0x0
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_BIT_CAPABILITIES_MASK 0x1f0000
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_BIT_CAPABILITIES__SHIFT 0x10
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_STREAM_FORMATS__STREAM_FORMATS_MASK 0xffffffff
+#define AZALIA_F2_CODEC_CONVERTER_PARAMETER_STREAM_FORMATS__STREAM_FORMATS__SHIFT 0x0
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__NUMBER_OF_CHANNELS_MASK 0xf
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__NUMBER_OF_CHANNELS__SHIFT 0x0
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__BITS_PER_SAMPLE_MASK 0x70
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__BITS_PER_SAMPLE__SHIFT 0x4
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_DIVISOR_MASK 0x700
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_DIVISOR__SHIFT 0x8
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_MULTIPLE_MASK 0x3800
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_MULTIPLE__SHIFT 0xb
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_RATE_MASK 0x4000
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_RATE__SHIFT 0xe
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__STREAM_TYPE_MASK 0x8000
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__STREAM_TYPE__SHIFT 0xf
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__STREAM_TYPE_R_MASK 0x8000
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__STREAM_TYPE_R__SHIFT 0xf
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_CHANNEL_STREAM_ID__CHANNEL_ID_MASK 0xf
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_CHANNEL_STREAM_ID__CHANNEL_ID__SHIFT 0x0
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_CHANNEL_STREAM_ID__STREAM_ID_MASK 0xf0
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_CHANNEL_STREAM_ID__STREAM_ID__SHIFT 0x4
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__DIGEN_MASK 0x1
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__DIGEN__SHIFT 0x0
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__V_MASK 0x2
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__V__SHIFT 0x1
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__VCFG_MASK 0x4
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__VCFG__SHIFT 0x2
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__PRE_MASK 0x8
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__PRE__SHIFT 0x3
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__COPY_MASK 0x10
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__COPY__SHIFT 0x4
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__NON_AUDIO_MASK 0x20
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__NON_AUDIO__SHIFT 0x5
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__PRO_MASK 0x40
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__PRO__SHIFT 0x6
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__L_MASK 0x80
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__L__SHIFT 0x7
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__CC_MASK 0x7f00
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__CC__SHIFT 0x8
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__KEEPALIVE_MASK 0x800000
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__KEEPALIVE__SHIFT 0x17
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_2__CC_MASK 0x7f
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_2__CC__SHIFT 0x0
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_3__KEEPALIVE_MASK 0x80
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER_3__KEEPALIVE__SHIFT 0x7
+#define AZALIA_F2_CODEC_CONVERTER_STRIPE_CONTROL__STRIPE_CONTROL_MASK 0x3
+#define AZALIA_F2_CODEC_CONVERTER_STRIPE_CONTROL__STRIPE_CONTROL__SHIFT 0x0
+#define AZALIA_F2_CODEC_CONVERTER_STRIPE_CONTROL__STRIPE_CAPABILITY_MASK 0x700000
+#define AZALIA_F2_CODEC_CONVERTER_STRIPE_CONTROL__STRIPE_CAPABILITY__SHIFT 0x14
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_RAMP_RATE__RAMP_RATE_MASK 0xff
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_RAMP_RATE__RAMP_RATE__SHIFT 0x0
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_GTC_EMBEDDING__PRESENTATION_TIME_EMBEDDING_ENABLE_MASK 0x1
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_GTC_EMBEDDING__PRESENTATION_TIME_EMBEDDING_ENABLE__SHIFT 0x0
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_GTC_EMBEDDING__PRESENTATION_TIME_OFFSET_CHANGED_MASK 0x2
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_GTC_EMBEDDING__PRESENTATION_TIME_OFFSET_CHANGED__SHIFT 0x1
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_GTC_EMBEDDING__PRESENTATION_TIME_EMBEDDING_GROUP_MASK 0x70
+#define AZALIA_F2_CODEC_CONVERTER_CONTROL_GTC_EMBEDDING__PRESENTATION_TIME_EMBEDDING_GROUP__SHIFT 0x4
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_CHANNEL_CAPABILITIES_MASK 0x1
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_CHANNEL_CAPABILITIES__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__INPUT_AMPLIFIER_PRESENT_MASK 0x2
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__INPUT_AMPLIFIER_PRESENT__SHIFT 0x1
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__OUTPUT_AMPLIFIER_PRESENT_MASK 0x4
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__OUTPUT_AMPLIFIER_PRESENT__SHIFT 0x2
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AMPLIFIER_PARAMETER_OVERRIDE_MASK 0x8
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AMPLIFIER_PARAMETER_OVERRIDE__SHIFT 0x3
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__STRIPE_MASK 0x20
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__STRIPE__SHIFT 0x5
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__PROCESSING_WIDGET_MASK 0x40
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__PROCESSING_WIDGET__SHIFT 0x6
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__UNSOLICITED_RESPONSE_CAPABILITY_MASK 0x80
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__UNSOLICITED_RESPONSE_CAPABILITY__SHIFT 0x7
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__CONNECTION_LIST_MASK 0x100
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__CONNECTION_LIST__SHIFT 0x8
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__DIGITAL_MASK 0x200
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__DIGITAL__SHIFT 0x9
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__POWER_CONTROL_MASK 0x400
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__POWER_CONTROL__SHIFT 0xa
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__LR_SWAP_MASK 0x800
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__LR_SWAP__SHIFT 0xb
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_WIDGET_CAPABILITIES_DELAY_MASK 0xf0000
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_WIDGET_CAPABILITIES_DELAY__SHIFT 0x10
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__TYPE_MASK 0xf00000
+#define AZALIA_F2_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__TYPE__SHIFT 0x14
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES__IMPEDANCE_SENSE_CAPABLE_MASK 0x1
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES__IMPEDANCE_SENSE_CAPABLE__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES__TRIGGER_REQUIRED_MASK 0x2
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES__TRIGGER_REQUIRED__SHIFT 0x1
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES__JACK_DETECTION_CAPABILITY_MASK 0x4
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES__JACK_DETECTION_CAPABILITY__SHIFT 0x2
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES__HEADPHONE_DRIVE_CAPABLE_MASK 0x8
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES__HEADPHONE_DRIVE_CAPABLE__SHIFT 0x3
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES__OUTPUT_CAPABLE_MASK 0x10
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES__OUTPUT_CAPABLE__SHIFT 0x4
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES__INPUT_CAPABLE_MASK 0x20
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES__INPUT_CAPABLE__SHIFT 0x5
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES__BALANCED_I_O_PINS_MASK 0x40
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES__BALANCED_I_O_PINS__SHIFT 0x6
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES__HDMI_MASK 0x80
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES__HDMI__SHIFT 0x7
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES__VREF_CONTROL_MASK 0xff00
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES__VREF_CONTROL__SHIFT 0x8
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES__EAPD_CAPABLE_MASK 0x10000
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES__EAPD_CAPABLE__SHIFT 0x10
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES__DP_MASK 0x1000000
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CAPABILITIES__DP__SHIFT 0x18
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CONNECTION_LIST_LENGTH__CONNECTION_LIST_LENGTH_MASK 0xffffffff
+#define AZALIA_F2_CODEC_PIN_PARAMETER_CONNECTION_LIST_LENGTH__CONNECTION_LIST_LENGTH__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONNECTION_LIST_ENTRY__CONNECTION_LIST_ENTRY_MASK 0xffffffff
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONNECTION_LIST_ENTRY__CONNECTION_LIST_ENTRY__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_WIDGET_CONTROL__OUT_ENABLE_MASK 0x40
+#define AZALIA_F2_CODEC_PIN_CONTROL_WIDGET_CONTROL__OUT_ENABLE__SHIFT 0x6
+#define AZALIA_F2_CODEC_PIN_CONTROL_UNSOLICITED_RESPONSE__TAG_MASK 0x3f
+#define AZALIA_F2_CODEC_PIN_CONTROL_UNSOLICITED_RESPONSE__TAG__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_UNSOLICITED_RESPONSE__ENABLE_MASK 0x80
+#define AZALIA_F2_CODEC_PIN_CONTROL_UNSOLICITED_RESPONSE__ENABLE__SHIFT 0x7
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_PIN_SENSE__IMPEDANCE_SENSE_MASK 0x7fffffff
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_PIN_SENSE__IMPEDANCE_SENSE__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_PIN_SENSE__PRESENCE_DETECT_MASK 0x80000000
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_PIN_SENSE__PRESENCE_DETECT__SHIFT 0x1f
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__SEQUENCE_MASK 0xf
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__SEQUENCE__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__DEFAULT_ASSOCIATION_MASK 0xf0
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__DEFAULT_ASSOCIATION__SHIFT 0x4
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__MISC_MASK 0xf00
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__MISC__SHIFT 0x8
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__COLOR_MASK 0xf000
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__COLOR__SHIFT 0xc
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__CONNECTION_TYPE_MASK 0xf0000
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__CONNECTION_TYPE__SHIFT 0x10
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__DEFAULT_DEVICE_MASK 0xf00000
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__DEFAULT_DEVICE__SHIFT 0x14
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__LOCATION_MASK 0x3f000000
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__LOCATION__SHIFT 0x18
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__PORT_CONNECTIVITY_MASK 0xc0000000
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__PORT_CONNECTIVITY__SHIFT 0x1e
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_2__MISC_MASK 0xf
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_2__MISC__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_2__COLOR_MASK 0xf0
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_2__COLOR__SHIFT 0x4
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_3__CONNECTION_TYPE_MASK 0xf
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_3__CONNECTION_TYPE__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_3__DEFAULT_DEVICE_MASK 0xf0
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_3__DEFAULT_DEVICE__SHIFT 0x4
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_4__LOCATION_MASK 0x3f
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_4__LOCATION__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_4__PORT_CONNECTIVITY_MASK 0xc0
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_4__PORT_CONNECTIVITY__SHIFT 0x6
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_SPEAKER_ALLOCATION__SPEAKER_ALLOCATION_MASK 0x7f
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_SPEAKER_ALLOCATION__SPEAKER_ALLOCATION__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_SPEAKER_ALLOCATION__HDMI_CONNECTION_MASK 0x100
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_SPEAKER_ALLOCATION__HDMI_CONNECTION__SHIFT 0x8
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_SPEAKER_ALLOCATION__DP_CONNECTION_MASK 0x200
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_SPEAKER_ALLOCATION__DP_CONNECTION__SHIFT 0x9
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_SPEAKER_ALLOCATION__EXTRA_CONNECTION_INFO_MASK 0xfc00
+#define AZALIA_F2_CODEC_PIN_CONTROL_RESPONSE_SPEAKER_ALLOCATION__EXTRA_CONNECTION_INFO__SHIFT 0xa
+#define AZALIA_F2_CODEC_PIN_CONTROL_CHANNEL_ALLOCATION__CHANNEL_ALLOCATION_MASK 0xff
+#define AZALIA_F2_CODEC_PIN_CONTROL_CHANNEL_ALLOCATION__CHANNEL_ALLOCATION__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_DOWN_MIX_INFO__LFE_PLAYBACK_LEVEL_MASK 0x3
+#define AZALIA_F2_CODEC_PIN_CONTROL_DOWN_MIX_INFO__LFE_PLAYBACK_LEVEL__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_DOWN_MIX_INFO__LEVEL_SHIFT_MASK 0x78
+#define AZALIA_F2_CODEC_PIN_CONTROL_DOWN_MIX_INFO__LEVEL_SHIFT__SHIFT 0x3
+#define AZALIA_F2_CODEC_PIN_CONTROL_DOWN_MIX_INFO__DOWN_MIX_INHIBIT_MASK 0x80
+#define AZALIA_F2_CODEC_PIN_CONTROL_DOWN_MIX_INFO__DOWN_MIX_INHIBIT__SHIFT 0x7
+#define AZALIA_F2_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR__MAX_CHANNELS_MASK 0x7
+#define AZALIA_F2_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR__MAX_CHANNELS__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR__FORMAT_CODE_MASK 0x78
+#define AZALIA_F2_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR__FORMAT_CODE__SHIFT 0x3
+#define AZALIA_F2_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AZALIA_F2_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AZALIA_F2_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AZALIA_F2_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AZALIA_F2_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR__SUPPORTED_FREQUENCIES_STEREO_MASK 0xff000000
+#define AZALIA_F2_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR__SUPPORTED_FREQUENCIES_STEREO__SHIFT 0x18
+#define AZALIA_F2_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR_DATA__DESCRIPTOR_MASK 0xffffffff
+#define AZALIA_F2_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR_DATA__DESCRIPTOR__SHIFT 0x0
+#define AUDIO_DESCRIPTOR0__MAX_CHANNELS_MASK 0x7
+#define AUDIO_DESCRIPTOR0__MAX_CHANNELS__SHIFT 0x0
+#define AUDIO_DESCRIPTOR0__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AUDIO_DESCRIPTOR0__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AUDIO_DESCRIPTOR0__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AUDIO_DESCRIPTOR0__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AUDIO_DESCRIPTOR0__SUPPORTED_FREQUENCIES_STEREO_MASK 0xff000000
+#define AUDIO_DESCRIPTOR0__SUPPORTED_FREQUENCIES_STEREO__SHIFT 0x18
+#define AUDIO_DESCRIPTOR1__MAX_CHANNELS_MASK 0x7
+#define AUDIO_DESCRIPTOR1__MAX_CHANNELS__SHIFT 0x0
+#define AUDIO_DESCRIPTOR1__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AUDIO_DESCRIPTOR1__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AUDIO_DESCRIPTOR1__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AUDIO_DESCRIPTOR1__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AUDIO_DESCRIPTOR1__SUPPORTED_FREQUENCIES_STEREO_MASK 0xff000000
+#define AUDIO_DESCRIPTOR1__SUPPORTED_FREQUENCIES_STEREO__SHIFT 0x18
+#define AUDIO_DESCRIPTOR2__MAX_CHANNELS_MASK 0x7
+#define AUDIO_DESCRIPTOR2__MAX_CHANNELS__SHIFT 0x0
+#define AUDIO_DESCRIPTOR2__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AUDIO_DESCRIPTOR2__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AUDIO_DESCRIPTOR2__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AUDIO_DESCRIPTOR2__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AUDIO_DESCRIPTOR2__SUPPORTED_FREQUENCIES_STEREO_MASK 0xff000000
+#define AUDIO_DESCRIPTOR2__SUPPORTED_FREQUENCIES_STEREO__SHIFT 0x18
+#define AUDIO_DESCRIPTOR3__MAX_CHANNELS_MASK 0x7
+#define AUDIO_DESCRIPTOR3__MAX_CHANNELS__SHIFT 0x0
+#define AUDIO_DESCRIPTOR3__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AUDIO_DESCRIPTOR3__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AUDIO_DESCRIPTOR3__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AUDIO_DESCRIPTOR3__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AUDIO_DESCRIPTOR3__SUPPORTED_FREQUENCIES_STEREO_MASK 0xff000000
+#define AUDIO_DESCRIPTOR3__SUPPORTED_FREQUENCIES_STEREO__SHIFT 0x18
+#define AUDIO_DESCRIPTOR4__MAX_CHANNELS_MASK 0x7
+#define AUDIO_DESCRIPTOR4__MAX_CHANNELS__SHIFT 0x0
+#define AUDIO_DESCRIPTOR4__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AUDIO_DESCRIPTOR4__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AUDIO_DESCRIPTOR4__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AUDIO_DESCRIPTOR4__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AUDIO_DESCRIPTOR4__SUPPORTED_FREQUENCIES_STEREO_MASK 0xff000000
+#define AUDIO_DESCRIPTOR4__SUPPORTED_FREQUENCIES_STEREO__SHIFT 0x18
+#define AUDIO_DESCRIPTOR5__MAX_CHANNELS_MASK 0x7
+#define AUDIO_DESCRIPTOR5__MAX_CHANNELS__SHIFT 0x0
+#define AUDIO_DESCRIPTOR5__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AUDIO_DESCRIPTOR5__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AUDIO_DESCRIPTOR5__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AUDIO_DESCRIPTOR5__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AUDIO_DESCRIPTOR5__SUPPORTED_FREQUENCIES_STEREO_MASK 0xff000000
+#define AUDIO_DESCRIPTOR5__SUPPORTED_FREQUENCIES_STEREO__SHIFT 0x18
+#define AUDIO_DESCRIPTOR6__MAX_CHANNELS_MASK 0x7
+#define AUDIO_DESCRIPTOR6__MAX_CHANNELS__SHIFT 0x0
+#define AUDIO_DESCRIPTOR6__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AUDIO_DESCRIPTOR6__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AUDIO_DESCRIPTOR6__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AUDIO_DESCRIPTOR6__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AUDIO_DESCRIPTOR6__SUPPORTED_FREQUENCIES_STEREO_MASK 0xff000000
+#define AUDIO_DESCRIPTOR6__SUPPORTED_FREQUENCIES_STEREO__SHIFT 0x18
+#define AUDIO_DESCRIPTOR7__MAX_CHANNELS_MASK 0x7
+#define AUDIO_DESCRIPTOR7__MAX_CHANNELS__SHIFT 0x0
+#define AUDIO_DESCRIPTOR7__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AUDIO_DESCRIPTOR7__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AUDIO_DESCRIPTOR7__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AUDIO_DESCRIPTOR7__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AUDIO_DESCRIPTOR7__SUPPORTED_FREQUENCIES_STEREO_MASK 0xff000000
+#define AUDIO_DESCRIPTOR7__SUPPORTED_FREQUENCIES_STEREO__SHIFT 0x18
+#define AUDIO_DESCRIPTOR8__MAX_CHANNELS_MASK 0x7
+#define AUDIO_DESCRIPTOR8__MAX_CHANNELS__SHIFT 0x0
+#define AUDIO_DESCRIPTOR8__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AUDIO_DESCRIPTOR8__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AUDIO_DESCRIPTOR8__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AUDIO_DESCRIPTOR8__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AUDIO_DESCRIPTOR8__SUPPORTED_FREQUENCIES_STEREO_MASK 0xff000000
+#define AUDIO_DESCRIPTOR8__SUPPORTED_FREQUENCIES_STEREO__SHIFT 0x18
+#define AUDIO_DESCRIPTOR9__MAX_CHANNELS_MASK 0x7
+#define AUDIO_DESCRIPTOR9__MAX_CHANNELS__SHIFT 0x0
+#define AUDIO_DESCRIPTOR9__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AUDIO_DESCRIPTOR9__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AUDIO_DESCRIPTOR9__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AUDIO_DESCRIPTOR9__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AUDIO_DESCRIPTOR9__SUPPORTED_FREQUENCIES_STEREO_MASK 0xff000000
+#define AUDIO_DESCRIPTOR9__SUPPORTED_FREQUENCIES_STEREO__SHIFT 0x18
+#define AUDIO_DESCRIPTOR10__MAX_CHANNELS_MASK 0x7
+#define AUDIO_DESCRIPTOR10__MAX_CHANNELS__SHIFT 0x0
+#define AUDIO_DESCRIPTOR10__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AUDIO_DESCRIPTOR10__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AUDIO_DESCRIPTOR10__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AUDIO_DESCRIPTOR10__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AUDIO_DESCRIPTOR10__SUPPORTED_FREQUENCIES_STEREO_MASK 0xff000000
+#define AUDIO_DESCRIPTOR10__SUPPORTED_FREQUENCIES_STEREO__SHIFT 0x18
+#define AUDIO_DESCRIPTOR11__MAX_CHANNELS_MASK 0x7
+#define AUDIO_DESCRIPTOR11__MAX_CHANNELS__SHIFT 0x0
+#define AUDIO_DESCRIPTOR11__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AUDIO_DESCRIPTOR11__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AUDIO_DESCRIPTOR11__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AUDIO_DESCRIPTOR11__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AUDIO_DESCRIPTOR11__SUPPORTED_FREQUENCIES_STEREO_MASK 0xff000000
+#define AUDIO_DESCRIPTOR11__SUPPORTED_FREQUENCIES_STEREO__SHIFT 0x18
+#define AUDIO_DESCRIPTOR12__MAX_CHANNELS_MASK 0x7
+#define AUDIO_DESCRIPTOR12__MAX_CHANNELS__SHIFT 0x0
+#define AUDIO_DESCRIPTOR12__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AUDIO_DESCRIPTOR12__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AUDIO_DESCRIPTOR12__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AUDIO_DESCRIPTOR12__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AUDIO_DESCRIPTOR12__SUPPORTED_FREQUENCIES_STEREO_MASK 0xff000000
+#define AUDIO_DESCRIPTOR12__SUPPORTED_FREQUENCIES_STEREO__SHIFT 0x18
+#define AUDIO_DESCRIPTOR13__MAX_CHANNELS_MASK 0x7
+#define AUDIO_DESCRIPTOR13__MAX_CHANNELS__SHIFT 0x0
+#define AUDIO_DESCRIPTOR13__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AUDIO_DESCRIPTOR13__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AUDIO_DESCRIPTOR13__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AUDIO_DESCRIPTOR13__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AUDIO_DESCRIPTOR13__SUPPORTED_FREQUENCIES_STEREO_MASK 0xff000000
+#define AUDIO_DESCRIPTOR13__SUPPORTED_FREQUENCIES_STEREO__SHIFT 0x18
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL01_ENABLE__MULTICHANNEL01_ENABLE_MASK 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL01_ENABLE__MULTICHANNEL01_ENABLE__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL01_ENABLE__MULTICHANNEL01_MUTE_MASK 0x2
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL01_ENABLE__MULTICHANNEL01_MUTE__SHIFT 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL01_ENABLE__MULTICHANNEL01_CHANNEL_ID_MASK 0xf0
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL01_ENABLE__MULTICHANNEL01_CHANNEL_ID__SHIFT 0x4
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL23_ENABLE__MULTICHANNEL23_ENABLE_MASK 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL23_ENABLE__MULTICHANNEL23_ENABLE__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL23_ENABLE__MULTICHANNEL23_MUTE_MASK 0x2
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL23_ENABLE__MULTICHANNEL23_MUTE__SHIFT 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL23_ENABLE__MULTICHANNEL23_CHANNEL_ID_MASK 0xf0
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL23_ENABLE__MULTICHANNEL23_CHANNEL_ID__SHIFT 0x4
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL45_ENABLE__MULTICHANNEL45_ENABLE_MASK 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL45_ENABLE__MULTICHANNEL45_ENABLE__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL45_ENABLE__MULTICHANNEL45_MUTE_MASK 0x2
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL45_ENABLE__MULTICHANNEL45_MUTE__SHIFT 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL45_ENABLE__MULTICHANNEL45_CHANNEL_ID_MASK 0xf0
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL45_ENABLE__MULTICHANNEL45_CHANNEL_ID__SHIFT 0x4
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL67_ENABLE__MULTICHANNEL67_ENABLE_MASK 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL67_ENABLE__MULTICHANNEL67_ENABLE__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL67_ENABLE__MULTICHANNEL67_MUTE_MASK 0x2
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL67_ENABLE__MULTICHANNEL67_MUTE__SHIFT 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL67_ENABLE__MULTICHANNEL67_CHANNEL_ID_MASK 0xf0
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL67_ENABLE__MULTICHANNEL67_CHANNEL_ID__SHIFT 0x4
+#define AZALIA_F2_CODEC_PIN_CONTROL_LIPSYNC__VIDEO_LIPSYNC_MASK 0xff
+#define AZALIA_F2_CODEC_PIN_CONTROL_LIPSYNC__VIDEO_LIPSYNC__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_LIPSYNC__AUDIO_LIPSYNC_MASK 0xff00
+#define AZALIA_F2_CODEC_PIN_CONTROL_LIPSYNC__AUDIO_LIPSYNC__SHIFT 0x8
+#define AZALIA_F2_CODEC_PIN_CONTROL_HBR__HBR_CAPABLE_MASK 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_HBR__HBR_CAPABLE__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_HBR__HBR_ENABLE_MASK 0x10
+#define AZALIA_F2_CODEC_PIN_CONTROL_HBR__HBR_ENABLE__SHIFT 0x4
+#define AZALIA_F2_CODEC_PIN_CONTROL_AUDIO_SINK_INFO_INDEX__SINK_INFO_INDEX_MASK 0xff
+#define AZALIA_F2_CODEC_PIN_CONTROL_AUDIO_SINK_INFO_INDEX__SINK_INFO_INDEX__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_AUDIO_SINK_INFO_DATA__SINK_DATA_MASK 0xffffffff
+#define AZALIA_F2_CODEC_PIN_CONTROL_AUDIO_SINK_INFO_DATA__SINK_DATA__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_MANUFACTURER_ID__MANUFACTURER_ID_MASK 0xffff
+#define AZALIA_F2_CODEC_PIN_CONTROL_MANUFACTURER_ID__MANUFACTURER_ID__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_PRODUCT_ID__PRODUCT_ID_MASK 0xffff
+#define AZALIA_F2_CODEC_PIN_CONTROL_PRODUCT_ID__PRODUCT_ID__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_SINK_DESCRIPTION_LEN__SINK_DESCRIPTION_LEN_MASK 0xff
+#define AZALIA_F2_CODEC_PIN_CONTROL_SINK_DESCRIPTION_LEN__SINK_DESCRIPTION_LEN__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_PORTID0__PORTID_MASK 0xffffffff
+#define AZALIA_F2_CODEC_PIN_CONTROL_PORTID0__PORTID__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_PORTID1__PORTID_MASK 0xffffffff
+#define AZALIA_F2_CODEC_PIN_CONTROL_PORTID1__PORTID__SHIFT 0x0
+#define SINK_DESCRIPTION0__DESCRIPTION_MASK 0xff
+#define SINK_DESCRIPTION0__DESCRIPTION__SHIFT 0x0
+#define SINK_DESCRIPTION1__DESCRIPTION_MASK 0xff
+#define SINK_DESCRIPTION1__DESCRIPTION__SHIFT 0x0
+#define SINK_DESCRIPTION2__DESCRIPTION_MASK 0xff
+#define SINK_DESCRIPTION2__DESCRIPTION__SHIFT 0x0
+#define SINK_DESCRIPTION3__DESCRIPTION_MASK 0xff
+#define SINK_DESCRIPTION3__DESCRIPTION__SHIFT 0x0
+#define SINK_DESCRIPTION4__DESCRIPTION_MASK 0xff
+#define SINK_DESCRIPTION4__DESCRIPTION__SHIFT 0x0
+#define SINK_DESCRIPTION5__DESCRIPTION_MASK 0xff
+#define SINK_DESCRIPTION5__DESCRIPTION__SHIFT 0x0
+#define SINK_DESCRIPTION6__DESCRIPTION_MASK 0xff
+#define SINK_DESCRIPTION6__DESCRIPTION__SHIFT 0x0
+#define SINK_DESCRIPTION7__DESCRIPTION_MASK 0xff
+#define SINK_DESCRIPTION7__DESCRIPTION__SHIFT 0x0
+#define SINK_DESCRIPTION8__DESCRIPTION_MASK 0xff
+#define SINK_DESCRIPTION8__DESCRIPTION__SHIFT 0x0
+#define SINK_DESCRIPTION9__DESCRIPTION_MASK 0xff
+#define SINK_DESCRIPTION9__DESCRIPTION__SHIFT 0x0
+#define SINK_DESCRIPTION10__DESCRIPTION_MASK 0xff
+#define SINK_DESCRIPTION10__DESCRIPTION__SHIFT 0x0
+#define SINK_DESCRIPTION11__DESCRIPTION_MASK 0xff
+#define SINK_DESCRIPTION11__DESCRIPTION__SHIFT 0x0
+#define SINK_DESCRIPTION12__DESCRIPTION_MASK 0xff
+#define SINK_DESCRIPTION12__DESCRIPTION__SHIFT 0x0
+#define SINK_DESCRIPTION13__DESCRIPTION_MASK 0xff
+#define SINK_DESCRIPTION13__DESCRIPTION__SHIFT 0x0
+#define SINK_DESCRIPTION14__DESCRIPTION_MASK 0xff
+#define SINK_DESCRIPTION14__DESCRIPTION__SHIFT 0x0
+#define SINK_DESCRIPTION15__DESCRIPTION_MASK 0xff
+#define SINK_DESCRIPTION15__DESCRIPTION__SHIFT 0x0
+#define SINK_DESCRIPTION16__DESCRIPTION_MASK 0xff
+#define SINK_DESCRIPTION16__DESCRIPTION__SHIFT 0x0
+#define SINK_DESCRIPTION17__DESCRIPTION_MASK 0xff
+#define SINK_DESCRIPTION17__DESCRIPTION__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL1_ENABLE__MULTICHANNEL1_ENABLE_MASK 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL1_ENABLE__MULTICHANNEL1_ENABLE__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL1_ENABLE__MULTICHANNEL1_MUTE_MASK 0x2
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL1_ENABLE__MULTICHANNEL1_MUTE__SHIFT 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL1_ENABLE__MULTICHANNEL1_CHANNEL_ID_MASK 0xf0
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL1_ENABLE__MULTICHANNEL1_CHANNEL_ID__SHIFT 0x4
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL3_ENABLE__MULTICHANNEL3_ENABLE_MASK 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL3_ENABLE__MULTICHANNEL3_ENABLE__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL3_ENABLE__MULTICHANNEL3_MUTE_MASK 0x2
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL3_ENABLE__MULTICHANNEL3_MUTE__SHIFT 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL3_ENABLE__MULTICHANNEL3_CHANNEL_ID_MASK 0xf0
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL3_ENABLE__MULTICHANNEL3_CHANNEL_ID__SHIFT 0x4
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL5_ENABLE__MULTICHANNEL5_ENABLE_MASK 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL5_ENABLE__MULTICHANNEL5_ENABLE__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL5_ENABLE__MULTICHANNEL5_MUTE_MASK 0x2
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL5_ENABLE__MULTICHANNEL5_MUTE__SHIFT 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL5_ENABLE__MULTICHANNEL5_CHANNEL_ID_MASK 0xf0
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL5_ENABLE__MULTICHANNEL5_CHANNEL_ID__SHIFT 0x4
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL7_ENABLE__MULTICHANNEL7_ENABLE_MASK 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL7_ENABLE__MULTICHANNEL7_ENABLE__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL7_ENABLE__MULTICHANNEL7_MUTE_MASK 0x2
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL7_ENABLE__MULTICHANNEL7_MUTE__SHIFT 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL7_ENABLE__MULTICHANNEL7_CHANNEL_ID_MASK 0xf0
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL7_ENABLE__MULTICHANNEL7_CHANNEL_ID__SHIFT 0x4
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL_MODE__MULTICHANNEL_MODE_MASK 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_MULTICHANNEL_MODE__MULTICHANNEL_MODE__SHIFT 0x0
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_0__IEC_60958_CS_MODE_MASK 0x3
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_0__IEC_60958_CS_MODE__SHIFT 0x0
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_0__IEC_60958_CS_SOURCE_NUMBER_MASK 0x3c
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_0__IEC_60958_CS_SOURCE_NUMBER__SHIFT 0x2
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_1__IEC_60958_CS_CLOCK_ACCURACY_MASK 0x3
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_1__IEC_60958_CS_CLOCK_ACCURACY__SHIFT 0x0
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_1__IEC_60958_CS_CLOCK_ACCURACY_OVRRD_EN_MASK 0x4
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_1__IEC_60958_CS_CLOCK_ACCURACY_OVRRD_EN__SHIFT 0x2
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_1__IEC_60958_CS_WORD_LENGTH_MASK 0x78
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_1__IEC_60958_CS_WORD_LENGTH__SHIFT 0x3
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_1__IEC_60958_CS_WORD_LENGTH_OVRRD_EN_MASK 0x80
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_1__IEC_60958_CS_WORD_LENGTH_OVRRD_EN__SHIFT 0x7
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_2__IEC_60958_CS_SAMPLING_FREQUENCY_MASK 0x3f
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_2__IEC_60958_CS_SAMPLING_FREQUENCY__SHIFT 0x0
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_2__IEC_60958_CS_SAMPLING_FREQUENCY_OVRRD_EN_MASK 0x40
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_2__IEC_60958_CS_SAMPLING_FREQUENCY_OVRRD_EN__SHIFT 0x6
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_3__IEC_60958_CS_ORIGINAL_SAMPLING_FREQUENCY_MASK 0xf
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_3__IEC_60958_CS_ORIGINAL_SAMPLING_FREQUENCY__SHIFT 0x0
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_3__IEC_60958_CS_ORIGINAL_SAMPLING_FREQUENCY_OVRRD_EN_MASK 0x10
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_3__IEC_60958_CS_ORIGINAL_SAMPLING_FREQUENCY_OVRRD_EN__SHIFT 0x4
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_4__IEC_60958_CS_SAMPLING_FREQUENCY_COEFF_MASK 0xf
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_4__IEC_60958_CS_SAMPLING_FREQUENCY_COEFF__SHIFT 0x0
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_4__IEC_60958_CS_MPEG_SURROUND_INFO_MASK 0x10
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_4__IEC_60958_CS_MPEG_SURROUND_INFO__SHIFT 0x4
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_4__IEC_60958_CS_CGMS_A_MASK 0x60
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_4__IEC_60958_CS_CGMS_A__SHIFT 0x5
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_4__IEC_60958_CS_CGMS_A_VALID_MASK 0x80
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_4__IEC_60958_CS_CGMS_A_VALID__SHIFT 0x7
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_5__IEC_60958_CS_CHANNEL_NUMBER_L_MASK 0xf
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_5__IEC_60958_CS_CHANNEL_NUMBER_L__SHIFT 0x0
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_5__IEC_60958_CS_CHANNEL_NUMBER_R_MASK 0xf0
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_5__IEC_60958_CS_CHANNEL_NUMBER_R__SHIFT 0x4
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_6__IEC_60958_CS_CHANNEL_NUMBER_2_MASK 0xf
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_6__IEC_60958_CS_CHANNEL_NUMBER_2__SHIFT 0x0
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_6__IEC_60958_CS_CHANNEL_NUMBER_3_MASK 0xf0
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_6__IEC_60958_CS_CHANNEL_NUMBER_3__SHIFT 0x4
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_7__IEC_60958_CS_CHANNEL_NUMBER_4_MASK 0xf
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_7__IEC_60958_CS_CHANNEL_NUMBER_4__SHIFT 0x0
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_7__IEC_60958_CS_CHANNEL_NUMBER_5_MASK 0xf0
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_7__IEC_60958_CS_CHANNEL_NUMBER_5__SHIFT 0x4
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_8__IEC_60958_CS_CHANNEL_NUMBER_6_MASK 0xf
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_8__IEC_60958_CS_CHANNEL_NUMBER_6__SHIFT 0x0
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_8__IEC_60958_CS_CHANNEL_NUMBER_7_MASK 0xf0
+#define AZALIA_F2_PIN_CONTROL_CODEC_CS_OVERRIDE_8__IEC_60958_CS_CHANNEL_NUMBER_7__SHIFT 0x4
+#define AZALIA_F2_CODEC_PIN_ASSOCIATION_INFO__ASSOCIATION_INFO_MASK 0xffffffff
+#define AZALIA_F2_CODEC_PIN_ASSOCIATION_INFO__ASSOCIATION_INFO__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_DIGITAL_OUTPUT_STATUS__OUTPUT_ACTIVE_MASK 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_DIGITAL_OUTPUT_STATUS__OUTPUT_ACTIVE__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_LPIB_SNAPSHOT_CONTROL__LPIB_SNAPSHOT_LOCK_MASK 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_LPIB_SNAPSHOT_CONTROL__LPIB_SNAPSHOT_LOCK__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_LPIB_SNAPSHOT_CONTROL__CYCLIC_BUFFER_WRAP_COUNT_MASK 0xff00
+#define AZALIA_F2_CODEC_PIN_CONTROL_LPIB_SNAPSHOT_CONTROL__CYCLIC_BUFFER_WRAP_COUNT__SHIFT 0x8
+#define AZALIA_F2_CODEC_PIN_CONTROL_LPIB__LPIB_MASK 0xffffffff
+#define AZALIA_F2_CODEC_PIN_CONTROL_LPIB__LPIB__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_LPIB_TIMER_SNAPSHOT__LPIB_TIMER_SNAPSHOT_MASK 0xffffffff
+#define AZALIA_F2_CODEC_PIN_CONTROL_LPIB_TIMER_SNAPSHOT__LPIB_TIMER_SNAPSHOT__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_CODING_TYPE__CODING_TYPE_MASK 0xff
+#define AZALIA_F2_CODEC_PIN_CONTROL_CODING_TYPE__CODING_TYPE__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_FORMAT_CHANGED__FORMAT_CHANGED_MASK 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_FORMAT_CHANGED__FORMAT_CHANGED__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_FORMAT_CHANGED__FORMAT_CHANGED_ACK_UR_ENABLE_MASK 0x2
+#define AZALIA_F2_CODEC_PIN_CONTROL_FORMAT_CHANGED__FORMAT_CHANGED_ACK_UR_ENABLE__SHIFT 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_FORMAT_CHANGED__FORMAT_CHANGE_REASON_MASK 0xff00
+#define AZALIA_F2_CODEC_PIN_CONTROL_FORMAT_CHANGED__FORMAT_CHANGE_REASON__SHIFT 0x8
+#define AZALIA_F2_CODEC_PIN_CONTROL_FORMAT_CHANGED__FORMAT_CHANGE_RESPONSE_MASK 0xff0000
+#define AZALIA_F2_CODEC_PIN_CONTROL_FORMAT_CHANGED__FORMAT_CHANGE_RESPONSE__SHIFT 0x10
+#define AZALIA_F2_CODEC_PIN_CONTROL_WIRELESS_DISPLAY_IDENTIFICATION__WIRELESS_DISPLAY_IDENTIFICATION_MASK 0x3
+#define AZALIA_F2_CODEC_PIN_CONTROL_WIRELESS_DISPLAY_IDENTIFICATION__WIRELESS_DISPLAY_IDENTIFICATION__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_REMOTE_KEEPALIVE__REMOTE_KEEP_ALIVE_ENABLE_MASK 0x1
+#define AZALIA_F2_CODEC_PIN_CONTROL_REMOTE_KEEPALIVE__REMOTE_KEEP_ALIVE_ENABLE__SHIFT 0x0
+#define AZALIA_F2_CODEC_PIN_CONTROL_REMOTE_KEEPALIVE__REMOTE_KEEP_ALIVE_CAPABILITY_MASK 0x10
+#define AZALIA_F2_CODEC_PIN_CONTROL_REMOTE_KEEPALIVE__REMOTE_KEEP_ALIVE_CAPABILITY__SHIFT 0x4
+#define AZALIA_CONTROLLER_CLOCK_GATING__ENABLE_CLOCK_GATING_MASK 0x1
+#define AZALIA_CONTROLLER_CLOCK_GATING__ENABLE_CLOCK_GATING__SHIFT 0x0
+#define AZALIA_CONTROLLER_CLOCK_GATING__CLOCK_ON_STATE_MASK 0x10
+#define AZALIA_CONTROLLER_CLOCK_GATING__CLOCK_ON_STATE__SHIFT 0x4
+#define AZALIA_AUDIO_DTO__AZALIA_AUDIO_DTO_PHASE_MASK 0xffff
+#define AZALIA_AUDIO_DTO__AZALIA_AUDIO_DTO_PHASE__SHIFT 0x0
+#define AZALIA_AUDIO_DTO__AZALIA_AUDIO_DTO_MODULE_MASK 0xffff0000
+#define AZALIA_AUDIO_DTO__AZALIA_AUDIO_DTO_MODULE__SHIFT 0x10
+#define AZALIA_AUDIO_DTO_CONTROL__AZALIA_AUDIO_FORCE_DTO_MASK 0x300
+#define AZALIA_AUDIO_DTO_CONTROL__AZALIA_AUDIO_FORCE_DTO__SHIFT 0x8
+#define AZALIA_SCLK_CONTROL__AUDIO_SCLK_CONTROL_MASK 0x30
+#define AZALIA_SCLK_CONTROL__AUDIO_SCLK_CONTROL__SHIFT 0x4
+#define AZALIA_UNDERFLOW_FILLER_SAMPLE__AZALIA_UNDERFLOW_FILLER_SAMPLE_MASK 0xffffffff
+#define AZALIA_UNDERFLOW_FILLER_SAMPLE__AZALIA_UNDERFLOW_FILLER_SAMPLE__SHIFT 0x0
+#define AZALIA_DATA_DMA_CONTROL__DATA_DMA_NON_SNOOP_MASK 0x3
+#define AZALIA_DATA_DMA_CONTROL__DATA_DMA_NON_SNOOP__SHIFT 0x0
+#define AZALIA_DATA_DMA_CONTROL__INPUT_DATA_DMA_NON_SNOOP_MASK 0xc
+#define AZALIA_DATA_DMA_CONTROL__INPUT_DATA_DMA_NON_SNOOP__SHIFT 0x2
+#define AZALIA_DATA_DMA_CONTROL__DATA_DMA_ISOCHRONOUS_MASK 0x30
+#define AZALIA_DATA_DMA_CONTROL__DATA_DMA_ISOCHRONOUS__SHIFT 0x4
+#define AZALIA_DATA_DMA_CONTROL__INPUT_DATA_DMA_ISOCHRONOUS_MASK 0xc0
+#define AZALIA_DATA_DMA_CONTROL__INPUT_DATA_DMA_ISOCHRONOUS__SHIFT 0x6
+#define AZALIA_DATA_DMA_CONTROL__AZALIA_IOC_GENERATION_METHOD_MASK 0x10000
+#define AZALIA_DATA_DMA_CONTROL__AZALIA_IOC_GENERATION_METHOD__SHIFT 0x10
+#define AZALIA_DATA_DMA_CONTROL__AZALIA_UNDERFLOW_CONTROL_MASK 0x20000
+#define AZALIA_DATA_DMA_CONTROL__AZALIA_UNDERFLOW_CONTROL__SHIFT 0x11
+#define AZALIA_BDL_DMA_CONTROL__BDL_DMA_NON_SNOOP_MASK 0x3
+#define AZALIA_BDL_DMA_CONTROL__BDL_DMA_NON_SNOOP__SHIFT 0x0
+#define AZALIA_BDL_DMA_CONTROL__INPUT_BDL_DMA_NON_SNOOP_MASK 0xc
+#define AZALIA_BDL_DMA_CONTROL__INPUT_BDL_DMA_NON_SNOOP__SHIFT 0x2
+#define AZALIA_BDL_DMA_CONTROL__BDL_DMA_ISOCHRONOUS_MASK 0x30
+#define AZALIA_BDL_DMA_CONTROL__BDL_DMA_ISOCHRONOUS__SHIFT 0x4
+#define AZALIA_BDL_DMA_CONTROL__INPUT_BDL_DMA_ISOCHRONOUS_MASK 0xc0
+#define AZALIA_BDL_DMA_CONTROL__INPUT_BDL_DMA_ISOCHRONOUS__SHIFT 0x6
+#define AZALIA_RIRB_AND_DP_CONTROL__RIRB_NON_SNOOP_MASK 0x1
+#define AZALIA_RIRB_AND_DP_CONTROL__RIRB_NON_SNOOP__SHIFT 0x0
+#define AZALIA_RIRB_AND_DP_CONTROL__DP_DMA_NON_SNOOP_MASK 0x10
+#define AZALIA_RIRB_AND_DP_CONTROL__DP_DMA_NON_SNOOP__SHIFT 0x4
+#define AZALIA_RIRB_AND_DP_CONTROL__DP_UPDATE_FREQ_DIVIDER_MASK 0x1e0
+#define AZALIA_RIRB_AND_DP_CONTROL__DP_UPDATE_FREQ_DIVIDER__SHIFT 0x5
+#define AZALIA_CORB_DMA_CONTROL__CORB_DMA_NON_SNOOP_MASK 0x1
+#define AZALIA_CORB_DMA_CONTROL__CORB_DMA_NON_SNOOP__SHIFT 0x0
+#define AZALIA_CORB_DMA_CONTROL__CORB_DMA_ISOCHRONOUS_MASK 0x10
+#define AZALIA_CORB_DMA_CONTROL__CORB_DMA_ISOCHRONOUS__SHIFT 0x4
+#define AZALIA_APPLICATION_POSITION_IN_CYCLIC_BUFFER__APPLICATION_POSITION_IN_CYCLIC_BUFFER_MASK 0xffffffff
+#define AZALIA_APPLICATION_POSITION_IN_CYCLIC_BUFFER__APPLICATION_POSITION_IN_CYCLIC_BUFFER__SHIFT 0x0
+#define AZALIA_CYCLIC_BUFFER_SYNC__CYCLIC_BUFFER_SYNC_ENABLE_MASK 0x1
+#define AZALIA_CYCLIC_BUFFER_SYNC__CYCLIC_BUFFER_SYNC_ENABLE__SHIFT 0x0
+#define AZALIA_GLOBAL_CAPABILITIES__NUMBER_OF_SERIAL_DATA_OUTPUT_SIGNALS_MASK 0x6
+#define AZALIA_GLOBAL_CAPABILITIES__NUMBER_OF_SERIAL_DATA_OUTPUT_SIGNALS__SHIFT 0x1
+#define AZALIA_OUTPUT_PAYLOAD_CAPABILITY__OUTPUT_PAYLOAD_CAPABILITY_MASK 0xffff
+#define AZALIA_OUTPUT_PAYLOAD_CAPABILITY__OUTPUT_PAYLOAD_CAPABILITY__SHIFT 0x0
+#define AZALIA_OUTPUT_PAYLOAD_CAPABILITY__OUTSTRMPAY_MASK 0xffff0000
+#define AZALIA_OUTPUT_PAYLOAD_CAPABILITY__OUTSTRMPAY__SHIFT 0x10
+#define AZALIA_OUTPUT_STREAM_ARBITER_CONTROL__LATENCY_HIDING_LEVEL_MASK 0xff
+#define AZALIA_OUTPUT_STREAM_ARBITER_CONTROL__LATENCY_HIDING_LEVEL__SHIFT 0x0
+#define AZALIA_OUTPUT_STREAM_ARBITER_CONTROL__SYS_MEM_ACTIVE_ENABLE_MASK 0x100
+#define AZALIA_OUTPUT_STREAM_ARBITER_CONTROL__SYS_MEM_ACTIVE_ENABLE__SHIFT 0x8
+#define AZALIA_OUTPUT_STREAM_ARBITER_CONTROL__INPUT_LATENCY_HIDING_LEVEL_MASK 0xff0000
+#define AZALIA_OUTPUT_STREAM_ARBITER_CONTROL__INPUT_LATENCY_HIDING_LEVEL__SHIFT 0x10
+#define AZALIA_INPUT_PAYLOAD_CAPABILITY__INPUT_PAYLOAD_CAPABILITY_MASK 0xffff
+#define AZALIA_INPUT_PAYLOAD_CAPABILITY__INPUT_PAYLOAD_CAPABILITY__SHIFT 0x0
+#define AZALIA_INPUT_PAYLOAD_CAPABILITY__INSTRMPAY_MASK 0xffff0000
+#define AZALIA_INPUT_PAYLOAD_CAPABILITY__INSTRMPAY__SHIFT 0x10
+#define AZALIA_CONTROLLER_DEBUG__CONTROLLER_DEBUG_MASK 0xffffffff
+#define AZALIA_CONTROLLER_DEBUG__CONTROLLER_DEBUG__SHIFT 0x0
+#define AZALIA_MEM_PWR_CTRL__AZ_MEM_PWR_FORCE_MASK 0x3
+#define AZALIA_MEM_PWR_CTRL__AZ_MEM_PWR_FORCE__SHIFT 0x0
+#define AZALIA_MEM_PWR_CTRL__AZ_MEM_PWR_DIS_MASK 0x4
+#define AZALIA_MEM_PWR_CTRL__AZ_MEM_PWR_DIS__SHIFT 0x2
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM0_MEM_PWR_FORCE_MASK 0x18
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM0_MEM_PWR_FORCE__SHIFT 0x3
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM0_MEM_PWR_DIS_MASK 0x20
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM0_MEM_PWR_DIS__SHIFT 0x5
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM1_MEM_PWR_FORCE_MASK 0xc0
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM1_MEM_PWR_FORCE__SHIFT 0x6
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM1_MEM_PWR_DIS_MASK 0x100
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM1_MEM_PWR_DIS__SHIFT 0x8
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM2_MEM_PWR_FORCE_MASK 0x600
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM2_MEM_PWR_FORCE__SHIFT 0x9
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM2_MEM_PWR_DIS_MASK 0x800
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM2_MEM_PWR_DIS__SHIFT 0xb
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM3_MEM_PWR_FORCE_MASK 0x3000
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM3_MEM_PWR_FORCE__SHIFT 0xc
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM3_MEM_PWR_DIS_MASK 0x4000
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM3_MEM_PWR_DIS__SHIFT 0xe
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM4_MEM_PWR_FORCE_MASK 0x18000
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM4_MEM_PWR_FORCE__SHIFT 0xf
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM4_MEM_PWR_DIS_MASK 0x20000
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM4_MEM_PWR_DIS__SHIFT 0x11
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM5_MEM_PWR_FORCE_MASK 0xc0000
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM5_MEM_PWR_FORCE__SHIFT 0x12
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM5_MEM_PWR_DIS_MASK 0x100000
+#define AZALIA_MEM_PWR_CTRL__AZ_INPUT_STREAM5_MEM_PWR_DIS__SHIFT 0x14
+#define AZALIA_MEM_PWR_CTRL__AZ_MEM_PWR_MODE_SEL_MASK 0x30000000
+#define AZALIA_MEM_PWR_CTRL__AZ_MEM_PWR_MODE_SEL__SHIFT 0x1c
+#define AZALIA_MEM_PWR_STATUS__AZ_MEM_PWR_STATE_MASK 0x3
+#define AZALIA_MEM_PWR_STATUS__AZ_MEM_PWR_STATE__SHIFT 0x0
+#define AZALIA_MEM_PWR_STATUS__AZ_INPUT_STREAM0_MEM_PWR_STATE_MASK 0xc
+#define AZALIA_MEM_PWR_STATUS__AZ_INPUT_STREAM0_MEM_PWR_STATE__SHIFT 0x2
+#define AZALIA_MEM_PWR_STATUS__AZ_INPUT_STREAM1_MEM_PWR_STATE_MASK 0x30
+#define AZALIA_MEM_PWR_STATUS__AZ_INPUT_STREAM1_MEM_PWR_STATE__SHIFT 0x4
+#define AZALIA_MEM_PWR_STATUS__AZ_INPUT_STREAM2_MEM_PWR_STATE_MASK 0xc0
+#define AZALIA_MEM_PWR_STATUS__AZ_INPUT_STREAM2_MEM_PWR_STATE__SHIFT 0x6
+#define AZALIA_MEM_PWR_STATUS__AZ_INPUT_STREAM3_MEM_PWR_STATE_MASK 0x300
+#define AZALIA_MEM_PWR_STATUS__AZ_INPUT_STREAM3_MEM_PWR_STATE__SHIFT 0x8
+#define AZALIA_MEM_PWR_STATUS__AZ_INPUT_STREAM4_MEM_PWR_STATE_MASK 0xc00
+#define AZALIA_MEM_PWR_STATUS__AZ_INPUT_STREAM4_MEM_PWR_STATE__SHIFT 0xa
+#define AZALIA_MEM_PWR_STATUS__AZ_INPUT_STREAM5_MEM_PWR_STATE_MASK 0x3000
+#define AZALIA_MEM_PWR_STATUS__AZ_INPUT_STREAM5_MEM_PWR_STATE__SHIFT 0xc
+#define DCI_PG_DEBUG_CONFIG__DCI_PG_DBG_EN_MASK 0x1
+#define DCI_PG_DEBUG_CONFIG__DCI_PG_DBG_EN__SHIFT 0x0
+#define AZALIA_INPUT_CRC0_CONTROL0__INPUT_CRC_EN_MASK 0x1
+#define AZALIA_INPUT_CRC0_CONTROL0__INPUT_CRC_EN__SHIFT 0x0
+#define AZALIA_INPUT_CRC0_CONTROL0__INPUT_CRC_BLOCK_MODE_MASK 0x10
+#define AZALIA_INPUT_CRC0_CONTROL0__INPUT_CRC_BLOCK_MODE__SHIFT 0x4
+#define AZALIA_INPUT_CRC0_CONTROL0__INPUT_CRC_INSTANCE_SEL_MASK 0x700
+#define AZALIA_INPUT_CRC0_CONTROL0__INPUT_CRC_INSTANCE_SEL__SHIFT 0x8
+#define AZALIA_INPUT_CRC0_CONTROL1__INPUT_CRC_BLOCK_SIZE_MASK 0xffffffff
+#define AZALIA_INPUT_CRC0_CONTROL1__INPUT_CRC_BLOCK_SIZE__SHIFT 0x0
+#define AZALIA_INPUT_CRC0_CONTROL2__INPUT_CRC_BLOCK_ITERATION_MASK 0xffff
+#define AZALIA_INPUT_CRC0_CONTROL2__INPUT_CRC_BLOCK_ITERATION__SHIFT 0x0
+#define AZALIA_INPUT_CRC0_CONTROL3__INPUT_CRC_COMPLETE_MASK 0x1
+#define AZALIA_INPUT_CRC0_CONTROL3__INPUT_CRC_COMPLETE__SHIFT 0x0
+#define AZALIA_INPUT_CRC0_CONTROL3__INPUT_CRC_BLOCK_COMPLETE_PHASE_MASK 0x10
+#define AZALIA_INPUT_CRC0_CONTROL3__INPUT_CRC_BLOCK_COMPLETE_PHASE__SHIFT 0x4
+#define AZALIA_INPUT_CRC0_CONTROL3__INPUT_CRC_CHANNEL_RESULT_SEL_MASK 0x700
+#define AZALIA_INPUT_CRC0_CONTROL3__INPUT_CRC_CHANNEL_RESULT_SEL__SHIFT 0x8
+#define AZALIA_INPUT_CRC0_RESULT__INPUT_CRC_RESULT_MASK 0xffffffff
+#define AZALIA_INPUT_CRC0_RESULT__INPUT_CRC_RESULT__SHIFT 0x0
+#define AZALIA_INPUT_CRC0_CHANNEL0__INPUT_CRC_CHANNEL0_MASK 0xffffffff
+#define AZALIA_INPUT_CRC0_CHANNEL0__INPUT_CRC_CHANNEL0__SHIFT 0x0
+#define AZALIA_INPUT_CRC0_CHANNEL1__INPUT_CRC_CHANNEL1_MASK 0xffffffff
+#define AZALIA_INPUT_CRC0_CHANNEL1__INPUT_CRC_CHANNEL1__SHIFT 0x0
+#define AZALIA_INPUT_CRC0_CHANNEL2__INPUT_CRC_CHANNEL2_MASK 0xffffffff
+#define AZALIA_INPUT_CRC0_CHANNEL2__INPUT_CRC_CHANNEL2__SHIFT 0x0
+#define AZALIA_INPUT_CRC0_CHANNEL3__INPUT_CRC_CHANNEL3_MASK 0xffffffff
+#define AZALIA_INPUT_CRC0_CHANNEL3__INPUT_CRC_CHANNEL3__SHIFT 0x0
+#define AZALIA_INPUT_CRC0_CHANNEL4__INPUT_CRC_CHANNEL4_MASK 0xffffffff
+#define AZALIA_INPUT_CRC0_CHANNEL4__INPUT_CRC_CHANNEL4__SHIFT 0x0
+#define AZALIA_INPUT_CRC0_CHANNEL5__INPUT_CRC_CHANNEL5_MASK 0xffffffff
+#define AZALIA_INPUT_CRC0_CHANNEL5__INPUT_CRC_CHANNEL5__SHIFT 0x0
+#define AZALIA_INPUT_CRC0_CHANNEL6__INPUT_CRC_CHANNEL6_MASK 0xffffffff
+#define AZALIA_INPUT_CRC0_CHANNEL6__INPUT_CRC_CHANNEL6__SHIFT 0x0
+#define AZALIA_INPUT_CRC0_CHANNEL7__INPUT_CRC_CHANNEL7_MASK 0xffffffff
+#define AZALIA_INPUT_CRC0_CHANNEL7__INPUT_CRC_CHANNEL7__SHIFT 0x0
+#define AZALIA_INPUT_CRC1_CONTROL0__INPUT_CRC_EN_MASK 0x1
+#define AZALIA_INPUT_CRC1_CONTROL0__INPUT_CRC_EN__SHIFT 0x0
+#define AZALIA_INPUT_CRC1_CONTROL0__INPUT_CRC_BLOCK_MODE_MASK 0x10
+#define AZALIA_INPUT_CRC1_CONTROL0__INPUT_CRC_BLOCK_MODE__SHIFT 0x4
+#define AZALIA_INPUT_CRC1_CONTROL0__INPUT_CRC_INSTANCE_SEL_MASK 0x700
+#define AZALIA_INPUT_CRC1_CONTROL0__INPUT_CRC_INSTANCE_SEL__SHIFT 0x8
+#define AZALIA_INPUT_CRC1_CONTROL1__INPUT_CRC_BLOCK_SIZE_MASK 0xffffffff
+#define AZALIA_INPUT_CRC1_CONTROL1__INPUT_CRC_BLOCK_SIZE__SHIFT 0x0
+#define AZALIA_INPUT_CRC1_CONTROL2__INPUT_CRC_BLOCK_ITERATION_MASK 0xffff
+#define AZALIA_INPUT_CRC1_CONTROL2__INPUT_CRC_BLOCK_ITERATION__SHIFT 0x0
+#define AZALIA_INPUT_CRC1_CONTROL3__INPUT_CRC_COMPLETE_MASK 0x1
+#define AZALIA_INPUT_CRC1_CONTROL3__INPUT_CRC_COMPLETE__SHIFT 0x0
+#define AZALIA_INPUT_CRC1_CONTROL3__INPUT_CRC_BLOCK_COMPLETE_PHASE_MASK 0x10
+#define AZALIA_INPUT_CRC1_CONTROL3__INPUT_CRC_BLOCK_COMPLETE_PHASE__SHIFT 0x4
+#define AZALIA_INPUT_CRC1_CONTROL3__INPUT_CRC_CHANNEL_RESULT_SEL_MASK 0x700
+#define AZALIA_INPUT_CRC1_CONTROL3__INPUT_CRC_CHANNEL_RESULT_SEL__SHIFT 0x8
+#define AZALIA_INPUT_CRC1_RESULT__INPUT_CRC_RESULT_MASK 0xffffffff
+#define AZALIA_INPUT_CRC1_RESULT__INPUT_CRC_RESULT__SHIFT 0x0
+#define AZALIA_INPUT_CRC1_CHANNEL0__INPUT_CRC_CHANNEL0_MASK 0xffffffff
+#define AZALIA_INPUT_CRC1_CHANNEL0__INPUT_CRC_CHANNEL0__SHIFT 0x0
+#define AZALIA_INPUT_CRC1_CHANNEL1__INPUT_CRC_CHANNEL1_MASK 0xffffffff
+#define AZALIA_INPUT_CRC1_CHANNEL1__INPUT_CRC_CHANNEL1__SHIFT 0x0
+#define AZALIA_INPUT_CRC1_CHANNEL2__INPUT_CRC_CHANNEL2_MASK 0xffffffff
+#define AZALIA_INPUT_CRC1_CHANNEL2__INPUT_CRC_CHANNEL2__SHIFT 0x0
+#define AZALIA_INPUT_CRC1_CHANNEL3__INPUT_CRC_CHANNEL3_MASK 0xffffffff
+#define AZALIA_INPUT_CRC1_CHANNEL3__INPUT_CRC_CHANNEL3__SHIFT 0x0
+#define AZALIA_INPUT_CRC1_CHANNEL4__INPUT_CRC_CHANNEL4_MASK 0xffffffff
+#define AZALIA_INPUT_CRC1_CHANNEL4__INPUT_CRC_CHANNEL4__SHIFT 0x0
+#define AZALIA_INPUT_CRC1_CHANNEL5__INPUT_CRC_CHANNEL5_MASK 0xffffffff
+#define AZALIA_INPUT_CRC1_CHANNEL5__INPUT_CRC_CHANNEL5__SHIFT 0x0
+#define AZALIA_INPUT_CRC1_CHANNEL6__INPUT_CRC_CHANNEL6_MASK 0xffffffff
+#define AZALIA_INPUT_CRC1_CHANNEL6__INPUT_CRC_CHANNEL6__SHIFT 0x0
+#define AZALIA_INPUT_CRC1_CHANNEL7__INPUT_CRC_CHANNEL7_MASK 0xffffffff
+#define AZALIA_INPUT_CRC1_CHANNEL7__INPUT_CRC_CHANNEL7__SHIFT 0x0
+#define AZALIA_CRC0_CONTROL0__CRC_EN_MASK 0x1
+#define AZALIA_CRC0_CONTROL0__CRC_EN__SHIFT 0x0
+#define AZALIA_CRC0_CONTROL0__CRC_BLOCK_MODE_MASK 0x10
+#define AZALIA_CRC0_CONTROL0__CRC_BLOCK_MODE__SHIFT 0x4
+#define AZALIA_CRC0_CONTROL0__CRC_INSTANCE_SEL_MASK 0x700
+#define AZALIA_CRC0_CONTROL0__CRC_INSTANCE_SEL__SHIFT 0x8
+#define AZALIA_CRC0_CONTROL0__CRC_SOURCE_SEL_MASK 0x1000
+#define AZALIA_CRC0_CONTROL0__CRC_SOURCE_SEL__SHIFT 0xc
+#define AZALIA_CRC0_CONTROL1__CRC_BLOCK_SIZE_MASK 0xffffffff
+#define AZALIA_CRC0_CONTROL1__CRC_BLOCK_SIZE__SHIFT 0x0
+#define AZALIA_CRC0_CONTROL2__CRC_BLOCK_ITERATION_MASK 0xffff
+#define AZALIA_CRC0_CONTROL2__CRC_BLOCK_ITERATION__SHIFT 0x0
+#define AZALIA_CRC0_CONTROL3__CRC_COMPLETE_MASK 0x1
+#define AZALIA_CRC0_CONTROL3__CRC_COMPLETE__SHIFT 0x0
+#define AZALIA_CRC0_CONTROL3__CRC_BLOCK_COMPLETE_PHASE_MASK 0x10
+#define AZALIA_CRC0_CONTROL3__CRC_BLOCK_COMPLETE_PHASE__SHIFT 0x4
+#define AZALIA_CRC0_CONTROL3__CRC_CHANNEL_RESULT_SEL_MASK 0x700
+#define AZALIA_CRC0_CONTROL3__CRC_CHANNEL_RESULT_SEL__SHIFT 0x8
+#define AZALIA_CRC0_RESULT__CRC_RESULT_MASK 0xffffffff
+#define AZALIA_CRC0_RESULT__CRC_RESULT__SHIFT 0x0
+#define AZALIA_CRC0_CHANNEL0__CRC_CHANNEL0_MASK 0xffffffff
+#define AZALIA_CRC0_CHANNEL0__CRC_CHANNEL0__SHIFT 0x0
+#define AZALIA_CRC0_CHANNEL1__CRC_CHANNEL1_MASK 0xffffffff
+#define AZALIA_CRC0_CHANNEL1__CRC_CHANNEL1__SHIFT 0x0
+#define AZALIA_CRC0_CHANNEL2__CRC_CHANNEL2_MASK 0xffffffff
+#define AZALIA_CRC0_CHANNEL2__CRC_CHANNEL2__SHIFT 0x0
+#define AZALIA_CRC0_CHANNEL3__CRC_CHANNEL3_MASK 0xffffffff
+#define AZALIA_CRC0_CHANNEL3__CRC_CHANNEL3__SHIFT 0x0
+#define AZALIA_CRC0_CHANNEL4__CRC_CHANNEL4_MASK 0xffffffff
+#define AZALIA_CRC0_CHANNEL4__CRC_CHANNEL4__SHIFT 0x0
+#define AZALIA_CRC0_CHANNEL5__CRC_CHANNEL5_MASK 0xffffffff
+#define AZALIA_CRC0_CHANNEL5__CRC_CHANNEL5__SHIFT 0x0
+#define AZALIA_CRC0_CHANNEL6__CRC_CHANNEL6_MASK 0xffffffff
+#define AZALIA_CRC0_CHANNEL6__CRC_CHANNEL6__SHIFT 0x0
+#define AZALIA_CRC0_CHANNEL7__CRC_CHANNEL7_MASK 0xffffffff
+#define AZALIA_CRC0_CHANNEL7__CRC_CHANNEL7__SHIFT 0x0
+#define AZALIA_CRC1_CONTROL0__CRC_EN_MASK 0x1
+#define AZALIA_CRC1_CONTROL0__CRC_EN__SHIFT 0x0
+#define AZALIA_CRC1_CONTROL0__CRC_BLOCK_MODE_MASK 0x10
+#define AZALIA_CRC1_CONTROL0__CRC_BLOCK_MODE__SHIFT 0x4
+#define AZALIA_CRC1_CONTROL0__CRC_INSTANCE_SEL_MASK 0x700
+#define AZALIA_CRC1_CONTROL0__CRC_INSTANCE_SEL__SHIFT 0x8
+#define AZALIA_CRC1_CONTROL0__CRC_SOURCE_SEL_MASK 0x1000
+#define AZALIA_CRC1_CONTROL0__CRC_SOURCE_SEL__SHIFT 0xc
+#define AZALIA_CRC1_CONTROL1__CRC_BLOCK_SIZE_MASK 0xffffffff
+#define AZALIA_CRC1_CONTROL1__CRC_BLOCK_SIZE__SHIFT 0x0
+#define AZALIA_CRC1_CONTROL2__CRC_BLOCK_ITERATION_MASK 0xffff
+#define AZALIA_CRC1_CONTROL2__CRC_BLOCK_ITERATION__SHIFT 0x0
+#define AZALIA_CRC1_CONTROL3__CRC_COMPLETE_MASK 0x1
+#define AZALIA_CRC1_CONTROL3__CRC_COMPLETE__SHIFT 0x0
+#define AZALIA_CRC1_CONTROL3__CRC_BLOCK_COMPLETE_PHASE_MASK 0x10
+#define AZALIA_CRC1_CONTROL3__CRC_BLOCK_COMPLETE_PHASE__SHIFT 0x4
+#define AZALIA_CRC1_CONTROL3__CRC_CHANNEL_RESULT_SEL_MASK 0x700
+#define AZALIA_CRC1_CONTROL3__CRC_CHANNEL_RESULT_SEL__SHIFT 0x8
+#define AZALIA_CRC1_RESULT__CRC_RESULT_MASK 0xffffffff
+#define AZALIA_CRC1_RESULT__CRC_RESULT__SHIFT 0x0
+#define AZALIA_CRC1_CHANNEL0__CRC_CHANNEL0_MASK 0xffffffff
+#define AZALIA_CRC1_CHANNEL0__CRC_CHANNEL0__SHIFT 0x0
+#define AZALIA_CRC1_CHANNEL1__CRC_CHANNEL1_MASK 0xffffffff
+#define AZALIA_CRC1_CHANNEL1__CRC_CHANNEL1__SHIFT 0x0
+#define AZALIA_CRC1_CHANNEL2__CRC_CHANNEL2_MASK 0xffffffff
+#define AZALIA_CRC1_CHANNEL2__CRC_CHANNEL2__SHIFT 0x0
+#define AZALIA_CRC1_CHANNEL3__CRC_CHANNEL3_MASK 0xffffffff
+#define AZALIA_CRC1_CHANNEL3__CRC_CHANNEL3__SHIFT 0x0
+#define AZALIA_CRC1_CHANNEL4__CRC_CHANNEL4_MASK 0xffffffff
+#define AZALIA_CRC1_CHANNEL4__CRC_CHANNEL4__SHIFT 0x0
+#define AZALIA_CRC1_CHANNEL5__CRC_CHANNEL5_MASK 0xffffffff
+#define AZALIA_CRC1_CHANNEL5__CRC_CHANNEL5__SHIFT 0x0
+#define AZALIA_CRC1_CHANNEL6__CRC_CHANNEL6_MASK 0xffffffff
+#define AZALIA_CRC1_CHANNEL6__CRC_CHANNEL6__SHIFT 0x0
+#define AZALIA_CRC1_CHANNEL7__CRC_CHANNEL7_MASK 0xffffffff
+#define AZALIA_CRC1_CHANNEL7__CRC_CHANNEL7__SHIFT 0x0
+#define AZ_TEST_DEBUG_INDEX__AZ_TEST_DEBUG_INDEX_MASK 0xff
+#define AZ_TEST_DEBUG_INDEX__AZ_TEST_DEBUG_INDEX__SHIFT 0x0
+#define AZ_TEST_DEBUG_INDEX__AZ_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define AZ_TEST_DEBUG_INDEX__AZ_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define AZ_TEST_DEBUG_DATA__AZ_TEST_DEBUG_DATA_MASK 0xffffffff
+#define AZ_TEST_DEBUG_DATA__AZ_TEST_DEBUG_DATA__SHIFT 0x0
+#define AZALIA_STREAM_INDEX__AZALIA_STREAM_REG_INDEX_MASK 0xff
+#define AZALIA_STREAM_INDEX__AZALIA_STREAM_REG_INDEX__SHIFT 0x0
+#define AZALIA_STREAM_INDEX__AZALIA_STREAM_REG_WRITE_EN_MASK 0x100
+#define AZALIA_STREAM_INDEX__AZALIA_STREAM_REG_WRITE_EN__SHIFT 0x8
+#define AZALIA_STREAM_DATA__AZALIA_STREAM_REG_DATA_MASK 0xffffffff
+#define AZALIA_STREAM_DATA__AZALIA_STREAM_REG_DATA__SHIFT 0x0
+#define AZALIA_FIFO_SIZE_CONTROL__MIN_FIFO_SIZE_MASK 0x7f
+#define AZALIA_FIFO_SIZE_CONTROL__MIN_FIFO_SIZE__SHIFT 0x0
+#define AZALIA_FIFO_SIZE_CONTROL__MAX_FIFO_SIZE_MASK 0x7f00
+#define AZALIA_FIFO_SIZE_CONTROL__MAX_FIFO_SIZE__SHIFT 0x8
+#define AZALIA_FIFO_SIZE_CONTROL__MAX_LATENCY_SUPPORT_MASK 0xff0000
+#define AZALIA_FIFO_SIZE_CONTROL__MAX_LATENCY_SUPPORT__SHIFT 0x10
+#define AZALIA_LATENCY_COUNTER_CONTROL__AZALIA_LATENCY_COUNTER_RESET_MASK 0x1
+#define AZALIA_LATENCY_COUNTER_CONTROL__AZALIA_LATENCY_COUNTER_RESET__SHIFT 0x0
+#define AZALIA_WORSTCASE_LATENCY_COUNT__AZALIA_WORSTCASE_LATENCY_COUNT_MASK 0xffffffff
+#define AZALIA_WORSTCASE_LATENCY_COUNT__AZALIA_WORSTCASE_LATENCY_COUNT__SHIFT 0x0
+#define AZALIA_CUMULATIVE_LATENCY_COUNT__AZALIA_CUMULATIVE_LATENCY_COUNT_MASK 0xffffffff
+#define AZALIA_CUMULATIVE_LATENCY_COUNT__AZALIA_CUMULATIVE_LATENCY_COUNT__SHIFT 0x0
+#define AZALIA_CUMULATIVE_REQUEST_COUNT__AZALIA_CUMULATIVE_REQUEST_COUNT_MASK 0xffffffff
+#define AZALIA_CUMULATIVE_REQUEST_COUNT__AZALIA_CUMULATIVE_REQUEST_COUNT__SHIFT 0x0
+#define AZALIA_STREAM_DEBUG__STREAM_DEBUG_DATA_MASK 0xffffffff
+#define AZALIA_STREAM_DEBUG__STREAM_DEBUG_DATA__SHIFT 0x0
+#define AZALIA_F0_CODEC_ENDPOINT_INDEX__AZALIA_ENDPOINT_REG_INDEX_MASK 0x3fff
+#define AZALIA_F0_CODEC_ENDPOINT_INDEX__AZALIA_ENDPOINT_REG_INDEX__SHIFT 0x0
+#define AZALIA_F0_CODEC_ENDPOINT_DATA__AZALIA_ENDPOINT_REG_DATA_MASK 0xffffffff
+#define AZALIA_F0_CODEC_ENDPOINT_DATA__AZALIA_ENDPOINT_REG_DATA__SHIFT 0x0
+#define AZALIA_F0_CODEC_CONVERTER_PIN_DEBUG__AZALIA_DEBUG__SHIFT 0x0
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_CHANNEL_CAPABILITIES_MASK 0x1
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_CHANNEL_CAPABILITIES__SHIFT 0x0
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__INPUT_AMPLIFIER_PRESENT_MASK 0x2
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__INPUT_AMPLIFIER_PRESENT__SHIFT 0x1
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__OUTPUT_AMPLIFIER_PRESENT_MASK 0x4
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__OUTPUT_AMPLIFIER_PRESENT__SHIFT 0x2
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AMPLIFIER_PARAMETER_OVERRIDE_MASK 0x8
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AMPLIFIER_PARAMETER_OVERRIDE__SHIFT 0x3
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__FORMAT_OVERRIDE_MASK 0x10
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__FORMAT_OVERRIDE__SHIFT 0x4
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__STRIPE_MASK 0x20
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__STRIPE__SHIFT 0x5
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__PROCESSING_WIDGET_MASK 0x40
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__PROCESSING_WIDGET__SHIFT 0x6
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__UNSOLICITED_RESPONSE_CAPABILITY_MASK 0x80
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__UNSOLICITED_RESPONSE_CAPABILITY__SHIFT 0x7
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__CONNECTION_LIST_MASK 0x100
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__CONNECTION_LIST__SHIFT 0x8
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__DIGITAL_MASK 0x200
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__DIGITAL__SHIFT 0x9
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__POWER_CONTROL_MASK 0x400
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__POWER_CONTROL__SHIFT 0xa
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__LR_SWAP_MASK 0x800
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__LR_SWAP__SHIFT 0xb
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_WIDGET_CAPABILITIES_DELAY_MASK 0xf0000
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_WIDGET_CAPABILITIES_DELAY__SHIFT 0x10
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__TYPE_MASK 0xf00000
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__TYPE__SHIFT 0x14
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__NUMBER_OF_CHANNELS_MASK 0xf
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__NUMBER_OF_CHANNELS__SHIFT 0x0
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__BITS_PER_SAMPLE_MASK 0x70
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__BITS_PER_SAMPLE__SHIFT 0x4
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_DIVISOR_MASK 0x700
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_DIVISOR__SHIFT 0x8
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_MULTIPLE_MASK 0x3800
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_MULTIPLE__SHIFT 0xb
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_RATE_MASK 0x4000
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_RATE__SHIFT 0xe
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__STREAM_TYPE_MASK 0x8000
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_CONVERTER_FORMAT__STREAM_TYPE__SHIFT 0xf
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_CHANNEL_STREAM_ID__CHANNEL_ID_MASK 0xf
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_CHANNEL_STREAM_ID__CHANNEL_ID__SHIFT 0x0
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_CHANNEL_STREAM_ID__STREAM_ID_MASK 0xf0
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_CHANNEL_STREAM_ID__STREAM_ID__SHIFT 0x4
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__DIGEN_MASK 0x1
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__DIGEN__SHIFT 0x0
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__V_MASK 0x2
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__V__SHIFT 0x1
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__VCFG_MASK 0x4
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__VCFG__SHIFT 0x2
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__PRE_MASK 0x8
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__PRE__SHIFT 0x3
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__COPY_MASK 0x10
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__COPY__SHIFT 0x4
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__NON_AUDIO_MASK 0x20
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__NON_AUDIO__SHIFT 0x5
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__PRO_MASK 0x40
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__PRO__SHIFT 0x6
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__L_MASK 0x80
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__L__SHIFT 0x7
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__CC_MASK 0x7f00
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__CC__SHIFT 0x8
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__KEEPALIVE_MASK 0x800000
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_DIGITAL_CONVERTER__KEEPALIVE__SHIFT 0x17
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_STREAM_FORMATS__STREAM_FORMATS_MASK 0xffffffff
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_STREAM_FORMATS__STREAM_FORMATS__SHIFT 0x0
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_RATE_CAPABILITIES_MASK 0xfff
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_RATE_CAPABILITIES__SHIFT 0x0
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_BIT_CAPABILITIES_MASK 0x1f0000
+#define AZALIA_F0_CODEC_CONVERTER_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_BIT_CAPABILITIES__SHIFT 0x10
+#define AZALIA_F0_CODEC_CONVERTER_STRIPE_CONTROL__STRIPE_CONTROL_MASK 0x3
+#define AZALIA_F0_CODEC_CONVERTER_STRIPE_CONTROL__STRIPE_CONTROL__SHIFT 0x0
+#define AZALIA_F0_CODEC_CONVERTER_STRIPE_CONTROL__STRIPE_CAPABILITY_MASK 0x700000
+#define AZALIA_F0_CODEC_CONVERTER_STRIPE_CONTROL__STRIPE_CAPABILITY__SHIFT 0x14
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_RAMP_RATE__RAMP_RATE_MASK 0xff
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_RAMP_RATE__RAMP_RATE__SHIFT 0x0
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_GTC_EMBEDDING__PRESENTATION_TIME_EMBEDDING_ENABLE_MASK 0x1
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_GTC_EMBEDDING__PRESENTATION_TIME_EMBEDDING_ENABLE__SHIFT 0x0
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_GTC_EMBEDDING__PRESENTATION_TIME_OFFSET_CHANGED_MASK 0x2
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_GTC_EMBEDDING__PRESENTATION_TIME_OFFSET_CHANGED__SHIFT 0x1
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_GTC_EMBEDDING__CLEAR_GTC_COUNTER_MIN_MAX_DELTA_MASK 0x4
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_GTC_EMBEDDING__CLEAR_GTC_COUNTER_MIN_MAX_DELTA__SHIFT 0x2
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_GTC_EMBEDDING__PRESENTATION_TIME_EMBEDDING_GROUP_MASK 0x70
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_GTC_EMBEDDING__PRESENTATION_TIME_EMBEDDING_GROUP__SHIFT 0x4
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_GTC_OFFSET_DEBUG__PRESENTATION_TIME_OFFSET_DEBUG_MASK 0xffffffff
+#define AZALIA_F0_CODEC_CONVERTER_CONTROL_GTC_OFFSET_DEBUG__PRESENTATION_TIME_OFFSET_DEBUG__SHIFT 0x0
+#define AZALIA_F0_CODEC_CONVERTER_GTC_COUNTER_DELTA__GTC_COUNTER_DELTA_MASK 0xffffffff
+#define AZALIA_F0_CODEC_CONVERTER_GTC_COUNTER_DELTA__GTC_COUNTER_DELTA__SHIFT 0x0
+#define AZALIA_F0_CODEC_CONVERTER_GTC_COUNTER_DELTA_MIN__GTC_COUNTER_DELTA_MIN_MASK 0xffffffff
+#define AZALIA_F0_CODEC_CONVERTER_GTC_COUNTER_DELTA_MIN__GTC_COUNTER_DELTA_MIN__SHIFT 0x0
+#define AZALIA_F0_CODEC_CONVERTER_GTC_COUNTER_DELTA_MAX__GTC_COUNTER_DELTA_MAX_MASK 0xffffffff
+#define AZALIA_F0_CODEC_CONVERTER_GTC_COUNTER_DELTA_MAX__GTC_COUNTER_DELTA_MAX__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_CHANNEL_CAPABILITIES_MASK 0x1
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_CHANNEL_CAPABILITIES__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__INPUT_AMPLIFIER_PRESENT_MASK 0x2
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__INPUT_AMPLIFIER_PRESENT__SHIFT 0x1
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__OUTPUT_AMPLIFIER_PRESENT_MASK 0x4
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__OUTPUT_AMPLIFIER_PRESENT__SHIFT 0x2
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AMPLIFIER_PARAMETER_OVERRIDE_MASK 0x8
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AMPLIFIER_PARAMETER_OVERRIDE__SHIFT 0x3
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__STRIPE_MASK 0x20
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__STRIPE__SHIFT 0x5
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__PROCESSING_WIDGET_MASK 0x40
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__PROCESSING_WIDGET__SHIFT 0x6
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__UNSOLICITED_RESPONSE_CAPABILITY_MASK 0x80
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__UNSOLICITED_RESPONSE_CAPABILITY__SHIFT 0x7
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__CONNECTION_LIST_MASK 0x100
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__CONNECTION_LIST__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__DIGITAL_MASK 0x200
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__DIGITAL__SHIFT 0x9
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__POWER_CONTROL_MASK 0x400
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__POWER_CONTROL__SHIFT 0xa
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__LR_SWAP_MASK 0x800
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__LR_SWAP__SHIFT 0xb
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_WIDGET_CAPABILITIES_DELAY_MASK 0xf0000
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_WIDGET_CAPABILITIES_DELAY__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__TYPE_MASK 0xf00000
+#define AZALIA_F0_CODEC_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__TYPE__SHIFT 0x14
+#define AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES__IMPEDANCE_SENSE_CAPABLE_MASK 0x1
+#define AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES__IMPEDANCE_SENSE_CAPABLE__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES__TRIGGER_REQUIRED_MASK 0x2
+#define AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES__TRIGGER_REQUIRED__SHIFT 0x1
+#define AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES__JACK_DETECTION_CAPABILITY_MASK 0x4
+#define AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES__JACK_DETECTION_CAPABILITY__SHIFT 0x2
+#define AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES__HEADPHONE_DRIVE_CAPABLE_MASK 0x8
+#define AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES__HEADPHONE_DRIVE_CAPABLE__SHIFT 0x3
+#define AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES__OUTPUT_CAPABLE_MASK 0x10
+#define AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES__OUTPUT_CAPABLE__SHIFT 0x4
+#define AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES__INPUT_CAPABLE_MASK 0x20
+#define AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES__INPUT_CAPABLE__SHIFT 0x5
+#define AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES__BALANCED_I_O_PINS_MASK 0x40
+#define AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES__BALANCED_I_O_PINS__SHIFT 0x6
+#define AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES__HDMI_MASK 0x80
+#define AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES__HDMI__SHIFT 0x7
+#define AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES__VREF_CONTROL_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES__VREF_CONTROL__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES__EAPD_CAPABLE_MASK 0x10000
+#define AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES__EAPD_CAPABLE__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES__DP_MASK 0x1000000
+#define AZALIA_F0_CODEC_PIN_PARAMETER_CAPABILITIES__DP__SHIFT 0x18
+#define AZALIA_F0_CODEC_PIN_CONTROL_UNSOLICITED_RESPONSE__TAG_MASK 0x3f
+#define AZALIA_F0_CODEC_PIN_CONTROL_UNSOLICITED_RESPONSE__TAG__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_UNSOLICITED_RESPONSE__ENABLE_MASK 0x80
+#define AZALIA_F0_CODEC_PIN_CONTROL_UNSOLICITED_RESPONSE__ENABLE__SHIFT 0x7
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_PIN_SENSE__IMPEDANCE_SENSE_MASK 0x7fffffff
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_PIN_SENSE__IMPEDANCE_SENSE__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_WIDGET_CONTROL__OUT_ENABLE_MASK 0x40
+#define AZALIA_F0_CODEC_PIN_CONTROL_WIDGET_CONTROL__OUT_ENABLE__SHIFT 0x6
+#define AZALIA_F0_CODEC_PIN_CONTROL_CHANNEL_SPEAKER__SPEAKER_ALLOCATION_MASK 0x7f
+#define AZALIA_F0_CODEC_PIN_CONTROL_CHANNEL_SPEAKER__SPEAKER_ALLOCATION__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_CHANNEL_SPEAKER__CHANNEL_ALLOCATION_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_CHANNEL_SPEAKER__CHANNEL_ALLOCATION__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_CHANNEL_SPEAKER__HDMI_CONNECTION_MASK 0x10000
+#define AZALIA_F0_CODEC_PIN_CONTROL_CHANNEL_SPEAKER__HDMI_CONNECTION__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_CHANNEL_SPEAKER__DP_CONNECTION_MASK 0x20000
+#define AZALIA_F0_CODEC_PIN_CONTROL_CHANNEL_SPEAKER__DP_CONNECTION__SHIFT 0x11
+#define AZALIA_F0_CODEC_PIN_CONTROL_CHANNEL_SPEAKER__EXTRA_CONNECTION_INFO_MASK 0xfc0000
+#define AZALIA_F0_CODEC_PIN_CONTROL_CHANNEL_SPEAKER__EXTRA_CONNECTION_INFO__SHIFT 0x12
+#define AZALIA_F0_CODEC_PIN_CONTROL_CHANNEL_SPEAKER__LFE_PLAYBACK_LEVEL_MASK 0x3000000
+#define AZALIA_F0_CODEC_PIN_CONTROL_CHANNEL_SPEAKER__LFE_PLAYBACK_LEVEL__SHIFT 0x18
+#define AZALIA_F0_CODEC_PIN_CONTROL_CHANNEL_SPEAKER__LEVEL_SHIFT_MASK 0x78000000
+#define AZALIA_F0_CODEC_PIN_CONTROL_CHANNEL_SPEAKER__LEVEL_SHIFT__SHIFT 0x1b
+#define AZALIA_F0_CODEC_PIN_CONTROL_CHANNEL_SPEAKER__DOWN_MIX_INHIBIT_MASK 0x80000000
+#define AZALIA_F0_CODEC_PIN_CONTROL_CHANNEL_SPEAKER__DOWN_MIX_INHIBIT__SHIFT 0x1f
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR0__MAX_CHANNELS_MASK 0x7
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR0__MAX_CHANNELS__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR0__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR0__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR0__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR0__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR0__SUPPORTED_FREQUENCIES_STEREO_MASK 0xff000000
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR0__SUPPORTED_FREQUENCIES_STEREO__SHIFT 0x18
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR1__MAX_CHANNELS_MASK 0x7
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR1__MAX_CHANNELS__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR1__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR1__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR1__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR1__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR2__MAX_CHANNELS_MASK 0x7
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR2__MAX_CHANNELS__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR2__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR2__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR2__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR2__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR3__MAX_CHANNELS_MASK 0x7
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR3__MAX_CHANNELS__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR3__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR3__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR3__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR3__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR4__MAX_CHANNELS_MASK 0x7
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR4__MAX_CHANNELS__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR4__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR4__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR4__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR4__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR5__MAX_CHANNELS_MASK 0x7
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR5__MAX_CHANNELS__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR5__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR5__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR5__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR5__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR6__MAX_CHANNELS_MASK 0x7
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR6__MAX_CHANNELS__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR6__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR6__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR6__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR6__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR7__MAX_CHANNELS_MASK 0x7
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR7__MAX_CHANNELS__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR7__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR7__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR7__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR7__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR8__MAX_CHANNELS_MASK 0x7
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR8__MAX_CHANNELS__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR8__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR8__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR8__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR8__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR9__MAX_CHANNELS_MASK 0x7
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR9__MAX_CHANNELS__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR9__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR9__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR9__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR9__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR10__MAX_CHANNELS_MASK 0x7
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR10__MAX_CHANNELS__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR10__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR10__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR10__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR10__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR11__MAX_CHANNELS_MASK 0x7
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR11__MAX_CHANNELS__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR11__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR11__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR11__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR11__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR12__MAX_CHANNELS_MASK 0x7
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR12__MAX_CHANNELS__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR12__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR12__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR12__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR12__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR13__MAX_CHANNELS_MASK 0x7
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR13__MAX_CHANNELS__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR13__SUPPORTED_FREQUENCIES_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR13__SUPPORTED_FREQUENCIES__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR13__DESCRIPTOR_BYTE_2_MASK 0xff0000
+#define AZALIA_F0_CODEC_PIN_CONTROL_AUDIO_DESCRIPTOR13__DESCRIPTOR_BYTE_2__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL01_ENABLE_MASK 0x1
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL01_ENABLE__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL01_MUTE_MASK 0x2
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL01_MUTE__SHIFT 0x1
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL01_CHANNEL_ID_MASK 0xf0
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL01_CHANNEL_ID__SHIFT 0x4
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL23_ENABLE_MASK 0x100
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL23_ENABLE__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL23_MUTE_MASK 0x200
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL23_MUTE__SHIFT 0x9
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL23_CHANNEL_ID_MASK 0xf000
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL23_CHANNEL_ID__SHIFT 0xc
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL45_ENABLE_MASK 0x10000
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL45_ENABLE__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL45_MUTE_MASK 0x20000
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL45_MUTE__SHIFT 0x11
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL45_CHANNEL_ID_MASK 0xf00000
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL45_CHANNEL_ID__SHIFT 0x14
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL67_ENABLE_MASK 0x1000000
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL67_ENABLE__SHIFT 0x18
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL67_MUTE_MASK 0x2000000
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL67_MUTE__SHIFT 0x19
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL67_CHANNEL_ID_MASK 0xf0000000
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL67_CHANNEL_ID__SHIFT 0x1c
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL1_ENABLE_MASK 0x1
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL1_ENABLE__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL1_MUTE_MASK 0x2
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL1_MUTE__SHIFT 0x1
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL1_CHANNEL_ID_MASK 0xf0
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL1_CHANNEL_ID__SHIFT 0x4
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL3_ENABLE_MASK 0x100
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL3_ENABLE__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL3_MUTE_MASK 0x200
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL3_MUTE__SHIFT 0x9
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL3_CHANNEL_ID_MASK 0xf000
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL3_CHANNEL_ID__SHIFT 0xc
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL5_ENABLE_MASK 0x10000
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL5_ENABLE__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL5_MUTE_MASK 0x20000
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL5_MUTE__SHIFT 0x11
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL5_CHANNEL_ID_MASK 0xf00000
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL5_CHANNEL_ID__SHIFT 0x14
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL7_ENABLE_MASK 0x1000000
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL7_ENABLE__SHIFT 0x18
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL7_MUTE_MASK 0x2000000
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL7_MUTE__SHIFT 0x19
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL7_CHANNEL_ID_MASK 0xf0000000
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL7_CHANNEL_ID__SHIFT 0x1c
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_MODE__MULTICHANNEL_MODE_MASK 0x1
+#define AZALIA_F0_CODEC_PIN_CONTROL_MULTICHANNEL_MODE__MULTICHANNEL_MODE__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_LIPSYNC__VIDEO_LIPSYNC_MASK 0xff
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_LIPSYNC__VIDEO_LIPSYNC__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_LIPSYNC__AUDIO_LIPSYNC_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_LIPSYNC__AUDIO_LIPSYNC__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_HBR__HBR_CAPABLE_MASK 0x1
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_HBR__HBR_CAPABLE__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_HBR__HBR_ENABLE_MASK 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_HBR__HBR_ENABLE__SHIFT 0x4
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO0__MANUFACTURER_ID_MASK 0xffff
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO0__MANUFACTURER_ID__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO0__PRODUCT_ID_MASK 0xffff0000
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO0__PRODUCT_ID__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO1__SINK_DESCRIPTION_LEN_MASK 0xff
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO1__SINK_DESCRIPTION_LEN__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO2__PORT_ID0_MASK 0xffffffff
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO2__PORT_ID0__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO3__PORT_ID1_MASK 0xffffffff
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO3__PORT_ID1__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO4__DESCRIPTION0_MASK 0xff
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO4__DESCRIPTION0__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO4__DESCRIPTION1_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO4__DESCRIPTION1__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO4__DESCRIPTION2_MASK 0xff0000
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO4__DESCRIPTION2__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO4__DESCRIPTION3_MASK 0xff000000
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO4__DESCRIPTION3__SHIFT 0x18
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO5__DESCRIPTION4_MASK 0xff
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO5__DESCRIPTION4__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO5__DESCRIPTION5_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO5__DESCRIPTION5__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO5__DESCRIPTION6_MASK 0xff0000
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO5__DESCRIPTION6__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO5__DESCRIPTION7_MASK 0xff000000
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO5__DESCRIPTION7__SHIFT 0x18
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO6__DESCRIPTION8_MASK 0xff
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO6__DESCRIPTION8__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO6__DESCRIPTION9_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO6__DESCRIPTION9__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO6__DESCRIPTION10_MASK 0xff0000
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO6__DESCRIPTION10__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO6__DESCRIPTION11_MASK 0xff000000
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO6__DESCRIPTION11__SHIFT 0x18
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO7__DESCRIPTION12_MASK 0xff
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO7__DESCRIPTION12__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO7__DESCRIPTION13_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO7__DESCRIPTION13__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO7__DESCRIPTION14_MASK 0xff0000
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO7__DESCRIPTION14__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO7__DESCRIPTION15_MASK 0xff000000
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO7__DESCRIPTION15__SHIFT 0x18
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO8__DESCRIPTION16_MASK 0xff
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO8__DESCRIPTION16__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO8__DESCRIPTION17_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_SINK_INFO8__DESCRIPTION17__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_HOT_PLUG_CONTROL__CLOCK_GATING_DISABLE_MASK 0x1
+#define AZALIA_F0_CODEC_PIN_CONTROL_HOT_PLUG_CONTROL__CLOCK_GATING_DISABLE__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_HOT_PLUG_CONTROL__CLOCK_ON_STATE_MASK 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_HOT_PLUG_CONTROL__CLOCK_ON_STATE__SHIFT 0x4
+#define AZALIA_F0_CODEC_PIN_CONTROL_HOT_PLUG_CONTROL__AUDIO_ENABLED_MASK 0x80000000
+#define AZALIA_F0_CODEC_PIN_CONTROL_HOT_PLUG_CONTROL__AUDIO_ENABLED__SHIFT 0x1f
+#define AZALIA_F0_CODEC_PIN_CONTROL_UNSOLICITED_RESPONSE_FORCE__UNSOLICITED_RESPONSE_PAYLOAD_MASK 0x3ffffff
+#define AZALIA_F0_CODEC_PIN_CONTROL_UNSOLICITED_RESPONSE_FORCE__UNSOLICITED_RESPONSE_PAYLOAD__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_UNSOLICITED_RESPONSE_FORCE__UNSOLICITED_RESPONSE_FORCE_MASK 0x10000000
+#define AZALIA_F0_CODEC_PIN_CONTROL_UNSOLICITED_RESPONSE_FORCE__UNSOLICITED_RESPONSE_FORCE__SHIFT 0x1c
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__SEQUENCE_MASK 0xf
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__SEQUENCE__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__DEFAULT_ASSOCIATION_MASK 0xf0
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__DEFAULT_ASSOCIATION__SHIFT 0x4
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__MISC_MASK 0xf00
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__MISC__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__COLOR_MASK 0xf000
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__COLOR__SHIFT 0xc
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__CONNECTION_TYPE_MASK 0xf0000
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__CONNECTION_TYPE__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__DEFAULT_DEVICE_MASK 0xf00000
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__DEFAULT_DEVICE__SHIFT 0x14
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__LOCATION_MASK 0x3f000000
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__LOCATION__SHIFT 0x18
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__PORT_CONNECTIVITY_MASK 0xc0000000
+#define AZALIA_F0_CODEC_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__PORT_CONNECTIVITY__SHIFT 0x1e
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_0__IEC_60958_CS_MODE_MASK 0x3
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_0__IEC_60958_CS_MODE__SHIFT 0x0
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_0__IEC_60958_CS_SOURCE_NUMBER_MASK 0x3c
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_0__IEC_60958_CS_SOURCE_NUMBER__SHIFT 0x2
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_1__IEC_60958_CS_CLOCK_ACCURACY_MASK 0x3
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_1__IEC_60958_CS_CLOCK_ACCURACY__SHIFT 0x0
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_1__IEC_60958_CS_CLOCK_ACCURACY_OVRRD_EN_MASK 0x4
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_1__IEC_60958_CS_CLOCK_ACCURACY_OVRRD_EN__SHIFT 0x2
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_1__IEC_60958_CS_WORD_LENGTH_MASK 0x78
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_1__IEC_60958_CS_WORD_LENGTH__SHIFT 0x3
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_1__IEC_60958_CS_WORD_LENGTH_OVRRD_EN_MASK 0x80
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_1__IEC_60958_CS_WORD_LENGTH_OVRRD_EN__SHIFT 0x7
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_2__IEC_60958_CS_SAMPLING_FREQUENCY_MASK 0x3f
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_2__IEC_60958_CS_SAMPLING_FREQUENCY__SHIFT 0x0
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_2__IEC_60958_CS_SAMPLING_FREQUENCY_OVRRD_EN_MASK 0x40
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_2__IEC_60958_CS_SAMPLING_FREQUENCY_OVRRD_EN__SHIFT 0x6
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_3__IEC_60958_CS_ORIGINAL_SAMPLING_FREQUENCY_MASK 0xf
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_3__IEC_60958_CS_ORIGINAL_SAMPLING_FREQUENCY__SHIFT 0x0
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_3__IEC_60958_CS_ORIGINAL_SAMPLING_FREQUENCY_OVRRD_EN_MASK 0x10
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_3__IEC_60958_CS_ORIGINAL_SAMPLING_FREQUENCY_OVRRD_EN__SHIFT 0x4
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_4__IEC_60958_CS_SAMPLING_FREQUENCY_COEFF_MASK 0xf
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_4__IEC_60958_CS_SAMPLING_FREQUENCY_COEFF__SHIFT 0x0
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_4__IEC_60958_CS_MPEG_SURROUND_INFO_MASK 0x10
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_4__IEC_60958_CS_MPEG_SURROUND_INFO__SHIFT 0x4
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_4__IEC_60958_CS_CGMS_A_MASK 0x60
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_4__IEC_60958_CS_CGMS_A__SHIFT 0x5
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_4__IEC_60958_CS_CGMS_A_VALID_MASK 0x80
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_4__IEC_60958_CS_CGMS_A_VALID__SHIFT 0x7
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_5__IEC_60958_CS_CHANNEL_NUMBER_L_MASK 0xf
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_5__IEC_60958_CS_CHANNEL_NUMBER_L__SHIFT 0x0
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_5__IEC_60958_CS_CHANNEL_NUMBER_R_MASK 0xf0
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_5__IEC_60958_CS_CHANNEL_NUMBER_R__SHIFT 0x4
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_6__IEC_60958_CS_CHANNEL_NUMBER_2_MASK 0xf
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_6__IEC_60958_CS_CHANNEL_NUMBER_2__SHIFT 0x0
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_6__IEC_60958_CS_CHANNEL_NUMBER_3_MASK 0xf0
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_6__IEC_60958_CS_CHANNEL_NUMBER_3__SHIFT 0x4
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_7__IEC_60958_CS_CHANNEL_NUMBER_4_MASK 0xf
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_7__IEC_60958_CS_CHANNEL_NUMBER_4__SHIFT 0x0
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_7__IEC_60958_CS_CHANNEL_NUMBER_5_MASK 0xf0
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_7__IEC_60958_CS_CHANNEL_NUMBER_5__SHIFT 0x4
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_8__IEC_60958_CS_CHANNEL_NUMBER_6_MASK 0xf
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_8__IEC_60958_CS_CHANNEL_NUMBER_6__SHIFT 0x0
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_8__IEC_60958_CS_CHANNEL_NUMBER_7_MASK 0xf0
+#define AZALIA_F0_PIN_CONTROL_CODEC_CS_OVERRIDE_8__IEC_60958_CS_CHANNEL_NUMBER_7__SHIFT 0x4
+#define AZALIA_F0_CODEC_PIN_ASSOCIATION_INFO__ASSOCIATION_INFO_MASK 0xffffffff
+#define AZALIA_F0_CODEC_PIN_ASSOCIATION_INFO__ASSOCIATION_INFO__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_DIGITAL_OUTPUT_STATUS__OUTPUT_ACTIVE_MASK 0x1
+#define AZALIA_F0_CODEC_PIN_CONTROL_DIGITAL_OUTPUT_STATUS__OUTPUT_ACTIVE__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_LPIB_SNAPSHOT_CONTROL__LPIB_SNAPSHOT_LOCK_MASK 0x1
+#define AZALIA_F0_CODEC_PIN_CONTROL_LPIB_SNAPSHOT_CONTROL__LPIB_SNAPSHOT_LOCK__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_LPIB_SNAPSHOT_CONTROL__CYCLIC_BUFFER_WRAP_COUNT_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_LPIB_SNAPSHOT_CONTROL__CYCLIC_BUFFER_WRAP_COUNT__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_LPIB__LPIB_MASK 0xffffffff
+#define AZALIA_F0_CODEC_PIN_CONTROL_LPIB__LPIB__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_LPIB_TIMER_SNAPSHOT__LPIB_TIMER_SNAPSHOT_MASK 0xffffffff
+#define AZALIA_F0_CODEC_PIN_CONTROL_LPIB_TIMER_SNAPSHOT__LPIB_TIMER_SNAPSHOT__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_CODING_TYPE__CODING_TYPE_MASK 0xff
+#define AZALIA_F0_CODEC_PIN_CONTROL_CODING_TYPE__CODING_TYPE__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_FORMAT_CHANGED__FORMAT_CHANGED_MASK 0x1
+#define AZALIA_F0_CODEC_PIN_CONTROL_FORMAT_CHANGED__FORMAT_CHANGED__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_FORMAT_CHANGED__FORMAT_CHANGED_ACK_UR_ENABLE_MASK 0x2
+#define AZALIA_F0_CODEC_PIN_CONTROL_FORMAT_CHANGED__FORMAT_CHANGED_ACK_UR_ENABLE__SHIFT 0x1
+#define AZALIA_F0_CODEC_PIN_CONTROL_FORMAT_CHANGED__FORMAT_CHANGE_REASON_MASK 0xff00
+#define AZALIA_F0_CODEC_PIN_CONTROL_FORMAT_CHANGED__FORMAT_CHANGE_REASON__SHIFT 0x8
+#define AZALIA_F0_CODEC_PIN_CONTROL_FORMAT_CHANGED__FORMAT_CHANGE_RESPONSE_MASK 0xff0000
+#define AZALIA_F0_CODEC_PIN_CONTROL_FORMAT_CHANGED__FORMAT_CHANGE_RESPONSE__SHIFT 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_WIRELESS_DISPLAY_IDENTIFICATION__WIRELESS_DISPLAY_IDENTIFICATION_MASK 0x3
+#define AZALIA_F0_CODEC_PIN_CONTROL_WIRELESS_DISPLAY_IDENTIFICATION__WIRELESS_DISPLAY_IDENTIFICATION__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_REMOTE_KEEPALIVE__REMOTE_KEEP_ALIVE_ENABLE_MASK 0x1
+#define AZALIA_F0_CODEC_PIN_CONTROL_REMOTE_KEEPALIVE__REMOTE_KEEP_ALIVE_ENABLE__SHIFT 0x0
+#define AZALIA_F0_CODEC_PIN_CONTROL_REMOTE_KEEPALIVE__REMOTE_KEEP_ALIVE_CAPABILITY_MASK 0x10
+#define AZALIA_F0_CODEC_PIN_CONTROL_REMOTE_KEEPALIVE__REMOTE_KEEP_ALIVE_CAPABILITY__SHIFT 0x4
+#define AZALIA_F0_AUDIO_ENABLE_STATUS__AUDIO_ENABLE_STATUS_MASK 0x1
+#define AZALIA_F0_AUDIO_ENABLE_STATUS__AUDIO_ENABLE_STATUS__SHIFT 0x0
+#define AZALIA_F0_AUDIO_ENABLED_INT_STATUS__AUDIO_ENABLED_FLAG_MASK 0x1
+#define AZALIA_F0_AUDIO_ENABLED_INT_STATUS__AUDIO_ENABLED_FLAG__SHIFT 0x0
+#define AZALIA_F0_AUDIO_ENABLED_INT_STATUS__AUDIO_ENABLED_MASK_MASK 0x10
+#define AZALIA_F0_AUDIO_ENABLED_INT_STATUS__AUDIO_ENABLED_MASK__SHIFT 0x4
+#define AZALIA_F0_AUDIO_ENABLED_INT_STATUS__AUDIO_ENABLED_TYPE_MASK 0x100
+#define AZALIA_F0_AUDIO_ENABLED_INT_STATUS__AUDIO_ENABLED_TYPE__SHIFT 0x8
+#define AZALIA_F0_AUDIO_DISABLED_INT_STATUS__AUDIO_DISABLED_FLAG_MASK 0x1
+#define AZALIA_F0_AUDIO_DISABLED_INT_STATUS__AUDIO_DISABLED_FLAG__SHIFT 0x0
+#define AZALIA_F0_AUDIO_DISABLED_INT_STATUS__AUDIO_DISABLED_MASK_MASK 0x10
+#define AZALIA_F0_AUDIO_DISABLED_INT_STATUS__AUDIO_DISABLED_MASK__SHIFT 0x4
+#define AZALIA_F0_AUDIO_DISABLED_INT_STATUS__AUDIO_DISABLED_TYPE_MASK 0x100
+#define AZALIA_F0_AUDIO_DISABLED_INT_STATUS__AUDIO_DISABLED_TYPE__SHIFT 0x8
+#define AZALIA_F0_AUDIO_FORMAT_CHANGED_INT_STATUS__AUDIO_FORMAT_CHANGED_FLAG_MASK 0x1
+#define AZALIA_F0_AUDIO_FORMAT_CHANGED_INT_STATUS__AUDIO_FORMAT_CHANGED_FLAG__SHIFT 0x0
+#define AZALIA_F0_AUDIO_FORMAT_CHANGED_INT_STATUS__AUDIO_FORMAT_CHANGED_MASK_MASK 0x10
+#define AZALIA_F0_AUDIO_FORMAT_CHANGED_INT_STATUS__AUDIO_FORMAT_CHANGED_MASK__SHIFT 0x4
+#define AZALIA_F0_AUDIO_FORMAT_CHANGED_INT_STATUS__AUDIO_FORMAT_CHANGED_TYPE_MASK 0x100
+#define AZALIA_F0_AUDIO_FORMAT_CHANGED_INT_STATUS__AUDIO_FORMAT_CHANGED_TYPE__SHIFT 0x8
+#define AZALIA_F0_CODEC_INPUT_ENDPOINT_INDEX__AZALIA_INPUT_ENDPOINT_REG_INDEX_MASK 0x3fff
+#define AZALIA_F0_CODEC_INPUT_ENDPOINT_INDEX__AZALIA_INPUT_ENDPOINT_REG_INDEX__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_ENDPOINT_DATA__AZALIA_INPUT_ENDPOINT_REG_DATA_MASK 0xffffffff
+#define AZALIA_F0_CODEC_INPUT_ENDPOINT_DATA__AZALIA_INPUT_ENDPOINT_REG_DATA__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PIN_DEBUG__AZALIA_INPUT_DEBUG_MASK 0xffffffff
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PIN_DEBUG__AZALIA_INPUT_DEBUG__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_CHANNEL_CAPABILITIES_MASK 0x1
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_CHANNEL_CAPABILITIES__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__INPUT_AMPLIFIER_PRESENT_MASK 0x2
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__INPUT_AMPLIFIER_PRESENT__SHIFT 0x1
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__OUTPUT_AMPLIFIER_PRESENT_MASK 0x4
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__OUTPUT_AMPLIFIER_PRESENT__SHIFT 0x2
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AMPLIFIER_PARAMETER_OVERRIDE_MASK 0x8
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AMPLIFIER_PARAMETER_OVERRIDE__SHIFT 0x3
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__FORMAT_OVERRIDE_MASK 0x10
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__FORMAT_OVERRIDE__SHIFT 0x4
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__STRIPE_MASK 0x20
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__STRIPE__SHIFT 0x5
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__PROCESSING_WIDGET_MASK 0x40
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__PROCESSING_WIDGET__SHIFT 0x6
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__UNSOLICITED_RESPONSE_CAPABILITY_MASK 0x80
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__UNSOLICITED_RESPONSE_CAPABILITY__SHIFT 0x7
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__CONNECTION_LIST_MASK 0x100
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__CONNECTION_LIST__SHIFT 0x8
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__DIGITAL_MASK 0x200
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__DIGITAL__SHIFT 0x9
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__POWER_CONTROL_MASK 0x400
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__POWER_CONTROL__SHIFT 0xa
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__LR_SWAP_MASK 0x800
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__LR_SWAP__SHIFT 0xb
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_WIDGET_CAPABILITIES_DELAY_MASK 0xf0000
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_WIDGET_CAPABILITIES_DELAY__SHIFT 0x10
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__TYPE_MASK 0xf00000
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__TYPE__SHIFT 0x14
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__NUMBER_OF_CHANNELS_MASK 0xf
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__NUMBER_OF_CHANNELS__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__BITS_PER_SAMPLE_MASK 0x70
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__BITS_PER_SAMPLE__SHIFT 0x4
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_DIVISOR_MASK 0x700
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_DIVISOR__SHIFT 0x8
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_MULTIPLE_MASK 0x3800
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_MULTIPLE__SHIFT 0xb
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_RATE_MASK 0x4000
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_RATE__SHIFT 0xe
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__STREAM_TYPE_MASK 0x8000
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__STREAM_TYPE__SHIFT 0xf
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_CHANNEL_STREAM_ID__CHANNEL_ID_MASK 0xf
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_CHANNEL_STREAM_ID__CHANNEL_ID__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_CHANNEL_STREAM_ID__STREAM_ID_MASK 0xf0
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_CHANNEL_STREAM_ID__STREAM_ID__SHIFT 0x4
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__DIGEN_MASK 0x1
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__DIGEN__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__V_MASK 0x2
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__V__SHIFT 0x1
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__VCFG_MASK 0x4
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__VCFG__SHIFT 0x2
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__PRE_MASK 0x8
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__PRE__SHIFT 0x3
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__COPY_MASK 0x10
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__COPY__SHIFT 0x4
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__NON_AUDIO_MASK 0x20
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__NON_AUDIO__SHIFT 0x5
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__PRO_MASK 0x40
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__PRO__SHIFT 0x6
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__L_MASK 0x80
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__L__SHIFT 0x7
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__CC_MASK 0x7f00
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__CC__SHIFT 0x8
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__KEEPALIVE_MASK 0x800000
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__KEEPALIVE__SHIFT 0x17
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_STREAM_FORMATS__STREAM_FORMATS_MASK 0xffffffff
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_STREAM_FORMATS__STREAM_FORMATS__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_RATE_CAPABILITIES_MASK 0xfff
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_RATE_CAPABILITIES__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_BIT_CAPABILITIES_MASK 0x1f0000
+#define AZALIA_F0_CODEC_INPUT_CONVERTER_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_BIT_CAPABILITIES__SHIFT 0x10
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_CHANNEL_CAPABILITIES_MASK 0x1
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_CHANNEL_CAPABILITIES__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__INPUT_AMPLIFIER_PRESENT_MASK 0x2
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__INPUT_AMPLIFIER_PRESENT__SHIFT 0x1
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__OUTPUT_AMPLIFIER_PRESENT_MASK 0x4
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__OUTPUT_AMPLIFIER_PRESENT__SHIFT 0x2
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AMPLIFIER_PARAMETER_OVERRIDE_MASK 0x8
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AMPLIFIER_PARAMETER_OVERRIDE__SHIFT 0x3
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__STRIPE_MASK 0x20
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__STRIPE__SHIFT 0x5
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__PROCESSING_WIDGET_MASK 0x40
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__PROCESSING_WIDGET__SHIFT 0x6
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__UNSOLICITED_RESPONSE_CAPABILITY_MASK 0x80
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__UNSOLICITED_RESPONSE_CAPABILITY__SHIFT 0x7
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__CONNECTION_LIST_MASK 0x100
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__CONNECTION_LIST__SHIFT 0x8
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__DIGITAL_MASK 0x200
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__DIGITAL__SHIFT 0x9
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__POWER_CONTROL_MASK 0x400
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__POWER_CONTROL__SHIFT 0xa
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__LR_SWAP_MASK 0x800
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__LR_SWAP__SHIFT 0xb
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_WIDGET_CAPABILITIES_DELAY_MASK 0xf0000
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_WIDGET_CAPABILITIES_DELAY__SHIFT 0x10
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__TYPE_MASK 0xf00000
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__TYPE__SHIFT 0x14
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__IMPEDANCE_SENSE_CAPABLE_MASK 0x1
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__IMPEDANCE_SENSE_CAPABLE__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__TRIGGER_REQUIRED_MASK 0x2
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__TRIGGER_REQUIRED__SHIFT 0x1
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__JACK_DETECTION_CAPABILITY_MASK 0x4
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__JACK_DETECTION_CAPABILITY__SHIFT 0x2
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__HEADPHONE_DRIVE_CAPABLE_MASK 0x8
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__HEADPHONE_DRIVE_CAPABLE__SHIFT 0x3
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__OUTPUT_CAPABLE_MASK 0x10
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__OUTPUT_CAPABLE__SHIFT 0x4
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__INPUT_CAPABLE_MASK 0x20
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__INPUT_CAPABLE__SHIFT 0x5
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__BALANCED_I_O_PINS_MASK 0x40
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__BALANCED_I_O_PINS__SHIFT 0x6
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__HDMI_MASK 0x80
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__HDMI__SHIFT 0x7
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__VREF_CONTROL_MASK 0xff00
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__VREF_CONTROL__SHIFT 0x8
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__EAPD_CAPABLE_MASK 0x10000
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__EAPD_CAPABLE__SHIFT 0x10
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__DP_MASK 0x1000000
+#define AZALIA_F0_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__DP__SHIFT 0x18
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_UNSOLICITED_RESPONSE__TAG_MASK 0x3f
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_UNSOLICITED_RESPONSE__TAG__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_UNSOLICITED_RESPONSE__ENABLE_MASK 0x80
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_UNSOLICITED_RESPONSE__ENABLE__SHIFT 0x7
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_INPUT_PIN_SENSE__IMPEDANCE_SENSE_MASK 0x7fffffff
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_INPUT_PIN_SENSE__IMPEDANCE_SENSE__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_INPUT_PIN_SENSE__PRESENCE_DETECT_MASK 0x80000000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_INPUT_PIN_SENSE__PRESENCE_DETECT__SHIFT 0x1f
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_WIDGET_CONTROL__IN_ENABLE_MASK 0x20
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_WIDGET_CONTROL__IN_ENABLE__SHIFT 0x5
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL0_ENABLE_MASK 0x1
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL0_ENABLE__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL0_MUTE_MASK 0x2
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL0_MUTE__SHIFT 0x1
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL0_CHANNEL_ID_MASK 0xf0
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL0_CHANNEL_ID__SHIFT 0x4
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL1_ENABLE_MASK 0x100
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL1_ENABLE__SHIFT 0x8
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL1_MUTE_MASK 0x200
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL1_MUTE__SHIFT 0x9
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL1_CHANNEL_ID_MASK 0xf000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL1_CHANNEL_ID__SHIFT 0xc
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL2_ENABLE_MASK 0x10000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL2_ENABLE__SHIFT 0x10
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL2_MUTE_MASK 0x20000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL2_MUTE__SHIFT 0x11
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL2_CHANNEL_ID_MASK 0xf00000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL2_CHANNEL_ID__SHIFT 0x14
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL3_ENABLE_MASK 0x1000000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL3_ENABLE__SHIFT 0x18
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL3_MUTE_MASK 0x2000000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL3_MUTE__SHIFT 0x19
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL3_CHANNEL_ID_MASK 0xf0000000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE__MULTICHANNEL3_CHANNEL_ID__SHIFT 0x1c
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL4_ENABLE_MASK 0x1
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL4_ENABLE__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL4_MUTE_MASK 0x2
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL4_MUTE__SHIFT 0x1
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL4_CHANNEL_ID_MASK 0xf0
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL4_CHANNEL_ID__SHIFT 0x4
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL5_ENABLE_MASK 0x100
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL5_ENABLE__SHIFT 0x8
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL5_MUTE_MASK 0x200
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL5_MUTE__SHIFT 0x9
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL5_CHANNEL_ID_MASK 0xf000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL5_CHANNEL_ID__SHIFT 0xc
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL6_ENABLE_MASK 0x10000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL6_ENABLE__SHIFT 0x10
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL6_MUTE_MASK 0x20000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL6_MUTE__SHIFT 0x11
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL6_CHANNEL_ID_MASK 0xf00000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL6_CHANNEL_ID__SHIFT 0x14
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL7_ENABLE_MASK 0x1000000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL7_ENABLE__SHIFT 0x18
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL7_MUTE_MASK 0x2000000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL7_MUTE__SHIFT 0x19
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL7_CHANNEL_ID_MASK 0xf0000000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL_ENABLE2__MULTICHANNEL7_CHANNEL_ID__SHIFT 0x1c
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_HBR__HBR_CAPABLE_MASK 0x1
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_HBR__HBR_CAPABLE__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_HBR__HBR_ENABLE_MASK 0x10
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_HBR__HBR_ENABLE__SHIFT 0x4
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_CHANNEL_ALLOCATION__CHANNEL_ALLOCATION_MASK 0xff
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_CHANNEL_ALLOCATION__CHANNEL_ALLOCATION__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_HOT_PLUG_CONTROL__CLOCK_GATING_DISABLE_MASK 0x1
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_HOT_PLUG_CONTROL__CLOCK_GATING_DISABLE__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_HOT_PLUG_CONTROL__CLOCK_ON_STATE_MASK 0x10
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_HOT_PLUG_CONTROL__CLOCK_ON_STATE__SHIFT 0x4
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_HOT_PLUG_CONTROL__AUDIO_ENABLED_MASK 0x80000000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_HOT_PLUG_CONTROL__AUDIO_ENABLED__SHIFT 0x1f
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_UNSOLICITED_RESPONSE_FORCE__UNSOLICITED_RESPONSE_PAYLOAD_MASK 0x3ffffff
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_UNSOLICITED_RESPONSE_FORCE__UNSOLICITED_RESPONSE_PAYLOAD__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_UNSOLICITED_RESPONSE_FORCE__UNSOLICITED_RESPONSE_FORCE_MASK 0x10000000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_UNSOLICITED_RESPONSE_FORCE__UNSOLICITED_RESPONSE_FORCE__SHIFT 0x1c
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__SEQUENCE_MASK 0xf
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__SEQUENCE__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__DEFAULT_ASSOCIATION_MASK 0xf0
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__DEFAULT_ASSOCIATION__SHIFT 0x4
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__MISC_MASK 0xf00
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__MISC__SHIFT 0x8
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__COLOR_MASK 0xf000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__COLOR__SHIFT 0xc
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__CONNECTION_TYPE_MASK 0xf0000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__CONNECTION_TYPE__SHIFT 0x10
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__DEFAULT_DEVICE_MASK 0xf00000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__DEFAULT_DEVICE__SHIFT 0x14
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__LOCATION_MASK 0x3f000000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__LOCATION__SHIFT 0x18
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__PORT_CONNECTIVITY_MASK 0xc0000000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__PORT_CONNECTIVITY__SHIFT 0x1e
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_INPUT_STATUS_CONTROL__INPUT_ACTIVITY_MASK 0x1
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_INPUT_STATUS_CONTROL__INPUT_ACTIVITY__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_INPUT_STATUS_CONTROL__CHANNEL_LAYOUT_MASK 0x6
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_INPUT_STATUS_CONTROL__CHANNEL_LAYOUT__SHIFT 0x1
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_INPUT_STATUS_CONTROL__INPUT_ACTIVITY_UR_ENABLE_MASK 0x10
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_INPUT_STATUS_CONTROL__INPUT_ACTIVITY_UR_ENABLE__SHIFT 0x4
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_INPUT_STATUS_CONTROL__INPUT_CL_CS_INFOFRAME_CHANGE_UR_ENABLE_MASK 0x20
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_INPUT_STATUS_CONTROL__INPUT_CL_CS_INFOFRAME_CHANGE_UR_ENABLE__SHIFT 0x5
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_INFOFRAME__CHANNEL_COUNT_MASK 0x7
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_INFOFRAME__CHANNEL_COUNT__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_INFOFRAME__CHANNEL_ALLOCATION_MASK 0xff00
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_INFOFRAME__CHANNEL_ALLOCATION__SHIFT 0x8
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_INFOFRAME__INFOFRAME_BYTE_5_MASK 0xff0000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_INFOFRAME__INFOFRAME_BYTE_5__SHIFT 0x10
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_INFOFRAME__INFOFRAME_VALID_MASK 0x80000000
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_INFOFRAME__INFOFRAME_VALID__SHIFT 0x1f
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_LPIB_SNAPSHOT_CONTROL__LPIB_SNAPSHOT_LOCK_MASK 0x1
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_LPIB_SNAPSHOT_CONTROL__LPIB_SNAPSHOT_LOCK__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_LPIB_SNAPSHOT_CONTROL__CYCLIC_BUFFER_WRAP_COUNT_MASK 0xff00
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_LPIB_SNAPSHOT_CONTROL__CYCLIC_BUFFER_WRAP_COUNT__SHIFT 0x8
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_LPIB__LPIB_MASK 0xffffffff
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_LPIB__LPIB__SHIFT 0x0
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_LPIB_TIMER_SNAPSHOT__LPIB_TIMER_SNAPSHOT_MASK 0xffffffff
+#define AZALIA_F0_CODEC_INPUT_PIN_CONTROL_LPIB_TIMER_SNAPSHOT__LPIB_TIMER_SNAPSHOT__SHIFT 0x0
+#define AZENDPOINT_IMMEDIATE_COMMAND_INPUT_INTERFACE_INDEX__IMMEDIATE_COMMAND_WRITE_MASK 0x1ffff
+#define AZENDPOINT_IMMEDIATE_COMMAND_INPUT_INTERFACE_INDEX__IMMEDIATE_COMMAND_WRITE__SHIFT 0x0
+#define AZENDPOINT_IMMEDIATE_COMMAND_INPUT_INTERFACE_DATA__IMMEDIATE_COMMAND_WRITE_MASK 0xffffffff
+#define AZENDPOINT_IMMEDIATE_COMMAND_INPUT_INTERFACE_DATA__IMMEDIATE_COMMAND_WRITE__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_CHANNEL_CAPABILITIES_MASK 0x1
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_CHANNEL_CAPABILITIES__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__INPUT_AMPLIFIER_PRESENT_MASK 0x2
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__INPUT_AMPLIFIER_PRESENT__SHIFT 0x1
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__OUTPUT_AMPLIFIER_PRESENT_MASK 0x4
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__OUTPUT_AMPLIFIER_PRESENT__SHIFT 0x2
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AMPLIFIER_PARAMETER_OVERRIDE_MASK 0x8
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AMPLIFIER_PARAMETER_OVERRIDE__SHIFT 0x3
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__FORMAT_OVERRIDE_MASK 0x10
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__FORMAT_OVERRIDE__SHIFT 0x4
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__STRIPE_MASK 0x20
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__STRIPE__SHIFT 0x5
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__PROCESSING_WIDGET_MASK 0x40
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__PROCESSING_WIDGET__SHIFT 0x6
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__UNSOLICITED_RESPONSE_CAPABILITY_MASK 0x80
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__UNSOLICITED_RESPONSE_CAPABILITY__SHIFT 0x7
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__CONNECTION_LIST_MASK 0x100
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__CONNECTION_LIST__SHIFT 0x8
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__DIGITAL_MASK 0x200
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__DIGITAL__SHIFT 0x9
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__POWER_CONTROL_MASK 0x400
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__POWER_CONTROL__SHIFT 0xa
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__LR_SWAP_MASK 0x800
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__LR_SWAP__SHIFT 0xb
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_WIDGET_CAPABILITIES_DELAY_MASK 0xf0000
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_WIDGET_CAPABILITIES_DELAY__SHIFT 0x10
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__TYPE_MASK 0xf00000
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_AUDIO_WIDGET_CAPABILITIES__TYPE__SHIFT 0x14
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_RATE_CAPABILITIES_MASK 0xfff
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_RATE_CAPABILITIES__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_BIT_CAPABILITIES_MASK 0x1f0000
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_SUPPORTED_SIZE_RATES__AUDIO_BIT_CAPABILITIES__SHIFT 0x10
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_STREAM_FORMATS__STREAM_FORMATS_MASK 0xffffffff
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_PARAMETER_STREAM_FORMATS__STREAM_FORMATS__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__NUMBER_OF_CHANNELS_MASK 0xf
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__NUMBER_OF_CHANNELS__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__BITS_PER_SAMPLE_MASK 0x70
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__BITS_PER_SAMPLE__SHIFT 0x4
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_DIVISOR_MASK 0x700
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_DIVISOR__SHIFT 0x8
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_MULTIPLE_MASK 0x3800
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_MULTIPLE__SHIFT 0xb
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_RATE_MASK 0x4000
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__SAMPLE_BASE_RATE__SHIFT 0xe
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__STREAM_TYPE_MASK 0x8000
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CONVERTER_FORMAT__STREAM_TYPE__SHIFT 0xf
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CHANNEL_STREAM_ID__CHANNEL_ID_MASK 0xf
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CHANNEL_STREAM_ID__CHANNEL_ID__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CHANNEL_STREAM_ID__STREAM_ID_MASK 0xf0
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_CHANNEL_STREAM_ID__STREAM_ID__SHIFT 0x4
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__DIGEN_MASK 0x1
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__DIGEN__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__V_MASK 0x2
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__V__SHIFT 0x1
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__VCFG_MASK 0x4
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__VCFG__SHIFT 0x2
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__PRE_MASK 0x8
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__PRE__SHIFT 0x3
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__COPY_MASK 0x10
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__COPY__SHIFT 0x4
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__NON_AUDIO_MASK 0x20
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__NON_AUDIO__SHIFT 0x5
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__PRO_MASK 0x40
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__PRO__SHIFT 0x6
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__L_MASK 0x80
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__L__SHIFT 0x7
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__CC_MASK 0x7f00
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__CC__SHIFT 0x8
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__KEEPALIVE_MASK 0x800000
+#define AZALIA_F2_CODEC_INPUT_CONVERTER_CONTROL_DIGITAL_CONVERTER__KEEPALIVE__SHIFT 0x17
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_CHANNEL_CAPABILITIES_MASK 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_CHANNEL_CAPABILITIES__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__INPUT_AMPLIFIER_PRESENT_MASK 0x2
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__INPUT_AMPLIFIER_PRESENT__SHIFT 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__OUTPUT_AMPLIFIER_PRESENT_MASK 0x4
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__OUTPUT_AMPLIFIER_PRESENT__SHIFT 0x2
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AMPLIFIER_PARAMETER_OVERRIDE_MASK 0x8
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AMPLIFIER_PARAMETER_OVERRIDE__SHIFT 0x3
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__STRIPE_MASK 0x20
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__STRIPE__SHIFT 0x5
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__PROCESSING_WIDGET_MASK 0x40
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__PROCESSING_WIDGET__SHIFT 0x6
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__UNSOLICITED_RESPONSE_CAPABILITY_MASK 0x80
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__UNSOLICITED_RESPONSE_CAPABILITY__SHIFT 0x7
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__CONNECTION_LIST_MASK 0x100
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__CONNECTION_LIST__SHIFT 0x8
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__DIGITAL_MASK 0x200
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__DIGITAL__SHIFT 0x9
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__POWER_CONTROL_MASK 0x400
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__POWER_CONTROL__SHIFT 0xa
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__LR_SWAP_MASK 0x800
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__LR_SWAP__SHIFT 0xb
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_WIDGET_CAPABILITIES_DELAY_MASK 0xf0000
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__AUDIO_WIDGET_CAPABILITIES_DELAY__SHIFT 0x10
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__TYPE_MASK 0xf00000
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_AUDIO_WIDGET_CAPABILITIES__TYPE__SHIFT 0x14
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__IMPEDANCE_SENSE_CAPABLE_MASK 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__IMPEDANCE_SENSE_CAPABLE__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__TRIGGER_REQUIRED_MASK 0x2
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__TRIGGER_REQUIRED__SHIFT 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__JACK_DETECTION_CAPABILITY_MASK 0x4
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__JACK_DETECTION_CAPABILITY__SHIFT 0x2
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__HEADPHONE_DRIVE_CAPABLE_MASK 0x8
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__HEADPHONE_DRIVE_CAPABLE__SHIFT 0x3
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__OUTPUT_CAPABLE_MASK 0x10
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__OUTPUT_CAPABLE__SHIFT 0x4
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__INPUT_CAPABLE_MASK 0x20
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__INPUT_CAPABLE__SHIFT 0x5
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__BALANCED_I_O_PINS_MASK 0x40
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__BALANCED_I_O_PINS__SHIFT 0x6
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__HDMI_MASK 0x80
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__HDMI__SHIFT 0x7
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__VREF_CONTROL_MASK 0xff00
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__VREF_CONTROL__SHIFT 0x8
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__EAPD_CAPABLE_MASK 0x10000
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__EAPD_CAPABLE__SHIFT 0x10
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__DP_MASK 0x1000000
+#define AZALIA_F2_CODEC_INPUT_PIN_PARAMETER_CAPABILITIES__DP__SHIFT 0x18
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_WIDGET_CONTROL__IN_ENABLE_MASK 0x20
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_WIDGET_CONTROL__IN_ENABLE__SHIFT 0x5
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_UNSOLICITED_RESPONSE__TAG_MASK 0x3f
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_UNSOLICITED_RESPONSE__TAG__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_UNSOLICITED_RESPONSE__ENABLE_MASK 0x80
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_UNSOLICITED_RESPONSE__ENABLE__SHIFT 0x7
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_PIN_SENSE__IMPEDANCE_SENSE_MASK 0x7fffffff
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_PIN_SENSE__IMPEDANCE_SENSE__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_PIN_SENSE__PRESENCE_DETECT_MASK 0x80000000
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_PIN_SENSE__PRESENCE_DETECT__SHIFT 0x1f
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__SEQUENCE_MASK 0xf
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__SEQUENCE__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__DEFAULT_ASSOCIATION_MASK 0xf0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__DEFAULT_ASSOCIATION__SHIFT 0x4
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__MISC_MASK 0xf00
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__MISC__SHIFT 0x8
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__COLOR_MASK 0xf000
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__COLOR__SHIFT 0xc
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__CONNECTION_TYPE_MASK 0xf0000
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__CONNECTION_TYPE__SHIFT 0x10
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__DEFAULT_DEVICE_MASK 0xf00000
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__DEFAULT_DEVICE__SHIFT 0x14
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__LOCATION_MASK 0x3f000000
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__LOCATION__SHIFT 0x18
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__PORT_CONNECTIVITY_MASK 0xc0000000
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT__PORT_CONNECTIVITY__SHIFT 0x1e
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_2__MISC_MASK 0xf
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_2__MISC__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_2__COLOR_MASK 0xf0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_2__COLOR__SHIFT 0x4
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_3__CONNECTION_TYPE_MASK 0xf
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_3__CONNECTION_TYPE__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_3__DEFAULT_DEVICE_MASK 0xf0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_3__DEFAULT_DEVICE__SHIFT 0x4
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_4__LOCATION_MASK 0x3f
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_4__LOCATION__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_4__PORT_CONNECTIVITY_MASK 0xc0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_RESPONSE_CONFIGURATION_DEFAULT_4__PORT_CONNECTIVITY__SHIFT 0x6
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL0_ENABLE__MULTICHANNEL0_ENABLE_MASK 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL0_ENABLE__MULTICHANNEL0_ENABLE__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL0_ENABLE__MULTICHANNEL0_MUTE_MASK 0x2
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL0_ENABLE__MULTICHANNEL0_MUTE__SHIFT 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL0_ENABLE__MULTICHANNEL0_CHANNEL_ID_MASK 0xf0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL0_ENABLE__MULTICHANNEL0_CHANNEL_ID__SHIFT 0x4
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL1_ENABLE__MULTICHANNEL1_ENABLE_MASK 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL1_ENABLE__MULTICHANNEL1_ENABLE__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL1_ENABLE__MULTICHANNEL1_MUTE_MASK 0x2
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL1_ENABLE__MULTICHANNEL1_MUTE__SHIFT 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL1_ENABLE__MULTICHANNEL1_CHANNEL_ID_MASK 0xf0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL1_ENABLE__MULTICHANNEL1_CHANNEL_ID__SHIFT 0x4
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL2_ENABLE__MULTICHANNEL2_ENABLE_MASK 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL2_ENABLE__MULTICHANNEL2_ENABLE__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL2_ENABLE__MULTICHANNEL2_MUTE_MASK 0x2
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL2_ENABLE__MULTICHANNEL2_MUTE__SHIFT 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL2_ENABLE__MULTICHANNEL2_CHANNEL_ID_MASK 0xf0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL2_ENABLE__MULTICHANNEL2_CHANNEL_ID__SHIFT 0x4
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL3_ENABLE__MULTICHANNEL3_ENABLE_MASK 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL3_ENABLE__MULTICHANNEL3_ENABLE__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL3_ENABLE__MULTICHANNEL3_MUTE_MASK 0x2
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL3_ENABLE__MULTICHANNEL3_MUTE__SHIFT 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL3_ENABLE__MULTICHANNEL3_CHANNEL_ID_MASK 0xf0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL3_ENABLE__MULTICHANNEL3_CHANNEL_ID__SHIFT 0x4
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_HBR__HBR_CAPABLE_MASK 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_HBR__HBR_CAPABLE__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_HBR__HBR_ENABLE_MASK 0x10
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_HBR__HBR_ENABLE__SHIFT 0x4
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL4_ENABLE__MULTICHANNEL4_ENABLE_MASK 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL4_ENABLE__MULTICHANNEL4_ENABLE__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL4_ENABLE__MULTICHANNEL4_MUTE_MASK 0x2
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL4_ENABLE__MULTICHANNEL4_MUTE__SHIFT 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL4_ENABLE__MULTICHANNEL4_CHANNEL_ID_MASK 0xf0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL4_ENABLE__MULTICHANNEL4_CHANNEL_ID__SHIFT 0x4
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL5_ENABLE__MULTICHANNEL5_ENABLE_MASK 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL5_ENABLE__MULTICHANNEL5_ENABLE__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL5_ENABLE__MULTICHANNEL5_MUTE_MASK 0x2
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL5_ENABLE__MULTICHANNEL5_MUTE__SHIFT 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL5_ENABLE__MULTICHANNEL5_CHANNEL_ID_MASK 0xf0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL5_ENABLE__MULTICHANNEL5_CHANNEL_ID__SHIFT 0x4
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL6_ENABLE__MULTICHANNEL6_ENABLE_MASK 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL6_ENABLE__MULTICHANNEL6_ENABLE__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL6_ENABLE__MULTICHANNEL6_MUTE_MASK 0x2
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL6_ENABLE__MULTICHANNEL6_MUTE__SHIFT 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL6_ENABLE__MULTICHANNEL6_CHANNEL_ID_MASK 0xf0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL6_ENABLE__MULTICHANNEL6_CHANNEL_ID__SHIFT 0x4
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL7_ENABLE__MULTICHANNEL7_ENABLE_MASK 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL7_ENABLE__MULTICHANNEL7_ENABLE__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL7_ENABLE__MULTICHANNEL7_MUTE_MASK 0x2
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL7_ENABLE__MULTICHANNEL7_MUTE__SHIFT 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL7_ENABLE__MULTICHANNEL7_CHANNEL_ID_MASK 0xf0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_MULTICHANNEL7_ENABLE__MULTICHANNEL7_CHANNEL_ID__SHIFT 0x4
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_CHANNEL_ALLOCATION__CHANNEL_ALLOCATION_MASK 0xff
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_CHANNEL_ALLOCATION__CHANNEL_ALLOCATION__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_INPUT_STATUS_CONTROL__INPUT_ACTIVITY_MASK 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_INPUT_STATUS_CONTROL__INPUT_ACTIVITY__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_INPUT_STATUS_CONTROL__CHANNEL_LAYOUT_MASK 0x6
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_INPUT_STATUS_CONTROL__CHANNEL_LAYOUT__SHIFT 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_INPUT_STATUS_CONTROL__INPUT_ACTIVITY_UR_ENABLE_MASK 0x10
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_INPUT_STATUS_CONTROL__INPUT_ACTIVITY_UR_ENABLE__SHIFT 0x4
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_INPUT_STATUS_CONTROL__INPUT_CL_CS_INFOFRAME_CHANGE_UR_ENABLE_MASK 0x20
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_INPUT_STATUS_CONTROL__INPUT_CL_CS_INFOFRAME_CHANGE_UR_ENABLE__SHIFT 0x5
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_INFOFRAME__CHANNEL_COUNT_MASK 0x7
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_INFOFRAME__CHANNEL_COUNT__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_INFOFRAME__CHANNEL_ALLOCATION_MASK 0xff00
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_INFOFRAME__CHANNEL_ALLOCATION__SHIFT 0x8
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_INFOFRAME__INFOFRAME_BYTE_5_MASK 0xff0000
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_INFOFRAME__INFOFRAME_BYTE_5__SHIFT 0x10
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_INFOFRAME__INFOFRAME_VALID_MASK 0x80000000
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_INFOFRAME__INFOFRAME_VALID__SHIFT 0x1f
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_CHANNEL_STATUS_L__CHANNEL_STATUS_L_MASK 0xffffffff
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_CHANNEL_STATUS_L__CHANNEL_STATUS_L__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_CHANNEL_STATUS_H__CHANNEL_STATUS_H_MASK 0xffffffff
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_CHANNEL_STATUS_H__CHANNEL_STATUS_H__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_LPIB_SNAPSHOT_CONTROL__LPIB_SNAPSHOT_LOCK_MASK 0x1
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_LPIB_SNAPSHOT_CONTROL__LPIB_SNAPSHOT_LOCK__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_LPIB_SNAPSHOT_CONTROL__CYCLIC_BUFFER_WRAP_COUNT_MASK 0xff00
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_LPIB_SNAPSHOT_CONTROL__CYCLIC_BUFFER_WRAP_COUNT__SHIFT 0x8
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_LPIB__LPIB_MASK 0xffffffff
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_LPIB__LPIB__SHIFT 0x0
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_LPIB_TIMER_SNAPSHOT__LPIB_TIMER_SNAPSHOT_MASK 0xffffffff
+#define AZALIA_F2_CODEC_INPUT_PIN_CONTROL_LPIB_TIMER_SNAPSHOT__LPIB_TIMER_SNAPSHOT__SHIFT 0x0
+#define BLND_CONTROL__BLND_GLOBAL_GAIN_MASK 0xff
+#define BLND_CONTROL__BLND_GLOBAL_GAIN__SHIFT 0x0
+#define BLND_CONTROL__BLND_MODE_MASK 0x300
+#define BLND_CONTROL__BLND_MODE__SHIFT 0x8
+#define BLND_CONTROL__BLND_STEREO_TYPE_MASK 0xc00
+#define BLND_CONTROL__BLND_STEREO_TYPE__SHIFT 0xa
+#define BLND_CONTROL__BLND_STEREO_POLARITY_MASK 0x1000
+#define BLND_CONTROL__BLND_STEREO_POLARITY__SHIFT 0xc
+#define BLND_CONTROL__BLND_FEEDTHROUGH_EN_MASK 0x2000
+#define BLND_CONTROL__BLND_FEEDTHROUGH_EN__SHIFT 0xd
+#define BLND_CONTROL__BLND_ALPHA_MODE_MASK 0x30000
+#define BLND_CONTROL__BLND_ALPHA_MODE__SHIFT 0x10
+#define BLND_CONTROL__BLND_ACTIVE_OVERLAP_ONLY_MASK 0x40000
+#define BLND_CONTROL__BLND_ACTIVE_OVERLAP_ONLY__SHIFT 0x12
+#define BLND_CONTROL__BLND_MULTIPLIED_MODE_MASK 0x100000
+#define BLND_CONTROL__BLND_MULTIPLIED_MODE__SHIFT 0x14
+#define BLND_CONTROL__BLND_GLOBAL_ALPHA_MASK 0xff000000
+#define BLND_CONTROL__BLND_GLOBAL_ALPHA__SHIFT 0x18
+#define BLND_SM_CONTROL2__SM_MODE_MASK 0x7
+#define BLND_SM_CONTROL2__SM_MODE__SHIFT 0x0
+#define BLND_SM_CONTROL2__SM_FRAME_ALTERNATE_MASK 0x10
+#define BLND_SM_CONTROL2__SM_FRAME_ALTERNATE__SHIFT 0x4
+#define BLND_SM_CONTROL2__SM_FIELD_ALTERNATE_MASK 0x20
+#define BLND_SM_CONTROL2__SM_FIELD_ALTERNATE__SHIFT 0x5
+#define BLND_SM_CONTROL2__SM_FORCE_NEXT_FRAME_POL_MASK 0x300
+#define BLND_SM_CONTROL2__SM_FORCE_NEXT_FRAME_POL__SHIFT 0x8
+#define BLND_SM_CONTROL2__SM_FORCE_NEXT_TOP_POL_MASK 0x30000
+#define BLND_SM_CONTROL2__SM_FORCE_NEXT_TOP_POL__SHIFT 0x10
+#define BLND_SM_CONTROL2__SM_CURRENT_FRAME_POL_MASK 0x1000000
+#define BLND_SM_CONTROL2__SM_CURRENT_FRAME_POL__SHIFT 0x18
+#define BLND_CONTROL2__PTI_ENABLE_MASK 0x1
+#define BLND_CONTROL2__PTI_ENABLE__SHIFT 0x0
+#define BLND_CONTROL2__PTI_NEW_PIXEL_GAP_MASK 0x30
+#define BLND_CONTROL2__PTI_NEW_PIXEL_GAP__SHIFT 0x4
+#define BLND_CONTROL2__BLND_NEW_PIXEL_MODE_MASK 0x40
+#define BLND_CONTROL2__BLND_NEW_PIXEL_MODE__SHIFT 0x6
+#define BLND_CONTROL2__BLND_SUPERAA_DEGAMMA_EN_MASK 0x80
+#define BLND_CONTROL2__BLND_SUPERAA_DEGAMMA_EN__SHIFT 0x7
+#define BLND_CONTROL2__BLND_SUPERAA_REGAMMA_EN_MASK 0x100
+#define BLND_CONTROL2__BLND_SUPERAA_REGAMMA_EN__SHIFT 0x8
+#define BLND_UPDATE__BLND_UPDATE_PENDING_MASK 0x1
+#define BLND_UPDATE__BLND_UPDATE_PENDING__SHIFT 0x0
+#define BLND_UPDATE__BLND_UPDATE_TAKEN_MASK 0x100
+#define BLND_UPDATE__BLND_UPDATE_TAKEN__SHIFT 0x8
+#define BLND_UPDATE__BLND_UPDATE_LOCK_MASK 0x10000
+#define BLND_UPDATE__BLND_UPDATE_LOCK__SHIFT 0x10
+#define BLND_UNDERFLOW_INTERRUPT__BLND_UNDERFLOW_INT_OCCURED_MASK 0x1
+#define BLND_UNDERFLOW_INTERRUPT__BLND_UNDERFLOW_INT_OCCURED__SHIFT 0x0
+#define BLND_UNDERFLOW_INTERRUPT__BLND_UNDERFLOW_INT_ACK_MASK 0x100
+#define BLND_UNDERFLOW_INTERRUPT__BLND_UNDERFLOW_INT_ACK__SHIFT 0x8
+#define BLND_UNDERFLOW_INTERRUPT__BLND_UNDERFLOW_INT_MASK_MASK 0x1000
+#define BLND_UNDERFLOW_INTERRUPT__BLND_UNDERFLOW_INT_MASK__SHIFT 0xc
+#define BLND_UNDERFLOW_INTERRUPT__BLND_UNDERFLOW_INT_PIPE_INDEX_MASK 0x30000
+#define BLND_UNDERFLOW_INTERRUPT__BLND_UNDERFLOW_INT_PIPE_INDEX__SHIFT 0x10
+#define BLND_V_UPDATE_LOCK__BLND_DCP_GRPH_V_UPDATE_LOCK_MASK 0x1
+#define BLND_V_UPDATE_LOCK__BLND_DCP_GRPH_V_UPDATE_LOCK__SHIFT 0x0
+#define BLND_V_UPDATE_LOCK__BLND_DCP_GRPH_SURF_V_UPDATE_LOCK_MASK 0x2
+#define BLND_V_UPDATE_LOCK__BLND_DCP_GRPH_SURF_V_UPDATE_LOCK__SHIFT 0x1
+#define BLND_V_UPDATE_LOCK__BLND_DCP_CUR_V_UPDATE_LOCK_MASK 0x10000
+#define BLND_V_UPDATE_LOCK__BLND_DCP_CUR_V_UPDATE_LOCK__SHIFT 0x10
+#define BLND_V_UPDATE_LOCK__BLND_DCP_CUR2_V_UPDATE_LOCK_MASK 0x1000000
+#define BLND_V_UPDATE_LOCK__BLND_DCP_CUR2_V_UPDATE_LOCK__SHIFT 0x18
+#define BLND_V_UPDATE_LOCK__BLND_SCL_V_UPDATE_LOCK_MASK 0x10000000
+#define BLND_V_UPDATE_LOCK__BLND_SCL_V_UPDATE_LOCK__SHIFT 0x1c
+#define BLND_V_UPDATE_LOCK__BLND_BLND_V_UPDATE_LOCK_MASK 0x20000000
+#define BLND_V_UPDATE_LOCK__BLND_BLND_V_UPDATE_LOCK__SHIFT 0x1d
+#define BLND_V_UPDATE_LOCK__BLND_V_UPDATE_LOCK_MODE_MASK 0x80000000
+#define BLND_V_UPDATE_LOCK__BLND_V_UPDATE_LOCK_MODE__SHIFT 0x1f
+#define BLND_REG_UPDATE_STATUS__DCP_BLNDC_GRPH_UPDATE_PENDING_MASK 0x1
+#define BLND_REG_UPDATE_STATUS__DCP_BLNDC_GRPH_UPDATE_PENDING__SHIFT 0x0
+#define BLND_REG_UPDATE_STATUS__DCP_BLNDO_GRPH_UPDATE_PENDING_MASK 0x2
+#define BLND_REG_UPDATE_STATUS__DCP_BLNDO_GRPH_UPDATE_PENDING__SHIFT 0x1
+#define BLND_REG_UPDATE_STATUS__DCP_BLNDC_GRPH_SURF_UPDATE_PENDING_MASK 0x4
+#define BLND_REG_UPDATE_STATUS__DCP_BLNDC_GRPH_SURF_UPDATE_PENDING__SHIFT 0x2
+#define BLND_REG_UPDATE_STATUS__DCP_BLNDO_GRPH_SURF_UPDATE_PENDING_MASK 0x8
+#define BLND_REG_UPDATE_STATUS__DCP_BLNDO_GRPH_SURF_UPDATE_PENDING__SHIFT 0x3
+#define BLND_REG_UPDATE_STATUS__DCP_BLNDC_CUR_UPDATE_PENDING_MASK 0x40
+#define BLND_REG_UPDATE_STATUS__DCP_BLNDC_CUR_UPDATE_PENDING__SHIFT 0x6
+#define BLND_REG_UPDATE_STATUS__DCP_BLNDO_CUR_UPDATE_PENDING_MASK 0x80
+#define BLND_REG_UPDATE_STATUS__DCP_BLNDO_CUR_UPDATE_PENDING__SHIFT 0x7
+#define BLND_REG_UPDATE_STATUS__SCL_BLNDC_UPDATE_PENDING_MASK 0x100
+#define BLND_REG_UPDATE_STATUS__SCL_BLNDC_UPDATE_PENDING__SHIFT 0x8
+#define BLND_REG_UPDATE_STATUS__SCL_BLNDO_UPDATE_PENDING_MASK 0x200
+#define BLND_REG_UPDATE_STATUS__SCL_BLNDO_UPDATE_PENDING__SHIFT 0x9
+#define BLND_REG_UPDATE_STATUS__BLND_BLNDC_UPDATE_PENDING_MASK 0x400
+#define BLND_REG_UPDATE_STATUS__BLND_BLNDC_UPDATE_PENDING__SHIFT 0xa
+#define BLND_REG_UPDATE_STATUS__BLND_BLNDO_UPDATE_PENDING_MASK 0x800
+#define BLND_REG_UPDATE_STATUS__BLND_BLNDO_UPDATE_PENDING__SHIFT 0xb
+#define BLND_DEBUG__BLND_CNV_MUX_SELECT_MASK 0x1
+#define BLND_DEBUG__BLND_CNV_MUX_SELECT__SHIFT 0x0
+#define BLND_DEBUG__BLND_DEBUG_MASK 0xfffffffe
+#define BLND_DEBUG__BLND_DEBUG__SHIFT 0x1
+#define BLND_TEST_DEBUG_INDEX__BLND_TEST_DEBUG_INDEX_MASK 0xff
+#define BLND_TEST_DEBUG_INDEX__BLND_TEST_DEBUG_INDEX__SHIFT 0x0
+#define BLND_TEST_DEBUG_INDEX__BLND_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define BLND_TEST_DEBUG_INDEX__BLND_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define BLND_TEST_DEBUG_DATA__BLND_TEST_DEBUG_DATA_MASK 0xffffffff
+#define BLND_TEST_DEBUG_DATA__BLND_TEST_DEBUG_DATA__SHIFT 0x0
+#define WB_ENABLE__WB_ENABLE_MASK 0x1
+#define WB_ENABLE__WB_ENABLE__SHIFT 0x0
+#define WB_EC_CONFIG__DISPCLK_R_WB_GATE_DIS_MASK 0x1
+#define WB_EC_CONFIG__DISPCLK_R_WB_GATE_DIS__SHIFT 0x0
+#define WB_EC_CONFIG__DISPCLK_G_WB_GATE_DIS_MASK 0x2
+#define WB_EC_CONFIG__DISPCLK_G_WB_GATE_DIS__SHIFT 0x1
+#define WB_EC_CONFIG__DISPCLK_G_WBSCL_GATE_DIS_MASK 0x4
+#define WB_EC_CONFIG__DISPCLK_G_WBSCL_GATE_DIS__SHIFT 0x2
+#define WB_EC_CONFIG__WB_TEST_CLK_SEL_MASK 0x78
+#define WB_EC_CONFIG__WB_TEST_CLK_SEL__SHIFT 0x3
+#define WB_EC_CONFIG__WB_LB_LS_DIS_MASK 0x80
+#define WB_EC_CONFIG__WB_LB_LS_DIS__SHIFT 0x7
+#define WB_EC_CONFIG__WB_LB_SD_DIS_MASK 0x100
+#define WB_EC_CONFIG__WB_LB_SD_DIS__SHIFT 0x8
+#define WB_EC_CONFIG__WB_LUT_LS_DIS_MASK 0x200
+#define WB_EC_CONFIG__WB_LUT_LS_DIS__SHIFT 0x9
+#define WB_EC_CONFIG__WBSCL_LB_MEM_PWR_MODE_SEL_MASK 0x3000
+#define WB_EC_CONFIG__WBSCL_LB_MEM_PWR_MODE_SEL__SHIFT 0xc
+#define WB_EC_CONFIG__WBSCL_LB_MEM_PWR_DIS_MASK 0x4000
+#define WB_EC_CONFIG__WBSCL_LB_MEM_PWR_DIS__SHIFT 0xe
+#define WB_EC_CONFIG__WBSCL_LB_MEM_PWR_FORCE_MASK 0x18000
+#define WB_EC_CONFIG__WBSCL_LB_MEM_PWR_FORCE__SHIFT 0xf
+#define WB_EC_CONFIG__WBSCL_LB_MEM_PWR_STATE_SM_MASK 0x60000
+#define WB_EC_CONFIG__WBSCL_LB_MEM_PWR_STATE_SM__SHIFT 0x11
+#define WB_EC_CONFIG__WBSCL_LB_MEM_PWR_STATE_BG_MASK 0x180000
+#define WB_EC_CONFIG__WBSCL_LB_MEM_PWR_STATE_BG__SHIFT 0x13
+#define WB_EC_CONFIG__WBSCL_LB_MEM_PWR_STATE_MASK 0x600000
+#define WB_EC_CONFIG__WBSCL_LB_MEM_PWR_STATE__SHIFT 0x15
+#define WB_EC_CONFIG__WB_RAM_PW_SAVE_MODE_MASK 0x800000
+#define WB_EC_CONFIG__WB_RAM_PW_SAVE_MODE__SHIFT 0x17
+#define WB_EC_CONFIG__LB_MEM_PWR_STATE_SM_MASK 0x3000000
+#define WB_EC_CONFIG__LB_MEM_PWR_STATE_SM__SHIFT 0x18
+#define WB_EC_CONFIG__LB_MEM_PWR_STATE_BG_MASK 0xc000000
+#define WB_EC_CONFIG__LB_MEM_PWR_STATE_BG__SHIFT 0x1a
+#define WB_EC_CONFIG__LB_MEM_PWR_STATE_MASK 0x30000000
+#define WB_EC_CONFIG__LB_MEM_PWR_STATE__SHIFT 0x1c
+#define WB_EC_CONFIG__LUT_MEM_PWR_STATE_MASK 0xc0000000
+#define WB_EC_CONFIG__LUT_MEM_PWR_STATE__SHIFT 0x1e
+#define CNV_MODE__CNV_FRAME_CAPTURE_RATE_MASK 0x300
+#define CNV_MODE__CNV_FRAME_CAPTURE_RATE__SHIFT 0x8
+#define CNV_MODE__CNV_WINDOW_CROP_EN_MASK 0x1000
+#define CNV_MODE__CNV_WINDOW_CROP_EN__SHIFT 0xc
+#define CNV_MODE__CNV_STEREO_TYPE_MASK 0x6000
+#define CNV_MODE__CNV_STEREO_TYPE__SHIFT 0xd
+#define CNV_MODE__CNV_INTERLACED_MODE_MASK 0x8000
+#define CNV_MODE__CNV_INTERLACED_MODE__SHIFT 0xf
+#define CNV_MODE__CNV_EYE_SELECTION_MASK 0x30000
+#define CNV_MODE__CNV_EYE_SELECTION__SHIFT 0x10
+#define CNV_MODE__CNV_STEREO_POLARITY_MASK 0x40000
+#define CNV_MODE__CNV_STEREO_POLARITY__SHIFT 0x12
+#define CNV_MODE__CNV_INTERLACED_FIELD_ORDER_MASK 0x80000
+#define CNV_MODE__CNV_INTERLACED_FIELD_ORDER__SHIFT 0x13
+#define CNV_MODE__CNV_STEREO_SPLIT_MASK 0x100000
+#define CNV_MODE__CNV_STEREO_SPLIT__SHIFT 0x14
+#define CNV_MODE__CNV_NEW_CONTENT_MASK 0x1000000
+#define CNV_MODE__CNV_NEW_CONTENT__SHIFT 0x18
+#define CNV_MODE__CNV_FRAME_CAPTURE_EN_MASK 0x80000000
+#define CNV_MODE__CNV_FRAME_CAPTURE_EN__SHIFT 0x1f
+#define CNV_WINDOW_START__CNV_WINDOW_START_X_MASK 0xfff
+#define CNV_WINDOW_START__CNV_WINDOW_START_X__SHIFT 0x0
+#define CNV_WINDOW_START__CNV_WINDOW_START_Y_MASK 0xfff0000
+#define CNV_WINDOW_START__CNV_WINDOW_START_Y__SHIFT 0x10
+#define CNV_WINDOW_SIZE__CNV_WINDOW_WIDTH_MASK 0xfff
+#define CNV_WINDOW_SIZE__CNV_WINDOW_WIDTH__SHIFT 0x0
+#define CNV_WINDOW_SIZE__CNV_WINDOW_HEIGHT_MASK 0xfff0000
+#define CNV_WINDOW_SIZE__CNV_WINDOW_HEIGHT__SHIFT 0x10
+#define CNV_UPDATE__CNV_UPDATE_PENDING_MASK 0x1
+#define CNV_UPDATE__CNV_UPDATE_PENDING__SHIFT 0x0
+#define CNV_UPDATE__CNV_UPDATE_TAKEN_MASK 0x100
+#define CNV_UPDATE__CNV_UPDATE_TAKEN__SHIFT 0x8
+#define CNV_UPDATE__CNV_UPDATE_LOCK_MASK 0x10000
+#define CNV_UPDATE__CNV_UPDATE_LOCK__SHIFT 0x10
+#define CNV_SOURCE_SIZE__CNV_SOURCE_WIDTH_MASK 0x7fff
+#define CNV_SOURCE_SIZE__CNV_SOURCE_WIDTH__SHIFT 0x0
+#define CNV_SOURCE_SIZE__CNV_SOURCE_HEIGHT_MASK 0x7fff0000
+#define CNV_SOURCE_SIZE__CNV_SOURCE_HEIGHT__SHIFT 0x10
+#define CNV_CSC_CONTROL__CNV_CSC_BYPASS_MASK 0x1
+#define CNV_CSC_CONTROL__CNV_CSC_BYPASS__SHIFT 0x0
+#define CNV_CSC_C11_C12__CNV_CSC_C11_MASK 0x1fff
+#define CNV_CSC_C11_C12__CNV_CSC_C11__SHIFT 0x0
+#define CNV_CSC_C11_C12__CNV_CSC_C12_MASK 0x1fff0000
+#define CNV_CSC_C11_C12__CNV_CSC_C12__SHIFT 0x10
+#define CNV_CSC_C13_C14__CNV_CSC_C13_MASK 0x1fff
+#define CNV_CSC_C13_C14__CNV_CSC_C13__SHIFT 0x0
+#define CNV_CSC_C13_C14__CNV_CSC_C14_MASK 0x7fff0000
+#define CNV_CSC_C13_C14__CNV_CSC_C14__SHIFT 0x10
+#define CNV_CSC_C21_C22__CNV_CSC_C21_MASK 0x1fff
+#define CNV_CSC_C21_C22__CNV_CSC_C21__SHIFT 0x0
+#define CNV_CSC_C21_C22__CNV_CSC_C22_MASK 0x1fff0000
+#define CNV_CSC_C21_C22__CNV_CSC_C22__SHIFT 0x10
+#define CNV_CSC_C23_C24__CNV_CSC_C23_MASK 0x1fff
+#define CNV_CSC_C23_C24__CNV_CSC_C23__SHIFT 0x0
+#define CNV_CSC_C23_C24__CNV_CSC_C24_MASK 0x7fff0000
+#define CNV_CSC_C23_C24__CNV_CSC_C24__SHIFT 0x10
+#define CNV_CSC_C31_C32__CNV_CSC_C31_MASK 0x1fff
+#define CNV_CSC_C31_C32__CNV_CSC_C31__SHIFT 0x0
+#define CNV_CSC_C31_C32__CNV_CSC_C32_MASK 0x1fff0000
+#define CNV_CSC_C31_C32__CNV_CSC_C32__SHIFT 0x10
+#define CNV_CSC_C33_C34__CNV_CSC_C33_MASK 0x1fff
+#define CNV_CSC_C33_C34__CNV_CSC_C33__SHIFT 0x0
+#define CNV_CSC_C33_C34__CNV_CSC_C34_MASK 0x7fff0000
+#define CNV_CSC_C33_C34__CNV_CSC_C34__SHIFT 0x10
+#define CNV_CSC_ROUND_OFFSET_R__CNV_CSC_ROUND_OFFSET_R_MASK 0xffff
+#define CNV_CSC_ROUND_OFFSET_R__CNV_CSC_ROUND_OFFSET_R__SHIFT 0x0
+#define CNV_CSC_ROUND_OFFSET_G__CNV_CSC_ROUND_OFFSET_G_MASK 0xffff
+#define CNV_CSC_ROUND_OFFSET_G__CNV_CSC_ROUND_OFFSET_G__SHIFT 0x0
+#define CNV_CSC_ROUND_OFFSET_B__CNV_CSC_ROUND_OFFSET_B_MASK 0xffff
+#define CNV_CSC_ROUND_OFFSET_B__CNV_CSC_ROUND_OFFSET_B__SHIFT 0x0
+#define CNV_CSC_CLAMP_R__CNV_CSC_CLAMP_UPPER_R_MASK 0xffff
+#define CNV_CSC_CLAMP_R__CNV_CSC_CLAMP_UPPER_R__SHIFT 0x0
+#define CNV_CSC_CLAMP_R__CNV_CSC_CLAMP_LOWER_R_MASK 0xffff0000
+#define CNV_CSC_CLAMP_R__CNV_CSC_CLAMP_LOWER_R__SHIFT 0x10
+#define CNV_CSC_CLAMP_G__CNV_CSC_CLAMP_UPPER_G_MASK 0xffff
+#define CNV_CSC_CLAMP_G__CNV_CSC_CLAMP_UPPER_G__SHIFT 0x0
+#define CNV_CSC_CLAMP_G__CNV_CSC_CLAMP_LOWER_G_MASK 0xffff0000
+#define CNV_CSC_CLAMP_G__CNV_CSC_CLAMP_LOWER_G__SHIFT 0x10
+#define CNV_CSC_CLAMP_B__CNV_CSC_CLAMP_UPPER_B_MASK 0xffff
+#define CNV_CSC_CLAMP_B__CNV_CSC_CLAMP_UPPER_B__SHIFT 0x0
+#define CNV_CSC_CLAMP_B__CNV_CSC_CLAMP_LOWER_B_MASK 0xffff0000
+#define CNV_CSC_CLAMP_B__CNV_CSC_CLAMP_LOWER_B__SHIFT 0x10
+#define CNV_TEST_CNTL__CNV_TEST_CRC_EN_MASK 0x10
+#define CNV_TEST_CNTL__CNV_TEST_CRC_EN__SHIFT 0x4
+#define CNV_TEST_CNTL__CNV_TEST_CRC_CONT_EN_MASK 0x100
+#define CNV_TEST_CNTL__CNV_TEST_CRC_CONT_EN__SHIFT 0x8
+#define CNV_TEST_CNTL__CNV_TEST_CRC_DE_ONLY_MASK 0x10000
+#define CNV_TEST_CNTL__CNV_TEST_CRC_DE_ONLY__SHIFT 0x10
+#define CNV_TEST_CRC_RED__CNV_TEST_CRC_RED_MASK_MASK 0xfff0
+#define CNV_TEST_CRC_RED__CNV_TEST_CRC_RED_MASK__SHIFT 0x4
+#define CNV_TEST_CRC_RED__CNV_TEST_CRC_SIG_RED_MASK 0xffff0000
+#define CNV_TEST_CRC_RED__CNV_TEST_CRC_SIG_RED__SHIFT 0x10
+#define CNV_TEST_CRC_GREEN__CNV_TEST_CRC_GREEN_MASK_MASK 0xfff0
+#define CNV_TEST_CRC_GREEN__CNV_TEST_CRC_GREEN_MASK__SHIFT 0x4
+#define CNV_TEST_CRC_GREEN__CNV_TEST_CRC_SIG_GREEN_MASK 0xffff0000
+#define CNV_TEST_CRC_GREEN__CNV_TEST_CRC_SIG_GREEN__SHIFT 0x10
+#define CNV_TEST_CRC_BLUE__CNV_TEST_CRC_BLUE_MASK_MASK 0xfff0
+#define CNV_TEST_CRC_BLUE__CNV_TEST_CRC_BLUE_MASK__SHIFT 0x4
+#define CNV_TEST_CRC_BLUE__CNV_TEST_CRC_SIG_BLUE_MASK 0xffff0000
+#define CNV_TEST_CRC_BLUE__CNV_TEST_CRC_SIG_BLUE__SHIFT 0x10
+#define WB_DEBUG_CTRL__WB_DEBUG_EN_MASK 0x1
+#define WB_DEBUG_CTRL__WB_DEBUG_EN__SHIFT 0x0
+#define WB_DEBUG_CTRL__WB_DEBUG_SEL_MASK 0xc0
+#define WB_DEBUG_CTRL__WB_DEBUG_SEL__SHIFT 0x6
+#define WB_DBG_MODE__WB_DBG_MODE_EN_MASK 0x1
+#define WB_DBG_MODE__WB_DBG_MODE_EN__SHIFT 0x0
+#define WB_DBG_MODE__WB_DBG_DIN_FMT_MASK 0x2
+#define WB_DBG_MODE__WB_DBG_DIN_FMT__SHIFT 0x1
+#define WB_DBG_MODE__WB_DBG_36MODE_MASK 0x4
+#define WB_DBG_MODE__WB_DBG_36MODE__SHIFT 0x2
+#define WB_DBG_MODE__WB_DBG_CMAP_MASK 0x8
+#define WB_DBG_MODE__WB_DBG_CMAP__SHIFT 0x3
+#define WB_DBG_MODE__WB_DBG_PXLRATE_ERROR_MASK 0x100
+#define WB_DBG_MODE__WB_DBG_PXLRATE_ERROR__SHIFT 0x8
+#define WB_DBG_MODE__WB_DBG_SOURCE_WIDTH_MASK 0x7fff0000
+#define WB_DBG_MODE__WB_DBG_SOURCE_WIDTH__SHIFT 0x10
+#define WB_HW_DEBUG__WB_HW_DEBUG_MASK 0xffffffff
+#define WB_HW_DEBUG__WB_HW_DEBUG__SHIFT 0x0
+#define CNV_INPUT_SELECT__CNV_INPUT_SRC_SELECT_MASK 0x3
+#define CNV_INPUT_SELECT__CNV_INPUT_SRC_SELECT__SHIFT 0x0
+#define CNV_INPUT_SELECT__CNV_INPUT_PIPE_SELECT_MASK 0x1c
+#define CNV_INPUT_SELECT__CNV_INPUT_PIPE_SELECT__SHIFT 0x2
+#define WB_SOFT_RESET__WB_SOFT_RESET_MASK 0x1
+#define WB_SOFT_RESET__WB_SOFT_RESET__SHIFT 0x0
+#define WB_WARM_UP_MODE_CTL1__WIDTH_WARMUP_MASK 0x7fff
+#define WB_WARM_UP_MODE_CTL1__WIDTH_WARMUP__SHIFT 0x0
+#define WB_WARM_UP_MODE_CTL1__HEIGHT_WARMUP_MASK 0x7fff0000
+#define WB_WARM_UP_MODE_CTL1__HEIGHT_WARMUP__SHIFT 0x10
+#define WB_WARM_UP_MODE_CTL1__GMC_WARM_UP_ENABLE_MASK 0x80000000
+#define WB_WARM_UP_MODE_CTL1__GMC_WARM_UP_ENABLE__SHIFT 0x1f
+#define WB_WARM_UP_MODE_CTL2__DATA_VALUE_WARMUP_MASK 0xff
+#define WB_WARM_UP_MODE_CTL2__DATA_VALUE_WARMUP__SHIFT 0x0
+#define WB_WARM_UP_MODE_CTL2__MODE_WARMUP_MASK 0x100
+#define WB_WARM_UP_MODE_CTL2__MODE_WARMUP__SHIFT 0x8
+#define CNV_TEST_DEBUG_INDEX__CNV_TEST_DEBUG_INDEX_MASK 0xff
+#define CNV_TEST_DEBUG_INDEX__CNV_TEST_DEBUG_INDEX__SHIFT 0x0
+#define CNV_TEST_DEBUG_INDEX__CNV_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define CNV_TEST_DEBUG_INDEX__CNV_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define CNV_TEST_DEBUG_DATA__CNV_TEST_DEBUG_DATA_MASK 0xffffffff
+#define CNV_TEST_DEBUG_DATA__CNV_TEST_DEBUG_DATA__SHIFT 0x0
+#define DCFE_CLOCK_CONTROL__DISPCLK_R_DCFE_GATE_DISABLE_MASK 0x10
+#define DCFE_CLOCK_CONTROL__DISPCLK_R_DCFE_GATE_DISABLE__SHIFT 0x4
+#define DCFE_CLOCK_CONTROL__DISPCLK_G_DCP_GATE_DISABLE_MASK 0x100
+#define DCFE_CLOCK_CONTROL__DISPCLK_G_DCP_GATE_DISABLE__SHIFT 0x8
+#define DCFE_CLOCK_CONTROL__DISPCLK_G_SCL_GATE_DISABLE_MASK 0x1000
+#define DCFE_CLOCK_CONTROL__DISPCLK_G_SCL_GATE_DISABLE__SHIFT 0xc
+#define DCFE_CLOCK_CONTROL__DISPCLK_G_PSCL_GATE_DISABLE_MASK 0x8000
+#define DCFE_CLOCK_CONTROL__DISPCLK_G_PSCL_GATE_DISABLE__SHIFT 0xf
+#define DCFE_CLOCK_CONTROL__DISPCLK_G_PIPE_REQUEST_DIS_GATE_DISABLE_MASK 0x20000
+#define DCFE_CLOCK_CONTROL__DISPCLK_G_PIPE_REQUEST_DIS_GATE_DISABLE__SHIFT 0x11
+#define DCFE_CLOCK_CONTROL__DCFE_TEST_CLK_SEL_MASK 0x1f000000
+#define DCFE_CLOCK_CONTROL__DCFE_TEST_CLK_SEL__SHIFT 0x18
+#define DCFE_CLOCK_CONTROL__DCFE_CLOCK_ENABLE_MASK 0x80000000
+#define DCFE_CLOCK_CONTROL__DCFE_CLOCK_ENABLE__SHIFT 0x1f
+#define DCFE_SOFT_RESET__DCP_PIXPIPE_SOFT_RESET_MASK 0x1
+#define DCFE_SOFT_RESET__DCP_PIXPIPE_SOFT_RESET__SHIFT 0x0
+#define DCFE_SOFT_RESET__DCP_REQ_SOFT_RESET_MASK 0x2
+#define DCFE_SOFT_RESET__DCP_REQ_SOFT_RESET__SHIFT 0x1
+#define DCFE_SOFT_RESET__SCL_ALU_SOFT_RESET_MASK 0x4
+#define DCFE_SOFT_RESET__SCL_ALU_SOFT_RESET__SHIFT 0x2
+#define DCFE_SOFT_RESET__SCL_SOFT_RESET_MASK 0x8
+#define DCFE_SOFT_RESET__SCL_SOFT_RESET__SHIFT 0x3
+#define DCFE_SOFT_RESET__CRTC_SOFT_RESET_MASK 0x10
+#define DCFE_SOFT_RESET__CRTC_SOFT_RESET__SHIFT 0x4
+#define DCFE_SOFT_RESET__PSCL_SOFT_RESET_MASK 0x20
+#define DCFE_SOFT_RESET__PSCL_SOFT_RESET__SHIFT 0x5
+#define DCFE_DBG_CONFIG__DCFE_DBG_EN_MASK 0x1
+#define DCFE_DBG_CONFIG__DCFE_DBG_EN__SHIFT 0x0
+#define DCFE_DBG_CONFIG__DCFE_DBG_SEL_MASK 0xf0
+#define DCFE_DBG_CONFIG__DCFE_DBG_SEL__SHIFT 0x4
+#define DCFE_MEM_PWR_CTRL__DCP_LUT_MEM_PWR_FORCE_MASK 0x3
+#define DCFE_MEM_PWR_CTRL__DCP_LUT_MEM_PWR_FORCE__SHIFT 0x0
+#define DCFE_MEM_PWR_CTRL__DCP_LUT_MEM_PWR_DIS_MASK 0x4
+#define DCFE_MEM_PWR_CTRL__DCP_LUT_MEM_PWR_DIS__SHIFT 0x2
+#define DCFE_MEM_PWR_CTRL__DCP_REGAMMA_MEM_PWR_FORCE_MASK 0x18
+#define DCFE_MEM_PWR_CTRL__DCP_REGAMMA_MEM_PWR_FORCE__SHIFT 0x3
+#define DCFE_MEM_PWR_CTRL__DCP_REGAMMA_MEM_PWR_DIS_MASK 0x20
+#define DCFE_MEM_PWR_CTRL__DCP_REGAMMA_MEM_PWR_DIS__SHIFT 0x5
+#define DCFE_MEM_PWR_CTRL__SCL_COEFF_MEM_PWR_FORCE_MASK 0xc0
+#define DCFE_MEM_PWR_CTRL__SCL_COEFF_MEM_PWR_FORCE__SHIFT 0x6
+#define DCFE_MEM_PWR_CTRL__SCL_COEFF_MEM_PWR_DIS_MASK 0x100
+#define DCFE_MEM_PWR_CTRL__SCL_COEFF_MEM_PWR_DIS__SHIFT 0x8
+#define DCFE_MEM_PWR_CTRL__DCP_CURSOR_MEM_PWR_FORCE_MASK 0x600
+#define DCFE_MEM_PWR_CTRL__DCP_CURSOR_MEM_PWR_FORCE__SHIFT 0x9
+#define DCFE_MEM_PWR_CTRL__DCP_CURSOR_MEM_PWR_DIS_MASK 0x800
+#define DCFE_MEM_PWR_CTRL__DCP_CURSOR_MEM_PWR_DIS__SHIFT 0xb
+#define DCFE_MEM_PWR_CTRL__LB0_ALPHA_MEM_PWR_FORCE_MASK 0x3000
+#define DCFE_MEM_PWR_CTRL__LB0_ALPHA_MEM_PWR_FORCE__SHIFT 0xc
+#define DCFE_MEM_PWR_CTRL__LB0_ALPHA_MEM_PWR_DIS_MASK 0x4000
+#define DCFE_MEM_PWR_CTRL__LB0_ALPHA_MEM_PWR_DIS__SHIFT 0xe
+#define DCFE_MEM_PWR_CTRL__LB1_ALPHA_MEM_PWR_FORCE_MASK 0x18000
+#define DCFE_MEM_PWR_CTRL__LB1_ALPHA_MEM_PWR_FORCE__SHIFT 0xf
+#define DCFE_MEM_PWR_CTRL__LB1_ALPHA_MEM_PWR_DIS_MASK 0x20000
+#define DCFE_MEM_PWR_CTRL__LB1_ALPHA_MEM_PWR_DIS__SHIFT 0x11
+#define DCFE_MEM_PWR_CTRL__LB2_ALPHA_MEM_PWR_FORCE_MASK 0xc0000
+#define DCFE_MEM_PWR_CTRL__LB2_ALPHA_MEM_PWR_FORCE__SHIFT 0x12
+#define DCFE_MEM_PWR_CTRL__LB2_ALPHA_MEM_PWR_DIS_MASK 0x100000
+#define DCFE_MEM_PWR_CTRL__LB2_ALPHA_MEM_PWR_DIS__SHIFT 0x14
+#define DCFE_MEM_PWR_CTRL__LB0_MEM_PWR_FORCE_MASK 0x600000
+#define DCFE_MEM_PWR_CTRL__LB0_MEM_PWR_FORCE__SHIFT 0x15
+#define DCFE_MEM_PWR_CTRL__LB0_MEM_PWR_DIS_MASK 0x800000
+#define DCFE_MEM_PWR_CTRL__LB0_MEM_PWR_DIS__SHIFT 0x17
+#define DCFE_MEM_PWR_CTRL__LB1_MEM_PWR_FORCE_MASK 0x3000000
+#define DCFE_MEM_PWR_CTRL__LB1_MEM_PWR_FORCE__SHIFT 0x18
+#define DCFE_MEM_PWR_CTRL__LB1_MEM_PWR_DIS_MASK 0x4000000
+#define DCFE_MEM_PWR_CTRL__LB1_MEM_PWR_DIS__SHIFT 0x1a
+#define DCFE_MEM_PWR_CTRL__LB2_MEM_PWR_FORCE_MASK 0x18000000
+#define DCFE_MEM_PWR_CTRL__LB2_MEM_PWR_FORCE__SHIFT 0x1b
+#define DCFE_MEM_PWR_CTRL__LB2_MEM_PWR_DIS_MASK 0x20000000
+#define DCFE_MEM_PWR_CTRL__LB2_MEM_PWR_DIS__SHIFT 0x1d
+#define DCFE_MEM_PWR_CTRL2__DCP_LUT_MEM_PWR_MODE_SEL_MASK 0x3
+#define DCFE_MEM_PWR_CTRL2__DCP_LUT_MEM_PWR_MODE_SEL__SHIFT 0x0
+#define DCFE_MEM_PWR_CTRL2__DCP_REGAMMA_MEM_PWR_MODE_SEL_MASK 0xc
+#define DCFE_MEM_PWR_CTRL2__DCP_REGAMMA_MEM_PWR_MODE_SEL__SHIFT 0x2
+#define DCFE_MEM_PWR_CTRL2__SCL_COEFF_MEM_PWR_MODE_SEL_MASK 0x30
+#define DCFE_MEM_PWR_CTRL2__SCL_COEFF_MEM_PWR_MODE_SEL__SHIFT 0x4
+#define DCFE_MEM_PWR_CTRL2__DCP_CURSOR_MEM_PWR_MODE_SEL_MASK 0xc0
+#define DCFE_MEM_PWR_CTRL2__DCP_CURSOR_MEM_PWR_MODE_SEL__SHIFT 0x6
+#define DCFE_MEM_PWR_CTRL2__LB_ALPHA_MEM_PWR_MODE_SEL_MASK 0x300
+#define DCFE_MEM_PWR_CTRL2__LB_ALPHA_MEM_PWR_MODE_SEL__SHIFT 0x8
+#define DCFE_MEM_PWR_CTRL2__LB_MEM_PWR_MODE_SEL_MASK 0xc00
+#define DCFE_MEM_PWR_CTRL2__LB_MEM_PWR_MODE_SEL__SHIFT 0xa
+#define DCFE_MEM_PWR_CTRL2__DCP_CURSOR2_MEM_PWR_MODE_SEL_MASK 0x3000
+#define DCFE_MEM_PWR_CTRL2__DCP_CURSOR2_MEM_PWR_MODE_SEL__SHIFT 0xc
+#define DCFE_MEM_PWR_CTRL2__BLND_MEM_PWR_MODE_SEL_MASK 0xc000
+#define DCFE_MEM_PWR_CTRL2__BLND_MEM_PWR_MODE_SEL__SHIFT 0xe
+#define DCFE_MEM_PWR_CTRL2__BLND_MEM_PWR_FORCE_MASK 0x30000
+#define DCFE_MEM_PWR_CTRL2__BLND_MEM_PWR_FORCE__SHIFT 0x10
+#define DCFE_MEM_PWR_CTRL2__BLND_MEM_PWR_DIS_MASK 0x40000
+#define DCFE_MEM_PWR_CTRL2__BLND_MEM_PWR_DIS__SHIFT 0x12
+#define DCFE_MEM_PWR_CTRL2__DCP_CURSOR2_MEM_PWR_FORCE_MASK 0x600000
+#define DCFE_MEM_PWR_CTRL2__DCP_CURSOR2_MEM_PWR_FORCE__SHIFT 0x15
+#define DCFE_MEM_PWR_CTRL2__DCP_CURSOR2_MEM_PWR_DIS_MASK 0x800000
+#define DCFE_MEM_PWR_CTRL2__DCP_CURSOR2_MEM_PWR_DIS__SHIFT 0x17
+#define DCFE_MEM_PWR_STATUS__DCP_LUT_MEM_PWR_STATE_MASK 0x3
+#define DCFE_MEM_PWR_STATUS__DCP_LUT_MEM_PWR_STATE__SHIFT 0x0
+#define DCFE_MEM_PWR_STATUS__DCP_REGAMMA_MEM_PWR_STATE_MASK 0xc
+#define DCFE_MEM_PWR_STATUS__DCP_REGAMMA_MEM_PWR_STATE__SHIFT 0x2
+#define DCFE_MEM_PWR_STATUS__SCL_COEFF_MEM_PWR_STATE_MASK 0x30
+#define DCFE_MEM_PWR_STATUS__SCL_COEFF_MEM_PWR_STATE__SHIFT 0x4
+#define DCFE_MEM_PWR_STATUS__DCP_CURSOR_MEM_PWR_STATE_MASK 0xc0
+#define DCFE_MEM_PWR_STATUS__DCP_CURSOR_MEM_PWR_STATE__SHIFT 0x6
+#define DCFE_MEM_PWR_STATUS__DCP_CURSOR2_MEM_PWR_STATE_MASK 0x300
+#define DCFE_MEM_PWR_STATUS__DCP_CURSOR2_MEM_PWR_STATE__SHIFT 0x8
+#define DCFE_MEM_PWR_STATUS__LB0_ALPHA_MEM_PWR_STATE_MASK 0xc00
+#define DCFE_MEM_PWR_STATUS__LB0_ALPHA_MEM_PWR_STATE__SHIFT 0xa
+#define DCFE_MEM_PWR_STATUS__LB1_ALPHA_MEM_PWR_STATE_MASK 0x3000
+#define DCFE_MEM_PWR_STATUS__LB1_ALPHA_MEM_PWR_STATE__SHIFT 0xc
+#define DCFE_MEM_PWR_STATUS__LB2_ALPHA_MEM_PWR_STATE_MASK 0xc000
+#define DCFE_MEM_PWR_STATUS__LB2_ALPHA_MEM_PWR_STATE__SHIFT 0xe
+#define DCFE_MEM_PWR_STATUS__LB0_MEM_PWR_STATE_MASK 0x30000
+#define DCFE_MEM_PWR_STATUS__LB0_MEM_PWR_STATE__SHIFT 0x10
+#define DCFE_MEM_PWR_STATUS__LB1_MEM_PWR_STATE_MASK 0xc0000
+#define DCFE_MEM_PWR_STATUS__LB1_MEM_PWR_STATE__SHIFT 0x12
+#define DCFE_MEM_PWR_STATUS__LB2_MEM_PWR_STATE_MASK 0x300000
+#define DCFE_MEM_PWR_STATUS__LB2_MEM_PWR_STATE__SHIFT 0x14
+#define DCFE_MEM_PWR_STATUS__BLND_MEM_PWR_STATE_MASK 0xc00000
+#define DCFE_MEM_PWR_STATUS__BLND_MEM_PWR_STATE__SHIFT 0x16
+#define DCFE_MISC__DCFE_DPG_ALLOW_SR_ECO_EN_MASK 0x1
+#define DCFE_MISC__DCFE_DPG_ALLOW_SR_ECO_EN__SHIFT 0x0
+#define DCFE_FLUSH__FLUSH_OCCURED_MASK 0x1
+#define DCFE_FLUSH__FLUSH_OCCURED__SHIFT 0x0
+#define DCFE_FLUSH__CLEAR_FLUSH_OCCURED_MASK 0x2
+#define DCFE_FLUSH__CLEAR_FLUSH_OCCURED__SHIFT 0x1
+#define DCFE_FLUSH__FLUSH_DEEP_MASK 0x4
+#define DCFE_FLUSH__FLUSH_DEEP__SHIFT 0x2
+#define DCFE_FLUSH__CLEAR_FLUSH_DEEP_MASK 0x8
+#define DCFE_FLUSH__CLEAR_FLUSH_DEEP__SHIFT 0x3
+#define DCFE_FLUSH__ALL_MC_REQ_RET_MASK 0x10
+#define DCFE_FLUSH__ALL_MC_REQ_RET__SHIFT 0x4
+#define DCFEV_CLOCK_CONTROL__DISPCLK_R_DCFEV_GATE_DISABLE_MASK 0x8
+#define DCFEV_CLOCK_CONTROL__DISPCLK_R_DCFEV_GATE_DISABLE__SHIFT 0x3
+#define DCFEV_CLOCK_CONTROL__DISPCLK_G_UNP_GATE_DISABLE_MASK 0x80
+#define DCFEV_CLOCK_CONTROL__DISPCLK_G_UNP_GATE_DISABLE__SHIFT 0x7
+#define DCFEV_CLOCK_CONTROL__DISPCLK_G_SCLV_GATE_DISABLE_MASK 0x200
+#define DCFEV_CLOCK_CONTROL__DISPCLK_G_SCLV_GATE_DISABLE__SHIFT 0x9
+#define DCFEV_CLOCK_CONTROL__DISPCLK_G_COL_MAN_GATE_DISABLE_MASK 0x800
+#define DCFEV_CLOCK_CONTROL__DISPCLK_G_COL_MAN_GATE_DISABLE__SHIFT 0xb
+#define DCFEV_CLOCK_CONTROL__DISPCLK_G_PSCLV_GATE_DISABLE_MASK 0x2000
+#define DCFEV_CLOCK_CONTROL__DISPCLK_G_PSCLV_GATE_DISABLE__SHIFT 0xd
+#define DCFEV_CLOCK_CONTROL__DISPCLK_G_CRTC_GATE_DISABLE_MASK 0x8000
+#define DCFEV_CLOCK_CONTROL__DISPCLK_G_CRTC_GATE_DISABLE__SHIFT 0xf
+#define DCFEV_CLOCK_CONTROL__DCFEV_TEST_CLK_SEL_MASK 0x1f000000
+#define DCFEV_CLOCK_CONTROL__DCFEV_TEST_CLK_SEL__SHIFT 0x18
+#define DCFEV_CLOCK_CONTROL__DCFEV_CLOCK_ENABLE_MASK 0x80000000
+#define DCFEV_CLOCK_CONTROL__DCFEV_CLOCK_ENABLE__SHIFT 0x1f
+#define DCFEV_SOFT_RESET__UNP_PIXPIPE_SOFT_RESET_MASK 0x1
+#define DCFEV_SOFT_RESET__UNP_PIXPIPE_SOFT_RESET__SHIFT 0x0
+#define DCFEV_SOFT_RESET__UNP_REQ_SOFT_RESET_MASK 0x2
+#define DCFEV_SOFT_RESET__UNP_REQ_SOFT_RESET__SHIFT 0x1
+#define DCFEV_SOFT_RESET__SCLV_ALU_SOFT_RESET_MASK 0x4
+#define DCFEV_SOFT_RESET__SCLV_ALU_SOFT_RESET__SHIFT 0x2
+#define DCFEV_SOFT_RESET__SCLV_SOFT_RESET_MASK 0x8
+#define DCFEV_SOFT_RESET__SCLV_SOFT_RESET__SHIFT 0x3
+#define DCFEV_SOFT_RESET__CRTC_SOFT_RESET_MASK 0x10
+#define DCFEV_SOFT_RESET__CRTC_SOFT_RESET__SHIFT 0x4
+#define DCFEV_SOFT_RESET__PSCLV_SOFT_RESET_MASK 0x20
+#define DCFEV_SOFT_RESET__PSCLV_SOFT_RESET__SHIFT 0x5
+#define DCFEV_SOFT_RESET__COL_MAN_SOFT_RESET_MASK 0x40
+#define DCFEV_SOFT_RESET__COL_MAN_SOFT_RESET__SHIFT 0x6
+#define DCFEV_DMIFV_CLOCK_CONTROL__DMIFV_SCLK_G_DMIFTRK_GATE_DIS_MASK 0x8
+#define DCFEV_DMIFV_CLOCK_CONTROL__DMIFV_SCLK_G_DMIFTRK_GATE_DIS__SHIFT 0x3
+#define DCFEV_DMIFV_CLOCK_CONTROL__DMIFV_DISPCLK_G_DMIFVL_GATE_DIS_MASK 0x10
+#define DCFEV_DMIFV_CLOCK_CONTROL__DMIFV_DISPCLK_G_DMIFVL_GATE_DIS__SHIFT 0x4
+#define DCFEV_DMIFV_CLOCK_CONTROL__DMIFV_DISPCLK_G_DMIFVC_GATE_DIS_MASK 0x20
+#define DCFEV_DMIFV_CLOCK_CONTROL__DMIFV_DISPCLK_G_DMIFVC_GATE_DIS__SHIFT 0x5
+#define DCFEV_DMIFV_CLOCK_CONTROL__DMIFV_SOFT_RESET_MASK 0x40
+#define DCFEV_DMIFV_CLOCK_CONTROL__DMIFV_SOFT_RESET__SHIFT 0x6
+#define DCFEV_DMIFV_CLOCK_CONTROL__DMIFV_TEST_CLK_SEL_MASK 0x1f000000
+#define DCFEV_DMIFV_CLOCK_CONTROL__DMIFV_TEST_CLK_SEL__SHIFT 0x18
+#define DCFEV_DMIFV_CLOCK_CONTROL__DMIFV_BUFFER_MODE_MASK 0x80000000
+#define DCFEV_DMIFV_CLOCK_CONTROL__DMIFV_BUFFER_MODE__SHIFT 0x1f
+#define DCFEV_DBG_CONFIG__DCFEV_DBG_EN_MASK 0x1
+#define DCFEV_DBG_CONFIG__DCFEV_DBG_EN__SHIFT 0x0
+#define DCFEV_DBG_CONFIG__DCFEV_DBG_SEL_MASK 0xf0
+#define DCFEV_DBG_CONFIG__DCFEV_DBG_SEL__SHIFT 0x4
+#define DCFEV_DMIFV_MEM_PWR_CTRL__DMIFV_MEM_PWR_SEL_MASK 0x3
+#define DCFEV_DMIFV_MEM_PWR_CTRL__DMIFV_MEM_PWR_SEL__SHIFT 0x0
+#define DCFEV_DMIFV_MEM_PWR_CTRL__DMIFV_MEM_PWR_LUMA_0_FORCE_MASK 0x4
+#define DCFEV_DMIFV_MEM_PWR_CTRL__DMIFV_MEM_PWR_LUMA_0_FORCE__SHIFT 0x2
+#define DCFEV_DMIFV_MEM_PWR_CTRL__DMIFV_MEM_PWR_LUMA_1_FORCE_MASK 0x8
+#define DCFEV_DMIFV_MEM_PWR_CTRL__DMIFV_MEM_PWR_LUMA_1_FORCE__SHIFT 0x3
+#define DCFEV_DMIFV_MEM_PWR_CTRL__DMIFV_MEM_PWR_LUMA_2_FORCE_MASK 0x10
+#define DCFEV_DMIFV_MEM_PWR_CTRL__DMIFV_MEM_PWR_LUMA_2_FORCE__SHIFT 0x4
+#define DCFEV_DMIFV_MEM_PWR_CTRL__DMIFV_MEM_PWR_LUMA_3_FORCE_MASK 0x20
+#define DCFEV_DMIFV_MEM_PWR_CTRL__DMIFV_MEM_PWR_LUMA_3_FORCE__SHIFT 0x5
+#define DCFEV_DMIFV_MEM_PWR_CTRL__DMIFV_MEM_PWR_LUMA_4_FORCE_MASK 0x40
+#define DCFEV_DMIFV_MEM_PWR_CTRL__DMIFV_MEM_PWR_LUMA_4_FORCE__SHIFT 0x6
+#define DCFEV_DMIFV_MEM_PWR_CTRL__DMIFV_MEM_PWR_CHROMA_0_FORCE_MASK 0x80
+#define DCFEV_DMIFV_MEM_PWR_CTRL__DMIFV_MEM_PWR_CHROMA_0_FORCE__SHIFT 0x7
+#define DCFEV_DMIFV_MEM_PWR_CTRL__DMIFV_MEM_PWR_CHROMA_1_FORCE_MASK 0x100
+#define DCFEV_DMIFV_MEM_PWR_CTRL__DMIFV_MEM_PWR_CHROMA_1_FORCE__SHIFT 0x8
+#define DCFEV_DMIFV_MEM_PWR_CTRL__DMIFV_MEM_PWR_CHROMA_2_FORCE_MASK 0x200
+#define DCFEV_DMIFV_MEM_PWR_CTRL__DMIFV_MEM_PWR_CHROMA_2_FORCE__SHIFT 0x9
+#define DCFEV_DMIFV_MEM_PWR_CTRL__DMIFV_MEM_PWR_CHROMA_3_FORCE_MASK 0x400
+#define DCFEV_DMIFV_MEM_PWR_CTRL__DMIFV_MEM_PWR_CHROMA_3_FORCE__SHIFT 0xa
+#define DCFEV_DMIFV_MEM_PWR_CTRL__DMIFV_MEM_PWR_CHROMA_4_FORCE_MASK 0x800
+#define DCFEV_DMIFV_MEM_PWR_CTRL__DMIFV_MEM_PWR_CHROMA_4_FORCE__SHIFT 0xb
+#define DCFEV_DMIFV_MEM_PWR_STATUS__DMIFV_MEM_PWR_LUMA_0_STATE_MASK 0x3
+#define DCFEV_DMIFV_MEM_PWR_STATUS__DMIFV_MEM_PWR_LUMA_0_STATE__SHIFT 0x0
+#define DCFEV_DMIFV_MEM_PWR_STATUS__DMIFV_MEM_PWR_LUMA_1_STATE_MASK 0xc
+#define DCFEV_DMIFV_MEM_PWR_STATUS__DMIFV_MEM_PWR_LUMA_1_STATE__SHIFT 0x2
+#define DCFEV_DMIFV_MEM_PWR_STATUS__DMIFV_MEM_PWR_LUMA_2_STATE_MASK 0x30
+#define DCFEV_DMIFV_MEM_PWR_STATUS__DMIFV_MEM_PWR_LUMA_2_STATE__SHIFT 0x4
+#define DCFEV_DMIFV_MEM_PWR_STATUS__DMIFV_MEM_PWR_LUMA_3_STATE_MASK 0xc0
+#define DCFEV_DMIFV_MEM_PWR_STATUS__DMIFV_MEM_PWR_LUMA_3_STATE__SHIFT 0x6
+#define DCFEV_DMIFV_MEM_PWR_STATUS__DMIFV_MEM_PWR_LUMA_4_STATE_MASK 0x300
+#define DCFEV_DMIFV_MEM_PWR_STATUS__DMIFV_MEM_PWR_LUMA_4_STATE__SHIFT 0x8
+#define DCFEV_DMIFV_MEM_PWR_STATUS__DMIFV_MEM_PWR_CHROMA_0_STATE_MASK 0xc00
+#define DCFEV_DMIFV_MEM_PWR_STATUS__DMIFV_MEM_PWR_CHROMA_0_STATE__SHIFT 0xa
+#define DCFEV_DMIFV_MEM_PWR_STATUS__DMIFV_MEM_PWR_CHROMA_1_STATE_MASK 0x3000
+#define DCFEV_DMIFV_MEM_PWR_STATUS__DMIFV_MEM_PWR_CHROMA_1_STATE__SHIFT 0xc
+#define DCFEV_DMIFV_MEM_PWR_STATUS__DMIFV_MEM_PWR_CHROMA_2_STATE_MASK 0xc000
+#define DCFEV_DMIFV_MEM_PWR_STATUS__DMIFV_MEM_PWR_CHROMA_2_STATE__SHIFT 0xe
+#define DCFEV_DMIFV_MEM_PWR_STATUS__DMIFV_MEM_PWR_CHROMA_3_STATE_MASK 0x30000
+#define DCFEV_DMIFV_MEM_PWR_STATUS__DMIFV_MEM_PWR_CHROMA_3_STATE__SHIFT 0x10
+#define DCFEV_DMIFV_MEM_PWR_STATUS__DMIFV_MEM_PWR_CHROMA_4_STATE_MASK 0xc0000
+#define DCFEV_DMIFV_MEM_PWR_STATUS__DMIFV_MEM_PWR_CHROMA_4_STATE__SHIFT 0x12
+#define DCFEV_MEM_PWR_CTRL__COL_MAN_GAMMA_CORR_MEM_PWR_FORCE_MASK 0x3
+#define DCFEV_MEM_PWR_CTRL__COL_MAN_GAMMA_CORR_MEM_PWR_FORCE__SHIFT 0x0
+#define DCFEV_MEM_PWR_CTRL__COL_MAN_GAMMA_CORR_MEM_PWR_DIS_MASK 0x4
+#define DCFEV_MEM_PWR_CTRL__COL_MAN_GAMMA_CORR_MEM_PWR_DIS__SHIFT 0x2
+#define DCFEV_MEM_PWR_CTRL__COL_MAN_INPUT_GAMMA_MEM_PWR_FORCE_MASK 0x18
+#define DCFEV_MEM_PWR_CTRL__COL_MAN_INPUT_GAMMA_MEM_PWR_FORCE__SHIFT 0x3
+#define DCFEV_MEM_PWR_CTRL__COL_MAN_INPUT_GAMMA_MEM_PWR_DIS_MASK 0x20
+#define DCFEV_MEM_PWR_CTRL__COL_MAN_INPUT_GAMMA_MEM_PWR_DIS__SHIFT 0x5
+#define DCFEV_MEM_PWR_CTRL__SCLV_COEFF_MEM_PWR_FORCE_MASK 0xc0
+#define DCFEV_MEM_PWR_CTRL__SCLV_COEFF_MEM_PWR_FORCE__SHIFT 0x6
+#define DCFEV_MEM_PWR_CTRL__SCLV_COEFF_MEM_PWR_DIS_MASK 0x100
+#define DCFEV_MEM_PWR_CTRL__SCLV_COEFF_MEM_PWR_DIS__SHIFT 0x8
+#define DCFEV_MEM_PWR_CTRL__LBV0_MEM_PWR_FORCE_MASK 0x600
+#define DCFEV_MEM_PWR_CTRL__LBV0_MEM_PWR_FORCE__SHIFT 0x9
+#define DCFEV_MEM_PWR_CTRL__LBV0_MEM_PWR_DIS_MASK 0x800
+#define DCFEV_MEM_PWR_CTRL__LBV0_MEM_PWR_DIS__SHIFT 0xb
+#define DCFEV_MEM_PWR_CTRL__LBV1_MEM_PWR_FORCE_MASK 0x3000
+#define DCFEV_MEM_PWR_CTRL__LBV1_MEM_PWR_FORCE__SHIFT 0xc
+#define DCFEV_MEM_PWR_CTRL__LBV1_MEM_PWR_DIS_MASK 0x4000
+#define DCFEV_MEM_PWR_CTRL__LBV1_MEM_PWR_DIS__SHIFT 0xe
+#define DCFEV_MEM_PWR_CTRL__LBV2_MEM_PWR_FORCE_MASK 0x18000
+#define DCFEV_MEM_PWR_CTRL__LBV2_MEM_PWR_FORCE__SHIFT 0xf
+#define DCFEV_MEM_PWR_CTRL__LBV2_MEM_PWR_DIS_MASK 0x20000
+#define DCFEV_MEM_PWR_CTRL__LBV2_MEM_PWR_DIS__SHIFT 0x11
+#define DCFEV_MEM_PWR_CTRL2__COL_MAN_GAMMA_CORR_MEM_PWR_MODE_SEL_MASK 0x3
+#define DCFEV_MEM_PWR_CTRL2__COL_MAN_GAMMA_CORR_MEM_PWR_MODE_SEL__SHIFT 0x0
+#define DCFEV_MEM_PWR_CTRL2__COL_MAN_INPUT_GAMMA_MEM_PWR_MODE_SEL_MASK 0xc
+#define DCFEV_MEM_PWR_CTRL2__COL_MAN_INPUT_GAMMA_MEM_PWR_MODE_SEL__SHIFT 0x2
+#define DCFEV_MEM_PWR_CTRL2__SCLV_COEFF_MEM_PWR_MODE_SEL_MASK 0x30
+#define DCFEV_MEM_PWR_CTRL2__SCLV_COEFF_MEM_PWR_MODE_SEL__SHIFT 0x4
+#define DCFEV_MEM_PWR_CTRL2__LBV_MEM_PWR_MODE_SEL_MASK 0xc0
+#define DCFEV_MEM_PWR_CTRL2__LBV_MEM_PWR_MODE_SEL__SHIFT 0x6
+#define DCFEV_MEM_PWR_STATUS__COL_MAN_GAMMA_CORR_MEM_PWR_STATE_MASK 0x3
+#define DCFEV_MEM_PWR_STATUS__COL_MAN_GAMMA_CORR_MEM_PWR_STATE__SHIFT 0x0
+#define DCFEV_MEM_PWR_STATUS__COL_MAN_INPUT_GAMMA_MEM_PWR_STATE_MASK 0xc
+#define DCFEV_MEM_PWR_STATUS__COL_MAN_INPUT_GAMMA_MEM_PWR_STATE__SHIFT 0x2
+#define DCFEV_MEM_PWR_STATUS__SCLV_COEFF_MEM_PWR_STATE_MASK 0x30
+#define DCFEV_MEM_PWR_STATUS__SCLV_COEFF_MEM_PWR_STATE__SHIFT 0x4
+#define DCFEV_MEM_PWR_STATUS__LBV0_MEM_PWR_STATE_MASK 0xc0
+#define DCFEV_MEM_PWR_STATUS__LBV0_MEM_PWR_STATE__SHIFT 0x6
+#define DCFEV_MEM_PWR_STATUS__LBV1_MEM_PWR_STATE_MASK 0x300
+#define DCFEV_MEM_PWR_STATUS__LBV1_MEM_PWR_STATE__SHIFT 0x8
+#define DCFEV_MEM_PWR_STATUS__LBV2_MEM_PWR_STATE_MASK 0xc00
+#define DCFEV_MEM_PWR_STATUS__LBV2_MEM_PWR_STATE__SHIFT 0xa
+#define DCFEV_MEM_PWR_STATUS__LBV3_MEM_PWR_STATE_MASK 0x3000
+#define DCFEV_MEM_PWR_STATUS__LBV3_MEM_PWR_STATE__SHIFT 0xc
+#define DCFEV_L_FLUSH__FLUSH_OCCURED_MASK 0x1
+#define DCFEV_L_FLUSH__FLUSH_OCCURED__SHIFT 0x0
+#define DCFEV_L_FLUSH__CLEAR_FLUSH_OCCURED_MASK 0x2
+#define DCFEV_L_FLUSH__CLEAR_FLUSH_OCCURED__SHIFT 0x1
+#define DCFEV_L_FLUSH__FLUSH_DEEP_MASK 0x4
+#define DCFEV_L_FLUSH__FLUSH_DEEP__SHIFT 0x2
+#define DCFEV_L_FLUSH__CLEAR_FLUSH_DEEP_MASK 0x8
+#define DCFEV_L_FLUSH__CLEAR_FLUSH_DEEP__SHIFT 0x3
+#define DCFEV_L_FLUSH__ALL_MC_REQ_RET_MASK 0x10
+#define DCFEV_L_FLUSH__ALL_MC_REQ_RET__SHIFT 0x4
+#define DCFEV_C_FLUSH__FLUSH_OCCURED_MASK 0x1
+#define DCFEV_C_FLUSH__FLUSH_OCCURED__SHIFT 0x0
+#define DCFEV_C_FLUSH__CLEAR_FLUSH_OCCURED_MASK 0x2
+#define DCFEV_C_FLUSH__CLEAR_FLUSH_OCCURED__SHIFT 0x1
+#define DCFEV_C_FLUSH__FLUSH_DEEP_MASK 0x4
+#define DCFEV_C_FLUSH__FLUSH_DEEP__SHIFT 0x2
+#define DCFEV_C_FLUSH__CLEAR_FLUSH_DEEP_MASK 0x8
+#define DCFEV_C_FLUSH__CLEAR_FLUSH_DEEP__SHIFT 0x3
+#define DCFEV_C_FLUSH__ALL_MC_REQ_RET_MASK 0x10
+#define DCFEV_C_FLUSH__ALL_MC_REQ_RET__SHIFT 0x4
+#define DCFEV_DMIFV_DEBUG__DMIFV_DEBUG_BUS_SEL_MASK 0xf
+#define DCFEV_DMIFV_DEBUG__DMIFV_DEBUG_BUS_SEL__SHIFT 0x0
+#define DCFEV_DMIFV_DEBUG__DMIFV_DEBUG_LUMA_VS_CHROMA_MASK 0x10
+#define DCFEV_DMIFV_DEBUG__DMIFV_DEBUG_LUMA_VS_CHROMA__SHIFT 0x4
+#define DCFEV_DMIFV_DEBUG__DMIFV_DEBUG_LOWER_UPPER_MASK 0x20
+#define DCFEV_DMIFV_DEBUG__DMIFV_DEBUG_LOWER_UPPER__SHIFT 0x5
+#define DCFEV_MISC__DCFEV_DPG_ALLOW_SR_ECO_EN_MASK 0x1
+#define DCFEV_MISC__DCFEV_DPG_ALLOW_SR_ECO_EN__SHIFT 0x0
+#define DC_HPD_INT_STATUS__DC_HPD_INT_STATUS_MASK 0x1
+#define DC_HPD_INT_STATUS__DC_HPD_INT_STATUS__SHIFT 0x0
+#define DC_HPD_INT_STATUS__DC_HPD_SENSE_MASK 0x2
+#define DC_HPD_INT_STATUS__DC_HPD_SENSE__SHIFT 0x1
+#define DC_HPD_INT_STATUS__DC_HPD_SENSE_DELAYED_MASK 0x10
+#define DC_HPD_INT_STATUS__DC_HPD_SENSE_DELAYED__SHIFT 0x4
+#define DC_HPD_INT_STATUS__DC_HPD_RX_INT_STATUS_MASK 0x100
+#define DC_HPD_INT_STATUS__DC_HPD_RX_INT_STATUS__SHIFT 0x8
+#define DC_HPD_INT_STATUS__DC_HPD_TOGGLE_FILT_CON_TIMER_VAL_MASK 0xff000
+#define DC_HPD_INT_STATUS__DC_HPD_TOGGLE_FILT_CON_TIMER_VAL__SHIFT 0xc
+#define DC_HPD_INT_STATUS__DC_HPD_TOGGLE_FILT_DISCON_TIMER_VAL_MASK 0xff000000
+#define DC_HPD_INT_STATUS__DC_HPD_TOGGLE_FILT_DISCON_TIMER_VAL__SHIFT 0x18
+#define DC_HPD_INT_CONTROL__DC_HPD_INT_ACK_MASK 0x1
+#define DC_HPD_INT_CONTROL__DC_HPD_INT_ACK__SHIFT 0x0
+#define DC_HPD_INT_CONTROL__DC_HPD_INT_POLARITY_MASK 0x100
+#define DC_HPD_INT_CONTROL__DC_HPD_INT_POLARITY__SHIFT 0x8
+#define DC_HPD_INT_CONTROL__DC_HPD_INT_EN_MASK 0x10000
+#define DC_HPD_INT_CONTROL__DC_HPD_INT_EN__SHIFT 0x10
+#define DC_HPD_INT_CONTROL__DC_HPD_RX_INT_ACK_MASK 0x100000
+#define DC_HPD_INT_CONTROL__DC_HPD_RX_INT_ACK__SHIFT 0x14
+#define DC_HPD_INT_CONTROL__DC_HPD_RX_INT_EN_MASK 0x1000000
+#define DC_HPD_INT_CONTROL__DC_HPD_RX_INT_EN__SHIFT 0x18
+#define DC_HPD_CONTROL__DC_HPD_CONNECTION_TIMER_MASK 0x1fff
+#define DC_HPD_CONTROL__DC_HPD_CONNECTION_TIMER__SHIFT 0x0
+#define DC_HPD_CONTROL__DC_HPD_RX_INT_TIMER_MASK 0x3ff0000
+#define DC_HPD_CONTROL__DC_HPD_RX_INT_TIMER__SHIFT 0x10
+#define DC_HPD_CONTROL__DC_HPD_EN_MASK 0x10000000
+#define DC_HPD_CONTROL__DC_HPD_EN__SHIFT 0x1c
+#define DC_HPD_FAST_TRAIN_CNTL__DC_HPD_CONNECT_AUX_TX_DELAY_MASK 0xff
+#define DC_HPD_FAST_TRAIN_CNTL__DC_HPD_CONNECT_AUX_TX_DELAY__SHIFT 0x0
+#define DC_HPD_FAST_TRAIN_CNTL__DC_HPD_CONNECT_FAST_TRAIN_DELAY_MASK 0xff000
+#define DC_HPD_FAST_TRAIN_CNTL__DC_HPD_CONNECT_FAST_TRAIN_DELAY__SHIFT 0xc
+#define DC_HPD_FAST_TRAIN_CNTL__DC_HPD_CONNECT_AUX_TX_EN_MASK 0x1000000
+#define DC_HPD_FAST_TRAIN_CNTL__DC_HPD_CONNECT_AUX_TX_EN__SHIFT 0x18
+#define DC_HPD_FAST_TRAIN_CNTL__DC_HPD_CONNECT_FAST_TRAIN_EN_MASK 0x10000000
+#define DC_HPD_FAST_TRAIN_CNTL__DC_HPD_CONNECT_FAST_TRAIN_EN__SHIFT 0x1c
+#define DC_HPD_TOGGLE_FILT_CNTL__DC_HPD_CONNECT_INT_DELAY_MASK 0xff
+#define DC_HPD_TOGGLE_FILT_CNTL__DC_HPD_CONNECT_INT_DELAY__SHIFT 0x0
+#define DC_HPD_TOGGLE_FILT_CNTL__DC_HPD_DISCONNECT_INT_DELAY_MASK 0xff00000
+#define DC_HPD_TOGGLE_FILT_CNTL__DC_HPD_DISCONNECT_INT_DELAY__SHIFT 0x14
+#define DCO_SCRATCH0__DCO_SCRATCH0_MASK 0xffffffff
+#define DCO_SCRATCH0__DCO_SCRATCH0__SHIFT 0x0
+#define DCO_SCRATCH1__DCO_SCRATCH1_MASK 0xffffffff
+#define DCO_SCRATCH1__DCO_SCRATCH1__SHIFT 0x0
+#define DCO_SCRATCH2__DCO_SCRATCH2_MASK 0xffffffff
+#define DCO_SCRATCH2__DCO_SCRATCH2__SHIFT 0x0
+#define DCO_SCRATCH3__DCO_SCRATCH3_MASK 0xffffffff
+#define DCO_SCRATCH3__DCO_SCRATCH3__SHIFT 0x0
+#define DCO_SCRATCH4__DCO_SCRATCH4_MASK 0xffffffff
+#define DCO_SCRATCH4__DCO_SCRATCH4__SHIFT 0x0
+#define DCO_SCRATCH5__DCO_SCRATCH5_MASK 0xffffffff
+#define DCO_SCRATCH5__DCO_SCRATCH5__SHIFT 0x0
+#define DCO_SCRATCH6__DCO_SCRATCH6_MASK 0xffffffff
+#define DCO_SCRATCH6__DCO_SCRATCH6__SHIFT 0x0
+#define DCO_SCRATCH7__DCO_SCRATCH7_MASK 0xffffffff
+#define DCO_SCRATCH7__DCO_SCRATCH7__SHIFT 0x0
+#define DCE_VCE_CONTROL__DC_VCE_VIDEO_PIPE_SELECT_MASK 0x7
+#define DCE_VCE_CONTROL__DC_VCE_VIDEO_PIPE_SELECT__SHIFT 0x0
+#define DCE_VCE_CONTROL__DC_VCE_AUDIO_STREAM_SELECT_MASK 0x70
+#define DCE_VCE_CONTROL__DC_VCE_AUDIO_STREAM_SELECT__SHIFT 0x4
+#define DISP_INTERRUPT_STATUS__SCL_DISP1_MODE_CHANGE_INTERRUPT_MASK 0x1
+#define DISP_INTERRUPT_STATUS__SCL_DISP1_MODE_CHANGE_INTERRUPT__SHIFT 0x0
+#define DISP_INTERRUPT_STATUS__D1BLND_DATA_UNDERFLOW_INTERRUPT_MASK 0x2
+#define DISP_INTERRUPT_STATUS__D1BLND_DATA_UNDERFLOW_INTERRUPT__SHIFT 0x1
+#define DISP_INTERRUPT_STATUS__LB_D1_VLINE_INTERRUPT_MASK 0x4
+#define DISP_INTERRUPT_STATUS__LB_D1_VLINE_INTERRUPT__SHIFT 0x2
+#define DISP_INTERRUPT_STATUS__LB_D1_VBLANK_INTERRUPT_MASK 0x8
+#define DISP_INTERRUPT_STATUS__LB_D1_VBLANK_INTERRUPT__SHIFT 0x3
+#define DISP_INTERRUPT_STATUS__CRTC1_SNAPSHOT_INTERRUPT_MASK 0x10
+#define DISP_INTERRUPT_STATUS__CRTC1_SNAPSHOT_INTERRUPT__SHIFT 0x4
+#define DISP_INTERRUPT_STATUS__CRTC1_FORCE_VSYNC_NEXT_LINE_INTERRUPT_MASK 0x20
+#define DISP_INTERRUPT_STATUS__CRTC1_FORCE_VSYNC_NEXT_LINE_INTERRUPT__SHIFT 0x5
+#define DISP_INTERRUPT_STATUS__CRTC1_FORCE_COUNT_NOW_INTERRUPT_MASK 0x40
+#define DISP_INTERRUPT_STATUS__CRTC1_FORCE_COUNT_NOW_INTERRUPT__SHIFT 0x6
+#define DISP_INTERRUPT_STATUS__CRTC1_TRIGA_INTERRUPT_MASK 0x80
+#define DISP_INTERRUPT_STATUS__CRTC1_TRIGA_INTERRUPT__SHIFT 0x7
+#define DISP_INTERRUPT_STATUS__CRTC1_TRIGB_INTERRUPT_MASK 0x100
+#define DISP_INTERRUPT_STATUS__CRTC1_TRIGB_INTERRUPT__SHIFT 0x8
+#define DISP_INTERRUPT_STATUS__CRTC1_VSYNC_NOM_INTERRUPT_MASK 0x200
+#define DISP_INTERRUPT_STATUS__CRTC1_VSYNC_NOM_INTERRUPT__SHIFT 0x9
+#define DISP_INTERRUPT_STATUS__CRTC1_SET_V_TOTAL_MIN_EVENT_OCCURED_INT_MASK 0x400
+#define DISP_INTERRUPT_STATUS__CRTC1_SET_V_TOTAL_MIN_EVENT_OCCURED_INT__SHIFT 0xa
+#define DISP_INTERRUPT_STATUS__DIGA_DP_FAST_TRAINING_COMPLETE_INTERRUPT_MASK 0x8000
+#define DISP_INTERRUPT_STATUS__DIGA_DP_FAST_TRAINING_COMPLETE_INTERRUPT__SHIFT 0xf
+#define DISP_INTERRUPT_STATUS__DIGA_DP_VID_STREAM_DISABLE_INTERRUPT_MASK 0x10000
+#define DISP_INTERRUPT_STATUS__DIGA_DP_VID_STREAM_DISABLE_INTERRUPT__SHIFT 0x10
+#define DISP_INTERRUPT_STATUS__DC_HPD1_INTERRUPT_MASK 0x20000
+#define DISP_INTERRUPT_STATUS__DC_HPD1_INTERRUPT__SHIFT 0x11
+#define DISP_INTERRUPT_STATUS__DC_HPD1_RX_INTERRUPT_MASK 0x40000
+#define DISP_INTERRUPT_STATUS__DC_HPD1_RX_INTERRUPT__SHIFT 0x12
+#define DISP_INTERRUPT_STATUS__AUX1_SW_DONE_INTERRUPT_MASK 0x80000
+#define DISP_INTERRUPT_STATUS__AUX1_SW_DONE_INTERRUPT__SHIFT 0x13
+#define DISP_INTERRUPT_STATUS__AUX1_LS_DONE_INTERRUPT_MASK 0x100000
+#define DISP_INTERRUPT_STATUS__AUX1_LS_DONE_INTERRUPT__SHIFT 0x14
+#define DISP_INTERRUPT_STATUS__DIGA_DISPCLK_SWITCH_ALLOWED_INTERRUPT_MASK 0x200000
+#define DISP_INTERRUPT_STATUS__DIGA_DISPCLK_SWITCH_ALLOWED_INTERRUPT__SHIFT 0x15
+#define DISP_INTERRUPT_STATUS__DACA_AUTODETECT_INTERRUPT_MASK 0x400000
+#define DISP_INTERRUPT_STATUS__DACA_AUTODETECT_INTERRUPT__SHIFT 0x16
+#define DISP_INTERRUPT_STATUS__DACB_AUTODETECT_INTERRUPT_MASK 0x800000
+#define DISP_INTERRUPT_STATUS__DACB_AUTODETECT_INTERRUPT__SHIFT 0x17
+#define DISP_INTERRUPT_STATUS__DC_I2C_SW_DONE_INTERRUPT_MASK 0x1000000
+#define DISP_INTERRUPT_STATUS__DC_I2C_SW_DONE_INTERRUPT__SHIFT 0x18
+#define DISP_INTERRUPT_STATUS__DC_I2C_HW_DONE_INTERRUPT_MASK 0x2000000
+#define DISP_INTERRUPT_STATUS__DC_I2C_HW_DONE_INTERRUPT__SHIFT 0x19
+#define DISP_INTERRUPT_STATUS__DMCU_UC_INTERNAL_INT_MASK 0x4000000
+#define DISP_INTERRUPT_STATUS__DMCU_UC_INTERNAL_INT__SHIFT 0x1a
+#define DISP_INTERRUPT_STATUS__DMCU_SCP_INT_MASK 0x8000000
+#define DISP_INTERRUPT_STATUS__DMCU_SCP_INT__SHIFT 0x1b
+#define DISP_INTERRUPT_STATUS__ABM1_HG_READY_INT_MASK 0x10000000
+#define DISP_INTERRUPT_STATUS__ABM1_HG_READY_INT__SHIFT 0x1c
+#define DISP_INTERRUPT_STATUS__ABM1_LS_READY_INT_MASK 0x20000000
+#define DISP_INTERRUPT_STATUS__ABM1_LS_READY_INT__SHIFT 0x1d
+#define DISP_INTERRUPT_STATUS__ABM1_BL_UPDATE_INT_MASK 0x40000000
+#define DISP_INTERRUPT_STATUS__ABM1_BL_UPDATE_INT__SHIFT 0x1e
+#define DISP_INTERRUPT_STATUS__DISP_INTERRUPT_STATUS_CONTINUE_MASK 0x80000000
+#define DISP_INTERRUPT_STATUS__DISP_INTERRUPT_STATUS_CONTINUE__SHIFT 0x1f
+#define DISP_INTERRUPT_STATUS_CONTINUE__SCL_DISP2_MODE_CHANGE_INTERRUPT_MASK 0x1
+#define DISP_INTERRUPT_STATUS_CONTINUE__SCL_DISP2_MODE_CHANGE_INTERRUPT__SHIFT 0x0
+#define DISP_INTERRUPT_STATUS_CONTINUE__D2BLND_DATA_UNDERFLOW_INTERRUPT_MASK 0x2
+#define DISP_INTERRUPT_STATUS_CONTINUE__D2BLND_DATA_UNDERFLOW_INTERRUPT__SHIFT 0x1
+#define DISP_INTERRUPT_STATUS_CONTINUE__LB_D2_VLINE_INTERRUPT_MASK 0x4
+#define DISP_INTERRUPT_STATUS_CONTINUE__LB_D2_VLINE_INTERRUPT__SHIFT 0x2
+#define DISP_INTERRUPT_STATUS_CONTINUE__LB_D2_VBLANK_INTERRUPT_MASK 0x8
+#define DISP_INTERRUPT_STATUS_CONTINUE__LB_D2_VBLANK_INTERRUPT__SHIFT 0x3
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC2_SNAPSHOT_INTERRUPT_MASK 0x10
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC2_SNAPSHOT_INTERRUPT__SHIFT 0x4
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC2_FORCE_VSYNC_NEXT_LINE_INTERRUPT_MASK 0x20
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC2_FORCE_VSYNC_NEXT_LINE_INTERRUPT__SHIFT 0x5
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC2_FORCE_COUNT_NOW_INTERRUPT_MASK 0x40
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC2_FORCE_COUNT_NOW_INTERRUPT__SHIFT 0x6
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC2_TRIGA_INTERRUPT_MASK 0x80
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC2_TRIGA_INTERRUPT__SHIFT 0x7
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC2_TRIGB_INTERRUPT_MASK 0x100
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC2_TRIGB_INTERRUPT__SHIFT 0x8
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC2_VSYNC_NOM_INTERRUPT_MASK 0x200
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC2_VSYNC_NOM_INTERRUPT__SHIFT 0x9
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC2_SET_V_TOTAL_MIN_EVENT_OCCURED_INT_MASK 0x400
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC2_SET_V_TOTAL_MIN_EVENT_OCCURED_INT__SHIFT 0xa
+#define DISP_INTERRUPT_STATUS_CONTINUE__DIGB_DP_FAST_TRAINING_COMPLETE_INTERRUPT_MASK 0x8000
+#define DISP_INTERRUPT_STATUS_CONTINUE__DIGB_DP_FAST_TRAINING_COMPLETE_INTERRUPT__SHIFT 0xf
+#define DISP_INTERRUPT_STATUS_CONTINUE__DIGB_DP_VID_STREAM_DISABLE_INTERRUPT_MASK 0x10000
+#define DISP_INTERRUPT_STATUS_CONTINUE__DIGB_DP_VID_STREAM_DISABLE_INTERRUPT__SHIFT 0x10
+#define DISP_INTERRUPT_STATUS_CONTINUE__DC_HPD2_INTERRUPT_MASK 0x20000
+#define DISP_INTERRUPT_STATUS_CONTINUE__DC_HPD2_INTERRUPT__SHIFT 0x11
+#define DISP_INTERRUPT_STATUS_CONTINUE__DC_HPD2_RX_INTERRUPT_MASK 0x40000
+#define DISP_INTERRUPT_STATUS_CONTINUE__DC_HPD2_RX_INTERRUPT__SHIFT 0x12
+#define DISP_INTERRUPT_STATUS_CONTINUE__AUX2_SW_DONE_INTERRUPT_MASK 0x80000
+#define DISP_INTERRUPT_STATUS_CONTINUE__AUX2_SW_DONE_INTERRUPT__SHIFT 0x13
+#define DISP_INTERRUPT_STATUS_CONTINUE__AUX2_LS_DONE_INTERRUPT_MASK 0x100000
+#define DISP_INTERRUPT_STATUS_CONTINUE__AUX2_LS_DONE_INTERRUPT__SHIFT 0x14
+#define DISP_INTERRUPT_STATUS_CONTINUE__LB_D1_VLINE2_INTERRUPT_MASK 0x200000
+#define DISP_INTERRUPT_STATUS_CONTINUE__LB_D1_VLINE2_INTERRUPT__SHIFT 0x15
+#define DISP_INTERRUPT_STATUS_CONTINUE__LB_D2_VLINE2_INTERRUPT_MASK 0x400000
+#define DISP_INTERRUPT_STATUS_CONTINUE__LB_D2_VLINE2_INTERRUPT__SHIFT 0x16
+#define DISP_INTERRUPT_STATUS_CONTINUE__LB_D3_VLINE2_INTERRUPT_MASK 0x800000
+#define DISP_INTERRUPT_STATUS_CONTINUE__LB_D3_VLINE2_INTERRUPT__SHIFT 0x17
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC1_EXT_TIMING_SYNC_LOSS_INTERRUPT_MASK 0x2000000
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC1_EXT_TIMING_SYNC_LOSS_INTERRUPT__SHIFT 0x19
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC1_EXT_TIMING_SYNC_INTERRUPT_MASK 0x4000000
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC1_EXT_TIMING_SYNC_INTERRUPT__SHIFT 0x1a
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC1_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_MASK 0x8000000
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC1_EXT_TIMING_SYNC_SIGNAL_INTERRUPT__SHIFT 0x1b
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC1_VERTICAL_INTERRUPT0_MASK 0x10000000
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC1_VERTICAL_INTERRUPT0__SHIFT 0x1c
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC1_VERTICAL_INTERRUPT1_MASK 0x20000000
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC1_VERTICAL_INTERRUPT1__SHIFT 0x1d
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC1_VERTICAL_INTERRUPT2_MASK 0x40000000
+#define DISP_INTERRUPT_STATUS_CONTINUE__CRTC1_VERTICAL_INTERRUPT2__SHIFT 0x1e
+#define DISP_INTERRUPT_STATUS_CONTINUE__DISP_INTERRUPT_STATUS_CONTINUE2_MASK 0x80000000
+#define DISP_INTERRUPT_STATUS_CONTINUE__DISP_INTERRUPT_STATUS_CONTINUE2__SHIFT 0x1f
+#define DISP_INTERRUPT_STATUS_CONTINUE2__SCL_DISP3_MODE_CHANGE_INTERRUPT_MASK 0x1
+#define DISP_INTERRUPT_STATUS_CONTINUE2__SCL_DISP3_MODE_CHANGE_INTERRUPT__SHIFT 0x0
+#define DISP_INTERRUPT_STATUS_CONTINUE2__D3BLND_DATA_UNDERFLOW_INTERRUPT_MASK 0x2
+#define DISP_INTERRUPT_STATUS_CONTINUE2__D3BLND_DATA_UNDERFLOW_INTERRUPT__SHIFT 0x1
+#define DISP_INTERRUPT_STATUS_CONTINUE2__LB_D3_VLINE_INTERRUPT_MASK 0x4
+#define DISP_INTERRUPT_STATUS_CONTINUE2__LB_D3_VLINE_INTERRUPT__SHIFT 0x2
+#define DISP_INTERRUPT_STATUS_CONTINUE2__LB_D3_VBLANK_INTERRUPT_MASK 0x8
+#define DISP_INTERRUPT_STATUS_CONTINUE2__LB_D3_VBLANK_INTERRUPT__SHIFT 0x3
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC3_SNAPSHOT_INTERRUPT_MASK 0x10
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC3_SNAPSHOT_INTERRUPT__SHIFT 0x4
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC3_FORCE_VSYNC_NEXT_LINE_INTERRUPT_MASK 0x20
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC3_FORCE_VSYNC_NEXT_LINE_INTERRUPT__SHIFT 0x5
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC3_FORCE_COUNT_NOW_INTERRUPT_MASK 0x40
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC3_FORCE_COUNT_NOW_INTERRUPT__SHIFT 0x6
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC3_TRIGA_INTERRUPT_MASK 0x80
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC3_TRIGA_INTERRUPT__SHIFT 0x7
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC3_TRIGB_INTERRUPT_MASK 0x100
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC3_TRIGB_INTERRUPT__SHIFT 0x8
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC3_VSYNC_NOM_INTERRUPT_MASK 0x200
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC3_VSYNC_NOM_INTERRUPT__SHIFT 0x9
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC3_SET_V_TOTAL_MIN_EVENT_OCCURED_INT_MASK 0x400
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC3_SET_V_TOTAL_MIN_EVENT_OCCURED_INT__SHIFT 0xa
+#define DISP_INTERRUPT_STATUS_CONTINUE2__DIGC_DP_FAST_TRAINING_COMPLETE_INTERRUPT_MASK 0x8000
+#define DISP_INTERRUPT_STATUS_CONTINUE2__DIGC_DP_FAST_TRAINING_COMPLETE_INTERRUPT__SHIFT 0xf
+#define DISP_INTERRUPT_STATUS_CONTINUE2__DIGC_DP_VID_STREAM_DISABLE_INTERRUPT_MASK 0x10000
+#define DISP_INTERRUPT_STATUS_CONTINUE2__DIGC_DP_VID_STREAM_DISABLE_INTERRUPT__SHIFT 0x10
+#define DISP_INTERRUPT_STATUS_CONTINUE2__DC_HPD3_INTERRUPT_MASK 0x20000
+#define DISP_INTERRUPT_STATUS_CONTINUE2__DC_HPD3_INTERRUPT__SHIFT 0x11
+#define DISP_INTERRUPT_STATUS_CONTINUE2__DC_HPD3_RX_INTERRUPT_MASK 0x40000
+#define DISP_INTERRUPT_STATUS_CONTINUE2__DC_HPD3_RX_INTERRUPT__SHIFT 0x12
+#define DISP_INTERRUPT_STATUS_CONTINUE2__AUX3_SW_DONE_INTERRUPT_MASK 0x80000
+#define DISP_INTERRUPT_STATUS_CONTINUE2__AUX3_SW_DONE_INTERRUPT__SHIFT 0x13
+#define DISP_INTERRUPT_STATUS_CONTINUE2__AUX3_LS_DONE_INTERRUPT_MASK 0x100000
+#define DISP_INTERRUPT_STATUS_CONTINUE2__AUX3_LS_DONE_INTERRUPT__SHIFT 0x14
+#define DISP_INTERRUPT_STATUS_CONTINUE2__LB_D4_VLINE2_INTERRUPT_MASK 0x200000
+#define DISP_INTERRUPT_STATUS_CONTINUE2__LB_D4_VLINE2_INTERRUPT__SHIFT 0x15
+#define DISP_INTERRUPT_STATUS_CONTINUE2__LB_D5_VLINE2_INTERRUPT_MASK 0x400000
+#define DISP_INTERRUPT_STATUS_CONTINUE2__LB_D5_VLINE2_INTERRUPT__SHIFT 0x16
+#define DISP_INTERRUPT_STATUS_CONTINUE2__LB_D6_VLINE2_INTERRUPT_MASK 0x800000
+#define DISP_INTERRUPT_STATUS_CONTINUE2__LB_D6_VLINE2_INTERRUPT__SHIFT 0x17
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC2_EXT_TIMING_SYNC_LOSS_INTERRUPT_MASK 0x2000000
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC2_EXT_TIMING_SYNC_LOSS_INTERRUPT__SHIFT 0x19
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC2_EXT_TIMING_SYNC_INTERRUPT_MASK 0x4000000
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC2_EXT_TIMING_SYNC_INTERRUPT__SHIFT 0x1a
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC2_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_MASK 0x8000000
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC2_EXT_TIMING_SYNC_SIGNAL_INTERRUPT__SHIFT 0x1b
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC2_VERTICAL_INTERRUPT0_MASK 0x10000000
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC2_VERTICAL_INTERRUPT0__SHIFT 0x1c
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC2_VERTICAL_INTERRUPT1_MASK 0x20000000
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC2_VERTICAL_INTERRUPT1__SHIFT 0x1d
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC2_VERTICAL_INTERRUPT2_MASK 0x40000000
+#define DISP_INTERRUPT_STATUS_CONTINUE2__CRTC2_VERTICAL_INTERRUPT2__SHIFT 0x1e
+#define DISP_INTERRUPT_STATUS_CONTINUE2__DISP_INTERRUPT_STATUS_CONTINUE3_MASK 0x80000000
+#define DISP_INTERRUPT_STATUS_CONTINUE2__DISP_INTERRUPT_STATUS_CONTINUE3__SHIFT 0x1f
+#define DISP_INTERRUPT_STATUS_CONTINUE3__SCL_DISP4_MODE_CHANGE_INTERRUPT_MASK 0x1
+#define DISP_INTERRUPT_STATUS_CONTINUE3__SCL_DISP4_MODE_CHANGE_INTERRUPT__SHIFT 0x0
+#define DISP_INTERRUPT_STATUS_CONTINUE3__D4BLND_DATA_UNDERFLOW_INTERRUPT_MASK 0x2
+#define DISP_INTERRUPT_STATUS_CONTINUE3__D4BLND_DATA_UNDERFLOW_INTERRUPT__SHIFT 0x1
+#define DISP_INTERRUPT_STATUS_CONTINUE3__LB_D4_VLINE_INTERRUPT_MASK 0x4
+#define DISP_INTERRUPT_STATUS_CONTINUE3__LB_D4_VLINE_INTERRUPT__SHIFT 0x2
+#define DISP_INTERRUPT_STATUS_CONTINUE3__LB_D4_VBLANK_INTERRUPT_MASK 0x8
+#define DISP_INTERRUPT_STATUS_CONTINUE3__LB_D4_VBLANK_INTERRUPT__SHIFT 0x3
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC4_SNAPSHOT_INTERRUPT_MASK 0x10
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC4_SNAPSHOT_INTERRUPT__SHIFT 0x4
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC4_FORCE_VSYNC_NEXT_LINE_INTERRUPT_MASK 0x20
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC4_FORCE_VSYNC_NEXT_LINE_INTERRUPT__SHIFT 0x5
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC4_FORCE_COUNT_NOW_INTERRUPT_MASK 0x40
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC4_FORCE_COUNT_NOW_INTERRUPT__SHIFT 0x6
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC4_TRIGA_INTERRUPT_MASK 0x80
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC4_TRIGA_INTERRUPT__SHIFT 0x7
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC4_TRIGB_INTERRUPT_MASK 0x100
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC4_TRIGB_INTERRUPT__SHIFT 0x8
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC4_VSYNC_NOM_INTERRUPT_MASK 0x200
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC4_VSYNC_NOM_INTERRUPT__SHIFT 0x9
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC4_SET_V_TOTAL_MIN_EVENT_OCCURED_INT_MASK 0x400
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC4_SET_V_TOTAL_MIN_EVENT_OCCURED_INT__SHIFT 0xa
+#define DISP_INTERRUPT_STATUS_CONTINUE3__DIGD_DP_FAST_TRAINING_COMPLETE_INTERRUPT_MASK 0x8000
+#define DISP_INTERRUPT_STATUS_CONTINUE3__DIGD_DP_FAST_TRAINING_COMPLETE_INTERRUPT__SHIFT 0xf
+#define DISP_INTERRUPT_STATUS_CONTINUE3__DIGD_DP_VID_STREAM_DISABLE_INTERRUPT_MASK 0x10000
+#define DISP_INTERRUPT_STATUS_CONTINUE3__DIGD_DP_VID_STREAM_DISABLE_INTERRUPT__SHIFT 0x10
+#define DISP_INTERRUPT_STATUS_CONTINUE3__DC_HPD4_INTERRUPT_MASK 0x20000
+#define DISP_INTERRUPT_STATUS_CONTINUE3__DC_HPD4_INTERRUPT__SHIFT 0x11
+#define DISP_INTERRUPT_STATUS_CONTINUE3__DC_HPD4_RX_INTERRUPT_MASK 0x40000
+#define DISP_INTERRUPT_STATUS_CONTINUE3__DC_HPD4_RX_INTERRUPT__SHIFT 0x12
+#define DISP_INTERRUPT_STATUS_CONTINUE3__AUX4_SW_DONE_INTERRUPT_MASK 0x80000
+#define DISP_INTERRUPT_STATUS_CONTINUE3__AUX4_SW_DONE_INTERRUPT__SHIFT 0x13
+#define DISP_INTERRUPT_STATUS_CONTINUE3__AUX4_LS_DONE_INTERRUPT_MASK 0x100000
+#define DISP_INTERRUPT_STATUS_CONTINUE3__AUX4_LS_DONE_INTERRUPT__SHIFT 0x14
+#define DISP_INTERRUPT_STATUS_CONTINUE3__BUFMGR_IHIF_INTERRUPT_MASK 0x200000
+#define DISP_INTERRUPT_STATUS_CONTINUE3__BUFMGR_IHIF_INTERRUPT__SHIFT 0x15
+#define DISP_INTERRUPT_STATUS_CONTINUE3__WBSCL_HOST_CONFLICT_INTERRUPT_MASK 0x400000
+#define DISP_INTERRUPT_STATUS_CONTINUE3__WBSCL_HOST_CONFLICT_INTERRUPT__SHIFT 0x16
+#define DISP_INTERRUPT_STATUS_CONTINUE3__WBSCL_DATA_OVERFLOW_INTERRUPT_MASK 0x800000
+#define DISP_INTERRUPT_STATUS_CONTINUE3__WBSCL_DATA_OVERFLOW_INTERRUPT__SHIFT 0x17
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC3_EXT_TIMING_SYNC_LOSS_INTERRUPT_MASK 0x2000000
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC3_EXT_TIMING_SYNC_LOSS_INTERRUPT__SHIFT 0x19
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC3_EXT_TIMING_SYNC_INTERRUPT_MASK 0x4000000
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC3_EXT_TIMING_SYNC_INTERRUPT__SHIFT 0x1a
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC3_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_MASK 0x8000000
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC3_EXT_TIMING_SYNC_SIGNAL_INTERRUPT__SHIFT 0x1b
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC3_VERTICAL_INTERRUPT0_MASK 0x10000000
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC3_VERTICAL_INTERRUPT0__SHIFT 0x1c
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC3_VERTICAL_INTERRUPT1_MASK 0x20000000
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC3_VERTICAL_INTERRUPT1__SHIFT 0x1d
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC3_VERTICAL_INTERRUPT2_MASK 0x40000000
+#define DISP_INTERRUPT_STATUS_CONTINUE3__CRTC3_VERTICAL_INTERRUPT2__SHIFT 0x1e
+#define DISP_INTERRUPT_STATUS_CONTINUE3__DISP_INTERRUPT_STATUS_CONTINUE4_MASK 0x80000000
+#define DISP_INTERRUPT_STATUS_CONTINUE3__DISP_INTERRUPT_STATUS_CONTINUE4__SHIFT 0x1f
+#define DISP_INTERRUPT_STATUS_CONTINUE4__SCL_DISP5_MODE_CHANGE_INTERRUPT_MASK 0x1
+#define DISP_INTERRUPT_STATUS_CONTINUE4__SCL_DISP5_MODE_CHANGE_INTERRUPT__SHIFT 0x0
+#define DISP_INTERRUPT_STATUS_CONTINUE4__D5BLND_DATA_UNDERFLOW_INTERRUPT_MASK 0x2
+#define DISP_INTERRUPT_STATUS_CONTINUE4__D5BLND_DATA_UNDERFLOW_INTERRUPT__SHIFT 0x1
+#define DISP_INTERRUPT_STATUS_CONTINUE4__LB_D5_VLINE_INTERRUPT_MASK 0x4
+#define DISP_INTERRUPT_STATUS_CONTINUE4__LB_D5_VLINE_INTERRUPT__SHIFT 0x2
+#define DISP_INTERRUPT_STATUS_CONTINUE4__LB_D5_VBLANK_INTERRUPT_MASK 0x8
+#define DISP_INTERRUPT_STATUS_CONTINUE4__LB_D5_VBLANK_INTERRUPT__SHIFT 0x3
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC5_SNAPSHOT_INTERRUPT_MASK 0x10
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC5_SNAPSHOT_INTERRUPT__SHIFT 0x4
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC5_FORCE_VSYNC_NEXT_LINE_INTERRUPT_MASK 0x20
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC5_FORCE_VSYNC_NEXT_LINE_INTERRUPT__SHIFT 0x5
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC5_FORCE_COUNT_NOW_INTERRUPT_MASK 0x40
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC5_FORCE_COUNT_NOW_INTERRUPT__SHIFT 0x6
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC5_TRIGA_INTERRUPT_MASK 0x80
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC5_TRIGA_INTERRUPT__SHIFT 0x7
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC5_TRIGB_INTERRUPT_MASK 0x100
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC5_TRIGB_INTERRUPT__SHIFT 0x8
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC5_VSYNC_NOM_INTERRUPT_MASK 0x200
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC5_VSYNC_NOM_INTERRUPT__SHIFT 0x9
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC5_SET_V_TOTAL_MIN_EVENT_OCCURED_INT_MASK 0x400
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC5_SET_V_TOTAL_MIN_EVENT_OCCURED_INT__SHIFT 0xa
+#define DISP_INTERRUPT_STATUS_CONTINUE4__DIGE_DP_FAST_TRAINING_COMPLETE_INTERRUPT_MASK 0x8000
+#define DISP_INTERRUPT_STATUS_CONTINUE4__DIGE_DP_FAST_TRAINING_COMPLETE_INTERRUPT__SHIFT 0xf
+#define DISP_INTERRUPT_STATUS_CONTINUE4__DIGE_DP_VID_STREAM_DISABLE_INTERRUPT_MASK 0x10000
+#define DISP_INTERRUPT_STATUS_CONTINUE4__DIGE_DP_VID_STREAM_DISABLE_INTERRUPT__SHIFT 0x10
+#define DISP_INTERRUPT_STATUS_CONTINUE4__DC_HPD5_INTERRUPT_MASK 0x20000
+#define DISP_INTERRUPT_STATUS_CONTINUE4__DC_HPD5_INTERRUPT__SHIFT 0x11
+#define DISP_INTERRUPT_STATUS_CONTINUE4__DC_HPD5_RX_INTERRUPT_MASK 0x40000
+#define DISP_INTERRUPT_STATUS_CONTINUE4__DC_HPD5_RX_INTERRUPT__SHIFT 0x12
+#define DISP_INTERRUPT_STATUS_CONTINUE4__AUX5_SW_DONE_INTERRUPT_MASK 0x80000
+#define DISP_INTERRUPT_STATUS_CONTINUE4__AUX5_SW_DONE_INTERRUPT__SHIFT 0x13
+#define DISP_INTERRUPT_STATUS_CONTINUE4__AUX5_LS_DONE_INTERRUPT_MASK 0x100000
+#define DISP_INTERRUPT_STATUS_CONTINUE4__AUX5_LS_DONE_INTERRUPT__SHIFT 0x14
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC4_EXT_TIMING_SYNC_LOSS_INTERRUPT_MASK 0x400000
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC4_EXT_TIMING_SYNC_LOSS_INTERRUPT__SHIFT 0x16
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC4_EXT_TIMING_SYNC_INTERRUPT_MASK 0x800000
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC4_EXT_TIMING_SYNC_INTERRUPT__SHIFT 0x17
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC4_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_MASK 0x1000000
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC4_EXT_TIMING_SYNC_SIGNAL_INTERRUPT__SHIFT 0x18
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC5_EXT_TIMING_SYNC_LOSS_INTERRUPT_MASK 0x2000000
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC5_EXT_TIMING_SYNC_LOSS_INTERRUPT__SHIFT 0x19
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC5_EXT_TIMING_SYNC_INTERRUPT_MASK 0x4000000
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC5_EXT_TIMING_SYNC_INTERRUPT__SHIFT 0x1a
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC5_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_MASK 0x8000000
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC5_EXT_TIMING_SYNC_SIGNAL_INTERRUPT__SHIFT 0x1b
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC4_VERTICAL_INTERRUPT0_MASK 0x10000000
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC4_VERTICAL_INTERRUPT0__SHIFT 0x1c
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC4_VERTICAL_INTERRUPT1_MASK 0x20000000
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC4_VERTICAL_INTERRUPT1__SHIFT 0x1d
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC4_VERTICAL_INTERRUPT2_MASK 0x40000000
+#define DISP_INTERRUPT_STATUS_CONTINUE4__CRTC4_VERTICAL_INTERRUPT2__SHIFT 0x1e
+#define DISP_INTERRUPT_STATUS_CONTINUE4__DISP_INTERRUPT_STATUS_CONTINUE5_MASK 0x80000000
+#define DISP_INTERRUPT_STATUS_CONTINUE4__DISP_INTERRUPT_STATUS_CONTINUE5__SHIFT 0x1f
+#define DISP_INTERRUPT_STATUS_CONTINUE5__SCL_DISP6_MODE_CHANGE_INTERRUPT_MASK 0x1
+#define DISP_INTERRUPT_STATUS_CONTINUE5__SCL_DISP6_MODE_CHANGE_INTERRUPT__SHIFT 0x0
+#define DISP_INTERRUPT_STATUS_CONTINUE5__D6BLND_DATA_UNDERFLOW_INTERRUPT_MASK 0x2
+#define DISP_INTERRUPT_STATUS_CONTINUE5__D6BLND_DATA_UNDERFLOW_INTERRUPT__SHIFT 0x1
+#define DISP_INTERRUPT_STATUS_CONTINUE5__LB_D6_VLINE_INTERRUPT_MASK 0x4
+#define DISP_INTERRUPT_STATUS_CONTINUE5__LB_D6_VLINE_INTERRUPT__SHIFT 0x2
+#define DISP_INTERRUPT_STATUS_CONTINUE5__LB_D6_VBLANK_INTERRUPT_MASK 0x8
+#define DISP_INTERRUPT_STATUS_CONTINUE5__LB_D6_VBLANK_INTERRUPT__SHIFT 0x3
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_SNAPSHOT_INTERRUPT_MASK 0x10
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_SNAPSHOT_INTERRUPT__SHIFT 0x4
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_FORCE_VSYNC_NEXT_LINE_INTERRUPT_MASK 0x20
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_FORCE_VSYNC_NEXT_LINE_INTERRUPT__SHIFT 0x5
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_FORCE_COUNT_NOW_INTERRUPT_MASK 0x40
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_FORCE_COUNT_NOW_INTERRUPT__SHIFT 0x6
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_TRIGA_INTERRUPT_MASK 0x80
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_TRIGA_INTERRUPT__SHIFT 0x7
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_TRIGB_INTERRUPT_MASK 0x100
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_TRIGB_INTERRUPT__SHIFT 0x8
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_VSYNC_NOM_INTERRUPT_MASK 0x200
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_VSYNC_NOM_INTERRUPT__SHIFT 0x9
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_SET_V_TOTAL_MIN_EVENT_OCCURED_INT_MASK 0x400
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_SET_V_TOTAL_MIN_EVENT_OCCURED_INT__SHIFT 0xa
+#define DISP_INTERRUPT_STATUS_CONTINUE5__DIGF_DP_FAST_TRAINING_COMPLETE_INTERRUPT_MASK 0x8000
+#define DISP_INTERRUPT_STATUS_CONTINUE5__DIGF_DP_FAST_TRAINING_COMPLETE_INTERRUPT__SHIFT 0xf
+#define DISP_INTERRUPT_STATUS_CONTINUE5__DIGF_DP_VID_STREAM_DISABLE_INTERRUPT_MASK 0x10000
+#define DISP_INTERRUPT_STATUS_CONTINUE5__DIGF_DP_VID_STREAM_DISABLE_INTERRUPT__SHIFT 0x10
+#define DISP_INTERRUPT_STATUS_CONTINUE5__DC_HPD6_INTERRUPT_MASK 0x20000
+#define DISP_INTERRUPT_STATUS_CONTINUE5__DC_HPD6_INTERRUPT__SHIFT 0x11
+#define DISP_INTERRUPT_STATUS_CONTINUE5__DC_HPD6_RX_INTERRUPT_MASK 0x40000
+#define DISP_INTERRUPT_STATUS_CONTINUE5__DC_HPD6_RX_INTERRUPT__SHIFT 0x12
+#define DISP_INTERRUPT_STATUS_CONTINUE5__AUX6_SW_DONE_INTERRUPT_MASK 0x80000
+#define DISP_INTERRUPT_STATUS_CONTINUE5__AUX6_SW_DONE_INTERRUPT__SHIFT 0x13
+#define DISP_INTERRUPT_STATUS_CONTINUE5__AUX6_LS_DONE_INTERRUPT_MASK 0x100000
+#define DISP_INTERRUPT_STATUS_CONTINUE5__AUX6_LS_DONE_INTERRUPT__SHIFT 0x14
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_EXT_TIMING_SYNC_LOSS_INTERRUPT_MASK 0x400000
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_EXT_TIMING_SYNC_LOSS_INTERRUPT__SHIFT 0x16
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_EXT_TIMING_SYNC_INTERRUPT_MASK 0x800000
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_EXT_TIMING_SYNC_INTERRUPT__SHIFT 0x17
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_EXT_TIMING_SYNC_SIGNAL_INTERRUPT_MASK 0x1000000
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_EXT_TIMING_SYNC_SIGNAL_INTERRUPT__SHIFT 0x18
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC5_VERTICAL_INTERRUPT0_MASK 0x2000000
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC5_VERTICAL_INTERRUPT0__SHIFT 0x19
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC5_VERTICAL_INTERRUPT1_MASK 0x4000000
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC5_VERTICAL_INTERRUPT1__SHIFT 0x1a
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC5_VERTICAL_INTERRUPT2_MASK 0x8000000
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC5_VERTICAL_INTERRUPT2__SHIFT 0x1b
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_VERTICAL_INTERRUPT0_MASK 0x10000000
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_VERTICAL_INTERRUPT0__SHIFT 0x1c
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_VERTICAL_INTERRUPT1_MASK 0x20000000
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_VERTICAL_INTERRUPT1__SHIFT 0x1d
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_VERTICAL_INTERRUPT2_MASK 0x40000000
+#define DISP_INTERRUPT_STATUS_CONTINUE5__CRTC6_VERTICAL_INTERRUPT2__SHIFT 0x1e
+#define DISP_INTERRUPT_STATUS_CONTINUE5__DISP_INTERRUPT_STATUS_CONTINUE6_MASK 0x80000000
+#define DISP_INTERRUPT_STATUS_CONTINUE5__DISP_INTERRUPT_STATUS_CONTINUE6__SHIFT 0x1f
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DCRX_PERFMON_COUNTER0_INTERRUPT_MASK 0x1
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DCRX_PERFMON_COUNTER0_INTERRUPT__SHIFT 0x0
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DCRX_PERFMON_COUNTER1_INTERRUPT_MASK 0x2
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DCRX_PERFMON_COUNTER1_INTERRUPT__SHIFT 0x1
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DCRX_PERFMON_COUNTER2_INTERRUPT_MASK 0x4
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DCRX_PERFMON_COUNTER2_INTERRUPT__SHIFT 0x2
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DCRX_PERFMON_COUNTER3_INTERRUPT_MASK 0x8
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DCRX_PERFMON_COUNTER3_INTERRUPT__SHIFT 0x3
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DCRX_PERFMON_COUNTER4_INTERRUPT_MASK 0x10
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DCRX_PERFMON_COUNTER4_INTERRUPT__SHIFT 0x4
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DCRX_PERFMON_COUNTER5_INTERRUPT_MASK 0x20
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DCRX_PERFMON_COUNTER5_INTERRUPT__SHIFT 0x5
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DCRX_PERFMON_COUNTER6_INTERRUPT_MASK 0x40
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DCRX_PERFMON_COUNTER6_INTERRUPT__SHIFT 0x6
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DCRX_PERFMON_COUNTER7_INTERRUPT_MASK 0x80
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DCRX_PERFMON_COUNTER7_INTERRUPT__SHIFT 0x7
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DCRX_PERFMON_COUNTER_OFF_INTERRUPT_MASK 0x100
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DCRX_PERFMON_COUNTER_OFF_INTERRUPT__SHIFT 0x8
+#define DISP_INTERRUPT_STATUS_CONTINUE6__BUFMGR_CWB0_IHIF_INTERRUPT_MASK 0x200
+#define DISP_INTERRUPT_STATUS_CONTINUE6__BUFMGR_CWB0_IHIF_INTERRUPT__SHIFT 0x9
+#define DISP_INTERRUPT_STATUS_CONTINUE6__BUFMGR_CWB1_IHIF_INTERRUPT_MASK 0x400
+#define DISP_INTERRUPT_STATUS_CONTINUE6__BUFMGR_CWB1_IHIF_INTERRUPT__SHIFT 0xa
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DIGG_DP_FAST_TRAINING_COMPLETE_INTERRUPT_MASK 0x8000
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DIGG_DP_FAST_TRAINING_COMPLETE_INTERRUPT__SHIFT 0xf
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DIGG_DP_VID_STREAM_DISABLE_INTERRUPT_MASK 0x10000
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DIGG_DP_VID_STREAM_DISABLE_INTERRUPT__SHIFT 0x10
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX1_GTC_SYNC_LOCK_DONE_INTERRUPT_MASK 0x20000
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX1_GTC_SYNC_LOCK_DONE_INTERRUPT__SHIFT 0x11
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX1_GTC_SYNC_ERROR_INTERRUPT_MASK 0x40000
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX1_GTC_SYNC_ERROR_INTERRUPT__SHIFT 0x12
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX2_GTC_SYNC_LOCK_DONE_INTERRUPT_MASK 0x80000
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX2_GTC_SYNC_LOCK_DONE_INTERRUPT__SHIFT 0x13
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX2_GTC_SYNC_ERROR_INTERRUPT_MASK 0x100000
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX2_GTC_SYNC_ERROR_INTERRUPT__SHIFT 0x14
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX3_GTC_SYNC_LOCK_DONE_INTERRUPT_MASK 0x200000
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX3_GTC_SYNC_LOCK_DONE_INTERRUPT__SHIFT 0x15
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX3_GTC_SYNC_ERROR_INTERRUPT_MASK 0x400000
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX3_GTC_SYNC_ERROR_INTERRUPT__SHIFT 0x16
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX4_GTC_SYNC_LOCK_DONE_INTERRUPT_MASK 0x800000
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX4_GTC_SYNC_LOCK_DONE_INTERRUPT__SHIFT 0x17
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX4_GTC_SYNC_ERROR_INTERRUPT_MASK 0x1000000
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX4_GTC_SYNC_ERROR_INTERRUPT__SHIFT 0x18
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX5_GTC_SYNC_LOCK_DONE_INTERRUPT_MASK 0x2000000
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX5_GTC_SYNC_LOCK_DONE_INTERRUPT__SHIFT 0x19
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX5_GTC_SYNC_ERROR_INTERRUPT_MASK 0x4000000
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX5_GTC_SYNC_ERROR_INTERRUPT__SHIFT 0x1a
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX6_GTC_SYNC_LOCK_DONE_INTERRUPT_MASK 0x8000000
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX6_GTC_SYNC_LOCK_DONE_INTERRUPT__SHIFT 0x1b
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX6_GTC_SYNC_ERROR_INTERRUPT_MASK 0x10000000
+#define DISP_INTERRUPT_STATUS_CONTINUE6__AUX6_GTC_SYNC_ERROR_INTERRUPT__SHIFT 0x1c
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DISP_INTERRUPT_STATUS_CONTINUE7_MASK 0x80000000
+#define DISP_INTERRUPT_STATUS_CONTINUE6__DISP_INTERRUPT_STATUS_CONTINUE7__SHIFT 0x1f
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCCG_PERFMON_COUNTER0_INTERRUPT_MASK 0x1
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCCG_PERFMON_COUNTER0_INTERRUPT__SHIFT 0x0
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCCG_PERFMON_COUNTER1_INTERRUPT_MASK 0x2
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCCG_PERFMON_COUNTER1_INTERRUPT__SHIFT 0x1
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCCG_PERFMON_COUNTER2_INTERRUPT_MASK 0x4
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCCG_PERFMON_COUNTER2_INTERRUPT__SHIFT 0x2
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCCG_PERFMON_COUNTER3_INTERRUPT_MASK 0x8
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCCG_PERFMON_COUNTER3_INTERRUPT__SHIFT 0x3
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCCG_PERFMON_COUNTER4_INTERRUPT_MASK 0x10
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCCG_PERFMON_COUNTER4_INTERRUPT__SHIFT 0x4
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCCG_PERFMON_COUNTER5_INTERRUPT_MASK 0x20
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCCG_PERFMON_COUNTER5_INTERRUPT__SHIFT 0x5
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCCG_PERFMON_COUNTER6_INTERRUPT_MASK 0x40
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCCG_PERFMON_COUNTER6_INTERRUPT__SHIFT 0x6
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCCG_PERFMON_COUNTER7_INTERRUPT_MASK 0x80
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCCG_PERFMON_COUNTER7_INTERRUPT__SHIFT 0x7
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCCG_PERFMON_COUNTER_OFF_INTERRUPT_MASK 0x100
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCCG_PERFMON_COUNTER_OFF_INTERRUPT__SHIFT 0x8
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCI_PERFMON_COUNTER0_INTERRUPT_MASK 0x200
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCI_PERFMON_COUNTER0_INTERRUPT__SHIFT 0x9
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCI_PERFMON_COUNTER1_INTERRUPT_MASK 0x400
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCI_PERFMON_COUNTER1_INTERRUPT__SHIFT 0xa
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCI_PERFMON_COUNTER2_INTERRUPT_MASK 0x800
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCI_PERFMON_COUNTER2_INTERRUPT__SHIFT 0xb
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCI_PERFMON_COUNTER3_INTERRUPT_MASK 0x1000
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCI_PERFMON_COUNTER3_INTERRUPT__SHIFT 0xc
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCI_PERFMON_COUNTER4_INTERRUPT_MASK 0x2000
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCI_PERFMON_COUNTER4_INTERRUPT__SHIFT 0xd
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCI_PERFMON_COUNTER5_INTERRUPT_MASK 0x4000
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCI_PERFMON_COUNTER5_INTERRUPT__SHIFT 0xe
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCI_PERFMON_COUNTER6_INTERRUPT_MASK 0x8000
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCI_PERFMON_COUNTER6_INTERRUPT__SHIFT 0xf
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCI_PERFMON_COUNTER7_INTERRUPT_MASK 0x10000
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCI_PERFMON_COUNTER7_INTERRUPT__SHIFT 0x10
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCI_PERFMON_COUNTER_OFF_INTERRUPT_MASK 0x20000
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCI_PERFMON_COUNTER_OFF_INTERRUPT__SHIFT 0x11
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCO_PERFMON_COUNTER0_INTERRUPT_MASK 0x40000
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCO_PERFMON_COUNTER0_INTERRUPT__SHIFT 0x12
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCO_PERFMON_COUNTER1_INTERRUPT_MASK 0x80000
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCO_PERFMON_COUNTER1_INTERRUPT__SHIFT 0x13
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCO_PERFMON_COUNTER2_INTERRUPT_MASK 0x100000
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCO_PERFMON_COUNTER2_INTERRUPT__SHIFT 0x14
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCO_PERFMON_COUNTER3_INTERRUPT_MASK 0x200000
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCO_PERFMON_COUNTER3_INTERRUPT__SHIFT 0x15
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCO_PERFMON_COUNTER4_INTERRUPT_MASK 0x400000
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCO_PERFMON_COUNTER4_INTERRUPT__SHIFT 0x16
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCO_PERFMON_COUNTER5_INTERRUPT_MASK 0x800000
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCO_PERFMON_COUNTER5_INTERRUPT__SHIFT 0x17
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCO_PERFMON_COUNTER6_INTERRUPT_MASK 0x1000000
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCO_PERFMON_COUNTER6_INTERRUPT__SHIFT 0x18
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCO_PERFMON_COUNTER7_INTERRUPT_MASK 0x2000000
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCO_PERFMON_COUNTER7_INTERRUPT__SHIFT 0x19
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCO_PERFMON_COUNTER_OFF_INTERRUPT_MASK 0x4000000
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DCO_PERFMON_COUNTER_OFF_INTERRUPT__SHIFT 0x1a
+#define DISP_INTERRUPT_STATUS_CONTINUE7__WB_PERFMON_COUNTER0_INTERRUPT_MASK 0x8000000
+#define DISP_INTERRUPT_STATUS_CONTINUE7__WB_PERFMON_COUNTER0_INTERRUPT__SHIFT 0x1b
+#define DISP_INTERRUPT_STATUS_CONTINUE7__WB_PERFMON_COUNTER1_INTERRUPT_MASK 0x10000000
+#define DISP_INTERRUPT_STATUS_CONTINUE7__WB_PERFMON_COUNTER1_INTERRUPT__SHIFT 0x1c
+#define DISP_INTERRUPT_STATUS_CONTINUE7__WB_PERFMON_COUNTER2_INTERRUPT_MASK 0x20000000
+#define DISP_INTERRUPT_STATUS_CONTINUE7__WB_PERFMON_COUNTER2_INTERRUPT__SHIFT 0x1d
+#define DISP_INTERRUPT_STATUS_CONTINUE7__WB_PERFMON_COUNTER3_INTERRUPT_MASK 0x40000000
+#define DISP_INTERRUPT_STATUS_CONTINUE7__WB_PERFMON_COUNTER3_INTERRUPT__SHIFT 0x1e
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DISP_INTERRUPT_STATUS_CONTINUE8_MASK 0x80000000
+#define DISP_INTERRUPT_STATUS_CONTINUE7__DISP_INTERRUPT_STATUS_CONTINUE8__SHIFT 0x1f
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE0_PERFMON_COUNTER0_INTERRUPT_MASK 0x1
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE0_PERFMON_COUNTER0_INTERRUPT__SHIFT 0x0
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE0_PERFMON_COUNTER1_INTERRUPT_MASK 0x2
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE0_PERFMON_COUNTER1_INTERRUPT__SHIFT 0x1
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE0_PERFMON_COUNTER2_INTERRUPT_MASK 0x4
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE0_PERFMON_COUNTER2_INTERRUPT__SHIFT 0x2
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE0_PERFMON_COUNTER3_INTERRUPT_MASK 0x8
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE0_PERFMON_COUNTER3_INTERRUPT__SHIFT 0x3
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE0_PERFMON_COUNTER4_INTERRUPT_MASK 0x10
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE0_PERFMON_COUNTER4_INTERRUPT__SHIFT 0x4
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE0_PERFMON_COUNTER5_INTERRUPT_MASK 0x20
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE0_PERFMON_COUNTER5_INTERRUPT__SHIFT 0x5
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE0_PERFMON_COUNTER6_INTERRUPT_MASK 0x40
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE0_PERFMON_COUNTER6_INTERRUPT__SHIFT 0x6
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE0_PERFMON_COUNTER7_INTERRUPT_MASK 0x80
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE0_PERFMON_COUNTER7_INTERRUPT__SHIFT 0x7
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE0_PERFMON_COUNTER_OFF_INTERRUPT_MASK 0x100
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE0_PERFMON_COUNTER_OFF_INTERRUPT__SHIFT 0x8
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE1_PERFMON_COUNTER0_INTERRUPT_MASK 0x200
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE1_PERFMON_COUNTER0_INTERRUPT__SHIFT 0x9
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE1_PERFMON_COUNTER1_INTERRUPT_MASK 0x400
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE1_PERFMON_COUNTER1_INTERRUPT__SHIFT 0xa
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE1_PERFMON_COUNTER2_INTERRUPT_MASK 0x800
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE1_PERFMON_COUNTER2_INTERRUPT__SHIFT 0xb
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE1_PERFMON_COUNTER3_INTERRUPT_MASK 0x1000
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE1_PERFMON_COUNTER3_INTERRUPT__SHIFT 0xc
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE1_PERFMON_COUNTER4_INTERRUPT_MASK 0x2000
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE1_PERFMON_COUNTER4_INTERRUPT__SHIFT 0xd
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE1_PERFMON_COUNTER5_INTERRUPT_MASK 0x4000
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE1_PERFMON_COUNTER5_INTERRUPT__SHIFT 0xe
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE1_PERFMON_COUNTER6_INTERRUPT_MASK 0x8000
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE1_PERFMON_COUNTER6_INTERRUPT__SHIFT 0xf
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE1_PERFMON_COUNTER7_INTERRUPT_MASK 0x10000
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE1_PERFMON_COUNTER7_INTERRUPT__SHIFT 0x10
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE1_PERFMON_COUNTER_OFF_INTERRUPT_MASK 0x20000
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE1_PERFMON_COUNTER_OFF_INTERRUPT__SHIFT 0x11
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE2_PERFMON_COUNTER0_INTERRUPT_MASK 0x40000
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE2_PERFMON_COUNTER0_INTERRUPT__SHIFT 0x12
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE2_PERFMON_COUNTER1_INTERRUPT_MASK 0x80000
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE2_PERFMON_COUNTER1_INTERRUPT__SHIFT 0x13
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE2_PERFMON_COUNTER2_INTERRUPT_MASK 0x100000
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE2_PERFMON_COUNTER2_INTERRUPT__SHIFT 0x14
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE2_PERFMON_COUNTER3_INTERRUPT_MASK 0x200000
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE2_PERFMON_COUNTER3_INTERRUPT__SHIFT 0x15
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE2_PERFMON_COUNTER4_INTERRUPT_MASK 0x400000
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE2_PERFMON_COUNTER4_INTERRUPT__SHIFT 0x16
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE2_PERFMON_COUNTER5_INTERRUPT_MASK 0x800000
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE2_PERFMON_COUNTER5_INTERRUPT__SHIFT 0x17
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE2_PERFMON_COUNTER6_INTERRUPT_MASK 0x1000000
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE2_PERFMON_COUNTER6_INTERRUPT__SHIFT 0x18
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE2_PERFMON_COUNTER7_INTERRUPT_MASK 0x2000000
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE2_PERFMON_COUNTER7_INTERRUPT__SHIFT 0x19
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE2_PERFMON_COUNTER_OFF_INTERRUPT_MASK 0x4000000
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DCFE2_PERFMON_COUNTER_OFF_INTERRUPT__SHIFT 0x1a
+#define DISP_INTERRUPT_STATUS_CONTINUE8__WB_PERFMON_COUNTER4_INTERRUPT_MASK 0x8000000
+#define DISP_INTERRUPT_STATUS_CONTINUE8__WB_PERFMON_COUNTER4_INTERRUPT__SHIFT 0x1b
+#define DISP_INTERRUPT_STATUS_CONTINUE8__WB_PERFMON_COUNTER5_INTERRUPT_MASK 0x10000000
+#define DISP_INTERRUPT_STATUS_CONTINUE8__WB_PERFMON_COUNTER5_INTERRUPT__SHIFT 0x1c
+#define DISP_INTERRUPT_STATUS_CONTINUE8__WB_PERFMON_COUNTER6_INTERRUPT_MASK 0x20000000
+#define DISP_INTERRUPT_STATUS_CONTINUE8__WB_PERFMON_COUNTER6_INTERRUPT__SHIFT 0x1d
+#define DISP_INTERRUPT_STATUS_CONTINUE8__WB_PERFMON_COUNTER7_INTERRUPT_MASK 0x40000000
+#define DISP_INTERRUPT_STATUS_CONTINUE8__WB_PERFMON_COUNTER7_INTERRUPT__SHIFT 0x1e
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DISP_INTERRUPT_STATUS_CONTINUE9_MASK 0x80000000
+#define DISP_INTERRUPT_STATUS_CONTINUE8__DISP_INTERRUPT_STATUS_CONTINUE9__SHIFT 0x1f
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE3_PERFMON_COUNTER0_INTERRUPT_MASK 0x1
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE3_PERFMON_COUNTER0_INTERRUPT__SHIFT 0x0
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE3_PERFMON_COUNTER1_INTERRUPT_MASK 0x2
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE3_PERFMON_COUNTER1_INTERRUPT__SHIFT 0x1
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE3_PERFMON_COUNTER2_INTERRUPT_MASK 0x4
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE3_PERFMON_COUNTER2_INTERRUPT__SHIFT 0x2
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE3_PERFMON_COUNTER3_INTERRUPT_MASK 0x8
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE3_PERFMON_COUNTER3_INTERRUPT__SHIFT 0x3
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE3_PERFMON_COUNTER4_INTERRUPT_MASK 0x10
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE3_PERFMON_COUNTER4_INTERRUPT__SHIFT 0x4
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE3_PERFMON_COUNTER5_INTERRUPT_MASK 0x20
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE3_PERFMON_COUNTER5_INTERRUPT__SHIFT 0x5
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE3_PERFMON_COUNTER6_INTERRUPT_MASK 0x40
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE3_PERFMON_COUNTER6_INTERRUPT__SHIFT 0x6
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE3_PERFMON_COUNTER7_INTERRUPT_MASK 0x80
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE3_PERFMON_COUNTER7_INTERRUPT__SHIFT 0x7
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE3_PERFMON_COUNTER_OFF_INTERRUPT_MASK 0x100
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE3_PERFMON_COUNTER_OFF_INTERRUPT__SHIFT 0x8
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE4_PERFMON_COUNTER0_INTERRUPT_MASK 0x200
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE4_PERFMON_COUNTER0_INTERRUPT__SHIFT 0x9
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE4_PERFMON_COUNTER1_INTERRUPT_MASK 0x400
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE4_PERFMON_COUNTER1_INTERRUPT__SHIFT 0xa
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE4_PERFMON_COUNTER2_INTERRUPT_MASK 0x800
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE4_PERFMON_COUNTER2_INTERRUPT__SHIFT 0xb
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE4_PERFMON_COUNTER3_INTERRUPT_MASK 0x1000
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE4_PERFMON_COUNTER3_INTERRUPT__SHIFT 0xc
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE4_PERFMON_COUNTER4_INTERRUPT_MASK 0x2000
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE4_PERFMON_COUNTER4_INTERRUPT__SHIFT 0xd
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE4_PERFMON_COUNTER5_INTERRUPT_MASK 0x4000
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE4_PERFMON_COUNTER5_INTERRUPT__SHIFT 0xe
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE4_PERFMON_COUNTER6_INTERRUPT_MASK 0x8000
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE4_PERFMON_COUNTER6_INTERRUPT__SHIFT 0xf
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE4_PERFMON_COUNTER7_INTERRUPT_MASK 0x10000
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE4_PERFMON_COUNTER7_INTERRUPT__SHIFT 0x10
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE4_PERFMON_COUNTER_OFF_INTERRUPT_MASK 0x20000
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE4_PERFMON_COUNTER_OFF_INTERRUPT__SHIFT 0x11
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE5_PERFMON_COUNTER0_INTERRUPT_MASK 0x40000
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE5_PERFMON_COUNTER0_INTERRUPT__SHIFT 0x12
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE5_PERFMON_COUNTER1_INTERRUPT_MASK 0x80000
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE5_PERFMON_COUNTER1_INTERRUPT__SHIFT 0x13
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE5_PERFMON_COUNTER2_INTERRUPT_MASK 0x100000
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE5_PERFMON_COUNTER2_INTERRUPT__SHIFT 0x14
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE5_PERFMON_COUNTER3_INTERRUPT_MASK 0x200000
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE5_PERFMON_COUNTER3_INTERRUPT__SHIFT 0x15
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE5_PERFMON_COUNTER4_INTERRUPT_MASK 0x400000
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE5_PERFMON_COUNTER4_INTERRUPT__SHIFT 0x16
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE5_PERFMON_COUNTER5_INTERRUPT_MASK 0x800000
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE5_PERFMON_COUNTER5_INTERRUPT__SHIFT 0x17
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE5_PERFMON_COUNTER6_INTERRUPT_MASK 0x1000000
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE5_PERFMON_COUNTER6_INTERRUPT__SHIFT 0x18
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE5_PERFMON_COUNTER7_INTERRUPT_MASK 0x2000000
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE5_PERFMON_COUNTER7_INTERRUPT__SHIFT 0x19
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE5_PERFMON_COUNTER_OFF_INTERRUPT_MASK 0x4000000
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DCFE5_PERFMON_COUNTER_OFF_INTERRUPT__SHIFT 0x1a
+#define DISP_INTERRUPT_STATUS_CONTINUE9__WB_PERFMON_COUNTER_OFF_INTERRUPT_MASK 0x8000000
+#define DISP_INTERRUPT_STATUS_CONTINUE9__WB_PERFMON_COUNTER_OFF_INTERRUPT__SHIFT 0x1b
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DISP_INTERRUPT_STATUS_CONTINUE10_MASK 0x80000000
+#define DISP_INTERRUPT_STATUS_CONTINUE9__DISP_INTERRUPT_STATUS_CONTINUE10__SHIFT 0x1f
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DIGLPA_DP_FAST_TRAINING_COMPLETE_INTERRUPT_MASK 0x10
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DIGLPA_DP_FAST_TRAINING_COMPLETE_INTERRUPT__SHIFT 0x4
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DIGLPA_DP_VID_STREAM_DISABLE_INTERRUPT_MASK 0x20
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DIGLPA_DP_VID_STREAM_DISABLE_INTERRUPT__SHIFT 0x5
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DIGLPB_DP_FAST_TRAINING_COMPLETE_INTERRUPT_MASK 0x400
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DIGLPB_DP_FAST_TRAINING_COMPLETE_INTERRUPT__SHIFT 0xa
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DIGLPB_DP_VID_STREAM_DISABLE_INTERRUPT_MASK 0x800
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DIGLPB_DP_VID_STREAM_DISABLE_INTERRUPT__SHIFT 0xb
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DCCG_PERFMON2_COUNTER0_INTERRUPT_MASK 0x1000
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DCCG_PERFMON2_COUNTER0_INTERRUPT__SHIFT 0xc
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DCCG_PERFMON2_COUNTER1_INTERRUPT_MASK 0x2000
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DCCG_PERFMON2_COUNTER1_INTERRUPT__SHIFT 0xd
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DCCG_PERFMON2_COUNTER2_INTERRUPT_MASK 0x4000
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DCCG_PERFMON2_COUNTER2_INTERRUPT__SHIFT 0xe
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DCCG_PERFMON2_COUNTER3_INTERRUPT_MASK 0x8000
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DCCG_PERFMON2_COUNTER3_INTERRUPT__SHIFT 0xf
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DCCG_PERFMON2_COUNTER4_INTERRUPT_MASK 0x10000
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DCCG_PERFMON2_COUNTER4_INTERRUPT__SHIFT 0x10
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DCCG_PERFMON2_COUNTER5_INTERRUPT_MASK 0x20000
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DCCG_PERFMON2_COUNTER5_INTERRUPT__SHIFT 0x11
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DCCG_PERFMON2_COUNTER6_INTERRUPT_MASK 0x40000
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DCCG_PERFMON2_COUNTER6_INTERRUPT__SHIFT 0x12
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DCCG_PERFMON2_COUNTER7_INTERRUPT_MASK 0x80000
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DCCG_PERFMON2_COUNTER7_INTERRUPT__SHIFT 0x13
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DCCG_PERFMON2_COUNTER_OFF_INTERRUPT_MASK 0x100000
+#define DISP_INTERRUPT_STATUS_CONTINUE10__DCCG_PERFMON2_COUNTER_OFF_INTERRUPT__SHIFT 0x14
+#define DCO_MEM_PWR_STATUS__I2C_MEM_PWR_STATE_MASK 0x1
+#define DCO_MEM_PWR_STATUS__I2C_MEM_PWR_STATE__SHIFT 0x0
+#define DCO_MEM_PWR_STATUS__MVP_MEM_PWR_STATE_MASK 0x4
+#define DCO_MEM_PWR_STATUS__MVP_MEM_PWR_STATE__SHIFT 0x2
+#define DCO_MEM_PWR_STATUS__DPA_MEM_PWR_STATE_MASK 0x8
+#define DCO_MEM_PWR_STATUS__DPA_MEM_PWR_STATE__SHIFT 0x3
+#define DCO_MEM_PWR_STATUS__DPB_MEM_PWR_STATE_MASK 0x10
+#define DCO_MEM_PWR_STATUS__DPB_MEM_PWR_STATE__SHIFT 0x4
+#define DCO_MEM_PWR_STATUS__DPC_MEM_PWR_STATE_MASK 0x20
+#define DCO_MEM_PWR_STATUS__DPC_MEM_PWR_STATE__SHIFT 0x5
+#define DCO_MEM_PWR_STATUS__DPD_MEM_PWR_STATE_MASK 0x40
+#define DCO_MEM_PWR_STATUS__DPD_MEM_PWR_STATE__SHIFT 0x6
+#define DCO_MEM_PWR_STATUS__DPE_MEM_PWR_STATE_MASK 0x80
+#define DCO_MEM_PWR_STATUS__DPE_MEM_PWR_STATE__SHIFT 0x7
+#define DCO_MEM_PWR_STATUS__DPF_MEM_PWR_STATE_MASK 0x100
+#define DCO_MEM_PWR_STATUS__DPF_MEM_PWR_STATE__SHIFT 0x8
+#define DCO_MEM_PWR_STATUS__DPG_MEM_PWR_STATE_MASK 0x200
+#define DCO_MEM_PWR_STATUS__DPG_MEM_PWR_STATE__SHIFT 0x9
+#define DCO_MEM_PWR_STATUS__HDMI0_MEM_PWR_STATE_MASK 0xc00
+#define DCO_MEM_PWR_STATUS__HDMI0_MEM_PWR_STATE__SHIFT 0xa
+#define DCO_MEM_PWR_STATUS__HDMI1_MEM_PWR_STATE_MASK 0x3000
+#define DCO_MEM_PWR_STATUS__HDMI1_MEM_PWR_STATE__SHIFT 0xc
+#define DCO_MEM_PWR_STATUS__HDMI2_MEM_PWR_STATE_MASK 0xc000
+#define DCO_MEM_PWR_STATUS__HDMI2_MEM_PWR_STATE__SHIFT 0xe
+#define DCO_MEM_PWR_STATUS__HDMI3_MEM_PWR_STATE_MASK 0x30000
+#define DCO_MEM_PWR_STATUS__HDMI3_MEM_PWR_STATE__SHIFT 0x10
+#define DCO_MEM_PWR_STATUS__HDMI4_MEM_PWR_STATE_MASK 0xc0000
+#define DCO_MEM_PWR_STATUS__HDMI4_MEM_PWR_STATE__SHIFT 0x12
+#define DCO_MEM_PWR_STATUS__HDMI5_MEM_PWR_STATE_MASK 0x300000
+#define DCO_MEM_PWR_STATUS__HDMI5_MEM_PWR_STATE__SHIFT 0x14
+#define DCO_MEM_PWR_STATUS__HDMI6_MEM_PWR_STATE_MASK 0xc00000
+#define DCO_MEM_PWR_STATUS__HDMI6_MEM_PWR_STATE__SHIFT 0x16
+#define DCO_MEM_PWR_STATUS1__DPLPA_MEM_PWR_STATE_MASK 0x1
+#define DCO_MEM_PWR_STATUS1__DPLPA_MEM_PWR_STATE__SHIFT 0x0
+#define DCO_MEM_PWR_STATUS1__DPLPB_MEM_PWR_STATE_MASK 0x2
+#define DCO_MEM_PWR_STATUS1__DPLPB_MEM_PWR_STATE__SHIFT 0x1
+#define DCO_MEM_PWR_STATUS1__HDMILP0_MEM_PWR_STATE_MASK 0xc00
+#define DCO_MEM_PWR_STATUS1__HDMILP0_MEM_PWR_STATE__SHIFT 0xa
+#define DCO_MEM_PWR_STATUS1__HDMILP1_MEM_PWR_STATE_MASK 0x3000
+#define DCO_MEM_PWR_STATUS1__HDMILP1_MEM_PWR_STATE__SHIFT 0xc
+#define DCO_MEM_PWR_CTRL__I2C_LIGHT_SLEEP_FORCE_MASK 0x1
+#define DCO_MEM_PWR_CTRL__I2C_LIGHT_SLEEP_FORCE__SHIFT 0x0
+#define DCO_MEM_PWR_CTRL__I2C_LIGHT_SLEEP_DIS_MASK 0x2
+#define DCO_MEM_PWR_CTRL__I2C_LIGHT_SLEEP_DIS__SHIFT 0x1
+#define DCO_MEM_PWR_CTRL__MVP_LIGHT_SLEEP_DIS_MASK 0x8
+#define DCO_MEM_PWR_CTRL__MVP_LIGHT_SLEEP_DIS__SHIFT 0x3
+#define DCO_MEM_PWR_CTRL__DPA_LIGHT_SLEEP_DIS_MASK 0x10
+#define DCO_MEM_PWR_CTRL__DPA_LIGHT_SLEEP_DIS__SHIFT 0x4
+#define DCO_MEM_PWR_CTRL__DPB_LIGHT_SLEEP_DIS_MASK 0x20
+#define DCO_MEM_PWR_CTRL__DPB_LIGHT_SLEEP_DIS__SHIFT 0x5
+#define DCO_MEM_PWR_CTRL__DPC_LIGHT_SLEEP_DIS_MASK 0x40
+#define DCO_MEM_PWR_CTRL__DPC_LIGHT_SLEEP_DIS__SHIFT 0x6
+#define DCO_MEM_PWR_CTRL__DPD_LIGHT_SLEEP_DIS_MASK 0x80
+#define DCO_MEM_PWR_CTRL__DPD_LIGHT_SLEEP_DIS__SHIFT 0x7
+#define DCO_MEM_PWR_CTRL__DPE_LIGHT_SLEEP_DIS_MASK 0x100
+#define DCO_MEM_PWR_CTRL__DPE_LIGHT_SLEEP_DIS__SHIFT 0x8
+#define DCO_MEM_PWR_CTRL__DPF_LIGHT_SLEEP_DIS_MASK 0x200
+#define DCO_MEM_PWR_CTRL__DPF_LIGHT_SLEEP_DIS__SHIFT 0x9
+#define DCO_MEM_PWR_CTRL__DPG_LIGHT_SLEEP_DIS_MASK 0x400
+#define DCO_MEM_PWR_CTRL__DPG_LIGHT_SLEEP_DIS__SHIFT 0xa
+#define DCO_MEM_PWR_CTRL__HDMI0_MEM_PWR_FORCE_MASK 0x1800
+#define DCO_MEM_PWR_CTRL__HDMI0_MEM_PWR_FORCE__SHIFT 0xb
+#define DCO_MEM_PWR_CTRL__HDMI0_MEM_PWR_DIS_MASK 0x2000
+#define DCO_MEM_PWR_CTRL__HDMI0_MEM_PWR_DIS__SHIFT 0xd
+#define DCO_MEM_PWR_CTRL__HDMI1_MEM_PWR_FORCE_MASK 0xc000
+#define DCO_MEM_PWR_CTRL__HDMI1_MEM_PWR_FORCE__SHIFT 0xe
+#define DCO_MEM_PWR_CTRL__HDMI1_MEM_PWR_DIS_MASK 0x10000
+#define DCO_MEM_PWR_CTRL__HDMI1_MEM_PWR_DIS__SHIFT 0x10
+#define DCO_MEM_PWR_CTRL__HDMI2_MEM_PWR_FORCE_MASK 0x60000
+#define DCO_MEM_PWR_CTRL__HDMI2_MEM_PWR_FORCE__SHIFT 0x11
+#define DCO_MEM_PWR_CTRL__HDMI2_MEM_PWR_DIS_MASK 0x80000
+#define DCO_MEM_PWR_CTRL__HDMI2_MEM_PWR_DIS__SHIFT 0x13
+#define DCO_MEM_PWR_CTRL__HDMI3_MEM_PWR_FORCE_MASK 0x300000
+#define DCO_MEM_PWR_CTRL__HDMI3_MEM_PWR_FORCE__SHIFT 0x14
+#define DCO_MEM_PWR_CTRL__HDMI3_MEM_PWR_DIS_MASK 0x400000
+#define DCO_MEM_PWR_CTRL__HDMI3_MEM_PWR_DIS__SHIFT 0x16
+#define DCO_MEM_PWR_CTRL__HDMI4_MEM_PWR_FORCE_MASK 0x1800000
+#define DCO_MEM_PWR_CTRL__HDMI4_MEM_PWR_FORCE__SHIFT 0x17
+#define DCO_MEM_PWR_CTRL__HDMI4_MEM_PWR_DIS_MASK 0x2000000
+#define DCO_MEM_PWR_CTRL__HDMI4_MEM_PWR_DIS__SHIFT 0x19
+#define DCO_MEM_PWR_CTRL__HDMI5_MEM_PWR_FORCE_MASK 0xc000000
+#define DCO_MEM_PWR_CTRL__HDMI5_MEM_PWR_FORCE__SHIFT 0x1a
+#define DCO_MEM_PWR_CTRL__HDMI5_MEM_PWR_DIS_MASK 0x10000000
+#define DCO_MEM_PWR_CTRL__HDMI5_MEM_PWR_DIS__SHIFT 0x1c
+#define DCO_MEM_PWR_CTRL__HDMI6_MEM_PWR_FORCE_MASK 0x60000000
+#define DCO_MEM_PWR_CTRL__HDMI6_MEM_PWR_FORCE__SHIFT 0x1d
+#define DCO_MEM_PWR_CTRL__HDMI6_MEM_PWR_DIS_MASK 0x80000000
+#define DCO_MEM_PWR_CTRL__HDMI6_MEM_PWR_DIS__SHIFT 0x1f
+#define DCO_MEM_PWR_CTRL2__HDMI_MEM_PWR_MODE_SEL_MASK 0x3
+#define DCO_MEM_PWR_CTRL2__HDMI_MEM_PWR_MODE_SEL__SHIFT 0x0
+#define DCO_MEM_PWR_CTRL2__DPLPA_LIGHT_SLEEP_DIS_MASK 0x4
+#define DCO_MEM_PWR_CTRL2__DPLPA_LIGHT_SLEEP_DIS__SHIFT 0x2
+#define DCO_MEM_PWR_CTRL2__DPLPB_LIGHT_SLEEP_DIS_MASK 0x8
+#define DCO_MEM_PWR_CTRL2__DPLPB_LIGHT_SLEEP_DIS__SHIFT 0x3
+#define DCO_MEM_PWR_CTRL2__HDMILP0_MEM_PWR_FORCE_MASK 0x30000
+#define DCO_MEM_PWR_CTRL2__HDMILP0_MEM_PWR_FORCE__SHIFT 0x10
+#define DCO_MEM_PWR_CTRL2__HDMILP0_MEM_PWR_DIS_MASK 0x40000
+#define DCO_MEM_PWR_CTRL2__HDMILP0_MEM_PWR_DIS__SHIFT 0x12
+#define DCO_MEM_PWR_CTRL2__HDMILP1_MEM_PWR_FORCE_MASK 0x180000
+#define DCO_MEM_PWR_CTRL2__HDMILP1_MEM_PWR_FORCE__SHIFT 0x13
+#define DCO_MEM_PWR_CTRL2__HDMILP1_MEM_PWR_DIS_MASK 0x200000
+#define DCO_MEM_PWR_CTRL2__HDMILP1_MEM_PWR_DIS__SHIFT 0x15
+#define FMT_MEMORY0_CONTROL__FMT420_MEM0_SOURCE_SEL_MASK 0x7
+#define FMT_MEMORY0_CONTROL__FMT420_MEM0_SOURCE_SEL__SHIFT 0x0
+#define FMT_MEMORY0_CONTROL__FMT420_MEM0_PWR_FORCE_MASK 0x30
+#define FMT_MEMORY0_CONTROL__FMT420_MEM0_PWR_FORCE__SHIFT 0x4
+#define FMT_MEMORY0_CONTROL__FMT420_MEM0_PWR_DIS_MASK 0x100
+#define FMT_MEMORY0_CONTROL__FMT420_MEM0_PWR_DIS__SHIFT 0x8
+#define FMT_MEMORY0_CONTROL__FMT420_MEM0_PWR_STATE_MASK 0x3000
+#define FMT_MEMORY0_CONTROL__FMT420_MEM0_PWR_STATE__SHIFT 0xc
+#define FMT_MEMORY1_CONTROL__FMT420_MEM1_SOURCE_SEL_MASK 0x7
+#define FMT_MEMORY1_CONTROL__FMT420_MEM1_SOURCE_SEL__SHIFT 0x0
+#define FMT_MEMORY1_CONTROL__FMT420_MEM1_PWR_FORCE_MASK 0x30
+#define FMT_MEMORY1_CONTROL__FMT420_MEM1_PWR_FORCE__SHIFT 0x4
+#define FMT_MEMORY1_CONTROL__FMT420_MEM1_PWR_DIS_MASK 0x100
+#define FMT_MEMORY1_CONTROL__FMT420_MEM1_PWR_DIS__SHIFT 0x8
+#define FMT_MEMORY1_CONTROL__FMT420_MEM1_PWR_STATE_MASK 0x3000
+#define FMT_MEMORY1_CONTROL__FMT420_MEM1_PWR_STATE__SHIFT 0xc
+#define FMT_MEMORY2_CONTROL__FMT420_MEM2_SOURCE_SEL_MASK 0x7
+#define FMT_MEMORY2_CONTROL__FMT420_MEM2_SOURCE_SEL__SHIFT 0x0
+#define FMT_MEMORY2_CONTROL__FMT420_MEM2_PWR_FORCE_MASK 0x30
+#define FMT_MEMORY2_CONTROL__FMT420_MEM2_PWR_FORCE__SHIFT 0x4
+#define FMT_MEMORY2_CONTROL__FMT420_MEM2_PWR_DIS_MASK 0x100
+#define FMT_MEMORY2_CONTROL__FMT420_MEM2_PWR_DIS__SHIFT 0x8
+#define FMT_MEMORY2_CONTROL__FMT420_MEM2_PWR_STATE_MASK 0x3000
+#define FMT_MEMORY2_CONTROL__FMT420_MEM2_PWR_STATE__SHIFT 0xc
+#define FMT_MEMORY3_CONTROL__FMT420_MEM3_SOURCE_SEL_MASK 0x7
+#define FMT_MEMORY3_CONTROL__FMT420_MEM3_SOURCE_SEL__SHIFT 0x0
+#define FMT_MEMORY3_CONTROL__FMT420_MEM3_PWR_FORCE_MASK 0x30
+#define FMT_MEMORY3_CONTROL__FMT420_MEM3_PWR_FORCE__SHIFT 0x4
+#define FMT_MEMORY3_CONTROL__FMT420_MEM3_PWR_DIS_MASK 0x100
+#define FMT_MEMORY3_CONTROL__FMT420_MEM3_PWR_DIS__SHIFT 0x8
+#define FMT_MEMORY3_CONTROL__FMT420_MEM3_PWR_STATE_MASK 0x3000
+#define FMT_MEMORY3_CONTROL__FMT420_MEM3_PWR_STATE__SHIFT 0xc
+#define FMT_MEMORY4_CONTROL__FMT420_MEM4_SOURCE_SEL_MASK 0x7
+#define FMT_MEMORY4_CONTROL__FMT420_MEM4_SOURCE_SEL__SHIFT 0x0
+#define FMT_MEMORY4_CONTROL__FMT420_MEM4_PWR_FORCE_MASK 0x30
+#define FMT_MEMORY4_CONTROL__FMT420_MEM4_PWR_FORCE__SHIFT 0x4
+#define FMT_MEMORY4_CONTROL__FMT420_MEM4_PWR_DIS_MASK 0x100
+#define FMT_MEMORY4_CONTROL__FMT420_MEM4_PWR_DIS__SHIFT 0x8
+#define FMT_MEMORY4_CONTROL__FMT420_MEM4_PWR_STATE_MASK 0x3000
+#define FMT_MEMORY4_CONTROL__FMT420_MEM4_PWR_STATE__SHIFT 0xc
+#define FMT_MEMORY5_CONTROL__FMT420_MEM5_SOURCE_SEL_MASK 0x7
+#define FMT_MEMORY5_CONTROL__FMT420_MEM5_SOURCE_SEL__SHIFT 0x0
+#define FMT_MEMORY5_CONTROL__FMT420_MEM5_PWR_FORCE_MASK 0x30
+#define FMT_MEMORY5_CONTROL__FMT420_MEM5_PWR_FORCE__SHIFT 0x4
+#define FMT_MEMORY5_CONTROL__FMT420_MEM5_PWR_DIS_MASK 0x100
+#define FMT_MEMORY5_CONTROL__FMT420_MEM5_PWR_DIS__SHIFT 0x8
+#define FMT_MEMORY5_CONTROL__FMT420_MEM5_PWR_STATE_MASK 0x3000
+#define FMT_MEMORY5_CONTROL__FMT420_MEM5_PWR_STATE__SHIFT 0xc
+#define DCO_CLK_CNTL__DISPCLK_R_DCO_GATE_DIS_MASK 0x20
+#define DCO_CLK_CNTL__DISPCLK_R_DCO_GATE_DIS__SHIFT 0x5
+#define DCO_CLK_CNTL__DISPCLK_G_ABM_GATE_DIS_MASK 0x40
+#define DCO_CLK_CNTL__DISPCLK_G_ABM_GATE_DIS__SHIFT 0x6
+#define DCO_CLK_CNTL__DISPCLK_G_DVO_GATE_DIS_MASK 0x80
+#define DCO_CLK_CNTL__DISPCLK_G_DVO_GATE_DIS__SHIFT 0x7
+#define DCO_CLK_CNTL__DISPCLK_G_DACA_GATE_DIS_MASK 0x100
+#define DCO_CLK_CNTL__DISPCLK_G_DACA_GATE_DIS__SHIFT 0x8
+#define DCO_CLK_CNTL__DISPCLK_G_DACB_GATE_DIS_MASK 0x200
+#define DCO_CLK_CNTL__DISPCLK_G_DACB_GATE_DIS__SHIFT 0x9
+#define DCO_CLK_CNTL__REFCLK_R_DCO_GATE_DIS_MASK 0x400
+#define DCO_CLK_CNTL__REFCLK_R_DCO_GATE_DIS__SHIFT 0xa
+#define DCO_CLK_CNTL__DISPCLK_G_FMT0_GATE_DIS_MASK 0x10000
+#define DCO_CLK_CNTL__DISPCLK_G_FMT0_GATE_DIS__SHIFT 0x10
+#define DCO_CLK_CNTL__DISPCLK_G_FMT1_GATE_DIS_MASK 0x20000
+#define DCO_CLK_CNTL__DISPCLK_G_FMT1_GATE_DIS__SHIFT 0x11
+#define DCO_CLK_CNTL__DISPCLK_G_FMT2_GATE_DIS_MASK 0x40000
+#define DCO_CLK_CNTL__DISPCLK_G_FMT2_GATE_DIS__SHIFT 0x12
+#define DCO_CLK_CNTL__DISPCLK_G_FMT3_GATE_DIS_MASK 0x80000
+#define DCO_CLK_CNTL__DISPCLK_G_FMT3_GATE_DIS__SHIFT 0x13
+#define DCO_CLK_CNTL__DISPCLK_G_FMT4_GATE_DIS_MASK 0x100000
+#define DCO_CLK_CNTL__DISPCLK_G_FMT4_GATE_DIS__SHIFT 0x14
+#define DCO_CLK_CNTL__DISPCLK_G_FMT5_GATE_DIS_MASK 0x200000
+#define DCO_CLK_CNTL__DISPCLK_G_FMT5_GATE_DIS__SHIFT 0x15
+#define DCO_CLK_CNTL__DISPCLK_G_DIGLPA_GATE_DIS_MASK 0x400000
+#define DCO_CLK_CNTL__DISPCLK_G_DIGLPA_GATE_DIS__SHIFT 0x16
+#define DCO_CLK_CNTL__DISPCLK_G_DIGLPB_GATE_DIS_MASK 0x800000
+#define DCO_CLK_CNTL__DISPCLK_G_DIGLPB_GATE_DIS__SHIFT 0x17
+#define DCO_CLK_CNTL__DISPCLK_G_DIGA_GATE_DIS_MASK 0x1000000
+#define DCO_CLK_CNTL__DISPCLK_G_DIGA_GATE_DIS__SHIFT 0x18
+#define DCO_CLK_CNTL__DISPCLK_G_DIGB_GATE_DIS_MASK 0x2000000
+#define DCO_CLK_CNTL__DISPCLK_G_DIGB_GATE_DIS__SHIFT 0x19
+#define DCO_CLK_CNTL__DISPCLK_G_DIGC_GATE_DIS_MASK 0x4000000
+#define DCO_CLK_CNTL__DISPCLK_G_DIGC_GATE_DIS__SHIFT 0x1a
+#define DCO_CLK_CNTL__DISPCLK_G_DIGD_GATE_DIS_MASK 0x8000000
+#define DCO_CLK_CNTL__DISPCLK_G_DIGD_GATE_DIS__SHIFT 0x1b
+#define DCO_CLK_CNTL__DISPCLK_G_DIGE_GATE_DIS_MASK 0x10000000
+#define DCO_CLK_CNTL__DISPCLK_G_DIGE_GATE_DIS__SHIFT 0x1c
+#define DCO_CLK_CNTL__DISPCLK_G_DIGF_GATE_DIS_MASK 0x20000000
+#define DCO_CLK_CNTL__DISPCLK_G_DIGF_GATE_DIS__SHIFT 0x1d
+#define DCO_CLK_CNTL__DISPCLK_G_DIGG_GATE_DIS_MASK 0x40000000
+#define DCO_CLK_CNTL__DISPCLK_G_DIGG_GATE_DIS__SHIFT 0x1e
+#define DCO_CLK_CNTL2__DCO_TEST_CLK_SEL_MASK 0x7f
+#define DCO_CLK_CNTL2__DCO_TEST_CLK_SEL__SHIFT 0x0
+#define DCO_CLK_CNTL2__SCLK_G_AFMTA_GATE_DIS_MASK 0x80
+#define DCO_CLK_CNTL2__SCLK_G_AFMTA_GATE_DIS__SHIFT 0x7
+#define DCO_CLK_CNTL2__SCLK_G_AFMTB_GATE_DIS_MASK 0x100
+#define DCO_CLK_CNTL2__SCLK_G_AFMTB_GATE_DIS__SHIFT 0x8
+#define DCO_CLK_CNTL2__SCLK_G_AFMTC_GATE_DIS_MASK 0x200
+#define DCO_CLK_CNTL2__SCLK_G_AFMTC_GATE_DIS__SHIFT 0x9
+#define DCO_CLK_CNTL2__SCLK_G_AFMTD_GATE_DIS_MASK 0x400
+#define DCO_CLK_CNTL2__SCLK_G_AFMTD_GATE_DIS__SHIFT 0xa
+#define DCO_CLK_CNTL2__SCLK_G_AFMTE_GATE_DIS_MASK 0x800
+#define DCO_CLK_CNTL2__SCLK_G_AFMTE_GATE_DIS__SHIFT 0xb
+#define DCO_CLK_CNTL2__SCLK_G_AFMTF_GATE_DIS_MASK 0x1000
+#define DCO_CLK_CNTL2__SCLK_G_AFMTF_GATE_DIS__SHIFT 0xc
+#define DCO_CLK_CNTL2__SCLK_G_AFMTG_GATE_DIS_MASK 0x2000
+#define DCO_CLK_CNTL2__SCLK_G_AFMTG_GATE_DIS__SHIFT 0xd
+#define DCO_CLK_CNTL2__SCLK_G_AFMTLPA_GATE_DIS_MASK 0x8000
+#define DCO_CLK_CNTL2__SCLK_G_AFMTLPA_GATE_DIS__SHIFT 0xf
+#define DCO_CLK_CNTL2__SCLK_G_AFMTLPB_GATE_DIS_MASK 0x10000
+#define DCO_CLK_CNTL2__SCLK_G_AFMTLPB_GATE_DIS__SHIFT 0x10
+#define DCO_CLK_CNTL2__SYMCLKA_FE_G_AFMT_GATE_DIS_MASK 0x20000
+#define DCO_CLK_CNTL2__SYMCLKA_FE_G_AFMT_GATE_DIS__SHIFT 0x11
+#define DCO_CLK_CNTL2__SYMCLKB_FE_G_AFMT_GATE_DIS_MASK 0x40000
+#define DCO_CLK_CNTL2__SYMCLKB_FE_G_AFMT_GATE_DIS__SHIFT 0x12
+#define DCO_CLK_CNTL2__SYMCLKC_FE_G_AFMT_GATE_DIS_MASK 0x80000
+#define DCO_CLK_CNTL2__SYMCLKC_FE_G_AFMT_GATE_DIS__SHIFT 0x13
+#define DCO_CLK_CNTL2__SYMCLKD_FE_G_AFMT_GATE_DIS_MASK 0x100000
+#define DCO_CLK_CNTL2__SYMCLKD_FE_G_AFMT_GATE_DIS__SHIFT 0x14
+#define DCO_CLK_CNTL2__SYMCLKE_FE_G_AFMT_GATE_DIS_MASK 0x200000
+#define DCO_CLK_CNTL2__SYMCLKE_FE_G_AFMT_GATE_DIS__SHIFT 0x15
+#define DCO_CLK_CNTL2__SYMCLKF_FE_G_AFMT_GATE_DIS_MASK 0x400000
+#define DCO_CLK_CNTL2__SYMCLKF_FE_G_AFMT_GATE_DIS__SHIFT 0x16
+#define DCO_CLK_CNTL2__SYMCLKG_FE_G_AFMT_GATE_DIS_MASK 0x800000
+#define DCO_CLK_CNTL2__SYMCLKG_FE_G_AFMT_GATE_DIS__SHIFT 0x17
+#define DCO_CLK_CNTL2__SYMCLKLPA_FE_G_AFMT_GATE_DIS_MASK 0x2000000
+#define DCO_CLK_CNTL2__SYMCLKLPA_FE_G_AFMT_GATE_DIS__SHIFT 0x19
+#define DCO_CLK_CNTL2__SYMCLKLPB_FE_G_AFMT_GATE_DIS_MASK 0x4000000
+#define DCO_CLK_CNTL2__SYMCLKLPB_FE_G_AFMT_GATE_DIS__SHIFT 0x1a
+#define DCO_CLK_CNTL3__SYMCLKA_FE_G_TMDS_GATE_DIS_MASK 0x1
+#define DCO_CLK_CNTL3__SYMCLKA_FE_G_TMDS_GATE_DIS__SHIFT 0x0
+#define DCO_CLK_CNTL3__SYMCLKB_FE_G_TMDS_GATE_DIS_MASK 0x2
+#define DCO_CLK_CNTL3__SYMCLKB_FE_G_TMDS_GATE_DIS__SHIFT 0x1
+#define DCO_CLK_CNTL3__SYMCLKC_FE_G_TMDS_GATE_DIS_MASK 0x4
+#define DCO_CLK_CNTL3__SYMCLKC_FE_G_TMDS_GATE_DIS__SHIFT 0x2
+#define DCO_CLK_CNTL3__SYMCLKD_FE_G_TMDS_GATE_DIS_MASK 0x8
+#define DCO_CLK_CNTL3__SYMCLKD_FE_G_TMDS_GATE_DIS__SHIFT 0x3
+#define DCO_CLK_CNTL3__SYMCLKE_FE_G_TMDS_GATE_DIS_MASK 0x10
+#define DCO_CLK_CNTL3__SYMCLKE_FE_G_TMDS_GATE_DIS__SHIFT 0x4
+#define DCO_CLK_CNTL3__SYMCLKF_FE_G_TMDS_GATE_DIS_MASK 0x20
+#define DCO_CLK_CNTL3__SYMCLKF_FE_G_TMDS_GATE_DIS__SHIFT 0x5
+#define DCO_CLK_CNTL3__SYMCLKG_FE_G_TMDS_GATE_DIS_MASK 0x40
+#define DCO_CLK_CNTL3__SYMCLKG_FE_G_TMDS_GATE_DIS__SHIFT 0x6
+#define DCO_CLK_CNTL3__SYMCLKLPA_FE_G_TMDS_GATE_DIS_MASK 0x100
+#define DCO_CLK_CNTL3__SYMCLKLPA_FE_G_TMDS_GATE_DIS__SHIFT 0x8
+#define DCO_CLK_CNTL3__SYMCLKLPB_FE_G_TMDS_GATE_DIS_MASK 0x200
+#define DCO_CLK_CNTL3__SYMCLKLPB_FE_G_TMDS_GATE_DIS__SHIFT 0x9
+#define DCO_CLK_CNTL3__SYMCLKA_G_TMDS_GATE_DIS_MASK 0x400
+#define DCO_CLK_CNTL3__SYMCLKA_G_TMDS_GATE_DIS__SHIFT 0xa
+#define DCO_CLK_CNTL3__SYMCLKB_G_TMDS_GATE_DIS_MASK 0x800
+#define DCO_CLK_CNTL3__SYMCLKB_G_TMDS_GATE_DIS__SHIFT 0xb
+#define DCO_CLK_CNTL3__SYMCLKC_G_TMDS_GATE_DIS_MASK 0x1000
+#define DCO_CLK_CNTL3__SYMCLKC_G_TMDS_GATE_DIS__SHIFT 0xc
+#define DCO_CLK_CNTL3__SYMCLKD_G_TMDS_GATE_DIS_MASK 0x2000
+#define DCO_CLK_CNTL3__SYMCLKD_G_TMDS_GATE_DIS__SHIFT 0xd
+#define DCO_CLK_CNTL3__SYMCLKE_G_TMDS_GATE_DIS_MASK 0x4000
+#define DCO_CLK_CNTL3__SYMCLKE_G_TMDS_GATE_DIS__SHIFT 0xe
+#define DCO_CLK_CNTL3__SYMCLKF_G_TMDS_GATE_DIS_MASK 0x8000
+#define DCO_CLK_CNTL3__SYMCLKF_G_TMDS_GATE_DIS__SHIFT 0xf
+#define DCO_CLK_CNTL3__SYMCLKG_G_TMDS_GATE_DIS_MASK 0x10000
+#define DCO_CLK_CNTL3__SYMCLKG_G_TMDS_GATE_DIS__SHIFT 0x10
+#define DCO_CLK_CNTL3__SYMCLKLPA_G_TMDS_GATE_DIS_MASK 0x40000
+#define DCO_CLK_CNTL3__SYMCLKLPA_G_TMDS_GATE_DIS__SHIFT 0x12
+#define DCO_CLK_CNTL3__SYMCLKLPB_G_TMDS_GATE_DIS_MASK 0x80000
+#define DCO_CLK_CNTL3__SYMCLKLPB_G_TMDS_GATE_DIS__SHIFT 0x13
+#define DPDBG_CNTL__DPDBG_ENABLE_MASK 0x1
+#define DPDBG_CNTL__DPDBG_ENABLE__SHIFT 0x0
+#define DPDBG_CNTL__DPDBG_INPUT_ENABLE_MASK 0x2
+#define DPDBG_CNTL__DPDBG_INPUT_ENABLE__SHIFT 0x1
+#define DPDBG_CNTL__DPDBG_SYMCLK_ON_MASK 0x10
+#define DPDBG_CNTL__DPDBG_SYMCLK_ON__SHIFT 0x4
+#define DPDBG_CNTL__DPDBG_ERROR_DETECTION_MODE_MASK 0x100
+#define DPDBG_CNTL__DPDBG_ERROR_DETECTION_MODE__SHIFT 0x8
+#define DPDBG_CNTL__DPDBG_LINE_LENGTH_MASK 0xffff0000
+#define DPDBG_CNTL__DPDBG_LINE_LENGTH__SHIFT 0x10
+#define DPDBG_INTERRUPT__DPDBG_FIFO_OVERFLOW_INT_MASK_MASK 0x1
+#define DPDBG_INTERRUPT__DPDBG_FIFO_OVERFLOW_INT_MASK__SHIFT 0x0
+#define DPDBG_INTERRUPT__DPDBG_FIFO_OVERFLOW_INT_TYPE_MASK 0x2
+#define DPDBG_INTERRUPT__DPDBG_FIFO_OVERFLOW_INT_TYPE__SHIFT 0x1
+#define DPDBG_INTERRUPT__DPDBG_FIFO_OVERFLOW_INT_ACK_MASK 0x100
+#define DPDBG_INTERRUPT__DPDBG_FIFO_OVERFLOW_INT_ACK__SHIFT 0x8
+#define DPDBG_INTERRUPT__DPDBG_FIFO_OVERFLOW_OCCURRED_MASK 0x10000
+#define DPDBG_INTERRUPT__DPDBG_FIFO_OVERFLOW_OCCURRED__SHIFT 0x10
+#define DPDBG_INTERRUPT__DPDBG_FIFO_OVERFLOW_INT_STATUS_MASK 0x1000000
+#define DPDBG_INTERRUPT__DPDBG_FIFO_OVERFLOW_INT_STATUS__SHIFT 0x18
+#define DCO_POWER_MANAGEMENT_CNTL__PM_ASSERT_RESET_MASK 0x1
+#define DCO_POWER_MANAGEMENT_CNTL__PM_ASSERT_RESET__SHIFT 0x0
+#define DCO_POWER_MANAGEMENT_CNTL__PM_ALL_BUSY_OFF_MASK 0x100
+#define DCO_POWER_MANAGEMENT_CNTL__PM_ALL_BUSY_OFF__SHIFT 0x8
+#define DCO_SOFT_RESET__DACA_SOFT_RESET_MASK 0x1
+#define DCO_SOFT_RESET__DACA_SOFT_RESET__SHIFT 0x0
+#define DCO_SOFT_RESET__I2S0_SPDIF0_SOFT_RESET_MASK 0x10
+#define DCO_SOFT_RESET__I2S0_SPDIF0_SOFT_RESET__SHIFT 0x4
+#define DCO_SOFT_RESET__I2S1_SOFT_RESET_MASK 0x20
+#define DCO_SOFT_RESET__I2S1_SOFT_RESET__SHIFT 0x5
+#define DCO_SOFT_RESET__SPDIF1_SOFT_RESET_MASK 0x40
+#define DCO_SOFT_RESET__SPDIF1_SOFT_RESET__SHIFT 0x6
+#define DCO_SOFT_RESET__DB_CLK_SOFT_RESET_MASK 0x1000
+#define DCO_SOFT_RESET__DB_CLK_SOFT_RESET__SHIFT 0xc
+#define DCO_SOFT_RESET__FMT0_SOFT_RESET_MASK 0x10000
+#define DCO_SOFT_RESET__FMT0_SOFT_RESET__SHIFT 0x10
+#define DCO_SOFT_RESET__FMT1_SOFT_RESET_MASK 0x20000
+#define DCO_SOFT_RESET__FMT1_SOFT_RESET__SHIFT 0x11
+#define DCO_SOFT_RESET__FMT2_SOFT_RESET_MASK 0x40000
+#define DCO_SOFT_RESET__FMT2_SOFT_RESET__SHIFT 0x12
+#define DCO_SOFT_RESET__FMT3_SOFT_RESET_MASK 0x80000
+#define DCO_SOFT_RESET__FMT3_SOFT_RESET__SHIFT 0x13
+#define DCO_SOFT_RESET__FMT4_SOFT_RESET_MASK 0x100000
+#define DCO_SOFT_RESET__FMT4_SOFT_RESET__SHIFT 0x14
+#define DCO_SOFT_RESET__FMT5_SOFT_RESET_MASK 0x200000
+#define DCO_SOFT_RESET__FMT5_SOFT_RESET__SHIFT 0x15
+#define DCO_SOFT_RESET__MVP_SOFT_RESET_MASK 0x1000000
+#define DCO_SOFT_RESET__MVP_SOFT_RESET__SHIFT 0x18
+#define DCO_SOFT_RESET__ABM_SOFT_RESET_MASK 0x2000000
+#define DCO_SOFT_RESET__ABM_SOFT_RESET__SHIFT 0x19
+#define DCO_SOFT_RESET__DVO_SOFT_RESET_MASK 0x8000000
+#define DCO_SOFT_RESET__DVO_SOFT_RESET__SHIFT 0x1b
+#define DIG_SOFT_RESET__DIGA_FE_SOFT_RESET_MASK 0x1
+#define DIG_SOFT_RESET__DIGA_FE_SOFT_RESET__SHIFT 0x0
+#define DIG_SOFT_RESET__DIGA_BE_SOFT_RESET_MASK 0x2
+#define DIG_SOFT_RESET__DIGA_BE_SOFT_RESET__SHIFT 0x1
+#define DIG_SOFT_RESET__DIGB_FE_SOFT_RESET_MASK 0x10
+#define DIG_SOFT_RESET__DIGB_FE_SOFT_RESET__SHIFT 0x4
+#define DIG_SOFT_RESET__DIGB_BE_SOFT_RESET_MASK 0x20
+#define DIG_SOFT_RESET__DIGB_BE_SOFT_RESET__SHIFT 0x5
+#define DIG_SOFT_RESET__DIGC_FE_SOFT_RESET_MASK 0x100
+#define DIG_SOFT_RESET__DIGC_FE_SOFT_RESET__SHIFT 0x8
+#define DIG_SOFT_RESET__DIGC_BE_SOFT_RESET_MASK 0x200
+#define DIG_SOFT_RESET__DIGC_BE_SOFT_RESET__SHIFT 0x9
+#define DIG_SOFT_RESET__DIGD_FE_SOFT_RESET_MASK 0x1000
+#define DIG_SOFT_RESET__DIGD_FE_SOFT_RESET__SHIFT 0xc
+#define DIG_SOFT_RESET__DIGD_BE_SOFT_RESET_MASK 0x2000
+#define DIG_SOFT_RESET__DIGD_BE_SOFT_RESET__SHIFT 0xd
+#define DIG_SOFT_RESET__DIGE_FE_SOFT_RESET_MASK 0x10000
+#define DIG_SOFT_RESET__DIGE_FE_SOFT_RESET__SHIFT 0x10
+#define DIG_SOFT_RESET__DIGE_BE_SOFT_RESET_MASK 0x20000
+#define DIG_SOFT_RESET__DIGE_BE_SOFT_RESET__SHIFT 0x11
+#define DIG_SOFT_RESET__DIGF_FE_SOFT_RESET_MASK 0x100000
+#define DIG_SOFT_RESET__DIGF_FE_SOFT_RESET__SHIFT 0x14
+#define DIG_SOFT_RESET__DIGF_BE_SOFT_RESET_MASK 0x200000
+#define DIG_SOFT_RESET__DIGF_BE_SOFT_RESET__SHIFT 0x15
+#define DIG_SOFT_RESET__DIGG_FE_SOFT_RESET_MASK 0x1000000
+#define DIG_SOFT_RESET__DIGG_FE_SOFT_RESET__SHIFT 0x18
+#define DIG_SOFT_RESET__DIGG_BE_SOFT_RESET_MASK 0x2000000
+#define DIG_SOFT_RESET__DIGG_BE_SOFT_RESET__SHIFT 0x19
+#define DIG_SOFT_RESET__DPDBG_SOFT_RESET_MASK 0x80000000
+#define DIG_SOFT_RESET__DPDBG_SOFT_RESET__SHIFT 0x1f
+#define DIG_SOFT_RESET_2__DIGLPA_FE_SOFT_RESET_MASK 0x1
+#define DIG_SOFT_RESET_2__DIGLPA_FE_SOFT_RESET__SHIFT 0x0
+#define DIG_SOFT_RESET_2__DIGLPA_BE_SOFT_RESET_MASK 0x2
+#define DIG_SOFT_RESET_2__DIGLPA_BE_SOFT_RESET__SHIFT 0x1
+#define DIG_SOFT_RESET_2__DIGLPB_FE_SOFT_RESET_MASK 0x10
+#define DIG_SOFT_RESET_2__DIGLPB_FE_SOFT_RESET__SHIFT 0x4
+#define DIG_SOFT_RESET_2__DIGLPB_BE_SOFT_RESET_MASK 0x20
+#define DIG_SOFT_RESET_2__DIGLPB_BE_SOFT_RESET__SHIFT 0x5
+#define DCO_STEREOSYNC_SEL__GENERICA_STEREOSYNC_SEL_MASK 0x7
+#define DCO_STEREOSYNC_SEL__GENERICA_STEREOSYNC_SEL__SHIFT 0x0
+#define DCO_STEREOSYNC_SEL__GENERICB_STEREOSYNC_SEL_MASK 0x70000
+#define DCO_STEREOSYNC_SEL__GENERICB_STEREOSYNC_SEL__SHIFT 0x10
+#define DCO_HDMI_RXSTATUS_TIMER_CONTROL__DCO_HDMI_RXSTATUS_TIMER_ENABLE_MASK 0x1
+#define DCO_HDMI_RXSTATUS_TIMER_CONTROL__DCO_HDMI_RXSTATUS_TIMER_ENABLE__SHIFT 0x0
+#define DCO_HDMI_RXSTATUS_TIMER_CONTROL__DCO_HDMI_RXSTATUS_TIMER_TYPE_MASK 0x10
+#define DCO_HDMI_RXSTATUS_TIMER_CONTROL__DCO_HDMI_RXSTATUS_TIMER_TYPE__SHIFT 0x4
+#define DCO_HDMI_RXSTATUS_TIMER_CONTROL__DCO_HDMI_RXSTATUS_TIMER_STATUS_MASK 0x100
+#define DCO_HDMI_RXSTATUS_TIMER_CONTROL__DCO_HDMI_RXSTATUS_TIMER_STATUS__SHIFT 0x8
+#define DCO_HDMI_RXSTATUS_TIMER_CONTROL__DCO_HDMI_RXSTATUS_TIMER_MASK_MASK 0x1000
+#define DCO_HDMI_RXSTATUS_TIMER_CONTROL__DCO_HDMI_RXSTATUS_TIMER_MASK__SHIFT 0xc
+#define DCO_HDMI_RXSTATUS_TIMER_CONTROL__DCO_HDMI_RXSTATUS_TIMER_INTERVAL_MASK 0xfff0000
+#define DCO_HDMI_RXSTATUS_TIMER_CONTROL__DCO_HDMI_RXSTATUS_TIMER_INTERVAL__SHIFT 0x10
+#define DCO_PSP_INTERRUPT_STATUS__DCO_PSP_INTERRUPT_STATUS_MASK 0x1
+#define DCO_PSP_INTERRUPT_STATUS__DCO_PSP_INTERRUPT_STATUS__SHIFT 0x0
+#define DCO_PSP_INTERRUPT_STATUS__DCO_PSP_INTERRUPT_MESSAGE_MASK 0xfffffffe
+#define DCO_PSP_INTERRUPT_STATUS__DCO_PSP_INTERRUPT_MESSAGE__SHIFT 0x1
+#define DCO_PSP_INTERRUPT_CLEAR__DCO_PSP_INTERRUPT_CLEAR_MASK 0x1
+#define DCO_PSP_INTERRUPT_CLEAR__DCO_PSP_INTERRUPT_CLEAR__SHIFT 0x0
+#define DCO_GENERIC_INTERRUPT_MESSAGE__DCO_GENERIC_INTERRUPT_STATUS_MASK 0x1
+#define DCO_GENERIC_INTERRUPT_MESSAGE__DCO_GENERIC_INTERRUPT_STATUS__SHIFT 0x0
+#define DCO_GENERIC_INTERRUPT_MESSAGE__DCO_GENERIC_INTERRUPT_MESSAGE_MASK 0xfffffffe
+#define DCO_GENERIC_INTERRUPT_MESSAGE__DCO_GENERIC_INTERRUPT_MESSAGE__SHIFT 0x1
+#define DCO_GENERIC_INTERRUPT_CLEAR__DCO_GENERIC_INTERRUPT_CLEAR_MASK 0x1
+#define DCO_GENERIC_INTERRUPT_CLEAR__DCO_GENERIC_INTERRUPT_CLEAR__SHIFT 0x0
+#define DCO_TEST_DEBUG_INDEX__DCO_TEST_DEBUG_INDEX_MASK 0xff
+#define DCO_TEST_DEBUG_INDEX__DCO_TEST_DEBUG_INDEX__SHIFT 0x0
+#define DCO_TEST_DEBUG_INDEX__DCO_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define DCO_TEST_DEBUG_INDEX__DCO_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define DCO_TEST_DEBUG_DATA__DCO_TEST_DEBUG_DATA_MASK 0xffffffff
+#define DCO_TEST_DEBUG_DATA__DCO_TEST_DEBUG_DATA__SHIFT 0x0
+#define DC_I2C_CONTROL__DC_I2C_GO_MASK 0x1
+#define DC_I2C_CONTROL__DC_I2C_GO__SHIFT 0x0
+#define DC_I2C_CONTROL__DC_I2C_SOFT_RESET_MASK 0x2
+#define DC_I2C_CONTROL__DC_I2C_SOFT_RESET__SHIFT 0x1
+#define DC_I2C_CONTROL__DC_I2C_SEND_RESET_MASK 0x4
+#define DC_I2C_CONTROL__DC_I2C_SEND_RESET__SHIFT 0x2
+#define DC_I2C_CONTROL__DC_I2C_SW_STATUS_RESET_MASK 0x8
+#define DC_I2C_CONTROL__DC_I2C_SW_STATUS_RESET__SHIFT 0x3
+#define DC_I2C_CONTROL__DC_I2C_DDC_SELECT_MASK 0x700
+#define DC_I2C_CONTROL__DC_I2C_DDC_SELECT__SHIFT 0x8
+#define DC_I2C_CONTROL__DC_I2C_TRANSACTION_COUNT_MASK 0x300000
+#define DC_I2C_CONTROL__DC_I2C_TRANSACTION_COUNT__SHIFT 0x14
+#define DC_I2C_CONTROL__DC_I2C_DBG_REF_SEL_MASK 0x80000000
+#define DC_I2C_CONTROL__DC_I2C_DBG_REF_SEL__SHIFT 0x1f
+#define DC_I2C_ARBITRATION__DC_I2C_SW_PRIORITY_MASK 0x3
+#define DC_I2C_ARBITRATION__DC_I2C_SW_PRIORITY__SHIFT 0x0
+#define DC_I2C_ARBITRATION__DC_I2C_REG_RW_CNTL_STATUS_MASK 0xc
+#define DC_I2C_ARBITRATION__DC_I2C_REG_RW_CNTL_STATUS__SHIFT 0x2
+#define DC_I2C_ARBITRATION__DC_I2C_NO_QUEUED_SW_GO_MASK 0x10
+#define DC_I2C_ARBITRATION__DC_I2C_NO_QUEUED_SW_GO__SHIFT 0x4
+#define DC_I2C_ARBITRATION__DC_I2C_ABORT_HW_XFER_MASK 0x100
+#define DC_I2C_ARBITRATION__DC_I2C_ABORT_HW_XFER__SHIFT 0x8
+#define DC_I2C_ARBITRATION__DC_I2C_ABORT_SW_XFER_MASK 0x1000
+#define DC_I2C_ARBITRATION__DC_I2C_ABORT_SW_XFER__SHIFT 0xc
+#define DC_I2C_ARBITRATION__DC_I2C_SW_USE_I2C_REG_REQ_MASK 0x100000
+#define DC_I2C_ARBITRATION__DC_I2C_SW_USE_I2C_REG_REQ__SHIFT 0x14
+#define DC_I2C_ARBITRATION__DC_I2C_SW_DONE_USING_I2C_REG_MASK 0x200000
+#define DC_I2C_ARBITRATION__DC_I2C_SW_DONE_USING_I2C_REG__SHIFT 0x15
+#define DC_I2C_ARBITRATION__DC_I2C_DMCU_USE_I2C_REG_REQ_MASK 0x1000000
+#define DC_I2C_ARBITRATION__DC_I2C_DMCU_USE_I2C_REG_REQ__SHIFT 0x18
+#define DC_I2C_ARBITRATION__DC_I2C_DMCU_DONE_USING_I2C_REG_MASK 0x2000000
+#define DC_I2C_ARBITRATION__DC_I2C_DMCU_DONE_USING_I2C_REG__SHIFT 0x19
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_SW_DONE_INT_MASK 0x1
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_SW_DONE_INT__SHIFT 0x0
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_SW_DONE_ACK_MASK 0x2
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_SW_DONE_ACK__SHIFT 0x1
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_SW_DONE_MASK_MASK 0x4
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_SW_DONE_MASK__SHIFT 0x2
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC1_HW_DONE_INT_MASK 0x10
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC1_HW_DONE_INT__SHIFT 0x4
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC1_HW_DONE_ACK_MASK 0x20
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC1_HW_DONE_ACK__SHIFT 0x5
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC1_HW_DONE_MASK_MASK 0x40
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC1_HW_DONE_MASK__SHIFT 0x6
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC2_HW_DONE_INT_MASK 0x100
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC2_HW_DONE_INT__SHIFT 0x8
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC2_HW_DONE_ACK_MASK 0x200
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC2_HW_DONE_ACK__SHIFT 0x9
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC2_HW_DONE_MASK_MASK 0x400
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC2_HW_DONE_MASK__SHIFT 0xa
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC3_HW_DONE_INT_MASK 0x1000
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC3_HW_DONE_INT__SHIFT 0xc
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC3_HW_DONE_ACK_MASK 0x2000
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC3_HW_DONE_ACK__SHIFT 0xd
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC3_HW_DONE_MASK_MASK 0x4000
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC3_HW_DONE_MASK__SHIFT 0xe
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC4_HW_DONE_INT_MASK 0x10000
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC4_HW_DONE_INT__SHIFT 0x10
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC4_HW_DONE_ACK_MASK 0x20000
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC4_HW_DONE_ACK__SHIFT 0x11
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC4_HW_DONE_MASK_MASK 0x40000
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC4_HW_DONE_MASK__SHIFT 0x12
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC5_HW_DONE_INT_MASK 0x100000
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC5_HW_DONE_INT__SHIFT 0x14
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC5_HW_DONE_ACK_MASK 0x200000
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC5_HW_DONE_ACK__SHIFT 0x15
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC5_HW_DONE_MASK_MASK 0x400000
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC5_HW_DONE_MASK__SHIFT 0x16
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC6_HW_DONE_INT_MASK 0x1000000
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC6_HW_DONE_INT__SHIFT 0x18
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC6_HW_DONE_ACK_MASK 0x2000000
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC6_HW_DONE_ACK__SHIFT 0x19
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC6_HW_DONE_MASK_MASK 0x4000000
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDC6_HW_DONE_MASK__SHIFT 0x1a
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDCVGA_HW_DONE_INT_MASK 0x8000000
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDCVGA_HW_DONE_INT__SHIFT 0x1b
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDCVGA_HW_DONE_ACK_MASK 0x10000000
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDCVGA_HW_DONE_ACK__SHIFT 0x1c
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDCVGA_HW_DONE_MASK_MASK 0x20000000
+#define DC_I2C_INTERRUPT_CONTROL__DC_I2C_DDCVGA_HW_DONE_MASK__SHIFT 0x1d
+#define DC_I2C_SW_STATUS__DC_I2C_SW_STATUS_MASK 0x3
+#define DC_I2C_SW_STATUS__DC_I2C_SW_STATUS__SHIFT 0x0
+#define DC_I2C_SW_STATUS__DC_I2C_SW_DONE_MASK 0x4
+#define DC_I2C_SW_STATUS__DC_I2C_SW_DONE__SHIFT 0x2
+#define DC_I2C_SW_STATUS__DC_I2C_SW_ABORTED_MASK 0x10
+#define DC_I2C_SW_STATUS__DC_I2C_SW_ABORTED__SHIFT 0x4
+#define DC_I2C_SW_STATUS__DC_I2C_SW_TIMEOUT_MASK 0x20
+#define DC_I2C_SW_STATUS__DC_I2C_SW_TIMEOUT__SHIFT 0x5
+#define DC_I2C_SW_STATUS__DC_I2C_SW_INTERRUPTED_MASK 0x40
+#define DC_I2C_SW_STATUS__DC_I2C_SW_INTERRUPTED__SHIFT 0x6
+#define DC_I2C_SW_STATUS__DC_I2C_SW_BUFFER_OVERFLOW_MASK 0x80
+#define DC_I2C_SW_STATUS__DC_I2C_SW_BUFFER_OVERFLOW__SHIFT 0x7
+#define DC_I2C_SW_STATUS__DC_I2C_SW_STOPPED_ON_NACK_MASK 0x100
+#define DC_I2C_SW_STATUS__DC_I2C_SW_STOPPED_ON_NACK__SHIFT 0x8
+#define DC_I2C_SW_STATUS__DC_I2C_SW_NACK0_MASK 0x1000
+#define DC_I2C_SW_STATUS__DC_I2C_SW_NACK0__SHIFT 0xc
+#define DC_I2C_SW_STATUS__DC_I2C_SW_NACK1_MASK 0x2000
+#define DC_I2C_SW_STATUS__DC_I2C_SW_NACK1__SHIFT 0xd
+#define DC_I2C_SW_STATUS__DC_I2C_SW_NACK2_MASK 0x4000
+#define DC_I2C_SW_STATUS__DC_I2C_SW_NACK2__SHIFT 0xe
+#define DC_I2C_SW_STATUS__DC_I2C_SW_NACK3_MASK 0x8000
+#define DC_I2C_SW_STATUS__DC_I2C_SW_NACK3__SHIFT 0xf
+#define DC_I2C_SW_STATUS__DC_I2C_SW_REQ_MASK 0x40000
+#define DC_I2C_SW_STATUS__DC_I2C_SW_REQ__SHIFT 0x12
+#define DC_I2C_DDC1_HW_STATUS__DC_I2C_DDC1_HW_STATUS_MASK 0x3
+#define DC_I2C_DDC1_HW_STATUS__DC_I2C_DDC1_HW_STATUS__SHIFT 0x0
+#define DC_I2C_DDC1_HW_STATUS__DC_I2C_DDC1_HW_DONE_MASK 0x8
+#define DC_I2C_DDC1_HW_STATUS__DC_I2C_DDC1_HW_DONE__SHIFT 0x3
+#define DC_I2C_DDC1_HW_STATUS__DC_I2C_DDC1_HW_REQ_MASK 0x10000
+#define DC_I2C_DDC1_HW_STATUS__DC_I2C_DDC1_HW_REQ__SHIFT 0x10
+#define DC_I2C_DDC1_HW_STATUS__DC_I2C_DDC1_HW_URG_MASK 0x20000
+#define DC_I2C_DDC1_HW_STATUS__DC_I2C_DDC1_HW_URG__SHIFT 0x11
+#define DC_I2C_DDC1_HW_STATUS__DC_I2C_DDC1_EDID_DETECT_STATUS_MASK 0x100000
+#define DC_I2C_DDC1_HW_STATUS__DC_I2C_DDC1_EDID_DETECT_STATUS__SHIFT 0x14
+#define DC_I2C_DDC1_HW_STATUS__DC_I2C_DDC1_EDID_DETECT_NUM_VALID_TRIES_MASK 0xf000000
+#define DC_I2C_DDC1_HW_STATUS__DC_I2C_DDC1_EDID_DETECT_NUM_VALID_TRIES__SHIFT 0x18
+#define DC_I2C_DDC1_HW_STATUS__DC_I2C_DDC1_EDID_DETECT_STATE_MASK 0x70000000
+#define DC_I2C_DDC1_HW_STATUS__DC_I2C_DDC1_EDID_DETECT_STATE__SHIFT 0x1c
+#define DC_I2C_DDC2_HW_STATUS__DC_I2C_DDC2_HW_STATUS_MASK 0x3
+#define DC_I2C_DDC2_HW_STATUS__DC_I2C_DDC2_HW_STATUS__SHIFT 0x0
+#define DC_I2C_DDC2_HW_STATUS__DC_I2C_DDC2_HW_DONE_MASK 0x8
+#define DC_I2C_DDC2_HW_STATUS__DC_I2C_DDC2_HW_DONE__SHIFT 0x3
+#define DC_I2C_DDC2_HW_STATUS__DC_I2C_DDC2_HW_REQ_MASK 0x10000
+#define DC_I2C_DDC2_HW_STATUS__DC_I2C_DDC2_HW_REQ__SHIFT 0x10
+#define DC_I2C_DDC2_HW_STATUS__DC_I2C_DDC2_HW_URG_MASK 0x20000
+#define DC_I2C_DDC2_HW_STATUS__DC_I2C_DDC2_HW_URG__SHIFT 0x11
+#define DC_I2C_DDC2_HW_STATUS__DC_I2C_DDC2_EDID_DETECT_STATUS_MASK 0x100000
+#define DC_I2C_DDC2_HW_STATUS__DC_I2C_DDC2_EDID_DETECT_STATUS__SHIFT 0x14
+#define DC_I2C_DDC2_HW_STATUS__DC_I2C_DDC2_EDID_DETECT_NUM_VALID_TRIES_MASK 0xf000000
+#define DC_I2C_DDC2_HW_STATUS__DC_I2C_DDC2_EDID_DETECT_NUM_VALID_TRIES__SHIFT 0x18
+#define DC_I2C_DDC2_HW_STATUS__DC_I2C_DDC2_EDID_DETECT_STATE_MASK 0x70000000
+#define DC_I2C_DDC2_HW_STATUS__DC_I2C_DDC2_EDID_DETECT_STATE__SHIFT 0x1c
+#define DC_I2C_DDC3_HW_STATUS__DC_I2C_DDC3_HW_STATUS_MASK 0x3
+#define DC_I2C_DDC3_HW_STATUS__DC_I2C_DDC3_HW_STATUS__SHIFT 0x0
+#define DC_I2C_DDC3_HW_STATUS__DC_I2C_DDC3_HW_DONE_MASK 0x8
+#define DC_I2C_DDC3_HW_STATUS__DC_I2C_DDC3_HW_DONE__SHIFT 0x3
+#define DC_I2C_DDC3_HW_STATUS__DC_I2C_DDC3_HW_REQ_MASK 0x10000
+#define DC_I2C_DDC3_HW_STATUS__DC_I2C_DDC3_HW_REQ__SHIFT 0x10
+#define DC_I2C_DDC3_HW_STATUS__DC_I2C_DDC3_HW_URG_MASK 0x20000
+#define DC_I2C_DDC3_HW_STATUS__DC_I2C_DDC3_HW_URG__SHIFT 0x11
+#define DC_I2C_DDC3_HW_STATUS__DC_I2C_DDC3_EDID_DETECT_STATUS_MASK 0x100000
+#define DC_I2C_DDC3_HW_STATUS__DC_I2C_DDC3_EDID_DETECT_STATUS__SHIFT 0x14
+#define DC_I2C_DDC3_HW_STATUS__DC_I2C_DDC3_EDID_DETECT_NUM_VALID_TRIES_MASK 0xf000000
+#define DC_I2C_DDC3_HW_STATUS__DC_I2C_DDC3_EDID_DETECT_NUM_VALID_TRIES__SHIFT 0x18
+#define DC_I2C_DDC3_HW_STATUS__DC_I2C_DDC3_EDID_DETECT_STATE_MASK 0x70000000
+#define DC_I2C_DDC3_HW_STATUS__DC_I2C_DDC3_EDID_DETECT_STATE__SHIFT 0x1c
+#define DC_I2C_DDC4_HW_STATUS__DC_I2C_DDC4_HW_STATUS_MASK 0x3
+#define DC_I2C_DDC4_HW_STATUS__DC_I2C_DDC4_HW_STATUS__SHIFT 0x0
+#define DC_I2C_DDC4_HW_STATUS__DC_I2C_DDC4_HW_DONE_MASK 0x8
+#define DC_I2C_DDC4_HW_STATUS__DC_I2C_DDC4_HW_DONE__SHIFT 0x3
+#define DC_I2C_DDC4_HW_STATUS__DC_I2C_DDC4_HW_REQ_MASK 0x10000
+#define DC_I2C_DDC4_HW_STATUS__DC_I2C_DDC4_HW_REQ__SHIFT 0x10
+#define DC_I2C_DDC4_HW_STATUS__DC_I2C_DDC4_HW_URG_MASK 0x20000
+#define DC_I2C_DDC4_HW_STATUS__DC_I2C_DDC4_HW_URG__SHIFT 0x11
+#define DC_I2C_DDC4_HW_STATUS__DC_I2C_DDC4_EDID_DETECT_STATUS_MASK 0x100000
+#define DC_I2C_DDC4_HW_STATUS__DC_I2C_DDC4_EDID_DETECT_STATUS__SHIFT 0x14
+#define DC_I2C_DDC4_HW_STATUS__DC_I2C_DDC4_EDID_DETECT_NUM_VALID_TRIES_MASK 0xf000000
+#define DC_I2C_DDC4_HW_STATUS__DC_I2C_DDC4_EDID_DETECT_NUM_VALID_TRIES__SHIFT 0x18
+#define DC_I2C_DDC4_HW_STATUS__DC_I2C_DDC4_EDID_DETECT_STATE_MASK 0x70000000
+#define DC_I2C_DDC4_HW_STATUS__DC_I2C_DDC4_EDID_DETECT_STATE__SHIFT 0x1c
+#define DC_I2C_DDC5_HW_STATUS__DC_I2C_DDC5_HW_STATUS_MASK 0x3
+#define DC_I2C_DDC5_HW_STATUS__DC_I2C_DDC5_HW_STATUS__SHIFT 0x0
+#define DC_I2C_DDC5_HW_STATUS__DC_I2C_DDC5_HW_DONE_MASK 0x8
+#define DC_I2C_DDC5_HW_STATUS__DC_I2C_DDC5_HW_DONE__SHIFT 0x3
+#define DC_I2C_DDC5_HW_STATUS__DC_I2C_DDC5_HW_REQ_MASK 0x10000
+#define DC_I2C_DDC5_HW_STATUS__DC_I2C_DDC5_HW_REQ__SHIFT 0x10
+#define DC_I2C_DDC5_HW_STATUS__DC_I2C_DDC5_HW_URG_MASK 0x20000
+#define DC_I2C_DDC5_HW_STATUS__DC_I2C_DDC5_HW_URG__SHIFT 0x11
+#define DC_I2C_DDC5_HW_STATUS__DC_I2C_DDC5_EDID_DETECT_STATUS_MASK 0x100000
+#define DC_I2C_DDC5_HW_STATUS__DC_I2C_DDC5_EDID_DETECT_STATUS__SHIFT 0x14
+#define DC_I2C_DDC5_HW_STATUS__DC_I2C_DDC5_EDID_DETECT_NUM_VALID_TRIES_MASK 0xf000000
+#define DC_I2C_DDC5_HW_STATUS__DC_I2C_DDC5_EDID_DETECT_NUM_VALID_TRIES__SHIFT 0x18
+#define DC_I2C_DDC5_HW_STATUS__DC_I2C_DDC5_EDID_DETECT_STATE_MASK 0x70000000
+#define DC_I2C_DDC5_HW_STATUS__DC_I2C_DDC5_EDID_DETECT_STATE__SHIFT 0x1c
+#define DC_I2C_DDC6_HW_STATUS__DC_I2C_DDC6_HW_STATUS_MASK 0x3
+#define DC_I2C_DDC6_HW_STATUS__DC_I2C_DDC6_HW_STATUS__SHIFT 0x0
+#define DC_I2C_DDC6_HW_STATUS__DC_I2C_DDC6_HW_DONE_MASK 0x8
+#define DC_I2C_DDC6_HW_STATUS__DC_I2C_DDC6_HW_DONE__SHIFT 0x3
+#define DC_I2C_DDC6_HW_STATUS__DC_I2C_DDC6_HW_REQ_MASK 0x10000
+#define DC_I2C_DDC6_HW_STATUS__DC_I2C_DDC6_HW_REQ__SHIFT 0x10
+#define DC_I2C_DDC6_HW_STATUS__DC_I2C_DDC6_HW_URG_MASK 0x20000
+#define DC_I2C_DDC6_HW_STATUS__DC_I2C_DDC6_HW_URG__SHIFT 0x11
+#define DC_I2C_DDC6_HW_STATUS__DC_I2C_DDC6_EDID_DETECT_STATUS_MASK 0x100000
+#define DC_I2C_DDC6_HW_STATUS__DC_I2C_DDC6_EDID_DETECT_STATUS__SHIFT 0x14
+#define DC_I2C_DDC6_HW_STATUS__DC_I2C_DDC6_EDID_DETECT_NUM_VALID_TRIES_MASK 0xf000000
+#define DC_I2C_DDC6_HW_STATUS__DC_I2C_DDC6_EDID_DETECT_NUM_VALID_TRIES__SHIFT 0x18
+#define DC_I2C_DDC6_HW_STATUS__DC_I2C_DDC6_EDID_DETECT_STATE_MASK 0x70000000
+#define DC_I2C_DDC6_HW_STATUS__DC_I2C_DDC6_EDID_DETECT_STATE__SHIFT 0x1c
+#define DC_I2C_DDC1_SPEED__DC_I2C_DDC1_THRESHOLD_MASK 0x3
+#define DC_I2C_DDC1_SPEED__DC_I2C_DDC1_THRESHOLD__SHIFT 0x0
+#define DC_I2C_DDC1_SPEED__DC_I2C_DDC1_DISABLE_FILTER_DURING_STALL_MASK 0x10
+#define DC_I2C_DDC1_SPEED__DC_I2C_DDC1_DISABLE_FILTER_DURING_STALL__SHIFT 0x4
+#define DC_I2C_DDC1_SPEED__DC_I2C_DDC1_START_STOP_TIMING_CNTL_MASK 0x300
+#define DC_I2C_DDC1_SPEED__DC_I2C_DDC1_START_STOP_TIMING_CNTL__SHIFT 0x8
+#define DC_I2C_DDC1_SPEED__DC_I2C_DDC1_PRESCALE_MASK 0xffff0000
+#define DC_I2C_DDC1_SPEED__DC_I2C_DDC1_PRESCALE__SHIFT 0x10
+#define DC_I2C_DDC1_SETUP__DC_I2C_DDC1_DATA_DRIVE_EN_MASK 0x1
+#define DC_I2C_DDC1_SETUP__DC_I2C_DDC1_DATA_DRIVE_EN__SHIFT 0x0
+#define DC_I2C_DDC1_SETUP__DC_I2C_DDC1_DATA_DRIVE_SEL_MASK 0x2
+#define DC_I2C_DDC1_SETUP__DC_I2C_DDC1_DATA_DRIVE_SEL__SHIFT 0x1
+#define DC_I2C_DDC1_SETUP__DC_I2C_DDC1_EDID_DETECT_ENABLE_MASK 0x10
+#define DC_I2C_DDC1_SETUP__DC_I2C_DDC1_EDID_DETECT_ENABLE__SHIFT 0x4
+#define DC_I2C_DDC1_SETUP__DC_I2C_DDC1_EDID_DETECT_MODE_MASK 0x20
+#define DC_I2C_DDC1_SETUP__DC_I2C_DDC1_EDID_DETECT_MODE__SHIFT 0x5
+#define DC_I2C_DDC1_SETUP__DC_I2C_DDC1_ENABLE_MASK 0x40
+#define DC_I2C_DDC1_SETUP__DC_I2C_DDC1_ENABLE__SHIFT 0x6
+#define DC_I2C_DDC1_SETUP__DC_I2C_DDC1_CLK_DRIVE_EN_MASK 0x80
+#define DC_I2C_DDC1_SETUP__DC_I2C_DDC1_CLK_DRIVE_EN__SHIFT 0x7
+#define DC_I2C_DDC1_SETUP__DC_I2C_DDC1_INTRA_BYTE_DELAY_MASK 0xff00
+#define DC_I2C_DDC1_SETUP__DC_I2C_DDC1_INTRA_BYTE_DELAY__SHIFT 0x8
+#define DC_I2C_DDC1_SETUP__DC_I2C_DDC1_INTRA_TRANSACTION_DELAY_MASK 0xff0000
+#define DC_I2C_DDC1_SETUP__DC_I2C_DDC1_INTRA_TRANSACTION_DELAY__SHIFT 0x10
+#define DC_I2C_DDC1_SETUP__DC_I2C_DDC1_TIME_LIMIT_MASK 0xff000000
+#define DC_I2C_DDC1_SETUP__DC_I2C_DDC1_TIME_LIMIT__SHIFT 0x18
+#define DC_I2C_DDC2_SPEED__DC_I2C_DDC2_THRESHOLD_MASK 0x3
+#define DC_I2C_DDC2_SPEED__DC_I2C_DDC2_THRESHOLD__SHIFT 0x0
+#define DC_I2C_DDC2_SPEED__DC_I2C_DDC2_DISABLE_FILTER_DURING_STALL_MASK 0x10
+#define DC_I2C_DDC2_SPEED__DC_I2C_DDC2_DISABLE_FILTER_DURING_STALL__SHIFT 0x4
+#define DC_I2C_DDC2_SPEED__DC_I2C_DDC2_START_STOP_TIMING_CNTL_MASK 0x300
+#define DC_I2C_DDC2_SPEED__DC_I2C_DDC2_START_STOP_TIMING_CNTL__SHIFT 0x8
+#define DC_I2C_DDC2_SPEED__DC_I2C_DDC2_PRESCALE_MASK 0xffff0000
+#define DC_I2C_DDC2_SPEED__DC_I2C_DDC2_PRESCALE__SHIFT 0x10
+#define DC_I2C_DDC2_SETUP__DC_I2C_DDC2_DATA_DRIVE_EN_MASK 0x1
+#define DC_I2C_DDC2_SETUP__DC_I2C_DDC2_DATA_DRIVE_EN__SHIFT 0x0
+#define DC_I2C_DDC2_SETUP__DC_I2C_DDC2_DATA_DRIVE_SEL_MASK 0x2
+#define DC_I2C_DDC2_SETUP__DC_I2C_DDC2_DATA_DRIVE_SEL__SHIFT 0x1
+#define DC_I2C_DDC2_SETUP__DC_I2C_DDC2_EDID_DETECT_ENABLE_MASK 0x10
+#define DC_I2C_DDC2_SETUP__DC_I2C_DDC2_EDID_DETECT_ENABLE__SHIFT 0x4
+#define DC_I2C_DDC2_SETUP__DC_I2C_DDC2_EDID_DETECT_MODE_MASK 0x20
+#define DC_I2C_DDC2_SETUP__DC_I2C_DDC2_EDID_DETECT_MODE__SHIFT 0x5
+#define DC_I2C_DDC2_SETUP__DC_I2C_DDC2_ENABLE_MASK 0x40
+#define DC_I2C_DDC2_SETUP__DC_I2C_DDC2_ENABLE__SHIFT 0x6
+#define DC_I2C_DDC2_SETUP__DC_I2C_DDC2_CLK_DRIVE_EN_MASK 0x80
+#define DC_I2C_DDC2_SETUP__DC_I2C_DDC2_CLK_DRIVE_EN__SHIFT 0x7
+#define DC_I2C_DDC2_SETUP__DC_I2C_DDC2_INTRA_BYTE_DELAY_MASK 0xff00
+#define DC_I2C_DDC2_SETUP__DC_I2C_DDC2_INTRA_BYTE_DELAY__SHIFT 0x8
+#define DC_I2C_DDC2_SETUP__DC_I2C_DDC2_INTRA_TRANSACTION_DELAY_MASK 0xff0000
+#define DC_I2C_DDC2_SETUP__DC_I2C_DDC2_INTRA_TRANSACTION_DELAY__SHIFT 0x10
+#define DC_I2C_DDC2_SETUP__DC_I2C_DDC2_TIME_LIMIT_MASK 0xff000000
+#define DC_I2C_DDC2_SETUP__DC_I2C_DDC2_TIME_LIMIT__SHIFT 0x18
+#define DC_I2C_DDC3_SPEED__DC_I2C_DDC3_THRESHOLD_MASK 0x3
+#define DC_I2C_DDC3_SPEED__DC_I2C_DDC3_THRESHOLD__SHIFT 0x0
+#define DC_I2C_DDC3_SPEED__DC_I2C_DDC3_DISABLE_FILTER_DURING_STALL_MASK 0x10
+#define DC_I2C_DDC3_SPEED__DC_I2C_DDC3_DISABLE_FILTER_DURING_STALL__SHIFT 0x4
+#define DC_I2C_DDC3_SPEED__DC_I2C_DDC3_START_STOP_TIMING_CNTL_MASK 0x300
+#define DC_I2C_DDC3_SPEED__DC_I2C_DDC3_START_STOP_TIMING_CNTL__SHIFT 0x8
+#define DC_I2C_DDC3_SPEED__DC_I2C_DDC3_PRESCALE_MASK 0xffff0000
+#define DC_I2C_DDC3_SPEED__DC_I2C_DDC3_PRESCALE__SHIFT 0x10
+#define DC_I2C_DDC3_SETUP__DC_I2C_DDC3_DATA_DRIVE_EN_MASK 0x1
+#define DC_I2C_DDC3_SETUP__DC_I2C_DDC3_DATA_DRIVE_EN__SHIFT 0x0
+#define DC_I2C_DDC3_SETUP__DC_I2C_DDC3_DATA_DRIVE_SEL_MASK 0x2
+#define DC_I2C_DDC3_SETUP__DC_I2C_DDC3_DATA_DRIVE_SEL__SHIFT 0x1
+#define DC_I2C_DDC3_SETUP__DC_I2C_DDC3_EDID_DETECT_ENABLE_MASK 0x10
+#define DC_I2C_DDC3_SETUP__DC_I2C_DDC3_EDID_DETECT_ENABLE__SHIFT 0x4
+#define DC_I2C_DDC3_SETUP__DC_I2C_DDC3_EDID_DETECT_MODE_MASK 0x20
+#define DC_I2C_DDC3_SETUP__DC_I2C_DDC3_EDID_DETECT_MODE__SHIFT 0x5
+#define DC_I2C_DDC3_SETUP__DC_I2C_DDC3_ENABLE_MASK 0x40
+#define DC_I2C_DDC3_SETUP__DC_I2C_DDC3_ENABLE__SHIFT 0x6
+#define DC_I2C_DDC3_SETUP__DC_I2C_DDC3_CLK_DRIVE_EN_MASK 0x80
+#define DC_I2C_DDC3_SETUP__DC_I2C_DDC3_CLK_DRIVE_EN__SHIFT 0x7
+#define DC_I2C_DDC3_SETUP__DC_I2C_DDC3_INTRA_BYTE_DELAY_MASK 0xff00
+#define DC_I2C_DDC3_SETUP__DC_I2C_DDC3_INTRA_BYTE_DELAY__SHIFT 0x8
+#define DC_I2C_DDC3_SETUP__DC_I2C_DDC3_INTRA_TRANSACTION_DELAY_MASK 0xff0000
+#define DC_I2C_DDC3_SETUP__DC_I2C_DDC3_INTRA_TRANSACTION_DELAY__SHIFT 0x10
+#define DC_I2C_DDC3_SETUP__DC_I2C_DDC3_TIME_LIMIT_MASK 0xff000000
+#define DC_I2C_DDC3_SETUP__DC_I2C_DDC3_TIME_LIMIT__SHIFT 0x18
+#define DC_I2C_DDC4_SPEED__DC_I2C_DDC4_THRESHOLD_MASK 0x3
+#define DC_I2C_DDC4_SPEED__DC_I2C_DDC4_THRESHOLD__SHIFT 0x0
+#define DC_I2C_DDC4_SPEED__DC_I2C_DDC4_DISABLE_FILTER_DURING_STALL_MASK 0x10
+#define DC_I2C_DDC4_SPEED__DC_I2C_DDC4_DISABLE_FILTER_DURING_STALL__SHIFT 0x4
+#define DC_I2C_DDC4_SPEED__DC_I2C_DDC4_START_STOP_TIMING_CNTL_MASK 0x300
+#define DC_I2C_DDC4_SPEED__DC_I2C_DDC4_START_STOP_TIMING_CNTL__SHIFT 0x8
+#define DC_I2C_DDC4_SPEED__DC_I2C_DDC4_PRESCALE_MASK 0xffff0000
+#define DC_I2C_DDC4_SPEED__DC_I2C_DDC4_PRESCALE__SHIFT 0x10
+#define DC_I2C_DDC4_SETUP__DC_I2C_DDC4_DATA_DRIVE_EN_MASK 0x1
+#define DC_I2C_DDC4_SETUP__DC_I2C_DDC4_DATA_DRIVE_EN__SHIFT 0x0
+#define DC_I2C_DDC4_SETUP__DC_I2C_DDC4_DATA_DRIVE_SEL_MASK 0x2
+#define DC_I2C_DDC4_SETUP__DC_I2C_DDC4_DATA_DRIVE_SEL__SHIFT 0x1
+#define DC_I2C_DDC4_SETUP__DC_I2C_DDC4_EDID_DETECT_ENABLE_MASK 0x10
+#define DC_I2C_DDC4_SETUP__DC_I2C_DDC4_EDID_DETECT_ENABLE__SHIFT 0x4
+#define DC_I2C_DDC4_SETUP__DC_I2C_DDC4_EDID_DETECT_MODE_MASK 0x20
+#define DC_I2C_DDC4_SETUP__DC_I2C_DDC4_EDID_DETECT_MODE__SHIFT 0x5
+#define DC_I2C_DDC4_SETUP__DC_I2C_DDC4_ENABLE_MASK 0x40
+#define DC_I2C_DDC4_SETUP__DC_I2C_DDC4_ENABLE__SHIFT 0x6
+#define DC_I2C_DDC4_SETUP__DC_I2C_DDC4_CLK_DRIVE_EN_MASK 0x80
+#define DC_I2C_DDC4_SETUP__DC_I2C_DDC4_CLK_DRIVE_EN__SHIFT 0x7
+#define DC_I2C_DDC4_SETUP__DC_I2C_DDC4_INTRA_BYTE_DELAY_MASK 0xff00
+#define DC_I2C_DDC4_SETUP__DC_I2C_DDC4_INTRA_BYTE_DELAY__SHIFT 0x8
+#define DC_I2C_DDC4_SETUP__DC_I2C_DDC4_INTRA_TRANSACTION_DELAY_MASK 0xff0000
+#define DC_I2C_DDC4_SETUP__DC_I2C_DDC4_INTRA_TRANSACTION_DELAY__SHIFT 0x10
+#define DC_I2C_DDC4_SETUP__DC_I2C_DDC4_TIME_LIMIT_MASK 0xff000000
+#define DC_I2C_DDC4_SETUP__DC_I2C_DDC4_TIME_LIMIT__SHIFT 0x18
+#define DC_I2C_DDC5_SPEED__DC_I2C_DDC5_THRESHOLD_MASK 0x3
+#define DC_I2C_DDC5_SPEED__DC_I2C_DDC5_THRESHOLD__SHIFT 0x0
+#define DC_I2C_DDC5_SPEED__DC_I2C_DDC5_DISABLE_FILTER_DURING_STALL_MASK 0x10
+#define DC_I2C_DDC5_SPEED__DC_I2C_DDC5_DISABLE_FILTER_DURING_STALL__SHIFT 0x4
+#define DC_I2C_DDC5_SPEED__DC_I2C_DDC5_START_STOP_TIMING_CNTL_MASK 0x300
+#define DC_I2C_DDC5_SPEED__DC_I2C_DDC5_START_STOP_TIMING_CNTL__SHIFT 0x8
+#define DC_I2C_DDC5_SPEED__DC_I2C_DDC5_PRESCALE_MASK 0xffff0000
+#define DC_I2C_DDC5_SPEED__DC_I2C_DDC5_PRESCALE__SHIFT 0x10
+#define DC_I2C_DDC5_SETUP__DC_I2C_DDC5_DATA_DRIVE_EN_MASK 0x1
+#define DC_I2C_DDC5_SETUP__DC_I2C_DDC5_DATA_DRIVE_EN__SHIFT 0x0
+#define DC_I2C_DDC5_SETUP__DC_I2C_DDC5_DATA_DRIVE_SEL_MASK 0x2
+#define DC_I2C_DDC5_SETUP__DC_I2C_DDC5_DATA_DRIVE_SEL__SHIFT 0x1
+#define DC_I2C_DDC5_SETUP__DC_I2C_DDC5_EDID_DETECT_ENABLE_MASK 0x10
+#define DC_I2C_DDC5_SETUP__DC_I2C_DDC5_EDID_DETECT_ENABLE__SHIFT 0x4
+#define DC_I2C_DDC5_SETUP__DC_I2C_DDC5_EDID_DETECT_MODE_MASK 0x20
+#define DC_I2C_DDC5_SETUP__DC_I2C_DDC5_EDID_DETECT_MODE__SHIFT 0x5
+#define DC_I2C_DDC5_SETUP__DC_I2C_DDC5_ENABLE_MASK 0x40
+#define DC_I2C_DDC5_SETUP__DC_I2C_DDC5_ENABLE__SHIFT 0x6
+#define DC_I2C_DDC5_SETUP__DC_I2C_DDC5_CLK_DRIVE_EN_MASK 0x80
+#define DC_I2C_DDC5_SETUP__DC_I2C_DDC5_CLK_DRIVE_EN__SHIFT 0x7
+#define DC_I2C_DDC5_SETUP__DC_I2C_DDC5_INTRA_BYTE_DELAY_MASK 0xff00
+#define DC_I2C_DDC5_SETUP__DC_I2C_DDC5_INTRA_BYTE_DELAY__SHIFT 0x8
+#define DC_I2C_DDC5_SETUP__DC_I2C_DDC5_INTRA_TRANSACTION_DELAY_MASK 0xff0000
+#define DC_I2C_DDC5_SETUP__DC_I2C_DDC5_INTRA_TRANSACTION_DELAY__SHIFT 0x10
+#define DC_I2C_DDC5_SETUP__DC_I2C_DDC5_TIME_LIMIT_MASK 0xff000000
+#define DC_I2C_DDC5_SETUP__DC_I2C_DDC5_TIME_LIMIT__SHIFT 0x18
+#define DC_I2C_DDC6_SPEED__DC_I2C_DDC6_THRESHOLD_MASK 0x3
+#define DC_I2C_DDC6_SPEED__DC_I2C_DDC6_THRESHOLD__SHIFT 0x0
+#define DC_I2C_DDC6_SPEED__DC_I2C_DDC6_DISABLE_FILTER_DURING_STALL_MASK 0x10
+#define DC_I2C_DDC6_SPEED__DC_I2C_DDC6_DISABLE_FILTER_DURING_STALL__SHIFT 0x4
+#define DC_I2C_DDC6_SPEED__DC_I2C_DDC6_START_STOP_TIMING_CNTL_MASK 0x300
+#define DC_I2C_DDC6_SPEED__DC_I2C_DDC6_START_STOP_TIMING_CNTL__SHIFT 0x8
+#define DC_I2C_DDC6_SPEED__DC_I2C_DDC6_PRESCALE_MASK 0xffff0000
+#define DC_I2C_DDC6_SPEED__DC_I2C_DDC6_PRESCALE__SHIFT 0x10
+#define DC_I2C_DDC6_SETUP__DC_I2C_DDC6_DATA_DRIVE_EN_MASK 0x1
+#define DC_I2C_DDC6_SETUP__DC_I2C_DDC6_DATA_DRIVE_EN__SHIFT 0x0
+#define DC_I2C_DDC6_SETUP__DC_I2C_DDC6_DATA_DRIVE_SEL_MASK 0x2
+#define DC_I2C_DDC6_SETUP__DC_I2C_DDC6_DATA_DRIVE_SEL__SHIFT 0x1
+#define DC_I2C_DDC6_SETUP__DC_I2C_DDC6_EDID_DETECT_ENABLE_MASK 0x10
+#define DC_I2C_DDC6_SETUP__DC_I2C_DDC6_EDID_DETECT_ENABLE__SHIFT 0x4
+#define DC_I2C_DDC6_SETUP__DC_I2C_DDC6_EDID_DETECT_MODE_MASK 0x20
+#define DC_I2C_DDC6_SETUP__DC_I2C_DDC6_EDID_DETECT_MODE__SHIFT 0x5
+#define DC_I2C_DDC6_SETUP__DC_I2C_DDC6_ENABLE_MASK 0x40
+#define DC_I2C_DDC6_SETUP__DC_I2C_DDC6_ENABLE__SHIFT 0x6
+#define DC_I2C_DDC6_SETUP__DC_I2C_DDC6_CLK_DRIVE_EN_MASK 0x80
+#define DC_I2C_DDC6_SETUP__DC_I2C_DDC6_CLK_DRIVE_EN__SHIFT 0x7
+#define DC_I2C_DDC6_SETUP__DC_I2C_DDC6_INTRA_BYTE_DELAY_MASK 0xff00
+#define DC_I2C_DDC6_SETUP__DC_I2C_DDC6_INTRA_BYTE_DELAY__SHIFT 0x8
+#define DC_I2C_DDC6_SETUP__DC_I2C_DDC6_INTRA_TRANSACTION_DELAY_MASK 0xff0000
+#define DC_I2C_DDC6_SETUP__DC_I2C_DDC6_INTRA_TRANSACTION_DELAY__SHIFT 0x10
+#define DC_I2C_DDC6_SETUP__DC_I2C_DDC6_TIME_LIMIT_MASK 0xff000000
+#define DC_I2C_DDC6_SETUP__DC_I2C_DDC6_TIME_LIMIT__SHIFT 0x18
+#define DC_I2C_TRANSACTION0__DC_I2C_RW0_MASK 0x1
+#define DC_I2C_TRANSACTION0__DC_I2C_RW0__SHIFT 0x0
+#define DC_I2C_TRANSACTION0__DC_I2C_STOP_ON_NACK0_MASK 0x100
+#define DC_I2C_TRANSACTION0__DC_I2C_STOP_ON_NACK0__SHIFT 0x8
+#define DC_I2C_TRANSACTION0__DC_I2C_START0_MASK 0x1000
+#define DC_I2C_TRANSACTION0__DC_I2C_START0__SHIFT 0xc
+#define DC_I2C_TRANSACTION0__DC_I2C_STOP0_MASK 0x2000
+#define DC_I2C_TRANSACTION0__DC_I2C_STOP0__SHIFT 0xd
+#define DC_I2C_TRANSACTION0__DC_I2C_COUNT0_MASK 0x3ff0000
+#define DC_I2C_TRANSACTION0__DC_I2C_COUNT0__SHIFT 0x10
+#define DC_I2C_TRANSACTION1__DC_I2C_RW1_MASK 0x1
+#define DC_I2C_TRANSACTION1__DC_I2C_RW1__SHIFT 0x0
+#define DC_I2C_TRANSACTION1__DC_I2C_STOP_ON_NACK1_MASK 0x100
+#define DC_I2C_TRANSACTION1__DC_I2C_STOP_ON_NACK1__SHIFT 0x8
+#define DC_I2C_TRANSACTION1__DC_I2C_START1_MASK 0x1000
+#define DC_I2C_TRANSACTION1__DC_I2C_START1__SHIFT 0xc
+#define DC_I2C_TRANSACTION1__DC_I2C_STOP1_MASK 0x2000
+#define DC_I2C_TRANSACTION1__DC_I2C_STOP1__SHIFT 0xd
+#define DC_I2C_TRANSACTION1__DC_I2C_COUNT1_MASK 0x3ff0000
+#define DC_I2C_TRANSACTION1__DC_I2C_COUNT1__SHIFT 0x10
+#define DC_I2C_TRANSACTION2__DC_I2C_RW2_MASK 0x1
+#define DC_I2C_TRANSACTION2__DC_I2C_RW2__SHIFT 0x0
+#define DC_I2C_TRANSACTION2__DC_I2C_STOP_ON_NACK2_MASK 0x100
+#define DC_I2C_TRANSACTION2__DC_I2C_STOP_ON_NACK2__SHIFT 0x8
+#define DC_I2C_TRANSACTION2__DC_I2C_START2_MASK 0x1000
+#define DC_I2C_TRANSACTION2__DC_I2C_START2__SHIFT 0xc
+#define DC_I2C_TRANSACTION2__DC_I2C_STOP2_MASK 0x2000
+#define DC_I2C_TRANSACTION2__DC_I2C_STOP2__SHIFT 0xd
+#define DC_I2C_TRANSACTION2__DC_I2C_COUNT2_MASK 0x3ff0000
+#define DC_I2C_TRANSACTION2__DC_I2C_COUNT2__SHIFT 0x10
+#define DC_I2C_TRANSACTION3__DC_I2C_RW3_MASK 0x1
+#define DC_I2C_TRANSACTION3__DC_I2C_RW3__SHIFT 0x0
+#define DC_I2C_TRANSACTION3__DC_I2C_STOP_ON_NACK3_MASK 0x100
+#define DC_I2C_TRANSACTION3__DC_I2C_STOP_ON_NACK3__SHIFT 0x8
+#define DC_I2C_TRANSACTION3__DC_I2C_START3_MASK 0x1000
+#define DC_I2C_TRANSACTION3__DC_I2C_START3__SHIFT 0xc
+#define DC_I2C_TRANSACTION3__DC_I2C_STOP3_MASK 0x2000
+#define DC_I2C_TRANSACTION3__DC_I2C_STOP3__SHIFT 0xd
+#define DC_I2C_TRANSACTION3__DC_I2C_COUNT3_MASK 0x3ff0000
+#define DC_I2C_TRANSACTION3__DC_I2C_COUNT3__SHIFT 0x10
+#define DC_I2C_DATA__DC_I2C_DATA_RW_MASK 0x1
+#define DC_I2C_DATA__DC_I2C_DATA_RW__SHIFT 0x0
+#define DC_I2C_DATA__DC_I2C_DATA_MASK 0xff00
+#define DC_I2C_DATA__DC_I2C_DATA__SHIFT 0x8
+#define DC_I2C_DATA__DC_I2C_INDEX_MASK 0x3ff0000
+#define DC_I2C_DATA__DC_I2C_INDEX__SHIFT 0x10
+#define DC_I2C_DATA__DC_I2C_INDEX_WRITE_MASK 0x80000000
+#define DC_I2C_DATA__DC_I2C_INDEX_WRITE__SHIFT 0x1f
+#define DC_I2C_DDCVGA_HW_STATUS__DC_I2C_DDCVGA_HW_STATUS_MASK 0x3
+#define DC_I2C_DDCVGA_HW_STATUS__DC_I2C_DDCVGA_HW_STATUS__SHIFT 0x0
+#define DC_I2C_DDCVGA_HW_STATUS__DC_I2C_DDCVGA_HW_DONE_MASK 0x8
+#define DC_I2C_DDCVGA_HW_STATUS__DC_I2C_DDCVGA_HW_DONE__SHIFT 0x3
+#define DC_I2C_DDCVGA_HW_STATUS__DC_I2C_DDCVGA_HW_REQ_MASK 0x10000
+#define DC_I2C_DDCVGA_HW_STATUS__DC_I2C_DDCVGA_HW_REQ__SHIFT 0x10
+#define DC_I2C_DDCVGA_HW_STATUS__DC_I2C_DDCVGA_HW_URG_MASK 0x20000
+#define DC_I2C_DDCVGA_HW_STATUS__DC_I2C_DDCVGA_HW_URG__SHIFT 0x11
+#define DC_I2C_DDCVGA_HW_STATUS__DC_I2C_DDCVGA_EDID_DETECT_STATUS_MASK 0x100000
+#define DC_I2C_DDCVGA_HW_STATUS__DC_I2C_DDCVGA_EDID_DETECT_STATUS__SHIFT 0x14
+#define DC_I2C_DDCVGA_HW_STATUS__DC_I2C_DDCVGA_EDID_DETECT_NUM_VALID_TRIES_MASK 0xf000000
+#define DC_I2C_DDCVGA_HW_STATUS__DC_I2C_DDCVGA_EDID_DETECT_NUM_VALID_TRIES__SHIFT 0x18
+#define DC_I2C_DDCVGA_HW_STATUS__DC_I2C_DDCVGA_EDID_DETECT_STATE_MASK 0x70000000
+#define DC_I2C_DDCVGA_HW_STATUS__DC_I2C_DDCVGA_EDID_DETECT_STATE__SHIFT 0x1c
+#define DC_I2C_DDCVGA_SPEED__DC_I2C_DDCVGA_THRESHOLD_MASK 0x3
+#define DC_I2C_DDCVGA_SPEED__DC_I2C_DDCVGA_THRESHOLD__SHIFT 0x0
+#define DC_I2C_DDCVGA_SPEED__DC_I2C_DDCVGA_DISABLE_FILTER_DURING_STALL_MASK 0x10
+#define DC_I2C_DDCVGA_SPEED__DC_I2C_DDCVGA_DISABLE_FILTER_DURING_STALL__SHIFT 0x4
+#define DC_I2C_DDCVGA_SPEED__DC_I2C_DDCVGA_START_STOP_TIMING_CNTL_MASK 0x300
+#define DC_I2C_DDCVGA_SPEED__DC_I2C_DDCVGA_START_STOP_TIMING_CNTL__SHIFT 0x8
+#define DC_I2C_DDCVGA_SPEED__DC_I2C_DDCVGA_PRESCALE_MASK 0xffff0000
+#define DC_I2C_DDCVGA_SPEED__DC_I2C_DDCVGA_PRESCALE__SHIFT 0x10
+#define DC_I2C_DDCVGA_SETUP__DC_I2C_DDCVGA_DATA_DRIVE_EN_MASK 0x1
+#define DC_I2C_DDCVGA_SETUP__DC_I2C_DDCVGA_DATA_DRIVE_EN__SHIFT 0x0
+#define DC_I2C_DDCVGA_SETUP__DC_I2C_DDCVGA_DATA_DRIVE_SEL_MASK 0x2
+#define DC_I2C_DDCVGA_SETUP__DC_I2C_DDCVGA_DATA_DRIVE_SEL__SHIFT 0x1
+#define DC_I2C_DDCVGA_SETUP__DC_I2C_DDCVGA_EDID_DETECT_ENABLE_MASK 0x10
+#define DC_I2C_DDCVGA_SETUP__DC_I2C_DDCVGA_EDID_DETECT_ENABLE__SHIFT 0x4
+#define DC_I2C_DDCVGA_SETUP__DC_I2C_DDCVGA_EDID_DETECT_MODE_MASK 0x20
+#define DC_I2C_DDCVGA_SETUP__DC_I2C_DDCVGA_EDID_DETECT_MODE__SHIFT 0x5
+#define DC_I2C_DDCVGA_SETUP__DC_I2C_DDCVGA_ENABLE_MASK 0x40
+#define DC_I2C_DDCVGA_SETUP__DC_I2C_DDCVGA_ENABLE__SHIFT 0x6
+#define DC_I2C_DDCVGA_SETUP__DC_I2C_DDCVGA_CLK_DRIVE_EN_MASK 0x80
+#define DC_I2C_DDCVGA_SETUP__DC_I2C_DDCVGA_CLK_DRIVE_EN__SHIFT 0x7
+#define DC_I2C_DDCVGA_SETUP__DC_I2C_DDCVGA_INTRA_BYTE_DELAY_MASK 0xff00
+#define DC_I2C_DDCVGA_SETUP__DC_I2C_DDCVGA_INTRA_BYTE_DELAY__SHIFT 0x8
+#define DC_I2C_DDCVGA_SETUP__DC_I2C_DDCVGA_INTRA_TRANSACTION_DELAY_MASK 0xff0000
+#define DC_I2C_DDCVGA_SETUP__DC_I2C_DDCVGA_INTRA_TRANSACTION_DELAY__SHIFT 0x10
+#define DC_I2C_DDCVGA_SETUP__DC_I2C_DDCVGA_TIME_LIMIT_MASK 0xff000000
+#define DC_I2C_DDCVGA_SETUP__DC_I2C_DDCVGA_TIME_LIMIT__SHIFT 0x18
+#define DC_I2C_EDID_DETECT_CTRL__DC_I2C_EDID_DETECT_WAIT_TIME_MASK 0xffff
+#define DC_I2C_EDID_DETECT_CTRL__DC_I2C_EDID_DETECT_WAIT_TIME__SHIFT 0x0
+#define DC_I2C_EDID_DETECT_CTRL__DC_I2C_EDID_DETECT_NUM_TRIES_UNTIL_VALID_MASK 0xf00000
+#define DC_I2C_EDID_DETECT_CTRL__DC_I2C_EDID_DETECT_NUM_TRIES_UNTIL_VALID__SHIFT 0x14
+#define DC_I2C_EDID_DETECT_CTRL__DC_I2C_EDID_DETECT_SEND_RESET_MASK 0x10000000
+#define DC_I2C_EDID_DETECT_CTRL__DC_I2C_EDID_DETECT_SEND_RESET__SHIFT 0x1c
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC1_READ_REQUEST_OCCURRED_MASK 0x1
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC1_READ_REQUEST_OCCURRED__SHIFT 0x0
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC1_READ_REQUEST_INT_MASK 0x2
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC1_READ_REQUEST_INT__SHIFT 0x1
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC1_READ_REQUEST_ACK_MASK 0x4
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC1_READ_REQUEST_ACK__SHIFT 0x2
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC1_READ_REQUEST_MASK_MASK 0x8
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC1_READ_REQUEST_MASK__SHIFT 0x3
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC2_READ_REQUEST_OCCURRED_MASK 0x10
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC2_READ_REQUEST_OCCURRED__SHIFT 0x4
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC2_READ_REQUEST_INT_MASK 0x20
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC2_READ_REQUEST_INT__SHIFT 0x5
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC2_READ_REQUEST_ACK_MASK 0x40
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC2_READ_REQUEST_ACK__SHIFT 0x6
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC2_READ_REQUEST_MASK_MASK 0x80
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC2_READ_REQUEST_MASK__SHIFT 0x7
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC3_READ_REQUEST_OCCURRED_MASK 0x100
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC3_READ_REQUEST_OCCURRED__SHIFT 0x8
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC3_READ_REQUEST_INT_MASK 0x200
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC3_READ_REQUEST_INT__SHIFT 0x9
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC3_READ_REQUEST_ACK_MASK 0x400
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC3_READ_REQUEST_ACK__SHIFT 0xa
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC3_READ_REQUEST_MASK_MASK 0x800
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC3_READ_REQUEST_MASK__SHIFT 0xb
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC4_READ_REQUEST_OCCURRED_MASK 0x1000
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC4_READ_REQUEST_OCCURRED__SHIFT 0xc
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC4_READ_REQUEST_INT_MASK 0x2000
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC4_READ_REQUEST_INT__SHIFT 0xd
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC4_READ_REQUEST_ACK_MASK 0x4000
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC4_READ_REQUEST_ACK__SHIFT 0xe
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC4_READ_REQUEST_MASK_MASK 0x8000
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC4_READ_REQUEST_MASK__SHIFT 0xf
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC5_READ_REQUEST_OCCURRED_MASK 0x10000
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC5_READ_REQUEST_OCCURRED__SHIFT 0x10
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC5_READ_REQUEST_INT_MASK 0x20000
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC5_READ_REQUEST_INT__SHIFT 0x11
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC5_READ_REQUEST_ACK_MASK 0x40000
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC5_READ_REQUEST_ACK__SHIFT 0x12
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC5_READ_REQUEST_MASK_MASK 0x80000
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC5_READ_REQUEST_MASK__SHIFT 0x13
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC6_READ_REQUEST_OCCURRED_MASK 0x100000
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC6_READ_REQUEST_OCCURRED__SHIFT 0x14
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC6_READ_REQUEST_INT_MASK 0x200000
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC6_READ_REQUEST_INT__SHIFT 0x15
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC6_READ_REQUEST_ACK_MASK 0x400000
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC6_READ_REQUEST_ACK__SHIFT 0x16
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC6_READ_REQUEST_MASK_MASK 0x800000
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC6_READ_REQUEST_MASK__SHIFT 0x17
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDCVGA_READ_REQUEST_OCCURRED_MASK 0x1000000
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDCVGA_READ_REQUEST_OCCURRED__SHIFT 0x18
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDCVGA_READ_REQUEST_INT_MASK 0x2000000
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDCVGA_READ_REQUEST_INT__SHIFT 0x19
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDCVGA_READ_REQUEST_ACK_MASK 0x4000000
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDCVGA_READ_REQUEST_ACK__SHIFT 0x1a
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDCVGA_READ_REQUEST_MASK_MASK 0x8000000
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDCVGA_READ_REQUEST_MASK__SHIFT 0x1b
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC_READ_REQUEST_ACK_ENABLE_MASK 0x40000000
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC_READ_REQUEST_ACK_ENABLE__SHIFT 0x1e
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC_READ_REQUEST_INT_TYPE_MASK 0x80000000
+#define DC_I2C_READ_REQUEST_INTERRUPT__DC_I2C_DDC_READ_REQUEST_INT_TYPE__SHIFT 0x1f
+#define GENERIC_I2C_CONTROL__GENERIC_I2C_GO_MASK 0x1
+#define GENERIC_I2C_CONTROL__GENERIC_I2C_GO__SHIFT 0x0
+#define GENERIC_I2C_CONTROL__GENERIC_I2C_SOFT_RESET_MASK 0x2
+#define GENERIC_I2C_CONTROL__GENERIC_I2C_SOFT_RESET__SHIFT 0x1
+#define GENERIC_I2C_CONTROL__GENERIC_I2C_SEND_RESET_MASK 0x4
+#define GENERIC_I2C_CONTROL__GENERIC_I2C_SEND_RESET__SHIFT 0x2
+#define GENERIC_I2C_CONTROL__GENERIC_I2C_ENABLE_MASK 0x8
+#define GENERIC_I2C_CONTROL__GENERIC_I2C_ENABLE__SHIFT 0x3
+#define GENERIC_I2C_CONTROL__GENERIC_I2C_DBG_REF_SEL_MASK 0x80000000
+#define GENERIC_I2C_CONTROL__GENERIC_I2C_DBG_REF_SEL__SHIFT 0x1f
+#define GENERIC_I2C_INTERRUPT_CONTROL__GENERIC_I2C_DONE_INT_MASK 0x1
+#define GENERIC_I2C_INTERRUPT_CONTROL__GENERIC_I2C_DONE_INT__SHIFT 0x0
+#define GENERIC_I2C_INTERRUPT_CONTROL__GENERIC_I2C_DONE_ACK_MASK 0x2
+#define GENERIC_I2C_INTERRUPT_CONTROL__GENERIC_I2C_DONE_ACK__SHIFT 0x1
+#define GENERIC_I2C_INTERRUPT_CONTROL__GENERIC_I2C_DONE_MASK_MASK 0x4
+#define GENERIC_I2C_INTERRUPT_CONTROL__GENERIC_I2C_DONE_MASK__SHIFT 0x2
+#define GENERIC_I2C_INTERRUPT_CONTROL__GENERIC_I2C_DDC_READ_REQUEST_OCCURRED_MASK 0x100
+#define GENERIC_I2C_INTERRUPT_CONTROL__GENERIC_I2C_DDC_READ_REQUEST_OCCURRED__SHIFT 0x8
+#define GENERIC_I2C_INTERRUPT_CONTROL__GENERIC_I2C_DDC_READ_REQUEST_INT_MASK 0x200
+#define GENERIC_I2C_INTERRUPT_CONTROL__GENERIC_I2C_DDC_READ_REQUEST_INT__SHIFT 0x9
+#define GENERIC_I2C_INTERRUPT_CONTROL__GENERIC_I2C_DDC_READ_REQUEST_ACK_MASK 0x400
+#define GENERIC_I2C_INTERRUPT_CONTROL__GENERIC_I2C_DDC_READ_REQUEST_ACK__SHIFT 0xa
+#define GENERIC_I2C_INTERRUPT_CONTROL__GENERIC_I2C_DDC_READ_REQUEST_MASK_MASK 0x800
+#define GENERIC_I2C_INTERRUPT_CONTROL__GENERIC_I2C_DDC_READ_REQUEST_MASK__SHIFT 0xb
+#define GENERIC_I2C_INTERRUPT_CONTROL__GENERIC_I2C_DDC_READ_REQUEST_INT_TYPE_MASK 0x1000
+#define GENERIC_I2C_INTERRUPT_CONTROL__GENERIC_I2C_DDC_READ_REQUEST_INT_TYPE__SHIFT 0xc
+#define GENERIC_I2C_STATUS__GENERIC_I2C_STATUS_MASK 0xf
+#define GENERIC_I2C_STATUS__GENERIC_I2C_STATUS__SHIFT 0x0
+#define GENERIC_I2C_STATUS__GENERIC_I2C_DONE_MASK 0x10
+#define GENERIC_I2C_STATUS__GENERIC_I2C_DONE__SHIFT 0x4
+#define GENERIC_I2C_STATUS__GENERIC_I2C_ABORTED_MASK 0x20
+#define GENERIC_I2C_STATUS__GENERIC_I2C_ABORTED__SHIFT 0x5
+#define GENERIC_I2C_STATUS__GENERIC_I2C_TIMEOUT_MASK 0x40
+#define GENERIC_I2C_STATUS__GENERIC_I2C_TIMEOUT__SHIFT 0x6
+#define GENERIC_I2C_STATUS__GENERIC_I2C_STOPPED_ON_NACK_MASK 0x200
+#define GENERIC_I2C_STATUS__GENERIC_I2C_STOPPED_ON_NACK__SHIFT 0x9
+#define GENERIC_I2C_STATUS__GENERIC_I2C_NACK_MASK 0x400
+#define GENERIC_I2C_STATUS__GENERIC_I2C_NACK__SHIFT 0xa
+#define GENERIC_I2C_SPEED__GENERIC_I2C_THRESHOLD_MASK 0x3
+#define GENERIC_I2C_SPEED__GENERIC_I2C_THRESHOLD__SHIFT 0x0
+#define GENERIC_I2C_SPEED__GENERIC_I2C_DISABLE_FILTER_DURING_STALL_MASK 0x10
+#define GENERIC_I2C_SPEED__GENERIC_I2C_DISABLE_FILTER_DURING_STALL__SHIFT 0x4
+#define GENERIC_I2C_SPEED__GENERIC_I2C_START_STOP_TIMING_CNTL_MASK 0x300
+#define GENERIC_I2C_SPEED__GENERIC_I2C_START_STOP_TIMING_CNTL__SHIFT 0x8
+#define GENERIC_I2C_SPEED__GENERIC_I2C_PRESCALE_MASK 0xffff0000
+#define GENERIC_I2C_SPEED__GENERIC_I2C_PRESCALE__SHIFT 0x10
+#define GENERIC_I2C_SETUP__GENERIC_I2C_DATA_DRIVE_EN_MASK 0x1
+#define GENERIC_I2C_SETUP__GENERIC_I2C_DATA_DRIVE_EN__SHIFT 0x0
+#define GENERIC_I2C_SETUP__GENERIC_I2C_DATA_DRIVE_SEL_MASK 0x2
+#define GENERIC_I2C_SETUP__GENERIC_I2C_DATA_DRIVE_SEL__SHIFT 0x1
+#define GENERIC_I2C_SETUP__GENERIC_I2C_CLK_DRIVE_EN_MASK 0x80
+#define GENERIC_I2C_SETUP__GENERIC_I2C_CLK_DRIVE_EN__SHIFT 0x7
+#define GENERIC_I2C_SETUP__GENERIC_I2C_INTRA_BYTE_DELAY_MASK 0xff00
+#define GENERIC_I2C_SETUP__GENERIC_I2C_INTRA_BYTE_DELAY__SHIFT 0x8
+#define GENERIC_I2C_SETUP__GENERIC_I2C_TIME_LIMIT_MASK 0xff000000
+#define GENERIC_I2C_SETUP__GENERIC_I2C_TIME_LIMIT__SHIFT 0x18
+#define GENERIC_I2C_TRANSACTION__GENERIC_I2C_RW_MASK 0x1
+#define GENERIC_I2C_TRANSACTION__GENERIC_I2C_RW__SHIFT 0x0
+#define GENERIC_I2C_TRANSACTION__GENERIC_I2C_STOP_ON_NACK_MASK 0x100
+#define GENERIC_I2C_TRANSACTION__GENERIC_I2C_STOP_ON_NACK__SHIFT 0x8
+#define GENERIC_I2C_TRANSACTION__GENERIC_I2C_ACK_ON_READ_MASK 0x200
+#define GENERIC_I2C_TRANSACTION__GENERIC_I2C_ACK_ON_READ__SHIFT 0x9
+#define GENERIC_I2C_TRANSACTION__GENERIC_I2C_START_MASK 0x1000
+#define GENERIC_I2C_TRANSACTION__GENERIC_I2C_START__SHIFT 0xc
+#define GENERIC_I2C_TRANSACTION__GENERIC_I2C_STOP_MASK 0x2000
+#define GENERIC_I2C_TRANSACTION__GENERIC_I2C_STOP__SHIFT 0xd
+#define GENERIC_I2C_TRANSACTION__GENERIC_I2C_COUNT_MASK 0xf0000
+#define GENERIC_I2C_TRANSACTION__GENERIC_I2C_COUNT__SHIFT 0x10
+#define GENERIC_I2C_DATA__GENERIC_I2C_DATA_RW_MASK 0x1
+#define GENERIC_I2C_DATA__GENERIC_I2C_DATA_RW__SHIFT 0x0
+#define GENERIC_I2C_DATA__GENERIC_I2C_DATA_MASK 0xff00
+#define GENERIC_I2C_DATA__GENERIC_I2C_DATA__SHIFT 0x8
+#define GENERIC_I2C_DATA__GENERIC_I2C_INDEX_MASK 0xf0000
+#define GENERIC_I2C_DATA__GENERIC_I2C_INDEX__SHIFT 0x10
+#define GENERIC_I2C_DATA__GENERIC_I2C_INDEX_WRITE_MASK 0x80000000
+#define GENERIC_I2C_DATA__GENERIC_I2C_INDEX_WRITE__SHIFT 0x1f
+#define GENERIC_I2C_PIN_SELECTION__GENERIC_I2C_SCL_PIN_SEL_MASK 0x7f
+#define GENERIC_I2C_PIN_SELECTION__GENERIC_I2C_SCL_PIN_SEL__SHIFT 0x0
+#define GENERIC_I2C_PIN_SELECTION__GENERIC_I2C_SDA_PIN_SEL_MASK 0x7f00
+#define GENERIC_I2C_PIN_SELECTION__GENERIC_I2C_SDA_PIN_SEL__SHIFT 0x8
+#define GENERIC_I2C_PIN_DEBUG__GENERIC_I2C_SCL_OUTPUT_MASK 0x1
+#define GENERIC_I2C_PIN_DEBUG__GENERIC_I2C_SCL_OUTPUT__SHIFT 0x0
+#define GENERIC_I2C_PIN_DEBUG__GENERIC_I2C_SCL_INPUT_MASK 0x2
+#define GENERIC_I2C_PIN_DEBUG__GENERIC_I2C_SCL_INPUT__SHIFT 0x1
+#define GENERIC_I2C_PIN_DEBUG__GENERIC_I2C_SCL_EN_MASK 0x4
+#define GENERIC_I2C_PIN_DEBUG__GENERIC_I2C_SCL_EN__SHIFT 0x2
+#define GENERIC_I2C_PIN_DEBUG__GENERIC_I2C_SDA_OUTPUT_MASK 0x10
+#define GENERIC_I2C_PIN_DEBUG__GENERIC_I2C_SDA_OUTPUT__SHIFT 0x4
+#define GENERIC_I2C_PIN_DEBUG__GENERIC_I2C_SDA_INPUT_MASK 0x20
+#define GENERIC_I2C_PIN_DEBUG__GENERIC_I2C_SDA_INPUT__SHIFT 0x5
+#define GENERIC_I2C_PIN_DEBUG__GENERIC_I2C_SDA_EN_MASK 0x40
+#define GENERIC_I2C_PIN_DEBUG__GENERIC_I2C_SDA_EN__SHIFT 0x6
+#define BLNDV_CONTROL__BLND_GLOBAL_GAIN_MASK 0xff
+#define BLNDV_CONTROL__BLND_GLOBAL_GAIN__SHIFT 0x0
+#define BLNDV_CONTROL__BLND_MODE_MASK 0x300
+#define BLNDV_CONTROL__BLND_MODE__SHIFT 0x8
+#define BLNDV_CONTROL__BLND_STEREO_TYPE_MASK 0xc00
+#define BLNDV_CONTROL__BLND_STEREO_TYPE__SHIFT 0xa
+#define BLNDV_CONTROL__BLND_STEREO_POLARITY_MASK 0x1000
+#define BLNDV_CONTROL__BLND_STEREO_POLARITY__SHIFT 0xc
+#define BLNDV_CONTROL__BLND_FEEDTHROUGH_EN_MASK 0x2000
+#define BLNDV_CONTROL__BLND_FEEDTHROUGH_EN__SHIFT 0xd
+#define BLNDV_CONTROL__BLND_ALPHA_MODE_MASK 0x30000
+#define BLNDV_CONTROL__BLND_ALPHA_MODE__SHIFT 0x10
+#define BLNDV_CONTROL__BLND_ACTIVE_OVERLAP_ONLY_MASK 0x40000
+#define BLNDV_CONTROL__BLND_ACTIVE_OVERLAP_ONLY__SHIFT 0x12
+#define BLNDV_CONTROL__BLND_MULTIPLIED_MODE_MASK 0x100000
+#define BLNDV_CONTROL__BLND_MULTIPLIED_MODE__SHIFT 0x14
+#define BLNDV_CONTROL__BLND_GLOBAL_ALPHA_MASK 0xff000000
+#define BLNDV_CONTROL__BLND_GLOBAL_ALPHA__SHIFT 0x18
+#define BLNDV_SM_CONTROL2__SM_MODE_MASK 0x7
+#define BLNDV_SM_CONTROL2__SM_MODE__SHIFT 0x0
+#define BLNDV_SM_CONTROL2__SM_FRAME_ALTERNATE_MASK 0x10
+#define BLNDV_SM_CONTROL2__SM_FRAME_ALTERNATE__SHIFT 0x4
+#define BLNDV_SM_CONTROL2__SM_FIELD_ALTERNATE_MASK 0x20
+#define BLNDV_SM_CONTROL2__SM_FIELD_ALTERNATE__SHIFT 0x5
+#define BLNDV_SM_CONTROL2__SM_FORCE_NEXT_FRAME_POL_MASK 0x300
+#define BLNDV_SM_CONTROL2__SM_FORCE_NEXT_FRAME_POL__SHIFT 0x8
+#define BLNDV_SM_CONTROL2__SM_FORCE_NEXT_TOP_POL_MASK 0x30000
+#define BLNDV_SM_CONTROL2__SM_FORCE_NEXT_TOP_POL__SHIFT 0x10
+#define BLNDV_SM_CONTROL2__SM_CURRENT_FRAME_POL_MASK 0x1000000
+#define BLNDV_SM_CONTROL2__SM_CURRENT_FRAME_POL__SHIFT 0x18
+#define BLNDV_CONTROL2__PTI_ENABLE_MASK 0x1
+#define BLNDV_CONTROL2__PTI_ENABLE__SHIFT 0x0
+#define BLNDV_CONTROL2__PTI_NEW_PIXEL_GAP_MASK 0x30
+#define BLNDV_CONTROL2__PTI_NEW_PIXEL_GAP__SHIFT 0x4
+#define BLNDV_CONTROL2__BLND_NEW_PIXEL_MODE_MASK 0x40
+#define BLNDV_CONTROL2__BLND_NEW_PIXEL_MODE__SHIFT 0x6
+#define BLNDV_CONTROL2__BLND_SUPERAA_DEGAMMA_EN_MASK 0x80
+#define BLNDV_CONTROL2__BLND_SUPERAA_DEGAMMA_EN__SHIFT 0x7
+#define BLNDV_CONTROL2__BLND_SUPERAA_REGAMMA_EN_MASK 0x100
+#define BLNDV_CONTROL2__BLND_SUPERAA_REGAMMA_EN__SHIFT 0x8
+#define BLNDV_UPDATE__BLND_UPDATE_PENDING_MASK 0x1
+#define BLNDV_UPDATE__BLND_UPDATE_PENDING__SHIFT 0x0
+#define BLNDV_UPDATE__BLND_UPDATE_TAKEN_MASK 0x100
+#define BLNDV_UPDATE__BLND_UPDATE_TAKEN__SHIFT 0x8
+#define BLNDV_UPDATE__BLND_UPDATE_LOCK_MASK 0x10000
+#define BLNDV_UPDATE__BLND_UPDATE_LOCK__SHIFT 0x10
+#define BLNDV_UNDERFLOW_INTERRUPT__BLND_UNDERFLOW_INT_OCCURED_MASK 0x1
+#define BLNDV_UNDERFLOW_INTERRUPT__BLND_UNDERFLOW_INT_OCCURED__SHIFT 0x0
+#define BLNDV_UNDERFLOW_INTERRUPT__BLND_UNDERFLOW_INT_ACK_MASK 0x100
+#define BLNDV_UNDERFLOW_INTERRUPT__BLND_UNDERFLOW_INT_ACK__SHIFT 0x8
+#define BLNDV_UNDERFLOW_INTERRUPT__BLND_UNDERFLOW_INT_MASK_MASK 0x1000
+#define BLNDV_UNDERFLOW_INTERRUPT__BLND_UNDERFLOW_INT_MASK__SHIFT 0xc
+#define BLNDV_UNDERFLOW_INTERRUPT__BLND_UNDERFLOW_INT_PIPE_INDEX_MASK 0x30000
+#define BLNDV_UNDERFLOW_INTERRUPT__BLND_UNDERFLOW_INT_PIPE_INDEX__SHIFT 0x10
+#define BLNDV_V_UPDATE_LOCK__BLND_DCP_GRPH_V_UPDATE_LOCK_MASK 0x1
+#define BLNDV_V_UPDATE_LOCK__BLND_DCP_GRPH_V_UPDATE_LOCK__SHIFT 0x0
+#define BLNDV_V_UPDATE_LOCK__BLND_DCP_GRPH_SURF_V_UPDATE_LOCK_MASK 0x2
+#define BLNDV_V_UPDATE_LOCK__BLND_DCP_GRPH_SURF_V_UPDATE_LOCK__SHIFT 0x1
+#define BLNDV_V_UPDATE_LOCK__BLND_DCP_CUR_V_UPDATE_LOCK_MASK 0x10000
+#define BLNDV_V_UPDATE_LOCK__BLND_DCP_CUR_V_UPDATE_LOCK__SHIFT 0x10
+#define BLNDV_V_UPDATE_LOCK__BLND_DCP_CUR2_V_UPDATE_LOCK_MASK 0x1000000
+#define BLNDV_V_UPDATE_LOCK__BLND_DCP_CUR2_V_UPDATE_LOCK__SHIFT 0x18
+#define BLNDV_V_UPDATE_LOCK__BLND_SCL_V_UPDATE_LOCK_MASK 0x10000000
+#define BLNDV_V_UPDATE_LOCK__BLND_SCL_V_UPDATE_LOCK__SHIFT 0x1c
+#define BLNDV_V_UPDATE_LOCK__BLND_BLND_V_UPDATE_LOCK_MASK 0x20000000
+#define BLNDV_V_UPDATE_LOCK__BLND_BLND_V_UPDATE_LOCK__SHIFT 0x1d
+#define BLNDV_V_UPDATE_LOCK__BLND_V_UPDATE_LOCK_MODE_MASK 0x80000000
+#define BLNDV_V_UPDATE_LOCK__BLND_V_UPDATE_LOCK_MODE__SHIFT 0x1f
+#define BLNDV_REG_UPDATE_STATUS__DCP_BLNDC_GRPH_UPDATE_PENDING_MASK 0x1
+#define BLNDV_REG_UPDATE_STATUS__DCP_BLNDC_GRPH_UPDATE_PENDING__SHIFT 0x0
+#define BLNDV_REG_UPDATE_STATUS__DCP_BLNDO_GRPH_UPDATE_PENDING_MASK 0x2
+#define BLNDV_REG_UPDATE_STATUS__DCP_BLNDO_GRPH_UPDATE_PENDING__SHIFT 0x1
+#define BLNDV_REG_UPDATE_STATUS__DCP_BLNDC_GRPH_SURF_UPDATE_PENDING_MASK 0x4
+#define BLNDV_REG_UPDATE_STATUS__DCP_BLNDC_GRPH_SURF_UPDATE_PENDING__SHIFT 0x2
+#define BLNDV_REG_UPDATE_STATUS__DCP_BLNDO_GRPH_SURF_UPDATE_PENDING_MASK 0x8
+#define BLNDV_REG_UPDATE_STATUS__DCP_BLNDO_GRPH_SURF_UPDATE_PENDING__SHIFT 0x3
+#define BLNDV_REG_UPDATE_STATUS__DCP_BLNDC_CUR_UPDATE_PENDING_MASK 0x40
+#define BLNDV_REG_UPDATE_STATUS__DCP_BLNDC_CUR_UPDATE_PENDING__SHIFT 0x6
+#define BLNDV_REG_UPDATE_STATUS__DCP_BLNDO_CUR_UPDATE_PENDING_MASK 0x80
+#define BLNDV_REG_UPDATE_STATUS__DCP_BLNDO_CUR_UPDATE_PENDING__SHIFT 0x7
+#define BLNDV_REG_UPDATE_STATUS__SCL_BLNDC_UPDATE_PENDING_MASK 0x100
+#define BLNDV_REG_UPDATE_STATUS__SCL_BLNDC_UPDATE_PENDING__SHIFT 0x8
+#define BLNDV_REG_UPDATE_STATUS__SCL_BLNDO_UPDATE_PENDING_MASK 0x200
+#define BLNDV_REG_UPDATE_STATUS__SCL_BLNDO_UPDATE_PENDING__SHIFT 0x9
+#define BLNDV_REG_UPDATE_STATUS__BLND_BLNDC_UPDATE_PENDING_MASK 0x400
+#define BLNDV_REG_UPDATE_STATUS__BLND_BLNDC_UPDATE_PENDING__SHIFT 0xa
+#define BLNDV_REG_UPDATE_STATUS__BLND_BLNDO_UPDATE_PENDING_MASK 0x800
+#define BLNDV_REG_UPDATE_STATUS__BLND_BLNDO_UPDATE_PENDING__SHIFT 0xb
+#define BLNDV_DEBUG__BLND_CNV_MUX_SELECT_MASK 0x1
+#define BLNDV_DEBUG__BLND_CNV_MUX_SELECT__SHIFT 0x0
+#define BLNDV_DEBUG__BLND_DEBUG_MASK 0xfffffffe
+#define BLNDV_DEBUG__BLND_DEBUG__SHIFT 0x1
+#define BLNDV_TEST_DEBUG_INDEX__BLND_TEST_DEBUG_INDEX_MASK 0xff
+#define BLNDV_TEST_DEBUG_INDEX__BLND_TEST_DEBUG_INDEX__SHIFT 0x0
+#define BLNDV_TEST_DEBUG_INDEX__BLND_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define BLNDV_TEST_DEBUG_INDEX__BLND_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define BLNDV_TEST_DEBUG_DATA__BLND_TEST_DEBUG_DATA_MASK 0xffffffff
+#define BLNDV_TEST_DEBUG_DATA__BLND_TEST_DEBUG_DATA__SHIFT 0x0
+#define CRTCV_H_TOTAL__CRTC_H_TOTAL_MASK 0x3fff
+#define CRTCV_H_TOTAL__CRTC_H_TOTAL__SHIFT 0x0
+#define CRTCV_H_BLANK_START_END__CRTC_H_BLANK_START_MASK 0x3fff
+#define CRTCV_H_BLANK_START_END__CRTC_H_BLANK_START__SHIFT 0x0
+#define CRTCV_H_BLANK_START_END__CRTC_H_BLANK_END_MASK 0x3fff0000
+#define CRTCV_H_BLANK_START_END__CRTC_H_BLANK_END__SHIFT 0x10
+#define CRTCV_H_SYNC_A__CRTC_H_SYNC_A_START_MASK 0x3fff
+#define CRTCV_H_SYNC_A__CRTC_H_SYNC_A_START__SHIFT 0x0
+#define CRTCV_H_SYNC_A__CRTC_H_SYNC_A_END_MASK 0x3fff0000
+#define CRTCV_H_SYNC_A__CRTC_H_SYNC_A_END__SHIFT 0x10
+#define CRTCV_V_TOTAL__CRTC_V_TOTAL_MASK 0x3fff
+#define CRTCV_V_TOTAL__CRTC_V_TOTAL__SHIFT 0x0
+#define CRTCV_V_BLANK_START_END__CRTC_V_BLANK_START_MASK 0x3fff
+#define CRTCV_V_BLANK_START_END__CRTC_V_BLANK_START__SHIFT 0x0
+#define CRTCV_V_BLANK_START_END__CRTC_V_BLANK_END_MASK 0x3fff0000
+#define CRTCV_V_BLANK_START_END__CRTC_V_BLANK_END__SHIFT 0x10
+#define CRTCV_V_SYNC_A__CRTC_V_SYNC_A_START_MASK 0x3fff
+#define CRTCV_V_SYNC_A__CRTC_V_SYNC_A_START__SHIFT 0x0
+#define CRTCV_V_SYNC_A__CRTC_V_SYNC_A_END_MASK 0x3fff0000
+#define CRTCV_V_SYNC_A__CRTC_V_SYNC_A_END__SHIFT 0x10
+#define CRTCV_CONTROL__CRTC_MASTER_EN_MASK 0x1
+#define CRTCV_CONTROL__CRTC_MASTER_EN__SHIFT 0x0
+#define CRTCV_CONTROL__CRTC_SYNC_RESET_SEL_MASK 0x10
+#define CRTCV_CONTROL__CRTC_SYNC_RESET_SEL__SHIFT 0x4
+#define CRTCV_CONTROL__CRTC_DISABLE_POINT_CNTL_MASK 0x300
+#define CRTCV_CONTROL__CRTC_DISABLE_POINT_CNTL__SHIFT 0x8
+#define CRTCV_CONTROL__CRTC_START_POINT_CNTL_MASK 0x1000
+#define CRTCV_CONTROL__CRTC_START_POINT_CNTL__SHIFT 0xc
+#define CRTCV_CONTROL__CRTC_FIELD_NUMBER_CNTL_MASK 0x2000
+#define CRTCV_CONTROL__CRTC_FIELD_NUMBER_CNTL__SHIFT 0xd
+#define CRTCV_CONTROL__CRTC_FIELD_NUMBER_POLARITY_MASK 0x4000
+#define CRTCV_CONTROL__CRTC_FIELD_NUMBER_POLARITY__SHIFT 0xe
+#define CRTCV_CONTROL__CRTC_CURRENT_MASTER_EN_STATE_MASK 0x10000
+#define CRTCV_CONTROL__CRTC_CURRENT_MASTER_EN_STATE__SHIFT 0x10
+#define CRTCV_CONTROL__CRTC_HBLANK_EARLY_CONTROL_MASK 0x700000
+#define CRTCV_CONTROL__CRTC_HBLANK_EARLY_CONTROL__SHIFT 0x14
+#define CRTCV_CONTROL__CRTC_DISP_READ_REQUEST_DISABLE_MASK 0x1000000
+#define CRTCV_CONTROL__CRTC_DISP_READ_REQUEST_DISABLE__SHIFT 0x18
+#define CRTCV_CONTROL__CRTC_SOF_PULL_EN_MASK 0x20000000
+#define CRTCV_CONTROL__CRTC_SOF_PULL_EN__SHIFT 0x1d
+#define CRTCV_CONTROL__CRTC_AVSYNC_LOCK_SNAPSHOT_MASK 0x40000000
+#define CRTCV_CONTROL__CRTC_AVSYNC_LOCK_SNAPSHOT__SHIFT 0x1e
+#define CRTCV_CONTROL__CRTC_AVSYNC_VSYNC_N_HSYNC_MODE_MASK 0x80000000
+#define CRTCV_CONTROL__CRTC_AVSYNC_VSYNC_N_HSYNC_MODE__SHIFT 0x1f
+#define CRTCV_START_LINE_CONTROL__CRTC_PROGRESSIVE_START_LINE_EARLY_MASK 0x1
+#define CRTCV_START_LINE_CONTROL__CRTC_PROGRESSIVE_START_LINE_EARLY__SHIFT 0x0
+#define CRTCV_START_LINE_CONTROL__CRTC_INTERLACE_START_LINE_EARLY_MASK 0x2
+#define CRTCV_START_LINE_CONTROL__CRTC_INTERLACE_START_LINE_EARLY__SHIFT 0x1
+#define CRTCV_START_LINE_CONTROL__CRTC_PREFETCH_EN_MASK 0x4
+#define CRTCV_START_LINE_CONTROL__CRTC_PREFETCH_EN__SHIFT 0x2
+#define CRTCV_START_LINE_CONTROL__CRTC_LEGACY_REQUESTOR_EN_MASK 0x100
+#define CRTCV_START_LINE_CONTROL__CRTC_LEGACY_REQUESTOR_EN__SHIFT 0x8
+#define CRTCV_START_LINE_CONTROL__CRTC_ADVANCED_START_LINE_POSITION_MASK 0xff000
+#define CRTCV_START_LINE_CONTROL__CRTC_ADVANCED_START_LINE_POSITION__SHIFT 0xc
+#define CRTCV_OVERSCAN_COLOR__CRTC_OVERSCAN_COLOR_BLUE_MASK 0x3ff
+#define CRTCV_OVERSCAN_COLOR__CRTC_OVERSCAN_COLOR_BLUE__SHIFT 0x0
+#define CRTCV_OVERSCAN_COLOR__CRTC_OVERSCAN_COLOR_GREEN_MASK 0xffc00
+#define CRTCV_OVERSCAN_COLOR__CRTC_OVERSCAN_COLOR_GREEN__SHIFT 0xa
+#define CRTCV_OVERSCAN_COLOR__CRTC_OVERSCAN_COLOR_RED_MASK 0x3ff00000
+#define CRTCV_OVERSCAN_COLOR__CRTC_OVERSCAN_COLOR_RED__SHIFT 0x14
+#define CRTCV_OVERSCAN_COLOR_EXT__CRTC_OVERSCAN_COLOR_BLUE_EXT_MASK 0x3
+#define CRTCV_OVERSCAN_COLOR_EXT__CRTC_OVERSCAN_COLOR_BLUE_EXT__SHIFT 0x0
+#define CRTCV_OVERSCAN_COLOR_EXT__CRTC_OVERSCAN_COLOR_GREEN_EXT_MASK 0x300
+#define CRTCV_OVERSCAN_COLOR_EXT__CRTC_OVERSCAN_COLOR_GREEN_EXT__SHIFT 0x8
+#define CRTCV_OVERSCAN_COLOR_EXT__CRTC_OVERSCAN_COLOR_RED_EXT_MASK 0x30000
+#define CRTCV_OVERSCAN_COLOR_EXT__CRTC_OVERSCAN_COLOR_RED_EXT__SHIFT 0x10
+#define CRTCV_BLACK_COLOR__CRTC_BLACK_COLOR_B_CB_MASK 0x3ff
+#define CRTCV_BLACK_COLOR__CRTC_BLACK_COLOR_B_CB__SHIFT 0x0
+#define CRTCV_BLACK_COLOR__CRTC_BLACK_COLOR_G_Y_MASK 0xffc00
+#define CRTCV_BLACK_COLOR__CRTC_BLACK_COLOR_G_Y__SHIFT 0xa
+#define CRTCV_BLACK_COLOR__CRTC_BLACK_COLOR_R_CR_MASK 0x3ff00000
+#define CRTCV_BLACK_COLOR__CRTC_BLACK_COLOR_R_CR__SHIFT 0x14
+#define CRTCV_BLACK_COLOR_EXT__CRTC_BLACK_COLOR_B_CB_EXT_MASK 0x3
+#define CRTCV_BLACK_COLOR_EXT__CRTC_BLACK_COLOR_B_CB_EXT__SHIFT 0x0
+#define CRTCV_BLACK_COLOR_EXT__CRTC_BLACK_COLOR_G_Y_EXT_MASK 0x300
+#define CRTCV_BLACK_COLOR_EXT__CRTC_BLACK_COLOR_G_Y_EXT__SHIFT 0x8
+#define CRTCV_BLACK_COLOR_EXT__CRTC_BLACK_COLOR_R_CR_EXT_MASK 0x30000
+#define CRTCV_BLACK_COLOR_EXT__CRTC_BLACK_COLOR_R_CR_EXT__SHIFT 0x10
+#define CRTCV_CRC_CNTL__CRTC_CRC_EN_MASK 0x1
+#define CRTCV_CRC_CNTL__CRTC_CRC_EN__SHIFT 0x0
+#define CRTCV_CRC_CNTL__CRTC_CRC_CONT_EN_MASK 0x10
+#define CRTCV_CRC_CNTL__CRTC_CRC_CONT_EN__SHIFT 0x4
+#define CRTCV_CRC_CNTL__CRTC_CRC_STEREO_MODE_MASK 0x300
+#define CRTCV_CRC_CNTL__CRTC_CRC_STEREO_MODE__SHIFT 0x8
+#define CRTCV_CRC_CNTL__CRTC_CRC_INTERLACE_MODE_MASK 0x3000
+#define CRTCV_CRC_CNTL__CRTC_CRC_INTERLACE_MODE__SHIFT 0xc
+#define CRTCV_CRC_CNTL__CRTC_CRC_USE_NEW_AND_REPEATED_PIXELS_MASK 0x10000
+#define CRTCV_CRC_CNTL__CRTC_CRC_USE_NEW_AND_REPEATED_PIXELS__SHIFT 0x10
+#define CRTCV_CRC_CNTL__CRTC_CRC0_SELECT_MASK 0x700000
+#define CRTCV_CRC_CNTL__CRTC_CRC0_SELECT__SHIFT 0x14
+#define CRTCV_CRC_CNTL__CRTC_CRC1_SELECT_MASK 0x7000000
+#define CRTCV_CRC_CNTL__CRTC_CRC1_SELECT__SHIFT 0x18
+#define CRTCV_CRC0_WINDOWA_X_CONTROL__CRTC_CRC0_WINDOWA_X_START_MASK 0x3fff
+#define CRTCV_CRC0_WINDOWA_X_CONTROL__CRTC_CRC0_WINDOWA_X_START__SHIFT 0x0
+#define CRTCV_CRC0_WINDOWA_X_CONTROL__CRTC_CRC0_WINDOWA_X_END_MASK 0x3fff0000
+#define CRTCV_CRC0_WINDOWA_X_CONTROL__CRTC_CRC0_WINDOWA_X_END__SHIFT 0x10
+#define CRTCV_CRC0_WINDOWA_Y_CONTROL__CRTC_CRC0_WINDOWA_Y_START_MASK 0x3fff
+#define CRTCV_CRC0_WINDOWA_Y_CONTROL__CRTC_CRC0_WINDOWA_Y_START__SHIFT 0x0
+#define CRTCV_CRC0_WINDOWA_Y_CONTROL__CRTC_CRC0_WINDOWA_Y_END_MASK 0x3fff0000
+#define CRTCV_CRC0_WINDOWA_Y_CONTROL__CRTC_CRC0_WINDOWA_Y_END__SHIFT 0x10
+#define CRTCV_CRC0_WINDOWB_X_CONTROL__CRTC_CRC0_WINDOWB_X_START_MASK 0x3fff
+#define CRTCV_CRC0_WINDOWB_X_CONTROL__CRTC_CRC0_WINDOWB_X_START__SHIFT 0x0
+#define CRTCV_CRC0_WINDOWB_X_CONTROL__CRTC_CRC0_WINDOWB_X_END_MASK 0x3fff0000
+#define CRTCV_CRC0_WINDOWB_X_CONTROL__CRTC_CRC0_WINDOWB_X_END__SHIFT 0x10
+#define CRTCV_CRC0_WINDOWB_Y_CONTROL__CRTC_CRC0_WINDOWB_Y_START_MASK 0x3fff
+#define CRTCV_CRC0_WINDOWB_Y_CONTROL__CRTC_CRC0_WINDOWB_Y_START__SHIFT 0x0
+#define CRTCV_CRC0_WINDOWB_Y_CONTROL__CRTC_CRC0_WINDOWB_Y_END_MASK 0x3fff0000
+#define CRTCV_CRC0_WINDOWB_Y_CONTROL__CRTC_CRC0_WINDOWB_Y_END__SHIFT 0x10
+#define CRTCV_CRC0_DATA_RG__CRC0_R_CR_MASK 0xffff
+#define CRTCV_CRC0_DATA_RG__CRC0_R_CR__SHIFT 0x0
+#define CRTCV_CRC0_DATA_RG__CRC0_G_Y_MASK 0xffff0000
+#define CRTCV_CRC0_DATA_RG__CRC0_G_Y__SHIFT 0x10
+#define CRTCV_CRC0_DATA_B__CRC0_B_CB_MASK 0xffff
+#define CRTCV_CRC0_DATA_B__CRC0_B_CB__SHIFT 0x0
+#define CRTCV_CRC1_WINDOWA_X_CONTROL__CRTC_CRC1_WINDOWA_X_START_MASK 0x3fff
+#define CRTCV_CRC1_WINDOWA_X_CONTROL__CRTC_CRC1_WINDOWA_X_START__SHIFT 0x0
+#define CRTCV_CRC1_WINDOWA_X_CONTROL__CRTC_CRC1_WINDOWA_X_END_MASK 0x3fff0000
+#define CRTCV_CRC1_WINDOWA_X_CONTROL__CRTC_CRC1_WINDOWA_X_END__SHIFT 0x10
+#define CRTCV_CRC1_WINDOWA_Y_CONTROL__CRTC_CRC1_WINDOWA_Y_START_MASK 0x3fff
+#define CRTCV_CRC1_WINDOWA_Y_CONTROL__CRTC_CRC1_WINDOWA_Y_START__SHIFT 0x0
+#define CRTCV_CRC1_WINDOWA_Y_CONTROL__CRTC_CRC1_WINDOWA_Y_END_MASK 0x3fff0000
+#define CRTCV_CRC1_WINDOWA_Y_CONTROL__CRTC_CRC1_WINDOWA_Y_END__SHIFT 0x10
+#define CRTCV_CRC1_WINDOWB_X_CONTROL__CRTC_CRC1_WINDOWB_X_START_MASK 0x3fff
+#define CRTCV_CRC1_WINDOWB_X_CONTROL__CRTC_CRC1_WINDOWB_X_START__SHIFT 0x0
+#define CRTCV_CRC1_WINDOWB_X_CONTROL__CRTC_CRC1_WINDOWB_X_END_MASK 0x3fff0000
+#define CRTCV_CRC1_WINDOWB_X_CONTROL__CRTC_CRC1_WINDOWB_X_END__SHIFT 0x10
+#define CRTCV_CRC1_WINDOWB_Y_CONTROL__CRTC_CRC1_WINDOWB_Y_START_MASK 0x3fff
+#define CRTCV_CRC1_WINDOWB_Y_CONTROL__CRTC_CRC1_WINDOWB_Y_START__SHIFT 0x0
+#define CRTCV_CRC1_WINDOWB_Y_CONTROL__CRTC_CRC1_WINDOWB_Y_END_MASK 0x3fff0000
+#define CRTCV_CRC1_WINDOWB_Y_CONTROL__CRTC_CRC1_WINDOWB_Y_END__SHIFT 0x10
+#define CRTCV_CRC1_DATA_RG__CRC1_R_CR_MASK 0xffff
+#define CRTCV_CRC1_DATA_RG__CRC1_R_CR__SHIFT 0x0
+#define CRTCV_CRC1_DATA_RG__CRC1_G_Y_MASK 0xffff0000
+#define CRTCV_CRC1_DATA_RG__CRC1_G_Y__SHIFT 0x10
+#define CRTCV_CRC1_DATA_B__CRC1_B_CB_MASK 0xffff
+#define CRTCV_CRC1_DATA_B__CRC1_B_CB__SHIFT 0x0
+#define CRTCV_TEST_DEBUG_INDEX__CRTC_TEST_DEBUG_INDEX_MASK 0xff
+#define CRTCV_TEST_DEBUG_INDEX__CRTC_TEST_DEBUG_INDEX__SHIFT 0x0
+#define CRTCV_TEST_DEBUG_INDEX__CRTC_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define CRTCV_TEST_DEBUG_INDEX__CRTC_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define CRTCV_TEST_DEBUG_DATA__CRTC_TEST_DEBUG_DATA_MASK 0xffffffff
+#define CRTCV_TEST_DEBUG_DATA__CRTC_TEST_DEBUG_DATA__SHIFT 0x0
+#define XDMA_MC_PCIE_CLIENT_CONFIG__XDMA_MC_PCIE_SWAP_MASK 0x300
+#define XDMA_MC_PCIE_CLIENT_CONFIG__XDMA_MC_PCIE_SWAP__SHIFT 0x8
+#define XDMA_MC_PCIE_CLIENT_CONFIG__XDMA_MC_PCIE_VMID_MASK 0xf000
+#define XDMA_MC_PCIE_CLIENT_CONFIG__XDMA_MC_PCIE_VMID__SHIFT 0xc
+#define XDMA_MC_PCIE_CLIENT_CONFIG__XDMA_MC_PCIE_PRIV_MASK 0x10000
+#define XDMA_MC_PCIE_CLIENT_CONFIG__XDMA_MC_PCIE_PRIV__SHIFT 0x10
+#define XDMA_LOCAL_SURFACE_TILING1__XDMA_LOCAL_ARRAY_MODE_MASK 0xf
+#define XDMA_LOCAL_SURFACE_TILING1__XDMA_LOCAL_ARRAY_MODE__SHIFT 0x0
+#define XDMA_LOCAL_SURFACE_TILING1__XDMA_LOCAL_TILE_SPLIT_MASK 0x70
+#define XDMA_LOCAL_SURFACE_TILING1__XDMA_LOCAL_TILE_SPLIT__SHIFT 0x4
+#define XDMA_LOCAL_SURFACE_TILING1__XDMA_LOCAL_BANK_WIDTH_MASK 0x300
+#define XDMA_LOCAL_SURFACE_TILING1__XDMA_LOCAL_BANK_WIDTH__SHIFT 0x8
+#define XDMA_LOCAL_SURFACE_TILING1__XDMA_LOCAL_BANK_HEIGHT_MASK 0xc00
+#define XDMA_LOCAL_SURFACE_TILING1__XDMA_LOCAL_BANK_HEIGHT__SHIFT 0xa
+#define XDMA_LOCAL_SURFACE_TILING1__XDMA_LOCAL_MACRO_TILE_ASPECT_MASK 0x3000
+#define XDMA_LOCAL_SURFACE_TILING1__XDMA_LOCAL_MACRO_TILE_ASPECT__SHIFT 0xc
+#define XDMA_LOCAL_SURFACE_TILING1__XDMA_LOCAL_NUM_BANKS_MASK 0x300000
+#define XDMA_LOCAL_SURFACE_TILING1__XDMA_LOCAL_NUM_BANKS__SHIFT 0x14
+#define XDMA_LOCAL_SURFACE_TILING2__XDMA_LOCAL_PIPE_INTERLEAVE_SIZE_MASK 0x7
+#define XDMA_LOCAL_SURFACE_TILING2__XDMA_LOCAL_PIPE_INTERLEAVE_SIZE__SHIFT 0x0
+#define XDMA_LOCAL_SURFACE_TILING2__XDMA_LOCAL_MICRO_TILE_MODE_MASK 0x700000
+#define XDMA_LOCAL_SURFACE_TILING2__XDMA_LOCAL_MICRO_TILE_MODE__SHIFT 0x14
+#define XDMA_LOCAL_SURFACE_TILING2__XDMA_LOCAL_PIPE_CONFIG_MASK 0xf8000000
+#define XDMA_LOCAL_SURFACE_TILING2__XDMA_LOCAL_PIPE_CONFIG__SHIFT 0x1b
+#define XDMA_INTERRUPT__XDMA_MSTR_MEM_URGENT_STAT_MASK 0x100
+#define XDMA_INTERRUPT__XDMA_MSTR_MEM_URGENT_STAT__SHIFT 0x8
+#define XDMA_INTERRUPT__XDMA_MSTR_MEM_URGENT_MASK_MASK 0x200
+#define XDMA_INTERRUPT__XDMA_MSTR_MEM_URGENT_MASK__SHIFT 0x9
+#define XDMA_INTERRUPT__XDMA_MSTR_MEM_URGENT_ACK_MASK 0x400
+#define XDMA_INTERRUPT__XDMA_MSTR_MEM_URGENT_ACK__SHIFT 0xa
+#define XDMA_INTERRUPT__XDMA_SLV_READ_URGENT_STAT_MASK 0x10000
+#define XDMA_INTERRUPT__XDMA_SLV_READ_URGENT_STAT__SHIFT 0x10
+#define XDMA_INTERRUPT__XDMA_SLV_READ_URGENT_MASK_MASK 0x20000
+#define XDMA_INTERRUPT__XDMA_SLV_READ_URGENT_MASK__SHIFT 0x11
+#define XDMA_INTERRUPT__XDMA_SLV_READ_URGENT_ACK_MASK 0x40000
+#define XDMA_INTERRUPT__XDMA_SLV_READ_URGENT_ACK__SHIFT 0x12
+#define XDMA_INTERRUPT__XDMA_PERF_MEAS_STAT_MASK 0x100000
+#define XDMA_INTERRUPT__XDMA_PERF_MEAS_STAT__SHIFT 0x14
+#define XDMA_INTERRUPT__XDMA_PERF_MEAS_MASK_MASK 0x200000
+#define XDMA_INTERRUPT__XDMA_PERF_MEAS_MASK__SHIFT 0x15
+#define XDMA_INTERRUPT__XDMA_PERF_MEAS_ACK_MASK 0x400000
+#define XDMA_INTERRUPT__XDMA_PERF_MEAS_ACK__SHIFT 0x16
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_TURN_ON_DELAY_MASK 0xf
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_TURN_ON_DELAY__SHIFT 0x0
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_TURN_OFF_DELAY_MASK 0xff0
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_TURN_OFF_DELAY__SHIFT 0x4
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_GATE_DIS_MASK 0x8000
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_GATE_DIS__SHIFT 0xf
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_REG_GATE_DIS_MASK 0x10000
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_REG_GATE_DIS__SHIFT 0x10
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_G_MDYN_GATE_DIS_PIPE_0_MASK 0x20000
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_G_MDYN_GATE_DIS_PIPE_0__SHIFT 0x11
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_G_MDYN_GATE_DIS_PIPE_1_MASK 0x40000
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_G_MDYN_GATE_DIS_PIPE_1__SHIFT 0x12
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_G_MDYN_GATE_DIS_PIPE_2_MASK 0x80000
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_G_MDYN_GATE_DIS_PIPE_2__SHIFT 0x13
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_G_MDYN_GATE_DIS_PIPE_3_MASK 0x100000
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_G_MDYN_GATE_DIS_PIPE_3__SHIFT 0x14
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_G_MDYN_GATE_DIS_PIPE_4_MASK 0x200000
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_G_MDYN_GATE_DIS_PIPE_4__SHIFT 0x15
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_G_MDYN_GATE_DIS_PIPE_5_MASK 0x400000
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_G_MDYN_GATE_DIS_PIPE_5__SHIFT 0x16
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_G_SDYN_GATE_DIS_MASK 0x800000
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_G_SDYN_GATE_DIS__SHIFT 0x17
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_G_MSTAT_GATE_DIS_MASK 0x1000000
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_G_MSTAT_GATE_DIS__SHIFT 0x18
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_G_SSTAT_GATE_DIS_MASK 0x2000000
+#define XDMA_CLOCK_GATING_CNTL__XDMA_SCLK_G_SSTAT_GATE_DIS__SHIFT 0x19
+#define XDMA_MEM_POWER_CNTL__XDMA_MEM_CORE_IDLE_STATE_MASK 0x3
+#define XDMA_MEM_POWER_CNTL__XDMA_MEM_CORE_IDLE_STATE__SHIFT 0x0
+#define XDMA_MEM_POWER_CNTL__XDMA_MEM_IF_IDLE_STATE_MASK 0xc
+#define XDMA_MEM_POWER_CNTL__XDMA_MEM_IF_IDLE_STATE__SHIFT 0x2
+#define XDMA_MEM_POWER_CNTL__XDMA_MEM_IF_PCIE_STATE_MASK 0x180000
+#define XDMA_MEM_POWER_CNTL__XDMA_MEM_IF_PCIE_STATE__SHIFT 0x13
+#define XDMA_MEM_POWER_CNTL__XDMA_MEM_IF_PCIE_TRANS_MASK 0x200000
+#define XDMA_MEM_POWER_CNTL__XDMA_MEM_IF_PCIE_TRANS__SHIFT 0x15
+#define XDMA_MEM_POWER_CNTL__XDMA_MEM_IF_RD_STATE_MASK 0xc00000
+#define XDMA_MEM_POWER_CNTL__XDMA_MEM_IF_RD_STATE__SHIFT 0x16
+#define XDMA_MEM_POWER_CNTL__XDMA_MEM_IF_RD_TRANS_MASK 0x2000000
+#define XDMA_MEM_POWER_CNTL__XDMA_MEM_IF_RD_TRANS__SHIFT 0x19
+#define XDMA_MEM_POWER_CNTL__XDMA_MEM_IF_WR_STATE_MASK 0xc000000
+#define XDMA_MEM_POWER_CNTL__XDMA_MEM_IF_WR_STATE__SHIFT 0x1a
+#define XDMA_MEM_POWER_CNTL__XDMA_MEM_IF_WR_TRANS_MASK 0x10000000
+#define XDMA_MEM_POWER_CNTL__XDMA_MEM_IF_WR_TRANS__SHIFT 0x1c
+#define XDMA_MEM_POWER_CNTL__XDMA_MEM_IF_BIF_STATE_MASK 0x60000000
+#define XDMA_MEM_POWER_CNTL__XDMA_MEM_IF_BIF_STATE__SHIFT 0x1d
+#define XDMA_MEM_POWER_CNTL__XDMA_MEM_IF_BIF_TRANS_MASK 0x80000000
+#define XDMA_MEM_POWER_CNTL__XDMA_MEM_IF_BIF_TRANS__SHIFT 0x1f
+#define XDMA_IF_BIF_STATUS__XDMA_IF_BIF_ERROR_STATUS_MASK 0xf
+#define XDMA_IF_BIF_STATUS__XDMA_IF_BIF_ERROR_STATUS__SHIFT 0x0
+#define XDMA_IF_BIF_STATUS__XDMA_IF_BIF_ERROR_CLEAR_MASK 0x100
+#define XDMA_IF_BIF_STATUS__XDMA_IF_BIF_ERROR_CLEAR__SHIFT 0x8
+#define XDMA_PERF_MEAS_STATUS__XDMA_PERF_MEAS_STATUS_MASK 0xff
+#define XDMA_PERF_MEAS_STATUS__XDMA_PERF_MEAS_STATUS__SHIFT 0x0
+#define XDMA_IF_STATUS__XDMA_MC_PCIEWR_BUSY_MASK 0x1
+#define XDMA_IF_STATUS__XDMA_MC_PCIEWR_BUSY__SHIFT 0x0
+#define XDMA_TEST_DEBUG_INDEX__XDMA_TEST_DEBUG_INDEX_MASK 0xff
+#define XDMA_TEST_DEBUG_INDEX__XDMA_TEST_DEBUG_INDEX__SHIFT 0x0
+#define XDMA_TEST_DEBUG_INDEX__XDMA_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define XDMA_TEST_DEBUG_INDEX__XDMA_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define XDMA_TEST_DEBUG_DATA__XDMA_TEST_DEBUG_DATA_MASK 0xffffffff
+#define XDMA_TEST_DEBUG_DATA__XDMA_TEST_DEBUG_DATA__SHIFT 0x0
+#define XDMA_RBBMIF_RDWR_CNTL__XDMA_RBBMIF_RDWR_DELAY_MASK 0x7
+#define XDMA_RBBMIF_RDWR_CNTL__XDMA_RBBMIF_RDWR_DELAY__SHIFT 0x0
+#define XDMA_RBBMIF_RDWR_CNTL__XDMA_RBBMIF_RDWR_TIMEOUT_DIS_MASK 0x8
+#define XDMA_RBBMIF_RDWR_CNTL__XDMA_RBBMIF_RDWR_TIMEOUT_DIS__SHIFT 0x3
+#define XDMA_RBBMIF_RDWR_CNTL__XDMA_RBBMIF_TIMEOUT_DELAY_MASK 0xffff8000
+#define XDMA_RBBMIF_RDWR_CNTL__XDMA_RBBMIF_TIMEOUT_DELAY__SHIFT 0xf
+#define XDMA_PG_CONTROL__XDMA_PG_CONTROL_MASK 0xffffffff
+#define XDMA_PG_CONTROL__XDMA_PG_CONTROL__SHIFT 0x0
+#define XDMA_PG_WDATA__XDMA_PG_WDATA_MASK 0xffffffff
+#define XDMA_PG_WDATA__XDMA_PG_WDATA__SHIFT 0x0
+#define XDMA_PG_STATUS__XDMA_SERDES_RDATA_MASK 0xffffff
+#define XDMA_PG_STATUS__XDMA_SERDES_RDATA__SHIFT 0x0
+#define XDMA_PG_STATUS__XDMA_PGFSM_READ_READY_MASK 0x1000000
+#define XDMA_PG_STATUS__XDMA_PGFSM_READ_READY__SHIFT 0x18
+#define XDMA_PG_STATUS__XDMA_SERDES_BUSY_MASK 0x2000000
+#define XDMA_PG_STATUS__XDMA_SERDES_BUSY__SHIFT 0x19
+#define XDMA_PG_STATUS__XDMA_SERDES_SMU_POWER_STATUS_MASK 0x4000000
+#define XDMA_PG_STATUS__XDMA_SERDES_SMU_POWER_STATUS__SHIFT 0x1a
+#define XDMA_AON_TEST_DEBUG_INDEX__XDMA_AON_TEST_DEBUG_INDEX_MASK 0xff
+#define XDMA_AON_TEST_DEBUG_INDEX__XDMA_AON_TEST_DEBUG_INDEX__SHIFT 0x0
+#define XDMA_AON_TEST_DEBUG_INDEX__XDMA_AON_TEST_DEBUG_WRITE_EN_MASK 0x100
+#define XDMA_AON_TEST_DEBUG_INDEX__XDMA_AON_TEST_DEBUG_WRITE_EN__SHIFT 0x8
+#define XDMA_AON_TEST_DEBUG_INDEX__XDMA_DEBUG_SEL_MASK 0x200
+#define XDMA_AON_TEST_DEBUG_INDEX__XDMA_DEBUG_SEL__SHIFT 0x9
+#define XDMA_AON_TEST_DEBUG_INDEX__XDMA_DEBUG_OUT_EN_MASK 0x400
+#define XDMA_AON_TEST_DEBUG_INDEX__XDMA_DEBUG_OUT_EN__SHIFT 0xa
+#define XDMA_AON_TEST_DEBUG_DATA__XDMA_AON_TEST_DEBUG_DATA_MASK 0xffffffff
+#define XDMA_AON_TEST_DEBUG_DATA__XDMA_AON_TEST_DEBUG_DATA__SHIFT 0x0
+#define XDMA_MSTR_CNTL__XDMA_MSTR_ALPHA_POSITION_MASK 0x3000
+#define XDMA_MSTR_CNTL__XDMA_MSTR_ALPHA_POSITION__SHIFT 0xc
+#define XDMA_MSTR_CNTL__XDMA_MSTR_MEM_READY_MASK 0x4000
+#define XDMA_MSTR_CNTL__XDMA_MSTR_MEM_READY__SHIFT 0xe
+#define XDMA_MSTR_CNTL__XDMA_MSTR_ENABLE_MASK 0x10000
+#define XDMA_MSTR_CNTL__XDMA_MSTR_ENABLE__SHIFT 0x10
+#define XDMA_MSTR_CNTL__XDMA_MSTR_DEBUG_MODE_MASK 0x40000
+#define XDMA_MSTR_CNTL__XDMA_MSTR_DEBUG_MODE__SHIFT 0x12
+#define XDMA_MSTR_CNTL__XDMA_MSTR_SOFT_RESET_MASK 0x100000
+#define XDMA_MSTR_CNTL__XDMA_MSTR_SOFT_RESET__SHIFT 0x14
+#define XDMA_MSTR_CNTL__XDMA_MSTR_BIF_STALL_EN_MASK 0x200000
+#define XDMA_MSTR_CNTL__XDMA_MSTR_BIF_STALL_EN__SHIFT 0x15
+#define XDMA_MSTR_STATUS__XDMA_MSTR_VCOUNT_CURRENT_MASK 0x3fff
+#define XDMA_MSTR_STATUS__XDMA_MSTR_VCOUNT_CURRENT__SHIFT 0x0
+#define XDMA_MSTR_STATUS__XDMA_MSTR_WRITE_LINE_CURRENT_MASK 0xfff0000
+#define XDMA_MSTR_STATUS__XDMA_MSTR_WRITE_LINE_CURRENT__SHIFT 0x10
+#define XDMA_MSTR_STATUS__XDMA_MSTR_STATUS_SELECT_MASK 0x70000000
+#define XDMA_MSTR_STATUS__XDMA_MSTR_STATUS_SELECT__SHIFT 0x1c
+#define XDMA_MSTR_MEM_CLIENT_CONFIG__XDMA_MSTR_MEM_CLIENT_SWAP_MASK 0x300
+#define XDMA_MSTR_MEM_CLIENT_CONFIG__XDMA_MSTR_MEM_CLIENT_SWAP__SHIFT 0x8
+#define XDMA_MSTR_MEM_CLIENT_CONFIG__XDMA_MSTR_MEM_CLIENT_VMID_MASK 0xf000
+#define XDMA_MSTR_MEM_CLIENT_CONFIG__XDMA_MSTR_MEM_CLIENT_VMID__SHIFT 0xc
+#define XDMA_MSTR_MEM_CLIENT_CONFIG__XDMA_MSTR_MEM_CLIENT_PRIV_MASK 0x10000
+#define XDMA_MSTR_MEM_CLIENT_CONFIG__XDMA_MSTR_MEM_CLIENT_PRIV__SHIFT 0x10
+#define XDMA_MSTR_LOCAL_SURFACE_BASE_ADDR__XDMA_MSTR_LOCAL_SURFACE_BASE_ADDR_MASK 0xffffffff
+#define XDMA_MSTR_LOCAL_SURFACE_BASE_ADDR__XDMA_MSTR_LOCAL_SURFACE_BASE_ADDR__SHIFT 0x0
+#define XDMA_MSTR_LOCAL_SURFACE_BASE_ADDR_HIGH__XDMA_MSTR_LOCAL_SURFACE_BASE_ADDR_HIGH_MASK 0xff
+#define XDMA_MSTR_LOCAL_SURFACE_BASE_ADDR_HIGH__XDMA_MSTR_LOCAL_SURFACE_BASE_ADDR_HIGH__SHIFT 0x0
+#define XDMA_MSTR_LOCAL_SURFACE_PITCH__XDMA_MSTR_LOCAL_SURFACE_PITCH_MASK 0x3fff
+#define XDMA_MSTR_LOCAL_SURFACE_PITCH__XDMA_MSTR_LOCAL_SURFACE_PITCH__SHIFT 0x0
+#define XDMA_MSTR_CMD_URGENT_CNTL__XDMA_MSTR_CMD_CLIENT_STALL_MASK 0x1
+#define XDMA_MSTR_CMD_URGENT_CNTL__XDMA_MSTR_CMD_CLIENT_STALL__SHIFT 0x0
+#define XDMA_MSTR_CMD_URGENT_CNTL__XDMA_MSTR_CMD_URGENT_LEVEL_MASK 0xf00
+#define XDMA_MSTR_CMD_URGENT_CNTL__XDMA_MSTR_CMD_URGENT_LEVEL__SHIFT 0x8
+#define XDMA_MSTR_CMD_URGENT_CNTL__XDMA_MSTR_CMD_STALL_DELAY_MASK 0xf000
+#define XDMA_MSTR_CMD_URGENT_CNTL__XDMA_MSTR_CMD_STALL_DELAY__SHIFT 0xc
+#define XDMA_MSTR_MEM_URGENT_CNTL__XDMA_MSTR_MEM_CLIENT_STALL_MASK 0x1
+#define XDMA_MSTR_MEM_URGENT_CNTL__XDMA_MSTR_MEM_CLIENT_STALL__SHIFT 0x0
+#define XDMA_MSTR_MEM_URGENT_CNTL__XDMA_MSTR_MEM_URGENT_LIMIT_MASK 0xf0
+#define XDMA_MSTR_MEM_URGENT_CNTL__XDMA_MSTR_MEM_URGENT_LIMIT__SHIFT 0x4
+#define XDMA_MSTR_MEM_URGENT_CNTL__XDMA_MSTR_MEM_URGENT_LEVEL_MASK 0xf00
+#define XDMA_MSTR_MEM_URGENT_CNTL__XDMA_MSTR_MEM_URGENT_LEVEL__SHIFT 0x8
+#define XDMA_MSTR_MEM_URGENT_CNTL__XDMA_MSTR_MEM_STALL_DELAY_MASK 0xf000
+#define XDMA_MSTR_MEM_URGENT_CNTL__XDMA_MSTR_MEM_STALL_DELAY__SHIFT 0xc
+#define XDMA_MSTR_MEM_URGENT_CNTL__XDMA_MSTR_MEM_URGENT_TIMER_MASK 0xffff0000
+#define XDMA_MSTR_MEM_URGENT_CNTL__XDMA_MSTR_MEM_URGENT_TIMER__SHIFT 0x10
+#define XDMA_MSTR_PCIE_NACK_STATUS__XDMA_MSTR_PCIE_NACK_TAG_MASK 0x3ff
+#define XDMA_MSTR_PCIE_NACK_STATUS__XDMA_MSTR_PCIE_NACK_TAG__SHIFT 0x0
+#define XDMA_MSTR_PCIE_NACK_STATUS__XDMA_MSTR_PCIE_NACK_MASK 0x3000
+#define XDMA_MSTR_PCIE_NACK_STATUS__XDMA_MSTR_PCIE_NACK__SHIFT 0xc
+#define XDMA_MSTR_PCIE_NACK_STATUS__XDMA_MSTR_PCIE_NACK_CLR_MASK 0x10000
+#define XDMA_MSTR_PCIE_NACK_STATUS__XDMA_MSTR_PCIE_NACK_CLR__SHIFT 0x10
+#define XDMA_MSTR_MEM_NACK_STATUS__XDMA_MSTR_MEM_NACK_TAG_MASK 0x3ff
+#define XDMA_MSTR_MEM_NACK_STATUS__XDMA_MSTR_MEM_NACK_TAG__SHIFT 0x0
+#define XDMA_MSTR_MEM_NACK_STATUS__XDMA_MSTR_MEM_NACK_MASK 0x3000
+#define XDMA_MSTR_MEM_NACK_STATUS__XDMA_MSTR_MEM_NACK__SHIFT 0xc
+#define XDMA_MSTR_MEM_NACK_STATUS__XDMA_MSTR_MEM_NACK_CLR_MASK 0x10000
+#define XDMA_MSTR_MEM_NACK_STATUS__XDMA_MSTR_MEM_NACK_CLR__SHIFT 0x10
+#define XDMA_MSTR_VSYNC_GSL_CHECK__XDMA_MSTR_VSYNC_GSL_CHECK_SEL_MASK 0x7
+#define XDMA_MSTR_VSYNC_GSL_CHECK__XDMA_MSTR_VSYNC_GSL_CHECK_SEL__SHIFT 0x0
+#define XDMA_MSTR_VSYNC_GSL_CHECK__XDMA_MSTR_VSYNC_GSL_CHECK_V_COUNT_MASK 0x3fff00
+#define XDMA_MSTR_VSYNC_GSL_CHECK__XDMA_MSTR_VSYNC_GSL_CHECK_V_COUNT__SHIFT 0x8
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_CACHE_LINES_MASK 0xff
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_CACHE_LINES__SHIFT 0x0
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_READ_REQUEST_MASK 0x100
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_READ_REQUEST__SHIFT 0x8
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_PIPE_FRAME_MODE_MASK 0x200
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_PIPE_FRAME_MODE__SHIFT 0x9
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_PIPE_SOFT_RESET_MASK 0x400
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_PIPE_SOFT_RESET__SHIFT 0xa
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_CACHE_INVALIDATE_MASK 0x800
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_CACHE_INVALIDATE__SHIFT 0xb
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_REQUEST_CHANNEL_ID_MASK 0x7000
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_REQUEST_CHANNEL_ID__SHIFT 0xc
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_FLIP_MODE_MASK 0x8000
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_FLIP_MODE__SHIFT 0xf
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_REQUEST_MIN_MASK 0xff0000
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_REQUEST_MIN__SHIFT 0x10
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_PIPE_ACTIVE_MASK 0x1000000
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_PIPE_ACTIVE__SHIFT 0x18
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_PIPE_FLUSHING_MASK 0x2000000
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_PIPE_FLUSHING__SHIFT 0x19
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_PIPE_FLIP_PENDING_MASK 0x4000000
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_PIPE_FLIP_PENDING__SHIFT 0x1a
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_VSYNC_GSL_ENABLE_MASK 0x8000000
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_VSYNC_GSL_ENABLE__SHIFT 0x1b
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_SUPERAA_ENABLE_MASK 0x10000000
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_SUPERAA_ENABLE__SHIFT 0x1c
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_HSYNC_GSL_GROUP_MASK 0x60000000
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_HSYNC_GSL_GROUP__SHIFT 0x1d
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_GSL_GROUP_MASTER_MASK 0x80000000
+#define XDMA_MSTR_PIPE_CNTL__XDMA_MSTR_GSL_GROUP_MASTER__SHIFT 0x1f
+#define XDMA_MSTR_READ_COMMAND__XDMA_MSTR_REQUEST_SIZE_MASK 0x3fff
+#define XDMA_MSTR_READ_COMMAND__XDMA_MSTR_REQUEST_SIZE__SHIFT 0x0
+#define XDMA_MSTR_READ_COMMAND__XDMA_MSTR_REQUEST_PREFETCH_MASK 0x3fff0000
+#define XDMA_MSTR_READ_COMMAND__XDMA_MSTR_REQUEST_PREFETCH__SHIFT 0x10
+#define XDMA_MSTR_CHANNEL_DIM__XDMA_MSTR_CHANNEL_WIDTH_MASK 0x3fff
+#define XDMA_MSTR_CHANNEL_DIM__XDMA_MSTR_CHANNEL_WIDTH__SHIFT 0x0
+#define XDMA_MSTR_CHANNEL_DIM__XDMA_MSTR_CHANNEL_HEIGHT_MASK 0x3fff0000
+#define XDMA_MSTR_CHANNEL_DIM__XDMA_MSTR_CHANNEL_HEIGHT__SHIFT 0x10
+#define XDMA_MSTR_HEIGHT__XDMA_MSTR_ACTIVE_HEIGHT_MASK 0x3fff
+#define XDMA_MSTR_HEIGHT__XDMA_MSTR_ACTIVE_HEIGHT__SHIFT 0x0
+#define XDMA_MSTR_HEIGHT__XDMA_MSTR_FRAME_HEIGHT_MASK 0x3fff0000
+#define XDMA_MSTR_HEIGHT__XDMA_MSTR_FRAME_HEIGHT__SHIFT 0x10
+#define XDMA_MSTR_REMOTE_SURFACE_BASE__XDMA_MSTR_REMOTE_SURFACE_BASE_MASK 0xffffffff
+#define XDMA_MSTR_REMOTE_SURFACE_BASE__XDMA_MSTR_REMOTE_SURFACE_BASE__SHIFT 0x0
+#define XDMA_MSTR_REMOTE_SURFACE_BASE_HIGH__XDMA_MSTR_REMOTE_SURFACE_BASE_HIGH_MASK 0xff
+#define XDMA_MSTR_REMOTE_SURFACE_BASE_HIGH__XDMA_MSTR_REMOTE_SURFACE_BASE_HIGH__SHIFT 0x0
+#define XDMA_MSTR_REMOTE_GPU_ADDRESS__XDMA_MSTR_REMOTE_GPU_ADDRESS_MASK 0xffffffff
+#define XDMA_MSTR_REMOTE_GPU_ADDRESS__XDMA_MSTR_REMOTE_GPU_ADDRESS__SHIFT 0x0
+#define XDMA_MSTR_REMOTE_GPU_ADDRESS_HIGH__XDMA_MSTR_REMOTE_GPU_ADDRESS_HIGH_MASK 0xff
+#define XDMA_MSTR_REMOTE_GPU_ADDRESS_HIGH__XDMA_MSTR_REMOTE_GPU_ADDRESS_HIGH__SHIFT 0x0
+#define XDMA_MSTR_CACHE_BASE_ADDR__XDMA_MSTR_CACHE_BASE_ADDR_MASK 0xffffffff
+#define XDMA_MSTR_CACHE_BASE_ADDR__XDMA_MSTR_CACHE_BASE_ADDR__SHIFT 0x0
+#define XDMA_MSTR_CACHE_BASE_ADDR_HIGH__XDMA_MSTR_CACHE_BASE_ADDR_HIGH_MASK 0xff
+#define XDMA_MSTR_CACHE_BASE_ADDR_HIGH__XDMA_MSTR_CACHE_BASE_ADDR_HIGH__SHIFT 0x0
+#define XDMA_MSTR_CACHE__XDMA_MSTR_CACHE_PITCH_MASK 0x3fff
+#define XDMA_MSTR_CACHE__XDMA_MSTR_CACHE_PITCH__SHIFT 0x0
+#define XDMA_MSTR_CACHE__XDMA_MSTR_CACHE_TLB_PG_STATE_MASK 0x60000000
+#define XDMA_MSTR_CACHE__XDMA_MSTR_CACHE_TLB_PG_STATE__SHIFT 0x1d
+#define XDMA_MSTR_CACHE__XDMA_MSTR_CACHE_TLB_PG_TRANS_MASK 0x80000000
+#define XDMA_MSTR_CACHE__XDMA_MSTR_CACHE_TLB_PG_TRANS__SHIFT 0x1f
+#define XDMA_MSTR_CHANNEL_START__XDMA_MSTR_CHANNEL_START_X_MASK 0x3fff
+#define XDMA_MSTR_CHANNEL_START__XDMA_MSTR_CHANNEL_START_X__SHIFT 0x0
+#define XDMA_MSTR_CHANNEL_START__XDMA_MSTR_CHANNEL_START_Y_MASK 0x3fff0000
+#define XDMA_MSTR_CHANNEL_START__XDMA_MSTR_CHANNEL_START_Y__SHIFT 0x10
+#define XDMA_MSTR_PERFMEAS_STATUS__XDMA_MSTR_PERFMEAS_DATA_MASK 0xffffff
+#define XDMA_MSTR_PERFMEAS_STATUS__XDMA_MSTR_PERFMEAS_DATA__SHIFT 0x0
+#define XDMA_MSTR_PERFMEAS_STATUS__XDMA_MSTR_PERFMEAS_INDEX_MASK 0x7000000
+#define XDMA_MSTR_PERFMEAS_STATUS__XDMA_MSTR_PERFMEAS_INDEX__SHIFT 0x18
+#define XDMA_MSTR_PERFMEAS_STATUS__XDMA_MSTR_PERFMEAS_INDEX_MODE_MASK 0xc0000000
+#define XDMA_MSTR_PERFMEAS_STATUS__XDMA_MSTR_PERFMEAS_INDEX_MODE__SHIFT 0x1e
+#define XDMA_MSTR_PERFMEAS_CNTL__XDMA_MSTR_CACHE_BW_MEAS_ITER_MASK 0xfff
+#define XDMA_MSTR_PERFMEAS_CNTL__XDMA_MSTR_CACHE_BW_MEAS_ITER__SHIFT 0x0
+#define XDMA_MSTR_PERFMEAS_CNTL__XDMA_MSTR_CACHE_BW_SEGID_SEL_MASK 0x1f000
+#define XDMA_MSTR_PERFMEAS_CNTL__XDMA_MSTR_CACHE_BW_SEGID_SEL__SHIFT 0xc
+#define XDMA_MSTR_PERFMEAS_CNTL__XDMA_MSTR_CACHE_BW_COUNTER_RST_MASK 0x20000
+#define XDMA_MSTR_PERFMEAS_CNTL__XDMA_MSTR_CACHE_BW_COUNTER_RST__SHIFT 0x11
+#define XDMA_MSTR_PERFMEAS_CNTL__XDMA_MSTR_LT_MEAS_ITER_MASK 0x7ff80000
+#define XDMA_MSTR_PERFMEAS_CNTL__XDMA_MSTR_LT_MEAS_ITER__SHIFT 0x13
+#define XDMA_MSTR_PERFMEAS_CNTL__XDMA_MSTR_LT_COUNTER_RST_MASK 0x80000000
+#define XDMA_MSTR_PERFMEAS_CNTL__XDMA_MSTR_LT_COUNTER_RST__SHIFT 0x1f
+#define XDMA_SLV_CNTL__XDMA_SLV_READ_LINES_MASK 0x1
+#define XDMA_SLV_CNTL__XDMA_SLV_READ_LINES__SHIFT 0x0
+#define XDMA_SLV_CNTL__XDMA_SLV_MEM_READY_MASK 0x200
+#define XDMA_SLV_CNTL__XDMA_SLV_MEM_READY__SHIFT 0x9
+#define XDMA_SLV_CNTL__XDMA_SLV_ACTIVE_MASK 0x400
+#define XDMA_SLV_CNTL__XDMA_SLV_ACTIVE__SHIFT 0xa
+#define XDMA_SLV_CNTL__XDMA_SLV_ALPHA_POSITION_MASK 0x3000
+#define XDMA_SLV_CNTL__XDMA_SLV_ALPHA_POSITION__SHIFT 0xc
+#define XDMA_SLV_CNTL__XDMA_SLV_ENABLE_MASK 0x10000
+#define XDMA_SLV_CNTL__XDMA_SLV_ENABLE__SHIFT 0x10
+#define XDMA_SLV_CNTL__XDMA_SLV_READ_LAT_TEST_EN_MASK 0x80000
+#define XDMA_SLV_CNTL__XDMA_SLV_READ_LAT_TEST_EN__SHIFT 0x13
+#define XDMA_SLV_CNTL__XDMA_SLV_SOFT_RESET_MASK 0x100000
+#define XDMA_SLV_CNTL__XDMA_SLV_SOFT_RESET__SHIFT 0x14
+#define XDMA_SLV_CNTL__XDMA_SLV_REQ_MAXED_OUT_MASK 0x1000000
+#define XDMA_SLV_CNTL__XDMA_SLV_REQ_MAXED_OUT__SHIFT 0x18
+#define XDMA_SLV_CNTL__XDMA_SLV_WB_BURST_RESET_MASK 0x2000000
+#define XDMA_SLV_CNTL__XDMA_SLV_WB_BURST_RESET__SHIFT 0x19
+#define XDMA_SLV_MEM_CLIENT_CONFIG__XDMA_SLV_MEM_CLIENT_SWAP_MASK 0x300
+#define XDMA_SLV_MEM_CLIENT_CONFIG__XDMA_SLV_MEM_CLIENT_SWAP__SHIFT 0x8
+#define XDMA_SLV_MEM_CLIENT_CONFIG__XDMA_SLV_MEM_CLIENT_VMID_MASK 0xf000
+#define XDMA_SLV_MEM_CLIENT_CONFIG__XDMA_SLV_MEM_CLIENT_VMID__SHIFT 0xc
+#define XDMA_SLV_MEM_CLIENT_CONFIG__XDMA_SLV_MEM_CLIENT_PRIV_MASK 0x10000
+#define XDMA_SLV_MEM_CLIENT_CONFIG__XDMA_SLV_MEM_CLIENT_PRIV__SHIFT 0x10
+#define XDMA_SLV_SLS_PITCH__XDMA_SLV_SLS_PITCH_MASK 0x3fff
+#define XDMA_SLV_SLS_PITCH__XDMA_SLV_SLS_PITCH__SHIFT 0x0
+#define XDMA_SLV_SLS_PITCH__XDMA_SLV_SLS_WIDTH_MASK 0x3fff0000
+#define XDMA_SLV_SLS_PITCH__XDMA_SLV_SLS_WIDTH__SHIFT 0x10
+#define XDMA_SLV_READ_URGENT_CNTL__XDMA_SLV_READ_CLIENT_STALL_MASK 0x1
+#define XDMA_SLV_READ_URGENT_CNTL__XDMA_SLV_READ_CLIENT_STALL__SHIFT 0x0
+#define XDMA_SLV_READ_URGENT_CNTL__XDMA_SLV_READ_URGENT_LIMIT_MASK 0xf0
+#define XDMA_SLV_READ_URGENT_CNTL__XDMA_SLV_READ_URGENT_LIMIT__SHIFT 0x4
+#define XDMA_SLV_READ_URGENT_CNTL__XDMA_SLV_READ_URGENT_LEVEL_MASK 0xf00
+#define XDMA_SLV_READ_URGENT_CNTL__XDMA_SLV_READ_URGENT_LEVEL__SHIFT 0x8
+#define XDMA_SLV_READ_URGENT_CNTL__XDMA_SLV_READ_STALL_DELAY_MASK 0xf000
+#define XDMA_SLV_READ_URGENT_CNTL__XDMA_SLV_READ_STALL_DELAY__SHIFT 0xc
+#define XDMA_SLV_READ_URGENT_CNTL__XDMA_SLV_READ_URGENT_TIMER_MASK 0xffff0000
+#define XDMA_SLV_READ_URGENT_CNTL__XDMA_SLV_READ_URGENT_TIMER__SHIFT 0x10
+#define XDMA_SLV_WRITE_URGENT_CNTL__XDMA_SLV_WRITE_STALL_MASK 0x1
+#define XDMA_SLV_WRITE_URGENT_CNTL__XDMA_SLV_WRITE_STALL__SHIFT 0x0
+#define XDMA_SLV_WRITE_URGENT_CNTL__XDMA_SLV_WRITE_URGENT_LEVEL_MASK 0xf00
+#define XDMA_SLV_WRITE_URGENT_CNTL__XDMA_SLV_WRITE_URGENT_LEVEL__SHIFT 0x8
+#define XDMA_SLV_WRITE_URGENT_CNTL__XDMA_SLV_WRITE_STALL_DELAY_MASK 0xf000
+#define XDMA_SLV_WRITE_URGENT_CNTL__XDMA_SLV_WRITE_STALL_DELAY__SHIFT 0xc
+#define XDMA_SLV_WB_RATE_CNTL__XDMA_SLV_WB_BURST_SIZE_MASK 0x1ff
+#define XDMA_SLV_WB_RATE_CNTL__XDMA_SLV_WB_BURST_SIZE__SHIFT 0x0
+#define XDMA_SLV_WB_RATE_CNTL__XDMA_SLV_WB_BURST_PERIOD_MASK 0xffff0000
+#define XDMA_SLV_WB_RATE_CNTL__XDMA_SLV_WB_BURST_PERIOD__SHIFT 0x10
+#define XDMA_SLV_READ_LATENCY_MINMAX__XDMA_SLV_READ_LATENCY_MIN_MASK 0xffff
+#define XDMA_SLV_READ_LATENCY_MINMAX__XDMA_SLV_READ_LATENCY_MIN__SHIFT 0x0
+#define XDMA_SLV_READ_LATENCY_MINMAX__XDMA_SLV_READ_LATENCY_MAX_MASK 0xffff0000
+#define XDMA_SLV_READ_LATENCY_MINMAX__XDMA_SLV_READ_LATENCY_MAX__SHIFT 0x10
+#define XDMA_SLV_READ_LATENCY_AVE__XDMA_SLV_READ_LATENCY_ACC_MASK 0xfffff
+#define XDMA_SLV_READ_LATENCY_AVE__XDMA_SLV_READ_LATENCY_ACC__SHIFT 0x0
+#define XDMA_SLV_READ_LATENCY_AVE__XDMA_SLV_READ_LATENCY_COUNT_MASK 0xfff00000
+#define XDMA_SLV_READ_LATENCY_AVE__XDMA_SLV_READ_LATENCY_COUNT__SHIFT 0x14
+#define XDMA_SLV_PCIE_NACK_STATUS__XDMA_SLV_PCIE_NACK_TAG_MASK 0x3ff
+#define XDMA_SLV_PCIE_NACK_STATUS__XDMA_SLV_PCIE_NACK_TAG__SHIFT 0x0
+#define XDMA_SLV_PCIE_NACK_STATUS__XDMA_SLV_PCIE_NACK_MASK 0x3000
+#define XDMA_SLV_PCIE_NACK_STATUS__XDMA_SLV_PCIE_NACK__SHIFT 0xc
+#define XDMA_SLV_PCIE_NACK_STATUS__XDMA_SLV_PCIE_NACK_CLR_MASK 0x10000
+#define XDMA_SLV_PCIE_NACK_STATUS__XDMA_SLV_PCIE_NACK_CLR__SHIFT 0x10
+#define XDMA_SLV_MEM_NACK_STATUS__XDMA_SLV_MEM_NACK_TAG_MASK 0xffff
+#define XDMA_SLV_MEM_NACK_STATUS__XDMA_SLV_MEM_NACK_TAG__SHIFT 0x0
+#define XDMA_SLV_MEM_NACK_STATUS__XDMA_SLV_MEM_NACK_MASK 0x30000
+#define XDMA_SLV_MEM_NACK_STATUS__XDMA_SLV_MEM_NACK__SHIFT 0x10
+#define XDMA_SLV_MEM_NACK_STATUS__XDMA_SLV_MEM_NACK_CLR_MASK 0x80000000
+#define XDMA_SLV_MEM_NACK_STATUS__XDMA_SLV_MEM_NACK_CLR__SHIFT 0x1f
+#define XDMA_SLV_RDRET_BUF_STATUS__XDMA_SLV_RDRET_FREE_ENTRIES_MASK 0x3ff
+#define XDMA_SLV_RDRET_BUF_STATUS__XDMA_SLV_RDRET_FREE_ENTRIES__SHIFT 0x0
+#define XDMA_SLV_RDRET_BUF_STATUS__XDMA_SLV_RDRET_BUF_SIZE_MASK 0x3ff000
+#define XDMA_SLV_RDRET_BUF_STATUS__XDMA_SLV_RDRET_BUF_SIZE__SHIFT 0xc
+#define XDMA_SLV_RDRET_BUF_STATUS__XDMA_SLV_RDRET_PG_STATE_MASK 0xc00000
+#define XDMA_SLV_RDRET_BUF_STATUS__XDMA_SLV_RDRET_PG_STATE__SHIFT 0x16
+#define XDMA_SLV_RDRET_BUF_STATUS__XDMA_SLV_RDRET_PG_TRANS_MASK 0x1000000
+#define XDMA_SLV_RDRET_BUF_STATUS__XDMA_SLV_RDRET_PG_TRANS__SHIFT 0x18
+#define XDMA_SLV_READ_LATENCY_TIMER__XDMA_SLV_READ_LATENCY_TIMER_MASK 0xffff
+#define XDMA_SLV_READ_LATENCY_TIMER__XDMA_SLV_READ_LATENCY_TIMER__SHIFT 0x0
+#define XDMA_SLV_FLIP_PENDING__XDMA_SLV_FLIP_PENDING_MASK 0x1
+#define XDMA_SLV_FLIP_PENDING__XDMA_SLV_FLIP_PENDING__SHIFT 0x0
+#define XDMA_SLV_CHANNEL_CNTL__XDMA_SLV_CHANNEL_WEIGHT_MASK 0x1ff
+#define XDMA_SLV_CHANNEL_CNTL__XDMA_SLV_CHANNEL_WEIGHT__SHIFT 0x0
+#define XDMA_SLV_CHANNEL_CNTL__XDMA_SLV_STOP_TRANSFER_MASK 0x10000
+#define XDMA_SLV_CHANNEL_CNTL__XDMA_SLV_STOP_TRANSFER__SHIFT 0x10
+#define XDMA_SLV_CHANNEL_CNTL__XDMA_SLV_CHANNEL_SOFT_RESET_MASK 0x20000
+#define XDMA_SLV_CHANNEL_CNTL__XDMA_SLV_CHANNEL_SOFT_RESET__SHIFT 0x11
+#define XDMA_SLV_CHANNEL_CNTL__XDMA_SLV_CHANNEL_ACTIVE_MASK 0x1000000
+#define XDMA_SLV_CHANNEL_CNTL__XDMA_SLV_CHANNEL_ACTIVE__SHIFT 0x18
+#define XDMA_SLV_REMOTE_GPU_ADDRESS__XDMA_SLV_REMOTE_GPU_ADDRESS_MASK 0xffffffff
+#define XDMA_SLV_REMOTE_GPU_ADDRESS__XDMA_SLV_REMOTE_GPU_ADDRESS__SHIFT 0x0
+#define XDMA_SLV_REMOTE_GPU_ADDRESS_HIGH__XDMA_SLV_REMOTE_GPU_ADDRESS_HIGH_MASK 0xff
+#define XDMA_SLV_REMOTE_GPU_ADDRESS_HIGH__XDMA_SLV_REMOTE_GPU_ADDRESS_HIGH__SHIFT 0x0
+#define CMD_BUS_TX_CONTROL_LANE0__tx_pwr_MASK 0x7
+#define CMD_BUS_TX_CONTROL_LANE0__tx_pwr__SHIFT 0x0
+#define CMD_BUS_TX_CONTROL_LANE0__tx_pg_en_MASK 0x18
+#define CMD_BUS_TX_CONTROL_LANE0__tx_pg_en__SHIFT 0x3
+#define CMD_BUS_TX_CONTROL_LANE0__tx_rdy_MASK 0x100
+#define CMD_BUS_TX_CONTROL_LANE0__tx_rdy__SHIFT 0x8
+#define CMD_BUS_TX_CONTROL_LANE1__tx_pwr_MASK 0x7
+#define CMD_BUS_TX_CONTROL_LANE1__tx_pwr__SHIFT 0x0
+#define CMD_BUS_TX_CONTROL_LANE1__tx_pg_en_MASK 0x18
+#define CMD_BUS_TX_CONTROL_LANE1__tx_pg_en__SHIFT 0x3
+#define CMD_BUS_TX_CONTROL_LANE1__tx_rdy_MASK 0x100
+#define CMD_BUS_TX_CONTROL_LANE1__tx_rdy__SHIFT 0x8
+#define CMD_BUS_TX_CONTROL_LANE2__tx_pwr_MASK 0x7
+#define CMD_BUS_TX_CONTROL_LANE2__tx_pwr__SHIFT 0x0
+#define CMD_BUS_TX_CONTROL_LANE2__tx_pg_en_MASK 0x18
+#define CMD_BUS_TX_CONTROL_LANE2__tx_pg_en__SHIFT 0x3
+#define CMD_BUS_TX_CONTROL_LANE2__tx_rdy_MASK 0x100
+#define CMD_BUS_TX_CONTROL_LANE2__tx_rdy__SHIFT 0x8
+#define CMD_BUS_TX_CONTROL_LANE3__tx_pwr_MASK 0x7
+#define CMD_BUS_TX_CONTROL_LANE3__tx_pwr__SHIFT 0x0
+#define CMD_BUS_TX_CONTROL_LANE3__tx_pg_en_MASK 0x18
+#define CMD_BUS_TX_CONTROL_LANE3__tx_pg_en__SHIFT 0x3
+#define CMD_BUS_TX_CONTROL_LANE3__tx_rdy_MASK 0x100
+#define CMD_BUS_TX_CONTROL_LANE3__tx_rdy__SHIFT 0x8
+#define MARGIN_DEEMPH_LANE0__txmarg_sel_MASK 0x7
+#define MARGIN_DEEMPH_LANE0__txmarg_sel__SHIFT 0x0
+#define MARGIN_DEEMPH_LANE0__deemph_sel_MASK 0x18
+#define MARGIN_DEEMPH_LANE0__deemph_sel__SHIFT 0x3
+#define MARGIN_DEEMPH_LANE0__tx_margin_en_MASK 0x20
+#define MARGIN_DEEMPH_LANE0__tx_margin_en__SHIFT 0x5
+#define MARGIN_DEEMPH_LANE1__txmarg_sel_MASK 0x7
+#define MARGIN_DEEMPH_LANE1__txmarg_sel__SHIFT 0x0
+#define MARGIN_DEEMPH_LANE1__deemph_sel_MASK 0x18
+#define MARGIN_DEEMPH_LANE1__deemph_sel__SHIFT 0x3
+#define MARGIN_DEEMPH_LANE1__tx_margin_en_MASK 0x20
+#define MARGIN_DEEMPH_LANE1__tx_margin_en__SHIFT 0x5
+#define MARGIN_DEEMPH_LANE2__txmarg_sel_MASK 0x7
+#define MARGIN_DEEMPH_LANE2__txmarg_sel__SHIFT 0x0
+#define MARGIN_DEEMPH_LANE2__deemph_sel_MASK 0x18
+#define MARGIN_DEEMPH_LANE2__deemph_sel__SHIFT 0x3
+#define MARGIN_DEEMPH_LANE2__tx_margin_en_MASK 0x20
+#define MARGIN_DEEMPH_LANE2__tx_margin_en__SHIFT 0x5
+#define MARGIN_DEEMPH_LANE3__txmarg_sel_MASK 0x7
+#define MARGIN_DEEMPH_LANE3__txmarg_sel__SHIFT 0x0
+#define MARGIN_DEEMPH_LANE3__deemph_sel_MASK 0x18
+#define MARGIN_DEEMPH_LANE3__deemph_sel__SHIFT 0x3
+#define MARGIN_DEEMPH_LANE3__tx_margin_en_MASK 0x20
+#define MARGIN_DEEMPH_LANE3__tx_margin_en__SHIFT 0x5
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__twosym_en_MASK 0x6
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__twosym_en__SHIFT 0x1
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__link_speed_MASK 0x18
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__link_speed__SHIFT 0x3
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__gang_mode_MASK 0xe0
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__gang_mode__SHIFT 0x5
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__max_linkrate_MASK 0x300
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__max_linkrate__SHIFT 0x8
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__pcs_freq_MASK 0xc00
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__pcs_freq__SHIFT 0xa
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__pcs_clken_MASK 0x1000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__pcs_clken__SHIFT 0xc
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__pcs_clkdone_MASK 0x2000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__pcs_clkdone__SHIFT 0xd
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__pll1_always_on_MASK 0x4000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__pll1_always_on__SHIFT 0xe
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__rdclk_div2_en_MASK 0x8000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__rdclk_div2_en__SHIFT 0xf
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__tx_boost_adj_MASK 0xf0000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__tx_boost_adj__SHIFT 0x10
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__tx_boost_en_MASK 0x100000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__tx_boost_en__SHIFT 0x14
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__tx_binary_ron_code_offset_MASK 0xc00000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE0__tx_binary_ron_code_offset__SHIFT 0x16
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__twosym_en_MASK 0x6
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__twosym_en__SHIFT 0x1
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__link_speed_MASK 0x18
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__link_speed__SHIFT 0x3
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__gang_mode_MASK 0xe0
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__gang_mode__SHIFT 0x5
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__max_linkrate_MASK 0x300
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__max_linkrate__SHIFT 0x8
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__pcs_freq_MASK 0xc00
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__pcs_freq__SHIFT 0xa
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__pcs_clken_MASK 0x1000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__pcs_clken__SHIFT 0xc
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__pcs_clkdone_MASK 0x2000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__pcs_clkdone__SHIFT 0xd
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__pll1_always_on_MASK 0x4000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__pll1_always_on__SHIFT 0xe
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__rdclk_div2_en_MASK 0x8000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__rdclk_div2_en__SHIFT 0xf
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__tx_boost_adj_MASK 0xf0000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__tx_boost_adj__SHIFT 0x10
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__tx_boost_en_MASK 0x100000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__tx_boost_en__SHIFT 0x14
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__tx_binary_ron_code_offset_MASK 0xc00000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE1__tx_binary_ron_code_offset__SHIFT 0x16
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__twosym_en_MASK 0x6
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__twosym_en__SHIFT 0x1
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__link_speed_MASK 0x18
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__link_speed__SHIFT 0x3
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__gang_mode_MASK 0xe0
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__gang_mode__SHIFT 0x5
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__max_linkrate_MASK 0x300
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__max_linkrate__SHIFT 0x8
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__pcs_freq_MASK 0xc00
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__pcs_freq__SHIFT 0xa
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__pcs_clken_MASK 0x1000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__pcs_clken__SHIFT 0xc
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__pcs_clkdone_MASK 0x2000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__pcs_clkdone__SHIFT 0xd
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__pll1_always_on_MASK 0x4000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__pll1_always_on__SHIFT 0xe
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__rdclk_div2_en_MASK 0x8000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__rdclk_div2_en__SHIFT 0xf
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__tx_boost_adj_MASK 0xf0000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__tx_boost_adj__SHIFT 0x10
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__tx_boost_en_MASK 0x100000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__tx_boost_en__SHIFT 0x14
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__tx_binary_ron_code_offset_MASK 0xc00000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE2__tx_binary_ron_code_offset__SHIFT 0x16
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__twosym_en_MASK 0x6
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__twosym_en__SHIFT 0x1
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__link_speed_MASK 0x18
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__link_speed__SHIFT 0x3
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__gang_mode_MASK 0xe0
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__gang_mode__SHIFT 0x5
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__max_linkrate_MASK 0x300
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__max_linkrate__SHIFT 0x8
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__pcs_freq_MASK 0xc00
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__pcs_freq__SHIFT 0xa
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__pcs_clken_MASK 0x1000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__pcs_clken__SHIFT 0xc
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__pcs_clkdone_MASK 0x2000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__pcs_clkdone__SHIFT 0xd
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__pll1_always_on_MASK 0x4000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__pll1_always_on__SHIFT 0xe
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__rdclk_div2_en_MASK 0x8000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__rdclk_div2_en__SHIFT 0xf
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__tx_boost_adj_MASK 0xf0000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__tx_boost_adj__SHIFT 0x10
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__tx_boost_en_MASK 0x100000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__tx_boost_en__SHIFT 0x14
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__tx_binary_ron_code_offset_MASK 0xc00000
+#define CMD_BUS_GLOBAL_FOR_TX_LANE3__tx_binary_ron_code_offset__SHIFT 0x16
+#define TX_DISP_RFU0_LANE0__rfu_value0_MASK 0xffffffff
+#define TX_DISP_RFU0_LANE0__rfu_value0__SHIFT 0x0
+#define TX_DISP_RFU0_LANE1__rfu_value0_MASK 0xffffffff
+#define TX_DISP_RFU0_LANE1__rfu_value0__SHIFT 0x0
+#define TX_DISP_RFU0_LANE2__rfu_value0_MASK 0xffffffff
+#define TX_DISP_RFU0_LANE2__rfu_value0__SHIFT 0x0
+#define TX_DISP_RFU0_LANE3__rfu_value0_MASK 0xffffffff
+#define TX_DISP_RFU0_LANE3__rfu_value0__SHIFT 0x0
+#define TX_DISP_RFU1_LANE0__rfu_value1_MASK 0xffffffff
+#define TX_DISP_RFU1_LANE0__rfu_value1__SHIFT 0x0
+#define TX_DISP_RFU1_LANE1__rfu_value1_MASK 0xffffffff
+#define TX_DISP_RFU1_LANE1__rfu_value1__SHIFT 0x0
+#define TX_DISP_RFU1_LANE2__rfu_value1_MASK 0xffffffff
+#define TX_DISP_RFU1_LANE2__rfu_value1__SHIFT 0x0
+#define TX_DISP_RFU1_LANE3__rfu_value1_MASK 0xffffffff
+#define TX_DISP_RFU1_LANE3__rfu_value1__SHIFT 0x0
+#define TX_DISP_RFU2_LANE0__rfu_value2_MASK 0xffffffff
+#define TX_DISP_RFU2_LANE0__rfu_value2__SHIFT 0x0
+#define TX_DISP_RFU2_LANE1__rfu_value2_MASK 0xffffffff
+#define TX_DISP_RFU2_LANE1__rfu_value2__SHIFT 0x0
+#define TX_DISP_RFU2_LANE2__rfu_value2_MASK 0xffffffff
+#define TX_DISP_RFU2_LANE2__rfu_value2__SHIFT 0x0
+#define TX_DISP_RFU2_LANE3__rfu_value2_MASK 0xffffffff
+#define TX_DISP_RFU2_LANE3__rfu_value2__SHIFT 0x0
+#define TX_DISP_RFU3_LANE0__rfu_value3_MASK 0xffffffff
+#define TX_DISP_RFU3_LANE0__rfu_value3__SHIFT 0x0
+#define TX_DISP_RFU3_LANE1__rfu_value3_MASK 0xffffffff
+#define TX_DISP_RFU3_LANE1__rfu_value3__SHIFT 0x0
+#define TX_DISP_RFU3_LANE2__rfu_value3_MASK 0xffffffff
+#define TX_DISP_RFU3_LANE2__rfu_value3__SHIFT 0x0
+#define TX_DISP_RFU3_LANE3__rfu_value3_MASK 0xffffffff
+#define TX_DISP_RFU3_LANE3__rfu_value3__SHIFT 0x0
+#define TX_DISP_RFU4_LANE0__rfu_value4_MASK 0xffffffff
+#define TX_DISP_RFU4_LANE0__rfu_value4__SHIFT 0x0
+#define TX_DISP_RFU4_LANE1__rfu_value4_MASK 0xffffffff
+#define TX_DISP_RFU4_LANE1__rfu_value4__SHIFT 0x0
+#define TX_DISP_RFU4_LANE2__rfu_value4_MASK 0xffffffff
+#define TX_DISP_RFU4_LANE2__rfu_value4__SHIFT 0x0
+#define TX_DISP_RFU4_LANE3__rfu_value4_MASK 0xffffffff
+#define TX_DISP_RFU4_LANE3__rfu_value4__SHIFT 0x0
+#define TX_DISP_RFU5_LANE0__rfu_value5_MASK 0xffffffff
+#define TX_DISP_RFU5_LANE0__rfu_value5__SHIFT 0x0
+#define TX_DISP_RFU5_LANE1__rfu_value5_MASK 0xffffffff
+#define TX_DISP_RFU5_LANE1__rfu_value5__SHIFT 0x0
+#define TX_DISP_RFU5_LANE2__rfu_value5_MASK 0xffffffff
+#define TX_DISP_RFU5_LANE2__rfu_value5__SHIFT 0x0
+#define TX_DISP_RFU5_LANE3__rfu_value5_MASK 0xffffffff
+#define TX_DISP_RFU5_LANE3__rfu_value5__SHIFT 0x0
+#define TX_DISP_RFU6_LANE0__rfu_value6_MASK 0xffffffff
+#define TX_DISP_RFU6_LANE0__rfu_value6__SHIFT 0x0
+#define TX_DISP_RFU6_LANE1__rfu_value6_MASK 0xffffffff
+#define TX_DISP_RFU6_LANE1__rfu_value6__SHIFT 0x0
+#define TX_DISP_RFU6_LANE2__rfu_value6_MASK 0xffffffff
+#define TX_DISP_RFU6_LANE2__rfu_value6__SHIFT 0x0
+#define TX_DISP_RFU6_LANE3__rfu_value6_MASK 0xffffffff
+#define TX_DISP_RFU6_LANE3__rfu_value6__SHIFT 0x0
+#define TX_DISP_RFU7_LANE0__rfu_value7_MASK 0xffffffff
+#define TX_DISP_RFU7_LANE0__rfu_value7__SHIFT 0x0
+#define TX_DISP_RFU7_LANE1__rfu_value7_MASK 0xffffffff
+#define TX_DISP_RFU7_LANE1__rfu_value7__SHIFT 0x0
+#define TX_DISP_RFU7_LANE2__rfu_value7_MASK 0xffffffff
+#define TX_DISP_RFU7_LANE2__rfu_value7__SHIFT 0x0
+#define TX_DISP_RFU7_LANE3__rfu_value7_MASK 0xffffffff
+#define TX_DISP_RFU7_LANE3__rfu_value7__SHIFT 0x0
+#define TX_DISP_RFU8_LANE0__rfu_value8_MASK 0xffffffff
+#define TX_DISP_RFU8_LANE0__rfu_value8__SHIFT 0x0
+#define TX_DISP_RFU8_LANE1__rfu_value8_MASK 0xffffffff
+#define TX_DISP_RFU8_LANE1__rfu_value8__SHIFT 0x0
+#define TX_DISP_RFU8_LANE2__rfu_value8_MASK 0xffffffff
+#define TX_DISP_RFU8_LANE2__rfu_value8__SHIFT 0x0
+#define TX_DISP_RFU8_LANE3__rfu_value8_MASK 0xffffffff
+#define TX_DISP_RFU8_LANE3__rfu_value8__SHIFT 0x0
+#define TX_DISP_RFU9_LANE0__rfu_value9_MASK 0xffffffff
+#define TX_DISP_RFU9_LANE0__rfu_value9__SHIFT 0x0
+#define TX_DISP_RFU9_LANE1__rfu_value9_MASK 0xffffffff
+#define TX_DISP_RFU9_LANE1__rfu_value9__SHIFT 0x0
+#define TX_DISP_RFU9_LANE2__rfu_value9_MASK 0xffffffff
+#define TX_DISP_RFU9_LANE2__rfu_value9__SHIFT 0x0
+#define TX_DISP_RFU9_LANE3__rfu_value9_MASK 0xffffffff
+#define TX_DISP_RFU9_LANE3__rfu_value9__SHIFT 0x0
+#define TX_DISP_RFU10_LANE0__rfu_value10_MASK 0xffffffff
+#define TX_DISP_RFU10_LANE0__rfu_value10__SHIFT 0x0
+#define TX_DISP_RFU10_LANE1__rfu_value10_MASK 0xffffffff
+#define TX_DISP_RFU10_LANE1__rfu_value10__SHIFT 0x0
+#define TX_DISP_RFU10_LANE2__rfu_value10_MASK 0xffffffff
+#define TX_DISP_RFU10_LANE2__rfu_value10__SHIFT 0x0
+#define TX_DISP_RFU10_LANE3__rfu_value10_MASK 0xffffffff
+#define TX_DISP_RFU10_LANE3__rfu_value10__SHIFT 0x0
+#define TX_DISP_RFU11_LANE0__rfu_value11_MASK 0xffffffff
+#define TX_DISP_RFU11_LANE0__rfu_value11__SHIFT 0x0
+#define TX_DISP_RFU11_LANE1__rfu_value11_MASK 0xffffffff
+#define TX_DISP_RFU11_LANE1__rfu_value11__SHIFT 0x0
+#define TX_DISP_RFU11_LANE2__rfu_value11_MASK 0xffffffff
+#define TX_DISP_RFU11_LANE2__rfu_value11__SHIFT 0x0
+#define TX_DISP_RFU11_LANE3__rfu_value11_MASK 0xffffffff
+#define TX_DISP_RFU11_LANE3__rfu_value11__SHIFT 0x0
+#define TX_DISP_RFU12_LANE0__rfu_value12_MASK 0xffffffff
+#define TX_DISP_RFU12_LANE0__rfu_value12__SHIFT 0x0
+#define TX_DISP_RFU12_LANE1__rfu_value12_MASK 0xffffffff
+#define TX_DISP_RFU12_LANE1__rfu_value12__SHIFT 0x0
+#define TX_DISP_RFU12_LANE2__rfu_value12_MASK 0xffffffff
+#define TX_DISP_RFU12_LANE2__rfu_value12__SHIFT 0x0
+#define TX_DISP_RFU12_LANE3__rfu_value12_MASK 0xffffffff
+#define TX_DISP_RFU12_LANE3__rfu_value12__SHIFT 0x0
+#define COMMON_MAR_DEEMPH_NOM__tx_margin_nom_MASK 0xff
+#define COMMON_MAR_DEEMPH_NOM__tx_margin_nom__SHIFT 0x0
+#define COMMON_MAR_DEEMPH_NOM__deemph_gen1_nom_MASK 0xff00
+#define COMMON_MAR_DEEMPH_NOM__deemph_gen1_nom__SHIFT 0x8
+#define COMMON_MAR_DEEMPH_NOM__deemph35_gen2_nom_MASK 0xff0000
+#define COMMON_MAR_DEEMPH_NOM__deemph35_gen2_nom__SHIFT 0x10
+#define COMMON_MAR_DEEMPH_NOM__deemph60_gen2_nom_MASK 0xff000000
+#define COMMON_MAR_DEEMPH_NOM__deemph60_gen2_nom__SHIFT 0x18
+#define COMMON_LANE_PWRMGMT__pgdelay_MASK 0xf
+#define COMMON_LANE_PWRMGMT__pgdelay__SHIFT 0x0
+#define COMMON_LANE_PWRMGMT__pgmask_MASK 0x3f0
+#define COMMON_LANE_PWRMGMT__pgmask__SHIFT 0x4
+#define COMMON_LANE_PWRMGMT__vprot_en_MASK 0x800
+#define COMMON_LANE_PWRMGMT__vprot_en__SHIFT 0xb
+#define COMMON_TXCNTRL__rdptr_rst_val_gen3_MASK 0x1f
+#define COMMON_TXCNTRL__rdptr_rst_val_gen3__SHIFT 0x0
+#define COMMON_TXCNTRL__clkgate_dis_MASK 0x20
+#define COMMON_TXCNTRL__clkgate_dis__SHIFT 0x5
+#define COMMON_TXCNTRL__slew_rate_ctl_gen1_MASK 0x1c0
+#define COMMON_TXCNTRL__slew_rate_ctl_gen1__SHIFT 0x6
+#define COMMON_TXCNTRL__slew_rate_ctl_gen2_MASK 0xe00
+#define COMMON_TXCNTRL__slew_rate_ctl_gen2__SHIFT 0x9
+#define COMMON_TXCNTRL__slew_rate_ctl_gen3_MASK 0x7000
+#define COMMON_TXCNTRL__slew_rate_ctl_gen3__SHIFT 0xc
+#define COMMON_TXCNTRL__dual_dvi_mstr_en_MASK 0x8000
+#define COMMON_TXCNTRL__dual_dvi_mstr_en__SHIFT 0xf
+#define COMMON_TXCNTRL__dual_dvi_en_MASK 0x10000
+#define COMMON_TXCNTRL__dual_dvi_en__SHIFT 0x10
+#define COMMON_TMDP__tmdp_spare_MASK 0xffffffff
+#define COMMON_TMDP__tmdp_spare__SHIFT 0x0
+#define COMMON_LANE_RESETS__lane_0_reset_l_MASK 0x1
+#define COMMON_LANE_RESETS__lane_0_reset_l__SHIFT 0x0
+#define COMMON_LANE_RESETS__lane_1_reset_l_MASK 0x2
+#define COMMON_LANE_RESETS__lane_1_reset_l__SHIFT 0x1
+#define COMMON_LANE_RESETS__lane_2_reset_l_MASK 0x4
+#define COMMON_LANE_RESETS__lane_2_reset_l__SHIFT 0x2
+#define COMMON_LANE_RESETS__lane_3_reset_l_MASK 0x8
+#define COMMON_LANE_RESETS__lane_3_reset_l__SHIFT 0x3
+#define COMMON_LANE_RESETS__lane_4_reset_l_MASK 0x10
+#define COMMON_LANE_RESETS__lane_4_reset_l__SHIFT 0x4
+#define COMMON_LANE_RESETS__lane_5_reset_l_MASK 0x20
+#define COMMON_LANE_RESETS__lane_5_reset_l__SHIFT 0x5
+#define COMMON_LANE_RESETS__lane_6_reset_l_MASK 0x40
+#define COMMON_LANE_RESETS__lane_6_reset_l__SHIFT 0x6
+#define COMMON_LANE_RESETS__lane_7_reset_l_MASK 0x80
+#define COMMON_LANE_RESETS__lane_7_reset_l__SHIFT 0x7
+#define COMMON_ZCALCODE_CTRL__zcalcode_override_MASK 0x1
+#define COMMON_ZCALCODE_CTRL__zcalcode_override__SHIFT 0x0
+#define COMMON_ZCALCODE_CTRL__tx_binary_code_override_val_MASK 0x3e
+#define COMMON_ZCALCODE_CTRL__tx_binary_code_override_val__SHIFT 0x1
+#define COMMON_ZCALCODE_CTRL__tx_driver_fifty_ohms_MASK 0x200000
+#define COMMON_ZCALCODE_CTRL__tx_driver_fifty_ohms__SHIFT 0x15
+#define COMMON_DISP_RFU1__rfu_value1_MASK 0xffffffff
+#define COMMON_DISP_RFU1__rfu_value1__SHIFT 0x0
+#define COMMON_DISP_RFU2__rfu_value2_MASK 0xffffffff
+#define COMMON_DISP_RFU2__rfu_value2__SHIFT 0x0
+#define COMMON_DISP_RFU3__rfu_value3_MASK 0xffffffff
+#define COMMON_DISP_RFU3__rfu_value3__SHIFT 0x0
+#define COMMON_DISP_RFU4__rfu_value4_MASK 0xffffffff
+#define COMMON_DISP_RFU4__rfu_value4__SHIFT 0x0
+#define COMMON_DISP_RFU5__rfu_value5_MASK 0xffffffff
+#define COMMON_DISP_RFU5__rfu_value5__SHIFT 0x0
+#define COMMON_DISP_RFU6__rfu_value6_MASK 0xffffffff
+#define COMMON_DISP_RFU6__rfu_value6__SHIFT 0x0
+#define COMMON_DISP_RFU7__rfu_value7_MASK 0xffffffff
+#define COMMON_DISP_RFU7__rfu_value7__SHIFT 0x0
+#define FREQ_CTRL0__fcw0_frac_MASK 0xffff
+#define FREQ_CTRL0__fcw0_frac__SHIFT 0x0
+#define FREQ_CTRL0__fcw0_int_MASK 0x1ff0000
+#define FREQ_CTRL0__fcw0_int__SHIFT 0x10
+#define FREQ_CTRL1__fcw1_frac_MASK 0xffff
+#define FREQ_CTRL1__fcw1_frac__SHIFT 0x0
+#define FREQ_CTRL1__fcw1_int_MASK 0x1ff0000
+#define FREQ_CTRL1__fcw1_int__SHIFT 0x10
+#define FREQ_CTRL2__fcw_denom_MASK 0xffff
+#define FREQ_CTRL2__fcw_denom__SHIFT 0x0
+#define FREQ_CTRL2__fcw_slew_frac_MASK 0xffff0000
+#define FREQ_CTRL2__fcw_slew_frac__SHIFT 0x10
+#define FREQ_CTRL3__refclk_div_MASK 0x3
+#define FREQ_CTRL3__refclk_div__SHIFT 0x0
+#define FREQ_CTRL3__vco_pre_div_MASK 0x18
+#define FREQ_CTRL3__vco_pre_div__SHIFT 0x3
+#define FREQ_CTRL3__fracn_en_MASK 0x40
+#define FREQ_CTRL3__fracn_en__SHIFT 0x6
+#define FREQ_CTRL3__ssc_en_MASK 0x100
+#define FREQ_CTRL3__ssc_en__SHIFT 0x8
+#define FREQ_CTRL3__fcw_sel_MASK 0x400
+#define FREQ_CTRL3__fcw_sel__SHIFT 0xa
+#define FREQ_CTRL3__freq_jump_en_MASK 0x1000
+#define FREQ_CTRL3__freq_jump_en__SHIFT 0xc
+#define FREQ_CTRL3__tdc_resolution_MASK 0xff0000
+#define FREQ_CTRL3__tdc_resolution__SHIFT 0x10
+#define FREQ_CTRL3__dpll_cfg_1_MASK 0xff000000
+#define FREQ_CTRL3__dpll_cfg_1__SHIFT 0x18
+#define BW_CTRL_COARSE__gi_coarse_mant_MASK 0x3
+#define BW_CTRL_COARSE__gi_coarse_mant__SHIFT 0x0
+#define BW_CTRL_COARSE__gi_coarse_exp_MASK 0x3c
+#define BW_CTRL_COARSE__gi_coarse_exp__SHIFT 0x2
+#define BW_CTRL_COARSE__gp_coarse_mant_MASK 0x780
+#define BW_CTRL_COARSE__gp_coarse_mant__SHIFT 0x7
+#define BW_CTRL_COARSE__gp_coarse_exp_MASK 0xf000
+#define BW_CTRL_COARSE__gp_coarse_exp__SHIFT 0xc
+#define BW_CTRL_COARSE__nctl_coarse_res_MASK 0x7e0000
+#define BW_CTRL_COARSE__nctl_coarse_res__SHIFT 0x11
+#define BW_CTRL_COARSE__nctl_coarse_frac_res_MASK 0x3000000
+#define BW_CTRL_COARSE__nctl_coarse_frac_res__SHIFT 0x18
+#define BW_CTRL_FINE__dpll_cfg_3_MASK 0x3ff
+#define BW_CTRL_FINE__dpll_cfg_3__SHIFT 0x0
+#define CAL_CTRL__bypass_freq_lock_MASK 0x1
+#define CAL_CTRL__bypass_freq_lock__SHIFT 0x0
+#define CAL_CTRL__tdc_cal_en_MASK 0x2
+#define CAL_CTRL__tdc_cal_en__SHIFT 0x1
+#define CAL_CTRL__tdc_cal_ctrl_MASK 0x1f8
+#define CAL_CTRL__tdc_cal_ctrl__SHIFT 0x3
+#define CAL_CTRL__meas_win_sel_MASK 0x600
+#define CAL_CTRL__meas_win_sel__SHIFT 0x9
+#define CAL_CTRL__kdco_cal_dis_MASK 0x800
+#define CAL_CTRL__kdco_cal_dis__SHIFT 0xb
+#define CAL_CTRL__kdco_ratio_MASK 0x1fe000
+#define CAL_CTRL__kdco_ratio__SHIFT 0xd
+#define CAL_CTRL__kdco_incr_cal_dis_MASK 0x400000
+#define CAL_CTRL__kdco_incr_cal_dis__SHIFT 0x16
+#define CAL_CTRL__nctl_adj_dis_MASK 0x800000
+#define CAL_CTRL__nctl_adj_dis__SHIFT 0x17
+#define CAL_CTRL__refclk_rate_MASK 0xff000000
+#define CAL_CTRL__refclk_rate__SHIFT 0x18
+#define LOOP_CTRL__fbdiv_mask_en_MASK 0x1
+#define LOOP_CTRL__fbdiv_mask_en__SHIFT 0x0
+#define LOOP_CTRL__fb_slip_dis_MASK 0x4
+#define LOOP_CTRL__fb_slip_dis__SHIFT 0x2
+#define LOOP_CTRL__clk_tdc_sel_MASK 0x30
+#define LOOP_CTRL__clk_tdc_sel__SHIFT 0x4
+#define LOOP_CTRL__clk_nctl_sel_MASK 0x180
+#define LOOP_CTRL__clk_nctl_sel__SHIFT 0x7
+#define LOOP_CTRL__sig_del_patt_sel_MASK 0x400
+#define LOOP_CTRL__sig_del_patt_sel__SHIFT 0xa
+#define LOOP_CTRL__nctl_sig_del_dis_MASK 0x1000
+#define LOOP_CTRL__nctl_sig_del_dis__SHIFT 0xc
+#define LOOP_CTRL__fbclk_track_refclk_MASK 0x4000
+#define LOOP_CTRL__fbclk_track_refclk__SHIFT 0xe
+#define LOOP_CTRL__prbs_en_MASK 0x10000
+#define LOOP_CTRL__prbs_en__SHIFT 0x10
+#define LOOP_CTRL__tdc_clk_gate_en_MASK 0x40000
+#define LOOP_CTRL__tdc_clk_gate_en__SHIFT 0x12
+#define LOOP_CTRL__phase_offset_MASK 0x7f00000
+#define LOOP_CTRL__phase_offset__SHIFT 0x14
+#define VREG_CFG__bleeder_ac_MASK 0x1
+#define VREG_CFG__bleeder_ac__SHIFT 0x0
+#define VREG_CFG__bleeder_en_MASK 0x2
+#define VREG_CFG__bleeder_en__SHIFT 0x1
+#define VREG_CFG__is_1p2_MASK 0x4
+#define VREG_CFG__is_1p2__SHIFT 0x2
+#define VREG_CFG__reg_obs_sel_MASK 0x18
+#define VREG_CFG__reg_obs_sel__SHIFT 0x3
+#define VREG_CFG__reg_on_mode_MASK 0x60
+#define VREG_CFG__reg_on_mode__SHIFT 0x5
+#define VREG_CFG__rlad_tap_sel_MASK 0x780
+#define VREG_CFG__rlad_tap_sel__SHIFT 0x7
+#define VREG_CFG__reg_off_hi_MASK 0x800
+#define VREG_CFG__reg_off_hi__SHIFT 0xb
+#define VREG_CFG__reg_off_lo_MASK 0x1000
+#define VREG_CFG__reg_off_lo__SHIFT 0xc
+#define VREG_CFG__scale_driver_MASK 0x6000
+#define VREG_CFG__scale_driver__SHIFT 0xd
+#define VREG_CFG__sel_bump_MASK 0x8000
+#define VREG_CFG__sel_bump__SHIFT 0xf
+#define VREG_CFG__sel_rladder_x_MASK 0x10000
+#define VREG_CFG__sel_rladder_x__SHIFT 0x10
+#define VREG_CFG__short_rc_filt_x_MASK 0x20000
+#define VREG_CFG__short_rc_filt_x__SHIFT 0x11
+#define VREG_CFG__vref_pwr_on_MASK 0x40000
+#define VREG_CFG__vref_pwr_on__SHIFT 0x12
+#define VREG_CFG__dpll_cfg_2_MASK 0xff00000
+#define VREG_CFG__dpll_cfg_2__SHIFT 0x14
+#define OBSERVE0__lock_det_tdc_steps_MASK 0x1f
+#define OBSERVE0__lock_det_tdc_steps__SHIFT 0x0
+#define OBSERVE0__clear_sticky_lock_MASK 0x40
+#define OBSERVE0__clear_sticky_lock__SHIFT 0x6
+#define OBSERVE0__lock_det_dis_MASK 0x100
+#define OBSERVE0__lock_det_dis__SHIFT 0x8
+#define OBSERVE0__dco_cfg_MASK 0x3fc00
+#define OBSERVE0__dco_cfg__SHIFT 0xa
+#define OBSERVE0__anaobs_sel_MASK 0xe00000
+#define OBSERVE0__anaobs_sel__SHIFT 0x15
+#define OBSERVE1__digobs_sel_MASK 0xf
+#define OBSERVE1__digobs_sel__SHIFT 0x0
+#define OBSERVE1__digobs_trig_sel_MASK 0x1e0
+#define OBSERVE1__digobs_trig_sel__SHIFT 0x5
+#define OBSERVE1__digobs_div_MASK 0xc00
+#define OBSERVE1__digobs_div__SHIFT 0xa
+#define OBSERVE1__digobs_trig_div_MASK 0x6000
+#define OBSERVE1__digobs_trig_div__SHIFT 0xd
+#define OBSERVE1__lock_timer_MASK 0x3fff0000
+#define OBSERVE1__lock_timer__SHIFT 0x10
+#define DFT_OUT__dft_data_MASK 0xffffffff
+#define DFT_OUT__dft_data__SHIFT 0x0
+#define PLL_WRAP_CNTRL1__wrap_cfg_sel_clk_MASK 0x3
+#define PLL_WRAP_CNTRL1__wrap_cfg_sel_clk__SHIFT 0x0
+#define PLL_WRAP_CNTRL__wrap_cfg_pll_freq_programming_ovveride_MASK 0x1
+#define PLL_WRAP_CNTRL__wrap_cfg_pll_freq_programming_ovveride__SHIFT 0x0
+#define PLL_WRAP_CNTRL__wrap_cfg_pll_pwr_state_ovrride_MASK 0x2
+#define PLL_WRAP_CNTRL__wrap_cfg_pll_pwr_state_ovrride__SHIFT 0x1
+#define PLL_WRAP_CNTRL__wrap_cfg_pll_pwr_state_MASK 0xc
+#define PLL_WRAP_CNTRL__wrap_cfg_pll_pwr_state__SHIFT 0x2
+#define PLL_WRAP_CNTRL__wrap_cfg_tx_pdiv_val_MASK 0xe0
+#define PLL_WRAP_CNTRL__wrap_cfg_tx_pdiv_val__SHIFT 0x5
+#define PLL_WRAP_CNTRL__wrap_cfg_tx_pixdiv_val_MASK 0x100
+#define PLL_WRAP_CNTRL__wrap_cfg_tx_pixdiv_val__SHIFT 0x8
+#define PLL_WRAP_CNTRL__wrap_cfg_cml_cmos_sel_MASK 0x400
+#define PLL_WRAP_CNTRL__wrap_cfg_cml_cmos_sel__SHIFT 0xa
+#define PLL_WRAP_CNTRL__wrap_cfg_pll_rdy_MASK 0x2000
+#define PLL_WRAP_CNTRL__wrap_cfg_pll_rdy__SHIFT 0xd
+#define PLL_WRAP_CNTRL__wrap_cfg_pll_update_MASK 0x4000
+#define PLL_WRAP_CNTRL__wrap_cfg_pll_update__SHIFT 0xe
+#define PLL_WRAP_CNTRL__wrap_cfg_ref_values_chg_MASK 0x8000
+#define PLL_WRAP_CNTRL__wrap_cfg_ref_values_chg__SHIFT 0xf
+#define PLL_WRAP_CNTRL__wrap_cfg_clk_gate_w_rdy_MASK 0x10000
+#define PLL_WRAP_CNTRL__wrap_cfg_clk_gate_w_rdy__SHIFT 0x10
+#define PLL_WRAP_CNTRL__wrap_cfg_pll_dsm_sel_MASK 0xe0000
+#define PLL_WRAP_CNTRL__wrap_cfg_pll_dsm_sel__SHIFT 0x11
+#define PPLL_VREG_CFG__pw_pc_bleeder_ac_MASK 0x1
+#define PPLL_VREG_CFG__pw_pc_bleeder_ac__SHIFT 0x0
+#define PPLL_VREG_CFG__pw_pc_bleeder_en_MASK 0x2
+#define PPLL_VREG_CFG__pw_pc_bleeder_en__SHIFT 0x1
+#define PPLL_VREG_CFG__pw_pc_is_1p2_MASK 0x4
+#define PPLL_VREG_CFG__pw_pc_is_1p2__SHIFT 0x2
+#define PPLL_VREG_CFG__pw_pc_reg_obs_sel_MASK 0x18
+#define PPLL_VREG_CFG__pw_pc_reg_obs_sel__SHIFT 0x3
+#define PPLL_VREG_CFG__pw_pc_reg_on_mode_MASK 0x60
+#define PPLL_VREG_CFG__pw_pc_reg_on_mode__SHIFT 0x5
+#define PPLL_VREG_CFG__pw_pc_rlad_tap_sel_MASK 0x780
+#define PPLL_VREG_CFG__pw_pc_rlad_tap_sel__SHIFT 0x7
+#define PPLL_VREG_CFG__pw_pc_reg_off_hi_MASK 0x800
+#define PPLL_VREG_CFG__pw_pc_reg_off_hi__SHIFT 0xb
+#define PPLL_VREG_CFG__pw_pc_reg_off_lo_MASK 0x1000
+#define PPLL_VREG_CFG__pw_pc_reg_off_lo__SHIFT 0xc
+#define PPLL_VREG_CFG__pw_pc_scale_driver_MASK 0x6000
+#define PPLL_VREG_CFG__pw_pc_scale_driver__SHIFT 0xd
+#define PPLL_VREG_CFG__pw_pc_sel_bump_MASK 0x8000
+#define PPLL_VREG_CFG__pw_pc_sel_bump__SHIFT 0xf
+#define PPLL_VREG_CFG__pw_pc_sel_rladder_x_MASK 0x10000
+#define PPLL_VREG_CFG__pw_pc_sel_rladder_x__SHIFT 0x10
+#define PPLL_VREG_CFG__pw_pc_short_rc_filt_x_MASK 0x20000
+#define PPLL_VREG_CFG__pw_pc_short_rc_filt_x__SHIFT 0x11
+#define PPLL_VREG_CFG__pw_pc_vref_pwr_on_MASK 0x40000
+#define PPLL_VREG_CFG__pw_pc_vref_pwr_on__SHIFT 0x12
+#define PPLL_VREG_CFG__pw_pc_dpll_cfg_2_MASK 0xff00000
+#define PPLL_VREG_CFG__pw_pc_dpll_cfg_2__SHIFT 0x14
+#define PPLL_MODE_CNTL__pw_pc_refclk_gate_dis_MASK 0x1
+#define PPLL_MODE_CNTL__pw_pc_refclk_gate_dis__SHIFT 0x0
+#define PPLL_MODE_CNTL__pw_pc_multi_phase_en_MASK 0xf00
+#define PPLL_MODE_CNTL__pw_pc_multi_phase_en__SHIFT 0x8
+#define PPLL_MODE_CNTL__reg_tmg_pwr_state_MASK 0x30000
+#define PPLL_MODE_CNTL__reg_tmg_pwr_state__SHIFT 0x10
+#define PPLL_FREQ_CTRL0__reg_tmg_fcw0_frac_MASK 0xffff
+#define PPLL_FREQ_CTRL0__reg_tmg_fcw0_frac__SHIFT 0x0
+#define PPLL_FREQ_CTRL0__reg_tmg_fcw0_int_MASK 0x1ff0000
+#define PPLL_FREQ_CTRL0__reg_tmg_fcw0_int__SHIFT 0x10
+#define PPLL_FREQ_CTRL1__reg_tmg_fcw1_frac_MASK 0xffff
+#define PPLL_FREQ_CTRL1__reg_tmg_fcw1_frac__SHIFT 0x0
+#define PPLL_FREQ_CTRL1__reg_tmg_fcw1_int_MASK 0x1ff0000
+#define PPLL_FREQ_CTRL1__reg_tmg_fcw1_int__SHIFT 0x10
+#define PPLL_FREQ_CTRL2__reg_tmg_fcw_denom_MASK 0xffff
+#define PPLL_FREQ_CTRL2__reg_tmg_fcw_denom__SHIFT 0x0
+#define PPLL_FREQ_CTRL2__reg_tmg_fcw_slew_frac_MASK 0xffff0000
+#define PPLL_FREQ_CTRL2__reg_tmg_fcw_slew_frac__SHIFT 0x10
+#define PPLL_FREQ_CTRL3__reg_tmg_refclk_div_MASK 0x3
+#define PPLL_FREQ_CTRL3__reg_tmg_refclk_div__SHIFT 0x0
+#define PPLL_FREQ_CTRL3__reg_tmg_vco_pre_div_MASK 0x18
+#define PPLL_FREQ_CTRL3__reg_tmg_vco_pre_div__SHIFT 0x3
+#define PPLL_FREQ_CTRL3__reg_tmg_fracn_en_MASK 0x40
+#define PPLL_FREQ_CTRL3__reg_tmg_fracn_en__SHIFT 0x6
+#define PPLL_FREQ_CTRL3__reg_tmg_ssc_en_MASK 0x100
+#define PPLL_FREQ_CTRL3__reg_tmg_ssc_en__SHIFT 0x8
+#define PPLL_FREQ_CTRL3__reg_tmg_fcw_sel_MASK 0x400
+#define PPLL_FREQ_CTRL3__reg_tmg_fcw_sel__SHIFT 0xa
+#define PPLL_FREQ_CTRL3__reg_tmg_freq_jump_en_MASK 0x1000
+#define PPLL_FREQ_CTRL3__reg_tmg_freq_jump_en__SHIFT 0xc
+#define PPLL_FREQ_CTRL3__reg_tmg_tdc_resol_MASK 0xff0000
+#define PPLL_FREQ_CTRL3__reg_tmg_tdc_resol__SHIFT 0x10
+#define PPLL_FREQ_CTRL3__pw_pc_dpll_cfg_1_MASK 0xff000000
+#define PPLL_FREQ_CTRL3__pw_pc_dpll_cfg_1__SHIFT 0x18
+#define PPLL_BW_CTRL_COARSE__reg_tmg_gi_crse_mant_MASK 0x3
+#define PPLL_BW_CTRL_COARSE__reg_tmg_gi_crse_mant__SHIFT 0x0
+#define PPLL_BW_CTRL_COARSE__reg_tmg_gi_crse_exp_MASK 0x3c
+#define PPLL_BW_CTRL_COARSE__reg_tmg_gi_crse_exp__SHIFT 0x2
+#define PPLL_BW_CTRL_COARSE__reg_tmg_gp_crse_mant_MASK 0x780
+#define PPLL_BW_CTRL_COARSE__reg_tmg_gp_crse_mant__SHIFT 0x7
+#define PPLL_BW_CTRL_COARSE__reg_tmg_gp_crse_exp_MASK 0xf000
+#define PPLL_BW_CTRL_COARSE__reg_tmg_gp_crse_exp__SHIFT 0xc
+#define PPLL_BW_CTRL_COARSE__reg_tmg_nctl_crse_res_MASK 0x7e0000
+#define PPLL_BW_CTRL_COARSE__reg_tmg_nctl_crse_res__SHIFT 0x11
+#define PPLL_BW_CTRL_COARSE__reg_tmg_nctl_crse_frac_res_MASK 0x3000000
+#define PPLL_BW_CTRL_COARSE__reg_tmg_nctl_crse_frac_res__SHIFT 0x18
+#define PPLL_BW_CTRL_FINE__pw_pc_dpll_cfg_3_MASK 0x3ff
+#define PPLL_BW_CTRL_FINE__pw_pc_dpll_cfg_3__SHIFT 0x0
+#define PPLL_CAL_CTRL__pw_pc_bypass_freq_lock_MASK 0x1
+#define PPLL_CAL_CTRL__pw_pc_bypass_freq_lock__SHIFT 0x0
+#define PPLL_CAL_CTRL__pw_pc_tdc_cal_en_MASK 0x2
+#define PPLL_CAL_CTRL__pw_pc_tdc_cal_en__SHIFT 0x1
+#define PPLL_CAL_CTRL__pw_pc_tdc_cal_ctrl_MASK 0x1f8
+#define PPLL_CAL_CTRL__pw_pc_tdc_cal_ctrl__SHIFT 0x3
+#define PPLL_CAL_CTRL__pw_pc_meas_win_sel_MASK 0x600
+#define PPLL_CAL_CTRL__pw_pc_meas_win_sel__SHIFT 0x9
+#define PPLL_CAL_CTRL__pw_pc_kdco_cal_dis_MASK 0x800
+#define PPLL_CAL_CTRL__pw_pc_kdco_cal_dis__SHIFT 0xb
+#define PPLL_CAL_CTRL__pw_pc_kdco_ratio_MASK 0x1fe000
+#define PPLL_CAL_CTRL__pw_pc_kdco_ratio__SHIFT 0xd
+#define PPLL_CAL_CTRL__pw_pc_kdco_incr_cal_dis_MASK 0x400000
+#define PPLL_CAL_CTRL__pw_pc_kdco_incr_cal_dis__SHIFT 0x16
+#define PPLL_CAL_CTRL__pw_pc_nctl_adj_dis_MASK 0x800000
+#define PPLL_CAL_CTRL__pw_pc_nctl_adj_dis__SHIFT 0x17
+#define PPLL_CAL_CTRL__pw_pc_refclk_rate_MASK 0xff000000
+#define PPLL_CAL_CTRL__pw_pc_refclk_rate__SHIFT 0x18
+#define PPLL_LOOP_CTRL__pw_pc_fbdiv_mask_en_MASK 0x1
+#define PPLL_LOOP_CTRL__pw_pc_fbdiv_mask_en__SHIFT 0x0
+#define PPLL_LOOP_CTRL__pw_pc_fb_slip_dis_MASK 0x4
+#define PPLL_LOOP_CTRL__pw_pc_fb_slip_dis__SHIFT 0x2
+#define PPLL_LOOP_CTRL__pw_pc_clk_tdc_sel_MASK 0x30
+#define PPLL_LOOP_CTRL__pw_pc_clk_tdc_sel__SHIFT 0x4
+#define PPLL_LOOP_CTRL__pw_pc_clk_nctl_sel_MASK 0x180
+#define PPLL_LOOP_CTRL__pw_pc_clk_nctl_sel__SHIFT 0x7
+#define PPLL_LOOP_CTRL__pw_pc_sig_del_patt_sel_MASK 0x400
+#define PPLL_LOOP_CTRL__pw_pc_sig_del_patt_sel__SHIFT 0xa
+#define PPLL_LOOP_CTRL__pw_pc_nctl_sig_del_dis_MASK 0x1000
+#define PPLL_LOOP_CTRL__pw_pc_nctl_sig_del_dis__SHIFT 0xc
+#define PPLL_LOOP_CTRL__pw_pc_fbclk_track_refclk_MASK 0x4000
+#define PPLL_LOOP_CTRL__pw_pc_fbclk_track_refclk__SHIFT 0xe
+#define PPLL_LOOP_CTRL__pw_pc_prbs_en_MASK 0x10000
+#define PPLL_LOOP_CTRL__pw_pc_prbs_en__SHIFT 0x10
+#define PPLL_LOOP_CTRL__pw_pc_tdc_clk_gate_en_MASK 0x40000
+#define PPLL_LOOP_CTRL__pw_pc_tdc_clk_gate_en__SHIFT 0x12
+#define PPLL_LOOP_CTRL__pw_pc_phase_offset_MASK 0x7f00000
+#define PPLL_LOOP_CTRL__pw_pc_phase_offset__SHIFT 0x14
+#define PPLL_REFCLK_CNTL__regs_pw_refclk0_recv_en_MASK 0x1
+#define PPLL_REFCLK_CNTL__regs_pw_refclk0_recv_en__SHIFT 0x0
+#define PPLL_REFCLK_CNTL__regs_pw_refclk1_recv_en_MASK 0x2
+#define PPLL_REFCLK_CNTL__regs_pw_refclk1_recv_en__SHIFT 0x1
+#define PPLL_REFCLK_CNTL__regs_pw_refclk2_recv_en_MASK 0x4
+#define PPLL_REFCLK_CNTL__regs_pw_refclk2_recv_en__SHIFT 0x2
+#define PPLL_REFCLK_CNTL__regs_pw_refclk3_recv_en_MASK 0x8
+#define PPLL_REFCLK_CNTL__regs_pw_refclk3_recv_en__SHIFT 0x3
+#define PPLL_REFCLK_CNTL__regs_pw_refclk0_recv_sel_MASK 0x100
+#define PPLL_REFCLK_CNTL__regs_pw_refclk0_recv_sel__SHIFT 0x8
+#define PPLL_REFCLK_CNTL__regs_pw_refclk1_recv_sel_MASK 0x200
+#define PPLL_REFCLK_CNTL__regs_pw_refclk1_recv_sel__SHIFT 0x9
+#define PPLL_REFCLK_CNTL__regs_pw_refclk2_recv_sel_MASK 0x400
+#define PPLL_REFCLK_CNTL__regs_pw_refclk2_recv_sel__SHIFT 0xa
+#define PPLL_REFCLK_CNTL__regs_pw_refclk3_recv_sel_MASK 0x800
+#define PPLL_REFCLK_CNTL__regs_pw_refclk3_recv_sel__SHIFT 0xb
+#define PPLL_REFCLK_CNTL__regs_pw_refdivsrc_MASK 0xc000
+#define PPLL_REFCLK_CNTL__regs_pw_refdivsrc__SHIFT 0xe
+#define PPLL_REFCLK_CNTL__regs_pw_ref2core_sel_MASK 0x10000
+#define PPLL_REFCLK_CNTL__regs_pw_ref2core_sel__SHIFT 0x10
+#define PPLL_CLKOUT_CNTL__regs_pw_pixclk_pre_pdivsel_MASK 0x100
+#define PPLL_CLKOUT_CNTL__regs_pw_pixclk_pre_pdivsel__SHIFT 0x8
+#define PPLL_CLKOUT_CNTL__regs_pw_pixclk_pdivsel_MASK 0x200
+#define PPLL_CLKOUT_CNTL__regs_pw_pixclk_pdivsel__SHIFT 0x9
+#define PPLL_CLKOUT_CNTL__regs_pw_dvoclk_pre_pdivsel_MASK 0x400
+#define PPLL_CLKOUT_CNTL__regs_pw_dvoclk_pre_pdivsel__SHIFT 0xa
+#define PPLL_CLKOUT_CNTL__regs_pw_dvoclk_pdivsel_MASK 0x800
+#define PPLL_CLKOUT_CNTL__regs_pw_dvoclk_pdivsel__SHIFT 0xb
+#define PPLL_CLKOUT_CNTL__regs_pw_idclk_en_MASK 0x1000
+#define PPLL_CLKOUT_CNTL__regs_pw_idclk_en__SHIFT 0xc
+#define PPLL_CLKOUT_CNTL__regs_pw_idclk_pre_pdivsel_MASK 0x2000
+#define PPLL_CLKOUT_CNTL__regs_pw_idclk_pre_pdivsel__SHIFT 0xd
+#define PPLL_CLKOUT_CNTL__regs_pw_idclk_pdivsel_MASK 0x4000
+#define PPLL_CLKOUT_CNTL__regs_pw_idclk_pdivsel__SHIFT 0xe
+#define PPLL_CLKOUT_CNTL__regs_pw_idclk_obs_sel_MASK 0x8000
+#define PPLL_CLKOUT_CNTL__regs_pw_idclk_obs_sel__SHIFT 0xf
+#define PPLL_CLKOUT_CNTL__regs_pw_refclk_sel_MASK 0x30000
+#define PPLL_CLKOUT_CNTL__regs_pw_refclk_sel__SHIFT 0x10
+#define PPLL_CLKOUT_CNTL__regs_cc_resetb_MASK 0x100000
+#define PPLL_CLKOUT_CNTL__regs_cc_resetb__SHIFT 0x14
+#define PPLL_DFT_CNTL__regs_pw_obs_en_MASK 0x1
+#define PPLL_DFT_CNTL__regs_pw_obs_en__SHIFT 0x0
+#define PPLL_DFT_CNTL__regs_pw_obs_div_sel_1_MASK 0x6
+#define PPLL_DFT_CNTL__regs_pw_obs_div_sel_1__SHIFT 0x1
+#define PPLL_DFT_CNTL__regs_pw_obs_clk_sel_1_MASK 0xf0
+#define PPLL_DFT_CNTL__regs_pw_obs_clk_sel_1__SHIFT 0x4
+#define PPLL_DFT_CNTL__regs_pw_obs_clk_sel_2_MASK 0xf00
+#define PPLL_DFT_CNTL__regs_pw_obs_clk_sel_2__SHIFT 0x8
+#define PPLL_DFT_CNTL__regs_pw_obs_sel_MASK 0x3000
+#define PPLL_DFT_CNTL__regs_pw_obs_sel__SHIFT 0xc
+#define PPLL_ANALOG_CNTL__regs_pw_spare_MASK 0xff
+#define PPLL_ANALOG_CNTL__regs_pw_spare__SHIFT 0x0
+#define PPLL_POSTDIV__reg_tmg_postdiv_MASK 0xf00
+#define PPLL_POSTDIV__reg_tmg_postdiv__SHIFT 0x8
+#define PPLL_POSTDIV__reg_tmg_pixclk_pdiv2_MASK 0x1000
+#define PPLL_POSTDIV__reg_tmg_pixclk_pdiv2__SHIFT 0xc
+#define PPLL_DEBUG0__pw_pc_phase_jump_trig_MASK 0x2
+#define PPLL_DEBUG0__pw_pc_phase_jump_trig__SHIFT 0x1
+#define PPLL_DEBUG0__pw_pc_fine_tdc_dis_MASK 0x4
+#define PPLL_DEBUG0__pw_pc_fine_tdc_dis__SHIFT 0x2
+#define PPLL_DEBUG0__pw_pc_coarse_tdc_dis_MASK 0x8
+#define PPLL_DEBUG0__pw_pc_coarse_tdc_dis__SHIFT 0x3
+#define PPLL_DEBUG0__pw_pc_alt_nctl_en_MASK 0x10
+#define PPLL_DEBUG0__pw_pc_alt_nctl_en__SHIFT 0x4
+#define PPLL_DEBUG0__pw_pc_alt_nctl_MASK 0x1ffffe0
+#define PPLL_DEBUG0__pw_pc_alt_nctl__SHIFT 0x5
+#define PPLL_DEBUG0__pw_pc_nctl_coarse_step_dis_MASK 0x2000000
+#define PPLL_DEBUG0__pw_pc_nctl_coarse_step_dis__SHIFT 0x19
+#define PPLL_DEBUG0__pw_pc_trig_coarse_step_MASK 0x4000000
+#define PPLL_DEBUG0__pw_pc_trig_coarse_step__SHIFT 0x1a
+#define PPLL_DEBUG0__pw_pc_dft_sel_MASK 0x38000000
+#define PPLL_DEBUG0__pw_pc_dft_sel__SHIFT 0x1b
+#define PPLL_DEBUG0__pw_pc_dft_capture_MASK 0x40000000
+#define PPLL_DEBUG0__pw_pc_dft_capture__SHIFT 0x1e
+#define PPLL_OBSERVE0__pw_pc_lock_det_tdc_steps_MASK 0x1f
+#define PPLL_OBSERVE0__pw_pc_lock_det_tdc_steps__SHIFT 0x0
+#define PPLL_OBSERVE0__pw_pc_clear_sticky_lock_MASK 0x40
+#define PPLL_OBSERVE0__pw_pc_clear_sticky_lock__SHIFT 0x6
+#define PPLL_OBSERVE0__pw_pc_lock_det_dis_MASK 0x100
+#define PPLL_OBSERVE0__pw_pc_lock_det_dis__SHIFT 0x8
+#define PPLL_OBSERVE0__pw_pc_dco_cfg_MASK 0x3fc00
+#define PPLL_OBSERVE0__pw_pc_dco_cfg__SHIFT 0xa
+#define PPLL_OBSERVE0__pw_pc_anaobs_sel_MASK 0xe00000
+#define PPLL_OBSERVE0__pw_pc_anaobs_sel__SHIFT 0x15
+#define PPLL_OBSERVE1__pw_pc_digobs_sel_MASK 0xf
+#define PPLL_OBSERVE1__pw_pc_digobs_sel__SHIFT 0x0
+#define PPLL_OBSERVE1__pw_pc_digobs_trig_sel_MASK 0x1e0
+#define PPLL_OBSERVE1__pw_pc_digobs_trig_sel__SHIFT 0x5
+#define PPLL_OBSERVE1__pw_pc_digobs_div_MASK 0xc00
+#define PPLL_OBSERVE1__pw_pc_digobs_div__SHIFT 0xa
+#define PPLL_OBSERVE1__pw_pc_digobs_trig_div_MASK 0x3000
+#define PPLL_OBSERVE1__pw_pc_digobs_trig_div__SHIFT 0xc
+#define PPLL_OBSERVE1__reg_tmg_lock_timer_MASK 0x3fff0000
+#define PPLL_OBSERVE1__reg_tmg_lock_timer__SHIFT 0x10
+#define PPLL_UPDATE_CNTL__reg_tmg_PLL_UPDATE_LOCK_MASK 0x4
+#define PPLL_UPDATE_CNTL__reg_tmg_PLL_UPDATE_LOCK__SHIFT 0x2
+#define PPLL_UPDATE_CNTL__reg_tmg_PLL_UPDATE_POINT_MASK 0x8
+#define PPLL_UPDATE_CNTL__reg_tmg_PLL_UPDATE_POINT__SHIFT 0x3
+#define PPLL_UPDATE_CNTL__tmg_reg_UPDATE_PENDING_MASK 0x100
+#define PPLL_UPDATE_CNTL__tmg_reg_UPDATE_PENDING__SHIFT 0x8
+#define PPLL_UPDATE_CNTL__pc_pw_pll_rdy_MASK 0x200
+#define PPLL_UPDATE_CNTL__pc_pw_pll_rdy__SHIFT 0x9
+#define PPLL_UPDATE_CNTL__TieLow1_MASK 0x10000
+#define PPLL_UPDATE_CNTL__TieLow1__SHIFT 0x10
+#define PPLL_OBSERVE0_OUT__disppll_core_obsout_MASK 0xffffffff
+#define PPLL_OBSERVE0_OUT__disppll_core_obsout__SHIFT 0x0
+#define PPLL_STATUS_DEBUG1__dbg_pll_rdy_MASK 0x1
+#define PPLL_STATUS_DEBUG1__dbg_pll_rdy__SHIFT 0x0
+#define PPLL_STATUS_DEBUG1__core_disppll_pwr_ok_vddp_MASK 0x2
+#define PPLL_STATUS_DEBUG1__core_disppll_pwr_ok_vddp__SHIFT 0x1
+#define PPLL_STATUS_DEBUG1__core_disppll_rcu_dc_resetb_vddp_MASK 0x4
+#define PPLL_STATUS_DEBUG1__core_disppll_rcu_dc_resetb_vddp__SHIFT 0x2
+#define PPLL_DEBUG_MUX_CNTL__DEBUG_BUS_MUX_SEL_MASK 0x1f
+#define PPLL_DEBUG_MUX_CNTL__DEBUG_BUS_MUX_SEL__SHIFT 0x0
+#define PPLL_DIV_UPDATE_DEBUG__TieLow2_MASK 0x1
+#define PPLL_DIV_UPDATE_DEBUG__TieLow2__SHIFT 0x0
+#define PPLL_DIV_UPDATE_DEBUG__tmg_reg_FB_DIV_CHANGED_MASK 0x2
+#define PPLL_DIV_UPDATE_DEBUG__tmg_reg_FB_DIV_CHANGED__SHIFT 0x1
+#define PPLL_DIV_UPDATE_DEBUG__dbg_UPDATE_PENDING_MASK 0x4
+#define PPLL_DIV_UPDATE_DEBUG__dbg_UPDATE_PENDING__SHIFT 0x2
+#define PPLL_DIV_UPDATE_DEBUG__tmg_reg_CURRENT_STATE_MASK 0x18
+#define PPLL_DIV_UPDATE_DEBUG__tmg_reg_CURRENT_STATE__SHIFT 0x3
+#define PPLL_DIV_UPDATE_DEBUG__tmg_reg_UPDATE_ENABLE_MASK 0x20
+#define PPLL_DIV_UPDATE_DEBUG__tmg_reg_UPDATE_ENABLE__SHIFT 0x5
+#define PPLL_DIV_UPDATE_DEBUG__tmg_reg_UPDATE_REQ_MASK 0x40
+#define PPLL_DIV_UPDATE_DEBUG__tmg_reg_UPDATE_REQ__SHIFT 0x6
+#define PPLL_DIV_UPDATE_DEBUG__tmg_reg_UPDATE_ACK_MASK 0x80
+#define PPLL_DIV_UPDATE_DEBUG__tmg_reg_UPDATE_ACK__SHIFT 0x7
+#define PPLL_STATUS_DEBUG0__obsout_MASK 0xffffffff
+#define PPLL_STATUS_DEBUG0__obsout__SHIFT 0x0
+#define COMP_EN_CTL__comp_en_MASK 0x1
+#define COMP_EN_CTL__comp_en__SHIFT 0x0
+#define COMP_EN_CTL__comp_en_override_MASK 0x4
+#define COMP_EN_CTL__comp_en_override__SHIFT 0x2
+#define COMP_EN_CTL__comp_done_MASK 0x10
+#define COMP_EN_CTL__comp_done__SHIFT 0x4
+#define COMP_EN_CTL__zcal_code_override_MASK 0x40
+#define COMP_EN_CTL__zcal_code_override__SHIFT 0x6
+#define COMP_EN_CTL__zcal_cal_rtt_MASK 0x80
+#define COMP_EN_CTL__zcal_cal_rtt__SHIFT 0x7
+#define COMP_EN_CTL__zcal_base_en_MASK 0x100
+#define COMP_EN_CTL__zcal_base_en__SHIFT 0x8
+#define COMP_EN_CTL__zcal_ht_rtt_sel_MASK 0x200
+#define COMP_EN_CTL__zcal_ht_rtt_sel__SHIFT 0x9
+#define COMP_EN_CTL__zcal_code_MASK 0x7c00
+#define COMP_EN_CTL__zcal_code__SHIFT 0xa
+#define COMP_EN_CTL__zcal_ron_cal_mode_MASK 0x10000
+#define COMP_EN_CTL__zcal_ron_cal_mode__SHIFT 0x10
+#define COMP_EN_CTL__zcal_ana_dbg_sel_MASK 0x60000
+#define COMP_EN_CTL__zcal_ana_dbg_sel__SHIFT 0x11
+#define COMP_EN_CTL__cfg_cml_cmos_sel_MASK 0x80000
+#define COMP_EN_CTL__cfg_cml_cmos_sel__SHIFT 0x13
+#define COMP_EN_CTL__dsm_sel_MASK 0xf00000
+#define COMP_EN_CTL__dsm_sel__SHIFT 0x14
+#define DPCSTX_PHY_CNTL__DPCS_PHY_RESET_MASK 0x1
+#define DPCSTX_PHY_CNTL__DPCS_PHY_RESET__SHIFT 0x0
+#define DPCSTX_TX_CLOCK_CNTL__DPCS_SYMCLK_GATE_DIS_MASK 0x1
+#define DPCSTX_TX_CLOCK_CNTL__DPCS_SYMCLK_GATE_DIS__SHIFT 0x0
+#define DPCSTX_TX_CLOCK_CNTL__DPCS_SYMCLK_EN_MASK 0x2
+#define DPCSTX_TX_CLOCK_CNTL__DPCS_SYMCLK_EN__SHIFT 0x1
+#define DPCSTX_TX_CLOCK_CNTL__DPCS_SYMCLK_CLOCK_ON_MASK 0x4
+#define DPCSTX_TX_CLOCK_CNTL__DPCS_SYMCLK_CLOCK_ON__SHIFT 0x2
+#define DPCSTX_TX_CLOCK_CNTL__DPCS_SYMCLK_DIV2_CLOCK_ON_MASK 0x8
+#define DPCSTX_TX_CLOCK_CNTL__DPCS_SYMCLK_DIV2_CLOCK_ON__SHIFT 0x3
+#define DPCSTX_TX_CLOCK_CNTL__DPCS_SYMCLK_DIV2_TX0_EN_MASK 0x10
+#define DPCSTX_TX_CLOCK_CNTL__DPCS_SYMCLK_DIV2_TX0_EN__SHIFT 0x4
+#define DPCSTX_TX_CLOCK_CNTL__DPCS_SYMCLK_DIV2_TX1_EN_MASK 0x20
+#define DPCSTX_TX_CLOCK_CNTL__DPCS_SYMCLK_DIV2_TX1_EN__SHIFT 0x5
+#define DPCSTX_TX_CLOCK_CNTL__DPCS_SYMCLK_DIV2_TX2_EN_MASK 0x40
+#define DPCSTX_TX_CLOCK_CNTL__DPCS_SYMCLK_DIV2_TX2_EN__SHIFT 0x6
+#define DPCSTX_TX_CLOCK_CNTL__DPCS_SYMCLK_DIV2_TX3_EN_MASK 0x80
+#define DPCSTX_TX_CLOCK_CNTL__DPCS_SYMCLK_DIV2_TX3_EN__SHIFT 0x7
+#define DPCSTX_TX_CNTL__DPCS_TX_RESYNC_MASK 0x1
+#define DPCSTX_TX_CNTL__DPCS_TX_RESYNC__SHIFT 0x0
+#define DPCSTX_TX_CNTL__DPCS_TX_STAGGERING_EN_MASK 0x2
+#define DPCSTX_TX_CNTL__DPCS_TX_STAGGERING_EN__SHIFT 0x1
+#define DPCSTX_TX_CNTL__DPCS_TX_HIGH_IMP_IDLE_OVERRIDE_EN_MASK 0x4
+#define DPCSTX_TX_CNTL__DPCS_TX_HIGH_IMP_IDLE_OVERRIDE_EN__SHIFT 0x2
+#define DPCSTX_TX_CNTL__DPCS_TX_HIGH_IMP_IDLE_MASK 0xf0
+#define DPCSTX_TX_CNTL__DPCS_TX_HIGH_IMP_IDLE__SHIFT 0x4
+#define DPCSTX_TX_CNTL__DPCS_TX_STAGGERING_DELAY_MASK 0x700
+#define DPCSTX_TX_CNTL__DPCS_TX_STAGGERING_DELAY__SHIFT 0x8
+#define DPCSTX_TX_CNTL__DPCS_TX_PLL_UPDATE_REQ_MASK 0x1000
+#define DPCSTX_TX_CNTL__DPCS_TX_PLL_UPDATE_REQ__SHIFT 0xc
+#define DPCSTX_TX_CNTL__DPCS_TX_PLL_UPDATE_PENDING_MASK 0x2000
+#define DPCSTX_TX_CNTL__DPCS_TX_PLL_UPDATE_PENDING__SHIFT 0xd
+#define DPCSTX_TX_CNTL__DPCS_TX_DATA_SWAP_MASK 0x4000
+#define DPCSTX_TX_CNTL__DPCS_TX_DATA_SWAP__SHIFT 0xe
+#define DPCSTX_TX_CNTL__DPCS_TX_FIFO_EN_MASK 0x10000
+#define DPCSTX_TX_CNTL__DPCS_TX_FIFO_EN__SHIFT 0x10
+#define DPCSTX_TX_CNTL__DPCS_TX_FIFO_START_MASK 0x20000
+#define DPCSTX_TX_CNTL__DPCS_TX_FIFO_START__SHIFT 0x11
+#define DPCSTX_TX_CNTL__DPCS_TX_FIFO_WR_START_DELAY_MASK 0xf00000
+#define DPCSTX_TX_CNTL__DPCS_TX_FIFO_WR_START_DELAY__SHIFT 0x14
+#define DPCSTX_TX_CNTL__DPCS_TX_DVI_LINK_MODE_MASK 0x3000000
+#define DPCSTX_TX_CNTL__DPCS_TX_DVI_LINK_MODE__SHIFT 0x18
+#define DPCSTX_TX_CNTL__DPCS_TX_SOFT_RESET_MASK 0x80000000
+#define DPCSTX_TX_CNTL__DPCS_TX_SOFT_RESET__SHIFT 0x1f
+#define DPCSTX_CBUS_CNTL__DPCS_CBUS_WR_CMD_DELAY_MASK 0xf
+#define DPCSTX_CBUS_CNTL__DPCS_CBUS_WR_CMD_DELAY__SHIFT 0x0
+#define DPCSTX_CBUS_CNTL__DPCS_PHY_MASTER_REQ_DELAY_MASK 0xff00
+#define DPCSTX_CBUS_CNTL__DPCS_PHY_MASTER_REQ_DELAY__SHIFT 0x8
+#define DPCSTX_CBUS_CNTL__DPCS_CBUS_SOFT_RESET_MASK 0x80000000
+#define DPCSTX_CBUS_CNTL__DPCS_CBUS_SOFT_RESET__SHIFT 0x1f
+#define DPCSTX_REG_ERROR_STATUS__DPCS_REG_FIFO_OVERFLOW_MASK 0x1
+#define DPCSTX_REG_ERROR_STATUS__DPCS_REG_FIFO_OVERFLOW__SHIFT 0x0
+#define DPCSTX_REG_ERROR_STATUS__DPCS_REG_ERROR_CLR_MASK 0x2
+#define DPCSTX_REG_ERROR_STATUS__DPCS_REG_ERROR_CLR__SHIFT 0x1
+#define DPCSTX_REG_ERROR_STATUS__DPCS_REG_FIFO_ERROR_MASK_MASK 0x10
+#define DPCSTX_REG_ERROR_STATUS__DPCS_REG_FIFO_ERROR_MASK__SHIFT 0x4
+#define DPCSTX_TX_ERROR_STATUS__DPCS_TX0_FIFO_ERROR_MASK 0x1
+#define DPCSTX_TX_ERROR_STATUS__DPCS_TX0_FIFO_ERROR__SHIFT 0x0
+#define DPCSTX_TX_ERROR_STATUS__DPCS_TX1_FIFO_ERROR_MASK 0x2
+#define DPCSTX_TX_ERROR_STATUS__DPCS_TX1_FIFO_ERROR__SHIFT 0x1
+#define DPCSTX_TX_ERROR_STATUS__DPCS_TX2_FIFO_ERROR_MASK 0x4
+#define DPCSTX_TX_ERROR_STATUS__DPCS_TX2_FIFO_ERROR__SHIFT 0x2
+#define DPCSTX_TX_ERROR_STATUS__DPCS_TX3_FIFO_ERROR_MASK 0x8
+#define DPCSTX_TX_ERROR_STATUS__DPCS_TX3_FIFO_ERROR__SHIFT 0x3
+#define DPCSTX_TX_ERROR_STATUS__DPCS_TX_ERROR_CLR_MASK 0x100
+#define DPCSTX_TX_ERROR_STATUS__DPCS_TX_ERROR_CLR__SHIFT 0x8
+#define DPCSTX_TX_ERROR_STATUS__DPCS_TX_FIFO_ERROR_MASK_MASK 0x1000
+#define DPCSTX_TX_ERROR_STATUS__DPCS_TX_FIFO_ERROR_MASK__SHIFT 0xc
+#define DPCSTX_PLL_UPDATE_ADDR__DPCS_PLL_UPDATE_ADDR_MASK 0x3ffff
+#define DPCSTX_PLL_UPDATE_ADDR__DPCS_PLL_UPDATE_ADDR__SHIFT 0x0
+#define DPCSTX_PLL_UPDATE_DATA__DPCS_PLL_UPDATE_DATA_MASK 0xffffffff
+#define DPCSTX_PLL_UPDATE_DATA__DPCS_PLL_UPDATE_DATA__SHIFT 0x0
+#define DPCSTX_INDEX_MODE_ADDR__DPCS_INDEX_MODE_ADDR_MASK 0x3ffff
+#define DPCSTX_INDEX_MODE_ADDR__DPCS_INDEX_MODE_ADDR__SHIFT 0x0
+#define DPCSTX_INDEX_MODE_DATA__DPCS_INDEX_MODE_DATA_MASK 0xffffffff
+#define DPCSTX_INDEX_MODE_DATA__DPCS_INDEX_MODE_DATA__SHIFT 0x0
+#define DPCSTX_DEBUG_CONFIG__DPCS_DBG_EN_MASK 0x1
+#define DPCSTX_DEBUG_CONFIG__DPCS_DBG_EN__SHIFT 0x0
+#define DPCSTX_DEBUG_CONFIG__DPCS_DBG_CFGCLK_SEL_MASK 0x6
+#define DPCSTX_DEBUG_CONFIG__DPCS_DBG_CFGCLK_SEL__SHIFT 0x1
+#define DPCSTX_DEBUG_CONFIG__DPCS_DBG_TX_SYMCLK_SEL_MASK 0x38
+#define DPCSTX_DEBUG_CONFIG__DPCS_DBG_TX_SYMCLK_SEL__SHIFT 0x3
+#define DPCSTX_DEBUG_CONFIG__DPCS_DBG_CLOCK_SEL_MASK 0x700
+#define DPCSTX_DEBUG_CONFIG__DPCS_DBG_CLOCK_SEL__SHIFT 0x8
+#define DPCSTX_DEBUG_CONFIG__DPCS_DBG_BLOCK_SEL_MASK 0x3800
+#define DPCSTX_DEBUG_CONFIG__DPCS_DBG_BLOCK_SEL__SHIFT 0xb
+#define DPCSTX_DEBUG_CONFIG__DPCS_DBG_CBUS_DIS_MASK 0x4000
+#define DPCSTX_DEBUG_CONFIG__DPCS_DBG_CBUS_DIS__SHIFT 0xe
+#define DPCSTX_DEBUG_CONFIG__DPCS_TEST_DEBUG_WRITE_EN_MASK 0x10000
+#define DPCSTX_DEBUG_CONFIG__DPCS_TEST_DEBUG_WRITE_EN__SHIFT 0x10
+#define DPCSTX_DEBUG_CONFIG__DPCS_DBG_TX_SYMCLK_DIV2_SEL_MASK 0xe0000
+#define DPCSTX_DEBUG_CONFIG__DPCS_DBG_TX_SYMCLK_DIV2_SEL__SHIFT 0x11
+#define DPCSTX_DEBUG_CONFIG__DPCS_TEST_DEBUG_INDEX_MASK 0xff000000
+#define DPCSTX_DEBUG_CONFIG__DPCS_TEST_DEBUG_INDEX__SHIFT 0x18
+#define DPCSTX_TEST_DEBUG_DATA__DPCS_TEST_DEBUG_DATA_MASK 0xffffffff
+#define DPCSTX_TEST_DEBUG_DATA__DPCS_TEST_DEBUG_DATA__SHIFT 0x0
+
+#endif /* DCE_11_2_SH_MASK_H */
diff --git a/drivers/gpu/drm/amd/include/asic_reg/gca/gfx_8_0_d.h b/drivers/gpu/drm/amd/include/asic_reg/gca/gfx_8_0_d.h
index a9b6923..ebaf67b 100644
--- a/drivers/gpu/drm/amd/include/asic_reg/gca/gfx_8_0_d.h
+++ b/drivers/gpu/drm/amd/include/asic_reg/gca/gfx_8_0_d.h
@@ -1391,6 +1391,8 @@
#define mmRLC_CGTT_MGCG_OVERRIDE 0xec48
#define mmRLC_CGCG_CGLS_CTRL 0xec49
#define mmRLC_CGCG_RAMP_CTRL 0xec4a
+#define mmRLC_CGCG_CGLS_CTRL_3D 0xec9d
+#define mmRLC_CGCG_RAMP_CTRL_3D 0xec9e
#define mmRLC_DYN_PG_STATUS 0xec4b
#define mmRLC_DYN_PG_REQUEST 0xec4c
#define mmRLC_PG_DELAY 0xec4d
diff --git a/drivers/gpu/drm/amd/include/asic_reg/uvd/uvd_6_0_d.h b/drivers/gpu/drm/amd/include/asic_reg/uvd/uvd_6_0_d.h
index b2d4aaf..6f6fb34 100644
--- a/drivers/gpu/drm/amd/include/asic_reg/uvd/uvd_6_0_d.h
+++ b/drivers/gpu/drm/amd/include/asic_reg/uvd/uvd_6_0_d.h
@@ -111,5 +111,6 @@
#define mmUVD_MIF_RECON1_ADDR_CONFIG 0x39c5
#define ixUVD_MIF_SCLR_ADDR_CONFIG 0x4
#define mmUVD_JPEG_ADDR_CONFIG 0x3a1f
+#define mmUVD_GP_SCRATCH4 0x3d38
#endif /* UVD_6_0_D_H */
diff --git a/drivers/gpu/drm/amd/include/atombios.h b/drivers/gpu/drm/amd/include/atombios.h
index eaf451e..32f3e34 100644
--- a/drivers/gpu/drm/amd/include/atombios.h
+++ b/drivers/gpu/drm/amd/include/atombios.h
@@ -79,9 +79,23 @@
#define ATOM_PPLL0 2
#define ATOM_PPLL3 3
+#define ATOM_PHY_PLL0 4
+#define ATOM_PHY_PLL1 5
+
#define ATOM_EXT_PLL1 8
+#define ATOM_GCK_DFS 8
#define ATOM_EXT_PLL2 9
+#define ATOM_FCH_CLK 9
#define ATOM_EXT_CLOCK 10
+#define ATOM_DP_DTO 11
+
+#define ATOM_COMBOPHY_PLL0 20
+#define ATOM_COMBOPHY_PLL1 21
+#define ATOM_COMBOPHY_PLL2 22
+#define ATOM_COMBOPHY_PLL3 23
+#define ATOM_COMBOPHY_PLL4 24
+#define ATOM_COMBOPHY_PLL5 25
+
#define ATOM_PPLL_INVALID 0xFF
#define ENCODER_REFCLK_SRC_P1PLL 0
@@ -224,6 +238,31 @@
UCHAR ucReserved;
}ATOM_ROM_HEADER;
+
+typedef struct _ATOM_ROM_HEADER_V2_1
+{
+ ATOM_COMMON_TABLE_HEADER sHeader;
+ UCHAR uaFirmWareSignature[4]; //Signature to distinguish between Atombios and non-atombios,
+ //atombios should init it as "ATOM", don't change the position
+ USHORT usBiosRuntimeSegmentAddress;
+ USHORT usProtectedModeInfoOffset;
+ USHORT usConfigFilenameOffset;
+ USHORT usCRC_BlockOffset;
+ USHORT usBIOS_BootupMessageOffset;
+ USHORT usInt10Offset;
+ USHORT usPciBusDevInitCode;
+ USHORT usIoBaseAddress;
+ USHORT usSubsystemVendorID;
+ USHORT usSubsystemID;
+ USHORT usPCI_InfoOffset;
+ USHORT usMasterCommandTableOffset;//Offest for SW to get all command table offsets, Don't change the position
+ USHORT usMasterDataTableOffset; //Offest for SW to get all data table offsets, Don't change the position
+ UCHAR ucExtendedFunctionCode;
+ UCHAR ucReserved;
+ ULONG ulPSPDirTableOffset;
+}ATOM_ROM_HEADER_V2_1;
+
+
//==============================Command Table Portion====================================
@@ -272,12 +311,12 @@
USHORT GetSCLKOverMCLKRatio; //Atomic Table, only used by Bios
USHORT SetCRTC_Timing; //Atomic Table, directly used by various SW components,latest version 1.1
USHORT SetCRTC_OverScan; //Atomic Table, used by various SW components,latest version 1.1
- USHORT SetCRTC_Replication; //Atomic Table, used only by Bios
+ USHORT GetSMUClockInfo; //Atomic Table, used only by Bios
USHORT SelectCRTC_Source; //Atomic Table, directly used by various SW components,latest version 1.1
USHORT EnableGraphSurfaces; //Atomic Table, used only by Bios
USHORT UpdateCRTC_DoubleBufferRegisters; //Atomic Table, used only by Bios
USHORT LUT_AutoFill; //Atomic Table, only used by Bios
- USHORT EnableHW_IconCursor; //Atomic Table, only used by Bios
+ USHORT SetDCEClock; //Atomic Table, start from DCE11.1, shared by driver and VBIOS, change DISPCLK and DPREFCLK
USHORT GetMemoryClock; //Atomic Table, directly used by various SW components,latest version 1.1
USHORT GetEngineClock; //Atomic Table, directly used by various SW components,latest version 1.1
USHORT SetCRTC_UsingDTDTiming; //Atomic Table, directly used by various SW components,latest version 1.1
@@ -292,7 +331,7 @@
USHORT PowerConnectorDetection; //Atomic Table, directly used by various SW components,latest version 1.1
USHORT MC_Synchronization; //Atomic Table, indirectly used by various SW components,called from SetMemoryClock
USHORT ComputeMemoryEnginePLL; //Atomic Table, indirectly used by various SW components,called from SetMemory/EngineClock
- USHORT MemoryRefreshConversion; //Atomic Table, indirectly used by various SW components,called from SetMemory or SetEngineClock
+ USHORT Gfx_Init; //Atomic Table, indirectly used by various SW components,called from SetMemory or SetEngineClock
USHORT VRAM_GetCurrentInfoBlock; //Atomic Table, used only by Bios
USHORT DynamicMemorySettings; //Atomic Table, indirectly used by various SW components,called from SetMemoryClock
USHORT MemoryTraining; //Atomic Table, used only by Bios
@@ -333,6 +372,10 @@
#define LCD1OutputControl HW_Misc_Operation
#define TV1OutputControl Gfx_Harvesting
#define TVEncoderControl SMC_Init
+#define EnableHW_IconCursor SetDCEClock
+#define SetCRTC_Replication GetSMUClockInfo
+
+#define MemoryRefreshConversion Gfx_Init
typedef struct _ATOM_MASTER_COMMAND_TABLE
{
@@ -425,6 +468,9 @@
#define b3FIRST_TIME_CHANGE_CLOCK 0x08 //Applicable to both memory and engine clock change,when set, it means this is 1st time to change clock after ASIC bootup
#define b3SKIP_SW_PROGRAM_PLL 0x10 //Applicable to both memory and engine clock change, when set, it means the table will not program SPLL/MPLL
#define b3DRAM_SELF_REFRESH_EXIT 0x20 //Applicable to DRAM self refresh exit only. when set, it means it will go to program DRAM self refresh exit path
+#define b3SRIOV_INIT_BOOT 0x40 //Use by HV GPU driver only, to load uCode. for ASIC_InitTable SCLK parameter only
+#define b3SRIOV_LOAD_UCODE 0x40 //Use by HV GPU driver only, to load uCode. for ASIC_InitTable SCLK parameter only
+#define b3SRIOV_SKIP_ASIC_INIT 0x02 //Use by HV GPU driver only, skip ASIC_Init for primary adapter boot. for ASIC_InitTable SCLK parameter only
typedef struct _ATOM_COMPUTE_CLOCK_FREQ
{
@@ -518,6 +564,33 @@
//ucPllCntlFlag
#define SPLL_CNTL_FLAG_VCO_MODE_MASK 0x03
+typedef struct _COMPUTE_GPU_CLOCK_INPUT_PARAMETERS_V1_7
+{
+ ATOM_COMPUTE_CLOCK_FREQ ulClock; //Input Parameter
+ ULONG ulReserved[5];
+}COMPUTE_GPU_CLOCK_INPUT_PARAMETERS_V1_7;
+
+//ATOM_COMPUTE_CLOCK_FREQ.ulComputeClockFlag
+#define COMPUTE_GPUCLK_INPUT_FLAG_CLK_TYPE_MASK 0x0f
+#define COMPUTE_GPUCLK_INPUT_FLAG_DEFAULT_GPUCLK 0x00
+#define COMPUTE_GPUCLK_INPUT_FLAG_SCLK 0x01
+
+typedef struct _COMPUTE_GPU_CLOCK_OUTPUT_PARAMETERS_V1_7
+{
+ COMPUTE_MEMORY_ENGINE_PLL_PARAMETERS_V4 ulClock; //Output Parameter: ucPostDiv=DFS divider
+ USHORT usSclk_fcw_frac; //fractional divider of fcw = usSclk_fcw_frac/65536
+ USHORT usSclk_fcw_int; //integer divider of fcwc
+ UCHAR ucSclkPostDiv; //PLL post divider = 2^ucSclkPostDiv
+ UCHAR ucSclkVcoMode; //0: 4G~8Ghz, 1:3G~6Ghz,3: 2G~4Ghz, 2:Reserved
+ UCHAR ucSclkPllRange; //GreenTable SCLK PLL range entry index ( 0~7 )
+ UCHAR ucSscEnable;
+ USHORT usSsc_fcw1_frac; //fcw1_frac when SSC enable
+ USHORT usSsc_fcw1_int; //fcw1_int when SSC enable
+ USHORT usReserved;
+ USHORT usPcc_fcw_int;
+ USHORT usSsc_fcw_slew_frac; //fcw_slew_frac when SSC enable
+ USHORT usPcc_fcw_slew_frac;
+}COMPUTE_GPU_CLOCK_OUTPUT_PARAMETERS_V1_7;
// ucInputFlag
#define ATOM_PLL_INPUT_FLAG_PLL_STROBE_MODE_EN 1 // 1-StrobeMode, 0-PerformanceMode
@@ -557,12 +630,16 @@
ULONG ulReserved;
}COMPUTE_MEMORY_CLOCK_PARAM_PARAMETERS_V2_2;
+//Input parameter of DynamicMemorySettingsTable
+//when ATOM_COMPUTE_CLOCK_FREQ.ulComputeClockFlag = COMPUTE_MEMORY_PLL_PARAM
typedef struct _DYNAMICE_MEMORY_SETTINGS_PARAMETER
{
ATOM_COMPUTE_CLOCK_FREQ ulClock;
ULONG ulReserved[2];
}DYNAMICE_MEMORY_SETTINGS_PARAMETER;
+//Input parameter of DynamicMemorySettingsTable
+//when ATOM_COMPUTE_CLOCK_FREQ.ulComputeClockFlag == COMPUTE_ENGINE_PLL_PARAM
typedef struct _DYNAMICE_ENGINE_SETTINGS_PARAMETER
{
ATOM_COMPUTE_CLOCK_FREQ ulClock;
@@ -570,6 +647,29 @@
ULONG ulReserved;
}DYNAMICE_ENGINE_SETTINGS_PARAMETER;
+//Input parameter of DynamicMemorySettingsTable ver2.1 and above
+//when ATOM_COMPUTE_CLOCK_FREQ.ulComputeClockFlag == ADJUST_MC_SETTING_PARAM
+typedef struct _DYNAMICE_MC_DPM_SETTINGS_PARAMETER
+{
+ ATOM_COMPUTE_CLOCK_FREQ ulClock;
+ UCHAR ucMclkDPMState;
+ UCHAR ucReserved[3];
+ ULONG ulReserved;
+}DYNAMICE_MC_DPM_SETTINGS_PARAMETER;
+
+//ucMclkDPMState
+#define DYNAMIC_MC_DPM_SETTING_LOW_DPM_STATE 0
+#define DYNAMIC_MC_DPM_SETTING_MEDIUM_DPM_STATE 1
+#define DYNAMIC_MC_DPM_SETTING_HIGH_DPM_STATE 2
+
+typedef union _DYNAMICE_MEMORY_SETTINGS_PARAMETER_V2_1
+{
+ DYNAMICE_MEMORY_SETTINGS_PARAMETER asMCReg;
+ DYNAMICE_ENGINE_SETTINGS_PARAMETER asMCArbReg;
+ DYNAMICE_MC_DPM_SETTINGS_PARAMETER asDPMMCReg;
+}DYNAMICE_MEMORY_SETTINGS_PARAMETER_V2_1;
+
+
/****************************************************************************/
// Structures used by SetEngineClockTable
/****************************************************************************/
@@ -584,6 +684,13 @@
COMPUTE_MEMORY_ENGINE_PLL_PARAMETERS_PS_ALLOCATION sReserved;
}SET_ENGINE_CLOCK_PS_ALLOCATION;
+typedef struct _SET_ENGINE_CLOCK_PS_ALLOCATION_V1_2
+{
+ ULONG ulTargetEngineClock; //In 10Khz unit
+ COMPUTE_GPU_CLOCK_INPUT_PARAMETERS_V1_7 sReserved;
+}SET_ENGINE_CLOCK_PS_ALLOCATION_V1_2;
+
+
/****************************************************************************/
// Structures used by SetMemoryClockTable
/****************************************************************************/
@@ -827,6 +934,12 @@
#define ATOM_ENCODER_CMD_SETUP 0x0f
#define ATOM_ENCODER_CMD_SETUP_PANEL_MODE 0x10
+// New Command for DIGxEncoderControlTable v1.5
+#define ATOM_ENCODER_CMD_DP_LINK_TRAINING_PATTERN4 0x14
+#define ATOM_ENCODER_CMD_STREAM_SETUP 0x0F //change name ATOM_ENCODER_CMD_SETUP
+#define ATOM_ENCODER_CMD_LINK_SETUP 0x11 //internal use, called by other Command Table
+#define ATOM_ENCODER_CMD_ENCODER_BLANK 0x12 //internal use, called by other Command Table
+
// ucStatus
#define ATOM_ENCODER_STATUS_LINK_TRAINING_COMPLETE 0x10
#define ATOM_ENCODER_STATUS_LINK_TRAINING_INCOMPLETE 0x00
@@ -955,6 +1068,69 @@
#define DP_PANEL_MODE_INTERNAL_DP2_MODE 0x01
#define DP_PANEL_MODE_INTERNAL_DP1_MODE 0x11
+
+typedef struct _ENCODER_STREAM_SETUP_PARAMETERS_V5
+{
+ UCHAR ucDigId; // 0~6 map to DIG0~DIG6
+ UCHAR ucAction; // = ATOM_ENOCODER_CMD_STREAM_SETUP
+ UCHAR ucDigMode; // ATOM_ENCODER_MODE_DP/ATOM_ENCODER_MODE_DVI/ATOM_ENCODER_MODE_HDMI
+ UCHAR ucLaneNum; // Lane number
+ ULONG ulPixelClock; // Pixel Clock in 10Khz
+ UCHAR ucBitPerColor;
+ UCHAR ucLinkRateIn270Mhz;//= DP link rate/270Mhz, =6: 1.62G = 10: 2.7G, =20: 5.4Ghz, =30: 8.1Ghz etc
+ UCHAR ucReserved[2];
+}ENCODER_STREAM_SETUP_PARAMETERS_V5;
+
+typedef struct _ENCODER_LINK_SETUP_PARAMETERS_V5
+{
+ UCHAR ucDigId; // 0~6 map to DIG0~DIG6
+ UCHAR ucAction; // = ATOM_ENOCODER_CMD_LINK_SETUP
+ UCHAR ucDigMode; // ATOM_ENCODER_MODE_DP/ATOM_ENCODER_MODE_DVI/ATOM_ENCODER_MODE_HDMI
+ UCHAR ucLaneNum; // Lane number
+ ULONG ulSymClock; // Symbol Clock in 10Khz
+ UCHAR ucHPDSel;
+ UCHAR ucDigEncoderSel; // DIG stream( front-end ) selection, bit0 means DIG0 FE is enable,
+ UCHAR ucReserved[2];
+}ENCODER_LINK_SETUP_PARAMETERS_V5;
+
+typedef struct _DP_PANEL_MODE_SETUP_PARAMETERS_V5
+{
+ UCHAR ucDigId; // 0~6 map to DIG0~DIG6
+ UCHAR ucAction; // = ATOM_ENCODER_CMD_DPLINK_SETUP
+ UCHAR ucPanelMode; // =0: external DP
+ // =0x1: internal DP2
+ // =0x11: internal DP1 NutMeg/Travis DP Translator
+ UCHAR ucReserved;
+ ULONG ulReserved[2];
+}DP_PANEL_MODE_SETUP_PARAMETERS_V5;
+
+typedef struct _ENCODER_GENERIC_CMD_PARAMETERS_V5
+{
+ UCHAR ucDigId; // 0~6 map to DIG0~DIG6
+ UCHAR ucAction; // = rest of generic encoder command which does not carry any parameters
+ UCHAR ucReserved[2];
+ ULONG ulReserved[2];
+}ENCODER_GENERIC_CMD_PARAMETERS_V5;
+
+//ucDigId
+#define ATOM_ENCODER_CONFIG_V5_DIG0_ENCODER 0x00
+#define ATOM_ENCODER_CONFIG_V5_DIG1_ENCODER 0x01
+#define ATOM_ENCODER_CONFIG_V5_DIG2_ENCODER 0x02
+#define ATOM_ENCODER_CONFIG_V5_DIG3_ENCODER 0x03
+#define ATOM_ENCODER_CONFIG_V5_DIG4_ENCODER 0x04
+#define ATOM_ENCODER_CONFIG_V5_DIG5_ENCODER 0x05
+#define ATOM_ENCODER_CONFIG_V5_DIG6_ENCODER 0x06
+
+
+typedef union _DIG_ENCODER_CONTROL_PARAMETERS_V5
+{
+ ENCODER_GENERIC_CMD_PARAMETERS_V5 asCmdParam;
+ ENCODER_STREAM_SETUP_PARAMETERS_V5 asStreamParam;
+ ENCODER_LINK_SETUP_PARAMETERS_V5 asLinkParam;
+ DP_PANEL_MODE_SETUP_PARAMETERS_V5 asDPPanelModeParam;
+}DIG_ENCODER_CONTROL_PARAMETERS_V5;
+
+
/****************************************************************************/
// Structures used by UNIPHYTransmitterControlTable
// LVTMATransmitterControlTable
@@ -1371,6 +1547,49 @@
#define DIG_TRANSMITTER_CONTROL_PS_ALLOCATION_V1_5 DIG_TRANSMITTER_CONTROL_PARAMETERS_V1_5
+typedef struct _DIG_TRANSMITTER_CONTROL_PARAMETERS_V1_6
+{
+ UCHAR ucPhyId; // 0=UNIPHYA, 1=UNIPHYB, 2=UNIPHYC, 3=UNIPHYD, 4= UNIPHYE 5=UNIPHYF
+ UCHAR ucAction; // define as ATOM_TRANSMITER_ACTION_xxx
+ union
+ {
+ UCHAR ucDigMode; // ATOM_ENCODER_MODE_DP/ATOM_ENCODER_MODE_DVI/ATOM_ENCODER_MODE_HDMI
+ UCHAR ucDPLaneSet; // DP voltage swing and pre-emphasis value defined in DPCD DP_LANE_SET, "DP_LANE_SET__xDB_y_zV"
+ };
+ UCHAR ucLaneNum; // Lane number
+ ULONG ulSymClock; // Symbol Clock in 10Khz
+ UCHAR ucHPDSel; // =1: HPD1, =2: HPD2, .... =6: HPD6, =0: HPD is not assigned
+ UCHAR ucDigEncoderSel; // DIG stream( front-end ) selection, bit0 means DIG0 FE is enable,
+ UCHAR ucConnObjId; // Connector Object Id defined in ObjectId.h
+ UCHAR ucReserved;
+ ULONG ulReserved;
+}DIG_TRANSMITTER_CONTROL_PARAMETERS_V1_6;
+
+
+// ucDigEncoderSel
+#define ATOM_TRANMSITTER_V6__DIGA_SEL 0x01
+#define ATOM_TRANMSITTER_V6__DIGB_SEL 0x02
+#define ATOM_TRANMSITTER_V6__DIGC_SEL 0x04
+#define ATOM_TRANMSITTER_V6__DIGD_SEL 0x08
+#define ATOM_TRANMSITTER_V6__DIGE_SEL 0x10
+#define ATOM_TRANMSITTER_V6__DIGF_SEL 0x20
+#define ATOM_TRANMSITTER_V6__DIGG_SEL 0x40
+
+// ucDigMode
+#define ATOM_TRANSMITTER_DIGMODE_V6_DP 0
+#define ATOM_TRANSMITTER_DIGMODE_V6_DVI 2
+#define ATOM_TRANSMITTER_DIGMODE_V6_HDMI 3
+#define ATOM_TRANSMITTER_DIGMODE_V6_DP_MST 5
+
+//ucHPDSel
+#define ATOM_TRANSMITTER_V6_NO_HPD_SEL 0x00
+#define ATOM_TRANSMITTER_V6_HPD1_SEL 0x01
+#define ATOM_TRANSMITTER_V6_HPD2_SEL 0x02
+#define ATOM_TRANSMITTER_V6_HPD3_SEL 0x03
+#define ATOM_TRANSMITTER_V6_HPD4_SEL 0x04
+#define ATOM_TRANSMITTER_V6_HPD5_SEL 0x05
+#define ATOM_TRANSMITTER_V6_HPD6_SEL 0x06
+
/****************************************************************************/
// Structures used by ExternalEncoderControlTable V1.3
@@ -1784,6 +2003,101 @@
PIXEL_CLOCK_PARAMETERS_V5 sDispClkInput;
}GET_DISP_PLL_STATUS_INPUT_PARAMETERS_V3;
+typedef struct _PIXEL_CLOCK_PARAMETERS_V7
+{
+ ULONG ulPixelClock; // target the pixel clock to drive the CRTC timing in unit of 100Hz.
+
+ UCHAR ucPpll; // ATOM_PHY_PLL0/ATOM_PHY_PLL1/ATOM_PPLL0
+ UCHAR ucTransmitterID; // ASIC encoder id defined in objectId.h,
+ // indicate which graphic encoder will be used.
+ UCHAR ucEncoderMode; // Encoder mode:
+ UCHAR ucMiscInfo; // bit[0]= Force program PLL for pixclk
+ // bit[1]= Force program PHY PLL only ( internally used by VBIOS only in DP case which PHYPLL is programmed for SYMCLK, not Pixclk )
+ // bit[5:4]= RefClock source for PPLL.
+ // =0: XTLAIN( default mode )
+ // =1: pcie
+ // =2: GENLK
+ UCHAR ucCRTC; // ATOM_CRTC1~6, indicate the CRTC controller to
+ UCHAR ucDeepColorRatio; // HDMI panel bit depth: =0: 24bpp =1:30bpp, =2:36bpp
+ UCHAR ucReserved[2];
+ ULONG ulReserved;
+}PIXEL_CLOCK_PARAMETERS_V7;
+
+//ucMiscInfo
+#define PIXEL_CLOCK_V7_MISC_FORCE_PROG_PPLL 0x01
+#define PIXEL_CLOCK_V7_MISC_PROG_PHYPLL 0x02
+#define PIXEL_CLOCK_V7_MISC_YUV420_MODE 0x04
+#define PIXEL_CLOCK_V7_MISC_DVI_DUALLINK_EN 0x08
+#define PIXEL_CLOCK_V7_MISC_REF_DIV_SRC 0x30
+#define PIXEL_CLOCK_V7_MISC_REF_DIV_SRC_XTALIN 0x00
+#define PIXEL_CLOCK_V7_MISC_REF_DIV_SRC_PCIE 0x10
+#define PIXEL_CLOCK_V7_MISC_REF_DIV_SRC_GENLK 0x20
+
+//ucDeepColorRatio
+#define PIXEL_CLOCK_V7_DEEPCOLOR_RATIO_DIS 0x00 //00 - DCCG_DEEP_COLOR_DTO_DISABLE: Disable Deep Color DTO
+#define PIXEL_CLOCK_V7_DEEPCOLOR_RATIO_5_4 0x01 //01 - DCCG_DEEP_COLOR_DTO_5_4_RATIO: Set Deep Color DTO to 5:4
+#define PIXEL_CLOCK_V7_DEEPCOLOR_RATIO_3_2 0x02 //02 - DCCG_DEEP_COLOR_DTO_3_2_RATIO: Set Deep Color DTO to 3:2
+#define PIXEL_CLOCK_V7_DEEPCOLOR_RATIO_2_1 0x03 //03 - DCCG_DEEP_COLOR_DTO_2_1_RATIO: Set Deep Color DTO to 2:1
+
+// SetDCEClockTable input parameter for DCE11.1
+typedef struct _SET_DCE_CLOCK_PARAMETERS_V1_1
+{
+ ULONG ulDISPClkFreq; // target DISPCLK frquency in unit of 10kHz, return real DISPCLK frequency. when ucFlag[1]=1, in unit of 100Hz.
+ UCHAR ucFlag; // bit0=1: DPREFCLK bypass DFS bit0=0: DPREFCLK not bypass DFS
+ UCHAR ucCrtc; // use when enable DCCG pixel clock ucFlag[1]=1
+ UCHAR ucPpllId; // use when enable DCCG pixel clock ucFlag[1]=1
+ UCHAR ucDeepColorRatio; // use when enable DCCG pixel clock ucFlag[1]=1
+}SET_DCE_CLOCK_PARAMETERS_V1_1;
+
+
+typedef struct _SET_DCE_CLOCK_PS_ALLOCATION_V1_1
+{
+ SET_DCE_CLOCK_PARAMETERS_V1_1 asParam;
+ ULONG ulReserved[2];
+}SET_DCE_CLOCK_PS_ALLOCATION_V1_1;
+
+//SET_DCE_CLOCK_PARAMETERS_V1_1.ucFlag
+#define SET_DCE_CLOCK_FLAG_GEN_DPREFCLK 0x01
+#define SET_DCE_CLOCK_FLAG_DPREFCLK_BYPASS 0x01
+#define SET_DCE_CLOCK_FLAG_ENABLE_PIXCLK 0x02
+
+// SetDCEClockTable input parameter for DCE11.2( POLARIS10 and POLARIS11 ) and above
+typedef struct _SET_DCE_CLOCK_PARAMETERS_V2_1
+{
+ ULONG ulDCEClkFreq; // target DCE frequency in unit of 10KHZ, return real DISPCLK/DPREFCLK frequency.
+ UCHAR ucDCEClkType; // =0: DISPCLK =1: DPREFCLK =2: PIXCLK
+ UCHAR ucDCEClkSrc; // ATOM_PLL0 or ATOM_GCK_DFS or ATOM_FCH_CLK or ATOM_COMBOPHY_PLLx
+ UCHAR ucDCEClkFlag; // Bit [1:0] = PPLL ref clock source ( when ucDCEClkSrc= ATOM_PPLL0 )
+ UCHAR ucCRTC; // ucDisp Pipe Id, ATOM_CRTC0/1/2/..., use only when ucDCEClkType = PIXCLK
+}SET_DCE_CLOCK_PARAMETERS_V2_1;
+
+//ucDCEClkType
+#define DCE_CLOCK_TYPE_DISPCLK 0
+#define DCE_CLOCK_TYPE_DPREFCLK 1
+#define DCE_CLOCK_TYPE_PIXELCLK 2 // used by VBIOS internally, called by SetPixelClockTable
+
+//ucDCEClkFlag when ucDCEClkType == DPREFCLK
+#define DCE_CLOCK_FLAG_PLL_REFCLK_SRC_MASK 0x03
+#define DCE_CLOCK_FLAG_PLL_REFCLK_SRC_GENERICA 0x00
+#define DCE_CLOCK_FLAG_PLL_REFCLK_SRC_GENLK 0x01
+#define DCE_CLOCK_FLAG_PLL_REFCLK_SRC_PCIE 0x02
+#define DCE_CLOCK_FLAG_PLL_REFCLK_SRC_XTALIN 0x03
+
+//ucDCEClkFlag when ucDCEClkType == PIXCLK
+#define DCE_CLOCK_FLAG_PCLK_DEEPCOLOR_RATIO_MASK 0x03
+#define DCE_CLOCK_FLAG_PCLK_DEEPCOLOR_RATIO_DIS 0x00 //00 - DCCG_DEEP_COLOR_DTO_DISABLE: Disable Deep Color DTO
+#define DCE_CLOCK_FLAG_PCLK_DEEPCOLOR_RATIO_5_4 0x01 //01 - DCCG_DEEP_COLOR_DTO_5_4_RATIO: Set Deep Color DTO to 5:4
+#define DCE_CLOCK_FLAG_PCLK_DEEPCOLOR_RATIO_3_2 0x02 //02 - DCCG_DEEP_COLOR_DTO_3_2_RATIO: Set Deep Color DTO to 3:2
+#define DCE_CLOCK_FLAG_PCLK_DEEPCOLOR_RATIO_2_1 0x03 //03 - DCCG_DEEP_COLOR_DTO_2_1_RATIO: Set Deep Color DTO to 2:1
+#define DCE_CLOCK_FLAG_PIXCLK_YUV420_MODE 0x04
+
+typedef struct _SET_DCE_CLOCK_PS_ALLOCATION_V2_1
+{
+ SET_DCE_CLOCK_PARAMETERS_V2_1 asParam;
+ ULONG ulReserved[2];
+}SET_DCE_CLOCK_PS_ALLOCATION_V2_1;
+
+
/****************************************************************************/
// Structures used by AdjustDisplayPllTable
@@ -2300,6 +2614,11 @@
#define VOLTAGE_TYPE_VDDCI 4
#define VOLTAGE_TYPE_VDDGFX 5
#define VOLTAGE_TYPE_PCC 6
+#define VOLTAGE_TYPE_MVPP 7
+#define VOLTAGE_TYPE_LEDDPM 8
+#define VOLTAGE_TYPE_PCC_MVDD 9
+#define VOLTAGE_TYPE_PCIE_VDDC 10
+#define VOLTAGE_TYPE_PCIE_VDDR 11
#define VOLTAGE_TYPE_GENERIC_I2C_1 0x11
#define VOLTAGE_TYPE_GENERIC_I2C_2 0x12
@@ -2396,6 +2715,39 @@
USHORT usTDP_Power; // TDP_Current in unit of 0.1W
}GET_EVV_VOLTAGE_INFO_OUTPUT_PARAMETER_V1_2;
+
+// New Added from CI Hawaii for GetVoltageInfoTable, input parameter structure
+typedef struct _GET_VOLTAGE_INFO_INPUT_PARAMETER_V1_3
+{
+ UCHAR ucVoltageType; // Input: To tell which voltage to set up, VDDC/MVDDC/MVDDQ/VDDCI
+ UCHAR ucVoltageMode; // Input: Indicate action: Get voltage info
+ USHORT usVoltageLevel; // Input: real voltage level in unit of mv or Voltage Phase (0, 1, 2, .. ) or Leakage Id
+ ULONG ulSCLKFreq; // Input: when ucVoltageMode= ATOM_GET_VOLTAGE_EVV_VOLTAGE, DPM state SCLK frequency, Define in PPTable SCLK/Voltage dependence table
+ ULONG ulReserved[3];
+}GET_VOLTAGE_INFO_INPUT_PARAMETER_V1_3;
+
+// New Added from CI Hawaii for EVV feature
+typedef struct _GET_EVV_VOLTAGE_INFO_OUTPUT_PARAMETER_V1_3
+{
+ ULONG ulVoltageLevel; // real voltage level in unit of 0.01mv
+ ULONG ulReserved[4];
+}GET_EVV_VOLTAGE_INFO_OUTPUT_PARAMETER_V1_3;
+
+
+/****************************************************************************/
+// Structures used by GetSMUClockInfo
+/****************************************************************************/
+typedef struct _GET_SMU_CLOCK_INFO_INPUT_PARAMETER_V2_1
+{
+ ULONG ulDfsPllOutputFreq:24;
+ ULONG ucDfsDivider:8;
+}GET_SMU_CLOCK_INFO_INPUT_PARAMETER_V2_1;
+
+typedef struct _GET_SMU_CLOCK_INFO_OUTPUT_PARAMETER_V2_1
+{
+ ULONG ulDfsOutputFreq;
+}GET_SMU_CLOCK_INFO_OUTPUT_PARAMETER_V2_1;
+
/****************************************************************************/
// Structures used by TVEncoderControlTable
/****************************************************************************/
@@ -2429,13 +2781,13 @@
USHORT PaletteData; // Only used by BIOS
USHORT LCD_Info; // Shared by various SW components,latest version 1.3, was called LVDS_Info
USHORT DIGTransmitterInfo; // Internal used by VBIOS only version 3.1
- USHORT AnalogTV_Info; // Shared by various SW components,latest version 1.1
+ USHORT SMU_Info; // Shared by various SW components,latest version 1.1
USHORT SupportedDevicesInfo; // Will be obsolete from R600
USHORT GPIO_I2C_Info; // Shared by various SW components,latest version 1.2 will be used from R600
USHORT VRAM_UsageByFirmware; // Shared by various SW components,latest version 1.3 will be used from R600
USHORT GPIO_Pin_LUT; // Shared by various SW components,latest version 1.1
USHORT VESA_ToInternalModeLUT; // Only used by Bios
- USHORT ComponentVideoInfo; // Shared by various SW components,latest version 2.1 will be used from R600
+ USHORT GFX_Info; // Shared by various SW components,latest version 2.1 will be used from R600
USHORT PowerPlayInfo; // Shared by various SW components,latest version 2.1,new design from R600
USHORT GPUVirtualizationInfo; // Will be obsolete from R600
USHORT SaveRestoreInfo; // Only used by Bios
@@ -2455,7 +2807,7 @@
USHORT ASIC_ProfilingInfo; // New table name from R600, used to be called "ASIC_VDDCI_Info" for pre-R600
USHORT VoltageObjectInfo; // Shared by various SW components, latest version 1.1
USHORT PowerSourceInfo; // Shared by various SW components, latest versoin 1.1
- USHORT ServiceInfo;
+ USHORT ServiceInfo;
}ATOM_MASTER_LIST_OF_DATA_TABLES;
typedef struct _ATOM_MASTER_DATA_TABLE
@@ -2469,6 +2821,8 @@
#define DAC_Info PaletteData
#define TMDS_Info DIGTransmitterInfo
#define CompassionateData GPUVirtualizationInfo
+#define AnalogTV_Info SMU_Info
+#define ComponentVideoInfo GFX_Info
/****************************************************************************/
// Structure used in MultimediaCapabilityInfoTable
@@ -4278,10 +4632,15 @@
#define MAX_NUMBER_OF_EXT_DISPLAY_PATH 7
//usCaps
-#define EXT_DISPLAY_PATH_CAPS__HBR2_DISABLE 0x01
-#define EXT_DISPLAY_PATH_CAPS__DP_FIXED_VS_EN 0x02
-#define EXT_DISPLAY_PATH_CAPS__HDMI20_PI3EQX1204 0x04
-#define EXT_DISPLAY_PATH_CAPS__HDMI20_TISN65DP159RSBT 0x08
+#define EXT_DISPLAY_PATH_CAPS__HBR2_DISABLE 0x0001
+#define EXT_DISPLAY_PATH_CAPS__DP_FIXED_VS_EN 0x0002
+#define EXT_DISPLAY_PATH_CAPS__EXT_CHIP_MASK 0x007C
+#define EXT_DISPLAY_PATH_CAPS__HDMI20_PI3EQX1204 (0x01 << 2 ) //PI redriver chip
+#define EXT_DISPLAY_PATH_CAPS__HDMI20_TISN65DP159RSBT (0x02 << 2 ) //TI retimer chip
+#define EXT_DISPLAY_PATH_CAPS__HDMI20_PARADE_PS175 (0x03 << 2 ) //Parade DP->HDMI recoverter chip
+
+
+
typedef struct _ATOM_EXTERNAL_DISPLAY_CONNECTION_INFO
{
@@ -4325,10 +4684,10 @@
#define ATOM_CONNECTOR_REMOTE_CAP_RECORD_TYPE 19
#define ATOM_ENCODER_CAP_RECORD_TYPE 20
#define ATOM_BRACKET_LAYOUT_RECORD_TYPE 21
-
+#define ATOM_CONNECTOR_FORCED_TMDS_CAP_RECORD_TYPE 22
//Must be updated when new record type is added,equal to that record definition!
-#define ATOM_MAX_OBJECT_RECORD_NUMBER ATOM_ENCODER_CAP_RECORD_TYPE
+#define ATOM_MAX_OBJECT_RECORD_NUMBER ATOM_CONNECTOR_FORCED_TMDS_CAP_RECORD_TYPE
typedef struct _ATOM_I2C_RECORD
{
@@ -4458,10 +4817,12 @@
UCHAR ucPadding[2];
}ATOM_ENCODER_DVO_CF_RECORD;
-// Bit maps for ATOM_ENCODER_CAP_RECORD.ucEncoderCap
-#define ATOM_ENCODER_CAP_RECORD_HBR2 0x01 // DP1.2 HBR2 is supported by HW encoder
+// Bit maps for ATOM_ENCODER_CAP_RECORD.usEncoderCap
+#define ATOM_ENCODER_CAP_RECORD_HBR2 0x01 // DP1.2 HBR2 is supported by HW encoder, it is retired in NI. the real meaning from SI is MST_EN
+#define ATOM_ENCODER_CAP_RECORD_MST_EN 0x01 // from SI, this bit means DP MST is enable or not.
#define ATOM_ENCODER_CAP_RECORD_HBR2_EN 0x02 // DP1.2 HBR2 setting is qualified and HBR2 can be enabled
#define ATOM_ENCODER_CAP_RECORD_HDMI6Gbps_EN 0x04 // HDMI2.0 6Gbps enable or not.
+#define ATOM_ENCODER_CAP_RECORD_HBR3_EN 0x08 // DP1.3 HBR3 is supported by board.
typedef struct _ATOM_ENCODER_CAP_RECORD
{
@@ -4482,6 +4843,31 @@
};
}ATOM_ENCODER_CAP_RECORD;
+// Used after SI
+typedef struct _ATOM_ENCODER_CAP_RECORD_V2
+{
+ ATOM_COMMON_RECORD_HEADER sheader;
+ union {
+ USHORT usEncoderCap;
+ struct {
+#if ATOM_BIG_ENDIAN
+ USHORT usReserved:12; // Bit4-15 may be defined for other capability in future
+ USHORT usHBR3En:1; // bit3 is for DP1.3 HBR3 enable
+ USHORT usHDMI6GEn:1; // Bit2 is for HDMI6Gbps enable, this bit is used starting from CZ( APU) Ellemere (dGPU)
+ USHORT usHBR2En:1; // Bit1 is for DP1.2 HBR2 enable
+ USHORT usMSTEn:1; // Bit0 is for DP1.2 MST enable
+#else
+ USHORT usMSTEn:1; // Bit0 is for DP1.2 MST enable
+ USHORT usHBR2En:1; // Bit1 is for DP1.2 HBR2 enable
+ USHORT usHDMI6GEn:1; // Bit2 is for HDMI6Gbps enable, this bit is used starting from CZ( APU) Ellemere (dGPU)
+ USHORT usHBR3En:1; // bit3 is for DP1.3 HBR3 enable
+ USHORT usReserved:12; // Bit4-15 may be defined for other capability in future
+#endif
+ };
+ };
+}ATOM_ENCODER_CAP_RECORD_V2;
+
+
// value for ATOM_CONNECTOR_CF_RECORD.ucConnectedDvoBundle
#define ATOM_CONNECTOR_CF_RECORD_CONNECTED_UPPER12BITBUNDLEA 1
#define ATOM_CONNECTOR_CF_RECORD_CONNECTED_LOWER12BITBUNDLEB 2
@@ -4554,6 +4940,16 @@
USHORT usReserved;
}ATOM_CONNECTOR_REMOTE_CAP_RECORD;
+
+typedef struct _ATOM_CONNECTOR_FORCED_TMDS_CAP_RECORD
+{
+ ATOM_COMMON_RECORD_HEADER sheader;
+ // override TMDS capability on this connector when it operate in TMDS mode. usMaxTmdsClkRate = max TMDS Clock in Mhz/2.5
+ UCHAR ucMaxTmdsClkRateIn2_5Mhz;
+ UCHAR ucReserved;
+} ATOM_CONNECTOR_FORCED_TMDS_CAP_RECORD;
+
+
typedef struct _ATOM_CONNECTOR_LAYOUT_INFO
{
USHORT usConnectorObjectId;
@@ -4657,12 +5053,12 @@
#define VOLTAGE_CONTROL_ID_UP1801 0x0C
#define VOLTAGE_CONTROL_ID_ST6788A 0x0D
#define VOLTAGE_CONTROL_ID_CHLIR3564SVI2 0x0E
-#define VOLTAGE_CONTROL_ID_AD527x 0x0F
-#define VOLTAGE_CONTROL_ID_NCP81022 0x10
-#define VOLTAGE_CONTROL_ID_LTC2635 0x11
-#define VOLTAGE_CONTROL_ID_NCP4208 0x12
+#define VOLTAGE_CONTROL_ID_AD527x 0x0F
+#define VOLTAGE_CONTROL_ID_NCP81022 0x10
+#define VOLTAGE_CONTROL_ID_LTC2635 0x11
+#define VOLTAGE_CONTROL_ID_NCP4208 0x12
#define VOLTAGE_CONTROL_ID_IR35xx 0x13
-#define VOLTAGE_CONTROL_ID_RT9403 0x14
+#define VOLTAGE_CONTROL_ID_RT9403 0x14
#define VOLTAGE_CONTROL_ID_GENERIC_I2C 0x40
@@ -4784,11 +5180,38 @@
ULONG ulReserved;
}ATOM_SVID2_VOLTAGE_OBJECT_V3;
+
+
+typedef struct _ATOM_MERGED_VOLTAGE_OBJECT_V3
+{
+ ATOM_VOLTAGE_OBJECT_HEADER_V3 sHeader; // voltage mode = VOLTAGE_OBJ_MERGED_POWER
+ UCHAR ucMergedVType; // VDDC/VDCCI/....
+ UCHAR ucReserved[3];
+}ATOM_MERGED_VOLTAGE_OBJECT_V3;
+
+
+typedef struct _ATOM_EVV_DPM_INFO
+{
+ ULONG ulDPMSclk; // DPM state SCLK
+ USHORT usVAdjOffset; // Adjust Voltage offset in unit of mv
+ UCHAR ucDPMTblVIndex; // Voltage Index in SMC_DPM_Table structure VddcTable/VddGfxTable
+ UCHAR ucDPMState; // DPMState0~7
+} ATOM_EVV_DPM_INFO;
+
+// ucVoltageMode = VOLTAGE_OBJ_EVV
+typedef struct _ATOM_EVV_VOLTAGE_OBJECT_V3
+{
+ ATOM_VOLTAGE_OBJECT_HEADER_V3 sHeader; // voltage mode = VOLTAGE_OBJ_SVID2
+ ATOM_EVV_DPM_INFO asEvvDpmList[8];
+}ATOM_EVV_VOLTAGE_OBJECT_V3;
+
+
typedef union _ATOM_VOLTAGE_OBJECT_V3{
ATOM_GPIO_VOLTAGE_OBJECT_V3 asGpioVoltageObj;
ATOM_I2C_VOLTAGE_OBJECT_V3 asI2cVoltageObj;
ATOM_LEAKAGE_VOLTAGE_OBJECT_V3 asLeakageObj;
ATOM_SVID2_VOLTAGE_OBJECT_V3 asSVID2Obj;
+ ATOM_EVV_VOLTAGE_OBJECT_V3 asEvvObj;
}ATOM_VOLTAGE_OBJECT_V3;
typedef struct _ATOM_VOLTAGE_OBJECT_INFO_V3_1
@@ -4963,7 +5386,11 @@
ULONG ulLkgEncodeMax;
ULONG ulLkgEncodeMin;
ULONG ulEfuseLogisticAlpha;
+
+ union{
USHORT usPowerDpm0;
+ USHORT usParamNegFlag; //bit0 =1 :indicate ulRoBeta is Negative, bit1=1 indicate Kv_m max is postive
+ };
USHORT usPowerDpm1;
USHORT usPowerDpm2;
USHORT usPowerDpm3;
@@ -5067,6 +5494,86 @@
ULONG ulReserved[8]; // Reserved for future ASIC
}ATOM_ASIC_PROFILING_INFO_V3_4;
+// for Polaris10/Polaris11 speed EVV algorithm
+typedef struct _ATOM_ASIC_PROFILING_INFO_V3_5
+{
+ ATOM_COMMON_TABLE_HEADER asHeader;
+ ULONG ulMaxVddc; //Maximum voltage for all parts, in unit of 0.01mv
+ ULONG ulMinVddc; //Minimum voltage for all parts, in unit of 0.01mv
+ USHORT usLkgEuseIndex; //Efuse Lkg_FT address ( BYTE address )
+ UCHAR ucLkgEfuseBitLSB; //Efuse Lkg_FT bit shift in 32bit DWORD
+ UCHAR ucLkgEfuseLength; //Efuse Lkg_FT length
+ ULONG ulLkgEncodeLn_MaxDivMin; //value of ln(Max_Lkg_Ft/Min_Lkg_Ft ) in unit of 0.00001 ( unit=100000 )
+ ULONG ulLkgEncodeMax; //Maximum Lkg_Ft measured value ( or efuse decode value ), in unit of 0.00001 ( unit=100000 )
+ ULONG ulLkgEncodeMin; //Minimum Lkg_Ft measured value ( or efuse decode value ), in unit of 0.00001 ( unit=100000 )
+ EFUSE_LINEAR_FUNC_PARAM sRoFuse;//Efuse RO info: DWORD address, bit shift, length, max/min measure value. in unit of 1.
+ ULONG ulEvvDefaultVddc; //def="EVV_DEFAULT_VDDC" descr="return default VDDC(v) when Efuse not cut" unit="100000"/>
+ ULONG ulEvvNoCalcVddc; //def="EVV_NOCALC_VDDC" descr="return VDDC(v) when Calculation is bad" unit="100000"/>
+ ULONG ulSpeed_Model; //def="EVV_SPEED_MODEL" descr="0 = Greek model, 1 = multivariate model" unit="1"/>
+ ULONG ulSM_A0; //def="EVV_SM_A0" descr="Leakage coeff(Multivariant Mode)." unit="100000"/>
+ ULONG ulSM_A1; //def="EVV_SM_A1" descr="Leakage/SCLK coeff(Multivariant Mode)." unit="1000000"/>
+ ULONG ulSM_A2; //def="EVV_SM_A2" descr="Alpha( Greek Mode ) or VDDC/SCLK coeff(Multivariant Mode)." unit="100000"/>
+ ULONG ulSM_A3; //def="EVV_SM_A3" descr="Beta( Greek Mode ) or SCLK coeff(Multivariant Mode)." unit="100000"/>
+ ULONG ulSM_A4; //def="EVV_SM_A4" descr="VDDC^2/SCLK coeff(Multivariant Mode)." unit="100000"/>
+ ULONG ulSM_A5; //def="EVV_SM_A5" descr="VDDC^2 coeff(Multivariant Mode)." unit="100000"/>
+ ULONG ulSM_A6; //def="EVV_SM_A6" descr="Gamma( Greek Mode ) or VDDC coeff(Multivariant Mode)." unit="100000"/>
+ ULONG ulSM_A7; //def="EVV_SM_A7" descr="Epsilon( Greek Mode ) or constant(Multivariant Mode)." unit="100000"/>
+ UCHAR ucSM_A0_sign; //def="EVV_SM_A0_SIGN" descr="=0 SM_A0 is postive. =1: SM_A0 is negative" unit="1"/>
+ UCHAR ucSM_A1_sign; //def="EVV_SM_A1_SIGN" descr="=0 SM_A1 is postive. =1: SM_A1 is negative" unit="1"/>
+ UCHAR ucSM_A2_sign; //def="EVV_SM_A2_SIGN" descr="=0 SM_A2 is postive. =1: SM_A2 is negative" unit="1"/>
+ UCHAR ucSM_A3_sign; //def="EVV_SM_A3_SIGN" descr="=0 SM_A3 is postive. =1: SM_A3 is negative" unit="1"/>
+ UCHAR ucSM_A4_sign; //def="EVV_SM_A4_SIGN" descr="=0 SM_A4 is postive. =1: SM_A4 is negative" unit="1"/>
+ UCHAR ucSM_A5_sign; //def="EVV_SM_A5_SIGN" descr="=0 SM_A5 is postive. =1: SM_A5 is negative" unit="1"/>
+ UCHAR ucSM_A6_sign; //def="EVV_SM_A6_SIGN" descr="=0 SM_A6 is postive. =1: SM_A6 is negative" unit="1"/>
+ UCHAR ucSM_A7_sign; //def="EVV_SM_A7_SIGN" descr="=0 SM_A7 is postive. =1: SM_A7 is negative" unit="1"/>
+ ULONG ulMargin_RO_a; //def="EVV_MARGIN_RO_A" descr="A Term to represent RO equation in Ax2+Bx+C, unit=1"
+ ULONG ulMargin_RO_b; //def="EVV_MARGIN_RO_B" descr="B Term to represent RO equation in Ax2+Bx+C, unit=1"
+ ULONG ulMargin_RO_c; //def="EVV_MARGIN_RO_C" descr="C Term to represent RO equation in Ax2+Bx+C, unit=1"
+ ULONG ulMargin_fixed; //def="EVV_MARGIN_FIXED" descr="Fixed MHz to add to SCLK margin, unit=1" unit="1"/>
+ ULONG ulMargin_Fmax_mean; //def="EVV_MARGIN_FMAX_MEAN" descr="Percentage to add for Fmas mean margin unit=10000" unit="10000"/>
+ ULONG ulMargin_plat_mean; //def="EVV_MARGIN_PLAT_MEAN" descr="Percentage to add for platform mean margin unit=10000" unit="10000"/>
+ ULONG ulMargin_Fmax_sigma; //def="EVV_MARGIN_FMAX_SIGMA" descr="Percentage to add for Fmax sigma margin unit=10000" unit="10000"/>
+ ULONG ulMargin_plat_sigma; //def="EVV_MARGIN_PLAT_SIGMA" descr="Percentage to add for platform sigma margin unit=10000" unit="10000"/>
+ ULONG ulMargin_DC_sigma; //def="EVV_MARGIN_DC_SIGMA" descr="Regulator DC tolerance margin (mV) unit=100" unit="100"/>
+ ULONG ulReserved[12];
+}ATOM_ASIC_PROFILING_INFO_V3_5;
+
+
+typedef struct _ATOM_SCLK_FCW_RANGE_ENTRY_V1{
+ ULONG ulMaxSclkFreq;
+ UCHAR ucVco_setting; // 1: 3-6GHz, 3: 2-4GHz
+ UCHAR ucPostdiv; // divide by 2^n
+ USHORT ucFcw_pcc;
+ USHORT ucFcw_trans_upper;
+ USHORT ucRcw_trans_lower;
+}ATOM_SCLK_FCW_RANGE_ENTRY_V1;
+
+
+// SMU_InfoTable for Polaris10/Polaris11
+typedef struct _ATOM_SMU_INFO_V2_1
+{
+ ATOM_COMMON_TABLE_HEADER asHeader;
+ UCHAR ucSclkEntryNum; // for potential future extend, indicate the number of ATOM_SCLK_FCW_RANGE_ENTRY_V1
+ UCHAR ucReserved[3];
+ ATOM_SCLK_FCW_RANGE_ENTRY_V1 asSclkFcwRangeEntry[8];
+}ATOM_SMU_INFO_V2_1;
+
+
+// GFX_InfoTable for Polaris10/Polaris11
+typedef struct _ATOM_GFX_INFO_V2_1
+{
+ ATOM_COMMON_TABLE_HEADER asHeader;
+ UCHAR GfxIpMinVer;
+ UCHAR GfxIpMajVer;
+ UCHAR max_shader_engines;
+ UCHAR max_tile_pipes;
+ UCHAR max_cu_per_sh;
+ UCHAR max_sh_per_se;
+ UCHAR max_backends_per_se;
+ UCHAR max_texture_channel_caches;
+}ATOM_GFX_INFO_V2_1;
+
+
typedef struct _ATOM_POWER_SOURCE_OBJECT
{
UCHAR ucPwrSrcId; // Power source
@@ -5765,14 +6272,6 @@
**********************************************************************************************************************/
-// this Table is used for Kaveri/Kabini APU
-typedef struct _ATOM_FUSION_SYSTEM_INFO_V2
-{
- ATOM_INTEGRATED_SYSTEM_INFO_V1_8 sIntegratedSysInfo; // refer to ATOM_INTEGRATED_SYSTEM_INFO_V1_8 definition
- ULONG ulPowerplayTable[128]; // Update comments here to link new powerplay table definition structure
-}ATOM_FUSION_SYSTEM_INFO_V2;
-
-
typedef struct _ATOM_I2C_REG_INFO
{
UCHAR ucI2cRegIndex;
@@ -5859,7 +6358,50 @@
#define EDP_VS_VARIABLE_PREM_MODE 5
-// this IntegrateSystemInfoTable is used for Carrizo
+// ulGPUCapInfo
+#define SYS_INFO_V1_9_GPUCAPSINFO_DISABLE_AUX_MODE_DETECT 0x08
+#define SYS_INFO_V1_9_GPUCAPSINFO_ENABEL_DFS_BYPASS 0x10
+//ulGPUCapInfo[16]=1 indicate SMC firmware is able to support GNB fast resume function, so that driver can call SMC to program most of GNB register during resuming, from ML
+#define SYS_INFO_V1_9_GPUCAPSINFO_GNB_FAST_RESUME_CAPABLE 0x00010000
+//ulGPUCapInfo[18]=1 indicate the IOMMU is not available
+#define SYS_INFO_V1_9_GPUCAPINFO_IOMMU_DISABLE 0x00040000
+//ulGPUCapInfo[19]=1 indicate the MARC Aperture is opened.
+#define SYS_INFO_V1_9_GPUCAPINFO_MARC_APERTURE_ENABLE 0x00080000
+
+
+typedef struct _DPHY_TIMING_PARA
+{
+ UCHAR ucProfileID; // SENSOR_PROFILES
+ ULONG ucPara;
+} DPHY_TIMING_PARA;
+
+typedef struct _DPHY_ELEC_PARA
+{
+ USHORT usPara[3];
+} DPHY_ELEC_PARA;
+
+typedef struct _CAMERA_MODULE_INFO
+{
+ UCHAR ucID; // 0: Rear, 1: Front right of user, 2: Front left of user
+ UCHAR strModuleName[8];
+ DPHY_TIMING_PARA asTimingPara[6]; // Exact number is under estimation and confirmation from sensor vendor
+} CAMERA_MODULE_INFO;
+
+typedef struct _FLASHLIGHT_INFO
+{
+ UCHAR ucID; // 0: Rear, 1: Front
+ UCHAR strName[8];
+} FLASHLIGHT_INFO;
+
+typedef struct _CAMERA_DATA
+{
+ ULONG ulVersionCode;
+ CAMERA_MODULE_INFO asCameraInfo[3]; // Assuming 3 camera sensors max
+ FLASHLIGHT_INFO asFlashInfo; // Assuming 1 flashlight max
+ DPHY_ELEC_PARA asDphyElecPara;
+ ULONG ulCrcVal; // CRC
+}CAMERA_DATA;
+
typedef struct _ATOM_INTEGRATED_SYSTEM_INFO_V1_10
{
ATOM_COMMON_TABLE_HEADER sHeader;
@@ -5883,7 +6425,7 @@
USHORT usPanelRefreshRateRange;
UCHAR ucMemoryType;
UCHAR ucUMAChannelNumber;
- UCHAR strVBIOSMsg[40];
+ ULONG ulMsgReserved[10];
ATOM_TDP_CONFIG asTdpConfig;
ULONG ulReserved[7];
ATOM_CLK_VOLT_CAPABILITY_V2 sDispClkVoltageMapping[8];
@@ -5925,8 +6467,27 @@
UCHAR ucEDPv1_4VSMode;
UCHAR ucReserved2;
ATOM_EXTERNAL_DISPLAY_CONNECTION_INFO sExtDispConnInfo;
+ CAMERA_DATA asCameraInfo;
+ ULONG ulReserved8[29];
}ATOM_INTEGRATED_SYSTEM_INFO_V1_10;
+
+// this Table is used for Kaveri/Kabini APU
+typedef struct _ATOM_FUSION_SYSTEM_INFO_V2
+{
+ ATOM_INTEGRATED_SYSTEM_INFO_V1_8 sIntegratedSysInfo; // refer to ATOM_INTEGRATED_SYSTEM_INFO_V1_8 definition
+ ULONG ulPowerplayTable[128]; // Update comments here to link new powerplay table definition structure
+}ATOM_FUSION_SYSTEM_INFO_V2;
+
+
+typedef struct _ATOM_FUSION_SYSTEM_INFO_V3
+{
+ ATOM_INTEGRATED_SYSTEM_INFO_V1_10 sIntegratedSysInfo; // refer to ATOM_INTEGRATED_SYSTEM_INFO_V1_8 definition
+ ULONG ulPowerplayTable[192]; // Reserve 768 bytes space for PowerPlayInfoTable
+}ATOM_FUSION_SYSTEM_INFO_V3;
+
+#define FUSION_V3_OFFSET_FROM_TOP_OF_FB 0x800
+
/**************************************************************************/
// This portion is only used when ext thermal chip or engine/memory clock SS chip is populated on a design
//Memory SS Info Table
@@ -6193,12 +6754,12 @@
#define ATOM_S3_DFP1_ACTIVE 0x00000008L
#define ATOM_S3_CRT2_ACTIVE 0x00000010L
#define ATOM_S3_LCD2_ACTIVE 0x00000020L
-#define ATOM_S3_DFP6_ACTIVE 0x00000040L
+#define ATOM_S3_DFP6_ACTIVE 0x00000040L
#define ATOM_S3_DFP2_ACTIVE 0x00000080L
#define ATOM_S3_CV_ACTIVE 0x00000100L
-#define ATOM_S3_DFP3_ACTIVE 0x00000200L
-#define ATOM_S3_DFP4_ACTIVE 0x00000400L
-#define ATOM_S3_DFP5_ACTIVE 0x00000800L
+#define ATOM_S3_DFP3_ACTIVE 0x00000200L
+#define ATOM_S3_DFP4_ACTIVE 0x00000400L
+#define ATOM_S3_DFP5_ACTIVE 0x00000800L
#define ATOM_S3_DEVICE_ACTIVE_MASK 0x00000FFFL
@@ -6215,9 +6776,9 @@
#define ATOM_S3_DFP6_CRTC_ACTIVE 0x00400000L
#define ATOM_S3_DFP2_CRTC_ACTIVE 0x00800000L
#define ATOM_S3_CV_CRTC_ACTIVE 0x01000000L
-#define ATOM_S3_DFP3_CRTC_ACTIVE 0x02000000L
-#define ATOM_S3_DFP4_CRTC_ACTIVE 0x04000000L
-#define ATOM_S3_DFP5_CRTC_ACTIVE 0x08000000L
+#define ATOM_S3_DFP3_CRTC_ACTIVE 0x02000000L
+#define ATOM_S3_DFP4_CRTC_ACTIVE 0x04000000L
+#define ATOM_S3_DFP5_CRTC_ACTIVE 0x08000000L
#define ATOM_S3_DEVICE_CRTC_ACTIVE_MASK 0x0FFF0000L
@@ -6238,9 +6799,9 @@
#define ATOM_S3_DFP6_ACTIVEb0 0x40
#define ATOM_S3_DFP2_ACTIVEb0 0x80
#define ATOM_S3_CV_ACTIVEb1 0x01
-#define ATOM_S3_DFP3_ACTIVEb1 0x02
-#define ATOM_S3_DFP4_ACTIVEb1 0x04
-#define ATOM_S3_DFP5_ACTIVEb1 0x08
+#define ATOM_S3_DFP3_ACTIVEb1 0x02
+#define ATOM_S3_DFP4_ACTIVEb1 0x04
+#define ATOM_S3_DFP5_ACTIVEb1 0x08
#define ATOM_S3_ACTIVE_CRTC1w0 0xFFF
@@ -6254,9 +6815,9 @@
#define ATOM_S3_DFP6_CRTC_ACTIVEb2 0x40
#define ATOM_S3_DFP2_CRTC_ACTIVEb2 0x80
#define ATOM_S3_CV_CRTC_ACTIVEb3 0x01
-#define ATOM_S3_DFP3_CRTC_ACTIVEb3 0x02
-#define ATOM_S3_DFP4_CRTC_ACTIVEb3 0x04
-#define ATOM_S3_DFP5_CRTC_ACTIVEb3 0x08
+#define ATOM_S3_DFP3_CRTC_ACTIVEb3 0x02
+#define ATOM_S3_DFP4_CRTC_ACTIVEb3 0x04
+#define ATOM_S3_DFP5_CRTC_ACTIVEb3 0x08
#define ATOM_S3_ACTIVE_CRTC2w1 0xFFF
@@ -6878,15 +7439,18 @@
#define _32Mx16 0x32
#define _32Mx32 0x33
#define _32Mx128 0x35
-#define _64Mx32 0x43
#define _64Mx8 0x41
#define _64Mx16 0x42
+#define _64Mx32 0x43
+#define _64Mx128 0x45
#define _128Mx8 0x51
#define _128Mx16 0x52
#define _128Mx32 0x53
#define _256Mx8 0x61
#define _256Mx16 0x62
+#define _256Mx32 0x63
#define _512Mx8 0x71
+#define _512Mx16 0x72
#define SAMSUNG 0x1
@@ -7407,6 +7971,17 @@
}ATOM_MEMORY_TRAINING_INFO;
+typedef struct _ATOM_MEMORY_TRAINING_INFO_V3_1
+{
+ ATOM_COMMON_TABLE_HEADER sHeader;
+ ULONG ulMCUcodeVersion;
+ USHORT usMCIOInitLen; //len of ATOM_REG_INIT_SETTING array
+ USHORT usMCUcodeLen; //len of ATOM_MC_UCODE_DATA array
+ USHORT usMCIORegInitOffset; //point of offset of ATOM_REG_INIT_SETTING array
+ USHORT usMCUcodeOffset; //point of offset of MC uCode ULONG array.
+}ATOM_MEMORY_TRAINING_INFO_V3_1;
+
+
typedef struct SW_I2C_CNTL_DATA_PARAMETERS
{
UCHAR ucControl;
@@ -7623,7 +8198,7 @@
{
USHORT usTransmitterObjId;
USHORT usSupportDevice;
- UCHAR ucTransmitterCmdTblId;
+ UCHAR ucTransmitterCmdTblId;
UCHAR ucConfig;
UCHAR ucEncoderID; //available 1st encoder ( default )
UCHAR ucOptionEncoderID; //available 2nd encoder ( optional )
diff --git a/drivers/gpu/drm/amd/include/cgs_common.h b/drivers/gpu/drm/amd/include/cgs_common.h
index ab84d49..a461e15 100644
--- a/drivers/gpu/drm/amd/include/cgs_common.h
+++ b/drivers/gpu/drm/amd/include/cgs_common.h
@@ -26,6 +26,8 @@
#include "amd_shared.h"
+struct cgs_device;
+
/**
* enum cgs_gpu_mem_type - GPU memory types
*/
@@ -92,6 +94,7 @@
*/
enum cgs_ucode_id {
CGS_UCODE_ID_SMU = 0,
+ CGS_UCODE_ID_SMU_SK,
CGS_UCODE_ID_SDMA0,
CGS_UCODE_ID_SDMA1,
CGS_UCODE_ID_CP_CE,
@@ -111,6 +114,7 @@
CGS_SYSTEM_INFO_PCIE_MLW,
CGS_SYSTEM_INFO_CG_FLAGS,
CGS_SYSTEM_INFO_PG_FLAGS,
+ CGS_SYSTEM_INFO_GFX_CU_INFO,
CGS_SYSTEM_INFO_ID_MAXIMUM,
};
@@ -223,7 +227,7 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_gpu_mem_info_t)(void *cgs_device, enum cgs_gpu_mem_type type,
+typedef int (*cgs_gpu_mem_info_t)(struct cgs_device *cgs_device, enum cgs_gpu_mem_type type,
uint64_t *mc_start, uint64_t *mc_size,
uint64_t *mem_size);
@@ -239,7 +243,7 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_gmap_kmem_t)(void *cgs_device, void *kmem, uint64_t size,
+typedef int (*cgs_gmap_kmem_t)(struct cgs_device *cgs_device, void *kmem, uint64_t size,
uint64_t min_offset, uint64_t max_offset,
cgs_handle_t *kmem_handle, uint64_t *mcaddr);
@@ -250,7 +254,7 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_gunmap_kmem_t)(void *cgs_device, cgs_handle_t kmem_handle);
+typedef int (*cgs_gunmap_kmem_t)(struct cgs_device *cgs_device, cgs_handle_t kmem_handle);
/**
* cgs_alloc_gpu_mem() - Allocate GPU memory
@@ -279,7 +283,7 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_alloc_gpu_mem_t)(void *cgs_device, enum cgs_gpu_mem_type type,
+typedef int (*cgs_alloc_gpu_mem_t)(struct cgs_device *cgs_device, enum cgs_gpu_mem_type type,
uint64_t size, uint64_t align,
uint64_t min_offset, uint64_t max_offset,
cgs_handle_t *handle);
@@ -291,7 +295,7 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_free_gpu_mem_t)(void *cgs_device, cgs_handle_t handle);
+typedef int (*cgs_free_gpu_mem_t)(struct cgs_device *cgs_device, cgs_handle_t handle);
/**
* cgs_gmap_gpu_mem() - GPU-map GPU memory
@@ -303,7 +307,7 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_gmap_gpu_mem_t)(void *cgs_device, cgs_handle_t handle,
+typedef int (*cgs_gmap_gpu_mem_t)(struct cgs_device *cgs_device, cgs_handle_t handle,
uint64_t *mcaddr);
/**
@@ -315,7 +319,7 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_gunmap_gpu_mem_t)(void *cgs_device, cgs_handle_t handle);
+typedef int (*cgs_gunmap_gpu_mem_t)(struct cgs_device *cgs_device, cgs_handle_t handle);
/**
* cgs_kmap_gpu_mem() - Kernel-map GPU memory
@@ -326,7 +330,7 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_kmap_gpu_mem_t)(void *cgs_device, cgs_handle_t handle,
+typedef int (*cgs_kmap_gpu_mem_t)(struct cgs_device *cgs_device, cgs_handle_t handle,
void **map);
/**
@@ -336,7 +340,7 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_kunmap_gpu_mem_t)(void *cgs_device, cgs_handle_t handle);
+typedef int (*cgs_kunmap_gpu_mem_t)(struct cgs_device *cgs_device, cgs_handle_t handle);
/**
* cgs_read_register() - Read an MMIO register
@@ -345,7 +349,7 @@
*
* Return: register value
*/
-typedef uint32_t (*cgs_read_register_t)(void *cgs_device, unsigned offset);
+typedef uint32_t (*cgs_read_register_t)(struct cgs_device *cgs_device, unsigned offset);
/**
* cgs_write_register() - Write an MMIO register
@@ -353,7 +357,7 @@
* @offset: register offset
* @value: register value
*/
-typedef void (*cgs_write_register_t)(void *cgs_device, unsigned offset,
+typedef void (*cgs_write_register_t)(struct cgs_device *cgs_device, unsigned offset,
uint32_t value);
/**
@@ -363,7 +367,7 @@
*
* Return: register value
*/
-typedef uint32_t (*cgs_read_ind_register_t)(void *cgs_device, enum cgs_ind_reg space,
+typedef uint32_t (*cgs_read_ind_register_t)(struct cgs_device *cgs_device, enum cgs_ind_reg space,
unsigned index);
/**
@@ -372,7 +376,7 @@
* @offset: register offset
* @value: register value
*/
-typedef void (*cgs_write_ind_register_t)(void *cgs_device, enum cgs_ind_reg space,
+typedef void (*cgs_write_ind_register_t)(struct cgs_device *cgs_device, enum cgs_ind_reg space,
unsigned index, uint32_t value);
/**
@@ -382,7 +386,7 @@
*
* Return: Value read
*/
-typedef uint8_t (*cgs_read_pci_config_byte_t)(void *cgs_device, unsigned addr);
+typedef uint8_t (*cgs_read_pci_config_byte_t)(struct cgs_device *cgs_device, unsigned addr);
/**
* cgs_read_pci_config_word() - Read word from PCI configuration space
@@ -391,7 +395,7 @@
*
* Return: Value read
*/
-typedef uint16_t (*cgs_read_pci_config_word_t)(void *cgs_device, unsigned addr);
+typedef uint16_t (*cgs_read_pci_config_word_t)(struct cgs_device *cgs_device, unsigned addr);
/**
* cgs_read_pci_config_dword() - Read dword from PCI configuration space
@@ -400,7 +404,7 @@
*
* Return: Value read
*/
-typedef uint32_t (*cgs_read_pci_config_dword_t)(void *cgs_device,
+typedef uint32_t (*cgs_read_pci_config_dword_t)(struct cgs_device *cgs_device,
unsigned addr);
/**
@@ -409,7 +413,7 @@
* @addr: address
* @value: value to write
*/
-typedef void (*cgs_write_pci_config_byte_t)(void *cgs_device, unsigned addr,
+typedef void (*cgs_write_pci_config_byte_t)(struct cgs_device *cgs_device, unsigned addr,
uint8_t value);
/**
@@ -418,7 +422,7 @@
* @addr: address, must be word-aligned
* @value: value to write
*/
-typedef void (*cgs_write_pci_config_word_t)(void *cgs_device, unsigned addr,
+typedef void (*cgs_write_pci_config_word_t)(struct cgs_device *cgs_device, unsigned addr,
uint16_t value);
/**
@@ -427,7 +431,7 @@
* @addr: address, must be dword-aligned
* @value: value to write
*/
-typedef void (*cgs_write_pci_config_dword_t)(void *cgs_device, unsigned addr,
+typedef void (*cgs_write_pci_config_dword_t)(struct cgs_device *cgs_device, unsigned addr,
uint32_t value);
@@ -441,7 +445,7 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_get_pci_resource_t)(void *cgs_device,
+typedef int (*cgs_get_pci_resource_t)(struct cgs_device *cgs_device,
enum cgs_resource_type resource_type,
uint64_t size,
uint64_t offset,
@@ -458,7 +462,7 @@
* Return: Pointer to start of the table, or NULL on failure
*/
typedef const void *(*cgs_atom_get_data_table_t)(
- void *cgs_device, unsigned table,
+ struct cgs_device *cgs_device, unsigned table,
uint16_t *size, uint8_t *frev, uint8_t *crev);
/**
@@ -470,7 +474,7 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_atom_get_cmd_table_revs_t)(void *cgs_device, unsigned table,
+typedef int (*cgs_atom_get_cmd_table_revs_t)(struct cgs_device *cgs_device, unsigned table,
uint8_t *frev, uint8_t *crev);
/**
@@ -481,7 +485,7 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_atom_exec_cmd_table_t)(void *cgs_device,
+typedef int (*cgs_atom_exec_cmd_table_t)(struct cgs_device *cgs_device,
unsigned table, void *args);
/**
@@ -491,7 +495,7 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_create_pm_request_t)(void *cgs_device, cgs_handle_t *request);
+typedef int (*cgs_create_pm_request_t)(struct cgs_device *cgs_device, cgs_handle_t *request);
/**
* cgs_destroy_pm_request() - Destroy a power management request
@@ -500,7 +504,7 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_destroy_pm_request_t)(void *cgs_device, cgs_handle_t request);
+typedef int (*cgs_destroy_pm_request_t)(struct cgs_device *cgs_device, cgs_handle_t request);
/**
* cgs_set_pm_request() - Activate or deactiveate a PM request
@@ -516,7 +520,7 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_set_pm_request_t)(void *cgs_device, cgs_handle_t request,
+typedef int (*cgs_set_pm_request_t)(struct cgs_device *cgs_device, cgs_handle_t request,
int active);
/**
@@ -528,7 +532,7 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_pm_request_clock_t)(void *cgs_device, cgs_handle_t request,
+typedef int (*cgs_pm_request_clock_t)(struct cgs_device *cgs_device, cgs_handle_t request,
enum cgs_clock clock, unsigned freq);
/**
@@ -540,7 +544,7 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_pm_request_engine_t)(void *cgs_device, cgs_handle_t request,
+typedef int (*cgs_pm_request_engine_t)(struct cgs_device *cgs_device, cgs_handle_t request,
enum cgs_engine engine, int powered);
/**
@@ -551,7 +555,7 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_pm_query_clock_limits_t)(void *cgs_device,
+typedef int (*cgs_pm_query_clock_limits_t)(struct cgs_device *cgs_device,
enum cgs_clock clock,
struct cgs_clock_limits *limits);
@@ -563,7 +567,7 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_set_camera_voltages_t)(void *cgs_device, uint32_t mask,
+typedef int (*cgs_set_camera_voltages_t)(struct cgs_device *cgs_device, uint32_t mask,
const uint32_t *voltages);
/**
* cgs_get_firmware_info - Get the firmware information from core driver
@@ -573,25 +577,25 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_get_firmware_info)(void *cgs_device,
+typedef int (*cgs_get_firmware_info)(struct cgs_device *cgs_device,
enum cgs_ucode_id type,
struct cgs_firmware_info *info);
-typedef int(*cgs_set_powergating_state)(void *cgs_device,
+typedef int(*cgs_set_powergating_state)(struct cgs_device *cgs_device,
enum amd_ip_block_type block_type,
enum amd_powergating_state state);
-typedef int(*cgs_set_clockgating_state)(void *cgs_device,
+typedef int(*cgs_set_clockgating_state)(struct cgs_device *cgs_device,
enum amd_ip_block_type block_type,
enum amd_clockgating_state state);
typedef int(*cgs_get_active_displays_info)(
- void *cgs_device,
+ struct cgs_device *cgs_device,
struct cgs_display_info *info);
-typedef int (*cgs_notify_dpm_enabled)(void *cgs_device, bool enabled);
+typedef int (*cgs_notify_dpm_enabled)(struct cgs_device *cgs_device, bool enabled);
-typedef int (*cgs_call_acpi_method)(void *cgs_device,
+typedef int (*cgs_call_acpi_method)(struct cgs_device *cgs_device,
uint32_t acpi_method,
uint32_t acpi_function,
void *pinput, void *poutput,
@@ -599,7 +603,7 @@
uint32_t input_size,
uint32_t output_size);
-typedef int (*cgs_query_system_info)(void *cgs_device,
+typedef int (*cgs_query_system_info)(struct cgs_device *cgs_device,
struct cgs_system_info *sys_info);
struct cgs_ops {
diff --git a/drivers/gpu/drm/amd/include/cgs_linux.h b/drivers/gpu/drm/amd/include/cgs_linux.h
index 3b47ae3..ca4f600 100644
--- a/drivers/gpu/drm/amd/include/cgs_linux.h
+++ b/drivers/gpu/drm/amd/include/cgs_linux.h
@@ -66,7 +66,7 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_add_irq_source_t)(void *cgs_device, unsigned src_id,
+typedef int (*cgs_add_irq_source_t)(struct cgs_device *cgs_device, unsigned src_id,
unsigned num_types,
cgs_irq_source_set_func_t set,
cgs_irq_handler_func_t handler,
@@ -83,7 +83,7 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_irq_get_t)(void *cgs_device, unsigned src_id, unsigned type);
+typedef int (*cgs_irq_get_t)(struct cgs_device *cgs_device, unsigned src_id, unsigned type);
/**
* cgs_irq_put() - Indicate IRQ source is no longer needed
@@ -98,7 +98,7 @@
*
* Return: 0 on success, -errno otherwise
*/
-typedef int (*cgs_irq_put_t)(void *cgs_device, unsigned src_id, unsigned type);
+typedef int (*cgs_irq_put_t)(struct cgs_device *cgs_device, unsigned src_id, unsigned type);
struct cgs_os_ops {
/* IRQ handling */
diff --git a/drivers/gpu/drm/amd/powerplay/amd_powerplay.c b/drivers/gpu/drm/amd/powerplay/amd_powerplay.c
index 9d22900..8e345bf 100644
--- a/drivers/gpu/drm/amd/powerplay/amd_powerplay.c
+++ b/drivers/gpu/drm/amd/powerplay/amd_powerplay.c
@@ -37,6 +37,12 @@
return -EINVAL; \
} while (0)
+#define PP_CHECK_HW(hwmgr) \
+ do { \
+ if ((hwmgr) == NULL || (hwmgr)->hwmgr_func == NULL) \
+ return -EINVAL; \
+ } while (0)
+
static int pp_early_init(void *handle)
{
return 0;
@@ -54,22 +60,26 @@
pp_handle = (struct pp_instance *)handle;
hwmgr = pp_handle->hwmgr;
- if (hwmgr == NULL || hwmgr->pptable_func == NULL ||
- hwmgr->hwmgr_func == NULL ||
+ PP_CHECK_HW(hwmgr);
+
+ if (hwmgr->pptable_func == NULL ||
hwmgr->pptable_func->pptable_init == NULL ||
hwmgr->hwmgr_func->backend_init == NULL)
return -EINVAL;
ret = hwmgr->pptable_func->pptable_init(hwmgr);
-
- if (ret == 0)
- ret = hwmgr->hwmgr_func->backend_init(hwmgr);
-
if (ret)
- printk("amdgpu: powerplay initialization failed\n");
- else
- printk("amdgpu: powerplay initialized\n");
+ goto err;
+ ret = hwmgr->hwmgr_func->backend_init(hwmgr);
+ if (ret)
+ goto err;
+
+ pr_info("amdgpu: powerplay initialized\n");
+
+ return 0;
+err:
+ pr_err("amdgpu: powerplay initialization failed\n");
return ret;
}
@@ -85,8 +95,9 @@
pp_handle = (struct pp_instance *)handle;
hwmgr = pp_handle->hwmgr;
- if (hwmgr != NULL || hwmgr->hwmgr_func != NULL ||
- hwmgr->hwmgr_func->backend_fini != NULL)
+ PP_CHECK_HW(hwmgr);
+
+ if (hwmgr->hwmgr_func->backend_fini != NULL)
ret = hwmgr->hwmgr_func->backend_fini(hwmgr);
return ret;
@@ -172,21 +183,117 @@
return 0;
}
-static void pp_print_status(void *handle)
-{
-
-}
static int pp_set_clockgating_state(void *handle,
enum amd_clockgating_state state)
{
+ struct pp_hwmgr *hwmgr;
+ uint32_t msg_id, pp_state;
+
+ if (handle == NULL)
+ return -EINVAL;
+
+ hwmgr = ((struct pp_instance *)handle)->hwmgr;
+
+ PP_CHECK_HW(hwmgr);
+
+ if (hwmgr->hwmgr_func->update_clock_gatings == NULL) {
+ printk(KERN_INFO "%s was not implemented.\n", __func__);
+ return 0;
+ }
+
+ if (state == AMD_CG_STATE_UNGATE)
+ pp_state = 0;
+ else
+ pp_state = PP_STATE_CG | PP_STATE_LS;
+
+ /* Enable/disable GFX blocks clock gating through SMU */
+ msg_id = PP_CG_MSG_ID(PP_GROUP_GFX,
+ PP_BLOCK_GFX_CG,
+ PP_STATE_SUPPORT_CG | PP_STATE_SUPPORT_LS,
+ pp_state);
+ hwmgr->hwmgr_func->update_clock_gatings(hwmgr, &msg_id);
+ msg_id = PP_CG_MSG_ID(PP_GROUP_GFX,
+ PP_BLOCK_GFX_3D,
+ PP_STATE_SUPPORT_CG | PP_STATE_SUPPORT_LS,
+ pp_state);
+ hwmgr->hwmgr_func->update_clock_gatings(hwmgr, &msg_id);
+ msg_id = PP_CG_MSG_ID(PP_GROUP_GFX,
+ PP_BLOCK_GFX_RLC,
+ PP_STATE_SUPPORT_CG | PP_STATE_SUPPORT_LS,
+ pp_state);
+ hwmgr->hwmgr_func->update_clock_gatings(hwmgr, &msg_id);
+ msg_id = PP_CG_MSG_ID(PP_GROUP_GFX,
+ PP_BLOCK_GFX_CP,
+ PP_STATE_SUPPORT_CG | PP_STATE_SUPPORT_LS,
+ pp_state);
+ hwmgr->hwmgr_func->update_clock_gatings(hwmgr, &msg_id);
+ msg_id = PP_CG_MSG_ID(PP_GROUP_GFX,
+ PP_BLOCK_GFX_MG,
+ PP_STATE_SUPPORT_CG | PP_STATE_SUPPORT_LS,
+ pp_state);
+ hwmgr->hwmgr_func->update_clock_gatings(hwmgr, &msg_id);
+
+ /* Enable/disable System blocks clock gating through SMU */
+ msg_id = PP_CG_MSG_ID(PP_GROUP_SYS,
+ PP_BLOCK_SYS_BIF,
+ PP_STATE_SUPPORT_CG | PP_STATE_SUPPORT_LS,
+ pp_state);
+ hwmgr->hwmgr_func->update_clock_gatings(hwmgr, &msg_id);
+ msg_id = PP_CG_MSG_ID(PP_GROUP_SYS,
+ PP_BLOCK_SYS_BIF,
+ PP_STATE_SUPPORT_CG | PP_STATE_SUPPORT_LS,
+ pp_state);
+ hwmgr->hwmgr_func->update_clock_gatings(hwmgr, &msg_id);
+ msg_id = PP_CG_MSG_ID(PP_GROUP_SYS,
+ PP_BLOCK_SYS_MC,
+ PP_STATE_SUPPORT_CG | PP_STATE_SUPPORT_LS,
+ pp_state);
+ hwmgr->hwmgr_func->update_clock_gatings(hwmgr, &msg_id);
+ msg_id = PP_CG_MSG_ID(PP_GROUP_SYS,
+ PP_BLOCK_SYS_ROM,
+ PP_STATE_SUPPORT_CG | PP_STATE_SUPPORT_LS,
+ pp_state);
+ hwmgr->hwmgr_func->update_clock_gatings(hwmgr, &msg_id);
+ msg_id = PP_CG_MSG_ID(PP_GROUP_SYS,
+ PP_BLOCK_SYS_DRM,
+ PP_STATE_SUPPORT_CG | PP_STATE_SUPPORT_LS,
+ pp_state);
+ hwmgr->hwmgr_func->update_clock_gatings(hwmgr, &msg_id);
+ msg_id = PP_CG_MSG_ID(PP_GROUP_SYS,
+ PP_BLOCK_SYS_HDP,
+ PP_STATE_SUPPORT_CG | PP_STATE_SUPPORT_LS,
+ pp_state);
+ hwmgr->hwmgr_func->update_clock_gatings(hwmgr, &msg_id);
+ msg_id = PP_CG_MSG_ID(PP_GROUP_SYS,
+ PP_BLOCK_SYS_SDMA,
+ PP_STATE_SUPPORT_CG | PP_STATE_SUPPORT_LS,
+ pp_state);
+ hwmgr->hwmgr_func->update_clock_gatings(hwmgr, &msg_id);
+
return 0;
}
static int pp_set_powergating_state(void *handle,
enum amd_powergating_state state)
{
- return 0;
+ struct pp_hwmgr *hwmgr;
+
+ if (handle == NULL)
+ return -EINVAL;
+
+ hwmgr = ((struct pp_instance *)handle)->hwmgr;
+
+ PP_CHECK_HW(hwmgr);
+
+ if (hwmgr->hwmgr_func->enable_per_cu_power_gating == NULL) {
+ printk(KERN_INFO "%s was not implemented.\n", __func__);
+ return 0;
+ }
+
+ /* Enable/disable GFX per cu powergating through SMU */
+ return hwmgr->hwmgr_func->enable_per_cu_power_gating(hwmgr,
+ state == AMD_PG_STATE_GATE ? true : false);
}
static int pp_suspend(void *handle)
@@ -236,6 +343,7 @@
}
const struct amd_ip_funcs pp_ip_funcs = {
+ .name = "powerplay",
.early_init = pp_early_init,
.late_init = NULL,
.sw_init = pp_sw_init,
@@ -247,7 +355,6 @@
.is_idle = pp_is_idle,
.wait_for_idle = pp_wait_for_idle,
.soft_reset = pp_sw_reset,
- .print_status = pp_print_status,
.set_clockgating_state = pp_set_clockgating_state,
.set_powergating_state = pp_set_powergating_state,
};
@@ -275,9 +382,12 @@
hwmgr = pp_handle->hwmgr;
- if (hwmgr == NULL || hwmgr->hwmgr_func == NULL ||
- hwmgr->hwmgr_func->force_dpm_level == NULL)
- return -EINVAL;
+ PP_CHECK_HW(hwmgr);
+
+ if (hwmgr->hwmgr_func->force_dpm_level == NULL) {
+ printk(KERN_INFO "%s was not implemented.\n", __func__);
+ return 0;
+ }
hwmgr->hwmgr_func->force_dpm_level(hwmgr, level);
@@ -309,9 +419,12 @@
hwmgr = ((struct pp_instance *)handle)->hwmgr;
- if (hwmgr == NULL || hwmgr->hwmgr_func == NULL ||
- hwmgr->hwmgr_func->get_sclk == NULL)
- return -EINVAL;
+ PP_CHECK_HW(hwmgr);
+
+ if (hwmgr->hwmgr_func->get_sclk == NULL) {
+ printk(KERN_INFO "%s was not implemented.\n", __func__);
+ return 0;
+ }
return hwmgr->hwmgr_func->get_sclk(hwmgr, low);
}
@@ -325,9 +438,12 @@
hwmgr = ((struct pp_instance *)handle)->hwmgr;
- if (hwmgr == NULL || hwmgr->hwmgr_func == NULL ||
- hwmgr->hwmgr_func->get_mclk == NULL)
- return -EINVAL;
+ PP_CHECK_HW(hwmgr);
+
+ if (hwmgr->hwmgr_func->get_mclk == NULL) {
+ printk(KERN_INFO "%s was not implemented.\n", __func__);
+ return 0;
+ }
return hwmgr->hwmgr_func->get_mclk(hwmgr, low);
}
@@ -341,9 +457,12 @@
hwmgr = ((struct pp_instance *)handle)->hwmgr;
- if (hwmgr == NULL || hwmgr->hwmgr_func == NULL ||
- hwmgr->hwmgr_func->powergate_vce == NULL)
- return -EINVAL;
+ PP_CHECK_HW(hwmgr);
+
+ if (hwmgr->hwmgr_func->powergate_vce == NULL) {
+ printk(KERN_INFO "%s was not implemented.\n", __func__);
+ return 0;
+ }
return hwmgr->hwmgr_func->powergate_vce(hwmgr, gate);
}
@@ -357,9 +476,12 @@
hwmgr = ((struct pp_instance *)handle)->hwmgr;
- if (hwmgr == NULL || hwmgr->hwmgr_func == NULL ||
- hwmgr->hwmgr_func->powergate_uvd == NULL)
- return -EINVAL;
+ PP_CHECK_HW(hwmgr);
+
+ if (hwmgr->hwmgr_func->powergate_uvd == NULL) {
+ printk(KERN_INFO "%s was not implemented.\n", __func__);
+ return 0;
+ }
return hwmgr->hwmgr_func->powergate_uvd(hwmgr, gate);
}
@@ -455,10 +577,14 @@
hwmgr = ((struct pp_instance *)handle)->hwmgr;
- if (hwmgr == NULL || hwmgr->hwmgr_func == NULL ||
- hwmgr->hwmgr_func->print_current_perforce_level == NULL)
+ if (hwmgr == NULL || hwmgr->hwmgr_func == NULL)
return;
+ if (hwmgr->hwmgr_func->print_current_perforce_level == NULL) {
+ printk(KERN_INFO "%s was not implemented.\n", __func__);
+ return;
+ }
+
hwmgr->hwmgr_func->print_current_perforce_level(hwmgr, m);
}
@@ -471,9 +597,12 @@
hwmgr = ((struct pp_instance *)handle)->hwmgr;
- if (hwmgr == NULL || hwmgr->hwmgr_func == NULL ||
- hwmgr->hwmgr_func->set_fan_control_mode == NULL)
- return -EINVAL;
+ PP_CHECK_HW(hwmgr);
+
+ if (hwmgr->hwmgr_func->set_fan_control_mode == NULL) {
+ printk(KERN_INFO "%s was not implemented.\n", __func__);
+ return 0;
+ }
return hwmgr->hwmgr_func->set_fan_control_mode(hwmgr, mode);
}
@@ -487,9 +616,12 @@
hwmgr = ((struct pp_instance *)handle)->hwmgr;
- if (hwmgr == NULL || hwmgr->hwmgr_func == NULL ||
- hwmgr->hwmgr_func->get_fan_control_mode == NULL)
- return -EINVAL;
+ PP_CHECK_HW(hwmgr);
+
+ if (hwmgr->hwmgr_func->get_fan_control_mode == NULL) {
+ printk(KERN_INFO "%s was not implemented.\n", __func__);
+ return 0;
+ }
return hwmgr->hwmgr_func->get_fan_control_mode(hwmgr);
}
@@ -503,9 +635,12 @@
hwmgr = ((struct pp_instance *)handle)->hwmgr;
- if (hwmgr == NULL || hwmgr->hwmgr_func == NULL ||
- hwmgr->hwmgr_func->set_fan_speed_percent == NULL)
- return -EINVAL;
+ PP_CHECK_HW(hwmgr);
+
+ if (hwmgr->hwmgr_func->set_fan_speed_percent == NULL) {
+ printk(KERN_INFO "%s was not implemented.\n", __func__);
+ return 0;
+ }
return hwmgr->hwmgr_func->set_fan_speed_percent(hwmgr, percent);
}
@@ -519,9 +654,12 @@
hwmgr = ((struct pp_instance *)handle)->hwmgr;
- if (hwmgr == NULL || hwmgr->hwmgr_func == NULL ||
- hwmgr->hwmgr_func->get_fan_speed_percent == NULL)
- return -EINVAL;
+ PP_CHECK_HW(hwmgr);
+
+ if (hwmgr->hwmgr_func->get_fan_speed_percent == NULL) {
+ printk(KERN_INFO "%s was not implemented.\n", __func__);
+ return 0;
+ }
return hwmgr->hwmgr_func->get_fan_speed_percent(hwmgr, speed);
}
@@ -535,9 +673,12 @@
hwmgr = ((struct pp_instance *)handle)->hwmgr;
- if (hwmgr == NULL || hwmgr->hwmgr_func == NULL ||
- hwmgr->hwmgr_func->get_temperature == NULL)
- return -EINVAL;
+ PP_CHECK_HW(hwmgr);
+
+ if (hwmgr->hwmgr_func->get_temperature == NULL) {
+ printk(KERN_INFO "%s was not implemented.\n", __func__);
+ return 0;
+ }
return hwmgr->hwmgr_func->get_temperature(hwmgr);
}
@@ -591,9 +732,12 @@
hwmgr = ((struct pp_instance *)handle)->hwmgr;
- if (hwmgr == NULL || hwmgr->hwmgr_func == NULL ||
- hwmgr->hwmgr_func->get_pp_table == NULL)
- return -EINVAL;
+ PP_CHECK_HW(hwmgr);
+
+ if (hwmgr->hwmgr_func->get_pp_table == NULL) {
+ printk(KERN_INFO "%s was not implemented.\n", __func__);
+ return 0;
+ }
return hwmgr->hwmgr_func->get_pp_table(hwmgr, table);
}
@@ -607,15 +751,18 @@
hwmgr = ((struct pp_instance *)handle)->hwmgr;
- if (hwmgr == NULL || hwmgr->hwmgr_func == NULL ||
- hwmgr->hwmgr_func->set_pp_table == NULL)
- return -EINVAL;
+ PP_CHECK_HW(hwmgr);
+
+ if (hwmgr->hwmgr_func->set_pp_table == NULL) {
+ printk(KERN_INFO "%s was not implemented.\n", __func__);
+ return 0;
+ }
return hwmgr->hwmgr_func->set_pp_table(hwmgr, buf, size);
}
static int pp_dpm_force_clock_level(void *handle,
- enum pp_clock_type type, int level)
+ enum pp_clock_type type, uint32_t mask)
{
struct pp_hwmgr *hwmgr;
@@ -624,11 +771,14 @@
hwmgr = ((struct pp_instance *)handle)->hwmgr;
- if (hwmgr == NULL || hwmgr->hwmgr_func == NULL ||
- hwmgr->hwmgr_func->force_clock_level == NULL)
- return -EINVAL;
+ PP_CHECK_HW(hwmgr);
- return hwmgr->hwmgr_func->force_clock_level(hwmgr, type, level);
+ if (hwmgr->hwmgr_func->force_clock_level == NULL) {
+ printk(KERN_INFO "%s was not implemented.\n", __func__);
+ return 0;
+ }
+
+ return hwmgr->hwmgr_func->force_clock_level(hwmgr, type, mask);
}
static int pp_dpm_print_clock_levels(void *handle,
@@ -641,10 +791,12 @@
hwmgr = ((struct pp_instance *)handle)->hwmgr;
- if (hwmgr == NULL || hwmgr->hwmgr_func == NULL ||
- hwmgr->hwmgr_func->print_clock_levels == NULL)
- return -EINVAL;
+ PP_CHECK_HW(hwmgr);
+ if (hwmgr->hwmgr_func->print_clock_levels == NULL) {
+ printk(KERN_INFO "%s was not implemented.\n", __func__);
+ return 0;
+ }
return hwmgr->hwmgr_func->print_clock_levels(hwmgr, type, buf);
}
diff --git a/drivers/gpu/drm/amd/powerplay/eventmgr/eventactionchains.c b/drivers/gpu/drm/amd/powerplay/eventmgr/eventactionchains.c
index 56856a2..d6635cc 100644
--- a/drivers/gpu/drm/amd/powerplay/eventmgr/eventactionchains.c
+++ b/drivers/gpu/drm/amd/powerplay/eventmgr/eventactionchains.c
@@ -24,7 +24,7 @@
#include "eventactionchains.h"
#include "eventsubchains.h"
-static const pem_event_action *initialize_event[] = {
+static const pem_event_action * const initialize_event[] = {
block_adjust_power_state_tasks,
power_budget_tasks,
system_config_tasks,
@@ -45,7 +45,7 @@
initialize_event
};
-static const pem_event_action *uninitialize_event[] = {
+static const pem_event_action * const uninitialize_event[] = {
ungate_all_display_phys_tasks,
uninitialize_display_phy_access_tasks,
disable_gfx_voltage_island_power_gating_tasks,
@@ -64,7 +64,7 @@
uninitialize_event
};
-static const pem_event_action *power_source_change_event_pp_enabled[] = {
+static const pem_event_action * const power_source_change_event_pp_enabled[] = {
set_power_source_tasks,
set_power_saving_state_tasks,
adjust_power_state_tasks,
@@ -79,7 +79,7 @@
power_source_change_event_pp_enabled
};
-static const pem_event_action *power_source_change_event_pp_disabled[] = {
+static const pem_event_action * const power_source_change_event_pp_disabled[] = {
set_power_source_tasks,
set_nbmcu_state_tasks,
NULL
@@ -90,7 +90,7 @@
power_source_change_event_pp_disabled
};
-static const pem_event_action *power_source_change_event_hardware_dc[] = {
+static const pem_event_action * const power_source_change_event_hardware_dc[] = {
set_power_source_tasks,
set_power_saving_state_tasks,
adjust_power_state_tasks,
@@ -106,7 +106,7 @@
power_source_change_event_hardware_dc
};
-static const pem_event_action *suspend_event[] = {
+static const pem_event_action * const suspend_event[] = {
reset_display_phy_access_tasks,
unregister_interrupt_tasks,
disable_gfx_voltage_island_power_gating_tasks,
@@ -130,7 +130,7 @@
suspend_event
};
-static const pem_event_action *resume_event[] = {
+static const pem_event_action * const resume_event[] = {
unblock_hw_access_tasks,
resume_connected_standby_tasks,
notify_smu_resume_tasks,
@@ -164,7 +164,7 @@
resume_event
};
-static const pem_event_action *complete_init_event[] = {
+static const pem_event_action * const complete_init_event[] = {
unblock_adjust_power_state_tasks,
adjust_power_state_tasks,
enable_gfx_clock_gating_tasks,
@@ -178,7 +178,7 @@
complete_init_event
};
-static const pem_event_action *enable_gfx_clock_gating_event[] = {
+static const pem_event_action * const enable_gfx_clock_gating_event[] = {
enable_gfx_clock_gating_tasks,
NULL
};
@@ -188,7 +188,7 @@
enable_gfx_clock_gating_event
};
-static const pem_event_action *disable_gfx_clock_gating_event[] = {
+static const pem_event_action * const disable_gfx_clock_gating_event[] = {
disable_gfx_clock_gating_tasks,
NULL
};
@@ -198,7 +198,7 @@
disable_gfx_clock_gating_event
};
-static const pem_event_action *enable_cgpg_event[] = {
+static const pem_event_action * const enable_cgpg_event[] = {
enable_cgpg_tasks,
NULL
};
@@ -208,7 +208,7 @@
enable_cgpg_event
};
-static const pem_event_action *disable_cgpg_event[] = {
+static const pem_event_action * const disable_cgpg_event[] = {
disable_cgpg_tasks,
NULL
};
@@ -221,7 +221,7 @@
/* Enable user _2d performance and activate */
-static const pem_event_action *enable_user_state_event[] = {
+static const pem_event_action * const enable_user_state_event[] = {
create_new_user_performance_state_tasks,
adjust_power_state_tasks,
NULL
@@ -232,7 +232,7 @@
enable_user_state_event
};
-static const pem_event_action *enable_user_2d_performance_event[] = {
+static const pem_event_action * const enable_user_2d_performance_event[] = {
enable_user_2d_performance_tasks,
add_user_2d_performance_state_tasks,
set_performance_state_tasks,
@@ -247,7 +247,7 @@
};
-static const pem_event_action *disable_user_2d_performance_event[] = {
+static const pem_event_action * const disable_user_2d_performance_event[] = {
disable_user_2d_performance_tasks,
delete_user_2d_performance_state_tasks,
NULL
@@ -259,7 +259,7 @@
};
-static const pem_event_action *display_config_change_event[] = {
+static const pem_event_action * const display_config_change_event[] = {
/* countDisplayConfigurationChangeEventTasks, */
unblock_adjust_power_state_tasks,
set_cpu_power_state,
@@ -278,7 +278,7 @@
display_config_change_event
};
-static const pem_event_action *readjust_power_state_event[] = {
+static const pem_event_action * const readjust_power_state_event[] = {
adjust_power_state_tasks,
NULL
};
diff --git a/drivers/gpu/drm/amd/powerplay/eventmgr/eventmanagement.c b/drivers/gpu/drm/amd/powerplay/eventmgr/eventmanagement.c
index 1e2ad56..cd1ca07 100644
--- a/drivers/gpu/drm/amd/powerplay/eventmgr/eventmanagement.c
+++ b/drivers/gpu/drm/amd/powerplay/eventmgr/eventmanagement.c
@@ -62,7 +62,7 @@
int pem_excute_event_chain(struct pp_eventmgr *eventmgr, const struct action_chain *event_chain, struct pem_event_data *event_data)
{
- const pem_event_action **paction_chain;
+ const pem_event_action * const *paction_chain;
const pem_event_action *psub_chain;
int tmp_result = 0;
int result = 0;
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/Makefile b/drivers/gpu/drm/amd/powerplay/hwmgr/Makefile
index b664e34..f7ce4cb 100644
--- a/drivers/gpu/drm/amd/powerplay/hwmgr/Makefile
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/Makefile
@@ -8,7 +8,9 @@
tonga_processpptables.o ppatomctrl.o \
tonga_hwmgr.o pppcielanes.o tonga_thermal.o\
fiji_powertune.o fiji_hwmgr.o tonga_clockpowergating.o \
- fiji_clockpowergating.o fiji_thermal.o
+ fiji_clockpowergating.o fiji_thermal.o \
+ polaris10_hwmgr.o polaris10_powertune.o polaris10_thermal.o \
+ polaris10_clockpowergating.o
AMD_PP_HWMGR = $(addprefix $(AMD_PP_PATH)/hwmgr/,$(HARDWARE_MGR))
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/cz_clockpowergating.c b/drivers/gpu/drm/amd/powerplay/hwmgr/cz_clockpowergating.c
index ff08ce4..436fc16 100644
--- a/drivers/gpu/drm/amd/powerplay/hwmgr/cz_clockpowergating.c
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/cz_clockpowergating.c
@@ -237,7 +237,7 @@
}
-static struct phm_master_table_item cz_enable_clock_power_gatings_list[] = {
+static const struct phm_master_table_item cz_enable_clock_power_gatings_list[] = {
/*we don't need an exit table here, because there is only D3 cold on Kv*/
{ phm_cf_want_uvd_power_gating, cz_tf_uvd_power_gating_initialize },
{ phm_cf_want_vce_power_gating, cz_tf_vce_power_gating_initialize },
@@ -245,7 +245,7 @@
{ NULL, NULL }
};
-struct phm_master_table_header cz_phm_enable_clock_power_gatings_master = {
+const struct phm_master_table_header cz_phm_enable_clock_power_gatings_master = {
0,
PHM_MasterTableFlag_None,
cz_enable_clock_power_gatings_list
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/cz_clockpowergating.h b/drivers/gpu/drm/amd/powerplay/hwmgr/cz_clockpowergating.h
index bbbc057..1954cea 100644
--- a/drivers/gpu/drm/amd/powerplay/hwmgr/cz_clockpowergating.h
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/cz_clockpowergating.h
@@ -28,8 +28,7 @@
#include "pp_asicblocks.h"
extern int cz_phm_set_asic_block_gating(struct pp_hwmgr *hwmgr, enum PHM_AsicBlock block, enum PHM_ClockGateSetting gating);
-extern struct phm_master_table_header cz_phm_enable_clock_power_gatings_master;
-extern struct phm_master_table_header cz_phm_disable_clock_power_gatings_master;
+extern const struct phm_master_table_header cz_phm_enable_clock_power_gatings_master;
extern int cz_dpm_powergate_vce(struct pp_hwmgr *hwmgr, bool bgate);
extern int cz_dpm_powergate_uvd(struct pp_hwmgr *hwmgr, bool bgate);
extern int cz_enable_disable_vce_dpm(struct pp_hwmgr *hwmgr, bool enable);
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/cz_hwmgr.c b/drivers/gpu/drm/amd/powerplay/hwmgr/cz_hwmgr.c
index 5682490..1f14c47 100644
--- a/drivers/gpu/drm/amd/powerplay/hwmgr/cz_hwmgr.c
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/cz_hwmgr.c
@@ -915,7 +915,7 @@
return 0;
}
-static struct phm_master_table_item cz_set_power_state_list[] = {
+static const struct phm_master_table_item cz_set_power_state_list[] = {
{NULL, cz_tf_update_sclk_limit},
{NULL, cz_tf_set_deep_sleep_sclk_threshold},
{NULL, cz_tf_set_watermark_threshold},
@@ -925,13 +925,13 @@
{NULL, NULL}
};
-static struct phm_master_table_header cz_set_power_state_master = {
+static const struct phm_master_table_header cz_set_power_state_master = {
0,
PHM_MasterTableFlag_None,
cz_set_power_state_list
};
-static struct phm_master_table_item cz_setup_asic_list[] = {
+static const struct phm_master_table_item cz_setup_asic_list[] = {
{NULL, cz_tf_reset_active_process_mask},
{NULL, cz_tf_upload_pptable_to_smu},
{NULL, cz_tf_init_sclk_limit},
@@ -943,7 +943,7 @@
{NULL, NULL}
};
-static struct phm_master_table_header cz_setup_asic_master = {
+static const struct phm_master_table_header cz_setup_asic_master = {
0,
PHM_MasterTableFlag_None,
cz_setup_asic_list
@@ -984,14 +984,14 @@
return 0;
}
-static struct phm_master_table_item cz_power_down_asic_list[] = {
+static const struct phm_master_table_item cz_power_down_asic_list[] = {
{NULL, cz_tf_power_up_display_clock_sys_pll},
{NULL, cz_tf_clear_nb_dpm_flag},
{NULL, cz_tf_reset_cc6_data},
{NULL, NULL}
};
-static struct phm_master_table_header cz_power_down_asic_master = {
+static const struct phm_master_table_header cz_power_down_asic_master = {
0,
PHM_MasterTableFlag_None,
cz_power_down_asic_list
@@ -1095,19 +1095,19 @@
return 0;
}
-static struct phm_master_table_item cz_disable_dpm_list[] = {
+static const struct phm_master_table_item cz_disable_dpm_list[] = {
{ NULL, cz_tf_check_for_dpm_enabled},
{NULL, NULL},
};
-static struct phm_master_table_header cz_disable_dpm_master = {
+static const struct phm_master_table_header cz_disable_dpm_master = {
0,
PHM_MasterTableFlag_None,
cz_disable_dpm_list
};
-static struct phm_master_table_item cz_enable_dpm_list[] = {
+static const struct phm_master_table_item cz_enable_dpm_list[] = {
{ NULL, cz_tf_check_for_dpm_disabled },
{ NULL, cz_tf_program_voting_clients },
{ NULL, cz_tf_start_dpm},
@@ -1117,7 +1117,7 @@
{NULL, NULL},
};
-static struct phm_master_table_header cz_enable_dpm_master = {
+static const struct phm_master_table_header cz_enable_dpm_master = {
0,
PHM_MasterTableFlag_None,
cz_enable_dpm_list
@@ -1729,7 +1729,7 @@
}
static int cz_force_clock_level(struct pp_hwmgr *hwmgr,
- enum pp_clock_type type, int level)
+ enum pp_clock_type type, uint32_t mask)
{
if (hwmgr->dpm_level != AMD_DPM_FORCED_LEVEL_MANUAL)
return -EINVAL;
@@ -1738,10 +1738,10 @@
case PP_SCLK:
smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
PPSMC_MSG_SetSclkSoftMin,
- (1 << level));
+ mask);
smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
PPSMC_MSG_SetSclkSoftMax,
- (1 << level));
+ mask);
break;
default:
break;
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/fiji_clockpowergating.c b/drivers/gpu/drm/amd/powerplay/hwmgr/fiji_clockpowergating.c
index e68edf0..e1b649b 100644
--- a/drivers/gpu/drm/amd/powerplay/hwmgr/fiji_clockpowergating.c
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/fiji_clockpowergating.c
@@ -47,10 +47,17 @@
data->uvd_power_gated = bgate;
- if (bgate)
+ if (bgate) {
+ cgs_set_clockgating_state(hwmgr->device,
+ AMD_IP_BLOCK_TYPE_UVD,
+ AMD_CG_STATE_GATE);
fiji_update_uvd_dpm(hwmgr, true);
- else
+ } else {
fiji_update_uvd_dpm(hwmgr, false);
+ cgs_set_clockgating_state(hwmgr->device,
+ AMD_IP_BLOCK_TYPE_UVD,
+ AMD_PG_STATE_UNGATE);
+ }
return 0;
}
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/fiji_hwmgr.c b/drivers/gpu/drm/amd/powerplay/hwmgr/fiji_hwmgr.c
index 89f31bc..c94f9fa 100644
--- a/drivers/gpu/drm/amd/powerplay/hwmgr/fiji_hwmgr.c
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/fiji_hwmgr.c
@@ -95,23 +95,23 @@
/* [2.5%,~2.5%] Clock stretched is multiple of 2.5% vs
* not and [Fmin, Fmax, LDO_REFSEL, USE_FOR_LOW_FREQ]
*/
-uint16_t fiji_clock_stretcher_lookup_table[2][4] = { {600, 1050, 3, 0},
- {600, 1050, 6, 1} };
+static const uint16_t fiji_clock_stretcher_lookup_table[2][4] =
+{ {600, 1050, 3, 0}, {600, 1050, 6, 1} };
/* [FF, SS] type, [] 4 voltage ranges, and
* [Floor Freq, Boundary Freq, VID min , VID max]
*/
-uint32_t fiji_clock_stretcher_ddt_table[2][4][4] =
+static const uint32_t fiji_clock_stretcher_ddt_table[2][4][4] =
{ { {265, 529, 120, 128}, {325, 650, 96, 119}, {430, 860, 32, 95}, {0, 0, 0, 31} },
{ {275, 550, 104, 112}, {319, 638, 96, 103}, {360, 720, 64, 95}, {384, 768, 32, 63} } };
/* [Use_For_Low_freq] value, [0%, 5%, 10%, 7.14%, 14.28%, 20%]
* (coming from PWR_CKS_CNTL.stretch_amount reg spec)
*/
-uint8_t fiji_clock_stretch_amount_conversion[2][6] = { {0, 1, 3, 2, 4, 5},
- {0, 2, 4, 5, 6, 5} };
+static const uint8_t fiji_clock_stretch_amount_conversion[2][6] =
+{ {0, 1, 3, 2, 4, 5}, {0, 2, 4, 5, 6, 5} };
-const unsigned long PhwFiji_Magic = (unsigned long)(PHM_VIslands_Magic);
+static const unsigned long PhwFiji_Magic = (unsigned long)(PHM_VIslands_Magic);
struct fiji_power_state *cast_phw_fiji_power_state(
struct pp_hw_power_state *hw_ps)
@@ -465,14 +465,14 @@
table_info->vdd_dep_on_mclk;
PP_ASSERT_WITH_CODE(allowed_sclk_vdd_table != NULL,
- "VDD dependency on SCLK table is missing. \
+ "VDD dependency on SCLK table is missing. \
This table is mandatory", return -EINVAL);
PP_ASSERT_WITH_CODE(allowed_sclk_vdd_table->count >= 1,
- "VDD dependency on SCLK table has to have is missing. \
+ "VDD dependency on SCLK table has to have is missing. \
This table is mandatory", return -EINVAL);
PP_ASSERT_WITH_CODE(allowed_mclk_vdd_table != NULL,
- "VDD dependency on MCLK table is missing. \
+ "VDD dependency on MCLK table is missing. \
This table is mandatory", return -EINVAL);
PP_ASSERT_WITH_CODE(allowed_mclk_vdd_table->count >= 1,
"VDD dependency on MCLK table has to have is missing. \
@@ -579,6 +579,18 @@
return 0;
}
+static int fiji_hwmgr_backend_fini(struct pp_hwmgr *hwmgr)
+{
+ struct fiji_hwmgr *data = (struct fiji_hwmgr *)(hwmgr->backend);
+
+ if (data->soft_pp_table) {
+ kfree(data->soft_pp_table);
+ data->soft_pp_table = NULL;
+ }
+
+ return phm_hwmgr_backend_fini(hwmgr);
+}
+
static int fiji_hwmgr_backend_init(struct pp_hwmgr *hwmgr)
{
struct fiji_hwmgr *data = (struct fiji_hwmgr *)(hwmgr->backend);
@@ -734,7 +746,7 @@
data->pcie_lane_cap = (uint32_t)sys_info.value;
} else {
/* Ignore return value in here, we are cleaning up a mess. */
- tonga_hwmgr_backend_fini(hwmgr);
+ fiji_hwmgr_backend_fini(hwmgr);
}
return 0;
@@ -1885,6 +1897,23 @@
return 0;
}
+
+static uint8_t fiji_get_sleep_divider_id_from_clock(uint32_t clock,
+ uint32_t clock_insr)
+{
+ uint8_t i;
+ uint32_t temp;
+ uint32_t min = max(clock_insr, (uint32_t)FIJI_MINIMUM_ENGINE_CLOCK);
+
+ PP_ASSERT_WITH_CODE((clock >= min), "Engine clock can't satisfy stutter requirement!", return 0);
+ for (i = FIJI_MAX_DEEPSLEEP_DIVIDER_ID; ; i--) {
+ temp = clock >> i;
+
+ if (temp >= min || i == 0)
+ break;
+ }
+ return i;
+}
/**
* Populates single SMC SCLK structure using the provided engine clock
*
@@ -1928,17 +1957,13 @@
threshold = clock * data->fast_watermark_threshold / 100;
- /*
- * TODO: get minimum clocks from dal configaration
- * PECI_GetMinClockSettings(hwmgr->pPECI, &minClocks);
- */
- /* data->DisplayTiming.minClockInSR = minClocks.engineClockInSR; */
- /* get level->DeepSleepDivId
- if (phm_cap_enabled(hwmgr->platformDescriptor.platformCaps, PHM_PlatformCaps_SclkDeepSleep))
- {
- level->DeepSleepDivId = PhwFiji_GetSleepDividerIdFromClock(hwmgr, clock, minClocks.engineClockInSR);
- } */
+ data->display_timing.min_clock_in_sr = hwmgr->display_config.min_core_set_clock_in_sr;
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps, PHM_PlatformCaps_SclkDeepSleep))
+ level->DeepSleepDivId = fiji_get_sleep_divider_id_from_clock(clock,
+ hwmgr->display_config.min_core_set_clock_in_sr);
+
/* Default to slow, highest DPM level will be
* set to PPSMC_DISPLAY_WATERMARK_LOW later.
@@ -3364,7 +3389,7 @@
DPM_EVENT_SRC, src);
PHM_WRITE_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC, GENERAL_PWRMGT,
THERMAL_PROTECTION_DIS,
- phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ !phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
PHM_PlatformCaps_ThermalController));
} else
PHM_WRITE_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC, GENERAL_PWRMGT,
@@ -4066,7 +4091,6 @@
struct fiji_single_dpm_table *mclk_table = &(data->dpm_table.mclk_table);
uint32_t mclk = fiji_ps->performance_levels
[fiji_ps->performance_level_count - 1].memory_clock;
- struct PP_Clocks min_clocks = {0};
uint32_t i;
struct cgs_display_info info = {0};
@@ -4080,10 +4104,8 @@
if (i >= sclk_table->count)
data->need_update_smu7_dpm_table |= DPMTABLE_OD_UPDATE_SCLK;
else {
- /* TODO: Check SCLK in DAL's minimum clocks
- * in case DeepSleep divider update is required.
- */
- if(data->display_timing.min_clock_in_sr != min_clocks.engineClockInSR)
+ if(data->display_timing.min_clock_in_sr !=
+ hwmgr->display_config.min_core_set_clock_in_sr)
data->need_update_smu7_dpm_table |= DPMTABLE_UPDATE_SCLK;
}
@@ -5086,24 +5108,40 @@
{
struct fiji_hwmgr *data = (struct fiji_hwmgr *)(hwmgr->backend);
- *table = (char *)&data->smc_state_table;
+ if (!data->soft_pp_table) {
+ data->soft_pp_table = kzalloc(hwmgr->soft_pp_table_size, GFP_KERNEL);
+ if (!data->soft_pp_table)
+ return -ENOMEM;
+ memcpy(data->soft_pp_table, hwmgr->soft_pp_table,
+ hwmgr->soft_pp_table_size);
+ }
- return sizeof(struct SMU73_Discrete_DpmTable);
+ *table = (char *)&data->soft_pp_table;
+
+ return hwmgr->soft_pp_table_size;
}
static int fiji_set_pp_table(struct pp_hwmgr *hwmgr, const char *buf, size_t size)
{
struct fiji_hwmgr *data = (struct fiji_hwmgr *)(hwmgr->backend);
- void *table = (void *)&data->smc_state_table;
+ if (!data->soft_pp_table) {
+ data->soft_pp_table = kzalloc(hwmgr->soft_pp_table_size, GFP_KERNEL);
+ if (!data->soft_pp_table)
+ return -ENOMEM;
+ }
- memcpy(table, buf, size);
+ memcpy(data->soft_pp_table, buf, size);
+
+ hwmgr->soft_pp_table = data->soft_pp_table;
+
+ /* TODO: re-init powerplay to implement modified pptable */
return 0;
}
static int fiji_force_clock_level(struct pp_hwmgr *hwmgr,
- enum pp_clock_type type, int level)
+ enum pp_clock_type type, uint32_t mask)
{
struct fiji_hwmgr *data = (struct fiji_hwmgr *)(hwmgr->backend);
@@ -5115,20 +5153,30 @@
if (!data->sclk_dpm_key_disabled)
smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
PPSMC_MSG_SCLKDPM_SetEnabledMask,
- (1 << level));
+ data->dpm_level_enable_mask.sclk_dpm_enable_mask & mask);
break;
+
case PP_MCLK:
if (!data->mclk_dpm_key_disabled)
smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
PPSMC_MSG_MCLKDPM_SetEnabledMask,
- (1 << level));
+ data->dpm_level_enable_mask.mclk_dpm_enable_mask & mask);
break;
+
case PP_PCIE:
+ {
+ uint32_t tmp = mask & data->dpm_level_enable_mask.pcie_dpm_enable_mask;
+ uint32_t level = 0;
+
+ while (tmp >>= 1)
+ level++;
+
if (!data->pcie_dpm_key_disabled)
smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
PPSMC_MSG_PCIeDPM_ForceLevel,
- (1 << level));
+ level);
break;
+ }
default:
break;
}
@@ -5252,19 +5300,19 @@
if (data->display_timing.num_existing_displays != info.display_count)
is_update_required = true;
-/* TO DO NEED TO GET DEEP SLEEP CLOCK FROM DAL
- if (phm_cap_enabled(hwmgr->hwmgr->platform_descriptor.platformCaps, PHM_PlatformCaps_SclkDeepSleep)) {
- cgs_get_min_clock_settings(hwmgr->device, &min_clocks);
- if(min_clocks.engineClockInSR != data->display_timing.minClockInSR)
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps, PHM_PlatformCaps_SclkDeepSleep)) {
+ if(hwmgr->display_config.min_core_set_clock_in_sr != data->display_timing.min_clock_in_sr)
is_update_required = true;
-*/
+ }
+
return is_update_required;
}
static const struct pp_hwmgr_func fiji_hwmgr_funcs = {
.backend_init = &fiji_hwmgr_backend_init,
- .backend_fini = &tonga_hwmgr_backend_fini,
+ .backend_fini = &fiji_hwmgr_backend_fini,
.asic_setup = &fiji_setup_asic_task,
.dynamic_state_management_enable = &fiji_enable_dpm_tasks,
.force_dpm_level = &fiji_dpm_force_dpm_level,
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/fiji_hwmgr.h b/drivers/gpu/drm/amd/powerplay/hwmgr/fiji_hwmgr.h
index a16f7cd..170edf5 100644
--- a/drivers/gpu/drm/amd/powerplay/hwmgr/fiji_hwmgr.h
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/fiji_hwmgr.h
@@ -263,7 +263,7 @@
bool enable_tdc_limit_feature;
bool enable_pkg_pwr_tracking_feature;
bool disable_uvd_power_tune_feature;
- struct fiji_pt_defaults *power_tune_defaults;
+ const struct fiji_pt_defaults *power_tune_defaults;
struct SMU73_Discrete_PmFuses power_tune_table;
uint32_t dte_tj_offset;
uint32_t fast_watermark_threshold;
@@ -302,6 +302,9 @@
bool pg_acp_init;
bool frtc_enabled;
bool frtc_status_changed;
+
+ /* soft pptable for re-uploading into smu */
+ void *soft_pp_table;
};
/* To convert to Q8.8 format for firmware */
@@ -338,7 +341,6 @@
#define FIJI_UNUSED_GPIO_PIN 0x7F
extern int tonga_initializa_dynamic_state_adjustment_rule_settings(struct pp_hwmgr *hwmgr);
-extern int tonga_hwmgr_backend_fini(struct pp_hwmgr *hwmgr);
extern int tonga_get_mc_microcode_version (struct pp_hwmgr *hwmgr);
extern int tonga_notify_smc_display_config_after_ps_adjustment(struct pp_hwmgr *hwmgr);
extern int tonga_notify_smc_display_change(struct pp_hwmgr *hwmgr, bool has_display);
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/fiji_powertune.c b/drivers/gpu/drm/amd/powerplay/hwmgr/fiji_powertune.c
index 6efcb2b..db23a40 100644
--- a/drivers/gpu/drm/amd/powerplay/hwmgr/fiji_powertune.c
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/fiji_powertune.c
@@ -32,7 +32,7 @@
#define VOLTAGE_SCALE 4
#define POWERTUNE_DEFAULT_SET_MAX 1
-struct fiji_pt_defaults fiji_power_tune_data_set_array[POWERTUNE_DEFAULT_SET_MAX] = {
+const struct fiji_pt_defaults fiji_power_tune_data_set_array[POWERTUNE_DEFAULT_SET_MAX] = {
/*sviLoadLIneEn, SviLoadLineVddC, TDC_VDDC_ThrottleReleaseLimitPerc */
{1, 0xF, 0xFD,
/* TDC_MAWt, TdcWaterfallCtl, DTEAmbientTempBase */
@@ -143,7 +143,7 @@
int fiji_populate_bapm_parameters_in_dpm_table(struct pp_hwmgr *hwmgr)
{
struct fiji_hwmgr *data = (struct fiji_hwmgr *)(hwmgr->backend);
- struct fiji_pt_defaults *defaults = data->power_tune_defaults;
+ const struct fiji_pt_defaults *defaults = data->power_tune_defaults;
SMU73_Discrete_DpmTable *dpm_table = &(data->smc_state_table);
struct phm_ppt_v1_information *table_info =
(struct phm_ppt_v1_information *)(hwmgr->pptable);
@@ -222,7 +222,7 @@
static int fiji_populate_svi_load_line(struct pp_hwmgr *hwmgr)
{
struct fiji_hwmgr *data = (struct fiji_hwmgr *)(hwmgr->backend);
- struct fiji_pt_defaults *defaults = data->power_tune_defaults;
+ const struct fiji_pt_defaults *defaults = data->power_tune_defaults;
data->power_tune_table.SviLoadLineEn = defaults->SviLoadLineEn;
data->power_tune_table.SviLoadLineVddC = defaults->SviLoadLineVddC;
@@ -238,7 +238,7 @@
struct fiji_hwmgr *data = (struct fiji_hwmgr *)(hwmgr->backend);
struct phm_ppt_v1_information *table_info =
(struct phm_ppt_v1_information *)(hwmgr->pptable);
- struct fiji_pt_defaults *defaults = data->power_tune_defaults;
+ const struct fiji_pt_defaults *defaults = data->power_tune_defaults;
/* TDC number of fraction bits are changed from 8 to 7
* for Fiji as requested by SMC team
@@ -256,7 +256,7 @@
static int fiji_populate_dw8(struct pp_hwmgr *hwmgr, uint32_t fuse_table_offset)
{
struct fiji_hwmgr *data = (struct fiji_hwmgr *)(hwmgr->backend);
- struct fiji_pt_defaults *defaults = data->power_tune_defaults;
+ const struct fiji_pt_defaults *defaults = data->power_tune_defaults;
uint32_t temp;
if (fiji_read_smc_sram_dword(hwmgr->smumgr,
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/fiji_thermal.c b/drivers/gpu/drm/amd/powerplay/hwmgr/fiji_thermal.c
index e76a7de..92976b6 100644
--- a/drivers/gpu/drm/amd/powerplay/hwmgr/fiji_thermal.c
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/fiji_thermal.c
@@ -221,8 +221,8 @@
if (duty100 == 0)
return -EINVAL;
- tmp64 = (uint64_t)speed * 100;
- do_div(tmp64, duty100);
+ tmp64 = (uint64_t)speed * duty100;
+ do_div(tmp64, 100);
duty = (uint32_t)tmp64;
PHM_WRITE_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
@@ -615,7 +615,7 @@
return fiji_thermal_disable_alert(hwmgr);
}
-static struct phm_master_table_item
+static const struct phm_master_table_item
fiji_thermal_start_thermal_controller_master_list[] = {
{NULL, tf_fiji_thermal_initialize},
{NULL, tf_fiji_thermal_set_temperature_range},
@@ -630,14 +630,14 @@
{NULL, NULL}
};
-static struct phm_master_table_header
+static const struct phm_master_table_header
fiji_thermal_start_thermal_controller_master = {
0,
PHM_MasterTableFlag_None,
fiji_thermal_start_thermal_controller_master_list
};
-static struct phm_master_table_item
+static const struct phm_master_table_item
fiji_thermal_set_temperature_range_master_list[] = {
{NULL, tf_fiji_thermal_disable_alert},
{NULL, tf_fiji_thermal_set_temperature_range},
@@ -645,7 +645,7 @@
{NULL, NULL}
};
-struct phm_master_table_header
+static const struct phm_master_table_header
fiji_thermal_set_temperature_range_master = {
0,
PHM_MasterTableFlag_None,
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/functiontables.c b/drivers/gpu/drm/amd/powerplay/hwmgr/functiontables.c
index 72cfecc..7a705ce 100644
--- a/drivers/gpu/drm/amd/powerplay/hwmgr/functiontables.c
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/functiontables.c
@@ -84,7 +84,7 @@
}
int phm_construct_table(struct pp_hwmgr *hwmgr,
- struct phm_master_table_header *master_table,
+ const struct phm_master_table_header *master_table,
struct phm_runtime_table_header *rt_table)
{
uint32_t function_count = 0;
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/hwmgr.c b/drivers/gpu/drm/amd/powerplay/hwmgr/hwmgr.c
index 5fb98aa..7d69ed6 100644
--- a/drivers/gpu/drm/amd/powerplay/hwmgr/hwmgr.c
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/hwmgr.c
@@ -34,6 +34,7 @@
extern int cz_hwmgr_init(struct pp_hwmgr *hwmgr);
extern int tonga_hwmgr_init(struct pp_hwmgr *hwmgr);
extern int fiji_hwmgr_init(struct pp_hwmgr *hwmgr);
+extern int polaris10_hwmgr_init(struct pp_hwmgr *hwmgr);
int hwmgr_init(struct amd_pp_init *pp_init, struct pp_instance *handle)
{
@@ -67,6 +68,10 @@
case CHIP_FIJI:
fiji_hwmgr_init(hwmgr);
break;
+ case CHIP_POLARIS11:
+ case CHIP_POLARIS10:
+ polaris10_hwmgr_init(hwmgr);
+ break;
default:
return -EINVAL;
}
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/hwmgr_ppt.h b/drivers/gpu/drm/amd/powerplay/hwmgr/hwmgr_ppt.h
index c9e6c2d..347fef1 100644
--- a/drivers/gpu/drm/amd/powerplay/hwmgr/hwmgr_ppt.h
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/hwmgr_ppt.h
@@ -92,6 +92,8 @@
struct phm_ppt_v1_pcie_record {
uint8_t gen_speed;
uint8_t lane_width;
+ uint16_t usreserved;
+ uint32_t pcie_sclk;
};
typedef struct phm_ppt_v1_pcie_record phm_ppt_v1_pcie_record;
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_clockpowergating.c b/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_clockpowergating.c
new file mode 100644
index 0000000..8f142a7
--- /dev/null
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_clockpowergating.c
@@ -0,0 +1,430 @@
+/*
+ * Copyright 2016 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#include "polaris10_clockpowergating.h"
+
+int polaris10_phm_powerdown_uvd(struct pp_hwmgr *hwmgr)
+{
+ if (phm_cf_want_uvd_power_gating(hwmgr))
+ return smum_send_msg_to_smc(hwmgr->smumgr,
+ PPSMC_MSG_UVDPowerOFF);
+ return 0;
+}
+
+int polaris10_phm_powerup_uvd(struct pp_hwmgr *hwmgr)
+{
+ if (phm_cf_want_uvd_power_gating(hwmgr)) {
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_UVDDynamicPowerGating)) {
+ return smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_UVDPowerON, 1);
+ } else {
+ return smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_UVDPowerON, 0);
+ }
+ }
+
+ return 0;
+}
+
+int polaris10_phm_powerdown_vce(struct pp_hwmgr *hwmgr)
+{
+ if (phm_cf_want_vce_power_gating(hwmgr))
+ return smum_send_msg_to_smc(hwmgr->smumgr,
+ PPSMC_MSG_VCEPowerOFF);
+ return 0;
+}
+
+int polaris10_phm_powerup_vce(struct pp_hwmgr *hwmgr)
+{
+ if (phm_cf_want_vce_power_gating(hwmgr))
+ return smum_send_msg_to_smc(hwmgr->smumgr,
+ PPSMC_MSG_VCEPowerON);
+ return 0;
+}
+
+int polaris10_phm_powerdown_samu(struct pp_hwmgr *hwmgr)
+{
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_SamuPowerGating))
+ return smum_send_msg_to_smc(hwmgr->smumgr,
+ PPSMC_MSG_SAMPowerOFF);
+ return 0;
+}
+
+int polaris10_phm_powerup_samu(struct pp_hwmgr *hwmgr)
+{
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_SamuPowerGating))
+ return smum_send_msg_to_smc(hwmgr->smumgr,
+ PPSMC_MSG_SAMPowerON);
+ return 0;
+}
+
+int polaris10_phm_disable_clock_power_gating(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ data->uvd_power_gated = false;
+ data->vce_power_gated = false;
+ data->samu_power_gated = false;
+
+ polaris10_phm_powerup_uvd(hwmgr);
+ polaris10_phm_powerup_vce(hwmgr);
+ polaris10_phm_powerup_samu(hwmgr);
+
+ return 0;
+}
+
+int polaris10_phm_powergate_uvd(struct pp_hwmgr *hwmgr, bool bgate)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ if (data->uvd_power_gated == bgate)
+ return 0;
+
+ data->uvd_power_gated = bgate;
+
+ if (bgate) {
+ polaris10_update_uvd_dpm(hwmgr, true);
+ polaris10_phm_powerdown_uvd(hwmgr);
+ } else {
+ polaris10_phm_powerup_uvd(hwmgr);
+ polaris10_update_uvd_dpm(hwmgr, false);
+ }
+
+ return 0;
+}
+
+int polaris10_phm_powergate_vce(struct pp_hwmgr *hwmgr, bool bgate)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ if (data->vce_power_gated == bgate)
+ return 0;
+
+ data->vce_power_gated = bgate;
+
+ if (bgate)
+ polaris10_phm_powerdown_vce(hwmgr);
+ else
+ polaris10_phm_powerup_vce(hwmgr);
+
+ return 0;
+}
+
+int polaris10_phm_powergate_samu(struct pp_hwmgr *hwmgr, bool bgate)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ if (data->samu_power_gated == bgate)
+ return 0;
+
+ data->samu_power_gated = bgate;
+
+ if (bgate) {
+ polaris10_update_samu_dpm(hwmgr, true);
+ polaris10_phm_powerdown_samu(hwmgr);
+ } else {
+ polaris10_phm_powerup_samu(hwmgr);
+ polaris10_update_samu_dpm(hwmgr, false);
+ }
+
+ return 0;
+}
+
+int polaris10_phm_update_clock_gatings(struct pp_hwmgr *hwmgr,
+ const uint32_t *msg_id)
+{
+ PPSMC_Msg msg;
+ uint32_t value;
+
+ switch ((*msg_id & PP_GROUP_MASK) >> PP_GROUP_SHIFT) {
+ case PP_GROUP_GFX:
+ switch ((*msg_id & PP_BLOCK_MASK) >> PP_BLOCK_SHIFT) {
+ case PP_BLOCK_GFX_CG:
+ if (PP_STATE_SUPPORT_CG & *msg_id) {
+ msg = ((*msg_id & PP_STATE_MASK) & PP_STATE_CG) ?
+ PPSMC_MSG_EnableClockGatingFeature :
+ PPSMC_MSG_DisableClockGatingFeature;
+ value = CG_GFX_CGCG_MASK;
+
+ if (smum_send_msg_to_smc_with_parameter(
+ hwmgr->smumgr, msg, value))
+ return -1;
+ }
+ if (PP_STATE_SUPPORT_LS & *msg_id) {
+ msg = (*msg_id & PP_STATE_MASK) & PP_STATE_LS
+ ? PPSMC_MSG_EnableClockGatingFeature
+ : PPSMC_MSG_DisableClockGatingFeature;
+ value = CG_GFX_CGLS_MASK;
+
+ if (smum_send_msg_to_smc_with_parameter(
+ hwmgr->smumgr, msg, value))
+ return -1;
+ }
+ break;
+
+ case PP_BLOCK_GFX_3D:
+ if (PP_STATE_SUPPORT_CG & *msg_id) {
+ msg = ((*msg_id & PP_STATE_MASK) & PP_STATE_CG) ?
+ PPSMC_MSG_EnableClockGatingFeature :
+ PPSMC_MSG_DisableClockGatingFeature;
+ value = CG_GFX_3DCG_MASK;
+
+ if (smum_send_msg_to_smc_with_parameter(
+ hwmgr->smumgr, msg, value))
+ return -1;
+ }
+
+ if (PP_STATE_SUPPORT_LS & *msg_id) {
+ msg = (*msg_id & PP_STATE_MASK) & PP_STATE_LS ?
+ PPSMC_MSG_EnableClockGatingFeature :
+ PPSMC_MSG_DisableClockGatingFeature;
+ value = CG_GFX_3DLS_MASK;
+
+ if (smum_send_msg_to_smc_with_parameter(
+ hwmgr->smumgr, msg, value))
+ return -1;
+ }
+ break;
+
+ case PP_BLOCK_GFX_RLC:
+ if (PP_STATE_SUPPORT_LS & *msg_id) {
+ msg = (*msg_id & PP_STATE_MASK) & PP_STATE_LS ?
+ PPSMC_MSG_EnableClockGatingFeature :
+ PPSMC_MSG_DisableClockGatingFeature;
+ value = CG_GFX_RLC_LS_MASK;
+
+ if (smum_send_msg_to_smc_with_parameter(
+ hwmgr->smumgr, msg, value))
+ return -1;
+ }
+ break;
+
+ case PP_BLOCK_GFX_CP:
+ if (PP_STATE_SUPPORT_LS & *msg_id) {
+ msg = (*msg_id & PP_STATE_MASK) & PP_STATE_LS ?
+ PPSMC_MSG_EnableClockGatingFeature :
+ PPSMC_MSG_DisableClockGatingFeature;
+ value = CG_GFX_CP_LS_MASK;
+
+ if (smum_send_msg_to_smc_with_parameter(
+ hwmgr->smumgr, msg, value))
+ return -1;
+ }
+ break;
+
+ case PP_BLOCK_GFX_MG:
+ if (PP_STATE_SUPPORT_CG & *msg_id) {
+ msg = ((*msg_id & PP_STATE_MASK) & PP_STATE_CG) ?
+ PPSMC_MSG_EnableClockGatingFeature :
+ PPSMC_MSG_DisableClockGatingFeature;
+ value = (CG_CPF_MGCG_MASK | CG_RLC_MGCG_MASK |
+ CG_GFX_OTHERS_MGCG_MASK);
+
+ if (smum_send_msg_to_smc_with_parameter(
+ hwmgr->smumgr, msg, value))
+ return -1;
+ }
+ break;
+
+ default:
+ return -1;
+ }
+ break;
+
+ case PP_GROUP_SYS:
+ switch ((*msg_id & PP_BLOCK_MASK) >> PP_BLOCK_SHIFT) {
+ case PP_BLOCK_SYS_BIF:
+ if (PP_STATE_SUPPORT_CG & *msg_id) {
+ msg = (*msg_id & PP_STATE_MASK) & PP_STATE_CG ?
+ PPSMC_MSG_EnableClockGatingFeature :
+ PPSMC_MSG_DisableClockGatingFeature;
+ value = CG_SYS_BIF_MGCG_MASK;
+
+ if (smum_send_msg_to_smc_with_parameter(
+ hwmgr->smumgr, msg, value))
+ return -1;
+ }
+ if (PP_STATE_SUPPORT_LS & *msg_id) {
+ msg = (*msg_id & PP_STATE_MASK) & PP_STATE_LS ?
+ PPSMC_MSG_EnableClockGatingFeature :
+ PPSMC_MSG_DisableClockGatingFeature;
+ value = CG_SYS_BIF_MGLS_MASK;
+
+ if (smum_send_msg_to_smc_with_parameter(
+ hwmgr->smumgr, msg, value))
+ return -1;
+ }
+ break;
+
+ case PP_BLOCK_SYS_MC:
+ if (PP_STATE_SUPPORT_CG & *msg_id) {
+ msg = ((*msg_id & PP_STATE_MASK) & PP_STATE_CG) ?
+ PPSMC_MSG_EnableClockGatingFeature :
+ PPSMC_MSG_DisableClockGatingFeature;
+ value = CG_SYS_MC_MGCG_MASK;
+
+ if (smum_send_msg_to_smc_with_parameter(
+ hwmgr->smumgr, msg, value))
+ return -1;
+ }
+
+ if (PP_STATE_SUPPORT_LS & *msg_id) {
+ msg = (*msg_id & PP_STATE_MASK) & PP_STATE_LS ?
+ PPSMC_MSG_EnableClockGatingFeature :
+ PPSMC_MSG_DisableClockGatingFeature;
+ value = CG_SYS_MC_MGLS_MASK;
+
+ if (smum_send_msg_to_smc_with_parameter(
+ hwmgr->smumgr, msg, value))
+ return -1;
+ }
+ break;
+
+ case PP_BLOCK_SYS_DRM:
+ if (PP_STATE_SUPPORT_CG & *msg_id) {
+ msg = (*msg_id & PP_STATE_MASK) & PP_STATE_CG ?
+ PPSMC_MSG_EnableClockGatingFeature :
+ PPSMC_MSG_DisableClockGatingFeature;
+ value = CG_SYS_DRM_MGCG_MASK;
+
+ if (smum_send_msg_to_smc_with_parameter(
+ hwmgr->smumgr, msg, value))
+ return -1;
+ }
+ if (PP_STATE_SUPPORT_LS & *msg_id) {
+ msg = (*msg_id & PP_STATE_MASK) & PP_STATE_LS ?
+ PPSMC_MSG_EnableClockGatingFeature :
+ PPSMC_MSG_DisableClockGatingFeature;
+ value = CG_SYS_DRM_MGLS_MASK;
+
+ if (smum_send_msg_to_smc_with_parameter(
+ hwmgr->smumgr, msg, value))
+ return -1;
+ }
+ break;
+
+ case PP_BLOCK_SYS_HDP:
+ if (PP_STATE_SUPPORT_CG & *msg_id) {
+ msg = ((*msg_id & PP_STATE_MASK) & PP_STATE_CG) ?
+ PPSMC_MSG_EnableClockGatingFeature :
+ PPSMC_MSG_DisableClockGatingFeature;
+ value = CG_SYS_HDP_MGCG_MASK;
+
+ if (smum_send_msg_to_smc_with_parameter(
+ hwmgr->smumgr, msg, value))
+ return -1;
+ }
+
+ if (PP_STATE_SUPPORT_LS & *msg_id) {
+ msg = (*msg_id & PP_STATE_MASK) & PP_STATE_LS ?
+ PPSMC_MSG_EnableClockGatingFeature :
+ PPSMC_MSG_DisableClockGatingFeature;
+ value = CG_SYS_HDP_MGLS_MASK;
+
+ if (smum_send_msg_to_smc_with_parameter(
+ hwmgr->smumgr, msg, value))
+ return -1;
+ }
+ break;
+
+ case PP_BLOCK_SYS_SDMA:
+ if (PP_STATE_SUPPORT_CG & *msg_id) {
+ msg = ((*msg_id & PP_STATE_MASK) & PP_STATE_CG) ?
+ PPSMC_MSG_EnableClockGatingFeature :
+ PPSMC_MSG_DisableClockGatingFeature;
+ value = CG_SYS_SDMA_MGCG_MASK;
+
+ if (smum_send_msg_to_smc_with_parameter(
+ hwmgr->smumgr, msg, value))
+ return -1;
+ }
+
+ if (PP_STATE_SUPPORT_LS & *msg_id) {
+ msg = (*msg_id & PP_STATE_MASK) & PP_STATE_LS ?
+ PPSMC_MSG_EnableClockGatingFeature :
+ PPSMC_MSG_DisableClockGatingFeature;
+ value = CG_SYS_SDMA_MGLS_MASK;
+
+ if (smum_send_msg_to_smc_with_parameter(
+ hwmgr->smumgr, msg, value))
+ return -1;
+ }
+ break;
+
+ case PP_BLOCK_SYS_ROM:
+ if (PP_STATE_SUPPORT_CG & *msg_id) {
+ msg = ((*msg_id & PP_STATE_MASK) & PP_STATE_CG) ?
+ PPSMC_MSG_EnableClockGatingFeature :
+ PPSMC_MSG_DisableClockGatingFeature;
+ value = CG_SYS_ROM_MASK;
+
+ if (smum_send_msg_to_smc_with_parameter(
+ hwmgr->smumgr, msg, value))
+ return -1;
+ }
+ break;
+
+ default:
+ return -1;
+
+ }
+ break;
+
+ default:
+ return -1;
+
+ }
+
+ return 0;
+}
+
+/* This function is for Polaris11 only for now,
+ * Powerplay will only control the static per CU Power Gating.
+ * Dynamic per CU Power Gating will be done in gfx.
+ */
+int polaris10_phm_enable_per_cu_power_gating(struct pp_hwmgr *hwmgr, bool enable)
+{
+ struct cgs_system_info sys_info = {0};
+ uint32_t active_cus;
+ int result;
+
+ sys_info.size = sizeof(struct cgs_system_info);
+ sys_info.info_id = CGS_SYSTEM_INFO_GFX_CU_INFO;
+
+ result = cgs_query_system_info(hwmgr->device, &sys_info);
+
+ if (result)
+ return -EINVAL;
+ else
+ active_cus = sys_info.value;
+
+ if (enable)
+ return smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_GFX_CU_PG_ENABLE, active_cus);
+ else
+ return smum_send_msg_to_smc(hwmgr->smumgr,
+ PPSMC_MSG_GFX_CU_PG_DISABLE);
+}
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_clockpowergating.h b/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_clockpowergating.h
new file mode 100644
index 0000000..88d68cb
--- /dev/null
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_clockpowergating.h
@@ -0,0 +1,40 @@
+/*
+ * Copyright 2016 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#ifndef _POLARIS10_CLOCK_POWER_GATING_H_
+#define _POLARIS10_CLOCK_POWER_GATING_H_
+
+#include "polaris10_hwmgr.h"
+#include "pp_asicblocks.h"
+
+int polaris10_phm_powergate_vce(struct pp_hwmgr *hwmgr, bool bgate);
+int polaris10_phm_powergate_uvd(struct pp_hwmgr *hwmgr, bool bgate);
+int polaris10_phm_powerdown_uvd(struct pp_hwmgr *hwmgr);
+int polaris10_phm_powergate_samu(struct pp_hwmgr *hwmgr, bool bgate);
+int polaris10_phm_powergate_acp(struct pp_hwmgr *hwmgr, bool bgate);
+int polaris10_phm_disable_clock_power_gating(struct pp_hwmgr *hwmgr);
+int polaris10_phm_update_clock_gatings(struct pp_hwmgr *hwmgr,
+ const uint32_t *msg_id);
+int polaris10_phm_enable_per_cu_power_gating(struct pp_hwmgr *hwmgr, bool enable);
+
+#endif /* _POLARIS10_CLOCK_POWER_GATING_H_ */
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_dyn_defaults.h b/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_dyn_defaults.h
new file mode 100644
index 0000000..f78ffd9
--- /dev/null
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_dyn_defaults.h
@@ -0,0 +1,62 @@
+/*
+ * Copyright 2015 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#ifndef POLARIS10_DYN_DEFAULTS_H
+#define POLARIS10_DYN_DEFAULTS_H
+
+
+enum Polaris10dpm_TrendDetection {
+ Polaris10Adpm_TrendDetection_AUTO,
+ Polaris10Adpm_TrendDetection_UP,
+ Polaris10Adpm_TrendDetection_DOWN
+};
+typedef enum Polaris10dpm_TrendDetection Polaris10dpm_TrendDetection;
+
+/* We need to fill in the default values */
+
+
+#define PPPOLARIS10_VOTINGRIGHTSCLIENTS_DFLT0 0x3FFFC102
+#define PPPOLARIS10_VOTINGRIGHTSCLIENTS_DFLT1 0x000400
+#define PPPOLARIS10_VOTINGRIGHTSCLIENTS_DFLT2 0xC00080
+#define PPPOLARIS10_VOTINGRIGHTSCLIENTS_DFLT3 0xC00200
+#define PPPOLARIS10_VOTINGRIGHTSCLIENTS_DFLT4 0xC01680
+#define PPPOLARIS10_VOTINGRIGHTSCLIENTS_DFLT5 0xC00033
+#define PPPOLARIS10_VOTINGRIGHTSCLIENTS_DFLT6 0xC00033
+#define PPPOLARIS10_VOTINGRIGHTSCLIENTS_DFLT7 0x3FFFC000
+
+
+#define PPPOLARIS10_THERMALPROTECTCOUNTER_DFLT 0x200
+#define PPPOLARIS10_STATICSCREENTHRESHOLDUNIT_DFLT 0
+#define PPPOLARIS10_STATICSCREENTHRESHOLD_DFLT 0x00C8
+#define PPPOLARIS10_GFXIDLECLOCKSTOPTHRESHOLD_DFLT 0x200
+#define PPPOLARIS10_REFERENCEDIVIDER_DFLT 4
+
+#define PPPOLARIS10_ULVVOLTAGECHANGEDELAY_DFLT 1687
+
+#define PPPOLARIS10_CGULVPARAMETER_DFLT 0x00040035
+#define PPPOLARIS10_CGULVCONTROL_DFLT 0x00007450
+#define PPPOLARIS10_TARGETACTIVITY_DFLT 50
+#define PPPOLARIS10_MCLK_TARGETACTIVITY_DFLT 10
+
+#endif
+
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_hwmgr.c b/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_hwmgr.c
new file mode 100644
index 0000000..93768fa
--- /dev/null
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_hwmgr.c
@@ -0,0 +1,4995 @@
+/*
+ * Copyright 2015 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/fb.h>
+#include <asm/div64.h>
+#include "linux/delay.h"
+#include "pp_acpi.h"
+#include "hwmgr.h"
+#include "polaris10_hwmgr.h"
+#include "polaris10_powertune.h"
+#include "polaris10_dyn_defaults.h"
+#include "polaris10_smumgr.h"
+#include "pp_debug.h"
+#include "ppatomctrl.h"
+#include "atombios.h"
+#include "tonga_pptable.h"
+#include "pppcielanes.h"
+#include "amd_pcie_helpers.h"
+#include "hardwaremanager.h"
+#include "tonga_processpptables.h"
+#include "cgs_common.h"
+#include "smu74.h"
+#include "smu_ucode_xfer_vi.h"
+#include "smu74_discrete.h"
+#include "smu/smu_7_1_3_d.h"
+#include "smu/smu_7_1_3_sh_mask.h"
+#include "gmc/gmc_8_1_d.h"
+#include "gmc/gmc_8_1_sh_mask.h"
+#include "oss/oss_3_0_d.h"
+#include "gca/gfx_8_0_d.h"
+#include "bif/bif_5_0_d.h"
+#include "bif/bif_5_0_sh_mask.h"
+#include "gmc/gmc_8_1_d.h"
+#include "gmc/gmc_8_1_sh_mask.h"
+#include "bif/bif_5_0_d.h"
+#include "bif/bif_5_0_sh_mask.h"
+#include "dce/dce_10_0_d.h"
+#include "dce/dce_10_0_sh_mask.h"
+
+#include "polaris10_thermal.h"
+#include "polaris10_clockpowergating.h"
+
+#define MC_CG_ARB_FREQ_F0 0x0a
+#define MC_CG_ARB_FREQ_F1 0x0b
+#define MC_CG_ARB_FREQ_F2 0x0c
+#define MC_CG_ARB_FREQ_F3 0x0d
+
+#define MC_CG_SEQ_DRAMCONF_S0 0x05
+#define MC_CG_SEQ_DRAMCONF_S1 0x06
+#define MC_CG_SEQ_YCLK_SUSPEND 0x04
+#define MC_CG_SEQ_YCLK_RESUME 0x0a
+
+
+#define SMC_RAM_END 0x40000
+
+#define SMC_CG_IND_START 0xc0030000
+#define SMC_CG_IND_END 0xc0040000
+
+#define VOLTAGE_SCALE 4
+#define VOLTAGE_VID_OFFSET_SCALE1 625
+#define VOLTAGE_VID_OFFSET_SCALE2 100
+
+#define VDDC_VDDCI_DELTA 200
+
+#define MEM_FREQ_LOW_LATENCY 25000
+#define MEM_FREQ_HIGH_LATENCY 80000
+
+#define MEM_LATENCY_HIGH 45
+#define MEM_LATENCY_LOW 35
+#define MEM_LATENCY_ERR 0xFFFF
+
+#define MC_SEQ_MISC0_GDDR5_SHIFT 28
+#define MC_SEQ_MISC0_GDDR5_MASK 0xf0000000
+#define MC_SEQ_MISC0_GDDR5_VALUE 5
+
+
+#define PCIE_BUS_CLK 10000
+#define TCLK (PCIE_BUS_CLK / 10)
+
+
+static const uint16_t polaris10_clock_stretcher_lookup_table[2][4] =
+{ {600, 1050, 3, 0}, {600, 1050, 6, 1} };
+
+/* [FF, SS] type, [] 4 voltage ranges, and [Floor Freq, Boundary Freq, VID min , VID max] */
+static const uint32_t polaris10_clock_stretcher_ddt_table[2][4][4] =
+{ { {265, 529, 120, 128}, {325, 650, 96, 119}, {430, 860, 32, 95}, {0, 0, 0, 31} },
+ { {275, 550, 104, 112}, {319, 638, 96, 103}, {360, 720, 64, 95}, {384, 768, 32, 63} } };
+
+/* [Use_For_Low_freq] value, [0%, 5%, 10%, 7.14%, 14.28%, 20%] (coming from PWR_CKS_CNTL.stretch_amount reg spec) */
+static const uint8_t polaris10_clock_stretch_amount_conversion[2][6] =
+{ {0, 1, 3, 2, 4, 5}, {0, 2, 4, 5, 6, 5} };
+
+/** Values for the CG_THERMAL_CTRL::DPM_EVENT_SRC field. */
+enum DPM_EVENT_SRC {
+ DPM_EVENT_SRC_ANALOG = 0,
+ DPM_EVENT_SRC_EXTERNAL = 1,
+ DPM_EVENT_SRC_DIGITAL = 2,
+ DPM_EVENT_SRC_ANALOG_OR_EXTERNAL = 3,
+ DPM_EVENT_SRC_DIGITAL_OR_EXTERNAL = 4
+};
+
+static const unsigned long PhwPolaris10_Magic = (unsigned long)(PHM_VIslands_Magic);
+
+struct polaris10_power_state *cast_phw_polaris10_power_state(
+ struct pp_hw_power_state *hw_ps)
+{
+ PP_ASSERT_WITH_CODE((PhwPolaris10_Magic == hw_ps->magic),
+ "Invalid Powerstate Type!",
+ return NULL);
+
+ return (struct polaris10_power_state *)hw_ps;
+}
+
+const struct polaris10_power_state *cast_const_phw_polaris10_power_state(
+ const struct pp_hw_power_state *hw_ps)
+{
+ PP_ASSERT_WITH_CODE((PhwPolaris10_Magic == hw_ps->magic),
+ "Invalid Powerstate Type!",
+ return NULL);
+
+ return (const struct polaris10_power_state *)hw_ps;
+}
+
+static bool polaris10_is_dpm_running(struct pp_hwmgr *hwmgr)
+{
+ return (1 == PHM_READ_INDIRECT_FIELD(hwmgr->device,
+ CGS_IND_REG__SMC, FEATURE_STATUS, VOLTAGE_CONTROLLER_ON))
+ ? true : false;
+}
+
+/**
+ * Find the MC microcode version and store it in the HwMgr struct
+ *
+ * @param hwmgr the address of the powerplay hardware manager.
+ * @return always 0
+ */
+int phm_get_mc_microcode_version (struct pp_hwmgr *hwmgr)
+{
+ cgs_write_register(hwmgr->device, mmMC_SEQ_IO_DEBUG_INDEX, 0x9F);
+
+ hwmgr->microcode_version_info.MC = cgs_read_register(hwmgr->device, mmMC_SEQ_IO_DEBUG_DATA);
+
+ return 0;
+}
+
+uint16_t phm_get_current_pcie_speed(struct pp_hwmgr *hwmgr)
+{
+ uint32_t speedCntl = 0;
+
+ /* mmPCIE_PORT_INDEX rename as mmPCIE_INDEX */
+ speedCntl = cgs_read_ind_register(hwmgr->device, CGS_IND_REG__PCIE,
+ ixPCIE_LC_SPEED_CNTL);
+ return((uint16_t)PHM_GET_FIELD(speedCntl,
+ PCIE_LC_SPEED_CNTL, LC_CURRENT_DATA_RATE));
+}
+
+int phm_get_current_pcie_lane_number(struct pp_hwmgr *hwmgr)
+{
+ uint32_t link_width;
+
+ /* mmPCIE_PORT_INDEX rename as mmPCIE_INDEX */
+ link_width = PHM_READ_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__PCIE,
+ PCIE_LC_LINK_WIDTH_CNTL, LC_LINK_WIDTH_RD);
+
+ PP_ASSERT_WITH_CODE((7 >= link_width),
+ "Invalid PCIe lane width!", return 0);
+
+ return decode_pcie_lane_width(link_width);
+}
+
+void phm_apply_dal_min_voltage_request(struct pp_hwmgr *hwmgr)
+{
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)hwmgr->pptable;
+ struct phm_clock_voltage_dependency_table *table =
+ table_info->vddc_dep_on_dal_pwrl;
+ struct phm_ppt_v1_clock_voltage_dependency_table *vddc_table;
+ enum PP_DAL_POWERLEVEL dal_power_level = hwmgr->dal_power_level;
+ uint32_t req_vddc = 0, req_volt, i;
+
+ if (!table && !(dal_power_level >= PP_DAL_POWERLEVEL_ULTRALOW &&
+ dal_power_level <= PP_DAL_POWERLEVEL_PERFORMANCE))
+ return;
+
+ for (i = 0; i < table->count; i++) {
+ if (dal_power_level == table->entries[i].clk) {
+ req_vddc = table->entries[i].v;
+ break;
+ }
+ }
+
+ vddc_table = table_info->vdd_dep_on_sclk;
+ for (i = 0; i < vddc_table->count; i++) {
+ if (req_vddc <= vddc_table->entries[i].vddc) {
+ req_volt = (((uint32_t)vddc_table->entries[i].vddc) * VOLTAGE_SCALE)
+ << VDDC_SHIFT;
+ smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_VddC_Request, req_volt);
+ return;
+ }
+ }
+ printk(KERN_ERR "DAL requested level can not"
+ " found a available voltage in VDDC DPM Table \n");
+}
+
+/**
+* Enable voltage control
+*
+* @param pHwMgr the address of the powerplay hardware manager.
+* @return always PP_Result_OK
+*/
+int polaris10_enable_smc_voltage_controller(struct pp_hwmgr *hwmgr)
+{
+ PP_ASSERT_WITH_CODE(
+ (hwmgr->smumgr->smumgr_funcs->send_msg_to_smc(hwmgr->smumgr, PPSMC_MSG_Voltage_Cntl_Enable) == 0),
+ "Failed to enable voltage DPM during DPM Start Function!",
+ return 1;
+ );
+
+ return 0;
+}
+
+/**
+* Checks if we want to support voltage control
+*
+* @param hwmgr the address of the powerplay hardware manager.
+*/
+static bool polaris10_voltage_control(const struct pp_hwmgr *hwmgr)
+{
+ const struct polaris10_hwmgr *data =
+ (const struct polaris10_hwmgr *)(hwmgr->backend);
+
+ return (POLARIS10_VOLTAGE_CONTROL_NONE != data->voltage_control);
+}
+
+/**
+* Enable voltage control
+*
+* @param hwmgr the address of the powerplay hardware manager.
+* @return always 0
+*/
+static int polaris10_enable_voltage_control(struct pp_hwmgr *hwmgr)
+{
+ /* enable voltage control */
+ PHM_WRITE_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ GENERAL_PWRMGT, VOLT_PWRMGT_EN, 1);
+
+ return 0;
+}
+
+/**
+* Create Voltage Tables.
+*
+* @param hwmgr the address of the powerplay hardware manager.
+* @return always 0
+*/
+static int polaris10_construct_voltage_tables(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)hwmgr->pptable;
+ int result;
+
+ if (POLARIS10_VOLTAGE_CONTROL_BY_GPIO == data->mvdd_control) {
+ result = atomctrl_get_voltage_table_v3(hwmgr,
+ VOLTAGE_TYPE_MVDDC, VOLTAGE_OBJ_GPIO_LUT,
+ &(data->mvdd_voltage_table));
+ PP_ASSERT_WITH_CODE((0 == result),
+ "Failed to retrieve MVDD table.",
+ return result);
+ } else if (POLARIS10_VOLTAGE_CONTROL_BY_SVID2 == data->mvdd_control) {
+ result = phm_get_svi2_mvdd_voltage_table(&(data->mvdd_voltage_table),
+ table_info->vdd_dep_on_mclk);
+ PP_ASSERT_WITH_CODE((0 == result),
+ "Failed to retrieve SVI2 MVDD table from dependancy table.",
+ return result;);
+ }
+
+ if (POLARIS10_VOLTAGE_CONTROL_BY_GPIO == data->vddci_control) {
+ result = atomctrl_get_voltage_table_v3(hwmgr,
+ VOLTAGE_TYPE_VDDCI, VOLTAGE_OBJ_GPIO_LUT,
+ &(data->vddci_voltage_table));
+ PP_ASSERT_WITH_CODE((0 == result),
+ "Failed to retrieve VDDCI table.",
+ return result);
+ } else if (POLARIS10_VOLTAGE_CONTROL_BY_SVID2 == data->vddci_control) {
+ result = phm_get_svi2_vddci_voltage_table(&(data->vddci_voltage_table),
+ table_info->vdd_dep_on_mclk);
+ PP_ASSERT_WITH_CODE((0 == result),
+ "Failed to retrieve SVI2 VDDCI table from dependancy table.",
+ return result);
+ }
+
+ if (POLARIS10_VOLTAGE_CONTROL_BY_SVID2 == data->voltage_control) {
+ result = phm_get_svi2_vdd_voltage_table(&(data->vddc_voltage_table),
+ table_info->vddc_lookup_table);
+ PP_ASSERT_WITH_CODE((0 == result),
+ "Failed to retrieve SVI2 VDDC table from lookup table.",
+ return result);
+ }
+
+ PP_ASSERT_WITH_CODE(
+ (data->vddc_voltage_table.count <= (SMU74_MAX_LEVELS_VDDC)),
+ "Too many voltage values for VDDC. Trimming to fit state table.",
+ phm_trim_voltage_table_to_fit_state_table(SMU74_MAX_LEVELS_VDDC,
+ &(data->vddc_voltage_table)));
+
+ PP_ASSERT_WITH_CODE(
+ (data->vddci_voltage_table.count <= (SMU74_MAX_LEVELS_VDDCI)),
+ "Too many voltage values for VDDCI. Trimming to fit state table.",
+ phm_trim_voltage_table_to_fit_state_table(SMU74_MAX_LEVELS_VDDCI,
+ &(data->vddci_voltage_table)));
+
+ PP_ASSERT_WITH_CODE(
+ (data->mvdd_voltage_table.count <= (SMU74_MAX_LEVELS_MVDD)),
+ "Too many voltage values for MVDD. Trimming to fit state table.",
+ phm_trim_voltage_table_to_fit_state_table(SMU74_MAX_LEVELS_MVDD,
+ &(data->mvdd_voltage_table)));
+
+ return 0;
+}
+
+/**
+* Programs static screed detection parameters
+*
+* @param hwmgr the address of the powerplay hardware manager.
+* @return always 0
+*/
+static int polaris10_program_static_screen_threshold_parameters(
+ struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ /* Set static screen threshold unit */
+ PHM_WRITE_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_STATIC_SCREEN_PARAMETER, STATIC_SCREEN_THRESHOLD_UNIT,
+ data->static_screen_threshold_unit);
+ /* Set static screen threshold */
+ PHM_WRITE_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_STATIC_SCREEN_PARAMETER, STATIC_SCREEN_THRESHOLD,
+ data->static_screen_threshold);
+
+ return 0;
+}
+
+/**
+* Setup display gap for glitch free memory clock switching.
+*
+* @param hwmgr the address of the powerplay hardware manager.
+* @return always 0
+*/
+static int polaris10_enable_display_gap(struct pp_hwmgr *hwmgr)
+{
+ uint32_t display_gap =
+ cgs_read_ind_register(hwmgr->device, CGS_IND_REG__SMC,
+ ixCG_DISPLAY_GAP_CNTL);
+
+ display_gap = PHM_SET_FIELD(display_gap, CG_DISPLAY_GAP_CNTL,
+ DISP_GAP, DISPLAY_GAP_IGNORE);
+
+ display_gap = PHM_SET_FIELD(display_gap, CG_DISPLAY_GAP_CNTL,
+ DISP_GAP_MCHG, DISPLAY_GAP_VBLANK);
+
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC,
+ ixCG_DISPLAY_GAP_CNTL, display_gap);
+
+ return 0;
+}
+
+/**
+* Programs activity state transition voting clients
+*
+* @param hwmgr the address of the powerplay hardware manager.
+* @return always 0
+*/
+static int polaris10_program_voting_clients(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ /* Clear reset for voting clients before enabling DPM */
+ PHM_WRITE_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ SCLK_PWRMGT_CNTL, RESET_SCLK_CNT, 0);
+ PHM_WRITE_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ SCLK_PWRMGT_CNTL, RESET_BUSY_CNT, 0);
+
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC,
+ ixCG_FREQ_TRAN_VOTING_0, data->voting_rights_clients0);
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC,
+ ixCG_FREQ_TRAN_VOTING_1, data->voting_rights_clients1);
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC,
+ ixCG_FREQ_TRAN_VOTING_2, data->voting_rights_clients2);
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC,
+ ixCG_FREQ_TRAN_VOTING_3, data->voting_rights_clients3);
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC,
+ ixCG_FREQ_TRAN_VOTING_4, data->voting_rights_clients4);
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC,
+ ixCG_FREQ_TRAN_VOTING_5, data->voting_rights_clients5);
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC,
+ ixCG_FREQ_TRAN_VOTING_6, data->voting_rights_clients6);
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC,
+ ixCG_FREQ_TRAN_VOTING_7, data->voting_rights_clients7);
+
+ return 0;
+}
+
+/**
+* Get the location of various tables inside the FW image.
+*
+* @param hwmgr the address of the powerplay hardware manager.
+* @return always 0
+*/
+static int polaris10_process_firmware_header(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct polaris10_smumgr *smu_data = (struct polaris10_smumgr *)(hwmgr->smumgr->backend);
+ uint32_t tmp;
+ int result;
+ bool error = false;
+
+ result = polaris10_read_smc_sram_dword(hwmgr->smumgr,
+ SMU7_FIRMWARE_HEADER_LOCATION +
+ offsetof(SMU74_Firmware_Header, DpmTable),
+ &tmp, data->sram_end);
+
+ if (0 == result)
+ data->dpm_table_start = tmp;
+
+ error |= (0 != result);
+
+ result = polaris10_read_smc_sram_dword(hwmgr->smumgr,
+ SMU7_FIRMWARE_HEADER_LOCATION +
+ offsetof(SMU74_Firmware_Header, SoftRegisters),
+ &tmp, data->sram_end);
+
+ if (!result) {
+ data->soft_regs_start = tmp;
+ smu_data->soft_regs_start = tmp;
+ }
+
+ error |= (0 != result);
+
+ result = polaris10_read_smc_sram_dword(hwmgr->smumgr,
+ SMU7_FIRMWARE_HEADER_LOCATION +
+ offsetof(SMU74_Firmware_Header, mcRegisterTable),
+ &tmp, data->sram_end);
+
+ if (!result)
+ data->mc_reg_table_start = tmp;
+
+ result = polaris10_read_smc_sram_dword(hwmgr->smumgr,
+ SMU7_FIRMWARE_HEADER_LOCATION +
+ offsetof(SMU74_Firmware_Header, FanTable),
+ &tmp, data->sram_end);
+
+ if (!result)
+ data->fan_table_start = tmp;
+
+ error |= (0 != result);
+
+ result = polaris10_read_smc_sram_dword(hwmgr->smumgr,
+ SMU7_FIRMWARE_HEADER_LOCATION +
+ offsetof(SMU74_Firmware_Header, mcArbDramTimingTable),
+ &tmp, data->sram_end);
+
+ if (!result)
+ data->arb_table_start = tmp;
+
+ error |= (0 != result);
+
+ result = polaris10_read_smc_sram_dword(hwmgr->smumgr,
+ SMU7_FIRMWARE_HEADER_LOCATION +
+ offsetof(SMU74_Firmware_Header, Version),
+ &tmp, data->sram_end);
+
+ if (!result)
+ hwmgr->microcode_version_info.SMC = tmp;
+
+ error |= (0 != result);
+
+ return error ? -1 : 0;
+}
+
+/* Copy one arb setting to another and then switch the active set.
+ * arb_src and arb_dest is one of the MC_CG_ARB_FREQ_Fx constants.
+ */
+static int polaris10_copy_and_switch_arb_sets(struct pp_hwmgr *hwmgr,
+ uint32_t arb_src, uint32_t arb_dest)
+{
+ uint32_t mc_arb_dram_timing;
+ uint32_t mc_arb_dram_timing2;
+ uint32_t burst_time;
+ uint32_t mc_cg_config;
+
+ switch (arb_src) {
+ case MC_CG_ARB_FREQ_F0:
+ mc_arb_dram_timing = cgs_read_register(hwmgr->device, mmMC_ARB_DRAM_TIMING);
+ mc_arb_dram_timing2 = cgs_read_register(hwmgr->device, mmMC_ARB_DRAM_TIMING2);
+ burst_time = PHM_READ_FIELD(hwmgr->device, MC_ARB_BURST_TIME, STATE0);
+ break;
+ case MC_CG_ARB_FREQ_F1:
+ mc_arb_dram_timing = cgs_read_register(hwmgr->device, mmMC_ARB_DRAM_TIMING_1);
+ mc_arb_dram_timing2 = cgs_read_register(hwmgr->device, mmMC_ARB_DRAM_TIMING2_1);
+ burst_time = PHM_READ_FIELD(hwmgr->device, MC_ARB_BURST_TIME, STATE1);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ switch (arb_dest) {
+ case MC_CG_ARB_FREQ_F0:
+ cgs_write_register(hwmgr->device, mmMC_ARB_DRAM_TIMING, mc_arb_dram_timing);
+ cgs_write_register(hwmgr->device, mmMC_ARB_DRAM_TIMING2, mc_arb_dram_timing2);
+ PHM_WRITE_FIELD(hwmgr->device, MC_ARB_BURST_TIME, STATE0, burst_time);
+ break;
+ case MC_CG_ARB_FREQ_F1:
+ cgs_write_register(hwmgr->device, mmMC_ARB_DRAM_TIMING_1, mc_arb_dram_timing);
+ cgs_write_register(hwmgr->device, mmMC_ARB_DRAM_TIMING2_1, mc_arb_dram_timing2);
+ PHM_WRITE_FIELD(hwmgr->device, MC_ARB_BURST_TIME, STATE1, burst_time);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ mc_cg_config = cgs_read_register(hwmgr->device, mmMC_CG_CONFIG);
+ mc_cg_config |= 0x0000000F;
+ cgs_write_register(hwmgr->device, mmMC_CG_CONFIG, mc_cg_config);
+ PHM_WRITE_FIELD(hwmgr->device, MC_ARB_CG, CG_ARB_REQ, arb_dest);
+
+ return 0;
+}
+
+/**
+* Initial switch from ARB F0->F1
+*
+* @param hwmgr the address of the powerplay hardware manager.
+* @return always 0
+* This function is to be called from the SetPowerState table.
+*/
+static int polaris10_initial_switch_from_arbf0_to_f1(struct pp_hwmgr *hwmgr)
+{
+ return polaris10_copy_and_switch_arb_sets(hwmgr,
+ MC_CG_ARB_FREQ_F0, MC_CG_ARB_FREQ_F1);
+}
+
+static int polaris10_setup_default_pcie_table(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+ struct phm_ppt_v1_pcie_table *pcie_table = table_info->pcie_table;
+ uint32_t i, max_entry;
+
+ PP_ASSERT_WITH_CODE((data->use_pcie_performance_levels ||
+ data->use_pcie_power_saving_levels), "No pcie performance levels!",
+ return -EINVAL);
+
+ if (data->use_pcie_performance_levels &&
+ !data->use_pcie_power_saving_levels) {
+ data->pcie_gen_power_saving = data->pcie_gen_performance;
+ data->pcie_lane_power_saving = data->pcie_lane_performance;
+ } else if (!data->use_pcie_performance_levels &&
+ data->use_pcie_power_saving_levels) {
+ data->pcie_gen_performance = data->pcie_gen_power_saving;
+ data->pcie_lane_performance = data->pcie_lane_power_saving;
+ }
+
+ phm_reset_single_dpm_table(&data->dpm_table.pcie_speed_table,
+ SMU74_MAX_LEVELS_LINK,
+ MAX_REGULAR_DPM_NUMBER);
+
+ if (pcie_table != NULL) {
+ /* max_entry is used to make sure we reserve one PCIE level
+ * for boot level (fix for A+A PSPP issue).
+ * If PCIE table from PPTable have ULV entry + 8 entries,
+ * then ignore the last entry.*/
+ max_entry = (SMU74_MAX_LEVELS_LINK < pcie_table->count) ?
+ SMU74_MAX_LEVELS_LINK : pcie_table->count;
+ for (i = 1; i < max_entry; i++) {
+ phm_setup_pcie_table_entry(&data->dpm_table.pcie_speed_table, i - 1,
+ get_pcie_gen_support(data->pcie_gen_cap,
+ pcie_table->entries[i].gen_speed),
+ get_pcie_lane_support(data->pcie_lane_cap,
+ pcie_table->entries[i].lane_width));
+ }
+ data->dpm_table.pcie_speed_table.count = max_entry - 1;
+
+ /* Setup BIF_SCLK levels */
+ for (i = 0; i < max_entry; i++)
+ data->bif_sclk_table[i] = pcie_table->entries[i].pcie_sclk;
+ } else {
+ /* Hardcode Pcie Table */
+ phm_setup_pcie_table_entry(&data->dpm_table.pcie_speed_table, 0,
+ get_pcie_gen_support(data->pcie_gen_cap,
+ PP_Min_PCIEGen),
+ get_pcie_lane_support(data->pcie_lane_cap,
+ PP_Max_PCIELane));
+ phm_setup_pcie_table_entry(&data->dpm_table.pcie_speed_table, 1,
+ get_pcie_gen_support(data->pcie_gen_cap,
+ PP_Min_PCIEGen),
+ get_pcie_lane_support(data->pcie_lane_cap,
+ PP_Max_PCIELane));
+ phm_setup_pcie_table_entry(&data->dpm_table.pcie_speed_table, 2,
+ get_pcie_gen_support(data->pcie_gen_cap,
+ PP_Max_PCIEGen),
+ get_pcie_lane_support(data->pcie_lane_cap,
+ PP_Max_PCIELane));
+ phm_setup_pcie_table_entry(&data->dpm_table.pcie_speed_table, 3,
+ get_pcie_gen_support(data->pcie_gen_cap,
+ PP_Max_PCIEGen),
+ get_pcie_lane_support(data->pcie_lane_cap,
+ PP_Max_PCIELane));
+ phm_setup_pcie_table_entry(&data->dpm_table.pcie_speed_table, 4,
+ get_pcie_gen_support(data->pcie_gen_cap,
+ PP_Max_PCIEGen),
+ get_pcie_lane_support(data->pcie_lane_cap,
+ PP_Max_PCIELane));
+ phm_setup_pcie_table_entry(&data->dpm_table.pcie_speed_table, 5,
+ get_pcie_gen_support(data->pcie_gen_cap,
+ PP_Max_PCIEGen),
+ get_pcie_lane_support(data->pcie_lane_cap,
+ PP_Max_PCIELane));
+
+ data->dpm_table.pcie_speed_table.count = 6;
+ }
+ /* Populate last level for boot PCIE level, but do not increment count. */
+ phm_setup_pcie_table_entry(&data->dpm_table.pcie_speed_table,
+ data->dpm_table.pcie_speed_table.count,
+ get_pcie_gen_support(data->pcie_gen_cap,
+ PP_Min_PCIEGen),
+ get_pcie_lane_support(data->pcie_lane_cap,
+ PP_Max_PCIELane));
+
+ return 0;
+}
+
+/*
+ * This function is to initalize all DPM state tables
+ * for SMU7 based on the dependency table.
+ * Dynamic state patching function will then trim these
+ * state tables to the allowed range based
+ * on the power policy or external client requests,
+ * such as UVD request, etc.
+ */
+int polaris10_setup_default_dpm_tables(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+ uint32_t i;
+
+ struct phm_ppt_v1_clock_voltage_dependency_table *dep_sclk_table =
+ table_info->vdd_dep_on_sclk;
+ struct phm_ppt_v1_clock_voltage_dependency_table *dep_mclk_table =
+ table_info->vdd_dep_on_mclk;
+
+ PP_ASSERT_WITH_CODE(dep_sclk_table != NULL,
+ "SCLK dependency table is missing. This table is mandatory",
+ return -EINVAL);
+ PP_ASSERT_WITH_CODE(dep_sclk_table->count >= 1,
+ "SCLK dependency table has to have is missing."
+ "This table is mandatory",
+ return -EINVAL);
+
+ PP_ASSERT_WITH_CODE(dep_mclk_table != NULL,
+ "MCLK dependency table is missing. This table is mandatory",
+ return -EINVAL);
+ PP_ASSERT_WITH_CODE(dep_mclk_table->count >= 1,
+ "MCLK dependency table has to have is missing."
+ "This table is mandatory",
+ return -EINVAL);
+
+ /* clear the state table to reset everything to default */
+ phm_reset_single_dpm_table(
+ &data->dpm_table.sclk_table, SMU74_MAX_LEVELS_GRAPHICS, MAX_REGULAR_DPM_NUMBER);
+ phm_reset_single_dpm_table(
+ &data->dpm_table.mclk_table, SMU74_MAX_LEVELS_MEMORY, MAX_REGULAR_DPM_NUMBER);
+
+
+ /* Initialize Sclk DPM table based on allow Sclk values */
+ data->dpm_table.sclk_table.count = 0;
+ for (i = 0; i < dep_sclk_table->count; i++) {
+ if (i == 0 || data->dpm_table.sclk_table.dpm_levels[data->dpm_table.sclk_table.count - 1].value !=
+ dep_sclk_table->entries[i].clk) {
+
+ data->dpm_table.sclk_table.dpm_levels[data->dpm_table.sclk_table.count].value =
+ dep_sclk_table->entries[i].clk;
+
+ data->dpm_table.sclk_table.dpm_levels[data->dpm_table.sclk_table.count].enabled =
+ (i == 0) ? true : false;
+ data->dpm_table.sclk_table.count++;
+ }
+ }
+
+ /* Initialize Mclk DPM table based on allow Mclk values */
+ data->dpm_table.mclk_table.count = 0;
+ for (i = 0; i < dep_mclk_table->count; i++) {
+ if (i == 0 || data->dpm_table.mclk_table.dpm_levels
+ [data->dpm_table.mclk_table.count - 1].value !=
+ dep_mclk_table->entries[i].clk) {
+ data->dpm_table.mclk_table.dpm_levels[data->dpm_table.mclk_table.count].value =
+ dep_mclk_table->entries[i].clk;
+ data->dpm_table.mclk_table.dpm_levels[data->dpm_table.mclk_table.count].enabled =
+ (i == 0) ? true : false;
+ data->dpm_table.mclk_table.count++;
+ }
+ }
+
+ /* setup PCIE gen speed levels */
+ polaris10_setup_default_pcie_table(hwmgr);
+
+ /* save a copy of the default DPM table */
+ memcpy(&(data->golden_dpm_table), &(data->dpm_table),
+ sizeof(struct polaris10_dpm_table));
+
+ return 0;
+}
+
+uint8_t convert_to_vid(uint16_t vddc)
+{
+ return (uint8_t) ((6200 - (vddc * VOLTAGE_SCALE)) / 25);
+}
+
+/**
+ * Mvdd table preparation for SMC.
+ *
+ * @param *hwmgr The address of the hardware manager.
+ * @param *table The SMC DPM table structure to be populated.
+ * @return 0
+ */
+static int polaris10_populate_smc_mvdd_table(struct pp_hwmgr *hwmgr,
+ SMU74_Discrete_DpmTable *table)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ uint32_t count, level;
+
+ if (POLARIS10_VOLTAGE_CONTROL_BY_GPIO == data->mvdd_control) {
+ count = data->mvdd_voltage_table.count;
+ if (count > SMU_MAX_SMIO_LEVELS)
+ count = SMU_MAX_SMIO_LEVELS;
+ for (level = 0; level < count; level++) {
+ table->SmioTable2.Pattern[level].Voltage =
+ PP_HOST_TO_SMC_US(data->mvdd_voltage_table.entries[count].value * VOLTAGE_SCALE);
+ /* Index into DpmTable.Smio. Drive bits from Smio entry to get this voltage level.*/
+ table->SmioTable2.Pattern[level].Smio =
+ (uint8_t) level;
+ table->Smio[level] |=
+ data->mvdd_voltage_table.entries[level].smio_low;
+ }
+ table->SmioMask2 = data->vddci_voltage_table.mask_low;
+
+ table->MvddLevelCount = (uint32_t) PP_HOST_TO_SMC_UL(count);
+ }
+
+ return 0;
+}
+
+static int polaris10_populate_smc_vddci_table(struct pp_hwmgr *hwmgr,
+ struct SMU74_Discrete_DpmTable *table)
+{
+ uint32_t count, level;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ count = data->vddci_voltage_table.count;
+
+ if (POLARIS10_VOLTAGE_CONTROL_BY_GPIO == data->vddci_control) {
+ if (count > SMU_MAX_SMIO_LEVELS)
+ count = SMU_MAX_SMIO_LEVELS;
+ for (level = 0; level < count; ++level) {
+ table->SmioTable1.Pattern[level].Voltage =
+ PP_HOST_TO_SMC_US(data->vddci_voltage_table.entries[level].value * VOLTAGE_SCALE);
+ table->SmioTable1.Pattern[level].Smio = (uint8_t) level;
+
+ table->Smio[level] |= data->vddci_voltage_table.entries[level].smio_low;
+ }
+ }
+
+ table->SmioMask1 = data->vddci_voltage_table.mask_low;
+
+ return 0;
+}
+
+/**
+* Preparation of vddc and vddgfx CAC tables for SMC.
+*
+* @param hwmgr the address of the hardware manager
+* @param table the SMC DPM table structure to be populated
+* @return always 0
+*/
+static int polaris10_populate_cac_table(struct pp_hwmgr *hwmgr,
+ struct SMU74_Discrete_DpmTable *table)
+{
+ uint32_t count;
+ uint8_t index;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+ struct phm_ppt_v1_voltage_lookup_table *lookup_table =
+ table_info->vddc_lookup_table;
+ /* tables is already swapped, so in order to use the value from it,
+ * we need to swap it back.
+ * We are populating vddc CAC data to BapmVddc table
+ * in split and merged mode
+ */
+ for (count = 0; count < lookup_table->count; count++) {
+ index = phm_get_voltage_index(lookup_table,
+ data->vddc_voltage_table.entries[count].value);
+ table->BapmVddcVidLoSidd[count] = convert_to_vid(lookup_table->entries[index].us_cac_low);
+ table->BapmVddcVidHiSidd[count] = convert_to_vid(lookup_table->entries[index].us_cac_mid);
+ table->BapmVddcVidHiSidd2[count] = convert_to_vid(lookup_table->entries[index].us_cac_high);
+ }
+
+ return 0;
+}
+
+/**
+* Preparation of voltage tables for SMC.
+*
+* @param hwmgr the address of the hardware manager
+* @param table the SMC DPM table structure to be populated
+* @return always 0
+*/
+
+int polaris10_populate_smc_voltage_tables(struct pp_hwmgr *hwmgr,
+ struct SMU74_Discrete_DpmTable *table)
+{
+ polaris10_populate_smc_vddci_table(hwmgr, table);
+ polaris10_populate_smc_mvdd_table(hwmgr, table);
+ polaris10_populate_cac_table(hwmgr, table);
+
+ return 0;
+}
+
+static int polaris10_populate_ulv_level(struct pp_hwmgr *hwmgr,
+ struct SMU74_Discrete_Ulv *state)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+
+ state->CcPwrDynRm = 0;
+ state->CcPwrDynRm1 = 0;
+
+ state->VddcOffset = (uint16_t) table_info->us_ulv_voltage_offset;
+ state->VddcOffsetVid = (uint8_t)(table_info->us_ulv_voltage_offset *
+ VOLTAGE_VID_OFFSET_SCALE2 / VOLTAGE_VID_OFFSET_SCALE1);
+
+ state->VddcPhase = (data->vddc_phase_shed_control) ? 0 : 1;
+
+ CONVERT_FROM_HOST_TO_SMC_UL(state->CcPwrDynRm);
+ CONVERT_FROM_HOST_TO_SMC_UL(state->CcPwrDynRm1);
+ CONVERT_FROM_HOST_TO_SMC_US(state->VddcOffset);
+
+ return 0;
+}
+
+static int polaris10_populate_ulv_state(struct pp_hwmgr *hwmgr,
+ struct SMU74_Discrete_DpmTable *table)
+{
+ return polaris10_populate_ulv_level(hwmgr, &table->Ulv);
+}
+
+static int polaris10_populate_smc_link_level(struct pp_hwmgr *hwmgr,
+ struct SMU74_Discrete_DpmTable *table)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct polaris10_dpm_table *dpm_table = &data->dpm_table;
+ int i;
+
+ /* Index (dpm_table->pcie_speed_table.count)
+ * is reserved for PCIE boot level. */
+ for (i = 0; i <= dpm_table->pcie_speed_table.count; i++) {
+ table->LinkLevel[i].PcieGenSpeed =
+ (uint8_t)dpm_table->pcie_speed_table.dpm_levels[i].value;
+ table->LinkLevel[i].PcieLaneCount = (uint8_t)encode_pcie_lane_width(
+ dpm_table->pcie_speed_table.dpm_levels[i].param1);
+ table->LinkLevel[i].EnabledForActivity = 1;
+ table->LinkLevel[i].SPC = (uint8_t)(data->pcie_spc_cap & 0xff);
+ table->LinkLevel[i].DownThreshold = PP_HOST_TO_SMC_UL(5);
+ table->LinkLevel[i].UpThreshold = PP_HOST_TO_SMC_UL(30);
+ }
+
+ data->smc_state_table.LinkLevelCount =
+ (uint8_t)dpm_table->pcie_speed_table.count;
+ data->dpm_level_enable_mask.pcie_dpm_enable_mask =
+ phm_get_dpm_level_enable_mask_value(&dpm_table->pcie_speed_table);
+
+ return 0;
+}
+
+static uint32_t polaris10_get_xclk(struct pp_hwmgr *hwmgr)
+{
+ uint32_t reference_clock, tmp;
+ struct cgs_display_info info = {0};
+ struct cgs_mode_info mode_info;
+
+ info.mode_info = &mode_info;
+
+ tmp = PHM_READ_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC, CG_CLKPIN_CNTL_2, MUX_TCLK_TO_XCLK);
+
+ if (tmp)
+ return TCLK;
+
+ cgs_get_active_displays_info(hwmgr->device, &info);
+ reference_clock = mode_info.ref_clock;
+
+ tmp = PHM_READ_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC, CG_CLKPIN_CNTL, XTALIN_DIVIDE);
+
+ if (0 != tmp)
+ return reference_clock / 4;
+
+ return reference_clock;
+}
+
+/**
+* Calculates the SCLK dividers using the provided engine clock
+*
+* @param hwmgr the address of the hardware manager
+* @param clock the engine clock to use to populate the structure
+* @param sclk the SMC SCLK structure to be populated
+*/
+static int polaris10_calculate_sclk_params(struct pp_hwmgr *hwmgr,
+ uint32_t clock, SMU_SclkSetting *sclk_setting)
+{
+ const struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ const SMU74_Discrete_DpmTable *table = &(data->smc_state_table);
+ struct pp_atomctrl_clock_dividers_ai dividers;
+
+ uint32_t ref_clock;
+ uint32_t pcc_target_percent, pcc_target_freq, ss_target_percent, ss_target_freq;
+ uint8_t i;
+ int result;
+ uint64_t temp;
+
+ sclk_setting->SclkFrequency = clock;
+ /* get the engine clock dividers for this clock value */
+ result = atomctrl_get_engine_pll_dividers_ai(hwmgr, clock, ÷rs);
+ if (result == 0) {
+ sclk_setting->Fcw_int = dividers.usSclk_fcw_int;
+ sclk_setting->Fcw_frac = dividers.usSclk_fcw_frac;
+ sclk_setting->Pcc_fcw_int = dividers.usPcc_fcw_int;
+ sclk_setting->PllRange = dividers.ucSclkPllRange;
+ sclk_setting->Sclk_slew_rate = 0x400;
+ sclk_setting->Pcc_up_slew_rate = dividers.usPcc_fcw_slew_frac;
+ sclk_setting->Pcc_down_slew_rate = 0xffff;
+ sclk_setting->SSc_En = dividers.ucSscEnable;
+ sclk_setting->Fcw1_int = dividers.usSsc_fcw1_int;
+ sclk_setting->Fcw1_frac = dividers.usSsc_fcw1_frac;
+ sclk_setting->Sclk_ss_slew_rate = dividers.usSsc_fcw_slew_frac;
+ return result;
+ }
+
+ ref_clock = polaris10_get_xclk(hwmgr);
+
+ for (i = 0; i < NUM_SCLK_RANGE; i++) {
+ if (clock > data->range_table[i].trans_lower_frequency
+ && clock <= data->range_table[i].trans_upper_frequency) {
+ sclk_setting->PllRange = i;
+ break;
+ }
+ }
+
+ sclk_setting->Fcw_int = (uint16_t)((clock << table->SclkFcwRangeTable[sclk_setting->PllRange].postdiv) / ref_clock);
+ temp = clock << table->SclkFcwRangeTable[sclk_setting->PllRange].postdiv;
+ temp <<= 0x10;
+ do_div(temp, ref_clock);
+ sclk_setting->Fcw_frac = temp & 0xffff;
+
+ pcc_target_percent = 10; /* Hardcode 10% for now. */
+ pcc_target_freq = clock - (clock * pcc_target_percent / 100);
+ sclk_setting->Pcc_fcw_int = (uint16_t)((pcc_target_freq << table->SclkFcwRangeTable[sclk_setting->PllRange].postdiv) / ref_clock);
+
+ ss_target_percent = 2; /* Hardcode 2% for now. */
+ sclk_setting->SSc_En = 0;
+ if (ss_target_percent) {
+ sclk_setting->SSc_En = 1;
+ ss_target_freq = clock - (clock * ss_target_percent / 100);
+ sclk_setting->Fcw1_int = (uint16_t)((ss_target_freq << table->SclkFcwRangeTable[sclk_setting->PllRange].postdiv) / ref_clock);
+ temp = ss_target_freq << table->SclkFcwRangeTable[sclk_setting->PllRange].postdiv;
+ temp <<= 0x10;
+ do_div(temp, ref_clock);
+ sclk_setting->Fcw1_frac = temp & 0xffff;
+ }
+
+ return 0;
+}
+
+static int polaris10_get_dependency_volt_by_clk(struct pp_hwmgr *hwmgr,
+ struct phm_ppt_v1_clock_voltage_dependency_table *dep_table,
+ uint32_t clock, SMU_VoltageLevel *voltage, uint32_t *mvdd)
+{
+ uint32_t i;
+ uint16_t vddci;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ *voltage = *mvdd = 0;
+
+ /* clock - voltage dependency table is empty table */
+ if (dep_table->count == 0)
+ return -EINVAL;
+
+ for (i = 0; i < dep_table->count; i++) {
+ /* find first sclk bigger than request */
+ if (dep_table->entries[i].clk >= clock) {
+ *voltage |= (dep_table->entries[i].vddc *
+ VOLTAGE_SCALE) << VDDC_SHIFT;
+ if (POLARIS10_VOLTAGE_CONTROL_NONE == data->vddci_control)
+ *voltage |= (data->vbios_boot_state.vddci_bootup_value *
+ VOLTAGE_SCALE) << VDDCI_SHIFT;
+ else if (dep_table->entries[i].vddci)
+ *voltage |= (dep_table->entries[i].vddci *
+ VOLTAGE_SCALE) << VDDCI_SHIFT;
+ else {
+ vddci = phm_find_closest_vddci(&(data->vddci_voltage_table),
+ (dep_table->entries[i].vddc -
+ (uint16_t)data->vddc_vddci_delta));
+ *voltage |= (vddci * VOLTAGE_SCALE) << VDDCI_SHIFT;
+ }
+
+ if (POLARIS10_VOLTAGE_CONTROL_NONE == data->mvdd_control)
+ *mvdd = data->vbios_boot_state.mvdd_bootup_value *
+ VOLTAGE_SCALE;
+ else if (dep_table->entries[i].mvdd)
+ *mvdd = (uint32_t) dep_table->entries[i].mvdd *
+ VOLTAGE_SCALE;
+
+ *voltage |= 1 << PHASES_SHIFT;
+ return 0;
+ }
+ }
+
+ /* sclk is bigger than max sclk in the dependence table */
+ *voltage |= (dep_table->entries[i - 1].vddc * VOLTAGE_SCALE) << VDDC_SHIFT;
+
+ if (POLARIS10_VOLTAGE_CONTROL_NONE == data->vddci_control)
+ *voltage |= (data->vbios_boot_state.vddci_bootup_value *
+ VOLTAGE_SCALE) << VDDCI_SHIFT;
+ else if (dep_table->entries[i-1].vddci) {
+ vddci = phm_find_closest_vddci(&(data->vddci_voltage_table),
+ (dep_table->entries[i].vddc -
+ (uint16_t)data->vddc_vddci_delta));
+ *voltage |= (vddci * VOLTAGE_SCALE) << VDDCI_SHIFT;
+ }
+
+ if (POLARIS10_VOLTAGE_CONTROL_NONE == data->mvdd_control)
+ *mvdd = data->vbios_boot_state.mvdd_bootup_value * VOLTAGE_SCALE;
+ else if (dep_table->entries[i].mvdd)
+ *mvdd = (uint32_t) dep_table->entries[i - 1].mvdd * VOLTAGE_SCALE;
+
+ return 0;
+}
+
+static const sclkFcwRange_t Range_Table[NUM_SCLK_RANGE] =
+{ {VCO_2_4, POSTDIV_DIV_BY_16, 75, 160, 112},
+ {VCO_3_6, POSTDIV_DIV_BY_16, 112, 224, 160},
+ {VCO_2_4, POSTDIV_DIV_BY_8, 75, 160, 112},
+ {VCO_3_6, POSTDIV_DIV_BY_8, 112, 224, 160},
+ {VCO_2_4, POSTDIV_DIV_BY_4, 75, 160, 112},
+ {VCO_3_6, POSTDIV_DIV_BY_4, 112, 216, 160},
+ {VCO_2_4, POSTDIV_DIV_BY_2, 75, 160, 108},
+ {VCO_3_6, POSTDIV_DIV_BY_2, 112, 216, 160} };
+
+static void polaris10_get_sclk_range_table(struct pp_hwmgr *hwmgr)
+{
+ uint32_t i, ref_clk;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ SMU74_Discrete_DpmTable *table = &(data->smc_state_table);
+ struct pp_atom_ctrl_sclk_range_table range_table_from_vbios = { { {0} } };
+
+ ref_clk = polaris10_get_xclk(hwmgr);
+
+ if (0 == atomctrl_get_smc_sclk_range_table(hwmgr, &range_table_from_vbios)) {
+ for (i = 0; i < NUM_SCLK_RANGE; i++) {
+ table->SclkFcwRangeTable[i].vco_setting = range_table_from_vbios.entry[i].ucVco_setting;
+ table->SclkFcwRangeTable[i].postdiv = range_table_from_vbios.entry[i].ucPostdiv;
+ table->SclkFcwRangeTable[i].fcw_pcc = range_table_from_vbios.entry[i].usFcw_pcc;
+
+ table->SclkFcwRangeTable[i].fcw_trans_upper = range_table_from_vbios.entry[i].usFcw_trans_upper;
+ table->SclkFcwRangeTable[i].fcw_trans_lower = range_table_from_vbios.entry[i].usRcw_trans_lower;
+
+ CONVERT_FROM_HOST_TO_SMC_US(table->SclkFcwRangeTable[i].fcw_pcc);
+ CONVERT_FROM_HOST_TO_SMC_US(table->SclkFcwRangeTable[i].fcw_trans_upper);
+ CONVERT_FROM_HOST_TO_SMC_US(table->SclkFcwRangeTable[i].fcw_trans_lower);
+ }
+ return;
+ }
+
+ for (i = 0; i < NUM_SCLK_RANGE; i++) {
+
+ data->range_table[i].trans_lower_frequency = (ref_clk * Range_Table[i].fcw_trans_lower) >> Range_Table[i].postdiv;
+ data->range_table[i].trans_upper_frequency = (ref_clk * Range_Table[i].fcw_trans_upper) >> Range_Table[i].postdiv;
+
+ table->SclkFcwRangeTable[i].vco_setting = Range_Table[i].vco_setting;
+ table->SclkFcwRangeTable[i].postdiv = Range_Table[i].postdiv;
+ table->SclkFcwRangeTable[i].fcw_pcc = Range_Table[i].fcw_pcc;
+
+ table->SclkFcwRangeTable[i].fcw_trans_upper = Range_Table[i].fcw_trans_upper;
+ table->SclkFcwRangeTable[i].fcw_trans_lower = Range_Table[i].fcw_trans_lower;
+
+ CONVERT_FROM_HOST_TO_SMC_US(table->SclkFcwRangeTable[i].fcw_pcc);
+ CONVERT_FROM_HOST_TO_SMC_US(table->SclkFcwRangeTable[i].fcw_trans_upper);
+ CONVERT_FROM_HOST_TO_SMC_US(table->SclkFcwRangeTable[i].fcw_trans_lower);
+ }
+}
+
+/**
+* Populates single SMC SCLK structure using the provided engine clock
+*
+* @param hwmgr the address of the hardware manager
+* @param clock the engine clock to use to populate the structure
+* @param sclk the SMC SCLK structure to be populated
+*/
+
+static int polaris10_populate_single_graphic_level(struct pp_hwmgr *hwmgr,
+ uint32_t clock, uint16_t sclk_al_threshold,
+ struct SMU74_Discrete_GraphicsLevel *level)
+{
+ int result, i, temp;
+ /* PP_Clocks minClocks; */
+ uint32_t mvdd;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+ SMU_SclkSetting curr_sclk_setting = { 0 };
+
+ result = polaris10_calculate_sclk_params(hwmgr, clock, &curr_sclk_setting);
+
+ /* populate graphics levels */
+ result = polaris10_get_dependency_volt_by_clk(hwmgr,
+ table_info->vdd_dep_on_sclk, clock,
+ &level->MinVoltage, &mvdd);
+
+ PP_ASSERT_WITH_CODE((0 == result),
+ "can not find VDDC voltage value for "
+ "VDDC engine clock dependency table",
+ return result);
+ level->ActivityLevel = sclk_al_threshold;
+
+ level->CcPwrDynRm = 0;
+ level->CcPwrDynRm1 = 0;
+ level->EnabledForActivity = 0;
+ level->EnabledForThrottle = 1;
+ level->UpHyst = 10;
+ level->DownHyst = 0;
+ level->VoltageDownHyst = 0;
+ level->PowerThrottle = 0;
+
+ /*
+ * TODO: get minimum clocks from dal configaration
+ * PECI_GetMinClockSettings(hwmgr->pPECI, &minClocks);
+ */
+ /* data->DisplayTiming.minClockInSR = minClocks.engineClockInSR; */
+
+ /* get level->DeepSleepDivId
+ if (phm_cap_enabled(hwmgr->platformDescriptor.platformCaps, PHM_PlatformCaps_SclkDeepSleep))
+ level->DeepSleepDivId = PhwFiji_GetSleepDividerIdFromClock(hwmgr, clock, minClocks.engineClockInSR);
+ */
+ PP_ASSERT_WITH_CODE((clock >= POLARIS10_MINIMUM_ENGINE_CLOCK), "Engine clock can't satisfy stutter requirement!", return 0);
+ for (i = POLARIS10_MAX_DEEPSLEEP_DIVIDER_ID; ; i--) {
+ temp = clock >> i;
+
+ if (temp >= POLARIS10_MINIMUM_ENGINE_CLOCK || i == 0)
+ break;
+ }
+
+ level->DeepSleepDivId = i;
+
+ /* Default to slow, highest DPM level will be
+ * set to PPSMC_DISPLAY_WATERMARK_LOW later.
+ */
+ if (data->update_up_hyst)
+ level->UpHyst = (uint8_t)data->up_hyst;
+ if (data->update_down_hyst)
+ level->DownHyst = (uint8_t)data->down_hyst;
+
+ level->SclkSetting = curr_sclk_setting;
+
+ CONVERT_FROM_HOST_TO_SMC_UL(level->MinVoltage);
+ CONVERT_FROM_HOST_TO_SMC_UL(level->CcPwrDynRm);
+ CONVERT_FROM_HOST_TO_SMC_UL(level->CcPwrDynRm1);
+ CONVERT_FROM_HOST_TO_SMC_US(level->ActivityLevel);
+ CONVERT_FROM_HOST_TO_SMC_UL(level->SclkSetting.SclkFrequency);
+ CONVERT_FROM_HOST_TO_SMC_US(level->SclkSetting.Fcw_int);
+ CONVERT_FROM_HOST_TO_SMC_US(level->SclkSetting.Fcw_frac);
+ CONVERT_FROM_HOST_TO_SMC_US(level->SclkSetting.Pcc_fcw_int);
+ CONVERT_FROM_HOST_TO_SMC_US(level->SclkSetting.Sclk_slew_rate);
+ CONVERT_FROM_HOST_TO_SMC_US(level->SclkSetting.Pcc_up_slew_rate);
+ CONVERT_FROM_HOST_TO_SMC_US(level->SclkSetting.Pcc_down_slew_rate);
+ CONVERT_FROM_HOST_TO_SMC_US(level->SclkSetting.Fcw1_int);
+ CONVERT_FROM_HOST_TO_SMC_US(level->SclkSetting.Fcw1_frac);
+ CONVERT_FROM_HOST_TO_SMC_US(level->SclkSetting.Sclk_ss_slew_rate);
+ return 0;
+}
+
+/**
+* Populates all SMC SCLK levels' structure based on the trimmed allowed dpm engine clock states
+*
+* @param hwmgr the address of the hardware manager
+*/
+static int polaris10_populate_all_graphic_levels(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct polaris10_dpm_table *dpm_table = &data->dpm_table;
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+ struct phm_ppt_v1_pcie_table *pcie_table = table_info->pcie_table;
+ uint8_t pcie_entry_cnt = (uint8_t) data->dpm_table.pcie_speed_table.count;
+ int result = 0;
+ uint32_t array = data->dpm_table_start +
+ offsetof(SMU74_Discrete_DpmTable, GraphicsLevel);
+ uint32_t array_size = sizeof(struct SMU74_Discrete_GraphicsLevel) *
+ SMU74_MAX_LEVELS_GRAPHICS;
+ struct SMU74_Discrete_GraphicsLevel *levels =
+ data->smc_state_table.GraphicsLevel;
+ uint32_t i, max_entry;
+ uint8_t hightest_pcie_level_enabled = 0,
+ lowest_pcie_level_enabled = 0,
+ mid_pcie_level_enabled = 0,
+ count = 0;
+
+ polaris10_get_sclk_range_table(hwmgr);
+
+ for (i = 0; i < dpm_table->sclk_table.count; i++) {
+
+ result = polaris10_populate_single_graphic_level(hwmgr,
+ dpm_table->sclk_table.dpm_levels[i].value,
+ (uint16_t)data->activity_target[i],
+ &(data->smc_state_table.GraphicsLevel[i]));
+ if (result)
+ return result;
+
+ /* Making sure only DPM level 0-1 have Deep Sleep Div ID populated. */
+ if (i > 1)
+ levels[i].DeepSleepDivId = 0;
+ }
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_SPLLShutdownSupport))
+ data->smc_state_table.GraphicsLevel[0].SclkSetting.SSc_En = 0;
+
+ data->smc_state_table.GraphicsLevel[0].EnabledForActivity = 1;
+ data->smc_state_table.GraphicsDpmLevelCount =
+ (uint8_t)dpm_table->sclk_table.count;
+ data->dpm_level_enable_mask.sclk_dpm_enable_mask =
+ phm_get_dpm_level_enable_mask_value(&dpm_table->sclk_table);
+
+
+ if (pcie_table != NULL) {
+ PP_ASSERT_WITH_CODE((1 <= pcie_entry_cnt),
+ "There must be 1 or more PCIE levels defined in PPTable.",
+ return -EINVAL);
+ max_entry = pcie_entry_cnt - 1;
+ for (i = 0; i < dpm_table->sclk_table.count; i++)
+ levels[i].pcieDpmLevel =
+ (uint8_t) ((i < max_entry) ? i : max_entry);
+ } else {
+ while (data->dpm_level_enable_mask.pcie_dpm_enable_mask &&
+ ((data->dpm_level_enable_mask.pcie_dpm_enable_mask &
+ (1 << (hightest_pcie_level_enabled + 1))) != 0))
+ hightest_pcie_level_enabled++;
+
+ while (data->dpm_level_enable_mask.pcie_dpm_enable_mask &&
+ ((data->dpm_level_enable_mask.pcie_dpm_enable_mask &
+ (1 << lowest_pcie_level_enabled)) == 0))
+ lowest_pcie_level_enabled++;
+
+ while ((count < hightest_pcie_level_enabled) &&
+ ((data->dpm_level_enable_mask.pcie_dpm_enable_mask &
+ (1 << (lowest_pcie_level_enabled + 1 + count))) == 0))
+ count++;
+
+ mid_pcie_level_enabled = (lowest_pcie_level_enabled + 1 + count) <
+ hightest_pcie_level_enabled ?
+ (lowest_pcie_level_enabled + 1 + count) :
+ hightest_pcie_level_enabled;
+
+ /* set pcieDpmLevel to hightest_pcie_level_enabled */
+ for (i = 2; i < dpm_table->sclk_table.count; i++)
+ levels[i].pcieDpmLevel = hightest_pcie_level_enabled;
+
+ /* set pcieDpmLevel to lowest_pcie_level_enabled */
+ levels[0].pcieDpmLevel = lowest_pcie_level_enabled;
+
+ /* set pcieDpmLevel to mid_pcie_level_enabled */
+ levels[1].pcieDpmLevel = mid_pcie_level_enabled;
+ }
+ /* level count will send to smc once at init smc table and never change */
+ result = polaris10_copy_bytes_to_smc(hwmgr->smumgr, array, (uint8_t *)levels,
+ (uint32_t)array_size, data->sram_end);
+
+ return result;
+}
+
+static int polaris10_populate_single_memory_level(struct pp_hwmgr *hwmgr,
+ uint32_t clock, struct SMU74_Discrete_MemoryLevel *mem_level)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+ int result = 0;
+ struct cgs_display_info info = {0, 0, NULL};
+
+ cgs_get_active_displays_info(hwmgr->device, &info);
+
+ if (table_info->vdd_dep_on_mclk) {
+ result = polaris10_get_dependency_volt_by_clk(hwmgr,
+ table_info->vdd_dep_on_mclk, clock,
+ &mem_level->MinVoltage, &mem_level->MinMvdd);
+ PP_ASSERT_WITH_CODE((0 == result),
+ "can not find MinVddc voltage value from memory "
+ "VDDC voltage dependency table", return result);
+ }
+
+ mem_level->MclkFrequency = clock;
+ mem_level->StutterEnable = 0;
+ mem_level->EnabledForThrottle = 1;
+ mem_level->EnabledForActivity = 0;
+ mem_level->UpHyst = 0;
+ mem_level->DownHyst = 100;
+ mem_level->VoltageDownHyst = 0;
+ mem_level->ActivityLevel = (uint16_t)data->mclk_activity_target;
+ mem_level->StutterEnable = false;
+
+ mem_level->DisplayWatermark = PPSMC_DISPLAY_WATERMARK_LOW;
+
+ data->display_timing.num_existing_displays = info.display_count;
+
+ if ((data->mclk_stutter_mode_threshold) &&
+ (clock <= data->mclk_stutter_mode_threshold) &&
+ (PHM_READ_FIELD(hwmgr->device, DPG_PIPE_STUTTER_CONTROL,
+ STUTTER_ENABLE) & 0x1))
+ mem_level->StutterEnable = true;
+
+ if (!result) {
+ CONVERT_FROM_HOST_TO_SMC_UL(mem_level->MinMvdd);
+ CONVERT_FROM_HOST_TO_SMC_UL(mem_level->MclkFrequency);
+ CONVERT_FROM_HOST_TO_SMC_US(mem_level->ActivityLevel);
+ CONVERT_FROM_HOST_TO_SMC_UL(mem_level->MinVoltage);
+ }
+ return result;
+}
+
+/**
+* Populates all SMC MCLK levels' structure based on the trimmed allowed dpm memory clock states
+*
+* @param hwmgr the address of the hardware manager
+*/
+static int polaris10_populate_all_memory_levels(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct polaris10_dpm_table *dpm_table = &data->dpm_table;
+ int result;
+ /* populate MCLK dpm table to SMU7 */
+ uint32_t array = data->dpm_table_start +
+ offsetof(SMU74_Discrete_DpmTable, MemoryLevel);
+ uint32_t array_size = sizeof(SMU74_Discrete_MemoryLevel) *
+ SMU74_MAX_LEVELS_MEMORY;
+ struct SMU74_Discrete_MemoryLevel *levels =
+ data->smc_state_table.MemoryLevel;
+ uint32_t i;
+
+ for (i = 0; i < dpm_table->mclk_table.count; i++) {
+ PP_ASSERT_WITH_CODE((0 != dpm_table->mclk_table.dpm_levels[i].value),
+ "can not populate memory level as memory clock is zero",
+ return -EINVAL);
+ result = polaris10_populate_single_memory_level(hwmgr,
+ dpm_table->mclk_table.dpm_levels[i].value,
+ &levels[i]);
+ if (i == dpm_table->mclk_table.count - 1) {
+ levels[i].DisplayWatermark = PPSMC_DISPLAY_WATERMARK_HIGH;
+ levels[i].EnabledForActivity = 1;
+ }
+ if (result)
+ return result;
+ }
+
+ /* in order to prevent MC activity from stutter mode to push DPM up.
+ * the UVD change complements this by putting the MCLK in
+ * a higher state by default such that we are not effected by
+ * up threshold or and MCLK DPM latency.
+ */
+ levels[0].ActivityLevel = (uint16_t)data->mclk_dpm0_activity_target;
+ CONVERT_FROM_HOST_TO_SMC_US(levels[0].ActivityLevel);
+
+ data->smc_state_table.MemoryDpmLevelCount =
+ (uint8_t)dpm_table->mclk_table.count;
+ data->dpm_level_enable_mask.mclk_dpm_enable_mask =
+ phm_get_dpm_level_enable_mask_value(&dpm_table->mclk_table);
+
+ /* level count will send to smc once at init smc table and never change */
+ result = polaris10_copy_bytes_to_smc(hwmgr->smumgr, array, (uint8_t *)levels,
+ (uint32_t)array_size, data->sram_end);
+
+ return result;
+}
+
+/**
+* Populates the SMC MVDD structure using the provided memory clock.
+*
+* @param hwmgr the address of the hardware manager
+* @param mclk the MCLK value to be used in the decision if MVDD should be high or low.
+* @param voltage the SMC VOLTAGE structure to be populated
+*/
+int polaris10_populate_mvdd_value(struct pp_hwmgr *hwmgr,
+ uint32_t mclk, SMIO_Pattern *smio_pat)
+{
+ const struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+ uint32_t i = 0;
+
+ if (POLARIS10_VOLTAGE_CONTROL_NONE != data->mvdd_control) {
+ /* find mvdd value which clock is more than request */
+ for (i = 0; i < table_info->vdd_dep_on_mclk->count; i++) {
+ if (mclk <= table_info->vdd_dep_on_mclk->entries[i].clk) {
+ smio_pat->Voltage = data->mvdd_voltage_table.entries[i].value;
+ break;
+ }
+ }
+ PP_ASSERT_WITH_CODE(i < table_info->vdd_dep_on_mclk->count,
+ "MVDD Voltage is outside the supported range.",
+ return -EINVAL);
+ } else
+ return -EINVAL;
+
+ return 0;
+}
+
+static int polaris10_populate_smc_acpi_level(struct pp_hwmgr *hwmgr,
+ SMU74_Discrete_DpmTable *table)
+{
+ int result = 0;
+ uint32_t sclk_frequency;
+ const struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+ SMIO_Pattern vol_level;
+ uint32_t mvdd;
+ uint16_t us_mvdd;
+
+ table->ACPILevel.Flags &= ~PPSMC_SWSTATE_FLAG_DC;
+
+ if (!data->sclk_dpm_key_disabled) {
+ /* Get MinVoltage and Frequency from DPM0,
+ * already converted to SMC_UL */
+ sclk_frequency = data->dpm_table.sclk_table.dpm_levels[0].value;
+ result = polaris10_get_dependency_volt_by_clk(hwmgr,
+ table_info->vdd_dep_on_sclk,
+ table->ACPILevel.SclkFrequency,
+ &table->ACPILevel.MinVoltage, &mvdd);
+ PP_ASSERT_WITH_CODE((0 == result),
+ "Cannot find ACPI VDDC voltage value "
+ "in Clock Dependency Table", );
+ } else {
+ sclk_frequency = data->vbios_boot_state.sclk_bootup_value;
+ table->ACPILevel.MinVoltage =
+ data->vbios_boot_state.vddc_bootup_value * VOLTAGE_SCALE;
+ }
+
+ result = polaris10_calculate_sclk_params(hwmgr, sclk_frequency, &(table->ACPILevel.SclkSetting));
+ PP_ASSERT_WITH_CODE(result == 0, "Error retrieving Engine Clock dividers from VBIOS.", return result);
+
+ table->ACPILevel.DeepSleepDivId = 0;
+ table->ACPILevel.CcPwrDynRm = 0;
+ table->ACPILevel.CcPwrDynRm1 = 0;
+
+ CONVERT_FROM_HOST_TO_SMC_UL(table->ACPILevel.Flags);
+ CONVERT_FROM_HOST_TO_SMC_UL(table->ACPILevel.MinVoltage);
+ CONVERT_FROM_HOST_TO_SMC_UL(table->ACPILevel.CcPwrDynRm);
+ CONVERT_FROM_HOST_TO_SMC_UL(table->ACPILevel.CcPwrDynRm1);
+
+ CONVERT_FROM_HOST_TO_SMC_UL(table->ACPILevel.SclkSetting.SclkFrequency);
+ CONVERT_FROM_HOST_TO_SMC_US(table->ACPILevel.SclkSetting.Fcw_int);
+ CONVERT_FROM_HOST_TO_SMC_US(table->ACPILevel.SclkSetting.Fcw_frac);
+ CONVERT_FROM_HOST_TO_SMC_US(table->ACPILevel.SclkSetting.Pcc_fcw_int);
+ CONVERT_FROM_HOST_TO_SMC_US(table->ACPILevel.SclkSetting.Sclk_slew_rate);
+ CONVERT_FROM_HOST_TO_SMC_US(table->ACPILevel.SclkSetting.Pcc_up_slew_rate);
+ CONVERT_FROM_HOST_TO_SMC_US(table->ACPILevel.SclkSetting.Pcc_down_slew_rate);
+ CONVERT_FROM_HOST_TO_SMC_US(table->ACPILevel.SclkSetting.Fcw1_int);
+ CONVERT_FROM_HOST_TO_SMC_US(table->ACPILevel.SclkSetting.Fcw1_frac);
+ CONVERT_FROM_HOST_TO_SMC_US(table->ACPILevel.SclkSetting.Sclk_ss_slew_rate);
+
+ if (!data->mclk_dpm_key_disabled) {
+ /* Get MinVoltage and Frequency from DPM0, already converted to SMC_UL */
+ table->MemoryACPILevel.MclkFrequency =
+ data->dpm_table.mclk_table.dpm_levels[0].value;
+ result = polaris10_get_dependency_volt_by_clk(hwmgr,
+ table_info->vdd_dep_on_mclk,
+ table->MemoryACPILevel.MclkFrequency,
+ &table->MemoryACPILevel.MinVoltage, &mvdd);
+ PP_ASSERT_WITH_CODE((0 == result),
+ "Cannot find ACPI VDDCI voltage value "
+ "in Clock Dependency Table",
+ );
+ } else {
+ table->MemoryACPILevel.MclkFrequency =
+ data->vbios_boot_state.mclk_bootup_value;
+ table->MemoryACPILevel.MinVoltage =
+ data->vbios_boot_state.vddci_bootup_value * VOLTAGE_SCALE;
+ }
+
+ us_mvdd = 0;
+ if ((POLARIS10_VOLTAGE_CONTROL_NONE == data->mvdd_control) ||
+ (data->mclk_dpm_key_disabled))
+ us_mvdd = data->vbios_boot_state.mvdd_bootup_value;
+ else {
+ if (!polaris10_populate_mvdd_value(hwmgr,
+ data->dpm_table.mclk_table.dpm_levels[0].value,
+ &vol_level))
+ us_mvdd = vol_level.Voltage;
+ }
+
+ if (0 == polaris10_populate_mvdd_value(hwmgr, 0, &vol_level))
+ table->MemoryACPILevel.MinMvdd = PP_HOST_TO_SMC_UL(vol_level.Voltage);
+ else
+ table->MemoryACPILevel.MinMvdd = 0;
+
+ table->MemoryACPILevel.StutterEnable = false;
+
+ table->MemoryACPILevel.EnabledForThrottle = 0;
+ table->MemoryACPILevel.EnabledForActivity = 0;
+ table->MemoryACPILevel.UpHyst = 0;
+ table->MemoryACPILevel.DownHyst = 100;
+ table->MemoryACPILevel.VoltageDownHyst = 0;
+ table->MemoryACPILevel.ActivityLevel =
+ PP_HOST_TO_SMC_US((uint16_t)data->mclk_activity_target);
+
+ CONVERT_FROM_HOST_TO_SMC_UL(table->MemoryACPILevel.MclkFrequency);
+ CONVERT_FROM_HOST_TO_SMC_UL(table->MemoryACPILevel.MinVoltage);
+
+ return result;
+}
+
+static int polaris10_populate_smc_vce_level(struct pp_hwmgr *hwmgr,
+ SMU74_Discrete_DpmTable *table)
+{
+ int result = -EINVAL;
+ uint8_t count;
+ struct pp_atomctrl_clock_dividers_vi dividers;
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+ struct phm_ppt_v1_mm_clock_voltage_dependency_table *mm_table =
+ table_info->mm_dep_table;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ table->VceLevelCount = (uint8_t)(mm_table->count);
+ table->VceBootLevel = 0;
+
+ for (count = 0; count < table->VceLevelCount; count++) {
+ table->VceLevel[count].Frequency = mm_table->entries[count].eclk;
+ table->VceLevel[count].MinVoltage = 0;
+ table->VceLevel[count].MinVoltage |=
+ (mm_table->entries[count].vddc * VOLTAGE_SCALE) << VDDC_SHIFT;
+ table->VceLevel[count].MinVoltage |=
+ ((mm_table->entries[count].vddc - data->vddc_vddci_delta) *
+ VOLTAGE_SCALE) << VDDCI_SHIFT;
+ table->VceLevel[count].MinVoltage |= 1 << PHASES_SHIFT;
+
+ /*retrieve divider value for VBIOS */
+ result = atomctrl_get_dfs_pll_dividers_vi(hwmgr,
+ table->VceLevel[count].Frequency, ÷rs);
+ PP_ASSERT_WITH_CODE((0 == result),
+ "can not find divide id for VCE engine clock",
+ return result);
+
+ table->VceLevel[count].Divider = (uint8_t)dividers.pll_post_divider;
+
+ CONVERT_FROM_HOST_TO_SMC_UL(table->VceLevel[count].Frequency);
+ CONVERT_FROM_HOST_TO_SMC_UL(table->VceLevel[count].MinVoltage);
+ }
+ return result;
+}
+
+static int polaris10_populate_smc_samu_level(struct pp_hwmgr *hwmgr,
+ SMU74_Discrete_DpmTable *table)
+{
+ int result = -EINVAL;
+ uint8_t count;
+ struct pp_atomctrl_clock_dividers_vi dividers;
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+ struct phm_ppt_v1_mm_clock_voltage_dependency_table *mm_table =
+ table_info->mm_dep_table;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ table->SamuBootLevel = 0;
+ table->SamuLevelCount = (uint8_t)(mm_table->count);
+
+ for (count = 0; count < table->SamuLevelCount; count++) {
+ /* not sure whether we need evclk or not */
+ table->SamuLevel[count].MinVoltage = 0;
+ table->SamuLevel[count].Frequency = mm_table->entries[count].samclock;
+ table->SamuLevel[count].MinVoltage |= (mm_table->entries[count].vddc *
+ VOLTAGE_SCALE) << VDDC_SHIFT;
+ table->SamuLevel[count].MinVoltage |= ((mm_table->entries[count].vddc -
+ data->vddc_vddci_delta) * VOLTAGE_SCALE) << VDDCI_SHIFT;
+ table->SamuLevel[count].MinVoltage |= 1 << PHASES_SHIFT;
+
+ /* retrieve divider value for VBIOS */
+ result = atomctrl_get_dfs_pll_dividers_vi(hwmgr,
+ table->SamuLevel[count].Frequency, ÷rs);
+ PP_ASSERT_WITH_CODE((0 == result),
+ "can not find divide id for samu clock", return result);
+
+ table->SamuLevel[count].Divider = (uint8_t)dividers.pll_post_divider;
+
+ CONVERT_FROM_HOST_TO_SMC_UL(table->SamuLevel[count].Frequency);
+ CONVERT_FROM_HOST_TO_SMC_UL(table->SamuLevel[count].MinVoltage);
+ }
+ return result;
+}
+
+static int polaris10_populate_memory_timing_parameters(struct pp_hwmgr *hwmgr,
+ int32_t eng_clock, int32_t mem_clock,
+ SMU74_Discrete_MCArbDramTimingTableEntry *arb_regs)
+{
+ uint32_t dram_timing;
+ uint32_t dram_timing2;
+ uint32_t burst_time;
+ int result;
+
+ result = atomctrl_set_engine_dram_timings_rv770(hwmgr,
+ eng_clock, mem_clock);
+ PP_ASSERT_WITH_CODE(result == 0,
+ "Error calling VBIOS to set DRAM_TIMING.", return result);
+
+ dram_timing = cgs_read_register(hwmgr->device, mmMC_ARB_DRAM_TIMING);
+ dram_timing2 = cgs_read_register(hwmgr->device, mmMC_ARB_DRAM_TIMING2);
+ burst_time = PHM_READ_FIELD(hwmgr->device, MC_ARB_BURST_TIME, STATE0);
+
+
+ arb_regs->McArbDramTiming = PP_HOST_TO_SMC_UL(dram_timing);
+ arb_regs->McArbDramTiming2 = PP_HOST_TO_SMC_UL(dram_timing2);
+ arb_regs->McArbBurstTime = (uint8_t)burst_time;
+
+ return 0;
+}
+
+static int polaris10_program_memory_timing_parameters(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct SMU74_Discrete_MCArbDramTimingTable arb_regs;
+ uint32_t i, j;
+ int result = 0;
+
+ for (i = 0; i < data->dpm_table.sclk_table.count; i++) {
+ for (j = 0; j < data->dpm_table.mclk_table.count; j++) {
+ result = polaris10_populate_memory_timing_parameters(hwmgr,
+ data->dpm_table.sclk_table.dpm_levels[i].value,
+ data->dpm_table.mclk_table.dpm_levels[j].value,
+ &arb_regs.entries[i][j]);
+ if (result == 0)
+ result = atomctrl_set_ac_timing_ai(hwmgr, data->dpm_table.mclk_table.dpm_levels[j].value, j);
+ if (result != 0)
+ return result;
+ }
+ }
+
+ result = polaris10_copy_bytes_to_smc(
+ hwmgr->smumgr,
+ data->arb_table_start,
+ (uint8_t *)&arb_regs,
+ sizeof(SMU74_Discrete_MCArbDramTimingTable),
+ data->sram_end);
+ return result;
+}
+
+static int polaris10_populate_smc_uvd_level(struct pp_hwmgr *hwmgr,
+ struct SMU74_Discrete_DpmTable *table)
+{
+ int result = -EINVAL;
+ uint8_t count;
+ struct pp_atomctrl_clock_dividers_vi dividers;
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+ struct phm_ppt_v1_mm_clock_voltage_dependency_table *mm_table =
+ table_info->mm_dep_table;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ table->UvdLevelCount = (uint8_t)(mm_table->count);
+ table->UvdBootLevel = 0;
+
+ for (count = 0; count < table->UvdLevelCount; count++) {
+ table->UvdLevel[count].MinVoltage = 0;
+ table->UvdLevel[count].VclkFrequency = mm_table->entries[count].vclk;
+ table->UvdLevel[count].DclkFrequency = mm_table->entries[count].dclk;
+ table->UvdLevel[count].MinVoltage |= (mm_table->entries[count].vddc *
+ VOLTAGE_SCALE) << VDDC_SHIFT;
+ table->UvdLevel[count].MinVoltage |= ((mm_table->entries[count].vddc -
+ data->vddc_vddci_delta) * VOLTAGE_SCALE) << VDDCI_SHIFT;
+ table->UvdLevel[count].MinVoltage |= 1 << PHASES_SHIFT;
+
+ /* retrieve divider value for VBIOS */
+ result = atomctrl_get_dfs_pll_dividers_vi(hwmgr,
+ table->UvdLevel[count].VclkFrequency, ÷rs);
+ PP_ASSERT_WITH_CODE((0 == result),
+ "can not find divide id for Vclk clock", return result);
+
+ table->UvdLevel[count].VclkDivider = (uint8_t)dividers.pll_post_divider;
+
+ result = atomctrl_get_dfs_pll_dividers_vi(hwmgr,
+ table->UvdLevel[count].DclkFrequency, ÷rs);
+ PP_ASSERT_WITH_CODE((0 == result),
+ "can not find divide id for Dclk clock", return result);
+
+ table->UvdLevel[count].DclkDivider = (uint8_t)dividers.pll_post_divider;
+
+ CONVERT_FROM_HOST_TO_SMC_UL(table->UvdLevel[count].VclkFrequency);
+ CONVERT_FROM_HOST_TO_SMC_UL(table->UvdLevel[count].DclkFrequency);
+ CONVERT_FROM_HOST_TO_SMC_UL(table->UvdLevel[count].MinVoltage);
+
+ }
+ return result;
+}
+
+static int polaris10_populate_smc_boot_level(struct pp_hwmgr *hwmgr,
+ struct SMU74_Discrete_DpmTable *table)
+{
+ int result = 0;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ table->GraphicsBootLevel = 0;
+ table->MemoryBootLevel = 0;
+
+ /* find boot level from dpm table */
+ result = phm_find_boot_level(&(data->dpm_table.sclk_table),
+ data->vbios_boot_state.sclk_bootup_value,
+ (uint32_t *)&(table->GraphicsBootLevel));
+
+ result = phm_find_boot_level(&(data->dpm_table.mclk_table),
+ data->vbios_boot_state.mclk_bootup_value,
+ (uint32_t *)&(table->MemoryBootLevel));
+
+ table->BootVddc = data->vbios_boot_state.vddc_bootup_value *
+ VOLTAGE_SCALE;
+ table->BootVddci = data->vbios_boot_state.vddci_bootup_value *
+ VOLTAGE_SCALE;
+ table->BootMVdd = data->vbios_boot_state.mvdd_bootup_value *
+ VOLTAGE_SCALE;
+
+ CONVERT_FROM_HOST_TO_SMC_US(table->BootVddc);
+ CONVERT_FROM_HOST_TO_SMC_US(table->BootVddci);
+ CONVERT_FROM_HOST_TO_SMC_US(table->BootMVdd);
+
+ return 0;
+}
+
+
+static int polaris10_populate_smc_initailial_state(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+ uint8_t count, level;
+
+ count = (uint8_t)(table_info->vdd_dep_on_sclk->count);
+
+ for (level = 0; level < count; level++) {
+ if (table_info->vdd_dep_on_sclk->entries[level].clk >=
+ data->vbios_boot_state.sclk_bootup_value) {
+ data->smc_state_table.GraphicsBootLevel = level;
+ break;
+ }
+ }
+
+ count = (uint8_t)(table_info->vdd_dep_on_mclk->count);
+ for (level = 0; level < count; level++) {
+ if (table_info->vdd_dep_on_mclk->entries[level].clk >=
+ data->vbios_boot_state.mclk_bootup_value) {
+ data->smc_state_table.MemoryBootLevel = level;
+ break;
+ }
+ }
+
+ return 0;
+}
+
+static int polaris10_populate_clock_stretcher_data_table(struct pp_hwmgr *hwmgr)
+{
+ uint32_t ro, efuse, efuse2, clock_freq, volt_without_cks,
+ volt_with_cks, value;
+ uint16_t clock_freq_u16;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ uint8_t type, i, j, cks_setting, stretch_amount, stretch_amount2,
+ volt_offset = 0;
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+ struct phm_ppt_v1_clock_voltage_dependency_table *sclk_table =
+ table_info->vdd_dep_on_sclk;
+
+ stretch_amount = (uint8_t)table_info->cac_dtp_table->usClockStretchAmount;
+
+ /* Read SMU_Eefuse to read and calculate RO and determine
+ * if the part is SS or FF. if RO >= 1660MHz, part is FF.
+ */
+ efuse = cgs_read_ind_register(hwmgr->device, CGS_IND_REG__SMC,
+ ixSMU_EFUSE_0 + (146 * 4));
+ efuse2 = cgs_read_ind_register(hwmgr->device, CGS_IND_REG__SMC,
+ ixSMU_EFUSE_0 + (148 * 4));
+ efuse &= 0xFF000000;
+ efuse = efuse >> 24;
+ efuse2 &= 0xF;
+
+ if (efuse2 == 1)
+ ro = (2300 - 1350) * efuse / 255 + 1350;
+ else
+ ro = (2500 - 1000) * efuse / 255 + 1000;
+
+ if (ro >= 1660)
+ type = 0;
+ else
+ type = 1;
+
+ /* Populate Stretch amount */
+ data->smc_state_table.ClockStretcherAmount = stretch_amount;
+
+ /* Populate Sclk_CKS_masterEn0_7 and Sclk_voltageOffset */
+ for (i = 0; i < sclk_table->count; i++) {
+ data->smc_state_table.Sclk_CKS_masterEn0_7 |=
+ sclk_table->entries[i].cks_enable << i;
+ volt_without_cks = (uint32_t)((14041 *
+ (sclk_table->entries[i].clk/100) / 10000 + 3571 + 75 - ro) * 1000 /
+ (4026 - (13924 * (sclk_table->entries[i].clk/100) / 10000)));
+ volt_with_cks = (uint32_t)((13946 *
+ (sclk_table->entries[i].clk/100) / 10000 + 3320 + 45 - ro) * 1000 /
+ (3664 - (11454 * (sclk_table->entries[i].clk/100) / 10000)));
+ if (volt_without_cks >= volt_with_cks)
+ volt_offset = (uint8_t)(((volt_without_cks - volt_with_cks +
+ sclk_table->entries[i].cks_voffset) * 100 / 625) + 1);
+ data->smc_state_table.Sclk_voltageOffset[i] = volt_offset;
+ }
+
+ PHM_WRITE_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC, PWR_CKS_ENABLE,
+ STRETCH_ENABLE, 0x0);
+ PHM_WRITE_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC, PWR_CKS_ENABLE,
+ masterReset, 0x1);
+ /* PHM_WRITE_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC, PWR_CKS_ENABLE, staticEnable, 0x1); */
+ PHM_WRITE_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC, PWR_CKS_ENABLE,
+ masterReset, 0x0);
+
+ /* Populate CKS Lookup Table */
+ if (stretch_amount == 1 || stretch_amount == 2 || stretch_amount == 5)
+ stretch_amount2 = 0;
+ else if (stretch_amount == 3 || stretch_amount == 4)
+ stretch_amount2 = 1;
+ else {
+ phm_cap_unset(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_ClockStretcher);
+ PP_ASSERT_WITH_CODE(false,
+ "Stretch Amount in PPTable not supported\n",
+ return -EINVAL);
+ }
+
+ value = cgs_read_ind_register(hwmgr->device, CGS_IND_REG__SMC,
+ ixPWR_CKS_CNTL);
+ value &= 0xFFC2FF87;
+ data->smc_state_table.CKS_LOOKUPTable.CKS_LOOKUPTableEntry[0].minFreq =
+ polaris10_clock_stretcher_lookup_table[stretch_amount2][0];
+ data->smc_state_table.CKS_LOOKUPTable.CKS_LOOKUPTableEntry[0].maxFreq =
+ polaris10_clock_stretcher_lookup_table[stretch_amount2][1];
+ clock_freq_u16 = (uint16_t)(PP_SMC_TO_HOST_UL(data->smc_state_table.
+ GraphicsLevel[data->smc_state_table.GraphicsDpmLevelCount - 1].SclkSetting.SclkFrequency) / 100);
+ if (polaris10_clock_stretcher_lookup_table[stretch_amount2][0] < clock_freq_u16
+ && polaris10_clock_stretcher_lookup_table[stretch_amount2][1] > clock_freq_u16) {
+ /* Program PWR_CKS_CNTL. CKS_USE_FOR_LOW_FREQ */
+ value |= (polaris10_clock_stretcher_lookup_table[stretch_amount2][3]) << 16;
+ /* Program PWR_CKS_CNTL. CKS_LDO_REFSEL */
+ value |= (polaris10_clock_stretcher_lookup_table[stretch_amount2][2]) << 18;
+ /* Program PWR_CKS_CNTL. CKS_STRETCH_AMOUNT */
+ value |= (polaris10_clock_stretch_amount_conversion
+ [polaris10_clock_stretcher_lookup_table[stretch_amount2][3]]
+ [stretch_amount]) << 3;
+ }
+ CONVERT_FROM_HOST_TO_SMC_US(data->smc_state_table.CKS_LOOKUPTable.CKS_LOOKUPTableEntry[0].minFreq);
+ CONVERT_FROM_HOST_TO_SMC_US(data->smc_state_table.CKS_LOOKUPTable.CKS_LOOKUPTableEntry[0].maxFreq);
+ data->smc_state_table.CKS_LOOKUPTable.CKS_LOOKUPTableEntry[0].setting =
+ polaris10_clock_stretcher_lookup_table[stretch_amount2][2] & 0x7F;
+ data->smc_state_table.CKS_LOOKUPTable.CKS_LOOKUPTableEntry[0].setting |=
+ (polaris10_clock_stretcher_lookup_table[stretch_amount2][3]) << 7;
+
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC,
+ ixPWR_CKS_CNTL, value);
+
+ /* Populate DDT Lookup Table */
+ for (i = 0; i < 4; i++) {
+ /* Assign the minimum and maximum VID stored
+ * in the last row of Clock Stretcher Voltage Table.
+ */
+ data->smc_state_table.ClockStretcherDataTable.ClockStretcherDataTableEntry[i].minVID =
+ (uint8_t) polaris10_clock_stretcher_ddt_table[type][i][2];
+ data->smc_state_table.ClockStretcherDataTable.ClockStretcherDataTableEntry[i].maxVID =
+ (uint8_t) polaris10_clock_stretcher_ddt_table[type][i][3];
+ /* Loop through each SCLK and check the frequency
+ * to see if it lies within the frequency for clock stretcher.
+ */
+ for (j = 0; j < data->smc_state_table.GraphicsDpmLevelCount; j++) {
+ cks_setting = 0;
+ clock_freq = PP_SMC_TO_HOST_UL(
+ data->smc_state_table.GraphicsLevel[j].SclkSetting.SclkFrequency);
+ /* Check the allowed frequency against the sclk level[j].
+ * Sclk's endianness has already been converted,
+ * and it's in 10Khz unit,
+ * as opposed to Data table, which is in Mhz unit.
+ */
+ if (clock_freq >= (polaris10_clock_stretcher_ddt_table[type][i][0]) * 100) {
+ cks_setting |= 0x2;
+ if (clock_freq < (polaris10_clock_stretcher_ddt_table[type][i][1]) * 100)
+ cks_setting |= 0x1;
+ }
+ data->smc_state_table.ClockStretcherDataTable.ClockStretcherDataTableEntry[i].setting
+ |= cks_setting << (j * 2);
+ }
+ CONVERT_FROM_HOST_TO_SMC_US(
+ data->smc_state_table.ClockStretcherDataTable.ClockStretcherDataTableEntry[i].setting);
+ }
+
+ value = cgs_read_ind_register(hwmgr->device, CGS_IND_REG__SMC, ixPWR_CKS_CNTL);
+ value &= 0xFFFFFFFE;
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC, ixPWR_CKS_CNTL, value);
+
+ return 0;
+}
+
+/**
+* Populates the SMC VRConfig field in DPM table.
+*
+* @param hwmgr the address of the hardware manager
+* @param table the SMC DPM table structure to be populated
+* @return always 0
+*/
+static int polaris10_populate_vr_config(struct pp_hwmgr *hwmgr,
+ struct SMU74_Discrete_DpmTable *table)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ uint16_t config;
+
+ config = VR_MERGED_WITH_VDDC;
+ table->VRConfig |= (config << VRCONF_VDDGFX_SHIFT);
+
+ /* Set Vddc Voltage Controller */
+ if (POLARIS10_VOLTAGE_CONTROL_BY_SVID2 == data->voltage_control) {
+ config = VR_SVI2_PLANE_1;
+ table->VRConfig |= config;
+ } else {
+ PP_ASSERT_WITH_CODE(false,
+ "VDDC should be on SVI2 control in merged mode!",
+ );
+ }
+ /* Set Vddci Voltage Controller */
+ if (POLARIS10_VOLTAGE_CONTROL_BY_SVID2 == data->vddci_control) {
+ config = VR_SVI2_PLANE_2; /* only in merged mode */
+ table->VRConfig |= (config << VRCONF_VDDCI_SHIFT);
+ } else if (POLARIS10_VOLTAGE_CONTROL_BY_GPIO == data->vddci_control) {
+ config = VR_SMIO_PATTERN_1;
+ table->VRConfig |= (config << VRCONF_VDDCI_SHIFT);
+ } else {
+ config = VR_STATIC_VOLTAGE;
+ table->VRConfig |= (config << VRCONF_VDDCI_SHIFT);
+ }
+ /* Set Mvdd Voltage Controller */
+ if (POLARIS10_VOLTAGE_CONTROL_BY_SVID2 == data->mvdd_control) {
+ config = VR_SVI2_PLANE_2;
+ table->VRConfig |= (config << VRCONF_MVDD_SHIFT);
+ } else if (POLARIS10_VOLTAGE_CONTROL_BY_GPIO == data->mvdd_control) {
+ config = VR_SMIO_PATTERN_2;
+ table->VRConfig |= (config << VRCONF_MVDD_SHIFT);
+ } else {
+ config = VR_STATIC_VOLTAGE;
+ table->VRConfig |= (config << VRCONF_MVDD_SHIFT);
+ }
+
+ return 0;
+}
+
+/**
+* Initializes the SMC table and uploads it
+*
+* @param hwmgr the address of the powerplay hardware manager.
+* @return always 0
+*/
+static int polaris10_init_smc_table(struct pp_hwmgr *hwmgr)
+{
+ int result;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+ struct SMU74_Discrete_DpmTable *table = &(data->smc_state_table);
+ const struct polaris10_ulv_parm *ulv = &(data->ulv);
+ uint8_t i;
+ struct pp_atomctrl_gpio_pin_assignment gpio_pin;
+ pp_atomctrl_clock_dividers_vi dividers;
+
+ result = polaris10_setup_default_dpm_tables(hwmgr);
+ PP_ASSERT_WITH_CODE(0 == result,
+ "Failed to setup default DPM tables!", return result);
+
+ if (POLARIS10_VOLTAGE_CONTROL_NONE != data->voltage_control)
+ polaris10_populate_smc_voltage_tables(hwmgr, table);
+
+ table->SystemFlags = 0;
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_AutomaticDCTransition))
+ table->SystemFlags |= PPSMC_SYSTEMFLAG_GPIO_DC;
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_StepVddc))
+ table->SystemFlags |= PPSMC_SYSTEMFLAG_STEPVDDC;
+
+ if (data->is_memory_gddr5)
+ table->SystemFlags |= PPSMC_SYSTEMFLAG_GDDR5;
+
+ if (ulv->ulv_supported && table_info->us_ulv_voltage_offset) {
+ result = polaris10_populate_ulv_state(hwmgr, table);
+ PP_ASSERT_WITH_CODE(0 == result,
+ "Failed to initialize ULV state!", return result);
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC,
+ ixCG_ULV_PARAMETER, PPPOLARIS10_CGULVPARAMETER_DFLT);
+ }
+
+ result = polaris10_populate_smc_link_level(hwmgr, table);
+ PP_ASSERT_WITH_CODE(0 == result,
+ "Failed to initialize Link Level!", return result);
+
+ result = polaris10_populate_all_graphic_levels(hwmgr);
+ PP_ASSERT_WITH_CODE(0 == result,
+ "Failed to initialize Graphics Level!", return result);
+
+ result = polaris10_populate_all_memory_levels(hwmgr);
+ PP_ASSERT_WITH_CODE(0 == result,
+ "Failed to initialize Memory Level!", return result);
+
+ result = polaris10_populate_smc_acpi_level(hwmgr, table);
+ PP_ASSERT_WITH_CODE(0 == result,
+ "Failed to initialize ACPI Level!", return result);
+
+ result = polaris10_populate_smc_vce_level(hwmgr, table);
+ PP_ASSERT_WITH_CODE(0 == result,
+ "Failed to initialize VCE Level!", return result);
+
+ result = polaris10_populate_smc_samu_level(hwmgr, table);
+ PP_ASSERT_WITH_CODE(0 == result,
+ "Failed to initialize SAMU Level!", return result);
+
+ /* Since only the initial state is completely set up at this point
+ * (the other states are just copies of the boot state) we only
+ * need to populate the ARB settings for the initial state.
+ */
+ result = polaris10_program_memory_timing_parameters(hwmgr);
+ PP_ASSERT_WITH_CODE(0 == result,
+ "Failed to Write ARB settings for the initial state.", return result);
+
+ result = polaris10_populate_smc_uvd_level(hwmgr, table);
+ PP_ASSERT_WITH_CODE(0 == result,
+ "Failed to initialize UVD Level!", return result);
+
+ result = polaris10_populate_smc_boot_level(hwmgr, table);
+ PP_ASSERT_WITH_CODE(0 == result,
+ "Failed to initialize Boot Level!", return result);
+
+ result = polaris10_populate_smc_initailial_state(hwmgr);
+ PP_ASSERT_WITH_CODE(0 == result,
+ "Failed to initialize Boot State!", return result);
+
+ result = polaris10_populate_bapm_parameters_in_dpm_table(hwmgr);
+ PP_ASSERT_WITH_CODE(0 == result,
+ "Failed to populate BAPM Parameters!", return result);
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_ClockStretcher)) {
+ result = polaris10_populate_clock_stretcher_data_table(hwmgr);
+ PP_ASSERT_WITH_CODE(0 == result,
+ "Failed to populate Clock Stretcher Data Table!",
+ return result);
+ }
+
+ table->GraphicsVoltageChangeEnable = 1;
+ table->GraphicsThermThrottleEnable = 1;
+ table->GraphicsInterval = 1;
+ table->VoltageInterval = 1;
+ table->ThermalInterval = 1;
+ table->TemperatureLimitHigh =
+ table_info->cac_dtp_table->usTargetOperatingTemp *
+ POLARIS10_Q88_FORMAT_CONVERSION_UNIT;
+ table->TemperatureLimitLow =
+ (table_info->cac_dtp_table->usTargetOperatingTemp - 1) *
+ POLARIS10_Q88_FORMAT_CONVERSION_UNIT;
+ table->MemoryVoltageChangeEnable = 1;
+ table->MemoryInterval = 1;
+ table->VoltageResponseTime = 0;
+ table->PhaseResponseTime = 0;
+ table->MemoryThermThrottleEnable = 1;
+ table->PCIeBootLinkLevel = 0;
+ table->PCIeGenInterval = 1;
+ table->VRConfig = 0;
+
+ result = polaris10_populate_vr_config(hwmgr, table);
+ PP_ASSERT_WITH_CODE(0 == result,
+ "Failed to populate VRConfig setting!", return result);
+
+ table->ThermGpio = 17;
+ table->SclkStepSize = 0x4000;
+
+ if (atomctrl_get_pp_assign_pin(hwmgr, VDDC_VRHOT_GPIO_PINID, &gpio_pin)) {
+ table->VRHotGpio = gpio_pin.uc_gpio_pin_bit_shift;
+ } else {
+ table->VRHotGpio = POLARIS10_UNUSED_GPIO_PIN;
+ phm_cap_unset(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_RegulatorHot);
+ }
+
+ if (atomctrl_get_pp_assign_pin(hwmgr, PP_AC_DC_SWITCH_GPIO_PINID,
+ &gpio_pin)) {
+ table->AcDcGpio = gpio_pin.uc_gpio_pin_bit_shift;
+ phm_cap_set(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_AutomaticDCTransition);
+ } else {
+ table->AcDcGpio = POLARIS10_UNUSED_GPIO_PIN;
+ phm_cap_unset(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_AutomaticDCTransition);
+ }
+
+ /* Thermal Output GPIO */
+ if (atomctrl_get_pp_assign_pin(hwmgr, THERMAL_INT_OUTPUT_GPIO_PINID,
+ &gpio_pin)) {
+ phm_cap_set(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_ThermalOutGPIO);
+
+ table->ThermOutGpio = gpio_pin.uc_gpio_pin_bit_shift;
+
+ /* For porlarity read GPIOPAD_A with assigned Gpio pin
+ * since VBIOS will program this register to set 'inactive state',
+ * driver can then determine 'active state' from this and
+ * program SMU with correct polarity
+ */
+ table->ThermOutPolarity = (0 == (cgs_read_register(hwmgr->device, mmGPIOPAD_A)
+ & (1 << gpio_pin.uc_gpio_pin_bit_shift))) ? 1:0;
+ table->ThermOutMode = SMU7_THERM_OUT_MODE_THERM_ONLY;
+
+ /* if required, combine VRHot/PCC with thermal out GPIO */
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps, PHM_PlatformCaps_RegulatorHot)
+ && phm_cap_enabled(hwmgr->platform_descriptor.platformCaps, PHM_PlatformCaps_CombinePCCWithThermalSignal))
+ table->ThermOutMode = SMU7_THERM_OUT_MODE_THERM_VRHOT;
+ } else {
+ table->ThermOutGpio = 17;
+ table->ThermOutPolarity = 1;
+ table->ThermOutMode = SMU7_THERM_OUT_MODE_DISABLE;
+ }
+
+ /* Populate BIF_SCLK levels into SMC DPM table */
+ for (i = 0; i <= data->dpm_table.pcie_speed_table.count; i++) {
+ result = atomctrl_get_dfs_pll_dividers_vi(hwmgr, data->bif_sclk_table[i], ÷rs);
+ PP_ASSERT_WITH_CODE((result == 0), "Can not find DFS divide id for Sclk", return result);
+
+ if (i == 0)
+ table->Ulv.BifSclkDfs = PP_HOST_TO_SMC_US((USHORT)(dividers.pll_post_divider));
+ else
+ table->LinkLevel[i-1].BifSclkDfs = PP_HOST_TO_SMC_US((USHORT)(dividers.pll_post_divider));
+ }
+
+ for (i = 0; i < SMU74_MAX_ENTRIES_SMIO; i++)
+ table->Smio[i] = PP_HOST_TO_SMC_UL(table->Smio[i]);
+
+ CONVERT_FROM_HOST_TO_SMC_UL(table->SystemFlags);
+ CONVERT_FROM_HOST_TO_SMC_UL(table->VRConfig);
+ CONVERT_FROM_HOST_TO_SMC_UL(table->SmioMask1);
+ CONVERT_FROM_HOST_TO_SMC_UL(table->SmioMask2);
+ CONVERT_FROM_HOST_TO_SMC_UL(table->SclkStepSize);
+ CONVERT_FROM_HOST_TO_SMC_US(table->TemperatureLimitHigh);
+ CONVERT_FROM_HOST_TO_SMC_US(table->TemperatureLimitLow);
+ CONVERT_FROM_HOST_TO_SMC_US(table->VoltageResponseTime);
+ CONVERT_FROM_HOST_TO_SMC_US(table->PhaseResponseTime);
+
+ /* Upload all dpm data to SMC memory.(dpm level, dpm level count etc) */
+ result = polaris10_copy_bytes_to_smc(hwmgr->smumgr,
+ data->dpm_table_start +
+ offsetof(SMU74_Discrete_DpmTable, SystemFlags),
+ (uint8_t *)&(table->SystemFlags),
+ sizeof(SMU74_Discrete_DpmTable) - 3 * sizeof(SMU74_PIDController),
+ data->sram_end);
+ PP_ASSERT_WITH_CODE(0 == result,
+ "Failed to upload dpm data to SMC memory!", return result);
+
+ return 0;
+}
+
+/**
+* Initialize the ARB DRAM timing table's index field.
+*
+* @param hwmgr the address of the powerplay hardware manager.
+* @return always 0
+*/
+static int polaris10_init_arb_table_index(struct pp_hwmgr *hwmgr)
+{
+ const struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ uint32_t tmp;
+ int result;
+
+ /* This is a read-modify-write on the first byte of the ARB table.
+ * The first byte in the SMU73_Discrete_MCArbDramTimingTable structure
+ * is the field 'current'.
+ * This solution is ugly, but we never write the whole table only
+ * individual fields in it.
+ * In reality this field should not be in that structure
+ * but in a soft register.
+ */
+ result = polaris10_read_smc_sram_dword(hwmgr->smumgr,
+ data->arb_table_start, &tmp, data->sram_end);
+
+ if (result)
+ return result;
+
+ tmp &= 0x00FFFFFF;
+ tmp |= ((uint32_t)MC_CG_ARB_FREQ_F1) << 24;
+
+ return polaris10_write_smc_sram_dword(hwmgr->smumgr,
+ data->arb_table_start, tmp, data->sram_end);
+}
+
+static int polaris10_enable_vrhot_gpio_interrupt(struct pp_hwmgr *hwmgr)
+{
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_RegulatorHot))
+ return smum_send_msg_to_smc(hwmgr->smumgr,
+ PPSMC_MSG_EnableVRHotGPIOInterrupt);
+
+ return 0;
+}
+
+static int polaris10_enable_sclk_control(struct pp_hwmgr *hwmgr)
+{
+ PHM_WRITE_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC, SCLK_PWRMGT_CNTL,
+ SCLK_PWRMGT_OFF, 0);
+ return 0;
+}
+
+static int polaris10_enable_ulv(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct polaris10_ulv_parm *ulv = &(data->ulv);
+
+ if (ulv->ulv_supported)
+ return smum_send_msg_to_smc(hwmgr->smumgr, PPSMC_MSG_EnableULV);
+
+ return 0;
+}
+
+static int polaris10_enable_deep_sleep_master_switch(struct pp_hwmgr *hwmgr)
+{
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_SclkDeepSleep)) {
+ if (smum_send_msg_to_smc(hwmgr->smumgr, PPSMC_MSG_MASTER_DeepSleep_ON))
+ PP_ASSERT_WITH_CODE(false,
+ "Attempt to enable Master Deep Sleep switch failed!",
+ return -1);
+ } else {
+ if (smum_send_msg_to_smc(hwmgr->smumgr,
+ PPSMC_MSG_MASTER_DeepSleep_OFF)) {
+ PP_ASSERT_WITH_CODE(false,
+ "Attempt to disable Master Deep Sleep switch failed!",
+ return -1);
+ }
+ }
+
+ return 0;
+}
+
+static int polaris10_enable_sclk_mclk_dpm(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ /* enable SCLK dpm */
+ if (!data->sclk_dpm_key_disabled)
+ PP_ASSERT_WITH_CODE(
+ (0 == smum_send_msg_to_smc(hwmgr->smumgr, PPSMC_MSG_DPM_Enable)),
+ "Failed to enable SCLK DPM during DPM Start Function!",
+ return -1);
+
+ /* enable MCLK dpm */
+ if (0 == data->mclk_dpm_key_disabled) {
+
+ PP_ASSERT_WITH_CODE(
+ (0 == smum_send_msg_to_smc(hwmgr->smumgr,
+ PPSMC_MSG_MCLKDPM_Enable)),
+ "Failed to enable MCLK DPM during DPM Start Function!",
+ return -1);
+
+
+ PHM_WRITE_FIELD(hwmgr->device, MC_SEQ_CNTL_3, CAC_EN, 0x1);
+
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC, ixLCAC_MC0_CNTL, 0x5);
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC, ixLCAC_MC1_CNTL, 0x5);
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC, ixLCAC_CPL_CNTL, 0x100005);
+ udelay(10);
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC, ixLCAC_MC0_CNTL, 0x400005);
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC, ixLCAC_MC1_CNTL, 0x400005);
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC, ixLCAC_CPL_CNTL, 0x500005);
+ }
+
+ return 0;
+}
+
+static int polaris10_start_dpm(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ /*enable general power management */
+
+ PHM_WRITE_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC, GENERAL_PWRMGT,
+ GLOBAL_PWRMGT_EN, 1);
+
+ /* enable sclk deep sleep */
+
+ PHM_WRITE_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC, SCLK_PWRMGT_CNTL,
+ DYNAMIC_PM_EN, 1);
+
+ /* prepare for PCIE DPM */
+
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC,
+ data->soft_regs_start + offsetof(SMU74_SoftRegisters,
+ VoltageChangeTimeout), 0x1000);
+ PHM_WRITE_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__PCIE,
+ SWRST_COMMAND_1, RESETLC, 0x0);
+/*
+ PP_ASSERT_WITH_CODE(
+ (0 == smum_send_msg_to_smc(hwmgr->smumgr,
+ PPSMC_MSG_Voltage_Cntl_Enable)),
+ "Failed to enable voltage DPM during DPM Start Function!",
+ return -1);
+*/
+
+ if (polaris10_enable_sclk_mclk_dpm(hwmgr)) {
+ printk(KERN_ERR "Failed to enable Sclk DPM and Mclk DPM!");
+ return -1;
+ }
+
+ /* enable PCIE dpm */
+ if (0 == data->pcie_dpm_key_disabled) {
+ PP_ASSERT_WITH_CODE(
+ (0 == smum_send_msg_to_smc(hwmgr->smumgr,
+ PPSMC_MSG_PCIeDPM_Enable)),
+ "Failed to enable pcie DPM during DPM Start Function!",
+ return -1);
+ }
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_Falcon_QuickTransition)) {
+ PP_ASSERT_WITH_CODE((0 == smum_send_msg_to_smc(hwmgr->smumgr,
+ PPSMC_MSG_EnableACDCGPIOInterrupt)),
+ "Failed to enable AC DC GPIO Interrupt!",
+ );
+ }
+
+ return 0;
+}
+
+static void polaris10_set_dpm_event_sources(struct pp_hwmgr *hwmgr, uint32_t sources)
+{
+ bool protection;
+ enum DPM_EVENT_SRC src;
+
+ switch (sources) {
+ default:
+ printk(KERN_ERR "Unknown throttling event sources.");
+ /* fall through */
+ case 0:
+ protection = false;
+ /* src is unused */
+ break;
+ case (1 << PHM_AutoThrottleSource_Thermal):
+ protection = true;
+ src = DPM_EVENT_SRC_DIGITAL;
+ break;
+ case (1 << PHM_AutoThrottleSource_External):
+ protection = true;
+ src = DPM_EVENT_SRC_EXTERNAL;
+ break;
+ case (1 << PHM_AutoThrottleSource_External) |
+ (1 << PHM_AutoThrottleSource_Thermal):
+ protection = true;
+ src = DPM_EVENT_SRC_DIGITAL_OR_EXTERNAL;
+ break;
+ }
+ /* Order matters - don't enable thermal protection for the wrong source. */
+ if (protection) {
+ PHM_WRITE_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC, CG_THERMAL_CTRL,
+ DPM_EVENT_SRC, src);
+ PHM_WRITE_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC, GENERAL_PWRMGT,
+ THERMAL_PROTECTION_DIS,
+ !phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_ThermalController));
+ } else
+ PHM_WRITE_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC, GENERAL_PWRMGT,
+ THERMAL_PROTECTION_DIS, 1);
+}
+
+static int polaris10_enable_auto_throttle_source(struct pp_hwmgr *hwmgr,
+ PHM_AutoThrottleSource source)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ if (!(data->active_auto_throttle_sources & (1 << source))) {
+ data->active_auto_throttle_sources |= 1 << source;
+ polaris10_set_dpm_event_sources(hwmgr, data->active_auto_throttle_sources);
+ }
+ return 0;
+}
+
+static int polaris10_enable_thermal_auto_throttle(struct pp_hwmgr *hwmgr)
+{
+ return polaris10_enable_auto_throttle_source(hwmgr, PHM_AutoThrottleSource_Thermal);
+}
+
+int polaris10_pcie_performance_request(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ data->pcie_performance_request = true;
+
+ return 0;
+}
+
+int polaris10_enable_dpm_tasks(struct pp_hwmgr *hwmgr)
+{
+ int tmp_result, result = 0;
+ tmp_result = (!polaris10_is_dpm_running(hwmgr)) ? 0 : -1;
+ PP_ASSERT_WITH_CODE(result == 0,
+ "DPM is already running right now, no need to enable DPM!",
+ return 0);
+
+ if (polaris10_voltage_control(hwmgr)) {
+ tmp_result = polaris10_enable_voltage_control(hwmgr);
+ PP_ASSERT_WITH_CODE(tmp_result == 0,
+ "Failed to enable voltage control!",
+ result = tmp_result);
+
+ tmp_result = polaris10_construct_voltage_tables(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to contruct voltage tables!",
+ result = tmp_result);
+ }
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_EngineSpreadSpectrumSupport))
+ PHM_WRITE_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ GENERAL_PWRMGT, DYN_SPREAD_SPECTRUM_EN, 1);
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_ThermalController))
+ PHM_WRITE_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ GENERAL_PWRMGT, THERMAL_PROTECTION_DIS, 0);
+
+ tmp_result = polaris10_program_static_screen_threshold_parameters(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to program static screen threshold parameters!",
+ result = tmp_result);
+
+ tmp_result = polaris10_enable_display_gap(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to enable display gap!", result = tmp_result);
+
+ tmp_result = polaris10_program_voting_clients(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to program voting clients!", result = tmp_result);
+
+ tmp_result = polaris10_process_firmware_header(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to process firmware header!", result = tmp_result);
+
+ tmp_result = polaris10_initial_switch_from_arbf0_to_f1(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to initialize switch from ArbF0 to F1!",
+ result = tmp_result);
+
+ tmp_result = polaris10_init_smc_table(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to initialize SMC table!", result = tmp_result);
+
+ tmp_result = polaris10_init_arb_table_index(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to initialize ARB table index!", result = tmp_result);
+
+ tmp_result = polaris10_populate_pm_fuses(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to populate PM fuses!", result = tmp_result);
+
+ tmp_result = polaris10_enable_vrhot_gpio_interrupt(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to enable VR hot GPIO interrupt!", result = tmp_result);
+
+ tmp_result = polaris10_enable_sclk_control(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to enable SCLK control!", result = tmp_result);
+
+ tmp_result = polaris10_enable_smc_voltage_controller(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to enable voltage control!", result = tmp_result);
+
+ tmp_result = polaris10_enable_ulv(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to enable ULV!", result = tmp_result);
+
+ tmp_result = polaris10_enable_deep_sleep_master_switch(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to enable deep sleep master switch!", result = tmp_result);
+
+ tmp_result = polaris10_start_dpm(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to start DPM!", result = tmp_result);
+
+ tmp_result = polaris10_enable_smc_cac(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to enable SMC CAC!", result = tmp_result);
+
+ tmp_result = polaris10_enable_power_containment(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to enable power containment!", result = tmp_result);
+
+ tmp_result = polaris10_power_control_set_level(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to power control set level!", result = tmp_result);
+
+ tmp_result = polaris10_enable_thermal_auto_throttle(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to enable thermal auto throttle!", result = tmp_result);
+
+ tmp_result = polaris10_pcie_performance_request(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "pcie performance request failed!", result = tmp_result);
+
+ return result;
+}
+
+int polaris10_disable_dpm_tasks(struct pp_hwmgr *hwmgr)
+{
+
+ return 0;
+}
+
+int polaris10_reset_asic_tasks(struct pp_hwmgr *hwmgr)
+{
+
+ return 0;
+}
+
+int polaris10_hwmgr_backend_fini(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ if (data->soft_pp_table) {
+ kfree(data->soft_pp_table);
+ data->soft_pp_table = NULL;
+ }
+
+ return phm_hwmgr_backend_fini(hwmgr);
+}
+
+int polaris10_set_features_platform_caps(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ phm_cap_set(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_SclkDeepSleep);
+
+ phm_cap_set(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_DynamicPatchPowerState);
+
+ if (data->mvdd_control == POLARIS10_VOLTAGE_CONTROL_NONE)
+ phm_cap_unset(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_EnableMVDDControl);
+
+ if (data->vddci_control == POLARIS10_VOLTAGE_CONTROL_NONE)
+ phm_cap_unset(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_ControlVDDCI);
+
+ phm_cap_set(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_TablelessHardwareInterface);
+
+ phm_cap_set(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_EnableSMU7ThermalManagement);
+
+ phm_cap_set(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_DynamicPowerManagement);
+
+ phm_cap_set(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_UnTabledHardwareInterface);
+
+ phm_cap_set(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_TablelessHardwareInterface);
+
+ phm_cap_set(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_SMC);
+
+ phm_cap_set(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_NonABMSupportInPPLib);
+
+ phm_cap_set(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_DynamicUVDState);
+
+ /* power tune caps Assume disabled */
+ phm_cap_unset(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_SQRamping);
+ phm_cap_unset(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_DBRamping);
+ phm_cap_unset(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_TDRamping);
+ phm_cap_unset(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_TCPRamping);
+
+ phm_cap_set(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_PowerContainment);
+ phm_cap_set(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_CAC);
+
+ phm_cap_set(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_RegulatorHot);
+
+ phm_cap_set(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_AutomaticDCTransition);
+
+ phm_cap_set(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_ODFuzzyFanControlSupport);
+
+ phm_cap_set(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_FanSpeedInTableIsRPM);
+ if (hwmgr->chip_id == CHIP_POLARIS11)
+ phm_cap_set(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_SPLLShutdownSupport);
+ return 0;
+}
+
+static void polaris10_init_dpm_defaults(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ polaris10_initialize_power_tune_defaults(hwmgr);
+
+ data->pcie_gen_performance.max = PP_PCIEGen1;
+ data->pcie_gen_performance.min = PP_PCIEGen3;
+ data->pcie_gen_power_saving.max = PP_PCIEGen1;
+ data->pcie_gen_power_saving.min = PP_PCIEGen3;
+ data->pcie_lane_performance.max = 0;
+ data->pcie_lane_performance.min = 16;
+ data->pcie_lane_power_saving.max = 0;
+ data->pcie_lane_power_saving.min = 16;
+}
+
+/**
+* Get Leakage VDDC based on leakage ID.
+*
+* @param hwmgr the address of the powerplay hardware manager.
+* @return always 0
+*/
+static int polaris10_get_evv_voltages(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ uint16_t vv_id;
+ uint16_t vddc = 0;
+ uint16_t i, j;
+ uint32_t sclk = 0;
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)hwmgr->pptable;
+ struct phm_ppt_v1_clock_voltage_dependency_table *sclk_table =
+ table_info->vdd_dep_on_sclk;
+ int result;
+
+ for (i = 0; i < POLARIS10_MAX_LEAKAGE_COUNT; i++) {
+ vv_id = ATOM_VIRTUAL_VOLTAGE_ID0 + i;
+ if (!phm_get_sclk_for_voltage_evv(hwmgr,
+ table_info->vddc_lookup_table, vv_id, &sclk)) {
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_ClockStretcher)) {
+ for (j = 1; j < sclk_table->count; j++) {
+ if (sclk_table->entries[j].clk == sclk &&
+ sclk_table->entries[j].cks_enable == 0) {
+ sclk += 5000;
+ break;
+ }
+ }
+ }
+
+
+ PP_ASSERT_WITH_CODE(0 == atomctrl_get_voltage_evv_on_sclk_ai(hwmgr,
+ VOLTAGE_TYPE_VDDC, sclk, vv_id, &vddc),
+ "Error retrieving EVV voltage value!",
+ continue);
+
+
+ /* need to make sure vddc is less than 2v or else, it could burn the ASIC. */
+ PP_ASSERT_WITH_CODE((vddc < 2000 && vddc != 0),
+ "Invalid VDDC value", result = -EINVAL;);
+
+ /* the voltage should not be zero nor equal to leakage ID */
+ if (vddc != 0 && vddc != vv_id) {
+ data->vddc_leakage.actual_voltage[data->vddc_leakage.count] = (uint16_t)(vddc/100);
+ data->vddc_leakage.leakage_id[data->vddc_leakage.count] = vv_id;
+ data->vddc_leakage.count++;
+ }
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * Change virtual leakage voltage to actual value.
+ *
+ * @param hwmgr the address of the powerplay hardware manager.
+ * @param pointer to changing voltage
+ * @param pointer to leakage table
+ */
+static void polaris10_patch_with_vdd_leakage(struct pp_hwmgr *hwmgr,
+ uint16_t *voltage, struct polaris10_leakage_voltage *leakage_table)
+{
+ uint32_t index;
+
+ /* search for leakage voltage ID 0xff01 ~ 0xff08 */
+ for (index = 0; index < leakage_table->count; index++) {
+ /* if this voltage matches a leakage voltage ID */
+ /* patch with actual leakage voltage */
+ if (leakage_table->leakage_id[index] == *voltage) {
+ *voltage = leakage_table->actual_voltage[index];
+ break;
+ }
+ }
+
+ if (*voltage > ATOM_VIRTUAL_VOLTAGE_ID0)
+ printk(KERN_ERR "Voltage value looks like a Leakage ID but it's not patched \n");
+}
+
+/**
+* Patch voltage lookup table by EVV leakages.
+*
+* @param hwmgr the address of the powerplay hardware manager.
+* @param pointer to voltage lookup table
+* @param pointer to leakage table
+* @return always 0
+*/
+static int polaris10_patch_lookup_table_with_leakage(struct pp_hwmgr *hwmgr,
+ phm_ppt_v1_voltage_lookup_table *lookup_table,
+ struct polaris10_leakage_voltage *leakage_table)
+{
+ uint32_t i;
+
+ for (i = 0; i < lookup_table->count; i++)
+ polaris10_patch_with_vdd_leakage(hwmgr,
+ &lookup_table->entries[i].us_vdd, leakage_table);
+
+ return 0;
+}
+
+static int polaris10_patch_clock_voltage_limits_with_vddc_leakage(
+ struct pp_hwmgr *hwmgr, struct polaris10_leakage_voltage *leakage_table,
+ uint16_t *vddc)
+{
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+ polaris10_patch_with_vdd_leakage(hwmgr, (uint16_t *)vddc, leakage_table);
+ hwmgr->dyn_state.max_clock_voltage_on_dc.vddc =
+ table_info->max_clock_voltage_on_dc.vddc;
+ return 0;
+}
+
+static int polaris10_patch_voltage_dependency_tables_with_lookup_table(
+ struct pp_hwmgr *hwmgr)
+{
+ uint8_t entryId;
+ uint8_t voltageId;
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+
+ struct phm_ppt_v1_clock_voltage_dependency_table *sclk_table =
+ table_info->vdd_dep_on_sclk;
+ struct phm_ppt_v1_clock_voltage_dependency_table *mclk_table =
+ table_info->vdd_dep_on_mclk;
+ struct phm_ppt_v1_mm_clock_voltage_dependency_table *mm_table =
+ table_info->mm_dep_table;
+
+ for (entryId = 0; entryId < sclk_table->count; ++entryId) {
+ voltageId = sclk_table->entries[entryId].vddInd;
+ sclk_table->entries[entryId].vddc =
+ table_info->vddc_lookup_table->entries[voltageId].us_vdd;
+ }
+
+ for (entryId = 0; entryId < mclk_table->count; ++entryId) {
+ voltageId = mclk_table->entries[entryId].vddInd;
+ mclk_table->entries[entryId].vddc =
+ table_info->vddc_lookup_table->entries[voltageId].us_vdd;
+ }
+
+ for (entryId = 0; entryId < mm_table->count; ++entryId) {
+ voltageId = mm_table->entries[entryId].vddcInd;
+ mm_table->entries[entryId].vddc =
+ table_info->vddc_lookup_table->entries[voltageId].us_vdd;
+ }
+
+ return 0;
+
+}
+
+static int polaris10_calc_voltage_dependency_tables(struct pp_hwmgr *hwmgr)
+{
+ /* Need to determine if we need calculated voltage. */
+ return 0;
+}
+
+static int polaris10_calc_mm_voltage_dependency_table(struct pp_hwmgr *hwmgr)
+{
+ /* Need to determine if we need calculated voltage from mm table. */
+ return 0;
+}
+
+static int polaris10_sort_lookup_table(struct pp_hwmgr *hwmgr,
+ struct phm_ppt_v1_voltage_lookup_table *lookup_table)
+{
+ uint32_t table_size, i, j;
+ struct phm_ppt_v1_voltage_lookup_record tmp_voltage_lookup_record;
+ table_size = lookup_table->count;
+
+ PP_ASSERT_WITH_CODE(0 != lookup_table->count,
+ "Lookup table is empty", return -EINVAL);
+
+ /* Sorting voltages */
+ for (i = 0; i < table_size - 1; i++) {
+ for (j = i + 1; j > 0; j--) {
+ if (lookup_table->entries[j].us_vdd <
+ lookup_table->entries[j - 1].us_vdd) {
+ tmp_voltage_lookup_record = lookup_table->entries[j - 1];
+ lookup_table->entries[j - 1] = lookup_table->entries[j];
+ lookup_table->entries[j] = tmp_voltage_lookup_record;
+ }
+ }
+ }
+
+ return 0;
+}
+
+static int polaris10_complete_dependency_tables(struct pp_hwmgr *hwmgr)
+{
+ int result = 0;
+ int tmp_result;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+
+ tmp_result = polaris10_patch_lookup_table_with_leakage(hwmgr,
+ table_info->vddc_lookup_table, &(data->vddc_leakage));
+ if (tmp_result)
+ result = tmp_result;
+
+ tmp_result = polaris10_patch_clock_voltage_limits_with_vddc_leakage(hwmgr,
+ &(data->vddc_leakage), &table_info->max_clock_voltage_on_dc.vddc);
+ if (tmp_result)
+ result = tmp_result;
+
+ tmp_result = polaris10_patch_voltage_dependency_tables_with_lookup_table(hwmgr);
+ if (tmp_result)
+ result = tmp_result;
+
+ tmp_result = polaris10_calc_voltage_dependency_tables(hwmgr);
+ if (tmp_result)
+ result = tmp_result;
+
+ tmp_result = polaris10_calc_mm_voltage_dependency_table(hwmgr);
+ if (tmp_result)
+ result = tmp_result;
+
+ tmp_result = polaris10_sort_lookup_table(hwmgr, table_info->vddc_lookup_table);
+ if (tmp_result)
+ result = tmp_result;
+
+ return result;
+}
+
+static int polaris10_set_private_data_based_on_pptable(struct pp_hwmgr *hwmgr)
+{
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+
+ struct phm_ppt_v1_clock_voltage_dependency_table *allowed_sclk_vdd_table =
+ table_info->vdd_dep_on_sclk;
+ struct phm_ppt_v1_clock_voltage_dependency_table *allowed_mclk_vdd_table =
+ table_info->vdd_dep_on_mclk;
+
+ PP_ASSERT_WITH_CODE(allowed_sclk_vdd_table != NULL,
+ "VDD dependency on SCLK table is missing. \
+ This table is mandatory", return -EINVAL);
+ PP_ASSERT_WITH_CODE(allowed_sclk_vdd_table->count >= 1,
+ "VDD dependency on SCLK table has to have is missing. \
+ This table is mandatory", return -EINVAL);
+
+ PP_ASSERT_WITH_CODE(allowed_mclk_vdd_table != NULL,
+ "VDD dependency on MCLK table is missing. \
+ This table is mandatory", return -EINVAL);
+ PP_ASSERT_WITH_CODE(allowed_mclk_vdd_table->count >= 1,
+ "VDD dependency on MCLK table has to have is missing. \
+ This table is mandatory", return -EINVAL);
+
+ table_info->max_clock_voltage_on_ac.sclk =
+ allowed_sclk_vdd_table->entries[allowed_sclk_vdd_table->count - 1].clk;
+ table_info->max_clock_voltage_on_ac.mclk =
+ allowed_mclk_vdd_table->entries[allowed_mclk_vdd_table->count - 1].clk;
+ table_info->max_clock_voltage_on_ac.vddc =
+ allowed_sclk_vdd_table->entries[allowed_sclk_vdd_table->count - 1].vddc;
+ table_info->max_clock_voltage_on_ac.vddci =
+ allowed_mclk_vdd_table->entries[allowed_mclk_vdd_table->count - 1].vddci;
+
+ hwmgr->dyn_state.max_clock_voltage_on_ac.sclk = table_info->max_clock_voltage_on_ac.sclk;
+ hwmgr->dyn_state.max_clock_voltage_on_ac.mclk = table_info->max_clock_voltage_on_ac.mclk;
+ hwmgr->dyn_state.max_clock_voltage_on_ac.vddc = table_info->max_clock_voltage_on_ac.vddc;
+ hwmgr->dyn_state.max_clock_voltage_on_ac.vddci =table_info->max_clock_voltage_on_ac.vddci;
+
+ return 0;
+}
+
+int polaris10_hwmgr_backend_init(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct pp_atomctrl_gpio_pin_assignment gpio_pin_assignment;
+ uint32_t temp_reg;
+ int result;
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+
+ data->dll_default_on = false;
+ data->sram_end = SMC_RAM_END;
+ data->mclk_dpm0_activity_target = 0xa;
+ data->disable_dpm_mask = 0xFF;
+ data->static_screen_threshold = PPPOLARIS10_STATICSCREENTHRESHOLD_DFLT;
+ data->static_screen_threshold_unit = PPPOLARIS10_STATICSCREENTHRESHOLD_DFLT;
+ data->activity_target[0] = PPPOLARIS10_TARGETACTIVITY_DFLT;
+ data->activity_target[1] = PPPOLARIS10_TARGETACTIVITY_DFLT;
+ data->activity_target[2] = PPPOLARIS10_TARGETACTIVITY_DFLT;
+ data->activity_target[3] = PPPOLARIS10_TARGETACTIVITY_DFLT;
+ data->activity_target[4] = PPPOLARIS10_TARGETACTIVITY_DFLT;
+ data->activity_target[5] = PPPOLARIS10_TARGETACTIVITY_DFLT;
+ data->activity_target[6] = PPPOLARIS10_TARGETACTIVITY_DFLT;
+ data->activity_target[7] = PPPOLARIS10_TARGETACTIVITY_DFLT;
+
+ data->voting_rights_clients0 = PPPOLARIS10_VOTINGRIGHTSCLIENTS_DFLT0;
+ data->voting_rights_clients1 = PPPOLARIS10_VOTINGRIGHTSCLIENTS_DFLT1;
+ data->voting_rights_clients2 = PPPOLARIS10_VOTINGRIGHTSCLIENTS_DFLT2;
+ data->voting_rights_clients3 = PPPOLARIS10_VOTINGRIGHTSCLIENTS_DFLT3;
+ data->voting_rights_clients4 = PPPOLARIS10_VOTINGRIGHTSCLIENTS_DFLT4;
+ data->voting_rights_clients5 = PPPOLARIS10_VOTINGRIGHTSCLIENTS_DFLT5;
+ data->voting_rights_clients6 = PPPOLARIS10_VOTINGRIGHTSCLIENTS_DFLT6;
+ data->voting_rights_clients7 = PPPOLARIS10_VOTINGRIGHTSCLIENTS_DFLT7;
+
+ data->vddc_vddci_delta = VDDC_VDDCI_DELTA;
+
+ data->mclk_activity_target = PPPOLARIS10_MCLK_TARGETACTIVITY_DFLT;
+
+ /* need to set voltage control types before EVV patching */
+ data->voltage_control = POLARIS10_VOLTAGE_CONTROL_NONE;
+ data->vddci_control = POLARIS10_VOLTAGE_CONTROL_NONE;
+ data->mvdd_control = POLARIS10_VOLTAGE_CONTROL_NONE;
+
+ if (atomctrl_is_voltage_controled_by_gpio_v3(hwmgr,
+ VOLTAGE_TYPE_VDDC, VOLTAGE_OBJ_SVID2))
+ data->voltage_control = POLARIS10_VOLTAGE_CONTROL_BY_SVID2;
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_EnableMVDDControl)) {
+ if (atomctrl_is_voltage_controled_by_gpio_v3(hwmgr,
+ VOLTAGE_TYPE_MVDDC, VOLTAGE_OBJ_GPIO_LUT))
+ data->mvdd_control = POLARIS10_VOLTAGE_CONTROL_BY_GPIO;
+ else if (atomctrl_is_voltage_controled_by_gpio_v3(hwmgr,
+ VOLTAGE_TYPE_MVDDC, VOLTAGE_OBJ_SVID2))
+ data->mvdd_control = POLARIS10_VOLTAGE_CONTROL_BY_SVID2;
+ }
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_ControlVDDCI)) {
+ if (atomctrl_is_voltage_controled_by_gpio_v3(hwmgr,
+ VOLTAGE_TYPE_VDDCI, VOLTAGE_OBJ_GPIO_LUT))
+ data->vddci_control = POLARIS10_VOLTAGE_CONTROL_BY_GPIO;
+ else if (atomctrl_is_voltage_controled_by_gpio_v3(hwmgr,
+ VOLTAGE_TYPE_VDDCI, VOLTAGE_OBJ_SVID2))
+ data->vddci_control = POLARIS10_VOLTAGE_CONTROL_BY_SVID2;
+ }
+
+ polaris10_set_features_platform_caps(hwmgr);
+
+ polaris10_init_dpm_defaults(hwmgr);
+
+ /* Get leakage voltage based on leakage ID. */
+ result = polaris10_get_evv_voltages(hwmgr);
+
+ if (result) {
+ printk("Get EVV Voltage Failed. Abort Driver loading!\n");
+ return -1;
+ }
+
+ polaris10_complete_dependency_tables(hwmgr);
+ polaris10_set_private_data_based_on_pptable(hwmgr);
+
+ /* Initalize Dynamic State Adjustment Rule Settings */
+ result = phm_initializa_dynamic_state_adjustment_rule_settings(hwmgr);
+
+ if (0 == result) {
+ struct cgs_system_info sys_info = {0};
+
+ data->is_tlu_enabled = 0;
+
+ hwmgr->platform_descriptor.hardwareActivityPerformanceLevels =
+ POLARIS10_MAX_HARDWARE_POWERLEVELS;
+ hwmgr->platform_descriptor.hardwarePerformanceLevels = 2;
+ hwmgr->platform_descriptor.minimumClocksReductionPercentage = 50;
+
+
+ if (atomctrl_get_pp_assign_pin(hwmgr, VDDC_PCC_GPIO_PINID, &gpio_pin_assignment)) {
+ temp_reg = cgs_read_ind_register(hwmgr->device, CGS_IND_REG__SMC, ixCNB_PWRMGT_CNTL);
+ switch (gpio_pin_assignment.uc_gpio_pin_bit_shift) {
+ case 0:
+ temp_reg = PHM_SET_FIELD(temp_reg, CNB_PWRMGT_CNTL, GNB_SLOW_MODE, 0x1);
+ break;
+ case 1:
+ temp_reg = PHM_SET_FIELD(temp_reg, CNB_PWRMGT_CNTL, GNB_SLOW_MODE, 0x2);
+ break;
+ case 2:
+ temp_reg = PHM_SET_FIELD(temp_reg, CNB_PWRMGT_CNTL, GNB_SLOW, 0x1);
+ break;
+ case 3:
+ temp_reg = PHM_SET_FIELD(temp_reg, CNB_PWRMGT_CNTL, FORCE_NB_PS1, 0x1);
+ break;
+ case 4:
+ temp_reg = PHM_SET_FIELD(temp_reg, CNB_PWRMGT_CNTL, DPM_ENABLED, 0x1);
+ break;
+ default:
+ PP_ASSERT_WITH_CODE(0,
+ "Failed to setup PCC HW register! Wrong GPIO assigned for VDDC_PCC_GPIO_PINID!",
+ );
+ break;
+ }
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC, ixCNB_PWRMGT_CNTL, temp_reg);
+ }
+
+ if (table_info->cac_dtp_table->usDefaultTargetOperatingTemp != 0 &&
+ hwmgr->thermal_controller.advanceFanControlParameters.ucFanControlMode) {
+ hwmgr->thermal_controller.advanceFanControlParameters.usFanPWMMinLimit =
+ (uint16_t)hwmgr->thermal_controller.advanceFanControlParameters.ucMinimumPWMLimit;
+
+ hwmgr->thermal_controller.advanceFanControlParameters.usFanPWMMaxLimit =
+ (uint16_t)hwmgr->thermal_controller.advanceFanControlParameters.usDefaultMaxFanPWM;
+
+ hwmgr->thermal_controller.advanceFanControlParameters.usFanPWMStep = 1;
+
+ hwmgr->thermal_controller.advanceFanControlParameters.usFanRPMMaxLimit = 100;
+
+ hwmgr->thermal_controller.advanceFanControlParameters.usFanRPMMinLimit =
+ (uint16_t)hwmgr->thermal_controller.advanceFanControlParameters.ucMinimumPWMLimit;
+
+ hwmgr->thermal_controller.advanceFanControlParameters.usFanRPMStep = 1;
+
+ table_info->cac_dtp_table->usDefaultTargetOperatingTemp = (table_info->cac_dtp_table->usDefaultTargetOperatingTemp >= 50) ?
+ (table_info->cac_dtp_table->usDefaultTargetOperatingTemp -50) : 0;
+
+ table_info->cac_dtp_table->usOperatingTempMaxLimit = table_info->cac_dtp_table->usDefaultTargetOperatingTemp;
+ table_info->cac_dtp_table->usOperatingTempStep = 1;
+ table_info->cac_dtp_table->usOperatingTempHyst = 1;
+
+ hwmgr->thermal_controller.advanceFanControlParameters.usMaxFanPWM =
+ hwmgr->thermal_controller.advanceFanControlParameters.usDefaultMaxFanPWM;
+
+ hwmgr->thermal_controller.advanceFanControlParameters.usMaxFanRPM =
+ hwmgr->thermal_controller.advanceFanControlParameters.usDefaultMaxFanRPM;
+
+ hwmgr->dyn_state.cac_dtp_table->usOperatingTempMinLimit =
+ table_info->cac_dtp_table->usOperatingTempMinLimit;
+
+ hwmgr->dyn_state.cac_dtp_table->usOperatingTempMaxLimit =
+ table_info->cac_dtp_table->usOperatingTempMaxLimit;
+
+ hwmgr->dyn_state.cac_dtp_table->usDefaultTargetOperatingTemp =
+ table_info->cac_dtp_table->usDefaultTargetOperatingTemp;
+
+ hwmgr->dyn_state.cac_dtp_table->usOperatingTempStep =
+ table_info->cac_dtp_table->usOperatingTempStep;
+
+ hwmgr->dyn_state.cac_dtp_table->usTargetOperatingTemp =
+ table_info->cac_dtp_table->usTargetOperatingTemp;
+ }
+
+ sys_info.size = sizeof(struct cgs_system_info);
+ sys_info.info_id = CGS_SYSTEM_INFO_PCIE_GEN_INFO;
+ result = cgs_query_system_info(hwmgr->device, &sys_info);
+ if (result)
+ data->pcie_gen_cap = 0x30007;
+ else
+ data->pcie_gen_cap = (uint32_t)sys_info.value;
+ if (data->pcie_gen_cap & CAIL_PCIE_LINK_SPEED_SUPPORT_GEN3)
+ data->pcie_spc_cap = 20;
+ sys_info.size = sizeof(struct cgs_system_info);
+ sys_info.info_id = CGS_SYSTEM_INFO_PCIE_MLW;
+ result = cgs_query_system_info(hwmgr->device, &sys_info);
+ if (result)
+ data->pcie_lane_cap = 0x2f0000;
+ else
+ data->pcie_lane_cap = (uint32_t)sys_info.value;
+
+ hwmgr->platform_descriptor.vbiosInterruptId = 0x20000400; /* IRQ_SOURCE1_SW_INT */
+/* The true clock step depends on the frequency, typically 4.5 or 9 MHz. Here we use 5. */
+ hwmgr->platform_descriptor.clockStep.engineClock = 500;
+ hwmgr->platform_descriptor.clockStep.memoryClock = 500;
+ } else {
+ /* Ignore return value in here, we are cleaning up a mess. */
+ polaris10_hwmgr_backend_fini(hwmgr);
+ }
+
+ return 0;
+}
+
+static int polaris10_force_dpm_highest(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ uint32_t level, tmp;
+
+ if (!data->pcie_dpm_key_disabled) {
+ if (data->dpm_level_enable_mask.pcie_dpm_enable_mask) {
+ level = 0;
+ tmp = data->dpm_level_enable_mask.pcie_dpm_enable_mask;
+ while (tmp >>= 1)
+ level++;
+
+ if (level)
+ smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_PCIeDPM_ForceLevel, level);
+ }
+ }
+
+ if (!data->sclk_dpm_key_disabled) {
+ if (data->dpm_level_enable_mask.sclk_dpm_enable_mask) {
+ level = 0;
+ tmp = data->dpm_level_enable_mask.sclk_dpm_enable_mask;
+ while (tmp >>= 1)
+ level++;
+
+ if (level)
+ smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_SCLKDPM_SetEnabledMask,
+ (1 << level));
+ }
+ }
+
+ if (!data->mclk_dpm_key_disabled) {
+ if (data->dpm_level_enable_mask.mclk_dpm_enable_mask) {
+ level = 0;
+ tmp = data->dpm_level_enable_mask.mclk_dpm_enable_mask;
+ while (tmp >>= 1)
+ level++;
+
+ if (level)
+ smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_MCLKDPM_SetEnabledMask,
+ (1 << level));
+ }
+ }
+
+ return 0;
+}
+
+static int polaris10_upload_dpm_level_enable_mask(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ phm_apply_dal_min_voltage_request(hwmgr);
+
+ if (!data->sclk_dpm_key_disabled) {
+ if (data->dpm_level_enable_mask.sclk_dpm_enable_mask)
+ smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_SCLKDPM_SetEnabledMask,
+ data->dpm_level_enable_mask.sclk_dpm_enable_mask);
+ }
+
+ if (!data->mclk_dpm_key_disabled) {
+ if (data->dpm_level_enable_mask.mclk_dpm_enable_mask)
+ smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_MCLKDPM_SetEnabledMask,
+ data->dpm_level_enable_mask.mclk_dpm_enable_mask);
+ }
+
+ return 0;
+}
+
+static int polaris10_unforce_dpm_levels(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ if (!polaris10_is_dpm_running(hwmgr))
+ return -EINVAL;
+
+ if (!data->pcie_dpm_key_disabled) {
+ smum_send_msg_to_smc(hwmgr->smumgr,
+ PPSMC_MSG_PCIeDPM_UnForceLevel);
+ }
+
+ return polaris10_upload_dpm_level_enable_mask(hwmgr);
+}
+
+static int polaris10_force_dpm_lowest(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data =
+ (struct polaris10_hwmgr *)(hwmgr->backend);
+ uint32_t level;
+
+ if (!data->sclk_dpm_key_disabled)
+ if (data->dpm_level_enable_mask.sclk_dpm_enable_mask) {
+ level = phm_get_lowest_enabled_level(hwmgr,
+ data->dpm_level_enable_mask.sclk_dpm_enable_mask);
+ smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_SCLKDPM_SetEnabledMask,
+ (1 << level));
+
+ }
+
+ if (!data->mclk_dpm_key_disabled) {
+ if (data->dpm_level_enable_mask.mclk_dpm_enable_mask) {
+ level = phm_get_lowest_enabled_level(hwmgr,
+ data->dpm_level_enable_mask.mclk_dpm_enable_mask);
+ smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_MCLKDPM_SetEnabledMask,
+ (1 << level));
+ }
+ }
+
+ if (!data->pcie_dpm_key_disabled) {
+ if (data->dpm_level_enable_mask.pcie_dpm_enable_mask) {
+ level = phm_get_lowest_enabled_level(hwmgr,
+ data->dpm_level_enable_mask.pcie_dpm_enable_mask);
+ smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_PCIeDPM_ForceLevel,
+ (level));
+ }
+ }
+
+ return 0;
+
+}
+static int polaris10_force_dpm_level(struct pp_hwmgr *hwmgr,
+ enum amd_dpm_forced_level level)
+{
+ int ret = 0;
+
+ switch (level) {
+ case AMD_DPM_FORCED_LEVEL_HIGH:
+ ret = polaris10_force_dpm_highest(hwmgr);
+ if (ret)
+ return ret;
+ break;
+ case AMD_DPM_FORCED_LEVEL_LOW:
+ ret = polaris10_force_dpm_lowest(hwmgr);
+ if (ret)
+ return ret;
+ break;
+ case AMD_DPM_FORCED_LEVEL_AUTO:
+ ret = polaris10_unforce_dpm_levels(hwmgr);
+ if (ret)
+ return ret;
+ break;
+ default:
+ break;
+ }
+
+ hwmgr->dpm_level = level;
+
+ return ret;
+}
+
+static int polaris10_get_power_state_size(struct pp_hwmgr *hwmgr)
+{
+ return sizeof(struct polaris10_power_state);
+}
+
+
+static int polaris10_apply_state_adjust_rules(struct pp_hwmgr *hwmgr,
+ struct pp_power_state *request_ps,
+ const struct pp_power_state *current_ps)
+{
+
+ struct polaris10_power_state *polaris10_ps =
+ cast_phw_polaris10_power_state(&request_ps->hardware);
+ uint32_t sclk;
+ uint32_t mclk;
+ struct PP_Clocks minimum_clocks = {0};
+ bool disable_mclk_switching;
+ bool disable_mclk_switching_for_frame_lock;
+ struct cgs_display_info info = {0};
+ const struct phm_clock_and_voltage_limits *max_limits;
+ uint32_t i;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+ int32_t count;
+ int32_t stable_pstate_sclk = 0, stable_pstate_mclk = 0;
+
+ data->battery_state = (PP_StateUILabel_Battery ==
+ request_ps->classification.ui_label);
+
+ PP_ASSERT_WITH_CODE(polaris10_ps->performance_level_count == 2,
+ "VI should always have 2 performance levels",
+ );
+
+ max_limits = (PP_PowerSource_AC == hwmgr->power_source) ?
+ &(hwmgr->dyn_state.max_clock_voltage_on_ac) :
+ &(hwmgr->dyn_state.max_clock_voltage_on_dc);
+
+ /* Cap clock DPM tables at DC MAX if it is in DC. */
+ if (PP_PowerSource_DC == hwmgr->power_source) {
+ for (i = 0; i < polaris10_ps->performance_level_count; i++) {
+ if (polaris10_ps->performance_levels[i].memory_clock > max_limits->mclk)
+ polaris10_ps->performance_levels[i].memory_clock = max_limits->mclk;
+ if (polaris10_ps->performance_levels[i].engine_clock > max_limits->sclk)
+ polaris10_ps->performance_levels[i].engine_clock = max_limits->sclk;
+ }
+ }
+
+ polaris10_ps->vce_clks.evclk = hwmgr->vce_arbiter.evclk;
+ polaris10_ps->vce_clks.ecclk = hwmgr->vce_arbiter.ecclk;
+
+ cgs_get_active_displays_info(hwmgr->device, &info);
+
+ /*TO DO result = PHM_CheckVBlankTime(hwmgr, &vblankTooShort);*/
+
+ /* TO DO GetMinClockSettings(hwmgr->pPECI, &minimum_clocks); */
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_StablePState)) {
+ max_limits = &(hwmgr->dyn_state.max_clock_voltage_on_ac);
+ stable_pstate_sclk = (max_limits->sclk * 75) / 100;
+
+ for (count = table_info->vdd_dep_on_sclk->count - 1;
+ count >= 0; count--) {
+ if (stable_pstate_sclk >=
+ table_info->vdd_dep_on_sclk->entries[count].clk) {
+ stable_pstate_sclk =
+ table_info->vdd_dep_on_sclk->entries[count].clk;
+ break;
+ }
+ }
+
+ if (count < 0)
+ stable_pstate_sclk = table_info->vdd_dep_on_sclk->entries[0].clk;
+
+ stable_pstate_mclk = max_limits->mclk;
+
+ minimum_clocks.engineClock = stable_pstate_sclk;
+ minimum_clocks.memoryClock = stable_pstate_mclk;
+ }
+
+ if (minimum_clocks.engineClock < hwmgr->gfx_arbiter.sclk)
+ minimum_clocks.engineClock = hwmgr->gfx_arbiter.sclk;
+
+ if (minimum_clocks.memoryClock < hwmgr->gfx_arbiter.mclk)
+ minimum_clocks.memoryClock = hwmgr->gfx_arbiter.mclk;
+
+ polaris10_ps->sclk_threshold = hwmgr->gfx_arbiter.sclk_threshold;
+
+ if (0 != hwmgr->gfx_arbiter.sclk_over_drive) {
+ PP_ASSERT_WITH_CODE((hwmgr->gfx_arbiter.sclk_over_drive <=
+ hwmgr->platform_descriptor.overdriveLimit.engineClock),
+ "Overdrive sclk exceeds limit",
+ hwmgr->gfx_arbiter.sclk_over_drive =
+ hwmgr->platform_descriptor.overdriveLimit.engineClock);
+
+ if (hwmgr->gfx_arbiter.sclk_over_drive >= hwmgr->gfx_arbiter.sclk)
+ polaris10_ps->performance_levels[1].engine_clock =
+ hwmgr->gfx_arbiter.sclk_over_drive;
+ }
+
+ if (0 != hwmgr->gfx_arbiter.mclk_over_drive) {
+ PP_ASSERT_WITH_CODE((hwmgr->gfx_arbiter.mclk_over_drive <=
+ hwmgr->platform_descriptor.overdriveLimit.memoryClock),
+ "Overdrive mclk exceeds limit",
+ hwmgr->gfx_arbiter.mclk_over_drive =
+ hwmgr->platform_descriptor.overdriveLimit.memoryClock);
+
+ if (hwmgr->gfx_arbiter.mclk_over_drive >= hwmgr->gfx_arbiter.mclk)
+ polaris10_ps->performance_levels[1].memory_clock =
+ hwmgr->gfx_arbiter.mclk_over_drive;
+ }
+
+ disable_mclk_switching_for_frame_lock = phm_cap_enabled(
+ hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_DisableMclkSwitchingForFrameLock);
+
+ disable_mclk_switching = (1 < info.display_count) ||
+ disable_mclk_switching_for_frame_lock;
+
+ sclk = polaris10_ps->performance_levels[0].engine_clock;
+ mclk = polaris10_ps->performance_levels[0].memory_clock;
+
+ if (disable_mclk_switching)
+ mclk = polaris10_ps->performance_levels
+ [polaris10_ps->performance_level_count - 1].memory_clock;
+
+ if (sclk < minimum_clocks.engineClock)
+ sclk = (minimum_clocks.engineClock > max_limits->sclk) ?
+ max_limits->sclk : minimum_clocks.engineClock;
+
+ if (mclk < minimum_clocks.memoryClock)
+ mclk = (minimum_clocks.memoryClock > max_limits->mclk) ?
+ max_limits->mclk : minimum_clocks.memoryClock;
+
+ polaris10_ps->performance_levels[0].engine_clock = sclk;
+ polaris10_ps->performance_levels[0].memory_clock = mclk;
+
+ polaris10_ps->performance_levels[1].engine_clock =
+ (polaris10_ps->performance_levels[1].engine_clock >=
+ polaris10_ps->performance_levels[0].engine_clock) ?
+ polaris10_ps->performance_levels[1].engine_clock :
+ polaris10_ps->performance_levels[0].engine_clock;
+
+ if (disable_mclk_switching) {
+ if (mclk < polaris10_ps->performance_levels[1].memory_clock)
+ mclk = polaris10_ps->performance_levels[1].memory_clock;
+
+ polaris10_ps->performance_levels[0].memory_clock = mclk;
+ polaris10_ps->performance_levels[1].memory_clock = mclk;
+ } else {
+ if (polaris10_ps->performance_levels[1].memory_clock <
+ polaris10_ps->performance_levels[0].memory_clock)
+ polaris10_ps->performance_levels[1].memory_clock =
+ polaris10_ps->performance_levels[0].memory_clock;
+ }
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_StablePState)) {
+ for (i = 0; i < polaris10_ps->performance_level_count; i++) {
+ polaris10_ps->performance_levels[i].engine_clock = stable_pstate_sclk;
+ polaris10_ps->performance_levels[i].memory_clock = stable_pstate_mclk;
+ polaris10_ps->performance_levels[i].pcie_gen = data->pcie_gen_performance.max;
+ polaris10_ps->performance_levels[i].pcie_lane = data->pcie_gen_performance.max;
+ }
+ }
+ return 0;
+}
+
+
+static int polaris10_dpm_get_mclk(struct pp_hwmgr *hwmgr, bool low)
+{
+ struct pp_power_state *ps;
+ struct polaris10_power_state *polaris10_ps;
+
+ if (hwmgr == NULL)
+ return -EINVAL;
+
+ ps = hwmgr->request_ps;
+
+ if (ps == NULL)
+ return -EINVAL;
+
+ polaris10_ps = cast_phw_polaris10_power_state(&ps->hardware);
+
+ if (low)
+ return polaris10_ps->performance_levels[0].memory_clock;
+ else
+ return polaris10_ps->performance_levels
+ [polaris10_ps->performance_level_count-1].memory_clock;
+}
+
+static int polaris10_dpm_get_sclk(struct pp_hwmgr *hwmgr, bool low)
+{
+ struct pp_power_state *ps;
+ struct polaris10_power_state *polaris10_ps;
+
+ if (hwmgr == NULL)
+ return -EINVAL;
+
+ ps = hwmgr->request_ps;
+
+ if (ps == NULL)
+ return -EINVAL;
+
+ polaris10_ps = cast_phw_polaris10_power_state(&ps->hardware);
+
+ if (low)
+ return polaris10_ps->performance_levels[0].engine_clock;
+ else
+ return polaris10_ps->performance_levels
+ [polaris10_ps->performance_level_count-1].engine_clock;
+}
+
+static int polaris10_dpm_patch_boot_state(struct pp_hwmgr *hwmgr,
+ struct pp_hw_power_state *hw_ps)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct polaris10_power_state *ps = (struct polaris10_power_state *)hw_ps;
+ ATOM_FIRMWARE_INFO_V2_2 *fw_info;
+ uint16_t size;
+ uint8_t frev, crev;
+ int index = GetIndexIntoMasterTable(DATA, FirmwareInfo);
+
+ /* First retrieve the Boot clocks and VDDC from the firmware info table.
+ * We assume here that fw_info is unchanged if this call fails.
+ */
+ fw_info = (ATOM_FIRMWARE_INFO_V2_2 *)cgs_atom_get_data_table(
+ hwmgr->device, index,
+ &size, &frev, &crev);
+ if (!fw_info)
+ /* During a test, there is no firmware info table. */
+ return 0;
+
+ /* Patch the state. */
+ data->vbios_boot_state.sclk_bootup_value =
+ le32_to_cpu(fw_info->ulDefaultEngineClock);
+ data->vbios_boot_state.mclk_bootup_value =
+ le32_to_cpu(fw_info->ulDefaultMemoryClock);
+ data->vbios_boot_state.mvdd_bootup_value =
+ le16_to_cpu(fw_info->usBootUpMVDDCVoltage);
+ data->vbios_boot_state.vddc_bootup_value =
+ le16_to_cpu(fw_info->usBootUpVDDCVoltage);
+ data->vbios_boot_state.vddci_bootup_value =
+ le16_to_cpu(fw_info->usBootUpVDDCIVoltage);
+ data->vbios_boot_state.pcie_gen_bootup_value =
+ phm_get_current_pcie_speed(hwmgr);
+
+ data->vbios_boot_state.pcie_lane_bootup_value =
+ (uint16_t)phm_get_current_pcie_lane_number(hwmgr);
+
+ /* set boot power state */
+ ps->performance_levels[0].memory_clock = data->vbios_boot_state.mclk_bootup_value;
+ ps->performance_levels[0].engine_clock = data->vbios_boot_state.sclk_bootup_value;
+ ps->performance_levels[0].pcie_gen = data->vbios_boot_state.pcie_gen_bootup_value;
+ ps->performance_levels[0].pcie_lane = data->vbios_boot_state.pcie_lane_bootup_value;
+
+ return 0;
+}
+
+static int polaris10_get_pp_table_entry_callback_func(struct pp_hwmgr *hwmgr,
+ void *state, struct pp_power_state *power_state,
+ void *pp_table, uint32_t classification_flag)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct polaris10_power_state *polaris10_power_state =
+ (struct polaris10_power_state *)(&(power_state->hardware));
+ struct polaris10_performance_level *performance_level;
+ ATOM_Tonga_State *state_entry = (ATOM_Tonga_State *)state;
+ ATOM_Tonga_POWERPLAYTABLE *powerplay_table =
+ (ATOM_Tonga_POWERPLAYTABLE *)pp_table;
+ ATOM_Tonga_SCLK_Dependency_Table *sclk_dep_table =
+ (ATOM_Tonga_SCLK_Dependency_Table *)
+ (((unsigned long)powerplay_table) +
+ le16_to_cpu(powerplay_table->usSclkDependencyTableOffset));
+ ATOM_Tonga_MCLK_Dependency_Table *mclk_dep_table =
+ (ATOM_Tonga_MCLK_Dependency_Table *)
+ (((unsigned long)powerplay_table) +
+ le16_to_cpu(powerplay_table->usMclkDependencyTableOffset));
+
+ /* The following fields are not initialized here: id orderedList allStatesList */
+ power_state->classification.ui_label =
+ (le16_to_cpu(state_entry->usClassification) &
+ ATOM_PPLIB_CLASSIFICATION_UI_MASK) >>
+ ATOM_PPLIB_CLASSIFICATION_UI_SHIFT;
+ power_state->classification.flags = classification_flag;
+ /* NOTE: There is a classification2 flag in BIOS that is not being used right now */
+
+ power_state->classification.temporary_state = false;
+ power_state->classification.to_be_deleted = false;
+
+ power_state->validation.disallowOnDC =
+ (0 != (le32_to_cpu(state_entry->ulCapsAndSettings) &
+ ATOM_Tonga_DISALLOW_ON_DC));
+
+ power_state->pcie.lanes = 0;
+
+ power_state->display.disableFrameModulation = false;
+ power_state->display.limitRefreshrate = false;
+ power_state->display.enableVariBright =
+ (0 != (le32_to_cpu(state_entry->ulCapsAndSettings) &
+ ATOM_Tonga_ENABLE_VARIBRIGHT));
+
+ power_state->validation.supportedPowerLevels = 0;
+ power_state->uvd_clocks.VCLK = 0;
+ power_state->uvd_clocks.DCLK = 0;
+ power_state->temperatures.min = 0;
+ power_state->temperatures.max = 0;
+
+ performance_level = &(polaris10_power_state->performance_levels
+ [polaris10_power_state->performance_level_count++]);
+
+ PP_ASSERT_WITH_CODE(
+ (polaris10_power_state->performance_level_count < SMU74_MAX_LEVELS_GRAPHICS),
+ "Performance levels exceeds SMC limit!",
+ return -1);
+
+ PP_ASSERT_WITH_CODE(
+ (polaris10_power_state->performance_level_count <=
+ hwmgr->platform_descriptor.hardwareActivityPerformanceLevels),
+ "Performance levels exceeds Driver limit!",
+ return -1);
+
+ /* Performance levels are arranged from low to high. */
+ performance_level->memory_clock = mclk_dep_table->entries
+ [state_entry->ucMemoryClockIndexLow].ulMclk;
+ performance_level->engine_clock = sclk_dep_table->entries
+ [state_entry->ucEngineClockIndexLow].ulSclk;
+ performance_level->pcie_gen = get_pcie_gen_support(data->pcie_gen_cap,
+ state_entry->ucPCIEGenLow);
+ performance_level->pcie_lane = get_pcie_lane_support(data->pcie_lane_cap,
+ state_entry->ucPCIELaneHigh);
+
+ performance_level = &(polaris10_power_state->performance_levels
+ [polaris10_power_state->performance_level_count++]);
+ performance_level->memory_clock = mclk_dep_table->entries
+ [state_entry->ucMemoryClockIndexHigh].ulMclk;
+ performance_level->engine_clock = sclk_dep_table->entries
+ [state_entry->ucEngineClockIndexHigh].ulSclk;
+ performance_level->pcie_gen = get_pcie_gen_support(data->pcie_gen_cap,
+ state_entry->ucPCIEGenHigh);
+ performance_level->pcie_lane = get_pcie_lane_support(data->pcie_lane_cap,
+ state_entry->ucPCIELaneHigh);
+
+ return 0;
+}
+
+static int polaris10_get_pp_table_entry(struct pp_hwmgr *hwmgr,
+ unsigned long entry_index, struct pp_power_state *state)
+{
+ int result;
+ struct polaris10_power_state *ps;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+ struct phm_ppt_v1_clock_voltage_dependency_table *dep_mclk_table =
+ table_info->vdd_dep_on_mclk;
+
+ state->hardware.magic = PHM_VIslands_Magic;
+
+ ps = (struct polaris10_power_state *)(&state->hardware);
+
+ result = tonga_get_powerplay_table_entry(hwmgr, entry_index, state,
+ polaris10_get_pp_table_entry_callback_func);
+
+ /* This is the earliest time we have all the dependency table and the VBIOS boot state
+ * as PP_Tables_GetPowerPlayTableEntry retrieves the VBIOS boot state
+ * if there is only one VDDCI/MCLK level, check if it's the same as VBIOS boot state
+ */
+ if (dep_mclk_table != NULL && dep_mclk_table->count == 1) {
+ if (dep_mclk_table->entries[0].clk !=
+ data->vbios_boot_state.mclk_bootup_value)
+ printk(KERN_ERR "Single MCLK entry VDDCI/MCLK dependency table "
+ "does not match VBIOS boot MCLK level");
+ if (dep_mclk_table->entries[0].vddci !=
+ data->vbios_boot_state.vddci_bootup_value)
+ printk(KERN_ERR "Single VDDCI entry VDDCI/MCLK dependency table "
+ "does not match VBIOS boot VDDCI level");
+ }
+
+ /* set DC compatible flag if this state supports DC */
+ if (!state->validation.disallowOnDC)
+ ps->dc_compatible = true;
+
+ if (state->classification.flags & PP_StateClassificationFlag_ACPI)
+ data->acpi_pcie_gen = ps->performance_levels[0].pcie_gen;
+
+ ps->uvd_clks.vclk = state->uvd_clocks.VCLK;
+ ps->uvd_clks.dclk = state->uvd_clocks.DCLK;
+
+ if (!result) {
+ uint32_t i;
+
+ switch (state->classification.ui_label) {
+ case PP_StateUILabel_Performance:
+ data->use_pcie_performance_levels = true;
+
+ for (i = 0; i < ps->performance_level_count; i++) {
+ if (data->pcie_gen_performance.max <
+ ps->performance_levels[i].pcie_gen)
+ data->pcie_gen_performance.max =
+ ps->performance_levels[i].pcie_gen;
+
+ if (data->pcie_gen_performance.min >
+ ps->performance_levels[i].pcie_gen)
+ data->pcie_gen_performance.min =
+ ps->performance_levels[i].pcie_gen;
+
+ if (data->pcie_lane_performance.max <
+ ps->performance_levels[i].pcie_lane)
+ data->pcie_lane_performance.max =
+ ps->performance_levels[i].pcie_lane;
+
+ if (data->pcie_lane_performance.min >
+ ps->performance_levels[i].pcie_lane)
+ data->pcie_lane_performance.min =
+ ps->performance_levels[i].pcie_lane;
+ }
+ break;
+ case PP_StateUILabel_Battery:
+ data->use_pcie_power_saving_levels = true;
+
+ for (i = 0; i < ps->performance_level_count; i++) {
+ if (data->pcie_gen_power_saving.max <
+ ps->performance_levels[i].pcie_gen)
+ data->pcie_gen_power_saving.max =
+ ps->performance_levels[i].pcie_gen;
+
+ if (data->pcie_gen_power_saving.min >
+ ps->performance_levels[i].pcie_gen)
+ data->pcie_gen_power_saving.min =
+ ps->performance_levels[i].pcie_gen;
+
+ if (data->pcie_lane_power_saving.max <
+ ps->performance_levels[i].pcie_lane)
+ data->pcie_lane_power_saving.max =
+ ps->performance_levels[i].pcie_lane;
+
+ if (data->pcie_lane_power_saving.min >
+ ps->performance_levels[i].pcie_lane)
+ data->pcie_lane_power_saving.min =
+ ps->performance_levels[i].pcie_lane;
+ }
+ break;
+ default:
+ break;
+ }
+ }
+ return 0;
+}
+
+static void
+polaris10_print_current_perforce_level(struct pp_hwmgr *hwmgr, struct seq_file *m)
+{
+ uint32_t sclk, mclk, activity_percent;
+ uint32_t offset;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ smum_send_msg_to_smc(hwmgr->smumgr, PPSMC_MSG_API_GetSclkFrequency);
+
+ sclk = cgs_read_register(hwmgr->device, mmSMC_MSG_ARG_0);
+
+ smum_send_msg_to_smc(hwmgr->smumgr, PPSMC_MSG_API_GetMclkFrequency);
+
+ mclk = cgs_read_register(hwmgr->device, mmSMC_MSG_ARG_0);
+ seq_printf(m, "\n [ mclk ]: %u MHz\n\n [ sclk ]: %u MHz\n",
+ mclk / 100, sclk / 100);
+
+ offset = data->soft_regs_start + offsetof(SMU74_SoftRegisters, AverageGraphicsActivity);
+ activity_percent = cgs_read_ind_register(hwmgr->device, CGS_IND_REG__SMC, offset);
+ activity_percent += 0x80;
+ activity_percent >>= 8;
+
+ seq_printf(m, "\n [GPU load]: %u%%\n\n", activity_percent > 100 ? 100 : activity_percent);
+
+ seq_printf(m, "uvd %sabled\n", data->uvd_power_gated ? "dis" : "en");
+
+ seq_printf(m, "vce %sabled\n", data->vce_power_gated ? "dis" : "en");
+}
+
+static int polaris10_find_dpm_states_clocks_in_dpm_table(struct pp_hwmgr *hwmgr, const void *input)
+{
+ const struct phm_set_power_state_input *states =
+ (const struct phm_set_power_state_input *)input;
+ const struct polaris10_power_state *polaris10_ps =
+ cast_const_phw_polaris10_power_state(states->pnew_state);
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct polaris10_single_dpm_table *sclk_table = &(data->dpm_table.sclk_table);
+ uint32_t sclk = polaris10_ps->performance_levels
+ [polaris10_ps->performance_level_count - 1].engine_clock;
+ struct polaris10_single_dpm_table *mclk_table = &(data->dpm_table.mclk_table);
+ uint32_t mclk = polaris10_ps->performance_levels
+ [polaris10_ps->performance_level_count - 1].memory_clock;
+ struct PP_Clocks min_clocks = {0};
+ uint32_t i;
+ struct cgs_display_info info = {0};
+
+ data->need_update_smu7_dpm_table = 0;
+
+ for (i = 0; i < sclk_table->count; i++) {
+ if (sclk == sclk_table->dpm_levels[i].value)
+ break;
+ }
+
+ if (i >= sclk_table->count)
+ data->need_update_smu7_dpm_table |= DPMTABLE_OD_UPDATE_SCLK;
+ else {
+ /* TODO: Check SCLK in DAL's minimum clocks
+ * in case DeepSleep divider update is required.
+ */
+ if (data->display_timing.min_clock_in_sr != min_clocks.engineClockInSR &&
+ (min_clocks.engineClockInSR >= POLARIS10_MINIMUM_ENGINE_CLOCK ||
+ data->display_timing.min_clock_in_sr >= POLARIS10_MINIMUM_ENGINE_CLOCK))
+ data->need_update_smu7_dpm_table |= DPMTABLE_UPDATE_SCLK;
+ }
+
+ for (i = 0; i < mclk_table->count; i++) {
+ if (mclk == mclk_table->dpm_levels[i].value)
+ break;
+ }
+
+ if (i >= mclk_table->count)
+ data->need_update_smu7_dpm_table |= DPMTABLE_OD_UPDATE_MCLK;
+
+ cgs_get_active_displays_info(hwmgr->device, &info);
+
+ if (data->display_timing.num_existing_displays != info.display_count)
+ data->need_update_smu7_dpm_table |= DPMTABLE_UPDATE_MCLK;
+
+ return 0;
+}
+
+static uint16_t polaris10_get_maximum_link_speed(struct pp_hwmgr *hwmgr,
+ const struct polaris10_power_state *polaris10_ps)
+{
+ uint32_t i;
+ uint32_t sclk, max_sclk = 0;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct polaris10_dpm_table *dpm_table = &data->dpm_table;
+
+ for (i = 0; i < polaris10_ps->performance_level_count; i++) {
+ sclk = polaris10_ps->performance_levels[i].engine_clock;
+ if (max_sclk < sclk)
+ max_sclk = sclk;
+ }
+
+ for (i = 0; i < dpm_table->sclk_table.count; i++) {
+ if (dpm_table->sclk_table.dpm_levels[i].value == max_sclk)
+ return (uint16_t) ((i >= dpm_table->pcie_speed_table.count) ?
+ dpm_table->pcie_speed_table.dpm_levels
+ [dpm_table->pcie_speed_table.count - 1].value :
+ dpm_table->pcie_speed_table.dpm_levels[i].value);
+ }
+
+ return 0;
+}
+
+static int polaris10_request_link_speed_change_before_state_change(
+ struct pp_hwmgr *hwmgr, const void *input)
+{
+ const struct phm_set_power_state_input *states =
+ (const struct phm_set_power_state_input *)input;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ const struct polaris10_power_state *polaris10_nps =
+ cast_const_phw_polaris10_power_state(states->pnew_state);
+ const struct polaris10_power_state *polaris10_cps =
+ cast_const_phw_polaris10_power_state(states->pcurrent_state);
+
+ uint16_t target_link_speed = polaris10_get_maximum_link_speed(hwmgr, polaris10_nps);
+ uint16_t current_link_speed;
+
+ if (data->force_pcie_gen == PP_PCIEGenInvalid)
+ current_link_speed = polaris10_get_maximum_link_speed(hwmgr, polaris10_cps);
+ else
+ current_link_speed = data->force_pcie_gen;
+
+ data->force_pcie_gen = PP_PCIEGenInvalid;
+ data->pspp_notify_required = false;
+
+ if (target_link_speed > current_link_speed) {
+ switch (target_link_speed) {
+ case PP_PCIEGen3:
+ if (0 == acpi_pcie_perf_request(hwmgr->device, PCIE_PERF_REQ_GEN3, false))
+ break;
+ data->force_pcie_gen = PP_PCIEGen2;
+ if (current_link_speed == PP_PCIEGen2)
+ break;
+ case PP_PCIEGen2:
+ if (0 == acpi_pcie_perf_request(hwmgr->device, PCIE_PERF_REQ_GEN2, false))
+ break;
+ default:
+ data->force_pcie_gen = phm_get_current_pcie_speed(hwmgr);
+ break;
+ }
+ } else {
+ if (target_link_speed < current_link_speed)
+ data->pspp_notify_required = true;
+ }
+
+ return 0;
+}
+
+static int polaris10_freeze_sclk_mclk_dpm(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ if (0 == data->need_update_smu7_dpm_table)
+ return 0;
+
+ if ((0 == data->sclk_dpm_key_disabled) &&
+ (data->need_update_smu7_dpm_table &
+ (DPMTABLE_OD_UPDATE_SCLK + DPMTABLE_UPDATE_SCLK))) {
+ PP_ASSERT_WITH_CODE(true == polaris10_is_dpm_running(hwmgr),
+ "Trying to freeze SCLK DPM when DPM is disabled",
+ );
+ PP_ASSERT_WITH_CODE(0 == smum_send_msg_to_smc(hwmgr->smumgr,
+ PPSMC_MSG_SCLKDPM_FreezeLevel),
+ "Failed to freeze SCLK DPM during FreezeSclkMclkDPM Function!",
+ return -1);
+ }
+
+ if ((0 == data->mclk_dpm_key_disabled) &&
+ (data->need_update_smu7_dpm_table &
+ DPMTABLE_OD_UPDATE_MCLK)) {
+ PP_ASSERT_WITH_CODE(true == polaris10_is_dpm_running(hwmgr),
+ "Trying to freeze MCLK DPM when DPM is disabled",
+ );
+ PP_ASSERT_WITH_CODE(0 == smum_send_msg_to_smc(hwmgr->smumgr,
+ PPSMC_MSG_MCLKDPM_FreezeLevel),
+ "Failed to freeze MCLK DPM during FreezeSclkMclkDPM Function!",
+ return -1);
+ }
+
+ return 0;
+}
+
+static int polaris10_populate_and_upload_sclk_mclk_dpm_levels(
+ struct pp_hwmgr *hwmgr, const void *input)
+{
+ int result = 0;
+ const struct phm_set_power_state_input *states =
+ (const struct phm_set_power_state_input *)input;
+ const struct polaris10_power_state *polaris10_ps =
+ cast_const_phw_polaris10_power_state(states->pnew_state);
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ uint32_t sclk = polaris10_ps->performance_levels
+ [polaris10_ps->performance_level_count - 1].engine_clock;
+ uint32_t mclk = polaris10_ps->performance_levels
+ [polaris10_ps->performance_level_count - 1].memory_clock;
+ struct polaris10_dpm_table *dpm_table = &data->dpm_table;
+
+ struct polaris10_dpm_table *golden_dpm_table = &data->golden_dpm_table;
+ uint32_t dpm_count, clock_percent;
+ uint32_t i;
+
+ if (0 == data->need_update_smu7_dpm_table)
+ return 0;
+
+ if (data->need_update_smu7_dpm_table & DPMTABLE_OD_UPDATE_SCLK) {
+ dpm_table->sclk_table.dpm_levels
+ [dpm_table->sclk_table.count - 1].value = sclk;
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps, PHM_PlatformCaps_OD6PlusinACSupport) ||
+ phm_cap_enabled(hwmgr->platform_descriptor.platformCaps, PHM_PlatformCaps_OD6PlusinDCSupport)) {
+ /* Need to do calculation based on the golden DPM table
+ * as the Heatmap GPU Clock axis is also based on the default values
+ */
+ PP_ASSERT_WITH_CODE(
+ (golden_dpm_table->sclk_table.dpm_levels
+ [golden_dpm_table->sclk_table.count - 1].value != 0),
+ "Divide by 0!",
+ return -1);
+ dpm_count = dpm_table->sclk_table.count < 2 ? 0 : dpm_table->sclk_table.count - 2;
+
+ for (i = dpm_count; i > 1; i--) {
+ if (sclk > golden_dpm_table->sclk_table.dpm_levels[golden_dpm_table->sclk_table.count-1].value) {
+ clock_percent =
+ ((sclk
+ - golden_dpm_table->sclk_table.dpm_levels[golden_dpm_table->sclk_table.count-1].value
+ ) * 100)
+ / golden_dpm_table->sclk_table.dpm_levels[golden_dpm_table->sclk_table.count-1].value;
+
+ dpm_table->sclk_table.dpm_levels[i].value =
+ golden_dpm_table->sclk_table.dpm_levels[i].value +
+ (golden_dpm_table->sclk_table.dpm_levels[i].value *
+ clock_percent)/100;
+
+ } else if (golden_dpm_table->sclk_table.dpm_levels[dpm_table->sclk_table.count-1].value > sclk) {
+ clock_percent =
+ ((golden_dpm_table->sclk_table.dpm_levels[golden_dpm_table->sclk_table.count - 1].value
+ - sclk) * 100)
+ / golden_dpm_table->sclk_table.dpm_levels[golden_dpm_table->sclk_table.count-1].value;
+
+ dpm_table->sclk_table.dpm_levels[i].value =
+ golden_dpm_table->sclk_table.dpm_levels[i].value -
+ (golden_dpm_table->sclk_table.dpm_levels[i].value *
+ clock_percent) / 100;
+ } else
+ dpm_table->sclk_table.dpm_levels[i].value =
+ golden_dpm_table->sclk_table.dpm_levels[i].value;
+ }
+ }
+ }
+
+ if (data->need_update_smu7_dpm_table & DPMTABLE_OD_UPDATE_MCLK) {
+ dpm_table->mclk_table.dpm_levels
+ [dpm_table->mclk_table.count - 1].value = mclk;
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps, PHM_PlatformCaps_OD6PlusinACSupport) ||
+ phm_cap_enabled(hwmgr->platform_descriptor.platformCaps, PHM_PlatformCaps_OD6PlusinDCSupport)) {
+
+ PP_ASSERT_WITH_CODE(
+ (golden_dpm_table->mclk_table.dpm_levels
+ [golden_dpm_table->mclk_table.count-1].value != 0),
+ "Divide by 0!",
+ return -1);
+ dpm_count = dpm_table->mclk_table.count < 2 ? 0 : dpm_table->mclk_table.count - 2;
+ for (i = dpm_count; i > 1; i--) {
+ if (golden_dpm_table->mclk_table.dpm_levels[golden_dpm_table->mclk_table.count-1].value < mclk) {
+ clock_percent = ((mclk -
+ golden_dpm_table->mclk_table.dpm_levels[golden_dpm_table->mclk_table.count-1].value) * 100)
+ / golden_dpm_table->mclk_table.dpm_levels[golden_dpm_table->mclk_table.count-1].value;
+
+ dpm_table->mclk_table.dpm_levels[i].value =
+ golden_dpm_table->mclk_table.dpm_levels[i].value +
+ (golden_dpm_table->mclk_table.dpm_levels[i].value *
+ clock_percent) / 100;
+
+ } else if (golden_dpm_table->mclk_table.dpm_levels[dpm_table->mclk_table.count-1].value > mclk) {
+ clock_percent = (
+ (golden_dpm_table->mclk_table.dpm_levels[golden_dpm_table->mclk_table.count-1].value - mclk)
+ * 100)
+ / golden_dpm_table->mclk_table.dpm_levels[golden_dpm_table->mclk_table.count-1].value;
+
+ dpm_table->mclk_table.dpm_levels[i].value =
+ golden_dpm_table->mclk_table.dpm_levels[i].value -
+ (golden_dpm_table->mclk_table.dpm_levels[i].value *
+ clock_percent) / 100;
+ } else
+ dpm_table->mclk_table.dpm_levels[i].value =
+ golden_dpm_table->mclk_table.dpm_levels[i].value;
+ }
+ }
+ }
+
+ if (data->need_update_smu7_dpm_table &
+ (DPMTABLE_OD_UPDATE_SCLK + DPMTABLE_UPDATE_SCLK)) {
+ result = polaris10_populate_all_graphic_levels(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == result),
+ "Failed to populate SCLK during PopulateNewDPMClocksStates Function!",
+ return result);
+ }
+
+ if (data->need_update_smu7_dpm_table &
+ (DPMTABLE_OD_UPDATE_MCLK + DPMTABLE_UPDATE_MCLK)) {
+ /*populate MCLK dpm table to SMU7 */
+ result = polaris10_populate_all_memory_levels(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == result),
+ "Failed to populate MCLK during PopulateNewDPMClocksStates Function!",
+ return result);
+ }
+
+ return result;
+}
+
+static int polaris10_trim_single_dpm_states(struct pp_hwmgr *hwmgr,
+ struct polaris10_single_dpm_table *dpm_table,
+ uint32_t low_limit, uint32_t high_limit)
+{
+ uint32_t i;
+
+ for (i = 0; i < dpm_table->count; i++) {
+ if ((dpm_table->dpm_levels[i].value < low_limit)
+ || (dpm_table->dpm_levels[i].value > high_limit))
+ dpm_table->dpm_levels[i].enabled = false;
+ else
+ dpm_table->dpm_levels[i].enabled = true;
+ }
+
+ return 0;
+}
+
+static int polaris10_trim_dpm_states(struct pp_hwmgr *hwmgr,
+ const struct polaris10_power_state *polaris10_ps)
+{
+ int result = 0;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ uint32_t high_limit_count;
+
+ PP_ASSERT_WITH_CODE((polaris10_ps->performance_level_count >= 1),
+ "power state did not have any performance level",
+ return -1);
+
+ high_limit_count = (1 == polaris10_ps->performance_level_count) ? 0 : 1;
+
+ polaris10_trim_single_dpm_states(hwmgr,
+ &(data->dpm_table.sclk_table),
+ polaris10_ps->performance_levels[0].engine_clock,
+ polaris10_ps->performance_levels[high_limit_count].engine_clock);
+
+ polaris10_trim_single_dpm_states(hwmgr,
+ &(data->dpm_table.mclk_table),
+ polaris10_ps->performance_levels[0].memory_clock,
+ polaris10_ps->performance_levels[high_limit_count].memory_clock);
+
+ return result;
+}
+
+static int polaris10_generate_dpm_level_enable_mask(
+ struct pp_hwmgr *hwmgr, const void *input)
+{
+ int result;
+ const struct phm_set_power_state_input *states =
+ (const struct phm_set_power_state_input *)input;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ const struct polaris10_power_state *polaris10_ps =
+ cast_const_phw_polaris10_power_state(states->pnew_state);
+
+ result = polaris10_trim_dpm_states(hwmgr, polaris10_ps);
+ if (result)
+ return result;
+
+ data->dpm_level_enable_mask.sclk_dpm_enable_mask =
+ phm_get_dpm_level_enable_mask_value(&data->dpm_table.sclk_table);
+ data->dpm_level_enable_mask.mclk_dpm_enable_mask =
+ phm_get_dpm_level_enable_mask_value(&data->dpm_table.mclk_table);
+ data->dpm_level_enable_mask.pcie_dpm_enable_mask =
+ phm_get_dpm_level_enable_mask_value(&data->dpm_table.pcie_speed_table);
+
+ return 0;
+}
+
+int polaris10_enable_disable_uvd_dpm(struct pp_hwmgr *hwmgr, bool enable)
+{
+ return smum_send_msg_to_smc(hwmgr->smumgr, enable ?
+ PPSMC_MSG_UVDDPM_Enable :
+ PPSMC_MSG_UVDDPM_Disable);
+}
+
+int polaris10_enable_disable_vce_dpm(struct pp_hwmgr *hwmgr, bool enable)
+{
+ return smum_send_msg_to_smc(hwmgr->smumgr, enable?
+ PPSMC_MSG_VCEDPM_Enable :
+ PPSMC_MSG_VCEDPM_Disable);
+}
+
+int polaris10_enable_disable_samu_dpm(struct pp_hwmgr *hwmgr, bool enable)
+{
+ return smum_send_msg_to_smc(hwmgr->smumgr, enable?
+ PPSMC_MSG_SAMUDPM_Enable :
+ PPSMC_MSG_SAMUDPM_Disable);
+}
+
+int polaris10_update_uvd_dpm(struct pp_hwmgr *hwmgr, bool bgate)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ uint32_t mm_boot_level_offset, mm_boot_level_value;
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+
+ if (!bgate) {
+ data->smc_state_table.UvdBootLevel = 0;
+ if (table_info->mm_dep_table->count > 0)
+ data->smc_state_table.UvdBootLevel =
+ (uint8_t) (table_info->mm_dep_table->count - 1);
+ mm_boot_level_offset = data->dpm_table_start +
+ offsetof(SMU74_Discrete_DpmTable, UvdBootLevel);
+ mm_boot_level_offset /= 4;
+ mm_boot_level_offset *= 4;
+ mm_boot_level_value = cgs_read_ind_register(hwmgr->device,
+ CGS_IND_REG__SMC, mm_boot_level_offset);
+ mm_boot_level_value &= 0x00FFFFFF;
+ mm_boot_level_value |= data->smc_state_table.UvdBootLevel << 24;
+ cgs_write_ind_register(hwmgr->device,
+ CGS_IND_REG__SMC, mm_boot_level_offset, mm_boot_level_value);
+
+ if (!phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_UVDDPM) ||
+ phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_StablePState))
+ smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_UVDDPM_SetEnabledMask,
+ (uint32_t)(1 << data->smc_state_table.UvdBootLevel));
+ }
+
+ return polaris10_enable_disable_uvd_dpm(hwmgr, !bgate);
+}
+
+static int polaris10_update_vce_dpm(struct pp_hwmgr *hwmgr, const void *input)
+{
+ const struct phm_set_power_state_input *states =
+ (const struct phm_set_power_state_input *)input;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ const struct polaris10_power_state *polaris10_nps =
+ cast_const_phw_polaris10_power_state(states->pnew_state);
+ const struct polaris10_power_state *polaris10_cps =
+ cast_const_phw_polaris10_power_state(states->pcurrent_state);
+
+ uint32_t mm_boot_level_offset, mm_boot_level_value;
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+
+ if (polaris10_nps->vce_clks.evclk > 0 &&
+ (polaris10_cps == NULL || polaris10_cps->vce_clks.evclk == 0)) {
+
+ data->smc_state_table.VceBootLevel =
+ (uint8_t) (table_info->mm_dep_table->count - 1);
+
+ mm_boot_level_offset = data->dpm_table_start +
+ offsetof(SMU74_Discrete_DpmTable, VceBootLevel);
+ mm_boot_level_offset /= 4;
+ mm_boot_level_offset *= 4;
+ mm_boot_level_value = cgs_read_ind_register(hwmgr->device,
+ CGS_IND_REG__SMC, mm_boot_level_offset);
+ mm_boot_level_value &= 0xFF00FFFF;
+ mm_boot_level_value |= data->smc_state_table.VceBootLevel << 16;
+ cgs_write_ind_register(hwmgr->device,
+ CGS_IND_REG__SMC, mm_boot_level_offset, mm_boot_level_value);
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps, PHM_PlatformCaps_StablePState)) {
+ smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_VCEDPM_SetEnabledMask,
+ (uint32_t)1 << data->smc_state_table.VceBootLevel);
+
+ polaris10_enable_disable_vce_dpm(hwmgr, true);
+ } else if (polaris10_nps->vce_clks.evclk == 0 &&
+ polaris10_cps != NULL &&
+ polaris10_cps->vce_clks.evclk > 0)
+ polaris10_enable_disable_vce_dpm(hwmgr, false);
+ }
+
+ return 0;
+}
+
+int polaris10_update_samu_dpm(struct pp_hwmgr *hwmgr, bool bgate)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ uint32_t mm_boot_level_offset, mm_boot_level_value;
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+
+ if (!bgate) {
+ data->smc_state_table.SamuBootLevel =
+ (uint8_t) (table_info->mm_dep_table->count - 1);
+ mm_boot_level_offset = data->dpm_table_start +
+ offsetof(SMU74_Discrete_DpmTable, SamuBootLevel);
+ mm_boot_level_offset /= 4;
+ mm_boot_level_offset *= 4;
+ mm_boot_level_value = cgs_read_ind_register(hwmgr->device,
+ CGS_IND_REG__SMC, mm_boot_level_offset);
+ mm_boot_level_value &= 0xFFFFFF00;
+ mm_boot_level_value |= data->smc_state_table.SamuBootLevel << 0;
+ cgs_write_ind_register(hwmgr->device,
+ CGS_IND_REG__SMC, mm_boot_level_offset, mm_boot_level_value);
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_StablePState))
+ smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_SAMUDPM_SetEnabledMask,
+ (uint32_t)(1 << data->smc_state_table.SamuBootLevel));
+ }
+
+ return polaris10_enable_disable_samu_dpm(hwmgr, !bgate);
+}
+
+static int polaris10_update_sclk_threshold(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ int result = 0;
+ uint32_t low_sclk_interrupt_threshold = 0;
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_SclkThrottleLowNotification)
+ && (hwmgr->gfx_arbiter.sclk_threshold !=
+ data->low_sclk_interrupt_threshold)) {
+ data->low_sclk_interrupt_threshold =
+ hwmgr->gfx_arbiter.sclk_threshold;
+ low_sclk_interrupt_threshold =
+ data->low_sclk_interrupt_threshold;
+
+ CONVERT_FROM_HOST_TO_SMC_UL(low_sclk_interrupt_threshold);
+
+ result = polaris10_copy_bytes_to_smc(
+ hwmgr->smumgr,
+ data->dpm_table_start +
+ offsetof(SMU74_Discrete_DpmTable,
+ LowSclkInterruptThreshold),
+ (uint8_t *)&low_sclk_interrupt_threshold,
+ sizeof(uint32_t),
+ data->sram_end);
+ }
+
+ return result;
+}
+
+static int polaris10_program_mem_timing_parameters(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ if (data->need_update_smu7_dpm_table &
+ (DPMTABLE_OD_UPDATE_SCLK + DPMTABLE_OD_UPDATE_MCLK))
+ return polaris10_program_memory_timing_parameters(hwmgr);
+
+ return 0;
+}
+
+static int polaris10_unfreeze_sclk_mclk_dpm(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ if (0 == data->need_update_smu7_dpm_table)
+ return 0;
+
+ if ((0 == data->sclk_dpm_key_disabled) &&
+ (data->need_update_smu7_dpm_table &
+ (DPMTABLE_OD_UPDATE_SCLK + DPMTABLE_UPDATE_SCLK))) {
+
+ PP_ASSERT_WITH_CODE(true == polaris10_is_dpm_running(hwmgr),
+ "Trying to Unfreeze SCLK DPM when DPM is disabled",
+ );
+ PP_ASSERT_WITH_CODE(0 == smum_send_msg_to_smc(hwmgr->smumgr,
+ PPSMC_MSG_SCLKDPM_UnfreezeLevel),
+ "Failed to unfreeze SCLK DPM during UnFreezeSclkMclkDPM Function!",
+ return -1);
+ }
+
+ if ((0 == data->mclk_dpm_key_disabled) &&
+ (data->need_update_smu7_dpm_table & DPMTABLE_OD_UPDATE_MCLK)) {
+
+ PP_ASSERT_WITH_CODE(true == polaris10_is_dpm_running(hwmgr),
+ "Trying to Unfreeze MCLK DPM when DPM is disabled",
+ );
+ PP_ASSERT_WITH_CODE(0 == smum_send_msg_to_smc(hwmgr->smumgr,
+ PPSMC_MSG_SCLKDPM_UnfreezeLevel),
+ "Failed to unfreeze MCLK DPM during UnFreezeSclkMclkDPM Function!",
+ return -1);
+ }
+
+ data->need_update_smu7_dpm_table = 0;
+
+ return 0;
+}
+
+static int polaris10_notify_link_speed_change_after_state_change(
+ struct pp_hwmgr *hwmgr, const void *input)
+{
+ const struct phm_set_power_state_input *states =
+ (const struct phm_set_power_state_input *)input;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ const struct polaris10_power_state *polaris10_ps =
+ cast_const_phw_polaris10_power_state(states->pnew_state);
+ uint16_t target_link_speed = polaris10_get_maximum_link_speed(hwmgr, polaris10_ps);
+ uint8_t request;
+
+ if (data->pspp_notify_required) {
+ if (target_link_speed == PP_PCIEGen3)
+ request = PCIE_PERF_REQ_GEN3;
+ else if (target_link_speed == PP_PCIEGen2)
+ request = PCIE_PERF_REQ_GEN2;
+ else
+ request = PCIE_PERF_REQ_GEN1;
+
+ if (request == PCIE_PERF_REQ_GEN1 &&
+ phm_get_current_pcie_speed(hwmgr) > 0)
+ return 0;
+
+ if (acpi_pcie_perf_request(hwmgr->device, request, false)) {
+ if (PP_PCIEGen2 == target_link_speed)
+ printk("PSPP request to switch to Gen2 from Gen3 Failed!");
+ else
+ printk("PSPP request to switch to Gen1 from Gen2 Failed!");
+ }
+ }
+
+ return 0;
+}
+
+static int polaris10_set_power_state_tasks(struct pp_hwmgr *hwmgr, const void *input)
+{
+ int tmp_result, result = 0;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ tmp_result = polaris10_find_dpm_states_clocks_in_dpm_table(hwmgr, input);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to find DPM states clocks in DPM table!",
+ result = tmp_result);
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_PCIEPerformanceRequest)) {
+ tmp_result =
+ polaris10_request_link_speed_change_before_state_change(hwmgr, input);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to request link speed change before state change!",
+ result = tmp_result);
+ }
+
+ tmp_result = polaris10_freeze_sclk_mclk_dpm(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to freeze SCLK MCLK DPM!", result = tmp_result);
+
+ tmp_result = polaris10_populate_and_upload_sclk_mclk_dpm_levels(hwmgr, input);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to populate and upload SCLK MCLK DPM levels!",
+ result = tmp_result);
+
+ tmp_result = polaris10_generate_dpm_level_enable_mask(hwmgr, input);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to generate DPM level enabled mask!",
+ result = tmp_result);
+
+ tmp_result = polaris10_update_vce_dpm(hwmgr, input);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to update VCE DPM!",
+ result = tmp_result);
+
+ tmp_result = polaris10_update_sclk_threshold(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to update SCLK threshold!",
+ result = tmp_result);
+
+ tmp_result = polaris10_program_mem_timing_parameters(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to program memory timing parameters!",
+ result = tmp_result);
+
+ tmp_result = polaris10_unfreeze_sclk_mclk_dpm(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to unfreeze SCLK MCLK DPM!",
+ result = tmp_result);
+
+ tmp_result = polaris10_upload_dpm_level_enable_mask(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to upload DPM level enabled mask!",
+ result = tmp_result);
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_PCIEPerformanceRequest)) {
+ tmp_result =
+ polaris10_notify_link_speed_change_after_state_change(hwmgr, input);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to notify link speed change after state change!",
+ result = tmp_result);
+ }
+ data->apply_optimized_settings = false;
+ return result;
+}
+
+static int polaris10_set_max_fan_pwm_output(struct pp_hwmgr *hwmgr, uint16_t us_max_fan_pwm)
+{
+ hwmgr->thermal_controller.
+ advanceFanControlParameters.usMaxFanPWM = us_max_fan_pwm;
+
+ if (phm_is_hw_access_blocked(hwmgr))
+ return 0;
+
+ return smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_SetFanPwmMax, us_max_fan_pwm);
+}
+
+int polaris10_notify_smc_display_change(struct pp_hwmgr *hwmgr, bool has_display)
+{
+ PPSMC_Msg msg = has_display ? (PPSMC_Msg)PPSMC_HasDisplay : (PPSMC_Msg)PPSMC_NoDisplay;
+
+ return (smum_send_msg_to_smc(hwmgr->smumgr, msg) == 0) ? 0 : -1;
+}
+
+int polaris10_notify_smc_display_config_after_ps_adjustment(struct pp_hwmgr *hwmgr)
+{
+ uint32_t num_active_displays = 0;
+ struct cgs_display_info info = {0};
+ info.mode_info = NULL;
+
+ cgs_get_active_displays_info(hwmgr->device, &info);
+
+ num_active_displays = info.display_count;
+
+ if (num_active_displays > 1) /* to do && (pHwMgr->pPECI->displayConfiguration.bMultiMonitorInSync != TRUE)) */
+ polaris10_notify_smc_display_change(hwmgr, false);
+ else
+ polaris10_notify_smc_display_change(hwmgr, true);
+
+ return 0;
+}
+
+/**
+* Programs the display gap
+*
+* @param hwmgr the address of the powerplay hardware manager.
+* @return always OK
+*/
+int polaris10_program_display_gap(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ uint32_t num_active_displays = 0;
+ uint32_t display_gap = cgs_read_ind_register(hwmgr->device, CGS_IND_REG__SMC, ixCG_DISPLAY_GAP_CNTL);
+ uint32_t display_gap2;
+ uint32_t pre_vbi_time_in_us;
+ uint32_t frame_time_in_us;
+ uint32_t ref_clock;
+ uint32_t refresh_rate = 0;
+ struct cgs_display_info info = {0};
+ struct cgs_mode_info mode_info;
+
+ info.mode_info = &mode_info;
+
+ cgs_get_active_displays_info(hwmgr->device, &info);
+ num_active_displays = info.display_count;
+
+ display_gap = PHM_SET_FIELD(display_gap, CG_DISPLAY_GAP_CNTL, DISP_GAP, (num_active_displays > 0) ? DISPLAY_GAP_VBLANK_OR_WM : DISPLAY_GAP_IGNORE);
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC, ixCG_DISPLAY_GAP_CNTL, display_gap);
+
+ ref_clock = mode_info.ref_clock;
+ refresh_rate = mode_info.refresh_rate;
+
+ if (0 == refresh_rate)
+ refresh_rate = 60;
+
+ frame_time_in_us = 1000000 / refresh_rate;
+
+ pre_vbi_time_in_us = frame_time_in_us - 200 - mode_info.vblank_time_us;
+ display_gap2 = pre_vbi_time_in_us * (ref_clock / 100);
+
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC, ixCG_DISPLAY_GAP_CNTL2, display_gap2);
+
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC, data->soft_regs_start + offsetof(SMU74_SoftRegisters, PreVBlankGap), 0x64);
+
+ cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC, data->soft_regs_start + offsetof(SMU74_SoftRegisters, VBlankTimeout), (frame_time_in_us - pre_vbi_time_in_us));
+
+ polaris10_notify_smc_display_change(hwmgr, num_active_displays != 0);
+
+ return 0;
+}
+
+
+int polaris10_display_configuration_changed_task(struct pp_hwmgr *hwmgr)
+{
+ return polaris10_program_display_gap(hwmgr);
+}
+
+/**
+* Set maximum target operating fan output RPM
+*
+* @param hwmgr: the address of the powerplay hardware manager.
+* @param usMaxFanRpm: max operating fan RPM value.
+* @return The response that came from the SMC.
+*/
+static int polaris10_set_max_fan_rpm_output(struct pp_hwmgr *hwmgr, uint16_t us_max_fan_rpm)
+{
+ hwmgr->thermal_controller.
+ advanceFanControlParameters.usMaxFanRPM = us_max_fan_rpm;
+
+ if (phm_is_hw_access_blocked(hwmgr))
+ return 0;
+
+ return smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_SetFanRpmMax, us_max_fan_rpm);
+}
+
+int polaris10_register_internal_thermal_interrupt(struct pp_hwmgr *hwmgr,
+ const void *thermal_interrupt_info)
+{
+ return 0;
+}
+
+bool polaris10_check_smc_update_required_for_display_configuration(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ bool is_update_required = false;
+ struct cgs_display_info info = {0, 0, NULL};
+
+ cgs_get_active_displays_info(hwmgr->device, &info);
+
+ if (data->display_timing.num_existing_displays != info.display_count)
+ is_update_required = true;
+/* TO DO NEED TO GET DEEP SLEEP CLOCK FROM DAL
+ if (phm_cap_enabled(hwmgr->hwmgr->platform_descriptor.platformCaps, PHM_PlatformCaps_SclkDeepSleep)) {
+ cgs_get_min_clock_settings(hwmgr->device, &min_clocks);
+ if (min_clocks.engineClockInSR != data->display_timing.minClockInSR &&
+ (min_clocks.engineClockInSR >= POLARIS10_MINIMUM_ENGINE_CLOCK ||
+ data->display_timing.minClockInSR >= POLARIS10_MINIMUM_ENGINE_CLOCK))
+ is_update_required = true;
+*/
+ return is_update_required;
+}
+
+static inline bool polaris10_are_power_levels_equal(const struct polaris10_performance_level *pl1,
+ const struct polaris10_performance_level *pl2)
+{
+ return ((pl1->memory_clock == pl2->memory_clock) &&
+ (pl1->engine_clock == pl2->engine_clock) &&
+ (pl1->pcie_gen == pl2->pcie_gen) &&
+ (pl1->pcie_lane == pl2->pcie_lane));
+}
+
+int polaris10_check_states_equal(struct pp_hwmgr *hwmgr, const struct pp_hw_power_state *pstate1, const struct pp_hw_power_state *pstate2, bool *equal)
+{
+ const struct polaris10_power_state *psa = cast_const_phw_polaris10_power_state(pstate1);
+ const struct polaris10_power_state *psb = cast_const_phw_polaris10_power_state(pstate2);
+ int i;
+
+ if (pstate1 == NULL || pstate2 == NULL || equal == NULL)
+ return -EINVAL;
+
+ /* If the two states don't even have the same number of performance levels they cannot be the same state. */
+ if (psa->performance_level_count != psb->performance_level_count) {
+ *equal = false;
+ return 0;
+ }
+
+ for (i = 0; i < psa->performance_level_count; i++) {
+ if (!polaris10_are_power_levels_equal(&(psa->performance_levels[i]), &(psb->performance_levels[i]))) {
+ /* If we have found even one performance level pair that is different the states are different. */
+ *equal = false;
+ return 0;
+ }
+ }
+
+ /* If all performance levels are the same try to use the UVD clocks to break the tie.*/
+ *equal = ((psa->uvd_clks.vclk == psb->uvd_clks.vclk) && (psa->uvd_clks.dclk == psb->uvd_clks.dclk));
+ *equal &= ((psa->vce_clks.evclk == psb->vce_clks.evclk) && (psa->vce_clks.ecclk == psb->vce_clks.ecclk));
+ *equal &= (psa->sclk_threshold == psb->sclk_threshold);
+
+ return 0;
+}
+
+int polaris10_upload_mc_firmware(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ uint32_t vbios_version;
+
+ /* Read MC indirect register offset 0x9F bits [3:0] to see if VBIOS has already loaded a full version of MC ucode or not.*/
+
+ phm_get_mc_microcode_version(hwmgr);
+ vbios_version = hwmgr->microcode_version_info.MC & 0xf;
+ /* Full version of MC ucode has already been loaded. */
+ if (vbios_version == 0) {
+ data->need_long_memory_training = false;
+ return 0;
+ }
+
+ data->need_long_memory_training = true;
+
+/*
+ * PPMCME_FirmwareDescriptorEntry *pfd = NULL;
+ pfd = &tonga_mcmeFirmware;
+ if (0 == PHM_READ_FIELD(hwmgr->device, MC_SEQ_SUP_CNTL, RUN))
+ polaris10_load_mc_microcode(hwmgr, pfd->dpmThreshold,
+ pfd->cfgArray, pfd->cfgSize, pfd->ioDebugArray,
+ pfd->ioDebugSize, pfd->ucodeArray, pfd->ucodeSize);
+*/
+ return 0;
+}
+
+/**
+ * Read clock related registers.
+ *
+ * @param hwmgr the address of the powerplay hardware manager.
+ * @return always 0
+ */
+static int polaris10_read_clock_registers(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ data->clock_registers.vCG_SPLL_FUNC_CNTL = cgs_read_ind_register(hwmgr->device,
+ CGS_IND_REG__SMC, ixCG_SPLL_FUNC_CNTL)
+ & CG_SPLL_FUNC_CNTL__SPLL_BYPASS_EN_MASK;
+
+ data->clock_registers.vCG_SPLL_FUNC_CNTL_2 = cgs_read_ind_register(hwmgr->device,
+ CGS_IND_REG__SMC, ixCG_SPLL_FUNC_CNTL_2)
+ & CG_SPLL_FUNC_CNTL_2__SCLK_MUX_SEL_MASK;
+
+ data->clock_registers.vCG_SPLL_FUNC_CNTL_4 = cgs_read_ind_register(hwmgr->device,
+ CGS_IND_REG__SMC, ixCG_SPLL_FUNC_CNTL_4)
+ & CG_SPLL_FUNC_CNTL_4__SPLL_SPARE_MASK;
+
+ return 0;
+}
+
+/**
+ * Find out if memory is GDDR5.
+ *
+ * @param hwmgr the address of the powerplay hardware manager.
+ * @return always 0
+ */
+static int polaris10_get_memory_type(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ uint32_t temp;
+
+ temp = cgs_read_register(hwmgr->device, mmMC_SEQ_MISC0);
+
+ data->is_memory_gddr5 = (MC_SEQ_MISC0_GDDR5_VALUE ==
+ ((temp & MC_SEQ_MISC0_GDDR5_MASK) >>
+ MC_SEQ_MISC0_GDDR5_SHIFT));
+
+ return 0;
+}
+
+/**
+ * Enables Dynamic Power Management by SMC
+ *
+ * @param hwmgr the address of the powerplay hardware manager.
+ * @return always 0
+ */
+static int polaris10_enable_acpi_power_management(struct pp_hwmgr *hwmgr)
+{
+ PHM_WRITE_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ GENERAL_PWRMGT, STATIC_PM_EN, 1);
+
+ return 0;
+}
+
+/**
+ * Initialize PowerGating States for different engines
+ *
+ * @param hwmgr the address of the powerplay hardware manager.
+ * @return always 0
+ */
+static int polaris10_init_power_gate_state(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ data->uvd_power_gated = false;
+ data->vce_power_gated = false;
+ data->samu_power_gated = false;
+
+ return 0;
+}
+
+static int polaris10_init_sclk_threshold(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ data->low_sclk_interrupt_threshold = 0;
+
+ return 0;
+}
+
+int polaris10_setup_asic_task(struct pp_hwmgr *hwmgr)
+{
+ int tmp_result, result = 0;
+
+ polaris10_upload_mc_firmware(hwmgr);
+
+ tmp_result = polaris10_read_clock_registers(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to read clock registers!", result = tmp_result);
+
+ tmp_result = polaris10_get_memory_type(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to get memory type!", result = tmp_result);
+
+ tmp_result = polaris10_enable_acpi_power_management(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to enable ACPI power management!", result = tmp_result);
+
+ tmp_result = polaris10_init_power_gate_state(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to init power gate state!", result = tmp_result);
+
+ tmp_result = phm_get_mc_microcode_version(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to get MC microcode version!", result = tmp_result);
+
+ tmp_result = polaris10_init_sclk_threshold(hwmgr);
+ PP_ASSERT_WITH_CODE((0 == tmp_result),
+ "Failed to init sclk threshold!", result = tmp_result);
+
+ return result;
+}
+
+static int polaris10_get_pp_table(struct pp_hwmgr *hwmgr, char **table)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ if (!data->soft_pp_table) {
+ data->soft_pp_table = kzalloc(hwmgr->soft_pp_table_size, GFP_KERNEL);
+ if (!data->soft_pp_table)
+ return -ENOMEM;
+ memcpy(data->soft_pp_table, hwmgr->soft_pp_table,
+ hwmgr->soft_pp_table_size);
+ }
+
+ *table = (char *)&data->soft_pp_table;
+
+ return hwmgr->soft_pp_table_size;
+}
+
+static int polaris10_set_pp_table(struct pp_hwmgr *hwmgr, const char *buf, size_t size)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ if (!data->soft_pp_table) {
+ data->soft_pp_table = kzalloc(hwmgr->soft_pp_table_size, GFP_KERNEL);
+ if (!data->soft_pp_table)
+ return -ENOMEM;
+ }
+
+ memcpy(data->soft_pp_table, buf, size);
+
+ hwmgr->soft_pp_table = data->soft_pp_table;
+
+ /* TODO: re-init powerplay to implement modified pptable */
+
+ return 0;
+}
+
+static int polaris10_force_clock_level(struct pp_hwmgr *hwmgr,
+ enum pp_clock_type type, uint32_t mask)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ if (hwmgr->dpm_level != AMD_DPM_FORCED_LEVEL_MANUAL)
+ return -EINVAL;
+
+ switch (type) {
+ case PP_SCLK:
+ if (!data->sclk_dpm_key_disabled)
+ smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_SCLKDPM_SetEnabledMask,
+ data->dpm_level_enable_mask.sclk_dpm_enable_mask & mask);
+ break;
+ case PP_MCLK:
+ if (!data->mclk_dpm_key_disabled)
+ smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_MCLKDPM_SetEnabledMask,
+ data->dpm_level_enable_mask.mclk_dpm_enable_mask & mask);
+ break;
+ case PP_PCIE:
+ {
+ uint32_t tmp = mask & data->dpm_level_enable_mask.pcie_dpm_enable_mask;
+ uint32_t level = 0;
+
+ while (tmp >>= 1)
+ level++;
+
+ if (!data->pcie_dpm_key_disabled)
+ smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_PCIeDPM_ForceLevel,
+ level);
+ break;
+ }
+ default:
+ break;
+ }
+
+ return 0;
+}
+
+static uint16_t polaris10_get_current_pcie_speed(struct pp_hwmgr *hwmgr)
+{
+ uint32_t speedCntl = 0;
+
+ /* mmPCIE_PORT_INDEX rename as mmPCIE_INDEX */
+ speedCntl = cgs_read_ind_register(hwmgr->device, CGS_IND_REG__PCIE,
+ ixPCIE_LC_SPEED_CNTL);
+ return((uint16_t)PHM_GET_FIELD(speedCntl,
+ PCIE_LC_SPEED_CNTL, LC_CURRENT_DATA_RATE));
+}
+
+static int polaris10_print_clock_levels(struct pp_hwmgr *hwmgr,
+ enum pp_clock_type type, char *buf)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct polaris10_single_dpm_table *sclk_table = &(data->dpm_table.sclk_table);
+ struct polaris10_single_dpm_table *mclk_table = &(data->dpm_table.mclk_table);
+ struct polaris10_single_dpm_table *pcie_table = &(data->dpm_table.pcie_speed_table);
+ int i, now, size = 0;
+ uint32_t clock, pcie_speed;
+
+ switch (type) {
+ case PP_SCLK:
+ smum_send_msg_to_smc(hwmgr->smumgr, PPSMC_MSG_API_GetSclkFrequency);
+ clock = cgs_read_register(hwmgr->device, mmSMC_MSG_ARG_0);
+
+ for (i = 0; i < sclk_table->count; i++) {
+ if (clock > sclk_table->dpm_levels[i].value)
+ continue;
+ break;
+ }
+ now = i;
+
+ for (i = 0; i < sclk_table->count; i++)
+ size += sprintf(buf + size, "%d: %uMhz %s\n",
+ i, sclk_table->dpm_levels[i].value / 100,
+ (i == now) ? "*" : "");
+ break;
+ case PP_MCLK:
+ smum_send_msg_to_smc(hwmgr->smumgr, PPSMC_MSG_API_GetMclkFrequency);
+ clock = cgs_read_register(hwmgr->device, mmSMC_MSG_ARG_0);
+
+ for (i = 0; i < mclk_table->count; i++) {
+ if (clock > mclk_table->dpm_levels[i].value)
+ continue;
+ break;
+ }
+ now = i;
+
+ for (i = 0; i < mclk_table->count; i++)
+ size += sprintf(buf + size, "%d: %uMhz %s\n",
+ i, mclk_table->dpm_levels[i].value / 100,
+ (i == now) ? "*" : "");
+ break;
+ case PP_PCIE:
+ pcie_speed = polaris10_get_current_pcie_speed(hwmgr);
+ for (i = 0; i < pcie_table->count; i++) {
+ if (pcie_speed != pcie_table->dpm_levels[i].value)
+ continue;
+ break;
+ }
+ now = i;
+
+ for (i = 0; i < pcie_table->count; i++)
+ size += sprintf(buf + size, "%d: %s %s\n", i,
+ (pcie_table->dpm_levels[i].value == 0) ? "2.5GB, x8" :
+ (pcie_table->dpm_levels[i].value == 1) ? "5.0GB, x16" :
+ (pcie_table->dpm_levels[i].value == 2) ? "8.0GB, x16" : "",
+ (i == now) ? "*" : "");
+ break;
+ default:
+ break;
+ }
+ return size;
+}
+
+static int polaris10_set_fan_control_mode(struct pp_hwmgr *hwmgr, uint32_t mode)
+{
+ if (mode) {
+ /* stop auto-manage */
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_MicrocodeFanControl))
+ polaris10_fan_ctrl_stop_smc_fan_control(hwmgr);
+ polaris10_fan_ctrl_set_static_mode(hwmgr, mode);
+ } else
+ /* restart auto-manage */
+ polaris10_fan_ctrl_reset_fan_speed_to_default(hwmgr);
+
+ return 0;
+}
+
+static int polaris10_get_fan_control_mode(struct pp_hwmgr *hwmgr)
+{
+ if (hwmgr->fan_ctrl_is_in_default_mode)
+ return hwmgr->fan_ctrl_default_mode;
+ else
+ return PHM_READ_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_FDO_CTRL2, FDO_PWM_MODE);
+}
+
+static const struct pp_hwmgr_func polaris10_hwmgr_funcs = {
+ .backend_init = &polaris10_hwmgr_backend_init,
+ .backend_fini = &polaris10_hwmgr_backend_fini,
+ .asic_setup = &polaris10_setup_asic_task,
+ .dynamic_state_management_enable = &polaris10_enable_dpm_tasks,
+ .apply_state_adjust_rules = polaris10_apply_state_adjust_rules,
+ .force_dpm_level = &polaris10_force_dpm_level,
+ .power_state_set = polaris10_set_power_state_tasks,
+ .get_power_state_size = polaris10_get_power_state_size,
+ .get_mclk = polaris10_dpm_get_mclk,
+ .get_sclk = polaris10_dpm_get_sclk,
+ .patch_boot_state = polaris10_dpm_patch_boot_state,
+ .get_pp_table_entry = polaris10_get_pp_table_entry,
+ .get_num_of_pp_table_entries = tonga_get_number_of_powerplay_table_entries,
+ .print_current_perforce_level = polaris10_print_current_perforce_level,
+ .powerdown_uvd = polaris10_phm_powerdown_uvd,
+ .powergate_uvd = polaris10_phm_powergate_uvd,
+ .powergate_vce = polaris10_phm_powergate_vce,
+ .disable_clock_power_gating = polaris10_phm_disable_clock_power_gating,
+ .update_clock_gatings = polaris10_phm_update_clock_gatings,
+ .notify_smc_display_config_after_ps_adjustment = polaris10_notify_smc_display_config_after_ps_adjustment,
+ .display_config_changed = polaris10_display_configuration_changed_task,
+ .set_max_fan_pwm_output = polaris10_set_max_fan_pwm_output,
+ .set_max_fan_rpm_output = polaris10_set_max_fan_rpm_output,
+ .get_temperature = polaris10_thermal_get_temperature,
+ .stop_thermal_controller = polaris10_thermal_stop_thermal_controller,
+ .get_fan_speed_info = polaris10_fan_ctrl_get_fan_speed_info,
+ .get_fan_speed_percent = polaris10_fan_ctrl_get_fan_speed_percent,
+ .set_fan_speed_percent = polaris10_fan_ctrl_set_fan_speed_percent,
+ .reset_fan_speed_to_default = polaris10_fan_ctrl_reset_fan_speed_to_default,
+ .get_fan_speed_rpm = polaris10_fan_ctrl_get_fan_speed_rpm,
+ .set_fan_speed_rpm = polaris10_fan_ctrl_set_fan_speed_rpm,
+ .uninitialize_thermal_controller = polaris10_thermal_ctrl_uninitialize_thermal_controller,
+ .register_internal_thermal_interrupt = polaris10_register_internal_thermal_interrupt,
+ .check_smc_update_required_for_display_configuration = polaris10_check_smc_update_required_for_display_configuration,
+ .check_states_equal = polaris10_check_states_equal,
+ .set_fan_control_mode = polaris10_set_fan_control_mode,
+ .get_fan_control_mode = polaris10_get_fan_control_mode,
+ .get_pp_table = polaris10_get_pp_table,
+ .set_pp_table = polaris10_set_pp_table,
+ .force_clock_level = polaris10_force_clock_level,
+ .print_clock_levels = polaris10_print_clock_levels,
+ .enable_per_cu_power_gating = polaris10_phm_enable_per_cu_power_gating,
+};
+
+int polaris10_hwmgr_init(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data;
+
+ data = kzalloc (sizeof(struct polaris10_hwmgr), GFP_KERNEL);
+ if (data == NULL)
+ return -ENOMEM;
+
+ hwmgr->backend = data;
+ hwmgr->hwmgr_func = &polaris10_hwmgr_funcs;
+ hwmgr->pptable_func = &tonga_pptable_funcs;
+ pp_polaris10_thermal_initialize(hwmgr);
+
+ return 0;
+}
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_hwmgr.h b/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_hwmgr.h
new file mode 100644
index 0000000..beedf35
--- /dev/null
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_hwmgr.h
@@ -0,0 +1,357 @@
+/*
+ * Copyright 2015 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#ifndef POLARIS10_HWMGR_H
+#define POLARIS10_HWMGR_H
+
+#include "hwmgr.h"
+#include "smu74.h"
+#include "smu74_discrete.h"
+#include "ppatomctrl.h"
+#include "polaris10_ppsmc.h"
+#include "polaris10_powertune.h"
+
+#define POLARIS10_MAX_HARDWARE_POWERLEVELS 2
+
+#define POLARIS10_VOLTAGE_CONTROL_NONE 0x0
+#define POLARIS10_VOLTAGE_CONTROL_BY_GPIO 0x1
+#define POLARIS10_VOLTAGE_CONTROL_BY_SVID2 0x2
+#define POLARIS10_VOLTAGE_CONTROL_MERGED 0x3
+
+#define DPMTABLE_OD_UPDATE_SCLK 0x00000001
+#define DPMTABLE_OD_UPDATE_MCLK 0x00000002
+#define DPMTABLE_UPDATE_SCLK 0x00000004
+#define DPMTABLE_UPDATE_MCLK 0x00000008
+
+struct polaris10_performance_level {
+ uint32_t memory_clock;
+ uint32_t engine_clock;
+ uint16_t pcie_gen;
+ uint16_t pcie_lane;
+};
+
+struct polaris10_uvd_clocks {
+ uint32_t vclk;
+ uint32_t dclk;
+};
+
+struct polaris10_vce_clocks {
+ uint32_t evclk;
+ uint32_t ecclk;
+};
+
+struct polaris10_power_state {
+ uint32_t magic;
+ struct polaris10_uvd_clocks uvd_clks;
+ struct polaris10_vce_clocks vce_clks;
+ uint32_t sam_clk;
+ uint16_t performance_level_count;
+ bool dc_compatible;
+ uint32_t sclk_threshold;
+ struct polaris10_performance_level performance_levels[POLARIS10_MAX_HARDWARE_POWERLEVELS];
+};
+
+struct polaris10_dpm_level {
+ bool enabled;
+ uint32_t value;
+ uint32_t param1;
+};
+
+#define POLARIS10_MAX_DEEPSLEEP_DIVIDER_ID 5
+#define MAX_REGULAR_DPM_NUMBER 8
+#define POLARIS10_MINIMUM_ENGINE_CLOCK 2500
+
+struct polaris10_single_dpm_table {
+ uint32_t count;
+ struct polaris10_dpm_level dpm_levels[MAX_REGULAR_DPM_NUMBER];
+};
+
+struct polaris10_dpm_table {
+ struct polaris10_single_dpm_table sclk_table;
+ struct polaris10_single_dpm_table mclk_table;
+ struct polaris10_single_dpm_table pcie_speed_table;
+ struct polaris10_single_dpm_table vddc_table;
+ struct polaris10_single_dpm_table vddci_table;
+ struct polaris10_single_dpm_table mvdd_table;
+};
+
+struct polaris10_clock_registers {
+ uint32_t vCG_SPLL_FUNC_CNTL;
+ uint32_t vCG_SPLL_FUNC_CNTL_2;
+ uint32_t vCG_SPLL_FUNC_CNTL_3;
+ uint32_t vCG_SPLL_FUNC_CNTL_4;
+ uint32_t vCG_SPLL_SPREAD_SPECTRUM;
+ uint32_t vCG_SPLL_SPREAD_SPECTRUM_2;
+ uint32_t vDLL_CNTL;
+ uint32_t vMCLK_PWRMGT_CNTL;
+ uint32_t vMPLL_AD_FUNC_CNTL;
+ uint32_t vMPLL_DQ_FUNC_CNTL;
+ uint32_t vMPLL_FUNC_CNTL;
+ uint32_t vMPLL_FUNC_CNTL_1;
+ uint32_t vMPLL_FUNC_CNTL_2;
+ uint32_t vMPLL_SS1;
+ uint32_t vMPLL_SS2;
+};
+
+#define DISABLE_MC_LOADMICROCODE 1
+#define DISABLE_MC_CFGPROGRAMMING 2
+
+struct polaris10_voltage_smio_registers {
+ uint32_t vS0_VID_LOWER_SMIO_CNTL;
+};
+
+#define POLARIS10_MAX_LEAKAGE_COUNT 8
+
+struct polaris10_leakage_voltage {
+ uint16_t count;
+ uint16_t leakage_id[POLARIS10_MAX_LEAKAGE_COUNT];
+ uint16_t actual_voltage[POLARIS10_MAX_LEAKAGE_COUNT];
+};
+
+struct polaris10_vbios_boot_state {
+ uint16_t mvdd_bootup_value;
+ uint16_t vddc_bootup_value;
+ uint16_t vddci_bootup_value;
+ uint32_t sclk_bootup_value;
+ uint32_t mclk_bootup_value;
+ uint16_t pcie_gen_bootup_value;
+ uint16_t pcie_lane_bootup_value;
+};
+
+/* Ultra Low Voltage parameter structure */
+struct polaris10_ulv_parm {
+ bool ulv_supported;
+ uint32_t cg_ulv_parameter;
+ uint32_t ulv_volt_change_delay;
+ struct polaris10_performance_level ulv_power_level;
+};
+
+struct polaris10_display_timing {
+ uint32_t min_clock_in_sr;
+ uint32_t num_existing_displays;
+};
+
+struct polaris10_dpmlevel_enable_mask {
+ uint32_t uvd_dpm_enable_mask;
+ uint32_t vce_dpm_enable_mask;
+ uint32_t acp_dpm_enable_mask;
+ uint32_t samu_dpm_enable_mask;
+ uint32_t sclk_dpm_enable_mask;
+ uint32_t mclk_dpm_enable_mask;
+ uint32_t pcie_dpm_enable_mask;
+};
+
+struct polaris10_pcie_perf_range {
+ uint16_t max;
+ uint16_t min;
+};
+struct polaris10_range_table {
+ uint32_t trans_lower_frequency; /* in 10khz */
+ uint32_t trans_upper_frequency;
+};
+
+struct polaris10_hwmgr {
+ struct polaris10_dpm_table dpm_table;
+ struct polaris10_dpm_table golden_dpm_table;
+ SMU74_Discrete_DpmTable smc_state_table;
+ struct SMU74_Discrete_Ulv ulv_setting;
+
+ struct polaris10_range_table range_table[NUM_SCLK_RANGE];
+ uint32_t voting_rights_clients0;
+ uint32_t voting_rights_clients1;
+ uint32_t voting_rights_clients2;
+ uint32_t voting_rights_clients3;
+ uint32_t voting_rights_clients4;
+ uint32_t voting_rights_clients5;
+ uint32_t voting_rights_clients6;
+ uint32_t voting_rights_clients7;
+ uint32_t static_screen_threshold_unit;
+ uint32_t static_screen_threshold;
+ uint32_t voltage_control;
+ uint32_t vddc_vddci_delta;
+
+ uint32_t active_auto_throttle_sources;
+
+ struct polaris10_clock_registers clock_registers;
+ struct polaris10_voltage_smio_registers voltage_smio_registers;
+
+ bool is_memory_gddr5;
+ uint16_t acpi_vddc;
+ bool pspp_notify_required;
+ uint16_t force_pcie_gen;
+ uint16_t acpi_pcie_gen;
+ uint32_t pcie_gen_cap;
+ uint32_t pcie_lane_cap;
+ uint32_t pcie_spc_cap;
+ struct polaris10_leakage_voltage vddc_leakage;
+ struct polaris10_leakage_voltage Vddci_leakage;
+
+ uint32_t mvdd_control;
+ uint32_t vddc_mask_low;
+ uint32_t mvdd_mask_low;
+ uint16_t max_vddc_in_pptable;
+ uint16_t min_vddc_in_pptable;
+ uint16_t max_vddci_in_pptable;
+ uint16_t min_vddci_in_pptable;
+ uint32_t mclk_strobe_mode_threshold;
+ uint32_t mclk_stutter_mode_threshold;
+ uint32_t mclk_edc_enable_threshold;
+ uint32_t mclk_edcwr_enable_threshold;
+ bool is_uvd_enabled;
+ struct polaris10_vbios_boot_state vbios_boot_state;
+
+ bool pcie_performance_request;
+ bool battery_state;
+ bool is_tlu_enabled;
+
+ /* ---- SMC SRAM Address of firmware header tables ---- */
+ uint32_t sram_end;
+ uint32_t dpm_table_start;
+ uint32_t soft_regs_start;
+ uint32_t mc_reg_table_start;
+ uint32_t fan_table_start;
+ uint32_t arb_table_start;
+
+ /* ---- Stuff originally coming from Evergreen ---- */
+ uint32_t vddci_control;
+ struct pp_atomctrl_voltage_table vddc_voltage_table;
+ struct pp_atomctrl_voltage_table vddci_voltage_table;
+ struct pp_atomctrl_voltage_table mvdd_voltage_table;
+
+ uint32_t mgcg_cgtt_local2;
+ uint32_t mgcg_cgtt_local3;
+ uint32_t gpio_debug;
+ uint32_t mc_micro_code_feature;
+ uint32_t highest_mclk;
+ uint16_t acpi_vddci;
+ uint8_t mvdd_high_index;
+ uint8_t mvdd_low_index;
+ bool dll_default_on;
+ bool performance_request_registered;
+
+ /* ---- Low Power Features ---- */
+ struct polaris10_ulv_parm ulv;
+
+ /* ---- CAC Stuff ---- */
+ uint32_t cac_table_start;
+ bool cac_configuration_required;
+ bool driver_calculate_cac_leakage;
+ bool cac_enabled;
+
+ /* ---- DPM2 Parameters ---- */
+ uint32_t power_containment_features;
+ bool enable_dte_feature;
+ bool enable_tdc_limit_feature;
+ bool enable_pkg_pwr_tracking_feature;
+ bool disable_uvd_power_tune_feature;
+ const struct polaris10_pt_defaults *power_tune_defaults;
+ struct SMU74_Discrete_PmFuses power_tune_table;
+ uint32_t dte_tj_offset;
+ uint32_t fast_watermark_threshold;
+
+ /* ---- Phase Shedding ---- */
+ bool vddc_phase_shed_control;
+
+ /* ---- DI/DT ---- */
+ struct polaris10_display_timing display_timing;
+ uint32_t bif_sclk_table[SMU74_MAX_LEVELS_LINK];
+
+ /* ---- Thermal Temperature Setting ---- */
+ struct polaris10_dpmlevel_enable_mask dpm_level_enable_mask;
+ uint32_t need_update_smu7_dpm_table;
+ uint32_t sclk_dpm_key_disabled;
+ uint32_t mclk_dpm_key_disabled;
+ uint32_t pcie_dpm_key_disabled;
+ uint32_t min_engine_clocks;
+ struct polaris10_pcie_perf_range pcie_gen_performance;
+ struct polaris10_pcie_perf_range pcie_lane_performance;
+ struct polaris10_pcie_perf_range pcie_gen_power_saving;
+ struct polaris10_pcie_perf_range pcie_lane_power_saving;
+ bool use_pcie_performance_levels;
+ bool use_pcie_power_saving_levels;
+ uint32_t activity_target[SMU74_MAX_LEVELS_GRAPHICS];
+ uint32_t mclk_activity_target;
+ uint32_t mclk_dpm0_activity_target;
+ uint32_t low_sclk_interrupt_threshold;
+ uint32_t last_mclk_dpm_enable_mask;
+ bool uvd_enabled;
+
+ /* ---- Power Gating States ---- */
+ bool uvd_power_gated;
+ bool vce_power_gated;
+ bool samu_power_gated;
+ bool need_long_memory_training;
+
+ /* Application power optimization parameters */
+ bool update_up_hyst;
+ bool update_down_hyst;
+ uint32_t down_hyst;
+ uint32_t up_hyst;
+ uint32_t disable_dpm_mask;
+ bool apply_optimized_settings;
+
+ /* soft pptable for re-uploading into smu */
+ void *soft_pp_table;
+};
+
+/* To convert to Q8.8 format for firmware */
+#define POLARIS10_Q88_FORMAT_CONVERSION_UNIT 256
+
+enum Polaris10_I2CLineID {
+ Polaris10_I2CLineID_DDC1 = 0x90,
+ Polaris10_I2CLineID_DDC2 = 0x91,
+ Polaris10_I2CLineID_DDC3 = 0x92,
+ Polaris10_I2CLineID_DDC4 = 0x93,
+ Polaris10_I2CLineID_DDC5 = 0x94,
+ Polaris10_I2CLineID_DDC6 = 0x95,
+ Polaris10_I2CLineID_SCLSDA = 0x96,
+ Polaris10_I2CLineID_DDCVGA = 0x97
+};
+
+#define POLARIS10_I2C_DDC1DATA 0
+#define POLARIS10_I2C_DDC1CLK 1
+#define POLARIS10_I2C_DDC2DATA 2
+#define POLARIS10_I2C_DDC2CLK 3
+#define POLARIS10_I2C_DDC3DATA 4
+#define POLARIS10_I2C_DDC3CLK 5
+#define POLARIS10_I2C_SDA 40
+#define POLARIS10_I2C_SCL 41
+#define POLARIS10_I2C_DDC4DATA 65
+#define POLARIS10_I2C_DDC4CLK 66
+#define POLARIS10_I2C_DDC5DATA 0x48
+#define POLARIS10_I2C_DDC5CLK 0x49
+#define POLARIS10_I2C_DDC6DATA 0x4a
+#define POLARIS10_I2C_DDC6CLK 0x4b
+#define POLARIS10_I2C_DDCVGADATA 0x4c
+#define POLARIS10_I2C_DDCVGACLK 0x4d
+
+#define POLARIS10_UNUSED_GPIO_PIN 0x7F
+
+int polaris10_hwmgr_init(struct pp_hwmgr *hwmgr);
+
+int polaris10_update_uvd_dpm(struct pp_hwmgr *hwmgr, bool bgate);
+int polaris10_update_samu_dpm(struct pp_hwmgr *hwmgr, bool bgate);
+int polaris10_enable_disable_vce_dpm(struct pp_hwmgr *hwmgr, bool enable);
+
+#endif
+
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_powertune.c b/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_powertune.c
new file mode 100644
index 0000000..0b99ab3
--- /dev/null
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_powertune.c
@@ -0,0 +1,398 @@
+/*
+ * Copyright 2015 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#include "hwmgr.h"
+#include "smumgr.h"
+#include "polaris10_hwmgr.h"
+#include "polaris10_powertune.h"
+#include "polaris10_smumgr.h"
+#include "smu74_discrete.h"
+#include "pp_debug.h"
+
+#define VOLTAGE_SCALE 4
+#define POWERTUNE_DEFAULT_SET_MAX 1
+
+static const struct polaris10_pt_defaults polaris10_power_tune_data_set_array[POWERTUNE_DEFAULT_SET_MAX] = {
+ /* sviLoadLIneEn, SviLoadLineVddC, TDC_VDDC_ThrottleReleaseLimitPerc, TDC_MAWt,
+ * TdcWaterfallCtl, DTEAmbientTempBase, DisplayCac, BAPM_TEMP_GRADIENT */
+ { 1, 0xF, 0xFD, 0x19, 5, 45, 0, 0xB0000,
+ { 0x79, 0x253, 0x25D, 0xAE, 0x72, 0x80, 0x83, 0x86, 0x6F, 0xC8, 0xC9, 0xC9, 0x2F, 0x4D, 0x61},
+ { 0x17C, 0x172, 0x180, 0x1BC, 0x1B3, 0x1BD, 0x206, 0x200, 0x203, 0x25D, 0x25A, 0x255, 0x2C3, 0x2C5, 0x2B4 } },
+};
+
+void polaris10_initialize_power_tune_defaults(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *polaris10_hwmgr = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+
+ if (table_info &&
+ table_info->cac_dtp_table->usPowerTuneDataSetID <= POWERTUNE_DEFAULT_SET_MAX &&
+ table_info->cac_dtp_table->usPowerTuneDataSetID)
+ polaris10_hwmgr->power_tune_defaults =
+ &polaris10_power_tune_data_set_array
+ [table_info->cac_dtp_table->usPowerTuneDataSetID - 1];
+ else
+ polaris10_hwmgr->power_tune_defaults = &polaris10_power_tune_data_set_array[0];
+
+}
+
+static uint16_t scale_fan_gain_settings(uint16_t raw_setting)
+{
+ uint32_t tmp;
+ tmp = raw_setting * 4096 / 100;
+ return (uint16_t)tmp;
+}
+
+int polaris10_populate_bapm_parameters_in_dpm_table(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ const struct polaris10_pt_defaults *defaults = data->power_tune_defaults;
+ SMU74_Discrete_DpmTable *dpm_table = &(data->smc_state_table);
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+ struct phm_cac_tdp_table *cac_dtp_table = table_info->cac_dtp_table;
+ struct pp_advance_fan_control_parameters *fan_table=
+ &hwmgr->thermal_controller.advanceFanControlParameters;
+ int i, j, k;
+ const uint16_t *pdef1;
+ const uint16_t *pdef2;
+
+ dpm_table->DefaultTdp = PP_HOST_TO_SMC_US((uint16_t)(cac_dtp_table->usTDP * 128));
+ dpm_table->TargetTdp = PP_HOST_TO_SMC_US((uint16_t)(cac_dtp_table->usTDP * 128));
+
+ PP_ASSERT_WITH_CODE(cac_dtp_table->usTargetOperatingTemp <= 255,
+ "Target Operating Temp is out of Range!",
+ );
+
+ dpm_table->TemperatureLimitEdge = PP_HOST_TO_SMC_US(
+ cac_dtp_table->usTargetOperatingTemp * 256);
+ dpm_table->TemperatureLimitHotspot = PP_HOST_TO_SMC_US(
+ cac_dtp_table->usTemperatureLimitHotspot * 256);
+ dpm_table->FanGainEdge = PP_HOST_TO_SMC_US(
+ scale_fan_gain_settings(fan_table->usFanGainEdge));
+ dpm_table->FanGainHotspot = PP_HOST_TO_SMC_US(
+ scale_fan_gain_settings(fan_table->usFanGainHotspot));
+
+ pdef1 = defaults->BAPMTI_R;
+ pdef2 = defaults->BAPMTI_RC;
+
+ for (i = 0; i < SMU74_DTE_ITERATIONS; i++) {
+ for (j = 0; j < SMU74_DTE_SOURCES; j++) {
+ for (k = 0; k < SMU74_DTE_SINKS; k++) {
+ dpm_table->BAPMTI_R[i][j][k] = PP_HOST_TO_SMC_US(*pdef1);
+ dpm_table->BAPMTI_RC[i][j][k] = PP_HOST_TO_SMC_US(*pdef2);
+ pdef1++;
+ pdef2++;
+ }
+ }
+ }
+
+ return 0;
+}
+
+static int polaris10_populate_svi_load_line(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ const struct polaris10_pt_defaults *defaults = data->power_tune_defaults;
+
+ data->power_tune_table.SviLoadLineEn = defaults->SviLoadLineEn;
+ data->power_tune_table.SviLoadLineVddC = defaults->SviLoadLineVddC;
+ data->power_tune_table.SviLoadLineTrimVddC = 3;
+ data->power_tune_table.SviLoadLineOffsetVddC = 0;
+
+ return 0;
+}
+
+static int polaris10_populate_tdc_limit(struct pp_hwmgr *hwmgr)
+{
+ uint16_t tdc_limit;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+ const struct polaris10_pt_defaults *defaults = data->power_tune_defaults;
+
+ tdc_limit = (uint16_t)(table_info->cac_dtp_table->usTDC * 128);
+ data->power_tune_table.TDC_VDDC_PkgLimit =
+ CONVERT_FROM_HOST_TO_SMC_US(tdc_limit);
+ data->power_tune_table.TDC_VDDC_ThrottleReleaseLimitPerc =
+ defaults->TDC_VDDC_ThrottleReleaseLimitPerc;
+ data->power_tune_table.TDC_MAWt = defaults->TDC_MAWt;
+
+ return 0;
+}
+
+static int polaris10_populate_dw8(struct pp_hwmgr *hwmgr, uint32_t fuse_table_offset)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ const struct polaris10_pt_defaults *defaults = data->power_tune_defaults;
+ uint32_t temp;
+
+ if (polaris10_read_smc_sram_dword(hwmgr->smumgr,
+ fuse_table_offset +
+ offsetof(SMU74_Discrete_PmFuses, TdcWaterfallCtl),
+ (uint32_t *)&temp, data->sram_end))
+ PP_ASSERT_WITH_CODE(false,
+ "Attempt to read PmFuses.DW6 (SviLoadLineEn) from SMC Failed!",
+ return -EINVAL);
+ else {
+ data->power_tune_table.TdcWaterfallCtl = defaults->TdcWaterfallCtl;
+ data->power_tune_table.LPMLTemperatureMin =
+ (uint8_t)((temp >> 16) & 0xff);
+ data->power_tune_table.LPMLTemperatureMax =
+ (uint8_t)((temp >> 8) & 0xff);
+ data->power_tune_table.Reserved = (uint8_t)(temp & 0xff);
+ }
+ return 0;
+}
+
+static int polaris10_populate_temperature_scaler(struct pp_hwmgr *hwmgr)
+{
+ int i;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ /* Currently not used. Set all to zero. */
+ for (i = 0; i < 16; i++)
+ data->power_tune_table.LPMLTemperatureScaler[i] = 0;
+
+ return 0;
+}
+
+static int polaris10_populate_fuzzy_fan(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ if ((hwmgr->thermal_controller.advanceFanControlParameters.usFanOutputSensitivity & (1 << 15))
+ || 0 == hwmgr->thermal_controller.advanceFanControlParameters.usFanOutputSensitivity)
+ hwmgr->thermal_controller.advanceFanControlParameters.usFanOutputSensitivity =
+ hwmgr->thermal_controller.advanceFanControlParameters.usDefaultFanOutputSensitivity;
+
+ data->power_tune_table.FuzzyFan_PwmSetDelta = PP_HOST_TO_SMC_US(
+ hwmgr->thermal_controller.advanceFanControlParameters.usFanOutputSensitivity);
+ return 0;
+}
+
+static int polaris10_populate_gnb_lpml(struct pp_hwmgr *hwmgr)
+{
+ int i;
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ /* Currently not used. Set all to zero. */
+ for (i = 0; i < 16; i++)
+ data->power_tune_table.GnbLPML[i] = 0;
+
+ return 0;
+}
+
+static int polaris10_min_max_vgnb_lpml_id_from_bapm_vddc(struct pp_hwmgr *hwmgr)
+{
+ return 0;
+}
+
+static int polaris10_populate_bapm_vddc_base_leakage_sidd(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+ uint16_t hi_sidd = data->power_tune_table.BapmVddCBaseLeakageHiSidd;
+ uint16_t lo_sidd = data->power_tune_table.BapmVddCBaseLeakageLoSidd;
+ struct phm_cac_tdp_table *cac_table = table_info->cac_dtp_table;
+
+ hi_sidd = (uint16_t)(cac_table->usHighCACLeakage / 100 * 256);
+ lo_sidd = (uint16_t)(cac_table->usLowCACLeakage / 100 * 256);
+
+ data->power_tune_table.BapmVddCBaseLeakageHiSidd =
+ CONVERT_FROM_HOST_TO_SMC_US(hi_sidd);
+ data->power_tune_table.BapmVddCBaseLeakageLoSidd =
+ CONVERT_FROM_HOST_TO_SMC_US(lo_sidd);
+
+ return 0;
+}
+
+int polaris10_populate_pm_fuses(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ uint32_t pm_fuse_table_offset;
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_PowerContainment)) {
+ if (polaris10_read_smc_sram_dword(hwmgr->smumgr,
+ SMU7_FIRMWARE_HEADER_LOCATION +
+ offsetof(SMU74_Firmware_Header, PmFuseTable),
+ &pm_fuse_table_offset, data->sram_end))
+ PP_ASSERT_WITH_CODE(false,
+ "Attempt to get pm_fuse_table_offset Failed!",
+ return -EINVAL);
+
+ if (polaris10_populate_svi_load_line(hwmgr))
+ PP_ASSERT_WITH_CODE(false,
+ "Attempt to populate SviLoadLine Failed!",
+ return -EINVAL);
+
+ if (polaris10_populate_tdc_limit(hwmgr))
+ PP_ASSERT_WITH_CODE(false,
+ "Attempt to populate TDCLimit Failed!", return -EINVAL);
+
+ if (polaris10_populate_dw8(hwmgr, pm_fuse_table_offset))
+ PP_ASSERT_WITH_CODE(false,
+ "Attempt to populate TdcWaterfallCtl, "
+ "LPMLTemperature Min and Max Failed!",
+ return -EINVAL);
+
+ if (0 != polaris10_populate_temperature_scaler(hwmgr))
+ PP_ASSERT_WITH_CODE(false,
+ "Attempt to populate LPMLTemperatureScaler Failed!",
+ return -EINVAL);
+
+ if (polaris10_populate_fuzzy_fan(hwmgr))
+ PP_ASSERT_WITH_CODE(false,
+ "Attempt to populate Fuzzy Fan Control parameters Failed!",
+ return -EINVAL);
+
+ if (polaris10_populate_gnb_lpml(hwmgr))
+ PP_ASSERT_WITH_CODE(false,
+ "Attempt to populate GnbLPML Failed!",
+ return -EINVAL);
+
+ if (polaris10_min_max_vgnb_lpml_id_from_bapm_vddc(hwmgr))
+ PP_ASSERT_WITH_CODE(false,
+ "Attempt to populate GnbLPML Min and Max Vid Failed!",
+ return -EINVAL);
+
+ if (polaris10_populate_bapm_vddc_base_leakage_sidd(hwmgr))
+ PP_ASSERT_WITH_CODE(false,
+ "Attempt to populate BapmVddCBaseLeakage Hi and Lo "
+ "Sidd Failed!", return -EINVAL);
+
+ if (polaris10_copy_bytes_to_smc(hwmgr->smumgr, pm_fuse_table_offset,
+ (uint8_t *)&data->power_tune_table,
+ sizeof(struct SMU74_Discrete_PmFuses), data->sram_end))
+ PP_ASSERT_WITH_CODE(false,
+ "Attempt to download PmFuseTable Failed!",
+ return -EINVAL);
+ }
+ return 0;
+}
+
+int polaris10_enable_smc_cac(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ int result = 0;
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_CAC)) {
+ int smc_result;
+ smc_result = smum_send_msg_to_smc(hwmgr->smumgr,
+ (uint16_t)(PPSMC_MSG_EnableCac));
+ PP_ASSERT_WITH_CODE((0 == smc_result),
+ "Failed to enable CAC in SMC.", result = -1);
+
+ data->cac_enabled = (0 == smc_result) ? true : false;
+ }
+ return result;
+}
+
+int polaris10_set_power_limit(struct pp_hwmgr *hwmgr, uint32_t n)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+
+ if (data->power_containment_features &
+ POWERCONTAINMENT_FEATURE_PkgPwrLimit)
+ return smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_PkgPwrSetLimit, n);
+ return 0;
+}
+
+static int polaris10_set_overdriver_target_tdp(struct pp_hwmgr *pHwMgr, uint32_t target_tdp)
+{
+ return smum_send_msg_to_smc_with_parameter(pHwMgr->smumgr,
+ PPSMC_MSG_OverDriveSetTargetTdp, target_tdp);
+}
+
+int polaris10_enable_power_containment(struct pp_hwmgr *hwmgr)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+ int smc_result;
+ int result = 0;
+
+ data->power_containment_features = 0;
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_PowerContainment)) {
+
+ if (data->enable_tdc_limit_feature) {
+ smc_result = smum_send_msg_to_smc(hwmgr->smumgr,
+ (uint16_t)(PPSMC_MSG_TDCLimitEnable));
+ PP_ASSERT_WITH_CODE((0 == smc_result),
+ "Failed to enable TDCLimit in SMC.", result = -1;);
+ if (0 == smc_result)
+ data->power_containment_features |=
+ POWERCONTAINMENT_FEATURE_TDCLimit;
+ }
+
+ if (data->enable_pkg_pwr_tracking_feature) {
+ smc_result = smum_send_msg_to_smc(hwmgr->smumgr,
+ (uint16_t)(PPSMC_MSG_PkgPwrLimitEnable));
+ PP_ASSERT_WITH_CODE((0 == smc_result),
+ "Failed to enable PkgPwrTracking in SMC.", result = -1;);
+ if (0 == smc_result) {
+ struct phm_cac_tdp_table *cac_table =
+ table_info->cac_dtp_table;
+ uint32_t default_limit =
+ (uint32_t)(cac_table->usMaximumPowerDeliveryLimit * 256);
+
+ data->power_containment_features |=
+ POWERCONTAINMENT_FEATURE_PkgPwrLimit;
+
+ if (polaris10_set_power_limit(hwmgr, default_limit))
+ printk(KERN_ERR "Failed to set Default Power Limit in SMC!");
+ }
+ }
+ }
+ return result;
+}
+
+int polaris10_power_control_set_level(struct pp_hwmgr *hwmgr)
+{
+ struct phm_ppt_v1_information *table_info =
+ (struct phm_ppt_v1_information *)(hwmgr->pptable);
+ struct phm_cac_tdp_table *cac_table = table_info->cac_dtp_table;
+ int adjust_percent, target_tdp;
+ int result = 0;
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_PowerContainment)) {
+ /* adjustment percentage has already been validated */
+ adjust_percent = hwmgr->platform_descriptor.TDPAdjustmentPolarity ?
+ hwmgr->platform_descriptor.TDPAdjustment :
+ (-1 * hwmgr->platform_descriptor.TDPAdjustment);
+ /* SMC requested that target_tdp to be 7 bit fraction in DPM table
+ * but message to be 8 bit fraction for messages
+ */
+ target_tdp = ((100 + adjust_percent) * (int)(cac_table->usTDP * 256)) / 100;
+ result = polaris10_set_overdriver_target_tdp(hwmgr, (uint32_t)target_tdp);
+ }
+
+ return result;
+}
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_powertune.h b/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_powertune.h
new file mode 100644
index 0000000..68bc1cb
--- /dev/null
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_powertune.h
@@ -0,0 +1,70 @@
+/*
+ * Copyright 2015 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+#ifndef POLARIS10_POWERTUNE_H
+#define POLARIS10_POWERTUNE_H
+
+enum polaris10_pt_config_reg_type {
+ POLARIS10_CONFIGREG_MMR = 0,
+ POLARIS10_CONFIGREG_SMC_IND,
+ POLARIS10_CONFIGREG_DIDT_IND,
+ POLARIS10_CONFIGREG_CACHE,
+ POLARIS10_CONFIGREG_MAX
+};
+
+/* PowerContainment Features */
+#define POWERCONTAINMENT_FEATURE_DTE 0x00000001
+#define POWERCONTAINMENT_FEATURE_TDCLimit 0x00000002
+#define POWERCONTAINMENT_FEATURE_PkgPwrLimit 0x00000004
+
+struct polaris10_pt_config_reg {
+ uint32_t offset;
+ uint32_t mask;
+ uint32_t shift;
+ uint32_t value;
+ enum polaris10_pt_config_reg_type type;
+};
+
+struct polaris10_pt_defaults {
+ uint8_t SviLoadLineEn;
+ uint8_t SviLoadLineVddC;
+ uint8_t TDC_VDDC_ThrottleReleaseLimitPerc;
+ uint8_t TDC_MAWt;
+ uint8_t TdcWaterfallCtl;
+ uint8_t DTEAmbientTempBase;
+
+ uint32_t DisplayCac;
+ uint32_t BAPM_TEMP_GRADIENT;
+ uint16_t BAPMTI_R[SMU74_DTE_ITERATIONS * SMU74_DTE_SOURCES * SMU74_DTE_SINKS];
+ uint16_t BAPMTI_RC[SMU74_DTE_ITERATIONS * SMU74_DTE_SOURCES * SMU74_DTE_SINKS];
+};
+
+void polaris10_initialize_power_tune_defaults(struct pp_hwmgr *hwmgr);
+int polaris10_populate_bapm_parameters_in_dpm_table(struct pp_hwmgr *hwmgr);
+int polaris10_populate_pm_fuses(struct pp_hwmgr *hwmgr);
+int polaris10_enable_smc_cac(struct pp_hwmgr *hwmgr);
+int polaris10_enable_power_containment(struct pp_hwmgr *hwmgr);
+int polaris10_set_power_limit(struct pp_hwmgr *hwmgr, uint32_t n);
+int polaris10_power_control_set_level(struct pp_hwmgr *hwmgr);
+
+#endif /* POLARIS10_POWERTUNE_H */
+
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_thermal.c b/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_thermal.c
new file mode 100644
index 0000000..aba167f
--- /dev/null
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_thermal.c
@@ -0,0 +1,712 @@
+/*
+ * Copyright 2016 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#include <asm/div64.h>
+#include "polaris10_thermal.h"
+#include "polaris10_hwmgr.h"
+#include "polaris10_smumgr.h"
+#include "polaris10_ppsmc.h"
+#include "smu/smu_7_1_3_d.h"
+#include "smu/smu_7_1_3_sh_mask.h"
+
+int polaris10_fan_ctrl_get_fan_speed_info(struct pp_hwmgr *hwmgr,
+ struct phm_fan_speed_info *fan_speed_info)
+{
+ if (hwmgr->thermal_controller.fanInfo.bNoFan)
+ return 0;
+
+ fan_speed_info->supports_percent_read = true;
+ fan_speed_info->supports_percent_write = true;
+ fan_speed_info->min_percent = 0;
+ fan_speed_info->max_percent = 100;
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_FanSpeedInTableIsRPM) &&
+ hwmgr->thermal_controller.fanInfo.ucTachometerPulsesPerRevolution) {
+ fan_speed_info->supports_rpm_read = true;
+ fan_speed_info->supports_rpm_write = true;
+ fan_speed_info->min_rpm = hwmgr->thermal_controller.fanInfo.ulMinRPM;
+ fan_speed_info->max_rpm = hwmgr->thermal_controller.fanInfo.ulMaxRPM;
+ } else {
+ fan_speed_info->min_rpm = 0;
+ fan_speed_info->max_rpm = 0;
+ }
+
+ return 0;
+}
+
+int polaris10_fan_ctrl_get_fan_speed_percent(struct pp_hwmgr *hwmgr,
+ uint32_t *speed)
+{
+ uint32_t duty100;
+ uint32_t duty;
+ uint64_t tmp64;
+
+ if (hwmgr->thermal_controller.fanInfo.bNoFan)
+ return 0;
+
+ duty100 = PHM_READ_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_FDO_CTRL1, FMAX_DUTY100);
+ duty = PHM_READ_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_THERMAL_STATUS, FDO_PWM_DUTY);
+
+ if (duty100 == 0)
+ return -EINVAL;
+
+
+ tmp64 = (uint64_t)duty * 100;
+ do_div(tmp64, duty100);
+ *speed = (uint32_t)tmp64;
+
+ if (*speed > 100)
+ *speed = 100;
+
+ return 0;
+}
+
+int polaris10_fan_ctrl_get_fan_speed_rpm(struct pp_hwmgr *hwmgr, uint32_t *speed)
+{
+ uint32_t tach_period;
+ uint32_t crystal_clock_freq;
+
+ if (hwmgr->thermal_controller.fanInfo.bNoFan ||
+ (hwmgr->thermal_controller.fanInfo.
+ ucTachometerPulsesPerRevolution == 0))
+ return 0;
+
+ tach_period = PHM_READ_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_TACH_STATUS, TACH_PERIOD);
+
+ if (tach_period == 0)
+ return -EINVAL;
+
+ crystal_clock_freq = tonga_get_xclk(hwmgr);
+
+ *speed = 60 * crystal_clock_freq * 10000 / tach_period;
+
+ return 0;
+}
+
+/**
+* Set Fan Speed Control to static mode, so that the user can decide what speed to use.
+* @param hwmgr the address of the powerplay hardware manager.
+* mode the fan control mode, 0 default, 1 by percent, 5, by RPM
+* @exception Should always succeed.
+*/
+int polaris10_fan_ctrl_set_static_mode(struct pp_hwmgr *hwmgr, uint32_t mode)
+{
+
+ if (hwmgr->fan_ctrl_is_in_default_mode) {
+ hwmgr->fan_ctrl_default_mode =
+ PHM_READ_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_FDO_CTRL2, FDO_PWM_MODE);
+ hwmgr->tmin =
+ PHM_READ_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_FDO_CTRL2, TMIN);
+ hwmgr->fan_ctrl_is_in_default_mode = false;
+ }
+
+ PHM_WRITE_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_FDO_CTRL2, TMIN, 0);
+ PHM_WRITE_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_FDO_CTRL2, FDO_PWM_MODE, mode);
+
+ return 0;
+}
+
+/**
+* Reset Fan Speed Control to default mode.
+* @param hwmgr the address of the powerplay hardware manager.
+* @exception Should always succeed.
+*/
+int polaris10_fan_ctrl_set_default_mode(struct pp_hwmgr *hwmgr)
+{
+ if (!hwmgr->fan_ctrl_is_in_default_mode) {
+ PHM_WRITE_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_FDO_CTRL2, FDO_PWM_MODE, hwmgr->fan_ctrl_default_mode);
+ PHM_WRITE_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_FDO_CTRL2, TMIN, hwmgr->tmin);
+ hwmgr->fan_ctrl_is_in_default_mode = true;
+ }
+
+ return 0;
+}
+
+int polaris10_fan_ctrl_start_smc_fan_control(struct pp_hwmgr *hwmgr)
+{
+ int result;
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_ODFuzzyFanControlSupport)) {
+ cgs_write_register(hwmgr->device, mmSMC_MSG_ARG_0, FAN_CONTROL_FUZZY);
+ result = smum_send_msg_to_smc(hwmgr->smumgr, PPSMC_StartFanControl);
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_FanSpeedInTableIsRPM))
+ hwmgr->hwmgr_func->set_max_fan_rpm_output(hwmgr,
+ hwmgr->thermal_controller.
+ advanceFanControlParameters.usMaxFanRPM);
+ else
+ hwmgr->hwmgr_func->set_max_fan_pwm_output(hwmgr,
+ hwmgr->thermal_controller.
+ advanceFanControlParameters.usMaxFanPWM);
+
+ } else {
+ cgs_write_register(hwmgr->device, mmSMC_MSG_ARG_0, FAN_CONTROL_TABLE);
+ result = smum_send_msg_to_smc(hwmgr->smumgr, PPSMC_StartFanControl);
+ }
+
+ if (!result && hwmgr->thermal_controller.
+ advanceFanControlParameters.ucTargetTemperature)
+ result = smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_SetFanTemperatureTarget,
+ hwmgr->thermal_controller.
+ advanceFanControlParameters.ucTargetTemperature);
+
+ return result;
+}
+
+
+int polaris10_fan_ctrl_stop_smc_fan_control(struct pp_hwmgr *hwmgr)
+{
+ return smum_send_msg_to_smc(hwmgr->smumgr, PPSMC_StopFanControl);
+}
+
+/**
+* Set Fan Speed in percent.
+* @param hwmgr the address of the powerplay hardware manager.
+* @param speed is the percentage value (0% - 100%) to be set.
+* @exception Fails is the 100% setting appears to be 0.
+*/
+int polaris10_fan_ctrl_set_fan_speed_percent(struct pp_hwmgr *hwmgr,
+ uint32_t speed)
+{
+ uint32_t duty100;
+ uint32_t duty;
+ uint64_t tmp64;
+
+ if (hwmgr->thermal_controller.fanInfo.bNoFan)
+ return 0;
+
+ if (speed > 100)
+ speed = 100;
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_MicrocodeFanControl))
+ polaris10_fan_ctrl_stop_smc_fan_control(hwmgr);
+
+ duty100 = PHM_READ_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_FDO_CTRL1, FMAX_DUTY100);
+
+ if (duty100 == 0)
+ return -EINVAL;
+
+ tmp64 = (uint64_t)speed * duty100;
+ do_div(tmp64, 100);
+ duty = (uint32_t)tmp64;
+
+ PHM_WRITE_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_FDO_CTRL0, FDO_STATIC_DUTY, duty);
+
+ return polaris10_fan_ctrl_set_static_mode(hwmgr, FDO_PWM_MODE_STATIC);
+}
+
+/**
+* Reset Fan Speed to default.
+* @param hwmgr the address of the powerplay hardware manager.
+* @exception Always succeeds.
+*/
+int polaris10_fan_ctrl_reset_fan_speed_to_default(struct pp_hwmgr *hwmgr)
+{
+ int result;
+
+ if (hwmgr->thermal_controller.fanInfo.bNoFan)
+ return 0;
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_MicrocodeFanControl)) {
+ result = polaris10_fan_ctrl_set_static_mode(hwmgr, FDO_PWM_MODE_STATIC);
+ if (!result)
+ result = polaris10_fan_ctrl_start_smc_fan_control(hwmgr);
+ } else
+ result = polaris10_fan_ctrl_set_default_mode(hwmgr);
+
+ return result;
+}
+
+/**
+* Set Fan Speed in RPM.
+* @param hwmgr the address of the powerplay hardware manager.
+* @param speed is the percentage value (min - max) to be set.
+* @exception Fails is the speed not lie between min and max.
+*/
+int polaris10_fan_ctrl_set_fan_speed_rpm(struct pp_hwmgr *hwmgr, uint32_t speed)
+{
+ uint32_t tach_period;
+ uint32_t crystal_clock_freq;
+
+ if (hwmgr->thermal_controller.fanInfo.bNoFan ||
+ (hwmgr->thermal_controller.fanInfo.
+ ucTachometerPulsesPerRevolution == 0) ||
+ (speed < hwmgr->thermal_controller.fanInfo.ulMinRPM) ||
+ (speed > hwmgr->thermal_controller.fanInfo.ulMaxRPM))
+ return 0;
+
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_MicrocodeFanControl))
+ polaris10_fan_ctrl_stop_smc_fan_control(hwmgr);
+
+ crystal_clock_freq = tonga_get_xclk(hwmgr);
+
+ tach_period = 60 * crystal_clock_freq * 10000 / (8 * speed);
+
+ PHM_WRITE_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_TACH_STATUS, TACH_PERIOD, tach_period);
+
+ return polaris10_fan_ctrl_set_static_mode(hwmgr, FDO_PWM_MODE_STATIC);
+}
+
+/**
+* Reads the remote temperature from the SIslands thermal controller.
+*
+* @param hwmgr The address of the hardware manager.
+*/
+int polaris10_thermal_get_temperature(struct pp_hwmgr *hwmgr)
+{
+ int temp;
+
+ temp = PHM_READ_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_MULT_THERMAL_STATUS, CTF_TEMP);
+
+ /* Bit 9 means the reading is lower than the lowest usable value. */
+ if (temp & 0x200)
+ temp = POLARIS10_THERMAL_MAXIMUM_TEMP_READING;
+ else
+ temp = temp & 0x1ff;
+
+ temp *= PP_TEMPERATURE_UNITS_PER_CENTIGRADES;
+
+ return temp;
+}
+
+/**
+* Set the requested temperature range for high and low alert signals
+*
+* @param hwmgr The address of the hardware manager.
+* @param range Temperature range to be programmed for high and low alert signals
+* @exception PP_Result_BadInput if the input data is not valid.
+*/
+static int polaris10_thermal_set_temperature_range(struct pp_hwmgr *hwmgr,
+ uint32_t low_temp, uint32_t high_temp)
+{
+ uint32_t low = POLARIS10_THERMAL_MINIMUM_ALERT_TEMP *
+ PP_TEMPERATURE_UNITS_PER_CENTIGRADES;
+ uint32_t high = POLARIS10_THERMAL_MAXIMUM_ALERT_TEMP *
+ PP_TEMPERATURE_UNITS_PER_CENTIGRADES;
+
+ if (low < low_temp)
+ low = low_temp;
+ if (high > high_temp)
+ high = high_temp;
+
+ if (low > high)
+ return -EINVAL;
+
+ PHM_WRITE_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_THERMAL_INT, DIG_THERM_INTH,
+ (high / PP_TEMPERATURE_UNITS_PER_CENTIGRADES));
+ PHM_WRITE_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_THERMAL_INT, DIG_THERM_INTL,
+ (low / PP_TEMPERATURE_UNITS_PER_CENTIGRADES));
+ PHM_WRITE_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_THERMAL_CTRL, DIG_THERM_DPM,
+ (high / PP_TEMPERATURE_UNITS_PER_CENTIGRADES));
+
+ return 0;
+}
+
+/**
+* Programs thermal controller one-time setting registers
+*
+* @param hwmgr The address of the hardware manager.
+*/
+static int polaris10_thermal_initialize(struct pp_hwmgr *hwmgr)
+{
+ if (hwmgr->thermal_controller.fanInfo.ucTachometerPulsesPerRevolution)
+ PHM_WRITE_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_TACH_CTRL, EDGE_PER_REV,
+ hwmgr->thermal_controller.fanInfo.
+ ucTachometerPulsesPerRevolution - 1);
+
+ PHM_WRITE_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_FDO_CTRL2, TACH_PWM_RESP_RATE, 0x28);
+
+ return 0;
+}
+
+/**
+* Enable thermal alerts on the RV770 thermal controller.
+*
+* @param hwmgr The address of the hardware manager.
+*/
+static int polaris10_thermal_enable_alert(struct pp_hwmgr *hwmgr)
+{
+ uint32_t alert;
+
+ alert = PHM_READ_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_THERMAL_INT, THERM_INT_MASK);
+ alert &= ~(POLARIS10_THERMAL_HIGH_ALERT_MASK | POLARIS10_THERMAL_LOW_ALERT_MASK);
+ PHM_WRITE_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_THERMAL_INT, THERM_INT_MASK, alert);
+
+ /* send message to SMU to enable internal thermal interrupts */
+ return smum_send_msg_to_smc(hwmgr->smumgr, PPSMC_MSG_Thermal_Cntl_Enable);
+}
+
+/**
+* Disable thermal alerts on the RV770 thermal controller.
+* @param hwmgr The address of the hardware manager.
+*/
+static int polaris10_thermal_disable_alert(struct pp_hwmgr *hwmgr)
+{
+ uint32_t alert;
+
+ alert = PHM_READ_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_THERMAL_INT, THERM_INT_MASK);
+ alert |= (POLARIS10_THERMAL_HIGH_ALERT_MASK | POLARIS10_THERMAL_LOW_ALERT_MASK);
+ PHM_WRITE_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_THERMAL_INT, THERM_INT_MASK, alert);
+
+ /* send message to SMU to disable internal thermal interrupts */
+ return smum_send_msg_to_smc(hwmgr->smumgr, PPSMC_MSG_Thermal_Cntl_Disable);
+}
+
+/**
+* Uninitialize the thermal controller.
+* Currently just disables alerts.
+* @param hwmgr The address of the hardware manager.
+*/
+int polaris10_thermal_stop_thermal_controller(struct pp_hwmgr *hwmgr)
+{
+ int result = polaris10_thermal_disable_alert(hwmgr);
+
+ if (!hwmgr->thermal_controller.fanInfo.bNoFan)
+ polaris10_fan_ctrl_set_default_mode(hwmgr);
+
+ return result;
+}
+
+/**
+* Set up the fan table to control the fan using the SMC.
+* @param hwmgr the address of the powerplay hardware manager.
+* @param pInput the pointer to input data
+* @param pOutput the pointer to output data
+* @param pStorage the pointer to temporary storage
+* @param Result the last failure code
+* @return result from set temperature range routine
+*/
+int tf_polaris10_thermal_setup_fan_table(struct pp_hwmgr *hwmgr,
+ void *input, void *output, void *storage, int result)
+{
+ struct polaris10_hwmgr *data = (struct polaris10_hwmgr *)(hwmgr->backend);
+ SMU74_Discrete_FanTable fan_table = { FDO_MODE_HARDWARE };
+ uint32_t duty100;
+ uint32_t t_diff1, t_diff2, pwm_diff1, pwm_diff2;
+ uint16_t fdo_min, slope1, slope2;
+ uint32_t reference_clock;
+ int res;
+ uint64_t tmp64;
+
+ if (data->fan_table_start == 0) {
+ phm_cap_unset(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_MicrocodeFanControl);
+ return 0;
+ }
+
+ duty100 = PHM_READ_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC,
+ CG_FDO_CTRL1, FMAX_DUTY100);
+
+ if (duty100 == 0) {
+ phm_cap_unset(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_MicrocodeFanControl);
+ return 0;
+ }
+
+ tmp64 = hwmgr->thermal_controller.advanceFanControlParameters.
+ usPWMMin * duty100;
+ do_div(tmp64, 10000);
+ fdo_min = (uint16_t)tmp64;
+
+ t_diff1 = hwmgr->thermal_controller.advanceFanControlParameters.usTMed -
+ hwmgr->thermal_controller.advanceFanControlParameters.usTMin;
+ t_diff2 = hwmgr->thermal_controller.advanceFanControlParameters.usTHigh -
+ hwmgr->thermal_controller.advanceFanControlParameters.usTMed;
+
+ pwm_diff1 = hwmgr->thermal_controller.advanceFanControlParameters.usPWMMed -
+ hwmgr->thermal_controller.advanceFanControlParameters.usPWMMin;
+ pwm_diff2 = hwmgr->thermal_controller.advanceFanControlParameters.usPWMHigh -
+ hwmgr->thermal_controller.advanceFanControlParameters.usPWMMed;
+
+ slope1 = (uint16_t)((50 + ((16 * duty100 * pwm_diff1) / t_diff1)) / 100);
+ slope2 = (uint16_t)((50 + ((16 * duty100 * pwm_diff2) / t_diff2)) / 100);
+
+ fan_table.TempMin = cpu_to_be16((50 + hwmgr->
+ thermal_controller.advanceFanControlParameters.usTMin) / 100);
+ fan_table.TempMed = cpu_to_be16((50 + hwmgr->
+ thermal_controller.advanceFanControlParameters.usTMed) / 100);
+ fan_table.TempMax = cpu_to_be16((50 + hwmgr->
+ thermal_controller.advanceFanControlParameters.usTMax) / 100);
+
+ fan_table.Slope1 = cpu_to_be16(slope1);
+ fan_table.Slope2 = cpu_to_be16(slope2);
+
+ fan_table.FdoMin = cpu_to_be16(fdo_min);
+
+ fan_table.HystDown = cpu_to_be16(hwmgr->
+ thermal_controller.advanceFanControlParameters.ucTHyst);
+
+ fan_table.HystUp = cpu_to_be16(1);
+
+ fan_table.HystSlope = cpu_to_be16(1);
+
+ fan_table.TempRespLim = cpu_to_be16(5);
+
+ reference_clock = tonga_get_xclk(hwmgr);
+
+ fan_table.RefreshPeriod = cpu_to_be32((hwmgr->
+ thermal_controller.advanceFanControlParameters.ulCycleDelay *
+ reference_clock) / 1600);
+
+ fan_table.FdoMax = cpu_to_be16((uint16_t)duty100);
+
+ fan_table.TempSrc = (uint8_t)PHM_READ_VFPF_INDIRECT_FIELD(
+ hwmgr->device, CGS_IND_REG__SMC,
+ CG_MULT_THERMAL_CTRL, TEMP_SEL);
+
+ res = polaris10_copy_bytes_to_smc(hwmgr->smumgr, data->fan_table_start,
+ (uint8_t *)&fan_table, (uint32_t)sizeof(fan_table),
+ data->sram_end);
+
+ if (!res && hwmgr->thermal_controller.
+ advanceFanControlParameters.ucMinimumPWMLimit)
+ res = smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_SetFanMinPwm,
+ hwmgr->thermal_controller.
+ advanceFanControlParameters.ucMinimumPWMLimit);
+
+ if (!res && hwmgr->thermal_controller.
+ advanceFanControlParameters.ulMinFanSCLKAcousticLimit)
+ res = smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
+ PPSMC_MSG_SetFanSclkTarget,
+ hwmgr->thermal_controller.
+ advanceFanControlParameters.ulMinFanSCLKAcousticLimit);
+
+ if (res)
+ phm_cap_unset(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_MicrocodeFanControl);
+
+ return 0;
+}
+
+/**
+* Start the fan control on the SMC.
+* @param hwmgr the address of the powerplay hardware manager.
+* @param pInput the pointer to input data
+* @param pOutput the pointer to output data
+* @param pStorage the pointer to temporary storage
+* @param Result the last failure code
+* @return result from set temperature range routine
+*/
+int tf_polaris10_thermal_start_smc_fan_control(struct pp_hwmgr *hwmgr,
+ void *input, void *output, void *storage, int result)
+{
+/* If the fantable setup has failed we could have disabled
+ * PHM_PlatformCaps_MicrocodeFanControl even after
+ * this function was included in the table.
+ * Make sure that we still think controlling the fan is OK.
+*/
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_MicrocodeFanControl)) {
+ polaris10_fan_ctrl_start_smc_fan_control(hwmgr);
+ polaris10_fan_ctrl_set_static_mode(hwmgr, FDO_PWM_MODE_STATIC);
+ }
+
+ return 0;
+}
+
+/**
+* Set temperature range for high and low alerts
+* @param hwmgr the address of the powerplay hardware manager.
+* @param pInput the pointer to input data
+* @param pOutput the pointer to output data
+* @param pStorage the pointer to temporary storage
+* @param Result the last failure code
+* @return result from set temperature range routine
+*/
+int tf_polaris10_thermal_set_temperature_range(struct pp_hwmgr *hwmgr,
+ void *input, void *output, void *storage, int result)
+{
+ struct PP_TemperatureRange *range = (struct PP_TemperatureRange *)input;
+
+ if (range == NULL)
+ return -EINVAL;
+
+ return polaris10_thermal_set_temperature_range(hwmgr, range->min, range->max);
+}
+
+/**
+* Programs one-time setting registers
+* @param hwmgr the address of the powerplay hardware manager.
+* @param pInput the pointer to input data
+* @param pOutput the pointer to output data
+* @param pStorage the pointer to temporary storage
+* @param Result the last failure code
+* @return result from initialize thermal controller routine
+*/
+int tf_polaris10_thermal_initialize(struct pp_hwmgr *hwmgr,
+ void *input, void *output, void *storage, int result)
+{
+ return polaris10_thermal_initialize(hwmgr);
+}
+
+/**
+* Enable high and low alerts
+* @param hwmgr the address of the powerplay hardware manager.
+* @param pInput the pointer to input data
+* @param pOutput the pointer to output data
+* @param pStorage the pointer to temporary storage
+* @param Result the last failure code
+* @return result from enable alert routine
+*/
+int tf_polaris10_thermal_enable_alert(struct pp_hwmgr *hwmgr,
+ void *input, void *output, void *storage, int result)
+{
+ return polaris10_thermal_enable_alert(hwmgr);
+}
+
+/**
+* Disable high and low alerts
+* @param hwmgr the address of the powerplay hardware manager.
+* @param pInput the pointer to input data
+* @param pOutput the pointer to output data
+* @param pStorage the pointer to temporary storage
+* @param Result the last failure code
+* @return result from disable alert routine
+*/
+static int tf_polaris10_thermal_disable_alert(struct pp_hwmgr *hwmgr,
+ void *input, void *output, void *storage, int result)
+{
+ return polaris10_thermal_disable_alert(hwmgr);
+}
+
+static int tf_polaris10_thermal_avfs_enable(struct pp_hwmgr *hwmgr,
+ void *input, void *output, void *storage, int result)
+{
+ int ret;
+ struct pp_smumgr *smumgr = (struct pp_smumgr *)(hwmgr->smumgr);
+ struct polaris10_smumgr *smu_data = (struct polaris10_smumgr *)(smumgr->backend);
+
+ if (smu_data->avfs.avfs_btc_status != AVFS_BTC_ENABLEAVFS)
+ return 0;
+
+ ret = (smum_send_msg_to_smc(smumgr, PPSMC_MSG_EnableAvfs) == 0) ?
+ 0 : -1;
+
+ if (!ret)
+ /* If this param is not changed, this function could fire unnecessarily */
+ smu_data->avfs.avfs_btc_status = AVFS_BTC_COMPLETED_PREVIOUSLY;
+
+ return ret;
+}
+
+static const struct phm_master_table_item
+polaris10_thermal_start_thermal_controller_master_list[] = {
+ {NULL, tf_polaris10_thermal_initialize},
+ {NULL, tf_polaris10_thermal_set_temperature_range},
+ {NULL, tf_polaris10_thermal_enable_alert},
+ {NULL, tf_polaris10_thermal_avfs_enable},
+/* We should restrict performance levels to low before we halt the SMC.
+ * On the other hand we are still in boot state when we do this
+ * so it would be pointless.
+ * If this assumption changes we have to revisit this table.
+ */
+ {NULL, tf_polaris10_thermal_setup_fan_table},
+ {NULL, tf_polaris10_thermal_start_smc_fan_control},
+ {NULL, NULL}
+};
+
+static const struct phm_master_table_header
+polaris10_thermal_start_thermal_controller_master = {
+ 0,
+ PHM_MasterTableFlag_None,
+ polaris10_thermal_start_thermal_controller_master_list
+};
+
+static const struct phm_master_table_item
+polaris10_thermal_set_temperature_range_master_list[] = {
+ {NULL, tf_polaris10_thermal_disable_alert},
+ {NULL, tf_polaris10_thermal_set_temperature_range},
+ {NULL, tf_polaris10_thermal_enable_alert},
+ {NULL, NULL}
+};
+
+static const struct phm_master_table_header
+polaris10_thermal_set_temperature_range_master = {
+ 0,
+ PHM_MasterTableFlag_None,
+ polaris10_thermal_set_temperature_range_master_list
+};
+
+int polaris10_thermal_ctrl_uninitialize_thermal_controller(struct pp_hwmgr *hwmgr)
+{
+ if (!hwmgr->thermal_controller.fanInfo.bNoFan)
+ polaris10_fan_ctrl_set_default_mode(hwmgr);
+ return 0;
+}
+
+/**
+* Initializes the thermal controller related functions in the Hardware Manager structure.
+* @param hwmgr The address of the hardware manager.
+* @exception Any error code from the low-level communication.
+*/
+int pp_polaris10_thermal_initialize(struct pp_hwmgr *hwmgr)
+{
+ int result;
+
+ result = phm_construct_table(hwmgr,
+ &polaris10_thermal_set_temperature_range_master,
+ &(hwmgr->set_temperature_range));
+
+ if (!result) {
+ result = phm_construct_table(hwmgr,
+ &polaris10_thermal_start_thermal_controller_master,
+ &(hwmgr->start_thermal_controller));
+ if (result)
+ phm_destroy_table(hwmgr, &(hwmgr->set_temperature_range));
+ }
+
+ if (!result)
+ hwmgr->fan_ctrl_is_in_default_mode = true;
+ return result;
+}
+
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_thermal.h b/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_thermal.h
new file mode 100644
index 0000000..62f8cbc
--- /dev/null
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/polaris10_thermal.h
@@ -0,0 +1,62 @@
+/*
+ * Copyright 2016 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#ifndef _POLARIS10_THERMAL_H_
+#define _POLARIS10_THERMAL_H_
+
+#include "hwmgr.h"
+
+#define POLARIS10_THERMAL_HIGH_ALERT_MASK 0x1
+#define POLARIS10_THERMAL_LOW_ALERT_MASK 0x2
+
+#define POLARIS10_THERMAL_MINIMUM_TEMP_READING -256
+#define POLARIS10_THERMAL_MAXIMUM_TEMP_READING 255
+
+#define POLARIS10_THERMAL_MINIMUM_ALERT_TEMP 0
+#define POLARIS10_THERMAL_MAXIMUM_ALERT_TEMP 255
+
+#define FDO_PWM_MODE_STATIC 1
+#define FDO_PWM_MODE_STATIC_RPM 5
+
+
+extern int tf_polaris10_thermal_initialize(struct pp_hwmgr *hwmgr, void *input, void *output, void *storage, int result);
+extern int tf_polaris10_thermal_set_temperature_range(struct pp_hwmgr *hwmgr, void *input, void *output, void *storage, int result);
+extern int tf_polaris10_thermal_enable_alert(struct pp_hwmgr *hwmgr, void *input, void *output, void *storage, int result);
+
+extern int polaris10_thermal_get_temperature(struct pp_hwmgr *hwmgr);
+extern int polaris10_thermal_stop_thermal_controller(struct pp_hwmgr *hwmgr);
+extern int polaris10_fan_ctrl_get_fan_speed_info(struct pp_hwmgr *hwmgr, struct phm_fan_speed_info *fan_speed_info);
+extern int polaris10_fan_ctrl_get_fan_speed_percent(struct pp_hwmgr *hwmgr, uint32_t *speed);
+extern int polaris10_fan_ctrl_set_default_mode(struct pp_hwmgr *hwmgr);
+extern int polaris10_fan_ctrl_set_static_mode(struct pp_hwmgr *hwmgr, uint32_t mode);
+extern int polaris10_fan_ctrl_set_fan_speed_percent(struct pp_hwmgr *hwmgr, uint32_t speed);
+extern int polaris10_fan_ctrl_reset_fan_speed_to_default(struct pp_hwmgr *hwmgr);
+extern int pp_polaris10_thermal_initialize(struct pp_hwmgr *hwmgr);
+extern int polaris10_thermal_ctrl_uninitialize_thermal_controller(struct pp_hwmgr *hwmgr);
+extern int polaris10_fan_ctrl_set_fan_speed_rpm(struct pp_hwmgr *hwmgr, uint32_t speed);
+extern int polaris10_fan_ctrl_get_fan_speed_rpm(struct pp_hwmgr *hwmgr, uint32_t *speed);
+extern int polaris10_fan_ctrl_stop_smc_fan_control(struct pp_hwmgr *hwmgr);
+extern uint32_t tonga_get_xclk(struct pp_hwmgr *hwmgr);
+
+#endif
+
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/pp_acpi.c b/drivers/gpu/drm/amd/powerplay/hwmgr/pp_acpi.c
index 7b2d500..58742e0 100644
--- a/drivers/gpu/drm/amd/powerplay/hwmgr/pp_acpi.c
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/pp_acpi.c
@@ -1,3 +1,26 @@
+/*
+ * Copyright 2016 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
#include <linux/errno.h>
#include "linux/delay.h"
#include "hwmgr.h"
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/ppatomctrl.c b/drivers/gpu/drm/amd/powerplay/hwmgr/ppatomctrl.c
index 2a83a4a..da9f5f1 100644
--- a/drivers/gpu/drm/amd/powerplay/hwmgr/ppatomctrl.c
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/ppatomctrl.c
@@ -373,6 +373,37 @@
return result;
}
+int atomctrl_get_engine_pll_dividers_ai(struct pp_hwmgr *hwmgr,
+ uint32_t clock_value,
+ pp_atomctrl_clock_dividers_ai *dividers)
+{
+ COMPUTE_GPU_CLOCK_OUTPUT_PARAMETERS_V1_7 pll_patameters;
+ int result;
+
+ pll_patameters.ulClock.ulClock = clock_value;
+ pll_patameters.ulClock.ucPostDiv = COMPUTE_GPUCLK_INPUT_FLAG_SCLK;
+
+ result = cgs_atom_exec_cmd_table
+ (hwmgr->device,
+ GetIndexIntoMasterTable(COMMAND, ComputeMemoryEnginePLL),
+ &pll_patameters);
+
+ if (0 == result) {
+ dividers->usSclk_fcw_frac = le16_to_cpu(pll_patameters.usSclk_fcw_frac);
+ dividers->usSclk_fcw_int = le16_to_cpu(pll_patameters.usSclk_fcw_int);
+ dividers->ucSclkPostDiv = pll_patameters.ucSclkPostDiv;
+ dividers->ucSclkVcoMode = pll_patameters.ucSclkVcoMode;
+ dividers->ucSclkPllRange = pll_patameters.ucSclkPllRange;
+ dividers->ucSscEnable = pll_patameters.ucSscEnable;
+ dividers->usSsc_fcw1_frac = le16_to_cpu(pll_patameters.usSsc_fcw1_frac);
+ dividers->usSsc_fcw1_int = le16_to_cpu(pll_patameters.usSsc_fcw1_int);
+ dividers->usPcc_fcw_int = le16_to_cpu(pll_patameters.usPcc_fcw_int);
+ dividers->usSsc_fcw_slew_frac = le16_to_cpu(pll_patameters.usSsc_fcw_slew_frac);
+ dividers->usPcc_fcw_slew_frac = le16_to_cpu(pll_patameters.usPcc_fcw_slew_frac);
+ }
+ return result;
+}
+
int atomctrl_get_dfs_pll_dividers_vi(
struct pp_hwmgr *hwmgr,
uint32_t clock_value,
@@ -618,7 +649,7 @@
if (!getASICProfilingInfo)
return -1;
- if(getASICProfilingInfo->asHeader.ucTableFormatRevision < 3 ||
+ if (getASICProfilingInfo->asHeader.ucTableFormatRevision < 3 ||
(getASICProfilingInfo->asHeader.ucTableFormatRevision == 3 &&
getASICProfilingInfo->asHeader.ucTableContentRevision < 4))
return -1;
@@ -891,18 +922,18 @@
*-----------------------
*/
- fA_Term = fAdd(fMargin_RO_a, fAdd(fMultiply(fSM_A4,fSclk), fSM_A5));
+ fA_Term = fAdd(fMargin_RO_a, fAdd(fMultiply(fSM_A4, fSclk), fSM_A5));
fB_Term = fAdd(fAdd(fMultiply(fSM_A2, fSclk), fSM_A6), fMargin_RO_b);
fC_Term = fAdd(fMargin_RO_c,
fAdd(fMultiply(fSM_A0,fLkg_FT),
- fAdd(fMultiply(fSM_A1, fMultiply(fLkg_FT,fSclk)),
+ fAdd(fMultiply(fSM_A1, fMultiply(fLkg_FT, fSclk)),
fAdd(fMultiply(fSM_A3, fSclk),
- fSubtract(fSM_A7,fRO_fused)))));
+ fSubtract(fSM_A7, fRO_fused)))));
fVDDC_base = fSubtract(fRO_fused,
fSubtract(fMargin_RO_c,
fSubtract(fSM_A3, fMultiply(fSM_A1, fSclk))));
- fVDDC_base = fDivide(fVDDC_base, fAdd(fMultiply(fSM_A0,fSclk), fSM_A2));
+ fVDDC_base = fDivide(fVDDC_base, fAdd(fMultiply(fSM_A0, fSclk), fSM_A2));
repeat = fSubtract(fVDDC_base,
fDivide(fMargin_DC_sigma, ConvertToFraction(1000)));
@@ -916,7 +947,7 @@
fSubtract(fRO_DC_margin,
fSubtract(fSM_A3,
fMultiply(fSM_A2, repeat))));
- fDC_SCLK = fDivide(fDC_SCLK, fAdd(fMultiply(fSM_A0,repeat), fSM_A1));
+ fDC_SCLK = fDivide(fDC_SCLK, fAdd(fMultiply(fSM_A0, repeat), fSM_A1));
fSigma_DC = fSubtract(fSclk, fDC_SCLK);
@@ -996,7 +1027,7 @@
fV_NL = fRoundUpByStepSize(fV_NL, fStepSize, 0);
if (GreaterThan(fV_max, fV_NL) &&
- (GreaterThan(fV_NL,fEVV_V) ||
+ (GreaterThan(fV_NL, fEVV_V) ||
Equal(fV_NL, fEVV_V))) {
fV_NL = fMultiply(fV_NL, ConvertToFraction(1000));
@@ -1010,10 +1041,10 @@
}
/** atomctrl_get_voltage_evv_on_sclk gets voltage via call to ATOM COMMAND table.
- * @param hwmgr input: pointer to hwManager
+ * @param hwmgr input: pointer to hwManager
* @param voltage_type input: type of EVV voltage VDDC or VDDGFX
* @param sclk input: in 10Khz unit. DPM state SCLK frequency
- * which is define in PPTable SCLK/VDDC dependence
+ * which is define in PPTable SCLK/VDDC dependence
* table associated with this virtual_voltage_Id
* @param virtual_voltage_Id input: voltage id which match per voltage DPM state: 0xff01, 0xff02.. 0xff08
* @param voltage output: real voltage level in unit of mv
@@ -1205,3 +1236,69 @@
return result;
}
+
+int atomctrl_set_ac_timing_ai(struct pp_hwmgr *hwmgr, uint32_t memory_clock,
+ uint8_t level)
+{
+ DYNAMICE_MEMORY_SETTINGS_PARAMETER_V2_1 memory_clock_parameters;
+ int result;
+
+ memory_clock_parameters.asDPMMCReg.ulClock.ulClockFreq = memory_clock & SET_CLOCK_FREQ_MASK;
+ memory_clock_parameters.asDPMMCReg.ulClock.ulComputeClockFlag = ADJUST_MC_SETTING_PARAM;
+ memory_clock_parameters.asDPMMCReg.ucMclkDPMState = level;
+
+ result = cgs_atom_exec_cmd_table
+ (hwmgr->device,
+ GetIndexIntoMasterTable(COMMAND, DynamicMemorySettings),
+ &memory_clock_parameters);
+
+ return result;
+}
+
+int atomctrl_get_voltage_evv_on_sclk_ai(struct pp_hwmgr *hwmgr, uint8_t voltage_type,
+ uint32_t sclk, uint16_t virtual_voltage_Id, uint16_t *voltage)
+{
+
+ int result;
+ GET_VOLTAGE_INFO_INPUT_PARAMETER_V1_3 get_voltage_info_param_space;
+
+ get_voltage_info_param_space.ucVoltageType = voltage_type;
+ get_voltage_info_param_space.ucVoltageMode = ATOM_GET_VOLTAGE_EVV_VOLTAGE;
+ get_voltage_info_param_space.usVoltageLevel = virtual_voltage_Id;
+ get_voltage_info_param_space.ulSCLKFreq = sclk;
+
+ result = cgs_atom_exec_cmd_table(hwmgr->device,
+ GetIndexIntoMasterTable(COMMAND, GetVoltageInfo),
+ &get_voltage_info_param_space);
+
+ if (0 != result)
+ return result;
+
+ *voltage = get_voltage_info_param_space.usVoltageLevel;
+
+ return result;
+}
+
+int atomctrl_get_smc_sclk_range_table(struct pp_hwmgr *hwmgr, struct pp_atom_ctrl_sclk_range_table *table)
+{
+
+ int i;
+ u8 frev, crev;
+ u16 size;
+
+ ATOM_SMU_INFO_V2_1 *psmu_info =
+ (ATOM_SMU_INFO_V2_1 *)cgs_atom_get_data_table(hwmgr->device,
+ GetIndexIntoMasterTable(DATA, SMU_Info),
+ &size, &frev, &crev);
+
+
+ for (i = 0; i < psmu_info->ucSclkEntryNum; i++) {
+ table->entry[i].ucVco_setting = psmu_info->asSclkFcwRangeEntry[i].ucVco_setting;
+ table->entry[i].ucPostdiv = psmu_info->asSclkFcwRangeEntry[i].ucPostdiv;
+ table->entry[i].usFcw_pcc = psmu_info->asSclkFcwRangeEntry[i].ucFcw_pcc;
+ table->entry[i].usFcw_trans_upper = psmu_info->asSclkFcwRangeEntry[i].ucFcw_trans_upper;
+ table->entry[i].usRcw_trans_lower = psmu_info->asSclkFcwRangeEntry[i].ucRcw_trans_lower;
+ }
+
+ return 0;
+}
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/ppatomctrl.h b/drivers/gpu/drm/amd/powerplay/hwmgr/ppatomctrl.h
index 627420b..d24ebb5 100644
--- a/drivers/gpu/drm/amd/powerplay/hwmgr/ppatomctrl.h
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/ppatomctrl.h
@@ -101,6 +101,23 @@
};
typedef struct pp_atomctrl_clock_dividers_vi pp_atomctrl_clock_dividers_vi;
+struct pp_atomctrl_clock_dividers_ai {
+ u16 usSclk_fcw_frac;
+ u16 usSclk_fcw_int;
+ u8 ucSclkPostDiv;
+ u8 ucSclkVcoMode;
+ u8 ucSclkPllRange;
+ u8 ucSscEnable;
+ u16 usSsc_fcw1_frac;
+ u16 usSsc_fcw1_int;
+ u16 usReserved;
+ u16 usPcc_fcw_int;
+ u16 usSsc_fcw_slew_frac;
+ u16 usPcc_fcw_slew_frac;
+};
+typedef struct pp_atomctrl_clock_dividers_ai pp_atomctrl_clock_dividers_ai;
+
+
union pp_atomctrl_s_mpll_fb_divider {
struct {
uint32_t cl_kf : 12;
@@ -204,6 +221,21 @@
typedef struct pp_atomctrl_mc_register_address pp_atomctrl_mc_register_address;
+#define MAX_SCLK_RANGE 8
+
+struct pp_atom_ctrl_sclk_range_table_entry{
+ uint8_t ucVco_setting;
+ uint8_t ucPostdiv;
+ uint16_t usFcw_pcc;
+ uint16_t usFcw_trans_upper;
+ uint16_t usRcw_trans_lower;
+};
+
+
+struct pp_atom_ctrl_sclk_range_table{
+ struct pp_atom_ctrl_sclk_range_table_entry entry[MAX_SCLK_RANGE];
+};
+
struct pp_atomctrl_mc_reg_table {
uint8_t last; /* number of registers */
uint8_t num_entries; /* number of AC timing entries */
@@ -240,7 +272,11 @@
uint16_t end_index, uint32_t mask, uint32_t *efuse);
extern int atomctrl_calculate_voltage_evv_on_sclk(struct pp_hwmgr *hwmgr, uint8_t voltage_type,
uint32_t sclk, uint16_t virtual_voltage_Id, uint16_t *voltage, uint16_t dpm_level, bool debug);
-
-
+extern int atomctrl_get_engine_pll_dividers_ai(struct pp_hwmgr *hwmgr, uint32_t clock_value, pp_atomctrl_clock_dividers_ai *dividers);
+extern int atomctrl_set_ac_timing_ai(struct pp_hwmgr *hwmgr, uint32_t memory_clock,
+ uint8_t level);
+extern int atomctrl_get_voltage_evv_on_sclk_ai(struct pp_hwmgr *hwmgr, uint8_t voltage_type,
+ uint32_t sclk, uint16_t virtual_voltage_Id, uint16_t *voltage);
+extern int atomctrl_get_smc_sclk_range_table(struct pp_hwmgr *hwmgr, struct pp_atom_ctrl_sclk_range_table *table);
#endif
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/ppevvmath.h b/drivers/gpu/drm/amd/powerplay/hwmgr/ppevvmath.h
index b10df32..009bd59 100644
--- a/drivers/gpu/drm/amd/powerplay/hwmgr/ppevvmath.h
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/ppevvmath.h
@@ -127,8 +127,8 @@
fInt solution = fPositiveOne; /*Starting off with baseline of 1 */
fInt error_term;
- uint32_t k_array[11] = {55452, 27726, 13863, 6931, 4055, 2231, 1178, 606, 308, 155, 78};
- uint32_t expk_array[11] = {2560000, 160000, 40000, 20000, 15000, 12500, 11250, 10625, 10313, 10156, 10078};
+ static const uint32_t k_array[11] = {55452, 27726, 13863, 6931, 4055, 2231, 1178, 606, 308, 155, 78};
+ static const uint32_t expk_array[11] = {2560000, 160000, 40000, 20000, 15000, 12500, 11250, 10625, 10313, 10156, 10078};
if (GreaterThan(fZERO, exponent)) {
exponent = fNegate(exponent);
@@ -162,8 +162,8 @@
fInt solution = ConvertToFraction(0); /*Starting off with baseline of 0 */
fInt error_term;
- uint32_t k_array[10] = {160000, 40000, 20000, 15000, 12500, 11250, 10625, 10313, 10156, 10078};
- uint32_t logk_array[10] = {27726, 13863, 6931, 4055, 2231, 1178, 606, 308, 155, 78};
+ static const uint32_t k_array[10] = {160000, 40000, 20000, 15000, 12500, 11250, 10625, 10313, 10156, 10078};
+ static const uint32_t logk_array[10] = {27726, 13863, 6931, 4055, 2231, 1178, 606, 308, 155, 78};
while (GreaterThan(fAdd(value, fNegativeOne), upper_bound)) {
for (i = 0; i < 10; i++) {
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/tonga_hwmgr.c b/drivers/gpu/drm/amd/powerplay/hwmgr/tonga_hwmgr.c
index 0d5d837..1faad92 100644
--- a/drivers/gpu/drm/amd/powerplay/hwmgr/tonga_hwmgr.c
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/tonga_hwmgr.c
@@ -51,6 +51,9 @@
#include "bif/bif_5_0_d.h"
#include "bif/bif_5_0_sh_mask.h"
+#include "dce/dce_10_0_d.h"
+#include "dce/dce_10_0_sh_mask.h"
+
#include "cgs_linux.h"
#include "eventmgr.h"
#include "amd_pcie_helpers.h"
@@ -86,17 +89,17 @@
typedef uint32_t PECI_RegistryValue;
/* [2.5%,~2.5%] Clock stretched is multiple of 2.5% vs not and [Fmin, Fmax, LDO_REFSEL, USE_FOR_LOW_FREQ] */
-uint16_t PP_ClockStretcherLookupTable[2][4] = {
+static const uint16_t PP_ClockStretcherLookupTable[2][4] = {
{600, 1050, 3, 0},
{600, 1050, 6, 1} };
/* [FF, SS] type, [] 4 voltage ranges, and [Floor Freq, Boundary Freq, VID min , VID max] */
-uint32_t PP_ClockStretcherDDTTable[2][4][4] = {
+static const uint32_t PP_ClockStretcherDDTTable[2][4][4] = {
{ {265, 529, 120, 128}, {325, 650, 96, 119}, {430, 860, 32, 95}, {0, 0, 0, 31} },
{ {275, 550, 104, 112}, {319, 638, 96, 103}, {360, 720, 64, 95}, {384, 768, 32, 63} } };
/* [Use_For_Low_freq] value, [0%, 5%, 10%, 7.14%, 14.28%, 20%] (coming from PWR_CKS_CNTL.stretch_amount reg spec) */
-uint8_t PP_ClockStretchAmountConversion[2][6] = {
+static const uint8_t PP_ClockStretchAmountConversion[2][6] = {
{0, 1, 3, 2, 4, 5},
{0, 2, 4, 5, 6, 5} };
@@ -110,7 +113,7 @@
};
typedef enum DPM_EVENT_SRC DPM_EVENT_SRC;
-const unsigned long PhwTonga_Magic = (unsigned long)(PHM_VIslands_Magic);
+static const unsigned long PhwTonga_Magic = (unsigned long)(PHM_VIslands_Magic);
struct tonga_power_state *cast_phw_tonga_power_state(
struct pp_hw_power_state *hw_ps)
@@ -429,19 +432,20 @@
}
}
}
- PP_ASSERT_WITH_CODE(0 == atomctrl_get_voltage_evv_on_sclk
- (hwmgr, VOLTAGE_TYPE_VDDGFX, sclk,
- virtual_voltage_id, &vddgfx),
- "Error retrieving EVV voltage value!", continue);
+ if (0 == atomctrl_get_voltage_evv_on_sclk
+ (hwmgr, VOLTAGE_TYPE_VDDGFX, sclk,
+ virtual_voltage_id, &vddgfx)) {
+ /* need to make sure vddgfx is less than 2v or else, it could burn the ASIC. */
+ PP_ASSERT_WITH_CODE((vddgfx < 2000 && vddgfx != 0), "Invalid VDDGFX value!", return -1);
- /* need to make sure vddgfx is less than 2v or else, it could burn the ASIC. */
- PP_ASSERT_WITH_CODE((vddgfx < 2000 && vddgfx != 0), "Invalid VDDGFX value!", return -1);
-
- /* the voltage should not be zero nor equal to leakage ID */
- if (vddgfx != 0 && vddgfx != virtual_voltage_id) {
- data->vddcgfx_leakage.actual_voltage[data->vddcgfx_leakage.count] = vddgfx;
- data->vddcgfx_leakage.leakage_id[data->vddcgfx_leakage.count] = virtual_voltage_id;
- data->vddcgfx_leakage.count++;
+ /* the voltage should not be zero nor equal to leakage ID */
+ if (vddgfx != 0 && vddgfx != virtual_voltage_id) {
+ data->vddcgfx_leakage.actual_voltage[data->vddcgfx_leakage.count] = vddgfx;
+ data->vddcgfx_leakage.leakage_id[data->vddcgfx_leakage.count] = virtual_voltage_id;
+ data->vddcgfx_leakage.count++;
+ }
+ } else {
+ printk("Error retrieving EVV voltage value!\n");
}
}
} else {
@@ -449,20 +453,20 @@
if (0 == tonga_get_sclk_for_voltage_evv(hwmgr,
pptable_info->vddc_lookup_table,
virtual_voltage_id, &sclk)) {
- PP_ASSERT_WITH_CODE(0 == atomctrl_get_voltage_evv_on_sclk
- (hwmgr, VOLTAGE_TYPE_VDDC, sclk,
- virtual_voltage_id, &vddc),
- "Error retrieving EVV voltage value!", continue);
+ if (0 == atomctrl_get_voltage_evv_on_sclk
+ (hwmgr, VOLTAGE_TYPE_VDDC, sclk,
+ virtual_voltage_id, &vddc)) {
+ /* need to make sure vddc is less than 2v or else, it could burn the ASIC. */
+ PP_ASSERT_WITH_CODE(vddc < 2000, "Invalid VDDC value!", return -1);
- /* need to make sure vddc is less than 2v or else, it could burn the ASIC. */
- if (vddc > 2000)
- printk(KERN_ERR "[ powerplay ] Invalid VDDC value! \n");
-
- /* the voltage should not be zero nor equal to leakage ID */
- if (vddc != 0 && vddc != virtual_voltage_id) {
- data->vddc_leakage.actual_voltage[data->vddc_leakage.count] = vddc;
- data->vddc_leakage.leakage_id[data->vddc_leakage.count] = virtual_voltage_id;
- data->vddc_leakage.count++;
+ /* the voltage should not be zero nor equal to leakage ID */
+ if (vddc != 0 && vddc != virtual_voltage_id) {
+ data->vddc_leakage.actual_voltage[data->vddc_leakage.count] = vddc;
+ data->vddc_leakage.leakage_id[data->vddc_leakage.count] = virtual_voltage_id;
+ data->vddc_leakage.count++;
+ }
+ } else {
+ printk("Error retrieving EVV voltage value!\n");
}
}
}
@@ -2037,14 +2041,11 @@
data->display_timing.num_existing_displays = info.display_count;
if ((data->mclk_stutter_mode_threshold != 0) &&
- (memory_clock <= data->mclk_stutter_mode_threshold) &&
- (data->is_uvd_enabled == 0)
-#if defined(LINUX)
- && (PHM_READ_FIELD(hwmgr->device, DPG_PIPE_STUTTER_CONTROL, STUTTER_ENABLE) & 0x1)
- && (data->display_timing.num_existing_displays <= 2)
- && (data->display_timing.num_existing_displays != 0)
-#endif
- )
+ (memory_clock <= data->mclk_stutter_mode_threshold) &&
+ (data->is_uvd_enabled == 0)
+ && (PHM_READ_FIELD(hwmgr->device, DPG_PIPE_STUTTER_CONTROL, STUTTER_ENABLE) & 0x1)
+ && (data->display_timing.num_existing_displays <= 2)
+ && (data->display_timing.num_existing_displays != 0))
memory_level->StutterEnable = 1;
/* decide strobe mode*/
@@ -2415,6 +2416,24 @@
return 0;
}
+static uint8_t tonga_get_sleep_divider_id_from_clock(uint32_t engine_clock,
+ uint32_t min_engine_clock_in_sr)
+{
+ uint32_t i, temp;
+ uint32_t min = max(min_engine_clock_in_sr, (uint32_t)TONGA_MINIMUM_ENGINE_CLOCK);
+
+ PP_ASSERT_WITH_CODE((engine_clock >= min),
+ "Engine clock can't satisfy stutter requirement!", return 0);
+
+ for (i = TONGA_MAX_DEEPSLEEP_DIVIDER_ID;; i--) {
+ temp = engine_clock >> i;
+
+ if(temp >= min || i == 0)
+ break;
+ }
+ return (uint8_t)i;
+}
+
/**
* Populates single SMC SCLK structure using the provided engine clock
*
@@ -2463,12 +2482,12 @@
*get the DAL clock. do it in funture.
PECI_GetMinClockSettings(hwmgr->peci, &minClocks);
data->display_timing.min_clock_insr = minClocks.engineClockInSR;
-
- if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps, PHM_PlatformCaps_SclkDeepSleep))
- {
- graphic_level->DeepSleepDivId = PhwTonga_GetSleepDividerIdFromClock(hwmgr, engine_clock, minClocks.engineClockInSR);
- }
*/
+ if (phm_cap_enabled(hwmgr->platform_descriptor.platformCaps,
+ PHM_PlatformCaps_SclkDeepSleep))
+ graphic_level->DeepSleepDivId =
+ tonga_get_sleep_divider_id_from_clock(engine_clock,
+ data->display_timing.min_clock_insr);
/* Default to slow, highest DPM level will be set to PPSMC_DISPLAY_WATERMARK_LOW later.*/
graphic_level->DisplayWatermark = PPSMC_DISPLAY_WATERMARK_LOW;
@@ -2663,7 +2682,7 @@
struct TONGA_DLL_SPEED_SETTING {
uint16_t Min; /* Minimum Data Rate*/
uint16_t Max; /* Maximum Data Rate*/
- uint32_t dll_speed; /* The desired DLL_SPEED setting*/
+ uint32_t dll_speed; /* The desired DLL_SPEED setting*/
};
static int tonga_populate_clock_stretcher_data_table(struct pp_hwmgr *hwmgr)
@@ -3296,14 +3315,14 @@
pptable_info->vdd_dep_on_mclk;
PP_ASSERT_WITH_CODE(allowed_sclk_vdd_table != NULL,
- "VDD dependency on SCLK table is missing. \
+ "VDD dependency on SCLK table is missing. \
This table is mandatory", return -1);
PP_ASSERT_WITH_CODE(allowed_sclk_vdd_table->count >= 1,
- "VDD dependency on SCLK table has to have is missing. \
+ "VDD dependency on SCLK table has to have is missing. \
This table is mandatory", return -1);
PP_ASSERT_WITH_CODE(allowed_mclk_vdd_table != NULL,
- "VDD dependency on MCLK table is missing. \
+ "VDD dependency on MCLK table is missing. \
This table is mandatory", return -1);
PP_ASSERT_WITH_CODE(allowed_mclk_vdd_table->count >= 1,
"VDD dependency on MCLK table has to have is missing. \
@@ -4424,17 +4443,14 @@
int tonga_hwmgr_backend_fini(struct pp_hwmgr *hwmgr)
{
- if (NULL != hwmgr->dyn_state.vddc_dep_on_dal_pwrl) {
- kfree(hwmgr->dyn_state.vddc_dep_on_dal_pwrl);
- hwmgr->dyn_state.vddc_dep_on_dal_pwrl = NULL;
+ struct tonga_hwmgr *data = (struct tonga_hwmgr *)(hwmgr->backend);
+
+ if (data->soft_pp_table) {
+ kfree(data->soft_pp_table);
+ data->soft_pp_table = NULL;
}
- if (NULL != hwmgr->backend) {
- kfree(hwmgr->backend);
- hwmgr->backend = NULL;
- }
-
- return 0;
+ return phm_hwmgr_backend_fini(hwmgr);
}
/**
@@ -5874,7 +5890,7 @@
if (!fw_info)
return 0;
- reference_clock = le16_to_cpu(fw_info->usMinPixelClockPLL_Output);
+ reference_clock = le16_to_cpu(fw_info->usReferenceClock);
divide = PHM_READ_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC, CG_CLKPIN_CNTL, XTALIN_DIVIDE);
@@ -6039,24 +6055,40 @@
{
struct tonga_hwmgr *data = (struct tonga_hwmgr *)(hwmgr->backend);
- *table = (char *)&data->smc_state_table;
+ if (!data->soft_pp_table) {
+ data->soft_pp_table = kzalloc(hwmgr->soft_pp_table_size, GFP_KERNEL);
+ if (!data->soft_pp_table)
+ return -ENOMEM;
+ memcpy(data->soft_pp_table, hwmgr->soft_pp_table,
+ hwmgr->soft_pp_table_size);
+ }
- return sizeof(struct SMU72_Discrete_DpmTable);
+ *table = (char *)&data->soft_pp_table;
+
+ return hwmgr->soft_pp_table_size;
}
static int tonga_set_pp_table(struct pp_hwmgr *hwmgr, const char *buf, size_t size)
{
struct tonga_hwmgr *data = (struct tonga_hwmgr *)(hwmgr->backend);
- void *table = (void *)&data->smc_state_table;
+ if (!data->soft_pp_table) {
+ data->soft_pp_table = kzalloc(hwmgr->soft_pp_table_size, GFP_KERNEL);
+ if (!data->soft_pp_table)
+ return -ENOMEM;
+ }
- memcpy(table, buf, size);
+ memcpy(data->soft_pp_table, buf, size);
+
+ hwmgr->soft_pp_table = data->soft_pp_table;
+
+ /* TODO: re-init powerplay to implement modified pptable */
return 0;
}
static int tonga_force_clock_level(struct pp_hwmgr *hwmgr,
- enum pp_clock_type type, int level)
+ enum pp_clock_type type, uint32_t mask)
{
struct tonga_hwmgr *data = (struct tonga_hwmgr *)(hwmgr->backend);
@@ -6068,20 +6100,28 @@
if (!data->sclk_dpm_key_disabled)
smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
PPSMC_MSG_SCLKDPM_SetEnabledMask,
- (1 << level));
+ data->dpm_level_enable_mask.sclk_dpm_enable_mask & mask);
break;
case PP_MCLK:
if (!data->mclk_dpm_key_disabled)
smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
PPSMC_MSG_MCLKDPM_SetEnabledMask,
- (1 << level));
+ data->dpm_level_enable_mask.mclk_dpm_enable_mask & mask);
break;
case PP_PCIE:
+ {
+ uint32_t tmp = mask & data->dpm_level_enable_mask.pcie_dpm_enable_mask;
+ uint32_t level = 0;
+
+ while (tmp >>= 1)
+ level++;
+
if (!data->pcie_dpm_key_disabled)
smum_send_msg_to_smc_with_parameter(hwmgr->smumgr,
PPSMC_MSG_PCIeDPM_ForceLevel,
- (1 << level));
+ level);
break;
+ }
default:
break;
}
@@ -6173,6 +6213,7 @@
.powergate_uvd = tonga_phm_powergate_uvd,
.powergate_vce = tonga_phm_powergate_vce,
.disable_clock_power_gating = tonga_phm_disable_clock_power_gating,
+ .update_clock_gatings = tonga_phm_update_clock_gatings,
.notify_smc_display_config_after_ps_adjustment = tonga_notify_smc_display_config_after_ps_adjustment,
.display_config_changed = tonga_display_configuration_changed_task,
.set_max_fan_pwm_output = tonga_set_max_fan_pwm_output,
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/tonga_hwmgr.h b/drivers/gpu/drm/amd/powerplay/hwmgr/tonga_hwmgr.h
index f88d3bb..573cd39 100644
--- a/drivers/gpu/drm/amd/powerplay/hwmgr/tonga_hwmgr.h
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/tonga_hwmgr.h
@@ -74,7 +74,7 @@
};
struct _phw_tonga_dpm_level {
- bool enabled;
+ bool enabled;
uint32_t value;
uint32_t param1;
};
@@ -237,20 +237,20 @@
irq_handler_func_t ctf_callback;
void *ctf_context;
- phw_tonga_clock_registers clock_registers;
+ phw_tonga_clock_registers clock_registers;
phw_tonga_voltage_smio_registers voltage_smio_registers;
- bool is_memory_GDDR5;
+ bool is_memory_GDDR5;
uint16_t acpi_vddc;
- bool pspp_notify_required; /* Flag to indicate if PSPP notification to SBIOS is required */
+ bool pspp_notify_required; /* Flag to indicate if PSPP notification to SBIOS is required */
uint16_t force_pcie_gen; /* The forced PCI-E speed if not 0xffff */
uint16_t acpi_pcie_gen; /* The PCI-E speed at ACPI time */
uint32_t pcie_gen_cap; /* The PCI-E speed capabilities bitmap from CAIL */
uint32_t pcie_lane_cap; /* The PCI-E lane capabilities bitmap from CAIL */
uint32_t pcie_spc_cap; /* Symbol Per Clock Capabilities from registry */
- phw_tonga_leakage_voltage vddc_leakage; /* The Leakage VDDC supported (based on leakage ID).*/
- phw_tonga_leakage_voltage vddcgfx_leakage; /* The Leakage VDDC supported (based on leakage ID). */
- phw_tonga_leakage_voltage vddci_leakage; /* The Leakage VDDCI supported (based on leakage ID). */
+ phw_tonga_leakage_voltage vddc_leakage; /* The Leakage VDDC supported (based on leakage ID).*/
+ phw_tonga_leakage_voltage vddcgfx_leakage; /* The Leakage VDDC supported (based on leakage ID). */
+ phw_tonga_leakage_voltage vddci_leakage; /* The Leakage VDDCI supported (based on leakage ID). */
uint32_t mvdd_control;
uint32_t vddc_mask_low;
@@ -263,8 +263,8 @@
uint32_t mclk_stutter_mode_threshold;
uint32_t mclk_edc_enable_threshold;
uint32_t mclk_edc_wr_enable_threshold;
- bool is_uvd_enabled;
- bool is_xdma_enabled;
+ bool is_uvd_enabled;
+ bool is_xdma_enabled;
phw_tonga_vbios_boot_state vbios_boot_state;
bool battery_state;
@@ -353,6 +353,8 @@
bool acp_power_gated; /* 1: gated, 0:not gated */
bool pg_acp_init;
+ /* soft pptable for re-uploading into smu */
+ void *soft_pp_table;
};
typedef struct tonga_hwmgr tonga_hwmgr;
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/tonga_pptable.h b/drivers/gpu/drm/amd/powerplay/hwmgr/tonga_pptable.h
index 9a4456e..1b44f4e 100644
--- a/drivers/gpu/drm/amd/powerplay/hwmgr/tonga_pptable.h
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/tonga_pptable.h
@@ -209,6 +209,20 @@
ATOM_Tonga_PCIE_Record entries[1]; /* Dynamically allocate entries. */
} ATOM_Tonga_PCIE_Table;
+typedef struct _ATOM_Polaris10_PCIE_Record {
+ UCHAR ucPCIEGenSpeed;
+ UCHAR usPCIELaneWidth;
+ UCHAR ucReserved[2];
+ ULONG ulPCIE_Sclk;
+} ATOM_Polaris10_PCIE_Record;
+
+typedef struct _ATOM_Polaris10_PCIE_Table {
+ UCHAR ucRevId;
+ UCHAR ucNumEntries; /* Number of entries. */
+ ATOM_Polaris10_PCIE_Record entries[1]; /* Dynamically allocate entries. */
+} ATOM_Polaris10_PCIE_Table;
+
+
typedef struct _ATOM_Tonga_MM_Dependency_Record {
UCHAR ucVddcInd; /* VDDC voltage */
USHORT usVddgfxOffset; /* Offset relative to VDDC voltage */
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/tonga_processpptables.c b/drivers/gpu/drm/amd/powerplay/hwmgr/tonga_processpptables.c
index b156481..10e3630 100644
--- a/drivers/gpu/drm/amd/powerplay/hwmgr/tonga_processpptables.c
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/tonga_processpptables.c
@@ -138,12 +138,15 @@
u16 size;
u8 frev, crev;
- void *table_address;
+ void *table_address = (void *)hwmgr->soft_pp_table;
- table_address = (ATOM_Tonga_POWERPLAYTABLE *)
- cgs_atom_get_data_table(hwmgr->device, index, &size, &frev, &crev);
-
- hwmgr->soft_pp_table = table_address; /*Cache the result in RAM.*/
+ if (!table_address) {
+ table_address = (ATOM_Tonga_POWERPLAYTABLE *)
+ cgs_atom_get_data_table(hwmgr->device,
+ index, &size, &frev, &crev);
+ hwmgr->soft_pp_table = table_address; /*Cache the result in RAM.*/
+ hwmgr->soft_pp_table_size = size;
+ }
return table_address;
}
@@ -448,48 +451,91 @@
static int get_pcie_table(
struct pp_hwmgr *hwmgr,
phm_ppt_v1_pcie_table **pp_tonga_pcie_table,
- const ATOM_Tonga_PCIE_Table * atom_pcie_table
+ const PPTable_Generic_SubTable_Header * pTable
)
{
uint32_t table_size, i, pcie_count;
phm_ppt_v1_pcie_table *pcie_table;
struct phm_ppt_v1_information *pp_table_information =
(struct phm_ppt_v1_information *)(hwmgr->pptable);
- PP_ASSERT_WITH_CODE((0 != atom_pcie_table->ucNumEntries),
- "Invalid PowerPlay Table!", return -1);
- table_size = sizeof(uint32_t) +
- sizeof(phm_ppt_v1_pcie_record) * atom_pcie_table->ucNumEntries;
+ if (pTable->ucRevId < 1) {
+ const ATOM_Tonga_PCIE_Table *atom_pcie_table = (ATOM_Tonga_PCIE_Table *)pTable;
+ PP_ASSERT_WITH_CODE((atom_pcie_table->ucNumEntries != 0),
+ "Invalid PowerPlay Table!", return -1);
- pcie_table = (phm_ppt_v1_pcie_table *)kzalloc(table_size, GFP_KERNEL);
+ table_size = sizeof(uint32_t) +
+ sizeof(phm_ppt_v1_pcie_record) * atom_pcie_table->ucNumEntries;
- if (NULL == pcie_table)
- return -ENOMEM;
+ pcie_table = (phm_ppt_v1_pcie_table *)kzalloc(table_size, GFP_KERNEL);
- memset(pcie_table, 0x00, table_size);
+ if (pcie_table == NULL)
+ return -ENOMEM;
- /*
- * Make sure the number of pcie entries are less than or equal to sclk dpm levels.
- * Since first PCIE entry is for ULV, #pcie has to be <= SclkLevel + 1.
- */
- pcie_count = (pp_table_information->vdd_dep_on_sclk->count) + 1;
- if ((uint32_t)atom_pcie_table->ucNumEntries <= pcie_count)
- pcie_count = (uint32_t)atom_pcie_table->ucNumEntries;
- else
- printk(KERN_ERR "[ powerplay ] Number of Pcie Entries exceed the number of SCLK Dpm Levels! \
- Disregarding the excess entries... \n");
+ memset(pcie_table, 0x00, table_size);
- pcie_table->count = pcie_count;
+ /*
+ * Make sure the number of pcie entries are less than or equal to sclk dpm levels.
+ * Since first PCIE entry is for ULV, #pcie has to be <= SclkLevel + 1.
+ */
+ pcie_count = (pp_table_information->vdd_dep_on_sclk->count) + 1;
+ if ((uint32_t)atom_pcie_table->ucNumEntries <= pcie_count)
+ pcie_count = (uint32_t)atom_pcie_table->ucNumEntries;
+ else
+ printk(KERN_ERR "[ powerplay ] Number of Pcie Entries exceed the number of SCLK Dpm Levels! \
+ Disregarding the excess entries... \n");
- for (i = 0; i < pcie_count; i++) {
- pcie_table->entries[i].gen_speed =
- atom_pcie_table->entries[i].ucPCIEGenSpeed;
- pcie_table->entries[i].lane_width =
- atom_pcie_table->entries[i].usPCIELaneWidth;
+ pcie_table->count = pcie_count;
+
+ for (i = 0; i < pcie_count; i++) {
+ pcie_table->entries[i].gen_speed =
+ atom_pcie_table->entries[i].ucPCIEGenSpeed;
+ pcie_table->entries[i].lane_width =
+ atom_pcie_table->entries[i].usPCIELaneWidth;
+ }
+
+ *pp_tonga_pcie_table = pcie_table;
+ } else {
+ /* Polaris10/Polaris11 and newer. */
+ const ATOM_Polaris10_PCIE_Table *atom_pcie_table = (ATOM_Polaris10_PCIE_Table *)pTable;
+ PP_ASSERT_WITH_CODE((atom_pcie_table->ucNumEntries != 0),
+ "Invalid PowerPlay Table!", return -1);
+
+ table_size = sizeof(uint32_t) +
+ sizeof(phm_ppt_v1_pcie_record) * atom_pcie_table->ucNumEntries;
+
+ pcie_table = (phm_ppt_v1_pcie_table *)kzalloc(table_size, GFP_KERNEL);
+
+ if (pcie_table == NULL)
+ return -ENOMEM;
+
+ memset(pcie_table, 0x00, table_size);
+
+ /*
+ * Make sure the number of pcie entries are less than or equal to sclk dpm levels.
+ * Since first PCIE entry is for ULV, #pcie has to be <= SclkLevel + 1.
+ */
+ pcie_count = (pp_table_information->vdd_dep_on_sclk->count) + 1;
+ if ((uint32_t)atom_pcie_table->ucNumEntries <= pcie_count)
+ pcie_count = (uint32_t)atom_pcie_table->ucNumEntries;
+ else
+ printk(KERN_ERR "[ powerplay ] Number of Pcie Entries exceed the number of SCLK Dpm Levels! \
+ Disregarding the excess entries... \n");
+
+ pcie_table->count = pcie_count;
+
+ for (i = 0; i < pcie_count; i++) {
+ pcie_table->entries[i].gen_speed =
+ atom_pcie_table->entries[i].ucPCIEGenSpeed;
+ pcie_table->entries[i].lane_width =
+ atom_pcie_table->entries[i].usPCIELaneWidth;
+ pcie_table->entries[i].pcie_sclk =
+ atom_pcie_table->entries[i].ulPCIE_Sclk;
+ }
+
+ *pp_tonga_pcie_table = pcie_table;
}
- *pp_tonga_pcie_table = pcie_table;
-
return 0;
}
@@ -668,8 +714,8 @@
const ATOM_Tonga_Hard_Limit_Table *pHardLimits =
(const ATOM_Tonga_Hard_Limit_Table *)(((unsigned long) powerplay_table) +
le16_to_cpu(powerplay_table->usHardLimitTableOffset));
- const ATOM_Tonga_PCIE_Table *pcie_table =
- (const ATOM_Tonga_PCIE_Table *)(((unsigned long) powerplay_table) +
+ const PPTable_Generic_SubTable_Header *pcie_table =
+ (const PPTable_Generic_SubTable_Header *)(((unsigned long) powerplay_table) +
le16_to_cpu(powerplay_table->usPCIETableOffset));
pp_table_information->vdd_dep_on_sclk = NULL;
diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/tonga_thermal.c b/drivers/gpu/drm/amd/powerplay/hwmgr/tonga_thermal.c
index a188174..47ef1ca 100644
--- a/drivers/gpu/drm/amd/powerplay/hwmgr/tonga_thermal.c
+++ b/drivers/gpu/drm/amd/powerplay/hwmgr/tonga_thermal.c
@@ -195,8 +195,8 @@
if (0 == duty100)
return -EINVAL;
- tmp64 = (uint64_t)speed * 100;
- do_div(tmp64, duty100);
+ tmp64 = (uint64_t)speed * duty100;
+ do_div(tmp64, 100);
duty = (uint32_t)tmp64;
PHM_WRITE_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC, CG_FDO_CTRL0, FDO_STATIC_DUTY, duty);
@@ -525,7 +525,7 @@
return tonga_thermal_disable_alert(hwmgr);
}
-static struct phm_master_table_item tonga_thermal_start_thermal_controller_master_list[] = {
+static const struct phm_master_table_item tonga_thermal_start_thermal_controller_master_list[] = {
{ NULL, tf_tonga_thermal_initialize },
{ NULL, tf_tonga_thermal_set_temperature_range },
{ NULL, tf_tonga_thermal_enable_alert },
@@ -538,20 +538,20 @@
{ NULL, NULL }
};
-static struct phm_master_table_header tonga_thermal_start_thermal_controller_master = {
+static const struct phm_master_table_header tonga_thermal_start_thermal_controller_master = {
0,
PHM_MasterTableFlag_None,
tonga_thermal_start_thermal_controller_master_list
};
-static struct phm_master_table_item tonga_thermal_set_temperature_range_master_list[] = {
+static const struct phm_master_table_item tonga_thermal_set_temperature_range_master_list[] = {
{ NULL, tf_tonga_thermal_disable_alert},
{ NULL, tf_tonga_thermal_set_temperature_range},
{ NULL, tf_tonga_thermal_enable_alert},
{ NULL, NULL }
};
-struct phm_master_table_header tonga_thermal_set_temperature_range_master = {
+static const struct phm_master_table_header tonga_thermal_set_temperature_range_master = {
0,
PHM_MasterTableFlag_None,
tonga_thermal_set_temperature_range_master_list
diff --git a/drivers/gpu/drm/amd/powerplay/inc/amd_powerplay.h b/drivers/gpu/drm/amd/powerplay/inc/amd_powerplay.h
index 7255f7d..50b367d 100644
--- a/drivers/gpu/drm/amd/powerplay/inc/amd_powerplay.h
+++ b/drivers/gpu/drm/amd/powerplay/inc/amd_powerplay.h
@@ -289,6 +289,9 @@
#define PP_BLOCK_GFX_CG 0x01
#define PP_BLOCK_GFX_MG 0x02
+#define PP_BLOCK_GFX_3D 0x04
+#define PP_BLOCK_GFX_RLC 0x08
+#define PP_BLOCK_GFX_CP 0x10
#define PP_BLOCK_SYS_BIF 0x01
#define PP_BLOCK_SYS_MC 0x02
#define PP_BLOCK_SYS_ROM 0x04
@@ -337,7 +340,7 @@
int (*get_pp_num_states)(void *handle, struct pp_states_info *data);
int (*get_pp_table)(void *handle, char **table);
int (*set_pp_table)(void *handle, const char *buf, size_t size);
- int (*force_clock_level)(void *handle, enum pp_clock_type type, int level);
+ int (*force_clock_level)(void *handle, enum pp_clock_type type, uint32_t mask);
int (*print_clock_levels)(void *handle, enum pp_clock_type type, char *buf);
};
diff --git a/drivers/gpu/drm/amd/powerplay/inc/eventmgr.h b/drivers/gpu/drm/amd/powerplay/inc/eventmgr.h
index 10437dc..d63ef83 100644
--- a/drivers/gpu/drm/amd/powerplay/inc/eventmgr.h
+++ b/drivers/gpu/drm/amd/powerplay/inc/eventmgr.h
@@ -37,7 +37,7 @@
struct action_chain {
const char *description; /* action chain description for debugging purpose */
- const pem_event_action **action_chain; /* pointer to chain of event actions */
+ const pem_event_action * const *action_chain; /* pointer to chain of event actions */
};
struct pem_power_source_ui_state_info {
diff --git a/drivers/gpu/drm/amd/powerplay/inc/fiji_pwrvirus.h b/drivers/gpu/drm/amd/powerplay/inc/fiji_pwrvirus.h
index 0262ad3..8a31665 100644
--- a/drivers/gpu/drm/amd/powerplay/inc/fiji_pwrvirus.h
+++ b/drivers/gpu/drm/amd/powerplay/inc/fiji_pwrvirus.h
@@ -46,7 +46,7 @@
typedef struct PWR_Command_Table PWR_Command_Table;
#define PWR_VIRUS_TABLE_SIZE 10243
-static PWR_Command_Table PwrVirusTable[PWR_VIRUS_TABLE_SIZE] =
+static const PWR_Command_Table PwrVirusTable[PWR_VIRUS_TABLE_SIZE] =
{
{ PwrCmdWrite, 0x100100b6, mmPCIE_INDEX },
{ PwrCmdWrite, 0x00000000, mmPCIE_DATA },
diff --git a/drivers/gpu/drm/amd/powerplay/inc/hardwaremanager.h b/drivers/gpu/drm/amd/powerplay/inc/hardwaremanager.h
index 040d3f7..56f712c 100644
--- a/drivers/gpu/drm/amd/powerplay/inc/hardwaremanager.h
+++ b/drivers/gpu/drm/amd/powerplay/inc/hardwaremanager.h
@@ -211,6 +211,7 @@
PHM_PlatformCaps_ClockStretcher,
PHM_PlatformCaps_TablelessHardwareInterface,
PHM_PlatformCaps_EnableDriverEVV,
+ PHM_PlatformCaps_SPLLShutdownSupport,
PHM_PlatformCaps_Max
};
diff --git a/drivers/gpu/drm/amd/powerplay/inc/hwmgr.h b/drivers/gpu/drm/amd/powerplay/inc/hwmgr.h
index 928f5a7..fd4ce7a 100644
--- a/drivers/gpu/drm/amd/powerplay/inc/hwmgr.h
+++ b/drivers/gpu/drm/amd/powerplay/inc/hwmgr.h
@@ -141,7 +141,7 @@
struct phm_master_table_header {
uint32_t storage_size;
uint32_t flags;
- struct phm_master_table_item *master_list;
+ const struct phm_master_table_item *master_list;
};
struct phm_runtime_table_header {
@@ -199,7 +199,7 @@
void *input, void *output);
extern int phm_construct_table(struct pp_hwmgr *hwmgr,
- struct phm_master_table_header *master_table,
+ const struct phm_master_table_header *master_table,
struct phm_runtime_table_header *rt_table);
extern int phm_destroy_table(struct pp_hwmgr *hwmgr,
@@ -335,8 +335,9 @@
int (*power_off_asic)(struct pp_hwmgr *hwmgr);
int (*get_pp_table)(struct pp_hwmgr *hwmgr, char **table);
int (*set_pp_table)(struct pp_hwmgr *hwmgr, const char *buf, size_t size);
- int (*force_clock_level)(struct pp_hwmgr *hwmgr, enum pp_clock_type type, int level);
+ int (*force_clock_level)(struct pp_hwmgr *hwmgr, enum pp_clock_type type, uint32_t mask);
int (*print_clock_levels)(struct pp_hwmgr *hwmgr, enum pp_clock_type type, char *buf);
+ int (*enable_per_cu_power_gating)(struct pp_hwmgr *hwmgr, bool enable);
};
struct pp_table_func {
@@ -499,7 +500,7 @@
struct phm_ppm_table *ppm_parameter_table;
struct phm_cac_tdp_table *cac_dtp_table;
struct phm_clock_voltage_dependency_table *vdd_gfx_dependency_on_sclk;
- struct phm_vq_budgeting_table *vq_budgeting_table;
+ struct phm_vq_budgeting_table *vq_budgeting_table;
};
struct pp_fan_info {
@@ -576,6 +577,7 @@
void *device;
struct pp_smumgr *smumgr;
const void *soft_pp_table;
+ uint32_t soft_pp_table_size;
bool need_pp_table_upload;
enum amd_dpm_forced_level dpm_level;
bool block_hw_access;
diff --git a/drivers/gpu/drm/amd/powerplay/inc/polaris10_ppsmc.h b/drivers/gpu/drm/amd/powerplay/inc/polaris10_ppsmc.h
new file mode 100644
index 0000000..0c6a413
--- /dev/null
+++ b/drivers/gpu/drm/amd/powerplay/inc/polaris10_ppsmc.h
@@ -0,0 +1,409 @@
+/*
+ * Copyright 2015 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#ifndef POLARIS10_PP_SMC_H
+#define POLARIS10_PP_SMC_H
+
+
+#pragma pack(push, 1)
+
+
+#define PPSMC_SWSTATE_FLAG_DC 0x01
+#define PPSMC_SWSTATE_FLAG_UVD 0x02
+#define PPSMC_SWSTATE_FLAG_VCE 0x04
+
+#define PPSMC_THERMAL_PROTECT_TYPE_INTERNAL 0x00
+#define PPSMC_THERMAL_PROTECT_TYPE_EXTERNAL 0x01
+#define PPSMC_THERMAL_PROTECT_TYPE_NONE 0xff
+
+#define PPSMC_SYSTEMFLAG_GPIO_DC 0x01
+#define PPSMC_SYSTEMFLAG_STEPVDDC 0x02
+#define PPSMC_SYSTEMFLAG_GDDR5 0x04
+
+#define PPSMC_SYSTEMFLAG_DISABLE_BABYSTEP 0x08
+
+#define PPSMC_SYSTEMFLAG_REGULATOR_HOT 0x10
+#define PPSMC_SYSTEMFLAG_REGULATOR_HOT_ANALOG 0x20
+
+#define PPSMC_EXTRAFLAGS_AC2DC_ACTION_MASK 0x07
+#define PPSMC_EXTRAFLAGS_AC2DC_DONT_WAIT_FOR_VBLANK 0x08
+
+#define PPSMC_EXTRAFLAGS_AC2DC_ACTION_GOTODPMLOWSTATE 0x00
+#define PPSMC_EXTRAFLAGS_AC2DC_ACTION_GOTOINITIALSTATE 0x01
+
+
+#define PPSMC_DPM2FLAGS_TDPCLMP 0x01
+#define PPSMC_DPM2FLAGS_PWRSHFT 0x02
+#define PPSMC_DPM2FLAGS_OCP 0x04
+
+
+#define PPSMC_DISPLAY_WATERMARK_LOW 0
+#define PPSMC_DISPLAY_WATERMARK_HIGH 1
+
+
+#define PPSMC_STATEFLAG_AUTO_PULSE_SKIP 0x01
+#define PPSMC_STATEFLAG_POWERBOOST 0x02
+#define PPSMC_STATEFLAG_PSKIP_ON_TDP_FAULT 0x04
+#define PPSMC_STATEFLAG_POWERSHIFT 0x08
+#define PPSMC_STATEFLAG_SLOW_READ_MARGIN 0x10
+#define PPSMC_STATEFLAG_DEEPSLEEP_THROTTLE 0x20
+#define PPSMC_STATEFLAG_DEEPSLEEP_BYPASS 0x40
+
+
+#define FDO_MODE_HARDWARE 0
+#define FDO_MODE_PIECE_WISE_LINEAR 1
+
+enum FAN_CONTROL {
+ FAN_CONTROL_FUZZY,
+ FAN_CONTROL_TABLE
+};
+
+
+#define PPSMC_Result_OK ((uint16_t)0x01)
+#define PPSMC_Result_NoMore ((uint16_t)0x02)
+
+#define PPSMC_Result_NotNow ((uint16_t)0x03)
+#define PPSMC_Result_Failed ((uint16_t)0xFF)
+#define PPSMC_Result_UnknownCmd ((uint16_t)0xFE)
+#define PPSMC_Result_UnknownVT ((uint16_t)0xFD)
+
+typedef uint16_t PPSMC_Result;
+
+#define PPSMC_isERROR(x) ((uint16_t)0x80 & (x))
+
+
+#define PPSMC_MSG_Halt ((uint16_t)0x10)
+#define PPSMC_MSG_Resume ((uint16_t)0x11)
+#define PPSMC_MSG_EnableDPMLevel ((uint16_t)0x12)
+#define PPSMC_MSG_ZeroLevelsDisabled ((uint16_t)0x13)
+#define PPSMC_MSG_OneLevelsDisabled ((uint16_t)0x14)
+#define PPSMC_MSG_TwoLevelsDisabled ((uint16_t)0x15)
+#define PPSMC_MSG_EnableThermalInterrupt ((uint16_t)0x16)
+#define PPSMC_MSG_RunningOnAC ((uint16_t)0x17)
+#define PPSMC_MSG_LevelUp ((uint16_t)0x18)
+#define PPSMC_MSG_LevelDown ((uint16_t)0x19)
+#define PPSMC_MSG_ResetDPMCounters ((uint16_t)0x1a)
+#define PPSMC_MSG_SwitchToSwState ((uint16_t)0x20)
+#define PPSMC_MSG_SwitchToSwStateLast ((uint16_t)0x3f)
+#define PPSMC_MSG_SwitchToInitialState ((uint16_t)0x40)
+#define PPSMC_MSG_NoForcedLevel ((uint16_t)0x41)
+#define PPSMC_MSG_ForceHigh ((uint16_t)0x42)
+#define PPSMC_MSG_ForceMediumOrHigh ((uint16_t)0x43)
+#define PPSMC_MSG_SwitchToMinimumPower ((uint16_t)0x51)
+#define PPSMC_MSG_ResumeFromMinimumPower ((uint16_t)0x52)
+#define PPSMC_MSG_EnableCac ((uint16_t)0x53)
+#define PPSMC_MSG_DisableCac ((uint16_t)0x54)
+#define PPSMC_DPMStateHistoryStart ((uint16_t)0x55)
+#define PPSMC_DPMStateHistoryStop ((uint16_t)0x56)
+#define PPSMC_CACHistoryStart ((uint16_t)0x57)
+#define PPSMC_CACHistoryStop ((uint16_t)0x58)
+#define PPSMC_TDPClampingActive ((uint16_t)0x59)
+#define PPSMC_TDPClampingInactive ((uint16_t)0x5A)
+#define PPSMC_StartFanControl ((uint16_t)0x5B)
+#define PPSMC_StopFanControl ((uint16_t)0x5C)
+#define PPSMC_NoDisplay ((uint16_t)0x5D)
+#define PPSMC_HasDisplay ((uint16_t)0x5E)
+#define PPSMC_MSG_UVDPowerOFF ((uint16_t)0x60)
+#define PPSMC_MSG_UVDPowerON ((uint16_t)0x61)
+#define PPSMC_MSG_EnableULV ((uint16_t)0x62)
+#define PPSMC_MSG_DisableULV ((uint16_t)0x63)
+#define PPSMC_MSG_EnterULV ((uint16_t)0x64)
+#define PPSMC_MSG_ExitULV ((uint16_t)0x65)
+#define PPSMC_PowerShiftActive ((uint16_t)0x6A)
+#define PPSMC_PowerShiftInactive ((uint16_t)0x6B)
+#define PPSMC_OCPActive ((uint16_t)0x6C)
+#define PPSMC_OCPInactive ((uint16_t)0x6D)
+#define PPSMC_CACLongTermAvgEnable ((uint16_t)0x6E)
+#define PPSMC_CACLongTermAvgDisable ((uint16_t)0x6F)
+#define PPSMC_MSG_InferredStateSweep_Start ((uint16_t)0x70)
+#define PPSMC_MSG_InferredStateSweep_Stop ((uint16_t)0x71)
+#define PPSMC_MSG_SwitchToLowestInfState ((uint16_t)0x72)
+#define PPSMC_MSG_SwitchToNonInfState ((uint16_t)0x73)
+#define PPSMC_MSG_AllStateSweep_Start ((uint16_t)0x74)
+#define PPSMC_MSG_AllStateSweep_Stop ((uint16_t)0x75)
+#define PPSMC_MSG_SwitchNextLowerInfState ((uint16_t)0x76)
+#define PPSMC_MSG_SwitchNextHigherInfState ((uint16_t)0x77)
+#define PPSMC_MSG_MclkRetrainingTest ((uint16_t)0x78)
+#define PPSMC_MSG_ForceTDPClamping ((uint16_t)0x79)
+#define PPSMC_MSG_CollectCAC_PowerCorreln ((uint16_t)0x7A)
+#define PPSMC_MSG_CollectCAC_WeightCalib ((uint16_t)0x7B)
+#define PPSMC_MSG_CollectCAC_SQonly ((uint16_t)0x7C)
+#define PPSMC_MSG_CollectCAC_TemperaturePwr ((uint16_t)0x7D)
+
+#define PPSMC_MSG_ExtremitiesTest_Start ((uint16_t)0x7E)
+#define PPSMC_MSG_ExtremitiesTest_Stop ((uint16_t)0x7F)
+#define PPSMC_FlushDataCache ((uint16_t)0x80)
+#define PPSMC_FlushInstrCache ((uint16_t)0x81)
+
+#define PPSMC_MSG_SetEnabledLevels ((uint16_t)0x82)
+#define PPSMC_MSG_SetForcedLevels ((uint16_t)0x83)
+
+#define PPSMC_MSG_ResetToDefaults ((uint16_t)0x84)
+
+#define PPSMC_MSG_SetForcedLevelsAndJump ((uint16_t)0x85)
+#define PPSMC_MSG_SetCACHistoryMode ((uint16_t)0x86)
+#define PPSMC_MSG_EnableDTE ((uint16_t)0x87)
+#define PPSMC_MSG_DisableDTE ((uint16_t)0x88)
+
+#define PPSMC_MSG_SmcSpaceSetAddress ((uint16_t)0x89)
+#define PPSM_MSG_SmcSpaceWriteDWordInc ((uint16_t)0x8A)
+#define PPSM_MSG_SmcSpaceWriteWordInc ((uint16_t)0x8B)
+#define PPSM_MSG_SmcSpaceWriteByteInc ((uint16_t)0x8C)
+
+#define PPSMC_MSG_BREAK ((uint16_t)0xF8)
+
+#define PPSMC_MSG_Test ((uint16_t) 0x100)
+#define PPSMC_MSG_DPM_Voltage_Pwrmgt ((uint16_t) 0x101)
+#define PPSMC_MSG_DPM_Config ((uint16_t) 0x102)
+#define PPSMC_MSG_PM_Controller_Start ((uint16_t) 0x103)
+#define PPSMC_MSG_DPM_ForceState ((uint16_t) 0x104)
+#define PPSMC_MSG_PG_PowerDownSIMD ((uint16_t) 0x105)
+#define PPSMC_MSG_PG_PowerUpSIMD ((uint16_t) 0x106)
+#define PPSMC_MSG_PM_Controller_Stop ((uint16_t) 0x107)
+#define PPSMC_MSG_PG_SIMD_Config ((uint16_t) 0x108)
+#define PPSMC_MSG_Voltage_Cntl_Enable ((uint16_t) 0x109)
+#define PPSMC_MSG_Thermal_Cntl_Enable ((uint16_t) 0x10a)
+#define PPSMC_MSG_Reset_Service ((uint16_t) 0x10b)
+#define PPSMC_MSG_VCEPowerOFF ((uint16_t) 0x10e)
+#define PPSMC_MSG_VCEPowerON ((uint16_t) 0x10f)
+#define PPSMC_MSG_DPM_Disable_VCE_HS ((uint16_t) 0x110)
+#define PPSMC_MSG_DPM_Enable_VCE_HS ((uint16_t) 0x111)
+#define PPSMC_MSG_DPM_N_LevelsDisabled ((uint16_t) 0x112)
+#define PPSMC_MSG_DCEPowerOFF ((uint16_t) 0x113)
+#define PPSMC_MSG_DCEPowerON ((uint16_t) 0x114)
+#define PPSMC_MSG_PCIE_DDIPowerDown ((uint16_t) 0x117)
+#define PPSMC_MSG_PCIE_DDIPowerUp ((uint16_t) 0x118)
+#define PPSMC_MSG_PCIE_CascadePLLPowerDown ((uint16_t) 0x119)
+#define PPSMC_MSG_PCIE_CascadePLLPowerUp ((uint16_t) 0x11a)
+#define PPSMC_MSG_SYSPLLPowerOff ((uint16_t) 0x11b)
+#define PPSMC_MSG_SYSPLLPowerOn ((uint16_t) 0x11c)
+#define PPSMC_MSG_DCE_RemoveVoltageAdjustment ((uint16_t) 0x11d)
+#define PPSMC_MSG_DCE_AllowVoltageAdjustment ((uint16_t) 0x11e)
+#define PPSMC_MSG_DISPLAYPHYStatusNotify ((uint16_t) 0x11f)
+#define PPSMC_MSG_EnableBAPM ((uint16_t) 0x120)
+#define PPSMC_MSG_DisableBAPM ((uint16_t) 0x121)
+#define PPSMC_MSG_Spmi_Enable ((uint16_t) 0x122)
+#define PPSMC_MSG_Spmi_Timer ((uint16_t) 0x123)
+#define PPSMC_MSG_LCLK_DPM_Config ((uint16_t) 0x124)
+#define PPSMC_MSG_VddNB_Request ((uint16_t) 0x125)
+#define PPSMC_MSG_PCIE_DDIPhyPowerDown ((uint32_t) 0x126)
+#define PPSMC_MSG_PCIE_DDIPhyPowerUp ((uint32_t) 0x127)
+#define PPSMC_MSG_MCLKDPM_Config ((uint16_t) 0x128)
+
+#define PPSMC_MSG_UVDDPM_Config ((uint16_t) 0x129)
+#define PPSMC_MSG_VCEDPM_Config ((uint16_t) 0x12A)
+#define PPSMC_MSG_ACPDPM_Config ((uint16_t) 0x12B)
+#define PPSMC_MSG_SAMUDPM_Config ((uint16_t) 0x12C)
+#define PPSMC_MSG_UVDDPM_SetEnabledMask ((uint16_t) 0x12D)
+#define PPSMC_MSG_VCEDPM_SetEnabledMask ((uint16_t) 0x12E)
+#define PPSMC_MSG_ACPDPM_SetEnabledMask ((uint16_t) 0x12F)
+#define PPSMC_MSG_SAMUDPM_SetEnabledMask ((uint16_t) 0x130)
+#define PPSMC_MSG_MCLKDPM_ForceState ((uint16_t) 0x131)
+#define PPSMC_MSG_MCLKDPM_NoForcedLevel ((uint16_t) 0x132)
+#define PPSMC_MSG_Thermal_Cntl_Disable ((uint16_t) 0x133)
+#define PPSMC_MSG_SetTDPLimit ((uint16_t) 0x134)
+#define PPSMC_MSG_Voltage_Cntl_Disable ((uint16_t) 0x135)
+#define PPSMC_MSG_PCIeDPM_Enable ((uint16_t) 0x136)
+#define PPSMC_MSG_ACPPowerOFF ((uint16_t) 0x137)
+#define PPSMC_MSG_ACPPowerON ((uint16_t) 0x138)
+#define PPSMC_MSG_SAMPowerOFF ((uint16_t) 0x139)
+#define PPSMC_MSG_SAMPowerON ((uint16_t) 0x13a)
+#define PPSMC_MSG_SDMAPowerOFF ((uint16_t) 0x13b)
+#define PPSMC_MSG_SDMAPowerON ((uint16_t) 0x13c)
+#define PPSMC_MSG_PCIeDPM_Disable ((uint16_t) 0x13d)
+#define PPSMC_MSG_IOMMUPowerOFF ((uint16_t) 0x13e)
+#define PPSMC_MSG_IOMMUPowerON ((uint16_t) 0x13f)
+#define PPSMC_MSG_NBDPM_Enable ((uint16_t) 0x140)
+#define PPSMC_MSG_NBDPM_Disable ((uint16_t) 0x141)
+#define PPSMC_MSG_NBDPM_ForceNominal ((uint16_t) 0x142)
+#define PPSMC_MSG_NBDPM_ForcePerformance ((uint16_t) 0x143)
+#define PPSMC_MSG_NBDPM_UnForce ((uint16_t) 0x144)
+#define PPSMC_MSG_SCLKDPM_SetEnabledMask ((uint16_t) 0x145)
+#define PPSMC_MSG_MCLKDPM_SetEnabledMask ((uint16_t) 0x146)
+#define PPSMC_MSG_PCIeDPM_ForceLevel ((uint16_t) 0x147)
+#define PPSMC_MSG_PCIeDPM_UnForceLevel ((uint16_t) 0x148)
+#define PPSMC_MSG_EnableACDCGPIOInterrupt ((uint16_t) 0x149)
+#define PPSMC_MSG_EnableVRHotGPIOInterrupt ((uint16_t) 0x14a)
+#define PPSMC_MSG_SwitchToAC ((uint16_t) 0x14b)
+#define PPSMC_MSG_XDMAPowerOFF ((uint16_t) 0x14c)
+#define PPSMC_MSG_XDMAPowerON ((uint16_t) 0x14d)
+
+#define PPSMC_MSG_DPM_Enable ((uint16_t) 0x14e)
+#define PPSMC_MSG_DPM_Disable ((uint16_t) 0x14f)
+#define PPSMC_MSG_MCLKDPM_Enable ((uint16_t) 0x150)
+#define PPSMC_MSG_MCLKDPM_Disable ((uint16_t) 0x151)
+#define PPSMC_MSG_LCLKDPM_Enable ((uint16_t) 0x152)
+#define PPSMC_MSG_LCLKDPM_Disable ((uint16_t) 0x153)
+#define PPSMC_MSG_UVDDPM_Enable ((uint16_t) 0x154)
+#define PPSMC_MSG_UVDDPM_Disable ((uint16_t) 0x155)
+#define PPSMC_MSG_SAMUDPM_Enable ((uint16_t) 0x156)
+#define PPSMC_MSG_SAMUDPM_Disable ((uint16_t) 0x157)
+#define PPSMC_MSG_ACPDPM_Enable ((uint16_t) 0x158)
+#define PPSMC_MSG_ACPDPM_Disable ((uint16_t) 0x159)
+#define PPSMC_MSG_VCEDPM_Enable ((uint16_t) 0x15a)
+#define PPSMC_MSG_VCEDPM_Disable ((uint16_t) 0x15b)
+#define PPSMC_MSG_LCLKDPM_SetEnabledMask ((uint16_t) 0x15c)
+#define PPSMC_MSG_DPM_FPS_Mode ((uint16_t) 0x15d)
+#define PPSMC_MSG_DPM_Activity_Mode ((uint16_t) 0x15e)
+#define PPSMC_MSG_VddC_Request ((uint16_t) 0x15f)
+#define PPSMC_MSG_MCLKDPM_GetEnabledMask ((uint16_t) 0x160)
+#define PPSMC_MSG_LCLKDPM_GetEnabledMask ((uint16_t) 0x161)
+#define PPSMC_MSG_SCLKDPM_GetEnabledMask ((uint16_t) 0x162)
+#define PPSMC_MSG_UVDDPM_GetEnabledMask ((uint16_t) 0x163)
+#define PPSMC_MSG_SAMUDPM_GetEnabledMask ((uint16_t) 0x164)
+#define PPSMC_MSG_ACPDPM_GetEnabledMask ((uint16_t) 0x165)
+#define PPSMC_MSG_VCEDPM_GetEnabledMask ((uint16_t) 0x166)
+#define PPSMC_MSG_PCIeDPM_SetEnabledMask ((uint16_t) 0x167)
+#define PPSMC_MSG_PCIeDPM_GetEnabledMask ((uint16_t) 0x168)
+#define PPSMC_MSG_TDCLimitEnable ((uint16_t) 0x169)
+#define PPSMC_MSG_TDCLimitDisable ((uint16_t) 0x16a)
+#define PPSMC_MSG_DPM_AutoRotate_Mode ((uint16_t) 0x16b)
+#define PPSMC_MSG_DISPCLK_FROM_FCH ((uint16_t) 0x16c)
+#define PPSMC_MSG_DISPCLK_FROM_DFS ((uint16_t) 0x16d)
+#define PPSMC_MSG_DPREFCLK_FROM_FCH ((uint16_t) 0x16e)
+#define PPSMC_MSG_DPREFCLK_FROM_DFS ((uint16_t) 0x16f)
+#define PPSMC_MSG_PmStatusLogStart ((uint16_t) 0x170)
+#define PPSMC_MSG_PmStatusLogSample ((uint16_t) 0x171)
+#define PPSMC_MSG_SCLK_AutoDPM_ON ((uint16_t) 0x172)
+#define PPSMC_MSG_MCLK_AutoDPM_ON ((uint16_t) 0x173)
+#define PPSMC_MSG_LCLK_AutoDPM_ON ((uint16_t) 0x174)
+#define PPSMC_MSG_UVD_AutoDPM_ON ((uint16_t) 0x175)
+#define PPSMC_MSG_SAMU_AutoDPM_ON ((uint16_t) 0x176)
+#define PPSMC_MSG_ACP_AutoDPM_ON ((uint16_t) 0x177)
+#define PPSMC_MSG_VCE_AutoDPM_ON ((uint16_t) 0x178)
+#define PPSMC_MSG_PCIe_AutoDPM_ON ((uint16_t) 0x179)
+#define PPSMC_MSG_MASTER_AutoDPM_ON ((uint16_t) 0x17a)
+#define PPSMC_MSG_MASTER_AutoDPM_OFF ((uint16_t) 0x17b)
+#define PPSMC_MSG_DYNAMICDISPPHYPOWER ((uint16_t) 0x17c)
+#define PPSMC_MSG_CAC_COLLECTION_ON ((uint16_t) 0x17d)
+#define PPSMC_MSG_CAC_COLLECTION_OFF ((uint16_t) 0x17e)
+#define PPSMC_MSG_CAC_CORRELATION_ON ((uint16_t) 0x17f)
+#define PPSMC_MSG_CAC_CORRELATION_OFF ((uint16_t) 0x180)
+#define PPSMC_MSG_PM_STATUS_TO_DRAM_ON ((uint16_t) 0x181)
+#define PPSMC_MSG_PM_STATUS_TO_DRAM_OFF ((uint16_t) 0x182)
+#define PPSMC_MSG_ALLOW_LOWSCLK_INTERRUPT ((uint16_t) 0x184)
+#define PPSMC_MSG_PkgPwrLimitEnable ((uint16_t) 0x185)
+#define PPSMC_MSG_PkgPwrLimitDisable ((uint16_t) 0x186)
+#define PPSMC_MSG_PkgPwrSetLimit ((uint16_t) 0x187)
+#define PPSMC_MSG_OverDriveSetTargetTdp ((uint16_t) 0x188)
+#define PPSMC_MSG_SCLKDPM_FreezeLevel ((uint16_t) 0x189)
+#define PPSMC_MSG_SCLKDPM_UnfreezeLevel ((uint16_t) 0x18A)
+#define PPSMC_MSG_MCLKDPM_FreezeLevel ((uint16_t) 0x18B)
+#define PPSMC_MSG_MCLKDPM_UnfreezeLevel ((uint16_t) 0x18C)
+#define PPSMC_MSG_START_DRAM_LOGGING ((uint16_t) 0x18D)
+#define PPSMC_MSG_STOP_DRAM_LOGGING ((uint16_t) 0x18E)
+#define PPSMC_MSG_MASTER_DeepSleep_ON ((uint16_t) 0x18F)
+#define PPSMC_MSG_MASTER_DeepSleep_OFF ((uint16_t) 0x190)
+#define PPSMC_MSG_Remove_DC_Clamp ((uint16_t) 0x191)
+#define PPSMC_MSG_DisableACDCGPIOInterrupt ((uint16_t) 0x192)
+#define PPSMC_MSG_OverrideVoltageControl_SetVddc ((uint16_t) 0x193)
+#define PPSMC_MSG_OverrideVoltageControl_SetVddci ((uint16_t) 0x194)
+#define PPSMC_MSG_SetVidOffset_1 ((uint16_t) 0x195)
+#define PPSMC_MSG_SetVidOffset_2 ((uint16_t) 0x207)
+#define PPSMC_MSG_GetVidOffset_1 ((uint16_t) 0x196)
+#define PPSMC_MSG_GetVidOffset_2 ((uint16_t) 0x208)
+#define PPSMC_MSG_THERMAL_OVERDRIVE_Enable ((uint16_t) 0x197)
+#define PPSMC_MSG_THERMAL_OVERDRIVE_Disable ((uint16_t) 0x198)
+#define PPSMC_MSG_SetTjMax ((uint16_t) 0x199)
+#define PPSMC_MSG_SetFanPwmMax ((uint16_t) 0x19A)
+#define PPSMC_MSG_WaitForMclkSwitchFinish ((uint16_t) 0x19B)
+#define PPSMC_MSG_ENABLE_THERMAL_DPM ((uint16_t) 0x19C)
+#define PPSMC_MSG_DISABLE_THERMAL_DPM ((uint16_t) 0x19D)
+
+#define PPSMC_MSG_API_GetSclkFrequency ((uint16_t) 0x200)
+#define PPSMC_MSG_API_GetMclkFrequency ((uint16_t) 0x201)
+#define PPSMC_MSG_API_GetSclkBusy ((uint16_t) 0x202)
+#define PPSMC_MSG_API_GetMclkBusy ((uint16_t) 0x203)
+#define PPSMC_MSG_API_GetAsicPower ((uint16_t) 0x204)
+#define PPSMC_MSG_SetFanRpmMax ((uint16_t) 0x205)
+#define PPSMC_MSG_SetFanSclkTarget ((uint16_t) 0x206)
+#define PPSMC_MSG_SetFanMinPwm ((uint16_t) 0x209)
+#define PPSMC_MSG_SetFanTemperatureTarget ((uint16_t) 0x20A)
+
+#define PPSMC_MSG_BACO_StartMonitor ((uint16_t) 0x240)
+#define PPSMC_MSG_BACO_Cancel ((uint16_t) 0x241)
+#define PPSMC_MSG_EnableVddGfx ((uint16_t) 0x242)
+#define PPSMC_MSG_DisableVddGfx ((uint16_t) 0x243)
+#define PPSMC_MSG_UcodeAddressLow ((uint16_t) 0x244)
+#define PPSMC_MSG_UcodeAddressHigh ((uint16_t) 0x245)
+#define PPSMC_MSG_UcodeLoadStatus ((uint16_t) 0x246)
+
+#define PPSMC_MSG_DRV_DRAM_ADDR_HI ((uint16_t) 0x250)
+#define PPSMC_MSG_DRV_DRAM_ADDR_LO ((uint16_t) 0x251)
+#define PPSMC_MSG_SMU_DRAM_ADDR_HI ((uint16_t) 0x252)
+#define PPSMC_MSG_SMU_DRAM_ADDR_LO ((uint16_t) 0x253)
+#define PPSMC_MSG_LoadUcodes ((uint16_t) 0x254)
+#define PPSMC_MSG_PowerStateNotify ((uint16_t) 0x255)
+#define PPSMC_MSG_COND_EXEC_DRAM_ADDR_HI ((uint16_t) 0x256)
+#define PPSMC_MSG_COND_EXEC_DRAM_ADDR_LO ((uint16_t) 0x257)
+#define PPSMC_MSG_VBIOS_DRAM_ADDR_HI ((uint16_t) 0x258)
+#define PPSMC_MSG_VBIOS_DRAM_ADDR_LO ((uint16_t) 0x259)
+#define PPSMC_MSG_LoadVBios ((uint16_t) 0x25A)
+#define PPSMC_MSG_GetUcodeVersion ((uint16_t) 0x25B)
+#define DMCUSMC_MSG_PSREntry ((uint16_t) 0x25C)
+#define DMCUSMC_MSG_PSRExit ((uint16_t) 0x25D)
+#define PPSMC_MSG_EnableClockGatingFeature ((uint16_t) 0x260)
+#define PPSMC_MSG_DisableClockGatingFeature ((uint16_t) 0x261)
+#define PPSMC_MSG_IsDeviceRunning ((uint16_t) 0x262)
+#define PPSMC_MSG_LoadMetaData ((uint16_t) 0x263)
+#define PPSMC_MSG_TMON_AutoCaliberate_Enable ((uint16_t) 0x264)
+#define PPSMC_MSG_TMON_AutoCaliberate_Disable ((uint16_t) 0x265)
+#define PPSMC_MSG_GetTelemetry1Slope ((uint16_t) 0x266)
+#define PPSMC_MSG_GetTelemetry1Offset ((uint16_t) 0x267)
+#define PPSMC_MSG_GetTelemetry2Slope ((uint16_t) 0x268)
+#define PPSMC_MSG_GetTelemetry2Offset ((uint16_t) 0x269)
+#define PPSMC_MSG_EnableAvfs ((uint16_t) 0x26A)
+#define PPSMC_MSG_DisableAvfs ((uint16_t) 0x26B)
+
+#define PPSMC_MSG_PerformBtc ((uint16_t) 0x26C)
+#define PPSMC_MSG_VftTableIsValid ((uint16_t) 0x275)
+#define PPSMC_MSG_UseNewGPIOScheme ((uint16_t) 0x277)
+#define PPSMC_MSG_GetEnabledPsm ((uint16_t) 0x400)
+#define PPSMC_MSG_AgmStartPsm ((uint16_t) 0x401)
+#define PPSMC_MSG_AgmReadPsm ((uint16_t) 0x402)
+#define PPSMC_MSG_AgmResetPsm ((uint16_t) 0x403)
+#define PPSMC_MSG_ReadVftCell ((uint16_t) 0x404)
+
+#define PPSMC_MSG_GFX_CU_PG_ENABLE ((uint16_t) 0x280)
+#define PPSMC_MSG_GFX_CU_PG_DISABLE ((uint16_t) 0x281)
+#define PPSMC_MSG_GetCurrPkgPwr ((uint16_t) 0x282)
+
+#define PPSMC_MSG_SetGpuPllDfsForSclk ((uint16_t) 0x300)
+#define PPSMC_MSG_Didt_Block_Function ((uint16_t) 0x301)
+
+#define PPSMC_MSG_SecureSRBMWrite ((uint16_t) 0x600)
+#define PPSMC_MSG_SecureSRBMRead ((uint16_t) 0x601)
+#define PPSMC_MSG_SetAddress ((uint16_t) 0x800)
+#define PPSMC_MSG_GetData ((uint16_t) 0x801)
+#define PPSMC_MSG_SetData ((uint16_t) 0x802)
+
+typedef uint16_t PPSMC_Msg;
+
+#define PPSMC_EVENT_STATUS_THERMAL 0x00000001
+#define PPSMC_EVENT_STATUS_REGULATORHOT 0x00000002
+#define PPSMC_EVENT_STATUS_DC 0x00000004
+
+#pragma pack(pop)
+
+#endif
+
diff --git a/drivers/gpu/drm/amd/powerplay/inc/polaris10_pwrvirus.h b/drivers/gpu/drm/amd/powerplay/inc/polaris10_pwrvirus.h
new file mode 100644
index 0000000..f497e7d
--- /dev/null
+++ b/drivers/gpu/drm/amd/powerplay/inc/polaris10_pwrvirus.h
@@ -0,0 +1,10088 @@
+/*
+ * Copyright 2015 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+#ifndef _POLARIS10_PWRVIRUS_H
+#define _POLARIS10_PWRVIRUS_H
+
+#define mmSMC_IND_INDEX_11 0x01AC
+#define mmSMC_IND_DATA_11 0x01AD
+#define mmCP_HYP_MEC1_UCODE_ADDR 0xf81a
+#define mmCP_HYP_MEC1_UCODE_DATA 0xf81b
+#define mmCP_HYP_MEC2_UCODE_ADDR 0xf81c
+#define mmCP_HYP_MEC2_UCODE_DATA 0xf81d
+
+enum PWR_Command {
+ PwrCmdNull = 0,
+ PwrCmdWrite,
+ PwrCmdEnd,
+ PwrCmdMax
+};
+
+typedef enum PWR_Command PWR_Command;
+
+struct PWR_Command_Table {
+ PWR_Command command;
+ uint32_t data;
+ uint32_t reg;
+};
+
+typedef struct PWR_Command_Table PWR_Command_Table;
+
+
+#define PWR_VIRUS_TABLE_SIZE 10031
+
+static const PWR_Command_Table pwr_virus_table[PWR_VIRUS_TABLE_SIZE] = {
+ { PwrCmdWrite, 0x00000000, mmRLC_CNTL },
+ { PwrCmdWrite, 0x00000002, mmRLC_SRM_CNTL },
+ { PwrCmdWrite, 0x15000000, mmCP_ME_CNTL },
+ { PwrCmdWrite, 0x50000000, mmCP_MEC_CNTL },
+ { PwrCmdWrite, 0x80000004, mmCP_DFY_CNTL },
+ { PwrCmdWrite, 0x0840800a, mmCP_RB0_CNTL },
+ { PwrCmdWrite, 0xf30fff0f, mmTCC_CTRL },
+ { PwrCmdWrite, 0x00000002, mmTCC_EXE_DISABLE },
+ { PwrCmdWrite, 0x000000ff, mmTCP_ADDR_CONFIG },
+ { PwrCmdWrite, 0x540ff000, mmCP_CPC_IC_BASE_LO },
+ { PwrCmdWrite, 0x000000b4, mmCP_CPC_IC_BASE_HI },
+ { PwrCmdWrite, 0x00010000, mmCP_HYP_MEC1_UCODE_ADDR },
+ { PwrCmdWrite, 0x00041b75, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000710e8, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000910dd, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000a1081, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000b016f, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000c0e3c, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000d10ec, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000e0188, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00101b5d, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00150a6c, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00170c5e, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x001d0c8c, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x001e0cfe, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00221408, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00370d7b, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00390dcb, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x003c142f, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x003f0b27, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00400e63, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00500f62, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00460fa7, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00490fa7, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x005811d4, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00680ad6, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00760b00, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00780b0c, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00790af7, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x007d1aba, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x007e1abe, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00591260, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x005a12fb, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00861ac7, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x008c1b01, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x008d1b34, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00a014b9, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00a1152e, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00a216fb, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00a41890, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00a31906, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00a50b14, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00621387, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x005c0b27, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00160a75, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC1_UCODE_DATA },
+ { PwrCmdWrite, 0x00010000, mmCP_HYP_MEC2_UCODE_ADDR },
+ { PwrCmdWrite, 0x00041b75, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000710e8, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000910dd, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000a1081, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000b016f, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000c0e3c, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000d10ec, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000e0188, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00101b5d, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00150a6c, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00170c5e, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x001d0c8c, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x001e0cfe, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00221408, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00370d7b, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00390dcb, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x003c142f, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x003f0b27, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00400e63, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00500f62, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00460fa7, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00490fa7, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x005811d4, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00680ad6, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00760b00, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00780b0c, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00790af7, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x007d1aba, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x007e1abe, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00591260, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x005a12fb, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00861ac7, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x008c1b01, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x008d1b34, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00a014b9, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00a1152e, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00a216fb, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00a41890, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00a31906, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00a50b14, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00621387, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x005c0b27, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x00160a75, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x000f016a, mmCP_HYP_MEC2_UCODE_DATA },
+ { PwrCmdWrite, 0x80000004, mmCP_DFY_CNTL },
+ { PwrCmdWrite, 0x000000b4, mmCP_DFY_ADDR_HI },
+ { PwrCmdWrite, 0x540fe800, mmCP_DFY_ADDR_LO },
+ { PwrCmdWrite, 0x7e000200, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e020201, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e040204, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e060205, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a080500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a0a0303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xbf810000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x54106f00, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x000400b4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00004000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00804fac, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000004, mmCP_DFY_CNTL },
+ { PwrCmdWrite, 0x000000b4, mmCP_DFY_ADDR_HI },
+ { PwrCmdWrite, 0x540fef00, mmCP_DFY_ADDR_LO },
+ { PwrCmdWrite, 0xc0031502, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00001e00, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000004, mmCP_DFY_CNTL },
+ { PwrCmdWrite, 0x000000b4, mmCP_DFY_ADDR_HI },
+ { PwrCmdWrite, 0x540ff000, mmCP_DFY_ADDR_LO },
+ { PwrCmdWrite, 0xc424000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000145, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94800001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95400001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95800001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc810000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdcc10000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdd010000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdd410000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdd810000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4080061, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24ccffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3cd08000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9500fffd, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1cd0ffcf, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d018001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4140004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x050c0019, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x84c00000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000023, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000067, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000006a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000006d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000079, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000084, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000008f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000099, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800000a0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800000af, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400053, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4080007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x388c0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x08880002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04100003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c00005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x98800003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000002d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04100005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000043, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00050, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000055, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28080001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc000004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d808001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd88130b8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc180000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc140000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc100000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc0c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc800005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc080000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000168, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd013278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24cc0700, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400029, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113255, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd01324f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113254, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1d10ffdf, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd013254, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x10cc0014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1d10c017, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d0d000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd0130b7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x14cc0010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9c00036, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000005d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc00c4000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc130b5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x14d00011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9500fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc030000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c01b10, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc00e0080, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc130b5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000013b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc00e0800, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc130b5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000013b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400053, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04100006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000043, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00050, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000055, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x280c0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00052, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28180039, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400053, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04100007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000043, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00050, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000055, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x280c0010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00052, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28180039, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400053, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04100008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000043, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc0003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00050, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000055, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x280c0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00052, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28180039, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc030000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000069, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28080001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ca88004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc800079, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04280001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc00006f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000013b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04100010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000043, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000055, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28180080, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd013278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc00c4000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113254, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1d10c017, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc130b5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd0130b7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000013b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96800001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97400001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97800001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc810000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd4c0380, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdcc0388, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55dc0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdcc038c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce0c0390, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56200020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce0c0394, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce4c0398, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56640020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce4c039c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce8c03a0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56a80020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce8c03a4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcecc03a8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56ec0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcecc03ac, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf0c03b0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x57300020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf0c03b4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf4c03b8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x57740020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf4c03bc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf8c03c0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x57b80020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf8c03c4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfcc03c8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x57fc0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfcc03cc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9000033, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41c0009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25dc0010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c0fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41c000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05dc002f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc12009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d200a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc012009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9000034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25e01c00, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12200013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25e40300, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12640008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25e800c0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12a80002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25ec003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e25c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7eae400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7de5c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xddc10000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc02ee000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec1c200, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c005f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00037, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24d000ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31100006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9500007b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000190, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc1c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc1c200, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4df0388, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4d7038c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51540020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d5dc01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4e30390, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4d70394, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51540020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d62001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4e70398, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4d7039c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51540020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d66401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4eb03a0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4d703a4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51540020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d6a801a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4ef03a8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4d703ac, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51540020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d6ec01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4f303b0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4d703b4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51540020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d73001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4f703b8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4d703bc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51540020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d77401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4fb03c0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4d703c4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51540020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d7b801a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4ff03c8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4d703cc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51540020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d7fc01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc080000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4d70380, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4080001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1c88001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0083, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c00010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc0e0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c0000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0082, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24d00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9900000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18cc01e3, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3cd00004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95000008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0085, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18cc006a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x98c00005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0082, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18cc01e3, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3cd00004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9900fffa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc180000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc140000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc100000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc0c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc800004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc080000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4080001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1c88001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc180000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc140000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc100000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc0c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc800004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc080000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400051, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04180018, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32640002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a40001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4293265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x040c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1aac0027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2aa80080, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce813265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00017, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd80002f1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04080002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x08880001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080250, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080258, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080230, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080238, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080240, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080248, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080268, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080270, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080280, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080228, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000367, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9880fff3, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04080010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x08880001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd80c0309, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd80c0319, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04cc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9880fffc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc00e0100, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc130b5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000016e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4180032, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95800001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d0003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24d4001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24d80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x155c0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05e80180, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9900000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x202c003d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec1325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000168, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000aa7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000bfc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800012e9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4200007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a200001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001b70, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000190, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc410001b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000032, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000031, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9900091a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24d000ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05280196, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d4fe04, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29540008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800001b4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000032b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000350, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000352, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000035f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000701, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000047c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000019f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000800, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc419325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1d98001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd81325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4140004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04100002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000043, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc0002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00050, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0044, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27fc0003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c00006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc00c4000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc130b5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000055, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd88130b8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400028, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400029, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9400036, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193256, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3254, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x15540008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd40005b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd40005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd40005d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd840006d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc421325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d3249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11540015, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19a4003c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1998003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1af0007d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11dc000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1264001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x15dc000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d65400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13300018, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1a38003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dd5c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7df1c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800045, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00100, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc411326a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc415326b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc419326c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d326d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc425326e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4293279, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800077, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd000056, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400057, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800058, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00059, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x259c8000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c00004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce40005a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29988000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd813265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113248, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2510000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd000073, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc411326f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17300019, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25140fff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95400007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800003a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001b6d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4153279, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400077, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd00005f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000075, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26f00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x15100010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d190004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd000035, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000035, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1af07fe8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf00000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf00000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001427, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04340022, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x07740001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04300010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdf430000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c434001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4412e01, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0434001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdf430000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4400078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdf030000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4412e40, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc41c030, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc41c031, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43dc031, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04343000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113246, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3245, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf413267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51100020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dd1c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4353267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45dc0160, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc810001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b4c0057, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b700213, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b740199, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f4f400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f73400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55180020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2198003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1c00025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x248dfffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc12e00, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c434001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c434001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00142b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1af4007d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2bfc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33740003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26d80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1ae8003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9680000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253277, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26680001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96800009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2a640002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce413277, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253348, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce413348, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253348, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b400003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x958000d8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000315, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253277, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04303000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26680001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf013267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193246, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3245, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96800041, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51980020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b342010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d9d801a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1714000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25540800, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b30c012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x459801b0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d77400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f37000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2b300000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf00001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd180001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04240010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x199c01e2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e5e4002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3e5c0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3e540002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc80c0011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8140011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x54d00020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55580020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000282, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95400015, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc80c0011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a640002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x041c0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x54d00020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8140011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45980004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x041c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf00001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd180001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8180011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000282, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8140011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55580020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000282, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45980004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc80c0011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf00001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd180001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8100011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8140011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55580020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc1334e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd01334f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd413350, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd813351, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd881334d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193273, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3275, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40d3271, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113270, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4153274, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x50cc0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cd0c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cdcc011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05900008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd00006a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc0006b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3272, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d594002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x54d00020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc12e23, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd012e24, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc12e25, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193246, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3245, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x15540002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51980020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d9d801a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc81c001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b340057, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b280213, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b300199, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45980198, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f37000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f2b000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55e40020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf000024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1800025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd40000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd40000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40d3249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x20cc003c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc13249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113274, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdd430000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc01e0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29dc0002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04280000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000036, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc400078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc400078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2d540002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95400022, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x078c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x07d40000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00120d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001239, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001232, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04f80000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x057c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc414000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41c0019, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dd5c005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25dc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd840007c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400074, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400069, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c018a6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4412e22, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800007c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c018a2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0019, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cd4c005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24cc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9680fffc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800002e3, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0057, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cd0c002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9680fffd, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800002e3, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000069, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd013273, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd013275, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000074, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc414005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9540188f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40d3249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc013cfff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cd0c009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc13249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9680000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0077, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x38d00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99000006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04cc0002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdcc30000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c01882, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4400078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000304, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c41c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c41c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd840002f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41c0015, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400030, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41c0016, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000030, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41c0016, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41c0015, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc81c001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x49980198, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55e40020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x459801a0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf000024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1800025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04302000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf013267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96800004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000036, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000329, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc812e00, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04302000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf013267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193256, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d3249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x16ec001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000028, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1998003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec00031, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000036, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97800004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce00000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1a18003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd88130b8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04100000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d43c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4093249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1888003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94800015, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400074, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000671, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a400006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc419324c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x259c0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1598001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c0000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9580000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99000003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400036, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x14d80011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24dc00ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31e00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31dc0003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9580fff0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a000003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9c00036, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94800004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000074, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95801827, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800036, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8c00036, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc424000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32640002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a400004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4180014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9580ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd840002f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x14dc0011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c0fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00037, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000190, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800006d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3246, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193245, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51dc0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d9d801a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400028, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400029, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc420000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32200002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a0000ad, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04200032, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xde030000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400033, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04080000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27fc0002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c0fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c0015, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d3249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1af4003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9740004d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4080060, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ca88005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24880001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f4b4009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97400046, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313274, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4100057, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d33400c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97400009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28240100, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e6a4004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce400079, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1eecffdd, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec13249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf013273, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf013275, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800003c3, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc429326f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1aa80030, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96800006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28240001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06a80008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e6a8004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800035, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3272, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25cc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x10cc0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19e80042, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25dc0006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11dc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e8e800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7de9c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40d3271, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4293270, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x50cc0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ce8c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cd30011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11e80007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2aa80000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce80001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd300001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4300011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b30003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33300000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4240059, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1660001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e320009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0328000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e72400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0430000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04300008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc02ac000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d310002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17300002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2aa87600, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cd0c011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd0c00025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04280222, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4280058, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x22ec003d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec13249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd013273, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce813275, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800007b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8380018, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x57b00020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04343108, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc429325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x040c3000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13740008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2374007e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32a80003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc13267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40d3267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18ec0057, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18e40213, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18cc0199, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cecc00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ce4c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94800003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4400078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800003e7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04200022, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xde030000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1800025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04200010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xde030000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45980104, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1800025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x49980104, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc81c001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45980168, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55e00020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1800025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800003f2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000448, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x040c2000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc13267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40d3267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40d3249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18cc003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400030, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c0016, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000030, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c0016, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c0015, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4300025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380081, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf813279, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf41326e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01326d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c0000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x254c0700, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc424001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x10cc0010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1a641fe8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc0726, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2a640200, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc1237b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2264003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8813260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce41325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4240033, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4280034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9000036, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001427, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xde430000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce40000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c01755, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4400078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9680000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce80000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06a80002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xde830000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce80000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c0174c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4400078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00142b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4393265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2bb80040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400032, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf813265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4200012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a00ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4100044, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19180024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8100072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x551c003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95800010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000043d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc00c8000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd840006c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28200000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000043f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc00c4000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x282000f0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113255, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd01324f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd88130b8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc130b5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000053, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x195c00e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2555fff0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0360001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29540001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04240000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04280004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc420000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32200002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a000009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec1c200, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc5e124dc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0aa80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef6c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e624001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80fff9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc02ee000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2555fff0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec1c200, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29540008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc81c001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55e00020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d3255, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4353259, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8013260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45980158, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1800025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x49980158, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45980170, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4200012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x16200010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a00fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1800025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc429324f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd000008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40d325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d43c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x195400e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1154000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18dc00e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05e80488, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d0006c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18f807f0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18e40077, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18ec0199, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e6e400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000048e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000494, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800004de, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000685, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000686, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800006ac, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1ccc001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc1325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc411325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x251001ef, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd01325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4293254, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1264000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4300004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d79400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e7a400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52a8001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x15180001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d69401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x202c007d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec1325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95000008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95800028, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d3267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193246, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3245, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1aec0028, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40d325c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800004cc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d3256, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc419324e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26e8003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1aec003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12f4000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d324d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40d324f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d75401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04100002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d290004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f8f4001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f52800f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51980020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d9d801a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x50e00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a800002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800004d1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d0dc002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x6665fc00, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e5e401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7da1c011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd140000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2a644000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce400002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f534002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x6665fc00, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e76401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1800002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce400002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800004d7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193258, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1aec003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3257, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4213259, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12f4000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d75401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51980020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52200002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d9d801a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7da1c011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd140000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2a644000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce400002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x202c003d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf000008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec1325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x259c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x15980004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05e804e3, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800004e7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800004f0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000505, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000016a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc435325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x277401ef, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf41325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000671, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9640fff4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17e00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd84131db, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc430001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b301ff8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2b300400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2330003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26edf000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8413260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec1325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05a80507, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000050c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000528, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000057d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800005c2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800005f3, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000671, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a400012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bd400e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c004a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd40005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41c004d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec0005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c0000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4100019, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d150005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99000008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00063b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113277, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2511fffd, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd013277, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801326f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000624, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04240012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1be00fe4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce413260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000066, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400068, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000671, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bd400e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c004a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd40005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41c004d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec0005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c0000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4100019, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d150005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99000009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400067, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00063b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113277, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2511fffd, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd013277, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801326f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000624, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bd400e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c0060, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ed6c005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26ec0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113271, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4153270, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193272, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3273, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04280022, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51100020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d51401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113274, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4213275, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253276, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313248, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1400061, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2730000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13300010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7db1800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800060, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05dc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00062, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c3000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd000063, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000064, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce400065, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec13267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d3246, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313245, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4353267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce813260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52ec0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc820001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b700057, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b680213, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b740199, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x46ec0188, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f73400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f6b400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56240020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd2c00025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c2000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17e00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec13267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d3267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26e01000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a00fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9c131fc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113277, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41c000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc420000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11dc0002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7de1c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11dc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29dc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25140001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x191807e4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x192007ec, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95400004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc1334a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9580000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09980001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x041c0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95800005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09980001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51dc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x69dc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980fffd, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7de20014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x561c0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce013344, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc13345, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95400022, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c3000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec13267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d3246, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313245, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4353267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc425334d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26640001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9640fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc419334e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d334f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4213350, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253351, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52ec0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b680057, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b700213, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b740199, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x46ec01b0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f6b400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f73400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd2c00025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c2000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec13267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d3267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04280032, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce813260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800068, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2010007d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd01325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc411325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1910003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9500fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04100040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd00001b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc410000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9900ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04100060, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd00001b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc410000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9900ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2010003d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd01325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113277, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25140001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x191807e4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9540000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2511fffd, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd013277, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41c000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc420000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11dc0002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7de1c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11dc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc1334a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95800005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8013344, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8013345, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4180050, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41c0052, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04280042, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd813273, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc13275, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce813260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9000068, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400067, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x07d40000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00120d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00124f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001232, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x057c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c3000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec13267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d3246, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313245, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4353267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52ec0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b680057, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b700213, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b740199, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc820001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x46ec0190, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f6b400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f73400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56240020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd2c00025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c2000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec13267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4153249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2154003d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41c0019, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bd800e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dd9c005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25dc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c004a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd80005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc420004d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec0005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11dc0010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e1e000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd413249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce01326f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28340001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f598004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800035, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1be800e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c004a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce80005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801327a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800005f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000075, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800007f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc424004c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce41326e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec0005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28240100, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e6a4004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce400079, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc435325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x277401ef, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04240020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce41325e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8013260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf41325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xda000068, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113277, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41c000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc420000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11dc0002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7de1c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11dc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29dc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25140001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9540002d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc1334a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c3000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec13267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d3246, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313245, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4353267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc425334d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26640001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9640fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc419334e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d334f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4213350, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253351, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52ec0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b680057, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b700213, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b740199, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x46ec01b0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f6b400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f73400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd2c00025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c2000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec13267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d3267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41c000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc420000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11dc0002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7de1c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11dc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc1334a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc430000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33300002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04240000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1be000e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0360001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04280004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec1c200, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc63124dc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0aa80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef6c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e724001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80fff9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc02ee000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec1c200, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fc14001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40d3249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18cc003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x98c00005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x194c1c03, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc0003b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c002d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000697, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc420004a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x194c00e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc0005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c004c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431326d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27301fff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce00005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cf0c00d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x98c00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c0007e0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc430001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b301ff8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2b300400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2330003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc411325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x251001ef, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd01325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25100007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31100005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9900008e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000075e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x202c007d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec1325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4293265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4353254, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26a9feff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1374000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1774000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d30b8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc411325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x251001ef, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd01325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce813265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400100, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc00ac006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc00e0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28880700, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc0014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c0006de, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x14cc0010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x30d4000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04cc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x10cc0010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc0014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99400009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41530b8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19980028, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99400003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99800002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800006c8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc411325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x251001ef, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd01325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x15600008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8380023, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4180081, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11a00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fa38011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4100026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d1a0002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x282c2002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3e280008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4300027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd3800025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf000024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x202400d0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ca48001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc800026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28240006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a640001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a40fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a800004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32280000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a800002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24d8003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd840003c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec0003a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd81a2a4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25dc0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40d3249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18cc003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c0000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc420004a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x194c00e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc0005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c004c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431326d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27301fff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce00005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cf0c00d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000712, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x194c1c03, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc0003b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c002d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05e80714, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000071c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000720, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000747, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000071d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800007c4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000732, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000745, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000744, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x98c00006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000072e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x98c00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c0007e0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c0000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2a64008c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce413265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc430001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b301fe8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2b300400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2330003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8013260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04240000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000075e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x98c0fff1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c0007e0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000723, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41f02f1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8013247, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000743, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8813247, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd88130b8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd000008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x98c0ffde, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000072e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x98c00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c0007e0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x15600008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd84131db, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc430001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b301ff8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2b300400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2330003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8413260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04240000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x041c3000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc13267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25dc8000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41c004a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x195800e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd80005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418004c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd81326e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc0005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25dd7fff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc13265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3246, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193245, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d3267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51e00020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e1a001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x46200200, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04283247, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04300033, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1af80057, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1af40213, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f7b400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f6f400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd2000025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc6990000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x329c325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x329c3269, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c00006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x329c3267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc01defff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d9d8009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000078a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25980000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0b300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06a80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00fff2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc03e7ff0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f3f0009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1f30001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf013249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc03e4000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc13254, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8013254, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801324f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8013255, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8013247, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b300028, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00120d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001219, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001232, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9900000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd88130b8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9700000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43d30b5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bf0003a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b000b80, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x203c003a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc430000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300700, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13300014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2b300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf0130b7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc130b5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x46200008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd2000025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x043c2000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc13267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43d3267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4080007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x259c0003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31dc0003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x040c3000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc13267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40d3267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18ec0057, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18e40213, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18cc0199, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cecc00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ce4c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193246, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3245, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51980020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d9d801a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000448, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x040c2000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc13267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40d3267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc800010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31980002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x041c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19580066, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x15600008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x040c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0120001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11980003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04240004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7da18001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4200007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc1c200, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d24db, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cd0c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a640001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dd9c005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25dc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a40fff8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9580137b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc00ee000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc1c200, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd840004f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113269, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19080070, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x190c00e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2510003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2518000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd813268, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05a80809, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000080e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000080f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000898, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000946, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800009e1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000a5a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04a80811, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000815, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000834, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000085e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000085e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04341001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d3045, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec1c091, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31300021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9700000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd84002f1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43130b8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4293059, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56a8001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f2b000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b000241, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000084a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43130b6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b000003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc02f0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec130b6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4252087, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x5668001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26a80005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80fffd, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd80130b6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000084a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04341001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431ecaa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300080, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc02e0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec130b6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd80130b6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31300021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9700000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd84002f1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43130b8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4293059, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56a8001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f2b000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00021d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdd410000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x040c0005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd84802e9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001a41, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43b02f1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b800006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd88130b8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec80278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56f00020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf080280, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001608, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc140000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8813247, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd80802e9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000085e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31100011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x950001fa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc02e0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2aec0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc01c0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0180001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc00c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11a40006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7de6000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x10e40008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e26000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e2e000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113254, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1d10ffdf, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2110003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd013254, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801324f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8013255, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1d10ff9e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd013254, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8013247, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801325e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0245301, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce413249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801325f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc425326c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0121fff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29108eff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e524009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce41326c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc425325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0127ff0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e524009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce41325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc425325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0131fff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e524009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce41325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801326d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801326e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8013279, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x08cc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000866, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc00c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95800003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09980001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000866, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0100010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dd2400c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a400004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0180003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dd1c002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000866, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000a5a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04a8089a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000089e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800008fa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000945, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000945, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31300022, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43130b8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04183000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd813267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113246, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193245, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51100020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d91801a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x459801e0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2738000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b342010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x172c000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26ec0800, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b30c012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef7400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f37000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2b300000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf00001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd180001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8300011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000036, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd180001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8340011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9740002f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13b80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc79d3300, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc7a13301, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8393300, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0260001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce793301, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc424005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x964012a4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c028009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9740001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27580001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99800004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x57740001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06a80400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800008d2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4180006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29640001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce40001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x242c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06ec0400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x57740001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27580001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980fffd, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc02620c0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce41c078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce81c080, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c081, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01c082, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x57240020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce41c083, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0260400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e6e400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce41c084, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7eae8001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f2f0011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800008d2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4180006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdf93300, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce393301, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04182000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd813267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000903, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31240022, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43130b8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4af0280, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4b30278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52ec0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ec30011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32f80000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b800011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x043c0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04280000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x67180001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0bfc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x57300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95800006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001628, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a400003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd981325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000915, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9c1325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06a80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc0fff6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f818001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001606, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d838001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94800010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3259, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc421325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x16240014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12640014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1a2801f0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12a80010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2620ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e2a000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7de1c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e5e400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b800002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2264003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce41325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8013259, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00075e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4af0228, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x043c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x66d80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95800010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04300002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1330000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13f40014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f73400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04380040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf80001b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc438000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04380060, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf80001b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc438000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x07fc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56ec0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33e80010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9680ffec, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000a5a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000a5a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04a80948, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000094c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000099b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800009e0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800009e0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04183000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd813267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113246, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193245, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51100020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d91801a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x459801e0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2738000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b342010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x172c000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26ec0800, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b30c012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef7400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f37000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2b300000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf00001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd180001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8300011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000033, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd180001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8340011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9740002c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13b80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc79d3300, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc7a13301, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8393300, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0260001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce793301, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc424005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x964011fe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c028009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9740001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27580001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99800004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x57740001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06a80400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000978, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4180006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29640001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce40001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x242c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06ec0400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x57740001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27580001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980fffd, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0260010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce41c078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01c080, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x57240020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce41c081, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce81c082, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c083, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0260800, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e6e400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce41c084, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7eae8001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f2f0011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000978, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4180006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdf93300, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce393301, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04182000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd813267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193246, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3245, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51980020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dda801a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d41c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e838011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd84802e9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001802, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x469c0390, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04183000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd813267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b342010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x172c000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26ec0800, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b30c012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef7400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f37000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2b300000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf00001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45dc0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1c0001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4200011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45dc0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1c0001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4240011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45dc0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1c0001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4280011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45dc0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1c0001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c0011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45dc0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1c0001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4300011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45dc0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1c0001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45dc0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1c0001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04182000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd813267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x043c0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c0014df, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000a5a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000a5a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31280014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce8802ef, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a800062, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31280034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a800060, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04a809e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800009ec, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000a45, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000a59, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000a59, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113246, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193245, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51100020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d91801a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45980400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4b30258, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4a70250, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x53300020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e72401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b342010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x172c000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26ec0800, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b30c012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef7400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f37000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2b300000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf00001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x66740001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97400041, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04383000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf813267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4393267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b800001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd180001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc438000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4300011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b38007e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33b40003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b400003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x4598001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9740002f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45980004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd180001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc438000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45980004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd180001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc438000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4100011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45980004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd180001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc438000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf4002eb, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45980004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd180001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc438000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf4002ec, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45980004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd180001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc438000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf4002ed, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45980004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd180001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc438000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf4002ee, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45980004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04382000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf813267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd84802e9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001715, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04382000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf813267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56640001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0aec0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac0ffbc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04341001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94800005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431ecaa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300080, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000a55, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43130b6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x233c0032, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc130b6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf0130b6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc49302ef, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99000003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8413247, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000a5a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000a5a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04180001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x5198001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd813268, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193269, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2598000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd80002f1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8013268, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800004f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04380001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x53b8001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7db9801a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd813268, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000a5e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400029, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c01106, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc412e01, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc412e02, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc412e03, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc412e00, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000aa7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400029, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c010fd, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x50640020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ce4c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd0c00072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc80c0072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x58e801fc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12a80009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2aa80000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd0c0001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce80001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc424000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a40ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04240010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18dc01e2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e5e4002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3e5c0003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3e540002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8180011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8100011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8100011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55140020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000aa2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9540000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8180011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x44cc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55900020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd0c0001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc424000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a40ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4140011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000aa2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x44cc0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4180011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd0c0001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc424000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a40ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8100011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55140020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd812e01, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd012e02, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd412e03, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc412e00, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2aa80008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2264003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce413249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc410001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4140028, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95000005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1e64001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce413249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001b70, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x14d00010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4180030, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99000004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99400009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ab1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00037, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000190, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc420001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000032, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a0010ac, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000aa7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd880003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8c0003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001082, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8c00040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800010de, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc010ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d403f7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d0cc009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41b0367, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d958004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d85800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc1e0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc424000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32640002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d001fc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05280adc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000af1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000adf, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ae7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000ace, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8c00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd8d2000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c00010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d803f7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc010ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d0cc009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04140000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11940014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29544001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a400002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29544003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000af4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8c00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd44d2000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc424000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32640002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8c00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd44dc000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d0003c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95000006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000ace, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd8d2c00, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000b0a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd44d2c00, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28148004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24d800ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00019, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4593240, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400029, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c0105e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c410001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x50540020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c418001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2198003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x199c0034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00028, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2aa80008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d324f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313255, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef3400c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001b70, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x14e80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a8000af, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x041c0002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c01c8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000d61, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400029, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c01043, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c410001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x50540020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c418001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18a01fe8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3620005c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a00000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2464003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc6290ce7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x16ac001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c00004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26ac003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ee6c00d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c00005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06200001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2620000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a00fff8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000016a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000367, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc424005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9640102e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x199c0037, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19a00035, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2aa80008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c0005d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d3256, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2330003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x16f8001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9780000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253248, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc035f0ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e764009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19b401f8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13740008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e76400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce413248, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d15001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1000072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8100072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55140020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x199c0034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400029, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b800004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1ae4003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000b7c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4353254, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x16a80008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1aec003c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19a4003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12a80015, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12ec001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1374000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7eae800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc02e4000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1774000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7eae800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f6b400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43d3248, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bfc01e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13fc0018, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dbd800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1d98ff15, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x592c00fc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd80000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12e00016, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7da1800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x592c007e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12e00015, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7da1800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11a0000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1264001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1620000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e26000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e32000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12e4001b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e26000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x5924007e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12640017, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e26000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19a4003c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12640018, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e26000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce01325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd013257, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd413258, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc429325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c00fdb, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96800001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c410001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9780f5ca, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400100, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00120d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001219, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001232, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001b6d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d324e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431324d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52ec0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc435324f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4293256, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52ec0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x07740003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04240002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x269c003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e5e4004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f67000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f674002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0b740001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x53740002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef6c011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1ab42010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1ab8c006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x16a8000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26a80800, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2b740000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f7b400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f6b400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf40001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd2c0001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc438000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4180011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a000003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000bec, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000b47, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c001d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313256, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b34060b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b300077, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f37000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13300017, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04340100, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26ec00ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc03a8004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef6c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f3b000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec1325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000c16, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0032, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc410001d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc415325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c418001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c418001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18580037, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x251000ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc421325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x262001ef, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce01325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99800004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d15400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd41325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000168, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1d54001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd41325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12a80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26a80004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7eae800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340028, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x14f00010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380030, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd280200, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd680208, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcda80210, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b400014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b800017, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12a80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26a80004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7eae800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc6930200, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc6970208, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc69b0210, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b000005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00037, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000190, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000032, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000028, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000168, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd900003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd940003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001082, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9000040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9400040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800010de, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x14fc0011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24f800ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33b80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c0fffc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b800007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00037, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000190, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000032, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000028, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001b70, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd88130b8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04100000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04140000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d83c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4093249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1888003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94800020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400074, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000671, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a400009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc419324c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x259c0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1598001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00016, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95800015, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99000003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400036, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x14d80011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24e000ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x321c0002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32200001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9580ffee, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c00014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96000004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00037, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04140001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000c30, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9480000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000074, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95800f29, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000c16, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94800004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000074, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95800f23, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9c00036, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99400002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00037, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000c16, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94800004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000074, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95800f1a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00037, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800036, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001b70, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x041c0003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c01c8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000d61, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4200007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0077, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c418001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9600f502, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a200001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x98c0f500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2aa80008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a000f05, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d3256, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1f30001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x16e4001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9640f4f4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc434000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33740002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b40f4f1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4353254, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x16a80008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1aec003c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12a80015, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12ec001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1374000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7eae800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc02e4000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1774000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7eae800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f6b400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400100, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12780001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2bb80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc00ac005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc00e0002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc8000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28884900, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc0014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ff3, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17fc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc00004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400029, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc424005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400ee1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc41c40a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc41c40c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc41c40d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c414001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24d0007f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x15580010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x255400ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd01c411, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd81c40f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd41c40e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc41c410, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c414001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c418001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04200000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18e80033, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18ec0034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc41c414, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc41c415, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd81c413, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd41c412, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18dc0032, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c030011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c038011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431c417, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc435c416, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96800004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc439c419, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43dc418, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41c000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29dc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf413261, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf013262, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96800004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc13263, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf813264, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18dc0030, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00017, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17fc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d77000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc00015, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9700000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000cd6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51b80020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x53300020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f97801a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f37001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f3b000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc0000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97800002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000cd6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a000018, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28200001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ca7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18dc0031, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc435c40b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9740fffd, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001b70, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4280032, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2aa80008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40d325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800012c2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc438001d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bb81ff0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f8cc00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc1325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc411325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x251001ef, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd01325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001b70, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2aa80008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc438001d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13f4000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc00006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43d3256, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bf0060b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bfc0077, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ff3c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000cf4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43d325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bfc0677, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13fc0017, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04300100, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bb81fe8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f73400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc032800b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fb7800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ff3c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ffbc00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc1325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000c16, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d42011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17fc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d001e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24cc007f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cd4c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc00004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400029, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96800e6c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c414001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x50580020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d59401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1400072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8140072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x596001fc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12200009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ce0c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c418001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x505c0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d9d801a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c41c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x50600020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7de1c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c420001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc0001b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd140001d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd180001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1c00020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04300000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8240010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e5e800c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc00015, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b000024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x122c0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06ec0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0aec0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000d1f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8240010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x566c0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2aa80008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce413261, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec13262, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001b70, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340032, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2b740008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40d325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96800005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x566c0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce413261, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec13262, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800012c2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc438001d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bb81fe8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f8cc00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc1325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc411325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x251001ef, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd01325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001b70, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc438001d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2aa80008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13f4000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc00006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43d3256, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bf0060b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bfc0077, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ff3c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000d57, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43d325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bfc0677, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13fc0017, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04300100, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bb81fe8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f73400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0328009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fb7800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ff3c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ffbc00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc1325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000c16, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2bfc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253246, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113245, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04143000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd413267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52640020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e51001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4153267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d2d0011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19640057, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19580213, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19600199, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7da6400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e26400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1000025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce400024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04142000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd413267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4153267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99400001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d001e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d40030, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d80034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05280d83, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c420001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c424001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000d8a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000016a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000d95, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000db1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000016a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000d95, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000dbc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11540010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e010001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00187c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d75400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4610000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9580f3d8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc439c040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97800001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000016, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x526c0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18e80058, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e2ec01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd2c00072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc82c0072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x5ae0073a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ea2800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9940000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd2c00025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9580f3c6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc3a0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0bb80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd2c00025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80fffb, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980fff5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc02a0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2aa80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x16200002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce01c405, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd441c406, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9580f3b1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc439c409, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97800001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc424000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32640002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a40000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11540010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29540002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4610000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9580f3a5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc439c040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97800001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4400078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000168, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400029, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c00da7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x50500020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cd0c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd0c00072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8280072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x5aac007e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12d80017, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c41c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d9d800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56a00020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2620ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7da1800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51980020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e82400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e58c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19d4003d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28182002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99400030, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00104f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc430000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340035, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8140023, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4180081, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13300005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc011000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4240004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11a00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c908009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12640004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d614011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4100026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ca4800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d1a0002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cb0800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3e280008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x20880188, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x54ec0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cb4800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4300027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04380008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1400025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf000024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x20240090, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ca48001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc800026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28240004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a640001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a40fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a800005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32280000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a800002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c018001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000016, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf80003a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd901a2a4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001037, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc421326c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1624001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a40fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd841325f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800033, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27fc0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c0fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000039, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd0c00038, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0022, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc429325f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26ac0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac0fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26ac0002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001b70, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc430001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800033, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13f4000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b301ff0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2b300300, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2330003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f37000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9680000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27fc0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c0fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400039, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd0c00038, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0022, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000c16, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c0001a2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001b70, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc80003b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24b00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1330000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18ac0024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2b304000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18a800e5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1d980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12a80008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7da9800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1910003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd840003d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c410001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4400078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51100020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01326c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cd0c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc421326c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12a80014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2220003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e2a000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce01326c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800033, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27fc0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c0fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000039, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd0c00038, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0022, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001190, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18dc003d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x041c0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c01c8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000d61, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d40030, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d001e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18fc0034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24e8000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06a80e71, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c418001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c41c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000edd, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000e91, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000e91, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ea1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000eaa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000e7c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000e7f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000e7f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000e87, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000e8f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000016a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51dc0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d9e001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ee6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc420000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2a200008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4213262, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253261, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52200020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e26001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ee6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc420000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2a200008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4213264, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253263, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52200020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e26001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ee6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc820001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ee6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18e82005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51e00020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2aa80000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7da1801a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1800072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8180072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x59a001fc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12200009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ea2800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce80001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd180001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8200011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ee6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x15980002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd81c400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc421c401, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95400041, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc425c401, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52640020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e26001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ee6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31ac2580, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31ac260c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac0000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31ac0800, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac0000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31ac0828, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac0000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31ac2440, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31ac2390, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31ac0093, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31ac31dc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31ac31e6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c00004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ede, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x39ac7c06, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3db07c00, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ebc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x39acc337, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3db0c330, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ebc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x39acc335, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3db0c336, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ebc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x39ac9002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3db09001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ebc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x39ac9012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3db09011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ebc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x39acec70, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3db0ec6f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ebc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc5a10000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95400005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05980001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc5a50000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52640020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e26001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05280eea, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c418001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c41c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ef1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000016a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000efe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000f11, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000f2e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000efe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000f1f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce190000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95400005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05980001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56200020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce190000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c0f26f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc439c040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97800001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51ec0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18e80058, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7daec01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd2c00072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc82c0072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x5af8073a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7eba800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd2c00025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95400003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56240020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c0f25c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc02a0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2aa80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x15980002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd81c405, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce01c406, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95400003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56240020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce41c406, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c0f24e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc439c409, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97800001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc424000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32640002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a40f247, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce190000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95400004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05980001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56200020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce190000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c0f240, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc439c040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97800001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31ac2580, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31ac260c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac0000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31ac0800, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac0000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31ac0828, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac0000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31ac2440, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31ac2390, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31ac0093, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31ac31dc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31ac31e6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c00004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ef2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x39ac7c06, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3db07c00, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000f40, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x39acc337, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3db0c330, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000f40, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x39acc335, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3db0c336, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000f40, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x39acec70, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3db0ec6f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000f40, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x39ac9002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3db09002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000f40, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x39ac9012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3db09012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000f40, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ef1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x98c0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c410001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c414001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c418001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c41c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c43c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc434000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2b740008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2b780001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8c1325e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf80001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c034001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c038001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18e0007d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32240003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a400006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32240000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a400004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd01c080, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd41c081, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000f88, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51640020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e52401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd2400072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8280072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce81c080, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56ac0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26f0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01c081, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1af000fc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1334000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24e02000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f63400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18e00074, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32240003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a400006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32240000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a400004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd81c082, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc1c083, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000f9d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51e40020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e5a401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd2400072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8280072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce81c082, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56ac0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26f0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01c083, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1af000fc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13380016, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18e00039, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12200019, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fa3800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fb7800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18e0007d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1220001d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fa3800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18e00074, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12200014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fa3800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf81c078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc1c084, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000c16, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18dc003d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x041c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c01c8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000d61, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d001e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31140005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99400003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31140006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95400002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00104f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05280fb7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28140002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000fbe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000fbe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000fc2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000fbe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000fd1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ff2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ff2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24cc003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc1a2a4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c414001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18e80039, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52a8003b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x50580020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24cc003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d59401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1400072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8140072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d69401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41c0017, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd140004b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc1a2a4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc414000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04180001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24cc003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d958004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800035, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc1a2a4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2bfc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43d3249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bfc003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400074, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4100019, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d150005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9500000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c0fffc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4180021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x159c0011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x259800ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31a00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31a40001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e25800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c0fff5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9580fff4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000fef, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc411326f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1d100010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd01326f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000074, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001b70, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04380000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc430000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8140023, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4180081, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13300005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc011000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4240004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33b40003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97400003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0340008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000ffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340035, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11a00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c908009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12640004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d614011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4100026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ca4800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d1a0002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cb0800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x282c2002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x208801a8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3e280008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cb4800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4300027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1400025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf000024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x20240030, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ca48001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc800026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c414001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28340000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x507c0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d7d401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1400072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8140072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x557c0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28342002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32280000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000102f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a800005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32280000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a800002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c018001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1cccfe08, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec0003a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc1a2a4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2bfc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43d3249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bfc003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc00007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x16a80008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c00b33, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd840003c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4200025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7da2400f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7da28002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e1ac002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0aec0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d2ac002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3ef40010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b40f11d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04380030, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf81325e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000c16, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xde410000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdcc10000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdd010000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdd410000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdd810000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xddc10000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xde010000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c024001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8100086, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x5510003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40d3249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18cc003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x98c00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99000011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001075, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9900000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4100081, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4140025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d15800f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d15c002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d520002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a200001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95800002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cde0002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3e20001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a000009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x040c0030, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc1325e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001071, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9c00036, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400029, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c00b01, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04240001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc200000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc1c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc180000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc140000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc100000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc0c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc240000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc0c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000c16, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc240000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc40003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8c00010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4080029, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc80003b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18a800e5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1d980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12a80008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7da9800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18a400e5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12500009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x248c0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c00006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x200c006d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cd0c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc1326c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc421326c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x200c0228, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cd0c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc1326c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc421326c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c002a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc410002b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18881fe8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d4072c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18cc00d1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cd4c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3094000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x38d80000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x311c0003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99400006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x30940007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1620001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9940001d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a000023, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800010c4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9580001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c00019, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00041, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25140001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418002c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9940000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x259c007f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19a00030, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc0001b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400022, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc430000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a000012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400023, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800010cb, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x199c0fe8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc0001b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400023, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc430000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800010cb, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8c00010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000022, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000023, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc430005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000aac, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc434002e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2bfc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2020002c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce01326c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17780001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27740001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x07a810d8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc421326c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000168, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000aa7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000bfc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800012e9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000104c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc400040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4180032, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x200c007d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc1325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc411325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28240007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xde430000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4400078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001190, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc80003b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24b00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1330000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18a800e5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1d980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12a80008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7da9800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40d3249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18cc003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x98c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd840003d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2b304000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01326c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431326c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c410001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c414001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x192400fd, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x50580020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d59401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c41c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06681110, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c420001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc400078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18ac0024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19180070, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19100078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18f40058, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x5978073a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f7b400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001117, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001118, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001122, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000112d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001130, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001133, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000016a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000117b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24ec0f00, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32ec0600, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4300006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1400025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000117b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24ec0f00, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32ec0600, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4300006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1400025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000117b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc81c001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55e00020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001122, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc81c0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55e00020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001122, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00116b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc02a0200, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e8e8009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x22a8003d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x22a80074, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2774001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13740014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7eb6800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25ecffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55700020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x15f40010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13740002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x275c001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c018001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f41c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x15dc0002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x39e00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25dc0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dc1c01e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05dc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96000004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05e40008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00116e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001168, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dc2001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06200001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05e40008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e62000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a000004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7da58001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00116e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001165, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dc2001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06200001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e1a0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05cc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e0d000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95000007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e02401e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06640001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06640008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05d80008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00116e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001168, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dc2401e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06640001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7da58001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00116e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05e00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7da2000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9600ffe6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17640002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00116e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001190, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4200006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a00ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00116b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc420000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2a200001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce00001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce81c078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec1c080, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c081, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd41c082, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01c083, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12640002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x22640435, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce41c084, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0528117e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x312c0003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001190, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001185, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001182, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001182, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4300012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac0000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc03a0400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x15980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1198001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d81c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc130b7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf8130b5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04240008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41c0049, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19a000e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29a80008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7de2c00c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc421325e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26200010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc415326d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a000006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc420007d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96000004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce40003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800011a3, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d654001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd41326d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c020001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96000005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4100026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4240081, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4140025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800011b6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253279, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc415326d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431326c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2730003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3b380006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97800004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3f38000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b800004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800011b4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04300006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800011b4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0430000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04380002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fb10004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e57000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e578002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d67c002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0be40001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d3a4002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x202c002c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc421325e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04280020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec1326c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26200010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3e640010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96000003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce81325e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4300028, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc434002e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17780001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27740001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x07a811cf, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00feb8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc414005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x954009a7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000168, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000aa7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000bfc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800012e9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000168, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00120d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc1c07c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc41c07d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc41c08c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c410001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc41c079, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd01c07e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c414001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18f0012f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18f40612, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18cc00c1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f73400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cf7400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x39600004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0140004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11600001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18fc003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9740001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400041, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc425c07f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x166c001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800011ee, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1a6c003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c00006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04200002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a200001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a00ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800011e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428002c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96800010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26ac007f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec0001b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1ab00030, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1aac0fe8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc434000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b40ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec0001b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc434000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b40ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001205, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a200001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a00ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc425c07f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x166c001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11600001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac0fffa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001232, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000033, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc438000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27fc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c0fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd841c07f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43dc07f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bfc0078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ffbc00c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c0fffd, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc03a2800, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf81c07c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c07d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c08c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c079, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c07e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04380040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf80001b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc438000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04380060, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf80001b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc438000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04380002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0bb80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43dc07f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17fc001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04380010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc0fffa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801c07f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43dc07f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc03ae000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf81c200, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc03a0800, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf81c07c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c07d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c08c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c079, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c07e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04380040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf80001b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc438000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04380002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0bb80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43dc07f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17fc001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04380010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc0fffa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc03ae000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf81c200, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc03a4000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf81c07c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c07d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c08c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c079, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c07e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04380002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0bb80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43dc07f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17fc001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04380010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc0fffa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x30d00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99000052, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400029, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc424005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9640090f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c410001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1514001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19180038, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2aa80008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99400030, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x30dc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c0000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d324e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431324d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52ec0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc435324f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4293256, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1ab0c006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52ec0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000127f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d3258, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313257, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52ec0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4353259, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc429325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1ab0c012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x07740001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04240002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26a0003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e624004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f67800f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97800002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04340000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x53740002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef6c011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1ab42010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x16a8000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26a80800, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2b740000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f73400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f6b400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf40001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd2c0001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc438000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4100011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1514001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99400006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c0012e1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04100000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc424005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x964008d7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9800036, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000c16, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c001d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b300677, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11dc000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800012aa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313256, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b34060b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b300077, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f37000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13300017, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04340100, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26ec00ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc03a8002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef6c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7edec00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f3b000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec1325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000c16, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4140032, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc410001d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29540008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40d325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1858003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x251000ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99800007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d0cc00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc1325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc411325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x251001ef, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd01325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000168, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d0006c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d407f0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9900000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04100002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193256, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d324f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2598003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d190004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d5d4001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d52000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a000003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd41324f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800012d8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d514002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd41324f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800012d8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193259, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d958001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dd5c002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd813259, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc1325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc411325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x251001ef, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd01325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1ccc001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc1325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40d325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340028, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x14f00010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380030, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b000004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b40000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b000005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00037, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000190, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000032, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000028, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000168, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd980003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9c0003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001082, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9800040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd9c00040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800010de, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33f80003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97800051, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc80003b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24b00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1330000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18a800e5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1d980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12a80008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7da9800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4353249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b74003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b400002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd840003d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2b304000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01326c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431326c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c434001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b4c00f8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c410001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c414001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x50700020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04e81324, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18ac0024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c41c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x50600020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc400078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x30e40004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a400007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d71401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x596401fc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12640009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b74008d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e76400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2a640000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000016a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000016a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000016a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000016a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000132c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000133b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001344, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000016a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42530b5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1a68003a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2024003a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25980700, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11980014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d19000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd0130b7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce4130b5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001190, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce40001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd140001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4240011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7de6800f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80ffea, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001190, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce40001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd140001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc428000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8240011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7de1c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7de6800f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80ffe0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001190, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00104f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28182002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc430000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340035, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8140023, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4180081, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13300005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4240004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11a00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12640004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d614011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4100026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05980008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ca4800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d1a0002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cb0800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3e280008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cb4800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4300027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1400025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf000024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x20240030, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ca48001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc800026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c434001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b4c00f8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28340000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c414001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x507c0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x30e40004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a400005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d7d401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1400072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8140072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x557c0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28342002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a800005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32280000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a800002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c018001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04380028, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec0003a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf81a2a4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001037, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400029, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c007eb, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x50500020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d0d001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1000072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8100072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x591c01fc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11dc0009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45140210, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x595801fc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11980009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29dc0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc0001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd140001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4200011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1624001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400069, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce013249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1a307fe8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf00000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x23304076, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3254, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253256, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18cc00e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x10cc0015, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x4514020c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd140001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4200011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce013248, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1a2001e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12200014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2a204001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1a64003c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1264001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11dc0009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x15dc000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dcdc00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e5dc00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00100, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf00000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf00000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001427, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04340022, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x07740001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04300010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdf430000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c434001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4412e01, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0434001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdf430000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4400078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdf030000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4412e40, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc41c030, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc41c031, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x248dfffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc12e00, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc812e00, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c434001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c434001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00142b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45140248, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd140001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8200011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce013257, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56200020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce013258, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0434000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdb000024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1400025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45540008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd140001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9980ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8200011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce013259, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56200020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0337fff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f220009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce01325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55300020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d01c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c01d0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000d61, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06ec0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f01c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000d61, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x041c0002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c01c8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c000d61, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000aa7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400029, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x50500020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001427, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cd0c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4200007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd0c00072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8240072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd240001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c414001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19682011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x5a6c01fc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12ec0009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7eeac00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2aec0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec0001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc430000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4180011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c438001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99800007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdf830000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfa0000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00142b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4400078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001b70, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00142b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001b70, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4380007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17b80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d40038, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c410001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b800004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400029, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc414005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9540073d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18c80066, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c414001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x30880001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c418001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94800008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00187c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd910000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d410001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x043c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c41c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c420001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04240001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06200001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x4220000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a640001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc000078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a40fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24e80007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24ec0010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac00006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc5310000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001465, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51540020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d15001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1000072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc82c0072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd2c0001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18f02011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x5aec01fc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12ec0009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2aec0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec0001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4300011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96800012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12a80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0aa80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06a8146a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f1f0009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f1b400f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001478, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f1b400e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001478, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f1b400c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000147a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f1b400d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000147a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f1b400f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000147a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f1b400e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000147a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f334002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97400014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000147b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b400012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b800005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc0001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e024001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x043c0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000144a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0032, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc438001d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43d325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bb81ff0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fbfc00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc1325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc411325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x251001ef, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd01325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001b70, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94800007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00187c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42c0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd910000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b800003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001b70, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0032, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40d325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800012c2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc438001d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13f4000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc00006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43d3256, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bf0060b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bfc0077, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ff3c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800014a9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43d325a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bfc0677, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04300100, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bb81ff0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f73400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0328007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fb7800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13fc0017, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ff3c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ffbc00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc1325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc03a0002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf8130b5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000c16, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x043c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc414000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29540008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193246, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3245, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51980020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dd9c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45dc0390, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04183000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd813267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b380057, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b340213, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b300199, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f7b400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f73400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1c00025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc800026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c420001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c424001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c428001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c42c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c430001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c434001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c438001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04182000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd813267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd840004f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1a0800fd, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x109c000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dd9c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc13265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2620ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce080228, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9880000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce480250, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce880258, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080230, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080238, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080240, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080248, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080268, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080270, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080280, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800004f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c0ec75, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x040c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x041c0010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26180001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09dc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x16200001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95800002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04cc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c0fffb, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc80230, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080238, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080240, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080248, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x040c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce480250, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce880258, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52a80020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e6a401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x041c0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x66580001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09dc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56640001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95800002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04cc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c0fffb, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc80260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080268, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080270, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080280, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x040c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec80288, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf080290, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec80298, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf0802a0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x040c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x041c0010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf4802a8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27580001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09dc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17740001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95800002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04cc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c0fffb, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc802b0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd80802b8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x178c000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27b8003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cf8c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf8802c0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc802c8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf8802d0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf8802d8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800004f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43d3265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bc800ea, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c418001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25b8ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4930240, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc48f0238, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04cc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24cc000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cd2800c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc5230309, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2620ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e3a400c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a400004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2510000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001539, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd08034b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4400078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000168, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc48f0230, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4930240, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x98c00004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd880353, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00163f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc49b0353, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4930238, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc48f0228, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2510000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cd14005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25540001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99400004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2510000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000154f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc48f0230, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c41c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd080238, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd08034b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x08cc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2598ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3d200008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc80230, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd900309, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8100319, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04340801, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2198003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd910ce7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4190ce6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d918005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25980001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9580fffd, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d918004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd810ce6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a000003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdd1054f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000156e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x090c0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdcd050e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x040c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x110c0014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc4001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc41230a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc41230b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc41230c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc41230d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc480329, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc48032a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc4802e0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000055, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc48f02e0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24d8003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09940001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x44100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9580002c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95400005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09540001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x69100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000157f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24cc003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4970290, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc49b0288, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51540020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d59401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc49b02a0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc49f0298, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51980020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d9d801a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x041c0040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04200000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dcdc002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d924019, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d26400c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09dc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06200001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c0fffa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc48f0230, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4930240, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00163f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001579, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d010021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d914019, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4930238, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55580020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd480298, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd8802a0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x10d40010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12180016, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc51f0309, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d95800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d62000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dd9c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdd00309, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce113320, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc48f02e0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc49b02b0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18dc01e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dd9400e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc48f0230, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4930240, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c0001d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95400003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00163f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800015aa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc48f0238, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4a302b8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12240004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e5e400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4ab02a8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04100000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce4c0319, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d9d8002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ea14005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25540001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99400004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06200001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2620000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800015bc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09dc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04240001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e624004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06200001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d25000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2620000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c0fff4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd0d3330, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce0802b8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd8802b0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4ab02e0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1aa807f0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc48f02d0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc49702d8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc49b02c8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc49f02c0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96800028, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d4e000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9600000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d964002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e6a000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96000003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d694001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800015e9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cde4002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e6a000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96000008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7de94001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800015e9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cd64002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e6a000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96000003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d694001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800015e9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc48f0230, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4930240, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00163f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800015cd, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4930238, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d698002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd4802d8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x129c0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc50f0319, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11a0000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11140001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e1e000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1198000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd953300, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e0e000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12a8000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce953301, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce100319, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4b70280, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4b30278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f73800a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x536c0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9780eb68, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001608, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080280, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x043c0003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001609, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x043c0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x30b40000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b400011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4b70258, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4b30250, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x53780020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fb3801a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7faf8019, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04300020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04280000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x67b40001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0b300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x57b80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97400002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06a80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00fffb, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4bb0260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fab8001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf880260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04300020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04280000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x66f40001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0b300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56ec0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97400005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001628, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4353247, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f7f4009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b40fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06a80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00fff7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x269c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11dc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29dc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26a00018, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12200003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7de1c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26a00060, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06200020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x16200001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7de1c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x269c0018, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26a00007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26a40060, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11dc0006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12200006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x16640001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29dc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7de1c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7de5c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4b70228, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04cc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2510000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc80230, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f514005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25540001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99400004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2510000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001644, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4b30248, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd080240, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f130005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001688, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00120d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001219, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001232, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04340801, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f130004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01051e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42d051f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ed2c005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26ec0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96c0fffd, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01051f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000055, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc5170309, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x195c07f0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x196007f6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04340000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09dc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04340001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09dc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x53740001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x6b740001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001665, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4a702a0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4ab0298, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52640020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e6a401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f634014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e76401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4300004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56680020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8113320, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce480298, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce8802a0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc5170319, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4b702b0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x255c000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f5f4001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8113330, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf4802b0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11340001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x195c07e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x196007ee, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8353300, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e1e4001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8353301, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce4802d0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8100309, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8100319, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf000008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4970258, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc48f0250, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51540020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cd4c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4af0280, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4b30278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52ec0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04140020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04280000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x64d80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09540001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x54cc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95800060, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001628, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193247, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25980001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9580005c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dc24001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3248, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25dc000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dd2000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96000057, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3255, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc435324f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7df5c00c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c00004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25980040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9580fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc439325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bb0003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000049, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bb000e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33380003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b800046, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33300002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9700000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4393260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bb000e4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33300004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800016f1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc033ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2f3000ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc439325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f3b0009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc439325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27b800ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8c00033, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4300009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9700fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1a7003e6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27380003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13b80004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13300003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fb38001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1a7000e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fb38001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fb38001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x07b80002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1a700064, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33300002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17b00005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x07300003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf012082, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01203f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01203f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0b300003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800016df, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17b00005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf012082, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01203f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01203f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13300005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fb30002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4392083, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fb38005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27b80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffdf, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8c00034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc439325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27b000ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00ffca, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd841325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2030007b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800016f2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd841325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f2b0014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06a80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9940ff9c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001608, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080280, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd840004f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc414000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29540008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43d3265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bc800ea, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd80802e9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18fc0064, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc00042, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193246, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3245, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51980020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dd9801a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x45980400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x043c3000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc13267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43d3267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b380057, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b340213, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b300199, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f7b400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f73400a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x14f4001d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4bf02e9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc0001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c410001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x192807fa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4bf0258, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4a70250, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x53fc0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e7e401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04300000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x667c0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56640001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06ec0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c0fffd, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x07300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0aec0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7eebc00c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06ec0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c0fff8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0b300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x43300007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x53300002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7db30011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd3000025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc03ec005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2bfca200, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x192807fa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc01f007f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d1d0009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2110007d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001628, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x203c003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc13256, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c0017f5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd013254, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18fc01e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc13248, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00185b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8413247, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0b740001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b40ffd5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800004f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4bf02e9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c0ea24, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x14d4001d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4930260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d52400e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc49f0258, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4a30250, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51dc0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7de1801a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400017, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d534002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4af0270, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dae4005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26640001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32e0001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a400006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06ec0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec80270, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000174f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0b740001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00178a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b40fff3, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4af0280, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4b30278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52ec0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001608, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080280, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4ab0268, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7daa4005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26640001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32a0001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a400005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06a80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24280000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001765, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c410001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc01f007f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09540001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d1d0009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2110007d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001628, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8013256, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c0017f2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd013254, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4113248, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x15100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4b3034b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f13000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf013248, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4930260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001855, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32a4001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8413247, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800004f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06a80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24280000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd080260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce880268, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9940ffc0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ec28001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001628, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32e0001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253247, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26640001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9640005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4293265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253255, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431324f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e72400c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26a80040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a400002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9680fff7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc429325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1aa4003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400049, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1aa400e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32680003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a800046, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32640002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9640000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4293260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1aa400e4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32640004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc425325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26640010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a40fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800017e2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc027ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2e6400ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc429325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e6a4009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce41325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc429325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26a800ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8c00033, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4240009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26640008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9640fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19e403e6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26680003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12a80004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26640003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12640003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ea68001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19e400e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ea68001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12640001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ea68001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06a80002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19e40064, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x32640002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x16a40005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06640003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce412082, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01203f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01203f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a640003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800017d0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x16a40005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce412082, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01203f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01203f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12640005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ea64002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4292083, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ea68005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26a80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a80ffdf, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8c00034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc425325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26640010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a40fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc429325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26a400ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a40ffca, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd841325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2024007b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce41325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800017e3, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd841325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4a70280, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4ab0278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52640020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e6a401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04280001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7eae8014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e6a401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56680020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce480278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce880280, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06ec0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x042c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec80270, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c438001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c420001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800017fe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4bf02e9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc00006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c438001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c420001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800017fe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43b02eb, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42302ec, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf813245, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce013246, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52200020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fa3801a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x47b8020c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x15e00008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1220000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2a206032, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x513c001e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e3e001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4bf02e9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc00005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2bfc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000180f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b3c0077, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b300199, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ff3000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1330000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2b300032, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x043c3000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc13267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43d3267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd200000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4200007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd3800002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400018, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x043c2000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc13267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000018, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dc30001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc1e0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04380032, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf80000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001427, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc413248, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43d3269, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27fc000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33fc0003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c00011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x043c001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdfc30000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4413249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c43c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c43c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x043c0024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0bfc0021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdfc30000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd441326a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x173c0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b300303, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f3f0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x043c0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ff3c004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc13084, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001842, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x043c0024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdfc30000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4413249, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c43c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x23fc003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc1326d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0bb80026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdf830000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd441326e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c438001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c438001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4393265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1fb8ffc6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xddc30000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf813265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a000003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc0000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001852, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc0000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c00142b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c41c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c420001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc13252, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce013253, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001628, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001878, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc49f02e9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c00018, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c41c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c420001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc13252, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce013253, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2bfc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x043c3000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc13267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43d3267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41c0012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2bfc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x043c2000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc13267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001628, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001878, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41f02ed, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42302ee, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc13252, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce013253, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04200001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e2a0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce013084, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28340001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x313c0bcc, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc00010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x393c051f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc00004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3d3c050e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc0000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c0000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x393c0560, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc00004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3d3c054f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc00007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c00007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x393c1538, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc00005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3d3c1537, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2b740800, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43d3265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bc800ea, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18e8007c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c42c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06a8189a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000189e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800018c5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800018f2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000016a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c414001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d0007e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x50580020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09200001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d59401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1400072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc8140072, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09240002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c418001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c41c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99000011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4340004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc42130b5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1a24002c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a40fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2020002c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc418000d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1198001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x10cc0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x14cc0004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7cd8c00a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc130b7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce0130b5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000168, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd1400025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x5978073a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2bb80002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf800024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd800026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9600e8a8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4300012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9640e8a5, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800018a9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04140000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc55b0309, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3d5c0010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05540001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2598ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09780001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dad800c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c0ffd2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9580fff9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4970258, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4930250, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51540020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d15001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04140020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04280000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x442c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x65180001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09540001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9580000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001628, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3248, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f2b0014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25dc000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7df9c00c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8c13260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd901325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06a80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9940fff1, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04140020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04280000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x66d80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09540001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x56ec0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95800005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001628, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc421325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26240007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a40fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06a80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9940fff7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000189e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04140020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04280000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09540001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001628, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3254, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc023007f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19e4003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7de1c009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dee000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96000007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8c13260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd901325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc421325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x261c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99c0fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000189e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06a80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9940fff0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000189e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28cc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43d3265, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bc800ea, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18e00064, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06281911, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x14f4001d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24cc0003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x86800000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001915, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x800019af, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001a2b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8000016a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc48032b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc480333, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc48033b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc480343, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x98800011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4213246, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253245, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52200020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e26401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x46640400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04203000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce013267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4213267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b3c0057, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b200213, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b300199, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e3e000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e32000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4970258, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4930250, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51540020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d15001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4af0280, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4b30278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52ec0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04180000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04140020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04280000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f438001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001628, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3247, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25dc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00068, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4213254, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1a1c003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00065, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc01f007f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e1e0009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97800062, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0bb80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x43bc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fcbc001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc7df032b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e1fc00c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c0fffa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x043c0101, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x043c0102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc439325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bb0003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000049, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bb000e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33380003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b800046, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33300002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4393260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bb000e4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33300004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001994, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001628, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc033ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2f3000ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc439325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f3b0009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc439325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27b800ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8c00033, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4300009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9700fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19f003e6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27380003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13b80004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13300003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fb38001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19f000e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fb38001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fb38001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x07b80002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19f00064, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33300002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17b00005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x07300003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf012082, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01203f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01203f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0b300003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001982, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17b00005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf012082, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01203f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01203f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13300005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fb30002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4392083, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fb38005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27b80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffdf, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8c00034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc439325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27b000ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00ffcb, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc1325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2030007b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001995, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc1325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f2b0014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x98800009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x41bc0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x53fc0002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e7fc011, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd3c00025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0012, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9bc0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x653c0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dbd8001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06a80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09540001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9940ff8f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2bfc0008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x043c2000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcfc13267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080280, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000168, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c410001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04140000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc55b0309, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x3d5c0010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2598ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x05540001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d91800c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4400078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000168, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9580fff8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09780001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4970258, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4930250, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51540020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d15001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4af0280, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4b30278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52ec0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04140020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04280000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x65180001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09540001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9580005d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001628, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253247, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26640001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04200101, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400058, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dc24001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41d3248, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25dc000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7df9c00c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95c00053, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c00002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04200102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e41c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc425325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1a70003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000049, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1a7000e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33240003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a400046, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33300002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9700000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1a7000e4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33300004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001a21, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc033ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2f3000ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc425325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f270009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc425325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x266400ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a40fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8c00033, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4300009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9700fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19f003e6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27240003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12640004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13300003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e724001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19f000e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e724001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e724001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06640002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19f00064, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33300002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x16700005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x07300003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf012082, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01203f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01203f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0b300003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001a0f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x16700005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf012082, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01203f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01203f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13300005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e730002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4252083, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e724005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x26640001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a40ffdf, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8c00034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc425325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x267000ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00ffca, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce01325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2030007b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001a22, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce01325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f2b0014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06a80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9940ff9f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4400078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080280, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000168, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001a31, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4400078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8080280, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4213246, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4253245, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52200020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e26401a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x46640400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04203000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce013267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4213267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b180057, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b200213, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1b300199, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e1a000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e32000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce000024, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4970258, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4930250, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x51540020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d15001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4af0280, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4b30278, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x52ec0020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04140020, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04280000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x65180001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95800060, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x8c001628, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4193247, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x25980001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04200101, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c00005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x30f00005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04200005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04200102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95800056, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc439325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bb0003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000049, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bb000e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33380003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b800046, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33300002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9700000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4393260, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bb000e4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33300004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001aa2, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc033ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2f3000ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc439325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f3b0009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf01325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc439325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27b800ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8c00033, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4300009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9700fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19f003e6, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27380003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13b80004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13300003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fb38001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19f000e8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fb38001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fb38001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x07b80002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x19f00064, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33300002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17b00005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x07300003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf012082, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01203f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01203f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0b300003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001a90, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x17b00005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf012082, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01203f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01203f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x13300005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fb30002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4392083, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7fb38005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27b80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b80ffdf, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8c00034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc00013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc431325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27300010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc439325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27b000ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b00ffca, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce01325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2030007b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf00325b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001aa3, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce01325d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04300001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7f2b0014, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ef2c01a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc49b02e9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99800005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd2400025, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x4664001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000026, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400027, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x06a80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09540001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55100001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9940ff9c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc49b02e9, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99800008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc430000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2b300008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf000013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04302000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcf013267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc4313267, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x244c00ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc4c0200, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc44f0200, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc410000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc414000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d158010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x059cc000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccdd0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0037, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc000049, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c003a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24d00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9500e69a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d0003b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d40021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99400006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd840004a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c003c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x14cc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c00028, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000033, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc438000b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0009, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x27fc0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c0fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd841c07f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43dc07f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1bfc0078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7ffbc00c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x97c0fffd, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x99000004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0120840, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x282c0040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001ae8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0121841, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x282c001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd01c07c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c07d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c08c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c079, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c07e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04200004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcec0001b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a200001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9a00ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc425c07f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x166c001f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04200004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9ac0fffb, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc434000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9b40ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801c07f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc425c07f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce400078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8000034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9940e66b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800004a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0036, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24d00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9900fffe, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18cc0021, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc00047, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc000046, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0039, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c003d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x98c0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c40c001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24d003ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d47fea, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x18d87ff4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd00004c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd40004e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd80004d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd41c405, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc02a0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2aa80001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd01c406, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c406, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c406, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x98c0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc414000e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x29540008, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x295c0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8c1325e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcdc0001a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11980002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x4110000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0160800, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7d15000a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0164010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd41c078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c080, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c081, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd81c082, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc01c083, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd01c084, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x98c0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400048, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c003b, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x94c0ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000c16, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801c40a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd901c40d, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801c410, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801c40e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd801c40f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc40c0040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04140001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x09540001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9940ffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04140096, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8400013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc1c400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc411c401, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9500fffa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc424003e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04d00001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x11100002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd01c40c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0180034, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd81c411, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd841c414, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0a540001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcd41c412, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x2468000f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc419c416, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x41980003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc41c003f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7dda0001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x12200002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x10cc0002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xccc1c40c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd901c411, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce41c412, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd8800013, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xce292e40, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc412e01, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc412e02, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc412e03, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc412e00, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000aa7, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc43c0007, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc120000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x31144000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x95400005, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xdc030000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd800002a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xcc3c000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001b70, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x33f80003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd4400078, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x9780e601, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x188cfff0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x04e40002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001190, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc424005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400006, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x90000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc424005e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x96400003, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7c408001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x88000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80001b74, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000168, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110501, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120206, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130703, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92100400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92110105, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92120602, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x92130307, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xbf810000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000004, mmCP_DFY_CNTL },
+ { PwrCmdWrite, 0x000000b4, mmCP_DFY_ADDR_HI },
+ { PwrCmdWrite, 0x54106500, mmCP_DFY_ADDR_LO },
+ { PwrCmdWrite, 0x7e000200, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e020204, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc00a0505, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xbf8c007f, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xb8900904, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xb8911a04, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xb8920304, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xb8930b44, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x921c0d0c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x921c1c13, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x921d0c12, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x811c1d1c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x811c111c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x921cff1c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000400, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x921dff10, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000100, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x81181d1c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e040218, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0701000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0701000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0701000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0701000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0701000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0701000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050102, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xe0501000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80050302, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xbf810000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000004, mmCP_DFY_CNTL },
+ { PwrCmdWrite, 0x000000b4, mmCP_DFY_ADDR_HI },
+ { PwrCmdWrite, 0x54106900, mmCP_DFY_ADDR_LO },
+ { PwrCmdWrite, 0x7e080200, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x7e100204, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xbefc00ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00010000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x24200087, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x262200ff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x000001f0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x20222282, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x28182111, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd81a0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000040c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd81a0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000080c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd81a0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000040c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd81a0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000080c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd81a0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000040c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd81a0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000080c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd81a0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000040c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd81a0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000080c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd81a0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000040c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd81a0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000080c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd81a0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000040c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd81a0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000080c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd81a0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000040c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd81a0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000080c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd81a0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000040c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd81a0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000080c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd81a0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000040c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd81a0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000080c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd81a0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000040c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd81a0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000080c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xd86c0000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x1100000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xbf810000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x80000004, mmCP_DFY_CNTL },
+ { PwrCmdWrite, 0x000000b4, mmCP_DFY_ADDR_HI },
+ { PwrCmdWrite, 0x54116f00, mmCP_DFY_ADDR_LO },
+ { PwrCmdWrite, 0xc0310800, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xb4540fe8, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000041, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000000c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x07808000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xffffffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xffffffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xffffffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xffffffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xaaaaaaaa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xaaaaaaaa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xaaaaaaaa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xaaaaaaaa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55555555, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55555555, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55555555, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55555555, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x540fee40, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x000000b4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x54116f00, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x000000b4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00005301, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xb4540fef, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x540fee20, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x000000b4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x08000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0310800, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xb454105e, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x000000c0, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x07808000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xffffffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xffffffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xffffffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xffffffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xaaaaaaaa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xaaaaaaaa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xaaaaaaaa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xaaaaaaaa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55555555, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55555555, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55555555, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55555555, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x540fee40, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x000000b4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x54117300, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x000000b4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00005301, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xb4540fef, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x540fee20, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x000000b4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x08000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0310800, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xb4541065, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000500, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000001c, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x07808000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xffffffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xffffffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xffffffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xffffffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xaaaaaaaa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xaaaaaaaa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xaaaaaaaa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xaaaaaaaa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55555555, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55555555, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55555555, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55555555, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x540fee40, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x000000b4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x54117700, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x000000b4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00005301, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xb4540fef, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x540fee20, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x000000b4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x08000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xc0310800, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000040, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xb4541069, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000444, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x0000008a, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x07808000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xffffffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xffffffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xffffffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xffffffff, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000002, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xaaaaaaaa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xaaaaaaaa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xaaaaaaaa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xaaaaaaaa, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55555555, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55555555, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55555555, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x55555555, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x540fee40, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x000000b4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000010, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000001, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000004, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x54117b00, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x000000b4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00005301, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0xb4540fef, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x540fee20, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x000000b4, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x08000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_DFY_DATA_0 },
+ { PwrCmdWrite, 0x00000000, mmCP_MEC_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_MEC_CNTL },
+ { PwrCmdWrite, 0x00000004, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x54116f00, mmCP_MQD_BASE_ADDR },
+ { PwrCmdWrite, 0x000000b4, mmCP_MQD_BASE_ADDR_HI },
+ { PwrCmdWrite, 0xb4540fef, mmCP_HQD_PQ_BASE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_BASE_HI },
+ { PwrCmdWrite, 0x540fee20, mmCP_HQD_PQ_WPTR_POLL_ADDR },
+ { PwrCmdWrite, 0x000000b4, mmCP_HQD_PQ_WPTR_POLL_ADDR_HI },
+ { PwrCmdWrite, 0x00005301, mmCP_HQD_PERSISTENT_STATE },
+ { PwrCmdWrite, 0x00010000, mmCP_HQD_VMID },
+ { PwrCmdWrite, 0xc8318509, mmCP_HQD_PQ_CONTROL },
+ { PwrCmdWrite, 0x00000005, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x54117300, mmCP_MQD_BASE_ADDR },
+ { PwrCmdWrite, 0x000000b4, mmCP_MQD_BASE_ADDR_HI },
+ { PwrCmdWrite, 0xb4540fef, mmCP_HQD_PQ_BASE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_BASE_HI },
+ { PwrCmdWrite, 0x540fee20, mmCP_HQD_PQ_WPTR_POLL_ADDR },
+ { PwrCmdWrite, 0x000000b4, mmCP_HQD_PQ_WPTR_POLL_ADDR_HI },
+ { PwrCmdWrite, 0x00005301, mmCP_HQD_PERSISTENT_STATE },
+ { PwrCmdWrite, 0x00010000, mmCP_HQD_VMID },
+ { PwrCmdWrite, 0xc8318509, mmCP_HQD_PQ_CONTROL },
+ { PwrCmdWrite, 0x00000006, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x54117700, mmCP_MQD_BASE_ADDR },
+ { PwrCmdWrite, 0x000000b4, mmCP_MQD_BASE_ADDR_HI },
+ { PwrCmdWrite, 0xb4540fef, mmCP_HQD_PQ_BASE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_BASE_HI },
+ { PwrCmdWrite, 0x540fee20, mmCP_HQD_PQ_WPTR_POLL_ADDR },
+ { PwrCmdWrite, 0x000000b4, mmCP_HQD_PQ_WPTR_POLL_ADDR_HI },
+ { PwrCmdWrite, 0x00005301, mmCP_HQD_PERSISTENT_STATE },
+ { PwrCmdWrite, 0x00010000, mmCP_HQD_VMID },
+ { PwrCmdWrite, 0xc8318509, mmCP_HQD_PQ_CONTROL },
+ { PwrCmdWrite, 0x00000007, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x54117b00, mmCP_MQD_BASE_ADDR },
+ { PwrCmdWrite, 0x000000b4, mmCP_MQD_BASE_ADDR_HI },
+ { PwrCmdWrite, 0xb4540fef, mmCP_HQD_PQ_BASE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_BASE_HI },
+ { PwrCmdWrite, 0x540fee20, mmCP_HQD_PQ_WPTR_POLL_ADDR },
+ { PwrCmdWrite, 0x000000b4, mmCP_HQD_PQ_WPTR_POLL_ADDR_HI },
+ { PwrCmdWrite, 0x00005301, mmCP_HQD_PERSISTENT_STATE },
+ { PwrCmdWrite, 0x00010000, mmCP_HQD_VMID },
+ { PwrCmdWrite, 0xc8318509, mmCP_HQD_PQ_CONTROL },
+ { PwrCmdWrite, 0x00000004, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000104, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000204, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000304, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000404, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000504, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000604, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000704, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000005, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000105, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000205, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000305, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000405, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000505, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000605, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000705, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000006, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000106, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000206, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000306, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000406, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000506, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000606, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000706, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000007, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000107, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000207, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000307, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000407, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000507, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000607, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000707, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000008, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000108, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000208, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000308, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000408, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000508, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000608, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000708, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000009, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000109, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000209, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000309, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000409, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000509, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000609, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000709, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_RPTR },
+ { PwrCmdWrite, 0x00000000, mmCP_HQD_PQ_WPTR },
+ { PwrCmdWrite, 0x00000001, mmCP_HQD_ACTIVE },
+ { PwrCmdWrite, 0x00000004, mmSRBM_GFX_CNTL },
+ { PwrCmdWrite, 0x01010101, mmCP_PQ_WPTR_POLL_CNTL1 },
+ { PwrCmdWrite, 0x00000000, mmGRBM_STATUS },
+ { PwrCmdWrite, 0x00000000, mmGRBM_STATUS },
+ { PwrCmdWrite, 0x00000000, mmGRBM_STATUS },
+ { PwrCmdEnd, 0x00000000, 0x00000000 },
+};
+
+
+#endif
diff --git a/drivers/gpu/drm/amd/powerplay/inc/smu74.h b/drivers/gpu/drm/amd/powerplay/inc/smu74.h
new file mode 100644
index 0000000..1a12d85
--- /dev/null
+++ b/drivers/gpu/drm/amd/powerplay/inc/smu74.h
@@ -0,0 +1,774 @@
+/*
+ * Copyright 2014 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+
+#ifndef SMU74_H
+#define SMU74_H
+
+#pragma pack(push, 1)
+
+#define SMU__DGPU_ONLY
+
+#define SMU__NUM_SCLK_DPM_STATE 8
+#define SMU__NUM_MCLK_DPM_LEVELS 4
+#define SMU__NUM_LCLK_DPM_LEVELS 8
+#define SMU__NUM_PCIE_DPM_LEVELS 8
+
+enum SID_OPTION {
+ SID_OPTION_HI,
+ SID_OPTION_LO,
+ SID_OPTION_COUNT
+};
+
+enum Poly3rdOrderCoeff {
+ LEAKAGE_TEMPERATURE_SCALAR,
+ LEAKAGE_VOLTAGE_SCALAR,
+ DYNAMIC_VOLTAGE_SCALAR,
+ POLY_3RD_ORDER_COUNT
+};
+
+struct SMU7_Poly3rdOrder_Data {
+ int32_t a;
+ int32_t b;
+ int32_t c;
+ int32_t d;
+ uint8_t a_shift;
+ uint8_t b_shift;
+ uint8_t c_shift;
+ uint8_t x_shift;
+};
+
+typedef struct SMU7_Poly3rdOrder_Data SMU7_Poly3rdOrder_Data;
+
+struct Power_Calculator_Data {
+ uint16_t NoLoadVoltage;
+ uint16_t LoadVoltage;
+ uint16_t Resistance;
+ uint16_t Temperature;
+ uint16_t BaseLeakage;
+ uint16_t LkgTempScalar;
+ uint16_t LkgVoltScalar;
+ uint16_t LkgAreaScalar;
+ uint16_t LkgPower;
+ uint16_t DynVoltScalar;
+ uint32_t Cac;
+ uint32_t DynPower;
+ uint32_t TotalCurrent;
+ uint32_t TotalPower;
+};
+
+typedef struct Power_Calculator_Data PowerCalculatorData_t;
+
+struct Gc_Cac_Weight_Data {
+ uint8_t index;
+ uint32_t value;
+};
+
+typedef struct Gc_Cac_Weight_Data GcCacWeight_Data;
+
+
+typedef struct {
+ uint32_t high;
+ uint32_t low;
+} data_64_t;
+
+typedef struct {
+ data_64_t high;
+ data_64_t low;
+} data_128_t;
+
+#define SMU7_CONTEXT_ID_SMC 1
+#define SMU7_CONTEXT_ID_VBIOS 2
+
+#define SMU74_MAX_LEVELS_VDDC 16
+#define SMU74_MAX_LEVELS_VDDGFX 16
+#define SMU74_MAX_LEVELS_VDDCI 8
+#define SMU74_MAX_LEVELS_MVDD 4
+
+#define SMU_MAX_SMIO_LEVELS 4
+
+#define SMU74_MAX_LEVELS_GRAPHICS SMU__NUM_SCLK_DPM_STATE /* SCLK + SQ DPM + ULV */
+#define SMU74_MAX_LEVELS_MEMORY SMU__NUM_MCLK_DPM_LEVELS /* MCLK Levels DPM */
+#define SMU74_MAX_LEVELS_GIO SMU__NUM_LCLK_DPM_LEVELS /* LCLK Levels */
+#define SMU74_MAX_LEVELS_LINK SMU__NUM_PCIE_DPM_LEVELS /* PCIe speed and number of lanes */
+#define SMU74_MAX_LEVELS_UVD 8 /* VCLK/DCLK levels for UVD */
+#define SMU74_MAX_LEVELS_VCE 8 /* ECLK levels for VCE */
+#define SMU74_MAX_LEVELS_ACP 8 /* ACLK levels for ACP */
+#define SMU74_MAX_LEVELS_SAMU 8 /* SAMCLK levels for SAMU */
+#define SMU74_MAX_ENTRIES_SMIO 32 /* Number of entries in SMIO table */
+
+#define DPM_NO_LIMIT 0
+#define DPM_NO_UP 1
+#define DPM_GO_DOWN 2
+#define DPM_GO_UP 3
+
+#define SMU7_FIRST_DPM_GRAPHICS_LEVEL 0
+#define SMU7_FIRST_DPM_MEMORY_LEVEL 0
+
+#define GPIO_CLAMP_MODE_VRHOT 1
+#define GPIO_CLAMP_MODE_THERM 2
+#define GPIO_CLAMP_MODE_DC 4
+
+#define SCRATCH_B_TARG_PCIE_INDEX_SHIFT 0
+#define SCRATCH_B_TARG_PCIE_INDEX_MASK (0x7<<SCRATCH_B_TARG_PCIE_INDEX_SHIFT)
+#define SCRATCH_B_CURR_PCIE_INDEX_SHIFT 3
+#define SCRATCH_B_CURR_PCIE_INDEX_MASK (0x7<<SCRATCH_B_CURR_PCIE_INDEX_SHIFT)
+#define SCRATCH_B_TARG_UVD_INDEX_SHIFT 6
+#define SCRATCH_B_TARG_UVD_INDEX_MASK (0x7<<SCRATCH_B_TARG_UVD_INDEX_SHIFT)
+#define SCRATCH_B_CURR_UVD_INDEX_SHIFT 9
+#define SCRATCH_B_CURR_UVD_INDEX_MASK (0x7<<SCRATCH_B_CURR_UVD_INDEX_SHIFT)
+#define SCRATCH_B_TARG_VCE_INDEX_SHIFT 12
+#define SCRATCH_B_TARG_VCE_INDEX_MASK (0x7<<SCRATCH_B_TARG_VCE_INDEX_SHIFT)
+#define SCRATCH_B_CURR_VCE_INDEX_SHIFT 15
+#define SCRATCH_B_CURR_VCE_INDEX_MASK (0x7<<SCRATCH_B_CURR_VCE_INDEX_SHIFT)
+#define SCRATCH_B_TARG_ACP_INDEX_SHIFT 18
+#define SCRATCH_B_TARG_ACP_INDEX_MASK (0x7<<SCRATCH_B_TARG_ACP_INDEX_SHIFT)
+#define SCRATCH_B_CURR_ACP_INDEX_SHIFT 21
+#define SCRATCH_B_CURR_ACP_INDEX_MASK (0x7<<SCRATCH_B_CURR_ACP_INDEX_SHIFT)
+#define SCRATCH_B_TARG_SAMU_INDEX_SHIFT 24
+#define SCRATCH_B_TARG_SAMU_INDEX_MASK (0x7<<SCRATCH_B_TARG_SAMU_INDEX_SHIFT)
+#define SCRATCH_B_CURR_SAMU_INDEX_SHIFT 27
+#define SCRATCH_B_CURR_SAMU_INDEX_MASK (0x7<<SCRATCH_B_CURR_SAMU_INDEX_SHIFT)
+
+/* Virtualization Defines */
+#define CG_XDMA_MASK 0x1
+#define CG_XDMA_SHIFT 0
+#define CG_UVD_MASK 0x2
+#define CG_UVD_SHIFT 1
+#define CG_VCE_MASK 0x4
+#define CG_VCE_SHIFT 2
+#define CG_SAMU_MASK 0x8
+#define CG_SAMU_SHIFT 3
+#define CG_GFX_MASK 0x10
+#define CG_GFX_SHIFT 4
+#define CG_SDMA_MASK 0x20
+#define CG_SDMA_SHIFT 5
+#define CG_HDP_MASK 0x40
+#define CG_HDP_SHIFT 6
+#define CG_MC_MASK 0x80
+#define CG_MC_SHIFT 7
+#define CG_DRM_MASK 0x100
+#define CG_DRM_SHIFT 8
+#define CG_ROM_MASK 0x200
+#define CG_ROM_SHIFT 9
+#define CG_BIF_MASK 0x400
+#define CG_BIF_SHIFT 10
+
+
+#define SMU74_DTE_ITERATIONS 5
+#define SMU74_DTE_SOURCES 3
+#define SMU74_DTE_SINKS 1
+#define SMU74_NUM_CPU_TES 0
+#define SMU74_NUM_GPU_TES 1
+#define SMU74_NUM_NON_TES 2
+#define SMU74_DTE_FAN_SCALAR_MIN 0x100
+#define SMU74_DTE_FAN_SCALAR_MAX 0x166
+#define SMU74_DTE_FAN_TEMP_MAX 93
+#define SMU74_DTE_FAN_TEMP_MIN 83
+
+
+#if defined SMU__FUSION_ONLY
+#define SMU7_DTE_ITERATIONS 5
+#define SMU7_DTE_SOURCES 5
+#define SMU7_DTE_SINKS 3
+#define SMU7_NUM_CPU_TES 2
+#define SMU7_NUM_GPU_TES 1
+#define SMU7_NUM_NON_TES 2
+#endif
+
+struct SMU7_HystController_Data {
+ uint8_t waterfall_up;
+ uint8_t waterfall_down;
+ uint8_t waterfall_limit;
+ uint8_t spare;
+ uint16_t release_cnt;
+ uint16_t release_limit;
+};
+
+typedef struct SMU7_HystController_Data SMU7_HystController_Data;
+
+struct SMU74_PIDController {
+ uint32_t Ki;
+ int32_t LFWindupUpperLim;
+ int32_t LFWindupLowerLim;
+ uint32_t StatePrecision;
+ uint32_t LfPrecision;
+ uint32_t LfOffset;
+ uint32_t MaxState;
+ uint32_t MaxLfFraction;
+ uint32_t StateShift;
+};
+
+typedef struct SMU74_PIDController SMU74_PIDController;
+
+struct SMU7_LocalDpmScoreboard {
+ uint32_t PercentageBusy;
+
+ int32_t PIDError;
+ int32_t PIDIntegral;
+ int32_t PIDOutput;
+
+ uint32_t SigmaDeltaAccum;
+ uint32_t SigmaDeltaOutput;
+ uint32_t SigmaDeltaLevel;
+
+ uint32_t UtilizationSetpoint;
+
+ uint8_t TdpClampMode;
+ uint8_t TdcClampMode;
+ uint8_t ThermClampMode;
+ uint8_t VoltageBusy;
+
+ int8_t CurrLevel;
+ int8_t TargLevel;
+ uint8_t LevelChangeInProgress;
+ uint8_t UpHyst;
+
+ uint8_t DownHyst;
+ uint8_t VoltageDownHyst;
+ uint8_t DpmEnable;
+ uint8_t DpmRunning;
+
+ uint8_t DpmForce;
+ uint8_t DpmForceLevel;
+ uint8_t DisplayWatermark;
+ uint8_t McArbIndex;
+
+ uint32_t MinimumPerfSclk;
+
+ uint8_t AcpiReq;
+ uint8_t AcpiAck;
+ uint8_t GfxClkSlow;
+ uint8_t GpioClampMode;
+
+ uint8_t spare2;
+ uint8_t EnabledLevelsChange;
+ uint8_t DteClampMode;
+ uint8_t FpsClampMode;
+
+ uint16_t LevelResidencyCounters[SMU74_MAX_LEVELS_GRAPHICS];
+ uint16_t LevelSwitchCounters[SMU74_MAX_LEVELS_GRAPHICS];
+
+ void (*TargetStateCalculator)(uint8_t);
+ void (*SavedTargetStateCalculator)(uint8_t);
+
+ uint16_t AutoDpmInterval;
+ uint16_t AutoDpmRange;
+
+ uint8_t FpsEnabled;
+ uint8_t MaxPerfLevel;
+ uint8_t AllowLowClkInterruptToHost;
+ uint8_t FpsRunning;
+
+ uint32_t MaxAllowedFrequency;
+
+ uint32_t FilteredSclkFrequency;
+ uint32_t LastSclkFrequency;
+ uint32_t FilteredSclkFrequencyCnt;
+
+ uint8_t MinPerfLevel;
+ uint8_t padding[3];
+
+ uint16_t FpsAlpha;
+ uint16_t DeltaTime;
+ uint32_t CurrentFps;
+ uint32_t FilteredFps;
+ uint32_t FrameCount;
+ uint32_t FrameCountLast;
+ uint16_t FpsTargetScalar;
+ uint16_t FpsWaterfallLimitScalar;
+ uint16_t FpsAlphaScalar;
+ uint16_t spare8;
+ SMU7_HystController_Data HystControllerData;
+};
+
+typedef struct SMU7_LocalDpmScoreboard SMU7_LocalDpmScoreboard;
+
+#define SMU7_MAX_VOLTAGE_CLIENTS 12
+
+typedef uint8_t (*VoltageChangeHandler_t)(uint16_t, uint8_t);
+
+#define VDDC_MASK 0x00007FFF
+#define VDDC_SHIFT 0
+#define VDDCI_MASK 0x3FFF8000
+#define VDDCI_SHIFT 15
+#define PHASES_MASK 0xC0000000
+#define PHASES_SHIFT 30
+
+typedef uint32_t SMU_VoltageLevel;
+
+struct SMU7_VoltageScoreboard {
+
+ SMU_VoltageLevel TargetVoltage;
+ uint16_t MaxVid;
+ uint8_t HighestVidOffset;
+ uint8_t CurrentVidOffset;
+
+ uint16_t CurrentVddc;
+ uint16_t CurrentVddci;
+
+
+ uint8_t ControllerBusy;
+ uint8_t CurrentVid;
+ uint8_t CurrentVddciVid;
+ uint8_t padding;
+
+ SMU_VoltageLevel RequestedVoltage[SMU7_MAX_VOLTAGE_CLIENTS];
+ SMU_VoltageLevel TargetVoltageState;
+ uint8_t EnabledRequest[SMU7_MAX_VOLTAGE_CLIENTS];
+
+ uint8_t padding2;
+ uint8_t padding3;
+ uint8_t ControllerEnable;
+ uint8_t ControllerRunning;
+ uint16_t CurrentStdVoltageHiSidd;
+ uint16_t CurrentStdVoltageLoSidd;
+ uint8_t OverrideVoltage;
+ uint8_t padding4;
+ uint8_t padding5;
+ uint8_t CurrentPhases;
+
+ VoltageChangeHandler_t ChangeVddc;
+
+ VoltageChangeHandler_t ChangeVddci;
+ VoltageChangeHandler_t ChangePhase;
+ VoltageChangeHandler_t ChangeMvdd;
+
+ VoltageChangeHandler_t functionLinks[6];
+
+ uint16_t *VddcFollower1;
+
+ int16_t Driver_OD_RequestedVidOffset1;
+ int16_t Driver_OD_RequestedVidOffset2;
+};
+
+typedef struct SMU7_VoltageScoreboard SMU7_VoltageScoreboard;
+
+#define SMU7_MAX_PCIE_LINK_SPEEDS 3 /* 0:Gen1 1:Gen2 2:Gen3 */
+
+struct SMU7_PCIeLinkSpeedScoreboard {
+ uint8_t DpmEnable;
+ uint8_t DpmRunning;
+ uint8_t DpmForce;
+ uint8_t DpmForceLevel;
+
+ uint8_t CurrentLinkSpeed;
+ uint8_t EnabledLevelsChange;
+ uint16_t AutoDpmInterval;
+
+ uint16_t AutoDpmRange;
+ uint16_t AutoDpmCount;
+
+ uint8_t DpmMode;
+ uint8_t AcpiReq;
+ uint8_t AcpiAck;
+ uint8_t CurrentLinkLevel;
+
+};
+
+typedef struct SMU7_PCIeLinkSpeedScoreboard SMU7_PCIeLinkSpeedScoreboard;
+
+#define SMU7_LKGE_LUT_NUM_OF_TEMP_ENTRIES 16
+#define SMU7_LKGE_LUT_NUM_OF_VOLT_ENTRIES 16
+
+#define SMU7_SCALE_I 7
+#define SMU7_SCALE_R 12
+
+struct SMU7_PowerScoreboard {
+ PowerCalculatorData_t VddcPowerData[SID_OPTION_COUNT];
+
+ uint32_t TotalGpuPower;
+ uint32_t TdcCurrent;
+
+ uint16_t VddciTotalPower;
+ uint16_t sparesasfsdfd;
+ uint16_t Vddr1Power;
+ uint16_t RocPower;
+
+ uint16_t CalcMeasPowerBlend;
+ uint8_t SidOptionPower;
+ uint8_t SidOptionCurrent;
+
+ uint32_t WinTime;
+
+ uint16_t Telemetry_1_slope;
+ uint16_t Telemetry_2_slope;
+ int32_t Telemetry_1_offset;
+ int32_t Telemetry_2_offset;
+
+ uint32_t VddcCurrentTelemetry;
+ uint32_t VddGfxCurrentTelemetry;
+ uint32_t VddcPowerTelemetry;
+ uint32_t VddGfxPowerTelemetry;
+ uint32_t VddciPowerTelemetry;
+
+ uint32_t VddcPower;
+ uint32_t VddGfxPower;
+ uint32_t VddciPower;
+
+ uint32_t TelemetryCurrent[2];
+ uint32_t TelemetryVoltage[2];
+ uint32_t TelemetryPower[2];
+};
+
+typedef struct SMU7_PowerScoreboard SMU7_PowerScoreboard;
+
+struct SMU7_ThermalScoreboard {
+ int16_t GpuLimit;
+ int16_t GpuHyst;
+ uint16_t CurrGnbTemp;
+ uint16_t FilteredGnbTemp;
+
+ uint8_t ControllerEnable;
+ uint8_t ControllerRunning;
+ uint8_t AutoTmonCalInterval;
+ uint8_t AutoTmonCalEnable;
+
+ uint8_t ThermalDpmEnabled;
+ uint8_t SclkEnabledMask;
+ uint8_t spare[2];
+ int32_t temperature_gradient;
+
+ SMU7_HystController_Data HystControllerData;
+ int32_t WeightedSensorTemperature;
+ uint16_t TemperatureLimit[SMU74_MAX_LEVELS_GRAPHICS];
+ uint32_t Alpha;
+};
+
+typedef struct SMU7_ThermalScoreboard SMU7_ThermalScoreboard;
+
+#define SMU7_SCLK_DPM_CONFIG_MASK 0x01
+#define SMU7_VOLTAGE_CONTROLLER_CONFIG_MASK 0x02
+#define SMU7_THERMAL_CONTROLLER_CONFIG_MASK 0x04
+#define SMU7_MCLK_DPM_CONFIG_MASK 0x08
+#define SMU7_UVD_DPM_CONFIG_MASK 0x10
+#define SMU7_VCE_DPM_CONFIG_MASK 0x20
+#define SMU7_ACP_DPM_CONFIG_MASK 0x40
+#define SMU7_SAMU_DPM_CONFIG_MASK 0x80
+#define SMU7_PCIEGEN_DPM_CONFIG_MASK 0x100
+
+#define SMU7_ACP_MCLK_HANDSHAKE_DISABLE 0x00000001
+#define SMU7_ACP_SCLK_HANDSHAKE_DISABLE 0x00000002
+#define SMU7_UVD_MCLK_HANDSHAKE_DISABLE 0x00000100
+#define SMU7_UVD_SCLK_HANDSHAKE_DISABLE 0x00000200
+#define SMU7_VCE_MCLK_HANDSHAKE_DISABLE 0x00010000
+#define SMU7_VCE_SCLK_HANDSHAKE_DISABLE 0x00020000
+
+/* All 'soft registers' should be uint32_t. */
+struct SMU74_SoftRegisters {
+ uint32_t RefClockFrequency;
+ uint32_t PmTimerPeriod;
+ uint32_t FeatureEnables;
+
+ uint32_t PreVBlankGap;
+ uint32_t VBlankTimeout;
+ uint32_t TrainTimeGap;
+
+ uint32_t MvddSwitchTime;
+ uint32_t LongestAcpiTrainTime;
+ uint32_t AcpiDelay;
+ uint32_t G5TrainTime;
+ uint32_t DelayMpllPwron;
+ uint32_t VoltageChangeTimeout;
+
+ uint32_t HandshakeDisables;
+
+ uint8_t DisplayPhy1Config;
+ uint8_t DisplayPhy2Config;
+ uint8_t DisplayPhy3Config;
+ uint8_t DisplayPhy4Config;
+
+ uint8_t DisplayPhy5Config;
+ uint8_t DisplayPhy6Config;
+ uint8_t DisplayPhy7Config;
+ uint8_t DisplayPhy8Config;
+
+ uint32_t AverageGraphicsActivity;
+ uint32_t AverageMemoryActivity;
+ uint32_t AverageGioActivity;
+
+ uint8_t SClkDpmEnabledLevels;
+ uint8_t MClkDpmEnabledLevels;
+ uint8_t LClkDpmEnabledLevels;
+ uint8_t PCIeDpmEnabledLevels;
+
+ uint8_t UVDDpmEnabledLevels;
+ uint8_t SAMUDpmEnabledLevels;
+ uint8_t ACPDpmEnabledLevels;
+ uint8_t VCEDpmEnabledLevels;
+
+ uint32_t DRAM_LOG_ADDR_H;
+ uint32_t DRAM_LOG_ADDR_L;
+ uint32_t DRAM_LOG_PHY_ADDR_H;
+ uint32_t DRAM_LOG_PHY_ADDR_L;
+ uint32_t DRAM_LOG_BUFF_SIZE;
+ uint32_t UlvEnterCount;
+ uint32_t UlvTime;
+ uint32_t UcodeLoadStatus;
+ uint32_t AllowMvddSwitch;
+ uint8_t Activity_Weight;
+ uint8_t Reserved8[3];
+};
+
+typedef struct SMU74_SoftRegisters SMU74_SoftRegisters;
+
+struct SMU74_Firmware_Header {
+ uint32_t Digest[5];
+ uint32_t Version;
+ uint32_t HeaderSize;
+ uint32_t Flags;
+ uint32_t EntryPoint;
+ uint32_t CodeSize;
+ uint32_t ImageSize;
+
+ uint32_t Rtos;
+ uint32_t SoftRegisters;
+ uint32_t DpmTable;
+ uint32_t FanTable;
+ uint32_t CacConfigTable;
+ uint32_t CacStatusTable;
+
+
+ uint32_t mcRegisterTable;
+
+
+ uint32_t mcArbDramTimingTable;
+
+
+
+
+ uint32_t PmFuseTable;
+ uint32_t Globals;
+ uint32_t ClockStretcherTable;
+ uint32_t VftTable;
+ uint32_t Reserved[21];
+ uint32_t Signature;
+};
+
+typedef struct SMU74_Firmware_Header SMU74_Firmware_Header;
+
+#define SMU7_FIRMWARE_HEADER_LOCATION 0x20000
+
+enum DisplayConfig {
+ PowerDown = 1,
+ DP54x4,
+ DP54x2,
+ DP54x1,
+ DP27x4,
+ DP27x2,
+ DP27x1,
+ HDMI297,
+ HDMI162,
+ LVDS,
+ DP324x4,
+ DP324x2,
+ DP324x1
+};
+
+
+#define MC_BLOCK_COUNT 1
+#define CPL_BLOCK_COUNT 5
+#define SE_BLOCK_COUNT 15
+#define GC_BLOCK_COUNT 24
+
+struct SMU7_Local_Cac {
+ uint8_t BlockId;
+ uint8_t SignalId;
+ uint8_t Threshold;
+ uint8_t Padding;
+};
+
+typedef struct SMU7_Local_Cac SMU7_Local_Cac;
+
+struct SMU7_Local_Cac_Table {
+
+ SMU7_Local_Cac CplLocalCac[CPL_BLOCK_COUNT];
+ SMU7_Local_Cac McLocalCac[MC_BLOCK_COUNT];
+ SMU7_Local_Cac SeLocalCac[SE_BLOCK_COUNT];
+ SMU7_Local_Cac GcLocalCac[GC_BLOCK_COUNT];
+};
+
+typedef struct SMU7_Local_Cac_Table SMU7_Local_Cac_Table;
+
+#pragma pack(pop)
+
+/* Description of Clock Gating bitmask for Tonga:
+ * System Clock Gating
+ */
+#define CG_SYS_BITMASK_FIRST_BIT 0 /* First bit of Sys CG bitmask */
+#define CG_SYS_BITMASK_LAST_BIT 9 /* Last bit of Sys CG bitmask */
+#define CG_SYS_BIF_MGLS_SHIFT 0
+#define CG_SYS_ROM_SHIFT 1
+#define CG_SYS_MC_MGCG_SHIFT 2
+#define CG_SYS_MC_MGLS_SHIFT 3
+#define CG_SYS_SDMA_MGCG_SHIFT 4
+#define CG_SYS_SDMA_MGLS_SHIFT 5
+#define CG_SYS_DRM_MGCG_SHIFT 6
+#define CG_SYS_HDP_MGCG_SHIFT 7
+#define CG_SYS_HDP_MGLS_SHIFT 8
+#define CG_SYS_DRM_MGLS_SHIFT 9
+#define CG_SYS_BIF_MGCG_SHIFT 10
+
+#define CG_SYS_BIF_MGLS_MASK 0x1
+#define CG_SYS_ROM_MASK 0x2
+#define CG_SYS_MC_MGCG_MASK 0x4
+#define CG_SYS_MC_MGLS_MASK 0x8
+#define CG_SYS_SDMA_MGCG_MASK 0x10
+#define CG_SYS_SDMA_MGLS_MASK 0x20
+#define CG_SYS_DRM_MGCG_MASK 0x40
+#define CG_SYS_HDP_MGCG_MASK 0x80
+#define CG_SYS_HDP_MGLS_MASK 0x100
+#define CG_SYS_DRM_MGLS_MASK 0x200
+#define CG_SYS_BIF_MGCG_MASK 0x400
+
+/* Graphics Clock Gating */
+#define CG_GFX_BITMASK_FIRST_BIT 16 /* First bit of Gfx CG bitmask */
+#define CG_GFX_BITMASK_LAST_BIT 24 /* Last bit of Gfx CG bitmask */
+
+#define CG_GFX_CGCG_SHIFT 16
+#define CG_GFX_CGLS_SHIFT 17
+#define CG_CPF_MGCG_SHIFT 18
+#define CG_RLC_MGCG_SHIFT 19
+#define CG_GFX_OTHERS_MGCG_SHIFT 20
+#define CG_GFX_3DCG_SHIFT 21
+#define CG_GFX_3DLS_SHIFT 22
+#define CG_GFX_RLC_LS_SHIFT 23
+#define CG_GFX_CP_LS_SHIFT 24
+
+#define CG_GFX_CGCG_MASK 0x00010000
+#define CG_GFX_CGLS_MASK 0x00020000
+#define CG_CPF_MGCG_MASK 0x00040000
+#define CG_RLC_MGCG_MASK 0x00080000
+#define CG_GFX_OTHERS_MGCG_MASK 0x00100000
+#define CG_GFX_3DCG_MASK 0x00200000
+#define CG_GFX_3DLS_MASK 0x00400000
+#define CG_GFX_RLC_LS_MASK 0x00800000
+#define CG_GFX_CP_LS_MASK 0x01000000
+
+
+/* Voltage Regulator Configuration
+VR Config info is contained in dpmTable.VRConfig */
+
+#define VRCONF_VDDC_MASK 0x000000FF
+#define VRCONF_VDDC_SHIFT 0
+#define VRCONF_VDDGFX_MASK 0x0000FF00
+#define VRCONF_VDDGFX_SHIFT 8
+#define VRCONF_VDDCI_MASK 0x00FF0000
+#define VRCONF_VDDCI_SHIFT 16
+#define VRCONF_MVDD_MASK 0xFF000000
+#define VRCONF_MVDD_SHIFT 24
+
+#define VR_MERGED_WITH_VDDC 0
+#define VR_SVI2_PLANE_1 1
+#define VR_SVI2_PLANE_2 2
+#define VR_SMIO_PATTERN_1 3
+#define VR_SMIO_PATTERN_2 4
+#define VR_STATIC_VOLTAGE 5
+
+/* Clock Stretcher Configuration */
+
+#define CLOCK_STRETCHER_MAX_ENTRIES 0x4
+#define CKS_LOOKUPTable_MAX_ENTRIES 0x4
+
+/* The 'settings' field is subdivided in the following way: */
+#define CLOCK_STRETCHER_SETTING_DDT_MASK 0x01
+#define CLOCK_STRETCHER_SETTING_DDT_SHIFT 0x0
+#define CLOCK_STRETCHER_SETTING_STRETCH_AMOUNT_MASK 0x1E
+#define CLOCK_STRETCHER_SETTING_STRETCH_AMOUNT_SHIFT 0x1
+#define CLOCK_STRETCHER_SETTING_ENABLE_MASK 0x80
+#define CLOCK_STRETCHER_SETTING_ENABLE_SHIFT 0x7
+
+struct SMU_ClockStretcherDataTableEntry {
+ uint8_t minVID;
+ uint8_t maxVID;
+
+
+ uint16_t setting;
+};
+typedef struct SMU_ClockStretcherDataTableEntry SMU_ClockStretcherDataTableEntry;
+
+struct SMU_ClockStretcherDataTable {
+ SMU_ClockStretcherDataTableEntry ClockStretcherDataTableEntry[CLOCK_STRETCHER_MAX_ENTRIES];
+};
+typedef struct SMU_ClockStretcherDataTable SMU_ClockStretcherDataTable;
+
+struct SMU_CKS_LOOKUPTableEntry {
+ uint16_t minFreq;
+ uint16_t maxFreq;
+
+ uint8_t setting;
+ uint8_t padding[3];
+};
+typedef struct SMU_CKS_LOOKUPTableEntry SMU_CKS_LOOKUPTableEntry;
+
+struct SMU_CKS_LOOKUPTable {
+ SMU_CKS_LOOKUPTableEntry CKS_LOOKUPTableEntry[CKS_LOOKUPTable_MAX_ENTRIES];
+};
+typedef struct SMU_CKS_LOOKUPTable SMU_CKS_LOOKUPTable;
+
+struct AgmAvfsData_t {
+ uint16_t avgPsmCount[28];
+ uint16_t minPsmCount[28];
+};
+
+typedef struct AgmAvfsData_t AgmAvfsData_t;
+
+enum VFT_COLUMNS {
+ SCLK0,
+ SCLK1,
+ SCLK2,
+ SCLK3,
+ SCLK4,
+ SCLK5,
+ SCLK6,
+ SCLK7,
+
+ NUM_VFT_COLUMNS
+};
+
+#define VFT_TABLE_DEFINED
+
+#define TEMP_RANGE_MAXSTEPS 12
+
+struct VFT_CELL_t {
+ uint16_t Voltage;
+};
+
+typedef struct VFT_CELL_t VFT_CELL_t;
+
+struct VFT_TABLE_t {
+ VFT_CELL_t Cell[TEMP_RANGE_MAXSTEPS][NUM_VFT_COLUMNS];
+ uint16_t AvfsGbv[NUM_VFT_COLUMNS];
+ uint16_t BtcGbv[NUM_VFT_COLUMNS];
+ uint16_t Temperature[TEMP_RANGE_MAXSTEPS];
+
+ uint8_t NumTemperatureSteps;
+ uint8_t padding[3];
+};
+
+typedef struct VFT_TABLE_t VFT_TABLE_t;
+
+
+#endif
+
+
diff --git a/drivers/gpu/drm/amd/powerplay/inc/smu74_discrete.h b/drivers/gpu/drm/amd/powerplay/inc/smu74_discrete.h
new file mode 100644
index 0000000..0dfe823
--- /dev/null
+++ b/drivers/gpu/drm/amd/powerplay/inc/smu74_discrete.h
@@ -0,0 +1,828 @@
+/*
+ * Copyright 2014 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#ifndef SMU74_DISCRETE_H
+#define SMU74_DISCRETE_H
+
+#include "smu74.h"
+
+#pragma pack(push, 1)
+
+
+#define NUM_SCLK_RANGE 8
+
+#define VCO_3_6 1
+#define VCO_2_4 3
+
+#define POSTDIV_DIV_BY_1 0
+#define POSTDIV_DIV_BY_2 1
+#define POSTDIV_DIV_BY_4 2
+#define POSTDIV_DIV_BY_8 3
+#define POSTDIV_DIV_BY_16 4
+
+struct sclkFcwRange_t {
+ uint8_t vco_setting;
+ uint8_t postdiv;
+ uint16_t fcw_pcc;
+
+ uint16_t fcw_trans_upper;
+ uint16_t fcw_trans_lower;
+};
+typedef struct sclkFcwRange_t sclkFcwRange_t;
+
+struct SMIO_Pattern {
+ uint16_t Voltage;
+ uint8_t Smio;
+ uint8_t padding;
+};
+
+typedef struct SMIO_Pattern SMIO_Pattern;
+
+struct SMIO_Table {
+ SMIO_Pattern Pattern[SMU_MAX_SMIO_LEVELS];
+};
+
+typedef struct SMIO_Table SMIO_Table;
+
+struct SMU_SclkSetting {
+ uint32_t SclkFrequency;
+ uint16_t Fcw_int;
+ uint16_t Fcw_frac;
+ uint16_t Pcc_fcw_int;
+ uint8_t PllRange;
+ uint8_t SSc_En;
+ uint16_t Sclk_slew_rate;
+ uint16_t Pcc_up_slew_rate;
+ uint16_t Pcc_down_slew_rate;
+ uint16_t Fcw1_int;
+ uint16_t Fcw1_frac;
+ uint16_t Sclk_ss_slew_rate;
+};
+typedef struct SMU_SclkSetting SMU_SclkSetting;
+
+struct SMU74_Discrete_GraphicsLevel {
+ SMU_VoltageLevel MinVoltage;
+ uint8_t pcieDpmLevel;
+ uint8_t DeepSleepDivId;
+ uint16_t ActivityLevel;
+ uint32_t CgSpllFuncCntl3;
+ uint32_t CgSpllFuncCntl4;
+ uint32_t CcPwrDynRm;
+ uint32_t CcPwrDynRm1;
+ uint8_t SclkDid;
+ uint8_t padding;
+ uint8_t EnabledForActivity;
+ uint8_t EnabledForThrottle;
+ uint8_t UpHyst;
+ uint8_t DownHyst;
+ uint8_t VoltageDownHyst;
+ uint8_t PowerThrottle;
+ SMU_SclkSetting SclkSetting;
+};
+
+typedef struct SMU74_Discrete_GraphicsLevel SMU74_Discrete_GraphicsLevel;
+
+struct SMU74_Discrete_ACPILevel {
+ uint32_t Flags;
+ SMU_VoltageLevel MinVoltage;
+ uint32_t SclkFrequency;
+ uint8_t SclkDid;
+ uint8_t DisplayWatermark;
+ uint8_t DeepSleepDivId;
+ uint8_t padding;
+ uint32_t CcPwrDynRm;
+ uint32_t CcPwrDynRm1;
+
+ SMU_SclkSetting SclkSetting;
+};
+
+typedef struct SMU74_Discrete_ACPILevel SMU74_Discrete_ACPILevel;
+
+struct SMU74_Discrete_Ulv {
+ uint32_t CcPwrDynRm;
+ uint32_t CcPwrDynRm1;
+ uint16_t VddcOffset;
+ uint8_t VddcOffsetVid;
+ uint8_t VddcPhase;
+ uint16_t BifSclkDfs;
+ uint16_t Reserved;
+};
+
+typedef struct SMU74_Discrete_Ulv SMU74_Discrete_Ulv;
+
+struct SMU74_Discrete_MemoryLevel {
+ SMU_VoltageLevel MinVoltage;
+ uint32_t MinMvdd;
+
+ uint32_t MclkFrequency;
+
+ uint8_t StutterEnable;
+ uint8_t EnabledForThrottle;
+ uint8_t EnabledForActivity;
+ uint8_t padding_0;
+
+ uint8_t UpHyst;
+ uint8_t DownHyst;
+ uint8_t VoltageDownHyst;
+ uint8_t padding_1;
+
+ uint16_t ActivityLevel;
+ uint8_t DisplayWatermark;
+ uint8_t Reserved;
+};
+
+typedef struct SMU74_Discrete_MemoryLevel SMU74_Discrete_MemoryLevel;
+
+struct SMU74_Discrete_LinkLevel {
+ uint8_t PcieGenSpeed;
+ uint8_t PcieLaneCount;
+ uint8_t EnabledForActivity;
+ uint8_t SPC;
+ uint32_t DownThreshold;
+ uint32_t UpThreshold;
+ uint16_t BifSclkDfs;
+ uint16_t Reserved;
+};
+
+typedef struct SMU74_Discrete_LinkLevel SMU74_Discrete_LinkLevel;
+
+struct SMU74_Discrete_MCArbDramTimingTableEntry {
+ uint32_t McArbDramTiming;
+ uint32_t McArbDramTiming2;
+ uint8_t McArbBurstTime;
+ uint8_t padding[3];
+};
+
+typedef struct SMU74_Discrete_MCArbDramTimingTableEntry SMU74_Discrete_MCArbDramTimingTableEntry;
+
+struct SMU74_Discrete_MCArbDramTimingTable {
+ SMU74_Discrete_MCArbDramTimingTableEntry entries[SMU__NUM_SCLK_DPM_STATE][SMU__NUM_MCLK_DPM_LEVELS];
+};
+
+typedef struct SMU74_Discrete_MCArbDramTimingTable SMU74_Discrete_MCArbDramTimingTable;
+
+struct SMU74_Discrete_UvdLevel {
+ uint32_t VclkFrequency;
+ uint32_t DclkFrequency;
+ SMU_VoltageLevel MinVoltage;
+ uint8_t VclkDivider;
+ uint8_t DclkDivider;
+ uint8_t padding[2];
+};
+
+typedef struct SMU74_Discrete_UvdLevel SMU74_Discrete_UvdLevel;
+
+struct SMU74_Discrete_ExtClkLevel {
+ uint32_t Frequency;
+ SMU_VoltageLevel MinVoltage;
+ uint8_t Divider;
+ uint8_t padding[3];
+};
+
+typedef struct SMU74_Discrete_ExtClkLevel SMU74_Discrete_ExtClkLevel;
+
+struct SMU74_Discrete_StateInfo {
+ uint32_t SclkFrequency;
+ uint32_t MclkFrequency;
+ uint32_t VclkFrequency;
+ uint32_t DclkFrequency;
+ uint32_t SamclkFrequency;
+ uint32_t AclkFrequency;
+ uint32_t EclkFrequency;
+ uint16_t MvddVoltage;
+ uint16_t padding16;
+ uint8_t DisplayWatermark;
+ uint8_t McArbIndex;
+ uint8_t McRegIndex;
+ uint8_t SeqIndex;
+ uint8_t SclkDid;
+ int8_t SclkIndex;
+ int8_t MclkIndex;
+ uint8_t PCIeGen;
+};
+
+typedef struct SMU74_Discrete_StateInfo SMU74_Discrete_StateInfo;
+
+struct SMU74_Discrete_DpmTable {
+
+ SMU74_PIDController GraphicsPIDController;
+ SMU74_PIDController MemoryPIDController;
+ SMU74_PIDController LinkPIDController;
+
+ uint32_t SystemFlags;
+
+ uint32_t VRConfig;
+ uint32_t SmioMask1;
+ uint32_t SmioMask2;
+ SMIO_Table SmioTable1;
+ SMIO_Table SmioTable2;
+
+ uint32_t MvddLevelCount;
+
+
+ uint8_t BapmVddcVidHiSidd[SMU74_MAX_LEVELS_VDDC];
+ uint8_t BapmVddcVidLoSidd[SMU74_MAX_LEVELS_VDDC];
+ uint8_t BapmVddcVidHiSidd2[SMU74_MAX_LEVELS_VDDC];
+
+ uint8_t GraphicsDpmLevelCount;
+ uint8_t MemoryDpmLevelCount;
+ uint8_t LinkLevelCount;
+ uint8_t MasterDeepSleepControl;
+
+ uint8_t UvdLevelCount;
+ uint8_t VceLevelCount;
+ uint8_t AcpLevelCount;
+ uint8_t SamuLevelCount;
+
+ uint8_t ThermOutGpio;
+ uint8_t ThermOutPolarity;
+ uint8_t ThermOutMode;
+ uint8_t BootPhases;
+ uint32_t Reserved[4];
+
+ SMU74_Discrete_GraphicsLevel GraphicsLevel[SMU74_MAX_LEVELS_GRAPHICS];
+ SMU74_Discrete_MemoryLevel MemoryACPILevel;
+ SMU74_Discrete_MemoryLevel MemoryLevel[SMU74_MAX_LEVELS_MEMORY];
+ SMU74_Discrete_LinkLevel LinkLevel[SMU74_MAX_LEVELS_LINK];
+ SMU74_Discrete_ACPILevel ACPILevel;
+ SMU74_Discrete_UvdLevel UvdLevel[SMU74_MAX_LEVELS_UVD];
+ SMU74_Discrete_ExtClkLevel VceLevel[SMU74_MAX_LEVELS_VCE];
+ SMU74_Discrete_ExtClkLevel AcpLevel[SMU74_MAX_LEVELS_ACP];
+ SMU74_Discrete_ExtClkLevel SamuLevel[SMU74_MAX_LEVELS_SAMU];
+ SMU74_Discrete_Ulv Ulv;
+
+ uint8_t DisplayWatermark[SMU74_MAX_LEVELS_MEMORY][SMU74_MAX_LEVELS_GRAPHICS];
+
+ uint32_t SclkStepSize;
+ uint32_t Smio[SMU74_MAX_ENTRIES_SMIO];
+
+ uint8_t UvdBootLevel;
+ uint8_t VceBootLevel;
+ uint8_t AcpBootLevel;
+ uint8_t SamuBootLevel;
+
+ uint8_t GraphicsBootLevel;
+ uint8_t GraphicsVoltageChangeEnable;
+ uint8_t GraphicsThermThrottleEnable;
+ uint8_t GraphicsInterval;
+
+ uint8_t VoltageInterval;
+ uint8_t ThermalInterval;
+ uint16_t TemperatureLimitHigh;
+
+ uint16_t TemperatureLimitLow;
+ uint8_t MemoryBootLevel;
+ uint8_t MemoryVoltageChangeEnable;
+
+ uint16_t BootMVdd;
+ uint8_t MemoryInterval;
+ uint8_t MemoryThermThrottleEnable;
+
+ uint16_t VoltageResponseTime;
+ uint16_t PhaseResponseTime;
+
+ uint8_t PCIeBootLinkLevel;
+ uint8_t PCIeGenInterval;
+ uint8_t DTEInterval;
+ uint8_t DTEMode;
+
+ uint8_t SVI2Enable;
+ uint8_t VRHotGpio;
+ uint8_t AcDcGpio;
+ uint8_t ThermGpio;
+
+ uint16_t PPM_PkgPwrLimit;
+ uint16_t PPM_TemperatureLimit;
+
+ uint16_t DefaultTdp;
+ uint16_t TargetTdp;
+
+ uint16_t FpsHighThreshold;
+ uint16_t FpsLowThreshold;
+
+ uint16_t BAPMTI_R[SMU74_DTE_ITERATIONS][SMU74_DTE_SOURCES][SMU74_DTE_SINKS];
+ uint16_t BAPMTI_RC[SMU74_DTE_ITERATIONS][SMU74_DTE_SOURCES][SMU74_DTE_SINKS];
+
+ uint16_t TemperatureLimitEdge;
+ uint16_t TemperatureLimitHotspot;
+
+ uint16_t BootVddc;
+ uint16_t BootVddci;
+
+ uint16_t FanGainEdge;
+ uint16_t FanGainHotspot;
+
+ uint32_t LowSclkInterruptThreshold;
+ uint32_t VddGfxReChkWait;
+
+ uint8_t ClockStretcherAmount;
+ uint8_t Sclk_CKS_masterEn0_7;
+ uint8_t Sclk_CKS_masterEn8_15;
+ uint8_t DPMFreezeAndForced;
+
+ uint8_t Sclk_voltageOffset[8];
+
+ SMU_ClockStretcherDataTable ClockStretcherDataTable;
+ SMU_CKS_LOOKUPTable CKS_LOOKUPTable;
+
+ uint32_t CurrSclkPllRange;
+ sclkFcwRange_t SclkFcwRangeTable[NUM_SCLK_RANGE];
+};
+
+typedef struct SMU74_Discrete_DpmTable SMU74_Discrete_DpmTable;
+
+
+struct SMU74_Discrete_FanTable {
+ uint16_t FdoMode;
+ int16_t TempMin;
+ int16_t TempMed;
+ int16_t TempMax;
+ int16_t Slope1;
+ int16_t Slope2;
+ int16_t FdoMin;
+ int16_t HystUp;
+ int16_t HystDown;
+ int16_t HystSlope;
+ int16_t TempRespLim;
+ int16_t TempCurr;
+ int16_t SlopeCurr;
+ int16_t PwmCurr;
+ uint32_t RefreshPeriod;
+ int16_t FdoMax;
+ uint8_t TempSrc;
+ int8_t Padding;
+};
+
+typedef struct SMU74_Discrete_FanTable SMU74_Discrete_FanTable;
+
+#define SMU7_DISCRETE_GPIO_SCLK_DEBUG 4
+#define SMU7_DISCRETE_GPIO_SCLK_DEBUG_BIT (0x1 << SMU7_DISCRETE_GPIO_SCLK_DEBUG)
+
+
+struct SMU7_MclkDpmScoreboard {
+ uint32_t PercentageBusy;
+
+ int32_t PIDError;
+ int32_t PIDIntegral;
+ int32_t PIDOutput;
+
+ uint32_t SigmaDeltaAccum;
+ uint32_t SigmaDeltaOutput;
+ uint32_t SigmaDeltaLevel;
+
+ uint32_t UtilizationSetpoint;
+
+ uint8_t TdpClampMode;
+ uint8_t TdcClampMode;
+ uint8_t ThermClampMode;
+ uint8_t VoltageBusy;
+
+ int8_t CurrLevel;
+ int8_t TargLevel;
+ uint8_t LevelChangeInProgress;
+ uint8_t UpHyst;
+
+ uint8_t DownHyst;
+ uint8_t VoltageDownHyst;
+ uint8_t DpmEnable;
+ uint8_t DpmRunning;
+
+ uint8_t DpmForce;
+ uint8_t DpmForceLevel;
+ uint8_t padding2;
+ uint8_t McArbIndex;
+
+ uint32_t MinimumPerfMclk;
+
+ uint8_t AcpiReq;
+ uint8_t AcpiAck;
+ uint8_t MclkSwitchInProgress;
+ uint8_t MclkSwitchCritical;
+
+ uint8_t IgnoreVBlank;
+ uint8_t TargetMclkIndex;
+ uint16_t VbiFailureCount;
+ uint8_t VbiWaitCounter;
+ uint8_t EnabledLevelsChange;
+
+ uint16_t LevelResidencyCounters[SMU74_MAX_LEVELS_MEMORY];
+ uint16_t LevelSwitchCounters[SMU74_MAX_LEVELS_MEMORY];
+
+ void (*TargetStateCalculator)(uint8_t);
+ void (*SavedTargetStateCalculator)(uint8_t);
+
+ uint16_t AutoDpmInterval;
+ uint16_t AutoDpmRange;
+
+ uint16_t VbiTimeoutCount;
+ uint16_t MclkSwitchingTime;
+
+ uint8_t fastSwitch;
+ uint8_t Save_PIC_VDDGFX_EXIT;
+ uint8_t Save_PIC_VDDGFX_ENTER;
+ uint8_t padding;
+};
+
+typedef struct SMU7_MclkDpmScoreboard SMU7_MclkDpmScoreboard;
+
+struct SMU7_UlvScoreboard {
+ uint8_t EnterUlv;
+ uint8_t ExitUlv;
+ uint8_t UlvActive;
+ uint8_t WaitingForUlv;
+ uint8_t UlvEnable;
+ uint8_t UlvRunning;
+ uint8_t UlvMasterEnable;
+ uint8_t padding;
+ uint32_t UlvAbortedCount;
+ uint32_t UlvTimeStamp;
+};
+
+typedef struct SMU7_UlvScoreboard SMU7_UlvScoreboard;
+
+struct VddgfxSavedRegisters {
+ uint32_t GPU_DBG[3];
+ uint32_t MEC_BaseAddress_Hi;
+ uint32_t MEC_BaseAddress_Lo;
+ uint32_t THM_TMON0_CTRL2__RDIR_PRESENT;
+ uint32_t THM_TMON1_CTRL2__RDIR_PRESENT;
+ uint32_t CP_INT_CNTL;
+};
+
+typedef struct VddgfxSavedRegisters VddgfxSavedRegisters;
+
+struct SMU7_VddGfxScoreboard {
+ uint8_t VddGfxEnable;
+ uint8_t VddGfxActive;
+ uint8_t VPUResetOccured;
+ uint8_t padding;
+
+ uint32_t VddGfxEnteredCount;
+ uint32_t VddGfxAbortedCount;
+
+ uint32_t VddGfxVid;
+
+ VddgfxSavedRegisters SavedRegisters;
+};
+
+typedef struct SMU7_VddGfxScoreboard SMU7_VddGfxScoreboard;
+
+struct SMU7_TdcLimitScoreboard {
+ uint8_t Enable;
+ uint8_t Running;
+ uint16_t Alpha;
+ uint32_t FilteredIddc;
+ uint32_t IddcLimit;
+ uint32_t IddcHyst;
+ SMU7_HystController_Data HystControllerData;
+};
+
+typedef struct SMU7_TdcLimitScoreboard SMU7_TdcLimitScoreboard;
+
+struct SMU7_PkgPwrLimitScoreboard {
+ uint8_t Enable;
+ uint8_t Running;
+ uint16_t Alpha;
+ uint32_t FilteredPkgPwr;
+ uint32_t Limit;
+ uint32_t Hyst;
+ uint32_t LimitFromDriver;
+ SMU7_HystController_Data HystControllerData;
+};
+
+typedef struct SMU7_PkgPwrLimitScoreboard SMU7_PkgPwrLimitScoreboard;
+
+struct SMU7_BapmScoreboard {
+ uint32_t source_powers[SMU74_DTE_SOURCES];
+ uint32_t source_powers_last[SMU74_DTE_SOURCES];
+ int32_t entity_temperatures[SMU74_NUM_GPU_TES];
+ int32_t initial_entity_temperatures[SMU74_NUM_GPU_TES];
+ int32_t Limit;
+ int32_t Hyst;
+ int32_t therm_influence_coeff_table[SMU74_DTE_ITERATIONS * SMU74_DTE_SOURCES * SMU74_DTE_SINKS * 2];
+ int32_t therm_node_table[SMU74_DTE_ITERATIONS * SMU74_DTE_SOURCES * SMU74_DTE_SINKS];
+ uint16_t ConfigTDPPowerScalar;
+ uint16_t FanSpeedPowerScalar;
+ uint16_t OverDrivePowerScalar;
+ uint16_t OverDriveLimitScalar;
+ uint16_t FinalPowerScalar;
+ uint8_t VariantID;
+ uint8_t spare997;
+
+ SMU7_HystController_Data HystControllerData;
+
+ int32_t temperature_gradient_slope;
+ int32_t temperature_gradient;
+ uint32_t measured_temperature;
+};
+
+
+typedef struct SMU7_BapmScoreboard SMU7_BapmScoreboard;
+
+struct SMU7_AcpiScoreboard {
+ uint32_t SavedInterruptMask[2];
+ uint8_t LastACPIRequest;
+ uint8_t CgBifResp;
+ uint8_t RequestType;
+ uint8_t Padding;
+ SMU74_Discrete_ACPILevel D0Level;
+};
+
+typedef struct SMU7_AcpiScoreboard SMU7_AcpiScoreboard;
+
+struct SMU_QuadraticCoeffs {
+ int32_t m1;
+ uint32_t b;
+
+ int16_t m2;
+ uint8_t m1_shift;
+ uint8_t m2_shift;
+};
+typedef struct SMU_QuadraticCoeffs SMU_QuadraticCoeffs;
+
+struct SMU74_Discrete_PmFuses {
+ uint8_t BapmVddCVidHiSidd[8];
+ uint8_t BapmVddCVidLoSidd[8];
+ uint8_t VddCVid[8];
+ uint8_t SviLoadLineEn;
+ uint8_t SviLoadLineVddC;
+ uint8_t SviLoadLineTrimVddC;
+ uint8_t SviLoadLineOffsetVddC;
+ uint16_t TDC_VDDC_PkgLimit;
+ uint8_t TDC_VDDC_ThrottleReleaseLimitPerc;
+ uint8_t TDC_MAWt;
+ uint8_t TdcWaterfallCtl;
+ uint8_t LPMLTemperatureMin;
+ uint8_t LPMLTemperatureMax;
+ uint8_t Reserved;
+
+ uint8_t LPMLTemperatureScaler[16];
+
+ int16_t FuzzyFan_ErrorSetDelta;
+ int16_t FuzzyFan_ErrorRateSetDelta;
+ int16_t FuzzyFan_PwmSetDelta;
+ uint16_t Reserved6;
+
+ uint8_t GnbLPML[16];
+
+ uint8_t GnbLPMLMaxVid;
+ uint8_t GnbLPMLMinVid;
+ uint8_t Reserved1[2];
+
+ uint16_t BapmVddCBaseLeakageHiSidd;
+ uint16_t BapmVddCBaseLeakageLoSidd;
+
+ uint16_t VFT_Temp[3];
+ uint16_t padding;
+
+ SMU_QuadraticCoeffs VFT_ATE[3];
+
+ SMU_QuadraticCoeffs AVFS_GB;
+ SMU_QuadraticCoeffs ATE_ACBTC_GB;
+
+ SMU_QuadraticCoeffs P2V;
+
+ uint32_t PsmCharzFreq;
+
+ uint16_t InversionVoltage;
+ uint16_t PsmCharzTemp;
+
+ uint32_t EnabledAvfsModules;
+};
+
+typedef struct SMU74_Discrete_PmFuses SMU74_Discrete_PmFuses;
+
+struct SMU7_Discrete_Log_Header_Table {
+ uint32_t version;
+ uint32_t asic_id;
+ uint16_t flags;
+ uint16_t entry_size;
+ uint32_t total_size;
+ uint32_t num_of_entries;
+ uint8_t type;
+ uint8_t mode;
+ uint8_t filler_0[2];
+ uint32_t filler_1[2];
+};
+
+typedef struct SMU7_Discrete_Log_Header_Table SMU7_Discrete_Log_Header_Table;
+
+struct SMU7_Discrete_Log_Cntl {
+ uint8_t Enabled;
+ uint8_t Type;
+ uint8_t padding[2];
+ uint32_t BufferSize;
+ uint32_t SamplesLogged;
+ uint32_t SampleSize;
+ uint32_t AddrL;
+ uint32_t AddrH;
+};
+
+typedef struct SMU7_Discrete_Log_Cntl SMU7_Discrete_Log_Cntl;
+
+#if defined SMU__DGPU_ONLY
+#define CAC_ACC_NW_NUM_OF_SIGNALS 87
+#endif
+
+
+struct SMU7_Discrete_Cac_Collection_Table {
+ uint32_t temperature;
+ uint32_t cac_acc_nw[CAC_ACC_NW_NUM_OF_SIGNALS];
+};
+
+typedef struct SMU7_Discrete_Cac_Collection_Table SMU7_Discrete_Cac_Collection_Table;
+
+struct SMU7_Discrete_Cac_Verification_Table {
+ uint32_t VddcTotalPower;
+ uint32_t VddcLeakagePower;
+ uint32_t VddcConstantPower;
+ uint32_t VddcGfxDynamicPower;
+ uint32_t VddcUvdDynamicPower;
+ uint32_t VddcVceDynamicPower;
+ uint32_t VddcAcpDynamicPower;
+ uint32_t VddcPcieDynamicPower;
+ uint32_t VddcDceDynamicPower;
+ uint32_t VddcCurrent;
+ uint32_t VddcVoltage;
+ uint32_t VddciTotalPower;
+ uint32_t VddciLeakagePower;
+ uint32_t VddciConstantPower;
+ uint32_t VddciDynamicPower;
+ uint32_t Vddr1TotalPower;
+ uint32_t Vddr1LeakagePower;
+ uint32_t Vddr1ConstantPower;
+ uint32_t Vddr1DynamicPower;
+ uint32_t spare[4];
+ uint32_t temperature;
+};
+
+typedef struct SMU7_Discrete_Cac_Verification_Table SMU7_Discrete_Cac_Verification_Table;
+
+struct SMU7_Discrete_Pm_Status_Table {
+ int32_t T_meas_max;
+ int32_t T_meas_acc;
+ int32_t T_calc_max;
+ int32_t T_calc_acc;
+ uint32_t P_scalar_acc;
+ uint32_t P_calc_max;
+ uint32_t P_calc_acc;
+
+ uint32_t I_calc_max;
+ uint32_t I_calc_acc;
+ uint32_t I_calc_acc_vddci;
+ uint32_t V_calc_noload_acc;
+ uint32_t V_calc_load_acc;
+ uint32_t V_calc_noload_acc_vddci;
+ uint32_t P_meas_acc;
+ uint32_t V_meas_noload_acc;
+ uint32_t V_meas_load_acc;
+ uint32_t I_meas_acc;
+ uint32_t P_meas_acc_vddci;
+ uint32_t V_meas_noload_acc_vddci;
+ uint32_t V_meas_load_acc_vddci;
+ uint32_t I_meas_acc_vddci;
+
+ uint16_t Sclk_dpm_residency[8];
+ uint16_t Uvd_dpm_residency[8];
+ uint16_t Vce_dpm_residency[8];
+ uint16_t Mclk_dpm_residency[4];
+
+ uint32_t P_vddci_acc;
+ uint32_t P_vddr1_acc;
+ uint32_t P_nte1_acc;
+ uint32_t PkgPwr_max;
+ uint32_t PkgPwr_acc;
+ uint32_t MclkSwitchingTime_max;
+ uint32_t MclkSwitchingTime_acc;
+ uint32_t FanPwm_acc;
+ uint32_t FanRpm_acc;
+
+ uint32_t AccCnt;
+};
+
+typedef struct SMU7_Discrete_Pm_Status_Table SMU7_Discrete_Pm_Status_Table;
+
+#define SMU7_MAX_GFX_CU_COUNT 16
+
+struct SMU7_GfxCuPgScoreboard {
+ uint8_t Enabled;
+ uint8_t WaterfallUp;
+ uint8_t WaterfallDown;
+ uint8_t WaterfallLimit;
+ uint8_t CurrMaxCu;
+ uint8_t TargMaxCu;
+ uint8_t ClampMode;
+ uint8_t Active;
+ uint8_t MaxSupportedCu;
+ uint8_t MinSupportedCu;
+ uint8_t PendingGfxCuHostInterrupt;
+ uint8_t LastFilteredMaxCuInteger;
+ uint16_t FilteredMaxCu;
+ uint16_t FilteredMaxCuAlpha;
+ uint16_t FilterResetCount;
+ uint16_t FilterResetCountLimit;
+ uint8_t ForceCu;
+ uint8_t ForceCuCount;
+ uint8_t spare[2];
+};
+
+typedef struct SMU7_GfxCuPgScoreboard SMU7_GfxCuPgScoreboard;
+
+#define SMU7_SCLK_CAC 0x561
+#define SMU7_MCLK_CAC 0xF9
+#define SMU7_VCLK_CAC 0x2DE
+#define SMU7_DCLK_CAC 0x2DE
+#define SMU7_ECLK_CAC 0x25E
+#define SMU7_ACLK_CAC 0x25E
+#define SMU7_SAMCLK_CAC 0x25E
+#define SMU7_DISPCLK_CAC 0x100
+#define SMU7_CAC_CONSTANT 0x2EE3430
+#define SMU7_CAC_CONSTANT_SHIFT 18
+
+#define SMU7_VDDCI_MCLK_CONST 1765
+#define SMU7_VDDCI_MCLK_CONST_SHIFT 16
+#define SMU7_VDDCI_VDDCI_CONST 50958
+#define SMU7_VDDCI_VDDCI_CONST_SHIFT 14
+#define SMU7_VDDCI_CONST 11781
+#define SMU7_VDDCI_STROBE_PWR 1331
+
+#define SMU7_VDDR1_CONST 693
+#define SMU7_VDDR1_CAC_WEIGHT 20
+#define SMU7_VDDR1_CAC_WEIGHT_SHIFT 19
+#define SMU7_VDDR1_STROBE_PWR 512
+
+#define SMU7_AREA_COEFF_UVD 0xA78
+#define SMU7_AREA_COEFF_VCE 0x190A
+#define SMU7_AREA_COEFF_ACP 0x22D1
+#define SMU7_AREA_COEFF_SAMU 0x534
+
+#define SMU7_THERM_OUT_MODE_DISABLE 0x0
+#define SMU7_THERM_OUT_MODE_THERM_ONLY 0x1
+#define SMU7_THERM_OUT_MODE_THERM_VRHOT 0x2
+
+// DIDT Defines
+#define SQ_Enable_MASK 0x1
+#define SQ_IR_MASK 0x2
+#define SQ_PCC_MASK 0x4
+#define SQ_EDC_MASK 0x8
+
+#define TCP_Enable_MASK 0x100
+#define TCP_IR_MASK 0x200
+#define TCP_PCC_MASK 0x400
+#define TCP_EDC_MASK 0x800
+
+#define TD_Enable_MASK 0x10000
+#define TD_IR_MASK 0x20000
+#define TD_PCC_MASK 0x40000
+#define TD_EDC_MASK 0x80000
+
+#define DB_Enable_MASK 0x1000000
+#define DB_IR_MASK 0x2000000
+#define DB_PCC_MASK 0x4000000
+#define DB_EDC_MASK 0x8000000
+
+#define SQ_Enable_SHIFT 0
+#define SQ_IR_SHIFT 1
+#define SQ_PCC_SHIFT 2
+#define SQ_EDC_SHIFT 3
+
+#define TCP_Enable_SHIFT 8
+#define TCP_IR_SHIFT 9
+#define TCP_PCC_SHIFT 10
+#define TCP_EDC_SHIFT 11
+
+#define TD_Enable_SHIFT 16
+#define TD_IR_SHIFT 17
+#define TD_PCC_SHIFT 18
+#define TD_EDC_SHIFT 19
+
+#define DB_Enable_SHIFT 24
+#define DB_IR_SHIFT 25
+#define DB_PCC_SHIFT 26
+#define DB_EDC_SHIFT 27
+
+#pragma pack(pop)
+
+
+#endif
+
diff --git a/drivers/gpu/drm/amd/powerplay/inc/smu_ucode_xfer_cz.h b/drivers/gpu/drm/amd/powerplay/inc/smu_ucode_xfer_cz.h
index f8ba071..eb0f79f 100644
--- a/drivers/gpu/drm/amd/powerplay/inc/smu_ucode_xfer_cz.h
+++ b/drivers/gpu/drm/amd/powerplay/inc/smu_ucode_xfer_cz.h
@@ -1,3 +1,25 @@
+/*
+ * Copyright 2016 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
// CZ Ucode Loading Definitions
#ifndef SMU_UCODE_XFER_CZ_H
#define SMU_UCODE_XFER_CZ_H
diff --git a/drivers/gpu/drm/amd/powerplay/inc/smu_ucode_xfer_vi.h b/drivers/gpu/drm/amd/powerplay/inc/smu_ucode_xfer_vi.h
index c24a81e..880152c 100644
--- a/drivers/gpu/drm/amd/powerplay/inc/smu_ucode_xfer_vi.h
+++ b/drivers/gpu/drm/amd/powerplay/inc/smu_ucode_xfer_vi.h
@@ -44,6 +44,7 @@
#define UCODE_ID_IH_REG_RESTORE 11
#define UCODE_ID_VBIOS 12
#define UCODE_ID_MISC_METADATA 13
+#define UCODE_ID_SMU_SK 14
#define UCODE_ID_RLC_SCRATCH 32
#define UCODE_ID_RLC_SRM_ARAM 33
#define UCODE_ID_RLC_SRM_DRAM 34
diff --git a/drivers/gpu/drm/amd/powerplay/smumgr/Makefile b/drivers/gpu/drm/amd/powerplay/smumgr/Makefile
index 6c4ef13..f10fb64 100644
--- a/drivers/gpu/drm/amd/powerplay/smumgr/Makefile
+++ b/drivers/gpu/drm/amd/powerplay/smumgr/Makefile
@@ -2,7 +2,7 @@
# Makefile for the 'smu manager' sub-component of powerplay.
# It provides the smu management services for the driver.
-SMU_MGR = smumgr.o cz_smumgr.o tonga_smumgr.o fiji_smumgr.o
+SMU_MGR = smumgr.o cz_smumgr.o tonga_smumgr.o fiji_smumgr.o polaris10_smumgr.o
AMD_PP_SMUMGR = $(addprefix $(AMD_PP_PATH)/smumgr/,$(SMU_MGR))
diff --git a/drivers/gpu/drm/amd/powerplay/smumgr/cz_smumgr.c b/drivers/gpu/drm/amd/powerplay/smumgr/cz_smumgr.c
index ec222c6..da18f44 100644
--- a/drivers/gpu/drm/amd/powerplay/smumgr/cz_smumgr.c
+++ b/drivers/gpu/drm/amd/powerplay/smumgr/cz_smumgr.c
@@ -39,7 +39,7 @@
#define SIZE_ALIGN_32(x) (((x) + 31) / 32 * 32)
-static enum cz_scratch_entry firmware_list[] = {
+static const enum cz_scratch_entry firmware_list[] = {
CZ_SCRATCH_ENTRY_UCODE_ID_SDMA0,
CZ_SCRATCH_ENTRY_UCODE_ID_SDMA1,
CZ_SCRATCH_ENTRY_UCODE_ID_CP_CE,
diff --git a/drivers/gpu/drm/amd/powerplay/smumgr/fiji_smumgr.c b/drivers/gpu/drm/amd/powerplay/smumgr/fiji_smumgr.c
index cdbb9f8..673a75c 100644
--- a/drivers/gpu/drm/amd/powerplay/smumgr/fiji_smumgr.c
+++ b/drivers/gpu/drm/amd/powerplay/smumgr/fiji_smumgr.c
@@ -44,7 +44,7 @@
#define FIJI_SMC_SIZE 0x20000
-struct SMU73_Discrete_GraphicsLevel avfs_graphics_level[8] = {
+static const struct SMU73_Discrete_GraphicsLevel avfs_graphics_level[8] = {
/* Min Sclk pcie DeepSleep Activity CgSpll CgSpll spllSpread SpllSpread CcPwr CcPwr Sclk Display Enabled Enabled Voltage Power */
/* Voltage, Frequency, DpmLevel, DivId, Level, FuncCntl3, FuncCntl4, Spectrum, Spectrum2, DynRm, DynRm1 Did, Watermark, ForActivity, ForThrottle, UpHyst, DownHyst, DownHyst, Throttle */
{ 0x3c0fd047, 0x30750000, 0x00, 0x03, 0x1e00, 0x00200410, 0x87020000, 0x21680000, 0x0c000000, 0, 0, 0x16, 0x00, 0x01, 0x01, 0x00, 0x00, 0x00, 0x00 },
@@ -189,7 +189,7 @@
int fiji_program_jump_on_start(struct pp_smumgr *smumgr)
{
- static unsigned char data[] = { 0xE0, 0x00, 0x80, 0x40 };
+ static const unsigned char data[] = { 0xE0, 0x00, 0x80, 0x40 };
fiji_copy_bytes_to_smc(smumgr, 0x0, data, 4, sizeof(data) + 1);
@@ -665,7 +665,7 @@
{
int i, result = -1;
uint32_t reg, data;
- PWR_Command_Table *virus = PwrVirusTable;
+ const PWR_Command_Table *virus = PwrVirusTable;
struct fiji_smumgr *priv = (struct fiji_smumgr *)(smumgr->backend);
priv->avfs.AvfsBtcStatus = AVFS_LOAD_VIRUS;
diff --git a/drivers/gpu/drm/amd/powerplay/smumgr/polaris10_smumgr.c b/drivers/gpu/drm/amd/powerplay/smumgr/polaris10_smumgr.c
new file mode 100644
index 0000000..de618ea
--- /dev/null
+++ b/drivers/gpu/drm/amd/powerplay/smumgr/polaris10_smumgr.c
@@ -0,0 +1,984 @@
+/*
+ * Copyright 2015 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#include "smumgr.h"
+#include "smu74.h"
+#include "smu_ucode_xfer_vi.h"
+#include "polaris10_smumgr.h"
+#include "smu74_discrete.h"
+#include "smu/smu_7_1_3_d.h"
+#include "smu/smu_7_1_3_sh_mask.h"
+#include "gmc/gmc_8_1_d.h"
+#include "gmc/gmc_8_1_sh_mask.h"
+#include "oss/oss_3_0_d.h"
+#include "gca/gfx_8_0_d.h"
+#include "bif/bif_5_0_d.h"
+#include "bif/bif_5_0_sh_mask.h"
+#include "polaris10_pwrvirus.h"
+#include "ppatomctrl.h"
+#include "pp_debug.h"
+#include "cgs_common.h"
+
+#define POLARIS10_SMC_SIZE 0x20000
+#define VOLTAGE_SCALE 4
+
+/* Microcode file is stored in this buffer */
+#define BUFFER_SIZE 80000
+#define MAX_STRING_SIZE 15
+#define BUFFER_SIZETWO 131072 /* 128 *1024 */
+
+#define SMC_RAM_END 0x40000
+
+static const SMU74_Discrete_GraphicsLevel avfs_graphics_level_polaris10[8] = {
+ /* Min pcie DeepSleep Activity CgSpll CgSpll CcPwr CcPwr Sclk Enabled Enabled Voltage Power */
+ /* Voltage, DpmLevel, DivId, Level, FuncCntl3, FuncCntl4, DynRm, DynRm1 Did, Padding,ForActivity, ForThrottle, UpHyst, DownHyst, DownHyst, Throttle */
+ { 0x3c0fd047, 0x00, 0x03, 0x1e00, 0x00200410, 0x87020000, 0, 0, 0x16, 0, 0x01, 0x01, 0x00, 0x00, 0x00, 0x00, { 0x30750000, 0, 0, 0, 0, 0, 0, 0 } },
+ { 0xa00fd047, 0x01, 0x04, 0x1e00, 0x00800510, 0x87020000, 0, 0, 0x16, 0, 0x01, 0x01, 0x00, 0x00, 0x00, 0x00, { 0x409c0000, 0, 0, 0, 0, 0, 0, 0 } },
+ { 0x0410d047, 0x01, 0x00, 0x1e00, 0x00600410, 0x87020000, 0, 0, 0x0e, 0, 0x01, 0x01, 0x00, 0x00, 0x00, 0x00, { 0x50c30000, 0, 0, 0, 0, 0, 0, 0 } },
+ { 0x6810d047, 0x01, 0x00, 0x1e00, 0x00800410, 0x87020000, 0, 0, 0x0c, 0, 0x01, 0x01, 0x00, 0x00, 0x00, 0x00, { 0x60ea0000, 0, 0, 0, 0, 0, 0, 0 } },
+ { 0xcc10d047, 0x01, 0x00, 0x1e00, 0x00e00410, 0x87020000, 0, 0, 0x0c, 0, 0x01, 0x01, 0x00, 0x00, 0x00, 0x00, { 0xe8fd0000, 0, 0, 0, 0, 0, 0, 0 } },
+ { 0x3011d047, 0x01, 0x00, 0x1e00, 0x00400510, 0x87020000, 0, 0, 0x0c, 0, 0x01, 0x01, 0x00, 0x00, 0x00, 0x00, { 0x70110100, 0, 0, 0, 0, 0, 0, 0 } },
+ { 0x9411d047, 0x01, 0x00, 0x1e00, 0x00a00510, 0x87020000, 0, 0, 0x0c, 0, 0x01, 0x01, 0x00, 0x00, 0x00, 0x00, { 0xf8240100, 0, 0, 0, 0, 0, 0, 0 } },
+ { 0xf811d047, 0x01, 0x00, 0x1e00, 0x00000610, 0x87020000, 0, 0, 0x0c, 0, 0x01, 0x01, 0x00, 0x00, 0x00, 0x00, { 0x80380100, 0, 0, 0, 0, 0, 0, 0 } }
+};
+
+static const SMU74_Discrete_MemoryLevel avfs_memory_level_polaris10 =
+ {0x50140000, 0x50140000, 0x00320000, 0x00, 0x00,
+ 0x00, 0x10, 0x00, 0x00, 0x00, 0x00, 0x0000, 0x00, 0x00};
+
+/**
+* Set the address for reading/writing the SMC SRAM space.
+* @param smumgr the address of the powerplay hardware manager.
+* @param smcAddress the address in the SMC RAM to access.
+*/
+static int polaris10_set_smc_sram_address(struct pp_smumgr *smumgr, uint32_t smc_addr, uint32_t limit)
+{
+ PP_ASSERT_WITH_CODE((0 == (3 & smc_addr)), "SMC address must be 4 byte aligned.", return -EINVAL);
+ PP_ASSERT_WITH_CODE((limit > (smc_addr + 3)), "SMC addr is beyond the SMC RAM area.", return -EINVAL);
+
+ cgs_write_register(smumgr->device, mmSMC_IND_INDEX_11, smc_addr);
+ SMUM_WRITE_FIELD(smumgr->device, SMC_IND_ACCESS_CNTL, AUTO_INCREMENT_IND_11, 0);
+
+ return 0;
+}
+
+/**
+* Copy bytes from SMC RAM space into driver memory.
+*
+* @param smumgr the address of the powerplay SMU manager.
+* @param smc_start_address the start address in the SMC RAM to copy bytes from
+* @param src the byte array to copy the bytes to.
+* @param byte_count the number of bytes to copy.
+*/
+int polaris10_copy_bytes_from_smc(struct pp_smumgr *smumgr, uint32_t smc_start_address, uint32_t *dest, uint32_t byte_count, uint32_t limit)
+{
+ uint32_t data;
+ uint32_t addr;
+ uint8_t *dest_byte;
+ uint8_t i, data_byte[4] = {0};
+ uint32_t *pdata = (uint32_t *)&data_byte;
+
+ PP_ASSERT_WITH_CODE((0 == (3 & smc_start_address)), "SMC address must be 4 byte aligned.", return -1;);
+ PP_ASSERT_WITH_CODE((limit > (smc_start_address + byte_count)), "SMC address is beyond the SMC RAM area.", return -1);
+
+ addr = smc_start_address;
+
+ while (byte_count >= 4) {
+ polaris10_read_smc_sram_dword(smumgr, addr, &data, limit);
+
+ *dest = PP_SMC_TO_HOST_UL(data);
+
+ dest += 1;
+ byte_count -= 4;
+ addr += 4;
+ }
+
+ if (byte_count) {
+ polaris10_read_smc_sram_dword(smumgr, addr, &data, limit);
+ *pdata = PP_SMC_TO_HOST_UL(data);
+ /* Cast dest into byte type in dest_byte. This way, we don't overflow if the allocated memory is not 4-byte aligned. */
+ dest_byte = (uint8_t *)dest;
+ for (i = 0; i < byte_count; i++)
+ dest_byte[i] = data_byte[i];
+ }
+
+ return 0;
+}
+
+/**
+* Copy bytes from an array into the SMC RAM space.
+*
+* @param pSmuMgr the address of the powerplay SMU manager.
+* @param smc_start_address the start address in the SMC RAM to copy bytes to.
+* @param src the byte array to copy the bytes from.
+* @param byte_count the number of bytes to copy.
+*/
+int polaris10_copy_bytes_to_smc(struct pp_smumgr *smumgr, uint32_t smc_start_address,
+ const uint8_t *src, uint32_t byte_count, uint32_t limit)
+{
+ int result;
+ uint32_t data = 0;
+ uint32_t original_data;
+ uint32_t addr = 0;
+ uint32_t extra_shift;
+
+ PP_ASSERT_WITH_CODE((0 == (3 & smc_start_address)), "SMC address must be 4 byte aligned.", return -1);
+ PP_ASSERT_WITH_CODE((limit > (smc_start_address + byte_count)), "SMC address is beyond the SMC RAM area.", return -1);
+
+ addr = smc_start_address;
+
+ while (byte_count >= 4) {
+ /* Bytes are written into the SMC addres space with the MSB first. */
+ data = src[0] * 0x1000000 + src[1] * 0x10000 + src[2] * 0x100 + src[3];
+
+ result = polaris10_set_smc_sram_address(smumgr, addr, limit);
+
+ if (0 != result)
+ return result;
+
+ cgs_write_register(smumgr->device, mmSMC_IND_DATA_11, data);
+
+ src += 4;
+ byte_count -= 4;
+ addr += 4;
+ }
+
+ if (0 != byte_count) {
+
+ data = 0;
+
+ result = polaris10_set_smc_sram_address(smumgr, addr, limit);
+
+ if (0 != result)
+ return result;
+
+
+ original_data = cgs_read_register(smumgr->device, mmSMC_IND_DATA_11);
+
+ extra_shift = 8 * (4 - byte_count);
+
+ while (byte_count > 0) {
+ /* Bytes are written into the SMC addres space with the MSB first. */
+ data = (0x100 * data) + *src++;
+ byte_count--;
+ }
+
+ data <<= extra_shift;
+
+ data |= (original_data & ~((~0UL) << extra_shift));
+
+ result = polaris10_set_smc_sram_address(smumgr, addr, limit);
+
+ if (0 != result)
+ return result;
+
+ cgs_write_register(smumgr->device, mmSMC_IND_DATA_11, data);
+ }
+
+ return 0;
+}
+
+
+static int polaris10_program_jump_on_start(struct pp_smumgr *smumgr)
+{
+ static const unsigned char data[4] = { 0xE0, 0x00, 0x80, 0x40 };
+
+ polaris10_copy_bytes_to_smc(smumgr, 0x0, data, 4, sizeof(data)+1);
+
+ return 0;
+}
+
+/**
+* Return if the SMC is currently running.
+*
+* @param smumgr the address of the powerplay hardware manager.
+*/
+bool polaris10_is_smc_ram_running(struct pp_smumgr *smumgr)
+{
+ return ((0 == SMUM_READ_VFPF_INDIRECT_FIELD(smumgr->device, CGS_IND_REG__SMC, SMC_SYSCON_CLOCK_CNTL_0, ck_disable))
+ && (0x20100 <= cgs_read_ind_register(smumgr->device, CGS_IND_REG__SMC, ixSMC_PC_C)));
+}
+
+/**
+* Send a message to the SMC, and wait for its response.
+*
+* @param smumgr the address of the powerplay hardware manager.
+* @param msg the message to send.
+* @return The response that came from the SMC.
+*/
+int polaris10_send_msg_to_smc(struct pp_smumgr *smumgr, uint16_t msg)
+{
+ if (!polaris10_is_smc_ram_running(smumgr))
+ return -1;
+
+ SMUM_WAIT_FIELD_UNEQUAL(smumgr, SMC_RESP_0, SMC_RESP, 0);
+
+ if (1 != SMUM_READ_FIELD(smumgr->device, SMC_RESP_0, SMC_RESP))
+ printk("Failed to send Previous Message.\n");
+
+
+ cgs_write_register(smumgr->device, mmSMC_MESSAGE_0, msg);
+
+ SMUM_WAIT_FIELD_UNEQUAL(smumgr, SMC_RESP_0, SMC_RESP, 0);
+
+ if (1 != SMUM_READ_FIELD(smumgr->device, SMC_RESP_0, SMC_RESP))
+ printk("Failed to send Message.\n");
+
+ return 0;
+}
+
+
+/**
+* Send a message to the SMC, and do not wait for its response.
+*
+* @param smumgr the address of the powerplay hardware manager.
+* @param msg the message to send.
+* @return Always return 0.
+*/
+int polaris10_send_msg_to_smc_without_waiting(struct pp_smumgr *smumgr, uint16_t msg)
+{
+ cgs_write_register(smumgr->device, mmSMC_MESSAGE_0, msg);
+
+ return 0;
+}
+
+/**
+* Send a message to the SMC with parameter
+*
+* @param smumgr: the address of the powerplay hardware manager.
+* @param msg: the message to send.
+* @param parameter: the parameter to send
+* @return The response that came from the SMC.
+*/
+int polaris10_send_msg_to_smc_with_parameter(struct pp_smumgr *smumgr, uint16_t msg, uint32_t parameter)
+{
+ if (!polaris10_is_smc_ram_running(smumgr)) {
+ return -1;
+ }
+
+ SMUM_WAIT_FIELD_UNEQUAL(smumgr, SMC_RESP_0, SMC_RESP, 0);
+
+ cgs_write_register(smumgr->device, mmSMC_MSG_ARG_0, parameter);
+
+ return polaris10_send_msg_to_smc(smumgr, msg);
+}
+
+
+/**
+* Send a message to the SMC with parameter, do not wait for response
+*
+* @param smumgr: the address of the powerplay hardware manager.
+* @param msg: the message to send.
+* @param parameter: the parameter to send
+* @return The response that came from the SMC.
+*/
+int polaris10_send_msg_to_smc_with_parameter_without_waiting(struct pp_smumgr *smumgr, uint16_t msg, uint32_t parameter)
+{
+ cgs_write_register(smumgr->device, mmSMC_MSG_ARG_0, parameter);
+
+ return polaris10_send_msg_to_smc_without_waiting(smumgr, msg);
+}
+
+int polaris10_send_msg_to_smc_offset(struct pp_smumgr *smumgr)
+{
+ cgs_write_register(smumgr->device, mmSMC_MSG_ARG_0, 0x20000);
+
+ cgs_write_register(smumgr->device, mmSMC_MESSAGE_0, PPSMC_MSG_Test);
+
+ SMUM_WAIT_FIELD_UNEQUAL(smumgr, SMC_RESP_0, SMC_RESP, 0);
+
+ if (1 != SMUM_READ_FIELD(smumgr->device, SMC_RESP_0, SMC_RESP))
+ printk("Failed to send Message.\n");
+
+ return 0;
+}
+
+/**
+* Wait until the SMC is doing nithing. Doing nothing means that the SMC is either turned off or it is sitting on the STOP instruction.
+*
+* @param smumgr the address of the powerplay hardware manager.
+* @param msg the message to send.
+* @return The response that came from the SMC.
+*/
+int polaris10_wait_for_smc_inactive(struct pp_smumgr *smumgr)
+{
+ /* If the SMC is not even on it qualifies as inactive. */
+ if (!polaris10_is_smc_ram_running(smumgr))
+ return -1;
+
+ SMUM_WAIT_VFPF_INDIRECT_FIELD(smumgr, SMC_IND, SMC_SYSCON_CLOCK_CNTL_0, cken, 0);
+ return 0;
+}
+
+
+/**
+* Upload the SMC firmware to the SMC microcontroller.
+*
+* @param smumgr the address of the powerplay hardware manager.
+* @param pFirmware the data structure containing the various sections of the firmware.
+*/
+static int polaris10_upload_smc_firmware_data(struct pp_smumgr *smumgr, uint32_t length, uint32_t *src, uint32_t limit)
+{
+ uint32_t byte_count = length;
+
+ PP_ASSERT_WITH_CODE((limit >= byte_count), "SMC address is beyond the SMC RAM area.", return -1);
+
+ cgs_write_register(smumgr->device, mmSMC_IND_INDEX_11, 0x20000);
+ SMUM_WRITE_FIELD(smumgr->device, SMC_IND_ACCESS_CNTL, AUTO_INCREMENT_IND_11, 1);
+
+ for (; byte_count >= 4; byte_count -= 4)
+ cgs_write_register(smumgr->device, mmSMC_IND_DATA_11, *src++);
+
+ SMUM_WRITE_FIELD(smumgr->device, SMC_IND_ACCESS_CNTL, AUTO_INCREMENT_IND_11, 0);
+
+ PP_ASSERT_WITH_CODE((0 == byte_count), "SMC size must be dividable by 4.", return -1);
+
+ return 0;
+}
+
+static enum cgs_ucode_id polaris10_convert_fw_type_to_cgs(uint32_t fw_type)
+{
+ enum cgs_ucode_id result = CGS_UCODE_ID_MAXIMUM;
+
+ switch (fw_type) {
+ case UCODE_ID_SMU:
+ result = CGS_UCODE_ID_SMU;
+ break;
+ case UCODE_ID_SMU_SK:
+ result = CGS_UCODE_ID_SMU_SK;
+ break;
+ case UCODE_ID_SDMA0:
+ result = CGS_UCODE_ID_SDMA0;
+ break;
+ case UCODE_ID_SDMA1:
+ result = CGS_UCODE_ID_SDMA1;
+ break;
+ case UCODE_ID_CP_CE:
+ result = CGS_UCODE_ID_CP_CE;
+ break;
+ case UCODE_ID_CP_PFP:
+ result = CGS_UCODE_ID_CP_PFP;
+ break;
+ case UCODE_ID_CP_ME:
+ result = CGS_UCODE_ID_CP_ME;
+ break;
+ case UCODE_ID_CP_MEC:
+ result = CGS_UCODE_ID_CP_MEC;
+ break;
+ case UCODE_ID_CP_MEC_JT1:
+ result = CGS_UCODE_ID_CP_MEC_JT1;
+ break;
+ case UCODE_ID_CP_MEC_JT2:
+ result = CGS_UCODE_ID_CP_MEC_JT2;
+ break;
+ case UCODE_ID_RLC_G:
+ result = CGS_UCODE_ID_RLC_G;
+ break;
+ default:
+ break;
+ }
+
+ return result;
+}
+
+static int polaris10_upload_smu_firmware_image(struct pp_smumgr *smumgr)
+{
+ int result = 0;
+ struct polaris10_smumgr *smu_data = (struct polaris10_smumgr *)(smumgr->backend);
+
+ struct cgs_firmware_info info = {0};
+
+ if (smu_data->security_hard_key == 1)
+ cgs_get_firmware_info(smumgr->device,
+ polaris10_convert_fw_type_to_cgs(UCODE_ID_SMU), &info);
+ else
+ cgs_get_firmware_info(smumgr->device,
+ polaris10_convert_fw_type_to_cgs(UCODE_ID_SMU_SK), &info);
+
+ /* TO DO cgs_init_samu_load_smu(smumgr->device, (uint32_t *)info.kptr, info.image_size, smu_data->post_initial_boot);*/
+ result = polaris10_upload_smc_firmware_data(smumgr, info.image_size, (uint32_t *)info.kptr, POLARIS10_SMC_SIZE);
+
+ return result;
+}
+
+/**
+* Read a 32bit value from the SMC SRAM space.
+* ALL PARAMETERS ARE IN HOST BYTE ORDER.
+* @param smumgr the address of the powerplay hardware manager.
+* @param smcAddress the address in the SMC RAM to access.
+* @param value and output parameter for the data read from the SMC SRAM.
+*/
+int polaris10_read_smc_sram_dword(struct pp_smumgr *smumgr, uint32_t smc_addr, uint32_t *value, uint32_t limit)
+{
+ int result;
+
+ result = polaris10_set_smc_sram_address(smumgr, smc_addr, limit);
+
+ if (result)
+ return result;
+
+ *value = cgs_read_register(smumgr->device, mmSMC_IND_DATA_11);
+ return 0;
+}
+
+/**
+* Write a 32bit value to the SMC SRAM space.
+* ALL PARAMETERS ARE IN HOST BYTE ORDER.
+* @param smumgr the address of the powerplay hardware manager.
+* @param smc_addr the address in the SMC RAM to access.
+* @param value to write to the SMC SRAM.
+*/
+int polaris10_write_smc_sram_dword(struct pp_smumgr *smumgr, uint32_t smc_addr, uint32_t value, uint32_t limit)
+{
+ int result;
+
+ result = polaris10_set_smc_sram_address(smumgr, smc_addr, limit);
+
+ if (result)
+ return result;
+
+ cgs_write_register(smumgr->device, mmSMC_IND_DATA_11, value);
+
+ return 0;
+}
+
+
+int polaris10_smu_fini(struct pp_smumgr *smumgr)
+{
+ if (smumgr->backend) {
+ kfree(smumgr->backend);
+ smumgr->backend = NULL;
+ }
+ return 0;
+}
+
+/* Convert the firmware type to SMU type mask. For MEC, we need to check all MEC related type */
+static uint32_t polaris10_get_mask_for_firmware_type(uint32_t fw_type)
+{
+ uint32_t result = 0;
+
+ switch (fw_type) {
+ case UCODE_ID_SDMA0:
+ result = UCODE_ID_SDMA0_MASK;
+ break;
+ case UCODE_ID_SDMA1:
+ result = UCODE_ID_SDMA1_MASK;
+ break;
+ case UCODE_ID_CP_CE:
+ result = UCODE_ID_CP_CE_MASK;
+ break;
+ case UCODE_ID_CP_PFP:
+ result = UCODE_ID_CP_PFP_MASK;
+ break;
+ case UCODE_ID_CP_ME:
+ result = UCODE_ID_CP_ME_MASK;
+ break;
+ case UCODE_ID_CP_MEC_JT1:
+ case UCODE_ID_CP_MEC_JT2:
+ result = UCODE_ID_CP_MEC_MASK;
+ break;
+ case UCODE_ID_RLC_G:
+ result = UCODE_ID_RLC_G_MASK;
+ break;
+ default:
+ printk("UCode type is out of range! \n");
+ result = 0;
+ }
+
+ return result;
+}
+
+/* Populate one firmware image to the data structure */
+
+static int polaris10_populate_single_firmware_entry(struct pp_smumgr *smumgr,
+ uint32_t fw_type,
+ struct SMU_Entry *entry)
+{
+ int result = 0;
+ struct cgs_firmware_info info = {0};
+
+ result = cgs_get_firmware_info(smumgr->device,
+ polaris10_convert_fw_type_to_cgs(fw_type),
+ &info);
+
+ if (!result) {
+ entry->version = info.version;
+ entry->id = (uint16_t)fw_type;
+ entry->image_addr_high = smu_upper_32_bits(info.mc_addr);
+ entry->image_addr_low = smu_lower_32_bits(info.mc_addr);
+ entry->meta_data_addr_high = 0;
+ entry->meta_data_addr_low = 0;
+ entry->data_size_byte = info.image_size;
+ entry->num_register_entries = 0;
+ }
+
+ if (fw_type == UCODE_ID_RLC_G)
+ entry->flags = 1;
+ else
+ entry->flags = 0;
+
+ return 0;
+}
+
+static int polaris10_request_smu_load_fw(struct pp_smumgr *smumgr)
+{
+ struct polaris10_smumgr *smu_data = (struct polaris10_smumgr *)(smumgr->backend);
+ uint32_t fw_to_load;
+
+ int result = 0;
+ struct SMU_DRAMData_TOC *toc;
+
+ if (!smumgr->reload_fw) {
+ printk(KERN_INFO "[ powerplay ] skip reloading...\n");
+ return 0;
+ }
+
+ if (smu_data->soft_regs_start)
+ cgs_write_ind_register(smumgr->device, CGS_IND_REG__SMC,
+ smu_data->soft_regs_start + offsetof(SMU74_SoftRegisters, UcodeLoadStatus),
+ 0x0);
+
+ polaris10_send_msg_to_smc_with_parameter(smumgr, PPSMC_MSG_SMU_DRAM_ADDR_HI, smu_data->smu_buffer.mc_addr_high);
+ polaris10_send_msg_to_smc_with_parameter(smumgr, PPSMC_MSG_SMU_DRAM_ADDR_LO, smu_data->smu_buffer.mc_addr_low);
+
+ toc = (struct SMU_DRAMData_TOC *)smu_data->header;
+ toc->num_entries = 0;
+ toc->structure_version = 1;
+
+ PP_ASSERT_WITH_CODE(0 == polaris10_populate_single_firmware_entry(smumgr, UCODE_ID_RLC_G, &toc->entry[toc->num_entries++]), "Failed to Get Firmware Entry.", return -1);
+ PP_ASSERT_WITH_CODE(0 == polaris10_populate_single_firmware_entry(smumgr, UCODE_ID_CP_CE, &toc->entry[toc->num_entries++]), "Failed to Get Firmware Entry.", return -1);
+ PP_ASSERT_WITH_CODE(0 == polaris10_populate_single_firmware_entry(smumgr, UCODE_ID_CP_PFP, &toc->entry[toc->num_entries++]), "Failed to Get Firmware Entry.", return -1);
+ PP_ASSERT_WITH_CODE(0 == polaris10_populate_single_firmware_entry(smumgr, UCODE_ID_CP_ME, &toc->entry[toc->num_entries++]), "Failed to Get Firmware Entry.", return -1);
+ PP_ASSERT_WITH_CODE(0 == polaris10_populate_single_firmware_entry(smumgr, UCODE_ID_CP_MEC, &toc->entry[toc->num_entries++]), "Failed to Get Firmware Entry.", return -1);
+ PP_ASSERT_WITH_CODE(0 == polaris10_populate_single_firmware_entry(smumgr, UCODE_ID_CP_MEC_JT1, &toc->entry[toc->num_entries++]), "Failed to Get Firmware Entry.", return -1);
+ PP_ASSERT_WITH_CODE(0 == polaris10_populate_single_firmware_entry(smumgr, UCODE_ID_CP_MEC_JT2, &toc->entry[toc->num_entries++]), "Failed to Get Firmware Entry.", return -1);
+ PP_ASSERT_WITH_CODE(0 == polaris10_populate_single_firmware_entry(smumgr, UCODE_ID_SDMA0, &toc->entry[toc->num_entries++]), "Failed to Get Firmware Entry.", return -1);
+ PP_ASSERT_WITH_CODE(0 == polaris10_populate_single_firmware_entry(smumgr, UCODE_ID_SDMA1, &toc->entry[toc->num_entries++]), "Failed to Get Firmware Entry.", return -1);
+
+ polaris10_send_msg_to_smc_with_parameter(smumgr, PPSMC_MSG_DRV_DRAM_ADDR_HI, smu_data->header_buffer.mc_addr_high);
+ polaris10_send_msg_to_smc_with_parameter(smumgr, PPSMC_MSG_DRV_DRAM_ADDR_LO, smu_data->header_buffer.mc_addr_low);
+
+ fw_to_load = UCODE_ID_RLC_G_MASK
+ + UCODE_ID_SDMA0_MASK
+ + UCODE_ID_SDMA1_MASK
+ + UCODE_ID_CP_CE_MASK
+ + UCODE_ID_CP_ME_MASK
+ + UCODE_ID_CP_PFP_MASK
+ + UCODE_ID_CP_MEC_MASK;
+
+ if (polaris10_send_msg_to_smc_with_parameter(smumgr, PPSMC_MSG_LoadUcodes, fw_to_load))
+ printk(KERN_ERR "Fail to Request SMU Load uCode");
+
+ return result;
+}
+
+/* Check if the FW has been loaded, SMU will not return if loading has not finished. */
+static int polaris10_check_fw_load_finish(struct pp_smumgr *smumgr, uint32_t fw_type)
+{
+ struct polaris10_smumgr *smu_data = (struct polaris10_smumgr *)(smumgr->backend);
+ uint32_t fw_mask = polaris10_get_mask_for_firmware_type(fw_type);
+ uint32_t ret;
+ /* Check SOFT_REGISTERS_TABLE_28.UcodeLoadStatus */
+ ret = smum_wait_on_indirect_register(smumgr, mmSMC_IND_INDEX_11,
+ smu_data->soft_regs_start + offsetof(SMU74_SoftRegisters, UcodeLoadStatus),
+ fw_mask, fw_mask);
+
+ return ret;
+}
+
+static int polaris10_reload_firmware(struct pp_smumgr *smumgr)
+{
+ return smumgr->smumgr_funcs->start_smu(smumgr);
+}
+
+static int polaris10_setup_pwr_virus(struct pp_smumgr *smumgr)
+{
+ int i;
+ int result = -1;
+ uint32_t reg, data;
+
+ const PWR_Command_Table *pvirus = pwr_virus_table;
+ struct polaris10_smumgr *smu_data = (struct polaris10_smumgr *)(smumgr->backend);
+
+
+ for (i = 0; i < PWR_VIRUS_TABLE_SIZE; i++) {
+ switch (pvirus->command) {
+ case PwrCmdWrite:
+ reg = pvirus->reg;
+ data = pvirus->data;
+ cgs_write_register(smumgr->device, reg, data);
+ break;
+
+ case PwrCmdEnd:
+ result = 0;
+ break;
+
+ default:
+ printk("Table Exit with Invalid Command!");
+ smu_data->avfs.avfs_btc_status = AVFS_BTC_VIRUS_FAIL;
+ result = -1;
+ break;
+ }
+ pvirus++;
+ }
+
+ return result;
+}
+
+static int polaris10_perform_btc(struct pp_smumgr *smumgr)
+{
+ int result = 0;
+ struct polaris10_smumgr *smu_data = (struct polaris10_smumgr *)(smumgr->backend);
+
+ if (0 != smu_data->avfs.avfs_btc_param) {
+ if (0 != polaris10_send_msg_to_smc_with_parameter(smumgr, PPSMC_MSG_PerformBtc, smu_data->avfs.avfs_btc_param)) {
+ printk("[AVFS][SmuPolaris10_PerformBtc] PerformBTC SMU msg failed");
+ result = -1;
+ }
+ }
+ if (smu_data->avfs.avfs_btc_param > 1) {
+ /* Soft-Reset to reset the engine before loading uCode */
+ /* halt */
+ cgs_write_register(smumgr->device, mmCP_MEC_CNTL, 0x50000000);
+ /* reset everything */
+ cgs_write_register(smumgr->device, mmGRBM_SOFT_RESET, 0xffffffff);
+ cgs_write_register(smumgr->device, mmGRBM_SOFT_RESET, 0);
+ }
+ return result;
+}
+
+
+int polaris10_setup_graphics_level_structure(struct pp_smumgr *smumgr)
+{
+ uint32_t vr_config;
+ uint32_t dpm_table_start;
+
+ uint16_t u16_boot_mvdd;
+ uint32_t graphics_level_address, vr_config_address, graphics_level_size;
+
+ graphics_level_size = sizeof(avfs_graphics_level_polaris10);
+ u16_boot_mvdd = PP_HOST_TO_SMC_US(1300 * VOLTAGE_SCALE);
+
+ PP_ASSERT_WITH_CODE(0 == polaris10_read_smc_sram_dword(smumgr,
+ SMU7_FIRMWARE_HEADER_LOCATION + offsetof(SMU74_Firmware_Header, DpmTable),
+ &dpm_table_start, 0x40000),
+ "[AVFS][Polaris10_SetupGfxLvlStruct] SMU could not communicate starting address of DPM table",
+ return -1);
+
+ /* Default value for VRConfig = VR_MERGED_WITH_VDDC + VR_STATIC_VOLTAGE(VDDCI) */
+ vr_config = 0x01000500; /* Real value:0x50001 */
+
+ vr_config_address = dpm_table_start + offsetof(SMU74_Discrete_DpmTable, VRConfig);
+
+ PP_ASSERT_WITH_CODE(0 == polaris10_copy_bytes_to_smc(smumgr, vr_config_address,
+ (uint8_t *)&vr_config, sizeof(uint32_t), 0x40000),
+ "[AVFS][Polaris10_SetupGfxLvlStruct] Problems copying VRConfig value over to SMC",
+ return -1);
+
+ graphics_level_address = dpm_table_start + offsetof(SMU74_Discrete_DpmTable, GraphicsLevel);
+
+ PP_ASSERT_WITH_CODE(0 == polaris10_copy_bytes_to_smc(smumgr, graphics_level_address,
+ (uint8_t *)(&avfs_graphics_level_polaris10),
+ graphics_level_size, 0x40000),
+ "[AVFS][Polaris10_SetupGfxLvlStruct] Copying of SCLK DPM table failed!",
+ return -1);
+
+ graphics_level_address = dpm_table_start + offsetof(SMU74_Discrete_DpmTable, MemoryLevel);
+
+ PP_ASSERT_WITH_CODE(0 == polaris10_copy_bytes_to_smc(smumgr, graphics_level_address,
+ (uint8_t *)(&avfs_memory_level_polaris10), sizeof(avfs_memory_level_polaris10), 0x40000),
+ "[AVFS][Polaris10_SetupGfxLvlStruct] Copying of MCLK DPM table failed!",
+ return -1);
+
+ /* MVDD Boot value - neccessary for getting rid of the hang that occurs during Mclk DPM enablement */
+
+ graphics_level_address = dpm_table_start + offsetof(SMU74_Discrete_DpmTable, BootMVdd);
+
+ PP_ASSERT_WITH_CODE(0 == polaris10_copy_bytes_to_smc(smumgr, graphics_level_address,
+ (uint8_t *)(&u16_boot_mvdd), sizeof(u16_boot_mvdd), 0x40000),
+ "[AVFS][Polaris10_SetupGfxLvlStruct] Copying of DPM table failed!",
+ return -1);
+
+ return 0;
+}
+
+int polaris10_avfs_event_mgr(struct pp_smumgr *smumgr, bool SMU_VFT_INTACT)
+{
+ struct polaris10_smumgr *smu_data = (struct polaris10_smumgr *)(smumgr->backend);
+
+ switch (smu_data->avfs.avfs_btc_status) {
+ case AVFS_BTC_COMPLETED_PREVIOUSLY:
+ break;
+
+ case AVFS_BTC_BOOT: /* Cold Boot State - Post SMU Start */
+
+ smu_data->avfs.avfs_btc_status = AVFS_BTC_DPMTABLESETUP_FAILED;
+ PP_ASSERT_WITH_CODE(0 == polaris10_setup_graphics_level_structure(smumgr),
+ "[AVFS][Polaris10_AVFSEventMgr] Could not Copy Graphics Level table over to SMU",
+ return -1);
+
+ if (smu_data->avfs.avfs_btc_param > 1) {
+ printk("[AVFS][Polaris10_AVFSEventMgr] AC BTC has not been successfully verified on Fiji. There may be in this setting.");
+ smu_data->avfs.avfs_btc_status = AVFS_BTC_VIRUS_FAIL;
+ PP_ASSERT_WITH_CODE(-1 == polaris10_setup_pwr_virus(smumgr),
+ "[AVFS][Polaris10_AVFSEventMgr] Could not setup Pwr Virus for AVFS ",
+ return -1);
+ }
+
+ smu_data->avfs.avfs_btc_status = AVFS_BTC_FAILED;
+ PP_ASSERT_WITH_CODE(0 == polaris10_perform_btc(smumgr),
+ "[AVFS][Polaris10_AVFSEventMgr] Failure at SmuPolaris10_PerformBTC. AVFS Disabled",
+ return -1);
+
+ break;
+
+ case AVFS_BTC_DISABLED:
+ case AVFS_BTC_NOTSUPPORTED:
+ break;
+
+ default:
+ printk("[AVFS] Something is broken. See log!");
+ break;
+ }
+
+ return 0;
+}
+
+static int polaris10_start_smu_in_protection_mode(struct pp_smumgr *smumgr)
+{
+ int result = 0;
+
+ /* Wait for smc boot up */
+ /* SMUM_WAIT_VFPF_INDIRECT_FIELD_UNEQUAL(smumgr, SMC_IND, RCU_UC_EVENTS, boot_seq_done, 0) */
+
+ /* Assert reset */
+ SMUM_WRITE_VFPF_INDIRECT_FIELD(smumgr->device, CGS_IND_REG__SMC,
+ SMC_SYSCON_RESET_CNTL, rst_reg, 1);
+
+ result = polaris10_upload_smu_firmware_image(smumgr);
+ if (result != 0)
+ return result;
+
+ /* Clear status */
+ cgs_write_ind_register(smumgr->device, CGS_IND_REG__SMC, ixSMU_STATUS, 0);
+
+ SMUM_WRITE_VFPF_INDIRECT_FIELD(smumgr->device, CGS_IND_REG__SMC,
+ SMC_SYSCON_CLOCK_CNTL_0, ck_disable, 0);
+
+ /* De-assert reset */
+ SMUM_WRITE_VFPF_INDIRECT_FIELD(smumgr->device, CGS_IND_REG__SMC,
+ SMC_SYSCON_RESET_CNTL, rst_reg, 0);
+
+
+ SMUM_WAIT_VFPF_INDIRECT_FIELD(smumgr, SMC_IND, RCU_UC_EVENTS, INTERRUPTS_ENABLED, 1);
+
+
+ /* Call Test SMU message with 0x20000 offset to trigger SMU start */
+ polaris10_send_msg_to_smc_offset(smumgr);
+
+ /* Wait done bit to be set */
+ /* Check pass/failed indicator */
+
+ SMUM_WAIT_VFPF_INDIRECT_FIELD_UNEQUAL(smumgr, SMC_IND, SMU_STATUS, SMU_DONE, 0);
+
+ if (1 != SMUM_READ_VFPF_INDIRECT_FIELD(smumgr->device, CGS_IND_REG__SMC,
+ SMU_STATUS, SMU_PASS))
+ PP_ASSERT_WITH_CODE(false, "SMU Firmware start failed!", return -1);
+
+ cgs_write_ind_register(smumgr->device, CGS_IND_REG__SMC, ixFIRMWARE_FLAGS, 0);
+
+ SMUM_WRITE_VFPF_INDIRECT_FIELD(smumgr->device, CGS_IND_REG__SMC,
+ SMC_SYSCON_RESET_CNTL, rst_reg, 1);
+
+ SMUM_WRITE_VFPF_INDIRECT_FIELD(smumgr->device, CGS_IND_REG__SMC,
+ SMC_SYSCON_RESET_CNTL, rst_reg, 0);
+
+ /* Wait for firmware to initialize */
+ SMUM_WAIT_VFPF_INDIRECT_FIELD(smumgr, SMC_IND, FIRMWARE_FLAGS, INTERRUPTS_ENABLED, 1);
+
+ return result;
+}
+
+static int polaris10_start_smu_in_non_protection_mode(struct pp_smumgr *smumgr)
+{
+ int result = 0;
+
+ /* wait for smc boot up */
+ SMUM_WAIT_VFPF_INDIRECT_FIELD_UNEQUAL(smumgr, SMC_IND, RCU_UC_EVENTS, boot_seq_done, 0);
+
+ /* Clear firmware interrupt enable flag */
+ /* SMUM_WRITE_VFPF_INDIRECT_FIELD(pSmuMgr, SMC_IND, SMC_SYSCON_MISC_CNTL, pre_fetcher_en, 1); */
+ cgs_write_ind_register(smumgr->device, CGS_IND_REG__SMC,
+ ixFIRMWARE_FLAGS, 0);
+
+ SMUM_WRITE_VFPF_INDIRECT_FIELD(smumgr->device, CGS_IND_REG__SMC,
+ SMC_SYSCON_RESET_CNTL,
+ rst_reg, 1);
+
+ result = polaris10_upload_smu_firmware_image(smumgr);
+ if (result != 0)
+ return result;
+
+ /* Set smc instruct start point at 0x0 */
+ polaris10_program_jump_on_start(smumgr);
+
+ SMUM_WRITE_VFPF_INDIRECT_FIELD(smumgr->device, CGS_IND_REG__SMC,
+ SMC_SYSCON_CLOCK_CNTL_0, ck_disable, 0);
+
+ SMUM_WRITE_VFPF_INDIRECT_FIELD(smumgr->device, CGS_IND_REG__SMC,
+ SMC_SYSCON_RESET_CNTL, rst_reg, 0);
+
+ /* Wait for firmware to initialize */
+
+ SMUM_WAIT_VFPF_INDIRECT_FIELD(smumgr, SMC_IND,
+ FIRMWARE_FLAGS, INTERRUPTS_ENABLED, 1);
+
+ return result;
+}
+
+static int polaris10_start_smu(struct pp_smumgr *smumgr)
+{
+ int result = 0;
+ struct polaris10_smumgr *smu_data = (struct polaris10_smumgr *)(smumgr->backend);
+ bool SMU_VFT_INTACT;
+
+ /* Only start SMC if SMC RAM is not running */
+ if (!polaris10_is_smc_ram_running(smumgr)) {
+ SMU_VFT_INTACT = false;
+ smu_data->protected_mode = (uint8_t) (SMUM_READ_VFPF_INDIRECT_FIELD(smumgr->device, CGS_IND_REG__SMC, SMU_FIRMWARE, SMU_MODE));
+ smu_data->security_hard_key = (uint8_t) (SMUM_READ_VFPF_INDIRECT_FIELD(smumgr->device, CGS_IND_REG__SMC, SMU_FIRMWARE, SMU_SEL));
+
+ /* Check if SMU is running in protected mode */
+ if (smu_data->protected_mode == 0) {
+ result = polaris10_start_smu_in_non_protection_mode(smumgr);
+ } else {
+ result = polaris10_start_smu_in_protection_mode(smumgr);
+
+ /* If failed, try with different security Key. */
+ if (result != 0) {
+ smu_data->security_hard_key ^= 1;
+ result = polaris10_start_smu_in_protection_mode(smumgr);
+ }
+ }
+
+ if (result != 0)
+ PP_ASSERT_WITH_CODE(0, "Failed to load SMU ucode.", return result);
+
+ polaris10_avfs_event_mgr(smumgr, true);
+ } else
+ SMU_VFT_INTACT = true; /*Driver went offline but SMU was still alive and contains the VFT table */
+
+ smu_data->post_initial_boot = true;
+ polaris10_avfs_event_mgr(smumgr, SMU_VFT_INTACT);
+ /* Setup SoftRegsStart here for register lookup in case DummyBackEnd is used and ProcessFirmwareHeader is not executed */
+ polaris10_read_smc_sram_dword(smumgr, SMU7_FIRMWARE_HEADER_LOCATION + offsetof(SMU74_Firmware_Header, SoftRegisters),
+ &(smu_data->soft_regs_start), 0x40000);
+
+ result = polaris10_request_smu_load_fw(smumgr);
+
+ return result;
+}
+
+static int polaris10_smu_init(struct pp_smumgr *smumgr)
+{
+ struct polaris10_smumgr *smu_data;
+ uint8_t *internal_buf;
+ uint64_t mc_addr = 0;
+ /* Allocate memory for backend private data */
+ smu_data = (struct polaris10_smumgr *)(smumgr->backend);
+ smu_data->header_buffer.data_size =
+ ((sizeof(struct SMU_DRAMData_TOC) / 4096) + 1) * 4096;
+ smu_data->smu_buffer.data_size = 200*4096;
+ smu_data->avfs.avfs_btc_status = AVFS_BTC_NOTSUPPORTED;
+/* Allocate FW image data structure and header buffer and
+ * send the header buffer address to SMU */
+ smu_allocate_memory(smumgr->device,
+ smu_data->header_buffer.data_size,
+ CGS_GPU_MEM_TYPE__VISIBLE_CONTIG_FB,
+ PAGE_SIZE,
+ &mc_addr,
+ &smu_data->header_buffer.kaddr,
+ &smu_data->header_buffer.handle);
+
+ smu_data->header = smu_data->header_buffer.kaddr;
+ smu_data->header_buffer.mc_addr_high = smu_upper_32_bits(mc_addr);
+ smu_data->header_buffer.mc_addr_low = smu_lower_32_bits(mc_addr);
+
+ PP_ASSERT_WITH_CODE((NULL != smu_data->header),
+ "Out of memory.",
+ kfree(smumgr->backend);
+ cgs_free_gpu_mem(smumgr->device,
+ (cgs_handle_t)smu_data->header_buffer.handle);
+ return -1);
+
+/* Allocate buffer for SMU internal buffer and send the address to SMU.
+ * Iceland SMU does not need internal buffer.*/
+ smu_allocate_memory(smumgr->device,
+ smu_data->smu_buffer.data_size,
+ CGS_GPU_MEM_TYPE__VISIBLE_CONTIG_FB,
+ PAGE_SIZE,
+ &mc_addr,
+ &smu_data->smu_buffer.kaddr,
+ &smu_data->smu_buffer.handle);
+
+ internal_buf = smu_data->smu_buffer.kaddr;
+ smu_data->smu_buffer.mc_addr_high = smu_upper_32_bits(mc_addr);
+ smu_data->smu_buffer.mc_addr_low = smu_lower_32_bits(mc_addr);
+
+ PP_ASSERT_WITH_CODE((NULL != internal_buf),
+ "Out of memory.",
+ kfree(smumgr->backend);
+ cgs_free_gpu_mem(smumgr->device,
+ (cgs_handle_t)smu_data->smu_buffer.handle);
+ return -1;);
+
+ return 0;
+}
+
+static const struct pp_smumgr_func ellsemere_smu_funcs = {
+ .smu_init = polaris10_smu_init,
+ .smu_fini = polaris10_smu_fini,
+ .start_smu = polaris10_start_smu,
+ .check_fw_load_finish = polaris10_check_fw_load_finish,
+ .request_smu_load_fw = polaris10_reload_firmware,
+ .request_smu_load_specific_fw = NULL,
+ .send_msg_to_smc = polaris10_send_msg_to_smc,
+ .send_msg_to_smc_with_parameter = polaris10_send_msg_to_smc_with_parameter,
+ .download_pptable_settings = NULL,
+ .upload_pptable_settings = NULL,
+};
+
+int polaris10_smum_init(struct pp_smumgr *smumgr)
+{
+ struct polaris10_smumgr *polaris10_smu = NULL;
+
+ polaris10_smu = kzalloc(sizeof(struct polaris10_smumgr), GFP_KERNEL);
+
+ if (polaris10_smu == NULL)
+ return -1;
+
+ smumgr->backend = polaris10_smu;
+ smumgr->smumgr_funcs = &ellsemere_smu_funcs;
+
+ return 0;
+}
diff --git a/drivers/gpu/drm/amd/powerplay/smumgr/polaris10_smumgr.h b/drivers/gpu/drm/amd/powerplay/smumgr/polaris10_smumgr.h
new file mode 100644
index 0000000..e5377ae
--- /dev/null
+++ b/drivers/gpu/drm/amd/powerplay/smumgr/polaris10_smumgr.h
@@ -0,0 +1,68 @@
+/*
+ * Copyright 2015 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#ifndef _POLARIS10_SMUMANAGER_H
+#define _POLARIS10_SMUMANAGER_H
+
+#include <polaris10_ppsmc.h>
+#include <pp_endian.h>
+
+struct polaris10_avfs {
+ enum AVFS_BTC_STATUS avfs_btc_status;
+ uint32_t avfs_btc_param;
+};
+
+struct polaris10_buffer_entry {
+ uint32_t data_size;
+ uint32_t mc_addr_low;
+ uint32_t mc_addr_high;
+ void *kaddr;
+ unsigned long handle;
+};
+
+struct polaris10_smumgr {
+ uint8_t *header;
+ uint8_t *mec_image;
+ struct polaris10_buffer_entry smu_buffer;
+ struct polaris10_buffer_entry header_buffer;
+ uint32_t soft_regs_start;
+ uint8_t *read_rrm_straps;
+ uint32_t read_drm_straps_mc_address_high;
+ uint32_t read_drm_straps_mc_address_low;
+ uint32_t acpi_optimization;
+ bool post_initial_boot;
+ uint8_t protected_mode;
+ uint8_t security_hard_key;
+ struct polaris10_avfs avfs;
+};
+
+
+int polaris10_smum_init(struct pp_smumgr *smumgr);
+
+int polaris10_read_smc_sram_dword(struct pp_smumgr *smumgr, uint32_t smc_addr, uint32_t *value, uint32_t limit);
+int polaris10_write_smc_sram_dword(struct pp_smumgr *smumgr, uint32_t smc_addr, uint32_t value, uint32_t limit);
+int polaris10_copy_bytes_to_smc(struct pp_smumgr *smumgr, uint32_t smc_start_address,
+ const uint8_t *src, uint32_t byte_count, uint32_t limit);
+
+#endif
+
diff --git a/drivers/gpu/drm/amd/powerplay/smumgr/smumgr.c b/drivers/gpu/drm/amd/powerplay/smumgr/smumgr.c
index 063ae71..c483baf 100644
--- a/drivers/gpu/drm/amd/powerplay/smumgr/smumgr.c
+++ b/drivers/gpu/drm/amd/powerplay/smumgr/smumgr.c
@@ -30,6 +30,7 @@
#include "cz_smumgr.h"
#include "tonga_smumgr.h"
#include "fiji_smumgr.h"
+#include "polaris10_smumgr.h"
int smum_init(struct amd_pp_init *pp_init, struct pp_instance *handle)
{
@@ -62,6 +63,10 @@
case CHIP_FIJI:
fiji_smum_init(smumgr);
break;
+ case CHIP_POLARIS11:
+ case CHIP_POLARIS10:
+ polaris10_smum_init(smumgr);
+ break;
default:
return -EINVAL;
}
diff --git a/drivers/gpu/drm/amd/powerplay/smumgr/tonga_smumgr.c b/drivers/gpu/drm/amd/powerplay/smumgr/tonga_smumgr.c
index ebdb43a..32820b6 100644
--- a/drivers/gpu/drm/amd/powerplay/smumgr/tonga_smumgr.c
+++ b/drivers/gpu/drm/amd/powerplay/smumgr/tonga_smumgr.c
@@ -145,7 +145,7 @@
int tonga_program_jump_on_start(struct pp_smumgr *smumgr)
{
- static unsigned char pData[] = { 0xE0, 0x00, 0x80, 0x40 };
+ static const unsigned char pData[] = { 0xE0, 0x00, 0x80, 0x40 };
tonga_copy_bytes_to_smc(smumgr, 0x0, pData, 4, sizeof(pData)+1);
diff --git a/drivers/gpu/drm/amd/scheduler/gpu_scheduler.c b/drivers/gpu/drm/amd/scheduler/gpu_scheduler.c
index a5ff945..c16248c 100644
--- a/drivers/gpu/drm/amd/scheduler/gpu_scheduler.c
+++ b/drivers/gpu/drm/amd/scheduler/gpu_scheduler.c
@@ -319,6 +319,48 @@
return added;
}
+static void amd_sched_free_job(struct fence *f, struct fence_cb *cb) {
+ struct amd_sched_job *job = container_of(cb, struct amd_sched_job, cb_free_job);
+ schedule_work(&job->work_free_job);
+}
+
+/* job_finish is called after hw fence signaled, and
+ * the job had already been deleted from ring_mirror_list
+ */
+void amd_sched_job_finish(struct amd_sched_job *s_job)
+{
+ struct amd_sched_job *next;
+ struct amd_gpu_scheduler *sched = s_job->sched;
+
+ if (sched->timeout != MAX_SCHEDULE_TIMEOUT) {
+ if (cancel_delayed_work(&s_job->work_tdr))
+ amd_sched_job_put(s_job);
+
+ /* queue TDR for next job */
+ next = list_first_entry_or_null(&sched->ring_mirror_list,
+ struct amd_sched_job, node);
+
+ if (next) {
+ INIT_DELAYED_WORK(&next->work_tdr, s_job->timeout_callback);
+ amd_sched_job_get(next);
+ schedule_delayed_work(&next->work_tdr, sched->timeout);
+ }
+ }
+}
+
+void amd_sched_job_begin(struct amd_sched_job *s_job)
+{
+ struct amd_gpu_scheduler *sched = s_job->sched;
+
+ if (sched->timeout != MAX_SCHEDULE_TIMEOUT &&
+ list_first_entry_or_null(&sched->ring_mirror_list, struct amd_sched_job, node) == s_job)
+ {
+ INIT_DELAYED_WORK(&s_job->work_tdr, s_job->timeout_callback);
+ amd_sched_job_get(s_job);
+ schedule_delayed_work(&s_job->work_tdr, sched->timeout);
+ }
+}
+
/**
* Submit a job to the job queue
*
@@ -330,11 +372,39 @@
{
struct amd_sched_entity *entity = sched_job->s_entity;
+ sched_job->use_sched = 1;
+ fence_add_callback(&sched_job->s_fence->base,
+ &sched_job->cb_free_job, amd_sched_free_job);
trace_amd_sched_job(sched_job);
wait_event(entity->sched->job_scheduled,
amd_sched_entity_in(sched_job));
}
+/* init a sched_job with basic field */
+int amd_sched_job_init(struct amd_sched_job *job,
+ struct amd_gpu_scheduler *sched,
+ struct amd_sched_entity *entity,
+ void (*timeout_cb)(struct work_struct *work),
+ void (*free_cb)(struct kref *refcount),
+ void *owner, struct fence **fence)
+{
+ INIT_LIST_HEAD(&job->node);
+ kref_init(&job->refcount);
+ job->sched = sched;
+ job->s_entity = entity;
+ job->s_fence = amd_sched_fence_create(entity, owner);
+ if (!job->s_fence)
+ return -ENOMEM;
+
+ job->s_fence->s_job = job;
+ job->timeout_callback = timeout_cb;
+ job->free_callback = free_cb;
+
+ if (fence)
+ *fence = &job->s_fence->base;
+ return 0;
+}
+
/**
* Return ture if we can push more jobs to the hw.
*/
@@ -383,47 +453,26 @@
unsigned long flags;
atomic_dec(&sched->hw_rq_count);
+
+ /* remove job from ring_mirror_list */
+ spin_lock_irqsave(&sched->job_list_lock, flags);
+ list_del_init(&s_fence->s_job->node);
+ sched->ops->finish_job(s_fence->s_job);
+ spin_unlock_irqrestore(&sched->job_list_lock, flags);
+
amd_sched_fence_signal(s_fence);
- if (sched->timeout != MAX_SCHEDULE_TIMEOUT) {
- cancel_delayed_work(&s_fence->dwork);
- spin_lock_irqsave(&sched->fence_list_lock, flags);
- list_del_init(&s_fence->list);
- spin_unlock_irqrestore(&sched->fence_list_lock, flags);
- }
+
trace_amd_sched_process_job(s_fence);
fence_put(&s_fence->base);
wake_up_interruptible(&sched->wake_up_worker);
}
-static void amd_sched_fence_work_func(struct work_struct *work)
-{
- struct amd_sched_fence *s_fence =
- container_of(work, struct amd_sched_fence, dwork.work);
- struct amd_gpu_scheduler *sched = s_fence->sched;
- struct amd_sched_fence *entity, *tmp;
- unsigned long flags;
-
- DRM_ERROR("[%s] scheduler is timeout!\n", sched->name);
-
- /* Clean all pending fences */
- spin_lock_irqsave(&sched->fence_list_lock, flags);
- list_for_each_entry_safe(entity, tmp, &sched->fence_list, list) {
- DRM_ERROR(" fence no %d\n", entity->base.seqno);
- cancel_delayed_work(&entity->dwork);
- list_del_init(&entity->list);
- fence_put(&entity->base);
- }
- spin_unlock_irqrestore(&sched->fence_list_lock, flags);
-}
-
static int amd_sched_main(void *param)
{
struct sched_param sparam = {.sched_priority = 1};
struct amd_gpu_scheduler *sched = (struct amd_gpu_scheduler *)param;
int r, count;
- spin_lock_init(&sched->fence_list_lock);
- INIT_LIST_HEAD(&sched->fence_list);
sched_setscheduler(current, SCHED_FIFO, &sparam);
while (!kthread_should_stop()) {
@@ -431,7 +480,6 @@
struct amd_sched_fence *s_fence;
struct amd_sched_job *sched_job;
struct fence *fence;
- unsigned long flags;
wait_event_interruptible(sched->wake_up_worker,
(entity = amd_sched_select_entity(sched)) ||
@@ -446,15 +494,8 @@
s_fence = sched_job->s_fence;
- if (sched->timeout != MAX_SCHEDULE_TIMEOUT) {
- INIT_DELAYED_WORK(&s_fence->dwork, amd_sched_fence_work_func);
- schedule_delayed_work(&s_fence->dwork, sched->timeout);
- spin_lock_irqsave(&sched->fence_list_lock, flags);
- list_add_tail(&s_fence->list, &sched->fence_list);
- spin_unlock_irqrestore(&sched->fence_list_lock, flags);
- }
-
atomic_inc(&sched->hw_rq_count);
+ amd_sched_job_pre_schedule(sched, sched_job);
fence = sched->ops->run_job(sched_job);
amd_sched_fence_scheduled(s_fence);
if (fence) {
@@ -489,7 +530,7 @@
* Return 0 on success, otherwise error code.
*/
int amd_sched_init(struct amd_gpu_scheduler *sched,
- struct amd_sched_backend_ops *ops,
+ const struct amd_sched_backend_ops *ops,
unsigned hw_submission, long timeout, const char *name)
{
int i;
@@ -502,6 +543,8 @@
init_waitqueue_head(&sched->wake_up_worker);
init_waitqueue_head(&sched->job_scheduled);
+ INIT_LIST_HEAD(&sched->ring_mirror_list);
+ spin_lock_init(&sched->job_list_lock);
atomic_set(&sched->hw_rq_count, 0);
if (atomic_inc_return(&sched_fence_slab_ref) == 1) {
sched_fence_slab = kmem_cache_create(
diff --git a/drivers/gpu/drm/amd/scheduler/gpu_scheduler.h b/drivers/gpu/drm/amd/scheduler/gpu_scheduler.h
index 9403145..070095a 100644
--- a/drivers/gpu/drm/amd/scheduler/gpu_scheduler.h
+++ b/drivers/gpu/drm/amd/scheduler/gpu_scheduler.h
@@ -37,7 +37,7 @@
/**
* A scheduler entity is a wrapper around a job queue or a group
- * of other entities. Entities take turns emitting jobs from their
+ * of other entities. Entities take turns emitting jobs from their
* job queues to corresponding hardware ring based on scheduling
* policy.
*/
@@ -74,14 +74,21 @@
struct amd_gpu_scheduler *sched;
spinlock_t lock;
void *owner;
- struct delayed_work dwork;
- struct list_head list;
+ struct amd_sched_job *s_job;
};
struct amd_sched_job {
+ struct kref refcount;
struct amd_gpu_scheduler *sched;
struct amd_sched_entity *s_entity;
struct amd_sched_fence *s_fence;
+ bool use_sched; /* true if the job goes to scheduler */
+ struct fence_cb cb_free_job;
+ struct work_struct work_free_job;
+ struct list_head node;
+ struct delayed_work work_tdr;
+ void (*timeout_callback) (struct work_struct *work);
+ void (*free_callback)(struct kref *refcount);
};
extern const struct fence_ops amd_sched_fence_ops;
@@ -102,6 +109,8 @@
struct amd_sched_backend_ops {
struct fence *(*dependency)(struct amd_sched_job *sched_job);
struct fence *(*run_job)(struct amd_sched_job *sched_job);
+ void (*begin_job)(struct amd_sched_job *sched_job);
+ void (*finish_job)(struct amd_sched_job *sched_job);
};
enum amd_sched_priority {
@@ -114,7 +123,7 @@
* One scheduler is implemented for each hardware ring
*/
struct amd_gpu_scheduler {
- struct amd_sched_backend_ops *ops;
+ const struct amd_sched_backend_ops *ops;
uint32_t hw_submission_limit;
long timeout;
const char *name;
@@ -122,13 +131,13 @@
wait_queue_head_t wake_up_worker;
wait_queue_head_t job_scheduled;
atomic_t hw_rq_count;
- struct list_head fence_list;
- spinlock_t fence_list_lock;
struct task_struct *thread;
+ struct list_head ring_mirror_list;
+ spinlock_t job_list_lock;
};
int amd_sched_init(struct amd_gpu_scheduler *sched,
- struct amd_sched_backend_ops *ops,
+ const struct amd_sched_backend_ops *ops,
uint32_t hw_submission, long timeout, const char *name);
void amd_sched_fini(struct amd_gpu_scheduler *sched);
@@ -144,5 +153,24 @@
struct amd_sched_entity *s_entity, void *owner);
void amd_sched_fence_scheduled(struct amd_sched_fence *fence);
void amd_sched_fence_signal(struct amd_sched_fence *fence);
+int amd_sched_job_init(struct amd_sched_job *job,
+ struct amd_gpu_scheduler *sched,
+ struct amd_sched_entity *entity,
+ void (*timeout_cb)(struct work_struct *work),
+ void (*free_cb)(struct kref* refcount),
+ void *owner, struct fence **fence);
+void amd_sched_job_pre_schedule(struct amd_gpu_scheduler *sched ,
+ struct amd_sched_job *s_job);
+void amd_sched_job_finish(struct amd_sched_job *s_job);
+void amd_sched_job_begin(struct amd_sched_job *s_job);
+static inline void amd_sched_job_get(struct amd_sched_job *job) {
+ if (job)
+ kref_get(&job->refcount);
+}
+
+static inline void amd_sched_job_put(struct amd_sched_job *job) {
+ if (job)
+ kref_put(&job->refcount, job->free_callback);
+}
#endif
diff --git a/drivers/gpu/drm/amd/scheduler/sched_fence.c b/drivers/gpu/drm/amd/scheduler/sched_fence.c
index dc115ae..2a732c4 100644
--- a/drivers/gpu/drm/amd/scheduler/sched_fence.c
+++ b/drivers/gpu/drm/amd/scheduler/sched_fence.c
@@ -57,6 +57,16 @@
FENCE_TRACE(&fence->base, "was already signaled\n");
}
+void amd_sched_job_pre_schedule(struct amd_gpu_scheduler *sched ,
+ struct amd_sched_job *s_job)
+{
+ unsigned long flags;
+ spin_lock_irqsave(&sched->job_list_lock, flags);
+ list_add_tail(&s_job->node, &sched->ring_mirror_list);
+ sched->ops->begin_job(s_job);
+ spin_unlock_irqrestore(&sched->job_list_lock, flags);
+}
+
void amd_sched_fence_scheduled(struct amd_sched_fence *s_fence)
{
struct fence_cb *cur, *tmp;
diff --git a/drivers/gpu/drm/arc/Kconfig b/drivers/gpu/drm/arc/Kconfig
new file mode 100644
index 0000000..f9a13b6
--- /dev/null
+++ b/drivers/gpu/drm/arc/Kconfig
@@ -0,0 +1,10 @@
+config DRM_ARCPGU
+ tristate "ARC PGU"
+ depends on DRM && OF
+ select DRM_KMS_CMA_HELPER
+ select DRM_KMS_FB_HELPER
+ select DRM_KMS_HELPER
+ help
+ Choose this option if you have an ARC PGU controller.
+
+ If M is selected the module will be called arcpgu.
diff --git a/drivers/gpu/drm/arc/Makefile b/drivers/gpu/drm/arc/Makefile
new file mode 100644
index 0000000..d48fda7
--- /dev/null
+++ b/drivers/gpu/drm/arc/Makefile
@@ -0,0 +1,2 @@
+arcpgu-y := arcpgu_crtc.o arcpgu_hdmi.o arcpgu_drv.o
+obj-$(CONFIG_DRM_ARCPGU) += arcpgu.o
diff --git a/drivers/gpu/drm/arc/arcpgu.h b/drivers/gpu/drm/arc/arcpgu.h
new file mode 100644
index 0000000..86574b6
--- /dev/null
+++ b/drivers/gpu/drm/arc/arcpgu.h
@@ -0,0 +1,50 @@
+/*
+ * ARC PGU DRM driver.
+ *
+ * Copyright (C) 2016 Synopsys, Inc. (www.synopsys.com)
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#ifndef _ARCPGU_H_
+#define _ARCPGU_H_
+
+struct arcpgu_drm_private {
+ void __iomem *regs;
+ struct clk *clk;
+ struct drm_fbdev_cma *fbdev;
+ struct drm_framebuffer *fb;
+ struct list_head event_list;
+ struct drm_crtc crtc;
+ struct drm_plane *plane;
+};
+
+#define crtc_to_arcpgu_priv(x) container_of(x, struct arcpgu_drm_private, crtc)
+
+static inline void arc_pgu_write(struct arcpgu_drm_private *arcpgu,
+ unsigned int reg, u32 value)
+{
+ iowrite32(value, arcpgu->regs + reg);
+}
+
+static inline u32 arc_pgu_read(struct arcpgu_drm_private *arcpgu,
+ unsigned int reg)
+{
+ return ioread32(arcpgu->regs + reg);
+}
+
+int arc_pgu_setup_crtc(struct drm_device *dev);
+int arcpgu_drm_hdmi_init(struct drm_device *drm, struct device_node *np);
+struct drm_fbdev_cma *arcpgu_fbdev_cma_init(struct drm_device *dev,
+ unsigned int preferred_bpp, unsigned int num_crtc,
+ unsigned int max_conn_count);
+
+#endif
diff --git a/drivers/gpu/drm/arc/arcpgu_crtc.c b/drivers/gpu/drm/arc/arcpgu_crtc.c
new file mode 100644
index 0000000..92f8bef
--- /dev/null
+++ b/drivers/gpu/drm/arc/arcpgu_crtc.c
@@ -0,0 +1,257 @@
+/*
+ * ARC PGU DRM driver.
+ *
+ * Copyright (C) 2016 Synopsys, Inc. (www.synopsys.com)
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_fb_cma_helper.h>
+#include <drm/drm_gem_cma_helper.h>
+#include <drm/drm_plane_helper.h>
+#include <linux/clk.h>
+#include <linux/platform_data/simplefb.h>
+
+#include "arcpgu.h"
+#include "arcpgu_regs.h"
+
+#define ENCODE_PGU_XY(x, y) ((((x) - 1) << 16) | ((y) - 1))
+
+static struct simplefb_format supported_formats[] = {
+ { "r5g6b5", 16, {11, 5}, {5, 6}, {0, 5}, {0, 0}, DRM_FORMAT_RGB565 },
+ { "r8g8b8", 24, {16, 8}, {8, 8}, {0, 8}, {0, 0}, DRM_FORMAT_RGB888 },
+};
+
+static void arc_pgu_set_pxl_fmt(struct drm_crtc *crtc)
+{
+ struct arcpgu_drm_private *arcpgu = crtc_to_arcpgu_priv(crtc);
+ uint32_t pixel_format = crtc->primary->state->fb->pixel_format;
+ struct simplefb_format *format = NULL;
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(supported_formats); i++) {
+ if (supported_formats[i].fourcc == pixel_format)
+ format = &supported_formats[i];
+ }
+
+ if (WARN_ON(!format))
+ return;
+
+ if (format->fourcc == DRM_FORMAT_RGB888)
+ arc_pgu_write(arcpgu, ARCPGU_REG_CTRL,
+ arc_pgu_read(arcpgu, ARCPGU_REG_CTRL) |
+ ARCPGU_MODE_RGB888_MASK);
+
+}
+
+static const struct drm_crtc_funcs arc_pgu_crtc_funcs = {
+ .destroy = drm_crtc_cleanup,
+ .set_config = drm_atomic_helper_set_config,
+ .page_flip = drm_atomic_helper_page_flip,
+ .reset = drm_atomic_helper_crtc_reset,
+ .atomic_duplicate_state = drm_atomic_helper_crtc_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
+};
+
+static void arc_pgu_crtc_mode_set_nofb(struct drm_crtc *crtc)
+{
+ struct arcpgu_drm_private *arcpgu = crtc_to_arcpgu_priv(crtc);
+ struct drm_display_mode *m = &crtc->state->adjusted_mode;
+ u32 val;
+
+ arc_pgu_write(arcpgu, ARCPGU_REG_FMT,
+ ENCODE_PGU_XY(m->crtc_htotal, m->crtc_vtotal));
+
+ arc_pgu_write(arcpgu, ARCPGU_REG_HSYNC,
+ ENCODE_PGU_XY(m->crtc_hsync_start - m->crtc_hdisplay,
+ m->crtc_hsync_end - m->crtc_hdisplay));
+
+ arc_pgu_write(arcpgu, ARCPGU_REG_VSYNC,
+ ENCODE_PGU_XY(m->crtc_vsync_start - m->crtc_vdisplay,
+ m->crtc_vsync_end - m->crtc_vdisplay));
+
+ arc_pgu_write(arcpgu, ARCPGU_REG_ACTIVE,
+ ENCODE_PGU_XY(m->crtc_hblank_end - m->crtc_hblank_start,
+ m->crtc_vblank_end - m->crtc_vblank_start));
+
+ val = arc_pgu_read(arcpgu, ARCPGU_REG_CTRL);
+
+ if (m->flags & DRM_MODE_FLAG_PVSYNC)
+ val |= ARCPGU_CTRL_VS_POL_MASK << ARCPGU_CTRL_VS_POL_OFST;
+ else
+ val &= ~(ARCPGU_CTRL_VS_POL_MASK << ARCPGU_CTRL_VS_POL_OFST);
+
+ if (m->flags & DRM_MODE_FLAG_PHSYNC)
+ val |= ARCPGU_CTRL_HS_POL_MASK << ARCPGU_CTRL_HS_POL_OFST;
+ else
+ val &= ~(ARCPGU_CTRL_HS_POL_MASK << ARCPGU_CTRL_HS_POL_OFST);
+
+ arc_pgu_write(arcpgu, ARCPGU_REG_CTRL, val);
+ arc_pgu_write(arcpgu, ARCPGU_REG_STRIDE, 0);
+ arc_pgu_write(arcpgu, ARCPGU_REG_START_SET, 1);
+
+ arc_pgu_set_pxl_fmt(crtc);
+
+ clk_set_rate(arcpgu->clk, m->crtc_clock * 1000);
+}
+
+static void arc_pgu_crtc_enable(struct drm_crtc *crtc)
+{
+ struct arcpgu_drm_private *arcpgu = crtc_to_arcpgu_priv(crtc);
+
+ clk_prepare_enable(arcpgu->clk);
+ arc_pgu_write(arcpgu, ARCPGU_REG_CTRL,
+ arc_pgu_read(arcpgu, ARCPGU_REG_CTRL) |
+ ARCPGU_CTRL_ENABLE_MASK);
+}
+
+static void arc_pgu_crtc_disable(struct drm_crtc *crtc)
+{
+ struct arcpgu_drm_private *arcpgu = crtc_to_arcpgu_priv(crtc);
+
+ if (!crtc->primary->fb)
+ return;
+
+ clk_disable_unprepare(arcpgu->clk);
+ arc_pgu_write(arcpgu, ARCPGU_REG_CTRL,
+ arc_pgu_read(arcpgu, ARCPGU_REG_CTRL) &
+ ~ARCPGU_CTRL_ENABLE_MASK);
+}
+
+static int arc_pgu_crtc_atomic_check(struct drm_crtc *crtc,
+ struct drm_crtc_state *state)
+{
+ struct arcpgu_drm_private *arcpgu = crtc_to_arcpgu_priv(crtc);
+ struct drm_display_mode *mode = &state->adjusted_mode;
+ long rate, clk_rate = mode->clock * 1000;
+
+ rate = clk_round_rate(arcpgu->clk, clk_rate);
+ if (rate != clk_rate)
+ return -EINVAL;
+
+ return 0;
+}
+
+static void arc_pgu_crtc_atomic_begin(struct drm_crtc *crtc,
+ struct drm_crtc_state *state)
+{
+ struct arcpgu_drm_private *arcpgu = crtc_to_arcpgu_priv(crtc);
+ unsigned long flags;
+
+ if (crtc->state->event) {
+ struct drm_pending_vblank_event *event = crtc->state->event;
+
+ crtc->state->event = NULL;
+ event->pipe = drm_crtc_index(crtc);
+
+ WARN_ON(drm_crtc_vblank_get(crtc) != 0);
+
+ spin_lock_irqsave(&crtc->dev->event_lock, flags);
+ list_add_tail(&event->base.link, &arcpgu->event_list);
+ spin_unlock_irqrestore(&crtc->dev->event_lock, flags);
+ }
+}
+
+static const struct drm_crtc_helper_funcs arc_pgu_crtc_helper_funcs = {
+ .mode_set = drm_helper_crtc_mode_set,
+ .mode_set_base = drm_helper_crtc_mode_set_base,
+ .mode_set_nofb = arc_pgu_crtc_mode_set_nofb,
+ .enable = arc_pgu_crtc_enable,
+ .disable = arc_pgu_crtc_disable,
+ .prepare = arc_pgu_crtc_disable,
+ .commit = arc_pgu_crtc_enable,
+ .atomic_check = arc_pgu_crtc_atomic_check,
+ .atomic_begin = arc_pgu_crtc_atomic_begin,
+};
+
+static void arc_pgu_plane_atomic_update(struct drm_plane *plane,
+ struct drm_plane_state *state)
+{
+ struct arcpgu_drm_private *arcpgu;
+ struct drm_gem_cma_object *gem;
+
+ if (!plane->state->crtc || !plane->state->fb)
+ return;
+
+ arcpgu = crtc_to_arcpgu_priv(plane->state->crtc);
+ gem = drm_fb_cma_get_gem_obj(plane->state->fb, 0);
+ arc_pgu_write(arcpgu, ARCPGU_REG_BUF0_ADDR, gem->paddr);
+}
+
+static const struct drm_plane_helper_funcs arc_pgu_plane_helper_funcs = {
+ .prepare_fb = NULL,
+ .cleanup_fb = NULL,
+ .atomic_update = arc_pgu_plane_atomic_update,
+};
+
+static void arc_pgu_plane_destroy(struct drm_plane *plane)
+{
+ drm_plane_helper_disable(plane);
+ drm_plane_cleanup(plane);
+}
+
+static const struct drm_plane_funcs arc_pgu_plane_funcs = {
+ .update_plane = drm_atomic_helper_update_plane,
+ .disable_plane = drm_atomic_helper_disable_plane,
+ .destroy = arc_pgu_plane_destroy,
+ .reset = drm_atomic_helper_plane_reset,
+ .atomic_duplicate_state = drm_atomic_helper_plane_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_plane_destroy_state,
+};
+
+static struct drm_plane *arc_pgu_plane_init(struct drm_device *drm)
+{
+ struct arcpgu_drm_private *arcpgu = drm->dev_private;
+ struct drm_plane *plane = NULL;
+ u32 formats[ARRAY_SIZE(supported_formats)], i;
+ int ret;
+
+ plane = devm_kzalloc(drm->dev, sizeof(*plane), GFP_KERNEL);
+ if (!plane)
+ return ERR_PTR(-ENOMEM);
+
+ for (i = 0; i < ARRAY_SIZE(supported_formats); i++)
+ formats[i] = supported_formats[i].fourcc;
+
+ ret = drm_universal_plane_init(drm, plane, 0xff, &arc_pgu_plane_funcs,
+ formats, ARRAY_SIZE(formats),
+ DRM_PLANE_TYPE_PRIMARY, NULL);
+ if (ret)
+ return ERR_PTR(ret);
+
+ drm_plane_helper_add(plane, &arc_pgu_plane_helper_funcs);
+ arcpgu->plane = plane;
+
+ return plane;
+}
+
+int arc_pgu_setup_crtc(struct drm_device *drm)
+{
+ struct arcpgu_drm_private *arcpgu = drm->dev_private;
+ struct drm_plane *primary;
+ int ret;
+
+ primary = arc_pgu_plane_init(drm);
+ if (IS_ERR(primary))
+ return PTR_ERR(primary);
+
+ ret = drm_crtc_init_with_planes(drm, &arcpgu->crtc, primary, NULL,
+ &arc_pgu_crtc_funcs, NULL);
+ if (ret) {
+ arc_pgu_plane_destroy(primary);
+ return ret;
+ }
+
+ drm_crtc_helper_add(&arcpgu->crtc, &arc_pgu_crtc_helper_funcs);
+ return 0;
+}
diff --git a/drivers/gpu/drm/arc/arcpgu_drv.c b/drivers/gpu/drm/arc/arcpgu_drv.c
new file mode 100644
index 0000000..76e187a
--- /dev/null
+++ b/drivers/gpu/drm/arc/arcpgu_drv.c
@@ -0,0 +1,288 @@
+/*
+ * ARC PGU DRM driver.
+ *
+ * Copyright (C) 2016 Synopsys, Inc. (www.synopsys.com)
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include <linux/clk.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_fb_cma_helper.h>
+#include <drm/drm_gem_cma_helper.h>
+#include <drm/drm_atomic_helper.h>
+#include <linux/of_reserved_mem.h>
+
+#include "arcpgu.h"
+#include "arcpgu_regs.h"
+
+static void arcpgu_fb_output_poll_changed(struct drm_device *dev)
+{
+ struct arcpgu_drm_private *arcpgu = dev->dev_private;
+
+ if (arcpgu->fbdev)
+ drm_fbdev_cma_hotplug_event(arcpgu->fbdev);
+}
+
+static int arcpgu_atomic_commit(struct drm_device *dev,
+ struct drm_atomic_state *state, bool async)
+{
+ return drm_atomic_helper_commit(dev, state, false);
+}
+
+static struct drm_mode_config_funcs arcpgu_drm_modecfg_funcs = {
+ .fb_create = drm_fb_cma_create,
+ .output_poll_changed = arcpgu_fb_output_poll_changed,
+ .atomic_check = drm_atomic_helper_check,
+ .atomic_commit = arcpgu_atomic_commit,
+};
+
+static void arcpgu_setup_mode_config(struct drm_device *drm)
+{
+ drm_mode_config_init(drm);
+ drm->mode_config.min_width = 0;
+ drm->mode_config.min_height = 0;
+ drm->mode_config.max_width = 1920;
+ drm->mode_config.max_height = 1080;
+ drm->mode_config.funcs = &arcpgu_drm_modecfg_funcs;
+}
+
+int arcpgu_gem_mmap(struct file *filp, struct vm_area_struct *vma)
+{
+ int ret;
+
+ ret = drm_gem_mmap(filp, vma);
+ if (ret)
+ return ret;
+
+ vma->vm_page_prot = pgprot_noncached(vm_get_page_prot(vma->vm_flags));
+ return 0;
+}
+
+static const struct file_operations arcpgu_drm_ops = {
+ .owner = THIS_MODULE,
+ .open = drm_open,
+ .release = drm_release,
+ .unlocked_ioctl = drm_ioctl,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = drm_compat_ioctl,
+#endif
+ .poll = drm_poll,
+ .read = drm_read,
+ .llseek = no_llseek,
+ .mmap = arcpgu_gem_mmap,
+};
+
+static void arcpgu_preclose(struct drm_device *drm, struct drm_file *file)
+{
+ struct arcpgu_drm_private *arcpgu = drm->dev_private;
+ struct drm_pending_vblank_event *e, *t;
+ unsigned long flags;
+
+ spin_lock_irqsave(&drm->event_lock, flags);
+ list_for_each_entry_safe(e, t, &arcpgu->event_list, base.link) {
+ if (e->base.file_priv != file)
+ continue;
+ list_del(&e->base.link);
+ e->base.destroy(&e->base);
+ }
+ spin_unlock_irqrestore(&drm->event_lock, flags);
+}
+
+static void arcpgu_lastclose(struct drm_device *drm)
+{
+ struct arcpgu_drm_private *arcpgu = drm->dev_private;
+
+ drm_fbdev_cma_restore_mode(arcpgu->fbdev);
+}
+
+static int arcpgu_load(struct drm_device *drm)
+{
+ struct platform_device *pdev = to_platform_device(drm->dev);
+ struct arcpgu_drm_private *arcpgu;
+ struct device_node *encoder_node;
+ struct resource *res;
+ int ret;
+
+ arcpgu = devm_kzalloc(&pdev->dev, sizeof(*arcpgu), GFP_KERNEL);
+ if (arcpgu == NULL)
+ return -ENOMEM;
+
+ drm->dev_private = arcpgu;
+
+ arcpgu->clk = devm_clk_get(drm->dev, "pxlclk");
+ if (IS_ERR(arcpgu->clk))
+ return PTR_ERR(arcpgu->clk);
+
+ INIT_LIST_HEAD(&arcpgu->event_list);
+
+ arcpgu_setup_mode_config(drm);
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ arcpgu->regs = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(arcpgu->regs)) {
+ dev_err(drm->dev, "Could not remap IO mem\n");
+ return PTR_ERR(arcpgu->regs);
+ }
+
+ dev_info(drm->dev, "arc_pgu ID: 0x%x\n",
+ arc_pgu_read(arcpgu, ARCPGU_REG_ID));
+
+ /* Get the optional framebuffer memory resource */
+ ret = of_reserved_mem_device_init(drm->dev);
+ if (ret && ret != -ENODEV)
+ return ret;
+
+ if (dma_set_mask_and_coherent(drm->dev, DMA_BIT_MASK(32)))
+ return -ENODEV;
+
+ if (arc_pgu_setup_crtc(drm) < 0)
+ return -ENODEV;
+
+ /* find the encoder node and initialize it */
+ encoder_node = of_parse_phandle(drm->dev->of_node, "encoder-slave", 0);
+ if (!encoder_node) {
+ dev_err(drm->dev, "failed to get an encoder slave node\n");
+ return -ENODEV;
+ }
+
+ ret = arcpgu_drm_hdmi_init(drm, encoder_node);
+ if (ret < 0)
+ return ret;
+
+ drm_mode_config_reset(drm);
+ drm_kms_helper_poll_init(drm);
+
+ arcpgu->fbdev = drm_fbdev_cma_init(drm, 16,
+ drm->mode_config.num_crtc,
+ drm->mode_config.num_connector);
+ if (IS_ERR(arcpgu->fbdev)) {
+ ret = PTR_ERR(arcpgu->fbdev);
+ arcpgu->fbdev = NULL;
+ return -ENODEV;
+ }
+
+ platform_set_drvdata(pdev, arcpgu);
+ return 0;
+}
+
+int arcpgu_unload(struct drm_device *drm)
+{
+ struct arcpgu_drm_private *arcpgu = drm->dev_private;
+
+ if (arcpgu->fbdev) {
+ drm_fbdev_cma_fini(arcpgu->fbdev);
+ arcpgu->fbdev = NULL;
+ }
+ drm_kms_helper_poll_fini(drm);
+ drm_vblank_cleanup(drm);
+ drm_mode_config_cleanup(drm);
+
+ return 0;
+}
+
+static struct drm_driver arcpgu_drm_driver = {
+ .driver_features = DRIVER_MODESET | DRIVER_GEM | DRIVER_PRIME |
+ DRIVER_ATOMIC,
+ .preclose = arcpgu_preclose,
+ .lastclose = arcpgu_lastclose,
+ .name = "drm-arcpgu",
+ .desc = "ARC PGU Controller",
+ .date = "20160219",
+ .major = 1,
+ .minor = 0,
+ .patchlevel = 0,
+ .fops = &arcpgu_drm_ops,
+ .dumb_create = drm_gem_cma_dumb_create,
+ .dumb_map_offset = drm_gem_cma_dumb_map_offset,
+ .dumb_destroy = drm_gem_dumb_destroy,
+ .get_vblank_counter = drm_vblank_no_hw_counter,
+ .prime_handle_to_fd = drm_gem_prime_handle_to_fd,
+ .prime_fd_to_handle = drm_gem_prime_fd_to_handle,
+ .gem_free_object = drm_gem_cma_free_object,
+ .gem_vm_ops = &drm_gem_cma_vm_ops,
+ .gem_prime_export = drm_gem_prime_export,
+ .gem_prime_import = drm_gem_prime_import,
+ .gem_prime_get_sg_table = drm_gem_cma_prime_get_sg_table,
+ .gem_prime_import_sg_table = drm_gem_cma_prime_import_sg_table,
+ .gem_prime_vmap = drm_gem_cma_prime_vmap,
+ .gem_prime_vunmap = drm_gem_cma_prime_vunmap,
+ .gem_prime_mmap = drm_gem_cma_prime_mmap,
+};
+
+static int arcpgu_probe(struct platform_device *pdev)
+{
+ struct drm_device *drm;
+ int ret;
+
+ drm = drm_dev_alloc(&arcpgu_drm_driver, &pdev->dev);
+ if (!drm)
+ return -ENOMEM;
+
+ ret = arcpgu_load(drm);
+ if (ret)
+ goto err_unref;
+
+ ret = drm_dev_register(drm, 0);
+ if (ret)
+ goto err_unload;
+
+ ret = drm_connector_register_all(drm);
+ if (ret)
+ goto err_unregister;
+
+ return 0;
+
+err_unregister:
+ drm_dev_unregister(drm);
+
+err_unload:
+ arcpgu_unload(drm);
+
+err_unref:
+ drm_dev_unref(drm);
+
+ return ret;
+}
+
+static int arcpgu_remove(struct platform_device *pdev)
+{
+ struct drm_device *drm = platform_get_drvdata(pdev);
+
+ drm_connector_unregister_all(drm);
+ drm_dev_unregister(drm);
+ arcpgu_unload(drm);
+ drm_dev_unref(drm);
+
+ return 0;
+}
+
+static const struct of_device_id arcpgu_of_table[] = {
+ {.compatible = "snps,arcpgu"},
+ {}
+};
+
+MODULE_DEVICE_TABLE(of, arcpgu_of_table);
+
+static struct platform_driver arcpgu_platform_driver = {
+ .probe = arcpgu_probe,
+ .remove = arcpgu_remove,
+ .driver = {
+ .name = "arcpgu",
+ .of_match_table = arcpgu_of_table,
+ },
+};
+
+module_platform_driver(arcpgu_platform_driver);
+
+MODULE_AUTHOR("Carlos Palminha <palminha@synopsys.com>");
+MODULE_DESCRIPTION("ARC PGU DRM driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/gpu/drm/arc/arcpgu_hdmi.c b/drivers/gpu/drm/arc/arcpgu_hdmi.c
new file mode 100644
index 0000000..08b6bae
--- /dev/null
+++ b/drivers/gpu/drm/arc/arcpgu_hdmi.c
@@ -0,0 +1,201 @@
+/*
+ * ARC PGU DRM driver.
+ *
+ * Copyright (C) 2016 Synopsys, Inc. (www.synopsys.com)
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_encoder_slave.h>
+#include <drm/drm_atomic_helper.h>
+
+#include "arcpgu.h"
+
+struct arcpgu_drm_connector {
+ struct drm_connector connector;
+ struct drm_encoder_slave *encoder_slave;
+};
+
+static int arcpgu_drm_connector_get_modes(struct drm_connector *connector)
+{
+ const struct drm_encoder_slave_funcs *sfuncs;
+ struct drm_encoder_slave *slave;
+ struct arcpgu_drm_connector *con =
+ container_of(connector, struct arcpgu_drm_connector, connector);
+
+ slave = con->encoder_slave;
+ if (slave == NULL) {
+ dev_err(connector->dev->dev,
+ "connector_get_modes: cannot find slave encoder for connector\n");
+ return 0;
+ }
+
+ sfuncs = slave->slave_funcs;
+ if (sfuncs->get_modes == NULL)
+ return 0;
+
+ return sfuncs->get_modes(&slave->base, connector);
+}
+
+struct drm_encoder *
+arcpgu_drm_connector_best_encoder(struct drm_connector *connector)
+{
+ struct drm_encoder_slave *slave;
+ struct arcpgu_drm_connector *con =
+ container_of(connector, struct arcpgu_drm_connector, connector);
+
+ slave = con->encoder_slave;
+ if (slave == NULL) {
+ dev_err(connector->dev->dev,
+ "connector_best_encoder: cannot find slave encoder for connector\n");
+ return NULL;
+ }
+
+ return &slave->base;
+}
+
+static enum drm_connector_status
+arcpgu_drm_connector_detect(struct drm_connector *connector, bool force)
+{
+ enum drm_connector_status status = connector_status_unknown;
+ const struct drm_encoder_slave_funcs *sfuncs;
+ struct drm_encoder_slave *slave;
+
+ struct arcpgu_drm_connector *con =
+ container_of(connector, struct arcpgu_drm_connector, connector);
+
+ slave = con->encoder_slave;
+ if (slave == NULL) {
+ dev_err(connector->dev->dev,
+ "connector_detect: cannot find slave encoder for connector\n");
+ return status;
+ }
+
+ sfuncs = slave->slave_funcs;
+ if (sfuncs && sfuncs->detect)
+ return sfuncs->detect(&slave->base, connector);
+
+ dev_err(connector->dev->dev, "connector_detect: could not detect slave funcs\n");
+ return status;
+}
+
+static void arcpgu_drm_connector_destroy(struct drm_connector *connector)
+{
+ drm_connector_unregister(connector);
+ drm_connector_cleanup(connector);
+}
+
+static const struct drm_connector_helper_funcs
+arcpgu_drm_connector_helper_funcs = {
+ .get_modes = arcpgu_drm_connector_get_modes,
+ .best_encoder = arcpgu_drm_connector_best_encoder,
+};
+
+static const struct drm_connector_funcs arcpgu_drm_connector_funcs = {
+ .dpms = drm_helper_connector_dpms,
+ .reset = drm_atomic_helper_connector_reset,
+ .detect = arcpgu_drm_connector_detect,
+ .fill_modes = drm_helper_probe_single_connector_modes,
+ .destroy = arcpgu_drm_connector_destroy,
+ .atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
+};
+
+static struct drm_encoder_helper_funcs arcpgu_drm_encoder_helper_funcs = {
+ .dpms = drm_i2c_encoder_dpms,
+ .mode_fixup = drm_i2c_encoder_mode_fixup,
+ .mode_set = drm_i2c_encoder_mode_set,
+ .prepare = drm_i2c_encoder_prepare,
+ .commit = drm_i2c_encoder_commit,
+ .detect = drm_i2c_encoder_detect,
+};
+
+static struct drm_encoder_funcs arcpgu_drm_encoder_funcs = {
+ .destroy = drm_encoder_cleanup,
+};
+
+int arcpgu_drm_hdmi_init(struct drm_device *drm, struct device_node *np)
+{
+ struct arcpgu_drm_connector *arcpgu_connector;
+ struct drm_i2c_encoder_driver *driver;
+ struct drm_encoder_slave *encoder;
+ struct drm_connector *connector;
+ struct i2c_client *i2c_slave;
+ int ret;
+
+ encoder = devm_kzalloc(drm->dev, sizeof(*encoder), GFP_KERNEL);
+ if (encoder == NULL)
+ return -ENOMEM;
+
+ i2c_slave = of_find_i2c_device_by_node(np);
+ if (!i2c_slave || !i2c_get_clientdata(i2c_slave)) {
+ dev_err(drm->dev, "failed to find i2c slave encoder\n");
+ return -EPROBE_DEFER;
+ }
+
+ if (i2c_slave->dev.driver == NULL) {
+ dev_err(drm->dev, "failed to find i2c slave driver\n");
+ return -EPROBE_DEFER;
+ }
+
+ driver =
+ to_drm_i2c_encoder_driver(to_i2c_driver(i2c_slave->dev.driver));
+ ret = driver->encoder_init(i2c_slave, drm, encoder);
+ if (ret) {
+ dev_err(drm->dev, "failed to initialize i2c encoder slave\n");
+ return ret;
+ }
+
+ encoder->base.possible_crtcs = 1;
+ encoder->base.possible_clones = 0;
+ ret = drm_encoder_init(drm, &encoder->base, &arcpgu_drm_encoder_funcs,
+ DRM_MODE_ENCODER_TMDS, NULL);
+ if (ret)
+ return ret;
+
+ drm_encoder_helper_add(&encoder->base,
+ &arcpgu_drm_encoder_helper_funcs);
+
+ arcpgu_connector = devm_kzalloc(drm->dev, sizeof(*arcpgu_connector),
+ GFP_KERNEL);
+ if (!arcpgu_connector) {
+ ret = -ENOMEM;
+ goto error_encoder_cleanup;
+ }
+
+ connector = &arcpgu_connector->connector;
+ drm_connector_helper_add(connector, &arcpgu_drm_connector_helper_funcs);
+ ret = drm_connector_init(drm, connector, &arcpgu_drm_connector_funcs,
+ DRM_MODE_CONNECTOR_HDMIA);
+ if (ret < 0) {
+ dev_err(drm->dev, "failed to initialize drm connector\n");
+ goto error_encoder_cleanup;
+ }
+
+ ret = drm_mode_connector_attach_encoder(connector, &encoder->base);
+ if (ret < 0) {
+ dev_err(drm->dev, "could not attach connector to encoder\n");
+ drm_connector_unregister(connector);
+ goto error_connector_cleanup;
+ }
+
+ arcpgu_connector->encoder_slave = encoder;
+
+ return 0;
+
+error_connector_cleanup:
+ drm_connector_cleanup(connector);
+
+error_encoder_cleanup:
+ drm_encoder_cleanup(&encoder->base);
+ return ret;
+}
diff --git a/drivers/gpu/drm/arc/arcpgu_regs.h b/drivers/gpu/drm/arc/arcpgu_regs.h
new file mode 100644
index 0000000..95a13a8
--- /dev/null
+++ b/drivers/gpu/drm/arc/arcpgu_regs.h
@@ -0,0 +1,40 @@
+/*
+ * ARC PGU DRM driver.
+ *
+ * Copyright (C) 2016 Synopsys, Inc. (www.synopsys.com)
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#ifndef _ARC_PGU_REGS_H_
+#define _ARC_PGU_REGS_H_
+
+#define ARCPGU_REG_CTRL 0x00
+#define ARCPGU_REG_STAT 0x04
+#define ARCPGU_REG_FMT 0x10
+#define ARCPGU_REG_HSYNC 0x14
+#define ARCPGU_REG_VSYNC 0x18
+#define ARCPGU_REG_ACTIVE 0x1c
+#define ARCPGU_REG_BUF0_ADDR 0x40
+#define ARCPGU_REG_STRIDE 0x50
+#define ARCPGU_REG_START_SET 0x84
+
+#define ARCPGU_REG_ID 0x3FC
+
+#define ARCPGU_CTRL_ENABLE_MASK 0x02
+#define ARCPGU_CTRL_VS_POL_MASK 0x1
+#define ARCPGU_CTRL_VS_POL_OFST 0x3
+#define ARCPGU_CTRL_HS_POL_MASK 0x1
+#define ARCPGU_CTRL_HS_POL_OFST 0x4
+#define ARCPGU_MODE_RGB888_MASK 0x04
+#define ARCPGU_STAT_BUSY_MASK 0x02
+
+#endif
diff --git a/drivers/gpu/drm/arm/hdlcd_drv.c b/drivers/gpu/drm/arm/hdlcd_drv.c
index 3ac1ae4..b987c63 100644
--- a/drivers/gpu/drm/arm/hdlcd_drv.c
+++ b/drivers/gpu/drm/arm/hdlcd_drv.c
@@ -113,7 +113,7 @@
}
static int hdlcd_atomic_commit(struct drm_device *dev,
- struct drm_atomic_state *state, bool async)
+ struct drm_atomic_state *state, bool nonblock)
{
return drm_atomic_helper_commit(dev, state, false);
}
@@ -379,7 +379,6 @@
DRM_ERROR("failed to initialise vblank\n");
goto err_vblank;
}
- drm->vblank_disable_allowed = true;
drm_mode_config_reset(drm);
drm_kms_helper_poll_init(drm);
diff --git a/drivers/gpu/drm/armada/armada_crtc.c b/drivers/gpu/drm/armada/armada_crtc.c
index 0293eb7..3130aa8 100644
--- a/drivers/gpu/drm/armada/armada_crtc.c
+++ b/drivers/gpu/drm/armada/armada_crtc.c
@@ -897,7 +897,6 @@
static int armada_drm_crtc_cursor_set(struct drm_crtc *crtc,
struct drm_file *file, uint32_t handle, uint32_t w, uint32_t h)
{
- struct drm_device *dev = crtc->dev;
struct armada_crtc *dcrtc = drm_to_armada_crtc(crtc);
struct armada_gem_object *obj = NULL;
int ret;
@@ -911,7 +910,7 @@
if (w > 64 || h > 64 || (w > 32 && h > 32))
return -ENOMEM;
- obj = armada_gem_object_lookup(dev, file, handle);
+ obj = armada_gem_object_lookup(file, handle);
if (!obj)
return -ENOENT;
diff --git a/drivers/gpu/drm/armada/armada_drv.c b/drivers/gpu/drm/armada/armada_drv.c
index 82043c2..439824a 100644
--- a/drivers/gpu/drm/armada/armada_drv.c
+++ b/drivers/gpu/drm/armada/armada_drv.c
@@ -113,7 +113,6 @@
goto err_comp;
dev->irq_enabled = true;
- dev->vblank_disable_allowed = 1;
ret = armada_fbdev_init(dev);
if (ret)
diff --git a/drivers/gpu/drm/armada/armada_fb.c b/drivers/gpu/drm/armada/armada_fb.c
index 5fa4bf2..f03c212 100644
--- a/drivers/gpu/drm/armada/armada_fb.c
+++ b/drivers/gpu/drm/armada/armada_fb.c
@@ -120,7 +120,7 @@
goto err;
}
- obj = armada_gem_object_lookup(dev, dfile, mode->handles[0]);
+ obj = armada_gem_object_lookup(dfile, mode->handles[0]);
if (!obj) {
ret = -ENOENT;
goto err;
diff --git a/drivers/gpu/drm/armada/armada_gem.c b/drivers/gpu/drm/armada/armada_gem.c
index aca7f9c..88e7fc7 100644
--- a/drivers/gpu/drm/armada/armada_gem.c
+++ b/drivers/gpu/drm/armada/armada_gem.c
@@ -278,7 +278,7 @@
struct armada_gem_object *obj;
int ret = 0;
- obj = armada_gem_object_lookup(dev, file, handle);
+ obj = armada_gem_object_lookup(file, handle);
if (!obj) {
DRM_ERROR("failed to lookup gem object\n");
return -EINVAL;
@@ -348,7 +348,7 @@
struct armada_gem_object *dobj;
unsigned long addr;
- dobj = armada_gem_object_lookup(dev, file, args->handle);
+ dobj = armada_gem_object_lookup(file, args->handle);
if (dobj == NULL)
return -ENOENT;
@@ -391,7 +391,7 @@
if (ret)
return ret;
- dobj = armada_gem_object_lookup(dev, file, args->handle);
+ dobj = armada_gem_object_lookup(file, args->handle);
if (dobj == NULL)
return -ENOENT;
diff --git a/drivers/gpu/drm/armada/armada_gem.h b/drivers/gpu/drm/armada/armada_gem.h
index b000ea3..b88d2b9 100644
--- a/drivers/gpu/drm/armada/armada_gem.h
+++ b/drivers/gpu/drm/armada/armada_gem.h
@@ -45,9 +45,9 @@
int armada_gem_map_import(struct armada_gem_object *);
static inline struct armada_gem_object *armada_gem_object_lookup(
- struct drm_device *dev, struct drm_file *dfile, unsigned handle)
+ struct drm_file *dfile, unsigned handle)
{
- struct drm_gem_object *obj = drm_gem_object_lookup(dev, dfile, handle);
+ struct drm_gem_object *obj = drm_gem_object_lookup(dfile, handle);
return obj ? drm_to_armada_gem(obj) : NULL;
}
diff --git a/drivers/gpu/drm/ast/ast_drv.c b/drivers/gpu/drm/ast/ast_drv.c
index 9a32d9d..fcd9c07 100644
--- a/drivers/gpu/drm/ast/ast_drv.c
+++ b/drivers/gpu/drm/ast/ast_drv.c
@@ -218,10 +218,8 @@
static int __init ast_init(void)
{
-#ifdef CONFIG_VGA_CONSOLE
if (vgacon_text_force() && ast_modeset == -1)
return -EINVAL;
-#endif
if (ast_modeset == 0)
return -EINVAL;
diff --git a/drivers/gpu/drm/ast/ast_drv.h b/drivers/gpu/drm/ast/ast_drv.h
index eb57159..908011d 100644
--- a/drivers/gpu/drm/ast/ast_drv.h
+++ b/drivers/gpu/drm/ast/ast_drv.h
@@ -367,7 +367,7 @@
{
int ret;
- ret = ttm_bo_reserve(&bo->bo, true, no_wait, false, NULL);
+ ret = ttm_bo_reserve(&bo->bo, true, no_wait, NULL);
if (ret) {
if (ret != -ERESTARTSYS && ret != -EBUSY)
DRM_ERROR("reserve failed %p\n", bo);
diff --git a/drivers/gpu/drm/ast/ast_main.c b/drivers/gpu/drm/ast/ast_main.c
index b1480ac..7bc3aa6 100644
--- a/drivers/gpu/drm/ast/ast_main.c
+++ b/drivers/gpu/drm/ast/ast_main.c
@@ -333,7 +333,7 @@
struct ast_framebuffer *ast_fb;
int ret;
- obj = drm_gem_object_lookup(dev, filp, mode_cmd->handles[0]);
+ obj = drm_gem_object_lookup(filp, mode_cmd->handles[0]);
if (obj == NULL)
return ERR_PTR(-ENOENT);
@@ -574,7 +574,7 @@
struct drm_gem_object *obj;
struct ast_bo *bo;
- obj = drm_gem_object_lookup(dev, file, handle);
+ obj = drm_gem_object_lookup(file, handle);
if (obj == NULL)
return -ENOENT;
diff --git a/drivers/gpu/drm/ast/ast_mode.c b/drivers/gpu/drm/ast/ast_mode.c
index a965e7e..c3379226 100644
--- a/drivers/gpu/drm/ast/ast_mode.c
+++ b/drivers/gpu/drm/ast/ast_mode.c
@@ -1141,7 +1141,7 @@
if (width > AST_MAX_HWC_WIDTH || height > AST_MAX_HWC_HEIGHT)
return -EINVAL;
- obj = drm_gem_object_lookup(crtc->dev, file_priv, handle);
+ obj = drm_gem_object_lookup(file_priv, handle);
if (!obj) {
DRM_ERROR("Cannot find cursor object %x for crtc\n", handle);
return -ENOENT;
diff --git a/drivers/gpu/drm/ast/ast_ttm.c b/drivers/gpu/drm/ast/ast_ttm.c
index 08f82ea..59f2f93 100644
--- a/drivers/gpu/drm/ast/ast_ttm.c
+++ b/drivers/gpu/drm/ast/ast_ttm.c
@@ -245,6 +245,8 @@
.verify_access = ast_bo_verify_access,
.io_mem_reserve = &ast_ttm_io_mem_reserve,
.io_mem_free = &ast_ttm_io_mem_free,
+ .lru_tail = &ttm_bo_default_lru_tail,
+ .swap_lru_tail = &ttm_bo_default_swap_lru_tail,
};
int ast_mm_init(struct ast_private *ast)
diff --git a/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_crtc.c b/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_crtc.c
index 58c4f78..cf23a75 100644
--- a/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_crtc.c
+++ b/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_crtc.c
@@ -32,6 +32,23 @@
#include "atmel_hlcdc_dc.h"
/**
+ * Atmel HLCDC CRTC state structure
+ *
+ * @base: base CRTC state
+ * @output_mode: RGBXXX output mode
+ */
+struct atmel_hlcdc_crtc_state {
+ struct drm_crtc_state base;
+ unsigned int output_mode;
+};
+
+static inline struct atmel_hlcdc_crtc_state *
+drm_crtc_state_to_atmel_hlcdc_crtc_state(struct drm_crtc_state *state)
+{
+ return container_of(state, struct atmel_hlcdc_crtc_state, base);
+}
+
+/**
* Atmel HLCDC CRTC structure
*
* @base: base DRM CRTC structure
@@ -59,6 +76,7 @@
struct atmel_hlcdc_crtc *crtc = drm_crtc_to_atmel_hlcdc_crtc(c);
struct regmap *regmap = crtc->dc->hlcdc->regmap;
struct drm_display_mode *adj = &c->state->adjusted_mode;
+ struct atmel_hlcdc_crtc_state *state;
unsigned long mode_rate;
struct videomode vm;
unsigned long prate;
@@ -112,15 +130,27 @@
if (adj->flags & DRM_MODE_FLAG_NHSYNC)
cfg |= ATMEL_HLCDC_HSPOL;
+ state = drm_crtc_state_to_atmel_hlcdc_crtc_state(c->state);
+ cfg |= state->output_mode << 8;
+
regmap_update_bits(regmap, ATMEL_HLCDC_CFG(5),
ATMEL_HLCDC_HSPOL | ATMEL_HLCDC_VSPOL |
ATMEL_HLCDC_VSPDLYS | ATMEL_HLCDC_VSPDLYE |
ATMEL_HLCDC_DISPPOL | ATMEL_HLCDC_DISPDLY |
ATMEL_HLCDC_VSPSU | ATMEL_HLCDC_VSPHO |
- ATMEL_HLCDC_GUARDTIME_MASK,
+ ATMEL_HLCDC_GUARDTIME_MASK | ATMEL_HLCDC_MODE_MASK,
cfg);
}
+static bool atmel_hlcdc_crtc_mode_fixup(struct drm_crtc *c,
+ const struct drm_display_mode *mode,
+ struct drm_display_mode *adjusted_mode)
+{
+ struct atmel_hlcdc_crtc *crtc = drm_crtc_to_atmel_hlcdc_crtc(c);
+
+ return atmel_hlcdc_dc_mode_valid(crtc->dc, adjusted_mode) == MODE_OK;
+}
+
static void atmel_hlcdc_crtc_disable(struct drm_crtc *c)
{
struct drm_device *dev = c->dev;
@@ -221,15 +251,79 @@
}
}
+#define ATMEL_HLCDC_RGB444_OUTPUT BIT(0)
+#define ATMEL_HLCDC_RGB565_OUTPUT BIT(1)
+#define ATMEL_HLCDC_RGB666_OUTPUT BIT(2)
+#define ATMEL_HLCDC_RGB888_OUTPUT BIT(3)
+#define ATMEL_HLCDC_OUTPUT_MODE_MASK GENMASK(3, 0)
+
+static int atmel_hlcdc_crtc_select_output_mode(struct drm_crtc_state *state)
+{
+ unsigned int output_fmts = ATMEL_HLCDC_OUTPUT_MODE_MASK;
+ struct atmel_hlcdc_crtc_state *hstate;
+ struct drm_connector_state *cstate;
+ struct drm_connector *connector;
+ struct atmel_hlcdc_crtc *crtc;
+ int i;
+
+ crtc = drm_crtc_to_atmel_hlcdc_crtc(state->crtc);
+
+ for_each_connector_in_state(state->state, connector, cstate, i) {
+ struct drm_display_info *info = &connector->display_info;
+ unsigned int supported_fmts = 0;
+ int j;
+
+ if (!cstate->crtc)
+ continue;
+
+ for (j = 0; j < info->num_bus_formats; j++) {
+ switch (info->bus_formats[j]) {
+ case MEDIA_BUS_FMT_RGB444_1X12:
+ supported_fmts |= ATMEL_HLCDC_RGB444_OUTPUT;
+ break;
+ case MEDIA_BUS_FMT_RGB565_1X16:
+ supported_fmts |= ATMEL_HLCDC_RGB565_OUTPUT;
+ break;
+ case MEDIA_BUS_FMT_RGB666_1X18:
+ supported_fmts |= ATMEL_HLCDC_RGB666_OUTPUT;
+ break;
+ case MEDIA_BUS_FMT_RGB888_1X24:
+ supported_fmts |= ATMEL_HLCDC_RGB888_OUTPUT;
+ break;
+ default:
+ break;
+ }
+ }
+
+ if (crtc->dc->desc->conflicting_output_formats)
+ output_fmts &= supported_fmts;
+ else
+ output_fmts |= supported_fmts;
+ }
+
+ if (!output_fmts)
+ return -EINVAL;
+
+ hstate = drm_crtc_state_to_atmel_hlcdc_crtc_state(state);
+ hstate->output_mode = fls(output_fmts) - 1;
+
+ return 0;
+}
+
static int atmel_hlcdc_crtc_atomic_check(struct drm_crtc *c,
struct drm_crtc_state *s)
{
- struct atmel_hlcdc_crtc *crtc = drm_crtc_to_atmel_hlcdc_crtc(c);
+ int ret;
- if (atmel_hlcdc_dc_mode_valid(crtc->dc, &s->adjusted_mode) != MODE_OK)
- return -EINVAL;
+ ret = atmel_hlcdc_crtc_select_output_mode(s);
+ if (ret)
+ return ret;
- return atmel_hlcdc_plane_prepare_disc_area(s);
+ ret = atmel_hlcdc_plane_prepare_disc_area(s);
+ if (ret)
+ return ret;
+
+ return atmel_hlcdc_plane_prepare_ahb_routing(s);
}
static void atmel_hlcdc_crtc_atomic_begin(struct drm_crtc *c,
@@ -254,6 +348,7 @@
}
static const struct drm_crtc_helper_funcs lcdc_crtc_helper_funcs = {
+ .mode_fixup = atmel_hlcdc_crtc_mode_fixup,
.mode_set = drm_helper_crtc_mode_set,
.mode_set_nofb = atmel_hlcdc_crtc_mode_set_nofb,
.mode_set_base = drm_helper_crtc_mode_set_base,
@@ -292,13 +387,60 @@
atmel_hlcdc_crtc_finish_page_flip(drm_crtc_to_atmel_hlcdc_crtc(c));
}
+void atmel_hlcdc_crtc_reset(struct drm_crtc *crtc)
+{
+ struct atmel_hlcdc_crtc_state *state;
+
+ if (crtc->state && crtc->state->mode_blob)
+ drm_property_unreference_blob(crtc->state->mode_blob);
+
+ if (crtc->state) {
+ state = drm_crtc_state_to_atmel_hlcdc_crtc_state(crtc->state);
+ kfree(state);
+ }
+
+ state = kzalloc(sizeof(*state), GFP_KERNEL);
+ if (state) {
+ crtc->state = &state->base;
+ crtc->state->crtc = crtc;
+ }
+}
+
+static struct drm_crtc_state *
+atmel_hlcdc_crtc_duplicate_state(struct drm_crtc *crtc)
+{
+ struct atmel_hlcdc_crtc_state *state, *cur;
+
+ if (WARN_ON(!crtc->state))
+ return NULL;
+
+ state = kmalloc(sizeof(*state), GFP_KERNEL);
+ if (state)
+ __drm_atomic_helper_crtc_duplicate_state(crtc, &state->base);
+
+ cur = drm_crtc_state_to_atmel_hlcdc_crtc_state(crtc->state);
+ state->output_mode = cur->output_mode;
+
+ return &state->base;
+}
+
+static void atmel_hlcdc_crtc_destroy_state(struct drm_crtc *crtc,
+ struct drm_crtc_state *s)
+{
+ struct atmel_hlcdc_crtc_state *state;
+
+ state = drm_crtc_state_to_atmel_hlcdc_crtc_state(s);
+ __drm_atomic_helper_crtc_destroy_state(s);
+ kfree(state);
+}
+
static const struct drm_crtc_funcs atmel_hlcdc_crtc_funcs = {
.page_flip = drm_atomic_helper_page_flip,
.set_config = drm_atomic_helper_set_config,
.destroy = atmel_hlcdc_crtc_destroy,
- .reset = drm_atomic_helper_crtc_reset,
- .atomic_duplicate_state = drm_atomic_helper_crtc_duplicate_state,
- .atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
+ .reset = atmel_hlcdc_crtc_reset,
+ .atomic_duplicate_state = atmel_hlcdc_crtc_duplicate_state,
+ .atomic_destroy_state = atmel_hlcdc_crtc_destroy_state,
};
int atmel_hlcdc_crtc_create(struct drm_device *dev)
diff --git a/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_dc.c b/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_dc.c
index 3d8d164..8ded764 100644
--- a/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_dc.c
+++ b/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_dc.c
@@ -50,6 +50,10 @@
.min_height = 0,
.max_width = 1280,
.max_height = 860,
+ .max_spw = 0x3f,
+ .max_vpw = 0x3f,
+ .max_hpw = 0xff,
+ .conflicting_output_formats = true,
.nlayers = ARRAY_SIZE(atmel_hlcdc_at91sam9n12_layers),
.layers = atmel_hlcdc_at91sam9n12_layers,
};
@@ -134,6 +138,10 @@
.min_height = 0,
.max_width = 800,
.max_height = 600,
+ .max_spw = 0x3f,
+ .max_vpw = 0x3f,
+ .max_hpw = 0xff,
+ .conflicting_output_formats = true,
.nlayers = ARRAY_SIZE(atmel_hlcdc_at91sam9x5_layers),
.layers = atmel_hlcdc_at91sam9x5_layers,
};
@@ -237,6 +245,10 @@
.min_height = 0,
.max_width = 2048,
.max_height = 2048,
+ .max_spw = 0x3f,
+ .max_vpw = 0x3f,
+ .max_hpw = 0x1ff,
+ .conflicting_output_formats = true,
.nlayers = ARRAY_SIZE(atmel_hlcdc_sama5d3_layers),
.layers = atmel_hlcdc_sama5d3_layers,
};
@@ -320,6 +332,9 @@
.min_height = 0,
.max_width = 2048,
.max_height = 2048,
+ .max_spw = 0xff,
+ .max_vpw = 0xff,
+ .max_hpw = 0x3ff,
.nlayers = ARRAY_SIZE(atmel_hlcdc_sama5d4_layers),
.layers = atmel_hlcdc_sama5d4_layers,
};
@@ -358,19 +373,19 @@
int hback_porch = mode->htotal - mode->hsync_end;
int hsync_len = mode->hsync_end - mode->hsync_start;
- if (hsync_len > 0x40 || hsync_len < 1)
+ if (hsync_len > dc->desc->max_spw + 1 || hsync_len < 1)
return MODE_HSYNC;
- if (vsync_len > 0x40 || vsync_len < 1)
+ if (vsync_len > dc->desc->max_spw + 1 || vsync_len < 1)
return MODE_VSYNC;
- if (hfront_porch > 0x200 || hfront_porch < 1 ||
- hback_porch > 0x200 || hback_porch < 1 ||
+ if (hfront_porch > dc->desc->max_hpw + 1 || hfront_porch < 1 ||
+ hback_porch > dc->desc->max_hpw + 1 || hback_porch < 1 ||
mode->hdisplay < 1)
return MODE_H_ILLEGAL;
- if (vfront_porch > 0x40 || vfront_porch < 1 ||
- vback_porch > 0x40 || vback_porch < 0 ||
+ if (vfront_porch > dc->desc->max_vpw + 1 || vfront_porch < 1 ||
+ vback_porch > dc->desc->max_vpw || vback_porch < 0 ||
mode->vdisplay < 1)
return MODE_V_ILLEGAL;
@@ -427,11 +442,102 @@
}
}
+struct atmel_hlcdc_dc_commit {
+ struct work_struct work;
+ struct drm_device *dev;
+ struct drm_atomic_state *state;
+};
+
+static void
+atmel_hlcdc_dc_atomic_complete(struct atmel_hlcdc_dc_commit *commit)
+{
+ struct drm_device *dev = commit->dev;
+ struct atmel_hlcdc_dc *dc = dev->dev_private;
+ struct drm_atomic_state *old_state = commit->state;
+
+ /* Apply the atomic update. */
+ drm_atomic_helper_commit_modeset_disables(dev, old_state);
+ drm_atomic_helper_commit_planes(dev, old_state, false);
+ drm_atomic_helper_commit_modeset_enables(dev, old_state);
+
+ drm_atomic_helper_wait_for_vblanks(dev, old_state);
+
+ drm_atomic_helper_cleanup_planes(dev, old_state);
+
+ drm_atomic_state_free(old_state);
+
+ /* Complete the commit, wake up any waiter. */
+ spin_lock(&dc->commit.wait.lock);
+ dc->commit.pending = false;
+ wake_up_all_locked(&dc->commit.wait);
+ spin_unlock(&dc->commit.wait.lock);
+
+ kfree(commit);
+}
+
+static void atmel_hlcdc_dc_atomic_work(struct work_struct *work)
+{
+ struct atmel_hlcdc_dc_commit *commit =
+ container_of(work, struct atmel_hlcdc_dc_commit, work);
+
+ atmel_hlcdc_dc_atomic_complete(commit);
+}
+
+static int atmel_hlcdc_dc_atomic_commit(struct drm_device *dev,
+ struct drm_atomic_state *state,
+ bool async)
+{
+ struct atmel_hlcdc_dc *dc = dev->dev_private;
+ struct atmel_hlcdc_dc_commit *commit;
+ int ret;
+
+ ret = drm_atomic_helper_prepare_planes(dev, state);
+ if (ret)
+ return ret;
+
+ /* Allocate the commit object. */
+ commit = kzalloc(sizeof(*commit), GFP_KERNEL);
+ if (!commit) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ INIT_WORK(&commit->work, atmel_hlcdc_dc_atomic_work);
+ commit->dev = dev;
+ commit->state = state;
+
+ spin_lock(&dc->commit.wait.lock);
+ ret = wait_event_interruptible_locked(dc->commit.wait,
+ !dc->commit.pending);
+ if (ret == 0)
+ dc->commit.pending = true;
+ spin_unlock(&dc->commit.wait.lock);
+
+ if (ret) {
+ kfree(commit);
+ goto error;
+ }
+
+ /* Swap the state, this is the point of no return. */
+ drm_atomic_helper_swap_state(dev, state);
+
+ if (async)
+ queue_work(dc->wq, &commit->work);
+ else
+ atmel_hlcdc_dc_atomic_complete(commit);
+
+ return 0;
+
+error:
+ drm_atomic_helper_cleanup_planes(dev, state);
+ return ret;
+}
+
static const struct drm_mode_config_funcs mode_config_funcs = {
.fb_create = atmel_hlcdc_fb_create,
.output_poll_changed = atmel_hlcdc_fb_output_poll_changed,
.atomic_check = drm_atomic_helper_check,
- .atomic_commit = drm_atomic_helper_commit,
+ .atomic_commit = atmel_hlcdc_dc_atomic_commit,
};
static int atmel_hlcdc_dc_modeset_init(struct drm_device *dev)
@@ -445,7 +551,7 @@
ret = atmel_hlcdc_create_outputs(dev);
if (ret) {
- dev_err(dev->dev, "failed to create panel: %d\n", ret);
+ dev_err(dev->dev, "failed to create HLCDC outputs: %d\n", ret);
return ret;
}
@@ -509,6 +615,7 @@
if (!dc->wq)
return -ENOMEM;
+ init_waitqueue_head(&dc->commit.wait);
dc->desc = match->data;
dc->hlcdc = dev_get_drvdata(dev->dev->parent);
dev->dev_private = dc;
@@ -584,38 +691,10 @@
destroy_workqueue(dc->wq);
}
-static int atmel_hlcdc_dc_connector_plug_all(struct drm_device *dev)
-{
- struct drm_connector *connector, *failed;
- int ret;
-
- mutex_lock(&dev->mode_config.mutex);
- list_for_each_entry(connector, &dev->mode_config.connector_list, head) {
- ret = drm_connector_register(connector);
- if (ret) {
- failed = connector;
- goto err;
- }
- }
- mutex_unlock(&dev->mode_config.mutex);
- return 0;
-
-err:
- list_for_each_entry(connector, &dev->mode_config.connector_list, head) {
- if (failed == connector)
- break;
-
- drm_connector_unregister(connector);
- }
- mutex_unlock(&dev->mode_config.mutex);
-
- return ret;
-}
-
static void atmel_hlcdc_dc_connector_unplug_all(struct drm_device *dev)
{
mutex_lock(&dev->mode_config.mutex);
- drm_connector_unplug_all(dev);
+ drm_connector_unregister_all(dev);
mutex_unlock(&dev->mode_config.mutex);
}
@@ -736,7 +815,7 @@
if (ret)
goto err_unload;
- ret = atmel_hlcdc_dc_connector_plug_all(ddev);
+ ret = drm_connector_register_all(ddev);
if (ret)
goto err_unregister;
diff --git a/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_dc.h b/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_dc.h
index fed517f..7a47f8c 100644
--- a/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_dc.h
+++ b/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_dc.h
@@ -50,6 +50,11 @@
* @min_height: minimum height supported by the Display Controller
* @max_width: maximum width supported by the Display Controller
* @max_height: maximum height supported by the Display Controller
+ * @max_spw: maximum vertical/horizontal pulse width
+ * @max_vpw: maximum vertical back/front porch width
+ * @max_hpw: maximum horizontal back/front porch width
+ * @conflicting_output_formats: true if RGBXXX output formats conflict with
+ * each other.
* @layers: a layer description table describing available layers
* @nlayers: layer description table size
*/
@@ -58,6 +63,10 @@
int min_height;
int max_width;
int max_height;
+ int max_spw;
+ int max_vpw;
+ int max_hpw;
+ bool conflicting_output_formats;
const struct atmel_hlcdc_layer_desc *layers;
int nlayers;
};
@@ -128,6 +137,7 @@
* @planes: instantiated planes
* @layers: active HLCDC layer
* @wq: display controller workqueue
+ * @commit: used for async commit handling
*/
struct atmel_hlcdc_dc {
const struct atmel_hlcdc_dc_desc *desc;
@@ -137,6 +147,10 @@
struct atmel_hlcdc_planes *planes;
struct atmel_hlcdc_layer *layers[ATMEL_HLCDC_MAX_LAYERS];
struct workqueue_struct *wq;
+ struct {
+ wait_queue_head_t wait;
+ bool pending;
+ } commit;
};
extern struct atmel_hlcdc_formats atmel_hlcdc_plane_rgb_formats;
@@ -149,6 +163,7 @@
atmel_hlcdc_create_planes(struct drm_device *dev);
int atmel_hlcdc_plane_prepare_disc_area(struct drm_crtc_state *c_state);
+int atmel_hlcdc_plane_prepare_ahb_routing(struct drm_crtc_state *c_state);
void atmel_hlcdc_crtc_irq(struct drm_crtc *c);
diff --git a/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_output.c b/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_output.c
index 0f7ec01..39802c0 100644
--- a/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_output.c
+++ b/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_output.c
@@ -27,16 +27,6 @@
#include "atmel_hlcdc_dc.h"
/**
- * Atmel HLCDC RGB output mode
- */
-enum atmel_hlcdc_connector_rgb_mode {
- ATMEL_HLCDC_CONNECTOR_RGB444,
- ATMEL_HLCDC_CONNECTOR_RGB565,
- ATMEL_HLCDC_CONNECTOR_RGB666,
- ATMEL_HLCDC_CONNECTOR_RGB888,
-};
-
-/**
* Atmel HLCDC RGB connector structure
*
* This structure stores RGB slave device information.
@@ -44,13 +34,13 @@
* @connector: DRM connector
* @encoder: DRM encoder
* @dc: pointer to the atmel_hlcdc_dc structure
- * @dpms: current DPMS mode
+ * @panel: panel connected on the RGB output
*/
struct atmel_hlcdc_rgb_output {
struct drm_connector connector;
struct drm_encoder encoder;
struct atmel_hlcdc_dc *dc;
- int dpms;
+ struct drm_panel *panel;
};
static inline struct atmel_hlcdc_rgb_output *
@@ -66,91 +56,31 @@
return container_of(encoder, struct atmel_hlcdc_rgb_output, encoder);
}
-/**
- * Atmel HLCDC Panel device structure
- *
- * This structure is specialization of the slave device structure to
- * interface with drm panels.
- *
- * @base: base slave device fields
- * @panel: drm panel attached to this slave device
- */
-struct atmel_hlcdc_panel {
- struct atmel_hlcdc_rgb_output base;
- struct drm_panel *panel;
-};
-
-static inline struct atmel_hlcdc_panel *
-atmel_hlcdc_rgb_output_to_panel(struct atmel_hlcdc_rgb_output *output)
-{
- return container_of(output, struct atmel_hlcdc_panel, base);
-}
-
-static void atmel_hlcdc_panel_encoder_enable(struct drm_encoder *encoder)
+static void atmel_hlcdc_rgb_encoder_enable(struct drm_encoder *encoder)
{
struct atmel_hlcdc_rgb_output *rgb =
drm_encoder_to_atmel_hlcdc_rgb_output(encoder);
- struct atmel_hlcdc_panel *panel = atmel_hlcdc_rgb_output_to_panel(rgb);
- drm_panel_enable(panel->panel);
-}
-
-static void atmel_hlcdc_panel_encoder_disable(struct drm_encoder *encoder)
-{
- struct atmel_hlcdc_rgb_output *rgb =
- drm_encoder_to_atmel_hlcdc_rgb_output(encoder);
- struct atmel_hlcdc_panel *panel = atmel_hlcdc_rgb_output_to_panel(rgb);
-
- drm_panel_disable(panel->panel);
-}
-
-static bool
-atmel_hlcdc_panel_encoder_mode_fixup(struct drm_encoder *encoder,
- const struct drm_display_mode *mode,
- struct drm_display_mode *adjusted)
-{
- return true;
-}
-
-static void
-atmel_hlcdc_rgb_encoder_mode_set(struct drm_encoder *encoder,
- struct drm_display_mode *mode,
- struct drm_display_mode *adjusted)
-{
- struct atmel_hlcdc_rgb_output *rgb =
- drm_encoder_to_atmel_hlcdc_rgb_output(encoder);
- struct drm_display_info *info = &rgb->connector.display_info;
- unsigned int cfg;
-
- cfg = 0;
-
- if (info->num_bus_formats) {
- switch (info->bus_formats[0]) {
- case MEDIA_BUS_FMT_RGB565_1X16:
- cfg |= ATMEL_HLCDC_CONNECTOR_RGB565 << 8;
- break;
- case MEDIA_BUS_FMT_RGB666_1X18:
- cfg |= ATMEL_HLCDC_CONNECTOR_RGB666 << 8;
- break;
- case MEDIA_BUS_FMT_RGB888_1X24:
- cfg |= ATMEL_HLCDC_CONNECTOR_RGB888 << 8;
- break;
- case MEDIA_BUS_FMT_RGB444_1X12:
- default:
- break;
- }
+ if (rgb->panel) {
+ drm_panel_prepare(rgb->panel);
+ drm_panel_enable(rgb->panel);
}
+}
- regmap_update_bits(rgb->dc->hlcdc->regmap, ATMEL_HLCDC_CFG(5),
- ATMEL_HLCDC_MODE_MASK,
- cfg);
+static void atmel_hlcdc_rgb_encoder_disable(struct drm_encoder *encoder)
+{
+ struct atmel_hlcdc_rgb_output *rgb =
+ drm_encoder_to_atmel_hlcdc_rgb_output(encoder);
+
+ if (rgb->panel) {
+ drm_panel_disable(rgb->panel);
+ drm_panel_unprepare(rgb->panel);
+ }
}
static const struct drm_encoder_helper_funcs atmel_hlcdc_panel_encoder_helper_funcs = {
- .mode_fixup = atmel_hlcdc_panel_encoder_mode_fixup,
- .mode_set = atmel_hlcdc_rgb_encoder_mode_set,
- .disable = atmel_hlcdc_panel_encoder_disable,
- .enable = atmel_hlcdc_panel_encoder_enable,
+ .disable = atmel_hlcdc_rgb_encoder_disable,
+ .enable = atmel_hlcdc_rgb_encoder_enable,
};
static void atmel_hlcdc_rgb_encoder_destroy(struct drm_encoder *encoder)
@@ -167,9 +97,11 @@
{
struct atmel_hlcdc_rgb_output *rgb =
drm_connector_to_atmel_hlcdc_rgb_output(connector);
- struct atmel_hlcdc_panel *panel = atmel_hlcdc_rgb_output_to_panel(rgb);
- return panel->panel->funcs->get_modes(panel->panel);
+ if (rgb->panel)
+ return rgb->panel->funcs->get_modes(rgb->panel);
+
+ return 0;
}
static int atmel_hlcdc_rgb_mode_valid(struct drm_connector *connector,
@@ -201,7 +133,13 @@
static enum drm_connector_status
atmel_hlcdc_panel_connector_detect(struct drm_connector *connector, bool force)
{
- return connector_status_connected;
+ struct atmel_hlcdc_rgb_output *rgb =
+ drm_connector_to_atmel_hlcdc_rgb_output(connector);
+
+ if (rgb->panel)
+ return connector_status_connected;
+
+ return connector_status_disconnected;
}
static void
@@ -209,9 +147,10 @@
{
struct atmel_hlcdc_rgb_output *rgb =
drm_connector_to_atmel_hlcdc_rgb_output(connector);
- struct atmel_hlcdc_panel *panel = atmel_hlcdc_rgb_output_to_panel(rgb);
- drm_panel_detach(panel->panel);
+ if (rgb->panel)
+ drm_panel_detach(rgb->panel);
+
drm_connector_cleanup(connector);
}
@@ -225,88 +164,122 @@
.atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
};
-static int atmel_hlcdc_create_panel_output(struct drm_device *dev,
- struct of_endpoint *ep)
+static int atmel_hlcdc_check_endpoint(struct drm_device *dev,
+ const struct of_endpoint *ep)
{
- struct atmel_hlcdc_dc *dc = dev->dev_private;
struct device_node *np;
- struct drm_panel *p = NULL;
- struct atmel_hlcdc_panel *panel;
- int ret;
+ void *obj;
np = of_graph_get_remote_port_parent(ep->local_node);
- if (!np)
- return -EINVAL;
- p = of_drm_find_panel(np);
+ obj = of_drm_find_panel(np);
+ if (!obj)
+ obj = of_drm_find_bridge(np);
+
of_node_put(np);
- if (!p)
- return -EPROBE_DEFER;
+ return obj ? 0 : -EPROBE_DEFER;
+}
- panel = devm_kzalloc(dev->dev, sizeof(*panel), GFP_KERNEL);
- if (!panel)
+static int atmel_hlcdc_attach_endpoint(struct drm_device *dev,
+ const struct of_endpoint *ep)
+{
+ struct atmel_hlcdc_dc *dc = dev->dev_private;
+ struct atmel_hlcdc_rgb_output *output;
+ struct device_node *np;
+ struct drm_panel *panel;
+ struct drm_bridge *bridge;
+ int ret;
+
+ output = devm_kzalloc(dev->dev, sizeof(*output), GFP_KERNEL);
+ if (!output)
return -EINVAL;
- panel->base.dpms = DRM_MODE_DPMS_OFF;
+ output->dc = dc;
- panel->base.dc = dc;
-
- drm_encoder_helper_add(&panel->base.encoder,
+ drm_encoder_helper_add(&output->encoder,
&atmel_hlcdc_panel_encoder_helper_funcs);
- ret = drm_encoder_init(dev, &panel->base.encoder,
+ ret = drm_encoder_init(dev, &output->encoder,
&atmel_hlcdc_panel_encoder_funcs,
- DRM_MODE_ENCODER_LVDS, NULL);
+ DRM_MODE_ENCODER_NONE, NULL);
if (ret)
return ret;
- panel->base.connector.dpms = DRM_MODE_DPMS_OFF;
- panel->base.connector.polled = DRM_CONNECTOR_POLL_CONNECT;
- drm_connector_helper_add(&panel->base.connector,
- &atmel_hlcdc_panel_connector_helper_funcs);
- ret = drm_connector_init(dev, &panel->base.connector,
- &atmel_hlcdc_panel_connector_funcs,
- DRM_MODE_CONNECTOR_LVDS);
- if (ret)
- goto err_encoder_cleanup;
+ output->encoder.possible_crtcs = 0x1;
- drm_mode_connector_attach_encoder(&panel->base.connector,
- &panel->base.encoder);
- panel->base.encoder.possible_crtcs = 0x1;
+ np = of_graph_get_remote_port_parent(ep->local_node);
- drm_panel_attach(p, &panel->base.connector);
- panel->panel = p;
+ ret = -EPROBE_DEFER;
- return 0;
+ panel = of_drm_find_panel(np);
+ if (panel) {
+ of_node_put(np);
+ output->connector.dpms = DRM_MODE_DPMS_OFF;
+ output->connector.polled = DRM_CONNECTOR_POLL_CONNECT;
+ drm_connector_helper_add(&output->connector,
+ &atmel_hlcdc_panel_connector_helper_funcs);
+ ret = drm_connector_init(dev, &output->connector,
+ &atmel_hlcdc_panel_connector_funcs,
+ DRM_MODE_CONNECTOR_Unknown);
+ if (ret)
+ goto err_encoder_cleanup;
+
+ drm_mode_connector_attach_encoder(&output->connector,
+ &output->encoder);
+
+ ret = drm_panel_attach(panel, &output->connector);
+ if (ret) {
+ drm_connector_cleanup(&output->connector);
+ goto err_encoder_cleanup;
+ }
+
+ output->panel = panel;
+
+ return 0;
+ }
+
+ bridge = of_drm_find_bridge(np);
+ of_node_put(np);
+
+ if (bridge) {
+ output->encoder.bridge = bridge;
+ bridge->encoder = &output->encoder;
+ ret = drm_bridge_attach(dev, bridge);
+ if (!ret)
+ return 0;
+ }
err_encoder_cleanup:
- drm_encoder_cleanup(&panel->base.encoder);
+ drm_encoder_cleanup(&output->encoder);
return ret;
}
int atmel_hlcdc_create_outputs(struct drm_device *dev)
{
- struct device_node *port_np, *np;
+ struct device_node *ep_np = NULL;
struct of_endpoint ep;
int ret;
- port_np = of_get_child_by_name(dev->dev->of_node, "port");
- if (!port_np)
- return -EINVAL;
+ for_each_endpoint_of_node(dev->dev->of_node, ep_np) {
+ ret = of_graph_parse_endpoint(ep_np, &ep);
+ if (!ret)
+ ret = atmel_hlcdc_check_endpoint(dev, &ep);
- np = of_get_child_by_name(port_np, "endpoint");
- of_node_put(port_np);
+ of_node_put(ep_np);
+ if (ret)
+ return ret;
+ }
- if (!np)
- return -EINVAL;
+ for_each_endpoint_of_node(dev->dev->of_node, ep_np) {
+ ret = of_graph_parse_endpoint(ep_np, &ep);
+ if (!ret)
+ ret = atmel_hlcdc_attach_endpoint(dev, &ep);
- ret = of_graph_parse_endpoint(np, &ep);
- of_node_put(port_np);
+ of_node_put(ep_np);
+ if (ret)
+ return ret;
+ }
- if (ret)
- return ret;
-
- /* We currently only support panel output */
- return atmel_hlcdc_create_panel_output(dev, &ep);
+ return 0;
}
diff --git a/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_plane.c b/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_plane.c
index d65dcae..aef3ca8 100644
--- a/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_plane.c
+++ b/drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_plane.c
@@ -37,6 +37,7 @@
* @xstride: value to add to the pixel pointer between each line
* @pstride: value to add to the pixel pointer between each pixel
* @nplanes: number of planes (deduced from pixel_format)
+ * @prepared: plane update has been prepared
*/
struct atmel_hlcdc_plane_state {
struct drm_plane_state base;
@@ -58,12 +59,15 @@
int disc_w;
int disc_h;
+ int ahb_id;
+
/* These fields are private and should not be touched */
int bpp[ATMEL_HLCDC_MAX_PLANES];
unsigned int offsets[ATMEL_HLCDC_MAX_PLANES];
int xstride[ATMEL_HLCDC_MAX_PLANES];
int pstride[ATMEL_HLCDC_MAX_PLANES];
int nplanes;
+ bool prepared;
};
static inline struct atmel_hlcdc_plane_state *
@@ -359,8 +363,10 @@
atmel_hlcdc_layer_update_cfg(&plane->layer,
ATMEL_HLCDC_LAYER_DMA_CFG_ID,
- ATMEL_HLCDC_LAYER_DMA_BLEN_MASK,
- ATMEL_HLCDC_LAYER_DMA_BLEN_INCR16);
+ ATMEL_HLCDC_LAYER_DMA_BLEN_MASK |
+ ATMEL_HLCDC_LAYER_DMA_SIF,
+ ATMEL_HLCDC_LAYER_DMA_BLEN_INCR16 |
+ state->ahb_id);
atmel_hlcdc_layer_update_cfg(&plane->layer, layout->general_config,
ATMEL_HLCDC_LAYER_ITER2BL |
@@ -435,6 +441,41 @@
}
}
+int atmel_hlcdc_plane_prepare_ahb_routing(struct drm_crtc_state *c_state)
+{
+ unsigned int ahb_load[2] = { };
+ struct drm_plane *plane;
+
+ drm_atomic_crtc_state_for_each_plane(plane, c_state) {
+ struct atmel_hlcdc_plane_state *plane_state;
+ struct drm_plane_state *plane_s;
+ unsigned int pixels, load = 0;
+ int i;
+
+ plane_s = drm_atomic_get_plane_state(c_state->state, plane);
+ if (IS_ERR(plane_s))
+ return PTR_ERR(plane_s);
+
+ plane_state =
+ drm_plane_state_to_atmel_hlcdc_plane_state(plane_s);
+
+ pixels = (plane_state->src_w * plane_state->src_h) -
+ (plane_state->disc_w * plane_state->disc_h);
+
+ for (i = 0; i < plane_state->nplanes; i++)
+ load += pixels * plane_state->bpp[i];
+
+ if (ahb_load[0] <= ahb_load[1])
+ plane_state->ahb_id = 0;
+ else
+ plane_state->ahb_id = 1;
+
+ ahb_load[plane_state->ahb_id] += load;
+ }
+
+ return 0;
+}
+
int
atmel_hlcdc_plane_prepare_disc_area(struct drm_crtc_state *c_state)
{
@@ -714,12 +755,54 @@
static int atmel_hlcdc_plane_prepare_fb(struct drm_plane *p,
const struct drm_plane_state *new_state)
{
+ /*
+ * FIXME: we should avoid this const -> non-const cast but it's
+ * currently the only solution we have to modify the ->prepared
+ * state and rollback the update request.
+ * Ideally, we should rework the code to attach all the resources
+ * to atmel_hlcdc_plane_state (including the DMA desc allocation),
+ * but this require a complete rework of the atmel_hlcdc_layer
+ * code.
+ */
+ struct drm_plane_state *s = (struct drm_plane_state *)new_state;
struct atmel_hlcdc_plane *plane = drm_plane_to_atmel_hlcdc_plane(p);
+ struct atmel_hlcdc_plane_state *state =
+ drm_plane_state_to_atmel_hlcdc_plane_state(s);
+ int ret;
- if (!new_state->fb)
- return 0;
+ ret = atmel_hlcdc_layer_update_start(&plane->layer);
+ if (!ret)
+ state->prepared = true;
- return atmel_hlcdc_layer_update_start(&plane->layer);
+ return ret;
+}
+
+static void atmel_hlcdc_plane_cleanup_fb(struct drm_plane *p,
+ const struct drm_plane_state *old_state)
+{
+ /*
+ * FIXME: we should avoid this const -> non-const cast but it's
+ * currently the only solution we have to modify the ->prepared
+ * state and rollback the update request.
+ * Ideally, we should rework the code to attach all the resources
+ * to atmel_hlcdc_plane_state (including the DMA desc allocation),
+ * but this require a complete rework of the atmel_hlcdc_layer
+ * code.
+ */
+ struct drm_plane_state *s = (struct drm_plane_state *)old_state;
+ struct atmel_hlcdc_plane *plane = drm_plane_to_atmel_hlcdc_plane(p);
+ struct atmel_hlcdc_plane_state *state =
+ drm_plane_state_to_atmel_hlcdc_plane_state(s);
+
+ /*
+ * The Request has already been applied or cancelled, nothing to do
+ * here.
+ */
+ if (!state->prepared)
+ return;
+
+ atmel_hlcdc_layer_update_rollback(&plane->layer);
+ state->prepared = false;
}
static void atmel_hlcdc_plane_atomic_update(struct drm_plane *p,
@@ -844,6 +927,7 @@
static struct drm_plane_helper_funcs atmel_hlcdc_layer_plane_helper_funcs = {
.prepare_fb = atmel_hlcdc_plane_prepare_fb,
+ .cleanup_fb = atmel_hlcdc_plane_cleanup_fb,
.atomic_check = atmel_hlcdc_plane_atomic_check,
.atomic_update = atmel_hlcdc_plane_atomic_update,
.atomic_disable = atmel_hlcdc_plane_atomic_disable,
@@ -883,6 +967,7 @@
return NULL;
copy->disc_updated = false;
+ copy->prepared = false;
if (copy->base.fb)
drm_framebuffer_reference(copy->base.fb);
diff --git a/drivers/gpu/drm/bochs/bochs_fbdev.c b/drivers/gpu/drm/bochs/bochs_fbdev.c
index 7520bf8..e1ec498 100644
--- a/drivers/gpu/drm/bochs/bochs_fbdev.c
+++ b/drivers/gpu/drm/bochs/bochs_fbdev.c
@@ -82,7 +82,7 @@
bo = gem_to_bochs_bo(gobj);
- ret = ttm_bo_reserve(&bo->bo, true, false, false, NULL);
+ ret = ttm_bo_reserve(&bo->bo, true, false, NULL);
if (ret)
return ret;
@@ -162,22 +162,7 @@
return 0;
}
-void bochs_fb_gamma_set(struct drm_crtc *crtc, u16 red, u16 green,
- u16 blue, int regno)
-{
-}
-
-void bochs_fb_gamma_get(struct drm_crtc *crtc, u16 *red, u16 *green,
- u16 *blue, int regno)
-{
- *red = regno;
- *green = regno;
- *blue = regno;
-}
-
static const struct drm_fb_helper_funcs bochs_fb_helper_funcs = {
- .gamma_set = bochs_fb_gamma_set,
- .gamma_get = bochs_fb_gamma_get,
.fb_probe = bochsfb_create,
};
diff --git a/drivers/gpu/drm/bochs/bochs_kms.c b/drivers/gpu/drm/bochs/bochs_kms.c
index 96926f0..207a2cb 100644
--- a/drivers/gpu/drm/bochs/bochs_kms.c
+++ b/drivers/gpu/drm/bochs/bochs_kms.c
@@ -43,7 +43,7 @@
if (old_fb) {
bochs_fb = to_bochs_framebuffer(old_fb);
bo = gem_to_bochs_bo(bochs_fb->obj);
- ret = ttm_bo_reserve(&bo->bo, true, false, false, NULL);
+ ret = ttm_bo_reserve(&bo->bo, true, false, NULL);
if (ret) {
DRM_ERROR("failed to reserve old_fb bo\n");
} else {
@@ -57,7 +57,7 @@
bochs_fb = to_bochs_framebuffer(crtc->primary->fb);
bo = gem_to_bochs_bo(bochs_fb->obj);
- ret = ttm_bo_reserve(&bo->bo, true, false, false, NULL);
+ ret = ttm_bo_reserve(&bo->bo, true, false, NULL);
if (ret)
return ret;
@@ -93,11 +93,6 @@
{
}
-static void bochs_crtc_gamma_set(struct drm_crtc *crtc, u16 *red, u16 *green,
- u16 *blue, uint32_t start, uint32_t size)
-{
-}
-
static int bochs_crtc_page_flip(struct drm_crtc *crtc,
struct drm_framebuffer *fb,
struct drm_pending_vblank_event *event,
@@ -120,7 +115,6 @@
/* These provide the minimum set of functions required to handle a CRTC */
static const struct drm_crtc_funcs bochs_crtc_funcs = {
- .gamma_set = bochs_crtc_gamma_set,
.set_config = drm_crtc_helper_set_config,
.destroy = drm_crtc_cleanup,
.page_flip = bochs_crtc_page_flip,
@@ -140,7 +134,6 @@
struct drm_crtc *crtc = &bochs->crtc;
drm_crtc_init(dev, crtc, &bochs_crtc_funcs);
- drm_mode_crtc_set_gamma_size(crtc, 256);
drm_crtc_helper_add(crtc, &bochs_helper_funcs);
}
diff --git a/drivers/gpu/drm/bochs/bochs_mm.c b/drivers/gpu/drm/bochs/bochs_mm.c
index d812ad0..6cf912c 100644
--- a/drivers/gpu/drm/bochs/bochs_mm.c
+++ b/drivers/gpu/drm/bochs/bochs_mm.c
@@ -212,6 +212,8 @@
.verify_access = bochs_bo_verify_access,
.io_mem_reserve = &bochs_ttm_io_mem_reserve,
.io_mem_free = &bochs_ttm_io_mem_free,
+ .lru_tail = &ttm_bo_default_lru_tail,
+ .swap_lru_tail = &ttm_bo_default_swap_lru_tail,
};
int bochs_mm_init(struct bochs_device *bochs)
@@ -456,7 +458,7 @@
struct drm_gem_object *obj;
struct bochs_bo *bo;
- obj = drm_gem_object_lookup(dev, file, handle);
+ obj = drm_gem_object_lookup(file, handle);
if (obj == NULL)
return -ENOENT;
@@ -518,7 +520,7 @@
if (mode_cmd->pixel_format != DRM_FORMAT_XRGB8888)
return ERR_PTR(-ENOENT);
- obj = drm_gem_object_lookup(dev, filp, mode_cmd->handles[0]);
+ obj = drm_gem_object_lookup(filp, mode_cmd->handles[0]);
if (obj == NULL)
return ERR_PTR(-ENOENT);
diff --git a/drivers/gpu/drm/bridge/Kconfig b/drivers/gpu/drm/bridge/Kconfig
index 27e2022..8f7423f 100644
--- a/drivers/gpu/drm/bridge/Kconfig
+++ b/drivers/gpu/drm/bridge/Kconfig
@@ -7,6 +7,16 @@
menu "Display Interface Bridges"
depends on DRM && DRM_BRIDGE
+config DRM_ANALOGIX_ANX78XX
+ tristate "Analogix ANX78XX bridge"
+ select DRM_KMS_HELPER
+ select REGMAP_I2C
+ ---help---
+ ANX78XX is an ultra-low Full-HD SlimPort transmitter
+ designed for portable devices. The ANX78XX transforms
+ the HDMI output of an application processor to MyDP
+ or DisplayPort.
+
config DRM_DW_HDMI
tristate
select DRM_KMS_HELPER
@@ -40,4 +50,6 @@
---help---
Parade eDP-LVDS bridge chip driver.
+source "drivers/gpu/drm/bridge/analogix/Kconfig"
+
endmenu
diff --git a/drivers/gpu/drm/bridge/Makefile b/drivers/gpu/drm/bridge/Makefile
index f13c33d..96b13b3 100644
--- a/drivers/gpu/drm/bridge/Makefile
+++ b/drivers/gpu/drm/bridge/Makefile
@@ -1,6 +1,8 @@
ccflags-y := -Iinclude/drm
+obj-$(CONFIG_DRM_ANALOGIX_ANX78XX) += analogix-anx78xx.o
obj-$(CONFIG_DRM_DW_HDMI) += dw-hdmi.o
obj-$(CONFIG_DRM_DW_HDMI_AHB_AUDIO) += dw-hdmi-ahb-audio.o
obj-$(CONFIG_DRM_NXP_PTN3460) += nxp-ptn3460.o
obj-$(CONFIG_DRM_PARADE_PS8622) += parade-ps8622.o
+obj-$(CONFIG_DRM_ANALOGIX_DP) += analogix/
diff --git a/drivers/gpu/drm/bridge/analogix-anx78xx.c b/drivers/gpu/drm/bridge/analogix-anx78xx.c
new file mode 100644
index 0000000..d087b05
--- /dev/null
+++ b/drivers/gpu/drm/bridge/analogix-anx78xx.c
@@ -0,0 +1,1514 @@
+/*
+ * Copyright(c) 2016, Analogix Semiconductor.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * Based on anx7808 driver obtained from chromeos with copyright:
+ * Copyright(c) 2013, Google Inc.
+ *
+ */
+#include <linux/delay.h>
+#include <linux/err.h>
+#include <linux/interrupt.h>
+#include <linux/i2c.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/of_gpio.h>
+#include <linux/of_irq.h>
+#include <linux/of_platform.h>
+#include <linux/regmap.h>
+#include <linux/types.h>
+#include <linux/gpio/consumer.h>
+#include <linux/regulator/consumer.h>
+
+#include <drm/drmP.h>
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_crtc.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_dp_helper.h>
+#include <drm/drm_edid.h>
+
+#include "analogix-anx78xx.h"
+
+#define I2C_NUM_ADDRESSES 5
+#define I2C_IDX_TX_P0 0
+#define I2C_IDX_TX_P1 1
+#define I2C_IDX_TX_P2 2
+#define I2C_IDX_RX_P0 3
+#define I2C_IDX_RX_P1 4
+
+#define XTAL_CLK 270 /* 27M */
+#define AUX_CH_BUFFER_SIZE 16
+#define AUX_WAIT_TIMEOUT_MS 15
+
+static const u8 anx78xx_i2c_addresses[] = {
+ [I2C_IDX_TX_P0] = TX_P0,
+ [I2C_IDX_TX_P1] = TX_P1,
+ [I2C_IDX_TX_P2] = TX_P2,
+ [I2C_IDX_RX_P0] = RX_P0,
+ [I2C_IDX_RX_P1] = RX_P1,
+};
+
+struct anx78xx_platform_data {
+ struct regulator *dvdd10;
+ struct gpio_desc *gpiod_hpd;
+ struct gpio_desc *gpiod_pd;
+ struct gpio_desc *gpiod_reset;
+
+ int hpd_irq;
+ int intp_irq;
+};
+
+struct anx78xx {
+ struct drm_dp_aux aux;
+ struct drm_bridge bridge;
+ struct i2c_client *client;
+ struct edid *edid;
+ struct drm_connector connector;
+ struct drm_dp_link link;
+ struct anx78xx_platform_data pdata;
+ struct mutex lock;
+
+ /*
+ * I2C Slave addresses of ANX7814 are mapped as TX_P0, TX_P1, TX_P2,
+ * RX_P0 and RX_P1.
+ */
+ struct i2c_client *i2c_dummy[I2C_NUM_ADDRESSES];
+ struct regmap *map[I2C_NUM_ADDRESSES];
+
+ u16 chipid;
+ u8 dpcd[DP_RECEIVER_CAP_SIZE];
+
+ bool powered;
+};
+
+static inline struct anx78xx *connector_to_anx78xx(struct drm_connector *c)
+{
+ return container_of(c, struct anx78xx, connector);
+}
+
+static inline struct anx78xx *bridge_to_anx78xx(struct drm_bridge *bridge)
+{
+ return container_of(bridge, struct anx78xx, bridge);
+}
+
+static int anx78xx_set_bits(struct regmap *map, u8 reg, u8 mask)
+{
+ return regmap_update_bits(map, reg, mask, mask);
+}
+
+static int anx78xx_clear_bits(struct regmap *map, u8 reg, u8 mask)
+{
+ return regmap_update_bits(map, reg, mask, 0);
+}
+
+static bool anx78xx_aux_op_finished(struct anx78xx *anx78xx)
+{
+ unsigned int value;
+ int err;
+
+ err = regmap_read(anx78xx->map[I2C_IDX_TX_P0], SP_DP_AUX_CH_CTRL2_REG,
+ &value);
+ if (err < 0)
+ return false;
+
+ return (value & SP_AUX_EN) == 0;
+}
+
+static int anx78xx_aux_wait(struct anx78xx *anx78xx)
+{
+ unsigned long timeout;
+ unsigned int status;
+ int err;
+
+ timeout = jiffies + msecs_to_jiffies(AUX_WAIT_TIMEOUT_MS) + 1;
+
+ while (!anx78xx_aux_op_finished(anx78xx)) {
+ if (time_after(jiffies, timeout)) {
+ if (!anx78xx_aux_op_finished(anx78xx)) {
+ DRM_ERROR("Timed out waiting AUX to finish\n");
+ return -ETIMEDOUT;
+ }
+
+ break;
+ }
+
+ usleep_range(1000, 2000);
+ }
+
+ /* Read the AUX channel access status */
+ err = regmap_read(anx78xx->map[I2C_IDX_TX_P0], SP_AUX_CH_STATUS_REG,
+ &status);
+ if (err < 0) {
+ DRM_ERROR("Failed to read from AUX channel: %d\n", err);
+ return err;
+ }
+
+ if (status & SP_AUX_STATUS) {
+ DRM_ERROR("Failed to wait for AUX channel (status: %02x)\n",
+ status);
+ return -ETIMEDOUT;
+ }
+
+ return 0;
+}
+
+static int anx78xx_aux_address(struct anx78xx *anx78xx, unsigned int addr)
+{
+ int err;
+
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P0], SP_AUX_ADDR_7_0_REG,
+ addr & 0xff);
+ if (err)
+ return err;
+
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P0], SP_AUX_ADDR_15_8_REG,
+ (addr & 0xff00) >> 8);
+ if (err)
+ return err;
+
+ /*
+ * DP AUX CH Address Register #2, only update bits[3:0]
+ * [7:4] RESERVED
+ * [3:0] AUX_ADDR[19:16], Register control AUX CH address.
+ */
+ err = regmap_update_bits(anx78xx->map[I2C_IDX_TX_P0],
+ SP_AUX_ADDR_19_16_REG,
+ SP_AUX_ADDR_19_16_MASK,
+ (addr & 0xf0000) >> 16);
+
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static ssize_t anx78xx_aux_transfer(struct drm_dp_aux *aux,
+ struct drm_dp_aux_msg *msg)
+{
+ struct anx78xx *anx78xx = container_of(aux, struct anx78xx, aux);
+ u8 ctrl1 = msg->request;
+ u8 ctrl2 = SP_AUX_EN;
+ u8 *buffer = msg->buffer;
+ int err;
+
+ /* The DP AUX transmit and receive buffer has 16 bytes. */
+ if (WARN_ON(msg->size > AUX_CH_BUFFER_SIZE))
+ return -E2BIG;
+
+ /* Zero-sized messages specify address-only transactions. */
+ if (msg->size < 1)
+ ctrl2 |= SP_ADDR_ONLY;
+ else /* For non-zero-sized set the length field. */
+ ctrl1 |= (msg->size - 1) << SP_AUX_LENGTH_SHIFT;
+
+ if ((msg->request & DP_AUX_I2C_READ) == 0) {
+ /* When WRITE | MOT write values to data buffer */
+ err = regmap_bulk_write(anx78xx->map[I2C_IDX_TX_P0],
+ SP_DP_BUF_DATA0_REG, buffer,
+ msg->size);
+ if (err)
+ return err;
+ }
+
+ /* Write address and request */
+ err = anx78xx_aux_address(anx78xx, msg->address);
+ if (err)
+ return err;
+
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P0], SP_DP_AUX_CH_CTRL1_REG,
+ ctrl1);
+ if (err)
+ return err;
+
+ /* Start transaction */
+ err = regmap_update_bits(anx78xx->map[I2C_IDX_TX_P0],
+ SP_DP_AUX_CH_CTRL2_REG, SP_ADDR_ONLY |
+ SP_AUX_EN, ctrl2);
+ if (err)
+ return err;
+
+ err = anx78xx_aux_wait(anx78xx);
+ if (err)
+ return err;
+
+ msg->reply = DP_AUX_I2C_REPLY_ACK;
+
+ if ((msg->size > 0) && (msg->request & DP_AUX_I2C_READ)) {
+ /* Read values from data buffer */
+ err = regmap_bulk_read(anx78xx->map[I2C_IDX_TX_P0],
+ SP_DP_BUF_DATA0_REG, buffer,
+ msg->size);
+ if (err)
+ return err;
+ }
+
+ err = anx78xx_clear_bits(anx78xx->map[I2C_IDX_TX_P0],
+ SP_DP_AUX_CH_CTRL2_REG, SP_ADDR_ONLY);
+ if (err)
+ return err;
+
+ return msg->size;
+}
+
+static int anx78xx_set_hpd(struct anx78xx *anx78xx)
+{
+ int err;
+
+ err = anx78xx_clear_bits(anx78xx->map[I2C_IDX_RX_P0],
+ SP_TMDS_CTRL_BASE + 7, SP_PD_RT);
+ if (err)
+ return err;
+
+ err = anx78xx_set_bits(anx78xx->map[I2C_IDX_TX_P2], SP_VID_CTRL3_REG,
+ SP_HPD_OUT);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static int anx78xx_clear_hpd(struct anx78xx *anx78xx)
+{
+ int err;
+
+ err = anx78xx_clear_bits(anx78xx->map[I2C_IDX_TX_P2], SP_VID_CTRL3_REG,
+ SP_HPD_OUT);
+ if (err)
+ return err;
+
+ err = anx78xx_set_bits(anx78xx->map[I2C_IDX_RX_P0],
+ SP_TMDS_CTRL_BASE + 7, SP_PD_RT);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static const struct reg_sequence tmds_phy_initialization[] = {
+ { SP_TMDS_CTRL_BASE + 1, 0x90 },
+ { SP_TMDS_CTRL_BASE + 2, 0xa9 },
+ { SP_TMDS_CTRL_BASE + 6, 0x92 },
+ { SP_TMDS_CTRL_BASE + 7, 0x80 },
+ { SP_TMDS_CTRL_BASE + 20, 0xf2 },
+ { SP_TMDS_CTRL_BASE + 22, 0xc4 },
+ { SP_TMDS_CTRL_BASE + 23, 0x18 },
+};
+
+static int anx78xx_rx_initialization(struct anx78xx *anx78xx)
+{
+ int err;
+
+ err = regmap_write(anx78xx->map[I2C_IDX_RX_P0], SP_HDMI_MUTE_CTRL_REG,
+ SP_AUD_MUTE | SP_VID_MUTE);
+ if (err)
+ return err;
+
+ err = anx78xx_set_bits(anx78xx->map[I2C_IDX_RX_P0], SP_CHIP_CTRL_REG,
+ SP_MAN_HDMI5V_DET | SP_PLLLOCK_CKDT_EN |
+ SP_DIGITAL_CKDT_EN);
+ if (err)
+ return err;
+
+ err = anx78xx_set_bits(anx78xx->map[I2C_IDX_RX_P0],
+ SP_SOFTWARE_RESET1_REG, SP_HDCP_MAN_RST |
+ SP_SW_MAN_RST | SP_TMDS_RST | SP_VIDEO_RST);
+ if (err)
+ return err;
+
+ err = anx78xx_clear_bits(anx78xx->map[I2C_IDX_RX_P0],
+ SP_SOFTWARE_RESET1_REG, SP_HDCP_MAN_RST |
+ SP_SW_MAN_RST | SP_TMDS_RST | SP_VIDEO_RST);
+ if (err)
+ return err;
+
+ /* Sync detect change, GP set mute */
+ err = anx78xx_set_bits(anx78xx->map[I2C_IDX_RX_P0],
+ SP_AUD_EXCEPTION_ENABLE_BASE + 1, BIT(5) |
+ BIT(6));
+ if (err)
+ return err;
+
+ err = anx78xx_set_bits(anx78xx->map[I2C_IDX_RX_P0],
+ SP_AUD_EXCEPTION_ENABLE_BASE + 3,
+ SP_AEC_EN21);
+ if (err)
+ return err;
+
+ err = anx78xx_set_bits(anx78xx->map[I2C_IDX_RX_P0], SP_AUDVID_CTRL_REG,
+ SP_AVC_EN | SP_AAC_OE | SP_AAC_EN);
+ if (err)
+ return err;
+
+ err = anx78xx_clear_bits(anx78xx->map[I2C_IDX_RX_P0],
+ SP_SYSTEM_POWER_DOWN1_REG, SP_PWDN_CTRL);
+ if (err)
+ return err;
+
+ err = anx78xx_set_bits(anx78xx->map[I2C_IDX_RX_P0],
+ SP_VID_DATA_RANGE_CTRL_REG, SP_R2Y_INPUT_LIMIT);
+ if (err)
+ return err;
+
+ /* Enable DDC stretch */
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P0],
+ SP_DP_EXTRA_I2C_DEV_ADDR_REG, SP_I2C_EXTRA_ADDR);
+ if (err)
+ return err;
+
+ /* TMDS phy initialization */
+ err = regmap_multi_reg_write(anx78xx->map[I2C_IDX_RX_P0],
+ tmds_phy_initialization,
+ ARRAY_SIZE(tmds_phy_initialization));
+ if (err)
+ return err;
+
+ err = anx78xx_clear_hpd(anx78xx);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static const u8 dp_tx_output_precise_tune_bits[20] = {
+ 0x01, 0x03, 0x07, 0x7f, 0x71, 0x6b, 0x7f,
+ 0x73, 0x7f, 0x7f, 0x00, 0x00, 0x00, 0x00,
+ 0x0c, 0x42, 0x1e, 0x3e, 0x72, 0x7e,
+};
+
+static int anx78xx_link_phy_initialization(struct anx78xx *anx78xx)
+{
+ int err;
+
+ /*
+ * REVISIT : It is writing to a RESERVED bits in Analog Control 0
+ * register.
+ */
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P2], SP_ANALOG_CTRL0_REG,
+ 0x02);
+ if (err)
+ return err;
+
+ /*
+ * Write DP TX output emphasis precise tune bits.
+ */
+ err = regmap_bulk_write(anx78xx->map[I2C_IDX_TX_P1],
+ SP_DP_TX_LT_CTRL0_REG,
+ dp_tx_output_precise_tune_bits,
+ ARRAY_SIZE(dp_tx_output_precise_tune_bits));
+
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static int anx78xx_xtal_clk_sel(struct anx78xx *anx78xx)
+{
+ unsigned int value;
+ int err;
+
+ err = regmap_update_bits(anx78xx->map[I2C_IDX_TX_P2],
+ SP_ANALOG_DEBUG2_REG,
+ SP_XTAL_FRQ | SP_FORCE_SW_OFF_BYPASS,
+ SP_XTAL_FRQ_27M);
+ if (err)
+ return err;
+
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P0], SP_DP_AUX_CH_CTRL3_REG,
+ XTAL_CLK & SP_WAIT_COUNTER_7_0_MASK);
+ if (err)
+ return err;
+
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P0], SP_DP_AUX_CH_CTRL4_REG,
+ ((XTAL_CLK & 0xff00) >> 2) | (XTAL_CLK / 10));
+ if (err)
+ return err;
+
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P0],
+ SP_I2C_GEN_10US_TIMER0_REG, XTAL_CLK & 0xff);
+ if (err)
+ return err;
+
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P0],
+ SP_I2C_GEN_10US_TIMER1_REG,
+ (XTAL_CLK & 0xff00) >> 8);
+ if (err)
+ return err;
+
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P0], SP_AUX_MISC_CTRL_REG,
+ XTAL_CLK / 10 - 1);
+ if (err)
+ return err;
+
+ err = regmap_read(anx78xx->map[I2C_IDX_RX_P0],
+ SP_HDMI_US_TIMER_CTRL_REG,
+ &value);
+ if (err)
+ return err;
+
+ err = regmap_write(anx78xx->map[I2C_IDX_RX_P0],
+ SP_HDMI_US_TIMER_CTRL_REG,
+ (value & SP_MS_TIMER_MARGIN_10_8_MASK) |
+ ((((XTAL_CLK / 10) >> 1) - 2) << 3));
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static const struct reg_sequence otp_key_protect[] = {
+ { SP_OTP_KEY_PROTECT1_REG, SP_OTP_PSW1 },
+ { SP_OTP_KEY_PROTECT2_REG, SP_OTP_PSW2 },
+ { SP_OTP_KEY_PROTECT3_REG, SP_OTP_PSW3 },
+};
+
+static int anx78xx_tx_initialization(struct anx78xx *anx78xx)
+{
+ int err;
+
+ /* Set terminal resistor to 50 ohm */
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P0], SP_DP_AUX_CH_CTRL2_REG,
+ 0x30);
+ if (err)
+ return err;
+
+ /* Enable aux double diff output */
+ err = anx78xx_set_bits(anx78xx->map[I2C_IDX_TX_P0],
+ SP_DP_AUX_CH_CTRL2_REG, 0x08);
+ if (err)
+ return err;
+
+ err = anx78xx_clear_bits(anx78xx->map[I2C_IDX_TX_P0],
+ SP_DP_HDCP_CTRL_REG, SP_AUTO_EN |
+ SP_AUTO_START);
+ if (err)
+ return err;
+
+ err = regmap_multi_reg_write(anx78xx->map[I2C_IDX_TX_P0],
+ otp_key_protect,
+ ARRAY_SIZE(otp_key_protect));
+ if (err)
+ return err;
+
+ err = anx78xx_set_bits(anx78xx->map[I2C_IDX_TX_P0],
+ SP_HDCP_KEY_COMMAND_REG, SP_DISABLE_SYNC_HDCP);
+ if (err)
+ return err;
+
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P2], SP_VID_CTRL8_REG,
+ SP_VID_VRES_TH);
+ if (err)
+ return err;
+
+ /*
+ * DP HDCP auto authentication wait timer (when downstream starts to
+ * auth, DP side will wait for this period then do auth automatically)
+ */
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P0], SP_HDCP_AUTO_TIMER_REG,
+ 0x00);
+ if (err)
+ return err;
+
+ err = anx78xx_set_bits(anx78xx->map[I2C_IDX_TX_P0],
+ SP_DP_HDCP_CTRL_REG, SP_LINK_POLLING);
+ if (err)
+ return err;
+
+ err = anx78xx_set_bits(anx78xx->map[I2C_IDX_TX_P0],
+ SP_DP_LINK_DEBUG_CTRL_REG, SP_M_VID_DEBUG);
+ if (err)
+ return err;
+
+ err = anx78xx_set_bits(anx78xx->map[I2C_IDX_TX_P2],
+ SP_ANALOG_DEBUG2_REG, SP_POWERON_TIME_1P5MS);
+ if (err)
+ return err;
+
+ err = anx78xx_xtal_clk_sel(anx78xx);
+ if (err)
+ return err;
+
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P0], SP_AUX_DEFER_CTRL_REG,
+ SP_DEFER_CTRL_EN | 0x0c);
+ if (err)
+ return err;
+
+ err = anx78xx_set_bits(anx78xx->map[I2C_IDX_TX_P0],
+ SP_DP_POLLING_CTRL_REG,
+ SP_AUTO_POLLING_DISABLE);
+ if (err)
+ return err;
+
+ /*
+ * Short the link integrity check timer to speed up bstatus
+ * polling for HDCP CTS item 1A-07
+ */
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P0],
+ SP_HDCP_LINK_CHECK_TIMER_REG, 0x1d);
+ if (err)
+ return err;
+
+ err = anx78xx_set_bits(anx78xx->map[I2C_IDX_TX_P0],
+ SP_DP_MISC_CTRL_REG, SP_EQ_TRAINING_LOOP);
+ if (err)
+ return err;
+
+ /* Power down the main link by default */
+ err = anx78xx_set_bits(anx78xx->map[I2C_IDX_TX_P0],
+ SP_DP_ANALOG_POWER_DOWN_REG, SP_CH0_PD);
+ if (err)
+ return err;
+
+ err = anx78xx_link_phy_initialization(anx78xx);
+ if (err)
+ return err;
+
+ /* Gen m_clk with downspreading */
+ err = anx78xx_set_bits(anx78xx->map[I2C_IDX_TX_P0],
+ SP_DP_M_CALCULATION_CTRL_REG, SP_M_GEN_CLK_SEL);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static int anx78xx_enable_interrupts(struct anx78xx *anx78xx)
+{
+ int err;
+
+ /*
+ * BIT0: INT pin assertion polarity: 1 = assert high
+ * BIT1: INT pin output type: 0 = push/pull
+ */
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P2], SP_INT_CTRL_REG, 0x01);
+ if (err)
+ return err;
+
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P2],
+ SP_COMMON_INT_MASK4_REG, SP_HPD_LOST | SP_HPD_PLUG);
+ if (err)
+ return err;
+
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P2], SP_DP_INT_MASK1_REG,
+ SP_TRAINING_FINISH);
+ if (err)
+ return err;
+
+ err = regmap_write(anx78xx->map[I2C_IDX_RX_P0], SP_INT_MASK1_REG,
+ SP_CKDT_CHG | SP_SCDT_CHG);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static void anx78xx_poweron(struct anx78xx *anx78xx)
+{
+ struct anx78xx_platform_data *pdata = &anx78xx->pdata;
+ int err;
+
+ if (WARN_ON(anx78xx->powered))
+ return;
+
+ if (pdata->dvdd10) {
+ err = regulator_enable(pdata->dvdd10);
+ if (err) {
+ DRM_ERROR("Failed to enable DVDD10 regulator: %d\n",
+ err);
+ return;
+ }
+
+ usleep_range(1000, 2000);
+ }
+
+ gpiod_set_value_cansleep(pdata->gpiod_reset, 1);
+ usleep_range(1000, 2000);
+
+ gpiod_set_value_cansleep(pdata->gpiod_pd, 0);
+ usleep_range(1000, 2000);
+
+ gpiod_set_value_cansleep(pdata->gpiod_reset, 0);
+
+ /* Power on registers module */
+ anx78xx_set_bits(anx78xx->map[I2C_IDX_TX_P2], SP_POWERDOWN_CTRL_REG,
+ SP_HDCP_PD | SP_AUDIO_PD | SP_VIDEO_PD | SP_LINK_PD);
+ anx78xx_clear_bits(anx78xx->map[I2C_IDX_TX_P2], SP_POWERDOWN_CTRL_REG,
+ SP_REGISTER_PD | SP_TOTAL_PD);
+
+ anx78xx->powered = true;
+}
+
+static void anx78xx_poweroff(struct anx78xx *anx78xx)
+{
+ struct anx78xx_platform_data *pdata = &anx78xx->pdata;
+ int err;
+
+ if (WARN_ON(!anx78xx->powered))
+ return;
+
+ gpiod_set_value_cansleep(pdata->gpiod_reset, 1);
+ usleep_range(1000, 2000);
+
+ gpiod_set_value_cansleep(pdata->gpiod_pd, 1);
+ usleep_range(1000, 2000);
+
+ if (pdata->dvdd10) {
+ err = regulator_disable(pdata->dvdd10);
+ if (err) {
+ DRM_ERROR("Failed to disable DVDD10 regulator: %d\n",
+ err);
+ return;
+ }
+
+ usleep_range(1000, 2000);
+ }
+
+ anx78xx->powered = false;
+}
+
+static int anx78xx_start(struct anx78xx *anx78xx)
+{
+ int err;
+
+ /* Power on all modules */
+ err = anx78xx_clear_bits(anx78xx->map[I2C_IDX_TX_P2],
+ SP_POWERDOWN_CTRL_REG,
+ SP_HDCP_PD | SP_AUDIO_PD | SP_VIDEO_PD |
+ SP_LINK_PD);
+
+ err = anx78xx_enable_interrupts(anx78xx);
+ if (err) {
+ DRM_ERROR("Failed to enable interrupts: %d\n", err);
+ goto err_poweroff;
+ }
+
+ err = anx78xx_rx_initialization(anx78xx);
+ if (err) {
+ DRM_ERROR("Failed receiver initialization: %d\n", err);
+ goto err_poweroff;
+ }
+
+ err = anx78xx_tx_initialization(anx78xx);
+ if (err) {
+ DRM_ERROR("Failed transmitter initialization: %d\n", err);
+ goto err_poweroff;
+ }
+
+ /*
+ * This delay seems to help keep the hardware in a good state. Without
+ * it, there are times where it fails silently.
+ */
+ usleep_range(10000, 15000);
+
+ return 0;
+
+err_poweroff:
+ DRM_ERROR("Failed SlimPort transmitter initialization: %d\n", err);
+ anx78xx_poweroff(anx78xx);
+
+ return err;
+}
+
+static int anx78xx_init_pdata(struct anx78xx *anx78xx)
+{
+ struct anx78xx_platform_data *pdata = &anx78xx->pdata;
+ struct device *dev = &anx78xx->client->dev;
+
+ /* 1.0V digital core power regulator */
+ pdata->dvdd10 = devm_regulator_get(dev, "dvdd10");
+ if (IS_ERR(pdata->dvdd10)) {
+ DRM_ERROR("DVDD10 regulator not found\n");
+ return PTR_ERR(pdata->dvdd10);
+ }
+
+ /* GPIO for HPD */
+ pdata->gpiod_hpd = devm_gpiod_get(dev, "hpd", GPIOD_IN);
+ if (IS_ERR(pdata->gpiod_hpd))
+ return PTR_ERR(pdata->gpiod_hpd);
+
+ /* GPIO for chip power down */
+ pdata->gpiod_pd = devm_gpiod_get(dev, "pd", GPIOD_OUT_HIGH);
+ if (IS_ERR(pdata->gpiod_pd))
+ return PTR_ERR(pdata->gpiod_pd);
+
+ /* GPIO for chip reset */
+ pdata->gpiod_reset = devm_gpiod_get(dev, "reset", GPIOD_OUT_LOW);
+
+ return PTR_ERR_OR_ZERO(pdata->gpiod_reset);
+}
+
+static int anx78xx_dp_link_training(struct anx78xx *anx78xx)
+{
+ u8 dp_bw, value;
+ int err;
+
+ err = regmap_write(anx78xx->map[I2C_IDX_RX_P0], SP_HDMI_MUTE_CTRL_REG,
+ 0x0);
+ if (err)
+ return err;
+
+ err = anx78xx_clear_bits(anx78xx->map[I2C_IDX_TX_P2],
+ SP_POWERDOWN_CTRL_REG,
+ SP_TOTAL_PD);
+ if (err)
+ return err;
+
+ err = drm_dp_dpcd_readb(&anx78xx->aux, DP_MAX_LINK_RATE, &dp_bw);
+ if (err < 0)
+ return err;
+
+ switch (dp_bw) {
+ case DP_LINK_BW_1_62:
+ case DP_LINK_BW_2_7:
+ case DP_LINK_BW_5_4:
+ break;
+
+ default:
+ DRM_DEBUG_KMS("DP bandwidth (%#02x) not supported\n", dp_bw);
+ return -EINVAL;
+ }
+
+ err = anx78xx_set_bits(anx78xx->map[I2C_IDX_TX_P2], SP_VID_CTRL1_REG,
+ SP_VIDEO_MUTE);
+ if (err)
+ return err;
+
+ err = anx78xx_clear_bits(anx78xx->map[I2C_IDX_TX_P2],
+ SP_VID_CTRL1_REG, SP_VIDEO_EN);
+ if (err)
+ return err;
+
+ /* Get DPCD info */
+ err = drm_dp_dpcd_read(&anx78xx->aux, DP_DPCD_REV,
+ &anx78xx->dpcd, DP_RECEIVER_CAP_SIZE);
+ if (err < 0) {
+ DRM_ERROR("Failed to read DPCD: %d\n", err);
+ return err;
+ }
+
+ /* Clear channel x SERDES power down */
+ err = anx78xx_clear_bits(anx78xx->map[I2C_IDX_TX_P0],
+ SP_DP_ANALOG_POWER_DOWN_REG, SP_CH0_PD);
+ if (err)
+ return err;
+
+ /* Check link capabilities */
+ err = drm_dp_link_probe(&anx78xx->aux, &anx78xx->link);
+ if (err < 0) {
+ DRM_ERROR("Failed to probe link capabilities: %d\n", err);
+ return err;
+ }
+
+ /* Power up the sink */
+ err = drm_dp_link_power_up(&anx78xx->aux, &anx78xx->link);
+ if (err < 0) {
+ DRM_ERROR("Failed to power up DisplayPort link: %d\n", err);
+ return err;
+ }
+
+ /* Possibly enable downspread on the sink */
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P0],
+ SP_DP_DOWNSPREAD_CTRL1_REG, 0);
+ if (err)
+ return err;
+
+ if (anx78xx->dpcd[DP_MAX_DOWNSPREAD] & DP_MAX_DOWNSPREAD_0_5) {
+ DRM_DEBUG("Enable downspread on the sink\n");
+ /* 4000PPM */
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P0],
+ SP_DP_DOWNSPREAD_CTRL1_REG, 8);
+ if (err)
+ return err;
+
+ err = drm_dp_dpcd_writeb(&anx78xx->aux, DP_DOWNSPREAD_CTRL,
+ DP_SPREAD_AMP_0_5);
+ if (err < 0)
+ return err;
+ } else {
+ err = drm_dp_dpcd_writeb(&anx78xx->aux, DP_DOWNSPREAD_CTRL, 0);
+ if (err < 0)
+ return err;
+ }
+
+ /* Set the lane count and the link rate on the sink */
+ if (drm_dp_enhanced_frame_cap(anx78xx->dpcd))
+ err = anx78xx_set_bits(anx78xx->map[I2C_IDX_TX_P0],
+ SP_DP_SYSTEM_CTRL_BASE + 4,
+ SP_ENHANCED_MODE);
+ else
+ err = anx78xx_clear_bits(anx78xx->map[I2C_IDX_TX_P0],
+ SP_DP_SYSTEM_CTRL_BASE + 4,
+ SP_ENHANCED_MODE);
+ if (err)
+ return err;
+
+ value = drm_dp_link_rate_to_bw_code(anx78xx->link.rate);
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P0],
+ SP_DP_MAIN_LINK_BW_SET_REG, value);
+ if (err)
+ return err;
+
+ err = drm_dp_link_configure(&anx78xx->aux, &anx78xx->link);
+ if (err < 0) {
+ DRM_ERROR("Failed to configure DisplayPort link: %d\n", err);
+ return err;
+ }
+
+ /* Start training on the source */
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P0], SP_DP_LT_CTRL_REG,
+ SP_LT_EN);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static int anx78xx_config_dp_output(struct anx78xx *anx78xx)
+{
+ int err;
+
+ err = anx78xx_clear_bits(anx78xx->map[I2C_IDX_TX_P2], SP_VID_CTRL1_REG,
+ SP_VIDEO_MUTE);
+ if (err)
+ return err;
+
+ /* Enable DP output */
+ err = anx78xx_set_bits(anx78xx->map[I2C_IDX_TX_P2], SP_VID_CTRL1_REG,
+ SP_VIDEO_EN);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static int anx78xx_send_video_infoframe(struct anx78xx *anx78xx,
+ struct hdmi_avi_infoframe *frame)
+{
+ u8 buffer[HDMI_INFOFRAME_HEADER_SIZE + HDMI_AVI_INFOFRAME_SIZE];
+ int err;
+
+ err = hdmi_avi_infoframe_pack(frame, buffer, sizeof(buffer));
+ if (err < 0) {
+ DRM_ERROR("Failed to pack AVI infoframe: %d\n", err);
+ return err;
+ }
+
+ err = anx78xx_clear_bits(anx78xx->map[I2C_IDX_TX_P0],
+ SP_PACKET_SEND_CTRL_REG, SP_AVI_IF_EN);
+ if (err)
+ return err;
+
+ err = regmap_bulk_write(anx78xx->map[I2C_IDX_TX_P2],
+ SP_INFOFRAME_AVI_DB1_REG, buffer,
+ frame->length);
+ if (err)
+ return err;
+
+ err = anx78xx_set_bits(anx78xx->map[I2C_IDX_TX_P0],
+ SP_PACKET_SEND_CTRL_REG, SP_AVI_IF_UD);
+ if (err)
+ return err;
+
+ err = anx78xx_set_bits(anx78xx->map[I2C_IDX_TX_P0],
+ SP_PACKET_SEND_CTRL_REG, SP_AVI_IF_EN);
+ if (err)
+ return err;
+
+ return 0;
+}
+
+static int anx78xx_get_downstream_info(struct anx78xx *anx78xx)
+{
+ u8 value;
+ int err;
+
+ err = drm_dp_dpcd_readb(&anx78xx->aux, DP_SINK_COUNT, &value);
+ if (err < 0) {
+ DRM_ERROR("Get sink count failed %d\n", err);
+ return err;
+ }
+
+ if (!DP_GET_SINK_COUNT(value)) {
+ DRM_ERROR("Downstream disconnected\n");
+ return -EIO;
+ }
+
+ return 0;
+}
+
+static int anx78xx_get_modes(struct drm_connector *connector)
+{
+ struct anx78xx *anx78xx = connector_to_anx78xx(connector);
+ int err, num_modes = 0;
+
+ if (WARN_ON(!anx78xx->powered))
+ return 0;
+
+ if (anx78xx->edid)
+ return drm_add_edid_modes(connector, anx78xx->edid);
+
+ mutex_lock(&anx78xx->lock);
+
+ err = anx78xx_get_downstream_info(anx78xx);
+ if (err) {
+ DRM_ERROR("Failed to get downstream info: %d\n", err);
+ goto unlock;
+ }
+
+ anx78xx->edid = drm_get_edid(connector, &anx78xx->aux.ddc);
+ if (!anx78xx->edid) {
+ DRM_ERROR("Failed to read EDID\n");
+ goto unlock;
+ }
+
+ err = drm_mode_connector_update_edid_property(connector,
+ anx78xx->edid);
+ if (err) {
+ DRM_ERROR("Failed to update EDID property: %d\n", err);
+ goto unlock;
+ }
+
+ num_modes = drm_add_edid_modes(connector, anx78xx->edid);
+ /* Store the ELD */
+ drm_edid_to_eld(connector, anx78xx->edid);
+
+unlock:
+ mutex_unlock(&anx78xx->lock);
+
+ return num_modes;
+}
+
+static struct drm_encoder *anx78xx_best_encoder(struct drm_connector *connector)
+{
+ struct anx78xx *anx78xx = connector_to_anx78xx(connector);
+
+ return anx78xx->bridge.encoder;
+}
+
+static const struct drm_connector_helper_funcs anx78xx_connector_helper_funcs = {
+ .get_modes = anx78xx_get_modes,
+ .best_encoder = anx78xx_best_encoder,
+};
+
+static enum drm_connector_status anx78xx_detect(struct drm_connector *connector,
+ bool force)
+{
+ struct anx78xx *anx78xx = connector_to_anx78xx(connector);
+
+ if (!gpiod_get_value(anx78xx->pdata.gpiod_hpd))
+ return connector_status_disconnected;
+
+ return connector_status_connected;
+}
+
+static void anx78xx_connector_destroy(struct drm_connector *connector)
+{
+ drm_connector_cleanup(connector);
+}
+
+static const struct drm_connector_funcs anx78xx_connector_funcs = {
+ .dpms = drm_atomic_helper_connector_dpms,
+ .fill_modes = drm_helper_probe_single_connector_modes,
+ .detect = anx78xx_detect,
+ .destroy = anx78xx_connector_destroy,
+ .reset = drm_atomic_helper_connector_reset,
+ .atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
+};
+
+static int anx78xx_bridge_attach(struct drm_bridge *bridge)
+{
+ struct anx78xx *anx78xx = bridge_to_anx78xx(bridge);
+ int err;
+
+ if (!bridge->encoder) {
+ DRM_ERROR("Parent encoder object not found");
+ return -ENODEV;
+ }
+
+ /* Register aux channel */
+ anx78xx->aux.name = "DP-AUX";
+ anx78xx->aux.dev = &anx78xx->client->dev;
+ anx78xx->aux.transfer = anx78xx_aux_transfer;
+
+ err = drm_dp_aux_register(&anx78xx->aux);
+ if (err < 0) {
+ DRM_ERROR("Failed to register aux channel: %d\n", err);
+ return err;
+ }
+
+ err = drm_connector_init(bridge->dev, &anx78xx->connector,
+ &anx78xx_connector_funcs,
+ DRM_MODE_CONNECTOR_DisplayPort);
+ if (err) {
+ DRM_ERROR("Failed to initialize connector: %d\n", err);
+ return err;
+ }
+
+ drm_connector_helper_add(&anx78xx->connector,
+ &anx78xx_connector_helper_funcs);
+
+ err = drm_connector_register(&anx78xx->connector);
+ if (err) {
+ DRM_ERROR("Failed to register connector: %d\n", err);
+ return err;
+ }
+
+ anx78xx->connector.polled = DRM_CONNECTOR_POLL_HPD;
+
+ err = drm_mode_connector_attach_encoder(&anx78xx->connector,
+ bridge->encoder);
+ if (err) {
+ DRM_ERROR("Failed to link up connector to encoder: %d\n", err);
+ return err;
+ }
+
+ return 0;
+}
+
+static bool anx78xx_bridge_mode_fixup(struct drm_bridge *bridge,
+ const struct drm_display_mode *mode,
+ struct drm_display_mode *adjusted_mode)
+{
+ if (mode->flags & DRM_MODE_FLAG_INTERLACE)
+ return false;
+
+ /* Max 1200p at 5.4 Ghz, one lane */
+ if (mode->clock > 154000)
+ return false;
+
+ return true;
+}
+
+static void anx78xx_bridge_disable(struct drm_bridge *bridge)
+{
+ struct anx78xx *anx78xx = bridge_to_anx78xx(bridge);
+
+ /* Power off all modules except configuration registers access */
+ anx78xx_set_bits(anx78xx->map[I2C_IDX_TX_P2], SP_POWERDOWN_CTRL_REG,
+ SP_HDCP_PD | SP_AUDIO_PD | SP_VIDEO_PD | SP_LINK_PD);
+}
+
+static void anx78xx_bridge_mode_set(struct drm_bridge *bridge,
+ struct drm_display_mode *mode,
+ struct drm_display_mode *adjusted_mode)
+{
+ struct anx78xx *anx78xx = bridge_to_anx78xx(bridge);
+ struct hdmi_avi_infoframe frame;
+ int err;
+
+ if (WARN_ON(!anx78xx->powered))
+ return;
+
+ mutex_lock(&anx78xx->lock);
+
+ err = drm_hdmi_avi_infoframe_from_display_mode(&frame, adjusted_mode);
+ if (err) {
+ DRM_ERROR("Failed to setup AVI infoframe: %d\n", err);
+ goto unlock;
+ }
+
+ err = anx78xx_send_video_infoframe(anx78xx, &frame);
+ if (err)
+ DRM_ERROR("Failed to send AVI infoframe: %d\n", err);
+
+unlock:
+ mutex_unlock(&anx78xx->lock);
+}
+
+static void anx78xx_bridge_enable(struct drm_bridge *bridge)
+{
+ struct anx78xx *anx78xx = bridge_to_anx78xx(bridge);
+ int err;
+
+ err = anx78xx_start(anx78xx);
+ if (err) {
+ DRM_ERROR("Failed to initialize: %d\n", err);
+ return;
+ }
+
+ err = anx78xx_set_hpd(anx78xx);
+ if (err)
+ DRM_ERROR("Failed to set HPD: %d\n", err);
+}
+
+static const struct drm_bridge_funcs anx78xx_bridge_funcs = {
+ .attach = anx78xx_bridge_attach,
+ .mode_fixup = anx78xx_bridge_mode_fixup,
+ .disable = anx78xx_bridge_disable,
+ .mode_set = anx78xx_bridge_mode_set,
+ .enable = anx78xx_bridge_enable,
+};
+
+static irqreturn_t anx78xx_hpd_threaded_handler(int irq, void *data)
+{
+ struct anx78xx *anx78xx = data;
+ int err;
+
+ if (anx78xx->powered)
+ return IRQ_HANDLED;
+
+ mutex_lock(&anx78xx->lock);
+
+ /* Cable is pulled, power on the chip */
+ anx78xx_poweron(anx78xx);
+
+ err = anx78xx_enable_interrupts(anx78xx);
+ if (err)
+ DRM_ERROR("Failed to enable interrupts: %d\n", err);
+
+ mutex_unlock(&anx78xx->lock);
+
+ return IRQ_HANDLED;
+}
+
+static int anx78xx_handle_dp_int_1(struct anx78xx *anx78xx, u8 irq)
+{
+ int err;
+
+ DRM_DEBUG_KMS("Handle DP interrupt 1: %02x\n", irq);
+
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P2], SP_DP_INT_STATUS1_REG,
+ irq);
+ if (err)
+ return err;
+
+ if (irq & SP_TRAINING_FINISH) {
+ DRM_DEBUG_KMS("IRQ: hardware link training finished\n");
+ err = anx78xx_config_dp_output(anx78xx);
+ }
+
+ return err;
+}
+
+static bool anx78xx_handle_common_int_4(struct anx78xx *anx78xx, u8 irq)
+{
+ bool event = false;
+ int err;
+
+ DRM_DEBUG_KMS("Handle common interrupt 4: %02x\n", irq);
+
+ err = regmap_write(anx78xx->map[I2C_IDX_TX_P2],
+ SP_COMMON_INT_STATUS4_REG, irq);
+ if (err) {
+ DRM_ERROR("Failed to write SP_COMMON_INT_STATUS4 %d\n", err);
+ return event;
+ }
+
+ if (irq & SP_HPD_LOST) {
+ DRM_DEBUG_KMS("IRQ: Hot plug detect - cable is pulled out\n");
+ event = true;
+ anx78xx_poweroff(anx78xx);
+ /* Free cached EDID */
+ kfree(anx78xx->edid);
+ anx78xx->edid = NULL;
+ } else if (irq & SP_HPD_PLUG) {
+ DRM_DEBUG_KMS("IRQ: Hot plug detect - cable plug\n");
+ event = true;
+ }
+
+ return event;
+}
+
+static void anx78xx_handle_hdmi_int_1(struct anx78xx *anx78xx, u8 irq)
+{
+ unsigned int value;
+ int err;
+
+ DRM_DEBUG_KMS("Handle HDMI interrupt 1: %02x\n", irq);
+
+ err = regmap_write(anx78xx->map[I2C_IDX_RX_P0], SP_INT_STATUS1_REG,
+ irq);
+ if (err) {
+ DRM_ERROR("Write HDMI int 1 failed: %d\n", err);
+ return;
+ }
+
+ if ((irq & SP_CKDT_CHG) || (irq & SP_SCDT_CHG)) {
+ DRM_DEBUG_KMS("IRQ: HDMI input detected\n");
+
+ err = regmap_read(anx78xx->map[I2C_IDX_RX_P0],
+ SP_SYSTEM_STATUS_REG, &value);
+ if (err) {
+ DRM_ERROR("Read system status reg failed: %d\n", err);
+ return;
+ }
+
+ if (!(value & SP_TMDS_CLOCK_DET)) {
+ DRM_DEBUG_KMS("IRQ: *** Waiting for HDMI clock ***\n");
+ return;
+ }
+
+ if (!(value & SP_TMDS_DE_DET)) {
+ DRM_DEBUG_KMS("IRQ: *** Waiting for HDMI signal ***\n");
+ return;
+ }
+
+ err = anx78xx_dp_link_training(anx78xx);
+ if (err)
+ DRM_ERROR("Failed to start link training: %d\n", err);
+ }
+}
+
+static irqreturn_t anx78xx_intp_threaded_handler(int unused, void *data)
+{
+ struct anx78xx *anx78xx = data;
+ bool event = false;
+ unsigned int irq;
+ int err;
+
+ mutex_lock(&anx78xx->lock);
+
+ err = regmap_read(anx78xx->map[I2C_IDX_TX_P2], SP_DP_INT_STATUS1_REG,
+ &irq);
+ if (err) {
+ DRM_ERROR("Failed to read DP interrupt 1 status: %d\n", err);
+ goto unlock;
+ }
+
+ if (irq)
+ anx78xx_handle_dp_int_1(anx78xx, irq);
+
+ err = regmap_read(anx78xx->map[I2C_IDX_TX_P2],
+ SP_COMMON_INT_STATUS4_REG, &irq);
+ if (err) {
+ DRM_ERROR("Failed to read common interrupt 4 status: %d\n",
+ err);
+ goto unlock;
+ }
+
+ if (irq)
+ event = anx78xx_handle_common_int_4(anx78xx, irq);
+
+ /* Make sure we are still powered after handle HPD events */
+ if (!anx78xx->powered)
+ goto unlock;
+
+ err = regmap_read(anx78xx->map[I2C_IDX_RX_P0], SP_INT_STATUS1_REG,
+ &irq);
+ if (err) {
+ DRM_ERROR("Failed to read HDMI int 1 status: %d\n", err);
+ goto unlock;
+ }
+
+ if (irq)
+ anx78xx_handle_hdmi_int_1(anx78xx, irq);
+
+unlock:
+ mutex_unlock(&anx78xx->lock);
+
+ if (event)
+ drm_helper_hpd_irq_event(anx78xx->connector.dev);
+
+ return IRQ_HANDLED;
+}
+
+static void unregister_i2c_dummy_clients(struct anx78xx *anx78xx)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(anx78xx->i2c_dummy); i++)
+ if (anx78xx->i2c_dummy[i])
+ i2c_unregister_device(anx78xx->i2c_dummy[i]);
+}
+
+static const struct regmap_config anx78xx_regmap_config = {
+ .reg_bits = 8,
+ .val_bits = 8,
+};
+
+static const u16 anx78xx_chipid_list[] = {
+ 0x7812,
+ 0x7814,
+ 0x7818,
+};
+
+static int anx78xx_i2c_probe(struct i2c_client *client,
+ const struct i2c_device_id *id)
+{
+ struct anx78xx *anx78xx;
+ struct anx78xx_platform_data *pdata;
+ unsigned int i, idl, idh, version;
+ bool found = false;
+ int err;
+
+ anx78xx = devm_kzalloc(&client->dev, sizeof(*anx78xx), GFP_KERNEL);
+ if (!anx78xx)
+ return -ENOMEM;
+
+ pdata = &anx78xx->pdata;
+
+ mutex_init(&anx78xx->lock);
+
+#if IS_ENABLED(CONFIG_OF)
+ anx78xx->bridge.of_node = client->dev.of_node;
+#endif
+
+ anx78xx->client = client;
+ i2c_set_clientdata(client, anx78xx);
+
+ err = anx78xx_init_pdata(anx78xx);
+ if (err) {
+ DRM_ERROR("Failed to initialize pdata: %d\n", err);
+ return err;
+ }
+
+ pdata->hpd_irq = gpiod_to_irq(pdata->gpiod_hpd);
+ if (pdata->hpd_irq < 0) {
+ DRM_ERROR("Failed to get HPD IRQ: %d\n", pdata->hpd_irq);
+ return -ENODEV;
+ }
+
+ pdata->intp_irq = client->irq;
+ if (!pdata->intp_irq) {
+ DRM_ERROR("Failed to get CABLE_DET and INTP IRQ\n");
+ return -ENODEV;
+ }
+
+ /* Map slave addresses of ANX7814 */
+ for (i = 0; i < I2C_NUM_ADDRESSES; i++) {
+ anx78xx->i2c_dummy[i] = i2c_new_dummy(client->adapter,
+ anx78xx_i2c_addresses[i] >> 1);
+ if (!anx78xx->i2c_dummy[i]) {
+ err = -ENOMEM;
+ DRM_ERROR("Failed to reserve I2C bus %02x\n",
+ anx78xx_i2c_addresses[i]);
+ goto err_unregister_i2c;
+ }
+
+ anx78xx->map[i] = devm_regmap_init_i2c(anx78xx->i2c_dummy[i],
+ &anx78xx_regmap_config);
+ if (IS_ERR(anx78xx->map[i])) {
+ err = PTR_ERR(anx78xx->map[i]);
+ DRM_ERROR("Failed regmap initialization %02x\n",
+ anx78xx_i2c_addresses[i]);
+ goto err_unregister_i2c;
+ }
+ }
+
+ /* Look for supported chip ID */
+ anx78xx_poweron(anx78xx);
+
+ err = regmap_read(anx78xx->map[I2C_IDX_TX_P2], SP_DEVICE_IDL_REG,
+ &idl);
+ if (err)
+ goto err_poweroff;
+
+ err = regmap_read(anx78xx->map[I2C_IDX_TX_P2], SP_DEVICE_IDH_REG,
+ &idh);
+ if (err)
+ goto err_poweroff;
+
+ anx78xx->chipid = (u8)idl | ((u8)idh << 8);
+
+ err = regmap_read(anx78xx->map[I2C_IDX_TX_P2], SP_DEVICE_VERSION_REG,
+ &version);
+ if (err)
+ goto err_poweroff;
+
+ for (i = 0; i < ARRAY_SIZE(anx78xx_chipid_list); i++) {
+ if (anx78xx->chipid == anx78xx_chipid_list[i]) {
+ DRM_INFO("Found ANX%x (ver. %d) SlimPort Transmitter\n",
+ anx78xx->chipid, version);
+ found = true;
+ break;
+ }
+ }
+
+ if (!found) {
+ DRM_ERROR("ANX%x (ver. %d) not supported by this driver\n",
+ anx78xx->chipid, version);
+ err = -ENODEV;
+ goto err_poweroff;
+ }
+
+ err = devm_request_threaded_irq(&client->dev, pdata->hpd_irq, NULL,
+ anx78xx_hpd_threaded_handler,
+ IRQF_TRIGGER_RISING | IRQF_ONESHOT,
+ "anx78xx-hpd", anx78xx);
+ if (err) {
+ DRM_ERROR("Failed to request CABLE_DET threaded IRQ: %d\n",
+ err);
+ goto err_poweroff;
+ }
+
+ err = devm_request_threaded_irq(&client->dev, pdata->intp_irq, NULL,
+ anx78xx_intp_threaded_handler,
+ IRQF_TRIGGER_RISING | IRQF_ONESHOT,
+ "anx78xx-intp", anx78xx);
+ if (err) {
+ DRM_ERROR("Failed to request INTP threaded IRQ: %d\n", err);
+ goto err_poweroff;
+ }
+
+ anx78xx->bridge.funcs = &anx78xx_bridge_funcs;
+
+ err = drm_bridge_add(&anx78xx->bridge);
+ if (err < 0) {
+ DRM_ERROR("Failed to add drm bridge: %d\n", err);
+ goto err_poweroff;
+ }
+
+ /* If cable is pulled out, just poweroff and wait for HPD event */
+ if (!gpiod_get_value(anx78xx->pdata.gpiod_hpd))
+ anx78xx_poweroff(anx78xx);
+
+ return 0;
+
+err_poweroff:
+ anx78xx_poweroff(anx78xx);
+
+err_unregister_i2c:
+ unregister_i2c_dummy_clients(anx78xx);
+ return err;
+}
+
+static int anx78xx_i2c_remove(struct i2c_client *client)
+{
+ struct anx78xx *anx78xx = i2c_get_clientdata(client);
+
+ drm_bridge_remove(&anx78xx->bridge);
+
+ unregister_i2c_dummy_clients(anx78xx);
+
+ kfree(anx78xx->edid);
+
+ return 0;
+}
+
+static const struct i2c_device_id anx78xx_id[] = {
+ { "anx7814", 0 },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(i2c, anx78xx_id);
+
+#if IS_ENABLED(CONFIG_OF)
+static const struct of_device_id anx78xx_match_table[] = {
+ { .compatible = "analogix,anx7814", },
+ { /* sentinel */ },
+};
+MODULE_DEVICE_TABLE(of, anx78xx_match_table);
+#endif
+
+static struct i2c_driver anx78xx_driver = {
+ .driver = {
+ .name = "anx7814",
+ .of_match_table = of_match_ptr(anx78xx_match_table),
+ },
+ .probe = anx78xx_i2c_probe,
+ .remove = anx78xx_i2c_remove,
+ .id_table = anx78xx_id,
+};
+module_i2c_driver(anx78xx_driver);
+
+MODULE_DESCRIPTION("ANX78xx SlimPort Transmitter driver");
+MODULE_AUTHOR("Enric Balletbo i Serra <enric.balletbo@collabora.com>");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/gpu/drm/bridge/analogix-anx78xx.h b/drivers/gpu/drm/bridge/analogix-anx78xx.h
new file mode 100644
index 0000000..38753c8
--- /dev/null
+++ b/drivers/gpu/drm/bridge/analogix-anx78xx.h
@@ -0,0 +1,719 @@
+/*
+ * Copyright(c) 2016, Analogix Semiconductor. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#ifndef __ANX78xx_H
+#define __ANX78xx_H
+
+#define TX_P0 0x70
+#define TX_P1 0x7a
+#define TX_P2 0x72
+
+#define RX_P0 0x7e
+#define RX_P1 0x80
+
+/***************************************************************/
+/* Register definition of device address 0x7e */
+/***************************************************************/
+
+/*
+ * System Control and Status
+ */
+
+/* Software Reset Register 1 */
+#define SP_SOFTWARE_RESET1_REG 0x11
+#define SP_VIDEO_RST BIT(4)
+#define SP_HDCP_MAN_RST BIT(2)
+#define SP_TMDS_RST BIT(1)
+#define SP_SW_MAN_RST BIT(0)
+
+/* System Status Register */
+#define SP_SYSTEM_STATUS_REG 0x14
+#define SP_TMDS_CLOCK_DET BIT(1)
+#define SP_TMDS_DE_DET BIT(0)
+
+/* HDMI Status Register */
+#define SP_HDMI_STATUS_REG 0x15
+#define SP_HDMI_AUD_LAYOUT BIT(3)
+#define SP_HDMI_DET BIT(0)
+# define SP_DVI_MODE 0
+# define SP_HDMI_MODE 1
+
+/* HDMI Mute Control Register */
+#define SP_HDMI_MUTE_CTRL_REG 0x16
+#define SP_AUD_MUTE BIT(1)
+#define SP_VID_MUTE BIT(0)
+
+/* System Power Down Register 1 */
+#define SP_SYSTEM_POWER_DOWN1_REG 0x18
+#define SP_PWDN_CTRL BIT(0)
+
+/*
+ * Audio and Video Auto Control
+ */
+
+/* Auto Audio and Video Control register */
+#define SP_AUDVID_CTRL_REG 0x20
+#define SP_AVC_OE BIT(7)
+#define SP_AAC_OE BIT(6)
+#define SP_AVC_EN BIT(1)
+#define SP_AAC_EN BIT(0)
+
+/* Audio Exception Enable Registers */
+#define SP_AUD_EXCEPTION_ENABLE_BASE (0x24 - 1)
+/* Bits for Audio Exception Enable Register 3 */
+#define SP_AEC_EN21 BIT(5)
+
+/*
+ * Interrupt
+ */
+
+/* Interrupt Status Register 1 */
+#define SP_INT_STATUS1_REG 0x31
+/* Bits for Interrupt Status Register 1 */
+#define SP_HDMI_DVI BIT(7)
+#define SP_CKDT_CHG BIT(6)
+#define SP_SCDT_CHG BIT(5)
+#define SP_PCLK_CHG BIT(4)
+#define SP_PLL_UNLOCK BIT(3)
+#define SP_CABLE_PLUG_CHG BIT(2)
+#define SP_SET_MUTE BIT(1)
+#define SP_SW_INTR BIT(0)
+/* Bits for Interrupt Status Register 2 */
+#define SP_HDCP_ERR BIT(5)
+#define SP_AUDIO_SAMPLE_CHG BIT(0) /* undocumented */
+/* Bits for Interrupt Status Register 3 */
+#define SP_AUD_MODE_CHG BIT(0)
+/* Bits for Interrupt Status Register 5 */
+#define SP_AUDIO_RCV BIT(0)
+/* Bits for Interrupt Status Register 6 */
+#define SP_INT_STATUS6_REG 0x36
+#define SP_CTS_RCV BIT(7)
+#define SP_NEW_AUD_PKT BIT(4)
+#define SP_NEW_AVI_PKT BIT(1)
+#define SP_NEW_CP_PKT BIT(0)
+/* Bits for Interrupt Status Register 7 */
+#define SP_NO_VSI BIT(7)
+#define SP_NEW_VS BIT(4)
+
+/* Interrupt Mask 1 Status Registers */
+#define SP_INT_MASK1_REG 0x41
+
+/* HDMI US TIMER Control Register */
+#define SP_HDMI_US_TIMER_CTRL_REG 0x49
+#define SP_MS_TIMER_MARGIN_10_8_MASK 0x07
+
+/*
+ * TMDS Control
+ */
+
+/* TMDS Control Registers */
+#define SP_TMDS_CTRL_BASE (0x50 - 1)
+/* Bits for TMDS Control Register 7 */
+#define SP_PD_RT BIT(0)
+
+/*
+ * Video Control
+ */
+
+/* Video Status Register */
+#define SP_VIDEO_STATUS_REG 0x70
+#define SP_COLOR_DEPTH_MASK 0xf0
+#define SP_COLOR_DEPTH_SHIFT 4
+# define SP_COLOR_DEPTH_MODE_LEGACY 0x00
+# define SP_COLOR_DEPTH_MODE_24BIT 0x04
+# define SP_COLOR_DEPTH_MODE_30BIT 0x05
+# define SP_COLOR_DEPTH_MODE_36BIT 0x06
+# define SP_COLOR_DEPTH_MODE_48BIT 0x07
+
+/* Video Data Range Control Register */
+#define SP_VID_DATA_RANGE_CTRL_REG 0x83
+#define SP_R2Y_INPUT_LIMIT BIT(1)
+
+/* Pixel Clock High Resolution Counter Registers */
+#define SP_PCLK_HIGHRES_CNT_BASE (0x8c - 1)
+
+/*
+ * Audio Control
+ */
+
+/* Number of Audio Channels Status Registers */
+#define SP_AUD_CH_STATUS_REG_NUM 6
+
+/* Audio IN S/PDIF Channel Status Registers */
+#define SP_AUD_SPDIF_CH_STATUS_BASE 0xc7
+
+/* Audio IN S/PDIF Channel Status Register 4 */
+#define SP_FS_FREQ_MASK 0x0f
+# define SP_FS_FREQ_44100HZ 0x00
+# define SP_FS_FREQ_48000HZ 0x02
+# define SP_FS_FREQ_32000HZ 0x03
+# define SP_FS_FREQ_88200HZ 0x08
+# define SP_FS_FREQ_96000HZ 0x0a
+# define SP_FS_FREQ_176400HZ 0x0c
+# define SP_FS_FREQ_192000HZ 0x0e
+
+/*
+ * Micellaneous Control Block
+ */
+
+/* CHIP Control Register */
+#define SP_CHIP_CTRL_REG 0xe3
+#define SP_MAN_HDMI5V_DET BIT(3)
+#define SP_PLLLOCK_CKDT_EN BIT(2)
+#define SP_ANALOG_CKDT_EN BIT(1)
+#define SP_DIGITAL_CKDT_EN BIT(0)
+
+/* Packet Receiving Status Register */
+#define SP_PACKET_RECEIVING_STATUS_REG 0xf3
+#define SP_AVI_RCVD BIT(5)
+#define SP_VSI_RCVD BIT(1)
+
+/***************************************************************/
+/* Register definition of device address 0x80 */
+/***************************************************************/
+
+/* HDCP BCAPS Shadow Register */
+#define SP_HDCP_BCAPS_SHADOW_REG 0x2a
+#define SP_BCAPS_REPEATER BIT(5)
+
+/* HDCP Status Register */
+#define SP_RX_HDCP_STATUS_REG 0x3f
+#define SP_AUTH_EN BIT(4)
+
+/*
+ * InfoFrame and Control Packet Registers
+ */
+
+/* AVI InfoFrame packet checksum */
+#define SP_AVI_INFOFRAME_CHECKSUM 0xa3
+
+/* AVI InfoFrame Registers */
+#define SP_AVI_INFOFRAME_DATA_BASE 0xa4
+
+#define SP_AVI_COLOR_F_MASK 0x60
+#define SP_AVI_COLOR_F_SHIFT 5
+
+/* Audio InfoFrame Registers */
+#define SP_AUD_INFOFRAME_DATA_BASE 0xc4
+#define SP_AUD_INFOFRAME_LAYOUT_MASK 0x0f
+
+/* MPEG/HDMI Vendor Specific InfoFrame Packet type code */
+#define SP_MPEG_VS_INFOFRAME_TYPE_REG 0xe0
+
+/* MPEG/HDMI Vendor Specific InfoFrame Packet length */
+#define SP_MPEG_VS_INFOFRAME_LEN_REG 0xe2
+
+/* MPEG/HDMI Vendor Specific InfoFrame Packet version number */
+#define SP_MPEG_VS_INFOFRAME_VER_REG 0xe1
+
+/* MPEG/HDMI Vendor Specific InfoFrame Packet content */
+#define SP_MPEG_VS_INFOFRAME_DATA_BASE 0xe4
+
+/* General Control Packet Register */
+#define SP_GENERAL_CTRL_PACKET_REG 0x9f
+#define SP_CLEAR_AVMUTE BIT(4)
+#define SP_SET_AVMUTE BIT(0)
+
+/***************************************************************/
+/* Register definition of device address 0x70 */
+/***************************************************************/
+
+/* HDCP Status Register */
+#define SP_TX_HDCP_STATUS_REG 0x00
+#define SP_AUTH_FAIL BIT(5)
+#define SP_AUTHEN_PASS BIT(1)
+
+/* HDCP Control Register 0 */
+#define SP_HDCP_CTRL0_REG 0x01
+#define SP_RX_REPEATER BIT(6)
+#define SP_RE_AUTH BIT(5)
+#define SP_SW_AUTH_OK BIT(4)
+#define SP_HARD_AUTH_EN BIT(3)
+#define SP_HDCP_ENC_EN BIT(2)
+#define SP_BKSV_SRM_PASS BIT(1)
+#define SP_KSVLIST_VLD BIT(0)
+/* HDCP Function Enabled */
+#define SP_HDCP_FUNCTION_ENABLED (BIT(0) | BIT(1) | BIT(2) | BIT(3))
+
+/* HDCP Receiver BSTATUS Register 0 */
+#define SP_HDCP_RX_BSTATUS0_REG 0x1b
+/* HDCP Receiver BSTATUS Register 1 */
+#define SP_HDCP_RX_BSTATUS1_REG 0x1c
+
+/* HDCP Embedded "Blue Screen" Content Registers */
+#define SP_HDCP_VID0_BLUE_SCREEN_REG 0x2c
+#define SP_HDCP_VID1_BLUE_SCREEN_REG 0x2d
+#define SP_HDCP_VID2_BLUE_SCREEN_REG 0x2e
+
+/* HDCP Wait R0 Timing Register */
+#define SP_HDCP_WAIT_R0_TIME_REG 0x40
+
+/* HDCP Link Integrity Check Timer Register */
+#define SP_HDCP_LINK_CHECK_TIMER_REG 0x41
+
+/* HDCP Repeater Ready Wait Timer Register */
+#define SP_HDCP_RPTR_RDY_WAIT_TIME_REG 0x42
+
+/* HDCP Auto Timer Register */
+#define SP_HDCP_AUTO_TIMER_REG 0x51
+
+/* HDCP Key Status Register */
+#define SP_HDCP_KEY_STATUS_REG 0x5e
+
+/* HDCP Key Command Register */
+#define SP_HDCP_KEY_COMMAND_REG 0x5f
+#define SP_DISABLE_SYNC_HDCP BIT(2)
+
+/* OTP Memory Key Protection Registers */
+#define SP_OTP_KEY_PROTECT1_REG 0x60
+#define SP_OTP_KEY_PROTECT2_REG 0x61
+#define SP_OTP_KEY_PROTECT3_REG 0x62
+#define SP_OTP_PSW1 0xa2
+#define SP_OTP_PSW2 0x7e
+#define SP_OTP_PSW3 0xc6
+
+/* DP System Control Registers */
+#define SP_DP_SYSTEM_CTRL_BASE (0x80 - 1)
+/* Bits for DP System Control Register 2 */
+#define SP_CHA_STA BIT(2)
+/* Bits for DP System Control Register 3 */
+#define SP_HPD_STATUS BIT(6)
+#define SP_STRM_VALID BIT(2)
+/* Bits for DP System Control Register 4 */
+#define SP_ENHANCED_MODE BIT(3)
+
+/* DP Video Control Register */
+#define SP_DP_VIDEO_CTRL_REG 0x84
+#define SP_COLOR_F_MASK 0x06
+#define SP_COLOR_F_SHIFT 1
+#define SP_BPC_MASK 0xe0
+#define SP_BPC_SHIFT 5
+# define SP_BPC_6BITS 0x00
+# define SP_BPC_8BITS 0x01
+# define SP_BPC_10BITS 0x02
+# define SP_BPC_12BITS 0x03
+
+/* DP Audio Control Register */
+#define SP_DP_AUDIO_CTRL_REG 0x87
+#define SP_AUD_EN BIT(0)
+
+/* 10us Pulse Generate Timer Registers */
+#define SP_I2C_GEN_10US_TIMER0_REG 0x88
+#define SP_I2C_GEN_10US_TIMER1_REG 0x89
+
+/* Packet Send Control Register */
+#define SP_PACKET_SEND_CTRL_REG 0x90
+#define SP_AUD_IF_UP BIT(7)
+#define SP_AVI_IF_UD BIT(6)
+#define SP_MPEG_IF_UD BIT(5)
+#define SP_SPD_IF_UD BIT(4)
+#define SP_AUD_IF_EN BIT(3)
+#define SP_AVI_IF_EN BIT(2)
+#define SP_MPEG_IF_EN BIT(1)
+#define SP_SPD_IF_EN BIT(0)
+
+/* DP HDCP Control Register */
+#define SP_DP_HDCP_CTRL_REG 0x92
+#define SP_AUTO_EN BIT(7)
+#define SP_AUTO_START BIT(5)
+#define SP_LINK_POLLING BIT(1)
+
+/* DP Main Link Bandwidth Setting Register */
+#define SP_DP_MAIN_LINK_BW_SET_REG 0xa0
+#define SP_LINK_BW_SET_MASK 0x1f
+#define SP_INITIAL_SLIM_M_AUD_SEL BIT(5)
+
+/* DP Training Pattern Set Register */
+#define SP_DP_TRAINING_PATTERN_SET_REG 0xa2
+
+/* DP Lane 0 Link Training Control Register */
+#define SP_DP_LANE0_LT_CTRL_REG 0xa3
+#define SP_TX_SW_SET_MASK 0x1b
+#define SP_MAX_PRE_REACH BIT(5)
+#define SP_MAX_DRIVE_REACH BIT(4)
+#define SP_PRE_EMP_LEVEL1 BIT(3)
+#define SP_DRVIE_CURRENT_LEVEL1 BIT(0)
+
+/* DP Link Training Control Register */
+#define SP_DP_LT_CTRL_REG 0xa8
+#define SP_LT_ERROR_TYPE_MASK 0x70
+# define SP_LT_NO_ERROR 0x00
+# define SP_LT_AUX_WRITE_ERROR 0x01
+# define SP_LT_MAX_DRIVE_REACHED 0x02
+# define SP_LT_WRONG_LANE_COUNT_SET 0x03
+# define SP_LT_LOOP_SAME_5_TIME 0x04
+# define SP_LT_CR_FAIL_IN_EQ 0x05
+# define SP_LT_EQ_LOOP_5_TIME 0x06
+#define SP_LT_EN BIT(0)
+
+/* DP CEP Training Control Registers */
+#define SP_DP_CEP_TRAINING_CTRL0_REG 0xa9
+#define SP_DP_CEP_TRAINING_CTRL1_REG 0xaa
+
+/* DP Debug Register 1 */
+#define SP_DP_DEBUG1_REG 0xb0
+#define SP_DEBUG_PLL_LOCK BIT(4)
+#define SP_POLLING_EN BIT(1)
+
+/* DP Polling Control Register */
+#define SP_DP_POLLING_CTRL_REG 0xb4
+#define SP_AUTO_POLLING_DISABLE BIT(0)
+
+/* DP Link Debug Control Register */
+#define SP_DP_LINK_DEBUG_CTRL_REG 0xb8
+#define SP_M_VID_DEBUG BIT(5)
+#define SP_NEW_PRBS7 BIT(4)
+#define SP_INSERT_ER BIT(1)
+#define SP_PRBS31_EN BIT(0)
+
+/* AUX Misc control Register */
+#define SP_AUX_MISC_CTRL_REG 0xbf
+
+/* DP PLL control Register */
+#define SP_DP_PLL_CTRL_REG 0xc7
+#define SP_PLL_RST BIT(6)
+
+/* DP Analog Power Down Register */
+#define SP_DP_ANALOG_POWER_DOWN_REG 0xc8
+#define SP_CH0_PD BIT(0)
+
+/* DP Misc Control Register */
+#define SP_DP_MISC_CTRL_REG 0xcd
+#define SP_EQ_TRAINING_LOOP BIT(6)
+
+/* DP Extra I2C Device Address Register */
+#define SP_DP_EXTRA_I2C_DEV_ADDR_REG 0xce
+#define SP_I2C_STRETCH_DISABLE BIT(7)
+
+#define SP_I2C_EXTRA_ADDR 0x50
+
+/* DP Downspread Control Register 1 */
+#define SP_DP_DOWNSPREAD_CTRL1_REG 0xd0
+
+/* DP M Value Calculation Control Register */
+#define SP_DP_M_CALCULATION_CTRL_REG 0xd9
+#define SP_M_GEN_CLK_SEL BIT(0)
+
+/* AUX Channel Access Status Register */
+#define SP_AUX_CH_STATUS_REG 0xe0
+#define SP_AUX_STATUS 0x0f
+
+/* AUX Channel DEFER Control Register */
+#define SP_AUX_DEFER_CTRL_REG 0xe2
+#define SP_DEFER_CTRL_EN BIT(7)
+
+/* DP Buffer Data Count Register */
+#define SP_BUF_DATA_COUNT_REG 0xe4
+#define SP_BUF_DATA_COUNT_MASK 0x1f
+#define SP_BUF_CLR BIT(7)
+
+/* DP AUX Channel Control Register 1 */
+#define SP_DP_AUX_CH_CTRL1_REG 0xe5
+#define SP_AUX_TX_COMM_MASK 0x0f
+#define SP_AUX_LENGTH_MASK 0xf0
+#define SP_AUX_LENGTH_SHIFT 4
+
+/* DP AUX CH Address Register 0 */
+#define SP_AUX_ADDR_7_0_REG 0xe6
+
+/* DP AUX CH Address Register 1 */
+#define SP_AUX_ADDR_15_8_REG 0xe7
+
+/* DP AUX CH Address Register 2 */
+#define SP_AUX_ADDR_19_16_REG 0xe8
+#define SP_AUX_ADDR_19_16_MASK 0x0f
+
+/* DP AUX Channel Control Register 2 */
+#define SP_DP_AUX_CH_CTRL2_REG 0xe9
+#define SP_AUX_SEL_RXCM BIT(6)
+#define SP_AUX_CHSEL BIT(3)
+#define SP_AUX_PN_INV BIT(2)
+#define SP_ADDR_ONLY BIT(1)
+#define SP_AUX_EN BIT(0)
+
+/* DP Video Stream Control InfoFrame Register */
+#define SP_DP_3D_VSC_CTRL_REG 0xea
+#define SP_INFO_FRAME_VSC_EN BIT(0)
+
+/* DP Video Stream Data Byte 1 Register */
+#define SP_DP_VSC_DB1_REG 0xeb
+
+/* DP AUX Channel Control Register 3 */
+#define SP_DP_AUX_CH_CTRL3_REG 0xec
+#define SP_WAIT_COUNTER_7_0_MASK 0xff
+
+/* DP AUX Channel Control Register 4 */
+#define SP_DP_AUX_CH_CTRL4_REG 0xed
+
+/* DP AUX Buffer Data Registers */
+#define SP_DP_BUF_DATA0_REG 0xf0
+
+/***************************************************************/
+/* Register definition of device address 0x72 */
+/***************************************************************/
+
+/*
+ * Core Register Definitions
+ */
+
+/* Device ID Low Byte Register */
+#define SP_DEVICE_IDL_REG 0x02
+
+/* Device ID High Byte Register */
+#define SP_DEVICE_IDH_REG 0x03
+
+/* Device version register */
+#define SP_DEVICE_VERSION_REG 0x04
+
+/* Power Down Control Register */
+#define SP_POWERDOWN_CTRL_REG 0x05
+#define SP_REGISTER_PD BIT(7)
+#define SP_HDCP_PD BIT(5)
+#define SP_AUDIO_PD BIT(4)
+#define SP_VIDEO_PD BIT(3)
+#define SP_LINK_PD BIT(2)
+#define SP_TOTAL_PD BIT(1)
+
+/* Reset Control Register 1 */
+#define SP_RESET_CTRL1_REG 0x06
+#define SP_MISC_RST BIT(7)
+#define SP_VIDCAP_RST BIT(6)
+#define SP_VIDFIF_RST BIT(5)
+#define SP_AUDFIF_RST BIT(4)
+#define SP_AUDCAP_RST BIT(3)
+#define SP_HDCP_RST BIT(2)
+#define SP_SW_RST BIT(1)
+#define SP_HW_RST BIT(0)
+
+/* Reset Control Register 2 */
+#define SP_RESET_CTRL2_REG 0x07
+#define SP_AUX_RST BIT(2)
+#define SP_SERDES_FIFO_RST BIT(1)
+#define SP_I2C_REG_RST BIT(0)
+
+/* Video Control Register 1 */
+#define SP_VID_CTRL1_REG 0x08
+#define SP_VIDEO_EN BIT(7)
+#define SP_VIDEO_MUTE BIT(2)
+#define SP_DE_GEN BIT(1)
+#define SP_DEMUX BIT(0)
+
+/* Video Control Register 2 */
+#define SP_VID_CTRL2_REG 0x09
+#define SP_IN_COLOR_F_MASK 0x03
+#define SP_IN_YC_BIT_SEL BIT(2)
+#define SP_IN_BPC_MASK 0x70
+#define SP_IN_BPC_SHIFT 4
+# define SP_IN_BPC_12BIT 0x03
+# define SP_IN_BPC_10BIT 0x02
+# define SP_IN_BPC_8BIT 0x01
+# define SP_IN_BPC_6BIT 0x00
+#define SP_IN_D_RANGE BIT(7)
+
+/* Video Control Register 3 */
+#define SP_VID_CTRL3_REG 0x0a
+#define SP_HPD_OUT BIT(6)
+
+/* Video Control Register 5 */
+#define SP_VID_CTRL5_REG 0x0c
+#define SP_CSC_STD_SEL BIT(7)
+#define SP_XVYCC_RNG_LMT BIT(6)
+#define SP_RANGE_Y2R BIT(5)
+#define SP_CSPACE_Y2R BIT(4)
+#define SP_RGB_RNG_LMT BIT(3)
+#define SP_Y_RNG_LMT BIT(2)
+#define SP_RANGE_R2Y BIT(1)
+#define SP_CSPACE_R2Y BIT(0)
+
+/* Video Control Register 6 */
+#define SP_VID_CTRL6_REG 0x0d
+#define SP_TEST_PATTERN_EN BIT(7)
+#define SP_VIDEO_PROCESS_EN BIT(6)
+#define SP_VID_US_MODE BIT(3)
+#define SP_VID_DS_MODE BIT(2)
+#define SP_UP_SAMPLE BIT(1)
+#define SP_DOWN_SAMPLE BIT(0)
+
+/* Video Control Register 8 */
+#define SP_VID_CTRL8_REG 0x0f
+#define SP_VID_VRES_TH BIT(0)
+
+/* Total Line Status Low Byte Register */
+#define SP_TOTAL_LINE_STAL_REG 0x24
+
+/* Total Line Status High Byte Register */
+#define SP_TOTAL_LINE_STAH_REG 0x25
+
+/* Active Line Status Low Byte Register */
+#define SP_ACT_LINE_STAL_REG 0x26
+
+/* Active Line Status High Byte Register */
+#define SP_ACT_LINE_STAH_REG 0x27
+
+/* Vertical Front Porch Status Register */
+#define SP_V_F_PORCH_STA_REG 0x28
+
+/* Vertical SYNC Width Status Register */
+#define SP_V_SYNC_STA_REG 0x29
+
+/* Vertical Back Porch Status Register */
+#define SP_V_B_PORCH_STA_REG 0x2a
+
+/* Total Pixel Status Low Byte Register */
+#define SP_TOTAL_PIXEL_STAL_REG 0x2b
+
+/* Total Pixel Status High Byte Register */
+#define SP_TOTAL_PIXEL_STAH_REG 0x2c
+
+/* Active Pixel Status Low Byte Register */
+#define SP_ACT_PIXEL_STAL_REG 0x2d
+
+/* Active Pixel Status High Byte Register */
+#define SP_ACT_PIXEL_STAH_REG 0x2e
+
+/* Horizontal Front Porch Status Low Byte Register */
+#define SP_H_F_PORCH_STAL_REG 0x2f
+
+/* Horizontal Front Porch Statys High Byte Register */
+#define SP_H_F_PORCH_STAH_REG 0x30
+
+/* Horizontal SYNC Width Status Low Byte Register */
+#define SP_H_SYNC_STAL_REG 0x31
+
+/* Horizontal SYNC Width Status High Byte Register */
+#define SP_H_SYNC_STAH_REG 0x32
+
+/* Horizontal Back Porch Status Low Byte Register */
+#define SP_H_B_PORCH_STAL_REG 0x33
+
+/* Horizontal Back Porch Status High Byte Register */
+#define SP_H_B_PORCH_STAH_REG 0x34
+
+/* InfoFrame AVI Packet DB1 Register */
+#define SP_INFOFRAME_AVI_DB1_REG 0x70
+
+/* Bit Control Specific Register */
+#define SP_BIT_CTRL_SPECIFIC_REG 0x80
+#define SP_BIT_CTRL_SELECT_SHIFT 1
+#define SP_ENABLE_BIT_CTRL BIT(0)
+
+/* InfoFrame Audio Packet DB1 Register */
+#define SP_INFOFRAME_AUD_DB1_REG 0x83
+
+/* InfoFrame MPEG Packet DB1 Register */
+#define SP_INFOFRAME_MPEG_DB1_REG 0xb0
+
+/* Audio Channel Status Registers */
+#define SP_AUD_CH_STATUS_BASE 0xd0
+
+/* Audio Channel Num Register 5 */
+#define SP_I2S_CHANNEL_NUM_MASK 0xe0
+# define SP_I2S_CH_NUM_1 (0x00 << 5)
+# define SP_I2S_CH_NUM_2 (0x01 << 5)
+# define SP_I2S_CH_NUM_3 (0x02 << 5)
+# define SP_I2S_CH_NUM_4 (0x03 << 5)
+# define SP_I2S_CH_NUM_5 (0x04 << 5)
+# define SP_I2S_CH_NUM_6 (0x05 << 5)
+# define SP_I2S_CH_NUM_7 (0x06 << 5)
+# define SP_I2S_CH_NUM_8 (0x07 << 5)
+#define SP_EXT_VUCP BIT(2)
+#define SP_VBIT BIT(1)
+#define SP_AUDIO_LAYOUT BIT(0)
+
+/* Analog Debug Register 2 */
+#define SP_ANALOG_DEBUG2_REG 0xdd
+#define SP_FORCE_SW_OFF_BYPASS 0x20
+#define SP_XTAL_FRQ 0x1c
+# define SP_XTAL_FRQ_19M2 (0x00 << 2)
+# define SP_XTAL_FRQ_24M (0x01 << 2)
+# define SP_XTAL_FRQ_25M (0x02 << 2)
+# define SP_XTAL_FRQ_26M (0x03 << 2)
+# define SP_XTAL_FRQ_27M (0x04 << 2)
+# define SP_XTAL_FRQ_38M4 (0x05 << 2)
+# define SP_XTAL_FRQ_52M (0x06 << 2)
+#define SP_POWERON_TIME_1P5MS 0x03
+
+/* Analog Control 0 Register */
+#define SP_ANALOG_CTRL0_REG 0xe1
+
+/* Common Interrupt Status Register 1 */
+#define SP_COMMON_INT_STATUS_BASE (0xf1 - 1)
+#define SP_PLL_LOCK_CHG 0x40
+
+/* Common Interrupt Status Register 2 */
+#define SP_COMMON_INT_STATUS2 0xf2
+#define SP_HDCP_AUTH_CHG BIT(1)
+#define SP_HDCP_AUTH_DONE BIT(0)
+
+#define SP_HDCP_LINK_CHECK_FAIL BIT(0)
+
+/* Common Interrupt Status Register 4 */
+#define SP_COMMON_INT_STATUS4_REG 0xf4
+#define SP_HPD_IRQ BIT(6)
+#define SP_HPD_ESYNC_ERR BIT(4)
+#define SP_HPD_CHG BIT(2)
+#define SP_HPD_LOST BIT(1)
+#define SP_HPD_PLUG BIT(0)
+
+/* DP Interrupt Status Register */
+#define SP_DP_INT_STATUS1_REG 0xf7
+#define SP_TRAINING_FINISH BIT(5)
+#define SP_POLLING_ERR BIT(4)
+
+/* Common Interrupt Mask Register */
+#define SP_COMMON_INT_MASK_BASE (0xf8 - 1)
+
+#define SP_COMMON_INT_MASK4_REG 0xfb
+
+/* DP Interrupts Mask Register */
+#define SP_DP_INT_MASK1_REG 0xfe
+
+/* Interrupt Control Register */
+#define SP_INT_CTRL_REG 0xff
+
+/***************************************************************/
+/* Register definition of device address 0x7a */
+/***************************************************************/
+
+/* DP TX Link Training Control Register */
+#define SP_DP_TX_LT_CTRL0_REG 0x30
+
+/* PD 1.2 Lint Training 80bit Pattern Register */
+#define SP_DP_LT_80BIT_PATTERN0_REG 0x80
+#define SP_DP_LT_80BIT_PATTERN_REG_NUM 10
+
+/* Audio Interface Control Register 0 */
+#define SP_AUD_INTERFACE_CTRL0_REG 0x5f
+#define SP_AUD_INTERFACE_DISABLE 0x80
+
+/* Audio Interface Control Register 2 */
+#define SP_AUD_INTERFACE_CTRL2_REG 0x60
+#define SP_M_AUD_ADJUST_ST 0x04
+
+/* Audio Interface Control Register 3 */
+#define SP_AUD_INTERFACE_CTRL3_REG 0x62
+
+/* Audio Interface Control Register 4 */
+#define SP_AUD_INTERFACE_CTRL4_REG 0x67
+
+/* Audio Interface Control Register 5 */
+#define SP_AUD_INTERFACE_CTRL5_REG 0x68
+
+/* Audio Interface Control Register 6 */
+#define SP_AUD_INTERFACE_CTRL6_REG 0x69
+
+/* Firmware Version Register */
+#define SP_FW_VER_REG 0xb7
+
+#endif
diff --git a/drivers/gpu/drm/bridge/analogix/Kconfig b/drivers/gpu/drm/bridge/analogix/Kconfig
new file mode 100644
index 0000000..80f286f
--- /dev/null
+++ b/drivers/gpu/drm/bridge/analogix/Kconfig
@@ -0,0 +1,3 @@
+config DRM_ANALOGIX_DP
+ tristate
+ depends on DRM
diff --git a/drivers/gpu/drm/bridge/analogix/Makefile b/drivers/gpu/drm/bridge/analogix/Makefile
new file mode 100644
index 0000000..cd4010b
--- /dev/null
+++ b/drivers/gpu/drm/bridge/analogix/Makefile
@@ -0,0 +1,2 @@
+analogix_dp-objs := analogix_dp_core.o analogix_dp_reg.o
+obj-$(CONFIG_DRM_ANALOGIX_DP) += analogix_dp.o
diff --git a/drivers/gpu/drm/bridge/analogix/analogix_dp_core.c b/drivers/gpu/drm/bridge/analogix/analogix_dp_core.c
new file mode 100644
index 0000000..7699597
--- /dev/null
+++ b/drivers/gpu/drm/bridge/analogix/analogix_dp_core.c
@@ -0,0 +1,1430 @@
+/*
+* Analogix DP (Display Port) core interface driver.
+*
+* Copyright (C) 2012 Samsung Electronics Co., Ltd.
+* Author: Jingoo Han <jg1.han@samsung.com>
+*
+* This program is free software; you can redistribute it and/or modify it
+* under the terms of the GNU General Public License as published by the
+* Free Software Foundation; either version 2 of the License, or (at your
+* option) any later version.
+*/
+
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/err.h>
+#include <linux/clk.h>
+#include <linux/io.h>
+#include <linux/interrupt.h>
+#include <linux/of.h>
+#include <linux/of_gpio.h>
+#include <linux/gpio.h>
+#include <linux/component.h>
+#include <linux/phy/phy.h>
+
+#include <drm/drmP.h>
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_crtc.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_panel.h>
+
+#include <drm/bridge/analogix_dp.h>
+
+#include "analogix_dp_core.h"
+
+#define to_dp(nm) container_of(nm, struct analogix_dp_device, nm)
+
+struct bridge_init {
+ struct i2c_client *client;
+ struct device_node *node;
+};
+
+static void analogix_dp_init_dp(struct analogix_dp_device *dp)
+{
+ analogix_dp_reset(dp);
+
+ analogix_dp_swreset(dp);
+
+ analogix_dp_init_analog_param(dp);
+ analogix_dp_init_interrupt(dp);
+
+ /* SW defined function Normal operation */
+ analogix_dp_enable_sw_function(dp);
+
+ analogix_dp_config_interrupt(dp);
+ analogix_dp_init_analog_func(dp);
+
+ analogix_dp_init_hpd(dp);
+ analogix_dp_init_aux(dp);
+}
+
+static int analogix_dp_detect_hpd(struct analogix_dp_device *dp)
+{
+ int timeout_loop = 0;
+
+ while (timeout_loop < DP_TIMEOUT_LOOP_COUNT) {
+ if (analogix_dp_get_plug_in_status(dp) == 0)
+ return 0;
+
+ timeout_loop++;
+ usleep_range(10, 11);
+ }
+
+ /*
+ * Some edp screen do not have hpd signal, so we can't just
+ * return failed when hpd plug in detect failed, DT property
+ * "force-hpd" would indicate whether driver need this.
+ */
+ if (!dp->force_hpd)
+ return -ETIMEDOUT;
+
+ /*
+ * The eDP TRM indicate that if HPD_STATUS(RO) is 0, AUX CH
+ * will not work, so we need to give a force hpd action to
+ * set HPD_STATUS manually.
+ */
+ dev_dbg(dp->dev, "failed to get hpd plug status, try to force hpd\n");
+
+ analogix_dp_force_hpd(dp);
+
+ if (analogix_dp_get_plug_in_status(dp) != 0) {
+ dev_err(dp->dev, "failed to get hpd plug in status\n");
+ return -EINVAL;
+ }
+
+ dev_dbg(dp->dev, "success to get plug in status after force hpd\n");
+
+ return 0;
+}
+
+static unsigned char analogix_dp_calc_edid_check_sum(unsigned char *edid_data)
+{
+ int i;
+ unsigned char sum = 0;
+
+ for (i = 0; i < EDID_BLOCK_LENGTH; i++)
+ sum = sum + edid_data[i];
+
+ return sum;
+}
+
+static int analogix_dp_read_edid(struct analogix_dp_device *dp)
+{
+ unsigned char *edid = dp->edid;
+ unsigned int extend_block = 0;
+ unsigned char sum;
+ unsigned char test_vector;
+ int retval;
+
+ /*
+ * EDID device address is 0x50.
+ * However, if necessary, you must have set upper address
+ * into E-EDID in I2C device, 0x30.
+ */
+
+ /* Read Extension Flag, Number of 128-byte EDID extension blocks */
+ retval = analogix_dp_read_byte_from_i2c(dp, I2C_EDID_DEVICE_ADDR,
+ EDID_EXTENSION_FLAG,
+ &extend_block);
+ if (retval)
+ return retval;
+
+ if (extend_block > 0) {
+ dev_dbg(dp->dev, "EDID data includes a single extension!\n");
+
+ /* Read EDID data */
+ retval = analogix_dp_read_bytes_from_i2c(dp,
+ I2C_EDID_DEVICE_ADDR,
+ EDID_HEADER_PATTERN,
+ EDID_BLOCK_LENGTH,
+ &edid[EDID_HEADER_PATTERN]);
+ if (retval != 0) {
+ dev_err(dp->dev, "EDID Read failed!\n");
+ return -EIO;
+ }
+ sum = analogix_dp_calc_edid_check_sum(edid);
+ if (sum != 0) {
+ dev_err(dp->dev, "EDID bad checksum!\n");
+ return -EIO;
+ }
+
+ /* Read additional EDID data */
+ retval = analogix_dp_read_bytes_from_i2c(dp,
+ I2C_EDID_DEVICE_ADDR,
+ EDID_BLOCK_LENGTH,
+ EDID_BLOCK_LENGTH,
+ &edid[EDID_BLOCK_LENGTH]);
+ if (retval != 0) {
+ dev_err(dp->dev, "EDID Read failed!\n");
+ return -EIO;
+ }
+ sum = analogix_dp_calc_edid_check_sum(&edid[EDID_BLOCK_LENGTH]);
+ if (sum != 0) {
+ dev_err(dp->dev, "EDID bad checksum!\n");
+ return -EIO;
+ }
+
+ analogix_dp_read_byte_from_dpcd(dp, DP_TEST_REQUEST,
+ &test_vector);
+ if (test_vector & DP_TEST_LINK_EDID_READ) {
+ analogix_dp_write_byte_to_dpcd(dp,
+ DP_TEST_EDID_CHECKSUM,
+ edid[EDID_BLOCK_LENGTH + EDID_CHECKSUM]);
+ analogix_dp_write_byte_to_dpcd(dp,
+ DP_TEST_RESPONSE,
+ DP_TEST_EDID_CHECKSUM_WRITE);
+ }
+ } else {
+ dev_info(dp->dev, "EDID data does not include any extensions.\n");
+
+ /* Read EDID data */
+ retval = analogix_dp_read_bytes_from_i2c(dp,
+ I2C_EDID_DEVICE_ADDR, EDID_HEADER_PATTERN,
+ EDID_BLOCK_LENGTH, &edid[EDID_HEADER_PATTERN]);
+ if (retval != 0) {
+ dev_err(dp->dev, "EDID Read failed!\n");
+ return -EIO;
+ }
+ sum = analogix_dp_calc_edid_check_sum(edid);
+ if (sum != 0) {
+ dev_err(dp->dev, "EDID bad checksum!\n");
+ return -EIO;
+ }
+
+ analogix_dp_read_byte_from_dpcd(dp, DP_TEST_REQUEST,
+ &test_vector);
+ if (test_vector & DP_TEST_LINK_EDID_READ) {
+ analogix_dp_write_byte_to_dpcd(dp,
+ DP_TEST_EDID_CHECKSUM, edid[EDID_CHECKSUM]);
+ analogix_dp_write_byte_to_dpcd(dp,
+ DP_TEST_RESPONSE, DP_TEST_EDID_CHECKSUM_WRITE);
+ }
+ }
+
+ dev_dbg(dp->dev, "EDID Read success!\n");
+ return 0;
+}
+
+static int analogix_dp_handle_edid(struct analogix_dp_device *dp)
+{
+ u8 buf[12];
+ int i;
+ int retval;
+
+ /* Read DPCD DP_DPCD_REV~RECEIVE_PORT1_CAP_1 */
+ retval = analogix_dp_read_bytes_from_dpcd(dp, DP_DPCD_REV, 12, buf);
+ if (retval)
+ return retval;
+
+ /* Read EDID */
+ for (i = 0; i < 3; i++) {
+ retval = analogix_dp_read_edid(dp);
+ if (!retval)
+ break;
+ }
+
+ return retval;
+}
+
+static void
+analogix_dp_enable_rx_to_enhanced_mode(struct analogix_dp_device *dp,
+ bool enable)
+{
+ u8 data;
+
+ analogix_dp_read_byte_from_dpcd(dp, DP_LANE_COUNT_SET, &data);
+
+ if (enable)
+ analogix_dp_write_byte_to_dpcd(dp, DP_LANE_COUNT_SET,
+ DP_LANE_COUNT_ENHANCED_FRAME_EN |
+ DPCD_LANE_COUNT_SET(data));
+ else
+ analogix_dp_write_byte_to_dpcd(dp, DP_LANE_COUNT_SET,
+ DPCD_LANE_COUNT_SET(data));
+}
+
+static int analogix_dp_is_enhanced_mode_available(struct analogix_dp_device *dp)
+{
+ u8 data;
+ int retval;
+
+ analogix_dp_read_byte_from_dpcd(dp, DP_MAX_LANE_COUNT, &data);
+ retval = DPCD_ENHANCED_FRAME_CAP(data);
+
+ return retval;
+}
+
+static void analogix_dp_set_enhanced_mode(struct analogix_dp_device *dp)
+{
+ u8 data;
+
+ data = analogix_dp_is_enhanced_mode_available(dp);
+ analogix_dp_enable_rx_to_enhanced_mode(dp, data);
+ analogix_dp_enable_enhanced_mode(dp, data);
+}
+
+static void analogix_dp_training_pattern_dis(struct analogix_dp_device *dp)
+{
+ analogix_dp_set_training_pattern(dp, DP_NONE);
+
+ analogix_dp_write_byte_to_dpcd(dp, DP_TRAINING_PATTERN_SET,
+ DP_TRAINING_PATTERN_DISABLE);
+}
+
+static void
+analogix_dp_set_lane_lane_pre_emphasis(struct analogix_dp_device *dp,
+ int pre_emphasis, int lane)
+{
+ switch (lane) {
+ case 0:
+ analogix_dp_set_lane0_pre_emphasis(dp, pre_emphasis);
+ break;
+ case 1:
+ analogix_dp_set_lane1_pre_emphasis(dp, pre_emphasis);
+ break;
+
+ case 2:
+ analogix_dp_set_lane2_pre_emphasis(dp, pre_emphasis);
+ break;
+
+ case 3:
+ analogix_dp_set_lane3_pre_emphasis(dp, pre_emphasis);
+ break;
+ }
+}
+
+static int analogix_dp_link_start(struct analogix_dp_device *dp)
+{
+ u8 buf[4];
+ int lane, lane_count, pll_tries, retval;
+
+ lane_count = dp->link_train.lane_count;
+
+ dp->link_train.lt_state = CLOCK_RECOVERY;
+ dp->link_train.eq_loop = 0;
+
+ for (lane = 0; lane < lane_count; lane++)
+ dp->link_train.cr_loop[lane] = 0;
+
+ /* Set link rate and count as you want to establish*/
+ analogix_dp_set_link_bandwidth(dp, dp->link_train.link_rate);
+ analogix_dp_set_lane_count(dp, dp->link_train.lane_count);
+
+ /* Setup RX configuration */
+ buf[0] = dp->link_train.link_rate;
+ buf[1] = dp->link_train.lane_count;
+ retval = analogix_dp_write_bytes_to_dpcd(dp, DP_LINK_BW_SET, 2, buf);
+ if (retval)
+ return retval;
+
+ /* Set TX pre-emphasis to minimum */
+ for (lane = 0; lane < lane_count; lane++)
+ analogix_dp_set_lane_lane_pre_emphasis(dp,
+ PRE_EMPHASIS_LEVEL_0, lane);
+
+ /* Wait for PLL lock */
+ pll_tries = 0;
+ while (analogix_dp_get_pll_lock_status(dp) == PLL_UNLOCKED) {
+ if (pll_tries == DP_TIMEOUT_LOOP_COUNT) {
+ dev_err(dp->dev, "Wait for PLL lock timed out\n");
+ return -ETIMEDOUT;
+ }
+
+ pll_tries++;
+ usleep_range(90, 120);
+ }
+
+ /* Set training pattern 1 */
+ analogix_dp_set_training_pattern(dp, TRAINING_PTN1);
+
+ /* Set RX training pattern */
+ retval = analogix_dp_write_byte_to_dpcd(dp,
+ DP_TRAINING_PATTERN_SET,
+ DP_LINK_SCRAMBLING_DISABLE | DP_TRAINING_PATTERN_1);
+ if (retval)
+ return retval;
+
+ for (lane = 0; lane < lane_count; lane++)
+ buf[lane] = DP_TRAIN_PRE_EMPH_LEVEL_0 |
+ DP_TRAIN_VOLTAGE_SWING_LEVEL_0;
+
+ retval = analogix_dp_write_bytes_to_dpcd(dp, DP_TRAINING_LANE0_SET,
+ lane_count, buf);
+
+ return retval;
+}
+
+static unsigned char analogix_dp_get_lane_status(u8 link_status[2], int lane)
+{
+ int shift = (lane & 1) * 4;
+ u8 link_value = link_status[lane >> 1];
+
+ return (link_value >> shift) & 0xf;
+}
+
+static int analogix_dp_clock_recovery_ok(u8 link_status[2], int lane_count)
+{
+ int lane;
+ u8 lane_status;
+
+ for (lane = 0; lane < lane_count; lane++) {
+ lane_status = analogix_dp_get_lane_status(link_status, lane);
+ if ((lane_status & DP_LANE_CR_DONE) == 0)
+ return -EINVAL;
+ }
+ return 0;
+}
+
+static int analogix_dp_channel_eq_ok(u8 link_status[2], u8 link_align,
+ int lane_count)
+{
+ int lane;
+ u8 lane_status;
+
+ if ((link_align & DP_INTERLANE_ALIGN_DONE) == 0)
+ return -EINVAL;
+
+ for (lane = 0; lane < lane_count; lane++) {
+ lane_status = analogix_dp_get_lane_status(link_status, lane);
+ lane_status &= DP_CHANNEL_EQ_BITS;
+ if (lane_status != DP_CHANNEL_EQ_BITS)
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static unsigned char
+analogix_dp_get_adjust_request_voltage(u8 adjust_request[2], int lane)
+{
+ int shift = (lane & 1) * 4;
+ u8 link_value = adjust_request[lane >> 1];
+
+ return (link_value >> shift) & 0x3;
+}
+
+static unsigned char analogix_dp_get_adjust_request_pre_emphasis(
+ u8 adjust_request[2],
+ int lane)
+{
+ int shift = (lane & 1) * 4;
+ u8 link_value = adjust_request[lane >> 1];
+
+ return ((link_value >> shift) & 0xc) >> 2;
+}
+
+static void analogix_dp_set_lane_link_training(struct analogix_dp_device *dp,
+ u8 training_lane_set, int lane)
+{
+ switch (lane) {
+ case 0:
+ analogix_dp_set_lane0_link_training(dp, training_lane_set);
+ break;
+ case 1:
+ analogix_dp_set_lane1_link_training(dp, training_lane_set);
+ break;
+
+ case 2:
+ analogix_dp_set_lane2_link_training(dp, training_lane_set);
+ break;
+
+ case 3:
+ analogix_dp_set_lane3_link_training(dp, training_lane_set);
+ break;
+ }
+}
+
+static unsigned int
+analogix_dp_get_lane_link_training(struct analogix_dp_device *dp,
+ int lane)
+{
+ u32 reg;
+
+ switch (lane) {
+ case 0:
+ reg = analogix_dp_get_lane0_link_training(dp);
+ break;
+ case 1:
+ reg = analogix_dp_get_lane1_link_training(dp);
+ break;
+ case 2:
+ reg = analogix_dp_get_lane2_link_training(dp);
+ break;
+ case 3:
+ reg = analogix_dp_get_lane3_link_training(dp);
+ break;
+ default:
+ WARN_ON(1);
+ return 0;
+ }
+
+ return reg;
+}
+
+static void analogix_dp_reduce_link_rate(struct analogix_dp_device *dp)
+{
+ analogix_dp_training_pattern_dis(dp);
+ analogix_dp_set_enhanced_mode(dp);
+
+ dp->link_train.lt_state = FAILED;
+}
+
+static void analogix_dp_get_adjust_training_lane(struct analogix_dp_device *dp,
+ u8 adjust_request[2])
+{
+ int lane, lane_count;
+ u8 voltage_swing, pre_emphasis, training_lane;
+
+ lane_count = dp->link_train.lane_count;
+ for (lane = 0; lane < lane_count; lane++) {
+ voltage_swing = analogix_dp_get_adjust_request_voltage(
+ adjust_request, lane);
+ pre_emphasis = analogix_dp_get_adjust_request_pre_emphasis(
+ adjust_request, lane);
+ training_lane = DPCD_VOLTAGE_SWING_SET(voltage_swing) |
+ DPCD_PRE_EMPHASIS_SET(pre_emphasis);
+
+ if (voltage_swing == VOLTAGE_LEVEL_3)
+ training_lane |= DP_TRAIN_MAX_SWING_REACHED;
+ if (pre_emphasis == PRE_EMPHASIS_LEVEL_3)
+ training_lane |= DP_TRAIN_MAX_PRE_EMPHASIS_REACHED;
+
+ dp->link_train.training_lane[lane] = training_lane;
+ }
+}
+
+static int analogix_dp_process_clock_recovery(struct analogix_dp_device *dp)
+{
+ int lane, lane_count, retval;
+ u8 voltage_swing, pre_emphasis, training_lane;
+ u8 link_status[2], adjust_request[2];
+
+ usleep_range(100, 101);
+
+ lane_count = dp->link_train.lane_count;
+
+ retval = analogix_dp_read_bytes_from_dpcd(dp,
+ DP_LANE0_1_STATUS, 2, link_status);
+ if (retval)
+ return retval;
+
+ retval = analogix_dp_read_bytes_from_dpcd(dp,
+ DP_ADJUST_REQUEST_LANE0_1, 2, adjust_request);
+ if (retval)
+ return retval;
+
+ if (analogix_dp_clock_recovery_ok(link_status, lane_count) == 0) {
+ /* set training pattern 2 for EQ */
+ analogix_dp_set_training_pattern(dp, TRAINING_PTN2);
+
+ retval = analogix_dp_write_byte_to_dpcd(dp,
+ DP_TRAINING_PATTERN_SET,
+ DP_LINK_SCRAMBLING_DISABLE |
+ DP_TRAINING_PATTERN_2);
+ if (retval)
+ return retval;
+
+ dev_info(dp->dev, "Link Training Clock Recovery success\n");
+ dp->link_train.lt_state = EQUALIZER_TRAINING;
+ } else {
+ for (lane = 0; lane < lane_count; lane++) {
+ training_lane = analogix_dp_get_lane_link_training(
+ dp, lane);
+ voltage_swing = analogix_dp_get_adjust_request_voltage(
+ adjust_request, lane);
+ pre_emphasis = analogix_dp_get_adjust_request_pre_emphasis(
+ adjust_request, lane);
+
+ if (DPCD_VOLTAGE_SWING_GET(training_lane) ==
+ voltage_swing &&
+ DPCD_PRE_EMPHASIS_GET(training_lane) ==
+ pre_emphasis)
+ dp->link_train.cr_loop[lane]++;
+
+ if (dp->link_train.cr_loop[lane] == MAX_CR_LOOP ||
+ voltage_swing == VOLTAGE_LEVEL_3 ||
+ pre_emphasis == PRE_EMPHASIS_LEVEL_3) {
+ dev_err(dp->dev, "CR Max reached (%d,%d,%d)\n",
+ dp->link_train.cr_loop[lane],
+ voltage_swing, pre_emphasis);
+ analogix_dp_reduce_link_rate(dp);
+ return -EIO;
+ }
+ }
+ }
+
+ analogix_dp_get_adjust_training_lane(dp, adjust_request);
+
+ for (lane = 0; lane < lane_count; lane++)
+ analogix_dp_set_lane_link_training(dp,
+ dp->link_train.training_lane[lane], lane);
+
+ retval = analogix_dp_write_bytes_to_dpcd(dp,
+ DP_TRAINING_LANE0_SET, lane_count,
+ dp->link_train.training_lane);
+ if (retval)
+ return retval;
+
+ return retval;
+}
+
+static int analogix_dp_process_equalizer_training(struct analogix_dp_device *dp)
+{
+ int lane, lane_count, retval;
+ u32 reg;
+ u8 link_align, link_status[2], adjust_request[2];
+
+ usleep_range(400, 401);
+
+ lane_count = dp->link_train.lane_count;
+
+ retval = analogix_dp_read_bytes_from_dpcd(dp,
+ DP_LANE0_1_STATUS, 2, link_status);
+ if (retval)
+ return retval;
+
+ if (analogix_dp_clock_recovery_ok(link_status, lane_count)) {
+ analogix_dp_reduce_link_rate(dp);
+ return -EIO;
+ }
+
+ retval = analogix_dp_read_bytes_from_dpcd(dp,
+ DP_ADJUST_REQUEST_LANE0_1, 2, adjust_request);
+ if (retval)
+ return retval;
+
+ retval = analogix_dp_read_byte_from_dpcd(dp,
+ DP_LANE_ALIGN_STATUS_UPDATED, &link_align);
+ if (retval)
+ return retval;
+
+ analogix_dp_get_adjust_training_lane(dp, adjust_request);
+
+ if (!analogix_dp_channel_eq_ok(link_status, link_align, lane_count)) {
+ /* traing pattern Set to Normal */
+ analogix_dp_training_pattern_dis(dp);
+
+ dev_info(dp->dev, "Link Training success!\n");
+
+ analogix_dp_get_link_bandwidth(dp, ®);
+ dp->link_train.link_rate = reg;
+ dev_dbg(dp->dev, "final bandwidth = %.2x\n",
+ dp->link_train.link_rate);
+
+ analogix_dp_get_lane_count(dp, ®);
+ dp->link_train.lane_count = reg;
+ dev_dbg(dp->dev, "final lane count = %.2x\n",
+ dp->link_train.lane_count);
+
+ /* set enhanced mode if available */
+ analogix_dp_set_enhanced_mode(dp);
+ dp->link_train.lt_state = FINISHED;
+
+ return 0;
+ }
+
+ /* not all locked */
+ dp->link_train.eq_loop++;
+
+ if (dp->link_train.eq_loop > MAX_EQ_LOOP) {
+ dev_err(dp->dev, "EQ Max loop\n");
+ analogix_dp_reduce_link_rate(dp);
+ return -EIO;
+ }
+
+ for (lane = 0; lane < lane_count; lane++)
+ analogix_dp_set_lane_link_training(dp,
+ dp->link_train.training_lane[lane], lane);
+
+ retval = analogix_dp_write_bytes_to_dpcd(dp, DP_TRAINING_LANE0_SET,
+ lane_count, dp->link_train.training_lane);
+
+ return retval;
+}
+
+static void analogix_dp_get_max_rx_bandwidth(struct analogix_dp_device *dp,
+ u8 *bandwidth)
+{
+ u8 data;
+
+ /*
+ * For DP rev.1.1, Maximum link rate of Main Link lanes
+ * 0x06 = 1.62 Gbps, 0x0a = 2.7 Gbps
+ * For DP rev.1.2, Maximum link rate of Main Link lanes
+ * 0x06 = 1.62 Gbps, 0x0a = 2.7 Gbps, 0x14 = 5.4Gbps
+ */
+ analogix_dp_read_byte_from_dpcd(dp, DP_MAX_LINK_RATE, &data);
+ *bandwidth = data;
+}
+
+static void analogix_dp_get_max_rx_lane_count(struct analogix_dp_device *dp,
+ u8 *lane_count)
+{
+ u8 data;
+
+ /*
+ * For DP rev.1.1, Maximum number of Main Link lanes
+ * 0x01 = 1 lane, 0x02 = 2 lanes, 0x04 = 4 lanes
+ */
+ analogix_dp_read_byte_from_dpcd(dp, DP_MAX_LANE_COUNT, &data);
+ *lane_count = DPCD_MAX_LANE_COUNT(data);
+}
+
+static void analogix_dp_init_training(struct analogix_dp_device *dp,
+ enum link_lane_count_type max_lane,
+ int max_rate)
+{
+ /*
+ * MACRO_RST must be applied after the PLL_LOCK to avoid
+ * the DP inter pair skew issue for at least 10 us
+ */
+ analogix_dp_reset_macro(dp);
+
+ /* Initialize by reading RX's DPCD */
+ analogix_dp_get_max_rx_bandwidth(dp, &dp->link_train.link_rate);
+ analogix_dp_get_max_rx_lane_count(dp, &dp->link_train.lane_count);
+
+ if ((dp->link_train.link_rate != DP_LINK_BW_1_62) &&
+ (dp->link_train.link_rate != DP_LINK_BW_2_7) &&
+ (dp->link_train.link_rate != DP_LINK_BW_5_4)) {
+ dev_err(dp->dev, "Rx Max Link Rate is abnormal :%x !\n",
+ dp->link_train.link_rate);
+ dp->link_train.link_rate = DP_LINK_BW_1_62;
+ }
+
+ if (dp->link_train.lane_count == 0) {
+ dev_err(dp->dev, "Rx Max Lane count is abnormal :%x !\n",
+ dp->link_train.lane_count);
+ dp->link_train.lane_count = (u8)LANE_COUNT1;
+ }
+
+ /* Setup TX lane count & rate */
+ if (dp->link_train.lane_count > max_lane)
+ dp->link_train.lane_count = max_lane;
+ if (dp->link_train.link_rate > max_rate)
+ dp->link_train.link_rate = max_rate;
+
+ /* All DP analog module power up */
+ analogix_dp_set_analog_power_down(dp, POWER_ALL, 0);
+}
+
+static int analogix_dp_sw_link_training(struct analogix_dp_device *dp)
+{
+ int retval = 0, training_finished = 0;
+
+ dp->link_train.lt_state = START;
+
+ /* Process here */
+ while (!retval && !training_finished) {
+ switch (dp->link_train.lt_state) {
+ case START:
+ retval = analogix_dp_link_start(dp);
+ if (retval)
+ dev_err(dp->dev, "LT link start failed!\n");
+ break;
+ case CLOCK_RECOVERY:
+ retval = analogix_dp_process_clock_recovery(dp);
+ if (retval)
+ dev_err(dp->dev, "LT CR failed!\n");
+ break;
+ case EQUALIZER_TRAINING:
+ retval = analogix_dp_process_equalizer_training(dp);
+ if (retval)
+ dev_err(dp->dev, "LT EQ failed!\n");
+ break;
+ case FINISHED:
+ training_finished = 1;
+ break;
+ case FAILED:
+ return -EREMOTEIO;
+ }
+ }
+ if (retval)
+ dev_err(dp->dev, "eDP link training failed (%d)\n", retval);
+
+ return retval;
+}
+
+static int analogix_dp_set_link_train(struct analogix_dp_device *dp,
+ u32 count, u32 bwtype)
+{
+ int i;
+ int retval;
+
+ for (i = 0; i < DP_TIMEOUT_LOOP_COUNT; i++) {
+ analogix_dp_init_training(dp, count, bwtype);
+ retval = analogix_dp_sw_link_training(dp);
+ if (retval == 0)
+ break;
+
+ usleep_range(100, 110);
+ }
+
+ return retval;
+}
+
+static int analogix_dp_config_video(struct analogix_dp_device *dp)
+{
+ int retval = 0;
+ int timeout_loop = 0;
+ int done_count = 0;
+
+ analogix_dp_config_video_slave_mode(dp);
+
+ analogix_dp_set_video_color_format(dp);
+
+ if (analogix_dp_get_pll_lock_status(dp) == PLL_UNLOCKED) {
+ dev_err(dp->dev, "PLL is not locked yet.\n");
+ return -EINVAL;
+ }
+
+ for (;;) {
+ timeout_loop++;
+ if (analogix_dp_is_slave_video_stream_clock_on(dp) == 0)
+ break;
+ if (timeout_loop > DP_TIMEOUT_LOOP_COUNT) {
+ dev_err(dp->dev, "Timeout of video streamclk ok\n");
+ return -ETIMEDOUT;
+ }
+
+ usleep_range(1, 2);
+ }
+
+ /* Set to use the register calculated M/N video */
+ analogix_dp_set_video_cr_mn(dp, CALCULATED_M, 0, 0);
+
+ /* For video bist, Video timing must be generated by register */
+ analogix_dp_set_video_timing_mode(dp, VIDEO_TIMING_FROM_CAPTURE);
+
+ /* Disable video mute */
+ analogix_dp_enable_video_mute(dp, 0);
+
+ /* Configure video slave mode */
+ analogix_dp_enable_video_master(dp, 0);
+
+ timeout_loop = 0;
+
+ for (;;) {
+ timeout_loop++;
+ if (analogix_dp_is_video_stream_on(dp) == 0) {
+ done_count++;
+ if (done_count > 10)
+ break;
+ } else if (done_count) {
+ done_count = 0;
+ }
+ if (timeout_loop > DP_TIMEOUT_LOOP_COUNT) {
+ dev_err(dp->dev, "Timeout of video streamclk ok\n");
+ return -ETIMEDOUT;
+ }
+
+ usleep_range(1000, 1001);
+ }
+
+ if (retval != 0)
+ dev_err(dp->dev, "Video stream is not detected!\n");
+
+ return retval;
+}
+
+static void analogix_dp_enable_scramble(struct analogix_dp_device *dp,
+ bool enable)
+{
+ u8 data;
+
+ if (enable) {
+ analogix_dp_enable_scrambling(dp);
+
+ analogix_dp_read_byte_from_dpcd(dp, DP_TRAINING_PATTERN_SET,
+ &data);
+ analogix_dp_write_byte_to_dpcd(dp,
+ DP_TRAINING_PATTERN_SET,
+ (u8)(data & ~DP_LINK_SCRAMBLING_DISABLE));
+ } else {
+ analogix_dp_disable_scrambling(dp);
+
+ analogix_dp_read_byte_from_dpcd(dp, DP_TRAINING_PATTERN_SET,
+ &data);
+ analogix_dp_write_byte_to_dpcd(dp,
+ DP_TRAINING_PATTERN_SET,
+ (u8)(data | DP_LINK_SCRAMBLING_DISABLE));
+ }
+}
+
+static irqreturn_t analogix_dp_hardirq(int irq, void *arg)
+{
+ struct analogix_dp_device *dp = arg;
+ irqreturn_t ret = IRQ_NONE;
+ enum dp_irq_type irq_type;
+
+ irq_type = analogix_dp_get_irq_type(dp);
+ if (irq_type != DP_IRQ_TYPE_UNKNOWN) {
+ analogix_dp_mute_hpd_interrupt(dp);
+ ret = IRQ_WAKE_THREAD;
+ }
+
+ return ret;
+}
+
+static irqreturn_t analogix_dp_irq_thread(int irq, void *arg)
+{
+ struct analogix_dp_device *dp = arg;
+ enum dp_irq_type irq_type;
+
+ irq_type = analogix_dp_get_irq_type(dp);
+ if (irq_type & DP_IRQ_TYPE_HP_CABLE_IN ||
+ irq_type & DP_IRQ_TYPE_HP_CABLE_OUT) {
+ dev_dbg(dp->dev, "Detected cable status changed!\n");
+ if (dp->drm_dev)
+ drm_helper_hpd_irq_event(dp->drm_dev);
+ }
+
+ if (irq_type != DP_IRQ_TYPE_UNKNOWN) {
+ analogix_dp_clear_hotplug_interrupts(dp);
+ analogix_dp_unmute_hpd_interrupt(dp);
+ }
+
+ return IRQ_HANDLED;
+}
+
+static void analogix_dp_commit(struct analogix_dp_device *dp)
+{
+ int ret;
+
+ /* Keep the panel disabled while we configure video */
+ if (dp->plat_data->panel) {
+ if (drm_panel_disable(dp->plat_data->panel))
+ DRM_ERROR("failed to disable the panel\n");
+ }
+
+ ret = analogix_dp_set_link_train(dp, dp->video_info.max_lane_count,
+ dp->video_info.max_link_rate);
+ if (ret) {
+ dev_err(dp->dev, "unable to do link train\n");
+ return;
+ }
+
+ analogix_dp_enable_scramble(dp, 1);
+ analogix_dp_enable_rx_to_enhanced_mode(dp, 1);
+ analogix_dp_enable_enhanced_mode(dp, 1);
+
+ analogix_dp_init_video(dp);
+ ret = analogix_dp_config_video(dp);
+ if (ret)
+ dev_err(dp->dev, "unable to config video\n");
+
+ /* Safe to enable the panel now */
+ if (dp->plat_data->panel) {
+ if (drm_panel_enable(dp->plat_data->panel))
+ DRM_ERROR("failed to enable the panel\n");
+ }
+
+ /* Enable video */
+ analogix_dp_start_video(dp);
+}
+
+int analogix_dp_get_modes(struct drm_connector *connector)
+{
+ struct analogix_dp_device *dp = to_dp(connector);
+ struct edid *edid = (struct edid *)dp->edid;
+ int num_modes = 0;
+
+ if (analogix_dp_handle_edid(dp) == 0) {
+ drm_mode_connector_update_edid_property(&dp->connector, edid);
+ num_modes += drm_add_edid_modes(&dp->connector, edid);
+ }
+
+ if (dp->plat_data->panel)
+ num_modes += drm_panel_get_modes(dp->plat_data->panel);
+
+ if (dp->plat_data->get_modes)
+ num_modes += dp->plat_data->get_modes(dp->plat_data);
+
+ return num_modes;
+}
+
+static struct drm_encoder *
+analogix_dp_best_encoder(struct drm_connector *connector)
+{
+ struct analogix_dp_device *dp = to_dp(connector);
+
+ return dp->encoder;
+}
+
+static const struct drm_connector_helper_funcs analogix_dp_connector_helper_funcs = {
+ .get_modes = analogix_dp_get_modes,
+ .best_encoder = analogix_dp_best_encoder,
+};
+
+enum drm_connector_status
+analogix_dp_detect(struct drm_connector *connector, bool force)
+{
+ struct analogix_dp_device *dp = to_dp(connector);
+
+ if (analogix_dp_detect_hpd(dp))
+ return connector_status_disconnected;
+
+ return connector_status_connected;
+}
+
+static void analogix_dp_connector_destroy(struct drm_connector *connector)
+{
+ drm_connector_unregister(connector);
+ drm_connector_cleanup(connector);
+
+}
+
+static const struct drm_connector_funcs analogix_dp_connector_funcs = {
+ .dpms = drm_atomic_helper_connector_dpms,
+ .fill_modes = drm_helper_probe_single_connector_modes,
+ .detect = analogix_dp_detect,
+ .destroy = analogix_dp_connector_destroy,
+ .reset = drm_atomic_helper_connector_reset,
+ .atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
+};
+
+static int analogix_dp_bridge_attach(struct drm_bridge *bridge)
+{
+ struct analogix_dp_device *dp = bridge->driver_private;
+ struct drm_encoder *encoder = dp->encoder;
+ struct drm_connector *connector = &dp->connector;
+ int ret;
+
+ if (!bridge->encoder) {
+ DRM_ERROR("Parent encoder object not found");
+ return -ENODEV;
+ }
+
+ connector->polled = DRM_CONNECTOR_POLL_HPD;
+
+ ret = drm_connector_init(dp->drm_dev, connector,
+ &analogix_dp_connector_funcs,
+ DRM_MODE_CONNECTOR_eDP);
+ if (ret) {
+ DRM_ERROR("Failed to initialize connector with drm\n");
+ return ret;
+ }
+
+ drm_connector_helper_add(connector,
+ &analogix_dp_connector_helper_funcs);
+ drm_mode_connector_attach_encoder(connector, encoder);
+
+ /*
+ * NOTE: the connector registration is implemented in analogix
+ * platform driver, that to say connector would be exist after
+ * plat_data->attch return, that's why we record the connector
+ * point after plat attached.
+ */
+ if (dp->plat_data->attach) {
+ ret = dp->plat_data->attach(dp->plat_data, bridge, connector);
+ if (ret) {
+ DRM_ERROR("Failed at platform attch func\n");
+ return ret;
+ }
+ }
+
+ if (dp->plat_data->panel) {
+ ret = drm_panel_attach(dp->plat_data->panel, &dp->connector);
+ if (ret) {
+ DRM_ERROR("Failed to attach panel\n");
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+static void analogix_dp_bridge_enable(struct drm_bridge *bridge)
+{
+ struct analogix_dp_device *dp = bridge->driver_private;
+
+ if (dp->dpms_mode == DRM_MODE_DPMS_ON)
+ return;
+
+ pm_runtime_get_sync(dp->dev);
+
+ if (dp->plat_data->power_on)
+ dp->plat_data->power_on(dp->plat_data);
+
+ phy_power_on(dp->phy);
+ analogix_dp_init_dp(dp);
+ enable_irq(dp->irq);
+ analogix_dp_commit(dp);
+
+ dp->dpms_mode = DRM_MODE_DPMS_ON;
+}
+
+static void analogix_dp_bridge_disable(struct drm_bridge *bridge)
+{
+ struct analogix_dp_device *dp = bridge->driver_private;
+
+ if (dp->dpms_mode != DRM_MODE_DPMS_ON)
+ return;
+
+ if (dp->plat_data->panel) {
+ if (drm_panel_disable(dp->plat_data->panel)) {
+ DRM_ERROR("failed to disable the panel\n");
+ return;
+ }
+ }
+
+ disable_irq(dp->irq);
+ phy_power_off(dp->phy);
+
+ if (dp->plat_data->power_off)
+ dp->plat_data->power_off(dp->plat_data);
+
+ pm_runtime_put_sync(dp->dev);
+
+ dp->dpms_mode = DRM_MODE_DPMS_OFF;
+}
+
+static void analogix_dp_bridge_mode_set(struct drm_bridge *bridge,
+ struct drm_display_mode *orig_mode,
+ struct drm_display_mode *mode)
+{
+ struct analogix_dp_device *dp = bridge->driver_private;
+ struct drm_display_info *display_info = &dp->connector.display_info;
+ struct video_info *video = &dp->video_info;
+ struct device_node *dp_node = dp->dev->of_node;
+ int vic;
+
+ /* Input video interlaces & hsync pol & vsync pol */
+ video->interlaced = !!(mode->flags & DRM_MODE_FLAG_INTERLACE);
+ video->v_sync_polarity = !!(mode->flags & DRM_MODE_FLAG_NVSYNC);
+ video->h_sync_polarity = !!(mode->flags & DRM_MODE_FLAG_NHSYNC);
+
+ /* Input video dynamic_range & colorimetry */
+ vic = drm_match_cea_mode(mode);
+ if ((vic == 6) || (vic == 7) || (vic == 21) || (vic == 22) ||
+ (vic == 2) || (vic == 3) || (vic == 17) || (vic == 18)) {
+ video->dynamic_range = CEA;
+ video->ycbcr_coeff = COLOR_YCBCR601;
+ } else if (vic) {
+ video->dynamic_range = CEA;
+ video->ycbcr_coeff = COLOR_YCBCR709;
+ } else {
+ video->dynamic_range = VESA;
+ video->ycbcr_coeff = COLOR_YCBCR709;
+ }
+
+ /* Input vide bpc and color_formats */
+ switch (display_info->bpc) {
+ case 12:
+ video->color_depth = COLOR_12;
+ break;
+ case 10:
+ video->color_depth = COLOR_10;
+ break;
+ case 8:
+ video->color_depth = COLOR_8;
+ break;
+ case 6:
+ video->color_depth = COLOR_6;
+ break;
+ default:
+ video->color_depth = COLOR_8;
+ break;
+ }
+ if (display_info->color_formats & DRM_COLOR_FORMAT_YCRCB444)
+ video->color_space = COLOR_YCBCR444;
+ else if (display_info->color_formats & DRM_COLOR_FORMAT_YCRCB422)
+ video->color_space = COLOR_YCBCR422;
+ else if (display_info->color_formats & DRM_COLOR_FORMAT_RGB444)
+ video->color_space = COLOR_RGB;
+ else
+ video->color_space = COLOR_RGB;
+
+ /*
+ * NOTE: those property parsing code is used for providing backward
+ * compatibility for samsung platform.
+ * Due to we used the "of_property_read_u32" interfaces, when this
+ * property isn't present, the "video_info" can keep the original
+ * values and wouldn't be modified.
+ */
+ of_property_read_u32(dp_node, "samsung,color-space",
+ &video->color_space);
+ of_property_read_u32(dp_node, "samsung,dynamic-range",
+ &video->dynamic_range);
+ of_property_read_u32(dp_node, "samsung,ycbcr-coeff",
+ &video->ycbcr_coeff);
+ of_property_read_u32(dp_node, "samsung,color-depth",
+ &video->color_depth);
+ if (of_property_read_bool(dp_node, "hsync-active-high"))
+ video->h_sync_polarity = true;
+ if (of_property_read_bool(dp_node, "vsync-active-high"))
+ video->v_sync_polarity = true;
+ if (of_property_read_bool(dp_node, "interlaced"))
+ video->interlaced = true;
+}
+
+static void analogix_dp_bridge_nop(struct drm_bridge *bridge)
+{
+ /* do nothing */
+}
+
+static const struct drm_bridge_funcs analogix_dp_bridge_funcs = {
+ .enable = analogix_dp_bridge_enable,
+ .disable = analogix_dp_bridge_disable,
+ .pre_enable = analogix_dp_bridge_nop,
+ .post_disable = analogix_dp_bridge_nop,
+ .mode_set = analogix_dp_bridge_mode_set,
+ .attach = analogix_dp_bridge_attach,
+};
+
+static int analogix_dp_create_bridge(struct drm_device *drm_dev,
+ struct analogix_dp_device *dp)
+{
+ struct drm_bridge *bridge;
+ int ret;
+
+ bridge = devm_kzalloc(drm_dev->dev, sizeof(*bridge), GFP_KERNEL);
+ if (!bridge) {
+ DRM_ERROR("failed to allocate for drm bridge\n");
+ return -ENOMEM;
+ }
+
+ dp->bridge = bridge;
+
+ dp->encoder->bridge = bridge;
+ bridge->driver_private = dp;
+ bridge->encoder = dp->encoder;
+ bridge->funcs = &analogix_dp_bridge_funcs;
+
+ ret = drm_bridge_attach(drm_dev, bridge);
+ if (ret) {
+ DRM_ERROR("failed to attach drm bridge\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int analogix_dp_dt_parse_pdata(struct analogix_dp_device *dp)
+{
+ struct device_node *dp_node = dp->dev->of_node;
+ struct video_info *video_info = &dp->video_info;
+
+ switch (dp->plat_data->dev_type) {
+ case RK3288_DP:
+ /*
+ * Like Rk3288 DisplayPort TRM indicate that "Main link
+ * containing 4 physical lanes of 2.7/1.62 Gbps/lane".
+ */
+ video_info->max_link_rate = 0x0A;
+ video_info->max_lane_count = 0x04;
+ break;
+ case EXYNOS_DP:
+ /*
+ * NOTE: those property parseing code is used for
+ * providing backward compatibility for samsung platform.
+ */
+ of_property_read_u32(dp_node, "samsung,link-rate",
+ &video_info->max_link_rate);
+ of_property_read_u32(dp_node, "samsung,lane-count",
+ &video_info->max_lane_count);
+ break;
+ }
+
+ return 0;
+}
+
+int analogix_dp_bind(struct device *dev, struct drm_device *drm_dev,
+ struct analogix_dp_plat_data *plat_data)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct analogix_dp_device *dp;
+ struct resource *res;
+ unsigned int irq_flags;
+ int ret;
+
+ if (!plat_data) {
+ dev_err(dev, "Invalided input plat_data\n");
+ return -EINVAL;
+ }
+
+ dp = devm_kzalloc(dev, sizeof(struct analogix_dp_device), GFP_KERNEL);
+ if (!dp)
+ return -ENOMEM;
+
+ dev_set_drvdata(dev, dp);
+
+ dp->dev = &pdev->dev;
+ dp->dpms_mode = DRM_MODE_DPMS_OFF;
+
+ /*
+ * platform dp driver need containor_of the plat_data to get
+ * the driver private data, so we need to store the point of
+ * plat_data, not the context of plat_data.
+ */
+ dp->plat_data = plat_data;
+
+ ret = analogix_dp_dt_parse_pdata(dp);
+ if (ret)
+ return ret;
+
+ dp->phy = devm_phy_get(dp->dev, "dp");
+ if (IS_ERR(dp->phy)) {
+ dev_err(dp->dev, "no DP phy configured\n");
+ ret = PTR_ERR(dp->phy);
+ if (ret) {
+ /*
+ * phy itself is not enabled, so we can move forward
+ * assigning NULL to phy pointer.
+ */
+ if (ret == -ENOSYS || ret == -ENODEV)
+ dp->phy = NULL;
+ else
+ return ret;
+ }
+ }
+
+ dp->clock = devm_clk_get(&pdev->dev, "dp");
+ if (IS_ERR(dp->clock)) {
+ dev_err(&pdev->dev, "failed to get clock\n");
+ return PTR_ERR(dp->clock);
+ }
+
+ clk_prepare_enable(dp->clock);
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+
+ dp->reg_base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(dp->reg_base))
+ return PTR_ERR(dp->reg_base);
+
+ dp->force_hpd = of_property_read_bool(dev->of_node, "force-hpd");
+
+ dp->hpd_gpio = of_get_named_gpio(dev->of_node, "hpd-gpios", 0);
+ if (!gpio_is_valid(dp->hpd_gpio))
+ dp->hpd_gpio = of_get_named_gpio(dev->of_node,
+ "samsung,hpd-gpio", 0);
+
+ if (gpio_is_valid(dp->hpd_gpio)) {
+ /*
+ * Set up the hotplug GPIO from the device tree as an interrupt.
+ * Simply specifying a different interrupt in the device tree
+ * doesn't work since we handle hotplug rather differently when
+ * using a GPIO. We also need the actual GPIO specifier so
+ * that we can get the current state of the GPIO.
+ */
+ ret = devm_gpio_request_one(&pdev->dev, dp->hpd_gpio, GPIOF_IN,
+ "hpd_gpio");
+ if (ret) {
+ dev_err(&pdev->dev, "failed to get hpd gpio\n");
+ return ret;
+ }
+ dp->irq = gpio_to_irq(dp->hpd_gpio);
+ irq_flags = IRQF_TRIGGER_RISING | IRQF_TRIGGER_FALLING;
+ } else {
+ dp->hpd_gpio = -ENODEV;
+ dp->irq = platform_get_irq(pdev, 0);
+ irq_flags = 0;
+ }
+
+ if (dp->irq == -ENXIO) {
+ dev_err(&pdev->dev, "failed to get irq\n");
+ return -ENODEV;
+ }
+
+ pm_runtime_enable(dev);
+
+ phy_power_on(dp->phy);
+
+ if (dp->plat_data->panel) {
+ if (drm_panel_prepare(dp->plat_data->panel)) {
+ DRM_ERROR("failed to setup the panel\n");
+ return -EBUSY;
+ }
+ }
+
+ analogix_dp_init_dp(dp);
+
+ ret = devm_request_threaded_irq(&pdev->dev, dp->irq,
+ analogix_dp_hardirq,
+ analogix_dp_irq_thread,
+ irq_flags, "analogix-dp", dp);
+ if (ret) {
+ dev_err(&pdev->dev, "failed to request irq\n");
+ goto err_disable_pm_runtime;
+ }
+ disable_irq(dp->irq);
+
+ dp->drm_dev = drm_dev;
+ dp->encoder = dp->plat_data->encoder;
+
+ ret = analogix_dp_create_bridge(drm_dev, dp);
+ if (ret) {
+ DRM_ERROR("failed to create bridge (%d)\n", ret);
+ drm_encoder_cleanup(dp->encoder);
+ goto err_disable_pm_runtime;
+ }
+
+ return 0;
+
+err_disable_pm_runtime:
+ pm_runtime_disable(dev);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(analogix_dp_bind);
+
+void analogix_dp_unbind(struct device *dev, struct device *master,
+ void *data)
+{
+ struct analogix_dp_device *dp = dev_get_drvdata(dev);
+
+ analogix_dp_bridge_disable(dp->bridge);
+
+ if (dp->plat_data->panel) {
+ if (drm_panel_unprepare(dp->plat_data->panel))
+ DRM_ERROR("failed to turnoff the panel\n");
+ }
+
+ pm_runtime_disable(dev);
+}
+EXPORT_SYMBOL_GPL(analogix_dp_unbind);
+
+#ifdef CONFIG_PM
+int analogix_dp_suspend(struct device *dev)
+{
+ struct analogix_dp_device *dp = dev_get_drvdata(dev);
+
+ clk_disable_unprepare(dp->clock);
+
+ if (dp->plat_data->panel) {
+ if (drm_panel_unprepare(dp->plat_data->panel))
+ DRM_ERROR("failed to turnoff the panel\n");
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(analogix_dp_suspend);
+
+int analogix_dp_resume(struct device *dev)
+{
+ struct analogix_dp_device *dp = dev_get_drvdata(dev);
+ int ret;
+
+ ret = clk_prepare_enable(dp->clock);
+ if (ret < 0) {
+ DRM_ERROR("Failed to prepare_enable the clock clk [%d]\n", ret);
+ return ret;
+ }
+
+ if (dp->plat_data->panel) {
+ if (drm_panel_prepare(dp->plat_data->panel)) {
+ DRM_ERROR("failed to setup the panel\n");
+ return -EBUSY;
+ }
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(analogix_dp_resume);
+#endif
+
+MODULE_AUTHOR("Jingoo Han <jg1.han@samsung.com>");
+MODULE_DESCRIPTION("Analogix DP Core Driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/gpu/drm/bridge/analogix/analogix_dp_core.h b/drivers/gpu/drm/bridge/analogix/analogix_dp_core.h
new file mode 100644
index 0000000..f09275d
--- /dev/null
+++ b/drivers/gpu/drm/bridge/analogix/analogix_dp_core.h
@@ -0,0 +1,281 @@
+/*
+ * Header file for Analogix DP (Display Port) core interface driver.
+ *
+ * Copyright (C) 2012 Samsung Electronics Co., Ltd.
+ * Author: Jingoo Han <jg1.han@samsung.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#ifndef _ANALOGIX_DP_CORE_H
+#define _ANALOGIX_DP_CORE_H
+
+#include <drm/drm_crtc.h>
+#include <drm/drm_dp_helper.h>
+
+#define DP_TIMEOUT_LOOP_COUNT 100
+#define MAX_CR_LOOP 5
+#define MAX_EQ_LOOP 5
+
+/* I2C EDID Chip ID, Slave Address */
+#define I2C_EDID_DEVICE_ADDR 0x50
+#define I2C_E_EDID_DEVICE_ADDR 0x30
+
+#define EDID_BLOCK_LENGTH 0x80
+#define EDID_HEADER_PATTERN 0x00
+#define EDID_EXTENSION_FLAG 0x7e
+#define EDID_CHECKSUM 0x7f
+
+/* DP_MAX_LANE_COUNT */
+#define DPCD_ENHANCED_FRAME_CAP(x) (((x) >> 7) & 0x1)
+#define DPCD_MAX_LANE_COUNT(x) ((x) & 0x1f)
+
+/* DP_LANE_COUNT_SET */
+#define DPCD_LANE_COUNT_SET(x) ((x) & 0x1f)
+
+/* DP_TRAINING_LANE0_SET */
+#define DPCD_PRE_EMPHASIS_SET(x) (((x) & 0x3) << 3)
+#define DPCD_PRE_EMPHASIS_GET(x) (((x) >> 3) & 0x3)
+#define DPCD_VOLTAGE_SWING_SET(x) (((x) & 0x3) << 0)
+#define DPCD_VOLTAGE_SWING_GET(x) (((x) >> 0) & 0x3)
+
+enum link_lane_count_type {
+ LANE_COUNT1 = 1,
+ LANE_COUNT2 = 2,
+ LANE_COUNT4 = 4
+};
+
+enum link_training_state {
+ START,
+ CLOCK_RECOVERY,
+ EQUALIZER_TRAINING,
+ FINISHED,
+ FAILED
+};
+
+enum voltage_swing_level {
+ VOLTAGE_LEVEL_0,
+ VOLTAGE_LEVEL_1,
+ VOLTAGE_LEVEL_2,
+ VOLTAGE_LEVEL_3,
+};
+
+enum pre_emphasis_level {
+ PRE_EMPHASIS_LEVEL_0,
+ PRE_EMPHASIS_LEVEL_1,
+ PRE_EMPHASIS_LEVEL_2,
+ PRE_EMPHASIS_LEVEL_3,
+};
+
+enum pattern_set {
+ PRBS7,
+ D10_2,
+ TRAINING_PTN1,
+ TRAINING_PTN2,
+ DP_NONE
+};
+
+enum color_space {
+ COLOR_RGB,
+ COLOR_YCBCR422,
+ COLOR_YCBCR444
+};
+
+enum color_depth {
+ COLOR_6,
+ COLOR_8,
+ COLOR_10,
+ COLOR_12
+};
+
+enum color_coefficient {
+ COLOR_YCBCR601,
+ COLOR_YCBCR709
+};
+
+enum dynamic_range {
+ VESA,
+ CEA
+};
+
+enum pll_status {
+ PLL_UNLOCKED,
+ PLL_LOCKED
+};
+
+enum clock_recovery_m_value_type {
+ CALCULATED_M,
+ REGISTER_M
+};
+
+enum video_timing_recognition_type {
+ VIDEO_TIMING_FROM_CAPTURE,
+ VIDEO_TIMING_FROM_REGISTER
+};
+
+enum analog_power_block {
+ AUX_BLOCK,
+ CH0_BLOCK,
+ CH1_BLOCK,
+ CH2_BLOCK,
+ CH3_BLOCK,
+ ANALOG_TOTAL,
+ POWER_ALL
+};
+
+enum dp_irq_type {
+ DP_IRQ_TYPE_HP_CABLE_IN,
+ DP_IRQ_TYPE_HP_CABLE_OUT,
+ DP_IRQ_TYPE_HP_CHANGE,
+ DP_IRQ_TYPE_UNKNOWN,
+};
+
+struct video_info {
+ char *name;
+
+ bool h_sync_polarity;
+ bool v_sync_polarity;
+ bool interlaced;
+
+ enum color_space color_space;
+ enum dynamic_range dynamic_range;
+ enum color_coefficient ycbcr_coeff;
+ enum color_depth color_depth;
+
+ int max_link_rate;
+ enum link_lane_count_type max_lane_count;
+};
+
+struct link_train {
+ int eq_loop;
+ int cr_loop[4];
+
+ u8 link_rate;
+ u8 lane_count;
+ u8 training_lane[4];
+
+ enum link_training_state lt_state;
+};
+
+struct analogix_dp_device {
+ struct drm_encoder *encoder;
+ struct device *dev;
+ struct drm_device *drm_dev;
+ struct drm_connector connector;
+ struct drm_bridge *bridge;
+ struct clk *clock;
+ unsigned int irq;
+ void __iomem *reg_base;
+
+ struct video_info video_info;
+ struct link_train link_train;
+ struct phy *phy;
+ int dpms_mode;
+ int hpd_gpio;
+ bool force_hpd;
+ unsigned char edid[EDID_BLOCK_LENGTH * 2];
+
+ struct analogix_dp_plat_data *plat_data;
+};
+
+/* analogix_dp_reg.c */
+void analogix_dp_enable_video_mute(struct analogix_dp_device *dp, bool enable);
+void analogix_dp_stop_video(struct analogix_dp_device *dp);
+void analogix_dp_lane_swap(struct analogix_dp_device *dp, bool enable);
+void analogix_dp_init_analog_param(struct analogix_dp_device *dp);
+void analogix_dp_init_interrupt(struct analogix_dp_device *dp);
+void analogix_dp_reset(struct analogix_dp_device *dp);
+void analogix_dp_swreset(struct analogix_dp_device *dp);
+void analogix_dp_config_interrupt(struct analogix_dp_device *dp);
+void analogix_dp_mute_hpd_interrupt(struct analogix_dp_device *dp);
+void analogix_dp_unmute_hpd_interrupt(struct analogix_dp_device *dp);
+enum pll_status analogix_dp_get_pll_lock_status(struct analogix_dp_device *dp);
+void analogix_dp_set_pll_power_down(struct analogix_dp_device *dp, bool enable);
+void analogix_dp_set_analog_power_down(struct analogix_dp_device *dp,
+ enum analog_power_block block,
+ bool enable);
+void analogix_dp_init_analog_func(struct analogix_dp_device *dp);
+void analogix_dp_init_hpd(struct analogix_dp_device *dp);
+void analogix_dp_force_hpd(struct analogix_dp_device *dp);
+enum dp_irq_type analogix_dp_get_irq_type(struct analogix_dp_device *dp);
+void analogix_dp_clear_hotplug_interrupts(struct analogix_dp_device *dp);
+void analogix_dp_reset_aux(struct analogix_dp_device *dp);
+void analogix_dp_init_aux(struct analogix_dp_device *dp);
+int analogix_dp_get_plug_in_status(struct analogix_dp_device *dp);
+void analogix_dp_enable_sw_function(struct analogix_dp_device *dp);
+int analogix_dp_start_aux_transaction(struct analogix_dp_device *dp);
+int analogix_dp_write_byte_to_dpcd(struct analogix_dp_device *dp,
+ unsigned int reg_addr,
+ unsigned char data);
+int analogix_dp_read_byte_from_dpcd(struct analogix_dp_device *dp,
+ unsigned int reg_addr,
+ unsigned char *data);
+int analogix_dp_write_bytes_to_dpcd(struct analogix_dp_device *dp,
+ unsigned int reg_addr,
+ unsigned int count,
+ unsigned char data[]);
+int analogix_dp_read_bytes_from_dpcd(struct analogix_dp_device *dp,
+ unsigned int reg_addr,
+ unsigned int count,
+ unsigned char data[]);
+int analogix_dp_select_i2c_device(struct analogix_dp_device *dp,
+ unsigned int device_addr,
+ unsigned int reg_addr);
+int analogix_dp_read_byte_from_i2c(struct analogix_dp_device *dp,
+ unsigned int device_addr,
+ unsigned int reg_addr,
+ unsigned int *data);
+int analogix_dp_read_bytes_from_i2c(struct analogix_dp_device *dp,
+ unsigned int device_addr,
+ unsigned int reg_addr,
+ unsigned int count,
+ unsigned char edid[]);
+void analogix_dp_set_link_bandwidth(struct analogix_dp_device *dp, u32 bwtype);
+void analogix_dp_get_link_bandwidth(struct analogix_dp_device *dp, u32 *bwtype);
+void analogix_dp_set_lane_count(struct analogix_dp_device *dp, u32 count);
+void analogix_dp_get_lane_count(struct analogix_dp_device *dp, u32 *count);
+void analogix_dp_enable_enhanced_mode(struct analogix_dp_device *dp,
+ bool enable);
+void analogix_dp_set_training_pattern(struct analogix_dp_device *dp,
+ enum pattern_set pattern);
+void analogix_dp_set_lane0_pre_emphasis(struct analogix_dp_device *dp,
+ u32 level);
+void analogix_dp_set_lane1_pre_emphasis(struct analogix_dp_device *dp,
+ u32 level);
+void analogix_dp_set_lane2_pre_emphasis(struct analogix_dp_device *dp,
+ u32 level);
+void analogix_dp_set_lane3_pre_emphasis(struct analogix_dp_device *dp,
+ u32 level);
+void analogix_dp_set_lane0_link_training(struct analogix_dp_device *dp,
+ u32 training_lane);
+void analogix_dp_set_lane1_link_training(struct analogix_dp_device *dp,
+ u32 training_lane);
+void analogix_dp_set_lane2_link_training(struct analogix_dp_device *dp,
+ u32 training_lane);
+void analogix_dp_set_lane3_link_training(struct analogix_dp_device *dp,
+ u32 training_lane);
+u32 analogix_dp_get_lane0_link_training(struct analogix_dp_device *dp);
+u32 analogix_dp_get_lane1_link_training(struct analogix_dp_device *dp);
+u32 analogix_dp_get_lane2_link_training(struct analogix_dp_device *dp);
+u32 analogix_dp_get_lane3_link_training(struct analogix_dp_device *dp);
+void analogix_dp_reset_macro(struct analogix_dp_device *dp);
+void analogix_dp_init_video(struct analogix_dp_device *dp);
+
+void analogix_dp_set_video_color_format(struct analogix_dp_device *dp);
+int analogix_dp_is_slave_video_stream_clock_on(struct analogix_dp_device *dp);
+void analogix_dp_set_video_cr_mn(struct analogix_dp_device *dp,
+ enum clock_recovery_m_value_type type,
+ u32 m_value,
+ u32 n_value);
+void analogix_dp_set_video_timing_mode(struct analogix_dp_device *dp, u32 type);
+void analogix_dp_enable_video_master(struct analogix_dp_device *dp,
+ bool enable);
+void analogix_dp_start_video(struct analogix_dp_device *dp);
+int analogix_dp_is_video_stream_on(struct analogix_dp_device *dp);
+void analogix_dp_config_video_slave_mode(struct analogix_dp_device *dp);
+void analogix_dp_enable_scrambling(struct analogix_dp_device *dp);
+void analogix_dp_disable_scrambling(struct analogix_dp_device *dp);
+#endif /* _ANALOGIX_DP_CORE_H */
diff --git a/drivers/gpu/drm/bridge/analogix/analogix_dp_reg.c b/drivers/gpu/drm/bridge/analogix/analogix_dp_reg.c
new file mode 100644
index 0000000..49205ef
--- /dev/null
+++ b/drivers/gpu/drm/bridge/analogix/analogix_dp_reg.c
@@ -0,0 +1,1320 @@
+/*
+ * Analogix DP (Display port) core register interface driver.
+ *
+ * Copyright (C) 2012 Samsung Electronics Co., Ltd.
+ * Author: Jingoo Han <jg1.han@samsung.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#include <linux/device.h>
+#include <linux/io.h>
+#include <linux/delay.h>
+#include <linux/gpio.h>
+
+#include <drm/bridge/analogix_dp.h>
+
+#include "analogix_dp_core.h"
+#include "analogix_dp_reg.h"
+
+#define COMMON_INT_MASK_1 0
+#define COMMON_INT_MASK_2 0
+#define COMMON_INT_MASK_3 0
+#define COMMON_INT_MASK_4 (HOTPLUG_CHG | HPD_LOST | PLUG)
+#define INT_STA_MASK INT_HPD
+
+void analogix_dp_enable_video_mute(struct analogix_dp_device *dp, bool enable)
+{
+ u32 reg;
+
+ if (enable) {
+ reg = readl(dp->reg_base + ANALOGIX_DP_VIDEO_CTL_1);
+ reg |= HDCP_VIDEO_MUTE;
+ writel(reg, dp->reg_base + ANALOGIX_DP_VIDEO_CTL_1);
+ } else {
+ reg = readl(dp->reg_base + ANALOGIX_DP_VIDEO_CTL_1);
+ reg &= ~HDCP_VIDEO_MUTE;
+ writel(reg, dp->reg_base + ANALOGIX_DP_VIDEO_CTL_1);
+ }
+}
+
+void analogix_dp_stop_video(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_VIDEO_CTL_1);
+ reg &= ~VIDEO_EN;
+ writel(reg, dp->reg_base + ANALOGIX_DP_VIDEO_CTL_1);
+}
+
+void analogix_dp_lane_swap(struct analogix_dp_device *dp, bool enable)
+{
+ u32 reg;
+
+ if (enable)
+ reg = LANE3_MAP_LOGIC_LANE_0 | LANE2_MAP_LOGIC_LANE_1 |
+ LANE1_MAP_LOGIC_LANE_2 | LANE0_MAP_LOGIC_LANE_3;
+ else
+ reg = LANE3_MAP_LOGIC_LANE_3 | LANE2_MAP_LOGIC_LANE_2 |
+ LANE1_MAP_LOGIC_LANE_1 | LANE0_MAP_LOGIC_LANE_0;
+
+ writel(reg, dp->reg_base + ANALOGIX_DP_LANE_MAP);
+}
+
+void analogix_dp_init_analog_param(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ reg = TX_TERMINAL_CTRL_50_OHM;
+ writel(reg, dp->reg_base + ANALOGIX_DP_ANALOG_CTL_1);
+
+ reg = SEL_24M | TX_DVDD_BIT_1_0625V;
+ writel(reg, dp->reg_base + ANALOGIX_DP_ANALOG_CTL_2);
+
+ if (dp->plat_data && (dp->plat_data->dev_type == RK3288_DP)) {
+ writel(REF_CLK_24M, dp->reg_base + ANALOGIX_DP_PLL_REG_1);
+ writel(0x95, dp->reg_base + ANALOGIX_DP_PLL_REG_2);
+ writel(0x40, dp->reg_base + ANALOGIX_DP_PLL_REG_3);
+ writel(0x58, dp->reg_base + ANALOGIX_DP_PLL_REG_4);
+ writel(0x22, dp->reg_base + ANALOGIX_DP_PLL_REG_5);
+ }
+
+ reg = DRIVE_DVDD_BIT_1_0625V | VCO_BIT_600_MICRO;
+ writel(reg, dp->reg_base + ANALOGIX_DP_ANALOG_CTL_3);
+
+ reg = PD_RING_OSC | AUX_TERMINAL_CTRL_50_OHM |
+ TX_CUR1_2X | TX_CUR_16_MA;
+ writel(reg, dp->reg_base + ANALOGIX_DP_PLL_FILTER_CTL_1);
+
+ reg = CH3_AMP_400_MV | CH2_AMP_400_MV |
+ CH1_AMP_400_MV | CH0_AMP_400_MV;
+ writel(reg, dp->reg_base + ANALOGIX_DP_TX_AMP_TUNING_CTL);
+}
+
+void analogix_dp_init_interrupt(struct analogix_dp_device *dp)
+{
+ /* Set interrupt pin assertion polarity as high */
+ writel(INT_POL1 | INT_POL0, dp->reg_base + ANALOGIX_DP_INT_CTL);
+
+ /* Clear pending regisers */
+ writel(0xff, dp->reg_base + ANALOGIX_DP_COMMON_INT_STA_1);
+ writel(0x4f, dp->reg_base + ANALOGIX_DP_COMMON_INT_STA_2);
+ writel(0xe0, dp->reg_base + ANALOGIX_DP_COMMON_INT_STA_3);
+ writel(0xe7, dp->reg_base + ANALOGIX_DP_COMMON_INT_STA_4);
+ writel(0x63, dp->reg_base + ANALOGIX_DP_INT_STA);
+
+ /* 0:mask,1: unmask */
+ writel(0x00, dp->reg_base + ANALOGIX_DP_COMMON_INT_MASK_1);
+ writel(0x00, dp->reg_base + ANALOGIX_DP_COMMON_INT_MASK_2);
+ writel(0x00, dp->reg_base + ANALOGIX_DP_COMMON_INT_MASK_3);
+ writel(0x00, dp->reg_base + ANALOGIX_DP_COMMON_INT_MASK_4);
+ writel(0x00, dp->reg_base + ANALOGIX_DP_INT_STA_MASK);
+}
+
+void analogix_dp_reset(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ analogix_dp_stop_video(dp);
+ analogix_dp_enable_video_mute(dp, 0);
+
+ reg = MASTER_VID_FUNC_EN_N | SLAVE_VID_FUNC_EN_N |
+ AUD_FIFO_FUNC_EN_N | AUD_FUNC_EN_N |
+ HDCP_FUNC_EN_N | SW_FUNC_EN_N;
+ writel(reg, dp->reg_base + ANALOGIX_DP_FUNC_EN_1);
+
+ reg = SSC_FUNC_EN_N | AUX_FUNC_EN_N |
+ SERDES_FIFO_FUNC_EN_N |
+ LS_CLK_DOMAIN_FUNC_EN_N;
+ writel(reg, dp->reg_base + ANALOGIX_DP_FUNC_EN_2);
+
+ usleep_range(20, 30);
+
+ analogix_dp_lane_swap(dp, 0);
+
+ writel(0x0, dp->reg_base + ANALOGIX_DP_SYS_CTL_1);
+ writel(0x40, dp->reg_base + ANALOGIX_DP_SYS_CTL_2);
+ writel(0x0, dp->reg_base + ANALOGIX_DP_SYS_CTL_3);
+ writel(0x0, dp->reg_base + ANALOGIX_DP_SYS_CTL_4);
+
+ writel(0x0, dp->reg_base + ANALOGIX_DP_PKT_SEND_CTL);
+ writel(0x0, dp->reg_base + ANALOGIX_DP_HDCP_CTL);
+
+ writel(0x5e, dp->reg_base + ANALOGIX_DP_HPD_DEGLITCH_L);
+ writel(0x1a, dp->reg_base + ANALOGIX_DP_HPD_DEGLITCH_H);
+
+ writel(0x10, dp->reg_base + ANALOGIX_DP_LINK_DEBUG_CTL);
+
+ writel(0x0, dp->reg_base + ANALOGIX_DP_PHY_TEST);
+
+ writel(0x0, dp->reg_base + ANALOGIX_DP_VIDEO_FIFO_THRD);
+ writel(0x20, dp->reg_base + ANALOGIX_DP_AUDIO_MARGIN);
+
+ writel(0x4, dp->reg_base + ANALOGIX_DP_M_VID_GEN_FILTER_TH);
+ writel(0x2, dp->reg_base + ANALOGIX_DP_M_AUD_GEN_FILTER_TH);
+
+ writel(0x00000101, dp->reg_base + ANALOGIX_DP_SOC_GENERAL_CTL);
+}
+
+void analogix_dp_swreset(struct analogix_dp_device *dp)
+{
+ writel(RESET_DP_TX, dp->reg_base + ANALOGIX_DP_TX_SW_RESET);
+}
+
+void analogix_dp_config_interrupt(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ /* 0: mask, 1: unmask */
+ reg = COMMON_INT_MASK_1;
+ writel(reg, dp->reg_base + ANALOGIX_DP_COMMON_INT_MASK_1);
+
+ reg = COMMON_INT_MASK_2;
+ writel(reg, dp->reg_base + ANALOGIX_DP_COMMON_INT_MASK_2);
+
+ reg = COMMON_INT_MASK_3;
+ writel(reg, dp->reg_base + ANALOGIX_DP_COMMON_INT_MASK_3);
+
+ reg = COMMON_INT_MASK_4;
+ writel(reg, dp->reg_base + ANALOGIX_DP_COMMON_INT_MASK_4);
+
+ reg = INT_STA_MASK;
+ writel(reg, dp->reg_base + ANALOGIX_DP_INT_STA_MASK);
+}
+
+void analogix_dp_mute_hpd_interrupt(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ /* 0: mask, 1: unmask */
+ reg = readl(dp->reg_base + ANALOGIX_DP_COMMON_INT_MASK_4);
+ reg &= ~COMMON_INT_MASK_4;
+ writel(reg, dp->reg_base + ANALOGIX_DP_COMMON_INT_MASK_4);
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_INT_STA_MASK);
+ reg &= ~INT_STA_MASK;
+ writel(reg, dp->reg_base + ANALOGIX_DP_INT_STA_MASK);
+}
+
+void analogix_dp_unmute_hpd_interrupt(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ /* 0: mask, 1: unmask */
+ reg = COMMON_INT_MASK_4;
+ writel(reg, dp->reg_base + ANALOGIX_DP_COMMON_INT_MASK_4);
+
+ reg = INT_STA_MASK;
+ writel(reg, dp->reg_base + ANALOGIX_DP_INT_STA_MASK);
+}
+
+enum pll_status analogix_dp_get_pll_lock_status(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_DEBUG_CTL);
+ if (reg & PLL_LOCK)
+ return PLL_LOCKED;
+ else
+ return PLL_UNLOCKED;
+}
+
+void analogix_dp_set_pll_power_down(struct analogix_dp_device *dp, bool enable)
+{
+ u32 reg;
+
+ if (enable) {
+ reg = readl(dp->reg_base + ANALOGIX_DP_PLL_CTL);
+ reg |= DP_PLL_PD;
+ writel(reg, dp->reg_base + ANALOGIX_DP_PLL_CTL);
+ } else {
+ reg = readl(dp->reg_base + ANALOGIX_DP_PLL_CTL);
+ reg &= ~DP_PLL_PD;
+ writel(reg, dp->reg_base + ANALOGIX_DP_PLL_CTL);
+ }
+}
+
+void analogix_dp_set_analog_power_down(struct analogix_dp_device *dp,
+ enum analog_power_block block,
+ bool enable)
+{
+ u32 reg;
+ u32 phy_pd_addr = ANALOGIX_DP_PHY_PD;
+
+ if (dp->plat_data && (dp->plat_data->dev_type == RK3288_DP))
+ phy_pd_addr = ANALOGIX_DP_PD;
+
+ switch (block) {
+ case AUX_BLOCK:
+ if (enable) {
+ reg = readl(dp->reg_base + phy_pd_addr);
+ reg |= AUX_PD;
+ writel(reg, dp->reg_base + phy_pd_addr);
+ } else {
+ reg = readl(dp->reg_base + phy_pd_addr);
+ reg &= ~AUX_PD;
+ writel(reg, dp->reg_base + phy_pd_addr);
+ }
+ break;
+ case CH0_BLOCK:
+ if (enable) {
+ reg = readl(dp->reg_base + phy_pd_addr);
+ reg |= CH0_PD;
+ writel(reg, dp->reg_base + phy_pd_addr);
+ } else {
+ reg = readl(dp->reg_base + phy_pd_addr);
+ reg &= ~CH0_PD;
+ writel(reg, dp->reg_base + phy_pd_addr);
+ }
+ break;
+ case CH1_BLOCK:
+ if (enable) {
+ reg = readl(dp->reg_base + phy_pd_addr);
+ reg |= CH1_PD;
+ writel(reg, dp->reg_base + phy_pd_addr);
+ } else {
+ reg = readl(dp->reg_base + phy_pd_addr);
+ reg &= ~CH1_PD;
+ writel(reg, dp->reg_base + phy_pd_addr);
+ }
+ break;
+ case CH2_BLOCK:
+ if (enable) {
+ reg = readl(dp->reg_base + phy_pd_addr);
+ reg |= CH2_PD;
+ writel(reg, dp->reg_base + phy_pd_addr);
+ } else {
+ reg = readl(dp->reg_base + phy_pd_addr);
+ reg &= ~CH2_PD;
+ writel(reg, dp->reg_base + phy_pd_addr);
+ }
+ break;
+ case CH3_BLOCK:
+ if (enable) {
+ reg = readl(dp->reg_base + phy_pd_addr);
+ reg |= CH3_PD;
+ writel(reg, dp->reg_base + phy_pd_addr);
+ } else {
+ reg = readl(dp->reg_base + phy_pd_addr);
+ reg &= ~CH3_PD;
+ writel(reg, dp->reg_base + phy_pd_addr);
+ }
+ break;
+ case ANALOG_TOTAL:
+ if (enable) {
+ reg = readl(dp->reg_base + phy_pd_addr);
+ reg |= DP_PHY_PD;
+ writel(reg, dp->reg_base + phy_pd_addr);
+ } else {
+ reg = readl(dp->reg_base + phy_pd_addr);
+ reg &= ~DP_PHY_PD;
+ writel(reg, dp->reg_base + phy_pd_addr);
+ }
+ break;
+ case POWER_ALL:
+ if (enable) {
+ reg = DP_PHY_PD | AUX_PD | CH3_PD | CH2_PD |
+ CH1_PD | CH0_PD;
+ writel(reg, dp->reg_base + phy_pd_addr);
+ } else {
+ writel(0x00, dp->reg_base + phy_pd_addr);
+ }
+ break;
+ default:
+ break;
+ }
+}
+
+void analogix_dp_init_analog_func(struct analogix_dp_device *dp)
+{
+ u32 reg;
+ int timeout_loop = 0;
+
+ analogix_dp_set_analog_power_down(dp, POWER_ALL, 0);
+
+ reg = PLL_LOCK_CHG;
+ writel(reg, dp->reg_base + ANALOGIX_DP_COMMON_INT_STA_1);
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_DEBUG_CTL);
+ reg &= ~(F_PLL_LOCK | PLL_LOCK_CTRL);
+ writel(reg, dp->reg_base + ANALOGIX_DP_DEBUG_CTL);
+
+ /* Power up PLL */
+ if (analogix_dp_get_pll_lock_status(dp) == PLL_UNLOCKED) {
+ analogix_dp_set_pll_power_down(dp, 0);
+
+ while (analogix_dp_get_pll_lock_status(dp) == PLL_UNLOCKED) {
+ timeout_loop++;
+ if (DP_TIMEOUT_LOOP_COUNT < timeout_loop) {
+ dev_err(dp->dev, "failed to get pll lock status\n");
+ return;
+ }
+ usleep_range(10, 20);
+ }
+ }
+
+ /* Enable Serdes FIFO function and Link symbol clock domain module */
+ reg = readl(dp->reg_base + ANALOGIX_DP_FUNC_EN_2);
+ reg &= ~(SERDES_FIFO_FUNC_EN_N | LS_CLK_DOMAIN_FUNC_EN_N
+ | AUX_FUNC_EN_N);
+ writel(reg, dp->reg_base + ANALOGIX_DP_FUNC_EN_2);
+}
+
+void analogix_dp_clear_hotplug_interrupts(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ if (gpio_is_valid(dp->hpd_gpio))
+ return;
+
+ reg = HOTPLUG_CHG | HPD_LOST | PLUG;
+ writel(reg, dp->reg_base + ANALOGIX_DP_COMMON_INT_STA_4);
+
+ reg = INT_HPD;
+ writel(reg, dp->reg_base + ANALOGIX_DP_INT_STA);
+}
+
+void analogix_dp_init_hpd(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ if (gpio_is_valid(dp->hpd_gpio))
+ return;
+
+ analogix_dp_clear_hotplug_interrupts(dp);
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_SYS_CTL_3);
+ reg &= ~(F_HPD | HPD_CTRL);
+ writel(reg, dp->reg_base + ANALOGIX_DP_SYS_CTL_3);
+}
+
+void analogix_dp_force_hpd(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_SYS_CTL_3);
+ reg = (F_HPD | HPD_CTRL);
+ writel(reg, dp->reg_base + ANALOGIX_DP_SYS_CTL_3);
+}
+
+enum dp_irq_type analogix_dp_get_irq_type(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ if (gpio_is_valid(dp->hpd_gpio)) {
+ reg = gpio_get_value(dp->hpd_gpio);
+ if (reg)
+ return DP_IRQ_TYPE_HP_CABLE_IN;
+ else
+ return DP_IRQ_TYPE_HP_CABLE_OUT;
+ } else {
+ /* Parse hotplug interrupt status register */
+ reg = readl(dp->reg_base + ANALOGIX_DP_COMMON_INT_STA_4);
+
+ if (reg & PLUG)
+ return DP_IRQ_TYPE_HP_CABLE_IN;
+
+ if (reg & HPD_LOST)
+ return DP_IRQ_TYPE_HP_CABLE_OUT;
+
+ if (reg & HOTPLUG_CHG)
+ return DP_IRQ_TYPE_HP_CHANGE;
+
+ return DP_IRQ_TYPE_UNKNOWN;
+ }
+}
+
+void analogix_dp_reset_aux(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ /* Disable AUX channel module */
+ reg = readl(dp->reg_base + ANALOGIX_DP_FUNC_EN_2);
+ reg |= AUX_FUNC_EN_N;
+ writel(reg, dp->reg_base + ANALOGIX_DP_FUNC_EN_2);
+}
+
+void analogix_dp_init_aux(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ /* Clear inerrupts related to AUX channel */
+ reg = RPLY_RECEIV | AUX_ERR;
+ writel(reg, dp->reg_base + ANALOGIX_DP_INT_STA);
+
+ analogix_dp_reset_aux(dp);
+
+ /* Disable AUX transaction H/W retry */
+ if (dp->plat_data && (dp->plat_data->dev_type == RK3288_DP))
+ reg = AUX_BIT_PERIOD_EXPECTED_DELAY(0) |
+ AUX_HW_RETRY_COUNT_SEL(3) |
+ AUX_HW_RETRY_INTERVAL_600_MICROSECONDS;
+ else
+ reg = AUX_BIT_PERIOD_EXPECTED_DELAY(3) |
+ AUX_HW_RETRY_COUNT_SEL(0) |
+ AUX_HW_RETRY_INTERVAL_600_MICROSECONDS;
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_HW_RETRY_CTL);
+
+ /* Receive AUX Channel DEFER commands equal to DEFFER_COUNT*64 */
+ reg = DEFER_CTRL_EN | DEFER_COUNT(1);
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_CH_DEFER_CTL);
+
+ /* Enable AUX channel module */
+ reg = readl(dp->reg_base + ANALOGIX_DP_FUNC_EN_2);
+ reg &= ~AUX_FUNC_EN_N;
+ writel(reg, dp->reg_base + ANALOGIX_DP_FUNC_EN_2);
+}
+
+int analogix_dp_get_plug_in_status(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ if (gpio_is_valid(dp->hpd_gpio)) {
+ if (gpio_get_value(dp->hpd_gpio))
+ return 0;
+ } else {
+ reg = readl(dp->reg_base + ANALOGIX_DP_SYS_CTL_3);
+ if (reg & HPD_STATUS)
+ return 0;
+ }
+
+ return -EINVAL;
+}
+
+void analogix_dp_enable_sw_function(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_FUNC_EN_1);
+ reg &= ~SW_FUNC_EN_N;
+ writel(reg, dp->reg_base + ANALOGIX_DP_FUNC_EN_1);
+}
+
+int analogix_dp_start_aux_transaction(struct analogix_dp_device *dp)
+{
+ int reg;
+ int retval = 0;
+ int timeout_loop = 0;
+
+ /* Enable AUX CH operation */
+ reg = readl(dp->reg_base + ANALOGIX_DP_AUX_CH_CTL_2);
+ reg |= AUX_EN;
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_CH_CTL_2);
+
+ /* Is AUX CH command reply received? */
+ reg = readl(dp->reg_base + ANALOGIX_DP_INT_STA);
+ while (!(reg & RPLY_RECEIV)) {
+ timeout_loop++;
+ if (DP_TIMEOUT_LOOP_COUNT < timeout_loop) {
+ dev_err(dp->dev, "AUX CH command reply failed!\n");
+ return -ETIMEDOUT;
+ }
+ reg = readl(dp->reg_base + ANALOGIX_DP_INT_STA);
+ usleep_range(10, 11);
+ }
+
+ /* Clear interrupt source for AUX CH command reply */
+ writel(RPLY_RECEIV, dp->reg_base + ANALOGIX_DP_INT_STA);
+
+ /* Clear interrupt source for AUX CH access error */
+ reg = readl(dp->reg_base + ANALOGIX_DP_INT_STA);
+ if (reg & AUX_ERR) {
+ writel(AUX_ERR, dp->reg_base + ANALOGIX_DP_INT_STA);
+ return -EREMOTEIO;
+ }
+
+ /* Check AUX CH error access status */
+ reg = readl(dp->reg_base + ANALOGIX_DP_AUX_CH_STA);
+ if ((reg & AUX_STATUS_MASK) != 0) {
+ dev_err(dp->dev, "AUX CH error happens: %d\n\n",
+ reg & AUX_STATUS_MASK);
+ return -EREMOTEIO;
+ }
+
+ return retval;
+}
+
+int analogix_dp_write_byte_to_dpcd(struct analogix_dp_device *dp,
+ unsigned int reg_addr,
+ unsigned char data)
+{
+ u32 reg;
+ int i;
+ int retval;
+
+ for (i = 0; i < 3; i++) {
+ /* Clear AUX CH data buffer */
+ reg = BUF_CLR;
+ writel(reg, dp->reg_base + ANALOGIX_DP_BUFFER_DATA_CTL);
+
+ /* Select DPCD device address */
+ reg = AUX_ADDR_7_0(reg_addr);
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_ADDR_7_0);
+ reg = AUX_ADDR_15_8(reg_addr);
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_ADDR_15_8);
+ reg = AUX_ADDR_19_16(reg_addr);
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_ADDR_19_16);
+
+ /* Write data buffer */
+ reg = (unsigned int)data;
+ writel(reg, dp->reg_base + ANALOGIX_DP_BUF_DATA_0);
+
+ /*
+ * Set DisplayPort transaction and write 1 byte
+ * If bit 3 is 1, DisplayPort transaction.
+ * If Bit 3 is 0, I2C transaction.
+ */
+ reg = AUX_TX_COMM_DP_TRANSACTION | AUX_TX_COMM_WRITE;
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_CH_CTL_1);
+
+ /* Start AUX transaction */
+ retval = analogix_dp_start_aux_transaction(dp);
+ if (retval == 0)
+ break;
+
+ dev_dbg(dp->dev, "%s: Aux Transaction fail!\n", __func__);
+ }
+
+ return retval;
+}
+
+int analogix_dp_read_byte_from_dpcd(struct analogix_dp_device *dp,
+ unsigned int reg_addr,
+ unsigned char *data)
+{
+ u32 reg;
+ int i;
+ int retval;
+
+ for (i = 0; i < 3; i++) {
+ /* Clear AUX CH data buffer */
+ reg = BUF_CLR;
+ writel(reg, dp->reg_base + ANALOGIX_DP_BUFFER_DATA_CTL);
+
+ /* Select DPCD device address */
+ reg = AUX_ADDR_7_0(reg_addr);
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_ADDR_7_0);
+ reg = AUX_ADDR_15_8(reg_addr);
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_ADDR_15_8);
+ reg = AUX_ADDR_19_16(reg_addr);
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_ADDR_19_16);
+
+ /*
+ * Set DisplayPort transaction and read 1 byte
+ * If bit 3 is 1, DisplayPort transaction.
+ * If Bit 3 is 0, I2C transaction.
+ */
+ reg = AUX_TX_COMM_DP_TRANSACTION | AUX_TX_COMM_READ;
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_CH_CTL_1);
+
+ /* Start AUX transaction */
+ retval = analogix_dp_start_aux_transaction(dp);
+ if (retval == 0)
+ break;
+
+ dev_dbg(dp->dev, "%s: Aux Transaction fail!\n", __func__);
+ }
+
+ /* Read data buffer */
+ reg = readl(dp->reg_base + ANALOGIX_DP_BUF_DATA_0);
+ *data = (unsigned char)(reg & 0xff);
+
+ return retval;
+}
+
+int analogix_dp_write_bytes_to_dpcd(struct analogix_dp_device *dp,
+ unsigned int reg_addr,
+ unsigned int count,
+ unsigned char data[])
+{
+ u32 reg;
+ unsigned int start_offset;
+ unsigned int cur_data_count;
+ unsigned int cur_data_idx;
+ int i;
+ int retval = 0;
+
+ /* Clear AUX CH data buffer */
+ reg = BUF_CLR;
+ writel(reg, dp->reg_base + ANALOGIX_DP_BUFFER_DATA_CTL);
+
+ start_offset = 0;
+ while (start_offset < count) {
+ /* Buffer size of AUX CH is 16 * 4bytes */
+ if ((count - start_offset) > 16)
+ cur_data_count = 16;
+ else
+ cur_data_count = count - start_offset;
+
+ for (i = 0; i < 3; i++) {
+ /* Select DPCD device address */
+ reg = AUX_ADDR_7_0(reg_addr + start_offset);
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_ADDR_7_0);
+ reg = AUX_ADDR_15_8(reg_addr + start_offset);
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_ADDR_15_8);
+ reg = AUX_ADDR_19_16(reg_addr + start_offset);
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_ADDR_19_16);
+
+ for (cur_data_idx = 0; cur_data_idx < cur_data_count;
+ cur_data_idx++) {
+ reg = data[start_offset + cur_data_idx];
+ writel(reg, dp->reg_base +
+ ANALOGIX_DP_BUF_DATA_0 +
+ 4 * cur_data_idx);
+ }
+
+ /*
+ * Set DisplayPort transaction and write
+ * If bit 3 is 1, DisplayPort transaction.
+ * If Bit 3 is 0, I2C transaction.
+ */
+ reg = AUX_LENGTH(cur_data_count) |
+ AUX_TX_COMM_DP_TRANSACTION | AUX_TX_COMM_WRITE;
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_CH_CTL_1);
+
+ /* Start AUX transaction */
+ retval = analogix_dp_start_aux_transaction(dp);
+ if (retval == 0)
+ break;
+
+ dev_dbg(dp->dev, "%s: Aux Transaction fail!\n",
+ __func__);
+ }
+
+ start_offset += cur_data_count;
+ }
+
+ return retval;
+}
+
+int analogix_dp_read_bytes_from_dpcd(struct analogix_dp_device *dp,
+ unsigned int reg_addr,
+ unsigned int count,
+ unsigned char data[])
+{
+ u32 reg;
+ unsigned int start_offset;
+ unsigned int cur_data_count;
+ unsigned int cur_data_idx;
+ int i;
+ int retval = 0;
+
+ /* Clear AUX CH data buffer */
+ reg = BUF_CLR;
+ writel(reg, dp->reg_base + ANALOGIX_DP_BUFFER_DATA_CTL);
+
+ start_offset = 0;
+ while (start_offset < count) {
+ /* Buffer size of AUX CH is 16 * 4bytes */
+ if ((count - start_offset) > 16)
+ cur_data_count = 16;
+ else
+ cur_data_count = count - start_offset;
+
+ /* AUX CH Request Transaction process */
+ for (i = 0; i < 3; i++) {
+ /* Select DPCD device address */
+ reg = AUX_ADDR_7_0(reg_addr + start_offset);
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_ADDR_7_0);
+ reg = AUX_ADDR_15_8(reg_addr + start_offset);
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_ADDR_15_8);
+ reg = AUX_ADDR_19_16(reg_addr + start_offset);
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_ADDR_19_16);
+
+ /*
+ * Set DisplayPort transaction and read
+ * If bit 3 is 1, DisplayPort transaction.
+ * If Bit 3 is 0, I2C transaction.
+ */
+ reg = AUX_LENGTH(cur_data_count) |
+ AUX_TX_COMM_DP_TRANSACTION | AUX_TX_COMM_READ;
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_CH_CTL_1);
+
+ /* Start AUX transaction */
+ retval = analogix_dp_start_aux_transaction(dp);
+ if (retval == 0)
+ break;
+
+ dev_dbg(dp->dev, "%s: Aux Transaction fail!\n",
+ __func__);
+ }
+
+ for (cur_data_idx = 0; cur_data_idx < cur_data_count;
+ cur_data_idx++) {
+ reg = readl(dp->reg_base + ANALOGIX_DP_BUF_DATA_0
+ + 4 * cur_data_idx);
+ data[start_offset + cur_data_idx] =
+ (unsigned char)reg;
+ }
+
+ start_offset += cur_data_count;
+ }
+
+ return retval;
+}
+
+int analogix_dp_select_i2c_device(struct analogix_dp_device *dp,
+ unsigned int device_addr,
+ unsigned int reg_addr)
+{
+ u32 reg;
+ int retval;
+
+ /* Set EDID device address */
+ reg = device_addr;
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_ADDR_7_0);
+ writel(0x0, dp->reg_base + ANALOGIX_DP_AUX_ADDR_15_8);
+ writel(0x0, dp->reg_base + ANALOGIX_DP_AUX_ADDR_19_16);
+
+ /* Set offset from base address of EDID device */
+ writel(reg_addr, dp->reg_base + ANALOGIX_DP_BUF_DATA_0);
+
+ /*
+ * Set I2C transaction and write address
+ * If bit 3 is 1, DisplayPort transaction.
+ * If Bit 3 is 0, I2C transaction.
+ */
+ reg = AUX_TX_COMM_I2C_TRANSACTION | AUX_TX_COMM_MOT |
+ AUX_TX_COMM_WRITE;
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_CH_CTL_1);
+
+ /* Start AUX transaction */
+ retval = analogix_dp_start_aux_transaction(dp);
+ if (retval != 0)
+ dev_dbg(dp->dev, "%s: Aux Transaction fail!\n", __func__);
+
+ return retval;
+}
+
+int analogix_dp_read_byte_from_i2c(struct analogix_dp_device *dp,
+ unsigned int device_addr,
+ unsigned int reg_addr,
+ unsigned int *data)
+{
+ u32 reg;
+ int i;
+ int retval;
+
+ for (i = 0; i < 3; i++) {
+ /* Clear AUX CH data buffer */
+ reg = BUF_CLR;
+ writel(reg, dp->reg_base + ANALOGIX_DP_BUFFER_DATA_CTL);
+
+ /* Select EDID device */
+ retval = analogix_dp_select_i2c_device(dp, device_addr,
+ reg_addr);
+ if (retval != 0)
+ continue;
+
+ /*
+ * Set I2C transaction and read data
+ * If bit 3 is 1, DisplayPort transaction.
+ * If Bit 3 is 0, I2C transaction.
+ */
+ reg = AUX_TX_COMM_I2C_TRANSACTION |
+ AUX_TX_COMM_READ;
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_CH_CTL_1);
+
+ /* Start AUX transaction */
+ retval = analogix_dp_start_aux_transaction(dp);
+ if (retval == 0)
+ break;
+
+ dev_dbg(dp->dev, "%s: Aux Transaction fail!\n", __func__);
+ }
+
+ /* Read data */
+ if (retval == 0)
+ *data = readl(dp->reg_base + ANALOGIX_DP_BUF_DATA_0);
+
+ return retval;
+}
+
+int analogix_dp_read_bytes_from_i2c(struct analogix_dp_device *dp,
+ unsigned int device_addr,
+ unsigned int reg_addr,
+ unsigned int count,
+ unsigned char edid[])
+{
+ u32 reg;
+ unsigned int i, j;
+ unsigned int cur_data_idx;
+ unsigned int defer = 0;
+ int retval = 0;
+
+ for (i = 0; i < count; i += 16) {
+ for (j = 0; j < 3; j++) {
+ /* Clear AUX CH data buffer */
+ reg = BUF_CLR;
+ writel(reg, dp->reg_base + ANALOGIX_DP_BUFFER_DATA_CTL);
+
+ /* Set normal AUX CH command */
+ reg = readl(dp->reg_base + ANALOGIX_DP_AUX_CH_CTL_2);
+ reg &= ~ADDR_ONLY;
+ writel(reg, dp->reg_base + ANALOGIX_DP_AUX_CH_CTL_2);
+
+ /*
+ * If Rx sends defer, Tx sends only reads
+ * request without sending address
+ */
+ if (!defer)
+ retval = analogix_dp_select_i2c_device(dp,
+ device_addr, reg_addr + i);
+ else
+ defer = 0;
+
+ if (retval == 0) {
+ /*
+ * Set I2C transaction and write data
+ * If bit 3 is 1, DisplayPort transaction.
+ * If Bit 3 is 0, I2C transaction.
+ */
+ reg = AUX_LENGTH(16) |
+ AUX_TX_COMM_I2C_TRANSACTION |
+ AUX_TX_COMM_READ;
+ writel(reg, dp->reg_base +
+ ANALOGIX_DP_AUX_CH_CTL_1);
+
+ /* Start AUX transaction */
+ retval = analogix_dp_start_aux_transaction(dp);
+ if (retval == 0)
+ break;
+
+ dev_dbg(dp->dev, "%s: Aux Transaction fail!\n",
+ __func__);
+ }
+ /* Check if Rx sends defer */
+ reg = readl(dp->reg_base + ANALOGIX_DP_AUX_RX_COMM);
+ if (reg == AUX_RX_COMM_AUX_DEFER ||
+ reg == AUX_RX_COMM_I2C_DEFER) {
+ dev_err(dp->dev, "Defer: %d\n\n", reg);
+ defer = 1;
+ }
+ }
+
+ for (cur_data_idx = 0; cur_data_idx < 16; cur_data_idx++) {
+ reg = readl(dp->reg_base + ANALOGIX_DP_BUF_DATA_0
+ + 4 * cur_data_idx);
+ edid[i + cur_data_idx] = (unsigned char)reg;
+ }
+ }
+
+ return retval;
+}
+
+void analogix_dp_set_link_bandwidth(struct analogix_dp_device *dp, u32 bwtype)
+{
+ u32 reg;
+
+ reg = bwtype;
+ if ((bwtype == DP_LINK_BW_2_7) || (bwtype == DP_LINK_BW_1_62))
+ writel(reg, dp->reg_base + ANALOGIX_DP_LINK_BW_SET);
+}
+
+void analogix_dp_get_link_bandwidth(struct analogix_dp_device *dp, u32 *bwtype)
+{
+ u32 reg;
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_LINK_BW_SET);
+ *bwtype = reg;
+}
+
+void analogix_dp_set_lane_count(struct analogix_dp_device *dp, u32 count)
+{
+ u32 reg;
+
+ reg = count;
+ writel(reg, dp->reg_base + ANALOGIX_DP_LANE_COUNT_SET);
+}
+
+void analogix_dp_get_lane_count(struct analogix_dp_device *dp, u32 *count)
+{
+ u32 reg;
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_LANE_COUNT_SET);
+ *count = reg;
+}
+
+void analogix_dp_enable_enhanced_mode(struct analogix_dp_device *dp,
+ bool enable)
+{
+ u32 reg;
+
+ if (enable) {
+ reg = readl(dp->reg_base + ANALOGIX_DP_SYS_CTL_4);
+ reg |= ENHANCED;
+ writel(reg, dp->reg_base + ANALOGIX_DP_SYS_CTL_4);
+ } else {
+ reg = readl(dp->reg_base + ANALOGIX_DP_SYS_CTL_4);
+ reg &= ~ENHANCED;
+ writel(reg, dp->reg_base + ANALOGIX_DP_SYS_CTL_4);
+ }
+}
+
+void analogix_dp_set_training_pattern(struct analogix_dp_device *dp,
+ enum pattern_set pattern)
+{
+ u32 reg;
+
+ switch (pattern) {
+ case PRBS7:
+ reg = SCRAMBLING_ENABLE | LINK_QUAL_PATTERN_SET_PRBS7;
+ writel(reg, dp->reg_base + ANALOGIX_DP_TRAINING_PTN_SET);
+ break;
+ case D10_2:
+ reg = SCRAMBLING_ENABLE | LINK_QUAL_PATTERN_SET_D10_2;
+ writel(reg, dp->reg_base + ANALOGIX_DP_TRAINING_PTN_SET);
+ break;
+ case TRAINING_PTN1:
+ reg = SCRAMBLING_DISABLE | SW_TRAINING_PATTERN_SET_PTN1;
+ writel(reg, dp->reg_base + ANALOGIX_DP_TRAINING_PTN_SET);
+ break;
+ case TRAINING_PTN2:
+ reg = SCRAMBLING_DISABLE | SW_TRAINING_PATTERN_SET_PTN2;
+ writel(reg, dp->reg_base + ANALOGIX_DP_TRAINING_PTN_SET);
+ break;
+ case DP_NONE:
+ reg = SCRAMBLING_ENABLE |
+ LINK_QUAL_PATTERN_SET_DISABLE |
+ SW_TRAINING_PATTERN_SET_NORMAL;
+ writel(reg, dp->reg_base + ANALOGIX_DP_TRAINING_PTN_SET);
+ break;
+ default:
+ break;
+ }
+}
+
+void analogix_dp_set_lane0_pre_emphasis(struct analogix_dp_device *dp,
+ u32 level)
+{
+ u32 reg;
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_LN0_LINK_TRAINING_CTL);
+ reg &= ~PRE_EMPHASIS_SET_MASK;
+ reg |= level << PRE_EMPHASIS_SET_SHIFT;
+ writel(reg, dp->reg_base + ANALOGIX_DP_LN0_LINK_TRAINING_CTL);
+}
+
+void analogix_dp_set_lane1_pre_emphasis(struct analogix_dp_device *dp,
+ u32 level)
+{
+ u32 reg;
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_LN1_LINK_TRAINING_CTL);
+ reg &= ~PRE_EMPHASIS_SET_MASK;
+ reg |= level << PRE_EMPHASIS_SET_SHIFT;
+ writel(reg, dp->reg_base + ANALOGIX_DP_LN1_LINK_TRAINING_CTL);
+}
+
+void analogix_dp_set_lane2_pre_emphasis(struct analogix_dp_device *dp,
+ u32 level)
+{
+ u32 reg;
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_LN2_LINK_TRAINING_CTL);
+ reg &= ~PRE_EMPHASIS_SET_MASK;
+ reg |= level << PRE_EMPHASIS_SET_SHIFT;
+ writel(reg, dp->reg_base + ANALOGIX_DP_LN2_LINK_TRAINING_CTL);
+}
+
+void analogix_dp_set_lane3_pre_emphasis(struct analogix_dp_device *dp,
+ u32 level)
+{
+ u32 reg;
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_LN3_LINK_TRAINING_CTL);
+ reg &= ~PRE_EMPHASIS_SET_MASK;
+ reg |= level << PRE_EMPHASIS_SET_SHIFT;
+ writel(reg, dp->reg_base + ANALOGIX_DP_LN3_LINK_TRAINING_CTL);
+}
+
+void analogix_dp_set_lane0_link_training(struct analogix_dp_device *dp,
+ u32 training_lane)
+{
+ u32 reg;
+
+ reg = training_lane;
+ writel(reg, dp->reg_base + ANALOGIX_DP_LN0_LINK_TRAINING_CTL);
+}
+
+void analogix_dp_set_lane1_link_training(struct analogix_dp_device *dp,
+ u32 training_lane)
+{
+ u32 reg;
+
+ reg = training_lane;
+ writel(reg, dp->reg_base + ANALOGIX_DP_LN1_LINK_TRAINING_CTL);
+}
+
+void analogix_dp_set_lane2_link_training(struct analogix_dp_device *dp,
+ u32 training_lane)
+{
+ u32 reg;
+
+ reg = training_lane;
+ writel(reg, dp->reg_base + ANALOGIX_DP_LN2_LINK_TRAINING_CTL);
+}
+
+void analogix_dp_set_lane3_link_training(struct analogix_dp_device *dp,
+ u32 training_lane)
+{
+ u32 reg;
+
+ reg = training_lane;
+ writel(reg, dp->reg_base + ANALOGIX_DP_LN3_LINK_TRAINING_CTL);
+}
+
+u32 analogix_dp_get_lane0_link_training(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_LN0_LINK_TRAINING_CTL);
+ return reg;
+}
+
+u32 analogix_dp_get_lane1_link_training(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_LN1_LINK_TRAINING_CTL);
+ return reg;
+}
+
+u32 analogix_dp_get_lane2_link_training(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_LN2_LINK_TRAINING_CTL);
+ return reg;
+}
+
+u32 analogix_dp_get_lane3_link_training(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_LN3_LINK_TRAINING_CTL);
+ return reg;
+}
+
+void analogix_dp_reset_macro(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_PHY_TEST);
+ reg |= MACRO_RST;
+ writel(reg, dp->reg_base + ANALOGIX_DP_PHY_TEST);
+
+ /* 10 us is the minimum reset time. */
+ usleep_range(10, 20);
+
+ reg &= ~MACRO_RST;
+ writel(reg, dp->reg_base + ANALOGIX_DP_PHY_TEST);
+}
+
+void analogix_dp_init_video(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ reg = VSYNC_DET | VID_FORMAT_CHG | VID_CLK_CHG;
+ writel(reg, dp->reg_base + ANALOGIX_DP_COMMON_INT_STA_1);
+
+ reg = 0x0;
+ writel(reg, dp->reg_base + ANALOGIX_DP_SYS_CTL_1);
+
+ reg = CHA_CRI(4) | CHA_CTRL;
+ writel(reg, dp->reg_base + ANALOGIX_DP_SYS_CTL_2);
+
+ reg = 0x0;
+ writel(reg, dp->reg_base + ANALOGIX_DP_SYS_CTL_3);
+
+ reg = VID_HRES_TH(2) | VID_VRES_TH(0);
+ writel(reg, dp->reg_base + ANALOGIX_DP_VIDEO_CTL_8);
+}
+
+void analogix_dp_set_video_color_format(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ /* Configure the input color depth, color space, dynamic range */
+ reg = (dp->video_info.dynamic_range << IN_D_RANGE_SHIFT) |
+ (dp->video_info.color_depth << IN_BPC_SHIFT) |
+ (dp->video_info.color_space << IN_COLOR_F_SHIFT);
+ writel(reg, dp->reg_base + ANALOGIX_DP_VIDEO_CTL_2);
+
+ /* Set Input Color YCbCr Coefficients to ITU601 or ITU709 */
+ reg = readl(dp->reg_base + ANALOGIX_DP_VIDEO_CTL_3);
+ reg &= ~IN_YC_COEFFI_MASK;
+ if (dp->video_info.ycbcr_coeff)
+ reg |= IN_YC_COEFFI_ITU709;
+ else
+ reg |= IN_YC_COEFFI_ITU601;
+ writel(reg, dp->reg_base + ANALOGIX_DP_VIDEO_CTL_3);
+}
+
+int analogix_dp_is_slave_video_stream_clock_on(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_SYS_CTL_1);
+ writel(reg, dp->reg_base + ANALOGIX_DP_SYS_CTL_1);
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_SYS_CTL_1);
+
+ if (!(reg & DET_STA)) {
+ dev_dbg(dp->dev, "Input stream clock not detected.\n");
+ return -EINVAL;
+ }
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_SYS_CTL_2);
+ writel(reg, dp->reg_base + ANALOGIX_DP_SYS_CTL_2);
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_SYS_CTL_2);
+ dev_dbg(dp->dev, "wait SYS_CTL_2.\n");
+
+ if (reg & CHA_STA) {
+ dev_dbg(dp->dev, "Input stream clk is changing\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+void analogix_dp_set_video_cr_mn(struct analogix_dp_device *dp,
+ enum clock_recovery_m_value_type type,
+ u32 m_value, u32 n_value)
+{
+ u32 reg;
+
+ if (type == REGISTER_M) {
+ reg = readl(dp->reg_base + ANALOGIX_DP_SYS_CTL_4);
+ reg |= FIX_M_VID;
+ writel(reg, dp->reg_base + ANALOGIX_DP_SYS_CTL_4);
+ reg = m_value & 0xff;
+ writel(reg, dp->reg_base + ANALOGIX_DP_M_VID_0);
+ reg = (m_value >> 8) & 0xff;
+ writel(reg, dp->reg_base + ANALOGIX_DP_M_VID_1);
+ reg = (m_value >> 16) & 0xff;
+ writel(reg, dp->reg_base + ANALOGIX_DP_M_VID_2);
+
+ reg = n_value & 0xff;
+ writel(reg, dp->reg_base + ANALOGIX_DP_N_VID_0);
+ reg = (n_value >> 8) & 0xff;
+ writel(reg, dp->reg_base + ANALOGIX_DP_N_VID_1);
+ reg = (n_value >> 16) & 0xff;
+ writel(reg, dp->reg_base + ANALOGIX_DP_N_VID_2);
+ } else {
+ reg = readl(dp->reg_base + ANALOGIX_DP_SYS_CTL_4);
+ reg &= ~FIX_M_VID;
+ writel(reg, dp->reg_base + ANALOGIX_DP_SYS_CTL_4);
+
+ writel(0x00, dp->reg_base + ANALOGIX_DP_N_VID_0);
+ writel(0x80, dp->reg_base + ANALOGIX_DP_N_VID_1);
+ writel(0x00, dp->reg_base + ANALOGIX_DP_N_VID_2);
+ }
+}
+
+void analogix_dp_set_video_timing_mode(struct analogix_dp_device *dp, u32 type)
+{
+ u32 reg;
+
+ if (type == VIDEO_TIMING_FROM_CAPTURE) {
+ reg = readl(dp->reg_base + ANALOGIX_DP_VIDEO_CTL_10);
+ reg &= ~FORMAT_SEL;
+ writel(reg, dp->reg_base + ANALOGIX_DP_VIDEO_CTL_10);
+ } else {
+ reg = readl(dp->reg_base + ANALOGIX_DP_VIDEO_CTL_10);
+ reg |= FORMAT_SEL;
+ writel(reg, dp->reg_base + ANALOGIX_DP_VIDEO_CTL_10);
+ }
+}
+
+void analogix_dp_enable_video_master(struct analogix_dp_device *dp, bool enable)
+{
+ u32 reg;
+
+ if (enable) {
+ reg = readl(dp->reg_base + ANALOGIX_DP_SOC_GENERAL_CTL);
+ reg &= ~VIDEO_MODE_MASK;
+ reg |= VIDEO_MASTER_MODE_EN | VIDEO_MODE_MASTER_MODE;
+ writel(reg, dp->reg_base + ANALOGIX_DP_SOC_GENERAL_CTL);
+ } else {
+ reg = readl(dp->reg_base + ANALOGIX_DP_SOC_GENERAL_CTL);
+ reg &= ~VIDEO_MODE_MASK;
+ reg |= VIDEO_MODE_SLAVE_MODE;
+ writel(reg, dp->reg_base + ANALOGIX_DP_SOC_GENERAL_CTL);
+ }
+}
+
+void analogix_dp_start_video(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_VIDEO_CTL_1);
+ reg |= VIDEO_EN;
+ writel(reg, dp->reg_base + ANALOGIX_DP_VIDEO_CTL_1);
+}
+
+int analogix_dp_is_video_stream_on(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_SYS_CTL_3);
+ writel(reg, dp->reg_base + ANALOGIX_DP_SYS_CTL_3);
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_SYS_CTL_3);
+ if (!(reg & STRM_VALID)) {
+ dev_dbg(dp->dev, "Input video stream is not detected.\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+void analogix_dp_config_video_slave_mode(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_FUNC_EN_1);
+ reg &= ~(MASTER_VID_FUNC_EN_N | SLAVE_VID_FUNC_EN_N);
+ reg |= MASTER_VID_FUNC_EN_N;
+ writel(reg, dp->reg_base + ANALOGIX_DP_FUNC_EN_1);
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_VIDEO_CTL_10);
+ reg &= ~INTERACE_SCAN_CFG;
+ reg |= (dp->video_info.interlaced << 2);
+ writel(reg, dp->reg_base + ANALOGIX_DP_VIDEO_CTL_10);
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_VIDEO_CTL_10);
+ reg &= ~VSYNC_POLARITY_CFG;
+ reg |= (dp->video_info.v_sync_polarity << 1);
+ writel(reg, dp->reg_base + ANALOGIX_DP_VIDEO_CTL_10);
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_VIDEO_CTL_10);
+ reg &= ~HSYNC_POLARITY_CFG;
+ reg |= (dp->video_info.h_sync_polarity << 0);
+ writel(reg, dp->reg_base + ANALOGIX_DP_VIDEO_CTL_10);
+
+ reg = AUDIO_MODE_SPDIF_MODE | VIDEO_MODE_SLAVE_MODE;
+ writel(reg, dp->reg_base + ANALOGIX_DP_SOC_GENERAL_CTL);
+}
+
+void analogix_dp_enable_scrambling(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_TRAINING_PTN_SET);
+ reg &= ~SCRAMBLING_DISABLE;
+ writel(reg, dp->reg_base + ANALOGIX_DP_TRAINING_PTN_SET);
+}
+
+void analogix_dp_disable_scrambling(struct analogix_dp_device *dp)
+{
+ u32 reg;
+
+ reg = readl(dp->reg_base + ANALOGIX_DP_TRAINING_PTN_SET);
+ reg |= SCRAMBLING_DISABLE;
+ writel(reg, dp->reg_base + ANALOGIX_DP_TRAINING_PTN_SET);
+}
diff --git a/drivers/gpu/drm/bridge/analogix/analogix_dp_reg.h b/drivers/gpu/drm/bridge/analogix/analogix_dp_reg.h
new file mode 100644
index 0000000..337912b
--- /dev/null
+++ b/drivers/gpu/drm/bridge/analogix/analogix_dp_reg.h
@@ -0,0 +1,378 @@
+/*
+ * Register definition file for Analogix DP core driver
+ *
+ * Copyright (C) 2012 Samsung Electronics Co., Ltd.
+ * Author: Jingoo Han <jg1.han@samsung.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef _ANALOGIX_DP_REG_H
+#define _ANALOGIX_DP_REG_H
+
+#define ANALOGIX_DP_TX_SW_RESET 0x14
+#define ANALOGIX_DP_FUNC_EN_1 0x18
+#define ANALOGIX_DP_FUNC_EN_2 0x1C
+#define ANALOGIX_DP_VIDEO_CTL_1 0x20
+#define ANALOGIX_DP_VIDEO_CTL_2 0x24
+#define ANALOGIX_DP_VIDEO_CTL_3 0x28
+
+#define ANALOGIX_DP_VIDEO_CTL_8 0x3C
+#define ANALOGIX_DP_VIDEO_CTL_10 0x44
+
+#define ANALOGIX_DP_PLL_REG_1 0xfc
+#define ANALOGIX_DP_PLL_REG_2 0x9e4
+#define ANALOGIX_DP_PLL_REG_3 0x9e8
+#define ANALOGIX_DP_PLL_REG_4 0x9ec
+#define ANALOGIX_DP_PLL_REG_5 0xa00
+
+#define ANALOGIX_DP_PD 0x12c
+
+#define ANALOGIX_DP_LANE_MAP 0x35C
+
+#define ANALOGIX_DP_ANALOG_CTL_1 0x370
+#define ANALOGIX_DP_ANALOG_CTL_2 0x374
+#define ANALOGIX_DP_ANALOG_CTL_3 0x378
+#define ANALOGIX_DP_PLL_FILTER_CTL_1 0x37C
+#define ANALOGIX_DP_TX_AMP_TUNING_CTL 0x380
+
+#define ANALOGIX_DP_AUX_HW_RETRY_CTL 0x390
+
+#define ANALOGIX_DP_COMMON_INT_STA_1 0x3C4
+#define ANALOGIX_DP_COMMON_INT_STA_2 0x3C8
+#define ANALOGIX_DP_COMMON_INT_STA_3 0x3CC
+#define ANALOGIX_DP_COMMON_INT_STA_4 0x3D0
+#define ANALOGIX_DP_INT_STA 0x3DC
+#define ANALOGIX_DP_COMMON_INT_MASK_1 0x3E0
+#define ANALOGIX_DP_COMMON_INT_MASK_2 0x3E4
+#define ANALOGIX_DP_COMMON_INT_MASK_3 0x3E8
+#define ANALOGIX_DP_COMMON_INT_MASK_4 0x3EC
+#define ANALOGIX_DP_INT_STA_MASK 0x3F8
+#define ANALOGIX_DP_INT_CTL 0x3FC
+
+#define ANALOGIX_DP_SYS_CTL_1 0x600
+#define ANALOGIX_DP_SYS_CTL_2 0x604
+#define ANALOGIX_DP_SYS_CTL_3 0x608
+#define ANALOGIX_DP_SYS_CTL_4 0x60C
+
+#define ANALOGIX_DP_PKT_SEND_CTL 0x640
+#define ANALOGIX_DP_HDCP_CTL 0x648
+
+#define ANALOGIX_DP_LINK_BW_SET 0x680
+#define ANALOGIX_DP_LANE_COUNT_SET 0x684
+#define ANALOGIX_DP_TRAINING_PTN_SET 0x688
+#define ANALOGIX_DP_LN0_LINK_TRAINING_CTL 0x68C
+#define ANALOGIX_DP_LN1_LINK_TRAINING_CTL 0x690
+#define ANALOGIX_DP_LN2_LINK_TRAINING_CTL 0x694
+#define ANALOGIX_DP_LN3_LINK_TRAINING_CTL 0x698
+
+#define ANALOGIX_DP_DEBUG_CTL 0x6C0
+#define ANALOGIX_DP_HPD_DEGLITCH_L 0x6C4
+#define ANALOGIX_DP_HPD_DEGLITCH_H 0x6C8
+#define ANALOGIX_DP_LINK_DEBUG_CTL 0x6E0
+
+#define ANALOGIX_DP_M_VID_0 0x700
+#define ANALOGIX_DP_M_VID_1 0x704
+#define ANALOGIX_DP_M_VID_2 0x708
+#define ANALOGIX_DP_N_VID_0 0x70C
+#define ANALOGIX_DP_N_VID_1 0x710
+#define ANALOGIX_DP_N_VID_2 0x714
+
+#define ANALOGIX_DP_PLL_CTL 0x71C
+#define ANALOGIX_DP_PHY_PD 0x720
+#define ANALOGIX_DP_PHY_TEST 0x724
+
+#define ANALOGIX_DP_VIDEO_FIFO_THRD 0x730
+#define ANALOGIX_DP_AUDIO_MARGIN 0x73C
+
+#define ANALOGIX_DP_M_VID_GEN_FILTER_TH 0x764
+#define ANALOGIX_DP_M_AUD_GEN_FILTER_TH 0x778
+#define ANALOGIX_DP_AUX_CH_STA 0x780
+#define ANALOGIX_DP_AUX_CH_DEFER_CTL 0x788
+#define ANALOGIX_DP_AUX_RX_COMM 0x78C
+#define ANALOGIX_DP_BUFFER_DATA_CTL 0x790
+#define ANALOGIX_DP_AUX_CH_CTL_1 0x794
+#define ANALOGIX_DP_AUX_ADDR_7_0 0x798
+#define ANALOGIX_DP_AUX_ADDR_15_8 0x79C
+#define ANALOGIX_DP_AUX_ADDR_19_16 0x7A0
+#define ANALOGIX_DP_AUX_CH_CTL_2 0x7A4
+
+#define ANALOGIX_DP_BUF_DATA_0 0x7C0
+
+#define ANALOGIX_DP_SOC_GENERAL_CTL 0x800
+
+/* ANALOGIX_DP_TX_SW_RESET */
+#define RESET_DP_TX (0x1 << 0)
+
+/* ANALOGIX_DP_FUNC_EN_1 */
+#define MASTER_VID_FUNC_EN_N (0x1 << 7)
+#define SLAVE_VID_FUNC_EN_N (0x1 << 5)
+#define AUD_FIFO_FUNC_EN_N (0x1 << 4)
+#define AUD_FUNC_EN_N (0x1 << 3)
+#define HDCP_FUNC_EN_N (0x1 << 2)
+#define CRC_FUNC_EN_N (0x1 << 1)
+#define SW_FUNC_EN_N (0x1 << 0)
+
+/* ANALOGIX_DP_FUNC_EN_2 */
+#define SSC_FUNC_EN_N (0x1 << 7)
+#define AUX_FUNC_EN_N (0x1 << 2)
+#define SERDES_FIFO_FUNC_EN_N (0x1 << 1)
+#define LS_CLK_DOMAIN_FUNC_EN_N (0x1 << 0)
+
+/* ANALOGIX_DP_VIDEO_CTL_1 */
+#define VIDEO_EN (0x1 << 7)
+#define HDCP_VIDEO_MUTE (0x1 << 6)
+
+/* ANALOGIX_DP_VIDEO_CTL_1 */
+#define IN_D_RANGE_MASK (0x1 << 7)
+#define IN_D_RANGE_SHIFT (7)
+#define IN_D_RANGE_CEA (0x1 << 7)
+#define IN_D_RANGE_VESA (0x0 << 7)
+#define IN_BPC_MASK (0x7 << 4)
+#define IN_BPC_SHIFT (4)
+#define IN_BPC_12_BITS (0x3 << 4)
+#define IN_BPC_10_BITS (0x2 << 4)
+#define IN_BPC_8_BITS (0x1 << 4)
+#define IN_BPC_6_BITS (0x0 << 4)
+#define IN_COLOR_F_MASK (0x3 << 0)
+#define IN_COLOR_F_SHIFT (0)
+#define IN_COLOR_F_YCBCR444 (0x2 << 0)
+#define IN_COLOR_F_YCBCR422 (0x1 << 0)
+#define IN_COLOR_F_RGB (0x0 << 0)
+
+/* ANALOGIX_DP_VIDEO_CTL_3 */
+#define IN_YC_COEFFI_MASK (0x1 << 7)
+#define IN_YC_COEFFI_SHIFT (7)
+#define IN_YC_COEFFI_ITU709 (0x1 << 7)
+#define IN_YC_COEFFI_ITU601 (0x0 << 7)
+#define VID_CHK_UPDATE_TYPE_MASK (0x1 << 4)
+#define VID_CHK_UPDATE_TYPE_SHIFT (4)
+#define VID_CHK_UPDATE_TYPE_1 (0x1 << 4)
+#define VID_CHK_UPDATE_TYPE_0 (0x0 << 4)
+
+/* ANALOGIX_DP_VIDEO_CTL_8 */
+#define VID_HRES_TH(x) (((x) & 0xf) << 4)
+#define VID_VRES_TH(x) (((x) & 0xf) << 0)
+
+/* ANALOGIX_DP_VIDEO_CTL_10 */
+#define FORMAT_SEL (0x1 << 4)
+#define INTERACE_SCAN_CFG (0x1 << 2)
+#define VSYNC_POLARITY_CFG (0x1 << 1)
+#define HSYNC_POLARITY_CFG (0x1 << 0)
+
+/* ANALOGIX_DP_PLL_REG_1 */
+#define REF_CLK_24M (0x1 << 1)
+#define REF_CLK_27M (0x0 << 1)
+
+/* ANALOGIX_DP_LANE_MAP */
+#define LANE3_MAP_LOGIC_LANE_0 (0x0 << 6)
+#define LANE3_MAP_LOGIC_LANE_1 (0x1 << 6)
+#define LANE3_MAP_LOGIC_LANE_2 (0x2 << 6)
+#define LANE3_MAP_LOGIC_LANE_3 (0x3 << 6)
+#define LANE2_MAP_LOGIC_LANE_0 (0x0 << 4)
+#define LANE2_MAP_LOGIC_LANE_1 (0x1 << 4)
+#define LANE2_MAP_LOGIC_LANE_2 (0x2 << 4)
+#define LANE2_MAP_LOGIC_LANE_3 (0x3 << 4)
+#define LANE1_MAP_LOGIC_LANE_0 (0x0 << 2)
+#define LANE1_MAP_LOGIC_LANE_1 (0x1 << 2)
+#define LANE1_MAP_LOGIC_LANE_2 (0x2 << 2)
+#define LANE1_MAP_LOGIC_LANE_3 (0x3 << 2)
+#define LANE0_MAP_LOGIC_LANE_0 (0x0 << 0)
+#define LANE0_MAP_LOGIC_LANE_1 (0x1 << 0)
+#define LANE0_MAP_LOGIC_LANE_2 (0x2 << 0)
+#define LANE0_MAP_LOGIC_LANE_3 (0x3 << 0)
+
+/* ANALOGIX_DP_ANALOG_CTL_1 */
+#define TX_TERMINAL_CTRL_50_OHM (0x1 << 4)
+
+/* ANALOGIX_DP_ANALOG_CTL_2 */
+#define SEL_24M (0x1 << 3)
+#define TX_DVDD_BIT_1_0625V (0x4 << 0)
+
+/* ANALOGIX_DP_ANALOG_CTL_3 */
+#define DRIVE_DVDD_BIT_1_0625V (0x4 << 5)
+#define VCO_BIT_600_MICRO (0x5 << 0)
+
+/* ANALOGIX_DP_PLL_FILTER_CTL_1 */
+#define PD_RING_OSC (0x1 << 6)
+#define AUX_TERMINAL_CTRL_50_OHM (0x2 << 4)
+#define TX_CUR1_2X (0x1 << 2)
+#define TX_CUR_16_MA (0x3 << 0)
+
+/* ANALOGIX_DP_TX_AMP_TUNING_CTL */
+#define CH3_AMP_400_MV (0x0 << 24)
+#define CH2_AMP_400_MV (0x0 << 16)
+#define CH1_AMP_400_MV (0x0 << 8)
+#define CH0_AMP_400_MV (0x0 << 0)
+
+/* ANALOGIX_DP_AUX_HW_RETRY_CTL */
+#define AUX_BIT_PERIOD_EXPECTED_DELAY(x) (((x) & 0x7) << 8)
+#define AUX_HW_RETRY_INTERVAL_MASK (0x3 << 3)
+#define AUX_HW_RETRY_INTERVAL_600_MICROSECONDS (0x0 << 3)
+#define AUX_HW_RETRY_INTERVAL_800_MICROSECONDS (0x1 << 3)
+#define AUX_HW_RETRY_INTERVAL_1000_MICROSECONDS (0x2 << 3)
+#define AUX_HW_RETRY_INTERVAL_1800_MICROSECONDS (0x3 << 3)
+#define AUX_HW_RETRY_COUNT_SEL(x) (((x) & 0x7) << 0)
+
+/* ANALOGIX_DP_COMMON_INT_STA_1 */
+#define VSYNC_DET (0x1 << 7)
+#define PLL_LOCK_CHG (0x1 << 6)
+#define SPDIF_ERR (0x1 << 5)
+#define SPDIF_UNSTBL (0x1 << 4)
+#define VID_FORMAT_CHG (0x1 << 3)
+#define AUD_CLK_CHG (0x1 << 2)
+#define VID_CLK_CHG (0x1 << 1)
+#define SW_INT (0x1 << 0)
+
+/* ANALOGIX_DP_COMMON_INT_STA_2 */
+#define ENC_EN_CHG (0x1 << 6)
+#define HW_BKSV_RDY (0x1 << 3)
+#define HW_SHA_DONE (0x1 << 2)
+#define HW_AUTH_STATE_CHG (0x1 << 1)
+#define HW_AUTH_DONE (0x1 << 0)
+
+/* ANALOGIX_DP_COMMON_INT_STA_3 */
+#define AFIFO_UNDER (0x1 << 7)
+#define AFIFO_OVER (0x1 << 6)
+#define R0_CHK_FLAG (0x1 << 5)
+
+/* ANALOGIX_DP_COMMON_INT_STA_4 */
+#define PSR_ACTIVE (0x1 << 7)
+#define PSR_INACTIVE (0x1 << 6)
+#define SPDIF_BI_PHASE_ERR (0x1 << 5)
+#define HOTPLUG_CHG (0x1 << 2)
+#define HPD_LOST (0x1 << 1)
+#define PLUG (0x1 << 0)
+
+/* ANALOGIX_DP_INT_STA */
+#define INT_HPD (0x1 << 6)
+#define HW_TRAINING_FINISH (0x1 << 5)
+#define RPLY_RECEIV (0x1 << 1)
+#define AUX_ERR (0x1 << 0)
+
+/* ANALOGIX_DP_INT_CTL */
+#define SOFT_INT_CTRL (0x1 << 2)
+#define INT_POL1 (0x1 << 1)
+#define INT_POL0 (0x1 << 0)
+
+/* ANALOGIX_DP_SYS_CTL_1 */
+#define DET_STA (0x1 << 2)
+#define FORCE_DET (0x1 << 1)
+#define DET_CTRL (0x1 << 0)
+
+/* ANALOGIX_DP_SYS_CTL_2 */
+#define CHA_CRI(x) (((x) & 0xf) << 4)
+#define CHA_STA (0x1 << 2)
+#define FORCE_CHA (0x1 << 1)
+#define CHA_CTRL (0x1 << 0)
+
+/* ANALOGIX_DP_SYS_CTL_3 */
+#define HPD_STATUS (0x1 << 6)
+#define F_HPD (0x1 << 5)
+#define HPD_CTRL (0x1 << 4)
+#define HDCP_RDY (0x1 << 3)
+#define STRM_VALID (0x1 << 2)
+#define F_VALID (0x1 << 1)
+#define VALID_CTRL (0x1 << 0)
+
+/* ANALOGIX_DP_SYS_CTL_4 */
+#define FIX_M_AUD (0x1 << 4)
+#define ENHANCED (0x1 << 3)
+#define FIX_M_VID (0x1 << 2)
+#define M_VID_UPDATE_CTRL (0x3 << 0)
+
+/* ANALOGIX_DP_TRAINING_PTN_SET */
+#define SCRAMBLER_TYPE (0x1 << 9)
+#define HW_LINK_TRAINING_PATTERN (0x1 << 8)
+#define SCRAMBLING_DISABLE (0x1 << 5)
+#define SCRAMBLING_ENABLE (0x0 << 5)
+#define LINK_QUAL_PATTERN_SET_MASK (0x3 << 2)
+#define LINK_QUAL_PATTERN_SET_PRBS7 (0x3 << 2)
+#define LINK_QUAL_PATTERN_SET_D10_2 (0x1 << 2)
+#define LINK_QUAL_PATTERN_SET_DISABLE (0x0 << 2)
+#define SW_TRAINING_PATTERN_SET_MASK (0x3 << 0)
+#define SW_TRAINING_PATTERN_SET_PTN2 (0x2 << 0)
+#define SW_TRAINING_PATTERN_SET_PTN1 (0x1 << 0)
+#define SW_TRAINING_PATTERN_SET_NORMAL (0x0 << 0)
+
+/* ANALOGIX_DP_LN0_LINK_TRAINING_CTL */
+#define PRE_EMPHASIS_SET_MASK (0x3 << 3)
+#define PRE_EMPHASIS_SET_SHIFT (3)
+
+/* ANALOGIX_DP_DEBUG_CTL */
+#define PLL_LOCK (0x1 << 4)
+#define F_PLL_LOCK (0x1 << 3)
+#define PLL_LOCK_CTRL (0x1 << 2)
+#define PN_INV (0x1 << 0)
+
+/* ANALOGIX_DP_PLL_CTL */
+#define DP_PLL_PD (0x1 << 7)
+#define DP_PLL_RESET (0x1 << 6)
+#define DP_PLL_LOOP_BIT_DEFAULT (0x1 << 4)
+#define DP_PLL_REF_BIT_1_1250V (0x5 << 0)
+#define DP_PLL_REF_BIT_1_2500V (0x7 << 0)
+
+/* ANALOGIX_DP_PHY_PD */
+#define DP_PHY_PD (0x1 << 5)
+#define AUX_PD (0x1 << 4)
+#define CH3_PD (0x1 << 3)
+#define CH2_PD (0x1 << 2)
+#define CH1_PD (0x1 << 1)
+#define CH0_PD (0x1 << 0)
+
+/* ANALOGIX_DP_PHY_TEST */
+#define MACRO_RST (0x1 << 5)
+#define CH1_TEST (0x1 << 1)
+#define CH0_TEST (0x1 << 0)
+
+/* ANALOGIX_DP_AUX_CH_STA */
+#define AUX_BUSY (0x1 << 4)
+#define AUX_STATUS_MASK (0xf << 0)
+
+/* ANALOGIX_DP_AUX_CH_DEFER_CTL */
+#define DEFER_CTRL_EN (0x1 << 7)
+#define DEFER_COUNT(x) (((x) & 0x7f) << 0)
+
+/* ANALOGIX_DP_AUX_RX_COMM */
+#define AUX_RX_COMM_I2C_DEFER (0x2 << 2)
+#define AUX_RX_COMM_AUX_DEFER (0x2 << 0)
+
+/* ANALOGIX_DP_BUFFER_DATA_CTL */
+#define BUF_CLR (0x1 << 7)
+#define BUF_DATA_COUNT(x) (((x) & 0x1f) << 0)
+
+/* ANALOGIX_DP_AUX_CH_CTL_1 */
+#define AUX_LENGTH(x) (((x - 1) & 0xf) << 4)
+#define AUX_TX_COMM_MASK (0xf << 0)
+#define AUX_TX_COMM_DP_TRANSACTION (0x1 << 3)
+#define AUX_TX_COMM_I2C_TRANSACTION (0x0 << 3)
+#define AUX_TX_COMM_MOT (0x1 << 2)
+#define AUX_TX_COMM_WRITE (0x0 << 0)
+#define AUX_TX_COMM_READ (0x1 << 0)
+
+/* ANALOGIX_DP_AUX_ADDR_7_0 */
+#define AUX_ADDR_7_0(x) (((x) >> 0) & 0xff)
+
+/* ANALOGIX_DP_AUX_ADDR_15_8 */
+#define AUX_ADDR_15_8(x) (((x) >> 8) & 0xff)
+
+/* ANALOGIX_DP_AUX_ADDR_19_16 */
+#define AUX_ADDR_19_16(x) (((x) >> 16) & 0x0f)
+
+/* ANALOGIX_DP_AUX_CH_CTL_2 */
+#define ADDR_ONLY (0x1 << 1)
+#define AUX_EN (0x1 << 0)
+
+/* ANALOGIX_DP_SOC_GENERAL_CTL */
+#define AUDIO_MODE_SPDIF_MODE (0x1 << 8)
+#define AUDIO_MODE_MASTER_MODE (0x0 << 8)
+#define MASTER_VIDEO_INTERLACE_EN (0x1 << 4)
+#define VIDEO_MASTER_CLK_SEL (0x1 << 2)
+#define VIDEO_MASTER_MODE_EN (0x1 << 1)
+#define VIDEO_MODE_MASK (0x1 << 0)
+#define VIDEO_MODE_SLAVE_MODE (0x1 << 0)
+#define VIDEO_MODE_MASTER_MODE (0x0 << 0)
+
+#endif /* _ANALOGIX_DP_REG_H */
diff --git a/drivers/gpu/drm/bridge/dw-hdmi.c b/drivers/gpu/drm/bridge/dw-hdmi.c
index 9795b72..c9d9412 100644
--- a/drivers/gpu/drm/bridge/dw-hdmi.c
+++ b/drivers/gpu/drm/bridge/dw-hdmi.c
@@ -1413,11 +1413,6 @@
mutex_unlock(&hdmi->mutex);
}
-static void dw_hdmi_bridge_nop(struct drm_bridge *bridge)
-{
- /* do nothing */
-}
-
static enum drm_connector_status
dw_hdmi_connector_detect(struct drm_connector *connector, bool force)
{
@@ -1536,8 +1531,6 @@
static const struct drm_bridge_funcs dw_hdmi_bridge_funcs = {
.enable = dw_hdmi_bridge_enable,
.disable = dw_hdmi_bridge_disable,
- .pre_enable = dw_hdmi_bridge_nop,
- .post_disable = dw_hdmi_bridge_nop,
.mode_set = dw_hdmi_bridge_mode_set,
};
diff --git a/drivers/gpu/drm/cirrus/cirrus_drv.c b/drivers/gpu/drm/cirrus/cirrus_drv.c
index 7bc394e..dc83f69 100644
--- a/drivers/gpu/drm/cirrus/cirrus_drv.c
+++ b/drivers/gpu/drm/cirrus/cirrus_drv.c
@@ -163,10 +163,8 @@
static int __init cirrus_init(void)
{
-#ifdef CONFIG_VGA_CONSOLE
if (vgacon_text_force() && cirrus_modeset == -1)
return -EINVAL;
-#endif
if (cirrus_modeset == 0)
return -EINVAL;
diff --git a/drivers/gpu/drm/cirrus/cirrus_drv.h b/drivers/gpu/drm/cirrus/cirrus_drv.h
index b774d63..2188d6b 100644
--- a/drivers/gpu/drm/cirrus/cirrus_drv.h
+++ b/drivers/gpu/drm/cirrus/cirrus_drv.h
@@ -245,7 +245,7 @@
{
int ret;
- ret = ttm_bo_reserve(&bo->bo, true, no_wait, false, NULL);
+ ret = ttm_bo_reserve(&bo->bo, true, no_wait, NULL);
if (ret) {
if (ret != -ERESTARTSYS && ret != -EBUSY)
DRM_ERROR("reserve failed %p\n", bo);
diff --git a/drivers/gpu/drm/cirrus/cirrus_main.c b/drivers/gpu/drm/cirrus/cirrus_main.c
index 0907715..32d32c5 100644
--- a/drivers/gpu/drm/cirrus/cirrus_main.c
+++ b/drivers/gpu/drm/cirrus/cirrus_main.c
@@ -61,7 +61,7 @@
bpp, mode_cmd->pitches[0]))
return ERR_PTR(-EINVAL);
- obj = drm_gem_object_lookup(dev, filp, mode_cmd->handles[0]);
+ obj = drm_gem_object_lookup(filp, mode_cmd->handles[0]);
if (obj == NULL)
return ERR_PTR(-ENOENT);
@@ -295,7 +295,7 @@
struct drm_gem_object *obj;
struct cirrus_bo *bo;
- obj = drm_gem_object_lookup(dev, file, handle);
+ obj = drm_gem_object_lookup(file, handle);
if (obj == NULL)
return -ENOENT;
diff --git a/drivers/gpu/drm/cirrus/cirrus_ttm.c b/drivers/gpu/drm/cirrus/cirrus_ttm.c
index dfffd52..6768b7b 100644
--- a/drivers/gpu/drm/cirrus/cirrus_ttm.c
+++ b/drivers/gpu/drm/cirrus/cirrus_ttm.c
@@ -245,6 +245,8 @@
.verify_access = cirrus_bo_verify_access,
.io_mem_reserve = &cirrus_ttm_io_mem_reserve,
.io_mem_free = &cirrus_ttm_io_mem_free,
+ .lru_tail = &ttm_bo_default_lru_tail,
+ .swap_lru_tail = &ttm_bo_default_swap_lru_tail,
};
int cirrus_mm_init(struct cirrus_device *cirrus)
diff --git a/drivers/gpu/drm/drm_agpsupport.c b/drivers/gpu/drm/drm_agpsupport.c
index a10ea6a..605bd24 100644
--- a/drivers/gpu/drm/drm_agpsupport.c
+++ b/drivers/gpu/drm/drm_agpsupport.c
@@ -423,7 +423,7 @@
}
/**
- * drm_agp_clear - Clear AGP resource list
+ * drm_legacy_agp_clear - Clear AGP resource list
* @dev: DRM device
*
* Iterate over all AGP resources and remove them. But keep the AGP head
@@ -434,7 +434,7 @@
* resources from getting destroyed. Drivers are responsible of cleaning them up
* during device shutdown.
*/
-void drm_agp_clear(struct drm_device *dev)
+void drm_legacy_agp_clear(struct drm_device *dev)
{
struct drm_agp_mem *entry, *tempe;
diff --git a/drivers/gpu/drm/drm_atomic.c b/drivers/gpu/drm/drm_atomic.c
index 8ee1db8..3ff1ed7 100644
--- a/drivers/gpu/drm/drm_atomic.c
+++ b/drivers/gpu/drm/drm_atomic.c
@@ -31,6 +31,8 @@
#include <drm/drm_mode.h>
#include <drm/drm_plane_helper.h>
+#include "drm_crtc_internal.h"
+
/**
* drm_atomic_state_default_release -
* release memory initialized by drm_atomic_state_init
@@ -142,18 +144,11 @@
if (!connector)
continue;
- /*
- * FIXME: Async commits can race with connector unplugging and
- * there's currently nothing that prevents cleanup up state for
- * deleted connectors. As long as the callback doesn't look at
- * the connector we'll be fine though, so make sure that's the
- * case by setting all connector pointers to NULL.
- */
- state->connector_states[i]->connector = NULL;
- connector->funcs->atomic_destroy_state(NULL,
+ connector->funcs->atomic_destroy_state(connector,
state->connector_states[i]);
state->connectors[i] = NULL;
state->connector_states[i] = NULL;
+ drm_connector_unreference(connector);
}
for (i = 0; i < config->num_crtc; i++) {
@@ -261,6 +256,8 @@
int ret, index = drm_crtc_index(crtc);
struct drm_crtc_state *crtc_state;
+ WARN_ON(!state->acquire_ctx);
+
crtc_state = drm_atomic_get_existing_crtc_state(state, crtc);
if (crtc_state)
return crtc_state;
@@ -620,6 +617,8 @@
int ret, index = drm_plane_index(plane);
struct drm_plane_state *plane_state;
+ WARN_ON(!state->acquire_ctx);
+
plane_state = drm_atomic_get_existing_plane_state(state, plane);
if (plane_state)
return plane_state;
@@ -888,6 +887,8 @@
struct drm_mode_config *config = &connector->dev->mode_config;
struct drm_connector_state *connector_state;
+ WARN_ON(!state->acquire_ctx);
+
ret = drm_modeset_lock(&config->connection_mutex, state->acquire_ctx);
if (ret)
return ERR_PTR(ret);
@@ -924,6 +925,7 @@
if (!connector_state)
return ERR_PTR(-ENOMEM);
+ drm_connector_reference(connector);
state->connector_states[index] = connector_state;
state->connectors[index] = connector;
connector_state->state = state;
@@ -1158,12 +1160,18 @@
{
struct drm_crtc_state *crtc_state;
- if (conn_state->crtc && conn_state->crtc != crtc) {
+ if (conn_state->crtc == crtc)
+ return 0;
+
+ if (conn_state->crtc) {
crtc_state = drm_atomic_get_existing_crtc_state(conn_state->state,
conn_state->crtc);
crtc_state->connector_mask &=
~(1 << drm_connector_index(conn_state->connector));
+
+ drm_connector_unreference(conn_state->connector);
+ conn_state->crtc = NULL;
}
if (crtc) {
@@ -1173,16 +1181,16 @@
crtc_state->connector_mask |=
1 << drm_connector_index(conn_state->connector);
- }
- conn_state->crtc = crtc;
+ drm_connector_reference(conn_state->connector);
+ conn_state->crtc = crtc;
- if (crtc)
DRM_DEBUG_ATOMIC("Link connector state %p to [CRTC:%d:%s]\n",
conn_state, crtc->base.id, crtc->name);
- else
+ } else {
DRM_DEBUG_ATOMIC("Link connector state %p to [NOCRTC]\n",
conn_state);
+ }
return 0;
}
@@ -1388,7 +1396,7 @@
EXPORT_SYMBOL(drm_atomic_commit);
/**
- * drm_atomic_async_commit - atomic&async configuration commit
+ * drm_atomic_nonblocking_commit - atomic&nonblocking configuration commit
* @state: atomic configuration to check
*
* Note that this function can return -EDEADLK if the driver needed to acquire
@@ -1403,7 +1411,7 @@
* Returns:
* 0 on success, negative error code on failure.
*/
-int drm_atomic_async_commit(struct drm_atomic_state *state)
+int drm_atomic_nonblocking_commit(struct drm_atomic_state *state)
{
struct drm_mode_config *config = &state->dev->mode_config;
int ret;
@@ -1412,11 +1420,11 @@
if (ret)
return ret;
- DRM_DEBUG_ATOMIC("commiting %p asynchronously\n", state);
+ DRM_DEBUG_ATOMIC("commiting %p nonblocking\n", state);
return config->funcs->atomic_commit(state->dev, state, true);
}
-EXPORT_SYMBOL(drm_atomic_async_commit);
+EXPORT_SYMBOL(drm_atomic_nonblocking_commit);
/*
* The big monstor ioctl
@@ -1614,12 +1622,19 @@
}
obj = drm_mode_object_find(dev, obj_id, DRM_MODE_OBJECT_ANY);
- if (!obj || !obj->properties) {
+ if (!obj) {
+ ret = -ENOENT;
+ goto out;
+ }
+
+ if (!obj->properties) {
+ drm_mode_object_unreference(obj);
ret = -ENOENT;
goto out;
}
if (get_user(count_props, count_props_ptr + copied_objs)) {
+ drm_mode_object_unreference(obj);
ret = -EFAULT;
goto out;
}
@@ -1632,12 +1647,14 @@
struct drm_property *prop;
if (get_user(prop_id, props_ptr + copied_props)) {
+ drm_mode_object_unreference(obj);
ret = -EFAULT;
goto out;
}
prop = drm_property_find(dev, prop_id);
if (!prop) {
+ drm_mode_object_unreference(obj);
ret = -ENOENT;
goto out;
}
@@ -1645,13 +1662,16 @@
if (copy_from_user(&prop_value,
prop_values_ptr + copied_props,
sizeof(prop_value))) {
+ drm_mode_object_unreference(obj);
ret = -EFAULT;
goto out;
}
ret = atomic_set_prop(state, obj, prop, prop_value);
- if (ret)
+ if (ret) {
+ drm_mode_object_unreference(obj);
goto out;
+ }
copied_props++;
}
@@ -1662,6 +1682,7 @@
plane_mask |= (1 << drm_plane_index(plane));
plane->old_fb = plane->fb;
}
+ drm_mode_object_unreference(obj);
}
if (arg->flags & DRM_MODE_PAGE_FLIP_EVENT) {
@@ -1685,7 +1706,7 @@
*/
ret = drm_atomic_check_only(state);
} else if (arg->flags & DRM_MODE_ATOMIC_NONBLOCK) {
- ret = drm_atomic_async_commit(state);
+ ret = drm_atomic_nonblocking_commit(state);
} else {
ret = drm_atomic_commit(state);
}
diff --git a/drivers/gpu/drm/drm_atomic_helper.c b/drivers/gpu/drm/drm_atomic_helper.c
index 4befe25..ddfa0d1 100644
--- a/drivers/gpu/drm/drm_atomic_helper.c
+++ b/drivers/gpu/drm/drm_atomic_helper.c
@@ -384,8 +384,6 @@
*/
encoder = conn_state->best_encoder;
funcs = encoder->helper_private;
- if (!funcs)
- continue;
ret = drm_bridge_mode_fixup(encoder->bridge, &crtc_state->mode,
&crtc_state->adjusted_mode);
@@ -394,7 +392,7 @@
return -EINVAL;
}
- if (funcs->atomic_check) {
+ if (funcs && funcs->atomic_check) {
ret = funcs->atomic_check(encoder, crtc_state,
conn_state);
if (ret) {
@@ -402,7 +400,7 @@
encoder->base.id, encoder->name);
return ret;
}
- } else if (funcs->mode_fixup) {
+ } else if (funcs && funcs->mode_fixup) {
ret = funcs->mode_fixup(encoder, &crtc_state->mode,
&crtc_state->adjusted_mode);
if (!ret) {
@@ -707,12 +705,14 @@
drm_bridge_disable(encoder->bridge);
/* Right function depends upon target state. */
- if (connector->state->crtc && funcs->prepare)
- funcs->prepare(encoder);
- else if (funcs->disable)
- funcs->disable(encoder);
- else
- funcs->dpms(encoder, DRM_MODE_DPMS_OFF);
+ if (funcs) {
+ if (connector->state->crtc && funcs->prepare)
+ funcs->prepare(encoder);
+ else if (funcs->disable)
+ funcs->disable(encoder);
+ else if (funcs->dpms)
+ funcs->dpms(encoder, DRM_MODE_DPMS_OFF);
+ }
drm_bridge_post_disable(encoder->bridge);
}
@@ -873,7 +873,7 @@
* Each encoder has at most one connector (since we always steal
* it away), so we won't call mode_set hooks twice.
*/
- if (funcs->mode_set)
+ if (funcs && funcs->mode_set)
funcs->mode_set(encoder, mode, adjusted_mode);
drm_bridge_mode_set(encoder->bridge, mode, adjusted_mode);
@@ -974,17 +974,29 @@
*/
drm_bridge_pre_enable(encoder->bridge);
- if (funcs->enable)
- funcs->enable(encoder);
- else
- funcs->commit(encoder);
+ if (funcs) {
+ if (funcs->enable)
+ funcs->enable(encoder);
+ else if (funcs->commit)
+ funcs->commit(encoder);
+ }
drm_bridge_enable(encoder->bridge);
}
}
EXPORT_SYMBOL(drm_atomic_helper_commit_modeset_enables);
-static void wait_for_fences(struct drm_device *dev,
+/**
+ * drm_atomic_helper_wait_for_fences - wait for fences stashed in plane state
+ * @dev: DRM device
+ * @state: atomic state object with old state structures
+ *
+ * For implicit sync, driver should fish the exclusive fence out from the
+ * incoming fb's and stash it in the drm_plane_state. This is called after
+ * drm_atomic_helper_swap_state() so it uses the current plane state (and
+ * just uses the atomic state to find the changed planes)
+ */
+void drm_atomic_helper_wait_for_fences(struct drm_device *dev,
struct drm_atomic_state *state)
{
struct drm_plane *plane;
@@ -1002,6 +1014,7 @@
plane->state->fence = NULL;
}
}
+EXPORT_SYMBOL(drm_atomic_helper_wait_for_fences);
/**
* drm_atomic_helper_framebuffer_changed - check if framebuffer has changed
@@ -1092,6 +1105,8 @@
drm_crtc_vblank_count(crtc),
msecs_to_jiffies(50));
+ WARN(!ret, "[CRTC:%d] vblank wait timed out\n", crtc->base.id);
+
drm_crtc_vblank_put(crtc);
}
}
@@ -1101,13 +1116,13 @@
* drm_atomic_helper_commit - commit validated state object
* @dev: DRM device
* @state: the driver state object
- * @async: asynchronous commit
+ * @nonblocking: whether nonblocking behavior is requested.
*
* This function commits a with drm_atomic_helper_check() pre-validated state
* object. This can still fail when e.g. the framebuffer reservation fails. For
- * now this doesn't implement asynchronous commits.
+ * now this doesn't implement nonblocking commits.
*
- * Note that right now this function does not support async commits, and hence
+ * Note that right now this function does not support nonblocking commits, hence
* driver writers must implement their own version for now. Also note that the
* default ordering of how the various stages are called is to match the legacy
* modeset helper library closest. One peculiarity of that is that it doesn't
@@ -1128,11 +1143,11 @@
*/
int drm_atomic_helper_commit(struct drm_device *dev,
struct drm_atomic_state *state,
- bool async)
+ bool nonblock)
{
int ret;
- if (async)
+ if (nonblock)
return -EBUSY;
ret = drm_atomic_helper_prepare_planes(dev, state);
@@ -1163,7 +1178,7 @@
* current layout.
*/
- wait_for_fences(dev, state);
+ drm_atomic_helper_wait_for_fences(dev, state);
drm_atomic_helper_commit_modeset_disables(dev, state);
@@ -1182,20 +1197,20 @@
EXPORT_SYMBOL(drm_atomic_helper_commit);
/**
- * DOC: implementing async commit
+ * DOC: implementing nonblocking commit
*
- * For now the atomic helpers don't support async commit directly. If there is
- * real need it could be added though, using the dma-buf fence infrastructure
- * for generic synchronization with outstanding rendering.
+ * For now the atomic helpers don't support nonblocking commit directly. If
+ * there is real need it could be added though, using the dma-buf fence
+ * infrastructure for generic synchronization with outstanding rendering.
*
- * For now drivers have to implement async commit themselves, with the following
- * sequence being the recommended one:
+ * For now drivers have to implement nonblocking commit themselves, with the
+ * following sequence being the recommended one:
*
* 1. Run drm_atomic_helper_prepare_planes() first. This is the only function
* which commit needs to call which can fail, so we want to run it first and
* synchronously.
*
- * 2. Synchronize with any outstanding asynchronous commit worker threads which
+ * 2. Synchronize with any outstanding nonblocking commit worker threads which
* might be affected the new state update. This can be done by either cancelling
* or flushing the work items, depending upon whether the driver can deal with
* cancelled updates. Note that it is important to ensure that the framebuffer
@@ -1209,9 +1224,9 @@
* 3. The software state is updated synchronously with
* drm_atomic_helper_swap_state(). Doing this under the protection of all modeset
* locks means concurrent callers never see inconsistent state. And doing this
- * while it's guaranteed that no relevant async worker runs means that async
- * workers do not need grab any locks. Actually they must not grab locks, for
- * otherwise the work flushing will deadlock.
+ * while it's guaranteed that no relevant nonblocking worker runs means that
+ * nonblocking workers do not need grab any locks. Actually they must not grab
+ * locks, for otherwise the work flushing will deadlock.
*
* 4. Schedule a work item to do all subsequent steps, using the split-out
* commit helpers: a) pre-plane commit b) plane commit c) post-plane commit and
@@ -2358,11 +2373,11 @@
goto fail;
}
- ret = drm_atomic_async_commit(state);
+ ret = drm_atomic_nonblocking_commit(state);
if (ret != 0)
goto fail;
- /* Driver takes ownership of state on successful async commit. */
+ /* Driver takes ownership of state on successful commit. */
return 0;
fail:
if (ret == -EDEADLK)
@@ -2468,6 +2483,23 @@
EXPORT_SYMBOL(drm_atomic_helper_connector_dpms);
/**
+ * drm_atomic_helper_best_encoder - Helper for &drm_connector_helper_funcs
+ * ->best_encoder callback
+ * @connector: Connector control structure
+ *
+ * This is a &drm_connector_helper_funcs ->best_encoder callback helper for
+ * connectors that support exactly 1 encoder, statically determined at driver
+ * init time.
+ */
+struct drm_encoder *
+drm_atomic_helper_best_encoder(struct drm_connector *connector)
+{
+ WARN_ON(connector->encoder_ids[1]);
+ return drm_encoder_find(connector->dev, connector->encoder_ids[0]);
+}
+EXPORT_SYMBOL(drm_atomic_helper_best_encoder);
+
+/**
* DOC: atomic state reset and initialization
*
* Both the drm core and the atomic helpers assume that there is always the full
@@ -2497,12 +2529,9 @@
*/
void drm_atomic_helper_crtc_reset(struct drm_crtc *crtc)
{
- if (crtc->state) {
- drm_property_unreference_blob(crtc->state->mode_blob);
- drm_property_unreference_blob(crtc->state->degamma_lut);
- drm_property_unreference_blob(crtc->state->ctm);
- drm_property_unreference_blob(crtc->state->gamma_lut);
- }
+ if (crtc->state)
+ __drm_atomic_helper_crtc_destroy_state(crtc->state);
+
kfree(crtc->state);
crtc->state = kzalloc(sizeof(*crtc->state), GFP_KERNEL);
@@ -2566,15 +2595,13 @@
/**
* __drm_atomic_helper_crtc_destroy_state - release CRTC state
- * @crtc: CRTC object
* @state: CRTC state object to release
*
* Releases all resources stored in the CRTC state without actually freeing
* the memory of the CRTC state. This is useful for drivers that subclass the
* CRTC state.
*/
-void __drm_atomic_helper_crtc_destroy_state(struct drm_crtc *crtc,
- struct drm_crtc_state *state)
+void __drm_atomic_helper_crtc_destroy_state(struct drm_crtc_state *state)
{
drm_property_unreference_blob(state->mode_blob);
drm_property_unreference_blob(state->degamma_lut);
@@ -2594,7 +2621,7 @@
void drm_atomic_helper_crtc_destroy_state(struct drm_crtc *crtc,
struct drm_crtc_state *state)
{
- __drm_atomic_helper_crtc_destroy_state(crtc, state);
+ __drm_atomic_helper_crtc_destroy_state(state);
kfree(state);
}
EXPORT_SYMBOL(drm_atomic_helper_crtc_destroy_state);
@@ -2608,8 +2635,8 @@
*/
void drm_atomic_helper_plane_reset(struct drm_plane *plane)
{
- if (plane->state && plane->state->fb)
- drm_framebuffer_unreference(plane->state->fb);
+ if (plane->state)
+ __drm_atomic_helper_plane_destroy_state(plane->state);
kfree(plane->state);
plane->state = kzalloc(sizeof(*plane->state), GFP_KERNEL);
@@ -2664,15 +2691,13 @@
/**
* __drm_atomic_helper_plane_destroy_state - release plane state
- * @plane: plane object
* @state: plane state object to release
*
* Releases all resources stored in the plane state without actually freeing
* the memory of the plane state. This is useful for drivers that subclass the
* plane state.
*/
-void __drm_atomic_helper_plane_destroy_state(struct drm_plane *plane,
- struct drm_plane_state *state)
+void __drm_atomic_helper_plane_destroy_state(struct drm_plane_state *state)
{
if (state->fb)
drm_framebuffer_unreference(state->fb);
@@ -2690,7 +2715,7 @@
void drm_atomic_helper_plane_destroy_state(struct drm_plane *plane,
struct drm_plane_state *state)
{
- __drm_atomic_helper_plane_destroy_state(plane, state);
+ __drm_atomic_helper_plane_destroy_state(state);
kfree(state);
}
EXPORT_SYMBOL(drm_atomic_helper_plane_destroy_state);
@@ -2730,6 +2755,9 @@
struct drm_connector_state *conn_state =
kzalloc(sizeof(*conn_state), GFP_KERNEL);
+ if (connector->state)
+ __drm_atomic_helper_connector_destroy_state(connector->state);
+
kfree(connector->state);
__drm_atomic_helper_connector_reset(connector, conn_state);
}
@@ -2748,6 +2776,8 @@
struct drm_connector_state *state)
{
memcpy(state, connector->state, sizeof(*state));
+ if (state->crtc)
+ drm_connector_reference(connector);
}
EXPORT_SYMBOL(__drm_atomic_helper_connector_duplicate_state);
@@ -2859,7 +2889,6 @@
/**
* __drm_atomic_helper_connector_destroy_state - release connector state
- * @connector: connector object
* @state: connector state object to release
*
* Releases all resources stored in the connector state without actually
@@ -2867,14 +2896,15 @@
* subclass the connector state.
*/
void
-__drm_atomic_helper_connector_destroy_state(struct drm_connector *connector,
- struct drm_connector_state *state)
+__drm_atomic_helper_connector_destroy_state(struct drm_connector_state *state)
{
/*
* This is currently a placeholder so that drivers that subclass the
* state will automatically do the right thing if code is ever added
* to this function.
*/
+ if (state->crtc)
+ drm_connector_unreference(state->connector);
}
EXPORT_SYMBOL(__drm_atomic_helper_connector_destroy_state);
@@ -2889,7 +2919,7 @@
void drm_atomic_helper_connector_destroy_state(struct drm_connector *connector,
struct drm_connector_state *state)
{
- __drm_atomic_helper_connector_destroy_state(connector, state);
+ __drm_atomic_helper_connector_destroy_state(state);
kfree(state);
}
EXPORT_SYMBOL(drm_atomic_helper_connector_destroy_state);
diff --git a/drivers/gpu/drm/drm_bufs.c b/drivers/gpu/drm/drm_bufs.c
index f1a204d..9b34158c 100644
--- a/drivers/gpu/drm/drm_bufs.c
+++ b/drivers/gpu/drm/drm_bufs.c
@@ -396,6 +396,10 @@
if (!(capable(CAP_SYS_ADMIN) || map->type == _DRM_AGP || map->type == _DRM_SHM))
return -EPERM;
+ if (!drm_core_check_feature(dev, DRIVER_KMS_LEGACY_CONTEXT) &&
+ drm_core_check_feature(dev, DRIVER_MODESET))
+ return -EINVAL;
+
err = drm_addmap_core(dev, map->offset, map->size, map->type,
map->flags, &maplist);
@@ -416,6 +420,62 @@
return 0;
}
+/*
+ * Get a mapping information.
+ *
+ * \param inode device inode.
+ * \param file_priv DRM file private.
+ * \param cmd command.
+ * \param arg user argument, pointing to a drm_map structure.
+ *
+ * \return zero on success or a negative number on failure.
+ *
+ * Searches for the mapping with the specified offset and copies its information
+ * into userspace
+ */
+int drm_legacy_getmap_ioctl(struct drm_device *dev, void *data,
+ struct drm_file *file_priv)
+{
+ struct drm_map *map = data;
+ struct drm_map_list *r_list = NULL;
+ struct list_head *list;
+ int idx;
+ int i;
+
+ if (!drm_core_check_feature(dev, DRIVER_KMS_LEGACY_CONTEXT) &&
+ drm_core_check_feature(dev, DRIVER_MODESET))
+ return -EINVAL;
+
+ idx = map->offset;
+ if (idx < 0)
+ return -EINVAL;
+
+ i = 0;
+ mutex_lock(&dev->struct_mutex);
+ list_for_each(list, &dev->maplist) {
+ if (i == idx) {
+ r_list = list_entry(list, struct drm_map_list, head);
+ break;
+ }
+ i++;
+ }
+ if (!r_list || !r_list->map) {
+ mutex_unlock(&dev->struct_mutex);
+ return -EINVAL;
+ }
+
+ map->offset = r_list->map->offset;
+ map->size = r_list->map->size;
+ map->type = r_list->map->type;
+ map->flags = r_list->map->flags;
+ map->handle = (void *)(unsigned long) r_list->user_token;
+ map->mtrr = arch_phys_wc_index(r_list->map->mtrr);
+
+ mutex_unlock(&dev->struct_mutex);
+
+ return 0;
+}
+
/**
* Remove a map private from list and deallocate resources if the mapping
* isn't in use.
@@ -482,18 +542,35 @@
}
EXPORT_SYMBOL(drm_legacy_rmmap_locked);
-int drm_legacy_rmmap(struct drm_device *dev, struct drm_local_map *map)
+void drm_legacy_rmmap(struct drm_device *dev, struct drm_local_map *map)
{
- int ret;
+ if (!drm_core_check_feature(dev, DRIVER_KMS_LEGACY_CONTEXT) &&
+ drm_core_check_feature(dev, DRIVER_MODESET))
+ return;
mutex_lock(&dev->struct_mutex);
- ret = drm_legacy_rmmap_locked(dev, map);
+ drm_legacy_rmmap_locked(dev, map);
mutex_unlock(&dev->struct_mutex);
-
- return ret;
}
EXPORT_SYMBOL(drm_legacy_rmmap);
+void drm_legacy_master_rmmaps(struct drm_device *dev, struct drm_master *master)
+{
+ struct drm_map_list *r_list, *list_temp;
+
+ if (drm_core_check_feature(dev, DRIVER_MODESET))
+ return;
+
+ mutex_lock(&dev->struct_mutex);
+ list_for_each_entry_safe(r_list, list_temp, &dev->maplist, head) {
+ if (r_list->master == master) {
+ drm_legacy_rmmap_locked(dev, r_list->map);
+ r_list = NULL;
+ }
+ }
+ mutex_unlock(&dev->struct_mutex);
+}
+
/* The rmmap ioctl appears to be unnecessary. All mappings are torn down on
* the last close of the device, and this is necessary for cleanup when things
* exit uncleanly. Therefore, having userland manually remove mappings seems
@@ -517,6 +594,10 @@
struct drm_map_list *r_list;
int ret;
+ if (!drm_core_check_feature(dev, DRIVER_KMS_LEGACY_CONTEXT) &&
+ drm_core_check_feature(dev, DRIVER_MODESET))
+ return -EINVAL;
+
mutex_lock(&dev->struct_mutex);
list_for_each_entry(r_list, &dev->maplist, head) {
if (r_list->map &&
diff --git a/drivers/gpu/drm/drm_crtc.c b/drivers/gpu/drm/drm_crtc.c
index e08f962..d2a6d95 100644
--- a/drivers/gpu/drm/drm_crtc.c
+++ b/drivers/gpu/drm/drm_crtc.c
@@ -168,6 +168,7 @@
{ DRM_MODE_CONNECTOR_eDP, "eDP" },
{ DRM_MODE_CONNECTOR_VIRTUAL, "Virtual" },
{ DRM_MODE_CONNECTOR_DSI, "DSI" },
+ { DRM_MODE_CONNECTOR_DPI, "DPI" },
};
static const struct drm_prop_enum_list drm_encoder_enum_list[] = {
@@ -179,6 +180,7 @@
{ DRM_MODE_ENCODER_VIRTUAL, "Virtual" },
{ DRM_MODE_ENCODER_DSI, "DSI" },
{ DRM_MODE_ENCODER_DPMST, "DP MST" },
+ { DRM_MODE_ENCODER_DPI, "DPI" },
};
static const struct drm_prop_enum_list drm_subpixel_enum_list[] = {
@@ -275,7 +277,8 @@
static int drm_mode_object_get_reg(struct drm_device *dev,
struct drm_mode_object *obj,
uint32_t obj_type,
- bool register_obj)
+ bool register_obj,
+ void (*obj_free_cb)(struct kref *kref))
{
int ret;
@@ -288,6 +291,10 @@
*/
obj->id = ret;
obj->type = obj_type;
+ if (obj_free_cb) {
+ obj->free_cb = obj_free_cb;
+ kref_init(&obj->refcount);
+ }
}
mutex_unlock(&dev->mode_config.idr_mutex);
@@ -311,7 +318,7 @@
int drm_mode_object_get(struct drm_device *dev,
struct drm_mode_object *obj, uint32_t obj_type)
{
- return drm_mode_object_get_reg(dev, obj, obj_type, true);
+ return drm_mode_object_get_reg(dev, obj, obj_type, true, NULL);
}
static void drm_mode_object_register(struct drm_device *dev,
@@ -323,19 +330,24 @@
}
/**
- * drm_mode_object_put - free a modeset identifer
+ * drm_mode_object_unregister - free a modeset identifer
* @dev: DRM device
* @object: object to free
*
- * Free @id from @dev's unique identifier pool. Note that despite the _get
- * postfix modeset identifiers are _not_ reference counted. Hence don't use this
+ * Free @id from @dev's unique identifier pool.
+ * This function can be called multiple times, and guards against
+ * multiple removals.
+ * These modeset identifiers are _not_ reference counted. Hence don't use this
* for reference counted modeset objects like framebuffers.
*/
-void drm_mode_object_put(struct drm_device *dev,
+void drm_mode_object_unregister(struct drm_device *dev,
struct drm_mode_object *object)
{
mutex_lock(&dev->mode_config.idr_mutex);
- idr_remove(&dev->mode_config.crtc_idr, object->id);
+ if (object->id) {
+ idr_remove(&dev->mode_config.crtc_idr, object->id);
+ object->id = 0;
+ }
mutex_unlock(&dev->mode_config.idr_mutex);
}
@@ -350,11 +362,11 @@
obj = NULL;
if (obj && obj->id != id)
obj = NULL;
- /* don't leak out unref'd fb's */
- if (obj &&
- (obj->type == DRM_MODE_OBJECT_FB ||
- obj->type == DRM_MODE_OBJECT_BLOB))
- obj = NULL;
+
+ if (obj && obj->free_cb) {
+ if (!kref_get_unless_zero(&obj->refcount))
+ obj = NULL;
+ }
mutex_unlock(&dev->mode_config.idr_mutex);
return obj;
@@ -366,25 +378,70 @@
* @id: id of the mode object
* @type: type of the mode object
*
- * Note that framebuffers cannot be looked up with this functions - since those
- * are reference counted, they need special treatment. Even with
- * DRM_MODE_OBJECT_ANY (although that will simply return NULL
- * rather than WARN_ON()).
+ * This function is used to look up a modeset object. It will acquire a
+ * reference for reference counted objects. This reference must be dropped again
+ * by callind drm_mode_object_unreference().
*/
struct drm_mode_object *drm_mode_object_find(struct drm_device *dev,
uint32_t id, uint32_t type)
{
struct drm_mode_object *obj = NULL;
- /* Framebuffers are reference counted and need their own lookup
- * function.*/
- WARN_ON(type == DRM_MODE_OBJECT_FB || type == DRM_MODE_OBJECT_BLOB);
obj = _object_find(dev, id, type);
return obj;
}
EXPORT_SYMBOL(drm_mode_object_find);
/**
+ * drm_mode_object_unreference - decr the object refcnt
+ * @obj: mode_object
+ *
+ * This functions decrements the object's refcount if it is a refcounted modeset
+ * object. It is a no-op on any other object. This is used to drop references
+ * acquired with drm_mode_object_reference().
+ */
+void drm_mode_object_unreference(struct drm_mode_object *obj)
+{
+ if (obj->free_cb) {
+ DRM_DEBUG("OBJ ID: %d (%d)\n", obj->id, atomic_read(&obj->refcount.refcount));
+ kref_put(&obj->refcount, obj->free_cb);
+ }
+}
+EXPORT_SYMBOL(drm_mode_object_unreference);
+
+/**
+ * drm_mode_object_reference - incr the object refcnt
+ * @obj: mode_object
+ *
+ * This functions increments the object's refcount if it is a refcounted modeset
+ * object. It is a no-op on any other object. References should be dropped again
+ * by calling drm_mode_object_unreference().
+ */
+void drm_mode_object_reference(struct drm_mode_object *obj)
+{
+ if (obj->free_cb) {
+ DRM_DEBUG("OBJ ID: %d (%d)\n", obj->id, atomic_read(&obj->refcount.refcount));
+ kref_get(&obj->refcount);
+ }
+}
+EXPORT_SYMBOL(drm_mode_object_reference);
+
+static void drm_framebuffer_free(struct kref *kref)
+{
+ struct drm_framebuffer *fb =
+ container_of(kref, struct drm_framebuffer, base.refcount);
+ struct drm_device *dev = fb->dev;
+
+ /*
+ * The lookup idr holds a weak reference, which has not necessarily been
+ * removed at this point. Check for that.
+ */
+ drm_mode_object_unregister(dev, &fb->base);
+
+ fb->funcs->destroy(fb);
+}
+
+/**
* drm_framebuffer_init - initialize a framebuffer
* @dev: DRM device
* @fb: framebuffer to be initialized
@@ -407,71 +464,26 @@
{
int ret;
- mutex_lock(&dev->mode_config.fb_lock);
- kref_init(&fb->refcount);
INIT_LIST_HEAD(&fb->filp_head);
fb->dev = dev;
fb->funcs = funcs;
- ret = drm_mode_object_get(dev, &fb->base, DRM_MODE_OBJECT_FB);
+ ret = drm_mode_object_get_reg(dev, &fb->base, DRM_MODE_OBJECT_FB,
+ false, drm_framebuffer_free);
if (ret)
goto out;
+ mutex_lock(&dev->mode_config.fb_lock);
dev->mode_config.num_fb++;
list_add(&fb->head, &dev->mode_config.fb_list);
-out:
mutex_unlock(&dev->mode_config.fb_lock);
+ drm_mode_object_register(dev, &fb->base);
+out:
return ret;
}
EXPORT_SYMBOL(drm_framebuffer_init);
-/* dev->mode_config.fb_lock must be held! */
-static void __drm_framebuffer_unregister(struct drm_device *dev,
- struct drm_framebuffer *fb)
-{
- drm_mode_object_put(dev, &fb->base);
-
- fb->base.id = 0;
-}
-
-static void drm_framebuffer_free(struct kref *kref)
-{
- struct drm_framebuffer *fb =
- container_of(kref, struct drm_framebuffer, refcount);
- struct drm_device *dev = fb->dev;
-
- /*
- * The lookup idr holds a weak reference, which has not necessarily been
- * removed at this point. Check for that.
- */
- mutex_lock(&dev->mode_config.fb_lock);
- if (fb->base.id) {
- /* Mark fb as reaped and drop idr ref. */
- __drm_framebuffer_unregister(dev, fb);
- }
- mutex_unlock(&dev->mode_config.fb_lock);
-
- fb->funcs->destroy(fb);
-}
-
-static struct drm_framebuffer *__drm_framebuffer_lookup(struct drm_device *dev,
- uint32_t id)
-{
- struct drm_mode_object *obj = NULL;
- struct drm_framebuffer *fb;
-
- mutex_lock(&dev->mode_config.idr_mutex);
- obj = idr_find(&dev->mode_config.crtc_idr, id);
- if (!obj || (obj->type != DRM_MODE_OBJECT_FB) || (obj->id != id))
- fb = NULL;
- else
- fb = obj_to_fb(obj);
- mutex_unlock(&dev->mode_config.idr_mutex);
-
- return fb;
-}
-
/**
* drm_framebuffer_lookup - look up a drm framebuffer and grab a reference
* @dev: drm device
@@ -484,47 +496,17 @@
struct drm_framebuffer *drm_framebuffer_lookup(struct drm_device *dev,
uint32_t id)
{
- struct drm_framebuffer *fb;
+ struct drm_mode_object *obj;
+ struct drm_framebuffer *fb = NULL;
- mutex_lock(&dev->mode_config.fb_lock);
- fb = __drm_framebuffer_lookup(dev, id);
- if (fb) {
- if (!kref_get_unless_zero(&fb->refcount))
- fb = NULL;
- }
- mutex_unlock(&dev->mode_config.fb_lock);
-
+ obj = _object_find(dev, id, DRM_MODE_OBJECT_FB);
+ if (obj)
+ fb = obj_to_fb(obj);
return fb;
}
EXPORT_SYMBOL(drm_framebuffer_lookup);
/**
- * drm_framebuffer_unreference - unref a framebuffer
- * @fb: framebuffer to unref
- *
- * This functions decrements the fb's refcount and frees it if it drops to zero.
- */
-void drm_framebuffer_unreference(struct drm_framebuffer *fb)
-{
- DRM_DEBUG("%p: FB ID: %d (%d)\n", fb, fb->base.id, atomic_read(&fb->refcount.refcount));
- kref_put(&fb->refcount, drm_framebuffer_free);
-}
-EXPORT_SYMBOL(drm_framebuffer_unreference);
-
-/**
- * drm_framebuffer_reference - incr the fb refcnt
- * @fb: framebuffer
- *
- * This functions increments the fb's refcount.
- */
-void drm_framebuffer_reference(struct drm_framebuffer *fb)
-{
- DRM_DEBUG("%p: FB ID: %d (%d)\n", fb, fb->base.id, atomic_read(&fb->refcount.refcount));
- kref_get(&fb->refcount);
-}
-EXPORT_SYMBOL(drm_framebuffer_reference);
-
-/**
* drm_framebuffer_unregister_private - unregister a private fb from the lookup idr
* @fb: fb to unregister
*
@@ -542,10 +524,8 @@
dev = fb->dev;
- mutex_lock(&dev->mode_config.fb_lock);
/* Mark fb as reaped and drop idr ref. */
- __drm_framebuffer_unregister(dev, fb);
- mutex_unlock(&dev->mode_config.fb_lock);
+ drm_mode_object_unregister(dev, &fb->base);
}
EXPORT_SYMBOL(drm_framebuffer_unregister_private);
@@ -619,7 +599,7 @@
* in-use fb with fb-id == 0. Userspace is allowed to shoot its own foot
* in this manner.
*/
- if (atomic_read(&fb->refcount.refcount) > 1) {
+ if (drm_framebuffer_read_refcount(fb) > 1) {
drm_modeset_lock_all(dev);
/* remove from any CRTC */
drm_for_each_crtc(crtc, dev) {
@@ -705,7 +685,7 @@
drm_num_crtcs(dev));
}
if (!crtc->name) {
- drm_mode_object_put(dev, &crtc->base);
+ drm_mode_object_unregister(dev, &crtc->base);
return -ENOMEM;
}
@@ -747,7 +727,7 @@
drm_modeset_lock_fini(&crtc->mutex);
- drm_mode_object_put(dev, &crtc->base);
+ drm_mode_object_unregister(dev, &crtc->base);
list_del(&crtc->head);
dev->mode_config.num_crtc--;
@@ -884,6 +864,16 @@
mode->interlace ? " interlaced" : "");
}
+static void drm_connector_free(struct kref *kref)
+{
+ struct drm_connector *connector =
+ container_of(kref, struct drm_connector, base.refcount);
+ struct drm_device *dev = connector->dev;
+
+ drm_mode_object_unregister(dev, &connector->base);
+ connector->funcs->destroy(connector);
+}
+
/**
* drm_connector_init - Init a preallocated connector
* @dev: DRM device
@@ -909,7 +899,9 @@
drm_modeset_lock_all(dev);
- ret = drm_mode_object_get_reg(dev, &connector->base, DRM_MODE_OBJECT_CONNECTOR, false);
+ ret = drm_mode_object_get_reg(dev, &connector->base,
+ DRM_MODE_OBJECT_CONNECTOR,
+ false, drm_connector_free);
if (ret)
goto out_unlock;
@@ -972,7 +964,7 @@
ida_remove(&config->connector_ida, connector->connector_id);
out_put:
if (ret)
- drm_mode_object_put(dev, &connector->base);
+ drm_mode_object_unregister(dev, &connector->base);
out_unlock:
drm_modeset_unlock_all(dev);
@@ -1010,7 +1002,7 @@
connector->connector_id);
kfree(connector->display_info.bus_formats);
- drm_mode_object_put(dev, &connector->base);
+ drm_mode_object_unregister(dev, &connector->base);
kfree(connector->name);
connector->name = NULL;
list_del(&connector->head);
@@ -1038,8 +1030,6 @@
{
int ret;
- drm_mode_object_register(connector->dev, &connector->base);
-
ret = drm_sysfs_connector_add(connector);
if (ret)
return ret;
@@ -1050,6 +1040,8 @@
return ret;
}
+ drm_mode_object_register(connector->dev, &connector->base);
+
return 0;
}
EXPORT_SYMBOL(drm_connector_register);
@@ -1067,25 +1059,65 @@
}
EXPORT_SYMBOL(drm_connector_unregister);
-
/**
- * drm_connector_unplug_all - unregister connector userspace interfaces
+ * drm_connector_register_all - register all connectors
* @dev: drm device
*
- * This function unregisters all connector userspace interfaces in sysfs. Should
- * be call when the device is disconnected, e.g. from an usb driver's
- * ->disconnect callback.
+ * This function registers all connectors in sysfs and other places so that
+ * userspace can start to access them. Drivers can call it after calling
+ * drm_dev_register() to complete the device registration, if they don't call
+ * drm_connector_register() on each connector individually.
+ *
+ * When a device is unplugged and should be removed from userspace access,
+ * call drm_connector_unregister_all(), which is the inverse of this
+ * function.
+ *
+ * Returns:
+ * Zero on success, error code on failure.
*/
-void drm_connector_unplug_all(struct drm_device *dev)
+int drm_connector_register_all(struct drm_device *dev)
+{
+ struct drm_connector *connector;
+ int ret;
+
+ mutex_lock(&dev->mode_config.mutex);
+
+ drm_for_each_connector(connector, dev) {
+ ret = drm_connector_register(connector);
+ if (ret)
+ goto err;
+ }
+
+ mutex_unlock(&dev->mode_config.mutex);
+
+ return 0;
+
+err:
+ mutex_unlock(&dev->mode_config.mutex);
+ drm_connector_unregister_all(dev);
+ return ret;
+}
+EXPORT_SYMBOL(drm_connector_register_all);
+
+/**
+ * drm_connector_unregister_all - unregister connector userspace interfaces
+ * @dev: drm device
+ *
+ * This functions unregisters all connectors from sysfs and other places so
+ * that userspace can no longer access them. Drivers should call this as the
+ * first step tearing down the device instace, or when the underlying
+ * physical device disappeared (e.g. USB unplug), right before calling
+ * drm_dev_unregister().
+ */
+void drm_connector_unregister_all(struct drm_device *dev)
{
struct drm_connector *connector;
/* FIXME: taking the mode config mutex ends up in a clash with sysfs */
list_for_each_entry(connector, &dev->mode_config.connector_list, head)
drm_connector_unregister(connector);
-
}
-EXPORT_SYMBOL(drm_connector_unplug_all);
+EXPORT_SYMBOL(drm_connector_unregister_all);
/**
* drm_encoder_init - Init a preallocated encoder
@@ -1138,7 +1170,7 @@
out_put:
if (ret)
- drm_mode_object_put(dev, &encoder->base);
+ drm_mode_object_unregister(dev, &encoder->base);
out_unlock:
drm_modeset_unlock_all(dev);
@@ -1181,7 +1213,7 @@
struct drm_device *dev = encoder->dev;
drm_modeset_lock_all(dev);
- drm_mode_object_put(dev, &encoder->base);
+ drm_mode_object_unregister(dev, &encoder->base);
kfree(encoder->name);
list_del(&encoder->head);
dev->mode_config.num_encoder--;
@@ -1242,7 +1274,7 @@
GFP_KERNEL);
if (!plane->format_types) {
DRM_DEBUG_KMS("out of memory when allocating plane\n");
- drm_mode_object_put(dev, &plane->base);
+ drm_mode_object_unregister(dev, &plane->base);
return -ENOMEM;
}
@@ -1258,7 +1290,7 @@
}
if (!plane->name) {
kfree(plane->format_types);
- drm_mode_object_put(dev, &plane->base);
+ drm_mode_object_unregister(dev, &plane->base);
return -ENOMEM;
}
@@ -1338,7 +1370,7 @@
drm_modeset_lock_all(dev);
kfree(plane->format_types);
- drm_mode_object_put(dev, &plane->base);
+ drm_mode_object_unregister(dev, &plane->base);
BUG_ON(list_empty(&plane->head));
@@ -1918,8 +1950,6 @@
copied = 0;
crtc_id = (uint32_t __user *)(unsigned long)card_res->crtc_id_ptr;
drm_for_each_crtc(crtc, dev) {
- DRM_DEBUG_KMS("[CRTC:%d:%s]\n",
- crtc->base.id, crtc->name);
if (put_user(crtc->base.id, crtc_id + copied)) {
ret = -EFAULT;
goto out;
@@ -1934,8 +1964,6 @@
copied = 0;
encoder_id = (uint32_t __user *)(unsigned long)card_res->encoder_id_ptr;
drm_for_each_encoder(encoder, dev) {
- DRM_DEBUG_KMS("[ENCODER:%d:%s]\n", encoder->base.id,
- encoder->name);
if (put_user(encoder->base.id, encoder_id +
copied)) {
ret = -EFAULT;
@@ -1951,9 +1979,6 @@
copied = 0;
connector_id = (uint32_t __user *)(unsigned long)card_res->connector_id_ptr;
drm_for_each_connector(connector, dev) {
- DRM_DEBUG_KMS("[CONNECTOR:%d:%s]\n",
- connector->base.id,
- connector->name);
if (put_user(connector->base.id,
connector_id + copied)) {
ret = -EFAULT;
@@ -1964,9 +1989,6 @@
}
card_res->count_connectors = connector_count;
- DRM_DEBUG_KMS("CRTC[%d] CONNECTORS[%d] ENCODERS[%d]\n", card_res->count_crtcs,
- card_res->count_connectors, card_res->count_encoders);
-
out:
mutex_unlock(&dev->mode_config.mutex);
return ret;
@@ -2125,11 +2147,9 @@
memset(&u_mode, 0, sizeof(struct drm_mode_modeinfo));
- DRM_DEBUG_KMS("[CONNECTOR:%d:?]\n", out_resp->connector_id);
-
mutex_lock(&dev->mode_config.mutex);
- connector = drm_connector_find(dev, out_resp->connector_id);
+ connector = drm_connector_lookup(dev, out_resp->connector_id);
if (!connector) {
ret = -ENOENT;
goto out_unlock;
@@ -2213,6 +2233,7 @@
out:
drm_modeset_unlock(&dev->mode_config.connection_mutex);
+ drm_connector_unreference(connector);
out_unlock:
mutex_unlock(&dev->mode_config.mutex);
@@ -2857,13 +2878,14 @@
}
for (i = 0; i < crtc_req->count_connectors; i++) {
+ connector_set[i] = NULL;
set_connectors_ptr = (uint32_t __user *)(unsigned long)crtc_req->set_connectors_ptr;
if (get_user(out_id, &set_connectors_ptr[i])) {
ret = -EFAULT;
goto out;
}
- connector = drm_connector_find(dev, out_id);
+ connector = drm_connector_lookup(dev, out_id);
if (!connector) {
DRM_DEBUG_KMS("Connector id %d unknown\n",
out_id);
@@ -2891,6 +2913,12 @@
if (fb)
drm_framebuffer_unreference(fb);
+ if (connector_set) {
+ for (i = 0; i < crtc_req->count_connectors; i++) {
+ if (connector_set[i])
+ drm_connector_unreference(connector_set[i]);
+ }
+ }
kfree(connector_set);
drm_mode_destroy(dev, mode);
drm_modeset_unlock_all(dev);
@@ -3423,17 +3451,35 @@
if (IS_ERR(fb))
return PTR_ERR(fb);
- /* Transfer ownership to the filp for reaping on close */
-
DRM_DEBUG_KMS("[FB:%d]\n", fb->base.id);
- mutex_lock(&file_priv->fbs_lock);
r->fb_id = fb->base.id;
+
+ /* Transfer ownership to the filp for reaping on close */
+ mutex_lock(&file_priv->fbs_lock);
list_add(&fb->filp_head, &file_priv->fbs);
mutex_unlock(&file_priv->fbs_lock);
return 0;
}
+struct drm_mode_rmfb_work {
+ struct work_struct work;
+ struct list_head fbs;
+};
+
+static void drm_mode_rmfb_work_fn(struct work_struct *w)
+{
+ struct drm_mode_rmfb_work *arg = container_of(w, typeof(*arg), work);
+
+ while (!list_empty(&arg->fbs)) {
+ struct drm_framebuffer *fb =
+ list_first_entry(&arg->fbs, typeof(*fb), filp_head);
+
+ list_del_init(&fb->filp_head);
+ drm_framebuffer_remove(fb);
+ }
+}
+
/**
* drm_mode_rmfb - remove an FB from the configuration
* @dev: drm device for the ioctl
@@ -3458,30 +3504,49 @@
if (!drm_core_check_feature(dev, DRIVER_MODESET))
return -EINVAL;
- mutex_lock(&file_priv->fbs_lock);
- mutex_lock(&dev->mode_config.fb_lock);
- fb = __drm_framebuffer_lookup(dev, *id);
+ fb = drm_framebuffer_lookup(dev, *id);
if (!fb)
- goto fail_lookup;
+ return -ENOENT;
+ mutex_lock(&file_priv->fbs_lock);
list_for_each_entry(fbl, &file_priv->fbs, filp_head)
if (fb == fbl)
found = 1;
- if (!found)
- goto fail_lookup;
+ if (!found) {
+ mutex_unlock(&file_priv->fbs_lock);
+ goto fail_unref;
+ }
list_del_init(&fb->filp_head);
- mutex_unlock(&dev->mode_config.fb_lock);
mutex_unlock(&file_priv->fbs_lock);
+ /* drop the reference we picked up in framebuffer lookup */
drm_framebuffer_unreference(fb);
+ /*
+ * we now own the reference that was stored in the fbs list
+ *
+ * drm_framebuffer_remove may fail with -EINTR on pending signals,
+ * so run this in a separate stack as there's no way to correctly
+ * handle this after the fb is already removed from the lookup table.
+ */
+ if (drm_framebuffer_read_refcount(fb) > 1) {
+ struct drm_mode_rmfb_work arg;
+
+ INIT_WORK_ONSTACK(&arg.work, drm_mode_rmfb_work_fn);
+ INIT_LIST_HEAD(&arg.fbs);
+ list_add_tail(&fb->filp_head, &arg.fbs);
+
+ schedule_work(&arg.work);
+ flush_work(&arg.work);
+ destroy_work_on_stack(&arg.work);
+ } else
+ drm_framebuffer_unreference(fb);
+
return 0;
-fail_lookup:
- mutex_unlock(&dev->mode_config.fb_lock);
- mutex_unlock(&file_priv->fbs_lock);
-
+fail_unref:
+ drm_framebuffer_unreference(fb);
return -ENOENT;
}
@@ -3627,7 +3692,6 @@
return ret;
}
-
/**
* drm_fb_release - remove and free the FBs on this file
* @priv: drm file for the ioctl
@@ -3642,6 +3706,9 @@
void drm_fb_release(struct drm_file *priv)
{
struct drm_framebuffer *fb, *tfb;
+ struct drm_mode_rmfb_work arg;
+
+ INIT_LIST_HEAD(&arg.fbs);
/*
* When the file gets released that means no one else can access the fb
@@ -3654,10 +3721,22 @@
* at it any more.
*/
list_for_each_entry_safe(fb, tfb, &priv->fbs, filp_head) {
- list_del_init(&fb->filp_head);
+ if (drm_framebuffer_read_refcount(fb) > 1) {
+ list_move_tail(&fb->filp_head, &arg.fbs);
+ } else {
+ list_del_init(&fb->filp_head);
- /* This drops the fpriv->fbs reference. */
- drm_framebuffer_unreference(fb);
+ /* This drops the fpriv->fbs reference. */
+ drm_framebuffer_unreference(fb);
+ }
+ }
+
+ if (!list_empty(&arg.fbs)) {
+ INIT_WORK_ONSTACK(&arg.work, drm_mode_rmfb_work_fn);
+
+ schedule_work(&arg.work);
+ flush_work(&arg.work);
+ destroy_work_on_stack(&arg.work);
}
}
@@ -4029,7 +4108,7 @@
if (property->num_values)
kfree(property->values);
- drm_mode_object_put(dev, &property->base);
+ drm_mode_object_unregister(dev, &property->base);
list_del(&property->head);
kfree(property);
}
@@ -4234,6 +4313,20 @@
return ret;
}
+static void drm_property_free_blob(struct kref *kref)
+{
+ struct drm_property_blob *blob =
+ container_of(kref, struct drm_property_blob, base.refcount);
+
+ mutex_lock(&blob->dev->mode_config.blob_lock);
+ list_del(&blob->head_global);
+ mutex_unlock(&blob->dev->mode_config.blob_lock);
+
+ drm_mode_object_unregister(blob->dev, &blob->base);
+
+ kfree(blob);
+}
+
/**
* drm_property_create_blob - Create new blob property
*
@@ -4271,20 +4364,16 @@
if (data)
memcpy(blob->data, data, length);
- mutex_lock(&dev->mode_config.blob_lock);
-
- ret = drm_mode_object_get(dev, &blob->base, DRM_MODE_OBJECT_BLOB);
+ ret = drm_mode_object_get_reg(dev, &blob->base, DRM_MODE_OBJECT_BLOB,
+ true, drm_property_free_blob);
if (ret) {
kfree(blob);
- mutex_unlock(&dev->mode_config.blob_lock);
return ERR_PTR(-EINVAL);
}
- kref_init(&blob->refcount);
-
+ mutex_lock(&dev->mode_config.blob_lock);
list_add_tail(&blob->head_global,
&dev->mode_config.property_blob_list);
-
mutex_unlock(&dev->mode_config.blob_lock);
return blob;
@@ -4292,27 +4381,6 @@
EXPORT_SYMBOL(drm_property_create_blob);
/**
- * drm_property_free_blob - Blob property destructor
- *
- * Internal free function for blob properties; must not be used directly.
- *
- * @kref: Reference
- */
-static void drm_property_free_blob(struct kref *kref)
-{
- struct drm_property_blob *blob =
- container_of(kref, struct drm_property_blob, refcount);
-
- WARN_ON(!mutex_is_locked(&blob->dev->mode_config.blob_lock));
-
- list_del(&blob->head_global);
- list_del(&blob->head_file);
- drm_mode_object_put(blob->dev, &blob->base);
-
- kfree(blob);
-}
-
-/**
* drm_property_unreference_blob - Unreference a blob property
*
* Drop a reference on a blob property. May free the object.
@@ -4321,42 +4389,14 @@
*/
void drm_property_unreference_blob(struct drm_property_blob *blob)
{
- struct drm_device *dev;
-
if (!blob)
return;
- dev = blob->dev;
-
- DRM_DEBUG("%p: blob ID: %d (%d)\n", blob, blob->base.id, atomic_read(&blob->refcount.refcount));
-
- if (kref_put_mutex(&blob->refcount, drm_property_free_blob,
- &dev->mode_config.blob_lock))
- mutex_unlock(&dev->mode_config.blob_lock);
- else
- might_lock(&dev->mode_config.blob_lock);
+ drm_mode_object_unreference(&blob->base);
}
EXPORT_SYMBOL(drm_property_unreference_blob);
/**
- * drm_property_unreference_blob_locked - Unreference a blob property with blob_lock held
- *
- * Drop a reference on a blob property. May free the object. This must be
- * called with blob_lock held.
- *
- * @blob: Pointer to blob property
- */
-static void drm_property_unreference_blob_locked(struct drm_property_blob *blob)
-{
- if (!blob)
- return;
-
- DRM_DEBUG("%p: blob ID: %d (%d)\n", blob, blob->base.id, atomic_read(&blob->refcount.refcount));
-
- kref_put(&blob->refcount, drm_property_free_blob);
-}
-
-/**
* drm_property_destroy_user_blobs - destroy all blobs created by this client
* @dev: DRM device
* @file_priv: destroy all blobs owned by this file handle
@@ -4366,14 +4406,14 @@
{
struct drm_property_blob *blob, *bt;
- mutex_lock(&dev->mode_config.blob_lock);
-
+ /*
+ * When the file gets released that means no one else can access the
+ * blob list any more, so no need to grab dev->blob_lock.
+ */
list_for_each_entry_safe(blob, bt, &file_priv->blobs, head_file) {
list_del_init(&blob->head_file);
- drm_property_unreference_blob_locked(blob);
+ drm_property_unreference_blob(blob);
}
-
- mutex_unlock(&dev->mode_config.blob_lock);
}
/**
@@ -4385,35 +4425,11 @@
*/
struct drm_property_blob *drm_property_reference_blob(struct drm_property_blob *blob)
{
- DRM_DEBUG("%p: blob ID: %d (%d)\n", blob, blob->base.id, atomic_read(&blob->refcount.refcount));
- kref_get(&blob->refcount);
+ drm_mode_object_reference(&blob->base);
return blob;
}
EXPORT_SYMBOL(drm_property_reference_blob);
-/*
- * Like drm_property_lookup_blob, but does not return an additional reference.
- * Must be called with blob_lock held.
- */
-static struct drm_property_blob *__drm_property_lookup_blob(struct drm_device *dev,
- uint32_t id)
-{
- struct drm_mode_object *obj = NULL;
- struct drm_property_blob *blob;
-
- WARN_ON(!mutex_is_locked(&dev->mode_config.blob_lock));
-
- mutex_lock(&dev->mode_config.idr_mutex);
- obj = idr_find(&dev->mode_config.crtc_idr, id);
- if (!obj || (obj->type != DRM_MODE_OBJECT_BLOB) || (obj->id != id))
- blob = NULL;
- else
- blob = obj_to_blob(obj);
- mutex_unlock(&dev->mode_config.idr_mutex);
-
- return blob;
-}
-
/**
* drm_property_lookup_blob - look up a blob property and take a reference
* @dev: drm device
@@ -4426,16 +4442,12 @@
struct drm_property_blob *drm_property_lookup_blob(struct drm_device *dev,
uint32_t id)
{
- struct drm_property_blob *blob;
+ struct drm_mode_object *obj;
+ struct drm_property_blob *blob = NULL;
- mutex_lock(&dev->mode_config.blob_lock);
- blob = __drm_property_lookup_blob(dev, id);
- if (blob) {
- if (!kref_get_unless_zero(&blob->refcount))
- blob = NULL;
- }
- mutex_unlock(&dev->mode_config.blob_lock);
-
+ obj = _object_find(dev, id, DRM_MODE_OBJECT_BLOB);
+ if (obj)
+ blob = obj_to_blob(obj);
return blob;
}
EXPORT_SYMBOL(drm_property_lookup_blob);
@@ -4540,26 +4552,21 @@
if (!drm_core_check_feature(dev, DRIVER_MODESET))
return -EINVAL;
- drm_modeset_lock_all(dev);
- mutex_lock(&dev->mode_config.blob_lock);
- blob = __drm_property_lookup_blob(dev, out_resp->blob_id);
- if (!blob) {
- ret = -ENOENT;
- goto done;
- }
+ blob = drm_property_lookup_blob(dev, out_resp->blob_id);
+ if (!blob)
+ return -ENOENT;
if (out_resp->length == blob->length) {
blob_ptr = (void __user *)(unsigned long)out_resp->data;
if (copy_to_user(blob_ptr, blob->data, blob->length)) {
ret = -EFAULT;
- goto done;
+ goto unref;
}
}
out_resp->length = blob->length;
+unref:
+ drm_property_unreference_blob(blob);
-done:
- mutex_unlock(&dev->mode_config.blob_lock);
- drm_modeset_unlock_all(dev);
return ret;
}
@@ -4638,13 +4645,11 @@
if (!drm_core_check_feature(dev, DRIVER_MODESET))
return -EINVAL;
- mutex_lock(&dev->mode_config.blob_lock);
- blob = __drm_property_lookup_blob(dev, out_resp->blob_id);
- if (!blob) {
- ret = -ENOENT;
- goto err;
- }
+ blob = drm_property_lookup_blob(dev, out_resp->blob_id);
+ if (!blob)
+ return -ENOENT;
+ mutex_lock(&dev->mode_config.blob_lock);
/* Ensure the property was actually created by this user. */
list_for_each_entry(bt, &file_priv->blobs, head_file) {
if (bt == blob) {
@@ -4661,13 +4666,18 @@
/* We must drop head_file here, because we may not be the last
* reference on the blob. */
list_del_init(&blob->head_file);
- drm_property_unreference_blob_locked(blob);
mutex_unlock(&dev->mode_config.blob_lock);
+ /* One reference from lookup, and one from the filp. */
+ drm_property_unreference_blob(blob);
+ drm_property_unreference_blob(blob);
+
return 0;
err:
mutex_unlock(&dev->mode_config.blob_lock);
+ drm_property_unreference_blob(blob);
+
return ret;
}
@@ -4831,19 +4841,7 @@
if (value == 0)
return true;
- /* handle refcnt'd objects specially: */
- if (property->values[0] == DRM_MODE_OBJECT_FB) {
- struct drm_framebuffer *fb;
- fb = drm_framebuffer_lookup(property->dev, value);
- if (fb) {
- *ref = &fb->base;
- return true;
- } else {
- return false;
- }
- } else {
- return _object_find(property->dev, value, property->values[0]) != NULL;
- }
+ return _object_find(property->dev, value, property->values[0]) != NULL;
}
for (i = 0; i < property->num_values; i++)
@@ -4859,8 +4857,7 @@
return;
if (drm_property_type_is(property, DRM_MODE_PROP_OBJECT)) {
- if (property->values[0] == DRM_MODE_OBJECT_FB)
- drm_framebuffer_unreference(obj_to_fb(ref));
+ drm_mode_object_unreference(ref);
} else if (drm_property_type_is(property, DRM_MODE_PROP_BLOB))
drm_property_unreference_blob(obj_to_blob(ref));
}
@@ -4991,7 +4988,7 @@
}
if (!obj->properties) {
ret = -EINVAL;
- goto out;
+ goto out_unref;
}
ret = get_properties(obj, file_priv->atomic,
@@ -4999,6 +4996,8 @@
(uint64_t __user *)(unsigned long)(arg->prop_values_ptr),
&arg->count_props);
+out_unref:
+ drm_mode_object_unreference(obj);
out:
drm_modeset_unlock_all(dev);
return ret;
@@ -5041,25 +5040,25 @@
goto out;
}
if (!arg_obj->properties)
- goto out;
+ goto out_unref;
for (i = 0; i < arg_obj->properties->count; i++)
if (arg_obj->properties->properties[i]->base.id == arg->prop_id)
break;
if (i == arg_obj->properties->count)
- goto out;
+ goto out_unref;
prop_obj = drm_mode_object_find(dev, arg->prop_id,
DRM_MODE_OBJECT_PROPERTY);
if (!prop_obj) {
ret = -ENOENT;
- goto out;
+ goto out_unref;
}
property = obj_to_property(prop_obj);
if (!drm_property_change_valid_get(property, arg->value, &ref))
- goto out;
+ goto out_unref;
switch (arg_obj->type) {
case DRM_MODE_OBJECT_CONNECTOR:
@@ -5077,6 +5076,8 @@
drm_property_change_valid_put(property, ref);
+out_unref:
+ drm_mode_object_unreference(arg_obj);
out:
drm_modeset_unlock_all(dev);
return ret;
@@ -5914,6 +5915,15 @@
drm_property_destroy(dev, property);
}
+ list_for_each_entry_safe(plane, plt, &dev->mode_config.plane_list,
+ head) {
+ plane->funcs->destroy(plane);
+ }
+
+ list_for_each_entry_safe(crtc, ct, &dev->mode_config.crtc_list, head) {
+ crtc->funcs->destroy(crtc);
+ }
+
list_for_each_entry_safe(blob, bt, &dev->mode_config.property_blob_list,
head_global) {
drm_property_unreference_blob(blob);
@@ -5929,16 +5939,7 @@
*/
WARN_ON(!list_empty(&dev->mode_config.fb_list));
list_for_each_entry_safe(fb, fbt, &dev->mode_config.fb_list, head) {
- drm_framebuffer_free(&fb->refcount);
- }
-
- list_for_each_entry_safe(plane, plt, &dev->mode_config.plane_list,
- head) {
- plane->funcs->destroy(plane);
- }
-
- list_for_each_entry_safe(crtc, ct, &dev->mode_config.crtc_list, head) {
- crtc->funcs->destroy(crtc);
+ drm_framebuffer_free(&fb->base.refcount);
}
ida_destroy(&dev->mode_config.connector_ida);
diff --git a/drivers/gpu/drm/drm_crtc_helper.c b/drivers/gpu/drm/drm_crtc_helper.c
index 79555d2..a6e4243 100644
--- a/drivers/gpu/drm/drm_crtc_helper.c
+++ b/drivers/gpu/drm/drm_crtc_helper.c
@@ -170,11 +170,14 @@
{
const struct drm_encoder_helper_funcs *encoder_funcs = encoder->helper_private;
+ if (!encoder_funcs)
+ return;
+
drm_bridge_disable(encoder->bridge);
if (encoder_funcs->disable)
(*encoder_funcs->disable)(encoder);
- else
+ else if (encoder_funcs->dpms)
(*encoder_funcs->dpms)(encoder, DRM_MODE_DPMS_OFF);
drm_bridge_post_disable(encoder->bridge);
@@ -248,6 +251,9 @@
drm_for_each_encoder(encoder, dev) {
encoder_funcs = encoder->helper_private;
+ if (!encoder_funcs)
+ continue;
+
/* Disable unused encoders */
if (encoder->crtc == NULL)
drm_encoder_disable(encoder);
@@ -326,6 +332,10 @@
if (encoder->crtc != crtc)
continue;
+ encoder_funcs = encoder->helper_private;
+ if (!encoder_funcs)
+ continue;
+
ret = drm_bridge_mode_fixup(encoder->bridge,
mode, adjusted_mode);
if (!ret) {
@@ -360,11 +370,15 @@
if (encoder->crtc != crtc)
continue;
+ encoder_funcs = encoder->helper_private;
+ if (!encoder_funcs)
+ continue;
+
drm_bridge_disable(encoder->bridge);
- encoder_funcs = encoder->helper_private;
/* Disable the encoders as the first thing we do. */
- encoder_funcs->prepare(encoder);
+ if (encoder_funcs->prepare)
+ encoder_funcs->prepare(encoder);
drm_bridge_post_disable(encoder->bridge);
}
@@ -385,11 +399,15 @@
if (encoder->crtc != crtc)
continue;
+ encoder_funcs = encoder->helper_private;
+ if (!encoder_funcs)
+ continue;
+
DRM_DEBUG_KMS("[ENCODER:%d:%s] set [MODE:%d:%s]\n",
encoder->base.id, encoder->name,
mode->base.id, mode->name);
- encoder_funcs = encoder->helper_private;
- encoder_funcs->mode_set(encoder, mode, adjusted_mode);
+ if (encoder_funcs->mode_set)
+ encoder_funcs->mode_set(encoder, mode, adjusted_mode);
drm_bridge_mode_set(encoder->bridge, mode, adjusted_mode);
}
@@ -402,10 +420,14 @@
if (encoder->crtc != crtc)
continue;
+ encoder_funcs = encoder->helper_private;
+ if (!encoder_funcs)
+ continue;
+
drm_bridge_pre_enable(encoder->bridge);
- encoder_funcs = encoder->helper_private;
- encoder_funcs->commit(encoder);
+ if (encoder_funcs->commit)
+ encoder_funcs->commit(encoder);
drm_bridge_enable(encoder->bridge);
}
@@ -456,6 +478,9 @@
* between them is henceforth no longer available.
*/
connector->dpms = DRM_MODE_DPMS_OFF;
+
+ /* we keep a reference while the encoder is bound */
+ drm_connector_unreference(connector);
}
}
@@ -606,6 +631,11 @@
mode_changed = true;
}
+ /* take a reference on all connectors in set */
+ for (ro = 0; ro < set->num_connectors; ro++) {
+ drm_connector_reference(set->connectors[ro]);
+ }
+
/* a) traverse passed in connector list and get encoders for them */
count = 0;
drm_for_each_connector(connector, dev) {
@@ -724,6 +754,12 @@
}
}
+ /* after fail drop reference on all connectors in save set */
+ count = 0;
+ drm_for_each_connector(connector, dev) {
+ drm_connector_unreference(&save_connectors[count++]);
+ }
+
kfree(save_connectors);
kfree(save_encoders);
return 0;
@@ -740,6 +776,11 @@
*connector = save_connectors[count++];
}
+ /* after fail drop reference on all connectors in set */
+ for (ro = 0; ro < set->num_connectors; ro++) {
+ drm_connector_unreference(set->connectors[ro]);
+ }
+
/* Try to restore the config */
if (mode_changed &&
!drm_crtc_helper_set_mode(save_set.crtc, save_set.mode, save_set.x,
@@ -771,12 +812,15 @@
struct drm_bridge *bridge = encoder->bridge;
const struct drm_encoder_helper_funcs *encoder_funcs;
+ encoder_funcs = encoder->helper_private;
+ if (!encoder_funcs)
+ return;
+
if (mode == DRM_MODE_DPMS_ON)
drm_bridge_pre_enable(bridge);
else
drm_bridge_disable(bridge);
- encoder_funcs = encoder->helper_private;
if (encoder_funcs->dpms)
encoder_funcs->dpms(encoder, mode);
@@ -1053,10 +1097,12 @@
if (plane->funcs->atomic_duplicate_state)
plane_state = plane->funcs->atomic_duplicate_state(plane);
- else if (plane->state)
+ else {
+ if (!plane->state)
+ drm_atomic_helper_plane_reset(plane);
+
plane_state = drm_atomic_helper_plane_duplicate_state(plane);
- else
- plane_state = kzalloc(sizeof(*plane_state), GFP_KERNEL);
+ }
if (!plane_state)
return -ENOMEM;
plane_state->plane = plane;
diff --git a/drivers/gpu/drm/drm_crtc_internal.h b/drivers/gpu/drm/drm_crtc_internal.h
index 247dc8b..a78c138 100644
--- a/drivers/gpu/drm/drm_crtc_internal.h
+++ b/drivers/gpu/drm/drm_crtc_internal.h
@@ -33,8 +33,8 @@
int drm_mode_object_get(struct drm_device *dev,
struct drm_mode_object *obj, uint32_t obj_type);
-void drm_mode_object_put(struct drm_device *dev,
- struct drm_mode_object *object);
+void drm_mode_object_unregister(struct drm_device *dev,
+ struct drm_mode_object *object);
/* drm_atomic.c */
int drm_atomic_get_property(struct drm_mode_object *obj,
diff --git a/drivers/gpu/drm/drm_dp_aux_dev.c b/drivers/gpu/drm/drm_dp_aux_dev.c
index f73b38b..3334baa 100644
--- a/drivers/gpu/drm/drm_dp_aux_dev.c
+++ b/drivers/gpu/drm/drm_dp_aux_dev.c
@@ -159,6 +159,12 @@
uint8_t localbuf[DP_AUX_MAX_PAYLOAD_BYTES];
ssize_t todo = min_t(size_t, bytes_pending, sizeof(localbuf));
+ if (signal_pending(current)) {
+ res = num_bytes_processed ?
+ num_bytes_processed : -ERESTARTSYS;
+ goto out;
+ }
+
res = drm_dp_dpcd_read(aux_dev->aux, *offset, localbuf, todo);
if (res <= 0) {
res = num_bytes_processed ? num_bytes_processed : res;
@@ -202,6 +208,12 @@
uint8_t localbuf[DP_AUX_MAX_PAYLOAD_BYTES];
ssize_t todo = min_t(size_t, bytes_pending, sizeof(localbuf));
+ if (signal_pending(current)) {
+ res = num_bytes_processed ?
+ num_bytes_processed : -ERESTARTSYS;
+ goto out;
+ }
+
if (__copy_from_user(localbuf,
buf + num_bytes_processed, todo)) {
res = num_bytes_processed ?
diff --git a/drivers/gpu/drm/drm_dp_helper.c b/drivers/gpu/drm/drm_dp_helper.c
index df64ed1..eeaf5a7 100644
--- a/drivers/gpu/drm/drm_dp_helper.c
+++ b/drivers/gpu/drm/drm_dp_helper.c
@@ -178,8 +178,8 @@
unsigned int offset, void *buffer, size_t size)
{
struct drm_dp_aux_msg msg;
- unsigned int retry;
- int err = 0;
+ unsigned int retry, native_reply;
+ int err = 0, ret = 0;
memset(&msg, 0, sizeof(msg));
msg.address = offset;
@@ -196,38 +196,39 @@
* sufficient, bump to 32 which makes Dell 4k monitors happier.
*/
for (retry = 0; retry < 32; retry++) {
-
- err = aux->transfer(aux, &msg);
- if (err < 0) {
- if (err == -EBUSY)
- continue;
-
- goto unlock;
+ if (ret != 0 && ret != -ETIMEDOUT) {
+ usleep_range(AUX_RETRY_INTERVAL,
+ AUX_RETRY_INTERVAL + 100);
}
+ ret = aux->transfer(aux, &msg);
- switch (msg.reply & DP_AUX_NATIVE_REPLY_MASK) {
- case DP_AUX_NATIVE_REPLY_ACK:
- if (err < size)
- err = -EPROTO;
- goto unlock;
+ if (ret > 0) {
+ native_reply = msg.reply & DP_AUX_NATIVE_REPLY_MASK;
+ if (native_reply == DP_AUX_NATIVE_REPLY_ACK) {
+ if (ret == size)
+ goto unlock;
- case DP_AUX_NATIVE_REPLY_NACK:
- err = -EIO;
- goto unlock;
-
- case DP_AUX_NATIVE_REPLY_DEFER:
- usleep_range(AUX_RETRY_INTERVAL, AUX_RETRY_INTERVAL + 100);
- break;
+ ret = -EPROTO;
+ } else
+ ret = -EIO;
}
+
+ /*
+ * We want the error we return to be the error we received on
+ * the first transaction, since we may get a different error the
+ * next time we retry
+ */
+ if (!err)
+ err = ret;
}
DRM_DEBUG_KMS("too many retries, giving up\n");
- err = -EIO;
+ ret = err;
unlock:
mutex_unlock(&aux->hw_mutex);
- return err;
+ return ret;
}
/**
@@ -247,6 +248,25 @@
ssize_t drm_dp_dpcd_read(struct drm_dp_aux *aux, unsigned int offset,
void *buffer, size_t size)
{
+ int ret;
+
+ /*
+ * HP ZR24w corrupts the first DPCD access after entering power save
+ * mode. Eg. on a read, the entire buffer will be filled with the same
+ * byte. Do a throw away read to avoid corrupting anything we care
+ * about. Afterwards things will work correctly until the monitor
+ * gets woken up and subsequently re-enters power save mode.
+ *
+ * The user pressing any button on the monitor is enough to wake it
+ * up, so there is no particularly good place to do the workaround.
+ * We just have to do it before any DPCD access and hope that the
+ * monitor doesn't power down exactly after the throw away read.
+ */
+ ret = drm_dp_dpcd_access(aux, DP_AUX_NATIVE_READ, DP_DPCD_REV, buffer,
+ 1);
+ if (ret != 1)
+ return ret;
+
return drm_dp_dpcd_access(aux, DP_AUX_NATIVE_READ, offset, buffer,
size);
}
diff --git a/drivers/gpu/drm/drm_dp_mst_topology.c b/drivers/gpu/drm/drm_dp_mst_topology.c
index 71ea052..a13edf5 100644
--- a/drivers/gpu/drm/drm_dp_mst_topology.c
+++ b/drivers/gpu/drm/drm_dp_mst_topology.c
@@ -2756,7 +2756,7 @@
seq_printf(m, "%smst: %p, %d\n", prefix, mstb, mstb->num_ports);
list_for_each_entry(port, &mstb->ports, next) {
- seq_printf(m, "%sport: %d: ddps: %d ldps: %d, sdp: %d/%d, %p, conn: %p\n", prefix, port->port_num, port->ddps, port->ldps, port->num_sdp_streams, port->num_sdp_stream_sinks, port, port->connector);
+ seq_printf(m, "%sport: %d: input: %d: pdt: %d, ddps: %d ldps: %d, sdp: %d/%d, %p, conn: %p\n", prefix, port->port_num, port->input, port->pdt, port->ddps, port->ldps, port->num_sdp_streams, port->num_sdp_stream_sinks, port, port->connector);
if (port->mstb)
drm_dp_mst_dump_mstb(m, port->mstb);
}
@@ -2777,6 +2777,16 @@
return false;
}
+static void fetch_monitor_name(struct drm_dp_mst_topology_mgr *mgr,
+ struct drm_dp_mst_port *port, char *name,
+ int namelen)
+{
+ struct edid *mst_edid;
+
+ mst_edid = drm_dp_mst_get_edid(port->connector, mgr, port);
+ drm_edid_get_monitor_name(mst_edid, name, namelen);
+}
+
/**
* drm_dp_mst_dump_topology(): dump topology to seq file.
* @m: seq_file to dump output to
@@ -2789,6 +2799,7 @@
{
int i;
struct drm_dp_mst_port *port;
+
mutex_lock(&mgr->lock);
if (mgr->mst_primary)
drm_dp_mst_dump_mstb(m, mgr->mst_primary);
@@ -2797,14 +2808,21 @@
mutex_unlock(&mgr->lock);
mutex_lock(&mgr->payload_lock);
- seq_printf(m, "vcpi: %lx %lx\n", mgr->payload_mask, mgr->vcpi_mask);
+ seq_printf(m, "vcpi: %lx %lx %d\n", mgr->payload_mask, mgr->vcpi_mask,
+ mgr->max_payloads);
for (i = 0; i < mgr->max_payloads; i++) {
if (mgr->proposed_vcpis[i]) {
+ char name[14];
+
port = container_of(mgr->proposed_vcpis[i], struct drm_dp_mst_port, vcpi);
- seq_printf(m, "vcpi %d: %d %d %d\n", i, port->port_num, port->vcpi.vcpi, port->vcpi.num_slots);
+ fetch_monitor_name(mgr, port, name, sizeof(name));
+ seq_printf(m, "vcpi %d: %d %d %d sink name: %s\n", i,
+ port->port_num, port->vcpi.vcpi,
+ port->vcpi.num_slots,
+ (*name != 0) ? name : "Unknown");
} else
- seq_printf(m, "vcpi %d:unsed\n", i);
+ seq_printf(m, "vcpi %d:unused\n", i);
}
for (i = 0; i < mgr->max_payloads; i++) {
seq_printf(m, "payload %d: %d, %d, %d\n",
@@ -2844,8 +2862,9 @@
for (i = 0; i < 0x3; i++)
seq_printf(m, "%02x", buf[i]);
seq_printf(m, " devid: ");
- for (i = 0x3; i < 0x8; i++)
+ for (i = 0x3; i < 0x8 && buf[i]; i++)
seq_printf(m, "%c", buf[i]);
+
seq_printf(m, " revision: hw: %x.%x sw: %x.%x", buf[0x9] >> 4, buf[0x9] & 0xf, buf[0xa], buf[0xb]);
seq_printf(m, "\n");
bret = dump_dp_payload_table(mgr, buf);
diff --git a/drivers/gpu/drm/drm_drv.c b/drivers/gpu/drm/drm_drv.c
index 167c8d3..bff8922 100644
--- a/drivers/gpu/drm/drm_drv.c
+++ b/drivers/gpu/drm/drm_drv.c
@@ -37,13 +37,23 @@
#include "drm_legacy.h"
#include "drm_internal.h"
-unsigned int drm_debug = 0; /* bitmask of DRM_UT_x */
+/*
+ * drm_debug: Enable debug output.
+ * Bitmask of DRM_UT_x. See include/drm/drmP.h for details.
+ */
+unsigned int drm_debug = 0;
EXPORT_SYMBOL(drm_debug);
MODULE_AUTHOR(CORE_AUTHOR);
MODULE_DESCRIPTION(CORE_DESC);
MODULE_LICENSE("GPL and additional rights");
-MODULE_PARM_DESC(debug, "Enable debug output");
+MODULE_PARM_DESC(debug, "Enable debug output, where each bit enables a debug category.\n"
+"\t\tBit 0 (0x01) will enable CORE messages (drm core code)\n"
+"\t\tBit 1 (0x02) will enable DRIVER messages (drm controller code)\n"
+"\t\tBit 2 (0x04) will enable KMS messages (modesetting code)\n"
+"\t\tBit 3 (0x08) will enable PRIME messages (prime code)\n"
+"\t\tBit 4 (0x10) will enable ATOMIC messages (atomic code)\n"
+"\t\tBit 5 (0x20) will enable VBL messages (vblank code)");
module_param_named(debug, drm_debug, int, 0600);
static DEFINE_SPINLOCK(drm_minor_lock);
@@ -111,19 +121,11 @@
{
struct drm_master *master = container_of(kref, struct drm_master, refcount);
struct drm_device *dev = master->minor->dev;
- struct drm_map_list *r_list, *list_temp;
- mutex_lock(&dev->struct_mutex);
if (dev->driver->master_destroy)
dev->driver->master_destroy(dev, master);
- list_for_each_entry_safe(r_list, list_temp, &dev->maplist, head) {
- if (r_list->master == master) {
- drm_legacy_rmmap_locked(dev, r_list->map);
- r_list = NULL;
- }
- }
- mutex_unlock(&dev->struct_mutex);
+ drm_legacy_master_rmmaps(dev, master);
idr_destroy(&master->magic_map);
kfree(master->unique);
@@ -588,6 +590,7 @@
spin_lock_init(&dev->buf_lock);
spin_lock_init(&dev->event_lock);
mutex_init(&dev->struct_mutex);
+ mutex_init(&dev->filelist_mutex);
mutex_init(&dev->ctxlist_mutex);
mutex_init(&dev->master_mutex);
@@ -715,7 +718,11 @@
*
* Register the DRM device @dev with the system, advertise device to user-space
* and start normal device operation. @dev must be allocated via drm_dev_alloc()
- * previously.
+ * previously. Right after drm_dev_register() the driver should call
+ * drm_connector_register_all() to register all connectors in sysfs. This is
+ * a separate call for backward compatibility with drivers still using
+ * the deprecated ->load() callback, where connectors are registered from within
+ * the ->load() callback.
*
* Never call this twice on any device!
*
diff --git a/drivers/gpu/drm/drm_edid.c b/drivers/gpu/drm/drm_edid.c
index 558ef9f..7df26d4 100644
--- a/drivers/gpu/drm/drm_edid.c
+++ b/drivers/gpu/drm/drm_edid.c
@@ -3293,6 +3293,46 @@
*(u8 **)data = t->data.other_data.data.str.str;
}
+static int get_monitor_name(struct edid *edid, char name[13])
+{
+ char *edid_name = NULL;
+ int mnl;
+
+ if (!edid || !name)
+ return 0;
+
+ drm_for_each_detailed_block((u8 *)edid, monitor_name, &edid_name);
+ for (mnl = 0; edid_name && mnl < 13; mnl++) {
+ if (edid_name[mnl] == 0x0a)
+ break;
+
+ name[mnl] = edid_name[mnl];
+ }
+
+ return mnl;
+}
+
+/**
+ * drm_edid_get_monitor_name - fetch the monitor name from the edid
+ * @edid: monitor EDID information
+ * @name: pointer to a character array to hold the name of the monitor
+ * @bufsize: The size of the name buffer (should be at least 14 chars.)
+ *
+ */
+void drm_edid_get_monitor_name(struct edid *edid, char *name, int bufsize)
+{
+ int name_length;
+ char buf[13];
+
+ if (bufsize <= 0)
+ return;
+
+ name_length = min(get_monitor_name(edid, buf), bufsize - 1);
+ memcpy(name, buf, name_length);
+ name[name_length] = '\0';
+}
+EXPORT_SYMBOL(drm_edid_get_monitor_name);
+
/**
* drm_edid_to_eld - build ELD from EDID
* @connector: connector corresponding to the HDMI/DP sink
@@ -3306,7 +3346,6 @@
{
uint8_t *eld = connector->eld;
u8 *cea;
- u8 *name;
u8 *db;
int total_sad_count = 0;
int mnl;
@@ -3320,14 +3359,8 @@
return;
}
- name = NULL;
- drm_for_each_detailed_block((u8 *)edid, monitor_name, &name);
- /* max: 13 bytes EDID, 16 bytes ELD */
- for (mnl = 0; name && mnl < 13; mnl++) {
- if (name[mnl] == 0x0a)
- break;
- eld[20 + mnl] = name[mnl];
- }
+ mnl = get_monitor_name(edid, eld + 20);
+
eld[4] = (cea[1] << 5) | mnl;
DRM_DEBUG_KMS("ELD monitor %s\n", eld + 20);
@@ -3868,6 +3901,133 @@
info->color_formats |= DRM_COLOR_FORMAT_YCRCB422;
}
+static int validate_displayid(u8 *displayid, int length, int idx)
+{
+ int i;
+ u8 csum = 0;
+ struct displayid_hdr *base;
+
+ base = (struct displayid_hdr *)&displayid[idx];
+
+ DRM_DEBUG_KMS("base revision 0x%x, length %d, %d %d\n",
+ base->rev, base->bytes, base->prod_id, base->ext_count);
+
+ if (base->bytes + 5 > length - idx)
+ return -EINVAL;
+ for (i = idx; i <= base->bytes + 5; i++) {
+ csum += displayid[i];
+ }
+ if (csum) {
+ DRM_ERROR("DisplayID checksum invalid, remainder is %d\n", csum);
+ return -EINVAL;
+ }
+ return 0;
+}
+
+static struct drm_display_mode *drm_mode_displayid_detailed(struct drm_device *dev,
+ struct displayid_detailed_timings_1 *timings)
+{
+ struct drm_display_mode *mode;
+ unsigned pixel_clock = (timings->pixel_clock[0] |
+ (timings->pixel_clock[1] << 8) |
+ (timings->pixel_clock[2] << 16));
+ unsigned hactive = (timings->hactive[0] | timings->hactive[1] << 8) + 1;
+ unsigned hblank = (timings->hblank[0] | timings->hblank[1] << 8) + 1;
+ unsigned hsync = (timings->hsync[0] | (timings->hsync[1] & 0x7f) << 8) + 1;
+ unsigned hsync_width = (timings->hsw[0] | timings->hsw[1] << 8) + 1;
+ unsigned vactive = (timings->vactive[0] | timings->vactive[1] << 8) + 1;
+ unsigned vblank = (timings->vblank[0] | timings->vblank[1] << 8) + 1;
+ unsigned vsync = (timings->vsync[0] | (timings->vsync[1] & 0x7f) << 8) + 1;
+ unsigned vsync_width = (timings->vsw[0] | timings->vsw[1] << 8) + 1;
+ bool hsync_positive = (timings->hsync[1] >> 7) & 0x1;
+ bool vsync_positive = (timings->vsync[1] >> 7) & 0x1;
+ mode = drm_mode_create(dev);
+ if (!mode)
+ return NULL;
+
+ mode->clock = pixel_clock * 10;
+ mode->hdisplay = hactive;
+ mode->hsync_start = mode->hdisplay + hsync;
+ mode->hsync_end = mode->hsync_start + hsync_width;
+ mode->htotal = mode->hdisplay + hblank;
+
+ mode->vdisplay = vactive;
+ mode->vsync_start = mode->vdisplay + vsync;
+ mode->vsync_end = mode->vsync_start + vsync_width;
+ mode->vtotal = mode->vdisplay + vblank;
+
+ mode->flags = 0;
+ mode->flags |= hsync_positive ? DRM_MODE_FLAG_PHSYNC : DRM_MODE_FLAG_NHSYNC;
+ mode->flags |= vsync_positive ? DRM_MODE_FLAG_PVSYNC : DRM_MODE_FLAG_NVSYNC;
+ mode->type = DRM_MODE_TYPE_DRIVER;
+
+ if (timings->flags & 0x80)
+ mode->type |= DRM_MODE_TYPE_PREFERRED;
+ mode->vrefresh = drm_mode_vrefresh(mode);
+ drm_mode_set_name(mode);
+
+ return mode;
+}
+
+static int add_displayid_detailed_1_modes(struct drm_connector *connector,
+ struct displayid_block *block)
+{
+ struct displayid_detailed_timing_block *det = (struct displayid_detailed_timing_block *)block;
+ int i;
+ int num_timings;
+ struct drm_display_mode *newmode;
+ int num_modes = 0;
+ /* blocks must be multiple of 20 bytes length */
+ if (block->num_bytes % 20)
+ return 0;
+
+ num_timings = block->num_bytes / 20;
+ for (i = 0; i < num_timings; i++) {
+ struct displayid_detailed_timings_1 *timings = &det->timings[i];
+
+ newmode = drm_mode_displayid_detailed(connector->dev, timings);
+ if (!newmode)
+ continue;
+
+ drm_mode_probed_add(connector, newmode);
+ num_modes++;
+ }
+ return num_modes;
+}
+
+static int add_displayid_detailed_modes(struct drm_connector *connector,
+ struct edid *edid)
+{
+ u8 *displayid;
+ int ret;
+ int idx = 1;
+ int length = EDID_LENGTH;
+ struct displayid_block *block;
+ int num_modes = 0;
+
+ displayid = drm_find_displayid_extension(edid);
+ if (!displayid)
+ return 0;
+
+ ret = validate_displayid(displayid, length, idx);
+ if (ret)
+ return 0;
+
+ idx += sizeof(struct displayid_hdr);
+ while (block = (struct displayid_block *)&displayid[idx],
+ idx + sizeof(struct displayid_block) <= length &&
+ idx + sizeof(struct displayid_block) + block->num_bytes <= length &&
+ block->num_bytes > 0) {
+ idx += block->num_bytes + sizeof(struct displayid_block);
+ switch (block->tag) {
+ case DATA_BLOCK_TYPE_1_DETAILED_TIMING:
+ num_modes += add_displayid_detailed_1_modes(connector, block);
+ break;
+ }
+ }
+ return num_modes;
+}
+
/**
* drm_add_edid_modes - add modes from EDID data, if available
* @connector: connector we're probing
@@ -3913,6 +4073,7 @@
num_modes += add_established_modes(connector, edid);
num_modes += add_cea_modes(connector, edid);
num_modes += add_alternate_cea_modes(connector, edid);
+ num_modes += add_displayid_detailed_modes(connector, edid);
if (edid->features & DRM_EDID_FEATURE_DEFAULT_GTF)
num_modes += add_inferred_modes(connector, edid);
@@ -4119,96 +4280,98 @@
}
EXPORT_SYMBOL(drm_hdmi_vendor_infoframe_from_display_mode);
+static int drm_parse_tiled_block(struct drm_connector *connector,
+ struct displayid_block *block)
+{
+ struct displayid_tiled_block *tile = (struct displayid_tiled_block *)block;
+ u16 w, h;
+ u8 tile_v_loc, tile_h_loc;
+ u8 num_v_tile, num_h_tile;
+ struct drm_tile_group *tg;
+
+ w = tile->tile_size[0] | tile->tile_size[1] << 8;
+ h = tile->tile_size[2] | tile->tile_size[3] << 8;
+
+ num_v_tile = (tile->topo[0] & 0xf) | (tile->topo[2] & 0x30);
+ num_h_tile = (tile->topo[0] >> 4) | ((tile->topo[2] >> 2) & 0x30);
+ tile_v_loc = (tile->topo[1] & 0xf) | ((tile->topo[2] & 0x3) << 4);
+ tile_h_loc = (tile->topo[1] >> 4) | (((tile->topo[2] >> 2) & 0x3) << 4);
+
+ connector->has_tile = true;
+ if (tile->tile_cap & 0x80)
+ connector->tile_is_single_monitor = true;
+
+ connector->num_h_tile = num_h_tile + 1;
+ connector->num_v_tile = num_v_tile + 1;
+ connector->tile_h_loc = tile_h_loc;
+ connector->tile_v_loc = tile_v_loc;
+ connector->tile_h_size = w + 1;
+ connector->tile_v_size = h + 1;
+
+ DRM_DEBUG_KMS("tile cap 0x%x\n", tile->tile_cap);
+ DRM_DEBUG_KMS("tile_size %d x %d\n", w + 1, h + 1);
+ DRM_DEBUG_KMS("topo num tiles %dx%d, location %dx%d\n",
+ num_h_tile + 1, num_v_tile + 1, tile_h_loc, tile_v_loc);
+ DRM_DEBUG_KMS("vend %c%c%c\n", tile->topology_id[0], tile->topology_id[1], tile->topology_id[2]);
+
+ tg = drm_mode_get_tile_group(connector->dev, tile->topology_id);
+ if (!tg) {
+ tg = drm_mode_create_tile_group(connector->dev, tile->topology_id);
+ }
+ if (!tg)
+ return -ENOMEM;
+
+ if (connector->tile_group != tg) {
+ /* if we haven't got a pointer,
+ take the reference, drop ref to old tile group */
+ if (connector->tile_group) {
+ drm_mode_put_tile_group(connector->dev, connector->tile_group);
+ }
+ connector->tile_group = tg;
+ } else
+ /* if same tile group, then release the ref we just took. */
+ drm_mode_put_tile_group(connector->dev, tg);
+ return 0;
+}
+
static int drm_parse_display_id(struct drm_connector *connector,
u8 *displayid, int length,
bool is_edid_extension)
{
/* if this is an EDID extension the first byte will be 0x70 */
int idx = 0;
- struct displayid_hdr *base;
struct displayid_block *block;
- u8 csum = 0;
- int i;
+ int ret;
if (is_edid_extension)
idx = 1;
- base = (struct displayid_hdr *)&displayid[idx];
+ ret = validate_displayid(displayid, length, idx);
+ if (ret)
+ return ret;
- DRM_DEBUG_KMS("base revision 0x%x, length %d, %d %d\n",
- base->rev, base->bytes, base->prod_id, base->ext_count);
+ idx += sizeof(struct displayid_hdr);
+ while (block = (struct displayid_block *)&displayid[idx],
+ idx + sizeof(struct displayid_block) <= length &&
+ idx + sizeof(struct displayid_block) + block->num_bytes <= length &&
+ block->num_bytes > 0) {
+ idx += block->num_bytes + sizeof(struct displayid_block);
+ DRM_DEBUG_KMS("block id 0x%x, rev %d, len %d\n",
+ block->tag, block->rev, block->num_bytes);
- if (base->bytes + 5 > length - idx)
- return -EINVAL;
-
- for (i = idx; i <= base->bytes + 5; i++) {
- csum += displayid[i];
- }
- if (csum) {
- DRM_ERROR("DisplayID checksum invalid, remainder is %d\n", csum);
- return -EINVAL;
- }
-
- block = (struct displayid_block *)&displayid[idx + 4];
- DRM_DEBUG_KMS("block id %d, rev %d, len %d\n",
- block->tag, block->rev, block->num_bytes);
-
- switch (block->tag) {
- case DATA_BLOCK_TILED_DISPLAY: {
- struct displayid_tiled_block *tile = (struct displayid_tiled_block *)block;
-
- u16 w, h;
- u8 tile_v_loc, tile_h_loc;
- u8 num_v_tile, num_h_tile;
- struct drm_tile_group *tg;
-
- w = tile->tile_size[0] | tile->tile_size[1] << 8;
- h = tile->tile_size[2] | tile->tile_size[3] << 8;
-
- num_v_tile = (tile->topo[0] & 0xf) | (tile->topo[2] & 0x30);
- num_h_tile = (tile->topo[0] >> 4) | ((tile->topo[2] >> 2) & 0x30);
- tile_v_loc = (tile->topo[1] & 0xf) | ((tile->topo[2] & 0x3) << 4);
- tile_h_loc = (tile->topo[1] >> 4) | (((tile->topo[2] >> 2) & 0x3) << 4);
-
- connector->has_tile = true;
- if (tile->tile_cap & 0x80)
- connector->tile_is_single_monitor = true;
-
- connector->num_h_tile = num_h_tile + 1;
- connector->num_v_tile = num_v_tile + 1;
- connector->tile_h_loc = tile_h_loc;
- connector->tile_v_loc = tile_v_loc;
- connector->tile_h_size = w + 1;
- connector->tile_v_size = h + 1;
-
- DRM_DEBUG_KMS("tile cap 0x%x\n", tile->tile_cap);
- DRM_DEBUG_KMS("tile_size %d x %d\n", w + 1, h + 1);
- DRM_DEBUG_KMS("topo num tiles %dx%d, location %dx%d\n",
- num_h_tile + 1, num_v_tile + 1, tile_h_loc, tile_v_loc);
- DRM_DEBUG_KMS("vend %c%c%c\n", tile->topology_id[0], tile->topology_id[1], tile->topology_id[2]);
-
- tg = drm_mode_get_tile_group(connector->dev, tile->topology_id);
- if (!tg) {
- tg = drm_mode_create_tile_group(connector->dev, tile->topology_id);
+ switch (block->tag) {
+ case DATA_BLOCK_TILED_DISPLAY:
+ ret = drm_parse_tiled_block(connector, block);
+ if (ret)
+ return ret;
+ break;
+ case DATA_BLOCK_TYPE_1_DETAILED_TIMING:
+ /* handled in mode gathering code. */
+ break;
+ default:
+ DRM_DEBUG_KMS("found DisplayID tag 0x%x, unhandled\n", block->tag);
+ break;
}
- if (!tg)
- return -ENOMEM;
-
- if (connector->tile_group != tg) {
- /* if we haven't got a pointer,
- take the reference, drop ref to old tile group */
- if (connector->tile_group) {
- drm_mode_put_tile_group(connector->dev, connector->tile_group);
- }
- connector->tile_group = tg;
- } else
- /* if same tile group, then release the ref we just took. */
- drm_mode_put_tile_group(connector->dev, tg);
- }
- break;
- default:
- printk("unknown displayid tag %d\n", block->tag);
- break;
}
return 0;
}
diff --git a/drivers/gpu/drm/drm_fb_cma_helper.c b/drivers/gpu/drm/drm_fb_cma_helper.c
index bb88e3d..172cafe 100644
--- a/drivers/gpu/drm/drm_fb_cma_helper.c
+++ b/drivers/gpu/drm/drm_fb_cma_helper.c
@@ -25,6 +25,8 @@
#include <drm/drm_fb_cma_helper.h>
#include <linux/module.h>
+#define DEFAULT_FBDEFIO_DELAY_MS 50
+
struct drm_fb_cma {
struct drm_framebuffer fb;
struct drm_gem_cma_object *obj[4];
@@ -35,6 +37,59 @@
struct drm_fb_cma *fb;
};
+/**
+ * DOC: framebuffer cma helper functions
+ *
+ * Provides helper functions for creating a cma (contiguous memory allocator)
+ * backed framebuffer.
+ *
+ * drm_fb_cma_create() is used in the &drm_mode_config_funcs ->fb_create
+ * callback function to create a cma backed framebuffer.
+ *
+ * An fbdev framebuffer backed by cma is also available by calling
+ * drm_fbdev_cma_init(). drm_fbdev_cma_fini() tears it down.
+ * If the &drm_framebuffer_funcs ->dirty callback is set, fb_deferred_io
+ * will be set up automatically. dirty() is called by
+ * drm_fb_helper_deferred_io() in process context (struct delayed_work).
+ *
+ * Example fbdev deferred io code:
+ *
+ * static int driver_fbdev_fb_dirty(struct drm_framebuffer *fb,
+ * struct drm_file *file_priv,
+ * unsigned flags, unsigned color,
+ * struct drm_clip_rect *clips,
+ * unsigned num_clips)
+ * {
+ * struct drm_gem_cma_object *cma = drm_fb_cma_get_gem_obj(fb, 0);
+ * ... push changes ...
+ * return 0;
+ * }
+ *
+ * static struct drm_framebuffer_funcs driver_fbdev_fb_funcs = {
+ * .destroy = drm_fb_cma_destroy,
+ * .create_handle = drm_fb_cma_create_handle,
+ * .dirty = driver_fbdev_fb_dirty,
+ * };
+ *
+ * static int driver_fbdev_create(struct drm_fb_helper *helper,
+ * struct drm_fb_helper_surface_size *sizes)
+ * {
+ * return drm_fbdev_cma_create_with_funcs(helper, sizes,
+ * &driver_fbdev_fb_funcs);
+ * }
+ *
+ * static const struct drm_fb_helper_funcs driver_fb_helper_funcs = {
+ * .fb_probe = driver_fbdev_create,
+ * };
+ *
+ * Initialize:
+ * fbdev = drm_fbdev_cma_init_with_funcs(dev, 16,
+ * dev->mode_config.num_crtc,
+ * dev->mode_config.num_connector,
+ * &driver_fb_helper_funcs);
+ *
+ */
+
static inline struct drm_fbdev_cma *to_fbdev_cma(struct drm_fb_helper *helper)
{
return container_of(helper, struct drm_fbdev_cma, fb_helper);
@@ -45,7 +100,7 @@
return container_of(fb, struct drm_fb_cma, fb);
}
-static void drm_fb_cma_destroy(struct drm_framebuffer *fb)
+void drm_fb_cma_destroy(struct drm_framebuffer *fb)
{
struct drm_fb_cma *fb_cma = to_fb_cma(fb);
int i;
@@ -58,8 +113,9 @@
drm_framebuffer_cleanup(fb);
kfree(fb_cma);
}
+EXPORT_SYMBOL(drm_fb_cma_destroy);
-static int drm_fb_cma_create_handle(struct drm_framebuffer *fb,
+int drm_fb_cma_create_handle(struct drm_framebuffer *fb,
struct drm_file *file_priv, unsigned int *handle)
{
struct drm_fb_cma *fb_cma = to_fb_cma(fb);
@@ -67,6 +123,7 @@
return drm_gem_handle_create(file_priv,
&fb_cma->obj[0]->base, handle);
}
+EXPORT_SYMBOL(drm_fb_cma_create_handle);
static struct drm_framebuffer_funcs drm_fb_cma_funcs = {
.destroy = drm_fb_cma_destroy,
@@ -76,7 +133,7 @@
static struct drm_fb_cma *drm_fb_cma_alloc(struct drm_device *dev,
const struct drm_mode_fb_cmd2 *mode_cmd,
struct drm_gem_cma_object **obj,
- unsigned int num_planes)
+ unsigned int num_planes, const struct drm_framebuffer_funcs *funcs)
{
struct drm_fb_cma *fb_cma;
int ret;
@@ -91,7 +148,7 @@
for (i = 0; i < num_planes; i++)
fb_cma->obj[i] = obj[i];
- ret = drm_framebuffer_init(dev, &fb_cma->fb, &drm_fb_cma_funcs);
+ ret = drm_framebuffer_init(dev, &fb_cma->fb, funcs);
if (ret) {
dev_err(dev->dev, "Failed to initialize framebuffer: %d\n", ret);
kfree(fb_cma);
@@ -102,13 +159,17 @@
}
/**
- * drm_fb_cma_create() - (struct drm_mode_config_funcs *)->fb_create callback function
+ * drm_fb_cma_create_with_funcs() - helper function for the
+ * &drm_mode_config_funcs ->fb_create
+ * callback function
*
- * If your hardware has special alignment or pitch requirements these should be
- * checked before calling this function.
+ * This can be used to set &drm_framebuffer_funcs for drivers that need the
+ * dirty() callback. Use drm_fb_cma_create() if you don't need to change
+ * &drm_framebuffer_funcs.
*/
-struct drm_framebuffer *drm_fb_cma_create(struct drm_device *dev,
- struct drm_file *file_priv, const struct drm_mode_fb_cmd2 *mode_cmd)
+struct drm_framebuffer *drm_fb_cma_create_with_funcs(struct drm_device *dev,
+ struct drm_file *file_priv, const struct drm_mode_fb_cmd2 *mode_cmd,
+ const struct drm_framebuffer_funcs *funcs)
{
struct drm_fb_cma *fb_cma;
struct drm_gem_cma_object *objs[4];
@@ -126,7 +187,7 @@
unsigned int height = mode_cmd->height / (i ? vsub : 1);
unsigned int min_size;
- obj = drm_gem_object_lookup(dev, file_priv, mode_cmd->handles[i]);
+ obj = drm_gem_object_lookup(file_priv, mode_cmd->handles[i]);
if (!obj) {
dev_err(dev->dev, "Failed to lookup GEM object\n");
ret = -ENXIO;
@@ -145,7 +206,7 @@
objs[i] = to_drm_gem_cma_obj(obj);
}
- fb_cma = drm_fb_cma_alloc(dev, mode_cmd, objs, i);
+ fb_cma = drm_fb_cma_alloc(dev, mode_cmd, objs, i, funcs);
if (IS_ERR(fb_cma)) {
ret = PTR_ERR(fb_cma);
goto err_gem_object_unreference;
@@ -158,6 +219,21 @@
drm_gem_object_unreference_unlocked(&objs[i]->base);
return ERR_PTR(ret);
}
+EXPORT_SYMBOL_GPL(drm_fb_cma_create_with_funcs);
+
+/**
+ * drm_fb_cma_create() - &drm_mode_config_funcs ->fb_create callback function
+ *
+ * If your hardware has special alignment or pitch requirements these should be
+ * checked before calling this function. Use drm_fb_cma_create_with_funcs() if
+ * you need to set &drm_framebuffer_funcs ->dirty.
+ */
+struct drm_framebuffer *drm_fb_cma_create(struct drm_device *dev,
+ struct drm_file *file_priv, const struct drm_mode_fb_cmd2 *mode_cmd)
+{
+ return drm_fb_cma_create_with_funcs(dev, file_priv, mode_cmd,
+ &drm_fb_cma_funcs);
+}
EXPORT_SYMBOL_GPL(drm_fb_cma_create);
/**
@@ -233,8 +309,67 @@
.fb_setcmap = drm_fb_helper_setcmap,
};
-static int drm_fbdev_cma_create(struct drm_fb_helper *helper,
- struct drm_fb_helper_surface_size *sizes)
+static int drm_fbdev_cma_deferred_io_mmap(struct fb_info *info,
+ struct vm_area_struct *vma)
+{
+ fb_deferred_io_mmap(info, vma);
+ vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
+
+ return 0;
+}
+
+static int drm_fbdev_cma_defio_init(struct fb_info *fbi,
+ struct drm_gem_cma_object *cma_obj)
+{
+ struct fb_deferred_io *fbdefio;
+ struct fb_ops *fbops;
+
+ /*
+ * Per device structures are needed because:
+ * fbops: fb_deferred_io_cleanup() clears fbops.fb_mmap
+ * fbdefio: individual delays
+ */
+ fbdefio = kzalloc(sizeof(*fbdefio), GFP_KERNEL);
+ fbops = kzalloc(sizeof(*fbops), GFP_KERNEL);
+ if (!fbdefio || !fbops) {
+ kfree(fbdefio);
+ return -ENOMEM;
+ }
+
+ /* can't be offset from vaddr since dirty() uses cma_obj */
+ fbi->screen_buffer = cma_obj->vaddr;
+ /* fb_deferred_io_fault() needs a physical address */
+ fbi->fix.smem_start = page_to_phys(virt_to_page(fbi->screen_buffer));
+
+ *fbops = *fbi->fbops;
+ fbi->fbops = fbops;
+
+ fbdefio->delay = msecs_to_jiffies(DEFAULT_FBDEFIO_DELAY_MS);
+ fbdefio->deferred_io = drm_fb_helper_deferred_io;
+ fbi->fbdefio = fbdefio;
+ fb_deferred_io_init(fbi);
+ fbi->fbops->fb_mmap = drm_fbdev_cma_deferred_io_mmap;
+
+ return 0;
+}
+
+static void drm_fbdev_cma_defio_fini(struct fb_info *fbi)
+{
+ if (!fbi->fbdefio)
+ return;
+
+ fb_deferred_io_cleanup(fbi);
+ kfree(fbi->fbdefio);
+ kfree(fbi->fbops);
+}
+
+/*
+ * For use in a (struct drm_fb_helper_funcs *)->fb_probe callback function that
+ * needs custom struct drm_framebuffer_funcs, like dirty() for deferred_io use.
+ */
+int drm_fbdev_cma_create_with_funcs(struct drm_fb_helper *helper,
+ struct drm_fb_helper_surface_size *sizes,
+ const struct drm_framebuffer_funcs *funcs)
{
struct drm_fbdev_cma *fbdev_cma = to_fbdev_cma(helper);
struct drm_mode_fb_cmd2 mode_cmd = { 0 };
@@ -270,7 +405,7 @@
goto err_gem_free_object;
}
- fbdev_cma->fb = drm_fb_cma_alloc(dev, &mode_cmd, &obj, 1);
+ fbdev_cma->fb = drm_fb_cma_alloc(dev, &mode_cmd, &obj, 1, funcs);
if (IS_ERR(fbdev_cma->fb)) {
dev_err(dev->dev, "Failed to allocate DRM framebuffer.\n");
ret = PTR_ERR(fbdev_cma->fb);
@@ -296,31 +431,48 @@
fbi->screen_size = size;
fbi->fix.smem_len = size;
+ if (funcs->dirty) {
+ ret = drm_fbdev_cma_defio_init(fbi, obj);
+ if (ret)
+ goto err_cma_destroy;
+ }
+
return 0;
+err_cma_destroy:
+ drm_framebuffer_unregister_private(&fbdev_cma->fb->fb);
+ drm_fb_cma_destroy(&fbdev_cma->fb->fb);
err_fb_info_destroy:
drm_fb_helper_release_fbi(helper);
err_gem_free_object:
dev->driver->gem_free_object(&obj->base);
return ret;
}
+EXPORT_SYMBOL(drm_fbdev_cma_create_with_funcs);
+
+static int drm_fbdev_cma_create(struct drm_fb_helper *helper,
+ struct drm_fb_helper_surface_size *sizes)
+{
+ return drm_fbdev_cma_create_with_funcs(helper, sizes, &drm_fb_cma_funcs);
+}
static const struct drm_fb_helper_funcs drm_fb_cma_helper_funcs = {
.fb_probe = drm_fbdev_cma_create,
};
/**
- * drm_fbdev_cma_init() - Allocate and initializes a drm_fbdev_cma struct
+ * drm_fbdev_cma_init_with_funcs() - Allocate and initializes a drm_fbdev_cma struct
* @dev: DRM device
* @preferred_bpp: Preferred bits per pixel for the device
* @num_crtc: Number of CRTCs
* @max_conn_count: Maximum number of connectors
+ * @funcs: fb helper functions, in particular fb_probe()
*
* Returns a newly allocated drm_fbdev_cma struct or a ERR_PTR.
*/
-struct drm_fbdev_cma *drm_fbdev_cma_init(struct drm_device *dev,
+struct drm_fbdev_cma *drm_fbdev_cma_init_with_funcs(struct drm_device *dev,
unsigned int preferred_bpp, unsigned int num_crtc,
- unsigned int max_conn_count)
+ unsigned int max_conn_count, const struct drm_fb_helper_funcs *funcs)
{
struct drm_fbdev_cma *fbdev_cma;
struct drm_fb_helper *helper;
@@ -334,7 +486,7 @@
helper = &fbdev_cma->fb_helper;
- drm_fb_helper_prepare(dev, helper, &drm_fb_cma_helper_funcs);
+ drm_fb_helper_prepare(dev, helper, funcs);
ret = drm_fb_helper_init(dev, helper, num_crtc, max_conn_count);
if (ret < 0) {
@@ -364,6 +516,24 @@
return ERR_PTR(ret);
}
+EXPORT_SYMBOL_GPL(drm_fbdev_cma_init_with_funcs);
+
+/**
+ * drm_fbdev_cma_init() - Allocate and initializes a drm_fbdev_cma struct
+ * @dev: DRM device
+ * @preferred_bpp: Preferred bits per pixel for the device
+ * @num_crtc: Number of CRTCs
+ * @max_conn_count: Maximum number of connectors
+ *
+ * Returns a newly allocated drm_fbdev_cma struct or a ERR_PTR.
+ */
+struct drm_fbdev_cma *drm_fbdev_cma_init(struct drm_device *dev,
+ unsigned int preferred_bpp, unsigned int num_crtc,
+ unsigned int max_conn_count)
+{
+ return drm_fbdev_cma_init_with_funcs(dev, preferred_bpp, num_crtc,
+ max_conn_count, &drm_fb_cma_helper_funcs);
+}
EXPORT_SYMBOL_GPL(drm_fbdev_cma_init);
/**
@@ -373,6 +543,7 @@
void drm_fbdev_cma_fini(struct drm_fbdev_cma *fbdev_cma)
{
drm_fb_helper_unregister_fbi(&fbdev_cma->fb_helper);
+ drm_fbdev_cma_defio_fini(fbdev_cma->fb_helper.fbdev);
drm_fb_helper_release_fbi(&fbdev_cma->fb_helper);
if (fbdev_cma->fb) {
diff --git a/drivers/gpu/drm/drm_fb_helper.c b/drivers/gpu/drm/drm_fb_helper.c
index 855108e..7c2eb75 100644
--- a/drivers/gpu/drm/drm_fb_helper.c
+++ b/drivers/gpu/drm/drm_fb_helper.c
@@ -84,6 +84,15 @@
* and set up an initial configuration using the detected hardware, drivers
* should call drm_fb_helper_single_add_all_connectors() followed by
* drm_fb_helper_initial_config().
+ *
+ * If &drm_framebuffer_funcs ->dirty is set, the
+ * drm_fb_helper_{cfb,sys}_{write,fillrect,copyarea,imageblit} functions will
+ * accumulate changes and schedule &drm_fb_helper ->dirty_work to run right
+ * away. This worker then calls the dirty() function ensuring that it will
+ * always run in process context since the fb_*() function could be running in
+ * atomic context. If drm_fb_helper_deferred_io() is used as the deferred_io
+ * callback it will also schedule dirty_work with the damage collected from the
+ * mmap page writes.
*/
/**
@@ -153,40 +162,13 @@
if (!fb_helper_connector)
return -ENOMEM;
+ drm_connector_reference(connector);
fb_helper_connector->connector = connector;
fb_helper->connector_info[fb_helper->connector_count++] = fb_helper_connector;
return 0;
}
EXPORT_SYMBOL(drm_fb_helper_add_one_connector);
-static void remove_from_modeset(struct drm_mode_set *set,
- struct drm_connector *connector)
-{
- int i, j;
-
- for (i = 0; i < set->num_connectors; i++) {
- if (set->connectors[i] == connector)
- break;
- }
-
- if (i == set->num_connectors)
- return;
-
- for (j = i + 1; j < set->num_connectors; j++) {
- set->connectors[j - 1] = set->connectors[j];
- }
- set->num_connectors--;
-
- /*
- * TODO maybe need to makes sure we set it back to !=NULL somewhere?
- */
- if (set->num_connectors == 0) {
- set->fb = NULL;
- drm_mode_destroy(connector->dev, set->mode);
- set->mode = NULL;
- }
-}
-
int drm_fb_helper_remove_one_connector(struct drm_fb_helper *fb_helper,
struct drm_connector *connector)
{
@@ -206,6 +188,7 @@
if (i == fb_helper->connector_count)
return -EINVAL;
fb_helper_connector = fb_helper->connector_info[i];
+ drm_connector_unreference(fb_helper_connector->connector);
for (j = i + 1; j < fb_helper->connector_count; j++) {
fb_helper->connector_info[j - 1] = fb_helper->connector_info[j];
@@ -213,10 +196,6 @@
fb_helper->connector_count--;
kfree(fb_helper_connector);
- /* also cleanup dangling references to the connector: */
- for (i = 0; i < fb_helper->crtc_count; i++)
- remove_from_modeset(&fb_helper->crtc_info[i].mode_set, connector);
-
return 0;
}
EXPORT_SYMBOL(drm_fb_helper_remove_one_connector);
@@ -626,8 +605,10 @@
{
int i;
- for (i = 0; i < helper->connector_count; i++)
+ for (i = 0; i < helper->connector_count; i++) {
+ drm_connector_unreference(helper->connector_info[i]->connector);
kfree(helper->connector_info[i]);
+ }
kfree(helper->connector_info);
for (i = 0; i < helper->crtc_count; i++) {
kfree(helper->crtc_info[i].mode_set.connectors);
@@ -637,6 +618,23 @@
kfree(helper->crtc_info);
}
+static void drm_fb_helper_dirty_work(struct work_struct *work)
+{
+ struct drm_fb_helper *helper = container_of(work, struct drm_fb_helper,
+ dirty_work);
+ struct drm_clip_rect *clip = &helper->dirty_clip;
+ struct drm_clip_rect clip_copy;
+ unsigned long flags;
+
+ spin_lock_irqsave(&helper->dirty_lock, flags);
+ clip_copy = *clip;
+ clip->x1 = clip->y1 = ~0;
+ clip->x2 = clip->y2 = 0;
+ spin_unlock_irqrestore(&helper->dirty_lock, flags);
+
+ helper->fb->funcs->dirty(helper->fb, NULL, 0, 0, &clip_copy, 1);
+}
+
/**
* drm_fb_helper_prepare - setup a drm_fb_helper structure
* @dev: DRM device
@@ -650,6 +648,9 @@
const struct drm_fb_helper_funcs *funcs)
{
INIT_LIST_HEAD(&helper->kernel_fb_list);
+ spin_lock_init(&helper->dirty_lock);
+ INIT_WORK(&helper->dirty_work, drm_fb_helper_dirty_work);
+ helper->dirty_clip.x1 = helper->dirty_clip.y1 = ~0;
helper->funcs = funcs;
helper->dev = dev;
}
@@ -834,6 +835,59 @@
}
EXPORT_SYMBOL(drm_fb_helper_unlink_fbi);
+static void drm_fb_helper_dirty(struct fb_info *info, u32 x, u32 y,
+ u32 width, u32 height)
+{
+ struct drm_fb_helper *helper = info->par;
+ struct drm_clip_rect *clip = &helper->dirty_clip;
+ unsigned long flags;
+
+ if (!helper->fb->funcs->dirty)
+ return;
+
+ spin_lock_irqsave(&helper->dirty_lock, flags);
+ clip->x1 = min_t(u32, clip->x1, x);
+ clip->y1 = min_t(u32, clip->y1, y);
+ clip->x2 = max_t(u32, clip->x2, x + width);
+ clip->y2 = max_t(u32, clip->y2, y + height);
+ spin_unlock_irqrestore(&helper->dirty_lock, flags);
+
+ schedule_work(&helper->dirty_work);
+}
+
+/**
+ * drm_fb_helper_deferred_io() - fbdev deferred_io callback function
+ * @info: fb_info struct pointer
+ * @pagelist: list of dirty mmap framebuffer pages
+ *
+ * This function is used as the &fb_deferred_io ->deferred_io
+ * callback function for flushing the fbdev mmap writes.
+ */
+void drm_fb_helper_deferred_io(struct fb_info *info,
+ struct list_head *pagelist)
+{
+ unsigned long start, end, min, max;
+ struct page *page;
+ u32 y1, y2;
+
+ min = ULONG_MAX;
+ max = 0;
+ list_for_each_entry(page, pagelist, lru) {
+ start = page->index << PAGE_SHIFT;
+ end = start + PAGE_SIZE - 1;
+ min = min(min, start);
+ max = max(max, end);
+ }
+
+ if (min < max) {
+ y1 = min / info->fix.line_length;
+ y2 = min_t(u32, DIV_ROUND_UP(max, info->fix.line_length),
+ info->var.yres);
+ drm_fb_helper_dirty(info, 0, y1, info->var.xres, y2 - y1);
+ }
+}
+EXPORT_SYMBOL(drm_fb_helper_deferred_io);
+
/**
* drm_fb_helper_sys_read - wrapper around fb_sys_read
* @info: fb_info struct pointer
@@ -862,7 +916,14 @@
ssize_t drm_fb_helper_sys_write(struct fb_info *info, const char __user *buf,
size_t count, loff_t *ppos)
{
- return fb_sys_write(info, buf, count, ppos);
+ ssize_t ret;
+
+ ret = fb_sys_write(info, buf, count, ppos);
+ if (ret > 0)
+ drm_fb_helper_dirty(info, 0, 0, info->var.xres,
+ info->var.yres);
+
+ return ret;
}
EXPORT_SYMBOL(drm_fb_helper_sys_write);
@@ -877,6 +938,8 @@
const struct fb_fillrect *rect)
{
sys_fillrect(info, rect);
+ drm_fb_helper_dirty(info, rect->dx, rect->dy,
+ rect->width, rect->height);
}
EXPORT_SYMBOL(drm_fb_helper_sys_fillrect);
@@ -891,6 +954,8 @@
const struct fb_copyarea *area)
{
sys_copyarea(info, area);
+ drm_fb_helper_dirty(info, area->dx, area->dy,
+ area->width, area->height);
}
EXPORT_SYMBOL(drm_fb_helper_sys_copyarea);
@@ -905,6 +970,8 @@
const struct fb_image *image)
{
sys_imageblit(info, image);
+ drm_fb_helper_dirty(info, image->dx, image->dy,
+ image->width, image->height);
}
EXPORT_SYMBOL(drm_fb_helper_sys_imageblit);
@@ -919,6 +986,8 @@
const struct fb_fillrect *rect)
{
cfb_fillrect(info, rect);
+ drm_fb_helper_dirty(info, rect->dx, rect->dy,
+ rect->width, rect->height);
}
EXPORT_SYMBOL(drm_fb_helper_cfb_fillrect);
@@ -933,6 +1002,8 @@
const struct fb_copyarea *area)
{
cfb_copyarea(info, area);
+ drm_fb_helper_dirty(info, area->dx, area->dy,
+ area->width, area->height);
}
EXPORT_SYMBOL(drm_fb_helper_cfb_copyarea);
@@ -947,6 +1018,8 @@
const struct fb_image *image)
{
cfb_imageblit(info, image);
+ drm_fb_helper_dirty(info, image->dx, image->dy,
+ image->width, image->height);
}
EXPORT_SYMBOL(drm_fb_helper_cfb_imageblit);
@@ -1895,7 +1968,6 @@
int n, int width, int height)
{
int c, o;
- struct drm_device *dev = fb_helper->dev;
struct drm_connector *connector;
const struct drm_connector_helper_funcs *connector_funcs;
struct drm_encoder *encoder;
@@ -1914,7 +1986,7 @@
if (modes[n] == NULL)
return best_score;
- crtcs = kzalloc(dev->mode_config.num_connector *
+ crtcs = kzalloc(fb_helper->connector_count *
sizeof(struct drm_fb_helper_crtc *), GFP_KERNEL);
if (!crtcs)
return best_score;
@@ -1960,7 +2032,7 @@
if (score > best_score) {
best_score = score;
memcpy(best_crtcs, crtcs,
- dev->mode_config.num_connector *
+ fb_helper->connector_count *
sizeof(struct drm_fb_helper_crtc *));
}
}
@@ -2104,8 +2176,8 @@
* cmdline option.
*
* The other option is to just disable fbdev emulation since very likely the
- * first modest from userspace will crash in the same way, and is even easier to
- * debug. This can be done by setting the drm_kms_helper.fbdev_emulation=0
+ * first modeset from userspace will crash in the same way, and is even easier
+ * to debug. This can be done by setting the drm_kms_helper.fbdev_emulation=0
* kernel cmdline option.
*
* RETURNS:
@@ -2150,7 +2222,7 @@
* hotplug interrupt).
*
* Note that drivers may call this even before calling
- * drm_fb_helper_initial_config but only aftert drm_fb_helper_init. This allows
+ * drm_fb_helper_initial_config but only after drm_fb_helper_init. This allows
* for a race-free fbcon setup and will make sure that the fbdev emulation will
* not miss any hotplug events.
*
diff --git a/drivers/gpu/drm/drm_fops.c b/drivers/gpu/drm/drm_fops.c
index aeef58e..7af7f8b 100644
--- a/drivers/gpu/drm/drm_fops.c
+++ b/drivers/gpu/drm/drm_fops.c
@@ -297,9 +297,9 @@
}
mutex_unlock(&dev->master_mutex);
- mutex_lock(&dev->struct_mutex);
+ mutex_lock(&dev->filelist_mutex);
list_add(&priv->lhead, &dev->filelist);
- mutex_unlock(&dev->struct_mutex);
+ mutex_unlock(&dev->filelist_mutex);
#ifdef __alpha__
/*
@@ -381,14 +381,26 @@
*/
static void drm_legacy_dev_reinit(struct drm_device *dev)
{
- if (drm_core_check_feature(dev, DRIVER_MODESET))
- return;
+ if (dev->irq_enabled)
+ drm_irq_uninstall(dev);
+
+ mutex_lock(&dev->struct_mutex);
+
+ drm_legacy_agp_clear(dev);
+
+ drm_legacy_sg_cleanup(dev);
+ drm_legacy_vma_flush(dev);
+ drm_legacy_dma_takedown(dev);
+
+ mutex_unlock(&dev->struct_mutex);
dev->sigdata.lock = NULL;
dev->context_flag = 0;
dev->last_context = 0;
dev->if_version = 0;
+
+ DRM_DEBUG("lastclose completed\n");
}
/*
@@ -400,7 +412,7 @@
*
* \sa drm_device
*/
-int drm_lastclose(struct drm_device * dev)
+void drm_lastclose(struct drm_device * dev)
{
DRM_DEBUG("\n");
@@ -408,23 +420,8 @@
dev->driver->lastclose(dev);
DRM_DEBUG("driver lastclose completed\n");
- if (dev->irq_enabled && !drm_core_check_feature(dev, DRIVER_MODESET))
- drm_irq_uninstall(dev);
-
- mutex_lock(&dev->struct_mutex);
-
- drm_agp_clear(dev);
-
- drm_legacy_sg_cleanup(dev);
- drm_legacy_vma_flush(dev);
- drm_legacy_dma_takedown(dev);
-
- mutex_unlock(&dev->struct_mutex);
-
- drm_legacy_dev_reinit(dev);
-
- DRM_DEBUG("lastclose completed\n");
- return 0;
+ if (!drm_core_check_feature(dev, DRIVER_MODESET))
+ drm_legacy_dev_reinit(dev);
}
/**
@@ -445,14 +442,16 @@
struct drm_file *file_priv = filp->private_data;
struct drm_minor *minor = file_priv->minor;
struct drm_device *dev = minor->dev;
- int retcode = 0;
mutex_lock(&drm_global_mutex);
DRM_DEBUG("open_count = %d\n", dev->open_count);
- mutex_lock(&dev->struct_mutex);
+ mutex_lock(&dev->filelist_mutex);
list_del(&file_priv->lhead);
+ mutex_unlock(&dev->filelist_mutex);
+
+ mutex_lock(&dev->struct_mutex);
if (file_priv->magic)
idr_remove(&file_priv->master->magic_map, file_priv->magic);
mutex_unlock(&dev->struct_mutex);
@@ -538,7 +537,7 @@
*/
if (!--dev->open_count) {
- retcode = drm_lastclose(dev);
+ drm_lastclose(dev);
if (drm_device_is_unplugged(dev))
drm_put_dev(dev);
}
@@ -546,7 +545,7 @@
drm_minor_release(minor);
- return retcode;
+ return 0;
}
EXPORT_SYMBOL(drm_release);
diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index da0c532..3215606 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -279,7 +279,6 @@
int
drm_gem_handle_delete(struct drm_file *filp, u32 handle)
{
- struct drm_device *dev;
struct drm_gem_object *obj;
/* This is gross. The idr system doesn't let us try a delete and
@@ -294,18 +293,19 @@
spin_lock(&filp->table_lock);
/* Check if we currently have a reference on the object */
- obj = idr_find(&filp->object_idr, handle);
- if (obj == NULL) {
- spin_unlock(&filp->table_lock);
+ obj = idr_replace(&filp->object_idr, NULL, handle);
+ spin_unlock(&filp->table_lock);
+ if (IS_ERR_OR_NULL(obj))
return -EINVAL;
- }
- dev = obj->dev;
- /* Release reference and decrement refcount. */
+ /* Release driver's reference and decrement refcount. */
+ drm_gem_object_release_handle(handle, obj, filp);
+
+ /* And finally make the handle available for future allocations. */
+ spin_lock(&filp->table_lock);
idr_remove(&filp->object_idr, handle);
spin_unlock(&filp->table_lock);
- drm_gem_object_release_handle(handle, obj, filp);
return 0;
}
EXPORT_SYMBOL(drm_gem_handle_delete);
@@ -422,6 +422,10 @@
* @obj: obj in question
*
* This routine frees fake offsets allocated by drm_gem_create_mmap_offset().
+ *
+ * Note that drm_gem_object_release() already calls this function, so drivers
+ * don't have to take care of releasing the mmap offset themselves when freeing
+ * the GEM object.
*/
void
drm_gem_free_mmap_offset(struct drm_gem_object *obj)
@@ -445,6 +449,9 @@
* This routine allocates and attaches a fake offset for @obj, in cases where
* the virtual size differs from the physical size (ie. obj->size). Otherwise
* just use drm_gem_create_mmap_offset().
+ *
+ * This function is idempotent and handles an already allocated mmap offset
+ * transparently. Drivers do not need to check for this case.
*/
int
drm_gem_create_mmap_offset_size(struct drm_gem_object *obj, size_t size)
@@ -466,6 +473,9 @@
* structures.
*
* This routine allocates and attaches a fake offset for @obj.
+ *
+ * Drivers can call drm_gem_free_mmap_offset() before freeing @obj to release
+ * the fake offset again.
*/
int drm_gem_create_mmap_offset(struct drm_gem_object *obj)
{
@@ -578,7 +588,6 @@
/**
* drm_gem_object_lookup - look up a GEM object from it's handle
- * @dev: DRM device
* @filp: DRM file private date
* @handle: userspace handle
*
@@ -588,8 +597,7 @@
* otherwise.
*/
struct drm_gem_object *
-drm_gem_object_lookup(struct drm_device *dev, struct drm_file *filp,
- u32 handle)
+drm_gem_object_lookup(struct drm_file *filp, u32 handle)
{
struct drm_gem_object *obj;
@@ -597,12 +605,8 @@
/* Check if we currently have a reference on the object */
obj = idr_find(&filp->object_idr, handle);
- if (obj == NULL) {
- spin_unlock(&filp->table_lock);
- return NULL;
- }
-
- drm_gem_object_reference(obj);
+ if (obj)
+ drm_gem_object_reference(obj);
spin_unlock(&filp->table_lock);
@@ -655,7 +659,7 @@
if (!drm_core_check_feature(dev, DRIVER_GEM))
return -ENODEV;
- obj = drm_gem_object_lookup(dev, file_priv, args->handle);
+ obj = drm_gem_object_lookup(file_priv, args->handle);
if (obj == NULL)
return -ENOENT;
@@ -759,6 +763,13 @@
idr_destroy(&file_private->object_idr);
}
+/**
+ * drm_gem_object_release - release GEM buffer object resources
+ * @obj: GEM buffer object
+ *
+ * This releases any structures and resources used by @obj and is the invers of
+ * drm_gem_object_init().
+ */
void
drm_gem_object_release(struct drm_gem_object *obj)
{
@@ -787,14 +798,67 @@
container_of(kref, struct drm_gem_object, refcount);
struct drm_device *dev = obj->dev;
- WARN_ON(!mutex_is_locked(&dev->struct_mutex));
+ if (dev->driver->gem_free_object_unlocked) {
+ dev->driver->gem_free_object_unlocked(obj);
+ } else if (dev->driver->gem_free_object) {
+ WARN_ON(!mutex_is_locked(&dev->struct_mutex));
- if (dev->driver->gem_free_object != NULL)
dev->driver->gem_free_object(obj);
+ }
}
EXPORT_SYMBOL(drm_gem_object_free);
/**
+ * drm_gem_object_unreference_unlocked - release a GEM BO reference
+ * @obj: GEM buffer object
+ *
+ * This releases a reference to @obj. Callers must not hold the
+ * dev->struct_mutex lock when calling this function.
+ *
+ * See also __drm_gem_object_unreference().
+ */
+void
+drm_gem_object_unreference_unlocked(struct drm_gem_object *obj)
+{
+ struct drm_device *dev;
+
+ if (!obj)
+ return;
+
+ dev = obj->dev;
+ might_lock(&dev->struct_mutex);
+
+ if (dev->driver->gem_free_object_unlocked)
+ kref_put(&obj->refcount, drm_gem_object_free);
+ else if (kref_put_mutex(&obj->refcount, drm_gem_object_free,
+ &dev->struct_mutex))
+ mutex_unlock(&dev->struct_mutex);
+}
+EXPORT_SYMBOL(drm_gem_object_unreference_unlocked);
+
+/**
+ * drm_gem_object_unreference - release a GEM BO reference
+ * @obj: GEM buffer object
+ *
+ * This releases a reference to @obj. Callers must hold the dev->struct_mutex
+ * lock when calling this function, even when the driver doesn't use
+ * dev->struct_mutex for anything.
+ *
+ * For drivers not encumbered with legacy locking use
+ * drm_gem_object_unreference_unlocked() instead.
+ */
+void
+drm_gem_object_unreference(struct drm_gem_object *obj)
+{
+ if (obj) {
+ WARN_ON(!mutex_is_locked(&obj->dev->struct_mutex));
+
+ kref_put(&obj->refcount, drm_gem_object_free);
+ }
+}
+EXPORT_SYMBOL(drm_gem_object_unreference);
+
+/**
* drm_gem_vm_open - vma->ops->open implementation for GEM
* @vma: VM area structure
*
diff --git a/drivers/gpu/drm/drm_gem_cma_helper.c b/drivers/gpu/drm/drm_gem_cma_helper.c
index 1f500a1..e1ab008 100644
--- a/drivers/gpu/drm/drm_gem_cma_helper.c
+++ b/drivers/gpu/drm/drm_gem_cma_helper.c
@@ -291,7 +291,7 @@
{
struct drm_gem_object *gem_obj;
- gem_obj = drm_gem_object_lookup(drm, file_priv, handle);
+ gem_obj = drm_gem_object_lookup(file_priv, handle);
if (!gem_obj) {
dev_err(drm->dev, "failed to lookup GEM object\n");
return -EINVAL;
diff --git a/drivers/gpu/drm/drm_info.c b/drivers/gpu/drm/drm_info.c
index cbb4fc0..5d469b2 100644
--- a/drivers/gpu/drm/drm_info.c
+++ b/drivers/gpu/drm/drm_info.c
@@ -174,7 +174,7 @@
/* dev->filelist is sorted youngest first, but we want to present
* oldest first (i.e. kernel, servers, clients), so walk backwardss.
*/
- mutex_lock(&dev->struct_mutex);
+ mutex_lock(&dev->filelist_mutex);
list_for_each_entry_reverse(priv, &dev->filelist, lhead) {
struct task_struct *task;
@@ -190,7 +190,7 @@
priv->magic);
rcu_read_unlock();
}
- mutex_unlock(&dev->struct_mutex);
+ mutex_unlock(&dev->filelist_mutex);
return 0;
}
diff --git a/drivers/gpu/drm/drm_internal.h b/drivers/gpu/drm/drm_internal.h
index 43cbda3..902cf6a 100644
--- a/drivers/gpu/drm/drm_internal.h
+++ b/drivers/gpu/drm/drm_internal.h
@@ -26,7 +26,7 @@
/* drm_fops.c */
extern struct mutex drm_global_mutex;
-int drm_lastclose(struct drm_device *dev);
+void drm_lastclose(struct drm_device *dev);
/* drm_pci.c */
int drm_pci_set_unique(struct drm_device *dev,
@@ -37,8 +37,6 @@
/* drm_vm.c */
int drm_vma_info(struct seq_file *m, void *data);
-void drm_vm_open_locked(struct drm_device *dev, struct vm_area_struct *vma);
-void drm_vm_close_locked(struct drm_device *dev, struct vm_area_struct *vma);
/* drm_prime.c */
int drm_prime_handle_to_fd_ioctl(struct drm_device *dev, void *data,
diff --git a/drivers/gpu/drm/drm_ioctl.c b/drivers/gpu/drm/drm_ioctl.c
index 8ce2a0c..b7a39771c 100644
--- a/drivers/gpu/drm/drm_ioctl.c
+++ b/drivers/gpu/drm/drm_ioctl.c
@@ -150,58 +150,6 @@
}
/*
- * Get a mapping information.
- *
- * \param inode device inode.
- * \param file_priv DRM file private.
- * \param cmd command.
- * \param arg user argument, pointing to a drm_map structure.
- *
- * \return zero on success or a negative number on failure.
- *
- * Searches for the mapping with the specified offset and copies its information
- * into userspace
- */
-static int drm_getmap(struct drm_device *dev, void *data,
- struct drm_file *file_priv)
-{
- struct drm_map *map = data;
- struct drm_map_list *r_list = NULL;
- struct list_head *list;
- int idx;
- int i;
-
- idx = map->offset;
- if (idx < 0)
- return -EINVAL;
-
- i = 0;
- mutex_lock(&dev->struct_mutex);
- list_for_each(list, &dev->maplist) {
- if (i == idx) {
- r_list = list_entry(list, struct drm_map_list, head);
- break;
- }
- i++;
- }
- if (!r_list || !r_list->map) {
- mutex_unlock(&dev->struct_mutex);
- return -EINVAL;
- }
-
- map->offset = r_list->map->offset;
- map->size = r_list->map->size;
- map->type = r_list->map->type;
- map->flags = r_list->map->flags;
- map->handle = (void *)(unsigned long) r_list->user_token;
- map->mtrr = arch_phys_wc_index(r_list->map->mtrr);
-
- mutex_unlock(&dev->struct_mutex);
-
- return 0;
-}
-
-/*
* Get client information.
*
* \param inode device inode.
@@ -558,7 +506,7 @@
DRM_IOCTL_DEF(DRM_IOCTL_GET_UNIQUE, drm_getunique, 0),
DRM_IOCTL_DEF(DRM_IOCTL_GET_MAGIC, drm_getmagic, 0),
DRM_IOCTL_DEF(DRM_IOCTL_IRQ_BUSID, drm_irq_by_busid, DRM_MASTER|DRM_ROOT_ONLY),
- DRM_IOCTL_DEF(DRM_IOCTL_GET_MAP, drm_getmap, DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_IOCTL_GET_MAP, drm_legacy_getmap_ioctl, DRM_UNLOCKED),
DRM_IOCTL_DEF(DRM_IOCTL_GET_CLIENT, drm_getclient, DRM_UNLOCKED),
DRM_IOCTL_DEF(DRM_IOCTL_GET_STATS, drm_getstats, DRM_UNLOCKED),
DRM_IOCTL_DEF(DRM_IOCTL_GET_CAP, drm_getcap, DRM_UNLOCKED|DRM_RENDER_ALLOW),
diff --git a/drivers/gpu/drm/drm_irq.c b/drivers/gpu/drm/drm_irq.c
index 881c5a6..0fac801 100644
--- a/drivers/gpu/drm/drm_irq.c
+++ b/drivers/gpu/drm/drm_irq.c
@@ -348,9 +348,6 @@
unsigned int pipe = vblank->pipe;
unsigned long irqflags;
- if (!dev->vblank_disable_allowed)
- return;
-
spin_lock_irqsave(&dev->vbl_lock, irqflags);
if (atomic_read(&vblank->refcount) == 0 && vblank->enabled) {
DRM_DEBUG("disabling vblank on crtc %u\n", pipe);
@@ -437,8 +434,6 @@
"get_vblank_timestamp == NULL\n");
}
- dev->vblank_disable_allowed = false;
-
return 0;
err:
@@ -863,10 +858,7 @@
/* Subtract time delta from raw timestamp to get final
* vblank_time timestamp for end of vblank.
*/
- if (delta_ns < 0)
- etime = ktime_add_ns(etime, -delta_ns);
- else
- etime = ktime_sub_ns(etime, delta_ns);
+ etime = ktime_sub_ns(etime, delta_ns);
*vblank_time = ktime_to_timeval(etime);
DRM_DEBUG_VBL("crtc %u : v 0x%x p(%d,%d)@ %ld.%ld -> %ld.%ld [e %d us, %d rep]\n",
@@ -1588,7 +1580,6 @@
if (vblank->inmodeset) {
spin_lock_irqsave(&dev->vbl_lock, irqflags);
- dev->vblank_disable_allowed = true;
drm_reset_vblank_timestamp(dev, pipe);
spin_unlock_irqrestore(&dev->vbl_lock, irqflags);
diff --git a/drivers/gpu/drm/drm_legacy.h b/drivers/gpu/drm/drm_legacy.h
index 9b73178..d3b6ee3 100644
--- a/drivers/gpu/drm/drm_legacy.h
+++ b/drivers/gpu/drm/drm_legacy.h
@@ -63,6 +63,8 @@
#define DRM_MAP_HASH_OFFSET 0x10000000
+int drm_legacy_getmap_ioctl(struct drm_device *dev, void *data,
+ struct drm_file *file_priv);
int drm_legacy_addmap_ioctl(struct drm_device *d, void *v, struct drm_file *f);
int drm_legacy_rmmap_ioctl(struct drm_device *d, void *v, struct drm_file *f);
int drm_legacy_addbufs(struct drm_device *d, void *v, struct drm_file *f);
diff --git a/drivers/gpu/drm/drm_modes.c b/drivers/gpu/drm/drm_modes.c
index f7448a5..7def3d5 100644
--- a/drivers/gpu/drm/drm_modes.c
+++ b/drivers/gpu/drm/drm_modes.c
@@ -98,7 +98,7 @@
if (!mode)
return;
- drm_mode_object_put(dev, &mode->base);
+ drm_mode_object_unregister(dev, &mode->base);
kfree(mode);
}
diff --git a/drivers/gpu/drm/drm_panel.c b/drivers/gpu/drm/drm_panel.c
index 2ef988e..3dfe3c8 100644
--- a/drivers/gpu/drm/drm_panel.c
+++ b/drivers/gpu/drm/drm_panel.c
@@ -30,12 +30,36 @@
static DEFINE_MUTEX(panel_lock);
static LIST_HEAD(panel_list);
+/**
+ * DOC: drm panel
+ *
+ * The DRM panel helpers allow drivers to register panel objects with a
+ * central registry and provide functions to retrieve those panels in display
+ * drivers.
+ */
+
+/**
+ * drm_panel_init - initialize a panel
+ * @panel: DRM panel
+ *
+ * Sets up internal fields of the panel so that it can subsequently be added
+ * to the registry.
+ */
void drm_panel_init(struct drm_panel *panel)
{
INIT_LIST_HEAD(&panel->list);
}
EXPORT_SYMBOL(drm_panel_init);
+/**
+ * drm_panel_add - add a panel to the global registry
+ * @panel: panel to add
+ *
+ * Add a panel to the global registry so that it can be looked up by display
+ * drivers.
+ *
+ * Return: 0 on success or a negative error code on failure.
+ */
int drm_panel_add(struct drm_panel *panel)
{
mutex_lock(&panel_lock);
@@ -46,6 +70,12 @@
}
EXPORT_SYMBOL(drm_panel_add);
+/**
+ * drm_panel_remove - remove a panel from the global registry
+ * @panel: DRM panel
+ *
+ * Removes a panel from the global registry.
+ */
void drm_panel_remove(struct drm_panel *panel)
{
mutex_lock(&panel_lock);
@@ -54,6 +84,18 @@
}
EXPORT_SYMBOL(drm_panel_remove);
+/**
+ * drm_panel_attach - attach a panel to a connector
+ * @panel: DRM panel
+ * @connector: DRM connector
+ *
+ * After obtaining a pointer to a DRM panel a display driver calls this
+ * function to attach a panel to a connector.
+ *
+ * An error is returned if the panel is already attached to another connector.
+ *
+ * Return: 0 on success or a negative error code on failure.
+ */
int drm_panel_attach(struct drm_panel *panel, struct drm_connector *connector)
{
if (panel->connector)
@@ -66,6 +108,15 @@
}
EXPORT_SYMBOL(drm_panel_attach);
+/**
+ * drm_panel_detach - detach a panel from a connector
+ * @panel: DRM panel
+ *
+ * Detaches a panel from the connector it is attached to. If a panel is not
+ * attached to any connector this is effectively a no-op.
+ *
+ * Return: 0 on success or a negative error code on failure.
+ */
int drm_panel_detach(struct drm_panel *panel)
{
panel->connector = NULL;
@@ -76,6 +127,16 @@
EXPORT_SYMBOL(drm_panel_detach);
#ifdef CONFIG_OF
+/**
+ * of_drm_find_panel - look up a panel using a device tree node
+ * @np: device tree node of the panel
+ *
+ * Searches the set of registered panels for one that matches the given device
+ * tree node. If a matching panel is found, return a pointer to it.
+ *
+ * Return: A pointer to the panel registered for the specified device tree
+ * node or NULL if no panel matching the device tree node can be found.
+ */
struct drm_panel *of_drm_find_panel(struct device_node *np)
{
struct drm_panel *panel;
diff --git a/drivers/gpu/drm/drm_pci.c b/drivers/gpu/drm/drm_pci.c
index a1fff11..29d5a54 100644
--- a/drivers/gpu/drm/drm_pci.c
+++ b/drivers/gpu/drm/drm_pci.c
@@ -250,7 +250,7 @@
{
if (dev->agp) {
arch_phys_wc_del(dev->agp->agp_mtrr);
- drm_agp_clear(dev);
+ drm_legacy_agp_clear(dev);
kfree(dev->agp);
dev->agp = NULL;
}
diff --git a/drivers/gpu/drm/drm_prime.c b/drivers/gpu/drm/drm_prime.c
index df6cdc7..aab0f3f 100644
--- a/drivers/gpu/drm/drm_prime.c
+++ b/drivers/gpu/drm/drm_prime.c
@@ -407,7 +407,7 @@
struct dma_buf *dmabuf;
mutex_lock(&file_priv->prime.lock);
- obj = drm_gem_object_lookup(dev, file_priv, handle);
+ obj = drm_gem_object_lookup(file_priv, handle);
if (!obj) {
ret = -ENOENT;
goto out_unlock;
diff --git a/drivers/gpu/drm/drm_probe_helper.c b/drivers/gpu/drm/drm_probe_helper.c
index e714b5a..0329080 100644
--- a/drivers/gpu/drm/drm_probe_helper.c
+++ b/drivers/gpu/drm/drm_probe_helper.c
@@ -264,10 +264,8 @@
count = drm_add_edid_modes(connector, edid);
drm_edid_to_eld(connector, edid);
} else {
-#ifdef CONFIG_DRM_LOAD_EDID_FIRMWARE
count = drm_load_edid_firmware(connector);
if (count == 0)
-#endif
count = (*connector_funcs->get_modes)(connector);
}
diff --git a/drivers/gpu/drm/drm_sysfs.c b/drivers/gpu/drm/drm_sysfs.c
index d503f8e..fa7fadc 100644
--- a/drivers/gpu/drm/drm_sysfs.c
+++ b/drivers/gpu/drm/drm_sysfs.c
@@ -208,9 +208,12 @@
char *buf)
{
struct drm_connector *connector = to_drm_connector(device);
+ enum drm_connector_status status;
+
+ status = READ_ONCE(connector->status);
return snprintf(buf, PAGE_SIZE, "%s\n",
- drm_get_connector_status_name(connector->status));
+ drm_get_connector_status_name(status));
}
static ssize_t dpms_show(struct device *device,
@@ -231,9 +234,11 @@
char *buf)
{
struct drm_connector *connector = to_drm_connector(device);
+ bool enabled;
- return snprintf(buf, PAGE_SIZE, "%s\n", connector->encoder ? "enabled" :
- "disabled");
+ enabled = READ_ONCE(connector->encoder);
+
+ return snprintf(buf, PAGE_SIZE, enabled ? "enabled\n" : "disabled\n");
}
static ssize_t edid_show(struct file *filp, struct kobject *kobj,
@@ -287,102 +292,6 @@
return written;
}
-static ssize_t tv_subconnector_show(struct device *device,
- struct device_attribute *attr,
- char *buf)
-{
- struct drm_connector *connector = to_drm_connector(device);
- struct drm_device *dev = connector->dev;
- struct drm_property *prop;
- uint64_t subconnector;
- int ret;
-
- prop = dev->mode_config.tv_subconnector_property;
- if (!prop) {
- DRM_ERROR("Unable to find subconnector property\n");
- return 0;
- }
-
- ret = drm_object_property_get_value(&connector->base, prop, &subconnector);
- if (ret)
- return 0;
-
- return snprintf(buf, PAGE_SIZE, "%s",
- drm_get_tv_subconnector_name((int)subconnector));
-}
-
-static ssize_t tv_select_subconnector_show(struct device *device,
- struct device_attribute *attr,
- char *buf)
-{
- struct drm_connector *connector = to_drm_connector(device);
- struct drm_device *dev = connector->dev;
- struct drm_property *prop;
- uint64_t subconnector;
- int ret;
-
- prop = dev->mode_config.tv_select_subconnector_property;
- if (!prop) {
- DRM_ERROR("Unable to find select subconnector property\n");
- return 0;
- }
-
- ret = drm_object_property_get_value(&connector->base, prop, &subconnector);
- if (ret)
- return 0;
-
- return snprintf(buf, PAGE_SIZE, "%s",
- drm_get_tv_select_name((int)subconnector));
-}
-
-static ssize_t dvii_subconnector_show(struct device *device,
- struct device_attribute *attr,
- char *buf)
-{
- struct drm_connector *connector = to_drm_connector(device);
- struct drm_device *dev = connector->dev;
- struct drm_property *prop;
- uint64_t subconnector;
- int ret;
-
- prop = dev->mode_config.dvi_i_subconnector_property;
- if (!prop) {
- DRM_ERROR("Unable to find subconnector property\n");
- return 0;
- }
-
- ret = drm_object_property_get_value(&connector->base, prop, &subconnector);
- if (ret)
- return 0;
-
- return snprintf(buf, PAGE_SIZE, "%s",
- drm_get_dvi_i_subconnector_name((int)subconnector));
-}
-
-static ssize_t dvii_select_subconnector_show(struct device *device,
- struct device_attribute *attr,
- char *buf)
-{
- struct drm_connector *connector = to_drm_connector(device);
- struct drm_device *dev = connector->dev;
- struct drm_property *prop;
- uint64_t subconnector;
- int ret;
-
- prop = dev->mode_config.dvi_i_select_subconnector_property;
- if (!prop) {
- DRM_ERROR("Unable to find select subconnector property\n");
- return 0;
- }
-
- ret = drm_object_property_get_value(&connector->base, prop, &subconnector);
- if (ret)
- return 0;
-
- return snprintf(buf, PAGE_SIZE, "%s",
- drm_get_dvi_i_select_name((int)subconnector));
-}
-
static DEVICE_ATTR_RW(status);
static DEVICE_ATTR_RO(enabled);
static DEVICE_ATTR_RO(dpms);
@@ -396,54 +305,6 @@
NULL
};
-static DEVICE_ATTR_RO(tv_subconnector);
-static DEVICE_ATTR_RO(tv_select_subconnector);
-
-static struct attribute *connector_tv_dev_attrs[] = {
- &dev_attr_tv_subconnector.attr,
- &dev_attr_tv_select_subconnector.attr,
- NULL
-};
-
-static DEVICE_ATTR_RO(dvii_subconnector);
-static DEVICE_ATTR_RO(dvii_select_subconnector);
-
-static struct attribute *connector_dvii_dev_attrs[] = {
- &dev_attr_dvii_subconnector.attr,
- &dev_attr_dvii_select_subconnector.attr,
- NULL
-};
-
-/* Connector type related helpers */
-static int kobj_connector_type(struct kobject *kobj)
-{
- struct device *dev = kobj_to_dev(kobj);
- struct drm_connector *connector = to_drm_connector(dev);
-
- return connector->connector_type;
-}
-
-static umode_t connector_is_dvii(struct kobject *kobj,
- struct attribute *attr, int idx)
-{
- return kobj_connector_type(kobj) == DRM_MODE_CONNECTOR_DVII ?
- attr->mode : 0;
-}
-
-static umode_t connector_is_tv(struct kobject *kobj,
- struct attribute *attr, int idx)
-{
- switch (kobj_connector_type(kobj)) {
- case DRM_MODE_CONNECTOR_Composite:
- case DRM_MODE_CONNECTOR_SVIDEO:
- case DRM_MODE_CONNECTOR_Component:
- case DRM_MODE_CONNECTOR_TV:
- return attr->mode;
- }
-
- return 0;
-}
-
static struct bin_attribute edid_attr = {
.attr.name = "edid",
.attr.mode = 0444,
@@ -461,20 +322,8 @@
.bin_attrs = connector_bin_attrs,
};
-static const struct attribute_group connector_tv_dev_group = {
- .attrs = connector_tv_dev_attrs,
- .is_visible = connector_is_tv,
-};
-
-static const struct attribute_group connector_dvii_dev_group = {
- .attrs = connector_dvii_dev_attrs,
- .is_visible = connector_is_dvii,
-};
-
static const struct attribute_group *connector_dev_groups[] = {
&connector_dev_group,
- &connector_tv_dev_group,
- &connector_dvii_dev_group,
NULL
};
diff --git a/drivers/gpu/drm/drm_vm.c b/drivers/gpu/drm/drm_vm.c
index f90bd5f..ac9f4b3 100644
--- a/drivers/gpu/drm/drm_vm.c
+++ b/drivers/gpu/drm/drm_vm.c
@@ -395,16 +395,8 @@
.close = drm_vm_close,
};
-/**
- * \c open method for shared virtual memory.
- *
- * \param vma virtual memory area.
- *
- * Create a new drm_vma_entry structure as the \p vma private data entry and
- * add it to drm_device::vmalist.
- */
-void drm_vm_open_locked(struct drm_device *dev,
- struct vm_area_struct *vma)
+static void drm_vm_open_locked(struct drm_device *dev,
+ struct vm_area_struct *vma)
{
struct drm_vma_entry *vma_entry;
@@ -429,8 +421,8 @@
mutex_unlock(&dev->struct_mutex);
}
-void drm_vm_close_locked(struct drm_device *dev,
- struct vm_area_struct *vma)
+static void drm_vm_close_locked(struct drm_device *dev,
+ struct vm_area_struct *vma)
{
struct drm_vma_entry *pt, *temp;
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_drv.c b/drivers/gpu/drm/etnaviv/etnaviv_drv.c
index e885898..3d4f56d 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_drv.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_drv.c
@@ -314,7 +314,7 @@
if (args->op & ~(ETNA_PREP_READ | ETNA_PREP_WRITE | ETNA_PREP_NOSYNC))
return -EINVAL;
- obj = drm_gem_object_lookup(dev, file, args->handle);
+ obj = drm_gem_object_lookup(file, args->handle);
if (!obj)
return -ENOENT;
@@ -335,7 +335,7 @@
if (args->flags)
return -EINVAL;
- obj = drm_gem_object_lookup(dev, file, args->handle);
+ obj = drm_gem_object_lookup(file, args->handle);
if (!obj)
return -ENOENT;
@@ -356,7 +356,7 @@
if (args->pad)
return -EINVAL;
- obj = drm_gem_object_lookup(dev, file, args->handle);
+ obj = drm_gem_object_lookup(file, args->handle);
if (!obj)
return -ENOENT;
@@ -441,7 +441,7 @@
if (!gpu)
return -ENXIO;
- obj = drm_gem_object_lookup(dev, file, args->handle);
+ obj = drm_gem_object_lookup(file, args->handle);
if (!obj)
return -ENOENT;
@@ -497,7 +497,7 @@
.open = etnaviv_open,
.preclose = etnaviv_preclose,
.set_busid = drm_platform_set_busid,
- .gem_free_object = etnaviv_gem_free_object,
+ .gem_free_object_unlocked = etnaviv_gem_free_object,
.gem_vm_ops = &vm_ops,
.prime_handle_to_fd = drm_gem_prime_handle_to_fd,
.prime_fd_to_handle = drm_gem_prime_fd_to_handle,
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem.c b/drivers/gpu/drm/etnaviv/etnaviv_gem.c
index 281c6ec..df9bcba 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gem.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gem.c
@@ -129,10 +129,9 @@
/* when we start tracking the pin count, then do something here */
}
-static int etnaviv_gem_mmap_obj(struct drm_gem_object *obj,
+static int etnaviv_gem_mmap_obj(struct etnaviv_gem_object *etnaviv_obj,
struct vm_area_struct *vma)
{
- struct etnaviv_gem_object *etnaviv_obj = to_etnaviv_bo(obj);
pgprot_t vm_page_prot;
vma->vm_flags &= ~VM_PFNMAP;
@@ -151,9 +150,9 @@
* in particular in the case of mmap'd dmabufs)
*/
fput(vma->vm_file);
- get_file(obj->filp);
+ get_file(etnaviv_obj->base.filp);
vma->vm_pgoff = 0;
- vma->vm_file = obj->filp;
+ vma->vm_file = etnaviv_obj->base.filp;
vma->vm_page_prot = vm_page_prot;
}
@@ -173,7 +172,7 @@
}
obj = to_etnaviv_bo(vma->vm_private_data);
- return etnaviv_gem_mmap_obj(vma->vm_private_data, vma);
+ return obj->ops->mmap(obj, vma);
}
int etnaviv_gem_fault(struct vm_area_struct *vma, struct vm_fault *vmf)
@@ -545,6 +544,7 @@
.get_pages = etnaviv_gem_shmem_get_pages,
.release = etnaviv_gem_shmem_release,
.vmap = etnaviv_gem_vmap_impl,
+ .mmap = etnaviv_gem_mmap_obj,
};
void etnaviv_gem_free_object(struct drm_gem_object *obj)
@@ -886,10 +886,17 @@
put_task_struct(etnaviv_obj->userptr.task);
}
+static int etnaviv_gem_userptr_mmap_obj(struct etnaviv_gem_object *etnaviv_obj,
+ struct vm_area_struct *vma)
+{
+ return -EINVAL;
+}
+
static const struct etnaviv_gem_ops etnaviv_gem_userptr_ops = {
.get_pages = etnaviv_gem_userptr_get_pages,
.release = etnaviv_gem_userptr_release,
.vmap = etnaviv_gem_vmap_impl,
+ .mmap = etnaviv_gem_userptr_mmap_obj,
};
int etnaviv_gem_new_userptr(struct drm_device *dev, struct drm_file *file,
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem.h b/drivers/gpu/drm/etnaviv/etnaviv_gem.h
index 02665d8..e63ff11 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gem.h
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gem.h
@@ -79,6 +79,7 @@
int (*get_pages)(struct etnaviv_gem_object *);
void (*release)(struct etnaviv_gem_object *);
void *(*vmap)(struct etnaviv_gem_object *);
+ int (*mmap)(struct etnaviv_gem_object *, struct vm_area_struct *);
};
static inline bool is_active(struct etnaviv_gem_object *etnaviv_obj)
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem_prime.c b/drivers/gpu/drm/etnaviv/etnaviv_gem_prime.c
index 4e67395..b93618c 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gem_prime.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gem_prime.c
@@ -84,10 +84,17 @@
return dma_buf_vmap(etnaviv_obj->base.import_attach->dmabuf);
}
+static int etnaviv_gem_prime_mmap_obj(struct etnaviv_gem_object *etnaviv_obj,
+ struct vm_area_struct *vma)
+{
+ return dma_buf_mmap(etnaviv_obj->base.dma_buf, vma, 0);
+}
+
static const struct etnaviv_gem_ops etnaviv_gem_prime_ops = {
/* .get_pages should never be called */
.release = etnaviv_gem_prime_release,
.vmap = etnaviv_gem_prime_vmap_impl,
+ .mmap = etnaviv_gem_prime_mmap_obj,
};
struct drm_gem_object *etnaviv_gem_prime_import_sg_table(struct drm_device *dev,
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
index 236ada9..afdd55d 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
@@ -28,11 +28,6 @@
#define BO_LOCKED 0x4000
#define BO_PINNED 0x2000
-static inline void __user *to_user_ptr(u64 address)
-{
- return (void __user *)(uintptr_t)address;
-}
-
static struct etnaviv_gem_submit *submit_create(struct drm_device *dev,
struct etnaviv_gpu *gpu, size_t nr)
{
@@ -347,21 +342,21 @@
cmdbuf->exec_state = args->exec_state;
cmdbuf->ctx = file->driver_priv;
- ret = copy_from_user(bos, to_user_ptr(args->bos),
+ ret = copy_from_user(bos, u64_to_user_ptr(args->bos),
args->nr_bos * sizeof(*bos));
if (ret) {
ret = -EFAULT;
goto err_submit_cmds;
}
- ret = copy_from_user(relocs, to_user_ptr(args->relocs),
+ ret = copy_from_user(relocs, u64_to_user_ptr(args->relocs),
args->nr_relocs * sizeof(*relocs));
if (ret) {
ret = -EFAULT;
goto err_submit_cmds;
}
- ret = copy_from_user(stream, to_user_ptr(args->stream),
+ ret = copy_from_user(stream, u64_to_user_ptr(args->stream),
args->stream_size);
if (ret) {
ret = -EFAULT;
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
index d8a9a9c..ff6aa5d 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
@@ -1528,8 +1528,8 @@
INIT_WORK(&gpu->recover_work, recover_worker);
init_waitqueue_head(&gpu->fence_event);
- setup_timer(&gpu->hangcheck_timer, hangcheck_handler,
- (unsigned long)gpu);
+ setup_deferrable_timer(&gpu->hangcheck_timer, hangcheck_handler,
+ (unsigned long)gpu);
priv->gpu[priv->num_gpus++] = gpu;
diff --git a/drivers/gpu/drm/exynos/Kconfig b/drivers/gpu/drm/exynos/Kconfig
index baddf33..d814b304 100644
--- a/drivers/gpu/drm/exynos/Kconfig
+++ b/drivers/gpu/drm/exynos/Kconfig
@@ -71,8 +71,9 @@
This enables support for Exynos MIPI-DSI device.
config DRM_EXYNOS_DP
- bool "Display Port"
+ bool "EXYNOS specific extensions for Analogix DP driver"
depends on DRM_EXYNOS_FIMD || DRM_EXYNOS7_DECON
+ select DRM_ANALOGIX_DP
default DRM_EXYNOS
select DRM_PANEL
help
diff --git a/drivers/gpu/drm/exynos/Makefile b/drivers/gpu/drm/exynos/Makefile
index 23d2f95..f663490 100644
--- a/drivers/gpu/drm/exynos/Makefile
+++ b/drivers/gpu/drm/exynos/Makefile
@@ -12,7 +12,7 @@
exynosdrm-$(CONFIG_DRM_EXYNOS7_DECON) += exynos7_drm_decon.o
exynosdrm-$(CONFIG_DRM_EXYNOS_DPI) += exynos_drm_dpi.o
exynosdrm-$(CONFIG_DRM_EXYNOS_DSI) += exynos_drm_dsi.o
-exynosdrm-$(CONFIG_DRM_EXYNOS_DP) += exynos_dp_core.o exynos_dp_reg.o
+exynosdrm-$(CONFIG_DRM_EXYNOS_DP) += exynos_dp.o
exynosdrm-$(CONFIG_DRM_EXYNOS_MIXER) += exynos_mixer.o
exynosdrm-$(CONFIG_DRM_EXYNOS_HDMI) += exynos_hdmi.o
exynosdrm-$(CONFIG_DRM_EXYNOS_VIDI) += exynos_drm_vidi.o
diff --git a/drivers/gpu/drm/exynos/exynos5433_drm_decon.c b/drivers/gpu/drm/exynos/exynos5433_drm_decon.c
index 5245bc5..ac21b40 100644
--- a/drivers/gpu/drm/exynos/exynos5433_drm_decon.c
+++ b/drivers/gpu/drm/exynos/exynos5433_drm_decon.c
@@ -28,6 +28,10 @@
#define WINDOWS_NR 3
#define MIN_FB_WIDTH_FOR_16WORD_BURST 128
+#define IFTYPE_I80 (1 << 0)
+#define I80_HW_TRG (1 << 1)
+#define IFTYPE_HDMI (1 << 2)
+
static const char * const decon_clks_name[] = {
"pclk",
"aclk_decon",
@@ -38,12 +42,6 @@
"sclk_decon_eclk",
};
-enum decon_iftype {
- IFTYPE_RGB,
- IFTYPE_I80,
- IFTYPE_HDMI
-};
-
enum decon_flag_bits {
BIT_CLKS_ENABLED,
BIT_IRQS_ENABLED,
@@ -61,7 +59,7 @@
struct clk *clks[ARRAY_SIZE(decon_clks_name)];
int pipe;
unsigned long flags;
- enum decon_iftype out_type;
+ unsigned long out_type;
int first_win;
};
@@ -95,7 +93,7 @@
if (!test_and_set_bit(BIT_IRQS_ENABLED, &ctx->flags)) {
val = VIDINTCON0_INTEN;
- if (ctx->out_type == IFTYPE_I80)
+ if (ctx->out_type & IFTYPE_I80)
val |= VIDINTCON0_FRAMEDONE;
else
val |= VIDINTCON0_INTFRMEN;
@@ -119,11 +117,11 @@
static void decon_setup_trigger(struct decon_context *ctx)
{
- u32 val = (ctx->out_type != IFTYPE_HDMI)
+ u32 val = !(ctx->out_type & I80_HW_TRG)
? TRIGCON_TRIGEN_PER_F | TRIGCON_TRIGEN_F |
TRIGCON_TE_AUTO_MASK | TRIGCON_SWTRIGEN
: TRIGCON_TRIGEN_PER_F | TRIGCON_TRIGEN_F |
- TRIGCON_HWTRIGMASK_I80_RGB | TRIGCON_HWTRIGEN_I80_RGB;
+ TRIGCON_HWTRIGMASK | TRIGCON_HWTRIGEN;
writel(val, ctx->addr + DECON_TRIGCON);
}
@@ -136,7 +134,7 @@
if (test_bit(BIT_SUSPENDED, &ctx->flags))
return;
- if (ctx->out_type == IFTYPE_HDMI) {
+ if (ctx->out_type & IFTYPE_HDMI) {
m->crtc_hsync_start = m->crtc_hdisplay + 10;
m->crtc_hsync_end = m->crtc_htotal - 92;
m->crtc_vsync_start = m->crtc_vdisplay + 1;
@@ -149,19 +147,24 @@
val = CMU_CLKGAGE_MODE_SFR_F | CMU_CLKGAGE_MODE_MEM_F;
writel(val, ctx->addr + DECON_CMU);
+ if (ctx->out_type & (IFTYPE_I80 | I80_HW_TRG))
+ decon_setup_trigger(ctx);
+
/* lcd on and use command if */
val = VIDOUT_LCD_ON;
- if (ctx->out_type == IFTYPE_I80)
+ if (ctx->out_type & IFTYPE_I80) {
val |= VIDOUT_COMMAND_IF;
- else
+ } else {
val |= VIDOUT_RGB_IF;
+ }
+
writel(val, ctx->addr + DECON_VIDOUTCON0);
val = VIDTCON2_LINEVAL(m->vdisplay - 1) |
VIDTCON2_HOZVAL(m->hdisplay - 1);
writel(val, ctx->addr + DECON_VIDTCON2);
- if (ctx->out_type != IFTYPE_I80) {
+ if (!(ctx->out_type & IFTYPE_I80)) {
val = VIDTCON00_VBPD_F(
m->crtc_vtotal - m->crtc_vsync_end - 1) |
VIDTCON00_VFPD_F(
@@ -183,10 +186,10 @@
writel(val, ctx->addr + DECON_VIDTCON11);
}
- decon_setup_trigger(ctx);
-
/* enable output and display signal */
decon_set_bits(ctx, DECON_VIDCON0, VIDCON0_ENVID | VIDCON0_ENVID_F, ~0);
+
+ decon_set_bits(ctx, DECON_UPDATE, STANDALONE_UPDATE_F, ~0);
}
static void decon_win_set_pixfmt(struct decon_context *ctx, unsigned int win,
@@ -300,7 +303,7 @@
val = dma_addr + pitch * state->src.h;
writel(val, ctx->addr + DECON_VIDW0xADD1B0(win));
- if (ctx->out_type != IFTYPE_HDMI)
+ if (!(ctx->out_type & IFTYPE_HDMI))
val = BIT_VAL(pitch - state->crtc.w * bpp, 27, 14)
| BIT_VAL(state->crtc.w * bpp, 13, 0);
else
@@ -312,9 +315,6 @@
/* window enable */
decon_set_bits(ctx, DECON_WINCONx(win), WINCONx_ENWIN_F, ~0);
-
- /* standalone update */
- decon_set_bits(ctx, DECON_UPDATE, STANDALONE_UPDATE_F, ~0);
}
static void decon_disable_plane(struct exynos_drm_crtc *crtc,
@@ -326,15 +326,7 @@
if (test_bit(BIT_SUSPENDED, &ctx->flags))
return;
- decon_shadow_protect_win(ctx, win, true);
-
- /* window disable */
decon_set_bits(ctx, DECON_WINCONx(win), WINCONx_ENWIN_F, 0);
-
- decon_shadow_protect_win(ctx, win, false);
-
- /* standalone update */
- decon_set_bits(ctx, DECON_UPDATE, STANDALONE_UPDATE_F, ~0);
}
static void decon_atomic_flush(struct exynos_drm_crtc *crtc)
@@ -348,7 +340,10 @@
for (i = ctx->first_win; i < WINDOWS_NR; i++)
decon_shadow_protect_win(ctx, i, false);
- if (ctx->out_type == IFTYPE_I80)
+ /* standalone update */
+ decon_set_bits(ctx, DECON_UPDATE, STANDALONE_UPDATE_F, ~0);
+
+ if (ctx->out_type & IFTYPE_I80)
set_bit(BIT_WIN_UPDATED, &ctx->flags);
}
@@ -374,7 +369,7 @@
WARN(tries == 0, "failed to software reset DECON\n");
- if (ctx->out_type != IFTYPE_HDMI)
+ if (!(ctx->out_type & IFTYPE_HDMI))
return;
writel(VIDCON0_CLKVALUP | VIDCON0_VLCKFREE, ctx->addr + DECON_VIDCON0);
@@ -383,7 +378,6 @@
writel(VIDCON1_VCLK_RUN_VDEN_DISABLE, ctx->addr + DECON_VIDCON1);
writel(CRCCTRL_CRCEN | CRCCTRL_CRCSTART_F | CRCCTRL_CRCCLKEN,
ctx->addr + DECON_CRCCTRL);
- decon_setup_trigger(ctx);
}
static void decon_enable(struct exynos_drm_crtc *crtc)
@@ -395,8 +389,12 @@
pm_runtime_get_sync(ctx->dev);
+ exynos_drm_pipe_clk_enable(crtc, true);
+
set_bit(BIT_CLKS_ENABLED, &ctx->flags);
+ decon_swreset(ctx);
+
/* if vblank was enabled status, enable it again. */
if (test_and_clear_bit(BIT_IRQS_ENABLED, &ctx->flags))
decon_enable_vblank(ctx->crtc);
@@ -424,6 +422,8 @@
clear_bit(BIT_CLKS_ENABLED, &ctx->flags);
+ exynos_drm_pipe_clk_enable(crtc, false);
+
pm_runtime_put_sync(ctx->dev);
set_bit(BIT_SUSPENDED, &ctx->flags);
@@ -433,13 +433,12 @@
{
struct decon_context *ctx = crtc->ctx;
- if (!test_bit(BIT_CLKS_ENABLED, &ctx->flags))
+ if (!test_bit(BIT_CLKS_ENABLED, &ctx->flags) ||
+ (ctx->out_type & I80_HW_TRG))
return;
if (test_and_clear_bit(BIT_WIN_UPDATED, &ctx->flags))
decon_set_bits(ctx, DECON_TRIGCON, TRIGCON_SWTRIGCMD, ~0);
-
- drm_crtc_handle_vblank(&ctx->crtc->base);
}
static void decon_clear_channels(struct exynos_drm_crtc *crtc)
@@ -459,8 +458,10 @@
decon_shadow_protect_win(ctx, win, true);
decon_set_bits(ctx, DECON_WINCONx(win), WINCONx_ENWIN_F, 0);
decon_shadow_protect_win(ctx, win, false);
- decon_set_bits(ctx, DECON_UPDATE, STANDALONE_UPDATE_F, ~0);
}
+
+ decon_set_bits(ctx, DECON_UPDATE, STANDALONE_UPDATE_F, ~0);
+
/* TODO: wait for possible vsync */
msleep(50);
@@ -509,7 +510,7 @@
}
exynos_plane = &ctx->planes[ctx->first_win];
- out_type = (ctx->out_type == IFTYPE_HDMI) ? EXYNOS_DISPLAY_TYPE_HDMI
+ out_type = (ctx->out_type & IFTYPE_HDMI) ? EXYNOS_DISPLAY_TYPE_HDMI
: EXYNOS_DISPLAY_TYPE_LCD;
ctx->crtc = exynos_drm_crtc_create(drm_dev, &exynos_plane->base,
ctx->pipe, out_type,
@@ -570,6 +571,7 @@
/* clear */
writel(val, ctx->addr + DECON_VIDINTCON1);
+ drm_crtc_handle_vblank(&ctx->crtc->base);
}
out:
@@ -617,11 +619,11 @@
static const struct of_device_id exynos5433_decon_driver_dt_match[] = {
{
.compatible = "samsung,exynos5433-decon",
- .data = (void *)IFTYPE_RGB
+ .data = (void *)I80_HW_TRG
},
{
.compatible = "samsung,exynos5433-decon-tv",
- .data = (void *)IFTYPE_HDMI
+ .data = (void *)(I80_HW_TRG | IFTYPE_HDMI)
},
{},
};
@@ -629,7 +631,6 @@
static int exynos5433_decon_probe(struct platform_device *pdev)
{
- const struct of_device_id *of_id;
struct device *dev = &pdev->dev;
struct decon_context *ctx;
struct resource *res;
@@ -642,14 +643,13 @@
__set_bit(BIT_SUSPENDED, &ctx->flags);
ctx->dev = dev;
+ ctx->out_type = (unsigned long)of_device_get_match_data(dev);
- of_id = of_match_device(exynos5433_decon_driver_dt_match, &pdev->dev);
- ctx->out_type = (enum decon_iftype)of_id->data;
-
- if (ctx->out_type == IFTYPE_HDMI)
+ if (ctx->out_type & IFTYPE_HDMI) {
ctx->first_win = 1;
- else if (of_get_child_by_name(dev->of_node, "i80-if-timings"))
- ctx->out_type = IFTYPE_I80;
+ } else if (of_get_child_by_name(dev->of_node, "i80-if-timings")) {
+ ctx->out_type |= IFTYPE_I80;
+ }
for (i = 0; i < ARRAY_SIZE(decon_clks_name); i++) {
struct clk *clk;
@@ -674,7 +674,7 @@
}
res = platform_get_resource_byname(pdev, IORESOURCE_IRQ,
- (ctx->out_type == IFTYPE_I80) ? "lcd_sys" : "vsync");
+ (ctx->out_type & IFTYPE_I80) ? "lcd_sys" : "vsync");
if (!res) {
dev_err(dev, "cannot find IRQ resource\n");
return -ENXIO;
diff --git a/drivers/gpu/drm/exynos/exynos7_drm_decon.c b/drivers/gpu/drm/exynos/exynos7_drm_decon.c
index 9336107..f6223f9 100644
--- a/drivers/gpu/drm/exynos/exynos7_drm_decon.c
+++ b/drivers/gpu/drm/exynos/exynos7_drm_decon.c
@@ -593,7 +593,6 @@
.commit = decon_commit,
.enable_vblank = decon_enable_vblank,
.disable_vblank = decon_disable_vblank,
- .wait_for_vblank = decon_wait_for_vblank,
.atomic_begin = decon_atomic_begin,
.update_plane = decon_update_plane,
.disable_plane = decon_disable_plane,
diff --git a/drivers/gpu/drm/exynos/exynos_dp.c b/drivers/gpu/drm/exynos/exynos_dp.c
new file mode 100644
index 0000000..468498e
--- /dev/null
+++ b/drivers/gpu/drm/exynos/exynos_dp.c
@@ -0,0 +1,311 @@
+/*
+ * Samsung SoC DP (Display Port) interface driver.
+ *
+ * Copyright (C) 2012 Samsung Electronics Co., Ltd.
+ * Author: Jingoo Han <jg1.han@samsung.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/err.h>
+#include <linux/clk.h>
+#include <linux/of_graph.h>
+#include <linux/component.h>
+#include <video/of_display_timing.h>
+#include <video/of_videomode.h>
+#include <video/videomode.h>
+
+#include <drm/drmP.h>
+#include <drm/drm_crtc.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_panel.h>
+
+#include <drm/bridge/analogix_dp.h>
+#include <drm/exynos_drm.h>
+
+#include "exynos_drm_crtc.h"
+
+#define to_dp(nm) container_of(nm, struct exynos_dp_device, nm)
+
+struct exynos_dp_device {
+ struct drm_encoder encoder;
+ struct drm_connector connector;
+ struct drm_bridge *ptn_bridge;
+ struct drm_device *drm_dev;
+ struct device *dev;
+
+ struct videomode vm;
+ struct analogix_dp_plat_data plat_data;
+};
+
+int exynos_dp_crtc_clock_enable(struct analogix_dp_plat_data *plat_data,
+ bool enable)
+{
+ struct exynos_dp_device *dp = to_dp(plat_data);
+ struct drm_encoder *encoder = &dp->encoder;
+
+ if (!encoder->crtc)
+ return -EPERM;
+
+ exynos_drm_pipe_clk_enable(to_exynos_crtc(encoder->crtc), enable);
+
+ return 0;
+}
+
+static int exynos_dp_poweron(struct analogix_dp_plat_data *plat_data)
+{
+ return exynos_dp_crtc_clock_enable(plat_data, true);
+}
+
+static int exynos_dp_poweroff(struct analogix_dp_plat_data *plat_data)
+{
+ return exynos_dp_crtc_clock_enable(plat_data, false);
+}
+
+static int exynos_dp_get_modes(struct analogix_dp_plat_data *plat_data)
+{
+ struct exynos_dp_device *dp = to_dp(plat_data);
+ struct drm_connector *connector = &dp->connector;
+ struct drm_display_mode *mode;
+ int num_modes = 0;
+
+ if (dp->plat_data.panel)
+ return num_modes;
+
+ mode = drm_mode_create(connector->dev);
+ if (!mode) {
+ DRM_ERROR("failed to create a new display mode.\n");
+ return num_modes;
+ }
+
+ drm_display_mode_from_videomode(&dp->vm, mode);
+ connector->display_info.width_mm = mode->width_mm;
+ connector->display_info.height_mm = mode->height_mm;
+
+ mode->type = DRM_MODE_TYPE_DRIVER | DRM_MODE_TYPE_PREFERRED;
+ drm_mode_set_name(mode);
+ drm_mode_probed_add(connector, mode);
+
+ return num_modes + 1;
+}
+
+static int exynos_dp_bridge_attach(struct analogix_dp_plat_data *plat_data,
+ struct drm_bridge *bridge,
+ struct drm_connector *connector)
+{
+ struct exynos_dp_device *dp = to_dp(plat_data);
+ struct drm_encoder *encoder = &dp->encoder;
+ int ret;
+
+ drm_connector_register(connector);
+
+ /* Pre-empt DP connector creation if there's a bridge */
+ if (dp->ptn_bridge) {
+ bridge->next = dp->ptn_bridge;
+ dp->ptn_bridge->encoder = encoder;
+ ret = drm_bridge_attach(encoder->dev, dp->ptn_bridge);
+ if (ret) {
+ DRM_ERROR("Failed to attach bridge to drm\n");
+ bridge->next = NULL;
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+static void exynos_dp_mode_set(struct drm_encoder *encoder,
+ struct drm_display_mode *mode,
+ struct drm_display_mode *adjusted_mode)
+{
+}
+
+static void exynos_dp_nop(struct drm_encoder *encoder)
+{
+ /* do nothing */
+}
+
+static const struct drm_encoder_helper_funcs exynos_dp_encoder_helper_funcs = {
+ .mode_set = exynos_dp_mode_set,
+ .enable = exynos_dp_nop,
+ .disable = exynos_dp_nop,
+};
+
+static const struct drm_encoder_funcs exynos_dp_encoder_funcs = {
+ .destroy = drm_encoder_cleanup,
+};
+
+static int exynos_dp_dt_parse_panel(struct exynos_dp_device *dp)
+{
+ int ret;
+
+ ret = of_get_videomode(dp->dev->of_node, &dp->vm, OF_USE_NATIVE_MODE);
+ if (ret) {
+ DRM_ERROR("failed: of_get_videomode() : %d\n", ret);
+ return ret;
+ }
+ return 0;
+}
+
+static int exynos_dp_bind(struct device *dev, struct device *master, void *data)
+{
+ struct exynos_dp_device *dp = dev_get_drvdata(dev);
+ struct drm_encoder *encoder = &dp->encoder;
+ struct drm_device *drm_dev = data;
+ int pipe, ret;
+
+ /*
+ * Just like the probe function said, we don't need the
+ * device drvrate anymore, we should leave the charge to
+ * analogix dp driver, set the device drvdata to NULL.
+ */
+ dev_set_drvdata(dev, NULL);
+
+ dp->dev = dev;
+ dp->drm_dev = drm_dev;
+
+ dp->plat_data.dev_type = EXYNOS_DP;
+ dp->plat_data.power_on = exynos_dp_poweron;
+ dp->plat_data.power_off = exynos_dp_poweroff;
+ dp->plat_data.attach = exynos_dp_bridge_attach;
+ dp->plat_data.get_modes = exynos_dp_get_modes;
+
+ if (!dp->plat_data.panel && !dp->ptn_bridge) {
+ ret = exynos_dp_dt_parse_panel(dp);
+ if (ret)
+ return ret;
+ }
+
+ pipe = exynos_drm_crtc_get_pipe_from_type(drm_dev,
+ EXYNOS_DISPLAY_TYPE_LCD);
+ if (pipe < 0)
+ return pipe;
+
+ encoder->possible_crtcs = 1 << pipe;
+
+ DRM_DEBUG_KMS("possible_crtcs = 0x%x\n", encoder->possible_crtcs);
+
+ drm_encoder_init(drm_dev, encoder, &exynos_dp_encoder_funcs,
+ DRM_MODE_ENCODER_TMDS, NULL);
+
+ drm_encoder_helper_add(encoder, &exynos_dp_encoder_helper_funcs);
+
+ dp->plat_data.encoder = encoder;
+
+ return analogix_dp_bind(dev, dp->drm_dev, &dp->plat_data);
+}
+
+static void exynos_dp_unbind(struct device *dev, struct device *master,
+ void *data)
+{
+ return analogix_dp_unbind(dev, master, data);
+}
+
+static const struct component_ops exynos_dp_ops = {
+ .bind = exynos_dp_bind,
+ .unbind = exynos_dp_unbind,
+};
+
+static int exynos_dp_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *np = NULL, *endpoint = NULL;
+ struct exynos_dp_device *dp;
+
+ dp = devm_kzalloc(&pdev->dev, sizeof(struct exynos_dp_device),
+ GFP_KERNEL);
+ if (!dp)
+ return -ENOMEM;
+
+ /*
+ * We just use the drvdata until driver run into component
+ * add function, and then we would set drvdata to null, so
+ * that analogix dp driver would take charge of the drvdata.
+ */
+ platform_set_drvdata(pdev, dp);
+
+ /* This is for the backward compatibility. */
+ np = of_parse_phandle(dev->of_node, "panel", 0);
+ if (np) {
+ dp->plat_data.panel = of_drm_find_panel(np);
+ of_node_put(np);
+ if (!dp->plat_data.panel)
+ return -EPROBE_DEFER;
+ goto out;
+ }
+
+ endpoint = of_graph_get_next_endpoint(dev->of_node, NULL);
+ if (endpoint) {
+ np = of_graph_get_remote_port_parent(endpoint);
+ if (np) {
+ /* The remote port can be either a panel or a bridge */
+ dp->plat_data.panel = of_drm_find_panel(np);
+ if (!dp->plat_data.panel) {
+ dp->ptn_bridge = of_drm_find_bridge(np);
+ if (!dp->ptn_bridge) {
+ of_node_put(np);
+ return -EPROBE_DEFER;
+ }
+ }
+ of_node_put(np);
+ } else {
+ DRM_ERROR("no remote endpoint device node found.\n");
+ return -EINVAL;
+ }
+ } else {
+ DRM_ERROR("no port endpoint subnode found.\n");
+ return -EINVAL;
+ }
+
+out:
+ return component_add(&pdev->dev, &exynos_dp_ops);
+}
+
+static int exynos_dp_remove(struct platform_device *pdev)
+{
+ component_del(&pdev->dev, &exynos_dp_ops);
+
+ return 0;
+}
+
+#ifdef CONFIG_PM
+static int exynos_dp_suspend(struct device *dev)
+{
+ return analogix_dp_suspend(dev);
+}
+
+static int exynos_dp_resume(struct device *dev)
+{
+ return analogix_dp_resume(dev);
+}
+#endif
+
+static const struct dev_pm_ops exynos_dp_pm_ops = {
+ SET_RUNTIME_PM_OPS(exynos_dp_suspend, exynos_dp_resume, NULL)
+};
+
+static const struct of_device_id exynos_dp_match[] = {
+ { .compatible = "samsung,exynos5-dp" },
+ {},
+};
+MODULE_DEVICE_TABLE(of, exynos_dp_match);
+
+struct platform_driver dp_driver = {
+ .probe = exynos_dp_probe,
+ .remove = exynos_dp_remove,
+ .driver = {
+ .name = "exynos-dp",
+ .owner = THIS_MODULE,
+ .pm = &exynos_dp_pm_ops,
+ .of_match_table = exynos_dp_match,
+ },
+};
+
+MODULE_AUTHOR("Jingoo Han <jg1.han@samsung.com>");
+MODULE_DESCRIPTION("Samsung Specific Analogix-DP Driver Extension");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/gpu/drm/exynos/exynos_dp_core.c b/drivers/gpu/drm/exynos/exynos_dp_core.c
deleted file mode 100644
index cff8dc7..0000000
--- a/drivers/gpu/drm/exynos/exynos_dp_core.c
+++ /dev/null
@@ -1,1499 +0,0 @@
-/*
- * Samsung SoC DP (Display Port) interface driver.
- *
- * Copyright (C) 2012 Samsung Electronics Co., Ltd.
- * Author: Jingoo Han <jg1.han@samsung.com>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- */
-
-#include <linux/module.h>
-#include <linux/platform_device.h>
-#include <linux/err.h>
-#include <linux/clk.h>
-#include <linux/io.h>
-#include <linux/interrupt.h>
-#include <linux/of.h>
-#include <linux/of_gpio.h>
-#include <linux/of_graph.h>
-#include <linux/gpio.h>
-#include <linux/component.h>
-#include <linux/phy/phy.h>
-#include <video/of_display_timing.h>
-#include <video/of_videomode.h>
-
-#include <drm/drmP.h>
-#include <drm/drm_crtc.h>
-#include <drm/drm_crtc_helper.h>
-#include <drm/drm_atomic_helper.h>
-#include <drm/drm_panel.h>
-
-#include "exynos_dp_core.h"
-#include "exynos_drm_crtc.h"
-
-#define ctx_from_connector(c) container_of(c, struct exynos_dp_device, \
- connector)
-
-static inline struct exynos_drm_crtc *dp_to_crtc(struct exynos_dp_device *dp)
-{
- return to_exynos_crtc(dp->encoder.crtc);
-}
-
-static inline struct exynos_dp_device *encoder_to_dp(
- struct drm_encoder *e)
-{
- return container_of(e, struct exynos_dp_device, encoder);
-}
-
-struct bridge_init {
- struct i2c_client *client;
- struct device_node *node;
-};
-
-static void exynos_dp_init_dp(struct exynos_dp_device *dp)
-{
- exynos_dp_reset(dp);
-
- exynos_dp_swreset(dp);
-
- exynos_dp_init_analog_param(dp);
- exynos_dp_init_interrupt(dp);
-
- /* SW defined function Normal operation */
- exynos_dp_enable_sw_function(dp);
-
- exynos_dp_config_interrupt(dp);
- exynos_dp_init_analog_func(dp);
-
- exynos_dp_init_hpd(dp);
- exynos_dp_init_aux(dp);
-}
-
-static int exynos_dp_detect_hpd(struct exynos_dp_device *dp)
-{
- int timeout_loop = 0;
-
- while (exynos_dp_get_plug_in_status(dp) != 0) {
- timeout_loop++;
- if (DP_TIMEOUT_LOOP_COUNT < timeout_loop) {
- dev_err(dp->dev, "failed to get hpd plug status\n");
- return -ETIMEDOUT;
- }
- usleep_range(10, 11);
- }
-
- return 0;
-}
-
-static unsigned char exynos_dp_calc_edid_check_sum(unsigned char *edid_data)
-{
- int i;
- unsigned char sum = 0;
-
- for (i = 0; i < EDID_BLOCK_LENGTH; i++)
- sum = sum + edid_data[i];
-
- return sum;
-}
-
-static int exynos_dp_read_edid(struct exynos_dp_device *dp)
-{
- unsigned char edid[EDID_BLOCK_LENGTH * 2];
- unsigned int extend_block = 0;
- unsigned char sum;
- unsigned char test_vector;
- int retval;
-
- /*
- * EDID device address is 0x50.
- * However, if necessary, you must have set upper address
- * into E-EDID in I2C device, 0x30.
- */
-
- /* Read Extension Flag, Number of 128-byte EDID extension blocks */
- retval = exynos_dp_read_byte_from_i2c(dp, I2C_EDID_DEVICE_ADDR,
- EDID_EXTENSION_FLAG,
- &extend_block);
- if (retval)
- return retval;
-
- if (extend_block > 0) {
- dev_dbg(dp->dev, "EDID data includes a single extension!\n");
-
- /* Read EDID data */
- retval = exynos_dp_read_bytes_from_i2c(dp, I2C_EDID_DEVICE_ADDR,
- EDID_HEADER_PATTERN,
- EDID_BLOCK_LENGTH,
- &edid[EDID_HEADER_PATTERN]);
- if (retval != 0) {
- dev_err(dp->dev, "EDID Read failed!\n");
- return -EIO;
- }
- sum = exynos_dp_calc_edid_check_sum(edid);
- if (sum != 0) {
- dev_err(dp->dev, "EDID bad checksum!\n");
- return -EIO;
- }
-
- /* Read additional EDID data */
- retval = exynos_dp_read_bytes_from_i2c(dp,
- I2C_EDID_DEVICE_ADDR,
- EDID_BLOCK_LENGTH,
- EDID_BLOCK_LENGTH,
- &edid[EDID_BLOCK_LENGTH]);
- if (retval != 0) {
- dev_err(dp->dev, "EDID Read failed!\n");
- return -EIO;
- }
- sum = exynos_dp_calc_edid_check_sum(&edid[EDID_BLOCK_LENGTH]);
- if (sum != 0) {
- dev_err(dp->dev, "EDID bad checksum!\n");
- return -EIO;
- }
-
- exynos_dp_read_byte_from_dpcd(dp, DP_TEST_REQUEST,
- &test_vector);
- if (test_vector & DP_TEST_LINK_EDID_READ) {
- exynos_dp_write_byte_to_dpcd(dp,
- DP_TEST_EDID_CHECKSUM,
- edid[EDID_BLOCK_LENGTH + EDID_CHECKSUM]);
- exynos_dp_write_byte_to_dpcd(dp,
- DP_TEST_RESPONSE,
- DP_TEST_EDID_CHECKSUM_WRITE);
- }
- } else {
- dev_info(dp->dev, "EDID data does not include any extensions.\n");
-
- /* Read EDID data */
- retval = exynos_dp_read_bytes_from_i2c(dp,
- I2C_EDID_DEVICE_ADDR,
- EDID_HEADER_PATTERN,
- EDID_BLOCK_LENGTH,
- &edid[EDID_HEADER_PATTERN]);
- if (retval != 0) {
- dev_err(dp->dev, "EDID Read failed!\n");
- return -EIO;
- }
- sum = exynos_dp_calc_edid_check_sum(edid);
- if (sum != 0) {
- dev_err(dp->dev, "EDID bad checksum!\n");
- return -EIO;
- }
-
- exynos_dp_read_byte_from_dpcd(dp,
- DP_TEST_REQUEST,
- &test_vector);
- if (test_vector & DP_TEST_LINK_EDID_READ) {
- exynos_dp_write_byte_to_dpcd(dp,
- DP_TEST_EDID_CHECKSUM,
- edid[EDID_CHECKSUM]);
- exynos_dp_write_byte_to_dpcd(dp,
- DP_TEST_RESPONSE,
- DP_TEST_EDID_CHECKSUM_WRITE);
- }
- }
-
- dev_dbg(dp->dev, "EDID Read success!\n");
- return 0;
-}
-
-static int exynos_dp_handle_edid(struct exynos_dp_device *dp)
-{
- u8 buf[12];
- int i;
- int retval;
-
- /* Read DPCD DP_DPCD_REV~RECEIVE_PORT1_CAP_1 */
- retval = exynos_dp_read_bytes_from_dpcd(dp, DP_DPCD_REV,
- 12, buf);
- if (retval)
- return retval;
-
- /* Read EDID */
- for (i = 0; i < 3; i++) {
- retval = exynos_dp_read_edid(dp);
- if (!retval)
- break;
- }
-
- return retval;
-}
-
-static void exynos_dp_enable_rx_to_enhanced_mode(struct exynos_dp_device *dp,
- bool enable)
-{
- u8 data;
-
- exynos_dp_read_byte_from_dpcd(dp, DP_LANE_COUNT_SET, &data);
-
- if (enable)
- exynos_dp_write_byte_to_dpcd(dp, DP_LANE_COUNT_SET,
- DP_LANE_COUNT_ENHANCED_FRAME_EN |
- DPCD_LANE_COUNT_SET(data));
- else
- exynos_dp_write_byte_to_dpcd(dp, DP_LANE_COUNT_SET,
- DPCD_LANE_COUNT_SET(data));
-}
-
-static int exynos_dp_is_enhanced_mode_available(struct exynos_dp_device *dp)
-{
- u8 data;
- int retval;
-
- exynos_dp_read_byte_from_dpcd(dp, DP_MAX_LANE_COUNT, &data);
- retval = DPCD_ENHANCED_FRAME_CAP(data);
-
- return retval;
-}
-
-static void exynos_dp_set_enhanced_mode(struct exynos_dp_device *dp)
-{
- u8 data;
-
- data = exynos_dp_is_enhanced_mode_available(dp);
- exynos_dp_enable_rx_to_enhanced_mode(dp, data);
- exynos_dp_enable_enhanced_mode(dp, data);
-}
-
-static void exynos_dp_training_pattern_dis(struct exynos_dp_device *dp)
-{
- exynos_dp_set_training_pattern(dp, DP_NONE);
-
- exynos_dp_write_byte_to_dpcd(dp,
- DP_TRAINING_PATTERN_SET,
- DP_TRAINING_PATTERN_DISABLE);
-}
-
-static void exynos_dp_set_lane_lane_pre_emphasis(struct exynos_dp_device *dp,
- int pre_emphasis, int lane)
-{
- switch (lane) {
- case 0:
- exynos_dp_set_lane0_pre_emphasis(dp, pre_emphasis);
- break;
- case 1:
- exynos_dp_set_lane1_pre_emphasis(dp, pre_emphasis);
- break;
-
- case 2:
- exynos_dp_set_lane2_pre_emphasis(dp, pre_emphasis);
- break;
-
- case 3:
- exynos_dp_set_lane3_pre_emphasis(dp, pre_emphasis);
- break;
- }
-}
-
-static int exynos_dp_link_start(struct exynos_dp_device *dp)
-{
- u8 buf[4];
- int lane, lane_count, pll_tries, retval;
-
- lane_count = dp->link_train.lane_count;
-
- dp->link_train.lt_state = CLOCK_RECOVERY;
- dp->link_train.eq_loop = 0;
-
- for (lane = 0; lane < lane_count; lane++)
- dp->link_train.cr_loop[lane] = 0;
-
- /* Set link rate and count as you want to establish*/
- exynos_dp_set_link_bandwidth(dp, dp->link_train.link_rate);
- exynos_dp_set_lane_count(dp, dp->link_train.lane_count);
-
- /* Setup RX configuration */
- buf[0] = dp->link_train.link_rate;
- buf[1] = dp->link_train.lane_count;
- retval = exynos_dp_write_bytes_to_dpcd(dp, DP_LINK_BW_SET,
- 2, buf);
- if (retval)
- return retval;
-
- /* Set TX pre-emphasis to minimum */
- for (lane = 0; lane < lane_count; lane++)
- exynos_dp_set_lane_lane_pre_emphasis(dp,
- PRE_EMPHASIS_LEVEL_0, lane);
-
- /* Wait for PLL lock */
- pll_tries = 0;
- while (exynos_dp_get_pll_lock_status(dp) == PLL_UNLOCKED) {
- if (pll_tries == DP_TIMEOUT_LOOP_COUNT) {
- dev_err(dp->dev, "Wait for PLL lock timed out\n");
- return -ETIMEDOUT;
- }
-
- pll_tries++;
- usleep_range(90, 120);
- }
-
- /* Set training pattern 1 */
- exynos_dp_set_training_pattern(dp, TRAINING_PTN1);
-
- /* Set RX training pattern */
- retval = exynos_dp_write_byte_to_dpcd(dp,
- DP_TRAINING_PATTERN_SET,
- DP_LINK_SCRAMBLING_DISABLE | DP_TRAINING_PATTERN_1);
- if (retval)
- return retval;
-
- for (lane = 0; lane < lane_count; lane++)
- buf[lane] = DP_TRAIN_PRE_EMPH_LEVEL_0 |
- DP_TRAIN_VOLTAGE_SWING_LEVEL_0;
-
- retval = exynos_dp_write_bytes_to_dpcd(dp, DP_TRAINING_LANE0_SET,
- lane_count, buf);
-
- return retval;
-}
-
-static unsigned char exynos_dp_get_lane_status(u8 link_status[2], int lane)
-{
- int shift = (lane & 1) * 4;
- u8 link_value = link_status[lane>>1];
-
- return (link_value >> shift) & 0xf;
-}
-
-static int exynos_dp_clock_recovery_ok(u8 link_status[2], int lane_count)
-{
- int lane;
- u8 lane_status;
-
- for (lane = 0; lane < lane_count; lane++) {
- lane_status = exynos_dp_get_lane_status(link_status, lane);
- if ((lane_status & DP_LANE_CR_DONE) == 0)
- return -EINVAL;
- }
- return 0;
-}
-
-static int exynos_dp_channel_eq_ok(u8 link_status[2], u8 link_align,
- int lane_count)
-{
- int lane;
- u8 lane_status;
-
- if ((link_align & DP_INTERLANE_ALIGN_DONE) == 0)
- return -EINVAL;
-
- for (lane = 0; lane < lane_count; lane++) {
- lane_status = exynos_dp_get_lane_status(link_status, lane);
- lane_status &= DP_CHANNEL_EQ_BITS;
- if (lane_status != DP_CHANNEL_EQ_BITS)
- return -EINVAL;
- }
-
- return 0;
-}
-
-static unsigned char exynos_dp_get_adjust_request_voltage(u8 adjust_request[2],
- int lane)
-{
- int shift = (lane & 1) * 4;
- u8 link_value = adjust_request[lane>>1];
-
- return (link_value >> shift) & 0x3;
-}
-
-static unsigned char exynos_dp_get_adjust_request_pre_emphasis(
- u8 adjust_request[2],
- int lane)
-{
- int shift = (lane & 1) * 4;
- u8 link_value = adjust_request[lane>>1];
-
- return ((link_value >> shift) & 0xc) >> 2;
-}
-
-static void exynos_dp_set_lane_link_training(struct exynos_dp_device *dp,
- u8 training_lane_set, int lane)
-{
- switch (lane) {
- case 0:
- exynos_dp_set_lane0_link_training(dp, training_lane_set);
- break;
- case 1:
- exynos_dp_set_lane1_link_training(dp, training_lane_set);
- break;
-
- case 2:
- exynos_dp_set_lane2_link_training(dp, training_lane_set);
- break;
-
- case 3:
- exynos_dp_set_lane3_link_training(dp, training_lane_set);
- break;
- }
-}
-
-static unsigned int exynos_dp_get_lane_link_training(
- struct exynos_dp_device *dp,
- int lane)
-{
- u32 reg;
-
- switch (lane) {
- case 0:
- reg = exynos_dp_get_lane0_link_training(dp);
- break;
- case 1:
- reg = exynos_dp_get_lane1_link_training(dp);
- break;
- case 2:
- reg = exynos_dp_get_lane2_link_training(dp);
- break;
- case 3:
- reg = exynos_dp_get_lane3_link_training(dp);
- break;
- default:
- WARN_ON(1);
- return 0;
- }
-
- return reg;
-}
-
-static void exynos_dp_reduce_link_rate(struct exynos_dp_device *dp)
-{
- exynos_dp_training_pattern_dis(dp);
- exynos_dp_set_enhanced_mode(dp);
-
- dp->link_train.lt_state = FAILED;
-}
-
-static void exynos_dp_get_adjust_training_lane(struct exynos_dp_device *dp,
- u8 adjust_request[2])
-{
- int lane, lane_count;
- u8 voltage_swing, pre_emphasis, training_lane;
-
- lane_count = dp->link_train.lane_count;
- for (lane = 0; lane < lane_count; lane++) {
- voltage_swing = exynos_dp_get_adjust_request_voltage(
- adjust_request, lane);
- pre_emphasis = exynos_dp_get_adjust_request_pre_emphasis(
- adjust_request, lane);
- training_lane = DPCD_VOLTAGE_SWING_SET(voltage_swing) |
- DPCD_PRE_EMPHASIS_SET(pre_emphasis);
-
- if (voltage_swing == VOLTAGE_LEVEL_3)
- training_lane |= DP_TRAIN_MAX_SWING_REACHED;
- if (pre_emphasis == PRE_EMPHASIS_LEVEL_3)
- training_lane |= DP_TRAIN_MAX_PRE_EMPHASIS_REACHED;
-
- dp->link_train.training_lane[lane] = training_lane;
- }
-}
-
-static int exynos_dp_process_clock_recovery(struct exynos_dp_device *dp)
-{
- int lane, lane_count, retval;
- u8 voltage_swing, pre_emphasis, training_lane;
- u8 link_status[2], adjust_request[2];
-
- usleep_range(100, 101);
-
- lane_count = dp->link_train.lane_count;
-
- retval = exynos_dp_read_bytes_from_dpcd(dp,
- DP_LANE0_1_STATUS, 2, link_status);
- if (retval)
- return retval;
-
- retval = exynos_dp_read_bytes_from_dpcd(dp,
- DP_ADJUST_REQUEST_LANE0_1, 2, adjust_request);
- if (retval)
- return retval;
-
- if (exynos_dp_clock_recovery_ok(link_status, lane_count) == 0) {
- /* set training pattern 2 for EQ */
- exynos_dp_set_training_pattern(dp, TRAINING_PTN2);
-
- retval = exynos_dp_write_byte_to_dpcd(dp,
- DP_TRAINING_PATTERN_SET,
- DP_LINK_SCRAMBLING_DISABLE |
- DP_TRAINING_PATTERN_2);
- if (retval)
- return retval;
-
- dev_info(dp->dev, "Link Training Clock Recovery success\n");
- dp->link_train.lt_state = EQUALIZER_TRAINING;
- } else {
- for (lane = 0; lane < lane_count; lane++) {
- training_lane = exynos_dp_get_lane_link_training(
- dp, lane);
- voltage_swing = exynos_dp_get_adjust_request_voltage(
- adjust_request, lane);
- pre_emphasis = exynos_dp_get_adjust_request_pre_emphasis(
- adjust_request, lane);
-
- if (DPCD_VOLTAGE_SWING_GET(training_lane) ==
- voltage_swing &&
- DPCD_PRE_EMPHASIS_GET(training_lane) ==
- pre_emphasis)
- dp->link_train.cr_loop[lane]++;
-
- if (dp->link_train.cr_loop[lane] == MAX_CR_LOOP ||
- voltage_swing == VOLTAGE_LEVEL_3 ||
- pre_emphasis == PRE_EMPHASIS_LEVEL_3) {
- dev_err(dp->dev, "CR Max reached (%d,%d,%d)\n",
- dp->link_train.cr_loop[lane],
- voltage_swing, pre_emphasis);
- exynos_dp_reduce_link_rate(dp);
- return -EIO;
- }
- }
- }
-
- exynos_dp_get_adjust_training_lane(dp, adjust_request);
-
- for (lane = 0; lane < lane_count; lane++)
- exynos_dp_set_lane_link_training(dp,
- dp->link_train.training_lane[lane], lane);
-
- retval = exynos_dp_write_bytes_to_dpcd(dp,
- DP_TRAINING_LANE0_SET, lane_count,
- dp->link_train.training_lane);
- if (retval)
- return retval;
-
- return retval;
-}
-
-static int exynos_dp_process_equalizer_training(struct exynos_dp_device *dp)
-{
- int lane, lane_count, retval;
- u32 reg;
- u8 link_align, link_status[2], adjust_request[2];
-
- usleep_range(400, 401);
-
- lane_count = dp->link_train.lane_count;
-
- retval = exynos_dp_read_bytes_from_dpcd(dp,
- DP_LANE0_1_STATUS, 2, link_status);
- if (retval)
- return retval;
-
- if (exynos_dp_clock_recovery_ok(link_status, lane_count)) {
- exynos_dp_reduce_link_rate(dp);
- return -EIO;
- }
-
- retval = exynos_dp_read_bytes_from_dpcd(dp,
- DP_ADJUST_REQUEST_LANE0_1, 2, adjust_request);
- if (retval)
- return retval;
-
- retval = exynos_dp_read_byte_from_dpcd(dp,
- DP_LANE_ALIGN_STATUS_UPDATED, &link_align);
- if (retval)
- return retval;
-
- exynos_dp_get_adjust_training_lane(dp, adjust_request);
-
- if (!exynos_dp_channel_eq_ok(link_status, link_align, lane_count)) {
- /* traing pattern Set to Normal */
- exynos_dp_training_pattern_dis(dp);
-
- dev_info(dp->dev, "Link Training success!\n");
-
- exynos_dp_get_link_bandwidth(dp, ®);
- dp->link_train.link_rate = reg;
- dev_dbg(dp->dev, "final bandwidth = %.2x\n",
- dp->link_train.link_rate);
-
- exynos_dp_get_lane_count(dp, ®);
- dp->link_train.lane_count = reg;
- dev_dbg(dp->dev, "final lane count = %.2x\n",
- dp->link_train.lane_count);
-
- /* set enhanced mode if available */
- exynos_dp_set_enhanced_mode(dp);
- dp->link_train.lt_state = FINISHED;
-
- return 0;
- }
-
- /* not all locked */
- dp->link_train.eq_loop++;
-
- if (dp->link_train.eq_loop > MAX_EQ_LOOP) {
- dev_err(dp->dev, "EQ Max loop\n");
- exynos_dp_reduce_link_rate(dp);
- return -EIO;
- }
-
- for (lane = 0; lane < lane_count; lane++)
- exynos_dp_set_lane_link_training(dp,
- dp->link_train.training_lane[lane], lane);
-
- retval = exynos_dp_write_bytes_to_dpcd(dp, DP_TRAINING_LANE0_SET,
- lane_count, dp->link_train.training_lane);
-
- return retval;
-}
-
-static void exynos_dp_get_max_rx_bandwidth(struct exynos_dp_device *dp,
- u8 *bandwidth)
-{
- u8 data;
-
- /*
- * For DP rev.1.1, Maximum link rate of Main Link lanes
- * 0x06 = 1.62 Gbps, 0x0a = 2.7 Gbps
- */
- exynos_dp_read_byte_from_dpcd(dp, DP_MAX_LINK_RATE, &data);
- *bandwidth = data;
-}
-
-static void exynos_dp_get_max_rx_lane_count(struct exynos_dp_device *dp,
- u8 *lane_count)
-{
- u8 data;
-
- /*
- * For DP rev.1.1, Maximum number of Main Link lanes
- * 0x01 = 1 lane, 0x02 = 2 lanes, 0x04 = 4 lanes
- */
- exynos_dp_read_byte_from_dpcd(dp, DP_MAX_LANE_COUNT, &data);
- *lane_count = DPCD_MAX_LANE_COUNT(data);
-}
-
-static void exynos_dp_init_training(struct exynos_dp_device *dp,
- enum link_lane_count_type max_lane,
- enum link_rate_type max_rate)
-{
- /*
- * MACRO_RST must be applied after the PLL_LOCK to avoid
- * the DP inter pair skew issue for at least 10 us
- */
- exynos_dp_reset_macro(dp);
-
- /* Initialize by reading RX's DPCD */
- exynos_dp_get_max_rx_bandwidth(dp, &dp->link_train.link_rate);
- exynos_dp_get_max_rx_lane_count(dp, &dp->link_train.lane_count);
-
- if ((dp->link_train.link_rate != LINK_RATE_1_62GBPS) &&
- (dp->link_train.link_rate != LINK_RATE_2_70GBPS)) {
- dev_err(dp->dev, "Rx Max Link Rate is abnormal :%x !\n",
- dp->link_train.link_rate);
- dp->link_train.link_rate = LINK_RATE_1_62GBPS;
- }
-
- if (dp->link_train.lane_count == 0) {
- dev_err(dp->dev, "Rx Max Lane count is abnormal :%x !\n",
- dp->link_train.lane_count);
- dp->link_train.lane_count = (u8)LANE_COUNT1;
- }
-
- /* Setup TX lane count & rate */
- if (dp->link_train.lane_count > max_lane)
- dp->link_train.lane_count = max_lane;
- if (dp->link_train.link_rate > max_rate)
- dp->link_train.link_rate = max_rate;
-
- /* All DP analog module power up */
- exynos_dp_set_analog_power_down(dp, POWER_ALL, 0);
-}
-
-static int exynos_dp_sw_link_training(struct exynos_dp_device *dp)
-{
- int retval = 0, training_finished = 0;
-
- dp->link_train.lt_state = START;
-
- /* Process here */
- while (!retval && !training_finished) {
- switch (dp->link_train.lt_state) {
- case START:
- retval = exynos_dp_link_start(dp);
- if (retval)
- dev_err(dp->dev, "LT link start failed!\n");
- break;
- case CLOCK_RECOVERY:
- retval = exynos_dp_process_clock_recovery(dp);
- if (retval)
- dev_err(dp->dev, "LT CR failed!\n");
- break;
- case EQUALIZER_TRAINING:
- retval = exynos_dp_process_equalizer_training(dp);
- if (retval)
- dev_err(dp->dev, "LT EQ failed!\n");
- break;
- case FINISHED:
- training_finished = 1;
- break;
- case FAILED:
- return -EREMOTEIO;
- }
- }
- if (retval)
- dev_err(dp->dev, "eDP link training failed (%d)\n", retval);
-
- return retval;
-}
-
-static int exynos_dp_set_link_train(struct exynos_dp_device *dp,
- u32 count,
- u32 bwtype)
-{
- int i;
- int retval;
-
- for (i = 0; i < DP_TIMEOUT_LOOP_COUNT; i++) {
- exynos_dp_init_training(dp, count, bwtype);
- retval = exynos_dp_sw_link_training(dp);
- if (retval == 0)
- break;
-
- usleep_range(100, 110);
- }
-
- return retval;
-}
-
-static int exynos_dp_config_video(struct exynos_dp_device *dp)
-{
- int retval = 0;
- int timeout_loop = 0;
- int done_count = 0;
-
- exynos_dp_config_video_slave_mode(dp);
-
- exynos_dp_set_video_color_format(dp);
-
- if (exynos_dp_get_pll_lock_status(dp) == PLL_UNLOCKED) {
- dev_err(dp->dev, "PLL is not locked yet.\n");
- return -EINVAL;
- }
-
- for (;;) {
- timeout_loop++;
- if (exynos_dp_is_slave_video_stream_clock_on(dp) == 0)
- break;
- if (DP_TIMEOUT_LOOP_COUNT < timeout_loop) {
- dev_err(dp->dev, "Timeout of video streamclk ok\n");
- return -ETIMEDOUT;
- }
-
- usleep_range(1, 2);
- }
-
- /* Set to use the register calculated M/N video */
- exynos_dp_set_video_cr_mn(dp, CALCULATED_M, 0, 0);
-
- /* For video bist, Video timing must be generated by register */
- exynos_dp_set_video_timing_mode(dp, VIDEO_TIMING_FROM_CAPTURE);
-
- /* Disable video mute */
- exynos_dp_enable_video_mute(dp, 0);
-
- /* Configure video slave mode */
- exynos_dp_enable_video_master(dp, 0);
-
- timeout_loop = 0;
-
- for (;;) {
- timeout_loop++;
- if (exynos_dp_is_video_stream_on(dp) == 0) {
- done_count++;
- if (done_count > 10)
- break;
- } else if (done_count) {
- done_count = 0;
- }
- if (DP_TIMEOUT_LOOP_COUNT < timeout_loop) {
- dev_err(dp->dev, "Timeout of video streamclk ok\n");
- return -ETIMEDOUT;
- }
-
- usleep_range(1000, 1001);
- }
-
- if (retval != 0)
- dev_err(dp->dev, "Video stream is not detected!\n");
-
- return retval;
-}
-
-static void exynos_dp_enable_scramble(struct exynos_dp_device *dp, bool enable)
-{
- u8 data;
-
- if (enable) {
- exynos_dp_enable_scrambling(dp);
-
- exynos_dp_read_byte_from_dpcd(dp,
- DP_TRAINING_PATTERN_SET,
- &data);
- exynos_dp_write_byte_to_dpcd(dp,
- DP_TRAINING_PATTERN_SET,
- (u8)(data & ~DP_LINK_SCRAMBLING_DISABLE));
- } else {
- exynos_dp_disable_scrambling(dp);
-
- exynos_dp_read_byte_from_dpcd(dp,
- DP_TRAINING_PATTERN_SET,
- &data);
- exynos_dp_write_byte_to_dpcd(dp,
- DP_TRAINING_PATTERN_SET,
- (u8)(data | DP_LINK_SCRAMBLING_DISABLE));
- }
-}
-
-static irqreturn_t exynos_dp_irq_handler(int irq, void *arg)
-{
- struct exynos_dp_device *dp = arg;
-
- enum dp_irq_type irq_type;
-
- irq_type = exynos_dp_get_irq_type(dp);
- switch (irq_type) {
- case DP_IRQ_TYPE_HP_CABLE_IN:
- dev_dbg(dp->dev, "Received irq - cable in\n");
- schedule_work(&dp->hotplug_work);
- exynos_dp_clear_hotplug_interrupts(dp);
- break;
- case DP_IRQ_TYPE_HP_CABLE_OUT:
- dev_dbg(dp->dev, "Received irq - cable out\n");
- exynos_dp_clear_hotplug_interrupts(dp);
- break;
- case DP_IRQ_TYPE_HP_CHANGE:
- /*
- * We get these change notifications once in a while, but there
- * is nothing we can do with them. Just ignore it for now and
- * only handle cable changes.
- */
- dev_dbg(dp->dev, "Received irq - hotplug change; ignoring.\n");
- exynos_dp_clear_hotplug_interrupts(dp);
- break;
- default:
- dev_err(dp->dev, "Received irq - unknown type!\n");
- break;
- }
- return IRQ_HANDLED;
-}
-
-static void exynos_dp_hotplug(struct work_struct *work)
-{
- struct exynos_dp_device *dp;
-
- dp = container_of(work, struct exynos_dp_device, hotplug_work);
-
- if (dp->drm_dev)
- drm_helper_hpd_irq_event(dp->drm_dev);
-}
-
-static void exynos_dp_commit(struct drm_encoder *encoder)
-{
- struct exynos_dp_device *dp = encoder_to_dp(encoder);
- int ret;
-
- /* Keep the panel disabled while we configure video */
- if (dp->panel) {
- if (drm_panel_disable(dp->panel))
- DRM_ERROR("failed to disable the panel\n");
- }
-
- ret = exynos_dp_detect_hpd(dp);
- if (ret) {
- /* Cable has been disconnected, we're done */
- return;
- }
-
- ret = exynos_dp_handle_edid(dp);
- if (ret) {
- dev_err(dp->dev, "unable to handle edid\n");
- return;
- }
-
- ret = exynos_dp_set_link_train(dp, dp->video_info->lane_count,
- dp->video_info->link_rate);
- if (ret) {
- dev_err(dp->dev, "unable to do link train\n");
- return;
- }
-
- exynos_dp_enable_scramble(dp, 1);
- exynos_dp_enable_rx_to_enhanced_mode(dp, 1);
- exynos_dp_enable_enhanced_mode(dp, 1);
-
- exynos_dp_set_lane_count(dp, dp->video_info->lane_count);
- exynos_dp_set_link_bandwidth(dp, dp->video_info->link_rate);
-
- exynos_dp_init_video(dp);
- ret = exynos_dp_config_video(dp);
- if (ret)
- dev_err(dp->dev, "unable to config video\n");
-
- /* Safe to enable the panel now */
- if (dp->panel) {
- if (drm_panel_enable(dp->panel))
- DRM_ERROR("failed to enable the panel\n");
- }
-
- /* Enable video */
- exynos_dp_start_video(dp);
-}
-
-static enum drm_connector_status exynos_dp_detect(
- struct drm_connector *connector, bool force)
-{
- return connector_status_connected;
-}
-
-static void exynos_dp_connector_destroy(struct drm_connector *connector)
-{
- drm_connector_unregister(connector);
- drm_connector_cleanup(connector);
-}
-
-static const struct drm_connector_funcs exynos_dp_connector_funcs = {
- .dpms = drm_atomic_helper_connector_dpms,
- .fill_modes = drm_helper_probe_single_connector_modes,
- .detect = exynos_dp_detect,
- .destroy = exynos_dp_connector_destroy,
- .reset = drm_atomic_helper_connector_reset,
- .atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
- .atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
-};
-
-static int exynos_dp_get_modes(struct drm_connector *connector)
-{
- struct exynos_dp_device *dp = ctx_from_connector(connector);
- struct drm_display_mode *mode;
-
- if (dp->panel)
- return drm_panel_get_modes(dp->panel);
-
- mode = drm_mode_create(connector->dev);
- if (!mode) {
- DRM_ERROR("failed to create a new display mode.\n");
- return 0;
- }
-
- drm_display_mode_from_videomode(&dp->vm, mode);
- connector->display_info.width_mm = mode->width_mm;
- connector->display_info.height_mm = mode->height_mm;
-
- mode->type = DRM_MODE_TYPE_DRIVER | DRM_MODE_TYPE_PREFERRED;
- drm_mode_set_name(mode);
- drm_mode_probed_add(connector, mode);
-
- return 1;
-}
-
-static struct drm_encoder *exynos_dp_best_encoder(
- struct drm_connector *connector)
-{
- struct exynos_dp_device *dp = ctx_from_connector(connector);
-
- return &dp->encoder;
-}
-
-static const struct drm_connector_helper_funcs exynos_dp_connector_helper_funcs = {
- .get_modes = exynos_dp_get_modes,
- .best_encoder = exynos_dp_best_encoder,
-};
-
-/* returns the number of bridges attached */
-static int exynos_drm_attach_lcd_bridge(struct exynos_dp_device *dp,
- struct drm_encoder *encoder)
-{
- int ret;
-
- encoder->bridge->next = dp->ptn_bridge;
- dp->ptn_bridge->encoder = encoder;
- ret = drm_bridge_attach(encoder->dev, dp->ptn_bridge);
- if (ret) {
- DRM_ERROR("Failed to attach bridge to drm\n");
- return ret;
- }
-
- return 0;
-}
-
-static int exynos_dp_bridge_attach(struct drm_bridge *bridge)
-{
- struct exynos_dp_device *dp = bridge->driver_private;
- struct drm_encoder *encoder = &dp->encoder;
- struct drm_connector *connector = &dp->connector;
- int ret;
-
- /* Pre-empt DP connector creation if there's a bridge */
- if (dp->ptn_bridge) {
- ret = exynos_drm_attach_lcd_bridge(dp, encoder);
- if (!ret)
- return 0;
- }
-
- connector->polled = DRM_CONNECTOR_POLL_HPD;
-
- ret = drm_connector_init(dp->drm_dev, connector,
- &exynos_dp_connector_funcs, DRM_MODE_CONNECTOR_eDP);
- if (ret) {
- DRM_ERROR("Failed to initialize connector with drm\n");
- return ret;
- }
-
- drm_connector_helper_add(connector, &exynos_dp_connector_helper_funcs);
- drm_connector_register(connector);
- drm_mode_connector_attach_encoder(connector, encoder);
-
- if (dp->panel)
- ret = drm_panel_attach(dp->panel, &dp->connector);
-
- return ret;
-}
-
-static void exynos_dp_bridge_enable(struct drm_bridge *bridge)
-{
- struct exynos_dp_device *dp = bridge->driver_private;
- struct exynos_drm_crtc *crtc = dp_to_crtc(dp);
-
- if (dp->dpms_mode == DRM_MODE_DPMS_ON)
- return;
-
- pm_runtime_get_sync(dp->dev);
-
- if (dp->panel) {
- if (drm_panel_prepare(dp->panel)) {
- DRM_ERROR("failed to setup the panel\n");
- return;
- }
- }
-
- if (crtc->ops->clock_enable)
- crtc->ops->clock_enable(dp_to_crtc(dp), true);
-
- phy_power_on(dp->phy);
- exynos_dp_init_dp(dp);
- enable_irq(dp->irq);
- exynos_dp_commit(&dp->encoder);
-
- dp->dpms_mode = DRM_MODE_DPMS_ON;
-}
-
-static void exynos_dp_bridge_disable(struct drm_bridge *bridge)
-{
- struct exynos_dp_device *dp = bridge->driver_private;
- struct exynos_drm_crtc *crtc = dp_to_crtc(dp);
-
- if (dp->dpms_mode != DRM_MODE_DPMS_ON)
- return;
-
- if (dp->panel) {
- if (drm_panel_disable(dp->panel)) {
- DRM_ERROR("failed to disable the panel\n");
- return;
- }
- }
-
- disable_irq(dp->irq);
- flush_work(&dp->hotplug_work);
- phy_power_off(dp->phy);
-
- if (crtc->ops->clock_enable)
- crtc->ops->clock_enable(dp_to_crtc(dp), false);
-
- if (dp->panel) {
- if (drm_panel_unprepare(dp->panel))
- DRM_ERROR("failed to turnoff the panel\n");
- }
-
- pm_runtime_put_sync(dp->dev);
-
- dp->dpms_mode = DRM_MODE_DPMS_OFF;
-}
-
-static void exynos_dp_bridge_nop(struct drm_bridge *bridge)
-{
- /* do nothing */
-}
-
-static const struct drm_bridge_funcs exynos_dp_bridge_funcs = {
- .enable = exynos_dp_bridge_enable,
- .disable = exynos_dp_bridge_disable,
- .pre_enable = exynos_dp_bridge_nop,
- .post_disable = exynos_dp_bridge_nop,
- .attach = exynos_dp_bridge_attach,
-};
-
-static int exynos_dp_create_connector(struct drm_encoder *encoder)
-{
- struct exynos_dp_device *dp = encoder_to_dp(encoder);
- struct drm_device *drm_dev = dp->drm_dev;
- struct drm_bridge *bridge;
- int ret;
-
- bridge = devm_kzalloc(drm_dev->dev, sizeof(*bridge), GFP_KERNEL);
- if (!bridge) {
- DRM_ERROR("failed to allocate for drm bridge\n");
- return -ENOMEM;
- }
-
- dp->bridge = bridge;
-
- encoder->bridge = bridge;
- bridge->driver_private = dp;
- bridge->encoder = encoder;
- bridge->funcs = &exynos_dp_bridge_funcs;
-
- ret = drm_bridge_attach(drm_dev, bridge);
- if (ret) {
- DRM_ERROR("failed to attach drm bridge\n");
- return -EINVAL;
- }
-
- return 0;
-}
-
-static void exynos_dp_mode_set(struct drm_encoder *encoder,
- struct drm_display_mode *mode,
- struct drm_display_mode *adjusted_mode)
-{
-}
-
-static void exynos_dp_enable(struct drm_encoder *encoder)
-{
-}
-
-static void exynos_dp_disable(struct drm_encoder *encoder)
-{
-}
-
-static const struct drm_encoder_helper_funcs exynos_dp_encoder_helper_funcs = {
- .mode_set = exynos_dp_mode_set,
- .enable = exynos_dp_enable,
- .disable = exynos_dp_disable,
-};
-
-static const struct drm_encoder_funcs exynos_dp_encoder_funcs = {
- .destroy = drm_encoder_cleanup,
-};
-
-static struct video_info *exynos_dp_dt_parse_pdata(struct device *dev)
-{
- struct device_node *dp_node = dev->of_node;
- struct video_info *dp_video_config;
-
- dp_video_config = devm_kzalloc(dev,
- sizeof(*dp_video_config), GFP_KERNEL);
- if (!dp_video_config)
- return ERR_PTR(-ENOMEM);
-
- dp_video_config->h_sync_polarity =
- of_property_read_bool(dp_node, "hsync-active-high");
-
- dp_video_config->v_sync_polarity =
- of_property_read_bool(dp_node, "vsync-active-high");
-
- dp_video_config->interlaced =
- of_property_read_bool(dp_node, "interlaced");
-
- if (of_property_read_u32(dp_node, "samsung,color-space",
- &dp_video_config->color_space)) {
- dev_err(dev, "failed to get color-space\n");
- return ERR_PTR(-EINVAL);
- }
-
- if (of_property_read_u32(dp_node, "samsung,dynamic-range",
- &dp_video_config->dynamic_range)) {
- dev_err(dev, "failed to get dynamic-range\n");
- return ERR_PTR(-EINVAL);
- }
-
- if (of_property_read_u32(dp_node, "samsung,ycbcr-coeff",
- &dp_video_config->ycbcr_coeff)) {
- dev_err(dev, "failed to get ycbcr-coeff\n");
- return ERR_PTR(-EINVAL);
- }
-
- if (of_property_read_u32(dp_node, "samsung,color-depth",
- &dp_video_config->color_depth)) {
- dev_err(dev, "failed to get color-depth\n");
- return ERR_PTR(-EINVAL);
- }
-
- if (of_property_read_u32(dp_node, "samsung,link-rate",
- &dp_video_config->link_rate)) {
- dev_err(dev, "failed to get link-rate\n");
- return ERR_PTR(-EINVAL);
- }
-
- if (of_property_read_u32(dp_node, "samsung,lane-count",
- &dp_video_config->lane_count)) {
- dev_err(dev, "failed to get lane-count\n");
- return ERR_PTR(-EINVAL);
- }
-
- return dp_video_config;
-}
-
-static int exynos_dp_dt_parse_panel(struct exynos_dp_device *dp)
-{
- int ret;
-
- ret = of_get_videomode(dp->dev->of_node, &dp->vm, OF_USE_NATIVE_MODE);
- if (ret) {
- DRM_ERROR("failed: of_get_videomode() : %d\n", ret);
- return ret;
- }
- return 0;
-}
-
-static int exynos_dp_bind(struct device *dev, struct device *master, void *data)
-{
- struct exynos_dp_device *dp = dev_get_drvdata(dev);
- struct platform_device *pdev = to_platform_device(dev);
- struct drm_device *drm_dev = data;
- struct drm_encoder *encoder = &dp->encoder;
- struct resource *res;
- unsigned int irq_flags;
- int pipe, ret = 0;
-
- dp->dev = &pdev->dev;
- dp->dpms_mode = DRM_MODE_DPMS_OFF;
-
- dp->video_info = exynos_dp_dt_parse_pdata(&pdev->dev);
- if (IS_ERR(dp->video_info))
- return PTR_ERR(dp->video_info);
-
- dp->phy = devm_phy_get(dp->dev, "dp");
- if (IS_ERR(dp->phy)) {
- dev_err(dp->dev, "no DP phy configured\n");
- ret = PTR_ERR(dp->phy);
- if (ret) {
- /*
- * phy itself is not enabled, so we can move forward
- * assigning NULL to phy pointer.
- */
- if (ret == -ENOSYS || ret == -ENODEV)
- dp->phy = NULL;
- else
- return ret;
- }
- }
-
- if (!dp->panel && !dp->ptn_bridge) {
- ret = exynos_dp_dt_parse_panel(dp);
- if (ret)
- return ret;
- }
-
- dp->clock = devm_clk_get(&pdev->dev, "dp");
- if (IS_ERR(dp->clock)) {
- dev_err(&pdev->dev, "failed to get clock\n");
- return PTR_ERR(dp->clock);
- }
-
- clk_prepare_enable(dp->clock);
-
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
-
- dp->reg_base = devm_ioremap_resource(&pdev->dev, res);
- if (IS_ERR(dp->reg_base))
- return PTR_ERR(dp->reg_base);
-
- dp->hpd_gpio = of_get_named_gpio(dev->of_node, "samsung,hpd-gpio", 0);
-
- if (gpio_is_valid(dp->hpd_gpio)) {
- /*
- * Set up the hotplug GPIO from the device tree as an interrupt.
- * Simply specifying a different interrupt in the device tree
- * doesn't work since we handle hotplug rather differently when
- * using a GPIO. We also need the actual GPIO specifier so
- * that we can get the current state of the GPIO.
- */
- ret = devm_gpio_request_one(&pdev->dev, dp->hpd_gpio, GPIOF_IN,
- "hpd_gpio");
- if (ret) {
- dev_err(&pdev->dev, "failed to get hpd gpio\n");
- return ret;
- }
- dp->irq = gpio_to_irq(dp->hpd_gpio);
- irq_flags = IRQF_TRIGGER_RISING | IRQF_TRIGGER_FALLING;
- } else {
- dp->hpd_gpio = -ENODEV;
- dp->irq = platform_get_irq(pdev, 0);
- irq_flags = 0;
- }
-
- if (dp->irq == -ENXIO) {
- dev_err(&pdev->dev, "failed to get irq\n");
- return -ENODEV;
- }
-
- INIT_WORK(&dp->hotplug_work, exynos_dp_hotplug);
-
- ret = devm_request_irq(&pdev->dev, dp->irq, exynos_dp_irq_handler,
- irq_flags, "exynos-dp", dp);
- if (ret) {
- dev_err(&pdev->dev, "failed to request irq\n");
- return ret;
- }
- disable_irq(dp->irq);
-
- dp->drm_dev = drm_dev;
-
- pipe = exynos_drm_crtc_get_pipe_from_type(drm_dev,
- EXYNOS_DISPLAY_TYPE_LCD);
- if (pipe < 0)
- return pipe;
-
- encoder->possible_crtcs = 1 << pipe;
-
- DRM_DEBUG_KMS("possible_crtcs = 0x%x\n", encoder->possible_crtcs);
-
- drm_encoder_init(drm_dev, encoder, &exynos_dp_encoder_funcs,
- DRM_MODE_ENCODER_TMDS, NULL);
-
- drm_encoder_helper_add(encoder, &exynos_dp_encoder_helper_funcs);
-
- ret = exynos_dp_create_connector(encoder);
- if (ret) {
- DRM_ERROR("failed to create connector ret = %d\n", ret);
- drm_encoder_cleanup(encoder);
- return ret;
- }
-
- return 0;
-}
-
-static void exynos_dp_unbind(struct device *dev, struct device *master,
- void *data)
-{
- struct exynos_dp_device *dp = dev_get_drvdata(dev);
-
- exynos_dp_disable(&dp->encoder);
-}
-
-static const struct component_ops exynos_dp_ops = {
- .bind = exynos_dp_bind,
- .unbind = exynos_dp_unbind,
-};
-
-static int exynos_dp_probe(struct platform_device *pdev)
-{
- struct device *dev = &pdev->dev;
- struct device_node *np = NULL, *endpoint = NULL;
- struct exynos_dp_device *dp;
- int ret;
-
- dp = devm_kzalloc(&pdev->dev, sizeof(struct exynos_dp_device),
- GFP_KERNEL);
- if (!dp)
- return -ENOMEM;
-
- platform_set_drvdata(pdev, dp);
-
- /* This is for the backward compatibility. */
- np = of_parse_phandle(dev->of_node, "panel", 0);
- if (np) {
- dp->panel = of_drm_find_panel(np);
- of_node_put(np);
- if (!dp->panel)
- return -EPROBE_DEFER;
- goto out;
- }
-
- endpoint = of_graph_get_next_endpoint(dev->of_node, NULL);
- if (endpoint) {
- np = of_graph_get_remote_port_parent(endpoint);
- if (np) {
- /* The remote port can be either a panel or a bridge */
- dp->panel = of_drm_find_panel(np);
- if (!dp->panel) {
- dp->ptn_bridge = of_drm_find_bridge(np);
- if (!dp->ptn_bridge) {
- of_node_put(np);
- return -EPROBE_DEFER;
- }
- }
- of_node_put(np);
- } else {
- DRM_ERROR("no remote endpoint device node found.\n");
- return -EINVAL;
- }
- } else {
- DRM_ERROR("no port endpoint subnode found.\n");
- return -EINVAL;
- }
-
-out:
- pm_runtime_enable(dev);
-
- ret = component_add(&pdev->dev, &exynos_dp_ops);
- if (ret)
- goto err_disable_pm_runtime;
-
- return ret;
-
-err_disable_pm_runtime:
- pm_runtime_disable(dev);
-
- return ret;
-}
-
-static int exynos_dp_remove(struct platform_device *pdev)
-{
- pm_runtime_disable(&pdev->dev);
- component_del(&pdev->dev, &exynos_dp_ops);
-
- return 0;
-}
-
-#ifdef CONFIG_PM
-static int exynos_dp_suspend(struct device *dev)
-{
- struct exynos_dp_device *dp = dev_get_drvdata(dev);
-
- clk_disable_unprepare(dp->clock);
-
- return 0;
-}
-
-static int exynos_dp_resume(struct device *dev)
-{
- struct exynos_dp_device *dp = dev_get_drvdata(dev);
- int ret;
-
- ret = clk_prepare_enable(dp->clock);
- if (ret < 0) {
- DRM_ERROR("Failed to prepare_enable the clock clk [%d]\n", ret);
- return ret;
- }
-
- return 0;
-}
-#endif
-
-static const struct dev_pm_ops exynos_dp_pm_ops = {
- SET_RUNTIME_PM_OPS(exynos_dp_suspend, exynos_dp_resume, NULL)
-};
-
-static const struct of_device_id exynos_dp_match[] = {
- { .compatible = "samsung,exynos5-dp" },
- {},
-};
-MODULE_DEVICE_TABLE(of, exynos_dp_match);
-
-struct platform_driver dp_driver = {
- .probe = exynos_dp_probe,
- .remove = exynos_dp_remove,
- .driver = {
- .name = "exynos-dp",
- .owner = THIS_MODULE,
- .pm = &exynos_dp_pm_ops,
- .of_match_table = exynos_dp_match,
- },
-};
-
-MODULE_AUTHOR("Jingoo Han <jg1.han@samsung.com>");
-MODULE_DESCRIPTION("Samsung SoC DP Driver");
-MODULE_LICENSE("GPL v2");
diff --git a/drivers/gpu/drm/exynos/exynos_dp_core.h b/drivers/gpu/drm/exynos/exynos_dp_core.h
deleted file mode 100644
index b5c2d8f..0000000
--- a/drivers/gpu/drm/exynos/exynos_dp_core.h
+++ /dev/null
@@ -1,282 +0,0 @@
-/*
- * Header file for Samsung DP (Display Port) interface driver.
- *
- * Copyright (C) 2012 Samsung Electronics Co., Ltd.
- * Author: Jingoo Han <jg1.han@samsung.com>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- */
-
-#ifndef _EXYNOS_DP_CORE_H
-#define _EXYNOS_DP_CORE_H
-
-#include <drm/drm_crtc.h>
-#include <drm/drm_dp_helper.h>
-#include <drm/exynos_drm.h>
-#include <video/videomode.h>
-
-#include "exynos_drm_drv.h"
-
-#define DP_TIMEOUT_LOOP_COUNT 100
-#define MAX_CR_LOOP 5
-#define MAX_EQ_LOOP 5
-
-enum link_rate_type {
- LINK_RATE_1_62GBPS = 0x06,
- LINK_RATE_2_70GBPS = 0x0a
-};
-
-enum link_lane_count_type {
- LANE_COUNT1 = 1,
- LANE_COUNT2 = 2,
- LANE_COUNT4 = 4
-};
-
-enum link_training_state {
- START,
- CLOCK_RECOVERY,
- EQUALIZER_TRAINING,
- FINISHED,
- FAILED
-};
-
-enum voltage_swing_level {
- VOLTAGE_LEVEL_0,
- VOLTAGE_LEVEL_1,
- VOLTAGE_LEVEL_2,
- VOLTAGE_LEVEL_3,
-};
-
-enum pre_emphasis_level {
- PRE_EMPHASIS_LEVEL_0,
- PRE_EMPHASIS_LEVEL_1,
- PRE_EMPHASIS_LEVEL_2,
- PRE_EMPHASIS_LEVEL_3,
-};
-
-enum pattern_set {
- PRBS7,
- D10_2,
- TRAINING_PTN1,
- TRAINING_PTN2,
- DP_NONE
-};
-
-enum color_space {
- COLOR_RGB,
- COLOR_YCBCR422,
- COLOR_YCBCR444
-};
-
-enum color_depth {
- COLOR_6,
- COLOR_8,
- COLOR_10,
- COLOR_12
-};
-
-enum color_coefficient {
- COLOR_YCBCR601,
- COLOR_YCBCR709
-};
-
-enum dynamic_range {
- VESA,
- CEA
-};
-
-enum pll_status {
- PLL_UNLOCKED,
- PLL_LOCKED
-};
-
-enum clock_recovery_m_value_type {
- CALCULATED_M,
- REGISTER_M
-};
-
-enum video_timing_recognition_type {
- VIDEO_TIMING_FROM_CAPTURE,
- VIDEO_TIMING_FROM_REGISTER
-};
-
-enum analog_power_block {
- AUX_BLOCK,
- CH0_BLOCK,
- CH1_BLOCK,
- CH2_BLOCK,
- CH3_BLOCK,
- ANALOG_TOTAL,
- POWER_ALL
-};
-
-enum dp_irq_type {
- DP_IRQ_TYPE_HP_CABLE_IN,
- DP_IRQ_TYPE_HP_CABLE_OUT,
- DP_IRQ_TYPE_HP_CHANGE,
- DP_IRQ_TYPE_UNKNOWN,
-};
-
-struct video_info {
- char *name;
-
- bool h_sync_polarity;
- bool v_sync_polarity;
- bool interlaced;
-
- enum color_space color_space;
- enum dynamic_range dynamic_range;
- enum color_coefficient ycbcr_coeff;
- enum color_depth color_depth;
-
- enum link_rate_type link_rate;
- enum link_lane_count_type lane_count;
-};
-
-struct link_train {
- int eq_loop;
- int cr_loop[4];
-
- u8 link_rate;
- u8 lane_count;
- u8 training_lane[4];
-
- enum link_training_state lt_state;
-};
-
-struct exynos_dp_device {
- struct drm_encoder encoder;
- struct device *dev;
- struct drm_device *drm_dev;
- struct drm_connector connector;
- struct drm_panel *panel;
- struct drm_bridge *bridge;
- struct drm_bridge *ptn_bridge;
- struct clk *clock;
- unsigned int irq;
- void __iomem *reg_base;
-
- struct video_info *video_info;
- struct link_train link_train;
- struct work_struct hotplug_work;
- struct phy *phy;
- int dpms_mode;
- int hpd_gpio;
- struct videomode vm;
-};
-
-/* exynos_dp_reg.c */
-void exynos_dp_enable_video_mute(struct exynos_dp_device *dp, bool enable);
-void exynos_dp_stop_video(struct exynos_dp_device *dp);
-void exynos_dp_lane_swap(struct exynos_dp_device *dp, bool enable);
-void exynos_dp_init_analog_param(struct exynos_dp_device *dp);
-void exynos_dp_init_interrupt(struct exynos_dp_device *dp);
-void exynos_dp_reset(struct exynos_dp_device *dp);
-void exynos_dp_swreset(struct exynos_dp_device *dp);
-void exynos_dp_config_interrupt(struct exynos_dp_device *dp);
-enum pll_status exynos_dp_get_pll_lock_status(struct exynos_dp_device *dp);
-void exynos_dp_set_pll_power_down(struct exynos_dp_device *dp, bool enable);
-void exynos_dp_set_analog_power_down(struct exynos_dp_device *dp,
- enum analog_power_block block,
- bool enable);
-void exynos_dp_init_analog_func(struct exynos_dp_device *dp);
-void exynos_dp_init_hpd(struct exynos_dp_device *dp);
-enum dp_irq_type exynos_dp_get_irq_type(struct exynos_dp_device *dp);
-void exynos_dp_clear_hotplug_interrupts(struct exynos_dp_device *dp);
-void exynos_dp_reset_aux(struct exynos_dp_device *dp);
-void exynos_dp_init_aux(struct exynos_dp_device *dp);
-int exynos_dp_get_plug_in_status(struct exynos_dp_device *dp);
-void exynos_dp_enable_sw_function(struct exynos_dp_device *dp);
-int exynos_dp_start_aux_transaction(struct exynos_dp_device *dp);
-int exynos_dp_write_byte_to_dpcd(struct exynos_dp_device *dp,
- unsigned int reg_addr,
- unsigned char data);
-int exynos_dp_read_byte_from_dpcd(struct exynos_dp_device *dp,
- unsigned int reg_addr,
- unsigned char *data);
-int exynos_dp_write_bytes_to_dpcd(struct exynos_dp_device *dp,
- unsigned int reg_addr,
- unsigned int count,
- unsigned char data[]);
-int exynos_dp_read_bytes_from_dpcd(struct exynos_dp_device *dp,
- unsigned int reg_addr,
- unsigned int count,
- unsigned char data[]);
-int exynos_dp_select_i2c_device(struct exynos_dp_device *dp,
- unsigned int device_addr,
- unsigned int reg_addr);
-int exynos_dp_read_byte_from_i2c(struct exynos_dp_device *dp,
- unsigned int device_addr,
- unsigned int reg_addr,
- unsigned int *data);
-int exynos_dp_read_bytes_from_i2c(struct exynos_dp_device *dp,
- unsigned int device_addr,
- unsigned int reg_addr,
- unsigned int count,
- unsigned char edid[]);
-void exynos_dp_set_link_bandwidth(struct exynos_dp_device *dp, u32 bwtype);
-void exynos_dp_get_link_bandwidth(struct exynos_dp_device *dp, u32 *bwtype);
-void exynos_dp_set_lane_count(struct exynos_dp_device *dp, u32 count);
-void exynos_dp_get_lane_count(struct exynos_dp_device *dp, u32 *count);
-void exynos_dp_enable_enhanced_mode(struct exynos_dp_device *dp, bool enable);
-void exynos_dp_set_training_pattern(struct exynos_dp_device *dp,
- enum pattern_set pattern);
-void exynos_dp_set_lane0_pre_emphasis(struct exynos_dp_device *dp, u32 level);
-void exynos_dp_set_lane1_pre_emphasis(struct exynos_dp_device *dp, u32 level);
-void exynos_dp_set_lane2_pre_emphasis(struct exynos_dp_device *dp, u32 level);
-void exynos_dp_set_lane3_pre_emphasis(struct exynos_dp_device *dp, u32 level);
-void exynos_dp_set_lane0_link_training(struct exynos_dp_device *dp,
- u32 training_lane);
-void exynos_dp_set_lane1_link_training(struct exynos_dp_device *dp,
- u32 training_lane);
-void exynos_dp_set_lane2_link_training(struct exynos_dp_device *dp,
- u32 training_lane);
-void exynos_dp_set_lane3_link_training(struct exynos_dp_device *dp,
- u32 training_lane);
-u32 exynos_dp_get_lane0_link_training(struct exynos_dp_device *dp);
-u32 exynos_dp_get_lane1_link_training(struct exynos_dp_device *dp);
-u32 exynos_dp_get_lane2_link_training(struct exynos_dp_device *dp);
-u32 exynos_dp_get_lane3_link_training(struct exynos_dp_device *dp);
-void exynos_dp_reset_macro(struct exynos_dp_device *dp);
-void exynos_dp_init_video(struct exynos_dp_device *dp);
-
-void exynos_dp_set_video_color_format(struct exynos_dp_device *dp);
-int exynos_dp_is_slave_video_stream_clock_on(struct exynos_dp_device *dp);
-void exynos_dp_set_video_cr_mn(struct exynos_dp_device *dp,
- enum clock_recovery_m_value_type type,
- u32 m_value,
- u32 n_value);
-void exynos_dp_set_video_timing_mode(struct exynos_dp_device *dp, u32 type);
-void exynos_dp_enable_video_master(struct exynos_dp_device *dp, bool enable);
-void exynos_dp_start_video(struct exynos_dp_device *dp);
-int exynos_dp_is_video_stream_on(struct exynos_dp_device *dp);
-void exynos_dp_config_video_slave_mode(struct exynos_dp_device *dp);
-void exynos_dp_enable_scrambling(struct exynos_dp_device *dp);
-void exynos_dp_disable_scrambling(struct exynos_dp_device *dp);
-
-/* I2C EDID Chip ID, Slave Address */
-#define I2C_EDID_DEVICE_ADDR 0x50
-#define I2C_E_EDID_DEVICE_ADDR 0x30
-
-#define EDID_BLOCK_LENGTH 0x80
-#define EDID_HEADER_PATTERN 0x00
-#define EDID_EXTENSION_FLAG 0x7e
-#define EDID_CHECKSUM 0x7f
-
-/* DP_MAX_LANE_COUNT */
-#define DPCD_ENHANCED_FRAME_CAP(x) (((x) >> 7) & 0x1)
-#define DPCD_MAX_LANE_COUNT(x) ((x) & 0x1f)
-
-/* DP_LANE_COUNT_SET */
-#define DPCD_LANE_COUNT_SET(x) ((x) & 0x1f)
-
-/* DP_TRAINING_LANE0_SET */
-#define DPCD_PRE_EMPHASIS_SET(x) (((x) & 0x3) << 3)
-#define DPCD_PRE_EMPHASIS_GET(x) (((x) >> 3) & 0x3)
-#define DPCD_VOLTAGE_SWING_SET(x) (((x) & 0x3) << 0)
-#define DPCD_VOLTAGE_SWING_GET(x) (((x) >> 0) & 0x3)
-
-#endif /* _EXYNOS_DP_CORE_H */
diff --git a/drivers/gpu/drm/exynos/exynos_dp_reg.c b/drivers/gpu/drm/exynos/exynos_dp_reg.c
deleted file mode 100644
index c1f87a2..0000000
--- a/drivers/gpu/drm/exynos/exynos_dp_reg.c
+++ /dev/null
@@ -1,1263 +0,0 @@
-/*
- * Samsung DP (Display port) register interface driver.
- *
- * Copyright (C) 2012 Samsung Electronics Co., Ltd.
- * Author: Jingoo Han <jg1.han@samsung.com>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- */
-
-#include <linux/device.h>
-#include <linux/io.h>
-#include <linux/delay.h>
-#include <linux/gpio.h>
-
-#include "exynos_dp_core.h"
-#include "exynos_dp_reg.h"
-
-#define COMMON_INT_MASK_1 0
-#define COMMON_INT_MASK_2 0
-#define COMMON_INT_MASK_3 0
-#define COMMON_INT_MASK_4 (HOTPLUG_CHG | HPD_LOST | PLUG)
-#define INT_STA_MASK INT_HPD
-
-void exynos_dp_enable_video_mute(struct exynos_dp_device *dp, bool enable)
-{
- u32 reg;
-
- if (enable) {
- reg = readl(dp->reg_base + EXYNOS_DP_VIDEO_CTL_1);
- reg |= HDCP_VIDEO_MUTE;
- writel(reg, dp->reg_base + EXYNOS_DP_VIDEO_CTL_1);
- } else {
- reg = readl(dp->reg_base + EXYNOS_DP_VIDEO_CTL_1);
- reg &= ~HDCP_VIDEO_MUTE;
- writel(reg, dp->reg_base + EXYNOS_DP_VIDEO_CTL_1);
- }
-}
-
-void exynos_dp_stop_video(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- reg = readl(dp->reg_base + EXYNOS_DP_VIDEO_CTL_1);
- reg &= ~VIDEO_EN;
- writel(reg, dp->reg_base + EXYNOS_DP_VIDEO_CTL_1);
-}
-
-void exynos_dp_lane_swap(struct exynos_dp_device *dp, bool enable)
-{
- u32 reg;
-
- if (enable)
- reg = LANE3_MAP_LOGIC_LANE_0 | LANE2_MAP_LOGIC_LANE_1 |
- LANE1_MAP_LOGIC_LANE_2 | LANE0_MAP_LOGIC_LANE_3;
- else
- reg = LANE3_MAP_LOGIC_LANE_3 | LANE2_MAP_LOGIC_LANE_2 |
- LANE1_MAP_LOGIC_LANE_1 | LANE0_MAP_LOGIC_LANE_0;
-
- writel(reg, dp->reg_base + EXYNOS_DP_LANE_MAP);
-}
-
-void exynos_dp_init_analog_param(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- reg = TX_TERMINAL_CTRL_50_OHM;
- writel(reg, dp->reg_base + EXYNOS_DP_ANALOG_CTL_1);
-
- reg = SEL_24M | TX_DVDD_BIT_1_0625V;
- writel(reg, dp->reg_base + EXYNOS_DP_ANALOG_CTL_2);
-
- reg = DRIVE_DVDD_BIT_1_0625V | VCO_BIT_600_MICRO;
- writel(reg, dp->reg_base + EXYNOS_DP_ANALOG_CTL_3);
-
- reg = PD_RING_OSC | AUX_TERMINAL_CTRL_50_OHM |
- TX_CUR1_2X | TX_CUR_16_MA;
- writel(reg, dp->reg_base + EXYNOS_DP_PLL_FILTER_CTL_1);
-
- reg = CH3_AMP_400_MV | CH2_AMP_400_MV |
- CH1_AMP_400_MV | CH0_AMP_400_MV;
- writel(reg, dp->reg_base + EXYNOS_DP_TX_AMP_TUNING_CTL);
-}
-
-void exynos_dp_init_interrupt(struct exynos_dp_device *dp)
-{
- /* Set interrupt pin assertion polarity as high */
- writel(INT_POL1 | INT_POL0, dp->reg_base + EXYNOS_DP_INT_CTL);
-
- /* Clear pending regisers */
- writel(0xff, dp->reg_base + EXYNOS_DP_COMMON_INT_STA_1);
- writel(0x4f, dp->reg_base + EXYNOS_DP_COMMON_INT_STA_2);
- writel(0xe0, dp->reg_base + EXYNOS_DP_COMMON_INT_STA_3);
- writel(0xe7, dp->reg_base + EXYNOS_DP_COMMON_INT_STA_4);
- writel(0x63, dp->reg_base + EXYNOS_DP_INT_STA);
-
- /* 0:mask,1: unmask */
- writel(0x00, dp->reg_base + EXYNOS_DP_COMMON_INT_MASK_1);
- writel(0x00, dp->reg_base + EXYNOS_DP_COMMON_INT_MASK_2);
- writel(0x00, dp->reg_base + EXYNOS_DP_COMMON_INT_MASK_3);
- writel(0x00, dp->reg_base + EXYNOS_DP_COMMON_INT_MASK_4);
- writel(0x00, dp->reg_base + EXYNOS_DP_INT_STA_MASK);
-}
-
-void exynos_dp_reset(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- exynos_dp_stop_video(dp);
- exynos_dp_enable_video_mute(dp, 0);
-
- reg = MASTER_VID_FUNC_EN_N | SLAVE_VID_FUNC_EN_N |
- AUD_FIFO_FUNC_EN_N | AUD_FUNC_EN_N |
- HDCP_FUNC_EN_N | SW_FUNC_EN_N;
- writel(reg, dp->reg_base + EXYNOS_DP_FUNC_EN_1);
-
- reg = SSC_FUNC_EN_N | AUX_FUNC_EN_N |
- SERDES_FIFO_FUNC_EN_N |
- LS_CLK_DOMAIN_FUNC_EN_N;
- writel(reg, dp->reg_base + EXYNOS_DP_FUNC_EN_2);
-
- usleep_range(20, 30);
-
- exynos_dp_lane_swap(dp, 0);
-
- writel(0x0, dp->reg_base + EXYNOS_DP_SYS_CTL_1);
- writel(0x40, dp->reg_base + EXYNOS_DP_SYS_CTL_2);
- writel(0x0, dp->reg_base + EXYNOS_DP_SYS_CTL_3);
- writel(0x0, dp->reg_base + EXYNOS_DP_SYS_CTL_4);
-
- writel(0x0, dp->reg_base + EXYNOS_DP_PKT_SEND_CTL);
- writel(0x0, dp->reg_base + EXYNOS_DP_HDCP_CTL);
-
- writel(0x5e, dp->reg_base + EXYNOS_DP_HPD_DEGLITCH_L);
- writel(0x1a, dp->reg_base + EXYNOS_DP_HPD_DEGLITCH_H);
-
- writel(0x10, dp->reg_base + EXYNOS_DP_LINK_DEBUG_CTL);
-
- writel(0x0, dp->reg_base + EXYNOS_DP_PHY_TEST);
-
- writel(0x0, dp->reg_base + EXYNOS_DP_VIDEO_FIFO_THRD);
- writel(0x20, dp->reg_base + EXYNOS_DP_AUDIO_MARGIN);
-
- writel(0x4, dp->reg_base + EXYNOS_DP_M_VID_GEN_FILTER_TH);
- writel(0x2, dp->reg_base + EXYNOS_DP_M_AUD_GEN_FILTER_TH);
-
- writel(0x00000101, dp->reg_base + EXYNOS_DP_SOC_GENERAL_CTL);
-}
-
-void exynos_dp_swreset(struct exynos_dp_device *dp)
-{
- writel(RESET_DP_TX, dp->reg_base + EXYNOS_DP_TX_SW_RESET);
-}
-
-void exynos_dp_config_interrupt(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- /* 0: mask, 1: unmask */
- reg = COMMON_INT_MASK_1;
- writel(reg, dp->reg_base + EXYNOS_DP_COMMON_INT_MASK_1);
-
- reg = COMMON_INT_MASK_2;
- writel(reg, dp->reg_base + EXYNOS_DP_COMMON_INT_MASK_2);
-
- reg = COMMON_INT_MASK_3;
- writel(reg, dp->reg_base + EXYNOS_DP_COMMON_INT_MASK_3);
-
- reg = COMMON_INT_MASK_4;
- writel(reg, dp->reg_base + EXYNOS_DP_COMMON_INT_MASK_4);
-
- reg = INT_STA_MASK;
- writel(reg, dp->reg_base + EXYNOS_DP_INT_STA_MASK);
-}
-
-enum pll_status exynos_dp_get_pll_lock_status(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- reg = readl(dp->reg_base + EXYNOS_DP_DEBUG_CTL);
- if (reg & PLL_LOCK)
- return PLL_LOCKED;
- else
- return PLL_UNLOCKED;
-}
-
-void exynos_dp_set_pll_power_down(struct exynos_dp_device *dp, bool enable)
-{
- u32 reg;
-
- if (enable) {
- reg = readl(dp->reg_base + EXYNOS_DP_PLL_CTL);
- reg |= DP_PLL_PD;
- writel(reg, dp->reg_base + EXYNOS_DP_PLL_CTL);
- } else {
- reg = readl(dp->reg_base + EXYNOS_DP_PLL_CTL);
- reg &= ~DP_PLL_PD;
- writel(reg, dp->reg_base + EXYNOS_DP_PLL_CTL);
- }
-}
-
-void exynos_dp_set_analog_power_down(struct exynos_dp_device *dp,
- enum analog_power_block block,
- bool enable)
-{
- u32 reg;
-
- switch (block) {
- case AUX_BLOCK:
- if (enable) {
- reg = readl(dp->reg_base + EXYNOS_DP_PHY_PD);
- reg |= AUX_PD;
- writel(reg, dp->reg_base + EXYNOS_DP_PHY_PD);
- } else {
- reg = readl(dp->reg_base + EXYNOS_DP_PHY_PD);
- reg &= ~AUX_PD;
- writel(reg, dp->reg_base + EXYNOS_DP_PHY_PD);
- }
- break;
- case CH0_BLOCK:
- if (enable) {
- reg = readl(dp->reg_base + EXYNOS_DP_PHY_PD);
- reg |= CH0_PD;
- writel(reg, dp->reg_base + EXYNOS_DP_PHY_PD);
- } else {
- reg = readl(dp->reg_base + EXYNOS_DP_PHY_PD);
- reg &= ~CH0_PD;
- writel(reg, dp->reg_base + EXYNOS_DP_PHY_PD);
- }
- break;
- case CH1_BLOCK:
- if (enable) {
- reg = readl(dp->reg_base + EXYNOS_DP_PHY_PD);
- reg |= CH1_PD;
- writel(reg, dp->reg_base + EXYNOS_DP_PHY_PD);
- } else {
- reg = readl(dp->reg_base + EXYNOS_DP_PHY_PD);
- reg &= ~CH1_PD;
- writel(reg, dp->reg_base + EXYNOS_DP_PHY_PD);
- }
- break;
- case CH2_BLOCK:
- if (enable) {
- reg = readl(dp->reg_base + EXYNOS_DP_PHY_PD);
- reg |= CH2_PD;
- writel(reg, dp->reg_base + EXYNOS_DP_PHY_PD);
- } else {
- reg = readl(dp->reg_base + EXYNOS_DP_PHY_PD);
- reg &= ~CH2_PD;
- writel(reg, dp->reg_base + EXYNOS_DP_PHY_PD);
- }
- break;
- case CH3_BLOCK:
- if (enable) {
- reg = readl(dp->reg_base + EXYNOS_DP_PHY_PD);
- reg |= CH3_PD;
- writel(reg, dp->reg_base + EXYNOS_DP_PHY_PD);
- } else {
- reg = readl(dp->reg_base + EXYNOS_DP_PHY_PD);
- reg &= ~CH3_PD;
- writel(reg, dp->reg_base + EXYNOS_DP_PHY_PD);
- }
- break;
- case ANALOG_TOTAL:
- if (enable) {
- reg = readl(dp->reg_base + EXYNOS_DP_PHY_PD);
- reg |= DP_PHY_PD;
- writel(reg, dp->reg_base + EXYNOS_DP_PHY_PD);
- } else {
- reg = readl(dp->reg_base + EXYNOS_DP_PHY_PD);
- reg &= ~DP_PHY_PD;
- writel(reg, dp->reg_base + EXYNOS_DP_PHY_PD);
- }
- break;
- case POWER_ALL:
- if (enable) {
- reg = DP_PHY_PD | AUX_PD | CH3_PD | CH2_PD |
- CH1_PD | CH0_PD;
- writel(reg, dp->reg_base + EXYNOS_DP_PHY_PD);
- } else {
- writel(0x00, dp->reg_base + EXYNOS_DP_PHY_PD);
- }
- break;
- default:
- break;
- }
-}
-
-void exynos_dp_init_analog_func(struct exynos_dp_device *dp)
-{
- u32 reg;
- int timeout_loop = 0;
-
- exynos_dp_set_analog_power_down(dp, POWER_ALL, 0);
-
- reg = PLL_LOCK_CHG;
- writel(reg, dp->reg_base + EXYNOS_DP_COMMON_INT_STA_1);
-
- reg = readl(dp->reg_base + EXYNOS_DP_DEBUG_CTL);
- reg &= ~(F_PLL_LOCK | PLL_LOCK_CTRL);
- writel(reg, dp->reg_base + EXYNOS_DP_DEBUG_CTL);
-
- /* Power up PLL */
- if (exynos_dp_get_pll_lock_status(dp) == PLL_UNLOCKED) {
- exynos_dp_set_pll_power_down(dp, 0);
-
- while (exynos_dp_get_pll_lock_status(dp) == PLL_UNLOCKED) {
- timeout_loop++;
- if (DP_TIMEOUT_LOOP_COUNT < timeout_loop) {
- dev_err(dp->dev, "failed to get pll lock status\n");
- return;
- }
- usleep_range(10, 20);
- }
- }
-
- /* Enable Serdes FIFO function and Link symbol clock domain module */
- reg = readl(dp->reg_base + EXYNOS_DP_FUNC_EN_2);
- reg &= ~(SERDES_FIFO_FUNC_EN_N | LS_CLK_DOMAIN_FUNC_EN_N
- | AUX_FUNC_EN_N);
- writel(reg, dp->reg_base + EXYNOS_DP_FUNC_EN_2);
-}
-
-void exynos_dp_clear_hotplug_interrupts(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- if (gpio_is_valid(dp->hpd_gpio))
- return;
-
- reg = HOTPLUG_CHG | HPD_LOST | PLUG;
- writel(reg, dp->reg_base + EXYNOS_DP_COMMON_INT_STA_4);
-
- reg = INT_HPD;
- writel(reg, dp->reg_base + EXYNOS_DP_INT_STA);
-}
-
-void exynos_dp_init_hpd(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- if (gpio_is_valid(dp->hpd_gpio))
- return;
-
- exynos_dp_clear_hotplug_interrupts(dp);
-
- reg = readl(dp->reg_base + EXYNOS_DP_SYS_CTL_3);
- reg &= ~(F_HPD | HPD_CTRL);
- writel(reg, dp->reg_base + EXYNOS_DP_SYS_CTL_3);
-}
-
-enum dp_irq_type exynos_dp_get_irq_type(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- if (gpio_is_valid(dp->hpd_gpio)) {
- reg = gpio_get_value(dp->hpd_gpio);
- if (reg)
- return DP_IRQ_TYPE_HP_CABLE_IN;
- else
- return DP_IRQ_TYPE_HP_CABLE_OUT;
- } else {
- /* Parse hotplug interrupt status register */
- reg = readl(dp->reg_base + EXYNOS_DP_COMMON_INT_STA_4);
-
- if (reg & PLUG)
- return DP_IRQ_TYPE_HP_CABLE_IN;
-
- if (reg & HPD_LOST)
- return DP_IRQ_TYPE_HP_CABLE_OUT;
-
- if (reg & HOTPLUG_CHG)
- return DP_IRQ_TYPE_HP_CHANGE;
-
- return DP_IRQ_TYPE_UNKNOWN;
- }
-}
-
-void exynos_dp_reset_aux(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- /* Disable AUX channel module */
- reg = readl(dp->reg_base + EXYNOS_DP_FUNC_EN_2);
- reg |= AUX_FUNC_EN_N;
- writel(reg, dp->reg_base + EXYNOS_DP_FUNC_EN_2);
-}
-
-void exynos_dp_init_aux(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- /* Clear inerrupts related to AUX channel */
- reg = RPLY_RECEIV | AUX_ERR;
- writel(reg, dp->reg_base + EXYNOS_DP_INT_STA);
-
- exynos_dp_reset_aux(dp);
-
- /* Disable AUX transaction H/W retry */
- reg = AUX_BIT_PERIOD_EXPECTED_DELAY(3) | AUX_HW_RETRY_COUNT_SEL(0)|
- AUX_HW_RETRY_INTERVAL_600_MICROSECONDS;
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_HW_RETRY_CTL);
-
- /* Receive AUX Channel DEFER commands equal to DEFFER_COUNT*64 */
- reg = DEFER_CTRL_EN | DEFER_COUNT(1);
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_CH_DEFER_CTL);
-
- /* Enable AUX channel module */
- reg = readl(dp->reg_base + EXYNOS_DP_FUNC_EN_2);
- reg &= ~AUX_FUNC_EN_N;
- writel(reg, dp->reg_base + EXYNOS_DP_FUNC_EN_2);
-}
-
-int exynos_dp_get_plug_in_status(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- if (gpio_is_valid(dp->hpd_gpio)) {
- if (gpio_get_value(dp->hpd_gpio))
- return 0;
- } else {
- reg = readl(dp->reg_base + EXYNOS_DP_SYS_CTL_3);
- if (reg & HPD_STATUS)
- return 0;
- }
-
- return -EINVAL;
-}
-
-void exynos_dp_enable_sw_function(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- reg = readl(dp->reg_base + EXYNOS_DP_FUNC_EN_1);
- reg &= ~SW_FUNC_EN_N;
- writel(reg, dp->reg_base + EXYNOS_DP_FUNC_EN_1);
-}
-
-int exynos_dp_start_aux_transaction(struct exynos_dp_device *dp)
-{
- int reg;
- int retval = 0;
- int timeout_loop = 0;
-
- /* Enable AUX CH operation */
- reg = readl(dp->reg_base + EXYNOS_DP_AUX_CH_CTL_2);
- reg |= AUX_EN;
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_CH_CTL_2);
-
- /* Is AUX CH command reply received? */
- reg = readl(dp->reg_base + EXYNOS_DP_INT_STA);
- while (!(reg & RPLY_RECEIV)) {
- timeout_loop++;
- if (DP_TIMEOUT_LOOP_COUNT < timeout_loop) {
- dev_err(dp->dev, "AUX CH command reply failed!\n");
- return -ETIMEDOUT;
- }
- reg = readl(dp->reg_base + EXYNOS_DP_INT_STA);
- usleep_range(10, 11);
- }
-
- /* Clear interrupt source for AUX CH command reply */
- writel(RPLY_RECEIV, dp->reg_base + EXYNOS_DP_INT_STA);
-
- /* Clear interrupt source for AUX CH access error */
- reg = readl(dp->reg_base + EXYNOS_DP_INT_STA);
- if (reg & AUX_ERR) {
- writel(AUX_ERR, dp->reg_base + EXYNOS_DP_INT_STA);
- return -EREMOTEIO;
- }
-
- /* Check AUX CH error access status */
- reg = readl(dp->reg_base + EXYNOS_DP_AUX_CH_STA);
- if ((reg & AUX_STATUS_MASK) != 0) {
- dev_err(dp->dev, "AUX CH error happens: %d\n\n",
- reg & AUX_STATUS_MASK);
- return -EREMOTEIO;
- }
-
- return retval;
-}
-
-int exynos_dp_write_byte_to_dpcd(struct exynos_dp_device *dp,
- unsigned int reg_addr,
- unsigned char data)
-{
- u32 reg;
- int i;
- int retval;
-
- for (i = 0; i < 3; i++) {
- /* Clear AUX CH data buffer */
- reg = BUF_CLR;
- writel(reg, dp->reg_base + EXYNOS_DP_BUFFER_DATA_CTL);
-
- /* Select DPCD device address */
- reg = AUX_ADDR_7_0(reg_addr);
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_ADDR_7_0);
- reg = AUX_ADDR_15_8(reg_addr);
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_ADDR_15_8);
- reg = AUX_ADDR_19_16(reg_addr);
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_ADDR_19_16);
-
- /* Write data buffer */
- reg = (unsigned int)data;
- writel(reg, dp->reg_base + EXYNOS_DP_BUF_DATA_0);
-
- /*
- * Set DisplayPort transaction and write 1 byte
- * If bit 3 is 1, DisplayPort transaction.
- * If Bit 3 is 0, I2C transaction.
- */
- reg = AUX_TX_COMM_DP_TRANSACTION | AUX_TX_COMM_WRITE;
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_CH_CTL_1);
-
- /* Start AUX transaction */
- retval = exynos_dp_start_aux_transaction(dp);
- if (retval == 0)
- break;
- else
- dev_dbg(dp->dev, "%s: Aux Transaction fail!\n",
- __func__);
- }
-
- return retval;
-}
-
-int exynos_dp_read_byte_from_dpcd(struct exynos_dp_device *dp,
- unsigned int reg_addr,
- unsigned char *data)
-{
- u32 reg;
- int i;
- int retval;
-
- for (i = 0; i < 3; i++) {
- /* Clear AUX CH data buffer */
- reg = BUF_CLR;
- writel(reg, dp->reg_base + EXYNOS_DP_BUFFER_DATA_CTL);
-
- /* Select DPCD device address */
- reg = AUX_ADDR_7_0(reg_addr);
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_ADDR_7_0);
- reg = AUX_ADDR_15_8(reg_addr);
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_ADDR_15_8);
- reg = AUX_ADDR_19_16(reg_addr);
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_ADDR_19_16);
-
- /*
- * Set DisplayPort transaction and read 1 byte
- * If bit 3 is 1, DisplayPort transaction.
- * If Bit 3 is 0, I2C transaction.
- */
- reg = AUX_TX_COMM_DP_TRANSACTION | AUX_TX_COMM_READ;
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_CH_CTL_1);
-
- /* Start AUX transaction */
- retval = exynos_dp_start_aux_transaction(dp);
- if (retval == 0)
- break;
- else
- dev_dbg(dp->dev, "%s: Aux Transaction fail!\n",
- __func__);
- }
-
- /* Read data buffer */
- reg = readl(dp->reg_base + EXYNOS_DP_BUF_DATA_0);
- *data = (unsigned char)(reg & 0xff);
-
- return retval;
-}
-
-int exynos_dp_write_bytes_to_dpcd(struct exynos_dp_device *dp,
- unsigned int reg_addr,
- unsigned int count,
- unsigned char data[])
-{
- u32 reg;
- unsigned int start_offset;
- unsigned int cur_data_count;
- unsigned int cur_data_idx;
- int i;
- int retval = 0;
-
- /* Clear AUX CH data buffer */
- reg = BUF_CLR;
- writel(reg, dp->reg_base + EXYNOS_DP_BUFFER_DATA_CTL);
-
- start_offset = 0;
- while (start_offset < count) {
- /* Buffer size of AUX CH is 16 * 4bytes */
- if ((count - start_offset) > 16)
- cur_data_count = 16;
- else
- cur_data_count = count - start_offset;
-
- for (i = 0; i < 3; i++) {
- /* Select DPCD device address */
- reg = AUX_ADDR_7_0(reg_addr + start_offset);
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_ADDR_7_0);
- reg = AUX_ADDR_15_8(reg_addr + start_offset);
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_ADDR_15_8);
- reg = AUX_ADDR_19_16(reg_addr + start_offset);
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_ADDR_19_16);
-
- for (cur_data_idx = 0; cur_data_idx < cur_data_count;
- cur_data_idx++) {
- reg = data[start_offset + cur_data_idx];
- writel(reg, dp->reg_base + EXYNOS_DP_BUF_DATA_0
- + 4 * cur_data_idx);
- }
-
- /*
- * Set DisplayPort transaction and write
- * If bit 3 is 1, DisplayPort transaction.
- * If Bit 3 is 0, I2C transaction.
- */
- reg = AUX_LENGTH(cur_data_count) |
- AUX_TX_COMM_DP_TRANSACTION | AUX_TX_COMM_WRITE;
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_CH_CTL_1);
-
- /* Start AUX transaction */
- retval = exynos_dp_start_aux_transaction(dp);
- if (retval == 0)
- break;
- else
- dev_dbg(dp->dev, "%s: Aux Transaction fail!\n",
- __func__);
- }
-
- start_offset += cur_data_count;
- }
-
- return retval;
-}
-
-int exynos_dp_read_bytes_from_dpcd(struct exynos_dp_device *dp,
- unsigned int reg_addr,
- unsigned int count,
- unsigned char data[])
-{
- u32 reg;
- unsigned int start_offset;
- unsigned int cur_data_count;
- unsigned int cur_data_idx;
- int i;
- int retval = 0;
-
- /* Clear AUX CH data buffer */
- reg = BUF_CLR;
- writel(reg, dp->reg_base + EXYNOS_DP_BUFFER_DATA_CTL);
-
- start_offset = 0;
- while (start_offset < count) {
- /* Buffer size of AUX CH is 16 * 4bytes */
- if ((count - start_offset) > 16)
- cur_data_count = 16;
- else
- cur_data_count = count - start_offset;
-
- /* AUX CH Request Transaction process */
- for (i = 0; i < 3; i++) {
- /* Select DPCD device address */
- reg = AUX_ADDR_7_0(reg_addr + start_offset);
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_ADDR_7_0);
- reg = AUX_ADDR_15_8(reg_addr + start_offset);
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_ADDR_15_8);
- reg = AUX_ADDR_19_16(reg_addr + start_offset);
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_ADDR_19_16);
-
- /*
- * Set DisplayPort transaction and read
- * If bit 3 is 1, DisplayPort transaction.
- * If Bit 3 is 0, I2C transaction.
- */
- reg = AUX_LENGTH(cur_data_count) |
- AUX_TX_COMM_DP_TRANSACTION | AUX_TX_COMM_READ;
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_CH_CTL_1);
-
- /* Start AUX transaction */
- retval = exynos_dp_start_aux_transaction(dp);
- if (retval == 0)
- break;
- else
- dev_dbg(dp->dev, "%s: Aux Transaction fail!\n",
- __func__);
- }
-
- for (cur_data_idx = 0; cur_data_idx < cur_data_count;
- cur_data_idx++) {
- reg = readl(dp->reg_base + EXYNOS_DP_BUF_DATA_0
- + 4 * cur_data_idx);
- data[start_offset + cur_data_idx] =
- (unsigned char)reg;
- }
-
- start_offset += cur_data_count;
- }
-
- return retval;
-}
-
-int exynos_dp_select_i2c_device(struct exynos_dp_device *dp,
- unsigned int device_addr,
- unsigned int reg_addr)
-{
- u32 reg;
- int retval;
-
- /* Set EDID device address */
- reg = device_addr;
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_ADDR_7_0);
- writel(0x0, dp->reg_base + EXYNOS_DP_AUX_ADDR_15_8);
- writel(0x0, dp->reg_base + EXYNOS_DP_AUX_ADDR_19_16);
-
- /* Set offset from base address of EDID device */
- writel(reg_addr, dp->reg_base + EXYNOS_DP_BUF_DATA_0);
-
- /*
- * Set I2C transaction and write address
- * If bit 3 is 1, DisplayPort transaction.
- * If Bit 3 is 0, I2C transaction.
- */
- reg = AUX_TX_COMM_I2C_TRANSACTION | AUX_TX_COMM_MOT |
- AUX_TX_COMM_WRITE;
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_CH_CTL_1);
-
- /* Start AUX transaction */
- retval = exynos_dp_start_aux_transaction(dp);
- if (retval != 0)
- dev_dbg(dp->dev, "%s: Aux Transaction fail!\n", __func__);
-
- return retval;
-}
-
-int exynos_dp_read_byte_from_i2c(struct exynos_dp_device *dp,
- unsigned int device_addr,
- unsigned int reg_addr,
- unsigned int *data)
-{
- u32 reg;
- int i;
- int retval;
-
- for (i = 0; i < 3; i++) {
- /* Clear AUX CH data buffer */
- reg = BUF_CLR;
- writel(reg, dp->reg_base + EXYNOS_DP_BUFFER_DATA_CTL);
-
- /* Select EDID device */
- retval = exynos_dp_select_i2c_device(dp, device_addr, reg_addr);
- if (retval != 0)
- continue;
-
- /*
- * Set I2C transaction and read data
- * If bit 3 is 1, DisplayPort transaction.
- * If Bit 3 is 0, I2C transaction.
- */
- reg = AUX_TX_COMM_I2C_TRANSACTION |
- AUX_TX_COMM_READ;
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_CH_CTL_1);
-
- /* Start AUX transaction */
- retval = exynos_dp_start_aux_transaction(dp);
- if (retval == 0)
- break;
- else
- dev_dbg(dp->dev, "%s: Aux Transaction fail!\n",
- __func__);
- }
-
- /* Read data */
- if (retval == 0)
- *data = readl(dp->reg_base + EXYNOS_DP_BUF_DATA_0);
-
- return retval;
-}
-
-int exynos_dp_read_bytes_from_i2c(struct exynos_dp_device *dp,
- unsigned int device_addr,
- unsigned int reg_addr,
- unsigned int count,
- unsigned char edid[])
-{
- u32 reg;
- unsigned int i, j;
- unsigned int cur_data_idx;
- unsigned int defer = 0;
- int retval = 0;
-
- for (i = 0; i < count; i += 16) {
- for (j = 0; j < 3; j++) {
- /* Clear AUX CH data buffer */
- reg = BUF_CLR;
- writel(reg, dp->reg_base + EXYNOS_DP_BUFFER_DATA_CTL);
-
- /* Set normal AUX CH command */
- reg = readl(dp->reg_base + EXYNOS_DP_AUX_CH_CTL_2);
- reg &= ~ADDR_ONLY;
- writel(reg, dp->reg_base + EXYNOS_DP_AUX_CH_CTL_2);
-
- /*
- * If Rx sends defer, Tx sends only reads
- * request without sending address
- */
- if (!defer)
- retval = exynos_dp_select_i2c_device(dp,
- device_addr, reg_addr + i);
- else
- defer = 0;
-
- if (retval == 0) {
- /*
- * Set I2C transaction and write data
- * If bit 3 is 1, DisplayPort transaction.
- * If Bit 3 is 0, I2C transaction.
- */
- reg = AUX_LENGTH(16) |
- AUX_TX_COMM_I2C_TRANSACTION |
- AUX_TX_COMM_READ;
- writel(reg, dp->reg_base +
- EXYNOS_DP_AUX_CH_CTL_1);
-
- /* Start AUX transaction */
- retval = exynos_dp_start_aux_transaction(dp);
- if (retval == 0)
- break;
- else
- dev_dbg(dp->dev,
- "%s: Aux Transaction fail!\n",
- __func__);
- }
- /* Check if Rx sends defer */
- reg = readl(dp->reg_base + EXYNOS_DP_AUX_RX_COMM);
- if (reg == AUX_RX_COMM_AUX_DEFER ||
- reg == AUX_RX_COMM_I2C_DEFER) {
- dev_err(dp->dev, "Defer: %d\n\n", reg);
- defer = 1;
- }
- }
-
- for (cur_data_idx = 0; cur_data_idx < 16; cur_data_idx++) {
- reg = readl(dp->reg_base + EXYNOS_DP_BUF_DATA_0
- + 4 * cur_data_idx);
- edid[i + cur_data_idx] = (unsigned char)reg;
- }
- }
-
- return retval;
-}
-
-void exynos_dp_set_link_bandwidth(struct exynos_dp_device *dp, u32 bwtype)
-{
- u32 reg;
-
- reg = bwtype;
- if ((bwtype == LINK_RATE_2_70GBPS) || (bwtype == LINK_RATE_1_62GBPS))
- writel(reg, dp->reg_base + EXYNOS_DP_LINK_BW_SET);
-}
-
-void exynos_dp_get_link_bandwidth(struct exynos_dp_device *dp, u32 *bwtype)
-{
- u32 reg;
-
- reg = readl(dp->reg_base + EXYNOS_DP_LINK_BW_SET);
- *bwtype = reg;
-}
-
-void exynos_dp_set_lane_count(struct exynos_dp_device *dp, u32 count)
-{
- u32 reg;
-
- reg = count;
- writel(reg, dp->reg_base + EXYNOS_DP_LANE_COUNT_SET);
-}
-
-void exynos_dp_get_lane_count(struct exynos_dp_device *dp, u32 *count)
-{
- u32 reg;
-
- reg = readl(dp->reg_base + EXYNOS_DP_LANE_COUNT_SET);
- *count = reg;
-}
-
-void exynos_dp_enable_enhanced_mode(struct exynos_dp_device *dp, bool enable)
-{
- u32 reg;
-
- if (enable) {
- reg = readl(dp->reg_base + EXYNOS_DP_SYS_CTL_4);
- reg |= ENHANCED;
- writel(reg, dp->reg_base + EXYNOS_DP_SYS_CTL_4);
- } else {
- reg = readl(dp->reg_base + EXYNOS_DP_SYS_CTL_4);
- reg &= ~ENHANCED;
- writel(reg, dp->reg_base + EXYNOS_DP_SYS_CTL_4);
- }
-}
-
-void exynos_dp_set_training_pattern(struct exynos_dp_device *dp,
- enum pattern_set pattern)
-{
- u32 reg;
-
- switch (pattern) {
- case PRBS7:
- reg = SCRAMBLING_ENABLE | LINK_QUAL_PATTERN_SET_PRBS7;
- writel(reg, dp->reg_base + EXYNOS_DP_TRAINING_PTN_SET);
- break;
- case D10_2:
- reg = SCRAMBLING_ENABLE | LINK_QUAL_PATTERN_SET_D10_2;
- writel(reg, dp->reg_base + EXYNOS_DP_TRAINING_PTN_SET);
- break;
- case TRAINING_PTN1:
- reg = SCRAMBLING_DISABLE | SW_TRAINING_PATTERN_SET_PTN1;
- writel(reg, dp->reg_base + EXYNOS_DP_TRAINING_PTN_SET);
- break;
- case TRAINING_PTN2:
- reg = SCRAMBLING_DISABLE | SW_TRAINING_PATTERN_SET_PTN2;
- writel(reg, dp->reg_base + EXYNOS_DP_TRAINING_PTN_SET);
- break;
- case DP_NONE:
- reg = SCRAMBLING_ENABLE |
- LINK_QUAL_PATTERN_SET_DISABLE |
- SW_TRAINING_PATTERN_SET_NORMAL;
- writel(reg, dp->reg_base + EXYNOS_DP_TRAINING_PTN_SET);
- break;
- default:
- break;
- }
-}
-
-void exynos_dp_set_lane0_pre_emphasis(struct exynos_dp_device *dp, u32 level)
-{
- u32 reg;
-
- reg = readl(dp->reg_base + EXYNOS_DP_LN0_LINK_TRAINING_CTL);
- reg &= ~PRE_EMPHASIS_SET_MASK;
- reg |= level << PRE_EMPHASIS_SET_SHIFT;
- writel(reg, dp->reg_base + EXYNOS_DP_LN0_LINK_TRAINING_CTL);
-}
-
-void exynos_dp_set_lane1_pre_emphasis(struct exynos_dp_device *dp, u32 level)
-{
- u32 reg;
-
- reg = readl(dp->reg_base + EXYNOS_DP_LN1_LINK_TRAINING_CTL);
- reg &= ~PRE_EMPHASIS_SET_MASK;
- reg |= level << PRE_EMPHASIS_SET_SHIFT;
- writel(reg, dp->reg_base + EXYNOS_DP_LN1_LINK_TRAINING_CTL);
-}
-
-void exynos_dp_set_lane2_pre_emphasis(struct exynos_dp_device *dp, u32 level)
-{
- u32 reg;
-
- reg = readl(dp->reg_base + EXYNOS_DP_LN2_LINK_TRAINING_CTL);
- reg &= ~PRE_EMPHASIS_SET_MASK;
- reg |= level << PRE_EMPHASIS_SET_SHIFT;
- writel(reg, dp->reg_base + EXYNOS_DP_LN2_LINK_TRAINING_CTL);
-}
-
-void exynos_dp_set_lane3_pre_emphasis(struct exynos_dp_device *dp, u32 level)
-{
- u32 reg;
-
- reg = readl(dp->reg_base + EXYNOS_DP_LN3_LINK_TRAINING_CTL);
- reg &= ~PRE_EMPHASIS_SET_MASK;
- reg |= level << PRE_EMPHASIS_SET_SHIFT;
- writel(reg, dp->reg_base + EXYNOS_DP_LN3_LINK_TRAINING_CTL);
-}
-
-void exynos_dp_set_lane0_link_training(struct exynos_dp_device *dp,
- u32 training_lane)
-{
- u32 reg;
-
- reg = training_lane;
- writel(reg, dp->reg_base + EXYNOS_DP_LN0_LINK_TRAINING_CTL);
-}
-
-void exynos_dp_set_lane1_link_training(struct exynos_dp_device *dp,
- u32 training_lane)
-{
- u32 reg;
-
- reg = training_lane;
- writel(reg, dp->reg_base + EXYNOS_DP_LN1_LINK_TRAINING_CTL);
-}
-
-void exynos_dp_set_lane2_link_training(struct exynos_dp_device *dp,
- u32 training_lane)
-{
- u32 reg;
-
- reg = training_lane;
- writel(reg, dp->reg_base + EXYNOS_DP_LN2_LINK_TRAINING_CTL);
-}
-
-void exynos_dp_set_lane3_link_training(struct exynos_dp_device *dp,
- u32 training_lane)
-{
- u32 reg;
-
- reg = training_lane;
- writel(reg, dp->reg_base + EXYNOS_DP_LN3_LINK_TRAINING_CTL);
-}
-
-u32 exynos_dp_get_lane0_link_training(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- reg = readl(dp->reg_base + EXYNOS_DP_LN0_LINK_TRAINING_CTL);
- return reg;
-}
-
-u32 exynos_dp_get_lane1_link_training(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- reg = readl(dp->reg_base + EXYNOS_DP_LN1_LINK_TRAINING_CTL);
- return reg;
-}
-
-u32 exynos_dp_get_lane2_link_training(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- reg = readl(dp->reg_base + EXYNOS_DP_LN2_LINK_TRAINING_CTL);
- return reg;
-}
-
-u32 exynos_dp_get_lane3_link_training(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- reg = readl(dp->reg_base + EXYNOS_DP_LN3_LINK_TRAINING_CTL);
- return reg;
-}
-
-void exynos_dp_reset_macro(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- reg = readl(dp->reg_base + EXYNOS_DP_PHY_TEST);
- reg |= MACRO_RST;
- writel(reg, dp->reg_base + EXYNOS_DP_PHY_TEST);
-
- /* 10 us is the minimum reset time. */
- usleep_range(10, 20);
-
- reg &= ~MACRO_RST;
- writel(reg, dp->reg_base + EXYNOS_DP_PHY_TEST);
-}
-
-void exynos_dp_init_video(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- reg = VSYNC_DET | VID_FORMAT_CHG | VID_CLK_CHG;
- writel(reg, dp->reg_base + EXYNOS_DP_COMMON_INT_STA_1);
-
- reg = 0x0;
- writel(reg, dp->reg_base + EXYNOS_DP_SYS_CTL_1);
-
- reg = CHA_CRI(4) | CHA_CTRL;
- writel(reg, dp->reg_base + EXYNOS_DP_SYS_CTL_2);
-
- reg = 0x0;
- writel(reg, dp->reg_base + EXYNOS_DP_SYS_CTL_3);
-
- reg = VID_HRES_TH(2) | VID_VRES_TH(0);
- writel(reg, dp->reg_base + EXYNOS_DP_VIDEO_CTL_8);
-}
-
-void exynos_dp_set_video_color_format(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- /* Configure the input color depth, color space, dynamic range */
- reg = (dp->video_info->dynamic_range << IN_D_RANGE_SHIFT) |
- (dp->video_info->color_depth << IN_BPC_SHIFT) |
- (dp->video_info->color_space << IN_COLOR_F_SHIFT);
- writel(reg, dp->reg_base + EXYNOS_DP_VIDEO_CTL_2);
-
- /* Set Input Color YCbCr Coefficients to ITU601 or ITU709 */
- reg = readl(dp->reg_base + EXYNOS_DP_VIDEO_CTL_3);
- reg &= ~IN_YC_COEFFI_MASK;
- if (dp->video_info->ycbcr_coeff)
- reg |= IN_YC_COEFFI_ITU709;
- else
- reg |= IN_YC_COEFFI_ITU601;
- writel(reg, dp->reg_base + EXYNOS_DP_VIDEO_CTL_3);
-}
-
-int exynos_dp_is_slave_video_stream_clock_on(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- reg = readl(dp->reg_base + EXYNOS_DP_SYS_CTL_1);
- writel(reg, dp->reg_base + EXYNOS_DP_SYS_CTL_1);
-
- reg = readl(dp->reg_base + EXYNOS_DP_SYS_CTL_1);
-
- if (!(reg & DET_STA)) {
- dev_dbg(dp->dev, "Input stream clock not detected.\n");
- return -EINVAL;
- }
-
- reg = readl(dp->reg_base + EXYNOS_DP_SYS_CTL_2);
- writel(reg, dp->reg_base + EXYNOS_DP_SYS_CTL_2);
-
- reg = readl(dp->reg_base + EXYNOS_DP_SYS_CTL_2);
- dev_dbg(dp->dev, "wait SYS_CTL_2.\n");
-
- if (reg & CHA_STA) {
- dev_dbg(dp->dev, "Input stream clk is changing\n");
- return -EINVAL;
- }
-
- return 0;
-}
-
-void exynos_dp_set_video_cr_mn(struct exynos_dp_device *dp,
- enum clock_recovery_m_value_type type,
- u32 m_value,
- u32 n_value)
-{
- u32 reg;
-
- if (type == REGISTER_M) {
- reg = readl(dp->reg_base + EXYNOS_DP_SYS_CTL_4);
- reg |= FIX_M_VID;
- writel(reg, dp->reg_base + EXYNOS_DP_SYS_CTL_4);
- reg = m_value & 0xff;
- writel(reg, dp->reg_base + EXYNOS_DP_M_VID_0);
- reg = (m_value >> 8) & 0xff;
- writel(reg, dp->reg_base + EXYNOS_DP_M_VID_1);
- reg = (m_value >> 16) & 0xff;
- writel(reg, dp->reg_base + EXYNOS_DP_M_VID_2);
-
- reg = n_value & 0xff;
- writel(reg, dp->reg_base + EXYNOS_DP_N_VID_0);
- reg = (n_value >> 8) & 0xff;
- writel(reg, dp->reg_base + EXYNOS_DP_N_VID_1);
- reg = (n_value >> 16) & 0xff;
- writel(reg, dp->reg_base + EXYNOS_DP_N_VID_2);
- } else {
- reg = readl(dp->reg_base + EXYNOS_DP_SYS_CTL_4);
- reg &= ~FIX_M_VID;
- writel(reg, dp->reg_base + EXYNOS_DP_SYS_CTL_4);
-
- writel(0x00, dp->reg_base + EXYNOS_DP_N_VID_0);
- writel(0x80, dp->reg_base + EXYNOS_DP_N_VID_1);
- writel(0x00, dp->reg_base + EXYNOS_DP_N_VID_2);
- }
-}
-
-void exynos_dp_set_video_timing_mode(struct exynos_dp_device *dp, u32 type)
-{
- u32 reg;
-
- if (type == VIDEO_TIMING_FROM_CAPTURE) {
- reg = readl(dp->reg_base + EXYNOS_DP_VIDEO_CTL_10);
- reg &= ~FORMAT_SEL;
- writel(reg, dp->reg_base + EXYNOS_DP_VIDEO_CTL_10);
- } else {
- reg = readl(dp->reg_base + EXYNOS_DP_VIDEO_CTL_10);
- reg |= FORMAT_SEL;
- writel(reg, dp->reg_base + EXYNOS_DP_VIDEO_CTL_10);
- }
-}
-
-void exynos_dp_enable_video_master(struct exynos_dp_device *dp, bool enable)
-{
- u32 reg;
-
- if (enable) {
- reg = readl(dp->reg_base + EXYNOS_DP_SOC_GENERAL_CTL);
- reg &= ~VIDEO_MODE_MASK;
- reg |= VIDEO_MASTER_MODE_EN | VIDEO_MODE_MASTER_MODE;
- writel(reg, dp->reg_base + EXYNOS_DP_SOC_GENERAL_CTL);
- } else {
- reg = readl(dp->reg_base + EXYNOS_DP_SOC_GENERAL_CTL);
- reg &= ~VIDEO_MODE_MASK;
- reg |= VIDEO_MODE_SLAVE_MODE;
- writel(reg, dp->reg_base + EXYNOS_DP_SOC_GENERAL_CTL);
- }
-}
-
-void exynos_dp_start_video(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- reg = readl(dp->reg_base + EXYNOS_DP_VIDEO_CTL_1);
- reg |= VIDEO_EN;
- writel(reg, dp->reg_base + EXYNOS_DP_VIDEO_CTL_1);
-}
-
-int exynos_dp_is_video_stream_on(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- reg = readl(dp->reg_base + EXYNOS_DP_SYS_CTL_3);
- writel(reg, dp->reg_base + EXYNOS_DP_SYS_CTL_3);
-
- reg = readl(dp->reg_base + EXYNOS_DP_SYS_CTL_3);
- if (!(reg & STRM_VALID)) {
- dev_dbg(dp->dev, "Input video stream is not detected.\n");
- return -EINVAL;
- }
-
- return 0;
-}
-
-void exynos_dp_config_video_slave_mode(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- reg = readl(dp->reg_base + EXYNOS_DP_FUNC_EN_1);
- reg &= ~(MASTER_VID_FUNC_EN_N|SLAVE_VID_FUNC_EN_N);
- reg |= MASTER_VID_FUNC_EN_N;
- writel(reg, dp->reg_base + EXYNOS_DP_FUNC_EN_1);
-
- reg = readl(dp->reg_base + EXYNOS_DP_VIDEO_CTL_10);
- reg &= ~INTERACE_SCAN_CFG;
- reg |= (dp->video_info->interlaced << 2);
- writel(reg, dp->reg_base + EXYNOS_DP_VIDEO_CTL_10);
-
- reg = readl(dp->reg_base + EXYNOS_DP_VIDEO_CTL_10);
- reg &= ~VSYNC_POLARITY_CFG;
- reg |= (dp->video_info->v_sync_polarity << 1);
- writel(reg, dp->reg_base + EXYNOS_DP_VIDEO_CTL_10);
-
- reg = readl(dp->reg_base + EXYNOS_DP_VIDEO_CTL_10);
- reg &= ~HSYNC_POLARITY_CFG;
- reg |= (dp->video_info->h_sync_polarity << 0);
- writel(reg, dp->reg_base + EXYNOS_DP_VIDEO_CTL_10);
-
- reg = AUDIO_MODE_SPDIF_MODE | VIDEO_MODE_SLAVE_MODE;
- writel(reg, dp->reg_base + EXYNOS_DP_SOC_GENERAL_CTL);
-}
-
-void exynos_dp_enable_scrambling(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- reg = readl(dp->reg_base + EXYNOS_DP_TRAINING_PTN_SET);
- reg &= ~SCRAMBLING_DISABLE;
- writel(reg, dp->reg_base + EXYNOS_DP_TRAINING_PTN_SET);
-}
-
-void exynos_dp_disable_scrambling(struct exynos_dp_device *dp)
-{
- u32 reg;
-
- reg = readl(dp->reg_base + EXYNOS_DP_TRAINING_PTN_SET);
- reg |= SCRAMBLING_DISABLE;
- writel(reg, dp->reg_base + EXYNOS_DP_TRAINING_PTN_SET);
-}
diff --git a/drivers/gpu/drm/exynos/exynos_dp_reg.h b/drivers/gpu/drm/exynos/exynos_dp_reg.h
deleted file mode 100644
index 2e9bd0e..0000000
--- a/drivers/gpu/drm/exynos/exynos_dp_reg.h
+++ /dev/null
@@ -1,366 +0,0 @@
-/*
- * Register definition file for Samsung DP driver
- *
- * Copyright (C) 2012 Samsung Electronics Co., Ltd.
- * Author: Jingoo Han <jg1.han@samsung.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-
-#ifndef _EXYNOS_DP_REG_H
-#define _EXYNOS_DP_REG_H
-
-#define EXYNOS_DP_TX_SW_RESET 0x14
-#define EXYNOS_DP_FUNC_EN_1 0x18
-#define EXYNOS_DP_FUNC_EN_2 0x1C
-#define EXYNOS_DP_VIDEO_CTL_1 0x20
-#define EXYNOS_DP_VIDEO_CTL_2 0x24
-#define EXYNOS_DP_VIDEO_CTL_3 0x28
-
-#define EXYNOS_DP_VIDEO_CTL_8 0x3C
-#define EXYNOS_DP_VIDEO_CTL_10 0x44
-
-#define EXYNOS_DP_LANE_MAP 0x35C
-
-#define EXYNOS_DP_ANALOG_CTL_1 0x370
-#define EXYNOS_DP_ANALOG_CTL_2 0x374
-#define EXYNOS_DP_ANALOG_CTL_3 0x378
-#define EXYNOS_DP_PLL_FILTER_CTL_1 0x37C
-#define EXYNOS_DP_TX_AMP_TUNING_CTL 0x380
-
-#define EXYNOS_DP_AUX_HW_RETRY_CTL 0x390
-
-#define EXYNOS_DP_COMMON_INT_STA_1 0x3C4
-#define EXYNOS_DP_COMMON_INT_STA_2 0x3C8
-#define EXYNOS_DP_COMMON_INT_STA_3 0x3CC
-#define EXYNOS_DP_COMMON_INT_STA_4 0x3D0
-#define EXYNOS_DP_INT_STA 0x3DC
-#define EXYNOS_DP_COMMON_INT_MASK_1 0x3E0
-#define EXYNOS_DP_COMMON_INT_MASK_2 0x3E4
-#define EXYNOS_DP_COMMON_INT_MASK_3 0x3E8
-#define EXYNOS_DP_COMMON_INT_MASK_4 0x3EC
-#define EXYNOS_DP_INT_STA_MASK 0x3F8
-#define EXYNOS_DP_INT_CTL 0x3FC
-
-#define EXYNOS_DP_SYS_CTL_1 0x600
-#define EXYNOS_DP_SYS_CTL_2 0x604
-#define EXYNOS_DP_SYS_CTL_3 0x608
-#define EXYNOS_DP_SYS_CTL_4 0x60C
-
-#define EXYNOS_DP_PKT_SEND_CTL 0x640
-#define EXYNOS_DP_HDCP_CTL 0x648
-
-#define EXYNOS_DP_LINK_BW_SET 0x680
-#define EXYNOS_DP_LANE_COUNT_SET 0x684
-#define EXYNOS_DP_TRAINING_PTN_SET 0x688
-#define EXYNOS_DP_LN0_LINK_TRAINING_CTL 0x68C
-#define EXYNOS_DP_LN1_LINK_TRAINING_CTL 0x690
-#define EXYNOS_DP_LN2_LINK_TRAINING_CTL 0x694
-#define EXYNOS_DP_LN3_LINK_TRAINING_CTL 0x698
-
-#define EXYNOS_DP_DEBUG_CTL 0x6C0
-#define EXYNOS_DP_HPD_DEGLITCH_L 0x6C4
-#define EXYNOS_DP_HPD_DEGLITCH_H 0x6C8
-#define EXYNOS_DP_LINK_DEBUG_CTL 0x6E0
-
-#define EXYNOS_DP_M_VID_0 0x700
-#define EXYNOS_DP_M_VID_1 0x704
-#define EXYNOS_DP_M_VID_2 0x708
-#define EXYNOS_DP_N_VID_0 0x70C
-#define EXYNOS_DP_N_VID_1 0x710
-#define EXYNOS_DP_N_VID_2 0x714
-
-#define EXYNOS_DP_PLL_CTL 0x71C
-#define EXYNOS_DP_PHY_PD 0x720
-#define EXYNOS_DP_PHY_TEST 0x724
-
-#define EXYNOS_DP_VIDEO_FIFO_THRD 0x730
-#define EXYNOS_DP_AUDIO_MARGIN 0x73C
-
-#define EXYNOS_DP_M_VID_GEN_FILTER_TH 0x764
-#define EXYNOS_DP_M_AUD_GEN_FILTER_TH 0x778
-#define EXYNOS_DP_AUX_CH_STA 0x780
-#define EXYNOS_DP_AUX_CH_DEFER_CTL 0x788
-#define EXYNOS_DP_AUX_RX_COMM 0x78C
-#define EXYNOS_DP_BUFFER_DATA_CTL 0x790
-#define EXYNOS_DP_AUX_CH_CTL_1 0x794
-#define EXYNOS_DP_AUX_ADDR_7_0 0x798
-#define EXYNOS_DP_AUX_ADDR_15_8 0x79C
-#define EXYNOS_DP_AUX_ADDR_19_16 0x7A0
-#define EXYNOS_DP_AUX_CH_CTL_2 0x7A4
-
-#define EXYNOS_DP_BUF_DATA_0 0x7C0
-
-#define EXYNOS_DP_SOC_GENERAL_CTL 0x800
-
-/* EXYNOS_DP_TX_SW_RESET */
-#define RESET_DP_TX (0x1 << 0)
-
-/* EXYNOS_DP_FUNC_EN_1 */
-#define MASTER_VID_FUNC_EN_N (0x1 << 7)
-#define SLAVE_VID_FUNC_EN_N (0x1 << 5)
-#define AUD_FIFO_FUNC_EN_N (0x1 << 4)
-#define AUD_FUNC_EN_N (0x1 << 3)
-#define HDCP_FUNC_EN_N (0x1 << 2)
-#define CRC_FUNC_EN_N (0x1 << 1)
-#define SW_FUNC_EN_N (0x1 << 0)
-
-/* EXYNOS_DP_FUNC_EN_2 */
-#define SSC_FUNC_EN_N (0x1 << 7)
-#define AUX_FUNC_EN_N (0x1 << 2)
-#define SERDES_FIFO_FUNC_EN_N (0x1 << 1)
-#define LS_CLK_DOMAIN_FUNC_EN_N (0x1 << 0)
-
-/* EXYNOS_DP_VIDEO_CTL_1 */
-#define VIDEO_EN (0x1 << 7)
-#define HDCP_VIDEO_MUTE (0x1 << 6)
-
-/* EXYNOS_DP_VIDEO_CTL_1 */
-#define IN_D_RANGE_MASK (0x1 << 7)
-#define IN_D_RANGE_SHIFT (7)
-#define IN_D_RANGE_CEA (0x1 << 7)
-#define IN_D_RANGE_VESA (0x0 << 7)
-#define IN_BPC_MASK (0x7 << 4)
-#define IN_BPC_SHIFT (4)
-#define IN_BPC_12_BITS (0x3 << 4)
-#define IN_BPC_10_BITS (0x2 << 4)
-#define IN_BPC_8_BITS (0x1 << 4)
-#define IN_BPC_6_BITS (0x0 << 4)
-#define IN_COLOR_F_MASK (0x3 << 0)
-#define IN_COLOR_F_SHIFT (0)
-#define IN_COLOR_F_YCBCR444 (0x2 << 0)
-#define IN_COLOR_F_YCBCR422 (0x1 << 0)
-#define IN_COLOR_F_RGB (0x0 << 0)
-
-/* EXYNOS_DP_VIDEO_CTL_3 */
-#define IN_YC_COEFFI_MASK (0x1 << 7)
-#define IN_YC_COEFFI_SHIFT (7)
-#define IN_YC_COEFFI_ITU709 (0x1 << 7)
-#define IN_YC_COEFFI_ITU601 (0x0 << 7)
-#define VID_CHK_UPDATE_TYPE_MASK (0x1 << 4)
-#define VID_CHK_UPDATE_TYPE_SHIFT (4)
-#define VID_CHK_UPDATE_TYPE_1 (0x1 << 4)
-#define VID_CHK_UPDATE_TYPE_0 (0x0 << 4)
-
-/* EXYNOS_DP_VIDEO_CTL_8 */
-#define VID_HRES_TH(x) (((x) & 0xf) << 4)
-#define VID_VRES_TH(x) (((x) & 0xf) << 0)
-
-/* EXYNOS_DP_VIDEO_CTL_10 */
-#define FORMAT_SEL (0x1 << 4)
-#define INTERACE_SCAN_CFG (0x1 << 2)
-#define VSYNC_POLARITY_CFG (0x1 << 1)
-#define HSYNC_POLARITY_CFG (0x1 << 0)
-
-/* EXYNOS_DP_LANE_MAP */
-#define LANE3_MAP_LOGIC_LANE_0 (0x0 << 6)
-#define LANE3_MAP_LOGIC_LANE_1 (0x1 << 6)
-#define LANE3_MAP_LOGIC_LANE_2 (0x2 << 6)
-#define LANE3_MAP_LOGIC_LANE_3 (0x3 << 6)
-#define LANE2_MAP_LOGIC_LANE_0 (0x0 << 4)
-#define LANE2_MAP_LOGIC_LANE_1 (0x1 << 4)
-#define LANE2_MAP_LOGIC_LANE_2 (0x2 << 4)
-#define LANE2_MAP_LOGIC_LANE_3 (0x3 << 4)
-#define LANE1_MAP_LOGIC_LANE_0 (0x0 << 2)
-#define LANE1_MAP_LOGIC_LANE_1 (0x1 << 2)
-#define LANE1_MAP_LOGIC_LANE_2 (0x2 << 2)
-#define LANE1_MAP_LOGIC_LANE_3 (0x3 << 2)
-#define LANE0_MAP_LOGIC_LANE_0 (0x0 << 0)
-#define LANE0_MAP_LOGIC_LANE_1 (0x1 << 0)
-#define LANE0_MAP_LOGIC_LANE_2 (0x2 << 0)
-#define LANE0_MAP_LOGIC_LANE_3 (0x3 << 0)
-
-/* EXYNOS_DP_ANALOG_CTL_1 */
-#define TX_TERMINAL_CTRL_50_OHM (0x1 << 4)
-
-/* EXYNOS_DP_ANALOG_CTL_2 */
-#define SEL_24M (0x1 << 3)
-#define TX_DVDD_BIT_1_0625V (0x4 << 0)
-
-/* EXYNOS_DP_ANALOG_CTL_3 */
-#define DRIVE_DVDD_BIT_1_0625V (0x4 << 5)
-#define VCO_BIT_600_MICRO (0x5 << 0)
-
-/* EXYNOS_DP_PLL_FILTER_CTL_1 */
-#define PD_RING_OSC (0x1 << 6)
-#define AUX_TERMINAL_CTRL_50_OHM (0x2 << 4)
-#define TX_CUR1_2X (0x1 << 2)
-#define TX_CUR_16_MA (0x3 << 0)
-
-/* EXYNOS_DP_TX_AMP_TUNING_CTL */
-#define CH3_AMP_400_MV (0x0 << 24)
-#define CH2_AMP_400_MV (0x0 << 16)
-#define CH1_AMP_400_MV (0x0 << 8)
-#define CH0_AMP_400_MV (0x0 << 0)
-
-/* EXYNOS_DP_AUX_HW_RETRY_CTL */
-#define AUX_BIT_PERIOD_EXPECTED_DELAY(x) (((x) & 0x7) << 8)
-#define AUX_HW_RETRY_INTERVAL_MASK (0x3 << 3)
-#define AUX_HW_RETRY_INTERVAL_600_MICROSECONDS (0x0 << 3)
-#define AUX_HW_RETRY_INTERVAL_800_MICROSECONDS (0x1 << 3)
-#define AUX_HW_RETRY_INTERVAL_1000_MICROSECONDS (0x2 << 3)
-#define AUX_HW_RETRY_INTERVAL_1800_MICROSECONDS (0x3 << 3)
-#define AUX_HW_RETRY_COUNT_SEL(x) (((x) & 0x7) << 0)
-
-/* EXYNOS_DP_COMMON_INT_STA_1 */
-#define VSYNC_DET (0x1 << 7)
-#define PLL_LOCK_CHG (0x1 << 6)
-#define SPDIF_ERR (0x1 << 5)
-#define SPDIF_UNSTBL (0x1 << 4)
-#define VID_FORMAT_CHG (0x1 << 3)
-#define AUD_CLK_CHG (0x1 << 2)
-#define VID_CLK_CHG (0x1 << 1)
-#define SW_INT (0x1 << 0)
-
-/* EXYNOS_DP_COMMON_INT_STA_2 */
-#define ENC_EN_CHG (0x1 << 6)
-#define HW_BKSV_RDY (0x1 << 3)
-#define HW_SHA_DONE (0x1 << 2)
-#define HW_AUTH_STATE_CHG (0x1 << 1)
-#define HW_AUTH_DONE (0x1 << 0)
-
-/* EXYNOS_DP_COMMON_INT_STA_3 */
-#define AFIFO_UNDER (0x1 << 7)
-#define AFIFO_OVER (0x1 << 6)
-#define R0_CHK_FLAG (0x1 << 5)
-
-/* EXYNOS_DP_COMMON_INT_STA_4 */
-#define PSR_ACTIVE (0x1 << 7)
-#define PSR_INACTIVE (0x1 << 6)
-#define SPDIF_BI_PHASE_ERR (0x1 << 5)
-#define HOTPLUG_CHG (0x1 << 2)
-#define HPD_LOST (0x1 << 1)
-#define PLUG (0x1 << 0)
-
-/* EXYNOS_DP_INT_STA */
-#define INT_HPD (0x1 << 6)
-#define HW_TRAINING_FINISH (0x1 << 5)
-#define RPLY_RECEIV (0x1 << 1)
-#define AUX_ERR (0x1 << 0)
-
-/* EXYNOS_DP_INT_CTL */
-#define SOFT_INT_CTRL (0x1 << 2)
-#define INT_POL1 (0x1 << 1)
-#define INT_POL0 (0x1 << 0)
-
-/* EXYNOS_DP_SYS_CTL_1 */
-#define DET_STA (0x1 << 2)
-#define FORCE_DET (0x1 << 1)
-#define DET_CTRL (0x1 << 0)
-
-/* EXYNOS_DP_SYS_CTL_2 */
-#define CHA_CRI(x) (((x) & 0xf) << 4)
-#define CHA_STA (0x1 << 2)
-#define FORCE_CHA (0x1 << 1)
-#define CHA_CTRL (0x1 << 0)
-
-/* EXYNOS_DP_SYS_CTL_3 */
-#define HPD_STATUS (0x1 << 6)
-#define F_HPD (0x1 << 5)
-#define HPD_CTRL (0x1 << 4)
-#define HDCP_RDY (0x1 << 3)
-#define STRM_VALID (0x1 << 2)
-#define F_VALID (0x1 << 1)
-#define VALID_CTRL (0x1 << 0)
-
-/* EXYNOS_DP_SYS_CTL_4 */
-#define FIX_M_AUD (0x1 << 4)
-#define ENHANCED (0x1 << 3)
-#define FIX_M_VID (0x1 << 2)
-#define M_VID_UPDATE_CTRL (0x3 << 0)
-
-/* EXYNOS_DP_TRAINING_PTN_SET */
-#define SCRAMBLER_TYPE (0x1 << 9)
-#define HW_LINK_TRAINING_PATTERN (0x1 << 8)
-#define SCRAMBLING_DISABLE (0x1 << 5)
-#define SCRAMBLING_ENABLE (0x0 << 5)
-#define LINK_QUAL_PATTERN_SET_MASK (0x3 << 2)
-#define LINK_QUAL_PATTERN_SET_PRBS7 (0x3 << 2)
-#define LINK_QUAL_PATTERN_SET_D10_2 (0x1 << 2)
-#define LINK_QUAL_PATTERN_SET_DISABLE (0x0 << 2)
-#define SW_TRAINING_PATTERN_SET_MASK (0x3 << 0)
-#define SW_TRAINING_PATTERN_SET_PTN2 (0x2 << 0)
-#define SW_TRAINING_PATTERN_SET_PTN1 (0x1 << 0)
-#define SW_TRAINING_PATTERN_SET_NORMAL (0x0 << 0)
-
-/* EXYNOS_DP_LN0_LINK_TRAINING_CTL */
-#define PRE_EMPHASIS_SET_MASK (0x3 << 3)
-#define PRE_EMPHASIS_SET_SHIFT (3)
-
-/* EXYNOS_DP_DEBUG_CTL */
-#define PLL_LOCK (0x1 << 4)
-#define F_PLL_LOCK (0x1 << 3)
-#define PLL_LOCK_CTRL (0x1 << 2)
-#define PN_INV (0x1 << 0)
-
-/* EXYNOS_DP_PLL_CTL */
-#define DP_PLL_PD (0x1 << 7)
-#define DP_PLL_RESET (0x1 << 6)
-#define DP_PLL_LOOP_BIT_DEFAULT (0x1 << 4)
-#define DP_PLL_REF_BIT_1_1250V (0x5 << 0)
-#define DP_PLL_REF_BIT_1_2500V (0x7 << 0)
-
-/* EXYNOS_DP_PHY_PD */
-#define DP_PHY_PD (0x1 << 5)
-#define AUX_PD (0x1 << 4)
-#define CH3_PD (0x1 << 3)
-#define CH2_PD (0x1 << 2)
-#define CH1_PD (0x1 << 1)
-#define CH0_PD (0x1 << 0)
-
-/* EXYNOS_DP_PHY_TEST */
-#define MACRO_RST (0x1 << 5)
-#define CH1_TEST (0x1 << 1)
-#define CH0_TEST (0x1 << 0)
-
-/* EXYNOS_DP_AUX_CH_STA */
-#define AUX_BUSY (0x1 << 4)
-#define AUX_STATUS_MASK (0xf << 0)
-
-/* EXYNOS_DP_AUX_CH_DEFER_CTL */
-#define DEFER_CTRL_EN (0x1 << 7)
-#define DEFER_COUNT(x) (((x) & 0x7f) << 0)
-
-/* EXYNOS_DP_AUX_RX_COMM */
-#define AUX_RX_COMM_I2C_DEFER (0x2 << 2)
-#define AUX_RX_COMM_AUX_DEFER (0x2 << 0)
-
-/* EXYNOS_DP_BUFFER_DATA_CTL */
-#define BUF_CLR (0x1 << 7)
-#define BUF_DATA_COUNT(x) (((x) & 0x1f) << 0)
-
-/* EXYNOS_DP_AUX_CH_CTL_1 */
-#define AUX_LENGTH(x) (((x - 1) & 0xf) << 4)
-#define AUX_TX_COMM_MASK (0xf << 0)
-#define AUX_TX_COMM_DP_TRANSACTION (0x1 << 3)
-#define AUX_TX_COMM_I2C_TRANSACTION (0x0 << 3)
-#define AUX_TX_COMM_MOT (0x1 << 2)
-#define AUX_TX_COMM_WRITE (0x0 << 0)
-#define AUX_TX_COMM_READ (0x1 << 0)
-
-/* EXYNOS_DP_AUX_ADDR_7_0 */
-#define AUX_ADDR_7_0(x) (((x) >> 0) & 0xff)
-
-/* EXYNOS_DP_AUX_ADDR_15_8 */
-#define AUX_ADDR_15_8(x) (((x) >> 8) & 0xff)
-
-/* EXYNOS_DP_AUX_ADDR_19_16 */
-#define AUX_ADDR_19_16(x) (((x) >> 16) & 0x0f)
-
-/* EXYNOS_DP_AUX_CH_CTL_2 */
-#define ADDR_ONLY (0x1 << 1)
-#define AUX_EN (0x1 << 0)
-
-/* EXYNOS_DP_SOC_GENERAL_CTL */
-#define AUDIO_MODE_SPDIF_MODE (0x1 << 8)
-#define AUDIO_MODE_MASTER_MODE (0x0 << 8)
-#define MASTER_VIDEO_INTERLACE_EN (0x1 << 4)
-#define VIDEO_MASTER_CLK_SEL (0x1 << 2)
-#define VIDEO_MASTER_MODE_EN (0x1 << 1)
-#define VIDEO_MODE_MASK (0x1 << 0)
-#define VIDEO_MODE_SLAVE_MODE (0x1 << 0)
-#define VIDEO_MODE_MASTER_MODE (0x0 << 0)
-
-#endif /* _EXYNOS_DP_REG_H */
diff --git a/drivers/gpu/drm/exynos/exynos_drm_crtc.c b/drivers/gpu/drm/exynos/exynos_drm_crtc.c
index e36579c..785ffa6 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_crtc.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_crtc.c
@@ -157,9 +157,8 @@
int exynos_drm_crtc_enable_vblank(struct drm_device *dev, unsigned int pipe)
{
- struct exynos_drm_private *private = dev->dev_private;
- struct exynos_drm_crtc *exynos_crtc =
- to_exynos_crtc(private->crtc[pipe]);
+ struct exynos_drm_crtc *exynos_crtc = exynos_drm_crtc_from_pipe(dev,
+ pipe);
if (exynos_crtc->ops->enable_vblank)
return exynos_crtc->ops->enable_vblank(exynos_crtc);
@@ -169,9 +168,8 @@
void exynos_drm_crtc_disable_vblank(struct drm_device *dev, unsigned int pipe)
{
- struct exynos_drm_private *private = dev->dev_private;
- struct exynos_drm_crtc *exynos_crtc =
- to_exynos_crtc(private->crtc[pipe]);
+ struct exynos_drm_crtc *exynos_crtc = exynos_drm_crtc_from_pipe(dev,
+ pipe);
if (exynos_crtc->ops->disable_vblank)
exynos_crtc->ops->disable_vblank(exynos_crtc);
@@ -235,20 +233,15 @@
unsigned long flags;
spin_lock_irqsave(&crtc->dev->event_lock, flags);
+
e = exynos_crtc->event;
if (e && e->base.file_priv == file) {
exynos_crtc->event = NULL;
- /*
- * event will be destroyed by core part
- * so below line should be removed later with core changes
- */
- e->base.destroy(&e->base);
- /*
- * event_space will be increased by core part
- * so below line should be removed later with core changes.
- */
- file->event_space += sizeof(e->event);
atomic_dec(&exynos_crtc->pending_update);
}
+
spin_unlock_irqrestore(&crtc->dev->event_lock, flags);
+
+ if (e && e->base.file_priv == file)
+ drm_event_cancel_free(crtc->dev, &e->base);
}
diff --git a/drivers/gpu/drm/exynos/exynos_drm_dpi.c b/drivers/gpu/drm/exynos/exynos_drm_dpi.c
index 75e570f..5e38e74 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_dpi.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_dpi.c
@@ -15,6 +15,7 @@
#include <drm/drm_panel.h>
#include <drm/drm_atomic_helper.h>
+#include <linux/of_graph.h>
#include <linux/regulator/consumer.h>
#include <video/of_videomode.h>
@@ -164,67 +165,6 @@
.destroy = drm_encoder_cleanup,
};
-/* of_* functions will be removed after merge of of_graph patches */
-static struct device_node *
-of_get_child_by_name_reg(struct device_node *parent, const char *name, u32 reg)
-{
- struct device_node *np;
-
- for_each_child_of_node(parent, np) {
- u32 r;
-
- if (!np->name || of_node_cmp(np->name, name))
- continue;
-
- if (of_property_read_u32(np, "reg", &r) < 0)
- r = 0;
-
- if (reg == r)
- break;
- }
-
- return np;
-}
-
-static struct device_node *of_graph_get_port_by_reg(struct device_node *parent,
- u32 reg)
-{
- struct device_node *ports, *port;
-
- ports = of_get_child_by_name(parent, "ports");
- if (ports)
- parent = ports;
-
- port = of_get_child_by_name_reg(parent, "port", reg);
-
- of_node_put(ports);
-
- return port;
-}
-
-static struct device_node *
-of_graph_get_endpoint_by_reg(struct device_node *port, u32 reg)
-{
- return of_get_child_by_name_reg(port, "endpoint", reg);
-}
-
-static struct device_node *
-of_graph_get_remote_port_parent(const struct device_node *node)
-{
- struct device_node *np;
- unsigned int depth;
-
- np = of_parse_phandle(node, "remote-endpoint", 0);
-
- /* Walk 3 levels up only if there is 'ports' node. */
- for (depth = 3; depth && np; depth--) {
- np = of_get_next_parent(np);
- if (depth == 2 && of_node_cmp(np->name, "ports"))
- break;
- }
- return np;
-}
-
enum {
FIMD_PORT_IN0,
FIMD_PORT_IN1,
@@ -237,12 +177,7 @@
{
struct device_node *np, *ep;
- np = of_graph_get_port_by_reg(dev->of_node, FIMD_PORT_RGB);
- if (!np)
- return NULL;
-
- ep = of_graph_get_endpoint_by_reg(np, 0);
- of_node_put(np);
+ ep = of_graph_get_endpoint_by_regs(dev->of_node, FIMD_PORT_RGB, 0);
if (!ep)
return NULL;
diff --git a/drivers/gpu/drm/exynos/exynos_drm_drv.c b/drivers/gpu/drm/exynos/exynos_drm_drv.c
index 5344940..2dd820e 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_drv.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_drv.c
@@ -212,13 +212,6 @@
*/
dev->irq_enabled = true;
- /*
- * with vblank_disable_allowed = true, vblank interrupt will be disabled
- * by drm timer once a current process gives up ownership of
- * vblank event.(after drm_vblank_put function is called)
- */
- dev->vblank_disable_allowed = true;
-
/* init kms poll for handling hpd */
drm_kms_helper_poll_init(dev);
@@ -270,7 +263,7 @@
}
int exynos_atomic_commit(struct drm_device *dev, struct drm_atomic_state *state,
- bool async)
+ bool nonblock)
{
struct exynos_drm_private *priv = dev->dev_private;
struct exynos_atomic_commit *commit;
@@ -308,7 +301,7 @@
drm_atomic_helper_swap_state(dev, state);
- if (async)
+ if (nonblock)
schedule_work(&commit->work);
else
exynos_atomic_commit_complete(commit);
@@ -418,7 +411,7 @@
.get_vblank_counter = drm_vblank_no_hw_counter,
.enable_vblank = exynos_drm_crtc_enable_vblank,
.disable_vblank = exynos_drm_crtc_disable_vblank,
- .gem_free_object = exynos_drm_gem_free_object,
+ .gem_free_object_unlocked = exynos_drm_gem_free_object,
.gem_vm_ops = &exynos_drm_gem_vm_ops,
.dumb_create = exynos_drm_gem_dumb_create,
.dumb_map_offset = exynos_drm_gem_dumb_map_offset,
@@ -431,6 +424,7 @@
.gem_prime_import_sg_table = exynos_drm_gem_prime_import_sg_table,
.gem_prime_vmap = exynos_drm_gem_prime_vmap,
.gem_prime_vunmap = exynos_drm_gem_prime_vunmap,
+ .gem_prime_mmap = exynos_drm_gem_prime_mmap,
.ioctls = exynos_ioctls,
.num_ioctls = ARRAY_SIZE(exynos_ioctls),
.fops = &exynos_drm_driver_fops,
diff --git a/drivers/gpu/drm/exynos/exynos_drm_drv.h b/drivers/gpu/drm/exynos/exynos_drm_drv.h
index 502f750..cc33ec9 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_drv.h
+++ b/drivers/gpu/drm/exynos/exynos_drm_drv.h
@@ -120,8 +120,6 @@
* @commit: set current hw specific display mode to hw.
* @enable_vblank: specific driver callback for enabling vblank interrupt.
* @disable_vblank: specific driver callback for disabling vblank interrupt.
- * @wait_for_vblank: wait for vblank interrupt to make sure that
- * hardware overlay is updated.
* @atomic_check: validate state
* @atomic_begin: prepare device to receive an update
* @atomic_flush: mark the end of device update
@@ -129,10 +127,6 @@
* @disable_plane: disable hardware specific overlay.
* @te_handler: trigger to transfer video image at the tearing effect
* synchronization signal if there is a page flip request.
- * @clock_enable: optional function enabling/disabling display domain clock,
- * called from exynos-dp driver before powering up (with
- * 'enable' argument as true) and after powering down (with
- * 'enable' as false).
*/
struct exynos_drm_crtc;
struct exynos_drm_crtc_ops {
@@ -141,7 +135,6 @@
void (*commit)(struct exynos_drm_crtc *crtc);
int (*enable_vblank)(struct exynos_drm_crtc *crtc);
void (*disable_vblank)(struct exynos_drm_crtc *crtc);
- void (*wait_for_vblank)(struct exynos_drm_crtc *crtc);
int (*atomic_check)(struct exynos_drm_crtc *crtc,
struct drm_crtc_state *state);
void (*atomic_begin)(struct exynos_drm_crtc *crtc);
@@ -151,7 +144,10 @@
struct exynos_drm_plane *plane);
void (*atomic_flush)(struct exynos_drm_crtc *crtc);
void (*te_handler)(struct exynos_drm_crtc *crtc);
- void (*clock_enable)(struct exynos_drm_crtc *crtc, bool enable);
+};
+
+struct exynos_drm_clk {
+ void (*enable)(struct exynos_drm_clk *clk, bool enable);
};
/*
@@ -182,8 +178,16 @@
atomic_t pending_update;
const struct exynos_drm_crtc_ops *ops;
void *ctx;
+ struct exynos_drm_clk *pipe_clk;
};
+static inline void exynos_drm_pipe_clk_enable(struct exynos_drm_crtc *crtc,
+ bool enable)
+{
+ if (crtc->pipe_clk)
+ crtc->pipe_clk->enable(crtc->pipe_clk, enable);
+}
+
struct exynos_drm_g2d_private {
struct device *dev;
struct list_head inuse_cmdlist;
@@ -232,6 +236,14 @@
wait_queue_head_t wait;
};
+static inline struct exynos_drm_crtc *
+exynos_drm_crtc_from_pipe(struct drm_device *dev, int pipe)
+{
+ struct exynos_drm_private *private = dev->dev_private;
+
+ return to_exynos_crtc(private->crtc[pipe]);
+}
+
static inline struct device *to_dma_dev(struct drm_device *dev)
{
struct exynos_drm_private *priv = dev->dev_private;
@@ -296,7 +308,7 @@
#endif
int exynos_atomic_commit(struct drm_device *dev, struct drm_atomic_state *state,
- bool async);
+ bool nonblock);
extern struct platform_driver fimd_driver;
diff --git a/drivers/gpu/drm/exynos/exynos_drm_dsi.c b/drivers/gpu/drm/exynos/exynos_drm_dsi.c
index 63c84a1..601ecf8 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_dsi.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_dsi.c
@@ -280,7 +280,7 @@
spinlock_t transfer_lock; /* protects transfer_list */
struct list_head transfer_list;
- struct exynos_dsi_driver_data *driver_data;
+ const struct exynos_dsi_driver_data *driver_data;
struct device_node *bridge_node;
};
@@ -532,15 +532,6 @@
{ }
};
-static inline struct exynos_dsi_driver_data *exynos_dsi_get_driver_data(
- struct platform_device *pdev)
-{
- const struct of_device_id *of_id =
- of_match_device(exynos_dsi_of_match, &pdev->dev);
-
- return (struct exynos_dsi_driver_data *)of_id->data;
-}
-
static void exynos_dsi_wait_for_reset(struct exynos_dsi *dsi)
{
if (wait_for_completion_timeout(&dsi->completed, msecs_to_jiffies(300)))
@@ -564,7 +555,7 @@
static unsigned long exynos_dsi_pll_find_pms(struct exynos_dsi *dsi,
unsigned long fin, unsigned long fout, u8 *p, u16 *m, u8 *s)
{
- struct exynos_dsi_driver_data *driver_data = dsi->driver_data;
+ const struct exynos_dsi_driver_data *driver_data = dsi->driver_data;
unsigned long best_freq = 0;
u32 min_delta = 0xffffffff;
u8 p_min, p_max;
@@ -618,7 +609,7 @@
static unsigned long exynos_dsi_set_pll(struct exynos_dsi *dsi,
unsigned long freq)
{
- struct exynos_dsi_driver_data *driver_data = dsi->driver_data;
+ const struct exynos_dsi_driver_data *driver_data = dsi->driver_data;
unsigned long fin, fout;
int timeout;
u8 p, s;
@@ -712,7 +703,7 @@
static void exynos_dsi_set_phy_ctrl(struct exynos_dsi *dsi)
{
- struct exynos_dsi_driver_data *driver_data = dsi->driver_data;
+ const struct exynos_dsi_driver_data *driver_data = dsi->driver_data;
const unsigned int *reg_values = driver_data->reg_values;
u32 reg;
@@ -790,7 +781,7 @@
static int exynos_dsi_init_link(struct exynos_dsi *dsi)
{
- struct exynos_dsi_driver_data *driver_data = dsi->driver_data;
+ const struct exynos_dsi_driver_data *driver_data = dsi->driver_data;
int timeout;
u32 reg;
u32 lanes_mask;
@@ -1334,7 +1325,7 @@
static int exynos_dsi_init(struct exynos_dsi *dsi)
{
- struct exynos_dsi_driver_data *driver_data = dsi->driver_data;
+ const struct exynos_dsi_driver_data *driver_data = dsi->driver_data;
exynos_dsi_reset(dsi);
exynos_dsi_enable_irq(dsi);
@@ -1641,50 +1632,6 @@
MODULE_DEVICE_TABLE(of, exynos_dsi_of_match);
-/* of_* functions will be removed after merge of of_graph patches */
-static struct device_node *
-of_get_child_by_name_reg(struct device_node *parent, const char *name, u32 reg)
-{
- struct device_node *np;
-
- for_each_child_of_node(parent, np) {
- u32 r;
-
- if (!np->name || of_node_cmp(np->name, name))
- continue;
-
- if (of_property_read_u32(np, "reg", &r) < 0)
- r = 0;
-
- if (reg == r)
- break;
- }
-
- return np;
-}
-
-static struct device_node *of_graph_get_port_by_reg(struct device_node *parent,
- u32 reg)
-{
- struct device_node *ports, *port;
-
- ports = of_get_child_by_name(parent, "ports");
- if (ports)
- parent = ports;
-
- port = of_get_child_by_name_reg(parent, "port", reg);
-
- of_node_put(ports);
-
- return port;
-}
-
-static struct device_node *
-of_graph_get_endpoint_by_reg(struct device_node *port, u32 reg)
-{
- return of_get_child_by_name_reg(port, "endpoint", reg);
-}
-
static int exynos_dsi_of_read_u32(const struct device_node *np,
const char *propname, u32 *out_value)
{
@@ -1706,7 +1653,7 @@
{
struct device *dev = dsi->dev;
struct device_node *node = dev->of_node;
- struct device_node *port, *ep;
+ struct device_node *ep;
int ret;
ret = exynos_dsi_of_read_u32(node, "samsung,pll-clock-frequency",
@@ -1714,16 +1661,9 @@
if (ret < 0)
return ret;
- port = of_graph_get_port_by_reg(node, DSI_PORT_OUT);
- if (!port) {
- dev_err(dev, "no output port specified\n");
- return -EINVAL;
- }
-
- ep = of_graph_get_endpoint_by_reg(port, 0);
- of_node_put(port);
+ ep = of_graph_get_endpoint_by_regs(node, DSI_PORT_OUT, 0);
if (!ep) {
- dev_err(dev, "no endpoint specified in output port\n");
+ dev_err(dev, "no output port with endpoint specified\n");
return -EINVAL;
}
@@ -1833,7 +1773,7 @@
dsi->dsi_host.dev = dev;
dsi->dev = dev;
- dsi->driver_data = exynos_dsi_get_driver_data(pdev);
+ dsi->driver_data = of_device_get_match_data(dev);
ret = exynos_dsi_parse_dt(dsi);
if (ret)
@@ -1917,7 +1857,7 @@
{
struct drm_encoder *encoder = dev_get_drvdata(dev);
struct exynos_dsi *dsi = encoder_to_dsi(encoder);
- struct exynos_dsi_driver_data *driver_data = dsi->driver_data;
+ const struct exynos_dsi_driver_data *driver_data = dsi->driver_data;
int ret, i;
usleep_range(10000, 20000);
@@ -1948,7 +1888,7 @@
{
struct drm_encoder *encoder = dev_get_drvdata(dev);
struct exynos_dsi *dsi = encoder_to_dsi(encoder);
- struct exynos_dsi_driver_data *driver_data = dsi->driver_data;
+ const struct exynos_dsi_driver_data *driver_data = dsi->driver_data;
int ret, i;
ret = regulator_bulk_enable(ARRAY_SIZE(dsi->supplies), dsi->supplies);
diff --git a/drivers/gpu/drm/exynos/exynos_drm_fb.c b/drivers/gpu/drm/exynos/exynos_drm_fb.c
index 81cc553..e016640 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_fb.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_fb.c
@@ -97,20 +97,9 @@
&exynos_fb->exynos_gem[0]->base, handle);
}
-static int exynos_drm_fb_dirty(struct drm_framebuffer *fb,
- struct drm_file *file_priv, unsigned flags,
- unsigned color, struct drm_clip_rect *clips,
- unsigned num_clips)
-{
- /* TODO */
-
- return 0;
-}
-
static const struct drm_framebuffer_funcs exynos_drm_fb_funcs = {
.destroy = exynos_drm_fb_destroy,
.create_handle = exynos_drm_fb_create_handle,
- .dirty = exynos_drm_fb_dirty,
};
struct drm_framebuffer *
@@ -163,8 +152,7 @@
int ret;
for (i = 0; i < drm_format_num_planes(mode_cmd->pixel_format); i++) {
- obj = drm_gem_object_lookup(dev, file_priv,
- mode_cmd->handles[i]);
+ obj = drm_gem_object_lookup(file_priv, mode_cmd->handles[i]);
if (!obj) {
DRM_ERROR("failed to lookup gem object\n");
ret = -ENOENT;
diff --git a/drivers/gpu/drm/exynos/exynos_drm_fbdev.c b/drivers/gpu/drm/exynos/exynos_drm_fbdev.c
index 72d7c0b..67dcd68 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_fbdev.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_fbdev.c
@@ -138,8 +138,6 @@
mode_cmd.pixel_format = drm_mode_legacy_fb_format(sizes->surface_bpp,
sizes->surface_depth);
- mutex_lock(&dev->struct_mutex);
-
size = mode_cmd.pitches[0] * mode_cmd.height;
exynos_gem = exynos_drm_gem_create(dev, EXYNOS_BO_CONTIG, size);
@@ -154,10 +152,8 @@
size);
}
- if (IS_ERR(exynos_gem)) {
- ret = PTR_ERR(exynos_gem);
- goto out;
- }
+ if (IS_ERR(exynos_gem))
+ return PTR_ERR(exynos_gem);
exynos_fbdev->exynos_gem = exynos_gem;
@@ -173,7 +169,6 @@
if (ret < 0)
goto err_destroy_framebuffer;
- mutex_unlock(&dev->struct_mutex);
return ret;
err_destroy_framebuffer:
@@ -181,13 +176,12 @@
err_destroy_gem:
exynos_drm_gem_destroy(exynos_gem);
-/*
- * if failed, all resources allocated above would be released by
- * drm_mode_config_cleanup() when drm_load() had been called prior
- * to any specific driver such as fimd or hdmi driver.
- */
-out:
- mutex_unlock(&dev->struct_mutex);
+ /*
+ * if failed, all resources allocated above would be released by
+ * drm_mode_config_cleanup() when drm_load() had been called prior
+ * to any specific driver such as fimd or hdmi driver.
+ */
+
return ret;
}
diff --git a/drivers/gpu/drm/exynos/exynos_drm_fimd.c b/drivers/gpu/drm/exynos/exynos_drm_fimd.c
index 018449f..3efe1aa 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_fimd.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_fimd.c
@@ -68,10 +68,15 @@
/* color key value register for hardware window 1 ~ 4. */
#define WKEYCON1_BASE(x) ((WKEYCON1 + 0x140) + ((x - 1) * 8))
-/* I80 / RGB trigger control register */
+/* I80 trigger control register */
#define TRIGCON 0x1A4
-#define TRGMODE_I80_RGB_ENABLE_I80 (1 << 0)
-#define SWTRGCMD_I80_RGB_ENABLE (1 << 1)
+#define TRGMODE_ENABLE (1 << 0)
+#define SWTRGCMD_ENABLE (1 << 1)
+/* Exynos3250, 3472, 4415, 5260 5410, 5420 and 5422 only supported. */
+#define HWTRGEN_ENABLE (1 << 3)
+#define HWTRGMASK_ENABLE (1 << 4)
+/* Exynos3250, 3472, 4415, 5260, 5420 and 5422 only supported. */
+#define HWTRIGEN_PER_ENABLE (1 << 31)
/* display mode change control register except exynos4 */
#define VIDOUT_CON 0x000
@@ -89,12 +94,16 @@
/* FIMD has totally five hardware windows. */
#define WINDOWS_NR 5
+/* HW trigger flag on i80 panel. */
+#define I80_HW_TRG (1 << 1)
+
struct fimd_driver_data {
unsigned int timing_base;
unsigned int lcdblk_offset;
unsigned int lcdblk_vt_shift;
unsigned int lcdblk_bypass_shift;
unsigned int lcdblk_mic_bypass_shift;
+ unsigned int trg_type;
unsigned int has_shadowcon:1;
unsigned int has_clksel:1;
@@ -102,20 +111,26 @@
unsigned int has_vidoutcon:1;
unsigned int has_vtsel:1;
unsigned int has_mic_bypass:1;
+ unsigned int has_dp_clk:1;
+ unsigned int has_hw_trigger:1;
+ unsigned int has_trigger_per_te:1;
};
static struct fimd_driver_data s3c64xx_fimd_driver_data = {
.timing_base = 0x0,
.has_clksel = 1,
.has_limited_fmt = 1,
+ .has_hw_trigger = 1,
};
static struct fimd_driver_data exynos3_fimd_driver_data = {
.timing_base = 0x20000,
.lcdblk_offset = 0x210,
.lcdblk_bypass_shift = 1,
+ .trg_type = I80_HW_TRG,
.has_shadowcon = 1,
.has_vidoutcon = 1,
+ .has_trigger_per_te = 1,
};
static struct fimd_driver_data exynos4_fimd_driver_data = {
@@ -132,9 +147,11 @@
.lcdblk_offset = 0x210,
.lcdblk_vt_shift = 10,
.lcdblk_bypass_shift = 1,
+ .trg_type = I80_HW_TRG,
.has_shadowcon = 1,
.has_vidoutcon = 1,
.has_vtsel = 1,
+ .has_trigger_per_te = 1,
};
static struct fimd_driver_data exynos5_fimd_driver_data = {
@@ -145,6 +162,7 @@
.has_shadowcon = 1,
.has_vidoutcon = 1,
.has_vtsel = 1,
+ .has_dp_clk = 1,
};
static struct fimd_driver_data exynos5420_fimd_driver_data = {
@@ -153,10 +171,14 @@
.lcdblk_vt_shift = 24,
.lcdblk_bypass_shift = 15,
.lcdblk_mic_bypass_shift = 11,
+ .trg_type = I80_HW_TRG,
.has_shadowcon = 1,
.has_vidoutcon = 1,
.has_vtsel = 1,
.has_mic_bypass = 1,
+ .has_dp_clk = 1,
+ .has_hw_trigger = 1,
+ .has_trigger_per_te = 1,
};
struct fimd_context {
@@ -182,8 +204,9 @@
atomic_t win_updated;
atomic_t triggering;
- struct fimd_driver_data *driver_data;
+ const struct fimd_driver_data *driver_data;
struct drm_encoder *encoder;
+ struct exynos_drm_clk dp_clk;
};
static const struct of_device_id fimd_driver_dt_match[] = {
@@ -219,15 +242,6 @@
DRM_FORMAT_ARGB8888,
};
-static inline struct fimd_driver_data *drm_fimd_get_driver_data(
- struct platform_device *pdev)
-{
- const struct of_device_id *of_id =
- of_match_device(fimd_driver_dt_match, &pdev->dev);
-
- return (struct fimd_driver_data *)of_id->data;
-}
-
static int fimd_enable_vblank(struct exynos_drm_crtc *crtc)
{
struct fimd_context *ctx = crtc->ctx;
@@ -383,9 +397,16 @@
static u32 fimd_calc_clkdiv(struct fimd_context *ctx,
const struct drm_display_mode *mode)
{
- unsigned long ideal_clk = mode->htotal * mode->vtotal * mode->vrefresh;
+ unsigned long ideal_clk;
u32 clkdiv;
+ if (mode->clock == 0) {
+ DRM_ERROR("Mode has zero clock value.\n");
+ return 0xff;
+ }
+
+ ideal_clk = mode->clock * 1000;
+
if (ctx->i80_if) {
/*
* The frame done interrupt should be occurred prior to the
@@ -400,11 +421,31 @@
return (clkdiv < 0x100) ? clkdiv : 0xff;
}
+static void fimd_setup_trigger(struct fimd_context *ctx)
+{
+ void __iomem *timing_base = ctx->regs + ctx->driver_data->timing_base;
+ u32 trg_type = ctx->driver_data->trg_type;
+ u32 val = readl(timing_base + TRIGCON);
+
+ val &= ~(TRGMODE_ENABLE);
+
+ if (trg_type == I80_HW_TRG) {
+ if (ctx->driver_data->has_hw_trigger)
+ val |= HWTRGEN_ENABLE | HWTRGMASK_ENABLE;
+ if (ctx->driver_data->has_trigger_per_te)
+ val |= HWTRIGEN_PER_ENABLE;
+ } else {
+ val |= TRGMODE_ENABLE;
+ }
+
+ writel(val, timing_base + TRIGCON);
+}
+
static void fimd_commit(struct exynos_drm_crtc *crtc)
{
struct fimd_context *ctx = crtc->ctx;
struct drm_display_mode *mode = &crtc->base.state->adjusted_mode;
- struct fimd_driver_data *driver_data = ctx->driver_data;
+ const struct fimd_driver_data *driver_data = ctx->driver_data;
void *timing_base = ctx->regs + driver_data->timing_base;
u32 val, clkdiv;
@@ -495,6 +536,8 @@
VIDTCON2_HOZVAL_E(mode->hdisplay - 1);
writel(val, ctx->regs + driver_data->timing_base + VIDTCON2);
+ fimd_setup_trigger(ctx);
+
/*
* fields of register with prefix '_F' would be updated
* at vsync(same as dma start)
@@ -827,7 +870,7 @@
static void fimd_trigger(struct device *dev)
{
struct fimd_context *ctx = dev_get_drvdata(dev);
- struct fimd_driver_data *driver_data = ctx->driver_data;
+ const struct fimd_driver_data *driver_data = ctx->driver_data;
void *timing_base = ctx->regs + driver_data->timing_base;
u32 reg;
@@ -842,7 +885,7 @@
atomic_set(&ctx->triggering, 1);
reg = readl(timing_base + TRIGCON);
- reg |= (TRGMODE_I80_RGB_ENABLE_I80 | SWTRGCMD_I80_RGB_ENABLE);
+ reg |= (TRGMODE_ENABLE | SWTRGCMD_ENABLE);
writel(reg, timing_base + TRIGCON);
/*
@@ -856,11 +899,15 @@
static void fimd_te_handler(struct exynos_drm_crtc *crtc)
{
struct fimd_context *ctx = crtc->ctx;
+ u32 trg_type = ctx->driver_data->trg_type;
/* Checks the crtc is detached already from encoder */
if (ctx->pipe < 0 || !ctx->drm_dev)
return;
+ if (trg_type == I80_HW_TRG)
+ goto out;
+
/*
* If there is a page flip request, triggers and handles the page flip
* event so that current fb can be updated into panel GRAM.
@@ -868,6 +915,7 @@
if (atomic_add_unless(&ctx->win_updated, -1, 0))
fimd_trigger(ctx->dev);
+out:
/* Wakes up vsync event queue */
if (atomic_read(&ctx->wait_vsync_event)) {
atomic_set(&ctx->wait_vsync_event, 0);
@@ -878,21 +926,11 @@
drm_crtc_handle_vblank(&ctx->crtc->base);
}
-static void fimd_dp_clock_enable(struct exynos_drm_crtc *crtc, bool enable)
+static void fimd_dp_clock_enable(struct exynos_drm_clk *clk, bool enable)
{
- struct fimd_context *ctx = crtc->ctx;
- u32 val;
-
- /*
- * Only Exynos 5250, 5260, 5410 and 542x requires enabling DP/MIE
- * clock. On these SoCs the bootloader may enable it but any
- * power domain off/on will reset it to disable state.
- */
- if (ctx->driver_data != &exynos5_fimd_driver_data &&
- ctx->driver_data != &exynos5420_fimd_driver_data)
- return;
-
- val = enable ? DP_MIE_CLK_DP_ENABLE : DP_MIE_CLK_DISABLE;
+ struct fimd_context *ctx = container_of(clk, struct fimd_context,
+ dp_clk);
+ u32 val = enable ? DP_MIE_CLK_DP_ENABLE : DP_MIE_CLK_DISABLE;
writel(val, ctx->regs + DP_MIE_CLKCON);
}
@@ -902,13 +940,11 @@
.commit = fimd_commit,
.enable_vblank = fimd_enable_vblank,
.disable_vblank = fimd_disable_vblank,
- .wait_for_vblank = fimd_wait_for_vblank,
.atomic_begin = fimd_atomic_begin,
.update_plane = fimd_update_plane,
.disable_plane = fimd_disable_plane,
.atomic_flush = fimd_atomic_flush,
.te_handler = fimd_te_handler,
- .clock_enable = fimd_dp_clock_enable,
};
static irqreturn_t fimd_irq_handler(int irq, void *dev_id)
@@ -987,6 +1023,11 @@
if (IS_ERR(ctx->crtc))
return PTR_ERR(ctx->crtc);
+ if (ctx->driver_data->has_dp_clk) {
+ ctx->dp_clk.enable = fimd_dp_clock_enable;
+ ctx->crtc->pipe_clk = &ctx->dp_clk;
+ }
+
if (ctx->encoder)
exynos_dpi_bind(drm_dev, ctx->encoder);
@@ -1035,7 +1076,7 @@
ctx->dev = dev;
ctx->suspended = true;
- ctx->driver_data = drm_fimd_get_driver_data(pdev);
+ ctx->driver_data = of_device_get_match_data(dev);
if (of_property_read_bool(dev->of_node, "samsung,invert-vden"))
ctx->vidcon1 |= VIDCON1_INV_VDEN;
diff --git a/drivers/gpu/drm/exynos/exynos_drm_g2d.c b/drivers/gpu/drm/exynos/exynos_drm_g2d.c
index 193d360..4935523 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_g2d.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_g2d.c
@@ -383,8 +383,8 @@
return;
out:
- exynos_gem_unmap_sgt_from_dma(drm_dev, g2d_userptr->sgt,
- DMA_BIDIRECTIONAL);
+ dma_unmap_sg(to_dma_dev(drm_dev), g2d_userptr->sgt->sgl,
+ g2d_userptr->sgt->nents, DMA_BIDIRECTIONAL);
pages = frame_vector_pages(g2d_userptr->vec);
if (!IS_ERR(pages)) {
@@ -501,10 +501,10 @@
g2d_userptr->sgt = sgt;
- ret = exynos_gem_map_sgt_with_dma(drm_dev, g2d_userptr->sgt,
- DMA_BIDIRECTIONAL);
- if (ret < 0) {
+ if (!dma_map_sg(to_dma_dev(drm_dev), sgt->sgl, sgt->nents,
+ DMA_BIDIRECTIONAL)) {
DRM_ERROR("failed to map sgt with dma region.\n");
+ ret = -ENOMEM;
goto err_sg_free_table;
}
diff --git a/drivers/gpu/drm/exynos/exynos_drm_gem.c b/drivers/gpu/drm/exynos/exynos_drm_gem.c
index 2914d62..cdf9f1a 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_gem.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_gem.c
@@ -177,7 +177,7 @@
struct exynos_drm_gem *exynos_gem;
struct drm_gem_object *obj;
- obj = drm_gem_object_lookup(dev, file_priv, gem_handle);
+ obj = drm_gem_object_lookup(file_priv, gem_handle);
if (!obj) {
DRM_ERROR("failed to lookup gem object.\n");
return 0;
@@ -296,7 +296,7 @@
struct exynos_drm_gem *exynos_gem;
struct drm_gem_object *obj;
- obj = drm_gem_object_lookup(dev, filp, gem_handle);
+ obj = drm_gem_object_lookup(filp, gem_handle);
if (!obj) {
DRM_ERROR("failed to lookup gem object.\n");
return ERR_PTR(-EINVAL);
@@ -313,7 +313,7 @@
{
struct drm_gem_object *obj;
- obj = drm_gem_object_lookup(dev, filp, gem_handle);
+ obj = drm_gem_object_lookup(filp, gem_handle);
if (!obj) {
DRM_ERROR("failed to lookup gem object.\n");
return;
@@ -362,12 +362,9 @@
struct drm_exynos_gem_info *args = data;
struct drm_gem_object *obj;
- mutex_lock(&dev->struct_mutex);
-
- obj = drm_gem_object_lookup(dev, file_priv, args->handle);
+ obj = drm_gem_object_lookup(file_priv, args->handle);
if (!obj) {
DRM_ERROR("failed to lookup gem object.\n");
- mutex_unlock(&dev->struct_mutex);
return -EINVAL;
}
@@ -376,38 +373,11 @@
args->flags = exynos_gem->flags;
args->size = exynos_gem->size;
- drm_gem_object_unreference(obj);
- mutex_unlock(&dev->struct_mutex);
+ drm_gem_object_unreference_unlocked(obj);
return 0;
}
-int exynos_gem_map_sgt_with_dma(struct drm_device *drm_dev,
- struct sg_table *sgt,
- enum dma_data_direction dir)
-{
- int nents;
-
- mutex_lock(&drm_dev->struct_mutex);
-
- nents = dma_map_sg(to_dma_dev(drm_dev), sgt->sgl, sgt->nents, dir);
- if (!nents) {
- DRM_ERROR("failed to map sgl with dma.\n");
- mutex_unlock(&drm_dev->struct_mutex);
- return nents;
- }
-
- mutex_unlock(&drm_dev->struct_mutex);
- return 0;
-}
-
-void exynos_gem_unmap_sgt_from_dma(struct drm_device *drm_dev,
- struct sg_table *sgt,
- enum dma_data_direction dir)
-{
- dma_unmap_sg(to_dma_dev(drm_dev), sgt->sgl, sgt->nents, dir);
-}
-
void exynos_drm_gem_free_object(struct drm_gem_object *obj)
{
exynos_drm_gem_destroy(to_exynos_gem(obj));
@@ -458,27 +428,22 @@
struct drm_gem_object *obj;
int ret = 0;
- mutex_lock(&dev->struct_mutex);
-
/*
* get offset of memory allocated for drm framebuffer.
* - this callback would be called by user application
* with DRM_IOCTL_MODE_MAP_DUMB command.
*/
- obj = drm_gem_object_lookup(dev, file_priv, handle);
+ obj = drm_gem_object_lookup(file_priv, handle);
if (!obj) {
DRM_ERROR("failed to lookup gem object.\n");
- ret = -EINVAL;
- goto unlock;
+ return -EINVAL;
}
*offset = drm_vma_node_offset_addr(&obj->vma_node);
DRM_DEBUG_KMS("offset = 0x%lx\n", (unsigned long)*offset);
- drm_gem_object_unreference(obj);
-unlock:
- mutex_unlock(&dev->struct_mutex);
+ drm_gem_object_unreference_unlocked(obj);
return ret;
}
@@ -516,22 +481,12 @@
}
}
-int exynos_drm_gem_mmap(struct file *filp, struct vm_area_struct *vma)
+static int exynos_drm_gem_mmap_obj(struct drm_gem_object *obj,
+ struct vm_area_struct *vma)
{
- struct exynos_drm_gem *exynos_gem;
- struct drm_gem_object *obj;
+ struct exynos_drm_gem *exynos_gem = to_exynos_gem(obj);
int ret;
- /* set vm_area_struct. */
- ret = drm_gem_mmap(filp, vma);
- if (ret < 0) {
- DRM_ERROR("failed to mmap.\n");
- return ret;
- }
-
- obj = vma->vm_private_data;
- exynos_gem = to_exynos_gem(obj);
-
DRM_DEBUG_KMS("flags = 0x%x\n", exynos_gem->flags);
/* non-cachable as default. */
@@ -556,6 +511,26 @@
return ret;
}
+int exynos_drm_gem_mmap(struct file *filp, struct vm_area_struct *vma)
+{
+ struct drm_gem_object *obj;
+ int ret;
+
+ /* set vm_area_struct. */
+ ret = drm_gem_mmap(filp, vma);
+ if (ret < 0) {
+ DRM_ERROR("failed to mmap.\n");
+ return ret;
+ }
+
+ obj = vma->vm_private_data;
+
+ if (obj->import_attach)
+ return dma_buf_mmap(obj->dma_buf, vma, 0);
+
+ return exynos_drm_gem_mmap_obj(obj, vma);
+}
+
/* low-level interface prime helpers */
struct sg_table *exynos_drm_gem_prime_get_sg_table(struct drm_gem_object *obj)
{
@@ -630,3 +605,15 @@
{
/* Nothing to do */
}
+
+int exynos_drm_gem_prime_mmap(struct drm_gem_object *obj,
+ struct vm_area_struct *vma)
+{
+ int ret;
+
+ ret = drm_gem_mmap_obj(obj, obj->size, vma);
+ if (ret < 0)
+ return ret;
+
+ return exynos_drm_gem_mmap_obj(obj, vma);
+}
diff --git a/drivers/gpu/drm/exynos/exynos_drm_gem.h b/drivers/gpu/drm/exynos/exynos_drm_gem.h
index 0022305..7810074 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_gem.h
+++ b/drivers/gpu/drm/exynos/exynos_drm_gem.h
@@ -121,16 +121,6 @@
/* set vm_flags and we can change the vm attribute to other one at here. */
int exynos_drm_gem_mmap(struct file *filp, struct vm_area_struct *vma);
-/* map sgt with dma region. */
-int exynos_gem_map_sgt_with_dma(struct drm_device *drm_dev,
- struct sg_table *sgt,
- enum dma_data_direction dir);
-
-/* unmap sgt from dma region. */
-void exynos_gem_unmap_sgt_from_dma(struct drm_device *drm_dev,
- struct sg_table *sgt,
- enum dma_data_direction dir);
-
/* low-level interface prime helpers */
struct sg_table *exynos_drm_gem_prime_get_sg_table(struct drm_gem_object *obj);
struct drm_gem_object *
@@ -139,5 +129,7 @@
struct sg_table *sgt);
void *exynos_drm_gem_prime_vmap(struct drm_gem_object *obj);
void exynos_drm_gem_prime_vunmap(struct drm_gem_object *obj, void *vaddr);
+int exynos_drm_gem_prime_mmap(struct drm_gem_object *obj,
+ struct vm_area_struct *vma);
#endif
diff --git a/drivers/gpu/drm/exynos/exynos_drm_plane.c b/drivers/gpu/drm/exynos/exynos_drm_plane.c
index 50185ac..55f1d37 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_plane.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_plane.c
@@ -166,7 +166,7 @@
{
struct exynos_drm_plane_state *old_exynos_state =
to_exynos_plane_state(old_state);
- __drm_atomic_helper_plane_destroy_state(plane, old_state);
+ __drm_atomic_helper_plane_destroy_state(old_state);
kfree(old_exynos_state);
}
diff --git a/drivers/gpu/drm/exynos/exynos_drm_rotator.c b/drivers/gpu/drm/exynos/exynos_drm_rotator.c
index f18fbe4..404367a 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_rotator.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_rotator.c
@@ -15,6 +15,7 @@
#include <linux/io.h>
#include <linux/platform_device.h>
#include <linux/clk.h>
+#include <linux/of_device.h>
#include <linux/pm_runtime.h>
#include <drm/drmP.h>
@@ -696,7 +697,6 @@
struct device *dev = &pdev->dev;
struct rot_context *rot;
struct exynos_drm_ippdrv *ippdrv;
- const struct of_device_id *match;
int ret;
if (!dev->of_node) {
@@ -708,13 +708,8 @@
if (!rot)
return -ENOMEM;
- match = of_match_node(exynos_rotator_match, dev->of_node);
- if (!match) {
- dev_err(dev, "failed to match node\n");
- return -ENODEV;
- }
- rot->limit_tbl = (struct rot_limit_table *)match->data;
-
+ rot->limit_tbl = (struct rot_limit_table *)
+ of_device_get_match_data(dev);
rot->regs_res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
rot->regs = devm_ioremap_resource(dev, rot->regs_res);
if (IS_ERR(rot->regs))
diff --git a/drivers/gpu/drm/exynos/exynos_hdmi.c b/drivers/gpu/drm/exynos/exynos_hdmi.c
index e148d72..58de5a4 100644
--- a/drivers/gpu/drm/exynos/exynos_hdmi.c
+++ b/drivers/gpu/drm/exynos/exynos_hdmi.c
@@ -7,9 +7,9 @@
*
* Based on drivers/media/video/s5p-tv/hdmi_drv.c
*
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
* option) any later version.
*
*/
@@ -49,14 +49,16 @@
/* AVI header and aspect ratio */
#define HDMI_AVI_VERSION 0x02
-#define HDMI_AVI_LENGTH 0x0D
+#define HDMI_AVI_LENGTH 0x0d
/* AUI header info */
-#define HDMI_AUI_VERSION 0x01
-#define HDMI_AUI_LENGTH 0x0A
-#define AVI_SAME_AS_PIC_ASPECT_RATIO 0x8
-#define AVI_4_3_CENTER_RATIO 0x9
-#define AVI_16_9_CENTER_RATIO 0xa
+#define HDMI_AUI_VERSION 0x01
+#define HDMI_AUI_LENGTH 0x0a
+
+/* AVI active format aspect ratio */
+#define AVI_SAME_AS_PIC_ASPECT_RATIO 0x08
+#define AVI_4_3_CENTER_RATIO 0x09
+#define AVI_16_9_CENTER_RATIO 0x0a
enum hdmi_type {
HDMI_TYPE13,
@@ -90,11 +92,34 @@
"vdd_pll",
};
+struct hdmiphy_config {
+ int pixel_clock;
+ u8 conf[32];
+};
+
+struct hdmiphy_configs {
+ int count;
+ const struct hdmiphy_config *data;
+};
+
+struct string_array_spec {
+ int count;
+ const char * const *data;
+};
+
+#define INIT_ARRAY_SPEC(a) { .count = ARRAY_SIZE(a), .data = a }
+
struct hdmi_driver_data {
unsigned int type;
- const struct hdmiphy_config *phy_confs;
- unsigned int phy_conf_count;
unsigned int is_apb_phy:1;
+ unsigned int has_sysreg:1;
+ struct hdmiphy_configs phy_confs;
+ struct string_array_spec clk_gates;
+ /*
+ * Array of triplets (p_off, p_on, clock), where p_off and p_on are
+ * required parents of clock when HDMI-PHY is respectively off or on.
+ */
+ struct string_array_spec clk_muxes;
};
struct hdmi_context {
@@ -116,13 +141,12 @@
struct gpio_desc *hpd_gpio;
int irq;
struct regmap *pmureg;
- struct clk *hdmi;
- struct clk *sclk_hdmi;
- struct clk *sclk_pixel;
- struct clk *sclk_hdmiphy;
- struct clk *mout_hdmi;
+ struct regmap *sysreg;
+ struct clk **clk_gates;
+ struct clk **clk_muxes;
struct regulator_bulk_data regul_bulk[ARRAY_SIZE(supply)];
struct regulator *reg_hdmi_en;
+ struct exynos_drm_clk phy_clk;
};
static inline struct hdmi_context *encoder_to_hdmi(struct drm_encoder *e)
@@ -135,12 +159,6 @@
return container_of(c, struct hdmi_context, connector);
}
-struct hdmiphy_config {
- int pixel_clock;
- u8 conf[32];
-};
-
-/* list of phy config settings */
static const struct hdmiphy_config hdmiphy_v13_configs[] = {
{
.pixel_clock = 27000000,
@@ -501,25 +519,136 @@
},
};
-static struct hdmi_driver_data exynos5420_hdmi_driver_data = {
- .type = HDMI_TYPE14,
- .phy_confs = hdmiphy_5420_configs,
- .phy_conf_count = ARRAY_SIZE(hdmiphy_5420_configs),
- .is_apb_phy = 1,
+static const struct hdmiphy_config hdmiphy_5433_configs[] = {
+ {
+ .pixel_clock = 27000000,
+ .conf = {
+ 0x01, 0x51, 0x22, 0x51, 0x08, 0xfc, 0x88, 0x46,
+ 0x72, 0x50, 0x24, 0x0c, 0x24, 0x0f, 0x7c, 0xa5,
+ 0xd4, 0x2b, 0x87, 0x00, 0x00, 0x04, 0x00, 0x30,
+ 0x08, 0x10, 0x01, 0x01, 0x48, 0x40, 0x00, 0x40,
+ },
+ },
+ {
+ .pixel_clock = 27027000,
+ .conf = {
+ 0x01, 0x51, 0x2d, 0x72, 0x64, 0x09, 0x88, 0xc3,
+ 0x71, 0x50, 0x24, 0x14, 0x24, 0x0f, 0x7c, 0xa5,
+ 0xd4, 0x2b, 0x87, 0x00, 0x00, 0x04, 0x00, 0x30,
+ 0x28, 0x10, 0x01, 0x01, 0x48, 0x40, 0x00, 0x40,
+ },
+ },
+ {
+ .pixel_clock = 40000000,
+ .conf = {
+ 0x01, 0x51, 0x32, 0x55, 0x01, 0x00, 0x88, 0x02,
+ 0x4d, 0x50, 0x44, 0x8C, 0x27, 0x00, 0x7C, 0xAC,
+ 0xD6, 0x2B, 0x67, 0x00, 0x00, 0x04, 0x00, 0x30,
+ 0x08, 0x10, 0x01, 0x01, 0x48, 0x40, 0x00, 0x40,
+ },
+ },
+ {
+ .pixel_clock = 50000000,
+ .conf = {
+ 0x01, 0x51, 0x34, 0x40, 0x64, 0x09, 0x88, 0xc3,
+ 0x3d, 0x50, 0x44, 0x8C, 0x27, 0x00, 0x7C, 0xAC,
+ 0xD6, 0x2B, 0x67, 0x00, 0x00, 0x04, 0x00, 0x30,
+ 0x08, 0x10, 0x01, 0x01, 0x48, 0x40, 0x00, 0x40,
+ },
+ },
+ {
+ .pixel_clock = 65000000,
+ .conf = {
+ 0x01, 0x51, 0x36, 0x31, 0x40, 0x10, 0x04, 0xc6,
+ 0x2e, 0xe8, 0x44, 0x8C, 0x27, 0x00, 0x7C, 0xAC,
+ 0xD6, 0x2B, 0x67, 0x00, 0x00, 0x04, 0x00, 0x30,
+ 0x08, 0x10, 0x01, 0x01, 0x48, 0x40, 0x00, 0x40,
+ },
+ },
+ {
+ .pixel_clock = 74176000,
+ .conf = {
+ 0x01, 0x51, 0x3E, 0x35, 0x5B, 0xDE, 0x88, 0x42,
+ 0x53, 0x51, 0x44, 0x8C, 0x27, 0x00, 0x7C, 0xAC,
+ 0xD6, 0x2B, 0x67, 0x00, 0x00, 0x04, 0x00, 0x30,
+ 0x08, 0x10, 0x01, 0x01, 0x48, 0x40, 0x00, 0x40,
+ },
+ },
+ {
+ .pixel_clock = 74250000,
+ .conf = {
+ 0x01, 0x51, 0x3E, 0x35, 0x40, 0xF0, 0x88, 0xC2,
+ 0x52, 0x51, 0x44, 0x8C, 0x27, 0x00, 0x7C, 0xAC,
+ 0xD6, 0x2B, 0x67, 0x00, 0x00, 0x04, 0x00, 0x30,
+ 0x08, 0x10, 0x01, 0x01, 0x48, 0x40, 0x00, 0x40,
+ },
+ },
+ {
+ .pixel_clock = 108000000,
+ .conf = {
+ 0x01, 0x51, 0x2d, 0x15, 0x01, 0x00, 0x88, 0x02,
+ 0x72, 0x52, 0x44, 0x8C, 0x27, 0x00, 0x7C, 0xAC,
+ 0xD6, 0x2B, 0x67, 0x00, 0x00, 0x04, 0x00, 0x30,
+ 0x08, 0x10, 0x01, 0x01, 0x48, 0x40, 0x00, 0x40,
+ },
+ },
+ {
+ .pixel_clock = 148500000,
+ .conf = {
+ 0x01, 0x51, 0x1f, 0x00, 0x40, 0xf8, 0x88, 0xc1,
+ 0x52, 0x52, 0x24, 0x0c, 0x24, 0x0f, 0x7c, 0xa5,
+ 0xd4, 0x2b, 0x87, 0x00, 0x00, 0x04, 0x00, 0x30,
+ 0x08, 0x10, 0x01, 0x01, 0x48, 0x4a, 0x00, 0x40,
+ },
+ },
};
-static struct hdmi_driver_data exynos4212_hdmi_driver_data = {
- .type = HDMI_TYPE14,
- .phy_confs = hdmiphy_v14_configs,
- .phy_conf_count = ARRAY_SIZE(hdmiphy_v14_configs),
- .is_apb_phy = 0,
+static const char * const hdmi_clk_gates4[] = {
+ "hdmi", "sclk_hdmi"
};
-static struct hdmi_driver_data exynos4210_hdmi_driver_data = {
+static const char * const hdmi_clk_muxes4[] = {
+ "sclk_pixel", "sclk_hdmiphy", "mout_hdmi"
+};
+
+static const char * const hdmi_clk_gates5433[] = {
+ "hdmi_pclk", "hdmi_i_pclk", "i_tmds_clk", "i_pixel_clk", "i_spdif_clk"
+};
+
+static const char * const hdmi_clk_muxes5433[] = {
+ "oscclk", "tmds_clko", "tmds_clko_user",
+ "oscclk", "pixel_clko", "pixel_clko_user"
+};
+
+static const struct hdmi_driver_data exynos4210_hdmi_driver_data = {
.type = HDMI_TYPE13,
- .phy_confs = hdmiphy_v13_configs,
- .phy_conf_count = ARRAY_SIZE(hdmiphy_v13_configs),
- .is_apb_phy = 0,
+ .phy_confs = INIT_ARRAY_SPEC(hdmiphy_v13_configs),
+ .clk_gates = INIT_ARRAY_SPEC(hdmi_clk_gates4),
+ .clk_muxes = INIT_ARRAY_SPEC(hdmi_clk_muxes4),
+};
+
+static const struct hdmi_driver_data exynos4212_hdmi_driver_data = {
+ .type = HDMI_TYPE14,
+ .phy_confs = INIT_ARRAY_SPEC(hdmiphy_v14_configs),
+ .clk_gates = INIT_ARRAY_SPEC(hdmi_clk_gates4),
+ .clk_muxes = INIT_ARRAY_SPEC(hdmi_clk_muxes4),
+};
+
+static const struct hdmi_driver_data exynos5420_hdmi_driver_data = {
+ .type = HDMI_TYPE14,
+ .is_apb_phy = 1,
+ .phy_confs = INIT_ARRAY_SPEC(hdmiphy_5420_configs),
+ .clk_gates = INIT_ARRAY_SPEC(hdmi_clk_gates4),
+ .clk_muxes = INIT_ARRAY_SPEC(hdmi_clk_muxes4),
+};
+
+static const struct hdmi_driver_data exynos5433_hdmi_driver_data = {
+ .type = HDMI_TYPE14,
+ .is_apb_phy = 1,
+ .has_sysreg = 1,
+ .phy_confs = INIT_ARRAY_SPEC(hdmiphy_5433_configs),
+ .clk_gates = INIT_ARRAY_SPEC(hdmi_clk_gates5433),
+ .clk_muxes = INIT_ARRAY_SPEC(hdmi_clk_muxes5433),
};
static inline u32 hdmi_map_reg(struct hdmi_context *hdata, u32 reg_id)
@@ -585,266 +714,52 @@
}
}
-static void hdmi_v13_regs_dump(struct hdmi_context *hdata, char *prefix)
+static int hdmi_clk_enable_gates(struct hdmi_context *hdata)
{
-#define DUMPREG(reg_id) \
- DRM_DEBUG_KMS("%s:" #reg_id " = %08x\n", prefix, \
- readl(hdata->regs + reg_id))
- DRM_DEBUG_KMS("%s: ---- CONTROL REGISTERS ----\n", prefix);
- DUMPREG(HDMI_INTC_FLAG);
- DUMPREG(HDMI_INTC_CON);
- DUMPREG(HDMI_HPD_STATUS);
- DUMPREG(HDMI_V13_PHY_RSTOUT);
- DUMPREG(HDMI_V13_PHY_VPLL);
- DUMPREG(HDMI_V13_PHY_CMU);
- DUMPREG(HDMI_V13_CORE_RSTOUT);
+ int i, ret;
- DRM_DEBUG_KMS("%s: ---- CORE REGISTERS ----\n", prefix);
- DUMPREG(HDMI_CON_0);
- DUMPREG(HDMI_CON_1);
- DUMPREG(HDMI_CON_2);
- DUMPREG(HDMI_SYS_STATUS);
- DUMPREG(HDMI_V13_PHY_STATUS);
- DUMPREG(HDMI_STATUS_EN);
- DUMPREG(HDMI_HPD);
- DUMPREG(HDMI_MODE_SEL);
- DUMPREG(HDMI_V13_HPD_GEN);
- DUMPREG(HDMI_V13_DC_CONTROL);
- DUMPREG(HDMI_V13_VIDEO_PATTERN_GEN);
+ for (i = 0; i < hdata->drv_data->clk_gates.count; ++i) {
+ ret = clk_prepare_enable(hdata->clk_gates[i]);
+ if (!ret)
+ continue;
- DRM_DEBUG_KMS("%s: ---- CORE SYNC REGISTERS ----\n", prefix);
- DUMPREG(HDMI_H_BLANK_0);
- DUMPREG(HDMI_H_BLANK_1);
- DUMPREG(HDMI_V13_V_BLANK_0);
- DUMPREG(HDMI_V13_V_BLANK_1);
- DUMPREG(HDMI_V13_V_BLANK_2);
- DUMPREG(HDMI_V13_H_V_LINE_0);
- DUMPREG(HDMI_V13_H_V_LINE_1);
- DUMPREG(HDMI_V13_H_V_LINE_2);
- DUMPREG(HDMI_VSYNC_POL);
- DUMPREG(HDMI_INT_PRO_MODE);
- DUMPREG(HDMI_V13_V_BLANK_F_0);
- DUMPREG(HDMI_V13_V_BLANK_F_1);
- DUMPREG(HDMI_V13_V_BLANK_F_2);
- DUMPREG(HDMI_V13_H_SYNC_GEN_0);
- DUMPREG(HDMI_V13_H_SYNC_GEN_1);
- DUMPREG(HDMI_V13_H_SYNC_GEN_2);
- DUMPREG(HDMI_V13_V_SYNC_GEN_1_0);
- DUMPREG(HDMI_V13_V_SYNC_GEN_1_1);
- DUMPREG(HDMI_V13_V_SYNC_GEN_1_2);
- DUMPREG(HDMI_V13_V_SYNC_GEN_2_0);
- DUMPREG(HDMI_V13_V_SYNC_GEN_2_1);
- DUMPREG(HDMI_V13_V_SYNC_GEN_2_2);
- DUMPREG(HDMI_V13_V_SYNC_GEN_3_0);
- DUMPREG(HDMI_V13_V_SYNC_GEN_3_1);
- DUMPREG(HDMI_V13_V_SYNC_GEN_3_2);
+ dev_err(hdata->dev, "Cannot enable clock '%s', %d\n",
+ hdata->drv_data->clk_gates.data[i], ret);
+ while (i--)
+ clk_disable_unprepare(hdata->clk_gates[i]);
+ return ret;
+ }
- DRM_DEBUG_KMS("%s: ---- TG REGISTERS ----\n", prefix);
- DUMPREG(HDMI_TG_CMD);
- DUMPREG(HDMI_TG_H_FSZ_L);
- DUMPREG(HDMI_TG_H_FSZ_H);
- DUMPREG(HDMI_TG_HACT_ST_L);
- DUMPREG(HDMI_TG_HACT_ST_H);
- DUMPREG(HDMI_TG_HACT_SZ_L);
- DUMPREG(HDMI_TG_HACT_SZ_H);
- DUMPREG(HDMI_TG_V_FSZ_L);
- DUMPREG(HDMI_TG_V_FSZ_H);
- DUMPREG(HDMI_TG_VSYNC_L);
- DUMPREG(HDMI_TG_VSYNC_H);
- DUMPREG(HDMI_TG_VSYNC2_L);
- DUMPREG(HDMI_TG_VSYNC2_H);
- DUMPREG(HDMI_TG_VACT_ST_L);
- DUMPREG(HDMI_TG_VACT_ST_H);
- DUMPREG(HDMI_TG_VACT_SZ_L);
- DUMPREG(HDMI_TG_VACT_SZ_H);
- DUMPREG(HDMI_TG_FIELD_CHG_L);
- DUMPREG(HDMI_TG_FIELD_CHG_H);
- DUMPREG(HDMI_TG_VACT_ST2_L);
- DUMPREG(HDMI_TG_VACT_ST2_H);
- DUMPREG(HDMI_TG_VSYNC_TOP_HDMI_L);
- DUMPREG(HDMI_TG_VSYNC_TOP_HDMI_H);
- DUMPREG(HDMI_TG_VSYNC_BOT_HDMI_L);
- DUMPREG(HDMI_TG_VSYNC_BOT_HDMI_H);
- DUMPREG(HDMI_TG_FIELD_TOP_HDMI_L);
- DUMPREG(HDMI_TG_FIELD_TOP_HDMI_H);
- DUMPREG(HDMI_TG_FIELD_BOT_HDMI_L);
- DUMPREG(HDMI_TG_FIELD_BOT_HDMI_H);
-#undef DUMPREG
+ return 0;
}
-static void hdmi_v14_regs_dump(struct hdmi_context *hdata, char *prefix)
+static void hdmi_clk_disable_gates(struct hdmi_context *hdata)
{
+ int i = hdata->drv_data->clk_gates.count;
+
+ while (i--)
+ clk_disable_unprepare(hdata->clk_gates[i]);
+}
+
+static int hdmi_clk_set_parents(struct hdmi_context *hdata, bool to_phy)
+{
+ struct device *dev = hdata->dev;
+ int ret = 0;
int i;
-#define DUMPREG(reg_id) \
- DRM_DEBUG_KMS("%s:" #reg_id " = %08x\n", prefix, \
- readl(hdata->regs + reg_id))
+ for (i = 0; i < hdata->drv_data->clk_muxes.count; i += 3) {
+ struct clk **c = &hdata->clk_muxes[i];
- DRM_DEBUG_KMS("%s: ---- CONTROL REGISTERS ----\n", prefix);
- DUMPREG(HDMI_INTC_CON);
- DUMPREG(HDMI_INTC_FLAG);
- DUMPREG(HDMI_HPD_STATUS);
- DUMPREG(HDMI_INTC_CON_1);
- DUMPREG(HDMI_INTC_FLAG_1);
- DUMPREG(HDMI_PHY_STATUS_0);
- DUMPREG(HDMI_PHY_STATUS_PLL);
- DUMPREG(HDMI_PHY_CON_0);
- DUMPREG(HDMI_V14_PHY_RSTOUT);
- DUMPREG(HDMI_PHY_VPLL);
- DUMPREG(HDMI_PHY_CMU);
- DUMPREG(HDMI_CORE_RSTOUT);
+ ret = clk_set_parent(c[2], c[to_phy]);
+ if (!ret)
+ continue;
- DRM_DEBUG_KMS("%s: ---- CORE REGISTERS ----\n", prefix);
- DUMPREG(HDMI_CON_0);
- DUMPREG(HDMI_CON_1);
- DUMPREG(HDMI_CON_2);
- DUMPREG(HDMI_SYS_STATUS);
- DUMPREG(HDMI_PHY_STATUS_0);
- DUMPREG(HDMI_STATUS_EN);
- DUMPREG(HDMI_HPD);
- DUMPREG(HDMI_MODE_SEL);
- DUMPREG(HDMI_ENC_EN);
- DUMPREG(HDMI_DC_CONTROL);
- DUMPREG(HDMI_VIDEO_PATTERN_GEN);
+ dev_err(dev, "Cannot set clock parent of '%s' to '%s', %d\n",
+ hdata->drv_data->clk_muxes.data[i + 2],
+ hdata->drv_data->clk_muxes.data[i + to_phy], ret);
+ }
- DRM_DEBUG_KMS("%s: ---- CORE SYNC REGISTERS ----\n", prefix);
- DUMPREG(HDMI_H_BLANK_0);
- DUMPREG(HDMI_H_BLANK_1);
- DUMPREG(HDMI_V2_BLANK_0);
- DUMPREG(HDMI_V2_BLANK_1);
- DUMPREG(HDMI_V1_BLANK_0);
- DUMPREG(HDMI_V1_BLANK_1);
- DUMPREG(HDMI_V_LINE_0);
- DUMPREG(HDMI_V_LINE_1);
- DUMPREG(HDMI_H_LINE_0);
- DUMPREG(HDMI_H_LINE_1);
- DUMPREG(HDMI_HSYNC_POL);
-
- DUMPREG(HDMI_VSYNC_POL);
- DUMPREG(HDMI_INT_PRO_MODE);
- DUMPREG(HDMI_V_BLANK_F0_0);
- DUMPREG(HDMI_V_BLANK_F0_1);
- DUMPREG(HDMI_V_BLANK_F1_0);
- DUMPREG(HDMI_V_BLANK_F1_1);
-
- DUMPREG(HDMI_H_SYNC_START_0);
- DUMPREG(HDMI_H_SYNC_START_1);
- DUMPREG(HDMI_H_SYNC_END_0);
- DUMPREG(HDMI_H_SYNC_END_1);
-
- DUMPREG(HDMI_V_SYNC_LINE_BEF_2_0);
- DUMPREG(HDMI_V_SYNC_LINE_BEF_2_1);
- DUMPREG(HDMI_V_SYNC_LINE_BEF_1_0);
- DUMPREG(HDMI_V_SYNC_LINE_BEF_1_1);
-
- DUMPREG(HDMI_V_SYNC_LINE_AFT_2_0);
- DUMPREG(HDMI_V_SYNC_LINE_AFT_2_1);
- DUMPREG(HDMI_V_SYNC_LINE_AFT_1_0);
- DUMPREG(HDMI_V_SYNC_LINE_AFT_1_1);
-
- DUMPREG(HDMI_V_SYNC_LINE_AFT_PXL_2_0);
- DUMPREG(HDMI_V_SYNC_LINE_AFT_PXL_2_1);
- DUMPREG(HDMI_V_SYNC_LINE_AFT_PXL_1_0);
- DUMPREG(HDMI_V_SYNC_LINE_AFT_PXL_1_1);
-
- DUMPREG(HDMI_V_BLANK_F2_0);
- DUMPREG(HDMI_V_BLANK_F2_1);
- DUMPREG(HDMI_V_BLANK_F3_0);
- DUMPREG(HDMI_V_BLANK_F3_1);
- DUMPREG(HDMI_V_BLANK_F4_0);
- DUMPREG(HDMI_V_BLANK_F4_1);
- DUMPREG(HDMI_V_BLANK_F5_0);
- DUMPREG(HDMI_V_BLANK_F5_1);
-
- DUMPREG(HDMI_V_SYNC_LINE_AFT_3_0);
- DUMPREG(HDMI_V_SYNC_LINE_AFT_3_1);
- DUMPREG(HDMI_V_SYNC_LINE_AFT_4_0);
- DUMPREG(HDMI_V_SYNC_LINE_AFT_4_1);
- DUMPREG(HDMI_V_SYNC_LINE_AFT_5_0);
- DUMPREG(HDMI_V_SYNC_LINE_AFT_5_1);
- DUMPREG(HDMI_V_SYNC_LINE_AFT_6_0);
- DUMPREG(HDMI_V_SYNC_LINE_AFT_6_1);
-
- DUMPREG(HDMI_V_SYNC_LINE_AFT_PXL_3_0);
- DUMPREG(HDMI_V_SYNC_LINE_AFT_PXL_3_1);
- DUMPREG(HDMI_V_SYNC_LINE_AFT_PXL_4_0);
- DUMPREG(HDMI_V_SYNC_LINE_AFT_PXL_4_1);
- DUMPREG(HDMI_V_SYNC_LINE_AFT_PXL_5_0);
- DUMPREG(HDMI_V_SYNC_LINE_AFT_PXL_5_1);
- DUMPREG(HDMI_V_SYNC_LINE_AFT_PXL_6_0);
- DUMPREG(HDMI_V_SYNC_LINE_AFT_PXL_6_1);
-
- DUMPREG(HDMI_VACT_SPACE_1_0);
- DUMPREG(HDMI_VACT_SPACE_1_1);
- DUMPREG(HDMI_VACT_SPACE_2_0);
- DUMPREG(HDMI_VACT_SPACE_2_1);
- DUMPREG(HDMI_VACT_SPACE_3_0);
- DUMPREG(HDMI_VACT_SPACE_3_1);
- DUMPREG(HDMI_VACT_SPACE_4_0);
- DUMPREG(HDMI_VACT_SPACE_4_1);
- DUMPREG(HDMI_VACT_SPACE_5_0);
- DUMPREG(HDMI_VACT_SPACE_5_1);
- DUMPREG(HDMI_VACT_SPACE_6_0);
- DUMPREG(HDMI_VACT_SPACE_6_1);
-
- DRM_DEBUG_KMS("%s: ---- TG REGISTERS ----\n", prefix);
- DUMPREG(HDMI_TG_CMD);
- DUMPREG(HDMI_TG_H_FSZ_L);
- DUMPREG(HDMI_TG_H_FSZ_H);
- DUMPREG(HDMI_TG_HACT_ST_L);
- DUMPREG(HDMI_TG_HACT_ST_H);
- DUMPREG(HDMI_TG_HACT_SZ_L);
- DUMPREG(HDMI_TG_HACT_SZ_H);
- DUMPREG(HDMI_TG_V_FSZ_L);
- DUMPREG(HDMI_TG_V_FSZ_H);
- DUMPREG(HDMI_TG_VSYNC_L);
- DUMPREG(HDMI_TG_VSYNC_H);
- DUMPREG(HDMI_TG_VSYNC2_L);
- DUMPREG(HDMI_TG_VSYNC2_H);
- DUMPREG(HDMI_TG_VACT_ST_L);
- DUMPREG(HDMI_TG_VACT_ST_H);
- DUMPREG(HDMI_TG_VACT_SZ_L);
- DUMPREG(HDMI_TG_VACT_SZ_H);
- DUMPREG(HDMI_TG_FIELD_CHG_L);
- DUMPREG(HDMI_TG_FIELD_CHG_H);
- DUMPREG(HDMI_TG_VACT_ST2_L);
- DUMPREG(HDMI_TG_VACT_ST2_H);
- DUMPREG(HDMI_TG_VACT_ST3_L);
- DUMPREG(HDMI_TG_VACT_ST3_H);
- DUMPREG(HDMI_TG_VACT_ST4_L);
- DUMPREG(HDMI_TG_VACT_ST4_H);
- DUMPREG(HDMI_TG_VSYNC_TOP_HDMI_L);
- DUMPREG(HDMI_TG_VSYNC_TOP_HDMI_H);
- DUMPREG(HDMI_TG_VSYNC_BOT_HDMI_L);
- DUMPREG(HDMI_TG_VSYNC_BOT_HDMI_H);
- DUMPREG(HDMI_TG_FIELD_TOP_HDMI_L);
- DUMPREG(HDMI_TG_FIELD_TOP_HDMI_H);
- DUMPREG(HDMI_TG_FIELD_BOT_HDMI_L);
- DUMPREG(HDMI_TG_FIELD_BOT_HDMI_H);
- DUMPREG(HDMI_TG_3D);
-
- DRM_DEBUG_KMS("%s: ---- PACKET REGISTERS ----\n", prefix);
- DUMPREG(HDMI_AVI_CON);
- DUMPREG(HDMI_AVI_HEADER0);
- DUMPREG(HDMI_AVI_HEADER1);
- DUMPREG(HDMI_AVI_HEADER2);
- DUMPREG(HDMI_AVI_CHECK_SUM);
- DUMPREG(HDMI_VSI_CON);
- DUMPREG(HDMI_VSI_HEADER0);
- DUMPREG(HDMI_VSI_HEADER1);
- DUMPREG(HDMI_VSI_HEADER2);
- for (i = 0; i < 7; ++i)
- DUMPREG(HDMI_VSI_DATA(i));
-
-#undef DUMPREG
-}
-
-static void hdmi_regs_dump(struct hdmi_context *hdata, char *prefix)
-{
- if (hdata->drv_data->type == HDMI_TYPE13)
- hdmi_v13_regs_dump(hdata, prefix);
- else
- hdmi_v14_regs_dump(hdata, prefix);
+ return ret;
}
static u8 hdmi_chksum(struct hdmi_context *hdata,
@@ -993,10 +908,11 @@
static int hdmi_find_phy_conf(struct hdmi_context *hdata, u32 pixel_clock)
{
+ const struct hdmiphy_configs *confs = &hdata->drv_data->phy_confs;
int i;
- for (i = 0; i < hdata->drv_data->phy_conf_count; i++)
- if (hdata->drv_data->phy_confs[i].pixel_clock == pixel_clock)
+ for (i = 0; i < confs->count; i++)
+ if (confs->data[i].pixel_clock == pixel_clock)
return i;
DRM_DEBUG_KMS("Could not find phy config for %d\n", pixel_clock);
@@ -1078,13 +994,11 @@
mode_ok = hdmi_mode_valid(connector, adjusted_mode);
- /* just return if user desired mode exists. */
if (mode_ok == MODE_OK)
return true;
/*
- * otherwise, find the most suitable mode among modes and change it
- * to adjusted_mode.
+ * Find the most suitable mode and copy it to adjusted_mode.
*/
list_for_each_entry(m, &connector->modes, head) {
mode_ok = hdmi_mode_valid(connector, m);
@@ -1129,15 +1043,15 @@
switch (bits_per_sample) {
case 20:
data_num = 2;
- bit_ch = 1;
+ bit_ch = 1;
break;
case 24:
data_num = 3;
- bit_ch = 1;
+ bit_ch = 1;
break;
default:
data_num = 1;
- bit_ch = 0;
+ bit_ch = 0;
break;
}
@@ -1230,13 +1144,12 @@
/* choose HDMI mode */
hdmi_reg_writemask(hdata, HDMI_MODE_SEL,
HDMI_MODE_HDMI_EN, HDMI_MODE_MASK);
- /* Apply Video preable and Guard band in HDMI mode only */
+ /* apply video pre-amble and guard band in HDMI mode only */
hdmi_reg_writeb(hdata, HDMI_CON_2, 0);
/* disable bluescreen */
hdmi_reg_writemask(hdata, HDMI_CON_0, 0, HDMI_BLUE_SCR_EN);
if (hdata->dvi_mode) {
- /* choose DVI mode */
hdmi_reg_writemask(hdata, HDMI_MODE_SEL,
HDMI_MODE_DVI_EN, HDMI_MODE_MASK);
hdmi_reg_writeb(hdata, HDMI_CON_2,
@@ -1308,7 +1221,7 @@
val = (m->hsync_start - m->hdisplay - 2);
val |= ((m->hsync_end - m->hdisplay - 2) << 10);
- val |= ((m->flags & DRM_MODE_FLAG_NHSYNC) ? 1 : 0)<<20;
+ val |= ((m->flags & DRM_MODE_FLAG_NHSYNC) ? 1 : 0)<<20;
hdmi_reg_writev(hdata, HDMI_V13_H_SYNC_GEN_0, 3, val);
/*
@@ -1319,7 +1232,6 @@
/* Following values & calculations differ for different type of modes */
if (m->flags & DRM_MODE_FLAG_INTERLACE) {
- /* Interlaced Mode */
val = ((m->vsync_end - m->vdisplay) / 2);
val |= ((m->vsync_start - m->vdisplay) / 2) << 12;
hdmi_reg_writev(hdata, HDMI_V13_V_SYNC_GEN_1_0, 3, val);
@@ -1348,8 +1260,6 @@
hdmi_reg_writev(hdata, HDMI_TG_VACT_ST2_L, 2, 0x249);
} else {
- /* Progressive Mode */
-
val = m->vtotal;
val |= (m->vtotal - m->vdisplay) << 11;
hdmi_reg_writev(hdata, HDMI_V13_V_BLANK_0, 3, val);
@@ -1365,21 +1275,12 @@
hdmi_reg_writev(hdata, HDMI_TG_VACT_ST_L, 2,
m->vtotal - m->vdisplay);
hdmi_reg_writev(hdata, HDMI_TG_VACT_SZ_L, 2, m->vdisplay);
- hdmi_reg_writev(hdata, HDMI_TG_VACT_ST2_L, 2, 0x248);
}
- /* Timing generator registers */
hdmi_reg_writev(hdata, HDMI_TG_H_FSZ_L, 2, m->htotal);
hdmi_reg_writev(hdata, HDMI_TG_HACT_ST_L, 2, m->htotal - m->hdisplay);
hdmi_reg_writev(hdata, HDMI_TG_HACT_SZ_L, 2, m->hdisplay);
hdmi_reg_writev(hdata, HDMI_TG_V_FSZ_L, 2, m->vtotal);
- hdmi_reg_writev(hdata, HDMI_TG_VSYNC_L, 2, 0x1);
- hdmi_reg_writev(hdata, HDMI_TG_VSYNC2_L, 2, 0x233);
- hdmi_reg_writev(hdata, HDMI_TG_FIELD_CHG_L, 2, 0x233);
- hdmi_reg_writev(hdata, HDMI_TG_VSYNC_TOP_HDMI_L, 2, 0x1);
- hdmi_reg_writev(hdata, HDMI_TG_VSYNC_BOT_HDMI_L, 2, 0x233);
- hdmi_reg_writev(hdata, HDMI_TG_FIELD_TOP_HDMI_L, 2, 0x1);
- hdmi_reg_writev(hdata, HDMI_TG_FIELD_BOT_HDMI_L, 2, 0x233);
}
static void hdmi_v14_mode_apply(struct hdmi_context *hdata)
@@ -1390,7 +1291,7 @@
hdmi_reg_writev(hdata, HDMI_V_LINE_0, 2, m->vtotal);
hdmi_reg_writev(hdata, HDMI_H_LINE_0, 2, m->htotal);
hdmi_reg_writev(hdata, HDMI_HSYNC_POL, 1,
- (m->flags & DRM_MODE_FLAG_NHSYNC) ? 1 : 0);
+ (m->flags & DRM_MODE_FLAG_NHSYNC) ? 1 : 0);
hdmi_reg_writev(hdata, HDMI_VSYNC_POL, 1,
(m->flags & DRM_MODE_FLAG_NVSYNC) ? 1 : 0);
hdmi_reg_writev(hdata, HDMI_INT_PRO_MODE, 1,
@@ -1404,7 +1305,6 @@
/* Following values & calculations differ for different type of modes */
if (m->flags & DRM_MODE_FLAG_INTERLACE) {
- /* Interlaced Mode */
hdmi_reg_writev(hdata, HDMI_V_SYNC_LINE_BEF_2_0, 2,
(m->vsync_end - m->vdisplay) / 2);
hdmi_reg_writev(hdata, HDMI_V_SYNC_LINE_BEF_1_0, 2,
@@ -1437,7 +1337,6 @@
hdmi_reg_writev(hdata, HDMI_TG_VACT_ST3_L, 2, 0x0);
hdmi_reg_writev(hdata, HDMI_TG_VACT_ST4_L, 2, 0x0);
} else {
- /* Progressive Mode */
hdmi_reg_writev(hdata, HDMI_V_SYNC_LINE_BEF_2_0, 2,
m->vsync_end - m->vdisplay);
hdmi_reg_writev(hdata, HDMI_V_SYNC_LINE_BEF_1_0, 2,
@@ -1454,15 +1353,8 @@
hdmi_reg_writev(hdata, HDMI_TG_VACT_ST_L, 2,
m->vtotal - m->vdisplay);
hdmi_reg_writev(hdata, HDMI_TG_VACT_SZ_L, 2, m->vdisplay);
- hdmi_reg_writev(hdata, HDMI_TG_VACT_ST2_L, 2, 0x248);
- hdmi_reg_writev(hdata, HDMI_TG_VACT_ST3_L, 2, 0x47b);
- hdmi_reg_writev(hdata, HDMI_TG_VACT_ST4_L, 2, 0x6ae);
- hdmi_reg_writev(hdata, HDMI_TG_VSYNC2_L, 2, 0x233);
- hdmi_reg_writev(hdata, HDMI_TG_VSYNC_BOT_HDMI_L, 2, 0x233);
- hdmi_reg_writev(hdata, HDMI_TG_FIELD_BOT_HDMI_L, 2, 0x233);
}
- /* Following values & calculations are same irrespective of mode type */
hdmi_reg_writev(hdata, HDMI_H_SYNC_START_0, 2,
m->hsync_start - m->hdisplay - 2);
hdmi_reg_writev(hdata, HDMI_H_SYNC_END_0, 2,
@@ -1486,16 +1378,12 @@
hdmi_reg_writev(hdata, HDMI_V_SYNC_LINE_AFT_PXL_5_0, 2, 0xffff);
hdmi_reg_writev(hdata, HDMI_V_SYNC_LINE_AFT_PXL_6_0, 2, 0xffff);
- /* Timing generator registers */
hdmi_reg_writev(hdata, HDMI_TG_H_FSZ_L, 2, m->htotal);
hdmi_reg_writev(hdata, HDMI_TG_HACT_ST_L, 2, m->htotal - m->hdisplay);
hdmi_reg_writev(hdata, HDMI_TG_HACT_SZ_L, 2, m->hdisplay);
hdmi_reg_writev(hdata, HDMI_TG_V_FSZ_L, 2, m->vtotal);
- hdmi_reg_writev(hdata, HDMI_TG_VSYNC_L, 2, 0x1);
- hdmi_reg_writev(hdata, HDMI_TG_FIELD_CHG_L, 2, 0x233);
- hdmi_reg_writev(hdata, HDMI_TG_VSYNC_TOP_HDMI_L, 2, 0x1);
- hdmi_reg_writev(hdata, HDMI_TG_FIELD_TOP_HDMI_L, 2, 0x1);
- hdmi_reg_writev(hdata, HDMI_TG_3D, 1, 0x0);
+ if (hdata->drv_data == &exynos5433_hdmi_driver_data)
+ hdmi_reg_writeb(hdata, HDMI_TG_DECON_EN, 1);
}
static void hdmi_mode_apply(struct hdmi_context *hdata)
@@ -1505,62 +1393,64 @@
else
hdmi_v14_mode_apply(hdata);
- hdmiphy_wait_for_pll(hdata);
-
- clk_set_parent(hdata->mout_hdmi, hdata->sclk_hdmiphy);
-
- /* enable HDMI and timing generator */
hdmi_start(hdata, true);
}
static void hdmiphy_conf_reset(struct hdmi_context *hdata)
{
- clk_set_parent(hdata->mout_hdmi, hdata->sclk_pixel);
-
- /* reset hdmiphy */
+ hdmi_reg_writemask(hdata, HDMI_CORE_RSTOUT, 0, 1);
+ usleep_range(10000, 12000);
+ hdmi_reg_writemask(hdata, HDMI_CORE_RSTOUT, ~0, 1);
+ usleep_range(10000, 12000);
hdmi_reg_writemask(hdata, HDMI_PHY_RSTOUT, ~0, HDMI_PHY_SW_RSTOUT);
usleep_range(10000, 12000);
- hdmi_reg_writemask(hdata, HDMI_PHY_RSTOUT, 0, HDMI_PHY_SW_RSTOUT);
+ hdmi_reg_writemask(hdata, HDMI_PHY_RSTOUT, 0, HDMI_PHY_SW_RSTOUT);
usleep_range(10000, 12000);
}
+static void hdmiphy_enable_mode_set(struct hdmi_context *hdata, bool enable)
+{
+ u8 v = enable ? HDMI_PHY_ENABLE_MODE_SET : HDMI_PHY_DISABLE_MODE_SET;
+
+ if (hdata->drv_data == &exynos5433_hdmi_driver_data)
+ writel(v, hdata->regs_hdmiphy + HDMIPHY5433_MODE_SET_DONE);
+}
+
static void hdmiphy_conf_apply(struct hdmi_context *hdata)
{
int ret;
- int i;
+ const u8 *phy_conf;
- /* pixel clock */
- i = hdmi_find_phy_conf(hdata, hdata->current_mode.clock * 1000);
- if (i < 0) {
+ ret = hdmi_find_phy_conf(hdata, hdata->current_mode.clock * 1000);
+ if (ret < 0) {
DRM_ERROR("failed to find hdmiphy conf\n");
return;
}
+ phy_conf = hdata->drv_data->phy_confs.data[ret].conf;
- ret = hdmiphy_reg_write_buf(hdata, 0,
- hdata->drv_data->phy_confs[i].conf, 32);
+ hdmi_clk_set_parents(hdata, false);
+
+ hdmiphy_conf_reset(hdata);
+
+ hdmiphy_enable_mode_set(hdata, true);
+ ret = hdmiphy_reg_write_buf(hdata, 0, phy_conf, 32);
if (ret) {
DRM_ERROR("failed to configure hdmiphy\n");
return;
}
-
+ hdmiphy_enable_mode_set(hdata, false);
+ hdmi_clk_set_parents(hdata, true);
usleep_range(10000, 12000);
+ hdmiphy_wait_for_pll(hdata);
}
static void hdmi_conf_apply(struct hdmi_context *hdata)
{
- hdmiphy_conf_reset(hdata);
- hdmiphy_conf_apply(hdata);
-
hdmi_start(hdata, false);
hdmi_conf_init(hdata);
-
hdmi_audio_init(hdata);
-
- /* setting core registers */
hdmi_mode_apply(hdata);
hdmi_audio_control(hdata, true);
-
- hdmi_regs_dump(hdata, "start");
}
static void hdmi_mode_set(struct drm_encoder *encoder,
@@ -1579,10 +1469,17 @@
hdata->cea_video_id = drm_match_cea_mode(mode);
}
-static void hdmi_enable(struct drm_encoder *encoder)
+static void hdmi_set_refclk(struct hdmi_context *hdata, bool on)
{
- struct hdmi_context *hdata = encoder_to_hdmi(encoder);
+ if (!hdata->sysreg)
+ return;
+ regmap_update_bits(hdata->sysreg, EXYNOS5433_SYSREG_DISP_HDMI_PHY,
+ SYSREG_HDMI_REFCLK_INT_CLK, on ? ~0 : 0);
+}
+
+static void hdmiphy_enable(struct hdmi_context *hdata)
+{
if (hdata->powered)
return;
@@ -1591,15 +1488,47 @@
if (regulator_bulk_enable(ARRAY_SIZE(supply), hdata->regul_bulk))
DRM_DEBUG_KMS("failed to enable regulator bulk\n");
- /* set pmu hdmiphy control bit to enable hdmiphy */
regmap_update_bits(hdata->pmureg, PMU_HDMI_PHY_CONTROL,
PMU_HDMI_PHY_ENABLE_BIT, 1);
- hdmi_conf_apply(hdata);
+ hdmi_set_refclk(hdata, true);
+
+ hdmi_reg_writemask(hdata, HDMI_PHY_CON_0, 0, HDMI_PHY_POWER_OFF_EN);
+
+ hdmiphy_conf_apply(hdata);
hdata->powered = true;
}
+static void hdmiphy_disable(struct hdmi_context *hdata)
+{
+ if (!hdata->powered)
+ return;
+
+ hdmi_reg_writemask(hdata, HDMI_CON_0, 0, HDMI_EN);
+
+ hdmi_reg_writemask(hdata, HDMI_PHY_CON_0, ~0, HDMI_PHY_POWER_OFF_EN);
+
+ hdmi_set_refclk(hdata, false);
+
+ regmap_update_bits(hdata->pmureg, PMU_HDMI_PHY_CONTROL,
+ PMU_HDMI_PHY_ENABLE_BIT, 0);
+
+ regulator_bulk_disable(ARRAY_SIZE(supply), hdata->regul_bulk);
+
+ pm_runtime_put_sync(hdata->dev);
+
+ hdata->powered = false;
+}
+
+static void hdmi_enable(struct drm_encoder *encoder)
+{
+ struct hdmi_context *hdata = encoder_to_hdmi(encoder);
+
+ hdmiphy_enable(hdata);
+ hdmi_conf_apply(hdata);
+}
+
static void hdmi_disable(struct drm_encoder *encoder)
{
struct hdmi_context *hdata = encoder_to_hdmi(encoder);
@@ -1623,20 +1552,9 @@
if (funcs && funcs->disable)
(*funcs->disable)(crtc);
- /* HDMI System Disable */
- hdmi_reg_writemask(hdata, HDMI_CON_0, 0, HDMI_EN);
-
cancel_delayed_work(&hdata->hotplug_work);
- /* reset pmu hdmiphy control bit to disable hdmiphy */
- regmap_update_bits(hdata->pmureg, PMU_HDMI_PHY_CONTROL,
- PMU_HDMI_PHY_ENABLE_BIT, 0);
-
- regulator_bulk_disable(ARRAY_SIZE(supply), hdata->regul_bulk);
-
- pm_runtime_put_sync(hdata->dev);
-
- hdata->powered = false;
+ hdmiphy_disable(hdata);
}
static const struct drm_encoder_helper_funcs exynos_hdmi_encoder_helper_funcs = {
@@ -1670,6 +1588,68 @@
return IRQ_HANDLED;
}
+static int hdmi_clks_get(struct hdmi_context *hdata,
+ const struct string_array_spec *names,
+ struct clk **clks)
+{
+ struct device *dev = hdata->dev;
+ int i;
+
+ for (i = 0; i < names->count; ++i) {
+ struct clk *clk = devm_clk_get(dev, names->data[i]);
+
+ if (IS_ERR(clk)) {
+ int ret = PTR_ERR(clk);
+
+ dev_err(dev, "Cannot get clock %s, %d\n",
+ names->data[i], ret);
+
+ return ret;
+ }
+
+ clks[i] = clk;
+ }
+
+ return 0;
+}
+
+static int hdmi_clk_init(struct hdmi_context *hdata)
+{
+ const struct hdmi_driver_data *drv_data = hdata->drv_data;
+ int count = drv_data->clk_gates.count + drv_data->clk_muxes.count;
+ struct device *dev = hdata->dev;
+ struct clk **clks;
+ int ret;
+
+ if (!count)
+ return 0;
+
+ clks = devm_kzalloc(dev, sizeof(*clks) * count, GFP_KERNEL);
+ if (!clks)
+ return -ENOMEM;
+
+ hdata->clk_gates = clks;
+ hdata->clk_muxes = clks + drv_data->clk_gates.count;
+
+ ret = hdmi_clks_get(hdata, &drv_data->clk_gates, hdata->clk_gates);
+ if (ret)
+ return ret;
+
+ return hdmi_clks_get(hdata, &drv_data->clk_muxes, hdata->clk_muxes);
+}
+
+
+static void hdmiphy_clk_enable(struct exynos_drm_clk *clk, bool enable)
+{
+ struct hdmi_context *hdata = container_of(clk, struct hdmi_context,
+ phy_clk);
+
+ if (enable)
+ hdmiphy_enable(hdata);
+ else
+ hdmiphy_disable(hdata);
+}
+
static int hdmi_resources_init(struct hdmi_context *hdata)
{
struct device *dev = hdata->dev;
@@ -1688,39 +1668,14 @@
DRM_ERROR("failed to get GPIO irq\n");
return hdata->irq;
}
- /* get clocks, power */
- hdata->hdmi = devm_clk_get(dev, "hdmi");
- if (IS_ERR(hdata->hdmi)) {
- DRM_ERROR("failed to get clock 'hdmi'\n");
- ret = PTR_ERR(hdata->hdmi);
- goto fail;
- }
- hdata->sclk_hdmi = devm_clk_get(dev, "sclk_hdmi");
- if (IS_ERR(hdata->sclk_hdmi)) {
- DRM_ERROR("failed to get clock 'sclk_hdmi'\n");
- ret = PTR_ERR(hdata->sclk_hdmi);
- goto fail;
- }
- hdata->sclk_pixel = devm_clk_get(dev, "sclk_pixel");
- if (IS_ERR(hdata->sclk_pixel)) {
- DRM_ERROR("failed to get clock 'sclk_pixel'\n");
- ret = PTR_ERR(hdata->sclk_pixel);
- goto fail;
- }
- hdata->sclk_hdmiphy = devm_clk_get(dev, "sclk_hdmiphy");
- if (IS_ERR(hdata->sclk_hdmiphy)) {
- DRM_ERROR("failed to get clock 'sclk_hdmiphy'\n");
- ret = PTR_ERR(hdata->sclk_hdmiphy);
- goto fail;
- }
- hdata->mout_hdmi = devm_clk_get(dev, "mout_hdmi");
- if (IS_ERR(hdata->mout_hdmi)) {
- DRM_ERROR("failed to get clock 'mout_hdmi'\n");
- ret = PTR_ERR(hdata->mout_hdmi);
- goto fail;
- }
- clk_set_parent(hdata->mout_hdmi, hdata->sclk_pixel);
+ ret = hdmi_clk_init(hdata);
+ if (ret)
+ return ret;
+
+ ret = hdmi_clk_set_parents(hdata, false);
+ if (ret)
+ return ret;
for (i = 0; i < ARRAY_SIZE(supply); ++i) {
hdata->regul_bulk[i].supply = supply[i];
@@ -1728,7 +1683,8 @@
}
ret = devm_regulator_bulk_get(dev, ARRAY_SIZE(supply), hdata->regul_bulk);
if (ret) {
- DRM_ERROR("failed to get regulators\n");
+ if (ret != -EPROBE_DEFER)
+ DRM_ERROR("failed to get regulators\n");
return ret;
}
@@ -1745,9 +1701,6 @@
DRM_ERROR("failed to enable hdmi-en regulator\n");
return ret;
-fail:
- DRM_ERROR("HDMI resource init - failed\n");
- return ret;
}
static struct of_device_id hdmi_match_types[] = {
@@ -1761,6 +1714,9 @@
.compatible = "samsung,exynos5420-hdmi",
.data = &exynos5420_hdmi_driver_data,
}, {
+ .compatible = "samsung,exynos5433-hdmi",
+ .data = &exynos5433_hdmi_driver_data,
+ }, {
/* end node */
}
};
@@ -1780,6 +1736,10 @@
if (pipe < 0)
return pipe;
+ hdata->phy_clk.enable = hdmiphy_clk_enable;
+
+ exynos_drm_crtc_from_pipe(drm_dev, pipe)->pipe_clk = &hdata->phy_clk;
+
encoder->possible_crtcs = 1 << pipe;
DRM_DEBUG_KMS("possible_crtcs = 0x%x\n", encoder->possible_crtcs);
@@ -1830,7 +1790,6 @@
static int hdmi_probe(struct platform_device *pdev)
{
struct device_node *ddc_node, *phy_node;
- const struct of_device_id *match;
struct device *dev = &pdev->dev;
struct hdmi_context *hdata;
struct resource *res;
@@ -1840,11 +1799,7 @@
if (!hdata)
return -ENOMEM;
- match = of_match_device(hdmi_match_types, dev);
- if (!match)
- return -ENODEV;
-
- hdata->drv_data = match->data;
+ hdata->drv_data = of_device_get_match_data(dev);
platform_set_drvdata(pdev, hdata);
@@ -1852,7 +1807,8 @@
ret = hdmi_resources_init(hdata);
if (ret) {
- DRM_ERROR("hdmi_resources_init failed\n");
+ if (ret != -EPROBE_DEFER)
+ DRM_ERROR("hdmi_resources_init failed\n");
return ret;
}
@@ -1867,7 +1823,6 @@
if (ddc_node)
goto out_get_ddc_adpt;
- /* DDC i2c driver */
ddc_node = of_parse_phandle(dev->of_node, "ddc", 0);
if (!ddc_node) {
DRM_ERROR("Failed to find ddc node in device tree\n");
@@ -1885,7 +1840,6 @@
if (phy_node)
goto out_get_phy_port;
- /* hdmiphy i2c driver */
phy_node = of_parse_phandle(dev->of_node, "phy", 0);
if (!phy_node) {
DRM_ERROR("Failed to find hdmiphy node in device tree\n");
@@ -1929,6 +1883,16 @@
goto err_hdmiphy;
}
+ if (hdata->drv_data->has_sysreg) {
+ hdata->sysreg = syscon_regmap_lookup_by_phandle(dev->of_node,
+ "samsung,sysreg-phandle");
+ if (IS_ERR(hdata->sysreg)) {
+ DRM_ERROR("sysreg regmap lookup failed.\n");
+ ret = -EPROBE_DEFER;
+ goto err_hdmiphy;
+ }
+ }
+
pm_runtime_enable(dev);
ret = component_add(&pdev->dev, &hdmi_component_ops);
@@ -1975,8 +1939,7 @@
{
struct hdmi_context *hdata = dev_get_drvdata(dev);
- clk_disable_unprepare(hdata->sclk_hdmi);
- clk_disable_unprepare(hdata->hdmi);
+ hdmi_clk_disable_gates(hdata);
return 0;
}
@@ -1986,17 +1949,9 @@
struct hdmi_context *hdata = dev_get_drvdata(dev);
int ret;
- ret = clk_prepare_enable(hdata->hdmi);
- if (ret < 0) {
- DRM_ERROR("Failed to prepare_enable the hdmi clk [%d]\n", ret);
+ ret = hdmi_clk_enable_gates(hdata);
+ if (ret < 0)
return ret;
- }
- ret = clk_prepare_enable(hdata->sclk_hdmi);
- if (ret < 0) {
- DRM_ERROR("Failed to prepare_enable the sclk_mixer clk [%d]\n",
- ret);
- return ret;
- }
return 0;
}
diff --git a/drivers/gpu/drm/exynos/exynos_mixer.c b/drivers/gpu/drm/exynos/exynos_mixer.c
index 0a5a600..74a4269 100644
--- a/drivers/gpu/drm/exynos/exynos_mixer.c
+++ b/drivers/gpu/drm/exynos/exynos_mixer.c
@@ -31,6 +31,7 @@
#include <linux/clk.h>
#include <linux/regulator/consumer.h>
#include <linux/of.h>
+#include <linux/of_device.h>
#include <linux/component.h>
#include <drm/exynos_drm.h>
@@ -103,8 +104,6 @@
struct mixer_resources mixer_res;
enum mixer_version_id mxr_ver;
- wait_queue_head_t wait_vsync_queue;
- atomic_t wait_vsync_event;
};
struct mixer_drv_data {
@@ -787,12 +786,6 @@
exynos_drm_crtc_finish_update(ctx->crtc, plane);
}
-
- /* set wait vsync event to zero and wake up queue. */
- if (atomic_read(&ctx->wait_vsync_event)) {
- atomic_set(&ctx->wait_vsync_event, 0);
- wake_up(&ctx->wait_vsync_queue);
- }
}
out:
@@ -1027,34 +1020,6 @@
mixer_vsync_set_update(mixer_ctx, true);
}
-static void mixer_wait_for_vblank(struct exynos_drm_crtc *crtc)
-{
- struct mixer_context *mixer_ctx = crtc->ctx;
- int err;
-
- if (!test_bit(MXR_BIT_POWERED, &mixer_ctx->flags))
- return;
-
- err = drm_vblank_get(mixer_ctx->drm_dev, mixer_ctx->pipe);
- if (err < 0) {
- DRM_DEBUG_KMS("failed to acquire vblank counter\n");
- return;
- }
-
- atomic_set(&mixer_ctx->wait_vsync_event, 1);
-
- /*
- * wait for MIXER to signal VSYNC interrupt or return after
- * timeout which is set to 50ms (refresh rate of 20).
- */
- if (!wait_event_timeout(mixer_ctx->wait_vsync_queue,
- !atomic_read(&mixer_ctx->wait_vsync_event),
- HZ/20))
- DRM_DEBUG_KMS("vblank wait timed out.\n");
-
- drm_vblank_put(mixer_ctx->drm_dev, mixer_ctx->pipe);
-}
-
static void mixer_enable(struct exynos_drm_crtc *crtc)
{
struct mixer_context *ctx = crtc->ctx;
@@ -1065,6 +1030,8 @@
pm_runtime_get_sync(ctx->dev);
+ exynos_drm_pipe_clk_enable(crtc, true);
+
mixer_vsync_set_update(ctx, false);
mixer_reg_writemask(res, MXR_STATUS, ~0, MXR_STATUS_SOFT_RESET);
@@ -1094,6 +1061,8 @@
for (i = 0; i < MIXER_WIN_NR; i++)
mixer_disable_plane(crtc, &ctx->planes[i]);
+ exynos_drm_pipe_clk_enable(crtc, false);
+
pm_runtime_put(ctx->dev);
clear_bit(MXR_BIT_POWERED, &ctx->flags);
@@ -1126,7 +1095,6 @@
.disable = mixer_disable,
.enable_vblank = mixer_enable_vblank,
.disable_vblank = mixer_disable_vblank,
- .wait_for_vblank = mixer_wait_for_vblank,
.atomic_begin = mixer_atomic_begin,
.update_plane = mixer_update_plane,
.disable_plane = mixer_disable_plane,
@@ -1155,18 +1123,6 @@
.has_sclk = 1,
};
-static const struct platform_device_id mixer_driver_types[] = {
- {
- .name = "s5p-mixer",
- .driver_data = (unsigned long)&exynos4210_mxr_drv_data,
- }, {
- .name = "exynos5-mixer",
- .driver_data = (unsigned long)&exynos5250_mxr_drv_data,
- }, {
- /* end node */
- }
-};
-
static struct of_device_id mixer_match_types[] = {
{
.compatible = "samsung,exynos4210-mixer",
@@ -1243,7 +1199,7 @@
static int mixer_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
- struct mixer_drv_data *drv;
+ const struct mixer_drv_data *drv;
struct mixer_context *ctx;
int ret;
@@ -1253,23 +1209,13 @@
return -ENOMEM;
}
- if (dev->of_node) {
- const struct of_device_id *match;
-
- match = of_match_node(mixer_match_types, dev->of_node);
- drv = (struct mixer_drv_data *)match->data;
- } else {
- drv = (struct mixer_drv_data *)
- platform_get_device_id(pdev)->driver_data;
- }
+ drv = of_device_get_match_data(dev);
ctx->pdev = pdev;
ctx->dev = dev;
ctx->vp_enabled = drv->is_vp_enabled;
ctx->has_sclk = drv->has_sclk;
ctx->mxr_ver = drv->version;
- init_waitqueue_head(&ctx->wait_vsync_queue);
- atomic_set(&ctx->wait_vsync_event, 0);
platform_set_drvdata(pdev, ctx);
@@ -1355,5 +1301,4 @@
},
.probe = mixer_probe,
.remove = mixer_remove,
- .id_table = mixer_driver_types,
};
diff --git a/drivers/gpu/drm/exynos/regs-hdmi.h b/drivers/gpu/drm/exynos/regs-hdmi.h
index 8c891e5..169667a 100644
--- a/drivers/gpu/drm/exynos/regs-hdmi.h
+++ b/drivers/gpu/drm/exynos/regs-hdmi.h
@@ -586,10 +586,12 @@
#define HDMI_TG_VACT_ST4_L HDMI_TG_BASE(0x0070)
#define HDMI_TG_VACT_ST4_H HDMI_TG_BASE(0x0074)
#define HDMI_TG_3D HDMI_TG_BASE(0x00F0)
+#define HDMI_TG_DECON_EN HDMI_TG_BASE(0x01e0)
/* HDMI PHY Registers Offsets*/
-#define HDMIPHY_POWER (0x74 >> 2)
-#define HDMIPHY_MODE_SET_DONE (0x7c >> 2)
+#define HDMIPHY_POWER 0x74
+#define HDMIPHY_MODE_SET_DONE 0x7c
+#define HDMIPHY5433_MODE_SET_DONE 0x84
/* HDMI PHY Values */
#define HDMI_PHY_POWER_ON 0x80
@@ -603,4 +605,7 @@
#define PMU_HDMI_PHY_CONTROL 0x700
#define PMU_HDMI_PHY_ENABLE_BIT BIT(0)
+#define EXYNOS5433_SYSREG_DISP_HDMI_PHY 0x1008
+#define SYSREG_HDMI_REFCLK_INT_CLK 1
+
#endif /* SAMSUNG_REGS_HDMI_H */
diff --git a/drivers/gpu/drm/fsl-dcu/Kconfig b/drivers/gpu/drm/fsl-dcu/Kconfig
index c78cf3f..b9c714d 100644
--- a/drivers/gpu/drm/fsl-dcu/Kconfig
+++ b/drivers/gpu/drm/fsl-dcu/Kconfig
@@ -1,6 +1,6 @@
config DRM_FSL_DCU
tristate "DRM Support for Freescale DCU"
- depends on DRM && OF && ARM
+ depends on DRM && OF && ARM && COMMON_CLK
select BACKLIGHT_CLASS_DEVICE
select BACKLIGHT_LCD_SUPPORT
select DRM_KMS_HELPER
diff --git a/drivers/gpu/drm/fsl-dcu/Makefile b/drivers/gpu/drm/fsl-dcu/Makefile
index 6ea1523..b35a292 100644
--- a/drivers/gpu/drm/fsl-dcu/Makefile
+++ b/drivers/gpu/drm/fsl-dcu/Makefile
@@ -3,5 +3,6 @@
fsl_dcu_drm_rgb.o \
fsl_dcu_drm_plane.o \
fsl_dcu_drm_crtc.o \
- fsl_dcu_drm_fbdev.o
+ fsl_dcu_drm_fbdev.o \
+ fsl_tcon.o
obj-$(CONFIG_DRM_FSL_DCU) += fsl-dcu-drm.o
diff --git a/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_crtc.c b/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_crtc.c
index 4ed7798..89c0084 100644
--- a/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_crtc.c
+++ b/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_crtc.c
@@ -66,13 +66,12 @@
{
struct drm_device *dev = crtc->dev;
struct fsl_dcu_drm_device *fsl_dev = dev->dev_private;
+ struct drm_connector *con = &fsl_dev->connector.base;
struct drm_display_mode *mode = &crtc->state->mode;
- unsigned int hbp, hfp, hsw, vbp, vfp, vsw, div, index, pol = 0;
- unsigned long dcuclk;
+ unsigned int hbp, hfp, hsw, vbp, vfp, vsw, index, pol = 0;
index = drm_crtc_index(crtc);
- dcuclk = clk_get_rate(fsl_dev->clk);
- div = dcuclk / mode->clock / 1000;
+ clk_set_rate(fsl_dev->pix_clk, mode->clock * 1000);
/* Configure timings: */
hbp = mode->htotal - mode->hsync_end;
@@ -82,6 +81,10 @@
vfp = mode->vsync_start - mode->vdisplay;
vsw = mode->vsync_end - mode->vsync_start;
+ /* INV_PXCK as default (most display sample data on rising edge) */
+ if (!(con->display_info.bus_flags & DRM_BUS_FLAG_PIXDATA_POSEDGE))
+ pol |= DCU_SYN_POL_INV_PXCK;
+
if (mode->flags & DRM_MODE_FLAG_NHSYNC)
pol |= DCU_SYN_POL_INV_HS_LOW;
@@ -99,7 +102,6 @@
regmap_write(fsl_dev->regmap, DCU_DISP_SIZE,
DCU_DISP_SIZE_DELTA_Y(mode->vdisplay) |
DCU_DISP_SIZE_DELTA_X(mode->hdisplay));
- regmap_write(fsl_dev->regmap, DCU_DIV_RATIO, div);
regmap_write(fsl_dev->regmap, DCU_SYN_POL, pol);
regmap_write(fsl_dev->regmap, DCU_BGND, DCU_BGND_R(0) |
DCU_BGND_G(0) | DCU_BGND_B(0));
diff --git a/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_drv.c b/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_drv.c
index e8d9337..0ec1ad9 100644
--- a/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_drv.c
+++ b/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_drv.c
@@ -23,10 +23,12 @@
#include <drm/drmP.h>
#include <drm/drm_crtc_helper.h>
+#include <drm/drm_fb_cma_helper.h>
#include <drm/drm_gem_cma_helper.h>
#include "fsl_dcu_drm_crtc.h"
#include "fsl_dcu_drm_drv.h"
+#include "fsl_tcon.h"
static bool fsl_dcu_drm_is_volatile_reg(struct device *dev, unsigned int reg)
{
@@ -62,46 +64,54 @@
return ret;
}
-static int fsl_dcu_load(struct drm_device *drm, unsigned long flags)
+static int fsl_dcu_load(struct drm_device *dev, unsigned long flags)
{
- struct device *dev = drm->dev;
- struct fsl_dcu_drm_device *fsl_dev = drm->dev_private;
+ struct fsl_dcu_drm_device *fsl_dev = dev->dev_private;
int ret;
ret = fsl_dcu_drm_modeset_init(fsl_dev);
if (ret < 0) {
- dev_err(dev, "failed to initialize mode setting\n");
+ dev_err(dev->dev, "failed to initialize mode setting\n");
return ret;
}
- ret = drm_vblank_init(drm, drm->mode_config.num_crtc);
+ ret = drm_vblank_init(dev, dev->mode_config.num_crtc);
if (ret < 0) {
- dev_err(dev, "failed to initialize vblank\n");
+ dev_err(dev->dev, "failed to initialize vblank\n");
goto done;
}
- drm->vblank_disable_allowed = true;
- ret = fsl_dcu_drm_irq_init(drm);
+ ret = fsl_dcu_drm_irq_init(dev);
if (ret < 0)
goto done;
- drm->irq_enabled = true;
+ dev->irq_enabled = true;
- fsl_dcu_fbdev_init(drm);
+ fsl_dcu_fbdev_init(dev);
return 0;
done:
- if (ret) {
- drm_mode_config_cleanup(drm);
- drm_vblank_cleanup(drm);
- drm_irq_uninstall(drm);
- drm->dev_private = NULL;
- }
+ drm_kms_helper_poll_fini(dev);
+
+ if (fsl_dev->fbdev)
+ drm_fbdev_cma_fini(fsl_dev->fbdev);
+
+ drm_mode_config_cleanup(dev);
+ drm_vblank_cleanup(dev);
+ drm_irq_uninstall(dev);
+ dev->dev_private = NULL;
return ret;
}
static int fsl_dcu_unload(struct drm_device *dev)
{
+ struct fsl_dcu_drm_device *fsl_dev = dev->dev_private;
+
+ drm_kms_helper_poll_fini(dev);
+
+ if (fsl_dev->fbdev)
+ drm_fbdev_cma_fini(fsl_dev->fbdev);
+
drm_mode_config_cleanup(dev);
drm_vblank_cleanup(dev);
drm_irq_uninstall(dev);
@@ -157,6 +167,13 @@
regmap_write(fsl_dev->regmap, DCU_INT_MASK, value);
}
+static void fsl_dcu_drm_lastclose(struct drm_device *dev)
+{
+ struct fsl_dcu_drm_device *fsl_dev = dev->dev_private;
+
+ drm_fbdev_cma_restore_mode(fsl_dev->fbdev);
+}
+
static const struct file_operations fsl_dcu_drm_fops = {
.owner = THIS_MODULE,
.open = drm_open,
@@ -174,6 +191,7 @@
static struct drm_driver fsl_dcu_drm_driver = {
.driver_features = DRIVER_HAVE_IRQ | DRIVER_GEM | DRIVER_MODESET
| DRIVER_PRIME | DRIVER_ATOMIC,
+ .lastclose = fsl_dcu_drm_lastclose,
.load = fsl_dcu_load,
.unload = fsl_dcu_unload,
.irq_handler = fsl_dcu_drm_irq,
@@ -197,9 +215,9 @@
.fops = &fsl_dcu_drm_fops,
.name = "fsl-dcu-drm",
.desc = "Freescale DCU DRM",
- .date = "20150213",
+ .date = "20160425",
.major = 1,
- .minor = 0,
+ .minor = 1,
};
#ifdef CONFIG_PM_SLEEP
@@ -283,6 +301,9 @@
struct resource *res;
void __iomem *base;
struct drm_driver *driver = &fsl_dcu_drm_driver;
+ struct clk *pix_clk_in;
+ char pix_clk_name[32];
+ const char *pix_clk_in_name;
const struct of_device_id *id;
int ret;
@@ -290,6 +311,11 @@
if (!fsl_dev)
return -ENOMEM;
+ id = of_match_node(fsl_dcu_of_match, pdev->dev.of_node);
+ if (!id)
+ return -ENODEV;
+ fsl_dev->soc = id->data;
+
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
if (!res) {
dev_err(dev, "could not get memory IO resource\n");
@@ -308,24 +334,6 @@
return -ENXIO;
}
- fsl_dev->clk = devm_clk_get(dev, "dcu");
- if (IS_ERR(fsl_dev->clk)) {
- ret = PTR_ERR(fsl_dev->clk);
- dev_err(dev, "failed to get dcu clock\n");
- return ret;
- }
- ret = clk_prepare(fsl_dev->clk);
- if (ret < 0) {
- dev_err(dev, "failed to prepare dcu clk\n");
- return ret;
- }
- ret = clk_enable(fsl_dev->clk);
- if (ret < 0) {
- dev_err(dev, "failed to enable dcu clk\n");
- clk_unprepare(fsl_dev->clk);
- return ret;
- }
-
fsl_dev->regmap = devm_regmap_init_mmio(dev, base,
&fsl_dcu_regmap_config);
if (IS_ERR(fsl_dev->regmap)) {
@@ -333,14 +341,47 @@
return PTR_ERR(fsl_dev->regmap);
}
- id = of_match_node(fsl_dcu_of_match, pdev->dev.of_node);
- if (!id)
- return -ENODEV;
- fsl_dev->soc = id->data;
+ fsl_dev->clk = devm_clk_get(dev, "dcu");
+ if (IS_ERR(fsl_dev->clk)) {
+ dev_err(dev, "failed to get dcu clock\n");
+ return PTR_ERR(fsl_dev->clk);
+ }
+ ret = clk_prepare_enable(fsl_dev->clk);
+ if (ret < 0) {
+ dev_err(dev, "failed to enable dcu clk\n");
+ return ret;
+ }
+
+ pix_clk_in = devm_clk_get(dev, "pix");
+ if (IS_ERR(pix_clk_in)) {
+ /* legancy binding, use dcu clock as pixel clock input */
+ pix_clk_in = fsl_dev->clk;
+ }
+
+ pix_clk_in_name = __clk_get_name(pix_clk_in);
+ snprintf(pix_clk_name, sizeof(pix_clk_name), "%s_pix", pix_clk_in_name);
+ fsl_dev->pix_clk = clk_register_divider(dev, pix_clk_name,
+ pix_clk_in_name, 0, base + DCU_DIV_RATIO,
+ 0, 8, CLK_DIVIDER_ROUND_CLOSEST, NULL);
+ if (IS_ERR(fsl_dev->pix_clk)) {
+ dev_err(dev, "failed to register pix clk\n");
+ ret = PTR_ERR(fsl_dev->pix_clk);
+ goto disable_clk;
+ }
+
+ ret = clk_prepare_enable(fsl_dev->pix_clk);
+ if (ret < 0) {
+ dev_err(dev, "failed to enable pix clk\n");
+ goto unregister_pix_clk;
+ }
+
+ fsl_dev->tcon = fsl_tcon_init(dev);
drm = drm_dev_alloc(driver, dev);
- if (!drm)
- return -ENOMEM;
+ if (!drm) {
+ ret = -ENOMEM;
+ goto disable_pix_clk;
+ }
fsl_dev->dev = dev;
fsl_dev->drm = drm;
@@ -360,6 +401,12 @@
unref:
drm_dev_unref(drm);
+disable_pix_clk:
+ clk_disable_unprepare(fsl_dev->pix_clk);
+unregister_pix_clk:
+ clk_unregister(fsl_dev->pix_clk);
+disable_clk:
+ clk_disable_unprepare(fsl_dev->clk);
return ret;
}
@@ -367,6 +414,9 @@
{
struct fsl_dcu_drm_device *fsl_dev = platform_get_drvdata(pdev);
+ clk_disable_unprepare(fsl_dev->clk);
+ clk_disable_unprepare(fsl_dev->pix_clk);
+ clk_unregister(fsl_dev->pix_clk);
drm_put_dev(fsl_dev->drm);
return 0;
diff --git a/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_drv.h b/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_drv.h
index 6413ac9..c275f90 100644
--- a/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_drv.h
+++ b/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_drv.h
@@ -47,8 +47,8 @@
#define DCU_VSYN_PARA_FP(x) (x)
#define DCU_SYN_POL 0x0024
-#define DCU_SYN_POL_INV_PXCK_FALL (0 << 6)
-#define DCU_SYN_POL_NEG_REMAIN (0 << 5)
+#define DCU_SYN_POL_INV_PXCK BIT(6)
+#define DCU_SYN_POL_NEG BIT(5)
#define DCU_SYN_POL_INV_VS_LOW BIT(1)
#define DCU_SYN_POL_INV_HS_LOW BIT(0)
@@ -183,6 +183,8 @@
struct regmap *regmap;
int irq;
struct clk *clk;
+ struct clk *pix_clk;
+ struct fsl_tcon *tcon;
/*protects hardware register*/
spinlock_t irq_lock;
struct drm_device *drm;
diff --git a/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_rgb.c b/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_rgb.c
index 8780deb..98c998d 100644
--- a/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_rgb.c
+++ b/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_rgb.c
@@ -17,6 +17,7 @@
#include <drm/drm_panel.h>
#include "fsl_dcu_drm_drv.h"
+#include "fsl_tcon.h"
static int
fsl_dcu_drm_encoder_atomic_check(struct drm_encoder *encoder,
@@ -28,10 +29,20 @@
static void fsl_dcu_drm_encoder_disable(struct drm_encoder *encoder)
{
+ struct drm_device *dev = encoder->dev;
+ struct fsl_dcu_drm_device *fsl_dev = dev->dev_private;
+
+ if (fsl_dev->tcon)
+ fsl_tcon_bypass_disable(fsl_dev->tcon);
}
static void fsl_dcu_drm_encoder_enable(struct drm_encoder *encoder)
{
+ struct drm_device *dev = encoder->dev;
+ struct fsl_dcu_drm_device *fsl_dev = dev->dev_private;
+
+ if (fsl_dev->tcon)
+ fsl_tcon_bypass_enable(fsl_dev->tcon);
}
static const struct drm_encoder_helper_funcs encoder_helper_funcs = {
@@ -68,7 +79,10 @@
static void fsl_dcu_drm_connector_destroy(struct drm_connector *connector)
{
+ struct fsl_dcu_drm_connector *fsl_con = to_fsl_dcu_connector(connector);
+
drm_connector_unregister(connector);
+ drm_panel_detach(fsl_con->panel);
drm_connector_cleanup(connector);
}
@@ -131,7 +145,7 @@
struct drm_encoder *encoder)
{
struct drm_connector *connector = &fsl_dev->connector.base;
- struct drm_mode_config mode_config = fsl_dev->drm->mode_config;
+ struct drm_mode_config *mode_config = &fsl_dev->drm->mode_config;
struct device_node *panel_node;
int ret;
@@ -153,19 +167,23 @@
goto err_sysfs;
drm_object_property_set_value(&connector->base,
- mode_config.dpms_property,
+ mode_config->dpms_property,
DRM_MODE_DPMS_OFF);
panel_node = of_parse_phandle(fsl_dev->np, "fsl,panel", 0);
- if (panel_node) {
- fsl_dev->connector.panel = of_drm_find_panel(panel_node);
- if (!fsl_dev->connector.panel) {
- ret = -EPROBE_DEFER;
- goto err_sysfs;
- }
- of_node_put(panel_node);
+ if (!panel_node) {
+ dev_err(fsl_dev->dev, "fsl,panel property not found\n");
+ ret = -ENODEV;
+ goto err_sysfs;
}
+ fsl_dev->connector.panel = of_drm_find_panel(panel_node);
+ if (!fsl_dev->connector.panel) {
+ ret = -EPROBE_DEFER;
+ goto err_panel;
+ }
+ of_node_put(panel_node);
+
ret = drm_panel_attach(fsl_dev->connector.panel, connector);
if (ret) {
dev_err(fsl_dev->dev, "failed to attach panel\n");
@@ -174,6 +192,8 @@
return 0;
+err_panel:
+ of_node_put(panel_node);
err_sysfs:
drm_connector_unregister(connector);
err_cleanup:
diff --git a/drivers/gpu/drm/fsl-dcu/fsl_tcon.c b/drivers/gpu/drm/fsl-dcu/fsl_tcon.c
new file mode 100644
index 0000000..bbe34f1
--- /dev/null
+++ b/drivers/gpu/drm/fsl-dcu/fsl_tcon.c
@@ -0,0 +1,111 @@
+/*
+ * Copyright 2015 Toradex AG
+ *
+ * Stefan Agner <stefan@agner.ch>
+ *
+ * Freescale TCON device driver
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+
+#include <linux/clk.h>
+#include <linux/io.h>
+#include <linux/mm.h>
+#include <linux/of_address.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+
+#include "fsl_tcon.h"
+
+void fsl_tcon_bypass_disable(struct fsl_tcon *tcon)
+{
+ regmap_update_bits(tcon->regs, FSL_TCON_CTRL1,
+ FSL_TCON_CTRL1_TCON_BYPASS, 0);
+}
+
+void fsl_tcon_bypass_enable(struct fsl_tcon *tcon)
+{
+ regmap_update_bits(tcon->regs, FSL_TCON_CTRL1,
+ FSL_TCON_CTRL1_TCON_BYPASS,
+ FSL_TCON_CTRL1_TCON_BYPASS);
+}
+
+static struct regmap_config fsl_tcon_regmap_config = {
+ .reg_bits = 32,
+ .reg_stride = 4,
+ .val_bits = 32,
+
+ .name = "tcon",
+};
+
+static int fsl_tcon_init_regmap(struct device *dev,
+ struct fsl_tcon *tcon,
+ struct device_node *np)
+{
+ struct resource res;
+ void __iomem *regs;
+
+ if (of_address_to_resource(np, 0, &res))
+ return -EINVAL;
+
+ regs = devm_ioremap_resource(dev, &res);
+ if (IS_ERR(regs))
+ return PTR_ERR(regs);
+
+ tcon->regs = devm_regmap_init_mmio(dev, regs,
+ &fsl_tcon_regmap_config);
+ if (IS_ERR(tcon->regs))
+ return PTR_ERR(tcon->regs);
+
+ return 0;
+}
+
+struct fsl_tcon *fsl_tcon_init(struct device *dev)
+{
+ struct fsl_tcon *tcon;
+ struct device_node *np;
+ int ret;
+
+ /* TCON node is not mandatory, some devices do not provide TCON */
+ np = of_parse_phandle(dev->of_node, "fsl,tcon", 0);
+ if (!np)
+ return NULL;
+
+ tcon = devm_kzalloc(dev, sizeof(*tcon), GFP_KERNEL);
+ if (!tcon) {
+ ret = -ENOMEM;
+ goto err_node_put;
+ }
+
+ ret = fsl_tcon_init_regmap(dev, tcon, np);
+ if (ret) {
+ dev_err(dev, "Couldn't create the TCON regmap\n");
+ goto err_node_put;
+ }
+
+ tcon->ipg_clk = of_clk_get_by_name(np, "ipg");
+ if (IS_ERR(tcon->ipg_clk)) {
+ dev_err(dev, "Couldn't get the TCON bus clock\n");
+ goto err_node_put;
+ }
+
+ clk_prepare_enable(tcon->ipg_clk);
+
+ dev_info(dev, "Using TCON in bypass mode\n");
+
+ return tcon;
+
+err_node_put:
+ of_node_put(np);
+ return NULL;
+}
+
+void fsl_tcon_free(struct fsl_tcon *tcon)
+{
+ clk_disable_unprepare(tcon->ipg_clk);
+ clk_put(tcon->ipg_clk);
+}
+
diff --git a/drivers/gpu/drm/fsl-dcu/fsl_tcon.h b/drivers/gpu/drm/fsl-dcu/fsl_tcon.h
new file mode 100644
index 0000000..80a7617
--- /dev/null
+++ b/drivers/gpu/drm/fsl-dcu/fsl_tcon.h
@@ -0,0 +1,33 @@
+/*
+ * Copyright 2015 Toradex AG
+ *
+ * Stefan Agner <stefan@agner.ch>
+ *
+ * Freescale TCON device driver
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+
+#ifndef __FSL_TCON_H__
+#define __FSL_TCON_H__
+
+#include <linux/bitops.h>
+
+#define FSL_TCON_CTRL1 0x0
+#define FSL_TCON_CTRL1_TCON_BYPASS BIT(29)
+
+struct fsl_tcon {
+ struct regmap *regs;
+ struct clk *ipg_clk;
+};
+
+struct fsl_tcon *fsl_tcon_init(struct device *dev);
+void fsl_tcon_free(struct fsl_tcon *tcon);
+
+void fsl_tcon_bypass_disable(struct fsl_tcon *tcon);
+void fsl_tcon_bypass_enable(struct fsl_tcon *tcon);
+
+#endif /* __FSL_TCON_H__ */
diff --git a/drivers/gpu/drm/gma500/framebuffer.c b/drivers/gpu/drm/gma500/framebuffer.c
index 033d894..7440bf9 100644
--- a/drivers/gpu/drm/gma500/framebuffer.c
+++ b/drivers/gpu/drm/gma500/framebuffer.c
@@ -411,7 +411,7 @@
info = drm_fb_helper_alloc_fbi(&fbdev->psb_fb_helper);
if (IS_ERR(info)) {
ret = PTR_ERR(info);
- goto out_err1;
+ goto err_free_range;
}
info->par = fbdev;
@@ -419,7 +419,7 @@
ret = psb_framebuffer_init(dev, psbfb, &mode_cmd, backing);
if (ret)
- goto out_unref;
+ goto err_release;
fb = &psbfb->base;
psbfb->fbdev = info;
@@ -464,14 +464,9 @@
psbfb->base.width, psbfb->base.height);
return 0;
-out_unref:
- if (backing->stolen)
- psb_gtt_free_range(dev, backing);
- else
- drm_gem_object_unreference_unlocked(&backing->gem);
-
+err_release:
drm_fb_helper_release_fbi(&fbdev->psb_fb_helper);
-out_err1:
+err_free_range:
psb_gtt_free_range(dev, backing);
return ret;
}
@@ -495,7 +490,7 @@
* Find the GEM object and thus the gtt range object that is
* to back this space
*/
- obj = drm_gem_object_lookup(dev, filp, cmd->handles[0]);
+ obj = drm_gem_object_lookup(filp, cmd->handles[0]);
if (obj == NULL)
return ERR_PTR(-ENOENT);
diff --git a/drivers/gpu/drm/gma500/gem.c b/drivers/gpu/drm/gma500/gem.c
index 506224b3..6d1cb6b 100644
--- a/drivers/gpu/drm/gma500/gem.c
+++ b/drivers/gpu/drm/gma500/gem.c
@@ -63,7 +63,7 @@
struct drm_gem_object *obj;
/* GEM does all our handle to object mapping */
- obj = drm_gem_object_lookup(dev, file, handle);
+ obj = drm_gem_object_lookup(file, handle);
if (obj == NULL)
return -ENOENT;
diff --git a/drivers/gpu/drm/gma500/gma_display.c b/drivers/gpu/drm/gma500/gma_display.c
index 5bf765d..c95406e 100644
--- a/drivers/gpu/drm/gma500/gma_display.c
+++ b/drivers/gpu/drm/gma500/gma_display.c
@@ -372,7 +372,7 @@
return -EINVAL;
}
- obj = drm_gem_object_lookup(dev, file_priv, handle);
+ obj = drm_gem_object_lookup(file_priv, handle);
if (!obj) {
ret = -ENOENT;
goto unlock;
diff --git a/drivers/gpu/drm/gma500/mdfld_dsi_dpi.c b/drivers/gpu/drm/gma500/mdfld_dsi_dpi.c
index 7cd87a0..a05c0206 100644
--- a/drivers/gpu/drm/gma500/mdfld_dsi_dpi.c
+++ b/drivers/gpu/drm/gma500/mdfld_dsi_dpi.c
@@ -979,11 +979,7 @@
return NULL;
}
- if (dsi_connector->pipe)
- dpi_output->panel_on = 0;
- else
- dpi_output->panel_on = 0;
-
+ dpi_output->panel_on = 0;
dpi_output->dev = dev;
if (mdfld_get_panel_type(dev, pipe) != TC35876X)
dpi_output->p_funcs = p_funcs;
diff --git a/drivers/gpu/drm/gma500/mdfld_dsi_pkg_sender.c b/drivers/gpu/drm/gma500/mdfld_dsi_pkg_sender.c
index 6b43ae3..1616af2 100644
--- a/drivers/gpu/drm/gma500/mdfld_dsi_pkg_sender.c
+++ b/drivers/gpu/drm/gma500/mdfld_dsi_pkg_sender.c
@@ -72,7 +72,7 @@
"RX Prot Violation",
"HS Generic Write FIFO Full",
"LP Generic Write FIFO Full",
- "Generic Read Data Avail"
+ "Generic Read Data Avail",
"Special Packet Sent",
"Tearing Effect",
};
diff --git a/drivers/gpu/drm/gma500/psb_drv.c b/drivers/gpu/drm/gma500/psb_drv.c
index 4e1c685..82b8ce4 100644
--- a/drivers/gpu/drm/gma500/psb_drv.c
+++ b/drivers/gpu/drm/gma500/psb_drv.c
@@ -374,7 +374,6 @@
drm_irq_install(dev, dev->pdev->irq);
- dev->vblank_disable_allowed = true;
dev->max_vblank_count = 0xffffff; /* only 24 bits of frame count */
dev->driver->get_vblank_counter = psb_get_vblank_counter;
diff --git a/drivers/gpu/drm/hisilicon/Kconfig b/drivers/gpu/drm/hisilicon/Kconfig
new file mode 100644
index 0000000..558c61b
--- /dev/null
+++ b/drivers/gpu/drm/hisilicon/Kconfig
@@ -0,0 +1,5 @@
+#
+# hisilicon drm device configuration.
+# Please keep this list sorted alphabetically
+
+source "drivers/gpu/drm/hisilicon/kirin/Kconfig"
diff --git a/drivers/gpu/drm/hisilicon/Makefile b/drivers/gpu/drm/hisilicon/Makefile
new file mode 100644
index 0000000..e3f6d49
--- /dev/null
+++ b/drivers/gpu/drm/hisilicon/Makefile
@@ -0,0 +1,5 @@
+#
+# Makefile for hisilicon drm drivers.
+# Please keep this list sorted alphabetically
+
+obj-$(CONFIG_DRM_HISI_KIRIN) += kirin/
diff --git a/drivers/gpu/drm/hisilicon/kirin/Kconfig b/drivers/gpu/drm/hisilicon/kirin/Kconfig
new file mode 100644
index 0000000..ea0df61
--- /dev/null
+++ b/drivers/gpu/drm/hisilicon/kirin/Kconfig
@@ -0,0 +1,18 @@
+config DRM_HISI_KIRIN
+ tristate "DRM Support for Hisilicon Kirin series SoCs Platform"
+ depends on DRM && OF && ARM64
+ select DRM_KMS_HELPER
+ select DRM_GEM_CMA_HELPER
+ select DRM_KMS_CMA_HELPER
+ help
+ Choose this option if you have a hisilicon Kirin chipsets(hi6220).
+ If M is selected the module will be called kirin-drm.
+
+config HISI_KIRIN_DW_DSI
+ tristate "HiSilicon Kirin specific extensions for Synopsys DW MIPI DSI"
+ depends on DRM_HISI_KIRIN
+ select DRM_MIPI_DSI
+ help
+ This selects support for HiSilicon Kirin SoC specific extensions for
+ the Synopsys DesignWare DSI driver. If you want to enable MIPI DSI on
+ hi6220 based SoC, you should selet this option.
diff --git a/drivers/gpu/drm/hisilicon/kirin/Makefile b/drivers/gpu/drm/hisilicon/kirin/Makefile
new file mode 100644
index 0000000..cdf6158
--- /dev/null
+++ b/drivers/gpu/drm/hisilicon/kirin/Makefile
@@ -0,0 +1,6 @@
+kirin-drm-y := kirin_drm_drv.o \
+ kirin_drm_ade.o
+
+obj-$(CONFIG_DRM_HISI_KIRIN) += kirin-drm.o
+
+obj-$(CONFIG_HISI_KIRIN_DW_DSI) += dw_drm_dsi.o
diff --git a/drivers/gpu/drm/hisilicon/kirin/dw_drm_dsi.c b/drivers/gpu/drm/hisilicon/kirin/dw_drm_dsi.c
new file mode 100644
index 0000000..998452a
--- /dev/null
+++ b/drivers/gpu/drm/hisilicon/kirin/dw_drm_dsi.c
@@ -0,0 +1,858 @@
+/*
+ * DesignWare MIPI DSI Host Controller v1.02 driver
+ *
+ * Copyright (c) 2016 Linaro Limited.
+ * Copyright (c) 2014-2016 Hisilicon Limited.
+ *
+ * Author:
+ * Xinliang Liu <z.liuxinliang@hisilicon.com>
+ * Xinliang Liu <xinliang.liu@linaro.org>
+ * Xinwei Kong <kong.kongxinwei@hisilicon.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#include <linux/clk.h>
+#include <linux/component.h>
+#include <linux/of_graph.h>
+
+#include <drm/drm_of.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_mipi_dsi.h>
+#include <drm/drm_encoder_slave.h>
+#include <drm/drm_atomic_helper.h>
+
+#include "dw_dsi_reg.h"
+
+#define MAX_TX_ESC_CLK 10
+#define ROUND(x, y) ((x) / (y) + \
+ ((x) % (y) * 10 / (y) >= 5 ? 1 : 0))
+#define PHY_REF_CLK_RATE 19200000
+#define PHY_REF_CLK_PERIOD_PS (1000000000 / (PHY_REF_CLK_RATE / 1000))
+
+#define encoder_to_dsi(encoder) \
+ container_of(encoder, struct dw_dsi, encoder)
+#define host_to_dsi(host) \
+ container_of(host, struct dw_dsi, host)
+
+struct mipi_phy_params {
+ u32 clk_t_lpx;
+ u32 clk_t_hs_prepare;
+ u32 clk_t_hs_zero;
+ u32 clk_t_hs_trial;
+ u32 clk_t_wakeup;
+ u32 data_t_lpx;
+ u32 data_t_hs_prepare;
+ u32 data_t_hs_zero;
+ u32 data_t_hs_trial;
+ u32 data_t_ta_go;
+ u32 data_t_ta_get;
+ u32 data_t_wakeup;
+ u32 hstx_ckg_sel;
+ u32 pll_fbd_div5f;
+ u32 pll_fbd_div1f;
+ u32 pll_fbd_2p;
+ u32 pll_enbwt;
+ u32 pll_fbd_p;
+ u32 pll_fbd_s;
+ u32 pll_pre_div1p;
+ u32 pll_pre_p;
+ u32 pll_vco_750M;
+ u32 pll_lpf_rs;
+ u32 pll_lpf_cs;
+ u32 clklp2hs_time;
+ u32 clkhs2lp_time;
+ u32 lp2hs_time;
+ u32 hs2lp_time;
+ u32 clk_to_data_delay;
+ u32 data_to_clk_delay;
+ u32 lane_byte_clk_kHz;
+ u32 clk_division;
+};
+
+struct dsi_hw_ctx {
+ void __iomem *base;
+ struct clk *pclk;
+};
+
+struct dw_dsi {
+ struct drm_encoder encoder;
+ struct drm_bridge *bridge;
+ struct mipi_dsi_host host;
+ struct drm_display_mode cur_mode;
+ struct dsi_hw_ctx *ctx;
+ struct mipi_phy_params phy;
+
+ u32 lanes;
+ enum mipi_dsi_pixel_format format;
+ unsigned long mode_flags;
+ bool enable;
+};
+
+struct dsi_data {
+ struct dw_dsi dsi;
+ struct dsi_hw_ctx ctx;
+};
+
+struct dsi_phy_range {
+ u32 min_range_kHz;
+ u32 max_range_kHz;
+ u32 pll_vco_750M;
+ u32 hstx_ckg_sel;
+};
+
+static const struct dsi_phy_range dphy_range_info[] = {
+ { 46875, 62500, 1, 7 },
+ { 62500, 93750, 0, 7 },
+ { 93750, 125000, 1, 6 },
+ { 125000, 187500, 0, 6 },
+ { 187500, 250000, 1, 5 },
+ { 250000, 375000, 0, 5 },
+ { 375000, 500000, 1, 4 },
+ { 500000, 750000, 0, 4 },
+ { 750000, 1000000, 1, 0 },
+ { 1000000, 1500000, 0, 0 }
+};
+
+static u32 dsi_calc_phy_rate(u32 req_kHz, struct mipi_phy_params *phy)
+{
+ u32 ref_clk_ps = PHY_REF_CLK_PERIOD_PS;
+ u32 tmp_kHz = req_kHz;
+ u32 i = 0;
+ u32 q_pll = 1;
+ u32 m_pll = 0;
+ u32 n_pll = 0;
+ u32 r_pll = 1;
+ u32 m_n = 0;
+ u32 m_n_int = 0;
+ u32 f_kHz = 0;
+ u64 temp;
+
+ /*
+ * Find a rate >= req_kHz.
+ */
+ do {
+ f_kHz = tmp_kHz;
+
+ for (i = 0; i < ARRAY_SIZE(dphy_range_info); i++)
+ if (f_kHz >= dphy_range_info[i].min_range_kHz &&
+ f_kHz <= dphy_range_info[i].max_range_kHz)
+ break;
+
+ if (i == ARRAY_SIZE(dphy_range_info)) {
+ DRM_ERROR("%dkHz out of range\n", f_kHz);
+ return 0;
+ }
+
+ phy->pll_vco_750M = dphy_range_info[i].pll_vco_750M;
+ phy->hstx_ckg_sel = dphy_range_info[i].hstx_ckg_sel;
+
+ if (phy->hstx_ckg_sel <= 7 &&
+ phy->hstx_ckg_sel >= 4)
+ q_pll = 0x10 >> (7 - phy->hstx_ckg_sel);
+
+ temp = f_kHz * (u64)q_pll * (u64)ref_clk_ps;
+ m_n_int = temp / (u64)1000000000;
+ m_n = (temp % (u64)1000000000) / (u64)100000000;
+
+ if (m_n_int % 2 == 0) {
+ if (m_n * 6 >= 50) {
+ n_pll = 2;
+ m_pll = (m_n_int + 1) * n_pll;
+ } else if (m_n * 6 >= 30) {
+ n_pll = 3;
+ m_pll = m_n_int * n_pll + 2;
+ } else {
+ n_pll = 1;
+ m_pll = m_n_int * n_pll;
+ }
+ } else {
+ if (m_n * 6 >= 50) {
+ n_pll = 1;
+ m_pll = (m_n_int + 1) * n_pll;
+ } else if (m_n * 6 >= 30) {
+ n_pll = 1;
+ m_pll = (m_n_int + 1) * n_pll;
+ } else if (m_n * 6 >= 10) {
+ n_pll = 3;
+ m_pll = m_n_int * n_pll + 1;
+ } else {
+ n_pll = 2;
+ m_pll = m_n_int * n_pll;
+ }
+ }
+
+ if (n_pll == 1) {
+ phy->pll_fbd_p = 0;
+ phy->pll_pre_div1p = 1;
+ } else {
+ phy->pll_fbd_p = n_pll;
+ phy->pll_pre_div1p = 0;
+ }
+
+ if (phy->pll_fbd_2p <= 7 && phy->pll_fbd_2p >= 4)
+ r_pll = 0x10 >> (7 - phy->pll_fbd_2p);
+
+ if (m_pll == 2) {
+ phy->pll_pre_p = 0;
+ phy->pll_fbd_s = 0;
+ phy->pll_fbd_div1f = 0;
+ phy->pll_fbd_div5f = 1;
+ } else if (m_pll >= 2 * 2 * r_pll && m_pll <= 2 * 4 * r_pll) {
+ phy->pll_pre_p = m_pll / (2 * r_pll);
+ phy->pll_fbd_s = 0;
+ phy->pll_fbd_div1f = 1;
+ phy->pll_fbd_div5f = 0;
+ } else if (m_pll >= 2 * 5 * r_pll && m_pll <= 2 * 150 * r_pll) {
+ if (((m_pll / (2 * r_pll)) % 2) == 0) {
+ phy->pll_pre_p =
+ (m_pll / (2 * r_pll)) / 2 - 1;
+ phy->pll_fbd_s =
+ (m_pll / (2 * r_pll)) % 2 + 2;
+ } else {
+ phy->pll_pre_p =
+ (m_pll / (2 * r_pll)) / 2;
+ phy->pll_fbd_s =
+ (m_pll / (2 * r_pll)) % 2;
+ }
+ phy->pll_fbd_div1f = 0;
+ phy->pll_fbd_div5f = 0;
+ } else {
+ phy->pll_pre_p = 0;
+ phy->pll_fbd_s = 0;
+ phy->pll_fbd_div1f = 0;
+ phy->pll_fbd_div5f = 1;
+ }
+
+ f_kHz = (u64)1000000000 * (u64)m_pll /
+ ((u64)ref_clk_ps * (u64)n_pll * (u64)q_pll);
+
+ if (f_kHz >= req_kHz)
+ break;
+
+ tmp_kHz += 10;
+
+ } while (true);
+
+ return f_kHz;
+}
+
+static void dsi_get_phy_params(u32 phy_req_kHz,
+ struct mipi_phy_params *phy)
+{
+ u32 ref_clk_ps = PHY_REF_CLK_PERIOD_PS;
+ u32 phy_rate_kHz;
+ u32 ui;
+
+ memset(phy, 0, sizeof(*phy));
+
+ phy_rate_kHz = dsi_calc_phy_rate(phy_req_kHz, phy);
+ if (!phy_rate_kHz)
+ return;
+
+ ui = 1000000 / phy_rate_kHz;
+
+ phy->clk_t_lpx = ROUND(50, 8 * ui);
+ phy->clk_t_hs_prepare = ROUND(133, 16 * ui) - 1;
+
+ phy->clk_t_hs_zero = ROUND(262, 8 * ui);
+ phy->clk_t_hs_trial = 2 * (ROUND(60, 8 * ui) - 1);
+ phy->clk_t_wakeup = ROUND(1000000, (ref_clk_ps / 1000) - 1);
+ if (phy->clk_t_wakeup > 0xff)
+ phy->clk_t_wakeup = 0xff;
+ phy->data_t_wakeup = phy->clk_t_wakeup;
+ phy->data_t_lpx = phy->clk_t_lpx;
+ phy->data_t_hs_prepare = ROUND(125 + 10 * ui, 16 * ui) - 1;
+ phy->data_t_hs_zero = ROUND(105 + 6 * ui, 8 * ui);
+ phy->data_t_hs_trial = 2 * (ROUND(60 + 4 * ui, 8 * ui) - 1);
+ phy->data_t_ta_go = 3;
+ phy->data_t_ta_get = 4;
+
+ phy->pll_enbwt = 1;
+ phy->clklp2hs_time = ROUND(407, 8 * ui) + 12;
+ phy->clkhs2lp_time = ROUND(105 + 12 * ui, 8 * ui);
+ phy->lp2hs_time = ROUND(240 + 12 * ui, 8 * ui) + 1;
+ phy->hs2lp_time = phy->clkhs2lp_time;
+ phy->clk_to_data_delay = 1 + phy->clklp2hs_time;
+ phy->data_to_clk_delay = ROUND(60 + 52 * ui, 8 * ui) +
+ phy->clkhs2lp_time;
+
+ phy->lane_byte_clk_kHz = phy_rate_kHz / 8;
+ phy->clk_division =
+ DIV_ROUND_UP(phy->lane_byte_clk_kHz, MAX_TX_ESC_CLK);
+}
+
+static u32 dsi_get_dpi_color_coding(enum mipi_dsi_pixel_format format)
+{
+ u32 val;
+
+ /*
+ * TODO: only support RGB888 now, to support more
+ */
+ switch (format) {
+ case MIPI_DSI_FMT_RGB888:
+ val = DSI_24BITS_1;
+ break;
+ default:
+ val = DSI_24BITS_1;
+ break;
+ }
+
+ return val;
+}
+
+/*
+ * dsi phy reg write function
+ */
+static void dsi_phy_tst_set(void __iomem *base, u32 reg, u32 val)
+{
+ u32 reg_write = 0x10000 + reg;
+
+ /*
+ * latch reg first
+ */
+ writel(reg_write, base + PHY_TST_CTRL1);
+ writel(0x02, base + PHY_TST_CTRL0);
+ writel(0x00, base + PHY_TST_CTRL0);
+
+ /*
+ * then latch value
+ */
+ writel(val, base + PHY_TST_CTRL1);
+ writel(0x02, base + PHY_TST_CTRL0);
+ writel(0x00, base + PHY_TST_CTRL0);
+}
+
+static void dsi_set_phy_timer(void __iomem *base,
+ struct mipi_phy_params *phy,
+ u32 lanes)
+{
+ u32 val;
+
+ /*
+ * Set lane value and phy stop wait time.
+ */
+ val = (lanes - 1) | (PHY_STOP_WAIT_TIME << 8);
+ writel(val, base + PHY_IF_CFG);
+
+ /*
+ * Set phy clk division.
+ */
+ val = readl(base + CLKMGR_CFG) | phy->clk_division;
+ writel(val, base + CLKMGR_CFG);
+
+ /*
+ * Set lp and hs switching params.
+ */
+ dw_update_bits(base + PHY_TMR_CFG, 24, MASK(8), phy->hs2lp_time);
+ dw_update_bits(base + PHY_TMR_CFG, 16, MASK(8), phy->lp2hs_time);
+ dw_update_bits(base + PHY_TMR_LPCLK_CFG, 16, MASK(10),
+ phy->clkhs2lp_time);
+ dw_update_bits(base + PHY_TMR_LPCLK_CFG, 0, MASK(10),
+ phy->clklp2hs_time);
+ dw_update_bits(base + CLK_DATA_TMR_CFG, 8, MASK(8),
+ phy->data_to_clk_delay);
+ dw_update_bits(base + CLK_DATA_TMR_CFG, 0, MASK(8),
+ phy->clk_to_data_delay);
+}
+
+static void dsi_set_mipi_phy(void __iomem *base,
+ struct mipi_phy_params *phy,
+ u32 lanes)
+{
+ u32 delay_count;
+ u32 val;
+ u32 i;
+
+ /* phy timer setting */
+ dsi_set_phy_timer(base, phy, lanes);
+
+ /*
+ * Reset to clean up phy tst params.
+ */
+ writel(0, base + PHY_RSTZ);
+ writel(0, base + PHY_TST_CTRL0);
+ writel(1, base + PHY_TST_CTRL0);
+ writel(0, base + PHY_TST_CTRL0);
+
+ /*
+ * Clock lane timing control setting: TLPX, THS-PREPARE,
+ * THS-ZERO, THS-TRAIL, TWAKEUP.
+ */
+ dsi_phy_tst_set(base, CLK_TLPX, phy->clk_t_lpx);
+ dsi_phy_tst_set(base, CLK_THS_PREPARE, phy->clk_t_hs_prepare);
+ dsi_phy_tst_set(base, CLK_THS_ZERO, phy->clk_t_hs_zero);
+ dsi_phy_tst_set(base, CLK_THS_TRAIL, phy->clk_t_hs_trial);
+ dsi_phy_tst_set(base, CLK_TWAKEUP, phy->clk_t_wakeup);
+
+ /*
+ * Data lane timing control setting: TLPX, THS-PREPARE,
+ * THS-ZERO, THS-TRAIL, TTA-GO, TTA-GET, TWAKEUP.
+ */
+ for (i = 0; i < lanes; i++) {
+ dsi_phy_tst_set(base, DATA_TLPX(i), phy->data_t_lpx);
+ dsi_phy_tst_set(base, DATA_THS_PREPARE(i),
+ phy->data_t_hs_prepare);
+ dsi_phy_tst_set(base, DATA_THS_ZERO(i), phy->data_t_hs_zero);
+ dsi_phy_tst_set(base, DATA_THS_TRAIL(i), phy->data_t_hs_trial);
+ dsi_phy_tst_set(base, DATA_TTA_GO(i), phy->data_t_ta_go);
+ dsi_phy_tst_set(base, DATA_TTA_GET(i), phy->data_t_ta_get);
+ dsi_phy_tst_set(base, DATA_TWAKEUP(i), phy->data_t_wakeup);
+ }
+
+ /*
+ * physical configuration: I, pll I, pll II, pll III,
+ * pll IV, pll V.
+ */
+ dsi_phy_tst_set(base, PHY_CFG_I, phy->hstx_ckg_sel);
+ val = (phy->pll_fbd_div5f << 5) + (phy->pll_fbd_div1f << 4) +
+ (phy->pll_fbd_2p << 1) + phy->pll_enbwt;
+ dsi_phy_tst_set(base, PHY_CFG_PLL_I, val);
+ dsi_phy_tst_set(base, PHY_CFG_PLL_II, phy->pll_fbd_p);
+ dsi_phy_tst_set(base, PHY_CFG_PLL_III, phy->pll_fbd_s);
+ val = (phy->pll_pre_div1p << 7) + phy->pll_pre_p;
+ dsi_phy_tst_set(base, PHY_CFG_PLL_IV, val);
+ val = (5 << 5) + (phy->pll_vco_750M << 4) + (phy->pll_lpf_rs << 2) +
+ phy->pll_lpf_cs;
+ dsi_phy_tst_set(base, PHY_CFG_PLL_V, val);
+
+ writel(PHY_ENABLECLK, base + PHY_RSTZ);
+ udelay(1);
+ writel(PHY_ENABLECLK | PHY_UNSHUTDOWNZ, base + PHY_RSTZ);
+ udelay(1);
+ writel(PHY_ENABLECLK | PHY_UNRSTZ | PHY_UNSHUTDOWNZ, base + PHY_RSTZ);
+ usleep_range(1000, 1500);
+
+ /*
+ * wait for phy's clock ready
+ */
+ delay_count = 100;
+ while (delay_count) {
+ val = readl(base + PHY_STATUS);
+ if ((BIT(0) | BIT(2)) & val)
+ break;
+
+ udelay(1);
+ delay_count--;
+ }
+
+ if (!delay_count)
+ DRM_INFO("phylock and phystopstateclklane is not ready.\n");
+}
+
+static void dsi_set_mode_timing(void __iomem *base,
+ u32 lane_byte_clk_kHz,
+ struct drm_display_mode *mode,
+ enum mipi_dsi_pixel_format format)
+{
+ u32 hfp, hbp, hsw, vfp, vbp, vsw;
+ u32 hline_time;
+ u32 hsa_time;
+ u32 hbp_time;
+ u32 pixel_clk_kHz;
+ int htot, vtot;
+ u32 val;
+ u64 tmp;
+
+ val = dsi_get_dpi_color_coding(format);
+ writel(val, base + DPI_COLOR_CODING);
+
+ val = (mode->flags & DRM_MODE_FLAG_NHSYNC ? 1 : 0) << 2;
+ val |= (mode->flags & DRM_MODE_FLAG_NVSYNC ? 1 : 0) << 1;
+ writel(val, base + DPI_CFG_POL);
+
+ /*
+ * The DSI IP accepts vertical timing using lines as normal,
+ * but horizontal timing is a mixture of pixel-clocks for the
+ * active region and byte-lane clocks for the blanking-related
+ * timings. hfp is specified as the total hline_time in byte-
+ * lane clocks minus hsa, hbp and active.
+ */
+ pixel_clk_kHz = mode->clock;
+ htot = mode->htotal;
+ vtot = mode->vtotal;
+ hfp = mode->hsync_start - mode->hdisplay;
+ hbp = mode->htotal - mode->hsync_end;
+ hsw = mode->hsync_end - mode->hsync_start;
+ vfp = mode->vsync_start - mode->vdisplay;
+ vbp = mode->vtotal - mode->vsync_end;
+ vsw = mode->vsync_end - mode->vsync_start;
+ if (vsw > 15) {
+ DRM_DEBUG_DRIVER("vsw exceeded 15\n");
+ vsw = 15;
+ }
+
+ hsa_time = (hsw * lane_byte_clk_kHz) / pixel_clk_kHz;
+ hbp_time = (hbp * lane_byte_clk_kHz) / pixel_clk_kHz;
+ tmp = (u64)htot * (u64)lane_byte_clk_kHz;
+ hline_time = DIV_ROUND_UP(tmp, pixel_clk_kHz);
+
+ /* all specified in byte-lane clocks */
+ writel(hsa_time, base + VID_HSA_TIME);
+ writel(hbp_time, base + VID_HBP_TIME);
+ writel(hline_time, base + VID_HLINE_TIME);
+
+ writel(vsw, base + VID_VSA_LINES);
+ writel(vbp, base + VID_VBP_LINES);
+ writel(vfp, base + VID_VFP_LINES);
+ writel(mode->vdisplay, base + VID_VACTIVE_LINES);
+ writel(mode->hdisplay, base + VID_PKT_SIZE);
+
+ DRM_DEBUG_DRIVER("htot=%d, hfp=%d, hbp=%d, hsw=%d\n",
+ htot, hfp, hbp, hsw);
+ DRM_DEBUG_DRIVER("vtol=%d, vfp=%d, vbp=%d, vsw=%d\n",
+ vtot, vfp, vbp, vsw);
+ DRM_DEBUG_DRIVER("hsa_time=%d, hbp_time=%d, hline_time=%d\n",
+ hsa_time, hbp_time, hline_time);
+}
+
+static void dsi_set_video_mode(void __iomem *base, unsigned long flags)
+{
+ u32 val;
+ u32 mode_mask = MIPI_DSI_MODE_VIDEO | MIPI_DSI_MODE_VIDEO_BURST |
+ MIPI_DSI_MODE_VIDEO_SYNC_PULSE;
+ u32 non_burst_sync_pulse = MIPI_DSI_MODE_VIDEO |
+ MIPI_DSI_MODE_VIDEO_SYNC_PULSE;
+ u32 non_burst_sync_event = MIPI_DSI_MODE_VIDEO;
+
+ /*
+ * choose video mode type
+ */
+ if ((flags & mode_mask) == non_burst_sync_pulse)
+ val = DSI_NON_BURST_SYNC_PULSES;
+ else if ((flags & mode_mask) == non_burst_sync_event)
+ val = DSI_NON_BURST_SYNC_EVENTS;
+ else
+ val = DSI_BURST_SYNC_PULSES_1;
+ writel(val, base + VID_MODE_CFG);
+
+ writel(PHY_TXREQUESTCLKHS, base + LPCLK_CTRL);
+ writel(DSI_VIDEO_MODE, base + MODE_CFG);
+}
+
+static void dsi_mipi_init(struct dw_dsi *dsi)
+{
+ struct dsi_hw_ctx *ctx = dsi->ctx;
+ struct mipi_phy_params *phy = &dsi->phy;
+ struct drm_display_mode *mode = &dsi->cur_mode;
+ u32 bpp = mipi_dsi_pixel_format_to_bpp(dsi->format);
+ void __iomem *base = ctx->base;
+ u32 dphy_req_kHz;
+
+ /*
+ * count phy params
+ */
+ dphy_req_kHz = mode->clock * bpp / dsi->lanes;
+ dsi_get_phy_params(dphy_req_kHz, phy);
+
+ /* reset Core */
+ writel(RESET, base + PWR_UP);
+
+ /* set dsi phy params */
+ dsi_set_mipi_phy(base, phy, dsi->lanes);
+
+ /* set dsi mode timing */
+ dsi_set_mode_timing(base, phy->lane_byte_clk_kHz, mode, dsi->format);
+
+ /* set dsi video mode */
+ dsi_set_video_mode(base, dsi->mode_flags);
+
+ /* dsi wake up */
+ writel(POWERUP, base + PWR_UP);
+
+ DRM_DEBUG_DRIVER("lanes=%d, pixel_clk=%d kHz, bytes_freq=%d kHz\n",
+ dsi->lanes, mode->clock, phy->lane_byte_clk_kHz);
+}
+
+static void dsi_encoder_disable(struct drm_encoder *encoder)
+{
+ struct dw_dsi *dsi = encoder_to_dsi(encoder);
+ struct dsi_hw_ctx *ctx = dsi->ctx;
+ void __iomem *base = ctx->base;
+
+ if (!dsi->enable)
+ return;
+
+ writel(0, base + PWR_UP);
+ writel(0, base + LPCLK_CTRL);
+ writel(0, base + PHY_RSTZ);
+ clk_disable_unprepare(ctx->pclk);
+
+ dsi->enable = false;
+}
+
+static void dsi_encoder_enable(struct drm_encoder *encoder)
+{
+ struct dw_dsi *dsi = encoder_to_dsi(encoder);
+ struct dsi_hw_ctx *ctx = dsi->ctx;
+ int ret;
+
+ if (dsi->enable)
+ return;
+
+ ret = clk_prepare_enable(ctx->pclk);
+ if (ret) {
+ DRM_ERROR("fail to enable pclk: %d\n", ret);
+ return;
+ }
+
+ dsi_mipi_init(dsi);
+
+ dsi->enable = true;
+}
+
+static void dsi_encoder_mode_set(struct drm_encoder *encoder,
+ struct drm_display_mode *mode,
+ struct drm_display_mode *adj_mode)
+{
+ struct dw_dsi *dsi = encoder_to_dsi(encoder);
+
+ drm_mode_copy(&dsi->cur_mode, adj_mode);
+}
+
+static int dsi_encoder_atomic_check(struct drm_encoder *encoder,
+ struct drm_crtc_state *crtc_state,
+ struct drm_connector_state *conn_state)
+{
+ /* do nothing */
+ return 0;
+}
+
+static const struct drm_encoder_helper_funcs dw_encoder_helper_funcs = {
+ .atomic_check = dsi_encoder_atomic_check,
+ .mode_set = dsi_encoder_mode_set,
+ .enable = dsi_encoder_enable,
+ .disable = dsi_encoder_disable
+};
+
+static const struct drm_encoder_funcs dw_encoder_funcs = {
+ .destroy = drm_encoder_cleanup,
+};
+
+static int dw_drm_encoder_init(struct device *dev,
+ struct drm_device *drm_dev,
+ struct drm_encoder *encoder)
+{
+ int ret;
+ u32 crtc_mask = drm_of_find_possible_crtcs(drm_dev, dev->of_node);
+
+ if (!crtc_mask) {
+ DRM_ERROR("failed to find crtc mask\n");
+ return -EINVAL;
+ }
+
+ encoder->possible_crtcs = crtc_mask;
+ ret = drm_encoder_init(drm_dev, encoder, &dw_encoder_funcs,
+ DRM_MODE_ENCODER_DSI, NULL);
+ if (ret) {
+ DRM_ERROR("failed to init dsi encoder\n");
+ return ret;
+ }
+
+ drm_encoder_helper_add(encoder, &dw_encoder_helper_funcs);
+
+ return 0;
+}
+
+static int dsi_host_attach(struct mipi_dsi_host *host,
+ struct mipi_dsi_device *mdsi)
+{
+ struct dw_dsi *dsi = host_to_dsi(host);
+
+ if (mdsi->lanes < 1 || mdsi->lanes > 4) {
+ DRM_ERROR("dsi device params invalid\n");
+ return -EINVAL;
+ }
+
+ dsi->lanes = mdsi->lanes;
+ dsi->format = mdsi->format;
+ dsi->mode_flags = mdsi->mode_flags;
+
+ return 0;
+}
+
+static int dsi_host_detach(struct mipi_dsi_host *host,
+ struct mipi_dsi_device *mdsi)
+{
+ /* do nothing */
+ return 0;
+}
+
+static const struct mipi_dsi_host_ops dsi_host_ops = {
+ .attach = dsi_host_attach,
+ .detach = dsi_host_detach,
+};
+
+static int dsi_host_init(struct device *dev, struct dw_dsi *dsi)
+{
+ struct mipi_dsi_host *host = &dsi->host;
+ int ret;
+
+ host->dev = dev;
+ host->ops = &dsi_host_ops;
+ ret = mipi_dsi_host_register(host);
+ if (ret) {
+ DRM_ERROR("failed to register dsi host\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+static int dsi_bridge_init(struct drm_device *dev, struct dw_dsi *dsi)
+{
+ struct drm_encoder *encoder = &dsi->encoder;
+ struct drm_bridge *bridge = dsi->bridge;
+ int ret;
+
+ /* associate the bridge to dsi encoder */
+ encoder->bridge = bridge;
+ bridge->encoder = encoder;
+
+ ret = drm_bridge_attach(dev, bridge);
+ if (ret) {
+ DRM_ERROR("failed to attach external bridge\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+static int dsi_bind(struct device *dev, struct device *master, void *data)
+{
+ struct dsi_data *ddata = dev_get_drvdata(dev);
+ struct dw_dsi *dsi = &ddata->dsi;
+ struct drm_device *drm_dev = data;
+ int ret;
+
+ ret = dw_drm_encoder_init(dev, drm_dev, &dsi->encoder);
+ if (ret)
+ return ret;
+
+ ret = dsi_host_init(dev, dsi);
+ if (ret)
+ return ret;
+
+ ret = dsi_bridge_init(drm_dev, dsi);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static void dsi_unbind(struct device *dev, struct device *master, void *data)
+{
+ /* do nothing */
+}
+
+static const struct component_ops dsi_ops = {
+ .bind = dsi_bind,
+ .unbind = dsi_unbind,
+};
+
+static int dsi_parse_dt(struct platform_device *pdev, struct dw_dsi *dsi)
+{
+ struct dsi_hw_ctx *ctx = dsi->ctx;
+ struct device_node *np = pdev->dev.of_node;
+ struct device_node *endpoint, *bridge_node;
+ struct drm_bridge *bridge;
+ struct resource *res;
+
+ /*
+ * Get the endpoint node. In our case, dsi has one output port1
+ * to which the external HDMI bridge is connected.
+ */
+ endpoint = of_graph_get_endpoint_by_regs(np, 1, -1);
+ if (!endpoint) {
+ DRM_ERROR("no valid endpoint node\n");
+ return -ENODEV;
+ }
+ of_node_put(endpoint);
+
+ bridge_node = of_graph_get_remote_port_parent(endpoint);
+ if (!bridge_node) {
+ DRM_ERROR("no valid bridge node\n");
+ return -ENODEV;
+ }
+ of_node_put(bridge_node);
+
+ bridge = of_drm_find_bridge(bridge_node);
+ if (!bridge) {
+ DRM_INFO("wait for external HDMI bridge driver.\n");
+ return -EPROBE_DEFER;
+ }
+ dsi->bridge = bridge;
+
+ ctx->pclk = devm_clk_get(&pdev->dev, "pclk");
+ if (IS_ERR(ctx->pclk)) {
+ DRM_ERROR("failed to get pclk clock\n");
+ return PTR_ERR(ctx->pclk);
+ }
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ ctx->base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(ctx->base)) {
+ DRM_ERROR("failed to remap dsi io region\n");
+ return PTR_ERR(ctx->base);
+ }
+
+ return 0;
+}
+
+static int dsi_probe(struct platform_device *pdev)
+{
+ struct dsi_data *data;
+ struct dw_dsi *dsi;
+ struct dsi_hw_ctx *ctx;
+ int ret;
+
+ data = devm_kzalloc(&pdev->dev, sizeof(*data), GFP_KERNEL);
+ if (!data) {
+ DRM_ERROR("failed to allocate dsi data.\n");
+ return -ENOMEM;
+ }
+ dsi = &data->dsi;
+ ctx = &data->ctx;
+ dsi->ctx = ctx;
+
+ ret = dsi_parse_dt(pdev, dsi);
+ if (ret)
+ return ret;
+
+ platform_set_drvdata(pdev, data);
+
+ return component_add(&pdev->dev, &dsi_ops);
+}
+
+static int dsi_remove(struct platform_device *pdev)
+{
+ component_del(&pdev->dev, &dsi_ops);
+
+ return 0;
+}
+
+static const struct of_device_id dsi_of_match[] = {
+ {.compatible = "hisilicon,hi6220-dsi"},
+ { }
+};
+MODULE_DEVICE_TABLE(of, dsi_of_match);
+
+static struct platform_driver dsi_driver = {
+ .probe = dsi_probe,
+ .remove = dsi_remove,
+ .driver = {
+ .name = "dw-dsi",
+ .of_match_table = dsi_of_match,
+ },
+};
+
+module_platform_driver(dsi_driver);
+
+MODULE_AUTHOR("Xinliang Liu <xinliang.liu@linaro.org>");
+MODULE_AUTHOR("Xinliang Liu <z.liuxinliang@hisilicon.com>");
+MODULE_AUTHOR("Xinwei Kong <kong.kongxinwei@hisilicon.com>");
+MODULE_DESCRIPTION("DesignWare MIPI DSI Host Controller v1.02 driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/gpu/drm/hisilicon/kirin/dw_dsi_reg.h b/drivers/gpu/drm/hisilicon/kirin/dw_dsi_reg.h
new file mode 100644
index 0000000..18808fc
--- /dev/null
+++ b/drivers/gpu/drm/hisilicon/kirin/dw_dsi_reg.h
@@ -0,0 +1,103 @@
+/*
+ * Copyright (c) 2016 Linaro Limited.
+ * Copyright (c) 2014-2016 Hisilicon Limited.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#ifndef __DW_DSI_REG_H__
+#define __DW_DSI_REG_H__
+
+#define MASK(x) (BIT(x) - 1)
+
+/*
+ * regs
+ */
+#define PWR_UP 0x04 /* Core power-up */
+#define RESET 0
+#define POWERUP BIT(0)
+#define PHY_IF_CFG 0xA4 /* D-PHY interface configuration */
+#define CLKMGR_CFG 0x08 /* the internal clock dividers */
+#define PHY_RSTZ 0xA0 /* D-PHY reset control */
+#define PHY_ENABLECLK BIT(2)
+#define PHY_UNRSTZ BIT(1)
+#define PHY_UNSHUTDOWNZ BIT(0)
+#define PHY_TST_CTRL0 0xB4 /* D-PHY test interface control 0 */
+#define PHY_TST_CTRL1 0xB8 /* D-PHY test interface control 1 */
+#define CLK_TLPX 0x10
+#define CLK_THS_PREPARE 0x11
+#define CLK_THS_ZERO 0x12
+#define CLK_THS_TRAIL 0x13
+#define CLK_TWAKEUP 0x14
+#define DATA_TLPX(x) (0x20 + ((x) << 4))
+#define DATA_THS_PREPARE(x) (0x21 + ((x) << 4))
+#define DATA_THS_ZERO(x) (0x22 + ((x) << 4))
+#define DATA_THS_TRAIL(x) (0x23 + ((x) << 4))
+#define DATA_TTA_GO(x) (0x24 + ((x) << 4))
+#define DATA_TTA_GET(x) (0x25 + ((x) << 4))
+#define DATA_TWAKEUP(x) (0x26 + ((x) << 4))
+#define PHY_CFG_I 0x60
+#define PHY_CFG_PLL_I 0x63
+#define PHY_CFG_PLL_II 0x64
+#define PHY_CFG_PLL_III 0x65
+#define PHY_CFG_PLL_IV 0x66
+#define PHY_CFG_PLL_V 0x67
+#define DPI_COLOR_CODING 0x10 /* DPI color coding */
+#define DPI_CFG_POL 0x14 /* DPI polarity configuration */
+#define VID_HSA_TIME 0x48 /* Horizontal Sync Active time */
+#define VID_HBP_TIME 0x4C /* Horizontal Back Porch time */
+#define VID_HLINE_TIME 0x50 /* Line time */
+#define VID_VSA_LINES 0x54 /* Vertical Sync Active period */
+#define VID_VBP_LINES 0x58 /* Vertical Back Porch period */
+#define VID_VFP_LINES 0x5C /* Vertical Front Porch period */
+#define VID_VACTIVE_LINES 0x60 /* Vertical resolution */
+#define VID_PKT_SIZE 0x3C /* Video packet size */
+#define VID_MODE_CFG 0x38 /* Video mode configuration */
+#define PHY_TMR_CFG 0x9C /* Data lanes timing configuration */
+#define BTA_TO_CNT 0x8C /* Response timeout definition */
+#define PHY_TMR_LPCLK_CFG 0x98 /* clock lane timing configuration */
+#define CLK_DATA_TMR_CFG 0xCC
+#define LPCLK_CTRL 0x94 /* Low-power in clock lane */
+#define PHY_TXREQUESTCLKHS BIT(0)
+#define MODE_CFG 0x34 /* Video or Command mode selection */
+#define PHY_STATUS 0xB0 /* D-PHY PPI status interface */
+
+#define PHY_STOP_WAIT_TIME 0x30
+
+/*
+ * regs relevant enum
+ */
+enum dpi_color_coding {
+ DSI_24BITS_1 = 5,
+};
+
+enum dsi_video_mode_type {
+ DSI_NON_BURST_SYNC_PULSES = 0,
+ DSI_NON_BURST_SYNC_EVENTS,
+ DSI_BURST_SYNC_PULSES_1,
+ DSI_BURST_SYNC_PULSES_2
+};
+
+enum dsi_work_mode {
+ DSI_VIDEO_MODE = 0,
+ DSI_COMMAND_MODE
+};
+
+/*
+ * Register Write/Read Helper functions
+ */
+static inline void dw_update_bits(void __iomem *addr, u32 bit_start,
+ u32 mask, u32 val)
+{
+ u32 tmp, orig;
+
+ orig = readl(addr);
+ tmp = orig & ~(mask << bit_start);
+ tmp |= (val & mask) << bit_start;
+ writel(tmp, addr);
+}
+
+#endif /* __DW_DRM_DSI_H__ */
diff --git a/drivers/gpu/drm/hisilicon/kirin/kirin_ade_reg.h b/drivers/gpu/drm/hisilicon/kirin/kirin_ade_reg.h
new file mode 100644
index 0000000..4cf281b7
--- /dev/null
+++ b/drivers/gpu/drm/hisilicon/kirin/kirin_ade_reg.h
@@ -0,0 +1,230 @@
+/*
+ * Copyright (c) 2016 Linaro Limited.
+ * Copyright (c) 2014-2016 Hisilicon Limited.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#ifndef __KIRIN_ADE_REG_H__
+#define __KIRIN_ADE_REG_H__
+
+/*
+ * ADE Registers
+ */
+#define MASK(x) (BIT(x) - 1)
+
+#define ADE_CTRL 0x0004
+#define FRM_END_START_OFST 0
+#define FRM_END_START_MASK MASK(2)
+#define AUTO_CLK_GATE_EN_OFST 0
+#define AUTO_CLK_GATE_EN BIT(0)
+#define ADE_DISP_SRC_CFG 0x0018
+#define ADE_CTRL1 0x008C
+#define ADE_EN 0x0100
+#define ADE_DISABLE 0
+#define ADE_ENABLE 1
+/* reset and reload regs */
+#define ADE_SOFT_RST_SEL(x) (0x0078 + (x) * 0x4)
+#define ADE_RELOAD_DIS(x) (0x00AC + (x) * 0x4)
+#define RDMA_OFST 0
+#define CLIP_OFST 15
+#define SCL_OFST 21
+#define CTRAN_OFST 24
+#define OVLY_OFST 37 /* 32+5 */
+/* channel regs */
+#define RD_CH_CTRL(x) (0x1004 + (x) * 0x80)
+#define RD_CH_ADDR(x) (0x1008 + (x) * 0x80)
+#define RD_CH_SIZE(x) (0x100C + (x) * 0x80)
+#define RD_CH_STRIDE(x) (0x1010 + (x) * 0x80)
+#define RD_CH_SPACE(x) (0x1014 + (x) * 0x80)
+#define RD_CH_EN(x) (0x1020 + (x) * 0x80)
+/* overlay regs */
+#define ADE_OVLY1_TRANS_CFG 0x002C
+#define ADE_OVLY_CTL 0x0098
+#define ADE_OVLY_CH_XY0(x) (0x2004 + (x) * 4)
+#define ADE_OVLY_CH_XY1(x) (0x2024 + (x) * 4)
+#define ADE_OVLY_CH_CTL(x) (0x204C + (x) * 4)
+#define ADE_OVLY_OUTPUT_SIZE(x) (0x2070 + (x) * 8)
+#define OUTPUT_XSIZE_OFST 16
+#define ADE_OVLYX_CTL(x) (0x209C + (x) * 4)
+#define CH_OVLY_SEL_OFST(x) ((x) * 4)
+#define CH_OVLY_SEL_MASK MASK(2)
+#define CH_OVLY_SEL_VAL(x) ((x) + 1)
+#define CH_ALP_MODE_OFST 0
+#define CH_ALP_SEL_OFST 2
+#define CH_UNDER_ALP_SEL_OFST 4
+#define CH_EN_OFST 6
+#define CH_ALP_GBL_OFST 15
+#define CH_SEL_OFST 28
+/* ctran regs */
+#define ADE_CTRAN_DIS(x) (0x5004 + (x) * 0x100)
+#define CTRAN_BYPASS_ON 1
+#define CTRAN_BYPASS_OFF 0
+#define ADE_CTRAN_IMAGE_SIZE(x) (0x503C + (x) * 0x100)
+/* clip regs */
+#define ADE_CLIP_DISABLE(x) (0x6800 + (x) * 0x100)
+#define ADE_CLIP_SIZE0(x) (0x6804 + (x) * 0x100)
+#define ADE_CLIP_SIZE1(x) (0x6808 + (x) * 0x100)
+
+/*
+ * LDI Registers
+ */
+#define LDI_HRZ_CTRL0 0x7400
+#define HBP_OFST 20
+#define LDI_HRZ_CTRL1 0x7404
+#define LDI_VRT_CTRL0 0x7408
+#define VBP_OFST 20
+#define LDI_VRT_CTRL1 0x740C
+#define LDI_PLR_CTRL 0x7410
+#define FLAG_NVSYNC BIT(0)
+#define FLAG_NHSYNC BIT(1)
+#define FLAG_NPIXCLK BIT(2)
+#define FLAG_NDE BIT(3)
+#define LDI_DSP_SIZE 0x7414
+#define VSIZE_OFST 20
+#define LDI_INT_EN 0x741C
+#define FRAME_END_INT_EN_OFST 1
+#define LDI_CTRL 0x7420
+#define BPP_OFST 3
+#define DATA_GATE_EN BIT(2)
+#define LDI_EN BIT(0)
+#define LDI_MSK_INT 0x7428
+#define LDI_INT_CLR 0x742C
+#define LDI_WORK_MODE 0x7430
+#define LDI_HDMI_DSI_GT 0x7434
+
+/*
+ * ADE media bus service regs
+ */
+#define ADE0_QOSGENERATOR_MODE 0x010C
+#define QOSGENERATOR_MODE_MASK MASK(2)
+#define ADE0_QOSGENERATOR_EXTCONTROL 0x0118
+#define SOCKET_QOS_EN BIT(0)
+#define ADE1_QOSGENERATOR_MODE 0x020C
+#define ADE1_QOSGENERATOR_EXTCONTROL 0x0218
+
+/*
+ * ADE regs relevant enums
+ */
+enum frame_end_start {
+ /* regs take effect in every vsync */
+ REG_EFFECTIVE_IN_VSYNC = 0,
+ /* regs take effect in fist ade en and every frame end */
+ REG_EFFECTIVE_IN_ADEEN_FRMEND,
+ /* regs take effect in ade en immediately */
+ REG_EFFECTIVE_IN_ADEEN,
+ /* regs take effect in first vsync and every frame end */
+ REG_EFFECTIVE_IN_VSYNC_FRMEND
+};
+
+enum ade_fb_format {
+ ADE_RGB_565 = 0,
+ ADE_BGR_565,
+ ADE_XRGB_8888,
+ ADE_XBGR_8888,
+ ADE_ARGB_8888,
+ ADE_ABGR_8888,
+ ADE_RGBA_8888,
+ ADE_BGRA_8888,
+ ADE_RGB_888,
+ ADE_BGR_888 = 9,
+ ADE_FORMAT_UNSUPPORT = 800
+};
+
+enum ade_channel {
+ ADE_CH1 = 0, /* channel 1 for primary plane */
+ ADE_CH_NUM
+};
+
+enum ade_scale {
+ ADE_SCL1 = 0,
+ ADE_SCL2,
+ ADE_SCL3,
+ ADE_SCL_NUM
+};
+
+enum ade_ctran {
+ ADE_CTRAN1 = 0,
+ ADE_CTRAN2,
+ ADE_CTRAN3,
+ ADE_CTRAN4,
+ ADE_CTRAN5,
+ ADE_CTRAN6,
+ ADE_CTRAN_NUM
+};
+
+enum ade_overlay {
+ ADE_OVLY1 = 0,
+ ADE_OVLY2,
+ ADE_OVLY3,
+ ADE_OVLY_NUM
+};
+
+enum ade_alpha_mode {
+ ADE_ALP_GLOBAL = 0,
+ ADE_ALP_PIXEL,
+ ADE_ALP_PIXEL_AND_GLB
+};
+
+enum ade_alpha_blending_mode {
+ ADE_ALP_MUL_COEFF_0 = 0, /* alpha */
+ ADE_ALP_MUL_COEFF_1, /* 1-alpha */
+ ADE_ALP_MUL_COEFF_2, /* 0 */
+ ADE_ALP_MUL_COEFF_3 /* 1 */
+};
+
+/*
+ * LDI regs relevant enums
+ */
+enum dsi_pclk_en {
+ DSI_PCLK_ON = 0,
+ DSI_PCLK_OFF
+};
+
+enum ldi_output_format {
+ LDI_OUT_RGB_565 = 0,
+ LDI_OUT_RGB_666,
+ LDI_OUT_RGB_888
+};
+
+enum ldi_work_mode {
+ TEST_MODE = 0,
+ NORMAL_MODE
+};
+
+enum ldi_input_source {
+ DISP_SRC_NONE = 0,
+ DISP_SRC_OVLY2,
+ DISP_SRC_DISP,
+ DISP_SRC_ROT,
+ DISP_SRC_SCL2
+};
+
+/*
+ * ADE media bus service relevant enums
+ */
+enum qos_generator_mode {
+ FIXED_MODE = 0,
+ LIMITER_MODE,
+ BYPASS_MODE,
+ REGULATOR_MODE
+};
+
+/*
+ * Register Write/Read Helper functions
+ */
+static inline void ade_update_bits(void __iomem *addr, u32 bit_start,
+ u32 mask, u32 val)
+{
+ u32 tmp, orig;
+
+ orig = readl(addr);
+ tmp = orig & ~(mask << bit_start);
+ tmp |= (val & mask) << bit_start;
+ writel(tmp, addr);
+}
+
+#endif
diff --git a/drivers/gpu/drm/hisilicon/kirin/kirin_drm_ade.c b/drivers/gpu/drm/hisilicon/kirin/kirin_drm_ade.c
new file mode 100644
index 0000000..fba6372
--- /dev/null
+++ b/drivers/gpu/drm/hisilicon/kirin/kirin_drm_ade.c
@@ -0,0 +1,1057 @@
+/*
+ * Hisilicon Hi6220 SoC ADE(Advanced Display Engine)'s crtc&plane driver
+ *
+ * Copyright (c) 2016 Linaro Limited.
+ * Copyright (c) 2014-2016 Hisilicon Limited.
+ *
+ * Author:
+ * Xinliang Liu <z.liuxinliang@hisilicon.com>
+ * Xinliang Liu <xinliang.liu@linaro.org>
+ * Xinwei Kong <kong.kongxinwei@hisilicon.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#include <linux/bitops.h>
+#include <linux/clk.h>
+#include <video/display_timing.h>
+#include <linux/mfd/syscon.h>
+#include <linux/regmap.h>
+#include <linux/reset.h>
+
+#include <drm/drmP.h>
+#include <drm/drm_crtc.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_atomic.h>
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_plane_helper.h>
+#include <drm/drm_gem_cma_helper.h>
+#include <drm/drm_fb_cma_helper.h>
+
+#include "kirin_drm_drv.h"
+#include "kirin_ade_reg.h"
+
+#define PRIMARY_CH ADE_CH1 /* primary plane */
+#define OUT_OVLY ADE_OVLY2 /* output overlay compositor */
+#define ADE_DEBUG 1
+
+#define to_ade_crtc(crtc) \
+ container_of(crtc, struct ade_crtc, base)
+
+#define to_ade_plane(plane) \
+ container_of(plane, struct ade_plane, base)
+
+struct ade_hw_ctx {
+ void __iomem *base;
+ struct regmap *noc_regmap;
+ struct clk *ade_core_clk;
+ struct clk *media_noc_clk;
+ struct clk *ade_pix_clk;
+ struct reset_control *reset;
+ bool power_on;
+ int irq;
+};
+
+struct ade_crtc {
+ struct drm_crtc base;
+ struct ade_hw_ctx *ctx;
+ bool enable;
+ u32 out_format;
+};
+
+struct ade_plane {
+ struct drm_plane base;
+ void *ctx;
+ u8 ch; /* channel */
+};
+
+struct ade_data {
+ struct ade_crtc acrtc;
+ struct ade_plane aplane[ADE_CH_NUM];
+ struct ade_hw_ctx ctx;
+};
+
+/* ade-format info: */
+struct ade_format {
+ u32 pixel_format;
+ enum ade_fb_format ade_format;
+};
+
+static const struct ade_format ade_formats[] = {
+ /* 16bpp RGB: */
+ { DRM_FORMAT_RGB565, ADE_RGB_565 },
+ { DRM_FORMAT_BGR565, ADE_BGR_565 },
+ /* 24bpp RGB: */
+ { DRM_FORMAT_RGB888, ADE_RGB_888 },
+ { DRM_FORMAT_BGR888, ADE_BGR_888 },
+ /* 32bpp [A]RGB: */
+ { DRM_FORMAT_XRGB8888, ADE_XRGB_8888 },
+ { DRM_FORMAT_XBGR8888, ADE_XBGR_8888 },
+ { DRM_FORMAT_RGBA8888, ADE_RGBA_8888 },
+ { DRM_FORMAT_BGRA8888, ADE_BGRA_8888 },
+ { DRM_FORMAT_ARGB8888, ADE_ARGB_8888 },
+ { DRM_FORMAT_ABGR8888, ADE_ABGR_8888 },
+};
+
+static const u32 channel_formats1[] = {
+ /* channel 1,2,3,4 */
+ DRM_FORMAT_RGB565, DRM_FORMAT_BGR565, DRM_FORMAT_RGB888,
+ DRM_FORMAT_BGR888, DRM_FORMAT_XRGB8888, DRM_FORMAT_XBGR8888,
+ DRM_FORMAT_RGBA8888, DRM_FORMAT_BGRA8888, DRM_FORMAT_ARGB8888,
+ DRM_FORMAT_ABGR8888
+};
+
+u32 ade_get_channel_formats(u8 ch, const u32 **formats)
+{
+ switch (ch) {
+ case ADE_CH1:
+ *formats = channel_formats1;
+ return ARRAY_SIZE(channel_formats1);
+ default:
+ DRM_ERROR("no this channel %d\n", ch);
+ *formats = NULL;
+ return 0;
+ }
+}
+
+/* convert from fourcc format to ade format */
+static u32 ade_get_format(u32 pixel_format)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(ade_formats); i++)
+ if (ade_formats[i].pixel_format == pixel_format)
+ return ade_formats[i].ade_format;
+
+ /* not found */
+ DRM_ERROR("Not found pixel format!!fourcc_format= %d\n",
+ pixel_format);
+ return ADE_FORMAT_UNSUPPORT;
+}
+
+static void ade_update_reload_bit(void __iomem *base, u32 bit_num, u32 val)
+{
+ u32 bit_ofst, reg_num;
+
+ bit_ofst = bit_num % 32;
+ reg_num = bit_num / 32;
+
+ ade_update_bits(base + ADE_RELOAD_DIS(reg_num), bit_ofst,
+ MASK(1), !!val);
+}
+
+static u32 ade_read_reload_bit(void __iomem *base, u32 bit_num)
+{
+ u32 tmp, bit_ofst, reg_num;
+
+ bit_ofst = bit_num % 32;
+ reg_num = bit_num / 32;
+
+ tmp = readl(base + ADE_RELOAD_DIS(reg_num));
+ return !!(BIT(bit_ofst) & tmp);
+}
+
+static void ade_init(struct ade_hw_ctx *ctx)
+{
+ void __iomem *base = ctx->base;
+
+ /* enable clk gate */
+ ade_update_bits(base + ADE_CTRL1, AUTO_CLK_GATE_EN_OFST,
+ AUTO_CLK_GATE_EN, ADE_ENABLE);
+ /* clear overlay */
+ writel(0, base + ADE_OVLY1_TRANS_CFG);
+ writel(0, base + ADE_OVLY_CTL);
+ writel(0, base + ADE_OVLYX_CTL(OUT_OVLY));
+ /* clear reset and reload regs */
+ writel(MASK(32), base + ADE_SOFT_RST_SEL(0));
+ writel(MASK(32), base + ADE_SOFT_RST_SEL(1));
+ writel(MASK(32), base + ADE_RELOAD_DIS(0));
+ writel(MASK(32), base + ADE_RELOAD_DIS(1));
+ /*
+ * for video mode, all the ade registers should
+ * become effective at frame end.
+ */
+ ade_update_bits(base + ADE_CTRL, FRM_END_START_OFST,
+ FRM_END_START_MASK, REG_EFFECTIVE_IN_ADEEN_FRMEND);
+}
+
+static void ade_set_pix_clk(struct ade_hw_ctx *ctx,
+ struct drm_display_mode *mode,
+ struct drm_display_mode *adj_mode)
+{
+ u32 clk_Hz = mode->clock * 1000;
+ int ret;
+
+ /*
+ * Success should be guaranteed in mode_valid call back,
+ * so failure shouldn't happen here
+ */
+ ret = clk_set_rate(ctx->ade_pix_clk, clk_Hz);
+ if (ret)
+ DRM_ERROR("failed to set pixel clk %dHz (%d)\n", clk_Hz, ret);
+ adj_mode->clock = clk_get_rate(ctx->ade_pix_clk) / 1000;
+}
+
+static void ade_ldi_set_mode(struct ade_crtc *acrtc,
+ struct drm_display_mode *mode,
+ struct drm_display_mode *adj_mode)
+{
+ struct ade_hw_ctx *ctx = acrtc->ctx;
+ void __iomem *base = ctx->base;
+ u32 width = mode->hdisplay;
+ u32 height = mode->vdisplay;
+ u32 hfp, hbp, hsw, vfp, vbp, vsw;
+ u32 plr_flags;
+
+ plr_flags = (mode->flags & DRM_MODE_FLAG_NVSYNC) ? FLAG_NVSYNC : 0;
+ plr_flags |= (mode->flags & DRM_MODE_FLAG_NHSYNC) ? FLAG_NHSYNC : 0;
+ hfp = mode->hsync_start - mode->hdisplay;
+ hbp = mode->htotal - mode->hsync_end;
+ hsw = mode->hsync_end - mode->hsync_start;
+ vfp = mode->vsync_start - mode->vdisplay;
+ vbp = mode->vtotal - mode->vsync_end;
+ vsw = mode->vsync_end - mode->vsync_start;
+ if (vsw > 15) {
+ DRM_DEBUG_DRIVER("vsw exceeded 15\n");
+ vsw = 15;
+ }
+
+ writel((hbp << HBP_OFST) | hfp, base + LDI_HRZ_CTRL0);
+ /* the configured value is actual value - 1 */
+ writel(hsw - 1, base + LDI_HRZ_CTRL1);
+ writel((vbp << VBP_OFST) | vfp, base + LDI_VRT_CTRL0);
+ /* the configured value is actual value - 1 */
+ writel(vsw - 1, base + LDI_VRT_CTRL1);
+ /* the configured value is actual value - 1 */
+ writel(((height - 1) << VSIZE_OFST) | (width - 1),
+ base + LDI_DSP_SIZE);
+ writel(plr_flags, base + LDI_PLR_CTRL);
+
+ /* set overlay compositor output size */
+ writel(((width - 1) << OUTPUT_XSIZE_OFST) | (height - 1),
+ base + ADE_OVLY_OUTPUT_SIZE(OUT_OVLY));
+
+ /* ctran6 setting */
+ writel(CTRAN_BYPASS_ON, base + ADE_CTRAN_DIS(ADE_CTRAN6));
+ /* the configured value is actual value - 1 */
+ writel(width * height - 1, base + ADE_CTRAN_IMAGE_SIZE(ADE_CTRAN6));
+ ade_update_reload_bit(base, CTRAN_OFST + ADE_CTRAN6, 0);
+
+ ade_set_pix_clk(ctx, mode, adj_mode);
+
+ DRM_DEBUG_DRIVER("set mode: %dx%d\n", width, height);
+}
+
+static int ade_power_up(struct ade_hw_ctx *ctx)
+{
+ int ret;
+
+ ret = clk_prepare_enable(ctx->media_noc_clk);
+ if (ret) {
+ DRM_ERROR("failed to enable media_noc_clk (%d)\n", ret);
+ return ret;
+ }
+
+ ret = reset_control_deassert(ctx->reset);
+ if (ret) {
+ DRM_ERROR("failed to deassert reset\n");
+ return ret;
+ }
+
+ ret = clk_prepare_enable(ctx->ade_core_clk);
+ if (ret) {
+ DRM_ERROR("failed to enable ade_core_clk (%d)\n", ret);
+ return ret;
+ }
+
+ ade_init(ctx);
+ ctx->power_on = true;
+ return 0;
+}
+
+static void ade_power_down(struct ade_hw_ctx *ctx)
+{
+ void __iomem *base = ctx->base;
+
+ writel(ADE_DISABLE, base + LDI_CTRL);
+ /* dsi pixel off */
+ writel(DSI_PCLK_OFF, base + LDI_HDMI_DSI_GT);
+
+ clk_disable_unprepare(ctx->ade_core_clk);
+ reset_control_assert(ctx->reset);
+ clk_disable_unprepare(ctx->media_noc_clk);
+ ctx->power_on = false;
+}
+
+static void ade_set_medianoc_qos(struct ade_crtc *acrtc)
+{
+ struct ade_hw_ctx *ctx = acrtc->ctx;
+ struct regmap *map = ctx->noc_regmap;
+
+ regmap_update_bits(map, ADE0_QOSGENERATOR_MODE,
+ QOSGENERATOR_MODE_MASK, BYPASS_MODE);
+ regmap_update_bits(map, ADE0_QOSGENERATOR_EXTCONTROL,
+ SOCKET_QOS_EN, SOCKET_QOS_EN);
+
+ regmap_update_bits(map, ADE1_QOSGENERATOR_MODE,
+ QOSGENERATOR_MODE_MASK, BYPASS_MODE);
+ regmap_update_bits(map, ADE1_QOSGENERATOR_EXTCONTROL,
+ SOCKET_QOS_EN, SOCKET_QOS_EN);
+}
+
+static int ade_enable_vblank(struct drm_device *dev, unsigned int pipe)
+{
+ struct kirin_drm_private *priv = dev->dev_private;
+ struct ade_crtc *acrtc = to_ade_crtc(priv->crtc[pipe]);
+ struct ade_hw_ctx *ctx = acrtc->ctx;
+ void __iomem *base = ctx->base;
+
+ if (!ctx->power_on)
+ (void)ade_power_up(ctx);
+
+ ade_update_bits(base + LDI_INT_EN, FRAME_END_INT_EN_OFST,
+ MASK(1), 1);
+
+ return 0;
+}
+
+static void ade_disable_vblank(struct drm_device *dev, unsigned int pipe)
+{
+ struct kirin_drm_private *priv = dev->dev_private;
+ struct ade_crtc *acrtc = to_ade_crtc(priv->crtc[pipe]);
+ struct ade_hw_ctx *ctx = acrtc->ctx;
+ void __iomem *base = ctx->base;
+
+ if (!ctx->power_on) {
+ DRM_ERROR("power is down! vblank disable fail\n");
+ return;
+ }
+
+ ade_update_bits(base + LDI_INT_EN, FRAME_END_INT_EN_OFST,
+ MASK(1), 0);
+}
+
+static irqreturn_t ade_irq_handler(int irq, void *data)
+{
+ struct ade_crtc *acrtc = data;
+ struct ade_hw_ctx *ctx = acrtc->ctx;
+ struct drm_crtc *crtc = &acrtc->base;
+ void __iomem *base = ctx->base;
+ u32 status;
+
+ status = readl(base + LDI_MSK_INT);
+ DRM_DEBUG_VBL("LDI IRQ: status=0x%X\n", status);
+
+ /* vblank irq */
+ if (status & BIT(FRAME_END_INT_EN_OFST)) {
+ ade_update_bits(base + LDI_INT_CLR, FRAME_END_INT_EN_OFST,
+ MASK(1), 1);
+ drm_crtc_handle_vblank(crtc);
+ }
+
+ return IRQ_HANDLED;
+}
+
+static void ade_display_enable(struct ade_crtc *acrtc)
+{
+ struct ade_hw_ctx *ctx = acrtc->ctx;
+ void __iomem *base = ctx->base;
+ u32 out_fmt = acrtc->out_format;
+
+ /* enable output overlay compositor */
+ writel(ADE_ENABLE, base + ADE_OVLYX_CTL(OUT_OVLY));
+ ade_update_reload_bit(base, OVLY_OFST + OUT_OVLY, 0);
+
+ /* display source setting */
+ writel(DISP_SRC_OVLY2, base + ADE_DISP_SRC_CFG);
+
+ /* enable ade */
+ writel(ADE_ENABLE, base + ADE_EN);
+ /* enable ldi */
+ writel(NORMAL_MODE, base + LDI_WORK_MODE);
+ writel((out_fmt << BPP_OFST) | DATA_GATE_EN | LDI_EN,
+ base + LDI_CTRL);
+ /* dsi pixel on */
+ writel(DSI_PCLK_ON, base + LDI_HDMI_DSI_GT);
+}
+
+#if ADE_DEBUG
+static void ade_rdma_dump_regs(void __iomem *base, u32 ch)
+{
+ u32 reg_ctrl, reg_addr, reg_size, reg_stride, reg_space, reg_en;
+ u32 val;
+
+ reg_ctrl = RD_CH_CTRL(ch);
+ reg_addr = RD_CH_ADDR(ch);
+ reg_size = RD_CH_SIZE(ch);
+ reg_stride = RD_CH_STRIDE(ch);
+ reg_space = RD_CH_SPACE(ch);
+ reg_en = RD_CH_EN(ch);
+
+ val = ade_read_reload_bit(base, RDMA_OFST + ch);
+ DRM_DEBUG_DRIVER("[rdma%d]: reload(%d)\n", ch + 1, val);
+ val = readl(base + reg_ctrl);
+ DRM_DEBUG_DRIVER("[rdma%d]: reg_ctrl(0x%08x)\n", ch + 1, val);
+ val = readl(base + reg_addr);
+ DRM_DEBUG_DRIVER("[rdma%d]: reg_addr(0x%08x)\n", ch + 1, val);
+ val = readl(base + reg_size);
+ DRM_DEBUG_DRIVER("[rdma%d]: reg_size(0x%08x)\n", ch + 1, val);
+ val = readl(base + reg_stride);
+ DRM_DEBUG_DRIVER("[rdma%d]: reg_stride(0x%08x)\n", ch + 1, val);
+ val = readl(base + reg_space);
+ DRM_DEBUG_DRIVER("[rdma%d]: reg_space(0x%08x)\n", ch + 1, val);
+ val = readl(base + reg_en);
+ DRM_DEBUG_DRIVER("[rdma%d]: reg_en(0x%08x)\n", ch + 1, val);
+}
+
+static void ade_clip_dump_regs(void __iomem *base, u32 ch)
+{
+ u32 val;
+
+ val = ade_read_reload_bit(base, CLIP_OFST + ch);
+ DRM_DEBUG_DRIVER("[clip%d]: reload(%d)\n", ch + 1, val);
+ val = readl(base + ADE_CLIP_DISABLE(ch));
+ DRM_DEBUG_DRIVER("[clip%d]: reg_clip_disable(0x%08x)\n", ch + 1, val);
+ val = readl(base + ADE_CLIP_SIZE0(ch));
+ DRM_DEBUG_DRIVER("[clip%d]: reg_clip_size0(0x%08x)\n", ch + 1, val);
+ val = readl(base + ADE_CLIP_SIZE1(ch));
+ DRM_DEBUG_DRIVER("[clip%d]: reg_clip_size1(0x%08x)\n", ch + 1, val);
+}
+
+static void ade_compositor_routing_dump_regs(void __iomem *base, u32 ch)
+{
+ u8 ovly_ch = 0; /* TODO: Only primary plane now */
+ u32 val;
+
+ val = readl(base + ADE_OVLY_CH_XY0(ovly_ch));
+ DRM_DEBUG_DRIVER("[overlay ch%d]: reg_ch_xy0(0x%08x)\n", ovly_ch, val);
+ val = readl(base + ADE_OVLY_CH_XY1(ovly_ch));
+ DRM_DEBUG_DRIVER("[overlay ch%d]: reg_ch_xy1(0x%08x)\n", ovly_ch, val);
+ val = readl(base + ADE_OVLY_CH_CTL(ovly_ch));
+ DRM_DEBUG_DRIVER("[overlay ch%d]: reg_ch_ctl(0x%08x)\n", ovly_ch, val);
+}
+
+static void ade_dump_overlay_compositor_regs(void __iomem *base, u32 comp)
+{
+ u32 val;
+
+ val = ade_read_reload_bit(base, OVLY_OFST + comp);
+ DRM_DEBUG_DRIVER("[overlay%d]: reload(%d)\n", comp + 1, val);
+ writel(ADE_ENABLE, base + ADE_OVLYX_CTL(comp));
+ DRM_DEBUG_DRIVER("[overlay%d]: reg_ctl(0x%08x)\n", comp + 1, val);
+ val = readl(base + ADE_OVLY_CTL);
+ DRM_DEBUG_DRIVER("ovly_ctl(0x%08x)\n", val);
+}
+
+static void ade_dump_regs(void __iomem *base)
+{
+ u32 i;
+
+ /* dump channel regs */
+ for (i = 0; i < ADE_CH_NUM; i++) {
+ /* dump rdma regs */
+ ade_rdma_dump_regs(base, i);
+
+ /* dump clip regs */
+ ade_clip_dump_regs(base, i);
+
+ /* dump compositor routing regs */
+ ade_compositor_routing_dump_regs(base, i);
+ }
+
+ /* dump overlay compositor regs */
+ ade_dump_overlay_compositor_regs(base, OUT_OVLY);
+}
+#else
+static void ade_dump_regs(void __iomem *base) { }
+#endif
+
+static void ade_crtc_enable(struct drm_crtc *crtc)
+{
+ struct ade_crtc *acrtc = to_ade_crtc(crtc);
+ struct ade_hw_ctx *ctx = acrtc->ctx;
+ int ret;
+
+ if (acrtc->enable)
+ return;
+
+ if (!ctx->power_on) {
+ ret = ade_power_up(ctx);
+ if (ret)
+ return;
+ }
+
+ ade_set_medianoc_qos(acrtc);
+ ade_display_enable(acrtc);
+ ade_dump_regs(ctx->base);
+ acrtc->enable = true;
+}
+
+static void ade_crtc_disable(struct drm_crtc *crtc)
+{
+ struct ade_crtc *acrtc = to_ade_crtc(crtc);
+ struct ade_hw_ctx *ctx = acrtc->ctx;
+
+ if (!acrtc->enable)
+ return;
+
+ ade_power_down(ctx);
+ acrtc->enable = false;
+}
+
+static int ade_crtc_atomic_check(struct drm_crtc *crtc,
+ struct drm_crtc_state *state)
+{
+ /* do nothing */
+ return 0;
+}
+
+static void ade_crtc_mode_set_nofb(struct drm_crtc *crtc)
+{
+ struct ade_crtc *acrtc = to_ade_crtc(crtc);
+ struct ade_hw_ctx *ctx = acrtc->ctx;
+ struct drm_display_mode *mode = &crtc->state->mode;
+ struct drm_display_mode *adj_mode = &crtc->state->adjusted_mode;
+
+ if (!ctx->power_on)
+ (void)ade_power_up(ctx);
+ ade_ldi_set_mode(acrtc, mode, adj_mode);
+}
+
+static void ade_crtc_atomic_begin(struct drm_crtc *crtc,
+ struct drm_crtc_state *old_state)
+{
+ struct ade_crtc *acrtc = to_ade_crtc(crtc);
+ struct ade_hw_ctx *ctx = acrtc->ctx;
+
+ if (!ctx->power_on)
+ (void)ade_power_up(ctx);
+}
+
+static void ade_crtc_atomic_flush(struct drm_crtc *crtc,
+ struct drm_crtc_state *old_state)
+
+{
+ struct ade_crtc *acrtc = to_ade_crtc(crtc);
+ struct ade_hw_ctx *ctx = acrtc->ctx;
+ void __iomem *base = ctx->base;
+
+ /* only crtc is enabled regs take effect */
+ if (acrtc->enable) {
+ ade_dump_regs(base);
+ /* flush ade registers */
+ writel(ADE_ENABLE, base + ADE_EN);
+ }
+}
+
+static const struct drm_crtc_helper_funcs ade_crtc_helper_funcs = {
+ .enable = ade_crtc_enable,
+ .disable = ade_crtc_disable,
+ .atomic_check = ade_crtc_atomic_check,
+ .mode_set_nofb = ade_crtc_mode_set_nofb,
+ .atomic_begin = ade_crtc_atomic_begin,
+ .atomic_flush = ade_crtc_atomic_flush,
+};
+
+static const struct drm_crtc_funcs ade_crtc_funcs = {
+ .destroy = drm_crtc_cleanup,
+ .set_config = drm_atomic_helper_set_config,
+ .page_flip = drm_atomic_helper_page_flip,
+ .reset = drm_atomic_helper_crtc_reset,
+ .set_property = drm_atomic_helper_crtc_set_property,
+ .atomic_duplicate_state = drm_atomic_helper_crtc_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
+};
+
+static int ade_crtc_init(struct drm_device *dev, struct drm_crtc *crtc,
+ struct drm_plane *plane)
+{
+ struct kirin_drm_private *priv = dev->dev_private;
+ struct device_node *port;
+ int ret;
+
+ /* set crtc port so that
+ * drm_of_find_possible_crtcs call works
+ */
+ port = of_get_child_by_name(dev->dev->of_node, "port");
+ if (!port) {
+ DRM_ERROR("no port node found in %s\n",
+ dev->dev->of_node->full_name);
+ return -EINVAL;
+ }
+ of_node_put(port);
+ crtc->port = port;
+
+ ret = drm_crtc_init_with_planes(dev, crtc, plane, NULL,
+ &ade_crtc_funcs, NULL);
+ if (ret) {
+ DRM_ERROR("failed to init crtc.\n");
+ return ret;
+ }
+
+ drm_crtc_helper_add(crtc, &ade_crtc_helper_funcs);
+ priv->crtc[drm_crtc_index(crtc)] = crtc;
+
+ return 0;
+}
+
+static void ade_rdma_set(void __iomem *base, struct drm_framebuffer *fb,
+ u32 ch, u32 y, u32 in_h, u32 fmt)
+{
+ struct drm_gem_cma_object *obj = drm_fb_cma_get_gem_obj(fb, 0);
+ u32 reg_ctrl, reg_addr, reg_size, reg_stride, reg_space, reg_en;
+ u32 stride = fb->pitches[0];
+ u32 addr = (u32)obj->paddr + y * stride;
+
+ DRM_DEBUG_DRIVER("rdma%d: (y=%d, height=%d), stride=%d, paddr=0x%x\n",
+ ch + 1, y, in_h, stride, (u32)obj->paddr);
+ DRM_DEBUG_DRIVER("addr=0x%x, fb:%dx%d, pixel_format=%d(%s)\n",
+ addr, fb->width, fb->height, fmt,
+ drm_get_format_name(fb->pixel_format));
+
+ /* get reg offset */
+ reg_ctrl = RD_CH_CTRL(ch);
+ reg_addr = RD_CH_ADDR(ch);
+ reg_size = RD_CH_SIZE(ch);
+ reg_stride = RD_CH_STRIDE(ch);
+ reg_space = RD_CH_SPACE(ch);
+ reg_en = RD_CH_EN(ch);
+
+ /*
+ * TODO: set rotation
+ */
+ writel((fmt << 16) & 0x1f0000, base + reg_ctrl);
+ writel(addr, base + reg_addr);
+ writel((in_h << 16) | stride, base + reg_size);
+ writel(stride, base + reg_stride);
+ writel(in_h * stride, base + reg_space);
+ writel(ADE_ENABLE, base + reg_en);
+ ade_update_reload_bit(base, RDMA_OFST + ch, 0);
+}
+
+static void ade_rdma_disable(void __iomem *base, u32 ch)
+{
+ u32 reg_en;
+
+ /* get reg offset */
+ reg_en = RD_CH_EN(ch);
+ writel(0, base + reg_en);
+ ade_update_reload_bit(base, RDMA_OFST + ch, 1);
+}
+
+static void ade_clip_set(void __iomem *base, u32 ch, u32 fb_w, u32 x,
+ u32 in_w, u32 in_h)
+{
+ u32 disable_val;
+ u32 clip_left;
+ u32 clip_right;
+
+ /*
+ * clip width, no need to clip height
+ */
+ if (fb_w == in_w) { /* bypass */
+ disable_val = 1;
+ clip_left = 0;
+ clip_right = 0;
+ } else {
+ disable_val = 0;
+ clip_left = x;
+ clip_right = fb_w - (x + in_w) - 1;
+ }
+
+ DRM_DEBUG_DRIVER("clip%d: clip_left=%d, clip_right=%d\n",
+ ch + 1, clip_left, clip_right);
+
+ writel(disable_val, base + ADE_CLIP_DISABLE(ch));
+ writel((fb_w - 1) << 16 | (in_h - 1), base + ADE_CLIP_SIZE0(ch));
+ writel(clip_left << 16 | clip_right, base + ADE_CLIP_SIZE1(ch));
+ ade_update_reload_bit(base, CLIP_OFST + ch, 0);
+}
+
+static void ade_clip_disable(void __iomem *base, u32 ch)
+{
+ writel(1, base + ADE_CLIP_DISABLE(ch));
+ ade_update_reload_bit(base, CLIP_OFST + ch, 1);
+}
+
+static bool has_Alpha_channel(int format)
+{
+ switch (format) {
+ case ADE_ARGB_8888:
+ case ADE_ABGR_8888:
+ case ADE_RGBA_8888:
+ case ADE_BGRA_8888:
+ return true;
+ default:
+ return false;
+ }
+}
+
+static void ade_get_blending_params(u32 fmt, u8 glb_alpha, u8 *alp_mode,
+ u8 *alp_sel, u8 *under_alp_sel)
+{
+ bool has_alpha = has_Alpha_channel(fmt);
+
+ /*
+ * get alp_mode
+ */
+ if (has_alpha && glb_alpha < 255)
+ *alp_mode = ADE_ALP_PIXEL_AND_GLB;
+ else if (has_alpha)
+ *alp_mode = ADE_ALP_PIXEL;
+ else
+ *alp_mode = ADE_ALP_GLOBAL;
+
+ /*
+ * get alp sel
+ */
+ *alp_sel = ADE_ALP_MUL_COEFF_3; /* 1 */
+ *under_alp_sel = ADE_ALP_MUL_COEFF_2; /* 0 */
+}
+
+static void ade_compositor_routing_set(void __iomem *base, u8 ch,
+ u32 x0, u32 y0,
+ u32 in_w, u32 in_h, u32 fmt)
+{
+ u8 ovly_ch = 0; /* TODO: This is the zpos, only one plane now */
+ u8 glb_alpha = 255;
+ u32 x1 = x0 + in_w - 1;
+ u32 y1 = y0 + in_h - 1;
+ u32 val;
+ u8 alp_sel;
+ u8 under_alp_sel;
+ u8 alp_mode;
+
+ ade_get_blending_params(fmt, glb_alpha, &alp_mode, &alp_sel,
+ &under_alp_sel);
+
+ /* overlay routing setting
+ */
+ writel(x0 << 16 | y0, base + ADE_OVLY_CH_XY0(ovly_ch));
+ writel(x1 << 16 | y1, base + ADE_OVLY_CH_XY1(ovly_ch));
+ val = (ch + 1) << CH_SEL_OFST | BIT(CH_EN_OFST) |
+ alp_sel << CH_ALP_SEL_OFST |
+ under_alp_sel << CH_UNDER_ALP_SEL_OFST |
+ glb_alpha << CH_ALP_GBL_OFST |
+ alp_mode << CH_ALP_MODE_OFST;
+ writel(val, base + ADE_OVLY_CH_CTL(ovly_ch));
+ /* connect this plane/channel to overlay2 compositor */
+ ade_update_bits(base + ADE_OVLY_CTL, CH_OVLY_SEL_OFST(ovly_ch),
+ CH_OVLY_SEL_MASK, CH_OVLY_SEL_VAL(OUT_OVLY));
+}
+
+static void ade_compositor_routing_disable(void __iomem *base, u32 ch)
+{
+ u8 ovly_ch = 0; /* TODO: Only primary plane now */
+
+ /* disable this plane/channel */
+ ade_update_bits(base + ADE_OVLY_CH_CTL(ovly_ch), CH_EN_OFST,
+ MASK(1), 0);
+ /* dis-connect this plane/channel of overlay2 compositor */
+ ade_update_bits(base + ADE_OVLY_CTL, CH_OVLY_SEL_OFST(ovly_ch),
+ CH_OVLY_SEL_MASK, 0);
+}
+
+/*
+ * Typicaly, a channel looks like: DMA-->clip-->scale-->ctrans-->compositor
+ */
+static void ade_update_channel(struct ade_plane *aplane,
+ struct drm_framebuffer *fb, int crtc_x,
+ int crtc_y, unsigned int crtc_w,
+ unsigned int crtc_h, u32 src_x,
+ u32 src_y, u32 src_w, u32 src_h)
+{
+ struct ade_hw_ctx *ctx = aplane->ctx;
+ void __iomem *base = ctx->base;
+ u32 fmt = ade_get_format(fb->pixel_format);
+ u32 ch = aplane->ch;
+ u32 in_w;
+ u32 in_h;
+
+ DRM_DEBUG_DRIVER("channel%d: src:(%d, %d)-%dx%d, crtc:(%d, %d)-%dx%d",
+ ch + 1, src_x, src_y, src_w, src_h,
+ crtc_x, crtc_y, crtc_w, crtc_h);
+
+ /* 1) DMA setting */
+ in_w = src_w;
+ in_h = src_h;
+ ade_rdma_set(base, fb, ch, src_y, in_h, fmt);
+
+ /* 2) clip setting */
+ ade_clip_set(base, ch, fb->width, src_x, in_w, in_h);
+
+ /* 3) TODO: scale setting for overlay planes */
+
+ /* 4) TODO: ctran/csc setting for overlay planes */
+
+ /* 5) compositor routing setting */
+ ade_compositor_routing_set(base, ch, crtc_x, crtc_y, in_w, in_h, fmt);
+}
+
+static void ade_disable_channel(struct ade_plane *aplane)
+{
+ struct ade_hw_ctx *ctx = aplane->ctx;
+ void __iomem *base = ctx->base;
+ u32 ch = aplane->ch;
+
+ DRM_DEBUG_DRIVER("disable channel%d\n", ch + 1);
+
+ /* disable read DMA */
+ ade_rdma_disable(base, ch);
+
+ /* disable clip */
+ ade_clip_disable(base, ch);
+
+ /* disable compositor routing */
+ ade_compositor_routing_disable(base, ch);
+}
+
+static int ade_plane_prepare_fb(struct drm_plane *plane,
+ const struct drm_plane_state *new_state)
+{
+ /* do nothing */
+ return 0;
+}
+
+static void ade_plane_cleanup_fb(struct drm_plane *plane,
+ const struct drm_plane_state *old_state)
+{
+ /* do nothing */
+}
+
+static int ade_plane_atomic_check(struct drm_plane *plane,
+ struct drm_plane_state *state)
+{
+ struct drm_framebuffer *fb = state->fb;
+ struct drm_crtc *crtc = state->crtc;
+ struct drm_crtc_state *crtc_state;
+ u32 src_x = state->src_x >> 16;
+ u32 src_y = state->src_y >> 16;
+ u32 src_w = state->src_w >> 16;
+ u32 src_h = state->src_h >> 16;
+ int crtc_x = state->crtc_x;
+ int crtc_y = state->crtc_y;
+ u32 crtc_w = state->crtc_w;
+ u32 crtc_h = state->crtc_h;
+ u32 fmt;
+
+ if (!crtc || !fb)
+ return 0;
+
+ fmt = ade_get_format(fb->pixel_format);
+ if (fmt == ADE_FORMAT_UNSUPPORT)
+ return -EINVAL;
+
+ crtc_state = drm_atomic_get_crtc_state(state->state, crtc);
+ if (IS_ERR(crtc_state))
+ return PTR_ERR(crtc_state);
+
+ if (src_w != crtc_w || src_h != crtc_h) {
+ DRM_ERROR("Scale not support!!!\n");
+ return -EINVAL;
+ }
+
+ if (src_x + src_w > fb->width ||
+ src_y + src_h > fb->height)
+ return -EINVAL;
+
+ if (crtc_x < 0 || crtc_y < 0)
+ return -EINVAL;
+
+ if (crtc_x + crtc_w > crtc_state->adjusted_mode.hdisplay ||
+ crtc_y + crtc_h > crtc_state->adjusted_mode.vdisplay)
+ return -EINVAL;
+
+ return 0;
+}
+
+static void ade_plane_atomic_update(struct drm_plane *plane,
+ struct drm_plane_state *old_state)
+{
+ struct drm_plane_state *state = plane->state;
+ struct ade_plane *aplane = to_ade_plane(plane);
+
+ ade_update_channel(aplane, state->fb, state->crtc_x, state->crtc_y,
+ state->crtc_w, state->crtc_h,
+ state->src_x >> 16, state->src_y >> 16,
+ state->src_w >> 16, state->src_h >> 16);
+}
+
+static void ade_plane_atomic_disable(struct drm_plane *plane,
+ struct drm_plane_state *old_state)
+{
+ struct ade_plane *aplane = to_ade_plane(plane);
+
+ ade_disable_channel(aplane);
+}
+
+static const struct drm_plane_helper_funcs ade_plane_helper_funcs = {
+ .prepare_fb = ade_plane_prepare_fb,
+ .cleanup_fb = ade_plane_cleanup_fb,
+ .atomic_check = ade_plane_atomic_check,
+ .atomic_update = ade_plane_atomic_update,
+ .atomic_disable = ade_plane_atomic_disable,
+};
+
+static struct drm_plane_funcs ade_plane_funcs = {
+ .update_plane = drm_atomic_helper_update_plane,
+ .disable_plane = drm_atomic_helper_disable_plane,
+ .set_property = drm_atomic_helper_plane_set_property,
+ .destroy = drm_plane_cleanup,
+ .reset = drm_atomic_helper_plane_reset,
+ .atomic_duplicate_state = drm_atomic_helper_plane_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_plane_destroy_state,
+};
+
+static int ade_plane_init(struct drm_device *dev, struct ade_plane *aplane,
+ enum drm_plane_type type)
+{
+ const u32 *fmts;
+ u32 fmts_cnt;
+ int ret = 0;
+
+ /* get properties */
+ fmts_cnt = ade_get_channel_formats(aplane->ch, &fmts);
+ if (ret)
+ return ret;
+
+ ret = drm_universal_plane_init(dev, &aplane->base, 1, &ade_plane_funcs,
+ fmts, fmts_cnt, type, NULL);
+ if (ret) {
+ DRM_ERROR("fail to init plane, ch=%d\n", aplane->ch);
+ return ret;
+ }
+
+ drm_plane_helper_add(&aplane->base, &ade_plane_helper_funcs);
+
+ return 0;
+}
+
+static int ade_dts_parse(struct platform_device *pdev, struct ade_hw_ctx *ctx)
+{
+ struct resource *res;
+ struct device *dev = &pdev->dev;
+ struct device_node *np = pdev->dev.of_node;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ ctx->base = devm_ioremap_resource(dev, res);
+ if (IS_ERR(ctx->base)) {
+ DRM_ERROR("failed to remap ade io base\n");
+ return PTR_ERR(ctx->base);
+ }
+
+ ctx->reset = devm_reset_control_get(dev, NULL);
+ if (IS_ERR(ctx->reset))
+ return PTR_ERR(ctx->reset);
+
+ ctx->noc_regmap =
+ syscon_regmap_lookup_by_phandle(np, "hisilicon,noc-syscon");
+ if (IS_ERR(ctx->noc_regmap)) {
+ DRM_ERROR("failed to get noc regmap\n");
+ return PTR_ERR(ctx->noc_regmap);
+ }
+
+ ctx->irq = platform_get_irq(pdev, 0);
+ if (ctx->irq < 0) {
+ DRM_ERROR("failed to get irq\n");
+ return -ENODEV;
+ }
+
+ ctx->ade_core_clk = devm_clk_get(dev, "clk_ade_core");
+ if (!ctx->ade_core_clk) {
+ DRM_ERROR("failed to parse clk ADE_CORE\n");
+ return -ENODEV;
+ }
+
+ ctx->media_noc_clk = devm_clk_get(dev, "clk_codec_jpeg");
+ if (!ctx->media_noc_clk) {
+ DRM_ERROR("failed to parse clk CODEC_JPEG\n");
+ return -ENODEV;
+ }
+
+ ctx->ade_pix_clk = devm_clk_get(dev, "clk_ade_pix");
+ if (!ctx->ade_pix_clk) {
+ DRM_ERROR("failed to parse clk ADE_PIX\n");
+ return -ENODEV;
+ }
+
+ return 0;
+}
+
+static int ade_drm_init(struct drm_device *dev)
+{
+ struct platform_device *pdev = dev->platformdev;
+ struct ade_data *ade;
+ struct ade_hw_ctx *ctx;
+ struct ade_crtc *acrtc;
+ struct ade_plane *aplane;
+ enum drm_plane_type type;
+ int ret;
+ int i;
+
+ ade = devm_kzalloc(dev->dev, sizeof(*ade), GFP_KERNEL);
+ if (!ade) {
+ DRM_ERROR("failed to alloc ade_data\n");
+ return -ENOMEM;
+ }
+ platform_set_drvdata(pdev, ade);
+
+ ctx = &ade->ctx;
+ acrtc = &ade->acrtc;
+ acrtc->ctx = ctx;
+ acrtc->out_format = LDI_OUT_RGB_888;
+
+ ret = ade_dts_parse(pdev, ctx);
+ if (ret)
+ return ret;
+
+ /*
+ * plane init
+ * TODO: Now only support primary plane, overlay planes
+ * need to do.
+ */
+ for (i = 0; i < ADE_CH_NUM; i++) {
+ aplane = &ade->aplane[i];
+ aplane->ch = i;
+ aplane->ctx = ctx;
+ type = i == PRIMARY_CH ? DRM_PLANE_TYPE_PRIMARY :
+ DRM_PLANE_TYPE_OVERLAY;
+
+ ret = ade_plane_init(dev, aplane, type);
+ if (ret)
+ return ret;
+ }
+
+ /* crtc init */
+ ret = ade_crtc_init(dev, &acrtc->base, &ade->aplane[PRIMARY_CH].base);
+ if (ret)
+ return ret;
+
+ /* vblank irq init */
+ ret = devm_request_irq(dev->dev, ctx->irq, ade_irq_handler,
+ IRQF_SHARED, dev->driver->name, acrtc);
+ if (ret)
+ return ret;
+ dev->driver->get_vblank_counter = drm_vblank_no_hw_counter;
+ dev->driver->enable_vblank = ade_enable_vblank;
+ dev->driver->disable_vblank = ade_disable_vblank;
+
+ return 0;
+}
+
+static void ade_drm_cleanup(struct drm_device *dev)
+{
+ struct platform_device *pdev = dev->platformdev;
+ struct ade_data *ade = platform_get_drvdata(pdev);
+ struct drm_crtc *crtc = &ade->acrtc.base;
+
+ drm_crtc_cleanup(crtc);
+}
+
+const struct kirin_dc_ops ade_dc_ops = {
+ .init = ade_drm_init,
+ .cleanup = ade_drm_cleanup
+};
diff --git a/drivers/gpu/drm/hisilicon/kirin/kirin_drm_drv.c b/drivers/gpu/drm/hisilicon/kirin/kirin_drm_drv.c
new file mode 100644
index 0000000..3f94785
--- /dev/null
+++ b/drivers/gpu/drm/hisilicon/kirin/kirin_drm_drv.c
@@ -0,0 +1,343 @@
+/*
+ * Hisilicon Kirin SoCs drm master driver
+ *
+ * Copyright (c) 2016 Linaro Limited.
+ * Copyright (c) 2014-2016 Hisilicon Limited.
+ *
+ * Author:
+ * Xinliang Liu <z.liuxinliang@hisilicon.com>
+ * Xinliang Liu <xinliang.liu@linaro.org>
+ * Xinwei Kong <kong.kongxinwei@hisilicon.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#include <linux/of_platform.h>
+#include <linux/component.h>
+#include <linux/of_graph.h>
+
+#include <drm/drmP.h>
+#include <drm/drm_gem_cma_helper.h>
+#include <drm/drm_fb_cma_helper.h>
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_crtc_helper.h>
+
+#include "kirin_drm_drv.h"
+
+static struct kirin_dc_ops *dc_ops;
+
+static int kirin_drm_kms_cleanup(struct drm_device *dev)
+{
+ struct kirin_drm_private *priv = dev->dev_private;
+
+#ifdef CONFIG_DRM_FBDEV_EMULATION
+ if (priv->fbdev) {
+ drm_fbdev_cma_fini(priv->fbdev);
+ priv->fbdev = NULL;
+ }
+#endif
+ drm_kms_helper_poll_fini(dev);
+ drm_vblank_cleanup(dev);
+ dc_ops->cleanup(dev);
+ drm_mode_config_cleanup(dev);
+ devm_kfree(dev->dev, priv);
+ dev->dev_private = NULL;
+
+ return 0;
+}
+
+#ifdef CONFIG_DRM_FBDEV_EMULATION
+static void kirin_fbdev_output_poll_changed(struct drm_device *dev)
+{
+ struct kirin_drm_private *priv = dev->dev_private;
+
+ if (priv->fbdev) {
+ drm_fbdev_cma_hotplug_event(priv->fbdev);
+ } else {
+ priv->fbdev = drm_fbdev_cma_init(dev, 32,
+ dev->mode_config.num_crtc,
+ dev->mode_config.num_connector);
+ if (IS_ERR(priv->fbdev))
+ priv->fbdev = NULL;
+ }
+}
+#endif
+
+static const struct drm_mode_config_funcs kirin_drm_mode_config_funcs = {
+ .fb_create = drm_fb_cma_create,
+#ifdef CONFIG_DRM_FBDEV_EMULATION
+ .output_poll_changed = kirin_fbdev_output_poll_changed,
+#endif
+ .atomic_check = drm_atomic_helper_check,
+ .atomic_commit = drm_atomic_helper_commit,
+};
+
+static void kirin_drm_mode_config_init(struct drm_device *dev)
+{
+ dev->mode_config.min_width = 0;
+ dev->mode_config.min_height = 0;
+
+ dev->mode_config.max_width = 2048;
+ dev->mode_config.max_height = 2048;
+
+ dev->mode_config.funcs = &kirin_drm_mode_config_funcs;
+}
+
+static int kirin_drm_kms_init(struct drm_device *dev)
+{
+ struct kirin_drm_private *priv;
+ int ret;
+
+ priv = devm_kzalloc(dev->dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ dev->dev_private = priv;
+ dev_set_drvdata(dev->dev, dev);
+
+ /* dev->mode_config initialization */
+ drm_mode_config_init(dev);
+ kirin_drm_mode_config_init(dev);
+
+ /* display controller init */
+ ret = dc_ops->init(dev);
+ if (ret)
+ goto err_mode_config_cleanup;
+
+ /* bind and init sub drivers */
+ ret = component_bind_all(dev->dev, dev);
+ if (ret) {
+ DRM_ERROR("failed to bind all component.\n");
+ goto err_dc_cleanup;
+ }
+
+ /* vblank init */
+ ret = drm_vblank_init(dev, dev->mode_config.num_crtc);
+ if (ret) {
+ DRM_ERROR("failed to initialize vblank.\n");
+ goto err_unbind_all;
+ }
+ /* with irq_enabled = true, we can use the vblank feature. */
+ dev->irq_enabled = true;
+
+ /* reset all the states of crtc/plane/encoder/connector */
+ drm_mode_config_reset(dev);
+
+ /* init kms poll for handling hpd */
+ drm_kms_helper_poll_init(dev);
+
+ /* force detection after connectors init */
+ (void)drm_helper_hpd_irq_event(dev);
+
+ return 0;
+
+err_unbind_all:
+ component_unbind_all(dev->dev, dev);
+err_dc_cleanup:
+ dc_ops->cleanup(dev);
+err_mode_config_cleanup:
+ drm_mode_config_cleanup(dev);
+ devm_kfree(dev->dev, priv);
+ dev->dev_private = NULL;
+
+ return ret;
+}
+
+static const struct file_operations kirin_drm_fops = {
+ .owner = THIS_MODULE,
+ .open = drm_open,
+ .release = drm_release,
+ .unlocked_ioctl = drm_ioctl,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = drm_compat_ioctl,
+#endif
+ .poll = drm_poll,
+ .read = drm_read,
+ .llseek = no_llseek,
+ .mmap = drm_gem_cma_mmap,
+};
+
+static int kirin_gem_cma_dumb_create(struct drm_file *file,
+ struct drm_device *dev,
+ struct drm_mode_create_dumb *args)
+{
+ return drm_gem_cma_dumb_create_internal(file, dev, args);
+}
+
+static struct drm_driver kirin_drm_driver = {
+ .driver_features = DRIVER_GEM | DRIVER_MODESET | DRIVER_PRIME |
+ DRIVER_ATOMIC | DRIVER_HAVE_IRQ,
+ .fops = &kirin_drm_fops,
+ .set_busid = drm_platform_set_busid,
+
+ .gem_free_object = drm_gem_cma_free_object,
+ .gem_vm_ops = &drm_gem_cma_vm_ops,
+ .dumb_create = kirin_gem_cma_dumb_create,
+ .dumb_map_offset = drm_gem_cma_dumb_map_offset,
+ .dumb_destroy = drm_gem_dumb_destroy,
+
+ .prime_handle_to_fd = drm_gem_prime_handle_to_fd,
+ .prime_fd_to_handle = drm_gem_prime_fd_to_handle,
+ .gem_prime_export = drm_gem_prime_export,
+ .gem_prime_import = drm_gem_prime_import,
+ .gem_prime_get_sg_table = drm_gem_cma_prime_get_sg_table,
+ .gem_prime_import_sg_table = drm_gem_cma_prime_import_sg_table,
+ .gem_prime_vmap = drm_gem_cma_prime_vmap,
+ .gem_prime_vunmap = drm_gem_cma_prime_vunmap,
+ .gem_prime_mmap = drm_gem_cma_prime_mmap,
+
+ .name = "kirin",
+ .desc = "Hisilicon Kirin SoCs' DRM Driver",
+ .date = "20150718",
+ .major = 1,
+ .minor = 0,
+};
+
+static int compare_of(struct device *dev, void *data)
+{
+ return dev->of_node == data;
+}
+
+static int kirin_drm_bind(struct device *dev)
+{
+ struct drm_driver *driver = &kirin_drm_driver;
+ struct drm_device *drm_dev;
+ int ret;
+
+ drm_dev = drm_dev_alloc(driver, dev);
+ if (!drm_dev)
+ return -ENOMEM;
+
+ drm_dev->platformdev = to_platform_device(dev);
+
+ ret = kirin_drm_kms_init(drm_dev);
+ if (ret)
+ goto err_drm_dev_unref;
+
+ ret = drm_dev_register(drm_dev, 0);
+ if (ret)
+ goto err_kms_cleanup;
+
+ /* connectors should be registered after drm device register */
+ ret = drm_connector_register_all(drm_dev);
+ if (ret)
+ goto err_drm_dev_unregister;
+
+ DRM_INFO("Initialized %s %d.%d.%d %s on minor %d\n",
+ driver->name, driver->major, driver->minor, driver->patchlevel,
+ driver->date, drm_dev->primary->index);
+
+ return 0;
+
+err_drm_dev_unregister:
+ drm_dev_unregister(drm_dev);
+err_kms_cleanup:
+ kirin_drm_kms_cleanup(drm_dev);
+err_drm_dev_unref:
+ drm_dev_unref(drm_dev);
+
+ return ret;
+}
+
+static void kirin_drm_unbind(struct device *dev)
+{
+ struct drm_device *drm_dev = dev_get_drvdata(dev);
+
+ drm_connector_unregister_all(drm_dev);
+ drm_dev_unregister(drm_dev);
+ kirin_drm_kms_cleanup(drm_dev);
+ drm_dev_unref(drm_dev);
+}
+
+static const struct component_master_ops kirin_drm_ops = {
+ .bind = kirin_drm_bind,
+ .unbind = kirin_drm_unbind,
+};
+
+static struct device_node *kirin_get_remote_node(struct device_node *np)
+{
+ struct device_node *endpoint, *remote;
+
+ /* get the first endpoint, in our case only one remote node
+ * is connected to display controller.
+ */
+ endpoint = of_graph_get_next_endpoint(np, NULL);
+ if (!endpoint) {
+ DRM_ERROR("no valid endpoint node\n");
+ return ERR_PTR(-ENODEV);
+ }
+ of_node_put(endpoint);
+
+ remote = of_graph_get_remote_port_parent(endpoint);
+ if (!remote) {
+ DRM_ERROR("no valid remote node\n");
+ return ERR_PTR(-ENODEV);
+ }
+ of_node_put(remote);
+
+ if (!of_device_is_available(remote)) {
+ DRM_ERROR("not available for remote node\n");
+ return ERR_PTR(-ENODEV);
+ }
+
+ return remote;
+}
+
+static int kirin_drm_platform_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *np = dev->of_node;
+ struct component_match *match = NULL;
+ struct device_node *remote;
+
+ dc_ops = (struct kirin_dc_ops *)of_device_get_match_data(dev);
+ if (!dc_ops) {
+ DRM_ERROR("failed to get dt id data\n");
+ return -EINVAL;
+ }
+
+ remote = kirin_get_remote_node(np);
+ if (IS_ERR(remote))
+ return PTR_ERR(remote);
+
+ component_match_add(dev, &match, compare_of, remote);
+
+ return component_master_add_with_match(dev, &kirin_drm_ops, match);
+
+ return 0;
+}
+
+static int kirin_drm_platform_remove(struct platform_device *pdev)
+{
+ component_master_del(&pdev->dev, &kirin_drm_ops);
+ dc_ops = NULL;
+ return 0;
+}
+
+static const struct of_device_id kirin_drm_dt_ids[] = {
+ { .compatible = "hisilicon,hi6220-ade",
+ .data = &ade_dc_ops,
+ },
+ { /* end node */ },
+};
+MODULE_DEVICE_TABLE(of, kirin_drm_dt_ids);
+
+static struct platform_driver kirin_drm_platform_driver = {
+ .probe = kirin_drm_platform_probe,
+ .remove = kirin_drm_platform_remove,
+ .driver = {
+ .name = "kirin-drm",
+ .of_match_table = kirin_drm_dt_ids,
+ },
+};
+
+module_platform_driver(kirin_drm_platform_driver);
+
+MODULE_AUTHOR("Xinliang Liu <xinliang.liu@linaro.org>");
+MODULE_AUTHOR("Xinliang Liu <z.liuxinliang@hisilicon.com>");
+MODULE_AUTHOR("Xinwei Kong <kong.kongxinwei@hisilicon.com>");
+MODULE_DESCRIPTION("hisilicon Kirin SoCs' DRM master driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/gpu/drm/hisilicon/kirin/kirin_drm_drv.h b/drivers/gpu/drm/hisilicon/kirin/kirin_drm_drv.h
new file mode 100644
index 0000000..1a07caf
--- /dev/null
+++ b/drivers/gpu/drm/hisilicon/kirin/kirin_drm_drv.h
@@ -0,0 +1,31 @@
+/*
+ * Copyright (c) 2016 Linaro Limited.
+ * Copyright (c) 2014-2016 Hisilicon Limited.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#ifndef __KIRIN_DRM_DRV_H__
+#define __KIRIN_DRM_DRV_H__
+
+#define MAX_CRTC 2
+
+/* display controller init/cleanup ops */
+struct kirin_dc_ops {
+ int (*init)(struct drm_device *dev);
+ void (*cleanup)(struct drm_device *dev);
+};
+
+struct kirin_drm_private {
+ struct drm_crtc *crtc[MAX_CRTC];
+#ifdef CONFIG_DRM_FBDEV_EMULATION
+ struct drm_fbdev_cma *fbdev;
+#endif
+};
+
+extern const struct kirin_dc_ops ade_dc_ops;
+
+#endif /* __KIRIN_DRM_DRV_H__ */
diff --git a/drivers/gpu/drm/i915/Kconfig b/drivers/gpu/drm/i915/Kconfig
index 20a5d04..29a32b1 100644
--- a/drivers/gpu/drm/i915/Kconfig
+++ b/drivers/gpu/drm/i915/Kconfig
@@ -56,3 +56,9 @@
selected to enabled full userptr support.
If in doubt, say "Y".
+
+menu "drm/i915 Debugging"
+depends on DRM_I915
+depends on EXPERT
+source drivers/gpu/drm/i915/Kconfig.debug
+endmenu
diff --git a/drivers/gpu/drm/i915/Kconfig.debug b/drivers/gpu/drm/i915/Kconfig.debug
new file mode 100644
index 0000000..8f40410
--- /dev/null
+++ b/drivers/gpu/drm/i915/Kconfig.debug
@@ -0,0 +1,41 @@
+config DRM_I915_WERROR
+ bool "Force GCC to throw an error instead of a warning when compiling"
+ # As this may inadvertently break the build, only allow the user
+ # to shoot oneself in the foot iff they aim really hard
+ depends on EXPERT
+ # We use the dependency on !COMPILE_TEST to not be enabled in
+ # allmodconfig or allyesconfig configurations
+ depends on !COMPILE_TEST
+ default n
+ help
+ Add -Werror to the build flags for (and only for) i915.ko.
+ Do not enable this unless you are writing code for the i915.ko module.
+
+ Recommended for driver developers only.
+
+ If in doubt, say "N".
+
+config DRM_I915_DEBUG
+ bool "Enable additional driver debugging"
+ depends on DRM_I915
+ default n
+ help
+ Choose this option to turn on extra driver debugging that may affect
+ performance but will catch some internal issues.
+
+ Recommended for driver developers only.
+
+ If in doubt, say "N".
+
+config DRM_I915_DEBUG_GEM
+ bool "Insert extra checks into the GEM internals"
+ default n
+ depends on DRM_I915_WERROR
+ help
+ Enable extra sanity checks (including BUGs) along the GEM driver
+ paths that may slow the system down and if hit hang the machine.
+
+ Recommended for driver developers only.
+
+ If in doubt, say "N".
+
diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile
index 0851de07..0b88ba0 100644
--- a/drivers/gpu/drm/i915/Makefile
+++ b/drivers/gpu/drm/i915/Makefile
@@ -2,6 +2,8 @@
# Makefile for the drm device driver. This driver provides support for the
# Direct Rendering Infrastructure (DRI) in XFree86 4.1.0 and higher.
+subdir-ccflags-$(CONFIG_DRM_I915_WERROR) := -Werror
+
# Please keep these build lists sorted!
# core driver code
@@ -55,7 +57,9 @@
intel_atomic.o \
intel_atomic_plane.o \
intel_bios.o \
+ intel_color.o \
intel_display.o \
+ intel_dpll_mgr.o \
intel_fbc.o \
intel_fifo_underrun.o \
intel_frontbuffer.o \
diff --git a/drivers/gpu/drm/i915/i915_cmd_parser.c b/drivers/gpu/drm/i915/i915_cmd_parser.c
index 814d894..a337f33 100644
--- a/drivers/gpu/drm/i915/i915_cmd_parser.c
+++ b/drivers/gpu/drm/i915/i915_cmd_parser.c
@@ -444,6 +444,7 @@
REG64(CL_PRIMITIVES_COUNT),
REG64(PS_INVOCATION_COUNT),
REG64(PS_DEPTH_COUNT),
+ REG64_IDX(RING_TIMESTAMP, RENDER_RING_BASE),
REG32(OACONTROL), /* Only allowed for LRI and SRM. See below. */
REG64(MI_PREDICATE_SRC0),
REG64(MI_PREDICATE_SRC1),
@@ -471,6 +472,25 @@
REG32(GEN7_L3SQCREG1),
REG32(GEN7_L3CNTLREG2),
REG32(GEN7_L3CNTLREG3),
+};
+
+static const struct drm_i915_reg_descriptor hsw_render_regs[] = {
+ REG64_IDX(HSW_CS_GPR, 0),
+ REG64_IDX(HSW_CS_GPR, 1),
+ REG64_IDX(HSW_CS_GPR, 2),
+ REG64_IDX(HSW_CS_GPR, 3),
+ REG64_IDX(HSW_CS_GPR, 4),
+ REG64_IDX(HSW_CS_GPR, 5),
+ REG64_IDX(HSW_CS_GPR, 6),
+ REG64_IDX(HSW_CS_GPR, 7),
+ REG64_IDX(HSW_CS_GPR, 8),
+ REG64_IDX(HSW_CS_GPR, 9),
+ REG64_IDX(HSW_CS_GPR, 10),
+ REG64_IDX(HSW_CS_GPR, 11),
+ REG64_IDX(HSW_CS_GPR, 12),
+ REG64_IDX(HSW_CS_GPR, 13),
+ REG64_IDX(HSW_CS_GPR, 14),
+ REG64_IDX(HSW_CS_GPR, 15),
REG32(HSW_SCRATCH1,
.mask = ~HSW_SCRATCH1_L3_DATA_ATOMICS_DISABLE,
.value = 0),
@@ -500,6 +520,33 @@
#undef REG64
#undef REG32
+struct drm_i915_reg_table {
+ const struct drm_i915_reg_descriptor *regs;
+ int num_regs;
+ bool master;
+};
+
+static const struct drm_i915_reg_table ivb_render_reg_tables[] = {
+ { gen7_render_regs, ARRAY_SIZE(gen7_render_regs), false },
+ { ivb_master_regs, ARRAY_SIZE(ivb_master_regs), true },
+};
+
+static const struct drm_i915_reg_table ivb_blt_reg_tables[] = {
+ { gen7_blt_regs, ARRAY_SIZE(gen7_blt_regs), false },
+ { ivb_master_regs, ARRAY_SIZE(ivb_master_regs), true },
+};
+
+static const struct drm_i915_reg_table hsw_render_reg_tables[] = {
+ { gen7_render_regs, ARRAY_SIZE(gen7_render_regs), false },
+ { hsw_render_regs, ARRAY_SIZE(hsw_render_regs), false },
+ { hsw_master_regs, ARRAY_SIZE(hsw_master_regs), true },
+};
+
+static const struct drm_i915_reg_table hsw_blt_reg_tables[] = {
+ { gen7_blt_regs, ARRAY_SIZE(gen7_blt_regs), false },
+ { hsw_master_regs, ARRAY_SIZE(hsw_master_regs), true },
+};
+
static u32 gen7_render_get_cmd_length_mask(u32 cmd_header)
{
u32 client = (cmd_header & INSTR_CLIENT_MASK) >> INSTR_CLIENT_SHIFT;
@@ -555,7 +602,7 @@
return 0;
}
-static bool validate_cmds_sorted(struct intel_engine_cs *ring,
+static bool validate_cmds_sorted(struct intel_engine_cs *engine,
const struct drm_i915_cmd_table *cmd_tables,
int cmd_table_count)
{
@@ -577,7 +624,7 @@
if (curr < previous) {
DRM_ERROR("CMD: table not sorted ring=%d table=%d entry=%d cmd=0x%08X prev=0x%08X\n",
- ring->id, i, j, curr, previous);
+ engine->id, i, j, curr, previous);
ret = false;
}
@@ -611,11 +658,18 @@
return ret;
}
-static bool validate_regs_sorted(struct intel_engine_cs *ring)
+static bool validate_regs_sorted(struct intel_engine_cs *engine)
{
- return check_sorted(ring->id, ring->reg_table, ring->reg_count) &&
- check_sorted(ring->id, ring->master_reg_table,
- ring->master_reg_count);
+ int i;
+ const struct drm_i915_reg_table *table;
+
+ for (i = 0; i < engine->reg_table_count; i++) {
+ table = &engine->reg_tables[i];
+ if (!check_sorted(engine->id, table->regs, table->num_regs))
+ return false;
+ }
+
+ return true;
}
struct cmd_node {
@@ -639,13 +693,13 @@
*/
#define CMD_HASH_MASK STD_MI_OPCODE_MASK
-static int init_hash_table(struct intel_engine_cs *ring,
+static int init_hash_table(struct intel_engine_cs *engine,
const struct drm_i915_cmd_table *cmd_tables,
int cmd_table_count)
{
int i, j;
- hash_init(ring->cmd_hash);
+ hash_init(engine->cmd_hash);
for (i = 0; i < cmd_table_count; i++) {
const struct drm_i915_cmd_table *table = &cmd_tables[i];
@@ -660,7 +714,7 @@
return -ENOMEM;
desc_node->desc = desc;
- hash_add(ring->cmd_hash, &desc_node->node,
+ hash_add(engine->cmd_hash, &desc_node->node,
desc->cmd.value & CMD_HASH_MASK);
}
}
@@ -668,13 +722,13 @@
return 0;
}
-static void fini_hash_table(struct intel_engine_cs *ring)
+static void fini_hash_table(struct intel_engine_cs *engine)
{
struct hlist_node *tmp;
struct cmd_node *desc_node;
int i;
- hash_for_each_safe(ring->cmd_hash, i, tmp, desc_node, node) {
+ hash_for_each_safe(engine->cmd_hash, i, tmp, desc_node, node) {
hash_del(&desc_node->node);
kfree(desc_node);
}
@@ -690,18 +744,18 @@
*
* Return: non-zero if initialization fails
*/
-int i915_cmd_parser_init_ring(struct intel_engine_cs *ring)
+int i915_cmd_parser_init_ring(struct intel_engine_cs *engine)
{
const struct drm_i915_cmd_table *cmd_tables;
int cmd_table_count;
int ret;
- if (!IS_GEN7(ring->dev))
+ if (!IS_GEN7(engine->dev))
return 0;
- switch (ring->id) {
+ switch (engine->id) {
case RCS:
- if (IS_HASWELL(ring->dev)) {
+ if (IS_HASWELL(engine->dev)) {
cmd_tables = hsw_render_ring_cmds;
cmd_table_count =
ARRAY_SIZE(hsw_render_ring_cmds);
@@ -710,26 +764,23 @@
cmd_table_count = ARRAY_SIZE(gen7_render_cmds);
}
- ring->reg_table = gen7_render_regs;
- ring->reg_count = ARRAY_SIZE(gen7_render_regs);
-
- if (IS_HASWELL(ring->dev)) {
- ring->master_reg_table = hsw_master_regs;
- ring->master_reg_count = ARRAY_SIZE(hsw_master_regs);
+ if (IS_HASWELL(engine->dev)) {
+ engine->reg_tables = hsw_render_reg_tables;
+ engine->reg_table_count = ARRAY_SIZE(hsw_render_reg_tables);
} else {
- ring->master_reg_table = ivb_master_regs;
- ring->master_reg_count = ARRAY_SIZE(ivb_master_regs);
+ engine->reg_tables = ivb_render_reg_tables;
+ engine->reg_table_count = ARRAY_SIZE(ivb_render_reg_tables);
}
- ring->get_cmd_length_mask = gen7_render_get_cmd_length_mask;
+ engine->get_cmd_length_mask = gen7_render_get_cmd_length_mask;
break;
case VCS:
cmd_tables = gen7_video_cmds;
cmd_table_count = ARRAY_SIZE(gen7_video_cmds);
- ring->get_cmd_length_mask = gen7_bsd_get_cmd_length_mask;
+ engine->get_cmd_length_mask = gen7_bsd_get_cmd_length_mask;
break;
case BCS:
- if (IS_HASWELL(ring->dev)) {
+ if (IS_HASWELL(engine->dev)) {
cmd_tables = hsw_blt_ring_cmds;
cmd_table_count = ARRAY_SIZE(hsw_blt_ring_cmds);
} else {
@@ -737,44 +788,41 @@
cmd_table_count = ARRAY_SIZE(gen7_blt_cmds);
}
- ring->reg_table = gen7_blt_regs;
- ring->reg_count = ARRAY_SIZE(gen7_blt_regs);
-
- if (IS_HASWELL(ring->dev)) {
- ring->master_reg_table = hsw_master_regs;
- ring->master_reg_count = ARRAY_SIZE(hsw_master_regs);
+ if (IS_HASWELL(engine->dev)) {
+ engine->reg_tables = hsw_blt_reg_tables;
+ engine->reg_table_count = ARRAY_SIZE(hsw_blt_reg_tables);
} else {
- ring->master_reg_table = ivb_master_regs;
- ring->master_reg_count = ARRAY_SIZE(ivb_master_regs);
+ engine->reg_tables = ivb_blt_reg_tables;
+ engine->reg_table_count = ARRAY_SIZE(ivb_blt_reg_tables);
}
- ring->get_cmd_length_mask = gen7_blt_get_cmd_length_mask;
+ engine->get_cmd_length_mask = gen7_blt_get_cmd_length_mask;
break;
case VECS:
cmd_tables = hsw_vebox_cmds;
cmd_table_count = ARRAY_SIZE(hsw_vebox_cmds);
/* VECS can use the same length_mask function as VCS */
- ring->get_cmd_length_mask = gen7_bsd_get_cmd_length_mask;
+ engine->get_cmd_length_mask = gen7_bsd_get_cmd_length_mask;
break;
default:
DRM_ERROR("CMD: cmd_parser_init with unknown ring: %d\n",
- ring->id);
+ engine->id);
BUG();
}
- BUG_ON(!validate_cmds_sorted(ring, cmd_tables, cmd_table_count));
- BUG_ON(!validate_regs_sorted(ring));
+ BUG_ON(!validate_cmds_sorted(engine, cmd_tables, cmd_table_count));
+ BUG_ON(!validate_regs_sorted(engine));
- WARN_ON(!hash_empty(ring->cmd_hash));
+ WARN_ON(!hash_empty(engine->cmd_hash));
- ret = init_hash_table(ring, cmd_tables, cmd_table_count);
+ ret = init_hash_table(engine, cmd_tables, cmd_table_count);
if (ret) {
DRM_ERROR("CMD: cmd_parser_init failed!\n");
- fini_hash_table(ring);
+ fini_hash_table(engine);
return ret;
}
- ring->needs_cmd_parser = true;
+ engine->needs_cmd_parser = true;
return 0;
}
@@ -786,21 +834,21 @@
* Releases any resources related to command parsing that may have been
* initialized for the specified ring.
*/
-void i915_cmd_parser_fini_ring(struct intel_engine_cs *ring)
+void i915_cmd_parser_fini_ring(struct intel_engine_cs *engine)
{
- if (!ring->needs_cmd_parser)
+ if (!engine->needs_cmd_parser)
return;
- fini_hash_table(ring);
+ fini_hash_table(engine);
}
static const struct drm_i915_cmd_descriptor*
-find_cmd_in_table(struct intel_engine_cs *ring,
+find_cmd_in_table(struct intel_engine_cs *engine,
u32 cmd_header)
{
struct cmd_node *desc_node;
- hash_for_each_possible(ring->cmd_hash, desc_node, node,
+ hash_for_each_possible(engine->cmd_hash, desc_node, node,
cmd_header & CMD_HASH_MASK) {
const struct drm_i915_cmd_descriptor *desc = desc_node->desc;
u32 masked_cmd = desc->cmd.mask & cmd_header;
@@ -822,18 +870,18 @@
* ring's default length encoding and returns default_desc.
*/
static const struct drm_i915_cmd_descriptor*
-find_cmd(struct intel_engine_cs *ring,
+find_cmd(struct intel_engine_cs *engine,
u32 cmd_header,
struct drm_i915_cmd_descriptor *default_desc)
{
const struct drm_i915_cmd_descriptor *desc;
u32 mask;
- desc = find_cmd_in_table(ring, cmd_header);
+ desc = find_cmd_in_table(engine, cmd_header);
if (desc)
return desc;
- mask = ring->get_cmd_length_mask(cmd_header);
+ mask = engine->get_cmd_length_mask(cmd_header);
if (!mask)
return NULL;
@@ -848,12 +896,31 @@
find_reg(const struct drm_i915_reg_descriptor *table,
int count, u32 addr)
{
- if (table) {
- int i;
+ int i;
- for (i = 0; i < count; i++) {
- if (i915_mmio_reg_offset(table[i].addr) == addr)
- return &table[i];
+ for (i = 0; i < count; i++) {
+ if (i915_mmio_reg_offset(table[i].addr) == addr)
+ return &table[i];
+ }
+
+ return NULL;
+}
+
+static const struct drm_i915_reg_descriptor *
+find_reg_in_tables(const struct drm_i915_reg_table *tables,
+ int count, bool is_master, u32 addr)
+{
+ int i;
+ const struct drm_i915_reg_table *table;
+ const struct drm_i915_reg_descriptor *reg;
+
+ for (i = 0; i < count; i++) {
+ table = &tables[i];
+ if (!table->master || is_master) {
+ reg = find_reg(table->regs, table->num_regs,
+ addr);
+ if (reg != NULL)
+ return reg;
}
}
@@ -963,18 +1030,18 @@
*
* Return: true if the ring requires software command parsing
*/
-bool i915_needs_cmd_parser(struct intel_engine_cs *ring)
+bool i915_needs_cmd_parser(struct intel_engine_cs *engine)
{
- if (!ring->needs_cmd_parser)
+ if (!engine->needs_cmd_parser)
return false;
- if (!USES_PPGTT(ring->dev))
+ if (!USES_PPGTT(engine->dev))
return false;
return (i915.enable_cmd_parser == 1);
}
-static bool check_cmd(const struct intel_engine_cs *ring,
+static bool check_cmd(const struct intel_engine_cs *engine,
const struct drm_i915_cmd_descriptor *desc,
const u32 *cmd, u32 length,
const bool is_master,
@@ -1004,17 +1071,14 @@
offset += step) {
const u32 reg_addr = cmd[offset] & desc->reg.mask;
const struct drm_i915_reg_descriptor *reg =
- find_reg(ring->reg_table, ring->reg_count,
- reg_addr);
-
- if (!reg && is_master)
- reg = find_reg(ring->master_reg_table,
- ring->master_reg_count,
- reg_addr);
+ find_reg_in_tables(engine->reg_tables,
+ engine->reg_table_count,
+ is_master,
+ reg_addr);
if (!reg) {
DRM_DEBUG_DRIVER("CMD: Rejected register 0x%08X in command: 0x%08X (ring=%d)\n",
- reg_addr, *cmd, ring->id);
+ reg_addr, *cmd, engine->id);
return false;
}
@@ -1087,7 +1151,7 @@
*cmd,
desc->bits[i].mask,
desc->bits[i].expected,
- dword, ring->id);
+ dword, engine->id);
return false;
}
}
@@ -1113,7 +1177,7 @@
* Return: non-zero if the parser finds violations or otherwise fails; -EACCES
* if the batch appears legal but should use hardware parsing
*/
-int i915_parse_cmds(struct intel_engine_cs *ring,
+int i915_parse_cmds(struct intel_engine_cs *engine,
struct drm_i915_gem_object *batch_obj,
struct drm_i915_gem_object *shadow_batch_obj,
u32 batch_start_offset,
@@ -1147,7 +1211,7 @@
if (*cmd == MI_BATCH_BUFFER_END)
break;
- desc = find_cmd(ring, *cmd, &default_desc);
+ desc = find_cmd(engine, *cmd, &default_desc);
if (!desc) {
DRM_DEBUG_DRIVER("CMD: Unrecognized command: 0x%08X\n",
*cmd);
@@ -1179,7 +1243,7 @@
break;
}
- if (!check_cmd(ring, desc, cmd, length, is_master,
+ if (!check_cmd(engine, desc, cmd, length, is_master,
&oacontrol_set)) {
ret = -EINVAL;
break;
@@ -1223,6 +1287,7 @@
* 3. Allow access to the GPGPU_THREADS_DISPATCHED register.
* 4. L3 atomic chicken bits of HSW_SCRATCH1 and HSW_ROW_CHICKEN3.
* 5. GPGPU dispatch compute indirect registers.
+ * 6. TIMESTAMP register and Haswell CS GPR registers
*/
- return 5;
+ return 6;
}
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index e3f4c72..3269033 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -89,27 +89,34 @@
return 0;
}
-static const char *get_pin_flag(struct drm_i915_gem_object *obj)
+static const char get_active_flag(struct drm_i915_gem_object *obj)
{
- if (obj->pin_display)
- return "p";
- else
- return " ";
+ return obj->active ? '*' : ' ';
}
-static const char *get_tiling_flag(struct drm_i915_gem_object *obj)
+static const char get_pin_flag(struct drm_i915_gem_object *obj)
+{
+ return obj->pin_display ? 'p' : ' ';
+}
+
+static const char get_tiling_flag(struct drm_i915_gem_object *obj)
{
switch (obj->tiling_mode) {
default:
- case I915_TILING_NONE: return " ";
- case I915_TILING_X: return "X";
- case I915_TILING_Y: return "Y";
+ case I915_TILING_NONE: return ' ';
+ case I915_TILING_X: return 'X';
+ case I915_TILING_Y: return 'Y';
}
}
-static inline const char *get_global_flag(struct drm_i915_gem_object *obj)
+static inline const char get_global_flag(struct drm_i915_gem_object *obj)
{
- return i915_gem_obj_to_ggtt(obj) ? "g" : " ";
+ return i915_gem_obj_to_ggtt(obj) ? 'g' : ' ';
+}
+
+static inline const char get_pin_mapped_flag(struct drm_i915_gem_object *obj)
+{
+ return obj->mapping ? 'M' : ' ';
}
static u64 i915_gem_obj_total_ggtt_size(struct drm_i915_gem_object *obj)
@@ -129,23 +136,26 @@
describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj)
{
struct drm_i915_private *dev_priv = to_i915(obj->base.dev);
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
struct i915_vma *vma;
int pin_count = 0;
- int i;
+ enum intel_engine_id id;
- seq_printf(m, "%pK: %s%s%s%s %8zdKiB %02x %02x [ ",
+ lockdep_assert_held(&obj->base.dev->struct_mutex);
+
+ seq_printf(m, "%pK: %c%c%c%c%c %8zdKiB %02x %02x [ ",
&obj->base,
- obj->active ? "*" : " ",
+ get_active_flag(obj),
get_pin_flag(obj),
get_tiling_flag(obj),
get_global_flag(obj),
+ get_pin_mapped_flag(obj),
obj->base.size / 1024,
obj->base.read_domains,
obj->base.write_domain);
- for_each_ring(ring, dev_priv, i)
+ for_each_engine_id(engine, dev_priv, id)
seq_printf(m, "%x ",
- i915_gem_request_get_seqno(obj->last_read_req[i]));
+ i915_gem_request_get_seqno(obj->last_read_req[id]));
seq_printf(m, "] %x %x%s%s%s",
i915_gem_request_get_seqno(obj->last_write_req),
i915_gem_request_get_seqno(obj->last_fenced_req),
@@ -184,7 +194,7 @@
}
if (obj->last_write_req != NULL)
seq_printf(m, " (%s)",
- i915_gem_request_get_ring(obj->last_write_req)->name);
+ i915_gem_request_get_engine(obj->last_write_req)->name);
if (obj->frontbuffer_bits)
seq_printf(m, " (frontbuffer: 0x%03x)", obj->frontbuffer_bits);
}
@@ -202,8 +212,8 @@
uintptr_t list = (uintptr_t) node->info_ent->data;
struct list_head *head;
struct drm_device *dev = node->minor->dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
- struct i915_address_space *vm = &dev_priv->gtt.base;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct i915_vma *vma;
u64 total_obj_size, total_gtt_size;
int count, ret;
@@ -216,11 +226,11 @@
switch (list) {
case ACTIVE_LIST:
seq_puts(m, "Active:\n");
- head = &vm->active_list;
+ head = &ggtt->base.active_list;
break;
case INACTIVE_LIST:
seq_puts(m, "Inactive:\n");
- head = &vm->inactive_list;
+ head = &ggtt->base.inactive_list;
break;
default:
mutex_unlock(&dev->struct_mutex);
@@ -397,15 +407,15 @@
{
struct drm_i915_gem_object *obj;
struct file_stats stats;
- struct intel_engine_cs *ring;
- int i, j;
+ struct intel_engine_cs *engine;
+ int j;
memset(&stats, 0, sizeof(stats));
- for_each_ring(ring, dev_priv, i) {
- for (j = 0; j < ARRAY_SIZE(ring->batch_pool.cache_list); j++) {
+ for_each_engine(engine, dev_priv) {
+ for (j = 0; j < ARRAY_SIZE(engine->batch_pool.cache_list); j++) {
list_for_each_entry(obj,
- &ring->batch_pool.cache_list[j],
+ &engine->batch_pool.cache_list[j],
batch_pool_link)
per_file_stats(0, obj, &stats);
}
@@ -429,11 +439,13 @@
{
struct drm_info_node *node = m->private;
struct drm_device *dev = node->minor->dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
u32 count, mappable_count, purgeable_count;
u64 size, mappable_size, purgeable_size;
+ unsigned long pin_mapped_count = 0, pin_mapped_purgeable_count = 0;
+ u64 pin_mapped_size = 0, pin_mapped_purgeable_size = 0;
struct drm_i915_gem_object *obj;
- struct i915_address_space *vm = &dev_priv->gtt.base;
struct drm_file *file;
struct i915_vma *vma;
int ret;
@@ -452,12 +464,12 @@
count, mappable_count, size, mappable_size);
size = count = mappable_size = mappable_count = 0;
- count_vmas(&vm->active_list, vm_link);
+ count_vmas(&ggtt->base.active_list, vm_link);
seq_printf(m, " %u [%u] active objects, %llu [%llu] bytes\n",
count, mappable_count, size, mappable_size);
size = count = mappable_size = mappable_count = 0;
- count_vmas(&vm->inactive_list, vm_link);
+ count_vmas(&ggtt->base.inactive_list, vm_link);
seq_printf(m, " %u [%u] inactive objects, %llu [%llu] bytes\n",
count, mappable_count, size, mappable_size);
@@ -466,6 +478,14 @@
size += obj->base.size, ++count;
if (obj->madv == I915_MADV_DONTNEED)
purgeable_size += obj->base.size, ++purgeable_count;
+ if (obj->mapping) {
+ pin_mapped_count++;
+ pin_mapped_size += obj->base.size;
+ if (obj->pages_pin_count == 0) {
+ pin_mapped_purgeable_count++;
+ pin_mapped_purgeable_size += obj->base.size;
+ }
+ }
}
seq_printf(m, "%u unbound objects, %llu bytes\n", count, size);
@@ -483,6 +503,14 @@
purgeable_size += obj->base.size;
++purgeable_count;
}
+ if (obj->mapping) {
+ pin_mapped_count++;
+ pin_mapped_size += obj->base.size;
+ if (obj->pages_pin_count == 0) {
+ pin_mapped_purgeable_count++;
+ pin_mapped_purgeable_size += obj->base.size;
+ }
+ }
}
seq_printf(m, "%u purgeable objects, %llu bytes\n",
purgeable_count, purgeable_size);
@@ -490,13 +518,20 @@
mappable_count, mappable_size);
seq_printf(m, "%u fault mappable objects, %llu bytes\n",
count, size);
+ seq_printf(m,
+ "%lu [%lu] pin mapped objects, %llu [%llu] bytes [purgeable]\n",
+ pin_mapped_count, pin_mapped_purgeable_count,
+ pin_mapped_size, pin_mapped_purgeable_size);
seq_printf(m, "%llu [%llu] gtt total\n",
- dev_priv->gtt.base.total,
- (u64)dev_priv->gtt.mappable_end - dev_priv->gtt.base.start);
+ ggtt->base.total, ggtt->mappable_end - ggtt->base.start);
seq_putc(m, '\n');
print_batch_pool_stats(m, dev_priv);
+
+ mutex_unlock(&dev->struct_mutex);
+
+ mutex_lock(&dev->filelist_mutex);
list_for_each_entry_reverse(file, &dev->filelist, lhead) {
struct file_stats stats;
struct task_struct *task;
@@ -517,8 +552,7 @@
print_file_stats(m, task ? task->comm : "<unknown>", stats);
rcu_read_unlock();
}
-
- mutex_unlock(&dev->struct_mutex);
+ mutex_unlock(&dev->filelist_mutex);
return 0;
}
@@ -591,14 +625,13 @@
pipe, plane);
}
if (work->flip_queued_req) {
- struct intel_engine_cs *ring =
- i915_gem_request_get_ring(work->flip_queued_req);
+ struct intel_engine_cs *engine = i915_gem_request_get_engine(work->flip_queued_req);
seq_printf(m, "Flip queued on %s at seqno %x, next seqno %x [current breadcrumb %x], completed? %d\n",
- ring->name,
+ engine->name,
i915_gem_request_get_seqno(work->flip_queued_req),
dev_priv->next_seqno,
- ring->get_seqno(ring, true),
+ engine->get_seqno(engine),
i915_gem_request_completed(work->flip_queued_req, true));
} else
seq_printf(m, "Flip not associated with any ring\n");
@@ -637,28 +670,28 @@
struct drm_device *dev = node->minor->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
struct drm_i915_gem_object *obj;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
int total = 0;
- int ret, i, j;
+ int ret, j;
ret = mutex_lock_interruptible(&dev->struct_mutex);
if (ret)
return ret;
- for_each_ring(ring, dev_priv, i) {
- for (j = 0; j < ARRAY_SIZE(ring->batch_pool.cache_list); j++) {
+ for_each_engine(engine, dev_priv) {
+ for (j = 0; j < ARRAY_SIZE(engine->batch_pool.cache_list); j++) {
int count;
count = 0;
list_for_each_entry(obj,
- &ring->batch_pool.cache_list[j],
+ &engine->batch_pool.cache_list[j],
batch_pool_link)
count++;
seq_printf(m, "%s cache[%d]: %d objects\n",
- ring->name, j, count);
+ engine->name, j, count);
list_for_each_entry(obj,
- &ring->batch_pool.cache_list[j],
+ &engine->batch_pool.cache_list[j],
batch_pool_link) {
seq_puts(m, " ");
describe_obj(m, obj);
@@ -681,26 +714,26 @@
struct drm_info_node *node = m->private;
struct drm_device *dev = node->minor->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
struct drm_i915_gem_request *req;
- int ret, any, i;
+ int ret, any;
ret = mutex_lock_interruptible(&dev->struct_mutex);
if (ret)
return ret;
any = 0;
- for_each_ring(ring, dev_priv, i) {
+ for_each_engine(engine, dev_priv) {
int count;
count = 0;
- list_for_each_entry(req, &ring->request_list, list)
+ list_for_each_entry(req, &engine->request_list, list)
count++;
if (count == 0)
continue;
- seq_printf(m, "%s requests: %d\n", ring->name, count);
- list_for_each_entry(req, &ring->request_list, list) {
+ seq_printf(m, "%s requests: %d\n", engine->name, count);
+ list_for_each_entry(req, &engine->request_list, list) {
struct task_struct *task;
rcu_read_lock();
@@ -726,12 +759,12 @@
}
static void i915_ring_seqno_info(struct seq_file *m,
- struct intel_engine_cs *ring)
+ struct intel_engine_cs *engine)
{
- if (ring->get_seqno) {
- seq_printf(m, "Current sequence (%s): %x\n",
- ring->name, ring->get_seqno(ring, false));
- }
+ seq_printf(m, "Current sequence (%s): %x\n",
+ engine->name, engine->get_seqno(engine));
+ seq_printf(m, "Current user interrupts (%s): %x\n",
+ engine->name, READ_ONCE(engine->user_interrupts));
}
static int i915_gem_seqno_info(struct seq_file *m, void *data)
@@ -739,16 +772,16 @@
struct drm_info_node *node = m->private;
struct drm_device *dev = node->minor->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
- int ret, i;
+ struct intel_engine_cs *engine;
+ int ret;
ret = mutex_lock_interruptible(&dev->struct_mutex);
if (ret)
return ret;
intel_runtime_pm_get(dev_priv);
- for_each_ring(ring, dev_priv, i)
- i915_ring_seqno_info(m, ring);
+ for_each_engine(engine, dev_priv)
+ i915_ring_seqno_info(m, engine);
intel_runtime_pm_put(dev_priv);
mutex_unlock(&dev->struct_mutex);
@@ -762,7 +795,7 @@
struct drm_info_node *node = m->private;
struct drm_device *dev = node->minor->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
int ret, i, pipe;
ret = mutex_lock_interruptible(&dev->struct_mutex);
@@ -934,13 +967,13 @@
seq_printf(m, "Graphics Interrupt mask: %08x\n",
I915_READ(GTIMR));
}
- for_each_ring(ring, dev_priv, i) {
+ for_each_engine(engine, dev_priv) {
if (INTEL_INFO(dev)->gen >= 6) {
seq_printf(m,
"Graphics Interrupt mask (%s): %08x\n",
- ring->name, I915_READ_IMR(ring));
+ engine->name, I915_READ_IMR(engine));
}
- i915_ring_seqno_info(m, ring);
+ i915_ring_seqno_info(m, engine);
}
intel_runtime_pm_put(dev_priv);
mutex_unlock(&dev->struct_mutex);
@@ -981,12 +1014,12 @@
struct drm_info_node *node = m->private;
struct drm_device *dev = node->minor->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
const u32 *hws;
int i;
- ring = &dev_priv->ring[(uintptr_t)node->info_ent->data];
- hws = ring->status_page.page_addr;
+ engine = &dev_priv->engine[(uintptr_t)node->info_ent->data];
+ hws = engine->status_page.page_addr;
if (hws == NULL)
return 0;
@@ -1216,12 +1249,12 @@
rpdeclimit = I915_READ(GEN6_RP_DOWN_THRESHOLD);
rpstat = I915_READ(GEN6_RPSTAT1);
- rpupei = I915_READ(GEN6_RP_CUR_UP_EI);
- rpcurup = I915_READ(GEN6_RP_CUR_UP);
- rpprevup = I915_READ(GEN6_RP_PREV_UP);
- rpdownei = I915_READ(GEN6_RP_CUR_DOWN_EI);
- rpcurdown = I915_READ(GEN6_RP_CUR_DOWN);
- rpprevdown = I915_READ(GEN6_RP_PREV_DOWN);
+ rpupei = I915_READ(GEN6_RP_CUR_UP_EI) & GEN6_CURICONT_MASK;
+ rpcurup = I915_READ(GEN6_RP_CUR_UP) & GEN6_CURBSYTAVG_MASK;
+ rpprevup = I915_READ(GEN6_RP_PREV_UP) & GEN6_CURBSYTAVG_MASK;
+ rpdownei = I915_READ(GEN6_RP_CUR_DOWN_EI) & GEN6_CURIAVG_MASK;
+ rpcurdown = I915_READ(GEN6_RP_CUR_DOWN) & GEN6_CURBSYTAVG_MASK;
+ rpprevdown = I915_READ(GEN6_RP_PREV_DOWN) & GEN6_CURBSYTAVG_MASK;
if (IS_GEN9(dev))
cagf = (rpstat & GEN9_CAGF_MASK) >> GEN9_CAGF_SHIFT;
else if (IS_HASWELL(dev) || IS_BROADWELL(dev))
@@ -1261,21 +1294,21 @@
seq_printf(m, "RPDECLIMIT: 0x%08x\n", rpdeclimit);
seq_printf(m, "RPNSWREQ: %dMHz\n", reqf);
seq_printf(m, "CAGF: %dMHz\n", cagf);
- seq_printf(m, "RP CUR UP EI: %dus\n", rpupei &
- GEN6_CURICONT_MASK);
- seq_printf(m, "RP CUR UP: %dus\n", rpcurup &
- GEN6_CURBSYTAVG_MASK);
- seq_printf(m, "RP PREV UP: %dus\n", rpprevup &
- GEN6_CURBSYTAVG_MASK);
+ seq_printf(m, "RP CUR UP EI: %d (%dus)\n",
+ rpupei, GT_PM_INTERVAL_TO_US(dev_priv, rpupei));
+ seq_printf(m, "RP CUR UP: %d (%dus)\n",
+ rpcurup, GT_PM_INTERVAL_TO_US(dev_priv, rpcurup));
+ seq_printf(m, "RP PREV UP: %d (%dus)\n",
+ rpprevup, GT_PM_INTERVAL_TO_US(dev_priv, rpprevup));
seq_printf(m, "Up threshold: %d%%\n",
dev_priv->rps.up_threshold);
- seq_printf(m, "RP CUR DOWN EI: %dus\n", rpdownei &
- GEN6_CURIAVG_MASK);
- seq_printf(m, "RP CUR DOWN: %dus\n", rpcurdown &
- GEN6_CURBSYTAVG_MASK);
- seq_printf(m, "RP PREV DOWN: %dus\n", rpprevdown &
- GEN6_CURBSYTAVG_MASK);
+ seq_printf(m, "RP CUR DOWN EI: %d (%dus)\n",
+ rpdownei, GT_PM_INTERVAL_TO_US(dev_priv, rpdownei));
+ seq_printf(m, "RP CUR DOWN: %d (%dus)\n",
+ rpcurdown, GT_PM_INTERVAL_TO_US(dev_priv, rpcurdown));
+ seq_printf(m, "RP PREV DOWN: %d (%dus)\n",
+ rpprevdown, GT_PM_INTERVAL_TO_US(dev_priv, rpprevdown));
seq_printf(m, "Down threshold: %d%%\n",
dev_priv->rps.down_threshold);
@@ -1331,11 +1364,12 @@
struct drm_info_node *node = m->private;
struct drm_device *dev = node->minor->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
- u64 acthd[I915_NUM_RINGS];
- u32 seqno[I915_NUM_RINGS];
+ struct intel_engine_cs *engine;
+ u64 acthd[I915_NUM_ENGINES];
+ u32 seqno[I915_NUM_ENGINES];
u32 instdone[I915_NUM_INSTDONE_REG];
- int i, j;
+ enum intel_engine_id id;
+ int j;
if (!i915.enable_hangcheck) {
seq_printf(m, "Hangcheck disabled\n");
@@ -1344,9 +1378,9 @@
intel_runtime_pm_get(dev_priv);
- for_each_ring(ring, dev_priv, i) {
- seqno[i] = ring->get_seqno(ring, false);
- acthd[i] = intel_ring_get_active_head(ring);
+ for_each_engine_id(engine, dev_priv, id) {
+ acthd[id] = intel_ring_get_active_head(engine);
+ seqno[id] = engine->get_seqno(engine);
}
i915_get_extra_instdone(dev, instdone);
@@ -1360,19 +1394,22 @@
} else
seq_printf(m, "Hangcheck inactive\n");
- for_each_ring(ring, dev_priv, i) {
- seq_printf(m, "%s:\n", ring->name);
- seq_printf(m, "\tseqno = %x [current %x]\n",
- ring->hangcheck.seqno, seqno[i]);
+ for_each_engine_id(engine, dev_priv, id) {
+ seq_printf(m, "%s:\n", engine->name);
+ seq_printf(m, "\tseqno = %x [current %x, last %x]\n",
+ engine->hangcheck.seqno,
+ seqno[id],
+ engine->last_submitted_seqno);
+ seq_printf(m, "\tuser interrupts = %x [current %x]\n",
+ engine->hangcheck.user_interrupts,
+ READ_ONCE(engine->user_interrupts));
seq_printf(m, "\tACTHD = 0x%08llx [current 0x%08llx]\n",
- (long long)ring->hangcheck.acthd,
- (long long)acthd[i]);
- seq_printf(m, "\tmax ACTHD = 0x%08llx\n",
- (long long)ring->hangcheck.max_acthd);
- seq_printf(m, "\tscore = %d\n", ring->hangcheck.score);
- seq_printf(m, "\taction = %d\n", ring->hangcheck.action);
+ (long long)engine->hangcheck.acthd,
+ (long long)acthd[id]);
+ seq_printf(m, "\tscore = %d\n", engine->hangcheck.score);
+ seq_printf(m, "\taction = %d\n", engine->hangcheck.action);
- if (ring->id == RCS) {
+ if (engine->id == RCS) {
seq_puts(m, "\tinstdone read =");
for (j = 0; j < I915_NUM_INSTDONE_REG; j++)
@@ -1382,7 +1419,7 @@
for (j = 0; j < I915_NUM_INSTDONE_REG; j++)
seq_printf(m, " 0x%08x",
- ring->hangcheck.instdone[j]);
+ engine->hangcheck.instdone[j]);
seq_puts(m, "\n");
}
@@ -1465,12 +1502,11 @@
struct drm_device *dev = node->minor->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
struct intel_uncore_forcewake_domain *fw_domain;
- int i;
spin_lock_irq(&dev_priv->uncore.lock);
- for_each_fw_domain(fw_domain, dev_priv, i) {
+ for_each_fw_domain(fw_domain, dev_priv) {
seq_printf(m, "%s.wake_count = %u\n",
- intel_uncore_forcewake_domain_to_str(i),
+ intel_uncore_forcewake_domain_to_str(fw_domain->id),
fw_domain->wake_count);
}
spin_unlock_irq(&dev_priv->uncore.lock);
@@ -1897,6 +1933,11 @@
struct drm_device *dev = node->minor->dev;
struct intel_framebuffer *fbdev_fb = NULL;
struct drm_framebuffer *drm_fb;
+ int ret;
+
+ ret = mutex_lock_interruptible(&dev->struct_mutex);
+ if (ret)
+ return ret;
#ifdef CONFIG_DRM_FBDEV_EMULATION
if (to_i915(dev)->fbdev) {
@@ -1908,7 +1949,7 @@
fbdev_fb->base.depth,
fbdev_fb->base.bits_per_pixel,
fbdev_fb->base.modifier[0],
- atomic_read(&fbdev_fb->base.refcount.refcount));
+ drm_framebuffer_read_refcount(&fbdev_fb->base));
describe_obj(m, fbdev_fb->obj);
seq_putc(m, '\n');
}
@@ -1926,11 +1967,12 @@
fb->base.depth,
fb->base.bits_per_pixel,
fb->base.modifier[0],
- atomic_read(&fb->base.refcount.refcount));
+ drm_framebuffer_read_refcount(&fb->base));
describe_obj(m, fb->obj);
seq_putc(m, '\n');
}
mutex_unlock(&dev->mode_config.fb_lock);
+ mutex_unlock(&dev->struct_mutex);
return 0;
}
@@ -1948,9 +1990,10 @@
struct drm_info_node *node = m->private;
struct drm_device *dev = node->minor->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
struct intel_context *ctx;
- int ret, i;
+ enum intel_engine_id id;
+ int ret;
ret = mutex_lock_interruptible(&dev->struct_mutex);
if (ret)
@@ -1968,13 +2011,13 @@
if (i915.enable_execlists) {
seq_putc(m, '\n');
- for_each_ring(ring, dev_priv, i) {
+ for_each_engine_id(engine, dev_priv, id) {
struct drm_i915_gem_object *ctx_obj =
- ctx->engine[i].state;
+ ctx->engine[id].state;
struct intel_ringbuffer *ringbuf =
- ctx->engine[i].ringbuf;
+ ctx->engine[id].ringbuf;
- seq_printf(m, "%s: ", ring->name);
+ seq_printf(m, "%s: ", engine->name);
if (ctx_obj)
describe_obj(m, ctx_obj);
if (ringbuf)
@@ -1995,22 +2038,22 @@
static void i915_dump_lrc_obj(struct seq_file *m,
struct intel_context *ctx,
- struct intel_engine_cs *ring)
+ struct intel_engine_cs *engine)
{
struct page *page;
uint32_t *reg_state;
int j;
- struct drm_i915_gem_object *ctx_obj = ctx->engine[ring->id].state;
+ struct drm_i915_gem_object *ctx_obj = ctx->engine[engine->id].state;
unsigned long ggtt_offset = 0;
if (ctx_obj == NULL) {
seq_printf(m, "Context on %s with no gem object\n",
- ring->name);
+ engine->name);
return;
}
- seq_printf(m, "CONTEXT: %s %u\n", ring->name,
- intel_execlists_ctx_id(ctx, ring));
+ seq_printf(m, "CONTEXT: %s %u\n", engine->name,
+ intel_execlists_ctx_id(ctx, engine));
if (!i915_gem_obj_ggtt_bound(ctx_obj))
seq_puts(m, "\tNot bound in GGTT\n");
@@ -2043,9 +2086,9 @@
struct drm_info_node *node = (struct drm_info_node *) m->private;
struct drm_device *dev = node->minor->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
struct intel_context *ctx;
- int ret, i;
+ int ret;
if (!i915.enable_execlists) {
seq_printf(m, "Logical Ring Contexts are disabled\n");
@@ -2058,8 +2101,8 @@
list_for_each_entry(ctx, &dev_priv->context_list, link)
if (ctx != dev_priv->kernel_context)
- for_each_ring(ring, dev_priv, i)
- i915_dump_lrc_obj(m, ctx, ring);
+ for_each_engine(engine, dev_priv)
+ i915_dump_lrc_obj(m, ctx, engine);
mutex_unlock(&dev->struct_mutex);
@@ -2071,15 +2114,14 @@
struct drm_info_node *node = (struct drm_info_node *)m->private;
struct drm_device *dev = node->minor->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
u32 status_pointer;
u8 read_pointer;
u8 write_pointer;
u32 status;
u32 ctx_id;
struct list_head *cursor;
- int ring_id, i;
- int ret;
+ int i, ret;
if (!i915.enable_execlists) {
seq_puts(m, "Logical Ring Contexts are disabled\n");
@@ -2092,22 +2134,21 @@
intel_runtime_pm_get(dev_priv);
- for_each_ring(ring, dev_priv, ring_id) {
+ for_each_engine(engine, dev_priv) {
struct drm_i915_gem_request *head_req = NULL;
int count = 0;
- unsigned long flags;
- seq_printf(m, "%s\n", ring->name);
+ seq_printf(m, "%s\n", engine->name);
- status = I915_READ(RING_EXECLIST_STATUS_LO(ring));
- ctx_id = I915_READ(RING_EXECLIST_STATUS_HI(ring));
+ status = I915_READ(RING_EXECLIST_STATUS_LO(engine));
+ ctx_id = I915_READ(RING_EXECLIST_STATUS_HI(engine));
seq_printf(m, "\tExeclist status: 0x%08X, context: %u\n",
status, ctx_id);
- status_pointer = I915_READ(RING_CONTEXT_STATUS_PTR(ring));
+ status_pointer = I915_READ(RING_CONTEXT_STATUS_PTR(engine));
seq_printf(m, "\tStatus pointer: 0x%08X\n", status_pointer);
- read_pointer = ring->next_context_status_buffer;
+ read_pointer = engine->next_context_status_buffer;
write_pointer = GEN8_CSB_WRITE_PTR(status_pointer);
if (read_pointer > write_pointer)
write_pointer += GEN8_CSB_ENTRIES;
@@ -2115,24 +2156,25 @@
read_pointer, write_pointer);
for (i = 0; i < GEN8_CSB_ENTRIES; i++) {
- status = I915_READ(RING_CONTEXT_STATUS_BUF_LO(ring, i));
- ctx_id = I915_READ(RING_CONTEXT_STATUS_BUF_HI(ring, i));
+ status = I915_READ(RING_CONTEXT_STATUS_BUF_LO(engine, i));
+ ctx_id = I915_READ(RING_CONTEXT_STATUS_BUF_HI(engine, i));
seq_printf(m, "\tStatus buffer %d: 0x%08X, context: %u\n",
i, status, ctx_id);
}
- spin_lock_irqsave(&ring->execlist_lock, flags);
- list_for_each(cursor, &ring->execlist_queue)
+ spin_lock_bh(&engine->execlist_lock);
+ list_for_each(cursor, &engine->execlist_queue)
count++;
- head_req = list_first_entry_or_null(&ring->execlist_queue,
- struct drm_i915_gem_request, execlist_link);
- spin_unlock_irqrestore(&ring->execlist_lock, flags);
+ head_req = list_first_entry_or_null(&engine->execlist_queue,
+ struct drm_i915_gem_request,
+ execlist_link);
+ spin_unlock_bh(&engine->execlist_lock);
seq_printf(m, "\t%d requests in queue\n", count);
if (head_req) {
seq_printf(m, "\tHead request id: %u\n",
- intel_execlists_ctx_id(head_req->ctx, ring));
+ intel_execlists_ctx_id(head_req->ctx, engine));
seq_printf(m, "\tHead request tail: %u\n",
head_req->tail);
}
@@ -2248,19 +2290,19 @@
static void gen8_ppgtt_info(struct seq_file *m, struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
struct i915_hw_ppgtt *ppgtt = dev_priv->mm.aliasing_ppgtt;
- int unused, i;
+ int i;
if (!ppgtt)
return;
- for_each_ring(ring, dev_priv, unused) {
- seq_printf(m, "%s\n", ring->name);
+ for_each_engine(engine, dev_priv) {
+ seq_printf(m, "%s\n", engine->name);
for (i = 0; i < 4; i++) {
- u64 pdp = I915_READ(GEN8_RING_PDP_UDW(ring, i));
+ u64 pdp = I915_READ(GEN8_RING_PDP_UDW(engine, i));
pdp <<= 32;
- pdp |= I915_READ(GEN8_RING_PDP_LDW(ring, i));
+ pdp |= I915_READ(GEN8_RING_PDP_LDW(engine, i));
seq_printf(m, "\tPDP%d 0x%016llx\n", i, pdp);
}
}
@@ -2269,19 +2311,22 @@
static void gen6_ppgtt_info(struct seq_file *m, struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
- int i;
+ struct intel_engine_cs *engine;
if (INTEL_INFO(dev)->gen == 6)
seq_printf(m, "GFX_MODE: 0x%08x\n", I915_READ(GFX_MODE));
- for_each_ring(ring, dev_priv, i) {
- seq_printf(m, "%s\n", ring->name);
+ for_each_engine(engine, dev_priv) {
+ seq_printf(m, "%s\n", engine->name);
if (INTEL_INFO(dev)->gen == 7)
- seq_printf(m, "GFX_MODE: 0x%08x\n", I915_READ(RING_MODE_GEN7(ring)));
- seq_printf(m, "PP_DIR_BASE: 0x%08x\n", I915_READ(RING_PP_DIR_BASE(ring)));
- seq_printf(m, "PP_DIR_BASE_READ: 0x%08x\n", I915_READ(RING_PP_DIR_BASE_READ(ring)));
- seq_printf(m, "PP_DIR_DCLV: 0x%08x\n", I915_READ(RING_PP_DIR_DCLV(ring)));
+ seq_printf(m, "GFX_MODE: 0x%08x\n",
+ I915_READ(RING_MODE_GEN7(engine)));
+ seq_printf(m, "PP_DIR_BASE: 0x%08x\n",
+ I915_READ(RING_PP_DIR_BASE(engine)));
+ seq_printf(m, "PP_DIR_BASE_READ: 0x%08x\n",
+ I915_READ(RING_PP_DIR_BASE_READ(engine)));
+ seq_printf(m, "PP_DIR_DCLV: 0x%08x\n",
+ I915_READ(RING_PP_DIR_DCLV(engine)));
}
if (dev_priv->mm.aliasing_ppgtt) {
struct i915_hw_ppgtt *ppgtt = dev_priv->mm.aliasing_ppgtt;
@@ -2312,6 +2357,7 @@
else if (INTEL_INFO(dev)->gen >= 6)
gen6_ppgtt_info(m, dev);
+ mutex_lock(&dev->filelist_mutex);
list_for_each_entry_reverse(file, &dev->filelist, lhead) {
struct drm_i915_file_private *file_priv = file->driver_priv;
struct task_struct *task;
@@ -2326,6 +2372,7 @@
idr_for_each(&file_priv->context_idr, per_file_ctx,
(void *)(unsigned long)m);
}
+ mutex_unlock(&dev->filelist_mutex);
out_put:
intel_runtime_pm_put(dev_priv);
@@ -2336,12 +2383,11 @@
static int count_irq_waiters(struct drm_i915_private *i915)
{
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
int count = 0;
- int i;
- for_each_ring(ring, i915, i)
- count += ring->irq_refcount;
+ for_each_engine(engine, i915)
+ count += engine->irq_refcount;
return count;
}
@@ -2362,6 +2408,8 @@
intel_gpu_freq(dev_priv, dev_priv->rps.min_freq_softlimit),
intel_gpu_freq(dev_priv, dev_priv->rps.max_freq_softlimit),
intel_gpu_freq(dev_priv, dev_priv->rps.max_freq));
+
+ mutex_lock(&dev->filelist_mutex);
spin_lock(&dev_priv->rps.client_lock);
list_for_each_entry_reverse(file, &dev->filelist, lhead) {
struct drm_i915_file_private *file_priv = file->driver_priv;
@@ -2384,6 +2432,7 @@
list_empty(&dev_priv->rps.mmioflips.link) ? "" : ", active");
seq_printf(m, "Kernel boosts: %d\n", dev_priv->rps.boosts);
spin_unlock(&dev_priv->rps.client_lock);
+ mutex_unlock(&dev->filelist_mutex);
return 0;
}
@@ -2393,10 +2442,11 @@
struct drm_info_node *node = m->private;
struct drm_device *dev = node->minor->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
+ const bool edram = INTEL_GEN(dev_priv) > 8;
- /* Size calculation for LLC is a bit of a pain. Ignore for now. */
seq_printf(m, "LLC: %s\n", yesno(HAS_LLC(dev)));
- seq_printf(m, "eLLC: %zuMB\n", dev_priv->ellc_size);
+ seq_printf(m, "%s: %lluMB\n", edram ? "eDRAM" : "eLLC",
+ intel_uncore_edram_size(dev_priv)/1024/1024);
return 0;
}
@@ -2408,7 +2458,7 @@
struct intel_guc_fw *guc_fw = &dev_priv->guc.guc_fw;
u32 tmp, i;
- if (!HAS_GUC_UCODE(dev_priv->dev))
+ if (!HAS_GUC_UCODE(dev_priv))
return 0;
seq_printf(m, "GuC firmware status:\n");
@@ -2449,9 +2499,8 @@
struct drm_i915_private *dev_priv,
struct i915_guc_client *client)
{
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
uint64_t tot = 0;
- uint32_t i;
seq_printf(m, "\tPriority %d, GuC ctx index: %u, PD offset 0x%x\n",
client->priority, client->ctx_index, client->proc_desc_offset);
@@ -2464,11 +2513,11 @@
seq_printf(m, "\tFailed doorbell: %u\n", client->b_fail);
seq_printf(m, "\tLast submission result: %d\n", client->retcode);
- for_each_ring(ring, dev_priv, i) {
+ for_each_engine(engine, dev_priv) {
seq_printf(m, "\tSubmissions: %llu %s\n",
- client->submissions[ring->guc_id],
- ring->name);
- tot += client->submissions[ring->guc_id];
+ client->submissions[engine->guc_id],
+ engine->name);
+ tot += client->submissions[engine->guc_id];
}
seq_printf(m, "\tTotal: %llu\n", tot);
}
@@ -2480,11 +2529,10 @@
struct drm_i915_private *dev_priv = dev->dev_private;
struct intel_guc guc;
struct i915_guc_client client = {};
- struct intel_engine_cs *ring;
- enum intel_ring_id i;
+ struct intel_engine_cs *engine;
u64 total = 0;
- if (!HAS_GUC_SCHED(dev_priv->dev))
+ if (!HAS_GUC_SCHED(dev_priv))
return 0;
if (mutex_lock_interruptible(&dev->struct_mutex))
@@ -2504,11 +2552,11 @@
seq_printf(m, "GuC last action error code: %d\n", guc.action_err);
seq_printf(m, "\nGuC submissions:\n");
- for_each_ring(ring, dev_priv, i) {
+ for_each_engine(engine, dev_priv) {
seq_printf(m, "\t%-24s: %10llu, last seqno 0x%08x\n",
- ring->name, guc.submissions[ring->guc_id],
- guc.last_seqno[ring->guc_id]);
- total += guc.submissions[ring->guc_id];
+ engine->name, guc.submissions[engine->guc_id],
+ guc.last_seqno[engine->guc_id]);
+ total += guc.submissions[engine->guc_id];
}
seq_printf(m, "\t%s: %llu\n", "Total", total);
@@ -2688,10 +2736,8 @@
struct drm_device *dev = node->minor->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- if (!HAS_RUNTIME_PM(dev)) {
- seq_puts(m, "not supported\n");
- return 0;
- }
+ if (!HAS_RUNTIME_PM(dev_priv))
+ seq_puts(m, "Runtime power management not supported\n");
seq_printf(m, "GPU idle: %s\n", yesno(!dev_priv->mm.busy));
seq_printf(m, "IRQs disabled: %s\n",
@@ -2702,6 +2748,9 @@
#else
seq_printf(m, "Device Power Management (CONFIG_PM) disabled\n");
#endif
+ seq_printf(m, "PCI device power state: %s [%d]\n",
+ pci_power_name(dev_priv->dev->pdev->current_state),
+ dev_priv->dev->pdev->current_state);
return 0;
}
@@ -3114,9 +3163,10 @@
struct drm_info_node *node = (struct drm_info_node *) m->private;
struct drm_device *dev = node->minor->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
int num_rings = hweight32(INTEL_INFO(dev)->ring_mask);
- int i, j, ret;
+ enum intel_engine_id id;
+ int j, ret;
if (!i915_semaphore_is_enabled(dev)) {
seq_puts(m, "Semaphores are disabled\n");
@@ -3135,14 +3185,14 @@
page = i915_gem_object_get_page(dev_priv->semaphore_obj, 0);
seqno = (uint64_t *)kmap_atomic(page);
- for_each_ring(ring, dev_priv, i) {
+ for_each_engine_id(engine, dev_priv, id) {
uint64_t offset;
- seq_printf(m, "%s\n", ring->name);
+ seq_printf(m, "%s\n", engine->name);
seq_puts(m, " Last signal:");
for (j = 0; j < num_rings; j++) {
- offset = i * I915_NUM_RINGS + j;
+ offset = id * I915_NUM_ENGINES + j;
seq_printf(m, "0x%08llx (0x%02llx) ",
seqno[offset], offset * 8);
}
@@ -3150,7 +3200,7 @@
seq_puts(m, " Last wait: ");
for (j = 0; j < num_rings; j++) {
- offset = i + (j * I915_NUM_RINGS);
+ offset = id + (j * I915_NUM_ENGINES);
seq_printf(m, "0x%08llx (0x%02llx) ",
seqno[offset], offset * 8);
}
@@ -3160,18 +3210,18 @@
kunmap_atomic(seqno);
} else {
seq_puts(m, " Last signal:");
- for_each_ring(ring, dev_priv, i)
+ for_each_engine(engine, dev_priv)
for (j = 0; j < num_rings; j++)
seq_printf(m, "0x%08x\n",
- I915_READ(ring->semaphore.mbox.signal[j]));
+ I915_READ(engine->semaphore.mbox.signal[j]));
seq_putc(m, '\n');
}
seq_puts(m, "\nSync seqno:\n");
- for_each_ring(ring, dev_priv, i) {
- for (j = 0; j < num_rings; j++) {
- seq_printf(m, " 0x%08x ", ring->semaphore.sync_seqno[j]);
- }
+ for_each_engine(engine, dev_priv) {
+ for (j = 0; j < num_rings; j++)
+ seq_printf(m, " 0x%08x ",
+ engine->semaphore.sync_seqno[j]);
seq_putc(m, '\n');
}
seq_putc(m, '\n');
@@ -3193,8 +3243,8 @@
struct intel_shared_dpll *pll = &dev_priv->shared_dplls[i];
seq_printf(m, "DPLL%i: %s, id: %i\n", i, pll->name, pll->id);
- seq_printf(m, " crtc_mask: 0x%08x, active: %d, on: %s\n",
- pll->config.crtc_mask, pll->active, yesno(pll->on));
+ seq_printf(m, " crtc_mask: 0x%08x, active: 0x%x, on: %s\n",
+ pll->config.crtc_mask, pll->active_mask, yesno(pll->on));
seq_printf(m, " tracked hardware state:\n");
seq_printf(m, " dpll: 0x%08x\n", pll->config.hw_state.dpll);
seq_printf(m, " dpll_md: 0x%08x\n",
@@ -3212,11 +3262,12 @@
{
int i;
int ret;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
struct drm_info_node *node = (struct drm_info_node *) m->private;
struct drm_device *dev = node->minor->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
struct i915_workarounds *workarounds = &dev_priv->workarounds;
+ enum intel_engine_id id;
ret = mutex_lock_interruptible(&dev->struct_mutex);
if (ret)
@@ -3225,9 +3276,9 @@
intel_runtime_pm_get(dev_priv);
seq_printf(m, "Workarounds applied: %d\n", workarounds->count);
- for_each_ring(ring, dev_priv, i)
+ for_each_engine_id(engine, dev_priv, id)
seq_printf(m, "HW whitelist count for %s: %d\n",
- ring->name, workarounds->hw_whitelist_count[i]);
+ engine->name, workarounds->hw_whitelist_count[id]);
for (i = 0; i < workarounds->count; ++i) {
i915_reg_t addr;
u32 mask, value, read;
@@ -3417,7 +3468,8 @@
intel_dig_port = enc_to_dig_port(encoder);
if (!intel_dig_port->dp.can_mst)
continue;
-
+ seq_printf(m, "MST Source Port %c\n",
+ port_name(intel_dig_port->port));
drm_dp_mst_dump_topology(m, &intel_dig_port->dp.mst_mgr);
}
drm_modeset_unlock_all(dev);
@@ -4693,7 +4745,7 @@
struct drm_device *dev = data;
struct drm_i915_private *dev_priv = dev->dev_private;
- *val = atomic_read(&dev_priv->gpu_error.reset_counter);
+ *val = i915_terminally_wedged(&dev_priv->gpu_error);
return 0;
}
diff --git a/drivers/gpu/drm/i915/i915_dma.c b/drivers/gpu/drm/i915/i915_dma.c
index 1c6d227..15615fb 100644
--- a/drivers/gpu/drm/i915/i915_dma.c
+++ b/drivers/gpu/drm/i915/i915_dma.c
@@ -50,6 +50,66 @@
#include <linux/pm_runtime.h>
#include <linux/oom.h>
+static unsigned int i915_load_fail_count;
+
+bool __i915_inject_load_failure(const char *func, int line)
+{
+ if (i915_load_fail_count >= i915.inject_load_failure)
+ return false;
+
+ if (++i915_load_fail_count == i915.inject_load_failure) {
+ DRM_INFO("Injecting failure at checkpoint %u [%s:%d]\n",
+ i915.inject_load_failure, func, line);
+ return true;
+ }
+
+ return false;
+}
+
+#define FDO_BUG_URL "https://bugs.freedesktop.org/enter_bug.cgi?product=DRI"
+#define FDO_BUG_MSG "Please file a bug at " FDO_BUG_URL " against DRM/Intel " \
+ "providing the dmesg log by booting with drm.debug=0xf"
+
+void
+__i915_printk(struct drm_i915_private *dev_priv, const char *level,
+ const char *fmt, ...)
+{
+ static bool shown_bug_once;
+ struct device *dev = dev_priv->dev->dev;
+ bool is_error = level[1] <= KERN_ERR[1];
+ bool is_debug = level[1] == KERN_DEBUG[1];
+ struct va_format vaf;
+ va_list args;
+
+ if (is_debug && !(drm_debug & DRM_UT_DRIVER))
+ return;
+
+ va_start(args, fmt);
+
+ vaf.fmt = fmt;
+ vaf.va = &args;
+
+ dev_printk(level, dev, "[" DRM_NAME ":%ps] %pV",
+ __builtin_return_address(0), &vaf);
+
+ if (is_error && !shown_bug_once) {
+ dev_notice(dev, "%s", FDO_BUG_MSG);
+ shown_bug_once = true;
+ }
+
+ va_end(args);
+}
+
+static bool i915_error_injected(struct drm_i915_private *dev_priv)
+{
+ return i915.inject_load_failure &&
+ i915_load_fail_count == i915.inject_load_failure;
+}
+
+#define i915_load_error(dev_priv, fmt, ...) \
+ __i915_printk(dev_priv, \
+ i915_error_injected(dev_priv) ? KERN_DEBUG : KERN_ERR, \
+ fmt, ##__VA_ARGS__)
static int i915_getparam(struct drm_device *dev, void *data,
struct drm_file *file_priv)
@@ -87,16 +147,16 @@
value = 1;
break;
case I915_PARAM_HAS_BSD:
- value = intel_ring_initialized(&dev_priv->ring[VCS]);
+ value = intel_engine_initialized(&dev_priv->engine[VCS]);
break;
case I915_PARAM_HAS_BLT:
- value = intel_ring_initialized(&dev_priv->ring[BCS]);
+ value = intel_engine_initialized(&dev_priv->engine[BCS]);
break;
case I915_PARAM_HAS_VEBOX:
- value = intel_ring_initialized(&dev_priv->ring[VECS]);
+ value = intel_engine_initialized(&dev_priv->engine[VECS]);
break;
case I915_PARAM_HAS_BSD2:
- value = intel_ring_initialized(&dev_priv->ring[VCS2]);
+ value = intel_engine_initialized(&dev_priv->engine[VCS2]);
break;
case I915_PARAM_HAS_RELAXED_FENCING:
value = 1;
@@ -197,13 +257,6 @@
return 0;
}
-#define MCHBAR_I915 0x44
-#define MCHBAR_I965 0x48
-#define MCHBAR_SIZE (4*4096)
-
-#define DEVEN_REG 0x54
-#define DEVEN_MCHBAR_EN (1 << 28)
-
/* Allocate space for the MCH regs if needed, return nonzero on error */
static int
intel_alloc_mchbar_resource(struct drm_device *dev)
@@ -265,7 +318,7 @@
dev_priv->mchbar_need_disable = false;
if (IS_I915G(dev) || IS_I915GM(dev)) {
- pci_read_config_dword(dev_priv->bridge_dev, DEVEN_REG, &temp);
+ pci_read_config_dword(dev_priv->bridge_dev, DEVEN, &temp);
enabled = !!(temp & DEVEN_MCHBAR_EN);
} else {
pci_read_config_dword(dev_priv->bridge_dev, mchbar_reg, &temp);
@@ -283,7 +336,7 @@
/* Space is allocated or reserved, so enable it. */
if (IS_I915G(dev) || IS_I915GM(dev)) {
- pci_write_config_dword(dev_priv->bridge_dev, DEVEN_REG,
+ pci_write_config_dword(dev_priv->bridge_dev, DEVEN,
temp | DEVEN_MCHBAR_EN);
} else {
pci_read_config_dword(dev_priv->bridge_dev, mchbar_reg, &temp);
@@ -296,17 +349,24 @@
{
struct drm_i915_private *dev_priv = dev->dev_private;
int mchbar_reg = INTEL_INFO(dev)->gen >= 4 ? MCHBAR_I965 : MCHBAR_I915;
- u32 temp;
if (dev_priv->mchbar_need_disable) {
if (IS_I915G(dev) || IS_I915GM(dev)) {
- pci_read_config_dword(dev_priv->bridge_dev, DEVEN_REG, &temp);
- temp &= ~DEVEN_MCHBAR_EN;
- pci_write_config_dword(dev_priv->bridge_dev, DEVEN_REG, temp);
+ u32 deven_val;
+
+ pci_read_config_dword(dev_priv->bridge_dev, DEVEN,
+ &deven_val);
+ deven_val &= ~DEVEN_MCHBAR_EN;
+ pci_write_config_dword(dev_priv->bridge_dev, DEVEN,
+ deven_val);
} else {
- pci_read_config_dword(dev_priv->bridge_dev, mchbar_reg, &temp);
- temp &= ~1;
- pci_write_config_dword(dev_priv->bridge_dev, mchbar_reg, temp);
+ u32 mchbar_val;
+
+ pci_read_config_dword(dev_priv->bridge_dev, mchbar_reg,
+ &mchbar_val);
+ mchbar_val &= ~1;
+ pci_write_config_dword(dev_priv->bridge_dev, mchbar_reg,
+ mchbar_val);
}
}
@@ -370,6 +430,9 @@
struct drm_i915_private *dev_priv = dev->dev_private;
int ret;
+ if (i915_inject_load_failure())
+ return -ENODEV;
+
ret = intel_bios_init(dev_priv);
if (ret)
DRM_INFO("failed to find VBIOS tables\n");
@@ -413,9 +476,6 @@
intel_modeset_gem_init(dev);
- /* Always safe in the mode setting case. */
- /* FIXME: do pre/post-mode set stuff in core KMS code */
- dev->vblank_disable_allowed = true;
if (INTEL_INFO(dev)->num_pipes == 0)
return 0;
@@ -444,7 +504,7 @@
cleanup_gem:
mutex_lock(&dev->struct_mutex);
- i915_gem_cleanup_ringbuffer(dev);
+ i915_gem_cleanup_engines(dev);
i915_gem_context_fini(dev);
mutex_unlock(&dev->struct_mutex);
cleanup_irq:
@@ -453,6 +513,7 @@
intel_teardown_gmbus(dev);
cleanup_csr:
intel_csr_ucode_fini(dev_priv);
+ intel_power_domains_fini(dev_priv);
vga_switcheroo_unregister_client(dev->pdev);
cleanup_vga_client:
vga_client_register(dev->pdev, NULL, NULL, NULL);
@@ -465,6 +526,7 @@
{
struct apertures_struct *ap;
struct pci_dev *pdev = dev_priv->dev->pdev;
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
bool primary;
int ret;
@@ -472,8 +534,8 @@
if (!ap)
return -ENOMEM;
- ap->ranges[0].base = dev_priv->gtt.mappable_base;
- ap->ranges[0].size = dev_priv->gtt.mappable_end;
+ ap->ranges[0].base = ggtt->mappable_base;
+ ap->ranges[0].size = ggtt->mappable_end;
primary =
pdev->resource[PCI_ROM_RESOURCE].flags & IORESOURCE_ROM_SHADOW;
@@ -853,6 +915,10 @@
else if (INTEL_INFO(dev)->gen >= 9)
gen9_sseu_info_init(dev);
+ /* Snooping is broken on BXT A stepping. */
+ info->has_snoop = !info->has_llc;
+ info->has_snoop &= !IS_BXT_REVID(dev, 0, BXT_REVID_A1);
+
DRM_DEBUG_DRIVER("slice total: %u\n", info->slice_total);
DRM_DEBUG_DRIVER("subslice total: %u\n", info->subslice_total);
DRM_DEBUG_DRIVER("subslice per slice: %u\n", info->subslice_per_slice);
@@ -929,6 +995,84 @@
destroy_workqueue(dev_priv->wq);
}
+/**
+ * i915_driver_init_early - setup state not requiring device access
+ * @dev_priv: device private
+ *
+ * Initialize everything that is a "SW-only" state, that is state not
+ * requiring accessing the device or exposing the driver via kernel internal
+ * or userspace interfaces. Example steps belonging here: lock initialization,
+ * system memory allocation, setting up device specific attributes and
+ * function hooks not requiring accessing the device.
+ */
+static int i915_driver_init_early(struct drm_i915_private *dev_priv,
+ struct drm_device *dev,
+ struct intel_device_info *info)
+{
+ struct intel_device_info *device_info;
+ int ret = 0;
+
+ if (i915_inject_load_failure())
+ return -ENODEV;
+
+ /* Setup the write-once "constant" device info */
+ device_info = (struct intel_device_info *)&dev_priv->info;
+ memcpy(device_info, info, sizeof(dev_priv->info));
+ device_info->device_id = dev->pdev->device;
+
+ spin_lock_init(&dev_priv->irq_lock);
+ spin_lock_init(&dev_priv->gpu_error.lock);
+ mutex_init(&dev_priv->backlight_lock);
+ spin_lock_init(&dev_priv->uncore.lock);
+ spin_lock_init(&dev_priv->mm.object_stat_lock);
+ spin_lock_init(&dev_priv->mmio_flip_lock);
+ mutex_init(&dev_priv->sb_lock);
+ mutex_init(&dev_priv->modeset_restore_lock);
+ mutex_init(&dev_priv->av_mutex);
+ mutex_init(&dev_priv->wm.wm_mutex);
+ mutex_init(&dev_priv->pps_mutex);
+
+ ret = i915_workqueues_init(dev_priv);
+ if (ret < 0)
+ return ret;
+
+ /* This must be called before any calls to HAS_PCH_* */
+ intel_detect_pch(dev);
+
+ intel_pm_setup(dev);
+ intel_init_dpio(dev_priv);
+ intel_power_domains_init(dev_priv);
+ intel_irq_init(dev_priv);
+ intel_init_display_hooks(dev_priv);
+ intel_init_clock_gating_hooks(dev_priv);
+ intel_init_audio_hooks(dev_priv);
+ i915_gem_load_init(dev);
+
+ intel_display_crc_init(dev);
+
+ i915_dump_device_info(dev_priv);
+
+ /* Not all pre-production machines fall into this category, only the
+ * very first ones. Almost everything should work, except for maybe
+ * suspend/resume. And we don't implement workarounds that affect only
+ * pre-production machines. */
+ if (IS_HSW_EARLY_SDV(dev))
+ DRM_INFO("This is an early pre-production Haswell machine. "
+ "It may not be fully functional.\n");
+
+ return 0;
+}
+
+/**
+ * i915_driver_cleanup_early - cleanup the setup done in i915_driver_init_early()
+ * @dev_priv: device private
+ */
+static void i915_driver_cleanup_early(struct drm_i915_private *dev_priv)
+{
+ i915_gem_load_cleanup(dev_priv->dev);
+ i915_workqueues_cleanup(dev_priv);
+}
+
static int i915_mmio_setup(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = to_i915(dev);
@@ -970,97 +1114,87 @@
}
/**
- * i915_driver_load - setup chip and create an initial config
- * @dev: DRM device
- * @flags: startup flags
+ * i915_driver_init_mmio - setup device MMIO
+ * @dev_priv: device private
*
- * The driver load routine has to do several things:
- * - drive output discovery via intel_modeset_init()
- * - initialize the memory manager
- * - allocate initial config memory
- * - setup the DRM framebuffer with the allocated memory
+ * Setup minimal device state necessary for MMIO accesses later in the
+ * initialization sequence. The setup here should avoid any other device-wide
+ * side effects or exposing the driver via kernel internal or user space
+ * interfaces.
*/
-int i915_driver_load(struct drm_device *dev, unsigned long flags)
+static int i915_driver_init_mmio(struct drm_i915_private *dev_priv)
{
- struct drm_i915_private *dev_priv;
- struct intel_device_info *info, *device_info;
- int ret = 0;
- uint32_t aperture_size;
+ struct drm_device *dev = dev_priv->dev;
+ int ret;
- info = (struct intel_device_info *) flags;
+ if (i915_inject_load_failure())
+ return -ENODEV;
- dev_priv = kzalloc(sizeof(*dev_priv), GFP_KERNEL);
- if (dev_priv == NULL)
- return -ENOMEM;
-
- dev->dev_private = dev_priv;
- dev_priv->dev = dev;
-
- /* Setup the write-once "constant" device info */
- device_info = (struct intel_device_info *)&dev_priv->info;
- memcpy(device_info, info, sizeof(dev_priv->info));
- device_info->device_id = dev->pdev->device;
-
- spin_lock_init(&dev_priv->irq_lock);
- spin_lock_init(&dev_priv->gpu_error.lock);
- mutex_init(&dev_priv->backlight_lock);
- spin_lock_init(&dev_priv->uncore.lock);
- spin_lock_init(&dev_priv->mm.object_stat_lock);
- spin_lock_init(&dev_priv->mmio_flip_lock);
- mutex_init(&dev_priv->sb_lock);
- mutex_init(&dev_priv->modeset_restore_lock);
- mutex_init(&dev_priv->av_mutex);
-
- ret = i915_workqueues_init(dev_priv);
- if (ret < 0)
- goto out_free_priv;
-
- intel_pm_setup(dev);
-
- intel_runtime_pm_get(dev_priv);
-
- intel_display_crc_init(dev);
-
- i915_dump_device_info(dev_priv);
-
- /* Not all pre-production machines fall into this category, only the
- * very first ones. Almost everything should work, except for maybe
- * suspend/resume. And we don't implement workarounds that affect only
- * pre-production machines. */
- if (IS_HSW_EARLY_SDV(dev))
- DRM_INFO("This is an early pre-production Haswell machine. "
- "It may not be fully functional.\n");
-
- if (i915_get_bridge_dev(dev)) {
- ret = -EIO;
- goto out_runtime_pm_put;
- }
+ if (i915_get_bridge_dev(dev))
+ return -EIO;
ret = i915_mmio_setup(dev);
if (ret < 0)
goto put_bridge;
- /* This must be called before any calls to HAS_PCH_* */
- intel_detect_pch(dev);
-
intel_uncore_init(dev);
- ret = i915_gem_gtt_init(dev);
+ return 0;
+
+put_bridge:
+ pci_dev_put(dev_priv->bridge_dev);
+
+ return ret;
+}
+
+/**
+ * i915_driver_cleanup_mmio - cleanup the setup done in i915_driver_init_mmio()
+ * @dev_priv: device private
+ */
+static void i915_driver_cleanup_mmio(struct drm_i915_private *dev_priv)
+{
+ struct drm_device *dev = dev_priv->dev;
+
+ intel_uncore_fini(dev);
+ i915_mmio_cleanup(dev);
+ pci_dev_put(dev_priv->bridge_dev);
+}
+
+/**
+ * i915_driver_init_hw - setup state requiring device access
+ * @dev_priv: device private
+ *
+ * Setup state that requires accessing the device, but doesn't require
+ * exposing the driver via kernel internal or userspace interfaces.
+ */
+static int i915_driver_init_hw(struct drm_i915_private *dev_priv)
+{
+ struct drm_device *dev = dev_priv->dev;
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
+ uint32_t aperture_size;
+ int ret;
+
+ if (i915_inject_load_failure())
+ return -ENODEV;
+
+ intel_device_info_runtime_init(dev);
+
+ ret = i915_ggtt_init_hw(dev);
if (ret)
- goto out_uncore_fini;
+ return ret;
/* WARNING: Apparently we must kick fbdev drivers before vgacon,
* otherwise the vga fbdev driver falls over. */
ret = i915_kick_out_firmware_fb(dev_priv);
if (ret) {
DRM_ERROR("failed to remove conflicting framebuffer drivers\n");
- goto out_gtt;
+ goto out_ggtt;
}
ret = i915_kick_out_vgacon(dev_priv);
if (ret) {
DRM_ERROR("failed to remove conflicting VGA console\n");
- goto out_gtt;
+ goto out_ggtt;
}
pci_set_master(dev->pdev);
@@ -1080,26 +1214,27 @@
if (IS_BROADWATER(dev) || IS_CRESTLINE(dev))
dma_set_coherent_mask(&dev->pdev->dev, DMA_BIT_MASK(32));
- aperture_size = dev_priv->gtt.mappable_end;
+ aperture_size = ggtt->mappable_end;
- dev_priv->gtt.mappable =
- io_mapping_create_wc(dev_priv->gtt.mappable_base,
+ ggtt->mappable =
+ io_mapping_create_wc(ggtt->mappable_base,
aperture_size);
- if (dev_priv->gtt.mappable == NULL) {
+ if (!ggtt->mappable) {
ret = -EIO;
- goto out_gtt;
+ goto out_ggtt;
}
- dev_priv->gtt.mtrr = arch_phys_wc_add(dev_priv->gtt.mappable_base,
+ ggtt->mtrr = arch_phys_wc_add(ggtt->mappable_base,
aperture_size);
- intel_irq_init(dev_priv);
+ pm_qos_add_request(&dev_priv->pm_qos, PM_QOS_CPU_DMA_LATENCY,
+ PM_QOS_DEFAULT_VALUE);
+
intel_uncore_sanitize(dev);
intel_opregion_setup(dev);
- i915_gem_load_init(dev);
- i915_gem_shrinker_init(dev_priv);
+ i915_gem_load_init_fences(dev_priv);
/* On the 945G/GM, the chipset reports the MSI capability on the
* integrated graphics even though the support isn't actually there
@@ -1117,24 +1252,44 @@
DRM_DEBUG_DRIVER("can't enable MSI");
}
- intel_device_info_runtime_init(dev);
+ return 0;
- intel_init_dpio(dev_priv);
+out_ggtt:
+ i915_ggtt_cleanup_hw(dev);
- if (INTEL_INFO(dev)->num_pipes) {
- ret = drm_vblank_init(dev, INTEL_INFO(dev)->num_pipes);
- if (ret)
- goto out_gem_unload;
- }
+ return ret;
+}
- intel_power_domains_init(dev_priv);
+/**
+ * i915_driver_cleanup_hw - cleanup the setup done in i915_driver_init_hw()
+ * @dev_priv: device private
+ */
+static void i915_driver_cleanup_hw(struct drm_i915_private *dev_priv)
+{
+ struct drm_device *dev = dev_priv->dev;
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
- ret = i915_load_modeset_init(dev);
- if (ret < 0) {
- DRM_ERROR("failed to init modeset\n");
- goto out_power_well;
- }
+ if (dev->pdev->msi_enabled)
+ pci_disable_msi(dev->pdev);
+ pm_qos_remove_request(&dev_priv->pm_qos);
+ arch_phys_wc_del(ggtt->mtrr);
+ io_mapping_free(ggtt->mappable);
+ i915_ggtt_cleanup_hw(dev);
+}
+
+/**
+ * i915_driver_register - register the driver with the rest of the system
+ * @dev_priv: device private
+ *
+ * Perform any steps necessary to make the driver available via kernel
+ * internal or userspace interfaces.
+ */
+static void i915_driver_register(struct drm_i915_private *dev_priv)
+{
+ struct drm_device *dev = dev_priv->dev;
+
+ i915_gem_shrinker_init(dev_priv);
/*
* Notify a valid surface after modesetting,
* when running inside a VM.
@@ -1144,48 +1299,107 @@
i915_setup_sysfs(dev);
- if (INTEL_INFO(dev)->num_pipes) {
+ if (INTEL_INFO(dev_priv)->num_pipes) {
/* Must be done after probing outputs */
intel_opregion_init(dev);
acpi_video_register();
}
- if (IS_GEN5(dev))
+ if (IS_GEN5(dev_priv))
intel_gpu_ips_init(dev_priv);
- intel_runtime_pm_enable(dev_priv);
-
i915_audio_component_init(dev_priv);
+}
+
+/**
+ * i915_driver_unregister - cleanup the registration done in i915_driver_regiser()
+ * @dev_priv: device private
+ */
+static void i915_driver_unregister(struct drm_i915_private *dev_priv)
+{
+ i915_audio_component_cleanup(dev_priv);
+ intel_gpu_ips_teardown();
+ acpi_video_unregister();
+ intel_opregion_fini(dev_priv->dev);
+ i915_teardown_sysfs(dev_priv->dev);
+ i915_gem_shrinker_cleanup(dev_priv);
+}
+
+/**
+ * i915_driver_load - setup chip and create an initial config
+ * @dev: DRM device
+ * @flags: startup flags
+ *
+ * The driver load routine has to do several things:
+ * - drive output discovery via intel_modeset_init()
+ * - initialize the memory manager
+ * - allocate initial config memory
+ * - setup the DRM framebuffer with the allocated memory
+ */
+int i915_driver_load(struct drm_device *dev, unsigned long flags)
+{
+ struct drm_i915_private *dev_priv;
+ int ret = 0;
+
+ dev_priv = kzalloc(sizeof(*dev_priv), GFP_KERNEL);
+ if (dev_priv == NULL)
+ return -ENOMEM;
+
+ dev->dev_private = dev_priv;
+ /* Must be set before calling __i915_printk */
+ dev_priv->dev = dev;
+
+ ret = i915_driver_init_early(dev_priv, dev,
+ (struct intel_device_info *)flags);
+
+ if (ret < 0)
+ goto out_free_priv;
+
+ intel_runtime_pm_get(dev_priv);
+
+ ret = i915_driver_init_mmio(dev_priv);
+ if (ret < 0)
+ goto out_runtime_pm_put;
+
+ ret = i915_driver_init_hw(dev_priv);
+ if (ret < 0)
+ goto out_cleanup_mmio;
+
+ /*
+ * TODO: move the vblank init and parts of modeset init steps into one
+ * of the i915_driver_init_/i915_driver_register functions according
+ * to the role/effect of the given init step.
+ */
+ if (INTEL_INFO(dev)->num_pipes) {
+ ret = drm_vblank_init(dev, INTEL_INFO(dev)->num_pipes);
+ if (ret)
+ goto out_cleanup_hw;
+ }
+
+ ret = i915_load_modeset_init(dev);
+ if (ret < 0)
+ goto out_cleanup_vblank;
+
+ i915_driver_register(dev_priv);
+
+ intel_runtime_pm_enable(dev_priv);
intel_runtime_pm_put(dev_priv);
return 0;
-out_power_well:
- intel_power_domains_fini(dev_priv);
+out_cleanup_vblank:
drm_vblank_cleanup(dev);
-out_gem_unload:
- i915_gem_shrinker_cleanup(dev_priv);
-
- if (dev->pdev->msi_enabled)
- pci_disable_msi(dev->pdev);
-
- intel_teardown_mchbar(dev);
- pm_qos_remove_request(&dev_priv->pm_qos);
- arch_phys_wc_del(dev_priv->gtt.mtrr);
- io_mapping_free(dev_priv->gtt.mappable);
-out_gtt:
- i915_global_gtt_cleanup(dev);
-out_uncore_fini:
- intel_uncore_fini(dev);
- i915_mmio_cleanup(dev);
-put_bridge:
- pci_dev_put(dev_priv->bridge_dev);
- i915_gem_load_cleanup(dev);
+out_cleanup_hw:
+ i915_driver_cleanup_hw(dev_priv);
+out_cleanup_mmio:
+ i915_driver_cleanup_mmio(dev_priv);
out_runtime_pm_put:
intel_runtime_pm_put(dev_priv);
- i915_workqueues_cleanup(dev_priv);
+ i915_driver_cleanup_early(dev_priv);
out_free_priv:
+ i915_load_error(dev_priv, "Device initialization failed (%d)\n", ret);
+
kfree(dev_priv);
return ret;
@@ -1198,26 +1412,15 @@
intel_fbdev_fini(dev);
- i915_audio_component_cleanup(dev_priv);
-
ret = i915_gem_suspend(dev);
if (ret) {
DRM_ERROR("failed to idle hardware: %d\n", ret);
return ret;
}
- intel_power_domains_fini(dev_priv);
+ intel_display_power_get(dev_priv, POWER_DOMAIN_INIT);
- intel_gpu_ips_teardown();
-
- i915_teardown_sysfs(dev);
-
- i915_gem_shrinker_cleanup(dev_priv);
-
- io_mapping_free(dev_priv->gtt.mappable);
- arch_phys_wc_del(dev_priv->gtt.mtrr);
-
- acpi_video_unregister();
+ i915_driver_unregister(dev_priv);
drm_vblank_cleanup(dev);
@@ -1246,31 +1449,24 @@
cancel_delayed_work_sync(&dev_priv->gpu_error.hangcheck_work);
i915_destroy_error_state(dev);
- if (dev->pdev->msi_enabled)
- pci_disable_msi(dev->pdev);
-
- intel_opregion_fini(dev);
-
/* Flush any outstanding unpin_work. */
flush_workqueue(dev_priv->wq);
intel_guc_ucode_fini(dev);
mutex_lock(&dev->struct_mutex);
- i915_gem_cleanup_ringbuffer(dev);
+ i915_gem_cleanup_engines(dev);
i915_gem_context_fini(dev);
mutex_unlock(&dev->struct_mutex);
intel_fbc_cleanup_cfb(dev_priv);
- pm_qos_remove_request(&dev_priv->pm_qos);
+ intel_power_domains_fini(dev_priv);
- i915_global_gtt_cleanup(dev);
+ i915_driver_cleanup_hw(dev_priv);
+ i915_driver_cleanup_mmio(dev_priv);
- intel_uncore_fini(dev);
- i915_mmio_cleanup(dev);
+ intel_display_power_put(dev_priv, POWER_DOMAIN_INIT);
- i915_gem_load_cleanup(dev);
- pci_dev_put(dev_priv->bridge_dev);
- i915_workqueues_cleanup(dev_priv);
+ i915_driver_cleanup_early(dev_priv);
kfree(dev_priv);
return 0;
diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c
index 6d2fb3f..d37c0a6 100644
--- a/drivers/gpu/drm/i915/i915_drv.c
+++ b/drivers/gpu/drm/i915/i915_drv.c
@@ -66,6 +66,11 @@
#define IVB_CURSOR_OFFSETS \
.cursor_offsets = { CURSOR_A_OFFSET, IVB_CURSOR_B_OFFSET, IVB_CURSOR_C_OFFSET }
+#define BDW_COLORS \
+ .color = { .degamma_lut_size = 512, .gamma_lut_size = 512 }
+#define CHV_COLORS \
+ .color = { .degamma_lut_size = 65, .gamma_lut_size = 257 }
+
static const struct intel_device_info intel_i830_info = {
.gen = 2, .is_mobile = 1, .cursor_needs_physical = 1, .num_pipes = 2,
.has_overlay = 1, .overlay_needs_physical = 1,
@@ -288,24 +293,28 @@
.is_mobile = 1,
};
+#define BDW_FEATURES \
+ HSW_FEATURES, \
+ BDW_COLORS
+
static const struct intel_device_info intel_broadwell_d_info = {
- HSW_FEATURES,
+ BDW_FEATURES,
.gen = 8,
};
static const struct intel_device_info intel_broadwell_m_info = {
- HSW_FEATURES,
+ BDW_FEATURES,
.gen = 8, .is_mobile = 1,
};
static const struct intel_device_info intel_broadwell_gt3d_info = {
- HSW_FEATURES,
+ BDW_FEATURES,
.gen = 8,
.ring_mask = RENDER_RING | BSD_RING | BLT_RING | VEBOX_RING | BSD2_RING,
};
static const struct intel_device_info intel_broadwell_gt3m_info = {
- HSW_FEATURES,
+ BDW_FEATURES,
.gen = 8, .is_mobile = 1,
.ring_mask = RENDER_RING | BSD_RING | BLT_RING | VEBOX_RING | BSD2_RING,
};
@@ -318,16 +327,17 @@
.display_mmio_offset = VLV_DISPLAY_BASE,
GEN_CHV_PIPEOFFSETS,
CURSOR_OFFSETS,
+ CHV_COLORS,
};
static const struct intel_device_info intel_skylake_info = {
- HSW_FEATURES,
+ BDW_FEATURES,
.is_skylake = 1,
.gen = 9,
};
static const struct intel_device_info intel_skylake_gt3_info = {
- HSW_FEATURES,
+ BDW_FEATURES,
.is_skylake = 1,
.gen = 9,
.ring_mask = RENDER_RING | BSD_RING | BLT_RING | VEBOX_RING | BSD2_RING,
@@ -345,18 +355,17 @@
.has_fbc = 1,
GEN_DEFAULT_PIPEOFFSETS,
IVB_CURSOR_OFFSETS,
+ BDW_COLORS,
};
static const struct intel_device_info intel_kabylake_info = {
- HSW_FEATURES,
- .is_preliminary = 1,
+ BDW_FEATURES,
.is_kabylake = 1,
.gen = 9,
};
static const struct intel_device_info intel_kabylake_gt3_info = {
- HSW_FEATURES,
- .is_preliminary = 1,
+ BDW_FEATURES,
.is_kabylake = 1,
.gen = 9,
.ring_mask = RENDER_RING | BSD_RING | BLT_RING | VEBOX_RING | BSD2_RING,
@@ -504,6 +513,7 @@
WARN_ON(!IS_SKYLAKE(dev) &&
!IS_KABYLAKE(dev));
} else if ((id == INTEL_PCH_P2X_DEVICE_ID_TYPE) ||
+ (id == INTEL_PCH_P3X_DEVICE_ID_TYPE) ||
((id == INTEL_PCH_QEMU_DEVICE_ID_TYPE) &&
pch->subsystem_vendor == 0x1af4 &&
pch->subsystem_device == 0x1100)) {
@@ -557,10 +567,9 @@
drm_modeset_unlock_all(dev);
}
-static int intel_suspend_complete(struct drm_i915_private *dev_priv);
static int vlv_resume_prepare(struct drm_i915_private *dev_priv,
bool rpm_resume);
-static int bxt_resume_prepare(struct drm_i915_private *dev_priv);
+static int vlv_suspend_complete(struct drm_i915_private *dev_priv);
static bool suspend_to_idle(struct drm_i915_private *dev_priv)
{
@@ -630,8 +639,7 @@
intel_display_set_init_power(dev_priv, false);
- if (HAS_CSR(dev_priv))
- flush_work(&dev_priv->csr.work);
+ intel_csr_ucode_suspend(dev_priv);
out:
enable_rpm_wakeref_asserts(dev_priv);
@@ -647,7 +655,8 @@
disable_rpm_wakeref_asserts(dev_priv);
- fw_csr = suspend_to_idle(dev_priv) && dev_priv->csr.dmc_payload;
+ fw_csr = !IS_BROXTON(dev_priv) &&
+ suspend_to_idle(dev_priv) && dev_priv->csr.dmc_payload;
/*
* In case of firmware assisted context save/restore don't manually
* deinit the power domains. This also means the CSR/DMC firmware will
@@ -658,7 +667,13 @@
if (!fw_csr)
intel_power_domains_suspend(dev_priv);
- ret = intel_suspend_complete(dev_priv);
+ ret = 0;
+ if (IS_BROXTON(dev_priv))
+ bxt_enable_dc9(dev_priv);
+ else if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv))
+ hsw_enable_pc8(dev_priv);
+ else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
+ ret = vlv_suspend_complete(dev_priv);
if (ret) {
DRM_ERROR("Suspend complete failed: %d\n", ret);
@@ -722,6 +737,8 @@
disable_rpm_wakeref_asserts(dev_priv);
+ intel_csr_ucode_resume(dev_priv);
+
mutex_lock(&dev->struct_mutex);
i915_gem_restore_gtt_mappings(dev);
mutex_unlock(&dev->struct_mutex);
@@ -850,21 +867,25 @@
intel_uncore_early_sanitize(dev, true);
- if (IS_BROXTON(dev))
- ret = bxt_resume_prepare(dev_priv);
- else if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv))
+ if (IS_BROXTON(dev)) {
+ if (!dev_priv->suspended_to_idle)
+ gen9_sanitize_dc_state(dev_priv);
+ bxt_disable_dc9(dev_priv);
+ } else if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv)) {
hsw_disable_pc8(dev_priv);
+ }
intel_uncore_sanitize(dev);
- if (!(dev_priv->suspended_to_idle && dev_priv->csr.dmc_payload))
+ if (IS_BROXTON(dev_priv) ||
+ !(dev_priv->suspended_to_idle && dev_priv->csr.dmc_payload))
intel_power_domains_init_hw(dev_priv, true);
+ enable_rpm_wakeref_asserts(dev_priv);
+
out:
dev_priv->suspended_to_idle = false;
- enable_rpm_wakeref_asserts(dev_priv);
-
return ret;
}
@@ -900,23 +921,32 @@
int i915_reset(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- bool simulated;
+ struct i915_gpu_error *error = &dev_priv->gpu_error;
+ unsigned reset_counter;
int ret;
intel_reset_gt_powersave(dev);
mutex_lock(&dev->struct_mutex);
+ /* Clear any previous failed attempts at recovery. Time to try again. */
+ atomic_andnot(I915_WEDGED, &error->reset_counter);
+
+ /* Clear the reset-in-progress flag and increment the reset epoch. */
+ reset_counter = atomic_inc_return(&error->reset_counter);
+ if (WARN_ON(__i915_reset_in_progress(reset_counter))) {
+ ret = -EIO;
+ goto error;
+ }
+
i915_gem_reset(dev);
- simulated = dev_priv->gpu_error.stop_rings != 0;
-
- ret = intel_gpu_reset(dev);
+ ret = intel_gpu_reset(dev, ALL_ENGINES);
/* Also reset the gpu hangman. */
- if (simulated) {
+ if (error->stop_rings != 0) {
DRM_INFO("Simulated gpu hang, resetting stop_rings\n");
- dev_priv->gpu_error.stop_rings = 0;
+ error->stop_rings = 0;
if (ret == -ENODEV) {
DRM_INFO("Reset not implemented, but ignoring "
"error for simulated gpu hangs\n");
@@ -928,9 +958,11 @@
pr_notice("drm/i915: Resetting chip after gpu hang\n");
if (ret) {
- DRM_ERROR("Failed to reset chip: %i\n", ret);
- mutex_unlock(&dev->struct_mutex);
- return ret;
+ if (ret != -ENODEV)
+ DRM_ERROR("Failed to reset chip: %i\n", ret);
+ else
+ DRM_DEBUG_DRIVER("GPU reset disabled\n");
+ goto error;
}
intel_overlay_reset(dev_priv);
@@ -949,20 +981,14 @@
* was running at the time of the reset (i.e. we weren't VT
* switched away).
*/
-
- /* Used to prevent gem_check_wedged returning -EAGAIN during gpu reset */
- dev_priv->gpu_error.reload_in_reset = true;
-
ret = i915_gem_init_hw(dev);
-
- dev_priv->gpu_error.reload_in_reset = false;
-
- mutex_unlock(&dev->struct_mutex);
if (ret) {
DRM_ERROR("Failed hw init on reset %d\n", ret);
- return ret;
+ goto error;
}
+ mutex_unlock(&dev->struct_mutex);
+
/*
* rps/rc6 re-init is necessary to restore state lost after the
* reset and the re-install of gt irqs. Skip for ironlake per
@@ -973,6 +999,11 @@
intel_enable_gt_powersave(dev);
return 0;
+
+error:
+ atomic_or(I915_WEDGED, &error->reset_counter);
+ mutex_unlock(&dev->struct_mutex);
+ return ret;
}
static int i915_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
@@ -1079,44 +1110,6 @@
return i915_drm_resume(drm_dev);
}
-static int hsw_suspend_complete(struct drm_i915_private *dev_priv)
-{
- hsw_enable_pc8(dev_priv);
-
- return 0;
-}
-
-static int bxt_suspend_complete(struct drm_i915_private *dev_priv)
-{
- struct drm_device *dev = dev_priv->dev;
-
- /* TODO: when DC5 support is added disable DC5 here. */
-
- broxton_ddi_phy_uninit(dev);
- broxton_uninit_cdclk(dev);
- bxt_enable_dc9(dev_priv);
-
- return 0;
-}
-
-static int bxt_resume_prepare(struct drm_i915_private *dev_priv)
-{
- struct drm_device *dev = dev_priv->dev;
-
- /* TODO: when CSR FW support is added make sure the FW is loaded */
-
- bxt_disable_dc9(dev_priv);
-
- /*
- * TODO: when DC5 support is added enable DC5 here if the CSR FW
- * is available.
- */
- broxton_init_cdclk(dev);
- broxton_ddi_phy_init(dev);
-
- return 0;
-}
-
/*
* Save all Gunit registers that may be lost after a D3 and a subsequent
* S0i[R123] transition. The list of registers needing a save/restore is
@@ -1420,7 +1413,7 @@
if (err)
goto err2;
- if (!IS_CHERRYVIEW(dev_priv->dev))
+ if (!IS_CHERRYVIEW(dev_priv))
vlv_save_gunit_s0ix_state(dev_priv);
err = vlv_force_gfx_clock(dev_priv, false);
@@ -1452,7 +1445,7 @@
*/
ret = vlv_force_gfx_clock(dev_priv, true);
- if (!IS_CHERRYVIEW(dev_priv->dev))
+ if (!IS_CHERRYVIEW(dev_priv))
vlv_restore_gunit_s0ix_state(dev_priv);
err = vlv_allow_gt_wake(dev_priv, true);
@@ -1522,7 +1515,16 @@
intel_suspend_gt_powersave(dev);
intel_runtime_pm_disable_interrupts(dev_priv);
- ret = intel_suspend_complete(dev_priv);
+ ret = 0;
+ if (IS_BROXTON(dev_priv)) {
+ bxt_display_core_uninit(dev_priv);
+ bxt_enable_dc9(dev_priv);
+ } else if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv)) {
+ hsw_enable_pc8(dev_priv);
+ } else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) {
+ ret = vlv_suspend_complete(dev_priv);
+ }
+
if (ret) {
DRM_ERROR("Runtime suspend failed, disabling it (%d)\n", ret);
intel_runtime_pm_enable_interrupts(dev_priv);
@@ -1596,12 +1598,17 @@
if (IS_GEN6(dev_priv))
intel_init_pch_refclk(dev);
- if (IS_BROXTON(dev))
- ret = bxt_resume_prepare(dev_priv);
- else if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv))
+ if (IS_BROXTON(dev)) {
+ bxt_disable_dc9(dev_priv);
+ bxt_display_core_init(dev_priv, true);
+ if (dev_priv->csr.dmc_payload &&
+ (dev_priv->csr.allowed_dc_mask & DC_STATE_EN_UPTO_DC5))
+ gen9_enable_dc5(dev_priv);
+ } else if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv)) {
hsw_disable_pc8(dev_priv);
- else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
+ } else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) {
ret = vlv_resume_prepare(dev_priv, true);
+ }
/*
* No point of rolling back things in case of an error, as the best
@@ -1632,26 +1639,6 @@
return ret;
}
-/*
- * This function implements common functionality of runtime and system
- * suspend sequence.
- */
-static int intel_suspend_complete(struct drm_i915_private *dev_priv)
-{
- int ret;
-
- if (IS_BROXTON(dev_priv))
- ret = bxt_suspend_complete(dev_priv);
- else if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv))
- ret = hsw_suspend_complete(dev_priv);
- else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
- ret = vlv_suspend_complete(dev_priv);
- else
- ret = 0;
-
- return ret;
-}
-
static const struct dev_pm_ops i915_pm_ops = {
/*
* S0ix (via system suspend) and S3 event handlers [PMSG_SUSPEND,
@@ -1772,10 +1759,8 @@
if (i915.modeset == 0)
driver.driver_features &= ~DRIVER_MODESET;
-#ifdef CONFIG_VGA_CONSOLE
if (vgacon_text_force() && i915.modeset == -1)
driver.driver_features &= ~DRIVER_MODESET;
-#endif
if (!(driver.driver_features & DRIVER_MODESET)) {
/* Silently fail loading to not upset userspace. */
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index daba7eb..b87ca4f 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -33,33 +33,40 @@
#include <uapi/drm/i915_drm.h>
#include <uapi/drm/drm_fourcc.h>
-#include <drm/drmP.h>
-#include "i915_params.h"
-#include "i915_reg.h"
-#include "intel_bios.h"
-#include "intel_ringbuffer.h"
-#include "intel_lrc.h"
-#include "i915_gem_gtt.h"
-#include "i915_gem_render_state.h"
#include <linux/io-mapping.h>
#include <linux/i2c.h>
#include <linux/i2c-algo-bit.h>
-#include <drm/intel-gtt.h>
-#include <drm/drm_legacy.h> /* for struct drm_dma_handle */
-#include <drm/drm_gem.h>
#include <linux/backlight.h>
#include <linux/hashtable.h>
#include <linux/intel-iommu.h>
#include <linux/kref.h>
#include <linux/pm_qos.h>
+#include <linux/shmem_fs.h>
+
+#include <drm/drmP.h>
+#include <drm/intel-gtt.h>
+#include <drm/drm_legacy.h> /* for struct drm_dma_handle */
+#include <drm/drm_gem.h>
+
+#include "i915_params.h"
+#include "i915_reg.h"
+
+#include "intel_bios.h"
+#include "intel_dpll_mgr.h"
#include "intel_guc.h"
+#include "intel_lrc.h"
+#include "intel_ringbuffer.h"
+
+#include "i915_gem.h"
+#include "i915_gem_gtt.h"
+#include "i915_gem_render_state.h"
/* General customization:
*/
#define DRIVER_NAME "i915"
#define DRIVER_DESC "Intel Graphics"
-#define DRIVER_DATE "20160229"
+#define DRIVER_DATE "20160425"
#undef WARN_ON
/* Many gcc seem to no see through this and fall over :( */
@@ -97,6 +104,10 @@
#define I915_STATE_WARN_ON(x) \
I915_STATE_WARN((x), "%s", "WARN_ON(" __stringify(x) ")")
+bool __i915_inject_load_failure(const char *func, int line);
+#define i915_inject_load_failure() \
+ __i915_inject_load_failure(__func__, __LINE__)
+
static inline const char *yesno(bool v)
{
return v ? "yes" : "no";
@@ -122,9 +133,35 @@
TRANSCODER_B,
TRANSCODER_C,
TRANSCODER_EDP,
+ TRANSCODER_DSI_A,
+ TRANSCODER_DSI_C,
I915_MAX_TRANSCODERS
};
-#define transcoder_name(t) ((t) + 'A')
+
+static inline const char *transcoder_name(enum transcoder transcoder)
+{
+ switch (transcoder) {
+ case TRANSCODER_A:
+ return "A";
+ case TRANSCODER_B:
+ return "B";
+ case TRANSCODER_C:
+ return "C";
+ case TRANSCODER_EDP:
+ return "EDP";
+ case TRANSCODER_DSI_A:
+ return "DSI A";
+ case TRANSCODER_DSI_C:
+ return "DSI C";
+ default:
+ return "<invalid>";
+ }
+}
+
+static inline bool transcoder_is_dsi(enum transcoder transcoder)
+{
+ return transcoder == TRANSCODER_DSI_A || transcoder == TRANSCODER_DSI_C;
+}
/*
* I915_MAX_PLANES in the enum below is the maximum (across all platforms)
@@ -176,6 +213,8 @@
POWER_DOMAIN_TRANSCODER_B,
POWER_DOMAIN_TRANSCODER_C,
POWER_DOMAIN_TRANSCODER_EDP,
+ POWER_DOMAIN_TRANSCODER_DSI_A,
+ POWER_DOMAIN_TRANSCODER_DSI_C,
POWER_DOMAIN_PORT_DDI_A_LANES,
POWER_DOMAIN_PORT_DDI_B_LANES,
POWER_DOMAIN_PORT_DDI_C_LANES,
@@ -273,6 +312,10 @@
(__s) < INTEL_INFO(__dev_priv)->num_sprites[(__p)]; \
(__s)++)
+#define for_each_port_masked(__port, __ports_mask) \
+ for ((__port) = PORT_A; (__port) < I915_MAX_PORTS; (__port)++) \
+ for_each_if ((__ports_mask) & (1 << (__port)))
+
#define for_each_crtc(dev, crtc) \
list_for_each_entry(crtc, &dev->mode_config.crtc_list, head)
@@ -340,81 +383,6 @@
unsigned int bsd_ring;
};
-enum intel_dpll_id {
- DPLL_ID_PRIVATE = -1, /* non-shared dpll in use */
- /* real shared dpll ids must be >= 0 */
- DPLL_ID_PCH_PLL_A = 0,
- DPLL_ID_PCH_PLL_B = 1,
- /* hsw/bdw */
- DPLL_ID_WRPLL1 = 0,
- DPLL_ID_WRPLL2 = 1,
- DPLL_ID_SPLL = 2,
-
- /* skl */
- DPLL_ID_SKL_DPLL1 = 0,
- DPLL_ID_SKL_DPLL2 = 1,
- DPLL_ID_SKL_DPLL3 = 2,
-};
-#define I915_NUM_PLLS 3
-
-struct intel_dpll_hw_state {
- /* i9xx, pch plls */
- uint32_t dpll;
- uint32_t dpll_md;
- uint32_t fp0;
- uint32_t fp1;
-
- /* hsw, bdw */
- uint32_t wrpll;
- uint32_t spll;
-
- /* skl */
- /*
- * DPLL_CTRL1 has 6 bits for each each this DPLL. We store those in
- * lower part of ctrl1 and they get shifted into position when writing
- * the register. This allows us to easily compare the state to share
- * the DPLL.
- */
- uint32_t ctrl1;
- /* HDMI only, 0 when used for DP */
- uint32_t cfgcr1, cfgcr2;
-
- /* bxt */
- uint32_t ebb0, ebb4, pll0, pll1, pll2, pll3, pll6, pll8, pll9, pll10,
- pcsdw12;
-};
-
-struct intel_shared_dpll_config {
- unsigned crtc_mask; /* mask of CRTCs sharing this PLL */
- struct intel_dpll_hw_state hw_state;
-};
-
-struct intel_shared_dpll {
- struct intel_shared_dpll_config config;
-
- int active; /* count of number of active CRTCs (i.e. DPMS on) */
- bool on; /* is the PLL actually active? Disabled during modeset */
- const char *name;
- /* should match the index in the dev_priv->shared_dplls array */
- enum intel_dpll_id id;
- /* The mode_set hook is optional and should be used together with the
- * intel_prepare_shared_dpll function. */
- void (*mode_set)(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll);
- void (*enable)(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll);
- void (*disable)(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll);
- bool (*get_hw_state)(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll,
- struct intel_dpll_hw_state *hw_state);
-};
-
-#define SKL_DPLL0 0
-#define SKL_DPLL1 1
-#define SKL_DPLL2 2
-#define SKL_DPLL3 3
-
/* Used by dp and fdi links */
struct intel_link_m_n {
uint32_t tu;
@@ -533,7 +501,8 @@
u32 cpu_ring_head;
u32 cpu_ring_tail;
- u32 semaphore_seqno[I915_NUM_RINGS - 1];
+ u32 last_seqno;
+ u32 semaphore_seqno[I915_NUM_ENGINES - 1];
/* Register state */
u32 start;
@@ -553,7 +522,7 @@
u32 fault_reg;
u64 faddr;
u32 rc_psmi; /* sleep state */
- u32 semaphore_mboxes[I915_NUM_RINGS - 1];
+ u32 semaphore_mboxes[I915_NUM_ENGINES - 1];
struct drm_i915_error_object {
int page_count;
@@ -561,6 +530,8 @@
u32 *pages[0];
} *ringbuffer, *batchbuffer, *wa_batchbuffer, *ctx, *hws_page;
+ struct drm_i915_error_object *wa_ctx;
+
struct drm_i915_error_request {
long jiffies;
u32 seqno;
@@ -577,12 +548,12 @@
pid_t pid;
char comm[TASK_COMM_LEN];
- } ring[I915_NUM_RINGS];
+ } ring[I915_NUM_ENGINES];
struct drm_i915_error_buffer {
u32 size;
u32 name;
- u32 rseqno[I915_NUM_RINGS], wseqno;
+ u32 rseqno[I915_NUM_ENGINES], wseqno;
u64 gtt_offset;
u32 read_domains;
u32 write_domain;
@@ -611,27 +582,12 @@
struct drm_i915_display_funcs {
int (*get_display_clock_speed)(struct drm_device *dev);
int (*get_fifo_size)(struct drm_device *dev, int plane);
- /**
- * find_dpll() - Find the best values for the PLL
- * @limit: limits for the PLL
- * @crtc: current CRTC
- * @target: target frequency in kHz
- * @refclk: reference clock frequency in kHz
- * @match_clock: if provided, @best_clock P divider must
- * match the P divider from @match_clock
- * used for LVDS downclocking
- * @best_clock: best PLL values found
- *
- * Returns true on success, false on failure.
- */
- bool (*find_dpll)(const struct intel_limit *limit,
- struct intel_crtc_state *crtc_state,
- int target, int refclk,
- struct dpll *match_clock,
- struct dpll *best_clock);
- int (*compute_pipe_wm)(struct intel_crtc *crtc,
- struct drm_atomic_state *state);
- void (*program_watermarks)(struct intel_crtc_state *cstate);
+ int (*compute_pipe_wm)(struct intel_crtc_state *cstate);
+ int (*compute_intermediate_wm)(struct drm_device *dev,
+ struct intel_crtc *intel_crtc,
+ struct intel_crtc_state *newstate);
+ void (*initial_watermarks)(struct intel_crtc_state *cstate);
+ void (*optimize_watermarks)(struct intel_crtc_state *cstate);
void (*update_wm)(struct drm_crtc *crtc);
int (*modeset_calc_cdclk)(struct drm_atomic_state *state);
void (*modeset_commit_cdclk)(struct drm_atomic_state *state);
@@ -662,6 +618,9 @@
/* render clock increase/decrease */
/* display clock increase/decrease */
/* pll clock increase/decrease */
+
+ void (*load_csc_matrix)(struct drm_crtc_state *crtc_state);
+ void (*load_luts)(struct drm_crtc_state *crtc_state);
};
enum forcewake_domain_id {
@@ -681,6 +640,13 @@
FORCEWAKE_MEDIA)
};
+#define FW_REG_READ (1)
+#define FW_REG_WRITE (2)
+
+enum forcewake_domains
+intel_uncore_forcewake_for_reg(struct drm_i915_private *dev_priv,
+ i915_reg_t reg, unsigned int op);
+
struct intel_uncore_funcs {
void (*force_wake_get)(struct drm_i915_private *dev_priv,
enum forcewake_domains domains);
@@ -713,8 +679,9 @@
struct intel_uncore_forcewake_domain {
struct drm_i915_private *i915;
enum forcewake_domain_id id;
+ enum forcewake_domains mask;
unsigned wake_count;
- struct timer_list timer;
+ struct hrtimer timer;
i915_reg_t reg_set;
u32 val_set;
u32 val_clear;
@@ -727,14 +694,14 @@
};
/* Iterate over initialised fw domains */
-#define for_each_fw_domain_mask(domain__, mask__, dev_priv__, i__) \
- for ((i__) = 0, (domain__) = &(dev_priv__)->uncore.fw_domain[0]; \
- (i__) < FW_DOMAIN_ID_COUNT; \
- (i__)++, (domain__) = &(dev_priv__)->uncore.fw_domain[i__]) \
- for_each_if (((mask__) & (dev_priv__)->uncore.fw_domains) & (1 << (i__)))
+#define for_each_fw_domain_masked(domain__, mask__, dev_priv__) \
+ for ((domain__) = &(dev_priv__)->uncore.fw_domain[0]; \
+ (domain__) < &(dev_priv__)->uncore.fw_domain[FW_DOMAIN_ID_COUNT]; \
+ (domain__)++) \
+ for_each_if ((mask__) & (domain__)->mask)
-#define for_each_fw_domain(domain__, dev_priv__, i__) \
- for_each_fw_domain_mask(domain__, FORCEWAKE_ALL, dev_priv__, i__)
+#define for_each_fw_domain(domain__, dev_priv__) \
+ for_each_fw_domain_masked(domain__, FORCEWAKE_ALL, dev_priv__)
#define CSR_VERSION(major, minor) ((major) << 16 | (minor))
#define CSR_VERSION_MAJOR(version) ((version) >> 16)
@@ -750,6 +717,7 @@
i915_reg_t mmioaddr[8];
uint32_t mmiodata[8];
uint32_t dc_state;
+ uint32_t allowed_dc_mask;
};
#define DEV_INFO_FOR_EACH_FLAG(func, sep) \
@@ -779,6 +747,7 @@
func(overlay_needs_physical) sep \
func(supports_tv) sep \
func(has_llc) sep \
+ func(has_snoop) sep \
func(has_ddi) sep \
func(has_fpga_dbg)
@@ -810,6 +779,11 @@
u8 has_slice_pg:1;
u8 has_subslice_pg:1;
u8 has_eu_pg:1;
+
+ struct color_luts {
+ u16 degamma_lut_size;
+ u16 gamma_lut_size;
+ } color;
};
#undef DEFINE_FLAG
@@ -891,7 +865,7 @@
struct i915_vma *lrc_vma;
u64 lrc_desc;
uint32_t *lrc_reg_state;
- } engine[I915_NUM_RINGS];
+ } engine[I915_NUM_ENGINES];
struct list_head link;
};
@@ -1036,6 +1010,7 @@
struct intel_gmbus {
struct i2c_adapter adapter;
+#define GMBUS_FORCE_BIT_RETRY (1U << 31)
u32 force_bit;
u32 reg0;
i915_reg_t gpio_reg;
@@ -1159,6 +1134,7 @@
u8 efficient_freq; /* AKA RPe. Pre-determined balanced frequency */
u8 rp1_freq; /* "less than" RP0 power/freqency */
u8 rp0_freq; /* Non-overclocked max frequency. */
+ u16 gpll_ref_freq; /* vlv/chv GPLL reference frequency */
u8 up_threshold; /* Current %busy required to uplock */
u8 down_threshold; /* Current %busy required to downclock */
@@ -1298,6 +1274,7 @@
struct i915_hw_ppgtt *aliasing_ppgtt;
struct notifier_block oom_notifier;
+ struct notifier_block vmap_notifier;
struct shrinker shrinker;
bool shrinker_no_lock_stealing;
@@ -1423,9 +1400,6 @@
/* For missed irq/seqno simulation. */
unsigned int test_irq_rings;
-
- /* Used to prevent gem_check_wedged returning -EAGAIN during gpu reset */
- bool reload_in_reset;
};
enum modeset_restore {
@@ -1482,21 +1456,23 @@
unsigned int lvds_use_ssc:1;
unsigned int display_clock_mode:1;
unsigned int fdi_rx_polarity_inverted:1;
- unsigned int has_mipi:1;
+ unsigned int panel_type:4;
int lvds_ssc_freq;
unsigned int bios_lvds_val; /* initial [PCH_]LVDS reg val in VBIOS */
enum drrs_support_type drrs_type;
- /* eDP */
- int edp_rate;
- int edp_lanes;
- int edp_preemphasis;
- int edp_vswing;
- bool edp_initialized;
- bool edp_support;
- int edp_bpp;
- struct edp_power_seq edp_pps;
+ struct {
+ int rate;
+ int lanes;
+ int preemphasis;
+ int vswing;
+ bool low_vswing;
+ bool initialized;
+ bool support;
+ int bpp;
+ struct edp_power_seq pps;
+ } edp;
struct {
bool full_link;
@@ -1516,7 +1492,6 @@
/* MIPI DSI */
struct {
- u16 port;
u16 panel_id;
struct mipi_config *config;
struct mipi_pps_data *pps;
@@ -1532,6 +1507,7 @@
union child_device_config *child_dev;
struct ddi_vbt_port_info ddi_port_info[I915_MAX_PORTS];
+ struct sdvo_device_mapping sdvo_mappings[2];
};
enum intel_ddb_partitioning {
@@ -1706,7 +1682,7 @@
struct i915_workarounds {
struct i915_wa_reg reg[I915_MAX_WA_REGS];
u32 count;
- u32 hw_whitelist_count[I915_NUM_RINGS];
+ u32 hw_whitelist_count[I915_NUM_ENGINES];
};
struct i915_virtual_gpu {
@@ -1719,7 +1695,7 @@
uint32_t dispatch_flags;
uint32_t args_batch_start_offset;
uint64_t batch_obj_vm_offset;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
struct drm_i915_gem_object *batch_obj;
struct intel_context *ctx;
struct drm_i915_gem_request *request;
@@ -1771,7 +1747,7 @@
wait_queue_head_t gmbus_wait_queue;
struct pci_dev *bridge_dev;
- struct intel_engine_cs ring[I915_NUM_RINGS];
+ struct intel_engine_cs engine[I915_NUM_ENGINES];
struct drm_i915_gem_object *semaphore_obj;
uint32_t last_seqno, next_seqno;
@@ -1829,6 +1805,7 @@
unsigned int skl_boot_cdclk;
unsigned int cdclk_freq, max_cdclk_freq, atomic_cdclk_freq;
unsigned int max_dotclk_freq;
+ unsigned int rawclk_freq;
unsigned int hpll_freq;
unsigned int czclk_freq;
@@ -1855,7 +1832,7 @@
struct drm_atomic_state *modeset_restore_state;
struct list_head vm_list; /* Global list of all address spaces */
- struct i915_gtt gtt; /* VM representing the global address space */
+ struct i915_ggtt ggtt; /* VM representing the global address space */
struct i915_gem_mm mm;
DECLARE_HASHTABLE(mm_structs, 7);
@@ -1863,8 +1840,6 @@
/* Kernel Modesetting */
- struct sdvo_device_mapping sdvo_mappings[2];
-
struct drm_crtc *plane_to_crtc_mapping[I915_MAX_PIPES];
struct drm_crtc *pipe_to_crtc_mapping[I915_MAX_PIPES];
wait_queue_head_t pending_flip_queue;
@@ -1876,6 +1851,14 @@
/* dpll and cdclk state is protected by connection_mutex */
int num_shared_dpll;
struct intel_shared_dpll shared_dplls[I915_NUM_PLLS];
+ const struct intel_dpll_mgr *dpll_mgr;
+
+ /*
+ * dpll_lock serializes intel_{prepare,enable,disable}_shared_dpll.
+ * Must be global rather than per dpll, because on some platforms
+ * plls share registers.
+ */
+ struct mutex dpll_lock;
unsigned int active_crtcs;
unsigned int min_pixclk[I915_MAX_PIPES];
@@ -1884,9 +1867,6 @@
struct i915_workarounds workarounds;
- /* Reclocking support */
- bool render_reclock_avail;
-
struct i915_frontbuffer_tracking fb_tracking;
u16 orig_clock;
@@ -1896,7 +1876,7 @@
struct intel_l3_parity l3_parity;
/* Cannot be determined by PCIID. You must always read a register. */
- size_t ellc_size;
+ u32 edram_cap;
/* gen6+ rps state */
struct intel_gen6_power_mgmt rps;
@@ -1936,7 +1916,15 @@
u32 fdi_rx_config;
+ /* Shadow for DISPLAY_PHY_CONTROL which can't be safely read */
u32 chv_phy_control;
+ /*
+ * Shadows for CHV DPLL_MD regs to keep the state
+ * checker somewhat working in the presence hardware
+ * crappiness (can't read out DPLL_MD for pipes B & C).
+ */
+ u32 chv_dpll_md[I915_MAX_PIPES];
+ u32 bxt_phy_grc;
u32 suspend_count;
bool suspended_to_idle;
@@ -1980,6 +1968,13 @@
};
uint8_t max_level;
+
+ /*
+ * Should be held around atomic WM register writing; also
+ * protects * intel_crtc->wm.active and
+ * cstate->wm.need_postvbl_update.
+ */
+ struct mutex wm_mutex;
} wm;
struct i915_runtime_pm pm;
@@ -1989,15 +1984,13 @@
int (*execbuf_submit)(struct i915_execbuffer_params *params,
struct drm_i915_gem_execbuffer2 *args,
struct list_head *vmas);
- int (*init_rings)(struct drm_device *dev);
- void (*cleanup_ring)(struct intel_engine_cs *ring);
- void (*stop_ring)(struct intel_engine_cs *ring);
+ int (*init_engines)(struct drm_device *dev);
+ void (*cleanup_engine)(struct intel_engine_cs *engine);
+ void (*stop_engine)(struct intel_engine_cs *engine);
} gt;
struct intel_context *kernel_context;
- bool edp_low_vswing;
-
/* perform PHY state sanity checks? */
bool chv_phy_assert[2];
@@ -2024,10 +2017,28 @@
return container_of(guc, struct drm_i915_private, guc);
}
-/* Iterate over initialised rings */
-#define for_each_ring(ring__, dev_priv__, i__) \
- for ((i__) = 0; (i__) < I915_NUM_RINGS; (i__)++) \
- for_each_if ((((ring__) = &(dev_priv__)->ring[(i__)]), intel_ring_initialized((ring__))))
+/* Simple iterator over all initialised engines */
+#define for_each_engine(engine__, dev_priv__) \
+ for ((engine__) = &(dev_priv__)->engine[0]; \
+ (engine__) < &(dev_priv__)->engine[I915_NUM_ENGINES]; \
+ (engine__)++) \
+ for_each_if (intel_engine_initialized(engine__))
+
+/* Iterator with engine_id */
+#define for_each_engine_id(engine__, dev_priv__, id__) \
+ for ((engine__) = &(dev_priv__)->engine[0], (id__) = 0; \
+ (engine__) < &(dev_priv__)->engine[I915_NUM_ENGINES]; \
+ (engine__)++) \
+ for_each_if (((id__) = (engine__)->id, \
+ intel_engine_initialized(engine__)))
+
+/* Iterator over subset of engines selected by mask */
+#define for_each_engine_masked(engine__, dev_priv__, mask__) \
+ for ((engine__) = &(dev_priv__)->engine[0]; \
+ (engine__) < &(dev_priv__)->engine[I915_NUM_ENGINES]; \
+ (engine__)++) \
+ for_each_if (((mask__) & intel_engine_flag(engine__)) && \
+ intel_engine_initialized(engine__))
enum hdmi_force_audio {
HDMI_AUDIO_OFF_DVI = -2, /* no aux data for HDMI-DVI converter */
@@ -2097,7 +2108,7 @@
struct drm_mm_node *stolen;
struct list_head global_list;
- struct list_head ring_list[I915_NUM_RINGS];
+ struct list_head engine_list[I915_NUM_ENGINES];
/** Used in execbuf to temporarily hold a ref */
struct list_head obj_exec_link;
@@ -2108,7 +2119,7 @@
* rendering and so a non-zero seqno), and is not set if it i s on
* inactive (ready to be unbound) list.
*/
- unsigned int active:I915_NUM_RINGS;
+ unsigned int active:I915_NUM_ENGINES;
/**
* This is set if the object has been written to since last bound
@@ -2172,10 +2183,7 @@
struct scatterlist *sg;
int last;
} get_page;
-
- /* prime dma-buf support */
- void *dma_buf_vmapping;
- int vmapping_count;
+ void *mapping;
/** Breadcrumb of last rendering to the buffer.
* There can only be one writer, but we allow for multiple readers.
@@ -2187,7 +2195,7 @@
* read request. This allows for the CPU to read from an active
* buffer by only waiting for the write to complete.
* */
- struct drm_i915_gem_request *last_read_req[I915_NUM_RINGS];
+ struct drm_i915_gem_request *last_read_req[I915_NUM_ENGINES];
struct drm_i915_gem_request *last_write_req;
/** Breadcrumb of last fenced GPU access to the buffer. */
struct drm_i915_gem_request *last_fenced_req;
@@ -2242,7 +2250,8 @@
/** On Which ring this request was generated */
struct drm_i915_private *i915;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
+ unsigned reset_counter;
/** GEM sequence number associated with the previous request,
* when the HWS breadcrumb is equal to this the GPU is processing
@@ -2323,7 +2332,6 @@
struct drm_i915_gem_request * __must_check
i915_gem_request_alloc(struct intel_engine_cs *engine,
struct intel_context *ctx);
-void i915_gem_request_cancel(struct drm_i915_gem_request *req);
void i915_gem_request_free(struct kref *req_ref);
int i915_gem_request_add_to_client(struct drm_i915_gem_request *req,
struct drm_file *file);
@@ -2335,9 +2343,9 @@
}
static inline struct intel_engine_cs *
-i915_gem_request_get_ring(struct drm_i915_gem_request *req)
+i915_gem_request_get_engine(struct drm_i915_gem_request *req)
{
- return req ? req->ring : NULL;
+ return req ? req->engine : NULL;
}
static inline struct drm_i915_gem_request *
@@ -2351,7 +2359,7 @@
static inline void
i915_gem_request_unreference(struct drm_i915_gem_request *req)
{
- WARN_ON(!mutex_is_locked(&req->ring->dev->struct_mutex));
+ WARN_ON(!mutex_is_locked(&req->engine->dev->struct_mutex));
kref_put(&req->ref, i915_gem_request_free);
}
@@ -2363,7 +2371,7 @@
if (!req)
return;
- dev = req->ring->dev;
+ dev = req->engine->dev;
if (kref_put_mutex(&req->ref, i915_gem_request_free, &dev->struct_mutex))
mutex_unlock(&dev->struct_mutex);
}
@@ -2493,6 +2501,7 @@
__p; \
})
#define INTEL_INFO(p) (&__I915__(p)->info)
+#define INTEL_GEN(p) (INTEL_INFO(p)->gen)
#define INTEL_DEVID(p) (INTEL_INFO(p)->device_id)
#define INTEL_REVID(p) (__I915__(p)->dev->pdev->revision)
@@ -2611,13 +2620,17 @@
#define BLT_RING (1<<BCS)
#define VEBOX_RING (1<<VECS)
#define BSD2_RING (1<<VCS2)
+#define ALL_ENGINES (~0)
+
#define HAS_BSD(dev) (INTEL_INFO(dev)->ring_mask & BSD_RING)
#define HAS_BSD2(dev) (INTEL_INFO(dev)->ring_mask & BSD2_RING)
#define HAS_BLT(dev) (INTEL_INFO(dev)->ring_mask & BLT_RING)
#define HAS_VEBOX(dev) (INTEL_INFO(dev)->ring_mask & VEBOX_RING)
#define HAS_LLC(dev) (INTEL_INFO(dev)->has_llc)
+#define HAS_SNOOP(dev) (INTEL_INFO(dev)->has_snoop)
+#define HAS_EDRAM(dev) (__I915__(dev)->edram_cap & EDRAM_ENABLED)
#define HAS_WT(dev) ((IS_HASWELL(dev) || IS_BROADWELL(dev)) && \
- __I915__(dev)->ellc_size)
+ HAS_EDRAM(dev))
#define I915_NEED_GFX_HWS(dev) (INTEL_INFO(dev)->need_gfx_hws)
#define HAS_HW_CONTEXTS(dev) (INTEL_INFO(dev)->gen >= 6)
@@ -2671,7 +2684,7 @@
#define HAS_RUNTIME_PM(dev) (IS_GEN6(dev) || IS_HASWELL(dev) || \
IS_BROADWELL(dev) || IS_VALLEYVIEW(dev) || \
IS_CHERRYVIEW(dev) || IS_SKYLAKE(dev) || \
- IS_KABYLAKE(dev))
+ IS_KABYLAKE(dev) || IS_BROXTON(dev))
#define HAS_RC6(dev) (INTEL_INFO(dev)->gen >= 6)
#define HAS_RC6p(dev) (INTEL_INFO(dev)->gen == 6 || IS_IVYBRIDGE(dev))
@@ -2696,6 +2709,7 @@
#define INTEL_PCH_SPT_DEVICE_ID_TYPE 0xA100
#define INTEL_PCH_SPT_LP_DEVICE_ID_TYPE 0x9D00
#define INTEL_PCH_P2X_DEVICE_ID_TYPE 0x7100
+#define INTEL_PCH_P3X_DEVICE_ID_TYPE 0x7000
#define INTEL_PCH_QEMU_DEVICE_ID_TYPE 0x2900 /* qemu q35 has 2918 */
#define INTEL_PCH_TYPE(dev) (__I915__(dev)->pch_type)
@@ -2727,6 +2741,13 @@
extern int i915_resume_switcheroo(struct drm_device *dev);
/* i915_dma.c */
+void __printf(3, 4)
+__i915_printk(struct drm_i915_private *dev_priv, const char *level,
+ const char *fmt, ...);
+
+#define i915_report_error(dev_priv, fmt, ...) \
+ __i915_printk(dev_priv, KERN_ERR, fmt, ##__VA_ARGS__)
+
extern int i915_driver_load(struct drm_device *, unsigned long flags);
extern int i915_driver_unload(struct drm_device *);
extern int i915_driver_open(struct drm_device *dev, struct drm_file *file);
@@ -2739,9 +2760,11 @@
extern long i915_compat_ioctl(struct file *filp, unsigned int cmd,
unsigned long arg);
#endif
-extern int intel_gpu_reset(struct drm_device *dev);
+extern int intel_gpu_reset(struct drm_device *dev, u32 engine_mask);
extern bool intel_has_gpu_reset(struct drm_device *dev);
extern int i915_reset(struct drm_device *dev);
+extern int intel_guc_reset(struct drm_i915_private *dev_priv);
+extern void intel_engine_init_hangcheck(struct intel_engine_cs *engine);
extern unsigned long i915_chipset_val(struct drm_i915_private *dev_priv);
extern unsigned long i915_mch_val(struct drm_i915_private *dev_priv);
extern unsigned long i915_gfx_val(struct drm_i915_private *dev_priv);
@@ -2758,7 +2781,7 @@
/* i915_irq.c */
void i915_queue_hangcheck(struct drm_device *dev);
__printf(3, 4)
-void i915_handle_error(struct drm_device *dev, bool wedged,
+void i915_handle_error(struct drm_device *dev, u32 engine_mask,
const char *fmt, ...);
extern void intel_irq_init(struct drm_i915_private *dev_priv);
@@ -2785,6 +2808,8 @@
enum forcewake_domains domains);
void intel_uncore_forcewake_put__locked(struct drm_i915_private *dev_priv,
enum forcewake_domains domains);
+u64 intel_uncore_edram_size(struct drm_i915_private *dev_priv);
+
void assert_forcewakes_inactive(struct drm_i915_private *dev_priv);
static inline bool intel_vgpu_active(struct drm_device *dev)
{
@@ -2863,7 +2888,6 @@
struct drm_file *file_priv);
void i915_gem_execbuffer_move_to_active(struct list_head *vmas,
struct drm_i915_gem_request *req);
-void i915_gem_execbuffer_retire_commands(struct i915_execbuffer_params *params);
int i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
struct drm_i915_gem_execbuffer2 *args,
struct list_head *vmas);
@@ -2894,6 +2918,7 @@
struct drm_file *file_priv);
void i915_gem_load_init(struct drm_device *dev);
void i915_gem_load_cleanup(struct drm_device *dev);
+void i915_gem_load_init_fences(struct drm_i915_private *dev_priv);
void *i915_gem_object_alloc(struct drm_device *dev);
void i915_gem_object_free(struct drm_i915_gem_object *obj);
void i915_gem_object_init(struct drm_i915_gem_object *obj,
@@ -2978,12 +3003,46 @@
BUG_ON(obj->pages == NULL);
obj->pages_pin_count++;
}
+
static inline void i915_gem_object_unpin_pages(struct drm_i915_gem_object *obj)
{
BUG_ON(obj->pages_pin_count == 0);
obj->pages_pin_count--;
}
+/**
+ * i915_gem_object_pin_map - return a contiguous mapping of the entire object
+ * @obj - the object to map into kernel address space
+ *
+ * Calls i915_gem_object_pin_pages() to prevent reaping of the object's
+ * pages and then returns a contiguous mapping of the backing storage into
+ * the kernel address space.
+ *
+ * The caller must hold the struct_mutex, and is responsible for calling
+ * i915_gem_object_unpin_map() when the mapping is no longer required.
+ *
+ * Returns the pointer through which to access the mapped object, or an
+ * ERR_PTR() on error.
+ */
+void *__must_check i915_gem_object_pin_map(struct drm_i915_gem_object *obj);
+
+/**
+ * i915_gem_object_unpin_map - releases an earlier mapping
+ * @obj - the object to unmap
+ *
+ * After pinning the object and mapping its pages, once you are finished
+ * with your access, call i915_gem_object_unpin_map() to release the pin
+ * upon the mapping. Once the pin count reaches zero, that mapping may be
+ * removed.
+ *
+ * The caller must hold the struct_mutex.
+ */
+static inline void i915_gem_object_unpin_map(struct drm_i915_gem_object *obj)
+{
+ lockdep_assert_held(&obj->base.dev->struct_mutex);
+ i915_gem_object_unpin_pages(obj);
+}
+
int __must_check i915_mutex_lock_interruptible(struct drm_device *dev);
int i915_gem_object_sync(struct drm_i915_gem_object *obj,
struct intel_engine_cs *to,
@@ -3007,42 +3066,68 @@
static inline bool i915_gem_request_started(struct drm_i915_gem_request *req,
bool lazy_coherency)
{
- u32 seqno = req->ring->get_seqno(req->ring, lazy_coherency);
- return i915_seqno_passed(seqno, req->previous_seqno);
+ if (!lazy_coherency && req->engine->irq_seqno_barrier)
+ req->engine->irq_seqno_barrier(req->engine);
+ return i915_seqno_passed(req->engine->get_seqno(req->engine),
+ req->previous_seqno);
}
static inline bool i915_gem_request_completed(struct drm_i915_gem_request *req,
bool lazy_coherency)
{
- u32 seqno = req->ring->get_seqno(req->ring, lazy_coherency);
- return i915_seqno_passed(seqno, req->seqno);
+ if (!lazy_coherency && req->engine->irq_seqno_barrier)
+ req->engine->irq_seqno_barrier(req->engine);
+ return i915_seqno_passed(req->engine->get_seqno(req->engine),
+ req->seqno);
}
int __must_check i915_gem_get_seqno(struct drm_device *dev, u32 *seqno);
int __must_check i915_gem_set_seqno(struct drm_device *dev, u32 seqno);
struct drm_i915_gem_request *
-i915_gem_find_active_request(struct intel_engine_cs *ring);
+i915_gem_find_active_request(struct intel_engine_cs *engine);
bool i915_gem_retire_requests(struct drm_device *dev);
-void i915_gem_retire_requests_ring(struct intel_engine_cs *ring);
-int __must_check i915_gem_check_wedge(struct i915_gpu_error *error,
- bool interruptible);
+void i915_gem_retire_requests_ring(struct intel_engine_cs *engine);
+
+static inline u32 i915_reset_counter(struct i915_gpu_error *error)
+{
+ return atomic_read(&error->reset_counter);
+}
+
+static inline bool __i915_reset_in_progress(u32 reset)
+{
+ return unlikely(reset & I915_RESET_IN_PROGRESS_FLAG);
+}
+
+static inline bool __i915_reset_in_progress_or_wedged(u32 reset)
+{
+ return unlikely(reset & (I915_RESET_IN_PROGRESS_FLAG | I915_WEDGED));
+}
+
+static inline bool __i915_terminally_wedged(u32 reset)
+{
+ return unlikely(reset & I915_WEDGED);
+}
static inline bool i915_reset_in_progress(struct i915_gpu_error *error)
{
- return unlikely(atomic_read(&error->reset_counter)
- & (I915_RESET_IN_PROGRESS_FLAG | I915_WEDGED));
+ return __i915_reset_in_progress(i915_reset_counter(error));
+}
+
+static inline bool i915_reset_in_progress_or_wedged(struct i915_gpu_error *error)
+{
+ return __i915_reset_in_progress_or_wedged(i915_reset_counter(error));
}
static inline bool i915_terminally_wedged(struct i915_gpu_error *error)
{
- return atomic_read(&error->reset_counter) & I915_WEDGED;
+ return __i915_terminally_wedged(i915_reset_counter(error));
}
static inline u32 i915_reset_count(struct i915_gpu_error *error)
{
- return ((atomic_read(&error->reset_counter) & ~I915_WEDGED) + 1) / 2;
+ return ((i915_reset_counter(error) & ~I915_WEDGED) + 1) / 2;
}
static inline bool i915_stop_ring_allow_ban(struct drm_i915_private *dev_priv)
@@ -3060,11 +3145,11 @@
void i915_gem_reset(struct drm_device *dev);
bool i915_gem_clflush_object(struct drm_i915_gem_object *obj, bool force);
int __must_check i915_gem_init(struct drm_device *dev);
-int i915_gem_init_rings(struct drm_device *dev);
+int i915_gem_init_engines(struct drm_device *dev);
int __must_check i915_gem_init_hw(struct drm_device *dev);
int i915_gem_l3_remap(struct drm_i915_gem_request *req, int slice);
void i915_gem_init_swizzling(struct drm_device *dev);
-void i915_gem_cleanup_ringbuffer(struct drm_device *dev);
+void i915_gem_cleanup_engines(struct drm_device *dev);
int __must_check i915_gpu_idle(struct drm_device *dev);
int __must_check i915_gem_suspend(struct drm_device *dev);
void __i915_add_request(struct drm_i915_gem_request *req,
@@ -3075,7 +3160,6 @@
#define i915_add_request_no_flush(req) \
__i915_add_request(req, NULL, false)
int __i915_wait_request(struct drm_i915_gem_request *req,
- unsigned reset_counter,
bool interruptible,
s64 *timeout,
struct intel_rps_client *rps);
@@ -3155,13 +3239,9 @@
bool i915_gem_obj_is_pinned(struct drm_i915_gem_object *obj);
/* Some GGTT VM helpers */
-#define i915_obj_to_ggtt(obj) \
- (&((struct drm_i915_private *)(obj)->base.dev->dev_private)->gtt.base)
-
static inline struct i915_hw_ppgtt *
i915_vm_to_ppgtt(struct i915_address_space *vm)
{
- WARN_ON(i915_is_ggtt(vm));
return container_of(vm, struct i915_hw_ppgtt, base);
}
@@ -3174,7 +3254,10 @@
static inline unsigned long
i915_gem_obj_ggtt_size(struct drm_i915_gem_object *obj)
{
- return i915_gem_obj_size(obj, i915_obj_to_ggtt(obj));
+ struct drm_i915_private *dev_priv = to_i915(obj->base.dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
+
+ return i915_gem_obj_size(obj, &ggtt->base);
}
static inline int __must_check
@@ -3182,7 +3265,10 @@
uint32_t alignment,
unsigned flags)
{
- return i915_gem_object_pin(obj, i915_obj_to_ggtt(obj),
+ struct drm_i915_private *dev_priv = to_i915(obj->base.dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
+
+ return i915_gem_object_pin(obj, &ggtt->base,
alignment, flags | PIN_GLOBAL);
}
@@ -3297,6 +3383,7 @@
#define I915_SHRINK_UNBOUND 0x2
#define I915_SHRINK_BOUND 0x4
#define I915_SHRINK_ACTIVE 0x8
+#define I915_SHRINK_VMAPS 0x10
unsigned long i915_gem_shrink_all(struct drm_i915_private *dev_priv);
void i915_gem_shrinker_init(struct drm_i915_private *dev_priv);
void i915_gem_shrinker_cleanup(struct drm_i915_private *dev_priv);
@@ -3343,7 +3430,7 @@
{
kfree(eb->buf);
}
-void i915_capture_error_state(struct drm_device *dev, bool wedge,
+void i915_capture_error_state(struct drm_device *dev, u32 engine_mask,
const char *error_msg);
void i915_error_state_get(struct drm_device *dev,
struct i915_error_state_file_priv *error_priv);
@@ -3355,10 +3442,10 @@
/* i915_cmd_parser.c */
int i915_cmd_parser_get_version(void);
-int i915_cmd_parser_init_ring(struct intel_engine_cs *ring);
-void i915_cmd_parser_fini_ring(struct intel_engine_cs *ring);
-bool i915_needs_cmd_parser(struct intel_engine_cs *ring);
-int i915_parse_cmds(struct intel_engine_cs *ring,
+int i915_cmd_parser_init_ring(struct intel_engine_cs *engine);
+void i915_cmd_parser_fini_ring(struct intel_engine_cs *engine);
+bool i915_needs_cmd_parser(struct intel_engine_cs *engine);
+int i915_parse_cmds(struct intel_engine_cs *engine,
struct drm_i915_gem_object *batch_obj,
struct drm_i915_gem_object *shadow_batch_obj,
u32 batch_start_offset,
@@ -3392,6 +3479,12 @@
/* intel_bios.c */
int intel_bios_init(struct drm_i915_private *dev_priv);
bool intel_bios_is_valid_vbt(const void *buf, size_t size);
+bool intel_bios_is_tv_present(struct drm_i915_private *dev_priv);
+bool intel_bios_is_lvds_present(struct drm_i915_private *dev_priv, u8 *i2c_pin);
+bool intel_bios_is_port_edp(struct drm_i915_private *dev_priv, enum port port);
+bool intel_bios_is_dsi_present(struct drm_i915_private *dev_priv, enum port *port);
+bool intel_bios_is_port_hpd_inverted(struct drm_i915_private *dev_priv,
+ enum port port);
/* intel_opregion.c */
#ifdef CONFIG_ACPI
@@ -3403,6 +3496,7 @@
bool enable);
extern int intel_opregion_notify_adapter(struct drm_device *dev,
pci_power_t state);
+extern int intel_opregion_get_panel_type(struct drm_device *dev);
#else
static inline int intel_opregion_setup(struct drm_device *dev) { return 0; }
static inline void intel_opregion_init(struct drm_device *dev) { return; }
@@ -3418,6 +3512,10 @@
{
return 0;
}
+static inline int intel_opregion_get_panel_type(struct drm_device *dev)
+{
+ return -ENODEV;
+}
#endif
/* intel_acpi.c */
@@ -3577,11 +3675,6 @@
return VGACNTRL;
}
-static inline void __user *to_user_ptr(u64 address)
-{
- return (void __user *)(uintptr_t)address;
-}
-
static inline unsigned long msecs_to_jiffies_timeout(const unsigned int m)
{
unsigned long j = msecs_to_jiffies(m);
@@ -3629,11 +3722,11 @@
}
}
-static inline void i915_trace_irq_get(struct intel_engine_cs *ring,
+static inline void i915_trace_irq_get(struct intel_engine_cs *engine,
struct drm_i915_gem_request *req)
{
- if (ring->trace_irq_req == NULL && ring->irq_get(ring))
- i915_gem_request_assign(&ring->trace_irq_req, req);
+ if (engine->trace_irq_req == NULL && engine->irq_get(engine))
+ i915_gem_request_assign(&engine->trace_irq_req, req);
}
#endif
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index f2cb9a9..9b99490 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -32,14 +32,13 @@
#include "i915_vgpu.h"
#include "i915_trace.h"
#include "intel_drv.h"
+#include "intel_mocs.h"
#include <linux/shmem_fs.h>
#include <linux/slab.h>
#include <linux/swap.h>
#include <linux/pci.h>
#include <linux/dma-buf.h>
-#define RQ_BUG_ON(expr)
-
static void i915_gem_object_flush_gtt_write_domain(struct drm_i915_gem_object *obj);
static void i915_gem_object_flush_cpu_write_domain(struct drm_i915_gem_object *obj);
static void
@@ -85,9 +84,7 @@
{
int ret;
-#define EXIT_COND (!i915_reset_in_progress(error) || \
- i915_terminally_wedged(error))
- if (EXIT_COND)
+ if (!i915_reset_in_progress(error))
return 0;
/*
@@ -96,17 +93,16 @@
* we should simply try to bail out and fail as gracefully as possible.
*/
ret = wait_event_interruptible_timeout(error->reset_queue,
- EXIT_COND,
+ !i915_reset_in_progress(error),
10*HZ);
if (ret == 0) {
DRM_ERROR("Timed out waiting for the gpu reset to complete\n");
return -EIO;
} else if (ret < 0) {
return ret;
+ } else {
+ return 0;
}
-#undef EXIT_COND
-
- return 0;
}
int i915_mutex_lock_interruptible(struct drm_device *dev)
@@ -130,9 +126,9 @@
i915_gem_get_aperture_ioctl(struct drm_device *dev, void *data,
struct drm_file *file)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct drm_i915_gem_get_aperture *args = data;
- struct i915_gtt *ggtt = &dev_priv->gtt;
struct i915_vma *vma;
size_t pinned;
@@ -146,7 +142,7 @@
pinned += vma->node.size;
mutex_unlock(&dev->struct_mutex);
- args->aper_size = dev_priv->gtt.base.total;
+ args->aper_size = ggtt->base.total;
args->aper_available_size = args->aper_size - pinned;
return 0;
@@ -211,11 +207,10 @@
BUG_ON(obj->madv == __I915_MADV_PURGED);
ret = i915_gem_object_set_to_cpu_domain(obj, true);
- if (ret) {
+ if (WARN_ON(ret)) {
/* In the event of a disaster, abandon all caches and
* hope for the best.
*/
- WARN_ON(ret != -EIO);
obj->base.read_domains = obj->base.write_domain = I915_GEM_DOMAIN_CPU;
}
@@ -324,7 +319,7 @@
{
struct drm_device *dev = obj->base.dev;
void *vaddr = obj->phys_handle->vaddr + args->offset;
- char __user *user_data = to_user_ptr(args->data_ptr);
+ char __user *user_data = u64_to_user_ptr(args->data_ptr);
int ret = 0;
/* We manually control the domain here and pretend that it
@@ -605,7 +600,7 @@
int needs_clflush = 0;
struct sg_page_iter sg_iter;
- user_data = to_user_ptr(args->data_ptr);
+ user_data = u64_to_user_ptr(args->data_ptr);
remain = args->size;
obj_do_bit17_swizzling = i915_gem_object_needs_bit17_swizzle(obj);
@@ -692,7 +687,7 @@
return 0;
if (!access_ok(VERIFY_WRITE,
- to_user_ptr(args->data_ptr),
+ u64_to_user_ptr(args->data_ptr),
args->size))
return -EFAULT;
@@ -700,7 +695,7 @@
if (ret)
return ret;
- obj = to_intel_bo(drm_gem_object_lookup(dev, file, args->handle));
+ obj = to_intel_bo(drm_gem_object_lookup(file, args->handle));
if (&obj->base == NULL) {
ret = -ENOENT;
goto unlock;
@@ -765,7 +760,8 @@
struct drm_i915_gem_pwrite *args,
struct drm_file *file)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
ssize_t remain;
loff_t offset, page_base;
char __user *user_data;
@@ -783,7 +779,7 @@
if (ret)
goto out_unpin;
- user_data = to_user_ptr(args->data_ptr);
+ user_data = u64_to_user_ptr(args->data_ptr);
remain = args->size;
offset = i915_gem_obj_ggtt_offset(obj) + args->offset;
@@ -807,7 +803,7 @@
* source page isn't available. Return the error and we'll
* retry in the slow path.
*/
- if (fast_user_write(dev_priv->gtt.mappable, page_base,
+ if (fast_user_write(ggtt->mappable, page_base,
page_offset, user_data, page_length)) {
ret = -EFAULT;
goto out_flush;
@@ -907,7 +903,7 @@
int needs_clflush_before = 0;
struct sg_page_iter sg_iter;
- user_data = to_user_ptr(args->data_ptr);
+ user_data = u64_to_user_ptr(args->data_ptr);
remain = args->size;
obj_do_bit17_swizzling = i915_gem_object_needs_bit17_swizzle(obj);
@@ -1036,12 +1032,12 @@
return 0;
if (!access_ok(VERIFY_READ,
- to_user_ptr(args->data_ptr),
+ u64_to_user_ptr(args->data_ptr),
args->size))
return -EFAULT;
if (likely(!i915.prefault_disable)) {
- ret = fault_in_multipages_readable(to_user_ptr(args->data_ptr),
+ ret = fault_in_multipages_readable(u64_to_user_ptr(args->data_ptr),
args->size);
if (ret)
return -EFAULT;
@@ -1053,7 +1049,7 @@
if (ret)
goto put_rpm;
- obj = to_intel_bo(drm_gem_object_lookup(dev, file, args->handle));
+ obj = to_intel_bo(drm_gem_object_lookup(file, args->handle));
if (&obj->base == NULL) {
ret = -ENOENT;
goto unlock;
@@ -1109,27 +1105,19 @@
return ret;
}
-int
-i915_gem_check_wedge(struct i915_gpu_error *error,
- bool interruptible)
+static int
+i915_gem_check_wedge(unsigned reset_counter, bool interruptible)
{
- if (i915_reset_in_progress(error)) {
+ if (__i915_terminally_wedged(reset_counter))
+ return -EIO;
+
+ if (__i915_reset_in_progress(reset_counter)) {
/* Non-interruptible callers can't handle -EAGAIN, hence return
* -EIO unconditionally for these. */
if (!interruptible)
return -EIO;
- /* Recovery complete, but the reset failed ... */
- if (i915_terminally_wedged(error))
- return -EIO;
-
- /*
- * Check if GPU Reset is in progress - we need intel_ring_begin
- * to work properly to reinit the hw state while the gpu is
- * still marked as reset-in-progress. Handle this with a flag.
- */
- if (!error->reload_in_reset)
- return -EAGAIN;
+ return -EAGAIN;
}
return 0;
@@ -1141,9 +1129,9 @@
}
static bool missed_irq(struct drm_i915_private *dev_priv,
- struct intel_engine_cs *ring)
+ struct intel_engine_cs *engine)
{
- return test_bit(ring->id, &dev_priv->gpu_error.missed_irq_rings);
+ return test_bit(engine->id, &dev_priv->gpu_error.missed_irq_rings);
}
static unsigned long local_clock_us(unsigned *cpu)
@@ -1193,7 +1181,7 @@
* takes to sleep on a request, on the order of a microsecond.
*/
- if (req->ring->irq_refcount)
+ if (req->engine->irq_refcount)
return -EBUSY;
/* Only spin if we know the GPU is processing this request */
@@ -1223,7 +1211,6 @@
/**
* __i915_wait_request - wait until execution of request has finished
* @req: duh!
- * @reset_counter: reset sequence associated with the given request
* @interruptible: do an interruptible wait (normally yes)
* @timeout: in - how long to wait (NULL forever); out - how much time remaining
*
@@ -1238,16 +1225,15 @@
* errno with remaining time filled in timeout argument.
*/
int __i915_wait_request(struct drm_i915_gem_request *req,
- unsigned reset_counter,
bool interruptible,
s64 *timeout,
struct intel_rps_client *rps)
{
- struct intel_engine_cs *ring = i915_gem_request_get_ring(req);
- struct drm_device *dev = ring->dev;
+ struct intel_engine_cs *engine = i915_gem_request_get_engine(req);
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
const bool irq_test_in_progress =
- ACCESS_ONCE(dev_priv->gpu_error.test_irq_rings) & intel_ring_flag(ring);
+ ACCESS_ONCE(dev_priv->gpu_error.test_irq_rings) & intel_engine_flag(engine);
int state = interruptible ? TASK_INTERRUPTIBLE : TASK_UNINTERRUPTIBLE;
DEFINE_WAIT(wait);
unsigned long timeout_expire;
@@ -1288,7 +1274,7 @@
if (ret == 0)
goto out;
- if (!irq_test_in_progress && WARN_ON(!ring->irq_get(ring))) {
+ if (!irq_test_in_progress && WARN_ON(!engine->irq_get(engine))) {
ret = -ENODEV;
goto out;
}
@@ -1296,16 +1282,17 @@
for (;;) {
struct timer_list timer;
- prepare_to_wait(&ring->irq_queue, &wait, state);
+ prepare_to_wait(&engine->irq_queue, &wait, state);
/* We need to check whether any gpu reset happened in between
- * the caller grabbing the seqno and now ... */
- if (reset_counter != atomic_read(&dev_priv->gpu_error.reset_counter)) {
- /* ... but upgrade the -EAGAIN to an -EIO if the gpu
- * is truely gone. */
- ret = i915_gem_check_wedge(&dev_priv->gpu_error, interruptible);
- if (ret == 0)
- ret = -EAGAIN;
+ * the request being submitted and now. If a reset has occurred,
+ * the request is effectively complete (we either are in the
+ * process of or have discarded the rendering and completely
+ * reset the GPU. The results of the request are lost and we
+ * are free to continue on with the original operation.
+ */
+ if (req->reset_counter != i915_reset_counter(&dev_priv->gpu_error)) {
+ ret = 0;
break;
}
@@ -1325,11 +1312,11 @@
}
timer.function = NULL;
- if (timeout || missed_irq(dev_priv, ring)) {
+ if (timeout || missed_irq(dev_priv, engine)) {
unsigned long expire;
setup_timer_on_stack(&timer, fake_irq, (unsigned long)current);
- expire = missed_irq(dev_priv, ring) ? jiffies + 1 : timeout_expire;
+ expire = missed_irq(dev_priv, engine) ? jiffies + 1 : timeout_expire;
mod_timer(&timer, expire);
}
@@ -1341,9 +1328,9 @@
}
}
if (!irq_test_in_progress)
- ring->irq_put(ring);
+ engine->irq_put(engine);
- finish_wait(&ring->irq_queue, &wait);
+ finish_wait(&engine->irq_queue, &wait);
out:
trace_i915_gem_request_wait_end(req);
@@ -1370,7 +1357,6 @@
int i915_gem_request_add_to_client(struct drm_i915_gem_request *req,
struct drm_file *file)
{
- struct drm_i915_private *dev_private;
struct drm_i915_file_private *file_priv;
WARN_ON(!req || !file || req->file_priv);
@@ -1381,7 +1367,6 @@
if (req->file_priv)
return -EINVAL;
- dev_private = req->ring->dev->dev_private;
file_priv = file->driver_priv;
spin_lock(&file_priv->mm.lock);
@@ -1434,7 +1419,7 @@
static void
__i915_gem_request_retire__upto(struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *engine = req->ring;
+ struct intel_engine_cs *engine = req->engine;
struct drm_i915_gem_request *tmp;
lockdep_assert_held(&engine->dev->struct_mutex);
@@ -1459,26 +1444,15 @@
int
i915_wait_request(struct drm_i915_gem_request *req)
{
- struct drm_device *dev;
- struct drm_i915_private *dev_priv;
+ struct drm_i915_private *dev_priv = req->i915;
bool interruptible;
int ret;
- BUG_ON(req == NULL);
-
- dev = req->ring->dev;
- dev_priv = dev->dev_private;
interruptible = dev_priv->mm.interruptible;
- BUG_ON(!mutex_is_locked(&dev->struct_mutex));
+ BUG_ON(!mutex_is_locked(&dev_priv->dev->struct_mutex));
- ret = i915_gem_check_wedge(&dev_priv->gpu_error, interruptible);
- if (ret)
- return ret;
-
- ret = __i915_wait_request(req,
- atomic_read(&dev_priv->gpu_error.reset_counter),
- interruptible, NULL, NULL);
+ ret = __i915_wait_request(req, interruptible, NULL, NULL);
if (ret)
return ret;
@@ -1505,14 +1479,14 @@
if (ret)
return ret;
- i = obj->last_write_req->ring->id;
+ i = obj->last_write_req->engine->id;
if (obj->last_read_req[i] == obj->last_write_req)
i915_gem_object_retire__read(obj, i);
else
i915_gem_object_retire__write(obj);
}
} else {
- for (i = 0; i < I915_NUM_RINGS; i++) {
+ for (i = 0; i < I915_NUM_ENGINES; i++) {
if (obj->last_read_req[i] == NULL)
continue;
@@ -1522,7 +1496,7 @@
i915_gem_object_retire__read(obj, i);
}
- RQ_BUG_ON(obj->active);
+ GEM_BUG_ON(obj->active);
}
return 0;
@@ -1532,7 +1506,7 @@
i915_gem_object_retire_request(struct drm_i915_gem_object *obj,
struct drm_i915_gem_request *req)
{
- int ring = req->ring->id;
+ int ring = req->engine->id;
if (obj->last_read_req[ring] == req)
i915_gem_object_retire__read(obj, ring);
@@ -1552,8 +1526,7 @@
{
struct drm_device *dev = obj->base.dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- struct drm_i915_gem_request *requests[I915_NUM_RINGS];
- unsigned reset_counter;
+ struct drm_i915_gem_request *requests[I915_NUM_ENGINES];
int ret, i, n = 0;
BUG_ON(!mutex_is_locked(&dev->struct_mutex));
@@ -1562,12 +1535,6 @@
if (!obj->active)
return 0;
- ret = i915_gem_check_wedge(&dev_priv->gpu_error, true);
- if (ret)
- return ret;
-
- reset_counter = atomic_read(&dev_priv->gpu_error.reset_counter);
-
if (readonly) {
struct drm_i915_gem_request *req;
@@ -1577,7 +1544,7 @@
requests[n++] = i915_gem_request_reference(req);
} else {
- for (i = 0; i < I915_NUM_RINGS; i++) {
+ for (i = 0; i < I915_NUM_ENGINES; i++) {
struct drm_i915_gem_request *req;
req = obj->last_read_req[i];
@@ -1589,9 +1556,9 @@
}
mutex_unlock(&dev->struct_mutex);
+ ret = 0;
for (i = 0; ret == 0 && i < n; i++)
- ret = __i915_wait_request(requests[i], reset_counter, true,
- NULL, rps);
+ ret = __i915_wait_request(requests[i], true, NULL, rps);
mutex_lock(&dev->struct_mutex);
for (i = 0; i < n; i++) {
@@ -1640,7 +1607,7 @@
if (ret)
return ret;
- obj = to_intel_bo(drm_gem_object_lookup(dev, file, args->handle));
+ obj = to_intel_bo(drm_gem_object_lookup(file, args->handle));
if (&obj->base == NULL) {
ret = -ENOENT;
goto unlock;
@@ -1688,7 +1655,7 @@
if (ret)
return ret;
- obj = to_intel_bo(drm_gem_object_lookup(dev, file, args->handle));
+ obj = to_intel_bo(drm_gem_object_lookup(file, args->handle));
if (&obj->base == NULL) {
ret = -ENOENT;
goto unlock;
@@ -1735,7 +1702,7 @@
if (args->flags & I915_MMAP_WC && !boot_cpu_has(X86_FEATURE_PAT))
return -ENODEV;
- obj = drm_gem_object_lookup(dev, file, args->handle);
+ obj = drm_gem_object_lookup(file, args->handle);
if (obj == NULL)
return -ENOENT;
@@ -1754,7 +1721,10 @@
struct mm_struct *mm = current->mm;
struct vm_area_struct *vma;
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem)) {
+ drm_gem_object_unreference_unlocked(obj);
+ return -EINTR;
+ }
vma = find_vma(mm, addr);
if (vma)
vma->vm_page_prot =
@@ -1792,7 +1762,8 @@
{
struct drm_i915_gem_object *obj = to_intel_bo(vma->vm_private_data);
struct drm_device *dev = obj->base.dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct i915_ggtt_view view = i915_ggtt_view_normal;
pgoff_t page_offset;
unsigned long pfn;
@@ -1827,7 +1798,7 @@
}
/* Use a partial view if the object is bigger than the aperture. */
- if (obj->base.size >= dev_priv->gtt.mappable_end &&
+ if (obj->base.size >= ggtt->mappable_end &&
obj->tiling_mode == I915_TILING_NONE) {
static const unsigned int chunk_size = 256; // 1 MiB
@@ -1855,7 +1826,7 @@
goto unpin;
/* Finally, remap it using the new GTT offset */
- pfn = dev_priv->gtt.mappable_base +
+ pfn = ggtt->mappable_base +
i915_gem_obj_ggtt_offset_view(obj, &view);
pfn >>= PAGE_SHIFT;
@@ -1964,11 +1935,27 @@
void
i915_gem_release_mmap(struct drm_i915_gem_object *obj)
{
+ /* Serialisation between user GTT access and our code depends upon
+ * revoking the CPU's PTE whilst the mutex is held. The next user
+ * pagefault then has to wait until we release the mutex.
+ */
+ lockdep_assert_held(&obj->base.dev->struct_mutex);
+
if (!obj->fault_mappable)
return;
drm_vma_node_unmap(&obj->base.vma_node,
obj->base.dev->anon_inode->i_mapping);
+
+ /* Ensure that the CPU's PTE are revoked and there are not outstanding
+ * memory transactions from userspace before we return. The TLB
+ * flushing implied above by changing the PTE above *should* be
+ * sufficient, an extra barrier here just provides us with a bit
+ * of paranoid documentation about our requirement to serialise
+ * memory writes before touching registers / GSM.
+ */
+ wmb();
+
obj->fault_mappable = false;
}
@@ -2033,9 +2020,6 @@
struct drm_i915_private *dev_priv = obj->base.dev->dev_private;
int ret;
- if (drm_vma_node_has_offset(&obj->base.vma_node))
- return 0;
-
dev_priv->mm.shrinker_no_lock_stealing = true;
ret = drm_gem_create_mmap_offset(&obj->base);
@@ -2084,7 +2068,7 @@
if (ret)
return ret;
- obj = to_intel_bo(drm_gem_object_lookup(dev, file, handle));
+ obj = to_intel_bo(drm_gem_object_lookup(file, handle));
if (&obj->base == NULL) {
ret = -ENOENT;
goto unlock;
@@ -2180,11 +2164,10 @@
BUG_ON(obj->madv == __I915_MADV_PURGED);
ret = i915_gem_object_set_to_cpu_domain(obj, true);
- if (ret) {
+ if (WARN_ON(ret)) {
/* In the event of a disaster, abandon all caches and
* hope for the best.
*/
- WARN_ON(ret != -EIO);
i915_gem_clflush_object(obj, true);
obj->base.read_domains = obj->base.write_domain = I915_GEM_DOMAIN_CPU;
}
@@ -2232,6 +2215,14 @@
* lists early. */
list_del(&obj->global_list);
+ if (obj->mapping) {
+ if (is_vmalloc_addr(obj->mapping))
+ vunmap(obj->mapping);
+ else
+ kunmap(kmap_to_page(obj->mapping));
+ obj->mapping = NULL;
+ }
+
ops->put_pages(obj);
obj->pages = NULL;
@@ -2400,21 +2391,64 @@
return 0;
}
+void *i915_gem_object_pin_map(struct drm_i915_gem_object *obj)
+{
+ int ret;
+
+ lockdep_assert_held(&obj->base.dev->struct_mutex);
+
+ ret = i915_gem_object_get_pages(obj);
+ if (ret)
+ return ERR_PTR(ret);
+
+ i915_gem_object_pin_pages(obj);
+
+ if (obj->mapping == NULL) {
+ struct page **pages;
+
+ pages = NULL;
+ if (obj->base.size == PAGE_SIZE)
+ obj->mapping = kmap(sg_page(obj->pages->sgl));
+ else
+ pages = drm_malloc_gfp(obj->base.size >> PAGE_SHIFT,
+ sizeof(*pages),
+ GFP_TEMPORARY);
+ if (pages != NULL) {
+ struct sg_page_iter sg_iter;
+ int n;
+
+ n = 0;
+ for_each_sg_page(obj->pages->sgl, &sg_iter,
+ obj->pages->nents, 0)
+ pages[n++] = sg_page_iter_page(&sg_iter);
+
+ obj->mapping = vmap(pages, n, 0, PAGE_KERNEL);
+ drm_free_large(pages);
+ }
+ if (obj->mapping == NULL) {
+ i915_gem_object_unpin_pages(obj);
+ return ERR_PTR(-ENOMEM);
+ }
+ }
+
+ return obj->mapping;
+}
+
void i915_vma_move_to_active(struct i915_vma *vma,
struct drm_i915_gem_request *req)
{
struct drm_i915_gem_object *obj = vma->obj;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
- ring = i915_gem_request_get_ring(req);
+ engine = i915_gem_request_get_engine(req);
/* Add a reference if we're newly entering the active list. */
if (obj->active == 0)
drm_gem_object_reference(&obj->base);
- obj->active |= intel_ring_flag(ring);
+ obj->active |= intel_engine_flag(engine);
- list_move_tail(&obj->ring_list[ring->id], &ring->active_list);
- i915_gem_request_assign(&obj->last_read_req[ring->id], req);
+ list_move_tail(&obj->engine_list[engine->id], &engine->active_list);
+ i915_gem_request_assign(&obj->last_read_req[engine->id], req);
list_move_tail(&vma->vm_link, &vma->vm->active_list);
}
@@ -2422,8 +2456,8 @@
static void
i915_gem_object_retire__write(struct drm_i915_gem_object *obj)
{
- RQ_BUG_ON(obj->last_write_req == NULL);
- RQ_BUG_ON(!(obj->active & intel_ring_flag(obj->last_write_req->ring)));
+ GEM_BUG_ON(obj->last_write_req == NULL);
+ GEM_BUG_ON(!(obj->active & intel_engine_flag(obj->last_write_req->engine)));
i915_gem_request_assign(&obj->last_write_req, NULL);
intel_fb_obj_flush(obj, true, ORIGIN_CS);
@@ -2434,13 +2468,13 @@
{
struct i915_vma *vma;
- RQ_BUG_ON(obj->last_read_req[ring] == NULL);
- RQ_BUG_ON(!(obj->active & (1 << ring)));
+ GEM_BUG_ON(obj->last_read_req[ring] == NULL);
+ GEM_BUG_ON(!(obj->active & (1 << ring)));
- list_del_init(&obj->ring_list[ring]);
+ list_del_init(&obj->engine_list[ring]);
i915_gem_request_assign(&obj->last_read_req[ring], NULL);
- if (obj->last_write_req && obj->last_write_req->ring->id == ring)
+ if (obj->last_write_req && obj->last_write_req->engine->id == ring)
i915_gem_object_retire__write(obj);
obj->active &= ~(1 << ring);
@@ -2467,24 +2501,20 @@
i915_gem_init_seqno(struct drm_device *dev, u32 seqno)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
- int ret, i, j;
+ struct intel_engine_cs *engine;
+ int ret;
/* Carefully retire all requests without writing to the rings */
- for_each_ring(ring, dev_priv, i) {
- ret = intel_ring_idle(ring);
+ for_each_engine(engine, dev_priv) {
+ ret = intel_engine_idle(engine);
if (ret)
return ret;
}
i915_gem_retire_requests(dev);
/* Finally reset hw state */
- for_each_ring(ring, dev_priv, i) {
- intel_ring_init_seqno(ring, seqno);
-
- for (j = 0; j < ARRAY_SIZE(ring->semaphore.sync_seqno); j++)
- ring->semaphore.sync_seqno[j] = 0;
- }
+ for_each_engine(engine, dev_priv)
+ intel_ring_init_seqno(engine, seqno);
return 0;
}
@@ -2542,7 +2572,7 @@
struct drm_i915_gem_object *obj,
bool flush_caches)
{
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
struct drm_i915_private *dev_priv;
struct intel_ringbuffer *ringbuf;
u32 request_start;
@@ -2551,8 +2581,8 @@
if (WARN_ON(request == NULL))
return;
- ring = request->ring;
- dev_priv = ring->dev->dev_private;
+ engine = request->engine;
+ dev_priv = request->i915;
ringbuf = request->ringbuf;
/*
@@ -2579,22 +2609,7 @@
WARN(ret, "*_ring_flush_all_caches failed: %d!\n", ret);
}
- /* Record the position of the start of the request so that
- * should we detect the updated seqno part-way through the
- * GPU processing the request, we never over-estimate the
- * position of the head.
- */
- request->postfix = intel_ring_get_tail(ringbuf);
-
- if (i915.enable_execlists)
- ret = ring->emit_request(request);
- else {
- ret = ring->add_request(request);
-
- request->tail = intel_ring_get_tail(ringbuf);
- }
- /* Not allowed to fail! */
- WARN(ret, "emit|add_request failed: %d!\n", ret);
+ trace_i915_gem_request_add(request);
request->head = request_start;
@@ -2606,14 +2621,34 @@
*/
request->batch_obj = obj;
+ /* Seal the request and mark it as pending execution. Note that
+ * we may inspect this state, without holding any locks, during
+ * hangcheck. Hence we apply the barrier to ensure that we do not
+ * see a more recent value in the hws than we are tracking.
+ */
request->emitted_jiffies = jiffies;
- request->previous_seqno = ring->last_submitted_seqno;
- ring->last_submitted_seqno = request->seqno;
- list_add_tail(&request->list, &ring->request_list);
+ request->previous_seqno = engine->last_submitted_seqno;
+ smp_store_mb(engine->last_submitted_seqno, request->seqno);
+ list_add_tail(&request->list, &engine->request_list);
- trace_i915_gem_request_add(request);
+ /* Record the position of the start of the request so that
+ * should we detect the updated seqno part-way through the
+ * GPU processing the request, we never over-estimate the
+ * position of the head.
+ */
+ request->postfix = intel_ring_get_tail(ringbuf);
- i915_queue_hangcheck(ring->dev);
+ if (i915.enable_execlists)
+ ret = engine->emit_request(request);
+ else {
+ ret = engine->add_request(request);
+
+ request->tail = intel_ring_get_tail(ringbuf);
+ }
+ /* Not allowed to fail! */
+ WARN(ret, "emit|add_request failed: %d!\n", ret);
+
+ i915_queue_hangcheck(engine->dev);
queue_delayed_work(dev_priv->wq,
&dev_priv->mm.retire_work,
@@ -2680,7 +2715,7 @@
if (ctx) {
if (i915.enable_execlists && ctx != req->i915->kernel_context)
- intel_lr_context_unpin(ctx, req->ring);
+ intel_lr_context_unpin(ctx, req->engine);
i915_gem_context_unreference(ctx);
}
@@ -2689,11 +2724,12 @@
}
static inline int
-__i915_gem_request_alloc(struct intel_engine_cs *ring,
+__i915_gem_request_alloc(struct intel_engine_cs *engine,
struct intel_context *ctx,
struct drm_i915_gem_request **req_out)
{
- struct drm_i915_private *dev_priv = to_i915(ring->dev);
+ struct drm_i915_private *dev_priv = to_i915(engine->dev);
+ unsigned reset_counter = i915_reset_counter(&dev_priv->gpu_error);
struct drm_i915_gem_request *req;
int ret;
@@ -2702,17 +2738,26 @@
*req_out = NULL;
+ /* ABI: Before userspace accesses the GPU (e.g. execbuffer), report
+ * EIO if the GPU is already wedged, or EAGAIN to drop the struct_mutex
+ * and restart.
+ */
+ ret = i915_gem_check_wedge(reset_counter, dev_priv->mm.interruptible);
+ if (ret)
+ return ret;
+
req = kmem_cache_zalloc(dev_priv->requests, GFP_KERNEL);
if (req == NULL)
return -ENOMEM;
- ret = i915_gem_get_seqno(ring->dev, &req->seqno);
+ ret = i915_gem_get_seqno(engine->dev, &req->seqno);
if (ret)
goto err;
kref_init(&req->ref);
req->i915 = dev_priv;
- req->ring = ring;
+ req->engine = engine;
+ req->reset_counter = reset_counter;
req->ctx = ctx;
i915_gem_context_reference(req->ctx);
@@ -2742,7 +2787,8 @@
* fully prepared. Thus it can be cleaned up using the proper
* free code.
*/
- i915_gem_request_cancel(req);
+ intel_ring_reserved_space_cancel(req->ringbuf);
+ i915_gem_request_unreference(req);
return ret;
}
@@ -2779,19 +2825,12 @@
return err ? ERR_PTR(err) : req;
}
-void i915_gem_request_cancel(struct drm_i915_gem_request *req)
-{
- intel_ring_reserved_space_cancel(req->ringbuf);
-
- i915_gem_request_unreference(req);
-}
-
struct drm_i915_gem_request *
-i915_gem_find_active_request(struct intel_engine_cs *ring)
+i915_gem_find_active_request(struct intel_engine_cs *engine)
{
struct drm_i915_gem_request *request;
- list_for_each_entry(request, &ring->request_list, list) {
+ list_for_each_entry(request, &engine->request_list, list) {
if (i915_gem_request_completed(request, false))
continue;
@@ -2801,38 +2840,38 @@
return NULL;
}
-static void i915_gem_reset_ring_status(struct drm_i915_private *dev_priv,
- struct intel_engine_cs *ring)
+static void i915_gem_reset_engine_status(struct drm_i915_private *dev_priv,
+ struct intel_engine_cs *engine)
{
struct drm_i915_gem_request *request;
bool ring_hung;
- request = i915_gem_find_active_request(ring);
+ request = i915_gem_find_active_request(engine);
if (request == NULL)
return;
- ring_hung = ring->hangcheck.score >= HANGCHECK_SCORE_RING_HUNG;
+ ring_hung = engine->hangcheck.score >= HANGCHECK_SCORE_RING_HUNG;
i915_set_reset_status(dev_priv, request->ctx, ring_hung);
- list_for_each_entry_continue(request, &ring->request_list, list)
+ list_for_each_entry_continue(request, &engine->request_list, list)
i915_set_reset_status(dev_priv, request->ctx, false);
}
-static void i915_gem_reset_ring_cleanup(struct drm_i915_private *dev_priv,
- struct intel_engine_cs *ring)
+static void i915_gem_reset_engine_cleanup(struct drm_i915_private *dev_priv,
+ struct intel_engine_cs *engine)
{
struct intel_ringbuffer *buffer;
- while (!list_empty(&ring->active_list)) {
+ while (!list_empty(&engine->active_list)) {
struct drm_i915_gem_object *obj;
- obj = list_first_entry(&ring->active_list,
+ obj = list_first_entry(&engine->active_list,
struct drm_i915_gem_object,
- ring_list[ring->id]);
+ engine_list[engine->id]);
- i915_gem_object_retire__read(obj, ring->id);
+ i915_gem_object_retire__read(obj, engine->id);
}
/*
@@ -2842,14 +2881,16 @@
*/
if (i915.enable_execlists) {
- spin_lock_irq(&ring->execlist_lock);
+ /* Ensure irq handler finishes or is cancelled. */
+ tasklet_kill(&engine->irq_tasklet);
+ spin_lock_bh(&engine->execlist_lock);
/* list_splice_tail_init checks for empty lists */
- list_splice_tail_init(&ring->execlist_queue,
- &ring->execlist_retired_req_list);
+ list_splice_tail_init(&engine->execlist_queue,
+ &engine->execlist_retired_req_list);
+ spin_unlock_bh(&engine->execlist_lock);
- spin_unlock_irq(&ring->execlist_lock);
- intel_execlists_retire_requests(ring);
+ intel_execlists_retire_requests(engine);
}
/*
@@ -2859,10 +2900,10 @@
* implicit references on things like e.g. ppgtt address spaces through
* the request.
*/
- while (!list_empty(&ring->request_list)) {
+ while (!list_empty(&engine->request_list)) {
struct drm_i915_gem_request *request;
- request = list_first_entry(&ring->request_list,
+ request = list_first_entry(&engine->request_list,
struct drm_i915_gem_request,
list);
@@ -2876,28 +2917,29 @@
* upon reset is less than when we start. Do one more pass over
* all the ringbuffers to reset last_retired_head.
*/
- list_for_each_entry(buffer, &ring->buffers, link) {
+ list_for_each_entry(buffer, &engine->buffers, link) {
buffer->last_retired_head = buffer->tail;
intel_ring_update_space(buffer);
}
+
+ intel_ring_init_seqno(engine, engine->last_submitted_seqno);
}
void i915_gem_reset(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
- int i;
+ struct intel_engine_cs *engine;
/*
* Before we free the objects from the requests, we need to inspect
* them for finding the guilty party. As the requests only borrow
* their reference to the objects, the inspection must be done first.
*/
- for_each_ring(ring, dev_priv, i)
- i915_gem_reset_ring_status(dev_priv, ring);
+ for_each_engine(engine, dev_priv)
+ i915_gem_reset_engine_status(dev_priv, engine);
- for_each_ring(ring, dev_priv, i)
- i915_gem_reset_ring_cleanup(dev_priv, ring);
+ for_each_engine(engine, dev_priv)
+ i915_gem_reset_engine_cleanup(dev_priv, engine);
i915_gem_context_reset(dev);
@@ -2910,19 +2952,19 @@
* This function clears the request list as sequence numbers are passed.
*/
void
-i915_gem_retire_requests_ring(struct intel_engine_cs *ring)
+i915_gem_retire_requests_ring(struct intel_engine_cs *engine)
{
- WARN_ON(i915_verify_lists(ring->dev));
+ WARN_ON(i915_verify_lists(engine->dev));
/* Retire requests first as we use it above for the early return.
* If we retire requests last, we may use a later seqno and so clear
* the requests lists without clearing the active list, leading to
* confusion.
*/
- while (!list_empty(&ring->request_list)) {
+ while (!list_empty(&engine->request_list)) {
struct drm_i915_gem_request *request;
- request = list_first_entry(&ring->request_list,
+ request = list_first_entry(&engine->request_list,
struct drm_i915_gem_request,
list);
@@ -2936,45 +2978,44 @@
* by the ringbuffer to the flushing/inactive lists as appropriate,
* before we free the context associated with the requests.
*/
- while (!list_empty(&ring->active_list)) {
+ while (!list_empty(&engine->active_list)) {
struct drm_i915_gem_object *obj;
- obj = list_first_entry(&ring->active_list,
- struct drm_i915_gem_object,
- ring_list[ring->id]);
+ obj = list_first_entry(&engine->active_list,
+ struct drm_i915_gem_object,
+ engine_list[engine->id]);
- if (!list_empty(&obj->last_read_req[ring->id]->list))
+ if (!list_empty(&obj->last_read_req[engine->id]->list))
break;
- i915_gem_object_retire__read(obj, ring->id);
+ i915_gem_object_retire__read(obj, engine->id);
}
- if (unlikely(ring->trace_irq_req &&
- i915_gem_request_completed(ring->trace_irq_req, true))) {
- ring->irq_put(ring);
- i915_gem_request_assign(&ring->trace_irq_req, NULL);
+ if (unlikely(engine->trace_irq_req &&
+ i915_gem_request_completed(engine->trace_irq_req, true))) {
+ engine->irq_put(engine);
+ i915_gem_request_assign(&engine->trace_irq_req, NULL);
}
- WARN_ON(i915_verify_lists(ring->dev));
+ WARN_ON(i915_verify_lists(engine->dev));
}
bool
i915_gem_retire_requests(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
bool idle = true;
- int i;
- for_each_ring(ring, dev_priv, i) {
- i915_gem_retire_requests_ring(ring);
- idle &= list_empty(&ring->request_list);
+ for_each_engine(engine, dev_priv) {
+ i915_gem_retire_requests_ring(engine);
+ idle &= list_empty(&engine->request_list);
if (i915.enable_execlists) {
- spin_lock_irq(&ring->execlist_lock);
- idle &= list_empty(&ring->execlist_queue);
- spin_unlock_irq(&ring->execlist_lock);
+ spin_lock_bh(&engine->execlist_lock);
+ idle &= list_empty(&engine->execlist_queue);
+ spin_unlock_bh(&engine->execlist_lock);
- intel_execlists_retire_requests(ring);
+ intel_execlists_retire_requests(engine);
}
}
@@ -3011,25 +3052,21 @@
struct drm_i915_private *dev_priv =
container_of(work, typeof(*dev_priv), mm.idle_work.work);
struct drm_device *dev = dev_priv->dev;
- struct intel_engine_cs *ring;
- int i;
+ struct intel_engine_cs *engine;
- for_each_ring(ring, dev_priv, i)
- if (!list_empty(&ring->request_list))
+ for_each_engine(engine, dev_priv)
+ if (!list_empty(&engine->request_list))
return;
/* we probably should sync with hangcheck here, using cancel_work_sync.
- * Also locking seems to be fubar here, ring->request_list is protected
+ * Also locking seems to be fubar here, engine->request_list is protected
* by dev->struct_mutex. */
intel_mark_idle(dev);
if (mutex_trylock(&dev->struct_mutex)) {
- struct intel_engine_cs *ring;
- int i;
-
- for_each_ring(ring, dev_priv, i)
- i915_gem_batch_pool_fini(&ring->batch_pool);
+ for_each_engine(engine, dev_priv)
+ i915_gem_batch_pool_fini(&engine->batch_pool);
mutex_unlock(&dev->struct_mutex);
}
@@ -3048,7 +3085,7 @@
if (!obj->active)
return 0;
- for (i = 0; i < I915_NUM_RINGS; i++) {
+ for (i = 0; i < I915_NUM_ENGINES; i++) {
struct drm_i915_gem_request *req;
req = obj->last_read_req[i];
@@ -3093,11 +3130,9 @@
int
i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
struct drm_i915_gem_wait *args = data;
struct drm_i915_gem_object *obj;
- struct drm_i915_gem_request *req[I915_NUM_RINGS];
- unsigned reset_counter;
+ struct drm_i915_gem_request *req[I915_NUM_ENGINES];
int i, n = 0;
int ret;
@@ -3108,7 +3143,7 @@
if (ret)
return ret;
- obj = to_intel_bo(drm_gem_object_lookup(dev, file, args->bo_handle));
+ obj = to_intel_bo(drm_gem_object_lookup(file, args->bo_handle));
if (&obj->base == NULL) {
mutex_unlock(&dev->struct_mutex);
return -ENOENT;
@@ -3131,9 +3166,8 @@
}
drm_gem_object_unreference(&obj->base);
- reset_counter = atomic_read(&dev_priv->gpu_error.reset_counter);
- for (i = 0; i < I915_NUM_RINGS; i++) {
+ for (i = 0; i < I915_NUM_ENGINES; i++) {
if (obj->last_read_req[i] == NULL)
continue;
@@ -3144,7 +3178,7 @@
for (i = 0; i < n; i++) {
if (ret == 0)
- ret = __i915_wait_request(req[i], reset_counter, true,
+ ret = __i915_wait_request(req[i], true,
args->timeout_ns > 0 ? &args->timeout_ns : NULL,
to_rps_client(file));
i915_gem_request_unreference__unlocked(req[i]);
@@ -3166,7 +3200,7 @@
struct intel_engine_cs *from;
int ret;
- from = i915_gem_request_get_ring(from_req);
+ from = i915_gem_request_get_engine(from_req);
if (to == from)
return 0;
@@ -3176,7 +3210,6 @@
if (!i915_semaphore_is_enabled(obj->base.dev)) {
struct drm_i915_private *i915 = to_i915(obj->base.dev);
ret = __i915_wait_request(from_req,
- atomic_read(&i915->gpu_error.reset_counter),
i915->mm.interruptible,
NULL,
&i915->rps.semaphores);
@@ -3260,7 +3293,7 @@
struct drm_i915_gem_request **to_req)
{
const bool readonly = obj->base.pending_write_domain == 0;
- struct drm_i915_gem_request *req[I915_NUM_RINGS];
+ struct drm_i915_gem_request *req[I915_NUM_ENGINES];
int ret, i, n;
if (!obj->active)
@@ -3274,7 +3307,7 @@
if (obj->last_write_req)
req[n++] = obj->last_write_req;
} else {
- for (i = 0; i < I915_NUM_RINGS; i++)
+ for (i = 0; i < I915_NUM_ENGINES; i++)
if (obj->last_read_req[i])
req[n++] = obj->last_read_req[i];
}
@@ -3297,9 +3330,6 @@
if ((obj->base.read_domains & I915_GEM_DOMAIN_GTT) == 0)
return;
- /* Wait for any direct GTT access to complete */
- mb();
-
old_read_domains = obj->base.read_domains;
old_write_domain = obj->base.write_domain;
@@ -3391,28 +3421,25 @@
int i915_gpu_idle(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
- int ret, i;
+ struct intel_engine_cs *engine;
+ int ret;
/* Flush everything onto the inactive list. */
- for_each_ring(ring, dev_priv, i) {
+ for_each_engine(engine, dev_priv) {
if (!i915.enable_execlists) {
struct drm_i915_gem_request *req;
- req = i915_gem_request_alloc(ring, NULL);
+ req = i915_gem_request_alloc(engine, NULL);
if (IS_ERR(req))
return PTR_ERR(req);
ret = i915_switch_context(req);
- if (ret) {
- i915_gem_request_cancel(req);
- return ret;
- }
-
i915_add_request_no_flush(req);
+ if (ret)
+ return ret;
}
- ret = intel_ring_idle(ring);
+ ret = intel_engine_idle(engine);
if (ret)
return ret;
}
@@ -3466,7 +3493,8 @@
uint64_t flags)
{
struct drm_device *dev = obj->base.dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
u32 fence_alignment, unfenced_alignment;
u32 search_flag, alloc_flag;
u64 start, end;
@@ -3513,7 +3541,7 @@
start = flags & PIN_OFFSET_BIAS ? flags & PIN_OFFSET_MASK : 0;
end = vm->total;
if (flags & PIN_MAPPABLE)
- end = min_t(u64, end, dev_priv->gtt.mappable_end);
+ end = min_t(u64, end, ggtt->mappable_end);
if (flags & PIN_ZONE_4G)
end = min_t(u64, end, (1ULL << 32) - PAGE_SIZE);
@@ -3720,6 +3748,9 @@
int
i915_gem_object_set_to_gtt_domain(struct drm_i915_gem_object *obj, bool write)
{
+ struct drm_device *dev = obj->base.dev;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
uint32_t old_write_domain, old_read_domains;
struct i915_vma *vma;
int ret;
@@ -3774,7 +3805,7 @@
vma = i915_gem_obj_to_ggtt(obj);
if (vma && drm_mm_node_allocated(&vma->node) && !obj->active)
list_move_tail(&vma->vm_link,
- &to_i915(obj->base.dev)->gtt.base.inactive_list);
+ &ggtt->base.inactive_list);
return 0;
}
@@ -3906,7 +3937,7 @@
struct drm_i915_gem_caching *args = data;
struct drm_i915_gem_object *obj;
- obj = to_intel_bo(drm_gem_object_lookup(dev, file, args->handle));
+ obj = to_intel_bo(drm_gem_object_lookup(file, args->handle));
if (&obj->base == NULL)
return -ENOENT;
@@ -3949,7 +3980,7 @@
* cacheline, whereas normally such cachelines would get
* invalidated.
*/
- if (IS_BXT_REVID(dev, 0, BXT_REVID_A1))
+ if (!HAS_LLC(dev) && !HAS_SNOOP(dev))
return -ENODEV;
level = I915_CACHE_LLC;
@@ -3967,7 +3998,7 @@
if (ret)
goto rpm_put;
- obj = to_intel_bo(drm_gem_object_lookup(dev, file, args->handle));
+ obj = to_intel_bo(drm_gem_object_lookup(file, args->handle));
if (&obj->base == NULL) {
ret = -ENOENT;
goto unlock;
@@ -4128,16 +4159,15 @@
struct drm_i915_file_private *file_priv = file->driver_priv;
unsigned long recent_enough = jiffies - DRM_I915_THROTTLE_JIFFIES;
struct drm_i915_gem_request *request, *target = NULL;
- unsigned reset_counter;
int ret;
ret = i915_gem_wait_for_error(&dev_priv->gpu_error);
if (ret)
return ret;
- ret = i915_gem_check_wedge(&dev_priv->gpu_error, false);
- if (ret)
- return ret;
+ /* ABI: return -EIO if already wedged */
+ if (i915_terminally_wedged(&dev_priv->gpu_error))
+ return -EIO;
spin_lock(&file_priv->mm.lock);
list_for_each_entry(request, &file_priv->mm.request_list, client_list) {
@@ -4153,7 +4183,6 @@
target = request;
}
- reset_counter = atomic_read(&dev_priv->gpu_error.reset_counter);
if (target)
i915_gem_request_reference(target);
spin_unlock(&file_priv->mm.lock);
@@ -4161,7 +4190,7 @@
if (target == NULL)
return 0;
- ret = __i915_wait_request(target, reset_counter, true, NULL, NULL);
+ ret = __i915_wait_request(target, true, NULL, NULL);
if (ret == 0)
queue_delayed_work(dev_priv->wq, &dev_priv->mm.retire_work, 0);
@@ -4211,7 +4240,7 @@
(vma->node.start & (fence_alignment - 1)) == 0);
mappable = (vma->node.start + fence_size <=
- to_i915(obj->base.dev)->gtt.mappable_end);
+ to_i915(obj->base.dev)->ggtt.mappable_end);
obj->map_and_fenceable = mappable && fenceable;
}
@@ -4243,9 +4272,6 @@
vma = ggtt_view ? i915_gem_obj_to_ggtt_view(obj, ggtt_view) :
i915_gem_obj_to_vma(obj, vm);
- if (IS_ERR(vma))
- return PTR_ERR(vma);
-
if (vma) {
if (WARN_ON(vma->pin_count == DRM_I915_GEM_OBJECT_MAX_PIN_COUNT))
return -EBUSY;
@@ -4308,10 +4334,13 @@
uint32_t alignment,
uint64_t flags)
{
- if (WARN_ONCE(!view, "no view specified"))
- return -EINVAL;
+ struct drm_device *dev = obj->base.dev;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
- return i915_gem_object_do_pin(obj, i915_obj_to_ggtt(obj), view,
+ BUG_ON(!view);
+
+ return i915_gem_object_do_pin(obj, &ggtt->base, view,
alignment, flags | PIN_GLOBAL);
}
@@ -4321,7 +4350,6 @@
{
struct i915_vma *vma = i915_gem_obj_to_ggtt_view(obj, view);
- BUG_ON(!vma);
WARN_ON(vma->pin_count == 0);
WARN_ON(!i915_gem_obj_ggtt_bound_view(obj, view));
@@ -4340,7 +4368,7 @@
if (ret)
return ret;
- obj = to_intel_bo(drm_gem_object_lookup(dev, file, args->handle));
+ obj = to_intel_bo(drm_gem_object_lookup(file, args->handle));
if (&obj->base == NULL) {
ret = -ENOENT;
goto unlock;
@@ -4359,15 +4387,15 @@
if (obj->active) {
int i;
- for (i = 0; i < I915_NUM_RINGS; i++) {
+ for (i = 0; i < I915_NUM_ENGINES; i++) {
struct drm_i915_gem_request *req;
req = obj->last_read_req[i];
if (req)
- args->busy |= 1 << (16 + req->ring->exec_id);
+ args->busy |= 1 << (16 + req->engine->exec_id);
}
if (obj->last_write_req)
- args->busy |= obj->last_write_req->ring->exec_id;
+ args->busy |= obj->last_write_req->engine->exec_id;
}
unref:
@@ -4405,7 +4433,7 @@
if (ret)
return ret;
- obj = to_intel_bo(drm_gem_object_lookup(dev, file_priv, args->handle));
+ obj = to_intel_bo(drm_gem_object_lookup(file_priv, args->handle));
if (&obj->base == NULL) {
ret = -ENOENT;
goto unlock;
@@ -4447,8 +4475,8 @@
int i;
INIT_LIST_HEAD(&obj->global_list);
- for (i = 0; i < I915_NUM_RINGS; i++)
- INIT_LIST_HEAD(&obj->ring_list[i]);
+ for (i = 0; i < I915_NUM_ENGINES; i++)
+ INIT_LIST_HEAD(&obj->engine_list[i]);
INIT_LIST_HEAD(&obj->obj_exec_link);
INIT_LIST_HEAD(&obj->vma_list);
INIT_LIST_HEAD(&obj->batch_pool_link);
@@ -4623,14 +4651,15 @@
struct i915_vma *i915_gem_obj_to_ggtt_view(struct drm_i915_gem_object *obj,
const struct i915_ggtt_view *view)
{
- struct i915_address_space *ggtt = i915_obj_to_ggtt(obj);
+ struct drm_device *dev = obj->base.dev;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct i915_vma *vma;
- if (WARN_ONCE(!view, "no view specified"))
- return ERR_PTR(-EINVAL);
+ BUG_ON(!view);
list_for_each_entry(vma, &obj->vma_list, obj_link)
- if (vma->vm == ggtt &&
+ if (vma->vm == &ggtt->base &&
i915_ggtt_view_equal(&vma->ggtt_view, view))
return vma;
return NULL;
@@ -4653,14 +4682,13 @@
}
static void
-i915_gem_stop_ringbuffers(struct drm_device *dev)
+i915_gem_stop_engines(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
- int i;
+ struct intel_engine_cs *engine;
- for_each_ring(ring, dev_priv, i)
- dev_priv->gt.stop_ring(ring);
+ for_each_engine(engine, dev_priv)
+ dev_priv->gt.stop_engine(engine);
}
int
@@ -4676,7 +4704,7 @@
i915_gem_retire_requests(dev);
- i915_gem_stop_ringbuffers(dev);
+ i915_gem_stop_engines(dev);
mutex_unlock(&dev->struct_mutex);
cancel_delayed_work_sync(&dev_priv->gpu_error.hangcheck_work);
@@ -4697,8 +4725,8 @@
int i915_gem_l3_remap(struct drm_i915_gem_request *req, int slice)
{
- struct intel_engine_cs *ring = req->ring;
- struct drm_device *dev = ring->dev;
+ struct intel_engine_cs *engine = req->engine;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
u32 *remap_info = dev_priv->l3_parity.remap_info[slice];
int i, ret;
@@ -4716,12 +4744,12 @@
* at initialization time.
*/
for (i = 0; i < GEN7_L3LOG_SIZE / 4; i++) {
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
- intel_ring_emit_reg(ring, GEN7_L3LOG(slice, i));
- intel_ring_emit(ring, remap_info[i]);
+ intel_ring_emit(engine, MI_LOAD_REGISTER_IMM(1));
+ intel_ring_emit_reg(engine, GEN7_L3LOG(slice, i));
+ intel_ring_emit(engine, remap_info[i]);
}
- intel_ring_advance(ring);
+ intel_ring_advance(engine);
return ret;
}
@@ -4778,7 +4806,7 @@
}
}
-int i915_gem_init_rings(struct drm_device *dev)
+int i915_gem_init_engines(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
int ret;
@@ -4814,13 +4842,13 @@
return 0;
cleanup_vebox_ring:
- intel_cleanup_ring_buffer(&dev_priv->ring[VECS]);
+ intel_cleanup_engine(&dev_priv->engine[VECS]);
cleanup_blt_ring:
- intel_cleanup_ring_buffer(&dev_priv->ring[BCS]);
+ intel_cleanup_engine(&dev_priv->engine[BCS]);
cleanup_bsd_ring:
- intel_cleanup_ring_buffer(&dev_priv->ring[VCS]);
+ intel_cleanup_engine(&dev_priv->engine[VCS]);
cleanup_render_ring:
- intel_cleanup_ring_buffer(&dev_priv->ring[RCS]);
+ intel_cleanup_engine(&dev_priv->engine[RCS]);
return ret;
}
@@ -4829,8 +4857,8 @@
i915_gem_init_hw(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
- int ret, i, j;
+ struct intel_engine_cs *engine;
+ int ret, j;
if (INTEL_INFO(dev)->gen < 6 && !intel_enable_gtt())
return -EIO;
@@ -4838,7 +4866,7 @@
/* Double layer security blanket, see i915_gem_init() */
intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
- if (dev_priv->ellc_size)
+ if (HAS_EDRAM(dev) && INTEL_GEN(dev_priv) < 9)
I915_WRITE(HSW_IDICR, I915_READ(HSW_IDICR) | IDIHASHMSK(0xf));
if (IS_HASWELL(dev))
@@ -4876,12 +4904,14 @@
}
/* Need to do basic initialisation of all rings first: */
- for_each_ring(ring, dev_priv, i) {
- ret = ring->init_hw(ring);
+ for_each_engine(engine, dev_priv) {
+ ret = engine->init_hw(engine);
if (ret)
goto out;
}
+ intel_mocs_init_l3cc_table(dev);
+
/* We can't enable contexts until all firmware is loaded */
if (HAS_GUC_UCODE(dev)) {
ret = intel_guc_ucode_load(dev);
@@ -4901,38 +4931,39 @@
goto out;
/* Now it is safe to go back round and do everything else: */
- for_each_ring(ring, dev_priv, i) {
+ for_each_engine(engine, dev_priv) {
struct drm_i915_gem_request *req;
- req = i915_gem_request_alloc(ring, NULL);
+ req = i915_gem_request_alloc(engine, NULL);
if (IS_ERR(req)) {
ret = PTR_ERR(req);
- i915_gem_cleanup_ringbuffer(dev);
- goto out;
+ break;
}
- if (ring->id == RCS) {
- for (j = 0; j < NUM_L3_SLICES(dev); j++)
- i915_gem_l3_remap(req, j);
+ if (engine->id == RCS) {
+ for (j = 0; j < NUM_L3_SLICES(dev); j++) {
+ ret = i915_gem_l3_remap(req, j);
+ if (ret)
+ goto err_request;
+ }
}
ret = i915_ppgtt_init_ring(req);
- if (ret && ret != -EIO) {
- DRM_ERROR("PPGTT enable ring #%d failed %d\n", i, ret);
- i915_gem_request_cancel(req);
- i915_gem_cleanup_ringbuffer(dev);
- goto out;
- }
+ if (ret)
+ goto err_request;
ret = i915_gem_context_enable(req);
- if (ret && ret != -EIO) {
- DRM_ERROR("Context enable ring #%d failed %d\n", i, ret);
- i915_gem_request_cancel(req);
- i915_gem_cleanup_ringbuffer(dev);
- goto out;
- }
+ if (ret)
+ goto err_request;
+err_request:
i915_add_request_no_flush(req);
+ if (ret) {
+ DRM_ERROR("Failed to enable %s, error=%d\n",
+ engine->name, ret);
+ i915_gem_cleanup_engines(dev);
+ break;
+ }
}
out:
@@ -4952,14 +4983,14 @@
if (!i915.enable_execlists) {
dev_priv->gt.execbuf_submit = i915_gem_ringbuffer_submission;
- dev_priv->gt.init_rings = i915_gem_init_rings;
- dev_priv->gt.cleanup_ring = intel_cleanup_ring_buffer;
- dev_priv->gt.stop_ring = intel_stop_ring_buffer;
+ dev_priv->gt.init_engines = i915_gem_init_engines;
+ dev_priv->gt.cleanup_engine = intel_cleanup_engine;
+ dev_priv->gt.stop_engine = intel_stop_engine;
} else {
dev_priv->gt.execbuf_submit = intel_execlists_submission;
- dev_priv->gt.init_rings = intel_logical_rings_init;
- dev_priv->gt.cleanup_ring = intel_logical_ring_cleanup;
- dev_priv->gt.stop_ring = intel_logical_ring_stop;
+ dev_priv->gt.init_engines = intel_logical_rings_init;
+ dev_priv->gt.cleanup_engine = intel_logical_ring_cleanup;
+ dev_priv->gt.stop_engine = intel_logical_ring_stop;
}
/* This is just a security blanket to placate dragons.
@@ -4974,13 +5005,13 @@
if (ret)
goto out_unlock;
- i915_gem_init_global_gtt(dev);
+ i915_gem_init_ggtt(dev);
ret = i915_gem_context_init(dev);
if (ret)
goto out_unlock;
- ret = dev_priv->gt.init_rings(dev);
+ ret = dev_priv->gt.init_engines(dev);
if (ret)
goto out_unlock;
@@ -5003,29 +5034,52 @@
}
void
-i915_gem_cleanup_ringbuffer(struct drm_device *dev)
+i915_gem_cleanup_engines(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
- int i;
+ struct intel_engine_cs *engine;
- for_each_ring(ring, dev_priv, i)
- dev_priv->gt.cleanup_ring(ring);
+ for_each_engine(engine, dev_priv)
+ dev_priv->gt.cleanup_engine(engine);
- if (i915.enable_execlists)
- /*
- * Neither the BIOS, ourselves or any other kernel
- * expects the system to be in execlists mode on startup,
- * so we need to reset the GPU back to legacy mode.
- */
- intel_gpu_reset(dev);
+ if (i915.enable_execlists)
+ /*
+ * Neither the BIOS, ourselves or any other kernel
+ * expects the system to be in execlists mode on startup,
+ * so we need to reset the GPU back to legacy mode.
+ */
+ intel_gpu_reset(dev, ALL_ENGINES);
}
static void
-init_ring_lists(struct intel_engine_cs *ring)
+init_engine_lists(struct intel_engine_cs *engine)
{
- INIT_LIST_HEAD(&ring->active_list);
- INIT_LIST_HEAD(&ring->request_list);
+ INIT_LIST_HEAD(&engine->active_list);
+ INIT_LIST_HEAD(&engine->request_list);
+}
+
+void
+i915_gem_load_init_fences(struct drm_i915_private *dev_priv)
+{
+ struct drm_device *dev = dev_priv->dev;
+
+ if (INTEL_INFO(dev_priv)->gen >= 7 && !IS_VALLEYVIEW(dev_priv) &&
+ !IS_CHERRYVIEW(dev_priv))
+ dev_priv->num_fence_regs = 32;
+ else if (INTEL_INFO(dev_priv)->gen >= 4 || IS_I945G(dev_priv) ||
+ IS_I945GM(dev_priv) || IS_G33(dev_priv))
+ dev_priv->num_fence_regs = 16;
+ else
+ dev_priv->num_fence_regs = 8;
+
+ if (intel_vgpu_active(dev))
+ dev_priv->num_fence_regs =
+ I915_READ(vgtif_reg(avail_rs.fence_num));
+
+ /* Initialize fence registers to zero */
+ i915_gem_restore_fences(dev);
+
+ i915_gem_detect_bit_6_swizzle(dev);
}
void
@@ -5055,8 +5109,8 @@
INIT_LIST_HEAD(&dev_priv->mm.unbound_list);
INIT_LIST_HEAD(&dev_priv->mm.bound_list);
INIT_LIST_HEAD(&dev_priv->mm.fence_list);
- for (i = 0; i < I915_NUM_RINGS; i++)
- init_ring_lists(&dev_priv->ring[i]);
+ for (i = 0; i < I915_NUM_ENGINES; i++)
+ init_engine_lists(&dev_priv->engine[i]);
for (i = 0; i < I915_MAX_NUM_FENCES; i++)
INIT_LIST_HEAD(&dev_priv->fence_regs[i].lru_list);
INIT_DELAYED_WORK(&dev_priv->mm.retire_work,
@@ -5067,17 +5121,6 @@
dev_priv->relative_constants_mode = I915_EXEC_CONSTANTS_REL_GENERAL;
- if (INTEL_INFO(dev)->gen >= 7 && !IS_VALLEYVIEW(dev) && !IS_CHERRYVIEW(dev))
- dev_priv->num_fence_regs = 32;
- else if (INTEL_INFO(dev)->gen >= 4 || IS_I945G(dev) || IS_I945GM(dev) || IS_G33(dev))
- dev_priv->num_fence_regs = 16;
- else
- dev_priv->num_fence_regs = 8;
-
- if (intel_vgpu_active(dev))
- dev_priv->num_fence_regs =
- I915_READ(vgtif_reg(avail_rs.fence_num));
-
/*
* Set initial sequence number for requests.
* Using this number allows the wraparound to happen early,
@@ -5086,11 +5129,8 @@
dev_priv->next_seqno = ((u32)~0 - 0x1100);
dev_priv->last_seqno = ((u32)~0 - 0x1101);
- /* Initialize fence registers to zero */
INIT_LIST_HEAD(&dev_priv->mm.fence_list);
- i915_gem_restore_fences(dev);
- i915_gem_detect_bit_6_swizzle(dev);
init_waitqueue_head(&dev_priv->pending_flip_queue);
dev_priv->mm.interruptible = true;
@@ -5213,11 +5253,12 @@
u64 i915_gem_obj_ggtt_offset_view(struct drm_i915_gem_object *o,
const struct i915_ggtt_view *view)
{
- struct i915_address_space *ggtt = i915_obj_to_ggtt(o);
+ struct drm_i915_private *dev_priv = to_i915(o->base.dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct i915_vma *vma;
list_for_each_entry(vma, &o->vma_list, obj_link)
- if (vma->vm == ggtt &&
+ if (vma->vm == &ggtt->base &&
i915_ggtt_view_equal(&vma->ggtt_view, view))
return vma->node.start;
@@ -5244,11 +5285,12 @@
bool i915_gem_obj_ggtt_bound_view(struct drm_i915_gem_object *o,
const struct i915_ggtt_view *view)
{
- struct i915_address_space *ggtt = i915_obj_to_ggtt(o);
+ struct drm_i915_private *dev_priv = to_i915(o->base.dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct i915_vma *vma;
list_for_each_entry(vma, &o->vma_list, obj_link)
- if (vma->vm == ggtt &&
+ if (vma->vm == &ggtt->base &&
i915_ggtt_view_equal(&vma->ggtt_view, view) &&
drm_mm_node_allocated(&vma->node))
return true;
diff --git a/drivers/gpu/drm/i915/i915_gem.h b/drivers/gpu/drm/i915/i915_gem.h
new file mode 100644
index 0000000..8292e79
--- /dev/null
+++ b/drivers/gpu/drm/i915/i915_gem.h
@@ -0,0 +1,34 @@
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __I915_GEM_H__
+#define __I915_GEM_H__
+
+#ifdef CONFIG_DRM_I915_DEBUG_GEM
+#define GEM_BUG_ON(expr) BUG_ON(expr)
+#else
+#define GEM_BUG_ON(expr)
+#endif
+
+#endif /* __I915_GEM_H__ */
diff --git a/drivers/gpu/drm/i915/i915_gem_context.c b/drivers/gpu/drm/i915/i915_gem_context.c
index 5dd84e1..e5acc39 100644
--- a/drivers/gpu/drm/i915/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/i915_gem_context.c
@@ -342,15 +342,15 @@
struct intel_context *ctx;
list_for_each_entry(ctx, &dev_priv->context_list, link)
- intel_lr_context_reset(dev, ctx);
+ intel_lr_context_reset(dev_priv, ctx);
}
- for (i = 0; i < I915_NUM_RINGS; i++) {
- struct intel_engine_cs *ring = &dev_priv->ring[i];
+ for (i = 0; i < I915_NUM_ENGINES; i++) {
+ struct intel_engine_cs *engine = &dev_priv->engine[i];
- if (ring->last_context) {
- i915_gem_context_unpin(ring->last_context, ring);
- ring->last_context = NULL;
+ if (engine->last_context) {
+ i915_gem_context_unpin(engine->last_context, engine);
+ engine->last_context = NULL;
}
}
@@ -413,7 +413,7 @@
/* The only known way to stop the gpu from accessing the hw context is
* to reset it. Do this as the very last operation to avoid confusing
* other code, leading to spurious errors. */
- intel_gpu_reset(dev);
+ intel_gpu_reset(dev, ALL_ENGINES);
/* When default context is created and switched to, base object refcount
* will be 2 (+1 from object creation and +1 from do_switch()).
@@ -421,17 +421,17 @@
* to default context. So we need to unreference the base object once
* to offset the do_switch part, so that i915_gem_context_unreference()
* can then free the base object correctly. */
- WARN_ON(!dev_priv->ring[RCS].last_context);
+ WARN_ON(!dev_priv->engine[RCS].last_context);
i915_gem_object_ggtt_unpin(dctx->legacy_hw_ctx.rcs_state);
}
- for (i = I915_NUM_RINGS; --i >= 0;) {
- struct intel_engine_cs *ring = &dev_priv->ring[i];
+ for (i = I915_NUM_ENGINES; --i >= 0;) {
+ struct intel_engine_cs *engine = &dev_priv->engine[i];
- if (ring->last_context) {
- i915_gem_context_unpin(ring->last_context, ring);
- ring->last_context = NULL;
+ if (engine->last_context) {
+ i915_gem_context_unpin(engine->last_context, engine);
+ engine->last_context = NULL;
}
}
@@ -441,14 +441,14 @@
int i915_gem_context_enable(struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
int ret;
if (i915.enable_execlists) {
- if (ring->init_context == NULL)
+ if (engine->init_context == NULL)
return 0;
- ret = ring->init_context(req);
+ ret = engine->init_context(req);
} else
ret = i915_switch_context(req);
@@ -510,133 +510,147 @@
static inline int
mi_set_context(struct drm_i915_gem_request *req, u32 hw_flags)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
u32 flags = hw_flags | MI_MM_SPACE_GTT;
const int num_rings =
/* Use an extended w/a on ivb+ if signalling from other rings */
- i915_semaphore_is_enabled(ring->dev) ?
- hweight32(INTEL_INFO(ring->dev)->ring_mask) - 1 :
+ i915_semaphore_is_enabled(engine->dev) ?
+ hweight32(INTEL_INFO(engine->dev)->ring_mask) - 1 :
0;
- int len, i, ret;
+ int len, ret;
/* w/a: If Flush TLB Invalidation Mode is enabled, driver must do a TLB
* invalidation prior to MI_SET_CONTEXT. On GEN6 we don't set the value
* explicitly, so we rely on the value at ring init, stored in
* itlb_before_ctx_switch.
*/
- if (IS_GEN6(ring->dev)) {
- ret = ring->flush(req, I915_GEM_GPU_DOMAINS, 0);
+ if (IS_GEN6(engine->dev)) {
+ ret = engine->flush(req, I915_GEM_GPU_DOMAINS, 0);
if (ret)
return ret;
}
/* These flags are for resource streamer on HSW+ */
- if (IS_HASWELL(ring->dev) || INTEL_INFO(ring->dev)->gen >= 8)
+ if (IS_HASWELL(engine->dev) || INTEL_INFO(engine->dev)->gen >= 8)
flags |= (HSW_MI_RS_SAVE_STATE_EN | HSW_MI_RS_RESTORE_STATE_EN);
- else if (INTEL_INFO(ring->dev)->gen < 8)
+ else if (INTEL_INFO(engine->dev)->gen < 8)
flags |= (MI_SAVE_EXT_STATE_EN | MI_RESTORE_EXT_STATE_EN);
len = 4;
- if (INTEL_INFO(ring->dev)->gen >= 7)
- len += 2 + (num_rings ? 4*num_rings + 2 : 0);
+ if (INTEL_INFO(engine->dev)->gen >= 7)
+ len += 2 + (num_rings ? 4*num_rings + 6 : 0);
ret = intel_ring_begin(req, len);
if (ret)
return ret;
/* WaProgramMiArbOnOffAroundMiSetContext:ivb,vlv,hsw,bdw,chv */
- if (INTEL_INFO(ring->dev)->gen >= 7) {
- intel_ring_emit(ring, MI_ARB_ON_OFF | MI_ARB_DISABLE);
+ if (INTEL_INFO(engine->dev)->gen >= 7) {
+ intel_ring_emit(engine, MI_ARB_ON_OFF | MI_ARB_DISABLE);
if (num_rings) {
struct intel_engine_cs *signaller;
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(num_rings));
- for_each_ring(signaller, to_i915(ring->dev), i) {
- if (signaller == ring)
+ intel_ring_emit(engine,
+ MI_LOAD_REGISTER_IMM(num_rings));
+ for_each_engine(signaller, to_i915(engine->dev)) {
+ if (signaller == engine)
continue;
- intel_ring_emit_reg(ring, RING_PSMI_CTL(signaller->mmio_base));
- intel_ring_emit(ring, _MASKED_BIT_ENABLE(GEN6_PSMI_SLEEP_MSG_DISABLE));
+ intel_ring_emit_reg(engine,
+ RING_PSMI_CTL(signaller->mmio_base));
+ intel_ring_emit(engine,
+ _MASKED_BIT_ENABLE(GEN6_PSMI_SLEEP_MSG_DISABLE));
}
}
}
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_emit(ring, MI_SET_CONTEXT);
- intel_ring_emit(ring, i915_gem_obj_ggtt_offset(req->ctx->legacy_hw_ctx.rcs_state) |
+ intel_ring_emit(engine, MI_NOOP);
+ intel_ring_emit(engine, MI_SET_CONTEXT);
+ intel_ring_emit(engine,
+ i915_gem_obj_ggtt_offset(req->ctx->legacy_hw_ctx.rcs_state) |
flags);
/*
* w/a: MI_SET_CONTEXT must always be followed by MI_NOOP
* WaMiSetContext_Hang:snb,ivb,vlv
*/
- intel_ring_emit(ring, MI_NOOP);
+ intel_ring_emit(engine, MI_NOOP);
- if (INTEL_INFO(ring->dev)->gen >= 7) {
+ if (INTEL_INFO(engine->dev)->gen >= 7) {
if (num_rings) {
struct intel_engine_cs *signaller;
+ i915_reg_t last_reg = {}; /* keep gcc quiet */
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(num_rings));
- for_each_ring(signaller, to_i915(ring->dev), i) {
- if (signaller == ring)
+ intel_ring_emit(engine,
+ MI_LOAD_REGISTER_IMM(num_rings));
+ for_each_engine(signaller, to_i915(engine->dev)) {
+ if (signaller == engine)
continue;
- intel_ring_emit_reg(ring, RING_PSMI_CTL(signaller->mmio_base));
- intel_ring_emit(ring, _MASKED_BIT_DISABLE(GEN6_PSMI_SLEEP_MSG_DISABLE));
+ last_reg = RING_PSMI_CTL(signaller->mmio_base);
+ intel_ring_emit_reg(engine, last_reg);
+ intel_ring_emit(engine,
+ _MASKED_BIT_DISABLE(GEN6_PSMI_SLEEP_MSG_DISABLE));
}
+
+ /* Insert a delay before the next switch! */
+ intel_ring_emit(engine,
+ MI_STORE_REGISTER_MEM |
+ MI_SRM_LRM_GLOBAL_GTT);
+ intel_ring_emit_reg(engine, last_reg);
+ intel_ring_emit(engine, engine->scratch.gtt_offset);
+ intel_ring_emit(engine, MI_NOOP);
}
- intel_ring_emit(ring, MI_ARB_ON_OFF | MI_ARB_ENABLE);
+ intel_ring_emit(engine, MI_ARB_ON_OFF | MI_ARB_ENABLE);
}
- intel_ring_advance(ring);
+ intel_ring_advance(engine);
return ret;
}
-static inline bool should_skip_switch(struct intel_engine_cs *ring,
- struct intel_context *from,
- struct intel_context *to)
+static inline bool skip_rcs_switch(struct intel_engine_cs *engine,
+ struct intel_context *to)
{
if (to->remap_slice)
return false;
- if (to->ppgtt && from == to &&
- !(intel_ring_flag(ring) & to->ppgtt->pd_dirty_rings))
+ if (!to->legacy_hw_ctx.initialized)
+ return false;
+
+ if (to->ppgtt &&
+ !(intel_engine_flag(engine) & to->ppgtt->pd_dirty_rings))
+ return false;
+
+ return to == engine->last_context;
+}
+
+static bool
+needs_pd_load_pre(struct intel_engine_cs *engine, struct intel_context *to)
+{
+ if (!to->ppgtt)
+ return false;
+
+ if (engine->last_context == to &&
+ !(intel_engine_flag(engine) & to->ppgtt->pd_dirty_rings))
+ return false;
+
+ if (engine->id != RCS)
+ return true;
+
+ if (INTEL_INFO(engine->dev)->gen < 8)
return true;
return false;
}
static bool
-needs_pd_load_pre(struct intel_engine_cs *ring, struct intel_context *to)
+needs_pd_load_post(struct intel_context *to, u32 hw_flags)
{
- struct drm_i915_private *dev_priv = ring->dev->dev_private;
-
if (!to->ppgtt)
return false;
- if (INTEL_INFO(ring->dev)->gen < 8)
- return true;
-
- if (ring != &dev_priv->ring[RCS])
- return true;
-
- return false;
-}
-
-static bool
-needs_pd_load_post(struct intel_engine_cs *ring, struct intel_context *to,
- u32 hw_flags)
-{
- struct drm_i915_private *dev_priv = ring->dev->dev_private;
-
- if (!to->ppgtt)
- return false;
-
- if (!IS_GEN8(ring->dev))
- return false;
-
- if (ring != &dev_priv->ring[RCS])
+ if (!IS_GEN8(to->i915))
return false;
if (hw_flags & MI_RESTORE_INHIBIT)
@@ -645,58 +659,32 @@
return false;
}
-static int do_switch(struct drm_i915_gem_request *req)
+static int do_rcs_switch(struct drm_i915_gem_request *req)
{
struct intel_context *to = req->ctx;
- struct intel_engine_cs *ring = req->ring;
- struct drm_i915_private *dev_priv = ring->dev->dev_private;
- struct intel_context *from = ring->last_context;
- u32 hw_flags = 0;
- bool uninitialized = false;
+ struct intel_engine_cs *engine = req->engine;
+ struct intel_context *from;
+ u32 hw_flags;
int ret, i;
- if (from != NULL && ring == &dev_priv->ring[RCS]) {
- BUG_ON(from->legacy_hw_ctx.rcs_state == NULL);
- BUG_ON(!i915_gem_obj_is_pinned(from->legacy_hw_ctx.rcs_state));
- }
-
- if (should_skip_switch(ring, from, to))
+ if (skip_rcs_switch(engine, to))
return 0;
/* Trying to pin first makes error handling easier. */
- if (ring == &dev_priv->ring[RCS]) {
- ret = i915_gem_obj_ggtt_pin(to->legacy_hw_ctx.rcs_state,
- get_context_alignment(ring->dev), 0);
- if (ret)
- return ret;
- }
+ ret = i915_gem_obj_ggtt_pin(to->legacy_hw_ctx.rcs_state,
+ get_context_alignment(engine->dev),
+ 0);
+ if (ret)
+ return ret;
/*
* Pin can switch back to the default context if we end up calling into
* evict_everything - as a last ditch gtt defrag effort that also
* switches to the default context. Hence we need to reload from here.
+ *
+ * XXX: Doing so is painfully broken!
*/
- from = ring->last_context;
-
- if (needs_pd_load_pre(ring, to)) {
- /* Older GENs and non render rings still want the load first,
- * "PP_DCLV followed by PP_DIR_BASE register through Load
- * Register Immediate commands in Ring Buffer before submitting
- * a context."*/
- trace_switch_mm(ring, to);
- ret = to->ppgtt->switch_mm(to->ppgtt, req);
- if (ret)
- goto unpin_out;
-
- /* Doing a PD load always reloads the page dirs */
- to->ppgtt->pd_dirty_rings &= ~intel_ring_flag(ring);
- }
-
- if (ring != &dev_priv->ring[RCS]) {
- if (from)
- i915_gem_context_unreference(from);
- goto done;
- }
+ from = engine->last_context;
/*
* Clear this page out of any CPU caches for coherent swap-in/out. Note
@@ -710,53 +698,37 @@
if (ret)
goto unpin_out;
- if (!to->legacy_hw_ctx.initialized || i915_gem_context_is_default(to)) {
- hw_flags |= MI_RESTORE_INHIBIT;
+ if (needs_pd_load_pre(engine, to)) {
+ /* Older GENs and non render rings still want the load first,
+ * "PP_DCLV followed by PP_DIR_BASE register through Load
+ * Register Immediate commands in Ring Buffer before submitting
+ * a context."*/
+ trace_switch_mm(engine, to);
+ ret = to->ppgtt->switch_mm(to->ppgtt, req);
+ if (ret)
+ goto unpin_out;
+ }
+
+ if (!to->legacy_hw_ctx.initialized || i915_gem_context_is_default(to))
/* NB: If we inhibit the restore, the context is not allowed to
* die because future work may end up depending on valid address
* space. This means we must enforce that a page table load
* occur when this occurs. */
- } else if (to->ppgtt &&
- (intel_ring_flag(ring) & to->ppgtt->pd_dirty_rings)) {
- hw_flags |= MI_FORCE_RESTORE;
- to->ppgtt->pd_dirty_rings &= ~intel_ring_flag(ring);
- }
+ hw_flags = MI_RESTORE_INHIBIT;
+ else if (to->ppgtt &&
+ intel_engine_flag(engine) & to->ppgtt->pd_dirty_rings)
+ hw_flags = MI_FORCE_RESTORE;
+ else
+ hw_flags = 0;
/* We should never emit switch_mm more than once */
- WARN_ON(needs_pd_load_pre(ring, to) &&
- needs_pd_load_post(ring, to, hw_flags));
+ WARN_ON(needs_pd_load_pre(engine, to) &&
+ needs_pd_load_post(to, hw_flags));
- ret = mi_set_context(req, hw_flags);
- if (ret)
- goto unpin_out;
-
- /* GEN8 does *not* require an explicit reload if the PDPs have been
- * setup, and we do not wish to move them.
- */
- if (needs_pd_load_post(ring, to, hw_flags)) {
- trace_switch_mm(ring, to);
- ret = to->ppgtt->switch_mm(to->ppgtt, req);
- /* The hardware context switch is emitted, but we haven't
- * actually changed the state - so it's probably safe to bail
- * here. Still, let the user know something dangerous has
- * happened.
- */
- if (ret) {
- DRM_ERROR("Failed to change address space on context switch\n");
- goto unpin_out;
- }
- }
-
- for (i = 0; i < MAX_L3_SLICES; i++) {
- if (!(to->remap_slice & (1<<i)))
- continue;
-
- ret = i915_gem_l3_remap(req, i);
- /* If it failed, try again next round */
+ if (to != from || (hw_flags & MI_FORCE_RESTORE)) {
+ ret = mi_set_context(req, hw_flags);
if (ret)
- DRM_DEBUG_DRIVER("L3 remapping failed\n");
- else
- to->remap_slice &= ~(1<<i);
+ goto unpin_out;
}
/* The backing object for the context is done after switching to the
@@ -781,27 +753,51 @@
i915_gem_object_ggtt_unpin(from->legacy_hw_ctx.rcs_state);
i915_gem_context_unreference(from);
}
-
- uninitialized = !to->legacy_hw_ctx.initialized;
- to->legacy_hw_ctx.initialized = true;
-
-done:
i915_gem_context_reference(to);
- ring->last_context = to;
+ engine->last_context = to;
- if (uninitialized) {
- if (ring->init_context) {
- ret = ring->init_context(req);
+ /* GEN8 does *not* require an explicit reload if the PDPs have been
+ * setup, and we do not wish to move them.
+ */
+ if (needs_pd_load_post(to, hw_flags)) {
+ trace_switch_mm(engine, to);
+ ret = to->ppgtt->switch_mm(to->ppgtt, req);
+ /* The hardware context switch is emitted, but we haven't
+ * actually changed the state - so it's probably safe to bail
+ * here. Still, let the user know something dangerous has
+ * happened.
+ */
+ if (ret)
+ return ret;
+ }
+
+ if (to->ppgtt)
+ to->ppgtt->pd_dirty_rings &= ~intel_engine_flag(engine);
+
+ for (i = 0; i < MAX_L3_SLICES; i++) {
+ if (!(to->remap_slice & (1<<i)))
+ continue;
+
+ ret = i915_gem_l3_remap(req, i);
+ if (ret)
+ return ret;
+
+ to->remap_slice &= ~(1<<i);
+ }
+
+ if (!to->legacy_hw_ctx.initialized) {
+ if (engine->init_context) {
+ ret = engine->init_context(req);
if (ret)
- DRM_ERROR("ring init context: %d\n", ret);
+ return ret;
}
+ to->legacy_hw_ctx.initialized = true;
}
return 0;
unpin_out:
- if (ring->id == RCS)
- i915_gem_object_ggtt_unpin(to->legacy_hw_ctx.rcs_state);
+ i915_gem_object_ggtt_unpin(to->legacy_hw_ctx.rcs_state);
return ret;
}
@@ -820,23 +816,39 @@
*/
int i915_switch_context(struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *ring = req->ring;
- struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct intel_engine_cs *engine = req->engine;
+ struct drm_i915_private *dev_priv = req->i915;
WARN_ON(i915.enable_execlists);
WARN_ON(!mutex_is_locked(&dev_priv->dev->struct_mutex));
- if (req->ctx->legacy_hw_ctx.rcs_state == NULL) { /* We have the fake context */
- if (req->ctx != ring->last_context) {
- i915_gem_context_reference(req->ctx);
- if (ring->last_context)
- i915_gem_context_unreference(ring->last_context);
- ring->last_context = req->ctx;
+ if (engine->id != RCS ||
+ req->ctx->legacy_hw_ctx.rcs_state == NULL) {
+ struct intel_context *to = req->ctx;
+
+ if (needs_pd_load_pre(engine, to)) {
+ int ret;
+
+ trace_switch_mm(engine, to);
+ ret = to->ppgtt->switch_mm(to->ppgtt, req);
+ if (ret)
+ return ret;
+
+ /* Doing a PD load always reloads the page dirs */
+ to->ppgtt->pd_dirty_rings &= ~intel_engine_flag(engine);
}
+
+ if (to != engine->last_context) {
+ i915_gem_context_reference(to);
+ if (engine->last_context)
+ i915_gem_context_unreference(engine->last_context);
+ engine->last_context = to;
+ }
+
return 0;
}
- return do_switch(req);
+ return do_rcs_switch(req);
}
static bool contexts_enabled(struct drm_device *dev)
@@ -937,7 +949,7 @@
else if (to_i915(dev)->mm.aliasing_ppgtt)
args->value = to_i915(dev)->mm.aliasing_ppgtt->base.total;
else
- args->value = to_i915(dev)->gtt.base.total;
+ args->value = to_i915(dev)->ggtt.base.total;
break;
default:
ret = -EINVAL;
diff --git a/drivers/gpu/drm/i915/i915_gem_debug.c b/drivers/gpu/drm/i915/i915_gem_debug.c
index 17299d0..a565164 100644
--- a/drivers/gpu/drm/i915/i915_gem_debug.c
+++ b/drivers/gpu/drm/i915/i915_gem_debug.c
@@ -36,29 +36,29 @@
static int warned;
struct drm_i915_private *dev_priv = to_i915(dev);
struct drm_i915_gem_object *obj;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
int err = 0;
- int i;
if (warned)
return 0;
- for_each_ring(ring, dev_priv, i) {
- list_for_each_entry(obj, &ring->active_list, ring_list[ring->id]) {
+ for_each_engine(engine, dev_priv) {
+ list_for_each_entry(obj, &engine->active_list,
+ engine_list[engine->id]) {
if (obj->base.dev != dev ||
!atomic_read(&obj->base.refcount.refcount)) {
DRM_ERROR("%s: freed active obj %p\n",
- ring->name, obj);
+ engine->name, obj);
err++;
break;
} else if (!obj->active ||
- obj->last_read_req[ring->id] == NULL) {
+ obj->last_read_req[engine->id] == NULL) {
DRM_ERROR("%s: invalid active obj %p\n",
- ring->name, obj);
+ engine->name, obj);
err++;
} else if (obj->base.write_domain) {
DRM_ERROR("%s: invalid write obj %p (w %x)\n",
- ring->name,
+ engine->name,
obj, obj->base.write_domain);
err++;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/i915_gem_dmabuf.c
index 0506016..80bbe43 100644
--- a/drivers/gpu/drm/i915/i915_gem_dmabuf.c
+++ b/drivers/gpu/drm/i915/i915_gem_dmabuf.c
@@ -95,14 +95,12 @@
{
struct drm_i915_gem_object *obj = dma_buf_to_obj(attachment->dmabuf);
- mutex_lock(&obj->base.dev->struct_mutex);
-
dma_unmap_sg(attachment->dev, sg->sgl, sg->nents, dir);
sg_free_table(sg);
kfree(sg);
+ mutex_lock(&obj->base.dev->struct_mutex);
i915_gem_object_unpin_pages(obj);
-
mutex_unlock(&obj->base.dev->struct_mutex);
}
@@ -110,51 +108,17 @@
{
struct drm_i915_gem_object *obj = dma_buf_to_obj(dma_buf);
struct drm_device *dev = obj->base.dev;
- struct sg_page_iter sg_iter;
- struct page **pages;
- int ret, i;
+ void *addr;
+ int ret;
ret = i915_mutex_lock_interruptible(dev);
if (ret)
return ERR_PTR(ret);
- if (obj->dma_buf_vmapping) {
- obj->vmapping_count++;
- goto out_unlock;
- }
-
- ret = i915_gem_object_get_pages(obj);
- if (ret)
- goto err;
-
- i915_gem_object_pin_pages(obj);
-
- ret = -ENOMEM;
-
- pages = drm_malloc_ab(obj->base.size >> PAGE_SHIFT, sizeof(*pages));
- if (pages == NULL)
- goto err_unpin;
-
- i = 0;
- for_each_sg_page(obj->pages->sgl, &sg_iter, obj->pages->nents, 0)
- pages[i++] = sg_page_iter_page(&sg_iter);
-
- obj->dma_buf_vmapping = vmap(pages, i, 0, PAGE_KERNEL);
- drm_free_large(pages);
-
- if (!obj->dma_buf_vmapping)
- goto err_unpin;
-
- obj->vmapping_count = 1;
-out_unlock:
+ addr = i915_gem_object_pin_map(obj);
mutex_unlock(&dev->struct_mutex);
- return obj->dma_buf_vmapping;
-err_unpin:
- i915_gem_object_unpin_pages(obj);
-err:
- mutex_unlock(&dev->struct_mutex);
- return ERR_PTR(ret);
+ return addr;
}
static void i915_gem_dmabuf_vunmap(struct dma_buf *dma_buf, void *vaddr)
@@ -163,12 +127,7 @@
struct drm_device *dev = obj->base.dev;
mutex_lock(&dev->struct_mutex);
- if (--obj->vmapping_count == 0) {
- vunmap(obj->dma_buf_vmapping);
- obj->dma_buf_vmapping = NULL;
-
- i915_gem_object_unpin_pages(obj);
- }
+ i915_gem_object_unpin_map(obj);
mutex_unlock(&dev->struct_mutex);
}
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index b845f46..33df74d 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -313,7 +313,8 @@
uint64_t target_offset)
{
struct drm_device *dev = obj->base.dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
uint64_t delta = relocation_target(reloc, target_offset);
uint64_t offset;
void __iomem *reloc_page;
@@ -330,7 +331,7 @@
/* Map the page containing the relocation we're going to perform. */
offset = i915_gem_obj_ggtt_offset(obj);
offset += reloc->offset;
- reloc_page = io_mapping_map_atomic_wc(dev_priv->gtt.mappable,
+ reloc_page = io_mapping_map_atomic_wc(ggtt->mappable,
offset & PAGE_MASK);
iowrite32(lower_32_bits(delta), reloc_page + offset_in_page(offset));
@@ -340,7 +341,7 @@
if (offset_in_page(offset) == 0) {
io_mapping_unmap_atomic(reloc_page);
reloc_page =
- io_mapping_map_atomic_wc(dev_priv->gtt.mappable,
+ io_mapping_map_atomic_wc(ggtt->mappable,
offset);
}
@@ -514,7 +515,7 @@
struct drm_i915_gem_exec_object2 *entry = vma->exec_entry;
int remain, ret;
- user_relocs = to_user_ptr(entry->relocs_ptr);
+ user_relocs = u64_to_user_ptr(entry->relocs_ptr);
remain = entry->relocation_count;
while (remain) {
@@ -535,9 +536,7 @@
return ret;
if (r->presumed_offset != offset &&
- __copy_to_user_inatomic(&user_relocs->presumed_offset,
- &r->presumed_offset,
- sizeof(r->presumed_offset))) {
+ __put_user(r->presumed_offset, &user_relocs->presumed_offset)) {
return -EFAULT;
}
@@ -599,7 +598,7 @@
static int
i915_gem_execbuffer_reserve_vma(struct i915_vma *vma,
- struct intel_engine_cs *ring,
+ struct intel_engine_cs *engine,
bool *need_reloc)
{
struct drm_i915_gem_object *obj = vma->obj;
@@ -713,7 +712,7 @@
}
static int
-i915_gem_execbuffer_reserve(struct intel_engine_cs *ring,
+i915_gem_execbuffer_reserve(struct intel_engine_cs *engine,
struct list_head *vmas,
struct intel_context *ctx,
bool *need_relocs)
@@ -723,10 +722,10 @@
struct i915_address_space *vm;
struct list_head ordered_vmas;
struct list_head pinned_vmas;
- bool has_fenced_gpu_access = INTEL_INFO(ring->dev)->gen < 4;
+ bool has_fenced_gpu_access = INTEL_INFO(engine->dev)->gen < 4;
int retry;
- i915_gem_retire_requests_ring(ring);
+ i915_gem_retire_requests_ring(engine);
vm = list_first_entry(vmas, struct i915_vma, exec_list)->vm;
@@ -788,7 +787,9 @@
if (eb_vma_misplaced(vma))
ret = i915_vma_unbind(vma);
else
- ret = i915_gem_execbuffer_reserve_vma(vma, ring, need_relocs);
+ ret = i915_gem_execbuffer_reserve_vma(vma,
+ engine,
+ need_relocs);
if (ret)
goto err;
}
@@ -798,7 +799,8 @@
if (drm_mm_node_allocated(&vma->node))
continue;
- ret = i915_gem_execbuffer_reserve_vma(vma, ring, need_relocs);
+ ret = i915_gem_execbuffer_reserve_vma(vma, engine,
+ need_relocs);
if (ret)
goto err;
}
@@ -821,7 +823,7 @@
i915_gem_execbuffer_relocate_slow(struct drm_device *dev,
struct drm_i915_gem_execbuffer2 *args,
struct drm_file *file,
- struct intel_engine_cs *ring,
+ struct intel_engine_cs *engine,
struct eb_vmas *eb,
struct drm_i915_gem_exec_object2 *exec,
struct intel_context *ctx)
@@ -865,7 +867,7 @@
u64 invalid_offset = (u64)-1;
int j;
- user_relocs = to_user_ptr(exec[i].relocs_ptr);
+ user_relocs = u64_to_user_ptr(exec[i].relocs_ptr);
if (copy_from_user(reloc+total, user_relocs,
exec[i].relocation_count * sizeof(*reloc))) {
@@ -910,7 +912,8 @@
goto err;
need_relocs = (args->flags & I915_EXEC_NO_RELOC) == 0;
- ret = i915_gem_execbuffer_reserve(ring, &eb->vmas, ctx, &need_relocs);
+ ret = i915_gem_execbuffer_reserve(engine, &eb->vmas, ctx,
+ &need_relocs);
if (ret)
goto err;
@@ -938,7 +941,7 @@
i915_gem_execbuffer_move_to_gpu(struct drm_i915_gem_request *req,
struct list_head *vmas)
{
- const unsigned other_rings = ~intel_ring_flag(req->ring);
+ const unsigned other_rings = ~intel_engine_flag(req->engine);
struct i915_vma *vma;
uint32_t flush_domains = 0;
bool flush_chipset = false;
@@ -948,7 +951,7 @@
struct drm_i915_gem_object *obj = vma->obj;
if (obj->active & other_rings) {
- ret = i915_gem_object_sync(obj, req->ring, &req);
+ ret = i915_gem_object_sync(obj, req->engine, &req);
if (ret)
return ret;
}
@@ -960,7 +963,7 @@
}
if (flush_chipset)
- i915_gem_chipset_flush(req->ring->dev);
+ i915_gem_chipset_flush(req->engine->dev);
if (flush_domains & I915_GEM_DOMAIN_GTT)
wmb();
@@ -1009,7 +1012,7 @@
invalid_flags |= EXEC_OBJECT_NEEDS_GTT;
for (i = 0; i < count; i++) {
- char __user *ptr = to_user_ptr(exec[i].relocs_ptr);
+ char __user *ptr = u64_to_user_ptr(exec[i].relocs_ptr);
int length; /* limited by fault_in_pages_readable() */
if (exec[i].flags & invalid_flags)
@@ -1062,12 +1065,12 @@
static struct intel_context *
i915_gem_validate_context(struct drm_device *dev, struct drm_file *file,
- struct intel_engine_cs *ring, const u32 ctx_id)
+ struct intel_engine_cs *engine, const u32 ctx_id)
{
struct intel_context *ctx = NULL;
struct i915_ctx_hang_stats *hs;
- if (ring->id != RCS && ctx_id != DEFAULT_CONTEXT_HANDLE)
+ if (engine->id != RCS && ctx_id != DEFAULT_CONTEXT_HANDLE)
return ERR_PTR(-EINVAL);
ctx = i915_gem_context_get(file->driver_priv, ctx_id);
@@ -1080,8 +1083,8 @@
return ERR_PTR(-EIO);
}
- if (i915.enable_execlists && !ctx->engine[ring->id].state) {
- int ret = intel_lr_context_deferred_alloc(ctx, ring);
+ if (i915.enable_execlists && !ctx->engine[engine->id].state) {
+ int ret = intel_lr_context_deferred_alloc(ctx, engine);
if (ret) {
DRM_DEBUG("Could not create LRC %u: %d\n", ctx_id, ret);
return ERR_PTR(ret);
@@ -1095,7 +1098,7 @@
i915_gem_execbuffer_move_to_active(struct list_head *vmas,
struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *ring = i915_gem_request_get_ring(req);
+ struct intel_engine_cs *engine = i915_gem_request_get_engine(req);
struct i915_vma *vma;
list_for_each_entry(vma, vmas, exec_list) {
@@ -1122,7 +1125,7 @@
if (entry->flags & EXEC_OBJECT_NEEDS_FENCE) {
i915_gem_request_assign(&obj->last_fenced_req, req);
if (entry->flags & __EXEC_OBJECT_HAS_FENCE) {
- struct drm_i915_private *dev_priv = to_i915(ring->dev);
+ struct drm_i915_private *dev_priv = to_i915(engine->dev);
list_move_tail(&dev_priv->fence_regs[obj->fence_reg].lru_list,
&dev_priv->mm.fence_list);
}
@@ -1132,11 +1135,11 @@
}
}
-void
+static void
i915_gem_execbuffer_retire_commands(struct i915_execbuffer_params *params)
{
/* Unconditionally force add_request to emit a full flush. */
- params->ring->gpu_caches_dirty = true;
+ params->engine->gpu_caches_dirty = true;
/* Add a breadcrumb for the completion of the batch buffer */
__i915_add_request(params->request, params->batch_obj, true);
@@ -1146,11 +1149,11 @@
i915_reset_gen7_sol_offsets(struct drm_device *dev,
struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
struct drm_i915_private *dev_priv = dev->dev_private;
int ret, i;
- if (!IS_GEN7(dev) || ring != &dev_priv->ring[RCS]) {
+ if (!IS_GEN7(dev) || engine != &dev_priv->engine[RCS]) {
DRM_DEBUG("sol reset is gen7/rcs only\n");
return -EINVAL;
}
@@ -1160,18 +1163,18 @@
return ret;
for (i = 0; i < 4; i++) {
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
- intel_ring_emit_reg(ring, GEN7_SO_WRITE_OFFSET(i));
- intel_ring_emit(ring, 0);
+ intel_ring_emit(engine, MI_LOAD_REGISTER_IMM(1));
+ intel_ring_emit_reg(engine, GEN7_SO_WRITE_OFFSET(i));
+ intel_ring_emit(engine, 0);
}
- intel_ring_advance(ring);
+ intel_ring_advance(engine);
return 0;
}
static struct drm_i915_gem_object*
-i915_gem_execbuffer_parse(struct intel_engine_cs *ring,
+i915_gem_execbuffer_parse(struct intel_engine_cs *engine,
struct drm_i915_gem_exec_object2 *shadow_exec_entry,
struct eb_vmas *eb,
struct drm_i915_gem_object *batch_obj,
@@ -1183,12 +1186,12 @@
struct i915_vma *vma;
int ret;
- shadow_batch_obj = i915_gem_batch_pool_get(&ring->batch_pool,
+ shadow_batch_obj = i915_gem_batch_pool_get(&engine->batch_pool,
PAGE_ALIGN(batch_len));
if (IS_ERR(shadow_batch_obj))
return shadow_batch_obj;
- ret = i915_parse_cmds(ring,
+ ret = i915_parse_cmds(engine,
batch_obj,
shadow_batch_obj,
batch_start_offset,
@@ -1229,7 +1232,7 @@
struct list_head *vmas)
{
struct drm_device *dev = params->dev;
- struct intel_engine_cs *ring = params->ring;
+ struct intel_engine_cs *engine = params->engine;
struct drm_i915_private *dev_priv = dev->dev_private;
u64 exec_start, exec_len;
int instp_mode;
@@ -1244,8 +1247,8 @@
if (ret)
return ret;
- WARN(params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1<<ring->id),
- "%s didn't clear reload\n", ring->name);
+ WARN(params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1<<engine->id),
+ "%s didn't clear reload\n", engine->name);
instp_mode = args->flags & I915_EXEC_CONSTANTS_MASK;
instp_mask = I915_EXEC_CONSTANTS_MASK;
@@ -1253,7 +1256,7 @@
case I915_EXEC_CONSTANTS_REL_GENERAL:
case I915_EXEC_CONSTANTS_ABSOLUTE:
case I915_EXEC_CONSTANTS_REL_SURFACE:
- if (instp_mode != 0 && ring != &dev_priv->ring[RCS]) {
+ if (instp_mode != 0 && engine != &dev_priv->engine[RCS]) {
DRM_DEBUG("non-0 rel constants mode on non-RCS\n");
return -EINVAL;
}
@@ -1280,17 +1283,17 @@
return -EINVAL;
}
- if (ring == &dev_priv->ring[RCS] &&
+ if (engine == &dev_priv->engine[RCS] &&
instp_mode != dev_priv->relative_constants_mode) {
ret = intel_ring_begin(params->request, 4);
if (ret)
return ret;
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
- intel_ring_emit_reg(ring, INSTPM);
- intel_ring_emit(ring, instp_mask << 16 | instp_mode);
- intel_ring_advance(ring);
+ intel_ring_emit(engine, MI_NOOP);
+ intel_ring_emit(engine, MI_LOAD_REGISTER_IMM(1));
+ intel_ring_emit_reg(engine, INSTPM);
+ intel_ring_emit(engine, instp_mask << 16 | instp_mode);
+ intel_ring_advance(engine);
dev_priv->relative_constants_mode = instp_mode;
}
@@ -1308,7 +1311,7 @@
if (exec_len == 0)
exec_len = params->batch_obj->base.size;
- ret = ring->dispatch_execbuffer(params->request,
+ ret = engine->dispatch_execbuffer(params->request,
exec_start, exec_len,
params->dispatch_flags);
if (ret)
@@ -1317,7 +1320,6 @@
trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
i915_gem_execbuffer_move_to_active(vmas, params->request);
- i915_gem_execbuffer_retire_commands(params);
return 0;
}
@@ -1365,7 +1367,7 @@
#define I915_USER_RINGS (4)
-static const enum intel_ring_id user_ring_map[I915_USER_RINGS + 1] = {
+static const enum intel_engine_id user_ring_map[I915_USER_RINGS + 1] = {
[I915_EXEC_DEFAULT] = RCS,
[I915_EXEC_RENDER] = RCS,
[I915_EXEC_BLT] = BCS,
@@ -1408,12 +1410,12 @@
return -EINVAL;
}
- *ring = &dev_priv->ring[_VCS(bsd_idx)];
+ *ring = &dev_priv->engine[_VCS(bsd_idx)];
} else {
- *ring = &dev_priv->ring[user_ring_map[user_ring_id]];
+ *ring = &dev_priv->engine[user_ring_map[user_ring_id]];
}
- if (!intel_ring_initialized(*ring)) {
+ if (!intel_engine_initialized(*ring)) {
DRM_DEBUG("execbuf with invalid ring: %u\n", user_ring_id);
return -EINVAL;
}
@@ -1427,12 +1429,13 @@
struct drm_i915_gem_execbuffer2 *args,
struct drm_i915_gem_exec_object2 *exec)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct drm_i915_gem_request *req = NULL;
struct eb_vmas *eb;
struct drm_i915_gem_object *batch_obj;
struct drm_i915_gem_exec_object2 shadow_exec_entry;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
struct intel_context *ctx;
struct i915_address_space *vm;
struct i915_execbuffer_params params_master; /* XXX: will be removed later */
@@ -1459,7 +1462,7 @@
if (args->flags & I915_EXEC_IS_PINNED)
dispatch_flags |= I915_DISPATCH_PINNED;
- ret = eb_select_ring(dev_priv, file, args, &ring);
+ ret = eb_select_ring(dev_priv, file, args, &engine);
if (ret)
return ret;
@@ -1473,9 +1476,9 @@
DRM_DEBUG("RS is only allowed for Haswell, Gen8 and above\n");
return -EINVAL;
}
- if (ring->id != RCS) {
+ if (engine->id != RCS) {
DRM_DEBUG("RS is not available on %s\n",
- ring->name);
+ engine->name);
return -EINVAL;
}
@@ -1488,7 +1491,7 @@
if (ret)
goto pre_mutex_err;
- ctx = i915_gem_validate_context(dev, file, ring, ctx_id);
+ ctx = i915_gem_validate_context(dev, file, engine, ctx_id);
if (IS_ERR(ctx)) {
mutex_unlock(&dev->struct_mutex);
ret = PTR_ERR(ctx);
@@ -1500,7 +1503,7 @@
if (ctx->ppgtt)
vm = &ctx->ppgtt->base;
else
- vm = &dev_priv->gtt.base;
+ vm = &ggtt->base;
memset(¶ms_master, 0x00, sizeof(params_master));
@@ -1522,7 +1525,8 @@
/* Move the objects en-masse into the GTT, evicting if necessary. */
need_relocs = (args->flags & I915_EXEC_NO_RELOC) == 0;
- ret = i915_gem_execbuffer_reserve(ring, &eb->vmas, ctx, &need_relocs);
+ ret = i915_gem_execbuffer_reserve(engine, &eb->vmas, ctx,
+ &need_relocs);
if (ret)
goto err;
@@ -1531,7 +1535,8 @@
ret = i915_gem_execbuffer_relocate(eb);
if (ret) {
if (ret == -EFAULT) {
- ret = i915_gem_execbuffer_relocate_slow(dev, args, file, ring,
+ ret = i915_gem_execbuffer_relocate_slow(dev, args, file,
+ engine,
eb, exec, ctx);
BUG_ON(!mutex_is_locked(&dev->struct_mutex));
}
@@ -1547,16 +1552,16 @@
}
params->args_batch_start_offset = args->batch_start_offset;
- if (i915_needs_cmd_parser(ring) && args->batch_len) {
+ if (i915_needs_cmd_parser(engine) && args->batch_len) {
struct drm_i915_gem_object *parsed_batch_obj;
- parsed_batch_obj = i915_gem_execbuffer_parse(ring,
- &shadow_exec_entry,
- eb,
- batch_obj,
- args->batch_start_offset,
- args->batch_len,
- file->is_master);
+ parsed_batch_obj = i915_gem_execbuffer_parse(engine,
+ &shadow_exec_entry,
+ eb,
+ batch_obj,
+ args->batch_start_offset,
+ args->batch_len,
+ file->is_master);
if (IS_ERR(parsed_batch_obj)) {
ret = PTR_ERR(parsed_batch_obj);
goto err;
@@ -1608,7 +1613,7 @@
params->batch_obj_vm_offset = i915_gem_obj_offset(batch_obj, vm);
/* Allocate a request for this batch buffer nice and early. */
- req = i915_gem_request_alloc(ring, ctx);
+ req = i915_gem_request_alloc(engine, ctx);
if (IS_ERR(req)) {
ret = PTR_ERR(req);
goto err_batch_unpin;
@@ -1616,7 +1621,7 @@
ret = i915_gem_request_add_to_client(req, file);
if (ret)
- goto err_batch_unpin;
+ goto err_request;
/*
* Save assorted stuff away to pass through to *_submission().
@@ -1626,13 +1631,15 @@
*/
params->dev = dev;
params->file = file;
- params->ring = ring;
+ params->engine = engine;
params->dispatch_flags = dispatch_flags;
params->batch_obj = batch_obj;
params->ctx = ctx;
params->request = req;
ret = dev_priv->gt.execbuf_submit(params, args, &eb->vmas);
+err_request:
+ i915_gem_execbuffer_retire_commands(params);
err_batch_unpin:
/*
@@ -1649,14 +1656,6 @@
i915_gem_context_unreference(ctx);
eb_destroy(eb);
- /*
- * If the request was created but not successfully submitted then it
- * must be freed again. If it was submitted then it is being tracked
- * on the active request list and no clean up is required here.
- */
- if (ret && !IS_ERR_OR_NULL(req))
- i915_gem_request_cancel(req);
-
mutex_unlock(&dev->struct_mutex);
pre_mutex_err:
@@ -1696,7 +1695,7 @@
return -ENOMEM;
}
ret = copy_from_user(exec_list,
- to_user_ptr(args->buffers_ptr),
+ u64_to_user_ptr(args->buffers_ptr),
sizeof(*exec_list) * args->buffer_count);
if (ret != 0) {
DRM_DEBUG("copy %d exec entries failed %d\n",
@@ -1732,7 +1731,7 @@
ret = i915_gem_do_execbuffer(dev, data, file, &exec2, exec2_list);
if (!ret) {
struct drm_i915_gem_exec_object __user *user_exec_list =
- to_user_ptr(args->buffers_ptr);
+ u64_to_user_ptr(args->buffers_ptr);
/* Copy the new buffer offsets back to the user's exec list. */
for (i = 0; i < args->buffer_count; i++) {
@@ -1775,18 +1774,16 @@
return -EINVAL;
}
- exec2_list = kmalloc(sizeof(*exec2_list)*args->buffer_count,
- GFP_TEMPORARY | __GFP_NOWARN | __GFP_NORETRY);
- if (exec2_list == NULL)
- exec2_list = drm_malloc_ab(sizeof(*exec2_list),
- args->buffer_count);
+ exec2_list = drm_malloc_gfp(args->buffer_count,
+ sizeof(*exec2_list),
+ GFP_TEMPORARY);
if (exec2_list == NULL) {
DRM_DEBUG("Failed to allocate exec list for %d buffers\n",
args->buffer_count);
return -ENOMEM;
}
ret = copy_from_user(exec2_list,
- to_user_ptr(args->buffers_ptr),
+ u64_to_user_ptr(args->buffers_ptr),
sizeof(*exec2_list) * args->buffer_count);
if (ret != 0) {
DRM_DEBUG("copy %d exec entries failed %d\n",
@@ -1799,7 +1796,7 @@
if (!ret) {
/* Copy the new buffer offsets back to the user's exec list. */
struct drm_i915_gem_exec_object2 __user *user_exec_list =
- to_user_ptr(args->buffers_ptr);
+ u64_to_user_ptr(args->buffers_ptr);
int i;
for (i = 0; i < args->buffer_count; i++) {
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
index 49e4f26..0d666b3 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
@@ -658,7 +658,7 @@
unsigned entry,
dma_addr_t addr)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
int ret;
BUG_ON(entry >= 4);
@@ -667,13 +667,13 @@
if (ret)
return ret;
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
- intel_ring_emit_reg(ring, GEN8_RING_PDP_UDW(ring, entry));
- intel_ring_emit(ring, upper_32_bits(addr));
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
- intel_ring_emit_reg(ring, GEN8_RING_PDP_LDW(ring, entry));
- intel_ring_emit(ring, lower_32_bits(addr));
- intel_ring_advance(ring);
+ intel_ring_emit(engine, MI_LOAD_REGISTER_IMM(1));
+ intel_ring_emit_reg(engine, GEN8_RING_PDP_UDW(engine, entry));
+ intel_ring_emit(engine, upper_32_bits(addr));
+ intel_ring_emit(engine, MI_LOAD_REGISTER_IMM(1));
+ intel_ring_emit_reg(engine, GEN8_RING_PDP_LDW(engine, entry));
+ intel_ring_emit(engine, lower_32_bits(addr));
+ intel_ring_advance(engine);
return 0;
}
@@ -706,8 +706,7 @@
uint64_t length,
gen8_pte_t scratch_pte)
{
- struct i915_hw_ppgtt *ppgtt =
- container_of(vm, struct i915_hw_ppgtt, base);
+ struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
gen8_pte_t *pt_vaddr;
unsigned pdpe = gen8_pdpe_index(start);
unsigned pde = gen8_pde_index(start);
@@ -746,7 +745,7 @@
num_entries--;
}
- kunmap_px(ppgtt, pt);
+ kunmap_px(ppgtt, pt_vaddr);
pte = 0;
if (++pde == I915_PDES) {
@@ -762,8 +761,7 @@
uint64_t length,
bool use_scratch)
{
- struct i915_hw_ppgtt *ppgtt =
- container_of(vm, struct i915_hw_ppgtt, base);
+ struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
gen8_pte_t scratch_pte = gen8_pte_encode(px_dma(vm->scratch_page),
I915_CACHE_LLC, use_scratch);
@@ -788,8 +786,7 @@
uint64_t start,
enum i915_cache_level cache_level)
{
- struct i915_hw_ppgtt *ppgtt =
- container_of(vm, struct i915_hw_ppgtt, base);
+ struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
gen8_pte_t *pt_vaddr;
unsigned pdpe = gen8_pdpe_index(start);
unsigned pde = gen8_pde_index(start);
@@ -829,8 +826,7 @@
enum i915_cache_level cache_level,
u32 unused)
{
- struct i915_hw_ppgtt *ppgtt =
- container_of(vm, struct i915_hw_ppgtt, base);
+ struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
struct sg_page_iter sg_iter;
__sg_page_iter_start(&sg_iter, pages->sgl, sg_nents(pages->sgl), 0);
@@ -909,11 +905,10 @@
static int gen8_ppgtt_notify_vgt(struct i915_hw_ppgtt *ppgtt, bool create)
{
enum vgt_g2v_type msg;
- struct drm_device *dev = ppgtt->base.dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(ppgtt->base.dev);
int i;
- if (USES_FULL_48BIT_PPGTT(dev)) {
+ if (USES_FULL_48BIT_PPGTT(dev_priv)) {
u64 daddr = px_dma(&ppgtt->pml4);
I915_WRITE(vgtif_reg(pdp[0].lo), lower_32_bits(daddr));
@@ -981,8 +976,7 @@
static void gen8_ppgtt_cleanup(struct i915_address_space *vm)
{
- struct i915_hw_ppgtt *ppgtt =
- container_of(vm, struct i915_hw_ppgtt, base);
+ struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
if (intel_vgpu_active(vm->dev))
gen8_ppgtt_notify_vgt(ppgtt, false);
@@ -1216,8 +1210,7 @@
uint64_t start,
uint64_t length)
{
- struct i915_hw_ppgtt *ppgtt =
- container_of(vm, struct i915_hw_ppgtt, base);
+ struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
unsigned long *new_page_dirs, *new_page_tables;
struct drm_device *dev = vm->dev;
struct i915_page_directory *pd;
@@ -1329,8 +1322,7 @@
uint64_t length)
{
DECLARE_BITMAP(new_pdps, GEN8_PML4ES_PER_PML4);
- struct i915_hw_ppgtt *ppgtt =
- container_of(vm, struct i915_hw_ppgtt, base);
+ struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
struct i915_page_directory_pointer *pdp;
uint64_t pml4e;
int ret = 0;
@@ -1376,8 +1368,7 @@
static int gen8_alloc_va_range(struct i915_address_space *vm,
uint64_t start, uint64_t length)
{
- struct i915_hw_ppgtt *ppgtt =
- container_of(vm, struct i915_hw_ppgtt, base);
+ struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
if (USES_FULL_48BIT_PPGTT(vm->dev))
return gen8_alloc_va_range_4lvl(vm, &ppgtt->pml4, start, length);
@@ -1629,6 +1620,7 @@
struct i915_page_directory *pd,
uint32_t start, uint32_t length)
{
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct i915_page_table *pt;
uint32_t pde, temp;
@@ -1637,7 +1629,7 @@
/* Make sure write is complete before other code can use this page
* table. Also require for WC mapped PTEs */
- readl(dev_priv->gtt.gsm);
+ readl(ggtt->gsm);
}
static uint32_t get_pd_offset(struct i915_hw_ppgtt *ppgtt)
@@ -1650,11 +1642,11 @@
static int hsw_mm_switch(struct i915_hw_ppgtt *ppgtt,
struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
int ret;
/* NB: TLBs must be flushed and invalidated before a switch */
- ret = ring->flush(req, I915_GEM_GPU_DOMAINS, I915_GEM_GPU_DOMAINS);
+ ret = engine->flush(req, I915_GEM_GPU_DOMAINS, I915_GEM_GPU_DOMAINS);
if (ret)
return ret;
@@ -1662,13 +1654,13 @@
if (ret)
return ret;
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(2));
- intel_ring_emit_reg(ring, RING_PP_DIR_DCLV(ring));
- intel_ring_emit(ring, PP_DIR_DCLV_2G);
- intel_ring_emit_reg(ring, RING_PP_DIR_BASE(ring));
- intel_ring_emit(ring, get_pd_offset(ppgtt));
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ intel_ring_emit(engine, MI_LOAD_REGISTER_IMM(2));
+ intel_ring_emit_reg(engine, RING_PP_DIR_DCLV(engine));
+ intel_ring_emit(engine, PP_DIR_DCLV_2G);
+ intel_ring_emit_reg(engine, RING_PP_DIR_BASE(engine));
+ intel_ring_emit(engine, get_pd_offset(ppgtt));
+ intel_ring_emit(engine, MI_NOOP);
+ intel_ring_advance(engine);
return 0;
}
@@ -1676,22 +1668,22 @@
static int vgpu_mm_switch(struct i915_hw_ppgtt *ppgtt,
struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
struct drm_i915_private *dev_priv = to_i915(ppgtt->base.dev);
- I915_WRITE(RING_PP_DIR_DCLV(ring), PP_DIR_DCLV_2G);
- I915_WRITE(RING_PP_DIR_BASE(ring), get_pd_offset(ppgtt));
+ I915_WRITE(RING_PP_DIR_DCLV(engine), PP_DIR_DCLV_2G);
+ I915_WRITE(RING_PP_DIR_BASE(engine), get_pd_offset(ppgtt));
return 0;
}
static int gen7_mm_switch(struct i915_hw_ppgtt *ppgtt,
struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
int ret;
/* NB: TLBs must be flushed and invalidated before a switch */
- ret = ring->flush(req, I915_GEM_GPU_DOMAINS, I915_GEM_GPU_DOMAINS);
+ ret = engine->flush(req, I915_GEM_GPU_DOMAINS, I915_GEM_GPU_DOMAINS);
if (ret)
return ret;
@@ -1699,17 +1691,17 @@
if (ret)
return ret;
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(2));
- intel_ring_emit_reg(ring, RING_PP_DIR_DCLV(ring));
- intel_ring_emit(ring, PP_DIR_DCLV_2G);
- intel_ring_emit_reg(ring, RING_PP_DIR_BASE(ring));
- intel_ring_emit(ring, get_pd_offset(ppgtt));
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ intel_ring_emit(engine, MI_LOAD_REGISTER_IMM(2));
+ intel_ring_emit_reg(engine, RING_PP_DIR_DCLV(engine));
+ intel_ring_emit(engine, PP_DIR_DCLV_2G);
+ intel_ring_emit_reg(engine, RING_PP_DIR_BASE(engine));
+ intel_ring_emit(engine, get_pd_offset(ppgtt));
+ intel_ring_emit(engine, MI_NOOP);
+ intel_ring_advance(engine);
/* XXX: RCS is the only one to auto invalidate the TLBs? */
- if (ring->id != RCS) {
- ret = ring->flush(req, I915_GEM_GPU_DOMAINS, I915_GEM_GPU_DOMAINS);
+ if (engine->id != RCS) {
+ ret = engine->flush(req, I915_GEM_GPU_DOMAINS, I915_GEM_GPU_DOMAINS);
if (ret)
return ret;
}
@@ -1720,15 +1712,15 @@
static int gen6_mm_switch(struct i915_hw_ppgtt *ppgtt,
struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
struct drm_device *dev = ppgtt->base.dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- I915_WRITE(RING_PP_DIR_DCLV(ring), PP_DIR_DCLV_2G);
- I915_WRITE(RING_PP_DIR_BASE(ring), get_pd_offset(ppgtt));
+ I915_WRITE(RING_PP_DIR_DCLV(engine), PP_DIR_DCLV_2G);
+ I915_WRITE(RING_PP_DIR_BASE(engine), get_pd_offset(ppgtt));
- POSTING_READ(RING_PP_DIR_DCLV(ring));
+ POSTING_READ(RING_PP_DIR_DCLV(engine));
return 0;
}
@@ -1736,12 +1728,11 @@
static void gen8_ppgtt_enable(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
- int j;
+ struct intel_engine_cs *engine;
- for_each_ring(ring, dev_priv, j) {
+ for_each_engine(engine, dev_priv) {
u32 four_level = USES_FULL_48BIT_PPGTT(dev) ? GEN8_GFX_PPGTT_48B : 0;
- I915_WRITE(RING_MODE_GEN7(ring),
+ I915_WRITE(RING_MODE_GEN7(engine),
_MASKED_BIT_ENABLE(GFX_PPGTT_ENABLE | four_level));
}
}
@@ -1749,9 +1740,8 @@
static void gen7_ppgtt_enable(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
uint32_t ecochk, ecobits;
- int i;
ecobits = I915_READ(GAC_ECO_BITS);
I915_WRITE(GAC_ECO_BITS, ecobits | ECOBITS_PPGTT_CACHE64B);
@@ -1765,9 +1755,9 @@
}
I915_WRITE(GAM_ECOCHK, ecochk);
- for_each_ring(ring, dev_priv, i) {
+ for_each_engine(engine, dev_priv) {
/* GFX_MODE is per-ring on gen7+ */
- I915_WRITE(RING_MODE_GEN7(ring),
+ I915_WRITE(RING_MODE_GEN7(engine),
_MASKED_BIT_ENABLE(GFX_PPGTT_ENABLE));
}
}
@@ -1796,8 +1786,7 @@
uint64_t length,
bool use_scratch)
{
- struct i915_hw_ppgtt *ppgtt =
- container_of(vm, struct i915_hw_ppgtt, base);
+ struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
gen6_pte_t *pt_vaddr, scratch_pte;
unsigned first_entry = start >> PAGE_SHIFT;
unsigned num_entries = length >> PAGE_SHIFT;
@@ -1831,8 +1820,7 @@
uint64_t start,
enum i915_cache_level cache_level, u32 flags)
{
- struct i915_hw_ppgtt *ppgtt =
- container_of(vm, struct i915_hw_ppgtt, base);
+ struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
gen6_pte_t *pt_vaddr;
unsigned first_entry = start >> PAGE_SHIFT;
unsigned act_pt = first_entry / GEN6_PTES;
@@ -1864,9 +1852,9 @@
{
DECLARE_BITMAP(new_page_tables, I915_PDES);
struct drm_device *dev = vm->dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
- struct i915_hw_ppgtt *ppgtt =
- container_of(vm, struct i915_hw_ppgtt, base);
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
+ struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
struct i915_page_table *pt;
uint32_t start, length, start_save, length_save;
uint32_t pde, temp;
@@ -1932,7 +1920,7 @@
/* Make sure write is complete before other code can use this page
* table. Also require for WC mapped PTEs */
- readl(dev_priv->gtt.gsm);
+ readl(ggtt->gsm);
mark_tlbs_dirty(ppgtt);
return 0;
@@ -1978,8 +1966,7 @@
static void gen6_ppgtt_cleanup(struct i915_address_space *vm)
{
- struct i915_hw_ppgtt *ppgtt =
- container_of(vm, struct i915_hw_ppgtt, base);
+ struct i915_hw_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
struct i915_page_table *pt;
uint32_t pde;
@@ -1997,7 +1984,8 @@
{
struct i915_address_space *vm = &ppgtt->base;
struct drm_device *dev = ppgtt->base.dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
bool retried = false;
int ret;
@@ -2005,23 +1993,23 @@
* allocator works in address space sizes, so it's multiplied by page
* size. We allocate at the top of the GTT to avoid fragmentation.
*/
- BUG_ON(!drm_mm_initialized(&dev_priv->gtt.base.mm));
+ BUG_ON(!drm_mm_initialized(&ggtt->base.mm));
ret = gen6_init_scratch(vm);
if (ret)
return ret;
alloc:
- ret = drm_mm_insert_node_in_range_generic(&dev_priv->gtt.base.mm,
+ ret = drm_mm_insert_node_in_range_generic(&ggtt->base.mm,
&ppgtt->node, GEN6_PD_SIZE,
GEN6_PD_ALIGN, 0,
- 0, dev_priv->gtt.base.total,
+ 0, ggtt->base.total,
DRM_MM_TOPDOWN);
if (ret == -ENOSPC && !retried) {
- ret = i915_gem_evict_something(dev, &dev_priv->gtt.base,
+ ret = i915_gem_evict_something(dev, &ggtt->base,
GEN6_PD_SIZE, GEN6_PD_ALIGN,
I915_CACHE_NONE,
- 0, dev_priv->gtt.base.total,
+ 0, ggtt->base.total,
0);
if (ret)
goto err_out;
@@ -2034,7 +2022,7 @@
goto err_out;
- if (ppgtt->node.start < dev_priv->gtt.mappable_end)
+ if (ppgtt->node.start < ggtt->mappable_end)
DRM_DEBUG("Forced to use aperture for PDEs\n");
return 0;
@@ -2062,10 +2050,11 @@
static int gen6_ppgtt_init(struct i915_hw_ppgtt *ppgtt)
{
struct drm_device *dev = ppgtt->base.dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
int ret;
- ppgtt->base.pte_encode = dev_priv->gtt.base.pte_encode;
+ ppgtt->base.pte_encode = ggtt->base.pte_encode;
if (IS_GEN6(dev)) {
ppgtt->switch_mm = gen6_mm_switch;
} else if (IS_HASWELL(dev)) {
@@ -2095,7 +2084,7 @@
ppgtt->pd.base.ggtt_offset =
ppgtt->node.start / PAGE_SIZE * sizeof(gen6_pte_t);
- ppgtt->pd_addr = (gen6_pte_t __iomem *)dev_priv->gtt.gsm +
+ ppgtt->pd_addr = (gen6_pte_t __iomem *)ggtt->gsm +
ppgtt->pd.base.ggtt_offset / sizeof(gen6_pte_t);
gen6_scratch_va_range(ppgtt, 0, ppgtt->base.total);
@@ -2192,7 +2181,7 @@
int i915_ppgtt_init_ring(struct drm_i915_gem_request *req)
{
- struct drm_i915_private *dev_priv = req->ring->dev->dev_private;
+ struct drm_i915_private *dev_priv = req->i915;
struct i915_hw_ppgtt *ppgtt = dev_priv->mm.aliasing_ppgtt;
if (i915.enable_execlists)
@@ -2263,9 +2252,10 @@
static bool do_idling(struct drm_i915_private *dev_priv)
{
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
bool ret = dev_priv->mm.interruptible;
- if (unlikely(dev_priv->gtt.do_idle_maps)) {
+ if (unlikely(ggtt->do_idle_maps)) {
dev_priv->mm.interruptible = false;
if (i915_gpu_idle(dev_priv->dev)) {
DRM_ERROR("Couldn't idle GPU\n");
@@ -2279,22 +2269,23 @@
static void undo_idling(struct drm_i915_private *dev_priv, bool interruptible)
{
- if (unlikely(dev_priv->gtt.do_idle_maps))
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
+
+ if (unlikely(ggtt->do_idle_maps))
dev_priv->mm.interruptible = interruptible;
}
void i915_check_and_clear_faults(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
- int i;
+ struct intel_engine_cs *engine;
if (INTEL_INFO(dev)->gen < 6)
return;
- for_each_ring(ring, dev_priv, i) {
+ for_each_engine(engine, dev_priv) {
u32 fault_reg;
- fault_reg = I915_READ(RING_FAULT_REG(ring));
+ fault_reg = I915_READ(RING_FAULT_REG(engine));
if (fault_reg & RING_FAULT_VALID) {
DRM_DEBUG_DRIVER("Unexpected fault\n"
"\tAddr: 0x%08lx\n"
@@ -2305,16 +2296,16 @@
fault_reg & RING_FAULT_GTTSEL_MASK ? "GGTT" : "PPGTT",
RING_FAULT_SRCID(fault_reg),
RING_FAULT_FAULT_TYPE(fault_reg));
- I915_WRITE(RING_FAULT_REG(ring),
+ I915_WRITE(RING_FAULT_REG(engine),
fault_reg & ~RING_FAULT_VALID);
}
}
- POSTING_READ(RING_FAULT_REG(&dev_priv->ring[RCS]));
+ POSTING_READ(RING_FAULT_REG(&dev_priv->engine[RCS]));
}
static void i915_ggtt_flush(struct drm_i915_private *dev_priv)
{
- if (INTEL_INFO(dev_priv->dev)->gen < 6) {
+ if (INTEL_INFO(dev_priv)->gen < 6) {
intel_gtt_chipset_flush();
} else {
I915_WRITE(GFX_FLSH_CNTL_GEN6, GFX_FLSH_CNTL_EN);
@@ -2324,7 +2315,8 @@
void i915_gem_suspend_gtt_mappings(struct drm_device *dev)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
/* Don't bother messing with faults pre GEN6 as we have little
* documentation supporting that it's a good idea.
@@ -2334,10 +2326,8 @@
i915_check_and_clear_faults(dev);
- dev_priv->gtt.base.clear_range(&dev_priv->gtt.base,
- dev_priv->gtt.base.start,
- dev_priv->gtt.base.total,
- true);
+ ggtt->base.clear_range(&ggtt->base, ggtt->base.start, ggtt->base.total,
+ true);
i915_ggtt_flush(dev_priv);
}
@@ -2367,10 +2357,11 @@
uint64_t start,
enum i915_cache_level level, u32 unused)
{
- struct drm_i915_private *dev_priv = vm->dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(vm->dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
unsigned first_entry = start >> PAGE_SHIFT;
gen8_pte_t __iomem *gtt_entries =
- (gen8_pte_t __iomem *)dev_priv->gtt.gsm + first_entry;
+ (gen8_pte_t __iomem *)ggtt->gsm + first_entry;
int i = 0;
struct sg_page_iter sg_iter;
dma_addr_t addr = 0; /* shut up gcc */
@@ -2444,10 +2435,11 @@
uint64_t start,
enum i915_cache_level level, u32 flags)
{
- struct drm_i915_private *dev_priv = vm->dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(vm->dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
unsigned first_entry = start >> PAGE_SHIFT;
gen6_pte_t __iomem *gtt_entries =
- (gen6_pte_t __iomem *)dev_priv->gtt.gsm + first_entry;
+ (gen6_pte_t __iomem *)ggtt->gsm + first_entry;
int i = 0;
struct sg_page_iter sg_iter;
dma_addr_t addr = 0;
@@ -2487,12 +2479,13 @@
uint64_t length,
bool use_scratch)
{
- struct drm_i915_private *dev_priv = vm->dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(vm->dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
unsigned first_entry = start >> PAGE_SHIFT;
unsigned num_entries = length >> PAGE_SHIFT;
gen8_pte_t scratch_pte, __iomem *gtt_base =
- (gen8_pte_t __iomem *) dev_priv->gtt.gsm + first_entry;
- const int max_entries = gtt_total_entries(dev_priv->gtt) - first_entry;
+ (gen8_pte_t __iomem *)ggtt->gsm + first_entry;
+ const int max_entries = ggtt_total_entries(ggtt) - first_entry;
int i;
int rpm_atomic_seq;
@@ -2518,12 +2511,13 @@
uint64_t length,
bool use_scratch)
{
- struct drm_i915_private *dev_priv = vm->dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(vm->dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
unsigned first_entry = start >> PAGE_SHIFT;
unsigned num_entries = length >> PAGE_SHIFT;
gen6_pte_t scratch_pte, __iomem *gtt_base =
- (gen6_pte_t __iomem *) dev_priv->gtt.gsm + first_entry;
- const int max_entries = gtt_total_entries(dev_priv->gtt) - first_entry;
+ (gen6_pte_t __iomem *)ggtt->gsm + first_entry;
+ const int max_entries = ggtt_total_entries(ggtt) - first_entry;
int i;
int rpm_atomic_seq;
@@ -2613,32 +2607,31 @@
enum i915_cache_level cache_level,
u32 flags)
{
- struct drm_device *dev = vma->vm->dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
- struct drm_i915_gem_object *obj = vma->obj;
- struct sg_table *pages = obj->pages;
- u32 pte_flags = 0;
+ u32 pte_flags;
int ret;
ret = i915_get_ggtt_vma_pages(vma);
if (ret)
return ret;
- pages = vma->ggtt_view.pages;
/* Currently applicable only to VLV */
- if (obj->gt_ro)
+ pte_flags = 0;
+ if (vma->obj->gt_ro)
pte_flags |= PTE_READ_ONLY;
if (flags & GLOBAL_BIND) {
- vma->vm->insert_entries(vma->vm, pages,
+ vma->vm->insert_entries(vma->vm,
+ vma->ggtt_view.pages,
vma->node.start,
cache_level, pte_flags);
}
if (flags & LOCAL_BIND) {
- struct i915_hw_ppgtt *appgtt = dev_priv->mm.aliasing_ppgtt;
- appgtt->base.insert_entries(&appgtt->base, pages,
+ struct i915_hw_ppgtt *appgtt =
+ to_i915(vma->vm->dev)->mm.aliasing_ppgtt;
+ appgtt->base.insert_entries(&appgtt->base,
+ vma->ggtt_view.pages,
vma->node.start,
cache_level, pte_flags);
}
@@ -2717,8 +2710,8 @@
* aperture. One page should be enough to keep any prefetching inside
* of the aperture.
*/
- struct drm_i915_private *dev_priv = dev->dev_private;
- struct i915_address_space *ggtt_vm = &dev_priv->gtt.base;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct drm_mm_node *entry;
struct drm_i915_gem_object *obj;
unsigned long hole_start, hole_end;
@@ -2726,13 +2719,13 @@
BUG_ON(mappable_end > end);
- ggtt_vm->start = start;
+ ggtt->base.start = start;
/* Subtract the guard page before address space initialization to
* shrink the range used by drm_mm */
- ggtt_vm->total = end - start - PAGE_SIZE;
- i915_address_space_init(ggtt_vm, dev_priv);
- ggtt_vm->total += PAGE_SIZE;
+ ggtt->base.total = end - start - PAGE_SIZE;
+ i915_address_space_init(&ggtt->base, dev_priv);
+ ggtt->base.total += PAGE_SIZE;
if (intel_vgpu_active(dev)) {
ret = intel_vgt_balloon(dev);
@@ -2741,36 +2734,36 @@
}
if (!HAS_LLC(dev))
- ggtt_vm->mm.color_adjust = i915_gtt_color_adjust;
+ ggtt->base.mm.color_adjust = i915_gtt_color_adjust;
/* Mark any preallocated objects as occupied */
list_for_each_entry(obj, &dev_priv->mm.bound_list, global_list) {
- struct i915_vma *vma = i915_gem_obj_to_vma(obj, ggtt_vm);
+ struct i915_vma *vma = i915_gem_obj_to_vma(obj, &ggtt->base);
DRM_DEBUG_KMS("reserving preallocated space: %llx + %zx\n",
i915_gem_obj_ggtt_offset(obj), obj->base.size);
WARN_ON(i915_gem_obj_ggtt_bound(obj));
- ret = drm_mm_reserve_node(&ggtt_vm->mm, &vma->node);
+ ret = drm_mm_reserve_node(&ggtt->base.mm, &vma->node);
if (ret) {
DRM_DEBUG_KMS("Reservation failed: %i\n", ret);
return ret;
}
vma->bound |= GLOBAL_BIND;
__i915_vma_set_map_and_fenceable(vma);
- list_add_tail(&vma->vm_link, &ggtt_vm->inactive_list);
+ list_add_tail(&vma->vm_link, &ggtt->base.inactive_list);
}
/* Clear any non-preallocated blocks */
- drm_mm_for_each_hole(entry, &ggtt_vm->mm, hole_start, hole_end) {
+ drm_mm_for_each_hole(entry, &ggtt->base.mm, hole_start, hole_end) {
DRM_DEBUG_KMS("clearing unused GTT space: [%lx, %lx]\n",
hole_start, hole_end);
- ggtt_vm->clear_range(ggtt_vm, hole_start,
+ ggtt->base.clear_range(&ggtt->base, hole_start,
hole_end - hole_start, true);
}
/* And finally clear the reserved guard page */
- ggtt_vm->clear_range(ggtt_vm, end - PAGE_SIZE, PAGE_SIZE, true);
+ ggtt->base.clear_range(&ggtt->base, end - PAGE_SIZE, PAGE_SIZE, true);
if (USES_PPGTT(dev) && !USES_FULL_PPGTT(dev)) {
struct i915_hw_ppgtt *ppgtt;
@@ -2801,28 +2794,33 @@
true);
dev_priv->mm.aliasing_ppgtt = ppgtt;
- WARN_ON(dev_priv->gtt.base.bind_vma != ggtt_bind_vma);
- dev_priv->gtt.base.bind_vma = aliasing_gtt_bind_vma;
+ WARN_ON(ggtt->base.bind_vma != ggtt_bind_vma);
+ ggtt->base.bind_vma = aliasing_gtt_bind_vma;
}
return 0;
}
-void i915_gem_init_global_gtt(struct drm_device *dev)
+/**
+ * i915_gem_init_ggtt - Initialize GEM for Global GTT
+ * @dev: DRM device
+ */
+void i915_gem_init_ggtt(struct drm_device *dev)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
- u64 gtt_size, mappable_size;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
- gtt_size = dev_priv->gtt.base.total;
- mappable_size = dev_priv->gtt.mappable_end;
-
- i915_gem_setup_global_gtt(dev, 0, mappable_size, gtt_size);
+ i915_gem_setup_global_gtt(dev, 0, ggtt->mappable_end, ggtt->base.total);
}
-void i915_global_gtt_cleanup(struct drm_device *dev)
+/**
+ * i915_ggtt_cleanup_hw - Clean up GGTT hardware initialization
+ * @dev: DRM device
+ */
+void i915_ggtt_cleanup_hw(struct drm_device *dev)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
- struct i915_address_space *vm = &dev_priv->gtt.base;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
if (dev_priv->mm.aliasing_ppgtt) {
struct i915_hw_ppgtt *ppgtt = dev_priv->mm.aliasing_ppgtt;
@@ -2832,15 +2830,15 @@
i915_gem_cleanup_stolen(dev);
- if (drm_mm_initialized(&vm->mm)) {
+ if (drm_mm_initialized(&ggtt->base.mm)) {
if (intel_vgpu_active(dev))
intel_vgt_deballoon();
- drm_mm_takedown(&vm->mm);
- list_del(&vm->global_link);
+ drm_mm_takedown(&ggtt->base.mm);
+ list_del(&ggtt->base.global_link);
}
- vm->cleanup(vm);
+ ggtt->base.cleanup(&ggtt->base);
}
static unsigned int gen6_get_total_gtt_size(u16 snb_gmch_ctl)
@@ -2924,13 +2922,14 @@
static int ggtt_probe_common(struct drm_device *dev,
size_t gtt_size)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct i915_page_scratch *scratch_page;
- phys_addr_t gtt_phys_addr;
+ phys_addr_t ggtt_phys_addr;
/* For Modern GENs the PTEs and register space are split in the BAR */
- gtt_phys_addr = pci_resource_start(dev->pdev, 0) +
- (pci_resource_len(dev->pdev, 0) / 2);
+ ggtt_phys_addr = pci_resource_start(dev->pdev, 0) +
+ (pci_resource_len(dev->pdev, 0) / 2);
/*
* On BXT writes larger than 64 bit to the GTT pagetable range will be
@@ -2940,10 +2939,10 @@
* readback check when writing GTT PTE entries.
*/
if (IS_BROXTON(dev))
- dev_priv->gtt.gsm = ioremap_nocache(gtt_phys_addr, gtt_size);
+ ggtt->gsm = ioremap_nocache(ggtt_phys_addr, gtt_size);
else
- dev_priv->gtt.gsm = ioremap_wc(gtt_phys_addr, gtt_size);
- if (!dev_priv->gtt.gsm) {
+ ggtt->gsm = ioremap_wc(ggtt_phys_addr, gtt_size);
+ if (!ggtt->gsm) {
DRM_ERROR("Failed to map the gtt page table\n");
return -ENOMEM;
}
@@ -2952,11 +2951,11 @@
if (IS_ERR(scratch_page)) {
DRM_ERROR("Scratch setup failed\n");
/* iounmap will also get called at remove, but meh */
- iounmap(dev_priv->gtt.gsm);
+ iounmap(ggtt->gsm);
return PTR_ERR(scratch_page);
}
- dev_priv->gtt.base.scratch_page = scratch_page;
+ ggtt->base.scratch_page = scratch_page;
return 0;
}
@@ -2977,7 +2976,7 @@
GEN8_PPAT(6, GEN8_PPAT_WB | GEN8_PPAT_LLCELLC | GEN8_PPAT_AGE(2)) |
GEN8_PPAT(7, GEN8_PPAT_WB | GEN8_PPAT_LLCELLC | GEN8_PPAT_AGE(3));
- if (!USES_PPGTT(dev_priv->dev))
+ if (!USES_PPGTT(dev_priv))
/* Spec: "For GGTT, there is NO pat_sel[2:0] from the entry,
* so RTL will always use the value corresponding to
* pat_sel = 000".
@@ -3034,20 +3033,16 @@
I915_WRITE(GEN8_PRIVATE_PAT_HI, pat >> 32);
}
-static int gen8_gmch_probe(struct drm_device *dev,
- u64 *gtt_total,
- size_t *stolen,
- phys_addr_t *mappable_base,
- u64 *mappable_end)
+static int gen8_gmch_probe(struct i915_ggtt *ggtt)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
- u64 gtt_size;
+ struct drm_device *dev = ggtt->base.dev;
+ struct drm_i915_private *dev_priv = to_i915(dev);
u16 snb_gmch_ctl;
int ret;
/* TODO: We're not aware of mappable constraints on gen8 yet */
- *mappable_base = pci_resource_start(dev->pdev, 2);
- *mappable_end = pci_resource_len(dev->pdev, 2);
+ ggtt->mappable_base = pci_resource_start(dev->pdev, 2);
+ ggtt->mappable_end = pci_resource_len(dev->pdev, 2);
if (!pci_set_dma_mask(dev->pdev, DMA_BIT_MASK(39)))
pci_set_consistent_dma_mask(dev->pdev, DMA_BIT_MASK(39));
@@ -3055,56 +3050,50 @@
pci_read_config_word(dev->pdev, SNB_GMCH_CTRL, &snb_gmch_ctl);
if (INTEL_INFO(dev)->gen >= 9) {
- *stolen = gen9_get_stolen_size(snb_gmch_ctl);
- gtt_size = gen8_get_total_gtt_size(snb_gmch_ctl);
+ ggtt->stolen_size = gen9_get_stolen_size(snb_gmch_ctl);
+ ggtt->size = gen8_get_total_gtt_size(snb_gmch_ctl);
} else if (IS_CHERRYVIEW(dev)) {
- *stolen = chv_get_stolen_size(snb_gmch_ctl);
- gtt_size = chv_get_total_gtt_size(snb_gmch_ctl);
+ ggtt->stolen_size = chv_get_stolen_size(snb_gmch_ctl);
+ ggtt->size = chv_get_total_gtt_size(snb_gmch_ctl);
} else {
- *stolen = gen8_get_stolen_size(snb_gmch_ctl);
- gtt_size = gen8_get_total_gtt_size(snb_gmch_ctl);
+ ggtt->stolen_size = gen8_get_stolen_size(snb_gmch_ctl);
+ ggtt->size = gen8_get_total_gtt_size(snb_gmch_ctl);
}
- *gtt_total = (gtt_size / sizeof(gen8_pte_t)) << PAGE_SHIFT;
+ ggtt->base.total = (ggtt->size / sizeof(gen8_pte_t)) << PAGE_SHIFT;
if (IS_CHERRYVIEW(dev) || IS_BROXTON(dev))
chv_setup_private_ppat(dev_priv);
else
bdw_setup_private_ppat(dev_priv);
- ret = ggtt_probe_common(dev, gtt_size);
+ ret = ggtt_probe_common(dev, ggtt->size);
- dev_priv->gtt.base.clear_range = gen8_ggtt_clear_range;
- dev_priv->gtt.base.insert_entries = gen8_ggtt_insert_entries;
- dev_priv->gtt.base.bind_vma = ggtt_bind_vma;
- dev_priv->gtt.base.unbind_vma = ggtt_unbind_vma;
-
+ ggtt->base.clear_range = gen8_ggtt_clear_range;
if (IS_CHERRYVIEW(dev_priv))
- dev_priv->gtt.base.insert_entries = gen8_ggtt_insert_entries__BKL;
+ ggtt->base.insert_entries = gen8_ggtt_insert_entries__BKL;
+ else
+ ggtt->base.insert_entries = gen8_ggtt_insert_entries;
+ ggtt->base.bind_vma = ggtt_bind_vma;
+ ggtt->base.unbind_vma = ggtt_unbind_vma;
return ret;
}
-static int gen6_gmch_probe(struct drm_device *dev,
- u64 *gtt_total,
- size_t *stolen,
- phys_addr_t *mappable_base,
- u64 *mappable_end)
+static int gen6_gmch_probe(struct i915_ggtt *ggtt)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
- unsigned int gtt_size;
+ struct drm_device *dev = ggtt->base.dev;
u16 snb_gmch_ctl;
int ret;
- *mappable_base = pci_resource_start(dev->pdev, 2);
- *mappable_end = pci_resource_len(dev->pdev, 2);
+ ggtt->mappable_base = pci_resource_start(dev->pdev, 2);
+ ggtt->mappable_end = pci_resource_len(dev->pdev, 2);
/* 64/512MB is the current min/max we actually know of, but this is just
* a coarse sanity check.
*/
- if ((*mappable_end < (64<<20) || (*mappable_end > (512<<20)))) {
- DRM_ERROR("Unknown GMADR size (%llx)\n",
- dev_priv->gtt.mappable_end);
+ if ((ggtt->mappable_end < (64<<20) || (ggtt->mappable_end > (512<<20)))) {
+ DRM_ERROR("Unknown GMADR size (%llx)\n", ggtt->mappable_end);
return -ENXIO;
}
@@ -3112,37 +3101,32 @@
pci_set_consistent_dma_mask(dev->pdev, DMA_BIT_MASK(40));
pci_read_config_word(dev->pdev, SNB_GMCH_CTRL, &snb_gmch_ctl);
- *stolen = gen6_get_stolen_size(snb_gmch_ctl);
+ ggtt->stolen_size = gen6_get_stolen_size(snb_gmch_ctl);
+ ggtt->size = gen6_get_total_gtt_size(snb_gmch_ctl);
+ ggtt->base.total = (ggtt->size / sizeof(gen6_pte_t)) << PAGE_SHIFT;
- gtt_size = gen6_get_total_gtt_size(snb_gmch_ctl);
- *gtt_total = (gtt_size / sizeof(gen6_pte_t)) << PAGE_SHIFT;
+ ret = ggtt_probe_common(dev, ggtt->size);
- ret = ggtt_probe_common(dev, gtt_size);
-
- dev_priv->gtt.base.clear_range = gen6_ggtt_clear_range;
- dev_priv->gtt.base.insert_entries = gen6_ggtt_insert_entries;
- dev_priv->gtt.base.bind_vma = ggtt_bind_vma;
- dev_priv->gtt.base.unbind_vma = ggtt_unbind_vma;
+ ggtt->base.clear_range = gen6_ggtt_clear_range;
+ ggtt->base.insert_entries = gen6_ggtt_insert_entries;
+ ggtt->base.bind_vma = ggtt_bind_vma;
+ ggtt->base.unbind_vma = ggtt_unbind_vma;
return ret;
}
static void gen6_gmch_remove(struct i915_address_space *vm)
{
+ struct i915_ggtt *ggtt = container_of(vm, struct i915_ggtt, base);
- struct i915_gtt *gtt = container_of(vm, struct i915_gtt, base);
-
- iounmap(gtt->gsm);
+ iounmap(ggtt->gsm);
free_scratch_page(vm->dev, vm->scratch_page);
}
-static int i915_gmch_probe(struct drm_device *dev,
- u64 *gtt_total,
- size_t *stolen,
- phys_addr_t *mappable_base,
- u64 *mappable_end)
+static int i915_gmch_probe(struct i915_ggtt *ggtt)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_device *dev = ggtt->base.dev;
+ struct drm_i915_private *dev_priv = to_i915(dev);
int ret;
ret = intel_gmch_probe(dev_priv->bridge_dev, dev_priv->dev->pdev, NULL);
@@ -3151,15 +3135,16 @@
return -EIO;
}
- intel_gtt_get(gtt_total, stolen, mappable_base, mappable_end);
+ intel_gtt_get(&ggtt->base.total, &ggtt->stolen_size,
+ &ggtt->mappable_base, &ggtt->mappable_end);
- dev_priv->gtt.do_idle_maps = needs_idle_maps(dev_priv->dev);
- dev_priv->gtt.base.insert_entries = i915_ggtt_insert_entries;
- dev_priv->gtt.base.clear_range = i915_ggtt_clear_range;
- dev_priv->gtt.base.bind_vma = ggtt_bind_vma;
- dev_priv->gtt.base.unbind_vma = ggtt_unbind_vma;
+ ggtt->do_idle_maps = needs_idle_maps(dev_priv->dev);
+ ggtt->base.insert_entries = i915_ggtt_insert_entries;
+ ggtt->base.clear_range = i915_ggtt_clear_range;
+ ggtt->base.bind_vma = ggtt_bind_vma;
+ ggtt->base.unbind_vma = ggtt_unbind_vma;
- if (unlikely(dev_priv->gtt.do_idle_maps))
+ if (unlikely(ggtt->do_idle_maps))
DRM_INFO("applying Ironlake quirks for intel_iommu\n");
return 0;
@@ -3170,41 +3155,53 @@
intel_gmch_remove();
}
-int i915_gem_gtt_init(struct drm_device *dev)
+/**
+ * i915_ggtt_init_hw - Initialize GGTT hardware
+ * @dev: DRM device
+ */
+int i915_ggtt_init_hw(struct drm_device *dev)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
- struct i915_gtt *gtt = &dev_priv->gtt;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
int ret;
if (INTEL_INFO(dev)->gen <= 5) {
- gtt->gtt_probe = i915_gmch_probe;
- gtt->base.cleanup = i915_gmch_remove;
+ ggtt->probe = i915_gmch_probe;
+ ggtt->base.cleanup = i915_gmch_remove;
} else if (INTEL_INFO(dev)->gen < 8) {
- gtt->gtt_probe = gen6_gmch_probe;
- gtt->base.cleanup = gen6_gmch_remove;
- if (IS_HASWELL(dev) && dev_priv->ellc_size)
- gtt->base.pte_encode = iris_pte_encode;
+ ggtt->probe = gen6_gmch_probe;
+ ggtt->base.cleanup = gen6_gmch_remove;
+
+ if (HAS_EDRAM(dev))
+ ggtt->base.pte_encode = iris_pte_encode;
else if (IS_HASWELL(dev))
- gtt->base.pte_encode = hsw_pte_encode;
+ ggtt->base.pte_encode = hsw_pte_encode;
else if (IS_VALLEYVIEW(dev))
- gtt->base.pte_encode = byt_pte_encode;
+ ggtt->base.pte_encode = byt_pte_encode;
else if (INTEL_INFO(dev)->gen >= 7)
- gtt->base.pte_encode = ivb_pte_encode;
+ ggtt->base.pte_encode = ivb_pte_encode;
else
- gtt->base.pte_encode = snb_pte_encode;
+ ggtt->base.pte_encode = snb_pte_encode;
} else {
- dev_priv->gtt.gtt_probe = gen8_gmch_probe;
- dev_priv->gtt.base.cleanup = gen6_gmch_remove;
+ ggtt->probe = gen8_gmch_probe;
+ ggtt->base.cleanup = gen6_gmch_remove;
}
- gtt->base.dev = dev;
- gtt->base.is_ggtt = true;
+ ggtt->base.dev = dev;
+ ggtt->base.is_ggtt = true;
- ret = gtt->gtt_probe(dev, >t->base.total, >t->stolen_size,
- >t->mappable_base, >t->mappable_end);
+ ret = ggtt->probe(ggtt);
if (ret)
return ret;
+ if ((ggtt->base.total - 1) >> 32) {
+ DRM_ERROR("We never expected a Global GTT with more than 32bits"
+ "of address space! Found %lldM!\n",
+ ggtt->base.total >> 20);
+ ggtt->base.total = 1ULL << 32;
+ ggtt->mappable_end = min(ggtt->mappable_end, ggtt->base.total);
+ }
+
/*
* Initialise stolen early so that we may reserve preallocated
* objects for the BIOS to KMS transition.
@@ -3215,9 +3212,9 @@
/* GMADR is the PCI mmio aperture into the global GTT. */
DRM_INFO("Memory usable by graphics device = %lluM\n",
- gtt->base.total >> 20);
- DRM_DEBUG_DRIVER("GMADR size = %lldM\n", gtt->mappable_end >> 20);
- DRM_DEBUG_DRIVER("GTT stolen size = %zdM\n", gtt->stolen_size >> 20);
+ ggtt->base.total >> 20);
+ DRM_DEBUG_DRIVER("GMADR size = %lldM\n", ggtt->mappable_end >> 20);
+ DRM_DEBUG_DRIVER("GTT stolen size = %zdM\n", ggtt->stolen_size >> 20);
#ifdef CONFIG_INTEL_IOMMU
if (intel_iommu_gfx_mapped)
DRM_INFO("VT-d active for gfx access\n");
@@ -3234,33 +3231,30 @@
return 0;
out_gtt_cleanup:
- gtt->base.cleanup(&dev_priv->gtt.base);
+ ggtt->base.cleanup(&ggtt->base);
return ret;
}
void i915_gem_restore_gtt_mappings(struct drm_device *dev)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct drm_i915_gem_object *obj;
- struct i915_address_space *vm;
struct i915_vma *vma;
bool flush;
i915_check_and_clear_faults(dev);
/* First fill our portion of the GTT with scratch pages */
- dev_priv->gtt.base.clear_range(&dev_priv->gtt.base,
- dev_priv->gtt.base.start,
- dev_priv->gtt.base.total,
- true);
+ ggtt->base.clear_range(&ggtt->base, ggtt->base.start, ggtt->base.total,
+ true);
/* Cache flush objects bound into GGTT and rebind them. */
- vm = &dev_priv->gtt.base;
list_for_each_entry(obj, &dev_priv->mm.bound_list, global_list) {
flush = false;
list_for_each_entry(vma, &obj->vma_list, obj_link) {
- if (vma->vm != vm)
+ if (vma->vm != &ggtt->base)
continue;
WARN_ON(i915_vma_bind(vma, obj->cache_level,
@@ -3283,15 +3277,17 @@
}
if (USES_PPGTT(dev)) {
+ struct i915_address_space *vm;
+
list_for_each_entry(vm, &dev_priv->vm_list, global_link) {
/* TODO: Perhaps it shouldn't be gen6 specific */
- struct i915_hw_ppgtt *ppgtt =
- container_of(vm, struct i915_hw_ppgtt,
- base);
+ struct i915_hw_ppgtt *ppgtt;
- if (i915_is_ggtt(vm))
+ if (vm->is_ggtt)
ppgtt = dev_priv->mm.aliasing_ppgtt;
+ else
+ ppgtt = i915_vm_to_ppgtt(vm);
gen6_write_page_range(dev_priv, &ppgtt->pd,
0, ppgtt->base.total);
@@ -3350,19 +3346,13 @@
i915_gem_obj_lookup_or_create_ggtt_vma(struct drm_i915_gem_object *obj,
const struct i915_ggtt_view *view)
{
- struct i915_address_space *ggtt = i915_obj_to_ggtt(obj);
- struct i915_vma *vma;
-
- if (WARN_ON(!view))
- return ERR_PTR(-EINVAL);
-
- vma = i915_gem_obj_to_ggtt_view(obj, view);
-
- if (IS_ERR(vma))
- return vma;
+ struct drm_device *dev = obj->base.dev;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
+ struct i915_vma *vma = i915_gem_obj_to_ggtt_view(obj, view);
if (!vma)
- vma = __i915_gem_vma_create(obj, ggtt, view);
+ vma = __i915_gem_vma_create(obj, &ggtt->base, view);
return vma;
@@ -3377,11 +3367,6 @@
unsigned int column, row;
unsigned int src_idx;
- if (!sg) {
- st->nents = 0;
- sg = st->sgl;
- }
-
for (column = 0; column < width; column++) {
src_idx = stride * (height - 1) + column;
for (row = 0; row < height; row++) {
@@ -3405,7 +3390,7 @@
intel_rotate_fb_obj_pages(struct intel_rotation_info *rot_info,
struct drm_i915_gem_object *obj)
{
- unsigned int size_pages = rot_info->size >> PAGE_SHIFT;
+ unsigned int size_pages = rot_info->plane[0].width * rot_info->plane[0].height;
unsigned int size_pages_uv;
struct sg_page_iter sg_iter;
unsigned long i;
@@ -3416,14 +3401,15 @@
int ret = -ENOMEM;
/* Allocate a temporary list of source pages for random access. */
- page_addr_list = drm_malloc_ab(obj->base.size / PAGE_SIZE,
- sizeof(dma_addr_t));
+ page_addr_list = drm_malloc_gfp(obj->base.size / PAGE_SIZE,
+ sizeof(dma_addr_t),
+ GFP_TEMPORARY);
if (!page_addr_list)
return ERR_PTR(ret);
/* Account for UV plane with NV12. */
if (rot_info->pixel_format == DRM_FORMAT_NV12)
- size_pages_uv = rot_info->size_uv >> PAGE_SHIFT;
+ size_pages_uv = rot_info->plane[1].width * rot_info->plane[1].height;
else
size_pages_uv = 0;
@@ -3443,11 +3429,14 @@
i++;
}
+ st->nents = 0;
+ sg = st->sgl;
+
/* Rotate the pages. */
sg = rotate_pages(page_addr_list, 0,
- rot_info->width_pages, rot_info->height_pages,
- rot_info->width_pages,
- st, NULL);
+ rot_info->plane[0].width, rot_info->plane[0].height,
+ rot_info->plane[0].width,
+ st, sg);
/* Append the UV plane if NV12. */
if (rot_info->pixel_format == DRM_FORMAT_NV12) {
@@ -3459,18 +3448,15 @@
rot_info->uv_start_page = uv_start_page;
- rotate_pages(page_addr_list, uv_start_page,
- rot_info->width_pages_uv,
- rot_info->height_pages_uv,
- rot_info->width_pages_uv,
- st, sg);
+ sg = rotate_pages(page_addr_list, rot_info->uv_start_page,
+ rot_info->plane[1].width, rot_info->plane[1].height,
+ rot_info->plane[1].width,
+ st, sg);
}
- DRM_DEBUG_KMS(
- "Created rotated page mapping for object size %zu (pitch=%u, height=%u, pixel_format=0x%x, %ux%u tiles, %u pages (%u plane 0)).\n",
- obj->base.size, rot_info->pitch, rot_info->height,
- rot_info->pixel_format, rot_info->width_pages,
- rot_info->height_pages, size_pages + size_pages_uv,
+ DRM_DEBUG_KMS("Created rotated page mapping for object size %zu (%ux%u tiles, %u pages (%u plane 0)).\n",
+ obj->base.size, rot_info->plane[0].width,
+ rot_info->plane[0].height, size_pages + size_pages_uv,
size_pages);
drm_free_large(page_addr_list);
@@ -3482,11 +3468,9 @@
err_st_alloc:
drm_free_large(page_addr_list);
- DRM_DEBUG_KMS(
- "Failed to create rotated mapping for object size %zu! (%d) (pitch=%u, height=%u, pixel_format=0x%x, %ux%u tiles, %u pages (%u plane 0))\n",
- obj->base.size, ret, rot_info->pitch, rot_info->height,
- rot_info->pixel_format, rot_info->width_pages,
- rot_info->height_pages, size_pages + size_pages_uv,
+ DRM_DEBUG_KMS("Failed to create rotated mapping for object size %zu! (%d) (%ux%u tiles, %u pages (%u plane 0))\n",
+ obj->base.size, ret, rot_info->plane[0].width,
+ rot_info->plane[0].height, size_pages + size_pages_uv,
size_pages);
return ERR_PTR(ret);
}
@@ -3634,7 +3618,7 @@
if (view->type == I915_GGTT_VIEW_NORMAL) {
return obj->base.size;
} else if (view->type == I915_GGTT_VIEW_ROTATED) {
- return view->params.rotated.size;
+ return intel_rotation_info_size(&view->params.rotated) << PAGE_SHIFT;
} else if (view->type == I915_GGTT_VIEW_PARTIAL) {
return view->params.partial.size << PAGE_SHIFT;
} else {
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.h b/drivers/gpu/drm/i915/i915_gem_gtt.h
index 8774f1b..d7dd3d8 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.h
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.h
@@ -42,7 +42,7 @@
typedef uint64_t gen8_ppgtt_pdpe_t;
typedef uint64_t gen8_ppgtt_pml4e_t;
-#define gtt_total_entries(gtt) ((gtt).base.total >> PAGE_SHIFT)
+#define ggtt_total_entries(ggtt) ((ggtt)->base.total >> PAGE_SHIFT)
/* gen6-hsw has bit 11-4 for physical addr bit 39-32 */
#define GEN6_GTT_ADDR_ENCODE(addr) ((addr) | (((addr) >> 28) & 0xff0))
@@ -135,16 +135,13 @@
};
struct intel_rotation_info {
- unsigned int height;
- unsigned int pitch;
unsigned int uv_offset;
uint32_t pixel_format;
- uint64_t fb_modifier;
- unsigned int width_pages, height_pages;
- uint64_t size;
- unsigned int width_pages_uv, height_pages_uv;
- uint64_t size_uv;
unsigned int uv_start_page;
+ struct {
+ /* tiles */
+ unsigned int width, height;
+ } plane[2];
};
struct i915_ggtt_view {
@@ -342,13 +339,14 @@
* and correct (in cases like swizzling). That region is referred to as GMADR in
* the spec.
*/
-struct i915_gtt {
+struct i915_ggtt {
struct i915_address_space base;
size_t stolen_size; /* Total size of stolen memory */
size_t stolen_usable_size; /* Total size minus BIOS reserved */
size_t stolen_reserved_base;
size_t stolen_reserved_size;
+ size_t size; /* Total size of Global GTT */
u64 mappable_end; /* End offset that we can CPU map */
struct io_mapping *mappable; /* Mapping to our CPU mappable region */
phys_addr_t mappable_base; /* PA of our GMADR */
@@ -360,10 +358,7 @@
int mtrr;
- /* global gtt ops */
- int (*gtt_probe)(struct drm_device *dev, u64 *gtt_total,
- size_t *stolen, phys_addr_t *mappable_base,
- u64 *mappable_end);
+ int (*probe)(struct i915_ggtt *ggtt);
};
struct i915_hw_ppgtt {
@@ -518,10 +513,9 @@
px_dma(ppgtt->base.scratch_pd);
}
-int i915_gem_gtt_init(struct drm_device *dev);
-void i915_gem_init_global_gtt(struct drm_device *dev);
-void i915_global_gtt_cleanup(struct drm_device *dev);
-
+int i915_ggtt_init_hw(struct drm_device *dev);
+void i915_gem_init_ggtt(struct drm_device *dev);
+void i915_ggtt_cleanup_hw(struct drm_device *dev);
int i915_ppgtt_init(struct drm_device *dev, struct i915_hw_ppgtt *ppgtt);
int i915_ppgtt_init_hw(struct drm_device *dev);
diff --git a/drivers/gpu/drm/i915/i915_gem_render_state.c b/drivers/gpu/drm/i915/i915_gem_render_state.c
index fc7e6d5..71611bf 100644
--- a/drivers/gpu/drm/i915/i915_gem_render_state.c
+++ b/drivers/gpu/drm/i915/i915_gem_render_state.c
@@ -169,15 +169,15 @@
drm_gem_object_unreference(&so->obj->base);
}
-int i915_gem_render_state_prepare(struct intel_engine_cs *ring,
+int i915_gem_render_state_prepare(struct intel_engine_cs *engine,
struct render_state *so)
{
int ret;
- if (WARN_ON(ring->id != RCS))
+ if (WARN_ON(engine->id != RCS))
return -ENOENT;
- ret = render_state_init(so, ring->dev);
+ ret = render_state_init(so, engine->dev);
if (ret)
return ret;
@@ -198,21 +198,21 @@
struct render_state so;
int ret;
- ret = i915_gem_render_state_prepare(req->ring, &so);
+ ret = i915_gem_render_state_prepare(req->engine, &so);
if (ret)
return ret;
if (so.rodata == NULL)
return 0;
- ret = req->ring->dispatch_execbuffer(req, so.ggtt_offset,
+ ret = req->engine->dispatch_execbuffer(req, so.ggtt_offset,
so.rodata->batch_items * 4,
I915_DISPATCH_SECURE);
if (ret)
goto out;
if (so.aux_batch_size > 8) {
- ret = req->ring->dispatch_execbuffer(req,
+ ret = req->engine->dispatch_execbuffer(req,
(so.ggtt_offset +
so.aux_batch_offset),
so.aux_batch_size,
diff --git a/drivers/gpu/drm/i915/i915_gem_render_state.h b/drivers/gpu/drm/i915/i915_gem_render_state.h
index e641bb0..6aaa3a1 100644
--- a/drivers/gpu/drm/i915/i915_gem_render_state.h
+++ b/drivers/gpu/drm/i915/i915_gem_render_state.h
@@ -43,7 +43,7 @@
int i915_gem_render_state_init(struct drm_i915_gem_request *req);
void i915_gem_render_state_fini(struct render_state *so);
-int i915_gem_render_state_prepare(struct intel_engine_cs *ring,
+int i915_gem_render_state_prepare(struct intel_engine_cs *engine,
struct render_state *so);
#endif /* _I915_GEM_RENDER_STATE_H_ */
diff --git a/drivers/gpu/drm/i915/i915_gem_shrinker.c b/drivers/gpu/drm/i915/i915_gem_shrinker.c
index d3c473f..425e721 100644
--- a/drivers/gpu/drm/i915/i915_gem_shrinker.c
+++ b/drivers/gpu/drm/i915/i915_gem_shrinker.c
@@ -28,6 +28,7 @@
#include <linux/swap.h>
#include <linux/pci.h>
#include <linux/dma-buf.h>
+#include <linux/vmalloc.h>
#include <drm/drmP.h>
#include <drm/i915_drm.h>
@@ -69,6 +70,10 @@
static bool can_release_pages(struct drm_i915_gem_object *obj)
{
+ /* Only shmemfs objects are backed by swap */
+ if (!obj->base.filp)
+ return false;
+
/* Only report true if by unbinding the object and putting its pages
* we can actually make forward progress towards freeing physical
* pages.
@@ -166,6 +171,10 @@
obj->madv != I915_MADV_DONTNEED)
continue;
+ if (flags & I915_SHRINK_VMAPS &&
+ !is_vmalloc_addr(obj->mapping))
+ continue;
+
if ((flags & I915_SHRINK_ACTIVE) == 0 && obj->active)
continue;
@@ -246,7 +255,7 @@
count = 0;
list_for_each_entry(obj, &dev_priv->mm.unbound_list, global_list)
- if (obj->pages_pin_count == 0)
+ if (can_release_pages(obj))
count += obj->base.size >> PAGE_SHIFT;
list_for_each_entry(obj, &dev_priv->mm.bound_list, global_list) {
@@ -288,67 +297,82 @@
return freed;
}
+struct shrinker_lock_uninterruptible {
+ bool was_interruptible;
+ bool unlock;
+};
+
+static bool
+i915_gem_shrinker_lock_uninterruptible(struct drm_i915_private *dev_priv,
+ struct shrinker_lock_uninterruptible *slu,
+ int timeout_ms)
+{
+ unsigned long timeout = msecs_to_jiffies(timeout_ms) + 1;
+
+ while (!i915_gem_shrinker_lock(dev_priv->dev, &slu->unlock)) {
+ schedule_timeout_killable(1);
+ if (fatal_signal_pending(current))
+ return false;
+ if (--timeout == 0) {
+ pr_err("Unable to lock GPU to purge memory.\n");
+ return false;
+ }
+ }
+
+ slu->was_interruptible = dev_priv->mm.interruptible;
+ dev_priv->mm.interruptible = false;
+ return true;
+}
+
+static void
+i915_gem_shrinker_unlock_uninterruptible(struct drm_i915_private *dev_priv,
+ struct shrinker_lock_uninterruptible *slu)
+{
+ dev_priv->mm.interruptible = slu->was_interruptible;
+ if (slu->unlock)
+ mutex_unlock(&dev_priv->dev->struct_mutex);
+}
+
static int
i915_gem_shrinker_oom(struct notifier_block *nb, unsigned long event, void *ptr)
{
struct drm_i915_private *dev_priv =
container_of(nb, struct drm_i915_private, mm.oom_notifier);
- struct drm_device *dev = dev_priv->dev;
+ struct shrinker_lock_uninterruptible slu;
struct drm_i915_gem_object *obj;
- unsigned long timeout = msecs_to_jiffies(5000) + 1;
- unsigned long pinned, bound, unbound, freed_pages;
- bool was_interruptible;
- bool unlock;
+ unsigned long unevictable, bound, unbound, freed_pages;
- while (!i915_gem_shrinker_lock(dev, &unlock) && --timeout) {
- schedule_timeout_killable(1);
- if (fatal_signal_pending(current))
- return NOTIFY_DONE;
- }
- if (timeout == 0) {
- pr_err("Unable to purge GPU memory due lock contention.\n");
+ if (!i915_gem_shrinker_lock_uninterruptible(dev_priv, &slu, 5000))
return NOTIFY_DONE;
- }
-
- was_interruptible = dev_priv->mm.interruptible;
- dev_priv->mm.interruptible = false;
freed_pages = i915_gem_shrink_all(dev_priv);
- dev_priv->mm.interruptible = was_interruptible;
-
/* Because we may be allocating inside our own driver, we cannot
* assert that there are no objects with pinned pages that are not
* being pointed to by hardware.
*/
- unbound = bound = pinned = 0;
+ unbound = bound = unevictable = 0;
list_for_each_entry(obj, &dev_priv->mm.unbound_list, global_list) {
- if (!obj->base.filp) /* not backed by a freeable object */
- continue;
-
- if (obj->pages_pin_count)
- pinned += obj->base.size;
+ if (!can_release_pages(obj))
+ unevictable += obj->base.size >> PAGE_SHIFT;
else
- unbound += obj->base.size;
+ unbound += obj->base.size >> PAGE_SHIFT;
}
list_for_each_entry(obj, &dev_priv->mm.bound_list, global_list) {
- if (!obj->base.filp)
- continue;
-
- if (obj->pages_pin_count)
- pinned += obj->base.size;
+ if (!can_release_pages(obj))
+ unevictable += obj->base.size >> PAGE_SHIFT;
else
- bound += obj->base.size;
+ bound += obj->base.size >> PAGE_SHIFT;
}
- if (unlock)
- mutex_unlock(&dev->struct_mutex);
+ i915_gem_shrinker_unlock_uninterruptible(dev_priv, &slu);
if (freed_pages || unbound || bound)
- pr_info("Purging GPU memory, %lu bytes freed, %lu bytes still pinned.\n",
- freed_pages << PAGE_SHIFT, pinned);
+ pr_info("Purging GPU memory, %lu pages freed, "
+ "%lu pages still pinned.\n",
+ freed_pages, unevictable);
if (unbound || bound)
- pr_err("%lu and %lu bytes still available in the "
+ pr_err("%lu and %lu pages still available in the "
"bound and unbound GPU page lists.\n",
bound, unbound);
@@ -356,6 +380,29 @@
return NOTIFY_DONE;
}
+static int
+i915_gem_shrinker_vmap(struct notifier_block *nb, unsigned long event, void *ptr)
+{
+ struct drm_i915_private *dev_priv =
+ container_of(nb, struct drm_i915_private, mm.vmap_notifier);
+ struct shrinker_lock_uninterruptible slu;
+ unsigned long freed_pages;
+
+ if (!i915_gem_shrinker_lock_uninterruptible(dev_priv, &slu, 5000))
+ return NOTIFY_DONE;
+
+ freed_pages = i915_gem_shrink(dev_priv, -1UL,
+ I915_SHRINK_BOUND |
+ I915_SHRINK_UNBOUND |
+ I915_SHRINK_ACTIVE |
+ I915_SHRINK_VMAPS);
+
+ i915_gem_shrinker_unlock_uninterruptible(dev_priv, &slu);
+
+ *(unsigned long *)ptr += freed_pages;
+ return NOTIFY_DONE;
+}
+
/**
* i915_gem_shrinker_init - Initialize i915 shrinker
* @dev_priv: i915 device
@@ -371,6 +418,9 @@
dev_priv->mm.oom_notifier.notifier_call = i915_gem_shrinker_oom;
WARN_ON(register_oom_notifier(&dev_priv->mm.oom_notifier));
+
+ dev_priv->mm.vmap_notifier.notifier_call = i915_gem_shrinker_vmap;
+ WARN_ON(register_vmap_purge_notifier(&dev_priv->mm.vmap_notifier));
}
/**
@@ -381,6 +431,7 @@
*/
void i915_gem_shrinker_cleanup(struct drm_i915_private *dev_priv)
{
+ WARN_ON(unregister_vmap_purge_notifier(&dev_priv->mm.vmap_notifier));
WARN_ON(unregister_oom_notifier(&dev_priv->mm.oom_notifier));
unregister_shrinker(&dev_priv->mm.shrinker);
}
diff --git a/drivers/gpu/drm/i915/i915_gem_stolen.c b/drivers/gpu/drm/i915/i915_gem_stolen.c
index 2e6e9fb..b7ce963 100644
--- a/drivers/gpu/drm/i915/i915_gem_stolen.c
+++ b/drivers/gpu/drm/i915/i915_gem_stolen.c
@@ -72,9 +72,11 @@
struct drm_mm_node *node, u64 size,
unsigned alignment)
{
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
+
return i915_gem_stolen_insert_node_in_range(dev_priv, node, size,
- alignment, 0,
- dev_priv->gtt.stolen_usable_size);
+ alignment, 0,
+ ggtt->stolen_usable_size);
}
void i915_gem_stolen_remove_node(struct drm_i915_private *dev_priv,
@@ -87,14 +89,15 @@
static unsigned long i915_stolen_to_physical(struct drm_device *dev)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct resource *r;
u32 base;
/* Almost universally we can find the Graphics Base of Stolen Memory
- * at offset 0x5c in the igfx configuration space. On a few (desktop)
- * machines this is also mirrored in the bridge device at different
- * locations, or in the MCHBAR.
+ * at register BSM (0x5c) in the igfx configuration space. On a few
+ * (desktop) machines this is also mirrored in the bridge device at
+ * different locations, or in the MCHBAR.
*
* On 865 we just check the TOUD register.
*
@@ -104,9 +107,11 @@
*/
base = 0;
if (INTEL_INFO(dev)->gen >= 3) {
- /* Read Graphics Base of Stolen Memory directly */
- pci_read_config_dword(dev->pdev, 0x5c, &base);
- base &= ~((1<<20) - 1);
+ u32 bsm;
+
+ pci_read_config_dword(dev->pdev, BSM, &bsm);
+
+ base = bsm & BSM_MASK;
} else if (IS_I865G(dev)) {
u16 toud = 0;
@@ -134,7 +139,7 @@
I85X_DRB3, &tmp);
tom = tmp * MB(32);
- base = tom - tseg_size - dev_priv->gtt.stolen_size;
+ base = tom - tseg_size - ggtt->stolen_size;
} else if (IS_845G(dev)) {
u32 tseg_size = 0;
u32 tom;
@@ -158,7 +163,7 @@
I830_DRB3, &tmp);
tom = tmp * MB(32);
- base = tom - tseg_size - dev_priv->gtt.stolen_size;
+ base = tom - tseg_size - ggtt->stolen_size;
} else if (IS_I830(dev)) {
u32 tseg_size = 0;
u32 tom;
@@ -178,7 +183,7 @@
I830_DRB3, &tmp);
tom = tmp * MB(32);
- base = tom - tseg_size - dev_priv->gtt.stolen_size;
+ base = tom - tseg_size - ggtt->stolen_size;
}
if (base == 0)
@@ -189,41 +194,41 @@
struct {
u32 start, end;
} stolen[2] = {
- { .start = base, .end = base + dev_priv->gtt.stolen_size, },
- { .start = base, .end = base + dev_priv->gtt.stolen_size, },
+ { .start = base, .end = base + ggtt->stolen_size, },
+ { .start = base, .end = base + ggtt->stolen_size, },
};
- u64 gtt_start, gtt_end;
+ u64 ggtt_start, ggtt_end;
- gtt_start = I915_READ(PGTBL_CTL);
+ ggtt_start = I915_READ(PGTBL_CTL);
if (IS_GEN4(dev))
- gtt_start = (gtt_start & PGTBL_ADDRESS_LO_MASK) |
- (gtt_start & PGTBL_ADDRESS_HI_MASK) << 28;
+ ggtt_start = (ggtt_start & PGTBL_ADDRESS_LO_MASK) |
+ (ggtt_start & PGTBL_ADDRESS_HI_MASK) << 28;
else
- gtt_start &= PGTBL_ADDRESS_LO_MASK;
- gtt_end = gtt_start + gtt_total_entries(dev_priv->gtt) * 4;
+ ggtt_start &= PGTBL_ADDRESS_LO_MASK;
+ ggtt_end = ggtt_start + ggtt_total_entries(ggtt) * 4;
- if (gtt_start >= stolen[0].start && gtt_start < stolen[0].end)
- stolen[0].end = gtt_start;
- if (gtt_end > stolen[1].start && gtt_end <= stolen[1].end)
- stolen[1].start = gtt_end;
+ if (ggtt_start >= stolen[0].start && ggtt_start < stolen[0].end)
+ stolen[0].end = ggtt_start;
+ if (ggtt_end > stolen[1].start && ggtt_end <= stolen[1].end)
+ stolen[1].start = ggtt_end;
/* pick the larger of the two chunks */
if (stolen[0].end - stolen[0].start >
stolen[1].end - stolen[1].start) {
base = stolen[0].start;
- dev_priv->gtt.stolen_size = stolen[0].end - stolen[0].start;
+ ggtt->stolen_size = stolen[0].end - stolen[0].start;
} else {
base = stolen[1].start;
- dev_priv->gtt.stolen_size = stolen[1].end - stolen[1].start;
+ ggtt->stolen_size = stolen[1].end - stolen[1].start;
}
if (stolen[0].start != stolen[1].start ||
stolen[0].end != stolen[1].end) {
DRM_DEBUG_KMS("GTT within stolen memory at 0x%llx-0x%llx\n",
- (unsigned long long) gtt_start,
- (unsigned long long) gtt_end - 1);
+ (unsigned long long)ggtt_start,
+ (unsigned long long)ggtt_end - 1);
DRM_DEBUG_KMS("Stolen memory adjusted to 0x%x-0x%x\n",
- base, base + (u32) dev_priv->gtt.stolen_size - 1);
+ base, base + (u32)ggtt->stolen_size - 1);
}
}
@@ -233,7 +238,7 @@
* kernel. So if the region is already marked as busy, something
* is seriously wrong.
*/
- r = devm_request_mem_region(dev->dev, base, dev_priv->gtt.stolen_size,
+ r = devm_request_mem_region(dev->dev, base, ggtt->stolen_size,
"Graphics Stolen Memory");
if (r == NULL) {
/*
@@ -245,7 +250,7 @@
* reservation starting from 1 instead of 0.
*/
r = devm_request_mem_region(dev->dev, base + 1,
- dev_priv->gtt.stolen_size - 1,
+ ggtt->stolen_size - 1,
"Graphics Stolen Memory");
/*
* GEN3 firmware likes to smash pci bridges into the stolen
@@ -253,7 +258,7 @@
*/
if (r == NULL && !IS_GEN3(dev)) {
DRM_ERROR("conflict detected with stolen region: [0x%08x - 0x%08x]\n",
- base, base + (uint32_t)dev_priv->gtt.stolen_size);
+ base, base + (uint32_t)ggtt->stolen_size);
base = 0;
}
}
@@ -274,11 +279,12 @@
static void g4x_get_stolen_reserved(struct drm_i915_private *dev_priv,
unsigned long *base, unsigned long *size)
{
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
uint32_t reg_val = I915_READ(IS_GM45(dev_priv) ?
CTG_STOLEN_RESERVED :
ELK_STOLEN_RESERVED);
unsigned long stolen_top = dev_priv->mm.stolen_base +
- dev_priv->gtt.stolen_size;
+ ggtt->stolen_size;
*base = (reg_val & G4X_STOLEN_RESERVED_ADDR2_MASK) << 16;
@@ -369,10 +375,11 @@
static void bdw_get_stolen_reserved(struct drm_i915_private *dev_priv,
unsigned long *base, unsigned long *size)
{
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
uint32_t reg_val = I915_READ(GEN6_STOLEN_RESERVED);
unsigned long stolen_top;
- stolen_top = dev_priv->mm.stolen_base + dev_priv->gtt.stolen_size;
+ stolen_top = dev_priv->mm.stolen_base + ggtt->stolen_size;
*base = reg_val & GEN6_STOLEN_RESERVED_ADDR_MASK;
@@ -388,7 +395,8 @@
int i915_gem_init_stolen(struct drm_device *dev)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
unsigned long reserved_total, reserved_base = 0, reserved_size;
unsigned long stolen_top;
@@ -401,14 +409,14 @@
}
#endif
- if (dev_priv->gtt.stolen_size == 0)
+ if (ggtt->stolen_size == 0)
return 0;
dev_priv->mm.stolen_base = i915_stolen_to_physical(dev);
if (dev_priv->mm.stolen_base == 0)
return 0;
- stolen_top = dev_priv->mm.stolen_base + dev_priv->gtt.stolen_size;
+ stolen_top = dev_priv->mm.stolen_base + ggtt->stolen_size;
switch (INTEL_INFO(dev_priv)->gen) {
case 2:
@@ -458,19 +466,18 @@
return 0;
}
- dev_priv->gtt.stolen_reserved_base = reserved_base;
- dev_priv->gtt.stolen_reserved_size = reserved_size;
+ ggtt->stolen_reserved_base = reserved_base;
+ ggtt->stolen_reserved_size = reserved_size;
/* It is possible for the reserved area to end before the end of stolen
* memory, so just consider the start. */
reserved_total = stolen_top - reserved_base;
DRM_DEBUG_KMS("Memory reserved for graphics device: %zuK, usable: %luK\n",
- dev_priv->gtt.stolen_size >> 10,
- (dev_priv->gtt.stolen_size - reserved_total) >> 10);
+ ggtt->stolen_size >> 10,
+ (ggtt->stolen_size - reserved_total) >> 10);
- dev_priv->gtt.stolen_usable_size = dev_priv->gtt.stolen_size -
- reserved_total;
+ ggtt->stolen_usable_size = ggtt->stolen_size - reserved_total;
/*
* Basic memrange allocator for stolen space.
@@ -483,7 +490,7 @@
* i915_gem_stolen_insert_node_in_range(). We may want to fix the fbcon
* problem later.
*/
- drm_mm_init(&dev_priv->mm.stolen, 0, dev_priv->gtt.stolen_usable_size);
+ drm_mm_init(&dev_priv->mm.stolen, 0, ggtt->stolen_usable_size);
return 0;
}
@@ -492,12 +499,13 @@
i915_pages_create_for_stolen(struct drm_device *dev,
u32 offset, u32 size)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct sg_table *st;
struct scatterlist *sg;
DRM_DEBUG_DRIVER("offset=0x%x, size=%d\n", offset, size);
- BUG_ON(offset > dev_priv->gtt.stolen_size - size);
+ BUG_ON(offset > ggtt->stolen_size - size);
/* We hide that we have no struct page backing our stolen object
* by wrapping the contiguous physical allocation with a fake
@@ -628,8 +636,8 @@
u32 gtt_offset,
u32 size)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
- struct i915_address_space *ggtt = &dev_priv->gtt.base;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct drm_i915_gem_object *obj;
struct drm_mm_node *stolen;
struct i915_vma *vma;
@@ -675,7 +683,7 @@
if (gtt_offset == I915_GTT_OFFSET_NONE)
return obj;
- vma = i915_gem_obj_lookup_or_create_vma(obj, ggtt);
+ vma = i915_gem_obj_lookup_or_create_vma(obj, &ggtt->base);
if (IS_ERR(vma)) {
ret = PTR_ERR(vma);
goto err;
@@ -688,8 +696,8 @@
*/
vma->node.start = gtt_offset;
vma->node.size = size;
- if (drm_mm_initialized(&ggtt->mm)) {
- ret = drm_mm_reserve_node(&ggtt->mm, &vma->node);
+ if (drm_mm_initialized(&ggtt->base.mm)) {
+ ret = drm_mm_reserve_node(&ggtt->base.mm, &vma->node);
if (ret) {
DRM_DEBUG_KMS("failed to allocate stolen GTT space\n");
goto err;
@@ -697,7 +705,7 @@
vma->bound |= GLOBAL_BIND;
__i915_vma_set_map_and_fenceable(vma);
- list_add_tail(&vma->vm_link, &ggtt->inactive_list);
+ list_add_tail(&vma->vm_link, &ggtt->base.inactive_list);
}
list_add_tail(&obj->global_list, &dev_priv->mm.bound_list);
diff --git a/drivers/gpu/drm/i915/i915_gem_tiling.c b/drivers/gpu/drm/i915/i915_gem_tiling.c
index 7410f6c..b9bdb340 100644
--- a/drivers/gpu/drm/i915/i915_gem_tiling.c
+++ b/drivers/gpu/drm/i915/i915_gem_tiling.c
@@ -166,7 +166,7 @@
struct drm_i915_gem_object *obj;
int ret = 0;
- obj = to_intel_bo(drm_gem_object_lookup(dev, file, args->handle));
+ obj = to_intel_bo(drm_gem_object_lookup(file, args->handle));
if (&obj->base == NULL)
return -ENOENT;
@@ -297,7 +297,7 @@
struct drm_i915_private *dev_priv = dev->dev_private;
struct drm_i915_gem_object *obj;
- obj = to_intel_bo(drm_gem_object_lookup(dev, file, args->handle));
+ obj = to_intel_bo(drm_gem_object_lookup(file, args->handle));
if (&obj->base == NULL)
return -ENOENT;
diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c
index 4d30b60..32d9726 100644
--- a/drivers/gpu/drm/i915/i915_gem_userptr.c
+++ b/drivers/gpu/drm/i915/i915_gem_userptr.c
@@ -34,7 +34,7 @@
struct i915_mm_struct {
struct mm_struct *mm;
- struct drm_device *dev;
+ struct drm_i915_private *i915;
struct i915_mmu_notifier *mn;
struct hlist_node node;
struct kref kref;
@@ -49,6 +49,7 @@
struct hlist_node node;
struct mmu_notifier mn;
struct rb_root objects;
+ struct workqueue_struct *wq;
};
struct i915_mmu_object {
@@ -60,6 +61,37 @@
bool attached;
};
+static void wait_rendering(struct drm_i915_gem_object *obj)
+{
+ struct drm_device *dev = obj->base.dev;
+ struct drm_i915_gem_request *requests[I915_NUM_ENGINES];
+ int i, n;
+
+ if (!obj->active)
+ return;
+
+ n = 0;
+ for (i = 0; i < I915_NUM_ENGINES; i++) {
+ struct drm_i915_gem_request *req;
+
+ req = obj->last_read_req[i];
+ if (req == NULL)
+ continue;
+
+ requests[n++] = i915_gem_request_reference(req);
+ }
+
+ mutex_unlock(&dev->struct_mutex);
+
+ for (i = 0; i < n; i++)
+ __i915_wait_request(requests[i], false, NULL, NULL);
+
+ mutex_lock(&dev->struct_mutex);
+
+ for (i = 0; i < n; i++)
+ i915_gem_request_unreference(requests[i]);
+}
+
static void cancel_userptr(struct work_struct *work)
{
struct i915_mmu_object *mo = container_of(work, typeof(*mo), work);
@@ -75,13 +107,13 @@
struct i915_vma *vma, *tmp;
bool was_interruptible;
+ wait_rendering(obj);
+
was_interruptible = dev_priv->mm.interruptible;
dev_priv->mm.interruptible = false;
- list_for_each_entry_safe(vma, tmp, &obj->vma_list, obj_link) {
- int ret = i915_vma_unbind(vma);
- WARN_ON(ret && ret != -EIO);
- }
+ list_for_each_entry_safe(vma, tmp, &obj->vma_list, obj_link)
+ WARN_ON(i915_vma_unbind(vma));
WARN_ON(i915_gem_object_put_pages(obj));
dev_priv->mm.interruptible = was_interruptible;
@@ -140,7 +172,7 @@
*/
mo = container_of(it, struct i915_mmu_object, it);
if (kref_get_unless_zero(&mo->obj->base.refcount))
- schedule_work(&mo->work);
+ queue_work(mn->wq, &mo->work);
list_add(&mo->link, &cancelled);
it = interval_tree_iter_next(it, start, end);
@@ -148,6 +180,8 @@
list_for_each_entry(mo, &cancelled, link)
del_object(mo);
spin_unlock(&mn->lock);
+
+ flush_workqueue(mn->wq);
}
static const struct mmu_notifier_ops i915_gem_userptr_notifier = {
@@ -167,10 +201,16 @@
spin_lock_init(&mn->lock);
mn->mn.ops = &i915_gem_userptr_notifier;
mn->objects = RB_ROOT;
+ mn->wq = alloc_workqueue("i915-userptr-release", WQ_UNBOUND, 0);
+ if (mn->wq == NULL) {
+ kfree(mn);
+ return ERR_PTR(-ENOMEM);
+ }
/* Protected by mmap_sem (write-lock) */
ret = __mmu_notifier_register(&mn->mn, mm);
if (ret) {
+ destroy_workqueue(mn->wq);
kfree(mn);
return ERR_PTR(ret);
}
@@ -205,13 +245,13 @@
return mn;
down_write(&mm->mm->mmap_sem);
- mutex_lock(&to_i915(mm->dev)->mm_lock);
+ mutex_lock(&mm->i915->mm_lock);
if ((mn = mm->mn) == NULL) {
mn = i915_mmu_notifier_create(mm->mm);
if (!IS_ERR(mn))
mm->mn = mn;
}
- mutex_unlock(&to_i915(mm->dev)->mm_lock);
+ mutex_unlock(&mm->i915->mm_lock);
up_write(&mm->mm->mmap_sem);
return mn;
@@ -256,6 +296,7 @@
return;
mmu_notifier_unregister(&mn->mn, mm);
+ destroy_workqueue(mn->wq);
kfree(mn);
}
@@ -327,7 +368,7 @@
}
kref_init(&mm->kref);
- mm->dev = obj->base.dev;
+ mm->i915 = to_i915(obj->base.dev);
mm->mm = current->mm;
atomic_inc(¤t->mm->mm_count);
@@ -362,7 +403,7 @@
/* Protected by dev_priv->mm_lock */
hash_del(&mm->node);
- mutex_unlock(&to_i915(mm->dev)->mm_lock);
+ mutex_unlock(&mm->i915->mm_lock);
INIT_WORK(&mm->work, __i915_mm_struct_free__worker);
schedule_work(&mm->work);
@@ -494,10 +535,7 @@
ret = -ENOMEM;
pinned = 0;
- pvec = kmalloc(npages*sizeof(struct page *),
- GFP_TEMPORARY | __GFP_NOWARN | __GFP_NORETRY);
- if (pvec == NULL)
- pvec = drm_malloc_ab(npages, sizeof(struct page *));
+ pvec = drm_malloc_gfp(npages, sizeof(struct page *), GFP_TEMPORARY);
if (pvec != NULL) {
struct mm_struct *mm = obj->userptr.mm->mm;
@@ -639,14 +677,11 @@
pvec = NULL;
pinned = 0;
if (obj->userptr.mm->mm == current->mm) {
- pvec = kmalloc(num_pages*sizeof(struct page *),
- GFP_TEMPORARY | __GFP_NOWARN | __GFP_NORETRY);
+ pvec = drm_malloc_gfp(num_pages, sizeof(struct page *),
+ GFP_TEMPORARY);
if (pvec == NULL) {
- pvec = drm_malloc_ab(num_pages, sizeof(struct page *));
- if (pvec == NULL) {
- __i915_gem_userptr_set_active(obj, false);
- return -ENOMEM;
- }
+ __i915_gem_userptr_set_active(obj, false);
+ return -ENOMEM;
}
pinned = __get_user_pages_fast(obj->userptr.ptr, num_pages,
@@ -763,6 +798,13 @@
int ret;
u32 handle;
+ if (!HAS_LLC(dev) && !HAS_SNOOP(dev)) {
+ /* We cannot support coherent userptr objects on hw without
+ * LLC and broken snooping.
+ */
+ return -ENODEV;
+ }
+
if (args->flags & ~(I915_USERPTR_READ_ONLY |
I915_USERPTR_UNSYNCHRONIZED))
return -EINVAL;
diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c
index 831895b..89725c9 100644
--- a/drivers/gpu/drm/i915/i915_gpu_error.c
+++ b/drivers/gpu/drm/i915/i915_gpu_error.c
@@ -198,7 +198,7 @@
err->size,
err->read_domains,
err->write_domain);
- for (i = 0; i < I915_NUM_RINGS; i++)
+ for (i = 0; i < I915_NUM_ENGINES; i++)
err_printf(m, "%02x ", err->rseqno[i]);
err_printf(m, "] %02x", err->wseqno);
@@ -230,8 +230,6 @@
return "wait";
case HANGCHECK_ACTIVE:
return "active";
- case HANGCHECK_ACTIVE_LOOP:
- return "active (loop)";
case HANGCHECK_KICK:
return "kick";
case HANGCHECK_HUNG:
@@ -298,6 +296,7 @@
}
}
err_printf(m, " seqno: 0x%08x\n", ring->seqno);
+ err_printf(m, " last_seqno: 0x%08x\n", ring->last_seqno);
err_printf(m, " waiting: %s\n", yesno(ring->waiting));
err_printf(m, " ring->head: 0x%08x\n", ring->cpu_ring_head);
err_printf(m, " ring->tail: 0x%08x\n", ring->cpu_ring_tail);
@@ -433,7 +432,7 @@
for (i = 0; i < ARRAY_SIZE(error->ring); i++) {
obj = error->ring[i].batchbuffer;
if (obj) {
- err_puts(m, dev_priv->ring[i].name);
+ err_puts(m, dev_priv->engine[i].name);
if (error->ring[i].pid != -1)
err_printf(m, " (submitted by %s [%d])",
error->ring[i].comm,
@@ -447,14 +446,14 @@
obj = error->ring[i].wa_batchbuffer;
if (obj) {
err_printf(m, "%s (w/a) --- gtt_offset = 0x%08x\n",
- dev_priv->ring[i].name,
+ dev_priv->engine[i].name,
lower_32_bits(obj->gtt_offset));
print_error_obj(m, obj);
}
if (error->ring[i].num_requests) {
err_printf(m, "%s --- %d requests\n",
- dev_priv->ring[i].name,
+ dev_priv->engine[i].name,
error->ring[i].num_requests);
for (j = 0; j < error->ring[i].num_requests; j++) {
err_printf(m, " seqno 0x%08x, emitted %ld, tail 0x%08x\n",
@@ -466,7 +465,7 @@
if ((obj = error->ring[i].ringbuffer)) {
err_printf(m, "%s --- ringbuffer = 0x%08x\n",
- dev_priv->ring[i].name,
+ dev_priv->engine[i].name,
lower_32_bits(obj->gtt_offset));
print_error_obj(m, obj);
}
@@ -480,7 +479,7 @@
hws_page = &obj->pages[LRC_PPHWSP_PN][0];
}
err_printf(m, "%s --- HW Status = 0x%08llx\n",
- dev_priv->ring[i].name, hws_offset);
+ dev_priv->engine[i].name, hws_offset);
offset = 0;
for (elt = 0; elt < PAGE_SIZE/16; elt += 4) {
err_printf(m, "[%04x] %08x %08x %08x %08x\n",
@@ -493,9 +492,31 @@
}
}
+ obj = error->ring[i].wa_ctx;
+ if (obj) {
+ u64 wa_ctx_offset = obj->gtt_offset;
+ u32 *wa_ctx_page = &obj->pages[0][0];
+ struct intel_engine_cs *engine = &dev_priv->engine[RCS];
+ u32 wa_ctx_size = (engine->wa_ctx.indirect_ctx.size +
+ engine->wa_ctx.per_ctx.size);
+
+ err_printf(m, "%s --- WA ctx batch buffer = 0x%08llx\n",
+ dev_priv->engine[i].name, wa_ctx_offset);
+ offset = 0;
+ for (elt = 0; elt < wa_ctx_size; elt += 4) {
+ err_printf(m, "[%04x] %08x %08x %08x %08x\n",
+ offset,
+ wa_ctx_page[elt + 0],
+ wa_ctx_page[elt + 1],
+ wa_ctx_page[elt + 2],
+ wa_ctx_page[elt + 3]);
+ offset += 16;
+ }
+ }
+
if ((obj = error->ring[i].ctx)) {
err_printf(m, "%s --- HW Context = 0x%08x\n",
- dev_priv->ring[i].name,
+ dev_priv->engine[i].name,
lower_32_bits(obj->gtt_offset));
print_error_obj(m, obj);
}
@@ -585,6 +606,7 @@
i915_error_object_free(error->ring[i].hws_page);
i915_error_object_free(error->ring[i].ctx);
kfree(error->ring[i].requests);
+ i915_error_object_free(error->ring[i].wa_ctx);
}
i915_error_object_free(error->semaphore_obj);
@@ -606,6 +628,7 @@
struct drm_i915_gem_object *src,
struct i915_address_space *vm)
{
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct drm_i915_error_object *dst;
struct i915_vma *vma = NULL;
int num_pages;
@@ -632,7 +655,7 @@
vma = i915_gem_obj_to_ggtt(src);
use_ggtt = (src->cache_level == I915_CACHE_NONE &&
vma && (vma->bound & GLOBAL_BIND) &&
- reloc_offset + num_pages * PAGE_SIZE <= dev_priv->gtt.mappable_end);
+ reloc_offset + num_pages * PAGE_SIZE <= ggtt->mappable_end);
/* Cannot access stolen address directly, try to use the aperture */
if (src->stolen) {
@@ -642,12 +665,13 @@
goto unwind;
reloc_offset = i915_gem_obj_ggtt_offset(src);
- if (reloc_offset + num_pages * PAGE_SIZE > dev_priv->gtt.mappable_end)
+ if (reloc_offset + num_pages * PAGE_SIZE > ggtt->mappable_end)
goto unwind;
}
/* Cannot access snooped pages through the aperture */
- if (use_ggtt && src->cache_level != I915_CACHE_NONE && !HAS_LLC(dev_priv->dev))
+ if (use_ggtt && src->cache_level != I915_CACHE_NONE &&
+ !HAS_LLC(dev_priv))
goto unwind;
dst->page_count = num_pages;
@@ -668,7 +692,7 @@
* captures what the GPU read.
*/
- s = io_mapping_map_atomic_wc(dev_priv->gtt.mappable,
+ s = io_mapping_map_atomic_wc(ggtt->mappable,
reloc_offset);
memcpy_fromio(d, s, PAGE_SIZE);
io_mapping_unmap_atomic(s);
@@ -701,7 +725,7 @@
return NULL;
}
#define i915_error_ggtt_object_create(dev_priv, src) \
- i915_error_object_create((dev_priv), (src), &(dev_priv)->gtt.base)
+ i915_error_object_create((dev_priv), (src), &(dev_priv)->ggtt.base)
static void capture_bo(struct drm_i915_error_buffer *err,
struct i915_vma *vma)
@@ -711,7 +735,7 @@
err->size = obj->base.size;
err->name = obj->base.name;
- for (i = 0; i < I915_NUM_RINGS; i++)
+ for (i = 0; i < I915_NUM_ENGINES; i++)
err->rseqno[i] = i915_gem_request_get_seqno(obj->last_read_req[i]);
err->wseqno = i915_gem_request_get_seqno(obj->last_write_req);
err->gtt_offset = vma->node.start;
@@ -726,7 +750,7 @@
err->purgeable = obj->madv != I915_MADV_WILLNEED;
err->userptr = obj->userptr.mm != NULL;
err->ring = obj->last_write_req ?
- i915_gem_request_get_ring(obj->last_write_req)->id : -1;
+ i915_gem_request_get_engine(obj->last_write_req)->id : -1;
err->cache_level = obj->cache_level;
}
@@ -788,7 +812,7 @@
* synchronization commands which almost always appear in the case
* strictly a client bug. Use instdone to differentiate those some.
*/
- for (i = 0; i < I915_NUM_RINGS; i++) {
+ for (i = 0; i < I915_NUM_ENGINES; i++) {
if (error->ring[i].hangcheck_action == HANGCHECK_HUNG) {
if (ring_id)
*ring_id = i;
@@ -821,11 +845,11 @@
static void gen8_record_semaphore_state(struct drm_i915_private *dev_priv,
struct drm_i915_error_state *error,
- struct intel_engine_cs *ring,
+ struct intel_engine_cs *engine,
struct drm_i915_error_ring *ering)
{
struct intel_engine_cs *to;
- int i;
+ enum intel_engine_id id;
if (!i915_semaphore_is_enabled(dev_priv->dev))
return;
@@ -835,68 +859,69 @@
i915_error_ggtt_object_create(dev_priv,
dev_priv->semaphore_obj);
- for_each_ring(to, dev_priv, i) {
+ for_each_engine_id(to, dev_priv, id) {
int idx;
u16 signal_offset;
u32 *tmp;
- if (ring == to)
+ if (engine == to)
continue;
- signal_offset = (GEN8_SIGNAL_OFFSET(ring, i) & (PAGE_SIZE - 1))
+ signal_offset = (GEN8_SIGNAL_OFFSET(engine, id) & (PAGE_SIZE - 1))
/ 4;
tmp = error->semaphore_obj->pages[0];
- idx = intel_ring_sync_index(ring, to);
+ idx = intel_ring_sync_index(engine, to);
ering->semaphore_mboxes[idx] = tmp[signal_offset];
- ering->semaphore_seqno[idx] = ring->semaphore.sync_seqno[idx];
+ ering->semaphore_seqno[idx] = engine->semaphore.sync_seqno[idx];
}
}
static void gen6_record_semaphore_state(struct drm_i915_private *dev_priv,
- struct intel_engine_cs *ring,
+ struct intel_engine_cs *engine,
struct drm_i915_error_ring *ering)
{
- ering->semaphore_mboxes[0] = I915_READ(RING_SYNC_0(ring->mmio_base));
- ering->semaphore_mboxes[1] = I915_READ(RING_SYNC_1(ring->mmio_base));
- ering->semaphore_seqno[0] = ring->semaphore.sync_seqno[0];
- ering->semaphore_seqno[1] = ring->semaphore.sync_seqno[1];
+ ering->semaphore_mboxes[0] = I915_READ(RING_SYNC_0(engine->mmio_base));
+ ering->semaphore_mboxes[1] = I915_READ(RING_SYNC_1(engine->mmio_base));
+ ering->semaphore_seqno[0] = engine->semaphore.sync_seqno[0];
+ ering->semaphore_seqno[1] = engine->semaphore.sync_seqno[1];
- if (HAS_VEBOX(dev_priv->dev)) {
+ if (HAS_VEBOX(dev_priv)) {
ering->semaphore_mboxes[2] =
- I915_READ(RING_SYNC_2(ring->mmio_base));
- ering->semaphore_seqno[2] = ring->semaphore.sync_seqno[2];
+ I915_READ(RING_SYNC_2(engine->mmio_base));
+ ering->semaphore_seqno[2] = engine->semaphore.sync_seqno[2];
}
}
static void i915_record_ring_state(struct drm_device *dev,
struct drm_i915_error_state *error,
- struct intel_engine_cs *ring,
+ struct intel_engine_cs *engine,
struct drm_i915_error_ring *ering)
{
struct drm_i915_private *dev_priv = dev->dev_private;
if (INTEL_INFO(dev)->gen >= 6) {
- ering->rc_psmi = I915_READ(RING_PSMI_CTL(ring->mmio_base));
- ering->fault_reg = I915_READ(RING_FAULT_REG(ring));
+ ering->rc_psmi = I915_READ(RING_PSMI_CTL(engine->mmio_base));
+ ering->fault_reg = I915_READ(RING_FAULT_REG(engine));
if (INTEL_INFO(dev)->gen >= 8)
- gen8_record_semaphore_state(dev_priv, error, ring, ering);
+ gen8_record_semaphore_state(dev_priv, error, engine,
+ ering);
else
- gen6_record_semaphore_state(dev_priv, ring, ering);
+ gen6_record_semaphore_state(dev_priv, engine, ering);
}
if (INTEL_INFO(dev)->gen >= 4) {
- ering->faddr = I915_READ(RING_DMA_FADD(ring->mmio_base));
- ering->ipeir = I915_READ(RING_IPEIR(ring->mmio_base));
- ering->ipehr = I915_READ(RING_IPEHR(ring->mmio_base));
- ering->instdone = I915_READ(RING_INSTDONE(ring->mmio_base));
- ering->instps = I915_READ(RING_INSTPS(ring->mmio_base));
- ering->bbaddr = I915_READ(RING_BBADDR(ring->mmio_base));
+ ering->faddr = I915_READ(RING_DMA_FADD(engine->mmio_base));
+ ering->ipeir = I915_READ(RING_IPEIR(engine->mmio_base));
+ ering->ipehr = I915_READ(RING_IPEHR(engine->mmio_base));
+ ering->instdone = I915_READ(RING_INSTDONE(engine->mmio_base));
+ ering->instps = I915_READ(RING_INSTPS(engine->mmio_base));
+ ering->bbaddr = I915_READ(RING_BBADDR(engine->mmio_base));
if (INTEL_INFO(dev)->gen >= 8) {
- ering->faddr |= (u64) I915_READ(RING_DMA_FADD_UDW(ring->mmio_base)) << 32;
- ering->bbaddr |= (u64) I915_READ(RING_BBADDR_UDW(ring->mmio_base)) << 32;
+ ering->faddr |= (u64) I915_READ(RING_DMA_FADD_UDW(engine->mmio_base)) << 32;
+ ering->bbaddr |= (u64) I915_READ(RING_BBADDR_UDW(engine->mmio_base)) << 32;
}
- ering->bbstate = I915_READ(RING_BBSTATE(ring->mmio_base));
+ ering->bbstate = I915_READ(RING_BBSTATE(engine->mmio_base));
} else {
ering->faddr = I915_READ(DMA_FADD_I8XX);
ering->ipeir = I915_READ(IPEIR);
@@ -904,20 +929,21 @@
ering->instdone = I915_READ(GEN2_INSTDONE);
}
- ering->waiting = waitqueue_active(&ring->irq_queue);
- ering->instpm = I915_READ(RING_INSTPM(ring->mmio_base));
- ering->seqno = ring->get_seqno(ring, false);
- ering->acthd = intel_ring_get_active_head(ring);
- ering->start = I915_READ_START(ring);
- ering->head = I915_READ_HEAD(ring);
- ering->tail = I915_READ_TAIL(ring);
- ering->ctl = I915_READ_CTL(ring);
+ ering->waiting = waitqueue_active(&engine->irq_queue);
+ ering->instpm = I915_READ(RING_INSTPM(engine->mmio_base));
+ ering->acthd = intel_ring_get_active_head(engine);
+ ering->seqno = engine->get_seqno(engine);
+ ering->last_seqno = engine->last_submitted_seqno;
+ ering->start = I915_READ_START(engine);
+ ering->head = I915_READ_HEAD(engine);
+ ering->tail = I915_READ_TAIL(engine);
+ ering->ctl = I915_READ_CTL(engine);
if (I915_NEED_GFX_HWS(dev)) {
i915_reg_t mmio;
if (IS_GEN7(dev)) {
- switch (ring->id) {
+ switch (engine->id) {
default:
case RCS:
mmio = RENDER_HWS_PGA_GEN7;
@@ -932,51 +958,51 @@
mmio = VEBOX_HWS_PGA_GEN7;
break;
}
- } else if (IS_GEN6(ring->dev)) {
- mmio = RING_HWS_PGA_GEN6(ring->mmio_base);
+ } else if (IS_GEN6(engine->dev)) {
+ mmio = RING_HWS_PGA_GEN6(engine->mmio_base);
} else {
/* XXX: gen8 returns to sanity */
- mmio = RING_HWS_PGA(ring->mmio_base);
+ mmio = RING_HWS_PGA(engine->mmio_base);
}
ering->hws = I915_READ(mmio);
}
- ering->hangcheck_score = ring->hangcheck.score;
- ering->hangcheck_action = ring->hangcheck.action;
+ ering->hangcheck_score = engine->hangcheck.score;
+ ering->hangcheck_action = engine->hangcheck.action;
if (USES_PPGTT(dev)) {
int i;
- ering->vm_info.gfx_mode = I915_READ(RING_MODE_GEN7(ring));
+ ering->vm_info.gfx_mode = I915_READ(RING_MODE_GEN7(engine));
if (IS_GEN6(dev))
ering->vm_info.pp_dir_base =
- I915_READ(RING_PP_DIR_BASE_READ(ring));
+ I915_READ(RING_PP_DIR_BASE_READ(engine));
else if (IS_GEN7(dev))
ering->vm_info.pp_dir_base =
- I915_READ(RING_PP_DIR_BASE(ring));
+ I915_READ(RING_PP_DIR_BASE(engine));
else if (INTEL_INFO(dev)->gen >= 8)
for (i = 0; i < 4; i++) {
ering->vm_info.pdp[i] =
- I915_READ(GEN8_RING_PDP_UDW(ring, i));
+ I915_READ(GEN8_RING_PDP_UDW(engine, i));
ering->vm_info.pdp[i] <<= 32;
ering->vm_info.pdp[i] |=
- I915_READ(GEN8_RING_PDP_LDW(ring, i));
+ I915_READ(GEN8_RING_PDP_LDW(engine, i));
}
}
}
-static void i915_gem_record_active_context(struct intel_engine_cs *ring,
+static void i915_gem_record_active_context(struct intel_engine_cs *engine,
struct drm_i915_error_state *error,
struct drm_i915_error_ring *ering)
{
- struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct drm_i915_private *dev_priv = engine->dev->dev_private;
struct drm_i915_gem_object *obj;
/* Currently render ring is the only HW context user */
- if (ring->id != RCS || !error->ccid)
+ if (engine->id != RCS || !error->ccid)
return;
list_for_each_entry(obj, &dev_priv->mm.bound_list, global_list) {
@@ -993,30 +1019,31 @@
static void i915_gem_record_rings(struct drm_device *dev,
struct drm_i915_error_state *error)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct drm_i915_gem_request *request;
int i, count;
- for (i = 0; i < I915_NUM_RINGS; i++) {
- struct intel_engine_cs *ring = &dev_priv->ring[i];
+ for (i = 0; i < I915_NUM_ENGINES; i++) {
+ struct intel_engine_cs *engine = &dev_priv->engine[i];
struct intel_ringbuffer *rbuf;
error->ring[i].pid = -1;
- if (ring->dev == NULL)
+ if (engine->dev == NULL)
continue;
error->ring[i].valid = true;
- i915_record_ring_state(dev, error, ring, &error->ring[i]);
+ i915_record_ring_state(dev, error, engine, &error->ring[i]);
- request = i915_gem_find_active_request(ring);
+ request = i915_gem_find_active_request(engine);
if (request) {
struct i915_address_space *vm;
vm = request->ctx && request->ctx->ppgtt ?
&request->ctx->ppgtt->base :
- &dev_priv->gtt.base;
+ &ggtt->base;
/* We need to copy these to an anonymous buffer
* as the simplest method to avoid being overwritten
@@ -1027,10 +1054,10 @@
request->batch_obj,
vm);
- if (HAS_BROKEN_CS_TLB(dev_priv->dev))
+ if (HAS_BROKEN_CS_TLB(dev_priv))
error->ring[i].wa_batchbuffer =
i915_error_ggtt_object_create(dev_priv,
- ring->scratch.obj);
+ engine->scratch.obj);
if (request->pid) {
struct task_struct *task;
@@ -1052,11 +1079,11 @@
* executed).
*/
if (request)
- rbuf = request->ctx->engine[ring->id].ringbuf;
+ rbuf = request->ctx->engine[engine->id].ringbuf;
else
- rbuf = dev_priv->kernel_context->engine[ring->id].ringbuf;
+ rbuf = dev_priv->kernel_context->engine[engine->id].ringbuf;
} else
- rbuf = ring->buffer;
+ rbuf = engine->buffer;
error->ring[i].cpu_ring_head = rbuf->head;
error->ring[i].cpu_ring_tail = rbuf->tail;
@@ -1065,12 +1092,19 @@
i915_error_ggtt_object_create(dev_priv, rbuf->obj);
error->ring[i].hws_page =
- i915_error_ggtt_object_create(dev_priv, ring->status_page.obj);
+ i915_error_ggtt_object_create(dev_priv,
+ engine->status_page.obj);
- i915_gem_record_active_context(ring, error, &error->ring[i]);
+ if (engine->wa_ctx.obj) {
+ error->ring[i].wa_ctx =
+ i915_error_ggtt_object_create(dev_priv,
+ engine->wa_ctx.obj);
+ }
+
+ i915_gem_record_active_context(engine, error, &error->ring[i]);
count = 0;
- list_for_each_entry(request, &ring->request_list, list)
+ list_for_each_entry(request, &engine->request_list, list)
count++;
error->ring[i].num_requests = count;
@@ -1083,7 +1117,7 @@
}
count = 0;
- list_for_each_entry(request, &ring->request_list, list) {
+ list_for_each_entry(request, &engine->request_list, list) {
struct drm_i915_error_request *erq;
if (count >= error->ring[i].num_requests) {
@@ -1272,7 +1306,7 @@
static void i915_error_capture_msg(struct drm_device *dev,
struct drm_i915_error_state *error,
- bool wedged,
+ u32 engine_mask,
const char *error_msg)
{
struct drm_i915_private *dev_priv = dev->dev_private;
@@ -1295,7 +1329,7 @@
scnprintf(error->error_msg + len, sizeof(error->error_msg) - len,
", reason: %s, action: %s",
error_msg,
- wedged ? "reset" : "continue");
+ engine_mask ? "reset" : "continue");
}
static void i915_capture_gen_state(struct drm_i915_private *dev_priv,
@@ -1318,7 +1352,7 @@
* out a structure which becomes available in debugfs for user level tools
* to pick up.
*/
-void i915_capture_error_state(struct drm_device *dev, bool wedged,
+void i915_capture_error_state(struct drm_device *dev, u32 engine_mask,
const char *error_msg)
{
static bool warned;
@@ -1346,7 +1380,7 @@
error->overlay = intel_overlay_capture_error_state(dev);
error->display = intel_display_capture_error_state(dev);
- i915_error_capture_msg(dev, error, wedged, error_msg);
+ i915_error_capture_msg(dev, error, engine_mask, error_msg);
DRM_INFO("%s\n", error->error_msg);
spin_lock_irqsave(&dev_priv->gpu_error.lock, flags);
diff --git a/drivers/gpu/drm/i915/i915_guc_reg.h b/drivers/gpu/drm/i915/i915_guc_reg.h
index e4ba582..80786d9 100644
--- a/drivers/gpu/drm/i915/i915_guc_reg.h
+++ b/drivers/gpu/drm/i915/i915_guc_reg.h
@@ -27,9 +27,12 @@
/* Definitions of GuC H/W registers, bits, etc */
#define GUC_STATUS _MMIO(0xc000)
+#define GS_RESET_SHIFT 0
+#define GS_MIA_IN_RESET (0x01 << GS_RESET_SHIFT)
#define GS_BOOTROM_SHIFT 1
#define GS_BOOTROM_MASK (0x7F << GS_BOOTROM_SHIFT)
#define GS_BOOTROM_RSA_FAILED (0x50 << GS_BOOTROM_SHIFT)
+#define GS_BOOTROM_JUMP_PASSED (0x76 << GS_BOOTROM_SHIFT)
#define GS_UKERNEL_SHIFT 8
#define GS_UKERNEL_MASK (0xFF << GS_UKERNEL_SHIFT)
#define GS_UKERNEL_LAPIC_DONE (0x30 << GS_UKERNEL_SHIFT)
@@ -37,7 +40,13 @@
#define GS_UKERNEL_READY (0xF0 << GS_UKERNEL_SHIFT)
#define GS_MIA_SHIFT 16
#define GS_MIA_MASK (0x07 << GS_MIA_SHIFT)
-#define GS_MIA_CORE_STATE (1 << GS_MIA_SHIFT)
+#define GS_MIA_CORE_STATE (0x01 << GS_MIA_SHIFT)
+#define GS_MIA_HALT_REQUESTED (0x02 << GS_MIA_SHIFT)
+#define GS_MIA_ISR_ENTRY (0x04 << GS_MIA_SHIFT)
+#define GS_AUTH_STATUS_SHIFT 30
+#define GS_AUTH_STATUS_MASK (0x03 << GS_AUTH_STATUS_SHIFT)
+#define GS_AUTH_STATUS_BAD (0x01 << GS_AUTH_STATUS_SHIFT)
+#define GS_AUTH_STATUS_GOOD (0x02 << GS_AUTH_STATUS_SHIFT)
#define SOFT_SCRATCH(n) _MMIO(0xc180 + (n) * 4)
#define SOFT_SCRATCH_COUNT 16
diff --git a/drivers/gpu/drm/i915/i915_guc_submission.c b/drivers/gpu/drm/i915/i915_guc_submission.c
index d7543ef..d40c13f 100644
--- a/drivers/gpu/drm/i915/i915_guc_submission.c
+++ b/drivers/gpu/drm/i915/i915_guc_submission.c
@@ -179,15 +179,11 @@
struct i915_guc_client *client)
{
struct guc_doorbell_info *doorbell;
- void *base;
- base = kmap_atomic(i915_gem_object_get_page(client->client_obj, 0));
- doorbell = base + client->doorbell_offset;
+ doorbell = client->client_base + client->doorbell_offset;
- doorbell->db_status = 1;
+ doorbell->db_status = GUC_DOORBELL_ENABLED;
doorbell->cookie = 0;
-
- kunmap_atomic(base);
}
static int guc_ring_doorbell(struct i915_guc_client *gc)
@@ -195,11 +191,9 @@
struct guc_process_desc *desc;
union guc_doorbell_qw db_cmp, db_exc, db_ret;
union guc_doorbell_qw *db;
- void *base;
int attempt = 2, ret = -EAGAIN;
- base = kmap_atomic(i915_gem_object_get_page(gc->client_obj, 0));
- desc = base + gc->proc_desc_offset;
+ desc = gc->client_base + gc->proc_desc_offset;
/* Update the tail so it is visible to GuC */
desc->tail = gc->wq_tail;
@@ -215,7 +209,7 @@
db_exc.cookie = 1;
/* pointer of current doorbell cacheline */
- db = base + gc->doorbell_offset;
+ db = gc->client_base + gc->doorbell_offset;
while (attempt--) {
/* lets ring the doorbell */
@@ -244,10 +238,6 @@
db_exc.cookie = 1;
}
- /* Finally, update the cached copy of the GuC's WQ head */
- gc->wq_head = desc->head;
-
- kunmap_atomic(base);
return ret;
}
@@ -256,16 +246,12 @@
{
struct drm_i915_private *dev_priv = guc_to_i915(guc);
struct guc_doorbell_info *doorbell;
- void *base;
i915_reg_t drbreg = GEN8_DRBREGL(client->doorbell_id);
int value;
- base = kmap_atomic(i915_gem_object_get_page(client->client_obj, 0));
- doorbell = base + client->doorbell_offset;
+ doorbell = client->client_base + client->doorbell_offset;
- doorbell->db_status = 0;
-
- kunmap_atomic(base);
+ doorbell->db_status = GUC_DOORBELL_DISABLED;
I915_WRITE(drbreg, I915_READ(drbreg) & ~GEN8_DRB_VALID);
@@ -341,10 +327,8 @@
struct i915_guc_client *client)
{
struct guc_process_desc *desc;
- void *base;
- base = kmap_atomic(i915_gem_object_get_page(client->client_obj, 0));
- desc = base + client->proc_desc_offset;
+ desc = client->client_base + client->proc_desc_offset;
memset(desc, 0, sizeof(*desc));
@@ -361,8 +345,6 @@
desc->wq_size_bytes = client->wq_size;
desc->wq_status = WQ_STATUS_ACTIVE;
desc->priority = client->priority;
-
- kunmap_atomic(base);
}
/*
@@ -376,12 +358,14 @@
static void guc_init_ctx_desc(struct intel_guc *guc,
struct i915_guc_client *client)
{
+ struct drm_i915_gem_object *client_obj = client->client_obj;
struct drm_i915_private *dev_priv = guc_to_i915(guc);
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
struct intel_context *ctx = client->owner;
struct guc_context_desc desc;
struct sg_table *sg;
- int i;
+ enum intel_engine_id id;
+ u32 gfx_addr;
memset(&desc, 0, sizeof(desc));
@@ -390,8 +374,8 @@
desc.priority = client->priority;
desc.db_id = client->doorbell_id;
- for_each_ring(ring, dev_priv, i) {
- struct guc_execlist_context *lrc = &desc.lrc[ring->guc_id];
+ for_each_engine_id(engine, dev_priv, id) {
+ struct guc_execlist_context *lrc = &desc.lrc[engine->guc_id];
struct drm_i915_gem_object *obj;
uint64_t ctx_desc;
@@ -402,48 +386,44 @@
* for now who owns a GuC client. But for future owner of GuC
* client, need to make sure lrc is pinned prior to enter here.
*/
- obj = ctx->engine[i].state;
+ obj = ctx->engine[id].state;
if (!obj)
break; /* XXX: continue? */
- ctx_desc = intel_lr_context_descriptor(ctx, ring);
+ ctx_desc = intel_lr_context_descriptor(ctx, engine);
lrc->context_desc = (u32)ctx_desc;
/* The state page is after PPHWSP */
- lrc->ring_lcra = i915_gem_obj_ggtt_offset(obj) +
- LRC_STATE_PN * PAGE_SIZE;
+ gfx_addr = i915_gem_obj_ggtt_offset(obj);
+ lrc->ring_lcra = gfx_addr + LRC_STATE_PN * PAGE_SIZE;
lrc->context_id = (client->ctx_index << GUC_ELC_CTXID_OFFSET) |
- (ring->guc_id << GUC_ELC_ENGINE_OFFSET);
+ (engine->guc_id << GUC_ELC_ENGINE_OFFSET);
- obj = ctx->engine[i].ringbuf->obj;
+ obj = ctx->engine[id].ringbuf->obj;
+ gfx_addr = i915_gem_obj_ggtt_offset(obj);
- lrc->ring_begin = i915_gem_obj_ggtt_offset(obj);
- lrc->ring_end = lrc->ring_begin + obj->base.size - 1;
- lrc->ring_next_free_location = lrc->ring_begin;
+ lrc->ring_begin = gfx_addr;
+ lrc->ring_end = gfx_addr + obj->base.size - 1;
+ lrc->ring_next_free_location = gfx_addr;
lrc->ring_current_tail_pointer_value = 0;
- desc.engines_used |= (1 << ring->guc_id);
+ desc.engines_used |= (1 << engine->guc_id);
}
WARN_ON(desc.engines_used == 0);
/*
- * The CPU address is only needed at certain points, so kmap_atomic on
- * demand instead of storing it in the ctx descriptor.
- * XXX: May make debug easier to have it mapped
+ * The doorbell, process descriptor, and workqueue are all parts
+ * of the client object, which the GuC will reference via the GGTT
*/
- desc.db_trigger_cpu = 0;
- desc.db_trigger_uk = client->doorbell_offset +
- i915_gem_obj_ggtt_offset(client->client_obj);
- desc.db_trigger_phy = client->doorbell_offset +
- sg_dma_address(client->client_obj->pages->sgl);
-
- desc.process_desc = client->proc_desc_offset +
- i915_gem_obj_ggtt_offset(client->client_obj);
-
- desc.wq_addr = client->wq_offset +
- i915_gem_obj_ggtt_offset(client->client_obj);
-
+ gfx_addr = i915_gem_obj_ggtt_offset(client_obj);
+ desc.db_trigger_phy = sg_dma_address(client_obj->pages->sgl) +
+ client->doorbell_offset;
+ desc.db_trigger_cpu = (uintptr_t)client->client_base +
+ client->doorbell_offset;
+ desc.db_trigger_uk = gfx_addr + client->doorbell_offset;
+ desc.process_desc = gfx_addr + client->proc_desc_offset;
+ desc.wq_addr = gfx_addr + client->wq_offset;
desc.wq_size = client->wq_size;
/*
@@ -474,25 +454,16 @@
int i915_guc_wq_check_space(struct i915_guc_client *gc)
{
struct guc_process_desc *desc;
- void *base;
u32 size = sizeof(struct guc_wq_item);
int ret = -ETIMEDOUT, timeout_counter = 200;
if (!gc)
return 0;
- /* Quickly return if wq space is available since last time we cache the
- * head position. */
- if (CIRC_SPACE(gc->wq_tail, gc->wq_head, gc->wq_size) >= size)
- return 0;
-
- base = kmap_atomic(i915_gem_object_get_page(gc->client_obj, 0));
- desc = base + gc->proc_desc_offset;
+ desc = gc->client_base + gc->proc_desc_offset;
while (timeout_counter-- > 0) {
- gc->wq_head = desc->head;
-
- if (CIRC_SPACE(gc->wq_tail, gc->wq_head, gc->wq_size) >= size) {
+ if (CIRC_SPACE(gc->wq_tail, desc->head, gc->wq_size) >= size) {
ret = 0;
break;
}
@@ -501,19 +472,19 @@
usleep_range(1000, 2000);
};
- kunmap_atomic(base);
-
return ret;
}
static int guc_add_workqueue_item(struct i915_guc_client *gc,
struct drm_i915_gem_request *rq)
{
+ struct guc_process_desc *desc;
struct guc_wq_item *wqi;
void *base;
u32 tail, wq_len, wq_off, space;
- space = CIRC_SPACE(gc->wq_tail, gc->wq_head, gc->wq_size);
+ desc = gc->client_base + gc->proc_desc_offset;
+ space = CIRC_SPACE(gc->wq_tail, desc->head, gc->wq_size);
if (WARN_ON(space < sizeof(struct guc_wq_item)))
return -ENOSPC; /* shouldn't happen */
@@ -542,11 +513,12 @@
wq_len = sizeof(struct guc_wq_item) / sizeof(u32) - 1;
wqi->header = WQ_TYPE_INORDER |
(wq_len << WQ_LEN_SHIFT) |
- (rq->ring->guc_id << WQ_TARGET_SHIFT) |
+ (rq->engine->guc_id << WQ_TARGET_SHIFT) |
WQ_NO_WCFLUSH_WAIT;
/* The GuC wants only the low-order word of the context descriptor */
- wqi->context_desc = (u32)intel_lr_context_descriptor(rq->ctx, rq->ring);
+ wqi->context_desc = (u32)intel_lr_context_descriptor(rq->ctx,
+ rq->engine);
/* The GuC firmware wants the tail index in QWords, not bytes */
tail = rq->ringbuf->tail >> 3;
@@ -569,7 +541,7 @@
struct drm_i915_gem_request *rq)
{
struct intel_guc *guc = client->guc;
- unsigned int engine_id = rq->ring->guc_id;
+ unsigned int engine_id = rq->engine->guc_id;
int q_ret, b_ret;
q_ret = guc_add_workqueue_item(client, rq);
@@ -660,21 +632,28 @@
if (!client)
return;
- if (client->doorbell_id != GUC_INVALID_DOORBELL_ID) {
- /*
- * First disable the doorbell, then tell the GuC we've
- * finished with it, finally deallocate it in our bitmap
- */
- guc_disable_doorbell(guc, client);
- host2guc_release_doorbell(guc, client);
- release_doorbell(guc, client->doorbell_id);
- }
-
/*
* XXX: wait for any outstanding submissions before freeing memory.
* Be sure to drop any locks
*/
+ if (client->client_base) {
+ /*
+ * If we got as far as setting up a doorbell, make sure
+ * we shut it down before unmapping & deallocating the
+ * memory. So first disable the doorbell, then tell the
+ * GuC that we've finished with it, finally deallocate
+ * it in our bitmap
+ */
+ if (client->doorbell_id != GUC_INVALID_DOORBELL_ID) {
+ guc_disable_doorbell(guc, client);
+ host2guc_release_doorbell(guc, client);
+ release_doorbell(guc, client->doorbell_id);
+ }
+
+ kunmap(kmap_to_page(client->client_base));
+ }
+
gem_release_guc_obj(client->client_obj);
if (client->ctx_index != GUC_INVALID_CTX_ID) {
@@ -695,7 +674,7 @@
* @ctx: the context that owns the client (we use the default render
* context)
*
- * Return: An i915_guc_client object if success.
+ * Return: An i915_guc_client object if success, else NULL.
*/
static struct i915_guc_client *guc_client_alloc(struct drm_device *dev,
uint32_t priority,
@@ -727,7 +706,9 @@
if (!obj)
goto err;
+ /* We'll keep just the first (doorbell/proc) page permanently kmap'd. */
client->client_obj = obj;
+ client->client_base = kmap(i915_gem_object_get_page(obj, 0));
client->wq_offset = GUC_DB_SIZE;
client->wq_size = GUC_WQ_SIZE;
@@ -839,9 +820,9 @@
struct guc_ads *ads;
struct guc_policies *policies;
struct guc_mmio_reg_state *reg_state;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
struct page *page;
- u32 size, i;
+ u32 size;
/* The ads obj includes the struct itself and buffers passed to GuC */
size = sizeof(struct guc_ads) + sizeof(struct guc_policies) +
@@ -867,11 +848,11 @@
* so its address won't change after we've told the GuC where
* to find it.
*/
- ring = &dev_priv->ring[RCS];
- ads->golden_context_lrca = ring->status_page.gfx_addr;
+ engine = &dev_priv->engine[RCS];
+ ads->golden_context_lrca = engine->status_page.gfx_addr;
- for_each_ring(ring, dev_priv, i)
- ads->eng_state_size[ring->guc_id] = intel_lr_context_size(ring);
+ for_each_engine(engine, dev_priv)
+ ads->eng_state_size[engine->guc_id] = intel_lr_context_size(engine);
/* GuC scheduling policies */
policies = (void *)ads + sizeof(struct guc_ads);
@@ -883,12 +864,12 @@
/* MMIO reg state */
reg_state = (void *)policies + sizeof(struct guc_policies);
- for_each_ring(ring, dev_priv, i) {
- reg_state->mmio_white_list[ring->guc_id].mmio_start =
- ring->mmio_base + GUC_MMIO_WHITE_LIST_START;
+ for_each_engine(engine, dev_priv) {
+ reg_state->mmio_white_list[engine->guc_id].mmio_start =
+ engine->mmio_base + GUC_MMIO_WHITE_LIST_START;
/* Nothing to be saved or restored for now. */
- reg_state->mmio_white_list[ring->guc_id].count = 0;
+ reg_state->mmio_white_list[engine->guc_id].count = 0;
}
ads->reg_state_addr = ads->scheduler_policies +
diff --git a/drivers/gpu/drm/i915/i915_irq.c b/drivers/gpu/drm/i915/i915_irq.c
index 1c21220..2f6fd33 100644
--- a/drivers/gpu/drm/i915/i915_irq.c
+++ b/drivers/gpu/drm/i915/i915_irq.c
@@ -994,14 +994,15 @@
return;
}
-static void notify_ring(struct intel_engine_cs *ring)
+static void notify_ring(struct intel_engine_cs *engine)
{
- if (!intel_ring_initialized(ring))
+ if (!intel_engine_initialized(engine))
return;
- trace_i915_gem_request_notify(ring);
+ trace_i915_gem_request_notify(engine);
+ engine->user_interrupts++;
- wake_up_all(&ring->irq_queue);
+ wake_up_all(&engine->irq_queue);
}
static void vlv_c0_read(struct drm_i915_private *dev_priv,
@@ -1079,11 +1080,10 @@
static bool any_waiters(struct drm_i915_private *dev_priv)
{
- struct intel_engine_cs *ring;
- int i;
+ struct intel_engine_cs *engine;
- for_each_ring(ring, dev_priv, i)
- if (ring->irq_refcount)
+ for_each_engine(engine, dev_priv)
+ if (engine->irq_refcount)
return true;
return false;
@@ -1219,7 +1219,7 @@
i915_reg_t reg;
slice--;
- if (WARN_ON_ONCE(slice >= NUM_L3_SLICES(dev_priv->dev)))
+ if (WARN_ON_ONCE(slice >= NUM_L3_SLICES(dev_priv)))
break;
dev_priv->l3_parity.which_slice &= ~(1<<slice);
@@ -1258,24 +1258,23 @@
out:
WARN_ON(dev_priv->l3_parity.which_slice);
spin_lock_irq(&dev_priv->irq_lock);
- gen5_enable_gt_irq(dev_priv, GT_PARITY_ERROR(dev_priv->dev));
+ gen5_enable_gt_irq(dev_priv, GT_PARITY_ERROR(dev_priv));
spin_unlock_irq(&dev_priv->irq_lock);
mutex_unlock(&dev_priv->dev->struct_mutex);
}
-static void ivybridge_parity_error_irq_handler(struct drm_device *dev, u32 iir)
+static void ivybridge_parity_error_irq_handler(struct drm_i915_private *dev_priv,
+ u32 iir)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
-
- if (!HAS_L3_DPF(dev))
+ if (!HAS_L3_DPF(dev_priv))
return;
spin_lock(&dev_priv->irq_lock);
- gen5_disable_gt_irq(dev_priv, GT_PARITY_ERROR(dev));
+ gen5_disable_gt_irq(dev_priv, GT_PARITY_ERROR(dev_priv));
spin_unlock(&dev_priv->irq_lock);
- iir &= GT_PARITY_ERROR(dev);
+ iir &= GT_PARITY_ERROR(dev_priv);
if (iir & GT_RENDER_L3_PARITY_ERROR_INTERRUPT_S1)
dev_priv->l3_parity.which_slice |= 1 << 1;
@@ -1285,102 +1284,85 @@
queue_work(dev_priv->wq, &dev_priv->l3_parity.error_work);
}
-static void ilk_gt_irq_handler(struct drm_device *dev,
- struct drm_i915_private *dev_priv,
+static void ilk_gt_irq_handler(struct drm_i915_private *dev_priv,
u32 gt_iir)
{
if (gt_iir &
(GT_RENDER_USER_INTERRUPT | GT_RENDER_PIPECTL_NOTIFY_INTERRUPT))
- notify_ring(&dev_priv->ring[RCS]);
+ notify_ring(&dev_priv->engine[RCS]);
if (gt_iir & ILK_BSD_USER_INTERRUPT)
- notify_ring(&dev_priv->ring[VCS]);
+ notify_ring(&dev_priv->engine[VCS]);
}
-static void snb_gt_irq_handler(struct drm_device *dev,
- struct drm_i915_private *dev_priv,
+static void snb_gt_irq_handler(struct drm_i915_private *dev_priv,
u32 gt_iir)
{
if (gt_iir &
(GT_RENDER_USER_INTERRUPT | GT_RENDER_PIPECTL_NOTIFY_INTERRUPT))
- notify_ring(&dev_priv->ring[RCS]);
+ notify_ring(&dev_priv->engine[RCS]);
if (gt_iir & GT_BSD_USER_INTERRUPT)
- notify_ring(&dev_priv->ring[VCS]);
+ notify_ring(&dev_priv->engine[VCS]);
if (gt_iir & GT_BLT_USER_INTERRUPT)
- notify_ring(&dev_priv->ring[BCS]);
+ notify_ring(&dev_priv->engine[BCS]);
if (gt_iir & (GT_BLT_CS_ERROR_INTERRUPT |
GT_BSD_CS_ERROR_INTERRUPT |
GT_RENDER_CS_MASTER_ERROR_INTERRUPT))
DRM_DEBUG("Command parser error, gt_iir 0x%08x\n", gt_iir);
- if (gt_iir & GT_PARITY_ERROR(dev))
- ivybridge_parity_error_irq_handler(dev, gt_iir);
+ if (gt_iir & GT_PARITY_ERROR(dev_priv))
+ ivybridge_parity_error_irq_handler(dev_priv, gt_iir);
}
static __always_inline void
-gen8_cs_irq_handler(struct intel_engine_cs *ring, u32 iir, int test_shift)
+gen8_cs_irq_handler(struct intel_engine_cs *engine, u32 iir, int test_shift)
{
if (iir & (GT_RENDER_USER_INTERRUPT << test_shift))
- notify_ring(ring);
+ notify_ring(engine);
if (iir & (GT_CONTEXT_SWITCH_INTERRUPT << test_shift))
- intel_lrc_irq_handler(ring);
+ tasklet_schedule(&engine->irq_tasklet);
}
-static irqreturn_t gen8_gt_irq_handler(struct drm_i915_private *dev_priv,
- u32 master_ctl)
+static irqreturn_t gen8_gt_irq_ack(struct drm_i915_private *dev_priv,
+ u32 master_ctl,
+ u32 gt_iir[4])
{
irqreturn_t ret = IRQ_NONE;
if (master_ctl & (GEN8_GT_RCS_IRQ | GEN8_GT_BCS_IRQ)) {
- u32 iir = I915_READ_FW(GEN8_GT_IIR(0));
- if (iir) {
- I915_WRITE_FW(GEN8_GT_IIR(0), iir);
+ gt_iir[0] = I915_READ_FW(GEN8_GT_IIR(0));
+ if (gt_iir[0]) {
+ I915_WRITE_FW(GEN8_GT_IIR(0), gt_iir[0]);
ret = IRQ_HANDLED;
-
- gen8_cs_irq_handler(&dev_priv->ring[RCS],
- iir, GEN8_RCS_IRQ_SHIFT);
-
- gen8_cs_irq_handler(&dev_priv->ring[BCS],
- iir, GEN8_BCS_IRQ_SHIFT);
} else
DRM_ERROR("The master control interrupt lied (GT0)!\n");
}
if (master_ctl & (GEN8_GT_VCS1_IRQ | GEN8_GT_VCS2_IRQ)) {
- u32 iir = I915_READ_FW(GEN8_GT_IIR(1));
- if (iir) {
- I915_WRITE_FW(GEN8_GT_IIR(1), iir);
+ gt_iir[1] = I915_READ_FW(GEN8_GT_IIR(1));
+ if (gt_iir[1]) {
+ I915_WRITE_FW(GEN8_GT_IIR(1), gt_iir[1]);
ret = IRQ_HANDLED;
-
- gen8_cs_irq_handler(&dev_priv->ring[VCS],
- iir, GEN8_VCS1_IRQ_SHIFT);
-
- gen8_cs_irq_handler(&dev_priv->ring[VCS2],
- iir, GEN8_VCS2_IRQ_SHIFT);
} else
DRM_ERROR("The master control interrupt lied (GT1)!\n");
}
if (master_ctl & GEN8_GT_VECS_IRQ) {
- u32 iir = I915_READ_FW(GEN8_GT_IIR(3));
- if (iir) {
- I915_WRITE_FW(GEN8_GT_IIR(3), iir);
+ gt_iir[3] = I915_READ_FW(GEN8_GT_IIR(3));
+ if (gt_iir[3]) {
+ I915_WRITE_FW(GEN8_GT_IIR(3), gt_iir[3]);
ret = IRQ_HANDLED;
-
- gen8_cs_irq_handler(&dev_priv->ring[VECS],
- iir, GEN8_VECS_IRQ_SHIFT);
} else
DRM_ERROR("The master control interrupt lied (GT3)!\n");
}
if (master_ctl & GEN8_GT_PM_IRQ) {
- u32 iir = I915_READ_FW(GEN8_GT_IIR(2));
- if (iir & dev_priv->pm_rps_events) {
+ gt_iir[2] = I915_READ_FW(GEN8_GT_IIR(2));
+ if (gt_iir[2] & dev_priv->pm_rps_events) {
I915_WRITE_FW(GEN8_GT_IIR(2),
- iir & dev_priv->pm_rps_events);
+ gt_iir[2] & dev_priv->pm_rps_events);
ret = IRQ_HANDLED;
- gen6_rps_irq_handler(dev_priv, iir);
} else
DRM_ERROR("The master control interrupt lied (PM)!\n");
}
@@ -1388,6 +1370,31 @@
return ret;
}
+static void gen8_gt_irq_handler(struct drm_i915_private *dev_priv,
+ u32 gt_iir[4])
+{
+ if (gt_iir[0]) {
+ gen8_cs_irq_handler(&dev_priv->engine[RCS],
+ gt_iir[0], GEN8_RCS_IRQ_SHIFT);
+ gen8_cs_irq_handler(&dev_priv->engine[BCS],
+ gt_iir[0], GEN8_BCS_IRQ_SHIFT);
+ }
+
+ if (gt_iir[1]) {
+ gen8_cs_irq_handler(&dev_priv->engine[VCS],
+ gt_iir[1], GEN8_VCS1_IRQ_SHIFT);
+ gen8_cs_irq_handler(&dev_priv->engine[VCS2],
+ gt_iir[1], GEN8_VCS2_IRQ_SHIFT);
+ }
+
+ if (gt_iir[3])
+ gen8_cs_irq_handler(&dev_priv->engine[VECS],
+ gt_iir[3], GEN8_VECS_IRQ_SHIFT);
+
+ if (gt_iir[2] & dev_priv->pm_rps_events)
+ gen6_rps_irq_handler(dev_priv, gt_iir[2]);
+}
+
static bool bxt_port_hotplug_long_detect(enum port port, u32 val)
{
switch (port) {
@@ -1627,9 +1634,9 @@
if (INTEL_INFO(dev_priv)->gen >= 8)
return;
- if (HAS_VEBOX(dev_priv->dev)) {
+ if (HAS_VEBOX(dev_priv)) {
if (pm_iir & PM_VEBOX_USER_INTERRUPT)
- notify_ring(&dev_priv->ring[VECS]);
+ notify_ring(&dev_priv->engine[VECS]);
if (pm_iir & PM_VEBOX_CS_ERROR_INTERRUPT)
DRM_DEBUG("Command parser error, pm_iir 0x%08x\n", pm_iir);
@@ -1644,10 +1651,10 @@
return true;
}
-static void valleyview_pipestat_irq_handler(struct drm_device *dev, u32 iir)
+static void valleyview_pipestat_irq_ack(struct drm_device *dev, u32 iir,
+ u32 pipe_stats[I915_MAX_PIPES])
{
struct drm_i915_private *dev_priv = dev->dev_private;
- u32 pipe_stats[I915_MAX_PIPES] = { };
int pipe;
spin_lock(&dev_priv->irq_lock);
@@ -1701,6 +1708,13 @@
I915_WRITE(reg, pipe_stats[pipe]);
}
spin_unlock(&dev_priv->irq_lock);
+}
+
+static void valleyview_pipestat_irq_handler(struct drm_device *dev,
+ u32 pipe_stats[I915_MAX_PIPES])
+{
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ enum pipe pipe;
for_each_pipe(dev_priv, pipe) {
if (pipe_stats[pipe] & PIPE_START_VBLANK_INTERRUPT_STATUS &&
@@ -1723,22 +1737,21 @@
gmbus_irq_handler(dev);
}
-static void i9xx_hpd_irq_handler(struct drm_device *dev)
+static u32 i9xx_hpd_irq_ack(struct drm_i915_private *dev_priv)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
u32 hotplug_status = I915_READ(PORT_HOTPLUG_STAT);
+
+ if (hotplug_status)
+ I915_WRITE(PORT_HOTPLUG_STAT, hotplug_status);
+
+ return hotplug_status;
+}
+
+static void i9xx_hpd_irq_handler(struct drm_device *dev,
+ u32 hotplug_status)
+{
u32 pin_mask = 0, long_mask = 0;
- if (!hotplug_status)
- return;
-
- I915_WRITE(PORT_HOTPLUG_STAT, hotplug_status);
- /*
- * Make sure hotplug status is cleared before we clear IIR, or else we
- * may miss hotplug events.
- */
- POSTING_READ(PORT_HOTPLUG_STAT);
-
if (IS_G4X(dev) || IS_VALLEYVIEW(dev) || IS_CHERRYVIEW(dev)) {
u32 hotplug_trigger = hotplug_status & HOTPLUG_INT_STATUS_G4X;
@@ -1768,59 +1781,6 @@
{
struct drm_device *dev = arg;
struct drm_i915_private *dev_priv = dev->dev_private;
- u32 iir, gt_iir, pm_iir;
- irqreturn_t ret = IRQ_NONE;
-
- if (!intel_irqs_enabled(dev_priv))
- return IRQ_NONE;
-
- /* IRQs are synced during runtime_suspend, we don't require a wakeref */
- disable_rpm_wakeref_asserts(dev_priv);
-
- while (true) {
- /* Find, clear, then process each source of interrupt */
-
- gt_iir = I915_READ(GTIIR);
- if (gt_iir)
- I915_WRITE(GTIIR, gt_iir);
-
- pm_iir = I915_READ(GEN6_PMIIR);
- if (pm_iir)
- I915_WRITE(GEN6_PMIIR, pm_iir);
-
- iir = I915_READ(VLV_IIR);
- if (iir) {
- /* Consume port before clearing IIR or we'll miss events */
- if (iir & I915_DISPLAY_PORT_INTERRUPT)
- i9xx_hpd_irq_handler(dev);
- I915_WRITE(VLV_IIR, iir);
- }
-
- if (gt_iir == 0 && pm_iir == 0 && iir == 0)
- goto out;
-
- ret = IRQ_HANDLED;
-
- if (gt_iir)
- snb_gt_irq_handler(dev, dev_priv, gt_iir);
- if (pm_iir)
- gen6_rps_irq_handler(dev_priv, pm_iir);
- /* Call regardless, as some status bits might not be
- * signalled in iir */
- valleyview_pipestat_irq_handler(dev, iir);
- }
-
-out:
- enable_rpm_wakeref_asserts(dev_priv);
-
- return ret;
-}
-
-static irqreturn_t cherryview_irq_handler(int irq, void *arg)
-{
- struct drm_device *dev = arg;
- struct drm_i915_private *dev_priv = dev->dev_private;
- u32 master_ctl, iir;
irqreturn_t ret = IRQ_NONE;
if (!intel_irqs_enabled(dev_priv))
@@ -1830,6 +1790,95 @@
disable_rpm_wakeref_asserts(dev_priv);
do {
+ u32 iir, gt_iir, pm_iir;
+ u32 pipe_stats[I915_MAX_PIPES] = {};
+ u32 hotplug_status = 0;
+ u32 ier = 0;
+
+ gt_iir = I915_READ(GTIIR);
+ pm_iir = I915_READ(GEN6_PMIIR);
+ iir = I915_READ(VLV_IIR);
+
+ if (gt_iir == 0 && pm_iir == 0 && iir == 0)
+ break;
+
+ ret = IRQ_HANDLED;
+
+ /*
+ * Theory on interrupt generation, based on empirical evidence:
+ *
+ * x = ((VLV_IIR & VLV_IER) ||
+ * (((GT_IIR & GT_IER) || (GEN6_PMIIR & GEN6_PMIER)) &&
+ * (VLV_MASTER_IER & MASTER_INTERRUPT_ENABLE)));
+ *
+ * A CPU interrupt will only be raised when 'x' has a 0->1 edge.
+ * Hence we clear MASTER_INTERRUPT_ENABLE and VLV_IER to
+ * guarantee the CPU interrupt will be raised again even if we
+ * don't end up clearing all the VLV_IIR, GT_IIR, GEN6_PMIIR
+ * bits this time around.
+ */
+ I915_WRITE(VLV_MASTER_IER, 0);
+ ier = I915_READ(VLV_IER);
+ I915_WRITE(VLV_IER, 0);
+
+ if (gt_iir)
+ I915_WRITE(GTIIR, gt_iir);
+ if (pm_iir)
+ I915_WRITE(GEN6_PMIIR, pm_iir);
+
+ if (iir & I915_DISPLAY_PORT_INTERRUPT)
+ hotplug_status = i9xx_hpd_irq_ack(dev_priv);
+
+ /* Call regardless, as some status bits might not be
+ * signalled in iir */
+ valleyview_pipestat_irq_ack(dev, iir, pipe_stats);
+
+ /*
+ * VLV_IIR is single buffered, and reflects the level
+ * from PIPESTAT/PORT_HOTPLUG_STAT, hence clear it last.
+ */
+ if (iir)
+ I915_WRITE(VLV_IIR, iir);
+
+ I915_WRITE(VLV_IER, ier);
+ I915_WRITE(VLV_MASTER_IER, MASTER_INTERRUPT_ENABLE);
+ POSTING_READ(VLV_MASTER_IER);
+
+ if (gt_iir)
+ snb_gt_irq_handler(dev_priv, gt_iir);
+ if (pm_iir)
+ gen6_rps_irq_handler(dev_priv, pm_iir);
+
+ if (hotplug_status)
+ i9xx_hpd_irq_handler(dev, hotplug_status);
+
+ valleyview_pipestat_irq_handler(dev, pipe_stats);
+ } while (0);
+
+ enable_rpm_wakeref_asserts(dev_priv);
+
+ return ret;
+}
+
+static irqreturn_t cherryview_irq_handler(int irq, void *arg)
+{
+ struct drm_device *dev = arg;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ irqreturn_t ret = IRQ_NONE;
+
+ if (!intel_irqs_enabled(dev_priv))
+ return IRQ_NONE;
+
+ /* IRQs are synced during runtime_suspend, we don't require a wakeref */
+ disable_rpm_wakeref_asserts(dev_priv);
+
+ do {
+ u32 master_ctl, iir;
+ u32 gt_iir[4] = {};
+ u32 pipe_stats[I915_MAX_PIPES] = {};
+ u32 hotplug_status = 0;
+ u32 ier = 0;
+
master_ctl = I915_READ(GEN8_MASTER_IRQ) & ~GEN8_MASTER_IRQ_CONTROL;
iir = I915_READ(VLV_IIR);
@@ -1838,25 +1887,49 @@
ret = IRQ_HANDLED;
+ /*
+ * Theory on interrupt generation, based on empirical evidence:
+ *
+ * x = ((VLV_IIR & VLV_IER) ||
+ * ((GEN8_MASTER_IRQ & ~GEN8_MASTER_IRQ_CONTROL) &&
+ * (GEN8_MASTER_IRQ & GEN8_MASTER_IRQ_CONTROL)));
+ *
+ * A CPU interrupt will only be raised when 'x' has a 0->1 edge.
+ * Hence we clear GEN8_MASTER_IRQ_CONTROL and VLV_IER to
+ * guarantee the CPU interrupt will be raised again even if we
+ * don't end up clearing all the VLV_IIR and GEN8_MASTER_IRQ_CONTROL
+ * bits this time around.
+ */
I915_WRITE(GEN8_MASTER_IRQ, 0);
+ ier = I915_READ(VLV_IER);
+ I915_WRITE(VLV_IER, 0);
- /* Find, clear, then process each source of interrupt */
+ gen8_gt_irq_ack(dev_priv, master_ctl, gt_iir);
- if (iir) {
- /* Consume port before clearing IIR or we'll miss events */
- if (iir & I915_DISPLAY_PORT_INTERRUPT)
- i9xx_hpd_irq_handler(dev);
- I915_WRITE(VLV_IIR, iir);
- }
-
- gen8_gt_irq_handler(dev_priv, master_ctl);
+ if (iir & I915_DISPLAY_PORT_INTERRUPT)
+ hotplug_status = i9xx_hpd_irq_ack(dev_priv);
/* Call regardless, as some status bits might not be
* signalled in iir */
- valleyview_pipestat_irq_handler(dev, iir);
+ valleyview_pipestat_irq_ack(dev, iir, pipe_stats);
- I915_WRITE(GEN8_MASTER_IRQ, DE_MASTER_IRQ_CONTROL);
+ /*
+ * VLV_IIR is single buffered, and reflects the level
+ * from PIPESTAT/PORT_HOTPLUG_STAT, hence clear it last.
+ */
+ if (iir)
+ I915_WRITE(VLV_IIR, iir);
+
+ I915_WRITE(VLV_IER, ier);
+ I915_WRITE(GEN8_MASTER_IRQ, GEN8_MASTER_IRQ_CONTROL);
POSTING_READ(GEN8_MASTER_IRQ);
+
+ gen8_gt_irq_handler(dev_priv, gt_iir);
+
+ if (hotplug_status)
+ i9xx_hpd_irq_handler(dev, hotplug_status);
+
+ valleyview_pipestat_irq_handler(dev, pipe_stats);
} while (0);
enable_rpm_wakeref_asserts(dev_priv);
@@ -2217,9 +2290,9 @@
I915_WRITE(GTIIR, gt_iir);
ret = IRQ_HANDLED;
if (INTEL_INFO(dev)->gen >= 6)
- snb_gt_irq_handler(dev, dev_priv, gt_iir);
+ snb_gt_irq_handler(dev_priv, gt_iir);
else
- ilk_gt_irq_handler(dev, dev_priv, gt_iir);
+ ilk_gt_irq_handler(dev_priv, gt_iir);
}
de_iir = I915_READ(DEIIR);
@@ -2419,6 +2492,7 @@
struct drm_device *dev = arg;
struct drm_i915_private *dev_priv = dev->dev_private;
u32 master_ctl;
+ u32 gt_iir[4] = {};
irqreturn_t ret;
if (!intel_irqs_enabled(dev_priv))
@@ -2435,7 +2509,8 @@
disable_rpm_wakeref_asserts(dev_priv);
/* Find, clear, then process each source of interrupt */
- ret = gen8_gt_irq_handler(dev_priv, master_ctl);
+ ret = gen8_gt_irq_ack(dev_priv, master_ctl, gt_iir);
+ gen8_gt_irq_handler(dev_priv, gt_iir);
ret |= gen8_de_irq_handler(dev_priv, master_ctl);
I915_WRITE_FW(GEN8_MASTER_IRQ, GEN8_MASTER_IRQ_CONTROL);
@@ -2449,8 +2524,7 @@
static void i915_error_wake_up(struct drm_i915_private *dev_priv,
bool reset_completed)
{
- struct intel_engine_cs *ring;
- int i;
+ struct intel_engine_cs *engine;
/*
* Notify all waiters for GPU completion events that reset state has
@@ -2460,8 +2534,8 @@
*/
/* Wake up __wait_seqno, potentially holding dev->struct_mutex. */
- for_each_ring(ring, dev_priv, i)
- wake_up_all(&ring->irq_queue);
+ for_each_engine(engine, dev_priv)
+ wake_up_all(&engine->irq_queue);
/* Wake up intel_crtc_wait_for_pending_flips, holding crtc->mutex. */
wake_up_all(&dev_priv->pending_flip_queue);
@@ -2484,7 +2558,6 @@
static void i915_reset_and_wakeup(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = to_i915(dev);
- struct i915_gpu_error *error = &dev_priv->gpu_error;
char *error_event[] = { I915_ERROR_UEVENT "=1", NULL };
char *reset_event[] = { I915_RESET_UEVENT "=1", NULL };
char *reset_done_event[] = { I915_ERROR_UEVENT "=0", NULL };
@@ -2502,7 +2575,7 @@
* the reset in-progress bit is only ever set by code outside of this
* work we don't need to worry about any other races.
*/
- if (i915_reset_in_progress(error) && !i915_terminally_wedged(error)) {
+ if (i915_reset_in_progress(&dev_priv->gpu_error)) {
DRM_DEBUG_DRIVER("resetting chip\n");
kobject_uevent_env(&dev->primary->kdev->kobj, KOBJ_CHANGE,
reset_event);
@@ -2530,25 +2603,9 @@
intel_runtime_pm_put(dev_priv);
- if (ret == 0) {
- /*
- * After all the gem state is reset, increment the reset
- * counter and wake up everyone waiting for the reset to
- * complete.
- *
- * Since unlock operations are a one-sided barrier only,
- * we need to insert a barrier here to order any seqno
- * updates before
- * the counter increment.
- */
- smp_mb__before_atomic();
- atomic_inc(&dev_priv->gpu_error.reset_counter);
-
+ if (ret == 0)
kobject_uevent_env(&dev->primary->kdev->kobj,
KOBJ_CHANGE, reset_done_event);
- } else {
- atomic_or(I915_WEDGED, &error->reset_counter);
- }
/*
* Note: The wake_up also serves as a memory barrier so that
@@ -2653,14 +2710,14 @@
/**
* i915_handle_error - handle a gpu error
* @dev: drm device
- *
+ * @engine_mask: mask representing engines that are hung
* Do some basic checking of register state at error time and
* dump it to the syslog. Also call i915_capture_error_state() to make
* sure we get a record and make it available in debugfs. Fire a uevent
* so userspace knows something bad happened (should trigger collection
* of a ring dump etc.).
*/
-void i915_handle_error(struct drm_device *dev, bool wedged,
+void i915_handle_error(struct drm_device *dev, u32 engine_mask,
const char *fmt, ...)
{
struct drm_i915_private *dev_priv = dev->dev_private;
@@ -2671,10 +2728,10 @@
vscnprintf(error_msg, sizeof(error_msg), fmt, args);
va_end(args);
- i915_capture_error_state(dev, wedged, error_msg);
+ i915_capture_error_state(dev, engine_mask, error_msg);
i915_report_and_clear_eir(dev);
- if (wedged) {
+ if (engine_mask) {
atomic_or(I915_RESET_IN_PROGRESS_FLAG,
&dev_priv->gpu_error.reset_counter);
@@ -2805,10 +2862,10 @@
}
static bool
-ring_idle(struct intel_engine_cs *ring, u32 seqno)
+ring_idle(struct intel_engine_cs *engine, u32 seqno)
{
- return (list_empty(&ring->request_list) ||
- i915_seqno_passed(seqno, ring->last_submitted_seqno));
+ return i915_seqno_passed(seqno,
+ READ_ONCE(engine->last_submitted_seqno));
}
static bool
@@ -2824,42 +2881,42 @@
}
static struct intel_engine_cs *
-semaphore_wait_to_signaller_ring(struct intel_engine_cs *ring, u32 ipehr, u64 offset)
+semaphore_wait_to_signaller_ring(struct intel_engine_cs *engine, u32 ipehr,
+ u64 offset)
{
- struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct drm_i915_private *dev_priv = engine->dev->dev_private;
struct intel_engine_cs *signaller;
- int i;
- if (INTEL_INFO(dev_priv->dev)->gen >= 8) {
- for_each_ring(signaller, dev_priv, i) {
- if (ring == signaller)
+ if (INTEL_INFO(dev_priv)->gen >= 8) {
+ for_each_engine(signaller, dev_priv) {
+ if (engine == signaller)
continue;
- if (offset == signaller->semaphore.signal_ggtt[ring->id])
+ if (offset == signaller->semaphore.signal_ggtt[engine->id])
return signaller;
}
} else {
u32 sync_bits = ipehr & MI_SEMAPHORE_SYNC_MASK;
- for_each_ring(signaller, dev_priv, i) {
- if(ring == signaller)
+ for_each_engine(signaller, dev_priv) {
+ if(engine == signaller)
continue;
- if (sync_bits == signaller->semaphore.mbox.wait[ring->id])
+ if (sync_bits == signaller->semaphore.mbox.wait[engine->id])
return signaller;
}
}
DRM_ERROR("No signaller ring found for ring %i, ipehr 0x%08x, offset 0x%016llx\n",
- ring->id, ipehr, offset);
+ engine->id, ipehr, offset);
return NULL;
}
static struct intel_engine_cs *
-semaphore_waits_for(struct intel_engine_cs *ring, u32 *seqno)
+semaphore_waits_for(struct intel_engine_cs *engine, u32 *seqno)
{
- struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct drm_i915_private *dev_priv = engine->dev->dev_private;
u32 cmd, ipehr, head;
u64 offset = 0;
int i, backwards;
@@ -2881,11 +2938,11 @@
* Therefore, this function does not support execlist mode in its
* current form. Just return NULL and move on.
*/
- if (ring->buffer == NULL)
+ if (engine->buffer == NULL)
return NULL;
- ipehr = I915_READ(RING_IPEHR(ring->mmio_base));
- if (!ipehr_is_semaphore_wait(ring->dev, ipehr))
+ ipehr = I915_READ(RING_IPEHR(engine->mmio_base));
+ if (!ipehr_is_semaphore_wait(engine->dev, ipehr))
return NULL;
/*
@@ -2896,8 +2953,8 @@
* point at at batch, and semaphores are always emitted into the
* ringbuffer itself.
*/
- head = I915_READ_HEAD(ring) & HEAD_ADDR;
- backwards = (INTEL_INFO(ring->dev)->gen >= 8) ? 5 : 4;
+ head = I915_READ_HEAD(engine) & HEAD_ADDR;
+ backwards = (INTEL_INFO(engine->dev)->gen >= 8) ? 5 : 4;
for (i = backwards; i; --i) {
/*
@@ -2905,10 +2962,10 @@
* our ring is smaller than what the hardware (and hence
* HEAD_ADDR) allows. Also handles wrap-around.
*/
- head &= ring->buffer->size - 1;
+ head &= engine->buffer->size - 1;
/* This here seems to blow up */
- cmd = ioread32(ring->buffer->virtual_start + head);
+ cmd = ioread32(engine->buffer->virtual_start + head);
if (cmd == ipehr)
break;
@@ -2918,32 +2975,32 @@
if (!i)
return NULL;
- *seqno = ioread32(ring->buffer->virtual_start + head + 4) + 1;
- if (INTEL_INFO(ring->dev)->gen >= 8) {
- offset = ioread32(ring->buffer->virtual_start + head + 12);
+ *seqno = ioread32(engine->buffer->virtual_start + head + 4) + 1;
+ if (INTEL_INFO(engine->dev)->gen >= 8) {
+ offset = ioread32(engine->buffer->virtual_start + head + 12);
offset <<= 32;
- offset = ioread32(ring->buffer->virtual_start + head + 8);
+ offset = ioread32(engine->buffer->virtual_start + head + 8);
}
- return semaphore_wait_to_signaller_ring(ring, ipehr, offset);
+ return semaphore_wait_to_signaller_ring(engine, ipehr, offset);
}
-static int semaphore_passed(struct intel_engine_cs *ring)
+static int semaphore_passed(struct intel_engine_cs *engine)
{
- struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct drm_i915_private *dev_priv = engine->dev->dev_private;
struct intel_engine_cs *signaller;
u32 seqno;
- ring->hangcheck.deadlock++;
+ engine->hangcheck.deadlock++;
- signaller = semaphore_waits_for(ring, &seqno);
+ signaller = semaphore_waits_for(engine, &seqno);
if (signaller == NULL)
return -1;
/* Prevent pathological recursion due to driver bugs */
- if (signaller->hangcheck.deadlock >= I915_NUM_RINGS)
+ if (signaller->hangcheck.deadlock >= I915_NUM_ENGINES)
return -1;
- if (i915_seqno_passed(signaller->get_seqno(signaller, false), seqno))
+ if (i915_seqno_passed(signaller->get_seqno(signaller), seqno))
return 1;
/* cursory check for an unkickable deadlock */
@@ -2956,23 +3013,22 @@
static void semaphore_clear_deadlocks(struct drm_i915_private *dev_priv)
{
- struct intel_engine_cs *ring;
- int i;
+ struct intel_engine_cs *engine;
- for_each_ring(ring, dev_priv, i)
- ring->hangcheck.deadlock = 0;
+ for_each_engine(engine, dev_priv)
+ engine->hangcheck.deadlock = 0;
}
-static bool subunits_stuck(struct intel_engine_cs *ring)
+static bool subunits_stuck(struct intel_engine_cs *engine)
{
u32 instdone[I915_NUM_INSTDONE_REG];
bool stuck;
int i;
- if (ring->id != RCS)
+ if (engine->id != RCS)
return true;
- i915_get_extra_instdone(ring->dev, instdone);
+ i915_get_extra_instdone(engine->dev, instdone);
/* There might be unstable subunit states even when
* actual head is not moving. Filter out the unstable ones by
@@ -2981,49 +3037,44 @@
*/
stuck = true;
for (i = 0; i < I915_NUM_INSTDONE_REG; i++) {
- const u32 tmp = instdone[i] | ring->hangcheck.instdone[i];
+ const u32 tmp = instdone[i] | engine->hangcheck.instdone[i];
- if (tmp != ring->hangcheck.instdone[i])
+ if (tmp != engine->hangcheck.instdone[i])
stuck = false;
- ring->hangcheck.instdone[i] |= tmp;
+ engine->hangcheck.instdone[i] |= tmp;
}
return stuck;
}
static enum intel_ring_hangcheck_action
-head_stuck(struct intel_engine_cs *ring, u64 acthd)
+head_stuck(struct intel_engine_cs *engine, u64 acthd)
{
- if (acthd != ring->hangcheck.acthd) {
+ if (acthd != engine->hangcheck.acthd) {
/* Clear subunit states on head movement */
- memset(ring->hangcheck.instdone, 0,
- sizeof(ring->hangcheck.instdone));
+ memset(engine->hangcheck.instdone, 0,
+ sizeof(engine->hangcheck.instdone));
- if (acthd > ring->hangcheck.max_acthd) {
- ring->hangcheck.max_acthd = acthd;
- return HANGCHECK_ACTIVE;
- }
-
- return HANGCHECK_ACTIVE_LOOP;
+ return HANGCHECK_ACTIVE;
}
- if (!subunits_stuck(ring))
+ if (!subunits_stuck(engine))
return HANGCHECK_ACTIVE;
return HANGCHECK_HUNG;
}
static enum intel_ring_hangcheck_action
-ring_stuck(struct intel_engine_cs *ring, u64 acthd)
+ring_stuck(struct intel_engine_cs *engine, u64 acthd)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
enum intel_ring_hangcheck_action ha;
u32 tmp;
- ha = head_stuck(ring, acthd);
+ ha = head_stuck(engine, acthd);
if (ha != HANGCHECK_HUNG)
return ha;
@@ -3035,24 +3086,24 @@
* and break the hang. This should work on
* all but the second generation chipsets.
*/
- tmp = I915_READ_CTL(ring);
+ tmp = I915_READ_CTL(engine);
if (tmp & RING_WAIT) {
- i915_handle_error(dev, false,
+ i915_handle_error(dev, 0,
"Kicking stuck wait on %s",
- ring->name);
- I915_WRITE_CTL(ring, tmp);
+ engine->name);
+ I915_WRITE_CTL(engine, tmp);
return HANGCHECK_KICK;
}
if (INTEL_INFO(dev)->gen >= 6 && tmp & RING_WAIT_SEMAPHORE) {
- switch (semaphore_passed(ring)) {
+ switch (semaphore_passed(engine)) {
default:
return HANGCHECK_HUNG;
case 1:
- i915_handle_error(dev, false,
+ i915_handle_error(dev, 0,
"Kicking stuck semaphore on %s",
- ring->name);
- I915_WRITE_CTL(ring, tmp);
+ engine->name);
+ I915_WRITE_CTL(engine, tmp);
return HANGCHECK_KICK;
case 0:
return HANGCHECK_WAIT;
@@ -3062,6 +3113,24 @@
return HANGCHECK_HUNG;
}
+static unsigned kick_waiters(struct intel_engine_cs *engine)
+{
+ struct drm_i915_private *i915 = to_i915(engine->dev);
+ unsigned user_interrupts = READ_ONCE(engine->user_interrupts);
+
+ if (engine->hangcheck.user_interrupts == user_interrupts &&
+ !test_and_set_bit(engine->id, &i915->gpu_error.missed_irq_rings)) {
+ if (!(i915->gpu_error.test_irq_rings & intel_engine_flag(engine)))
+ DRM_ERROR("Hangcheck timer elapsed... %s idle\n",
+ engine->name);
+ else
+ DRM_INFO("Fake missed irq on %s\n",
+ engine->name);
+ wake_up_all(&engine->irq_queue);
+ }
+
+ return user_interrupts;
+}
/*
* This is called when the chip hasn't reported back with completed
* batchbuffers in a long time. We keep track per ring seqno progress and
@@ -3076,13 +3145,14 @@
container_of(work, typeof(*dev_priv),
gpu_error.hangcheck_work.work);
struct drm_device *dev = dev_priv->dev;
- struct intel_engine_cs *ring;
- int i;
+ struct intel_engine_cs *engine;
+ enum intel_engine_id id;
int busy_count = 0, rings_hung = 0;
- bool stuck[I915_NUM_RINGS] = { 0 };
+ bool stuck[I915_NUM_ENGINES] = { 0 };
#define BUSY 1
#define KICK 5
#define HUNG 20
+#define ACTIVE_DECAY 15
if (!i915.enable_hangcheck)
return;
@@ -3100,33 +3170,37 @@
*/
intel_uncore_arm_unclaimed_mmio_detection(dev_priv);
- for_each_ring(ring, dev_priv, i) {
+ for_each_engine_id(engine, dev_priv, id) {
u64 acthd;
u32 seqno;
+ unsigned user_interrupts;
bool busy = true;
semaphore_clear_deadlocks(dev_priv);
- seqno = ring->get_seqno(ring, false);
- acthd = intel_ring_get_active_head(ring);
+ /* We don't strictly need an irq-barrier here, as we are not
+ * serving an interrupt request, be paranoid in case the
+ * barrier has side-effects (such as preventing a broken
+ * cacheline snoop) and so be sure that we can see the seqno
+ * advance. If the seqno should stick, due to a stale
+ * cacheline, we would erroneously declare the GPU hung.
+ */
+ if (engine->irq_seqno_barrier)
+ engine->irq_seqno_barrier(engine);
- if (ring->hangcheck.seqno == seqno) {
- if (ring_idle(ring, seqno)) {
- ring->hangcheck.action = HANGCHECK_IDLE;
+ acthd = intel_ring_get_active_head(engine);
+ seqno = engine->get_seqno(engine);
- if (waitqueue_active(&ring->irq_queue)) {
- /* Issue a wake-up to catch stuck h/w. */
- if (!test_and_set_bit(ring->id, &dev_priv->gpu_error.missed_irq_rings)) {
- if (!(dev_priv->gpu_error.test_irq_rings & intel_ring_flag(ring)))
- DRM_ERROR("Hangcheck timer elapsed... %s idle\n",
- ring->name);
- else
- DRM_INFO("Fake missed irq on %s\n",
- ring->name);
- wake_up_all(&ring->irq_queue);
- }
+ /* Reset stuck interrupts between batch advances */
+ user_interrupts = 0;
+
+ if (engine->hangcheck.seqno == seqno) {
+ if (ring_idle(engine, seqno)) {
+ engine->hangcheck.action = HANGCHECK_IDLE;
+ if (waitqueue_active(&engine->irq_queue)) {
/* Safeguard against driver failure */
- ring->hangcheck.score += BUSY;
+ user_interrupts = kick_waiters(engine);
+ engine->hangcheck.score += BUSY;
} else
busy = false;
} else {
@@ -3145,58 +3219,60 @@
* being repeatedly kicked and so responsible
* for stalling the machine.
*/
- ring->hangcheck.action = ring_stuck(ring,
- acthd);
+ engine->hangcheck.action = ring_stuck(engine,
+ acthd);
- switch (ring->hangcheck.action) {
+ switch (engine->hangcheck.action) {
case HANGCHECK_IDLE:
case HANGCHECK_WAIT:
- case HANGCHECK_ACTIVE:
break;
- case HANGCHECK_ACTIVE_LOOP:
- ring->hangcheck.score += BUSY;
+ case HANGCHECK_ACTIVE:
+ engine->hangcheck.score += BUSY;
break;
case HANGCHECK_KICK:
- ring->hangcheck.score += KICK;
+ engine->hangcheck.score += KICK;
break;
case HANGCHECK_HUNG:
- ring->hangcheck.score += HUNG;
- stuck[i] = true;
+ engine->hangcheck.score += HUNG;
+ stuck[id] = true;
break;
}
}
} else {
- ring->hangcheck.action = HANGCHECK_ACTIVE;
+ engine->hangcheck.action = HANGCHECK_ACTIVE;
/* Gradually reduce the count so that we catch DoS
* attempts across multiple batches.
*/
- if (ring->hangcheck.score > 0)
- ring->hangcheck.score--;
+ if (engine->hangcheck.score > 0)
+ engine->hangcheck.score -= ACTIVE_DECAY;
+ if (engine->hangcheck.score < 0)
+ engine->hangcheck.score = 0;
/* Clear head and subunit states on seqno movement */
- ring->hangcheck.acthd = ring->hangcheck.max_acthd = 0;
+ acthd = 0;
- memset(ring->hangcheck.instdone, 0,
- sizeof(ring->hangcheck.instdone));
+ memset(engine->hangcheck.instdone, 0,
+ sizeof(engine->hangcheck.instdone));
}
- ring->hangcheck.seqno = seqno;
- ring->hangcheck.acthd = acthd;
+ engine->hangcheck.seqno = seqno;
+ engine->hangcheck.acthd = acthd;
+ engine->hangcheck.user_interrupts = user_interrupts;
busy_count += busy;
}
- for_each_ring(ring, dev_priv, i) {
- if (ring->hangcheck.score >= HANGCHECK_SCORE_RING_HUNG) {
+ for_each_engine_id(engine, dev_priv, id) {
+ if (engine->hangcheck.score >= HANGCHECK_SCORE_RING_HUNG) {
DRM_INFO("%s on %s\n",
- stuck[i] ? "stuck" : "no progress",
- ring->name);
- rings_hung++;
+ stuck[id] ? "stuck" : "no progress",
+ engine->name);
+ rings_hung |= intel_engine_flag(engine);
}
}
if (rings_hung) {
- i915_handle_error(dev, true, "Ring hung");
+ i915_handle_error(dev, rings_hung, "Engine(s) hung");
goto out;
}
@@ -3267,6 +3343,55 @@
GEN5_IRQ_RESET(GEN6_PM);
}
+static void vlv_display_irq_reset(struct drm_i915_private *dev_priv)
+{
+ enum pipe pipe;
+
+ if (IS_CHERRYVIEW(dev_priv))
+ I915_WRITE(DPINVGTT, DPINVGTT_STATUS_MASK_CHV);
+ else
+ I915_WRITE(DPINVGTT, DPINVGTT_STATUS_MASK);
+
+ i915_hotplug_interrupt_update_locked(dev_priv, 0xffffffff, 0);
+ I915_WRITE(PORT_HOTPLUG_STAT, I915_READ(PORT_HOTPLUG_STAT));
+
+ for_each_pipe(dev_priv, pipe) {
+ I915_WRITE(PIPESTAT(pipe),
+ PIPE_FIFO_UNDERRUN_STATUS |
+ PIPESTAT_INT_STATUS_MASK);
+ dev_priv->pipestat_irq_mask[pipe] = 0;
+ }
+
+ GEN5_IRQ_RESET(VLV_);
+ dev_priv->irq_mask = ~0;
+}
+
+static void vlv_display_irq_postinstall(struct drm_i915_private *dev_priv)
+{
+ u32 pipestat_mask;
+ u32 enable_mask;
+ enum pipe pipe;
+
+ pipestat_mask = PLANE_FLIP_DONE_INT_STATUS_VLV |
+ PIPE_CRC_DONE_INTERRUPT_STATUS;
+
+ i915_enable_pipestat(dev_priv, PIPE_A, PIPE_GMBUS_INTERRUPT_STATUS);
+ for_each_pipe(dev_priv, pipe)
+ i915_enable_pipestat(dev_priv, pipe, pipestat_mask);
+
+ enable_mask = I915_DISPLAY_PORT_INTERRUPT |
+ I915_DISPLAY_PIPE_A_EVENT_INTERRUPT |
+ I915_DISPLAY_PIPE_B_EVENT_INTERRUPT;
+ if (IS_CHERRYVIEW(dev_priv))
+ enable_mask |= I915_DISPLAY_PIPE_C_EVENT_INTERRUPT;
+
+ WARN_ON(dev_priv->irq_mask != ~0);
+
+ dev_priv->irq_mask = ~enable_mask;
+
+ GEN5_IRQ_INIT(VLV_, dev_priv->irq_mask, enable_mask);
+}
+
/* drm_dma.h hooks
*/
static void ironlake_irq_reset(struct drm_device *dev)
@@ -3284,34 +3409,19 @@
ibx_irq_reset(dev);
}
-static void vlv_display_irq_reset(struct drm_i915_private *dev_priv)
-{
- enum pipe pipe;
-
- i915_hotplug_interrupt_update(dev_priv, 0xFFFFFFFF, 0);
- I915_WRITE(PORT_HOTPLUG_STAT, I915_READ(PORT_HOTPLUG_STAT));
-
- for_each_pipe(dev_priv, pipe)
- I915_WRITE(PIPESTAT(pipe), 0xffff);
-
- GEN5_IRQ_RESET(VLV_);
-}
-
static void valleyview_irq_preinstall(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- /* VLV magic */
- I915_WRITE(VLV_IMR, 0);
- I915_WRITE(RING_IMR(RENDER_RING_BASE), 0);
- I915_WRITE(RING_IMR(GEN6_BSD_RING_BASE), 0);
- I915_WRITE(RING_IMR(BLT_RING_BASE), 0);
+ I915_WRITE(VLV_MASTER_IER, 0);
+ POSTING_READ(VLV_MASTER_IER);
gen5_gt_irq_reset(dev);
- I915_WRITE(DPINVGTT, DPINVGTT_STATUS_MASK);
-
- vlv_display_irq_reset(dev_priv);
+ spin_lock_irq(&dev_priv->irq_lock);
+ if (dev_priv->display_irqs_enabled)
+ vlv_display_irq_reset(dev_priv);
+ spin_unlock_irq(&dev_priv->irq_lock);
}
static void gen8_gt_irq_reset(struct drm_i915_private *dev_priv)
@@ -3384,9 +3494,10 @@
GEN5_IRQ_RESET(GEN8_PCU_);
- I915_WRITE(DPINVGTT, DPINVGTT_STATUS_MASK_CHV);
-
- vlv_display_irq_reset(dev_priv);
+ spin_lock_irq(&dev_priv->irq_lock);
+ if (dev_priv->display_irqs_enabled)
+ vlv_display_irq_reset(dev_priv);
+ spin_unlock_irq(&dev_priv->irq_lock);
}
static u32 intel_hpd_enabled_irqs(struct drm_device *dev,
@@ -3506,6 +3617,26 @@
hotplug = I915_READ(PCH_PORT_HOTPLUG);
hotplug |= PORTC_HOTPLUG_ENABLE | PORTB_HOTPLUG_ENABLE |
PORTA_HOTPLUG_ENABLE;
+
+ DRM_DEBUG_KMS("Invert bit setting: hp_ctl:%x hp_port:%x\n",
+ hotplug, enabled_irqs);
+ hotplug &= ~BXT_DDI_HPD_INVERT_MASK;
+
+ /*
+ * For BXT invert bit has to be set based on AOB design
+ * for HPD detection logic, update it based on VBT fields.
+ */
+
+ if ((enabled_irqs & BXT_DE_PORT_HP_DDIA) &&
+ intel_bios_is_port_hpd_inverted(dev_priv, PORT_A))
+ hotplug |= BXT_DDIA_HPD_INVERT;
+ if ((enabled_irqs & BXT_DE_PORT_HP_DDIB) &&
+ intel_bios_is_port_hpd_inverted(dev_priv, PORT_B))
+ hotplug |= BXT_DDIB_HPD_INVERT;
+ if ((enabled_irqs & BXT_DE_PORT_HP_DDIC) &&
+ intel_bios_is_port_hpd_inverted(dev_priv, PORT_C))
+ hotplug |= BXT_DDIC_HPD_INVERT;
+
I915_WRITE(PCH_PORT_HOTPLUG, hotplug);
}
@@ -3613,74 +3744,6 @@
return 0;
}
-static void valleyview_display_irqs_install(struct drm_i915_private *dev_priv)
-{
- u32 pipestat_mask;
- u32 iir_mask;
- enum pipe pipe;
-
- pipestat_mask = PIPESTAT_INT_STATUS_MASK |
- PIPE_FIFO_UNDERRUN_STATUS;
-
- for_each_pipe(dev_priv, pipe)
- I915_WRITE(PIPESTAT(pipe), pipestat_mask);
- POSTING_READ(PIPESTAT(PIPE_A));
-
- pipestat_mask = PLANE_FLIP_DONE_INT_STATUS_VLV |
- PIPE_CRC_DONE_INTERRUPT_STATUS;
-
- i915_enable_pipestat(dev_priv, PIPE_A, PIPE_GMBUS_INTERRUPT_STATUS);
- for_each_pipe(dev_priv, pipe)
- i915_enable_pipestat(dev_priv, pipe, pipestat_mask);
-
- iir_mask = I915_DISPLAY_PORT_INTERRUPT |
- I915_DISPLAY_PIPE_A_EVENT_INTERRUPT |
- I915_DISPLAY_PIPE_B_EVENT_INTERRUPT;
- if (IS_CHERRYVIEW(dev_priv))
- iir_mask |= I915_DISPLAY_PIPE_C_EVENT_INTERRUPT;
- dev_priv->irq_mask &= ~iir_mask;
-
- I915_WRITE(VLV_IIR, iir_mask);
- I915_WRITE(VLV_IIR, iir_mask);
- I915_WRITE(VLV_IER, ~dev_priv->irq_mask);
- I915_WRITE(VLV_IMR, dev_priv->irq_mask);
- POSTING_READ(VLV_IMR);
-}
-
-static void valleyview_display_irqs_uninstall(struct drm_i915_private *dev_priv)
-{
- u32 pipestat_mask;
- u32 iir_mask;
- enum pipe pipe;
-
- iir_mask = I915_DISPLAY_PORT_INTERRUPT |
- I915_DISPLAY_PIPE_A_EVENT_INTERRUPT |
- I915_DISPLAY_PIPE_B_EVENT_INTERRUPT;
- if (IS_CHERRYVIEW(dev_priv))
- iir_mask |= I915_DISPLAY_PIPE_C_EVENT_INTERRUPT;
-
- dev_priv->irq_mask |= iir_mask;
- I915_WRITE(VLV_IMR, dev_priv->irq_mask);
- I915_WRITE(VLV_IER, ~dev_priv->irq_mask);
- I915_WRITE(VLV_IIR, iir_mask);
- I915_WRITE(VLV_IIR, iir_mask);
- POSTING_READ(VLV_IIR);
-
- pipestat_mask = PLANE_FLIP_DONE_INT_STATUS_VLV |
- PIPE_CRC_DONE_INTERRUPT_STATUS;
-
- i915_disable_pipestat(dev_priv, PIPE_A, PIPE_GMBUS_INTERRUPT_STATUS);
- for_each_pipe(dev_priv, pipe)
- i915_disable_pipestat(dev_priv, pipe, pipestat_mask);
-
- pipestat_mask = PIPESTAT_INT_STATUS_MASK |
- PIPE_FIFO_UNDERRUN_STATUS;
-
- for_each_pipe(dev_priv, pipe)
- I915_WRITE(PIPESTAT(pipe), pipestat_mask);
- POSTING_READ(PIPESTAT(PIPE_A));
-}
-
void valleyview_enable_display_irqs(struct drm_i915_private *dev_priv)
{
assert_spin_locked(&dev_priv->irq_lock);
@@ -3690,8 +3753,10 @@
dev_priv->display_irqs_enabled = true;
- if (intel_irqs_enabled(dev_priv))
- valleyview_display_irqs_install(dev_priv);
+ if (intel_irqs_enabled(dev_priv)) {
+ vlv_display_irq_reset(dev_priv);
+ vlv_display_irq_postinstall(dev_priv);
+ }
}
void valleyview_disable_display_irqs(struct drm_i915_private *dev_priv)
@@ -3704,45 +3769,23 @@
dev_priv->display_irqs_enabled = false;
if (intel_irqs_enabled(dev_priv))
- valleyview_display_irqs_uninstall(dev_priv);
+ vlv_display_irq_reset(dev_priv);
}
-static void vlv_display_irq_postinstall(struct drm_i915_private *dev_priv)
-{
- dev_priv->irq_mask = ~0;
-
- i915_hotplug_interrupt_update(dev_priv, 0xffffffff, 0);
- POSTING_READ(PORT_HOTPLUG_EN);
-
- I915_WRITE(VLV_IIR, 0xffffffff);
- I915_WRITE(VLV_IIR, 0xffffffff);
- I915_WRITE(VLV_IER, ~dev_priv->irq_mask);
- I915_WRITE(VLV_IMR, dev_priv->irq_mask);
- POSTING_READ(VLV_IMR);
-
- /* Interrupt setup is already guaranteed to be single-threaded, this is
- * just to make the assert_spin_locked check happy. */
- spin_lock_irq(&dev_priv->irq_lock);
- if (dev_priv->display_irqs_enabled)
- valleyview_display_irqs_install(dev_priv);
- spin_unlock_irq(&dev_priv->irq_lock);
-}
static int valleyview_irq_postinstall(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- vlv_display_irq_postinstall(dev_priv);
-
gen5_gt_irq_postinstall(dev);
- /* ack & enable invalid PTE error interrupts */
-#if 0 /* FIXME: add support to irq handler for checking these bits */
- I915_WRITE(DPINVGTT, DPINVGTT_STATUS_MASK);
- I915_WRITE(DPINVGTT, DPINVGTT_EN_MASK);
-#endif
+ spin_lock_irq(&dev_priv->irq_lock);
+ if (dev_priv->display_irqs_enabled)
+ vlv_display_irq_postinstall(dev_priv);
+ spin_unlock_irq(&dev_priv->irq_lock);
I915_WRITE(VLV_MASTER_IER, MASTER_INTERRUPT_ENABLE);
+ POSTING_READ(VLV_MASTER_IER);
return 0;
}
@@ -3753,7 +3796,6 @@
uint32_t gt_interrupts[] = {
GT_RENDER_USER_INTERRUPT << GEN8_RCS_IRQ_SHIFT |
GT_CONTEXT_SWITCH_INTERRUPT << GEN8_RCS_IRQ_SHIFT |
- GT_RENDER_L3_PARITY_ERROR_INTERRUPT |
GT_RENDER_USER_INTERRUPT << GEN8_BCS_IRQ_SHIFT |
GT_CONTEXT_SWITCH_INTERRUPT << GEN8_BCS_IRQ_SHIFT,
GT_RENDER_USER_INTERRUPT << GEN8_VCS1_IRQ_SHIFT |
@@ -3765,6 +3807,9 @@
GT_CONTEXT_SWITCH_INTERRUPT << GEN8_VECS_IRQ_SHIFT
};
+ if (HAS_L3_DPF(dev_priv))
+ gt_interrupts[0] |= GT_RENDER_L3_PARITY_ERROR_INTERRUPT;
+
dev_priv->pm_irq_mask = 0xffffffff;
GEN8_IRQ_INIT_NDX(GT, 0, ~gt_interrupts[0], gt_interrupts[0]);
GEN8_IRQ_INIT_NDX(GT, 1, ~gt_interrupts[1], gt_interrupts[1]);
@@ -3832,7 +3877,7 @@
if (HAS_PCH_SPLIT(dev))
ibx_irq_postinstall(dev);
- I915_WRITE(GEN8_MASTER_IRQ, DE_MASTER_IRQ_CONTROL);
+ I915_WRITE(GEN8_MASTER_IRQ, GEN8_MASTER_IRQ_CONTROL);
POSTING_READ(GEN8_MASTER_IRQ);
return 0;
@@ -3842,11 +3887,14 @@
{
struct drm_i915_private *dev_priv = dev->dev_private;
- vlv_display_irq_postinstall(dev_priv);
-
gen8_gt_irq_postinstall(dev_priv);
- I915_WRITE(GEN8_MASTER_IRQ, MASTER_INTERRUPT_ENABLE);
+ spin_lock_irq(&dev_priv->irq_lock);
+ if (dev_priv->display_irqs_enabled)
+ vlv_display_irq_postinstall(dev_priv);
+ spin_unlock_irq(&dev_priv->irq_lock);
+
+ I915_WRITE(GEN8_MASTER_IRQ, GEN8_MASTER_IRQ_CONTROL);
POSTING_READ(GEN8_MASTER_IRQ);
return 0;
@@ -3862,20 +3910,6 @@
gen8_irq_reset(dev);
}
-static void vlv_display_irq_uninstall(struct drm_i915_private *dev_priv)
-{
- /* Interrupt setup is already guaranteed to be single-threaded, this is
- * just to make the assert_spin_locked check happy. */
- spin_lock_irq(&dev_priv->irq_lock);
- if (dev_priv->display_irqs_enabled)
- valleyview_display_irqs_uninstall(dev_priv);
- spin_unlock_irq(&dev_priv->irq_lock);
-
- vlv_display_irq_reset(dev_priv);
-
- dev_priv->irq_mask = ~0;
-}
-
static void valleyview_irq_uninstall(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
@@ -3884,12 +3918,16 @@
return;
I915_WRITE(VLV_MASTER_IER, 0);
+ POSTING_READ(VLV_MASTER_IER);
gen5_gt_irq_reset(dev);
I915_WRITE(HWSTAM, 0xffffffff);
- vlv_display_irq_uninstall(dev_priv);
+ spin_lock_irq(&dev_priv->irq_lock);
+ if (dev_priv->display_irqs_enabled)
+ vlv_display_irq_reset(dev_priv);
+ spin_unlock_irq(&dev_priv->irq_lock);
}
static void cherryview_irq_uninstall(struct drm_device *dev)
@@ -3906,7 +3944,10 @@
GEN5_IRQ_RESET(GEN8_PCU_);
- vlv_display_irq_uninstall(dev_priv);
+ spin_lock_irq(&dev_priv->irq_lock);
+ if (dev_priv->display_irqs_enabled)
+ vlv_display_irq_reset(dev_priv);
+ spin_unlock_irq(&dev_priv->irq_lock);
}
static void ironlake_irq_uninstall(struct drm_device *dev)
@@ -4044,7 +4085,7 @@
new_iir = I915_READ16(IIR); /* Flush posted writes */
if (iir & I915_USER_INTERRUPT)
- notify_ring(&dev_priv->ring[RCS]);
+ notify_ring(&dev_priv->engine[RCS]);
for_each_pipe(dev_priv, pipe) {
int plane = pipe;
@@ -4233,14 +4274,17 @@
/* Consume port. Then clear IIR or we'll miss events */
if (I915_HAS_HOTPLUG(dev) &&
- iir & I915_DISPLAY_PORT_INTERRUPT)
- i9xx_hpd_irq_handler(dev);
+ iir & I915_DISPLAY_PORT_INTERRUPT) {
+ u32 hotplug_status = i9xx_hpd_irq_ack(dev_priv);
+ if (hotplug_status)
+ i9xx_hpd_irq_handler(dev, hotplug_status);
+ }
I915_WRITE(IIR, iir & ~flip_mask);
new_iir = I915_READ(IIR); /* Flush posted writes */
if (iir & I915_USER_INTERRUPT)
- notify_ring(&dev_priv->ring[RCS]);
+ notify_ring(&dev_priv->engine[RCS]);
for_each_pipe(dev_priv, pipe) {
int plane = pipe;
@@ -4463,16 +4507,19 @@
ret = IRQ_HANDLED;
/* Consume port. Then clear IIR or we'll miss events */
- if (iir & I915_DISPLAY_PORT_INTERRUPT)
- i9xx_hpd_irq_handler(dev);
+ if (iir & I915_DISPLAY_PORT_INTERRUPT) {
+ u32 hotplug_status = i9xx_hpd_irq_ack(dev_priv);
+ if (hotplug_status)
+ i9xx_hpd_irq_handler(dev, hotplug_status);
+ }
I915_WRITE(IIR, iir & ~flip_mask);
new_iir = I915_READ(IIR); /* Flush posted writes */
if (iir & I915_USER_INTERRUPT)
- notify_ring(&dev_priv->ring[RCS]);
+ notify_ring(&dev_priv->engine[RCS]);
if (iir & I915_BSD_USER_INTERRUPT)
- notify_ring(&dev_priv->ring[VCS]);
+ notify_ring(&dev_priv->engine[VCS]);
for_each_pipe(dev_priv, pipe) {
if (pipe_stats[pipe] & PIPE_START_VBLANK_INTERRUPT_STATUS &&
@@ -4567,8 +4614,6 @@
INIT_DELAYED_WORK(&dev_priv->gpu_error.hangcheck_work,
i915_hangcheck_elapsed);
- pm_qos_add_request(&dev_priv->pm_qos, PM_QOS_CPU_DMA_LATENCY, PM_QOS_DEFAULT_VALUE);
-
if (IS_GEN2(dev_priv)) {
dev->max_vblank_count = 0;
dev->driver->get_vblank_counter = i8xx_get_vblank_counter;
diff --git a/drivers/gpu/drm/i915/i915_params.c b/drivers/gpu/drm/i915/i915_params.c
index 278c9c4..1779f02 100644
--- a/drivers/gpu/drm/i915/i915_params.c
+++ b/drivers/gpu/drm/i915/i915_params.c
@@ -56,6 +56,8 @@
.edp_vswing = 0,
.enable_guc_submission = false,
.guc_log_level = -1,
+ .enable_dp_mst = true,
+ .inject_load_failure = 0,
};
module_param_named(modeset, i915.modeset, int, 0400);
@@ -201,3 +203,10 @@
module_param_named(guc_log_level, i915.guc_log_level, int, 0400);
MODULE_PARM_DESC(guc_log_level,
"GuC firmware logging level (-1:disabled (default), 0-3:enabled)");
+
+module_param_named_unsafe(enable_dp_mst, i915.enable_dp_mst, bool, 0600);
+MODULE_PARM_DESC(enable_dp_mst,
+ "Enable multi-stream transport (MST) for new DisplayPort sinks. (default: true)");
+module_param_named_unsafe(inject_load_failure, i915.inject_load_failure, uint, 0400);
+MODULE_PARM_DESC(inject_load_failure,
+ "Force an error after a number of failure check points (0:disabled (default), N:force failure at the Nth failure check point)");
diff --git a/drivers/gpu/drm/i915/i915_params.h b/drivers/gpu/drm/i915/i915_params.h
index bd5026b..02bc278 100644
--- a/drivers/gpu/drm/i915/i915_params.h
+++ b/drivers/gpu/drm/i915/i915_params.h
@@ -49,6 +49,7 @@
int use_mmio_flip;
int mmio_debug;
int edp_vswing;
+ unsigned int inject_load_failure;
/* leave bools at the end to not create holes */
bool enable_hangcheck;
bool fastboot;
@@ -59,6 +60,7 @@
bool enable_guc_submission;
bool verbose_state_checks;
bool nuclear_pageflip;
+ bool enable_dp_mst;
};
extern struct i915_params i915 __read_mostly;
diff --git a/drivers/gpu/drm/i915/i915_reg.h b/drivers/gpu/drm/i915/i915_reg.h
index 363bd79..b407411 100644
--- a/drivers/gpu/drm/i915/i915_reg.h
+++ b/drivers/gpu/drm/i915/i915_reg.h
@@ -79,6 +79,16 @@
/* PCI config space */
+#define MCHBAR_I915 0x44
+#define MCHBAR_I965 0x48
+#define MCHBAR_SIZE (4 * 4096)
+
+#define DEVEN 0x54
+#define DEVEN_MCHBAR_EN (1 << 28)
+
+#define BSM 0x5c
+#define BSM_MASK (0xFFFF << 20)
+
#define HPLLCC 0xc0 /* 85x only */
#define GC_CLOCK_CONTROL_MASK (0x7 << 0)
#define GC_CLOCK_133_200 (0 << 0)
@@ -90,6 +100,16 @@
#define GC_CLOCK_166_266 (6 << 0)
#define GC_CLOCK_166_250 (7 << 0)
+#define I915_GDRST 0xc0 /* PCI config register */
+#define GRDOM_FULL (0 << 2)
+#define GRDOM_RENDER (1 << 2)
+#define GRDOM_MEDIA (3 << 2)
+#define GRDOM_MASK (3 << 2)
+#define GRDOM_RESET_STATUS (1 << 1)
+#define GRDOM_RESET_ENABLE (1 << 0)
+
+#define GCDGMBUS 0xcc
+
#define GCFGC2 0xda
#define GCFGC 0xf0 /* 915+ only */
#define GC_LOW_FREQUENCY_ENABLE (1 << 7)
@@ -121,18 +141,16 @@
#define I915_GC_RENDER_CLOCK_166_MHZ (0 << 0)
#define I915_GC_RENDER_CLOCK_200_MHZ (1 << 0)
#define I915_GC_RENDER_CLOCK_333_MHZ (4 << 0)
-#define GCDGMBUS 0xcc
-#define PCI_LBPC 0xf4 /* legacy/combination backlight modes, also called LBB */
+#define ASLE 0xe4
+#define ASLS 0xfc
-/* Graphics reset regs */
-#define I915_GDRST 0xc0 /* PCI config register */
-#define GRDOM_FULL (0<<2)
-#define GRDOM_RENDER (1<<2)
-#define GRDOM_MEDIA (3<<2)
-#define GRDOM_MASK (3<<2)
-#define GRDOM_RESET_STATUS (1<<1)
-#define GRDOM_RESET_ENABLE (1<<0)
+#define SWSCI 0xe8
+#define SWSCI_SCISEL (1 << 15)
+#define SWSCI_GSSCIE (1 << 0)
+
+#define LBPC 0xf4 /* legacy/combination backlight modes, also called LBB */
+
#define ILK_GDSR _MMIO(MCHBAR_MIRROR_BASE + 0x2ca4)
#define ILK_GRDOM_FULL (0<<1)
@@ -164,6 +182,9 @@
#define GEN6_GRDOM_RENDER (1 << 1)
#define GEN6_GRDOM_MEDIA (1 << 2)
#define GEN6_GRDOM_BLT (1 << 3)
+#define GEN6_GRDOM_VECS (1 << 4)
+#define GEN9_GRDOM_GUC (1 << 5)
+#define GEN8_GRDOM_MEDIA2 (1 << 7)
#define RING_PP_DIR_BASE(ring) _MMIO((ring)->mmio_base+0x228)
#define RING_PP_DIR_BASE_READ(ring) _MMIO((ring)->mmio_base+0x518)
@@ -586,6 +607,10 @@
#define GEN7_GPGPU_DISPATCHDIMY _MMIO(0x2504)
#define GEN7_GPGPU_DISPATCHDIMZ _MMIO(0x2508)
+/* There are the 16 64-bit CS General Purpose Registers */
+#define HSW_CS_GPR(n) _MMIO(0x2600 + (n) * 8)
+#define HSW_CS_GPR_UDW(n) _MMIO(0x2600 + (n) * 8 + 4)
+
#define OACONTROL _MMIO(0x2360)
#define _GEN7_PIPEA_DE_LOAD_SL 0x70068
@@ -621,6 +646,10 @@
#define IOSF_PORT_GPIO_SC 0x48
#define IOSF_PORT_GPIO_SUS 0xa8
#define IOSF_PORT_CCU 0xa9
+#define CHV_IOSF_PORT_GPIO_N 0x13
+#define CHV_IOSF_PORT_GPIO_SE 0x48
+#define CHV_IOSF_PORT_GPIO_E 0xa8
+#define CHV_IOSF_PORT_GPIO_SW 0xb2
#define VLV_IOSF_DATA _MMIO(VLV_DISPLAY_BASE + 0x2104)
#define VLV_IOSF_ADDR _MMIO(VLV_DISPLAY_BASE + 0x2108)
@@ -785,7 +814,9 @@
#define DSI_PLL_M1_DIV_SHIFT 0
#define DSI_PLL_M1_DIV_MASK (0x1ff << 0)
#define CCK_CZ_CLOCK_CONTROL 0x62
+#define CCK_GPLL_CLOCK_CONTROL 0x67
#define CCK_DISPLAY_CLOCK_CONTROL 0x6b
+#define CCK_DISPLAY_REF_CLOCK_CONTROL 0x6c
#define CCK_TRUNK_FORCE_ON (1 << 17)
#define CCK_TRUNK_FORCE_OFF (1 << 16)
#define CCK_FREQUENCY_STATUS (0x1f << 8)
@@ -1317,6 +1348,7 @@
#define _PORT_CL1CM_DW0_A 0x162000
#define _PORT_CL1CM_DW0_BC 0x6C000
#define PHY_POWER_GOOD (1 << 16)
+#define PHY_RESERVED (1 << 7)
#define BXT_PORT_CL1CM_DW0(phy) _BXT_PHY((phy), _PORT_CL1CM_DW0_BC, \
_PORT_CL1CM_DW0_A)
@@ -1361,14 +1393,10 @@
#define _PORT_REF_DW6_A 0x162198
#define _PORT_REF_DW6_BC 0x6C198
-/*
- * FIXME: BSpec/CHV ConfigDB disagrees on the following two fields, fix them
- * after testing.
- */
-#define GRC_CODE_SHIFT 23
-#define GRC_CODE_MASK (0x1FF << GRC_CODE_SHIFT)
+#define GRC_CODE_SHIFT 24
+#define GRC_CODE_MASK (0xFF << GRC_CODE_SHIFT)
#define GRC_CODE_FAST_SHIFT 16
-#define GRC_CODE_FAST_MASK (0x7F << GRC_CODE_FAST_SHIFT)
+#define GRC_CODE_FAST_MASK (0xFF << GRC_CODE_FAST_SHIFT)
#define GRC_CODE_SLOW_SHIFT 8
#define GRC_CODE_SLOW_MASK (0xFF << GRC_CODE_SLOW_SHIFT)
#define GRC_CODE_NOM_MASK 0xFF
@@ -1776,6 +1804,18 @@
#define GEN9_IZ_HASHING_MASK(slice) (0x3 << ((slice) * 2))
#define GEN9_IZ_HASHING(slice, val) ((val) << ((slice) * 2))
+/* WaClearTdlStateAckDirtyBits */
+#define GEN8_STATE_ACK _MMIO(0x20F0)
+#define GEN9_STATE_ACK_SLICE1 _MMIO(0x20F8)
+#define GEN9_STATE_ACK_SLICE2 _MMIO(0x2100)
+#define GEN9_STATE_ACK_TDL0 (1 << 12)
+#define GEN9_STATE_ACK_TDL1 (1 << 13)
+#define GEN9_STATE_ACK_TDL2 (1 << 14)
+#define GEN9_STATE_ACK_TDL3 (1 << 15)
+#define GEN9_SUBSLICE_TDL_ACK_BITS \
+ (GEN9_STATE_ACK_TDL3 | GEN9_STATE_ACK_TDL2 | \
+ GEN9_STATE_ACK_TDL1 | GEN9_STATE_ACK_TDL0)
+
#define GFX_MODE _MMIO(0x2520)
#define GFX_MODE_GEN7 _MMIO(0x229c)
#define RING_MODE_GEN7(ring) _MMIO((ring)->mmio_base+0x29c)
@@ -1795,6 +1835,7 @@
#define VLV_DISPLAY_BASE 0x180000
#define VLV_MIPI_BASE VLV_DISPLAY_BASE
+#define BXT_MIPI_BASE 0x60000
#define VLV_GU_CTL0 _MMIO(VLV_DISPLAY_BASE + 0x2030)
#define VLV_GU_CTL1 _MMIO(VLV_DISPLAY_BASE + 0x2034)
@@ -2923,6 +2964,15 @@
INTERVAL_1_33_US(us)) : \
INTERVAL_1_28_US(us))
+#define INTERVAL_1_28_TO_US(interval) (((interval) << 7) / 100)
+#define INTERVAL_1_33_TO_US(interval) (((interval) << 2) / 3)
+#define INTERVAL_0_833_TO_US(interval) (((interval) * 5) / 6)
+#define GT_PM_INTERVAL_TO_US(dev_priv, interval) (IS_GEN9(dev_priv) ? \
+ (IS_BROXTON(dev_priv) ? \
+ INTERVAL_0_833_TO_US(interval) : \
+ INTERVAL_1_33_TO_US(interval)) : \
+ INTERVAL_1_28_TO_US(interval))
+
/*
* Logical Context regs
*/
@@ -4784,6 +4834,10 @@
#define CBR_PND_DEADLINE_DISABLE (1<<31)
#define CBR_PWM_CLOCK_MUX_SELECT (1<<30)
+#define CBR4_VLV _MMIO(VLV_DISPLAY_BASE + 0x70450)
+#define CBR_DPLLBMD_PIPE_C (1<<29)
+#define CBR_DPLLBMD_PIPE_B (1<<18)
+
/* FIFO watermark sizes etc */
#define G4X_FIFO_LINE_SIZE 64
#define I915_FIFO_LINE_SIZE 64
@@ -6184,6 +6238,7 @@
/* digital port hotplug */
#define PCH_PORT_HOTPLUG _MMIO(0xc4030) /* SHOTPLUG_CTL */
#define PORTA_HOTPLUG_ENABLE (1 << 28) /* LPT:LP+ & BXT */
+#define BXT_DDIA_HPD_INVERT (1 << 27)
#define PORTA_HOTPLUG_STATUS_MASK (3 << 24) /* SPT+ & BXT */
#define PORTA_HOTPLUG_NO_DETECT (0 << 24) /* SPT+ & BXT */
#define PORTA_HOTPLUG_SHORT_DETECT (1 << 24) /* SPT+ & BXT */
@@ -6199,6 +6254,7 @@
#define PORTD_HOTPLUG_SHORT_DETECT (1 << 16)
#define PORTD_HOTPLUG_LONG_DETECT (2 << 16)
#define PORTC_HOTPLUG_ENABLE (1 << 12)
+#define BXT_DDIC_HPD_INVERT (1 << 11)
#define PORTC_PULSE_DURATION_2ms (0 << 10) /* pre-LPT */
#define PORTC_PULSE_DURATION_4_5ms (1 << 10) /* pre-LPT */
#define PORTC_PULSE_DURATION_6ms (2 << 10) /* pre-LPT */
@@ -6209,6 +6265,7 @@
#define PORTC_HOTPLUG_SHORT_DETECT (1 << 8)
#define PORTC_HOTPLUG_LONG_DETECT (2 << 8)
#define PORTB_HOTPLUG_ENABLE (1 << 4)
+#define BXT_DDIB_HPD_INVERT (1 << 3)
#define PORTB_PULSE_DURATION_2ms (0 << 2) /* pre-LPT */
#define PORTB_PULSE_DURATION_4_5ms (1 << 2) /* pre-LPT */
#define PORTB_PULSE_DURATION_6ms (2 << 2) /* pre-LPT */
@@ -6218,6 +6275,9 @@
#define PORTB_HOTPLUG_NO_DETECT (0 << 0)
#define PORTB_HOTPLUG_SHORT_DETECT (1 << 0)
#define PORTB_HOTPLUG_LONG_DETECT (2 << 0)
+#define BXT_DDI_HPD_INVERT_MASK (BXT_DDIA_HPD_INVERT | \
+ BXT_DDIB_HPD_INVERT | \
+ BXT_DDIC_HPD_INVERT)
#define PCH_PORT_HOTPLUG2 _MMIO(0xc403C) /* SHOTPLUG_CTL2 SPT+ */
#define PORTE_HOTPLUG_ENABLE (1 << 4)
@@ -6836,6 +6896,8 @@
#define VLV_SPAREG2H _MMIO(0xA194)
#define GTFIFODBG _MMIO(0x120000)
+#define GT_FIFO_SBDEDICATE_FREE_ENTRY_CHV (0x1f << 20)
+#define GT_FIFO_FREE_ENTRIES_CHV (0x7f << 13)
#define GT_FIFO_SBDROPERR (1<<6)
#define GT_FIFO_BLOBDROPERR (1<<5)
#define GT_FIFO_SB_READ_ABORTERR (1<<4)
@@ -6852,8 +6914,11 @@
#define HSW_IDICR _MMIO(0x9008)
#define IDIHASHMSK(x) (((x) & 0x3f) << 16)
-#define HSW_EDRAM_PRESENT _MMIO(0x120010)
+#define HSW_EDRAM_CAP _MMIO(0x120010)
#define EDRAM_ENABLED 0x1
+#define EDRAM_NUM_BANKS(cap) (((cap) >> 1) & 0xf)
+#define EDRAM_WAYS_IDX(cap) (((cap) >> 5) & 0x7)
+#define EDRAM_SETS_IDX(cap) (((cap) >> 8) & 0x3)
#define GEN6_UCGCTL1 _MMIO(0x9400)
# define GEN6_EU_TCUNIT_CLOCK_GATE_DISABLE (1 << 16)
@@ -7109,6 +7174,7 @@
#define GEN9_CCS_TLB_PREFETCH_ENABLE (1<<3)
#define GEN8_ROW_CHICKEN _MMIO(0xe4f0)
+#define FLOW_CONTROL_ENABLE (1<<15)
#define PARTIAL_INSTRUCTION_SHOOTDOWN_DISABLE (1<<8)
#define STALL_DOP_GATING_DISABLE (1<<5)
@@ -7130,6 +7196,7 @@
#define GEN9_HALF_SLICE_CHICKEN7 _MMIO(0xe194)
#define GEN9_ENABLE_YV12_BUGFIX (1<<4)
+#define GEN9_ENABLE_GPGPU_PREEMPTION (1<<2)
/* Audio */
#define G4X_AUD_VID_DID _MMIO(dev_priv->info.display_mmio_offset + 0x62020)
@@ -7369,9 +7436,11 @@
/* SBI offsets */
#define SBI_SSCDIVINTPHASE 0x0200
#define SBI_SSCDIVINTPHASE6 0x0600
-#define SBI_SSCDIVINTPHASE_DIVSEL_MASK ((0x7f)<<1)
+#define SBI_SSCDIVINTPHASE_DIVSEL_SHIFT 1
+#define SBI_SSCDIVINTPHASE_DIVSEL_MASK (0x7f<<1)
#define SBI_SSCDIVINTPHASE_DIVSEL(x) ((x)<<1)
-#define SBI_SSCDIVINTPHASE_INCVAL_MASK ((0x7f)<<8)
+#define SBI_SSCDIVINTPHASE_INCVAL_SHIFT 8
+#define SBI_SSCDIVINTPHASE_INCVAL_MASK (0x7f<<8)
#define SBI_SSCDIVINTPHASE_INCVAL(x) ((x)<<8)
#define SBI_SSCDIVINTPHASE_DIR(x) ((x)<<15)
#define SBI_SSCDIVINTPHASE_PROPAGATE (1<<0)
@@ -7381,6 +7450,8 @@
#define SBI_SSCCTL_PATHALT (1<<3)
#define SBI_SSCCTL_DISABLE (1<<0)
#define SBI_SSCAUXDIV6 0x0610
+#define SBI_SSCAUXDIV_FINALDIV2SEL_SHIFT 4
+#define SBI_SSCAUXDIV_FINALDIV2SEL_MASK (1<<4)
#define SBI_SSCAUXDIV_FINALDIV2SEL(x) ((x)<<4)
#define SBI_DBUFF0 0x2a00
#define SBI_GEN0 0x1f00
@@ -7660,6 +7731,59 @@
#define PIPE_CSC_POSTOFF_ME(pipe) _MMIO_PIPE(pipe, _PIPE_A_CSC_POSTOFF_ME, _PIPE_B_CSC_POSTOFF_ME)
#define PIPE_CSC_POSTOFF_LO(pipe) _MMIO_PIPE(pipe, _PIPE_A_CSC_POSTOFF_LO, _PIPE_B_CSC_POSTOFF_LO)
+/* pipe degamma/gamma LUTs on IVB+ */
+#define _PAL_PREC_INDEX_A 0x4A400
+#define _PAL_PREC_INDEX_B 0x4AC00
+#define _PAL_PREC_INDEX_C 0x4B400
+#define PAL_PREC_10_12_BIT (0 << 31)
+#define PAL_PREC_SPLIT_MODE (1 << 31)
+#define PAL_PREC_AUTO_INCREMENT (1 << 15)
+#define _PAL_PREC_DATA_A 0x4A404
+#define _PAL_PREC_DATA_B 0x4AC04
+#define _PAL_PREC_DATA_C 0x4B404
+#define _PAL_PREC_GC_MAX_A 0x4A410
+#define _PAL_PREC_GC_MAX_B 0x4AC10
+#define _PAL_PREC_GC_MAX_C 0x4B410
+#define _PAL_PREC_EXT_GC_MAX_A 0x4A420
+#define _PAL_PREC_EXT_GC_MAX_B 0x4AC20
+#define _PAL_PREC_EXT_GC_MAX_C 0x4B420
+
+#define PREC_PAL_INDEX(pipe) _MMIO_PIPE(pipe, _PAL_PREC_INDEX_A, _PAL_PREC_INDEX_B)
+#define PREC_PAL_DATA(pipe) _MMIO_PIPE(pipe, _PAL_PREC_DATA_A, _PAL_PREC_DATA_B)
+#define PREC_PAL_GC_MAX(pipe, i) _MMIO(_PIPE(pipe, _PAL_PREC_GC_MAX_A, _PAL_PREC_GC_MAX_B) + (i) * 4)
+#define PREC_PAL_EXT_GC_MAX(pipe, i) _MMIO(_PIPE(pipe, _PAL_PREC_EXT_GC_MAX_A, _PAL_PREC_EXT_GC_MAX_B) + (i) * 4)
+
+/* pipe CSC & degamma/gamma LUTs on CHV */
+#define _CGM_PIPE_A_CSC_COEFF01 (VLV_DISPLAY_BASE + 0x67900)
+#define _CGM_PIPE_A_CSC_COEFF23 (VLV_DISPLAY_BASE + 0x67904)
+#define _CGM_PIPE_A_CSC_COEFF45 (VLV_DISPLAY_BASE + 0x67908)
+#define _CGM_PIPE_A_CSC_COEFF67 (VLV_DISPLAY_BASE + 0x6790C)
+#define _CGM_PIPE_A_CSC_COEFF8 (VLV_DISPLAY_BASE + 0x67910)
+#define _CGM_PIPE_A_DEGAMMA (VLV_DISPLAY_BASE + 0x66000)
+#define _CGM_PIPE_A_GAMMA (VLV_DISPLAY_BASE + 0x67000)
+#define _CGM_PIPE_A_MODE (VLV_DISPLAY_BASE + 0x67A00)
+#define CGM_PIPE_MODE_GAMMA (1 << 2)
+#define CGM_PIPE_MODE_CSC (1 << 1)
+#define CGM_PIPE_MODE_DEGAMMA (1 << 0)
+
+#define _CGM_PIPE_B_CSC_COEFF01 (VLV_DISPLAY_BASE + 0x69900)
+#define _CGM_PIPE_B_CSC_COEFF23 (VLV_DISPLAY_BASE + 0x69904)
+#define _CGM_PIPE_B_CSC_COEFF45 (VLV_DISPLAY_BASE + 0x69908)
+#define _CGM_PIPE_B_CSC_COEFF67 (VLV_DISPLAY_BASE + 0x6990C)
+#define _CGM_PIPE_B_CSC_COEFF8 (VLV_DISPLAY_BASE + 0x69910)
+#define _CGM_PIPE_B_DEGAMMA (VLV_DISPLAY_BASE + 0x68000)
+#define _CGM_PIPE_B_GAMMA (VLV_DISPLAY_BASE + 0x69000)
+#define _CGM_PIPE_B_MODE (VLV_DISPLAY_BASE + 0x69A00)
+
+#define CGM_PIPE_CSC_COEFF01(pipe) _MMIO_PIPE(pipe, _CGM_PIPE_A_CSC_COEFF01, _CGM_PIPE_B_CSC_COEFF01)
+#define CGM_PIPE_CSC_COEFF23(pipe) _MMIO_PIPE(pipe, _CGM_PIPE_A_CSC_COEFF23, _CGM_PIPE_B_CSC_COEFF23)
+#define CGM_PIPE_CSC_COEFF45(pipe) _MMIO_PIPE(pipe, _CGM_PIPE_A_CSC_COEFF45, _CGM_PIPE_B_CSC_COEFF45)
+#define CGM_PIPE_CSC_COEFF67(pipe) _MMIO_PIPE(pipe, _CGM_PIPE_A_CSC_COEFF67, _CGM_PIPE_B_CSC_COEFF67)
+#define CGM_PIPE_CSC_COEFF8(pipe) _MMIO_PIPE(pipe, _CGM_PIPE_A_CSC_COEFF8, _CGM_PIPE_B_CSC_COEFF8)
+#define CGM_PIPE_DEGAMMA(pipe, i, w) _MMIO(_PIPE(pipe, _CGM_PIPE_A_DEGAMMA, _CGM_PIPE_B_DEGAMMA) + (i) * 8 + (w) * 4)
+#define CGM_PIPE_GAMMA(pipe, i, w) _MMIO(_PIPE(pipe, _CGM_PIPE_A_GAMMA, _CGM_PIPE_B_GAMMA) + (i) * 8 + (w) * 4)
+#define CGM_PIPE_MODE(pipe) _MMIO_PIPE(pipe, _CGM_PIPE_A_MODE, _CGM_PIPE_B_MODE)
+
/* MIPI DSI registers */
#define _MIPI_PORT(port, a, c) _PORT3(port, a, 0, c) /* ports A and C only */
@@ -7674,58 +7798,62 @@
#define BXT_MIPI_DIV_SHIFT(port) \
_MIPI_PORT(port, BXT_MIPI1_DIV_SHIFT, \
BXT_MIPI2_DIV_SHIFT)
-/* Var clock divider to generate TX source. Result must be < 39.5 M */
-#define BXT_MIPI1_ESCLK_VAR_DIV_MASK (0x3F << 26)
-#define BXT_MIPI2_ESCLK_VAR_DIV_MASK (0x3F << 10)
-#define BXT_MIPI_ESCLK_VAR_DIV_MASK(port) \
- _MIPI_PORT(port, BXT_MIPI1_ESCLK_VAR_DIV_MASK, \
- BXT_MIPI2_ESCLK_VAR_DIV_MASK)
-#define BXT_MIPI_ESCLK_VAR_DIV(port, val) \
- (val << BXT_MIPI_DIV_SHIFT(port))
/* TX control divider to select actual TX clock output from (8x/var) */
-#define BXT_MIPI1_TX_ESCLK_SHIFT 21
-#define BXT_MIPI2_TX_ESCLK_SHIFT 5
+#define BXT_MIPI1_TX_ESCLK_SHIFT 26
+#define BXT_MIPI2_TX_ESCLK_SHIFT 10
#define BXT_MIPI_TX_ESCLK_SHIFT(port) \
_MIPI_PORT(port, BXT_MIPI1_TX_ESCLK_SHIFT, \
BXT_MIPI2_TX_ESCLK_SHIFT)
-#define BXT_MIPI1_TX_ESCLK_FIXDIV_MASK (3 << 21)
-#define BXT_MIPI2_TX_ESCLK_FIXDIV_MASK (3 << 5)
+#define BXT_MIPI1_TX_ESCLK_FIXDIV_MASK (0x3F << 26)
+#define BXT_MIPI2_TX_ESCLK_FIXDIV_MASK (0x3F << 10)
#define BXT_MIPI_TX_ESCLK_FIXDIV_MASK(port) \
_MIPI_PORT(port, BXT_MIPI1_TX_ESCLK_FIXDIV_MASK, \
- BXT_MIPI2_TX_ESCLK_FIXDIV_MASK)
-#define BXT_MIPI_TX_ESCLK_8XDIV_BY2(port) \
- (0x0 << BXT_MIPI_TX_ESCLK_SHIFT(port))
-#define BXT_MIPI_TX_ESCLK_8XDIV_BY4(port) \
- (0x1 << BXT_MIPI_TX_ESCLK_SHIFT(port))
-#define BXT_MIPI_TX_ESCLK_8XDIV_BY8(port) \
- (0x2 << BXT_MIPI_TX_ESCLK_SHIFT(port))
-/* RX control divider to select actual RX clock output from 8x*/
-#define BXT_MIPI1_RX_ESCLK_SHIFT 19
-#define BXT_MIPI2_RX_ESCLK_SHIFT 3
-#define BXT_MIPI_RX_ESCLK_SHIFT(port) \
- _MIPI_PORT(port, BXT_MIPI1_RX_ESCLK_SHIFT, \
- BXT_MIPI2_RX_ESCLK_SHIFT)
-#define BXT_MIPI1_RX_ESCLK_FIXDIV_MASK (3 << 19)
-#define BXT_MIPI2_RX_ESCLK_FIXDIV_MASK (3 << 3)
-#define BXT_MIPI_RX_ESCLK_FIXDIV_MASK(port) \
- (3 << BXT_MIPI_RX_ESCLK_SHIFT(port))
-#define BXT_MIPI_RX_ESCLK_8X_BY2(port) \
- (1 << BXT_MIPI_RX_ESCLK_SHIFT(port))
-#define BXT_MIPI_RX_ESCLK_8X_BY3(port) \
- (2 << BXT_MIPI_RX_ESCLK_SHIFT(port))
-#define BXT_MIPI_RX_ESCLK_8X_BY4(port) \
- (3 << BXT_MIPI_RX_ESCLK_SHIFT(port))
-/* BXT-A WA: Always prog DPHY dividers to 00 */
-#define BXT_MIPI1_DPHY_DIV_SHIFT 16
-#define BXT_MIPI2_DPHY_DIV_SHIFT 0
-#define BXT_MIPI_DPHY_DIV_SHIFT(port) \
- _MIPI_PORT(port, BXT_MIPI1_DPHY_DIV_SHIFT, \
- BXT_MIPI2_DPHY_DIV_SHIFT)
-#define BXT_MIPI_1_DPHY_DIVIDER_MASK (3 << 16)
-#define BXT_MIPI_2_DPHY_DIVIDER_MASK (3 << 0)
-#define BXT_MIPI_DPHY_DIVIDER_MASK(port) \
- (3 << BXT_MIPI_DPHY_DIV_SHIFT(port))
+ BXT_MIPI2_TX_ESCLK_FIXDIV_MASK)
+#define BXT_MIPI_TX_ESCLK_DIVIDER(port, val) \
+ ((val & 0x3F) << BXT_MIPI_TX_ESCLK_SHIFT(port))
+/* RX upper control divider to select actual RX clock output from 8x */
+#define BXT_MIPI1_RX_ESCLK_UPPER_SHIFT 21
+#define BXT_MIPI2_RX_ESCLK_UPPER_SHIFT 5
+#define BXT_MIPI_RX_ESCLK_UPPER_SHIFT(port) \
+ _MIPI_PORT(port, BXT_MIPI1_RX_ESCLK_UPPER_SHIFT, \
+ BXT_MIPI2_RX_ESCLK_UPPER_SHIFT)
+#define BXT_MIPI1_RX_ESCLK_UPPER_FIXDIV_MASK (3 << 21)
+#define BXT_MIPI2_RX_ESCLK_UPPER_FIXDIV_MASK (3 << 5)
+#define BXT_MIPI_RX_ESCLK_UPPER_FIXDIV_MASK(port) \
+ _MIPI_PORT(port, BXT_MIPI1_RX_ESCLK_UPPER_FIXDIV_MASK, \
+ BXT_MIPI2_RX_ESCLK_UPPER_FIXDIV_MASK)
+#define BXT_MIPI_RX_ESCLK_UPPER_DIVIDER(port, val) \
+ ((val & 3) << BXT_MIPI_RX_ESCLK_UPPER_SHIFT(port))
+/* 8/3X divider to select the actual 8/3X clock output from 8x */
+#define BXT_MIPI1_8X_BY3_SHIFT 19
+#define BXT_MIPI2_8X_BY3_SHIFT 3
+#define BXT_MIPI_8X_BY3_SHIFT(port) \
+ _MIPI_PORT(port, BXT_MIPI1_8X_BY3_SHIFT, \
+ BXT_MIPI2_8X_BY3_SHIFT)
+#define BXT_MIPI1_8X_BY3_DIVIDER_MASK (3 << 19)
+#define BXT_MIPI2_8X_BY3_DIVIDER_MASK (3 << 3)
+#define BXT_MIPI_8X_BY3_DIVIDER_MASK(port) \
+ _MIPI_PORT(port, BXT_MIPI1_8X_BY3_DIVIDER_MASK, \
+ BXT_MIPI2_8X_BY3_DIVIDER_MASK)
+#define BXT_MIPI_8X_BY3_DIVIDER(port, val) \
+ ((val & 3) << BXT_MIPI_8X_BY3_SHIFT(port))
+/* RX lower control divider to select actual RX clock output from 8x */
+#define BXT_MIPI1_RX_ESCLK_LOWER_SHIFT 16
+#define BXT_MIPI2_RX_ESCLK_LOWER_SHIFT 0
+#define BXT_MIPI_RX_ESCLK_LOWER_SHIFT(port) \
+ _MIPI_PORT(port, BXT_MIPI1_RX_ESCLK_LOWER_SHIFT, \
+ BXT_MIPI2_RX_ESCLK_LOWER_SHIFT)
+#define BXT_MIPI1_RX_ESCLK_LOWER_FIXDIV_MASK (3 << 16)
+#define BXT_MIPI2_RX_ESCLK_LOWER_FIXDIV_MASK (3 << 0)
+#define BXT_MIPI_RX_ESCLK_LOWER_FIXDIV_MASK(port) \
+ _MIPI_PORT(port, BXT_MIPI1_RX_ESCLK_LOWER_FIXDIV_MASK, \
+ BXT_MIPI2_RX_ESCLK_LOWER_FIXDIV_MASK)
+#define BXT_MIPI_RX_ESCLK_LOWER_DIVIDER(port, val) \
+ ((val & 3) << BXT_MIPI_RX_ESCLK_LOWER_SHIFT(port))
+
+#define RX_DIVIDER_BIT_1_2 0x3
+#define RX_DIVIDER_BIT_3_4 0xC
/* BXT MIPI mode configure */
#define _BXT_MIPIA_TRANS_HACTIVE 0x6B0F8
@@ -7750,9 +7878,11 @@
#define BXT_DSIC_16X_BY2 (1 << 10)
#define BXT_DSIC_16X_BY3 (2 << 10)
#define BXT_DSIC_16X_BY4 (3 << 10)
+#define BXT_DSIC_16X_MASK (3 << 10)
#define BXT_DSIA_16X_BY2 (1 << 8)
#define BXT_DSIA_16X_BY3 (2 << 8)
#define BXT_DSIA_16X_BY4 (3 << 8)
+#define BXT_DSIA_16X_MASK (3 << 8)
#define BXT_DSI_FREQ_SEL_SHIFT 8
#define BXT_DSI_FREQ_SEL_MASK (0xF << BXT_DSI_FREQ_SEL_SHIFT)
@@ -7887,8 +8017,8 @@
#define VID_MODE_FORMAT_MASK (0xf << 7)
#define VID_MODE_NOT_SUPPORTED (0 << 7)
#define VID_MODE_FORMAT_RGB565 (1 << 7)
-#define VID_MODE_FORMAT_RGB666 (2 << 7)
-#define VID_MODE_FORMAT_RGB666_LOOSE (3 << 7)
+#define VID_MODE_FORMAT_RGB666_PACKED (2 << 7)
+#define VID_MODE_FORMAT_RGB666 (3 << 7)
#define VID_MODE_FORMAT_RGB888 (4 << 7)
#define CMD_MODE_CHANNEL_NUMBER_SHIFT 5
#define CMD_MODE_CHANNEL_NUMBER_MASK (3 << 5)
@@ -8144,6 +8274,7 @@
#define READ_REQUEST_PRIORITY_HIGH (3 << 3)
#define RGB_FLIP_TO_BGR (1 << 2)
+#define BXT_PIPE_SELECT_SHIFT 7
#define BXT_PIPE_SELECT_MASK (7 << 7)
#define BXT_PIPE_SELECT(pipe) ((pipe) << 7)
diff --git a/drivers/gpu/drm/i915/i915_sysfs.c b/drivers/gpu/drm/i915/i915_sysfs.c
index c6188dd..2d576b7 100644
--- a/drivers/gpu/drm/i915/i915_sysfs.c
+++ b/drivers/gpu/drm/i915/i915_sysfs.c
@@ -370,6 +370,8 @@
flush_delayed_work(&dev_priv->rps.delayed_resume_work);
+ intel_runtime_pm_get(dev_priv);
+
mutex_lock(&dev_priv->rps.hw_lock);
val = intel_freq_opcode(dev_priv, val);
@@ -378,6 +380,7 @@
val > dev_priv->rps.max_freq ||
val < dev_priv->rps.min_freq_softlimit) {
mutex_unlock(&dev_priv->rps.hw_lock);
+ intel_runtime_pm_put(dev_priv);
return -EINVAL;
}
@@ -398,6 +401,8 @@
mutex_unlock(&dev_priv->rps.hw_lock);
+ intel_runtime_pm_put(dev_priv);
+
return count;
}
@@ -433,6 +438,8 @@
flush_delayed_work(&dev_priv->rps.delayed_resume_work);
+ intel_runtime_pm_get(dev_priv);
+
mutex_lock(&dev_priv->rps.hw_lock);
val = intel_freq_opcode(dev_priv, val);
@@ -441,6 +448,7 @@
val > dev_priv->rps.max_freq ||
val > dev_priv->rps.max_freq_softlimit) {
mutex_unlock(&dev_priv->rps.hw_lock);
+ intel_runtime_pm_put(dev_priv);
return -EINVAL;
}
@@ -457,6 +465,8 @@
mutex_unlock(&dev_priv->rps.hw_lock);
+ intel_runtime_pm_put(dev_priv);
+
return count;
}
diff --git a/drivers/gpu/drm/i915/i915_trace.h b/drivers/gpu/drm/i915/i915_trace.h
index fa09e55..dc0def2 100644
--- a/drivers/gpu/drm/i915/i915_trace.h
+++ b/drivers/gpu/drm/i915/i915_trace.h
@@ -464,7 +464,7 @@
TP_fast_assign(
__entry->dev = from->dev->primary->index;
__entry->sync_from = from->id;
- __entry->sync_to = to_req->ring->id;
+ __entry->sync_to = to_req->engine->id;
__entry->seqno = i915_gem_request_get_seqno(req);
),
@@ -486,13 +486,13 @@
),
TP_fast_assign(
- struct intel_engine_cs *ring =
- i915_gem_request_get_ring(req);
- __entry->dev = ring->dev->primary->index;
- __entry->ring = ring->id;
+ struct intel_engine_cs *engine =
+ i915_gem_request_get_engine(req);
+ __entry->dev = engine->dev->primary->index;
+ __entry->ring = engine->id;
__entry->seqno = i915_gem_request_get_seqno(req);
__entry->flags = flags;
- i915_trace_irq_get(ring, req);
+ i915_trace_irq_get(engine, req);
),
TP_printk("dev=%u, ring=%u, seqno=%u, flags=%x",
@@ -511,8 +511,8 @@
),
TP_fast_assign(
- __entry->dev = req->ring->dev->primary->index;
- __entry->ring = req->ring->id;
+ __entry->dev = req->engine->dev->primary->index;
+ __entry->ring = req->engine->id;
__entry->invalidate = invalidate;
__entry->flush = flush;
),
@@ -533,10 +533,10 @@
),
TP_fast_assign(
- struct intel_engine_cs *ring =
- i915_gem_request_get_ring(req);
- __entry->dev = ring->dev->primary->index;
- __entry->ring = ring->id;
+ struct intel_engine_cs *engine =
+ i915_gem_request_get_engine(req);
+ __entry->dev = engine->dev->primary->index;
+ __entry->ring = engine->id;
__entry->seqno = i915_gem_request_get_seqno(req);
),
@@ -550,8 +550,8 @@
);
TRACE_EVENT(i915_gem_request_notify,
- TP_PROTO(struct intel_engine_cs *ring),
- TP_ARGS(ring),
+ TP_PROTO(struct intel_engine_cs *engine),
+ TP_ARGS(engine),
TP_STRUCT__entry(
__field(u32, dev)
@@ -560,9 +560,9 @@
),
TP_fast_assign(
- __entry->dev = ring->dev->primary->index;
- __entry->ring = ring->id;
- __entry->seqno = ring->get_seqno(ring, false);
+ __entry->dev = engine->dev->primary->index;
+ __entry->ring = engine->id;
+ __entry->seqno = engine->get_seqno(engine);
),
TP_printk("dev=%u, ring=%u, seqno=%u",
@@ -597,13 +597,13 @@
* less desirable.
*/
TP_fast_assign(
- struct intel_engine_cs *ring =
- i915_gem_request_get_ring(req);
- __entry->dev = ring->dev->primary->index;
- __entry->ring = ring->id;
+ struct intel_engine_cs *engine =
+ i915_gem_request_get_engine(req);
+ __entry->dev = engine->dev->primary->index;
+ __entry->ring = engine->id;
__entry->seqno = i915_gem_request_get_seqno(req);
__entry->blocking =
- mutex_is_locked(&ring->dev->struct_mutex);
+ mutex_is_locked(&engine->dev->struct_mutex);
),
TP_printk("dev=%u, ring=%u, seqno=%u, blocking=%s",
@@ -777,9 +777,9 @@
* called only if full ppgtt is enabled.
*/
TRACE_EVENT(switch_mm,
- TP_PROTO(struct intel_engine_cs *ring, struct intel_context *to),
+ TP_PROTO(struct intel_engine_cs *engine, struct intel_context *to),
- TP_ARGS(ring, to),
+ TP_ARGS(engine, to),
TP_STRUCT__entry(
__field(u32, ring)
@@ -789,10 +789,10 @@
),
TP_fast_assign(
- __entry->ring = ring->id;
+ __entry->ring = engine->id;
__entry->to = to;
__entry->vm = to->ppgtt? &to->ppgtt->base : NULL;
- __entry->dev = ring->dev->primary->index;
+ __entry->dev = engine->dev->primary->index;
),
TP_printk("dev=%u, ring=%u, ctx=%p, ctx_vm=%p",
diff --git a/drivers/gpu/drm/i915/i915_vgpu.c b/drivers/gpu/drm/i915/i915_vgpu.c
index dea7429..d02efb8 100644
--- a/drivers/gpu/drm/i915/i915_vgpu.c
+++ b/drivers/gpu/drm/i915/i915_vgpu.c
@@ -181,8 +181,8 @@
int intel_vgt_balloon(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = to_i915(dev);
- struct i915_address_space *ggtt_vm = &dev_priv->gtt.base;
- unsigned long ggtt_vm_end = ggtt_vm->start + ggtt_vm->total;
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
+ unsigned long ggtt_end = ggtt->base.start + ggtt->base.total;
unsigned long mappable_base, mappable_size, mappable_end;
unsigned long unmappable_base, unmappable_size, unmappable_end;
@@ -202,19 +202,19 @@
DRM_INFO("Unmappable graphic memory: base 0x%lx size %ldKiB\n",
unmappable_base, unmappable_size / 1024);
- if (mappable_base < ggtt_vm->start ||
- mappable_end > dev_priv->gtt.mappable_end ||
- unmappable_base < dev_priv->gtt.mappable_end ||
- unmappable_end > ggtt_vm_end) {
+ if (mappable_base < ggtt->base.start ||
+ mappable_end > ggtt->mappable_end ||
+ unmappable_base < ggtt->mappable_end ||
+ unmappable_end > ggtt_end) {
DRM_ERROR("Invalid ballooning configuration!\n");
return -EINVAL;
}
/* Unmappable graphic memory ballooning */
- if (unmappable_base > dev_priv->gtt.mappable_end) {
- ret = vgt_balloon_space(&ggtt_vm->mm,
+ if (unmappable_base > ggtt->mappable_end) {
+ ret = vgt_balloon_space(&ggtt->base.mm,
&bl_info.space[2],
- dev_priv->gtt.mappable_end,
+ ggtt->mappable_end,
unmappable_base);
if (ret)
@@ -225,30 +225,30 @@
* No need to partition out the last physical page,
* because it is reserved to the guard page.
*/
- if (unmappable_end < ggtt_vm_end - PAGE_SIZE) {
- ret = vgt_balloon_space(&ggtt_vm->mm,
+ if (unmappable_end < ggtt_end - PAGE_SIZE) {
+ ret = vgt_balloon_space(&ggtt->base.mm,
&bl_info.space[3],
unmappable_end,
- ggtt_vm_end - PAGE_SIZE);
+ ggtt_end - PAGE_SIZE);
if (ret)
goto err;
}
/* Mappable graphic memory ballooning */
- if (mappable_base > ggtt_vm->start) {
- ret = vgt_balloon_space(&ggtt_vm->mm,
+ if (mappable_base > ggtt->base.start) {
+ ret = vgt_balloon_space(&ggtt->base.mm,
&bl_info.space[0],
- ggtt_vm->start, mappable_base);
+ ggtt->base.start, mappable_base);
if (ret)
goto err;
}
- if (mappable_end < dev_priv->gtt.mappable_end) {
- ret = vgt_balloon_space(&ggtt_vm->mm,
+ if (mappable_end < ggtt->mappable_end) {
+ ret = vgt_balloon_space(&ggtt->base.mm,
&bl_info.space[1],
mappable_end,
- dev_priv->gtt.mappable_end);
+ ggtt->mappable_end);
if (ret)
goto err;
diff --git a/drivers/gpu/drm/i915/intel_atomic.c b/drivers/gpu/drm/i915/intel_atomic.c
index 8e579a8..50ff90a 100644
--- a/drivers/gpu/drm/i915/intel_atomic.c
+++ b/drivers/gpu/drm/i915/intel_atomic.c
@@ -96,8 +96,11 @@
crtc_state->update_pipe = false;
crtc_state->disable_lp_wm = false;
crtc_state->disable_cxsr = false;
- crtc_state->wm_changed = false;
+ crtc_state->update_wm_pre = false;
+ crtc_state->update_wm_post = false;
crtc_state->fb_changed = false;
+ crtc_state->wm.need_postvbl_update = false;
+ crtc_state->fb_bits = 0;
return &crtc_state->base;
}
diff --git a/drivers/gpu/drm/i915/intel_atomic_plane.c b/drivers/gpu/drm/i915/intel_atomic_plane.c
index e0b851a..7de7721 100644
--- a/drivers/gpu/drm/i915/intel_atomic_plane.c
+++ b/drivers/gpu/drm/i915/intel_atomic_plane.c
@@ -195,12 +195,10 @@
struct intel_plane_state *intel_state =
to_intel_plane_state(plane->state);
struct drm_crtc *crtc = plane->state->crtc ?: old_state->crtc;
- struct drm_crtc_state *crtc_state =
- drm_atomic_get_existing_crtc_state(old_state->state, crtc);
if (intel_state->visible)
intel_plane->update_plane(plane,
- to_intel_crtc_state(crtc_state),
+ to_intel_crtc_state(crtc->state),
intel_state);
else
intel_plane->disable_plane(plane, crtc);
diff --git a/drivers/gpu/drm/i915/intel_audio.c b/drivers/gpu/drm/i915/intel_audio.c
index 7d281b4..02a7527 100644
--- a/drivers/gpu/drm/i915/intel_audio.c
+++ b/drivers/gpu/drm/i915/intel_audio.c
@@ -372,7 +372,7 @@
if (WARN_ON(port == PORT_A))
return;
- if (HAS_PCH_IBX(dev_priv->dev)) {
+ if (HAS_PCH_IBX(dev_priv)) {
aud_config = IBX_AUD_CFG(pipe);
aud_cntrl_st2 = IBX_AUD_CNTL_ST2;
} else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) {
@@ -561,23 +561,21 @@
}
/**
- * intel_init_audio - Set up chip specific audio functions
- * @dev: drm device
+ * intel_init_audio_hooks - Set up chip specific audio hooks
+ * @dev_priv: device private
*/
-void intel_init_audio(struct drm_device *dev)
+void intel_init_audio_hooks(struct drm_i915_private *dev_priv)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
-
- if (IS_G4X(dev)) {
+ if (IS_G4X(dev_priv)) {
dev_priv->display.audio_codec_enable = g4x_audio_codec_enable;
dev_priv->display.audio_codec_disable = g4x_audio_codec_disable;
- } else if (IS_VALLEYVIEW(dev) || IS_CHERRYVIEW(dev)) {
+ } else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) {
dev_priv->display.audio_codec_enable = ilk_audio_codec_enable;
dev_priv->display.audio_codec_disable = ilk_audio_codec_disable;
- } else if (IS_HASWELL(dev) || INTEL_INFO(dev)->gen >= 8) {
+ } else if (IS_HASWELL(dev_priv) || INTEL_INFO(dev_priv)->gen >= 8) {
dev_priv->display.audio_codec_enable = hsw_audio_codec_enable;
dev_priv->display.audio_codec_disable = hsw_audio_codec_disable;
- } else if (HAS_PCH_SPLIT(dev)) {
+ } else if (HAS_PCH_SPLIT(dev_priv)) {
dev_priv->display.audio_codec_enable = ilk_audio_codec_enable;
dev_priv->display.audio_codec_disable = ilk_audio_codec_disable;
}
diff --git a/drivers/gpu/drm/i915/intel_bios.c b/drivers/gpu/drm/i915/intel_bios.c
index bf62a19..e72dd9a 100644
--- a/drivers/gpu/drm/i915/intel_bios.c
+++ b/drivers/gpu/drm/i915/intel_bios.c
@@ -29,7 +29,9 @@
#include <drm/drmP.h>
#include <drm/i915_drm.h>
#include "i915_drv.h"
-#include "intel_bios.h"
+
+#define _INTEL_BIOS_PRIVATE
+#include "intel_vbt_defs.h"
/**
* DOC: Video BIOS Table (VBT)
@@ -56,8 +58,6 @@
#define SLAVE_ADDR1 0x70
#define SLAVE_ADDR2 0x72
-static int panel_type;
-
/* Get BDB block size given a pointer to Block ID. */
static u32 _get_blocksize(const u8 *block_base)
{
@@ -203,17 +203,32 @@
const struct lvds_dvo_timing *panel_dvo_timing;
const struct lvds_fp_timing *fp_timing;
struct drm_display_mode *panel_fixed_mode;
+ int panel_type;
int drrs_mode;
+ int ret;
lvds_options = find_section(bdb, BDB_LVDS_OPTIONS);
if (!lvds_options)
return;
dev_priv->vbt.lvds_dither = lvds_options->pixel_dither;
- if (lvds_options->panel_type == 0xff)
- return;
- panel_type = lvds_options->panel_type;
+ ret = intel_opregion_get_panel_type(dev_priv->dev);
+ if (ret >= 0) {
+ WARN_ON(ret > 0xf);
+ panel_type = ret;
+ DRM_DEBUG_KMS("Panel type: %d (OpRegion)\n", panel_type);
+ } else {
+ if (lvds_options->panel_type > 0xf) {
+ DRM_DEBUG_KMS("Invalid VBT panel type 0x%x\n",
+ lvds_options->panel_type);
+ return;
+ }
+ panel_type = lvds_options->panel_type;
+ DRM_DEBUG_KMS("Panel type: %d (VBT)\n", panel_type);
+ }
+
+ dev_priv->vbt.panel_type = panel_type;
drrs_mode = (lvds_options->dps_panel_type_bits
>> (panel_type * 2)) & MODE_MASK;
@@ -249,7 +264,7 @@
panel_dvo_timing = get_lvds_dvo_timing(lvds_lfp_data,
lvds_lfp_data_ptrs,
- lvds_options->panel_type);
+ panel_type);
panel_fixed_mode = kzalloc(sizeof(*panel_fixed_mode), GFP_KERNEL);
if (!panel_fixed_mode)
@@ -264,7 +279,7 @@
fp_timing = get_lvds_fp_timing(bdb, lvds_lfp_data,
lvds_lfp_data_ptrs,
- lvds_options->panel_type);
+ panel_type);
if (fp_timing) {
/* check the resolution, just to be sure */
if (fp_timing->x_res == panel_fixed_mode->hdisplay &&
@@ -282,6 +297,7 @@
{
const struct bdb_lfp_backlight_data *backlight_data;
const struct bdb_lfp_backlight_data_entry *entry;
+ int panel_type = dev_priv->vbt.panel_type;
backlight_data = find_section(bdb, BDB_LVDS_BACKLIGHT);
if (!backlight_data)
@@ -480,7 +496,7 @@
child->slave_addr,
(child->dvo_port == DEVICE_PORT_DVOB) ?
"SDVOB" : "SDVOC");
- p_mapping = &(dev_priv->sdvo_mappings[child->dvo_port - 1]);
+ p_mapping = &dev_priv->vbt.sdvo_mappings[child->dvo_port - 1];
if (!p_mapping->initialized) {
p_mapping->dvo_port = child->dvo_port;
p_mapping->slave_addr = child->slave_addr;
@@ -525,10 +541,7 @@
return;
if (driver->lvds_config == BDB_DRIVER_FEATURE_EDP)
- dev_priv->vbt.edp_support = 1;
-
- if (driver->dual_frequency)
- dev_priv->render_reclock_avail = true;
+ dev_priv->vbt.edp.support = 1;
DRM_DEBUG_KMS("DRRS State Enabled:%d\n", driver->drrs_enabled);
/*
@@ -547,23 +560,24 @@
const struct bdb_edp *edp;
const struct edp_power_seq *edp_pps;
const struct edp_link_params *edp_link_params;
+ int panel_type = dev_priv->vbt.panel_type;
edp = find_section(bdb, BDB_EDP);
if (!edp) {
- if (dev_priv->vbt.edp_support)
+ if (dev_priv->vbt.edp.support)
DRM_DEBUG_KMS("No eDP BDB found but eDP panel supported.\n");
return;
}
switch ((edp->color_depth >> (panel_type * 2)) & 3) {
case EDP_18BPP:
- dev_priv->vbt.edp_bpp = 18;
+ dev_priv->vbt.edp.bpp = 18;
break;
case EDP_24BPP:
- dev_priv->vbt.edp_bpp = 24;
+ dev_priv->vbt.edp.bpp = 24;
break;
case EDP_30BPP:
- dev_priv->vbt.edp_bpp = 30;
+ dev_priv->vbt.edp.bpp = 30;
break;
}
@@ -571,14 +585,14 @@
edp_pps = &edp->power_seqs[panel_type];
edp_link_params = &edp->link_params[panel_type];
- dev_priv->vbt.edp_pps = *edp_pps;
+ dev_priv->vbt.edp.pps = *edp_pps;
switch (edp_link_params->rate) {
case EDP_RATE_1_62:
- dev_priv->vbt.edp_rate = DP_LINK_BW_1_62;
+ dev_priv->vbt.edp.rate = DP_LINK_BW_1_62;
break;
case EDP_RATE_2_7:
- dev_priv->vbt.edp_rate = DP_LINK_BW_2_7;
+ dev_priv->vbt.edp.rate = DP_LINK_BW_2_7;
break;
default:
DRM_DEBUG_KMS("VBT has unknown eDP link rate value %u\n",
@@ -588,13 +602,13 @@
switch (edp_link_params->lanes) {
case EDP_LANE_1:
- dev_priv->vbt.edp_lanes = 1;
+ dev_priv->vbt.edp.lanes = 1;
break;
case EDP_LANE_2:
- dev_priv->vbt.edp_lanes = 2;
+ dev_priv->vbt.edp.lanes = 2;
break;
case EDP_LANE_4:
- dev_priv->vbt.edp_lanes = 4;
+ dev_priv->vbt.edp.lanes = 4;
break;
default:
DRM_DEBUG_KMS("VBT has unknown eDP lane count value %u\n",
@@ -604,16 +618,16 @@
switch (edp_link_params->preemphasis) {
case EDP_PREEMPHASIS_NONE:
- dev_priv->vbt.edp_preemphasis = DP_TRAIN_PRE_EMPH_LEVEL_0;
+ dev_priv->vbt.edp.preemphasis = DP_TRAIN_PRE_EMPH_LEVEL_0;
break;
case EDP_PREEMPHASIS_3_5dB:
- dev_priv->vbt.edp_preemphasis = DP_TRAIN_PRE_EMPH_LEVEL_1;
+ dev_priv->vbt.edp.preemphasis = DP_TRAIN_PRE_EMPH_LEVEL_1;
break;
case EDP_PREEMPHASIS_6dB:
- dev_priv->vbt.edp_preemphasis = DP_TRAIN_PRE_EMPH_LEVEL_2;
+ dev_priv->vbt.edp.preemphasis = DP_TRAIN_PRE_EMPH_LEVEL_2;
break;
case EDP_PREEMPHASIS_9_5dB:
- dev_priv->vbt.edp_preemphasis = DP_TRAIN_PRE_EMPH_LEVEL_3;
+ dev_priv->vbt.edp.preemphasis = DP_TRAIN_PRE_EMPH_LEVEL_3;
break;
default:
DRM_DEBUG_KMS("VBT has unknown eDP pre-emphasis value %u\n",
@@ -623,16 +637,16 @@
switch (edp_link_params->vswing) {
case EDP_VSWING_0_4V:
- dev_priv->vbt.edp_vswing = DP_TRAIN_VOLTAGE_SWING_LEVEL_0;
+ dev_priv->vbt.edp.vswing = DP_TRAIN_VOLTAGE_SWING_LEVEL_0;
break;
case EDP_VSWING_0_6V:
- dev_priv->vbt.edp_vswing = DP_TRAIN_VOLTAGE_SWING_LEVEL_1;
+ dev_priv->vbt.edp.vswing = DP_TRAIN_VOLTAGE_SWING_LEVEL_1;
break;
case EDP_VSWING_0_8V:
- dev_priv->vbt.edp_vswing = DP_TRAIN_VOLTAGE_SWING_LEVEL_2;
+ dev_priv->vbt.edp.vswing = DP_TRAIN_VOLTAGE_SWING_LEVEL_2;
break;
case EDP_VSWING_1_2V:
- dev_priv->vbt.edp_vswing = DP_TRAIN_VOLTAGE_SWING_LEVEL_3;
+ dev_priv->vbt.edp.vswing = DP_TRAIN_VOLTAGE_SWING_LEVEL_3;
break;
default:
DRM_DEBUG_KMS("VBT has unknown eDP voltage swing value %u\n",
@@ -645,10 +659,10 @@
/* Don't read from VBT if module parameter has valid value*/
if (i915.edp_vswing) {
- dev_priv->edp_low_vswing = i915.edp_vswing == 1;
+ dev_priv->vbt.edp.low_vswing = i915.edp_vswing == 1;
} else {
vswing = (edp->edp_vswing_preemph >> (panel_type * 4)) & 0xF;
- dev_priv->edp_low_vswing = vswing == 0;
+ dev_priv->vbt.edp.low_vswing = vswing == 0;
}
}
}
@@ -658,6 +672,7 @@
{
const struct bdb_psr *psr;
const struct psr_table *psr_table;
+ int panel_type = dev_priv->vbt.panel_type;
psr = find_section(bdb, BDB_PSR);
if (!psr) {
@@ -704,9 +719,10 @@
const struct bdb_mipi_config *start;
const struct mipi_config *config;
const struct mipi_pps_data *pps;
+ int panel_type = dev_priv->vbt.panel_type;
/* parse MIPI blocks only if LFP type is MIPI */
- if (!dev_priv->vbt.has_mipi)
+ if (!intel_bios_is_dsi_present(dev_priv, NULL))
return;
/* Initialize this to undefined indicating no generic MIPI support */
@@ -911,6 +927,7 @@
parse_mipi_sequence(struct drm_i915_private *dev_priv,
const struct bdb_header *bdb)
{
+ int panel_type = dev_priv->vbt.panel_type;
const struct bdb_mipi_sequence *sequence;
const u8 *seq_data;
u32 seq_size;
@@ -1124,7 +1141,7 @@
}
/* Parse the I_boost config for SKL and above */
- if (bdb->version >= 196 && (child->common.flags_1 & IBOOST_ENABLE)) {
+ if (bdb->version >= 196 && child->common.iboost) {
info->dp_boost_level = translate_iboost(child->common.iboost_level & 0xF);
DRM_DEBUG_KMS("VBT (e)DP boost level for port %c: %d\n",
port_name(port), info->dp_boost_level);
@@ -1232,14 +1249,6 @@
continue;
}
- if (p_child->common.dvo_port >= DVO_PORT_MIPIA
- && p_child->common.dvo_port <= DVO_PORT_MIPID
- &&p_child->common.device_type & DEVICE_TYPE_MIPI_OUTPUT) {
- DRM_DEBUG_KMS("Found MIPI as LFP\n");
- dev_priv->vbt.has_mipi = 1;
- dev_priv->vbt.dsi.port = p_child->common.dvo_port;
- }
-
child_dev_ptr = dev_priv->vbt.child_dev + count;
count++;
@@ -1250,6 +1259,19 @@
*/
memcpy(child_dev_ptr, p_child,
min_t(size_t, p_defs->child_dev_size, sizeof(*p_child)));
+
+ /*
+ * copied full block, now init values when they are not
+ * available in current version
+ */
+ if (bdb->version < 196) {
+ /* Set default values for bits added from v196 */
+ child_dev_ptr->common.iboost = 0;
+ child_dev_ptr->common.hpd_invert = 0;
+ }
+
+ if (bdb->version < 192)
+ child_dev_ptr->common.lspcon = 0;
}
return;
}
@@ -1431,3 +1453,210 @@
return 0;
}
+
+/**
+ * intel_bios_is_tv_present - is integrated TV present in VBT
+ * @dev_priv: i915 device instance
+ *
+ * Return true if TV is present. If no child devices were parsed from VBT,
+ * assume TV is present.
+ */
+bool intel_bios_is_tv_present(struct drm_i915_private *dev_priv)
+{
+ union child_device_config *p_child;
+ int i;
+
+ if (!dev_priv->vbt.int_tv_support)
+ return false;
+
+ if (!dev_priv->vbt.child_dev_num)
+ return true;
+
+ for (i = 0; i < dev_priv->vbt.child_dev_num; i++) {
+ p_child = dev_priv->vbt.child_dev + i;
+ /*
+ * If the device type is not TV, continue.
+ */
+ switch (p_child->old.device_type) {
+ case DEVICE_TYPE_INT_TV:
+ case DEVICE_TYPE_TV:
+ case DEVICE_TYPE_TV_SVIDEO_COMPOSITE:
+ break;
+ default:
+ continue;
+ }
+ /* Only when the addin_offset is non-zero, it is regarded
+ * as present.
+ */
+ if (p_child->old.addin_offset)
+ return true;
+ }
+
+ return false;
+}
+
+/**
+ * intel_bios_is_lvds_present - is LVDS present in VBT
+ * @dev_priv: i915 device instance
+ * @i2c_pin: i2c pin for LVDS if present
+ *
+ * Return true if LVDS is present. If no child devices were parsed from VBT,
+ * assume LVDS is present.
+ */
+bool intel_bios_is_lvds_present(struct drm_i915_private *dev_priv, u8 *i2c_pin)
+{
+ int i;
+
+ if (!dev_priv->vbt.child_dev_num)
+ return true;
+
+ for (i = 0; i < dev_priv->vbt.child_dev_num; i++) {
+ union child_device_config *uchild = dev_priv->vbt.child_dev + i;
+ struct old_child_dev_config *child = &uchild->old;
+
+ /* If the device type is not LFP, continue.
+ * We have to check both the new identifiers as well as the
+ * old for compatibility with some BIOSes.
+ */
+ if (child->device_type != DEVICE_TYPE_INT_LFP &&
+ child->device_type != DEVICE_TYPE_LFP)
+ continue;
+
+ if (intel_gmbus_is_valid_pin(dev_priv, child->i2c_pin))
+ *i2c_pin = child->i2c_pin;
+
+ /* However, we cannot trust the BIOS writers to populate
+ * the VBT correctly. Since LVDS requires additional
+ * information from AIM blocks, a non-zero addin offset is
+ * a good indicator that the LVDS is actually present.
+ */
+ if (child->addin_offset)
+ return true;
+
+ /* But even then some BIOS writers perform some black magic
+ * and instantiate the device without reference to any
+ * additional data. Trust that if the VBT was written into
+ * the OpRegion then they have validated the LVDS's existence.
+ */
+ if (dev_priv->opregion.vbt)
+ return true;
+ }
+
+ return false;
+}
+
+/**
+ * intel_bios_is_port_edp - is the device in given port eDP
+ * @dev_priv: i915 device instance
+ * @port: port to check
+ *
+ * Return true if the device in %port is eDP.
+ */
+bool intel_bios_is_port_edp(struct drm_i915_private *dev_priv, enum port port)
+{
+ union child_device_config *p_child;
+ static const short port_mapping[] = {
+ [PORT_B] = DVO_PORT_DPB,
+ [PORT_C] = DVO_PORT_DPC,
+ [PORT_D] = DVO_PORT_DPD,
+ [PORT_E] = DVO_PORT_DPE,
+ };
+ int i;
+
+ if (!dev_priv->vbt.child_dev_num)
+ return false;
+
+ for (i = 0; i < dev_priv->vbt.child_dev_num; i++) {
+ p_child = dev_priv->vbt.child_dev + i;
+
+ if (p_child->common.dvo_port == port_mapping[port] &&
+ (p_child->common.device_type & DEVICE_TYPE_eDP_BITS) ==
+ (DEVICE_TYPE_eDP & DEVICE_TYPE_eDP_BITS))
+ return true;
+ }
+
+ return false;
+}
+
+/**
+ * intel_bios_is_dsi_present - is DSI present in VBT
+ * @dev_priv: i915 device instance
+ * @port: port for DSI if present
+ *
+ * Return true if DSI is present, and return the port in %port.
+ */
+bool intel_bios_is_dsi_present(struct drm_i915_private *dev_priv,
+ enum port *port)
+{
+ union child_device_config *p_child;
+ u8 dvo_port;
+ int i;
+
+ for (i = 0; i < dev_priv->vbt.child_dev_num; i++) {
+ p_child = dev_priv->vbt.child_dev + i;
+
+ if (!(p_child->common.device_type & DEVICE_TYPE_MIPI_OUTPUT))
+ continue;
+
+ dvo_port = p_child->common.dvo_port;
+
+ switch (dvo_port) {
+ case DVO_PORT_MIPIA:
+ case DVO_PORT_MIPIC:
+ if (port)
+ *port = dvo_port - DVO_PORT_MIPIA;
+ return true;
+ case DVO_PORT_MIPIB:
+ case DVO_PORT_MIPID:
+ DRM_DEBUG_KMS("VBT has unsupported DSI port %c\n",
+ port_name(dvo_port - DVO_PORT_MIPIA));
+ break;
+ }
+ }
+
+ return false;
+}
+
+/**
+ * intel_bios_is_port_hpd_inverted - is HPD inverted for %port
+ * @dev_priv: i915 device instance
+ * @port: port to check
+ *
+ * Return true if HPD should be inverted for %port.
+ */
+bool
+intel_bios_is_port_hpd_inverted(struct drm_i915_private *dev_priv,
+ enum port port)
+{
+ int i;
+
+ if (WARN_ON_ONCE(!IS_BROXTON(dev_priv)))
+ return false;
+
+ for (i = 0; i < dev_priv->vbt.child_dev_num; i++) {
+ if (!dev_priv->vbt.child_dev[i].common.hpd_invert)
+ continue;
+
+ switch (dev_priv->vbt.child_dev[i].common.dvo_port) {
+ case DVO_PORT_DPA:
+ case DVO_PORT_HDMIA:
+ if (port == PORT_A)
+ return true;
+ break;
+ case DVO_PORT_DPB:
+ case DVO_PORT_HDMIB:
+ if (port == PORT_B)
+ return true;
+ break;
+ case DVO_PORT_DPC:
+ case DVO_PORT_HDMIC:
+ if (port == PORT_C)
+ return true;
+ break;
+ default:
+ break;
+ }
+ }
+
+ return false;
+}
diff --git a/drivers/gpu/drm/i915/intel_bios.h b/drivers/gpu/drm/i915/intel_bios.h
index 350d4e0..ab0ea31 100644
--- a/drivers/gpu/drm/i915/intel_bios.h
+++ b/drivers/gpu/drm/i915/intel_bios.h
@@ -1,5 +1,5 @@
/*
- * Copyright © 2006 Intel Corporation
+ * Copyright © 2016 Intel Corporation
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
@@ -19,544 +19,17 @@
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
- *
- * Authors:
- * Eric Anholt <eric@anholt.net>
- *
+ */
+
+/*
+ * Please use intel_vbt_defs.h for VBT private data, to hide and abstract away
+ * the VBT from the rest of the driver. Add the parsed, clean data to struct
+ * intel_vbt_data within struct drm_i915_private.
*/
#ifndef _INTEL_BIOS_H_
#define _INTEL_BIOS_H_
-/**
- * struct vbt_header - VBT Header structure
- * @signature: VBT signature, always starts with "$VBT"
- * @version: Version of this structure
- * @header_size: Size of this structure
- * @vbt_size: Size of VBT (VBT Header, BDB Header and data blocks)
- * @vbt_checksum: Checksum
- * @reserved0: Reserved
- * @bdb_offset: Offset of &struct bdb_header from beginning of VBT
- * @aim_offset: Offsets of add-in data blocks from beginning of VBT
- */
-struct vbt_header {
- u8 signature[20];
- u16 version;
- u16 header_size;
- u16 vbt_size;
- u8 vbt_checksum;
- u8 reserved0;
- u32 bdb_offset;
- u32 aim_offset[4];
-} __packed;
-
-/**
- * struct bdb_header - BDB Header structure
- * @signature: BDB signature "BIOS_DATA_BLOCK"
- * @version: Version of the data block definitions
- * @header_size: Size of this structure
- * @bdb_size: Size of BDB (BDB Header and data blocks)
- */
-struct bdb_header {
- u8 signature[16];
- u16 version;
- u16 header_size;
- u16 bdb_size;
-} __packed;
-
-/* strictly speaking, this is a "skip" block, but it has interesting info */
-struct vbios_data {
- u8 type; /* 0 == desktop, 1 == mobile */
- u8 relstage;
- u8 chipset;
- u8 lvds_present:1;
- u8 tv_present:1;
- u8 rsvd2:6; /* finish byte */
- u8 rsvd3[4];
- u8 signon[155];
- u8 copyright[61];
- u16 code_segment;
- u8 dos_boot_mode;
- u8 bandwidth_percent;
- u8 rsvd4; /* popup memory size */
- u8 resize_pci_bios;
- u8 rsvd5; /* is crt already on ddc2 */
-} __packed;
-
-/*
- * There are several types of BIOS data blocks (BDBs), each block has
- * an ID and size in the first 3 bytes (ID in first, size in next 2).
- * Known types are listed below.
- */
-#define BDB_GENERAL_FEATURES 1
-#define BDB_GENERAL_DEFINITIONS 2
-#define BDB_OLD_TOGGLE_LIST 3
-#define BDB_MODE_SUPPORT_LIST 4
-#define BDB_GENERIC_MODE_TABLE 5
-#define BDB_EXT_MMIO_REGS 6
-#define BDB_SWF_IO 7
-#define BDB_SWF_MMIO 8
-#define BDB_PSR 9
-#define BDB_MODE_REMOVAL_TABLE 10
-#define BDB_CHILD_DEVICE_TABLE 11
-#define BDB_DRIVER_FEATURES 12
-#define BDB_DRIVER_PERSISTENCE 13
-#define BDB_EXT_TABLE_PTRS 14
-#define BDB_DOT_CLOCK_OVERRIDE 15
-#define BDB_DISPLAY_SELECT 16
-/* 17 rsvd */
-#define BDB_DRIVER_ROTATION 18
-#define BDB_DISPLAY_REMOVE 19
-#define BDB_OEM_CUSTOM 20
-#define BDB_EFP_LIST 21 /* workarounds for VGA hsync/vsync */
-#define BDB_SDVO_LVDS_OPTIONS 22
-#define BDB_SDVO_PANEL_DTDS 23
-#define BDB_SDVO_LVDS_PNP_IDS 24
-#define BDB_SDVO_LVDS_POWER_SEQ 25
-#define BDB_TV_OPTIONS 26
-#define BDB_EDP 27
-#define BDB_LVDS_OPTIONS 40
-#define BDB_LVDS_LFP_DATA_PTRS 41
-#define BDB_LVDS_LFP_DATA 42
-#define BDB_LVDS_BACKLIGHT 43
-#define BDB_LVDS_POWER 44
-#define BDB_MIPI_CONFIG 52
-#define BDB_MIPI_SEQUENCE 53
-#define BDB_SKIP 254 /* VBIOS private block, ignore */
-
-struct bdb_general_features {
- /* bits 1 */
- u8 panel_fitting:2;
- u8 flexaim:1;
- u8 msg_enable:1;
- u8 clear_screen:3;
- u8 color_flip:1;
-
- /* bits 2 */
- u8 download_ext_vbt:1;
- u8 enable_ssc:1;
- u8 ssc_freq:1;
- u8 enable_lfp_on_override:1;
- u8 disable_ssc_ddt:1;
- u8 rsvd7:1;
- u8 display_clock_mode:1;
- u8 rsvd8:1; /* finish byte */
-
- /* bits 3 */
- u8 disable_smooth_vision:1;
- u8 single_dvi:1;
- u8 rsvd9:1;
- u8 fdi_rx_polarity_inverted:1;
- u8 rsvd10:4; /* finish byte */
-
- /* bits 4 */
- u8 legacy_monitor_detect;
-
- /* bits 5 */
- u8 int_crt_support:1;
- u8 int_tv_support:1;
- u8 int_efp_support:1;
- u8 dp_ssc_enb:1; /* PCH attached eDP supports SSC */
- u8 dp_ssc_freq:1; /* SSC freq for PCH attached eDP */
- u8 rsvd11:3; /* finish byte */
-} __packed;
-
-/* pre-915 */
-#define GPIO_PIN_DVI_LVDS 0x03 /* "DVI/LVDS DDC GPIO pins" */
-#define GPIO_PIN_ADD_I2C 0x05 /* "ADDCARD I2C GPIO pins" */
-#define GPIO_PIN_ADD_DDC 0x04 /* "ADDCARD DDC GPIO pins" */
-#define GPIO_PIN_ADD_DDC_I2C 0x06 /* "ADDCARD DDC/I2C GPIO pins" */
-
-/* Pre 915 */
-#define DEVICE_TYPE_NONE 0x00
-#define DEVICE_TYPE_CRT 0x01
-#define DEVICE_TYPE_TV 0x09
-#define DEVICE_TYPE_EFP 0x12
-#define DEVICE_TYPE_LFP 0x22
-/* On 915+ */
-#define DEVICE_TYPE_CRT_DPMS 0x6001
-#define DEVICE_TYPE_CRT_DPMS_HOTPLUG 0x4001
-#define DEVICE_TYPE_TV_COMPOSITE 0x0209
-#define DEVICE_TYPE_TV_MACROVISION 0x0289
-#define DEVICE_TYPE_TV_RF_COMPOSITE 0x020c
-#define DEVICE_TYPE_TV_SVIDEO_COMPOSITE 0x0609
-#define DEVICE_TYPE_TV_SCART 0x0209
-#define DEVICE_TYPE_TV_CODEC_HOTPLUG_PWR 0x6009
-#define DEVICE_TYPE_EFP_HOTPLUG_PWR 0x6012
-#define DEVICE_TYPE_EFP_DVI_HOTPLUG_PWR 0x6052
-#define DEVICE_TYPE_EFP_DVI_I 0x6053
-#define DEVICE_TYPE_EFP_DVI_D_DUAL 0x6152
-#define DEVICE_TYPE_EFP_DVI_D_HDCP 0x60d2
-#define DEVICE_TYPE_OPENLDI_HOTPLUG_PWR 0x6062
-#define DEVICE_TYPE_OPENLDI_DUALPIX 0x6162
-#define DEVICE_TYPE_LFP_PANELLINK 0x5012
-#define DEVICE_TYPE_LFP_CMOS_PWR 0x5042
-#define DEVICE_TYPE_LFP_LVDS_PWR 0x5062
-#define DEVICE_TYPE_LFP_LVDS_DUAL 0x5162
-#define DEVICE_TYPE_LFP_LVDS_DUAL_HDCP 0x51e2
-
-#define DEVICE_CFG_NONE 0x00
-#define DEVICE_CFG_12BIT_DVOB 0x01
-#define DEVICE_CFG_12BIT_DVOC 0x02
-#define DEVICE_CFG_24BIT_DVOBC 0x09
-#define DEVICE_CFG_24BIT_DVOCB 0x0a
-#define DEVICE_CFG_DUAL_DVOB 0x11
-#define DEVICE_CFG_DUAL_DVOC 0x12
-#define DEVICE_CFG_DUAL_DVOBC 0x13
-#define DEVICE_CFG_DUAL_LINK_DVOBC 0x19
-#define DEVICE_CFG_DUAL_LINK_DVOCB 0x1a
-
-#define DEVICE_WIRE_NONE 0x00
-#define DEVICE_WIRE_DVOB 0x01
-#define DEVICE_WIRE_DVOC 0x02
-#define DEVICE_WIRE_DVOBC 0x03
-#define DEVICE_WIRE_DVOBB 0x05
-#define DEVICE_WIRE_DVOCC 0x06
-#define DEVICE_WIRE_DVOB_MASTER 0x0d
-#define DEVICE_WIRE_DVOC_MASTER 0x0e
-
-#define DEVICE_PORT_DVOA 0x00 /* none on 845+ */
-#define DEVICE_PORT_DVOB 0x01
-#define DEVICE_PORT_DVOC 0x02
-
-/*
- * We used to keep this struct but without any version control. We should avoid
- * using it in the future, but it should be safe to keep using it in the old
- * code. Do not change; we rely on its size.
- */
-struct old_child_dev_config {
- u16 handle;
- u16 device_type;
- u8 device_id[10]; /* ascii string */
- u16 addin_offset;
- u8 dvo_port; /* See Device_PORT_* above */
- u8 i2c_pin;
- u8 slave_addr;
- u8 ddc_pin;
- u16 edid_ptr;
- u8 dvo_cfg; /* See DEVICE_CFG_* above */
- u8 dvo2_port;
- u8 i2c2_pin;
- u8 slave2_addr;
- u8 ddc2_pin;
- u8 capabilities;
- u8 dvo_wiring;/* See DEVICE_WIRE_* above */
- u8 dvo2_wiring;
- u16 extended_type;
- u8 dvo_function;
-} __packed;
-
-/* This one contains field offsets that are known to be common for all BDB
- * versions. Notice that the meaning of the contents contents may still change,
- * but at least the offsets are consistent. */
-
-/* Definitions for flags_1 */
-#define IBOOST_ENABLE (1<<3)
-
-struct common_child_dev_config {
- u16 handle;
- u16 device_type;
- u8 not_common1[12];
- u8 dvo_port;
- u8 not_common2[2];
- u8 ddc_pin;
- u16 edid_ptr;
- u8 obsolete;
- u8 flags_1;
- u8 not_common3[13];
- u8 iboost_level;
-} __packed;
-
-
-/* This field changes depending on the BDB version, so the most reliable way to
- * read it is by checking the BDB version and reading the raw pointer. */
-union child_device_config {
- /* This one is safe to be used anywhere, but the code should still check
- * the BDB version. */
- u8 raw[33];
- /* This one should only be kept for legacy code. */
- struct old_child_dev_config old;
- /* This one should also be safe to use anywhere, even without version
- * checks. */
- struct common_child_dev_config common;
-} __packed;
-
-struct bdb_general_definitions {
- /* DDC GPIO */
- u8 crt_ddc_gmbus_pin;
-
- /* DPMS bits */
- u8 dpms_acpi:1;
- u8 skip_boot_crt_detect:1;
- u8 dpms_aim:1;
- u8 rsvd1:5; /* finish byte */
-
- /* boot device bits */
- u8 boot_display[2];
- u8 child_dev_size;
-
- /*
- * Device info:
- * If TV is present, it'll be at devices[0].
- * LVDS will be next, either devices[0] or [1], if present.
- * On some platforms the number of device is 6. But could be as few as
- * 4 if both TV and LVDS are missing.
- * And the device num is related with the size of general definition
- * block. It is obtained by using the following formula:
- * number = (block_size - sizeof(bdb_general_definitions))/
- * defs->child_dev_size;
- */
- uint8_t devices[0];
-} __packed;
-
-/* Mask for DRRS / Panel Channel / SSC / BLT control bits extraction */
-#define MODE_MASK 0x3
-
-struct bdb_lvds_options {
- u8 panel_type;
- u8 rsvd1;
- /* LVDS capabilities, stored in a dword */
- u8 pfit_mode:2;
- u8 pfit_text_mode_enhanced:1;
- u8 pfit_gfx_mode_enhanced:1;
- u8 pfit_ratio_auto:1;
- u8 pixel_dither:1;
- u8 lvds_edid:1;
- u8 rsvd2:1;
- u8 rsvd4;
- /* LVDS Panel channel bits stored here */
- u32 lvds_panel_channel_bits;
- /* LVDS SSC (Spread Spectrum Clock) bits stored here. */
- u16 ssc_bits;
- u16 ssc_freq;
- u16 ssc_ddt;
- /* Panel color depth defined here */
- u16 panel_color_depth;
- /* LVDS panel type bits stored here */
- u32 dps_panel_type_bits;
- /* LVDS backlight control type bits stored here */
- u32 blt_control_type_bits;
-} __packed;
-
-/* LFP pointer table contains entries to the struct below */
-struct bdb_lvds_lfp_data_ptr {
- u16 fp_timing_offset; /* offsets are from start of bdb */
- u8 fp_table_size;
- u16 dvo_timing_offset;
- u8 dvo_table_size;
- u16 panel_pnp_id_offset;
- u8 pnp_table_size;
-} __packed;
-
-struct bdb_lvds_lfp_data_ptrs {
- u8 lvds_entries; /* followed by one or more lvds_data_ptr structs */
- struct bdb_lvds_lfp_data_ptr ptr[16];
-} __packed;
-
-/* LFP data has 3 blocks per entry */
-struct lvds_fp_timing {
- u16 x_res;
- u16 y_res;
- u32 lvds_reg;
- u32 lvds_reg_val;
- u32 pp_on_reg;
- u32 pp_on_reg_val;
- u32 pp_off_reg;
- u32 pp_off_reg_val;
- u32 pp_cycle_reg;
- u32 pp_cycle_reg_val;
- u32 pfit_reg;
- u32 pfit_reg_val;
- u16 terminator;
-} __packed;
-
-struct lvds_dvo_timing {
- u16 clock; /**< In 10khz */
- u8 hactive_lo;
- u8 hblank_lo;
- u8 hblank_hi:4;
- u8 hactive_hi:4;
- u8 vactive_lo;
- u8 vblank_lo;
- u8 vblank_hi:4;
- u8 vactive_hi:4;
- u8 hsync_off_lo;
- u8 hsync_pulse_width;
- u8 vsync_pulse_width:4;
- u8 vsync_off:4;
- u8 rsvd0:6;
- u8 hsync_off_hi:2;
- u8 h_image;
- u8 v_image;
- u8 max_hv;
- u8 h_border;
- u8 v_border;
- u8 rsvd1:3;
- u8 digital:2;
- u8 vsync_positive:1;
- u8 hsync_positive:1;
- u8 rsvd2:1;
-} __packed;
-
-struct lvds_pnp_id {
- u16 mfg_name;
- u16 product_code;
- u32 serial;
- u8 mfg_week;
- u8 mfg_year;
-} __packed;
-
-struct bdb_lvds_lfp_data_entry {
- struct lvds_fp_timing fp_timing;
- struct lvds_dvo_timing dvo_timing;
- struct lvds_pnp_id pnp_id;
-} __packed;
-
-struct bdb_lvds_lfp_data {
- struct bdb_lvds_lfp_data_entry data[16];
-} __packed;
-
-#define BDB_BACKLIGHT_TYPE_NONE 0
-#define BDB_BACKLIGHT_TYPE_PWM 2
-
-struct bdb_lfp_backlight_data_entry {
- u8 type:2;
- u8 active_low_pwm:1;
- u8 obsolete1:5;
- u16 pwm_freq_hz;
- u8 min_brightness;
- u8 obsolete2;
- u8 obsolete3;
-} __packed;
-
-struct bdb_lfp_backlight_data {
- u8 entry_size;
- struct bdb_lfp_backlight_data_entry data[16];
- u8 level[16];
-} __packed;
-
-struct aimdb_header {
- char signature[16];
- char oem_device[20];
- u16 aimdb_version;
- u16 aimdb_header_size;
- u16 aimdb_size;
-} __packed;
-
-struct aimdb_block {
- u8 aimdb_id;
- u16 aimdb_size;
-} __packed;
-
-struct vch_panel_data {
- u16 fp_timing_offset;
- u8 fp_timing_size;
- u16 dvo_timing_offset;
- u8 dvo_timing_size;
- u16 text_fitting_offset;
- u8 text_fitting_size;
- u16 graphics_fitting_offset;
- u8 graphics_fitting_size;
-} __packed;
-
-struct vch_bdb_22 {
- struct aimdb_block aimdb_block;
- struct vch_panel_data panels[16];
-} __packed;
-
-struct bdb_sdvo_lvds_options {
- u8 panel_backlight;
- u8 h40_set_panel_type;
- u8 panel_type;
- u8 ssc_clk_freq;
- u16 als_low_trip;
- u16 als_high_trip;
- u8 sclalarcoeff_tab_row_num;
- u8 sclalarcoeff_tab_row_size;
- u8 coefficient[8];
- u8 panel_misc_bits_1;
- u8 panel_misc_bits_2;
- u8 panel_misc_bits_3;
- u8 panel_misc_bits_4;
-} __packed;
-
-
-#define BDB_DRIVER_FEATURE_NO_LVDS 0
-#define BDB_DRIVER_FEATURE_INT_LVDS 1
-#define BDB_DRIVER_FEATURE_SDVO_LVDS 2
-#define BDB_DRIVER_FEATURE_EDP 3
-
-struct bdb_driver_features {
- u8 boot_dev_algorithm:1;
- u8 block_display_switch:1;
- u8 allow_display_switch:1;
- u8 hotplug_dvo:1;
- u8 dual_view_zoom:1;
- u8 int15h_hook:1;
- u8 sprite_in_clone:1;
- u8 primary_lfp_id:1;
-
- u16 boot_mode_x;
- u16 boot_mode_y;
- u8 boot_mode_bpp;
- u8 boot_mode_refresh;
-
- u16 enable_lfp_primary:1;
- u16 selective_mode_pruning:1;
- u16 dual_frequency:1;
- u16 render_clock_freq:1; /* 0: high freq; 1: low freq */
- u16 nt_clone_support:1;
- u16 power_scheme_ui:1; /* 0: CUI; 1: 3rd party */
- u16 sprite_display_assign:1; /* 0: secondary; 1: primary */
- u16 cui_aspect_scaling:1;
- u16 preserve_aspect_ratio:1;
- u16 sdvo_device_power_down:1;
- u16 crt_hotplug:1;
- u16 lvds_config:2;
- u16 tv_hotplug:1;
- u16 hdmi_config:2;
-
- u8 static_display:1;
- u8 reserved2:7;
- u16 legacy_crt_max_x;
- u16 legacy_crt_max_y;
- u8 legacy_crt_max_refresh;
-
- u8 hdmi_termination;
- u8 custom_vbt_version;
- /* Driver features data block */
- u16 rmpm_enabled:1;
- u16 s2ddt_enabled:1;
- u16 dpst_enabled:1;
- u16 bltclt_enabled:1;
- u16 adb_enabled:1;
- u16 drrs_enabled:1;
- u16 grs_enabled:1;
- u16 gpmt_enabled:1;
- u16 tbt_enabled:1;
- u16 psr_enabled:1;
- u16 ips_enabled:1;
- u16 reserved3:4;
- u16 pc_feature_valid:1;
-} __packed;
-
-#define EDP_18BPP 0
-#define EDP_24BPP 1
-#define EDP_30BPP 2
-#define EDP_RATE_1_62 0
-#define EDP_RATE_2_7 1
-#define EDP_LANE_1 0
-#define EDP_LANE_2 1
-#define EDP_LANE_4 3
-#define EDP_PREEMPHASIS_NONE 0
-#define EDP_PREEMPHASIS_3_5dB 1
-#define EDP_PREEMPHASIS_6dB 2
-#define EDP_PREEMPHASIS_9_5dB 3
-#define EDP_VSWING_0_4V 0
-#define EDP_VSWING_0_6V 1
-#define EDP_VSWING_0_8V 2
-#define EDP_VSWING_1_2V 3
-
struct edp_power_seq {
u16 t1_t3;
u16 t8;
@@ -565,245 +38,37 @@
u16 t11_t12;
} __packed;
-struct edp_link_params {
- u8 rate:4;
- u8 lanes:4;
- u8 preemphasis:4;
- u8 vswing:4;
-} __packed;
+/* MIPI Sequence Block definitions */
+enum mipi_seq {
+ MIPI_SEQ_END = 0,
+ MIPI_SEQ_ASSERT_RESET,
+ MIPI_SEQ_INIT_OTP,
+ MIPI_SEQ_DISPLAY_ON,
+ MIPI_SEQ_DISPLAY_OFF,
+ MIPI_SEQ_DEASSERT_RESET,
+ MIPI_SEQ_BACKLIGHT_ON, /* sequence block v2+ */
+ MIPI_SEQ_BACKLIGHT_OFF, /* sequence block v2+ */
+ MIPI_SEQ_TEAR_ON, /* sequence block v2+ */
+ MIPI_SEQ_TEAR_OFF, /* sequence block v3+ */
+ MIPI_SEQ_POWER_ON, /* sequence block v3+ */
+ MIPI_SEQ_POWER_OFF, /* sequence block v3+ */
+ MIPI_SEQ_MAX
+};
-struct bdb_edp {
- struct edp_power_seq power_seqs[16];
- u32 color_depth;
- struct edp_link_params link_params[16];
- u32 sdrrs_msa_timing_delay;
-
- /* ith bit indicates enabled/disabled for (i+1)th panel */
- u16 edp_s3d_feature;
- u16 edp_t3_optimization;
- u64 edp_vswing_preemph; /* v173 */
-} __packed;
-
-struct psr_table {
- /* Feature bits */
- u8 full_link:1;
- u8 require_aux_to_wakeup:1;
- u8 feature_bits_rsvd:6;
-
- /* Wait times */
- u8 idle_frames:4;
- u8 lines_to_wait:3;
- u8 wait_times_rsvd:1;
-
- /* TP wake up time in multiple of 100 */
- u16 tp1_wakeup_time;
- u16 tp2_tp3_wakeup_time;
-} __packed;
-
-struct bdb_psr {
- struct psr_table psr_table[16];
-} __packed;
-
-/*
- * Driver<->VBIOS interaction occurs through scratch bits in
- * GR18 & SWF*.
- */
-
-/* GR18 bits are set on display switch and hotkey events */
-#define GR18_DRIVER_SWITCH_EN (1<<7) /* 0: VBIOS control, 1: driver control */
-#define GR18_HOTKEY_MASK 0x78 /* See also SWF4 15:0 */
-#define GR18_HK_NONE (0x0<<3)
-#define GR18_HK_LFP_STRETCH (0x1<<3)
-#define GR18_HK_TOGGLE_DISP (0x2<<3)
-#define GR18_HK_DISP_SWITCH (0x4<<3) /* see SWF14 15:0 for what to enable */
-#define GR18_HK_POPUP_DISABLED (0x6<<3)
-#define GR18_HK_POPUP_ENABLED (0x7<<3)
-#define GR18_HK_PFIT (0x8<<3)
-#define GR18_HK_APM_CHANGE (0xa<<3)
-#define GR18_HK_MULTIPLE (0xc<<3)
-#define GR18_USER_INT_EN (1<<2)
-#define GR18_A0000_FLUSH_EN (1<<1)
-#define GR18_SMM_EN (1<<0)
-
-/* Set by driver, cleared by VBIOS */
-#define SWF00_YRES_SHIFT 16
-#define SWF00_XRES_SHIFT 0
-#define SWF00_RES_MASK 0xffff
-
-/* Set by VBIOS at boot time and driver at runtime */
-#define SWF01_TV2_FORMAT_SHIFT 8
-#define SWF01_TV1_FORMAT_SHIFT 0
-#define SWF01_TV_FORMAT_MASK 0xffff
-
-#define SWF10_VBIOS_BLC_I2C_EN (1<<29)
-#define SWF10_GTT_OVERRIDE_EN (1<<28)
-#define SWF10_LFP_DPMS_OVR (1<<27) /* override DPMS on display switch */
-#define SWF10_ACTIVE_TOGGLE_LIST_MASK (7<<24)
-#define SWF10_OLD_TOGGLE 0x0
-#define SWF10_TOGGLE_LIST_1 0x1
-#define SWF10_TOGGLE_LIST_2 0x2
-#define SWF10_TOGGLE_LIST_3 0x3
-#define SWF10_TOGGLE_LIST_4 0x4
-#define SWF10_PANNING_EN (1<<23)
-#define SWF10_DRIVER_LOADED (1<<22)
-#define SWF10_EXTENDED_DESKTOP (1<<21)
-#define SWF10_EXCLUSIVE_MODE (1<<20)
-#define SWF10_OVERLAY_EN (1<<19)
-#define SWF10_PLANEB_HOLDOFF (1<<18)
-#define SWF10_PLANEA_HOLDOFF (1<<17)
-#define SWF10_VGA_HOLDOFF (1<<16)
-#define SWF10_ACTIVE_DISP_MASK 0xffff
-#define SWF10_PIPEB_LFP2 (1<<15)
-#define SWF10_PIPEB_EFP2 (1<<14)
-#define SWF10_PIPEB_TV2 (1<<13)
-#define SWF10_PIPEB_CRT2 (1<<12)
-#define SWF10_PIPEB_LFP (1<<11)
-#define SWF10_PIPEB_EFP (1<<10)
-#define SWF10_PIPEB_TV (1<<9)
-#define SWF10_PIPEB_CRT (1<<8)
-#define SWF10_PIPEA_LFP2 (1<<7)
-#define SWF10_PIPEA_EFP2 (1<<6)
-#define SWF10_PIPEA_TV2 (1<<5)
-#define SWF10_PIPEA_CRT2 (1<<4)
-#define SWF10_PIPEA_LFP (1<<3)
-#define SWF10_PIPEA_EFP (1<<2)
-#define SWF10_PIPEA_TV (1<<1)
-#define SWF10_PIPEA_CRT (1<<0)
-
-#define SWF11_MEMORY_SIZE_SHIFT 16
-#define SWF11_SV_TEST_EN (1<<15)
-#define SWF11_IS_AGP (1<<14)
-#define SWF11_DISPLAY_HOLDOFF (1<<13)
-#define SWF11_DPMS_REDUCED (1<<12)
-#define SWF11_IS_VBE_MODE (1<<11)
-#define SWF11_PIPEB_ACCESS (1<<10) /* 0 here means pipe a */
-#define SWF11_DPMS_MASK 0x07
-#define SWF11_DPMS_OFF (1<<2)
-#define SWF11_DPMS_SUSPEND (1<<1)
-#define SWF11_DPMS_STANDBY (1<<0)
-#define SWF11_DPMS_ON 0
-
-#define SWF14_GFX_PFIT_EN (1<<31)
-#define SWF14_TEXT_PFIT_EN (1<<30)
-#define SWF14_LID_STATUS_CLOSED (1<<29) /* 0 here means open */
-#define SWF14_POPUP_EN (1<<28)
-#define SWF14_DISPLAY_HOLDOFF (1<<27)
-#define SWF14_DISP_DETECT_EN (1<<26)
-#define SWF14_DOCKING_STATUS_DOCKED (1<<25) /* 0 here means undocked */
-#define SWF14_DRIVER_STATUS (1<<24)
-#define SWF14_OS_TYPE_WIN9X (1<<23)
-#define SWF14_OS_TYPE_WINNT (1<<22)
-/* 21:19 rsvd */
-#define SWF14_PM_TYPE_MASK 0x00070000
-#define SWF14_PM_ACPI_VIDEO (0x4 << 16)
-#define SWF14_PM_ACPI (0x3 << 16)
-#define SWF14_PM_APM_12 (0x2 << 16)
-#define SWF14_PM_APM_11 (0x1 << 16)
-#define SWF14_HK_REQUEST_MASK 0x0000ffff /* see GR18 6:3 for event type */
- /* if GR18 indicates a display switch */
-#define SWF14_DS_PIPEB_LFP2_EN (1<<15)
-#define SWF14_DS_PIPEB_EFP2_EN (1<<14)
-#define SWF14_DS_PIPEB_TV2_EN (1<<13)
-#define SWF14_DS_PIPEB_CRT2_EN (1<<12)
-#define SWF14_DS_PIPEB_LFP_EN (1<<11)
-#define SWF14_DS_PIPEB_EFP_EN (1<<10)
-#define SWF14_DS_PIPEB_TV_EN (1<<9)
-#define SWF14_DS_PIPEB_CRT_EN (1<<8)
-#define SWF14_DS_PIPEA_LFP2_EN (1<<7)
-#define SWF14_DS_PIPEA_EFP2_EN (1<<6)
-#define SWF14_DS_PIPEA_TV2_EN (1<<5)
-#define SWF14_DS_PIPEA_CRT2_EN (1<<4)
-#define SWF14_DS_PIPEA_LFP_EN (1<<3)
-#define SWF14_DS_PIPEA_EFP_EN (1<<2)
-#define SWF14_DS_PIPEA_TV_EN (1<<1)
-#define SWF14_DS_PIPEA_CRT_EN (1<<0)
- /* if GR18 indicates a panel fitting request */
-#define SWF14_PFIT_EN (1<<0) /* 0 means disable */
- /* if GR18 indicates an APM change request */
-#define SWF14_APM_HIBERNATE 0x4
-#define SWF14_APM_SUSPEND 0x3
-#define SWF14_APM_STANDBY 0x1
-#define SWF14_APM_RESTORE 0x0
-
-/* Add the device class for LFP, TV, HDMI */
-#define DEVICE_TYPE_INT_LFP 0x1022
-#define DEVICE_TYPE_INT_TV 0x1009
-#define DEVICE_TYPE_HDMI 0x60D2
-#define DEVICE_TYPE_DP 0x68C6
-#define DEVICE_TYPE_eDP 0x78C6
-
-#define DEVICE_TYPE_CLASS_EXTENSION (1 << 15)
-#define DEVICE_TYPE_POWER_MANAGEMENT (1 << 14)
-#define DEVICE_TYPE_HOTPLUG_SIGNALING (1 << 13)
-#define DEVICE_TYPE_INTERNAL_CONNECTOR (1 << 12)
-#define DEVICE_TYPE_NOT_HDMI_OUTPUT (1 << 11)
-#define DEVICE_TYPE_MIPI_OUTPUT (1 << 10)
-#define DEVICE_TYPE_COMPOSITE_OUTPUT (1 << 9)
-#define DEVICE_TYPE_DUAL_CHANNEL (1 << 8)
-#define DEVICE_TYPE_HIGH_SPEED_LINK (1 << 6)
-#define DEVICE_TYPE_LVDS_SINGALING (1 << 5)
-#define DEVICE_TYPE_TMDS_DVI_SIGNALING (1 << 4)
-#define DEVICE_TYPE_VIDEO_SIGNALING (1 << 3)
-#define DEVICE_TYPE_DISPLAYPORT_OUTPUT (1 << 2)
-#define DEVICE_TYPE_DIGITAL_OUTPUT (1 << 1)
-#define DEVICE_TYPE_ANALOG_OUTPUT (1 << 0)
-
-/*
- * Bits we care about when checking for DEVICE_TYPE_eDP
- * Depending on the system, the other bits may or may not
- * be set for eDP outputs.
- */
-#define DEVICE_TYPE_eDP_BITS \
- (DEVICE_TYPE_INTERNAL_CONNECTOR | \
- DEVICE_TYPE_MIPI_OUTPUT | \
- DEVICE_TYPE_COMPOSITE_OUTPUT | \
- DEVICE_TYPE_DUAL_CHANNEL | \
- DEVICE_TYPE_LVDS_SINGALING | \
- DEVICE_TYPE_TMDS_DVI_SIGNALING | \
- DEVICE_TYPE_VIDEO_SIGNALING | \
- DEVICE_TYPE_DISPLAYPORT_OUTPUT | \
- DEVICE_TYPE_ANALOG_OUTPUT)
-
-/* define the DVO port for HDMI output type */
-#define DVO_B 1
-#define DVO_C 2
-#define DVO_D 3
-
-/* Possible values for the "DVO Port" field for versions >= 155: */
-#define DVO_PORT_HDMIA 0
-#define DVO_PORT_HDMIB 1
-#define DVO_PORT_HDMIC 2
-#define DVO_PORT_HDMID 3
-#define DVO_PORT_LVDS 4
-#define DVO_PORT_TV 5
-#define DVO_PORT_CRT 6
-#define DVO_PORT_DPB 7
-#define DVO_PORT_DPC 8
-#define DVO_PORT_DPD 9
-#define DVO_PORT_DPA 10
-#define DVO_PORT_DPE 11
-#define DVO_PORT_HDMIE 12
-#define DVO_PORT_MIPIA 21
-#define DVO_PORT_MIPIB 22
-#define DVO_PORT_MIPIC 23
-#define DVO_PORT_MIPID 24
-
-/* Block 52 contains MIPI Panel info
- * 6 such enteries will there. Index into correct
- * entery is based on the panel_index in #40 LFP
- */
-#define MAX_MIPI_CONFIGURATIONS 6
+enum mipi_seq_element {
+ MIPI_SEQ_ELEM_END = 0,
+ MIPI_SEQ_ELEM_SEND_PKT,
+ MIPI_SEQ_ELEM_DELAY,
+ MIPI_SEQ_ELEM_GPIO,
+ MIPI_SEQ_ELEM_I2C, /* sequence block v2+ */
+ MIPI_SEQ_ELEM_SPI, /* sequence block v3+ */
+ MIPI_SEQ_ELEM_PMIC, /* sequence block v3+ */
+ MIPI_SEQ_ELEM_MAX
+};
#define MIPI_DSI_UNDEFINED_PANEL_ID 0
#define MIPI_DSI_GENERIC_PANEL_ID 1
-/*
- * PMIC vs SoC Backlight support specified in pwm_blc
- * field in mipi_config block below.
-*/
-#define PPS_BLC_PMIC 0
-#define PPS_BLC_SOC 1
-
struct mipi_config {
u16 panel_id;
@@ -821,6 +86,8 @@
u32 video_transfer_mode:2;
u32 cabc_supported:1;
+#define PPS_BLC_PMIC 0
+#define PPS_BLC_SOC 1
u32 pwm_blc:1;
/* Bit 13:10 */
@@ -924,12 +191,7 @@
} __packed;
-/* Block 52 contains MIPI configuration block
- * 6 * bdb_mipi_config, followed by 6 pps data
- * block below
- *
- * all delays has a unit of 100us
- */
+/* all delays have a unit of 100us */
struct mipi_pps_data {
u16 panel_on_delay;
u16 bl_enable_delay;
@@ -938,57 +200,4 @@
u16 panel_power_cycle_delay;
} __packed;
-struct bdb_mipi_config {
- struct mipi_config config[MAX_MIPI_CONFIGURATIONS];
- struct mipi_pps_data pps[MAX_MIPI_CONFIGURATIONS];
-} __packed;
-
-/* Block 53 contains MIPI sequences as needed by the panel
- * for enabling it. This block can be variable in size and
- * can be maximum of 6 blocks
- */
-struct bdb_mipi_sequence {
- u8 version;
- u8 data[0];
-} __packed;
-
-/* MIPI Sequnece Block definitions */
-enum mipi_seq {
- MIPI_SEQ_END = 0,
- MIPI_SEQ_ASSERT_RESET,
- MIPI_SEQ_INIT_OTP,
- MIPI_SEQ_DISPLAY_ON,
- MIPI_SEQ_DISPLAY_OFF,
- MIPI_SEQ_DEASSERT_RESET,
- MIPI_SEQ_BACKLIGHT_ON, /* sequence block v2+ */
- MIPI_SEQ_BACKLIGHT_OFF, /* sequence block v2+ */
- MIPI_SEQ_TEAR_ON, /* sequence block v2+ */
- MIPI_SEQ_TEAR_OFF, /* sequence block v3+ */
- MIPI_SEQ_POWER_ON, /* sequence block v3+ */
- MIPI_SEQ_POWER_OFF, /* sequence block v3+ */
- MIPI_SEQ_MAX
-};
-
-enum mipi_seq_element {
- MIPI_SEQ_ELEM_END = 0,
- MIPI_SEQ_ELEM_SEND_PKT,
- MIPI_SEQ_ELEM_DELAY,
- MIPI_SEQ_ELEM_GPIO,
- MIPI_SEQ_ELEM_I2C, /* sequence block v2+ */
- MIPI_SEQ_ELEM_SPI, /* sequence block v3+ */
- MIPI_SEQ_ELEM_PMIC, /* sequence block v3+ */
- MIPI_SEQ_ELEM_MAX
-};
-
-enum mipi_gpio_pin_index {
- MIPI_GPIO_UNDEFINED = 0,
- MIPI_GPIO_PANEL_ENABLE,
- MIPI_GPIO_BL_ENABLE,
- MIPI_GPIO_PWM_ENABLE,
- MIPI_GPIO_RESET_N,
- MIPI_GPIO_PWR_DOWN_R,
- MIPI_GPIO_STDBY_RST_N,
- MIPI_GPIO_MAX
-};
-
#endif /* _INTEL_BIOS_H_ */
diff --git a/drivers/gpu/drm/i915/intel_color.c b/drivers/gpu/drm/i915/intel_color.c
new file mode 100644
index 0000000..1b3f974
--- /dev/null
+++ b/drivers/gpu/drm/i915/intel_color.c
@@ -0,0 +1,553 @@
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#include "intel_drv.h"
+
+#define CTM_COEFF_SIGN (1ULL << 63)
+
+#define CTM_COEFF_1_0 (1ULL << 32)
+#define CTM_COEFF_2_0 (CTM_COEFF_1_0 << 1)
+#define CTM_COEFF_4_0 (CTM_COEFF_2_0 << 1)
+#define CTM_COEFF_8_0 (CTM_COEFF_4_0 << 1)
+#define CTM_COEFF_0_5 (CTM_COEFF_1_0 >> 1)
+#define CTM_COEFF_0_25 (CTM_COEFF_0_5 >> 1)
+#define CTM_COEFF_0_125 (CTM_COEFF_0_25 >> 1)
+
+#define CTM_COEFF_LIMITED_RANGE ((235ULL - 16ULL) * CTM_COEFF_1_0 / 255)
+
+#define CTM_COEFF_NEGATIVE(coeff) (((coeff) & CTM_COEFF_SIGN) != 0)
+#define CTM_COEFF_ABS(coeff) ((coeff) & (CTM_COEFF_SIGN - 1))
+
+#define LEGACY_LUT_LENGTH (sizeof(struct drm_color_lut) * 256)
+
+/*
+ * Extract the CSC coefficient from a CTM coefficient (in U32.32 fixed point
+ * format). This macro takes the coefficient we want transformed and the
+ * number of fractional bits.
+ *
+ * We only have a 9 bits precision window which slides depending on the value
+ * of the CTM coefficient and we write the value from bit 3. We also round the
+ * value.
+ */
+#define I9XX_CSC_COEFF_FP(coeff, fbits) \
+ (clamp_val(((coeff) >> (32 - (fbits) - 3)) + 4, 0, 0xfff) & 0xff8)
+
+#define I9XX_CSC_COEFF_LIMITED_RANGE \
+ I9XX_CSC_COEFF_FP(CTM_COEFF_LIMITED_RANGE, 9)
+#define I9XX_CSC_COEFF_1_0 \
+ ((7 << 12) | I9XX_CSC_COEFF_FP(CTM_COEFF_1_0, 8))
+
+static bool crtc_state_is_legacy(struct drm_crtc_state *state)
+{
+ return !state->degamma_lut &&
+ !state->ctm &&
+ state->gamma_lut &&
+ state->gamma_lut->length == LEGACY_LUT_LENGTH;
+}
+
+/*
+ * When using limited range, multiply the matrix given by userspace by
+ * the matrix that we would use for the limited range. We do the
+ * multiplication in U2.30 format.
+ */
+static void ctm_mult_by_limited(uint64_t *result, int64_t *input)
+{
+ int i;
+
+ for (i = 0; i < 9; i++)
+ result[i] = 0;
+
+ for (i = 0; i < 3; i++) {
+ int64_t user_coeff = input[i * 3 + i];
+ uint64_t limited_coeff = CTM_COEFF_LIMITED_RANGE >> 2;
+ uint64_t abs_coeff = clamp_val(CTM_COEFF_ABS(user_coeff),
+ 0,
+ CTM_COEFF_4_0 - 1) >> 2;
+
+ result[i * 3 + i] = (limited_coeff * abs_coeff) >> 27;
+ if (CTM_COEFF_NEGATIVE(user_coeff))
+ result[i * 3 + i] |= CTM_COEFF_SIGN;
+ }
+}
+
+/* Set up the pipe CSC unit. */
+static void i9xx_load_csc_matrix(struct drm_crtc_state *crtc_state)
+{
+ struct drm_crtc *crtc = crtc_state->crtc;
+ struct drm_device *dev = crtc->dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
+ int i, pipe = intel_crtc->pipe;
+ uint16_t coeffs[9] = { 0, };
+
+ if (crtc_state->ctm) {
+ struct drm_color_ctm *ctm =
+ (struct drm_color_ctm *)crtc_state->ctm->data;
+ uint64_t input[9] = { 0, };
+
+ if (intel_crtc->config->limited_color_range) {
+ ctm_mult_by_limited(input, ctm->matrix);
+ } else {
+ for (i = 0; i < ARRAY_SIZE(input); i++)
+ input[i] = ctm->matrix[i];
+ }
+
+ /*
+ * Convert fixed point S31.32 input to format supported by the
+ * hardware.
+ */
+ for (i = 0; i < ARRAY_SIZE(coeffs); i++) {
+ uint64_t abs_coeff = ((1ULL << 63) - 1) & input[i];
+
+ /*
+ * Clamp input value to min/max supported by
+ * hardware.
+ */
+ abs_coeff = clamp_val(abs_coeff, 0, CTM_COEFF_4_0 - 1);
+
+ /* sign bit */
+ if (CTM_COEFF_NEGATIVE(input[i]))
+ coeffs[i] |= 1 << 15;
+
+ if (abs_coeff < CTM_COEFF_0_125)
+ coeffs[i] |= (3 << 12) |
+ I9XX_CSC_COEFF_FP(abs_coeff, 12);
+ else if (abs_coeff < CTM_COEFF_0_25)
+ coeffs[i] |= (2 << 12) |
+ I9XX_CSC_COEFF_FP(abs_coeff, 11);
+ else if (abs_coeff < CTM_COEFF_0_5)
+ coeffs[i] |= (1 << 12) |
+ I9XX_CSC_COEFF_FP(abs_coeff, 10);
+ else if (abs_coeff < CTM_COEFF_1_0)
+ coeffs[i] |= I9XX_CSC_COEFF_FP(abs_coeff, 9);
+ else if (abs_coeff < CTM_COEFF_2_0)
+ coeffs[i] |= (7 << 12) |
+ I9XX_CSC_COEFF_FP(abs_coeff, 8);
+ else
+ coeffs[i] |= (6 << 12) |
+ I9XX_CSC_COEFF_FP(abs_coeff, 7);
+ }
+ } else {
+ /*
+ * Load an identity matrix if no coefficients are provided.
+ *
+ * TODO: Check what kind of values actually come out of the
+ * pipe with these coeff/postoff values and adjust to get the
+ * best accuracy. Perhaps we even need to take the bpc value
+ * into consideration.
+ */
+ for (i = 0; i < 3; i++) {
+ if (intel_crtc->config->limited_color_range)
+ coeffs[i * 3 + i] =
+ I9XX_CSC_COEFF_LIMITED_RANGE;
+ else
+ coeffs[i * 3 + i] = I9XX_CSC_COEFF_1_0;
+ }
+ }
+
+ I915_WRITE(PIPE_CSC_COEFF_RY_GY(pipe), coeffs[0] << 16 | coeffs[1]);
+ I915_WRITE(PIPE_CSC_COEFF_BY(pipe), coeffs[2] << 16);
+
+ I915_WRITE(PIPE_CSC_COEFF_RU_GU(pipe), coeffs[3] << 16 | coeffs[4]);
+ I915_WRITE(PIPE_CSC_COEFF_BU(pipe), coeffs[5] << 16);
+
+ I915_WRITE(PIPE_CSC_COEFF_RV_GV(pipe), coeffs[6] << 16 | coeffs[7]);
+ I915_WRITE(PIPE_CSC_COEFF_BV(pipe), coeffs[8] << 16);
+
+ I915_WRITE(PIPE_CSC_PREOFF_HI(pipe), 0);
+ I915_WRITE(PIPE_CSC_PREOFF_ME(pipe), 0);
+ I915_WRITE(PIPE_CSC_PREOFF_LO(pipe), 0);
+
+ if (INTEL_INFO(dev)->gen > 6) {
+ uint16_t postoff = 0;
+
+ if (intel_crtc->config->limited_color_range)
+ postoff = (16 * (1 << 12) / 255) & 0x1fff;
+
+ I915_WRITE(PIPE_CSC_POSTOFF_HI(pipe), postoff);
+ I915_WRITE(PIPE_CSC_POSTOFF_ME(pipe), postoff);
+ I915_WRITE(PIPE_CSC_POSTOFF_LO(pipe), postoff);
+
+ I915_WRITE(PIPE_CSC_MODE(pipe), 0);
+ } else {
+ uint32_t mode = CSC_MODE_YUV_TO_RGB;
+
+ if (intel_crtc->config->limited_color_range)
+ mode |= CSC_BLACK_SCREEN_OFFSET;
+
+ I915_WRITE(PIPE_CSC_MODE(pipe), mode);
+ }
+}
+
+/*
+ * Set up the pipe CSC unit on CherryView.
+ */
+static void cherryview_load_csc_matrix(struct drm_crtc_state *state)
+{
+ struct drm_crtc *crtc = state->crtc;
+ struct drm_device *dev = crtc->dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ int pipe = to_intel_crtc(crtc)->pipe;
+ uint32_t mode;
+
+ if (state->ctm) {
+ struct drm_color_ctm *ctm =
+ (struct drm_color_ctm *) state->ctm->data;
+ uint16_t coeffs[9] = { 0, };
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(coeffs); i++) {
+ uint64_t abs_coeff =
+ ((1ULL << 63) - 1) & ctm->matrix[i];
+
+ /* Round coefficient. */
+ abs_coeff += 1 << (32 - 13);
+ /* Clamp to hardware limits. */
+ abs_coeff = clamp_val(abs_coeff, 0, CTM_COEFF_8_0 - 1);
+
+ /* Write coefficients in S3.12 format. */
+ if (ctm->matrix[i] & (1ULL << 63))
+ coeffs[i] = 1 << 15;
+ coeffs[i] |= ((abs_coeff >> 32) & 7) << 12;
+ coeffs[i] |= (abs_coeff >> 20) & 0xfff;
+ }
+
+ I915_WRITE(CGM_PIPE_CSC_COEFF01(pipe),
+ coeffs[1] << 16 | coeffs[0]);
+ I915_WRITE(CGM_PIPE_CSC_COEFF23(pipe),
+ coeffs[3] << 16 | coeffs[2]);
+ I915_WRITE(CGM_PIPE_CSC_COEFF45(pipe),
+ coeffs[5] << 16 | coeffs[4]);
+ I915_WRITE(CGM_PIPE_CSC_COEFF67(pipe),
+ coeffs[7] << 16 | coeffs[6]);
+ I915_WRITE(CGM_PIPE_CSC_COEFF8(pipe), coeffs[8]);
+ }
+
+ mode = (state->ctm ? CGM_PIPE_MODE_CSC : 0);
+ if (!crtc_state_is_legacy(state)) {
+ mode |= (state->degamma_lut ? CGM_PIPE_MODE_DEGAMMA : 0) |
+ (state->gamma_lut ? CGM_PIPE_MODE_GAMMA : 0);
+ }
+ I915_WRITE(CGM_PIPE_MODE(pipe), mode);
+}
+
+void intel_color_set_csc(struct drm_crtc_state *crtc_state)
+{
+ struct drm_device *dev = crtc_state->crtc->dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+
+ if (dev_priv->display.load_csc_matrix)
+ dev_priv->display.load_csc_matrix(crtc_state);
+}
+
+/* Loads the legacy palette/gamma unit for the CRTC. */
+static void i9xx_load_luts_internal(struct drm_crtc *crtc,
+ struct drm_property_blob *blob)
+{
+ struct drm_device *dev = crtc->dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
+ enum pipe pipe = intel_crtc->pipe;
+ int i;
+
+ if (HAS_GMCH_DISPLAY(dev)) {
+ if (intel_crtc->config->has_dsi_encoder)
+ assert_dsi_pll_enabled(dev_priv);
+ else
+ assert_pll_enabled(dev_priv, pipe);
+ }
+
+ if (blob) {
+ struct drm_color_lut *lut = (struct drm_color_lut *) blob->data;
+ for (i = 0; i < 256; i++) {
+ uint32_t word =
+ (drm_color_lut_extract(lut[i].red, 8) << 16) |
+ (drm_color_lut_extract(lut[i].green, 8) << 8) |
+ drm_color_lut_extract(lut[i].blue, 8);
+
+ if (HAS_GMCH_DISPLAY(dev))
+ I915_WRITE(PALETTE(pipe, i), word);
+ else
+ I915_WRITE(LGC_PALETTE(pipe, i), word);
+ }
+ } else {
+ for (i = 0; i < 256; i++) {
+ uint32_t word = (i << 16) | (i << 8) | i;
+
+ if (HAS_GMCH_DISPLAY(dev))
+ I915_WRITE(PALETTE(pipe, i), word);
+ else
+ I915_WRITE(LGC_PALETTE(pipe, i), word);
+ }
+ }
+}
+
+static void i9xx_load_luts(struct drm_crtc_state *crtc_state)
+{
+ i9xx_load_luts_internal(crtc_state->crtc, crtc_state->gamma_lut);
+}
+
+/* Loads the legacy palette/gamma unit for the CRTC on Haswell. */
+static void haswell_load_luts(struct drm_crtc_state *crtc_state)
+{
+ struct drm_crtc *crtc = crtc_state->crtc;
+ struct drm_device *dev = crtc->dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
+ struct intel_crtc_state *intel_crtc_state =
+ to_intel_crtc_state(crtc_state);
+ bool reenable_ips = false;
+
+ /*
+ * Workaround : Do not read or write the pipe palette/gamma data while
+ * GAMMA_MODE is configured for split gamma and IPS_CTL has IPS enabled.
+ */
+ if (IS_HASWELL(dev) && intel_crtc->config->ips_enabled &&
+ (intel_crtc_state->gamma_mode == GAMMA_MODE_MODE_SPLIT)) {
+ hsw_disable_ips(intel_crtc);
+ reenable_ips = true;
+ }
+
+ intel_crtc_state->gamma_mode = GAMMA_MODE_MODE_8BIT;
+ I915_WRITE(GAMMA_MODE(intel_crtc->pipe), GAMMA_MODE_MODE_8BIT);
+
+ i9xx_load_luts(crtc_state);
+
+ if (reenable_ips)
+ hsw_enable_ips(intel_crtc);
+}
+
+/* Loads the palette/gamma unit for the CRTC on Broadwell+. */
+static void broadwell_load_luts(struct drm_crtc_state *state)
+{
+ struct drm_crtc *crtc = state->crtc;
+ struct drm_device *dev = crtc->dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct intel_crtc_state *intel_state = to_intel_crtc_state(state);
+ enum pipe pipe = to_intel_crtc(crtc)->pipe;
+ uint32_t i, lut_size = INTEL_INFO(dev)->color.degamma_lut_size;
+
+ if (crtc_state_is_legacy(state)) {
+ haswell_load_luts(state);
+ return;
+ }
+
+ I915_WRITE(PREC_PAL_INDEX(pipe),
+ PAL_PREC_SPLIT_MODE | PAL_PREC_AUTO_INCREMENT);
+
+ if (state->degamma_lut) {
+ struct drm_color_lut *lut =
+ (struct drm_color_lut *) state->degamma_lut->data;
+
+ for (i = 0; i < lut_size; i++) {
+ uint32_t word =
+ drm_color_lut_extract(lut[i].red, 10) << 20 |
+ drm_color_lut_extract(lut[i].green, 10) << 10 |
+ drm_color_lut_extract(lut[i].blue, 10);
+
+ I915_WRITE(PREC_PAL_DATA(pipe), word);
+ }
+ } else {
+ for (i = 0; i < lut_size; i++) {
+ uint32_t v = (i * ((1 << 10) - 1)) / (lut_size - 1);
+
+ I915_WRITE(PREC_PAL_DATA(pipe),
+ (v << 20) | (v << 10) | v);
+ }
+ }
+
+ if (state->gamma_lut) {
+ struct drm_color_lut *lut =
+ (struct drm_color_lut *) state->gamma_lut->data;
+
+ for (i = 0; i < lut_size; i++) {
+ uint32_t word =
+ (drm_color_lut_extract(lut[i].red, 10) << 20) |
+ (drm_color_lut_extract(lut[i].green, 10) << 10) |
+ drm_color_lut_extract(lut[i].blue, 10);
+
+ I915_WRITE(PREC_PAL_DATA(pipe), word);
+ }
+
+ /* Program the max register to clamp values > 1.0. */
+ I915_WRITE(PREC_PAL_GC_MAX(pipe, 0),
+ drm_color_lut_extract(lut[i].red, 16));
+ I915_WRITE(PREC_PAL_GC_MAX(pipe, 1),
+ drm_color_lut_extract(lut[i].green, 16));
+ I915_WRITE(PREC_PAL_GC_MAX(pipe, 2),
+ drm_color_lut_extract(lut[i].blue, 16));
+ } else {
+ for (i = 0; i < lut_size; i++) {
+ uint32_t v = (i * ((1 << 10) - 1)) / (lut_size - 1);
+
+ I915_WRITE(PREC_PAL_DATA(pipe),
+ (v << 20) | (v << 10) | v);
+ }
+
+ I915_WRITE(PREC_PAL_GC_MAX(pipe, 0), (1 << 16) - 1);
+ I915_WRITE(PREC_PAL_GC_MAX(pipe, 1), (1 << 16) - 1);
+ I915_WRITE(PREC_PAL_GC_MAX(pipe, 2), (1 << 16) - 1);
+ }
+
+ intel_state->gamma_mode = GAMMA_MODE_MODE_SPLIT;
+ I915_WRITE(GAMMA_MODE(pipe), GAMMA_MODE_MODE_SPLIT);
+ POSTING_READ(GAMMA_MODE(pipe));
+
+ /*
+ * Reset the index, otherwise it prevents the legacy palette to be
+ * written properly.
+ */
+ I915_WRITE(PREC_PAL_INDEX(pipe), 0);
+}
+
+/* Loads the palette/gamma unit for the CRTC on CherryView. */
+static void cherryview_load_luts(struct drm_crtc_state *state)
+{
+ struct drm_crtc *crtc = state->crtc;
+ struct drm_device *dev = crtc->dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ enum pipe pipe = to_intel_crtc(crtc)->pipe;
+ struct drm_color_lut *lut;
+ uint32_t i, lut_size;
+ uint32_t word0, word1;
+
+ if (crtc_state_is_legacy(state)) {
+ /* Turn off degamma/gamma on CGM block. */
+ I915_WRITE(CGM_PIPE_MODE(pipe),
+ (state->ctm ? CGM_PIPE_MODE_CSC : 0));
+ i9xx_load_luts_internal(crtc, state->gamma_lut);
+ return;
+ }
+
+ if (state->degamma_lut) {
+ lut = (struct drm_color_lut *) state->degamma_lut->data;
+ lut_size = INTEL_INFO(dev)->color.degamma_lut_size;
+ for (i = 0; i < lut_size; i++) {
+ /* Write LUT in U0.14 format. */
+ word0 =
+ (drm_color_lut_extract(lut[i].green, 14) << 16) |
+ drm_color_lut_extract(lut[i].blue, 14);
+ word1 = drm_color_lut_extract(lut[i].red, 14);
+
+ I915_WRITE(CGM_PIPE_DEGAMMA(pipe, i, 0), word0);
+ I915_WRITE(CGM_PIPE_DEGAMMA(pipe, i, 1), word1);
+ }
+ }
+
+ if (state->gamma_lut) {
+ lut = (struct drm_color_lut *) state->gamma_lut->data;
+ lut_size = INTEL_INFO(dev)->color.gamma_lut_size;
+ for (i = 0; i < lut_size; i++) {
+ /* Write LUT in U0.10 format. */
+ word0 =
+ (drm_color_lut_extract(lut[i].green, 10) << 16) |
+ drm_color_lut_extract(lut[i].blue, 10);
+ word1 = drm_color_lut_extract(lut[i].red, 10);
+
+ I915_WRITE(CGM_PIPE_GAMMA(pipe, i, 0), word0);
+ I915_WRITE(CGM_PIPE_GAMMA(pipe, i, 1), word1);
+ }
+ }
+
+ I915_WRITE(CGM_PIPE_MODE(pipe),
+ (state->ctm ? CGM_PIPE_MODE_CSC : 0) |
+ (state->degamma_lut ? CGM_PIPE_MODE_DEGAMMA : 0) |
+ (state->gamma_lut ? CGM_PIPE_MODE_GAMMA : 0));
+
+ /*
+ * Also program a linear LUT in the legacy block (behind the
+ * CGM block).
+ */
+ i9xx_load_luts_internal(crtc, NULL);
+}
+
+void intel_color_load_luts(struct drm_crtc_state *crtc_state)
+{
+ struct drm_device *dev = crtc_state->crtc->dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+
+ dev_priv->display.load_luts(crtc_state);
+}
+
+int intel_color_check(struct drm_crtc *crtc,
+ struct drm_crtc_state *crtc_state)
+{
+ struct drm_device *dev = crtc->dev;
+ size_t gamma_length, degamma_length;
+
+ degamma_length = INTEL_INFO(dev)->color.degamma_lut_size *
+ sizeof(struct drm_color_lut);
+ gamma_length = INTEL_INFO(dev)->color.gamma_lut_size *
+ sizeof(struct drm_color_lut);
+
+ /*
+ * We allow both degamma & gamma luts at the right size or
+ * NULL.
+ */
+ if ((!crtc_state->degamma_lut ||
+ crtc_state->degamma_lut->length == degamma_length) &&
+ (!crtc_state->gamma_lut ||
+ crtc_state->gamma_lut->length == gamma_length))
+ return 0;
+
+ /*
+ * We also allow no degamma lut and a gamma lut at the legacy
+ * size (256 entries).
+ */
+ if (!crtc_state->degamma_lut &&
+ crtc_state->gamma_lut &&
+ crtc_state->gamma_lut->length == LEGACY_LUT_LENGTH)
+ return 0;
+
+ return -EINVAL;
+}
+
+void intel_color_init(struct drm_crtc *crtc)
+{
+ struct drm_device *dev = crtc->dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+
+ drm_mode_crtc_set_gamma_size(crtc, 256);
+
+ if (IS_CHERRYVIEW(dev)) {
+ dev_priv->display.load_csc_matrix = cherryview_load_csc_matrix;
+ dev_priv->display.load_luts = cherryview_load_luts;
+ } else if (IS_HASWELL(dev)) {
+ dev_priv->display.load_csc_matrix = i9xx_load_csc_matrix;
+ dev_priv->display.load_luts = haswell_load_luts;
+ } else if (IS_BROADWELL(dev) || IS_SKYLAKE(dev) ||
+ IS_BROXTON(dev) || IS_KABYLAKE(dev)) {
+ dev_priv->display.load_csc_matrix = i9xx_load_csc_matrix;
+ dev_priv->display.load_luts = broadwell_load_luts;
+ } else {
+ dev_priv->display.load_luts = i9xx_load_luts;
+ }
+
+ /* Enable color management support when we have degamma & gamma LUTs. */
+ if (INTEL_INFO(dev)->color.degamma_lut_size != 0 &&
+ INTEL_INFO(dev)->color.gamma_lut_size != 0)
+ drm_helper_crtc_enable_color_mgmt(crtc,
+ INTEL_INFO(dev)->color.degamma_lut_size,
+ INTEL_INFO(dev)->color.gamma_lut_size);
+}
diff --git a/drivers/gpu/drm/i915/intel_crt.c b/drivers/gpu/drm/i915/intel_crt.c
index 0364292..3fbb6fc 100644
--- a/drivers/gpu/drm/i915/intel_crt.c
+++ b/drivers/gpu/drm/i915/intel_crt.c
@@ -120,22 +120,16 @@
static void intel_crt_get_config(struct intel_encoder *encoder,
struct intel_crtc_state *pipe_config)
{
- struct drm_device *dev = encoder->base.dev;
- int dotclock;
-
pipe_config->base.adjusted_mode.flags |= intel_crt_get_flags(encoder);
- dotclock = pipe_config->port_clock;
-
- if (HAS_PCH_SPLIT(dev))
- ironlake_check_encoder_dotclock(pipe_config, dotclock);
-
- pipe_config->base.adjusted_mode.crtc_clock = dotclock;
+ pipe_config->base.adjusted_mode.crtc_clock = pipe_config->port_clock;
}
static void hsw_crt_get_config(struct intel_encoder *encoder,
struct intel_crtc_state *pipe_config)
{
+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
+
intel_ddi_get_config(encoder, pipe_config);
pipe_config->base.adjusted_mode.flags &= ~(DRM_MODE_FLAG_PHSYNC |
@@ -143,6 +137,8 @@
DRM_MODE_FLAG_PVSYNC |
DRM_MODE_FLAG_NVSYNC);
pipe_config->base.adjusted_mode.flags |= intel_crt_get_flags(encoder);
+
+ pipe_config->base.adjusted_mode.crtc_clock = lpt_get_iclkip(dev_priv);
}
/* Note: The caller is required to filter out dpms modes not supported by the
@@ -222,18 +218,26 @@
{
struct drm_device *dev = connector->dev;
int max_dotclk = to_i915(dev)->max_dotclk_freq;
+ int max_clock;
- int max_clock = 0;
if (mode->flags & DRM_MODE_FLAG_DBLSCAN)
return MODE_NO_DBLESCAN;
if (mode->clock < 25000)
return MODE_CLOCK_LOW;
- if (IS_GEN2(dev))
- max_clock = 350000;
- else
+ if (HAS_PCH_LPT(dev))
+ max_clock = 180000;
+ else if (IS_VALLEYVIEW(dev))
+ /*
+ * 270 MHz due to current DPLL limits,
+ * DAC limit supposedly 355 MHz.
+ */
+ max_clock = 270000;
+ else if (IS_GEN3(dev) || IS_GEN4(dev))
max_clock = 400000;
+ else
+ max_clock = 350000;
if (mode->clock > max_clock)
return MODE_CLOCK_HIGH;
@@ -267,15 +271,9 @@
}
/* FDI must always be 2.7 GHz */
- if (HAS_DDI(dev)) {
- pipe_config->ddi_pll_sel = PORT_CLK_SEL_SPLL;
+ if (HAS_DDI(dev))
pipe_config->port_clock = 135000 * 2;
- pipe_config->dpll_hw_state.wrpll = 0;
- pipe_config->dpll_hw_state.spll =
- SPLL_PLL_ENABLE | SPLL_PLL_FREQ_1350MHz | SPLL_PLL_SSC;
- }
-
return true;
}
@@ -658,6 +656,8 @@
else if (INTEL_INFO(dev)->gen < 4)
status = intel_crt_load_detect(crt,
to_intel_crtc(connector->state->crtc)->pipe);
+ else if (i915.load_detect_test)
+ status = connector_status_disconnected;
else
status = connector_status_unknown;
intel_release_load_detect_pipe(connector, &tmp, &ctx);
diff --git a/drivers/gpu/drm/i915/intel_csr.c b/drivers/gpu/drm/i915/intel_csr.c
index 902054e..a34c23e 100644
--- a/drivers/gpu/drm/i915/intel_csr.c
+++ b/drivers/gpu/drm/i915/intel_csr.c
@@ -50,6 +50,7 @@
MODULE_FIRMWARE(I915_CSR_BXT);
#define SKL_CSR_VERSION_REQUIRED CSR_VERSION(1, 23)
+#define BXT_CSR_VERSION_REQUIRED CSR_VERSION(1, 7)
#define CSR_MAX_FW_SIZE 0x2FFF
#define CSR_DEFAULT_FW_OFFSET 0xFFFFFFFF
@@ -188,28 +189,49 @@
{'B', '0'}, {'B', '1'}, {'B', '2'}
};
-static const struct stepping_info *intel_get_stepping_info(struct drm_device *dev)
+static const struct stepping_info no_stepping_info = { '*', '*' };
+
+static const struct stepping_info *
+intel_get_stepping_info(struct drm_i915_private *dev_priv)
{
const struct stepping_info *si;
unsigned int size;
- if (IS_KABYLAKE(dev)) {
+ if (IS_KABYLAKE(dev_priv)) {
size = ARRAY_SIZE(kbl_stepping_info);
si = kbl_stepping_info;
- } else if (IS_SKYLAKE(dev)) {
+ } else if (IS_SKYLAKE(dev_priv)) {
size = ARRAY_SIZE(skl_stepping_info);
si = skl_stepping_info;
- } else if (IS_BROXTON(dev)) {
+ } else if (IS_BROXTON(dev_priv)) {
size = ARRAY_SIZE(bxt_stepping_info);
si = bxt_stepping_info;
} else {
- return NULL;
+ size = 0;
}
- if (INTEL_REVID(dev) < size)
- return si + INTEL_REVID(dev);
+ if (INTEL_REVID(dev_priv) < size)
+ return si + INTEL_REVID(dev_priv);
- return NULL;
+ return &no_stepping_info;
+}
+
+static void gen9_set_dc_state_debugmask(struct drm_i915_private *dev_priv)
+{
+ uint32_t val, mask;
+
+ mask = DC_STATE_DEBUG_MASK_MEMORY_UP;
+
+ if (IS_BROXTON(dev_priv))
+ mask |= DC_STATE_DEBUG_MASK_CORES;
+
+ /* The below bit doesn't need to be cleared ever afterwards */
+ val = I915_READ(DC_STATE_DEBUG);
+ if ((val & mask) != mask) {
+ val |= mask;
+ I915_WRITE(DC_STATE_DEBUG, val);
+ POSTING_READ(DC_STATE_DEBUG);
+ }
}
/**
@@ -220,19 +242,19 @@
* Everytime display comes back from low power state this function is called to
* copy the firmware from internal memory to registers.
*/
-bool intel_csr_load_program(struct drm_i915_private *dev_priv)
+void intel_csr_load_program(struct drm_i915_private *dev_priv)
{
u32 *payload = dev_priv->csr.dmc_payload;
uint32_t i, fw_size;
if (!IS_GEN9(dev_priv)) {
DRM_ERROR("No CSR support available for this platform\n");
- return false;
+ return;
}
if (!dev_priv->csr.dmc_payload) {
DRM_ERROR("Tried to program CSR with empty payload\n");
- return false;
+ return;
}
fw_size = dev_priv->csr.dmc_fw_size;
@@ -246,34 +268,25 @@
dev_priv->csr.dc_state = 0;
- return true;
+ gen9_set_dc_state_debugmask(dev_priv);
}
static uint32_t *parse_csr_fw(struct drm_i915_private *dev_priv,
const struct firmware *fw)
{
- struct drm_device *dev = dev_priv->dev;
struct intel_css_header *css_header;
struct intel_package_header *package_header;
struct intel_dmc_header *dmc_header;
struct intel_csr *csr = &dev_priv->csr;
- const struct stepping_info *stepping_info = intel_get_stepping_info(dev);
- char stepping, substepping;
+ const struct stepping_info *si = intel_get_stepping_info(dev_priv);
uint32_t dmc_offset = CSR_DEFAULT_FW_OFFSET, readcount = 0, nbytes;
uint32_t i;
uint32_t *dmc_payload;
+ uint32_t required_min_version;
if (!fw)
return NULL;
- if (!stepping_info) {
- DRM_ERROR("Unknown stepping info, firmware loading failed\n");
- return NULL;
- }
-
- stepping = stepping_info->stepping;
- substepping = stepping_info->substepping;
-
/* Extract CSS Header information*/
css_header = (struct intel_css_header *)fw->data;
if (sizeof(struct intel_css_header) !=
@@ -285,15 +298,23 @@
csr->version = css_header->version;
- if ((IS_SKYLAKE(dev) || IS_KABYLAKE(dev)) &&
- csr->version < SKL_CSR_VERSION_REQUIRED) {
- DRM_INFO("Refusing to load old Skylake DMC firmware v%u.%u,"
+ if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) {
+ required_min_version = SKL_CSR_VERSION_REQUIRED;
+ } else if (IS_BROXTON(dev_priv)) {
+ required_min_version = BXT_CSR_VERSION_REQUIRED;
+ } else {
+ MISSING_CASE(INTEL_REVID(dev_priv));
+ required_min_version = 0;
+ }
+
+ if (csr->version < required_min_version) {
+ DRM_INFO("Refusing to load old DMC firmware v%u.%u,"
" please upgrade to v%u.%u or later"
" [" FIRMWARE_URL "].\n",
CSR_VERSION_MAJOR(csr->version),
CSR_VERSION_MINOR(csr->version),
- CSR_VERSION_MAJOR(SKL_CSR_VERSION_REQUIRED),
- CSR_VERSION_MINOR(SKL_CSR_VERSION_REQUIRED));
+ CSR_VERSION_MAJOR(required_min_version),
+ CSR_VERSION_MINOR(required_min_version));
return NULL;
}
@@ -313,11 +334,11 @@
/* Search for dmc_offset to find firware binary. */
for (i = 0; i < package_header->num_entries; i++) {
if (package_header->fw_info[i].substepping == '*' &&
- stepping == package_header->fw_info[i].stepping) {
+ si->stepping == package_header->fw_info[i].stepping) {
dmc_offset = package_header->fw_info[i].offset;
break;
- } else if (stepping == package_header->fw_info[i].stepping &&
- substepping == package_header->fw_info[i].substepping) {
+ } else if (si->stepping == package_header->fw_info[i].stepping &&
+ si->substepping == package_header->fw_info[i].substepping) {
dmc_offset = package_header->fw_info[i].offset;
break;
} else if (package_header->fw_info[i].stepping == '*' &&
@@ -325,7 +346,8 @@
dmc_offset = package_header->fw_info[i].offset;
}
if (dmc_offset == CSR_DEFAULT_FW_OFFSET) {
- DRM_ERROR("Firmware not supported for %c stepping\n", stepping);
+ DRM_ERROR("Firmware not supported for %c stepping\n",
+ si->stepping);
return NULL;
}
readcount += dmc_offset;
@@ -371,9 +393,7 @@
return NULL;
}
- memcpy(dmc_payload, &fw->data[readcount], nbytes);
-
- return dmc_payload;
+ return memcpy(dmc_payload, &fw->data[readcount], nbytes);
}
static void csr_load_work_fn(struct work_struct *work)
@@ -388,18 +408,12 @@
ret = request_firmware(&fw, dev_priv->csr.fw_path,
&dev_priv->dev->pdev->dev);
- if (!fw)
- goto out;
+ if (fw)
+ dev_priv->csr.dmc_payload = parse_csr_fw(dev_priv, fw);
- dev_priv->csr.dmc_payload = parse_csr_fw(dev_priv, fw);
- if (!dev_priv->csr.dmc_payload)
- goto out;
-
- /* load csr program during system boot, as needed for DC states */
- intel_csr_load_program(dev_priv);
-
-out:
if (dev_priv->csr.dmc_payload) {
+ intel_csr_load_program(dev_priv);
+
intel_display_power_put(dev_priv, POWER_DOMAIN_INIT);
DRM_INFO("Finished loading %s (v%u.%u)\n",
@@ -453,10 +467,50 @@
}
/**
+ * intel_csr_ucode_suspend() - prepare CSR firmware before system suspend
+ * @dev_priv: i915 drm device
+ *
+ * Prepare the DMC firmware before entering system suspend. This includes
+ * flushing pending work items and releasing any resources acquired during
+ * init.
+ */
+void intel_csr_ucode_suspend(struct drm_i915_private *dev_priv)
+{
+ if (!HAS_CSR(dev_priv))
+ return;
+
+ flush_work(&dev_priv->csr.work);
+
+ /* Drop the reference held in case DMC isn't loaded. */
+ if (!dev_priv->csr.dmc_payload)
+ intel_display_power_put(dev_priv, POWER_DOMAIN_INIT);
+}
+
+/**
+ * intel_csr_ucode_resume() - init CSR firmware during system resume
+ * @dev_priv: i915 drm device
+ *
+ * Reinitialize the DMC firmware during system resume, reacquiring any
+ * resources released in intel_csr_ucode_suspend().
+ */
+void intel_csr_ucode_resume(struct drm_i915_private *dev_priv)
+{
+ if (!HAS_CSR(dev_priv))
+ return;
+
+ /*
+ * Reacquire the reference to keep RPM disabled in case DMC isn't
+ * loaded.
+ */
+ if (!dev_priv->csr.dmc_payload)
+ intel_display_power_get(dev_priv, POWER_DOMAIN_INIT);
+}
+
+/**
* intel_csr_ucode_fini() - unload the CSR firmware.
* @dev_priv: i915 drm device.
*
- * Firmmware unloading includes freeing the internal momory and reset the
+ * Firmmware unloading includes freeing the internal memory and reset the
* firmware loading status.
*/
void intel_csr_ucode_fini(struct drm_i915_private *dev_priv)
@@ -464,7 +518,7 @@
if (!HAS_CSR(dev_priv))
return;
- flush_work(&dev_priv->csr.work);
+ intel_csr_ucode_suspend(dev_priv);
kfree(dev_priv->csr.dmc_payload);
}
diff --git a/drivers/gpu/drm/i915/intel_ddi.c b/drivers/gpu/drm/i915/intel_ddi.c
index 96ffcc5..3fac046 100644
--- a/drivers/gpu/drm/i915/intel_ddi.c
+++ b/drivers/gpu/drm/i915/intel_ddi.c
@@ -315,6 +315,9 @@
*dig_port = enc_to_mst(encoder)->primary;
*port = (*dig_port)->port;
break;
+ default:
+ WARN(1, "Invalid DDI encoder type %d\n", intel_encoder->type);
+ /* fallthrough and treat as unknown */
case INTEL_OUTPUT_DISPLAYPORT:
case INTEL_OUTPUT_EDP:
case INTEL_OUTPUT_HDMI:
@@ -326,9 +329,6 @@
*dig_port = NULL;
*port = PORT_E;
break;
- default:
- WARN(1, "Invalid DDI encoder type %d\n", intel_encoder->type);
- break;
}
}
@@ -360,7 +360,7 @@
static const struct ddi_buf_trans *
skl_get_buf_trans_edp(struct drm_i915_private *dev_priv, int *n_entries)
{
- if (dev_priv->edp_low_vswing) {
+ if (dev_priv->vbt.edp.low_vswing) {
if (IS_SKL_ULX(dev_priv) || IS_KBL_ULX(dev_priv)) {
*n_entries = ARRAY_SIZE(skl_y_ddi_translations_edp);
return skl_y_ddi_translations_edp;
@@ -444,7 +444,7 @@
ddi_translations_fdi = bdw_ddi_translations_fdi;
ddi_translations_dp = bdw_ddi_translations_dp;
- if (dev_priv->edp_low_vswing) {
+ if (dev_priv->vbt.edp.low_vswing) {
ddi_translations_edp = bdw_ddi_translations_edp;
n_edp_entries = ARRAY_SIZE(bdw_ddi_translations_edp);
} else {
@@ -637,6 +637,10 @@
break;
}
+ rx_ctl_val &= ~FDI_RX_ENABLE;
+ I915_WRITE(FDI_RX_CTL(PIPE_A), rx_ctl_val);
+ POSTING_READ(FDI_RX_CTL(PIPE_A));
+
temp = I915_READ(DDI_BUF_CTL(PORT_E));
temp &= ~DDI_BUF_CTL_ENABLE;
I915_WRITE(DDI_BUF_CTL(PORT_E), temp);
@@ -651,10 +655,6 @@
intel_wait_ddi_buf_idle(dev_priv, PORT_E);
- rx_ctl_val &= ~FDI_RX_ENABLE;
- I915_WRITE(FDI_RX_CTL(PIPE_A), rx_ctl_val);
- POSTING_READ(FDI_RX_CTL(PIPE_A));
-
/* Reset FDI_RX_MISC pwrdn lanes */
temp = I915_READ(FDI_RX_MISC(PIPE_A));
temp &= ~(FDI_RX_PWRDN_LANE1_MASK | FDI_RX_PWRDN_LANE0_MASK);
@@ -732,160 +732,6 @@
}
#define LC_FREQ 2700
-#define LC_FREQ_2K U64_C(LC_FREQ * 2000)
-
-#define P_MIN 2
-#define P_MAX 64
-#define P_INC 2
-
-/* Constraints for PLL good behavior */
-#define REF_MIN 48
-#define REF_MAX 400
-#define VCO_MIN 2400
-#define VCO_MAX 4800
-
-#define abs_diff(a, b) ({ \
- typeof(a) __a = (a); \
- typeof(b) __b = (b); \
- (void) (&__a == &__b); \
- __a > __b ? (__a - __b) : (__b - __a); })
-
-struct hsw_wrpll_rnp {
- unsigned p, n2, r2;
-};
-
-static unsigned hsw_wrpll_get_budget_for_freq(int clock)
-{
- unsigned budget;
-
- switch (clock) {
- case 25175000:
- case 25200000:
- case 27000000:
- case 27027000:
- case 37762500:
- case 37800000:
- case 40500000:
- case 40541000:
- case 54000000:
- case 54054000:
- case 59341000:
- case 59400000:
- case 72000000:
- case 74176000:
- case 74250000:
- case 81000000:
- case 81081000:
- case 89012000:
- case 89100000:
- case 108000000:
- case 108108000:
- case 111264000:
- case 111375000:
- case 148352000:
- case 148500000:
- case 162000000:
- case 162162000:
- case 222525000:
- case 222750000:
- case 296703000:
- case 297000000:
- budget = 0;
- break;
- case 233500000:
- case 245250000:
- case 247750000:
- case 253250000:
- case 298000000:
- budget = 1500;
- break;
- case 169128000:
- case 169500000:
- case 179500000:
- case 202000000:
- budget = 2000;
- break;
- case 256250000:
- case 262500000:
- case 270000000:
- case 272500000:
- case 273750000:
- case 280750000:
- case 281250000:
- case 286000000:
- case 291750000:
- budget = 4000;
- break;
- case 267250000:
- case 268500000:
- budget = 5000;
- break;
- default:
- budget = 1000;
- break;
- }
-
- return budget;
-}
-
-static void hsw_wrpll_update_rnp(uint64_t freq2k, unsigned budget,
- unsigned r2, unsigned n2, unsigned p,
- struct hsw_wrpll_rnp *best)
-{
- uint64_t a, b, c, d, diff, diff_best;
-
- /* No best (r,n,p) yet */
- if (best->p == 0) {
- best->p = p;
- best->n2 = n2;
- best->r2 = r2;
- return;
- }
-
- /*
- * Output clock is (LC_FREQ_2K / 2000) * N / (P * R), which compares to
- * freq2k.
- *
- * delta = 1e6 *
- * abs(freq2k - (LC_FREQ_2K * n2/(p * r2))) /
- * freq2k;
- *
- * and we would like delta <= budget.
- *
- * If the discrepancy is above the PPM-based budget, always prefer to
- * improve upon the previous solution. However, if you're within the
- * budget, try to maximize Ref * VCO, that is N / (P * R^2).
- */
- a = freq2k * budget * p * r2;
- b = freq2k * budget * best->p * best->r2;
- diff = abs_diff(freq2k * p * r2, LC_FREQ_2K * n2);
- diff_best = abs_diff(freq2k * best->p * best->r2,
- LC_FREQ_2K * best->n2);
- c = 1000000 * diff;
- d = 1000000 * diff_best;
-
- if (a < c && b < d) {
- /* If both are above the budget, pick the closer */
- if (best->p * best->r2 * diff < p * r2 * diff_best) {
- best->p = p;
- best->n2 = n2;
- best->r2 = r2;
- }
- } else if (a >= c && b < d) {
- /* If A is below the threshold but B is above it? Update. */
- best->p = p;
- best->n2 = n2;
- best->r2 = r2;
- } else if (a >= c && b >= d) {
- /* Both are below the limit, so pick the higher n2/(r2*r2) */
- if (n2 * best->r2 * best->r2 > best->n2 * r2 * r2) {
- best->p = p;
- best->n2 = n2;
- best->r2 = r2;
- }
- }
- /* Otherwise a < c && b >= d, do nothing */
-}
static int hsw_ddi_calc_wrpll_link(struct drm_i915_private *dev_priv,
i915_reg_t reg)
@@ -1147,363 +993,20 @@
bxt_ddi_clock_get(encoder, pipe_config);
}
-static void
-hsw_ddi_calculate_wrpll(int clock /* in Hz */,
- unsigned *r2_out, unsigned *n2_out, unsigned *p_out)
-{
- uint64_t freq2k;
- unsigned p, n2, r2;
- struct hsw_wrpll_rnp best = { 0, 0, 0 };
- unsigned budget;
-
- freq2k = clock / 100;
-
- budget = hsw_wrpll_get_budget_for_freq(clock);
-
- /* Special case handling for 540 pixel clock: bypass WR PLL entirely
- * and directly pass the LC PLL to it. */
- if (freq2k == 5400000) {
- *n2_out = 2;
- *p_out = 1;
- *r2_out = 2;
- return;
- }
-
- /*
- * Ref = LC_FREQ / R, where Ref is the actual reference input seen by
- * the WR PLL.
- *
- * We want R so that REF_MIN <= Ref <= REF_MAX.
- * Injecting R2 = 2 * R gives:
- * REF_MAX * r2 > LC_FREQ * 2 and
- * REF_MIN * r2 < LC_FREQ * 2
- *
- * Which means the desired boundaries for r2 are:
- * LC_FREQ * 2 / REF_MAX < r2 < LC_FREQ * 2 / REF_MIN
- *
- */
- for (r2 = LC_FREQ * 2 / REF_MAX + 1;
- r2 <= LC_FREQ * 2 / REF_MIN;
- r2++) {
-
- /*
- * VCO = N * Ref, that is: VCO = N * LC_FREQ / R
- *
- * Once again we want VCO_MIN <= VCO <= VCO_MAX.
- * Injecting R2 = 2 * R and N2 = 2 * N, we get:
- * VCO_MAX * r2 > n2 * LC_FREQ and
- * VCO_MIN * r2 < n2 * LC_FREQ)
- *
- * Which means the desired boundaries for n2 are:
- * VCO_MIN * r2 / LC_FREQ < n2 < VCO_MAX * r2 / LC_FREQ
- */
- for (n2 = VCO_MIN * r2 / LC_FREQ + 1;
- n2 <= VCO_MAX * r2 / LC_FREQ;
- n2++) {
-
- for (p = P_MIN; p <= P_MAX; p += P_INC)
- hsw_wrpll_update_rnp(freq2k, budget,
- r2, n2, p, &best);
- }
- }
-
- *n2_out = best.n2;
- *p_out = best.p;
- *r2_out = best.r2;
-}
-
static bool
hsw_ddi_pll_select(struct intel_crtc *intel_crtc,
struct intel_crtc_state *crtc_state,
struct intel_encoder *intel_encoder)
{
- int clock = crtc_state->port_clock;
+ struct intel_shared_dpll *pll;
- if (intel_encoder->type == INTEL_OUTPUT_HDMI) {
- struct intel_shared_dpll *pll;
- uint32_t val;
- unsigned p, n2, r2;
+ pll = intel_get_shared_dpll(intel_crtc, crtc_state,
+ intel_encoder);
+ if (!pll)
+ DRM_DEBUG_DRIVER("failed to find PLL for pipe %c\n",
+ pipe_name(intel_crtc->pipe));
- hsw_ddi_calculate_wrpll(clock * 1000, &r2, &n2, &p);
-
- val = WRPLL_PLL_ENABLE | WRPLL_PLL_LCPLL |
- WRPLL_DIVIDER_REFERENCE(r2) | WRPLL_DIVIDER_FEEDBACK(n2) |
- WRPLL_DIVIDER_POST(p);
-
- memset(&crtc_state->dpll_hw_state, 0,
- sizeof(crtc_state->dpll_hw_state));
-
- crtc_state->dpll_hw_state.wrpll = val;
-
- pll = intel_get_shared_dpll(intel_crtc, crtc_state);
- if (pll == NULL) {
- DRM_DEBUG_DRIVER("failed to find PLL for pipe %c\n",
- pipe_name(intel_crtc->pipe));
- return false;
- }
-
- crtc_state->ddi_pll_sel = PORT_CLK_SEL_WRPLL(pll->id);
- } else if (crtc_state->ddi_pll_sel == PORT_CLK_SEL_SPLL) {
- struct drm_atomic_state *state = crtc_state->base.state;
- struct intel_shared_dpll_config *spll =
- &intel_atomic_get_shared_dpll_state(state)[DPLL_ID_SPLL];
-
- if (spll->crtc_mask &&
- WARN_ON(spll->hw_state.spll != crtc_state->dpll_hw_state.spll))
- return false;
-
- crtc_state->shared_dpll = DPLL_ID_SPLL;
- spll->hw_state.spll = crtc_state->dpll_hw_state.spll;
- spll->crtc_mask |= 1 << intel_crtc->pipe;
- }
-
- return true;
-}
-
-struct skl_wrpll_context {
- uint64_t min_deviation; /* current minimal deviation */
- uint64_t central_freq; /* chosen central freq */
- uint64_t dco_freq; /* chosen dco freq */
- unsigned int p; /* chosen divider */
-};
-
-static void skl_wrpll_context_init(struct skl_wrpll_context *ctx)
-{
- memset(ctx, 0, sizeof(*ctx));
-
- ctx->min_deviation = U64_MAX;
-}
-
-/* DCO freq must be within +1%/-6% of the DCO central freq */
-#define SKL_DCO_MAX_PDEVIATION 100
-#define SKL_DCO_MAX_NDEVIATION 600
-
-static void skl_wrpll_try_divider(struct skl_wrpll_context *ctx,
- uint64_t central_freq,
- uint64_t dco_freq,
- unsigned int divider)
-{
- uint64_t deviation;
-
- deviation = div64_u64(10000 * abs_diff(dco_freq, central_freq),
- central_freq);
-
- /* positive deviation */
- if (dco_freq >= central_freq) {
- if (deviation < SKL_DCO_MAX_PDEVIATION &&
- deviation < ctx->min_deviation) {
- ctx->min_deviation = deviation;
- ctx->central_freq = central_freq;
- ctx->dco_freq = dco_freq;
- ctx->p = divider;
- }
- /* negative deviation */
- } else if (deviation < SKL_DCO_MAX_NDEVIATION &&
- deviation < ctx->min_deviation) {
- ctx->min_deviation = deviation;
- ctx->central_freq = central_freq;
- ctx->dco_freq = dco_freq;
- ctx->p = divider;
- }
-}
-
-static void skl_wrpll_get_multipliers(unsigned int p,
- unsigned int *p0 /* out */,
- unsigned int *p1 /* out */,
- unsigned int *p2 /* out */)
-{
- /* even dividers */
- if (p % 2 == 0) {
- unsigned int half = p / 2;
-
- if (half == 1 || half == 2 || half == 3 || half == 5) {
- *p0 = 2;
- *p1 = 1;
- *p2 = half;
- } else if (half % 2 == 0) {
- *p0 = 2;
- *p1 = half / 2;
- *p2 = 2;
- } else if (half % 3 == 0) {
- *p0 = 3;
- *p1 = half / 3;
- *p2 = 2;
- } else if (half % 7 == 0) {
- *p0 = 7;
- *p1 = half / 7;
- *p2 = 2;
- }
- } else if (p == 3 || p == 9) { /* 3, 5, 7, 9, 15, 21, 35 */
- *p0 = 3;
- *p1 = 1;
- *p2 = p / 3;
- } else if (p == 5 || p == 7) {
- *p0 = p;
- *p1 = 1;
- *p2 = 1;
- } else if (p == 15) {
- *p0 = 3;
- *p1 = 1;
- *p2 = 5;
- } else if (p == 21) {
- *p0 = 7;
- *p1 = 1;
- *p2 = 3;
- } else if (p == 35) {
- *p0 = 7;
- *p1 = 1;
- *p2 = 5;
- }
-}
-
-struct skl_wrpll_params {
- uint32_t dco_fraction;
- uint32_t dco_integer;
- uint32_t qdiv_ratio;
- uint32_t qdiv_mode;
- uint32_t kdiv;
- uint32_t pdiv;
- uint32_t central_freq;
-};
-
-static void skl_wrpll_params_populate(struct skl_wrpll_params *params,
- uint64_t afe_clock,
- uint64_t central_freq,
- uint32_t p0, uint32_t p1, uint32_t p2)
-{
- uint64_t dco_freq;
-
- switch (central_freq) {
- case 9600000000ULL:
- params->central_freq = 0;
- break;
- case 9000000000ULL:
- params->central_freq = 1;
- break;
- case 8400000000ULL:
- params->central_freq = 3;
- }
-
- switch (p0) {
- case 1:
- params->pdiv = 0;
- break;
- case 2:
- params->pdiv = 1;
- break;
- case 3:
- params->pdiv = 2;
- break;
- case 7:
- params->pdiv = 4;
- break;
- default:
- WARN(1, "Incorrect PDiv\n");
- }
-
- switch (p2) {
- case 5:
- params->kdiv = 0;
- break;
- case 2:
- params->kdiv = 1;
- break;
- case 3:
- params->kdiv = 2;
- break;
- case 1:
- params->kdiv = 3;
- break;
- default:
- WARN(1, "Incorrect KDiv\n");
- }
-
- params->qdiv_ratio = p1;
- params->qdiv_mode = (params->qdiv_ratio == 1) ? 0 : 1;
-
- dco_freq = p0 * p1 * p2 * afe_clock;
-
- /*
- * Intermediate values are in Hz.
- * Divide by MHz to match bsepc
- */
- params->dco_integer = div_u64(dco_freq, 24 * MHz(1));
- params->dco_fraction =
- div_u64((div_u64(dco_freq, 24) -
- params->dco_integer * MHz(1)) * 0x8000, MHz(1));
-}
-
-static bool
-skl_ddi_calculate_wrpll(int clock /* in Hz */,
- struct skl_wrpll_params *wrpll_params)
-{
- uint64_t afe_clock = clock * 5; /* AFE Clock is 5x Pixel clock */
- uint64_t dco_central_freq[3] = {8400000000ULL,
- 9000000000ULL,
- 9600000000ULL};
- static const int even_dividers[] = { 4, 6, 8, 10, 12, 14, 16, 18, 20,
- 24, 28, 30, 32, 36, 40, 42, 44,
- 48, 52, 54, 56, 60, 64, 66, 68,
- 70, 72, 76, 78, 80, 84, 88, 90,
- 92, 96, 98 };
- static const int odd_dividers[] = { 3, 5, 7, 9, 15, 21, 35 };
- static const struct {
- const int *list;
- int n_dividers;
- } dividers[] = {
- { even_dividers, ARRAY_SIZE(even_dividers) },
- { odd_dividers, ARRAY_SIZE(odd_dividers) },
- };
- struct skl_wrpll_context ctx;
- unsigned int dco, d, i;
- unsigned int p0, p1, p2;
-
- skl_wrpll_context_init(&ctx);
-
- for (d = 0; d < ARRAY_SIZE(dividers); d++) {
- for (dco = 0; dco < ARRAY_SIZE(dco_central_freq); dco++) {
- for (i = 0; i < dividers[d].n_dividers; i++) {
- unsigned int p = dividers[d].list[i];
- uint64_t dco_freq = p * afe_clock;
-
- skl_wrpll_try_divider(&ctx,
- dco_central_freq[dco],
- dco_freq,
- p);
- /*
- * Skip the remaining dividers if we're sure to
- * have found the definitive divider, we can't
- * improve a 0 deviation.
- */
- if (ctx.min_deviation == 0)
- goto skip_remaining_dividers;
- }
- }
-
-skip_remaining_dividers:
- /*
- * If a solution is found with an even divider, prefer
- * this one.
- */
- if (d == 0 && ctx.p)
- break;
- }
-
- if (!ctx.p) {
- DRM_DEBUG_DRIVER("No valid divider found for %dHz\n", clock);
- return false;
- }
-
- /*
- * gcc incorrectly analyses that these can be used without being
- * initialized. To be fair, it's hard to guess.
- */
- p0 = p1 = p2 = 0;
- skl_wrpll_get_multipliers(ctx.p, &p0, &p1, &p2);
- skl_wrpll_params_populate(wrpll_params, afe_clock, ctx.central_freq,
- p0, p1, p2);
-
- return true;
+ return pll;
}
static bool
@@ -1512,218 +1015,23 @@
struct intel_encoder *intel_encoder)
{
struct intel_shared_dpll *pll;
- uint32_t ctrl1, cfgcr1, cfgcr2;
- int clock = crtc_state->port_clock;
- /*
- * See comment in intel_dpll_hw_state to understand why we always use 0
- * as the DPLL id in this function.
- */
-
- ctrl1 = DPLL_CTRL1_OVERRIDE(0);
-
- if (intel_encoder->type == INTEL_OUTPUT_HDMI) {
- struct skl_wrpll_params wrpll_params = { 0, };
-
- ctrl1 |= DPLL_CTRL1_HDMI_MODE(0);
-
- if (!skl_ddi_calculate_wrpll(clock * 1000, &wrpll_params))
- return false;
-
- cfgcr1 = DPLL_CFGCR1_FREQ_ENABLE |
- DPLL_CFGCR1_DCO_FRACTION(wrpll_params.dco_fraction) |
- wrpll_params.dco_integer;
-
- cfgcr2 = DPLL_CFGCR2_QDIV_RATIO(wrpll_params.qdiv_ratio) |
- DPLL_CFGCR2_QDIV_MODE(wrpll_params.qdiv_mode) |
- DPLL_CFGCR2_KDIV(wrpll_params.kdiv) |
- DPLL_CFGCR2_PDIV(wrpll_params.pdiv) |
- wrpll_params.central_freq;
- } else if (intel_encoder->type == INTEL_OUTPUT_DISPLAYPORT ||
- intel_encoder->type == INTEL_OUTPUT_DP_MST) {
- switch (crtc_state->port_clock / 2) {
- case 81000:
- ctrl1 |= DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_810, 0);
- break;
- case 135000:
- ctrl1 |= DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_1350, 0);
- break;
- case 270000:
- ctrl1 |= DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_2700, 0);
- break;
- }
-
- cfgcr1 = cfgcr2 = 0;
- } else if (intel_encoder->type == INTEL_OUTPUT_EDP) {
- return true;
- } else
- return false;
-
- memset(&crtc_state->dpll_hw_state, 0,
- sizeof(crtc_state->dpll_hw_state));
-
- crtc_state->dpll_hw_state.ctrl1 = ctrl1;
- crtc_state->dpll_hw_state.cfgcr1 = cfgcr1;
- crtc_state->dpll_hw_state.cfgcr2 = cfgcr2;
-
- pll = intel_get_shared_dpll(intel_crtc, crtc_state);
+ pll = intel_get_shared_dpll(intel_crtc, crtc_state, intel_encoder);
if (pll == NULL) {
DRM_DEBUG_DRIVER("failed to find PLL for pipe %c\n",
pipe_name(intel_crtc->pipe));
return false;
}
- /* shared DPLL id 0 is DPLL 1 */
- crtc_state->ddi_pll_sel = pll->id + 1;
-
return true;
}
-/* bxt clock parameters */
-struct bxt_clk_div {
- int clock;
- uint32_t p1;
- uint32_t p2;
- uint32_t m2_int;
- uint32_t m2_frac;
- bool m2_frac_en;
- uint32_t n;
-};
-
-/* pre-calculated values for DP linkrates */
-static const struct bxt_clk_div bxt_dp_clk_val[] = {
- {162000, 4, 2, 32, 1677722, 1, 1},
- {270000, 4, 1, 27, 0, 0, 1},
- {540000, 2, 1, 27, 0, 0, 1},
- {216000, 3, 2, 32, 1677722, 1, 1},
- {243000, 4, 1, 24, 1258291, 1, 1},
- {324000, 4, 1, 32, 1677722, 1, 1},
- {432000, 3, 1, 32, 1677722, 1, 1}
-};
-
static bool
bxt_ddi_pll_select(struct intel_crtc *intel_crtc,
struct intel_crtc_state *crtc_state,
struct intel_encoder *intel_encoder)
{
- struct intel_shared_dpll *pll;
- struct bxt_clk_div clk_div = {0};
- int vco = 0;
- uint32_t prop_coef, int_coef, gain_ctl, targ_cnt;
- uint32_t lanestagger;
- int clock = crtc_state->port_clock;
-
- if (intel_encoder->type == INTEL_OUTPUT_HDMI) {
- intel_clock_t best_clock;
-
- /* Calculate HDMI div */
- /*
- * FIXME: tie the following calculation into
- * i9xx_crtc_compute_clock
- */
- if (!bxt_find_best_dpll(crtc_state, clock, &best_clock)) {
- DRM_DEBUG_DRIVER("no PLL dividers found for clock %d pipe %c\n",
- clock, pipe_name(intel_crtc->pipe));
- return false;
- }
-
- clk_div.p1 = best_clock.p1;
- clk_div.p2 = best_clock.p2;
- WARN_ON(best_clock.m1 != 2);
- clk_div.n = best_clock.n;
- clk_div.m2_int = best_clock.m2 >> 22;
- clk_div.m2_frac = best_clock.m2 & ((1 << 22) - 1);
- clk_div.m2_frac_en = clk_div.m2_frac != 0;
-
- vco = best_clock.vco;
- } else if (intel_encoder->type == INTEL_OUTPUT_DISPLAYPORT ||
- intel_encoder->type == INTEL_OUTPUT_EDP) {
- int i;
-
- clk_div = bxt_dp_clk_val[0];
- for (i = 0; i < ARRAY_SIZE(bxt_dp_clk_val); ++i) {
- if (bxt_dp_clk_val[i].clock == clock) {
- clk_div = bxt_dp_clk_val[i];
- break;
- }
- }
- vco = clock * 10 / 2 * clk_div.p1 * clk_div.p2;
- }
-
- if (vco >= 6200000 && vco <= 6700000) {
- prop_coef = 4;
- int_coef = 9;
- gain_ctl = 3;
- targ_cnt = 8;
- } else if ((vco > 5400000 && vco < 6200000) ||
- (vco >= 4800000 && vco < 5400000)) {
- prop_coef = 5;
- int_coef = 11;
- gain_ctl = 3;
- targ_cnt = 9;
- } else if (vco == 5400000) {
- prop_coef = 3;
- int_coef = 8;
- gain_ctl = 1;
- targ_cnt = 9;
- } else {
- DRM_ERROR("Invalid VCO\n");
- return false;
- }
-
- memset(&crtc_state->dpll_hw_state, 0,
- sizeof(crtc_state->dpll_hw_state));
-
- if (clock > 270000)
- lanestagger = 0x18;
- else if (clock > 135000)
- lanestagger = 0x0d;
- else if (clock > 67000)
- lanestagger = 0x07;
- else if (clock > 33000)
- lanestagger = 0x04;
- else
- lanestagger = 0x02;
-
- crtc_state->dpll_hw_state.ebb0 =
- PORT_PLL_P1(clk_div.p1) | PORT_PLL_P2(clk_div.p2);
- crtc_state->dpll_hw_state.pll0 = clk_div.m2_int;
- crtc_state->dpll_hw_state.pll1 = PORT_PLL_N(clk_div.n);
- crtc_state->dpll_hw_state.pll2 = clk_div.m2_frac;
-
- if (clk_div.m2_frac_en)
- crtc_state->dpll_hw_state.pll3 =
- PORT_PLL_M2_FRAC_ENABLE;
-
- crtc_state->dpll_hw_state.pll6 =
- prop_coef | PORT_PLL_INT_COEFF(int_coef);
- crtc_state->dpll_hw_state.pll6 |=
- PORT_PLL_GAIN_CTL(gain_ctl);
-
- crtc_state->dpll_hw_state.pll8 = targ_cnt;
-
- crtc_state->dpll_hw_state.pll9 = 5 << PORT_PLL_LOCK_THRESHOLD_SHIFT;
-
- crtc_state->dpll_hw_state.pll10 =
- PORT_PLL_DCO_AMP(PORT_PLL_DCO_AMP_DEFAULT)
- | PORT_PLL_DCO_AMP_OVR_EN_H;
-
- crtc_state->dpll_hw_state.ebb4 = PORT_PLL_10BIT_CLK_ENABLE;
-
- crtc_state->dpll_hw_state.pcsdw12 =
- LANESTAGGER_STRAP_OVRD | lanestagger;
-
- pll = intel_get_shared_dpll(intel_crtc, crtc_state);
- if (pll == NULL) {
- DRM_DEBUG_DRIVER("failed to find PLL for pipe %c\n",
- pipe_name(intel_crtc->pipe));
- return false;
- }
-
- /* shared DPLL id 0 is DPLL A */
- crtc_state->ddi_pll_sel = pll->id;
-
- return true;
+ return !!intel_get_shared_dpll(intel_crtc, crtc_state, intel_encoder);
}
/*
@@ -1761,6 +1069,8 @@
uint32_t temp;
if (type == INTEL_OUTPUT_DISPLAYPORT || type == INTEL_OUTPUT_EDP || type == INTEL_OUTPUT_DP_MST) {
+ WARN_ON(transcoder_is_dsi(cpu_transcoder));
+
temp = TRANS_MSA_SYNC_CLK;
switch (intel_crtc->config->pipe_bpp) {
case 18:
@@ -2129,7 +1439,7 @@
u32 n_entries, i;
uint32_t val;
- if (type == INTEL_OUTPUT_EDP && dev_priv->edp_low_vswing) {
+ if (type == INTEL_OUTPUT_EDP && dev_priv->vbt.edp.low_vswing) {
n_entries = ARRAY_SIZE(bxt_ddi_translations_edp);
ddi_translations = bxt_ddi_translations_edp;
} else if (type == INTEL_OUTPUT_DISPLAYPORT
@@ -2267,24 +1577,6 @@
uint32_t dpll = pipe_config->ddi_pll_sel;
uint32_t val;
- /*
- * DPLL0 is used for eDP and is the only "private" DPLL (as
- * opposed to shared) on SKL
- */
- if (encoder->type == INTEL_OUTPUT_EDP) {
- WARN_ON(dpll != SKL_DPLL0);
-
- val = I915_READ(DPLL_CTRL1);
-
- val &= ~(DPLL_CTRL1_HDMI_MODE(dpll) |
- DPLL_CTRL1_SSC(dpll) |
- DPLL_CTRL1_LINK_RATE_MASK(dpll));
- val |= pipe_config->dpll_hw_state.ctrl1 << (dpll * 6);
-
- I915_WRITE(DPLL_CTRL1, val);
- POSTING_READ(DPLL_CTRL1);
- }
-
/* DDI -> PLL mapping */
val = I915_READ(DPLL_CTRL2);
@@ -2438,251 +1730,101 @@
}
}
-static void hsw_ddi_wrpll_enable(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll)
+static bool broxton_phy_is_enabled(struct drm_i915_private *dev_priv,
+ enum dpio_phy phy)
{
- I915_WRITE(WRPLL_CTL(pll->id), pll->config.hw_state.wrpll);
- POSTING_READ(WRPLL_CTL(pll->id));
- udelay(20);
-}
-
-static void hsw_ddi_spll_enable(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll)
-{
- I915_WRITE(SPLL_CTL, pll->config.hw_state.spll);
- POSTING_READ(SPLL_CTL);
- udelay(20);
-}
-
-static void hsw_ddi_wrpll_disable(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll)
-{
- uint32_t val;
-
- val = I915_READ(WRPLL_CTL(pll->id));
- I915_WRITE(WRPLL_CTL(pll->id), val & ~WRPLL_PLL_ENABLE);
- POSTING_READ(WRPLL_CTL(pll->id));
-}
-
-static void hsw_ddi_spll_disable(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll)
-{
- uint32_t val;
-
- val = I915_READ(SPLL_CTL);
- I915_WRITE(SPLL_CTL, val & ~SPLL_PLL_ENABLE);
- POSTING_READ(SPLL_CTL);
-}
-
-static bool hsw_ddi_wrpll_get_hw_state(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll,
- struct intel_dpll_hw_state *hw_state)
-{
- uint32_t val;
-
- if (!intel_display_power_get_if_enabled(dev_priv, POWER_DOMAIN_PLLS))
+ if (!(I915_READ(BXT_P_CR_GT_DISP_PWRON) & GT_DISPLAY_POWER_ON(phy)))
return false;
- val = I915_READ(WRPLL_CTL(pll->id));
- hw_state->wrpll = val;
+ if ((I915_READ(BXT_PORT_CL1CM_DW0(phy)) &
+ (PHY_POWER_GOOD | PHY_RESERVED)) != PHY_POWER_GOOD) {
+ DRM_DEBUG_DRIVER("DDI PHY %d powered, but power hasn't settled\n",
+ phy);
- intel_display_power_put(dev_priv, POWER_DOMAIN_PLLS);
-
- return val & WRPLL_PLL_ENABLE;
-}
-
-static bool hsw_ddi_spll_get_hw_state(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll,
- struct intel_dpll_hw_state *hw_state)
-{
- uint32_t val;
-
- if (!intel_display_power_get_if_enabled(dev_priv, POWER_DOMAIN_PLLS))
return false;
-
- val = I915_READ(SPLL_CTL);
- hw_state->spll = val;
-
- intel_display_power_put(dev_priv, POWER_DOMAIN_PLLS);
-
- return val & SPLL_PLL_ENABLE;
-}
-
-
-static const char * const hsw_ddi_pll_names[] = {
- "WRPLL 1",
- "WRPLL 2",
- "SPLL"
-};
-
-static void hsw_shared_dplls_init(struct drm_i915_private *dev_priv)
-{
- int i;
-
- dev_priv->num_shared_dpll = 3;
-
- for (i = 0; i < 2; i++) {
- dev_priv->shared_dplls[i].id = i;
- dev_priv->shared_dplls[i].name = hsw_ddi_pll_names[i];
- dev_priv->shared_dplls[i].disable = hsw_ddi_wrpll_disable;
- dev_priv->shared_dplls[i].enable = hsw_ddi_wrpll_enable;
- dev_priv->shared_dplls[i].get_hw_state =
- hsw_ddi_wrpll_get_hw_state;
}
- /* SPLL is special, but needs to be initialized anyway.. */
- dev_priv->shared_dplls[i].id = i;
- dev_priv->shared_dplls[i].name = hsw_ddi_pll_names[i];
- dev_priv->shared_dplls[i].disable = hsw_ddi_spll_disable;
- dev_priv->shared_dplls[i].enable = hsw_ddi_spll_enable;
- dev_priv->shared_dplls[i].get_hw_state = hsw_ddi_spll_get_hw_state;
+ if (phy == DPIO_PHY1 &&
+ !(I915_READ(BXT_PORT_REF_DW3(DPIO_PHY1)) & GRC_DONE)) {
+ DRM_DEBUG_DRIVER("DDI PHY 1 powered, but GRC isn't done\n");
-}
-
-static const char * const skl_ddi_pll_names[] = {
- "DPLL 1",
- "DPLL 2",
- "DPLL 3",
-};
-
-struct skl_dpll_regs {
- i915_reg_t ctl, cfgcr1, cfgcr2;
-};
-
-/* this array is indexed by the *shared* pll id */
-static const struct skl_dpll_regs skl_dpll_regs[3] = {
- {
- /* DPLL 1 */
- .ctl = LCPLL2_CTL,
- .cfgcr1 = DPLL_CFGCR1(SKL_DPLL1),
- .cfgcr2 = DPLL_CFGCR2(SKL_DPLL1),
- },
- {
- /* DPLL 2 */
- .ctl = WRPLL_CTL(0),
- .cfgcr1 = DPLL_CFGCR1(SKL_DPLL2),
- .cfgcr2 = DPLL_CFGCR2(SKL_DPLL2),
- },
- {
- /* DPLL 3 */
- .ctl = WRPLL_CTL(1),
- .cfgcr1 = DPLL_CFGCR1(SKL_DPLL3),
- .cfgcr2 = DPLL_CFGCR2(SKL_DPLL3),
- },
-};
-
-static void skl_ddi_pll_enable(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll)
-{
- uint32_t val;
- unsigned int dpll;
- const struct skl_dpll_regs *regs = skl_dpll_regs;
-
- /* DPLL0 is not part of the shared DPLLs, so pll->id is 0 for DPLL1 */
- dpll = pll->id + 1;
-
- val = I915_READ(DPLL_CTRL1);
-
- val &= ~(DPLL_CTRL1_HDMI_MODE(dpll) | DPLL_CTRL1_SSC(dpll) |
- DPLL_CTRL1_LINK_RATE_MASK(dpll));
- val |= pll->config.hw_state.ctrl1 << (dpll * 6);
-
- I915_WRITE(DPLL_CTRL1, val);
- POSTING_READ(DPLL_CTRL1);
-
- I915_WRITE(regs[pll->id].cfgcr1, pll->config.hw_state.cfgcr1);
- I915_WRITE(regs[pll->id].cfgcr2, pll->config.hw_state.cfgcr2);
- POSTING_READ(regs[pll->id].cfgcr1);
- POSTING_READ(regs[pll->id].cfgcr2);
-
- /* the enable bit is always bit 31 */
- I915_WRITE(regs[pll->id].ctl,
- I915_READ(regs[pll->id].ctl) | LCPLL_PLL_ENABLE);
-
- if (wait_for(I915_READ(DPLL_STATUS) & DPLL_LOCK(dpll), 5))
- DRM_ERROR("DPLL %d not locked\n", dpll);
-}
-
-static void skl_ddi_pll_disable(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll)
-{
- const struct skl_dpll_regs *regs = skl_dpll_regs;
-
- /* the enable bit is always bit 31 */
- I915_WRITE(regs[pll->id].ctl,
- I915_READ(regs[pll->id].ctl) & ~LCPLL_PLL_ENABLE);
- POSTING_READ(regs[pll->id].ctl);
-}
-
-static bool skl_ddi_pll_get_hw_state(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll,
- struct intel_dpll_hw_state *hw_state)
-{
- uint32_t val;
- unsigned int dpll;
- const struct skl_dpll_regs *regs = skl_dpll_regs;
- bool ret;
-
- if (!intel_display_power_get_if_enabled(dev_priv, POWER_DOMAIN_PLLS))
return false;
-
- ret = false;
-
- /* DPLL0 is not part of the shared DPLLs, so pll->id is 0 for DPLL1 */
- dpll = pll->id + 1;
-
- val = I915_READ(regs[pll->id].ctl);
- if (!(val & LCPLL_PLL_ENABLE))
- goto out;
-
- val = I915_READ(DPLL_CTRL1);
- hw_state->ctrl1 = (val >> (dpll * 6)) & 0x3f;
-
- /* avoid reading back stale values if HDMI mode is not enabled */
- if (val & DPLL_CTRL1_HDMI_MODE(dpll)) {
- hw_state->cfgcr1 = I915_READ(regs[pll->id].cfgcr1);
- hw_state->cfgcr2 = I915_READ(regs[pll->id].cfgcr2);
}
- ret = true;
-out:
- intel_display_power_put(dev_priv, POWER_DOMAIN_PLLS);
+ if (!(I915_READ(BXT_PHY_CTL_FAMILY(phy)) & COMMON_RESET_DIS)) {
+ DRM_DEBUG_DRIVER("DDI PHY %d powered, but still in reset\n",
+ phy);
- return ret;
+ return false;
+ }
+
+ return true;
}
-static void skl_shared_dplls_init(struct drm_i915_private *dev_priv)
+static u32 broxton_get_grc(struct drm_i915_private *dev_priv, enum dpio_phy phy)
{
- int i;
+ u32 val = I915_READ(BXT_PORT_REF_DW6(phy));
- dev_priv->num_shared_dpll = 3;
-
- for (i = 0; i < dev_priv->num_shared_dpll; i++) {
- dev_priv->shared_dplls[i].id = i;
- dev_priv->shared_dplls[i].name = skl_ddi_pll_names[i];
- dev_priv->shared_dplls[i].disable = skl_ddi_pll_disable;
- dev_priv->shared_dplls[i].enable = skl_ddi_pll_enable;
- dev_priv->shared_dplls[i].get_hw_state =
- skl_ddi_pll_get_hw_state;
- }
+ return (val & GRC_CODE_MASK) >> GRC_CODE_SHIFT;
}
+static void broxton_phy_wait_grc_done(struct drm_i915_private *dev_priv,
+ enum dpio_phy phy)
+{
+ if (wait_for(I915_READ(BXT_PORT_REF_DW3(phy)) & GRC_DONE, 10))
+ DRM_ERROR("timeout waiting for PHY%d GRC\n", phy);
+}
+
+static bool broxton_phy_verify_state(struct drm_i915_private *dev_priv,
+ enum dpio_phy phy);
+
static void broxton_phy_init(struct drm_i915_private *dev_priv,
enum dpio_phy phy)
{
enum port port;
- uint32_t val;
+ u32 ports, val;
+
+ if (broxton_phy_is_enabled(dev_priv, phy)) {
+ /* Still read out the GRC value for state verification */
+ if (phy == DPIO_PHY0)
+ dev_priv->bxt_phy_grc = broxton_get_grc(dev_priv, phy);
+
+ if (broxton_phy_verify_state(dev_priv, phy)) {
+ DRM_DEBUG_DRIVER("DDI PHY %d already enabled, "
+ "won't reprogram it\n", phy);
+
+ return;
+ }
+
+ DRM_DEBUG_DRIVER("DDI PHY %d enabled with invalid state, "
+ "force reprogramming it\n", phy);
+ } else {
+ DRM_DEBUG_DRIVER("DDI PHY %d not enabled, enabling it\n", phy);
+ }
val = I915_READ(BXT_P_CR_GT_DISP_PWRON);
val |= GT_DISPLAY_POWER_ON(phy);
I915_WRITE(BXT_P_CR_GT_DISP_PWRON, val);
- /* Considering 10ms timeout until BSpec is updated */
- if (wait_for(I915_READ(BXT_PORT_CL1CM_DW0(phy)) & PHY_POWER_GOOD, 10))
+ /*
+ * The PHY registers start out inaccessible and respond to reads with
+ * all 1s. Eventually they become accessible as they power up, then
+ * the reserved bit will give the default 0. Poll on the reserved bit
+ * becoming 0 to find when the PHY is accessible.
+ * HW team confirmed that the time to reach phypowergood status is
+ * anywhere between 50 us and 100us.
+ */
+ if (wait_for_us(((I915_READ(BXT_PORT_CL1CM_DW0(phy)) &
+ (PHY_RESERVED | PHY_POWER_GOOD)) == PHY_POWER_GOOD), 100)) {
DRM_ERROR("timeout during PHY%d power on\n", phy);
+ }
- for (port = (phy == DPIO_PHY0 ? PORT_B : PORT_A);
- port <= (phy == DPIO_PHY0 ? PORT_C : PORT_A); port++) {
+ if (phy == DPIO_PHY0)
+ ports = BIT(PORT_B) | BIT(PORT_C);
+ else
+ ports = BIT(PORT_A);
+
+ for_each_port_masked(port, ports) {
int lane;
for (lane = 0; lane < 4; lane++) {
@@ -2730,6 +1872,9 @@
* enabled.
* TODO: port C is only connected on BXT-P, so on BXT0/1 we should
* power down the second channel on PHY0 as well.
+ *
+ * FIXME: Clarify programming of the following, the register is
+ * read-only with bit 6 fixed at 0 at least in stepping A.
*/
if (phy == DPIO_PHY1)
val |= OCL2_LDOFUSE_PWR_DIS;
@@ -2742,12 +1887,10 @@
* the corresponding calibrated value from PHY1, and disable
* the automatic calibration on PHY0.
*/
- if (wait_for(I915_READ(BXT_PORT_REF_DW3(DPIO_PHY1)) & GRC_DONE,
- 10))
- DRM_ERROR("timeout waiting for PHY1 GRC\n");
+ broxton_phy_wait_grc_done(dev_priv, DPIO_PHY1);
- val = I915_READ(BXT_PORT_REF_DW6(DPIO_PHY1));
- val = (val & GRC_CODE_MASK) >> GRC_CODE_SHIFT;
+ val = dev_priv->bxt_phy_grc = broxton_get_grc(dev_priv,
+ DPIO_PHY1);
grc_code = val << GRC_CODE_FAST_SHIFT |
val << GRC_CODE_SLOW_SHIFT |
val;
@@ -2757,17 +1900,27 @@
val |= GRC_DIS | GRC_RDY_OVRD;
I915_WRITE(BXT_PORT_REF_DW8(DPIO_PHY0), val);
}
+ /*
+ * During PHY1 init delay waiting for GRC calibration to finish, since
+ * it can happen in parallel with the subsequent PHY0 init.
+ */
val = I915_READ(BXT_PHY_CTL_FAMILY(phy));
val |= COMMON_RESET_DIS;
I915_WRITE(BXT_PHY_CTL_FAMILY(phy), val);
}
-void broxton_ddi_phy_init(struct drm_device *dev)
+void broxton_ddi_phy_init(struct drm_i915_private *dev_priv)
{
/* Enable PHY1 first since it provides Rcomp for PHY0 */
- broxton_phy_init(dev->dev_private, DPIO_PHY1);
- broxton_phy_init(dev->dev_private, DPIO_PHY0);
+ broxton_phy_init(dev_priv, DPIO_PHY1);
+ broxton_phy_init(dev_priv, DPIO_PHY0);
+
+ /*
+ * If BIOS enabled only PHY0 and not PHY1, we skipped waiting for the
+ * PHY1 GRC calibration to finish, so wait for it here.
+ */
+ broxton_phy_wait_grc_done(dev_priv, DPIO_PHY1);
}
static void broxton_phy_uninit(struct drm_i915_private *dev_priv,
@@ -2778,260 +1931,126 @@
val = I915_READ(BXT_PHY_CTL_FAMILY(phy));
val &= ~COMMON_RESET_DIS;
I915_WRITE(BXT_PHY_CTL_FAMILY(phy), val);
+
+ val = I915_READ(BXT_P_CR_GT_DISP_PWRON);
+ val &= ~GT_DISPLAY_POWER_ON(phy);
+ I915_WRITE(BXT_P_CR_GT_DISP_PWRON, val);
}
-void broxton_ddi_phy_uninit(struct drm_device *dev)
+void broxton_ddi_phy_uninit(struct drm_i915_private *dev_priv)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
-
broxton_phy_uninit(dev_priv, DPIO_PHY1);
broxton_phy_uninit(dev_priv, DPIO_PHY0);
-
- /* FIXME: do this in broxton_phy_uninit per phy */
- I915_WRITE(BXT_P_CR_GT_DISP_PWRON, 0);
}
-static const char * const bxt_ddi_pll_names[] = {
- "PORT PLL A",
- "PORT PLL B",
- "PORT PLL C",
-};
-
-static void bxt_ddi_pll_enable(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll)
+static bool __printf(6, 7)
+__phy_reg_verify_state(struct drm_i915_private *dev_priv, enum dpio_phy phy,
+ i915_reg_t reg, u32 mask, u32 expected,
+ const char *reg_fmt, ...)
{
- uint32_t temp;
- enum port port = (enum port)pll->id; /* 1:1 port->PLL mapping */
+ struct va_format vaf;
+ va_list args;
+ u32 val;
- temp = I915_READ(BXT_PORT_PLL_ENABLE(port));
- temp &= ~PORT_PLL_REF_SEL;
- /* Non-SSC reference */
- I915_WRITE(BXT_PORT_PLL_ENABLE(port), temp);
+ val = I915_READ(reg);
+ if ((val & mask) == expected)
+ return true;
- /* Disable 10 bit clock */
- temp = I915_READ(BXT_PORT_PLL_EBB_4(port));
- temp &= ~PORT_PLL_10BIT_CLK_ENABLE;
- I915_WRITE(BXT_PORT_PLL_EBB_4(port), temp);
+ va_start(args, reg_fmt);
+ vaf.fmt = reg_fmt;
+ vaf.va = &args;
- /* Write P1 & P2 */
- temp = I915_READ(BXT_PORT_PLL_EBB_0(port));
- temp &= ~(PORT_PLL_P1_MASK | PORT_PLL_P2_MASK);
- temp |= pll->config.hw_state.ebb0;
- I915_WRITE(BXT_PORT_PLL_EBB_0(port), temp);
+ DRM_DEBUG_DRIVER("DDI PHY %d reg %pV [%08x] state mismatch: "
+ "current %08x, expected %08x (mask %08x)\n",
+ phy, &vaf, reg.reg, val, (val & ~mask) | expected,
+ mask);
- /* Write M2 integer */
- temp = I915_READ(BXT_PORT_PLL(port, 0));
- temp &= ~PORT_PLL_M2_MASK;
- temp |= pll->config.hw_state.pll0;
- I915_WRITE(BXT_PORT_PLL(port, 0), temp);
+ va_end(args);
- /* Write N */
- temp = I915_READ(BXT_PORT_PLL(port, 1));
- temp &= ~PORT_PLL_N_MASK;
- temp |= pll->config.hw_state.pll1;
- I915_WRITE(BXT_PORT_PLL(port, 1), temp);
-
- /* Write M2 fraction */
- temp = I915_READ(BXT_PORT_PLL(port, 2));
- temp &= ~PORT_PLL_M2_FRAC_MASK;
- temp |= pll->config.hw_state.pll2;
- I915_WRITE(BXT_PORT_PLL(port, 2), temp);
-
- /* Write M2 fraction enable */
- temp = I915_READ(BXT_PORT_PLL(port, 3));
- temp &= ~PORT_PLL_M2_FRAC_ENABLE;
- temp |= pll->config.hw_state.pll3;
- I915_WRITE(BXT_PORT_PLL(port, 3), temp);
-
- /* Write coeff */
- temp = I915_READ(BXT_PORT_PLL(port, 6));
- temp &= ~PORT_PLL_PROP_COEFF_MASK;
- temp &= ~PORT_PLL_INT_COEFF_MASK;
- temp &= ~PORT_PLL_GAIN_CTL_MASK;
- temp |= pll->config.hw_state.pll6;
- I915_WRITE(BXT_PORT_PLL(port, 6), temp);
-
- /* Write calibration val */
- temp = I915_READ(BXT_PORT_PLL(port, 8));
- temp &= ~PORT_PLL_TARGET_CNT_MASK;
- temp |= pll->config.hw_state.pll8;
- I915_WRITE(BXT_PORT_PLL(port, 8), temp);
-
- temp = I915_READ(BXT_PORT_PLL(port, 9));
- temp &= ~PORT_PLL_LOCK_THRESHOLD_MASK;
- temp |= pll->config.hw_state.pll9;
- I915_WRITE(BXT_PORT_PLL(port, 9), temp);
-
- temp = I915_READ(BXT_PORT_PLL(port, 10));
- temp &= ~PORT_PLL_DCO_AMP_OVR_EN_H;
- temp &= ~PORT_PLL_DCO_AMP_MASK;
- temp |= pll->config.hw_state.pll10;
- I915_WRITE(BXT_PORT_PLL(port, 10), temp);
-
- /* Recalibrate with new settings */
- temp = I915_READ(BXT_PORT_PLL_EBB_4(port));
- temp |= PORT_PLL_RECALIBRATE;
- I915_WRITE(BXT_PORT_PLL_EBB_4(port), temp);
- temp &= ~PORT_PLL_10BIT_CLK_ENABLE;
- temp |= pll->config.hw_state.ebb4;
- I915_WRITE(BXT_PORT_PLL_EBB_4(port), temp);
-
- /* Enable PLL */
- temp = I915_READ(BXT_PORT_PLL_ENABLE(port));
- temp |= PORT_PLL_ENABLE;
- I915_WRITE(BXT_PORT_PLL_ENABLE(port), temp);
- POSTING_READ(BXT_PORT_PLL_ENABLE(port));
-
- if (wait_for_atomic_us((I915_READ(BXT_PORT_PLL_ENABLE(port)) &
- PORT_PLL_LOCK), 200))
- DRM_ERROR("PLL %d not locked\n", port);
-
- /*
- * While we write to the group register to program all lanes at once we
- * can read only lane registers and we pick lanes 0/1 for that.
- */
- temp = I915_READ(BXT_PORT_PCS_DW12_LN01(port));
- temp &= ~LANE_STAGGER_MASK;
- temp &= ~LANESTAGGER_STRAP_OVRD;
- temp |= pll->config.hw_state.pcsdw12;
- I915_WRITE(BXT_PORT_PCS_DW12_GRP(port), temp);
+ return false;
}
-static void bxt_ddi_pll_disable(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll)
+static bool broxton_phy_verify_state(struct drm_i915_private *dev_priv,
+ enum dpio_phy phy)
{
- enum port port = (enum port)pll->id; /* 1:1 port->PLL mapping */
- uint32_t temp;
+ enum port port;
+ u32 ports;
+ uint32_t mask;
+ bool ok;
- temp = I915_READ(BXT_PORT_PLL_ENABLE(port));
- temp &= ~PORT_PLL_ENABLE;
- I915_WRITE(BXT_PORT_PLL_ENABLE(port), temp);
- POSTING_READ(BXT_PORT_PLL_ENABLE(port));
-}
+#define _CHK(reg, mask, exp, fmt, ...) \
+ __phy_reg_verify_state(dev_priv, phy, reg, mask, exp, fmt, \
+ ## __VA_ARGS__)
-static bool bxt_ddi_pll_get_hw_state(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll,
- struct intel_dpll_hw_state *hw_state)
-{
- enum port port = (enum port)pll->id; /* 1:1 port->PLL mapping */
- uint32_t val;
- bool ret;
-
- if (!intel_display_power_get_if_enabled(dev_priv, POWER_DOMAIN_PLLS))
+ /* We expect the PHY to be always enabled */
+ if (!broxton_phy_is_enabled(dev_priv, phy))
return false;
- ret = false;
+ ok = true;
- val = I915_READ(BXT_PORT_PLL_ENABLE(port));
- if (!(val & PORT_PLL_ENABLE))
- goto out;
+ if (phy == DPIO_PHY0)
+ ports = BIT(PORT_B) | BIT(PORT_C);
+ else
+ ports = BIT(PORT_A);
- hw_state->ebb0 = I915_READ(BXT_PORT_PLL_EBB_0(port));
- hw_state->ebb0 &= PORT_PLL_P1_MASK | PORT_PLL_P2_MASK;
+ for_each_port_masked(port, ports) {
+ int lane;
- hw_state->ebb4 = I915_READ(BXT_PORT_PLL_EBB_4(port));
- hw_state->ebb4 &= PORT_PLL_10BIT_CLK_ENABLE;
+ for (lane = 0; lane < 4; lane++)
+ ok &= _CHK(BXT_PORT_TX_DW14_LN(port, lane),
+ LATENCY_OPTIM,
+ lane != 1 ? LATENCY_OPTIM : 0,
+ "BXT_PORT_TX_DW14_LN(%d, %d)", port, lane);
+ }
- hw_state->pll0 = I915_READ(BXT_PORT_PLL(port, 0));
- hw_state->pll0 &= PORT_PLL_M2_MASK;
+ /* PLL Rcomp code offset */
+ ok &= _CHK(BXT_PORT_CL1CM_DW9(phy),
+ IREF0RC_OFFSET_MASK, 0xe4 << IREF0RC_OFFSET_SHIFT,
+ "BXT_PORT_CL1CM_DW9(%d)", phy);
+ ok &= _CHK(BXT_PORT_CL1CM_DW10(phy),
+ IREF1RC_OFFSET_MASK, 0xe4 << IREF1RC_OFFSET_SHIFT,
+ "BXT_PORT_CL1CM_DW10(%d)", phy);
- hw_state->pll1 = I915_READ(BXT_PORT_PLL(port, 1));
- hw_state->pll1 &= PORT_PLL_N_MASK;
+ /* Power gating */
+ mask = OCL1_POWER_DOWN_EN | DW28_OLDO_DYN_PWR_DOWN_EN | SUS_CLK_CONFIG;
+ ok &= _CHK(BXT_PORT_CL1CM_DW28(phy), mask, mask,
+ "BXT_PORT_CL1CM_DW28(%d)", phy);
- hw_state->pll2 = I915_READ(BXT_PORT_PLL(port, 2));
- hw_state->pll2 &= PORT_PLL_M2_FRAC_MASK;
-
- hw_state->pll3 = I915_READ(BXT_PORT_PLL(port, 3));
- hw_state->pll3 &= PORT_PLL_M2_FRAC_ENABLE;
-
- hw_state->pll6 = I915_READ(BXT_PORT_PLL(port, 6));
- hw_state->pll6 &= PORT_PLL_PROP_COEFF_MASK |
- PORT_PLL_INT_COEFF_MASK |
- PORT_PLL_GAIN_CTL_MASK;
-
- hw_state->pll8 = I915_READ(BXT_PORT_PLL(port, 8));
- hw_state->pll8 &= PORT_PLL_TARGET_CNT_MASK;
-
- hw_state->pll9 = I915_READ(BXT_PORT_PLL(port, 9));
- hw_state->pll9 &= PORT_PLL_LOCK_THRESHOLD_MASK;
-
- hw_state->pll10 = I915_READ(BXT_PORT_PLL(port, 10));
- hw_state->pll10 &= PORT_PLL_DCO_AMP_OVR_EN_H |
- PORT_PLL_DCO_AMP_MASK;
+ if (phy == DPIO_PHY0)
+ ok &= _CHK(BXT_PORT_CL2CM_DW6_BC,
+ DW6_OLDO_DYN_PWR_DOWN_EN, DW6_OLDO_DYN_PWR_DOWN_EN,
+ "BXT_PORT_CL2CM_DW6_BC");
/*
- * While we write to the group register to program all lanes at once we
- * can read only lane registers. We configure all lanes the same way, so
- * here just read out lanes 0/1 and output a note if lanes 2/3 differ.
+ * TODO: Verify BXT_PORT_CL1CM_DW30 bit OCL2_LDOFUSE_PWR_DIS,
+ * at least on stepping A this bit is read-only and fixed at 0.
*/
- hw_state->pcsdw12 = I915_READ(BXT_PORT_PCS_DW12_LN01(port));
- if (I915_READ(BXT_PORT_PCS_DW12_LN23(port)) != hw_state->pcsdw12)
- DRM_DEBUG_DRIVER("lane stagger config different for lane 01 (%08x) and 23 (%08x)\n",
- hw_state->pcsdw12,
- I915_READ(BXT_PORT_PCS_DW12_LN23(port)));
- hw_state->pcsdw12 &= LANE_STAGGER_MASK | LANESTAGGER_STRAP_OVRD;
- ret = true;
+ if (phy == DPIO_PHY0) {
+ u32 grc_code = dev_priv->bxt_phy_grc;
-out:
- intel_display_power_put(dev_priv, POWER_DOMAIN_PLLS);
+ grc_code = grc_code << GRC_CODE_FAST_SHIFT |
+ grc_code << GRC_CODE_SLOW_SHIFT |
+ grc_code;
+ mask = GRC_CODE_FAST_MASK | GRC_CODE_SLOW_MASK |
+ GRC_CODE_NOM_MASK;
+ ok &= _CHK(BXT_PORT_REF_DW6(DPIO_PHY0), mask, grc_code,
+ "BXT_PORT_REF_DW6(%d)", DPIO_PHY0);
- return ret;
+ mask = GRC_DIS | GRC_RDY_OVRD;
+ ok &= _CHK(BXT_PORT_REF_DW8(DPIO_PHY0), mask, mask,
+ "BXT_PORT_REF_DW8(%d)", DPIO_PHY0);
+ }
+
+ return ok;
+#undef _CHK
}
-static void bxt_shared_dplls_init(struct drm_i915_private *dev_priv)
+void broxton_ddi_phy_verify_state(struct drm_i915_private *dev_priv)
{
- int i;
-
- dev_priv->num_shared_dpll = 3;
-
- for (i = 0; i < dev_priv->num_shared_dpll; i++) {
- dev_priv->shared_dplls[i].id = i;
- dev_priv->shared_dplls[i].name = bxt_ddi_pll_names[i];
- dev_priv->shared_dplls[i].disable = bxt_ddi_pll_disable;
- dev_priv->shared_dplls[i].enable = bxt_ddi_pll_enable;
- dev_priv->shared_dplls[i].get_hw_state =
- bxt_ddi_pll_get_hw_state;
- }
-}
-
-void intel_ddi_pll_init(struct drm_device *dev)
-{
- struct drm_i915_private *dev_priv = dev->dev_private;
- uint32_t val = I915_READ(LCPLL_CTL);
-
- if (IS_SKYLAKE(dev) || IS_KABYLAKE(dev))
- skl_shared_dplls_init(dev_priv);
- else if (IS_BROXTON(dev))
- bxt_shared_dplls_init(dev_priv);
- else
- hsw_shared_dplls_init(dev_priv);
-
- if (IS_SKYLAKE(dev) || IS_KABYLAKE(dev)) {
- int cdclk_freq;
-
- cdclk_freq = dev_priv->display.get_display_clock_speed(dev);
- dev_priv->skl_boot_cdclk = cdclk_freq;
- if (skl_sanitize_cdclk(dev_priv))
- DRM_DEBUG_KMS("Sanitized cdclk programmed by pre-os\n");
- if (!(I915_READ(LCPLL1_CTL) & LCPLL_PLL_ENABLE))
- DRM_ERROR("LCPLL1 is disabled\n");
- } else if (IS_BROXTON(dev)) {
- broxton_init_cdclk(dev);
- broxton_ddi_phy_init(dev);
- } else {
- /*
- * The LCPLL register should be turned on by the BIOS. For now
- * let's just check its state and print errors in case
- * something is wrong. Don't even try to turn it on.
- */
-
- if (val & LCPLL_CD_SOURCE_FCLK)
- DRM_ERROR("CDCLK source is not LCPLL\n");
-
- if (val & LCPLL_PLL_DISABLE)
- DRM_ERROR("LCPLL is disabled\n");
- }
+ if (!broxton_phy_verify_state(dev_priv, DPIO_PHY0) ||
+ !broxton_phy_verify_state(dev_priv, DPIO_PHY1))
+ i915_report_error(dev_priv, "DDI PHY state mismatch\n");
}
void intel_ddi_prepare_link_retrain(struct intel_dp *intel_dp)
@@ -3086,12 +2105,18 @@
struct intel_encoder *intel_encoder = intel_ddi_get_crtc_encoder(crtc);
uint32_t val;
- intel_ddi_post_disable(intel_encoder);
-
+ /*
+ * Bspec lists this as both step 13 (before DDI_BUF_CTL disable)
+ * and step 18 (after clearing PORT_CLK_SEL). Based on a BUN,
+ * step 13 is the correct place for it. Step 18 is where it was
+ * originally before the BUN.
+ */
val = I915_READ(FDI_RX_CTL(PIPE_A));
val &= ~FDI_RX_ENABLE;
I915_WRITE(FDI_RX_CTL(PIPE_A), val);
+ intel_ddi_post_disable(intel_encoder);
+
val = I915_READ(FDI_RX_MISC(PIPE_A));
val &= ~(FDI_RX_PWRDN_LANE1_MASK | FDI_RX_PWRDN_LANE0_MASK);
val |= FDI_RX_PWRDN_LANE1_VAL(2) | FDI_RX_PWRDN_LANE0_VAL(2);
@@ -3115,6 +2140,10 @@
struct intel_hdmi *intel_hdmi;
u32 temp, flags = 0;
+ /* XXX: DSI transcoder paranoia */
+ if (WARN_ON(transcoder_is_dsi(cpu_transcoder)))
+ return;
+
temp = I915_READ(TRANS_DDI_FUNC_CTL(cpu_transcoder));
if (temp & TRANS_DDI_PHSYNC)
flags |= DRM_MODE_FLAG_PHSYNC;
@@ -3172,8 +2201,8 @@
pipe_config->has_audio = true;
}
- if (encoder->type == INTEL_OUTPUT_EDP && dev_priv->vbt.edp_bpp &&
- pipe_config->pipe_bpp > dev_priv->vbt.edp_bpp) {
+ if (encoder->type == INTEL_OUTPUT_EDP && dev_priv->vbt.edp.bpp &&
+ pipe_config->pipe_bpp > dev_priv->vbt.edp.bpp) {
/*
* This is a big fat ugly hack.
*
@@ -3188,8 +2217,8 @@
* load.
*/
DRM_DEBUG_KMS("pipe has %d bpp for eDP panel, overriding BIOS-provided max %d bpp\n",
- pipe_config->pipe_bpp, dev_priv->vbt.edp_bpp);
- dev_priv->vbt.edp_bpp = pipe_config->pipe_bpp;
+ pipe_config->pipe_bpp, dev_priv->vbt.edp.bpp);
+ dev_priv->vbt.edp.bpp = pipe_config->pipe_bpp;
}
intel_ddi_clock_get(encoder, pipe_config);
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index 0104a06..46f9be3 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -36,6 +36,7 @@
#include "intel_drv.h"
#include <drm/i915_drm.h>
#include "i915_drv.h"
+#include "intel_dsi.h"
#include "i915_trace.h"
#include <drm/drm_atomic.h>
#include <drm/drm_atomic_helper.h>
@@ -96,12 +97,13 @@
struct drm_i915_gem_object *obj);
static void i9xx_set_pipeconf(struct intel_crtc *intel_crtc);
static void intel_set_pipe_timings(struct intel_crtc *intel_crtc);
+static void intel_set_pipe_src_size(struct intel_crtc *intel_crtc);
static void intel_cpu_transcoder_set_m_n(struct intel_crtc *crtc,
struct intel_link_m_n *m_n,
struct intel_link_m_n *m2_n2);
static void ironlake_set_pipeconf(struct drm_crtc *crtc);
static void haswell_set_pipeconf(struct drm_crtc *crtc);
-static void intel_set_pipe_csc(struct drm_crtc *crtc);
+static void haswell_set_pipemisc(struct drm_crtc *crtc);
static void vlv_prepare_pll(struct intel_crtc *crtc,
const struct intel_crtc_state *pipe_config);
static void chv_prepare_pll(struct intel_crtc *crtc,
@@ -110,13 +112,11 @@
static void intel_finish_crtc_commit(struct drm_crtc *, struct drm_crtc_state *);
static void skl_init_scalers(struct drm_device *dev, struct intel_crtc *intel_crtc,
struct intel_crtc_state *crtc_state);
-static int i9xx_get_refclk(const struct intel_crtc_state *crtc_state,
- int num_connectors);
static void skylake_pfit_enable(struct intel_crtc *crtc);
static void ironlake_pfit_disable(struct intel_crtc *crtc, bool force);
static void ironlake_pfit_enable(struct intel_crtc *crtc);
static void intel_modeset_setup_hw_state(struct drm_device *dev);
-static void intel_pre_disable_primary(struct drm_crtc *crtc);
+static void intel_pre_disable_primary_noatomic(struct drm_crtc *crtc);
typedef struct {
int min, max;
@@ -147,15 +147,12 @@
return vco_freq[hpll_freq] * 1000;
}
-static int vlv_get_cck_clock_hpll(struct drm_i915_private *dev_priv,
- const char *name, u32 reg)
+int vlv_get_cck_clock(struct drm_i915_private *dev_priv,
+ const char *name, u32 reg, int ref_freq)
{
u32 val;
int divider;
- if (dev_priv->hpll_freq == 0)
- dev_priv->hpll_freq = valleyview_get_vco(dev_priv);
-
mutex_lock(&dev_priv->sb_lock);
val = vlv_cck_read(dev_priv, reg);
mutex_unlock(&dev_priv->sb_lock);
@@ -166,52 +163,75 @@
(divider << CCK_FREQUENCY_STATUS_SHIFT),
"%s change in progress\n", name);
- return DIV_ROUND_CLOSEST(dev_priv->hpll_freq << 1, divider + 1);
+ return DIV_ROUND_CLOSEST(ref_freq << 1, divider + 1);
}
-int
-intel_pch_rawclk(struct drm_device *dev)
+static int vlv_get_cck_clock_hpll(struct drm_i915_private *dev_priv,
+ const char *name, u32 reg)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
+ if (dev_priv->hpll_freq == 0)
+ dev_priv->hpll_freq = valleyview_get_vco(dev_priv);
- WARN_ON(!HAS_PCH_SPLIT(dev));
-
- return I915_READ(PCH_RAWCLK_FREQ) & RAWCLK_FREQ_MASK;
+ return vlv_get_cck_clock(dev_priv, name, reg,
+ dev_priv->hpll_freq);
}
-/* hrawclock is 1/4 the FSB frequency */
-int intel_hrawclk(struct drm_device *dev)
+static int
+intel_pch_rawclk(struct drm_i915_private *dev_priv)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
+ return (I915_READ(PCH_RAWCLK_FREQ) & RAWCLK_FREQ_MASK) * 1000;
+}
+
+static int
+intel_vlv_hrawclk(struct drm_i915_private *dev_priv)
+{
+ return vlv_get_cck_clock_hpll(dev_priv, "hrawclk",
+ CCK_DISPLAY_REF_CLOCK_CONTROL);
+}
+
+static int
+intel_g4x_hrawclk(struct drm_i915_private *dev_priv)
+{
uint32_t clkcfg;
- /* There is no CLKCFG reg in Valleyview. VLV hrawclk is 200 MHz */
- if (IS_VALLEYVIEW(dev) || IS_CHERRYVIEW(dev))
- return 200;
-
+ /* hrawclock is 1/4 the FSB frequency */
clkcfg = I915_READ(CLKCFG);
switch (clkcfg & CLKCFG_FSB_MASK) {
case CLKCFG_FSB_400:
- return 100;
+ return 100000;
case CLKCFG_FSB_533:
- return 133;
+ return 133333;
case CLKCFG_FSB_667:
- return 166;
+ return 166667;
case CLKCFG_FSB_800:
- return 200;
+ return 200000;
case CLKCFG_FSB_1067:
- return 266;
+ return 266667;
case CLKCFG_FSB_1333:
- return 333;
+ return 333333;
/* these two are just a guess; one of them might be right */
case CLKCFG_FSB_1600:
case CLKCFG_FSB_1600_ALT:
- return 400;
+ return 400000;
default:
- return 133;
+ return 133333;
}
}
+static void intel_update_rawclk(struct drm_i915_private *dev_priv)
+{
+ if (HAS_PCH_SPLIT(dev_priv))
+ dev_priv->rawclk_freq = intel_pch_rawclk(dev_priv);
+ else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
+ dev_priv->rawclk_freq = intel_vlv_hrawclk(dev_priv);
+ else if (IS_G4X(dev_priv) || IS_PINEVIEW(dev_priv))
+ dev_priv->rawclk_freq = intel_g4x_hrawclk(dev_priv);
+ else
+ return; /* no rawclk on other platforms, or no need to know it */
+
+ DRM_DEBUG_DRIVER("rawclk rate: %d kHz\n", dev_priv->rawclk_freq);
+}
+
static void intel_update_czclk(struct drm_i915_private *dev_priv)
{
if (!(IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)))
@@ -224,13 +244,15 @@
}
static inline u32 /* units of 100MHz */
-intel_fdi_link_freq(struct drm_device *dev)
+intel_fdi_link_freq(struct drm_i915_private *dev_priv,
+ const struct intel_crtc_state *pipe_config)
{
- if (IS_GEN5(dev)) {
- struct drm_i915_private *dev_priv = dev->dev_private;
- return (I915_READ(FDI_PLL_BIOS_0) & FDI_PLL_FB_CLOCK_MASK) + 2;
- } else
- return 27;
+ if (HAS_DDI(dev_priv))
+ return pipe_config->port_clock; /* SPLL */
+ else if (IS_GEN5(dev_priv))
+ return ((I915_READ(FDI_PLL_BIOS_0) & FDI_PLL_FB_CLOCK_MASK) + 2) * 10000;
+ else
+ return 270000;
}
static const intel_limit_t intel_limits_i8xx_dac = {
@@ -550,89 +572,6 @@
return false;
}
-static const intel_limit_t *
-intel_ironlake_limit(struct intel_crtc_state *crtc_state, int refclk)
-{
- struct drm_device *dev = crtc_state->base.crtc->dev;
- const intel_limit_t *limit;
-
- if (intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_LVDS)) {
- if (intel_is_dual_link_lvds(dev)) {
- if (refclk == 100000)
- limit = &intel_limits_ironlake_dual_lvds_100m;
- else
- limit = &intel_limits_ironlake_dual_lvds;
- } else {
- if (refclk == 100000)
- limit = &intel_limits_ironlake_single_lvds_100m;
- else
- limit = &intel_limits_ironlake_single_lvds;
- }
- } else
- limit = &intel_limits_ironlake_dac;
-
- return limit;
-}
-
-static const intel_limit_t *
-intel_g4x_limit(struct intel_crtc_state *crtc_state)
-{
- struct drm_device *dev = crtc_state->base.crtc->dev;
- const intel_limit_t *limit;
-
- if (intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_LVDS)) {
- if (intel_is_dual_link_lvds(dev))
- limit = &intel_limits_g4x_dual_channel_lvds;
- else
- limit = &intel_limits_g4x_single_channel_lvds;
- } else if (intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_HDMI) ||
- intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_ANALOG)) {
- limit = &intel_limits_g4x_hdmi;
- } else if (intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_SDVO)) {
- limit = &intel_limits_g4x_sdvo;
- } else /* The option is for other outputs */
- limit = &intel_limits_i9xx_sdvo;
-
- return limit;
-}
-
-static const intel_limit_t *
-intel_limit(struct intel_crtc_state *crtc_state, int refclk)
-{
- struct drm_device *dev = crtc_state->base.crtc->dev;
- const intel_limit_t *limit;
-
- if (IS_BROXTON(dev))
- limit = &intel_limits_bxt;
- else if (HAS_PCH_SPLIT(dev))
- limit = intel_ironlake_limit(crtc_state, refclk);
- else if (IS_G4X(dev)) {
- limit = intel_g4x_limit(crtc_state);
- } else if (IS_PINEVIEW(dev)) {
- if (intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_LVDS))
- limit = &intel_limits_pineview_lvds;
- else
- limit = &intel_limits_pineview_sdvo;
- } else if (IS_CHERRYVIEW(dev)) {
- limit = &intel_limits_chv;
- } else if (IS_VALLEYVIEW(dev)) {
- limit = &intel_limits_vlv;
- } else if (!IS_GEN2(dev)) {
- if (intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_LVDS))
- limit = &intel_limits_i9xx_lvds;
- else
- limit = &intel_limits_i9xx_sdvo;
- } else {
- if (intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_LVDS))
- limit = &intel_limits_i8xx_lvds;
- else if (intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_DVO))
- limit = &intel_limits_i8xx_dvo;
- else
- limit = &intel_limits_i8xx_dac;
- }
- return limit;
-}
-
/*
* Platform specific helpers to calculate the port PLL loopback- (clock.m),
* and post-divider (clock.p) values, pre- (clock.vco) and post-divided fast
@@ -763,6 +702,16 @@
}
}
+/*
+ * Returns a set of divisors for the desired target clock with the given
+ * refclk, or FALSE. The returned values represent the clock equation:
+ * reflck * (5 * (m1 + 2) + (m2 + 2)) / (n + 2) / p1 / p2.
+ *
+ * Target and reference clocks are specified in kHz.
+ *
+ * If match_clock is provided, then best_clock P divider must match the P
+ * divider from @match_clock used for LVDS downclocking.
+ */
static bool
i9xx_find_best_dpll(const intel_limit_t *limit,
struct intel_crtc_state *crtc_state,
@@ -810,6 +759,16 @@
return (err != target);
}
+/*
+ * Returns a set of divisors for the desired target clock with the given
+ * refclk, or FALSE. The returned values represent the clock equation:
+ * reflck * (5 * (m1 + 2) + (m2 + 2)) / (n + 2) / p1 / p2.
+ *
+ * Target and reference clocks are specified in kHz.
+ *
+ * If match_clock is provided, then best_clock P divider must match the P
+ * divider from @match_clock used for LVDS downclocking.
+ */
static bool
pnv_find_best_dpll(const intel_limit_t *limit,
struct intel_crtc_state *crtc_state,
@@ -855,6 +814,16 @@
return (err != target);
}
+/*
+ * Returns a set of divisors for the desired target clock with the given
+ * refclk, or FALSE. The returned values represent the clock equation:
+ * reflck * (5 * (m1 + 2) + (m2 + 2)) / (n + 2) / p1 / p2.
+ *
+ * Target and reference clocks are specified in kHz.
+ *
+ * If match_clock is provided, then best_clock P divider must match the P
+ * divider from @match_clock used for LVDS downclocking.
+ */
static bool
g4x_find_best_dpll(const intel_limit_t *limit,
struct intel_crtc_state *crtc_state,
@@ -943,6 +912,11 @@
return *error_ppm + 10 < best_error_ppm;
}
+/*
+ * Returns a set of divisors for the desired target clock with the given
+ * refclk, or FALSE. The returned values represent the clock equation:
+ * reflck * (5 * (m1 + 2) + (m2 + 2)) / (n + 2) / p1 / p2.
+ */
static bool
vlv_find_best_dpll(const intel_limit_t *limit,
struct intel_crtc_state *crtc_state,
@@ -997,6 +971,11 @@
return found;
}
+/*
+ * Returns a set of divisors for the desired target clock with the given
+ * refclk, or FALSE. The returned values represent the clock equation:
+ * reflck * (5 * (m1 + 2) + (m2 + 2)) / (n + 2) / p1 / p2.
+ */
static bool
chv_find_best_dpll(const intel_limit_t *limit,
struct intel_crtc_state *crtc_state,
@@ -1058,9 +1037,10 @@
bool bxt_find_best_dpll(struct intel_crtc_state *crtc_state, int target_clock,
intel_clock_t *best_clock)
{
- int refclk = i9xx_get_refclk(crtc_state, 0);
+ int refclk = 100000;
+ const intel_limit_t *limit = &intel_limits_bxt;
- return chv_find_best_dpll(intel_limit(crtc_state, refclk), crtc_state,
+ return chv_find_best_dpll(limit, crtc_state,
target_clock, refclk, NULL, best_clock);
}
@@ -1165,7 +1145,7 @@
}
/* XXX: the dsi pll is shared between MIPI DSI ports */
-static void assert_dsi_pll(struct drm_i915_private *dev_priv, bool state)
+void assert_dsi_pll(struct drm_i915_private *dev_priv, bool state)
{
u32 val;
bool cur_state;
@@ -1179,36 +1159,6 @@
"DSI PLL state assertion failure (expected %s, current %s)\n",
onoff(state), onoff(cur_state));
}
-#define assert_dsi_pll_enabled(d) assert_dsi_pll(d, true)
-#define assert_dsi_pll_disabled(d) assert_dsi_pll(d, false)
-
-struct intel_shared_dpll *
-intel_crtc_to_shared_dpll(struct intel_crtc *crtc)
-{
- struct drm_i915_private *dev_priv = crtc->base.dev->dev_private;
-
- if (crtc->config->shared_dpll < 0)
- return NULL;
-
- return &dev_priv->shared_dplls[crtc->config->shared_dpll];
-}
-
-/* For ILK+ */
-void assert_shared_dpll(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll,
- bool state)
-{
- bool cur_state;
- struct intel_dpll_hw_state hw_state;
-
- if (WARN(!pll, "asserting DPLL %s with no DPLL\n", onoff(state)))
- return;
-
- cur_state = pll->get_hw_state(dev_priv, pll, &hw_state);
- I915_STATE_WARN(cur_state != state,
- "%s assertion failure (expected %s, current %s)\n",
- pll->name, onoff(state), onoff(cur_state));
-}
static void assert_fdi_tx(struct drm_i915_private *dev_priv,
enum pipe pipe, bool state)
@@ -1217,7 +1167,7 @@
enum transcoder cpu_transcoder = intel_pipe_to_cpu_transcoder(dev_priv,
pipe);
- if (HAS_DDI(dev_priv->dev)) {
+ if (HAS_DDI(dev_priv)) {
/* DDI does not have a specific FDI_TX register */
u32 val = I915_READ(TRANS_DDI_FUNC_CTL(cpu_transcoder));
cur_state = !!(val & TRANS_DDI_FUNC_ENABLE);
@@ -1253,11 +1203,11 @@
u32 val;
/* ILK FDI PLL is always enabled */
- if (INTEL_INFO(dev_priv->dev)->gen == 5)
+ if (INTEL_INFO(dev_priv)->gen == 5)
return;
/* On Haswell, DDI ports are responsible for the FDI PLL setup */
- if (HAS_DDI(dev_priv->dev))
+ if (HAS_DDI(dev_priv))
return;
val = I915_READ(FDI_TX_CTL(pipe));
@@ -1446,21 +1396,8 @@
drm_crtc_vblank_put(crtc);
}
-static void ibx_assert_pch_refclk_enabled(struct drm_i915_private *dev_priv)
-{
- u32 val;
- bool enabled;
-
- I915_STATE_WARN_ON(!(HAS_PCH_IBX(dev_priv->dev) || HAS_PCH_CPT(dev_priv->dev)));
-
- val = I915_READ(PCH_DREF_CONTROL);
- enabled = !!(val & (DREF_SSC_SOURCE_MASK | DREF_NONSPREAD_SOURCE_MASK |
- DREF_SUPERSPREAD_SOURCE_MASK));
- I915_STATE_WARN(!enabled, "PCH refclk assertion failure, should be active but is disabled\n");
-}
-
-static void assert_pch_transcoder_disabled(struct drm_i915_private *dev_priv,
- enum pipe pipe)
+void assert_pch_transcoder_disabled(struct drm_i915_private *dev_priv,
+ enum pipe pipe)
{
u32 val;
bool enabled;
@@ -1478,11 +1415,11 @@
if ((val & DP_PORT_EN) == 0)
return false;
- if (HAS_PCH_CPT(dev_priv->dev)) {
+ if (HAS_PCH_CPT(dev_priv)) {
u32 trans_dp_ctl = I915_READ(TRANS_DP_CTL(pipe));
if ((trans_dp_ctl & TRANS_DP_PORT_SEL_MASK) != port_sel)
return false;
- } else if (IS_CHERRYVIEW(dev_priv->dev)) {
+ } else if (IS_CHERRYVIEW(dev_priv)) {
if ((val & DP_PIPE_MASK_CHV) != DP_PIPE_SELECT_CHV(pipe))
return false;
} else {
@@ -1498,10 +1435,10 @@
if ((val & SDVO_ENABLE) == 0)
return false;
- if (HAS_PCH_CPT(dev_priv->dev)) {
+ if (HAS_PCH_CPT(dev_priv)) {
if ((val & SDVO_PIPE_SEL_MASK_CPT) != SDVO_PIPE_SEL_CPT(pipe))
return false;
- } else if (IS_CHERRYVIEW(dev_priv->dev)) {
+ } else if (IS_CHERRYVIEW(dev_priv)) {
if ((val & SDVO_PIPE_SEL_MASK_CHV) != SDVO_PIPE_SEL_CHV(pipe))
return false;
} else {
@@ -1517,7 +1454,7 @@
if ((val & LVDS_PORT_EN) == 0)
return false;
- if (HAS_PCH_CPT(dev_priv->dev)) {
+ if (HAS_PCH_CPT(dev_priv)) {
if ((val & PORT_TRANS_SEL_MASK) != PORT_TRANS_SEL_CPT(pipe))
return false;
} else {
@@ -1532,7 +1469,7 @@
{
if ((val & ADPA_DAC_ENABLE) == 0)
return false;
- if (HAS_PCH_CPT(dev_priv->dev)) {
+ if (HAS_PCH_CPT(dev_priv)) {
if ((val & PORT_TRANS_SEL_MASK) != PORT_TRANS_SEL_CPT(pipe))
return false;
} else {
@@ -1551,7 +1488,7 @@
"PCH DP (0x%08x) enabled on transcoder %c, should be disabled\n",
i915_mmio_reg_offset(reg), pipe_name(pipe));
- I915_STATE_WARN(HAS_PCH_IBX(dev_priv->dev) && (val & DP_PORT_EN) == 0
+ I915_STATE_WARN(HAS_PCH_IBX(dev_priv) && (val & DP_PORT_EN) == 0
&& (val & DP_PIPEB_SELECT),
"IBX PCH dp port still using transcoder B\n");
}
@@ -1564,7 +1501,7 @@
"PCH HDMI (0x%08x) enabled on transcoder %c, should be disabled\n",
i915_mmio_reg_offset(reg), pipe_name(pipe));
- I915_STATE_WARN(HAS_PCH_IBX(dev_priv->dev) && (val & SDVO_ENABLE) == 0
+ I915_STATE_WARN(HAS_PCH_IBX(dev_priv) && (val & SDVO_ENABLE) == 0
&& (val & SDVO_PIPE_B_SELECT),
"IBX PCH hdmi port still using transcoder B\n");
}
@@ -1593,53 +1530,47 @@
assert_pch_hdmi_disabled(dev_priv, pipe, PCH_HDMID);
}
+static void _vlv_enable_pll(struct intel_crtc *crtc,
+ const struct intel_crtc_state *pipe_config)
+{
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ enum pipe pipe = crtc->pipe;
+
+ I915_WRITE(DPLL(pipe), pipe_config->dpll_hw_state.dpll);
+ POSTING_READ(DPLL(pipe));
+ udelay(150);
+
+ if (wait_for(((I915_READ(DPLL(pipe)) & DPLL_LOCK_VLV) == DPLL_LOCK_VLV), 1))
+ DRM_ERROR("DPLL %d failed to lock\n", pipe);
+}
+
static void vlv_enable_pll(struct intel_crtc *crtc,
const struct intel_crtc_state *pipe_config)
{
- struct drm_device *dev = crtc->base.dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
- i915_reg_t reg = DPLL(crtc->pipe);
- u32 dpll = pipe_config->dpll_hw_state.dpll;
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ enum pipe pipe = crtc->pipe;
- assert_pipe_disabled(dev_priv, crtc->pipe);
+ assert_pipe_disabled(dev_priv, pipe);
/* PLL is protected by panel, make sure we can write it */
- if (IS_MOBILE(dev_priv->dev))
- assert_panel_unlocked(dev_priv, crtc->pipe);
+ assert_panel_unlocked(dev_priv, pipe);
- I915_WRITE(reg, dpll);
- POSTING_READ(reg);
- udelay(150);
+ if (pipe_config->dpll_hw_state.dpll & DPLL_VCO_ENABLE)
+ _vlv_enable_pll(crtc, pipe_config);
- if (wait_for(((I915_READ(reg) & DPLL_LOCK_VLV) == DPLL_LOCK_VLV), 1))
- DRM_ERROR("DPLL %d failed to lock\n", crtc->pipe);
-
- I915_WRITE(DPLL_MD(crtc->pipe), pipe_config->dpll_hw_state.dpll_md);
- POSTING_READ(DPLL_MD(crtc->pipe));
-
- /* We do this three times for luck */
- I915_WRITE(reg, dpll);
- POSTING_READ(reg);
- udelay(150); /* wait for warmup */
- I915_WRITE(reg, dpll);
- POSTING_READ(reg);
- udelay(150); /* wait for warmup */
- I915_WRITE(reg, dpll);
- POSTING_READ(reg);
- udelay(150); /* wait for warmup */
+ I915_WRITE(DPLL_MD(pipe), pipe_config->dpll_hw_state.dpll_md);
+ POSTING_READ(DPLL_MD(pipe));
}
-static void chv_enable_pll(struct intel_crtc *crtc,
- const struct intel_crtc_state *pipe_config)
+
+static void _chv_enable_pll(struct intel_crtc *crtc,
+ const struct intel_crtc_state *pipe_config)
{
- struct drm_device *dev = crtc->base.dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
- int pipe = crtc->pipe;
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ enum pipe pipe = crtc->pipe;
enum dpio_channel port = vlv_pipe_to_channel(pipe);
u32 tmp;
- assert_pipe_disabled(dev_priv, crtc->pipe);
-
mutex_lock(&dev_priv->sb_lock);
/* Enable back the 10bit clock to display controller */
@@ -1660,10 +1591,43 @@
/* Check PLL is locked */
if (wait_for(((I915_READ(DPLL(pipe)) & DPLL_LOCK_VLV) == DPLL_LOCK_VLV), 1))
DRM_ERROR("PLL %d failed to lock\n", pipe);
+}
- /* not sure when this should be written */
- I915_WRITE(DPLL_MD(pipe), pipe_config->dpll_hw_state.dpll_md);
- POSTING_READ(DPLL_MD(pipe));
+static void chv_enable_pll(struct intel_crtc *crtc,
+ const struct intel_crtc_state *pipe_config)
+{
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ enum pipe pipe = crtc->pipe;
+
+ assert_pipe_disabled(dev_priv, pipe);
+
+ /* PLL is protected by panel, make sure we can write it */
+ assert_panel_unlocked(dev_priv, pipe);
+
+ if (pipe_config->dpll_hw_state.dpll & DPLL_VCO_ENABLE)
+ _chv_enable_pll(crtc, pipe_config);
+
+ if (pipe != PIPE_A) {
+ /*
+ * WaPixelRepeatModeFixForC0:chv
+ *
+ * DPLLCMD is AWOL. Use chicken bits to propagate
+ * the value from DPLLBMD to either pipe B or C.
+ */
+ I915_WRITE(CBR4_VLV, pipe == PIPE_B ? CBR_DPLLBMD_PIPE_B : CBR_DPLLBMD_PIPE_C);
+ I915_WRITE(DPLL_MD(PIPE_B), pipe_config->dpll_hw_state.dpll_md);
+ I915_WRITE(CBR4_VLV, 0);
+ dev_priv->chv_dpll_md[pipe] = pipe_config->dpll_hw_state.dpll_md;
+
+ /*
+ * DPLLB VGA mode also seems to cause problems.
+ * We should always have it disabled.
+ */
+ WARN_ON((I915_READ(DPLL(PIPE_B)) & DPLL_VGA_MODE_DIS) == 0);
+ } else {
+ I915_WRITE(DPLL_MD(pipe), pipe_config->dpll_hw_state.dpll_md);
+ POSTING_READ(DPLL_MD(pipe));
+ }
}
static int intel_num_dvo_pipes(struct drm_device *dev)
@@ -1687,9 +1651,6 @@
assert_pipe_disabled(dev_priv, crtc->pipe);
- /* No really, not for ILK+ */
- BUG_ON(INTEL_INFO(dev)->gen >= 5);
-
/* PLL is protected by panel, make sure we can write it */
if (IS_MOBILE(dev) && !IS_I830(dev))
assert_panel_unlocked(dev_priv, crtc->pipe);
@@ -1788,16 +1749,13 @@
/* Make sure the pipe isn't still relying on us */
assert_pipe_disabled(dev_priv, pipe);
- /*
- * Leave integrated clock source and reference clock enabled for pipe B.
- * The latter is needed for VGA hotplug / manual detection.
- */
- val = DPLL_VGA_MODE_DIS;
- if (pipe == PIPE_B)
- val = DPLL_INTEGRATED_CRI_CLK_VLV | DPLL_REF_CLK_ENABLE_VLV;
+ val = DPLL_INTEGRATED_REF_CLK_VLV |
+ DPLL_REF_CLK_ENABLE_VLV | DPLL_VGA_MODE_DIS;
+ if (pipe != PIPE_A)
+ val |= DPLL_INTEGRATED_CRI_CLK_VLV;
+
I915_WRITE(DPLL(pipe), val);
POSTING_READ(DPLL(pipe));
-
}
static void chv_disable_pll(struct drm_i915_private *dev_priv, enum pipe pipe)
@@ -1808,11 +1766,11 @@
/* Make sure the pipe isn't still relying on us */
assert_pipe_disabled(dev_priv, pipe);
- /* Set PLL en = 0 */
val = DPLL_SSC_REF_CLK_CHV |
DPLL_REF_CLK_ENABLE_VLV | DPLL_VGA_MODE_DIS;
if (pipe != PIPE_A)
val |= DPLL_INTEGRATED_CRI_CLK_VLV;
+
I915_WRITE(DPLL(pipe), val);
POSTING_READ(DPLL(pipe));
@@ -1856,100 +1814,6 @@
port_name(dport->port), I915_READ(dpll_reg) & port_mask, expected_mask);
}
-static void intel_prepare_shared_dpll(struct intel_crtc *crtc)
-{
- struct drm_device *dev = crtc->base.dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_shared_dpll *pll = intel_crtc_to_shared_dpll(crtc);
-
- if (WARN_ON(pll == NULL))
- return;
-
- WARN_ON(!pll->config.crtc_mask);
- if (pll->active == 0) {
- DRM_DEBUG_DRIVER("setting up %s\n", pll->name);
- WARN_ON(pll->on);
- assert_shared_dpll_disabled(dev_priv, pll);
-
- pll->mode_set(dev_priv, pll);
- }
-}
-
-/**
- * intel_enable_shared_dpll - enable PCH PLL
- * @dev_priv: i915 private structure
- * @pipe: pipe PLL to enable
- *
- * The PCH PLL needs to be enabled before the PCH transcoder, since it
- * drives the transcoder clock.
- */
-static void intel_enable_shared_dpll(struct intel_crtc *crtc)
-{
- struct drm_device *dev = crtc->base.dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_shared_dpll *pll = intel_crtc_to_shared_dpll(crtc);
-
- if (WARN_ON(pll == NULL))
- return;
-
- if (WARN_ON(pll->config.crtc_mask == 0))
- return;
-
- DRM_DEBUG_KMS("enable %s (active %d, on? %d) for crtc %d\n",
- pll->name, pll->active, pll->on,
- crtc->base.base.id);
-
- if (pll->active++) {
- WARN_ON(!pll->on);
- assert_shared_dpll_enabled(dev_priv, pll);
- return;
- }
- WARN_ON(pll->on);
-
- intel_display_power_get(dev_priv, POWER_DOMAIN_PLLS);
-
- DRM_DEBUG_KMS("enabling %s\n", pll->name);
- pll->enable(dev_priv, pll);
- pll->on = true;
-}
-
-static void intel_disable_shared_dpll(struct intel_crtc *crtc)
-{
- struct drm_device *dev = crtc->base.dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_shared_dpll *pll = intel_crtc_to_shared_dpll(crtc);
-
- /* PCH only available on ILK+ */
- if (INTEL_INFO(dev)->gen < 5)
- return;
-
- if (pll == NULL)
- return;
-
- if (WARN_ON(!(pll->config.crtc_mask & (1 << drm_crtc_index(&crtc->base)))))
- return;
-
- DRM_DEBUG_KMS("disable %s (active %d, on? %d) for crtc %d\n",
- pll->name, pll->active, pll->on,
- crtc->base.base.id);
-
- if (WARN_ON(pll->active == 0)) {
- assert_shared_dpll_disabled(dev_priv, pll);
- return;
- }
-
- assert_shared_dpll_enabled(dev_priv, pll);
- WARN_ON(!pll->on);
- if (--pll->active)
- return;
-
- DRM_DEBUG_KMS("disabling %s\n", pll->name);
- pll->disable(dev_priv, pll);
- pll->on = false;
-
- intel_display_power_put(dev_priv, POWER_DOMAIN_PLLS);
-}
-
static void ironlake_enable_pch_transcoder(struct drm_i915_private *dev_priv,
enum pipe pipe)
{
@@ -1959,12 +1823,8 @@
i915_reg_t reg;
uint32_t val, pipeconf_val;
- /* PCH only available on ILK+ */
- BUG_ON(!HAS_PCH_SPLIT(dev));
-
/* Make sure PCH DPLL is enabled */
- assert_shared_dpll_enabled(dev_priv,
- intel_crtc_to_shared_dpll(intel_crtc));
+ assert_shared_dpll_enabled(dev_priv, intel_crtc->config->shared_dpll);
/* FDI must be feeding us bits for PCH ports */
assert_fdi_tx_enabled(dev_priv, pipe);
@@ -1983,7 +1843,7 @@
val = I915_READ(reg);
pipeconf_val = I915_READ(PIPECONF(pipe));
- if (HAS_PCH_IBX(dev_priv->dev)) {
+ if (HAS_PCH_IBX(dev_priv)) {
/*
* Make the BPC in transcoder be consistent with
* that in pipeconf reg. For HDMI we must use 8bpc
@@ -1998,7 +1858,7 @@
val &= ~TRANS_INTERLACE_MASK;
if ((pipeconf_val & PIPECONF_INTERLACE_MASK) == PIPECONF_INTERLACED_ILK)
- if (HAS_PCH_IBX(dev_priv->dev) &&
+ if (HAS_PCH_IBX(dev_priv) &&
intel_pipe_has_type(intel_crtc, INTEL_OUTPUT_SDVO))
val |= TRANS_LEGACY_INTERLACED_ILK;
else
@@ -2016,9 +1876,6 @@
{
u32 val, pipeconf_val;
- /* PCH only available on ILK+ */
- BUG_ON(!HAS_PCH_SPLIT(dev_priv->dev));
-
/* FDI must be feeding us bits for PCH ports */
assert_fdi_tx_enabled(dev_priv, (enum pipe) cpu_transcoder);
assert_fdi_rx_enabled(dev_priv, TRANSCODER_A);
@@ -2113,7 +1970,7 @@
assert_cursor_disabled(dev_priv, pipe);
assert_sprites_disabled(dev_priv, pipe);
- if (HAS_PCH_LPT(dev_priv->dev))
+ if (HAS_PCH_LPT(dev_priv))
pch_transcoder = TRANSCODER_A;
else
pch_transcoder = pipe;
@@ -2123,7 +1980,7 @@
* a plane. On ILK+ the pipe PLLs are integrated, so we don't
* need the check.
*/
- if (HAS_GMCH_DISPLAY(dev_priv->dev))
+ if (HAS_GMCH_DISPLAY(dev_priv))
if (crtc->config->has_dsi_encoder)
assert_dsi_pll_enabled(dev_priv);
else
@@ -2225,8 +2082,8 @@
return IS_GEN2(dev_priv) ? 2048 : 4096;
}
-static unsigned int intel_tile_width(const struct drm_i915_private *dev_priv,
- uint64_t fb_modifier, unsigned int cpp)
+static unsigned int intel_tile_width_bytes(const struct drm_i915_private *dev_priv,
+ uint64_t fb_modifier, unsigned int cpp)
{
switch (fb_modifier) {
case DRM_FORMAT_MOD_NONE:
@@ -2269,7 +2126,21 @@
return 1;
else
return intel_tile_size(dev_priv) /
- intel_tile_width(dev_priv, fb_modifier, cpp);
+ intel_tile_width_bytes(dev_priv, fb_modifier, cpp);
+}
+
+/* Return the tile dimensions in pixel units */
+static void intel_tile_dims(const struct drm_i915_private *dev_priv,
+ unsigned int *tile_width,
+ unsigned int *tile_height,
+ uint64_t fb_modifier,
+ unsigned int cpp)
+{
+ unsigned int tile_width_bytes =
+ intel_tile_width_bytes(dev_priv, fb_modifier, cpp);
+
+ *tile_width = tile_width_bytes / cpp;
+ *tile_height = intel_tile_size(dev_priv) / tile_width_bytes;
}
unsigned int
@@ -2282,48 +2153,54 @@
return ALIGN(height, tile_height);
}
-static void
-intel_fill_fb_ggtt_view(struct i915_ggtt_view *view, struct drm_framebuffer *fb,
- const struct drm_plane_state *plane_state)
+unsigned int intel_rotation_info_size(const struct intel_rotation_info *rot_info)
{
- struct drm_i915_private *dev_priv = to_i915(fb->dev);
- struct intel_rotation_info *info = &view->params.rotated;
+ unsigned int size = 0;
+ int i;
+
+ for (i = 0 ; i < ARRAY_SIZE(rot_info->plane); i++)
+ size += rot_info->plane[i].width * rot_info->plane[i].height;
+
+ return size;
+}
+
+static void
+intel_fill_fb_ggtt_view(struct i915_ggtt_view *view,
+ const struct drm_framebuffer *fb,
+ unsigned int rotation)
+{
+ if (intel_rotation_90_or_270(rotation)) {
+ *view = i915_ggtt_view_rotated;
+ view->params.rotated = to_intel_framebuffer(fb)->rot_info;
+ } else {
+ *view = i915_ggtt_view_normal;
+ }
+}
+
+static void
+intel_fill_fb_info(struct drm_i915_private *dev_priv,
+ struct drm_framebuffer *fb)
+{
+ struct intel_rotation_info *info = &to_intel_framebuffer(fb)->rot_info;
unsigned int tile_size, tile_width, tile_height, cpp;
- *view = i915_ggtt_view_normal;
-
- if (!plane_state)
- return;
-
- if (!intel_rotation_90_or_270(plane_state->rotation))
- return;
-
- *view = i915_ggtt_view_rotated;
-
- info->height = fb->height;
- info->pixel_format = fb->pixel_format;
- info->pitch = fb->pitches[0];
- info->uv_offset = fb->offsets[1];
- info->fb_modifier = fb->modifier[0];
-
tile_size = intel_tile_size(dev_priv);
cpp = drm_format_plane_cpp(fb->pixel_format, 0);
- tile_width = intel_tile_width(dev_priv, fb->modifier[0], cpp);
- tile_height = tile_size / tile_width;
+ intel_tile_dims(dev_priv, &tile_width, &tile_height,
+ fb->modifier[0], cpp);
- info->width_pages = DIV_ROUND_UP(fb->pitches[0], tile_width);
- info->height_pages = DIV_ROUND_UP(fb->height, tile_height);
- info->size = info->width_pages * info->height_pages * tile_size;
+ info->plane[0].width = DIV_ROUND_UP(fb->pitches[0], tile_width * cpp);
+ info->plane[0].height = DIV_ROUND_UP(fb->height, tile_height);
if (info->pixel_format == DRM_FORMAT_NV12) {
cpp = drm_format_plane_cpp(fb->pixel_format, 1);
- tile_width = intel_tile_width(dev_priv, fb->modifier[1], cpp);
- tile_height = tile_size / tile_width;
+ intel_tile_dims(dev_priv, &tile_width, &tile_height,
+ fb->modifier[1], cpp);
- info->width_pages_uv = DIV_ROUND_UP(fb->pitches[1], tile_width);
- info->height_pages_uv = DIV_ROUND_UP(fb->height / 2, tile_height);
- info->size_uv = info->width_pages_uv * info->height_pages_uv * tile_size;
+ info->uv_offset = fb->offsets[1];
+ info->plane[1].width = DIV_ROUND_UP(fb->pitches[1], tile_width * cpp);
+ info->plane[1].height = DIV_ROUND_UP(fb->height / 2, tile_height);
}
}
@@ -2360,9 +2237,8 @@
}
int
-intel_pin_and_fence_fb_obj(struct drm_plane *plane,
- struct drm_framebuffer *fb,
- const struct drm_plane_state *plane_state)
+intel_pin_and_fence_fb_obj(struct drm_framebuffer *fb,
+ unsigned int rotation)
{
struct drm_device *dev = fb->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
@@ -2375,7 +2251,7 @@
alignment = intel_surf_alignment(dev_priv, fb->modifier[0]);
- intel_fill_fb_ggtt_view(&view, fb, plane_state);
+ intel_fill_fb_ggtt_view(&view, fb, rotation);
/* Note that the w/a also requires 64 PTE of padding following the
* bo. We currently fill all unused PTE with the shadow page and so
@@ -2433,15 +2309,14 @@
return ret;
}
-static void intel_unpin_fb_obj(struct drm_framebuffer *fb,
- const struct drm_plane_state *plane_state)
+static void intel_unpin_fb_obj(struct drm_framebuffer *fb, unsigned int rotation)
{
struct drm_i915_gem_object *obj = intel_fb_obj(fb);
struct i915_ggtt_view view;
WARN_ON(!mutex_is_locked(&obj->base.dev->struct_mutex));
- intel_fill_fb_ggtt_view(&view, fb, plane_state);
+ intel_fill_fb_ggtt_view(&view, fb, rotation);
if (view.type == I915_GGTT_VIEW_NORMAL)
i915_gem_object_unpin_fence(obj);
@@ -2449,38 +2324,93 @@
i915_gem_object_unpin_from_display_plane(obj, &view);
}
-/* Computes the linear offset to the base tile and adjusts x, y. bytes per pixel
- * is assumed to be a power-of-two. */
-u32 intel_compute_tile_offset(struct drm_i915_private *dev_priv,
- int *x, int *y,
- uint64_t fb_modifier,
- unsigned int cpp,
- unsigned int pitch)
+/*
+ * Adjust the tile offset by moving the difference into
+ * the x/y offsets.
+ *
+ * Input tile dimensions and pitch must already be
+ * rotated to match x and y, and in pixel units.
+ */
+static u32 intel_adjust_tile_offset(int *x, int *y,
+ unsigned int tile_width,
+ unsigned int tile_height,
+ unsigned int tile_size,
+ unsigned int pitch_tiles,
+ u32 old_offset,
+ u32 new_offset)
{
+ unsigned int tiles;
+
+ WARN_ON(old_offset & (tile_size - 1));
+ WARN_ON(new_offset & (tile_size - 1));
+ WARN_ON(new_offset > old_offset);
+
+ tiles = (old_offset - new_offset) / tile_size;
+
+ *y += tiles / pitch_tiles * tile_height;
+ *x += tiles % pitch_tiles * tile_width;
+
+ return new_offset;
+}
+
+/*
+ * Computes the linear offset to the base tile and adjusts
+ * x, y. bytes per pixel is assumed to be a power-of-two.
+ *
+ * In the 90/270 rotated case, x and y are assumed
+ * to be already rotated to match the rotated GTT view, and
+ * pitch is the tile_height aligned framebuffer height.
+ */
+u32 intel_compute_tile_offset(int *x, int *y,
+ const struct drm_framebuffer *fb, int plane,
+ unsigned int pitch,
+ unsigned int rotation)
+{
+ const struct drm_i915_private *dev_priv = to_i915(fb->dev);
+ uint64_t fb_modifier = fb->modifier[plane];
+ unsigned int cpp = drm_format_plane_cpp(fb->pixel_format, plane);
+ u32 offset, offset_aligned, alignment;
+
+ alignment = intel_surf_alignment(dev_priv, fb_modifier);
+ if (alignment)
+ alignment--;
+
if (fb_modifier != DRM_FORMAT_MOD_NONE) {
unsigned int tile_size, tile_width, tile_height;
- unsigned int tile_rows, tiles;
+ unsigned int tile_rows, tiles, pitch_tiles;
tile_size = intel_tile_size(dev_priv);
- tile_width = intel_tile_width(dev_priv, fb_modifier, cpp);
- tile_height = tile_size / tile_width;
+ intel_tile_dims(dev_priv, &tile_width, &tile_height,
+ fb_modifier, cpp);
+
+ if (intel_rotation_90_or_270(rotation)) {
+ pitch_tiles = pitch / tile_height;
+ swap(tile_width, tile_height);
+ } else {
+ pitch_tiles = pitch / (tile_width * cpp);
+ }
tile_rows = *y / tile_height;
*y %= tile_height;
- tiles = *x / (tile_width/cpp);
- *x %= tile_width/cpp;
+ tiles = *x / tile_width;
+ *x %= tile_width;
- return tile_rows * pitch * tile_height + tiles * tile_size;
+ offset = (tile_rows * pitch_tiles + tiles) * tile_size;
+ offset_aligned = offset & ~alignment;
+
+ intel_adjust_tile_offset(x, y, tile_width, tile_height,
+ tile_size, pitch_tiles,
+ offset, offset_aligned);
} else {
- unsigned int alignment = intel_linear_alignment(dev_priv) - 1;
- unsigned int offset;
-
offset = *y * pitch + *x * cpp;
+ offset_aligned = offset & ~alignment;
+
*y = (offset & alignment) / pitch;
*x = ((offset & alignment) - *y * pitch) / cpp;
- return offset & ~alignment;
}
+
+ return offset_aligned;
}
static int i9xx_format_to_fourcc(int format)
@@ -2536,6 +2466,7 @@
{
struct drm_device *dev = crtc->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct drm_i915_gem_object *obj = NULL;
struct drm_mode_fb_cmd2 mode_cmd = { 0 };
struct drm_framebuffer *fb = &plane_config->fb->base;
@@ -2551,7 +2482,7 @@
/* If the FB is too big, just don't use it since fbdev is not very
* important and we should probably use that space with FBC or other
* features. */
- if (size_aligned * 2 > dev_priv->gtt.stolen_usable_size)
+ if (size_aligned * 2 > ggtt->stolen_usable_size)
return false;
mutex_lock(&dev->struct_mutex);
@@ -2667,7 +2598,7 @@
*/
to_intel_plane_state(plane_state)->visible = false;
crtc_state->plane_mask &= ~(1 << drm_plane_index(primary));
- intel_pre_disable_primary(&intel_crtc->base);
+ intel_pre_disable_primary_noatomic(&intel_crtc->base);
intel_plane->disable_plane(primary, &intel_crtc->base);
return;
@@ -2716,6 +2647,7 @@
u32 linear_offset;
u32 dspcntr;
i915_reg_t reg = DSPCNTR(plane);
+ unsigned int rotation = plane_state->base.rotation;
int cpp = drm_format_plane_cpp(fb->pixel_format, 0);
int x = plane_state->src.x1 >> 16;
int y = plane_state->src.y1 >> 16;
@@ -2780,15 +2712,14 @@
if (INTEL_INFO(dev)->gen >= 4) {
intel_crtc->dspaddr_offset =
- intel_compute_tile_offset(dev_priv, &x, &y,
- fb->modifier[0], cpp,
- fb->pitches[0]);
+ intel_compute_tile_offset(&x, &y, fb, 0,
+ fb->pitches[0], rotation);
linear_offset -= intel_crtc->dspaddr_offset;
} else {
intel_crtc->dspaddr_offset = linear_offset;
}
- if (plane_state->base.rotation == BIT(DRM_ROTATE_180)) {
+ if (rotation == BIT(DRM_ROTATE_180)) {
dspcntr |= DISPPLANE_ROTATE_180;
x += (crtc_state->pipe_src_w - 1);
@@ -2846,6 +2777,7 @@
u32 linear_offset;
u32 dspcntr;
i915_reg_t reg = DSPCNTR(plane);
+ unsigned int rotation = plane_state->base.rotation;
int cpp = drm_format_plane_cpp(fb->pixel_format, 0);
int x = plane_state->src.x1 >> 16;
int y = plane_state->src.y1 >> 16;
@@ -2887,11 +2819,10 @@
linear_offset = y * fb->pitches[0] + x * cpp;
intel_crtc->dspaddr_offset =
- intel_compute_tile_offset(dev_priv, &x, &y,
- fb->modifier[0], cpp,
- fb->pitches[0]);
+ intel_compute_tile_offset(&x, &y, fb, 0,
+ fb->pitches[0], rotation);
linear_offset -= intel_crtc->dspaddr_offset;
- if (plane_state->base.rotation == BIT(DRM_ROTATE_180)) {
+ if (rotation == BIT(DRM_ROTATE_180)) {
dspcntr |= DISPPLANE_ROTATE_180;
if (!IS_HASWELL(dev) && !IS_BROADWELL(dev)) {
@@ -2931,7 +2862,7 @@
} else {
int cpp = drm_format_plane_cpp(pixel_format, 0);
- return intel_tile_width(dev_priv, fb_modifier, cpp);
+ return intel_tile_width_bytes(dev_priv, fb_modifier, cpp);
}
}
@@ -2944,7 +2875,7 @@
u64 offset;
intel_fill_fb_ggtt_view(&view, intel_plane->base.state->fb,
- intel_plane->base.state);
+ intel_plane->base.state->rotation);
vma = i915_gem_obj_to_ggtt_view(obj, &view);
if (WARN(!vma, "ggtt vma for display object not found! (view=%u)\n",
@@ -3284,12 +3215,12 @@
static bool intel_crtc_has_pending_flip(struct drm_crtc *crtc)
{
struct drm_device *dev = crtc->dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
+ unsigned reset_counter;
bool pending;
- if (i915_reset_in_progress(&dev_priv->gpu_error) ||
- intel_crtc->reset_counter != atomic_read(&dev_priv->gpu_error.reset_counter))
+ reset_counter = i915_reset_counter(&to_i915(dev)->gpu_error);
+ if (intel_crtc->reset_counter != reset_counter)
return false;
spin_lock_irq(&dev->event_lock);
@@ -3314,9 +3245,6 @@
old_crtc_state->pipe_src_w, old_crtc_state->pipe_src_h,
pipe_config->pipe_src_w, pipe_config->pipe_src_h);
- if (HAS_DDI(dev))
- intel_set_pipe_csc(&crtc->base);
-
/*
* Update pipe size and adjust fitter if needed: the reason for this is
* that in compute_mode_changes we check the native mode (not the pfit
@@ -3894,9 +3822,7 @@
intel_crtc->unpin_work = NULL;
if (work->event)
- drm_send_vblank_event(intel_crtc->base.dev,
- intel_crtc->pipe,
- work->event);
+ drm_crtc_send_vblank_event(&intel_crtc->base, work->event);
drm_crtc_vblank_put(&intel_crtc->base);
@@ -3955,37 +3881,35 @@
/* Program iCLKIP clock to the desired frequency */
static void lpt_program_iclkip(struct drm_crtc *crtc)
{
- struct drm_device *dev = crtc->dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(crtc->dev);
int clock = to_intel_crtc(crtc)->config->base.adjusted_mode.crtc_clock;
u32 divsel, phaseinc, auxdiv, phasedir = 0;
u32 temp;
lpt_disable_iclkip(dev_priv);
- /* 20MHz is a corner case which is out of range for the 7-bit divisor */
- if (clock == 20000) {
- auxdiv = 1;
- divsel = 0x41;
- phaseinc = 0x20;
- } else {
- /* The iCLK virtual clock root frequency is in MHz,
- * but the adjusted_mode->crtc_clock in in KHz. To get the
- * divisors, it is necessary to divide one by another, so we
- * convert the virtual clock precision to KHz here for higher
- * precision.
- */
+ /* The iCLK virtual clock root frequency is in MHz,
+ * but the adjusted_mode->crtc_clock in in KHz. To get the
+ * divisors, it is necessary to divide one by another, so we
+ * convert the virtual clock precision to KHz here for higher
+ * precision.
+ */
+ for (auxdiv = 0; auxdiv < 2; auxdiv++) {
u32 iclk_virtual_root_freq = 172800 * 1000;
u32 iclk_pi_range = 64;
- u32 desired_divisor, msb_divisor_value, pi_value;
+ u32 desired_divisor;
- desired_divisor = DIV_ROUND_CLOSEST(iclk_virtual_root_freq, clock);
- msb_divisor_value = desired_divisor / iclk_pi_range;
- pi_value = desired_divisor % iclk_pi_range;
+ desired_divisor = DIV_ROUND_CLOSEST(iclk_virtual_root_freq,
+ clock << auxdiv);
+ divsel = (desired_divisor / iclk_pi_range) - 2;
+ phaseinc = desired_divisor % iclk_pi_range;
- auxdiv = 0;
- divsel = msb_divisor_value - 2;
- phaseinc = pi_value;
+ /*
+ * Near 20MHz is a corner case which is
+ * out of range for the 7-bit divisor
+ */
+ if (divsel <= 0x7f)
+ break;
}
/* This should not happen with any sane values */
@@ -4032,6 +3956,43 @@
I915_WRITE(PIXCLK_GATE, PIXCLK_GATE_UNGATE);
}
+int lpt_get_iclkip(struct drm_i915_private *dev_priv)
+{
+ u32 divsel, phaseinc, auxdiv;
+ u32 iclk_virtual_root_freq = 172800 * 1000;
+ u32 iclk_pi_range = 64;
+ u32 desired_divisor;
+ u32 temp;
+
+ if ((I915_READ(PIXCLK_GATE) & PIXCLK_GATE_UNGATE) == 0)
+ return 0;
+
+ mutex_lock(&dev_priv->sb_lock);
+
+ temp = intel_sbi_read(dev_priv, SBI_SSCCTL6, SBI_ICLK);
+ if (temp & SBI_SSCCTL_DISABLE) {
+ mutex_unlock(&dev_priv->sb_lock);
+ return 0;
+ }
+
+ temp = intel_sbi_read(dev_priv, SBI_SSCDIVINTPHASE6, SBI_ICLK);
+ divsel = (temp & SBI_SSCDIVINTPHASE_DIVSEL_MASK) >>
+ SBI_SSCDIVINTPHASE_DIVSEL_SHIFT;
+ phaseinc = (temp & SBI_SSCDIVINTPHASE_INCVAL_MASK) >>
+ SBI_SSCDIVINTPHASE_INCVAL_SHIFT;
+
+ temp = intel_sbi_read(dev_priv, SBI_SSCAUXDIV6, SBI_ICLK);
+ auxdiv = (temp & SBI_SSCAUXDIV_FINALDIV2SEL_MASK) >>
+ SBI_SSCAUXDIV_FINALDIV2SEL_SHIFT;
+
+ mutex_unlock(&dev_priv->sb_lock);
+
+ desired_divisor = (divsel + 2) * iclk_pi_range + phaseinc;
+
+ return DIV_ROUND_CLOSEST(iclk_virtual_root_freq,
+ desired_divisor << auxdiv);
+}
+
static void ironlake_pch_transcoder_set_timings(struct intel_crtc *crtc,
enum pipe pch_transcoder)
{
@@ -4142,12 +4103,6 @@
I915_WRITE(FDI_RX_TUSIZE1(pipe),
I915_READ(PIPE_DATA_M1(pipe)) & TU_SIZE_MASK);
- /*
- * Sometimes spurious CPU pipe underruns happen during FDI
- * training, at least with VGA+HDMI cloning. Suppress them.
- */
- intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, false);
-
/* For PCH output, training FDI link */
dev_priv->display.fdi_link_train(crtc);
@@ -4159,7 +4114,8 @@
temp = I915_READ(PCH_DPLL_SEL);
temp |= TRANS_DPLL_ENABLE(pipe);
sel = TRANS_DPLLB_SEL(pipe);
- if (intel_crtc->config->shared_dpll == DPLL_ID_PCH_PLL_B)
+ if (intel_crtc->config->shared_dpll ==
+ intel_get_shared_dpll_by_id(dev_priv, DPLL_ID_PCH_PLL_B))
temp |= sel;
else
temp &= ~sel;
@@ -4181,8 +4137,6 @@
intel_fdi_normal_train(crtc);
- intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, true);
-
/* For PCH DP, enable TRANS_DP_CTL */
if (HAS_PCH_CPT(dev) && intel_crtc->config->has_dp_encoder) {
const struct drm_display_mode *adjusted_mode =
@@ -4238,113 +4192,6 @@
lpt_enable_pch_transcoder(dev_priv, cpu_transcoder);
}
-struct intel_shared_dpll *intel_get_shared_dpll(struct intel_crtc *crtc,
- struct intel_crtc_state *crtc_state)
-{
- struct drm_i915_private *dev_priv = crtc->base.dev->dev_private;
- struct intel_shared_dpll *pll;
- struct intel_shared_dpll_config *shared_dpll;
- enum intel_dpll_id i;
- int max = dev_priv->num_shared_dpll;
-
- shared_dpll = intel_atomic_get_shared_dpll_state(crtc_state->base.state);
-
- if (HAS_PCH_IBX(dev_priv->dev)) {
- /* Ironlake PCH has a fixed PLL->PCH pipe mapping. */
- i = (enum intel_dpll_id) crtc->pipe;
- pll = &dev_priv->shared_dplls[i];
-
- DRM_DEBUG_KMS("CRTC:%d using pre-allocated %s\n",
- crtc->base.base.id, pll->name);
-
- WARN_ON(shared_dpll[i].crtc_mask);
-
- goto found;
- }
-
- if (IS_BROXTON(dev_priv->dev)) {
- /* PLL is attached to port in bxt */
- struct intel_encoder *encoder;
- struct intel_digital_port *intel_dig_port;
-
- encoder = intel_ddi_get_crtc_new_encoder(crtc_state);
- if (WARN_ON(!encoder))
- return NULL;
-
- intel_dig_port = enc_to_dig_port(&encoder->base);
- /* 1:1 mapping between ports and PLLs */
- i = (enum intel_dpll_id)intel_dig_port->port;
- pll = &dev_priv->shared_dplls[i];
- DRM_DEBUG_KMS("CRTC:%d using pre-allocated %s\n",
- crtc->base.base.id, pll->name);
- WARN_ON(shared_dpll[i].crtc_mask);
-
- goto found;
- } else if (INTEL_INFO(dev_priv)->gen < 9 && HAS_DDI(dev_priv))
- /* Do not consider SPLL */
- max = 2;
-
- for (i = 0; i < max; i++) {
- pll = &dev_priv->shared_dplls[i];
-
- /* Only want to check enabled timings first */
- if (shared_dpll[i].crtc_mask == 0)
- continue;
-
- if (memcmp(&crtc_state->dpll_hw_state,
- &shared_dpll[i].hw_state,
- sizeof(crtc_state->dpll_hw_state)) == 0) {
- DRM_DEBUG_KMS("CRTC:%d sharing existing %s (crtc mask 0x%08x, ative %d)\n",
- crtc->base.base.id, pll->name,
- shared_dpll[i].crtc_mask,
- pll->active);
- goto found;
- }
- }
-
- /* Ok no matching timings, maybe there's a free one? */
- for (i = 0; i < dev_priv->num_shared_dpll; i++) {
- pll = &dev_priv->shared_dplls[i];
- if (shared_dpll[i].crtc_mask == 0) {
- DRM_DEBUG_KMS("CRTC:%d allocated %s\n",
- crtc->base.base.id, pll->name);
- goto found;
- }
- }
-
- return NULL;
-
-found:
- if (shared_dpll[i].crtc_mask == 0)
- shared_dpll[i].hw_state =
- crtc_state->dpll_hw_state;
-
- crtc_state->shared_dpll = i;
- DRM_DEBUG_DRIVER("using %s for pipe %c\n", pll->name,
- pipe_name(crtc->pipe));
-
- shared_dpll[i].crtc_mask |= 1 << crtc->pipe;
-
- return pll;
-}
-
-static void intel_shared_dpll_commit(struct drm_atomic_state *state)
-{
- struct drm_i915_private *dev_priv = to_i915(state->dev);
- struct intel_shared_dpll_config *shared_dpll;
- struct intel_shared_dpll *pll;
- enum intel_dpll_id i;
-
- if (!to_intel_atomic_state(state)->dpll_set)
- return;
-
- shared_dpll = to_intel_atomic_state(state)->shared_dpll;
- for (i = 0; i < dev_priv->num_shared_dpll; i++) {
- pll = &dev_priv->shared_dplls[i];
- pll->config = shared_dpll[i];
- }
-}
-
static void cpt_verify_modeset(struct drm_device *dev, int pipe)
{
struct drm_i915_private *dev_priv = dev->dev_private;
@@ -4576,8 +4423,11 @@
if (!crtc->config->ips_enabled)
return;
- /* We can only enable IPS after we enable a plane and wait for a vblank */
- intel_wait_for_vblank(dev, crtc->pipe);
+ /*
+ * We can only enable IPS after we enable a plane and wait for a vblank
+ * This function is called from post_plane_update, which is run after
+ * a vblank wait.
+ */
assert_plane_enabled(dev_priv, crtc->plane);
if (IS_BROADWELL(dev)) {
@@ -4626,55 +4476,6 @@
intel_wait_for_vblank(dev, crtc->pipe);
}
-/** Loads the palette/gamma unit for the CRTC with the prepared values */
-static void intel_crtc_load_lut(struct drm_crtc *crtc)
-{
- struct drm_device *dev = crtc->dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- enum pipe pipe = intel_crtc->pipe;
- int i;
- bool reenable_ips = false;
-
- /* The clocks have to be on to load the palette. */
- if (!crtc->state->active)
- return;
-
- if (HAS_GMCH_DISPLAY(dev_priv->dev)) {
- if (intel_crtc->config->has_dsi_encoder)
- assert_dsi_pll_enabled(dev_priv);
- else
- assert_pll_enabled(dev_priv, pipe);
- }
-
- /* Workaround : Do not read or write the pipe palette/gamma data while
- * GAMMA_MODE is configured for split gamma and IPS_CTL has IPS enabled.
- */
- if (IS_HASWELL(dev) && intel_crtc->config->ips_enabled &&
- ((I915_READ(GAMMA_MODE(pipe)) & GAMMA_MODE_MODE_MASK) ==
- GAMMA_MODE_MODE_SPLIT)) {
- hsw_disable_ips(intel_crtc);
- reenable_ips = true;
- }
-
- for (i = 0; i < 256; i++) {
- i915_reg_t palreg;
-
- if (HAS_GMCH_DISPLAY(dev))
- palreg = PALETTE(pipe, i);
- else
- palreg = LGC_PALETTE(pipe, i);
-
- I915_WRITE(palreg,
- (intel_crtc->lut_r[i] << 16) |
- (intel_crtc->lut_g[i] << 8) |
- intel_crtc->lut_b[i]);
- }
-
- if (reenable_ips)
- hsw_enable_ips(intel_crtc);
-}
-
static void intel_crtc_dpms_overlay_disable(struct intel_crtc *intel_crtc)
{
if (intel_crtc->overlay) {
@@ -4734,16 +4535,7 @@
intel_check_pch_fifo_underruns(dev_priv);
}
-/**
- * intel_pre_disable_primary - Perform operations before disabling primary plane
- * @crtc: the CRTC whose primary plane is to be disabled
- *
- * Performs potentially sleeping operations that must be done before the
- * primary plane is disabled, such as updating FBC and IPS. Note that this may
- * be called due to an explicit primary plane update, or due to an implicit
- * disable that is caused when a sprite plane completely hides the primary
- * plane.
- */
+/* FIXME move all this to pre_plane_update() with proper state tracking */
static void
intel_pre_disable_primary(struct drm_crtc *crtc)
{
@@ -4762,6 +4554,26 @@
intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, false);
/*
+ * FIXME IPS should be fine as long as one plane is
+ * enabled, but in practice it seems to have problems
+ * when going from primary only to sprite only and vice
+ * versa.
+ */
+ hsw_disable_ips(intel_crtc);
+}
+
+/* FIXME get rid of this and use pre_plane_update */
+static void
+intel_pre_disable_primary_noatomic(struct drm_crtc *crtc)
+{
+ struct drm_device *dev = crtc->dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
+ int pipe = intel_crtc->pipe;
+
+ intel_pre_disable_primary(crtc);
+
+ /*
* Vblank time updates from the shadow to live plane control register
* are blocked if the memory self-refresh mode is active at that
* moment. So to make sure the plane gets truly disabled, disable
@@ -4775,37 +4587,39 @@
dev_priv->wm.vlv.cxsr = false;
intel_wait_for_vblank(dev, pipe);
}
-
- /*
- * FIXME IPS should be fine as long as one plane is
- * enabled, but in practice it seems to have problems
- * when going from primary only to sprite only and vice
- * versa.
- */
- hsw_disable_ips(intel_crtc);
}
-static void intel_post_plane_update(struct intel_crtc *crtc)
+static void intel_post_plane_update(struct intel_crtc_state *old_crtc_state)
{
- struct intel_crtc_atomic_commit *atomic = &crtc->atomic;
+ struct intel_crtc *crtc = to_intel_crtc(old_crtc_state->base.crtc);
+ struct drm_atomic_state *old_state = old_crtc_state->base.state;
struct intel_crtc_state *pipe_config =
to_intel_crtc_state(crtc->base.state);
struct drm_device *dev = crtc->base.dev;
+ struct drm_plane *primary = crtc->base.primary;
+ struct drm_plane_state *old_pri_state =
+ drm_atomic_get_existing_plane_state(old_state, primary);
- intel_frontbuffer_flip(dev, atomic->fb_bits);
+ intel_frontbuffer_flip(dev, pipe_config->fb_bits);
crtc->wm.cxsr_allowed = true;
- if (pipe_config->wm_changed && pipe_config->base.active)
+ if (pipe_config->update_wm_post && pipe_config->base.active)
intel_update_watermarks(&crtc->base);
- if (atomic->update_fbc)
+ if (old_pri_state) {
+ struct intel_plane_state *primary_state =
+ to_intel_plane_state(primary->state);
+ struct intel_plane_state *old_primary_state =
+ to_intel_plane_state(old_pri_state);
+
intel_fbc_post_update(crtc);
- if (atomic->post_enable_primary)
- intel_post_enable_primary(&crtc->base);
-
- memset(atomic, 0, sizeof(*atomic));
+ if (primary_state->visible &&
+ (needs_modeset(&pipe_config->base) ||
+ !old_primary_state->visible))
+ intel_post_enable_primary(&crtc->base);
+ }
}
static void intel_pre_plane_update(struct intel_crtc_state *old_crtc_state)
@@ -4813,7 +4627,6 @@
struct intel_crtc *crtc = to_intel_crtc(old_crtc_state->base.crtc);
struct drm_device *dev = crtc->base.dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_crtc_atomic_commit *atomic = &crtc->atomic;
struct intel_crtc_state *pipe_config =
to_intel_crtc_state(crtc->base.state);
struct drm_atomic_state *old_state = old_crtc_state->base.state;
@@ -4822,15 +4635,14 @@
drm_atomic_get_existing_plane_state(old_state, primary);
bool modeset = needs_modeset(&pipe_config->base);
- if (atomic->update_fbc)
- intel_fbc_pre_update(crtc);
-
if (old_pri_state) {
struct intel_plane_state *primary_state =
to_intel_plane_state(primary->state);
struct intel_plane_state *old_primary_state =
to_intel_plane_state(old_pri_state);
+ intel_fbc_pre_update(crtc);
+
if (old_primary_state->visible &&
(modeset || !primary_state->visible))
intel_pre_disable_primary(&crtc->base);
@@ -4839,11 +4651,58 @@
if (pipe_config->disable_cxsr) {
crtc->wm.cxsr_allowed = false;
- if (old_crtc_state->base.active)
+ /*
+ * Vblank time updates from the shadow to live plane control register
+ * are blocked if the memory self-refresh mode is active at that
+ * moment. So to make sure the plane gets truly disabled, disable
+ * first the self-refresh mode. The self-refresh enable bit in turn
+ * will be checked/applied by the HW only at the next frame start
+ * event which is after the vblank start event, so we need to have a
+ * wait-for-vblank between disabling the plane and the pipe.
+ */
+ if (old_crtc_state->base.active) {
intel_set_memory_cxsr(dev_priv, false);
+ dev_priv->wm.vlv.cxsr = false;
+ intel_wait_for_vblank(dev, crtc->pipe);
+ }
}
- if (!needs_modeset(&pipe_config->base) && pipe_config->wm_changed)
+ /*
+ * IVB workaround: must disable low power watermarks for at least
+ * one frame before enabling scaling. LP watermarks can be re-enabled
+ * when scaling is disabled.
+ *
+ * WaCxSRDisabledForSpriteScaling:ivb
+ */
+ if (pipe_config->disable_lp_wm) {
+ ilk_disable_lp_wm(dev);
+ intel_wait_for_vblank(dev, crtc->pipe);
+ }
+
+ /*
+ * If we're doing a modeset, we're done. No need to do any pre-vblank
+ * watermark programming here.
+ */
+ if (needs_modeset(&pipe_config->base))
+ return;
+
+ /*
+ * For platforms that support atomic watermarks, program the
+ * 'intermediate' watermarks immediately. On pre-gen9 platforms, these
+ * will be the intermediate values that are safe for both pre- and
+ * post- vblank; when vblank happens, the 'active' values will be set
+ * to the final 'target' values and we'll do this again to get the
+ * optimal watermarks. For gen9+ platforms, the values we program here
+ * will be the final target values which will get automatically latched
+ * at vblank time; no further programming will be necessary.
+ *
+ * If a platform hasn't been transitioned to atomic watermarks yet,
+ * we'll continue to update watermarks the old way, if flags tell
+ * us to.
+ */
+ if (dev_priv->display.initial_watermarks != NULL)
+ dev_priv->display.initial_watermarks(pipe_config);
+ else if (pipe_config->update_wm_pre)
intel_update_watermarks(&crtc->base);
}
@@ -4874,10 +4733,24 @@
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
struct intel_encoder *encoder;
int pipe = intel_crtc->pipe;
+ struct intel_crtc_state *pipe_config =
+ to_intel_crtc_state(crtc->state);
if (WARN_ON(intel_crtc->active))
return;
+ /*
+ * Sometimes spurious CPU pipe underruns happen during FDI
+ * training, at least with VGA+HDMI cloning. Suppress them.
+ *
+ * On ILK we get an occasional spurious CPU pipe underruns
+ * between eDP port A enable and vdd enable. Also PCH port
+ * enable seems to result in the occasional CPU pipe underrun.
+ *
+ * Spurious PCH underruns also occur during PCH enabling.
+ */
+ if (intel_crtc->config->has_pch_encoder || IS_GEN5(dev_priv))
+ intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, false);
if (intel_crtc->config->has_pch_encoder)
intel_set_pch_fifo_underrun_reporting(dev_priv, pipe, false);
@@ -4888,6 +4761,7 @@
intel_dp_set_m_n(intel_crtc, M1_N1);
intel_set_pipe_timings(intel_crtc);
+ intel_set_pipe_src_size(intel_crtc);
if (intel_crtc->config->has_pch_encoder) {
intel_cpu_transcoder_set_m_n(intel_crtc,
@@ -4898,8 +4772,6 @@
intel_crtc->active = true;
- intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, true);
-
for_each_encoder_on_crtc(dev, crtc, encoder)
if (encoder->pre_enable)
encoder->pre_enable(encoder);
@@ -4920,9 +4792,10 @@
* On ILK+ LUT must be loaded before the pipe is running but with
* clocks enabled
*/
- intel_crtc_load_lut(crtc);
+ intel_color_load_luts(&pipe_config->base);
- intel_update_watermarks(crtc);
+ if (dev_priv->display.initial_watermarks != NULL)
+ dev_priv->display.initial_watermarks(intel_crtc->config);
intel_enable_pipe(intel_crtc);
if (intel_crtc->config->has_pch_encoder)
@@ -4940,6 +4813,7 @@
/* Must wait for vblank to avoid spurious PCH FIFO underruns */
if (intel_crtc->config->has_pch_encoder)
intel_wait_for_vblank(dev, pipe);
+ intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, true);
intel_set_pch_fifo_underrun_reporting(dev_priv, pipe, true);
}
@@ -4956,6 +4830,7 @@
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
struct intel_encoder *encoder;
int pipe = intel_crtc->pipe, hsw_workaround_pipe;
+ enum transcoder cpu_transcoder = intel_crtc->config->cpu_transcoder;
struct intel_crtc_state *pipe_config =
to_intel_crtc_state(crtc->state);
@@ -4966,16 +4841,20 @@
intel_set_pch_fifo_underrun_reporting(dev_priv, TRANSCODER_A,
false);
- if (intel_crtc_to_shared_dpll(intel_crtc))
+ if (intel_crtc->config->shared_dpll)
intel_enable_shared_dpll(intel_crtc);
if (intel_crtc->config->has_dp_encoder)
intel_dp_set_m_n(intel_crtc, M1_N1);
- intel_set_pipe_timings(intel_crtc);
+ if (!intel_crtc->config->has_dsi_encoder)
+ intel_set_pipe_timings(intel_crtc);
- if (intel_crtc->config->cpu_transcoder != TRANSCODER_EDP) {
- I915_WRITE(PIPE_MULT(intel_crtc->config->cpu_transcoder),
+ intel_set_pipe_src_size(intel_crtc);
+
+ if (cpu_transcoder != TRANSCODER_EDP &&
+ !transcoder_is_dsi(cpu_transcoder)) {
+ I915_WRITE(PIPE_MULT(cpu_transcoder),
intel_crtc->config->pixel_multiplier - 1);
}
@@ -4984,9 +4863,12 @@
&intel_crtc->config->fdi_m_n, NULL);
}
- haswell_set_pipeconf(crtc);
+ if (!intel_crtc->config->has_dsi_encoder)
+ haswell_set_pipeconf(crtc);
- intel_set_pipe_csc(crtc);
+ haswell_set_pipemisc(crtc);
+
+ intel_color_set_csc(&pipe_config->base);
intel_crtc->active = true;
@@ -5015,14 +4897,20 @@
* On ILK+ LUT must be loaded before the pipe is running but with
* clocks enabled
*/
- intel_crtc_load_lut(crtc);
+ intel_color_load_luts(&pipe_config->base);
intel_ddi_set_pipe_settings(crtc);
if (!intel_crtc->config->has_dsi_encoder)
intel_ddi_enable_transcoder_func(crtc);
- intel_update_watermarks(crtc);
- intel_enable_pipe(intel_crtc);
+ if (dev_priv->display.initial_watermarks != NULL)
+ dev_priv->display.initial_watermarks(pipe_config);
+ else
+ intel_update_watermarks(crtc);
+
+ /* XXX: Do the pipe assertions at the right place for BXT DSI. */
+ if (!intel_crtc->config->has_dsi_encoder)
+ intel_enable_pipe(intel_crtc);
if (intel_crtc->config->has_pch_encoder)
lpt_pch_enable(crtc);
@@ -5078,8 +4966,15 @@
struct intel_encoder *encoder;
int pipe = intel_crtc->pipe;
- if (intel_crtc->config->has_pch_encoder)
+ /*
+ * Sometimes spurious CPU pipe underruns happen when the
+ * pipe is already disabled, but FDI RX/TX is still enabled.
+ * Happens at least with VGA+HDMI cloning. Suppress them.
+ */
+ if (intel_crtc->config->has_pch_encoder) {
+ intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, false);
intel_set_pch_fifo_underrun_reporting(dev_priv, pipe, false);
+ }
for_each_encoder_on_crtc(dev, crtc, encoder)
encoder->disable(encoder);
@@ -5087,22 +4982,12 @@
drm_crtc_vblank_off(crtc);
assert_vblank_disabled(crtc);
- /*
- * Sometimes spurious CPU pipe underruns happen when the
- * pipe is already disabled, but FDI RX/TX is still enabled.
- * Happens at least with VGA+HDMI cloning. Suppress them.
- */
- if (intel_crtc->config->has_pch_encoder)
- intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, false);
-
intel_disable_pipe(intel_crtc);
ironlake_pfit_disable(intel_crtc, false);
- if (intel_crtc->config->has_pch_encoder) {
+ if (intel_crtc->config->has_pch_encoder)
ironlake_fdi_disable(crtc);
- intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, true);
- }
for_each_encoder_on_crtc(dev, crtc, encoder)
if (encoder->post_disable)
@@ -5132,6 +5017,7 @@
ironlake_fdi_pll_disable(intel_crtc);
}
+ intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, true);
intel_set_pch_fifo_underrun_reporting(dev_priv, pipe, true);
}
@@ -5155,7 +5041,9 @@
drm_crtc_vblank_off(crtc);
assert_vblank_disabled(crtc);
- intel_disable_pipe(intel_crtc);
+ /* XXX: Do the pipe assertions at the right place for BXT DSI. */
+ if (!intel_crtc->config->has_dsi_encoder)
+ intel_disable_pipe(intel_crtc);
if (intel_crtc->config->dp_encoder_is_mst)
intel_ddi_set_vc_payload_alloc(crtc, false);
@@ -5330,6 +5218,9 @@
mask |= BIT(intel_display_port_power_domain(intel_encoder));
}
+ if (crtc_state->shared_dpll)
+ mask |= BIT(POWER_DOMAIN_PLLS);
+
return mask;
}
@@ -5393,6 +5284,8 @@
dev_priv->max_cdclk_freq = 450000;
else
dev_priv->max_cdclk_freq = 337500;
+ } else if (IS_BROXTON(dev)) {
+ dev_priv->max_cdclk_freq = 624000;
} else if (IS_BROADWELL(dev)) {
/*
* FIXME with extra cooling we can allow
@@ -5452,9 +5345,8 @@
intel_update_max_cdclk(dev);
}
-static void broxton_set_cdclk(struct drm_device *dev, int frequency)
+static void broxton_set_cdclk(struct drm_i915_private *dev_priv, int frequency)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
uint32_t divider;
uint32_t ratio;
uint32_t current_freq;
@@ -5568,33 +5460,46 @@
return;
}
- intel_update_cdclk(dev);
+ intel_update_cdclk(dev_priv->dev);
}
-void broxton_init_cdclk(struct drm_device *dev)
+static bool broxton_cdclk_is_enabled(struct drm_i915_private *dev_priv)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
- uint32_t val;
+ if (!(I915_READ(BXT_DE_PLL_ENABLE) & BXT_DE_PLL_PLL_ENABLE))
+ return false;
- /*
- * NDE_RSTWRN_OPT RST PCH Handshake En must always be 0b on BXT
- * or else the reset will hang because there is no PCH to respond.
- * Move the handshake programming to initialization sequence.
- * Previously was left up to BIOS.
- */
- val = I915_READ(HSW_NDE_RSTWRN_OPT);
- val &= ~RESET_PCH_HANDSHAKE_ENABLE;
- I915_WRITE(HSW_NDE_RSTWRN_OPT, val);
+ /* TODO: Check for a valid CDCLK rate */
- /* Enable PG1 for cdclk */
- intel_display_power_get(dev_priv, POWER_DOMAIN_PLLS);
+ if (!(I915_READ(DBUF_CTL) & DBUF_POWER_REQUEST)) {
+ DRM_DEBUG_DRIVER("CDCLK enabled, but DBUF power not requested\n");
+ return false;
+ }
+
+ if (!(I915_READ(DBUF_CTL) & DBUF_POWER_STATE)) {
+ DRM_DEBUG_DRIVER("CDCLK enabled, but DBUF power hasn't settled\n");
+
+ return false;
+ }
+
+ return true;
+}
+
+bool broxton_cdclk_verify_state(struct drm_i915_private *dev_priv)
+{
+ return broxton_cdclk_is_enabled(dev_priv);
+}
+
+void broxton_init_cdclk(struct drm_i915_private *dev_priv)
+{
/* check if cd clock is enabled */
- if (I915_READ(BXT_DE_PLL_ENABLE) & BXT_DE_PLL_PLL_ENABLE) {
- DRM_DEBUG_KMS("Display already initialized\n");
+ if (broxton_cdclk_is_enabled(dev_priv)) {
+ DRM_DEBUG_KMS("CDCLK already enabled, won't reprogram it\n");
return;
}
+ DRM_DEBUG_KMS("CDCLK not enabled, enabling it\n");
+
/*
* FIXME:
* - The initial CDCLK needs to be read from VBT.
@@ -5602,7 +5507,7 @@
* - check if setting the max (or any) cdclk freq is really necessary
* here, it belongs to modeset time
*/
- broxton_set_cdclk(dev, 624000);
+ broxton_set_cdclk(dev_priv, 624000);
I915_WRITE(DBUF_CTL, I915_READ(DBUF_CTL) | DBUF_POWER_REQUEST);
POSTING_READ(DBUF_CTL);
@@ -5613,10 +5518,8 @@
DRM_ERROR("DBuf power enable timeout!\n");
}
-void broxton_uninit_cdclk(struct drm_device *dev)
+void broxton_uninit_cdclk(struct drm_i915_private *dev_priv)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
-
I915_WRITE(DBUF_CTL, I915_READ(DBUF_CTL) & ~DBUF_POWER_REQUEST);
POSTING_READ(DBUF_CTL);
@@ -5626,9 +5529,7 @@
DRM_ERROR("DBuf power disable timeout!\n");
/* Set minimum (bypass) frequency, in effect turning off the DE PLL */
- broxton_set_cdclk(dev, 19200);
-
- intel_display_power_put(dev_priv, POWER_DOMAIN_PLLS);
+ broxton_set_cdclk(dev_priv, 19200);
}
static const struct skl_cdclk_entry {
@@ -6165,6 +6066,8 @@
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
struct intel_encoder *encoder;
+ struct intel_crtc_state *pipe_config =
+ to_intel_crtc_state(crtc->state);
int pipe = intel_crtc->pipe;
if (WARN_ON(intel_crtc->active))
@@ -6174,6 +6077,7 @@
intel_dp_set_m_n(intel_crtc, M1_N1);
intel_set_pipe_timings(intel_crtc);
+ intel_set_pipe_src_size(intel_crtc);
if (IS_CHERRYVIEW(dev) && pipe == PIPE_B) {
struct drm_i915_private *dev_priv = dev->dev_private;
@@ -6192,14 +6096,12 @@
if (encoder->pre_pll_enable)
encoder->pre_pll_enable(encoder);
- if (!intel_crtc->config->has_dsi_encoder) {
- if (IS_CHERRYVIEW(dev)) {
- chv_prepare_pll(intel_crtc, intel_crtc->config);
- chv_enable_pll(intel_crtc, intel_crtc->config);
- } else {
- vlv_prepare_pll(intel_crtc, intel_crtc->config);
- vlv_enable_pll(intel_crtc, intel_crtc->config);
- }
+ if (IS_CHERRYVIEW(dev)) {
+ chv_prepare_pll(intel_crtc, intel_crtc->config);
+ chv_enable_pll(intel_crtc, intel_crtc->config);
+ } else {
+ vlv_prepare_pll(intel_crtc, intel_crtc->config);
+ vlv_enable_pll(intel_crtc, intel_crtc->config);
}
for_each_encoder_on_crtc(dev, crtc, encoder)
@@ -6208,8 +6110,9 @@
i9xx_pfit_enable(intel_crtc);
- intel_crtc_load_lut(crtc);
+ intel_color_load_luts(&pipe_config->base);
+ intel_update_watermarks(crtc);
intel_enable_pipe(intel_crtc);
assert_vblank_disabled(crtc);
@@ -6234,7 +6137,9 @@
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
struct intel_encoder *encoder;
- int pipe = intel_crtc->pipe;
+ struct intel_crtc_state *pipe_config =
+ to_intel_crtc_state(crtc->state);
+ enum pipe pipe = intel_crtc->pipe;
if (WARN_ON(intel_crtc->active))
return;
@@ -6245,6 +6150,7 @@
intel_dp_set_m_n(intel_crtc, M1_N1);
intel_set_pipe_timings(intel_crtc);
+ intel_set_pipe_src_size(intel_crtc);
i9xx_set_pipeconf(intel_crtc);
@@ -6261,7 +6167,7 @@
i9xx_pfit_enable(intel_crtc);
- intel_crtc_load_lut(crtc);
+ intel_color_load_luts(&pipe_config->base);
intel_update_watermarks(crtc);
intel_enable_pipe(intel_crtc);
@@ -6299,10 +6205,9 @@
/*
* On gen2 planes are double buffered but the pipe isn't, so we must
* wait for planes to fully turn off before disabling the pipe.
- * We also need to wait on all gmch platforms because of the
- * self-refresh mode constraint explained above.
*/
- intel_wait_for_vblank(dev, pipe);
+ if (IS_GEN2(dev))
+ intel_wait_for_vblank(dev, pipe);
for_each_encoder_on_crtc(dev, crtc, encoder)
encoder->disable(encoder);
@@ -6337,6 +6242,7 @@
static void intel_crtc_disable_noatomic(struct drm_crtc *crtc)
{
+ struct intel_encoder *encoder;
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
struct drm_i915_private *dev_priv = to_i915(crtc->dev);
enum intel_display_power_domain domain;
@@ -6348,14 +6254,27 @@
if (to_intel_plane_state(crtc->primary->state)->visible) {
WARN_ON(intel_crtc->unpin_work);
- intel_pre_disable_primary(crtc);
+ intel_pre_disable_primary_noatomic(crtc);
intel_crtc_disable_planes(crtc, 1 << drm_plane_index(crtc->primary));
to_intel_plane_state(crtc->primary->state)->visible = false;
}
dev_priv->display.crtc_disable(crtc);
+
+ DRM_DEBUG_KMS("[CRTC:%d] hw state adjusted, was enabled, now disabled\n",
+ crtc->base.id);
+
+ WARN_ON(drm_atomic_set_mode_for_crtc(crtc->state, NULL) < 0);
+ crtc->state->active = false;
intel_crtc->active = false;
+ crtc->enabled = false;
+ crtc->state->connector_mask = 0;
+ crtc->state->encoder_mask = 0;
+
+ for_each_encoder_on_crtc(crtc->dev, crtc, encoder)
+ encoder->base.crtc = NULL;
+
intel_fbc_disable(intel_crtc);
intel_update_watermarks(crtc);
intel_disable_shared_dpll(intel_crtc);
@@ -6398,7 +6317,7 @@
/* Cross check the actual hw state with our own modeset state tracking (and it's
* internal consistency). */
-static void intel_connector_check_state(struct intel_connector *connector)
+static void intel_connector_verify_state(struct intel_connector *connector)
{
struct drm_crtc *crtc = connector->base.state->crtc;
@@ -6568,7 +6487,7 @@
* Hence the bw of each lane in terms of the mode signal
* is:
*/
- link_bw = intel_fdi_link_freq(dev) * MHz(100)/KHz(1)/10;
+ link_bw = intel_fdi_link_freq(to_i915(dev), pipe_config);
fdi_dotclock = adjusted_mode->crtc_clock;
@@ -6580,8 +6499,7 @@
intel_link_compute_m_n(pipe_config->pipe_bpp, lane, fdi_dotclock,
link_bw, &pipe_config->fdi_m_n);
- ret = ironlake_check_fdi_lanes(intel_crtc->base.dev,
- intel_crtc->pipe, pipe_config);
+ ret = ironlake_check_fdi_lanes(dev, intel_crtc->pipe, pipe_config);
if (ret == -EINVAL && pipe_config->pipe_bpp > 6*3) {
pipe_config->pipe_bpp -= 2*3;
DRM_DEBUG_KMS("fdi link bw constraint, reducing pipe bpp to %i\n",
@@ -6605,7 +6523,7 @@
return false;
/* HSW can handle pixel rate up to cdclk? */
- if (IS_HASWELL(dev_priv->dev))
+ if (IS_HASWELL(dev_priv))
return true;
/*
@@ -7133,30 +7051,6 @@
&& !(dev_priv->quirks & QUIRK_LVDS_SSC_DISABLE);
}
-static int i9xx_get_refclk(const struct intel_crtc_state *crtc_state,
- int num_connectors)
-{
- struct drm_device *dev = crtc_state->base.crtc->dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
- int refclk;
-
- WARN_ON(!crtc_state->base.state);
-
- if (IS_VALLEYVIEW(dev) || IS_CHERRYVIEW(dev) || IS_BROXTON(dev)) {
- refclk = 100000;
- } else if (intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_LVDS) &&
- intel_panel_use_ssc(dev_priv) && num_connectors < 2) {
- refclk = dev_priv->vbt.lvds_ssc_freq;
- DRM_DEBUG_KMS("using SSC reference clock of %d kHz\n", refclk);
- } else if (!IS_GEN2(dev)) {
- refclk = 96000;
- } else {
- refclk = 48000;
- }
-
- return refclk;
-}
-
static uint32_t pnv_dpll_compute_fp(struct dpll *dpll)
{
return (1 << dpll->n) << 16 | dpll->m2;
@@ -7300,24 +7194,34 @@
static void vlv_compute_dpll(struct intel_crtc *crtc,
struct intel_crtc_state *pipe_config)
{
- u32 dpll, dpll_md;
+ pipe_config->dpll_hw_state.dpll = DPLL_INTEGRATED_REF_CLK_VLV |
+ DPLL_REF_CLK_ENABLE_VLV | DPLL_VGA_MODE_DIS;
+ if (crtc->pipe != PIPE_A)
+ pipe_config->dpll_hw_state.dpll |= DPLL_INTEGRATED_CRI_CLK_VLV;
- /*
- * Enable DPIO clock input. We should never disable the reference
- * clock for pipe B, since VGA hotplug / manual detection depends
- * on it.
- */
- dpll = DPLL_EXT_BUFFER_ENABLE_VLV | DPLL_REF_CLK_ENABLE_VLV |
- DPLL_VGA_MODE_DIS | DPLL_INTEGRATED_REF_CLK_VLV;
- /* We should never disable this, set it here for state tracking */
- if (crtc->pipe == PIPE_B)
- dpll |= DPLL_INTEGRATED_CRI_CLK_VLV;
- dpll |= DPLL_VCO_ENABLE;
- pipe_config->dpll_hw_state.dpll = dpll;
+ /* DPLL not used with DSI, but still need the rest set up */
+ if (!pipe_config->has_dsi_encoder)
+ pipe_config->dpll_hw_state.dpll |= DPLL_VCO_ENABLE |
+ DPLL_EXT_BUFFER_ENABLE_VLV;
- dpll_md = (pipe_config->pixel_multiplier - 1)
- << DPLL_MD_UDI_MULTIPLIER_SHIFT;
- pipe_config->dpll_hw_state.dpll_md = dpll_md;
+ pipe_config->dpll_hw_state.dpll_md =
+ (pipe_config->pixel_multiplier - 1) << DPLL_MD_UDI_MULTIPLIER_SHIFT;
+}
+
+static void chv_compute_dpll(struct intel_crtc *crtc,
+ struct intel_crtc_state *pipe_config)
+{
+ pipe_config->dpll_hw_state.dpll = DPLL_SSC_REF_CLK_CHV |
+ DPLL_REF_CLK_ENABLE_VLV | DPLL_VGA_MODE_DIS;
+ if (crtc->pipe != PIPE_A)
+ pipe_config->dpll_hw_state.dpll |= DPLL_INTEGRATED_CRI_CLK_VLV;
+
+ /* DPLL not used with DSI, but still need the rest set up */
+ if (!pipe_config->has_dsi_encoder)
+ pipe_config->dpll_hw_state.dpll |= DPLL_VCO_ENABLE;
+
+ pipe_config->dpll_hw_state.dpll_md =
+ (pipe_config->pixel_multiplier - 1) << DPLL_MD_UDI_MULTIPLIER_SHIFT;
}
static void vlv_prepare_pll(struct intel_crtc *crtc,
@@ -7325,11 +7229,20 @@
{
struct drm_device *dev = crtc->base.dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- int pipe = crtc->pipe;
+ enum pipe pipe = crtc->pipe;
u32 mdiv;
u32 bestn, bestm1, bestm2, bestp1, bestp2;
u32 coreclk, reg_val;
+ /* Enable Refclk */
+ I915_WRITE(DPLL(pipe),
+ pipe_config->dpll_hw_state.dpll &
+ ~(DPLL_VCO_ENABLE | DPLL_EXT_BUFFER_ENABLE_VLV));
+
+ /* No need to actually set up the DPLL with DSI */
+ if ((pipe_config->dpll_hw_state.dpll & DPLL_VCO_ENABLE) == 0)
+ return;
+
mutex_lock(&dev_priv->sb_lock);
bestn = pipe_config->dpll.n;
@@ -7411,32 +7324,26 @@
mutex_unlock(&dev_priv->sb_lock);
}
-static void chv_compute_dpll(struct intel_crtc *crtc,
- struct intel_crtc_state *pipe_config)
-{
- pipe_config->dpll_hw_state.dpll = DPLL_SSC_REF_CLK_CHV |
- DPLL_REF_CLK_ENABLE_VLV | DPLL_VGA_MODE_DIS |
- DPLL_VCO_ENABLE;
- if (crtc->pipe != PIPE_A)
- pipe_config->dpll_hw_state.dpll |= DPLL_INTEGRATED_CRI_CLK_VLV;
-
- pipe_config->dpll_hw_state.dpll_md =
- (pipe_config->pixel_multiplier - 1) << DPLL_MD_UDI_MULTIPLIER_SHIFT;
-}
-
static void chv_prepare_pll(struct intel_crtc *crtc,
const struct intel_crtc_state *pipe_config)
{
struct drm_device *dev = crtc->base.dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- int pipe = crtc->pipe;
- i915_reg_t dpll_reg = DPLL(crtc->pipe);
+ enum pipe pipe = crtc->pipe;
enum dpio_channel port = vlv_pipe_to_channel(pipe);
u32 loopfilter, tribuf_calcntr;
u32 bestn, bestm1, bestm2, bestp1, bestp2, bestm2_frac;
u32 dpio_val;
int vco;
+ /* Enable Refclk and SSC */
+ I915_WRITE(DPLL(pipe),
+ pipe_config->dpll_hw_state.dpll & ~DPLL_VCO_ENABLE);
+
+ /* No need to actually set up the DPLL with DSI */
+ if ((pipe_config->dpll_hw_state.dpll & DPLL_VCO_ENABLE) == 0)
+ return;
+
bestn = pipe_config->dpll.n;
bestm2_frac = pipe_config->dpll.m2 & 0x3fffff;
bestm1 = pipe_config->dpll.m1;
@@ -7447,12 +7354,6 @@
dpio_val = 0;
loopfilter = 0;
- /*
- * Enable Refclk and SSC
- */
- I915_WRITE(dpll_reg,
- pipe_config->dpll_hw_state.dpll & ~DPLL_VCO_ENABLE);
-
mutex_lock(&dev_priv->sb_lock);
/* p1 and p2 divider */
@@ -7586,8 +7487,7 @@
static void i9xx_compute_dpll(struct intel_crtc *crtc,
struct intel_crtc_state *crtc_state,
- intel_clock_t *reduced_clock,
- int num_connectors)
+ intel_clock_t *reduced_clock)
{
struct drm_device *dev = crtc->base.dev;
struct drm_i915_private *dev_priv = dev->dev_private;
@@ -7646,7 +7546,7 @@
if (crtc_state->sdvo_tv_clock)
dpll |= PLL_REF_INPUT_TVCLKINBC;
else if (intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_LVDS) &&
- intel_panel_use_ssc(dev_priv) && num_connectors < 2)
+ intel_panel_use_ssc(dev_priv))
dpll |= PLLB_REF_INPUT_SPREADSPECTRUMIN;
else
dpll |= PLL_REF_INPUT_DREFCLK;
@@ -7663,8 +7563,7 @@
static void i8xx_compute_dpll(struct intel_crtc *crtc,
struct intel_crtc_state *crtc_state,
- intel_clock_t *reduced_clock,
- int num_connectors)
+ intel_clock_t *reduced_clock)
{
struct drm_device *dev = crtc->base.dev;
struct drm_i915_private *dev_priv = dev->dev_private;
@@ -7690,7 +7589,7 @@
dpll |= DPLL_DVO_2X_MODE;
if (intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_LVDS) &&
- intel_panel_use_ssc(dev_priv) && num_connectors < 2)
+ intel_panel_use_ssc(dev_priv))
dpll |= PLLB_REF_INPUT_SPREADSPECTRUMIN;
else
dpll |= PLL_REF_INPUT_DREFCLK;
@@ -7759,6 +7658,14 @@
(pipe == PIPE_B || pipe == PIPE_C))
I915_WRITE(VTOTAL(pipe), I915_READ(VTOTAL(cpu_transcoder)));
+}
+
+static void intel_set_pipe_src_size(struct intel_crtc *intel_crtc)
+{
+ struct drm_device *dev = intel_crtc->base.dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ enum pipe pipe = intel_crtc->pipe;
+
/* pipesrc controls the size that is scaled from, which should
* always be the user's requested size.
*/
@@ -7800,6 +7707,14 @@
pipe_config->base.adjusted_mode.crtc_vtotal += 1;
pipe_config->base.adjusted_mode.crtc_vblank_end += 1;
}
+}
+
+static void intel_get_pipe_src_size(struct intel_crtc *crtc,
+ struct intel_crtc_state *pipe_config)
+{
+ struct drm_device *dev = crtc->base.dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ u32 tmp;
tmp = I915_READ(PIPESRC(crtc->pipe));
pipe_config->pipe_src_h = (tmp & 0xffff) + 1;
@@ -7897,69 +7812,192 @@
POSTING_READ(PIPECONF(intel_crtc->pipe));
}
+static int i8xx_crtc_compute_clock(struct intel_crtc *crtc,
+ struct intel_crtc_state *crtc_state)
+{
+ struct drm_device *dev = crtc->base.dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ const intel_limit_t *limit;
+ int refclk = 48000;
+
+ memset(&crtc_state->dpll_hw_state, 0,
+ sizeof(crtc_state->dpll_hw_state));
+
+ if (intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_LVDS)) {
+ if (intel_panel_use_ssc(dev_priv)) {
+ refclk = dev_priv->vbt.lvds_ssc_freq;
+ DRM_DEBUG_KMS("using SSC reference clock of %d kHz\n", refclk);
+ }
+
+ limit = &intel_limits_i8xx_lvds;
+ } else if (intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_DVO)) {
+ limit = &intel_limits_i8xx_dvo;
+ } else {
+ limit = &intel_limits_i8xx_dac;
+ }
+
+ if (!crtc_state->clock_set &&
+ !i9xx_find_best_dpll(limit, crtc_state, crtc_state->port_clock,
+ refclk, NULL, &crtc_state->dpll)) {
+ DRM_ERROR("Couldn't find PLL settings for mode!\n");
+ return -EINVAL;
+ }
+
+ i8xx_compute_dpll(crtc, crtc_state, NULL);
+
+ return 0;
+}
+
+static int g4x_crtc_compute_clock(struct intel_crtc *crtc,
+ struct intel_crtc_state *crtc_state)
+{
+ struct drm_device *dev = crtc->base.dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ const intel_limit_t *limit;
+ int refclk = 96000;
+
+ memset(&crtc_state->dpll_hw_state, 0,
+ sizeof(crtc_state->dpll_hw_state));
+
+ if (intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_LVDS)) {
+ if (intel_panel_use_ssc(dev_priv)) {
+ refclk = dev_priv->vbt.lvds_ssc_freq;
+ DRM_DEBUG_KMS("using SSC reference clock of %d kHz\n", refclk);
+ }
+
+ if (intel_is_dual_link_lvds(dev))
+ limit = &intel_limits_g4x_dual_channel_lvds;
+ else
+ limit = &intel_limits_g4x_single_channel_lvds;
+ } else if (intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_HDMI) ||
+ intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_ANALOG)) {
+ limit = &intel_limits_g4x_hdmi;
+ } else if (intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_SDVO)) {
+ limit = &intel_limits_g4x_sdvo;
+ } else {
+ /* The option is for other outputs */
+ limit = &intel_limits_i9xx_sdvo;
+ }
+
+ if (!crtc_state->clock_set &&
+ !g4x_find_best_dpll(limit, crtc_state, crtc_state->port_clock,
+ refclk, NULL, &crtc_state->dpll)) {
+ DRM_ERROR("Couldn't find PLL settings for mode!\n");
+ return -EINVAL;
+ }
+
+ i9xx_compute_dpll(crtc, crtc_state, NULL);
+
+ return 0;
+}
+
+static int pnv_crtc_compute_clock(struct intel_crtc *crtc,
+ struct intel_crtc_state *crtc_state)
+{
+ struct drm_device *dev = crtc->base.dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ const intel_limit_t *limit;
+ int refclk = 96000;
+
+ memset(&crtc_state->dpll_hw_state, 0,
+ sizeof(crtc_state->dpll_hw_state));
+
+ if (intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_LVDS)) {
+ if (intel_panel_use_ssc(dev_priv)) {
+ refclk = dev_priv->vbt.lvds_ssc_freq;
+ DRM_DEBUG_KMS("using SSC reference clock of %d kHz\n", refclk);
+ }
+
+ limit = &intel_limits_pineview_lvds;
+ } else {
+ limit = &intel_limits_pineview_sdvo;
+ }
+
+ if (!crtc_state->clock_set &&
+ !pnv_find_best_dpll(limit, crtc_state, crtc_state->port_clock,
+ refclk, NULL, &crtc_state->dpll)) {
+ DRM_ERROR("Couldn't find PLL settings for mode!\n");
+ return -EINVAL;
+ }
+
+ i9xx_compute_dpll(crtc, crtc_state, NULL);
+
+ return 0;
+}
+
static int i9xx_crtc_compute_clock(struct intel_crtc *crtc,
struct intel_crtc_state *crtc_state)
{
struct drm_device *dev = crtc->base.dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- int refclk, num_connectors = 0;
- intel_clock_t clock;
- bool ok;
const intel_limit_t *limit;
- struct drm_atomic_state *state = crtc_state->base.state;
- struct drm_connector *connector;
- struct drm_connector_state *connector_state;
- int i;
+ int refclk = 96000;
memset(&crtc_state->dpll_hw_state, 0,
sizeof(crtc_state->dpll_hw_state));
- if (crtc_state->has_dsi_encoder)
- return 0;
-
- for_each_connector_in_state(state, connector, connector_state, i) {
- if (connector_state->crtc == &crtc->base)
- num_connectors++;
- }
-
- if (!crtc_state->clock_set) {
- refclk = i9xx_get_refclk(crtc_state, num_connectors);
-
- /*
- * Returns a set of divisors for the desired target clock with
- * the given refclk, or FALSE. The returned values represent
- * the clock equation: reflck * (5 * (m1 + 2) + (m2 + 2)) / (n +
- * 2) / p1 / p2.
- */
- limit = intel_limit(crtc_state, refclk);
- ok = dev_priv->display.find_dpll(limit, crtc_state,
- crtc_state->port_clock,
- refclk, NULL, &clock);
- if (!ok) {
- DRM_ERROR("Couldn't find PLL settings for mode!\n");
- return -EINVAL;
+ if (intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_LVDS)) {
+ if (intel_panel_use_ssc(dev_priv)) {
+ refclk = dev_priv->vbt.lvds_ssc_freq;
+ DRM_DEBUG_KMS("using SSC reference clock of %d kHz\n", refclk);
}
- /* Compat-code for transition, will disappear. */
- crtc_state->dpll.n = clock.n;
- crtc_state->dpll.m1 = clock.m1;
- crtc_state->dpll.m2 = clock.m2;
- crtc_state->dpll.p1 = clock.p1;
- crtc_state->dpll.p2 = clock.p2;
+ limit = &intel_limits_i9xx_lvds;
+ } else {
+ limit = &intel_limits_i9xx_sdvo;
}
- if (IS_GEN2(dev)) {
- i8xx_compute_dpll(crtc, crtc_state, NULL,
- num_connectors);
- } else if (IS_CHERRYVIEW(dev)) {
- chv_compute_dpll(crtc, crtc_state);
- } else if (IS_VALLEYVIEW(dev)) {
- vlv_compute_dpll(crtc, crtc_state);
- } else {
- i9xx_compute_dpll(crtc, crtc_state, NULL,
- num_connectors);
+ if (!crtc_state->clock_set &&
+ !i9xx_find_best_dpll(limit, crtc_state, crtc_state->port_clock,
+ refclk, NULL, &crtc_state->dpll)) {
+ DRM_ERROR("Couldn't find PLL settings for mode!\n");
+ return -EINVAL;
}
+ i9xx_compute_dpll(crtc, crtc_state, NULL);
+
+ return 0;
+}
+
+static int chv_crtc_compute_clock(struct intel_crtc *crtc,
+ struct intel_crtc_state *crtc_state)
+{
+ int refclk = 100000;
+ const intel_limit_t *limit = &intel_limits_chv;
+
+ memset(&crtc_state->dpll_hw_state, 0,
+ sizeof(crtc_state->dpll_hw_state));
+
+ if (!crtc_state->clock_set &&
+ !chv_find_best_dpll(limit, crtc_state, crtc_state->port_clock,
+ refclk, NULL, &crtc_state->dpll)) {
+ DRM_ERROR("Couldn't find PLL settings for mode!\n");
+ return -EINVAL;
+ }
+
+ chv_compute_dpll(crtc, crtc_state);
+
+ return 0;
+}
+
+static int vlv_crtc_compute_clock(struct intel_crtc *crtc,
+ struct intel_crtc_state *crtc_state)
+{
+ int refclk = 100000;
+ const intel_limit_t *limit = &intel_limits_vlv;
+
+ memset(&crtc_state->dpll_hw_state, 0,
+ sizeof(crtc_state->dpll_hw_state));
+
+ if (!crtc_state->clock_set &&
+ !vlv_find_best_dpll(limit, crtc_state, crtc_state->port_clock,
+ refclk, NULL, &crtc_state->dpll)) {
+ DRM_ERROR("Couldn't find PLL settings for mode!\n");
+ return -EINVAL;
+ }
+
+ vlv_compute_dpll(crtc, crtc_state);
+
return 0;
}
@@ -8000,8 +8038,8 @@
u32 mdiv;
int refclk = 100000;
- /* In case of MIPI DPLL will not even be used */
- if (!(pipe_config->dpll_hw_state.dpll & DPLL_VCO_ENABLE))
+ /* In case of DSI, DPLL will not be used */
+ if ((pipe_config->dpll_hw_state.dpll & DPLL_VCO_ENABLE) == 0)
return;
mutex_lock(&dev_priv->sb_lock);
@@ -8097,6 +8135,10 @@
u32 cmn_dw13, pll_dw0, pll_dw1, pll_dw2, pll_dw3;
int refclk = 100000;
+ /* In case of DSI, DPLL will not be used */
+ if ((pipe_config->dpll_hw_state.dpll & DPLL_VCO_ENABLE) == 0)
+ return;
+
mutex_lock(&dev_priv->sb_lock);
cmn_dw13 = vlv_dpio_read(dev_priv, pipe, CHV_CMN_DW13(port));
pll_dw0 = vlv_dpio_read(dev_priv, pipe, CHV_PLL_DW0(port));
@@ -8130,7 +8172,7 @@
return false;
pipe_config->cpu_transcoder = (enum transcoder) crtc->pipe;
- pipe_config->shared_dpll = DPLL_ID_PRIVATE;
+ pipe_config->shared_dpll = NULL;
ret = false;
@@ -8162,11 +8204,16 @@
pipe_config->double_wide = tmp & PIPECONF_DOUBLE_WIDE;
intel_get_pipe_timings(crtc, pipe_config);
+ intel_get_pipe_src_size(crtc, pipe_config);
i9xx_get_pfit_config(crtc, pipe_config);
if (INTEL_INFO(dev)->gen >= 4) {
- tmp = I915_READ(DPLL_MD(crtc->pipe));
+ /* No way to read it out on pipes B and C */
+ if (IS_CHERRYVIEW(dev) && crtc->pipe != PIPE_A)
+ tmp = dev_priv->chv_dpll_md[crtc->pipe];
+ else
+ tmp = I915_READ(DPLL_MD(crtc->pipe));
pipe_config->pixel_multiplier =
((tmp & DPLL_MD_UDI_MULTIPLIER_MASK)
>> DPLL_MD_UDI_MULTIPLIER_SHIFT) + 1;
@@ -8635,42 +8682,6 @@
lpt_init_pch_refclk(dev);
}
-static int ironlake_get_refclk(struct intel_crtc_state *crtc_state)
-{
- struct drm_device *dev = crtc_state->base.crtc->dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
- struct drm_atomic_state *state = crtc_state->base.state;
- struct drm_connector *connector;
- struct drm_connector_state *connector_state;
- struct intel_encoder *encoder;
- int num_connectors = 0, i;
- bool is_lvds = false;
-
- for_each_connector_in_state(state, connector, connector_state, i) {
- if (connector_state->crtc != crtc_state->base.crtc)
- continue;
-
- encoder = to_intel_encoder(connector_state->best_encoder);
-
- switch (encoder->type) {
- case INTEL_OUTPUT_LVDS:
- is_lvds = true;
- break;
- default:
- break;
- }
- num_connectors++;
- }
-
- if (is_lvds && intel_panel_use_ssc(dev_priv) && num_connectors < 2) {
- DRM_DEBUG_KMS("using SSC reference clock of %d kHz\n",
- dev_priv->vbt.lvds_ssc_freq);
- return dev_priv->vbt.lvds_ssc_freq;
- }
-
- return 120000;
-}
-
static void ironlake_set_pipeconf(struct drm_crtc *crtc)
{
struct drm_i915_private *dev_priv = crtc->dev->dev_private;
@@ -8713,82 +8724,14 @@
POSTING_READ(PIPECONF(pipe));
}
-/*
- * Set up the pipe CSC unit.
- *
- * Currently only full range RGB to limited range RGB conversion
- * is supported, but eventually this should handle various
- * RGB<->YCbCr scenarios as well.
- */
-static void intel_set_pipe_csc(struct drm_crtc *crtc)
-{
- struct drm_device *dev = crtc->dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- int pipe = intel_crtc->pipe;
- uint16_t coeff = 0x7800; /* 1.0 */
-
- /*
- * TODO: Check what kind of values actually come out of the pipe
- * with these coeff/postoff values and adjust to get the best
- * accuracy. Perhaps we even need to take the bpc value into
- * consideration.
- */
-
- if (intel_crtc->config->limited_color_range)
- coeff = ((235 - 16) * (1 << 12) / 255) & 0xff8; /* 0.xxx... */
-
- /*
- * GY/GU and RY/RU should be the other way around according
- * to BSpec, but reality doesn't agree. Just set them up in
- * a way that results in the correct picture.
- */
- I915_WRITE(PIPE_CSC_COEFF_RY_GY(pipe), coeff << 16);
- I915_WRITE(PIPE_CSC_COEFF_BY(pipe), 0);
-
- I915_WRITE(PIPE_CSC_COEFF_RU_GU(pipe), coeff);
- I915_WRITE(PIPE_CSC_COEFF_BU(pipe), 0);
-
- I915_WRITE(PIPE_CSC_COEFF_RV_GV(pipe), 0);
- I915_WRITE(PIPE_CSC_COEFF_BV(pipe), coeff << 16);
-
- I915_WRITE(PIPE_CSC_PREOFF_HI(pipe), 0);
- I915_WRITE(PIPE_CSC_PREOFF_ME(pipe), 0);
- I915_WRITE(PIPE_CSC_PREOFF_LO(pipe), 0);
-
- if (INTEL_INFO(dev)->gen > 6) {
- uint16_t postoff = 0;
-
- if (intel_crtc->config->limited_color_range)
- postoff = (16 * (1 << 12) / 255) & 0x1fff;
-
- I915_WRITE(PIPE_CSC_POSTOFF_HI(pipe), postoff);
- I915_WRITE(PIPE_CSC_POSTOFF_ME(pipe), postoff);
- I915_WRITE(PIPE_CSC_POSTOFF_LO(pipe), postoff);
-
- I915_WRITE(PIPE_CSC_MODE(pipe), 0);
- } else {
- uint32_t mode = CSC_MODE_YUV_TO_RGB;
-
- if (intel_crtc->config->limited_color_range)
- mode |= CSC_BLACK_SCREEN_OFFSET;
-
- I915_WRITE(PIPE_CSC_MODE(pipe), mode);
- }
-}
-
static void haswell_set_pipeconf(struct drm_crtc *crtc)
{
- struct drm_device *dev = crtc->dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = crtc->dev->dev_private;
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- enum pipe pipe = intel_crtc->pipe;
enum transcoder cpu_transcoder = intel_crtc->config->cpu_transcoder;
- uint32_t val;
+ u32 val = 0;
- val = 0;
-
- if (IS_HASWELL(dev) && intel_crtc->config->dither)
+ if (IS_HASWELL(dev_priv) && intel_crtc->config->dither)
val |= (PIPECONF_DITHER_EN | PIPECONF_DITHER_TYPE_SP);
if (intel_crtc->config->base.adjusted_mode.flags & DRM_MODE_FLAG_INTERLACE)
@@ -8798,12 +8741,15 @@
I915_WRITE(PIPECONF(cpu_transcoder), val);
POSTING_READ(PIPECONF(cpu_transcoder));
+}
- I915_WRITE(GAMMA_MODE(intel_crtc->pipe), GAMMA_MODE_MODE_8BIT);
- POSTING_READ(GAMMA_MODE(intel_crtc->pipe));
+static void haswell_set_pipemisc(struct drm_crtc *crtc)
+{
+ struct drm_i915_private *dev_priv = crtc->dev->dev_private;
+ struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- if (IS_BROADWELL(dev) || INTEL_INFO(dev)->gen >= 9) {
- val = 0;
+ if (IS_BROADWELL(dev_priv) || INTEL_INFO(dev_priv)->gen >= 9) {
+ u32 val = 0;
switch (intel_crtc->config->pipe_bpp) {
case 18:
@@ -8826,39 +8772,10 @@
if (intel_crtc->config->dither)
val |= PIPEMISC_DITHER_ENABLE | PIPEMISC_DITHER_TYPE_SP;
- I915_WRITE(PIPEMISC(pipe), val);
+ I915_WRITE(PIPEMISC(intel_crtc->pipe), val);
}
}
-static bool ironlake_compute_clocks(struct drm_crtc *crtc,
- struct intel_crtc_state *crtc_state,
- intel_clock_t *clock,
- bool *has_reduced_clock,
- intel_clock_t *reduced_clock)
-{
- struct drm_device *dev = crtc->dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
- int refclk;
- const intel_limit_t *limit;
- bool ret;
-
- refclk = ironlake_get_refclk(crtc_state);
-
- /*
- * Returns a set of divisors for the desired target clock with the given
- * refclk, or FALSE. The returned values represent the clock equation:
- * reflck * (5 * (m1 + 2) + (m2 + 2)) / (n + 2) / p1 / p2.
- */
- limit = intel_limit(crtc_state, refclk);
- ret = dev_priv->display.find_dpll(limit, crtc_state,
- crtc_state->port_clock,
- refclk, NULL, clock);
- if (!ret)
- return false;
-
- return true;
-}
-
int ironlake_get_lanes_required(int target_clock, int link_bw, int bpp)
{
/*
@@ -8875,10 +8792,9 @@
return i9xx_dpll_compute_m(dpll) < factor * dpll->n;
}
-static uint32_t ironlake_compute_dpll(struct intel_crtc *intel_crtc,
- struct intel_crtc_state *crtc_state,
- u32 *fp,
- intel_clock_t *reduced_clock, u32 *fp2)
+static void ironlake_compute_dpll(struct intel_crtc *intel_crtc,
+ struct intel_crtc_state *crtc_state,
+ intel_clock_t *reduced_clock)
{
struct drm_crtc *crtc = &intel_crtc->base;
struct drm_device *dev = crtc->dev;
@@ -8887,8 +8803,8 @@
struct drm_connector *connector;
struct drm_connector_state *connector_state;
struct intel_encoder *encoder;
- uint32_t dpll;
- int factor, num_connectors = 0, i;
+ u32 dpll, fp, fp2;
+ int factor, i;
bool is_lvds = false, is_sdvo = false;
for_each_connector_in_state(state, connector, connector_state, i) {
@@ -8908,8 +8824,6 @@
default:
break;
}
-
- num_connectors++;
}
/* Enable autotuning of the PLL clock (if permissible) */
@@ -8922,11 +8836,19 @@
} else if (crtc_state->sdvo_tv_clock)
factor = 20;
- if (ironlake_needs_fb_cb_tune(&crtc_state->dpll, factor))
- *fp |= FP_CB_TUNE;
+ fp = i9xx_dpll_compute_fp(&crtc_state->dpll);
- if (fp2 && (reduced_clock->m < factor * reduced_clock->n))
- *fp2 |= FP_CB_TUNE;
+ if (ironlake_needs_fb_cb_tune(&crtc_state->dpll, factor))
+ fp |= FP_CB_TUNE;
+
+ if (reduced_clock) {
+ fp2 = i9xx_dpll_compute_fp(reduced_clock);
+
+ if (reduced_clock->m < factor * reduced_clock->n)
+ fp2 |= FP_CB_TUNE;
+ } else {
+ fp2 = fp;
+ }
dpll = 0;
@@ -8963,76 +8885,80 @@
break;
}
- if (is_lvds && intel_panel_use_ssc(dev_priv) && num_connectors < 2)
+ if (is_lvds && intel_panel_use_ssc(dev_priv))
dpll |= PLLB_REF_INPUT_SPREADSPECTRUMIN;
else
dpll |= PLL_REF_INPUT_DREFCLK;
- return dpll | DPLL_VCO_ENABLE;
+ dpll |= DPLL_VCO_ENABLE;
+
+ crtc_state->dpll_hw_state.dpll = dpll;
+ crtc_state->dpll_hw_state.fp0 = fp;
+ crtc_state->dpll_hw_state.fp1 = fp2;
}
static int ironlake_crtc_compute_clock(struct intel_crtc *crtc,
struct intel_crtc_state *crtc_state)
{
struct drm_device *dev = crtc->base.dev;
- intel_clock_t clock, reduced_clock;
- u32 dpll = 0, fp = 0, fp2 = 0;
- bool ok, has_reduced_clock = false;
- bool is_lvds = false;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ intel_clock_t reduced_clock;
+ bool has_reduced_clock = false;
struct intel_shared_dpll *pll;
+ const intel_limit_t *limit;
+ int refclk = 120000;
memset(&crtc_state->dpll_hw_state, 0,
sizeof(crtc_state->dpll_hw_state));
- is_lvds = intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_LVDS);
+ crtc->lowfreq_avail = false;
- WARN(!(HAS_PCH_IBX(dev) || HAS_PCH_CPT(dev)),
- "Unexpected PCH type %d\n", INTEL_PCH_TYPE(dev));
+ /* CPU eDP is the only output that doesn't need a PCH PLL of its own. */
+ if (!crtc_state->has_pch_encoder)
+ return 0;
- ok = ironlake_compute_clocks(&crtc->base, crtc_state, &clock,
- &has_reduced_clock, &reduced_clock);
- if (!ok && !crtc_state->clock_set) {
+ if (intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_LVDS)) {
+ if (intel_panel_use_ssc(dev_priv)) {
+ DRM_DEBUG_KMS("using SSC reference clock of %d kHz\n",
+ dev_priv->vbt.lvds_ssc_freq);
+ refclk = dev_priv->vbt.lvds_ssc_freq;
+ }
+
+ if (intel_is_dual_link_lvds(dev)) {
+ if (refclk == 100000)
+ limit = &intel_limits_ironlake_dual_lvds_100m;
+ else
+ limit = &intel_limits_ironlake_dual_lvds;
+ } else {
+ if (refclk == 100000)
+ limit = &intel_limits_ironlake_single_lvds_100m;
+ else
+ limit = &intel_limits_ironlake_single_lvds;
+ }
+ } else {
+ limit = &intel_limits_ironlake_dac;
+ }
+
+ if (!crtc_state->clock_set &&
+ !g4x_find_best_dpll(limit, crtc_state, crtc_state->port_clock,
+ refclk, NULL, &crtc_state->dpll)) {
DRM_ERROR("Couldn't find PLL settings for mode!\n");
return -EINVAL;
}
- /* Compat-code for transition, will disappear. */
- if (!crtc_state->clock_set) {
- crtc_state->dpll.n = clock.n;
- crtc_state->dpll.m1 = clock.m1;
- crtc_state->dpll.m2 = clock.m2;
- crtc_state->dpll.p1 = clock.p1;
- crtc_state->dpll.p2 = clock.p2;
+
+ ironlake_compute_dpll(crtc, crtc_state,
+ has_reduced_clock ? &reduced_clock : NULL);
+
+ pll = intel_get_shared_dpll(crtc, crtc_state, NULL);
+ if (pll == NULL) {
+ DRM_DEBUG_DRIVER("failed to find PLL for pipe %c\n",
+ pipe_name(crtc->pipe));
+ return -EINVAL;
}
- /* CPU eDP is the only output that doesn't need a PCH PLL of its own. */
- if (crtc_state->has_pch_encoder) {
- fp = i9xx_dpll_compute_fp(&crtc_state->dpll);
- if (has_reduced_clock)
- fp2 = i9xx_dpll_compute_fp(&reduced_clock);
-
- dpll = ironlake_compute_dpll(crtc, crtc_state,
- &fp, &reduced_clock,
- has_reduced_clock ? &fp2 : NULL);
-
- crtc_state->dpll_hw_state.dpll = dpll;
- crtc_state->dpll_hw_state.fp0 = fp;
- if (has_reduced_clock)
- crtc_state->dpll_hw_state.fp1 = fp2;
- else
- crtc_state->dpll_hw_state.fp1 = fp;
-
- pll = intel_get_shared_dpll(crtc, crtc_state);
- if (pll == NULL) {
- DRM_DEBUG_DRIVER("failed to find PLL for pipe %c\n",
- pipe_name(crtc->pipe));
- return -EINVAL;
- }
- }
-
- if (is_lvds && has_reduced_clock)
+ if (intel_pipe_will_have_type(crtc_state, INTEL_OUTPUT_LVDS) &&
+ has_reduced_clock)
crtc->lowfreq_avail = true;
- else
- crtc->lowfreq_avail = false;
return 0;
}
@@ -9334,7 +9260,7 @@
return false;
pipe_config->cpu_transcoder = (enum transcoder) crtc->pipe;
- pipe_config->shared_dpll = DPLL_ID_PRIVATE;
+ pipe_config->shared_dpll = NULL;
ret = false;
tmp = I915_READ(PIPECONF(crtc->pipe));
@@ -9363,6 +9289,7 @@
if (I915_READ(PCH_TRANSCONF(crtc->pipe)) & TRANS_ENABLE) {
struct intel_shared_dpll *pll;
+ enum intel_dpll_id pll_id;
pipe_config->has_pch_encoder = true;
@@ -9372,21 +9299,22 @@
ironlake_get_fdi_m_n_config(crtc, pipe_config);
- if (HAS_PCH_IBX(dev_priv->dev)) {
- pipe_config->shared_dpll =
- (enum intel_dpll_id) crtc->pipe;
+ if (HAS_PCH_IBX(dev_priv)) {
+ pll_id = (enum intel_dpll_id) crtc->pipe;
} else {
tmp = I915_READ(PCH_DPLL_SEL);
if (tmp & TRANS_DPLLB_SEL(crtc->pipe))
- pipe_config->shared_dpll = DPLL_ID_PCH_PLL_B;
+ pll_id = DPLL_ID_PCH_PLL_B;
else
- pipe_config->shared_dpll = DPLL_ID_PCH_PLL_A;
+ pll_id= DPLL_ID_PCH_PLL_A;
}
- pll = &dev_priv->shared_dplls[pipe_config->shared_dpll];
+ pipe_config->shared_dpll =
+ intel_get_shared_dpll_by_id(dev_priv, pll_id);
+ pll = pipe_config->shared_dpll;
- WARN_ON(!pll->get_hw_state(dev_priv, pll,
- &pipe_config->dpll_hw_state));
+ WARN_ON(!pll->funcs.get_hw_state(dev_priv, pll,
+ &pipe_config->dpll_hw_state));
tmp = pipe_config->dpll_hw_state.dpll;
pipe_config->pixel_multiplier =
@@ -9399,6 +9327,7 @@
}
intel_get_pipe_timings(crtc, pipe_config);
+ intel_get_pipe_src_size(crtc, pipe_config);
ironlake_get_pfit_config(crtc, pipe_config);
@@ -9638,7 +9567,7 @@
to_intel_atomic_state(old_state);
unsigned int req_cdclk = old_intel_state->dev_cdclk;
- broxton_set_cdclk(dev, req_cdclk);
+ broxton_set_cdclk(to_i915(dev), req_cdclk);
}
/* compute the max rate for new configuration */
@@ -9706,8 +9635,8 @@
val |= LCPLL_CD_SOURCE_FCLK;
I915_WRITE(LCPLL_CTL, val);
- if (wait_for_atomic_us(I915_READ(LCPLL_CTL) &
- LCPLL_CD_SOURCE_FCLK_DONE, 1))
+ if (wait_for_us(I915_READ(LCPLL_CTL) &
+ LCPLL_CD_SOURCE_FCLK_DONE, 1))
DRM_ERROR("Switching to FCLK failed\n");
val = I915_READ(LCPLL_CTL);
@@ -9741,8 +9670,8 @@
val &= ~LCPLL_CD_SOURCE_FCLK;
I915_WRITE(LCPLL_CTL, val);
- if (wait_for_atomic_us((I915_READ(LCPLL_CTL) &
- LCPLL_CD_SOURCE_FCLK_DONE) == 0, 1))
+ if (wait_for_us((I915_READ(LCPLL_CTL) &
+ LCPLL_CD_SOURCE_FCLK_DONE) == 0, 1))
DRM_ERROR("Switching back to LCPLL failed\n");
mutex_lock(&dev_priv->rps.hw_lock);
@@ -9821,72 +9750,193 @@
enum port port,
struct intel_crtc_state *pipe_config)
{
+ enum intel_dpll_id id;
+
switch (port) {
case PORT_A:
pipe_config->ddi_pll_sel = SKL_DPLL0;
- pipe_config->shared_dpll = DPLL_ID_SKL_DPLL1;
+ id = DPLL_ID_SKL_DPLL0;
break;
case PORT_B:
pipe_config->ddi_pll_sel = SKL_DPLL1;
- pipe_config->shared_dpll = DPLL_ID_SKL_DPLL2;
+ id = DPLL_ID_SKL_DPLL1;
break;
case PORT_C:
pipe_config->ddi_pll_sel = SKL_DPLL2;
- pipe_config->shared_dpll = DPLL_ID_SKL_DPLL3;
+ id = DPLL_ID_SKL_DPLL2;
break;
default:
DRM_ERROR("Incorrect port type\n");
+ return;
}
+
+ pipe_config->shared_dpll = intel_get_shared_dpll_by_id(dev_priv, id);
}
static void skylake_get_ddi_pll(struct drm_i915_private *dev_priv,
enum port port,
struct intel_crtc_state *pipe_config)
{
- u32 temp, dpll_ctl1;
+ enum intel_dpll_id id;
+ u32 temp;
temp = I915_READ(DPLL_CTRL2) & DPLL_CTRL2_DDI_CLK_SEL_MASK(port);
pipe_config->ddi_pll_sel = temp >> (port * 3 + 1);
switch (pipe_config->ddi_pll_sel) {
case SKL_DPLL0:
- /*
- * On SKL the eDP DPLL (DPLL0 as we don't use SSC) is not part
- * of the shared DPLL framework and thus needs to be read out
- * separately
- */
- dpll_ctl1 = I915_READ(DPLL_CTRL1);
- pipe_config->dpll_hw_state.ctrl1 = dpll_ctl1 & 0x3f;
+ id = DPLL_ID_SKL_DPLL0;
break;
case SKL_DPLL1:
- pipe_config->shared_dpll = DPLL_ID_SKL_DPLL1;
+ id = DPLL_ID_SKL_DPLL1;
break;
case SKL_DPLL2:
- pipe_config->shared_dpll = DPLL_ID_SKL_DPLL2;
+ id = DPLL_ID_SKL_DPLL2;
break;
case SKL_DPLL3:
- pipe_config->shared_dpll = DPLL_ID_SKL_DPLL3;
+ id = DPLL_ID_SKL_DPLL3;
break;
+ default:
+ MISSING_CASE(pipe_config->ddi_pll_sel);
+ return;
}
+
+ pipe_config->shared_dpll = intel_get_shared_dpll_by_id(dev_priv, id);
}
static void haswell_get_ddi_pll(struct drm_i915_private *dev_priv,
enum port port,
struct intel_crtc_state *pipe_config)
{
+ enum intel_dpll_id id;
+
pipe_config->ddi_pll_sel = I915_READ(PORT_CLK_SEL(port));
switch (pipe_config->ddi_pll_sel) {
case PORT_CLK_SEL_WRPLL1:
- pipe_config->shared_dpll = DPLL_ID_WRPLL1;
+ id = DPLL_ID_WRPLL1;
break;
case PORT_CLK_SEL_WRPLL2:
- pipe_config->shared_dpll = DPLL_ID_WRPLL2;
+ id = DPLL_ID_WRPLL2;
break;
case PORT_CLK_SEL_SPLL:
- pipe_config->shared_dpll = DPLL_ID_SPLL;
+ id = DPLL_ID_SPLL;
+ break;
+ case PORT_CLK_SEL_LCPLL_810:
+ id = DPLL_ID_LCPLL_810;
+ break;
+ case PORT_CLK_SEL_LCPLL_1350:
+ id = DPLL_ID_LCPLL_1350;
+ break;
+ case PORT_CLK_SEL_LCPLL_2700:
+ id = DPLL_ID_LCPLL_2700;
+ break;
+ default:
+ MISSING_CASE(pipe_config->ddi_pll_sel);
+ /* fall through */
+ case PORT_CLK_SEL_NONE:
+ return;
+ }
+
+ pipe_config->shared_dpll = intel_get_shared_dpll_by_id(dev_priv, id);
+}
+
+static bool hsw_get_transcoder_state(struct intel_crtc *crtc,
+ struct intel_crtc_state *pipe_config,
+ unsigned long *power_domain_mask)
+{
+ struct drm_device *dev = crtc->base.dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ enum intel_display_power_domain power_domain;
+ u32 tmp;
+
+ pipe_config->cpu_transcoder = (enum transcoder) crtc->pipe;
+
+ /*
+ * XXX: Do intel_display_power_get_if_enabled before reading this (for
+ * consistency and less surprising code; it's in always on power).
+ */
+ tmp = I915_READ(TRANS_DDI_FUNC_CTL(TRANSCODER_EDP));
+ if (tmp & TRANS_DDI_FUNC_ENABLE) {
+ enum pipe trans_edp_pipe;
+ switch (tmp & TRANS_DDI_EDP_INPUT_MASK) {
+ default:
+ WARN(1, "unknown pipe linked to edp transcoder\n");
+ case TRANS_DDI_EDP_INPUT_A_ONOFF:
+ case TRANS_DDI_EDP_INPUT_A_ON:
+ trans_edp_pipe = PIPE_A;
+ break;
+ case TRANS_DDI_EDP_INPUT_B_ONOFF:
+ trans_edp_pipe = PIPE_B;
+ break;
+ case TRANS_DDI_EDP_INPUT_C_ONOFF:
+ trans_edp_pipe = PIPE_C;
+ break;
+ }
+
+ if (trans_edp_pipe == crtc->pipe)
+ pipe_config->cpu_transcoder = TRANSCODER_EDP;
+ }
+
+ power_domain = POWER_DOMAIN_TRANSCODER(pipe_config->cpu_transcoder);
+ if (!intel_display_power_get_if_enabled(dev_priv, power_domain))
+ return false;
+ *power_domain_mask |= BIT(power_domain);
+
+ tmp = I915_READ(PIPECONF(pipe_config->cpu_transcoder));
+
+ return tmp & PIPECONF_ENABLE;
+}
+
+static bool bxt_get_dsi_transcoder_state(struct intel_crtc *crtc,
+ struct intel_crtc_state *pipe_config,
+ unsigned long *power_domain_mask)
+{
+ struct drm_device *dev = crtc->base.dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ enum intel_display_power_domain power_domain;
+ enum port port;
+ enum transcoder cpu_transcoder;
+ u32 tmp;
+
+ pipe_config->has_dsi_encoder = false;
+
+ for_each_port_masked(port, BIT(PORT_A) | BIT(PORT_C)) {
+ if (port == PORT_A)
+ cpu_transcoder = TRANSCODER_DSI_A;
+ else
+ cpu_transcoder = TRANSCODER_DSI_C;
+
+ power_domain = POWER_DOMAIN_TRANSCODER(cpu_transcoder);
+ if (!intel_display_power_get_if_enabled(dev_priv, power_domain))
+ continue;
+ *power_domain_mask |= BIT(power_domain);
+
+ /*
+ * The PLL needs to be enabled with a valid divider
+ * configuration, otherwise accessing DSI registers will hang
+ * the machine. See BSpec North Display Engine
+ * registers/MIPI[BXT]. We can break out here early, since we
+ * need the same DSI PLL to be enabled for both DSI ports.
+ */
+ if (!intel_dsi_pll_is_enabled(dev_priv))
+ break;
+
+ /* XXX: this works for video mode only */
+ tmp = I915_READ(BXT_MIPI_PORT_CTRL(port));
+ if (!(tmp & DPI_ENABLE))
+ continue;
+
+ tmp = I915_READ(MIPI_CTRL(port));
+ if ((tmp & BXT_PIPE_SELECT_MASK) != BXT_PIPE_SELECT(crtc->pipe))
+ continue;
+
+ pipe_config->cpu_transcoder = cpu_transcoder;
+ pipe_config->has_dsi_encoder = true;
break;
}
+
+ return pipe_config->has_dsi_encoder;
}
static void haswell_get_ddi_port_state(struct intel_crtc *crtc,
@@ -9909,11 +9959,10 @@
else
haswell_get_ddi_pll(dev_priv, port, pipe_config);
- if (pipe_config->shared_dpll >= 0) {
- pll = &dev_priv->shared_dplls[pipe_config->shared_dpll];
-
- WARN_ON(!pll->get_hw_state(dev_priv, pll,
- &pipe_config->dpll_hw_state));
+ pll = pipe_config->shared_dpll;
+ if (pll) {
+ WARN_ON(!pll->funcs.get_hw_state(dev_priv, pll,
+ &pipe_config->dpll_hw_state));
}
/*
@@ -9940,53 +9989,37 @@
struct drm_i915_private *dev_priv = dev->dev_private;
enum intel_display_power_domain power_domain;
unsigned long power_domain_mask;
- uint32_t tmp;
- bool ret;
+ bool active;
power_domain = POWER_DOMAIN_PIPE(crtc->pipe);
if (!intel_display_power_get_if_enabled(dev_priv, power_domain))
return false;
power_domain_mask = BIT(power_domain);
- ret = false;
+ pipe_config->shared_dpll = NULL;
- pipe_config->cpu_transcoder = (enum transcoder) crtc->pipe;
- pipe_config->shared_dpll = DPLL_ID_PRIVATE;
+ active = hsw_get_transcoder_state(crtc, pipe_config, &power_domain_mask);
- tmp = I915_READ(TRANS_DDI_FUNC_CTL(TRANSCODER_EDP));
- if (tmp & TRANS_DDI_FUNC_ENABLE) {
- enum pipe trans_edp_pipe;
- switch (tmp & TRANS_DDI_EDP_INPUT_MASK) {
- default:
- WARN(1, "unknown pipe linked to edp transcoder\n");
- case TRANS_DDI_EDP_INPUT_A_ONOFF:
- case TRANS_DDI_EDP_INPUT_A_ON:
- trans_edp_pipe = PIPE_A;
- break;
- case TRANS_DDI_EDP_INPUT_B_ONOFF:
- trans_edp_pipe = PIPE_B;
- break;
- case TRANS_DDI_EDP_INPUT_C_ONOFF:
- trans_edp_pipe = PIPE_C;
- break;
- }
-
- if (trans_edp_pipe == crtc->pipe)
- pipe_config->cpu_transcoder = TRANSCODER_EDP;
+ if (IS_BROXTON(dev_priv)) {
+ bxt_get_dsi_transcoder_state(crtc, pipe_config,
+ &power_domain_mask);
+ WARN_ON(active && pipe_config->has_dsi_encoder);
+ if (pipe_config->has_dsi_encoder)
+ active = true;
}
- power_domain = POWER_DOMAIN_TRANSCODER(pipe_config->cpu_transcoder);
- if (!intel_display_power_get_if_enabled(dev_priv, power_domain))
- goto out;
- power_domain_mask |= BIT(power_domain);
-
- tmp = I915_READ(PIPECONF(pipe_config->cpu_transcoder));
- if (!(tmp & PIPECONF_ENABLE))
+ if (!active)
goto out;
- haswell_get_ddi_port_state(crtc, pipe_config);
+ if (!pipe_config->has_dsi_encoder) {
+ haswell_get_ddi_port_state(crtc, pipe_config);
+ intel_get_pipe_timings(crtc, pipe_config);
+ }
- intel_get_pipe_timings(crtc, pipe_config);
+ intel_get_pipe_src_size(crtc, pipe_config);
+
+ pipe_config->gamma_mode =
+ I915_READ(GAMMA_MODE(crtc->pipe)) & GAMMA_MODE_MODE_MASK;
if (INTEL_INFO(dev)->gen >= 9) {
skl_init_scalers(dev, crtc, pipe_config);
@@ -10010,20 +10043,19 @@
pipe_config->ips_enabled = hsw_crtc_supports_ips(crtc) &&
(I915_READ(IPS_CTL) & IPS_ENABLE);
- if (pipe_config->cpu_transcoder != TRANSCODER_EDP) {
+ if (pipe_config->cpu_transcoder != TRANSCODER_EDP &&
+ !transcoder_is_dsi(pipe_config->cpu_transcoder)) {
pipe_config->pixel_multiplier =
I915_READ(PIPE_MULT(pipe_config->cpu_transcoder)) + 1;
} else {
pipe_config->pixel_multiplier = 1;
}
- ret = true;
-
out:
for_each_power_domain(power_domain, power_domain_mask)
intel_display_power_put(dev_priv, power_domain);
- return ret;
+ return active;
}
static void i845_update_cursor(struct drm_crtc *crtc, u32 base,
@@ -10216,21 +10248,6 @@
return true;
}
-static void intel_crtc_gamma_set(struct drm_crtc *crtc, u16 *red, u16 *green,
- u16 *blue, uint32_t start, uint32_t size)
-{
- int end = (start + size > 256) ? 256 : start + size, i;
- struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
-
- for (i = start; i < end; i++) {
- intel_crtc->lut_r[i] = red[i] >> 8;
- intel_crtc->lut_g[i] = green[i] >> 8;
- intel_crtc->lut_b[i] = blue[i] >> 8;
- }
-
- intel_crtc_load_lut(crtc);
-}
-
/* VESA 640x480x72Hz mode to set on the pipe */
static struct drm_display_mode load_detect_mode = {
DRM_MODE("640x480", DRM_MODE_TYPE_DEFAULT, 31500, 640, 664,
@@ -10718,19 +10735,18 @@
static void ironlake_pch_clock_get(struct intel_crtc *crtc,
struct intel_crtc_state *pipe_config)
{
- struct drm_device *dev = crtc->base.dev;
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
/* read out port_clock from the DPLL */
i9xx_crtc_clock_get(crtc, pipe_config);
/*
- * This value does not include pixel_multiplier.
- * We will check that port_clock and adjusted_mode.crtc_clock
- * agree once we know their relationship in the encoder's
- * get_config() function.
+ * In case there is an active pipe without active ports,
+ * we may need some idea for the dotclock anyway.
+ * Calculate one based on the FDI configuration.
*/
pipe_config->base.adjusted_mode.crtc_clock =
- intel_dotclock_calculate(intel_fdi_link_freq(dev) * 10000,
+ intel_dotclock_calculate(intel_fdi_link_freq(dev_priv, pipe_config),
&pipe_config->fdi_m_n);
}
@@ -10849,7 +10865,7 @@
struct drm_plane *primary = crtc->base.primary;
mutex_lock(&dev->struct_mutex);
- intel_unpin_fb_obj(work->old_fb, primary->state);
+ intel_unpin_fb_obj(work->old_fb, primary->state->rotation);
drm_gem_object_unreference(&work->pending_flip_obj->base);
if (work->flip_queued_req)
@@ -10923,9 +10939,10 @@
{
struct drm_device *dev = crtc->base.dev;
struct drm_i915_private *dev_priv = dev->dev_private;
+ unsigned reset_counter;
- if (i915_reset_in_progress(&dev_priv->gpu_error) ||
- crtc->reset_counter != atomic_read(&dev_priv->gpu_error.reset_counter))
+ reset_counter = i915_reset_counter(&dev_priv->gpu_error);
+ if (crtc->reset_counter != reset_counter)
return true;
/*
@@ -11003,7 +11020,7 @@
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
u32 flip_mask;
int ret;
@@ -11019,13 +11036,13 @@
flip_mask = MI_WAIT_FOR_PLANE_B_FLIP;
else
flip_mask = MI_WAIT_FOR_PLANE_A_FLIP;
- intel_ring_emit(ring, MI_WAIT_FOR_EVENT | flip_mask);
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_emit(ring, MI_DISPLAY_FLIP |
+ intel_ring_emit(engine, MI_WAIT_FOR_EVENT | flip_mask);
+ intel_ring_emit(engine, MI_NOOP);
+ intel_ring_emit(engine, MI_DISPLAY_FLIP |
MI_DISPLAY_FLIP_PLANE(intel_crtc->plane));
- intel_ring_emit(ring, fb->pitches[0]);
- intel_ring_emit(ring, intel_crtc->unpin_work->gtt_offset);
- intel_ring_emit(ring, 0); /* aux display base address, unused */
+ intel_ring_emit(engine, fb->pitches[0]);
+ intel_ring_emit(engine, intel_crtc->unpin_work->gtt_offset);
+ intel_ring_emit(engine, 0); /* aux display base address, unused */
intel_mark_page_flip_active(intel_crtc->unpin_work);
return 0;
@@ -11038,7 +11055,7 @@
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
u32 flip_mask;
int ret;
@@ -11051,13 +11068,13 @@
flip_mask = MI_WAIT_FOR_PLANE_B_FLIP;
else
flip_mask = MI_WAIT_FOR_PLANE_A_FLIP;
- intel_ring_emit(ring, MI_WAIT_FOR_EVENT | flip_mask);
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_emit(ring, MI_DISPLAY_FLIP_I915 |
+ intel_ring_emit(engine, MI_WAIT_FOR_EVENT | flip_mask);
+ intel_ring_emit(engine, MI_NOOP);
+ intel_ring_emit(engine, MI_DISPLAY_FLIP_I915 |
MI_DISPLAY_FLIP_PLANE(intel_crtc->plane));
- intel_ring_emit(ring, fb->pitches[0]);
- intel_ring_emit(ring, intel_crtc->unpin_work->gtt_offset);
- intel_ring_emit(ring, MI_NOOP);
+ intel_ring_emit(engine, fb->pitches[0]);
+ intel_ring_emit(engine, intel_crtc->unpin_work->gtt_offset);
+ intel_ring_emit(engine, MI_NOOP);
intel_mark_page_flip_active(intel_crtc->unpin_work);
return 0;
@@ -11070,7 +11087,7 @@
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
struct drm_i915_private *dev_priv = dev->dev_private;
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
uint32_t pf, pipesrc;
@@ -11084,10 +11101,10 @@
* Display Registers (which do not change across a page-flip)
* so we need only reprogram the base address.
*/
- intel_ring_emit(ring, MI_DISPLAY_FLIP |
+ intel_ring_emit(engine, MI_DISPLAY_FLIP |
MI_DISPLAY_FLIP_PLANE(intel_crtc->plane));
- intel_ring_emit(ring, fb->pitches[0]);
- intel_ring_emit(ring, intel_crtc->unpin_work->gtt_offset |
+ intel_ring_emit(engine, fb->pitches[0]);
+ intel_ring_emit(engine, intel_crtc->unpin_work->gtt_offset |
obj->tiling_mode);
/* XXX Enabling the panel-fitter across page-flip is so far
@@ -11096,7 +11113,7 @@
*/
pf = 0;
pipesrc = I915_READ(PIPESRC(intel_crtc->pipe)) & 0x0fff0fff;
- intel_ring_emit(ring, pf | pipesrc);
+ intel_ring_emit(engine, pf | pipesrc);
intel_mark_page_flip_active(intel_crtc->unpin_work);
return 0;
@@ -11109,7 +11126,7 @@
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
struct drm_i915_private *dev_priv = dev->dev_private;
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
uint32_t pf, pipesrc;
@@ -11119,10 +11136,10 @@
if (ret)
return ret;
- intel_ring_emit(ring, MI_DISPLAY_FLIP |
+ intel_ring_emit(engine, MI_DISPLAY_FLIP |
MI_DISPLAY_FLIP_PLANE(intel_crtc->plane));
- intel_ring_emit(ring, fb->pitches[0] | obj->tiling_mode);
- intel_ring_emit(ring, intel_crtc->unpin_work->gtt_offset);
+ intel_ring_emit(engine, fb->pitches[0] | obj->tiling_mode);
+ intel_ring_emit(engine, intel_crtc->unpin_work->gtt_offset);
/* Contrary to the suggestions in the documentation,
* "Enable Panel Fitter" does not seem to be required when page
@@ -11132,7 +11149,7 @@
*/
pf = 0;
pipesrc = I915_READ(PIPESRC(intel_crtc->pipe)) & 0x0fff0fff;
- intel_ring_emit(ring, pf | pipesrc);
+ intel_ring_emit(engine, pf | pipesrc);
intel_mark_page_flip_active(intel_crtc->unpin_work);
return 0;
@@ -11145,7 +11162,7 @@
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
uint32_t plane_bit = 0;
int len, ret;
@@ -11166,7 +11183,7 @@
}
len = 4;
- if (ring->id == RCS) {
+ if (engine->id == RCS) {
len += 6;
/*
* On Gen 8, SRM is now taking an extra dword to accommodate
@@ -11204,36 +11221,36 @@
* for the RCS also doesn't appear to drop events. Setting the DERRMR
* to zero does lead to lockups within MI_DISPLAY_FLIP.
*/
- if (ring->id == RCS) {
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
- intel_ring_emit_reg(ring, DERRMR);
- intel_ring_emit(ring, ~(DERRMR_PIPEA_PRI_FLIP_DONE |
- DERRMR_PIPEB_PRI_FLIP_DONE |
- DERRMR_PIPEC_PRI_FLIP_DONE));
+ if (engine->id == RCS) {
+ intel_ring_emit(engine, MI_LOAD_REGISTER_IMM(1));
+ intel_ring_emit_reg(engine, DERRMR);
+ intel_ring_emit(engine, ~(DERRMR_PIPEA_PRI_FLIP_DONE |
+ DERRMR_PIPEB_PRI_FLIP_DONE |
+ DERRMR_PIPEC_PRI_FLIP_DONE));
if (IS_GEN8(dev))
- intel_ring_emit(ring, MI_STORE_REGISTER_MEM_GEN8 |
+ intel_ring_emit(engine, MI_STORE_REGISTER_MEM_GEN8 |
MI_SRM_LRM_GLOBAL_GTT);
else
- intel_ring_emit(ring, MI_STORE_REGISTER_MEM |
+ intel_ring_emit(engine, MI_STORE_REGISTER_MEM |
MI_SRM_LRM_GLOBAL_GTT);
- intel_ring_emit_reg(ring, DERRMR);
- intel_ring_emit(ring, ring->scratch.gtt_offset + 256);
+ intel_ring_emit_reg(engine, DERRMR);
+ intel_ring_emit(engine, engine->scratch.gtt_offset + 256);
if (IS_GEN8(dev)) {
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, MI_NOOP);
+ intel_ring_emit(engine, 0);
+ intel_ring_emit(engine, MI_NOOP);
}
}
- intel_ring_emit(ring, MI_DISPLAY_FLIP_I915 | plane_bit);
- intel_ring_emit(ring, (fb->pitches[0] | obj->tiling_mode));
- intel_ring_emit(ring, intel_crtc->unpin_work->gtt_offset);
- intel_ring_emit(ring, (MI_NOOP));
+ intel_ring_emit(engine, MI_DISPLAY_FLIP_I915 | plane_bit);
+ intel_ring_emit(engine, (fb->pitches[0] | obj->tiling_mode));
+ intel_ring_emit(engine, intel_crtc->unpin_work->gtt_offset);
+ intel_ring_emit(engine, (MI_NOOP));
intel_mark_page_flip_active(intel_crtc->unpin_work);
return 0;
}
-static bool use_mmio_flip(struct intel_engine_cs *ring,
+static bool use_mmio_flip(struct intel_engine_cs *engine,
struct drm_i915_gem_object *obj)
{
/*
@@ -11244,10 +11261,10 @@
* So using MMIO flips there would disrupt this mechanism.
*/
- if (ring == NULL)
+ if (engine == NULL)
return true;
- if (INTEL_INFO(ring->dev)->gen < 5)
+ if (INTEL_INFO(engine->dev)->gen < 5)
return false;
if (i915.use_mmio_flip < 0)
@@ -11261,7 +11278,7 @@
false))
return true;
else
- return ring != i915_gem_request_get_ring(obj->last_write_req);
+ return engine != i915_gem_request_get_engine(obj->last_write_req);
}
static void skl_do_mmio_flip(struct intel_crtc *intel_crtc,
@@ -11379,7 +11396,6 @@
if (mmio_flip->req) {
WARN_ON(__i915_wait_request(mmio_flip->req,
- mmio_flip->crtc->reset_counter,
false, NULL,
&mmio_flip->i915->rps.mmioflips));
i915_gem_request_unreference__unlocked(mmio_flip->req);
@@ -11507,7 +11523,7 @@
struct drm_plane *primary = crtc->primary;
enum pipe pipe = intel_crtc->pipe;
struct intel_unpin_work *work;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
bool mmio_flip;
struct drm_i915_gem_request *request = NULL;
int ret;
@@ -11587,28 +11603,33 @@
if (ret)
goto cleanup;
+ intel_crtc->reset_counter = i915_reset_counter(&dev_priv->gpu_error);
+ if (__i915_reset_in_progress_or_wedged(intel_crtc->reset_counter)) {
+ ret = -EIO;
+ goto cleanup;
+ }
+
atomic_inc(&intel_crtc->unpin_work_count);
- intel_crtc->reset_counter = atomic_read(&dev_priv->gpu_error.reset_counter);
if (INTEL_INFO(dev)->gen >= 5 || IS_G4X(dev))
work->flip_count = I915_READ(PIPE_FLIPCOUNT_G4X(pipe)) + 1;
if (IS_VALLEYVIEW(dev) || IS_CHERRYVIEW(dev)) {
- ring = &dev_priv->ring[BCS];
+ engine = &dev_priv->engine[BCS];
if (obj->tiling_mode != intel_fb_obj(work->old_fb)->tiling_mode)
/* vlv: DISPLAY_FLIP fails to change tiling */
- ring = NULL;
+ engine = NULL;
} else if (IS_IVYBRIDGE(dev) || IS_HASWELL(dev)) {
- ring = &dev_priv->ring[BCS];
+ engine = &dev_priv->engine[BCS];
} else if (INTEL_INFO(dev)->gen >= 7) {
- ring = i915_gem_request_get_ring(obj->last_write_req);
- if (ring == NULL || ring->id != RCS)
- ring = &dev_priv->ring[BCS];
+ engine = i915_gem_request_get_engine(obj->last_write_req);
+ if (engine == NULL || engine->id != RCS)
+ engine = &dev_priv->engine[BCS];
} else {
- ring = &dev_priv->ring[RCS];
+ engine = &dev_priv->engine[RCS];
}
- mmio_flip = use_mmio_flip(ring, obj);
+ mmio_flip = use_mmio_flip(engine, obj);
/* When using CS flips, we want to emit semaphores between rings.
* However, when using mmio flips we will create a task to do the
@@ -11616,13 +11637,12 @@
* into the display plane and skip any waits.
*/
if (!mmio_flip) {
- ret = i915_gem_object_sync(obj, ring, &request);
+ ret = i915_gem_object_sync(obj, engine, &request);
if (ret)
goto cleanup_pending;
}
- ret = intel_pin_and_fence_fb_obj(crtc->primary, fb,
- crtc->primary->state);
+ ret = intel_pin_and_fence_fb_obj(fb, primary->state->rotation);
if (ret)
goto cleanup_pending;
@@ -11639,7 +11659,7 @@
obj->last_write_req);
} else {
if (!request) {
- request = i915_gem_request_alloc(ring, NULL);
+ request = i915_gem_request_alloc(engine, NULL);
if (IS_ERR(request)) {
ret = PTR_ERR(request);
goto cleanup_unpin;
@@ -11672,10 +11692,10 @@
return 0;
cleanup_unpin:
- intel_unpin_fb_obj(fb, crtc->primary->state);
+ intel_unpin_fb_obj(fb, crtc->primary->state->rotation);
cleanup_pending:
if (!IS_ERR_OR_NULL(request))
- i915_gem_request_cancel(request);
+ i915_add_request_no_flush(request);
atomic_dec(&intel_crtc->unpin_work_count);
mutex_unlock(&dev->struct_mutex);
cleanup:
@@ -11725,7 +11745,7 @@
if (ret == 0 && event) {
spin_lock_irq(&dev->event_lock);
- drm_send_vblank_event(dev, pipe, event);
+ drm_crtc_send_vblank_event(crtc, event);
spin_unlock_irq(&dev->event_lock);
}
}
@@ -11785,6 +11805,7 @@
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
struct drm_plane *plane = plane_state->plane;
struct drm_device *dev = crtc->dev;
+ struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_plane_state *old_plane_state =
to_intel_plane_state(plane->state);
int idx = intel_crtc->base.base.id, ret;
@@ -11833,42 +11854,43 @@
plane->base.id, was_visible, visible,
turn_off, turn_on, mode_changed);
- if (turn_on || turn_off) {
- pipe_config->wm_changed = true;
+ if (turn_on) {
+ pipe_config->update_wm_pre = true;
+
+ /* must disable cxsr around plane enable/disable */
+ if (plane->type != DRM_PLANE_TYPE_CURSOR)
+ pipe_config->disable_cxsr = true;
+ } else if (turn_off) {
+ pipe_config->update_wm_post = true;
/* must disable cxsr around plane enable/disable */
if (plane->type != DRM_PLANE_TYPE_CURSOR)
pipe_config->disable_cxsr = true;
} else if (intel_wm_need_update(plane, plane_state)) {
- pipe_config->wm_changed = true;
+ /* FIXME bollocks */
+ pipe_config->update_wm_pre = true;
+ pipe_config->update_wm_post = true;
}
+ /* Pre-gen9 platforms need two-step watermark updates */
+ if ((pipe_config->update_wm_pre || pipe_config->update_wm_post) &&
+ INTEL_INFO(dev)->gen < 9 && dev_priv->display.optimize_watermarks)
+ to_intel_crtc_state(crtc_state)->wm.need_postvbl_update = true;
+
if (visible || was_visible)
- intel_crtc->atomic.fb_bits |=
- to_intel_plane(plane)->frontbuffer_bit;
+ pipe_config->fb_bits |= to_intel_plane(plane)->frontbuffer_bit;
- switch (plane->type) {
- case DRM_PLANE_TYPE_PRIMARY:
- intel_crtc->atomic.post_enable_primary = turn_on;
- intel_crtc->atomic.update_fbc = true;
+ /*
+ * WaCxSRDisabledForSpriteScaling:ivb
+ *
+ * cstate->update_wm was already set above, so this flag will
+ * take effect when we commit and program watermarks.
+ */
+ if (plane->type == DRM_PLANE_TYPE_OVERLAY && IS_IVYBRIDGE(dev) &&
+ needs_scaling(to_intel_plane_state(plane_state)) &&
+ !needs_scaling(old_plane_state))
+ pipe_config->disable_lp_wm = true;
- break;
- case DRM_PLANE_TYPE_CURSOR:
- break;
- case DRM_PLANE_TYPE_OVERLAY:
- /*
- * WaCxSRDisabledForSpriteScaling:ivb
- *
- * cstate->update_wm was already set above, so this flag will
- * take effect when we commit and program watermarks.
- */
- if (IS_IVYBRIDGE(dev) &&
- needs_scaling(to_intel_plane_state(plane_state)) &&
- !needs_scaling(old_plane_state))
- pipe_config->disable_lp_wm = true;
-
- break;
- }
return 0;
}
@@ -11940,24 +11962,51 @@
}
if (mode_changed && !crtc_state->active)
- pipe_config->wm_changed = true;
+ pipe_config->update_wm_post = true;
if (mode_changed && crtc_state->enable &&
dev_priv->display.crtc_compute_clock &&
- !WARN_ON(pipe_config->shared_dpll != DPLL_ID_PRIVATE)) {
+ !WARN_ON(pipe_config->shared_dpll)) {
ret = dev_priv->display.crtc_compute_clock(intel_crtc,
pipe_config);
if (ret)
return ret;
}
- ret = 0;
- if (dev_priv->display.compute_pipe_wm) {
- ret = dev_priv->display.compute_pipe_wm(intel_crtc, state);
+ if (crtc_state->color_mgmt_changed) {
+ ret = intel_color_check(crtc, crtc_state);
if (ret)
return ret;
}
+ ret = 0;
+ if (dev_priv->display.compute_pipe_wm) {
+ ret = dev_priv->display.compute_pipe_wm(pipe_config);
+ if (ret) {
+ DRM_DEBUG_KMS("Target pipe watermarks are invalid\n");
+ return ret;
+ }
+ }
+
+ if (dev_priv->display.compute_intermediate_wm &&
+ !to_intel_atomic_state(state)->skip_intermediate_wm) {
+ if (WARN_ON(!dev_priv->display.compute_pipe_wm))
+ return 0;
+
+ /*
+ * Calculate 'intermediate' watermarks that satisfy both the
+ * old state and the new state. We can program these
+ * immediately.
+ */
+ ret = dev_priv->display.compute_intermediate_wm(crtc->dev,
+ intel_crtc,
+ pipe_config);
+ if (ret) {
+ DRM_DEBUG_KMS("No valid intermediate pipe watermarks are possible\n");
+ return ret;
+ }
+ }
+
if (INTEL_INFO(dev)->gen >= 9) {
if (mode_changed)
ret = skl_update_scaler_crtc(pipe_config);
@@ -11972,7 +12021,6 @@
static const struct drm_crtc_helper_funcs intel_helper_funcs = {
.mode_set_base_atomic = intel_pipe_set_base_atomic,
- .load_lut = intel_crtc_load_lut,
.atomic_begin = intel_begin_crtc_commit,
.atomic_flush = intel_finish_crtc_commit,
.atomic_check = intel_crtc_atomic_check,
@@ -11983,11 +12031,16 @@
struct intel_connector *connector;
for_each_intel_connector(dev, connector) {
+ if (connector->base.state->crtc)
+ drm_connector_unreference(&connector->base);
+
if (connector->base.encoder) {
connector->base.state->best_encoder =
connector->base.encoder;
connector->base.state->crtc =
connector->base.encoder->crtc;
+
+ drm_connector_reference(&connector->base);
} else {
connector->base.state->best_encoder = NULL;
connector->base.state->crtc = NULL;
@@ -12089,7 +12142,7 @@
DRM_DEBUG_KMS("[CRTC:%d]%s config %p for pipe %c\n", crtc->base.base.id,
context, pipe_config, pipe_name(crtc->pipe));
- DRM_DEBUG_KMS("cpu_transcoder: %c\n", transcoder_name(pipe_config->cpu_transcoder));
+ DRM_DEBUG_KMS("cpu_transcoder: %s\n", transcoder_name(pipe_config->cpu_transcoder));
DRM_DEBUG_KMS("pipe bpp: %i, dithering: %i\n",
pipe_config->pipe_bpp, pipe_config->dither);
DRM_DEBUG_KMS("fdi/pch: %i, lanes: %i, gmch_m: %u, gmch_n: %u, link_m: %u, link_n: %u, tu: %u\n",
@@ -12165,7 +12218,7 @@
pipe_config->dpll_hw_state.cfgcr1,
pipe_config->dpll_hw_state.cfgcr2);
} else if (HAS_DDI(dev)) {
- DRM_DEBUG_KMS("ddi_pll_sel: %u; dpll_hw_state: wrpll: 0x%x spll: 0x%x\n",
+ DRM_DEBUG_KMS("ddi_pll_sel: 0x%x; dpll_hw_state: wrpll: 0x%x spll: 0x%x\n",
pipe_config->ddi_pll_sel,
pipe_config->dpll_hw_state.wrpll,
pipe_config->dpll_hw_state.spll);
@@ -12268,7 +12321,7 @@
struct drm_crtc_state tmp_state;
struct intel_crtc_scaler_state scaler_state;
struct intel_dpll_hw_state dpll_hw_state;
- enum intel_dpll_id shared_dpll;
+ struct intel_shared_dpll *shared_dpll;
uint32_t ddi_pll_sel;
bool force_thru;
@@ -12538,6 +12591,15 @@
ret = false; \
}
+#define PIPE_CONF_CHECK_P(name) \
+ if (current_config->name != pipe_config->name) { \
+ INTEL_ERR_OR_DBG_KMS("mismatch in " #name " " \
+ "(expected %p, found %p)\n", \
+ current_config->name, \
+ pipe_config->name); \
+ ret = false; \
+ }
+
#define PIPE_CONF_CHECK_M_N(name) \
if (!intel_compare_link_m_n(¤t_config->name, \
&pipe_config->name,\
@@ -12558,6 +12620,11 @@
ret = false; \
}
+/* This is required for BDW+ where there is only one set of registers for
+ * switching between high and low RR.
+ * This macro can be used whenever a comparison has to be made between one
+ * hw state and multiple sw state variables.
+ */
#define PIPE_CONF_CHECK_M_N_ALT(name, alt_name) \
if (!intel_compare_link_m_n(¤t_config->name, \
&pipe_config->name, adjust) && \
@@ -12585,22 +12652,6 @@
ret = false; \
}
-/* This is required for BDW+ where there is only one set of registers for
- * switching between high and low RR.
- * This macro can be used whenever a comparison has to be made between one
- * hw state and multiple sw state variables.
- */
-#define PIPE_CONF_CHECK_I_ALT(name, alt_name) \
- if ((current_config->name != pipe_config->name) && \
- (current_config->alt_name != pipe_config->name)) { \
- INTEL_ERR_OR_DBG_KMS("mismatch in " #name " " \
- "(expected %i or %i, found %i)\n", \
- current_config->name, \
- current_config->alt_name, \
- pipe_config->name); \
- ret = false; \
- }
-
#define PIPE_CONF_CHECK_FLAGS(name, mask) \
if ((current_config->name ^ pipe_config->name) & (mask)) { \
INTEL_ERR_OR_DBG_KMS("mismatch in " #name "(" #mask ") " \
@@ -12681,7 +12732,7 @@
PIPE_CONF_CHECK_X(gmch_pfit.control);
/* pfit ratios are autocomputed by the hw on gen4+ */
if (INTEL_INFO(dev)->gen < 4)
- PIPE_CONF_CHECK_I(gmch_pfit.pgm_ratios);
+ PIPE_CONF_CHECK_X(gmch_pfit.pgm_ratios);
PIPE_CONF_CHECK_X(gmch_pfit.lvds_border_bits);
if (!adjust) {
@@ -12705,7 +12756,7 @@
PIPE_CONF_CHECK_X(ddi_pll_sel);
- PIPE_CONF_CHECK_I(shared_dpll);
+ PIPE_CONF_CHECK_P(shared_dpll);
PIPE_CONF_CHECK_X(dpll_hw_state.dpll);
PIPE_CONF_CHECK_X(dpll_hw_state.dpll_md);
PIPE_CONF_CHECK_X(dpll_hw_state.fp0);
@@ -12716,6 +12767,9 @@
PIPE_CONF_CHECK_X(dpll_hw_state.cfgcr1);
PIPE_CONF_CHECK_X(dpll_hw_state.cfgcr2);
+ PIPE_CONF_CHECK_X(dsi_pll.ctrl);
+ PIPE_CONF_CHECK_X(dsi_pll.div);
+
if (IS_G4X(dev) || INTEL_INFO(dev)->gen >= 5)
PIPE_CONF_CHECK_I(pipe_bpp);
@@ -12724,7 +12778,7 @@
#undef PIPE_CONF_CHECK_X
#undef PIPE_CONF_CHECK_I
-#undef PIPE_CONF_CHECK_I_ALT
+#undef PIPE_CONF_CHECK_P
#undef PIPE_CONF_CHECK_FLAGS
#undef PIPE_CONF_CHECK_CLOCK_FUZZY
#undef PIPE_CONF_QUIRK
@@ -12733,48 +12787,61 @@
return ret;
}
-static void check_wm_state(struct drm_device *dev)
+static void intel_pipe_config_sanity_check(struct drm_i915_private *dev_priv,
+ const struct intel_crtc_state *pipe_config)
{
+ if (pipe_config->has_pch_encoder) {
+ int fdi_dotclock = intel_dotclock_calculate(intel_fdi_link_freq(dev_priv, pipe_config),
+ &pipe_config->fdi_m_n);
+ int dotclock = pipe_config->base.adjusted_mode.crtc_clock;
+
+ /*
+ * FDI already provided one idea for the dotclock.
+ * Yell if the encoder disagrees.
+ */
+ WARN(!intel_fuzzy_clock_check(fdi_dotclock, dotclock),
+ "FDI dotclock and encoder dotclock mismatch, fdi: %i, encoder: %i\n",
+ fdi_dotclock, dotclock);
+ }
+}
+
+static void verify_wm_state(struct drm_crtc *crtc,
+ struct drm_crtc_state *new_state)
+{
+ struct drm_device *dev = crtc->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
struct skl_ddb_allocation hw_ddb, *sw_ddb;
- struct intel_crtc *intel_crtc;
+ struct skl_ddb_entry *hw_entry, *sw_entry;
+ struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
+ const enum pipe pipe = intel_crtc->pipe;
int plane;
- if (INTEL_INFO(dev)->gen < 9)
+ if (INTEL_INFO(dev)->gen < 9 || !new_state->active)
return;
skl_ddb_get_hw_state(dev_priv, &hw_ddb);
sw_ddb = &dev_priv->wm.skl_hw.ddb;
- for_each_intel_crtc(dev, intel_crtc) {
- struct skl_ddb_entry *hw_entry, *sw_entry;
- const enum pipe pipe = intel_crtc->pipe;
-
- if (!intel_crtc->active)
- continue;
-
- /* planes */
- for_each_plane(dev_priv, pipe, plane) {
- hw_entry = &hw_ddb.plane[pipe][plane];
- sw_entry = &sw_ddb->plane[pipe][plane];
-
- if (skl_ddb_entry_equal(hw_entry, sw_entry))
- continue;
-
- DRM_ERROR("mismatch in DDB state pipe %c plane %d "
- "(expected (%u,%u), found (%u,%u))\n",
- pipe_name(pipe), plane + 1,
- sw_entry->start, sw_entry->end,
- hw_entry->start, hw_entry->end);
- }
-
- /* cursor */
- hw_entry = &hw_ddb.plane[pipe][PLANE_CURSOR];
- sw_entry = &sw_ddb->plane[pipe][PLANE_CURSOR];
+ /* planes */
+ for_each_plane(dev_priv, pipe, plane) {
+ hw_entry = &hw_ddb.plane[pipe][plane];
+ sw_entry = &sw_ddb->plane[pipe][plane];
if (skl_ddb_entry_equal(hw_entry, sw_entry))
continue;
+ DRM_ERROR("mismatch in DDB state pipe %c plane %d "
+ "(expected (%u,%u), found (%u,%u))\n",
+ pipe_name(pipe), plane + 1,
+ sw_entry->start, sw_entry->end,
+ hw_entry->start, hw_entry->end);
+ }
+
+ /* cursor */
+ hw_entry = &hw_ddb.plane[pipe][PLANE_CURSOR];
+ sw_entry = &sw_ddb->plane[pipe][PLANE_CURSOR];
+
+ if (!skl_ddb_entry_equal(hw_entry, sw_entry)) {
DRM_ERROR("mismatch in DDB state pipe %c cursor "
"(expected (%u,%u), found (%u,%u))\n",
pipe_name(pipe),
@@ -12784,20 +12851,18 @@
}
static void
-check_connector_state(struct drm_device *dev,
- struct drm_atomic_state *old_state)
+verify_connector_state(struct drm_device *dev, struct drm_crtc *crtc)
{
- struct drm_connector_state *old_conn_state;
struct drm_connector *connector;
- int i;
- for_each_connector_in_state(old_state, connector, old_conn_state, i) {
+ drm_for_each_connector(connector, dev) {
struct drm_encoder *encoder = connector->encoder;
struct drm_connector_state *state = connector->state;
- /* This also checks the encoder/connector hw state with the
- * ->get_hw_state callbacks. */
- intel_connector_check_state(to_intel_connector(connector));
+ if (state->crtc != crtc)
+ continue;
+
+ intel_connector_verify_state(to_intel_connector(connector));
I915_STATE_WARN(state->best_encoder != encoder,
"connector's atomic encoder doesn't match legacy encoder\n");
@@ -12805,7 +12870,7 @@
}
static void
-check_encoder_state(struct drm_device *dev)
+verify_encoder_state(struct drm_device *dev)
{
struct intel_encoder *encoder;
struct intel_connector *connector;
@@ -12845,149 +12910,186 @@
}
static void
-check_crtc_state(struct drm_device *dev, struct drm_atomic_state *old_state)
+verify_crtc_state(struct drm_crtc *crtc,
+ struct drm_crtc_state *old_crtc_state,
+ struct drm_crtc_state *new_crtc_state)
{
+ struct drm_device *dev = crtc->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
struct intel_encoder *encoder;
- struct drm_crtc_state *old_crtc_state;
- struct drm_crtc *crtc;
- int i;
+ struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
+ struct intel_crtc_state *pipe_config, *sw_config;
+ struct drm_atomic_state *old_state;
+ bool active;
- for_each_crtc_in_state(old_state, crtc, old_crtc_state, i) {
- struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- struct intel_crtc_state *pipe_config, *sw_config;
- bool active;
+ old_state = old_crtc_state->state;
+ __drm_atomic_helper_crtc_destroy_state(old_crtc_state);
+ pipe_config = to_intel_crtc_state(old_crtc_state);
+ memset(pipe_config, 0, sizeof(*pipe_config));
+ pipe_config->base.crtc = crtc;
+ pipe_config->base.state = old_state;
- if (!needs_modeset(crtc->state) &&
- !to_intel_crtc_state(crtc->state)->update_pipe)
- continue;
+ DRM_DEBUG_KMS("[CRTC:%d]\n", crtc->base.id);
- __drm_atomic_helper_crtc_destroy_state(crtc, old_crtc_state);
- pipe_config = to_intel_crtc_state(old_crtc_state);
- memset(pipe_config, 0, sizeof(*pipe_config));
- pipe_config->base.crtc = crtc;
- pipe_config->base.state = old_state;
+ active = dev_priv->display.get_pipe_config(intel_crtc, pipe_config);
- DRM_DEBUG_KMS("[CRTC:%d]\n",
- crtc->base.id);
+ /* hw state is inconsistent with the pipe quirk */
+ if ((intel_crtc->pipe == PIPE_A && dev_priv->quirks & QUIRK_PIPEA_FORCE) ||
+ (intel_crtc->pipe == PIPE_B && dev_priv->quirks & QUIRK_PIPEB_FORCE))
+ active = new_crtc_state->active;
- active = dev_priv->display.get_pipe_config(intel_crtc,
- pipe_config);
+ I915_STATE_WARN(new_crtc_state->active != active,
+ "crtc active state doesn't match with hw state "
+ "(expected %i, found %i)\n", new_crtc_state->active, active);
- /* hw state is inconsistent with the pipe quirk */
- if ((intel_crtc->pipe == PIPE_A && dev_priv->quirks & QUIRK_PIPEA_FORCE) ||
- (intel_crtc->pipe == PIPE_B && dev_priv->quirks & QUIRK_PIPEB_FORCE))
- active = crtc->state->active;
+ I915_STATE_WARN(intel_crtc->active != new_crtc_state->active,
+ "transitional active state does not match atomic hw state "
+ "(expected %i, found %i)\n", new_crtc_state->active, intel_crtc->active);
- I915_STATE_WARN(crtc->state->active != active,
- "crtc active state doesn't match with hw state "
- "(expected %i, found %i)\n", crtc->state->active, active);
+ for_each_encoder_on_crtc(dev, crtc, encoder) {
+ enum pipe pipe;
- I915_STATE_WARN(intel_crtc->active != crtc->state->active,
- "transitional active state does not match atomic hw state "
- "(expected %i, found %i)\n", crtc->state->active, intel_crtc->active);
+ active = encoder->get_hw_state(encoder, &pipe);
+ I915_STATE_WARN(active != new_crtc_state->active,
+ "[ENCODER:%i] active %i with crtc active %i\n",
+ encoder->base.base.id, active, new_crtc_state->active);
- for_each_encoder_on_crtc(dev, crtc, encoder) {
- enum pipe pipe;
+ I915_STATE_WARN(active && intel_crtc->pipe != pipe,
+ "Encoder connected to wrong pipe %c\n",
+ pipe_name(pipe));
- active = encoder->get_hw_state(encoder, &pipe);
- I915_STATE_WARN(active != crtc->state->active,
- "[ENCODER:%i] active %i with crtc active %i\n",
- encoder->base.base.id, active, crtc->state->active);
+ if (active)
+ encoder->get_config(encoder, pipe_config);
+ }
- I915_STATE_WARN(active && intel_crtc->pipe != pipe,
- "Encoder connected to wrong pipe %c\n",
- pipe_name(pipe));
+ if (!new_crtc_state->active)
+ return;
- if (active)
- encoder->get_config(encoder, pipe_config);
- }
+ intel_pipe_config_sanity_check(dev_priv, pipe_config);
- if (!crtc->state->active)
- continue;
-
- sw_config = to_intel_crtc_state(crtc->state);
- if (!intel_pipe_config_compare(dev, sw_config,
- pipe_config, false)) {
- I915_STATE_WARN(1, "pipe state doesn't match!\n");
- intel_dump_pipe_config(intel_crtc, pipe_config,
- "[hw state]");
- intel_dump_pipe_config(intel_crtc, sw_config,
- "[sw state]");
- }
+ sw_config = to_intel_crtc_state(crtc->state);
+ if (!intel_pipe_config_compare(dev, sw_config,
+ pipe_config, false)) {
+ I915_STATE_WARN(1, "pipe state doesn't match!\n");
+ intel_dump_pipe_config(intel_crtc, pipe_config,
+ "[hw state]");
+ intel_dump_pipe_config(intel_crtc, sw_config,
+ "[sw state]");
}
}
static void
-check_shared_dpll_state(struct drm_device *dev)
+verify_single_dpll_state(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll,
+ struct drm_crtc *crtc,
+ struct drm_crtc_state *new_state)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_crtc *crtc;
struct intel_dpll_hw_state dpll_hw_state;
- int i;
+ unsigned crtc_mask;
+ bool active;
- for (i = 0; i < dev_priv->num_shared_dpll; i++) {
- struct intel_shared_dpll *pll = &dev_priv->shared_dplls[i];
- int enabled_crtcs = 0, active_crtcs = 0;
- bool active;
+ memset(&dpll_hw_state, 0, sizeof(dpll_hw_state));
- memset(&dpll_hw_state, 0, sizeof(dpll_hw_state));
+ DRM_DEBUG_KMS("%s\n", pll->name);
- DRM_DEBUG_KMS("%s\n", pll->name);
+ active = pll->funcs.get_hw_state(dev_priv, pll, &dpll_hw_state);
- active = pll->get_hw_state(dev_priv, pll, &dpll_hw_state);
-
- I915_STATE_WARN(pll->active > hweight32(pll->config.crtc_mask),
- "more active pll users than references: %i vs %i\n",
- pll->active, hweight32(pll->config.crtc_mask));
- I915_STATE_WARN(pll->active && !pll->on,
+ if (!(pll->flags & INTEL_DPLL_ALWAYS_ON)) {
+ I915_STATE_WARN(!pll->on && pll->active_mask,
"pll in active use but not on in sw tracking\n");
- I915_STATE_WARN(pll->on && !pll->active,
- "pll in on but not on in use in sw tracking\n");
+ I915_STATE_WARN(pll->on && !pll->active_mask,
+ "pll is on but not used by any active crtc\n");
I915_STATE_WARN(pll->on != active,
"pll on state mismatch (expected %i, found %i)\n",
pll->on, active);
+ }
- for_each_intel_crtc(dev, crtc) {
- if (crtc->base.state->enable && intel_crtc_to_shared_dpll(crtc) == pll)
- enabled_crtcs++;
- if (crtc->active && intel_crtc_to_shared_dpll(crtc) == pll)
- active_crtcs++;
- }
- I915_STATE_WARN(pll->active != active_crtcs,
- "pll active crtcs mismatch (expected %i, found %i)\n",
- pll->active, active_crtcs);
- I915_STATE_WARN(hweight32(pll->config.crtc_mask) != enabled_crtcs,
- "pll enabled crtcs mismatch (expected %i, found %i)\n",
- hweight32(pll->config.crtc_mask), enabled_crtcs);
+ if (!crtc) {
+ I915_STATE_WARN(pll->active_mask & ~pll->config.crtc_mask,
+ "more active pll users than references: %x vs %x\n",
+ pll->active_mask, pll->config.crtc_mask);
- I915_STATE_WARN(pll->on && memcmp(&pll->config.hw_state, &dpll_hw_state,
- sizeof(dpll_hw_state)),
- "pll hw state mismatch\n");
+ return;
+ }
+
+ crtc_mask = 1 << drm_crtc_index(crtc);
+
+ if (new_state->active)
+ I915_STATE_WARN(!(pll->active_mask & crtc_mask),
+ "pll active mismatch (expected pipe %c in active mask 0x%02x)\n",
+ pipe_name(drm_crtc_index(crtc)), pll->active_mask);
+ else
+ I915_STATE_WARN(pll->active_mask & crtc_mask,
+ "pll active mismatch (didn't expect pipe %c in active mask 0x%02x)\n",
+ pipe_name(drm_crtc_index(crtc)), pll->active_mask);
+
+ I915_STATE_WARN(!(pll->config.crtc_mask & crtc_mask),
+ "pll enabled crtcs mismatch (expected 0x%x in 0x%02x)\n",
+ crtc_mask, pll->config.crtc_mask);
+
+ I915_STATE_WARN(pll->on && memcmp(&pll->config.hw_state,
+ &dpll_hw_state,
+ sizeof(dpll_hw_state)),
+ "pll hw state mismatch\n");
+}
+
+static void
+verify_shared_dpll_state(struct drm_device *dev, struct drm_crtc *crtc,
+ struct drm_crtc_state *old_crtc_state,
+ struct drm_crtc_state *new_crtc_state)
+{
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct intel_crtc_state *old_state = to_intel_crtc_state(old_crtc_state);
+ struct intel_crtc_state *new_state = to_intel_crtc_state(new_crtc_state);
+
+ if (new_state->shared_dpll)
+ verify_single_dpll_state(dev_priv, new_state->shared_dpll, crtc, new_crtc_state);
+
+ if (old_state->shared_dpll &&
+ old_state->shared_dpll != new_state->shared_dpll) {
+ unsigned crtc_mask = 1 << drm_crtc_index(crtc);
+ struct intel_shared_dpll *pll = old_state->shared_dpll;
+
+ I915_STATE_WARN(pll->active_mask & crtc_mask,
+ "pll active mismatch (didn't expect pipe %c in active mask)\n",
+ pipe_name(drm_crtc_index(crtc)));
+ I915_STATE_WARN(pll->config.crtc_mask & crtc_mask,
+ "pll enabled crtcs mismatch (found %x in enabled mask)\n",
+ pipe_name(drm_crtc_index(crtc)));
}
}
static void
-intel_modeset_check_state(struct drm_device *dev,
- struct drm_atomic_state *old_state)
+intel_modeset_verify_crtc(struct drm_crtc *crtc,
+ struct drm_crtc_state *old_state,
+ struct drm_crtc_state *new_state)
{
- check_wm_state(dev);
- check_connector_state(dev, old_state);
- check_encoder_state(dev);
- check_crtc_state(dev, old_state);
- check_shared_dpll_state(dev);
+ if (!needs_modeset(new_state) &&
+ !to_intel_crtc_state(new_state)->update_pipe)
+ return;
+
+ verify_wm_state(crtc, new_state);
+ verify_connector_state(crtc->dev, crtc);
+ verify_crtc_state(crtc, old_state, new_state);
+ verify_shared_dpll_state(crtc->dev, crtc, old_state, new_state);
}
-void ironlake_check_encoder_dotclock(const struct intel_crtc_state *pipe_config,
- int dotclock)
+static void
+verify_disabled_dpll_state(struct drm_device *dev)
{
- /*
- * FDI already provided one idea for the dotclock.
- * Yell if the encoder disagrees.
- */
- WARN(!intel_fuzzy_clock_check(pipe_config->base.adjusted_mode.crtc_clock, dotclock),
- "FDI dotclock and encoder dotclock mismatch, fdi: %i, encoder: %i\n",
- pipe_config->base.adjusted_mode.crtc_clock, dotclock);
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ int i;
+
+ for (i = 0; i < dev_priv->num_shared_dpll; i++)
+ verify_single_dpll_state(dev_priv, &dev_priv->shared_dplls[i], NULL, NULL);
+}
+
+static void
+intel_modeset_verify_disabled(struct drm_device *dev)
+{
+ verify_encoder_state(dev);
+ verify_connector_state(dev, NULL);
+ verify_disabled_dpll_state(dev);
}
static void update_scanline_offset(struct intel_crtc *crtc)
@@ -13042,20 +13144,21 @@
for_each_crtc_in_state(state, crtc, crtc_state, i) {
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- int old_dpll = to_intel_crtc_state(crtc->state)->shared_dpll;
+ struct intel_shared_dpll *old_dpll =
+ to_intel_crtc_state(crtc->state)->shared_dpll;
if (!needs_modeset(crtc_state))
continue;
- to_intel_crtc_state(crtc_state)->shared_dpll = DPLL_ID_PRIVATE;
+ to_intel_crtc_state(crtc_state)->shared_dpll = NULL;
- if (old_dpll == DPLL_ID_PRIVATE)
+ if (!old_dpll)
continue;
if (!shared_dpll)
shared_dpll = intel_atomic_get_shared_dpll_state(state);
- shared_dpll[old_dpll].crtc_mask &= ~(1 << intel_crtc->pipe);
+ intel_shared_dpll_config_put(shared_dpll, old_dpll, intel_crtc);
}
}
@@ -13267,9 +13370,6 @@
struct intel_crtc_state *pipe_config =
to_intel_crtc_state(crtc_state);
- memset(&to_intel_crtc(crtc)->atomic, 0,
- sizeof(struct intel_crtc_atomic_commit));
-
/* Catch I915_MODE_FLAG_INHERITED */
if (crtc_state->mode.private_flags != crtc->state->mode.private_flags)
crtc_state->mode_changed = true;
@@ -13335,7 +13435,7 @@
static int intel_atomic_prepare_commit(struct drm_device *dev,
struct drm_atomic_state *state,
- bool async)
+ bool nonblock)
{
struct drm_i915_private *dev_priv = dev->dev_private;
struct drm_plane_state *plane_state;
@@ -13344,8 +13444,8 @@
struct drm_crtc *crtc;
int i, ret;
- if (async) {
- DRM_DEBUG_KMS("i915 does not yet support async commit\n");
+ if (nonblock) {
+ DRM_DEBUG_KMS("i915 does not yet support nonblocking commit\n");
return -EINVAL;
}
@@ -13366,12 +13466,9 @@
return ret;
ret = drm_atomic_helper_prepare_planes(dev, state);
- if (!ret && !async && !i915_reset_in_progress(&dev_priv->gpu_error)) {
- u32 reset_counter;
+ mutex_unlock(&dev->struct_mutex);
- reset_counter = atomic_read(&dev_priv->gpu_error.reset_counter);
- mutex_unlock(&dev->struct_mutex);
-
+ if (!ret && !nonblock) {
for_each_plane_in_state(state, plane, plane_state, i) {
struct intel_plane_state *intel_plane_state =
to_intel_plane_state(plane_state);
@@ -13380,25 +13477,18 @@
continue;
ret = __i915_wait_request(intel_plane_state->wait_req,
- reset_counter, true,
- NULL, NULL);
-
- /* Swallow -EIO errors to allow updates during hw lockup. */
- if (ret == -EIO)
- ret = 0;
-
- if (ret)
+ true, NULL, NULL);
+ if (ret) {
+ /* Any hang should be swallowed by the wait */
+ WARN_ON(ret == -EIO);
+ mutex_lock(&dev->struct_mutex);
+ drm_atomic_helper_cleanup_planes(dev, state);
+ mutex_unlock(&dev->struct_mutex);
break;
+ }
}
-
- if (!ret)
- return 0;
-
- mutex_lock(&dev->struct_mutex);
- drm_atomic_helper_cleanup_planes(dev, state);
}
- mutex_unlock(&dev->struct_mutex);
return ret;
}
@@ -13440,7 +13530,7 @@
drm_crtc_vblank_count(crtc),
msecs_to_jiffies(50));
- WARN_ON(!lret);
+ WARN(!lret, "pipe %c vblank wait timed out\n", pipe_name(pipe));
drm_crtc_vblank_put(crtc);
}
@@ -13453,12 +13543,12 @@
return true;
/* wm changes, need vblank before final wm's */
- if (crtc_state->wm_changed)
+ if (crtc_state->update_wm_post)
return true;
/*
* cxsr is re-enabled after vblank.
- * This is already handled by crtc_state->wm_changed,
+ * This is already handled by crtc_state->update_wm_post,
* but added for clarity.
*/
if (crtc_state->disable_cxsr)
@@ -13471,39 +13561,41 @@
* intel_atomic_commit - commit validated state object
* @dev: DRM device
* @state: the top-level driver state object
- * @async: asynchronous commit
+ * @nonblock: nonblocking commit
*
* This function commits a top-level state object that has been validated
* with drm_atomic_helper_check().
*
* FIXME: Atomic modeset support for i915 is not yet complete. At the moment
* we can only handle plane-related operations and do not yet support
- * asynchronous commit.
+ * nonblocking commit.
*
* RETURNS
* Zero for success or -errno.
*/
static int intel_atomic_commit(struct drm_device *dev,
struct drm_atomic_state *state,
- bool async)
+ bool nonblock)
{
struct intel_atomic_state *intel_state = to_intel_atomic_state(state);
struct drm_i915_private *dev_priv = dev->dev_private;
- struct drm_crtc_state *crtc_state;
+ struct drm_crtc_state *old_crtc_state;
struct drm_crtc *crtc;
+ struct intel_crtc_state *intel_cstate;
int ret = 0, i;
bool hw_check = intel_state->modeset;
unsigned long put_domains[I915_MAX_PIPES] = {};
unsigned crtc_vblank_mask = 0;
- ret = intel_atomic_prepare_commit(dev, state, async);
+ ret = intel_atomic_prepare_commit(dev, state, nonblock);
if (ret) {
DRM_DEBUG_ATOMIC("Preparing state failed with %i\n", ret);
return ret;
}
drm_atomic_helper_swap_state(dev, state);
- dev_priv->wm.config = to_intel_atomic_state(state)->wm_config;
+ dev_priv->wm.config = intel_state->wm_config;
+ intel_shared_dpll_commit(state);
if (intel_state->modeset) {
memcpy(dev_priv->min_pixclk, intel_state->min_pixclk,
@@ -13514,7 +13606,7 @@
intel_display_power_get(dev_priv, POWER_DOMAIN_MODESET);
}
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
+ for_each_crtc_in_state(state, crtc, old_crtc_state, i) {
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
if (needs_modeset(crtc->state) ||
@@ -13529,10 +13621,10 @@
if (!needs_modeset(crtc->state))
continue;
- intel_pre_plane_update(to_intel_crtc_state(crtc_state));
+ intel_pre_plane_update(to_intel_crtc_state(old_crtc_state));
- if (crtc_state->active) {
- intel_crtc_disable_planes(crtc, crtc_state->plane_mask);
+ if (old_crtc_state->active) {
+ intel_crtc_disable_planes(crtc, old_crtc_state->plane_mask);
dev_priv->display.crtc_disable(crtc);
intel_crtc->active = false;
intel_fbc_disable(intel_crtc);
@@ -13555,17 +13647,17 @@
intel_modeset_update_crtc_state(state);
if (intel_state->modeset) {
- intel_shared_dpll_commit(state);
-
drm_atomic_helper_update_legacy_modeset_state(state->dev, state);
if (dev_priv->display.modeset_commit_cdclk &&
intel_state->dev_cdclk != dev_priv->cdclk_freq)
dev_priv->display.modeset_commit_cdclk(state);
+
+ intel_modeset_verify_disabled(dev);
}
/* Now enable the clocks, plane, pipe, and connectors that we set up. */
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
+ for_each_crtc_in_state(state, crtc, old_crtc_state, i) {
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
bool modeset = needs_modeset(crtc->state);
struct intel_crtc_state *pipe_config =
@@ -13578,14 +13670,15 @@
}
if (!modeset)
- intel_pre_plane_update(to_intel_crtc_state(crtc_state));
+ intel_pre_plane_update(to_intel_crtc_state(old_crtc_state));
- if (crtc->state->active && intel_crtc->atomic.update_fbc)
+ if (crtc->state->active &&
+ drm_atomic_get_existing_plane_state(state, crtc->primary))
intel_fbc_enable(intel_crtc);
if (crtc->state->active &&
(crtc->state->planes_changed || update_pipe))
- drm_atomic_helper_commit_planes_on_crtc(crtc_state);
+ drm_atomic_helper_commit_planes_on_crtc(old_crtc_state);
if (pipe_config->base.active && needs_vblank_wait(pipe_config))
crtc_vblank_mask |= 1 << i;
@@ -13596,11 +13689,27 @@
if (!state->legacy_cursor_update)
intel_atomic_wait_for_vblanks(dev, dev_priv, crtc_vblank_mask);
- for_each_crtc_in_state(state, crtc, crtc_state, i) {
- intel_post_plane_update(to_intel_crtc(crtc));
+ /*
+ * Now that the vblank has passed, we can go ahead and program the
+ * optimal watermarks on platforms that need two-step watermark
+ * programming.
+ *
+ * TODO: Move this (and other cleanup) to an async worker eventually.
+ */
+ for_each_crtc_in_state(state, crtc, old_crtc_state, i) {
+ intel_cstate = to_intel_crtc_state(crtc->state);
+
+ if (dev_priv->display.optimize_watermarks)
+ dev_priv->display.optimize_watermarks(intel_cstate);
+ }
+
+ for_each_crtc_in_state(state, crtc, old_crtc_state, i) {
+ intel_post_plane_update(to_intel_crtc_state(old_crtc_state));
if (put_domains[i])
modeset_put_power_domains(dev_priv, put_domains[i]);
+
+ intel_modeset_verify_crtc(crtc, old_crtc_state, crtc->state);
}
if (intel_state->modeset)
@@ -13610,9 +13719,6 @@
drm_atomic_helper_cleanup_planes(dev, state);
mutex_unlock(&dev->struct_mutex);
- if (hw_check)
- intel_modeset_check_state(dev, state);
-
drm_atomic_state_free(state);
/* As one of the primary mmio accessors, KMS has a high likelihood
@@ -13672,116 +13778,15 @@
#undef for_each_intel_crtc_masked
static const struct drm_crtc_funcs intel_crtc_funcs = {
- .gamma_set = intel_crtc_gamma_set,
+ .gamma_set = drm_atomic_helper_legacy_gamma_set,
.set_config = drm_atomic_helper_set_config,
+ .set_property = drm_atomic_helper_crtc_set_property,
.destroy = intel_crtc_destroy,
.page_flip = intel_crtc_page_flip,
.atomic_duplicate_state = intel_crtc_duplicate_state,
.atomic_destroy_state = intel_crtc_destroy_state,
};
-static bool ibx_pch_dpll_get_hw_state(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll,
- struct intel_dpll_hw_state *hw_state)
-{
- uint32_t val;
-
- if (!intel_display_power_get_if_enabled(dev_priv, POWER_DOMAIN_PLLS))
- return false;
-
- val = I915_READ(PCH_DPLL(pll->id));
- hw_state->dpll = val;
- hw_state->fp0 = I915_READ(PCH_FP0(pll->id));
- hw_state->fp1 = I915_READ(PCH_FP1(pll->id));
-
- intel_display_power_put(dev_priv, POWER_DOMAIN_PLLS);
-
- return val & DPLL_VCO_ENABLE;
-}
-
-static void ibx_pch_dpll_mode_set(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll)
-{
- I915_WRITE(PCH_FP0(pll->id), pll->config.hw_state.fp0);
- I915_WRITE(PCH_FP1(pll->id), pll->config.hw_state.fp1);
-}
-
-static void ibx_pch_dpll_enable(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll)
-{
- /* PCH refclock must be enabled first */
- ibx_assert_pch_refclk_enabled(dev_priv);
-
- I915_WRITE(PCH_DPLL(pll->id), pll->config.hw_state.dpll);
-
- /* Wait for the clocks to stabilize. */
- POSTING_READ(PCH_DPLL(pll->id));
- udelay(150);
-
- /* The pixel multiplier can only be updated once the
- * DPLL is enabled and the clocks are stable.
- *
- * So write it again.
- */
- I915_WRITE(PCH_DPLL(pll->id), pll->config.hw_state.dpll);
- POSTING_READ(PCH_DPLL(pll->id));
- udelay(200);
-}
-
-static void ibx_pch_dpll_disable(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll)
-{
- struct drm_device *dev = dev_priv->dev;
- struct intel_crtc *crtc;
-
- /* Make sure no transcoder isn't still depending on us. */
- for_each_intel_crtc(dev, crtc) {
- if (intel_crtc_to_shared_dpll(crtc) == pll)
- assert_pch_transcoder_disabled(dev_priv, crtc->pipe);
- }
-
- I915_WRITE(PCH_DPLL(pll->id), 0);
- POSTING_READ(PCH_DPLL(pll->id));
- udelay(200);
-}
-
-static char *ibx_pch_dpll_names[] = {
- "PCH DPLL A",
- "PCH DPLL B",
-};
-
-static void ibx_pch_dpll_init(struct drm_device *dev)
-{
- struct drm_i915_private *dev_priv = dev->dev_private;
- int i;
-
- dev_priv->num_shared_dpll = 2;
-
- for (i = 0; i < dev_priv->num_shared_dpll; i++) {
- dev_priv->shared_dplls[i].id = i;
- dev_priv->shared_dplls[i].name = ibx_pch_dpll_names[i];
- dev_priv->shared_dplls[i].mode_set = ibx_pch_dpll_mode_set;
- dev_priv->shared_dplls[i].enable = ibx_pch_dpll_enable;
- dev_priv->shared_dplls[i].disable = ibx_pch_dpll_disable;
- dev_priv->shared_dplls[i].get_hw_state =
- ibx_pch_dpll_get_hw_state;
- }
-}
-
-static void intel_shared_dpll_init(struct drm_device *dev)
-{
- struct drm_i915_private *dev_priv = dev->dev_private;
-
- if (HAS_DDI(dev))
- intel_ddi_pll_init(dev);
- else if (HAS_PCH_IBX(dev) || HAS_PCH_CPT(dev))
- ibx_pch_dpll_init(dev);
- else
- dev_priv->num_shared_dpll = 0;
-
- BUG_ON(dev_priv->num_shared_dpll > I915_NUM_PLLS);
-}
-
/**
* intel_prepare_plane_fb - Prepare fb for usage on plane
* @plane: drm plane to prepare for
@@ -13827,10 +13832,11 @@
*/
if (needs_modeset(crtc_state))
ret = i915_gem_object_wait_rendering(old_obj, true);
-
- /* Swallow -EIO errors to allow updates during hw lockup. */
- if (ret && ret != -EIO)
+ if (ret) {
+ /* GPU hangs should have been swallowed by the wait */
+ WARN_ON(ret == -EIO);
return ret;
+ }
}
/* For framebuffer backed by dmabuf, wait for fence */
@@ -13855,7 +13861,7 @@
if (ret)
DRM_DEBUG_KMS("failed to attach phys object\n");
} else {
- ret = intel_pin_and_fence_fb_obj(plane, fb, new_state);
+ ret = intel_pin_and_fence_fb_obj(fb, new_state->rotation);
}
if (ret == 0) {
@@ -13899,7 +13905,7 @@
if (old_obj && (plane->type != DRM_PLANE_TYPE_CURSOR ||
!INTEL_INFO(dev)->cursor_needs_physical))
- intel_unpin_fb_obj(old_state->fb, old_state);
+ intel_unpin_fb_obj(old_state->fb, old_state->rotation);
/* prepare_fb aborted? */
if ((old_obj && (old_obj->frontbuffer_bits & intel_plane->frontbuffer_bit)) ||
@@ -13907,7 +13913,6 @@
i915_gem_track_fb(old_obj, obj, intel_plane->frontbuffer_bit);
i915_gem_request_assign(&old_intel_state->wait_req, NULL);
-
}
int
@@ -13982,6 +13987,11 @@
if (modeset)
return;
+ if (crtc->state->color_mgmt_changed || to_intel_crtc_state(crtc->state)->update_pipe) {
+ intel_color_set_csc(crtc->state);
+ intel_color_load_luts(crtc->state);
+ }
+
if (to_intel_crtc_state(crtc->state)->update_pipe)
intel_update_pipe_config(intel_crtc, old_intel_state);
else if (INTEL_INFO(dev)->gen >= 9)
@@ -14025,20 +14035,19 @@
static struct drm_plane *intel_primary_plane_create(struct drm_device *dev,
int pipe)
{
- struct intel_plane *primary;
- struct intel_plane_state *state;
+ struct intel_plane *primary = NULL;
+ struct intel_plane_state *state = NULL;
const uint32_t *intel_primary_formats;
unsigned int num_formats;
+ int ret;
primary = kzalloc(sizeof(*primary), GFP_KERNEL);
- if (primary == NULL)
- return NULL;
+ if (!primary)
+ goto fail;
state = intel_create_plane_state(&primary->base);
- if (!state) {
- kfree(primary);
- return NULL;
- }
+ if (!state)
+ goto fail;
primary->base.state = &state->base;
primary->can_scale = false;
@@ -14080,10 +14089,12 @@
primary->disable_plane = i9xx_disable_primary_plane;
}
- drm_universal_plane_init(dev, &primary->base, 0,
- &intel_plane_funcs,
- intel_primary_formats, num_formats,
- DRM_PLANE_TYPE_PRIMARY, NULL);
+ ret = drm_universal_plane_init(dev, &primary->base, 0,
+ &intel_plane_funcs,
+ intel_primary_formats, num_formats,
+ DRM_PLANE_TYPE_PRIMARY, NULL);
+ if (ret)
+ goto fail;
if (INTEL_INFO(dev)->gen >= 4)
intel_create_rotation_property(dev, primary);
@@ -14091,6 +14102,12 @@
drm_plane_helper_add(&primary->base, &intel_plane_helper_funcs);
return &primary->base;
+
+fail:
+ kfree(state);
+ kfree(primary);
+
+ return NULL;
}
void intel_create_rotation_property(struct drm_device *dev, struct intel_plane *plane)
@@ -14207,18 +14224,17 @@
static struct drm_plane *intel_cursor_plane_create(struct drm_device *dev,
int pipe)
{
- struct intel_plane *cursor;
- struct intel_plane_state *state;
+ struct intel_plane *cursor = NULL;
+ struct intel_plane_state *state = NULL;
+ int ret;
cursor = kzalloc(sizeof(*cursor), GFP_KERNEL);
- if (cursor == NULL)
- return NULL;
+ if (!cursor)
+ goto fail;
state = intel_create_plane_state(&cursor->base);
- if (!state) {
- kfree(cursor);
- return NULL;
- }
+ if (!state)
+ goto fail;
cursor->base.state = &state->base;
cursor->can_scale = false;
@@ -14230,11 +14246,13 @@
cursor->update_plane = intel_update_cursor_plane;
cursor->disable_plane = intel_disable_cursor_plane;
- drm_universal_plane_init(dev, &cursor->base, 0,
- &intel_plane_funcs,
- intel_cursor_formats,
- ARRAY_SIZE(intel_cursor_formats),
- DRM_PLANE_TYPE_CURSOR, NULL);
+ ret = drm_universal_plane_init(dev, &cursor->base, 0,
+ &intel_plane_funcs,
+ intel_cursor_formats,
+ ARRAY_SIZE(intel_cursor_formats),
+ DRM_PLANE_TYPE_CURSOR, NULL);
+ if (ret)
+ goto fail;
if (INTEL_INFO(dev)->gen >= 4) {
if (!dev->mode_config.rotation_property)
@@ -14254,6 +14272,12 @@
drm_plane_helper_add(&cursor->base, &intel_plane_helper_funcs);
return &cursor->base;
+
+fail:
+ kfree(state);
+ kfree(cursor);
+
+ return NULL;
}
static void skl_init_scalers(struct drm_device *dev, struct intel_crtc *intel_crtc,
@@ -14279,7 +14303,7 @@
struct intel_crtc_state *crtc_state = NULL;
struct drm_plane *primary = NULL;
struct drm_plane *cursor = NULL;
- int i, ret;
+ int ret;
intel_crtc = kzalloc(sizeof(*intel_crtc), GFP_KERNEL);
if (intel_crtc == NULL)
@@ -14315,13 +14339,6 @@
if (ret)
goto fail;
- drm_mode_crtc_set_gamma_size(&intel_crtc->base, 256);
- for (i = 0; i < 256; i++) {
- intel_crtc->lut_r[i] = i;
- intel_crtc->lut_g[i] = i;
- intel_crtc->lut_b[i] = i;
- }
-
/*
* On gen2/3 only plane A can do fbc, but the panel fitter and lvds port
* is hooked to pipe B. Hence we want plane A feeding pipe B.
@@ -14346,6 +14363,8 @@
drm_crtc_helper_add(&intel_crtc->base, &intel_helper_funcs);
+ intel_color_init(&intel_crtc->base);
+
WARN_ON(drm_crtc_index(&intel_crtc->base) != intel_crtc->pipe);
return;
@@ -14470,6 +14489,8 @@
intel_ddi_init(dev, PORT_A);
intel_ddi_init(dev, PORT_B);
intel_ddi_init(dev, PORT_C);
+
+ intel_dsi_init(dev);
} else if (HAS_DDI(dev)) {
int found;
@@ -14839,6 +14860,8 @@
drm_helper_mode_fill_fb_struct(&intel_fb->base, mode_cmd);
intel_fb->obj = obj;
+ intel_fill_fb_info(dev_priv, &intel_fb->base);
+
ret = drm_framebuffer_init(dev, &intel_fb->base, &intel_fb_funcs);
if (ret) {
DRM_ERROR("framebuffer init failed %d\n", ret);
@@ -14859,8 +14882,7 @@
struct drm_i915_gem_object *obj;
struct drm_mode_fb_cmd2 mode_cmd = *user_mode_cmd;
- obj = to_intel_bo(drm_gem_object_lookup(dev, filp,
- mode_cmd.handles[0]));
+ obj = to_intel_bo(drm_gem_object_lookup(filp, mode_cmd.handles[0]));
if (&obj->base == NULL)
return ERR_PTR(-ENOENT);
@@ -14886,23 +14908,13 @@
.atomic_state_clear = intel_atomic_state_clear,
};
-/* Set up chip specific display functions */
-static void intel_init_display(struct drm_device *dev)
+/**
+ * intel_init_display_hooks - initialize the display modesetting hooks
+ * @dev_priv: device private
+ */
+void intel_init_display_hooks(struct drm_i915_private *dev_priv)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
-
- if (HAS_PCH_SPLIT(dev) || IS_G4X(dev))
- dev_priv->display.find_dpll = g4x_find_best_dpll;
- else if (IS_CHERRYVIEW(dev))
- dev_priv->display.find_dpll = chv_find_best_dpll;
- else if (IS_VALLEYVIEW(dev))
- dev_priv->display.find_dpll = vlv_find_best_dpll;
- else if (IS_PINEVIEW(dev))
- dev_priv->display.find_dpll = pnv_find_best_dpll;
- else
- dev_priv->display.find_dpll = i9xx_find_best_dpll;
-
- if (INTEL_INFO(dev)->gen >= 9) {
+ if (INTEL_INFO(dev_priv)->gen >= 9) {
dev_priv->display.get_pipe_config = haswell_get_pipe_config;
dev_priv->display.get_initial_plane_config =
skylake_get_initial_plane_config;
@@ -14910,7 +14922,7 @@
haswell_crtc_compute_clock;
dev_priv->display.crtc_enable = haswell_crtc_enable;
dev_priv->display.crtc_disable = haswell_crtc_disable;
- } else if (HAS_DDI(dev)) {
+ } else if (HAS_DDI(dev_priv)) {
dev_priv->display.get_pipe_config = haswell_get_pipe_config;
dev_priv->display.get_initial_plane_config =
ironlake_get_initial_plane_config;
@@ -14918,7 +14930,7 @@
haswell_crtc_compute_clock;
dev_priv->display.crtc_enable = haswell_crtc_enable;
dev_priv->display.crtc_disable = haswell_crtc_disable;
- } else if (HAS_PCH_SPLIT(dev)) {
+ } else if (HAS_PCH_SPLIT(dev_priv)) {
dev_priv->display.get_pipe_config = ironlake_get_pipe_config;
dev_priv->display.get_initial_plane_config =
ironlake_get_initial_plane_config;
@@ -14926,106 +14938,134 @@
ironlake_crtc_compute_clock;
dev_priv->display.crtc_enable = ironlake_crtc_enable;
dev_priv->display.crtc_disable = ironlake_crtc_disable;
- } else if (IS_VALLEYVIEW(dev) || IS_CHERRYVIEW(dev)) {
+ } else if (IS_CHERRYVIEW(dev_priv)) {
dev_priv->display.get_pipe_config = i9xx_get_pipe_config;
dev_priv->display.get_initial_plane_config =
i9xx_get_initial_plane_config;
- dev_priv->display.crtc_compute_clock = i9xx_crtc_compute_clock;
+ dev_priv->display.crtc_compute_clock = chv_crtc_compute_clock;
dev_priv->display.crtc_enable = valleyview_crtc_enable;
dev_priv->display.crtc_disable = i9xx_crtc_disable;
- } else {
+ } else if (IS_VALLEYVIEW(dev_priv)) {
+ dev_priv->display.get_pipe_config = i9xx_get_pipe_config;
+ dev_priv->display.get_initial_plane_config =
+ i9xx_get_initial_plane_config;
+ dev_priv->display.crtc_compute_clock = vlv_crtc_compute_clock;
+ dev_priv->display.crtc_enable = valleyview_crtc_enable;
+ dev_priv->display.crtc_disable = i9xx_crtc_disable;
+ } else if (IS_G4X(dev_priv)) {
+ dev_priv->display.get_pipe_config = i9xx_get_pipe_config;
+ dev_priv->display.get_initial_plane_config =
+ i9xx_get_initial_plane_config;
+ dev_priv->display.crtc_compute_clock = g4x_crtc_compute_clock;
+ dev_priv->display.crtc_enable = i9xx_crtc_enable;
+ dev_priv->display.crtc_disable = i9xx_crtc_disable;
+ } else if (IS_PINEVIEW(dev_priv)) {
+ dev_priv->display.get_pipe_config = i9xx_get_pipe_config;
+ dev_priv->display.get_initial_plane_config =
+ i9xx_get_initial_plane_config;
+ dev_priv->display.crtc_compute_clock = pnv_crtc_compute_clock;
+ dev_priv->display.crtc_enable = i9xx_crtc_enable;
+ dev_priv->display.crtc_disable = i9xx_crtc_disable;
+ } else if (!IS_GEN2(dev_priv)) {
dev_priv->display.get_pipe_config = i9xx_get_pipe_config;
dev_priv->display.get_initial_plane_config =
i9xx_get_initial_plane_config;
dev_priv->display.crtc_compute_clock = i9xx_crtc_compute_clock;
dev_priv->display.crtc_enable = i9xx_crtc_enable;
dev_priv->display.crtc_disable = i9xx_crtc_disable;
+ } else {
+ dev_priv->display.get_pipe_config = i9xx_get_pipe_config;
+ dev_priv->display.get_initial_plane_config =
+ i9xx_get_initial_plane_config;
+ dev_priv->display.crtc_compute_clock = i8xx_crtc_compute_clock;
+ dev_priv->display.crtc_enable = i9xx_crtc_enable;
+ dev_priv->display.crtc_disable = i9xx_crtc_disable;
}
/* Returns the core display clock speed */
- if (IS_SKYLAKE(dev) || IS_KABYLAKE(dev))
+ if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv))
dev_priv->display.get_display_clock_speed =
skylake_get_display_clock_speed;
- else if (IS_BROXTON(dev))
+ else if (IS_BROXTON(dev_priv))
dev_priv->display.get_display_clock_speed =
broxton_get_display_clock_speed;
- else if (IS_BROADWELL(dev))
+ else if (IS_BROADWELL(dev_priv))
dev_priv->display.get_display_clock_speed =
broadwell_get_display_clock_speed;
- else if (IS_HASWELL(dev))
+ else if (IS_HASWELL(dev_priv))
dev_priv->display.get_display_clock_speed =
haswell_get_display_clock_speed;
- else if (IS_VALLEYVIEW(dev) || IS_CHERRYVIEW(dev))
+ else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
dev_priv->display.get_display_clock_speed =
valleyview_get_display_clock_speed;
- else if (IS_GEN5(dev))
+ else if (IS_GEN5(dev_priv))
dev_priv->display.get_display_clock_speed =
ilk_get_display_clock_speed;
- else if (IS_I945G(dev) || IS_BROADWATER(dev) ||
- IS_GEN6(dev) || IS_IVYBRIDGE(dev))
+ else if (IS_I945G(dev_priv) || IS_BROADWATER(dev_priv) ||
+ IS_GEN6(dev_priv) || IS_IVYBRIDGE(dev_priv))
dev_priv->display.get_display_clock_speed =
i945_get_display_clock_speed;
- else if (IS_GM45(dev))
+ else if (IS_GM45(dev_priv))
dev_priv->display.get_display_clock_speed =
gm45_get_display_clock_speed;
- else if (IS_CRESTLINE(dev))
+ else if (IS_CRESTLINE(dev_priv))
dev_priv->display.get_display_clock_speed =
i965gm_get_display_clock_speed;
- else if (IS_PINEVIEW(dev))
+ else if (IS_PINEVIEW(dev_priv))
dev_priv->display.get_display_clock_speed =
pnv_get_display_clock_speed;
- else if (IS_G33(dev) || IS_G4X(dev))
+ else if (IS_G33(dev_priv) || IS_G4X(dev_priv))
dev_priv->display.get_display_clock_speed =
g33_get_display_clock_speed;
- else if (IS_I915G(dev))
+ else if (IS_I915G(dev_priv))
dev_priv->display.get_display_clock_speed =
i915_get_display_clock_speed;
- else if (IS_I945GM(dev) || IS_845G(dev))
+ else if (IS_I945GM(dev_priv) || IS_845G(dev_priv))
dev_priv->display.get_display_clock_speed =
i9xx_misc_get_display_clock_speed;
- else if (IS_I915GM(dev))
+ else if (IS_I915GM(dev_priv))
dev_priv->display.get_display_clock_speed =
i915gm_get_display_clock_speed;
- else if (IS_I865G(dev))
+ else if (IS_I865G(dev_priv))
dev_priv->display.get_display_clock_speed =
i865_get_display_clock_speed;
- else if (IS_I85X(dev))
+ else if (IS_I85X(dev_priv))
dev_priv->display.get_display_clock_speed =
i85x_get_display_clock_speed;
else { /* 830 */
- WARN(!IS_I830(dev), "Unknown platform. Assuming 133 MHz CDCLK\n");
+ WARN(!IS_I830(dev_priv), "Unknown platform. Assuming 133 MHz CDCLK\n");
dev_priv->display.get_display_clock_speed =
i830_get_display_clock_speed;
}
- if (IS_GEN5(dev)) {
+ if (IS_GEN5(dev_priv)) {
dev_priv->display.fdi_link_train = ironlake_fdi_link_train;
- } else if (IS_GEN6(dev)) {
+ } else if (IS_GEN6(dev_priv)) {
dev_priv->display.fdi_link_train = gen6_fdi_link_train;
- } else if (IS_IVYBRIDGE(dev)) {
+ } else if (IS_IVYBRIDGE(dev_priv)) {
/* FIXME: detect B0+ stepping and use auto training */
dev_priv->display.fdi_link_train = ivb_manual_fdi_link_train;
- } else if (IS_HASWELL(dev) || IS_BROADWELL(dev)) {
+ } else if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv)) {
dev_priv->display.fdi_link_train = hsw_fdi_link_train;
- if (IS_BROADWELL(dev)) {
+ if (IS_BROADWELL(dev_priv)) {
dev_priv->display.modeset_commit_cdclk =
broadwell_modeset_commit_cdclk;
dev_priv->display.modeset_calc_cdclk =
broadwell_modeset_calc_cdclk;
}
- } else if (IS_VALLEYVIEW(dev) || IS_CHERRYVIEW(dev)) {
+ } else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) {
dev_priv->display.modeset_commit_cdclk =
valleyview_modeset_commit_cdclk;
dev_priv->display.modeset_calc_cdclk =
valleyview_modeset_calc_cdclk;
- } else if (IS_BROXTON(dev)) {
+ } else if (IS_BROXTON(dev_priv)) {
dev_priv->display.modeset_commit_cdclk =
broxton_modeset_commit_cdclk;
dev_priv->display.modeset_calc_cdclk =
broxton_modeset_calc_cdclk;
}
- switch (INTEL_INFO(dev)->gen) {
+ switch (INTEL_INFO(dev_priv)->gen) {
case 2:
dev_priv->display.queue_flip = intel_gen2_queue_flip;
break;
@@ -15052,8 +15092,6 @@
/* Default just returns -ENODEV to indicate unsupported */
dev_priv->display.queue_flip = intel_default_queue_flip;
}
-
- mutex_init(&dev_priv->pps_mutex);
}
/*
@@ -15276,7 +15314,7 @@
int i;
/* Only supported on platforms that use atomic watermark design */
- if (!dev_priv->display.program_watermarks)
+ if (!dev_priv->display.optimize_watermarks)
return;
/*
@@ -15297,6 +15335,13 @@
if (WARN_ON(IS_ERR(state)))
goto fail;
+ /*
+ * Hardware readout is the only time we don't want to calculate
+ * intermediate watermarks (since we don't trust the current
+ * watermarks).
+ */
+ to_intel_atomic_state(state)->skip_intermediate_wm = true;
+
ret = intel_atomic_check(dev, state);
if (ret) {
/*
@@ -15319,7 +15364,8 @@
for_each_crtc_in_state(state, crtc, cstate, i) {
struct intel_crtc_state *cs = to_intel_crtc_state(cstate);
- dev_priv->display.program_watermarks(cs);
+ cs->wm.need_postvbl_update = true;
+ dev_priv->display.optimize_watermarks(cs);
}
drm_atomic_state_free(state);
@@ -15330,7 +15376,8 @@
void intel_modeset_init(struct drm_device *dev)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
int sprite, ret;
enum pipe pipe;
struct intel_crtc *crtc;
@@ -15372,9 +15419,6 @@
}
}
- intel_init_display(dev);
- intel_init_audio(dev);
-
if (IS_GEN2(dev)) {
dev->mode_config.max_width = 2048;
dev->mode_config.max_height = 2048;
@@ -15397,7 +15441,7 @@
dev->mode_config.cursor_height = MAX_CURSOR_HEIGHT;
}
- dev->mode_config.fb_base = dev_priv->gtt.mappable_base;
+ dev->mode_config.fb_base = ggtt->mappable_base;
DRM_DEBUG_KMS("%d display pipe%s available.\n",
INTEL_INFO(dev)->num_pipes,
@@ -15414,6 +15458,7 @@
}
intel_update_czclk(dev_priv);
+ intel_update_rawclk(dev_priv);
intel_update_cdclk(dev);
intel_shared_dpll_init(dev);
@@ -15526,10 +15571,15 @@
{
struct drm_device *dev = crtc->base.dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- i915_reg_t reg = PIPECONF(crtc->config->cpu_transcoder);
+ enum transcoder cpu_transcoder = crtc->config->cpu_transcoder;
/* Clear any frame start delays used for debugging left by the BIOS */
- I915_WRITE(reg, I915_READ(reg) & ~PIPECONF_FRAME_START_DELAY_MASK);
+ if (!transcoder_is_dsi(cpu_transcoder)) {
+ i915_reg_t reg = PIPECONF(cpu_transcoder);
+
+ I915_WRITE(reg,
+ I915_READ(reg) & ~PIPECONF_FRAME_START_DELAY_MASK);
+ }
/* restore vblank interrupts to correct state */
drm_crtc_vblank_reset(&crtc->base);
@@ -15577,38 +15627,9 @@
/* Adjust the state of the output pipe according to whether we
* have active connectors/encoders. */
- if (!intel_crtc_has_encoders(crtc))
+ if (crtc->active && !intel_crtc_has_encoders(crtc))
intel_crtc_disable_noatomic(&crtc->base);
- if (crtc->active != crtc->base.state->active) {
- struct intel_encoder *encoder;
-
- /* This can happen either due to bugs in the get_hw_state
- * functions or because of calls to intel_crtc_disable_noatomic,
- * or because the pipe is force-enabled due to the
- * pipe A quirk. */
- DRM_DEBUG_KMS("[CRTC:%d] hw state adjusted, was %s, now %s\n",
- crtc->base.base.id,
- crtc->base.state->enable ? "enabled" : "disabled",
- crtc->active ? "enabled" : "disabled");
-
- WARN_ON(drm_atomic_set_mode_for_crtc(crtc->base.state, NULL) < 0);
- crtc->base.state->active = crtc->active;
- crtc->base.enabled = crtc->active;
- crtc->base.state->connector_mask = 0;
- crtc->base.state->encoder_mask = 0;
-
- /* Because we only establish the connector -> encoder ->
- * crtc links if something is active, this means the
- * crtc is now deactivated. Break the links. connector
- * -> encoder links are only establish when things are
- * actually up, hence no need to break them. */
- WARN_ON(crtc->active);
-
- for_each_encoder_on_crtc(dev, &crtc->base, encoder)
- encoder->base.crtc = NULL;
- }
-
if (crtc->active || HAS_GMCH_DISPLAY(dev)) {
/*
* We start out with underrun reporting disabled to avoid races.
@@ -15738,7 +15759,7 @@
struct intel_crtc_state *crtc_state = crtc->config;
int pixclk = 0;
- __drm_atomic_helper_crtc_destroy_state(&crtc->base, &crtc_state->base);
+ __drm_atomic_helper_crtc_destroy_state(&crtc_state->base);
memset(crtc_state, 0, sizeof(*crtc_state));
crtc_state->base.crtc = &crtc->base;
@@ -15777,22 +15798,17 @@
for (i = 0; i < dev_priv->num_shared_dpll; i++) {
struct intel_shared_dpll *pll = &dev_priv->shared_dplls[i];
- pll->on = pll->get_hw_state(dev_priv, pll,
- &pll->config.hw_state);
- pll->active = 0;
+ pll->on = pll->funcs.get_hw_state(dev_priv, pll,
+ &pll->config.hw_state);
pll->config.crtc_mask = 0;
for_each_intel_crtc(dev, crtc) {
- if (crtc->active && intel_crtc_to_shared_dpll(crtc) == pll) {
- pll->active++;
+ if (crtc->active && crtc->config->shared_dpll == pll)
pll->config.crtc_mask |= 1 << crtc->pipe;
- }
}
+ pll->active_mask = pll->config.crtc_mask;
DRM_DEBUG_KMS("%s hw state readout: crtc_mask 0x%08x, on %i\n",
pll->name, pll->config.crtc_mask, pll->on);
-
- if (pll->config.crtc_mask)
- intel_display_power_get(dev_priv, POWER_DOMAIN_PLLS);
}
for_each_intel_encoder(dev, encoder) {
@@ -15874,6 +15890,8 @@
drm_calc_timestamping_constants(&crtc->base, &crtc->base.hwmode);
update_scanline_offset(crtc);
}
+
+ intel_pipe_config_sanity_check(dev_priv, crtc->config);
}
}
@@ -15908,12 +15926,12 @@
for (i = 0; i < dev_priv->num_shared_dpll; i++) {
struct intel_shared_dpll *pll = &dev_priv->shared_dplls[i];
- if (!pll->on || pll->active)
+ if (!pll->on || pll->active_mask)
continue;
DRM_DEBUG_KMS("%s enabled but not in use, disabling\n", pll->name);
- pll->disable(dev_priv, pll);
+ pll->funcs.disable(dev_priv, pll);
pll->on = false;
}
@@ -16022,9 +16040,8 @@
continue;
mutex_lock(&dev->struct_mutex);
- ret = intel_pin_and_fence_fb_obj(c->primary,
- c->primary->fb,
- c->primary->state);
+ ret = intel_pin_and_fence_fb_obj(c->primary->fb,
+ c->primary->state->rotation);
mutex_unlock(&dev->struct_mutex);
if (ret) {
DRM_ERROR("failed to pin boot fb on pipe %d\n",
@@ -16233,8 +16250,9 @@
error->pipe[i].stat = I915_READ(PIPESTAT(i));
}
+ /* Note: this does not include DSI transcoders. */
error->num_transcoders = INTEL_INFO(dev)->num_pipes;
- if (HAS_DDI(dev_priv->dev))
+ if (HAS_DDI(dev_priv))
error->num_transcoders++; /* Account for eDP. */
for (i = 0; i < error->num_transcoders; i++) {
@@ -16305,7 +16323,7 @@
}
for (i = 0; i < error->num_transcoders; i++) {
- err_printf(m, "CPU transcoder: %c\n",
+ err_printf(m, "CPU transcoder: %s\n",
transcoder_name(error->transcoder[i].cpu_transcoder));
err_printf(m, " Power: %s\n",
onoff(error->transcoder[i].power_domain_on));
diff --git a/drivers/gpu/drm/i915/intel_dp.c b/drivers/gpu/drm/i915/intel_dp.c
index 412a34c..f192f58 100644
--- a/drivers/gpu/drm/i915/intel_dp.c
+++ b/drivers/gpu/drm/i915/intel_dp.c
@@ -129,6 +129,7 @@
static void vlv_init_panel_power_sequencer(struct intel_dp *intel_dp);
static void vlv_steal_power_sequencer(struct drm_device *dev,
enum pipe pipe);
+static void intel_dp_unset_edid(struct intel_dp *intel_dp);
static unsigned int intel_dp_unused_lane_mask(int lane_count)
{
@@ -671,60 +672,55 @@
return status;
}
-static uint32_t i9xx_get_aux_clock_divider(struct intel_dp *intel_dp, int index)
+static uint32_t g4x_get_aux_clock_divider(struct intel_dp *intel_dp, int index)
{
struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
- struct drm_device *dev = intel_dig_port->base.base.dev;
+ struct drm_i915_private *dev_priv = to_i915(intel_dig_port->base.base.dev);
+
+ if (index)
+ return 0;
/*
* The clock divider is based off the hrawclk, and would like to run at
- * 2MHz. So, take the hrawclk value and divide by 2 and use that
+ * 2MHz. So, take the hrawclk value and divide by 2000 and use that
*/
- return index ? 0 : DIV_ROUND_CLOSEST(intel_hrawclk(dev), 2);
+ return DIV_ROUND_CLOSEST(dev_priv->rawclk_freq, 2000);
}
static uint32_t ilk_get_aux_clock_divider(struct intel_dp *intel_dp, int index)
{
struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
- struct drm_device *dev = intel_dig_port->base.base.dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(intel_dig_port->base.base.dev);
if (index)
return 0;
- if (intel_dig_port->port == PORT_A) {
+ /*
+ * The clock divider is based off the cdclk or PCH rawclk, and would
+ * like to run at 2MHz. So, take the cdclk or PCH rawclk value and
+ * divide by 2000 and use that
+ */
+ if (intel_dig_port->port == PORT_A)
return DIV_ROUND_CLOSEST(dev_priv->cdclk_freq, 2000);
-
- } else {
- return DIV_ROUND_CLOSEST(intel_pch_rawclk(dev), 2);
- }
+ else
+ return DIV_ROUND_CLOSEST(dev_priv->rawclk_freq, 2000);
}
static uint32_t hsw_get_aux_clock_divider(struct intel_dp *intel_dp, int index)
{
struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
- struct drm_device *dev = intel_dig_port->base.base.dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(intel_dig_port->base.base.dev);
- if (intel_dig_port->port == PORT_A) {
- if (index)
- return 0;
- return DIV_ROUND_CLOSEST(dev_priv->cdclk_freq, 2000);
- } else if (HAS_PCH_LPT_H(dev_priv)) {
+ if (intel_dig_port->port != PORT_A && HAS_PCH_LPT_H(dev_priv)) {
/* Workaround for non-ULT HSW */
switch (index) {
case 0: return 63;
case 1: return 72;
default: return 0;
}
- } else {
- return index ? 0 : DIV_ROUND_CLOSEST(intel_pch_rawclk(dev), 2);
}
-}
-static uint32_t vlv_get_aux_clock_divider(struct intel_dp *intel_dp, int index)
-{
- return index ? 0 : 100;
+ return ilk_get_aux_clock_divider(intel_dp, index);
}
static uint32_t skl_get_aux_clock_divider(struct intel_dp *intel_dp, int index)
@@ -737,10 +733,10 @@
return index ? 0 : 1;
}
-static uint32_t i9xx_get_aux_send_ctl(struct intel_dp *intel_dp,
- bool has_aux_irq,
- int send_bytes,
- uint32_t aux_clock_divider)
+static uint32_t g4x_get_aux_send_ctl(struct intel_dp *intel_dp,
+ bool has_aux_irq,
+ int send_bytes,
+ uint32_t aux_clock_divider)
{
struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
struct drm_device *dev = intel_dig_port->base.base.dev;
@@ -1229,71 +1225,6 @@
intel_connector_unregister(intel_connector);
}
-static void
-skl_edp_set_pll_config(struct intel_crtc_state *pipe_config)
-{
- u32 ctrl1;
-
- memset(&pipe_config->dpll_hw_state, 0,
- sizeof(pipe_config->dpll_hw_state));
-
- pipe_config->ddi_pll_sel = SKL_DPLL0;
- pipe_config->dpll_hw_state.cfgcr1 = 0;
- pipe_config->dpll_hw_state.cfgcr2 = 0;
-
- ctrl1 = DPLL_CTRL1_OVERRIDE(SKL_DPLL0);
- switch (pipe_config->port_clock / 2) {
- case 81000:
- ctrl1 |= DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_810,
- SKL_DPLL0);
- break;
- case 135000:
- ctrl1 |= DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_1350,
- SKL_DPLL0);
- break;
- case 270000:
- ctrl1 |= DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_2700,
- SKL_DPLL0);
- break;
- case 162000:
- ctrl1 |= DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_1620,
- SKL_DPLL0);
- break;
- /* TBD: For DP link rates 2.16 GHz and 4.32 GHz, VCO is 8640 which
- results in CDCLK change. Need to handle the change of CDCLK by
- disabling pipes and re-enabling them */
- case 108000:
- ctrl1 |= DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_1080,
- SKL_DPLL0);
- break;
- case 216000:
- ctrl1 |= DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_2160,
- SKL_DPLL0);
- break;
-
- }
- pipe_config->dpll_hw_state.ctrl1 = ctrl1;
-}
-
-void
-hsw_dp_set_ddi_pll_sel(struct intel_crtc_state *pipe_config)
-{
- memset(&pipe_config->dpll_hw_state, 0,
- sizeof(pipe_config->dpll_hw_state));
-
- switch (pipe_config->port_clock / 2) {
- case 81000:
- pipe_config->ddi_pll_sel = PORT_CLK_SEL_LCPLL_810;
- break;
- case 135000:
- pipe_config->ddi_pll_sel = PORT_CLK_SEL_LCPLL_1350;
- break;
- case 270000:
- pipe_config->ddi_pll_sel = PORT_CLK_SEL_LCPLL_2700;
- break;
- }
-}
-
static int
intel_dp_sink_rates(struct intel_dp *intel_dp, const int **sink_rates)
{
@@ -1570,10 +1501,10 @@
/* Get bpp from vbt only for panels that dont have bpp in edid */
if (intel_connector->base.display_info.bpc == 0 &&
- (dev_priv->vbt.edp_bpp && dev_priv->vbt.edp_bpp < bpp)) {
+ (dev_priv->vbt.edp.bpp && dev_priv->vbt.edp.bpp < bpp)) {
DRM_DEBUG_KMS("clamping bpp for eDP panel to BIOS-provided %i\n",
- dev_priv->vbt.edp_bpp);
- bpp = dev_priv->vbt.edp_bpp;
+ dev_priv->vbt.edp.bpp);
+ bpp = dev_priv->vbt.edp.bpp;
}
/*
@@ -1651,13 +1582,7 @@
&pipe_config->dp_m2_n2);
}
- if ((IS_SKYLAKE(dev) || IS_KABYLAKE(dev)) && is_edp(intel_dp))
- skl_edp_set_pll_config(pipe_config);
- else if (IS_BROXTON(dev))
- /* handled in ddi */;
- else if (IS_HASWELL(dev) || IS_BROADWELL(dev))
- hsw_dp_set_ddi_pll_sel(pipe_config);
- else
+ if (!HAS_DDI(dev))
intel_dp_set_clock(encoder, pipe_config);
return true;
@@ -1779,11 +1704,11 @@
I915_READ(pp_stat_reg),
I915_READ(pp_ctrl_reg));
- if (_wait_for((I915_READ(pp_stat_reg) & mask) == value, 5000, 10)) {
+ if (_wait_for((I915_READ(pp_stat_reg) & mask) == value,
+ 5 * USEC_PER_SEC, 10 * USEC_PER_MSEC))
DRM_ERROR("Panel status timeout: status %08x control %08x\n",
I915_READ(pp_stat_reg),
I915_READ(pp_ctrl_reg));
- }
DRM_DEBUG_KMS("Wait complete\n");
}
@@ -2290,6 +2215,15 @@
POSTING_READ(DP_A);
udelay(500);
+ /*
+ * [DevILK] Work around required when enabling DP PLL
+ * while a pipe is enabled going to FDI:
+ * 1. Wait for the start of vertical blank on the enabled pipe going to FDI
+ * 2. Program DP PLL enable
+ */
+ if (IS_GEN5(dev_priv))
+ intel_wait_for_vblank_if_active(dev_priv->dev, !crtc->pipe);
+
intel_dp->DP |= DP_PLL_ENABLE;
I915_WRITE(DP_A, intel_dp->DP);
@@ -2409,7 +2343,6 @@
struct drm_i915_private *dev_priv = dev->dev_private;
enum port port = dp_to_dig_port(intel_dp)->port;
struct intel_crtc *crtc = to_intel_crtc(encoder->base.crtc);
- int dotclock;
tmp = I915_READ(intel_dp->output_reg);
@@ -2459,16 +2392,12 @@
pipe_config->port_clock = 270000;
}
- dotclock = intel_dotclock_calculate(pipe_config->port_clock,
- &pipe_config->dp_m_n);
+ pipe_config->base.adjusted_mode.crtc_clock =
+ intel_dotclock_calculate(pipe_config->port_clock,
+ &pipe_config->dp_m_n);
- if (HAS_PCH_SPLIT(dev_priv->dev) && port != PORT_A)
- ironlake_check_encoder_dotclock(pipe_config, dotclock);
-
- pipe_config->base.adjusted_mode.crtc_clock = dotclock;
-
- if (is_edp(intel_dp) && dev_priv->vbt.edp_bpp &&
- pipe_config->pipe_bpp > dev_priv->vbt.edp_bpp) {
+ if (is_edp(intel_dp) && dev_priv->vbt.edp.bpp &&
+ pipe_config->pipe_bpp > dev_priv->vbt.edp.bpp) {
/*
* This is a big fat ugly hack.
*
@@ -2483,8 +2412,8 @@
* load.
*/
DRM_DEBUG_KMS("pipe has %d bpp for eDP panel, overriding BIOS-provided max %d bpp\n",
- pipe_config->pipe_bpp, dev_priv->vbt.edp_bpp);
- dev_priv->vbt.edp_bpp = pipe_config->pipe_bpp;
+ pipe_config->pipe_bpp, dev_priv->vbt.edp.bpp);
+ dev_priv->vbt.edp.bpp = pipe_config->pipe_bpp;
}
}
@@ -2710,7 +2639,6 @@
struct drm_i915_private *dev_priv = dev->dev_private;
struct intel_crtc *crtc = to_intel_crtc(encoder->base.crtc);
uint32_t dp_reg = I915_READ(intel_dp->output_reg);
- enum port port = dp_to_dig_port(intel_dp)->port;
enum pipe pipe = crtc->pipe;
if (WARN_ON(dp_reg & DP_PORT_EN))
@@ -2721,35 +2649,12 @@
if (IS_VALLEYVIEW(dev) || IS_CHERRYVIEW(dev))
vlv_init_panel_power_sequencer(intel_dp);
- /*
- * We get an occasional spurious underrun between the port
- * enable and vdd enable, when enabling port A eDP.
- *
- * FIXME: Not sure if this applies to (PCH) port D eDP as well
- */
- if (port == PORT_A)
- intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, false);
-
intel_dp_enable_port(intel_dp);
- if (port == PORT_A && IS_GEN5(dev_priv)) {
- /*
- * Underrun reporting for the other pipe was disabled in
- * g4x_pre_enable_dp(). The eDP PLL and port have now been
- * enabled, so it's now safe to re-enable underrun reporting.
- */
- intel_wait_for_vblank_if_active(dev_priv->dev, !pipe);
- intel_set_cpu_fifo_underrun_reporting(dev_priv, !pipe, true);
- intel_set_pch_fifo_underrun_reporting(dev_priv, !pipe, true);
- }
-
edp_panel_vdd_on(intel_dp);
edp_panel_on(intel_dp);
edp_panel_vdd_off(intel_dp, true);
- if (port == PORT_A)
- intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, true);
-
pps_unlock(intel_dp);
if (IS_VALLEYVIEW(dev) || IS_CHERRYVIEW(dev)) {
@@ -2791,26 +2696,11 @@
static void g4x_pre_enable_dp(struct intel_encoder *encoder)
{
- struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
struct intel_dp *intel_dp = enc_to_intel_dp(&encoder->base);
enum port port = dp_to_dig_port(intel_dp)->port;
- enum pipe pipe = to_intel_crtc(encoder->base.crtc)->pipe;
intel_dp_prepare(encoder);
- if (port == PORT_A && IS_GEN5(dev_priv)) {
- /*
- * We get FIFO underruns on the other pipe when
- * enabling the CPU eDP PLL, and when enabling CPU
- * eDP port. We could potentially avoid the PLL
- * underrun with a vblank wait just prior to enabling
- * the PLL, but that doesn't appear to help the port
- * enable case. Just sweep it all under the rug.
- */
- intel_set_cpu_fifo_underrun_reporting(dev_priv, !pipe, false);
- intel_set_pch_fifo_underrun_reporting(dev_priv, !pipe, false);
- }
-
/* Only ilk+ has port A */
if (port == PORT_A)
ironlake_edp_pll_on(intel_dp);
@@ -3184,47 +3074,14 @@
}
/*
- * Native read with retry for link status and receiver capability reads for
- * cases where the sink may still be asleep.
- *
- * Sinks are *supposed* to come up within 1ms from an off state, but we're also
- * supposed to retry 3 times per the spec.
- */
-static ssize_t
-intel_dp_dpcd_read_wake(struct drm_dp_aux *aux, unsigned int offset,
- void *buffer, size_t size)
-{
- ssize_t ret;
- int i;
-
- /*
- * Sometime we just get the same incorrect byte repeated
- * over the entire buffer. Doing just one throw away read
- * initially seems to "solve" it.
- */
- drm_dp_dpcd_read(aux, DP_DPCD_REV, buffer, 1);
-
- for (i = 0; i < 3; i++) {
- ret = drm_dp_dpcd_read(aux, offset, buffer, size);
- if (ret == size)
- return ret;
- msleep(1);
- }
-
- return ret;
-}
-
-/*
* Fetch AUX CH registers 0x202 - 0x207 which contain
* link status information
*/
bool
intel_dp_get_link_status(struct intel_dp *intel_dp, uint8_t link_status[DP_LINK_STATUS_SIZE])
{
- return intel_dp_dpcd_read_wake(&intel_dp->aux,
- DP_LANE0_1_STATUS,
- link_status,
- DP_LINK_STATUS_SIZE) == DP_LINK_STATUS_SIZE;
+ return drm_dp_dpcd_read(&intel_dp->aux, DP_LANE0_1_STATUS, link_status,
+ DP_LINK_STATUS_SIZE) == DP_LINK_STATUS_SIZE;
}
/* These are source-specific values. */
@@ -3238,7 +3095,7 @@
if (IS_BROXTON(dev))
return DP_TRAIN_VOLTAGE_SWING_LEVEL_3;
else if (INTEL_INFO(dev)->gen >= 9) {
- if (dev_priv->edp_low_vswing && port == PORT_A)
+ if (dev_priv->vbt.edp.low_vswing && port == PORT_A)
return DP_TRAIN_VOLTAGE_SWING_LEVEL_3;
return DP_TRAIN_VOLTAGE_SWING_LEVEL_2;
} else if (IS_VALLEYVIEW(dev) || IS_CHERRYVIEW(dev))
@@ -3859,8 +3716,8 @@
struct drm_i915_private *dev_priv = dev->dev_private;
uint8_t rev;
- if (intel_dp_dpcd_read_wake(&intel_dp->aux, 0x000, intel_dp->dpcd,
- sizeof(intel_dp->dpcd)) < 0)
+ if (drm_dp_dpcd_read(&intel_dp->aux, 0x000, intel_dp->dpcd,
+ sizeof(intel_dp->dpcd)) < 0)
return false; /* aux transfer failed */
DRM_DEBUG_KMS("DPCD: %*ph\n", (int) sizeof(intel_dp->dpcd), intel_dp->dpcd);
@@ -3868,12 +3725,33 @@
if (intel_dp->dpcd[DP_DPCD_REV] == 0)
return false; /* DPCD not present */
+ if (drm_dp_dpcd_read(&intel_dp->aux, DP_SINK_COUNT,
+ &intel_dp->sink_count, 1) < 0)
+ return false;
+
+ /*
+ * Sink count can change between short pulse hpd hence
+ * a member variable in intel_dp will track any changes
+ * between short pulse interrupts.
+ */
+ intel_dp->sink_count = DP_GET_SINK_COUNT(intel_dp->sink_count);
+
+ /*
+ * SINK_COUNT == 0 and DOWNSTREAM_PORT_PRESENT == 1 implies that
+ * a dongle is present but no display. Unless we require to know
+ * if a dongle is present or not, we don't need to update
+ * downstream port information. So, an early return here saves
+ * time from performing other operations which are not required.
+ */
+ if (!is_edp(intel_dp) && !intel_dp->sink_count)
+ return false;
+
/* Check if the panel supports PSR */
memset(intel_dp->psr_dpcd, 0, sizeof(intel_dp->psr_dpcd));
if (is_edp(intel_dp)) {
- intel_dp_dpcd_read_wake(&intel_dp->aux, DP_PSR_SUPPORT,
- intel_dp->psr_dpcd,
- sizeof(intel_dp->psr_dpcd));
+ drm_dp_dpcd_read(&intel_dp->aux, DP_PSR_SUPPORT,
+ intel_dp->psr_dpcd,
+ sizeof(intel_dp->psr_dpcd));
if (intel_dp->psr_dpcd[0] & DP_PSR_IS_SUPPORTED) {
dev_priv->psr.sink_support = true;
DRM_DEBUG_KMS("Detected EDP PSR Panel.\n");
@@ -3884,9 +3762,9 @@
uint8_t frame_sync_cap;
dev_priv->psr.sink_support = true;
- intel_dp_dpcd_read_wake(&intel_dp->aux,
- DP_SINK_DEVICE_AUX_FRAME_SYNC_CAP,
- &frame_sync_cap, 1);
+ drm_dp_dpcd_read(&intel_dp->aux,
+ DP_SINK_DEVICE_AUX_FRAME_SYNC_CAP,
+ &frame_sync_cap, 1);
dev_priv->psr.aux_frame_sync = frame_sync_cap ? true : false;
/* PSR2 needs frame sync as well */
dev_priv->psr.psr2_support = dev_priv->psr.aux_frame_sync;
@@ -3902,15 +3780,13 @@
/* Intermediate frequency support */
if (is_edp(intel_dp) &&
(intel_dp->dpcd[DP_EDP_CONFIGURATION_CAP] & DP_DPCD_DISPLAY_CONTROL_CAPABLE) &&
- (intel_dp_dpcd_read_wake(&intel_dp->aux, DP_EDP_DPCD_REV, &rev, 1) == 1) &&
+ (drm_dp_dpcd_read(&intel_dp->aux, DP_EDP_DPCD_REV, &rev, 1) == 1) &&
(rev >= 0x03)) { /* eDp v1.4 or higher */
__le16 sink_rates[DP_MAX_SUPPORTED_RATES];
int i;
- intel_dp_dpcd_read_wake(&intel_dp->aux,
- DP_SUPPORTED_LINK_RATES,
- sink_rates,
- sizeof(sink_rates));
+ drm_dp_dpcd_read(&intel_dp->aux, DP_SUPPORTED_LINK_RATES,
+ sink_rates, sizeof(sink_rates));
for (i = 0; i < ARRAY_SIZE(sink_rates); i++) {
int val = le16_to_cpu(sink_rates[i]);
@@ -3933,9 +3809,9 @@
if (intel_dp->dpcd[DP_DPCD_REV] == 0x10)
return true; /* no per-port downstream info */
- if (intel_dp_dpcd_read_wake(&intel_dp->aux, DP_DOWNSTREAM_PORT_0,
- intel_dp->downstream_ports,
- DP_MAX_DOWNSTREAM_PORTS) < 0)
+ if (drm_dp_dpcd_read(&intel_dp->aux, DP_DOWNSTREAM_PORT_0,
+ intel_dp->downstream_ports,
+ DP_MAX_DOWNSTREAM_PORTS) < 0)
return false; /* downstream port status fetch failed */
return true;
@@ -3949,11 +3825,11 @@
if (!(intel_dp->dpcd[DP_DOWN_STREAM_PORT_COUNT] & DP_OUI_SUPPORT))
return;
- if (intel_dp_dpcd_read_wake(&intel_dp->aux, DP_SINK_OUI, buf, 3) == 3)
+ if (drm_dp_dpcd_read(&intel_dp->aux, DP_SINK_OUI, buf, 3) == 3)
DRM_DEBUG_KMS("Sink OUI: %02hx%02hx%02hx\n",
buf[0], buf[1], buf[2]);
- if (intel_dp_dpcd_read_wake(&intel_dp->aux, DP_BRANCH_OUI, buf, 3) == 3)
+ if (drm_dp_dpcd_read(&intel_dp->aux, DP_BRANCH_OUI, buf, 3) == 3)
DRM_DEBUG_KMS("Branch OUI: %02hx%02hx%02hx\n",
buf[0], buf[1], buf[2]);
}
@@ -3963,13 +3839,16 @@
{
u8 buf[1];
+ if (!i915.enable_dp_mst)
+ return false;
+
if (!intel_dp->can_mst)
return false;
if (intel_dp->dpcd[DP_DPCD_REV] < 0x12)
return false;
- if (intel_dp_dpcd_read_wake(&intel_dp->aux, DP_MSTM_CAP, buf, 1)) {
+ if (drm_dp_dpcd_read(&intel_dp->aux, DP_MSTM_CAP, buf, 1)) {
if (buf[0] & DP_MST_CAP) {
DRM_DEBUG_KMS("Sink is MST capable\n");
intel_dp->is_mst = true;
@@ -4106,7 +3985,7 @@
static bool
intel_dp_get_sink_irq(struct intel_dp *intel_dp, u8 *sink_irq_vector)
{
- return intel_dp_dpcd_read_wake(&intel_dp->aux,
+ return drm_dp_dpcd_read(&intel_dp->aux,
DP_DEVICE_SERVICE_IRQ_VECTOR,
sink_irq_vector, 1) == 1;
}
@@ -4116,7 +3995,7 @@
{
int ret;
- ret = intel_dp_dpcd_read_wake(&intel_dp->aux,
+ ret = drm_dp_dpcd_read(&intel_dp->aux,
DP_SINK_COUNT_ESI,
sink_irq_vector, 14);
if (ret != 14)
@@ -4292,6 +4171,36 @@
return -EINVAL;
}
+static void
+intel_dp_check_link_status(struct intel_dp *intel_dp)
+{
+ struct intel_encoder *intel_encoder = &dp_to_dig_port(intel_dp)->base;
+ struct drm_device *dev = intel_dp_to_dev(intel_dp);
+ u8 link_status[DP_LINK_STATUS_SIZE];
+
+ WARN_ON(!drm_modeset_is_locked(&dev->mode_config.connection_mutex));
+
+ if (!intel_dp_get_link_status(intel_dp, link_status)) {
+ DRM_ERROR("Failed to get link status\n");
+ return;
+ }
+
+ if (!intel_encoder->base.crtc)
+ return;
+
+ if (!to_intel_crtc(intel_encoder->base.crtc)->active)
+ return;
+
+ /* if link training is requested we should perform it always */
+ if ((intel_dp->compliance_test_type == DP_TEST_LINK_TRAINING) ||
+ (!drm_dp_channel_eq_ok(link_status, intel_dp->lane_count))) {
+ DRM_DEBUG_KMS("%s: channel EQ not ok, retraining\n",
+ intel_encoder->base.name);
+ intel_dp_start_link_train(intel_dp);
+ intel_dp_stop_link_train(intel_dp);
+ }
+}
+
/*
* According to DP spec
* 5.1.2:
@@ -4299,16 +4208,19 @@
* 2. Configure link according to Receiver Capabilities
* 3. Use Link Training from 2.5.3.3 and 3.5.1.3
* 4. Check link status on receipt of hot-plug interrupt
+ *
+ * intel_dp_short_pulse - handles short pulse interrupts
+ * when full detection is not required.
+ * Returns %true if short pulse is handled and full detection
+ * is NOT required and %false otherwise.
*/
-static void
-intel_dp_check_link_status(struct intel_dp *intel_dp)
+static bool
+intel_dp_short_pulse(struct intel_dp *intel_dp)
{
struct drm_device *dev = intel_dp_to_dev(intel_dp);
- struct intel_encoder *intel_encoder = &dp_to_dig_port(intel_dp)->base;
u8 sink_irq_vector;
- u8 link_status[DP_LINK_STATUS_SIZE];
-
- WARN_ON(!drm_modeset_is_locked(&dev->mode_config.connection_mutex));
+ u8 old_sink_count = intel_dp->sink_count;
+ bool ret;
/*
* Clearing compliance test variables to allow capturing
@@ -4318,20 +4230,17 @@
intel_dp->compliance_test_type = 0;
intel_dp->compliance_test_data = 0;
- if (!intel_encoder->base.crtc)
- return;
+ /*
+ * Now read the DPCD to see if it's actually running
+ * If the current value of sink count doesn't match with
+ * the value that was stored earlier or dpcd read failed
+ * we need to do full detection
+ */
+ ret = intel_dp_get_dpcd(intel_dp);
- if (!to_intel_crtc(intel_encoder->base.crtc)->active)
- return;
-
- /* Try to read receiver status if the link appears to be up */
- if (!intel_dp_get_link_status(intel_dp, link_status)) {
- return;
- }
-
- /* Now read the DPCD to see if it's actually running */
- if (!intel_dp_get_dpcd(intel_dp)) {
- return;
+ if ((old_sink_count != intel_dp->sink_count) || !ret) {
+ /* No need to proceed if we are going to do full detect */
+ return false;
}
/* Try to read the source of the interrupt */
@@ -4348,14 +4257,11 @@
DRM_DEBUG_DRIVER("CP or sink specific irq unhandled\n");
}
- /* if link training is requested we should perform it always */
- if ((intel_dp->compliance_test_type == DP_TEST_LINK_TRAINING) ||
- (!drm_dp_channel_eq_ok(link_status, intel_dp->lane_count))) {
- DRM_DEBUG_KMS("%s: channel EQ not ok, retraining\n",
- intel_encoder->base.name);
- intel_dp_start_link_train(intel_dp);
- intel_dp_stop_link_train(intel_dp);
- }
+ drm_modeset_lock(&dev->mode_config.connection_mutex, NULL);
+ intel_dp_check_link_status(intel_dp);
+ drm_modeset_unlock(&dev->mode_config.connection_mutex);
+
+ return true;
}
/* XXX this is probably wrong for multiple downstream ports */
@@ -4368,6 +4274,9 @@
if (!intel_dp_get_dpcd(intel_dp))
return connector_status_disconnected;
+ if (is_edp(intel_dp))
+ return connector_status_connected;
+
/* if there's no downstream port, we're done */
if (!(dpcd[DP_DOWNSTREAMPORT_PRESENT] & DP_DWN_STRM_PORT_PRESENT))
return connector_status_connected;
@@ -4375,14 +4284,9 @@
/* If we're HPD-aware, SINK_COUNT changes dynamically */
if (intel_dp->dpcd[DP_DPCD_REV] >= 0x11 &&
intel_dp->downstream_ports[0] & DP_DS_PORT_HPD) {
- uint8_t reg;
- if (intel_dp_dpcd_read_wake(&intel_dp->aux, DP_SINK_COUNT,
- ®, 1) < 0)
- return connector_status_unknown;
-
- return DP_GET_SINK_COUNT(reg) ? connector_status_connected
- : connector_status_disconnected;
+ return intel_dp->sink_count ?
+ connector_status_connected : connector_status_disconnected;
}
/* If no HPD, poke DDC gently */
@@ -4591,6 +4495,7 @@
struct intel_connector *intel_connector = intel_dp->attached_connector;
struct edid *edid;
+ intel_dp_unset_edid(intel_dp);
edid = intel_dp_get_edid(intel_dp);
intel_connector->detect_edid = edid;
@@ -4611,9 +4516,10 @@
intel_dp->has_audio = false;
}
-static enum drm_connector_status
-intel_dp_detect(struct drm_connector *connector, bool force)
+static void
+intel_dp_long_pulse(struct intel_connector *intel_connector)
{
+ struct drm_connector *connector = &intel_connector->base;
struct intel_dp *intel_dp = intel_attached_dp(connector);
struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
struct intel_encoder *intel_encoder = &intel_dig_port->base;
@@ -4623,17 +4529,6 @@
bool ret;
u8 sink_irq_vector;
- DRM_DEBUG_KMS("[CONNECTOR:%d:%s]\n",
- connector->base.id, connector->name);
- intel_dp_unset_edid(intel_dp);
-
- if (intel_dp->is_mst) {
- /* MST devices are disconnected from a monitor POV */
- if (intel_encoder->type != INTEL_OUTPUT_EDP)
- intel_encoder->type = INTEL_OUTPUT_DISPLAYPORT;
- return connector_status_disconnected;
- }
-
power_domain = intel_display_port_aux_power_domain(intel_encoder);
intel_display_power_get(to_i915(dev), power_domain);
@@ -4651,19 +4546,42 @@
intel_dp->compliance_test_type = 0;
intel_dp->compliance_test_data = 0;
+ if (intel_dp->is_mst) {
+ DRM_DEBUG_KMS("MST device may have disappeared %d vs %d\n",
+ intel_dp->is_mst,
+ intel_dp->mst_mgr.mst_state);
+ intel_dp->is_mst = false;
+ drm_dp_mst_topology_mgr_set_mst(&intel_dp->mst_mgr,
+ intel_dp->is_mst);
+ }
+
goto out;
}
+ if (intel_encoder->type != INTEL_OUTPUT_EDP)
+ intel_encoder->type = INTEL_OUTPUT_DISPLAYPORT;
+
intel_dp_probe_oui(intel_dp);
ret = intel_dp_probe_mst(intel_dp);
if (ret) {
- /* if we are in MST mode then this connector
- won't appear connected or have anything with EDID on it */
- if (intel_encoder->type != INTEL_OUTPUT_EDP)
- intel_encoder->type = INTEL_OUTPUT_DISPLAYPORT;
+ /*
+ * If we are in MST mode then this connector
+ * won't appear connected or have anything
+ * with EDID on it
+ */
status = connector_status_disconnected;
goto out;
+ } else if (connector->status == connector_status_connected) {
+ /*
+ * If display was connected already and is still connected
+ * check links status, there has been known issues of
+ * link loss triggerring long pulse!!!!
+ */
+ drm_modeset_lock(&dev->mode_config.connection_mutex, NULL);
+ intel_dp_check_link_status(intel_dp);
+ drm_modeset_unlock(&dev->mode_config.connection_mutex);
+ goto out;
}
/*
@@ -4676,9 +4594,8 @@
intel_dp_set_edid(intel_dp);
- if (intel_encoder->type != INTEL_OUTPUT_EDP)
- intel_encoder->type = INTEL_OUTPUT_DISPLAYPORT;
status = connector_status_connected;
+ intel_dp->detect_done = true;
/* Try to read the source of the interrupt */
if (intel_dp->dpcd[DP_DPCD_REV] >= 0x11 &&
@@ -4695,8 +4612,43 @@
}
out:
+ if ((status != connector_status_connected) &&
+ (intel_dp->is_mst == false))
+ intel_dp_unset_edid(intel_dp);
+
intel_display_power_put(to_i915(dev), power_domain);
- return status;
+ return;
+}
+
+static enum drm_connector_status
+intel_dp_detect(struct drm_connector *connector, bool force)
+{
+ struct intel_dp *intel_dp = intel_attached_dp(connector);
+ struct intel_digital_port *intel_dig_port = dp_to_dig_port(intel_dp);
+ struct intel_encoder *intel_encoder = &intel_dig_port->base;
+ struct intel_connector *intel_connector = to_intel_connector(connector);
+
+ DRM_DEBUG_KMS("[CONNECTOR:%d:%s]\n",
+ connector->base.id, connector->name);
+
+ if (intel_dp->is_mst) {
+ /* MST devices are disconnected from a monitor POV */
+ intel_dp_unset_edid(intel_dp);
+ if (intel_encoder->type != INTEL_OUTPUT_EDP)
+ intel_encoder->type = INTEL_OUTPUT_DISPLAYPORT;
+ return connector_status_disconnected;
+ }
+
+ /* If full detect is not performed yet, do a full detect */
+ if (!intel_dp->detect_done)
+ intel_dp_long_pulse(intel_dp->attached_connector);
+
+ intel_dp->detect_done = false;
+
+ if (intel_connector->detect_edid)
+ return connector_status_connected;
+ else
+ return connector_status_disconnected;
}
static void
@@ -4835,6 +4787,11 @@
DRM_DEBUG_KMS("no scaling not supported\n");
return -EINVAL;
}
+ if (HAS_GMCH_DISPLAY(dev_priv) &&
+ val == DRM_MODE_SCALE_CENTER) {
+ DRM_DEBUG_KMS("centering not supported\n");
+ return -EINVAL;
+ }
if (intel_connector->panel.fitting_mode == val) {
/* the eDP scaling property is not changed */
@@ -5023,44 +4980,37 @@
/* indicate that we need to restart link training */
intel_dp->train_set_valid = false;
- if (!intel_digital_port_connected(dev_priv, intel_dig_port))
- goto mst_fail;
+ intel_dp_long_pulse(intel_dp->attached_connector);
+ if (intel_dp->is_mst)
+ ret = IRQ_HANDLED;
+ goto put_power;
- if (!intel_dp_get_dpcd(intel_dp)) {
- goto mst_fail;
- }
-
- intel_dp_probe_oui(intel_dp);
-
- if (!intel_dp_probe_mst(intel_dp)) {
- drm_modeset_lock(&dev->mode_config.connection_mutex, NULL);
- intel_dp_check_link_status(intel_dp);
- drm_modeset_unlock(&dev->mode_config.connection_mutex);
- goto mst_fail;
- }
} else {
if (intel_dp->is_mst) {
- if (intel_dp_check_mst_status(intel_dp) == -EINVAL)
- goto mst_fail;
+ if (intel_dp_check_mst_status(intel_dp) == -EINVAL) {
+ /*
+ * If we were in MST mode, and device is not
+ * there, get out of MST mode
+ */
+ DRM_DEBUG_KMS("MST device may have disappeared %d vs %d\n",
+ intel_dp->is_mst, intel_dp->mst_mgr.mst_state);
+ intel_dp->is_mst = false;
+ drm_dp_mst_topology_mgr_set_mst(&intel_dp->mst_mgr,
+ intel_dp->is_mst);
+ goto put_power;
+ }
}
if (!intel_dp->is_mst) {
- drm_modeset_lock(&dev->mode_config.connection_mutex, NULL);
- intel_dp_check_link_status(intel_dp);
- drm_modeset_unlock(&dev->mode_config.connection_mutex);
+ if (!intel_dp_short_pulse(intel_dp)) {
+ intel_dp_long_pulse(intel_dp->attached_connector);
+ goto put_power;
+ }
}
}
ret = IRQ_HANDLED;
- goto put_power;
-mst_fail:
- /* if we were in MST mode, and device is not there get out of MST mode */
- if (intel_dp->is_mst) {
- DRM_DEBUG_KMS("MST device may have disappeared %d vs %d\n", intel_dp->is_mst, intel_dp->mst_mgr.mst_state);
- intel_dp->is_mst = false;
- drm_dp_mst_topology_mgr_set_mst(&intel_dp->mst_mgr, intel_dp->is_mst);
- }
put_power:
intel_display_power_put(dev_priv, power_domain);
@@ -5071,14 +5021,6 @@
bool intel_dp_is_edp(struct drm_device *dev, enum port port)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- union child_device_config *p_child;
- int i;
- static const short port_mapping[] = {
- [PORT_B] = DVO_PORT_DPB,
- [PORT_C] = DVO_PORT_DPC,
- [PORT_D] = DVO_PORT_DPD,
- [PORT_E] = DVO_PORT_DPE,
- };
/*
* eDP not supported on g4x. so bail out early just
@@ -5090,18 +5032,7 @@
if (port == PORT_A)
return true;
- if (!dev_priv->vbt.child_dev_num)
- return false;
-
- for (i = 0; i < dev_priv->vbt.child_dev_num; i++) {
- p_child = dev_priv->vbt.child_dev + i;
-
- if (p_child->common.dvo_port == port_mapping[port] &&
- (p_child->common.device_type & DEVICE_TYPE_eDP_BITS) ==
- (DEVICE_TYPE_eDP & DEVICE_TYPE_eDP_BITS))
- return true;
- }
- return false;
+ return intel_bios_is_port_edp(dev_priv, port);
}
void
@@ -5208,7 +5139,7 @@
DRM_DEBUG_KMS("cur t1_t3 %d t8 %d t9 %d t10 %d t11_t12 %d\n",
cur.t1_t3, cur.t8, cur.t9, cur.t10, cur.t11_t12);
- vbt = dev_priv->vbt.edp_pps;
+ vbt = dev_priv->vbt.edp.pps;
/* Upper limits from eDP 1.3 spec. Note that we use the clunky units of
* our hw here, which are all in 100usec. */
@@ -5259,7 +5190,7 @@
{
struct drm_i915_private *dev_priv = dev->dev_private;
u32 pp_on, pp_off, pp_div, port_sel = 0;
- int div = HAS_PCH_SPLIT(dev) ? intel_pch_rawclk(dev) : intel_hrawclk(dev);
+ int div = dev_priv->rawclk_freq / 1000;
i915_reg_t pp_on_reg, pp_off_reg, pp_div_reg, pp_ctrl_reg;
enum port port = dp_to_dig_port(intel_dp)->port;
const struct edp_power_seq *seq = &intel_dp->pps_delays;
@@ -5852,19 +5783,17 @@
/* intel_dp vfuncs */
if (INTEL_INFO(dev)->gen >= 9)
intel_dp->get_aux_clock_divider = skl_get_aux_clock_divider;
- else if (IS_VALLEYVIEW(dev) || IS_CHERRYVIEW(dev))
- intel_dp->get_aux_clock_divider = vlv_get_aux_clock_divider;
else if (IS_HASWELL(dev) || IS_BROADWELL(dev))
intel_dp->get_aux_clock_divider = hsw_get_aux_clock_divider;
else if (HAS_PCH_SPLIT(dev))
intel_dp->get_aux_clock_divider = ilk_get_aux_clock_divider;
else
- intel_dp->get_aux_clock_divider = i9xx_get_aux_clock_divider;
+ intel_dp->get_aux_clock_divider = g4x_get_aux_clock_divider;
if (INTEL_INFO(dev)->gen >= 9)
intel_dp->get_aux_send_ctl = skl_get_aux_send_ctl;
else
- intel_dp->get_aux_send_ctl = i9xx_get_aux_send_ctl;
+ intel_dp->get_aux_send_ctl = g4x_get_aux_send_ctl;
if (HAS_DDI(dev))
intel_dp->prepare_link_retrain = intel_ddi_prepare_link_retrain;
diff --git a/drivers/gpu/drm/i915/intel_dp_mst.c b/drivers/gpu/drm/i915/intel_dp_mst.c
index 2c99972..7a34090 100644
--- a/drivers/gpu/drm/i915/intel_dp_mst.c
+++ b/drivers/gpu/drm/i915/intel_dp_mst.c
@@ -33,7 +33,6 @@
static bool intel_dp_mst_compute_config(struct intel_encoder *encoder,
struct intel_crtc_state *pipe_config)
{
- struct drm_device *dev = encoder->base.dev;
struct intel_dp_mst_encoder *intel_mst = enc_to_mst(&encoder->base);
struct intel_digital_port *intel_dig_port = intel_mst->primary;
struct intel_dp *intel_dp = &intel_dig_port->dp;
@@ -90,9 +89,6 @@
pipe_config->dp_m_n.tu = slots;
- if (IS_HASWELL(dev) || IS_BROADWELL(dev))
- hsw_dp_set_ddi_pll_sel(pipe_config);
-
return true;
}
@@ -106,7 +102,7 @@
DRM_DEBUG_KMS("%d\n", intel_dp->active_mst_links);
- drm_dp_mst_reset_vcpi_slots(&intel_dp->mst_mgr, intel_mst->port);
+ drm_dp_mst_reset_vcpi_slots(&intel_dp->mst_mgr, intel_mst->connector->port);
ret = drm_dp_update_payload_part1(&intel_dp->mst_mgr);
if (ret) {
@@ -127,10 +123,11 @@
/* and this can also fail */
drm_dp_update_payload_part2(&intel_dp->mst_mgr);
- drm_dp_mst_deallocate_vcpi(&intel_dp->mst_mgr, intel_mst->port);
+ drm_dp_mst_deallocate_vcpi(&intel_dp->mst_mgr, intel_mst->connector->port);
intel_dp->active_mst_links--;
- intel_mst->port = NULL;
+
+ intel_mst->connector = NULL;
if (intel_dp->active_mst_links == 0) {
intel_dig_port->base.post_disable(&intel_dig_port->base);
intel_dp_sink_dpms(intel_dp, DRM_MODE_DPMS_OFF);
@@ -170,7 +167,8 @@
found->encoder = encoder;
DRM_DEBUG_KMS("%d\n", intel_dp->active_mst_links);
- intel_mst->port = found->port;
+
+ intel_mst->connector = found;
if (intel_dp->active_mst_links == 0) {
intel_prepare_ddi_buffer(&intel_dig_port->base);
@@ -188,7 +186,7 @@
}
ret = drm_dp_mst_allocate_vcpi(&intel_dp->mst_mgr,
- intel_mst->port,
+ intel_mst->connector->port,
intel_crtc->config->pbn, &slots);
if (ret == false) {
DRM_ERROR("failed to allocate vcpi\n");
@@ -229,7 +227,7 @@
{
struct intel_dp_mst_encoder *intel_mst = enc_to_mst(&encoder->base);
*pipe = intel_mst->pipe;
- if (intel_mst->port)
+ if (intel_mst->connector)
return true;
return false;
}
@@ -290,10 +288,11 @@
struct edid *edid;
int ret;
- edid = drm_dp_mst_get_edid(connector, &intel_dp->mst_mgr, intel_connector->port);
- if (!edid)
- return 0;
+ if (!intel_dp) {
+ return intel_connector_update_modes(connector, NULL);
+ }
+ edid = drm_dp_mst_get_edid(connector, &intel_dp->mst_mgr, intel_connector->port);
ret = intel_connector_update_modes(connector, edid);
kfree(edid);
@@ -306,6 +305,8 @@
struct intel_connector *intel_connector = to_intel_connector(connector);
struct intel_dp *intel_dp = intel_connector->mst_port;
+ if (!intel_dp)
+ return connector_status_disconnected;
return drm_dp_mst_detect_port(connector, &intel_dp->mst_mgr, intel_connector->port);
}
@@ -371,6 +372,8 @@
struct intel_dp *intel_dp = intel_connector->mst_port;
struct intel_crtc *crtc = to_intel_crtc(state->crtc);
+ if (!intel_dp)
+ return NULL;
return &intel_dp->mst_encoders[crtc->pipe]->base.base;
}
@@ -378,6 +381,8 @@
{
struct intel_connector *intel_connector = to_intel_connector(connector);
struct intel_dp *intel_dp = intel_connector->mst_port;
+ if (!intel_dp)
+ return NULL;
return &intel_dp->mst_encoders[0]->base.base;
}
@@ -488,23 +493,11 @@
/* need to nuke the connector */
drm_modeset_lock_all(dev);
- if (connector->state->crtc) {
- struct drm_mode_set set;
- int ret;
-
- memset(&set, 0, sizeof(set));
- set.crtc = connector->state->crtc,
-
- ret = drm_atomic_helper_set_config(&set);
-
- WARN(ret, "Disabling mst crtc failed with %i\n", ret);
- }
-
intel_connector_remove_from_fbdev(intel_connector);
- drm_connector_cleanup(connector);
+ intel_connector->mst_port = NULL;
drm_modeset_unlock_all(dev);
- kfree(intel_connector);
+ drm_connector_unreference(&intel_connector->base);
DRM_DEBUG_KMS("\n");
}
diff --git a/drivers/gpu/drm/i915/intel_dpll_mgr.c b/drivers/gpu/drm/i915/intel_dpll_mgr.c
new file mode 100644
index 0000000..639bf02
--- /dev/null
+++ b/drivers/gpu/drm/i915/intel_dpll_mgr.c
@@ -0,0 +1,1783 @@
+/*
+ * Copyright © 2006-2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#include "intel_drv.h"
+
+struct intel_shared_dpll *
+intel_get_shared_dpll_by_id(struct drm_i915_private *dev_priv,
+ enum intel_dpll_id id)
+{
+ return &dev_priv->shared_dplls[id];
+}
+
+enum intel_dpll_id
+intel_get_shared_dpll_id(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll)
+{
+ if (WARN_ON(pll < dev_priv->shared_dplls||
+ pll > &dev_priv->shared_dplls[dev_priv->num_shared_dpll]))
+ return -1;
+
+ return (enum intel_dpll_id) (pll - dev_priv->shared_dplls);
+}
+
+void
+intel_shared_dpll_config_get(struct intel_shared_dpll_config *config,
+ struct intel_shared_dpll *pll,
+ struct intel_crtc *crtc)
+{
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ enum intel_dpll_id id = intel_get_shared_dpll_id(dev_priv, pll);
+
+ config[id].crtc_mask |= 1 << crtc->pipe;
+}
+
+void
+intel_shared_dpll_config_put(struct intel_shared_dpll_config *config,
+ struct intel_shared_dpll *pll,
+ struct intel_crtc *crtc)
+{
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ enum intel_dpll_id id = intel_get_shared_dpll_id(dev_priv, pll);
+
+ config[id].crtc_mask &= ~(1 << crtc->pipe);
+}
+
+/* For ILK+ */
+void assert_shared_dpll(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll,
+ bool state)
+{
+ bool cur_state;
+ struct intel_dpll_hw_state hw_state;
+
+ if (WARN(!pll, "asserting DPLL %s with no DPLL\n", onoff(state)))
+ return;
+
+ cur_state = pll->funcs.get_hw_state(dev_priv, pll, &hw_state);
+ I915_STATE_WARN(cur_state != state,
+ "%s assertion failure (expected %s, current %s)\n",
+ pll->name, onoff(state), onoff(cur_state));
+}
+
+void intel_prepare_shared_dpll(struct intel_crtc *crtc)
+{
+ struct drm_device *dev = crtc->base.dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct intel_shared_dpll *pll = crtc->config->shared_dpll;
+
+ if (WARN_ON(pll == NULL))
+ return;
+
+ mutex_lock(&dev_priv->dpll_lock);
+ WARN_ON(!pll->config.crtc_mask);
+ if (!pll->active_mask) {
+ DRM_DEBUG_DRIVER("setting up %s\n", pll->name);
+ WARN_ON(pll->on);
+ assert_shared_dpll_disabled(dev_priv, pll);
+
+ pll->funcs.mode_set(dev_priv, pll);
+ }
+ mutex_unlock(&dev_priv->dpll_lock);
+}
+
+/**
+ * intel_enable_shared_dpll - enable PCH PLL
+ * @dev_priv: i915 private structure
+ * @pipe: pipe PLL to enable
+ *
+ * The PCH PLL needs to be enabled before the PCH transcoder, since it
+ * drives the transcoder clock.
+ */
+void intel_enable_shared_dpll(struct intel_crtc *crtc)
+{
+ struct drm_device *dev = crtc->base.dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct intel_shared_dpll *pll = crtc->config->shared_dpll;
+ unsigned crtc_mask = 1 << drm_crtc_index(&crtc->base);
+ unsigned old_mask;
+
+ if (WARN_ON(pll == NULL))
+ return;
+
+ mutex_lock(&dev_priv->dpll_lock);
+ old_mask = pll->active_mask;
+
+ if (WARN_ON(!(pll->config.crtc_mask & crtc_mask)) ||
+ WARN_ON(pll->active_mask & crtc_mask))
+ goto out;
+
+ pll->active_mask |= crtc_mask;
+
+ DRM_DEBUG_KMS("enable %s (active %x, on? %d) for crtc %d\n",
+ pll->name, pll->active_mask, pll->on,
+ crtc->base.base.id);
+
+ if (old_mask) {
+ WARN_ON(!pll->on);
+ assert_shared_dpll_enabled(dev_priv, pll);
+ goto out;
+ }
+ WARN_ON(pll->on);
+
+ DRM_DEBUG_KMS("enabling %s\n", pll->name);
+ pll->funcs.enable(dev_priv, pll);
+ pll->on = true;
+
+out:
+ mutex_unlock(&dev_priv->dpll_lock);
+}
+
+void intel_disable_shared_dpll(struct intel_crtc *crtc)
+{
+ struct drm_device *dev = crtc->base.dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct intel_shared_dpll *pll = crtc->config->shared_dpll;
+ unsigned crtc_mask = 1 << drm_crtc_index(&crtc->base);
+
+ /* PCH only available on ILK+ */
+ if (INTEL_INFO(dev)->gen < 5)
+ return;
+
+ if (pll == NULL)
+ return;
+
+ mutex_lock(&dev_priv->dpll_lock);
+ if (WARN_ON(!(pll->active_mask & crtc_mask)))
+ goto out;
+
+ DRM_DEBUG_KMS("disable %s (active %x, on? %d) for crtc %d\n",
+ pll->name, pll->active_mask, pll->on,
+ crtc->base.base.id);
+
+ assert_shared_dpll_enabled(dev_priv, pll);
+ WARN_ON(!pll->on);
+
+ pll->active_mask &= ~crtc_mask;
+ if (pll->active_mask)
+ goto out;
+
+ DRM_DEBUG_KMS("disabling %s\n", pll->name);
+ pll->funcs.disable(dev_priv, pll);
+ pll->on = false;
+
+out:
+ mutex_unlock(&dev_priv->dpll_lock);
+}
+
+static struct intel_shared_dpll *
+intel_find_shared_dpll(struct intel_crtc *crtc,
+ struct intel_crtc_state *crtc_state,
+ enum intel_dpll_id range_min,
+ enum intel_dpll_id range_max)
+{
+ struct drm_i915_private *dev_priv = crtc->base.dev->dev_private;
+ struct intel_shared_dpll *pll;
+ struct intel_shared_dpll_config *shared_dpll;
+ enum intel_dpll_id i;
+
+ shared_dpll = intel_atomic_get_shared_dpll_state(crtc_state->base.state);
+
+ for (i = range_min; i <= range_max; i++) {
+ pll = &dev_priv->shared_dplls[i];
+
+ /* Only want to check enabled timings first */
+ if (shared_dpll[i].crtc_mask == 0)
+ continue;
+
+ if (memcmp(&crtc_state->dpll_hw_state,
+ &shared_dpll[i].hw_state,
+ sizeof(crtc_state->dpll_hw_state)) == 0) {
+ DRM_DEBUG_KMS("CRTC:%d sharing existing %s (crtc mask 0x%08x, active %x)\n",
+ crtc->base.base.id, pll->name,
+ shared_dpll[i].crtc_mask,
+ pll->active_mask);
+ return pll;
+ }
+ }
+
+ /* Ok no matching timings, maybe there's a free one? */
+ for (i = range_min; i <= range_max; i++) {
+ pll = &dev_priv->shared_dplls[i];
+ if (shared_dpll[i].crtc_mask == 0) {
+ DRM_DEBUG_KMS("CRTC:%d allocated %s\n",
+ crtc->base.base.id, pll->name);
+ return pll;
+ }
+ }
+
+ return NULL;
+}
+
+static void
+intel_reference_shared_dpll(struct intel_shared_dpll *pll,
+ struct intel_crtc_state *crtc_state)
+{
+ struct intel_shared_dpll_config *shared_dpll;
+ struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
+ enum intel_dpll_id i = pll->id;
+
+ shared_dpll = intel_atomic_get_shared_dpll_state(crtc_state->base.state);
+
+ if (shared_dpll[i].crtc_mask == 0)
+ shared_dpll[i].hw_state =
+ crtc_state->dpll_hw_state;
+
+ crtc_state->shared_dpll = pll;
+ DRM_DEBUG_DRIVER("using %s for pipe %c\n", pll->name,
+ pipe_name(crtc->pipe));
+
+ intel_shared_dpll_config_get(shared_dpll, pll, crtc);
+}
+
+void intel_shared_dpll_commit(struct drm_atomic_state *state)
+{
+ struct drm_i915_private *dev_priv = to_i915(state->dev);
+ struct intel_shared_dpll_config *shared_dpll;
+ struct intel_shared_dpll *pll;
+ enum intel_dpll_id i;
+
+ if (!to_intel_atomic_state(state)->dpll_set)
+ return;
+
+ shared_dpll = to_intel_atomic_state(state)->shared_dpll;
+ for (i = 0; i < dev_priv->num_shared_dpll; i++) {
+ pll = &dev_priv->shared_dplls[i];
+ pll->config = shared_dpll[i];
+ }
+}
+
+static bool ibx_pch_dpll_get_hw_state(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll,
+ struct intel_dpll_hw_state *hw_state)
+{
+ uint32_t val;
+
+ if (!intel_display_power_get_if_enabled(dev_priv, POWER_DOMAIN_PLLS))
+ return false;
+
+ val = I915_READ(PCH_DPLL(pll->id));
+ hw_state->dpll = val;
+ hw_state->fp0 = I915_READ(PCH_FP0(pll->id));
+ hw_state->fp1 = I915_READ(PCH_FP1(pll->id));
+
+ intel_display_power_put(dev_priv, POWER_DOMAIN_PLLS);
+
+ return val & DPLL_VCO_ENABLE;
+}
+
+static void ibx_pch_dpll_mode_set(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll)
+{
+ I915_WRITE(PCH_FP0(pll->id), pll->config.hw_state.fp0);
+ I915_WRITE(PCH_FP1(pll->id), pll->config.hw_state.fp1);
+}
+
+static void ibx_assert_pch_refclk_enabled(struct drm_i915_private *dev_priv)
+{
+ u32 val;
+ bool enabled;
+
+ I915_STATE_WARN_ON(!(HAS_PCH_IBX(dev_priv) || HAS_PCH_CPT(dev_priv)));
+
+ val = I915_READ(PCH_DREF_CONTROL);
+ enabled = !!(val & (DREF_SSC_SOURCE_MASK | DREF_NONSPREAD_SOURCE_MASK |
+ DREF_SUPERSPREAD_SOURCE_MASK));
+ I915_STATE_WARN(!enabled, "PCH refclk assertion failure, should be active but is disabled\n");
+}
+
+static void ibx_pch_dpll_enable(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll)
+{
+ /* PCH refclock must be enabled first */
+ ibx_assert_pch_refclk_enabled(dev_priv);
+
+ I915_WRITE(PCH_DPLL(pll->id), pll->config.hw_state.dpll);
+
+ /* Wait for the clocks to stabilize. */
+ POSTING_READ(PCH_DPLL(pll->id));
+ udelay(150);
+
+ /* The pixel multiplier can only be updated once the
+ * DPLL is enabled and the clocks are stable.
+ *
+ * So write it again.
+ */
+ I915_WRITE(PCH_DPLL(pll->id), pll->config.hw_state.dpll);
+ POSTING_READ(PCH_DPLL(pll->id));
+ udelay(200);
+}
+
+static void ibx_pch_dpll_disable(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll)
+{
+ struct drm_device *dev = dev_priv->dev;
+ struct intel_crtc *crtc;
+
+ /* Make sure no transcoder isn't still depending on us. */
+ for_each_intel_crtc(dev, crtc) {
+ if (crtc->config->shared_dpll == pll)
+ assert_pch_transcoder_disabled(dev_priv, crtc->pipe);
+ }
+
+ I915_WRITE(PCH_DPLL(pll->id), 0);
+ POSTING_READ(PCH_DPLL(pll->id));
+ udelay(200);
+}
+
+static struct intel_shared_dpll *
+ibx_get_dpll(struct intel_crtc *crtc, struct intel_crtc_state *crtc_state,
+ struct intel_encoder *encoder)
+{
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct intel_shared_dpll *pll;
+ enum intel_dpll_id i;
+
+ if (HAS_PCH_IBX(dev_priv)) {
+ /* Ironlake PCH has a fixed PLL->PCH pipe mapping. */
+ i = (enum intel_dpll_id) crtc->pipe;
+ pll = &dev_priv->shared_dplls[i];
+
+ DRM_DEBUG_KMS("CRTC:%d using pre-allocated %s\n",
+ crtc->base.base.id, pll->name);
+ } else {
+ pll = intel_find_shared_dpll(crtc, crtc_state,
+ DPLL_ID_PCH_PLL_A,
+ DPLL_ID_PCH_PLL_B);
+ }
+
+ /* reference the pll */
+ intel_reference_shared_dpll(pll, crtc_state);
+
+ return pll;
+}
+
+static const struct intel_shared_dpll_funcs ibx_pch_dpll_funcs = {
+ .mode_set = ibx_pch_dpll_mode_set,
+ .enable = ibx_pch_dpll_enable,
+ .disable = ibx_pch_dpll_disable,
+ .get_hw_state = ibx_pch_dpll_get_hw_state,
+};
+
+static void hsw_ddi_wrpll_enable(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll)
+{
+ I915_WRITE(WRPLL_CTL(pll->id), pll->config.hw_state.wrpll);
+ POSTING_READ(WRPLL_CTL(pll->id));
+ udelay(20);
+}
+
+static void hsw_ddi_spll_enable(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll)
+{
+ I915_WRITE(SPLL_CTL, pll->config.hw_state.spll);
+ POSTING_READ(SPLL_CTL);
+ udelay(20);
+}
+
+static void hsw_ddi_wrpll_disable(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll)
+{
+ uint32_t val;
+
+ val = I915_READ(WRPLL_CTL(pll->id));
+ I915_WRITE(WRPLL_CTL(pll->id), val & ~WRPLL_PLL_ENABLE);
+ POSTING_READ(WRPLL_CTL(pll->id));
+}
+
+static void hsw_ddi_spll_disable(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll)
+{
+ uint32_t val;
+
+ val = I915_READ(SPLL_CTL);
+ I915_WRITE(SPLL_CTL, val & ~SPLL_PLL_ENABLE);
+ POSTING_READ(SPLL_CTL);
+}
+
+static bool hsw_ddi_wrpll_get_hw_state(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll,
+ struct intel_dpll_hw_state *hw_state)
+{
+ uint32_t val;
+
+ if (!intel_display_power_get_if_enabled(dev_priv, POWER_DOMAIN_PLLS))
+ return false;
+
+ val = I915_READ(WRPLL_CTL(pll->id));
+ hw_state->wrpll = val;
+
+ intel_display_power_put(dev_priv, POWER_DOMAIN_PLLS);
+
+ return val & WRPLL_PLL_ENABLE;
+}
+
+static bool hsw_ddi_spll_get_hw_state(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll,
+ struct intel_dpll_hw_state *hw_state)
+{
+ uint32_t val;
+
+ if (!intel_display_power_get_if_enabled(dev_priv, POWER_DOMAIN_PLLS))
+ return false;
+
+ val = I915_READ(SPLL_CTL);
+ hw_state->spll = val;
+
+ intel_display_power_put(dev_priv, POWER_DOMAIN_PLLS);
+
+ return val & SPLL_PLL_ENABLE;
+}
+
+static uint32_t hsw_pll_to_ddi_pll_sel(struct intel_shared_dpll *pll)
+{
+ switch (pll->id) {
+ case DPLL_ID_WRPLL1:
+ return PORT_CLK_SEL_WRPLL1;
+ case DPLL_ID_WRPLL2:
+ return PORT_CLK_SEL_WRPLL2;
+ case DPLL_ID_SPLL:
+ return PORT_CLK_SEL_SPLL;
+ case DPLL_ID_LCPLL_810:
+ return PORT_CLK_SEL_LCPLL_810;
+ case DPLL_ID_LCPLL_1350:
+ return PORT_CLK_SEL_LCPLL_1350;
+ case DPLL_ID_LCPLL_2700:
+ return PORT_CLK_SEL_LCPLL_2700;
+ default:
+ return PORT_CLK_SEL_NONE;
+ }
+}
+
+#define LC_FREQ 2700
+#define LC_FREQ_2K U64_C(LC_FREQ * 2000)
+
+#define P_MIN 2
+#define P_MAX 64
+#define P_INC 2
+
+/* Constraints for PLL good behavior */
+#define REF_MIN 48
+#define REF_MAX 400
+#define VCO_MIN 2400
+#define VCO_MAX 4800
+
+struct hsw_wrpll_rnp {
+ unsigned p, n2, r2;
+};
+
+static unsigned hsw_wrpll_get_budget_for_freq(int clock)
+{
+ unsigned budget;
+
+ switch (clock) {
+ case 25175000:
+ case 25200000:
+ case 27000000:
+ case 27027000:
+ case 37762500:
+ case 37800000:
+ case 40500000:
+ case 40541000:
+ case 54000000:
+ case 54054000:
+ case 59341000:
+ case 59400000:
+ case 72000000:
+ case 74176000:
+ case 74250000:
+ case 81000000:
+ case 81081000:
+ case 89012000:
+ case 89100000:
+ case 108000000:
+ case 108108000:
+ case 111264000:
+ case 111375000:
+ case 148352000:
+ case 148500000:
+ case 162000000:
+ case 162162000:
+ case 222525000:
+ case 222750000:
+ case 296703000:
+ case 297000000:
+ budget = 0;
+ break;
+ case 233500000:
+ case 245250000:
+ case 247750000:
+ case 253250000:
+ case 298000000:
+ budget = 1500;
+ break;
+ case 169128000:
+ case 169500000:
+ case 179500000:
+ case 202000000:
+ budget = 2000;
+ break;
+ case 256250000:
+ case 262500000:
+ case 270000000:
+ case 272500000:
+ case 273750000:
+ case 280750000:
+ case 281250000:
+ case 286000000:
+ case 291750000:
+ budget = 4000;
+ break;
+ case 267250000:
+ case 268500000:
+ budget = 5000;
+ break;
+ default:
+ budget = 1000;
+ break;
+ }
+
+ return budget;
+}
+
+static void hsw_wrpll_update_rnp(uint64_t freq2k, unsigned budget,
+ unsigned r2, unsigned n2, unsigned p,
+ struct hsw_wrpll_rnp *best)
+{
+ uint64_t a, b, c, d, diff, diff_best;
+
+ /* No best (r,n,p) yet */
+ if (best->p == 0) {
+ best->p = p;
+ best->n2 = n2;
+ best->r2 = r2;
+ return;
+ }
+
+ /*
+ * Output clock is (LC_FREQ_2K / 2000) * N / (P * R), which compares to
+ * freq2k.
+ *
+ * delta = 1e6 *
+ * abs(freq2k - (LC_FREQ_2K * n2/(p * r2))) /
+ * freq2k;
+ *
+ * and we would like delta <= budget.
+ *
+ * If the discrepancy is above the PPM-based budget, always prefer to
+ * improve upon the previous solution. However, if you're within the
+ * budget, try to maximize Ref * VCO, that is N / (P * R^2).
+ */
+ a = freq2k * budget * p * r2;
+ b = freq2k * budget * best->p * best->r2;
+ diff = abs_diff(freq2k * p * r2, LC_FREQ_2K * n2);
+ diff_best = abs_diff(freq2k * best->p * best->r2,
+ LC_FREQ_2K * best->n2);
+ c = 1000000 * diff;
+ d = 1000000 * diff_best;
+
+ if (a < c && b < d) {
+ /* If both are above the budget, pick the closer */
+ if (best->p * best->r2 * diff < p * r2 * diff_best) {
+ best->p = p;
+ best->n2 = n2;
+ best->r2 = r2;
+ }
+ } else if (a >= c && b < d) {
+ /* If A is below the threshold but B is above it? Update. */
+ best->p = p;
+ best->n2 = n2;
+ best->r2 = r2;
+ } else if (a >= c && b >= d) {
+ /* Both are below the limit, so pick the higher n2/(r2*r2) */
+ if (n2 * best->r2 * best->r2 > best->n2 * r2 * r2) {
+ best->p = p;
+ best->n2 = n2;
+ best->r2 = r2;
+ }
+ }
+ /* Otherwise a < c && b >= d, do nothing */
+}
+
+static void
+hsw_ddi_calculate_wrpll(int clock /* in Hz */,
+ unsigned *r2_out, unsigned *n2_out, unsigned *p_out)
+{
+ uint64_t freq2k;
+ unsigned p, n2, r2;
+ struct hsw_wrpll_rnp best = { 0, 0, 0 };
+ unsigned budget;
+
+ freq2k = clock / 100;
+
+ budget = hsw_wrpll_get_budget_for_freq(clock);
+
+ /* Special case handling for 540 pixel clock: bypass WR PLL entirely
+ * and directly pass the LC PLL to it. */
+ if (freq2k == 5400000) {
+ *n2_out = 2;
+ *p_out = 1;
+ *r2_out = 2;
+ return;
+ }
+
+ /*
+ * Ref = LC_FREQ / R, where Ref is the actual reference input seen by
+ * the WR PLL.
+ *
+ * We want R so that REF_MIN <= Ref <= REF_MAX.
+ * Injecting R2 = 2 * R gives:
+ * REF_MAX * r2 > LC_FREQ * 2 and
+ * REF_MIN * r2 < LC_FREQ * 2
+ *
+ * Which means the desired boundaries for r2 are:
+ * LC_FREQ * 2 / REF_MAX < r2 < LC_FREQ * 2 / REF_MIN
+ *
+ */
+ for (r2 = LC_FREQ * 2 / REF_MAX + 1;
+ r2 <= LC_FREQ * 2 / REF_MIN;
+ r2++) {
+
+ /*
+ * VCO = N * Ref, that is: VCO = N * LC_FREQ / R
+ *
+ * Once again we want VCO_MIN <= VCO <= VCO_MAX.
+ * Injecting R2 = 2 * R and N2 = 2 * N, we get:
+ * VCO_MAX * r2 > n2 * LC_FREQ and
+ * VCO_MIN * r2 < n2 * LC_FREQ)
+ *
+ * Which means the desired boundaries for n2 are:
+ * VCO_MIN * r2 / LC_FREQ < n2 < VCO_MAX * r2 / LC_FREQ
+ */
+ for (n2 = VCO_MIN * r2 / LC_FREQ + 1;
+ n2 <= VCO_MAX * r2 / LC_FREQ;
+ n2++) {
+
+ for (p = P_MIN; p <= P_MAX; p += P_INC)
+ hsw_wrpll_update_rnp(freq2k, budget,
+ r2, n2, p, &best);
+ }
+ }
+
+ *n2_out = best.n2;
+ *p_out = best.p;
+ *r2_out = best.r2;
+}
+
+static struct intel_shared_dpll *
+hsw_get_dpll(struct intel_crtc *crtc, struct intel_crtc_state *crtc_state,
+ struct intel_encoder *encoder)
+{
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct intel_shared_dpll *pll;
+ int clock = crtc_state->port_clock;
+
+ memset(&crtc_state->dpll_hw_state, 0,
+ sizeof(crtc_state->dpll_hw_state));
+
+ if (encoder->type == INTEL_OUTPUT_HDMI) {
+ uint32_t val;
+ unsigned p, n2, r2;
+
+ hsw_ddi_calculate_wrpll(clock * 1000, &r2, &n2, &p);
+
+ val = WRPLL_PLL_ENABLE | WRPLL_PLL_LCPLL |
+ WRPLL_DIVIDER_REFERENCE(r2) | WRPLL_DIVIDER_FEEDBACK(n2) |
+ WRPLL_DIVIDER_POST(p);
+
+ crtc_state->dpll_hw_state.wrpll = val;
+
+ pll = intel_find_shared_dpll(crtc, crtc_state,
+ DPLL_ID_WRPLL1, DPLL_ID_WRPLL2);
+
+ } else if (encoder->type == INTEL_OUTPUT_DISPLAYPORT ||
+ encoder->type == INTEL_OUTPUT_DP_MST ||
+ encoder->type == INTEL_OUTPUT_EDP) {
+ enum intel_dpll_id pll_id;
+
+ switch (clock / 2) {
+ case 81000:
+ pll_id = DPLL_ID_LCPLL_810;
+ break;
+ case 135000:
+ pll_id = DPLL_ID_LCPLL_1350;
+ break;
+ case 270000:
+ pll_id = DPLL_ID_LCPLL_2700;
+ break;
+ default:
+ DRM_DEBUG_KMS("Invalid clock for DP: %d\n", clock);
+ return NULL;
+ }
+
+ pll = intel_get_shared_dpll_by_id(dev_priv, pll_id);
+
+ } else if (encoder->type == INTEL_OUTPUT_ANALOG) {
+ if (WARN_ON(crtc_state->port_clock / 2 != 135000))
+ return NULL;
+
+ crtc_state->dpll_hw_state.spll =
+ SPLL_PLL_ENABLE | SPLL_PLL_FREQ_1350MHz | SPLL_PLL_SSC;
+
+ pll = intel_find_shared_dpll(crtc, crtc_state,
+ DPLL_ID_SPLL, DPLL_ID_SPLL);
+ } else {
+ return NULL;
+ }
+
+ if (!pll)
+ return NULL;
+
+ crtc_state->ddi_pll_sel = hsw_pll_to_ddi_pll_sel(pll);
+
+ intel_reference_shared_dpll(pll, crtc_state);
+
+ return pll;
+}
+
+
+static const struct intel_shared_dpll_funcs hsw_ddi_wrpll_funcs = {
+ .enable = hsw_ddi_wrpll_enable,
+ .disable = hsw_ddi_wrpll_disable,
+ .get_hw_state = hsw_ddi_wrpll_get_hw_state,
+};
+
+static const struct intel_shared_dpll_funcs hsw_ddi_spll_funcs = {
+ .enable = hsw_ddi_spll_enable,
+ .disable = hsw_ddi_spll_disable,
+ .get_hw_state = hsw_ddi_spll_get_hw_state,
+};
+
+static void hsw_ddi_lcpll_enable(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll)
+{
+}
+
+static void hsw_ddi_lcpll_disable(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll)
+{
+}
+
+static bool hsw_ddi_lcpll_get_hw_state(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll,
+ struct intel_dpll_hw_state *hw_state)
+{
+ return true;
+}
+
+static const struct intel_shared_dpll_funcs hsw_ddi_lcpll_funcs = {
+ .enable = hsw_ddi_lcpll_enable,
+ .disable = hsw_ddi_lcpll_disable,
+ .get_hw_state = hsw_ddi_lcpll_get_hw_state,
+};
+
+struct skl_dpll_regs {
+ i915_reg_t ctl, cfgcr1, cfgcr2;
+};
+
+/* this array is indexed by the *shared* pll id */
+static const struct skl_dpll_regs skl_dpll_regs[4] = {
+ {
+ /* DPLL 0 */
+ .ctl = LCPLL1_CTL,
+ /* DPLL 0 doesn't support HDMI mode */
+ },
+ {
+ /* DPLL 1 */
+ .ctl = LCPLL2_CTL,
+ .cfgcr1 = DPLL_CFGCR1(SKL_DPLL1),
+ .cfgcr2 = DPLL_CFGCR2(SKL_DPLL1),
+ },
+ {
+ /* DPLL 2 */
+ .ctl = WRPLL_CTL(0),
+ .cfgcr1 = DPLL_CFGCR1(SKL_DPLL2),
+ .cfgcr2 = DPLL_CFGCR2(SKL_DPLL2),
+ },
+ {
+ /* DPLL 3 */
+ .ctl = WRPLL_CTL(1),
+ .cfgcr1 = DPLL_CFGCR1(SKL_DPLL3),
+ .cfgcr2 = DPLL_CFGCR2(SKL_DPLL3),
+ },
+};
+
+static void skl_ddi_pll_write_ctrl1(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll)
+{
+ uint32_t val;
+
+ val = I915_READ(DPLL_CTRL1);
+
+ val &= ~(DPLL_CTRL1_HDMI_MODE(pll->id) | DPLL_CTRL1_SSC(pll->id) |
+ DPLL_CTRL1_LINK_RATE_MASK(pll->id));
+ val |= pll->config.hw_state.ctrl1 << (pll->id * 6);
+
+ I915_WRITE(DPLL_CTRL1, val);
+ POSTING_READ(DPLL_CTRL1);
+}
+
+static void skl_ddi_pll_enable(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll)
+{
+ const struct skl_dpll_regs *regs = skl_dpll_regs;
+
+ skl_ddi_pll_write_ctrl1(dev_priv, pll);
+
+ I915_WRITE(regs[pll->id].cfgcr1, pll->config.hw_state.cfgcr1);
+ I915_WRITE(regs[pll->id].cfgcr2, pll->config.hw_state.cfgcr2);
+ POSTING_READ(regs[pll->id].cfgcr1);
+ POSTING_READ(regs[pll->id].cfgcr2);
+
+ /* the enable bit is always bit 31 */
+ I915_WRITE(regs[pll->id].ctl,
+ I915_READ(regs[pll->id].ctl) | LCPLL_PLL_ENABLE);
+
+ if (wait_for(I915_READ(DPLL_STATUS) & DPLL_LOCK(pll->id), 5))
+ DRM_ERROR("DPLL %d not locked\n", pll->id);
+}
+
+static void skl_ddi_dpll0_enable(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll)
+{
+ skl_ddi_pll_write_ctrl1(dev_priv, pll);
+}
+
+static void skl_ddi_pll_disable(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll)
+{
+ const struct skl_dpll_regs *regs = skl_dpll_regs;
+
+ /* the enable bit is always bit 31 */
+ I915_WRITE(regs[pll->id].ctl,
+ I915_READ(regs[pll->id].ctl) & ~LCPLL_PLL_ENABLE);
+ POSTING_READ(regs[pll->id].ctl);
+}
+
+static void skl_ddi_dpll0_disable(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll)
+{
+}
+
+static bool skl_ddi_pll_get_hw_state(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll,
+ struct intel_dpll_hw_state *hw_state)
+{
+ uint32_t val;
+ const struct skl_dpll_regs *regs = skl_dpll_regs;
+ bool ret;
+
+ if (!intel_display_power_get_if_enabled(dev_priv, POWER_DOMAIN_PLLS))
+ return false;
+
+ ret = false;
+
+ val = I915_READ(regs[pll->id].ctl);
+ if (!(val & LCPLL_PLL_ENABLE))
+ goto out;
+
+ val = I915_READ(DPLL_CTRL1);
+ hw_state->ctrl1 = (val >> (pll->id * 6)) & 0x3f;
+
+ /* avoid reading back stale values if HDMI mode is not enabled */
+ if (val & DPLL_CTRL1_HDMI_MODE(pll->id)) {
+ hw_state->cfgcr1 = I915_READ(regs[pll->id].cfgcr1);
+ hw_state->cfgcr2 = I915_READ(regs[pll->id].cfgcr2);
+ }
+ ret = true;
+
+out:
+ intel_display_power_put(dev_priv, POWER_DOMAIN_PLLS);
+
+ return ret;
+}
+
+static bool skl_ddi_dpll0_get_hw_state(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll,
+ struct intel_dpll_hw_state *hw_state)
+{
+ uint32_t val;
+ const struct skl_dpll_regs *regs = skl_dpll_regs;
+ bool ret;
+
+ if (!intel_display_power_get_if_enabled(dev_priv, POWER_DOMAIN_PLLS))
+ return false;
+
+ ret = false;
+
+ /* DPLL0 is always enabled since it drives CDCLK */
+ val = I915_READ(regs[pll->id].ctl);
+ if (WARN_ON(!(val & LCPLL_PLL_ENABLE)))
+ goto out;
+
+ val = I915_READ(DPLL_CTRL1);
+ hw_state->ctrl1 = (val >> (pll->id * 6)) & 0x3f;
+
+ ret = true;
+
+out:
+ intel_display_power_put(dev_priv, POWER_DOMAIN_PLLS);
+
+ return ret;
+}
+
+struct skl_wrpll_context {
+ uint64_t min_deviation; /* current minimal deviation */
+ uint64_t central_freq; /* chosen central freq */
+ uint64_t dco_freq; /* chosen dco freq */
+ unsigned int p; /* chosen divider */
+};
+
+static void skl_wrpll_context_init(struct skl_wrpll_context *ctx)
+{
+ memset(ctx, 0, sizeof(*ctx));
+
+ ctx->min_deviation = U64_MAX;
+}
+
+/* DCO freq must be within +1%/-6% of the DCO central freq */
+#define SKL_DCO_MAX_PDEVIATION 100
+#define SKL_DCO_MAX_NDEVIATION 600
+
+static void skl_wrpll_try_divider(struct skl_wrpll_context *ctx,
+ uint64_t central_freq,
+ uint64_t dco_freq,
+ unsigned int divider)
+{
+ uint64_t deviation;
+
+ deviation = div64_u64(10000 * abs_diff(dco_freq, central_freq),
+ central_freq);
+
+ /* positive deviation */
+ if (dco_freq >= central_freq) {
+ if (deviation < SKL_DCO_MAX_PDEVIATION &&
+ deviation < ctx->min_deviation) {
+ ctx->min_deviation = deviation;
+ ctx->central_freq = central_freq;
+ ctx->dco_freq = dco_freq;
+ ctx->p = divider;
+ }
+ /* negative deviation */
+ } else if (deviation < SKL_DCO_MAX_NDEVIATION &&
+ deviation < ctx->min_deviation) {
+ ctx->min_deviation = deviation;
+ ctx->central_freq = central_freq;
+ ctx->dco_freq = dco_freq;
+ ctx->p = divider;
+ }
+}
+
+static void skl_wrpll_get_multipliers(unsigned int p,
+ unsigned int *p0 /* out */,
+ unsigned int *p1 /* out */,
+ unsigned int *p2 /* out */)
+{
+ /* even dividers */
+ if (p % 2 == 0) {
+ unsigned int half = p / 2;
+
+ if (half == 1 || half == 2 || half == 3 || half == 5) {
+ *p0 = 2;
+ *p1 = 1;
+ *p2 = half;
+ } else if (half % 2 == 0) {
+ *p0 = 2;
+ *p1 = half / 2;
+ *p2 = 2;
+ } else if (half % 3 == 0) {
+ *p0 = 3;
+ *p1 = half / 3;
+ *p2 = 2;
+ } else if (half % 7 == 0) {
+ *p0 = 7;
+ *p1 = half / 7;
+ *p2 = 2;
+ }
+ } else if (p == 3 || p == 9) { /* 3, 5, 7, 9, 15, 21, 35 */
+ *p0 = 3;
+ *p1 = 1;
+ *p2 = p / 3;
+ } else if (p == 5 || p == 7) {
+ *p0 = p;
+ *p1 = 1;
+ *p2 = 1;
+ } else if (p == 15) {
+ *p0 = 3;
+ *p1 = 1;
+ *p2 = 5;
+ } else if (p == 21) {
+ *p0 = 7;
+ *p1 = 1;
+ *p2 = 3;
+ } else if (p == 35) {
+ *p0 = 7;
+ *p1 = 1;
+ *p2 = 5;
+ }
+}
+
+struct skl_wrpll_params {
+ uint32_t dco_fraction;
+ uint32_t dco_integer;
+ uint32_t qdiv_ratio;
+ uint32_t qdiv_mode;
+ uint32_t kdiv;
+ uint32_t pdiv;
+ uint32_t central_freq;
+};
+
+static void skl_wrpll_params_populate(struct skl_wrpll_params *params,
+ uint64_t afe_clock,
+ uint64_t central_freq,
+ uint32_t p0, uint32_t p1, uint32_t p2)
+{
+ uint64_t dco_freq;
+
+ switch (central_freq) {
+ case 9600000000ULL:
+ params->central_freq = 0;
+ break;
+ case 9000000000ULL:
+ params->central_freq = 1;
+ break;
+ case 8400000000ULL:
+ params->central_freq = 3;
+ }
+
+ switch (p0) {
+ case 1:
+ params->pdiv = 0;
+ break;
+ case 2:
+ params->pdiv = 1;
+ break;
+ case 3:
+ params->pdiv = 2;
+ break;
+ case 7:
+ params->pdiv = 4;
+ break;
+ default:
+ WARN(1, "Incorrect PDiv\n");
+ }
+
+ switch (p2) {
+ case 5:
+ params->kdiv = 0;
+ break;
+ case 2:
+ params->kdiv = 1;
+ break;
+ case 3:
+ params->kdiv = 2;
+ break;
+ case 1:
+ params->kdiv = 3;
+ break;
+ default:
+ WARN(1, "Incorrect KDiv\n");
+ }
+
+ params->qdiv_ratio = p1;
+ params->qdiv_mode = (params->qdiv_ratio == 1) ? 0 : 1;
+
+ dco_freq = p0 * p1 * p2 * afe_clock;
+
+ /*
+ * Intermediate values are in Hz.
+ * Divide by MHz to match bsepc
+ */
+ params->dco_integer = div_u64(dco_freq, 24 * MHz(1));
+ params->dco_fraction =
+ div_u64((div_u64(dco_freq, 24) -
+ params->dco_integer * MHz(1)) * 0x8000, MHz(1));
+}
+
+static bool
+skl_ddi_calculate_wrpll(int clock /* in Hz */,
+ struct skl_wrpll_params *wrpll_params)
+{
+ uint64_t afe_clock = clock * 5; /* AFE Clock is 5x Pixel clock */
+ uint64_t dco_central_freq[3] = {8400000000ULL,
+ 9000000000ULL,
+ 9600000000ULL};
+ static const int even_dividers[] = { 4, 6, 8, 10, 12, 14, 16, 18, 20,
+ 24, 28, 30, 32, 36, 40, 42, 44,
+ 48, 52, 54, 56, 60, 64, 66, 68,
+ 70, 72, 76, 78, 80, 84, 88, 90,
+ 92, 96, 98 };
+ static const int odd_dividers[] = { 3, 5, 7, 9, 15, 21, 35 };
+ static const struct {
+ const int *list;
+ int n_dividers;
+ } dividers[] = {
+ { even_dividers, ARRAY_SIZE(even_dividers) },
+ { odd_dividers, ARRAY_SIZE(odd_dividers) },
+ };
+ struct skl_wrpll_context ctx;
+ unsigned int dco, d, i;
+ unsigned int p0, p1, p2;
+
+ skl_wrpll_context_init(&ctx);
+
+ for (d = 0; d < ARRAY_SIZE(dividers); d++) {
+ for (dco = 0; dco < ARRAY_SIZE(dco_central_freq); dco++) {
+ for (i = 0; i < dividers[d].n_dividers; i++) {
+ unsigned int p = dividers[d].list[i];
+ uint64_t dco_freq = p * afe_clock;
+
+ skl_wrpll_try_divider(&ctx,
+ dco_central_freq[dco],
+ dco_freq,
+ p);
+ /*
+ * Skip the remaining dividers if we're sure to
+ * have found the definitive divider, we can't
+ * improve a 0 deviation.
+ */
+ if (ctx.min_deviation == 0)
+ goto skip_remaining_dividers;
+ }
+ }
+
+skip_remaining_dividers:
+ /*
+ * If a solution is found with an even divider, prefer
+ * this one.
+ */
+ if (d == 0 && ctx.p)
+ break;
+ }
+
+ if (!ctx.p) {
+ DRM_DEBUG_DRIVER("No valid divider found for %dHz\n", clock);
+ return false;
+ }
+
+ /*
+ * gcc incorrectly analyses that these can be used without being
+ * initialized. To be fair, it's hard to guess.
+ */
+ p0 = p1 = p2 = 0;
+ skl_wrpll_get_multipliers(ctx.p, &p0, &p1, &p2);
+ skl_wrpll_params_populate(wrpll_params, afe_clock, ctx.central_freq,
+ p0, p1, p2);
+
+ return true;
+}
+
+static struct intel_shared_dpll *
+skl_get_dpll(struct intel_crtc *crtc, struct intel_crtc_state *crtc_state,
+ struct intel_encoder *encoder)
+{
+ struct intel_shared_dpll *pll;
+ uint32_t ctrl1, cfgcr1, cfgcr2;
+ int clock = crtc_state->port_clock;
+
+ /*
+ * See comment in intel_dpll_hw_state to understand why we always use 0
+ * as the DPLL id in this function.
+ */
+
+ ctrl1 = DPLL_CTRL1_OVERRIDE(0);
+
+ if (encoder->type == INTEL_OUTPUT_HDMI) {
+ struct skl_wrpll_params wrpll_params = { 0, };
+
+ ctrl1 |= DPLL_CTRL1_HDMI_MODE(0);
+
+ if (!skl_ddi_calculate_wrpll(clock * 1000, &wrpll_params))
+ return NULL;
+
+ cfgcr1 = DPLL_CFGCR1_FREQ_ENABLE |
+ DPLL_CFGCR1_DCO_FRACTION(wrpll_params.dco_fraction) |
+ wrpll_params.dco_integer;
+
+ cfgcr2 = DPLL_CFGCR2_QDIV_RATIO(wrpll_params.qdiv_ratio) |
+ DPLL_CFGCR2_QDIV_MODE(wrpll_params.qdiv_mode) |
+ DPLL_CFGCR2_KDIV(wrpll_params.kdiv) |
+ DPLL_CFGCR2_PDIV(wrpll_params.pdiv) |
+ wrpll_params.central_freq;
+ } else if (encoder->type == INTEL_OUTPUT_DISPLAYPORT ||
+ encoder->type == INTEL_OUTPUT_DP_MST ||
+ encoder->type == INTEL_OUTPUT_EDP) {
+ switch (crtc_state->port_clock / 2) {
+ case 81000:
+ ctrl1 |= DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_810, 0);
+ break;
+ case 135000:
+ ctrl1 |= DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_1350, 0);
+ break;
+ case 270000:
+ ctrl1 |= DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_2700, 0);
+ break;
+ /* eDP 1.4 rates */
+ case 162000:
+ ctrl1 |= DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_1620, 0);
+ break;
+ /* TBD: For DP link rates 2.16 GHz and 4.32 GHz, VCO is 8640 which
+ results in CDCLK change. Need to handle the change of CDCLK by
+ disabling pipes and re-enabling them */
+ case 108000:
+ ctrl1 |= DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_1080, 0);
+ break;
+ case 216000:
+ ctrl1 |= DPLL_CTRL1_LINK_RATE(DPLL_CTRL1_LINK_RATE_2160, 0);
+ break;
+ }
+
+ cfgcr1 = cfgcr2 = 0;
+ } else {
+ return NULL;
+ }
+
+ memset(&crtc_state->dpll_hw_state, 0,
+ sizeof(crtc_state->dpll_hw_state));
+
+ crtc_state->dpll_hw_state.ctrl1 = ctrl1;
+ crtc_state->dpll_hw_state.cfgcr1 = cfgcr1;
+ crtc_state->dpll_hw_state.cfgcr2 = cfgcr2;
+
+ if (encoder->type == INTEL_OUTPUT_EDP)
+ pll = intel_find_shared_dpll(crtc, crtc_state,
+ DPLL_ID_SKL_DPLL0,
+ DPLL_ID_SKL_DPLL0);
+ else
+ pll = intel_find_shared_dpll(crtc, crtc_state,
+ DPLL_ID_SKL_DPLL1,
+ DPLL_ID_SKL_DPLL3);
+ if (!pll)
+ return NULL;
+
+ crtc_state->ddi_pll_sel = pll->id;
+
+ intel_reference_shared_dpll(pll, crtc_state);
+
+ return pll;
+}
+
+static const struct intel_shared_dpll_funcs skl_ddi_pll_funcs = {
+ .enable = skl_ddi_pll_enable,
+ .disable = skl_ddi_pll_disable,
+ .get_hw_state = skl_ddi_pll_get_hw_state,
+};
+
+static const struct intel_shared_dpll_funcs skl_ddi_dpll0_funcs = {
+ .enable = skl_ddi_dpll0_enable,
+ .disable = skl_ddi_dpll0_disable,
+ .get_hw_state = skl_ddi_dpll0_get_hw_state,
+};
+
+static void bxt_ddi_pll_enable(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll)
+{
+ uint32_t temp;
+ enum port port = (enum port)pll->id; /* 1:1 port->PLL mapping */
+
+ /* Non-SSC reference */
+ temp = I915_READ(BXT_PORT_PLL_ENABLE(port));
+ temp |= PORT_PLL_REF_SEL;
+ I915_WRITE(BXT_PORT_PLL_ENABLE(port), temp);
+
+ /* Disable 10 bit clock */
+ temp = I915_READ(BXT_PORT_PLL_EBB_4(port));
+ temp &= ~PORT_PLL_10BIT_CLK_ENABLE;
+ I915_WRITE(BXT_PORT_PLL_EBB_4(port), temp);
+
+ /* Write P1 & P2 */
+ temp = I915_READ(BXT_PORT_PLL_EBB_0(port));
+ temp &= ~(PORT_PLL_P1_MASK | PORT_PLL_P2_MASK);
+ temp |= pll->config.hw_state.ebb0;
+ I915_WRITE(BXT_PORT_PLL_EBB_0(port), temp);
+
+ /* Write M2 integer */
+ temp = I915_READ(BXT_PORT_PLL(port, 0));
+ temp &= ~PORT_PLL_M2_MASK;
+ temp |= pll->config.hw_state.pll0;
+ I915_WRITE(BXT_PORT_PLL(port, 0), temp);
+
+ /* Write N */
+ temp = I915_READ(BXT_PORT_PLL(port, 1));
+ temp &= ~PORT_PLL_N_MASK;
+ temp |= pll->config.hw_state.pll1;
+ I915_WRITE(BXT_PORT_PLL(port, 1), temp);
+
+ /* Write M2 fraction */
+ temp = I915_READ(BXT_PORT_PLL(port, 2));
+ temp &= ~PORT_PLL_M2_FRAC_MASK;
+ temp |= pll->config.hw_state.pll2;
+ I915_WRITE(BXT_PORT_PLL(port, 2), temp);
+
+ /* Write M2 fraction enable */
+ temp = I915_READ(BXT_PORT_PLL(port, 3));
+ temp &= ~PORT_PLL_M2_FRAC_ENABLE;
+ temp |= pll->config.hw_state.pll3;
+ I915_WRITE(BXT_PORT_PLL(port, 3), temp);
+
+ /* Write coeff */
+ temp = I915_READ(BXT_PORT_PLL(port, 6));
+ temp &= ~PORT_PLL_PROP_COEFF_MASK;
+ temp &= ~PORT_PLL_INT_COEFF_MASK;
+ temp &= ~PORT_PLL_GAIN_CTL_MASK;
+ temp |= pll->config.hw_state.pll6;
+ I915_WRITE(BXT_PORT_PLL(port, 6), temp);
+
+ /* Write calibration val */
+ temp = I915_READ(BXT_PORT_PLL(port, 8));
+ temp &= ~PORT_PLL_TARGET_CNT_MASK;
+ temp |= pll->config.hw_state.pll8;
+ I915_WRITE(BXT_PORT_PLL(port, 8), temp);
+
+ temp = I915_READ(BXT_PORT_PLL(port, 9));
+ temp &= ~PORT_PLL_LOCK_THRESHOLD_MASK;
+ temp |= pll->config.hw_state.pll9;
+ I915_WRITE(BXT_PORT_PLL(port, 9), temp);
+
+ temp = I915_READ(BXT_PORT_PLL(port, 10));
+ temp &= ~PORT_PLL_DCO_AMP_OVR_EN_H;
+ temp &= ~PORT_PLL_DCO_AMP_MASK;
+ temp |= pll->config.hw_state.pll10;
+ I915_WRITE(BXT_PORT_PLL(port, 10), temp);
+
+ /* Recalibrate with new settings */
+ temp = I915_READ(BXT_PORT_PLL_EBB_4(port));
+ temp |= PORT_PLL_RECALIBRATE;
+ I915_WRITE(BXT_PORT_PLL_EBB_4(port), temp);
+ temp &= ~PORT_PLL_10BIT_CLK_ENABLE;
+ temp |= pll->config.hw_state.ebb4;
+ I915_WRITE(BXT_PORT_PLL_EBB_4(port), temp);
+
+ /* Enable PLL */
+ temp = I915_READ(BXT_PORT_PLL_ENABLE(port));
+ temp |= PORT_PLL_ENABLE;
+ I915_WRITE(BXT_PORT_PLL_ENABLE(port), temp);
+ POSTING_READ(BXT_PORT_PLL_ENABLE(port));
+
+ if (wait_for_atomic_us((I915_READ(BXT_PORT_PLL_ENABLE(port)) &
+ PORT_PLL_LOCK), 200))
+ DRM_ERROR("PLL %d not locked\n", port);
+
+ /*
+ * While we write to the group register to program all lanes at once we
+ * can read only lane registers and we pick lanes 0/1 for that.
+ */
+ temp = I915_READ(BXT_PORT_PCS_DW12_LN01(port));
+ temp &= ~LANE_STAGGER_MASK;
+ temp &= ~LANESTAGGER_STRAP_OVRD;
+ temp |= pll->config.hw_state.pcsdw12;
+ I915_WRITE(BXT_PORT_PCS_DW12_GRP(port), temp);
+}
+
+static void bxt_ddi_pll_disable(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll)
+{
+ enum port port = (enum port)pll->id; /* 1:1 port->PLL mapping */
+ uint32_t temp;
+
+ temp = I915_READ(BXT_PORT_PLL_ENABLE(port));
+ temp &= ~PORT_PLL_ENABLE;
+ I915_WRITE(BXT_PORT_PLL_ENABLE(port), temp);
+ POSTING_READ(BXT_PORT_PLL_ENABLE(port));
+}
+
+static bool bxt_ddi_pll_get_hw_state(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll,
+ struct intel_dpll_hw_state *hw_state)
+{
+ enum port port = (enum port)pll->id; /* 1:1 port->PLL mapping */
+ uint32_t val;
+ bool ret;
+
+ if (!intel_display_power_get_if_enabled(dev_priv, POWER_DOMAIN_PLLS))
+ return false;
+
+ ret = false;
+
+ val = I915_READ(BXT_PORT_PLL_ENABLE(port));
+ if (!(val & PORT_PLL_ENABLE))
+ goto out;
+
+ hw_state->ebb0 = I915_READ(BXT_PORT_PLL_EBB_0(port));
+ hw_state->ebb0 &= PORT_PLL_P1_MASK | PORT_PLL_P2_MASK;
+
+ hw_state->ebb4 = I915_READ(BXT_PORT_PLL_EBB_4(port));
+ hw_state->ebb4 &= PORT_PLL_10BIT_CLK_ENABLE;
+
+ hw_state->pll0 = I915_READ(BXT_PORT_PLL(port, 0));
+ hw_state->pll0 &= PORT_PLL_M2_MASK;
+
+ hw_state->pll1 = I915_READ(BXT_PORT_PLL(port, 1));
+ hw_state->pll1 &= PORT_PLL_N_MASK;
+
+ hw_state->pll2 = I915_READ(BXT_PORT_PLL(port, 2));
+ hw_state->pll2 &= PORT_PLL_M2_FRAC_MASK;
+
+ hw_state->pll3 = I915_READ(BXT_PORT_PLL(port, 3));
+ hw_state->pll3 &= PORT_PLL_M2_FRAC_ENABLE;
+
+ hw_state->pll6 = I915_READ(BXT_PORT_PLL(port, 6));
+ hw_state->pll6 &= PORT_PLL_PROP_COEFF_MASK |
+ PORT_PLL_INT_COEFF_MASK |
+ PORT_PLL_GAIN_CTL_MASK;
+
+ hw_state->pll8 = I915_READ(BXT_PORT_PLL(port, 8));
+ hw_state->pll8 &= PORT_PLL_TARGET_CNT_MASK;
+
+ hw_state->pll9 = I915_READ(BXT_PORT_PLL(port, 9));
+ hw_state->pll9 &= PORT_PLL_LOCK_THRESHOLD_MASK;
+
+ hw_state->pll10 = I915_READ(BXT_PORT_PLL(port, 10));
+ hw_state->pll10 &= PORT_PLL_DCO_AMP_OVR_EN_H |
+ PORT_PLL_DCO_AMP_MASK;
+
+ /*
+ * While we write to the group register to program all lanes at once we
+ * can read only lane registers. We configure all lanes the same way, so
+ * here just read out lanes 0/1 and output a note if lanes 2/3 differ.
+ */
+ hw_state->pcsdw12 = I915_READ(BXT_PORT_PCS_DW12_LN01(port));
+ if (I915_READ(BXT_PORT_PCS_DW12_LN23(port)) != hw_state->pcsdw12)
+ DRM_DEBUG_DRIVER("lane stagger config different for lane 01 (%08x) and 23 (%08x)\n",
+ hw_state->pcsdw12,
+ I915_READ(BXT_PORT_PCS_DW12_LN23(port)));
+ hw_state->pcsdw12 &= LANE_STAGGER_MASK | LANESTAGGER_STRAP_OVRD;
+
+ ret = true;
+
+out:
+ intel_display_power_put(dev_priv, POWER_DOMAIN_PLLS);
+
+ return ret;
+}
+
+/* bxt clock parameters */
+struct bxt_clk_div {
+ int clock;
+ uint32_t p1;
+ uint32_t p2;
+ uint32_t m2_int;
+ uint32_t m2_frac;
+ bool m2_frac_en;
+ uint32_t n;
+};
+
+/* pre-calculated values for DP linkrates */
+static const struct bxt_clk_div bxt_dp_clk_val[] = {
+ {162000, 4, 2, 32, 1677722, 1, 1},
+ {270000, 4, 1, 27, 0, 0, 1},
+ {540000, 2, 1, 27, 0, 0, 1},
+ {216000, 3, 2, 32, 1677722, 1, 1},
+ {243000, 4, 1, 24, 1258291, 1, 1},
+ {324000, 4, 1, 32, 1677722, 1, 1},
+ {432000, 3, 1, 32, 1677722, 1, 1}
+};
+
+static struct intel_shared_dpll *
+bxt_get_dpll(struct intel_crtc *crtc, struct intel_crtc_state *crtc_state,
+ struct intel_encoder *encoder)
+{
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ struct intel_shared_dpll *pll;
+ enum intel_dpll_id i;
+ struct intel_digital_port *intel_dig_port;
+ struct bxt_clk_div clk_div = {0};
+ int vco = 0;
+ uint32_t prop_coef, int_coef, gain_ctl, targ_cnt;
+ uint32_t lanestagger;
+ int clock = crtc_state->port_clock;
+
+ if (encoder->type == INTEL_OUTPUT_HDMI) {
+ intel_clock_t best_clock;
+
+ /* Calculate HDMI div */
+ /*
+ * FIXME: tie the following calculation into
+ * i9xx_crtc_compute_clock
+ */
+ if (!bxt_find_best_dpll(crtc_state, clock, &best_clock)) {
+ DRM_DEBUG_DRIVER("no PLL dividers found for clock %d pipe %c\n",
+ clock, pipe_name(crtc->pipe));
+ return NULL;
+ }
+
+ clk_div.p1 = best_clock.p1;
+ clk_div.p2 = best_clock.p2;
+ WARN_ON(best_clock.m1 != 2);
+ clk_div.n = best_clock.n;
+ clk_div.m2_int = best_clock.m2 >> 22;
+ clk_div.m2_frac = best_clock.m2 & ((1 << 22) - 1);
+ clk_div.m2_frac_en = clk_div.m2_frac != 0;
+
+ vco = best_clock.vco;
+ } else if (encoder->type == INTEL_OUTPUT_DISPLAYPORT ||
+ encoder->type == INTEL_OUTPUT_EDP) {
+ int i;
+
+ clk_div = bxt_dp_clk_val[0];
+ for (i = 0; i < ARRAY_SIZE(bxt_dp_clk_val); ++i) {
+ if (bxt_dp_clk_val[i].clock == clock) {
+ clk_div = bxt_dp_clk_val[i];
+ break;
+ }
+ }
+ vco = clock * 10 / 2 * clk_div.p1 * clk_div.p2;
+ }
+
+ if (vco >= 6200000 && vco <= 6700000) {
+ prop_coef = 4;
+ int_coef = 9;
+ gain_ctl = 3;
+ targ_cnt = 8;
+ } else if ((vco > 5400000 && vco < 6200000) ||
+ (vco >= 4800000 && vco < 5400000)) {
+ prop_coef = 5;
+ int_coef = 11;
+ gain_ctl = 3;
+ targ_cnt = 9;
+ } else if (vco == 5400000) {
+ prop_coef = 3;
+ int_coef = 8;
+ gain_ctl = 1;
+ targ_cnt = 9;
+ } else {
+ DRM_ERROR("Invalid VCO\n");
+ return NULL;
+ }
+
+ memset(&crtc_state->dpll_hw_state, 0,
+ sizeof(crtc_state->dpll_hw_state));
+
+ if (clock > 270000)
+ lanestagger = 0x18;
+ else if (clock > 135000)
+ lanestagger = 0x0d;
+ else if (clock > 67000)
+ lanestagger = 0x07;
+ else if (clock > 33000)
+ lanestagger = 0x04;
+ else
+ lanestagger = 0x02;
+
+ crtc_state->dpll_hw_state.ebb0 =
+ PORT_PLL_P1(clk_div.p1) | PORT_PLL_P2(clk_div.p2);
+ crtc_state->dpll_hw_state.pll0 = clk_div.m2_int;
+ crtc_state->dpll_hw_state.pll1 = PORT_PLL_N(clk_div.n);
+ crtc_state->dpll_hw_state.pll2 = clk_div.m2_frac;
+
+ if (clk_div.m2_frac_en)
+ crtc_state->dpll_hw_state.pll3 =
+ PORT_PLL_M2_FRAC_ENABLE;
+
+ crtc_state->dpll_hw_state.pll6 =
+ prop_coef | PORT_PLL_INT_COEFF(int_coef);
+ crtc_state->dpll_hw_state.pll6 |=
+ PORT_PLL_GAIN_CTL(gain_ctl);
+
+ crtc_state->dpll_hw_state.pll8 = targ_cnt;
+
+ crtc_state->dpll_hw_state.pll9 = 5 << PORT_PLL_LOCK_THRESHOLD_SHIFT;
+
+ crtc_state->dpll_hw_state.pll10 =
+ PORT_PLL_DCO_AMP(PORT_PLL_DCO_AMP_DEFAULT)
+ | PORT_PLL_DCO_AMP_OVR_EN_H;
+
+ crtc_state->dpll_hw_state.ebb4 = PORT_PLL_10BIT_CLK_ENABLE;
+
+ crtc_state->dpll_hw_state.pcsdw12 =
+ LANESTAGGER_STRAP_OVRD | lanestagger;
+
+ intel_dig_port = enc_to_dig_port(&encoder->base);
+
+ /* 1:1 mapping between ports and PLLs */
+ i = (enum intel_dpll_id) intel_dig_port->port;
+ pll = intel_get_shared_dpll_by_id(dev_priv, i);
+
+ DRM_DEBUG_KMS("CRTC:%d using pre-allocated %s\n",
+ crtc->base.base.id, pll->name);
+
+ intel_reference_shared_dpll(pll, crtc_state);
+
+ /* shared DPLL id 0 is DPLL A */
+ crtc_state->ddi_pll_sel = pll->id;
+
+ return pll;
+}
+
+static const struct intel_shared_dpll_funcs bxt_ddi_pll_funcs = {
+ .enable = bxt_ddi_pll_enable,
+ .disable = bxt_ddi_pll_disable,
+ .get_hw_state = bxt_ddi_pll_get_hw_state,
+};
+
+static void intel_ddi_pll_init(struct drm_device *dev)
+{
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ uint32_t val = I915_READ(LCPLL_CTL);
+
+ if (IS_SKYLAKE(dev) || IS_KABYLAKE(dev)) {
+ int cdclk_freq;
+
+ cdclk_freq = dev_priv->display.get_display_clock_speed(dev);
+ dev_priv->skl_boot_cdclk = cdclk_freq;
+ if (skl_sanitize_cdclk(dev_priv))
+ DRM_DEBUG_KMS("Sanitized cdclk programmed by pre-os\n");
+ if (!(I915_READ(LCPLL1_CTL) & LCPLL_PLL_ENABLE))
+ DRM_ERROR("LCPLL1 is disabled\n");
+ } else if (!IS_BROXTON(dev_priv)) {
+ /*
+ * The LCPLL register should be turned on by the BIOS. For now
+ * let's just check its state and print errors in case
+ * something is wrong. Don't even try to turn it on.
+ */
+
+ if (val & LCPLL_CD_SOURCE_FCLK)
+ DRM_ERROR("CDCLK source is not LCPLL\n");
+
+ if (val & LCPLL_PLL_DISABLE)
+ DRM_ERROR("LCPLL is disabled\n");
+ }
+}
+
+struct dpll_info {
+ const char *name;
+ const int id;
+ const struct intel_shared_dpll_funcs *funcs;
+ uint32_t flags;
+};
+
+struct intel_dpll_mgr {
+ const struct dpll_info *dpll_info;
+
+ struct intel_shared_dpll *(*get_dpll)(struct intel_crtc *crtc,
+ struct intel_crtc_state *crtc_state,
+ struct intel_encoder *encoder);
+};
+
+static const struct dpll_info pch_plls[] = {
+ { "PCH DPLL A", DPLL_ID_PCH_PLL_A, &ibx_pch_dpll_funcs, 0 },
+ { "PCH DPLL B", DPLL_ID_PCH_PLL_B, &ibx_pch_dpll_funcs, 0 },
+ { NULL, -1, NULL, 0 },
+};
+
+static const struct intel_dpll_mgr pch_pll_mgr = {
+ .dpll_info = pch_plls,
+ .get_dpll = ibx_get_dpll,
+};
+
+static const struct dpll_info hsw_plls[] = {
+ { "WRPLL 1", DPLL_ID_WRPLL1, &hsw_ddi_wrpll_funcs, 0 },
+ { "WRPLL 2", DPLL_ID_WRPLL2, &hsw_ddi_wrpll_funcs, 0 },
+ { "SPLL", DPLL_ID_SPLL, &hsw_ddi_spll_funcs, 0 },
+ { "LCPLL 810", DPLL_ID_LCPLL_810, &hsw_ddi_lcpll_funcs, INTEL_DPLL_ALWAYS_ON },
+ { "LCPLL 1350", DPLL_ID_LCPLL_1350, &hsw_ddi_lcpll_funcs, INTEL_DPLL_ALWAYS_ON },
+ { "LCPLL 2700", DPLL_ID_LCPLL_2700, &hsw_ddi_lcpll_funcs, INTEL_DPLL_ALWAYS_ON },
+ { NULL, -1, NULL, },
+};
+
+static const struct intel_dpll_mgr hsw_pll_mgr = {
+ .dpll_info = hsw_plls,
+ .get_dpll = hsw_get_dpll,
+};
+
+static const struct dpll_info skl_plls[] = {
+ { "DPLL 0", DPLL_ID_SKL_DPLL0, &skl_ddi_dpll0_funcs, INTEL_DPLL_ALWAYS_ON },
+ { "DPPL 1", DPLL_ID_SKL_DPLL1, &skl_ddi_pll_funcs, 0 },
+ { "DPPL 2", DPLL_ID_SKL_DPLL2, &skl_ddi_pll_funcs, 0 },
+ { "DPPL 3", DPLL_ID_SKL_DPLL3, &skl_ddi_pll_funcs, 0 },
+ { NULL, -1, NULL, },
+};
+
+static const struct intel_dpll_mgr skl_pll_mgr = {
+ .dpll_info = skl_plls,
+ .get_dpll = skl_get_dpll,
+};
+
+static const struct dpll_info bxt_plls[] = {
+ { "PORT PLL A", DPLL_ID_SKL_DPLL0, &bxt_ddi_pll_funcs, 0 },
+ { "PORT PLL B", DPLL_ID_SKL_DPLL1, &bxt_ddi_pll_funcs, 0 },
+ { "PORT PLL C", DPLL_ID_SKL_DPLL2, &bxt_ddi_pll_funcs, 0 },
+ { NULL, -1, NULL, },
+};
+
+static const struct intel_dpll_mgr bxt_pll_mgr = {
+ .dpll_info = bxt_plls,
+ .get_dpll = bxt_get_dpll,
+};
+
+void intel_shared_dpll_init(struct drm_device *dev)
+{
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ const struct intel_dpll_mgr *dpll_mgr = NULL;
+ const struct dpll_info *dpll_info;
+ int i;
+
+ if (IS_SKYLAKE(dev) || IS_KABYLAKE(dev))
+ dpll_mgr = &skl_pll_mgr;
+ else if (IS_BROXTON(dev))
+ dpll_mgr = &bxt_pll_mgr;
+ else if (HAS_DDI(dev))
+ dpll_mgr = &hsw_pll_mgr;
+ else if (HAS_PCH_IBX(dev) || HAS_PCH_CPT(dev))
+ dpll_mgr = &pch_pll_mgr;
+
+ if (!dpll_mgr) {
+ dev_priv->num_shared_dpll = 0;
+ return;
+ }
+
+ dpll_info = dpll_mgr->dpll_info;
+
+ for (i = 0; dpll_info[i].id >= 0; i++) {
+ WARN_ON(i != dpll_info[i].id);
+
+ dev_priv->shared_dplls[i].id = dpll_info[i].id;
+ dev_priv->shared_dplls[i].name = dpll_info[i].name;
+ dev_priv->shared_dplls[i].funcs = *dpll_info[i].funcs;
+ dev_priv->shared_dplls[i].flags = dpll_info[i].flags;
+ }
+
+ dev_priv->dpll_mgr = dpll_mgr;
+ dev_priv->num_shared_dpll = i;
+ mutex_init(&dev_priv->dpll_lock);
+
+ BUG_ON(dev_priv->num_shared_dpll > I915_NUM_PLLS);
+
+ /* FIXME: Move this to a more suitable place */
+ if (HAS_DDI(dev))
+ intel_ddi_pll_init(dev);
+}
+
+struct intel_shared_dpll *
+intel_get_shared_dpll(struct intel_crtc *crtc,
+ struct intel_crtc_state *crtc_state,
+ struct intel_encoder *encoder)
+{
+ struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
+ const struct intel_dpll_mgr *dpll_mgr = dev_priv->dpll_mgr;
+
+ if (WARN_ON(!dpll_mgr))
+ return NULL;
+
+ return dpll_mgr->get_dpll(crtc, crtc_state, encoder);
+}
diff --git a/drivers/gpu/drm/i915/intel_dpll_mgr.h b/drivers/gpu/drm/i915/intel_dpll_mgr.h
new file mode 100644
index 0000000..89c5ada
--- /dev/null
+++ b/drivers/gpu/drm/i915/intel_dpll_mgr.h
@@ -0,0 +1,164 @@
+/*
+ * Copyright © 2012-2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#ifndef _INTEL_DPLL_MGR_H_
+#define _INTEL_DPLL_MGR_H_
+
+/*FIXME: Move this to a more appropriate place. */
+#define abs_diff(a, b) ({ \
+ typeof(a) __a = (a); \
+ typeof(b) __b = (b); \
+ (void) (&__a == &__b); \
+ __a > __b ? (__a - __b) : (__b - __a); })
+
+struct drm_i915_private;
+struct intel_crtc;
+struct intel_crtc_state;
+struct intel_encoder;
+
+struct intel_shared_dpll;
+struct intel_dpll_mgr;
+
+enum intel_dpll_id {
+ DPLL_ID_PRIVATE = -1, /* non-shared dpll in use */
+ /* real shared dpll ids must be >= 0 */
+ DPLL_ID_PCH_PLL_A = 0,
+ DPLL_ID_PCH_PLL_B = 1,
+ /* hsw/bdw */
+ DPLL_ID_WRPLL1 = 0,
+ DPLL_ID_WRPLL2 = 1,
+ DPLL_ID_SPLL = 2,
+ DPLL_ID_LCPLL_810 = 3,
+ DPLL_ID_LCPLL_1350 = 4,
+ DPLL_ID_LCPLL_2700 = 5,
+
+ /* skl */
+ DPLL_ID_SKL_DPLL0 = 0,
+ DPLL_ID_SKL_DPLL1 = 1,
+ DPLL_ID_SKL_DPLL2 = 2,
+ DPLL_ID_SKL_DPLL3 = 3,
+};
+#define I915_NUM_PLLS 6
+
+/** Inform the state checker that the DPLL is kept enabled even if not
+ * in use by any crtc.
+ */
+#define INTEL_DPLL_ALWAYS_ON (1 << 0)
+
+struct intel_dpll_hw_state {
+ /* i9xx, pch plls */
+ uint32_t dpll;
+ uint32_t dpll_md;
+ uint32_t fp0;
+ uint32_t fp1;
+
+ /* hsw, bdw */
+ uint32_t wrpll;
+ uint32_t spll;
+
+ /* skl */
+ /*
+ * DPLL_CTRL1 has 6 bits for each each this DPLL. We store those in
+ * lower part of ctrl1 and they get shifted into position when writing
+ * the register. This allows us to easily compare the state to share
+ * the DPLL.
+ */
+ uint32_t ctrl1;
+ /* HDMI only, 0 when used for DP */
+ uint32_t cfgcr1, cfgcr2;
+
+ /* bxt */
+ uint32_t ebb0, ebb4, pll0, pll1, pll2, pll3, pll6, pll8, pll9, pll10,
+ pcsdw12;
+};
+
+struct intel_shared_dpll_config {
+ unsigned crtc_mask; /* mask of CRTCs sharing this PLL */
+ struct intel_dpll_hw_state hw_state;
+};
+
+struct intel_shared_dpll_funcs {
+ /* The mode_set hook is optional and should be used together with the
+ * intel_prepare_shared_dpll function. */
+ void (*mode_set)(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll);
+ void (*enable)(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll);
+ void (*disable)(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll);
+ bool (*get_hw_state)(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll,
+ struct intel_dpll_hw_state *hw_state);
+};
+
+struct intel_shared_dpll {
+ struct intel_shared_dpll_config config;
+
+ unsigned active_mask; /* mask of active CRTCs (i.e. DPMS on) */
+ bool on; /* is the PLL actually active? Disabled during modeset */
+ const char *name;
+ /* should match the index in the dev_priv->shared_dplls array */
+ enum intel_dpll_id id;
+
+ struct intel_shared_dpll_funcs funcs;
+
+ uint32_t flags;
+};
+
+#define SKL_DPLL0 0
+#define SKL_DPLL1 1
+#define SKL_DPLL2 2
+#define SKL_DPLL3 3
+
+/* shared dpll functions */
+struct intel_shared_dpll *
+intel_get_shared_dpll_by_id(struct drm_i915_private *dev_priv,
+ enum intel_dpll_id id);
+enum intel_dpll_id
+intel_get_shared_dpll_id(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll);
+void
+intel_shared_dpll_config_get(struct intel_shared_dpll_config *config,
+ struct intel_shared_dpll *pll,
+ struct intel_crtc *crtc);
+void
+intel_shared_dpll_config_put(struct intel_shared_dpll_config *config,
+ struct intel_shared_dpll *pll,
+ struct intel_crtc *crtc);
+void assert_shared_dpll(struct drm_i915_private *dev_priv,
+ struct intel_shared_dpll *pll,
+ bool state);
+#define assert_shared_dpll_enabled(d, p) assert_shared_dpll(d, p, true)
+#define assert_shared_dpll_disabled(d, p) assert_shared_dpll(d, p, false)
+struct intel_shared_dpll *intel_get_shared_dpll(struct intel_crtc *crtc,
+ struct intel_crtc_state *state,
+ struct intel_encoder *encoder);
+void intel_prepare_shared_dpll(struct intel_crtc *crtc);
+void intel_enable_shared_dpll(struct intel_crtc *crtc);
+void intel_disable_shared_dpll(struct intel_crtc *crtc);
+void intel_shared_dpll_commit(struct drm_atomic_state *state);
+void intel_shared_dpll_init(struct drm_device *dev);
+
+
+#endif /* _INTEL_DPLL_MGR_H_ */
diff --git a/drivers/gpu/drm/i915/intel_drv.h b/drivers/gpu/drm/i915/intel_drv.h
index 9d0770c..5da29a0 100644
--- a/drivers/gpu/drm/i915/intel_drv.h
+++ b/drivers/gpu/drm/i915/intel_drv.h
@@ -44,9 +44,13 @@
* contexts. Note that it's important that we check the condition again after
* having timed out, since the timeout could be due to preemption or similar and
* we've never had a chance to check the condition before the timeout.
+ *
+ * TODO: When modesetting has fully transitioned to atomic, the below
+ * drm_can_sleep() can be removed and in_atomic()/!in_atomic() asserts
+ * added.
*/
-#define _wait_for(COND, MS, W) ({ \
- unsigned long timeout__ = jiffies + msecs_to_jiffies(MS) + 1; \
+#define _wait_for(COND, US, W) ({ \
+ unsigned long timeout__ = jiffies + usecs_to_jiffies(US) + 1; \
int ret__ = 0; \
while (!(COND)) { \
if (time_after(jiffies, timeout__)) { \
@@ -55,7 +59,7 @@
break; \
} \
if ((W) && drm_can_sleep()) { \
- usleep_range((W)*1000, (W)*2000); \
+ usleep_range((W), (W)*2); \
} else { \
cpu_relax(); \
} \
@@ -63,10 +67,40 @@
ret__; \
})
-#define wait_for(COND, MS) _wait_for(COND, MS, 1)
-#define wait_for_atomic(COND, MS) _wait_for(COND, MS, 0)
-#define wait_for_atomic_us(COND, US) _wait_for((COND), \
- DIV_ROUND_UP((US), 1000), 0)
+#define wait_for(COND, MS) _wait_for((COND), (MS) * 1000, 1000)
+#define wait_for_us(COND, US) _wait_for((COND), (US), 1)
+
+/* If CONFIG_PREEMPT_COUNT is disabled, in_atomic() always reports false. */
+#if defined(CONFIG_DRM_I915_DEBUG) && defined(CONFIG_PREEMPT_COUNT)
+# define _WAIT_FOR_ATOMIC_CHECK WARN_ON_ONCE(!in_atomic())
+#else
+# define _WAIT_FOR_ATOMIC_CHECK do { } while (0)
+#endif
+
+#define _wait_for_atomic(COND, US) ({ \
+ unsigned long end__; \
+ int ret__ = 0; \
+ _WAIT_FOR_ATOMIC_CHECK; \
+ BUILD_BUG_ON((US) > 50000); \
+ end__ = (local_clock() >> 10) + (US) + 1; \
+ while (!(COND)) { \
+ if (time_after((unsigned long)(local_clock() >> 10), end__)) { \
+ /* Unlike the regular wait_for(), this atomic variant \
+ * cannot be preempted (and we'll just ignore the issue\
+ * of irq interruptions) and so we know that no time \
+ * has passed since the last check of COND and can \
+ * immediately report the timeout. \
+ */ \
+ ret__ = -ETIMEDOUT; \
+ break; \
+ } \
+ cpu_relax(); \
+ } \
+ ret__; \
+})
+
+#define wait_for_atomic(COND, MS) _wait_for_atomic((COND), (MS) * 1000)
+#define wait_for_atomic_us(COND, US) _wait_for_atomic((COND), (US))
#define KHz(x) (1000 * (x))
#define MHz(x) KHz(1000 * (x))
@@ -118,6 +152,7 @@
struct intel_framebuffer {
struct drm_framebuffer base;
struct drm_i915_gem_object *obj;
+ struct intel_rotation_info rot_info;
};
struct intel_fbdev {
@@ -260,6 +295,12 @@
struct intel_shared_dpll_config shared_dpll[I915_NUM_PLLS];
struct intel_wm_config wm_config;
+
+ /*
+ * Current watermarks can't be trusted during hardware readout, so
+ * don't bother calculating intermediate watermarks.
+ */
+ bool skip_intermediate_wm;
};
struct intel_plane_state {
@@ -349,6 +390,7 @@
struct intel_pipe_wm {
struct intel_wm_level wm[5];
+ struct intel_wm_level raw_wm[5];
uint32_t linetime;
bool fbc_wm_enabled;
bool pipe_enabled;
@@ -376,9 +418,10 @@
#define PIPE_CONFIG_QUIRK_MODE_SYNC_FLAGS (1<<0) /* unreliable sync mode.flags */
unsigned long quirks;
+ unsigned fb_bits; /* framebuffers to flip */
bool update_pipe; /* can a fast modeset be performed? */
bool disable_cxsr;
- bool wm_changed; /* watermarks are updated */
+ bool update_wm_pre, update_wm_post; /* watermarks are updated */
bool fb_changed; /* fb on any of the planes is changed */
/* Pipe source size (ie. panel fitter input size)
@@ -394,7 +437,8 @@
bool has_infoframe;
/* CPU Transcoder for the pipe. Currently this can only differ from the
- * pipe on Haswell (where we have a special eDP transcoder). */
+ * pipe on Haswell and later (where we have a special eDP transcoder)
+ * and Broxton (where we have special DSI transcoders). */
enum transcoder cpu_transcoder;
/*
@@ -441,8 +485,8 @@
* haswell. */
struct dpll dpll;
- /* Selected dpll when shared or DPLL_ID_PRIVATE. */
- enum intel_dpll_id shared_dpll;
+ /* Selected dpll when shared or NULL. */
+ struct intel_shared_dpll *shared_dpll;
/*
* - PORT_CLK_SEL for DDI ports on HSW/BDW.
@@ -453,6 +497,11 @@
/* Actual register state of the dpll, for shared dpll cross-checking. */
struct intel_dpll_hw_state dpll_hw_state;
+ /* DSI PLL registers */
+ struct {
+ u32 ctrl, div;
+ } dsi_pll;
+
int pipe_bpp;
struct intel_link_m_n dp_m_n;
@@ -510,14 +559,33 @@
struct {
/*
- * optimal watermarks, programmed post-vblank when this state
- * is committed
+ * Optimal watermarks, programmed post-vblank when this state
+ * is committed.
*/
union {
struct intel_pipe_wm ilk;
struct skl_pipe_wm skl;
} optimal;
+
+ /*
+ * Intermediate watermarks; these can be programmed immediately
+ * since they satisfy both the current configuration we're
+ * switching away from and the new configuration we're switching
+ * to.
+ */
+ struct intel_pipe_wm intermediate;
+
+ /*
+ * Platforms with two-step watermark programming will need to
+ * update watermark programming post-vblank to switch from the
+ * safe intermediate watermarks to the optimal final
+ * watermarks.
+ */
+ bool need_postvbl_update;
} wm;
+
+ /* Gamma mode programmed on the pipe */
+ uint32_t gamma_mode;
};
struct vlv_wm_state {
@@ -537,23 +605,6 @@
unsigned int rotation;
};
-/*
- * Tracking of operations that need to be performed at the beginning/end of an
- * atomic commit, outside the atomic section where interrupts are disabled.
- * These are generally operations that grab mutexes or might otherwise sleep
- * and thus can't be run with interrupts disabled.
- */
-struct intel_crtc_atomic_commit {
- /* Sleepable operations to perform before commit */
-
- /* Sleepable operations to perform after commit */
- unsigned fb_bits;
- bool post_enable_primary;
-
- /* Sleepable operations to perform before and after commit */
- bool update_fbc;
-};
-
struct intel_crtc {
struct drm_crtc base;
enum pipe pipe;
@@ -600,6 +651,7 @@
struct intel_pipe_wm ilk;
struct skl_pipe_wm skl;
} active;
+
/* allow CxSR on this pipe */
bool cxsr_allowed;
} wm;
@@ -613,8 +665,6 @@
int scanline_start;
} debug;
- struct intel_crtc_atomic_commit atomic;
-
/* scalers available on this crtc */
int num_scalers;
@@ -751,7 +801,9 @@
uint32_t DP;
int link_rate;
uint8_t lane_count;
+ uint8_t sink_count;
bool has_audio;
+ bool detect_done;
enum hdmi_force_audio force_audio;
bool limited_color_range;
bool color_range_auto;
@@ -831,7 +883,7 @@
struct intel_encoder base;
enum pipe pipe;
struct intel_digital_port *primary;
- void *port; /* store this opaque as its illegal to dereference it */
+ struct intel_connector *connector;
};
static inline enum dpio_channel
@@ -1007,7 +1059,6 @@
void intel_ddi_init(struct drm_device *dev, enum port port);
enum port intel_ddi_get_encoder_port(struct intel_encoder *intel_encoder);
bool intel_ddi_get_hw_state(struct intel_encoder *encoder, enum pipe *pipe);
-void intel_ddi_pll_init(struct drm_device *dev);
void intel_ddi_enable_transcoder_func(struct drm_crtc *crtc);
void intel_ddi_disable_transcoder_func(struct drm_i915_private *dev_priv,
enum transcoder cpu_transcoder);
@@ -1049,17 +1100,19 @@
uint64_t fb_modifier, uint32_t pixel_format);
/* intel_audio.c */
-void intel_init_audio(struct drm_device *dev);
+void intel_init_audio_hooks(struct drm_i915_private *dev_priv);
void intel_audio_codec_enable(struct intel_encoder *encoder);
void intel_audio_codec_disable(struct intel_encoder *encoder);
void i915_audio_component_init(struct drm_i915_private *dev_priv);
void i915_audio_component_cleanup(struct drm_i915_private *dev_priv);
/* intel_display.c */
+int vlv_get_cck_clock(struct drm_i915_private *dev_priv,
+ const char *name, u32 reg, int ref_freq);
extern const struct drm_plane_funcs intel_plane_funcs;
+void intel_init_display_hooks(struct drm_i915_private *dev_priv);
+unsigned int intel_rotation_info_size(const struct intel_rotation_info *rot_info);
bool intel_has_pending_fb_unpin(struct drm_device *dev);
-int intel_pch_rawclk(struct drm_device *dev);
-int intel_hrawclk(struct drm_device *dev);
void intel_mark_busy(struct drm_device *dev);
void intel_mark_idle(struct drm_device *dev);
void intel_crtc_restore_mode(struct drm_crtc *crtc);
@@ -1104,9 +1157,8 @@
void intel_release_load_detect_pipe(struct drm_connector *connector,
struct intel_load_detect_pipe *old,
struct drm_modeset_acquire_ctx *ctx);
-int intel_pin_and_fence_fb_obj(struct drm_plane *plane,
- struct drm_framebuffer *fb,
- const struct drm_plane_state *plane_state);
+int intel_pin_and_fence_fb_obj(struct drm_framebuffer *fb,
+ unsigned int rotation);
struct drm_framebuffer *
__intel_framebuffer_create(struct drm_device *dev,
struct drm_mode_fb_cmd2 *mode_cmd,
@@ -1142,19 +1194,13 @@
void intel_create_rotation_property(struct drm_device *dev,
struct intel_plane *plane);
-/* shared dpll functions */
-struct intel_shared_dpll *intel_crtc_to_shared_dpll(struct intel_crtc *crtc);
-void assert_shared_dpll(struct drm_i915_private *dev_priv,
- struct intel_shared_dpll *pll,
- bool state);
-#define assert_shared_dpll_enabled(d, p) assert_shared_dpll(d, p, true)
-#define assert_shared_dpll_disabled(d, p) assert_shared_dpll(d, p, false)
-struct intel_shared_dpll *intel_get_shared_dpll(struct intel_crtc *crtc,
- struct intel_crtc_state *state);
+void assert_pch_transcoder_disabled(struct drm_i915_private *dev_priv,
+ enum pipe pipe);
int vlv_force_pll_on(struct drm_device *dev, enum pipe pipe,
const struct dpll *dpll);
void vlv_force_pll_off(struct drm_device *dev, enum pipe pipe);
+int lpt_get_iclkip(struct drm_i915_private *dev_priv);
/* modesetting asserts */
void assert_panel_unlocked(struct drm_i915_private *dev_priv,
@@ -1163,6 +1209,9 @@
enum pipe pipe, bool state);
#define assert_pll_enabled(d, p) assert_pll(d, p, true)
#define assert_pll_disabled(d, p) assert_pll(d, p, false)
+void assert_dsi_pll(struct drm_i915_private *dev_priv, bool state);
+#define assert_dsi_pll_enabled(d) assert_dsi_pll(d, true)
+#define assert_dsi_pll_disabled(d) assert_dsi_pll(d, false)
void assert_fdi_rx_pll(struct drm_i915_private *dev_priv,
enum pipe pipe, bool state);
#define assert_fdi_rx_pll_enabled(d, p) assert_fdi_rx_pll(d, p, true)
@@ -1170,21 +1219,24 @@
void assert_pipe(struct drm_i915_private *dev_priv, enum pipe pipe, bool state);
#define assert_pipe_enabled(d, p) assert_pipe(d, p, true)
#define assert_pipe_disabled(d, p) assert_pipe(d, p, false)
-u32 intel_compute_tile_offset(struct drm_i915_private *dev_priv,
- int *x, int *y,
- uint64_t fb_modifier,
- unsigned int cpp,
- unsigned int pitch);
+u32 intel_compute_tile_offset(int *x, int *y,
+ const struct drm_framebuffer *fb, int plane,
+ unsigned int pitch,
+ unsigned int rotation);
void intel_prepare_reset(struct drm_device *dev);
void intel_finish_reset(struct drm_device *dev);
void hsw_enable_pc8(struct drm_i915_private *dev_priv);
void hsw_disable_pc8(struct drm_i915_private *dev_priv);
-void broxton_init_cdclk(struct drm_device *dev);
-void broxton_uninit_cdclk(struct drm_device *dev);
-void broxton_ddi_phy_init(struct drm_device *dev);
-void broxton_ddi_phy_uninit(struct drm_device *dev);
+void broxton_init_cdclk(struct drm_i915_private *dev_priv);
+void broxton_uninit_cdclk(struct drm_i915_private *dev_priv);
+bool broxton_cdclk_verify_state(struct drm_i915_private *dev_priv);
+void broxton_ddi_phy_init(struct drm_i915_private *dev_priv);
+void broxton_ddi_phy_uninit(struct drm_i915_private *dev_priv);
+void broxton_ddi_phy_verify_state(struct drm_i915_private *dev_priv);
+void gen9_sanitize_dc_state(struct drm_i915_private *dev_priv);
void bxt_enable_dc9(struct drm_i915_private *dev_priv);
void bxt_disable_dc9(struct drm_i915_private *dev_priv);
+void gen9_enable_dc5(struct drm_i915_private *dev_priv);
void skl_init_cdclk(struct drm_i915_private *dev_priv);
int skl_sanitize_cdclk(struct drm_i915_private *dev_priv);
void skl_uninit_cdclk(struct drm_i915_private *dev_priv);
@@ -1194,9 +1246,6 @@
struct intel_crtc_state *pipe_config);
void intel_dp_set_m_n(struct intel_crtc *crtc, enum link_m_n_set m_n);
int intel_dotclock_calculate(int link_freq, const struct intel_link_m_n *m_n);
-void
-ironlake_check_encoder_dotclock(const struct intel_crtc_state *pipe_config,
- int dotclock);
bool bxt_find_best_dpll(struct intel_crtc_state *crtc_state, int target_clock,
intel_clock_t *best_clock);
int chv_calc_dpll_params(int refclk, intel_clock_t *pll_clock);
@@ -1224,8 +1273,10 @@
/* intel_csr.c */
void intel_csr_ucode_init(struct drm_i915_private *);
-bool intel_csr_load_program(struct drm_i915_private *);
+void intel_csr_load_program(struct drm_i915_private *);
void intel_csr_ucode_fini(struct drm_i915_private *);
+void intel_csr_ucode_suspend(struct drm_i915_private *);
+void intel_csr_ucode_resume(struct drm_i915_private *);
/* intel_dp.c */
void intel_dp_init(struct drm_device *dev, i915_reg_t output_reg, enum port port);
@@ -1266,7 +1317,6 @@
void intel_edp_drrs_flush(struct drm_device *dev, unsigned frontbuffer_bits);
bool intel_digital_port_connected(struct drm_i915_private *dev_priv,
struct intel_digital_port *port);
-void hsw_dp_set_ddi_pll_sel(struct intel_crtc_state *pipe_config);
void
intel_dp_program_link_training_pattern(struct intel_dp *intel_dp,
@@ -1423,8 +1473,8 @@
void intel_power_domains_fini(struct drm_i915_private *);
void intel_power_domains_init_hw(struct drm_i915_private *dev_priv, bool resume);
void intel_power_domains_suspend(struct drm_i915_private *dev_priv);
-void skl_pw1_misc_io_init(struct drm_i915_private *dev_priv);
-void skl_pw1_misc_io_fini(struct drm_i915_private *dev_priv);
+void bxt_display_core_init(struct drm_i915_private *dev_priv, bool resume);
+void bxt_display_core_uninit(struct drm_i915_private *dev_priv);
void intel_runtime_pm_enable(struct drm_i915_private *dev_priv);
const char *
intel_display_power_domain_str(enum intel_display_power_domain domain);
@@ -1541,6 +1591,7 @@
int ilk_wm_max_level(const struct drm_device *dev);
void intel_update_watermarks(struct drm_crtc *crtc);
void intel_init_pm(struct drm_device *dev);
+void intel_init_clock_gating_hooks(struct drm_i915_private *dev_priv);
void intel_pm_setup(struct drm_device *dev);
void intel_gpu_ips_init(struct drm_i915_private *dev_priv);
void intel_gpu_ips_teardown(void);
@@ -1565,6 +1616,7 @@
void skl_ddb_get_hw_state(struct drm_i915_private *dev_priv,
struct skl_ddb_allocation *ddb /* out */);
uint32_t ilk_pipe_pixel_rate(const struct intel_crtc_state *pipe_config);
+bool ilk_disable_lp_wm(struct drm_device *dev);
int sanitize_rc6_option(const struct drm_device *dev, int enable_rc6);
/* intel_sdvo.c */
@@ -1606,6 +1658,18 @@
return to_intel_crtc_state(crtc_state);
}
+
+static inline struct intel_plane_state *
+intel_atomic_get_existing_plane_state(struct drm_atomic_state *state,
+ struct intel_plane *plane)
+{
+ struct drm_plane_state *plane_state;
+
+ plane_state = drm_atomic_get_existing_plane_state(state, &plane->base);
+
+ return to_intel_plane_state(plane_state);
+}
+
int intel_atomic_setup_scalers(struct drm_device *dev,
struct intel_crtc *intel_crtc,
struct intel_crtc_state *crtc_state);
@@ -1617,4 +1681,10 @@
struct drm_plane_state *state);
extern const struct drm_plane_helper_funcs intel_plane_helper_funcs;
+/* intel_color.c */
+void intel_color_init(struct drm_crtc *crtc);
+int intel_color_check(struct drm_crtc *crtc, struct drm_crtc_state *state);
+void intel_color_set_csc(struct drm_crtc_state *crtc_state);
+void intel_color_load_luts(struct drm_crtc_state *crtc_state);
+
#endif /* __INTEL_DRV_H__ */
diff --git a/drivers/gpu/drm/i915/intel_dsi.c b/drivers/gpu/drm/i915/intel_dsi.c
index 01b8e9f..2b22bb9 100644
--- a/drivers/gpu/drm/i915/intel_dsi.c
+++ b/drivers/gpu/drm/i915/intel_dsi.c
@@ -46,6 +46,24 @@
},
};
+enum mipi_dsi_pixel_format pixel_format_from_register_bits(u32 fmt)
+{
+ /* It just so happens the VBT matches register contents. */
+ switch (fmt) {
+ case VID_MODE_FORMAT_RGB888:
+ return MIPI_DSI_FMT_RGB888;
+ case VID_MODE_FORMAT_RGB666:
+ return MIPI_DSI_FMT_RGB666;
+ case VID_MODE_FORMAT_RGB666_PACKED:
+ return MIPI_DSI_FMT_RGB666_PACKED;
+ case VID_MODE_FORMAT_RGB565:
+ return MIPI_DSI_FMT_RGB565;
+ default:
+ MISSING_CASE(fmt);
+ return MIPI_DSI_FMT_RGB666;
+ }
+}
+
static void wait_for_dsi_fifo_empty(struct intel_dsi *intel_dsi, enum port port)
{
struct drm_encoder *encoder = &intel_dsi->base.base;
@@ -268,22 +286,47 @@
static bool intel_dsi_compute_config(struct intel_encoder *encoder,
struct intel_crtc_state *pipe_config)
{
+ struct drm_i915_private *dev_priv = encoder->base.dev->dev_private;
struct intel_dsi *intel_dsi = container_of(encoder, struct intel_dsi,
base);
struct intel_connector *intel_connector = intel_dsi->attached_connector;
- struct drm_display_mode *fixed_mode = intel_connector->panel.fixed_mode;
+ struct intel_crtc *crtc = to_intel_crtc(pipe_config->base.crtc);
+ const struct drm_display_mode *fixed_mode = intel_connector->panel.fixed_mode;
struct drm_display_mode *adjusted_mode = &pipe_config->base.adjusted_mode;
+ int ret;
DRM_DEBUG_KMS("\n");
pipe_config->has_dsi_encoder = true;
- if (fixed_mode)
+ if (fixed_mode) {
intel_fixed_panel_mode(fixed_mode, adjusted_mode);
+ if (HAS_GMCH_DISPLAY(dev_priv))
+ intel_gmch_panel_fitting(crtc, pipe_config,
+ intel_connector->panel.fitting_mode);
+ else
+ intel_pch_panel_fitting(crtc, pipe_config,
+ intel_connector->panel.fitting_mode);
+ }
+
/* DSI uses short packets for sync events, so clear mode flags for DSI */
adjusted_mode->flags = 0;
+ if (IS_BROXTON(dev_priv)) {
+ /* Dual link goes to DSI transcoder A. */
+ if (intel_dsi->ports == BIT(PORT_C))
+ pipe_config->cpu_transcoder = TRANSCODER_DSI_C;
+ else
+ pipe_config->cpu_transcoder = TRANSCODER_DSI_A;
+ }
+
+ ret = intel_compute_dsi_pll(encoder, pipe_config);
+ if (ret)
+ return false;
+
+ pipe_config->clock_set = true;
+
return true;
}
@@ -403,7 +446,7 @@
temp &= ~LANE_CONFIGURATION_MASK;
temp &= ~DUAL_LINK_MODE_MASK;
- if (intel_dsi->ports == ((1 << PORT_A) | (1 << PORT_C))) {
+ if (intel_dsi->ports == (BIT(PORT_A) | BIT(PORT_C))) {
temp |= (intel_dsi->dual_link - 1)
<< DUAL_LINK_MODE_SHIFT;
temp |= intel_crtc->pipe ?
@@ -471,14 +514,19 @@
struct drm_device *dev = encoder->base.dev;
struct drm_i915_private *dev_priv = dev->dev_private;
struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
- struct intel_crtc *intel_crtc = to_intel_crtc(encoder->base.crtc);
- enum pipe pipe = intel_crtc->pipe;
+ struct intel_crtc *crtc = to_intel_crtc(encoder->base.crtc);
enum port port;
u32 tmp;
DRM_DEBUG_KMS("\n");
- intel_enable_dsi_pll(encoder);
+ /*
+ * The BIOS may leave the PLL in a wonky state where it doesn't
+ * lock. It needs to be fully powered down to fix it.
+ */
+ intel_disable_dsi_pll(encoder);
+ intel_enable_dsi_pll(encoder, crtc->config);
+
intel_dsi_prepare(encoder);
/* Panel Enable over CRC PMIC */
@@ -488,19 +536,7 @@
msleep(intel_dsi->panel_on_delay);
if (IS_VALLEYVIEW(dev) || IS_CHERRYVIEW(dev)) {
- /*
- * Disable DPOunit clock gating, can stall pipe
- * and we need DPLL REFA always enabled
- */
- tmp = I915_READ(DPLL(pipe));
- tmp |= DPLL_REF_CLK_ENABLE_VLV;
- I915_WRITE(DPLL(pipe), tmp);
-
- /* update the hw state for DPLL */
- intel_crtc->config->dpll_hw_state.dpll =
- DPLL_INTEGRATED_REF_CLK_VLV |
- DPLL_REF_CLK_ENABLE_VLV | DPLL_VGA_MODE_DIS;
-
+ /* Disable DPOunit clock gating, can stall pipe */
tmp = I915_READ(DSPCLK_GATE_D);
tmp |= DPOUNIT_CLOCK_GATE_DISABLE;
I915_WRITE(DSPCLK_GATE_D, tmp);
@@ -652,11 +688,16 @@
drm_panel_unprepare(intel_dsi->panel);
msleep(intel_dsi->panel_off_delay);
- msleep(intel_dsi->panel_pwr_cycle_delay);
/* Panel Disable over CRC PMIC */
if (intel_dsi->gpio_panel)
gpiod_set_value_cansleep(intel_dsi->gpio_panel, 0);
+
+ /*
+ * FIXME As we do with eDP, just make a note of the time here
+ * and perform the wait before the next panel power on.
+ */
+ msleep(intel_dsi->panel_pwr_cycle_delay);
}
static bool intel_dsi_get_hw_state(struct intel_encoder *encoder,
@@ -667,7 +708,7 @@
struct drm_device *dev = encoder->base.dev;
enum intel_display_power_domain power_domain;
enum port port;
- bool ret;
+ bool active = false;
DRM_DEBUG_KMS("\n");
@@ -675,55 +716,133 @@
if (!intel_display_power_get_if_enabled(dev_priv, power_domain))
return false;
- ret = false;
+ /*
+ * On Broxton the PLL needs to be enabled with a valid divider
+ * configuration, otherwise accessing DSI registers will hang the
+ * machine. See BSpec North Display Engine registers/MIPI[BXT].
+ */
+ if (IS_BROXTON(dev_priv) && !intel_dsi_pll_is_enabled(dev_priv))
+ goto out_put_power;
/* XXX: this only works for one DSI output */
for_each_dsi_port(port, intel_dsi->ports) {
i915_reg_t ctrl_reg = IS_BROXTON(dev) ?
BXT_MIPI_PORT_CTRL(port) : MIPI_PORT_CTRL(port);
- u32 dpi_enabled, func;
+ bool enabled = I915_READ(ctrl_reg) & DPI_ENABLE;
- func = I915_READ(MIPI_DSI_FUNC_PRG(port));
- dpi_enabled = I915_READ(ctrl_reg) & DPI_ENABLE;
-
- /* Due to some hardware limitations on BYT, MIPI Port C DPI
- * Enable bit does not get set. To check whether DSI Port C
- * was enabled in BIOS, check the Pipe B enable bit
+ /*
+ * Due to some hardware limitations on VLV/CHV, the DPI enable
+ * bit in port C control register does not get set. As a
+ * workaround, check pipe B conf instead.
*/
- if (IS_VALLEYVIEW(dev) && port == PORT_C)
- dpi_enabled = I915_READ(PIPECONF(PIPE_B)) &
- PIPECONF_ENABLE;
+ if ((IS_VALLEYVIEW(dev) || IS_CHERRYVIEW(dev)) && port == PORT_C)
+ enabled = I915_READ(PIPECONF(PIPE_B)) & PIPECONF_ENABLE;
- if (dpi_enabled || (func & CMD_MODE_DATA_WIDTH_MASK)) {
- if (I915_READ(MIPI_DEVICE_READY(port)) & DEVICE_READY) {
- *pipe = port == PORT_A ? PIPE_A : PIPE_B;
- ret = true;
-
- goto out;
- }
+ /* Try command mode if video mode not enabled */
+ if (!enabled) {
+ u32 tmp = I915_READ(MIPI_DSI_FUNC_PRG(port));
+ enabled = tmp & CMD_MODE_DATA_WIDTH_MASK;
}
+
+ if (!enabled)
+ continue;
+
+ if (!(I915_READ(MIPI_DEVICE_READY(port)) & DEVICE_READY))
+ continue;
+
+ if (IS_BROXTON(dev_priv)) {
+ u32 tmp = I915_READ(MIPI_CTRL(port));
+ tmp &= BXT_PIPE_SELECT_MASK;
+ tmp >>= BXT_PIPE_SELECT_SHIFT;
+
+ if (WARN_ON(tmp > PIPE_C))
+ continue;
+
+ *pipe = tmp;
+ } else {
+ *pipe = port == PORT_A ? PIPE_A : PIPE_B;
+ }
+
+ active = true;
+ break;
}
-out:
+
+out_put_power:
intel_display_power_put(dev_priv, power_domain);
- return ret;
+ return active;
}
+static void bxt_dsi_get_pipe_config(struct intel_encoder *encoder,
+ struct intel_crtc_state *pipe_config)
+{
+ struct drm_device *dev = encoder->base.dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_display_mode *adjusted_mode =
+ &pipe_config->base.adjusted_mode;
+ struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
+ unsigned int bpp, fmt;
+ enum port port;
+ u16 vfp, vsync, vbp;
+
+ /*
+ * Atleast one port is active as encoder->get_config called only if
+ * encoder->get_hw_state() returns true.
+ */
+ for_each_dsi_port(port, intel_dsi->ports) {
+ if (I915_READ(BXT_MIPI_PORT_CTRL(port)) & DPI_ENABLE)
+ break;
+ }
+
+ fmt = I915_READ(MIPI_DSI_FUNC_PRG(port)) & VID_MODE_FORMAT_MASK;
+ pipe_config->pipe_bpp =
+ mipi_dsi_pixel_format_to_bpp(
+ pixel_format_from_register_bits(fmt));
+ bpp = pipe_config->pipe_bpp;
+
+ /* In terms of pixels */
+ adjusted_mode->crtc_hdisplay =
+ I915_READ(BXT_MIPI_TRANS_HACTIVE(port));
+ adjusted_mode->crtc_vdisplay =
+ I915_READ(BXT_MIPI_TRANS_VACTIVE(port));
+ adjusted_mode->crtc_vtotal =
+ I915_READ(BXT_MIPI_TRANS_VTOTAL(port));
+
+ /*
+ * TODO: Retrieve hfp, hsync and hbp. Adjust them for dual link and
+ * calculate hsync_start, hsync_end, htotal and hblank_end
+ */
+
+ /* vertical values are in terms of lines */
+ vfp = I915_READ(MIPI_VFP_COUNT(port));
+ vsync = I915_READ(MIPI_VSYNC_PADDING_COUNT(port));
+ vbp = I915_READ(MIPI_VBP_COUNT(port));
+
+ adjusted_mode->crtc_hblank_start = adjusted_mode->crtc_hdisplay;
+
+ adjusted_mode->crtc_vsync_start =
+ vfp + adjusted_mode->crtc_vdisplay;
+ adjusted_mode->crtc_vsync_end =
+ vsync + adjusted_mode->crtc_vsync_start;
+ adjusted_mode->crtc_vblank_start = adjusted_mode->crtc_vdisplay;
+ adjusted_mode->crtc_vblank_end = adjusted_mode->crtc_vtotal;
+}
+
+
static void intel_dsi_get_config(struct intel_encoder *encoder,
struct intel_crtc_state *pipe_config)
{
+ struct drm_device *dev = encoder->base.dev;
u32 pclk;
DRM_DEBUG_KMS("\n");
pipe_config->has_dsi_encoder = true;
- /*
- * DPLL_MD is not used in case of DSI, reading will get some default value
- * set dpll_md = 0
- */
- pipe_config->dpll_hw_state.dpll_md = 0;
+ if (IS_BROXTON(dev))
+ bxt_dsi_get_pipe_config(encoder, pipe_config);
- pclk = intel_dsi_get_pclk(encoder, pipe_config->pipe_bpp);
+ pclk = intel_dsi_get_pclk(encoder, pipe_config->pipe_bpp,
+ pipe_config);
if (!pclk)
return;
@@ -736,7 +855,7 @@
struct drm_display_mode *mode)
{
struct intel_connector *intel_connector = to_intel_connector(connector);
- struct drm_display_mode *fixed_mode = intel_connector->panel.fixed_mode;
+ const struct drm_display_mode *fixed_mode = intel_connector->panel.fixed_mode;
int max_dotclk = to_i915(connector->dev)->max_dotclk_freq;
DRM_DEBUG_KMS("\n");
@@ -787,7 +906,7 @@
struct drm_i915_private *dev_priv = dev->dev_private;
struct intel_dsi *intel_dsi = enc_to_intel_dsi(encoder);
enum port port;
- unsigned int bpp = dsi_pixel_format_bpp(intel_dsi->pixel_format);
+ unsigned int bpp = mipi_dsi_pixel_format_to_bpp(intel_dsi->pixel_format);
unsigned int lane_count = intel_dsi->lane_count;
u16 hactive, hfp, hsync, hbp, vfp, vsync, vbp;
@@ -849,6 +968,23 @@
}
}
+static u32 pixel_format_to_reg(enum mipi_dsi_pixel_format fmt)
+{
+ switch (fmt) {
+ case MIPI_DSI_FMT_RGB888:
+ return VID_MODE_FORMAT_RGB888;
+ case MIPI_DSI_FMT_RGB666:
+ return VID_MODE_FORMAT_RGB666;
+ case MIPI_DSI_FMT_RGB666_PACKED:
+ return VID_MODE_FORMAT_RGB666_PACKED;
+ case MIPI_DSI_FMT_RGB565:
+ return VID_MODE_FORMAT_RGB565;
+ default:
+ MISSING_CASE(fmt);
+ return VID_MODE_FORMAT_RGB666;
+ }
+}
+
static void intel_dsi_prepare(struct intel_encoder *intel_encoder)
{
struct drm_encoder *encoder = &intel_encoder->base;
@@ -858,7 +994,7 @@
struct intel_dsi *intel_dsi = enc_to_intel_dsi(encoder);
const struct drm_display_mode *adjusted_mode = &intel_crtc->config->base.adjusted_mode;
enum port port;
- unsigned int bpp = dsi_pixel_format_bpp(intel_dsi->pixel_format);
+ unsigned int bpp = mipi_dsi_pixel_format_to_bpp(intel_dsi->pixel_format);
u32 val, tmp;
u16 mode_hdisplay;
@@ -917,9 +1053,7 @@
val |= CMD_MODE_DATA_WIDTH_8_BIT; /* XXX */
} else {
val |= intel_dsi->channel << VID_MODE_CHANNEL_NUMBER_SHIFT;
-
- /* XXX: cross-check bpp vs. pixel format? */
- val |= intel_dsi->pixel_format;
+ val |= pixel_format_to_reg(intel_dsi->pixel_format);
}
tmp = 0;
@@ -1059,6 +1193,48 @@
return 1;
}
+static int intel_dsi_set_property(struct drm_connector *connector,
+ struct drm_property *property,
+ uint64_t val)
+{
+ struct drm_device *dev = connector->dev;
+ struct intel_connector *intel_connector = to_intel_connector(connector);
+ struct drm_crtc *crtc;
+ int ret;
+
+ ret = drm_object_property_set_value(&connector->base, property, val);
+ if (ret)
+ return ret;
+
+ if (property == dev->mode_config.scaling_mode_property) {
+ if (val == DRM_MODE_SCALE_NONE) {
+ DRM_DEBUG_KMS("no scaling not supported\n");
+ return -EINVAL;
+ }
+ if (HAS_GMCH_DISPLAY(dev) &&
+ val == DRM_MODE_SCALE_CENTER) {
+ DRM_DEBUG_KMS("centering not supported\n");
+ return -EINVAL;
+ }
+
+ if (intel_connector->panel.fitting_mode == val)
+ return 0;
+
+ intel_connector->panel.fitting_mode = val;
+ }
+
+ crtc = intel_attached_encoder(connector)->base.crtc;
+ if (crtc && crtc->state->enable) {
+ /*
+ * If the CRTC is enabled, the display will be changed
+ * according to the new panel fitting mode.
+ */
+ intel_crtc_restore_mode(crtc);
+ }
+
+ return 0;
+}
+
static void intel_dsi_connector_destroy(struct drm_connector *connector)
{
struct intel_connector *intel_connector = to_intel_connector(connector);
@@ -1101,11 +1277,25 @@
.detect = intel_dsi_detect,
.destroy = intel_dsi_connector_destroy,
.fill_modes = drm_helper_probe_single_connector_modes,
+ .set_property = intel_dsi_set_property,
.atomic_get_property = intel_connector_atomic_get_property,
.atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
.atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
};
+static void intel_dsi_add_properties(struct intel_connector *connector)
+{
+ struct drm_device *dev = connector->base.dev;
+
+ if (connector->panel.fixed_mode) {
+ drm_mode_create_scaling_mode_property(dev);
+ drm_object_attach_property(&connector->base.base,
+ dev->mode_config.scaling_mode_property,
+ DRM_MODE_SCALE_ASPECT);
+ connector->panel.fitting_mode = DRM_MODE_SCALE_ASPECT;
+ }
+}
+
void intel_dsi_init(struct drm_device *dev)
{
struct intel_dsi *intel_dsi;
@@ -1121,11 +1311,13 @@
DRM_DEBUG_KMS("\n");
/* There is no detection method for MIPI so rely on VBT */
- if (!dev_priv->vbt.has_mipi)
+ if (!intel_bios_is_dsi_present(dev_priv, &port))
return;
if (IS_VALLEYVIEW(dev) || IS_CHERRYVIEW(dev)) {
dev_priv->mipi_mmio_base = VLV_MIPI_BASE;
+ } else if (IS_BROXTON(dev)) {
+ dev_priv->mipi_mmio_base = BXT_MIPI_BASE;
} else {
DRM_ERROR("Unsupported Mipi device to reg base");
return;
@@ -1161,17 +1353,21 @@
intel_connector->get_hw_state = intel_connector_get_hw_state;
intel_connector->unregister = intel_connector_unregister;
- /* Pipe A maps to MIPI DSI port A, pipe B maps to MIPI DSI port C */
- if (dev_priv->vbt.dsi.port == DVO_PORT_MIPIA) {
- intel_encoder->crtc_mask = (1 << PIPE_A);
- intel_dsi->ports = (1 << PORT_A);
- } else if (dev_priv->vbt.dsi.port == DVO_PORT_MIPIC) {
- intel_encoder->crtc_mask = (1 << PIPE_B);
- intel_dsi->ports = (1 << PORT_C);
- }
+ /*
+ * On BYT/CHV, pipe A maps to MIPI DSI port A, pipe B maps to MIPI DSI
+ * port C. BXT isn't limited like this.
+ */
+ if (IS_BROXTON(dev_priv))
+ intel_encoder->crtc_mask = BIT(PIPE_A) | BIT(PIPE_B) | BIT(PIPE_C);
+ else if (port == PORT_A)
+ intel_encoder->crtc_mask = BIT(PIPE_A);
+ else
+ intel_encoder->crtc_mask = BIT(PIPE_B);
if (dev_priv->vbt.dsi.config->dual_link)
- intel_dsi->ports = ((1 << PORT_A) | (1 << PORT_C));
+ intel_dsi->ports = BIT(PORT_A) | BIT(PORT_C);
+ else
+ intel_dsi->ports = BIT(port);
/* Create a DSI host (and a device) for each port. */
for_each_dsi_port(port, intel_dsi->ports) {
@@ -1223,8 +1419,6 @@
intel_connector_attach_encoder(intel_connector, intel_encoder);
- drm_connector_register(connector);
-
drm_panel_attach(intel_dsi->panel, connector);
mutex_lock(&dev->mode_config.mutex);
@@ -1243,6 +1437,11 @@
}
intel_panel_init(&intel_connector->panel, fixed_mode, NULL);
+
+ intel_dsi_add_properties(intel_connector);
+
+ drm_connector_register(connector);
+
intel_panel_setup_backlight(connector, INVALID_PIPE);
return;
diff --git a/drivers/gpu/drm/i915/intel_dsi.h b/drivers/gpu/drm/i915/intel_dsi.h
index 92f3922..61a6957 100644
--- a/drivers/gpu/drm/i915/intel_dsi.h
+++ b/drivers/gpu/drm/i915/intel_dsi.h
@@ -34,8 +34,6 @@
#define DSI_DUAL_LINK_FRONT_BACK 1
#define DSI_DUAL_LINK_PIXEL_ALT 2
-int dsi_pixel_format_bpp(int pixel_format);
-
struct intel_dsi_host;
struct intel_dsi {
@@ -64,8 +62,12 @@
/* number of DSI lanes */
unsigned int lane_count;
- /* video mode pixel format for MIPI_DSI_FUNC_PRG register */
- u32 pixel_format;
+ /*
+ * video mode pixel format
+ *
+ * XXX: consolidate on .format in struct mipi_dsi_device.
+ */
+ enum mipi_dsi_pixel_format pixel_format;
/* video mode format for MIPI_VIDEO_MODE_FORMAT register */
u32 video_mode_format;
@@ -117,21 +119,25 @@
return container_of(h, struct intel_dsi_host, base);
}
-#define for_each_dsi_port(__port, __ports_mask) \
- for ((__port) = PORT_A; (__port) < I915_MAX_PORTS; (__port)++) \
- for_each_if ((__ports_mask) & (1 << (__port)))
+#define for_each_dsi_port(__port, __ports_mask) for_each_port_masked(__port, __ports_mask)
static inline struct intel_dsi *enc_to_intel_dsi(struct drm_encoder *encoder)
{
return container_of(encoder, struct intel_dsi, base.base);
}
-extern void intel_enable_dsi_pll(struct intel_encoder *encoder);
-extern void intel_disable_dsi_pll(struct intel_encoder *encoder);
-extern u32 intel_dsi_get_pclk(struct intel_encoder *encoder, int pipe_bpp);
-extern void intel_dsi_reset_clocks(struct intel_encoder *encoder,
- enum port port);
+bool intel_dsi_pll_is_enabled(struct drm_i915_private *dev_priv);
+int intel_compute_dsi_pll(struct intel_encoder *encoder,
+ struct intel_crtc_state *config);
+void intel_enable_dsi_pll(struct intel_encoder *encoder,
+ const struct intel_crtc_state *config);
+void intel_disable_dsi_pll(struct intel_encoder *encoder);
+u32 intel_dsi_get_pclk(struct intel_encoder *encoder, int pipe_bpp,
+ struct intel_crtc_state *config);
+void intel_dsi_reset_clocks(struct intel_encoder *encoder,
+ enum port port);
struct drm_panel *vbt_panel_init(struct intel_dsi *intel_dsi, u16 panel_id);
+enum mipi_dsi_pixel_format pixel_format_from_register_bits(u32 fmt);
#endif /* _INTEL_DSI_H */
diff --git a/drivers/gpu/drm/i915/intel_dsi_panel_vbt.c b/drivers/gpu/drm/i915/intel_dsi_panel_vbt.c
index 7f145b4..e498f1c 100644
--- a/drivers/gpu/drm/i915/intel_dsi_panel_vbt.c
+++ b/drivers/gpu/drm/i915/intel_dsi_panel_vbt.c
@@ -58,50 +58,41 @@
#define NS_KHZ_RATIO 1000000
-#define GPI0_NC_0_HV_DDI0_HPD 0x4130
-#define GPIO_NC_0_HV_DDI0_PAD 0x4138
-#define GPIO_NC_1_HV_DDI0_DDC_SDA 0x4120
-#define GPIO_NC_1_HV_DDI0_DDC_SDA_PAD 0x4128
-#define GPIO_NC_2_HV_DDI0_DDC_SCL 0x4110
-#define GPIO_NC_2_HV_DDI0_DDC_SCL_PAD 0x4118
-#define GPIO_NC_3_PANEL0_VDDEN 0x4140
-#define GPIO_NC_3_PANEL0_VDDEN_PAD 0x4148
-#define GPIO_NC_4_PANEL0_BLKEN 0x4150
-#define GPIO_NC_4_PANEL0_BLKEN_PAD 0x4158
-#define GPIO_NC_5_PANEL0_BLKCTL 0x4160
-#define GPIO_NC_5_PANEL0_BLKCTL_PAD 0x4168
-#define GPIO_NC_6_PCONF0 0x4180
-#define GPIO_NC_6_PAD 0x4188
-#define GPIO_NC_7_PCONF0 0x4190
-#define GPIO_NC_7_PAD 0x4198
-#define GPIO_NC_8_PCONF0 0x4170
-#define GPIO_NC_8_PAD 0x4178
-#define GPIO_NC_9_PCONF0 0x4100
-#define GPIO_NC_9_PAD 0x4108
-#define GPIO_NC_10_PCONF0 0x40E0
-#define GPIO_NC_10_PAD 0x40E8
-#define GPIO_NC_11_PCONF0 0x40F0
-#define GPIO_NC_11_PAD 0x40F8
+/* base offsets for gpio pads */
+#define VLV_GPIO_NC_0_HV_DDI0_HPD 0x4130
+#define VLV_GPIO_NC_1_HV_DDI0_DDC_SDA 0x4120
+#define VLV_GPIO_NC_2_HV_DDI0_DDC_SCL 0x4110
+#define VLV_GPIO_NC_3_PANEL0_VDDEN 0x4140
+#define VLV_GPIO_NC_4_PANEL0_BKLTEN 0x4150
+#define VLV_GPIO_NC_5_PANEL0_BKLTCTL 0x4160
+#define VLV_GPIO_NC_6_HV_DDI1_HPD 0x4180
+#define VLV_GPIO_NC_7_HV_DDI1_DDC_SDA 0x4190
+#define VLV_GPIO_NC_8_HV_DDI1_DDC_SCL 0x4170
+#define VLV_GPIO_NC_9_PANEL1_VDDEN 0x4100
+#define VLV_GPIO_NC_10_PANEL1_BKLTEN 0x40E0
+#define VLV_GPIO_NC_11_PANEL1_BKLTCTL 0x40F0
-struct gpio_table {
- u16 function_reg;
- u16 pad_reg;
- u8 init;
+#define VLV_GPIO_PCONF0(base_offset) (base_offset)
+#define VLV_GPIO_PAD_VAL(base_offset) ((base_offset) + 8)
+
+struct gpio_map {
+ u16 base_offset;
+ bool init;
};
-static struct gpio_table gtable[] = {
- { GPI0_NC_0_HV_DDI0_HPD, GPIO_NC_0_HV_DDI0_PAD, 0 },
- { GPIO_NC_1_HV_DDI0_DDC_SDA, GPIO_NC_1_HV_DDI0_DDC_SDA_PAD, 0 },
- { GPIO_NC_2_HV_DDI0_DDC_SCL, GPIO_NC_2_HV_DDI0_DDC_SCL_PAD, 0 },
- { GPIO_NC_3_PANEL0_VDDEN, GPIO_NC_3_PANEL0_VDDEN_PAD, 0 },
- { GPIO_NC_4_PANEL0_BLKEN, GPIO_NC_4_PANEL0_BLKEN_PAD, 0 },
- { GPIO_NC_5_PANEL0_BLKCTL, GPIO_NC_5_PANEL0_BLKCTL_PAD, 0 },
- { GPIO_NC_6_PCONF0, GPIO_NC_6_PAD, 0 },
- { GPIO_NC_7_PCONF0, GPIO_NC_7_PAD, 0 },
- { GPIO_NC_8_PCONF0, GPIO_NC_8_PAD, 0 },
- { GPIO_NC_9_PCONF0, GPIO_NC_9_PAD, 0 },
- { GPIO_NC_10_PCONF0, GPIO_NC_10_PAD, 0},
- { GPIO_NC_11_PCONF0, GPIO_NC_11_PAD, 0}
+static struct gpio_map vlv_gpio_table[] = {
+ { VLV_GPIO_NC_0_HV_DDI0_HPD },
+ { VLV_GPIO_NC_1_HV_DDI0_DDC_SDA },
+ { VLV_GPIO_NC_2_HV_DDI0_DDC_SCL },
+ { VLV_GPIO_NC_3_PANEL0_VDDEN },
+ { VLV_GPIO_NC_4_PANEL0_BKLTEN },
+ { VLV_GPIO_NC_5_PANEL0_BKLTCTL },
+ { VLV_GPIO_NC_6_HV_DDI1_HPD },
+ { VLV_GPIO_NC_7_HV_DDI1_DDC_SDA },
+ { VLV_GPIO_NC_8_HV_DDI1_DDC_SCL },
+ { VLV_GPIO_NC_9_PANEL1_VDDEN },
+ { VLV_GPIO_NC_10_PANEL1_BKLTEN },
+ { VLV_GPIO_NC_11_PANEL1_BKLTCTL },
};
static inline enum port intel_dsi_seq_port_to_port(u8 port)
@@ -196,56 +187,76 @@
return data;
}
+static void vlv_exec_gpio(struct drm_i915_private *dev_priv,
+ u8 gpio_source, u8 gpio_index, bool value)
+{
+ struct gpio_map *map;
+ u16 pconf0, padval;
+ u32 tmp;
+ u8 port;
+
+ if (gpio_index >= ARRAY_SIZE(vlv_gpio_table)) {
+ DRM_DEBUG_KMS("unknown gpio index %u\n", gpio_index);
+ return;
+ }
+
+ map = &vlv_gpio_table[gpio_index];
+
+ if (dev_priv->vbt.dsi.seq_version >= 3) {
+ DRM_DEBUG_KMS("GPIO element v3 not supported\n");
+ return;
+ } else {
+ if (gpio_source == 0) {
+ port = IOSF_PORT_GPIO_NC;
+ } else if (gpio_source == 1) {
+ port = IOSF_PORT_GPIO_SC;
+ } else {
+ DRM_DEBUG_KMS("unknown gpio source %u\n", gpio_source);
+ return;
+ }
+ }
+
+ pconf0 = VLV_GPIO_PCONF0(map->base_offset);
+ padval = VLV_GPIO_PAD_VAL(map->base_offset);
+
+ mutex_lock(&dev_priv->sb_lock);
+ if (!map->init) {
+ /* FIXME: remove constant below */
+ vlv_iosf_sb_write(dev_priv, port, pconf0, 0x2000CC00);
+ map->init = true;
+ }
+
+ tmp = 0x4 | value;
+ vlv_iosf_sb_write(dev_priv, port, padval, tmp);
+ mutex_unlock(&dev_priv->sb_lock);
+}
+
static const u8 *mipi_exec_gpio(struct intel_dsi *intel_dsi, const u8 *data)
{
- u8 gpio, action;
- u16 function, pad;
- u32 val;
struct drm_device *dev = intel_dsi->base.base.dev;
struct drm_i915_private *dev_priv = dev->dev_private;
+ u8 gpio_source, gpio_index;
+ bool value;
if (dev_priv->vbt.dsi.seq_version >= 3)
data++;
- gpio = *data++;
+ gpio_index = *data++;
+
+ /* gpio source in sequence v2 only */
+ if (dev_priv->vbt.dsi.seq_version == 2)
+ gpio_source = (*data >> 1) & 3;
+ else
+ gpio_source = 0;
/* pull up/down */
- action = *data++ & 1;
+ value = *data++ & 1;
- if (gpio >= ARRAY_SIZE(gtable)) {
- DRM_DEBUG_KMS("unknown gpio %u\n", gpio);
- goto out;
- }
-
- if (!IS_VALLEYVIEW(dev_priv)) {
+ if (IS_VALLEYVIEW(dev_priv))
+ vlv_exec_gpio(dev_priv, gpio_source, gpio_index, value);
+ else
DRM_DEBUG_KMS("GPIO element not supported on this platform\n");
- goto out;
- }
- if (dev_priv->vbt.dsi.seq_version >= 3) {
- DRM_DEBUG_KMS("GPIO element v3 not supported\n");
- goto out;
- }
-
- function = gtable[gpio].function_reg;
- pad = gtable[gpio].pad_reg;
-
- mutex_lock(&dev_priv->sb_lock);
- if (!gtable[gpio].init) {
- /* program the function */
- /* FIXME: remove constant below */
- vlv_iosf_sb_write(dev_priv, IOSF_PORT_GPIO_NC, function,
- 0x2000CC00);
- gtable[gpio].init = 1;
- }
-
- val = 0x4 | action;
-
- /* pull up/down */
- vlv_iosf_sb_write(dev_priv, IOSF_PORT_GPIO_NC, pad, val);
- mutex_unlock(&dev_priv->sb_lock);
-
-out:
return data;
}
@@ -420,7 +431,7 @@
struct mipi_pps_data *pps = dev_priv->vbt.dsi.pps;
struct drm_display_mode *mode = dev_priv->vbt.lfp_lvds_vbt_mode;
struct vbt_panel *vbt_panel;
- u32 bits_per_pixel = 24;
+ u32 bpp;
u32 tlpx_ns, extra_byte_count, bitrate, tlpx_ui;
u32 ui_num, ui_den;
u32 prepare_cnt, exit_zero_cnt, clk_zero_cnt, trail_cnt;
@@ -436,12 +447,13 @@
intel_dsi->eotp_pkt = mipi_config->eot_pkt_disabled ? 0 : 1;
intel_dsi->clock_stop = mipi_config->enable_clk_stop ? 1 : 0;
intel_dsi->lane_count = mipi_config->lane_cnt + 1;
- intel_dsi->pixel_format = mipi_config->videomode_color_format << 7;
+ intel_dsi->pixel_format =
+ pixel_format_from_register_bits(
+ mipi_config->videomode_color_format << 7);
+ bpp = mipi_dsi_pixel_format_to_bpp(intel_dsi->pixel_format);
+
intel_dsi->dual_link = mipi_config->dual_link;
intel_dsi->pixel_overlap = mipi_config->pixel_overlap;
-
- bits_per_pixel = dsi_pixel_format_bpp(intel_dsi->pixel_format);
-
intel_dsi->operation_mode = mipi_config->is_cmd_mode;
intel_dsi->video_mode_format = mipi_config->video_transfer_mode;
intel_dsi->escape_clk_div = mipi_config->byte_clk_sel;
@@ -475,8 +487,7 @@
*/
if (intel_dsi->video_mode_format == VIDEO_MODE_BURST) {
if (mipi_config->target_burst_mode_freq) {
- computed_ddr =
- (pclk * bits_per_pixel) / intel_dsi->lane_count;
+ computed_ddr = (pclk * bpp) / intel_dsi->lane_count;
if (mipi_config->target_burst_mode_freq <
computed_ddr) {
@@ -499,7 +510,7 @@
intel_dsi->burst_mode_ratio = burst_mode_ratio;
intel_dsi->pclk = pclk;
- bitrate = (pclk * bits_per_pixel) / intel_dsi->lane_count;
+ bitrate = (pclk * bpp) / intel_dsi->lane_count;
switch (intel_dsi->escape_clk_div) {
case 0:
diff --git a/drivers/gpu/drm/i915/intel_dsi_pll.c b/drivers/gpu/drm/i915/intel_dsi_pll.c
index 70883c5..1765e6e 100644
--- a/drivers/gpu/drm/i915/intel_dsi_pll.c
+++ b/drivers/gpu/drm/i915/intel_dsi_pll.c
@@ -30,33 +30,7 @@
#include "i915_drv.h"
#include "intel_dsi.h"
-int dsi_pixel_format_bpp(int pixel_format)
-{
- int bpp;
-
- switch (pixel_format) {
- default:
- case VID_MODE_FORMAT_RGB888:
- case VID_MODE_FORMAT_RGB666_LOOSE:
- bpp = 24;
- break;
- case VID_MODE_FORMAT_RGB666:
- bpp = 18;
- break;
- case VID_MODE_FORMAT_RGB565:
- bpp = 16;
- break;
- }
-
- return bpp;
-}
-
-struct dsi_mnp {
- u32 dsi_pll_ctrl;
- u32 dsi_pll_div;
-};
-
-static const u32 lfsr_converts[] = {
+static const u16 lfsr_converts[] = {
426, 469, 234, 373, 442, 221, 110, 311, 411, /* 62 - 70 */
461, 486, 243, 377, 188, 350, 175, 343, 427, 213, /* 71 - 80 */
106, 53, 282, 397, 454, 227, 113, 56, 284, 142, /* 81 - 90 */
@@ -64,10 +38,11 @@
};
/* Get DSI clock from pixel clock */
-static u32 dsi_clk_from_pclk(u32 pclk, int pixel_format, int lane_count)
+static u32 dsi_clk_from_pclk(u32 pclk, enum mipi_dsi_pixel_format fmt,
+ int lane_count)
{
u32 dsi_clk_khz;
- u32 bpp = dsi_pixel_format_bpp(pixel_format);
+ u32 bpp = mipi_dsi_pixel_format_to_bpp(fmt);
/* DSI data rate = pixel clock * bits per pixel / lane count
pixel clock is converted from KHz to Hz */
@@ -77,7 +52,8 @@
}
static int dsi_calc_mnp(struct drm_i915_private *dev_priv,
- struct dsi_mnp *dsi_mnp, int target_dsi_clk)
+ struct intel_crtc_state *config,
+ int target_dsi_clk)
{
unsigned int calc_m = 0, calc_p = 0;
unsigned int m_min, m_max, p_min = 2, p_max = 6;
@@ -123,8 +99,8 @@
/* register has log2(N1), this works fine for powers of two */
n = ffs(n) - 1;
m_seed = lfsr_converts[calc_m - 62];
- dsi_mnp->dsi_pll_ctrl = 1 << (DSI_PLL_P1_POST_DIV_SHIFT + calc_p - 2);
- dsi_mnp->dsi_pll_div = n << DSI_PLL_N1_DIV_SHIFT |
+ config->dsi_pll.ctrl = 1 << (DSI_PLL_P1_POST_DIV_SHIFT + calc_p - 2);
+ config->dsi_pll.div = n << DSI_PLL_N1_DIV_SHIFT |
m_seed << DSI_PLL_M1_DIV_SHIFT;
return 0;
@@ -134,54 +110,55 @@
* XXX: The muxing and gating is hard coded for now. Need to add support for
* sharing PLLs with two DSI outputs.
*/
-static void vlv_configure_dsi_pll(struct intel_encoder *encoder)
+static int vlv_compute_dsi_pll(struct intel_encoder *encoder,
+ struct intel_crtc_state *config)
{
struct drm_i915_private *dev_priv = encoder->base.dev->dev_private;
struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
int ret;
- struct dsi_mnp dsi_mnp;
u32 dsi_clk;
dsi_clk = dsi_clk_from_pclk(intel_dsi->pclk, intel_dsi->pixel_format,
intel_dsi->lane_count);
- ret = dsi_calc_mnp(dev_priv, &dsi_mnp, dsi_clk);
+ ret = dsi_calc_mnp(dev_priv, config, dsi_clk);
if (ret) {
DRM_DEBUG_KMS("dsi_calc_mnp failed\n");
- return;
+ return ret;
}
if (intel_dsi->ports & (1 << PORT_A))
- dsi_mnp.dsi_pll_ctrl |= DSI_PLL_CLK_GATE_DSI0_DSIPLL;
+ config->dsi_pll.ctrl |= DSI_PLL_CLK_GATE_DSI0_DSIPLL;
if (intel_dsi->ports & (1 << PORT_C))
- dsi_mnp.dsi_pll_ctrl |= DSI_PLL_CLK_GATE_DSI1_DSIPLL;
+ config->dsi_pll.ctrl |= DSI_PLL_CLK_GATE_DSI1_DSIPLL;
+
+ config->dsi_pll.ctrl |= DSI_PLL_VCO_EN;
DRM_DEBUG_KMS("dsi pll div %08x, ctrl %08x\n",
- dsi_mnp.dsi_pll_div, dsi_mnp.dsi_pll_ctrl);
+ config->dsi_pll.div, config->dsi_pll.ctrl);
- vlv_cck_write(dev_priv, CCK_REG_DSI_PLL_CONTROL, 0);
- vlv_cck_write(dev_priv, CCK_REG_DSI_PLL_DIVIDER, dsi_mnp.dsi_pll_div);
- vlv_cck_write(dev_priv, CCK_REG_DSI_PLL_CONTROL, dsi_mnp.dsi_pll_ctrl);
+ return 0;
}
-static void vlv_enable_dsi_pll(struct intel_encoder *encoder)
+static void vlv_enable_dsi_pll(struct intel_encoder *encoder,
+ const struct intel_crtc_state *config)
{
- struct drm_i915_private *dev_priv = encoder->base.dev->dev_private;
- u32 tmp;
+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
DRM_DEBUG_KMS("\n");
mutex_lock(&dev_priv->sb_lock);
- vlv_configure_dsi_pll(encoder);
+ vlv_cck_write(dev_priv, CCK_REG_DSI_PLL_CONTROL, 0);
+ vlv_cck_write(dev_priv, CCK_REG_DSI_PLL_DIVIDER, config->dsi_pll.div);
+ vlv_cck_write(dev_priv, CCK_REG_DSI_PLL_CONTROL,
+ config->dsi_pll.ctrl & ~DSI_PLL_VCO_EN);
/* wait at least 0.5 us after ungating before enabling VCO */
usleep_range(1, 10);
- tmp = vlv_cck_read(dev_priv, CCK_REG_DSI_PLL_CONTROL);
- tmp |= DSI_PLL_VCO_EN;
- vlv_cck_write(dev_priv, CCK_REG_DSI_PLL_CONTROL, tmp);
+ vlv_cck_write(dev_priv, CCK_REG_DSI_PLL_CONTROL, config->dsi_pll.ctrl);
if (wait_for(vlv_cck_read(dev_priv, CCK_REG_DSI_PLL_CONTROL) &
DSI_PLL_LOCK, 20)) {
@@ -197,7 +174,7 @@
static void vlv_disable_dsi_pll(struct intel_encoder *encoder)
{
- struct drm_i915_private *dev_priv = encoder->base.dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
u32 tmp;
DRM_DEBUG_KMS("\n");
@@ -212,9 +189,39 @@
mutex_unlock(&dev_priv->sb_lock);
}
+static bool bxt_dsi_pll_is_enabled(struct drm_i915_private *dev_priv)
+{
+ bool enabled;
+ u32 val;
+ u32 mask;
+
+ mask = BXT_DSI_PLL_DO_ENABLE | BXT_DSI_PLL_LOCKED;
+ val = I915_READ(BXT_DSI_PLL_ENABLE);
+ enabled = (val & mask) == mask;
+
+ if (!enabled)
+ return false;
+
+ /*
+ * Both dividers must be programmed with valid values even if only one
+ * of the PLL is used, see BSpec/Broxton Clocks. Check this here for
+ * paranoia, since BIOS is known to misconfigure PLLs in this way at
+ * times, and since accessing DSI registers with invalid dividers
+ * causes a system hang.
+ */
+ val = I915_READ(BXT_DSI_PLL_CTL);
+ if (!(val & BXT_DSIA_16X_MASK) || !(val & BXT_DSIC_16X_MASK)) {
+ DRM_DEBUG_DRIVER("PLL is enabled with invalid divider settings (%08x)\n",
+ val);
+ enabled = false;
+ }
+
+ return enabled;
+}
+
static void bxt_disable_dsi_pll(struct intel_encoder *encoder)
{
- struct drm_i915_private *dev_priv = encoder->base.dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
u32 val;
DRM_DEBUG_KMS("\n");
@@ -232,23 +239,24 @@
DRM_ERROR("Timeout waiting for PLL lock deassertion\n");
}
-static void assert_bpp_mismatch(int pixel_format, int pipe_bpp)
+static void assert_bpp_mismatch(enum mipi_dsi_pixel_format fmt, int pipe_bpp)
{
- int bpp = dsi_pixel_format_bpp(pixel_format);
+ int bpp = mipi_dsi_pixel_format_to_bpp(fmt);
WARN(bpp != pipe_bpp,
"bpp match assertion failure (expected %d, current %d)\n",
bpp, pipe_bpp);
}
-static u32 vlv_dsi_get_pclk(struct intel_encoder *encoder, int pipe_bpp)
+static u32 vlv_dsi_get_pclk(struct intel_encoder *encoder, int pipe_bpp,
+ struct intel_crtc_state *config)
{
- struct drm_i915_private *dev_priv = encoder->base.dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
u32 dsi_clock, pclk;
u32 pll_ctl, pll_div;
u32 m = 0, p = 0, n;
- int refclk = 25000;
+ int refclk = IS_CHERRYVIEW(dev_priv) ? 100000 : 25000;
int i;
DRM_DEBUG_KMS("\n");
@@ -258,6 +266,9 @@
pll_div = vlv_cck_read(dev_priv, CCK_REG_DSI_PLL_DIVIDER);
mutex_unlock(&dev_priv->sb_lock);
+ config->dsi_pll.ctrl = pll_ctl & ~DSI_PLL_LOCK;
+ config->dsi_pll.div = pll_div;
+
/* mask out other bits and extract the P1 divisor */
pll_ctl &= DSI_PLL_P1_POST_DIV_MASK;
pll_ctl = pll_ctl >> (DSI_PLL_P1_POST_DIV_SHIFT - 2);
@@ -303,7 +314,8 @@
return pclk;
}
-static u32 bxt_dsi_get_pclk(struct intel_encoder *encoder, int pipe_bpp)
+static u32 bxt_dsi_get_pclk(struct intel_encoder *encoder, int pipe_bpp,
+ struct intel_crtc_state *config)
{
u32 pclk;
u32 dsi_clk;
@@ -317,15 +329,9 @@
return 0;
}
- dsi_ratio = I915_READ(BXT_DSI_PLL_CTL) &
- BXT_DSI_PLL_RATIO_MASK;
+ config->dsi_pll.ctrl = I915_READ(BXT_DSI_PLL_CTL);
- /* Invalid DSI ratio ? */
- if (dsi_ratio < BXT_DSI_PLL_RATIO_MIN ||
- dsi_ratio > BXT_DSI_PLL_RATIO_MAX) {
- DRM_ERROR("Invalid DSI pll ratio(%u) programmed\n", dsi_ratio);
- return 0;
- }
+ dsi_ratio = config->dsi_pll.ctrl & BXT_DSI_PLL_RATIO_MASK;
dsi_clk = (dsi_ratio * BXT_REF_CLOCK_KHZ) / 2;
@@ -338,12 +344,13 @@
return pclk;
}
-u32 intel_dsi_get_pclk(struct intel_encoder *encoder, int pipe_bpp)
+u32 intel_dsi_get_pclk(struct intel_encoder *encoder, int pipe_bpp,
+ struct intel_crtc_state *config)
{
if (IS_BROXTON(encoder->base.dev))
- return bxt_dsi_get_pclk(encoder, pipe_bpp);
+ return bxt_dsi_get_pclk(encoder, pipe_bpp, config);
else
- return vlv_dsi_get_pclk(encoder, pipe_bpp);
+ return vlv_dsi_get_pclk(encoder, pipe_bpp, config);
}
static void vlv_dsi_reset_clocks(struct intel_encoder *encoder, enum port port)
@@ -360,51 +367,72 @@
}
/* Program BXT Mipi clocks and dividers */
-static void bxt_dsi_program_clocks(struct drm_device *dev, enum port port)
+static void bxt_dsi_program_clocks(struct drm_device *dev, enum port port,
+ const struct intel_crtc_state *config)
{
- u32 tmp;
- u32 divider;
- u32 dsi_rate;
- u32 pll_ratio;
struct drm_i915_private *dev_priv = dev->dev_private;
+ u32 tmp;
+ u32 dsi_rate = 0;
+ u32 pll_ratio = 0;
+ u32 rx_div;
+ u32 tx_div;
+ u32 rx_div_upper;
+ u32 rx_div_lower;
+ u32 mipi_8by3_divider;
/* Clear old configurations */
tmp = I915_READ(BXT_MIPI_CLOCK_CTL);
tmp &= ~(BXT_MIPI_TX_ESCLK_FIXDIV_MASK(port));
- tmp &= ~(BXT_MIPI_RX_ESCLK_FIXDIV_MASK(port));
- tmp &= ~(BXT_MIPI_ESCLK_VAR_DIV_MASK(port));
- tmp &= ~(BXT_MIPI_DPHY_DIVIDER_MASK(port));
+ tmp &= ~(BXT_MIPI_RX_ESCLK_UPPER_FIXDIV_MASK(port));
+ tmp &= ~(BXT_MIPI_8X_BY3_DIVIDER_MASK(port));
+ tmp &= ~(BXT_MIPI_RX_ESCLK_LOWER_FIXDIV_MASK(port));
/* Get the current DSI rate(actual) */
- pll_ratio = I915_READ(BXT_DSI_PLL_CTL) &
- BXT_DSI_PLL_RATIO_MASK;
+ pll_ratio = config->dsi_pll.ctrl & BXT_DSI_PLL_RATIO_MASK;
dsi_rate = (BXT_REF_CLOCK_KHZ * pll_ratio) / 2;
- /* Max possible output of clock is 39.5 MHz, program value -1 */
- divider = (dsi_rate / BXT_MAX_VAR_OUTPUT_KHZ) - 1;
- tmp |= BXT_MIPI_ESCLK_VAR_DIV(port, divider);
+ /*
+ * tx clock should be <= 20MHz and the div value must be
+ * subtracted by 1 as per bspec
+ */
+ tx_div = DIV_ROUND_UP(dsi_rate, 20000) - 1;
+ /*
+ * rx clock should be <= 150MHz and the div value must be
+ * subtracted by 1 as per bspec
+ */
+ rx_div = DIV_ROUND_UP(dsi_rate, 150000) - 1;
/*
- * Tx escape clock must be as close to 20MHz possible, but should
- * not exceed it. Hence select divide by 2
+ * rx divider value needs to be updated in the
+ * two differnt bit fields in the register hence splitting the
+ * rx divider value accordingly
*/
- tmp |= BXT_MIPI_TX_ESCLK_8XDIV_BY2(port);
+ rx_div_lower = rx_div & RX_DIVIDER_BIT_1_2;
+ rx_div_upper = (rx_div & RX_DIVIDER_BIT_3_4) >> 2;
- tmp |= BXT_MIPI_RX_ESCLK_8X_BY3(port);
+ /* As per bpsec program the 8/3X clock divider to the below value */
+ if (dev_priv->vbt.dsi.config->is_cmd_mode)
+ mipi_8by3_divider = 0x2;
+ else
+ mipi_8by3_divider = 0x3;
+
+ tmp |= BXT_MIPI_8X_BY3_DIVIDER(port, mipi_8by3_divider);
+ tmp |= BXT_MIPI_TX_ESCLK_DIVIDER(port, tx_div);
+ tmp |= BXT_MIPI_RX_ESCLK_LOWER_DIVIDER(port, rx_div_lower);
+ tmp |= BXT_MIPI_RX_ESCLK_UPPER_DIVIDER(port, rx_div_upper);
I915_WRITE(BXT_MIPI_CLOCK_CTL, tmp);
}
-static bool bxt_configure_dsi_pll(struct intel_encoder *encoder)
+static int bxt_compute_dsi_pll(struct intel_encoder *encoder,
+ struct intel_crtc_state *config)
{
- struct drm_i915_private *dev_priv = encoder->base.dev->dev_private;
struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
u8 dsi_ratio;
u32 dsi_clk;
- u32 val;
dsi_clk = dsi_clk_from_pclk(intel_dsi->pclk, intel_dsi->pixel_format,
- intel_dsi->lane_count);
+ intel_dsi->lane_count);
/*
* From clock diagram, to get PLL ratio divider, divide double of DSI
@@ -413,9 +441,9 @@
*/
dsi_ratio = DIV_ROUND_UP(dsi_clk * 2, BXT_REF_CLOCK_KHZ);
if (dsi_ratio < BXT_DSI_PLL_RATIO_MIN ||
- dsi_ratio > BXT_DSI_PLL_RATIO_MAX) {
+ dsi_ratio > BXT_DSI_PLL_RATIO_MAX) {
DRM_ERROR("Cant get a suitable ratio from DSI PLL ratios\n");
- return false;
+ return -ECHRNG;
}
/*
@@ -423,27 +451,19 @@
* Spec says both have to be programmed, even if one is not getting
* used. Configure MIPI_CLOCK_CTL dividers in modeset
*/
- val = I915_READ(BXT_DSI_PLL_CTL);
- val &= ~BXT_DSI_PLL_PVD_RATIO_MASK;
- val &= ~BXT_DSI_FREQ_SEL_MASK;
- val &= ~BXT_DSI_PLL_RATIO_MASK;
- val |= (dsi_ratio | BXT_DSIA_16X_BY2 | BXT_DSIC_16X_BY2);
+ config->dsi_pll.ctrl = dsi_ratio | BXT_DSIA_16X_BY2 | BXT_DSIC_16X_BY2;
/* As per recommendation from hardware team,
* Prog PVD ratio =1 if dsi ratio <= 50
*/
- if (dsi_ratio <= 50) {
- val &= ~BXT_DSI_PLL_PVD_RATIO_MASK;
- val |= BXT_DSI_PLL_PVD_RATIO_1;
- }
+ if (dsi_ratio <= 50)
+ config->dsi_pll.ctrl |= BXT_DSI_PLL_PVD_RATIO_1;
- I915_WRITE(BXT_DSI_PLL_CTL, val);
- POSTING_READ(BXT_DSI_PLL_CTL);
-
- return true;
+ return 0;
}
-static void bxt_enable_dsi_pll(struct intel_encoder *encoder)
+static void bxt_enable_dsi_pll(struct intel_encoder *encoder,
+ const struct intel_crtc_state *config)
{
struct drm_i915_private *dev_priv = encoder->base.dev->dev_private;
struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
@@ -452,23 +472,13 @@
DRM_DEBUG_KMS("\n");
- val = I915_READ(BXT_DSI_PLL_ENABLE);
-
- if (val & BXT_DSI_PLL_DO_ENABLE) {
- WARN(1, "DSI PLL already enabled. Disabling it.\n");
- val &= ~BXT_DSI_PLL_DO_ENABLE;
- I915_WRITE(BXT_DSI_PLL_ENABLE, val);
- }
-
/* Configure PLL vales */
- if (!bxt_configure_dsi_pll(encoder)) {
- DRM_ERROR("Configure DSI PLL failed, abort PLL enable\n");
- return;
- }
+ I915_WRITE(BXT_DSI_PLL_CTL, config->dsi_pll.ctrl);
+ POSTING_READ(BXT_DSI_PLL_CTL);
/* Program TX, RX, Dphy clocks */
for_each_dsi_port(port, intel_dsi->ports)
- bxt_dsi_program_clocks(encoder->base.dev, port);
+ bxt_dsi_program_clocks(encoder->base.dev, port, config);
/* Enable DSI PLL */
val = I915_READ(BXT_DSI_PLL_ENABLE);
@@ -484,14 +494,38 @@
DRM_DEBUG_KMS("DSI PLL locked\n");
}
-void intel_enable_dsi_pll(struct intel_encoder *encoder)
+bool intel_dsi_pll_is_enabled(struct drm_i915_private *dev_priv)
+{
+ if (IS_BROXTON(dev_priv))
+ return bxt_dsi_pll_is_enabled(dev_priv);
+
+ MISSING_CASE(INTEL_DEVID(dev_priv));
+
+ return false;
+}
+
+int intel_compute_dsi_pll(struct intel_encoder *encoder,
+ struct intel_crtc_state *config)
{
struct drm_device *dev = encoder->base.dev;
if (IS_VALLEYVIEW(dev) || IS_CHERRYVIEW(dev))
- vlv_enable_dsi_pll(encoder);
+ return vlv_compute_dsi_pll(encoder, config);
else if (IS_BROXTON(dev))
- bxt_enable_dsi_pll(encoder);
+ return bxt_compute_dsi_pll(encoder, config);
+
+ return -ENODEV;
+}
+
+void intel_enable_dsi_pll(struct intel_encoder *encoder,
+ const struct intel_crtc_state *config)
+{
+ struct drm_device *dev = encoder->base.dev;
+
+ if (IS_VALLEYVIEW(dev) || IS_CHERRYVIEW(dev))
+ vlv_enable_dsi_pll(encoder, config);
+ else if (IS_BROXTON(dev))
+ bxt_enable_dsi_pll(encoder, config);
}
void intel_disable_dsi_pll(struct intel_encoder *encoder)
@@ -513,9 +547,9 @@
/* Clear old configurations */
tmp = I915_READ(BXT_MIPI_CLOCK_CTL);
tmp &= ~(BXT_MIPI_TX_ESCLK_FIXDIV_MASK(port));
- tmp &= ~(BXT_MIPI_RX_ESCLK_FIXDIV_MASK(port));
- tmp &= ~(BXT_MIPI_ESCLK_VAR_DIV_MASK(port));
- tmp &= ~(BXT_MIPI_DPHY_DIVIDER_MASK(port));
+ tmp &= ~(BXT_MIPI_RX_ESCLK_UPPER_FIXDIV_MASK(port));
+ tmp &= ~(BXT_MIPI_8X_BY3_DIVIDER_MASK(port));
+ tmp &= ~(BXT_MIPI_RX_ESCLK_LOWER_FIXDIV_MASK(port));
I915_WRITE(BXT_MIPI_CLOCK_CTL, tmp);
I915_WRITE(MIPI_EOT_DISABLE(port), CLOCKSTOP);
}
diff --git a/drivers/gpu/drm/i915/intel_fbc.c b/drivers/gpu/drm/i915/intel_fbc.c
index 0f0492f..d5a7cfe 100644
--- a/drivers/gpu/drm/i915/intel_fbc.c
+++ b/drivers/gpu/drm/i915/intel_fbc.c
@@ -506,6 +506,7 @@
int size,
int fb_cpp)
{
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
int compression_threshold = 1;
int ret;
u64 end;
@@ -516,9 +517,9 @@
* underruns, even if that range is not reserved by the BIOS. */
if (IS_BROADWELL(dev_priv) ||
IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv))
- end = dev_priv->gtt.stolen_size - 8 * 1024 * 1024;
+ end = ggtt->stolen_size - 8 * 1024 * 1024;
else
- end = dev_priv->gtt.stolen_usable_size;
+ end = ggtt->stolen_usable_size;
/* HACK: This code depends on what we will do in *_enable_fbc. If that
* code changes, this code needs to change as well.
diff --git a/drivers/gpu/drm/i915/intel_fbdev.c b/drivers/gpu/drm/i915/intel_fbdev.c
index 97a91e63..ab8d09a 100644
--- a/drivers/gpu/drm/i915/intel_fbdev.c
+++ b/drivers/gpu/drm/i915/intel_fbdev.c
@@ -122,6 +122,7 @@
struct drm_framebuffer *fb;
struct drm_device *dev = helper->dev;
struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct drm_mode_fb_cmd2 mode_cmd = {};
struct drm_i915_gem_object *obj = NULL;
int size, ret;
@@ -146,7 +147,7 @@
/* If the FB is too big, just don't use it since fbdev is not very
* important and we should probably use that space with FBC or other
* features. */
- if (size * 2 < dev_priv->gtt.stolen_usable_size)
+ if (size * 2 < ggtt->stolen_usable_size)
obj = i915_gem_object_create_stolen(dev, size);
if (obj == NULL)
obj = i915_gem_alloc_object(dev, size);
@@ -181,7 +182,8 @@
container_of(helper, struct intel_fbdev, helper);
struct intel_framebuffer *intel_fb = ifbdev->fb;
struct drm_device *dev = helper->dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct fb_info *info;
struct drm_framebuffer *fb;
struct drm_i915_gem_object *obj;
@@ -220,7 +222,7 @@
* This also validates that any existing fb inherited from the
* BIOS is suitable for own access.
*/
- ret = intel_pin_and_fence_fb_obj(NULL, &ifbdev->fb->base, NULL);
+ ret = intel_pin_and_fence_fb_obj(&ifbdev->fb->base, BIT(DRM_ROTATE_0));
if (ret)
goto out_unlock;
@@ -244,13 +246,13 @@
/* setup aperture base/size for vesafb takeover */
info->apertures->ranges[0].base = dev->mode_config.fb_base;
- info->apertures->ranges[0].size = dev_priv->gtt.mappable_end;
+ info->apertures->ranges[0].size = ggtt->mappable_end;
info->fix.smem_start = dev->mode_config.fb_base + i915_gem_obj_ggtt_offset(obj);
info->fix.smem_len = size;
info->screen_base =
- ioremap_wc(dev_priv->gtt.mappable_base + i915_gem_obj_ggtt_offset(obj),
+ ioremap_wc(ggtt->mappable_base + i915_gem_obj_ggtt_offset(obj),
size);
if (!info->screen_base) {
DRM_ERROR("Failed to remap framebuffer into virtual memory\n");
@@ -366,12 +368,12 @@
uint64_t conn_configured = 0, mask;
int pass = 0;
- save_enabled = kcalloc(dev->mode_config.num_connector, sizeof(bool),
+ save_enabled = kcalloc(fb_helper->connector_count, sizeof(bool),
GFP_KERNEL);
if (!save_enabled)
return false;
- memcpy(save_enabled, enabled, dev->mode_config.num_connector);
+ memcpy(save_enabled, enabled, fb_helper->connector_count);
mask = (1 << fb_helper->connector_count) - 1;
retry:
for (i = 0; i < fb_helper->connector_count; i++) {
@@ -379,6 +381,7 @@
struct drm_connector *connector;
struct drm_encoder *encoder;
struct drm_fb_helper_crtc *new_crtc;
+ struct intel_crtc *intel_crtc;
fb_conn = fb_helper->connector_info[i];
connector = fb_conn->connector;
@@ -420,6 +423,13 @@
num_connectors_enabled++;
+ intel_crtc = to_intel_crtc(connector->state->crtc);
+ for (j = 0; j < 256; j++) {
+ intel_crtc->lut_r[j] = j;
+ intel_crtc->lut_g[j] = j;
+ intel_crtc->lut_b[j] = j;
+ }
+
new_crtc = intel_fb_helper_crtc(fb_helper, connector->state->crtc);
/*
@@ -510,7 +520,7 @@
if (fallback) {
bail:
DRM_DEBUG_KMS("Not using firmware configuration\n");
- memcpy(enabled, save_enabled, dev->mode_config.num_connector);
+ memcpy(enabled, save_enabled, fb_helper->connector_count);
kfree(save_enabled);
return false;
}
diff --git a/drivers/gpu/drm/i915/intel_fifo_underrun.c b/drivers/gpu/drm/i915/intel_fifo_underrun.c
index bda5266..9be839a 100644
--- a/drivers/gpu/drm/i915/intel_fifo_underrun.c
+++ b/drivers/gpu/drm/i915/intel_fifo_underrun.c
@@ -212,7 +212,7 @@
I915_WRITE(SERR_INT, SERR_INT_TRANS_FIFO_UNDERRUN(pch_transcoder));
POSTING_READ(SERR_INT);
- DRM_ERROR("pch fifo underrun on pch transcoder %c\n",
+ DRM_ERROR("pch fifo underrun on pch transcoder %s\n",
transcoder_name(pch_transcoder));
}
@@ -235,7 +235,7 @@
if (old && I915_READ(SERR_INT) &
SERR_INT_TRANS_FIFO_UNDERRUN(pch_transcoder)) {
- DRM_ERROR("uncleared pch fifo underrun on pch transcoder %c\n",
+ DRM_ERROR("uncleared pch fifo underrun on pch transcoder %s\n",
transcoder_name(pch_transcoder));
}
}
@@ -333,7 +333,7 @@
old = !intel_crtc->pch_fifo_underrun_disabled;
intel_crtc->pch_fifo_underrun_disabled = !enable;
- if (HAS_PCH_IBX(dev_priv->dev))
+ if (HAS_PCH_IBX(dev_priv))
ibx_set_fifo_underrun_reporting(dev_priv->dev, pch_transcoder,
enable);
else
@@ -363,7 +363,7 @@
return;
/* GMCH can't disable fifo underruns, filter them. */
- if (HAS_GMCH_DISPLAY(dev_priv->dev) &&
+ if (HAS_GMCH_DISPLAY(dev_priv) &&
to_intel_crtc(crtc)->cpu_fifo_underrun_disabled)
return;
@@ -386,7 +386,7 @@
{
if (intel_set_pch_fifo_underrun_reporting(dev_priv, pch_transcoder,
false))
- DRM_ERROR("PCH transcoder %c FIFO underrun\n",
+ DRM_ERROR("PCH transcoder %s FIFO underrun\n",
transcoder_name(pch_transcoder));
}
diff --git a/drivers/gpu/drm/i915/intel_guc.h b/drivers/gpu/drm/i915/intel_guc.h
index 73002e9..9d79c4c 100644
--- a/drivers/gpu/drm/i915/intel_guc.h
+++ b/drivers/gpu/drm/i915/intel_guc.h
@@ -27,8 +27,34 @@
#include "intel_guc_fwif.h"
#include "i915_guc_reg.h"
+struct drm_i915_gem_request;
+
+/*
+ * This structure primarily describes the GEM object shared with the GuC.
+ * The GEM object is held for the entire lifetime of our interaction with
+ * the GuC, being allocated before the GuC is loaded with its firmware.
+ * Because there's no way to update the address used by the GuC after
+ * initialisation, the shared object must stay pinned into the GGTT as
+ * long as the GuC is in use. We also keep the first page (only) mapped
+ * into kernel address space, as it includes shared data that must be
+ * updated on every request submission.
+ *
+ * The single GEM object described here is actually made up of several
+ * separate areas, as far as the GuC is concerned. The first page (kept
+ * kmap'd) includes the "process decriptor" which holds sequence data for
+ * the doorbell, and one cacheline which actually *is* the doorbell; a
+ * write to this will "ring the doorbell" (i.e. send an interrupt to the
+ * GuC). The subsequent pages of the client object constitute the work
+ * queue (a circular array of work items), again described in the process
+ * descriptor. Work queue pages are mapped momentarily as required.
+ *
+ * Finally, we also keep a few statistics here, including the number of
+ * submissions to each engine, and a record of the last submission failure
+ * (if any).
+ */
struct i915_guc_client {
struct drm_i915_gem_object *client_obj;
+ void *client_base; /* first page (only) of above */
struct intel_context *owner;
struct intel_guc *guc;
uint32_t priority;
@@ -43,13 +69,14 @@
uint32_t wq_offset;
uint32_t wq_size;
uint32_t wq_tail;
- uint32_t wq_head;
+ uint32_t unused; /* Was 'wq_head' */
/* GuC submission statistics & status */
uint64_t submissions[GUC_MAX_ENGINES_NUM];
uint32_t q_fail;
uint32_t b_fail;
int retcode;
+ int spare; /* pad to 32 DWords */
};
enum intel_guc_fw_status {
diff --git a/drivers/gpu/drm/i915/intel_guc_loader.c b/drivers/gpu/drm/i915/intel_guc_loader.c
index 82a3c03..876e5da 100644
--- a/drivers/gpu/drm/i915/intel_guc_loader.c
+++ b/drivers/gpu/drm/i915/intel_guc_loader.c
@@ -59,7 +59,7 @@
*
*/
-#define I915_SKL_GUC_UCODE "i915/skl_guc_ver4.bin"
+#define I915_SKL_GUC_UCODE "i915/skl_guc_ver6.bin"
MODULE_FIRMWARE(I915_SKL_GUC_UCODE);
/* User-friendly representation of an enum */
@@ -81,14 +81,14 @@
static void direct_interrupts_to_host(struct drm_i915_private *dev_priv)
{
- struct intel_engine_cs *ring;
- int i, irqs;
+ struct intel_engine_cs *engine;
+ int irqs;
/* tell all command streamers NOT to forward interrupts and vblank to GuC */
irqs = _MASKED_FIELD(GFX_FORWARD_VBLANK_MASK, GFX_FORWARD_VBLANK_NEVER);
irqs |= _MASKED_BIT_DISABLE(GFX_INTERRUPT_STEERING);
- for_each_ring(ring, dev_priv, i)
- I915_WRITE(RING_MODE_GEN7(ring), irqs);
+ for_each_engine(engine, dev_priv)
+ I915_WRITE(RING_MODE_GEN7(engine), irqs);
/* route all GT interrupts to the host */
I915_WRITE(GUC_BCS_RCS_IER, 0);
@@ -98,14 +98,14 @@
static void direct_interrupts_to_guc(struct drm_i915_private *dev_priv)
{
- struct intel_engine_cs *ring;
- int i, irqs;
+ struct intel_engine_cs *engine;
+ int irqs;
/* tell all command streamers to forward interrupts and vblank to GuC */
irqs = _MASKED_FIELD(GFX_FORWARD_VBLANK_MASK, GFX_FORWARD_VBLANK_ALWAYS);
irqs |= _MASKED_BIT_ENABLE(GFX_INTERRUPT_STEERING);
- for_each_ring(ring, dev_priv, i)
- I915_WRITE(RING_MODE_GEN7(ring), irqs);
+ for_each_engine(engine, dev_priv)
+ I915_WRITE(RING_MODE_GEN7(engine), irqs);
/* route USER_INTERRUPT to Host, all others are sent to GuC. */
irqs = GT_RENDER_USER_INTERRUPT << GEN8_RCS_IRQ_SHIFT |
@@ -353,6 +353,24 @@
return ret;
}
+static int i915_reset_guc(struct drm_i915_private *dev_priv)
+{
+ int ret;
+ u32 guc_status;
+
+ ret = intel_guc_reset(dev_priv);
+ if (ret) {
+ DRM_ERROR("GuC reset failed, ret = %d\n", ret);
+ return ret;
+ }
+
+ guc_status = I915_READ(GUC_STATUS);
+ WARN(!(guc_status & GS_MIA_IN_RESET),
+ "GuC status: 0x%x, MIA core expected to be in reset\n", guc_status);
+
+ return ret;
+}
+
/**
* intel_guc_ucode_load() - load GuC uCode into the device
* @dev: drm device
@@ -369,7 +387,7 @@
{
struct drm_i915_private *dev_priv = dev->dev_private;
struct intel_guc_fw *guc_fw = &dev_priv->guc.guc_fw;
- int err = 0;
+ int retries, err = 0;
if (!i915.enable_guc_submission)
return 0;
@@ -417,9 +435,33 @@
if (err)
goto fail;
- err = guc_ucode_xfer(dev_priv);
- if (err)
- goto fail;
+ /*
+ * WaEnableuKernelHeaderValidFix:skl,bxt
+ * For BXT, this is only upto B0 but below WA is required for later
+ * steppings also so this is extended as well.
+ */
+ /* WaEnableGuCBootHashCheckNotSet:skl,bxt */
+ for (retries = 3; ; ) {
+ /*
+ * Always reset the GuC just before (re)loading, so
+ * that the state and timing are fairly predictable
+ */
+ err = i915_reset_guc(dev_priv);
+ if (err) {
+ DRM_ERROR("GuC reset failed, err %d\n", err);
+ goto fail;
+ }
+
+ err = guc_ucode_xfer(dev_priv);
+ if (!err)
+ break;
+
+ if (--retries == 0)
+ goto fail;
+
+ DRM_INFO("GuC fw load failed, err %d; will reset and "
+ "retry %d more time(s)\n", err, retries);
+ }
guc_fw->guc_fw_load_status = GUC_FIRMWARE_SUCCESS;
@@ -440,6 +482,7 @@
return 0;
fail:
+ DRM_ERROR("GuC firmware load failed, err %d\n", err);
if (guc_fw->guc_fw_load_status == GUC_FIRMWARE_PENDING)
guc_fw->guc_fw_load_status = GUC_FIRMWARE_FAIL;
@@ -595,8 +638,8 @@
fw_path = NULL;
} else if (IS_SKYLAKE(dev)) {
fw_path = I915_SKL_GUC_UCODE;
- guc_fw->guc_fw_major_wanted = 4;
- guc_fw->guc_fw_minor_wanted = 3;
+ guc_fw->guc_fw_major_wanted = 6;
+ guc_fw->guc_fw_minor_wanted = 1;
} else {
i915.enable_guc_submission = false;
fw_path = ""; /* unknown device */
diff --git a/drivers/gpu/drm/i915/intel_hdmi.c b/drivers/gpu/drm/i915/intel_hdmi.c
index 1ab6f68..2cdab73 100644
--- a/drivers/gpu/drm/i915/intel_hdmi.c
+++ b/drivers/gpu/drm/i915/intel_hdmi.c
@@ -638,7 +638,7 @@
reg = HSW_TVIDEO_DIP_GCP(crtc->config->cpu_transcoder);
else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv))
reg = VLV_TVIDEO_DIP_GCP(crtc->pipe);
- else if (HAS_PCH_SPLIT(dev_priv->dev))
+ else if (HAS_PCH_SPLIT(dev_priv))
reg = TVIDEO_DIP_GCP(crtc->pipe);
else
return false;
@@ -952,9 +952,6 @@
if (pipe_config->pixel_multiplier)
dotclock /= pipe_config->pixel_multiplier;
- if (HAS_PCH_SPLIT(dev_priv->dev))
- ironlake_check_encoder_dotclock(pipe_config, dotclock);
-
pipe_config->base.adjusted_mode.crtc_clock = dotclock;
}
diff --git a/drivers/gpu/drm/i915/intel_i2c.c b/drivers/gpu/drm/i915/intel_i2c.c
index 52fbe53..81de230 100644
--- a/drivers/gpu/drm/i915/intel_i2c.c
+++ b/drivers/gpu/drm/i915/intel_i2c.c
@@ -124,7 +124,7 @@
u32 val;
/* When using bit bashing for I2C, this bit needs to be set to 1 */
- if (!IS_PINEVIEW(dev_priv->dev))
+ if (!IS_PINEVIEW(dev_priv))
return;
val = I915_READ(DSPCLK_GATE_D);
@@ -264,7 +264,7 @@
u32 gmbus2 = 0;
DEFINE_WAIT(wait);
- if (!HAS_GMBUS_IRQ(dev_priv->dev))
+ if (!HAS_GMBUS_IRQ(dev_priv))
gmbus4_irq_en = 0;
/* Important: The hw handles only the first bit, so set only one! Since
@@ -300,7 +300,7 @@
#define C ((I915_READ_NOTRACE(GMBUS2) & GMBUS_ACTIVE) == 0)
- if (!HAS_GMBUS_IRQ(dev_priv->dev))
+ if (!HAS_GMBUS_IRQ(dev_priv))
return wait_for(C, 10);
/* Important: The hw handles only the first bit, so set only one! */
@@ -571,15 +571,14 @@
goto out;
timeout:
- DRM_INFO("GMBUS [%s] timed out, falling back to bit banging on pin %d\n",
- bus->adapter.name, bus->reg0 & 0xff);
+ DRM_DEBUG_KMS("GMBUS [%s] timed out, falling back to bit banging on pin %d\n",
+ bus->adapter.name, bus->reg0 & 0xff);
I915_WRITE(GMBUS0, 0);
/*
* Hardware may not support GMBUS over these pins? Try GPIO bitbanging
* instead. Use EAGAIN to have i2c core retry.
*/
- bus->force_bit = 1;
ret = -EAGAIN;
out:
@@ -597,10 +596,15 @@
intel_display_power_get(dev_priv, POWER_DOMAIN_GMBUS);
mutex_lock(&dev_priv->gmbus_mutex);
- if (bus->force_bit)
+ if (bus->force_bit) {
ret = i2c_bit_algo.master_xfer(adapter, msgs, num);
- else
+ if (ret < 0)
+ bus->force_bit &= ~GMBUS_FORCE_BIT_RETRY;
+ } else {
ret = do_gmbus_xfer(adapter, msgs, num);
+ if (ret == -EAGAIN)
+ bus->force_bit |= GMBUS_FORCE_BIT_RETRY;
+ }
mutex_unlock(&dev_priv->gmbus_mutex);
intel_display_power_put(dev_priv, POWER_DOMAIN_GMBUS);
@@ -718,11 +722,16 @@
void intel_gmbus_force_bit(struct i2c_adapter *adapter, bool force_bit)
{
struct intel_gmbus *bus = to_intel_gmbus(adapter);
+ struct drm_i915_private *dev_priv = bus->dev_priv;
+
+ mutex_lock(&dev_priv->gmbus_mutex);
bus->force_bit += force_bit ? 1 : -1;
DRM_DEBUG_KMS("%sabling bit-banging on %s. force bit now %d\n",
force_bit ? "en" : "dis", adapter->name,
bus->force_bit);
+
+ mutex_unlock(&dev_priv->gmbus_mutex);
}
void intel_teardown_gmbus(struct drm_device *dev)
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 5c6080f..6179b59 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -131,6 +131,7 @@
* preemption, but just sampling the new tail pointer).
*
*/
+#include <linux/interrupt.h>
#include <drm/drmP.h>
#include <drm/i915_drm.h>
@@ -228,9 +229,6 @@
static int intel_lr_context_pin(struct intel_context *ctx,
struct intel_engine_cs *engine);
-static void lrc_setup_hardware_status_page(struct intel_engine_cs *ring,
- struct drm_i915_gem_object *default_ctx_obj);
-
/**
* intel_sanitize_enable_execlists() - sanitize i915.enable_execlists
@@ -266,20 +264,23 @@
}
static void
-logical_ring_init_platform_invariants(struct intel_engine_cs *ring)
+logical_ring_init_platform_invariants(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
- ring->disable_lite_restore_wa = (IS_SKL_REVID(dev, 0, SKL_REVID_B0) ||
+ if (IS_GEN8(dev) || IS_GEN9(dev))
+ engine->idle_lite_restore_wa = ~0;
+
+ engine->disable_lite_restore_wa = (IS_SKL_REVID(dev, 0, SKL_REVID_B0) ||
IS_BXT_REVID(dev, 0, BXT_REVID_A1)) &&
- (ring->id == VCS || ring->id == VCS2);
+ (engine->id == VCS || engine->id == VCS2);
- ring->ctx_desc_template = GEN8_CTX_VALID;
- ring->ctx_desc_template |= GEN8_CTX_ADDRESSING_MODE(dev) <<
+ engine->ctx_desc_template = GEN8_CTX_VALID;
+ engine->ctx_desc_template |= GEN8_CTX_ADDRESSING_MODE(dev) <<
GEN8_CTX_ADDRESSING_MODE_SHIFT;
if (IS_GEN8(dev))
- ring->ctx_desc_template |= GEN8_CTX_L3LLC_COHERENT;
- ring->ctx_desc_template |= GEN8_CTX_PRIVILEGE;
+ engine->ctx_desc_template |= GEN8_CTX_L3LLC_COHERENT;
+ engine->ctx_desc_template |= GEN8_CTX_PRIVILEGE;
/* TODO: WaDisableLiteRestore when we start using semaphore
* signalling between Command Streamers */
@@ -287,8 +288,8 @@
/* WaEnableForceRestoreInCtxtDescForVCS:skl */
/* WaEnableForceRestoreInCtxtDescForVCS:bxt */
- if (ring->disable_lite_restore_wa)
- ring->ctx_desc_template |= GEN8_CTX_FORCE_RESTORE;
+ if (engine->disable_lite_restore_wa)
+ engine->ctx_desc_template |= GEN8_CTX_FORCE_RESTORE;
}
/**
@@ -311,24 +312,24 @@
*/
static void
intel_lr_context_descriptor_update(struct intel_context *ctx,
- struct intel_engine_cs *ring)
+ struct intel_engine_cs *engine)
{
uint64_t lrca, desc;
- lrca = ctx->engine[ring->id].lrc_vma->node.start +
+ lrca = ctx->engine[engine->id].lrc_vma->node.start +
LRC_PPHWSP_PN * PAGE_SIZE;
- desc = ring->ctx_desc_template; /* bits 0-11 */
+ desc = engine->ctx_desc_template; /* bits 0-11 */
desc |= lrca; /* bits 12-31 */
desc |= (lrca >> PAGE_SHIFT) << GEN8_CTX_ID_SHIFT; /* bits 32-51 */
- ctx->engine[ring->id].lrc_desc = desc;
+ ctx->engine[engine->id].lrc_desc = desc;
}
uint64_t intel_lr_context_descriptor(struct intel_context *ctx,
- struct intel_engine_cs *ring)
+ struct intel_engine_cs *engine)
{
- return ctx->engine[ring->id].lrc_desc;
+ return ctx->engine[engine->id].lrc_desc;
}
/**
@@ -348,98 +349,103 @@
* Return: 20-bits globally unique context ID.
*/
u32 intel_execlists_ctx_id(struct intel_context *ctx,
- struct intel_engine_cs *ring)
+ struct intel_engine_cs *engine)
{
- return intel_lr_context_descriptor(ctx, ring) >> GEN8_CTX_ID_SHIFT;
+ return intel_lr_context_descriptor(ctx, engine) >> GEN8_CTX_ID_SHIFT;
}
static void execlists_elsp_write(struct drm_i915_gem_request *rq0,
struct drm_i915_gem_request *rq1)
{
- struct intel_engine_cs *ring = rq0->ring;
- struct drm_device *dev = ring->dev;
+ struct intel_engine_cs *engine = rq0->engine;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
uint64_t desc[2];
if (rq1) {
- desc[1] = intel_lr_context_descriptor(rq1->ctx, rq1->ring);
+ desc[1] = intel_lr_context_descriptor(rq1->ctx, rq1->engine);
rq1->elsp_submitted++;
} else {
desc[1] = 0;
}
- desc[0] = intel_lr_context_descriptor(rq0->ctx, rq0->ring);
+ desc[0] = intel_lr_context_descriptor(rq0->ctx, rq0->engine);
rq0->elsp_submitted++;
/* You must always write both descriptors in the order below. */
- spin_lock(&dev_priv->uncore.lock);
- intel_uncore_forcewake_get__locked(dev_priv, FORCEWAKE_ALL);
- I915_WRITE_FW(RING_ELSP(ring), upper_32_bits(desc[1]));
- I915_WRITE_FW(RING_ELSP(ring), lower_32_bits(desc[1]));
+ I915_WRITE_FW(RING_ELSP(engine), upper_32_bits(desc[1]));
+ I915_WRITE_FW(RING_ELSP(engine), lower_32_bits(desc[1]));
- I915_WRITE_FW(RING_ELSP(ring), upper_32_bits(desc[0]));
+ I915_WRITE_FW(RING_ELSP(engine), upper_32_bits(desc[0]));
/* The context is automatically loaded after the following */
- I915_WRITE_FW(RING_ELSP(ring), lower_32_bits(desc[0]));
+ I915_WRITE_FW(RING_ELSP(engine), lower_32_bits(desc[0]));
/* ELSP is a wo register, use another nearby reg for posting */
- POSTING_READ_FW(RING_EXECLIST_STATUS_LO(ring));
- intel_uncore_forcewake_put__locked(dev_priv, FORCEWAKE_ALL);
- spin_unlock(&dev_priv->uncore.lock);
+ POSTING_READ_FW(RING_EXECLIST_STATUS_LO(engine));
}
-static int execlists_update_context(struct drm_i915_gem_request *rq)
+static void
+execlists_update_context_pdps(struct i915_hw_ppgtt *ppgtt, u32 *reg_state)
{
- struct intel_engine_cs *ring = rq->ring;
+ ASSIGN_CTX_PDP(ppgtt, reg_state, 3);
+ ASSIGN_CTX_PDP(ppgtt, reg_state, 2);
+ ASSIGN_CTX_PDP(ppgtt, reg_state, 1);
+ ASSIGN_CTX_PDP(ppgtt, reg_state, 0);
+}
+
+static void execlists_update_context(struct drm_i915_gem_request *rq)
+{
+ struct intel_engine_cs *engine = rq->engine;
struct i915_hw_ppgtt *ppgtt = rq->ctx->ppgtt;
- uint32_t *reg_state = rq->ctx->engine[ring->id].lrc_reg_state;
+ uint32_t *reg_state = rq->ctx->engine[engine->id].lrc_reg_state;
reg_state[CTX_RING_TAIL+1] = rq->tail;
- if (ppgtt && !USES_FULL_48BIT_PPGTT(ppgtt->base.dev)) {
- /* True 32b PPGTT with dynamic page allocation: update PDP
- * registers and point the unallocated PDPs to scratch page.
- * PML4 is allocated during ppgtt init, so this is not needed
- * in 48-bit mode.
- */
- ASSIGN_CTX_PDP(ppgtt, reg_state, 3);
- ASSIGN_CTX_PDP(ppgtt, reg_state, 2);
- ASSIGN_CTX_PDP(ppgtt, reg_state, 1);
- ASSIGN_CTX_PDP(ppgtt, reg_state, 0);
- }
-
- return 0;
+ /* True 32b PPGTT with dynamic page allocation: update PDP
+ * registers and point the unallocated PDPs to scratch page.
+ * PML4 is allocated during ppgtt init, so this is not needed
+ * in 48-bit mode.
+ */
+ if (ppgtt && !USES_FULL_48BIT_PPGTT(ppgtt->base.dev))
+ execlists_update_context_pdps(ppgtt, reg_state);
}
static void execlists_submit_requests(struct drm_i915_gem_request *rq0,
struct drm_i915_gem_request *rq1)
{
+ struct drm_i915_private *dev_priv = rq0->i915;
+ unsigned int fw_domains = rq0->engine->fw_domains;
+
execlists_update_context(rq0);
if (rq1)
execlists_update_context(rq1);
+ spin_lock_irq(&dev_priv->uncore.lock);
+ intel_uncore_forcewake_get__locked(dev_priv, fw_domains);
+
execlists_elsp_write(rq0, rq1);
+
+ intel_uncore_forcewake_put__locked(dev_priv, fw_domains);
+ spin_unlock_irq(&dev_priv->uncore.lock);
}
-static void execlists_context_unqueue(struct intel_engine_cs *ring)
+static void execlists_context_unqueue(struct intel_engine_cs *engine)
{
struct drm_i915_gem_request *req0 = NULL, *req1 = NULL;
- struct drm_i915_gem_request *cursor = NULL, *tmp = NULL;
+ struct drm_i915_gem_request *cursor, *tmp;
- assert_spin_locked(&ring->execlist_lock);
+ assert_spin_locked(&engine->execlist_lock);
/*
* If irqs are not active generate a warning as batches that finish
* without the irqs may get lost and a GPU Hang may occur.
*/
- WARN_ON(!intel_irqs_enabled(ring->dev->dev_private));
-
- if (list_empty(&ring->execlist_queue))
- return;
+ WARN_ON(!intel_irqs_enabled(engine->dev->dev_private));
/* Try to read in pairs */
- list_for_each_entry_safe(cursor, tmp, &ring->execlist_queue,
+ list_for_each_entry_safe(cursor, tmp, &engine->execlist_queue,
execlist_link) {
if (!req0) {
req0 = cursor;
@@ -448,172 +454,179 @@
* will update tail past first request's workload */
cursor->elsp_submitted = req0->elsp_submitted;
list_move_tail(&req0->execlist_link,
- &ring->execlist_retired_req_list);
+ &engine->execlist_retired_req_list);
req0 = cursor;
} else {
req1 = cursor;
+ WARN_ON(req1->elsp_submitted);
break;
}
}
- if (IS_GEN8(ring->dev) || IS_GEN9(ring->dev)) {
+ if (unlikely(!req0))
+ return;
+
+ if (req0->elsp_submitted & engine->idle_lite_restore_wa) {
/*
- * WaIdleLiteRestore: make sure we never cause a lite
- * restore with HEAD==TAIL
+ * WaIdleLiteRestore: make sure we never cause a lite restore
+ * with HEAD==TAIL.
+ *
+ * Apply the wa NOOPS to prevent ring:HEAD == req:TAIL as we
+ * resubmit the request. See gen8_emit_request() for where we
+ * prepare the padding after the end of the request.
*/
- if (req0->elsp_submitted) {
- /*
- * Apply the wa NOOPS to prevent ring:HEAD == req:TAIL
- * as we resubmit the request. See gen8_emit_request()
- * for where we prepare the padding after the end of the
- * request.
- */
- struct intel_ringbuffer *ringbuf;
+ struct intel_ringbuffer *ringbuf;
- ringbuf = req0->ctx->engine[ring->id].ringbuf;
- req0->tail += 8;
- req0->tail &= ringbuf->size - 1;
- }
+ ringbuf = req0->ctx->engine[engine->id].ringbuf;
+ req0->tail += 8;
+ req0->tail &= ringbuf->size - 1;
}
- WARN_ON(req1 && req1->elsp_submitted);
-
execlists_submit_requests(req0, req1);
}
-static bool execlists_check_remove_request(struct intel_engine_cs *ring,
- u32 request_id)
+static unsigned int
+execlists_check_remove_request(struct intel_engine_cs *engine, u32 request_id)
{
struct drm_i915_gem_request *head_req;
- assert_spin_locked(&ring->execlist_lock);
+ assert_spin_locked(&engine->execlist_lock);
- head_req = list_first_entry_or_null(&ring->execlist_queue,
+ head_req = list_first_entry_or_null(&engine->execlist_queue,
struct drm_i915_gem_request,
execlist_link);
- if (head_req != NULL) {
- if (intel_execlists_ctx_id(head_req->ctx, ring) == request_id) {
- WARN(head_req->elsp_submitted == 0,
- "Never submitted head request\n");
+ if (!head_req)
+ return 0;
- if (--head_req->elsp_submitted <= 0) {
- list_move_tail(&head_req->execlist_link,
- &ring->execlist_retired_req_list);
- return true;
- }
- }
- }
+ if (unlikely(intel_execlists_ctx_id(head_req->ctx, engine) != request_id))
+ return 0;
- return false;
+ WARN(head_req->elsp_submitted == 0, "Never submitted head request\n");
+
+ if (--head_req->elsp_submitted > 0)
+ return 0;
+
+ list_move_tail(&head_req->execlist_link,
+ &engine->execlist_retired_req_list);
+
+ return 1;
}
-static void get_context_status(struct intel_engine_cs *ring,
- u8 read_pointer,
- u32 *status, u32 *context_id)
+static u32
+get_context_status(struct intel_engine_cs *engine, unsigned int read_pointer,
+ u32 *context_id)
{
- struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct drm_i915_private *dev_priv = engine->dev->dev_private;
+ u32 status;
- if (WARN_ON(read_pointer >= GEN8_CSB_ENTRIES))
- return;
+ read_pointer %= GEN8_CSB_ENTRIES;
- *status = I915_READ(RING_CONTEXT_STATUS_BUF_LO(ring, read_pointer));
- *context_id = I915_READ(RING_CONTEXT_STATUS_BUF_HI(ring, read_pointer));
+ status = I915_READ_FW(RING_CONTEXT_STATUS_BUF_LO(engine, read_pointer));
+
+ if (status & GEN8_CTX_STATUS_IDLE_ACTIVE)
+ return 0;
+
+ *context_id = I915_READ_FW(RING_CONTEXT_STATUS_BUF_HI(engine,
+ read_pointer));
+
+ return status;
}
/**
* intel_lrc_irq_handler() - handle Context Switch interrupts
- * @ring: Engine Command Streamer to handle.
+ * @engine: Engine Command Streamer to handle.
*
* Check the unread Context Status Buffers and manage the submission of new
* contexts to the ELSP accordingly.
*/
-void intel_lrc_irq_handler(struct intel_engine_cs *ring)
+static void intel_lrc_irq_handler(unsigned long data)
{
- struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct intel_engine_cs *engine = (struct intel_engine_cs *)data;
+ struct drm_i915_private *dev_priv = engine->dev->dev_private;
u32 status_pointer;
- u8 read_pointer;
- u8 write_pointer;
- u32 status = 0;
- u32 status_id;
- u32 submit_contexts = 0;
+ unsigned int read_pointer, write_pointer;
+ u32 csb[GEN8_CSB_ENTRIES][2];
+ unsigned int csb_read = 0, i;
+ unsigned int submit_contexts = 0;
- status_pointer = I915_READ(RING_CONTEXT_STATUS_PTR(ring));
+ intel_uncore_forcewake_get(dev_priv, engine->fw_domains);
- read_pointer = ring->next_context_status_buffer;
+ status_pointer = I915_READ_FW(RING_CONTEXT_STATUS_PTR(engine));
+
+ read_pointer = engine->next_context_status_buffer;
write_pointer = GEN8_CSB_WRITE_PTR(status_pointer);
if (read_pointer > write_pointer)
write_pointer += GEN8_CSB_ENTRIES;
- spin_lock(&ring->execlist_lock);
-
while (read_pointer < write_pointer) {
+ if (WARN_ON_ONCE(csb_read == GEN8_CSB_ENTRIES))
+ break;
+ csb[csb_read][0] = get_context_status(engine, ++read_pointer,
+ &csb[csb_read][1]);
+ csb_read++;
+ }
- get_context_status(ring, ++read_pointer % GEN8_CSB_ENTRIES,
- &status, &status_id);
+ engine->next_context_status_buffer = write_pointer % GEN8_CSB_ENTRIES;
- if (status & GEN8_CTX_STATUS_IDLE_ACTIVE)
- continue;
+ /* Update the read pointer to the old write pointer. Manual ringbuffer
+ * management ftw </sarcasm> */
+ I915_WRITE_FW(RING_CONTEXT_STATUS_PTR(engine),
+ _MASKED_FIELD(GEN8_CSB_READ_PTR_MASK,
+ engine->next_context_status_buffer << 8));
- if (status & GEN8_CTX_STATUS_PREEMPTED) {
- if (status & GEN8_CTX_STATUS_LITE_RESTORE) {
- if (execlists_check_remove_request(ring, status_id))
+ intel_uncore_forcewake_put(dev_priv, engine->fw_domains);
+
+ spin_lock(&engine->execlist_lock);
+
+ for (i = 0; i < csb_read; i++) {
+ if (unlikely(csb[i][0] & GEN8_CTX_STATUS_PREEMPTED)) {
+ if (csb[i][0] & GEN8_CTX_STATUS_LITE_RESTORE) {
+ if (execlists_check_remove_request(engine, csb[i][1]))
WARN(1, "Lite Restored request removed from queue\n");
} else
WARN(1, "Preemption without Lite Restore\n");
}
- if ((status & GEN8_CTX_STATUS_ACTIVE_IDLE) ||
- (status & GEN8_CTX_STATUS_ELEMENT_SWITCH)) {
- if (execlists_check_remove_request(ring, status_id))
- submit_contexts++;
- }
+ if (csb[i][0] & (GEN8_CTX_STATUS_ACTIVE_IDLE |
+ GEN8_CTX_STATUS_ELEMENT_SWITCH))
+ submit_contexts +=
+ execlists_check_remove_request(engine, csb[i][1]);
}
- if (ring->disable_lite_restore_wa) {
- /* Prevent a ctx to preempt itself */
- if ((status & GEN8_CTX_STATUS_ACTIVE_IDLE) &&
- (submit_contexts != 0))
- execlists_context_unqueue(ring);
- } else if (submit_contexts != 0) {
- execlists_context_unqueue(ring);
+ if (submit_contexts) {
+ if (!engine->disable_lite_restore_wa ||
+ (csb[i][0] & GEN8_CTX_STATUS_ACTIVE_IDLE))
+ execlists_context_unqueue(engine);
}
- spin_unlock(&ring->execlist_lock);
+ spin_unlock(&engine->execlist_lock);
if (unlikely(submit_contexts > 2))
DRM_ERROR("More than two context complete events?\n");
-
- ring->next_context_status_buffer = write_pointer % GEN8_CSB_ENTRIES;
-
- /* Update the read pointer to the old write pointer. Manual ringbuffer
- * management ftw </sarcasm> */
- I915_WRITE(RING_CONTEXT_STATUS_PTR(ring),
- _MASKED_FIELD(GEN8_CSB_READ_PTR_MASK,
- ring->next_context_status_buffer << 8));
}
-static int execlists_context_queue(struct drm_i915_gem_request *request)
+static void execlists_context_queue(struct drm_i915_gem_request *request)
{
- struct intel_engine_cs *ring = request->ring;
+ struct intel_engine_cs *engine = request->engine;
struct drm_i915_gem_request *cursor;
int num_elements = 0;
if (request->ctx != request->i915->kernel_context)
- intel_lr_context_pin(request->ctx, ring);
+ intel_lr_context_pin(request->ctx, engine);
i915_gem_request_reference(request);
- spin_lock_irq(&ring->execlist_lock);
+ spin_lock_bh(&engine->execlist_lock);
- list_for_each_entry(cursor, &ring->execlist_queue, execlist_link)
+ list_for_each_entry(cursor, &engine->execlist_queue, execlist_link)
if (++num_elements > 2)
break;
if (num_elements > 2) {
struct drm_i915_gem_request *tail_req;
- tail_req = list_last_entry(&ring->execlist_queue,
+ tail_req = list_last_entry(&engine->execlist_queue,
struct drm_i915_gem_request,
execlist_link);
@@ -621,41 +634,39 @@
WARN(tail_req->elsp_submitted != 0,
"More than 2 already-submitted reqs queued\n");
list_move_tail(&tail_req->execlist_link,
- &ring->execlist_retired_req_list);
+ &engine->execlist_retired_req_list);
}
}
- list_add_tail(&request->execlist_link, &ring->execlist_queue);
+ list_add_tail(&request->execlist_link, &engine->execlist_queue);
if (num_elements == 0)
- execlists_context_unqueue(ring);
+ execlists_context_unqueue(engine);
- spin_unlock_irq(&ring->execlist_lock);
-
- return 0;
+ spin_unlock_bh(&engine->execlist_lock);
}
static int logical_ring_invalidate_all_caches(struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
uint32_t flush_domains;
int ret;
flush_domains = 0;
- if (ring->gpu_caches_dirty)
+ if (engine->gpu_caches_dirty)
flush_domains = I915_GEM_GPU_DOMAINS;
- ret = ring->emit_flush(req, I915_GEM_GPU_DOMAINS, flush_domains);
+ ret = engine->emit_flush(req, I915_GEM_GPU_DOMAINS, flush_domains);
if (ret)
return ret;
- ring->gpu_caches_dirty = false;
+ engine->gpu_caches_dirty = false;
return 0;
}
static int execlists_move_to_gpu(struct drm_i915_gem_request *req,
struct list_head *vmas)
{
- const unsigned other_rings = ~intel_ring_flag(req->ring);
+ const unsigned other_rings = ~intel_engine_flag(req->engine);
struct i915_vma *vma;
uint32_t flush_domains = 0;
bool flush_chipset = false;
@@ -665,7 +676,7 @@
struct drm_i915_gem_object *obj = vma->obj;
if (obj->active & other_rings) {
- ret = i915_gem_object_sync(obj, req->ring, &req);
+ ret = i915_gem_object_sync(obj, req->engine, &req);
if (ret)
return ret;
}
@@ -689,7 +700,7 @@
{
int ret = 0;
- request->ringbuf = request->ctx->engine[request->ring->id].ringbuf;
+ request->ringbuf = request->ctx->engine[request->engine->id].ringbuf;
if (i915.enable_guc_submission) {
/*
@@ -705,7 +716,7 @@
}
if (request->ctx != request->i915->kernel_context)
- ret = intel_lr_context_pin(request->ctx, request->ring);
+ ret = intel_lr_context_pin(request->ctx, request->engine);
return ret;
}
@@ -714,7 +725,7 @@
int bytes)
{
struct intel_ringbuffer *ringbuf = req->ringbuf;
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
struct drm_i915_gem_request *target;
unsigned space;
int ret;
@@ -725,7 +736,7 @@
/* The whole point of reserving space is to not wait! */
WARN_ON(ringbuf->reserved_in_use);
- list_for_each_entry(target, &ring->request_list, list) {
+ list_for_each_entry(target, &engine->request_list, list) {
/*
* The request queue is per-engine, so can contain requests
* from multiple ringbuffers. Here, we must ignore any that
@@ -741,7 +752,7 @@
break;
}
- if (WARN_ON(&target->list == &ring->request_list))
+ if (WARN_ON(&target->list == &engine->request_list))
return -ENOSPC;
ret = i915_wait_request(target);
@@ -766,7 +777,7 @@
{
struct intel_ringbuffer *ringbuf = request->ringbuf;
struct drm_i915_private *dev_priv = request->i915;
- struct intel_engine_cs *engine = request->ring;
+ struct intel_engine_cs *engine = request->engine;
intel_logical_ring_advance(ringbuf);
request->tail = ringbuf->tail;
@@ -781,7 +792,7 @@
intel_logical_ring_emit(ringbuf, MI_NOOP);
intel_logical_ring_advance(ringbuf);
- if (intel_ring_stopped(engine))
+ if (intel_engine_stopped(engine))
return 0;
if (engine->last_context != request->ctx) {
@@ -879,17 +890,8 @@
*/
int intel_logical_ring_begin(struct drm_i915_gem_request *req, int num_dwords)
{
- struct drm_i915_private *dev_priv;
int ret;
- WARN_ON(req == NULL);
- dev_priv = req->ring->dev->dev_private;
-
- ret = i915_gem_check_wedge(&dev_priv->gpu_error,
- dev_priv->mm.interruptible);
- if (ret)
- return ret;
-
ret = logical_ring_prepare(req, num_dwords * sizeof(uint32_t));
if (ret)
return ret;
@@ -935,9 +937,9 @@
struct list_head *vmas)
{
struct drm_device *dev = params->dev;
- struct intel_engine_cs *ring = params->ring;
+ struct intel_engine_cs *engine = params->engine;
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_ringbuffer *ringbuf = params->ctx->engine[ring->id].ringbuf;
+ struct intel_ringbuffer *ringbuf = params->ctx->engine[engine->id].ringbuf;
u64 exec_start;
int instp_mode;
u32 instp_mask;
@@ -949,7 +951,7 @@
case I915_EXEC_CONSTANTS_REL_GENERAL:
case I915_EXEC_CONSTANTS_ABSOLUTE:
case I915_EXEC_CONSTANTS_REL_SURFACE:
- if (instp_mode != 0 && ring != &dev_priv->ring[RCS]) {
+ if (instp_mode != 0 && engine != &dev_priv->engine[RCS]) {
DRM_DEBUG("non-0 rel constants mode on non-RCS\n");
return -EINVAL;
}
@@ -978,7 +980,7 @@
if (ret)
return ret;
- if (ring == &dev_priv->ring[RCS] &&
+ if (engine == &dev_priv->engine[RCS] &&
instp_mode != dev_priv->relative_constants_mode) {
ret = intel_logical_ring_begin(params->request, 4);
if (ret)
@@ -996,116 +998,116 @@
exec_start = params->batch_obj_vm_offset +
args->batch_start_offset;
- ret = ring->emit_bb_start(params->request, exec_start, params->dispatch_flags);
+ ret = engine->emit_bb_start(params->request, exec_start, params->dispatch_flags);
if (ret)
return ret;
trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
i915_gem_execbuffer_move_to_active(vmas, params->request);
- i915_gem_execbuffer_retire_commands(params);
return 0;
}
-void intel_execlists_retire_requests(struct intel_engine_cs *ring)
+void intel_execlists_retire_requests(struct intel_engine_cs *engine)
{
struct drm_i915_gem_request *req, *tmp;
struct list_head retired_list;
- WARN_ON(!mutex_is_locked(&ring->dev->struct_mutex));
- if (list_empty(&ring->execlist_retired_req_list))
+ WARN_ON(!mutex_is_locked(&engine->dev->struct_mutex));
+ if (list_empty(&engine->execlist_retired_req_list))
return;
INIT_LIST_HEAD(&retired_list);
- spin_lock_irq(&ring->execlist_lock);
- list_replace_init(&ring->execlist_retired_req_list, &retired_list);
- spin_unlock_irq(&ring->execlist_lock);
+ spin_lock_bh(&engine->execlist_lock);
+ list_replace_init(&engine->execlist_retired_req_list, &retired_list);
+ spin_unlock_bh(&engine->execlist_lock);
list_for_each_entry_safe(req, tmp, &retired_list, execlist_link) {
struct intel_context *ctx = req->ctx;
struct drm_i915_gem_object *ctx_obj =
- ctx->engine[ring->id].state;
+ ctx->engine[engine->id].state;
if (ctx_obj && (ctx != req->i915->kernel_context))
- intel_lr_context_unpin(ctx, ring);
+ intel_lr_context_unpin(ctx, engine);
list_del(&req->execlist_link);
i915_gem_request_unreference(req);
}
}
-void intel_logical_ring_stop(struct intel_engine_cs *ring)
+void intel_logical_ring_stop(struct intel_engine_cs *engine)
{
- struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct drm_i915_private *dev_priv = engine->dev->dev_private;
int ret;
- if (!intel_ring_initialized(ring))
+ if (!intel_engine_initialized(engine))
return;
- ret = intel_ring_idle(ring);
- if (ret && !i915_reset_in_progress(&to_i915(ring->dev)->gpu_error))
+ ret = intel_engine_idle(engine);
+ if (ret)
DRM_ERROR("failed to quiesce %s whilst cleaning up: %d\n",
- ring->name, ret);
+ engine->name, ret);
/* TODO: Is this correct with Execlists enabled? */
- I915_WRITE_MODE(ring, _MASKED_BIT_ENABLE(STOP_RING));
- if (wait_for_atomic((I915_READ_MODE(ring) & MODE_IDLE) != 0, 1000)) {
- DRM_ERROR("%s :timed out trying to stop ring\n", ring->name);
+ I915_WRITE_MODE(engine, _MASKED_BIT_ENABLE(STOP_RING));
+ if (wait_for((I915_READ_MODE(engine) & MODE_IDLE) != 0, 1000)) {
+ DRM_ERROR("%s :timed out trying to stop ring\n", engine->name);
return;
}
- I915_WRITE_MODE(ring, _MASKED_BIT_DISABLE(STOP_RING));
+ I915_WRITE_MODE(engine, _MASKED_BIT_DISABLE(STOP_RING));
}
int logical_ring_flush_all_caches(struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
int ret;
- if (!ring->gpu_caches_dirty)
+ if (!engine->gpu_caches_dirty)
return 0;
- ret = ring->emit_flush(req, 0, I915_GEM_GPU_DOMAINS);
+ ret = engine->emit_flush(req, 0, I915_GEM_GPU_DOMAINS);
if (ret)
return ret;
- ring->gpu_caches_dirty = false;
+ engine->gpu_caches_dirty = false;
return 0;
}
static int intel_lr_context_do_pin(struct intel_context *ctx,
- struct intel_engine_cs *ring)
+ struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- struct drm_i915_gem_object *ctx_obj = ctx->engine[ring->id].state;
- struct intel_ringbuffer *ringbuf = ctx->engine[ring->id].ringbuf;
- struct page *lrc_state_page;
- uint32_t *lrc_reg_state;
+ struct drm_i915_gem_object *ctx_obj = ctx->engine[engine->id].state;
+ struct intel_ringbuffer *ringbuf = ctx->engine[engine->id].ringbuf;
+ void *vaddr;
+ u32 *lrc_reg_state;
int ret;
- WARN_ON(!mutex_is_locked(&ring->dev->struct_mutex));
+ WARN_ON(!mutex_is_locked(&engine->dev->struct_mutex));
ret = i915_gem_obj_ggtt_pin(ctx_obj, GEN8_LR_CONTEXT_ALIGN,
PIN_OFFSET_BIAS | GUC_WOPCM_TOP);
if (ret)
return ret;
- lrc_state_page = i915_gem_object_get_dirty_page(ctx_obj, LRC_STATE_PN);
- if (WARN_ON(!lrc_state_page)) {
- ret = -ENODEV;
+ vaddr = i915_gem_object_pin_map(ctx_obj);
+ if (IS_ERR(vaddr)) {
+ ret = PTR_ERR(vaddr);
goto unpin_ctx_obj;
}
- ret = intel_pin_and_map_ringbuffer_obj(ring->dev, ringbuf);
- if (ret)
- goto unpin_ctx_obj;
+ lrc_reg_state = vaddr + LRC_STATE_PN * PAGE_SIZE;
- ctx->engine[ring->id].lrc_vma = i915_gem_obj_to_ggtt(ctx_obj);
- intel_lr_context_descriptor_update(ctx, ring);
- lrc_reg_state = kmap(lrc_state_page);
+ ret = intel_pin_and_map_ringbuffer_obj(engine->dev, ringbuf);
+ if (ret)
+ goto unpin_map;
+
+ ctx->engine[engine->id].lrc_vma = i915_gem_obj_to_ggtt(ctx_obj);
+ intel_lr_context_descriptor_update(ctx, engine);
lrc_reg_state[CTX_RING_BUFFER_START+1] = ringbuf->vma->node.start;
- ctx->engine[ring->id].lrc_reg_state = lrc_reg_state;
+ ctx->engine[engine->id].lrc_reg_state = lrc_reg_state;
ctx_obj->dirty = true;
/* Invalidate GuC TLB. */
@@ -1114,6 +1116,8 @@
return ret;
+unpin_map:
+ i915_gem_object_unpin_map(ctx_obj);
unpin_ctx_obj:
i915_gem_object_ggtt_unpin(ctx_obj);
@@ -1146,7 +1150,7 @@
WARN_ON(!mutex_is_locked(&ctx->i915->dev->struct_mutex));
if (--ctx->engine[engine->id].pin_count == 0) {
- kunmap(kmap_to_page(ctx->engine[engine->id].lrc_reg_state));
+ i915_gem_object_unpin_map(ctx_obj);
intel_unpin_ringbuffer_obj(ctx->engine[engine->id].ringbuf);
i915_gem_object_ggtt_unpin(ctx_obj);
ctx->engine[engine->id].lrc_vma = NULL;
@@ -1160,16 +1164,16 @@
static int intel_logical_ring_workarounds_emit(struct drm_i915_gem_request *req)
{
int ret, i;
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
struct intel_ringbuffer *ringbuf = req->ringbuf;
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
struct i915_workarounds *w = &dev_priv->workarounds;
if (w->count == 0)
return 0;
- ring->gpu_caches_dirty = true;
+ engine->gpu_caches_dirty = true;
ret = logical_ring_flush_all_caches(req);
if (ret)
return ret;
@@ -1187,7 +1191,7 @@
intel_logical_ring_advance(ringbuf);
- ring->gpu_caches_dirty = true;
+ engine->gpu_caches_dirty = true;
ret = logical_ring_flush_all_caches(req);
if (ret)
return ret;
@@ -1223,7 +1227,7 @@
* This WA is also required for Gen9 so extracting as a function avoids
* code duplication.
*/
-static inline int gen8_emit_flush_coherentl3_wa(struct intel_engine_cs *ring,
+static inline int gen8_emit_flush_coherentl3_wa(struct intel_engine_cs *engine,
uint32_t *const batch,
uint32_t index)
{
@@ -1235,13 +1239,13 @@
* this batch updates GEN8_L3SQCREG4 with default value we need to
* set this bit here to retain the WA during flush.
*/
- if (IS_SKL_REVID(ring->dev, 0, SKL_REVID_E0))
+ if (IS_SKL_REVID(engine->dev, 0, SKL_REVID_E0))
l3sqc4_flush |= GEN8_LQSC_RO_PERF_DIS;
wa_ctx_emit(batch, index, (MI_STORE_REGISTER_MEM_GEN8 |
MI_SRM_LRM_GLOBAL_GTT));
wa_ctx_emit_reg(batch, index, GEN8_L3SQCREG4);
- wa_ctx_emit(batch, index, ring->scratch.gtt_offset + 256);
+ wa_ctx_emit(batch, index, engine->scratch.gtt_offset + 256);
wa_ctx_emit(batch, index, 0);
wa_ctx_emit(batch, index, MI_LOAD_REGISTER_IMM(1));
@@ -1259,7 +1263,7 @@
wa_ctx_emit(batch, index, (MI_LOAD_REGISTER_MEM_GEN8 |
MI_SRM_LRM_GLOBAL_GTT));
wa_ctx_emit_reg(batch, index, GEN8_L3SQCREG4);
- wa_ctx_emit(batch, index, ring->scratch.gtt_offset + 256);
+ wa_ctx_emit(batch, index, engine->scratch.gtt_offset + 256);
wa_ctx_emit(batch, index, 0);
return index;
@@ -1312,7 +1316,7 @@
* Return: non-zero if we exceed the PAGE_SIZE limit.
*/
-static int gen8_init_indirectctx_bb(struct intel_engine_cs *ring,
+static int gen8_init_indirectctx_bb(struct intel_engine_cs *engine,
struct i915_wa_ctx_bb *wa_ctx,
uint32_t *const batch,
uint32_t *offset)
@@ -1324,8 +1328,8 @@
wa_ctx_emit(batch, index, MI_ARB_ON_OFF | MI_ARB_DISABLE);
/* WaFlushCoherentL3CacheLinesAtContextSwitch:bdw */
- if (IS_BROADWELL(ring->dev)) {
- int rc = gen8_emit_flush_coherentl3_wa(ring, batch, index);
+ if (IS_BROADWELL(engine->dev)) {
+ int rc = gen8_emit_flush_coherentl3_wa(engine, batch, index);
if (rc < 0)
return rc;
index = rc;
@@ -1333,7 +1337,7 @@
/* WaClearSlmSpaceAtContextSwitch:bdw,chv */
/* Actual scratch location is at 128 bytes offset */
- scratch_addr = ring->scratch.gtt_offset + 2*CACHELINE_BYTES;
+ scratch_addr = engine->scratch.gtt_offset + 2*CACHELINE_BYTES;
wa_ctx_emit(batch, index, GFX_OP_PIPE_CONTROL(6));
wa_ctx_emit(batch, index, (PIPE_CONTROL_FLUSH_L3 |
@@ -1375,7 +1379,7 @@
* This batch is terminated with MI_BATCH_BUFFER_END and so we need not add padding
* to align it with cacheline as padding after MI_BATCH_BUFFER_END is redundant.
*/
-static int gen8_init_perctx_bb(struct intel_engine_cs *ring,
+static int gen8_init_perctx_bb(struct intel_engine_cs *engine,
struct i915_wa_ctx_bb *wa_ctx,
uint32_t *const batch,
uint32_t *offset)
@@ -1390,13 +1394,13 @@
return wa_ctx_end(wa_ctx, *offset = index, 1);
}
-static int gen9_init_indirectctx_bb(struct intel_engine_cs *ring,
+static int gen9_init_indirectctx_bb(struct intel_engine_cs *engine,
struct i915_wa_ctx_bb *wa_ctx,
uint32_t *const batch,
uint32_t *offset)
{
int ret;
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
uint32_t index = wa_ctx_start(wa_ctx, *offset, CACHELINE_DWORDS);
/* WaDisableCtxRestoreArbitration:skl,bxt */
@@ -1405,7 +1409,7 @@
wa_ctx_emit(batch, index, MI_ARB_ON_OFF | MI_ARB_DISABLE);
/* WaFlushCoherentL3CacheLinesAtContextSwitch:skl,bxt */
- ret = gen8_emit_flush_coherentl3_wa(ring, batch, index);
+ ret = gen8_emit_flush_coherentl3_wa(engine, batch, index);
if (ret < 0)
return ret;
index = ret;
@@ -1417,12 +1421,12 @@
return wa_ctx_end(wa_ctx, *offset = index, CACHELINE_DWORDS);
}
-static int gen9_init_perctx_bb(struct intel_engine_cs *ring,
+static int gen9_init_perctx_bb(struct intel_engine_cs *engine,
struct i915_wa_ctx_bb *wa_ctx,
uint32_t *const batch,
uint32_t *offset)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
uint32_t index = wa_ctx_start(wa_ctx, *offset, CACHELINE_DWORDS);
/* WaSetDisablePixMaskCammingAndRhwoInCommonSliceChicken:skl,bxt */
@@ -1435,6 +1439,25 @@
wa_ctx_emit(batch, index, MI_NOOP);
}
+ /* WaClearTdlStateAckDirtyBits:bxt */
+ if (IS_BXT_REVID(dev, 0, BXT_REVID_B0)) {
+ wa_ctx_emit(batch, index, MI_LOAD_REGISTER_IMM(4));
+
+ wa_ctx_emit_reg(batch, index, GEN8_STATE_ACK);
+ wa_ctx_emit(batch, index, _MASKED_BIT_DISABLE(GEN9_SUBSLICE_TDL_ACK_BITS));
+
+ wa_ctx_emit_reg(batch, index, GEN9_STATE_ACK_SLICE1);
+ wa_ctx_emit(batch, index, _MASKED_BIT_DISABLE(GEN9_SUBSLICE_TDL_ACK_BITS));
+
+ wa_ctx_emit_reg(batch, index, GEN9_STATE_ACK_SLICE2);
+ wa_ctx_emit(batch, index, _MASKED_BIT_DISABLE(GEN9_SUBSLICE_TDL_ACK_BITS));
+
+ wa_ctx_emit_reg(batch, index, GEN7_ROW_CHICKEN2);
+ /* dummy write to CS, mask bits are 0 to ensure the register is not modified */
+ wa_ctx_emit(batch, index, 0x0);
+ wa_ctx_emit(batch, index, MI_NOOP);
+ }
+
/* WaDisableCtxRestoreArbitration:skl,bxt */
if (IS_SKL_REVID(dev, 0, SKL_REVID_D0) ||
IS_BXT_REVID(dev, 0, BXT_REVID_A1))
@@ -1445,60 +1468,61 @@
return wa_ctx_end(wa_ctx, *offset = index, 1);
}
-static int lrc_setup_wa_ctx_obj(struct intel_engine_cs *ring, u32 size)
+static int lrc_setup_wa_ctx_obj(struct intel_engine_cs *engine, u32 size)
{
int ret;
- ring->wa_ctx.obj = i915_gem_alloc_object(ring->dev, PAGE_ALIGN(size));
- if (!ring->wa_ctx.obj) {
+ engine->wa_ctx.obj = i915_gem_alloc_object(engine->dev,
+ PAGE_ALIGN(size));
+ if (!engine->wa_ctx.obj) {
DRM_DEBUG_DRIVER("alloc LRC WA ctx backing obj failed.\n");
return -ENOMEM;
}
- ret = i915_gem_obj_ggtt_pin(ring->wa_ctx.obj, PAGE_SIZE, 0);
+ ret = i915_gem_obj_ggtt_pin(engine->wa_ctx.obj, PAGE_SIZE, 0);
if (ret) {
DRM_DEBUG_DRIVER("pin LRC WA ctx backing obj failed: %d\n",
ret);
- drm_gem_object_unreference(&ring->wa_ctx.obj->base);
+ drm_gem_object_unreference(&engine->wa_ctx.obj->base);
return ret;
}
return 0;
}
-static void lrc_destroy_wa_ctx_obj(struct intel_engine_cs *ring)
+static void lrc_destroy_wa_ctx_obj(struct intel_engine_cs *engine)
{
- if (ring->wa_ctx.obj) {
- i915_gem_object_ggtt_unpin(ring->wa_ctx.obj);
- drm_gem_object_unreference(&ring->wa_ctx.obj->base);
- ring->wa_ctx.obj = NULL;
+ if (engine->wa_ctx.obj) {
+ i915_gem_object_ggtt_unpin(engine->wa_ctx.obj);
+ drm_gem_object_unreference(&engine->wa_ctx.obj->base);
+ engine->wa_ctx.obj = NULL;
}
}
-static int intel_init_workaround_bb(struct intel_engine_cs *ring)
+static int intel_init_workaround_bb(struct intel_engine_cs *engine)
{
int ret;
uint32_t *batch;
uint32_t offset;
struct page *page;
- struct i915_ctx_workarounds *wa_ctx = &ring->wa_ctx;
+ struct i915_ctx_workarounds *wa_ctx = &engine->wa_ctx;
- WARN_ON(ring->id != RCS);
+ WARN_ON(engine->id != RCS);
/* update this when WA for higher Gen are added */
- if (INTEL_INFO(ring->dev)->gen > 9) {
+ if (INTEL_INFO(engine->dev)->gen > 9) {
DRM_ERROR("WA batch buffer is not initialized for Gen%d\n",
- INTEL_INFO(ring->dev)->gen);
+ INTEL_INFO(engine->dev)->gen);
return 0;
}
/* some WA perform writes to scratch page, ensure it is valid */
- if (ring->scratch.obj == NULL) {
- DRM_ERROR("scratch page not allocated for %s\n", ring->name);
+ if (engine->scratch.obj == NULL) {
+ DRM_ERROR("scratch page not allocated for %s\n", engine->name);
return -EINVAL;
}
- ret = lrc_setup_wa_ctx_obj(ring, PAGE_SIZE);
+ ret = lrc_setup_wa_ctx_obj(engine, PAGE_SIZE);
if (ret) {
DRM_DEBUG_DRIVER("Failed to setup context WA page: %d\n", ret);
return ret;
@@ -1508,29 +1532,29 @@
batch = kmap_atomic(page);
offset = 0;
- if (INTEL_INFO(ring->dev)->gen == 8) {
- ret = gen8_init_indirectctx_bb(ring,
+ if (INTEL_INFO(engine->dev)->gen == 8) {
+ ret = gen8_init_indirectctx_bb(engine,
&wa_ctx->indirect_ctx,
batch,
&offset);
if (ret)
goto out;
- ret = gen8_init_perctx_bb(ring,
+ ret = gen8_init_perctx_bb(engine,
&wa_ctx->per_ctx,
batch,
&offset);
if (ret)
goto out;
- } else if (INTEL_INFO(ring->dev)->gen == 9) {
- ret = gen9_init_indirectctx_bb(ring,
+ } else if (INTEL_INFO(engine->dev)->gen == 9) {
+ ret = gen9_init_indirectctx_bb(engine,
&wa_ctx->indirect_ctx,
batch,
&offset);
if (ret)
goto out;
- ret = gen9_init_perctx_bb(ring,
+ ret = gen9_init_perctx_bb(engine,
&wa_ctx->per_ctx,
batch,
&offset);
@@ -1541,27 +1565,36 @@
out:
kunmap_atomic(batch);
if (ret)
- lrc_destroy_wa_ctx_obj(ring);
+ lrc_destroy_wa_ctx_obj(engine);
return ret;
}
-static int gen8_init_common_ring(struct intel_engine_cs *ring)
+static void lrc_init_hws(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_i915_private *dev_priv = engine->dev->dev_private;
+
+ I915_WRITE(RING_HWS_PGA(engine->mmio_base),
+ (u32)engine->status_page.gfx_addr);
+ POSTING_READ(RING_HWS_PGA(engine->mmio_base));
+}
+
+static int gen8_init_common_ring(struct intel_engine_cs *engine)
+{
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- u8 next_context_status_buffer_hw;
+ unsigned int next_context_status_buffer_hw;
- lrc_setup_hardware_status_page(ring,
- dev_priv->kernel_context->engine[ring->id].state);
+ lrc_init_hws(engine);
- I915_WRITE_IMR(ring, ~(ring->irq_enable_mask | ring->irq_keep_mask));
- I915_WRITE(RING_HWSTAM(ring->mmio_base), 0xffffffff);
+ I915_WRITE_IMR(engine,
+ ~(engine->irq_enable_mask | engine->irq_keep_mask));
+ I915_WRITE(RING_HWSTAM(engine->mmio_base), 0xffffffff);
- I915_WRITE(RING_MODE_GEN7(ring),
+ I915_WRITE(RING_MODE_GEN7(engine),
_MASKED_BIT_DISABLE(GFX_REPLAY_MODE) |
_MASKED_BIT_ENABLE(GFX_RUN_LIST_ENABLE));
- POSTING_READ(RING_MODE_GEN7(ring));
+ POSTING_READ(RING_MODE_GEN7(engine));
/*
* Instead of resetting the Context Status Buffer (CSB) read pointer to
@@ -1576,7 +1609,7 @@
* BXT | ? | ? |
*/
next_context_status_buffer_hw =
- GEN8_CSB_WRITE_PTR(I915_READ(RING_CONTEXT_STATUS_PTR(ring)));
+ GEN8_CSB_WRITE_PTR(I915_READ(RING_CONTEXT_STATUS_PTR(engine)));
/*
* When the CSB registers are reset (also after power-up / gpu reset),
@@ -1586,21 +1619,21 @@
if (next_context_status_buffer_hw == GEN8_CSB_PTR_MASK)
next_context_status_buffer_hw = (GEN8_CSB_ENTRIES - 1);
- ring->next_context_status_buffer = next_context_status_buffer_hw;
- DRM_DEBUG_DRIVER("Execlists enabled for %s\n", ring->name);
+ engine->next_context_status_buffer = next_context_status_buffer_hw;
+ DRM_DEBUG_DRIVER("Execlists enabled for %s\n", engine->name);
- memset(&ring->hangcheck, 0, sizeof(ring->hangcheck));
+ intel_engine_init_hangcheck(engine);
- return 0;
+ return intel_mocs_init_engine(engine);
}
-static int gen8_init_render_ring(struct intel_engine_cs *ring)
+static int gen8_init_render_ring(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
int ret;
- ret = gen8_init_common_ring(ring);
+ ret = gen8_init_common_ring(engine);
if (ret)
return ret;
@@ -1614,24 +1647,24 @@
I915_WRITE(INSTPM, _MASKED_BIT_ENABLE(INSTPM_FORCE_ORDERING));
- return init_workarounds_ring(ring);
+ return init_workarounds_ring(engine);
}
-static int gen9_init_render_ring(struct intel_engine_cs *ring)
+static int gen9_init_render_ring(struct intel_engine_cs *engine)
{
int ret;
- ret = gen8_init_common_ring(ring);
+ ret = gen8_init_common_ring(engine);
if (ret)
return ret;
- return init_workarounds_ring(ring);
+ return init_workarounds_ring(engine);
}
static int intel_logical_ring_emit_pdps(struct drm_i915_gem_request *req)
{
struct i915_hw_ppgtt *ppgtt = req->ctx->ppgtt;
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
struct intel_ringbuffer *ringbuf = req->ringbuf;
const int num_lri_cmds = GEN8_LEGACY_PDPES * 2;
int i, ret;
@@ -1644,9 +1677,11 @@
for (i = GEN8_LEGACY_PDPES - 1; i >= 0; i--) {
const dma_addr_t pd_daddr = i915_page_dir_dma_addr(ppgtt, i);
- intel_logical_ring_emit_reg(ringbuf, GEN8_RING_PDP_UDW(ring, i));
+ intel_logical_ring_emit_reg(ringbuf,
+ GEN8_RING_PDP_UDW(engine, i));
intel_logical_ring_emit(ringbuf, upper_32_bits(pd_daddr));
- intel_logical_ring_emit_reg(ringbuf, GEN8_RING_PDP_LDW(ring, i));
+ intel_logical_ring_emit_reg(ringbuf,
+ GEN8_RING_PDP_LDW(engine, i));
intel_logical_ring_emit(ringbuf, lower_32_bits(pd_daddr));
}
@@ -1670,7 +1705,7 @@
* not idle). PML4 is allocated during ppgtt init so this is
* not needed in 48-bit.*/
if (req->ctx->ppgtt &&
- (intel_ring_flag(req->ring) & req->ctx->ppgtt->pd_dirty_rings)) {
+ (intel_engine_flag(req->engine) & req->ctx->ppgtt->pd_dirty_rings)) {
if (!USES_FULL_48BIT_PPGTT(req->i915) &&
!intel_vgpu_active(req->i915->dev)) {
ret = intel_logical_ring_emit_pdps(req);
@@ -1678,7 +1713,7 @@
return ret;
}
- req->ctx->ppgtt->pd_dirty_rings &= ~intel_ring_flag(req->ring);
+ req->ctx->ppgtt->pd_dirty_rings &= ~intel_engine_flag(req->engine);
}
ret = intel_logical_ring_begin(req, 4);
@@ -1698,9 +1733,9 @@
return 0;
}
-static bool gen8_logical_ring_get_irq(struct intel_engine_cs *ring)
+static bool gen8_logical_ring_get_irq(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
unsigned long flags;
@@ -1708,25 +1743,26 @@
return false;
spin_lock_irqsave(&dev_priv->irq_lock, flags);
- if (ring->irq_refcount++ == 0) {
- I915_WRITE_IMR(ring, ~(ring->irq_enable_mask | ring->irq_keep_mask));
- POSTING_READ(RING_IMR(ring->mmio_base));
+ if (engine->irq_refcount++ == 0) {
+ I915_WRITE_IMR(engine,
+ ~(engine->irq_enable_mask | engine->irq_keep_mask));
+ POSTING_READ(RING_IMR(engine->mmio_base));
}
spin_unlock_irqrestore(&dev_priv->irq_lock, flags);
return true;
}
-static void gen8_logical_ring_put_irq(struct intel_engine_cs *ring)
+static void gen8_logical_ring_put_irq(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
unsigned long flags;
spin_lock_irqsave(&dev_priv->irq_lock, flags);
- if (--ring->irq_refcount == 0) {
- I915_WRITE_IMR(ring, ~ring->irq_keep_mask);
- POSTING_READ(RING_IMR(ring->mmio_base));
+ if (--engine->irq_refcount == 0) {
+ I915_WRITE_IMR(engine, ~engine->irq_keep_mask);
+ POSTING_READ(RING_IMR(engine->mmio_base));
}
spin_unlock_irqrestore(&dev_priv->irq_lock, flags);
}
@@ -1736,8 +1772,8 @@
u32 unused)
{
struct intel_ringbuffer *ringbuf = request->ringbuf;
- struct intel_engine_cs *ring = ringbuf->ring;
- struct drm_device *dev = ring->dev;
+ struct intel_engine_cs *engine = ringbuf->engine;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
uint32_t cmd;
int ret;
@@ -1757,7 +1793,7 @@
if (invalidate_domains & I915_GEM_GPU_DOMAINS) {
cmd |= MI_INVALIDATE_TLB;
- if (ring == &dev_priv->ring[VCS])
+ if (engine == &dev_priv->engine[VCS])
cmd |= MI_INVALIDATE_BSD;
}
@@ -1777,8 +1813,8 @@
u32 flush_domains)
{
struct intel_ringbuffer *ringbuf = request->ringbuf;
- struct intel_engine_cs *ring = ringbuf->ring;
- u32 scratch_addr = ring->scratch.gtt_offset + 2 * CACHELINE_BYTES;
+ struct intel_engine_cs *engine = ringbuf->engine;
+ u32 scratch_addr = engine->scratch.gtt_offset + 2 * CACHELINE_BYTES;
bool vf_flush_wa = false;
u32 flags = 0;
int ret;
@@ -1806,7 +1842,7 @@
* On GEN9: before VF_CACHE_INVALIDATE we need to emit a NULL
* pipe control.
*/
- if (IS_GEN9(ring->dev))
+ if (IS_GEN9(engine->dev))
vf_flush_wa = true;
}
@@ -1834,19 +1870,18 @@
return 0;
}
-static u32 gen8_get_seqno(struct intel_engine_cs *ring, bool lazy_coherency)
+static u32 gen8_get_seqno(struct intel_engine_cs *engine)
{
- return intel_read_status_page(ring, I915_GEM_HWS_INDEX);
+ return intel_read_status_page(engine, I915_GEM_HWS_INDEX);
}
-static void gen8_set_seqno(struct intel_engine_cs *ring, u32 seqno)
+static void gen8_set_seqno(struct intel_engine_cs *engine, u32 seqno)
{
- intel_write_status_page(ring, I915_GEM_HWS_INDEX, seqno);
+ intel_write_status_page(engine, I915_GEM_HWS_INDEX, seqno);
}
-static u32 bxt_a_get_seqno(struct intel_engine_cs *ring, bool lazy_coherency)
+static void bxt_a_seqno_barrier(struct intel_engine_cs *engine)
{
-
/*
* On BXT A steppings there is a HW coherency issue whereby the
* MI_STORE_DATA_IMM storing the completed request's seqno
@@ -1857,19 +1892,15 @@
* bxt_a_set_seqno(), where we also do a clflush after the write. So
* this clflush in practice becomes an invalidate operation.
*/
-
- if (!lazy_coherency)
- intel_flush_status_page(ring, I915_GEM_HWS_INDEX);
-
- return intel_read_status_page(ring, I915_GEM_HWS_INDEX);
+ intel_flush_status_page(engine, I915_GEM_HWS_INDEX);
}
-static void bxt_a_set_seqno(struct intel_engine_cs *ring, u32 seqno)
+static void bxt_a_set_seqno(struct intel_engine_cs *engine, u32 seqno)
{
- intel_write_status_page(ring, I915_GEM_HWS_INDEX, seqno);
+ intel_write_status_page(engine, I915_GEM_HWS_INDEX, seqno);
/* See bxt_a_get_seqno() explaining the reason for the clflush. */
- intel_flush_status_page(ring, I915_GEM_HWS_INDEX);
+ intel_flush_status_page(engine, I915_GEM_HWS_INDEX);
}
/*
@@ -1899,7 +1930,7 @@
intel_logical_ring_emit(ringbuf,
(MI_FLUSH_DW + 1) | MI_FLUSH_DW_OP_STOREDW);
intel_logical_ring_emit(ringbuf,
- hws_seqno_address(request->ring) |
+ hws_seqno_address(request->engine) |
MI_FLUSH_DW_USE_GTT);
intel_logical_ring_emit(ringbuf, 0);
intel_logical_ring_emit(ringbuf, i915_gem_request_get_seqno(request));
@@ -1929,7 +1960,7 @@
(PIPE_CONTROL_GLOBAL_GTT_IVB |
PIPE_CONTROL_CS_STALL |
PIPE_CONTROL_QW_WRITE));
- intel_logical_ring_emit(ringbuf, hws_seqno_address(request->ring));
+ intel_logical_ring_emit(ringbuf, hws_seqno_address(request->engine));
intel_logical_ring_emit(ringbuf, 0);
intel_logical_ring_emit(ringbuf, i915_gem_request_get_seqno(request));
/* We're thrashing one dword of HWS. */
@@ -1944,19 +1975,19 @@
struct render_state so;
int ret;
- ret = i915_gem_render_state_prepare(req->ring, &so);
+ ret = i915_gem_render_state_prepare(req->engine, &so);
if (ret)
return ret;
if (so.rodata == NULL)
return 0;
- ret = req->ring->emit_bb_start(req, so.ggtt_offset,
+ ret = req->engine->emit_bb_start(req, so.ggtt_offset,
I915_DISPATCH_SECURE);
if (ret)
goto out;
- ret = req->ring->emit_bb_start(req,
+ ret = req->engine->emit_bb_start(req,
(so.ggtt_offset + so.aux_batch_offset),
I915_DISPATCH_SECURE);
if (ret)
@@ -1994,146 +2025,197 @@
* @ring: Engine Command Streamer.
*
*/
-void intel_logical_ring_cleanup(struct intel_engine_cs *ring)
+void intel_logical_ring_cleanup(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv;
- if (!intel_ring_initialized(ring))
+ if (!intel_engine_initialized(engine))
return;
- dev_priv = ring->dev->dev_private;
+ /*
+ * Tasklet cannot be active at this point due intel_mark_active/idle
+ * so this is just for documentation.
+ */
+ if (WARN_ON(test_bit(TASKLET_STATE_SCHED, &engine->irq_tasklet.state)))
+ tasklet_kill(&engine->irq_tasklet);
- if (ring->buffer) {
- intel_logical_ring_stop(ring);
- WARN_ON((I915_READ_MODE(ring) & MODE_IDLE) == 0);
+ dev_priv = engine->dev->dev_private;
+
+ if (engine->buffer) {
+ intel_logical_ring_stop(engine);
+ WARN_ON((I915_READ_MODE(engine) & MODE_IDLE) == 0);
}
- if (ring->cleanup)
- ring->cleanup(ring);
+ if (engine->cleanup)
+ engine->cleanup(engine);
- i915_cmd_parser_fini_ring(ring);
- i915_gem_batch_pool_fini(&ring->batch_pool);
+ i915_cmd_parser_fini_ring(engine);
+ i915_gem_batch_pool_fini(&engine->batch_pool);
- if (ring->status_page.obj) {
- kunmap(sg_page(ring->status_page.obj->pages->sgl));
- ring->status_page.obj = NULL;
+ if (engine->status_page.obj) {
+ i915_gem_object_unpin_map(engine->status_page.obj);
+ engine->status_page.obj = NULL;
}
- ring->disable_lite_restore_wa = false;
- ring->ctx_desc_template = 0;
+ engine->idle_lite_restore_wa = 0;
+ engine->disable_lite_restore_wa = false;
+ engine->ctx_desc_template = 0;
- lrc_destroy_wa_ctx_obj(ring);
- ring->dev = NULL;
+ lrc_destroy_wa_ctx_obj(engine);
+ engine->dev = NULL;
}
static void
logical_ring_default_vfuncs(struct drm_device *dev,
- struct intel_engine_cs *ring)
+ struct intel_engine_cs *engine)
{
/* Default vfuncs which can be overriden by each engine. */
- ring->init_hw = gen8_init_common_ring;
- ring->emit_request = gen8_emit_request;
- ring->emit_flush = gen8_emit_flush;
- ring->irq_get = gen8_logical_ring_get_irq;
- ring->irq_put = gen8_logical_ring_put_irq;
- ring->emit_bb_start = gen8_emit_bb_start;
+ engine->init_hw = gen8_init_common_ring;
+ engine->emit_request = gen8_emit_request;
+ engine->emit_flush = gen8_emit_flush;
+ engine->irq_get = gen8_logical_ring_get_irq;
+ engine->irq_put = gen8_logical_ring_put_irq;
+ engine->emit_bb_start = gen8_emit_bb_start;
+ engine->get_seqno = gen8_get_seqno;
+ engine->set_seqno = gen8_set_seqno;
if (IS_BXT_REVID(dev, 0, BXT_REVID_A1)) {
- ring->get_seqno = bxt_a_get_seqno;
- ring->set_seqno = bxt_a_set_seqno;
- } else {
- ring->get_seqno = gen8_get_seqno;
- ring->set_seqno = gen8_set_seqno;
+ engine->irq_seqno_barrier = bxt_a_seqno_barrier;
+ engine->set_seqno = bxt_a_set_seqno;
}
}
static inline void
-logical_ring_default_irqs(struct intel_engine_cs *ring, unsigned shift)
+logical_ring_default_irqs(struct intel_engine_cs *engine, unsigned shift)
{
- ring->irq_enable_mask = GT_RENDER_USER_INTERRUPT << shift;
- ring->irq_keep_mask = GT_CONTEXT_SWITCH_INTERRUPT << shift;
+ engine->irq_enable_mask = GT_RENDER_USER_INTERRUPT << shift;
+ engine->irq_keep_mask = GT_CONTEXT_SWITCH_INTERRUPT << shift;
}
static int
-logical_ring_init(struct drm_device *dev, struct intel_engine_cs *ring)
+lrc_setup_hws(struct intel_engine_cs *engine,
+ struct drm_i915_gem_object *dctx_obj)
{
- struct intel_context *dctx = to_i915(dev)->kernel_context;
+ void *hws;
+
+ /* The HWSP is part of the default context object in LRC mode. */
+ engine->status_page.gfx_addr = i915_gem_obj_ggtt_offset(dctx_obj) +
+ LRC_PPHWSP_PN * PAGE_SIZE;
+ hws = i915_gem_object_pin_map(dctx_obj);
+ if (IS_ERR(hws))
+ return PTR_ERR(hws);
+ engine->status_page.page_addr = hws + LRC_PPHWSP_PN * PAGE_SIZE;
+ engine->status_page.obj = dctx_obj;
+
+ return 0;
+}
+
+static int
+logical_ring_init(struct drm_device *dev, struct intel_engine_cs *engine)
+{
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct intel_context *dctx = dev_priv->kernel_context;
+ enum forcewake_domains fw_domains;
int ret;
/* Intentionally left blank. */
- ring->buffer = NULL;
+ engine->buffer = NULL;
- ring->dev = dev;
- INIT_LIST_HEAD(&ring->active_list);
- INIT_LIST_HEAD(&ring->request_list);
- i915_gem_batch_pool_init(dev, &ring->batch_pool);
- init_waitqueue_head(&ring->irq_queue);
+ engine->dev = dev;
+ INIT_LIST_HEAD(&engine->active_list);
+ INIT_LIST_HEAD(&engine->request_list);
+ i915_gem_batch_pool_init(dev, &engine->batch_pool);
+ init_waitqueue_head(&engine->irq_queue);
- INIT_LIST_HEAD(&ring->buffers);
- INIT_LIST_HEAD(&ring->execlist_queue);
- INIT_LIST_HEAD(&ring->execlist_retired_req_list);
- spin_lock_init(&ring->execlist_lock);
+ INIT_LIST_HEAD(&engine->buffers);
+ INIT_LIST_HEAD(&engine->execlist_queue);
+ INIT_LIST_HEAD(&engine->execlist_retired_req_list);
+ spin_lock_init(&engine->execlist_lock);
- logical_ring_init_platform_invariants(ring);
+ tasklet_init(&engine->irq_tasklet,
+ intel_lrc_irq_handler, (unsigned long)engine);
- ret = i915_cmd_parser_init_ring(ring);
+ logical_ring_init_platform_invariants(engine);
+
+ fw_domains = intel_uncore_forcewake_for_reg(dev_priv,
+ RING_ELSP(engine),
+ FW_REG_WRITE);
+
+ fw_domains |= intel_uncore_forcewake_for_reg(dev_priv,
+ RING_CONTEXT_STATUS_PTR(engine),
+ FW_REG_READ | FW_REG_WRITE);
+
+ fw_domains |= intel_uncore_forcewake_for_reg(dev_priv,
+ RING_CONTEXT_STATUS_BUF_BASE(engine),
+ FW_REG_READ);
+
+ engine->fw_domains = fw_domains;
+
+ ret = i915_cmd_parser_init_ring(engine);
if (ret)
goto error;
- ret = intel_lr_context_deferred_alloc(dctx, ring);
+ ret = intel_lr_context_deferred_alloc(dctx, engine);
if (ret)
goto error;
/* As this is the default context, always pin it */
- ret = intel_lr_context_do_pin(dctx, ring);
+ ret = intel_lr_context_do_pin(dctx, engine);
if (ret) {
DRM_ERROR(
"Failed to pin and map ringbuffer %s: %d\n",
- ring->name, ret);
+ engine->name, ret);
+ goto error;
+ }
+
+ /* And setup the hardware status page. */
+ ret = lrc_setup_hws(engine, dctx->engine[engine->id].state);
+ if (ret) {
+ DRM_ERROR("Failed to set up hws %s: %d\n", engine->name, ret);
goto error;
}
return 0;
error:
- intel_logical_ring_cleanup(ring);
+ intel_logical_ring_cleanup(engine);
return ret;
}
static int logical_render_ring_init(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring = &dev_priv->ring[RCS];
+ struct intel_engine_cs *engine = &dev_priv->engine[RCS];
int ret;
- ring->name = "render ring";
- ring->id = RCS;
- ring->exec_id = I915_EXEC_RENDER;
- ring->guc_id = GUC_RENDER_ENGINE;
- ring->mmio_base = RENDER_RING_BASE;
+ engine->name = "render ring";
+ engine->id = RCS;
+ engine->exec_id = I915_EXEC_RENDER;
+ engine->guc_id = GUC_RENDER_ENGINE;
+ engine->mmio_base = RENDER_RING_BASE;
- logical_ring_default_irqs(ring, GEN8_RCS_IRQ_SHIFT);
+ logical_ring_default_irqs(engine, GEN8_RCS_IRQ_SHIFT);
if (HAS_L3_DPF(dev))
- ring->irq_keep_mask |= GT_RENDER_L3_PARITY_ERROR_INTERRUPT;
+ engine->irq_keep_mask |= GT_RENDER_L3_PARITY_ERROR_INTERRUPT;
- logical_ring_default_vfuncs(dev, ring);
+ logical_ring_default_vfuncs(dev, engine);
/* Override some for render ring. */
if (INTEL_INFO(dev)->gen >= 9)
- ring->init_hw = gen9_init_render_ring;
+ engine->init_hw = gen9_init_render_ring;
else
- ring->init_hw = gen8_init_render_ring;
- ring->init_context = gen8_init_rcs_context;
- ring->cleanup = intel_fini_pipe_control;
- ring->emit_flush = gen8_emit_flush_render;
- ring->emit_request = gen8_emit_request_render;
+ engine->init_hw = gen8_init_render_ring;
+ engine->init_context = gen8_init_rcs_context;
+ engine->cleanup = intel_fini_pipe_control;
+ engine->emit_flush = gen8_emit_flush_render;
+ engine->emit_request = gen8_emit_request_render;
- ring->dev = dev;
+ engine->dev = dev;
- ret = intel_init_pipe_control(ring);
+ ret = intel_init_pipe_control(engine);
if (ret)
return ret;
- ret = intel_init_workaround_bb(ring);
+ ret = intel_init_workaround_bb(engine);
if (ret) {
/*
* We continue even if we fail to initialize WA batch
@@ -2144,9 +2226,9 @@
ret);
}
- ret = logical_ring_init(dev, ring);
+ ret = logical_ring_init(dev, engine);
if (ret) {
- lrc_destroy_wa_ctx_obj(ring);
+ lrc_destroy_wa_ctx_obj(engine);
}
return ret;
@@ -2155,69 +2237,69 @@
static int logical_bsd_ring_init(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring = &dev_priv->ring[VCS];
+ struct intel_engine_cs *engine = &dev_priv->engine[VCS];
- ring->name = "bsd ring";
- ring->id = VCS;
- ring->exec_id = I915_EXEC_BSD;
- ring->guc_id = GUC_VIDEO_ENGINE;
- ring->mmio_base = GEN6_BSD_RING_BASE;
+ engine->name = "bsd ring";
+ engine->id = VCS;
+ engine->exec_id = I915_EXEC_BSD;
+ engine->guc_id = GUC_VIDEO_ENGINE;
+ engine->mmio_base = GEN6_BSD_RING_BASE;
- logical_ring_default_irqs(ring, GEN8_VCS1_IRQ_SHIFT);
- logical_ring_default_vfuncs(dev, ring);
+ logical_ring_default_irqs(engine, GEN8_VCS1_IRQ_SHIFT);
+ logical_ring_default_vfuncs(dev, engine);
- return logical_ring_init(dev, ring);
+ return logical_ring_init(dev, engine);
}
static int logical_bsd2_ring_init(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring = &dev_priv->ring[VCS2];
+ struct intel_engine_cs *engine = &dev_priv->engine[VCS2];
- ring->name = "bsd2 ring";
- ring->id = VCS2;
- ring->exec_id = I915_EXEC_BSD;
- ring->guc_id = GUC_VIDEO_ENGINE2;
- ring->mmio_base = GEN8_BSD2_RING_BASE;
+ engine->name = "bsd2 ring";
+ engine->id = VCS2;
+ engine->exec_id = I915_EXEC_BSD;
+ engine->guc_id = GUC_VIDEO_ENGINE2;
+ engine->mmio_base = GEN8_BSD2_RING_BASE;
- logical_ring_default_irqs(ring, GEN8_VCS2_IRQ_SHIFT);
- logical_ring_default_vfuncs(dev, ring);
+ logical_ring_default_irqs(engine, GEN8_VCS2_IRQ_SHIFT);
+ logical_ring_default_vfuncs(dev, engine);
- return logical_ring_init(dev, ring);
+ return logical_ring_init(dev, engine);
}
static int logical_blt_ring_init(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring = &dev_priv->ring[BCS];
+ struct intel_engine_cs *engine = &dev_priv->engine[BCS];
- ring->name = "blitter ring";
- ring->id = BCS;
- ring->exec_id = I915_EXEC_BLT;
- ring->guc_id = GUC_BLITTER_ENGINE;
- ring->mmio_base = BLT_RING_BASE;
+ engine->name = "blitter ring";
+ engine->id = BCS;
+ engine->exec_id = I915_EXEC_BLT;
+ engine->guc_id = GUC_BLITTER_ENGINE;
+ engine->mmio_base = BLT_RING_BASE;
- logical_ring_default_irqs(ring, GEN8_BCS_IRQ_SHIFT);
- logical_ring_default_vfuncs(dev, ring);
+ logical_ring_default_irqs(engine, GEN8_BCS_IRQ_SHIFT);
+ logical_ring_default_vfuncs(dev, engine);
- return logical_ring_init(dev, ring);
+ return logical_ring_init(dev, engine);
}
static int logical_vebox_ring_init(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring = &dev_priv->ring[VECS];
+ struct intel_engine_cs *engine = &dev_priv->engine[VECS];
- ring->name = "video enhancement ring";
- ring->id = VECS;
- ring->exec_id = I915_EXEC_VEBOX;
- ring->guc_id = GUC_VIDEOENHANCE_ENGINE;
- ring->mmio_base = VEBOX_RING_BASE;
+ engine->name = "video enhancement ring";
+ engine->id = VECS;
+ engine->exec_id = I915_EXEC_VEBOX;
+ engine->guc_id = GUC_VIDEOENHANCE_ENGINE;
+ engine->mmio_base = VEBOX_RING_BASE;
- logical_ring_default_irqs(ring, GEN8_VECS_IRQ_SHIFT);
- logical_ring_default_vfuncs(dev, ring);
+ logical_ring_default_irqs(engine, GEN8_VECS_IRQ_SHIFT);
+ logical_ring_default_vfuncs(dev, engine);
- return logical_ring_init(dev, ring);
+ return logical_ring_init(dev, engine);
}
/**
@@ -2225,7 +2307,7 @@
* @dev: DRM device.
*
* This function inits the engines for an Execlists submission style (the equivalent in the
- * legacy ringbuffer submission world would be i915_gem_init_rings). It does it only for
+ * legacy ringbuffer submission world would be i915_gem_init_engines). It does it only for
* those engines that are present in the hardware.
*
* Return: non-zero if the initialization failed.
@@ -2266,13 +2348,13 @@
return 0;
cleanup_vebox_ring:
- intel_logical_ring_cleanup(&dev_priv->ring[VECS]);
+ intel_logical_ring_cleanup(&dev_priv->engine[VECS]);
cleanup_blt_ring:
- intel_logical_ring_cleanup(&dev_priv->ring[BCS]);
+ intel_logical_ring_cleanup(&dev_priv->engine[BCS]);
cleanup_bsd_ring:
- intel_logical_ring_cleanup(&dev_priv->ring[VCS]);
+ intel_logical_ring_cleanup(&dev_priv->engine[VCS]);
cleanup_render_ring:
- intel_logical_ring_cleanup(&dev_priv->ring[RCS]);
+ intel_logical_ring_cleanup(&dev_priv->engine[RCS]);
return ret;
}
@@ -2320,13 +2402,13 @@
return rpcs;
}
-static u32 intel_lr_indirect_ctx_offset(struct intel_engine_cs *ring)
+static u32 intel_lr_indirect_ctx_offset(struct intel_engine_cs *engine)
{
u32 indirect_ctx_offset;
- switch (INTEL_INFO(ring->dev)->gen) {
+ switch (INTEL_INFO(engine->dev)->gen) {
default:
- MISSING_CASE(INTEL_INFO(ring->dev)->gen);
+ MISSING_CASE(INTEL_INFO(engine->dev)->gen);
/* fall through */
case 9:
indirect_ctx_offset =
@@ -2342,14 +2424,16 @@
}
static int
-populate_lr_context(struct intel_context *ctx, struct drm_i915_gem_object *ctx_obj,
- struct intel_engine_cs *ring, struct intel_ringbuffer *ringbuf)
+populate_lr_context(struct intel_context *ctx,
+ struct drm_i915_gem_object *ctx_obj,
+ struct intel_engine_cs *engine,
+ struct intel_ringbuffer *ringbuf)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
struct i915_hw_ppgtt *ppgtt = ctx->ppgtt;
- struct page *page;
- uint32_t *reg_state;
+ void *vaddr;
+ u32 *reg_state;
int ret;
if (!ppgtt)
@@ -2361,18 +2445,17 @@
return ret;
}
- ret = i915_gem_object_get_pages(ctx_obj);
- if (ret) {
- DRM_DEBUG_DRIVER("Could not get object pages\n");
+ vaddr = i915_gem_object_pin_map(ctx_obj);
+ if (IS_ERR(vaddr)) {
+ ret = PTR_ERR(vaddr);
+ DRM_DEBUG_DRIVER("Could not map object pages! (%d)\n", ret);
return ret;
}
-
- i915_gem_object_pin_pages(ctx_obj);
+ ctx_obj->dirty = true;
/* The second page of the context object contains some fields which must
* be set up prior to the first execution. */
- page = i915_gem_object_get_dirty_page(ctx_obj, LRC_STATE_PN);
- reg_state = kmap_atomic(page);
+ reg_state = vaddr + LRC_STATE_PN * PAGE_SIZE;
/* A context is actually a big batch buffer with several MI_LOAD_REGISTER_IMM
* commands followed by (reg, value) pairs. The values we are setting here are
@@ -2380,33 +2463,47 @@
* recreate this batchbuffer with new values (including all the missing
* MI_LOAD_REGISTER_IMM commands that we are not initializing here). */
reg_state[CTX_LRI_HEADER_0] =
- MI_LOAD_REGISTER_IMM(ring->id == RCS ? 14 : 11) | MI_LRI_FORCE_POSTED;
- ASSIGN_CTX_REG(reg_state, CTX_CONTEXT_CONTROL, RING_CONTEXT_CONTROL(ring),
+ MI_LOAD_REGISTER_IMM(engine->id == RCS ? 14 : 11) | MI_LRI_FORCE_POSTED;
+ ASSIGN_CTX_REG(reg_state, CTX_CONTEXT_CONTROL,
+ RING_CONTEXT_CONTROL(engine),
_MASKED_BIT_ENABLE(CTX_CTRL_INHIBIT_SYN_CTX_SWITCH |
CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT |
(HAS_RESOURCE_STREAMER(dev) ?
CTX_CTRL_RS_CTX_ENABLE : 0)));
- ASSIGN_CTX_REG(reg_state, CTX_RING_HEAD, RING_HEAD(ring->mmio_base), 0);
- ASSIGN_CTX_REG(reg_state, CTX_RING_TAIL, RING_TAIL(ring->mmio_base), 0);
+ ASSIGN_CTX_REG(reg_state, CTX_RING_HEAD, RING_HEAD(engine->mmio_base),
+ 0);
+ ASSIGN_CTX_REG(reg_state, CTX_RING_TAIL, RING_TAIL(engine->mmio_base),
+ 0);
/* Ring buffer start address is not known until the buffer is pinned.
* It is written to the context image in execlists_update_context()
*/
- ASSIGN_CTX_REG(reg_state, CTX_RING_BUFFER_START, RING_START(ring->mmio_base), 0);
- ASSIGN_CTX_REG(reg_state, CTX_RING_BUFFER_CONTROL, RING_CTL(ring->mmio_base),
+ ASSIGN_CTX_REG(reg_state, CTX_RING_BUFFER_START,
+ RING_START(engine->mmio_base), 0);
+ ASSIGN_CTX_REG(reg_state, CTX_RING_BUFFER_CONTROL,
+ RING_CTL(engine->mmio_base),
((ringbuf->size - PAGE_SIZE) & RING_NR_PAGES) | RING_VALID);
- ASSIGN_CTX_REG(reg_state, CTX_BB_HEAD_U, RING_BBADDR_UDW(ring->mmio_base), 0);
- ASSIGN_CTX_REG(reg_state, CTX_BB_HEAD_L, RING_BBADDR(ring->mmio_base), 0);
- ASSIGN_CTX_REG(reg_state, CTX_BB_STATE, RING_BBSTATE(ring->mmio_base),
+ ASSIGN_CTX_REG(reg_state, CTX_BB_HEAD_U,
+ RING_BBADDR_UDW(engine->mmio_base), 0);
+ ASSIGN_CTX_REG(reg_state, CTX_BB_HEAD_L,
+ RING_BBADDR(engine->mmio_base), 0);
+ ASSIGN_CTX_REG(reg_state, CTX_BB_STATE,
+ RING_BBSTATE(engine->mmio_base),
RING_BB_PPGTT);
- ASSIGN_CTX_REG(reg_state, CTX_SECOND_BB_HEAD_U, RING_SBBADDR_UDW(ring->mmio_base), 0);
- ASSIGN_CTX_REG(reg_state, CTX_SECOND_BB_HEAD_L, RING_SBBADDR(ring->mmio_base), 0);
- ASSIGN_CTX_REG(reg_state, CTX_SECOND_BB_STATE, RING_SBBSTATE(ring->mmio_base), 0);
- if (ring->id == RCS) {
- ASSIGN_CTX_REG(reg_state, CTX_BB_PER_CTX_PTR, RING_BB_PER_CTX_PTR(ring->mmio_base), 0);
- ASSIGN_CTX_REG(reg_state, CTX_RCS_INDIRECT_CTX, RING_INDIRECT_CTX(ring->mmio_base), 0);
- ASSIGN_CTX_REG(reg_state, CTX_RCS_INDIRECT_CTX_OFFSET, RING_INDIRECT_CTX_OFFSET(ring->mmio_base), 0);
- if (ring->wa_ctx.obj) {
- struct i915_ctx_workarounds *wa_ctx = &ring->wa_ctx;
+ ASSIGN_CTX_REG(reg_state, CTX_SECOND_BB_HEAD_U,
+ RING_SBBADDR_UDW(engine->mmio_base), 0);
+ ASSIGN_CTX_REG(reg_state, CTX_SECOND_BB_HEAD_L,
+ RING_SBBADDR(engine->mmio_base), 0);
+ ASSIGN_CTX_REG(reg_state, CTX_SECOND_BB_STATE,
+ RING_SBBSTATE(engine->mmio_base), 0);
+ if (engine->id == RCS) {
+ ASSIGN_CTX_REG(reg_state, CTX_BB_PER_CTX_PTR,
+ RING_BB_PER_CTX_PTR(engine->mmio_base), 0);
+ ASSIGN_CTX_REG(reg_state, CTX_RCS_INDIRECT_CTX,
+ RING_INDIRECT_CTX(engine->mmio_base), 0);
+ ASSIGN_CTX_REG(reg_state, CTX_RCS_INDIRECT_CTX_OFFSET,
+ RING_INDIRECT_CTX_OFFSET(engine->mmio_base), 0);
+ if (engine->wa_ctx.obj) {
+ struct i915_ctx_workarounds *wa_ctx = &engine->wa_ctx;
uint32_t ggtt_offset = i915_gem_obj_ggtt_offset(wa_ctx->obj);
reg_state[CTX_RCS_INDIRECT_CTX+1] =
@@ -2414,7 +2511,7 @@
(wa_ctx->indirect_ctx.size / CACHELINE_DWORDS);
reg_state[CTX_RCS_INDIRECT_CTX_OFFSET+1] =
- intel_lr_indirect_ctx_offset(ring) << 6;
+ intel_lr_indirect_ctx_offset(engine) << 6;
reg_state[CTX_BB_PER_CTX_PTR+1] =
(ggtt_offset + wa_ctx->per_ctx.offset * sizeof(uint32_t)) |
@@ -2422,16 +2519,25 @@
}
}
reg_state[CTX_LRI_HEADER_1] = MI_LOAD_REGISTER_IMM(9) | MI_LRI_FORCE_POSTED;
- ASSIGN_CTX_REG(reg_state, CTX_CTX_TIMESTAMP, RING_CTX_TIMESTAMP(ring->mmio_base), 0);
+ ASSIGN_CTX_REG(reg_state, CTX_CTX_TIMESTAMP,
+ RING_CTX_TIMESTAMP(engine->mmio_base), 0);
/* PDP values well be assigned later if needed */
- ASSIGN_CTX_REG(reg_state, CTX_PDP3_UDW, GEN8_RING_PDP_UDW(ring, 3), 0);
- ASSIGN_CTX_REG(reg_state, CTX_PDP3_LDW, GEN8_RING_PDP_LDW(ring, 3), 0);
- ASSIGN_CTX_REG(reg_state, CTX_PDP2_UDW, GEN8_RING_PDP_UDW(ring, 2), 0);
- ASSIGN_CTX_REG(reg_state, CTX_PDP2_LDW, GEN8_RING_PDP_LDW(ring, 2), 0);
- ASSIGN_CTX_REG(reg_state, CTX_PDP1_UDW, GEN8_RING_PDP_UDW(ring, 1), 0);
- ASSIGN_CTX_REG(reg_state, CTX_PDP1_LDW, GEN8_RING_PDP_LDW(ring, 1), 0);
- ASSIGN_CTX_REG(reg_state, CTX_PDP0_UDW, GEN8_RING_PDP_UDW(ring, 0), 0);
- ASSIGN_CTX_REG(reg_state, CTX_PDP0_LDW, GEN8_RING_PDP_LDW(ring, 0), 0);
+ ASSIGN_CTX_REG(reg_state, CTX_PDP3_UDW, GEN8_RING_PDP_UDW(engine, 3),
+ 0);
+ ASSIGN_CTX_REG(reg_state, CTX_PDP3_LDW, GEN8_RING_PDP_LDW(engine, 3),
+ 0);
+ ASSIGN_CTX_REG(reg_state, CTX_PDP2_UDW, GEN8_RING_PDP_UDW(engine, 2),
+ 0);
+ ASSIGN_CTX_REG(reg_state, CTX_PDP2_LDW, GEN8_RING_PDP_LDW(engine, 2),
+ 0);
+ ASSIGN_CTX_REG(reg_state, CTX_PDP1_UDW, GEN8_RING_PDP_UDW(engine, 1),
+ 0);
+ ASSIGN_CTX_REG(reg_state, CTX_PDP1_LDW, GEN8_RING_PDP_LDW(engine, 1),
+ 0);
+ ASSIGN_CTX_REG(reg_state, CTX_PDP0_UDW, GEN8_RING_PDP_UDW(engine, 0),
+ 0);
+ ASSIGN_CTX_REG(reg_state, CTX_PDP0_LDW, GEN8_RING_PDP_LDW(engine, 0),
+ 0);
if (USES_FULL_48BIT_PPGTT(ppgtt->base.dev)) {
/* 64b PPGTT (48bit canonical)
@@ -2445,20 +2551,16 @@
* With dynamic page allocation, PDPs may not be allocated at
* this point. Point the unallocated PDPs to the scratch page
*/
- ASSIGN_CTX_PDP(ppgtt, reg_state, 3);
- ASSIGN_CTX_PDP(ppgtt, reg_state, 2);
- ASSIGN_CTX_PDP(ppgtt, reg_state, 1);
- ASSIGN_CTX_PDP(ppgtt, reg_state, 0);
+ execlists_update_context_pdps(ppgtt, reg_state);
}
- if (ring->id == RCS) {
+ if (engine->id == RCS) {
reg_state[CTX_LRI_HEADER_2] = MI_LOAD_REGISTER_IMM(1);
ASSIGN_CTX_REG(reg_state, CTX_R_PWR_CLK_STATE, GEN8_R_PWR_CLK_STATE,
make_rpcs(dev));
}
- kunmap_atomic(reg_state);
- i915_gem_object_unpin_pages(ctx_obj);
+ i915_gem_object_unpin_map(ctx_obj);
return 0;
}
@@ -2475,7 +2577,7 @@
{
int i;
- for (i = I915_NUM_RINGS; --i >= 0; ) {
+ for (i = I915_NUM_ENGINES; --i >= 0; ) {
struct intel_ringbuffer *ringbuf = ctx->engine[i].ringbuf;
struct drm_i915_gem_object *ctx_obj = ctx->engine[i].state;
@@ -2485,6 +2587,7 @@
if (ctx == ctx->i915->kernel_context) {
intel_unpin_ringbuffer_obj(ringbuf);
i915_gem_object_ggtt_unpin(ctx_obj);
+ i915_gem_object_unpin_map(ctx_obj);
}
WARN_ON(ctx->engine[i].pin_count);
@@ -2507,15 +2610,15 @@
* in LRC mode, but does not include the "shared data page" used with
* GuC submission. The caller should account for this if using the GuC.
*/
-uint32_t intel_lr_context_size(struct intel_engine_cs *ring)
+uint32_t intel_lr_context_size(struct intel_engine_cs *engine)
{
int ret = 0;
- WARN_ON(INTEL_INFO(ring->dev)->gen < 8);
+ WARN_ON(INTEL_INFO(engine->dev)->gen < 8);
- switch (ring->id) {
+ switch (engine->id) {
case RCS:
- if (INTEL_INFO(ring->dev)->gen >= 9)
+ if (INTEL_INFO(engine->dev)->gen >= 9)
ret = GEN9_LR_CONTEXT_RENDER_SIZE;
else
ret = GEN8_LR_CONTEXT_RENDER_SIZE;
@@ -2531,24 +2634,6 @@
return ret;
}
-static void lrc_setup_hardware_status_page(struct intel_engine_cs *ring,
- struct drm_i915_gem_object *default_ctx_obj)
-{
- struct drm_i915_private *dev_priv = ring->dev->dev_private;
- struct page *page;
-
- /* The HWSP is part of the default context object in LRC mode. */
- ring->status_page.gfx_addr = i915_gem_obj_ggtt_offset(default_ctx_obj)
- + LRC_PPHWSP_PN * PAGE_SIZE;
- page = i915_gem_object_get_page(default_ctx_obj, LRC_PPHWSP_PN);
- ring->status_page.page_addr = kmap(page);
- ring->status_page.obj = default_ctx_obj;
-
- I915_WRITE(RING_HWS_PGA(ring->mmio_base),
- (u32)ring->status_page.gfx_addr);
- POSTING_READ(RING_HWS_PGA(ring->mmio_base));
-}
-
/**
* intel_lr_context_deferred_alloc() - create the LRC specific bits of a context
* @ctx: LR context to create.
@@ -2564,18 +2649,18 @@
*/
int intel_lr_context_deferred_alloc(struct intel_context *ctx,
- struct intel_engine_cs *ring)
+ struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_gem_object *ctx_obj;
uint32_t context_size;
struct intel_ringbuffer *ringbuf;
int ret;
WARN_ON(ctx->legacy_hw_ctx.rcs_state != NULL);
- WARN_ON(ctx->engine[ring->id].state);
+ WARN_ON(ctx->engine[engine->id].state);
- context_size = round_up(intel_lr_context_size(ring), 4096);
+ context_size = round_up(intel_lr_context_size(engine), 4096);
/* One extra page as the sharing data between driver and GuC */
context_size += PAGE_SIZE * LRC_PPHWSP_PN;
@@ -2586,39 +2671,38 @@
return -ENOMEM;
}
- ringbuf = intel_engine_create_ringbuffer(ring, 4 * PAGE_SIZE);
+ ringbuf = intel_engine_create_ringbuffer(engine, 4 * PAGE_SIZE);
if (IS_ERR(ringbuf)) {
ret = PTR_ERR(ringbuf);
goto error_deref_obj;
}
- ret = populate_lr_context(ctx, ctx_obj, ring, ringbuf);
+ ret = populate_lr_context(ctx, ctx_obj, engine, ringbuf);
if (ret) {
DRM_DEBUG_DRIVER("Failed to populate LRC: %d\n", ret);
goto error_ringbuf;
}
- ctx->engine[ring->id].ringbuf = ringbuf;
- ctx->engine[ring->id].state = ctx_obj;
+ ctx->engine[engine->id].ringbuf = ringbuf;
+ ctx->engine[engine->id].state = ctx_obj;
- if (ctx != ctx->i915->kernel_context && ring->init_context) {
+ if (ctx != ctx->i915->kernel_context && engine->init_context) {
struct drm_i915_gem_request *req;
- req = i915_gem_request_alloc(ring, ctx);
+ req = i915_gem_request_alloc(engine, ctx);
if (IS_ERR(req)) {
ret = PTR_ERR(req);
DRM_ERROR("ring create req: %d\n", ret);
goto error_ringbuf;
}
- ret = ring->init_context(req);
+ ret = engine->init_context(req);
+ i915_add_request_no_flush(req);
if (ret) {
DRM_ERROR("ring init context: %d\n",
ret);
- i915_gem_request_cancel(req);
goto error_ringbuf;
}
- i915_add_request_no_flush(req);
}
return 0;
@@ -2626,40 +2710,38 @@
intel_ringbuffer_free(ringbuf);
error_deref_obj:
drm_gem_object_unreference(&ctx_obj->base);
- ctx->engine[ring->id].ringbuf = NULL;
- ctx->engine[ring->id].state = NULL;
+ ctx->engine[engine->id].ringbuf = NULL;
+ ctx->engine[engine->id].state = NULL;
return ret;
}
-void intel_lr_context_reset(struct drm_device *dev,
- struct intel_context *ctx)
+void intel_lr_context_reset(struct drm_i915_private *dev_priv,
+ struct intel_context *ctx)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
- int i;
+ struct intel_engine_cs *engine;
- for_each_ring(ring, dev_priv, i) {
+ for_each_engine(engine, dev_priv) {
struct drm_i915_gem_object *ctx_obj =
- ctx->engine[ring->id].state;
+ ctx->engine[engine->id].state;
struct intel_ringbuffer *ringbuf =
- ctx->engine[ring->id].ringbuf;
+ ctx->engine[engine->id].ringbuf;
+ void *vaddr;
uint32_t *reg_state;
- struct page *page;
if (!ctx_obj)
continue;
- if (i915_gem_object_get_pages(ctx_obj)) {
- WARN(1, "Failed get_pages for context obj\n");
+ vaddr = i915_gem_object_pin_map(ctx_obj);
+ if (WARN_ON(IS_ERR(vaddr)))
continue;
- }
- page = i915_gem_object_get_dirty_page(ctx_obj, LRC_STATE_PN);
- reg_state = kmap_atomic(page);
+
+ reg_state = vaddr + LRC_STATE_PN * PAGE_SIZE;
+ ctx_obj->dirty = true;
reg_state[CTX_RING_HEAD+1] = 0;
reg_state[CTX_RING_TAIL+1] = 0;
- kunmap_atomic(reg_state);
+ i915_gem_object_unpin_map(ctx_obj);
ringbuf->head = 0;
ringbuf->tail = 0;
diff --git a/drivers/gpu/drm/i915/intel_lrc.h b/drivers/gpu/drm/i915/intel_lrc.h
index e6cda3e..461f1ef 100644
--- a/drivers/gpu/drm/i915/intel_lrc.h
+++ b/drivers/gpu/drm/i915/intel_lrc.h
@@ -24,6 +24,8 @@
#ifndef _INTEL_LRC_H_
#define _INTEL_LRC_H_
+#include "intel_ringbuffer.h"
+
#define GEN8_LR_CONTEXT_ALIGN 4096
/* Execlists regs */
@@ -34,6 +36,7 @@
#define CTX_CTRL_INHIBIT_SYN_CTX_SWITCH (1 << 3)
#define CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT (1 << 0)
#define CTX_CTRL_RS_CTX_ENABLE (1 << 1)
+#define RING_CONTEXT_STATUS_BUF_BASE(ring) _MMIO((ring)->mmio_base + 0x370)
#define RING_CONTEXT_STATUS_BUF_LO(ring, i) _MMIO((ring)->mmio_base + 0x370 + (i) * 8)
#define RING_CONTEXT_STATUS_BUF_HI(ring, i) _MMIO((ring)->mmio_base + 0x370 + (i) * 8 + 4)
#define RING_CONTEXT_STATUS_PTR(ring) _MMIO((ring)->mmio_base + 0x3a0)
@@ -57,8 +60,8 @@
/* Logical Rings */
int intel_logical_ring_alloc_request_extras(struct drm_i915_gem_request *request);
int intel_logical_ring_reserve_space(struct drm_i915_gem_request *request);
-void intel_logical_ring_stop(struct intel_engine_cs *ring);
-void intel_logical_ring_cleanup(struct intel_engine_cs *ring);
+void intel_logical_ring_stop(struct intel_engine_cs *engine);
+void intel_logical_ring_cleanup(struct intel_engine_cs *engine);
int intel_logical_rings_init(struct drm_device *dev);
int intel_logical_ring_begin(struct drm_i915_gem_request *req, int num_dwords);
@@ -98,18 +101,21 @@
#define LRC_STATE_PN (LRC_PPHWSP_PN + 1)
void intel_lr_context_free(struct intel_context *ctx);
-uint32_t intel_lr_context_size(struct intel_engine_cs *ring);
+uint32_t intel_lr_context_size(struct intel_engine_cs *engine);
int intel_lr_context_deferred_alloc(struct intel_context *ctx,
- struct intel_engine_cs *ring);
+ struct intel_engine_cs *engine);
void intel_lr_context_unpin(struct intel_context *ctx,
struct intel_engine_cs *engine);
-void intel_lr_context_reset(struct drm_device *dev,
- struct intel_context *ctx);
+
+struct drm_i915_private;
+
+void intel_lr_context_reset(struct drm_i915_private *dev_priv,
+ struct intel_context *ctx);
uint64_t intel_lr_context_descriptor(struct intel_context *ctx,
- struct intel_engine_cs *ring);
+ struct intel_engine_cs *engine);
u32 intel_execlists_ctx_id(struct intel_context *ctx,
- struct intel_engine_cs *ring);
+ struct intel_engine_cs *engine);
/* Execlists */
int intel_sanitize_enable_execlists(struct drm_device *dev, int enable_execlists);
@@ -118,7 +124,6 @@
struct drm_i915_gem_execbuffer2 *args,
struct list_head *vmas);
-void intel_lrc_irq_handler(struct intel_engine_cs *ring);
-void intel_execlists_retire_requests(struct intel_engine_cs *ring);
+void intel_execlists_retire_requests(struct intel_engine_cs *engine);
#endif /* _INTEL_LRC_H_ */
diff --git a/drivers/gpu/drm/i915/intel_lvds.c b/drivers/gpu/drm/i915/intel_lvds.c
index 10dc351..bc53c0d 100644
--- a/drivers/gpu/drm/i915/intel_lvds.c
+++ b/drivers/gpu/drm/i915/intel_lvds.c
@@ -109,7 +109,6 @@
struct drm_i915_private *dev_priv = dev->dev_private;
struct intel_lvds_encoder *lvds_encoder = to_lvds_encoder(&encoder->base);
u32 tmp, flags = 0;
- int dotclock;
tmp = I915_READ(lvds_encoder->reg);
if (tmp & LVDS_HSYNC_POLARITY)
@@ -134,12 +133,7 @@
pipe_config->gmch_pfit.control |= tmp & PANEL_8TO6_DITHER_ENABLE;
}
- dotclock = pipe_config->port_clock;
-
- if (HAS_PCH_SPLIT(dev_priv->dev))
- ironlake_check_encoder_dotclock(pipe_config, dotclock);
-
- pipe_config->base.adjusted_mode.crtc_clock = dotclock;
+ pipe_config->base.adjusted_mode.crtc_clock = pipe_config->port_clock;
}
static void intel_pre_enable_lvds(struct intel_encoder *encoder)
@@ -155,7 +149,7 @@
if (HAS_PCH_SPLIT(dev)) {
assert_fdi_rx_pll_disabled(dev_priv, pipe);
assert_shared_dpll_disabled(dev_priv,
- intel_crtc_to_shared_dpll(crtc));
+ crtc->config->shared_dpll);
} else {
assert_pll_disabled(dev_priv, pipe);
}
@@ -782,57 +776,6 @@
{ } /* terminating entry */
};
-/*
- * Enumerate the child dev array parsed from VBT to check whether
- * the LVDS is present.
- * If it is present, return 1.
- * If it is not present, return false.
- * If no child dev is parsed from VBT, it assumes that the LVDS is present.
- */
-static bool lvds_is_present_in_vbt(struct drm_device *dev,
- u8 *i2c_pin)
-{
- struct drm_i915_private *dev_priv = dev->dev_private;
- int i;
-
- if (!dev_priv->vbt.child_dev_num)
- return true;
-
- for (i = 0; i < dev_priv->vbt.child_dev_num; i++) {
- union child_device_config *uchild = dev_priv->vbt.child_dev + i;
- struct old_child_dev_config *child = &uchild->old;
-
- /* If the device type is not LFP, continue.
- * We have to check both the new identifiers as well as the
- * old for compatibility with some BIOSes.
- */
- if (child->device_type != DEVICE_TYPE_INT_LFP &&
- child->device_type != DEVICE_TYPE_LFP)
- continue;
-
- if (intel_gmbus_is_valid_pin(dev_priv, child->i2c_pin))
- *i2c_pin = child->i2c_pin;
-
- /* However, we cannot trust the BIOS writers to populate
- * the VBT correctly. Since LVDS requires additional
- * information from AIM blocks, a non-zero addin offset is
- * a good indicator that the LVDS is actually present.
- */
- if (child->addin_offset)
- return true;
-
- /* But even then some BIOS writers perform some black magic
- * and instantiate the device without reference to any
- * additional data. Trust that if the VBT was written into
- * the OpRegion then they have validated the LVDS's existence.
- */
- if (dev_priv->opregion.vbt)
- return true;
- }
-
- return false;
-}
-
static int intel_dual_link_lvds_callback(const struct dmi_system_id *id)
{
DRM_INFO("Forcing lvds to dual link mode on %s\n", id->ident);
@@ -982,14 +925,14 @@
if (HAS_PCH_SPLIT(dev)) {
if ((lvds & LVDS_DETECTED) == 0)
return;
- if (dev_priv->vbt.edp_support) {
+ if (dev_priv->vbt.edp.support) {
DRM_DEBUG_KMS("disable LVDS for eDP support\n");
return;
}
}
pin = GMBUS_PIN_PANEL;
- if (!lvds_is_present_in_vbt(dev, &pin)) {
+ if (!intel_bios_is_lvds_present(dev_priv, &pin)) {
if ((lvds & LVDS_PORT_EN) == 0) {
DRM_DEBUG_KMS("LVDS is not present in VBT\n");
return;
diff --git a/drivers/gpu/drm/i915/intel_mocs.c b/drivers/gpu/drm/i915/intel_mocs.c
index fed7bea..23b8545 100644
--- a/drivers/gpu/drm/i915/intel_mocs.c
+++ b/drivers/gpu/drm/i915/intel_mocs.c
@@ -128,9 +128,9 @@
/**
* get_mocs_settings()
- * @dev: DRM device.
+ * @dev_priv: i915 device.
* @table: Output table that will be made to point at appropriate
- * MOCS values for the device.
+ * MOCS values for the device.
*
* This function will return the values of the MOCS table that needs to
* be programmed for the platform. It will return the values that need
@@ -138,28 +138,28 @@
*
* Return: true if there are applicable MOCS settings for the device.
*/
-static bool get_mocs_settings(struct drm_device *dev,
+static bool get_mocs_settings(struct drm_i915_private *dev_priv,
struct drm_i915_mocs_table *table)
{
bool result = false;
- if (IS_SKYLAKE(dev) || IS_KABYLAKE(dev)) {
+ if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) {
table->size = ARRAY_SIZE(skylake_mocs_table);
table->table = skylake_mocs_table;
result = true;
- } else if (IS_BROXTON(dev)) {
+ } else if (IS_BROXTON(dev_priv)) {
table->size = ARRAY_SIZE(broxton_mocs_table);
table->table = broxton_mocs_table;
result = true;
} else {
- WARN_ONCE(INTEL_INFO(dev)->gen >= 9,
+ WARN_ONCE(INTEL_INFO(dev_priv)->gen >= 9,
"Platform that should have a MOCS table does not.\n");
}
return result;
}
-static i915_reg_t mocs_register(enum intel_ring_id ring, int index)
+static i915_reg_t mocs_register(enum intel_engine_id ring, int index)
{
switch (ring) {
case RCS:
@@ -179,10 +179,49 @@
}
/**
+ * intel_mocs_init_engine() - emit the mocs control table
+ * @engine: The engine for whom to emit the registers.
+ *
+ * This function simply emits a MI_LOAD_REGISTER_IMM command for the
+ * given table starting at the given address.
+ *
+ * Return: 0 on success, otherwise the error status.
+ */
+int intel_mocs_init_engine(struct intel_engine_cs *engine)
+{
+ struct drm_i915_private *dev_priv = to_i915(engine->dev);
+ struct drm_i915_mocs_table table;
+ unsigned int index;
+
+ if (!get_mocs_settings(dev_priv, &table))
+ return 0;
+
+ if (WARN_ON(table.size > GEN9_NUM_MOCS_ENTRIES))
+ return -ENODEV;
+
+ for (index = 0; index < table.size; index++)
+ I915_WRITE(mocs_register(engine->id, index),
+ table.table[index].control_value);
+
+ /*
+ * Ok, now set the unused entries to uncached. These entries
+ * are officially undefined and no contract for the contents
+ * and settings is given for these entries.
+ *
+ * Entry 0 in the table is uncached - so we are just writing
+ * that value to all the used entries.
+ */
+ for (; index < GEN9_NUM_MOCS_ENTRIES; index++)
+ I915_WRITE(mocs_register(engine->id, index),
+ table.table[0].control_value);
+
+ return 0;
+}
+
+/**
* emit_mocs_control_table() - emit the mocs control table
* @req: Request to set up the MOCS table for.
* @table: The values to program into the control regs.
- * @ring: The engine for whom to emit the registers.
*
* This function simply emits a MI_LOAD_REGISTER_IMM command for the
* given table starting at the given address.
@@ -190,10 +229,10 @@
* Return: 0 on success, otherwise the error status.
*/
static int emit_mocs_control_table(struct drm_i915_gem_request *req,
- const struct drm_i915_mocs_table *table,
- enum intel_ring_id ring)
+ const struct drm_i915_mocs_table *table)
{
struct intel_ringbuffer *ringbuf = req->ringbuf;
+ enum intel_engine_id engine = req->engine->id;
unsigned int index;
int ret;
@@ -210,7 +249,8 @@
MI_LOAD_REGISTER_IMM(GEN9_NUM_MOCS_ENTRIES));
for (index = 0; index < table->size; index++) {
- intel_logical_ring_emit_reg(ringbuf, mocs_register(ring, index));
+ intel_logical_ring_emit_reg(ringbuf,
+ mocs_register(engine, index));
intel_logical_ring_emit(ringbuf,
table->table[index].control_value);
}
@@ -224,8 +264,10 @@
* that value to all the used entries.
*/
for (; index < GEN9_NUM_MOCS_ENTRIES; index++) {
- intel_logical_ring_emit_reg(ringbuf, mocs_register(ring, index));
- intel_logical_ring_emit(ringbuf, table->table[0].control_value);
+ intel_logical_ring_emit_reg(ringbuf,
+ mocs_register(engine, index));
+ intel_logical_ring_emit(ringbuf,
+ table->table[0].control_value);
}
intel_logical_ring_emit(ringbuf, MI_NOOP);
@@ -234,6 +276,14 @@
return 0;
}
+static inline u32 l3cc_combine(const struct drm_i915_mocs_table *table,
+ u16 low,
+ u16 high)
+{
+ return table->table[low].l3cc_value |
+ table->table[high].l3cc_value << 16;
+}
+
/**
* emit_mocs_l3cc_table() - emit the mocs control table
* @req: Request to set up the MOCS table for.
@@ -249,11 +299,7 @@
const struct drm_i915_mocs_table *table)
{
struct intel_ringbuffer *ringbuf = req->ringbuf;
- unsigned int count;
unsigned int i;
- u32 value;
- u32 filler = (table->table[0].l3cc_value & 0xffff) |
- ((table->table[0].l3cc_value & 0xffff) << 16);
int ret;
if (WARN_ON(table->size > GEN9_NUM_MOCS_ENTRIES))
@@ -268,20 +314,18 @@
intel_logical_ring_emit(ringbuf,
MI_LOAD_REGISTER_IMM(GEN9_NUM_MOCS_ENTRIES / 2));
- for (i = 0, count = 0; i < table->size / 2; i++, count += 2) {
- value = (table->table[count].l3cc_value & 0xffff) |
- ((table->table[count + 1].l3cc_value & 0xffff) << 16);
-
+ for (i = 0; i < table->size/2; i++) {
intel_logical_ring_emit_reg(ringbuf, GEN9_LNCFCMOCS(i));
- intel_logical_ring_emit(ringbuf, value);
+ intel_logical_ring_emit(ringbuf,
+ l3cc_combine(table, 2*i, 2*i+1));
}
if (table->size & 0x01) {
/* Odd table size - 1 left over */
- value = (table->table[count].l3cc_value & 0xffff) |
- ((table->table[0].l3cc_value & 0xffff) << 16);
- } else
- value = filler;
+ intel_logical_ring_emit_reg(ringbuf, GEN9_LNCFCMOCS(i));
+ intel_logical_ring_emit(ringbuf, l3cc_combine(table, 2*i, 0));
+ i++;
+ }
/*
* Now set the rest of the table to uncached - use entry 0 as
@@ -290,9 +334,7 @@
*/
for (; i < GEN9_NUM_MOCS_ENTRIES / 2; i++) {
intel_logical_ring_emit_reg(ringbuf, GEN9_LNCFCMOCS(i));
- intel_logical_ring_emit(ringbuf, value);
-
- value = filler;
+ intel_logical_ring_emit(ringbuf, l3cc_combine(table, 0, 0));
}
intel_logical_ring_emit(ringbuf, MI_NOOP);
@@ -302,6 +344,47 @@
}
/**
+ * intel_mocs_init_l3cc_table() - program the mocs control table
+ * @dev: The the device to be programmed.
+ *
+ * This function simply programs the mocs registers for the given table
+ * starting at the given address. This register set is programmed in pairs.
+ *
+ * These registers may get programmed more than once, it is simpler to
+ * re-program 32 registers than maintain the state of when they were programmed.
+ * We are always reprogramming with the same values and this only on context
+ * start.
+ *
+ * Return: Nothing.
+ */
+void intel_mocs_init_l3cc_table(struct drm_device *dev)
+{
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct drm_i915_mocs_table table;
+ unsigned int i;
+
+ if (!get_mocs_settings(dev_priv, &table))
+ return;
+
+ for (i = 0; i < table.size/2; i++)
+ I915_WRITE(GEN9_LNCFCMOCS(i), l3cc_combine(&table, 2*i, 2*i+1));
+
+ /* Odd table size - 1 left over */
+ if (table.size & 0x01) {
+ I915_WRITE(GEN9_LNCFCMOCS(i), l3cc_combine(&table, 2*i, 0));
+ i++;
+ }
+
+ /*
+ * Now set the rest of the table to uncached - use entry 0 as
+ * this will be uncached. Leave the last pair as initialised as
+ * they are reserved by the hardware.
+ */
+ for (; i < (GEN9_NUM_MOCS_ENTRIES / 2); i++)
+ I915_WRITE(GEN9_LNCFCMOCS(i), l3cc_combine(&table, 0, 0));
+}
+
+/**
* intel_rcs_context_init_mocs() - program the MOCS register.
* @req: Request to set up the MOCS tables for.
*
@@ -322,17 +405,11 @@
struct drm_i915_mocs_table t;
int ret;
- if (get_mocs_settings(req->ring->dev, &t)) {
- struct drm_i915_private *dev_priv = req->i915;
- struct intel_engine_cs *ring;
- enum intel_ring_id ring_id;
-
- /* Program the control registers */
- for_each_ring(ring, dev_priv, ring_id) {
- ret = emit_mocs_control_table(req, &t, ring_id);
- if (ret)
- return ret;
- }
+ if (get_mocs_settings(req->i915, &t)) {
+ /* Program the RCS control registers */
+ ret = emit_mocs_control_table(req, &t);
+ if (ret)
+ return ret;
/* Now program the l3cc registers */
ret = emit_mocs_l3cc_table(req, &t);
diff --git a/drivers/gpu/drm/i915/intel_mocs.h b/drivers/gpu/drm/i915/intel_mocs.h
index 76e45b1..4640299 100644
--- a/drivers/gpu/drm/i915/intel_mocs.h
+++ b/drivers/gpu/drm/i915/intel_mocs.h
@@ -53,5 +53,7 @@
#include "i915_drv.h"
int intel_rcs_context_init_mocs(struct drm_i915_gem_request *req);
+void intel_mocs_init_l3cc_table(struct drm_device *dev);
+int intel_mocs_init_engine(struct intel_engine_cs *ring);
#endif
diff --git a/drivers/gpu/drm/i915/intel_opregion.c b/drivers/gpu/drm/i915/intel_opregion.c
index c15718b..99e2603 100644
--- a/drivers/gpu/drm/i915/intel_opregion.c
+++ b/drivers/gpu/drm/i915/intel_opregion.c
@@ -34,12 +34,6 @@
#include "i915_drv.h"
#include "intel_drv.h"
-#define PCI_ASLE 0xe4
-#define PCI_ASLS 0xfc
-#define PCI_SWSCI 0xe8
-#define PCI_SWSCI_SCISEL (1 << 15)
-#define PCI_SWSCI_GSSCIE (1 << 0)
-
#define OPREGION_HEADER_OFFSET 0
#define OPREGION_ACPI_OFFSET 0x100
#define ACPI_CLID 0x01ac /* current lid state indicator */
@@ -246,13 +240,12 @@
#define MAX_DSLP 1500
-#ifdef CONFIG_ACPI
static int swsci(struct drm_device *dev, u32 function, u32 parm, u32 *parm_out)
{
struct drm_i915_private *dev_priv = dev->dev_private;
struct opregion_swsci *swsci = dev_priv->opregion.swsci;
u32 main_function, sub_function, scic;
- u16 pci_swsci;
+ u16 swsci_val;
u32 dslp;
if (!swsci)
@@ -300,16 +293,16 @@
swsci->scic = scic;
/* Ensure SCI event is selected and event trigger is cleared. */
- pci_read_config_word(dev->pdev, PCI_SWSCI, &pci_swsci);
- if (!(pci_swsci & PCI_SWSCI_SCISEL) || (pci_swsci & PCI_SWSCI_GSSCIE)) {
- pci_swsci |= PCI_SWSCI_SCISEL;
- pci_swsci &= ~PCI_SWSCI_GSSCIE;
- pci_write_config_word(dev->pdev, PCI_SWSCI, pci_swsci);
+ pci_read_config_word(dev->pdev, SWSCI, &swsci_val);
+ if (!(swsci_val & SWSCI_SCISEL) || (swsci_val & SWSCI_GSSCIE)) {
+ swsci_val |= SWSCI_SCISEL;
+ swsci_val &= ~SWSCI_GSSCIE;
+ pci_write_config_word(dev->pdev, SWSCI, swsci_val);
}
/* Use event trigger to tell bios to check the mail. */
- pci_swsci |= PCI_SWSCI_GSSCIE;
- pci_write_config_word(dev->pdev, PCI_SWSCI, pci_swsci);
+ swsci_val |= SWSCI_GSSCIE;
+ pci_write_config_word(dev->pdev, SWSCI, swsci_val);
/* Poll for the result. */
#define C (((scic = swsci->scic) & SWSCI_SCIC_INDICATOR) == 0)
@@ -905,9 +898,6 @@
opregion->swsci_gbda_sub_functions,
opregion->swsci_sbcb_sub_functions);
}
-#else /* CONFIG_ACPI */
-static inline void swsci_setup(struct drm_device *dev) {}
-#endif /* CONFIG_ACPI */
static int intel_no_opregion_vbt_callback(const struct dmi_system_id *id)
{
@@ -943,16 +933,14 @@
BUILD_BUG_ON(sizeof(struct opregion_asle) != 0x100);
BUILD_BUG_ON(sizeof(struct opregion_asle_ext) != 0x400);
- pci_read_config_dword(dev->pdev, PCI_ASLS, &asls);
+ pci_read_config_dword(dev->pdev, ASLS, &asls);
DRM_DEBUG_DRIVER("graphic opregion physical addr: 0x%x\n", asls);
if (asls == 0) {
DRM_DEBUG_DRIVER("ACPI OpRegion not supported!\n");
return -ENOTSUPP;
}
-#ifdef CONFIG_ACPI
INIT_WORK(&opregion->asle_work, asle_work);
-#endif
base = memremap(asls, OPREGION_SIZE, MEMREMAP_WB);
if (!base)
@@ -1024,3 +1012,31 @@
memunmap(base);
return err;
}
+
+int
+intel_opregion_get_panel_type(struct drm_device *dev)
+{
+ u32 panel_details;
+ int ret;
+
+ ret = swsci(dev, SWSCI_GBDA_PANEL_DETAILS, 0x0, &panel_details);
+ if (ret) {
+ DRM_DEBUG_KMS("Failed to get panel details from OpRegion (%d)\n",
+ ret);
+ return ret;
+ }
+
+ ret = (panel_details >> 8) & 0xff;
+ if (ret > 0x10) {
+ DRM_DEBUG_KMS("Invalid OpRegion panel type 0x%x\n", ret);
+ return -EINVAL;
+ }
+
+ /* fall back to VBT panel type? */
+ if (ret == 0x0) {
+ DRM_DEBUG_KMS("No panel type in OpRegion\n");
+ return -ENODEV;
+ }
+
+ return ret - 1;
+}
diff --git a/drivers/gpu/drm/i915/intel_overlay.c b/drivers/gpu/drm/i915/intel_overlay.c
index 9168413..bd38e49 100644
--- a/drivers/gpu/drm/i915/intel_overlay.c
+++ b/drivers/gpu/drm/i915/intel_overlay.c
@@ -190,13 +190,14 @@
static struct overlay_registers __iomem *
intel_overlay_map_regs(struct intel_overlay *overlay)
{
- struct drm_i915_private *dev_priv = overlay->dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(overlay->dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct overlay_registers __iomem *regs;
if (OVERLAY_NEEDS_PHYSICAL(overlay->dev))
regs = (struct overlay_registers __iomem *)overlay->reg_bo->phys_handle->vaddr;
else
- regs = io_mapping_map_wc(dev_priv->gtt.mappable,
+ regs = io_mapping_map_wc(ggtt->mappable,
i915_gem_obj_ggtt_offset(overlay->reg_bo));
return regs;
@@ -233,30 +234,30 @@
{
struct drm_device *dev = overlay->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring = &dev_priv->ring[RCS];
+ struct intel_engine_cs *engine = &dev_priv->engine[RCS];
struct drm_i915_gem_request *req;
int ret;
WARN_ON(overlay->active);
WARN_ON(IS_I830(dev) && !(dev_priv->quirks & QUIRK_PIPEA_FORCE));
- req = i915_gem_request_alloc(ring, NULL);
+ req = i915_gem_request_alloc(engine, NULL);
if (IS_ERR(req))
return PTR_ERR(req);
ret = intel_ring_begin(req, 4);
if (ret) {
- i915_gem_request_cancel(req);
+ i915_add_request_no_flush(req);
return ret;
}
overlay->active = true;
- intel_ring_emit(ring, MI_OVERLAY_FLIP | MI_OVERLAY_ON);
- intel_ring_emit(ring, overlay->flip_addr | OFC_UPDATE);
- intel_ring_emit(ring, MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP);
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ intel_ring_emit(engine, MI_OVERLAY_FLIP | MI_OVERLAY_ON);
+ intel_ring_emit(engine, overlay->flip_addr | OFC_UPDATE);
+ intel_ring_emit(engine, MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP);
+ intel_ring_emit(engine, MI_NOOP);
+ intel_ring_advance(engine);
return intel_overlay_do_wait_request(overlay, req, NULL);
}
@@ -267,7 +268,7 @@
{
struct drm_device *dev = overlay->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring = &dev_priv->ring[RCS];
+ struct intel_engine_cs *engine = &dev_priv->engine[RCS];
struct drm_i915_gem_request *req;
u32 flip_addr = overlay->flip_addr;
u32 tmp;
@@ -283,19 +284,19 @@
if (tmp & (1 << 17))
DRM_DEBUG("overlay underrun, DOVSTA: %x\n", tmp);
- req = i915_gem_request_alloc(ring, NULL);
+ req = i915_gem_request_alloc(engine, NULL);
if (IS_ERR(req))
return PTR_ERR(req);
ret = intel_ring_begin(req, 2);
if (ret) {
- i915_gem_request_cancel(req);
+ i915_add_request_no_flush(req);
return ret;
}
- intel_ring_emit(ring, MI_OVERLAY_FLIP | MI_OVERLAY_CONTINUE);
- intel_ring_emit(ring, flip_addr);
- intel_ring_advance(ring);
+ intel_ring_emit(engine, MI_OVERLAY_FLIP | MI_OVERLAY_CONTINUE);
+ intel_ring_emit(engine, flip_addr);
+ intel_ring_advance(engine);
WARN_ON(overlay->last_flip_req);
i915_gem_request_assign(&overlay->last_flip_req, req);
@@ -336,7 +337,7 @@
{
struct drm_device *dev = overlay->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring = &dev_priv->ring[RCS];
+ struct intel_engine_cs *engine = &dev_priv->engine[RCS];
struct drm_i915_gem_request *req;
u32 flip_addr = overlay->flip_addr;
int ret;
@@ -349,33 +350,34 @@
* of the hw. Do it in both cases */
flip_addr |= OFC_UPDATE;
- req = i915_gem_request_alloc(ring, NULL);
+ req = i915_gem_request_alloc(engine, NULL);
if (IS_ERR(req))
return PTR_ERR(req);
ret = intel_ring_begin(req, 6);
if (ret) {
- i915_gem_request_cancel(req);
+ i915_add_request_no_flush(req);
return ret;
}
/* wait for overlay to go idle */
- intel_ring_emit(ring, MI_OVERLAY_FLIP | MI_OVERLAY_CONTINUE);
- intel_ring_emit(ring, flip_addr);
- intel_ring_emit(ring, MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP);
+ intel_ring_emit(engine, MI_OVERLAY_FLIP | MI_OVERLAY_CONTINUE);
+ intel_ring_emit(engine, flip_addr);
+ intel_ring_emit(engine, MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP);
/* turn overlay off */
if (IS_I830(dev)) {
/* Workaround: Don't disable the overlay fully, since otherwise
* it dies on the next OVERLAY_ON cmd. */
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_emit(ring, MI_NOOP);
+ intel_ring_emit(engine, MI_NOOP);
+ intel_ring_emit(engine, MI_NOOP);
+ intel_ring_emit(engine, MI_NOOP);
} else {
- intel_ring_emit(ring, MI_OVERLAY_FLIP | MI_OVERLAY_OFF);
- intel_ring_emit(ring, flip_addr);
- intel_ring_emit(ring, MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP);
+ intel_ring_emit(engine, MI_OVERLAY_FLIP | MI_OVERLAY_OFF);
+ intel_ring_emit(engine, flip_addr);
+ intel_ring_emit(engine,
+ MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP);
}
- intel_ring_advance(ring);
+ intel_ring_advance(engine);
return intel_overlay_do_wait_request(overlay, req, intel_overlay_off_tail);
}
@@ -408,7 +410,7 @@
{
struct drm_device *dev = overlay->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring = &dev_priv->ring[RCS];
+ struct intel_engine_cs *engine = &dev_priv->engine[RCS];
int ret;
WARN_ON(!mutex_is_locked(&dev->struct_mutex));
@@ -423,19 +425,20 @@
/* synchronous slowpath */
struct drm_i915_gem_request *req;
- req = i915_gem_request_alloc(ring, NULL);
+ req = i915_gem_request_alloc(engine, NULL);
if (IS_ERR(req))
return PTR_ERR(req);
ret = intel_ring_begin(req, 2);
if (ret) {
- i915_gem_request_cancel(req);
+ i915_add_request_no_flush(req);
return ret;
}
- intel_ring_emit(ring, MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP);
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ intel_ring_emit(engine,
+ MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP);
+ intel_ring_emit(engine, MI_NOOP);
+ intel_ring_advance(engine);
ret = intel_overlay_do_wait_request(overlay, req,
intel_overlay_release_old_vid_tail);
@@ -1124,7 +1127,7 @@
}
crtc = to_intel_crtc(drmmode_crtc);
- new_bo = to_intel_bo(drm_gem_object_lookup(dev, file_priv,
+ new_bo = to_intel_bo(drm_gem_object_lookup(file_priv,
put_image_rec->bo_handle));
if (&new_bo->base == NULL) {
ret = -ENOENT;
@@ -1479,7 +1482,8 @@
static struct overlay_registers __iomem *
intel_overlay_map_regs_atomic(struct intel_overlay *overlay)
{
- struct drm_i915_private *dev_priv = overlay->dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(overlay->dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct overlay_registers __iomem *regs;
if (OVERLAY_NEEDS_PHYSICAL(overlay->dev))
@@ -1488,7 +1492,7 @@
regs = (struct overlay_registers __iomem *)
overlay->reg_bo->phys_handle->vaddr;
else
- regs = io_mapping_map_atomic_wc(dev_priv->gtt.mappable,
+ regs = io_mapping_map_atomic_wc(ggtt->mappable,
i915_gem_obj_ggtt_offset(overlay->reg_bo));
return regs;
diff --git a/drivers/gpu/drm/i915/intel_panel.c b/drivers/gpu/drm/i915/intel_panel.c
index 21ee647..a078876 100644
--- a/drivers/gpu/drm/i915/intel_panel.c
+++ b/drivers/gpu/drm/i915/intel_panel.c
@@ -504,7 +504,7 @@
if (panel->backlight.combination_mode) {
u8 lbpc;
- pci_read_config_byte(dev_priv->dev->pdev, PCI_LBPC, &lbpc);
+ pci_read_config_byte(dev_priv->dev->pdev, LBPC, &lbpc);
val *= lbpc;
}
@@ -592,7 +592,7 @@
lbpc = level * 0xfe / panel->backlight.max + 1;
level /= lbpc;
- pci_write_config_byte(dev_priv->dev->pdev, PCI_LBPC, lbpc);
+ pci_write_config_byte(dev_priv->dev->pdev, LBPC, lbpc);
}
if (IS_GEN4(dev_priv)) {
@@ -1240,7 +1240,7 @@
*/
static u32 bxt_hz_to_pwm(struct intel_connector *connector, u32 pwm_freq_hz)
{
- return KHz(19200) / pwm_freq_hz;
+ return DIV_ROUND_CLOSEST(KHz(19200), pwm_freq_hz);
}
/*
@@ -1251,16 +1251,14 @@
static u32 spt_hz_to_pwm(struct intel_connector *connector, u32 pwm_freq_hz)
{
struct drm_i915_private *dev_priv = to_i915(connector->base.dev);
- u32 mul, clock;
+ u32 mul;
if (I915_READ(SOUTH_CHICKEN1) & SPT_PWM_GRANULARITY)
mul = 128;
else
mul = 16;
- clock = MHz(24);
-
- return clock / (pwm_freq_hz * mul);
+ return DIV_ROUND_CLOSEST(MHz(24), pwm_freq_hz * mul);
}
/*
@@ -1283,7 +1281,7 @@
else
clock = MHz(24); /* LPT:LP */
- return clock / (pwm_freq_hz * mul);
+ return DIV_ROUND_CLOSEST(clock, pwm_freq_hz * mul);
}
/*
@@ -1292,10 +1290,9 @@
*/
static u32 pch_hz_to_pwm(struct intel_connector *connector, u32 pwm_freq_hz)
{
- struct drm_device *dev = connector->base.dev;
- int clock = MHz(intel_pch_rawclk(dev));
+ struct drm_i915_private *dev_priv = to_i915(connector->base.dev);
- return clock / (pwm_freq_hz * 128);
+ return DIV_ROUND_CLOSEST(KHz(dev_priv->rawclk_freq), pwm_freq_hz * 128);
}
/*
@@ -1308,16 +1305,15 @@
*/
static u32 i9xx_hz_to_pwm(struct intel_connector *connector, u32 pwm_freq_hz)
{
- struct drm_device *dev = connector->base.dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(connector->base.dev);
int clock;
- if (IS_PINEVIEW(dev))
- clock = MHz(intel_hrawclk(dev));
+ if (IS_PINEVIEW(dev_priv))
+ clock = KHz(dev_priv->rawclk_freq);
else
- clock = 1000 * dev_priv->cdclk_freq;
+ clock = KHz(dev_priv->cdclk_freq);
- return clock / (pwm_freq_hz * 32);
+ return DIV_ROUND_CLOSEST(clock, pwm_freq_hz * 32);
}
/*
@@ -1332,11 +1328,11 @@
int clock;
if (IS_G4X(dev_priv))
- clock = MHz(intel_hrawclk(dev));
+ clock = KHz(dev_priv->rawclk_freq);
else
- clock = 1000 * dev_priv->cdclk_freq;
+ clock = KHz(dev_priv->cdclk_freq);
- return clock / (pwm_freq_hz * 128);
+ return DIV_ROUND_CLOSEST(clock, pwm_freq_hz * 128);
}
/*
@@ -1346,19 +1342,21 @@
*/
static u32 vlv_hz_to_pwm(struct intel_connector *connector, u32 pwm_freq_hz)
{
- struct drm_device *dev = connector->base.dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
- int clock;
+ struct drm_i915_private *dev_priv = to_i915(connector->base.dev);
+ int mul, clock;
if ((I915_READ(CBR1_VLV) & CBR_PWM_CLOCK_MUX_SELECT) == 0) {
- if (IS_CHERRYVIEW(dev))
- return KHz(19200) / (pwm_freq_hz * 16);
+ if (IS_CHERRYVIEW(dev_priv))
+ clock = KHz(19200);
else
- return MHz(25) / (pwm_freq_hz * 16);
+ clock = MHz(25);
+ mul = 16;
} else {
- clock = intel_hrawclk(dev);
- return MHz(clock) / (pwm_freq_hz * 128);
+ clock = KHz(dev_priv->rawclk_freq);
+ mul = 128;
}
+
+ return DIV_ROUND_CLOSEST(clock, pwm_freq_hz * mul);
}
static u32 get_backlight_max_vbt(struct intel_connector *connector)
@@ -1745,7 +1743,7 @@
panel->backlight.get = pch_get_backlight;
panel->backlight.hz_to_pwm = pch_hz_to_pwm;
} else if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) {
- if (dev_priv->vbt.has_mipi) {
+ if (connector->base.connector_type == DRM_MODE_CONNECTOR_DSI) {
panel->backlight.setup = pwm_setup_backlight;
panel->backlight.enable = pwm_enable_backlight;
panel->backlight.disable = pwm_disable_backlight;
diff --git a/drivers/gpu/drm/i915/intel_pm.c b/drivers/gpu/drm/i915/intel_pm.c
index 3425d8e..4b60005 100644
--- a/drivers/gpu/drm/i915/intel_pm.c
+++ b/drivers/gpu/drm/i915/intel_pm.c
@@ -487,20 +487,6 @@
.guard_size = 2,
.cacheline_size = G4X_FIFO_LINE_SIZE,
};
-static const struct intel_watermark_params valleyview_wm_info = {
- .fifo_size = VALLEYVIEW_FIFO_SIZE,
- .max_wm = VALLEYVIEW_MAX_WM,
- .default_wm = VALLEYVIEW_MAX_WM,
- .guard_size = 2,
- .cacheline_size = G4X_FIFO_LINE_SIZE,
-};
-static const struct intel_watermark_params valleyview_cursor_wm_info = {
- .fifo_size = I965_CURSOR_FIFO,
- .max_wm = VALLEYVIEW_CURSOR_MAX_WM,
- .default_wm = I965_CURSOR_DFT_WM,
- .guard_size = 2,
- .cacheline_size = G4X_FIFO_LINE_SIZE,
-};
static const struct intel_watermark_params i965_cursor_wm_info = {
.fifo_size = I965_CURSOR_FIFO,
.max_wm = I965_CURSOR_MAX_WM,
@@ -2010,11 +1996,18 @@
cur_latency *= 5;
}
- result->pri_val = ilk_compute_pri_wm(cstate, pristate,
- pri_latency, level);
- result->spr_val = ilk_compute_spr_wm(cstate, sprstate, spr_latency);
- result->cur_val = ilk_compute_cur_wm(cstate, curstate, cur_latency);
- result->fbc_val = ilk_compute_fbc_wm(cstate, pristate, result->pri_val);
+ if (pristate) {
+ result->pri_val = ilk_compute_pri_wm(cstate, pristate,
+ pri_latency, level);
+ result->fbc_val = ilk_compute_fbc_wm(cstate, pristate, result->pri_val);
+ }
+
+ if (sprstate)
+ result->spr_val = ilk_compute_spr_wm(cstate, sprstate, spr_latency);
+
+ if (curstate)
+ result->cur_val = ilk_compute_cur_wm(cstate, curstate, cur_latency);
+
result->enable = true;
}
@@ -2278,100 +2271,171 @@
intel_print_wm_latency(dev, "Gen9 Plane", dev_priv->wm.skl_latency);
}
-/* Compute new watermarks for the pipe */
-static int ilk_compute_pipe_wm(struct intel_crtc *intel_crtc,
- struct drm_atomic_state *state)
+static bool ilk_validate_pipe_wm(struct drm_device *dev,
+ struct intel_pipe_wm *pipe_wm)
{
- struct intel_pipe_wm *pipe_wm;
- struct drm_device *dev = intel_crtc->base.dev;
- const struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_crtc_state *cstate = NULL;
- struct intel_plane *intel_plane;
- struct drm_plane_state *ps;
- struct intel_plane_state *pristate = NULL;
- struct intel_plane_state *sprstate = NULL;
- struct intel_plane_state *curstate = NULL;
- int level, max_level = ilk_wm_max_level(dev);
/* LP0 watermark maximums depend on this pipe alone */
- struct intel_wm_config config = {
+ const struct intel_wm_config config = {
.num_pipes_active = 1,
+ .sprites_enabled = pipe_wm->sprites_enabled,
+ .sprites_scaled = pipe_wm->sprites_scaled,
};
struct ilk_wm_maximums max;
- cstate = intel_atomic_get_crtc_state(state, intel_crtc);
- if (IS_ERR(cstate))
- return PTR_ERR(cstate);
-
- pipe_wm = &cstate->wm.optimal.ilk;
- memset(pipe_wm, 0, sizeof(*pipe_wm));
-
- for_each_intel_plane_on_crtc(dev, intel_crtc, intel_plane) {
- ps = drm_atomic_get_plane_state(state,
- &intel_plane->base);
- if (IS_ERR(ps))
- return PTR_ERR(ps);
-
- if (intel_plane->base.type == DRM_PLANE_TYPE_PRIMARY)
- pristate = to_intel_plane_state(ps);
- else if (intel_plane->base.type == DRM_PLANE_TYPE_OVERLAY)
- sprstate = to_intel_plane_state(ps);
- else if (intel_plane->base.type == DRM_PLANE_TYPE_CURSOR)
- curstate = to_intel_plane_state(ps);
- }
-
- config.sprites_enabled = sprstate->visible;
- config.sprites_scaled = sprstate->visible &&
- (drm_rect_width(&sprstate->dst) != drm_rect_width(&sprstate->src) >> 16 ||
- drm_rect_height(&sprstate->dst) != drm_rect_height(&sprstate->src) >> 16);
-
- pipe_wm->pipe_enabled = cstate->base.active;
- pipe_wm->sprites_enabled = config.sprites_enabled;
- pipe_wm->sprites_scaled = config.sprites_scaled;
-
- /* ILK/SNB: LP2+ watermarks only w/o sprites */
- if (INTEL_INFO(dev)->gen <= 6 && sprstate->visible)
- max_level = 1;
-
- /* ILK/SNB/IVB: LP1+ watermarks only w/o scaling */
- if (config.sprites_scaled)
- max_level = 0;
-
- ilk_compute_wm_level(dev_priv, intel_crtc, 0, cstate,
- pristate, sprstate, curstate, &pipe_wm->wm[0]);
-
- if (IS_HASWELL(dev) || IS_BROADWELL(dev))
- pipe_wm->linetime = hsw_compute_linetime_wm(dev, cstate);
-
/* LP0 watermarks always use 1/2 DDB partitioning */
ilk_compute_wm_maximums(dev, 0, &config, INTEL_DDB_PART_1_2, &max);
/* At least LP0 must be valid */
- if (!ilk_validate_wm_level(0, &max, &pipe_wm->wm[0]))
+ if (!ilk_validate_wm_level(0, &max, &pipe_wm->wm[0])) {
+ DRM_DEBUG_KMS("LP0 watermark invalid\n");
+ return false;
+ }
+
+ return true;
+}
+
+/* Compute new watermarks for the pipe */
+static int ilk_compute_pipe_wm(struct intel_crtc_state *cstate)
+{
+ struct drm_atomic_state *state = cstate->base.state;
+ struct intel_crtc *intel_crtc = to_intel_crtc(cstate->base.crtc);
+ struct intel_pipe_wm *pipe_wm;
+ struct drm_device *dev = state->dev;
+ const struct drm_i915_private *dev_priv = dev->dev_private;
+ struct intel_plane *intel_plane;
+ struct intel_plane_state *pristate = NULL;
+ struct intel_plane_state *sprstate = NULL;
+ struct intel_plane_state *curstate = NULL;
+ int level, max_level = ilk_wm_max_level(dev), usable_level;
+ struct ilk_wm_maximums max;
+
+ pipe_wm = &cstate->wm.optimal.ilk;
+
+ for_each_intel_plane_on_crtc(dev, intel_crtc, intel_plane) {
+ struct intel_plane_state *ps;
+
+ ps = intel_atomic_get_existing_plane_state(state,
+ intel_plane);
+ if (!ps)
+ continue;
+
+ if (intel_plane->base.type == DRM_PLANE_TYPE_PRIMARY)
+ pristate = ps;
+ else if (intel_plane->base.type == DRM_PLANE_TYPE_OVERLAY)
+ sprstate = ps;
+ else if (intel_plane->base.type == DRM_PLANE_TYPE_CURSOR)
+ curstate = ps;
+ }
+
+ pipe_wm->pipe_enabled = cstate->base.active;
+ if (sprstate) {
+ pipe_wm->sprites_enabled = sprstate->visible;
+ pipe_wm->sprites_scaled = sprstate->visible &&
+ (drm_rect_width(&sprstate->dst) != drm_rect_width(&sprstate->src) >> 16 ||
+ drm_rect_height(&sprstate->dst) != drm_rect_height(&sprstate->src) >> 16);
+ }
+
+ usable_level = max_level;
+
+ /* ILK/SNB: LP2+ watermarks only w/o sprites */
+ if (INTEL_INFO(dev)->gen <= 6 && pipe_wm->sprites_enabled)
+ usable_level = 1;
+
+ /* ILK/SNB/IVB: LP1+ watermarks only w/o scaling */
+ if (pipe_wm->sprites_scaled)
+ usable_level = 0;
+
+ ilk_compute_wm_level(dev_priv, intel_crtc, 0, cstate,
+ pristate, sprstate, curstate, &pipe_wm->raw_wm[0]);
+
+ memset(&pipe_wm->wm, 0, sizeof(pipe_wm->wm));
+ pipe_wm->wm[0] = pipe_wm->raw_wm[0];
+
+ if (IS_HASWELL(dev) || IS_BROADWELL(dev))
+ pipe_wm->linetime = hsw_compute_linetime_wm(dev, cstate);
+
+ if (!ilk_validate_pipe_wm(dev, pipe_wm))
return -EINVAL;
ilk_compute_wm_reg_maximums(dev, 1, &max);
for (level = 1; level <= max_level; level++) {
- struct intel_wm_level wm = {};
+ struct intel_wm_level *wm = &pipe_wm->raw_wm[level];
ilk_compute_wm_level(dev_priv, intel_crtc, level, cstate,
- pristate, sprstate, curstate, &wm);
+ pristate, sprstate, curstate, wm);
/*
* Disable any watermark level that exceeds the
* register maximums since such watermarks are
* always invalid.
*/
- if (!ilk_validate_wm_level(level, &max, &wm))
- break;
+ if (level > usable_level)
+ continue;
- pipe_wm->wm[level] = wm;
+ if (ilk_validate_wm_level(level, &max, wm))
+ pipe_wm->wm[level] = *wm;
+ else
+ usable_level = level;
}
return 0;
}
/*
+ * Build a set of 'intermediate' watermark values that satisfy both the old
+ * state and the new state. These can be programmed to the hardware
+ * immediately.
+ */
+static int ilk_compute_intermediate_wm(struct drm_device *dev,
+ struct intel_crtc *intel_crtc,
+ struct intel_crtc_state *newstate)
+{
+ struct intel_pipe_wm *a = &newstate->wm.intermediate;
+ struct intel_pipe_wm *b = &intel_crtc->wm.active.ilk;
+ int level, max_level = ilk_wm_max_level(dev);
+
+ /*
+ * Start with the final, target watermarks, then combine with the
+ * currently active watermarks to get values that are safe both before
+ * and after the vblank.
+ */
+ *a = newstate->wm.optimal.ilk;
+ a->pipe_enabled |= b->pipe_enabled;
+ a->sprites_enabled |= b->sprites_enabled;
+ a->sprites_scaled |= b->sprites_scaled;
+
+ for (level = 0; level <= max_level; level++) {
+ struct intel_wm_level *a_wm = &a->wm[level];
+ const struct intel_wm_level *b_wm = &b->wm[level];
+
+ a_wm->enable &= b_wm->enable;
+ a_wm->pri_val = max(a_wm->pri_val, b_wm->pri_val);
+ a_wm->spr_val = max(a_wm->spr_val, b_wm->spr_val);
+ a_wm->cur_val = max(a_wm->cur_val, b_wm->cur_val);
+ a_wm->fbc_val = max(a_wm->fbc_val, b_wm->fbc_val);
+ }
+
+ /*
+ * We need to make sure that these merged watermark values are
+ * actually a valid configuration themselves. If they're not,
+ * there's no safe way to transition from the old state to
+ * the new state, so we need to fail the atomic transaction.
+ */
+ if (!ilk_validate_pipe_wm(dev, a))
+ return -EINVAL;
+
+ /*
+ * If our intermediate WM are identical to the final WM, then we can
+ * omit the post-vblank programming; only update if it's different.
+ */
+ if (memcmp(a, &newstate->wm.optimal.ilk, sizeof(*a)) == 0)
+ newstate->wm.need_postvbl_update = false;
+
+ return 0;
+}
+
+/*
* Merge the watermarks from all active pipes for a specific level.
*/
static void ilk_merge_wm_level(struct drm_device *dev,
@@ -2383,9 +2447,7 @@
ret_wm->enable = true;
for_each_intel_crtc(dev, intel_crtc) {
- const struct intel_crtc_state *cstate =
- to_intel_crtc_state(intel_crtc->base.state);
- const struct intel_pipe_wm *active = &cstate->wm.optimal.ilk;
+ const struct intel_pipe_wm *active = &intel_crtc->wm.active.ilk;
const struct intel_wm_level *wm = &active->wm[level];
if (!active->pipe_enabled)
@@ -2421,7 +2483,7 @@
/* ILK/SNB/IVB: LP1+ watermarks only w/ single pipe */
if ((INTEL_INFO(dev)->gen <= 6 || IS_IVYBRIDGE(dev)) &&
config->num_pipes_active > 1)
- return;
+ last_enabled_level = 0;
/* ILK: FBC WM must be disabled always */
merged->fbc_wm_enabled = INTEL_INFO(dev)->gen >= 6;
@@ -2533,15 +2595,14 @@
/* LP0 register values */
for_each_intel_crtc(dev, intel_crtc) {
- const struct intel_crtc_state *cstate =
- to_intel_crtc_state(intel_crtc->base.state);
enum pipe pipe = intel_crtc->pipe;
- const struct intel_wm_level *r = &cstate->wm.optimal.ilk.wm[0];
+ const struct intel_wm_level *r =
+ &intel_crtc->wm.active.ilk.wm[0];
if (WARN_ON(!r->enable))
continue;
- results->wm_linetime[pipe] = cstate->wm.optimal.ilk.linetime;
+ results->wm_linetime[pipe] = intel_crtc->wm.active.ilk.linetime;
results->wm_pipe[pipe] =
(r->pri_val << WM0_PIPE_PLANE_SHIFT) |
@@ -2748,7 +2809,7 @@
dev_priv->wm.hw = *results;
}
-static bool ilk_disable_lp_wm(struct drm_device *dev)
+bool ilk_disable_lp_wm(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
@@ -3657,11 +3718,9 @@
}
}
-static void ilk_program_watermarks(struct intel_crtc_state *cstate)
+static void ilk_program_watermarks(struct drm_i915_private *dev_priv)
{
- struct drm_crtc *crtc = cstate->base.crtc;
- struct drm_device *dev = crtc->dev;
- struct drm_i915_private *dev_priv = to_i915(dev);
+ struct drm_device *dev = dev_priv->dev;
struct intel_pipe_wm lp_wm_1_2 = {}, lp_wm_5_6 = {}, *best_lp_wm;
struct ilk_wm_maximums max;
struct intel_wm_config config = {};
@@ -3692,28 +3751,28 @@
ilk_write_wm_values(dev_priv, &results);
}
-static void ilk_update_wm(struct drm_crtc *crtc)
+static void ilk_initial_watermarks(struct intel_crtc_state *cstate)
{
- struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- struct intel_crtc_state *cstate = to_intel_crtc_state(crtc->state);
+ struct drm_i915_private *dev_priv = to_i915(cstate->base.crtc->dev);
+ struct intel_crtc *intel_crtc = to_intel_crtc(cstate->base.crtc);
- WARN_ON(cstate->base.active != intel_crtc->active);
+ mutex_lock(&dev_priv->wm.wm_mutex);
+ intel_crtc->wm.active.ilk = cstate->wm.intermediate;
+ ilk_program_watermarks(dev_priv);
+ mutex_unlock(&dev_priv->wm.wm_mutex);
+}
- /*
- * IVB workaround: must disable low power watermarks for at least
- * one frame before enabling scaling. LP watermarks can be re-enabled
- * when scaling is disabled.
- *
- * WaCxSRDisabledForSpriteScaling:ivb
- */
- if (cstate->disable_lp_wm) {
- ilk_disable_lp_wm(crtc->dev);
- intel_wait_for_vblank(crtc->dev, intel_crtc->pipe);
+static void ilk_optimize_watermarks(struct intel_crtc_state *cstate)
+{
+ struct drm_i915_private *dev_priv = to_i915(cstate->base.crtc->dev);
+ struct intel_crtc *intel_crtc = to_intel_crtc(cstate->base.crtc);
+
+ mutex_lock(&dev_priv->wm.wm_mutex);
+ if (cstate->wm.need_postvbl_update) {
+ intel_crtc->wm.active.ilk = cstate->wm.optimal.ilk;
+ ilk_program_watermarks(dev_priv);
}
-
- intel_crtc->wm.active.ilk = cstate->wm.optimal.ilk;
-
- ilk_program_watermarks(cstate);
+ mutex_unlock(&dev_priv->wm.wm_mutex);
}
static void skl_pipe_wm_active_state(uint32_t val,
@@ -4243,7 +4302,7 @@
* the hw runs at the minimal clock before selecting the desired
* frequency, if the down threshold expires in that window we will not
* receive a down interrupt. */
- if (IS_GEN9(dev_priv->dev)) {
+ if (IS_GEN9(dev_priv)) {
limits = (dev_priv->rps.max_freq_softlimit) << 23;
if (val <= dev_priv->rps.min_freq_softlimit)
limits |= (dev_priv->rps.min_freq_softlimit) << 14;
@@ -4528,7 +4587,7 @@
gen6_set_rps(dev, val);
}
-static void gen9_disable_rps(struct drm_device *dev)
+static void gen9_disable_rc6(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
@@ -4536,12 +4595,20 @@
I915_WRITE(GEN9_PG_ENABLE, 0);
}
+static void gen9_disable_rps(struct drm_device *dev)
+{
+ struct drm_i915_private *dev_priv = dev->dev_private;
+
+ I915_WRITE(GEN6_RP_CONTROL, 0);
+}
+
static void gen6_disable_rps(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
I915_WRITE(GEN6_RC_CONTROL, 0);
I915_WRITE(GEN6_RPNSWREQ, 1 << 31);
+ I915_WRITE(GEN6_RP_CONTROL, 0);
}
static void cherryview_disable_rps(struct drm_device *dev)
@@ -4585,7 +4652,8 @@
static bool bxt_check_bios_rc6_setup(const struct drm_device *dev)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
bool enable_rc6 = true;
unsigned long rc6_ctx_base;
@@ -4599,9 +4667,9 @@
* for this check.
*/
rc6_ctx_base = I915_READ(RC6_CTX_BASE) & RC6_CTX_BASE_MASK;
- if (!((rc6_ctx_base >= dev_priv->gtt.stolen_reserved_base) &&
- (rc6_ctx_base + PAGE_SIZE <= dev_priv->gtt.stolen_reserved_base +
- dev_priv->gtt.stolen_reserved_size))) {
+ if (!((rc6_ctx_base >= ggtt->stolen_reserved_base) &&
+ (rc6_ctx_base + PAGE_SIZE <= ggtt->stolen_reserved_base +
+ ggtt->stolen_reserved_size))) {
DRM_DEBUG_KMS("RC6 Base address not as expected.\n");
enable_rc6 = false;
}
@@ -4744,6 +4812,16 @@
/* WaGsvDisableTurbo: Workaround to disable turbo on BXT A* */
if (IS_BXT_REVID(dev, 0, BXT_REVID_A1)) {
+ /*
+ * BIOS could leave the Hw Turbo enabled, so need to explicitly
+ * clear out the Control register just to avoid inconsitency
+ * with debugfs interface, which will show Turbo as enabled
+ * only and that is not expected by the User after adding the
+ * WaGsvDisableTurbo. Apart from this there is no problem even
+ * if the Turbo is left enabled in the Control register, as the
+ * Up/Down interrupts would remain masked.
+ */
+ gen9_disable_rps(dev);
intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
return;
}
@@ -4762,7 +4840,7 @@
* Up/Down EI & threshold registers, as well as the RP_CONTROL,
* RP_INTERRUPT_LIMITS & RPNSWREQ registers */
dev_priv->rps.power = HIGH_POWER; /* force a reset */
- gen6_set_rps(dev_priv->dev, dev_priv->rps.min_freq_softlimit);
+ gen6_set_rps(dev_priv->dev, dev_priv->rps.idle_freq);
intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
}
@@ -4770,9 +4848,8 @@
static void gen9_enable_rc6(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
uint32_t rc6_mask = 0;
- int unused;
/* 1a: Software RC state - RC0 */
I915_WRITE(GEN6_RC_STATE, 0);
@@ -4793,8 +4870,8 @@
I915_WRITE(GEN6_RC6_WAKE_RATE_LIMIT, 54 << 16);
I915_WRITE(GEN6_RC_EVALUATION_INTERVAL, 125000); /* 12500 * 1280ns */
I915_WRITE(GEN6_RC_IDLE_HYSTERSIS, 25); /* 25 * 1280ns */
- for_each_ring(ring, dev_priv, unused)
- I915_WRITE(RING_MAX_IDLE(ring->mmio_base), 10);
+ for_each_engine(engine, dev_priv)
+ I915_WRITE(RING_MAX_IDLE(engine->mmio_base), 10);
if (HAS_GUC_UCODE(dev))
I915_WRITE(GUC_MAX_IDLE_COUNT, 0xA);
@@ -4840,9 +4917,8 @@
static void gen8_enable_rps(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
uint32_t rc6_mask = 0;
- int unused;
/* 1a: Software RC state - RC0 */
I915_WRITE(GEN6_RC_STATE, 0);
@@ -4861,8 +4937,8 @@
I915_WRITE(GEN6_RC6_WAKE_RATE_LIMIT, 40 << 16);
I915_WRITE(GEN6_RC_EVALUATION_INTERVAL, 125000); /* 12500 * 1280ns */
I915_WRITE(GEN6_RC_IDLE_HYSTERSIS, 25); /* 25 * 1280ns */
- for_each_ring(ring, dev_priv, unused)
- I915_WRITE(RING_MAX_IDLE(ring->mmio_base), 10);
+ for_each_engine(engine, dev_priv)
+ I915_WRITE(RING_MAX_IDLE(engine->mmio_base), 10);
I915_WRITE(GEN6_RC_SLEEP, 0);
if (IS_BROADWELL(dev))
I915_WRITE(GEN6_RC6_THRESHOLD, 625); /* 800us/1.28 for TO */
@@ -4922,11 +4998,11 @@
static void gen6_enable_rps(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
u32 rc6vids, pcu_mbox = 0, rc6_mask = 0;
u32 gtfifodbg;
int rc6_mode;
- int i, ret;
+ int ret;
WARN_ON(!mutex_is_locked(&dev_priv->rps.hw_lock));
@@ -4939,7 +5015,8 @@
I915_WRITE(GEN6_RC_STATE, 0);
/* Clear the DBG now so we don't confuse earlier errors */
- if ((gtfifodbg = I915_READ(GTFIFODBG))) {
+ gtfifodbg = I915_READ(GTFIFODBG);
+ if (gtfifodbg) {
DRM_ERROR("GT fifo had a previous error %x\n", gtfifodbg);
I915_WRITE(GTFIFODBG, gtfifodbg);
}
@@ -4958,8 +5035,8 @@
I915_WRITE(GEN6_RC_EVALUATION_INTERVAL, 125000);
I915_WRITE(GEN6_RC_IDLE_HYSTERSIS, 25);
- for_each_ring(ring, dev_priv, i)
- I915_WRITE(RING_MAX_IDLE(ring->mmio_base), 10);
+ for_each_engine(engine, dev_priv)
+ I915_WRITE(RING_MAX_IDLE(engine->mmio_base), 10);
I915_WRITE(GEN6_RC_SLEEP, 0);
I915_WRITE(GEN6_RC1e_THRESHOLD, 1000);
@@ -5244,9 +5321,9 @@
static void cherryview_setup_pctx(struct drm_device *dev)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
unsigned long pctx_paddr, paddr;
- struct i915_gtt *gtt = &dev_priv->gtt;
u32 pcbr;
int pctx_size = 32*1024;
@@ -5254,7 +5331,7 @@
if ((pcbr >> VLV_PCBR_ADDR_SHIFT) == 0) {
DRM_DEBUG_DRIVER("BIOS didn't set up PCBR, fixing up\n");
paddr = (dev_priv->mm.stolen_base +
- (gtt->stolen_size - pctx_size));
+ (ggtt->stolen_size - pctx_size));
pctx_paddr = (paddr & (~4095));
I915_WRITE(VLV_PCBR, pctx_paddr);
@@ -5322,6 +5399,17 @@
dev_priv->vlv_pctx = NULL;
}
+static void vlv_init_gpll_ref_freq(struct drm_i915_private *dev_priv)
+{
+ dev_priv->rps.gpll_ref_freq =
+ vlv_get_cck_clock(dev_priv, "GPLL ref",
+ CCK_GPLL_CLOCK_CONTROL,
+ dev_priv->czclk_freq);
+
+ DRM_DEBUG_DRIVER("GPLL reference freq: %d kHz\n",
+ dev_priv->rps.gpll_ref_freq);
+}
+
static void valleyview_init_gt_powersave(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
@@ -5329,6 +5417,8 @@
valleyview_setup_pctx(dev);
+ vlv_init_gpll_ref_freq(dev_priv);
+
mutex_lock(&dev_priv->rps.hw_lock);
val = vlv_punit_read(dev_priv, PUNIT_REG_GPU_FREQ_STS);
@@ -5386,6 +5476,8 @@
cherryview_setup_pctx(dev);
+ vlv_init_gpll_ref_freq(dev_priv);
+
mutex_lock(&dev_priv->rps.hw_lock);
mutex_lock(&dev_priv->sb_lock);
@@ -5450,13 +5542,13 @@
static void cherryview_enable_rps(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
u32 gtfifodbg, val, rc6_mode = 0, pcbr;
- int i;
WARN_ON(!mutex_is_locked(&dev_priv->rps.hw_lock));
- gtfifodbg = I915_READ(GTFIFODBG);
+ gtfifodbg = I915_READ(GTFIFODBG) & ~(GT_FIFO_SBDEDICATE_FREE_ENTRY_CHV |
+ GT_FIFO_FREE_ENTRIES_CHV);
if (gtfifodbg) {
DRM_DEBUG_DRIVER("GT fifo had a previous error %x\n",
gtfifodbg);
@@ -5477,8 +5569,8 @@
I915_WRITE(GEN6_RC_EVALUATION_INTERVAL, 125000); /* 12500 * 1280ns */
I915_WRITE(GEN6_RC_IDLE_HYSTERSIS, 25); /* 25 * 1280ns */
- for_each_ring(ring, dev_priv, i)
- I915_WRITE(RING_MAX_IDLE(ring->mmio_base), 10);
+ for_each_engine(engine, dev_priv)
+ I915_WRITE(RING_MAX_IDLE(engine->mmio_base), 10);
I915_WRITE(GEN6_RC_SLEEP, 0);
/* TO threshold set to 500 us ( 0x186 * 1.28 us) */
@@ -5537,10 +5629,10 @@
dev_priv->rps.cur_freq);
DRM_DEBUG_DRIVER("setting GPU freq to %d MHz (%u)\n",
- intel_gpu_freq(dev_priv, dev_priv->rps.efficient_freq),
- dev_priv->rps.efficient_freq);
+ intel_gpu_freq(dev_priv, dev_priv->rps.idle_freq),
+ dev_priv->rps.idle_freq);
- valleyview_set_rps(dev_priv->dev, dev_priv->rps.efficient_freq);
+ valleyview_set_rps(dev_priv->dev, dev_priv->rps.idle_freq);
intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
}
@@ -5548,15 +5640,15 @@
static void valleyview_enable_rps(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
u32 gtfifodbg, val, rc6_mode = 0;
- int i;
WARN_ON(!mutex_is_locked(&dev_priv->rps.hw_lock));
valleyview_check_pctx(dev_priv);
- if ((gtfifodbg = I915_READ(GTFIFODBG))) {
+ gtfifodbg = I915_READ(GTFIFODBG);
+ if (gtfifodbg) {
DRM_DEBUG_DRIVER("GT fifo had a previous error %x\n",
gtfifodbg);
I915_WRITE(GTFIFODBG, gtfifodbg);
@@ -5588,8 +5680,8 @@
I915_WRITE(GEN6_RC_EVALUATION_INTERVAL, 125000);
I915_WRITE(GEN6_RC_IDLE_HYSTERSIS, 25);
- for_each_ring(ring, dev_priv, i)
- I915_WRITE(RING_MAX_IDLE(ring->mmio_base), 10);
+ for_each_engine(engine, dev_priv)
+ I915_WRITE(RING_MAX_IDLE(engine->mmio_base), 10);
I915_WRITE(GEN6_RC6_THRESHOLD, 0x557);
@@ -5627,10 +5719,10 @@
dev_priv->rps.cur_freq);
DRM_DEBUG_DRIVER("setting GPU freq to %d MHz (%u)\n",
- intel_gpu_freq(dev_priv, dev_priv->rps.efficient_freq),
- dev_priv->rps.efficient_freq);
+ intel_gpu_freq(dev_priv, dev_priv->rps.idle_freq),
+ dev_priv->rps.idle_freq);
- valleyview_set_rps(dev_priv->dev, dev_priv->rps.efficient_freq);
+ valleyview_set_rps(dev_priv->dev, dev_priv->rps.idle_freq);
intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
}
@@ -5965,17 +6057,16 @@
bool i915_gpu_busy(void)
{
struct drm_i915_private *dev_priv;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
bool ret = false;
- int i;
spin_lock_irq(&mchdev_lock);
if (!i915_mch_dev)
goto out_unlock;
dev_priv = i915_mch_dev;
- for_each_ring(ring, dev_priv, i)
- ret |= !list_empty(&ring->request_list);
+ for_each_engine(engine, dev_priv)
+ ret |= !list_empty(&engine->request_list);
out_unlock:
spin_unlock_irq(&mchdev_lock);
@@ -6195,9 +6286,10 @@
intel_suspend_gt_powersave(dev);
mutex_lock(&dev_priv->rps.hw_lock);
- if (INTEL_INFO(dev)->gen >= 9)
+ if (INTEL_INFO(dev)->gen >= 9) {
+ gen9_disable_rc6(dev);
gen9_disable_rps(dev);
- else if (IS_CHERRYVIEW(dev))
+ } else if (IS_CHERRYVIEW(dev))
cherryview_disable_rps(dev);
else if (IS_VALLEYVIEW(dev))
valleyview_disable_rps(dev);
@@ -6818,23 +6910,10 @@
gen6_check_mch_setup(dev);
}
-static void vlv_init_display_clock_gating(struct drm_i915_private *dev_priv)
-{
- I915_WRITE(DSPCLK_GATE_D, VRHUNIT_CLOCK_GATE_DISABLE);
-
- /*
- * Disable trickle feed and enable pnd deadline calculation
- */
- I915_WRITE(MI_ARB_VLV, MI_ARB_DISPLAY_TRICKLE_FEED_DISABLE);
- I915_WRITE(CBR1_VLV, 0);
-}
-
static void valleyview_init_clock_gating(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- vlv_init_display_clock_gating(dev_priv);
-
/* WaDisableEarlyCull:vlv */
I915_WRITE(_3D_CHICKEN3,
_MASKED_BIT_ENABLE(_3D_CHICKEN_SF_DISABLE_OBJEND_CULL));
@@ -6917,8 +6996,6 @@
{
struct drm_i915_private *dev_priv = dev->dev_private;
- vlv_init_display_clock_gating(dev_priv);
-
/* WaVSRefCountFullforceMissDisable:chv */
/* WaDSRefCountFullforceMissDisable:chv */
I915_WRITE(GEN7_FF_THREAD_MODE,
@@ -7058,8 +7135,7 @@
{
struct drm_i915_private *dev_priv = dev->dev_private;
- if (dev_priv->display.init_clock_gating)
- dev_priv->display.init_clock_gating(dev);
+ dev_priv->display.init_clock_gating(dev);
}
void intel_suspend_hw(struct drm_device *dev)
@@ -7068,6 +7144,60 @@
lpt_suspend_hw(dev);
}
+static void nop_init_clock_gating(struct drm_device *dev)
+{
+ DRM_DEBUG_KMS("No clock gating settings or workarounds applied.\n");
+}
+
+/**
+ * intel_init_clock_gating_hooks - setup the clock gating hooks
+ * @dev_priv: device private
+ *
+ * Setup the hooks that configure which clocks of a given platform can be
+ * gated and also apply various GT and display specific workarounds for these
+ * platforms. Note that some GT specific workarounds are applied separately
+ * when GPU contexts or batchbuffers start their execution.
+ */
+void intel_init_clock_gating_hooks(struct drm_i915_private *dev_priv)
+{
+ if (IS_SKYLAKE(dev_priv))
+ dev_priv->display.init_clock_gating = nop_init_clock_gating;
+ else if (IS_KABYLAKE(dev_priv))
+ dev_priv->display.init_clock_gating = nop_init_clock_gating;
+ else if (IS_BROXTON(dev_priv))
+ dev_priv->display.init_clock_gating = bxt_init_clock_gating;
+ else if (IS_BROADWELL(dev_priv))
+ dev_priv->display.init_clock_gating = broadwell_init_clock_gating;
+ else if (IS_CHERRYVIEW(dev_priv))
+ dev_priv->display.init_clock_gating = cherryview_init_clock_gating;
+ else if (IS_HASWELL(dev_priv))
+ dev_priv->display.init_clock_gating = haswell_init_clock_gating;
+ else if (IS_IVYBRIDGE(dev_priv))
+ dev_priv->display.init_clock_gating = ivybridge_init_clock_gating;
+ else if (IS_VALLEYVIEW(dev_priv))
+ dev_priv->display.init_clock_gating = valleyview_init_clock_gating;
+ else if (IS_GEN6(dev_priv))
+ dev_priv->display.init_clock_gating = gen6_init_clock_gating;
+ else if (IS_GEN5(dev_priv))
+ dev_priv->display.init_clock_gating = ironlake_init_clock_gating;
+ else if (IS_G4X(dev_priv))
+ dev_priv->display.init_clock_gating = g4x_init_clock_gating;
+ else if (IS_CRESTLINE(dev_priv))
+ dev_priv->display.init_clock_gating = crestline_init_clock_gating;
+ else if (IS_BROADWATER(dev_priv))
+ dev_priv->display.init_clock_gating = broadwater_init_clock_gating;
+ else if (IS_GEN3(dev_priv))
+ dev_priv->display.init_clock_gating = gen3_init_clock_gating;
+ else if (IS_I85X(dev_priv) || IS_I865G(dev_priv))
+ dev_priv->display.init_clock_gating = i85x_init_clock_gating;
+ else if (IS_GEN2(dev_priv))
+ dev_priv->display.init_clock_gating = i830_init_clock_gating;
+ else {
+ MISSING_CASE(INTEL_DEVID(dev_priv));
+ dev_priv->display.init_clock_gating = nop_init_clock_gating;
+ }
+}
+
/* Set up chip specific power management-related functions */
void intel_init_pm(struct drm_device *dev)
{
@@ -7084,10 +7214,6 @@
/* For FIFO watermark updates */
if (INTEL_INFO(dev)->gen >= 9) {
skl_setup_wm_latency(dev);
-
- if (IS_BROXTON(dev))
- dev_priv->display.init_clock_gating =
- bxt_init_clock_gating;
dev_priv->display.update_wm = skl_update_wm;
} else if (HAS_PCH_SPLIT(dev)) {
ilk_setup_wm_latency(dev);
@@ -7096,36 +7222,23 @@
dev_priv->wm.spr_latency[1] && dev_priv->wm.cur_latency[1]) ||
(!IS_GEN5(dev) && dev_priv->wm.pri_latency[0] &&
dev_priv->wm.spr_latency[0] && dev_priv->wm.cur_latency[0])) {
- dev_priv->display.update_wm = ilk_update_wm;
dev_priv->display.compute_pipe_wm = ilk_compute_pipe_wm;
- dev_priv->display.program_watermarks = ilk_program_watermarks;
+ dev_priv->display.compute_intermediate_wm =
+ ilk_compute_intermediate_wm;
+ dev_priv->display.initial_watermarks =
+ ilk_initial_watermarks;
+ dev_priv->display.optimize_watermarks =
+ ilk_optimize_watermarks;
} else {
DRM_DEBUG_KMS("Failed to read display plane latency. "
"Disable CxSR\n");
}
-
- if (IS_GEN5(dev))
- dev_priv->display.init_clock_gating = ironlake_init_clock_gating;
- else if (IS_GEN6(dev))
- dev_priv->display.init_clock_gating = gen6_init_clock_gating;
- else if (IS_IVYBRIDGE(dev))
- dev_priv->display.init_clock_gating = ivybridge_init_clock_gating;
- else if (IS_HASWELL(dev))
- dev_priv->display.init_clock_gating = haswell_init_clock_gating;
- else if (INTEL_INFO(dev)->gen == 8)
- dev_priv->display.init_clock_gating = broadwell_init_clock_gating;
} else if (IS_CHERRYVIEW(dev)) {
vlv_setup_wm_latency(dev);
-
dev_priv->display.update_wm = vlv_update_wm;
- dev_priv->display.init_clock_gating =
- cherryview_init_clock_gating;
} else if (IS_VALLEYVIEW(dev)) {
vlv_setup_wm_latency(dev);
-
dev_priv->display.update_wm = vlv_update_wm;
- dev_priv->display.init_clock_gating =
- valleyview_init_clock_gating;
} else if (IS_PINEVIEW(dev)) {
if (!intel_get_cxsr_latency(IS_PINEVIEW_G(dev),
dev_priv->is_ddr3,
@@ -7141,20 +7254,13 @@
dev_priv->display.update_wm = NULL;
} else
dev_priv->display.update_wm = pineview_update_wm;
- dev_priv->display.init_clock_gating = gen3_init_clock_gating;
} else if (IS_G4X(dev)) {
dev_priv->display.update_wm = g4x_update_wm;
- dev_priv->display.init_clock_gating = g4x_init_clock_gating;
} else if (IS_GEN4(dev)) {
dev_priv->display.update_wm = i965_update_wm;
- if (IS_CRESTLINE(dev))
- dev_priv->display.init_clock_gating = crestline_init_clock_gating;
- else if (IS_BROADWATER(dev))
- dev_priv->display.init_clock_gating = broadwater_init_clock_gating;
} else if (IS_GEN3(dev)) {
dev_priv->display.update_wm = i9xx_update_wm;
dev_priv->display.get_fifo_size = i9xx_get_fifo_size;
- dev_priv->display.init_clock_gating = gen3_init_clock_gating;
} else if (IS_GEN2(dev)) {
if (INTEL_INFO(dev)->num_pipes == 1) {
dev_priv->display.update_wm = i845_update_wm;
@@ -7163,11 +7269,6 @@
dev_priv->display.update_wm = i9xx_update_wm;
dev_priv->display.get_fifo_size = i830_get_fifo_size;
}
-
- if (IS_I85X(dev) || IS_I865G(dev))
- dev_priv->display.init_clock_gating = i85x_init_clock_gating;
- else
- dev_priv->display.init_clock_gating = i830_init_clock_gating;
} else {
DRM_ERROR("unexpected fall-through in intel_init_pm\n");
}
@@ -7221,78 +7322,43 @@
return 0;
}
-static int vlv_gpu_freq_div(unsigned int czclk_freq)
-{
- switch (czclk_freq) {
- case 200:
- return 10;
- case 267:
- return 12;
- case 320:
- case 333:
- return 16;
- case 400:
- return 20;
- default:
- return -1;
- }
-}
-
static int byt_gpu_freq(struct drm_i915_private *dev_priv, int val)
{
- int div, czclk_freq = DIV_ROUND_CLOSEST(dev_priv->czclk_freq, 1000);
-
- div = vlv_gpu_freq_div(czclk_freq);
- if (div < 0)
- return div;
-
- return DIV_ROUND_CLOSEST(czclk_freq * (val + 6 - 0xbd), div);
+ /*
+ * N = val - 0xb7
+ * Slow = Fast = GPLL ref * N
+ */
+ return DIV_ROUND_CLOSEST(dev_priv->rps.gpll_ref_freq * (val - 0xb7), 1000);
}
static int byt_freq_opcode(struct drm_i915_private *dev_priv, int val)
{
- int mul, czclk_freq = DIV_ROUND_CLOSEST(dev_priv->czclk_freq, 1000);
-
- mul = vlv_gpu_freq_div(czclk_freq);
- if (mul < 0)
- return mul;
-
- return DIV_ROUND_CLOSEST(mul * val, czclk_freq) + 0xbd - 6;
+ return DIV_ROUND_CLOSEST(1000 * val, dev_priv->rps.gpll_ref_freq) + 0xb7;
}
static int chv_gpu_freq(struct drm_i915_private *dev_priv, int val)
{
- int div, czclk_freq = DIV_ROUND_CLOSEST(dev_priv->czclk_freq, 1000);
-
- div = vlv_gpu_freq_div(czclk_freq);
- if (div < 0)
- return div;
- div /= 2;
-
- return DIV_ROUND_CLOSEST(czclk_freq * val, 2 * div) / 2;
+ /*
+ * N = val / 2
+ * CU (slow) = CU2x (fast) / 2 = GPLL ref * N / 2
+ */
+ return DIV_ROUND_CLOSEST(dev_priv->rps.gpll_ref_freq * val, 2 * 2 * 1000);
}
static int chv_freq_opcode(struct drm_i915_private *dev_priv, int val)
{
- int mul, czclk_freq = DIV_ROUND_CLOSEST(dev_priv->czclk_freq, 1000);
-
- mul = vlv_gpu_freq_div(czclk_freq);
- if (mul < 0)
- return mul;
- mul /= 2;
-
/* CHV needs even values */
- return DIV_ROUND_CLOSEST(val * 2 * mul, czclk_freq) * 2;
+ return DIV_ROUND_CLOSEST(2 * 1000 * val, dev_priv->rps.gpll_ref_freq) * 2;
}
int intel_gpu_freq(struct drm_i915_private *dev_priv, int val)
{
- if (IS_GEN9(dev_priv->dev))
+ if (IS_GEN9(dev_priv))
return DIV_ROUND_CLOSEST(val * GT_FREQUENCY_MULTIPLIER,
GEN9_FREQ_SCALER);
- else if (IS_CHERRYVIEW(dev_priv->dev))
+ else if (IS_CHERRYVIEW(dev_priv))
return chv_gpu_freq(dev_priv, val);
- else if (IS_VALLEYVIEW(dev_priv->dev))
+ else if (IS_VALLEYVIEW(dev_priv))
return byt_gpu_freq(dev_priv, val);
else
return val * GT_FREQUENCY_MULTIPLIER;
@@ -7300,12 +7366,12 @@
int intel_freq_opcode(struct drm_i915_private *dev_priv, int val)
{
- if (IS_GEN9(dev_priv->dev))
+ if (IS_GEN9(dev_priv))
return DIV_ROUND_CLOSEST(val * GEN9_FREQ_SCALER,
GT_FREQUENCY_MULTIPLIER);
- else if (IS_CHERRYVIEW(dev_priv->dev))
+ else if (IS_CHERRYVIEW(dev_priv))
return chv_freq_opcode(dev_priv, val);
- else if (IS_VALLEYVIEW(dev_priv->dev))
+ else if (IS_VALLEYVIEW(dev_priv))
return byt_freq_opcode(dev_priv, val);
else
return DIV_ROUND_CLOSEST(val, GT_FREQUENCY_MULTIPLIER);
@@ -7322,7 +7388,7 @@
struct drm_i915_gem_request *req = boost->req;
if (!i915_gem_request_completed(req, true))
- gen6_rps_boost(to_i915(req->ring->dev), NULL,
+ gen6_rps_boost(to_i915(req->engine->dev), NULL,
req->emitted_jiffies);
i915_gem_request_unreference__unlocked(req);
diff --git a/drivers/gpu/drm/i915/intel_psr.c b/drivers/gpu/drm/i915/intel_psr.c
index 0b42ada..c3abae4 100644
--- a/drivers/gpu/drm/i915/intel_psr.c
+++ b/drivers/gpu/drm/i915/intel_psr.c
@@ -507,7 +507,8 @@
/* Wait till PSR is idle */
if (_wait_for((I915_READ(EDP_PSR_STATUS_CTL) &
- EDP_PSR_STATUS_STATE_MASK) == 0, 2000, 10))
+ EDP_PSR_STATUS_STATE_MASK) == 0,
+ 2 * USEC_PER_SEC, 10 * USEC_PER_MSEC))
DRM_ERROR("Timed out waiting for PSR Idle State\n");
dev_priv->psr.active = false;
@@ -562,7 +563,7 @@
* PSR might take some time to get fully disabled
* and be ready for re-enable.
*/
- if (HAS_DDI(dev_priv->dev)) {
+ if (HAS_DDI(dev_priv)) {
if (wait_for((I915_READ(EDP_PSR_STATUS_CTL) &
EDP_PSR_STATUS_STATE_MASK) == 0, 50)) {
DRM_ERROR("Timed out waiting for PSR Idle for re-enable\n");
@@ -780,8 +781,7 @@
/* Per platform default */
if (i915.enable_psr == -1) {
- if (IS_HASWELL(dev) || IS_BROADWELL(dev) ||
- IS_VALLEYVIEW(dev) || IS_CHERRYVIEW(dev))
+ if (IS_HASWELL(dev) || IS_BROADWELL(dev))
i915.enable_psr = 1;
else
i915.enable_psr = 0;
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 9121646..245386e 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -59,19 +59,19 @@
return ringbuf->space;
}
-bool intel_ring_stopped(struct intel_engine_cs *ring)
+bool intel_engine_stopped(struct intel_engine_cs *engine)
{
- struct drm_i915_private *dev_priv = ring->dev->dev_private;
- return dev_priv->gpu_error.stop_rings & intel_ring_flag(ring);
+ struct drm_i915_private *dev_priv = engine->dev->dev_private;
+ return dev_priv->gpu_error.stop_rings & intel_engine_flag(engine);
}
-static void __intel_ring_advance(struct intel_engine_cs *ring)
+static void __intel_ring_advance(struct intel_engine_cs *engine)
{
- struct intel_ringbuffer *ringbuf = ring->buffer;
+ struct intel_ringbuffer *ringbuf = engine->buffer;
ringbuf->tail &= ringbuf->size - 1;
- if (intel_ring_stopped(ring))
+ if (intel_engine_stopped(engine))
return;
- ring->write_tail(ring, ringbuf->tail);
+ engine->write_tail(engine, ringbuf->tail);
}
static int
@@ -79,7 +79,7 @@
u32 invalidate_domains,
u32 flush_domains)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
u32 cmd;
int ret;
@@ -94,9 +94,9 @@
if (ret)
return ret;
- intel_ring_emit(ring, cmd);
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ intel_ring_emit(engine, cmd);
+ intel_ring_emit(engine, MI_NOOP);
+ intel_ring_advance(engine);
return 0;
}
@@ -106,8 +106,8 @@
u32 invalidate_domains,
u32 flush_domains)
{
- struct intel_engine_cs *ring = req->ring;
- struct drm_device *dev = ring->dev;
+ struct intel_engine_cs *engine = req->engine;
+ struct drm_device *dev = engine->dev;
u32 cmd;
int ret;
@@ -153,9 +153,9 @@
if (ret)
return ret;
- intel_ring_emit(ring, cmd);
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ intel_ring_emit(engine, cmd);
+ intel_ring_emit(engine, MI_NOOP);
+ intel_ring_advance(engine);
return 0;
}
@@ -200,34 +200,34 @@
static int
intel_emit_post_sync_nonzero_flush(struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *ring = req->ring;
- u32 scratch_addr = ring->scratch.gtt_offset + 2 * CACHELINE_BYTES;
+ struct intel_engine_cs *engine = req->engine;
+ u32 scratch_addr = engine->scratch.gtt_offset + 2 * CACHELINE_BYTES;
int ret;
ret = intel_ring_begin(req, 6);
if (ret)
return ret;
- intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(5));
- intel_ring_emit(ring, PIPE_CONTROL_CS_STALL |
+ intel_ring_emit(engine, GFX_OP_PIPE_CONTROL(5));
+ intel_ring_emit(engine, PIPE_CONTROL_CS_STALL |
PIPE_CONTROL_STALL_AT_SCOREBOARD);
- intel_ring_emit(ring, scratch_addr | PIPE_CONTROL_GLOBAL_GTT); /* address */
- intel_ring_emit(ring, 0); /* low dword */
- intel_ring_emit(ring, 0); /* high dword */
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ intel_ring_emit(engine, scratch_addr | PIPE_CONTROL_GLOBAL_GTT); /* address */
+ intel_ring_emit(engine, 0); /* low dword */
+ intel_ring_emit(engine, 0); /* high dword */
+ intel_ring_emit(engine, MI_NOOP);
+ intel_ring_advance(engine);
ret = intel_ring_begin(req, 6);
if (ret)
return ret;
- intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(5));
- intel_ring_emit(ring, PIPE_CONTROL_QW_WRITE);
- intel_ring_emit(ring, scratch_addr | PIPE_CONTROL_GLOBAL_GTT); /* address */
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ intel_ring_emit(engine, GFX_OP_PIPE_CONTROL(5));
+ intel_ring_emit(engine, PIPE_CONTROL_QW_WRITE);
+ intel_ring_emit(engine, scratch_addr | PIPE_CONTROL_GLOBAL_GTT); /* address */
+ intel_ring_emit(engine, 0);
+ intel_ring_emit(engine, 0);
+ intel_ring_emit(engine, MI_NOOP);
+ intel_ring_advance(engine);
return 0;
}
@@ -236,9 +236,9 @@
gen6_render_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate_domains, u32 flush_domains)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
u32 flags = 0;
- u32 scratch_addr = ring->scratch.gtt_offset + 2 * CACHELINE_BYTES;
+ u32 scratch_addr = engine->scratch.gtt_offset + 2 * CACHELINE_BYTES;
int ret;
/* Force SNB workarounds for PIPE_CONTROL flushes */
@@ -276,11 +276,11 @@
if (ret)
return ret;
- intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(4));
- intel_ring_emit(ring, flags);
- intel_ring_emit(ring, scratch_addr | PIPE_CONTROL_GLOBAL_GTT);
- intel_ring_emit(ring, 0);
- intel_ring_advance(ring);
+ intel_ring_emit(engine, GFX_OP_PIPE_CONTROL(4));
+ intel_ring_emit(engine, flags);
+ intel_ring_emit(engine, scratch_addr | PIPE_CONTROL_GLOBAL_GTT);
+ intel_ring_emit(engine, 0);
+ intel_ring_advance(engine);
return 0;
}
@@ -288,19 +288,19 @@
static int
gen7_render_ring_cs_stall_wa(struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
int ret;
ret = intel_ring_begin(req, 4);
if (ret)
return ret;
- intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(4));
- intel_ring_emit(ring, PIPE_CONTROL_CS_STALL |
+ intel_ring_emit(engine, GFX_OP_PIPE_CONTROL(4));
+ intel_ring_emit(engine, PIPE_CONTROL_CS_STALL |
PIPE_CONTROL_STALL_AT_SCOREBOARD);
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, 0);
- intel_ring_advance(ring);
+ intel_ring_emit(engine, 0);
+ intel_ring_emit(engine, 0);
+ intel_ring_advance(engine);
return 0;
}
@@ -309,9 +309,9 @@
gen7_render_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate_domains, u32 flush_domains)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
u32 flags = 0;
- u32 scratch_addr = ring->scratch.gtt_offset + 2 * CACHELINE_BYTES;
+ u32 scratch_addr = engine->scratch.gtt_offset + 2 * CACHELINE_BYTES;
int ret;
/*
@@ -360,11 +360,11 @@
if (ret)
return ret;
- intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(4));
- intel_ring_emit(ring, flags);
- intel_ring_emit(ring, scratch_addr);
- intel_ring_emit(ring, 0);
- intel_ring_advance(ring);
+ intel_ring_emit(engine, GFX_OP_PIPE_CONTROL(4));
+ intel_ring_emit(engine, flags);
+ intel_ring_emit(engine, scratch_addr);
+ intel_ring_emit(engine, 0);
+ intel_ring_advance(engine);
return 0;
}
@@ -373,20 +373,20 @@
gen8_emit_pipe_control(struct drm_i915_gem_request *req,
u32 flags, u32 scratch_addr)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
int ret;
ret = intel_ring_begin(req, 6);
if (ret)
return ret;
- intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(6));
- intel_ring_emit(ring, flags);
- intel_ring_emit(ring, scratch_addr);
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, 0);
- intel_ring_advance(ring);
+ intel_ring_emit(engine, GFX_OP_PIPE_CONTROL(6));
+ intel_ring_emit(engine, flags);
+ intel_ring_emit(engine, scratch_addr);
+ intel_ring_emit(engine, 0);
+ intel_ring_emit(engine, 0);
+ intel_ring_emit(engine, 0);
+ intel_ring_advance(engine);
return 0;
}
@@ -396,7 +396,7 @@
u32 invalidate_domains, u32 flush_domains)
{
u32 flags = 0;
- u32 scratch_addr = req->ring->scratch.gtt_offset + 2 * CACHELINE_BYTES;
+ u32 scratch_addr = req->engine->scratch.gtt_offset + 2 * CACHELINE_BYTES;
int ret;
flags |= PIPE_CONTROL_CS_STALL;
@@ -429,51 +429,51 @@
return gen8_emit_pipe_control(req, flags, scratch_addr);
}
-static void ring_write_tail(struct intel_engine_cs *ring,
+static void ring_write_tail(struct intel_engine_cs *engine,
u32 value)
{
- struct drm_i915_private *dev_priv = ring->dev->dev_private;
- I915_WRITE_TAIL(ring, value);
+ struct drm_i915_private *dev_priv = engine->dev->dev_private;
+ I915_WRITE_TAIL(engine, value);
}
-u64 intel_ring_get_active_head(struct intel_engine_cs *ring)
+u64 intel_ring_get_active_head(struct intel_engine_cs *engine)
{
- struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct drm_i915_private *dev_priv = engine->dev->dev_private;
u64 acthd;
- if (INTEL_INFO(ring->dev)->gen >= 8)
- acthd = I915_READ64_2x32(RING_ACTHD(ring->mmio_base),
- RING_ACTHD_UDW(ring->mmio_base));
- else if (INTEL_INFO(ring->dev)->gen >= 4)
- acthd = I915_READ(RING_ACTHD(ring->mmio_base));
+ if (INTEL_INFO(engine->dev)->gen >= 8)
+ acthd = I915_READ64_2x32(RING_ACTHD(engine->mmio_base),
+ RING_ACTHD_UDW(engine->mmio_base));
+ else if (INTEL_INFO(engine->dev)->gen >= 4)
+ acthd = I915_READ(RING_ACTHD(engine->mmio_base));
else
acthd = I915_READ(ACTHD);
return acthd;
}
-static void ring_setup_phys_status_page(struct intel_engine_cs *ring)
+static void ring_setup_phys_status_page(struct intel_engine_cs *engine)
{
- struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct drm_i915_private *dev_priv = engine->dev->dev_private;
u32 addr;
addr = dev_priv->status_page_dmah->busaddr;
- if (INTEL_INFO(ring->dev)->gen >= 4)
+ if (INTEL_INFO(engine->dev)->gen >= 4)
addr |= (dev_priv->status_page_dmah->busaddr >> 28) & 0xf0;
I915_WRITE(HWS_PGA, addr);
}
-static void intel_ring_setup_status_page(struct intel_engine_cs *ring)
+static void intel_ring_setup_status_page(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
- struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct drm_device *dev = engine->dev;
+ struct drm_i915_private *dev_priv = engine->dev->dev_private;
i915_reg_t mmio;
/* The ring status page addresses are no longer next to the rest of
* the ring registers as of gen7.
*/
if (IS_GEN7(dev)) {
- switch (ring->id) {
+ switch (engine->id) {
case RCS:
mmio = RENDER_HWS_PGA_GEN7;
break;
@@ -492,14 +492,14 @@
mmio = VEBOX_HWS_PGA_GEN7;
break;
}
- } else if (IS_GEN6(ring->dev)) {
- mmio = RING_HWS_PGA_GEN6(ring->mmio_base);
+ } else if (IS_GEN6(engine->dev)) {
+ mmio = RING_HWS_PGA_GEN6(engine->mmio_base);
} else {
/* XXX: gen8 returns to sanity */
- mmio = RING_HWS_PGA(ring->mmio_base);
+ mmio = RING_HWS_PGA(engine->mmio_base);
}
- I915_WRITE(mmio, (u32)ring->status_page.gfx_addr);
+ I915_WRITE(mmio, (u32)engine->status_page.gfx_addr);
POSTING_READ(mmio);
/*
@@ -510,10 +510,10 @@
* invalidating the TLB?
*/
if (INTEL_INFO(dev)->gen >= 6 && INTEL_INFO(dev)->gen < 8) {
- i915_reg_t reg = RING_INSTPM(ring->mmio_base);
+ i915_reg_t reg = RING_INSTPM(engine->mmio_base);
/* ring should be idle before issuing a sync flush*/
- WARN_ON((I915_READ_MODE(ring) & MODE_IDLE) == 0);
+ WARN_ON((I915_READ_MODE(engine) & MODE_IDLE) == 0);
I915_WRITE(reg,
_MASKED_BIT_ENABLE(INSTPM_TLB_INVALIDATE |
@@ -521,117 +521,125 @@
if (wait_for((I915_READ(reg) & INSTPM_SYNC_FLUSH) == 0,
1000))
DRM_ERROR("%s: wait for SyncFlush to complete for TLB invalidation timed out\n",
- ring->name);
+ engine->name);
}
}
-static bool stop_ring(struct intel_engine_cs *ring)
+static bool stop_ring(struct intel_engine_cs *engine)
{
- struct drm_i915_private *dev_priv = to_i915(ring->dev);
+ struct drm_i915_private *dev_priv = to_i915(engine->dev);
- if (!IS_GEN2(ring->dev)) {
- I915_WRITE_MODE(ring, _MASKED_BIT_ENABLE(STOP_RING));
- if (wait_for((I915_READ_MODE(ring) & MODE_IDLE) != 0, 1000)) {
- DRM_ERROR("%s : timed out trying to stop ring\n", ring->name);
+ if (!IS_GEN2(engine->dev)) {
+ I915_WRITE_MODE(engine, _MASKED_BIT_ENABLE(STOP_RING));
+ if (wait_for((I915_READ_MODE(engine) & MODE_IDLE) != 0, 1000)) {
+ DRM_ERROR("%s : timed out trying to stop ring\n",
+ engine->name);
/* Sometimes we observe that the idle flag is not
* set even though the ring is empty. So double
* check before giving up.
*/
- if (I915_READ_HEAD(ring) != I915_READ_TAIL(ring))
+ if (I915_READ_HEAD(engine) != I915_READ_TAIL(engine))
return false;
}
}
- I915_WRITE_CTL(ring, 0);
- I915_WRITE_HEAD(ring, 0);
- ring->write_tail(ring, 0);
+ I915_WRITE_CTL(engine, 0);
+ I915_WRITE_HEAD(engine, 0);
+ engine->write_tail(engine, 0);
- if (!IS_GEN2(ring->dev)) {
- (void)I915_READ_CTL(ring);
- I915_WRITE_MODE(ring, _MASKED_BIT_DISABLE(STOP_RING));
+ if (!IS_GEN2(engine->dev)) {
+ (void)I915_READ_CTL(engine);
+ I915_WRITE_MODE(engine, _MASKED_BIT_DISABLE(STOP_RING));
}
- return (I915_READ_HEAD(ring) & HEAD_ADDR) == 0;
+ return (I915_READ_HEAD(engine) & HEAD_ADDR) == 0;
}
-static int init_ring_common(struct intel_engine_cs *ring)
+void intel_engine_init_hangcheck(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ memset(&engine->hangcheck, 0, sizeof(engine->hangcheck));
+}
+
+static int init_ring_common(struct intel_engine_cs *engine)
+{
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_ringbuffer *ringbuf = ring->buffer;
+ struct intel_ringbuffer *ringbuf = engine->buffer;
struct drm_i915_gem_object *obj = ringbuf->obj;
int ret = 0;
intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
- if (!stop_ring(ring)) {
+ if (!stop_ring(engine)) {
/* G45 ring initialization often fails to reset head to zero */
DRM_DEBUG_KMS("%s head not reset to zero "
"ctl %08x head %08x tail %08x start %08x\n",
- ring->name,
- I915_READ_CTL(ring),
- I915_READ_HEAD(ring),
- I915_READ_TAIL(ring),
- I915_READ_START(ring));
+ engine->name,
+ I915_READ_CTL(engine),
+ I915_READ_HEAD(engine),
+ I915_READ_TAIL(engine),
+ I915_READ_START(engine));
- if (!stop_ring(ring)) {
+ if (!stop_ring(engine)) {
DRM_ERROR("failed to set %s head to zero "
"ctl %08x head %08x tail %08x start %08x\n",
- ring->name,
- I915_READ_CTL(ring),
- I915_READ_HEAD(ring),
- I915_READ_TAIL(ring),
- I915_READ_START(ring));
+ engine->name,
+ I915_READ_CTL(engine),
+ I915_READ_HEAD(engine),
+ I915_READ_TAIL(engine),
+ I915_READ_START(engine));
ret = -EIO;
goto out;
}
}
if (I915_NEED_GFX_HWS(dev))
- intel_ring_setup_status_page(ring);
+ intel_ring_setup_status_page(engine);
else
- ring_setup_phys_status_page(ring);
+ ring_setup_phys_status_page(engine);
/* Enforce ordering by reading HEAD register back */
- I915_READ_HEAD(ring);
+ I915_READ_HEAD(engine);
/* Initialize the ring. This must happen _after_ we've cleared the ring
* registers with the above sequence (the readback of the HEAD registers
* also enforces ordering), otherwise the hw might lose the new ring
* register values. */
- I915_WRITE_START(ring, i915_gem_obj_ggtt_offset(obj));
+ I915_WRITE_START(engine, i915_gem_obj_ggtt_offset(obj));
/* WaClearRingBufHeadRegAtInit:ctg,elk */
- if (I915_READ_HEAD(ring))
+ if (I915_READ_HEAD(engine))
DRM_DEBUG("%s initialization failed [head=%08x], fudging\n",
- ring->name, I915_READ_HEAD(ring));
- I915_WRITE_HEAD(ring, 0);
- (void)I915_READ_HEAD(ring);
+ engine->name, I915_READ_HEAD(engine));
+ I915_WRITE_HEAD(engine, 0);
+ (void)I915_READ_HEAD(engine);
- I915_WRITE_CTL(ring,
+ I915_WRITE_CTL(engine,
((ringbuf->size - PAGE_SIZE) & RING_NR_PAGES)
| RING_VALID);
/* If the head is still not zero, the ring is dead */
- if (wait_for((I915_READ_CTL(ring) & RING_VALID) != 0 &&
- I915_READ_START(ring) == i915_gem_obj_ggtt_offset(obj) &&
- (I915_READ_HEAD(ring) & HEAD_ADDR) == 0, 50)) {
+ if (wait_for((I915_READ_CTL(engine) & RING_VALID) != 0 &&
+ I915_READ_START(engine) == i915_gem_obj_ggtt_offset(obj) &&
+ (I915_READ_HEAD(engine) & HEAD_ADDR) == 0, 50)) {
DRM_ERROR("%s initialization failed "
"ctl %08x (valid? %d) head %08x tail %08x start %08x [expected %08lx]\n",
- ring->name,
- I915_READ_CTL(ring), I915_READ_CTL(ring) & RING_VALID,
- I915_READ_HEAD(ring), I915_READ_TAIL(ring),
- I915_READ_START(ring), (unsigned long)i915_gem_obj_ggtt_offset(obj));
+ engine->name,
+ I915_READ_CTL(engine),
+ I915_READ_CTL(engine) & RING_VALID,
+ I915_READ_HEAD(engine), I915_READ_TAIL(engine),
+ I915_READ_START(engine),
+ (unsigned long)i915_gem_obj_ggtt_offset(obj));
ret = -EIO;
goto out;
}
ringbuf->last_retired_head = -1;
- ringbuf->head = I915_READ_HEAD(ring);
- ringbuf->tail = I915_READ_TAIL(ring) & TAIL_ADDR;
+ ringbuf->head = I915_READ_HEAD(engine);
+ ringbuf->tail = I915_READ_TAIL(engine) & TAIL_ADDR;
intel_ring_update_space(ringbuf);
- memset(&ring->hangcheck, 0, sizeof(ring->hangcheck));
+ intel_engine_init_hangcheck(engine);
out:
intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
@@ -640,59 +648,60 @@
}
void
-intel_fini_pipe_control(struct intel_engine_cs *ring)
+intel_fini_pipe_control(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
- if (ring->scratch.obj == NULL)
+ if (engine->scratch.obj == NULL)
return;
if (INTEL_INFO(dev)->gen >= 5) {
- kunmap(sg_page(ring->scratch.obj->pages->sgl));
- i915_gem_object_ggtt_unpin(ring->scratch.obj);
+ kunmap(sg_page(engine->scratch.obj->pages->sgl));
+ i915_gem_object_ggtt_unpin(engine->scratch.obj);
}
- drm_gem_object_unreference(&ring->scratch.obj->base);
- ring->scratch.obj = NULL;
+ drm_gem_object_unreference(&engine->scratch.obj->base);
+ engine->scratch.obj = NULL;
}
int
-intel_init_pipe_control(struct intel_engine_cs *ring)
+intel_init_pipe_control(struct intel_engine_cs *engine)
{
int ret;
- WARN_ON(ring->scratch.obj);
+ WARN_ON(engine->scratch.obj);
- ring->scratch.obj = i915_gem_alloc_object(ring->dev, 4096);
- if (ring->scratch.obj == NULL) {
+ engine->scratch.obj = i915_gem_alloc_object(engine->dev, 4096);
+ if (engine->scratch.obj == NULL) {
DRM_ERROR("Failed to allocate seqno page\n");
ret = -ENOMEM;
goto err;
}
- ret = i915_gem_object_set_cache_level(ring->scratch.obj, I915_CACHE_LLC);
+ ret = i915_gem_object_set_cache_level(engine->scratch.obj,
+ I915_CACHE_LLC);
if (ret)
goto err_unref;
- ret = i915_gem_obj_ggtt_pin(ring->scratch.obj, 4096, 0);
+ ret = i915_gem_obj_ggtt_pin(engine->scratch.obj, 4096, 0);
if (ret)
goto err_unref;
- ring->scratch.gtt_offset = i915_gem_obj_ggtt_offset(ring->scratch.obj);
- ring->scratch.cpu_page = kmap(sg_page(ring->scratch.obj->pages->sgl));
- if (ring->scratch.cpu_page == NULL) {
+ engine->scratch.gtt_offset = i915_gem_obj_ggtt_offset(engine->scratch.obj);
+ engine->scratch.cpu_page = kmap(sg_page(engine->scratch.obj->pages->sgl));
+ if (engine->scratch.cpu_page == NULL) {
ret = -ENOMEM;
goto err_unpin;
}
DRM_DEBUG_DRIVER("%s pipe control offset: 0x%08x\n",
- ring->name, ring->scratch.gtt_offset);
+ engine->name, engine->scratch.gtt_offset);
return 0;
err_unpin:
- i915_gem_object_ggtt_unpin(ring->scratch.obj);
+ i915_gem_object_ggtt_unpin(engine->scratch.obj);
err_unref:
- drm_gem_object_unreference(&ring->scratch.obj->base);
+ drm_gem_object_unreference(&engine->scratch.obj->base);
err:
return ret;
}
@@ -700,15 +709,15 @@
static int intel_ring_workarounds_emit(struct drm_i915_gem_request *req)
{
int ret, i;
- struct intel_engine_cs *ring = req->ring;
- struct drm_device *dev = ring->dev;
+ struct intel_engine_cs *engine = req->engine;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
struct i915_workarounds *w = &dev_priv->workarounds;
if (w->count == 0)
return 0;
- ring->gpu_caches_dirty = true;
+ engine->gpu_caches_dirty = true;
ret = intel_ring_flush_all_caches(req);
if (ret)
return ret;
@@ -717,16 +726,16 @@
if (ret)
return ret;
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(w->count));
+ intel_ring_emit(engine, MI_LOAD_REGISTER_IMM(w->count));
for (i = 0; i < w->count; i++) {
- intel_ring_emit_reg(ring, w->reg[i].addr);
- intel_ring_emit(ring, w->reg[i].value);
+ intel_ring_emit_reg(engine, w->reg[i].addr);
+ intel_ring_emit(engine, w->reg[i].value);
}
- intel_ring_emit(ring, MI_NOOP);
+ intel_ring_emit(engine, MI_NOOP);
- intel_ring_advance(ring);
+ intel_ring_advance(engine);
- ring->gpu_caches_dirty = true;
+ engine->gpu_caches_dirty = true;
ret = intel_ring_flush_all_caches(req);
if (ret)
return ret;
@@ -789,25 +798,26 @@
#define WA_WRITE(addr, val) WA_REG(addr, 0xffffffff, val)
-static int wa_ring_whitelist_reg(struct intel_engine_cs *ring, i915_reg_t reg)
+static int wa_ring_whitelist_reg(struct intel_engine_cs *engine,
+ i915_reg_t reg)
{
- struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct drm_i915_private *dev_priv = engine->dev->dev_private;
struct i915_workarounds *wa = &dev_priv->workarounds;
- const uint32_t index = wa->hw_whitelist_count[ring->id];
+ const uint32_t index = wa->hw_whitelist_count[engine->id];
if (WARN_ON(index >= RING_MAX_NONPRIV_SLOTS))
return -EINVAL;
- WA_WRITE(RING_FORCE_TO_NONPRIV(ring->mmio_base, index),
+ WA_WRITE(RING_FORCE_TO_NONPRIV(engine->mmio_base, index),
i915_mmio_reg_offset(reg));
- wa->hw_whitelist_count[ring->id]++;
+ wa->hw_whitelist_count[engine->id]++;
return 0;
}
-static int gen8_init_workarounds(struct intel_engine_cs *ring)
+static int gen8_init_workarounds(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
WA_SET_BIT_MASKED(INSTPM, INSTPM_FORCE_ORDERING);
@@ -857,13 +867,13 @@
return 0;
}
-static int bdw_init_workarounds(struct intel_engine_cs *ring)
+static int bdw_init_workarounds(struct intel_engine_cs *engine)
{
int ret;
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- ret = gen8_init_workarounds(ring);
+ ret = gen8_init_workarounds(engine);
if (ret)
return ret;
@@ -886,13 +896,13 @@
return 0;
}
-static int chv_init_workarounds(struct intel_engine_cs *ring)
+static int chv_init_workarounds(struct intel_engine_cs *engine)
{
int ret;
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- ret = gen8_init_workarounds(ring);
+ ret = gen8_init_workarounds(engine);
if (ret)
return ret;
@@ -905,9 +915,9 @@
return 0;
}
-static int gen9_init_workarounds(struct intel_engine_cs *ring)
+static int gen9_init_workarounds(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
uint32_t tmp;
int ret;
@@ -920,8 +930,10 @@
I915_WRITE(GAM_ECOCHK, I915_READ(GAM_ECOCHK) |
ECOCHK_DIS_TLB);
+ /* WaClearFlowControlGpgpuContextSave:skl,bxt */
/* WaDisablePartialInstShootdown:skl,bxt */
WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN,
+ FLOW_CONTROL_ENABLE |
PARTIAL_INSTRUCTION_SHOOTDOWN_DISABLE);
/* Syncing dependencies between camera and graphics:skl,bxt */
@@ -947,9 +959,10 @@
}
/* WaEnableYV12BugFixInHalfSliceChicken7:skl,bxt */
- if (IS_SKL_REVID(dev, SKL_REVID_C0, REVID_FOREVER) || IS_BROXTON(dev))
- WA_SET_BIT_MASKED(GEN9_HALF_SLICE_CHICKEN7,
- GEN9_ENABLE_YV12_BUGFIX);
+ /* WaEnableSamplerGPGPUPreemptionSupport:skl,bxt */
+ WA_SET_BIT_MASKED(GEN9_HALF_SLICE_CHICKEN7,
+ GEN9_ENABLE_YV12_BUGFIX |
+ GEN9_ENABLE_GPGPU_PREEMPTION);
/* Wa4x4STCOptimizationDisable:skl,bxt */
/* WaDisablePartialResolveInVc:skl,bxt */
@@ -986,21 +999,21 @@
GEN8_LQSC_FLUSH_COHERENT_LINES));
/* WaEnablePreemptionGranularityControlByUMD:skl,bxt */
- ret= wa_ring_whitelist_reg(ring, GEN8_CS_CHICKEN1);
+ ret= wa_ring_whitelist_reg(engine, GEN8_CS_CHICKEN1);
if (ret)
return ret;
/* WaAllowUMDToModifyHDCChicken1:skl,bxt */
- ret = wa_ring_whitelist_reg(ring, GEN8_HDC_CHICKEN1);
+ ret = wa_ring_whitelist_reg(engine, GEN8_HDC_CHICKEN1);
if (ret)
return ret;
return 0;
}
-static int skl_tune_iz_hashing(struct intel_engine_cs *ring)
+static int skl_tune_iz_hashing(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
u8 vals[3] = { 0, 0, 0 };
unsigned int i;
@@ -1040,13 +1053,13 @@
return 0;
}
-static int skl_init_workarounds(struct intel_engine_cs *ring)
+static int skl_init_workarounds(struct intel_engine_cs *engine)
{
int ret;
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- ret = gen9_init_workarounds(ring);
+ ret = gen9_init_workarounds(engine);
if (ret)
return ret;
@@ -1114,20 +1127,20 @@
GEN7_SBE_SS_CACHE_DISPATCH_PORT_SHARING_DISABLE);
/* WaDisableLSQCROPERFforOCL:skl */
- ret = wa_ring_whitelist_reg(ring, GEN8_L3SQCREG4);
+ ret = wa_ring_whitelist_reg(engine, GEN8_L3SQCREG4);
if (ret)
return ret;
- return skl_tune_iz_hashing(ring);
+ return skl_tune_iz_hashing(engine);
}
-static int bxt_init_workarounds(struct intel_engine_cs *ring)
+static int bxt_init_workarounds(struct intel_engine_cs *engine)
{
int ret;
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- ret = gen9_init_workarounds(ring);
+ ret = gen9_init_workarounds(engine);
if (ret)
return ret;
@@ -1158,11 +1171,11 @@
/* WaDisableObjectLevelPreemtionForInstanceId:bxt */
/* WaDisableLSQCROPERFforOCL:bxt */
if (IS_BXT_REVID(dev, 0, BXT_REVID_A1)) {
- ret = wa_ring_whitelist_reg(ring, GEN9_CS_DEBUG_MODE1);
+ ret = wa_ring_whitelist_reg(engine, GEN9_CS_DEBUG_MODE1);
if (ret)
return ret;
- ret = wa_ring_whitelist_reg(ring, GEN8_L3SQCREG4);
+ ret = wa_ring_whitelist_reg(engine, GEN8_L3SQCREG4);
if (ret)
return ret;
}
@@ -1170,36 +1183,36 @@
return 0;
}
-int init_workarounds_ring(struct intel_engine_cs *ring)
+int init_workarounds_ring(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- WARN_ON(ring->id != RCS);
+ WARN_ON(engine->id != RCS);
dev_priv->workarounds.count = 0;
dev_priv->workarounds.hw_whitelist_count[RCS] = 0;
if (IS_BROADWELL(dev))
- return bdw_init_workarounds(ring);
+ return bdw_init_workarounds(engine);
if (IS_CHERRYVIEW(dev))
- return chv_init_workarounds(ring);
+ return chv_init_workarounds(engine);
if (IS_SKYLAKE(dev))
- return skl_init_workarounds(ring);
+ return skl_init_workarounds(engine);
if (IS_BROXTON(dev))
- return bxt_init_workarounds(ring);
+ return bxt_init_workarounds(engine);
return 0;
}
-static int init_render_ring(struct intel_engine_cs *ring)
+static int init_render_ring(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- int ret = init_ring_common(ring);
+ int ret = init_ring_common(engine);
if (ret)
return ret;
@@ -1242,14 +1255,14 @@
I915_WRITE(INSTPM, _MASKED_BIT_ENABLE(INSTPM_FORCE_ORDERING));
if (HAS_L3_DPF(dev))
- I915_WRITE_IMR(ring, ~GT_PARITY_ERROR(dev));
+ I915_WRITE_IMR(engine, ~GT_PARITY_ERROR(dev));
- return init_workarounds_ring(ring);
+ return init_workarounds_ring(engine);
}
-static void render_ring_cleanup(struct intel_engine_cs *ring)
+static void render_ring_cleanup(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
if (dev_priv->semaphore_obj) {
@@ -1258,18 +1271,19 @@
dev_priv->semaphore_obj = NULL;
}
- intel_fini_pipe_control(ring);
+ intel_fini_pipe_control(engine);
}
static int gen8_rcs_signal(struct drm_i915_gem_request *signaller_req,
unsigned int num_dwords)
{
#define MBOX_UPDATE_DWORDS 8
- struct intel_engine_cs *signaller = signaller_req->ring;
+ struct intel_engine_cs *signaller = signaller_req->engine;
struct drm_device *dev = signaller->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
struct intel_engine_cs *waiter;
- int i, ret, num_rings;
+ enum intel_engine_id id;
+ int ret, num_rings;
num_rings = hweight32(INTEL_INFO(dev)->ring_mask);
num_dwords += (num_rings-1) * MBOX_UPDATE_DWORDS;
@@ -1279,9 +1293,9 @@
if (ret)
return ret;
- for_each_ring(waiter, dev_priv, i) {
+ for_each_engine_id(waiter, dev_priv, id) {
u32 seqno;
- u64 gtt_offset = signaller->semaphore.signal_ggtt[i];
+ u64 gtt_offset = signaller->semaphore.signal_ggtt[id];
if (gtt_offset == MI_SEMAPHORE_SYNC_INVALID)
continue;
@@ -1306,11 +1320,12 @@
unsigned int num_dwords)
{
#define MBOX_UPDATE_DWORDS 6
- struct intel_engine_cs *signaller = signaller_req->ring;
+ struct intel_engine_cs *signaller = signaller_req->engine;
struct drm_device *dev = signaller->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
struct intel_engine_cs *waiter;
- int i, ret, num_rings;
+ enum intel_engine_id id;
+ int ret, num_rings;
num_rings = hweight32(INTEL_INFO(dev)->ring_mask);
num_dwords += (num_rings-1) * MBOX_UPDATE_DWORDS;
@@ -1320,9 +1335,9 @@
if (ret)
return ret;
- for_each_ring(waiter, dev_priv, i) {
+ for_each_engine_id(waiter, dev_priv, id) {
u32 seqno;
- u64 gtt_offset = signaller->semaphore.signal_ggtt[i];
+ u64 gtt_offset = signaller->semaphore.signal_ggtt[id];
if (gtt_offset == MI_SEMAPHORE_SYNC_INVALID)
continue;
@@ -1344,11 +1359,12 @@
static int gen6_signal(struct drm_i915_gem_request *signaller_req,
unsigned int num_dwords)
{
- struct intel_engine_cs *signaller = signaller_req->ring;
+ struct intel_engine_cs *signaller = signaller_req->engine;
struct drm_device *dev = signaller->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
struct intel_engine_cs *useless;
- int i, ret, num_rings;
+ enum intel_engine_id id;
+ int ret, num_rings;
#define MBOX_UPDATE_DWORDS 3
num_rings = hweight32(INTEL_INFO(dev)->ring_mask);
@@ -1359,8 +1375,8 @@
if (ret)
return ret;
- for_each_ring(useless, dev_priv, i) {
- i915_reg_t mbox_reg = signaller->semaphore.mbox.signal[i];
+ for_each_engine_id(useless, dev_priv, id) {
+ i915_reg_t mbox_reg = signaller->semaphore.mbox.signal[id];
if (i915_mmio_reg_valid(mbox_reg)) {
u32 seqno = i915_gem_request_get_seqno(signaller_req);
@@ -1389,22 +1405,23 @@
static int
gen6_add_request(struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
int ret;
- if (ring->semaphore.signal)
- ret = ring->semaphore.signal(req, 4);
+ if (engine->semaphore.signal)
+ ret = engine->semaphore.signal(req, 4);
else
ret = intel_ring_begin(req, 4);
if (ret)
return ret;
- intel_ring_emit(ring, MI_STORE_DWORD_INDEX);
- intel_ring_emit(ring, I915_GEM_HWS_INDEX << MI_STORE_DWORD_INDEX_SHIFT);
- intel_ring_emit(ring, i915_gem_request_get_seqno(req));
- intel_ring_emit(ring, MI_USER_INTERRUPT);
- __intel_ring_advance(ring);
+ intel_ring_emit(engine, MI_STORE_DWORD_INDEX);
+ intel_ring_emit(engine,
+ I915_GEM_HWS_INDEX << MI_STORE_DWORD_INDEX_SHIFT);
+ intel_ring_emit(engine, i915_gem_request_get_seqno(req));
+ intel_ring_emit(engine, MI_USER_INTERRUPT);
+ __intel_ring_advance(engine);
return 0;
}
@@ -1429,7 +1446,7 @@
struct intel_engine_cs *signaller,
u32 seqno)
{
- struct intel_engine_cs *waiter = waiter_req->ring;
+ struct intel_engine_cs *waiter = waiter_req->engine;
struct drm_i915_private *dev_priv = waiter->dev->dev_private;
int ret;
@@ -1455,7 +1472,7 @@
struct intel_engine_cs *signaller,
u32 seqno)
{
- struct intel_engine_cs *waiter = waiter_req->ring;
+ struct intel_engine_cs *waiter = waiter_req->engine;
u32 dw1 = MI_SEMAPHORE_MBOX |
MI_SEMAPHORE_COMPARE |
MI_SEMAPHORE_REGISTER;
@@ -1503,8 +1520,8 @@
static int
pc_render_add_request(struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *ring = req->ring;
- u32 scratch_addr = ring->scratch.gtt_offset + 2 * CACHELINE_BYTES;
+ struct intel_engine_cs *engine = req->engine;
+ u32 scratch_addr = engine->scratch.gtt_offset + 2 * CACHELINE_BYTES;
int ret;
/* For Ironlake, MI_USER_INTERRUPT was deprecated and apparently
@@ -1519,78 +1536,87 @@
if (ret)
return ret;
- intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(4) | PIPE_CONTROL_QW_WRITE |
+ intel_ring_emit(engine,
+ GFX_OP_PIPE_CONTROL(4) | PIPE_CONTROL_QW_WRITE |
PIPE_CONTROL_WRITE_FLUSH |
PIPE_CONTROL_TEXTURE_CACHE_INVALIDATE);
- intel_ring_emit(ring, ring->scratch.gtt_offset | PIPE_CONTROL_GLOBAL_GTT);
- intel_ring_emit(ring, i915_gem_request_get_seqno(req));
- intel_ring_emit(ring, 0);
- PIPE_CONTROL_FLUSH(ring, scratch_addr);
+ intel_ring_emit(engine,
+ engine->scratch.gtt_offset | PIPE_CONTROL_GLOBAL_GTT);
+ intel_ring_emit(engine, i915_gem_request_get_seqno(req));
+ intel_ring_emit(engine, 0);
+ PIPE_CONTROL_FLUSH(engine, scratch_addr);
scratch_addr += 2 * CACHELINE_BYTES; /* write to separate cachelines */
- PIPE_CONTROL_FLUSH(ring, scratch_addr);
+ PIPE_CONTROL_FLUSH(engine, scratch_addr);
scratch_addr += 2 * CACHELINE_BYTES;
- PIPE_CONTROL_FLUSH(ring, scratch_addr);
+ PIPE_CONTROL_FLUSH(engine, scratch_addr);
scratch_addr += 2 * CACHELINE_BYTES;
- PIPE_CONTROL_FLUSH(ring, scratch_addr);
+ PIPE_CONTROL_FLUSH(engine, scratch_addr);
scratch_addr += 2 * CACHELINE_BYTES;
- PIPE_CONTROL_FLUSH(ring, scratch_addr);
+ PIPE_CONTROL_FLUSH(engine, scratch_addr);
scratch_addr += 2 * CACHELINE_BYTES;
- PIPE_CONTROL_FLUSH(ring, scratch_addr);
+ PIPE_CONTROL_FLUSH(engine, scratch_addr);
- intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(4) | PIPE_CONTROL_QW_WRITE |
+ intel_ring_emit(engine,
+ GFX_OP_PIPE_CONTROL(4) | PIPE_CONTROL_QW_WRITE |
PIPE_CONTROL_WRITE_FLUSH |
PIPE_CONTROL_TEXTURE_CACHE_INVALIDATE |
PIPE_CONTROL_NOTIFY);
- intel_ring_emit(ring, ring->scratch.gtt_offset | PIPE_CONTROL_GLOBAL_GTT);
- intel_ring_emit(ring, i915_gem_request_get_seqno(req));
- intel_ring_emit(ring, 0);
- __intel_ring_advance(ring);
+ intel_ring_emit(engine,
+ engine->scratch.gtt_offset | PIPE_CONTROL_GLOBAL_GTT);
+ intel_ring_emit(engine, i915_gem_request_get_seqno(req));
+ intel_ring_emit(engine, 0);
+ __intel_ring_advance(engine);
return 0;
}
-static u32
-gen6_ring_get_seqno(struct intel_engine_cs *ring, bool lazy_coherency)
+static void
+gen6_seqno_barrier(struct intel_engine_cs *engine)
{
/* Workaround to force correct ordering between irq and seqno writes on
* ivb (and maybe also on snb) by reading from a CS register (like
- * ACTHD) before reading the status page. */
- if (!lazy_coherency) {
- struct drm_i915_private *dev_priv = ring->dev->dev_private;
- POSTING_READ(RING_ACTHD(ring->mmio_base));
- }
-
- return intel_read_status_page(ring, I915_GEM_HWS_INDEX);
+ * ACTHD) before reading the status page.
+ *
+ * Note that this effectively stalls the read by the time it takes to
+ * do a memory transaction, which more or less ensures that the write
+ * from the GPU has sufficient time to invalidate the CPU cacheline.
+ * Alternatively we could delay the interrupt from the CS ring to give
+ * the write time to land, but that would incur a delay after every
+ * batch i.e. much more frequent than a delay when waiting for the
+ * interrupt (with the same net latency).
+ */
+ struct drm_i915_private *dev_priv = engine->dev->dev_private;
+ POSTING_READ_FW(RING_ACTHD(engine->mmio_base));
}
static u32
-ring_get_seqno(struct intel_engine_cs *ring, bool lazy_coherency)
+ring_get_seqno(struct intel_engine_cs *engine)
{
- return intel_read_status_page(ring, I915_GEM_HWS_INDEX);
+ return intel_read_status_page(engine, I915_GEM_HWS_INDEX);
}
static void
-ring_set_seqno(struct intel_engine_cs *ring, u32 seqno)
+ring_set_seqno(struct intel_engine_cs *engine, u32 seqno)
{
- intel_write_status_page(ring, I915_GEM_HWS_INDEX, seqno);
+ intel_write_status_page(engine, I915_GEM_HWS_INDEX, seqno);
}
static u32
-pc_render_get_seqno(struct intel_engine_cs *ring, bool lazy_coherency)
+pc_render_get_seqno(struct intel_engine_cs *engine)
{
- return ring->scratch.cpu_page[0];
+ return engine->scratch.cpu_page[0];
}
static void
-pc_render_set_seqno(struct intel_engine_cs *ring, u32 seqno)
+pc_render_set_seqno(struct intel_engine_cs *engine, u32 seqno)
{
- ring->scratch.cpu_page[0] = seqno;
+ engine->scratch.cpu_page[0] = seqno;
}
static bool
-gen5_ring_get_irq(struct intel_engine_cs *ring)
+gen5_ring_get_irq(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
unsigned long flags;
@@ -1598,30 +1624,30 @@
return false;
spin_lock_irqsave(&dev_priv->irq_lock, flags);
- if (ring->irq_refcount++ == 0)
- gen5_enable_gt_irq(dev_priv, ring->irq_enable_mask);
+ if (engine->irq_refcount++ == 0)
+ gen5_enable_gt_irq(dev_priv, engine->irq_enable_mask);
spin_unlock_irqrestore(&dev_priv->irq_lock, flags);
return true;
}
static void
-gen5_ring_put_irq(struct intel_engine_cs *ring)
+gen5_ring_put_irq(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
unsigned long flags;
spin_lock_irqsave(&dev_priv->irq_lock, flags);
- if (--ring->irq_refcount == 0)
- gen5_disable_gt_irq(dev_priv, ring->irq_enable_mask);
+ if (--engine->irq_refcount == 0)
+ gen5_disable_gt_irq(dev_priv, engine->irq_enable_mask);
spin_unlock_irqrestore(&dev_priv->irq_lock, flags);
}
static bool
-i9xx_ring_get_irq(struct intel_engine_cs *ring)
+i9xx_ring_get_irq(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
unsigned long flags;
@@ -1629,8 +1655,8 @@
return false;
spin_lock_irqsave(&dev_priv->irq_lock, flags);
- if (ring->irq_refcount++ == 0) {
- dev_priv->irq_mask &= ~ring->irq_enable_mask;
+ if (engine->irq_refcount++ == 0) {
+ dev_priv->irq_mask &= ~engine->irq_enable_mask;
I915_WRITE(IMR, dev_priv->irq_mask);
POSTING_READ(IMR);
}
@@ -1640,15 +1666,15 @@
}
static void
-i9xx_ring_put_irq(struct intel_engine_cs *ring)
+i9xx_ring_put_irq(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
unsigned long flags;
spin_lock_irqsave(&dev_priv->irq_lock, flags);
- if (--ring->irq_refcount == 0) {
- dev_priv->irq_mask |= ring->irq_enable_mask;
+ if (--engine->irq_refcount == 0) {
+ dev_priv->irq_mask |= engine->irq_enable_mask;
I915_WRITE(IMR, dev_priv->irq_mask);
POSTING_READ(IMR);
}
@@ -1656,9 +1682,9 @@
}
static bool
-i8xx_ring_get_irq(struct intel_engine_cs *ring)
+i8xx_ring_get_irq(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
unsigned long flags;
@@ -1666,8 +1692,8 @@
return false;
spin_lock_irqsave(&dev_priv->irq_lock, flags);
- if (ring->irq_refcount++ == 0) {
- dev_priv->irq_mask &= ~ring->irq_enable_mask;
+ if (engine->irq_refcount++ == 0) {
+ dev_priv->irq_mask &= ~engine->irq_enable_mask;
I915_WRITE16(IMR, dev_priv->irq_mask);
POSTING_READ16(IMR);
}
@@ -1677,15 +1703,15 @@
}
static void
-i8xx_ring_put_irq(struct intel_engine_cs *ring)
+i8xx_ring_put_irq(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
unsigned long flags;
spin_lock_irqsave(&dev_priv->irq_lock, flags);
- if (--ring->irq_refcount == 0) {
- dev_priv->irq_mask |= ring->irq_enable_mask;
+ if (--engine->irq_refcount == 0) {
+ dev_priv->irq_mask |= engine->irq_enable_mask;
I915_WRITE16(IMR, dev_priv->irq_mask);
POSTING_READ16(IMR);
}
@@ -1697,42 +1723,43 @@
u32 invalidate_domains,
u32 flush_domains)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
int ret;
ret = intel_ring_begin(req, 2);
if (ret)
return ret;
- intel_ring_emit(ring, MI_FLUSH);
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ intel_ring_emit(engine, MI_FLUSH);
+ intel_ring_emit(engine, MI_NOOP);
+ intel_ring_advance(engine);
return 0;
}
static int
i9xx_add_request(struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
int ret;
ret = intel_ring_begin(req, 4);
if (ret)
return ret;
- intel_ring_emit(ring, MI_STORE_DWORD_INDEX);
- intel_ring_emit(ring, I915_GEM_HWS_INDEX << MI_STORE_DWORD_INDEX_SHIFT);
- intel_ring_emit(ring, i915_gem_request_get_seqno(req));
- intel_ring_emit(ring, MI_USER_INTERRUPT);
- __intel_ring_advance(ring);
+ intel_ring_emit(engine, MI_STORE_DWORD_INDEX);
+ intel_ring_emit(engine,
+ I915_GEM_HWS_INDEX << MI_STORE_DWORD_INDEX_SHIFT);
+ intel_ring_emit(engine, i915_gem_request_get_seqno(req));
+ intel_ring_emit(engine, MI_USER_INTERRUPT);
+ __intel_ring_advance(engine);
return 0;
}
static bool
-gen6_ring_get_irq(struct intel_engine_cs *ring)
+gen6_ring_get_irq(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
unsigned long flags;
@@ -1740,14 +1767,14 @@
return false;
spin_lock_irqsave(&dev_priv->irq_lock, flags);
- if (ring->irq_refcount++ == 0) {
- if (HAS_L3_DPF(dev) && ring->id == RCS)
- I915_WRITE_IMR(ring,
- ~(ring->irq_enable_mask |
+ if (engine->irq_refcount++ == 0) {
+ if (HAS_L3_DPF(dev) && engine->id == RCS)
+ I915_WRITE_IMR(engine,
+ ~(engine->irq_enable_mask |
GT_PARITY_ERROR(dev)));
else
- I915_WRITE_IMR(ring, ~ring->irq_enable_mask);
- gen5_enable_gt_irq(dev_priv, ring->irq_enable_mask);
+ I915_WRITE_IMR(engine, ~engine->irq_enable_mask);
+ gen5_enable_gt_irq(dev_priv, engine->irq_enable_mask);
}
spin_unlock_irqrestore(&dev_priv->irq_lock, flags);
@@ -1755,27 +1782,27 @@
}
static void
-gen6_ring_put_irq(struct intel_engine_cs *ring)
+gen6_ring_put_irq(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
unsigned long flags;
spin_lock_irqsave(&dev_priv->irq_lock, flags);
- if (--ring->irq_refcount == 0) {
- if (HAS_L3_DPF(dev) && ring->id == RCS)
- I915_WRITE_IMR(ring, ~GT_PARITY_ERROR(dev));
+ if (--engine->irq_refcount == 0) {
+ if (HAS_L3_DPF(dev) && engine->id == RCS)
+ I915_WRITE_IMR(engine, ~GT_PARITY_ERROR(dev));
else
- I915_WRITE_IMR(ring, ~0);
- gen5_disable_gt_irq(dev_priv, ring->irq_enable_mask);
+ I915_WRITE_IMR(engine, ~0);
+ gen5_disable_gt_irq(dev_priv, engine->irq_enable_mask);
}
spin_unlock_irqrestore(&dev_priv->irq_lock, flags);
}
static bool
-hsw_vebox_get_irq(struct intel_engine_cs *ring)
+hsw_vebox_get_irq(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
unsigned long flags;
@@ -1783,9 +1810,9 @@
return false;
spin_lock_irqsave(&dev_priv->irq_lock, flags);
- if (ring->irq_refcount++ == 0) {
- I915_WRITE_IMR(ring, ~ring->irq_enable_mask);
- gen6_enable_pm_irq(dev_priv, ring->irq_enable_mask);
+ if (engine->irq_refcount++ == 0) {
+ I915_WRITE_IMR(engine, ~engine->irq_enable_mask);
+ gen6_enable_pm_irq(dev_priv, engine->irq_enable_mask);
}
spin_unlock_irqrestore(&dev_priv->irq_lock, flags);
@@ -1793,24 +1820,24 @@
}
static void
-hsw_vebox_put_irq(struct intel_engine_cs *ring)
+hsw_vebox_put_irq(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
unsigned long flags;
spin_lock_irqsave(&dev_priv->irq_lock, flags);
- if (--ring->irq_refcount == 0) {
- I915_WRITE_IMR(ring, ~0);
- gen6_disable_pm_irq(dev_priv, ring->irq_enable_mask);
+ if (--engine->irq_refcount == 0) {
+ I915_WRITE_IMR(engine, ~0);
+ gen6_disable_pm_irq(dev_priv, engine->irq_enable_mask);
}
spin_unlock_irqrestore(&dev_priv->irq_lock, flags);
}
static bool
-gen8_ring_get_irq(struct intel_engine_cs *ring)
+gen8_ring_get_irq(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
unsigned long flags;
@@ -1818,15 +1845,15 @@
return false;
spin_lock_irqsave(&dev_priv->irq_lock, flags);
- if (ring->irq_refcount++ == 0) {
- if (HAS_L3_DPF(dev) && ring->id == RCS) {
- I915_WRITE_IMR(ring,
- ~(ring->irq_enable_mask |
+ if (engine->irq_refcount++ == 0) {
+ if (HAS_L3_DPF(dev) && engine->id == RCS) {
+ I915_WRITE_IMR(engine,
+ ~(engine->irq_enable_mask |
GT_RENDER_L3_PARITY_ERROR_INTERRUPT));
} else {
- I915_WRITE_IMR(ring, ~ring->irq_enable_mask);
+ I915_WRITE_IMR(engine, ~engine->irq_enable_mask);
}
- POSTING_READ(RING_IMR(ring->mmio_base));
+ POSTING_READ(RING_IMR(engine->mmio_base));
}
spin_unlock_irqrestore(&dev_priv->irq_lock, flags);
@@ -1834,21 +1861,21 @@
}
static void
-gen8_ring_put_irq(struct intel_engine_cs *ring)
+gen8_ring_put_irq(struct intel_engine_cs *engine)
{
- struct drm_device *dev = ring->dev;
+ struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
unsigned long flags;
spin_lock_irqsave(&dev_priv->irq_lock, flags);
- if (--ring->irq_refcount == 0) {
- if (HAS_L3_DPF(dev) && ring->id == RCS) {
- I915_WRITE_IMR(ring,
+ if (--engine->irq_refcount == 0) {
+ if (HAS_L3_DPF(dev) && engine->id == RCS) {
+ I915_WRITE_IMR(engine,
~GT_RENDER_L3_PARITY_ERROR_INTERRUPT);
} else {
- I915_WRITE_IMR(ring, ~0);
+ I915_WRITE_IMR(engine, ~0);
}
- POSTING_READ(RING_IMR(ring->mmio_base));
+ POSTING_READ(RING_IMR(engine->mmio_base));
}
spin_unlock_irqrestore(&dev_priv->irq_lock, flags);
}
@@ -1858,20 +1885,20 @@
u64 offset, u32 length,
unsigned dispatch_flags)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
int ret;
ret = intel_ring_begin(req, 2);
if (ret)
return ret;
- intel_ring_emit(ring,
+ intel_ring_emit(engine,
MI_BATCH_BUFFER_START |
MI_BATCH_GTT |
(dispatch_flags & I915_DISPATCH_SECURE ?
0 : MI_BATCH_NON_SECURE_I965));
- intel_ring_emit(ring, offset);
- intel_ring_advance(ring);
+ intel_ring_emit(engine, offset);
+ intel_ring_advance(engine);
return 0;
}
@@ -1885,8 +1912,8 @@
u64 offset, u32 len,
unsigned dispatch_flags)
{
- struct intel_engine_cs *ring = req->ring;
- u32 cs_offset = ring->scratch.gtt_offset;
+ struct intel_engine_cs *engine = req->engine;
+ u32 cs_offset = engine->scratch.gtt_offset;
int ret;
ret = intel_ring_begin(req, 6);
@@ -1894,13 +1921,13 @@
return ret;
/* Evict the invalid PTE TLBs */
- intel_ring_emit(ring, COLOR_BLT_CMD | BLT_WRITE_RGBA);
- intel_ring_emit(ring, BLT_DEPTH_32 | BLT_ROP_COLOR_COPY | 4096);
- intel_ring_emit(ring, I830_TLB_ENTRIES << 16 | 4); /* load each page */
- intel_ring_emit(ring, cs_offset);
- intel_ring_emit(ring, 0xdeadbeef);
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ intel_ring_emit(engine, COLOR_BLT_CMD | BLT_WRITE_RGBA);
+ intel_ring_emit(engine, BLT_DEPTH_32 | BLT_ROP_COLOR_COPY | 4096);
+ intel_ring_emit(engine, I830_TLB_ENTRIES << 16 | 4); /* load each page */
+ intel_ring_emit(engine, cs_offset);
+ intel_ring_emit(engine, 0xdeadbeef);
+ intel_ring_emit(engine, MI_NOOP);
+ intel_ring_advance(engine);
if ((dispatch_flags & I915_DISPATCH_PINNED) == 0) {
if (len > I830_BATCH_LIMIT)
@@ -1914,16 +1941,17 @@
* stable batch scratch bo area (so that the CS never
* stumbles over its tlb invalidation bug) ...
*/
- intel_ring_emit(ring, SRC_COPY_BLT_CMD | BLT_WRITE_RGBA);
- intel_ring_emit(ring, BLT_DEPTH_32 | BLT_ROP_SRC_COPY | 4096);
- intel_ring_emit(ring, DIV_ROUND_UP(len, 4096) << 16 | 4096);
- intel_ring_emit(ring, cs_offset);
- intel_ring_emit(ring, 4096);
- intel_ring_emit(ring, offset);
+ intel_ring_emit(engine, SRC_COPY_BLT_CMD | BLT_WRITE_RGBA);
+ intel_ring_emit(engine,
+ BLT_DEPTH_32 | BLT_ROP_SRC_COPY | 4096);
+ intel_ring_emit(engine, DIV_ROUND_UP(len, 4096) << 16 | 4096);
+ intel_ring_emit(engine, cs_offset);
+ intel_ring_emit(engine, 4096);
+ intel_ring_emit(engine, offset);
- intel_ring_emit(ring, MI_FLUSH);
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ intel_ring_emit(engine, MI_FLUSH);
+ intel_ring_emit(engine, MI_NOOP);
+ intel_ring_advance(engine);
/* ... and execute it. */
offset = cs_offset;
@@ -1933,10 +1961,10 @@
if (ret)
return ret;
- intel_ring_emit(ring, MI_BATCH_BUFFER_START | MI_BATCH_GTT);
- intel_ring_emit(ring, offset | (dispatch_flags & I915_DISPATCH_SECURE ?
- 0 : MI_BATCH_NON_SECURE));
- intel_ring_advance(ring);
+ intel_ring_emit(engine, MI_BATCH_BUFFER_START | MI_BATCH_GTT);
+ intel_ring_emit(engine, offset | (dispatch_flags & I915_DISPATCH_SECURE ?
+ 0 : MI_BATCH_NON_SECURE));
+ intel_ring_advance(engine);
return 0;
}
@@ -1946,55 +1974,55 @@
u64 offset, u32 len,
unsigned dispatch_flags)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
int ret;
ret = intel_ring_begin(req, 2);
if (ret)
return ret;
- intel_ring_emit(ring, MI_BATCH_BUFFER_START | MI_BATCH_GTT);
- intel_ring_emit(ring, offset | (dispatch_flags & I915_DISPATCH_SECURE ?
- 0 : MI_BATCH_NON_SECURE));
- intel_ring_advance(ring);
+ intel_ring_emit(engine, MI_BATCH_BUFFER_START | MI_BATCH_GTT);
+ intel_ring_emit(engine, offset | (dispatch_flags & I915_DISPATCH_SECURE ?
+ 0 : MI_BATCH_NON_SECURE));
+ intel_ring_advance(engine);
return 0;
}
-static void cleanup_phys_status_page(struct intel_engine_cs *ring)
+static void cleanup_phys_status_page(struct intel_engine_cs *engine)
{
- struct drm_i915_private *dev_priv = to_i915(ring->dev);
+ struct drm_i915_private *dev_priv = to_i915(engine->dev);
if (!dev_priv->status_page_dmah)
return;
- drm_pci_free(ring->dev, dev_priv->status_page_dmah);
- ring->status_page.page_addr = NULL;
+ drm_pci_free(engine->dev, dev_priv->status_page_dmah);
+ engine->status_page.page_addr = NULL;
}
-static void cleanup_status_page(struct intel_engine_cs *ring)
+static void cleanup_status_page(struct intel_engine_cs *engine)
{
struct drm_i915_gem_object *obj;
- obj = ring->status_page.obj;
+ obj = engine->status_page.obj;
if (obj == NULL)
return;
kunmap(sg_page(obj->pages->sgl));
i915_gem_object_ggtt_unpin(obj);
drm_gem_object_unreference(&obj->base);
- ring->status_page.obj = NULL;
+ engine->status_page.obj = NULL;
}
-static int init_status_page(struct intel_engine_cs *ring)
+static int init_status_page(struct intel_engine_cs *engine)
{
- struct drm_i915_gem_object *obj = ring->status_page.obj;
+ struct drm_i915_gem_object *obj = engine->status_page.obj;
if (obj == NULL) {
unsigned flags;
int ret;
- obj = i915_gem_alloc_object(ring->dev, 4096);
+ obj = i915_gem_alloc_object(engine->dev, 4096);
if (obj == NULL) {
DRM_ERROR("Failed to allocate status page\n");
return -ENOMEM;
@@ -2005,7 +2033,7 @@
goto err_unref;
flags = 0;
- if (!HAS_LLC(ring->dev))
+ if (!HAS_LLC(engine->dev))
/* On g33, we cannot place HWS above 256MiB, so
* restrict its pinning to the low mappable arena.
* Though this restriction is not documented for
@@ -2024,32 +2052,32 @@
return ret;
}
- ring->status_page.obj = obj;
+ engine->status_page.obj = obj;
}
- ring->status_page.gfx_addr = i915_gem_obj_ggtt_offset(obj);
- ring->status_page.page_addr = kmap(sg_page(obj->pages->sgl));
- memset(ring->status_page.page_addr, 0, PAGE_SIZE);
+ engine->status_page.gfx_addr = i915_gem_obj_ggtt_offset(obj);
+ engine->status_page.page_addr = kmap(sg_page(obj->pages->sgl));
+ memset(engine->status_page.page_addr, 0, PAGE_SIZE);
DRM_DEBUG_DRIVER("%s hws offset: 0x%08x\n",
- ring->name, ring->status_page.gfx_addr);
+ engine->name, engine->status_page.gfx_addr);
return 0;
}
-static int init_phys_status_page(struct intel_engine_cs *ring)
+static int init_phys_status_page(struct intel_engine_cs *engine)
{
- struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct drm_i915_private *dev_priv = engine->dev->dev_private;
if (!dev_priv->status_page_dmah) {
dev_priv->status_page_dmah =
- drm_pci_alloc(ring->dev, PAGE_SIZE, PAGE_SIZE);
+ drm_pci_alloc(engine->dev, PAGE_SIZE, PAGE_SIZE);
if (!dev_priv->status_page_dmah)
return -ENOMEM;
}
- ring->status_page.page_addr = dev_priv->status_page_dmah->vaddr;
- memset(ring->status_page.page_addr, 0, PAGE_SIZE);
+ engine->status_page.page_addr = dev_priv->status_page_dmah->vaddr;
+ memset(engine->status_page.page_addr, 0, PAGE_SIZE);
return 0;
}
@@ -2057,7 +2085,7 @@
void intel_unpin_ringbuffer_obj(struct intel_ringbuffer *ringbuf)
{
if (HAS_LLC(ringbuf->obj->base.dev) && !ringbuf->obj->stolen)
- vunmap(ringbuf->virtual_start);
+ i915_gem_object_unpin_map(ringbuf->obj);
else
iounmap(ringbuf->virtual_start);
ringbuf->virtual_start = NULL;
@@ -2065,34 +2093,15 @@
i915_gem_object_ggtt_unpin(ringbuf->obj);
}
-static u32 *vmap_obj(struct drm_i915_gem_object *obj)
-{
- struct sg_page_iter sg_iter;
- struct page **pages;
- void *addr;
- int i;
-
- pages = drm_malloc_ab(obj->base.size >> PAGE_SHIFT, sizeof(*pages));
- if (pages == NULL)
- return NULL;
-
- i = 0;
- for_each_sg_page(obj->pages->sgl, &sg_iter, obj->pages->nents, 0)
- pages[i++] = sg_page_iter_page(&sg_iter);
-
- addr = vmap(pages, i, 0, PAGE_KERNEL);
- drm_free_large(pages);
-
- return addr;
-}
-
int intel_pin_and_map_ringbuffer_obj(struct drm_device *dev,
struct intel_ringbuffer *ringbuf)
{
struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_ggtt *ggtt = &dev_priv->ggtt;
struct drm_i915_gem_object *obj = ringbuf->obj;
/* Ring wraparound at offset 0 sometimes hangs. No idea why. */
unsigned flags = PIN_OFFSET_BIAS | 4096;
+ void *addr;
int ret;
if (HAS_LLC(dev_priv) && !obj->stolen) {
@@ -2101,15 +2110,13 @@
return ret;
ret = i915_gem_object_set_to_cpu_domain(obj, true);
- if (ret) {
- i915_gem_object_ggtt_unpin(obj);
- return ret;
- }
+ if (ret)
+ goto err_unpin;
- ringbuf->virtual_start = vmap_obj(obj);
- if (ringbuf->virtual_start == NULL) {
- i915_gem_object_ggtt_unpin(obj);
- return -ENOMEM;
+ addr = i915_gem_object_pin_map(obj);
+ if (IS_ERR(addr)) {
+ ret = PTR_ERR(addr);
+ goto err_unpin;
}
} else {
ret = i915_gem_obj_ggtt_pin(obj, PAGE_SIZE,
@@ -2118,25 +2125,27 @@
return ret;
ret = i915_gem_object_set_to_gtt_domain(obj, true);
- if (ret) {
- i915_gem_object_ggtt_unpin(obj);
- return ret;
- }
+ if (ret)
+ goto err_unpin;
/* Access through the GTT requires the device to be awake. */
assert_rpm_wakelock_held(dev_priv);
- ringbuf->virtual_start = ioremap_wc(dev_priv->gtt.mappable_base +
- i915_gem_obj_ggtt_offset(obj), ringbuf->size);
- if (ringbuf->virtual_start == NULL) {
- i915_gem_object_ggtt_unpin(obj);
- return -EINVAL;
+ addr = ioremap_wc(ggtt->mappable_base +
+ i915_gem_obj_ggtt_offset(obj), ringbuf->size);
+ if (addr == NULL) {
+ ret = -ENOMEM;
+ goto err_unpin;
}
}
+ ringbuf->virtual_start = addr;
ringbuf->vma = i915_gem_obj_to_ggtt(obj);
-
return 0;
+
+err_unpin:
+ i915_gem_object_ggtt_unpin(obj);
+ return ret;
}
static void intel_destroy_ringbuffer_obj(struct intel_ringbuffer *ringbuf)
@@ -2179,7 +2188,7 @@
return ERR_PTR(-ENOMEM);
}
- ring->ring = engine;
+ ring->engine = engine;
list_add(&ring->link, &engine->buffers);
ring->size = size;
@@ -2215,37 +2224,38 @@
}
static int intel_init_ring_buffer(struct drm_device *dev,
- struct intel_engine_cs *ring)
+ struct intel_engine_cs *engine)
{
struct intel_ringbuffer *ringbuf;
int ret;
- WARN_ON(ring->buffer);
+ WARN_ON(engine->buffer);
- ring->dev = dev;
- INIT_LIST_HEAD(&ring->active_list);
- INIT_LIST_HEAD(&ring->request_list);
- INIT_LIST_HEAD(&ring->execlist_queue);
- INIT_LIST_HEAD(&ring->buffers);
- i915_gem_batch_pool_init(dev, &ring->batch_pool);
- memset(ring->semaphore.sync_seqno, 0, sizeof(ring->semaphore.sync_seqno));
+ engine->dev = dev;
+ INIT_LIST_HEAD(&engine->active_list);
+ INIT_LIST_HEAD(&engine->request_list);
+ INIT_LIST_HEAD(&engine->execlist_queue);
+ INIT_LIST_HEAD(&engine->buffers);
+ i915_gem_batch_pool_init(dev, &engine->batch_pool);
+ memset(engine->semaphore.sync_seqno, 0,
+ sizeof(engine->semaphore.sync_seqno));
- init_waitqueue_head(&ring->irq_queue);
+ init_waitqueue_head(&engine->irq_queue);
- ringbuf = intel_engine_create_ringbuffer(ring, 32 * PAGE_SIZE);
+ ringbuf = intel_engine_create_ringbuffer(engine, 32 * PAGE_SIZE);
if (IS_ERR(ringbuf)) {
ret = PTR_ERR(ringbuf);
goto error;
}
- ring->buffer = ringbuf;
+ engine->buffer = ringbuf;
if (I915_NEED_GFX_HWS(dev)) {
- ret = init_status_page(ring);
+ ret = init_status_page(engine);
if (ret)
goto error;
} else {
- WARN_ON(ring->id != RCS);
- ret = init_phys_status_page(ring);
+ WARN_ON(engine->id != RCS);
+ ret = init_phys_status_page(engine);
if (ret)
goto error;
}
@@ -2253,58 +2263,58 @@
ret = intel_pin_and_map_ringbuffer_obj(dev, ringbuf);
if (ret) {
DRM_ERROR("Failed to pin and map ringbuffer %s: %d\n",
- ring->name, ret);
+ engine->name, ret);
intel_destroy_ringbuffer_obj(ringbuf);
goto error;
}
- ret = i915_cmd_parser_init_ring(ring);
+ ret = i915_cmd_parser_init_ring(engine);
if (ret)
goto error;
return 0;
error:
- intel_cleanup_ring_buffer(ring);
+ intel_cleanup_engine(engine);
return ret;
}
-void intel_cleanup_ring_buffer(struct intel_engine_cs *ring)
+void intel_cleanup_engine(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv;
- if (!intel_ring_initialized(ring))
+ if (!intel_engine_initialized(engine))
return;
- dev_priv = to_i915(ring->dev);
+ dev_priv = to_i915(engine->dev);
- if (ring->buffer) {
- intel_stop_ring_buffer(ring);
- WARN_ON(!IS_GEN2(ring->dev) && (I915_READ_MODE(ring) & MODE_IDLE) == 0);
+ if (engine->buffer) {
+ intel_stop_engine(engine);
+ WARN_ON(!IS_GEN2(engine->dev) && (I915_READ_MODE(engine) & MODE_IDLE) == 0);
- intel_unpin_ringbuffer_obj(ring->buffer);
- intel_ringbuffer_free(ring->buffer);
- ring->buffer = NULL;
+ intel_unpin_ringbuffer_obj(engine->buffer);
+ intel_ringbuffer_free(engine->buffer);
+ engine->buffer = NULL;
}
- if (ring->cleanup)
- ring->cleanup(ring);
+ if (engine->cleanup)
+ engine->cleanup(engine);
- if (I915_NEED_GFX_HWS(ring->dev)) {
- cleanup_status_page(ring);
+ if (I915_NEED_GFX_HWS(engine->dev)) {
+ cleanup_status_page(engine);
} else {
- WARN_ON(ring->id != RCS);
- cleanup_phys_status_page(ring);
+ WARN_ON(engine->id != RCS);
+ cleanup_phys_status_page(engine);
}
- i915_cmd_parser_fini_ring(ring);
- i915_gem_batch_pool_fini(&ring->batch_pool);
- ring->dev = NULL;
+ i915_cmd_parser_fini_ring(engine);
+ i915_gem_batch_pool_fini(&engine->batch_pool);
+ engine->dev = NULL;
}
-static int ring_wait_for_space(struct intel_engine_cs *ring, int n)
+static int ring_wait_for_space(struct intel_engine_cs *engine, int n)
{
- struct intel_ringbuffer *ringbuf = ring->buffer;
+ struct intel_ringbuffer *ringbuf = engine->buffer;
struct drm_i915_gem_request *request;
unsigned space;
int ret;
@@ -2315,14 +2325,14 @@
/* The whole point of reserving space is to not wait! */
WARN_ON(ringbuf->reserved_in_use);
- list_for_each_entry(request, &ring->request_list, list) {
+ list_for_each_entry(request, &engine->request_list, list) {
space = __intel_ring_space(request->postfix, ringbuf->tail,
ringbuf->size);
if (space >= n)
break;
}
- if (WARN_ON(&request->list == &ring->request_list))
+ if (WARN_ON(&request->list == &engine->request_list))
return -ENOSPC;
ret = i915_wait_request(request);
@@ -2347,28 +2357,27 @@
intel_ring_update_space(ringbuf);
}
-int intel_ring_idle(struct intel_engine_cs *ring)
+int intel_engine_idle(struct intel_engine_cs *engine)
{
struct drm_i915_gem_request *req;
/* Wait upon the last request to be completed */
- if (list_empty(&ring->request_list))
+ if (list_empty(&engine->request_list))
return 0;
- req = list_entry(ring->request_list.prev,
- struct drm_i915_gem_request,
- list);
+ req = list_entry(engine->request_list.prev,
+ struct drm_i915_gem_request,
+ list);
/* Make sure we do not trigger any retires */
return __i915_wait_request(req,
- atomic_read(&to_i915(ring->dev)->gpu_error.reset_counter),
- to_i915(ring->dev)->mm.interruptible,
+ req->i915->mm.interruptible,
NULL, NULL);
}
int intel_ring_alloc_request_extras(struct drm_i915_gem_request *request)
{
- request->ringbuf = request->ring->buffer;
+ request->ringbuf = request->engine->buffer;
return 0;
}
@@ -2434,9 +2443,9 @@
ringbuf->reserved_in_use = false;
}
-static int __intel_ring_prepare(struct intel_engine_cs *ring, int bytes)
+static int __intel_ring_prepare(struct intel_engine_cs *engine, int bytes)
{
- struct intel_ringbuffer *ringbuf = ring->buffer;
+ struct intel_ringbuffer *ringbuf = engine->buffer;
int remain_usable = ringbuf->effective_size - ringbuf->tail;
int remain_actual = ringbuf->size - ringbuf->tail;
int ret, total_bytes, wait_bytes = 0;
@@ -2470,7 +2479,7 @@
}
if (wait_bytes) {
- ret = ring_wait_for_space(ring, wait_bytes);
+ ret = ring_wait_for_space(engine, wait_bytes);
if (unlikely(ret))
return ret;
@@ -2484,32 +2493,22 @@
int intel_ring_begin(struct drm_i915_gem_request *req,
int num_dwords)
{
- struct intel_engine_cs *ring;
- struct drm_i915_private *dev_priv;
+ struct intel_engine_cs *engine = req->engine;
int ret;
- WARN_ON(req == NULL);
- ring = req->ring;
- dev_priv = ring->dev->dev_private;
-
- ret = i915_gem_check_wedge(&dev_priv->gpu_error,
- dev_priv->mm.interruptible);
+ ret = __intel_ring_prepare(engine, num_dwords * sizeof(uint32_t));
if (ret)
return ret;
- ret = __intel_ring_prepare(ring, num_dwords * sizeof(uint32_t));
- if (ret)
- return ret;
-
- ring->buffer->space -= num_dwords * sizeof(uint32_t);
+ engine->buffer->space -= num_dwords * sizeof(uint32_t);
return 0;
}
/* Align the ring tail to a cacheline boundary */
int intel_ring_cacheline_align(struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *ring = req->ring;
- int num_dwords = (ring->buffer->tail & (CACHELINE_BYTES - 1)) / sizeof(uint32_t);
+ struct intel_engine_cs *engine = req->engine;
+ int num_dwords = (engine->buffer->tail & (CACHELINE_BYTES - 1)) / sizeof(uint32_t);
int ret;
if (num_dwords == 0)
@@ -2521,33 +2520,52 @@
return ret;
while (num_dwords--)
- intel_ring_emit(ring, MI_NOOP);
+ intel_ring_emit(engine, MI_NOOP);
- intel_ring_advance(ring);
+ intel_ring_advance(engine);
return 0;
}
-void intel_ring_init_seqno(struct intel_engine_cs *ring, u32 seqno)
+void intel_ring_init_seqno(struct intel_engine_cs *engine, u32 seqno)
{
- struct drm_device *dev = ring->dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_i915_private *dev_priv = to_i915(engine->dev);
- if (INTEL_INFO(dev)->gen == 6 || INTEL_INFO(dev)->gen == 7) {
- I915_WRITE(RING_SYNC_0(ring->mmio_base), 0);
- I915_WRITE(RING_SYNC_1(ring->mmio_base), 0);
- if (HAS_VEBOX(dev))
- I915_WRITE(RING_SYNC_2(ring->mmio_base), 0);
+ /* Our semaphore implementation is strictly monotonic (i.e. we proceed
+ * so long as the semaphore value in the register/page is greater
+ * than the sync value), so whenever we reset the seqno,
+ * so long as we reset the tracking semaphore value to 0, it will
+ * always be before the next request's seqno. If we don't reset
+ * the semaphore value, then when the seqno moves backwards all
+ * future waits will complete instantly (causing rendering corruption).
+ */
+ if (INTEL_INFO(dev_priv)->gen == 6 || INTEL_INFO(dev_priv)->gen == 7) {
+ I915_WRITE(RING_SYNC_0(engine->mmio_base), 0);
+ I915_WRITE(RING_SYNC_1(engine->mmio_base), 0);
+ if (HAS_VEBOX(dev_priv))
+ I915_WRITE(RING_SYNC_2(engine->mmio_base), 0);
}
+ if (dev_priv->semaphore_obj) {
+ struct drm_i915_gem_object *obj = dev_priv->semaphore_obj;
+ struct page *page = i915_gem_object_get_dirty_page(obj, 0);
+ void *semaphores = kmap(page);
+ memset(semaphores + GEN8_SEMAPHORE_OFFSET(engine->id, 0),
+ 0, I915_NUM_ENGINES * gen8_semaphore_seqno_size);
+ kunmap(page);
+ }
+ memset(engine->semaphore.sync_seqno, 0,
+ sizeof(engine->semaphore.sync_seqno));
- ring->set_seqno(ring, seqno);
- ring->hangcheck.seqno = seqno;
+ engine->set_seqno(engine, seqno);
+ engine->last_submitted_seqno = seqno;
+
+ engine->hangcheck.seqno = seqno;
}
-static void gen6_bsd_ring_write_tail(struct intel_engine_cs *ring,
+static void gen6_bsd_ring_write_tail(struct intel_engine_cs *engine,
u32 value)
{
- struct drm_i915_private *dev_priv = ring->dev->dev_private;
+ struct drm_i915_private *dev_priv = engine->dev->dev_private;
/* Every tail move must follow the sequence below */
@@ -2567,8 +2585,8 @@
DRM_ERROR("timed out waiting for the BSD ring to wake up\n");
/* Now that the ring is fully powered up, update the tail */
- I915_WRITE_TAIL(ring, value);
- POSTING_READ(RING_TAIL(ring->mmio_base));
+ I915_WRITE_TAIL(engine, value);
+ POSTING_READ(RING_TAIL(engine->mmio_base));
/* Let the ring send IDLE messages to the GT again,
* and so let it sleep to conserve power when idle.
@@ -2580,7 +2598,7 @@
static int gen6_bsd_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate, u32 flush)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
uint32_t cmd;
int ret;
@@ -2589,7 +2607,7 @@
return ret;
cmd = MI_FLUSH_DW;
- if (INTEL_INFO(ring->dev)->gen >= 8)
+ if (INTEL_INFO(engine->dev)->gen >= 8)
cmd += 1;
/* We always require a command barrier so that subsequent
@@ -2608,16 +2626,17 @@
if (invalidate & I915_GEM_GPU_DOMAINS)
cmd |= MI_INVALIDATE_TLB | MI_INVALIDATE_BSD;
- intel_ring_emit(ring, cmd);
- intel_ring_emit(ring, I915_GEM_HWS_SCRATCH_ADDR | MI_FLUSH_DW_USE_GTT);
- if (INTEL_INFO(ring->dev)->gen >= 8) {
- intel_ring_emit(ring, 0); /* upper addr */
- intel_ring_emit(ring, 0); /* value */
+ intel_ring_emit(engine, cmd);
+ intel_ring_emit(engine,
+ I915_GEM_HWS_SCRATCH_ADDR | MI_FLUSH_DW_USE_GTT);
+ if (INTEL_INFO(engine->dev)->gen >= 8) {
+ intel_ring_emit(engine, 0); /* upper addr */
+ intel_ring_emit(engine, 0); /* value */
} else {
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, MI_NOOP);
+ intel_ring_emit(engine, 0);
+ intel_ring_emit(engine, MI_NOOP);
}
- intel_ring_advance(ring);
+ intel_ring_advance(engine);
return 0;
}
@@ -2626,8 +2645,8 @@
u64 offset, u32 len,
unsigned dispatch_flags)
{
- struct intel_engine_cs *ring = req->ring;
- bool ppgtt = USES_PPGTT(ring->dev) &&
+ struct intel_engine_cs *engine = req->engine;
+ bool ppgtt = USES_PPGTT(engine->dev) &&
!(dispatch_flags & I915_DISPATCH_SECURE);
int ret;
@@ -2636,13 +2655,13 @@
return ret;
/* FIXME(BDW): Address space and security selectors. */
- intel_ring_emit(ring, MI_BATCH_BUFFER_START_GEN8 | (ppgtt<<8) |
+ intel_ring_emit(engine, MI_BATCH_BUFFER_START_GEN8 | (ppgtt<<8) |
(dispatch_flags & I915_DISPATCH_RS ?
MI_BATCH_RESOURCE_STREAMER : 0));
- intel_ring_emit(ring, lower_32_bits(offset));
- intel_ring_emit(ring, upper_32_bits(offset));
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ring);
+ intel_ring_emit(engine, lower_32_bits(offset));
+ intel_ring_emit(engine, upper_32_bits(offset));
+ intel_ring_emit(engine, MI_NOOP);
+ intel_ring_advance(engine);
return 0;
}
@@ -2652,22 +2671,22 @@
u64 offset, u32 len,
unsigned dispatch_flags)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
int ret;
ret = intel_ring_begin(req, 2);
if (ret)
return ret;
- intel_ring_emit(ring,
+ intel_ring_emit(engine,
MI_BATCH_BUFFER_START |
(dispatch_flags & I915_DISPATCH_SECURE ?
0 : MI_BATCH_PPGTT_HSW | MI_BATCH_NON_SECURE_HSW) |
(dispatch_flags & I915_DISPATCH_RS ?
MI_BATCH_RESOURCE_STREAMER : 0));
/* bit0-7 is the length on GEN6+ */
- intel_ring_emit(ring, offset);
- intel_ring_advance(ring);
+ intel_ring_emit(engine, offset);
+ intel_ring_advance(engine);
return 0;
}
@@ -2677,20 +2696,20 @@
u64 offset, u32 len,
unsigned dispatch_flags)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
int ret;
ret = intel_ring_begin(req, 2);
if (ret)
return ret;
- intel_ring_emit(ring,
+ intel_ring_emit(engine,
MI_BATCH_BUFFER_START |
(dispatch_flags & I915_DISPATCH_SECURE ?
0 : MI_BATCH_NON_SECURE_I965));
/* bit0-7 is the length on GEN6+ */
- intel_ring_emit(ring, offset);
- intel_ring_advance(ring);
+ intel_ring_emit(engine, offset);
+ intel_ring_advance(engine);
return 0;
}
@@ -2700,8 +2719,8 @@
static int gen6_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate, u32 flush)
{
- struct intel_engine_cs *ring = req->ring;
- struct drm_device *dev = ring->dev;
+ struct intel_engine_cs *engine = req->engine;
+ struct drm_device *dev = engine->dev;
uint32_t cmd;
int ret;
@@ -2728,16 +2747,17 @@
*/
if (invalidate & I915_GEM_DOMAIN_RENDER)
cmd |= MI_INVALIDATE_TLB;
- intel_ring_emit(ring, cmd);
- intel_ring_emit(ring, I915_GEM_HWS_SCRATCH_ADDR | MI_FLUSH_DW_USE_GTT);
+ intel_ring_emit(engine, cmd);
+ intel_ring_emit(engine,
+ I915_GEM_HWS_SCRATCH_ADDR | MI_FLUSH_DW_USE_GTT);
if (INTEL_INFO(dev)->gen >= 8) {
- intel_ring_emit(ring, 0); /* upper addr */
- intel_ring_emit(ring, 0); /* value */
+ intel_ring_emit(engine, 0); /* upper addr */
+ intel_ring_emit(engine, 0); /* value */
} else {
- intel_ring_emit(ring, 0);
- intel_ring_emit(ring, MI_NOOP);
+ intel_ring_emit(engine, 0);
+ intel_ring_emit(engine, MI_NOOP);
}
- intel_ring_advance(ring);
+ intel_ring_advance(engine);
return 0;
}
@@ -2745,14 +2765,14 @@
int intel_init_render_ring_buffer(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring = &dev_priv->ring[RCS];
+ struct intel_engine_cs *engine = &dev_priv->engine[RCS];
struct drm_i915_gem_object *obj;
int ret;
- ring->name = "render ring";
- ring->id = RCS;
- ring->exec_id = I915_EXEC_RENDER;
- ring->mmio_base = RENDER_RING_BASE;
+ engine->name = "render ring";
+ engine->id = RCS;
+ engine->exec_id = I915_EXEC_RENDER;
+ engine->mmio_base = RENDER_RING_BASE;
if (INTEL_INFO(dev)->gen >= 8) {
if (i915_semaphore_is_enabled(dev)) {
@@ -2772,34 +2792,36 @@
}
}
- ring->init_context = intel_rcs_ctx_init;
- ring->add_request = gen6_add_request;
- ring->flush = gen8_render_ring_flush;
- ring->irq_get = gen8_ring_get_irq;
- ring->irq_put = gen8_ring_put_irq;
- ring->irq_enable_mask = GT_RENDER_USER_INTERRUPT;
- ring->get_seqno = gen6_ring_get_seqno;
- ring->set_seqno = ring_set_seqno;
+ engine->init_context = intel_rcs_ctx_init;
+ engine->add_request = gen6_add_request;
+ engine->flush = gen8_render_ring_flush;
+ engine->irq_get = gen8_ring_get_irq;
+ engine->irq_put = gen8_ring_put_irq;
+ engine->irq_enable_mask = GT_RENDER_USER_INTERRUPT;
+ engine->irq_seqno_barrier = gen6_seqno_barrier;
+ engine->get_seqno = ring_get_seqno;
+ engine->set_seqno = ring_set_seqno;
if (i915_semaphore_is_enabled(dev)) {
WARN_ON(!dev_priv->semaphore_obj);
- ring->semaphore.sync_to = gen8_ring_sync;
- ring->semaphore.signal = gen8_rcs_signal;
- GEN8_RING_SEMAPHORE_INIT;
+ engine->semaphore.sync_to = gen8_ring_sync;
+ engine->semaphore.signal = gen8_rcs_signal;
+ GEN8_RING_SEMAPHORE_INIT(engine);
}
} else if (INTEL_INFO(dev)->gen >= 6) {
- ring->init_context = intel_rcs_ctx_init;
- ring->add_request = gen6_add_request;
- ring->flush = gen7_render_ring_flush;
+ engine->init_context = intel_rcs_ctx_init;
+ engine->add_request = gen6_add_request;
+ engine->flush = gen7_render_ring_flush;
if (INTEL_INFO(dev)->gen == 6)
- ring->flush = gen6_render_ring_flush;
- ring->irq_get = gen6_ring_get_irq;
- ring->irq_put = gen6_ring_put_irq;
- ring->irq_enable_mask = GT_RENDER_USER_INTERRUPT;
- ring->get_seqno = gen6_ring_get_seqno;
- ring->set_seqno = ring_set_seqno;
+ engine->flush = gen6_render_ring_flush;
+ engine->irq_get = gen6_ring_get_irq;
+ engine->irq_put = gen6_ring_put_irq;
+ engine->irq_enable_mask = GT_RENDER_USER_INTERRUPT;
+ engine->irq_seqno_barrier = gen6_seqno_barrier;
+ engine->get_seqno = ring_get_seqno;
+ engine->set_seqno = ring_set_seqno;
if (i915_semaphore_is_enabled(dev)) {
- ring->semaphore.sync_to = gen6_ring_sync;
- ring->semaphore.signal = gen6_signal;
+ engine->semaphore.sync_to = gen6_ring_sync;
+ engine->semaphore.signal = gen6_signal;
/*
* The current semaphore is only applied on pre-gen8
* platform. And there is no VCS2 ring on the pre-gen8
@@ -2807,59 +2829,59 @@
* initialized as INVALID. Gen8 will initialize the
* sema between VCS2 and RCS later.
*/
- ring->semaphore.mbox.wait[RCS] = MI_SEMAPHORE_SYNC_INVALID;
- ring->semaphore.mbox.wait[VCS] = MI_SEMAPHORE_SYNC_RV;
- ring->semaphore.mbox.wait[BCS] = MI_SEMAPHORE_SYNC_RB;
- ring->semaphore.mbox.wait[VECS] = MI_SEMAPHORE_SYNC_RVE;
- ring->semaphore.mbox.wait[VCS2] = MI_SEMAPHORE_SYNC_INVALID;
- ring->semaphore.mbox.signal[RCS] = GEN6_NOSYNC;
- ring->semaphore.mbox.signal[VCS] = GEN6_VRSYNC;
- ring->semaphore.mbox.signal[BCS] = GEN6_BRSYNC;
- ring->semaphore.mbox.signal[VECS] = GEN6_VERSYNC;
- ring->semaphore.mbox.signal[VCS2] = GEN6_NOSYNC;
+ engine->semaphore.mbox.wait[RCS] = MI_SEMAPHORE_SYNC_INVALID;
+ engine->semaphore.mbox.wait[VCS] = MI_SEMAPHORE_SYNC_RV;
+ engine->semaphore.mbox.wait[BCS] = MI_SEMAPHORE_SYNC_RB;
+ engine->semaphore.mbox.wait[VECS] = MI_SEMAPHORE_SYNC_RVE;
+ engine->semaphore.mbox.wait[VCS2] = MI_SEMAPHORE_SYNC_INVALID;
+ engine->semaphore.mbox.signal[RCS] = GEN6_NOSYNC;
+ engine->semaphore.mbox.signal[VCS] = GEN6_VRSYNC;
+ engine->semaphore.mbox.signal[BCS] = GEN6_BRSYNC;
+ engine->semaphore.mbox.signal[VECS] = GEN6_VERSYNC;
+ engine->semaphore.mbox.signal[VCS2] = GEN6_NOSYNC;
}
} else if (IS_GEN5(dev)) {
- ring->add_request = pc_render_add_request;
- ring->flush = gen4_render_ring_flush;
- ring->get_seqno = pc_render_get_seqno;
- ring->set_seqno = pc_render_set_seqno;
- ring->irq_get = gen5_ring_get_irq;
- ring->irq_put = gen5_ring_put_irq;
- ring->irq_enable_mask = GT_RENDER_USER_INTERRUPT |
+ engine->add_request = pc_render_add_request;
+ engine->flush = gen4_render_ring_flush;
+ engine->get_seqno = pc_render_get_seqno;
+ engine->set_seqno = pc_render_set_seqno;
+ engine->irq_get = gen5_ring_get_irq;
+ engine->irq_put = gen5_ring_put_irq;
+ engine->irq_enable_mask = GT_RENDER_USER_INTERRUPT |
GT_RENDER_PIPECTL_NOTIFY_INTERRUPT;
} else {
- ring->add_request = i9xx_add_request;
+ engine->add_request = i9xx_add_request;
if (INTEL_INFO(dev)->gen < 4)
- ring->flush = gen2_render_ring_flush;
+ engine->flush = gen2_render_ring_flush;
else
- ring->flush = gen4_render_ring_flush;
- ring->get_seqno = ring_get_seqno;
- ring->set_seqno = ring_set_seqno;
+ engine->flush = gen4_render_ring_flush;
+ engine->get_seqno = ring_get_seqno;
+ engine->set_seqno = ring_set_seqno;
if (IS_GEN2(dev)) {
- ring->irq_get = i8xx_ring_get_irq;
- ring->irq_put = i8xx_ring_put_irq;
+ engine->irq_get = i8xx_ring_get_irq;
+ engine->irq_put = i8xx_ring_put_irq;
} else {
- ring->irq_get = i9xx_ring_get_irq;
- ring->irq_put = i9xx_ring_put_irq;
+ engine->irq_get = i9xx_ring_get_irq;
+ engine->irq_put = i9xx_ring_put_irq;
}
- ring->irq_enable_mask = I915_USER_INTERRUPT;
+ engine->irq_enable_mask = I915_USER_INTERRUPT;
}
- ring->write_tail = ring_write_tail;
+ engine->write_tail = ring_write_tail;
if (IS_HASWELL(dev))
- ring->dispatch_execbuffer = hsw_ring_dispatch_execbuffer;
+ engine->dispatch_execbuffer = hsw_ring_dispatch_execbuffer;
else if (IS_GEN8(dev))
- ring->dispatch_execbuffer = gen8_ring_dispatch_execbuffer;
+ engine->dispatch_execbuffer = gen8_ring_dispatch_execbuffer;
else if (INTEL_INFO(dev)->gen >= 6)
- ring->dispatch_execbuffer = gen6_ring_dispatch_execbuffer;
+ engine->dispatch_execbuffer = gen6_ring_dispatch_execbuffer;
else if (INTEL_INFO(dev)->gen >= 4)
- ring->dispatch_execbuffer = i965_dispatch_execbuffer;
+ engine->dispatch_execbuffer = i965_dispatch_execbuffer;
else if (IS_I830(dev) || IS_845G(dev))
- ring->dispatch_execbuffer = i830_dispatch_execbuffer;
+ engine->dispatch_execbuffer = i830_dispatch_execbuffer;
else
- ring->dispatch_execbuffer = i915_dispatch_execbuffer;
- ring->init_hw = init_render_ring;
- ring->cleanup = render_ring_cleanup;
+ engine->dispatch_execbuffer = i915_dispatch_execbuffer;
+ engine->init_hw = init_render_ring;
+ engine->cleanup = render_ring_cleanup;
/* Workaround batchbuffer to combat CS tlb bug. */
if (HAS_BROKEN_CS_TLB(dev)) {
@@ -2876,16 +2898,16 @@
return ret;
}
- ring->scratch.obj = obj;
- ring->scratch.gtt_offset = i915_gem_obj_ggtt_offset(obj);
+ engine->scratch.obj = obj;
+ engine->scratch.gtt_offset = i915_gem_obj_ggtt_offset(obj);
}
- ret = intel_init_ring_buffer(dev, ring);
+ ret = intel_init_ring_buffer(dev, engine);
if (ret)
return ret;
if (INTEL_INFO(dev)->gen >= 5) {
- ret = intel_init_pipe_control(ring);
+ ret = intel_init_pipe_control(engine);
if (ret)
return ret;
}
@@ -2896,75 +2918,76 @@
int intel_init_bsd_ring_buffer(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring = &dev_priv->ring[VCS];
+ struct intel_engine_cs *engine = &dev_priv->engine[VCS];
- ring->name = "bsd ring";
- ring->id = VCS;
- ring->exec_id = I915_EXEC_BSD;
+ engine->name = "bsd ring";
+ engine->id = VCS;
+ engine->exec_id = I915_EXEC_BSD;
- ring->write_tail = ring_write_tail;
+ engine->write_tail = ring_write_tail;
if (INTEL_INFO(dev)->gen >= 6) {
- ring->mmio_base = GEN6_BSD_RING_BASE;
+ engine->mmio_base = GEN6_BSD_RING_BASE;
/* gen6 bsd needs a special wa for tail updates */
if (IS_GEN6(dev))
- ring->write_tail = gen6_bsd_ring_write_tail;
- ring->flush = gen6_bsd_ring_flush;
- ring->add_request = gen6_add_request;
- ring->get_seqno = gen6_ring_get_seqno;
- ring->set_seqno = ring_set_seqno;
+ engine->write_tail = gen6_bsd_ring_write_tail;
+ engine->flush = gen6_bsd_ring_flush;
+ engine->add_request = gen6_add_request;
+ engine->irq_seqno_barrier = gen6_seqno_barrier;
+ engine->get_seqno = ring_get_seqno;
+ engine->set_seqno = ring_set_seqno;
if (INTEL_INFO(dev)->gen >= 8) {
- ring->irq_enable_mask =
+ engine->irq_enable_mask =
GT_RENDER_USER_INTERRUPT << GEN8_VCS1_IRQ_SHIFT;
- ring->irq_get = gen8_ring_get_irq;
- ring->irq_put = gen8_ring_put_irq;
- ring->dispatch_execbuffer =
+ engine->irq_get = gen8_ring_get_irq;
+ engine->irq_put = gen8_ring_put_irq;
+ engine->dispatch_execbuffer =
gen8_ring_dispatch_execbuffer;
if (i915_semaphore_is_enabled(dev)) {
- ring->semaphore.sync_to = gen8_ring_sync;
- ring->semaphore.signal = gen8_xcs_signal;
- GEN8_RING_SEMAPHORE_INIT;
+ engine->semaphore.sync_to = gen8_ring_sync;
+ engine->semaphore.signal = gen8_xcs_signal;
+ GEN8_RING_SEMAPHORE_INIT(engine);
}
} else {
- ring->irq_enable_mask = GT_BSD_USER_INTERRUPT;
- ring->irq_get = gen6_ring_get_irq;
- ring->irq_put = gen6_ring_put_irq;
- ring->dispatch_execbuffer =
+ engine->irq_enable_mask = GT_BSD_USER_INTERRUPT;
+ engine->irq_get = gen6_ring_get_irq;
+ engine->irq_put = gen6_ring_put_irq;
+ engine->dispatch_execbuffer =
gen6_ring_dispatch_execbuffer;
if (i915_semaphore_is_enabled(dev)) {
- ring->semaphore.sync_to = gen6_ring_sync;
- ring->semaphore.signal = gen6_signal;
- ring->semaphore.mbox.wait[RCS] = MI_SEMAPHORE_SYNC_VR;
- ring->semaphore.mbox.wait[VCS] = MI_SEMAPHORE_SYNC_INVALID;
- ring->semaphore.mbox.wait[BCS] = MI_SEMAPHORE_SYNC_VB;
- ring->semaphore.mbox.wait[VECS] = MI_SEMAPHORE_SYNC_VVE;
- ring->semaphore.mbox.wait[VCS2] = MI_SEMAPHORE_SYNC_INVALID;
- ring->semaphore.mbox.signal[RCS] = GEN6_RVSYNC;
- ring->semaphore.mbox.signal[VCS] = GEN6_NOSYNC;
- ring->semaphore.mbox.signal[BCS] = GEN6_BVSYNC;
- ring->semaphore.mbox.signal[VECS] = GEN6_VEVSYNC;
- ring->semaphore.mbox.signal[VCS2] = GEN6_NOSYNC;
+ engine->semaphore.sync_to = gen6_ring_sync;
+ engine->semaphore.signal = gen6_signal;
+ engine->semaphore.mbox.wait[RCS] = MI_SEMAPHORE_SYNC_VR;
+ engine->semaphore.mbox.wait[VCS] = MI_SEMAPHORE_SYNC_INVALID;
+ engine->semaphore.mbox.wait[BCS] = MI_SEMAPHORE_SYNC_VB;
+ engine->semaphore.mbox.wait[VECS] = MI_SEMAPHORE_SYNC_VVE;
+ engine->semaphore.mbox.wait[VCS2] = MI_SEMAPHORE_SYNC_INVALID;
+ engine->semaphore.mbox.signal[RCS] = GEN6_RVSYNC;
+ engine->semaphore.mbox.signal[VCS] = GEN6_NOSYNC;
+ engine->semaphore.mbox.signal[BCS] = GEN6_BVSYNC;
+ engine->semaphore.mbox.signal[VECS] = GEN6_VEVSYNC;
+ engine->semaphore.mbox.signal[VCS2] = GEN6_NOSYNC;
}
}
} else {
- ring->mmio_base = BSD_RING_BASE;
- ring->flush = bsd_ring_flush;
- ring->add_request = i9xx_add_request;
- ring->get_seqno = ring_get_seqno;
- ring->set_seqno = ring_set_seqno;
+ engine->mmio_base = BSD_RING_BASE;
+ engine->flush = bsd_ring_flush;
+ engine->add_request = i9xx_add_request;
+ engine->get_seqno = ring_get_seqno;
+ engine->set_seqno = ring_set_seqno;
if (IS_GEN5(dev)) {
- ring->irq_enable_mask = ILK_BSD_USER_INTERRUPT;
- ring->irq_get = gen5_ring_get_irq;
- ring->irq_put = gen5_ring_put_irq;
+ engine->irq_enable_mask = ILK_BSD_USER_INTERRUPT;
+ engine->irq_get = gen5_ring_get_irq;
+ engine->irq_put = gen5_ring_put_irq;
} else {
- ring->irq_enable_mask = I915_BSD_USER_INTERRUPT;
- ring->irq_get = i9xx_ring_get_irq;
- ring->irq_put = i9xx_ring_put_irq;
+ engine->irq_enable_mask = I915_BSD_USER_INTERRUPT;
+ engine->irq_get = i9xx_ring_get_irq;
+ engine->irq_put = i9xx_ring_put_irq;
}
- ring->dispatch_execbuffer = i965_dispatch_execbuffer;
+ engine->dispatch_execbuffer = i965_dispatch_execbuffer;
}
- ring->init_hw = init_ring_common;
+ engine->init_hw = init_ring_common;
- return intel_init_ring_buffer(dev, ring);
+ return intel_init_ring_buffer(dev, engine);
}
/**
@@ -2973,68 +2996,70 @@
int intel_init_bsd2_ring_buffer(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring = &dev_priv->ring[VCS2];
+ struct intel_engine_cs *engine = &dev_priv->engine[VCS2];
- ring->name = "bsd2 ring";
- ring->id = VCS2;
- ring->exec_id = I915_EXEC_BSD;
+ engine->name = "bsd2 ring";
+ engine->id = VCS2;
+ engine->exec_id = I915_EXEC_BSD;
- ring->write_tail = ring_write_tail;
- ring->mmio_base = GEN8_BSD2_RING_BASE;
- ring->flush = gen6_bsd_ring_flush;
- ring->add_request = gen6_add_request;
- ring->get_seqno = gen6_ring_get_seqno;
- ring->set_seqno = ring_set_seqno;
- ring->irq_enable_mask =
+ engine->write_tail = ring_write_tail;
+ engine->mmio_base = GEN8_BSD2_RING_BASE;
+ engine->flush = gen6_bsd_ring_flush;
+ engine->add_request = gen6_add_request;
+ engine->irq_seqno_barrier = gen6_seqno_barrier;
+ engine->get_seqno = ring_get_seqno;
+ engine->set_seqno = ring_set_seqno;
+ engine->irq_enable_mask =
GT_RENDER_USER_INTERRUPT << GEN8_VCS2_IRQ_SHIFT;
- ring->irq_get = gen8_ring_get_irq;
- ring->irq_put = gen8_ring_put_irq;
- ring->dispatch_execbuffer =
+ engine->irq_get = gen8_ring_get_irq;
+ engine->irq_put = gen8_ring_put_irq;
+ engine->dispatch_execbuffer =
gen8_ring_dispatch_execbuffer;
if (i915_semaphore_is_enabled(dev)) {
- ring->semaphore.sync_to = gen8_ring_sync;
- ring->semaphore.signal = gen8_xcs_signal;
- GEN8_RING_SEMAPHORE_INIT;
+ engine->semaphore.sync_to = gen8_ring_sync;
+ engine->semaphore.signal = gen8_xcs_signal;
+ GEN8_RING_SEMAPHORE_INIT(engine);
}
- ring->init_hw = init_ring_common;
+ engine->init_hw = init_ring_common;
- return intel_init_ring_buffer(dev, ring);
+ return intel_init_ring_buffer(dev, engine);
}
int intel_init_blt_ring_buffer(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring = &dev_priv->ring[BCS];
+ struct intel_engine_cs *engine = &dev_priv->engine[BCS];
- ring->name = "blitter ring";
- ring->id = BCS;
- ring->exec_id = I915_EXEC_BLT;
+ engine->name = "blitter ring";
+ engine->id = BCS;
+ engine->exec_id = I915_EXEC_BLT;
- ring->mmio_base = BLT_RING_BASE;
- ring->write_tail = ring_write_tail;
- ring->flush = gen6_ring_flush;
- ring->add_request = gen6_add_request;
- ring->get_seqno = gen6_ring_get_seqno;
- ring->set_seqno = ring_set_seqno;
+ engine->mmio_base = BLT_RING_BASE;
+ engine->write_tail = ring_write_tail;
+ engine->flush = gen6_ring_flush;
+ engine->add_request = gen6_add_request;
+ engine->irq_seqno_barrier = gen6_seqno_barrier;
+ engine->get_seqno = ring_get_seqno;
+ engine->set_seqno = ring_set_seqno;
if (INTEL_INFO(dev)->gen >= 8) {
- ring->irq_enable_mask =
+ engine->irq_enable_mask =
GT_RENDER_USER_INTERRUPT << GEN8_BCS_IRQ_SHIFT;
- ring->irq_get = gen8_ring_get_irq;
- ring->irq_put = gen8_ring_put_irq;
- ring->dispatch_execbuffer = gen8_ring_dispatch_execbuffer;
+ engine->irq_get = gen8_ring_get_irq;
+ engine->irq_put = gen8_ring_put_irq;
+ engine->dispatch_execbuffer = gen8_ring_dispatch_execbuffer;
if (i915_semaphore_is_enabled(dev)) {
- ring->semaphore.sync_to = gen8_ring_sync;
- ring->semaphore.signal = gen8_xcs_signal;
- GEN8_RING_SEMAPHORE_INIT;
+ engine->semaphore.sync_to = gen8_ring_sync;
+ engine->semaphore.signal = gen8_xcs_signal;
+ GEN8_RING_SEMAPHORE_INIT(engine);
}
} else {
- ring->irq_enable_mask = GT_BLT_USER_INTERRUPT;
- ring->irq_get = gen6_ring_get_irq;
- ring->irq_put = gen6_ring_put_irq;
- ring->dispatch_execbuffer = gen6_ring_dispatch_execbuffer;
+ engine->irq_enable_mask = GT_BLT_USER_INTERRUPT;
+ engine->irq_get = gen6_ring_get_irq;
+ engine->irq_put = gen6_ring_put_irq;
+ engine->dispatch_execbuffer = gen6_ring_dispatch_execbuffer;
if (i915_semaphore_is_enabled(dev)) {
- ring->semaphore.signal = gen6_signal;
- ring->semaphore.sync_to = gen6_ring_sync;
+ engine->semaphore.signal = gen6_signal;
+ engine->semaphore.sync_to = gen6_ring_sync;
/*
* The current semaphore is only applied on pre-gen8
* platform. And there is no VCS2 ring on the pre-gen8
@@ -3042,127 +3067,128 @@
* initialized as INVALID. Gen8 will initialize the
* sema between BCS and VCS2 later.
*/
- ring->semaphore.mbox.wait[RCS] = MI_SEMAPHORE_SYNC_BR;
- ring->semaphore.mbox.wait[VCS] = MI_SEMAPHORE_SYNC_BV;
- ring->semaphore.mbox.wait[BCS] = MI_SEMAPHORE_SYNC_INVALID;
- ring->semaphore.mbox.wait[VECS] = MI_SEMAPHORE_SYNC_BVE;
- ring->semaphore.mbox.wait[VCS2] = MI_SEMAPHORE_SYNC_INVALID;
- ring->semaphore.mbox.signal[RCS] = GEN6_RBSYNC;
- ring->semaphore.mbox.signal[VCS] = GEN6_VBSYNC;
- ring->semaphore.mbox.signal[BCS] = GEN6_NOSYNC;
- ring->semaphore.mbox.signal[VECS] = GEN6_VEBSYNC;
- ring->semaphore.mbox.signal[VCS2] = GEN6_NOSYNC;
+ engine->semaphore.mbox.wait[RCS] = MI_SEMAPHORE_SYNC_BR;
+ engine->semaphore.mbox.wait[VCS] = MI_SEMAPHORE_SYNC_BV;
+ engine->semaphore.mbox.wait[BCS] = MI_SEMAPHORE_SYNC_INVALID;
+ engine->semaphore.mbox.wait[VECS] = MI_SEMAPHORE_SYNC_BVE;
+ engine->semaphore.mbox.wait[VCS2] = MI_SEMAPHORE_SYNC_INVALID;
+ engine->semaphore.mbox.signal[RCS] = GEN6_RBSYNC;
+ engine->semaphore.mbox.signal[VCS] = GEN6_VBSYNC;
+ engine->semaphore.mbox.signal[BCS] = GEN6_NOSYNC;
+ engine->semaphore.mbox.signal[VECS] = GEN6_VEBSYNC;
+ engine->semaphore.mbox.signal[VCS2] = GEN6_NOSYNC;
}
}
- ring->init_hw = init_ring_common;
+ engine->init_hw = init_ring_common;
- return intel_init_ring_buffer(dev, ring);
+ return intel_init_ring_buffer(dev, engine);
}
int intel_init_vebox_ring_buffer(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_engine_cs *ring = &dev_priv->ring[VECS];
+ struct intel_engine_cs *engine = &dev_priv->engine[VECS];
- ring->name = "video enhancement ring";
- ring->id = VECS;
- ring->exec_id = I915_EXEC_VEBOX;
+ engine->name = "video enhancement ring";
+ engine->id = VECS;
+ engine->exec_id = I915_EXEC_VEBOX;
- ring->mmio_base = VEBOX_RING_BASE;
- ring->write_tail = ring_write_tail;
- ring->flush = gen6_ring_flush;
- ring->add_request = gen6_add_request;
- ring->get_seqno = gen6_ring_get_seqno;
- ring->set_seqno = ring_set_seqno;
+ engine->mmio_base = VEBOX_RING_BASE;
+ engine->write_tail = ring_write_tail;
+ engine->flush = gen6_ring_flush;
+ engine->add_request = gen6_add_request;
+ engine->irq_seqno_barrier = gen6_seqno_barrier;
+ engine->get_seqno = ring_get_seqno;
+ engine->set_seqno = ring_set_seqno;
if (INTEL_INFO(dev)->gen >= 8) {
- ring->irq_enable_mask =
+ engine->irq_enable_mask =
GT_RENDER_USER_INTERRUPT << GEN8_VECS_IRQ_SHIFT;
- ring->irq_get = gen8_ring_get_irq;
- ring->irq_put = gen8_ring_put_irq;
- ring->dispatch_execbuffer = gen8_ring_dispatch_execbuffer;
+ engine->irq_get = gen8_ring_get_irq;
+ engine->irq_put = gen8_ring_put_irq;
+ engine->dispatch_execbuffer = gen8_ring_dispatch_execbuffer;
if (i915_semaphore_is_enabled(dev)) {
- ring->semaphore.sync_to = gen8_ring_sync;
- ring->semaphore.signal = gen8_xcs_signal;
- GEN8_RING_SEMAPHORE_INIT;
+ engine->semaphore.sync_to = gen8_ring_sync;
+ engine->semaphore.signal = gen8_xcs_signal;
+ GEN8_RING_SEMAPHORE_INIT(engine);
}
} else {
- ring->irq_enable_mask = PM_VEBOX_USER_INTERRUPT;
- ring->irq_get = hsw_vebox_get_irq;
- ring->irq_put = hsw_vebox_put_irq;
- ring->dispatch_execbuffer = gen6_ring_dispatch_execbuffer;
+ engine->irq_enable_mask = PM_VEBOX_USER_INTERRUPT;
+ engine->irq_get = hsw_vebox_get_irq;
+ engine->irq_put = hsw_vebox_put_irq;
+ engine->dispatch_execbuffer = gen6_ring_dispatch_execbuffer;
if (i915_semaphore_is_enabled(dev)) {
- ring->semaphore.sync_to = gen6_ring_sync;
- ring->semaphore.signal = gen6_signal;
- ring->semaphore.mbox.wait[RCS] = MI_SEMAPHORE_SYNC_VER;
- ring->semaphore.mbox.wait[VCS] = MI_SEMAPHORE_SYNC_VEV;
- ring->semaphore.mbox.wait[BCS] = MI_SEMAPHORE_SYNC_VEB;
- ring->semaphore.mbox.wait[VECS] = MI_SEMAPHORE_SYNC_INVALID;
- ring->semaphore.mbox.wait[VCS2] = MI_SEMAPHORE_SYNC_INVALID;
- ring->semaphore.mbox.signal[RCS] = GEN6_RVESYNC;
- ring->semaphore.mbox.signal[VCS] = GEN6_VVESYNC;
- ring->semaphore.mbox.signal[BCS] = GEN6_BVESYNC;
- ring->semaphore.mbox.signal[VECS] = GEN6_NOSYNC;
- ring->semaphore.mbox.signal[VCS2] = GEN6_NOSYNC;
+ engine->semaphore.sync_to = gen6_ring_sync;
+ engine->semaphore.signal = gen6_signal;
+ engine->semaphore.mbox.wait[RCS] = MI_SEMAPHORE_SYNC_VER;
+ engine->semaphore.mbox.wait[VCS] = MI_SEMAPHORE_SYNC_VEV;
+ engine->semaphore.mbox.wait[BCS] = MI_SEMAPHORE_SYNC_VEB;
+ engine->semaphore.mbox.wait[VECS] = MI_SEMAPHORE_SYNC_INVALID;
+ engine->semaphore.mbox.wait[VCS2] = MI_SEMAPHORE_SYNC_INVALID;
+ engine->semaphore.mbox.signal[RCS] = GEN6_RVESYNC;
+ engine->semaphore.mbox.signal[VCS] = GEN6_VVESYNC;
+ engine->semaphore.mbox.signal[BCS] = GEN6_BVESYNC;
+ engine->semaphore.mbox.signal[VECS] = GEN6_NOSYNC;
+ engine->semaphore.mbox.signal[VCS2] = GEN6_NOSYNC;
}
}
- ring->init_hw = init_ring_common;
+ engine->init_hw = init_ring_common;
- return intel_init_ring_buffer(dev, ring);
+ return intel_init_ring_buffer(dev, engine);
}
int
intel_ring_flush_all_caches(struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
int ret;
- if (!ring->gpu_caches_dirty)
+ if (!engine->gpu_caches_dirty)
return 0;
- ret = ring->flush(req, 0, I915_GEM_GPU_DOMAINS);
+ ret = engine->flush(req, 0, I915_GEM_GPU_DOMAINS);
if (ret)
return ret;
trace_i915_gem_ring_flush(req, 0, I915_GEM_GPU_DOMAINS);
- ring->gpu_caches_dirty = false;
+ engine->gpu_caches_dirty = false;
return 0;
}
int
intel_ring_invalidate_all_caches(struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *ring = req->ring;
+ struct intel_engine_cs *engine = req->engine;
uint32_t flush_domains;
int ret;
flush_domains = 0;
- if (ring->gpu_caches_dirty)
+ if (engine->gpu_caches_dirty)
flush_domains = I915_GEM_GPU_DOMAINS;
- ret = ring->flush(req, I915_GEM_GPU_DOMAINS, flush_domains);
+ ret = engine->flush(req, I915_GEM_GPU_DOMAINS, flush_domains);
if (ret)
return ret;
trace_i915_gem_ring_flush(req, I915_GEM_GPU_DOMAINS, flush_domains);
- ring->gpu_caches_dirty = false;
+ engine->gpu_caches_dirty = false;
return 0;
}
void
-intel_stop_ring_buffer(struct intel_engine_cs *ring)
+intel_stop_engine(struct intel_engine_cs *engine)
{
int ret;
- if (!intel_ring_initialized(ring))
+ if (!intel_engine_initialized(engine))
return;
- ret = intel_ring_idle(ring);
- if (ret && !i915_reset_in_progress(&to_i915(ring->dev)->gpu_error))
+ ret = intel_engine_idle(engine);
+ if (ret)
DRM_ERROR("failed to quiesce %s whilst cleaning up: %d\n",
- ring->name, ret);
+ engine->name, ret);
- stop_ring(ring);
+ stop_ring(engine);
}
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h
index 566b0ae..2ade194 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.h
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.h
@@ -52,34 +52,32 @@
/* seqno size is actually only a uint32, but since we plan to use MI_FLUSH_DW to
* do the writes, and that must have qw aligned offsets, simply pretend it's 8b.
*/
-#define i915_semaphore_seqno_size sizeof(uint64_t)
+#define gen8_semaphore_seqno_size sizeof(uint64_t)
+#define GEN8_SEMAPHORE_OFFSET(__from, __to) \
+ (((__from) * I915_NUM_ENGINES + (__to)) * gen8_semaphore_seqno_size)
#define GEN8_SIGNAL_OFFSET(__ring, to) \
(i915_gem_obj_ggtt_offset(dev_priv->semaphore_obj) + \
- ((__ring)->id * I915_NUM_RINGS * i915_semaphore_seqno_size) + \
- (i915_semaphore_seqno_size * (to)))
-
+ GEN8_SEMAPHORE_OFFSET((__ring)->id, (to)))
#define GEN8_WAIT_OFFSET(__ring, from) \
(i915_gem_obj_ggtt_offset(dev_priv->semaphore_obj) + \
- ((from) * I915_NUM_RINGS * i915_semaphore_seqno_size) + \
- (i915_semaphore_seqno_size * (__ring)->id))
+ GEN8_SEMAPHORE_OFFSET(from, (__ring)->id))
-#define GEN8_RING_SEMAPHORE_INIT do { \
+#define GEN8_RING_SEMAPHORE_INIT(e) do { \
if (!dev_priv->semaphore_obj) { \
break; \
} \
- ring->semaphore.signal_ggtt[RCS] = GEN8_SIGNAL_OFFSET(ring, RCS); \
- ring->semaphore.signal_ggtt[VCS] = GEN8_SIGNAL_OFFSET(ring, VCS); \
- ring->semaphore.signal_ggtt[BCS] = GEN8_SIGNAL_OFFSET(ring, BCS); \
- ring->semaphore.signal_ggtt[VECS] = GEN8_SIGNAL_OFFSET(ring, VECS); \
- ring->semaphore.signal_ggtt[VCS2] = GEN8_SIGNAL_OFFSET(ring, VCS2); \
- ring->semaphore.signal_ggtt[ring->id] = MI_SEMAPHORE_SYNC_INVALID; \
+ (e)->semaphore.signal_ggtt[RCS] = GEN8_SIGNAL_OFFSET((e), RCS); \
+ (e)->semaphore.signal_ggtt[VCS] = GEN8_SIGNAL_OFFSET((e), VCS); \
+ (e)->semaphore.signal_ggtt[BCS] = GEN8_SIGNAL_OFFSET((e), BCS); \
+ (e)->semaphore.signal_ggtt[VECS] = GEN8_SIGNAL_OFFSET((e), VECS); \
+ (e)->semaphore.signal_ggtt[VCS2] = GEN8_SIGNAL_OFFSET((e), VCS2); \
+ (e)->semaphore.signal_ggtt[(e)->id] = MI_SEMAPHORE_SYNC_INVALID; \
} while(0)
enum intel_ring_hangcheck_action {
HANGCHECK_IDLE = 0,
HANGCHECK_WAIT,
HANGCHECK_ACTIVE,
- HANGCHECK_ACTIVE_LOOP,
HANGCHECK_KICK,
HANGCHECK_HUNG,
};
@@ -88,8 +86,8 @@
struct intel_ring_hangcheck {
u64 acthd;
- u64 max_acthd;
u32 seqno;
+ unsigned user_interrupts;
int score;
enum intel_ring_hangcheck_action action;
int deadlock;
@@ -101,7 +99,7 @@
void __iomem *virtual_start;
struct i915_vma *vma;
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
struct list_head link;
u32 head;
@@ -125,7 +123,7 @@
};
struct intel_context;
-struct drm_i915_reg_descriptor;
+struct drm_i915_reg_table;
/*
* we use a single page to load ctx workarounds so all of these
@@ -148,14 +146,14 @@
struct intel_engine_cs {
const char *name;
- enum intel_ring_id {
+ enum intel_engine_id {
RCS = 0,
BCS,
VCS,
VCS2, /* Keep instances of the same type engine together. */
VECS
} id;
-#define I915_NUM_RINGS 5
+#define I915_NUM_ENGINES 5
#define _VCS(n) (VCS + (n))
unsigned int exec_id;
unsigned int guc_id;
@@ -196,8 +194,8 @@
* seen value is good enough. Note that the seqno will always be
* monotonic, even if not coherent.
*/
- u32 (*get_seqno)(struct intel_engine_cs *ring,
- bool lazy_coherency);
+ void (*irq_seqno_barrier)(struct intel_engine_cs *ring);
+ u32 (*get_seqno)(struct intel_engine_cs *ring);
void (*set_seqno)(struct intel_engine_cs *ring,
u32 seqno);
int (*dispatch_execbuffer)(struct drm_i915_gem_request *req,
@@ -246,16 +244,16 @@
* ie. transpose of f(x, y)
*/
struct {
- u32 sync_seqno[I915_NUM_RINGS-1];
+ u32 sync_seqno[I915_NUM_ENGINES-1];
union {
struct {
/* our mbox written by others */
- u32 wait[I915_NUM_RINGS];
+ u32 wait[I915_NUM_ENGINES];
/* mboxes this ring signals to */
- i915_reg_t signal[I915_NUM_RINGS];
+ i915_reg_t signal[I915_NUM_ENGINES];
} mbox;
- u64 signal_ggtt[I915_NUM_RINGS];
+ u64 signal_ggtt[I915_NUM_ENGINES];
};
/* AKA wait() */
@@ -268,10 +266,13 @@
} semaphore;
/* Execlists */
- spinlock_t execlist_lock;
+ struct tasklet_struct irq_tasklet;
+ spinlock_t execlist_lock; /* used inside tasklet, use spin_lock_bh */
struct list_head execlist_queue;
struct list_head execlist_retired_req_list;
- u8 next_context_status_buffer;
+ unsigned int fw_domains;
+ unsigned int next_context_status_buffer;
+ unsigned int idle_lite_restore_wa;
bool disable_lite_restore_wa;
u32 ctx_desc_template;
u32 irq_keep_mask; /* bitmask for interrupts that should not be masked */
@@ -306,6 +307,7 @@
* inspecting request list.
*/
u32 last_submitted_seqno;
+ unsigned user_interrupts;
bool gpu_caches_dirty;
@@ -332,15 +334,8 @@
/*
* Table of registers allowed in commands that read/write registers.
*/
- const struct drm_i915_reg_descriptor *reg_table;
- int reg_count;
-
- /*
- * Table of registers allowed in commands that read/write registers, but
- * only from the DRM master.
- */
- const struct drm_i915_reg_descriptor *master_reg_table;
- int master_reg_count;
+ const struct drm_i915_reg_table *reg_tables;
+ int reg_table_count;
/*
* Returns the bitmask for the length field of the specified command.
@@ -356,19 +351,19 @@
};
static inline bool
-intel_ring_initialized(struct intel_engine_cs *ring)
+intel_engine_initialized(struct intel_engine_cs *engine)
{
- return ring->dev != NULL;
+ return engine->dev != NULL;
}
static inline unsigned
-intel_ring_flag(struct intel_engine_cs *ring)
+intel_engine_flag(struct intel_engine_cs *engine)
{
- return 1 << ring->id;
+ return 1 << engine->id;
}
static inline u32
-intel_ring_sync_index(struct intel_engine_cs *ring,
+intel_ring_sync_index(struct intel_engine_cs *engine,
struct intel_engine_cs *other)
{
int idx;
@@ -381,34 +376,33 @@
* vcs2 -> 0 = rcs, 1 = vcs, 2 = bcs, 3 = vecs;
*/
- idx = (other - ring) - 1;
+ idx = (other - engine) - 1;
if (idx < 0)
- idx += I915_NUM_RINGS;
+ idx += I915_NUM_ENGINES;
return idx;
}
static inline void
-intel_flush_status_page(struct intel_engine_cs *ring, int reg)
+intel_flush_status_page(struct intel_engine_cs *engine, int reg)
{
- drm_clflush_virt_range(&ring->status_page.page_addr[reg],
- sizeof(uint32_t));
+ mb();
+ clflush(&engine->status_page.page_addr[reg]);
+ mb();
}
static inline u32
-intel_read_status_page(struct intel_engine_cs *ring,
- int reg)
+intel_read_status_page(struct intel_engine_cs *engine, int reg)
{
/* Ensure that the compiler doesn't optimize away the load. */
- barrier();
- return ring->status_page.page_addr[reg];
+ return READ_ONCE(engine->status_page.page_addr[reg]);
}
static inline void
-intel_write_status_page(struct intel_engine_cs *ring,
+intel_write_status_page(struct intel_engine_cs *engine,
int reg, u32 value)
{
- ring->status_page.page_addr[reg] = value;
+ engine->status_page.page_addr[reg] = value;
}
/*
@@ -439,42 +433,42 @@
void intel_unpin_ringbuffer_obj(struct intel_ringbuffer *ringbuf);
void intel_ringbuffer_free(struct intel_ringbuffer *ring);
-void intel_stop_ring_buffer(struct intel_engine_cs *ring);
-void intel_cleanup_ring_buffer(struct intel_engine_cs *ring);
+void intel_stop_engine(struct intel_engine_cs *engine);
+void intel_cleanup_engine(struct intel_engine_cs *engine);
int intel_ring_alloc_request_extras(struct drm_i915_gem_request *request);
int __must_check intel_ring_begin(struct drm_i915_gem_request *req, int n);
int __must_check intel_ring_cacheline_align(struct drm_i915_gem_request *req);
-static inline void intel_ring_emit(struct intel_engine_cs *ring,
+static inline void intel_ring_emit(struct intel_engine_cs *engine,
u32 data)
{
- struct intel_ringbuffer *ringbuf = ring->buffer;
+ struct intel_ringbuffer *ringbuf = engine->buffer;
iowrite32(data, ringbuf->virtual_start + ringbuf->tail);
ringbuf->tail += 4;
}
-static inline void intel_ring_emit_reg(struct intel_engine_cs *ring,
+static inline void intel_ring_emit_reg(struct intel_engine_cs *engine,
i915_reg_t reg)
{
- intel_ring_emit(ring, i915_mmio_reg_offset(reg));
+ intel_ring_emit(engine, i915_mmio_reg_offset(reg));
}
-static inline void intel_ring_advance(struct intel_engine_cs *ring)
+static inline void intel_ring_advance(struct intel_engine_cs *engine)
{
- struct intel_ringbuffer *ringbuf = ring->buffer;
+ struct intel_ringbuffer *ringbuf = engine->buffer;
ringbuf->tail &= ringbuf->size - 1;
}
int __intel_ring_space(int head, int tail, int size);
void intel_ring_update_space(struct intel_ringbuffer *ringbuf);
int intel_ring_space(struct intel_ringbuffer *ringbuf);
-bool intel_ring_stopped(struct intel_engine_cs *ring);
+bool intel_engine_stopped(struct intel_engine_cs *engine);
-int __must_check intel_ring_idle(struct intel_engine_cs *ring);
-void intel_ring_init_seqno(struct intel_engine_cs *ring, u32 seqno);
+int __must_check intel_engine_idle(struct intel_engine_cs *engine);
+void intel_ring_init_seqno(struct intel_engine_cs *engine, u32 seqno);
int intel_ring_flush_all_caches(struct drm_i915_gem_request *req);
int intel_ring_invalidate_all_caches(struct drm_i915_gem_request *req);
-void intel_fini_pipe_control(struct intel_engine_cs *ring);
-int intel_init_pipe_control(struct intel_engine_cs *ring);
+void intel_fini_pipe_control(struct intel_engine_cs *engine);
+int intel_init_pipe_control(struct intel_engine_cs *engine);
int intel_init_render_ring_buffer(struct drm_device *dev);
int intel_init_bsd_ring_buffer(struct drm_device *dev);
@@ -482,9 +476,9 @@
int intel_init_blt_ring_buffer(struct drm_device *dev);
int intel_init_vebox_ring_buffer(struct drm_device *dev);
-u64 intel_ring_get_active_head(struct intel_engine_cs *ring);
+u64 intel_ring_get_active_head(struct intel_engine_cs *engine);
-int init_workarounds_ring(struct intel_engine_cs *ring);
+int init_workarounds_ring(struct intel_engine_cs *engine);
static inline u32 intel_ring_get_tail(struct intel_ringbuffer *ringbuf)
{
diff --git a/drivers/gpu/drm/i915/intel_runtime_pm.c b/drivers/gpu/drm/i915/intel_runtime_pm.c
index 6e54d97..7fb1da4 100644
--- a/drivers/gpu/drm/i915/intel_runtime_pm.c
+++ b/drivers/gpu/drm/i915/intel_runtime_pm.c
@@ -89,6 +89,10 @@
return "TRANSCODER_C";
case POWER_DOMAIN_TRANSCODER_EDP:
return "TRANSCODER_EDP";
+ case POWER_DOMAIN_TRANSCODER_DSI_A:
+ return "TRANSCODER_DSI_A";
+ case POWER_DOMAIN_TRANSCODER_DSI_C:
+ return "TRANSCODER_DSI_C";
case POWER_DOMAIN_PORT_DDI_A_LANES:
return "PORT_DDI_A_LANES";
case POWER_DOMAIN_PORT_DDI_B_LANES:
@@ -393,11 +397,6 @@
BIT(POWER_DOMAIN_MODESET) | \
BIT(POWER_DOMAIN_AUX_A) | \
BIT(POWER_DOMAIN_INIT))
-#define SKL_DISPLAY_ALWAYS_ON_POWER_DOMAINS ( \
- (POWER_DOMAIN_MASK & ~( \
- SKL_DISPLAY_POWERWELL_2_POWER_DOMAINS | \
- SKL_DISPLAY_DC_OFF_POWER_DOMAINS)) | \
- BIT(POWER_DOMAIN_INIT))
#define BXT_DISPLAY_POWERWELL_2_POWER_DOMAINS ( \
BIT(POWER_DOMAIN_TRANSCODER_A) | \
@@ -415,36 +414,21 @@
BIT(POWER_DOMAIN_VGA) | \
BIT(POWER_DOMAIN_GMBUS) | \
BIT(POWER_DOMAIN_INIT))
-#define BXT_DISPLAY_POWERWELL_1_POWER_DOMAINS ( \
- BXT_DISPLAY_POWERWELL_2_POWER_DOMAINS | \
- BIT(POWER_DOMAIN_PIPE_A) | \
- BIT(POWER_DOMAIN_TRANSCODER_EDP) | \
- BIT(POWER_DOMAIN_PIPE_A_PANEL_FITTER) | \
- BIT(POWER_DOMAIN_PORT_DDI_A_LANES) | \
- BIT(POWER_DOMAIN_AUX_A) | \
- BIT(POWER_DOMAIN_PLLS) | \
- BIT(POWER_DOMAIN_INIT))
#define BXT_DISPLAY_DC_OFF_POWER_DOMAINS ( \
BXT_DISPLAY_POWERWELL_2_POWER_DOMAINS | \
BIT(POWER_DOMAIN_MODESET) | \
BIT(POWER_DOMAIN_AUX_A) | \
BIT(POWER_DOMAIN_INIT))
-#define BXT_DISPLAY_ALWAYS_ON_POWER_DOMAINS ( \
- (POWER_DOMAIN_MASK & ~(BXT_DISPLAY_POWERWELL_1_POWER_DOMAINS | \
- BXT_DISPLAY_POWERWELL_2_POWER_DOMAINS)) | \
- BIT(POWER_DOMAIN_INIT))
static void assert_can_enable_dc9(struct drm_i915_private *dev_priv)
{
- struct drm_device *dev = dev_priv->dev;
-
- WARN(!IS_BROXTON(dev), "Platform doesn't support DC9.\n");
- WARN((I915_READ(DC_STATE_EN) & DC_STATE_EN_DC9),
- "DC9 already programmed to be enabled.\n");
- WARN(I915_READ(DC_STATE_EN) & DC_STATE_EN_UPTO_DC5,
- "DC5 still not disabled to enable DC9.\n");
- WARN(I915_READ(HSW_PWR_WELL_DRIVER), "Power well on.\n");
- WARN(intel_irqs_enabled(dev_priv), "Interrupts not disabled yet.\n");
+ WARN_ONCE((I915_READ(DC_STATE_EN) & DC_STATE_EN_DC9),
+ "DC9 already programmed to be enabled.\n");
+ WARN_ONCE(I915_READ(DC_STATE_EN) & DC_STATE_EN_UPTO_DC5,
+ "DC5 still not disabled to enable DC9.\n");
+ WARN_ONCE(I915_READ(HSW_PWR_WELL_DRIVER), "Power well on.\n");
+ WARN_ONCE(intel_irqs_enabled(dev_priv),
+ "Interrupts not disabled yet.\n");
/*
* TODO: check for the following to verify the conditions to enter DC9
@@ -457,11 +441,10 @@
static void assert_can_disable_dc9(struct drm_i915_private *dev_priv)
{
- WARN(intel_irqs_enabled(dev_priv), "Interrupts not disabled yet.\n");
- WARN(!(I915_READ(DC_STATE_EN) & DC_STATE_EN_DC9),
- "DC9 already programmed to be disabled.\n");
- WARN(I915_READ(DC_STATE_EN) & DC_STATE_EN_UPTO_DC5,
- "DC5 still not disabled.\n");
+ WARN_ONCE(intel_irqs_enabled(dev_priv),
+ "Interrupts not disabled yet.\n");
+ WARN_ONCE(I915_READ(DC_STATE_EN) & DC_STATE_EN_UPTO_DC5,
+ "DC5 still not disabled.\n");
/*
* TODO: check for the following to verify DC9 state was indeed
@@ -472,24 +455,6 @@
*/
}
-static void gen9_set_dc_state_debugmask(struct drm_i915_private *dev_priv)
-{
- uint32_t val, mask;
-
- mask = DC_STATE_DEBUG_MASK_MEMORY_UP;
-
- if (IS_BROXTON(dev_priv))
- mask |= DC_STATE_DEBUG_MASK_CORES;
-
- /* The below bit doesn't need to be cleared ever afterwards */
- val = I915_READ(DC_STATE_DEBUG);
- if ((val & mask) != mask) {
- val |= mask;
- I915_WRITE(DC_STATE_DEBUG, val);
- POSTING_READ(DC_STATE_DEBUG);
- }
-}
-
static void gen9_write_dc_state(struct drm_i915_private *dev_priv,
u32 state)
{
@@ -527,10 +492,9 @@
state, rewrites);
}
-static void gen9_set_dc_state(struct drm_i915_private *dev_priv, uint32_t state)
+static u32 gen9_dc_mask(struct drm_i915_private *dev_priv)
{
- uint32_t val;
- uint32_t mask;
+ u32 mask;
mask = DC_STATE_EN_UPTO_DC5;
if (IS_BROXTON(dev_priv))
@@ -538,14 +502,30 @@
else
mask |= DC_STATE_EN_UPTO_DC6;
- WARN_ON_ONCE(state & ~mask);
+ return mask;
+}
- if (i915.enable_dc == 0)
- state = DC_STATE_DISABLE;
- else if (i915.enable_dc == 1 && state > DC_STATE_EN_UPTO_DC5)
- state = DC_STATE_EN_UPTO_DC5;
+void gen9_sanitize_dc_state(struct drm_i915_private *dev_priv)
+{
+ u32 val;
+
+ val = I915_READ(DC_STATE_EN) & gen9_dc_mask(dev_priv);
+
+ DRM_DEBUG_KMS("Resetting DC state tracking from %02x to %02x\n",
+ dev_priv->csr.dc_state, val);
+ dev_priv->csr.dc_state = val;
+}
+
+static void gen9_set_dc_state(struct drm_i915_private *dev_priv, uint32_t state)
+{
+ uint32_t val;
+ uint32_t mask;
+
+ if (WARN_ON_ONCE(state & ~dev_priv->csr.allowed_dc_mask))
+ state &= dev_priv->csr.allowed_dc_mask;
val = I915_READ(DC_STATE_EN);
+ mask = gen9_dc_mask(dev_priv);
DRM_DEBUG_KMS("Setting DC state from %02x to %02x\n",
val & mask, state);
@@ -590,13 +570,9 @@
static void assert_can_enable_dc5(struct drm_i915_private *dev_priv)
{
- struct drm_device *dev = dev_priv->dev;
bool pg2_enabled = intel_display_power_well_is_enabled(dev_priv,
SKL_DISP_PW_2);
- WARN_ONCE(!IS_SKYLAKE(dev) && !IS_KABYLAKE(dev),
- "Platform doesn't support DC5.\n");
- WARN_ONCE(!HAS_RUNTIME_PM(dev), "Runtime PM not enabled.\n");
WARN_ONCE(pg2_enabled, "PG2 not disabled to enable DC5.\n");
WARN_ONCE((I915_READ(DC_STATE_EN) & DC_STATE_EN_UPTO_DC5),
@@ -606,19 +582,7 @@
assert_csr_loaded(dev_priv);
}
-static void assert_can_disable_dc5(struct drm_i915_private *dev_priv)
-{
- /*
- * During initialization, the firmware may not be loaded yet.
- * We still want to make sure that the DC enabling flag is cleared.
- */
- if (dev_priv->power_domains.initializing)
- return;
-
- assert_rpm_wakelock_held(dev_priv);
-}
-
-static void gen9_enable_dc5(struct drm_i915_private *dev_priv)
+void gen9_enable_dc5(struct drm_i915_private *dev_priv)
{
assert_can_enable_dc5(dev_priv);
@@ -629,11 +593,6 @@
static void assert_can_enable_dc6(struct drm_i915_private *dev_priv)
{
- struct drm_device *dev = dev_priv->dev;
-
- WARN_ONCE(!IS_SKYLAKE(dev) && !IS_KABYLAKE(dev),
- "Platform doesn't support DC6.\n");
- WARN_ONCE(!HAS_RUNTIME_PM(dev), "Runtime PM not enabled.\n");
WARN_ONCE(I915_READ(UTIL_PIN_CTL) & UTIL_PIN_ENABLE,
"Backlight is not disabled.\n");
WARN_ONCE((I915_READ(DC_STATE_EN) & DC_STATE_EN_UPTO_DC6),
@@ -642,30 +601,6 @@
assert_csr_loaded(dev_priv);
}
-static void assert_can_disable_dc6(struct drm_i915_private *dev_priv)
-{
- /*
- * During initialization, the firmware may not be loaded yet.
- * We still want to make sure that the DC enabling flag is cleared.
- */
- if (dev_priv->power_domains.initializing)
- return;
-
- WARN_ONCE(!(I915_READ(DC_STATE_EN) & DC_STATE_EN_UPTO_DC6),
- "DC6 already programmed to be disabled.\n");
-}
-
-static void gen9_disable_dc5_dc6(struct drm_i915_private *dev_priv)
-{
- assert_can_disable_dc5(dev_priv);
-
- if ((IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) &&
- i915.enable_dc != 0 && i915.enable_dc != 1)
- assert_can_disable_dc6(dev_priv);
-
- gen9_set_dc_state(dev_priv, DC_STATE_DISABLE);
-}
-
void skl_enable_dc6(struct drm_i915_private *dev_priv)
{
assert_can_enable_dc6(dev_priv);
@@ -678,13 +613,50 @@
void skl_disable_dc6(struct drm_i915_private *dev_priv)
{
- assert_can_disable_dc6(dev_priv);
-
DRM_DEBUG_KMS("Disabling DC6\n");
gen9_set_dc_state(dev_priv, DC_STATE_DISABLE);
}
+static void
+gen9_sanitize_power_well_requests(struct drm_i915_private *dev_priv,
+ struct i915_power_well *power_well)
+{
+ enum skl_disp_power_wells power_well_id = power_well->data;
+ u32 val;
+ u32 mask;
+
+ mask = SKL_POWER_WELL_REQ(power_well_id);
+
+ val = I915_READ(HSW_PWR_WELL_KVMR);
+ if (WARN_ONCE(val & mask, "Clearing unexpected KVMR request for %s\n",
+ power_well->name))
+ I915_WRITE(HSW_PWR_WELL_KVMR, val & ~mask);
+
+ val = I915_READ(HSW_PWR_WELL_BIOS);
+ val |= I915_READ(HSW_PWR_WELL_DEBUG);
+
+ if (!(val & mask))
+ return;
+
+ /*
+ * DMC is known to force on the request bits for power well 1 on SKL
+ * and BXT and the misc IO power well on SKL but we don't expect any
+ * other request bits to be set, so WARN for those.
+ */
+ if (power_well_id == SKL_DISP_PW_1 ||
+ ((IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) &&
+ power_well_id == SKL_DISP_PW_MISC_IO))
+ DRM_DEBUG_DRIVER("Clearing auxiliary requests for %s forced on "
+ "by DMC\n", power_well->name);
+ else
+ WARN_ONCE(1, "Clearing unexpected auxiliary requests for %s\n",
+ power_well->name);
+
+ I915_WRITE(HSW_PWR_WELL_BIOS, val & ~mask);
+ I915_WRITE(HSW_PWR_WELL_DEBUG, val & ~mask);
+}
+
static void skl_set_power_well(struct drm_i915_private *dev_priv,
struct i915_power_well *power_well, bool enable)
{
@@ -739,10 +711,6 @@
if (!is_enabled) {
DRM_DEBUG_KMS("Enabling %s\n", power_well->name);
- if (wait_for((I915_READ(HSW_PWR_WELL_DRIVER) &
- state_mask), 1))
- DRM_ERROR("%s enable timeout\n",
- power_well->name);
check_fuse_status = true;
}
} else {
@@ -751,8 +719,16 @@
POSTING_READ(HSW_PWR_WELL_DRIVER);
DRM_DEBUG_KMS("Disabling %s\n", power_well->name);
}
+
+ if (IS_GEN9(dev_priv))
+ gen9_sanitize_power_well_requests(dev_priv, power_well);
}
+ if (wait_for(!!(I915_READ(HSW_PWR_WELL_DRIVER) & state_mask) == enable,
+ 1))
+ DRM_ERROR("%s %s timeout\n",
+ power_well->name, enable ? "enable" : "disable");
+
if (check_fuse_status) {
if (power_well->data == SKL_DISP_PW_1) {
if (wait_for((I915_READ(SKL_FUSE_STATUS) &
@@ -833,32 +809,33 @@
static void gen9_dc_off_power_well_enable(struct drm_i915_private *dev_priv,
struct i915_power_well *power_well)
{
- gen9_disable_dc5_dc6(dev_priv);
+ gen9_set_dc_state(dev_priv, DC_STATE_DISABLE);
+
+ if (IS_BROXTON(dev_priv)) {
+ broxton_cdclk_verify_state(dev_priv);
+ broxton_ddi_phy_verify_state(dev_priv);
+ }
}
static void gen9_dc_off_power_well_disable(struct drm_i915_private *dev_priv,
struct i915_power_well *power_well)
{
- if ((IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) &&
- i915.enable_dc != 0 && i915.enable_dc != 1)
+ if (!dev_priv->csr.dmc_payload)
+ return;
+
+ if (dev_priv->csr.allowed_dc_mask & DC_STATE_EN_UPTO_DC6)
skl_enable_dc6(dev_priv);
- else
+ else if (dev_priv->csr.allowed_dc_mask & DC_STATE_EN_UPTO_DC5)
gen9_enable_dc5(dev_priv);
}
static void gen9_dc_off_power_well_sync_hw(struct drm_i915_private *dev_priv,
struct i915_power_well *power_well)
{
- if (power_well->count > 0) {
- gen9_set_dc_state(dev_priv, DC_STATE_DISABLE);
- } else {
- if ((IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) &&
- i915.enable_dc != 0 &&
- i915.enable_dc != 1)
- gen9_set_dc_state(dev_priv, DC_STATE_EN_UPTO_DC6);
- else
- gen9_set_dc_state(dev_priv, DC_STATE_EN_UPTO_DC5);
- }
+ if (power_well->count > 0)
+ gen9_dc_off_power_well_enable(dev_priv, power_well);
+ else
+ gen9_dc_off_power_well_disable(dev_priv, power_well);
}
static void i9xx_always_on_power_well_noop(struct drm_i915_private *dev_priv,
@@ -962,6 +939,17 @@
return enabled;
}
+static void vlv_init_display_clock_gating(struct drm_i915_private *dev_priv)
+{
+ I915_WRITE(DSPCLK_GATE_D, VRHUNIT_CLOCK_GATE_DISABLE);
+
+ /*
+ * Disable trickle feed and enable pnd deadline calculation
+ */
+ I915_WRITE(MI_ARB_VLV, MI_ARB_DISPLAY_TRICKLE_FEED_DISABLE);
+ I915_WRITE(CBR1_VLV, 0);
+}
+
static void vlv_display_power_well_init(struct drm_i915_private *dev_priv)
{
enum pipe pipe;
@@ -984,6 +972,8 @@
I915_WRITE(DPLL(pipe), val);
}
+ vlv_init_display_clock_gating(dev_priv);
+
spin_lock_irq(&dev_priv->irq_lock);
valleyview_enable_display_irqs(dev_priv);
spin_unlock_irq(&dev_priv->irq_lock);
@@ -1622,34 +1612,56 @@
intel_runtime_pm_put(dev_priv);
}
-#define HSW_ALWAYS_ON_POWER_DOMAINS ( \
- BIT(POWER_DOMAIN_PIPE_A) | \
- BIT(POWER_DOMAIN_TRANSCODER_EDP) | \
- BIT(POWER_DOMAIN_PORT_DDI_A_LANES) | \
+#define HSW_DISPLAY_POWER_DOMAINS ( \
+ BIT(POWER_DOMAIN_PIPE_B) | \
+ BIT(POWER_DOMAIN_PIPE_C) | \
+ BIT(POWER_DOMAIN_PIPE_A_PANEL_FITTER) | \
+ BIT(POWER_DOMAIN_PIPE_B_PANEL_FITTER) | \
+ BIT(POWER_DOMAIN_PIPE_C_PANEL_FITTER) | \
+ BIT(POWER_DOMAIN_TRANSCODER_A) | \
+ BIT(POWER_DOMAIN_TRANSCODER_B) | \
+ BIT(POWER_DOMAIN_TRANSCODER_C) | \
BIT(POWER_DOMAIN_PORT_DDI_B_LANES) | \
BIT(POWER_DOMAIN_PORT_DDI_C_LANES) | \
BIT(POWER_DOMAIN_PORT_DDI_D_LANES) | \
- BIT(POWER_DOMAIN_PORT_CRT) | \
- BIT(POWER_DOMAIN_PLLS) | \
- BIT(POWER_DOMAIN_AUX_A) | \
- BIT(POWER_DOMAIN_AUX_B) | \
- BIT(POWER_DOMAIN_AUX_C) | \
- BIT(POWER_DOMAIN_AUX_D) | \
- BIT(POWER_DOMAIN_GMBUS) | \
- BIT(POWER_DOMAIN_INIT))
-#define HSW_DISPLAY_POWER_DOMAINS ( \
- (POWER_DOMAIN_MASK & ~HSW_ALWAYS_ON_POWER_DOMAINS) | \
+ BIT(POWER_DOMAIN_PORT_CRT) | /* DDI E */ \
+ BIT(POWER_DOMAIN_VGA) | \
+ BIT(POWER_DOMAIN_AUDIO) | \
BIT(POWER_DOMAIN_INIT))
-#define BDW_ALWAYS_ON_POWER_DOMAINS ( \
- HSW_ALWAYS_ON_POWER_DOMAINS | \
- BIT(POWER_DOMAIN_PIPE_A_PANEL_FITTER))
-#define BDW_DISPLAY_POWER_DOMAINS ( \
- (POWER_DOMAIN_MASK & ~BDW_ALWAYS_ON_POWER_DOMAINS) | \
+#define BDW_DISPLAY_POWER_DOMAINS ( \
+ BIT(POWER_DOMAIN_PIPE_B) | \
+ BIT(POWER_DOMAIN_PIPE_C) | \
+ BIT(POWER_DOMAIN_PIPE_B_PANEL_FITTER) | \
+ BIT(POWER_DOMAIN_PIPE_C_PANEL_FITTER) | \
+ BIT(POWER_DOMAIN_TRANSCODER_A) | \
+ BIT(POWER_DOMAIN_TRANSCODER_B) | \
+ BIT(POWER_DOMAIN_TRANSCODER_C) | \
+ BIT(POWER_DOMAIN_PORT_DDI_B_LANES) | \
+ BIT(POWER_DOMAIN_PORT_DDI_C_LANES) | \
+ BIT(POWER_DOMAIN_PORT_DDI_D_LANES) | \
+ BIT(POWER_DOMAIN_PORT_CRT) | /* DDI E */ \
+ BIT(POWER_DOMAIN_VGA) | \
+ BIT(POWER_DOMAIN_AUDIO) | \
BIT(POWER_DOMAIN_INIT))
-#define VLV_ALWAYS_ON_POWER_DOMAINS BIT(POWER_DOMAIN_INIT)
-#define VLV_DISPLAY_POWER_DOMAINS POWER_DOMAIN_MASK
+#define VLV_DISPLAY_POWER_DOMAINS ( \
+ BIT(POWER_DOMAIN_PIPE_A) | \
+ BIT(POWER_DOMAIN_PIPE_B) | \
+ BIT(POWER_DOMAIN_PIPE_A_PANEL_FITTER) | \
+ BIT(POWER_DOMAIN_PIPE_B_PANEL_FITTER) | \
+ BIT(POWER_DOMAIN_TRANSCODER_A) | \
+ BIT(POWER_DOMAIN_TRANSCODER_B) | \
+ BIT(POWER_DOMAIN_PORT_DDI_B_LANES) | \
+ BIT(POWER_DOMAIN_PORT_DDI_C_LANES) | \
+ BIT(POWER_DOMAIN_PORT_DSI) | \
+ BIT(POWER_DOMAIN_PORT_CRT) | \
+ BIT(POWER_DOMAIN_VGA) | \
+ BIT(POWER_DOMAIN_AUDIO) | \
+ BIT(POWER_DOMAIN_AUX_B) | \
+ BIT(POWER_DOMAIN_AUX_C) | \
+ BIT(POWER_DOMAIN_GMBUS) | \
+ BIT(POWER_DOMAIN_INIT))
#define VLV_DPIO_CMN_BC_POWER_DOMAINS ( \
BIT(POWER_DOMAIN_PORT_DDI_B_LANES) | \
@@ -1679,6 +1691,28 @@
BIT(POWER_DOMAIN_AUX_C) | \
BIT(POWER_DOMAIN_INIT))
+#define CHV_DISPLAY_POWER_DOMAINS ( \
+ BIT(POWER_DOMAIN_PIPE_A) | \
+ BIT(POWER_DOMAIN_PIPE_B) | \
+ BIT(POWER_DOMAIN_PIPE_C) | \
+ BIT(POWER_DOMAIN_PIPE_A_PANEL_FITTER) | \
+ BIT(POWER_DOMAIN_PIPE_B_PANEL_FITTER) | \
+ BIT(POWER_DOMAIN_PIPE_C_PANEL_FITTER) | \
+ BIT(POWER_DOMAIN_TRANSCODER_A) | \
+ BIT(POWER_DOMAIN_TRANSCODER_B) | \
+ BIT(POWER_DOMAIN_TRANSCODER_C) | \
+ BIT(POWER_DOMAIN_PORT_DDI_B_LANES) | \
+ BIT(POWER_DOMAIN_PORT_DDI_C_LANES) | \
+ BIT(POWER_DOMAIN_PORT_DDI_D_LANES) | \
+ BIT(POWER_DOMAIN_PORT_DSI) | \
+ BIT(POWER_DOMAIN_VGA) | \
+ BIT(POWER_DOMAIN_AUDIO) | \
+ BIT(POWER_DOMAIN_AUX_B) | \
+ BIT(POWER_DOMAIN_AUX_C) | \
+ BIT(POWER_DOMAIN_AUX_D) | \
+ BIT(POWER_DOMAIN_GMBUS) | \
+ BIT(POWER_DOMAIN_INIT))
+
#define CHV_DPIO_CMN_BC_POWER_DOMAINS ( \
BIT(POWER_DOMAIN_PORT_DDI_B_LANES) | \
BIT(POWER_DOMAIN_PORT_DDI_C_LANES) | \
@@ -1746,7 +1780,7 @@
{
.name = "always-on",
.always_on = 1,
- .domains = HSW_ALWAYS_ON_POWER_DOMAINS,
+ .domains = POWER_DOMAIN_MASK,
.ops = &i9xx_always_on_power_well_ops,
},
{
@@ -1760,7 +1794,7 @@
{
.name = "always-on",
.always_on = 1,
- .domains = BDW_ALWAYS_ON_POWER_DOMAINS,
+ .domains = POWER_DOMAIN_MASK,
.ops = &i9xx_always_on_power_well_ops,
},
{
@@ -1795,7 +1829,7 @@
{
.name = "always-on",
.always_on = 1,
- .domains = VLV_ALWAYS_ON_POWER_DOMAINS,
+ .domains = POWER_DOMAIN_MASK,
.ops = &i9xx_always_on_power_well_ops,
.data = PUNIT_POWER_WELL_ALWAYS_ON,
},
@@ -1853,7 +1887,7 @@
{
.name = "always-on",
.always_on = 1,
- .domains = VLV_ALWAYS_ON_POWER_DOMAINS,
+ .domains = POWER_DOMAIN_MASK,
.ops = &i9xx_always_on_power_well_ops,
},
{
@@ -1863,7 +1897,7 @@
* power wells don't actually exist. Pipe A power well is
* required for any pipe to work.
*/
- .domains = VLV_DISPLAY_POWER_DOMAINS,
+ .domains = CHV_DISPLAY_POWER_DOMAINS,
.data = PIPE_A,
.ops = &chv_pipe_power_well_ops,
},
@@ -1897,7 +1931,7 @@
{
.name = "always-on",
.always_on = 1,
- .domains = SKL_DISPLAY_ALWAYS_ON_POWER_DOMAINS,
+ .domains = POWER_DOMAIN_MASK,
.ops = &i9xx_always_on_power_well_ops,
.data = SKL_DISP_PW_ALWAYS_ON,
},
@@ -1953,44 +1987,16 @@
},
};
-void skl_pw1_misc_io_init(struct drm_i915_private *dev_priv)
-{
- struct i915_power_well *well;
-
- if (!(IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)))
- return;
-
- well = lookup_power_well(dev_priv, SKL_DISP_PW_1);
- intel_power_well_enable(dev_priv, well);
-
- well = lookup_power_well(dev_priv, SKL_DISP_PW_MISC_IO);
- intel_power_well_enable(dev_priv, well);
-}
-
-void skl_pw1_misc_io_fini(struct drm_i915_private *dev_priv)
-{
- struct i915_power_well *well;
-
- if (!(IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)))
- return;
-
- well = lookup_power_well(dev_priv, SKL_DISP_PW_1);
- intel_power_well_disable(dev_priv, well);
-
- well = lookup_power_well(dev_priv, SKL_DISP_PW_MISC_IO);
- intel_power_well_disable(dev_priv, well);
-}
-
static struct i915_power_well bxt_power_wells[] = {
{
.name = "always-on",
.always_on = 1,
- .domains = BXT_DISPLAY_ALWAYS_ON_POWER_DOMAINS,
+ .domains = POWER_DOMAIN_MASK,
.ops = &i9xx_always_on_power_well_ops,
},
{
.name = "power well 1",
- .domains = BXT_DISPLAY_POWERWELL_1_POWER_DOMAINS,
+ .domains = 0,
.ops = &skl_power_well_ops,
.data = SKL_DISP_PW_1,
},
@@ -2015,12 +2021,56 @@
if (disable_power_well >= 0)
return !!disable_power_well;
- if (IS_BROXTON(dev_priv)) {
- DRM_DEBUG_KMS("Disabling display power well support\n");
- return 0;
+ return 1;
+}
+
+static uint32_t get_allowed_dc_mask(const struct drm_i915_private *dev_priv,
+ int enable_dc)
+{
+ uint32_t mask;
+ int requested_dc;
+ int max_dc;
+
+ if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) {
+ max_dc = 2;
+ mask = 0;
+ } else if (IS_BROXTON(dev_priv)) {
+ max_dc = 1;
+ /*
+ * DC9 has a separate HW flow from the rest of the DC states,
+ * not depending on the DMC firmware. It's needed by system
+ * suspend/resume, so allow it unconditionally.
+ */
+ mask = DC_STATE_EN_DC9;
+ } else {
+ max_dc = 0;
+ mask = 0;
}
- return 1;
+ if (!i915.disable_power_well)
+ max_dc = 0;
+
+ if (enable_dc >= 0 && enable_dc <= max_dc) {
+ requested_dc = enable_dc;
+ } else if (enable_dc == -1) {
+ requested_dc = max_dc;
+ } else if (enable_dc > max_dc && enable_dc <= 2) {
+ DRM_DEBUG_KMS("Adjusting requested max DC state (%d->%d)\n",
+ enable_dc, max_dc);
+ requested_dc = max_dc;
+ } else {
+ DRM_ERROR("Unexpected value for enable_dc (%d)\n", enable_dc);
+ requested_dc = max_dc;
+ }
+
+ if (requested_dc > 1)
+ mask |= DC_STATE_EN_UPTO_DC6;
+ if (requested_dc > 0)
+ mask |= DC_STATE_EN_UPTO_DC5;
+
+ DRM_DEBUG_KMS("Allowed DC state mask %02x\n", mask);
+
+ return mask;
}
#define set_power_wells(power_domains, __power_wells) ({ \
@@ -2041,6 +2091,8 @@
i915.disable_power_well = sanitize_disable_power_well_option(dev_priv,
i915.disable_power_well);
+ dev_priv->csr.allowed_dc_mask = get_allowed_dc_mask(dev_priv,
+ i915.enable_dc);
BUILD_BUG_ON(POWER_DOMAIN_NUM > 31);
@@ -2050,17 +2102,17 @@
* The enabling order will be from lower to higher indexed wells,
* the disabling order is reversed.
*/
- if (IS_HASWELL(dev_priv->dev)) {
+ if (IS_HASWELL(dev_priv)) {
set_power_wells(power_domains, hsw_power_wells);
- } else if (IS_BROADWELL(dev_priv->dev)) {
+ } else if (IS_BROADWELL(dev_priv)) {
set_power_wells(power_domains, bdw_power_wells);
- } else if (IS_SKYLAKE(dev_priv->dev) || IS_KABYLAKE(dev_priv->dev)) {
+ } else if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv)) {
set_power_wells(power_domains, skl_power_wells);
- } else if (IS_BROXTON(dev_priv->dev)) {
+ } else if (IS_BROXTON(dev_priv)) {
set_power_wells(power_domains, bxt_power_wells);
- } else if (IS_CHERRYVIEW(dev_priv->dev)) {
+ } else if (IS_CHERRYVIEW(dev_priv)) {
set_power_wells(power_domains, chv_power_wells);
- } else if (IS_VALLEYVIEW(dev_priv->dev)) {
+ } else if (IS_VALLEYVIEW(dev_priv)) {
set_power_wells(power_domains, vlv_power_wells);
} else {
set_power_wells(power_domains, i9xx_always_on_power_well);
@@ -2120,9 +2172,10 @@
}
static void skl_display_core_init(struct drm_i915_private *dev_priv,
- bool resume)
+ bool resume)
{
struct i915_power_domains *power_domains = &dev_priv->power_domains;
+ struct i915_power_well *well;
uint32_t val;
gen9_set_dc_state(dev_priv, DC_STATE_DISABLE);
@@ -2133,7 +2186,13 @@
/* enable PG1 and Misc I/O */
mutex_lock(&power_domains->lock);
- skl_pw1_misc_io_init(dev_priv);
+
+ well = lookup_power_well(dev_priv, SKL_DISP_PW_1);
+ intel_power_well_enable(dev_priv, well);
+
+ well = lookup_power_well(dev_priv, SKL_DISP_PW_MISC_IO);
+ intel_power_well_enable(dev_priv, well);
+
mutex_unlock(&power_domains->lock);
if (!resume)
@@ -2141,13 +2200,14 @@
skl_init_cdclk(dev_priv);
- if (dev_priv->csr.dmc_payload && intel_csr_load_program(dev_priv))
- gen9_set_dc_state_debugmask(dev_priv);
+ if (dev_priv->csr.dmc_payload)
+ intel_csr_load_program(dev_priv);
}
static void skl_display_core_uninit(struct drm_i915_private *dev_priv)
{
struct i915_power_domains *power_domains = &dev_priv->power_domains;
+ struct i915_power_well *well;
gen9_set_dc_state(dev_priv, DC_STATE_DISABLE);
@@ -2155,8 +2215,73 @@
/* The spec doesn't call for removing the reset handshake flag */
/* disable PG1 and Misc I/O */
+
mutex_lock(&power_domains->lock);
- skl_pw1_misc_io_fini(dev_priv);
+
+ well = lookup_power_well(dev_priv, SKL_DISP_PW_MISC_IO);
+ intel_power_well_disable(dev_priv, well);
+
+ well = lookup_power_well(dev_priv, SKL_DISP_PW_1);
+ intel_power_well_disable(dev_priv, well);
+
+ mutex_unlock(&power_domains->lock);
+}
+
+void bxt_display_core_init(struct drm_i915_private *dev_priv,
+ bool resume)
+{
+ struct i915_power_domains *power_domains = &dev_priv->power_domains;
+ struct i915_power_well *well;
+ uint32_t val;
+
+ gen9_set_dc_state(dev_priv, DC_STATE_DISABLE);
+
+ /*
+ * NDE_RSTWRN_OPT RST PCH Handshake En must always be 0b on BXT
+ * or else the reset will hang because there is no PCH to respond.
+ * Move the handshake programming to initialization sequence.
+ * Previously was left up to BIOS.
+ */
+ val = I915_READ(HSW_NDE_RSTWRN_OPT);
+ val &= ~RESET_PCH_HANDSHAKE_ENABLE;
+ I915_WRITE(HSW_NDE_RSTWRN_OPT, val);
+
+ /* Enable PG1 */
+ mutex_lock(&power_domains->lock);
+
+ well = lookup_power_well(dev_priv, SKL_DISP_PW_1);
+ intel_power_well_enable(dev_priv, well);
+
+ mutex_unlock(&power_domains->lock);
+
+ broxton_init_cdclk(dev_priv);
+ broxton_ddi_phy_init(dev_priv);
+
+ broxton_cdclk_verify_state(dev_priv);
+ broxton_ddi_phy_verify_state(dev_priv);
+
+ if (resume && dev_priv->csr.dmc_payload)
+ intel_csr_load_program(dev_priv);
+}
+
+void bxt_display_core_uninit(struct drm_i915_private *dev_priv)
+{
+ struct i915_power_domains *power_domains = &dev_priv->power_domains;
+ struct i915_power_well *well;
+
+ gen9_set_dc_state(dev_priv, DC_STATE_DISABLE);
+
+ broxton_ddi_phy_uninit(dev_priv);
+ broxton_uninit_cdclk(dev_priv);
+
+ /* The spec doesn't call for removing the reset handshake flag */
+
+ /* Disable PG1 */
+ mutex_lock(&power_domains->lock);
+
+ well = lookup_power_well(dev_priv, SKL_DISP_PW_1);
+ intel_power_well_disable(dev_priv, well);
+
mutex_unlock(&power_domains->lock);
}
@@ -2291,6 +2416,8 @@
if (IS_SKYLAKE(dev) || IS_KABYLAKE(dev)) {
skl_display_core_init(dev_priv, resume);
+ } else if (IS_BROXTON(dev)) {
+ bxt_display_core_init(dev_priv, resume);
} else if (IS_CHERRYVIEW(dev)) {
mutex_lock(&power_domains->lock);
chv_phy_control_init(dev_priv);
@@ -2328,6 +2455,8 @@
if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv))
skl_display_core_uninit(dev_priv);
+ else if (IS_BROXTON(dev_priv))
+ bxt_display_core_uninit(dev_priv);
}
/**
diff --git a/drivers/gpu/drm/i915/intel_sdvo.c b/drivers/gpu/drm/i915/intel_sdvo.c
index 4ecc076..2128fae 100644
--- a/drivers/gpu/drm/i915/intel_sdvo.c
+++ b/drivers/gpu/drm/i915/intel_sdvo.c
@@ -1398,12 +1398,10 @@
}
dotclock = pipe_config->port_clock;
+
if (pipe_config->pixel_multiplier)
dotclock /= pipe_config->pixel_multiplier;
- if (HAS_PCH_SPLIT(dev))
- ironlake_check_encoder_dotclock(pipe_config, dotclock);
-
pipe_config->base.adjusted_mode.crtc_clock = dotclock;
/* Cross check the port pixel multiplier with the sdvo encoder state. */
@@ -2262,9 +2260,9 @@
struct sdvo_device_mapping *mapping;
if (sdvo->port == PORT_B)
- mapping = &(dev_priv->sdvo_mappings[0]);
+ mapping = &dev_priv->vbt.sdvo_mappings[0];
else
- mapping = &(dev_priv->sdvo_mappings[1]);
+ mapping = &dev_priv->vbt.sdvo_mappings[1];
if (mapping->initialized)
sdvo->ddc_bus = 1 << ((mapping->ddc_pin & 0xf0) >> 4);
@@ -2280,9 +2278,9 @@
u8 pin;
if (sdvo->port == PORT_B)
- mapping = &dev_priv->sdvo_mappings[0];
+ mapping = &dev_priv->vbt.sdvo_mappings[0];
else
- mapping = &dev_priv->sdvo_mappings[1];
+ mapping = &dev_priv->vbt.sdvo_mappings[1];
if (mapping->initialized &&
intel_gmbus_is_valid_pin(dev_priv, mapping->i2c_pin))
@@ -2318,11 +2316,11 @@
struct sdvo_device_mapping *my_mapping, *other_mapping;
if (sdvo->port == PORT_B) {
- my_mapping = &dev_priv->sdvo_mappings[0];
- other_mapping = &dev_priv->sdvo_mappings[1];
+ my_mapping = &dev_priv->vbt.sdvo_mappings[0];
+ other_mapping = &dev_priv->vbt.sdvo_mappings[1];
} else {
- my_mapping = &dev_priv->sdvo_mappings[1];
- other_mapping = &dev_priv->sdvo_mappings[0];
+ my_mapping = &dev_priv->vbt.sdvo_mappings[1];
+ other_mapping = &dev_priv->vbt.sdvo_mappings[0];
}
/* If the BIOS described our SDVO device, take advantage of it. */
diff --git a/drivers/gpu/drm/i915/intel_sprite.c b/drivers/gpu/drm/i915/intel_sprite.c
index a2582c4..0f3e230 100644
--- a/drivers/gpu/drm/i915/intel_sprite.c
+++ b/drivers/gpu/drm/i915/intel_sprite.c
@@ -193,7 +193,7 @@
const struct drm_intel_sprite_colorkey *key = &plane_state->ckey;
u32 surf_addr;
u32 tile_height, plane_offset, plane_size;
- unsigned int rotation;
+ unsigned int rotation = plane_state->base.rotation;
int x_offset, y_offset;
int crtc_x = plane_state->dst.x1;
int crtc_y = plane_state->dst.y1;
@@ -213,7 +213,6 @@
plane_ctl |= skl_plane_ctl_format(fb->pixel_format);
plane_ctl |= skl_plane_ctl_tiling(fb->modifier[0]);
- rotation = plane_state->base.rotation;
plane_ctl |= skl_plane_ctl_rotation(rotation);
stride_div = intel_fb_stride_alignment(dev_priv, fb->modifier[0],
@@ -351,6 +350,7 @@
int plane = intel_plane->plane;
u32 sprctl;
u32 sprsurf_offset, linear_offset;
+ unsigned int rotation = dplane->state->rotation;
int cpp = drm_format_plane_cpp(fb->pixel_format, 0);
const struct drm_intel_sprite_colorkey *key = &plane_state->ckey;
int crtc_x = plane_state->dst.x1;
@@ -423,12 +423,11 @@
crtc_h--;
linear_offset = y * fb->pitches[0] + x * cpp;
- sprsurf_offset = intel_compute_tile_offset(dev_priv, &x, &y,
- fb->modifier[0], cpp,
- fb->pitches[0]);
+ sprsurf_offset = intel_compute_tile_offset(&x, &y, fb, 0,
+ fb->pitches[0], rotation);
linear_offset -= sprsurf_offset;
- if (plane_state->base.rotation == BIT(DRM_ROTATE_180)) {
+ if (rotation == BIT(DRM_ROTATE_180)) {
sprctl |= SP_ROTATE_180;
x += src_w;
@@ -493,6 +492,7 @@
enum pipe pipe = intel_plane->pipe;
u32 sprctl, sprscale = 0;
u32 sprsurf_offset, linear_offset;
+ unsigned int rotation = plane_state->base.rotation;
int cpp = drm_format_plane_cpp(fb->pixel_format, 0);
const struct drm_intel_sprite_colorkey *key = &plane_state->ckey;
int crtc_x = plane_state->dst.x1;
@@ -556,12 +556,11 @@
sprscale = SPRITE_SCALE_ENABLE | (src_w << 16) | src_h;
linear_offset = y * fb->pitches[0] + x * cpp;
- sprsurf_offset = intel_compute_tile_offset(dev_priv, &x, &y,
- fb->modifier[0], cpp,
- fb->pitches[0]);
+ sprsurf_offset = intel_compute_tile_offset(&x, &y, fb, 0,
+ fb->pitches[0], rotation);
linear_offset -= sprsurf_offset;
- if (plane_state->base.rotation == BIT(DRM_ROTATE_180)) {
+ if (rotation == BIT(DRM_ROTATE_180)) {
sprctl |= SPRITE_ROTATE_180;
/* HSW and BDW does this automagically in hardware */
@@ -634,6 +633,7 @@
int pipe = intel_plane->pipe;
u32 dvscntr, dvsscale;
u32 dvssurf_offset, linear_offset;
+ unsigned int rotation = plane_state->base.rotation;
int cpp = drm_format_plane_cpp(fb->pixel_format, 0);
const struct drm_intel_sprite_colorkey *key = &plane_state->ckey;
int crtc_x = plane_state->dst.x1;
@@ -693,12 +693,11 @@
dvsscale = DVS_SCALE_ENABLE | (src_w << 16) | src_h;
linear_offset = y * fb->pitches[0] + x * cpp;
- dvssurf_offset = intel_compute_tile_offset(dev_priv, &x, &y,
- fb->modifier[0], cpp,
- fb->pitches[0]);
+ dvssurf_offset = intel_compute_tile_offset(&x, &y, fb, 0,
+ fb->pitches[0], rotation);
linear_offset -= dvssurf_offset;
- if (plane_state->base.rotation == BIT(DRM_ROTATE_180)) {
+ if (rotation == BIT(DRM_ROTATE_180)) {
dvscntr |= DVS_ROTATE_180;
x += src_w;
@@ -1026,8 +1025,8 @@
int
intel_plane_init(struct drm_device *dev, enum pipe pipe, int plane)
{
- struct intel_plane *intel_plane;
- struct intel_plane_state *state;
+ struct intel_plane *intel_plane = NULL;
+ struct intel_plane_state *state = NULL;
unsigned long possible_crtcs;
const uint32_t *plane_formats;
int num_plane_formats;
@@ -1037,13 +1036,15 @@
return -ENODEV;
intel_plane = kzalloc(sizeof(*intel_plane), GFP_KERNEL);
- if (!intel_plane)
- return -ENOMEM;
+ if (!intel_plane) {
+ ret = -ENOMEM;
+ goto fail;
+ }
state = intel_create_plane_state(&intel_plane->base);
if (!state) {
- kfree(intel_plane);
- return -ENOMEM;
+ ret = -ENOMEM;
+ goto fail;
}
intel_plane->base.state = &state->base;
@@ -1098,28 +1099,34 @@
num_plane_formats = ARRAY_SIZE(skl_plane_formats);
break;
default:
- kfree(intel_plane);
- return -ENODEV;
+ MISSING_CASE(INTEL_INFO(dev)->gen);
+ ret = -ENODEV;
+ goto fail;
}
intel_plane->pipe = pipe;
intel_plane->plane = plane;
intel_plane->frontbuffer_bit = INTEL_FRONTBUFFER_SPRITE(pipe, plane);
intel_plane->check_plane = intel_check_sprite_plane;
+
possible_crtcs = (1 << pipe);
+
ret = drm_universal_plane_init(dev, &intel_plane->base, possible_crtcs,
&intel_plane_funcs,
plane_formats, num_plane_formats,
DRM_PLANE_TYPE_OVERLAY, NULL);
- if (ret) {
- kfree(intel_plane);
- goto out;
- }
+ if (ret)
+ goto fail;
intel_create_rotation_property(dev, intel_plane);
drm_plane_helper_add(&intel_plane->base, &intel_plane_helper_funcs);
-out:
+ return 0;
+
+fail:
+ kfree(state);
+ kfree(intel_plane);
+
return ret;
}
diff --git a/drivers/gpu/drm/i915/intel_tv.c b/drivers/gpu/drm/i915/intel_tv.c
index 6745bad..223129d 100644
--- a/drivers/gpu/drm/i915/intel_tv.c
+++ b/drivers/gpu/drm/i915/intel_tv.c
@@ -326,24 +326,12 @@
.rv = 0x0100, .gv = 0x03ad, .bv = 0x074d, .av = 0x0200,
};
-static const struct color_conversion sdtv_csc_rgb = {
- .ry = 0x0000, .gy = 0x0f00, .by = 0x0000, .ay = 0x0166,
- .ru = 0x0000, .gu = 0x0000, .bu = 0x0f00, .au = 0x0166,
- .rv = 0x0f00, .gv = 0x0000, .bv = 0x0000, .av = 0x0166,
-};
-
static const struct color_conversion hdtv_csc_yprpb = {
.ry = 0x05b3, .gy = 0x016e, .by = 0x0728, .ay = 0x0145,
.ru = 0x07d5, .gu = 0x038b, .bu = 0x0100, .au = 0x0200,
.rv = 0x0100, .gv = 0x03d1, .bv = 0x06bc, .av = 0x0200,
};
-static const struct color_conversion hdtv_csc_rgb = {
- .ry = 0x0000, .gy = 0x0f00, .by = 0x0000, .ay = 0x0166,
- .ru = 0x0000, .gu = 0x0000, .bu = 0x0f00, .au = 0x0166,
- .rv = 0x0f00, .gv = 0x0000, .bv = 0x0000, .av = 0x0166,
-};
-
static const struct video_levels component_levels = {
.blank = 279, .black = 279, .burst = 0,
};
@@ -1531,47 +1519,6 @@
.destroy = intel_encoder_destroy,
};
-/*
- * Enumerate the child dev array parsed from VBT to check whether
- * the integrated TV is present.
- * If it is present, return 1.
- * If it is not present, return false.
- * If no child dev is parsed from VBT, it assumes that the TV is present.
- */
-static int tv_is_present_in_vbt(struct drm_device *dev)
-{
- struct drm_i915_private *dev_priv = dev->dev_private;
- union child_device_config *p_child;
- int i, ret;
-
- if (!dev_priv->vbt.child_dev_num)
- return 1;
-
- ret = 0;
- for (i = 0; i < dev_priv->vbt.child_dev_num; i++) {
- p_child = dev_priv->vbt.child_dev + i;
- /*
- * If the device type is not TV, continue.
- */
- switch (p_child->old.device_type) {
- case DEVICE_TYPE_INT_TV:
- case DEVICE_TYPE_TV:
- case DEVICE_TYPE_TV_SVIDEO_COMPOSITE:
- break;
- default:
- continue;
- }
- /* Only when the addin_offset is non-zero, it is regarded
- * as present.
- */
- if (p_child->old.addin_offset) {
- ret = 1;
- break;
- }
- }
- return ret;
-}
-
void
intel_tv_init(struct drm_device *dev)
{
@@ -1587,13 +1534,10 @@
if ((I915_READ(TV_CTL) & TV_FUSE_STATE_MASK) == TV_FUSE_STATE_DISABLED)
return;
- if (!tv_is_present_in_vbt(dev)) {
+ if (!intel_bios_is_tv_present(dev_priv)) {
DRM_DEBUG_KMS("Integrated TV is not present.\n");
return;
}
- /* Even if we have an encoder we may not have a connector */
- if (!dev_priv->vbt.int_tv_support)
- return;
/*
* Sanity check the TV output by checking to see if the
diff --git a/drivers/gpu/drm/i915/intel_uncore.c b/drivers/gpu/drm/i915/intel_uncore.c
index 68b6f69..4f1dfe6 100644
--- a/drivers/gpu/drm/i915/intel_uncore.c
+++ b/drivers/gpu/drm/i915/intel_uncore.c
@@ -60,7 +60,11 @@
static inline void
fw_domain_arm_timer(struct intel_uncore_forcewake_domain *d)
{
- mod_timer_pinned(&d->timer, jiffies + 1);
+ d->wake_count++;
+ hrtimer_start_range_ns(&d->timer,
+ ktime_set(0, NSEC_PER_MSEC),
+ NSEC_PER_MSEC,
+ HRTIMER_MODE_REL);
}
static inline void
@@ -107,22 +111,22 @@
fw_domains_get(struct drm_i915_private *dev_priv, enum forcewake_domains fw_domains)
{
struct intel_uncore_forcewake_domain *d;
- enum forcewake_domain_id id;
- for_each_fw_domain_mask(d, fw_domains, dev_priv, id) {
+ for_each_fw_domain_masked(d, fw_domains, dev_priv) {
fw_domain_wait_ack_clear(d);
fw_domain_get(d);
- fw_domain_wait_ack(d);
}
+
+ for_each_fw_domain_masked(d, fw_domains, dev_priv)
+ fw_domain_wait_ack(d);
}
static void
fw_domains_put(struct drm_i915_private *dev_priv, enum forcewake_domains fw_domains)
{
struct intel_uncore_forcewake_domain *d;
- enum forcewake_domain_id id;
- for_each_fw_domain_mask(d, fw_domains, dev_priv, id) {
+ for_each_fw_domain_masked(d, fw_domains, dev_priv) {
fw_domain_put(d);
fw_domain_posting_read(d);
}
@@ -132,10 +136,9 @@
fw_domains_posting_read(struct drm_i915_private *dev_priv)
{
struct intel_uncore_forcewake_domain *d;
- enum forcewake_domain_id id;
/* No need to do for all, just do for first found */
- for_each_fw_domain(d, dev_priv, id) {
+ for_each_fw_domain(d, dev_priv) {
fw_domain_posting_read(d);
break;
}
@@ -145,12 +148,11 @@
fw_domains_reset(struct drm_i915_private *dev_priv, enum forcewake_domains fw_domains)
{
struct intel_uncore_forcewake_domain *d;
- enum forcewake_domain_id id;
if (dev_priv->uncore.fw_domains == 0)
return;
- for_each_fw_domain_mask(d, fw_domains, dev_priv, id)
+ for_each_fw_domain_masked(d, fw_domains, dev_priv)
fw_domain_reset(d);
fw_domains_posting_read(dev_priv);
@@ -204,7 +206,7 @@
/* On VLV, FIFO will be shared by both SW and HW.
* So, we need to read the FREE_ENTRIES everytime */
- if (IS_VALLEYVIEW(dev_priv->dev))
+ if (IS_VALLEYVIEW(dev_priv))
dev_priv->uncore.fifo_count = fifo_free_entries(dev_priv);
if (dev_priv->uncore.fifo_count < GT_FIFO_NUM_RESERVED_ENTRIES) {
@@ -224,9 +226,11 @@
return ret;
}
-static void intel_uncore_fw_release_timer(unsigned long arg)
+static enum hrtimer_restart
+intel_uncore_fw_release_timer(struct hrtimer *timer)
{
- struct intel_uncore_forcewake_domain *domain = (void *)arg;
+ struct intel_uncore_forcewake_domain *domain =
+ container_of(timer, struct intel_uncore_forcewake_domain, timer);
unsigned long irqflags;
assert_rpm_device_not_suspended(domain->i915);
@@ -240,6 +244,8 @@
1 << domain->id);
spin_unlock_irqrestore(&domain->i915->uncore.lock, irqflags);
+
+ return HRTIMER_NORESTART;
}
void intel_uncore_forcewake_reset(struct drm_device *dev, bool restore)
@@ -248,7 +254,6 @@
unsigned long irqflags;
struct intel_uncore_forcewake_domain *domain;
int retry_count = 100;
- enum forcewake_domain_id id;
enum forcewake_domains fw = 0, active_domains;
/* Hold uncore.lock across reset to prevent any register access
@@ -258,18 +263,18 @@
while (1) {
active_domains = 0;
- for_each_fw_domain(domain, dev_priv, id) {
- if (del_timer_sync(&domain->timer) == 0)
+ for_each_fw_domain(domain, dev_priv) {
+ if (hrtimer_cancel(&domain->timer) == 0)
continue;
- intel_uncore_fw_release_timer((unsigned long)domain);
+ intel_uncore_fw_release_timer(&domain->timer);
}
spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
- for_each_fw_domain(domain, dev_priv, id) {
- if (timer_pending(&domain->timer))
- active_domains |= (1 << id);
+ for_each_fw_domain(domain, dev_priv) {
+ if (hrtimer_active(&domain->timer))
+ active_domains |= domain->mask;
}
if (active_domains == 0)
@@ -286,9 +291,9 @@
WARN_ON(active_domains);
- for_each_fw_domain(domain, dev_priv, id)
+ for_each_fw_domain(domain, dev_priv)
if (domain->wake_count)
- fw |= 1 << id;
+ fw |= domain->mask;
if (fw)
dev_priv->uncore.funcs.force_wake_put(dev_priv, fw);
@@ -310,21 +315,49 @@
spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
}
-static void intel_uncore_ellc_detect(struct drm_device *dev)
+static u64 gen9_edram_size(struct drm_i915_private *dev_priv)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
+ const unsigned int ways[8] = { 4, 8, 12, 16, 16, 16, 16, 16 };
+ const unsigned int sets[4] = { 1, 1, 2, 2 };
+ const u32 cap = dev_priv->edram_cap;
- if ((IS_HASWELL(dev) || IS_BROADWELL(dev) ||
- INTEL_INFO(dev)->gen >= 9) &&
- (__raw_i915_read32(dev_priv, HSW_EDRAM_PRESENT) & EDRAM_ENABLED)) {
- /* The docs do not explain exactly how the calculation can be
- * made. It is somewhat guessable, but for now, it's always
- * 128MB.
- * NB: We can't write IDICR yet because we do not have gt funcs
+ return EDRAM_NUM_BANKS(cap) *
+ ways[EDRAM_WAYS_IDX(cap)] *
+ sets[EDRAM_SETS_IDX(cap)] *
+ 1024 * 1024;
+}
+
+u64 intel_uncore_edram_size(struct drm_i915_private *dev_priv)
+{
+ if (!HAS_EDRAM(dev_priv))
+ return 0;
+
+ /* The needed capability bits for size calculation
+ * are not there with pre gen9 so return 128MB always.
+ */
+ if (INTEL_GEN(dev_priv) < 9)
+ return 128 * 1024 * 1024;
+
+ return gen9_edram_size(dev_priv);
+}
+
+static void intel_uncore_edram_detect(struct drm_i915_private *dev_priv)
+{
+ if (IS_HASWELL(dev_priv) ||
+ IS_BROADWELL(dev_priv) ||
+ INTEL_GEN(dev_priv) >= 9) {
+ dev_priv->edram_cap = __raw_i915_read32(dev_priv,
+ HSW_EDRAM_CAP);
+
+ /* NB: We can't write IDICR yet because we do not have gt funcs
* set up */
- dev_priv->ellc_size = 128;
- DRM_INFO("Found %zuMB of eLLC\n", dev_priv->ellc_size);
+ } else {
+ dev_priv->edram_cap = 0;
}
+
+ if (HAS_EDRAM(dev_priv))
+ DRM_INFO("Found %lluMB of eDRAM\n",
+ intel_uncore_edram_size(dev_priv) / (1024 * 1024));
}
static bool
@@ -410,16 +443,15 @@
enum forcewake_domains fw_domains)
{
struct intel_uncore_forcewake_domain *domain;
- enum forcewake_domain_id id;
if (!dev_priv->uncore.funcs.force_wake_get)
return;
fw_domains &= dev_priv->uncore.fw_domains;
- for_each_fw_domain_mask(domain, fw_domains, dev_priv, id) {
+ for_each_fw_domain_masked(domain, fw_domains, dev_priv) {
if (domain->wake_count++)
- fw_domains &= ~(1 << id);
+ fw_domains &= ~domain->mask;
}
if (fw_domains)
@@ -477,21 +509,19 @@
enum forcewake_domains fw_domains)
{
struct intel_uncore_forcewake_domain *domain;
- enum forcewake_domain_id id;
if (!dev_priv->uncore.funcs.force_wake_put)
return;
fw_domains &= dev_priv->uncore.fw_domains;
- for_each_fw_domain_mask(domain, fw_domains, dev_priv, id) {
+ for_each_fw_domain_masked(domain, fw_domains, dev_priv) {
if (WARN_ON(domain->wake_count == 0))
continue;
if (--domain->wake_count)
continue;
- domain->wake_count++;
fw_domain_arm_timer(domain);
}
}
@@ -539,18 +569,27 @@
void assert_forcewakes_inactive(struct drm_i915_private *dev_priv)
{
struct intel_uncore_forcewake_domain *domain;
- enum forcewake_domain_id id;
if (!dev_priv->uncore.funcs.force_wake_get)
return;
- for_each_fw_domain(domain, dev_priv, id)
+ for_each_fw_domain(domain, dev_priv)
WARN_ON(domain->wake_count);
}
/* We give fast paths for the really cool registers */
#define NEEDS_FORCE_WAKE(reg) ((reg) < 0x40000)
+#define __gen6_reg_read_fw_domains(offset) \
+({ \
+ enum forcewake_domains __fwd; \
+ if (NEEDS_FORCE_WAKE(offset)) \
+ __fwd = FORCEWAKE_RENDER; \
+ else \
+ __fwd = 0; \
+ __fwd; \
+})
+
#define REG_RANGE(reg, start, end) ((reg) >= (start) && (reg) < (end))
#define FORCEWAKE_VLV_RENDER_RANGE_OFFSET(reg) \
@@ -564,6 +603,48 @@
REG_RANGE((reg), 0x22000, 0x24000) || \
REG_RANGE((reg), 0x30000, 0x40000))
+#define __vlv_reg_read_fw_domains(offset) \
+({ \
+ enum forcewake_domains __fwd = 0; \
+ if (!NEEDS_FORCE_WAKE(offset)) \
+ __fwd = 0; \
+ else if (FORCEWAKE_VLV_RENDER_RANGE_OFFSET(offset)) \
+ __fwd = FORCEWAKE_RENDER; \
+ else if (FORCEWAKE_VLV_MEDIA_RANGE_OFFSET(offset)) \
+ __fwd = FORCEWAKE_MEDIA; \
+ __fwd; \
+})
+
+static const i915_reg_t gen8_shadowed_regs[] = {
+ GEN6_RPNSWREQ,
+ GEN6_RC_VIDEO_FREQ,
+ RING_TAIL(RENDER_RING_BASE),
+ RING_TAIL(GEN6_BSD_RING_BASE),
+ RING_TAIL(VEBOX_RING_BASE),
+ RING_TAIL(BLT_RING_BASE),
+ /* TODO: Other registers are not yet used */
+};
+
+static bool is_gen8_shadowed(u32 offset)
+{
+ int i;
+ for (i = 0; i < ARRAY_SIZE(gen8_shadowed_regs); i++)
+ if (offset == gen8_shadowed_regs[i].reg)
+ return true;
+
+ return false;
+}
+
+#define __gen8_reg_write_fw_domains(offset) \
+({ \
+ enum forcewake_domains __fwd; \
+ if (NEEDS_FORCE_WAKE(offset) && !is_gen8_shadowed(offset)) \
+ __fwd = FORCEWAKE_RENDER; \
+ else \
+ __fwd = 0; \
+ __fwd; \
+})
+
#define FORCEWAKE_CHV_RENDER_RANGE_OFFSET(reg) \
(REG_RANGE((reg), 0x2000, 0x4000) || \
REG_RANGE((reg), 0x5200, 0x8000) || \
@@ -586,6 +667,34 @@
REG_RANGE((reg), 0x9000, 0xB000) || \
REG_RANGE((reg), 0xF000, 0x10000))
+#define __chv_reg_read_fw_domains(offset) \
+({ \
+ enum forcewake_domains __fwd = 0; \
+ if (!NEEDS_FORCE_WAKE(offset)) \
+ __fwd = 0; \
+ else if (FORCEWAKE_CHV_RENDER_RANGE_OFFSET(offset)) \
+ __fwd = FORCEWAKE_RENDER; \
+ else if (FORCEWAKE_CHV_MEDIA_RANGE_OFFSET(offset)) \
+ __fwd = FORCEWAKE_MEDIA; \
+ else if (FORCEWAKE_CHV_COMMON_RANGE_OFFSET(offset)) \
+ __fwd = FORCEWAKE_RENDER | FORCEWAKE_MEDIA; \
+ __fwd; \
+})
+
+#define __chv_reg_write_fw_domains(offset) \
+({ \
+ enum forcewake_domains __fwd = 0; \
+ if (!NEEDS_FORCE_WAKE(offset) || is_gen8_shadowed(offset)) \
+ __fwd = 0; \
+ else if (FORCEWAKE_CHV_RENDER_RANGE_OFFSET(offset)) \
+ __fwd = FORCEWAKE_RENDER; \
+ else if (FORCEWAKE_CHV_MEDIA_RANGE_OFFSET(offset)) \
+ __fwd = FORCEWAKE_MEDIA; \
+ else if (FORCEWAKE_CHV_COMMON_RANGE_OFFSET(offset)) \
+ __fwd = FORCEWAKE_RENDER | FORCEWAKE_MEDIA; \
+ __fwd; \
+})
+
#define FORCEWAKE_GEN9_UNCORE_RANGE_OFFSET(reg) \
REG_RANGE((reg), 0xB00, 0x2000)
@@ -618,6 +727,61 @@
!FORCEWAKE_GEN9_MEDIA_RANGE_OFFSET(reg) && \
!FORCEWAKE_GEN9_COMMON_RANGE_OFFSET(reg))
+#define SKL_NEEDS_FORCE_WAKE(reg) \
+ ((reg) < 0x40000 && !FORCEWAKE_GEN9_UNCORE_RANGE_OFFSET(reg))
+
+#define __gen9_reg_read_fw_domains(offset) \
+({ \
+ enum forcewake_domains __fwd; \
+ if (!SKL_NEEDS_FORCE_WAKE(offset)) \
+ __fwd = 0; \
+ else if (FORCEWAKE_GEN9_RENDER_RANGE_OFFSET(offset)) \
+ __fwd = FORCEWAKE_RENDER; \
+ else if (FORCEWAKE_GEN9_MEDIA_RANGE_OFFSET(offset)) \
+ __fwd = FORCEWAKE_MEDIA; \
+ else if (FORCEWAKE_GEN9_COMMON_RANGE_OFFSET(offset)) \
+ __fwd = FORCEWAKE_RENDER | FORCEWAKE_MEDIA; \
+ else \
+ __fwd = FORCEWAKE_BLITTER; \
+ __fwd; \
+})
+
+static const i915_reg_t gen9_shadowed_regs[] = {
+ RING_TAIL(RENDER_RING_BASE),
+ RING_TAIL(GEN6_BSD_RING_BASE),
+ RING_TAIL(VEBOX_RING_BASE),
+ RING_TAIL(BLT_RING_BASE),
+ GEN6_RPNSWREQ,
+ GEN6_RC_VIDEO_FREQ,
+ /* TODO: Other registers are not yet used */
+};
+
+static bool is_gen9_shadowed(u32 offset)
+{
+ int i;
+ for (i = 0; i < ARRAY_SIZE(gen9_shadowed_regs); i++)
+ if (offset == gen9_shadowed_regs[i].reg)
+ return true;
+
+ return false;
+}
+
+#define __gen9_reg_write_fw_domains(offset) \
+({ \
+ enum forcewake_domains __fwd; \
+ if (!SKL_NEEDS_FORCE_WAKE(offset) || is_gen9_shadowed(offset)) \
+ __fwd = 0; \
+ else if (FORCEWAKE_GEN9_RENDER_RANGE_OFFSET(offset)) \
+ __fwd = FORCEWAKE_RENDER; \
+ else if (FORCEWAKE_GEN9_MEDIA_RANGE_OFFSET(offset)) \
+ __fwd = FORCEWAKE_MEDIA; \
+ else if (FORCEWAKE_GEN9_COMMON_RANGE_OFFSET(offset)) \
+ __fwd = FORCEWAKE_RENDER | FORCEWAKE_MEDIA; \
+ else \
+ __fwd = FORCEWAKE_BLITTER; \
+ __fwd; \
+})
+
static void
ilk_dummy_write(struct drm_i915_private *dev_priv)
{
@@ -633,15 +797,6 @@
const bool read,
const bool before)
{
- /* XXX. We limit the auto arming traces for mmio
- * debugs on these platforms. There are just too many
- * revealed by these and CI/Bat suffers from the noise.
- * Please fix and then re-enable the automatic traces.
- */
- if (i915.mmio_debug < 2 &&
- (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)))
- return;
-
if (WARN(check_for_unclaimed_mmio(dev_priv),
"Unclaimed register detected %s %s register 0x%x\n",
before ? "before" : "after",
@@ -716,23 +871,21 @@
trace_i915_reg_rw(false, reg, val, sizeof(val), trace); \
return val
-static inline void __force_wake_get(struct drm_i915_private *dev_priv,
- enum forcewake_domains fw_domains)
+static inline void __force_wake_auto(struct drm_i915_private *dev_priv,
+ enum forcewake_domains fw_domains)
{
struct intel_uncore_forcewake_domain *domain;
- enum forcewake_domain_id id;
if (WARN_ON(!fw_domains))
return;
/* Ideally GCC would be constant-fold and eliminate this loop */
- for_each_fw_domain_mask(domain, fw_domains, dev_priv, id) {
+ for_each_fw_domain_masked(domain, fw_domains, dev_priv) {
if (domain->wake_count) {
- fw_domains &= ~(1 << id);
+ fw_domains &= ~domain->mask;
continue;
}
- domain->wake_count++;
fw_domain_arm_timer(domain);
}
@@ -743,9 +896,11 @@
#define __gen6_read(x) \
static u##x \
gen6_read##x(struct drm_i915_private *dev_priv, i915_reg_t reg, bool trace) { \
+ enum forcewake_domains fw_engine; \
GEN6_READ_HEADER(x); \
- if (NEEDS_FORCE_WAKE(offset)) \
- __force_wake_get(dev_priv, FORCEWAKE_RENDER); \
+ fw_engine = __gen6_reg_read_fw_domains(offset); \
+ if (fw_engine) \
+ __force_wake_auto(dev_priv, fw_engine); \
val = __raw_i915_read##x(dev_priv, reg); \
GEN6_READ_FOOTER; \
}
@@ -753,16 +908,11 @@
#define __vlv_read(x) \
static u##x \
vlv_read##x(struct drm_i915_private *dev_priv, i915_reg_t reg, bool trace) { \
- enum forcewake_domains fw_engine = 0; \
+ enum forcewake_domains fw_engine; \
GEN6_READ_HEADER(x); \
- if (!NEEDS_FORCE_WAKE(offset)) \
- fw_engine = 0; \
- else if (FORCEWAKE_VLV_RENDER_RANGE_OFFSET(offset)) \
- fw_engine = FORCEWAKE_RENDER; \
- else if (FORCEWAKE_VLV_MEDIA_RANGE_OFFSET(offset)) \
- fw_engine = FORCEWAKE_MEDIA; \
+ fw_engine = __vlv_reg_read_fw_domains(offset); \
if (fw_engine) \
- __force_wake_get(dev_priv, fw_engine); \
+ __force_wake_auto(dev_priv, fw_engine); \
val = __raw_i915_read##x(dev_priv, reg); \
GEN6_READ_FOOTER; \
}
@@ -770,42 +920,23 @@
#define __chv_read(x) \
static u##x \
chv_read##x(struct drm_i915_private *dev_priv, i915_reg_t reg, bool trace) { \
- enum forcewake_domains fw_engine = 0; \
+ enum forcewake_domains fw_engine; \
GEN6_READ_HEADER(x); \
- if (!NEEDS_FORCE_WAKE(offset)) \
- fw_engine = 0; \
- else if (FORCEWAKE_CHV_RENDER_RANGE_OFFSET(offset)) \
- fw_engine = FORCEWAKE_RENDER; \
- else if (FORCEWAKE_CHV_MEDIA_RANGE_OFFSET(offset)) \
- fw_engine = FORCEWAKE_MEDIA; \
- else if (FORCEWAKE_CHV_COMMON_RANGE_OFFSET(offset)) \
- fw_engine = FORCEWAKE_RENDER | FORCEWAKE_MEDIA; \
+ fw_engine = __chv_reg_read_fw_domains(offset); \
if (fw_engine) \
- __force_wake_get(dev_priv, fw_engine); \
+ __force_wake_auto(dev_priv, fw_engine); \
val = __raw_i915_read##x(dev_priv, reg); \
GEN6_READ_FOOTER; \
}
-#define SKL_NEEDS_FORCE_WAKE(reg) \
- ((reg) < 0x40000 && !FORCEWAKE_GEN9_UNCORE_RANGE_OFFSET(reg))
-
#define __gen9_read(x) \
static u##x \
gen9_read##x(struct drm_i915_private *dev_priv, i915_reg_t reg, bool trace) { \
enum forcewake_domains fw_engine; \
GEN6_READ_HEADER(x); \
- if (!SKL_NEEDS_FORCE_WAKE(offset)) \
- fw_engine = 0; \
- else if (FORCEWAKE_GEN9_RENDER_RANGE_OFFSET(offset)) \
- fw_engine = FORCEWAKE_RENDER; \
- else if (FORCEWAKE_GEN9_MEDIA_RANGE_OFFSET(offset)) \
- fw_engine = FORCEWAKE_MEDIA; \
- else if (FORCEWAKE_GEN9_COMMON_RANGE_OFFSET(offset)) \
- fw_engine = FORCEWAKE_RENDER | FORCEWAKE_MEDIA; \
- else \
- fw_engine = FORCEWAKE_BLITTER; \
+ fw_engine = __gen9_reg_read_fw_domains(offset); \
if (fw_engine) \
- __force_wake_get(dev_priv, fw_engine); \
+ __force_wake_auto(dev_priv, fw_engine); \
val = __raw_i915_read##x(dev_priv, reg); \
GEN6_READ_FOOTER; \
}
@@ -942,34 +1073,14 @@
GEN6_WRITE_FOOTER; \
}
-static const i915_reg_t gen8_shadowed_regs[] = {
- FORCEWAKE_MT,
- GEN6_RPNSWREQ,
- GEN6_RC_VIDEO_FREQ,
- RING_TAIL(RENDER_RING_BASE),
- RING_TAIL(GEN6_BSD_RING_BASE),
- RING_TAIL(VEBOX_RING_BASE),
- RING_TAIL(BLT_RING_BASE),
- /* TODO: Other registers are not yet used */
-};
-
-static bool is_gen8_shadowed(struct drm_i915_private *dev_priv,
- i915_reg_t reg)
-{
- int i;
- for (i = 0; i < ARRAY_SIZE(gen8_shadowed_regs); i++)
- if (i915_mmio_reg_equal(reg, gen8_shadowed_regs[i]))
- return true;
-
- return false;
-}
-
#define __gen8_write(x) \
static void \
gen8_write##x(struct drm_i915_private *dev_priv, i915_reg_t reg, u##x val, bool trace) { \
+ enum forcewake_domains fw_engine; \
GEN6_WRITE_HEADER; \
- if (NEEDS_FORCE_WAKE(offset) && !is_gen8_shadowed(dev_priv, reg)) \
- __force_wake_get(dev_priv, FORCEWAKE_RENDER); \
+ fw_engine = __gen8_reg_write_fw_domains(offset); \
+ if (fw_engine) \
+ __force_wake_auto(dev_priv, fw_engine); \
__raw_i915_write##x(dev_priv, reg, val); \
GEN6_WRITE_FOOTER; \
}
@@ -977,66 +1088,24 @@
#define __chv_write(x) \
static void \
chv_write##x(struct drm_i915_private *dev_priv, i915_reg_t reg, u##x val, bool trace) { \
- enum forcewake_domains fw_engine = 0; \
+ enum forcewake_domains fw_engine; \
GEN6_WRITE_HEADER; \
- if (!NEEDS_FORCE_WAKE(offset) || \
- is_gen8_shadowed(dev_priv, reg)) \
- fw_engine = 0; \
- else if (FORCEWAKE_CHV_RENDER_RANGE_OFFSET(offset)) \
- fw_engine = FORCEWAKE_RENDER; \
- else if (FORCEWAKE_CHV_MEDIA_RANGE_OFFSET(offset)) \
- fw_engine = FORCEWAKE_MEDIA; \
- else if (FORCEWAKE_CHV_COMMON_RANGE_OFFSET(offset)) \
- fw_engine = FORCEWAKE_RENDER | FORCEWAKE_MEDIA; \
+ fw_engine = __chv_reg_write_fw_domains(offset); \
if (fw_engine) \
- __force_wake_get(dev_priv, fw_engine); \
+ __force_wake_auto(dev_priv, fw_engine); \
__raw_i915_write##x(dev_priv, reg, val); \
GEN6_WRITE_FOOTER; \
}
-static const i915_reg_t gen9_shadowed_regs[] = {
- RING_TAIL(RENDER_RING_BASE),
- RING_TAIL(GEN6_BSD_RING_BASE),
- RING_TAIL(VEBOX_RING_BASE),
- RING_TAIL(BLT_RING_BASE),
- FORCEWAKE_BLITTER_GEN9,
- FORCEWAKE_RENDER_GEN9,
- FORCEWAKE_MEDIA_GEN9,
- GEN6_RPNSWREQ,
- GEN6_RC_VIDEO_FREQ,
- /* TODO: Other registers are not yet used */
-};
-
-static bool is_gen9_shadowed(struct drm_i915_private *dev_priv,
- i915_reg_t reg)
-{
- int i;
- for (i = 0; i < ARRAY_SIZE(gen9_shadowed_regs); i++)
- if (i915_mmio_reg_equal(reg, gen9_shadowed_regs[i]))
- return true;
-
- return false;
-}
-
#define __gen9_write(x) \
static void \
gen9_write##x(struct drm_i915_private *dev_priv, i915_reg_t reg, u##x val, \
bool trace) { \
enum forcewake_domains fw_engine; \
GEN6_WRITE_HEADER; \
- if (!SKL_NEEDS_FORCE_WAKE(offset) || \
- is_gen9_shadowed(dev_priv, reg)) \
- fw_engine = 0; \
- else if (FORCEWAKE_GEN9_RENDER_RANGE_OFFSET(offset)) \
- fw_engine = FORCEWAKE_RENDER; \
- else if (FORCEWAKE_GEN9_MEDIA_RANGE_OFFSET(offset)) \
- fw_engine = FORCEWAKE_MEDIA; \
- else if (FORCEWAKE_GEN9_COMMON_RANGE_OFFSET(offset)) \
- fw_engine = FORCEWAKE_RENDER | FORCEWAKE_MEDIA; \
- else \
- fw_engine = FORCEWAKE_BLITTER; \
+ fw_engine = __gen9_reg_write_fw_domains(offset); \
if (fw_engine) \
- __force_wake_get(dev_priv, fw_engine); \
+ __force_wake_auto(dev_priv, fw_engine); \
__raw_i915_write##x(dev_priv, reg, val); \
GEN6_WRITE_FOOTER; \
}
@@ -1150,7 +1219,14 @@
d->i915 = dev_priv;
d->id = domain_id;
- setup_timer(&d->timer, intel_uncore_fw_release_timer, (unsigned long)d);
+ BUILD_BUG_ON(FORCEWAKE_RENDER != (1 << FW_DOMAIN_ID_RENDER));
+ BUILD_BUG_ON(FORCEWAKE_BLITTER != (1 << FW_DOMAIN_ID_BLITTER));
+ BUILD_BUG_ON(FORCEWAKE_MEDIA != (1 << FW_DOMAIN_ID_MEDIA));
+
+ d->mask = 1 << domain_id;
+
+ hrtimer_init(&d->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+ d->timer.function = intel_uncore_fw_release_timer;
dev_priv->uncore.fw_domains |= (1 << domain_id);
@@ -1161,7 +1237,7 @@
{
struct drm_i915_private *dev_priv = dev->dev_private;
- if (INTEL_INFO(dev_priv->dev)->gen <= 5)
+ if (INTEL_INFO(dev_priv)->gen <= 5)
return;
if (IS_GEN9(dev)) {
@@ -1257,7 +1333,7 @@
i915_check_vgpu(dev);
- intel_uncore_ellc_detect(dev);
+ intel_uncore_edram_detect(dev_priv);
intel_uncore_fw_domains_init(dev);
__intel_uncore_early_sanitize(dev, false);
@@ -1437,7 +1513,7 @@
return (gdrst & GRDOM_RESET_STATUS) == 0;
}
-static int i915_do_reset(struct drm_device *dev)
+static int i915_do_reset(struct drm_device *dev, unsigned engine_mask)
{
/* assert reset for at least 20 usec */
pci_write_config_byte(dev->pdev, I915_GDRST, GRDOM_RESET_ENABLE);
@@ -1454,13 +1530,13 @@
return (gdrst & GRDOM_RESET_ENABLE) == 0;
}
-static int g33_do_reset(struct drm_device *dev)
+static int g33_do_reset(struct drm_device *dev, unsigned engine_mask)
{
pci_write_config_byte(dev->pdev, I915_GDRST, GRDOM_RESET_ENABLE);
return wait_for(g4x_reset_complete(dev), 500);
}
-static int g4x_do_reset(struct drm_device *dev)
+static int g4x_do_reset(struct drm_device *dev, unsigned engine_mask)
{
struct drm_i915_private *dev_priv = dev->dev_private;
int ret;
@@ -1490,7 +1566,7 @@
return 0;
}
-static int ironlake_do_reset(struct drm_device *dev)
+static int ironlake_do_reset(struct drm_device *dev, unsigned engine_mask)
{
struct drm_i915_private *dev_priv = dev->dev_private;
int ret;
@@ -1514,75 +1590,132 @@
return 0;
}
-static int gen6_do_reset(struct drm_device *dev)
+/* Reset the hardware domains (GENX_GRDOM_*) specified by mask */
+static int gen6_hw_domain_reset(struct drm_i915_private *dev_priv,
+ u32 hw_domain_mask)
{
- struct drm_i915_private *dev_priv = dev->dev_private;
- int ret;
-
- /* Reset the chip */
+ int ret;
/* GEN6_GDRST is not in the gt power well, no need to check
* for fifo space for the write or forcewake the chip for
* the read
*/
- __raw_i915_write32(dev_priv, GEN6_GDRST, GEN6_GRDOM_FULL);
+ __raw_i915_write32(dev_priv, GEN6_GDRST, hw_domain_mask);
- /* Spin waiting for the device to ack the reset request */
- ret = wait_for((__raw_i915_read32(dev_priv, GEN6_GDRST) & GEN6_GRDOM_FULL) == 0, 500);
+#define ACKED ((__raw_i915_read32(dev_priv, GEN6_GDRST) & hw_domain_mask) == 0)
+ /* Spin waiting for the device to ack the reset requests */
+ ret = wait_for(ACKED, 500);
+#undef ACKED
+
+ return ret;
+}
+
+/**
+ * gen6_reset_engines - reset individual engines
+ * @dev: DRM device
+ * @engine_mask: mask of intel_ring_flag() engines or ALL_ENGINES for full reset
+ *
+ * This function will reset the individual engines that are set in engine_mask.
+ * If you provide ALL_ENGINES as mask, full global domain reset will be issued.
+ *
+ * Note: It is responsibility of the caller to handle the difference between
+ * asking full domain reset versus reset for all available individual engines.
+ *
+ * Returns 0 on success, nonzero on error.
+ */
+static int gen6_reset_engines(struct drm_device *dev, unsigned engine_mask)
+{
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct intel_engine_cs *engine;
+ const u32 hw_engine_mask[I915_NUM_ENGINES] = {
+ [RCS] = GEN6_GRDOM_RENDER,
+ [BCS] = GEN6_GRDOM_BLT,
+ [VCS] = GEN6_GRDOM_MEDIA,
+ [VCS2] = GEN8_GRDOM_MEDIA2,
+ [VECS] = GEN6_GRDOM_VECS,
+ };
+ u32 hw_mask;
+ int ret;
+
+ if (engine_mask == ALL_ENGINES) {
+ hw_mask = GEN6_GRDOM_FULL;
+ } else {
+ hw_mask = 0;
+ for_each_engine_masked(engine, dev_priv, engine_mask)
+ hw_mask |= hw_engine_mask[engine->id];
+ }
+
+ ret = gen6_hw_domain_reset(dev_priv, hw_mask);
intel_uncore_forcewake_reset(dev, true);
return ret;
}
-static int wait_for_register(struct drm_i915_private *dev_priv,
- i915_reg_t reg,
- const u32 mask,
- const u32 value,
- const unsigned long timeout_ms)
+static int wait_for_register_fw(struct drm_i915_private *dev_priv,
+ i915_reg_t reg,
+ const u32 mask,
+ const u32 value,
+ const unsigned long timeout_ms)
{
- return wait_for((I915_READ(reg) & mask) == value, timeout_ms);
+ return wait_for((I915_READ_FW(reg) & mask) == value, timeout_ms);
}
-static int gen8_do_reset(struct drm_device *dev)
+static int gen8_request_engine_reset(struct intel_engine_cs *engine)
+{
+ int ret;
+ struct drm_i915_private *dev_priv = engine->dev->dev_private;
+
+ I915_WRITE_FW(RING_RESET_CTL(engine->mmio_base),
+ _MASKED_BIT_ENABLE(RESET_CTL_REQUEST_RESET));
+
+ ret = wait_for_register_fw(dev_priv,
+ RING_RESET_CTL(engine->mmio_base),
+ RESET_CTL_READY_TO_RESET,
+ RESET_CTL_READY_TO_RESET,
+ 700);
+ if (ret)
+ DRM_ERROR("%s: reset request timeout\n", engine->name);
+
+ return ret;
+}
+
+static void gen8_unrequest_engine_reset(struct intel_engine_cs *engine)
+{
+ struct drm_i915_private *dev_priv = engine->dev->dev_private;
+
+ I915_WRITE_FW(RING_RESET_CTL(engine->mmio_base),
+ _MASKED_BIT_DISABLE(RESET_CTL_REQUEST_RESET));
+}
+
+static int gen8_reset_engines(struct drm_device *dev, unsigned engine_mask)
{
struct drm_i915_private *dev_priv = dev->dev_private;
struct intel_engine_cs *engine;
- int i;
- for_each_ring(engine, dev_priv, i) {
- I915_WRITE(RING_RESET_CTL(engine->mmio_base),
- _MASKED_BIT_ENABLE(RESET_CTL_REQUEST_RESET));
-
- if (wait_for_register(dev_priv,
- RING_RESET_CTL(engine->mmio_base),
- RESET_CTL_READY_TO_RESET,
- RESET_CTL_READY_TO_RESET,
- 700)) {
- DRM_ERROR("%s: reset request timeout\n", engine->name);
+ for_each_engine_masked(engine, dev_priv, engine_mask)
+ if (gen8_request_engine_reset(engine))
goto not_ready;
- }
- }
- return gen6_do_reset(dev);
+ return gen6_reset_engines(dev, engine_mask);
not_ready:
- for_each_ring(engine, dev_priv, i)
- I915_WRITE(RING_RESET_CTL(engine->mmio_base),
- _MASKED_BIT_DISABLE(RESET_CTL_REQUEST_RESET));
+ for_each_engine_masked(engine, dev_priv, engine_mask)
+ gen8_unrequest_engine_reset(engine);
return -EIO;
}
-static int (*intel_get_gpu_reset(struct drm_device *dev))(struct drm_device *)
+static int (*intel_get_gpu_reset(struct drm_device *dev))(struct drm_device *,
+ unsigned engine_mask)
{
if (!i915.reset)
return NULL;
if (INTEL_INFO(dev)->gen >= 8)
- return gen8_do_reset;
+ return gen8_reset_engines;
else if (INTEL_INFO(dev)->gen >= 6)
- return gen6_do_reset;
+ return gen6_reset_engines;
else if (IS_GEN5(dev))
return ironlake_do_reset;
else if (IS_G4X(dev))
@@ -1595,10 +1728,10 @@
return NULL;
}
-int intel_gpu_reset(struct drm_device *dev)
+int intel_gpu_reset(struct drm_device *dev, unsigned engine_mask)
{
struct drm_i915_private *dev_priv = to_i915(dev);
- int (*reset)(struct drm_device *);
+ int (*reset)(struct drm_device *, unsigned);
int ret;
reset = intel_get_gpu_reset(dev);
@@ -1609,7 +1742,7 @@
* request may be dropped and never completes (causing -EIO).
*/
intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
- ret = reset(dev);
+ ret = reset(dev, engine_mask);
intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
return ret;
@@ -1620,6 +1753,25 @@
return intel_get_gpu_reset(dev) != NULL;
}
+int intel_guc_reset(struct drm_i915_private *dev_priv)
+{
+ int ret;
+ unsigned long irqflags;
+
+ if (!i915.enable_guc_submission)
+ return -EINVAL;
+
+ intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
+ spin_lock_irqsave(&dev_priv->uncore.lock, irqflags);
+
+ ret = gen6_hw_domain_reset(dev_priv, GEN9_GRDOM_GUC);
+
+ spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags);
+ intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
+
+ return ret;
+}
+
bool intel_uncore_unclaimed_mmio(struct drm_i915_private *dev_priv)
{
return check_for_unclaimed_mmio(dev_priv);
@@ -1643,3 +1795,111 @@
return false;
}
+
+static enum forcewake_domains
+intel_uncore_forcewake_for_read(struct drm_i915_private *dev_priv,
+ i915_reg_t reg)
+{
+ enum forcewake_domains fw_domains;
+
+ if (intel_vgpu_active(dev_priv->dev))
+ return 0;
+
+ switch (INTEL_INFO(dev_priv)->gen) {
+ case 9:
+ fw_domains = __gen9_reg_read_fw_domains(i915_mmio_reg_offset(reg));
+ break;
+ case 8:
+ if (IS_CHERRYVIEW(dev_priv))
+ fw_domains = __chv_reg_read_fw_domains(i915_mmio_reg_offset(reg));
+ else
+ fw_domains = __gen6_reg_read_fw_domains(i915_mmio_reg_offset(reg));
+ break;
+ case 7:
+ case 6:
+ if (IS_VALLEYVIEW(dev_priv))
+ fw_domains = __vlv_reg_read_fw_domains(i915_mmio_reg_offset(reg));
+ else
+ fw_domains = __gen6_reg_read_fw_domains(i915_mmio_reg_offset(reg));
+ break;
+ default:
+ MISSING_CASE(INTEL_INFO(dev_priv)->gen);
+ case 5: /* forcewake was introduced with gen6 */
+ case 4:
+ case 3:
+ case 2:
+ return 0;
+ }
+
+ WARN_ON(fw_domains & ~dev_priv->uncore.fw_domains);
+
+ return fw_domains;
+}
+
+static enum forcewake_domains
+intel_uncore_forcewake_for_write(struct drm_i915_private *dev_priv,
+ i915_reg_t reg)
+{
+ enum forcewake_domains fw_domains;
+
+ if (intel_vgpu_active(dev_priv->dev))
+ return 0;
+
+ switch (INTEL_INFO(dev_priv)->gen) {
+ case 9:
+ fw_domains = __gen9_reg_write_fw_domains(i915_mmio_reg_offset(reg));
+ break;
+ case 8:
+ if (IS_CHERRYVIEW(dev_priv))
+ fw_domains = __chv_reg_write_fw_domains(i915_mmio_reg_offset(reg));
+ else
+ fw_domains = __gen8_reg_write_fw_domains(i915_mmio_reg_offset(reg));
+ break;
+ case 7:
+ case 6:
+ fw_domains = FORCEWAKE_RENDER;
+ break;
+ default:
+ MISSING_CASE(INTEL_INFO(dev_priv)->gen);
+ case 5:
+ case 4:
+ case 3:
+ case 2:
+ return 0;
+ }
+
+ WARN_ON(fw_domains & ~dev_priv->uncore.fw_domains);
+
+ return fw_domains;
+}
+
+/**
+ * intel_uncore_forcewake_for_reg - which forcewake domains are needed to access
+ * a register
+ * @dev_priv: pointer to struct drm_i915_private
+ * @reg: register in question
+ * @op: operation bitmask of FW_REG_READ and/or FW_REG_WRITE
+ *
+ * Returns a set of forcewake domains required to be taken with for example
+ * intel_uncore_forcewake_get for the specified register to be accessible in the
+ * specified mode (read, write or read/write) with raw mmio accessors.
+ *
+ * NOTE: On Gen6 and Gen7 write forcewake domain (FORCEWAKE_RENDER) requires the
+ * callers to do FIFO management on their own or risk losing writes.
+ */
+enum forcewake_domains
+intel_uncore_forcewake_for_reg(struct drm_i915_private *dev_priv,
+ i915_reg_t reg, unsigned int op)
+{
+ enum forcewake_domains fw_domains = 0;
+
+ WARN_ON(!op);
+
+ if (op & FW_REG_READ)
+ fw_domains = intel_uncore_forcewake_for_read(dev_priv, reg);
+
+ if (op & FW_REG_WRITE)
+ fw_domains |= intel_uncore_forcewake_for_write(dev_priv, reg);
+
+ return fw_domains;
+}
diff --git a/drivers/gpu/drm/i915/intel_vbt_defs.h b/drivers/gpu/drm/i915/intel_vbt_defs.h
new file mode 100644
index 0000000..9ff1e96
--- /dev/null
+++ b/drivers/gpu/drm/i915/intel_vbt_defs.h
@@ -0,0 +1,832 @@
+/*
+ * Copyright © 2006-2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ *
+ * Authors:
+ * Eric Anholt <eric@anholt.net>
+ *
+ */
+
+/*
+ * This information is private to VBT parsing in intel_bios.c.
+ *
+ * Please do NOT include anywhere else.
+ */
+#ifndef _INTEL_BIOS_PRIVATE
+#error "intel_vbt_defs.h is private to intel_bios.c"
+#endif
+
+#ifndef _INTEL_VBT_DEFS_H_
+#define _INTEL_VBT_DEFS_H_
+
+#include "intel_bios.h"
+
+/**
+ * struct vbt_header - VBT Header structure
+ * @signature: VBT signature, always starts with "$VBT"
+ * @version: Version of this structure
+ * @header_size: Size of this structure
+ * @vbt_size: Size of VBT (VBT Header, BDB Header and data blocks)
+ * @vbt_checksum: Checksum
+ * @reserved0: Reserved
+ * @bdb_offset: Offset of &struct bdb_header from beginning of VBT
+ * @aim_offset: Offsets of add-in data blocks from beginning of VBT
+ */
+struct vbt_header {
+ u8 signature[20];
+ u16 version;
+ u16 header_size;
+ u16 vbt_size;
+ u8 vbt_checksum;
+ u8 reserved0;
+ u32 bdb_offset;
+ u32 aim_offset[4];
+} __packed;
+
+/**
+ * struct bdb_header - BDB Header structure
+ * @signature: BDB signature "BIOS_DATA_BLOCK"
+ * @version: Version of the data block definitions
+ * @header_size: Size of this structure
+ * @bdb_size: Size of BDB (BDB Header and data blocks)
+ */
+struct bdb_header {
+ u8 signature[16];
+ u16 version;
+ u16 header_size;
+ u16 bdb_size;
+} __packed;
+
+/* strictly speaking, this is a "skip" block, but it has interesting info */
+struct vbios_data {
+ u8 type; /* 0 == desktop, 1 == mobile */
+ u8 relstage;
+ u8 chipset;
+ u8 lvds_present:1;
+ u8 tv_present:1;
+ u8 rsvd2:6; /* finish byte */
+ u8 rsvd3[4];
+ u8 signon[155];
+ u8 copyright[61];
+ u16 code_segment;
+ u8 dos_boot_mode;
+ u8 bandwidth_percent;
+ u8 rsvd4; /* popup memory size */
+ u8 resize_pci_bios;
+ u8 rsvd5; /* is crt already on ddc2 */
+} __packed;
+
+/*
+ * There are several types of BIOS data blocks (BDBs), each block has
+ * an ID and size in the first 3 bytes (ID in first, size in next 2).
+ * Known types are listed below.
+ */
+#define BDB_GENERAL_FEATURES 1
+#define BDB_GENERAL_DEFINITIONS 2
+#define BDB_OLD_TOGGLE_LIST 3
+#define BDB_MODE_SUPPORT_LIST 4
+#define BDB_GENERIC_MODE_TABLE 5
+#define BDB_EXT_MMIO_REGS 6
+#define BDB_SWF_IO 7
+#define BDB_SWF_MMIO 8
+#define BDB_PSR 9
+#define BDB_MODE_REMOVAL_TABLE 10
+#define BDB_CHILD_DEVICE_TABLE 11
+#define BDB_DRIVER_FEATURES 12
+#define BDB_DRIVER_PERSISTENCE 13
+#define BDB_EXT_TABLE_PTRS 14
+#define BDB_DOT_CLOCK_OVERRIDE 15
+#define BDB_DISPLAY_SELECT 16
+/* 17 rsvd */
+#define BDB_DRIVER_ROTATION 18
+#define BDB_DISPLAY_REMOVE 19
+#define BDB_OEM_CUSTOM 20
+#define BDB_EFP_LIST 21 /* workarounds for VGA hsync/vsync */
+#define BDB_SDVO_LVDS_OPTIONS 22
+#define BDB_SDVO_PANEL_DTDS 23
+#define BDB_SDVO_LVDS_PNP_IDS 24
+#define BDB_SDVO_LVDS_POWER_SEQ 25
+#define BDB_TV_OPTIONS 26
+#define BDB_EDP 27
+#define BDB_LVDS_OPTIONS 40
+#define BDB_LVDS_LFP_DATA_PTRS 41
+#define BDB_LVDS_LFP_DATA 42
+#define BDB_LVDS_BACKLIGHT 43
+#define BDB_LVDS_POWER 44
+#define BDB_MIPI_CONFIG 52
+#define BDB_MIPI_SEQUENCE 53
+#define BDB_SKIP 254 /* VBIOS private block, ignore */
+
+struct bdb_general_features {
+ /* bits 1 */
+ u8 panel_fitting:2;
+ u8 flexaim:1;
+ u8 msg_enable:1;
+ u8 clear_screen:3;
+ u8 color_flip:1;
+
+ /* bits 2 */
+ u8 download_ext_vbt:1;
+ u8 enable_ssc:1;
+ u8 ssc_freq:1;
+ u8 enable_lfp_on_override:1;
+ u8 disable_ssc_ddt:1;
+ u8 rsvd7:1;
+ u8 display_clock_mode:1;
+ u8 rsvd8:1; /* finish byte */
+
+ /* bits 3 */
+ u8 disable_smooth_vision:1;
+ u8 single_dvi:1;
+ u8 rsvd9:1;
+ u8 fdi_rx_polarity_inverted:1;
+ u8 rsvd10:4; /* finish byte */
+
+ /* bits 4 */
+ u8 legacy_monitor_detect;
+
+ /* bits 5 */
+ u8 int_crt_support:1;
+ u8 int_tv_support:1;
+ u8 int_efp_support:1;
+ u8 dp_ssc_enb:1; /* PCH attached eDP supports SSC */
+ u8 dp_ssc_freq:1; /* SSC freq for PCH attached eDP */
+ u8 rsvd11:3; /* finish byte */
+} __packed;
+
+/* pre-915 */
+#define GPIO_PIN_DVI_LVDS 0x03 /* "DVI/LVDS DDC GPIO pins" */
+#define GPIO_PIN_ADD_I2C 0x05 /* "ADDCARD I2C GPIO pins" */
+#define GPIO_PIN_ADD_DDC 0x04 /* "ADDCARD DDC GPIO pins" */
+#define GPIO_PIN_ADD_DDC_I2C 0x06 /* "ADDCARD DDC/I2C GPIO pins" */
+
+/* Pre 915 */
+#define DEVICE_TYPE_NONE 0x00
+#define DEVICE_TYPE_CRT 0x01
+#define DEVICE_TYPE_TV 0x09
+#define DEVICE_TYPE_EFP 0x12
+#define DEVICE_TYPE_LFP 0x22
+/* On 915+ */
+#define DEVICE_TYPE_CRT_DPMS 0x6001
+#define DEVICE_TYPE_CRT_DPMS_HOTPLUG 0x4001
+#define DEVICE_TYPE_TV_COMPOSITE 0x0209
+#define DEVICE_TYPE_TV_MACROVISION 0x0289
+#define DEVICE_TYPE_TV_RF_COMPOSITE 0x020c
+#define DEVICE_TYPE_TV_SVIDEO_COMPOSITE 0x0609
+#define DEVICE_TYPE_TV_SCART 0x0209
+#define DEVICE_TYPE_TV_CODEC_HOTPLUG_PWR 0x6009
+#define DEVICE_TYPE_EFP_HOTPLUG_PWR 0x6012
+#define DEVICE_TYPE_EFP_DVI_HOTPLUG_PWR 0x6052
+#define DEVICE_TYPE_EFP_DVI_I 0x6053
+#define DEVICE_TYPE_EFP_DVI_D_DUAL 0x6152
+#define DEVICE_TYPE_EFP_DVI_D_HDCP 0x60d2
+#define DEVICE_TYPE_OPENLDI_HOTPLUG_PWR 0x6062
+#define DEVICE_TYPE_OPENLDI_DUALPIX 0x6162
+#define DEVICE_TYPE_LFP_PANELLINK 0x5012
+#define DEVICE_TYPE_LFP_CMOS_PWR 0x5042
+#define DEVICE_TYPE_LFP_LVDS_PWR 0x5062
+#define DEVICE_TYPE_LFP_LVDS_DUAL 0x5162
+#define DEVICE_TYPE_LFP_LVDS_DUAL_HDCP 0x51e2
+
+#define DEVICE_CFG_NONE 0x00
+#define DEVICE_CFG_12BIT_DVOB 0x01
+#define DEVICE_CFG_12BIT_DVOC 0x02
+#define DEVICE_CFG_24BIT_DVOBC 0x09
+#define DEVICE_CFG_24BIT_DVOCB 0x0a
+#define DEVICE_CFG_DUAL_DVOB 0x11
+#define DEVICE_CFG_DUAL_DVOC 0x12
+#define DEVICE_CFG_DUAL_DVOBC 0x13
+#define DEVICE_CFG_DUAL_LINK_DVOBC 0x19
+#define DEVICE_CFG_DUAL_LINK_DVOCB 0x1a
+
+#define DEVICE_WIRE_NONE 0x00
+#define DEVICE_WIRE_DVOB 0x01
+#define DEVICE_WIRE_DVOC 0x02
+#define DEVICE_WIRE_DVOBC 0x03
+#define DEVICE_WIRE_DVOBB 0x05
+#define DEVICE_WIRE_DVOCC 0x06
+#define DEVICE_WIRE_DVOB_MASTER 0x0d
+#define DEVICE_WIRE_DVOC_MASTER 0x0e
+
+#define DEVICE_PORT_DVOA 0x00 /* none on 845+ */
+#define DEVICE_PORT_DVOB 0x01
+#define DEVICE_PORT_DVOC 0x02
+
+/*
+ * We used to keep this struct but without any version control. We should avoid
+ * using it in the future, but it should be safe to keep using it in the old
+ * code. Do not change; we rely on its size.
+ */
+struct old_child_dev_config {
+ u16 handle;
+ u16 device_type;
+ u8 device_id[10]; /* ascii string */
+ u16 addin_offset;
+ u8 dvo_port; /* See Device_PORT_* above */
+ u8 i2c_pin;
+ u8 slave_addr;
+ u8 ddc_pin;
+ u16 edid_ptr;
+ u8 dvo_cfg; /* See DEVICE_CFG_* above */
+ u8 dvo2_port;
+ u8 i2c2_pin;
+ u8 slave2_addr;
+ u8 ddc2_pin;
+ u8 capabilities;
+ u8 dvo_wiring;/* See DEVICE_WIRE_* above */
+ u8 dvo2_wiring;
+ u16 extended_type;
+ u8 dvo_function;
+} __packed;
+
+/* This one contains field offsets that are known to be common for all BDB
+ * versions. Notice that the meaning of the contents contents may still change,
+ * but at least the offsets are consistent. */
+
+struct common_child_dev_config {
+ u16 handle;
+ u16 device_type;
+ u8 not_common1[12];
+ u8 dvo_port;
+ u8 not_common2[2];
+ u8 ddc_pin;
+ u16 edid_ptr;
+ u8 dvo_cfg; /* See DEVICE_CFG_* above */
+ u8 efp_routed:1;
+ u8 lane_reversal:1;
+ u8 lspcon:1;
+ u8 iboost:1;
+ u8 hpd_invert:1;
+ u8 flag_reserved:3;
+ u8 hdmi_support:1;
+ u8 dp_support:1;
+ u8 tmds_support:1;
+ u8 support_reserved:5;
+ u8 not_common3[12];
+ u8 iboost_level;
+} __packed;
+
+
+/* This field changes depending on the BDB version, so the most reliable way to
+ * read it is by checking the BDB version and reading the raw pointer. */
+union child_device_config {
+ /* This one is safe to be used anywhere, but the code should still check
+ * the BDB version. */
+ u8 raw[33];
+ /* This one should only be kept for legacy code. */
+ struct old_child_dev_config old;
+ /* This one should also be safe to use anywhere, even without version
+ * checks. */
+ struct common_child_dev_config common;
+} __packed;
+
+struct bdb_general_definitions {
+ /* DDC GPIO */
+ u8 crt_ddc_gmbus_pin;
+
+ /* DPMS bits */
+ u8 dpms_acpi:1;
+ u8 skip_boot_crt_detect:1;
+ u8 dpms_aim:1;
+ u8 rsvd1:5; /* finish byte */
+
+ /* boot device bits */
+ u8 boot_display[2];
+ u8 child_dev_size;
+
+ /*
+ * Device info:
+ * If TV is present, it'll be at devices[0].
+ * LVDS will be next, either devices[0] or [1], if present.
+ * On some platforms the number of device is 6. But could be as few as
+ * 4 if both TV and LVDS are missing.
+ * And the device num is related with the size of general definition
+ * block. It is obtained by using the following formula:
+ * number = (block_size - sizeof(bdb_general_definitions))/
+ * defs->child_dev_size;
+ */
+ uint8_t devices[0];
+} __packed;
+
+/* Mask for DRRS / Panel Channel / SSC / BLT control bits extraction */
+#define MODE_MASK 0x3
+
+struct bdb_lvds_options {
+ u8 panel_type;
+ u8 rsvd1;
+ /* LVDS capabilities, stored in a dword */
+ u8 pfit_mode:2;
+ u8 pfit_text_mode_enhanced:1;
+ u8 pfit_gfx_mode_enhanced:1;
+ u8 pfit_ratio_auto:1;
+ u8 pixel_dither:1;
+ u8 lvds_edid:1;
+ u8 rsvd2:1;
+ u8 rsvd4;
+ /* LVDS Panel channel bits stored here */
+ u32 lvds_panel_channel_bits;
+ /* LVDS SSC (Spread Spectrum Clock) bits stored here. */
+ u16 ssc_bits;
+ u16 ssc_freq;
+ u16 ssc_ddt;
+ /* Panel color depth defined here */
+ u16 panel_color_depth;
+ /* LVDS panel type bits stored here */
+ u32 dps_panel_type_bits;
+ /* LVDS backlight control type bits stored here */
+ u32 blt_control_type_bits;
+} __packed;
+
+/* LFP pointer table contains entries to the struct below */
+struct bdb_lvds_lfp_data_ptr {
+ u16 fp_timing_offset; /* offsets are from start of bdb */
+ u8 fp_table_size;
+ u16 dvo_timing_offset;
+ u8 dvo_table_size;
+ u16 panel_pnp_id_offset;
+ u8 pnp_table_size;
+} __packed;
+
+struct bdb_lvds_lfp_data_ptrs {
+ u8 lvds_entries; /* followed by one or more lvds_data_ptr structs */
+ struct bdb_lvds_lfp_data_ptr ptr[16];
+} __packed;
+
+/* LFP data has 3 blocks per entry */
+struct lvds_fp_timing {
+ u16 x_res;
+ u16 y_res;
+ u32 lvds_reg;
+ u32 lvds_reg_val;
+ u32 pp_on_reg;
+ u32 pp_on_reg_val;
+ u32 pp_off_reg;
+ u32 pp_off_reg_val;
+ u32 pp_cycle_reg;
+ u32 pp_cycle_reg_val;
+ u32 pfit_reg;
+ u32 pfit_reg_val;
+ u16 terminator;
+} __packed;
+
+struct lvds_dvo_timing {
+ u16 clock; /**< In 10khz */
+ u8 hactive_lo;
+ u8 hblank_lo;
+ u8 hblank_hi:4;
+ u8 hactive_hi:4;
+ u8 vactive_lo;
+ u8 vblank_lo;
+ u8 vblank_hi:4;
+ u8 vactive_hi:4;
+ u8 hsync_off_lo;
+ u8 hsync_pulse_width;
+ u8 vsync_pulse_width:4;
+ u8 vsync_off:4;
+ u8 rsvd0:6;
+ u8 hsync_off_hi:2;
+ u8 h_image;
+ u8 v_image;
+ u8 max_hv;
+ u8 h_border;
+ u8 v_border;
+ u8 rsvd1:3;
+ u8 digital:2;
+ u8 vsync_positive:1;
+ u8 hsync_positive:1;
+ u8 rsvd2:1;
+} __packed;
+
+struct lvds_pnp_id {
+ u16 mfg_name;
+ u16 product_code;
+ u32 serial;
+ u8 mfg_week;
+ u8 mfg_year;
+} __packed;
+
+struct bdb_lvds_lfp_data_entry {
+ struct lvds_fp_timing fp_timing;
+ struct lvds_dvo_timing dvo_timing;
+ struct lvds_pnp_id pnp_id;
+} __packed;
+
+struct bdb_lvds_lfp_data {
+ struct bdb_lvds_lfp_data_entry data[16];
+} __packed;
+
+#define BDB_BACKLIGHT_TYPE_NONE 0
+#define BDB_BACKLIGHT_TYPE_PWM 2
+
+struct bdb_lfp_backlight_data_entry {
+ u8 type:2;
+ u8 active_low_pwm:1;
+ u8 obsolete1:5;
+ u16 pwm_freq_hz;
+ u8 min_brightness;
+ u8 obsolete2;
+ u8 obsolete3;
+} __packed;
+
+struct bdb_lfp_backlight_data {
+ u8 entry_size;
+ struct bdb_lfp_backlight_data_entry data[16];
+ u8 level[16];
+} __packed;
+
+struct aimdb_header {
+ char signature[16];
+ char oem_device[20];
+ u16 aimdb_version;
+ u16 aimdb_header_size;
+ u16 aimdb_size;
+} __packed;
+
+struct aimdb_block {
+ u8 aimdb_id;
+ u16 aimdb_size;
+} __packed;
+
+struct vch_panel_data {
+ u16 fp_timing_offset;
+ u8 fp_timing_size;
+ u16 dvo_timing_offset;
+ u8 dvo_timing_size;
+ u16 text_fitting_offset;
+ u8 text_fitting_size;
+ u16 graphics_fitting_offset;
+ u8 graphics_fitting_size;
+} __packed;
+
+struct vch_bdb_22 {
+ struct aimdb_block aimdb_block;
+ struct vch_panel_data panels[16];
+} __packed;
+
+struct bdb_sdvo_lvds_options {
+ u8 panel_backlight;
+ u8 h40_set_panel_type;
+ u8 panel_type;
+ u8 ssc_clk_freq;
+ u16 als_low_trip;
+ u16 als_high_trip;
+ u8 sclalarcoeff_tab_row_num;
+ u8 sclalarcoeff_tab_row_size;
+ u8 coefficient[8];
+ u8 panel_misc_bits_1;
+ u8 panel_misc_bits_2;
+ u8 panel_misc_bits_3;
+ u8 panel_misc_bits_4;
+} __packed;
+
+
+#define BDB_DRIVER_FEATURE_NO_LVDS 0
+#define BDB_DRIVER_FEATURE_INT_LVDS 1
+#define BDB_DRIVER_FEATURE_SDVO_LVDS 2
+#define BDB_DRIVER_FEATURE_EDP 3
+
+struct bdb_driver_features {
+ u8 boot_dev_algorithm:1;
+ u8 block_display_switch:1;
+ u8 allow_display_switch:1;
+ u8 hotplug_dvo:1;
+ u8 dual_view_zoom:1;
+ u8 int15h_hook:1;
+ u8 sprite_in_clone:1;
+ u8 primary_lfp_id:1;
+
+ u16 boot_mode_x;
+ u16 boot_mode_y;
+ u8 boot_mode_bpp;
+ u8 boot_mode_refresh;
+
+ u16 enable_lfp_primary:1;
+ u16 selective_mode_pruning:1;
+ u16 dual_frequency:1;
+ u16 render_clock_freq:1; /* 0: high freq; 1: low freq */
+ u16 nt_clone_support:1;
+ u16 power_scheme_ui:1; /* 0: CUI; 1: 3rd party */
+ u16 sprite_display_assign:1; /* 0: secondary; 1: primary */
+ u16 cui_aspect_scaling:1;
+ u16 preserve_aspect_ratio:1;
+ u16 sdvo_device_power_down:1;
+ u16 crt_hotplug:1;
+ u16 lvds_config:2;
+ u16 tv_hotplug:1;
+ u16 hdmi_config:2;
+
+ u8 static_display:1;
+ u8 reserved2:7;
+ u16 legacy_crt_max_x;
+ u16 legacy_crt_max_y;
+ u8 legacy_crt_max_refresh;
+
+ u8 hdmi_termination;
+ u8 custom_vbt_version;
+ /* Driver features data block */
+ u16 rmpm_enabled:1;
+ u16 s2ddt_enabled:1;
+ u16 dpst_enabled:1;
+ u16 bltclt_enabled:1;
+ u16 adb_enabled:1;
+ u16 drrs_enabled:1;
+ u16 grs_enabled:1;
+ u16 gpmt_enabled:1;
+ u16 tbt_enabled:1;
+ u16 psr_enabled:1;
+ u16 ips_enabled:1;
+ u16 reserved3:4;
+ u16 pc_feature_valid:1;
+} __packed;
+
+#define EDP_18BPP 0
+#define EDP_24BPP 1
+#define EDP_30BPP 2
+#define EDP_RATE_1_62 0
+#define EDP_RATE_2_7 1
+#define EDP_LANE_1 0
+#define EDP_LANE_2 1
+#define EDP_LANE_4 3
+#define EDP_PREEMPHASIS_NONE 0
+#define EDP_PREEMPHASIS_3_5dB 1
+#define EDP_PREEMPHASIS_6dB 2
+#define EDP_PREEMPHASIS_9_5dB 3
+#define EDP_VSWING_0_4V 0
+#define EDP_VSWING_0_6V 1
+#define EDP_VSWING_0_8V 2
+#define EDP_VSWING_1_2V 3
+
+
+struct edp_link_params {
+ u8 rate:4;
+ u8 lanes:4;
+ u8 preemphasis:4;
+ u8 vswing:4;
+} __packed;
+
+struct bdb_edp {
+ struct edp_power_seq power_seqs[16];
+ u32 color_depth;
+ struct edp_link_params link_params[16];
+ u32 sdrrs_msa_timing_delay;
+
+ /* ith bit indicates enabled/disabled for (i+1)th panel */
+ u16 edp_s3d_feature;
+ u16 edp_t3_optimization;
+ u64 edp_vswing_preemph; /* v173 */
+} __packed;
+
+struct psr_table {
+ /* Feature bits */
+ u8 full_link:1;
+ u8 require_aux_to_wakeup:1;
+ u8 feature_bits_rsvd:6;
+
+ /* Wait times */
+ u8 idle_frames:4;
+ u8 lines_to_wait:3;
+ u8 wait_times_rsvd:1;
+
+ /* TP wake up time in multiple of 100 */
+ u16 tp1_wakeup_time;
+ u16 tp2_tp3_wakeup_time;
+} __packed;
+
+struct bdb_psr {
+ struct psr_table psr_table[16];
+} __packed;
+
+/*
+ * Driver<->VBIOS interaction occurs through scratch bits in
+ * GR18 & SWF*.
+ */
+
+/* GR18 bits are set on display switch and hotkey events */
+#define GR18_DRIVER_SWITCH_EN (1<<7) /* 0: VBIOS control, 1: driver control */
+#define GR18_HOTKEY_MASK 0x78 /* See also SWF4 15:0 */
+#define GR18_HK_NONE (0x0<<3)
+#define GR18_HK_LFP_STRETCH (0x1<<3)
+#define GR18_HK_TOGGLE_DISP (0x2<<3)
+#define GR18_HK_DISP_SWITCH (0x4<<3) /* see SWF14 15:0 for what to enable */
+#define GR18_HK_POPUP_DISABLED (0x6<<3)
+#define GR18_HK_POPUP_ENABLED (0x7<<3)
+#define GR18_HK_PFIT (0x8<<3)
+#define GR18_HK_APM_CHANGE (0xa<<3)
+#define GR18_HK_MULTIPLE (0xc<<3)
+#define GR18_USER_INT_EN (1<<2)
+#define GR18_A0000_FLUSH_EN (1<<1)
+#define GR18_SMM_EN (1<<0)
+
+/* Set by driver, cleared by VBIOS */
+#define SWF00_YRES_SHIFT 16
+#define SWF00_XRES_SHIFT 0
+#define SWF00_RES_MASK 0xffff
+
+/* Set by VBIOS at boot time and driver at runtime */
+#define SWF01_TV2_FORMAT_SHIFT 8
+#define SWF01_TV1_FORMAT_SHIFT 0
+#define SWF01_TV_FORMAT_MASK 0xffff
+
+#define SWF10_VBIOS_BLC_I2C_EN (1<<29)
+#define SWF10_GTT_OVERRIDE_EN (1<<28)
+#define SWF10_LFP_DPMS_OVR (1<<27) /* override DPMS on display switch */
+#define SWF10_ACTIVE_TOGGLE_LIST_MASK (7<<24)
+#define SWF10_OLD_TOGGLE 0x0
+#define SWF10_TOGGLE_LIST_1 0x1
+#define SWF10_TOGGLE_LIST_2 0x2
+#define SWF10_TOGGLE_LIST_3 0x3
+#define SWF10_TOGGLE_LIST_4 0x4
+#define SWF10_PANNING_EN (1<<23)
+#define SWF10_DRIVER_LOADED (1<<22)
+#define SWF10_EXTENDED_DESKTOP (1<<21)
+#define SWF10_EXCLUSIVE_MODE (1<<20)
+#define SWF10_OVERLAY_EN (1<<19)
+#define SWF10_PLANEB_HOLDOFF (1<<18)
+#define SWF10_PLANEA_HOLDOFF (1<<17)
+#define SWF10_VGA_HOLDOFF (1<<16)
+#define SWF10_ACTIVE_DISP_MASK 0xffff
+#define SWF10_PIPEB_LFP2 (1<<15)
+#define SWF10_PIPEB_EFP2 (1<<14)
+#define SWF10_PIPEB_TV2 (1<<13)
+#define SWF10_PIPEB_CRT2 (1<<12)
+#define SWF10_PIPEB_LFP (1<<11)
+#define SWF10_PIPEB_EFP (1<<10)
+#define SWF10_PIPEB_TV (1<<9)
+#define SWF10_PIPEB_CRT (1<<8)
+#define SWF10_PIPEA_LFP2 (1<<7)
+#define SWF10_PIPEA_EFP2 (1<<6)
+#define SWF10_PIPEA_TV2 (1<<5)
+#define SWF10_PIPEA_CRT2 (1<<4)
+#define SWF10_PIPEA_LFP (1<<3)
+#define SWF10_PIPEA_EFP (1<<2)
+#define SWF10_PIPEA_TV (1<<1)
+#define SWF10_PIPEA_CRT (1<<0)
+
+#define SWF11_MEMORY_SIZE_SHIFT 16
+#define SWF11_SV_TEST_EN (1<<15)
+#define SWF11_IS_AGP (1<<14)
+#define SWF11_DISPLAY_HOLDOFF (1<<13)
+#define SWF11_DPMS_REDUCED (1<<12)
+#define SWF11_IS_VBE_MODE (1<<11)
+#define SWF11_PIPEB_ACCESS (1<<10) /* 0 here means pipe a */
+#define SWF11_DPMS_MASK 0x07
+#define SWF11_DPMS_OFF (1<<2)
+#define SWF11_DPMS_SUSPEND (1<<1)
+#define SWF11_DPMS_STANDBY (1<<0)
+#define SWF11_DPMS_ON 0
+
+#define SWF14_GFX_PFIT_EN (1<<31)
+#define SWF14_TEXT_PFIT_EN (1<<30)
+#define SWF14_LID_STATUS_CLOSED (1<<29) /* 0 here means open */
+#define SWF14_POPUP_EN (1<<28)
+#define SWF14_DISPLAY_HOLDOFF (1<<27)
+#define SWF14_DISP_DETECT_EN (1<<26)
+#define SWF14_DOCKING_STATUS_DOCKED (1<<25) /* 0 here means undocked */
+#define SWF14_DRIVER_STATUS (1<<24)
+#define SWF14_OS_TYPE_WIN9X (1<<23)
+#define SWF14_OS_TYPE_WINNT (1<<22)
+/* 21:19 rsvd */
+#define SWF14_PM_TYPE_MASK 0x00070000
+#define SWF14_PM_ACPI_VIDEO (0x4 << 16)
+#define SWF14_PM_ACPI (0x3 << 16)
+#define SWF14_PM_APM_12 (0x2 << 16)
+#define SWF14_PM_APM_11 (0x1 << 16)
+#define SWF14_HK_REQUEST_MASK 0x0000ffff /* see GR18 6:3 for event type */
+ /* if GR18 indicates a display switch */
+#define SWF14_DS_PIPEB_LFP2_EN (1<<15)
+#define SWF14_DS_PIPEB_EFP2_EN (1<<14)
+#define SWF14_DS_PIPEB_TV2_EN (1<<13)
+#define SWF14_DS_PIPEB_CRT2_EN (1<<12)
+#define SWF14_DS_PIPEB_LFP_EN (1<<11)
+#define SWF14_DS_PIPEB_EFP_EN (1<<10)
+#define SWF14_DS_PIPEB_TV_EN (1<<9)
+#define SWF14_DS_PIPEB_CRT_EN (1<<8)
+#define SWF14_DS_PIPEA_LFP2_EN (1<<7)
+#define SWF14_DS_PIPEA_EFP2_EN (1<<6)
+#define SWF14_DS_PIPEA_TV2_EN (1<<5)
+#define SWF14_DS_PIPEA_CRT2_EN (1<<4)
+#define SWF14_DS_PIPEA_LFP_EN (1<<3)
+#define SWF14_DS_PIPEA_EFP_EN (1<<2)
+#define SWF14_DS_PIPEA_TV_EN (1<<1)
+#define SWF14_DS_PIPEA_CRT_EN (1<<0)
+ /* if GR18 indicates a panel fitting request */
+#define SWF14_PFIT_EN (1<<0) /* 0 means disable */
+ /* if GR18 indicates an APM change request */
+#define SWF14_APM_HIBERNATE 0x4
+#define SWF14_APM_SUSPEND 0x3
+#define SWF14_APM_STANDBY 0x1
+#define SWF14_APM_RESTORE 0x0
+
+/* Add the device class for LFP, TV, HDMI */
+#define DEVICE_TYPE_INT_LFP 0x1022
+#define DEVICE_TYPE_INT_TV 0x1009
+#define DEVICE_TYPE_HDMI 0x60D2
+#define DEVICE_TYPE_DP 0x68C6
+#define DEVICE_TYPE_eDP 0x78C6
+
+#define DEVICE_TYPE_CLASS_EXTENSION (1 << 15)
+#define DEVICE_TYPE_POWER_MANAGEMENT (1 << 14)
+#define DEVICE_TYPE_HOTPLUG_SIGNALING (1 << 13)
+#define DEVICE_TYPE_INTERNAL_CONNECTOR (1 << 12)
+#define DEVICE_TYPE_NOT_HDMI_OUTPUT (1 << 11)
+#define DEVICE_TYPE_MIPI_OUTPUT (1 << 10)
+#define DEVICE_TYPE_COMPOSITE_OUTPUT (1 << 9)
+#define DEVICE_TYPE_DUAL_CHANNEL (1 << 8)
+#define DEVICE_TYPE_HIGH_SPEED_LINK (1 << 6)
+#define DEVICE_TYPE_LVDS_SINGALING (1 << 5)
+#define DEVICE_TYPE_TMDS_DVI_SIGNALING (1 << 4)
+#define DEVICE_TYPE_VIDEO_SIGNALING (1 << 3)
+#define DEVICE_TYPE_DISPLAYPORT_OUTPUT (1 << 2)
+#define DEVICE_TYPE_DIGITAL_OUTPUT (1 << 1)
+#define DEVICE_TYPE_ANALOG_OUTPUT (1 << 0)
+
+/*
+ * Bits we care about when checking for DEVICE_TYPE_eDP
+ * Depending on the system, the other bits may or may not
+ * be set for eDP outputs.
+ */
+#define DEVICE_TYPE_eDP_BITS \
+ (DEVICE_TYPE_INTERNAL_CONNECTOR | \
+ DEVICE_TYPE_MIPI_OUTPUT | \
+ DEVICE_TYPE_COMPOSITE_OUTPUT | \
+ DEVICE_TYPE_DUAL_CHANNEL | \
+ DEVICE_TYPE_LVDS_SINGALING | \
+ DEVICE_TYPE_TMDS_DVI_SIGNALING | \
+ DEVICE_TYPE_VIDEO_SIGNALING | \
+ DEVICE_TYPE_DISPLAYPORT_OUTPUT | \
+ DEVICE_TYPE_ANALOG_OUTPUT)
+
+/* define the DVO port for HDMI output type */
+#define DVO_B 1
+#define DVO_C 2
+#define DVO_D 3
+
+/* Possible values for the "DVO Port" field for versions >= 155: */
+#define DVO_PORT_HDMIA 0
+#define DVO_PORT_HDMIB 1
+#define DVO_PORT_HDMIC 2
+#define DVO_PORT_HDMID 3
+#define DVO_PORT_LVDS 4
+#define DVO_PORT_TV 5
+#define DVO_PORT_CRT 6
+#define DVO_PORT_DPB 7
+#define DVO_PORT_DPC 8
+#define DVO_PORT_DPD 9
+#define DVO_PORT_DPA 10
+#define DVO_PORT_DPE 11
+#define DVO_PORT_HDMIE 12
+#define DVO_PORT_MIPIA 21
+#define DVO_PORT_MIPIB 22
+#define DVO_PORT_MIPIC 23
+#define DVO_PORT_MIPID 24
+
+/* Block 52 contains MIPI configuration block
+ * 6 * bdb_mipi_config, followed by 6 pps data block
+ * block below
+ */
+#define MAX_MIPI_CONFIGURATIONS 6
+
+struct bdb_mipi_config {
+ struct mipi_config config[MAX_MIPI_CONFIGURATIONS];
+ struct mipi_pps_data pps[MAX_MIPI_CONFIGURATIONS];
+} __packed;
+
+/* Block 53 contains MIPI sequences as needed by the panel
+ * for enabling it. This block can be variable in size and
+ * can be maximum of 6 blocks
+ */
+struct bdb_mipi_sequence {
+ u8 version;
+ u8 data[0];
+} __packed;
+
+enum mipi_gpio_pin_index {
+ MIPI_GPIO_UNDEFINED = 0,
+ MIPI_GPIO_PANEL_ENABLE,
+ MIPI_GPIO_BL_ENABLE,
+ MIPI_GPIO_PWM_ENABLE,
+ MIPI_GPIO_RESET_N,
+ MIPI_GPIO_PWR_DOWN_R,
+ MIPI_GPIO_STDBY_RST_N,
+ MIPI_GPIO_MAX
+};
+
+#endif /* _INTEL_VBT_DEFS_H_ */
diff --git a/drivers/gpu/drm/imx/imx-drm-core.c b/drivers/gpu/drm/imx/imx-drm-core.c
index e26dcde..1080019 100644
--- a/drivers/gpu/drm/imx/imx-drm-core.c
+++ b/drivers/gpu/drm/imx/imx-drm-core.c
@@ -252,13 +252,6 @@
if (ret)
goto err_kms;
- /*
- * with vblank_disable_allowed = true, vblank interrupt will be
- * disabled by drm timer once a current process gives up ownership
- * of vblank event. (after drm_vblank_put function is called)
- */
- drm->vblank_disable_allowed = true;
-
platform_set_drvdata(drm->platformdev, drm);
/* Now try and bind all our sub-components */
@@ -411,7 +404,7 @@
.unload = imx_drm_driver_unload,
.lastclose = imx_drm_driver_lastclose,
.set_busid = drm_platform_set_busid,
- .gem_free_object = drm_gem_cma_free_object,
+ .gem_free_object_unlocked = drm_gem_cma_free_object,
.gem_vm_ops = &drm_gem_cma_vm_ops,
.dumb_create = drm_gem_cma_dumb_create,
.dumb_map_offset = drm_gem_cma_dumb_map_offset,
diff --git a/drivers/gpu/drm/mediatek/Kconfig b/drivers/gpu/drm/mediatek/Kconfig
new file mode 100644
index 0000000..eeefc97
--- /dev/null
+++ b/drivers/gpu/drm/mediatek/Kconfig
@@ -0,0 +1,16 @@
+config DRM_MEDIATEK
+ tristate "DRM Support for Mediatek SoCs"
+ depends on DRM
+ depends on ARCH_MEDIATEK || (ARM && COMPILE_TEST)
+ select DRM_GEM_CMA_HELPER
+ select DRM_KMS_HELPER
+ select DRM_MIPI_DSI
+ select DRM_PANEL
+ select IOMMU_DMA
+ select MEMORY
+ select MTK_SMI
+ help
+ Choose this option if you have a Mediatek SoCs.
+ The module will be called mediatek-drm
+ This driver provides kernel mode setting and
+ buffer management to userspace.
diff --git a/drivers/gpu/drm/mediatek/Makefile b/drivers/gpu/drm/mediatek/Makefile
new file mode 100644
index 0000000..5fcf58e
--- /dev/null
+++ b/drivers/gpu/drm/mediatek/Makefile
@@ -0,0 +1,14 @@
+mediatek-drm-y := mtk_disp_ovl.o \
+ mtk_disp_rdma.o \
+ mtk_drm_crtc.o \
+ mtk_drm_ddp.o \
+ mtk_drm_ddp_comp.o \
+ mtk_drm_drv.o \
+ mtk_drm_fb.o \
+ mtk_drm_gem.o \
+ mtk_drm_plane.o \
+ mtk_dsi.o \
+ mtk_mipi_tx.o \
+ mtk_dpi.o
+
+obj-$(CONFIG_DRM_MEDIATEK) += mediatek-drm.o
diff --git a/drivers/gpu/drm/mediatek/mtk_disp_ovl.c b/drivers/gpu/drm/mediatek/mtk_disp_ovl.c
new file mode 100644
index 0000000..8f62671f
--- /dev/null
+++ b/drivers/gpu/drm/mediatek/mtk_disp_ovl.c
@@ -0,0 +1,302 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <drm/drmP.h>
+#include <linux/clk.h>
+#include <linux/component.h>
+#include <linux/of_device.h>
+#include <linux/of_irq.h>
+#include <linux/platform_device.h>
+
+#include "mtk_drm_crtc.h"
+#include "mtk_drm_ddp_comp.h"
+
+#define DISP_REG_OVL_INTEN 0x0004
+#define OVL_FME_CPL_INT BIT(1)
+#define DISP_REG_OVL_INTSTA 0x0008
+#define DISP_REG_OVL_EN 0x000c
+#define DISP_REG_OVL_RST 0x0014
+#define DISP_REG_OVL_ROI_SIZE 0x0020
+#define DISP_REG_OVL_ROI_BGCLR 0x0028
+#define DISP_REG_OVL_SRC_CON 0x002c
+#define DISP_REG_OVL_CON(n) (0x0030 + 0x20 * (n))
+#define DISP_REG_OVL_SRC_SIZE(n) (0x0038 + 0x20 * (n))
+#define DISP_REG_OVL_OFFSET(n) (0x003c + 0x20 * (n))
+#define DISP_REG_OVL_PITCH(n) (0x0044 + 0x20 * (n))
+#define DISP_REG_OVL_RDMA_CTRL(n) (0x00c0 + 0x20 * (n))
+#define DISP_REG_OVL_RDMA_GMC(n) (0x00c8 + 0x20 * (n))
+#define DISP_REG_OVL_ADDR(n) (0x0f40 + 0x20 * (n))
+
+#define OVL_RDMA_MEM_GMC 0x40402020
+
+#define OVL_CON_BYTE_SWAP BIT(24)
+#define OVL_CON_CLRFMT_RGB565 (0 << 12)
+#define OVL_CON_CLRFMT_RGB888 (1 << 12)
+#define OVL_CON_CLRFMT_RGBA8888 (2 << 12)
+#define OVL_CON_CLRFMT_ARGB8888 (3 << 12)
+#define OVL_CON_AEN BIT(8)
+#define OVL_CON_ALPHA 0xff
+
+/**
+ * struct mtk_disp_ovl - DISP_OVL driver structure
+ * @ddp_comp - structure containing type enum and hardware resources
+ * @crtc - associated crtc to report vblank events to
+ */
+struct mtk_disp_ovl {
+ struct mtk_ddp_comp ddp_comp;
+ struct drm_crtc *crtc;
+};
+
+static irqreturn_t mtk_disp_ovl_irq_handler(int irq, void *dev_id)
+{
+ struct mtk_disp_ovl *priv = dev_id;
+ struct mtk_ddp_comp *ovl = &priv->ddp_comp;
+
+ /* Clear frame completion interrupt */
+ writel(0x0, ovl->regs + DISP_REG_OVL_INTSTA);
+
+ if (!priv->crtc)
+ return IRQ_NONE;
+
+ mtk_crtc_ddp_irq(priv->crtc, ovl);
+
+ return IRQ_HANDLED;
+}
+
+static void mtk_ovl_enable_vblank(struct mtk_ddp_comp *comp,
+ struct drm_crtc *crtc)
+{
+ struct mtk_disp_ovl *priv = container_of(comp, struct mtk_disp_ovl,
+ ddp_comp);
+
+ priv->crtc = crtc;
+ writel_relaxed(OVL_FME_CPL_INT, comp->regs + DISP_REG_OVL_INTEN);
+}
+
+static void mtk_ovl_disable_vblank(struct mtk_ddp_comp *comp)
+{
+ struct mtk_disp_ovl *priv = container_of(comp, struct mtk_disp_ovl,
+ ddp_comp);
+
+ priv->crtc = NULL;
+ writel_relaxed(0x0, comp->regs + DISP_REG_OVL_INTEN);
+}
+
+static void mtk_ovl_start(struct mtk_ddp_comp *comp)
+{
+ writel_relaxed(0x1, comp->regs + DISP_REG_OVL_EN);
+}
+
+static void mtk_ovl_stop(struct mtk_ddp_comp *comp)
+{
+ writel_relaxed(0x0, comp->regs + DISP_REG_OVL_EN);
+}
+
+static void mtk_ovl_config(struct mtk_ddp_comp *comp, unsigned int w,
+ unsigned int h, unsigned int vrefresh)
+{
+ if (w != 0 && h != 0)
+ writel_relaxed(h << 16 | w, comp->regs + DISP_REG_OVL_ROI_SIZE);
+ writel_relaxed(0x0, comp->regs + DISP_REG_OVL_ROI_BGCLR);
+
+ writel(0x1, comp->regs + DISP_REG_OVL_RST);
+ writel(0x0, comp->regs + DISP_REG_OVL_RST);
+}
+
+static void mtk_ovl_layer_on(struct mtk_ddp_comp *comp, unsigned int idx)
+{
+ unsigned int reg;
+
+ writel(0x1, comp->regs + DISP_REG_OVL_RDMA_CTRL(idx));
+ writel(OVL_RDMA_MEM_GMC, comp->regs + DISP_REG_OVL_RDMA_GMC(idx));
+
+ reg = readl(comp->regs + DISP_REG_OVL_SRC_CON);
+ reg = reg | BIT(idx);
+ writel(reg, comp->regs + DISP_REG_OVL_SRC_CON);
+}
+
+static void mtk_ovl_layer_off(struct mtk_ddp_comp *comp, unsigned int idx)
+{
+ unsigned int reg;
+
+ reg = readl(comp->regs + DISP_REG_OVL_SRC_CON);
+ reg = reg & ~BIT(idx);
+ writel(reg, comp->regs + DISP_REG_OVL_SRC_CON);
+
+ writel(0x0, comp->regs + DISP_REG_OVL_RDMA_CTRL(idx));
+}
+
+static unsigned int ovl_fmt_convert(unsigned int fmt)
+{
+ switch (fmt) {
+ default:
+ case DRM_FORMAT_RGB565:
+ return OVL_CON_CLRFMT_RGB565;
+ case DRM_FORMAT_BGR565:
+ return OVL_CON_CLRFMT_RGB565 | OVL_CON_BYTE_SWAP;
+ case DRM_FORMAT_RGB888:
+ return OVL_CON_CLRFMT_RGB888;
+ case DRM_FORMAT_BGR888:
+ return OVL_CON_CLRFMT_RGB888 | OVL_CON_BYTE_SWAP;
+ case DRM_FORMAT_RGBX8888:
+ case DRM_FORMAT_RGBA8888:
+ return OVL_CON_CLRFMT_ARGB8888;
+ case DRM_FORMAT_BGRX8888:
+ case DRM_FORMAT_BGRA8888:
+ return OVL_CON_CLRFMT_ARGB8888 | OVL_CON_BYTE_SWAP;
+ case DRM_FORMAT_XRGB8888:
+ case DRM_FORMAT_ARGB8888:
+ return OVL_CON_CLRFMT_RGBA8888;
+ case DRM_FORMAT_XBGR8888:
+ case DRM_FORMAT_ABGR8888:
+ return OVL_CON_CLRFMT_RGBA8888 | OVL_CON_BYTE_SWAP;
+ }
+}
+
+static void mtk_ovl_layer_config(struct mtk_ddp_comp *comp, unsigned int idx,
+ struct mtk_plane_state *state)
+{
+ struct mtk_plane_pending_state *pending = &state->pending;
+ unsigned int addr = pending->addr;
+ unsigned int pitch = pending->pitch & 0xffff;
+ unsigned int fmt = pending->format;
+ unsigned int offset = (pending->y << 16) | pending->x;
+ unsigned int src_size = (pending->height << 16) | pending->width;
+ unsigned int con;
+
+ if (!pending->enable)
+ mtk_ovl_layer_off(comp, idx);
+
+ con = ovl_fmt_convert(fmt);
+ if (idx != 0)
+ con |= OVL_CON_AEN | OVL_CON_ALPHA;
+
+ writel_relaxed(con, comp->regs + DISP_REG_OVL_CON(idx));
+ writel_relaxed(pitch, comp->regs + DISP_REG_OVL_PITCH(idx));
+ writel_relaxed(src_size, comp->regs + DISP_REG_OVL_SRC_SIZE(idx));
+ writel_relaxed(offset, comp->regs + DISP_REG_OVL_OFFSET(idx));
+ writel_relaxed(addr, comp->regs + DISP_REG_OVL_ADDR(idx));
+
+ if (pending->enable)
+ mtk_ovl_layer_on(comp, idx);
+}
+
+static const struct mtk_ddp_comp_funcs mtk_disp_ovl_funcs = {
+ .config = mtk_ovl_config,
+ .start = mtk_ovl_start,
+ .stop = mtk_ovl_stop,
+ .enable_vblank = mtk_ovl_enable_vblank,
+ .disable_vblank = mtk_ovl_disable_vblank,
+ .layer_on = mtk_ovl_layer_on,
+ .layer_off = mtk_ovl_layer_off,
+ .layer_config = mtk_ovl_layer_config,
+};
+
+static int mtk_disp_ovl_bind(struct device *dev, struct device *master,
+ void *data)
+{
+ struct mtk_disp_ovl *priv = dev_get_drvdata(dev);
+ struct drm_device *drm_dev = data;
+ int ret;
+
+ ret = mtk_ddp_comp_register(drm_dev, &priv->ddp_comp);
+ if (ret < 0) {
+ dev_err(dev, "Failed to register component %s: %d\n",
+ dev->of_node->full_name, ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static void mtk_disp_ovl_unbind(struct device *dev, struct device *master,
+ void *data)
+{
+ struct mtk_disp_ovl *priv = dev_get_drvdata(dev);
+ struct drm_device *drm_dev = data;
+
+ mtk_ddp_comp_unregister(drm_dev, &priv->ddp_comp);
+}
+
+static const struct component_ops mtk_disp_ovl_component_ops = {
+ .bind = mtk_disp_ovl_bind,
+ .unbind = mtk_disp_ovl_unbind,
+};
+
+static int mtk_disp_ovl_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct mtk_disp_ovl *priv;
+ int comp_id;
+ int irq;
+ int ret;
+
+ priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0)
+ return irq;
+
+ ret = devm_request_irq(dev, irq, mtk_disp_ovl_irq_handler,
+ IRQF_TRIGGER_NONE, dev_name(dev), priv);
+ if (ret < 0) {
+ dev_err(dev, "Failed to request irq %d: %d\n", irq, ret);
+ return ret;
+ }
+
+ comp_id = mtk_ddp_comp_get_id(dev->of_node, MTK_DISP_OVL);
+ if (comp_id < 0) {
+ dev_err(dev, "Failed to identify by alias: %d\n", comp_id);
+ return comp_id;
+ }
+
+ ret = mtk_ddp_comp_init(dev, dev->of_node, &priv->ddp_comp, comp_id,
+ &mtk_disp_ovl_funcs);
+ if (ret) {
+ dev_err(dev, "Failed to initialize component: %d\n", ret);
+ return ret;
+ }
+
+ platform_set_drvdata(pdev, priv);
+
+ ret = component_add(dev, &mtk_disp_ovl_component_ops);
+ if (ret)
+ dev_err(dev, "Failed to add component: %d\n", ret);
+
+ return ret;
+}
+
+static int mtk_disp_ovl_remove(struct platform_device *pdev)
+{
+ component_del(&pdev->dev, &mtk_disp_ovl_component_ops);
+
+ return 0;
+}
+
+static const struct of_device_id mtk_disp_ovl_driver_dt_match[] = {
+ { .compatible = "mediatek,mt8173-disp-ovl", },
+ {},
+};
+MODULE_DEVICE_TABLE(of, mtk_disp_ovl_driver_dt_match);
+
+struct platform_driver mtk_disp_ovl_driver = {
+ .probe = mtk_disp_ovl_probe,
+ .remove = mtk_disp_ovl_remove,
+ .driver = {
+ .name = "mediatek-disp-ovl",
+ .owner = THIS_MODULE,
+ .of_match_table = mtk_disp_ovl_driver_dt_match,
+ },
+};
diff --git a/drivers/gpu/drm/mediatek/mtk_disp_rdma.c b/drivers/gpu/drm/mediatek/mtk_disp_rdma.c
new file mode 100644
index 0000000..5fb80cb
--- /dev/null
+++ b/drivers/gpu/drm/mediatek/mtk_disp_rdma.c
@@ -0,0 +1,240 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <drm/drmP.h>
+#include <linux/clk.h>
+#include <linux/component.h>
+#include <linux/of_device.h>
+#include <linux/of_irq.h>
+#include <linux/platform_device.h>
+
+#include "mtk_drm_crtc.h"
+#include "mtk_drm_ddp_comp.h"
+
+#define DISP_REG_RDMA_INT_ENABLE 0x0000
+#define DISP_REG_RDMA_INT_STATUS 0x0004
+#define RDMA_TARGET_LINE_INT BIT(5)
+#define RDMA_FIFO_UNDERFLOW_INT BIT(4)
+#define RDMA_EOF_ABNORMAL_INT BIT(3)
+#define RDMA_FRAME_END_INT BIT(2)
+#define RDMA_FRAME_START_INT BIT(1)
+#define RDMA_REG_UPDATE_INT BIT(0)
+#define DISP_REG_RDMA_GLOBAL_CON 0x0010
+#define RDMA_ENGINE_EN BIT(0)
+#define DISP_REG_RDMA_SIZE_CON_0 0x0014
+#define DISP_REG_RDMA_SIZE_CON_1 0x0018
+#define DISP_REG_RDMA_TARGET_LINE 0x001c
+#define DISP_REG_RDMA_FIFO_CON 0x0040
+#define RDMA_FIFO_UNDERFLOW_EN BIT(31)
+#define RDMA_FIFO_PSEUDO_SIZE(bytes) (((bytes) / 16) << 16)
+#define RDMA_OUTPUT_VALID_FIFO_THRESHOLD(bytes) ((bytes) / 16)
+
+/**
+ * struct mtk_disp_rdma - DISP_RDMA driver structure
+ * @ddp_comp - structure containing type enum and hardware resources
+ * @crtc - associated crtc to report irq events to
+ */
+struct mtk_disp_rdma {
+ struct mtk_ddp_comp ddp_comp;
+ struct drm_crtc *crtc;
+};
+
+static irqreturn_t mtk_disp_rdma_irq_handler(int irq, void *dev_id)
+{
+ struct mtk_disp_rdma *priv = dev_id;
+ struct mtk_ddp_comp *rdma = &priv->ddp_comp;
+
+ /* Clear frame completion interrupt */
+ writel(0x0, rdma->regs + DISP_REG_RDMA_INT_STATUS);
+
+ if (!priv->crtc)
+ return IRQ_NONE;
+
+ mtk_crtc_ddp_irq(priv->crtc, rdma);
+
+ return IRQ_HANDLED;
+}
+
+static void rdma_update_bits(struct mtk_ddp_comp *comp, unsigned int reg,
+ unsigned int mask, unsigned int val)
+{
+ unsigned int tmp = readl(comp->regs + reg);
+
+ tmp = (tmp & ~mask) | (val & mask);
+ writel(tmp, comp->regs + reg);
+}
+
+static void mtk_rdma_enable_vblank(struct mtk_ddp_comp *comp,
+ struct drm_crtc *crtc)
+{
+ struct mtk_disp_rdma *priv = container_of(comp, struct mtk_disp_rdma,
+ ddp_comp);
+
+ priv->crtc = crtc;
+ rdma_update_bits(comp, DISP_REG_RDMA_INT_ENABLE, RDMA_FRAME_END_INT,
+ RDMA_FRAME_END_INT);
+}
+
+static void mtk_rdma_disable_vblank(struct mtk_ddp_comp *comp)
+{
+ struct mtk_disp_rdma *priv = container_of(comp, struct mtk_disp_rdma,
+ ddp_comp);
+
+ priv->crtc = NULL;
+ rdma_update_bits(comp, DISP_REG_RDMA_INT_ENABLE, RDMA_FRAME_END_INT, 0);
+}
+
+static void mtk_rdma_start(struct mtk_ddp_comp *comp)
+{
+ rdma_update_bits(comp, DISP_REG_RDMA_GLOBAL_CON, RDMA_ENGINE_EN,
+ RDMA_ENGINE_EN);
+}
+
+static void mtk_rdma_stop(struct mtk_ddp_comp *comp)
+{
+ rdma_update_bits(comp, DISP_REG_RDMA_GLOBAL_CON, RDMA_ENGINE_EN, 0);
+}
+
+static void mtk_rdma_config(struct mtk_ddp_comp *comp, unsigned int width,
+ unsigned int height, unsigned int vrefresh)
+{
+ unsigned int threshold;
+ unsigned int reg;
+
+ rdma_update_bits(comp, DISP_REG_RDMA_SIZE_CON_0, 0xfff, width);
+ rdma_update_bits(comp, DISP_REG_RDMA_SIZE_CON_1, 0xfffff, height);
+
+ /*
+ * Enable FIFO underflow since DSI and DPI can't be blocked.
+ * Keep the FIFO pseudo size reset default of 8 KiB. Set the
+ * output threshold to 6 microseconds with 7/6 overhead to
+ * account for blanking, and with a pixel depth of 4 bytes:
+ */
+ threshold = width * height * vrefresh * 4 * 7 / 1000000;
+ reg = RDMA_FIFO_UNDERFLOW_EN |
+ RDMA_FIFO_PSEUDO_SIZE(SZ_8K) |
+ RDMA_OUTPUT_VALID_FIFO_THRESHOLD(threshold);
+ writel(reg, comp->regs + DISP_REG_RDMA_FIFO_CON);
+}
+
+static const struct mtk_ddp_comp_funcs mtk_disp_rdma_funcs = {
+ .config = mtk_rdma_config,
+ .start = mtk_rdma_start,
+ .stop = mtk_rdma_stop,
+ .enable_vblank = mtk_rdma_enable_vblank,
+ .disable_vblank = mtk_rdma_disable_vblank,
+};
+
+static int mtk_disp_rdma_bind(struct device *dev, struct device *master,
+ void *data)
+{
+ struct mtk_disp_rdma *priv = dev_get_drvdata(dev);
+ struct drm_device *drm_dev = data;
+ int ret;
+
+ ret = mtk_ddp_comp_register(drm_dev, &priv->ddp_comp);
+ if (ret < 0) {
+ dev_err(dev, "Failed to register component %s: %d\n",
+ dev->of_node->full_name, ret);
+ return ret;
+ }
+
+ return 0;
+
+}
+
+static void mtk_disp_rdma_unbind(struct device *dev, struct device *master,
+ void *data)
+{
+ struct mtk_disp_rdma *priv = dev_get_drvdata(dev);
+ struct drm_device *drm_dev = data;
+
+ mtk_ddp_comp_unregister(drm_dev, &priv->ddp_comp);
+}
+
+static const struct component_ops mtk_disp_rdma_component_ops = {
+ .bind = mtk_disp_rdma_bind,
+ .unbind = mtk_disp_rdma_unbind,
+};
+
+static int mtk_disp_rdma_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct mtk_disp_rdma *priv;
+ int comp_id;
+ int irq;
+ int ret;
+
+ priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0)
+ return irq;
+
+ comp_id = mtk_ddp_comp_get_id(dev->of_node, MTK_DISP_RDMA);
+ if (comp_id < 0) {
+ dev_err(dev, "Failed to identify by alias: %d\n", comp_id);
+ return comp_id;
+ }
+
+ ret = mtk_ddp_comp_init(dev, dev->of_node, &priv->ddp_comp, comp_id,
+ &mtk_disp_rdma_funcs);
+ if (ret) {
+ dev_err(dev, "Failed to initialize component: %d\n", ret);
+ return ret;
+ }
+
+ /* Disable and clear pending interrupts */
+ writel(0x0, priv->ddp_comp.regs + DISP_REG_RDMA_INT_ENABLE);
+ writel(0x0, priv->ddp_comp.regs + DISP_REG_RDMA_INT_STATUS);
+
+ ret = devm_request_irq(dev, irq, mtk_disp_rdma_irq_handler,
+ IRQF_TRIGGER_NONE, dev_name(dev), priv);
+ if (ret < 0) {
+ dev_err(dev, "Failed to request irq %d: %d\n", irq, ret);
+ return ret;
+ }
+
+ platform_set_drvdata(pdev, priv);
+
+ ret = component_add(dev, &mtk_disp_rdma_component_ops);
+ if (ret)
+ dev_err(dev, "Failed to add component: %d\n", ret);
+
+ return ret;
+}
+
+static int mtk_disp_rdma_remove(struct platform_device *pdev)
+{
+ component_del(&pdev->dev, &mtk_disp_rdma_component_ops);
+
+ return 0;
+}
+
+static const struct of_device_id mtk_disp_rdma_driver_dt_match[] = {
+ { .compatible = "mediatek,mt8173-disp-rdma", },
+ {},
+};
+MODULE_DEVICE_TABLE(of, mtk_disp_rdma_driver_dt_match);
+
+struct platform_driver mtk_disp_rdma_driver = {
+ .probe = mtk_disp_rdma_probe,
+ .remove = mtk_disp_rdma_remove,
+ .driver = {
+ .name = "mediatek-disp-rdma",
+ .owner = THIS_MODULE,
+ .of_match_table = mtk_disp_rdma_driver_dt_match,
+ },
+};
diff --git a/drivers/gpu/drm/mediatek/mtk_dpi.c b/drivers/gpu/drm/mediatek/mtk_dpi.c
new file mode 100644
index 0000000..d05ca79
--- /dev/null
+++ b/drivers/gpu/drm/mediatek/mtk_dpi.c
@@ -0,0 +1,769 @@
+/*
+ * Copyright (c) 2014 MediaTek Inc.
+ * Author: Jie Qiu <jie.qiu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+#include <drm/drmP.h>
+#include <drm/drm_crtc.h>
+#include <drm/drm_crtc_helper.h>
+#include <linux/kernel.h>
+#include <linux/component.h>
+#include <linux/platform_device.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <linux/interrupt.h>
+#include <linux/types.h>
+#include <linux/clk.h>
+
+#include "mtk_dpi_regs.h"
+#include "mtk_drm_ddp_comp.h"
+
+enum mtk_dpi_out_bit_num {
+ MTK_DPI_OUT_BIT_NUM_8BITS,
+ MTK_DPI_OUT_BIT_NUM_10BITS,
+ MTK_DPI_OUT_BIT_NUM_12BITS,
+ MTK_DPI_OUT_BIT_NUM_16BITS
+};
+
+enum mtk_dpi_out_yc_map {
+ MTK_DPI_OUT_YC_MAP_RGB,
+ MTK_DPI_OUT_YC_MAP_CYCY,
+ MTK_DPI_OUT_YC_MAP_YCYC,
+ MTK_DPI_OUT_YC_MAP_CY,
+ MTK_DPI_OUT_YC_MAP_YC
+};
+
+enum mtk_dpi_out_channel_swap {
+ MTK_DPI_OUT_CHANNEL_SWAP_RGB,
+ MTK_DPI_OUT_CHANNEL_SWAP_GBR,
+ MTK_DPI_OUT_CHANNEL_SWAP_BRG,
+ MTK_DPI_OUT_CHANNEL_SWAP_RBG,
+ MTK_DPI_OUT_CHANNEL_SWAP_GRB,
+ MTK_DPI_OUT_CHANNEL_SWAP_BGR
+};
+
+enum mtk_dpi_out_color_format {
+ MTK_DPI_COLOR_FORMAT_RGB,
+ MTK_DPI_COLOR_FORMAT_RGB_FULL,
+ MTK_DPI_COLOR_FORMAT_YCBCR_444,
+ MTK_DPI_COLOR_FORMAT_YCBCR_422,
+ MTK_DPI_COLOR_FORMAT_XV_YCC,
+ MTK_DPI_COLOR_FORMAT_YCBCR_444_FULL,
+ MTK_DPI_COLOR_FORMAT_YCBCR_422_FULL
+};
+
+struct mtk_dpi {
+ struct mtk_ddp_comp ddp_comp;
+ struct drm_encoder encoder;
+ void __iomem *regs;
+ struct device *dev;
+ struct clk *engine_clk;
+ struct clk *pixel_clk;
+ struct clk *tvd_clk;
+ int irq;
+ struct drm_display_mode mode;
+ enum mtk_dpi_out_color_format color_format;
+ enum mtk_dpi_out_yc_map yc_map;
+ enum mtk_dpi_out_bit_num bit_num;
+ enum mtk_dpi_out_channel_swap channel_swap;
+ bool power_sta;
+ u8 power_ctl;
+};
+
+static inline struct mtk_dpi *mtk_dpi_from_encoder(struct drm_encoder *e)
+{
+ return container_of(e, struct mtk_dpi, encoder);
+}
+
+enum mtk_dpi_polarity {
+ MTK_DPI_POLARITY_RISING,
+ MTK_DPI_POLARITY_FALLING,
+};
+
+enum mtk_dpi_power_ctl {
+ DPI_POWER_START = BIT(0),
+ DPI_POWER_ENABLE = BIT(1),
+};
+
+struct mtk_dpi_polarities {
+ enum mtk_dpi_polarity de_pol;
+ enum mtk_dpi_polarity ck_pol;
+ enum mtk_dpi_polarity hsync_pol;
+ enum mtk_dpi_polarity vsync_pol;
+};
+
+struct mtk_dpi_sync_param {
+ u32 sync_width;
+ u32 front_porch;
+ u32 back_porch;
+ bool shift_half_line;
+};
+
+struct mtk_dpi_yc_limit {
+ u16 y_top;
+ u16 y_bottom;
+ u16 c_top;
+ u16 c_bottom;
+};
+
+static void mtk_dpi_mask(struct mtk_dpi *dpi, u32 offset, u32 val, u32 mask)
+{
+ u32 tmp = readl(dpi->regs + offset) & ~mask;
+
+ tmp |= (val & mask);
+ writel(tmp, dpi->regs + offset);
+}
+
+static void mtk_dpi_sw_reset(struct mtk_dpi *dpi, bool reset)
+{
+ mtk_dpi_mask(dpi, DPI_RET, reset ? RST : 0, RST);
+}
+
+static void mtk_dpi_enable(struct mtk_dpi *dpi)
+{
+ mtk_dpi_mask(dpi, DPI_EN, EN, EN);
+}
+
+static void mtk_dpi_disable(struct mtk_dpi *dpi)
+{
+ mtk_dpi_mask(dpi, DPI_EN, 0, EN);
+}
+
+static void mtk_dpi_config_hsync(struct mtk_dpi *dpi,
+ struct mtk_dpi_sync_param *sync)
+{
+ mtk_dpi_mask(dpi, DPI_TGEN_HWIDTH,
+ sync->sync_width << HPW, HPW_MASK);
+ mtk_dpi_mask(dpi, DPI_TGEN_HPORCH,
+ sync->back_porch << HBP, HBP_MASK);
+ mtk_dpi_mask(dpi, DPI_TGEN_HPORCH, sync->front_porch << HFP,
+ HFP_MASK);
+}
+
+static void mtk_dpi_config_vsync(struct mtk_dpi *dpi,
+ struct mtk_dpi_sync_param *sync,
+ u32 width_addr, u32 porch_addr)
+{
+ mtk_dpi_mask(dpi, width_addr,
+ sync->sync_width << VSYNC_WIDTH_SHIFT,
+ VSYNC_WIDTH_MASK);
+ mtk_dpi_mask(dpi, width_addr,
+ sync->shift_half_line << VSYNC_HALF_LINE_SHIFT,
+ VSYNC_HALF_LINE_MASK);
+ mtk_dpi_mask(dpi, porch_addr,
+ sync->back_porch << VSYNC_BACK_PORCH_SHIFT,
+ VSYNC_BACK_PORCH_MASK);
+ mtk_dpi_mask(dpi, porch_addr,
+ sync->front_porch << VSYNC_FRONT_PORCH_SHIFT,
+ VSYNC_FRONT_PORCH_MASK);
+}
+
+static void mtk_dpi_config_vsync_lodd(struct mtk_dpi *dpi,
+ struct mtk_dpi_sync_param *sync)
+{
+ mtk_dpi_config_vsync(dpi, sync, DPI_TGEN_VWIDTH, DPI_TGEN_VPORCH);
+}
+
+static void mtk_dpi_config_vsync_leven(struct mtk_dpi *dpi,
+ struct mtk_dpi_sync_param *sync)
+{
+ mtk_dpi_config_vsync(dpi, sync, DPI_TGEN_VWIDTH_LEVEN,
+ DPI_TGEN_VPORCH_LEVEN);
+}
+
+static void mtk_dpi_config_vsync_rodd(struct mtk_dpi *dpi,
+ struct mtk_dpi_sync_param *sync)
+{
+ mtk_dpi_config_vsync(dpi, sync, DPI_TGEN_VWIDTH_RODD,
+ DPI_TGEN_VPORCH_RODD);
+}
+
+static void mtk_dpi_config_vsync_reven(struct mtk_dpi *dpi,
+ struct mtk_dpi_sync_param *sync)
+{
+ mtk_dpi_config_vsync(dpi, sync, DPI_TGEN_VWIDTH_REVEN,
+ DPI_TGEN_VPORCH_REVEN);
+}
+
+static void mtk_dpi_config_pol(struct mtk_dpi *dpi,
+ struct mtk_dpi_polarities *dpi_pol)
+{
+ unsigned int pol;
+
+ pol = (dpi_pol->ck_pol == MTK_DPI_POLARITY_RISING ? 0 : CK_POL) |
+ (dpi_pol->de_pol == MTK_DPI_POLARITY_RISING ? 0 : DE_POL) |
+ (dpi_pol->hsync_pol == MTK_DPI_POLARITY_RISING ? 0 : HSYNC_POL) |
+ (dpi_pol->vsync_pol == MTK_DPI_POLARITY_RISING ? 0 : VSYNC_POL);
+ mtk_dpi_mask(dpi, DPI_OUTPUT_SETTING, pol,
+ CK_POL | DE_POL | HSYNC_POL | VSYNC_POL);
+}
+
+static void mtk_dpi_config_3d(struct mtk_dpi *dpi, bool en_3d)
+{
+ mtk_dpi_mask(dpi, DPI_CON, en_3d ? TDFP_EN : 0, TDFP_EN);
+}
+
+static void mtk_dpi_config_interface(struct mtk_dpi *dpi, bool inter)
+{
+ mtk_dpi_mask(dpi, DPI_CON, inter ? INTL_EN : 0, INTL_EN);
+}
+
+static void mtk_dpi_config_fb_size(struct mtk_dpi *dpi, u32 width, u32 height)
+{
+ mtk_dpi_mask(dpi, DPI_SIZE, width << HSIZE, HSIZE_MASK);
+ mtk_dpi_mask(dpi, DPI_SIZE, height << VSIZE, VSIZE_MASK);
+}
+
+static void mtk_dpi_config_channel_limit(struct mtk_dpi *dpi,
+ struct mtk_dpi_yc_limit *limit)
+{
+ mtk_dpi_mask(dpi, DPI_Y_LIMIT, limit->y_bottom << Y_LIMINT_BOT,
+ Y_LIMINT_BOT_MASK);
+ mtk_dpi_mask(dpi, DPI_Y_LIMIT, limit->y_top << Y_LIMINT_TOP,
+ Y_LIMINT_TOP_MASK);
+ mtk_dpi_mask(dpi, DPI_C_LIMIT, limit->c_bottom << C_LIMIT_BOT,
+ C_LIMIT_BOT_MASK);
+ mtk_dpi_mask(dpi, DPI_C_LIMIT, limit->c_top << C_LIMIT_TOP,
+ C_LIMIT_TOP_MASK);
+}
+
+static void mtk_dpi_config_bit_num(struct mtk_dpi *dpi,
+ enum mtk_dpi_out_bit_num num)
+{
+ u32 val;
+
+ switch (num) {
+ case MTK_DPI_OUT_BIT_NUM_8BITS:
+ val = OUT_BIT_8;
+ break;
+ case MTK_DPI_OUT_BIT_NUM_10BITS:
+ val = OUT_BIT_10;
+ break;
+ case MTK_DPI_OUT_BIT_NUM_12BITS:
+ val = OUT_BIT_12;
+ break;
+ case MTK_DPI_OUT_BIT_NUM_16BITS:
+ val = OUT_BIT_16;
+ break;
+ default:
+ val = OUT_BIT_8;
+ break;
+ }
+ mtk_dpi_mask(dpi, DPI_OUTPUT_SETTING, val << OUT_BIT,
+ OUT_BIT_MASK);
+}
+
+static void mtk_dpi_config_yc_map(struct mtk_dpi *dpi,
+ enum mtk_dpi_out_yc_map map)
+{
+ u32 val;
+
+ switch (map) {
+ case MTK_DPI_OUT_YC_MAP_RGB:
+ val = YC_MAP_RGB;
+ break;
+ case MTK_DPI_OUT_YC_MAP_CYCY:
+ val = YC_MAP_CYCY;
+ break;
+ case MTK_DPI_OUT_YC_MAP_YCYC:
+ val = YC_MAP_YCYC;
+ break;
+ case MTK_DPI_OUT_YC_MAP_CY:
+ val = YC_MAP_CY;
+ break;
+ case MTK_DPI_OUT_YC_MAP_YC:
+ val = YC_MAP_YC;
+ break;
+ default:
+ val = YC_MAP_RGB;
+ break;
+ }
+
+ mtk_dpi_mask(dpi, DPI_OUTPUT_SETTING, val << YC_MAP, YC_MAP_MASK);
+}
+
+static void mtk_dpi_config_channel_swap(struct mtk_dpi *dpi,
+ enum mtk_dpi_out_channel_swap swap)
+{
+ u32 val;
+
+ switch (swap) {
+ case MTK_DPI_OUT_CHANNEL_SWAP_RGB:
+ val = SWAP_RGB;
+ break;
+ case MTK_DPI_OUT_CHANNEL_SWAP_GBR:
+ val = SWAP_GBR;
+ break;
+ case MTK_DPI_OUT_CHANNEL_SWAP_BRG:
+ val = SWAP_BRG;
+ break;
+ case MTK_DPI_OUT_CHANNEL_SWAP_RBG:
+ val = SWAP_RBG;
+ break;
+ case MTK_DPI_OUT_CHANNEL_SWAP_GRB:
+ val = SWAP_GRB;
+ break;
+ case MTK_DPI_OUT_CHANNEL_SWAP_BGR:
+ val = SWAP_BGR;
+ break;
+ default:
+ val = SWAP_RGB;
+ break;
+ }
+
+ mtk_dpi_mask(dpi, DPI_OUTPUT_SETTING, val << CH_SWAP, CH_SWAP_MASK);
+}
+
+static void mtk_dpi_config_yuv422_enable(struct mtk_dpi *dpi, bool enable)
+{
+ mtk_dpi_mask(dpi, DPI_CON, enable ? YUV422_EN : 0, YUV422_EN);
+}
+
+static void mtk_dpi_config_csc_enable(struct mtk_dpi *dpi, bool enable)
+{
+ mtk_dpi_mask(dpi, DPI_CON, enable ? CSC_ENABLE : 0, CSC_ENABLE);
+}
+
+static void mtk_dpi_config_swap_input(struct mtk_dpi *dpi, bool enable)
+{
+ mtk_dpi_mask(dpi, DPI_CON, enable ? IN_RB_SWAP : 0, IN_RB_SWAP);
+}
+
+static void mtk_dpi_config_2n_h_fre(struct mtk_dpi *dpi)
+{
+ mtk_dpi_mask(dpi, DPI_H_FRE_CON, H_FRE_2N, H_FRE_2N);
+}
+
+static void mtk_dpi_config_color_format(struct mtk_dpi *dpi,
+ enum mtk_dpi_out_color_format format)
+{
+ if ((format == MTK_DPI_COLOR_FORMAT_YCBCR_444) ||
+ (format == MTK_DPI_COLOR_FORMAT_YCBCR_444_FULL)) {
+ mtk_dpi_config_yuv422_enable(dpi, false);
+ mtk_dpi_config_csc_enable(dpi, true);
+ mtk_dpi_config_swap_input(dpi, false);
+ mtk_dpi_config_channel_swap(dpi, MTK_DPI_OUT_CHANNEL_SWAP_BGR);
+ } else if ((format == MTK_DPI_COLOR_FORMAT_YCBCR_422) ||
+ (format == MTK_DPI_COLOR_FORMAT_YCBCR_422_FULL)) {
+ mtk_dpi_config_yuv422_enable(dpi, true);
+ mtk_dpi_config_csc_enable(dpi, true);
+ mtk_dpi_config_swap_input(dpi, true);
+ mtk_dpi_config_channel_swap(dpi, MTK_DPI_OUT_CHANNEL_SWAP_RGB);
+ } else {
+ mtk_dpi_config_yuv422_enable(dpi, false);
+ mtk_dpi_config_csc_enable(dpi, false);
+ mtk_dpi_config_swap_input(dpi, false);
+ mtk_dpi_config_channel_swap(dpi, MTK_DPI_OUT_CHANNEL_SWAP_RGB);
+ }
+}
+
+static void mtk_dpi_power_off(struct mtk_dpi *dpi, enum mtk_dpi_power_ctl pctl)
+{
+ dpi->power_ctl &= ~pctl;
+
+ if ((dpi->power_ctl & DPI_POWER_START) ||
+ (dpi->power_ctl & DPI_POWER_ENABLE))
+ return;
+
+ if (!dpi->power_sta)
+ return;
+
+ mtk_dpi_disable(dpi);
+ clk_disable_unprepare(dpi->pixel_clk);
+ clk_disable_unprepare(dpi->engine_clk);
+ dpi->power_sta = false;
+}
+
+static int mtk_dpi_power_on(struct mtk_dpi *dpi, enum mtk_dpi_power_ctl pctl)
+{
+ int ret;
+
+ dpi->power_ctl |= pctl;
+
+ if (!(dpi->power_ctl & DPI_POWER_START) &&
+ !(dpi->power_ctl & DPI_POWER_ENABLE))
+ return 0;
+
+ if (dpi->power_sta)
+ return 0;
+
+ ret = clk_prepare_enable(dpi->engine_clk);
+ if (ret) {
+ dev_err(dpi->dev, "Failed to enable engine clock: %d\n", ret);
+ goto err_eng;
+ }
+
+ ret = clk_prepare_enable(dpi->pixel_clk);
+ if (ret) {
+ dev_err(dpi->dev, "Failed to enable pixel clock: %d\n", ret);
+ goto err_pixel;
+ }
+
+ mtk_dpi_enable(dpi);
+ dpi->power_sta = true;
+ return 0;
+
+err_pixel:
+ clk_disable_unprepare(dpi->engine_clk);
+err_eng:
+ dpi->power_ctl &= ~pctl;
+ return ret;
+}
+
+static int mtk_dpi_set_display_mode(struct mtk_dpi *dpi,
+ struct drm_display_mode *mode)
+{
+ struct mtk_dpi_yc_limit limit;
+ struct mtk_dpi_polarities dpi_pol;
+ struct mtk_dpi_sync_param hsync;
+ struct mtk_dpi_sync_param vsync_lodd = { 0 };
+ struct mtk_dpi_sync_param vsync_leven = { 0 };
+ struct mtk_dpi_sync_param vsync_rodd = { 0 };
+ struct mtk_dpi_sync_param vsync_reven = { 0 };
+ unsigned long pix_rate;
+ unsigned long pll_rate;
+ unsigned int factor;
+
+ if (!dpi) {
+ dev_err(dpi->dev, "invalid argument\n");
+ return -EINVAL;
+ }
+
+ pix_rate = 1000UL * mode->clock;
+ if (mode->clock <= 74000)
+ factor = 8 * 3;
+ else
+ factor = 4 * 3;
+ pll_rate = pix_rate * factor;
+
+ dev_dbg(dpi->dev, "Want PLL %lu Hz, pixel clock %lu Hz\n",
+ pll_rate, pix_rate);
+
+ clk_set_rate(dpi->tvd_clk, pll_rate);
+ pll_rate = clk_get_rate(dpi->tvd_clk);
+
+ pix_rate = pll_rate / factor;
+ clk_set_rate(dpi->pixel_clk, pix_rate);
+ pix_rate = clk_get_rate(dpi->pixel_clk);
+
+ dev_dbg(dpi->dev, "Got PLL %lu Hz, pixel clock %lu Hz\n",
+ pll_rate, pix_rate);
+
+ limit.c_bottom = 0x0010;
+ limit.c_top = 0x0FE0;
+ limit.y_bottom = 0x0010;
+ limit.y_top = 0x0FE0;
+
+ dpi_pol.ck_pol = MTK_DPI_POLARITY_FALLING;
+ dpi_pol.de_pol = MTK_DPI_POLARITY_RISING;
+ dpi_pol.hsync_pol = mode->flags & DRM_MODE_FLAG_PHSYNC ?
+ MTK_DPI_POLARITY_FALLING : MTK_DPI_POLARITY_RISING;
+ dpi_pol.vsync_pol = mode->flags & DRM_MODE_FLAG_PVSYNC ?
+ MTK_DPI_POLARITY_FALLING : MTK_DPI_POLARITY_RISING;
+
+ hsync.sync_width = mode->hsync_end - mode->hsync_start;
+ hsync.back_porch = mode->htotal - mode->hsync_end;
+ hsync.front_porch = mode->hsync_start - mode->hdisplay;
+ hsync.shift_half_line = false;
+
+ vsync_lodd.sync_width = mode->vsync_end - mode->vsync_start;
+ vsync_lodd.back_porch = mode->vtotal - mode->vsync_end;
+ vsync_lodd.front_porch = mode->vsync_start - mode->vdisplay;
+ vsync_lodd.shift_half_line = false;
+
+ if (mode->flags & DRM_MODE_FLAG_INTERLACE &&
+ mode->flags & DRM_MODE_FLAG_3D_MASK) {
+ vsync_leven = vsync_lodd;
+ vsync_rodd = vsync_lodd;
+ vsync_reven = vsync_lodd;
+ vsync_leven.shift_half_line = true;
+ vsync_reven.shift_half_line = true;
+ } else if (mode->flags & DRM_MODE_FLAG_INTERLACE &&
+ !(mode->flags & DRM_MODE_FLAG_3D_MASK)) {
+ vsync_leven = vsync_lodd;
+ vsync_leven.shift_half_line = true;
+ } else if (!(mode->flags & DRM_MODE_FLAG_INTERLACE) &&
+ mode->flags & DRM_MODE_FLAG_3D_MASK) {
+ vsync_rodd = vsync_lodd;
+ }
+ mtk_dpi_sw_reset(dpi, true);
+ mtk_dpi_config_pol(dpi, &dpi_pol);
+
+ mtk_dpi_config_hsync(dpi, &hsync);
+ mtk_dpi_config_vsync_lodd(dpi, &vsync_lodd);
+ mtk_dpi_config_vsync_rodd(dpi, &vsync_rodd);
+ mtk_dpi_config_vsync_leven(dpi, &vsync_leven);
+ mtk_dpi_config_vsync_reven(dpi, &vsync_reven);
+
+ mtk_dpi_config_3d(dpi, !!(mode->flags & DRM_MODE_FLAG_3D_MASK));
+ mtk_dpi_config_interface(dpi, !!(mode->flags &
+ DRM_MODE_FLAG_INTERLACE));
+ if (mode->flags & DRM_MODE_FLAG_INTERLACE)
+ mtk_dpi_config_fb_size(dpi, mode->hdisplay, mode->vdisplay / 2);
+ else
+ mtk_dpi_config_fb_size(dpi, mode->hdisplay, mode->vdisplay);
+
+ mtk_dpi_config_channel_limit(dpi, &limit);
+ mtk_dpi_config_bit_num(dpi, dpi->bit_num);
+ mtk_dpi_config_channel_swap(dpi, dpi->channel_swap);
+ mtk_dpi_config_yc_map(dpi, dpi->yc_map);
+ mtk_dpi_config_color_format(dpi, dpi->color_format);
+ mtk_dpi_config_2n_h_fre(dpi);
+ mtk_dpi_sw_reset(dpi, false);
+
+ return 0;
+}
+
+static void mtk_dpi_encoder_destroy(struct drm_encoder *encoder)
+{
+ drm_encoder_cleanup(encoder);
+}
+
+static const struct drm_encoder_funcs mtk_dpi_encoder_funcs = {
+ .destroy = mtk_dpi_encoder_destroy,
+};
+
+static bool mtk_dpi_encoder_mode_fixup(struct drm_encoder *encoder,
+ const struct drm_display_mode *mode,
+ struct drm_display_mode *adjusted_mode)
+{
+ return true;
+}
+
+static void mtk_dpi_encoder_mode_set(struct drm_encoder *encoder,
+ struct drm_display_mode *mode,
+ struct drm_display_mode *adjusted_mode)
+{
+ struct mtk_dpi *dpi = mtk_dpi_from_encoder(encoder);
+
+ drm_mode_copy(&dpi->mode, adjusted_mode);
+}
+
+static void mtk_dpi_encoder_disable(struct drm_encoder *encoder)
+{
+ struct mtk_dpi *dpi = mtk_dpi_from_encoder(encoder);
+
+ mtk_dpi_power_off(dpi, DPI_POWER_ENABLE);
+}
+
+static void mtk_dpi_encoder_enable(struct drm_encoder *encoder)
+{
+ struct mtk_dpi *dpi = mtk_dpi_from_encoder(encoder);
+
+ mtk_dpi_power_on(dpi, DPI_POWER_ENABLE);
+ mtk_dpi_set_display_mode(dpi, &dpi->mode);
+}
+
+static int mtk_dpi_atomic_check(struct drm_encoder *encoder,
+ struct drm_crtc_state *crtc_state,
+ struct drm_connector_state *conn_state)
+{
+ return 0;
+}
+
+static const struct drm_encoder_helper_funcs mtk_dpi_encoder_helper_funcs = {
+ .mode_fixup = mtk_dpi_encoder_mode_fixup,
+ .mode_set = mtk_dpi_encoder_mode_set,
+ .disable = mtk_dpi_encoder_disable,
+ .enable = mtk_dpi_encoder_enable,
+ .atomic_check = mtk_dpi_atomic_check,
+};
+
+static void mtk_dpi_start(struct mtk_ddp_comp *comp)
+{
+ struct mtk_dpi *dpi = container_of(comp, struct mtk_dpi, ddp_comp);
+
+ mtk_dpi_power_on(dpi, DPI_POWER_START);
+}
+
+static void mtk_dpi_stop(struct mtk_ddp_comp *comp)
+{
+ struct mtk_dpi *dpi = container_of(comp, struct mtk_dpi, ddp_comp);
+
+ mtk_dpi_power_off(dpi, DPI_POWER_START);
+}
+
+static const struct mtk_ddp_comp_funcs mtk_dpi_funcs = {
+ .start = mtk_dpi_start,
+ .stop = mtk_dpi_stop,
+};
+
+static int mtk_dpi_bind(struct device *dev, struct device *master, void *data)
+{
+ struct mtk_dpi *dpi = dev_get_drvdata(dev);
+ struct drm_device *drm_dev = data;
+ int ret;
+
+ ret = mtk_ddp_comp_register(drm_dev, &dpi->ddp_comp);
+ if (ret < 0) {
+ dev_err(dev, "Failed to register component %s: %d\n",
+ dev->of_node->full_name, ret);
+ return ret;
+ }
+
+ ret = drm_encoder_init(drm_dev, &dpi->encoder, &mtk_dpi_encoder_funcs,
+ DRM_MODE_ENCODER_TMDS, NULL);
+ if (ret) {
+ dev_err(dev, "Failed to initialize decoder: %d\n", ret);
+ goto err_unregister;
+ }
+ drm_encoder_helper_add(&dpi->encoder, &mtk_dpi_encoder_helper_funcs);
+
+ /* Currently DPI0 is fixed to be driven by OVL1 */
+ dpi->encoder.possible_crtcs = BIT(1);
+
+ dpi->encoder.bridge->encoder = &dpi->encoder;
+ ret = drm_bridge_attach(dpi->encoder.dev, dpi->encoder.bridge);
+ if (ret) {
+ dev_err(dev, "Failed to attach bridge: %d\n", ret);
+ goto err_cleanup;
+ }
+
+ dpi->bit_num = MTK_DPI_OUT_BIT_NUM_8BITS;
+ dpi->channel_swap = MTK_DPI_OUT_CHANNEL_SWAP_RGB;
+ dpi->yc_map = MTK_DPI_OUT_YC_MAP_RGB;
+ dpi->color_format = MTK_DPI_COLOR_FORMAT_RGB;
+
+ return 0;
+
+err_cleanup:
+ drm_encoder_cleanup(&dpi->encoder);
+err_unregister:
+ mtk_ddp_comp_unregister(drm_dev, &dpi->ddp_comp);
+ return ret;
+}
+
+static void mtk_dpi_unbind(struct device *dev, struct device *master,
+ void *data)
+{
+ struct mtk_dpi *dpi = dev_get_drvdata(dev);
+ struct drm_device *drm_dev = data;
+
+ drm_encoder_cleanup(&dpi->encoder);
+ mtk_ddp_comp_unregister(drm_dev, &dpi->ddp_comp);
+}
+
+static const struct component_ops mtk_dpi_component_ops = {
+ .bind = mtk_dpi_bind,
+ .unbind = mtk_dpi_unbind,
+};
+
+static int mtk_dpi_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct mtk_dpi *dpi;
+ struct resource *mem;
+ struct device_node *ep, *bridge_node = NULL;
+ int comp_id;
+ int ret;
+
+ dpi = devm_kzalloc(dev, sizeof(*dpi), GFP_KERNEL);
+ if (!dpi)
+ return -ENOMEM;
+
+ dpi->dev = dev;
+
+ mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ dpi->regs = devm_ioremap_resource(dev, mem);
+ if (IS_ERR(dpi->regs)) {
+ ret = PTR_ERR(dpi->regs);
+ dev_err(dev, "Failed to ioremap mem resource: %d\n", ret);
+ return ret;
+ }
+
+ dpi->engine_clk = devm_clk_get(dev, "engine");
+ if (IS_ERR(dpi->engine_clk)) {
+ ret = PTR_ERR(dpi->engine_clk);
+ dev_err(dev, "Failed to get engine clock: %d\n", ret);
+ return ret;
+ }
+
+ dpi->pixel_clk = devm_clk_get(dev, "pixel");
+ if (IS_ERR(dpi->pixel_clk)) {
+ ret = PTR_ERR(dpi->pixel_clk);
+ dev_err(dev, "Failed to get pixel clock: %d\n", ret);
+ return ret;
+ }
+
+ dpi->tvd_clk = devm_clk_get(dev, "pll");
+ if (IS_ERR(dpi->tvd_clk)) {
+ ret = PTR_ERR(dpi->tvd_clk);
+ dev_err(dev, "Failed to get tvdpll clock: %d\n", ret);
+ return ret;
+ }
+
+ dpi->irq = platform_get_irq(pdev, 0);
+ if (dpi->irq <= 0) {
+ dev_err(dev, "Failed to get irq: %d\n", dpi->irq);
+ return -EINVAL;
+ }
+
+ ep = of_graph_get_next_endpoint(dev->of_node, NULL);
+ if (ep) {
+ bridge_node = of_graph_get_remote_port_parent(ep);
+ of_node_put(ep);
+ }
+ if (!bridge_node) {
+ dev_err(dev, "Failed to find bridge node\n");
+ return -ENODEV;
+ }
+
+ dev_info(dev, "Found bridge node: %s\n", bridge_node->full_name);
+
+ dpi->encoder.bridge = of_drm_find_bridge(bridge_node);
+ of_node_put(bridge_node);
+ if (!dpi->encoder.bridge)
+ return -EPROBE_DEFER;
+
+ comp_id = mtk_ddp_comp_get_id(dev->of_node, MTK_DPI);
+ if (comp_id < 0) {
+ dev_err(dev, "Failed to identify by alias: %d\n", comp_id);
+ return comp_id;
+ }
+
+ ret = mtk_ddp_comp_init(dev, dev->of_node, &dpi->ddp_comp, comp_id,
+ &mtk_dpi_funcs);
+ if (ret) {
+ dev_err(dev, "Failed to initialize component: %d\n", ret);
+ return ret;
+ }
+
+ platform_set_drvdata(pdev, dpi);
+
+ ret = component_add(dev, &mtk_dpi_component_ops);
+ if (ret) {
+ dev_err(dev, "Failed to add component: %d\n", ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int mtk_dpi_remove(struct platform_device *pdev)
+{
+ component_del(&pdev->dev, &mtk_dpi_component_ops);
+
+ return 0;
+}
+
+static const struct of_device_id mtk_dpi_of_ids[] = {
+ { .compatible = "mediatek,mt8173-dpi", },
+ {}
+};
+
+struct platform_driver mtk_dpi_driver = {
+ .probe = mtk_dpi_probe,
+ .remove = mtk_dpi_remove,
+ .driver = {
+ .name = "mediatek-dpi",
+ .of_match_table = mtk_dpi_of_ids,
+ },
+};
diff --git a/drivers/gpu/drm/mediatek/mtk_dpi_regs.h b/drivers/gpu/drm/mediatek/mtk_dpi_regs.h
new file mode 100644
index 0000000..4b6ad47
--- /dev/null
+++ b/drivers/gpu/drm/mediatek/mtk_dpi_regs.h
@@ -0,0 +1,228 @@
+/*
+ * Copyright (c) 2014 MediaTek Inc.
+ * Author: Jie Qiu <jie.qiu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+#ifndef __MTK_DPI_REGS_H
+#define __MTK_DPI_REGS_H
+
+#define DPI_EN 0x00
+#define EN BIT(0)
+
+#define DPI_RET 0x04
+#define RST BIT(0)
+
+#define DPI_INTEN 0x08
+#define INT_VSYNC_EN BIT(0)
+#define INT_VDE_EN BIT(1)
+#define INT_UNDERFLOW_EN BIT(2)
+
+#define DPI_INTSTA 0x0C
+#define INT_VSYNC_STA BIT(0)
+#define INT_VDE_STA BIT(1)
+#define INT_UNDERFLOW_STA BIT(2)
+
+#define DPI_CON 0x10
+#define BG_ENABLE BIT(0)
+#define IN_RB_SWAP BIT(1)
+#define INTL_EN BIT(2)
+#define TDFP_EN BIT(3)
+#define CLPF_EN BIT(4)
+#define YUV422_EN BIT(5)
+#define CSC_ENABLE BIT(6)
+#define R601_SEL BIT(7)
+#define EMBSYNC_EN BIT(8)
+#define VS_LODD_EN BIT(16)
+#define VS_LEVEN_EN BIT(17)
+#define VS_RODD_EN BIT(18)
+#define VS_REVEN BIT(19)
+#define FAKE_DE_LODD BIT(20)
+#define FAKE_DE_LEVEN BIT(21)
+#define FAKE_DE_RODD BIT(22)
+#define FAKE_DE_REVEN BIT(23)
+
+#define DPI_OUTPUT_SETTING 0x14
+#define CH_SWAP 0
+#define CH_SWAP_MASK (0x7 << 0)
+#define SWAP_RGB 0x00
+#define SWAP_GBR 0x01
+#define SWAP_BRG 0x02
+#define SWAP_RBG 0x03
+#define SWAP_GRB 0x04
+#define SWAP_BGR 0x05
+#define BIT_SWAP BIT(3)
+#define B_MASK BIT(4)
+#define G_MASK BIT(5)
+#define R_MASK BIT(6)
+#define DE_MASK BIT(8)
+#define HS_MASK BIT(9)
+#define VS_MASK BIT(10)
+#define DE_POL BIT(12)
+#define HSYNC_POL BIT(13)
+#define VSYNC_POL BIT(14)
+#define CK_POL BIT(15)
+#define OEN_OFF BIT(16)
+#define EDGE_SEL BIT(17)
+#define OUT_BIT 18
+#define OUT_BIT_MASK (0x3 << 18)
+#define OUT_BIT_8 0x00
+#define OUT_BIT_10 0x01
+#define OUT_BIT_12 0x02
+#define OUT_BIT_16 0x03
+#define YC_MAP 20
+#define YC_MAP_MASK (0x7 << 20)
+#define YC_MAP_RGB 0x00
+#define YC_MAP_CYCY 0x04
+#define YC_MAP_YCYC 0x05
+#define YC_MAP_CY 0x06
+#define YC_MAP_YC 0x07
+
+#define DPI_SIZE 0x18
+#define HSIZE 0
+#define HSIZE_MASK (0x1FFF << 0)
+#define VSIZE 16
+#define VSIZE_MASK (0x1FFF << 16)
+
+#define DPI_DDR_SETTING 0x1C
+#define DDR_EN BIT(0)
+#define DDDR_SEL BIT(1)
+#define DDR_4PHASE BIT(2)
+#define DDR_WIDTH (0x3 << 4)
+#define DDR_PAD_MODE (0x1 << 8)
+
+#define DPI_TGEN_HWIDTH 0x20
+#define HPW 0
+#define HPW_MASK (0xFFF << 0)
+
+#define DPI_TGEN_HPORCH 0x24
+#define HBP 0
+#define HBP_MASK (0xFFF << 0)
+#define HFP 16
+#define HFP_MASK (0xFFF << 16)
+
+#define DPI_TGEN_VWIDTH 0x28
+#define DPI_TGEN_VPORCH 0x2C
+
+#define VSYNC_WIDTH_SHIFT 0
+#define VSYNC_WIDTH_MASK (0xFFF << 0)
+#define VSYNC_HALF_LINE_SHIFT 16
+#define VSYNC_HALF_LINE_MASK BIT(16)
+#define VSYNC_BACK_PORCH_SHIFT 0
+#define VSYNC_BACK_PORCH_MASK (0xFFF << 0)
+#define VSYNC_FRONT_PORCH_SHIFT 16
+#define VSYNC_FRONT_PORCH_MASK (0xFFF << 16)
+
+#define DPI_BG_HCNTL 0x30
+#define BG_RIGHT (0x1FFF << 0)
+#define BG_LEFT (0x1FFF << 16)
+
+#define DPI_BG_VCNTL 0x34
+#define BG_BOT (0x1FFF << 0)
+#define BG_TOP (0x1FFF << 16)
+
+#define DPI_BG_COLOR 0x38
+#define BG_B (0xF << 0)
+#define BG_G (0xF << 8)
+#define BG_R (0xF << 16)
+
+#define DPI_FIFO_CTL 0x3C
+#define FIFO_VALID_SET (0x1F << 0)
+#define FIFO_RST_SEL (0x1 << 8)
+
+#define DPI_STATUS 0x40
+#define VCOUNTER (0x1FFF << 0)
+#define DPI_BUSY BIT(16)
+#define OUTEN BIT(17)
+#define FIELD BIT(20)
+#define TDLR BIT(21)
+
+#define DPI_TMODE 0x44
+#define DPI_OEN_ON BIT(0)
+
+#define DPI_CHECKSUM 0x48
+#define DPI_CHECKSUM_MASK (0xFFFFFF << 0)
+#define DPI_CHECKSUM_READY BIT(30)
+#define DPI_CHECKSUM_EN BIT(31)
+
+#define DPI_DUMMY 0x50
+#define DPI_DUMMY_MASK (0xFFFFFFFF << 0)
+
+#define DPI_TGEN_VWIDTH_LEVEN 0x68
+#define DPI_TGEN_VPORCH_LEVEN 0x6C
+#define DPI_TGEN_VWIDTH_RODD 0x70
+#define DPI_TGEN_VPORCH_RODD 0x74
+#define DPI_TGEN_VWIDTH_REVEN 0x78
+#define DPI_TGEN_VPORCH_REVEN 0x7C
+
+#define DPI_ESAV_VTIMING_LODD 0x80
+#define ESAV_VOFST_LODD (0xFFF << 0)
+#define ESAV_VWID_LODD (0xFFF << 16)
+
+#define DPI_ESAV_VTIMING_LEVEN 0x84
+#define ESAV_VOFST_LEVEN (0xFFF << 0)
+#define ESAV_VWID_LEVEN (0xFFF << 16)
+
+#define DPI_ESAV_VTIMING_RODD 0x88
+#define ESAV_VOFST_RODD (0xFFF << 0)
+#define ESAV_VWID_RODD (0xFFF << 16)
+
+#define DPI_ESAV_VTIMING_REVEN 0x8C
+#define ESAV_VOFST_REVEN (0xFFF << 0)
+#define ESAV_VWID_REVEN (0xFFF << 16)
+
+#define DPI_ESAV_FTIMING 0x90
+#define ESAV_FOFST_ODD (0xFFF << 0)
+#define ESAV_FOFST_EVEN (0xFFF << 16)
+
+#define DPI_CLPF_SETTING 0x94
+#define CLPF_TYPE (0x3 << 0)
+#define ROUND_EN BIT(4)
+
+#define DPI_Y_LIMIT 0x98
+#define Y_LIMINT_BOT 0
+#define Y_LIMINT_BOT_MASK (0xFFF << 0)
+#define Y_LIMINT_TOP 16
+#define Y_LIMINT_TOP_MASK (0xFFF << 16)
+
+#define DPI_C_LIMIT 0x9C
+#define C_LIMIT_BOT 0
+#define C_LIMIT_BOT_MASK (0xFFF << 0)
+#define C_LIMIT_TOP 16
+#define C_LIMIT_TOP_MASK (0xFFF << 16)
+
+#define DPI_YUV422_SETTING 0xA0
+#define UV_SWAP BIT(0)
+#define CR_DELSEL BIT(4)
+#define CB_DELSEL BIT(5)
+#define Y_DELSEL BIT(6)
+#define DE_DELSEL BIT(7)
+
+#define DPI_EMBSYNC_SETTING 0xA4
+#define EMBSYNC_R_CR_EN BIT(0)
+#define EMPSYNC_G_Y_EN BIT(1)
+#define EMPSYNC_B_CB_EN BIT(2)
+#define ESAV_F_INV BIT(4)
+#define ESAV_V_INV BIT(5)
+#define ESAV_H_INV BIT(6)
+#define ESAV_CODE_MAN BIT(8)
+#define VS_OUT_SEL (0x7 << 12)
+
+#define DPI_ESAV_CODE_SET0 0xA8
+#define ESAV_CODE0 (0xFFF << 0)
+#define ESAV_CODE1 (0xFFF << 16)
+
+#define DPI_ESAV_CODE_SET1 0xAC
+#define ESAV_CODE2 (0xFFF << 0)
+#define ESAV_CODE3_MSB BIT(16)
+
+#define DPI_H_FRE_CON 0xE0
+#define H_FRE_2N BIT(25)
+#endif /* __MTK_DPI_REGS_H */
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_crtc.c b/drivers/gpu/drm/mediatek/mtk_drm_crtc.c
new file mode 100644
index 0000000..24aa3ba
--- /dev/null
+++ b/drivers/gpu/drm/mediatek/mtk_drm_crtc.c
@@ -0,0 +1,582 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <asm/barrier.h>
+#include <drm/drmP.h>
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_plane_helper.h>
+#include <linux/clk.h>
+#include <linux/pm_runtime.h>
+#include <soc/mediatek/smi.h>
+
+#include "mtk_drm_drv.h"
+#include "mtk_drm_crtc.h"
+#include "mtk_drm_ddp.h"
+#include "mtk_drm_ddp_comp.h"
+#include "mtk_drm_gem.h"
+#include "mtk_drm_plane.h"
+
+/**
+ * struct mtk_drm_crtc - MediaTek specific crtc structure.
+ * @base: crtc object.
+ * @enabled: records whether crtc_enable succeeded
+ * @planes: array of 4 mtk_drm_plane structures, one for each overlay plane
+ * @pending_planes: whether any plane has pending changes to be applied
+ * @config_regs: memory mapped mmsys configuration register space
+ * @mutex: handle to one of the ten disp_mutex streams
+ * @ddp_comp_nr: number of components in ddp_comp
+ * @ddp_comp: array of pointers the mtk_ddp_comp structures used by this crtc
+ */
+struct mtk_drm_crtc {
+ struct drm_crtc base;
+ bool enabled;
+
+ bool pending_needs_vblank;
+ struct drm_pending_vblank_event *event;
+
+ struct mtk_drm_plane planes[OVL_LAYER_NR];
+ bool pending_planes;
+
+ void __iomem *config_regs;
+ struct mtk_disp_mutex *mutex;
+ unsigned int ddp_comp_nr;
+ struct mtk_ddp_comp **ddp_comp;
+};
+
+struct mtk_crtc_state {
+ struct drm_crtc_state base;
+
+ bool pending_config;
+ unsigned int pending_width;
+ unsigned int pending_height;
+ unsigned int pending_vrefresh;
+};
+
+static inline struct mtk_drm_crtc *to_mtk_crtc(struct drm_crtc *c)
+{
+ return container_of(c, struct mtk_drm_crtc, base);
+}
+
+static inline struct mtk_crtc_state *to_mtk_crtc_state(struct drm_crtc_state *s)
+{
+ return container_of(s, struct mtk_crtc_state, base);
+}
+
+static void mtk_drm_crtc_finish_page_flip(struct mtk_drm_crtc *mtk_crtc)
+{
+ struct drm_crtc *crtc = &mtk_crtc->base;
+ unsigned long flags;
+
+ spin_lock_irqsave(&crtc->dev->event_lock, flags);
+ drm_crtc_send_vblank_event(crtc, mtk_crtc->event);
+ drm_crtc_vblank_put(crtc);
+ mtk_crtc->event = NULL;
+ spin_unlock_irqrestore(&crtc->dev->event_lock, flags);
+}
+
+static void mtk_drm_finish_page_flip(struct mtk_drm_crtc *mtk_crtc)
+{
+ drm_crtc_handle_vblank(&mtk_crtc->base);
+ if (mtk_crtc->pending_needs_vblank) {
+ mtk_drm_crtc_finish_page_flip(mtk_crtc);
+ mtk_crtc->pending_needs_vblank = false;
+ }
+}
+
+static void mtk_drm_crtc_destroy(struct drm_crtc *crtc)
+{
+ struct mtk_drm_crtc *mtk_crtc = to_mtk_crtc(crtc);
+ int i;
+
+ for (i = 0; i < mtk_crtc->ddp_comp_nr; i++)
+ clk_unprepare(mtk_crtc->ddp_comp[i]->clk);
+
+ mtk_disp_mutex_put(mtk_crtc->mutex);
+
+ drm_crtc_cleanup(crtc);
+}
+
+static void mtk_drm_crtc_reset(struct drm_crtc *crtc)
+{
+ struct mtk_crtc_state *state;
+
+ if (crtc->state) {
+ if (crtc->state->mode_blob)
+ drm_property_unreference_blob(crtc->state->mode_blob);
+
+ state = to_mtk_crtc_state(crtc->state);
+ memset(state, 0, sizeof(*state));
+ } else {
+ state = kzalloc(sizeof(*state), GFP_KERNEL);
+ if (!state)
+ return;
+ crtc->state = &state->base;
+ }
+
+ state->base.crtc = crtc;
+}
+
+static struct drm_crtc_state *mtk_drm_crtc_duplicate_state(struct drm_crtc *crtc)
+{
+ struct mtk_crtc_state *state;
+
+ state = kzalloc(sizeof(*state), GFP_KERNEL);
+ if (!state)
+ return NULL;
+
+ __drm_atomic_helper_crtc_duplicate_state(crtc, &state->base);
+
+ WARN_ON(state->base.crtc != crtc);
+ state->base.crtc = crtc;
+
+ return &state->base;
+}
+
+static void mtk_drm_crtc_destroy_state(struct drm_crtc *crtc,
+ struct drm_crtc_state *state)
+{
+ __drm_atomic_helper_crtc_destroy_state(state);
+ kfree(to_mtk_crtc_state(state));
+}
+
+static bool mtk_drm_crtc_mode_fixup(struct drm_crtc *crtc,
+ const struct drm_display_mode *mode,
+ struct drm_display_mode *adjusted_mode)
+{
+ /* Nothing to do here, but this callback is mandatory. */
+ return true;
+}
+
+static void mtk_drm_crtc_mode_set_nofb(struct drm_crtc *crtc)
+{
+ struct mtk_crtc_state *state = to_mtk_crtc_state(crtc->state);
+
+ state->pending_width = crtc->mode.hdisplay;
+ state->pending_height = crtc->mode.vdisplay;
+ state->pending_vrefresh = crtc->mode.vrefresh;
+ wmb(); /* Make sure the above parameters are set before update */
+ state->pending_config = true;
+}
+
+int mtk_drm_crtc_enable_vblank(struct drm_device *drm, unsigned int pipe)
+{
+ struct mtk_drm_private *priv = drm->dev_private;
+ struct mtk_drm_crtc *mtk_crtc = to_mtk_crtc(priv->crtc[pipe]);
+ struct mtk_ddp_comp *ovl = mtk_crtc->ddp_comp[0];
+
+ mtk_ddp_comp_enable_vblank(ovl, &mtk_crtc->base);
+
+ return 0;
+}
+
+void mtk_drm_crtc_disable_vblank(struct drm_device *drm, unsigned int pipe)
+{
+ struct mtk_drm_private *priv = drm->dev_private;
+ struct mtk_drm_crtc *mtk_crtc = to_mtk_crtc(priv->crtc[pipe]);
+ struct mtk_ddp_comp *ovl = mtk_crtc->ddp_comp[0];
+
+ mtk_ddp_comp_disable_vblank(ovl);
+}
+
+static int mtk_crtc_ddp_clk_enable(struct mtk_drm_crtc *mtk_crtc)
+{
+ int ret;
+ int i;
+
+ DRM_DEBUG_DRIVER("%s\n", __func__);
+ for (i = 0; i < mtk_crtc->ddp_comp_nr; i++) {
+ ret = clk_enable(mtk_crtc->ddp_comp[i]->clk);
+ if (ret) {
+ DRM_ERROR("Failed to enable clock %d: %d\n", i, ret);
+ goto err;
+ }
+ }
+
+ return 0;
+err:
+ while (--i >= 0)
+ clk_disable(mtk_crtc->ddp_comp[i]->clk);
+ return ret;
+}
+
+static void mtk_crtc_ddp_clk_disable(struct mtk_drm_crtc *mtk_crtc)
+{
+ int i;
+
+ DRM_DEBUG_DRIVER("%s\n", __func__);
+ for (i = 0; i < mtk_crtc->ddp_comp_nr; i++)
+ clk_disable(mtk_crtc->ddp_comp[i]->clk);
+}
+
+static int mtk_crtc_ddp_hw_init(struct mtk_drm_crtc *mtk_crtc)
+{
+ struct drm_crtc *crtc = &mtk_crtc->base;
+ unsigned int width, height, vrefresh;
+ int ret;
+ int i;
+
+ DRM_DEBUG_DRIVER("%s\n", __func__);
+ if (WARN_ON(!crtc->state))
+ return -EINVAL;
+
+ width = crtc->state->adjusted_mode.hdisplay;
+ height = crtc->state->adjusted_mode.vdisplay;
+ vrefresh = crtc->state->adjusted_mode.vrefresh;
+
+ ret = pm_runtime_get_sync(crtc->dev->dev);
+ if (ret < 0) {
+ DRM_ERROR("Failed to enable power domain: %d\n", ret);
+ return ret;
+ }
+
+ ret = mtk_disp_mutex_prepare(mtk_crtc->mutex);
+ if (ret < 0) {
+ DRM_ERROR("Failed to enable mutex clock: %d\n", ret);
+ goto err_pm_runtime_put;
+ }
+
+ ret = mtk_crtc_ddp_clk_enable(mtk_crtc);
+ if (ret < 0) {
+ DRM_ERROR("Failed to enable component clocks: %d\n", ret);
+ goto err_mutex_unprepare;
+ }
+
+ DRM_DEBUG_DRIVER("mediatek_ddp_ddp_path_setup\n");
+ for (i = 0; i < mtk_crtc->ddp_comp_nr - 1; i++) {
+ mtk_ddp_add_comp_to_path(mtk_crtc->config_regs,
+ mtk_crtc->ddp_comp[i]->id,
+ mtk_crtc->ddp_comp[i + 1]->id);
+ mtk_disp_mutex_add_comp(mtk_crtc->mutex,
+ mtk_crtc->ddp_comp[i]->id);
+ }
+ mtk_disp_mutex_add_comp(mtk_crtc->mutex, mtk_crtc->ddp_comp[i]->id);
+ mtk_disp_mutex_enable(mtk_crtc->mutex);
+
+ for (i = 0; i < mtk_crtc->ddp_comp_nr; i++) {
+ struct mtk_ddp_comp *comp = mtk_crtc->ddp_comp[i];
+
+ mtk_ddp_comp_config(comp, width, height, vrefresh);
+ mtk_ddp_comp_start(comp);
+ }
+
+ /* Initially configure all planes */
+ for (i = 0; i < OVL_LAYER_NR; i++) {
+ struct drm_plane *plane = &mtk_crtc->planes[i].base;
+ struct mtk_plane_state *plane_state;
+
+ plane_state = to_mtk_plane_state(plane->state);
+ mtk_ddp_comp_layer_config(mtk_crtc->ddp_comp[0], i,
+ plane_state);
+ }
+
+ return 0;
+
+err_mutex_unprepare:
+ mtk_disp_mutex_unprepare(mtk_crtc->mutex);
+err_pm_runtime_put:
+ pm_runtime_put(crtc->dev->dev);
+ return ret;
+}
+
+static void mtk_crtc_ddp_hw_fini(struct mtk_drm_crtc *mtk_crtc)
+{
+ struct drm_device *drm = mtk_crtc->base.dev;
+ int i;
+
+ DRM_DEBUG_DRIVER("%s\n", __func__);
+ for (i = 0; i < mtk_crtc->ddp_comp_nr; i++)
+ mtk_ddp_comp_stop(mtk_crtc->ddp_comp[i]);
+ for (i = 0; i < mtk_crtc->ddp_comp_nr; i++)
+ mtk_disp_mutex_remove_comp(mtk_crtc->mutex,
+ mtk_crtc->ddp_comp[i]->id);
+ mtk_disp_mutex_disable(mtk_crtc->mutex);
+ for (i = 0; i < mtk_crtc->ddp_comp_nr - 1; i++) {
+ mtk_ddp_remove_comp_from_path(mtk_crtc->config_regs,
+ mtk_crtc->ddp_comp[i]->id,
+ mtk_crtc->ddp_comp[i + 1]->id);
+ mtk_disp_mutex_remove_comp(mtk_crtc->mutex,
+ mtk_crtc->ddp_comp[i]->id);
+ }
+ mtk_disp_mutex_remove_comp(mtk_crtc->mutex, mtk_crtc->ddp_comp[i]->id);
+ mtk_crtc_ddp_clk_disable(mtk_crtc);
+ mtk_disp_mutex_unprepare(mtk_crtc->mutex);
+
+ pm_runtime_put(drm->dev);
+}
+
+static void mtk_drm_crtc_enable(struct drm_crtc *crtc)
+{
+ struct mtk_drm_crtc *mtk_crtc = to_mtk_crtc(crtc);
+ struct mtk_ddp_comp *ovl = mtk_crtc->ddp_comp[0];
+ int ret;
+
+ DRM_DEBUG_DRIVER("%s %d\n", __func__, crtc->base.id);
+
+ ret = mtk_smi_larb_get(ovl->larb_dev);
+ if (ret) {
+ DRM_ERROR("Failed to get larb: %d\n", ret);
+ return;
+ }
+
+ ret = mtk_crtc_ddp_hw_init(mtk_crtc);
+ if (ret) {
+ mtk_smi_larb_put(ovl->larb_dev);
+ return;
+ }
+
+ drm_crtc_vblank_on(crtc);
+ mtk_crtc->enabled = true;
+}
+
+static void mtk_drm_crtc_disable(struct drm_crtc *crtc)
+{
+ struct mtk_drm_crtc *mtk_crtc = to_mtk_crtc(crtc);
+ struct mtk_ddp_comp *ovl = mtk_crtc->ddp_comp[0];
+ int i;
+
+ DRM_DEBUG_DRIVER("%s %d\n", __func__, crtc->base.id);
+ if (!mtk_crtc->enabled)
+ return;
+
+ /* Set all pending plane state to disabled */
+ for (i = 0; i < OVL_LAYER_NR; i++) {
+ struct drm_plane *plane = &mtk_crtc->planes[i].base;
+ struct mtk_plane_state *plane_state;
+
+ plane_state = to_mtk_plane_state(plane->state);
+ plane_state->pending.enable = false;
+ plane_state->pending.config = true;
+ }
+ mtk_crtc->pending_planes = true;
+
+ /* Wait for planes to be disabled */
+ drm_crtc_wait_one_vblank(crtc);
+
+ drm_crtc_vblank_off(crtc);
+ mtk_crtc_ddp_hw_fini(mtk_crtc);
+ mtk_smi_larb_put(ovl->larb_dev);
+
+ mtk_crtc->enabled = false;
+}
+
+static void mtk_drm_crtc_atomic_begin(struct drm_crtc *crtc,
+ struct drm_crtc_state *old_crtc_state)
+{
+ struct mtk_crtc_state *state = to_mtk_crtc_state(crtc->state);
+ struct mtk_drm_crtc *mtk_crtc = to_mtk_crtc(crtc);
+
+ if (mtk_crtc->event && state->base.event)
+ DRM_ERROR("new event while there is still a pending event\n");
+
+ if (state->base.event) {
+ state->base.event->pipe = drm_crtc_index(crtc);
+ WARN_ON(drm_crtc_vblank_get(crtc) != 0);
+ mtk_crtc->event = state->base.event;
+ state->base.event = NULL;
+ }
+}
+
+static void mtk_drm_crtc_atomic_flush(struct drm_crtc *crtc,
+ struct drm_crtc_state *old_crtc_state)
+{
+ struct mtk_drm_crtc *mtk_crtc = to_mtk_crtc(crtc);
+ unsigned int pending_planes = 0;
+ int i;
+
+ if (mtk_crtc->event)
+ mtk_crtc->pending_needs_vblank = true;
+ for (i = 0; i < OVL_LAYER_NR; i++) {
+ struct drm_plane *plane = &mtk_crtc->planes[i].base;
+ struct mtk_plane_state *plane_state;
+
+ plane_state = to_mtk_plane_state(plane->state);
+ if (plane_state->pending.dirty) {
+ plane_state->pending.config = true;
+ plane_state->pending.dirty = false;
+ pending_planes |= BIT(i);
+ }
+ }
+ if (pending_planes)
+ mtk_crtc->pending_planes = true;
+}
+
+static const struct drm_crtc_funcs mtk_crtc_funcs = {
+ .set_config = drm_atomic_helper_set_config,
+ .page_flip = drm_atomic_helper_page_flip,
+ .destroy = mtk_drm_crtc_destroy,
+ .reset = mtk_drm_crtc_reset,
+ .atomic_duplicate_state = mtk_drm_crtc_duplicate_state,
+ .atomic_destroy_state = mtk_drm_crtc_destroy_state,
+};
+
+static const struct drm_crtc_helper_funcs mtk_crtc_helper_funcs = {
+ .mode_fixup = mtk_drm_crtc_mode_fixup,
+ .mode_set_nofb = mtk_drm_crtc_mode_set_nofb,
+ .enable = mtk_drm_crtc_enable,
+ .disable = mtk_drm_crtc_disable,
+ .atomic_begin = mtk_drm_crtc_atomic_begin,
+ .atomic_flush = mtk_drm_crtc_atomic_flush,
+};
+
+static int mtk_drm_crtc_init(struct drm_device *drm,
+ struct mtk_drm_crtc *mtk_crtc,
+ struct drm_plane *primary,
+ struct drm_plane *cursor, unsigned int pipe)
+{
+ int ret;
+
+ ret = drm_crtc_init_with_planes(drm, &mtk_crtc->base, primary, cursor,
+ &mtk_crtc_funcs, NULL);
+ if (ret)
+ goto err_cleanup_crtc;
+
+ drm_crtc_helper_add(&mtk_crtc->base, &mtk_crtc_helper_funcs);
+
+ return 0;
+
+err_cleanup_crtc:
+ drm_crtc_cleanup(&mtk_crtc->base);
+ return ret;
+}
+
+void mtk_crtc_ddp_irq(struct drm_crtc *crtc, struct mtk_ddp_comp *ovl)
+{
+ struct mtk_drm_crtc *mtk_crtc = to_mtk_crtc(crtc);
+ struct mtk_crtc_state *state = to_mtk_crtc_state(mtk_crtc->base.state);
+ unsigned int i;
+
+ /*
+ * TODO: instead of updating the registers here, we should prepare
+ * working registers in atomic_commit and let the hardware command
+ * queue update module registers on vblank.
+ */
+ if (state->pending_config) {
+ mtk_ddp_comp_config(ovl, state->pending_width,
+ state->pending_height,
+ state->pending_vrefresh);
+
+ state->pending_config = false;
+ }
+
+ if (mtk_crtc->pending_planes) {
+ for (i = 0; i < OVL_LAYER_NR; i++) {
+ struct drm_plane *plane = &mtk_crtc->planes[i].base;
+ struct mtk_plane_state *plane_state;
+
+ plane_state = to_mtk_plane_state(plane->state);
+
+ if (plane_state->pending.config) {
+ mtk_ddp_comp_layer_config(ovl, i, plane_state);
+ plane_state->pending.config = false;
+ }
+ }
+ mtk_crtc->pending_planes = false;
+ }
+
+ mtk_drm_finish_page_flip(mtk_crtc);
+}
+
+int mtk_drm_crtc_create(struct drm_device *drm_dev,
+ const enum mtk_ddp_comp_id *path, unsigned int path_len)
+{
+ struct mtk_drm_private *priv = drm_dev->dev_private;
+ struct device *dev = drm_dev->dev;
+ struct mtk_drm_crtc *mtk_crtc;
+ enum drm_plane_type type;
+ unsigned int zpos;
+ int pipe = priv->num_pipes;
+ int ret;
+ int i;
+
+ for (i = 0; i < path_len; i++) {
+ enum mtk_ddp_comp_id comp_id = path[i];
+ struct device_node *node;
+
+ node = priv->comp_node[comp_id];
+ if (!node) {
+ dev_info(dev,
+ "Not creating crtc %d because component %d is disabled or missing\n",
+ pipe, comp_id);
+ return 0;
+ }
+ }
+
+ mtk_crtc = devm_kzalloc(dev, sizeof(*mtk_crtc), GFP_KERNEL);
+ if (!mtk_crtc)
+ return -ENOMEM;
+
+ mtk_crtc->config_regs = priv->config_regs;
+ mtk_crtc->ddp_comp_nr = path_len;
+ mtk_crtc->ddp_comp = devm_kmalloc_array(dev, mtk_crtc->ddp_comp_nr,
+ sizeof(*mtk_crtc->ddp_comp),
+ GFP_KERNEL);
+
+ mtk_crtc->mutex = mtk_disp_mutex_get(priv->mutex_dev, pipe);
+ if (IS_ERR(mtk_crtc->mutex)) {
+ ret = PTR_ERR(mtk_crtc->mutex);
+ dev_err(dev, "Failed to get mutex: %d\n", ret);
+ return ret;
+ }
+
+ for (i = 0; i < mtk_crtc->ddp_comp_nr; i++) {
+ enum mtk_ddp_comp_id comp_id = path[i];
+ struct mtk_ddp_comp *comp;
+ struct device_node *node;
+
+ node = priv->comp_node[comp_id];
+ comp = priv->ddp_comp[comp_id];
+ if (!comp) {
+ dev_err(dev, "Component %s not initialized\n",
+ node->full_name);
+ ret = -ENODEV;
+ goto unprepare;
+ }
+
+ ret = clk_prepare(comp->clk);
+ if (ret) {
+ dev_err(dev,
+ "Failed to prepare clock for component %s: %d\n",
+ node->full_name, ret);
+ goto unprepare;
+ }
+
+ mtk_crtc->ddp_comp[i] = comp;
+ }
+
+ for (zpos = 0; zpos < OVL_LAYER_NR; zpos++) {
+ type = (zpos == 0) ? DRM_PLANE_TYPE_PRIMARY :
+ (zpos == 1) ? DRM_PLANE_TYPE_CURSOR :
+ DRM_PLANE_TYPE_OVERLAY;
+ ret = mtk_plane_init(drm_dev, &mtk_crtc->planes[zpos],
+ BIT(pipe), type, zpos);
+ if (ret)
+ goto unprepare;
+ }
+
+ ret = mtk_drm_crtc_init(drm_dev, mtk_crtc, &mtk_crtc->planes[0].base,
+ &mtk_crtc->planes[1].base, pipe);
+ if (ret < 0)
+ goto unprepare;
+
+ priv->crtc[pipe] = &mtk_crtc->base;
+ priv->num_pipes++;
+
+ return 0;
+
+unprepare:
+ while (--i >= 0)
+ clk_unprepare(mtk_crtc->ddp_comp[i]->clk);
+
+ return ret;
+}
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_crtc.h b/drivers/gpu/drm/mediatek/mtk_drm_crtc.h
new file mode 100644
index 0000000..81e5566
--- /dev/null
+++ b/drivers/gpu/drm/mediatek/mtk_drm_crtc.h
@@ -0,0 +1,32 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef MTK_DRM_CRTC_H
+#define MTK_DRM_CRTC_H
+
+#include <drm/drm_crtc.h>
+#include "mtk_drm_ddp_comp.h"
+#include "mtk_drm_plane.h"
+
+#define OVL_LAYER_NR 4
+
+int mtk_drm_crtc_enable_vblank(struct drm_device *drm, unsigned int pipe);
+void mtk_drm_crtc_disable_vblank(struct drm_device *drm, unsigned int pipe);
+void mtk_drm_crtc_check_flush(struct drm_crtc *crtc);
+void mtk_drm_crtc_commit(struct drm_crtc *crtc);
+void mtk_crtc_ddp_irq(struct drm_crtc *crtc, struct mtk_ddp_comp *ovl);
+int mtk_drm_crtc_create(struct drm_device *drm_dev,
+ const enum mtk_ddp_comp_id *path,
+ unsigned int path_len);
+
+#endif /* MTK_DRM_CRTC_H */
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_ddp.c b/drivers/gpu/drm/mediatek/mtk_drm_ddp.c
new file mode 100644
index 0000000..17ba935
--- /dev/null
+++ b/drivers/gpu/drm/mediatek/mtk_drm_ddp.c
@@ -0,0 +1,353 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/clk.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+
+#include "mtk_drm_ddp.h"
+#include "mtk_drm_ddp_comp.h"
+
+#define DISP_REG_CONFIG_DISP_OVL0_MOUT_EN 0x040
+#define DISP_REG_CONFIG_DISP_OVL1_MOUT_EN 0x044
+#define DISP_REG_CONFIG_DISP_OD_MOUT_EN 0x048
+#define DISP_REG_CONFIG_DISP_GAMMA_MOUT_EN 0x04c
+#define DISP_REG_CONFIG_DISP_UFOE_MOUT_EN 0x050
+#define DISP_REG_CONFIG_DISP_COLOR0_SEL_IN 0x084
+#define DISP_REG_CONFIG_DISP_COLOR1_SEL_IN 0x088
+#define DISP_REG_CONFIG_DPI_SEL_IN 0x0ac
+#define DISP_REG_CONFIG_DISP_RDMA1_MOUT_EN 0x0c8
+#define DISP_REG_CONFIG_MMSYS_CG_CON0 0x100
+
+#define DISP_REG_MUTEX_EN(n) (0x20 + 0x20 * (n))
+#define DISP_REG_MUTEX_RST(n) (0x28 + 0x20 * (n))
+#define DISP_REG_MUTEX_MOD(n) (0x2c + 0x20 * (n))
+#define DISP_REG_MUTEX_SOF(n) (0x30 + 0x20 * (n))
+
+#define MUTEX_MOD_DISP_OVL0 BIT(11)
+#define MUTEX_MOD_DISP_OVL1 BIT(12)
+#define MUTEX_MOD_DISP_RDMA0 BIT(13)
+#define MUTEX_MOD_DISP_RDMA1 BIT(14)
+#define MUTEX_MOD_DISP_RDMA2 BIT(15)
+#define MUTEX_MOD_DISP_WDMA0 BIT(16)
+#define MUTEX_MOD_DISP_WDMA1 BIT(17)
+#define MUTEX_MOD_DISP_COLOR0 BIT(18)
+#define MUTEX_MOD_DISP_COLOR1 BIT(19)
+#define MUTEX_MOD_DISP_AAL BIT(20)
+#define MUTEX_MOD_DISP_GAMMA BIT(21)
+#define MUTEX_MOD_DISP_UFOE BIT(22)
+#define MUTEX_MOD_DISP_PWM0 BIT(23)
+#define MUTEX_MOD_DISP_PWM1 BIT(24)
+#define MUTEX_MOD_DISP_OD BIT(25)
+
+#define MUTEX_SOF_SINGLE_MODE 0
+#define MUTEX_SOF_DSI0 1
+#define MUTEX_SOF_DSI1 2
+#define MUTEX_SOF_DPI0 3
+
+#define OVL0_MOUT_EN_COLOR0 0x1
+#define OD_MOUT_EN_RDMA0 0x1
+#define UFOE_MOUT_EN_DSI0 0x1
+#define COLOR0_SEL_IN_OVL0 0x1
+#define OVL1_MOUT_EN_COLOR1 0x1
+#define GAMMA_MOUT_EN_RDMA1 0x1
+#define RDMA1_MOUT_DPI0 0x2
+#define DPI0_SEL_IN_RDMA1 0x1
+#define COLOR1_SEL_IN_OVL1 0x1
+
+struct mtk_disp_mutex {
+ int id;
+ bool claimed;
+};
+
+struct mtk_ddp {
+ struct device *dev;
+ struct clk *clk;
+ void __iomem *regs;
+ struct mtk_disp_mutex mutex[10];
+};
+
+static const unsigned int mutex_mod[DDP_COMPONENT_ID_MAX] = {
+ [DDP_COMPONENT_AAL] = MUTEX_MOD_DISP_AAL,
+ [DDP_COMPONENT_COLOR0] = MUTEX_MOD_DISP_COLOR0,
+ [DDP_COMPONENT_COLOR1] = MUTEX_MOD_DISP_COLOR1,
+ [DDP_COMPONENT_GAMMA] = MUTEX_MOD_DISP_GAMMA,
+ [DDP_COMPONENT_OD] = MUTEX_MOD_DISP_OD,
+ [DDP_COMPONENT_OVL0] = MUTEX_MOD_DISP_OVL0,
+ [DDP_COMPONENT_OVL1] = MUTEX_MOD_DISP_OVL1,
+ [DDP_COMPONENT_PWM0] = MUTEX_MOD_DISP_PWM0,
+ [DDP_COMPONENT_PWM1] = MUTEX_MOD_DISP_PWM1,
+ [DDP_COMPONENT_RDMA0] = MUTEX_MOD_DISP_RDMA0,
+ [DDP_COMPONENT_RDMA1] = MUTEX_MOD_DISP_RDMA1,
+ [DDP_COMPONENT_RDMA2] = MUTEX_MOD_DISP_RDMA2,
+ [DDP_COMPONENT_UFOE] = MUTEX_MOD_DISP_UFOE,
+ [DDP_COMPONENT_WDMA0] = MUTEX_MOD_DISP_WDMA0,
+ [DDP_COMPONENT_WDMA1] = MUTEX_MOD_DISP_WDMA1,
+};
+
+static unsigned int mtk_ddp_mout_en(enum mtk_ddp_comp_id cur,
+ enum mtk_ddp_comp_id next,
+ unsigned int *addr)
+{
+ unsigned int value;
+
+ if (cur == DDP_COMPONENT_OVL0 && next == DDP_COMPONENT_COLOR0) {
+ *addr = DISP_REG_CONFIG_DISP_OVL0_MOUT_EN;
+ value = OVL0_MOUT_EN_COLOR0;
+ } else if (cur == DDP_COMPONENT_OD && next == DDP_COMPONENT_RDMA0) {
+ *addr = DISP_REG_CONFIG_DISP_OD_MOUT_EN;
+ value = OD_MOUT_EN_RDMA0;
+ } else if (cur == DDP_COMPONENT_UFOE && next == DDP_COMPONENT_DSI0) {
+ *addr = DISP_REG_CONFIG_DISP_UFOE_MOUT_EN;
+ value = UFOE_MOUT_EN_DSI0;
+ } else if (cur == DDP_COMPONENT_OVL1 && next == DDP_COMPONENT_COLOR1) {
+ *addr = DISP_REG_CONFIG_DISP_OVL1_MOUT_EN;
+ value = OVL1_MOUT_EN_COLOR1;
+ } else if (cur == DDP_COMPONENT_GAMMA && next == DDP_COMPONENT_RDMA1) {
+ *addr = DISP_REG_CONFIG_DISP_GAMMA_MOUT_EN;
+ value = GAMMA_MOUT_EN_RDMA1;
+ } else if (cur == DDP_COMPONENT_RDMA1 && next == DDP_COMPONENT_DPI0) {
+ *addr = DISP_REG_CONFIG_DISP_RDMA1_MOUT_EN;
+ value = RDMA1_MOUT_DPI0;
+ } else {
+ value = 0;
+ }
+
+ return value;
+}
+
+static unsigned int mtk_ddp_sel_in(enum mtk_ddp_comp_id cur,
+ enum mtk_ddp_comp_id next,
+ unsigned int *addr)
+{
+ unsigned int value;
+
+ if (cur == DDP_COMPONENT_OVL0 && next == DDP_COMPONENT_COLOR0) {
+ *addr = DISP_REG_CONFIG_DISP_COLOR0_SEL_IN;
+ value = COLOR0_SEL_IN_OVL0;
+ } else if (cur == DDP_COMPONENT_RDMA1 && next == DDP_COMPONENT_DPI0) {
+ *addr = DISP_REG_CONFIG_DPI_SEL_IN;
+ value = DPI0_SEL_IN_RDMA1;
+ } else if (cur == DDP_COMPONENT_OVL1 && next == DDP_COMPONENT_COLOR1) {
+ *addr = DISP_REG_CONFIG_DISP_COLOR1_SEL_IN;
+ value = COLOR1_SEL_IN_OVL1;
+ } else {
+ value = 0;
+ }
+
+ return value;
+}
+
+void mtk_ddp_add_comp_to_path(void __iomem *config_regs,
+ enum mtk_ddp_comp_id cur,
+ enum mtk_ddp_comp_id next)
+{
+ unsigned int addr, value, reg;
+
+ value = mtk_ddp_mout_en(cur, next, &addr);
+ if (value) {
+ reg = readl_relaxed(config_regs + addr) | value;
+ writel_relaxed(reg, config_regs + addr);
+ }
+
+ value = mtk_ddp_sel_in(cur, next, &addr);
+ if (value) {
+ reg = readl_relaxed(config_regs + addr) | value;
+ writel_relaxed(reg, config_regs + addr);
+ }
+}
+
+void mtk_ddp_remove_comp_from_path(void __iomem *config_regs,
+ enum mtk_ddp_comp_id cur,
+ enum mtk_ddp_comp_id next)
+{
+ unsigned int addr, value, reg;
+
+ value = mtk_ddp_mout_en(cur, next, &addr);
+ if (value) {
+ reg = readl_relaxed(config_regs + addr) & ~value;
+ writel_relaxed(reg, config_regs + addr);
+ }
+
+ value = mtk_ddp_sel_in(cur, next, &addr);
+ if (value) {
+ reg = readl_relaxed(config_regs + addr) & ~value;
+ writel_relaxed(reg, config_regs + addr);
+ }
+}
+
+struct mtk_disp_mutex *mtk_disp_mutex_get(struct device *dev, unsigned int id)
+{
+ struct mtk_ddp *ddp = dev_get_drvdata(dev);
+
+ if (id >= 10)
+ return ERR_PTR(-EINVAL);
+ if (ddp->mutex[id].claimed)
+ return ERR_PTR(-EBUSY);
+
+ ddp->mutex[id].claimed = true;
+
+ return &ddp->mutex[id];
+}
+
+void mtk_disp_mutex_put(struct mtk_disp_mutex *mutex)
+{
+ struct mtk_ddp *ddp = container_of(mutex, struct mtk_ddp,
+ mutex[mutex->id]);
+
+ WARN_ON(&ddp->mutex[mutex->id] != mutex);
+
+ mutex->claimed = false;
+}
+
+int mtk_disp_mutex_prepare(struct mtk_disp_mutex *mutex)
+{
+ struct mtk_ddp *ddp = container_of(mutex, struct mtk_ddp,
+ mutex[mutex->id]);
+ return clk_prepare_enable(ddp->clk);
+}
+
+void mtk_disp_mutex_unprepare(struct mtk_disp_mutex *mutex)
+{
+ struct mtk_ddp *ddp = container_of(mutex, struct mtk_ddp,
+ mutex[mutex->id]);
+ clk_disable_unprepare(ddp->clk);
+}
+
+void mtk_disp_mutex_add_comp(struct mtk_disp_mutex *mutex,
+ enum mtk_ddp_comp_id id)
+{
+ struct mtk_ddp *ddp = container_of(mutex, struct mtk_ddp,
+ mutex[mutex->id]);
+ unsigned int reg;
+
+ WARN_ON(&ddp->mutex[mutex->id] != mutex);
+
+ switch (id) {
+ case DDP_COMPONENT_DSI0:
+ reg = MUTEX_SOF_DSI0;
+ break;
+ case DDP_COMPONENT_DSI1:
+ reg = MUTEX_SOF_DSI0;
+ break;
+ case DDP_COMPONENT_DPI0:
+ reg = MUTEX_SOF_DPI0;
+ break;
+ default:
+ reg = readl_relaxed(ddp->regs + DISP_REG_MUTEX_MOD(mutex->id));
+ reg |= mutex_mod[id];
+ writel_relaxed(reg, ddp->regs + DISP_REG_MUTEX_MOD(mutex->id));
+ return;
+ }
+
+ writel_relaxed(reg, ddp->regs + DISP_REG_MUTEX_SOF(mutex->id));
+}
+
+void mtk_disp_mutex_remove_comp(struct mtk_disp_mutex *mutex,
+ enum mtk_ddp_comp_id id)
+{
+ struct mtk_ddp *ddp = container_of(mutex, struct mtk_ddp,
+ mutex[mutex->id]);
+ unsigned int reg;
+
+ WARN_ON(&ddp->mutex[mutex->id] != mutex);
+
+ switch (id) {
+ case DDP_COMPONENT_DSI0:
+ case DDP_COMPONENT_DSI1:
+ case DDP_COMPONENT_DPI0:
+ writel_relaxed(MUTEX_SOF_SINGLE_MODE,
+ ddp->regs + DISP_REG_MUTEX_SOF(mutex->id));
+ break;
+ default:
+ reg = readl_relaxed(ddp->regs + DISP_REG_MUTEX_MOD(mutex->id));
+ reg &= ~mutex_mod[id];
+ writel_relaxed(reg, ddp->regs + DISP_REG_MUTEX_MOD(mutex->id));
+ break;
+ }
+}
+
+void mtk_disp_mutex_enable(struct mtk_disp_mutex *mutex)
+{
+ struct mtk_ddp *ddp = container_of(mutex, struct mtk_ddp,
+ mutex[mutex->id]);
+
+ WARN_ON(&ddp->mutex[mutex->id] != mutex);
+
+ writel(1, ddp->regs + DISP_REG_MUTEX_EN(mutex->id));
+}
+
+void mtk_disp_mutex_disable(struct mtk_disp_mutex *mutex)
+{
+ struct mtk_ddp *ddp = container_of(mutex, struct mtk_ddp,
+ mutex[mutex->id]);
+
+ WARN_ON(&ddp->mutex[mutex->id] != mutex);
+
+ writel(0, ddp->regs + DISP_REG_MUTEX_EN(mutex->id));
+}
+
+static int mtk_ddp_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct mtk_ddp *ddp;
+ struct resource *regs;
+ int i;
+
+ ddp = devm_kzalloc(dev, sizeof(*ddp), GFP_KERNEL);
+ if (!ddp)
+ return -ENOMEM;
+
+ for (i = 0; i < 10; i++)
+ ddp->mutex[i].id = i;
+
+ ddp->clk = devm_clk_get(dev, NULL);
+ if (IS_ERR(ddp->clk)) {
+ dev_err(dev, "Failed to get clock\n");
+ return PTR_ERR(ddp->clk);
+ }
+
+ regs = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ ddp->regs = devm_ioremap_resource(dev, regs);
+ if (IS_ERR(ddp->regs)) {
+ dev_err(dev, "Failed to map mutex registers\n");
+ return PTR_ERR(ddp->regs);
+ }
+
+ platform_set_drvdata(pdev, ddp);
+
+ return 0;
+}
+
+static int mtk_ddp_remove(struct platform_device *pdev)
+{
+ return 0;
+}
+
+static const struct of_device_id ddp_driver_dt_match[] = {
+ { .compatible = "mediatek,mt8173-disp-mutex" },
+ {},
+};
+MODULE_DEVICE_TABLE(of, ddp_driver_dt_match);
+
+struct platform_driver mtk_ddp_driver = {
+ .probe = mtk_ddp_probe,
+ .remove = mtk_ddp_remove,
+ .driver = {
+ .name = "mediatek-ddp",
+ .owner = THIS_MODULE,
+ .of_match_table = ddp_driver_dt_match,
+ },
+};
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_ddp.h b/drivers/gpu/drm/mediatek/mtk_drm_ddp.h
new file mode 100644
index 0000000..92c1175
--- /dev/null
+++ b/drivers/gpu/drm/mediatek/mtk_drm_ddp.h
@@ -0,0 +1,41 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef MTK_DRM_DDP_H
+#define MTK_DRM_DDP_H
+
+#include "mtk_drm_ddp_comp.h"
+
+struct regmap;
+struct device;
+struct mtk_disp_mutex;
+
+void mtk_ddp_add_comp_to_path(void __iomem *config_regs,
+ enum mtk_ddp_comp_id cur,
+ enum mtk_ddp_comp_id next);
+void mtk_ddp_remove_comp_from_path(void __iomem *config_regs,
+ enum mtk_ddp_comp_id cur,
+ enum mtk_ddp_comp_id next);
+
+struct mtk_disp_mutex *mtk_disp_mutex_get(struct device *dev, unsigned int id);
+int mtk_disp_mutex_prepare(struct mtk_disp_mutex *mutex);
+void mtk_disp_mutex_add_comp(struct mtk_disp_mutex *mutex,
+ enum mtk_ddp_comp_id id);
+void mtk_disp_mutex_enable(struct mtk_disp_mutex *mutex);
+void mtk_disp_mutex_disable(struct mtk_disp_mutex *mutex);
+void mtk_disp_mutex_remove_comp(struct mtk_disp_mutex *mutex,
+ enum mtk_ddp_comp_id id);
+void mtk_disp_mutex_unprepare(struct mtk_disp_mutex *mutex);
+void mtk_disp_mutex_put(struct mtk_disp_mutex *mutex);
+
+#endif /* MTK_DRM_DDP_H */
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_ddp_comp.c b/drivers/gpu/drm/mediatek/mtk_drm_ddp_comp.c
new file mode 100644
index 0000000..3970fcf
--- /dev/null
+++ b/drivers/gpu/drm/mediatek/mtk_drm_ddp_comp.c
@@ -0,0 +1,225 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Authors:
+ * YT Shen <yt.shen@mediatek.com>
+ * CK Hu <ck.hu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/clk.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/of_irq.h>
+#include <linux/of_platform.h>
+#include <linux/platform_device.h>
+#include <drm/drmP.h>
+#include "mtk_drm_drv.h"
+#include "mtk_drm_plane.h"
+#include "mtk_drm_ddp_comp.h"
+
+#define DISP_OD_EN 0x0000
+#define DISP_OD_INTEN 0x0008
+#define DISP_OD_INTSTA 0x000c
+#define DISP_OD_CFG 0x0020
+#define DISP_OD_SIZE 0x0030
+
+#define DISP_REG_UFO_START 0x0000
+
+#define DISP_COLOR_CFG_MAIN 0x0400
+#define DISP_COLOR_START 0x0c00
+#define DISP_COLOR_WIDTH 0x0c50
+#define DISP_COLOR_HEIGHT 0x0c54
+
+#define OD_RELAY_MODE BIT(0)
+
+#define UFO_BYPASS BIT(2)
+
+#define COLOR_BYPASS_ALL BIT(7)
+#define COLOR_SEQ_SEL BIT(13)
+
+static void mtk_color_config(struct mtk_ddp_comp *comp, unsigned int w,
+ unsigned int h, unsigned int vrefresh)
+{
+ writel(w, comp->regs + DISP_COLOR_WIDTH);
+ writel(h, comp->regs + DISP_COLOR_HEIGHT);
+}
+
+static void mtk_color_start(struct mtk_ddp_comp *comp)
+{
+ writel(COLOR_BYPASS_ALL | COLOR_SEQ_SEL,
+ comp->regs + DISP_COLOR_CFG_MAIN);
+ writel(0x1, comp->regs + DISP_COLOR_START);
+}
+
+static void mtk_od_config(struct mtk_ddp_comp *comp, unsigned int w,
+ unsigned int h, unsigned int vrefresh)
+{
+ writel(w << 16 | h, comp->regs + DISP_OD_SIZE);
+}
+
+static void mtk_od_start(struct mtk_ddp_comp *comp)
+{
+ writel(OD_RELAY_MODE, comp->regs + DISP_OD_CFG);
+ writel(1, comp->regs + DISP_OD_EN);
+}
+
+static void mtk_ufoe_start(struct mtk_ddp_comp *comp)
+{
+ writel(UFO_BYPASS, comp->regs + DISP_REG_UFO_START);
+}
+
+static const struct mtk_ddp_comp_funcs ddp_color = {
+ .config = mtk_color_config,
+ .start = mtk_color_start,
+};
+
+static const struct mtk_ddp_comp_funcs ddp_od = {
+ .config = mtk_od_config,
+ .start = mtk_od_start,
+};
+
+static const struct mtk_ddp_comp_funcs ddp_ufoe = {
+ .start = mtk_ufoe_start,
+};
+
+static const char * const mtk_ddp_comp_stem[MTK_DDP_COMP_TYPE_MAX] = {
+ [MTK_DISP_OVL] = "ovl",
+ [MTK_DISP_RDMA] = "rdma",
+ [MTK_DISP_WDMA] = "wdma",
+ [MTK_DISP_COLOR] = "color",
+ [MTK_DISP_AAL] = "aal",
+ [MTK_DISP_GAMMA] = "gamma",
+ [MTK_DISP_UFOE] = "ufoe",
+ [MTK_DSI] = "dsi",
+ [MTK_DPI] = "dpi",
+ [MTK_DISP_PWM] = "pwm",
+ [MTK_DISP_MUTEX] = "mutex",
+ [MTK_DISP_OD] = "od",
+};
+
+struct mtk_ddp_comp_match {
+ enum mtk_ddp_comp_type type;
+ int alias_id;
+ const struct mtk_ddp_comp_funcs *funcs;
+};
+
+static const struct mtk_ddp_comp_match mtk_ddp_matches[DDP_COMPONENT_ID_MAX] = {
+ [DDP_COMPONENT_AAL] = { MTK_DISP_AAL, 0, NULL },
+ [DDP_COMPONENT_COLOR0] = { MTK_DISP_COLOR, 0, &ddp_color },
+ [DDP_COMPONENT_COLOR1] = { MTK_DISP_COLOR, 1, &ddp_color },
+ [DDP_COMPONENT_DPI0] = { MTK_DPI, 0, NULL },
+ [DDP_COMPONENT_DSI0] = { MTK_DSI, 0, NULL },
+ [DDP_COMPONENT_DSI1] = { MTK_DSI, 1, NULL },
+ [DDP_COMPONENT_GAMMA] = { MTK_DISP_GAMMA, 0, NULL },
+ [DDP_COMPONENT_OD] = { MTK_DISP_OD, 0, &ddp_od },
+ [DDP_COMPONENT_OVL0] = { MTK_DISP_OVL, 0, NULL },
+ [DDP_COMPONENT_OVL1] = { MTK_DISP_OVL, 1, NULL },
+ [DDP_COMPONENT_PWM0] = { MTK_DISP_PWM, 0, NULL },
+ [DDP_COMPONENT_RDMA0] = { MTK_DISP_RDMA, 0, NULL },
+ [DDP_COMPONENT_RDMA1] = { MTK_DISP_RDMA, 1, NULL },
+ [DDP_COMPONENT_RDMA2] = { MTK_DISP_RDMA, 2, NULL },
+ [DDP_COMPONENT_UFOE] = { MTK_DISP_UFOE, 0, &ddp_ufoe },
+ [DDP_COMPONENT_WDMA0] = { MTK_DISP_WDMA, 0, NULL },
+ [DDP_COMPONENT_WDMA1] = { MTK_DISP_WDMA, 1, NULL },
+};
+
+int mtk_ddp_comp_get_id(struct device_node *node,
+ enum mtk_ddp_comp_type comp_type)
+{
+ int id = of_alias_get_id(node, mtk_ddp_comp_stem[comp_type]);
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(mtk_ddp_matches); i++) {
+ if (comp_type == mtk_ddp_matches[i].type &&
+ (id < 0 || id == mtk_ddp_matches[i].alias_id))
+ return i;
+ }
+
+ return -EINVAL;
+}
+
+int mtk_ddp_comp_init(struct device *dev, struct device_node *node,
+ struct mtk_ddp_comp *comp, enum mtk_ddp_comp_id comp_id,
+ const struct mtk_ddp_comp_funcs *funcs)
+{
+ enum mtk_ddp_comp_type type;
+ struct device_node *larb_node;
+ struct platform_device *larb_pdev;
+
+ if (comp_id < 0 || comp_id >= DDP_COMPONENT_ID_MAX)
+ return -EINVAL;
+
+ comp->id = comp_id;
+ comp->funcs = funcs ?: mtk_ddp_matches[comp_id].funcs;
+
+ if (comp_id == DDP_COMPONENT_DPI0 ||
+ comp_id == DDP_COMPONENT_DSI0 ||
+ comp_id == DDP_COMPONENT_PWM0) {
+ comp->regs = NULL;
+ comp->clk = NULL;
+ comp->irq = 0;
+ return 0;
+ }
+
+ comp->regs = of_iomap(node, 0);
+ comp->irq = of_irq_get(node, 0);
+ comp->clk = of_clk_get(node, 0);
+ if (IS_ERR(comp->clk))
+ comp->clk = NULL;
+
+ type = mtk_ddp_matches[comp_id].type;
+
+ /* Only DMA capable components need the LARB property */
+ comp->larb_dev = NULL;
+ if (type != MTK_DISP_OVL &&
+ type != MTK_DISP_RDMA &&
+ type != MTK_DISP_WDMA)
+ return 0;
+
+ larb_node = of_parse_phandle(node, "mediatek,larb", 0);
+ if (!larb_node) {
+ dev_err(dev,
+ "Missing mediadek,larb phandle in %s node\n",
+ node->full_name);
+ return -EINVAL;
+ }
+
+ larb_pdev = of_find_device_by_node(larb_node);
+ if (!larb_pdev) {
+ dev_warn(dev, "Waiting for larb device %s\n",
+ larb_node->full_name);
+ of_node_put(larb_node);
+ return -EPROBE_DEFER;
+ }
+ of_node_put(larb_node);
+
+ comp->larb_dev = &larb_pdev->dev;
+
+ return 0;
+}
+
+int mtk_ddp_comp_register(struct drm_device *drm, struct mtk_ddp_comp *comp)
+{
+ struct mtk_drm_private *private = drm->dev_private;
+
+ if (private->ddp_comp[comp->id])
+ return -EBUSY;
+
+ private->ddp_comp[comp->id] = comp;
+ return 0;
+}
+
+void mtk_ddp_comp_unregister(struct drm_device *drm, struct mtk_ddp_comp *comp)
+{
+ struct mtk_drm_private *private = drm->dev_private;
+
+ private->ddp_comp[comp->id] = NULL;
+}
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_ddp_comp.h b/drivers/gpu/drm/mediatek/mtk_drm_ddp_comp.h
new file mode 100644
index 0000000..6b13ba9
--- /dev/null
+++ b/drivers/gpu/drm/mediatek/mtk_drm_ddp_comp.h
@@ -0,0 +1,150 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef MTK_DRM_DDP_COMP_H
+#define MTK_DRM_DDP_COMP_H
+
+#include <linux/io.h>
+
+struct device;
+struct device_node;
+struct drm_crtc;
+struct drm_device;
+struct mtk_plane_state;
+
+enum mtk_ddp_comp_type {
+ MTK_DISP_OVL,
+ MTK_DISP_RDMA,
+ MTK_DISP_WDMA,
+ MTK_DISP_COLOR,
+ MTK_DISP_AAL,
+ MTK_DISP_GAMMA,
+ MTK_DISP_UFOE,
+ MTK_DSI,
+ MTK_DPI,
+ MTK_DISP_PWM,
+ MTK_DISP_MUTEX,
+ MTK_DISP_OD,
+ MTK_DDP_COMP_TYPE_MAX,
+};
+
+enum mtk_ddp_comp_id {
+ DDP_COMPONENT_AAL,
+ DDP_COMPONENT_COLOR0,
+ DDP_COMPONENT_COLOR1,
+ DDP_COMPONENT_DPI0,
+ DDP_COMPONENT_DSI0,
+ DDP_COMPONENT_DSI1,
+ DDP_COMPONENT_GAMMA,
+ DDP_COMPONENT_OD,
+ DDP_COMPONENT_OVL0,
+ DDP_COMPONENT_OVL1,
+ DDP_COMPONENT_PWM0,
+ DDP_COMPONENT_PWM1,
+ DDP_COMPONENT_RDMA0,
+ DDP_COMPONENT_RDMA1,
+ DDP_COMPONENT_RDMA2,
+ DDP_COMPONENT_UFOE,
+ DDP_COMPONENT_WDMA0,
+ DDP_COMPONENT_WDMA1,
+ DDP_COMPONENT_ID_MAX,
+};
+
+struct mtk_ddp_comp;
+
+struct mtk_ddp_comp_funcs {
+ void (*config)(struct mtk_ddp_comp *comp, unsigned int w,
+ unsigned int h, unsigned int vrefresh);
+ void (*start)(struct mtk_ddp_comp *comp);
+ void (*stop)(struct mtk_ddp_comp *comp);
+ void (*enable_vblank)(struct mtk_ddp_comp *comp, struct drm_crtc *crtc);
+ void (*disable_vblank)(struct mtk_ddp_comp *comp);
+ void (*layer_on)(struct mtk_ddp_comp *comp, unsigned int idx);
+ void (*layer_off)(struct mtk_ddp_comp *comp, unsigned int idx);
+ void (*layer_config)(struct mtk_ddp_comp *comp, unsigned int idx,
+ struct mtk_plane_state *state);
+};
+
+struct mtk_ddp_comp {
+ struct clk *clk;
+ void __iomem *regs;
+ int irq;
+ struct device *larb_dev;
+ enum mtk_ddp_comp_id id;
+ const struct mtk_ddp_comp_funcs *funcs;
+};
+
+static inline void mtk_ddp_comp_config(struct mtk_ddp_comp *comp,
+ unsigned int w, unsigned int h,
+ unsigned int vrefresh)
+{
+ if (comp->funcs && comp->funcs->config)
+ comp->funcs->config(comp, w, h, vrefresh);
+}
+
+static inline void mtk_ddp_comp_start(struct mtk_ddp_comp *comp)
+{
+ if (comp->funcs && comp->funcs->start)
+ comp->funcs->start(comp);
+}
+
+static inline void mtk_ddp_comp_stop(struct mtk_ddp_comp *comp)
+{
+ if (comp->funcs && comp->funcs->stop)
+ comp->funcs->stop(comp);
+}
+
+static inline void mtk_ddp_comp_enable_vblank(struct mtk_ddp_comp *comp,
+ struct drm_crtc *crtc)
+{
+ if (comp->funcs && comp->funcs->enable_vblank)
+ comp->funcs->enable_vblank(comp, crtc);
+}
+
+static inline void mtk_ddp_comp_disable_vblank(struct mtk_ddp_comp *comp)
+{
+ if (comp->funcs && comp->funcs->disable_vblank)
+ comp->funcs->disable_vblank(comp);
+}
+
+static inline void mtk_ddp_comp_layer_on(struct mtk_ddp_comp *comp,
+ unsigned int idx)
+{
+ if (comp->funcs && comp->funcs->layer_on)
+ comp->funcs->layer_on(comp, idx);
+}
+
+static inline void mtk_ddp_comp_layer_off(struct mtk_ddp_comp *comp,
+ unsigned int idx)
+{
+ if (comp->funcs && comp->funcs->layer_off)
+ comp->funcs->layer_off(comp, idx);
+}
+
+static inline void mtk_ddp_comp_layer_config(struct mtk_ddp_comp *comp,
+ unsigned int idx,
+ struct mtk_plane_state *state)
+{
+ if (comp->funcs && comp->funcs->layer_config)
+ comp->funcs->layer_config(comp, idx, state);
+}
+
+int mtk_ddp_comp_get_id(struct device_node *node,
+ enum mtk_ddp_comp_type comp_type);
+int mtk_ddp_comp_init(struct device *dev, struct device_node *comp_node,
+ struct mtk_ddp_comp *comp, enum mtk_ddp_comp_id comp_id,
+ const struct mtk_ddp_comp_funcs *funcs);
+int mtk_ddp_comp_register(struct drm_device *drm, struct mtk_ddp_comp *comp);
+void mtk_ddp_comp_unregister(struct drm_device *drm, struct mtk_ddp_comp *comp);
+
+#endif /* MTK_DRM_DDP_COMP_H */
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_drv.c b/drivers/gpu/drm/mediatek/mtk_drm_drv.c
new file mode 100644
index 0000000..b1223d5
--- /dev/null
+++ b/drivers/gpu/drm/mediatek/mtk_drm_drv.c
@@ -0,0 +1,567 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: YT SHEN <yt.shen@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <drm/drmP.h>
+#include <drm/drm_atomic.h>
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_gem.h>
+#include <drm/drm_gem_cma_helper.h>
+#include <linux/component.h>
+#include <linux/iommu.h>
+#include <linux/of_address.h>
+#include <linux/of_platform.h>
+#include <linux/pm_runtime.h>
+
+#include "mtk_drm_crtc.h"
+#include "mtk_drm_ddp.h"
+#include "mtk_drm_ddp_comp.h"
+#include "mtk_drm_drv.h"
+#include "mtk_drm_fb.h"
+#include "mtk_drm_gem.h"
+
+#define DRIVER_NAME "mediatek"
+#define DRIVER_DESC "Mediatek SoC DRM"
+#define DRIVER_DATE "20150513"
+#define DRIVER_MAJOR 1
+#define DRIVER_MINOR 0
+
+static void mtk_atomic_schedule(struct mtk_drm_private *private,
+ struct drm_atomic_state *state)
+{
+ private->commit.state = state;
+ schedule_work(&private->commit.work);
+}
+
+static void mtk_atomic_wait_for_fences(struct drm_atomic_state *state)
+{
+ struct drm_plane *plane;
+ struct drm_plane_state *plane_state;
+ int i;
+
+ for_each_plane_in_state(state, plane, plane_state, i)
+ mtk_fb_wait(plane->state->fb);
+}
+
+static void mtk_atomic_complete(struct mtk_drm_private *private,
+ struct drm_atomic_state *state)
+{
+ struct drm_device *drm = private->drm;
+
+ mtk_atomic_wait_for_fences(state);
+
+ drm_atomic_helper_commit_modeset_disables(drm, state);
+ drm_atomic_helper_commit_planes(drm, state, false);
+ drm_atomic_helper_commit_modeset_enables(drm, state);
+ drm_atomic_helper_wait_for_vblanks(drm, state);
+ drm_atomic_helper_cleanup_planes(drm, state);
+ drm_atomic_state_free(state);
+}
+
+static void mtk_atomic_work(struct work_struct *work)
+{
+ struct mtk_drm_private *private = container_of(work,
+ struct mtk_drm_private, commit.work);
+
+ mtk_atomic_complete(private, private->commit.state);
+}
+
+static int mtk_atomic_commit(struct drm_device *drm,
+ struct drm_atomic_state *state,
+ bool async)
+{
+ struct mtk_drm_private *private = drm->dev_private;
+ int ret;
+
+ ret = drm_atomic_helper_prepare_planes(drm, state);
+ if (ret)
+ return ret;
+
+ mutex_lock(&private->commit.lock);
+ flush_work(&private->commit.work);
+
+ drm_atomic_helper_swap_state(drm, state);
+
+ if (async)
+ mtk_atomic_schedule(private, state);
+ else
+ mtk_atomic_complete(private, state);
+
+ mutex_unlock(&private->commit.lock);
+
+ return 0;
+}
+
+static const struct drm_mode_config_funcs mtk_drm_mode_config_funcs = {
+ .fb_create = mtk_drm_mode_fb_create,
+ .atomic_check = drm_atomic_helper_check,
+ .atomic_commit = mtk_atomic_commit,
+};
+
+static const enum mtk_ddp_comp_id mtk_ddp_main[] = {
+ DDP_COMPONENT_OVL0,
+ DDP_COMPONENT_COLOR0,
+ DDP_COMPONENT_AAL,
+ DDP_COMPONENT_OD,
+ DDP_COMPONENT_RDMA0,
+ DDP_COMPONENT_UFOE,
+ DDP_COMPONENT_DSI0,
+ DDP_COMPONENT_PWM0,
+};
+
+static const enum mtk_ddp_comp_id mtk_ddp_ext[] = {
+ DDP_COMPONENT_OVL1,
+ DDP_COMPONENT_COLOR1,
+ DDP_COMPONENT_GAMMA,
+ DDP_COMPONENT_RDMA1,
+ DDP_COMPONENT_DPI0,
+};
+
+static int mtk_drm_kms_init(struct drm_device *drm)
+{
+ struct mtk_drm_private *private = drm->dev_private;
+ struct platform_device *pdev;
+ struct device_node *np;
+ int ret;
+
+ if (!iommu_present(&platform_bus_type))
+ return -EPROBE_DEFER;
+
+ pdev = of_find_device_by_node(private->mutex_node);
+ if (!pdev) {
+ dev_err(drm->dev, "Waiting for disp-mutex device %s\n",
+ private->mutex_node->full_name);
+ of_node_put(private->mutex_node);
+ return -EPROBE_DEFER;
+ }
+ private->mutex_dev = &pdev->dev;
+
+ drm_mode_config_init(drm);
+
+ drm->mode_config.min_width = 64;
+ drm->mode_config.min_height = 64;
+
+ /*
+ * set max width and height as default value(4096x4096).
+ * this value would be used to check framebuffer size limitation
+ * at drm_mode_addfb().
+ */
+ drm->mode_config.max_width = 4096;
+ drm->mode_config.max_height = 4096;
+ drm->mode_config.funcs = &mtk_drm_mode_config_funcs;
+
+ ret = component_bind_all(drm->dev, drm);
+ if (ret)
+ goto err_config_cleanup;
+
+ /*
+ * We currently support two fixed data streams, each optional,
+ * and each statically assigned to a crtc:
+ * OVL0 -> COLOR0 -> AAL -> OD -> RDMA0 -> UFOE -> DSI0 ...
+ */
+ ret = mtk_drm_crtc_create(drm, mtk_ddp_main, ARRAY_SIZE(mtk_ddp_main));
+ if (ret < 0)
+ goto err_component_unbind;
+ /* ... and OVL1 -> COLOR1 -> GAMMA -> RDMA1 -> DPI0. */
+ ret = mtk_drm_crtc_create(drm, mtk_ddp_ext, ARRAY_SIZE(mtk_ddp_ext));
+ if (ret < 0)
+ goto err_component_unbind;
+
+ /* Use OVL device for all DMA memory allocations */
+ np = private->comp_node[mtk_ddp_main[0]] ?:
+ private->comp_node[mtk_ddp_ext[0]];
+ pdev = of_find_device_by_node(np);
+ if (!pdev) {
+ ret = -ENODEV;
+ dev_err(drm->dev, "Need at least one OVL device\n");
+ goto err_component_unbind;
+ }
+
+ private->dma_dev = &pdev->dev;
+
+ /*
+ * We don't use the drm_irq_install() helpers provided by the DRM
+ * core, so we need to set this manually in order to allow the
+ * DRM_IOCTL_WAIT_VBLANK to operate correctly.
+ */
+ drm->irq_enabled = true;
+ ret = drm_vblank_init(drm, MAX_CRTC);
+ if (ret < 0)
+ goto err_component_unbind;
+
+ drm_kms_helper_poll_init(drm);
+ drm_mode_config_reset(drm);
+
+ return 0;
+
+err_component_unbind:
+ component_unbind_all(drm->dev, drm);
+err_config_cleanup:
+ drm_mode_config_cleanup(drm);
+
+ return ret;
+}
+
+static void mtk_drm_kms_deinit(struct drm_device *drm)
+{
+ drm_kms_helper_poll_fini(drm);
+
+ drm_vblank_cleanup(drm);
+ component_unbind_all(drm->dev, drm);
+ drm_mode_config_cleanup(drm);
+}
+
+static const struct file_operations mtk_drm_fops = {
+ .owner = THIS_MODULE,
+ .open = drm_open,
+ .release = drm_release,
+ .unlocked_ioctl = drm_ioctl,
+ .mmap = mtk_drm_gem_mmap,
+ .poll = drm_poll,
+ .read = drm_read,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = drm_compat_ioctl,
+#endif
+};
+
+static struct drm_driver mtk_drm_driver = {
+ .driver_features = DRIVER_MODESET | DRIVER_GEM | DRIVER_PRIME |
+ DRIVER_ATOMIC,
+
+ .get_vblank_counter = drm_vblank_count,
+ .enable_vblank = mtk_drm_crtc_enable_vblank,
+ .disable_vblank = mtk_drm_crtc_disable_vblank,
+
+ .gem_free_object = mtk_drm_gem_free_object,
+ .gem_vm_ops = &drm_gem_cma_vm_ops,
+ .dumb_create = mtk_drm_gem_dumb_create,
+ .dumb_map_offset = mtk_drm_gem_dumb_map_offset,
+ .dumb_destroy = drm_gem_dumb_destroy,
+
+ .prime_handle_to_fd = drm_gem_prime_handle_to_fd,
+ .prime_fd_to_handle = drm_gem_prime_fd_to_handle,
+ .gem_prime_export = drm_gem_prime_export,
+ .gem_prime_import = drm_gem_prime_import,
+ .gem_prime_get_sg_table = mtk_gem_prime_get_sg_table,
+ .gem_prime_import_sg_table = mtk_gem_prime_import_sg_table,
+ .gem_prime_mmap = mtk_drm_gem_mmap_buf,
+ .fops = &mtk_drm_fops,
+
+ .name = DRIVER_NAME,
+ .desc = DRIVER_DESC,
+ .date = DRIVER_DATE,
+ .major = DRIVER_MAJOR,
+ .minor = DRIVER_MINOR,
+};
+
+static int compare_of(struct device *dev, void *data)
+{
+ return dev->of_node == data;
+}
+
+static int mtk_drm_bind(struct device *dev)
+{
+ struct mtk_drm_private *private = dev_get_drvdata(dev);
+ struct drm_device *drm;
+ int ret;
+
+ drm = drm_dev_alloc(&mtk_drm_driver, dev);
+ if (!drm)
+ return -ENOMEM;
+
+ drm_dev_set_unique(drm, dev_name(dev));
+
+ drm->dev_private = private;
+ private->drm = drm;
+
+ ret = mtk_drm_kms_init(drm);
+ if (ret < 0)
+ goto err_free;
+
+ ret = drm_dev_register(drm, 0);
+ if (ret < 0)
+ goto err_deinit;
+
+ ret = drm_connector_register_all(drm);
+ if (ret < 0)
+ goto err_unregister;
+
+ return 0;
+
+err_unregister:
+ drm_dev_unregister(drm);
+err_deinit:
+ mtk_drm_kms_deinit(drm);
+err_free:
+ drm_dev_unref(drm);
+ return ret;
+}
+
+static void mtk_drm_unbind(struct device *dev)
+{
+ struct mtk_drm_private *private = dev_get_drvdata(dev);
+
+ drm_put_dev(private->drm);
+ private->drm = NULL;
+}
+
+static const struct component_master_ops mtk_drm_ops = {
+ .bind = mtk_drm_bind,
+ .unbind = mtk_drm_unbind,
+};
+
+static const struct of_device_id mtk_ddp_comp_dt_ids[] = {
+ { .compatible = "mediatek,mt8173-disp-ovl", .data = (void *)MTK_DISP_OVL },
+ { .compatible = "mediatek,mt8173-disp-rdma", .data = (void *)MTK_DISP_RDMA },
+ { .compatible = "mediatek,mt8173-disp-wdma", .data = (void *)MTK_DISP_WDMA },
+ { .compatible = "mediatek,mt8173-disp-color", .data = (void *)MTK_DISP_COLOR },
+ { .compatible = "mediatek,mt8173-disp-aal", .data = (void *)MTK_DISP_AAL},
+ { .compatible = "mediatek,mt8173-disp-gamma", .data = (void *)MTK_DISP_GAMMA, },
+ { .compatible = "mediatek,mt8173-disp-ufoe", .data = (void *)MTK_DISP_UFOE },
+ { .compatible = "mediatek,mt8173-dsi", .data = (void *)MTK_DSI },
+ { .compatible = "mediatek,mt8173-dpi", .data = (void *)MTK_DPI },
+ { .compatible = "mediatek,mt8173-disp-mutex", .data = (void *)MTK_DISP_MUTEX },
+ { .compatible = "mediatek,mt8173-disp-pwm", .data = (void *)MTK_DISP_PWM },
+ { .compatible = "mediatek,mt8173-disp-od", .data = (void *)MTK_DISP_OD },
+ { }
+};
+
+static int mtk_drm_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct mtk_drm_private *private;
+ struct resource *mem;
+ struct device_node *node;
+ struct component_match *match = NULL;
+ int ret;
+ int i;
+
+ private = devm_kzalloc(dev, sizeof(*private), GFP_KERNEL);
+ if (!private)
+ return -ENOMEM;
+
+ mutex_init(&private->commit.lock);
+ INIT_WORK(&private->commit.work, mtk_atomic_work);
+
+ mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ private->config_regs = devm_ioremap_resource(dev, mem);
+ if (IS_ERR(private->config_regs)) {
+ ret = PTR_ERR(private->config_regs);
+ dev_err(dev, "Failed to ioremap mmsys-config resource: %d\n",
+ ret);
+ return ret;
+ }
+
+ /* Iterate over sibling DISP function blocks */
+ for_each_child_of_node(dev->of_node->parent, node) {
+ const struct of_device_id *of_id;
+ enum mtk_ddp_comp_type comp_type;
+ int comp_id;
+
+ of_id = of_match_node(mtk_ddp_comp_dt_ids, node);
+ if (!of_id)
+ continue;
+
+ if (!of_device_is_available(node)) {
+ dev_dbg(dev, "Skipping disabled component %s\n",
+ node->full_name);
+ continue;
+ }
+
+ comp_type = (enum mtk_ddp_comp_type)of_id->data;
+
+ if (comp_type == MTK_DISP_MUTEX) {
+ private->mutex_node = of_node_get(node);
+ continue;
+ }
+
+ comp_id = mtk_ddp_comp_get_id(node, comp_type);
+ if (comp_id < 0) {
+ dev_warn(dev, "Skipping unknown component %s\n",
+ node->full_name);
+ continue;
+ }
+
+ private->comp_node[comp_id] = of_node_get(node);
+
+ /*
+ * Currently only the OVL, RDMA, DSI, and DPI blocks have
+ * separate component platform drivers and initialize their own
+ * DDP component structure. The others are initialized here.
+ */
+ if (comp_type == MTK_DISP_OVL ||
+ comp_type == MTK_DISP_RDMA ||
+ comp_type == MTK_DSI ||
+ comp_type == MTK_DPI) {
+ dev_info(dev, "Adding component match for %s\n",
+ node->full_name);
+ component_match_add(dev, &match, compare_of, node);
+ } else {
+ struct mtk_ddp_comp *comp;
+
+ comp = devm_kzalloc(dev, sizeof(*comp), GFP_KERNEL);
+ if (!comp) {
+ ret = -ENOMEM;
+ goto err_node;
+ }
+
+ ret = mtk_ddp_comp_init(dev, node, comp, comp_id, NULL);
+ if (ret)
+ goto err_node;
+
+ private->ddp_comp[comp_id] = comp;
+ }
+ }
+
+ if (!private->mutex_node) {
+ dev_err(dev, "Failed to find disp-mutex node\n");
+ ret = -ENODEV;
+ goto err_node;
+ }
+
+ pm_runtime_enable(dev);
+
+ platform_set_drvdata(pdev, private);
+
+ ret = component_master_add_with_match(dev, &mtk_drm_ops, match);
+ if (ret)
+ goto err_pm;
+
+ return 0;
+
+err_pm:
+ pm_runtime_disable(dev);
+err_node:
+ of_node_put(private->mutex_node);
+ for (i = 0; i < DDP_COMPONENT_ID_MAX; i++)
+ of_node_put(private->comp_node[i]);
+ return ret;
+}
+
+static int mtk_drm_remove(struct platform_device *pdev)
+{
+ struct mtk_drm_private *private = platform_get_drvdata(pdev);
+ struct drm_device *drm = private->drm;
+ int i;
+
+ drm_connector_unregister_all(drm);
+ drm_dev_unregister(drm);
+ mtk_drm_kms_deinit(drm);
+ drm_dev_unref(drm);
+
+ component_master_del(&pdev->dev, &mtk_drm_ops);
+ pm_runtime_disable(&pdev->dev);
+ of_node_put(private->mutex_node);
+ for (i = 0; i < DDP_COMPONENT_ID_MAX; i++)
+ of_node_put(private->comp_node[i]);
+
+ return 0;
+}
+
+#ifdef CONFIG_PM_SLEEP
+static int mtk_drm_sys_suspend(struct device *dev)
+{
+ struct mtk_drm_private *private = dev_get_drvdata(dev);
+ struct drm_device *drm = private->drm;
+
+ drm_kms_helper_poll_disable(drm);
+
+ private->suspend_state = drm_atomic_helper_suspend(drm);
+ if (IS_ERR(private->suspend_state)) {
+ drm_kms_helper_poll_enable(drm);
+ return PTR_ERR(private->suspend_state);
+ }
+
+ DRM_DEBUG_DRIVER("mtk_drm_sys_suspend\n");
+ return 0;
+}
+
+static int mtk_drm_sys_resume(struct device *dev)
+{
+ struct mtk_drm_private *private = dev_get_drvdata(dev);
+ struct drm_device *drm = private->drm;
+
+ drm_atomic_helper_resume(drm, private->suspend_state);
+ drm_kms_helper_poll_enable(drm);
+
+ DRM_DEBUG_DRIVER("mtk_drm_sys_resume\n");
+ return 0;
+}
+#endif
+
+static SIMPLE_DEV_PM_OPS(mtk_drm_pm_ops, mtk_drm_sys_suspend,
+ mtk_drm_sys_resume);
+
+static const struct of_device_id mtk_drm_of_ids[] = {
+ { .compatible = "mediatek,mt8173-mmsys", },
+ { }
+};
+
+static struct platform_driver mtk_drm_platform_driver = {
+ .probe = mtk_drm_probe,
+ .remove = mtk_drm_remove,
+ .driver = {
+ .name = "mediatek-drm",
+ .of_match_table = mtk_drm_of_ids,
+ .pm = &mtk_drm_pm_ops,
+ },
+};
+
+static struct platform_driver * const mtk_drm_drivers[] = {
+ &mtk_ddp_driver,
+ &mtk_disp_ovl_driver,
+ &mtk_disp_rdma_driver,
+ &mtk_dpi_driver,
+ &mtk_drm_platform_driver,
+ &mtk_dsi_driver,
+ &mtk_mipi_tx_driver,
+};
+
+static int __init mtk_drm_init(void)
+{
+ int ret;
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(mtk_drm_drivers); i++) {
+ ret = platform_driver_register(mtk_drm_drivers[i]);
+ if (ret < 0) {
+ pr_err("Failed to register %s driver: %d\n",
+ mtk_drm_drivers[i]->driver.name, ret);
+ goto err;
+ }
+ }
+
+ return 0;
+
+err:
+ while (--i >= 0)
+ platform_driver_unregister(mtk_drm_drivers[i]);
+
+ return ret;
+}
+
+static void __exit mtk_drm_exit(void)
+{
+ int i;
+
+ for (i = ARRAY_SIZE(mtk_drm_drivers) - 1; i >= 0; i--)
+ platform_driver_unregister(mtk_drm_drivers[i]);
+}
+
+module_init(mtk_drm_init);
+module_exit(mtk_drm_exit);
+
+MODULE_AUTHOR("YT SHEN <yt.shen@mediatek.com>");
+MODULE_DESCRIPTION("Mediatek SoC DRM driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_drv.h b/drivers/gpu/drm/mediatek/mtk_drm_drv.h
new file mode 100644
index 0000000..aa93894
--- /dev/null
+++ b/drivers/gpu/drm/mediatek/mtk_drm_drv.h
@@ -0,0 +1,60 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef MTK_DRM_DRV_H
+#define MTK_DRM_DRV_H
+
+#include <linux/io.h>
+#include "mtk_drm_ddp_comp.h"
+
+#define MAX_CRTC 2
+#define MAX_CONNECTOR 2
+
+struct device;
+struct device_node;
+struct drm_crtc;
+struct drm_device;
+struct drm_fb_helper;
+struct drm_property;
+struct regmap;
+
+struct mtk_drm_private {
+ struct drm_device *drm;
+ struct device *dma_dev;
+
+ struct drm_crtc *crtc[MAX_CRTC];
+ unsigned int num_pipes;
+
+ struct device_node *mutex_node;
+ struct device *mutex_dev;
+ void __iomem *config_regs;
+ struct device_node *comp_node[DDP_COMPONENT_ID_MAX];
+ struct mtk_ddp_comp *ddp_comp[DDP_COMPONENT_ID_MAX];
+
+ struct {
+ struct drm_atomic_state *state;
+ struct work_struct work;
+ struct mutex lock;
+ } commit;
+
+ struct drm_atomic_state *suspend_state;
+};
+
+extern struct platform_driver mtk_ddp_driver;
+extern struct platform_driver mtk_disp_ovl_driver;
+extern struct platform_driver mtk_disp_rdma_driver;
+extern struct platform_driver mtk_dpi_driver;
+extern struct platform_driver mtk_dsi_driver;
+extern struct platform_driver mtk_mipi_tx_driver;
+
+#endif /* MTK_DRM_DRV_H */
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_fb.c b/drivers/gpu/drm/mediatek/mtk_drm_fb.c
new file mode 100644
index 0000000..147df85
--- /dev/null
+++ b/drivers/gpu/drm/mediatek/mtk_drm_fb.c
@@ -0,0 +1,165 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <drm/drmP.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_fb_helper.h>
+#include <drm/drm_gem.h>
+#include <linux/dma-buf.h>
+#include <linux/reservation.h>
+
+#include "mtk_drm_drv.h"
+#include "mtk_drm_fb.h"
+#include "mtk_drm_gem.h"
+
+/*
+ * mtk specific framebuffer structure.
+ *
+ * @fb: drm framebuffer object.
+ * @gem_obj: array of gem objects.
+ */
+struct mtk_drm_fb {
+ struct drm_framebuffer base;
+ /* For now we only support a single plane */
+ struct drm_gem_object *gem_obj;
+};
+
+#define to_mtk_fb(x) container_of(x, struct mtk_drm_fb, base)
+
+struct drm_gem_object *mtk_fb_get_gem_obj(struct drm_framebuffer *fb)
+{
+ struct mtk_drm_fb *mtk_fb = to_mtk_fb(fb);
+
+ return mtk_fb->gem_obj;
+}
+
+static int mtk_drm_fb_create_handle(struct drm_framebuffer *fb,
+ struct drm_file *file_priv,
+ unsigned int *handle)
+{
+ struct mtk_drm_fb *mtk_fb = to_mtk_fb(fb);
+
+ return drm_gem_handle_create(file_priv, mtk_fb->gem_obj, handle);
+}
+
+static void mtk_drm_fb_destroy(struct drm_framebuffer *fb)
+{
+ struct mtk_drm_fb *mtk_fb = to_mtk_fb(fb);
+
+ drm_framebuffer_cleanup(fb);
+
+ drm_gem_object_unreference_unlocked(mtk_fb->gem_obj);
+
+ kfree(mtk_fb);
+}
+
+static const struct drm_framebuffer_funcs mtk_drm_fb_funcs = {
+ .create_handle = mtk_drm_fb_create_handle,
+ .destroy = mtk_drm_fb_destroy,
+};
+
+static struct mtk_drm_fb *mtk_drm_framebuffer_init(struct drm_device *dev,
+ const struct drm_mode_fb_cmd2 *mode,
+ struct drm_gem_object *obj)
+{
+ struct mtk_drm_fb *mtk_fb;
+ int ret;
+
+ if (drm_format_num_planes(mode->pixel_format) != 1)
+ return ERR_PTR(-EINVAL);
+
+ mtk_fb = kzalloc(sizeof(*mtk_fb), GFP_KERNEL);
+ if (!mtk_fb)
+ return ERR_PTR(-ENOMEM);
+
+ drm_helper_mode_fill_fb_struct(&mtk_fb->base, mode);
+
+ mtk_fb->gem_obj = obj;
+
+ ret = drm_framebuffer_init(dev, &mtk_fb->base, &mtk_drm_fb_funcs);
+ if (ret) {
+ DRM_ERROR("failed to initialize framebuffer\n");
+ kfree(mtk_fb);
+ return ERR_PTR(ret);
+ }
+
+ return mtk_fb;
+}
+
+/*
+ * Wait for any exclusive fence in fb's gem object's reservation object.
+ *
+ * Returns -ERESTARTSYS if interrupted, else 0.
+ */
+int mtk_fb_wait(struct drm_framebuffer *fb)
+{
+ struct drm_gem_object *gem;
+ struct reservation_object *resv;
+ long ret;
+
+ if (!fb)
+ return 0;
+
+ gem = mtk_fb_get_gem_obj(fb);
+ if (!gem || !gem->dma_buf || !gem->dma_buf->resv)
+ return 0;
+
+ resv = gem->dma_buf->resv;
+ ret = reservation_object_wait_timeout_rcu(resv, false, true,
+ MAX_SCHEDULE_TIMEOUT);
+ /* MAX_SCHEDULE_TIMEOUT on success, -ERESTARTSYS if interrupted */
+ if (WARN_ON(ret < 0))
+ return ret;
+
+ return 0;
+}
+
+struct drm_framebuffer *mtk_drm_mode_fb_create(struct drm_device *dev,
+ struct drm_file *file,
+ const struct drm_mode_fb_cmd2 *cmd)
+{
+ struct mtk_drm_fb *mtk_fb;
+ struct drm_gem_object *gem;
+ unsigned int width = cmd->width;
+ unsigned int height = cmd->height;
+ unsigned int size, bpp;
+ int ret;
+
+ if (drm_format_num_planes(cmd->pixel_format) != 1)
+ return ERR_PTR(-EINVAL);
+
+ gem = drm_gem_object_lookup(file, cmd->handles[0]);
+ if (!gem)
+ return ERR_PTR(-ENOENT);
+
+ bpp = drm_format_plane_cpp(cmd->pixel_format, 0);
+ size = (height - 1) * cmd->pitches[0] + width * bpp;
+ size += cmd->offsets[0];
+
+ if (gem->size < size) {
+ ret = -EINVAL;
+ goto unreference;
+ }
+
+ mtk_fb = mtk_drm_framebuffer_init(dev, cmd, gem);
+ if (IS_ERR(mtk_fb)) {
+ ret = PTR_ERR(mtk_fb);
+ goto unreference;
+ }
+
+ return &mtk_fb->base;
+
+unreference:
+ drm_gem_object_unreference_unlocked(gem);
+ return ERR_PTR(ret);
+}
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_fb.h b/drivers/gpu/drm/mediatek/mtk_drm_fb.h
new file mode 100644
index 0000000..9b2ae34
--- /dev/null
+++ b/drivers/gpu/drm/mediatek/mtk_drm_fb.h
@@ -0,0 +1,23 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef MTK_DRM_FB_H
+#define MTK_DRM_FB_H
+
+struct drm_gem_object *mtk_fb_get_gem_obj(struct drm_framebuffer *fb);
+int mtk_fb_wait(struct drm_framebuffer *fb);
+struct drm_framebuffer *mtk_drm_mode_fb_create(struct drm_device *dev,
+ struct drm_file *file,
+ const struct drm_mode_fb_cmd2 *cmd);
+
+#endif /* MTK_DRM_FB_H */
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_gem.c b/drivers/gpu/drm/mediatek/mtk_drm_gem.c
new file mode 100644
index 0000000..fa2ec0c
--- /dev/null
+++ b/drivers/gpu/drm/mediatek/mtk_drm_gem.c
@@ -0,0 +1,269 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <drm/drmP.h>
+#include <drm/drm_gem.h>
+#include <linux/dma-buf.h>
+
+#include "mtk_drm_drv.h"
+#include "mtk_drm_gem.h"
+
+static struct mtk_drm_gem_obj *mtk_drm_gem_init(struct drm_device *dev,
+ unsigned long size)
+{
+ struct mtk_drm_gem_obj *mtk_gem_obj;
+ int ret;
+
+ size = round_up(size, PAGE_SIZE);
+
+ mtk_gem_obj = kzalloc(sizeof(*mtk_gem_obj), GFP_KERNEL);
+ if (!mtk_gem_obj)
+ return ERR_PTR(-ENOMEM);
+
+ ret = drm_gem_object_init(dev, &mtk_gem_obj->base, size);
+ if (ret < 0) {
+ DRM_ERROR("failed to initialize gem object\n");
+ kfree(mtk_gem_obj);
+ return ERR_PTR(ret);
+ }
+
+ return mtk_gem_obj;
+}
+
+struct mtk_drm_gem_obj *mtk_drm_gem_create(struct drm_device *dev,
+ size_t size, bool alloc_kmap)
+{
+ struct mtk_drm_private *priv = dev->dev_private;
+ struct mtk_drm_gem_obj *mtk_gem;
+ struct drm_gem_object *obj;
+ int ret;
+
+ mtk_gem = mtk_drm_gem_init(dev, size);
+ if (IS_ERR(mtk_gem))
+ return ERR_CAST(mtk_gem);
+
+ obj = &mtk_gem->base;
+
+ init_dma_attrs(&mtk_gem->dma_attrs);
+ dma_set_attr(DMA_ATTR_WRITE_COMBINE, &mtk_gem->dma_attrs);
+
+ if (!alloc_kmap)
+ dma_set_attr(DMA_ATTR_NO_KERNEL_MAPPING, &mtk_gem->dma_attrs);
+
+ mtk_gem->cookie = dma_alloc_attrs(priv->dma_dev, obj->size,
+ &mtk_gem->dma_addr, GFP_KERNEL,
+ &mtk_gem->dma_attrs);
+ if (!mtk_gem->cookie) {
+ DRM_ERROR("failed to allocate %zx byte dma buffer", obj->size);
+ ret = -ENOMEM;
+ goto err_gem_free;
+ }
+
+ if (alloc_kmap)
+ mtk_gem->kvaddr = mtk_gem->cookie;
+
+ DRM_DEBUG_DRIVER("cookie = %p dma_addr = %pad size = %zu\n",
+ mtk_gem->cookie, &mtk_gem->dma_addr,
+ size);
+
+ return mtk_gem;
+
+err_gem_free:
+ drm_gem_object_release(obj);
+ kfree(mtk_gem);
+ return ERR_PTR(ret);
+}
+
+void mtk_drm_gem_free_object(struct drm_gem_object *obj)
+{
+ struct mtk_drm_gem_obj *mtk_gem = to_mtk_gem_obj(obj);
+ struct mtk_drm_private *priv = obj->dev->dev_private;
+
+ if (mtk_gem->sg)
+ drm_prime_gem_destroy(obj, mtk_gem->sg);
+ else
+ dma_free_attrs(priv->dma_dev, obj->size, mtk_gem->cookie,
+ mtk_gem->dma_addr, &mtk_gem->dma_attrs);
+
+ /* release file pointer to gem object. */
+ drm_gem_object_release(obj);
+
+ kfree(mtk_gem);
+}
+
+int mtk_drm_gem_dumb_create(struct drm_file *file_priv, struct drm_device *dev,
+ struct drm_mode_create_dumb *args)
+{
+ struct mtk_drm_gem_obj *mtk_gem;
+ int ret;
+
+ args->pitch = DIV_ROUND_UP(args->width * args->bpp, 8);
+ args->size = args->pitch * args->height;
+
+ mtk_gem = mtk_drm_gem_create(dev, args->size, false);
+ if (IS_ERR(mtk_gem))
+ return PTR_ERR(mtk_gem);
+
+ /*
+ * allocate a id of idr table where the obj is registered
+ * and handle has the id what user can see.
+ */
+ ret = drm_gem_handle_create(file_priv, &mtk_gem->base, &args->handle);
+ if (ret)
+ goto err_handle_create;
+
+ /* drop reference from allocate - handle holds it now. */
+ drm_gem_object_unreference_unlocked(&mtk_gem->base);
+
+ return 0;
+
+err_handle_create:
+ mtk_drm_gem_free_object(&mtk_gem->base);
+ return ret;
+}
+
+int mtk_drm_gem_dumb_map_offset(struct drm_file *file_priv,
+ struct drm_device *dev, uint32_t handle,
+ uint64_t *offset)
+{
+ struct drm_gem_object *obj;
+ int ret;
+
+ obj = drm_gem_object_lookup(file_priv, handle);
+ if (!obj) {
+ DRM_ERROR("failed to lookup gem object.\n");
+ return -EINVAL;
+ }
+
+ ret = drm_gem_create_mmap_offset(obj);
+ if (ret)
+ goto out;
+
+ *offset = drm_vma_node_offset_addr(&obj->vma_node);
+ DRM_DEBUG_KMS("offset = 0x%llx\n", *offset);
+
+out:
+ drm_gem_object_unreference_unlocked(obj);
+ return ret;
+}
+
+static int mtk_drm_gem_object_mmap(struct drm_gem_object *obj,
+ struct vm_area_struct *vma)
+
+{
+ int ret;
+ struct mtk_drm_gem_obj *mtk_gem = to_mtk_gem_obj(obj);
+ struct mtk_drm_private *priv = obj->dev->dev_private;
+
+ /*
+ * dma_alloc_attrs() allocated a struct page table for mtk_gem, so clear
+ * VM_PFNMAP flag that was set by drm_gem_mmap_obj()/drm_gem_mmap().
+ */
+ vma->vm_flags &= ~VM_PFNMAP;
+ vma->vm_pgoff = 0;
+
+ ret = dma_mmap_attrs(priv->dma_dev, vma, mtk_gem->cookie,
+ mtk_gem->dma_addr, obj->size, &mtk_gem->dma_attrs);
+ if (ret)
+ drm_gem_vm_close(vma);
+
+ return ret;
+}
+
+int mtk_drm_gem_mmap_buf(struct drm_gem_object *obj, struct vm_area_struct *vma)
+{
+ int ret;
+
+ ret = drm_gem_mmap_obj(obj, obj->size, vma);
+ if (ret)
+ return ret;
+
+ return mtk_drm_gem_object_mmap(obj, vma);
+}
+
+int mtk_drm_gem_mmap(struct file *filp, struct vm_area_struct *vma)
+{
+ struct drm_gem_object *obj;
+ int ret;
+
+ ret = drm_gem_mmap(filp, vma);
+ if (ret)
+ return ret;
+
+ obj = vma->vm_private_data;
+
+ return mtk_drm_gem_object_mmap(obj, vma);
+}
+
+/*
+ * Allocate a sg_table for this GEM object.
+ * Note: Both the table's contents, and the sg_table itself must be freed by
+ * the caller.
+ * Returns a pointer to the newly allocated sg_table, or an ERR_PTR() error.
+ */
+struct sg_table *mtk_gem_prime_get_sg_table(struct drm_gem_object *obj)
+{
+ struct mtk_drm_gem_obj *mtk_gem = to_mtk_gem_obj(obj);
+ struct mtk_drm_private *priv = obj->dev->dev_private;
+ struct sg_table *sgt;
+ int ret;
+
+ sgt = kzalloc(sizeof(*sgt), GFP_KERNEL);
+ if (!sgt)
+ return ERR_PTR(-ENOMEM);
+
+ ret = dma_get_sgtable_attrs(priv->dma_dev, sgt, mtk_gem->cookie,
+ mtk_gem->dma_addr, obj->size,
+ &mtk_gem->dma_attrs);
+ if (ret) {
+ DRM_ERROR("failed to allocate sgt, %d\n", ret);
+ kfree(sgt);
+ return ERR_PTR(ret);
+ }
+
+ return sgt;
+}
+
+struct drm_gem_object *mtk_gem_prime_import_sg_table(struct drm_device *dev,
+ struct dma_buf_attachment *attach, struct sg_table *sg)
+{
+ struct mtk_drm_gem_obj *mtk_gem;
+ int ret;
+ struct scatterlist *s;
+ unsigned int i;
+ dma_addr_t expected;
+
+ mtk_gem = mtk_drm_gem_init(dev, attach->dmabuf->size);
+
+ if (IS_ERR(mtk_gem))
+ return ERR_PTR(PTR_ERR(mtk_gem));
+
+ expected = sg_dma_address(sg->sgl);
+ for_each_sg(sg->sgl, s, sg->nents, i) {
+ if (sg_dma_address(s) != expected) {
+ DRM_ERROR("sg_table is not contiguous");
+ ret = -EINVAL;
+ goto err_gem_free;
+ }
+ expected = sg_dma_address(s) + sg_dma_len(s);
+ }
+
+ mtk_gem->dma_addr = sg_dma_address(sg->sgl);
+ mtk_gem->sg = sg;
+
+ return &mtk_gem->base;
+
+err_gem_free:
+ kfree(mtk_gem);
+ return ERR_PTR(ret);
+}
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_gem.h b/drivers/gpu/drm/mediatek/mtk_drm_gem.h
new file mode 100644
index 0000000..3a2a562
--- /dev/null
+++ b/drivers/gpu/drm/mediatek/mtk_drm_gem.h
@@ -0,0 +1,59 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _MTK_DRM_GEM_H_
+#define _MTK_DRM_GEM_H_
+
+#include <drm/drm_gem.h>
+
+/*
+ * mtk drm buffer structure.
+ *
+ * @base: a gem object.
+ * - a new handle to this gem object would be created
+ * by drm_gem_handle_create().
+ * @cookie: the return value of dma_alloc_attrs(), keep it for dma_free_attrs()
+ * @kvaddr: kernel virtual address of gem buffer.
+ * @dma_addr: dma address of gem buffer.
+ * @dma_attrs: dma attributes of gem buffer.
+ *
+ * P.S. this object would be transferred to user as kms_bo.handle so
+ * user can access the buffer through kms_bo.handle.
+ */
+struct mtk_drm_gem_obj {
+ struct drm_gem_object base;
+ void *cookie;
+ void *kvaddr;
+ dma_addr_t dma_addr;
+ struct dma_attrs dma_attrs;
+ struct sg_table *sg;
+};
+
+#define to_mtk_gem_obj(x) container_of(x, struct mtk_drm_gem_obj, base)
+
+void mtk_drm_gem_free_object(struct drm_gem_object *gem);
+struct mtk_drm_gem_obj *mtk_drm_gem_create(struct drm_device *dev, size_t size,
+ bool alloc_kmap);
+int mtk_drm_gem_dumb_create(struct drm_file *file_priv, struct drm_device *dev,
+ struct drm_mode_create_dumb *args);
+int mtk_drm_gem_dumb_map_offset(struct drm_file *file_priv,
+ struct drm_device *dev, uint32_t handle,
+ uint64_t *offset);
+int mtk_drm_gem_mmap(struct file *filp, struct vm_area_struct *vma);
+int mtk_drm_gem_mmap_buf(struct drm_gem_object *obj,
+ struct vm_area_struct *vma);
+struct sg_table *mtk_gem_prime_get_sg_table(struct drm_gem_object *obj);
+struct drm_gem_object *mtk_gem_prime_import_sg_table(struct drm_device *dev,
+ struct dma_buf_attachment *attach, struct sg_table *sg);
+
+#endif
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_plane.c b/drivers/gpu/drm/mediatek/mtk_drm_plane.c
new file mode 100644
index 0000000..51bc898
--- /dev/null
+++ b/drivers/gpu/drm/mediatek/mtk_drm_plane.c
@@ -0,0 +1,240 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: CK Hu <ck.hu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <drm/drmP.h>
+#include <drm/drm_atomic.h>
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_plane_helper.h>
+
+#include "mtk_drm_crtc.h"
+#include "mtk_drm_ddp_comp.h"
+#include "mtk_drm_drv.h"
+#include "mtk_drm_fb.h"
+#include "mtk_drm_gem.h"
+#include "mtk_drm_plane.h"
+
+static const u32 formats[] = {
+ DRM_FORMAT_XRGB8888,
+ DRM_FORMAT_ARGB8888,
+ DRM_FORMAT_RGB565,
+};
+
+static void mtk_plane_enable(struct mtk_drm_plane *mtk_plane, bool enable,
+ dma_addr_t addr, struct drm_rect *dest)
+{
+ struct drm_plane *plane = &mtk_plane->base;
+ struct mtk_plane_state *state = to_mtk_plane_state(plane->state);
+ unsigned int pitch, format;
+ int x, y;
+
+ if (WARN_ON(!plane->state || (enable && !plane->state->fb)))
+ return;
+
+ if (plane->state->fb) {
+ pitch = plane->state->fb->pitches[0];
+ format = plane->state->fb->pixel_format;
+ } else {
+ pitch = 0;
+ format = DRM_FORMAT_RGBA8888;
+ }
+
+ x = plane->state->crtc_x;
+ y = plane->state->crtc_y;
+
+ if (x < 0) {
+ addr -= x * 4;
+ x = 0;
+ }
+
+ if (y < 0) {
+ addr -= y * pitch;
+ y = 0;
+ }
+
+ state->pending.enable = enable;
+ state->pending.pitch = pitch;
+ state->pending.format = format;
+ state->pending.addr = addr;
+ state->pending.x = x;
+ state->pending.y = y;
+ state->pending.width = dest->x2 - dest->x1;
+ state->pending.height = dest->y2 - dest->y1;
+ wmb(); /* Make sure the above parameters are set before update */
+ state->pending.dirty = true;
+}
+
+static void mtk_plane_reset(struct drm_plane *plane)
+{
+ struct mtk_plane_state *state;
+
+ if (plane->state) {
+ if (plane->state->fb)
+ drm_framebuffer_unreference(plane->state->fb);
+
+ state = to_mtk_plane_state(plane->state);
+ memset(state, 0, sizeof(*state));
+ } else {
+ state = kzalloc(sizeof(*state), GFP_KERNEL);
+ if (!state)
+ return;
+ plane->state = &state->base;
+ }
+
+ state->base.plane = plane;
+ state->pending.format = DRM_FORMAT_RGB565;
+}
+
+static struct drm_plane_state *mtk_plane_duplicate_state(struct drm_plane *plane)
+{
+ struct mtk_plane_state *old_state = to_mtk_plane_state(plane->state);
+ struct mtk_plane_state *state;
+
+ state = kzalloc(sizeof(*state), GFP_KERNEL);
+ if (!state)
+ return NULL;
+
+ __drm_atomic_helper_plane_duplicate_state(plane, &state->base);
+
+ WARN_ON(state->base.plane != plane);
+
+ state->pending = old_state->pending;
+
+ return &state->base;
+}
+
+static void mtk_drm_plane_destroy_state(struct drm_plane *plane,
+ struct drm_plane_state *state)
+{
+ __drm_atomic_helper_plane_destroy_state(state);
+ kfree(to_mtk_plane_state(state));
+}
+
+static const struct drm_plane_funcs mtk_plane_funcs = {
+ .update_plane = drm_atomic_helper_update_plane,
+ .disable_plane = drm_atomic_helper_disable_plane,
+ .destroy = drm_plane_cleanup,
+ .reset = mtk_plane_reset,
+ .atomic_duplicate_state = mtk_plane_duplicate_state,
+ .atomic_destroy_state = mtk_drm_plane_destroy_state,
+};
+
+static int mtk_plane_atomic_check(struct drm_plane *plane,
+ struct drm_plane_state *state)
+{
+ struct drm_framebuffer *fb = state->fb;
+ struct drm_crtc_state *crtc_state;
+ bool visible;
+ struct drm_rect dest = {
+ .x1 = state->crtc_x,
+ .y1 = state->crtc_y,
+ .x2 = state->crtc_x + state->crtc_w,
+ .y2 = state->crtc_y + state->crtc_h,
+ };
+ struct drm_rect src = {
+ /* 16.16 fixed point */
+ .x1 = state->src_x,
+ .y1 = state->src_y,
+ .x2 = state->src_x + state->src_w,
+ .y2 = state->src_y + state->src_h,
+ };
+ struct drm_rect clip = { 0, };
+
+ if (!fb)
+ return 0;
+
+ if (!mtk_fb_get_gem_obj(fb)) {
+ DRM_DEBUG_KMS("buffer is null\n");
+ return -EFAULT;
+ }
+
+ if (!state->crtc)
+ return 0;
+
+ crtc_state = drm_atomic_get_crtc_state(state->state, state->crtc);
+ if (IS_ERR(crtc_state))
+ return PTR_ERR(crtc_state);
+
+ clip.x2 = crtc_state->mode.hdisplay;
+ clip.y2 = crtc_state->mode.vdisplay;
+
+ return drm_plane_helper_check_update(plane, state->crtc, fb,
+ &src, &dest, &clip,
+ DRM_PLANE_HELPER_NO_SCALING,
+ DRM_PLANE_HELPER_NO_SCALING,
+ true, true, &visible);
+}
+
+static void mtk_plane_atomic_update(struct drm_plane *plane,
+ struct drm_plane_state *old_state)
+{
+ struct mtk_plane_state *state = to_mtk_plane_state(plane->state);
+ struct drm_crtc *crtc = state->base.crtc;
+ struct drm_gem_object *gem;
+ struct mtk_drm_gem_obj *mtk_gem;
+ struct mtk_drm_plane *mtk_plane = to_mtk_plane(plane);
+ struct drm_rect dest = {
+ .x1 = state->base.crtc_x,
+ .y1 = state->base.crtc_y,
+ .x2 = state->base.crtc_x + state->base.crtc_w,
+ .y2 = state->base.crtc_y + state->base.crtc_h,
+ };
+ struct drm_rect clip = { 0, };
+
+ if (!crtc)
+ return;
+
+ clip.x2 = state->base.crtc->state->mode.hdisplay;
+ clip.y2 = state->base.crtc->state->mode.vdisplay;
+ drm_rect_intersect(&dest, &clip);
+
+ gem = mtk_fb_get_gem_obj(state->base.fb);
+ mtk_gem = to_mtk_gem_obj(gem);
+ mtk_plane_enable(mtk_plane, true, mtk_gem->dma_addr, &dest);
+}
+
+static void mtk_plane_atomic_disable(struct drm_plane *plane,
+ struct drm_plane_state *old_state)
+{
+ struct mtk_plane_state *state = to_mtk_plane_state(plane->state);
+
+ state->pending.enable = false;
+ wmb(); /* Make sure the above parameter is set before update */
+ state->pending.dirty = true;
+}
+
+static const struct drm_plane_helper_funcs mtk_plane_helper_funcs = {
+ .atomic_check = mtk_plane_atomic_check,
+ .atomic_update = mtk_plane_atomic_update,
+ .atomic_disable = mtk_plane_atomic_disable,
+};
+
+int mtk_plane_init(struct drm_device *dev, struct mtk_drm_plane *mtk_plane,
+ unsigned long possible_crtcs, enum drm_plane_type type,
+ unsigned int zpos)
+{
+ int err;
+
+ err = drm_universal_plane_init(dev, &mtk_plane->base, possible_crtcs,
+ &mtk_plane_funcs, formats,
+ ARRAY_SIZE(formats), type, NULL);
+ if (err) {
+ DRM_ERROR("failed to initialize plane\n");
+ return err;
+ }
+
+ drm_plane_helper_add(&mtk_plane->base, &mtk_plane_helper_funcs);
+ mtk_plane->idx = zpos;
+
+ return 0;
+}
diff --git a/drivers/gpu/drm/mediatek/mtk_drm_plane.h b/drivers/gpu/drm/mediatek/mtk_drm_plane.h
new file mode 100644
index 0000000..72a7b3e
--- /dev/null
+++ b/drivers/gpu/drm/mediatek/mtk_drm_plane.h
@@ -0,0 +1,59 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ * Author: CK Hu <ck.hu@mediatek.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _MTK_DRM_PLANE_H_
+#define _MTK_DRM_PLANE_H_
+
+#include <drm/drm_crtc.h>
+#include <linux/types.h>
+
+struct mtk_drm_plane {
+ struct drm_plane base;
+ unsigned int idx;
+};
+
+struct mtk_plane_pending_state {
+ bool config;
+ bool enable;
+ dma_addr_t addr;
+ unsigned int pitch;
+ unsigned int format;
+ unsigned int x;
+ unsigned int y;
+ unsigned int width;
+ unsigned int height;
+ bool dirty;
+};
+
+struct mtk_plane_state {
+ struct drm_plane_state base;
+ struct mtk_plane_pending_state pending;
+};
+
+static inline struct mtk_drm_plane *to_mtk_plane(struct drm_plane *plane)
+{
+ return container_of(plane, struct mtk_drm_plane, base);
+}
+
+static inline struct mtk_plane_state *
+to_mtk_plane_state(struct drm_plane_state *state)
+{
+ return container_of(state, struct mtk_plane_state, base);
+}
+
+int mtk_plane_init(struct drm_device *dev, struct mtk_drm_plane *mtk_plane,
+ unsigned long possible_crtcs, enum drm_plane_type type,
+ unsigned int zpos);
+
+#endif
diff --git a/drivers/gpu/drm/mediatek/mtk_dsi.c b/drivers/gpu/drm/mediatek/mtk_dsi.c
new file mode 100644
index 0000000..2d808e5
--- /dev/null
+++ b/drivers/gpu/drm/mediatek/mtk_dsi.c
@@ -0,0 +1,913 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <drm/drmP.h>
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_mipi_dsi.h>
+#include <drm/drm_panel.h>
+#include <linux/clk.h>
+#include <linux/component.h>
+#include <linux/of.h>
+#include <linux/of_platform.h>
+#include <linux/of_graph.h>
+#include <linux/phy/phy.h>
+#include <linux/platform_device.h>
+#include <video/videomode.h>
+
+#include "mtk_drm_ddp_comp.h"
+
+#define DSI_VIDEO_FIFO_DEPTH (1920 / 4)
+#define DSI_HOST_FIFO_DEPTH 64
+
+#define DSI_START 0x00
+
+#define DSI_CON_CTRL 0x10
+#define DSI_RESET BIT(0)
+#define DSI_EN BIT(1)
+
+#define DSI_MODE_CTRL 0x14
+#define MODE (3)
+#define CMD_MODE 0
+#define SYNC_PULSE_MODE 1
+#define SYNC_EVENT_MODE 2
+#define BURST_MODE 3
+#define FRM_MODE BIT(16)
+#define MIX_MODE BIT(17)
+
+#define DSI_TXRX_CTRL 0x18
+#define VC_NUM (2 << 0)
+#define LANE_NUM (0xf << 2)
+#define DIS_EOT BIT(6)
+#define NULL_EN BIT(7)
+#define TE_FREERUN BIT(8)
+#define EXT_TE_EN BIT(9)
+#define EXT_TE_EDGE BIT(10)
+#define MAX_RTN_SIZE (0xf << 12)
+#define HSTX_CKLP_EN BIT(16)
+
+#define DSI_PSCTRL 0x1c
+#define DSI_PS_WC 0x3fff
+#define DSI_PS_SEL (3 << 16)
+#define PACKED_PS_16BIT_RGB565 (0 << 16)
+#define LOOSELY_PS_18BIT_RGB666 (1 << 16)
+#define PACKED_PS_18BIT_RGB666 (2 << 16)
+#define PACKED_PS_24BIT_RGB888 (3 << 16)
+
+#define DSI_VSA_NL 0x20
+#define DSI_VBP_NL 0x24
+#define DSI_VFP_NL 0x28
+#define DSI_VACT_NL 0x2C
+#define DSI_HSA_WC 0x50
+#define DSI_HBP_WC 0x54
+#define DSI_HFP_WC 0x58
+
+#define DSI_HSTX_CKL_WC 0x64
+
+#define DSI_PHY_LCCON 0x104
+#define LC_HS_TX_EN BIT(0)
+#define LC_ULPM_EN BIT(1)
+#define LC_WAKEUP_EN BIT(2)
+
+#define DSI_PHY_LD0CON 0x108
+#define LD0_HS_TX_EN BIT(0)
+#define LD0_ULPM_EN BIT(1)
+#define LD0_WAKEUP_EN BIT(2)
+
+#define DSI_PHY_TIMECON0 0x110
+#define LPX (0xff << 0)
+#define HS_PRPR (0xff << 8)
+#define HS_ZERO (0xff << 16)
+#define HS_TRAIL (0xff << 24)
+
+#define DSI_PHY_TIMECON1 0x114
+#define TA_GO (0xff << 0)
+#define TA_SURE (0xff << 8)
+#define TA_GET (0xff << 16)
+#define DA_HS_EXIT (0xff << 24)
+
+#define DSI_PHY_TIMECON2 0x118
+#define CONT_DET (0xff << 0)
+#define CLK_ZERO (0xff << 16)
+#define CLK_TRAIL (0xff << 24)
+
+#define DSI_PHY_TIMECON3 0x11c
+#define CLK_HS_PRPR (0xff << 0)
+#define CLK_HS_POST (0xff << 8)
+#define CLK_HS_EXIT (0xff << 16)
+
+#define NS_TO_CYCLE(n, c) ((n) / (c) + (((n) % (c)) ? 1 : 0))
+
+struct phy;
+
+struct mtk_dsi {
+ struct mtk_ddp_comp ddp_comp;
+ struct device *dev;
+ struct mipi_dsi_host host;
+ struct drm_encoder encoder;
+ struct drm_connector conn;
+ struct drm_panel *panel;
+ struct drm_bridge *bridge;
+ struct phy *phy;
+
+ void __iomem *regs;
+
+ struct clk *engine_clk;
+ struct clk *digital_clk;
+ struct clk *hs_clk;
+
+ u32 data_rate;
+
+ unsigned long mode_flags;
+ enum mipi_dsi_pixel_format format;
+ unsigned int lanes;
+ struct videomode vm;
+ int refcount;
+ bool enabled;
+};
+
+static inline struct mtk_dsi *encoder_to_dsi(struct drm_encoder *e)
+{
+ return container_of(e, struct mtk_dsi, encoder);
+}
+
+static inline struct mtk_dsi *connector_to_dsi(struct drm_connector *c)
+{
+ return container_of(c, struct mtk_dsi, conn);
+}
+
+static inline struct mtk_dsi *host_to_dsi(struct mipi_dsi_host *h)
+{
+ return container_of(h, struct mtk_dsi, host);
+}
+
+static void mtk_dsi_mask(struct mtk_dsi *dsi, u32 offset, u32 mask, u32 data)
+{
+ u32 temp = readl(dsi->regs + offset);
+
+ writel((temp & ~mask) | (data & mask), dsi->regs + offset);
+}
+
+static void dsi_phy_timconfig(struct mtk_dsi *dsi)
+{
+ u32 timcon0, timcon1, timcon2, timcon3;
+ unsigned int ui, cycle_time;
+ unsigned int lpx;
+
+ ui = 1000 / dsi->data_rate + 0x01;
+ cycle_time = 8000 / dsi->data_rate + 0x01;
+ lpx = 5;
+
+ timcon0 = (8 << 24) | (0xa << 16) | (0x6 << 8) | lpx;
+ timcon1 = (7 << 24) | (5 * lpx << 16) | ((3 * lpx) / 2) << 8 |
+ (4 * lpx);
+ timcon2 = ((NS_TO_CYCLE(0x64, cycle_time) + 0xa) << 24) |
+ (NS_TO_CYCLE(0x150, cycle_time) << 16);
+ timcon3 = (2 * lpx) << 16 | NS_TO_CYCLE(80 + 52 * ui, cycle_time) << 8 |
+ NS_TO_CYCLE(0x40, cycle_time);
+
+ writel(timcon0, dsi->regs + DSI_PHY_TIMECON0);
+ writel(timcon1, dsi->regs + DSI_PHY_TIMECON1);
+ writel(timcon2, dsi->regs + DSI_PHY_TIMECON2);
+ writel(timcon3, dsi->regs + DSI_PHY_TIMECON3);
+}
+
+static void mtk_dsi_enable(struct mtk_dsi *dsi)
+{
+ mtk_dsi_mask(dsi, DSI_CON_CTRL, DSI_EN, DSI_EN);
+}
+
+static void mtk_dsi_disable(struct mtk_dsi *dsi)
+{
+ mtk_dsi_mask(dsi, DSI_CON_CTRL, DSI_EN, 0);
+}
+
+static void mtk_dsi_reset(struct mtk_dsi *dsi)
+{
+ mtk_dsi_mask(dsi, DSI_CON_CTRL, DSI_RESET, DSI_RESET);
+ mtk_dsi_mask(dsi, DSI_CON_CTRL, DSI_RESET, 0);
+}
+
+static int mtk_dsi_poweron(struct mtk_dsi *dsi)
+{
+ struct device *dev = dsi->dev;
+ int ret;
+
+ if (++dsi->refcount != 1)
+ return 0;
+
+ /**
+ * data_rate = (pixel_clock / 1000) * pixel_dipth * mipi_ratio;
+ * pixel_clock unit is Khz, data_rata unit is MHz, so need divide 1000.
+ * mipi_ratio is mipi clk coefficient for balance the pixel clk in mipi.
+ * we set mipi_ratio is 1.05.
+ */
+ dsi->data_rate = dsi->vm.pixelclock * 3 * 21 / (1 * 1000 * 10);
+
+ ret = clk_set_rate(dsi->hs_clk, dsi->data_rate * 1000000);
+ if (ret < 0) {
+ dev_err(dev, "Failed to set data rate: %d\n", ret);
+ goto err_refcount;
+ }
+
+ phy_power_on(dsi->phy);
+
+ ret = clk_prepare_enable(dsi->engine_clk);
+ if (ret < 0) {
+ dev_err(dev, "Failed to enable engine clock: %d\n", ret);
+ goto err_phy_power_off;
+ }
+
+ ret = clk_prepare_enable(dsi->digital_clk);
+ if (ret < 0) {
+ dev_err(dev, "Failed to enable digital clock: %d\n", ret);
+ goto err_disable_engine_clk;
+ }
+
+ mtk_dsi_enable(dsi);
+ mtk_dsi_reset(dsi);
+ dsi_phy_timconfig(dsi);
+
+ return 0;
+
+err_disable_engine_clk:
+ clk_disable_unprepare(dsi->engine_clk);
+err_phy_power_off:
+ phy_power_off(dsi->phy);
+err_refcount:
+ dsi->refcount--;
+ return ret;
+}
+
+static void dsi_clk_ulp_mode_enter(struct mtk_dsi *dsi)
+{
+ mtk_dsi_mask(dsi, DSI_PHY_LCCON, LC_HS_TX_EN, 0);
+ mtk_dsi_mask(dsi, DSI_PHY_LCCON, LC_ULPM_EN, 0);
+}
+
+static void dsi_clk_ulp_mode_leave(struct mtk_dsi *dsi)
+{
+ mtk_dsi_mask(dsi, DSI_PHY_LCCON, LC_ULPM_EN, 0);
+ mtk_dsi_mask(dsi, DSI_PHY_LCCON, LC_WAKEUP_EN, LC_WAKEUP_EN);
+ mtk_dsi_mask(dsi, DSI_PHY_LCCON, LC_WAKEUP_EN, 0);
+}
+
+static void dsi_lane0_ulp_mode_enter(struct mtk_dsi *dsi)
+{
+ mtk_dsi_mask(dsi, DSI_PHY_LD0CON, LD0_HS_TX_EN, 0);
+ mtk_dsi_mask(dsi, DSI_PHY_LD0CON, LD0_ULPM_EN, 0);
+}
+
+static void dsi_lane0_ulp_mode_leave(struct mtk_dsi *dsi)
+{
+ mtk_dsi_mask(dsi, DSI_PHY_LD0CON, LD0_ULPM_EN, 0);
+ mtk_dsi_mask(dsi, DSI_PHY_LD0CON, LD0_WAKEUP_EN, LD0_WAKEUP_EN);
+ mtk_dsi_mask(dsi, DSI_PHY_LD0CON, LD0_WAKEUP_EN, 0);
+}
+
+static bool dsi_clk_hs_state(struct mtk_dsi *dsi)
+{
+ u32 tmp_reg1;
+
+ tmp_reg1 = readl(dsi->regs + DSI_PHY_LCCON);
+ return ((tmp_reg1 & LC_HS_TX_EN) == 1) ? true : false;
+}
+
+static void dsi_clk_hs_mode(struct mtk_dsi *dsi, bool enter)
+{
+ if (enter && !dsi_clk_hs_state(dsi))
+ mtk_dsi_mask(dsi, DSI_PHY_LCCON, LC_HS_TX_EN, LC_HS_TX_EN);
+ else if (!enter && dsi_clk_hs_state(dsi))
+ mtk_dsi_mask(dsi, DSI_PHY_LCCON, LC_HS_TX_EN, 0);
+}
+
+static void dsi_set_mode(struct mtk_dsi *dsi)
+{
+ u32 vid_mode = CMD_MODE;
+
+ if (dsi->mode_flags & MIPI_DSI_MODE_VIDEO) {
+ vid_mode = SYNC_PULSE_MODE;
+
+ if ((dsi->mode_flags & MIPI_DSI_MODE_VIDEO_BURST) &&
+ !(dsi->mode_flags & MIPI_DSI_MODE_VIDEO_SYNC_PULSE))
+ vid_mode = BURST_MODE;
+ }
+
+ writel(vid_mode, dsi->regs + DSI_MODE_CTRL);
+}
+
+static void dsi_ps_control_vact(struct mtk_dsi *dsi)
+{
+ struct videomode *vm = &dsi->vm;
+ u32 dsi_buf_bpp, ps_wc;
+ u32 ps_bpp_mode;
+
+ if (dsi->format == MIPI_DSI_FMT_RGB565)
+ dsi_buf_bpp = 2;
+ else
+ dsi_buf_bpp = 3;
+
+ ps_wc = vm->hactive * dsi_buf_bpp;
+ ps_bpp_mode = ps_wc;
+
+ switch (dsi->format) {
+ case MIPI_DSI_FMT_RGB888:
+ ps_bpp_mode |= PACKED_PS_24BIT_RGB888;
+ break;
+ case MIPI_DSI_FMT_RGB666:
+ ps_bpp_mode |= PACKED_PS_18BIT_RGB666;
+ break;
+ case MIPI_DSI_FMT_RGB666_PACKED:
+ ps_bpp_mode |= LOOSELY_PS_18BIT_RGB666;
+ break;
+ case MIPI_DSI_FMT_RGB565:
+ ps_bpp_mode |= PACKED_PS_16BIT_RGB565;
+ break;
+ }
+
+ writel(vm->vactive, dsi->regs + DSI_VACT_NL);
+ writel(ps_bpp_mode, dsi->regs + DSI_PSCTRL);
+ writel(ps_wc, dsi->regs + DSI_HSTX_CKL_WC);
+}
+
+static void dsi_rxtx_control(struct mtk_dsi *dsi)
+{
+ u32 tmp_reg;
+
+ switch (dsi->lanes) {
+ case 1:
+ tmp_reg = 1 << 2;
+ break;
+ case 2:
+ tmp_reg = 3 << 2;
+ break;
+ case 3:
+ tmp_reg = 7 << 2;
+ break;
+ case 4:
+ tmp_reg = 0xf << 2;
+ break;
+ default:
+ tmp_reg = 0xf << 2;
+ break;
+ }
+
+ writel(tmp_reg, dsi->regs + DSI_TXRX_CTRL);
+}
+
+static void dsi_ps_control(struct mtk_dsi *dsi)
+{
+ unsigned int dsi_tmp_buf_bpp;
+ u32 tmp_reg;
+
+ switch (dsi->format) {
+ case MIPI_DSI_FMT_RGB888:
+ tmp_reg = PACKED_PS_24BIT_RGB888;
+ dsi_tmp_buf_bpp = 3;
+ break;
+ case MIPI_DSI_FMT_RGB666:
+ tmp_reg = LOOSELY_PS_18BIT_RGB666;
+ dsi_tmp_buf_bpp = 3;
+ break;
+ case MIPI_DSI_FMT_RGB666_PACKED:
+ tmp_reg = PACKED_PS_18BIT_RGB666;
+ dsi_tmp_buf_bpp = 3;
+ break;
+ case MIPI_DSI_FMT_RGB565:
+ tmp_reg = PACKED_PS_16BIT_RGB565;
+ dsi_tmp_buf_bpp = 2;
+ break;
+ default:
+ tmp_reg = PACKED_PS_24BIT_RGB888;
+ dsi_tmp_buf_bpp = 3;
+ break;
+ }
+
+ tmp_reg += dsi->vm.hactive * dsi_tmp_buf_bpp & DSI_PS_WC;
+ writel(tmp_reg, dsi->regs + DSI_PSCTRL);
+}
+
+static void dsi_config_vdo_timing(struct mtk_dsi *dsi)
+{
+ unsigned int horizontal_sync_active_byte;
+ unsigned int horizontal_backporch_byte;
+ unsigned int horizontal_frontporch_byte;
+ unsigned int dsi_tmp_buf_bpp;
+
+ struct videomode *vm = &dsi->vm;
+
+ if (dsi->format == MIPI_DSI_FMT_RGB565)
+ dsi_tmp_buf_bpp = 2;
+ else
+ dsi_tmp_buf_bpp = 3;
+
+ writel(vm->vsync_len, dsi->regs + DSI_VSA_NL);
+ writel(vm->vback_porch, dsi->regs + DSI_VBP_NL);
+ writel(vm->vfront_porch, dsi->regs + DSI_VFP_NL);
+ writel(vm->vactive, dsi->regs + DSI_VACT_NL);
+
+ horizontal_sync_active_byte = (vm->hsync_len * dsi_tmp_buf_bpp - 10);
+
+ if (dsi->mode_flags & MIPI_DSI_MODE_VIDEO_SYNC_PULSE)
+ horizontal_backporch_byte =
+ (vm->hback_porch * dsi_tmp_buf_bpp - 10);
+ else
+ horizontal_backporch_byte = ((vm->hback_porch + vm->hsync_len) *
+ dsi_tmp_buf_bpp - 10);
+
+ horizontal_frontporch_byte = (vm->hfront_porch * dsi_tmp_buf_bpp - 12);
+
+ writel(horizontal_sync_active_byte, dsi->regs + DSI_HSA_WC);
+ writel(horizontal_backporch_byte, dsi->regs + DSI_HBP_WC);
+ writel(horizontal_frontporch_byte, dsi->regs + DSI_HFP_WC);
+
+ dsi_ps_control(dsi);
+}
+
+static void mtk_dsi_start(struct mtk_dsi *dsi)
+{
+ writel(0, dsi->regs + DSI_START);
+ writel(1, dsi->regs + DSI_START);
+}
+
+static void mtk_dsi_poweroff(struct mtk_dsi *dsi)
+{
+ if (WARN_ON(dsi->refcount == 0))
+ return;
+
+ if (--dsi->refcount != 0)
+ return;
+
+ dsi_lane0_ulp_mode_enter(dsi);
+ dsi_clk_ulp_mode_enter(dsi);
+
+ mtk_dsi_disable(dsi);
+
+ clk_disable_unprepare(dsi->engine_clk);
+ clk_disable_unprepare(dsi->digital_clk);
+
+ phy_power_off(dsi->phy);
+}
+
+static void mtk_output_dsi_enable(struct mtk_dsi *dsi)
+{
+ int ret;
+
+ if (dsi->enabled)
+ return;
+
+ if (dsi->panel) {
+ if (drm_panel_prepare(dsi->panel)) {
+ DRM_ERROR("failed to setup the panel\n");
+ return;
+ }
+ }
+
+ ret = mtk_dsi_poweron(dsi);
+ if (ret < 0) {
+ DRM_ERROR("failed to power on dsi\n");
+ return;
+ }
+
+ dsi_rxtx_control(dsi);
+
+ dsi_clk_ulp_mode_leave(dsi);
+ dsi_lane0_ulp_mode_leave(dsi);
+ dsi_clk_hs_mode(dsi, 0);
+ dsi_set_mode(dsi);
+
+ dsi_ps_control_vact(dsi);
+ dsi_config_vdo_timing(dsi);
+
+ dsi_set_mode(dsi);
+ dsi_clk_hs_mode(dsi, 1);
+
+ mtk_dsi_start(dsi);
+
+ dsi->enabled = true;
+}
+
+static void mtk_output_dsi_disable(struct mtk_dsi *dsi)
+{
+ if (!dsi->enabled)
+ return;
+
+ if (dsi->panel) {
+ if (drm_panel_disable(dsi->panel)) {
+ DRM_ERROR("failed to disable the panel\n");
+ return;
+ }
+ }
+
+ mtk_dsi_poweroff(dsi);
+
+ dsi->enabled = false;
+}
+
+static void mtk_dsi_encoder_destroy(struct drm_encoder *encoder)
+{
+ drm_encoder_cleanup(encoder);
+}
+
+static const struct drm_encoder_funcs mtk_dsi_encoder_funcs = {
+ .destroy = mtk_dsi_encoder_destroy,
+};
+
+static bool mtk_dsi_encoder_mode_fixup(struct drm_encoder *encoder,
+ const struct drm_display_mode *mode,
+ struct drm_display_mode *adjusted_mode)
+{
+ return true;
+}
+
+static void mtk_dsi_encoder_mode_set(struct drm_encoder *encoder,
+ struct drm_display_mode *mode,
+ struct drm_display_mode *adjusted)
+{
+ struct mtk_dsi *dsi = encoder_to_dsi(encoder);
+
+ dsi->vm.pixelclock = adjusted->clock;
+ dsi->vm.hactive = adjusted->hdisplay;
+ dsi->vm.hback_porch = adjusted->htotal - adjusted->hsync_end;
+ dsi->vm.hfront_porch = adjusted->hsync_start - adjusted->hdisplay;
+ dsi->vm.hsync_len = adjusted->hsync_end - adjusted->hsync_start;
+
+ dsi->vm.vactive = adjusted->vdisplay;
+ dsi->vm.vback_porch = adjusted->vtotal - adjusted->vsync_end;
+ dsi->vm.vfront_porch = adjusted->vsync_start - adjusted->vdisplay;
+ dsi->vm.vsync_len = adjusted->vsync_end - adjusted->vsync_start;
+}
+
+static void mtk_dsi_encoder_disable(struct drm_encoder *encoder)
+{
+ struct mtk_dsi *dsi = encoder_to_dsi(encoder);
+
+ mtk_output_dsi_disable(dsi);
+}
+
+static void mtk_dsi_encoder_enable(struct drm_encoder *encoder)
+{
+ struct mtk_dsi *dsi = encoder_to_dsi(encoder);
+
+ mtk_output_dsi_enable(dsi);
+}
+
+static enum drm_connector_status mtk_dsi_connector_detect(
+ struct drm_connector *connector, bool force)
+{
+ return connector_status_connected;
+}
+
+static int mtk_dsi_connector_get_modes(struct drm_connector *connector)
+{
+ struct mtk_dsi *dsi = connector_to_dsi(connector);
+
+ return drm_panel_get_modes(dsi->panel);
+}
+
+static struct drm_encoder *mtk_dsi_connector_best_encoder(
+ struct drm_connector *connector)
+{
+ struct mtk_dsi *dsi = connector_to_dsi(connector);
+
+ return &dsi->encoder;
+}
+
+static const struct drm_encoder_helper_funcs mtk_dsi_encoder_helper_funcs = {
+ .mode_fixup = mtk_dsi_encoder_mode_fixup,
+ .mode_set = mtk_dsi_encoder_mode_set,
+ .disable = mtk_dsi_encoder_disable,
+ .enable = mtk_dsi_encoder_enable,
+};
+
+static const struct drm_connector_funcs mtk_dsi_connector_funcs = {
+ .dpms = drm_atomic_helper_connector_dpms,
+ .detect = mtk_dsi_connector_detect,
+ .fill_modes = drm_helper_probe_single_connector_modes,
+ .destroy = drm_connector_cleanup,
+ .reset = drm_atomic_helper_connector_reset,
+ .atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
+};
+
+static const struct drm_connector_helper_funcs
+ mtk_dsi_connector_helper_funcs = {
+ .get_modes = mtk_dsi_connector_get_modes,
+ .best_encoder = mtk_dsi_connector_best_encoder,
+};
+
+static int mtk_drm_attach_bridge(struct drm_bridge *bridge,
+ struct drm_encoder *encoder)
+{
+ int ret;
+
+ if (!bridge)
+ return -ENOENT;
+
+ encoder->bridge = bridge;
+ bridge->encoder = encoder;
+ ret = drm_bridge_attach(encoder->dev, bridge);
+ if (ret) {
+ DRM_ERROR("Failed to attach bridge to drm\n");
+ encoder->bridge = NULL;
+ bridge->encoder = NULL;
+ }
+
+ return ret;
+}
+
+static int mtk_dsi_create_connector(struct drm_device *drm, struct mtk_dsi *dsi)
+{
+ int ret;
+
+ ret = drm_connector_init(drm, &dsi->conn, &mtk_dsi_connector_funcs,
+ DRM_MODE_CONNECTOR_DSI);
+ if (ret) {
+ DRM_ERROR("Failed to connector init to drm\n");
+ return ret;
+ }
+
+ drm_connector_helper_add(&dsi->conn, &mtk_dsi_connector_helper_funcs);
+
+ dsi->conn.dpms = DRM_MODE_DPMS_OFF;
+ drm_mode_connector_attach_encoder(&dsi->conn, &dsi->encoder);
+
+ if (dsi->panel) {
+ ret = drm_panel_attach(dsi->panel, &dsi->conn);
+ if (ret) {
+ DRM_ERROR("Failed to attach panel to drm\n");
+ goto err_connector_cleanup;
+ }
+ }
+
+ return 0;
+
+err_connector_cleanup:
+ drm_connector_cleanup(&dsi->conn);
+ return ret;
+}
+
+static int mtk_dsi_create_conn_enc(struct drm_device *drm, struct mtk_dsi *dsi)
+{
+ int ret;
+
+ ret = drm_encoder_init(drm, &dsi->encoder, &mtk_dsi_encoder_funcs,
+ DRM_MODE_ENCODER_DSI, NULL);
+ if (ret) {
+ DRM_ERROR("Failed to encoder init to drm\n");
+ return ret;
+ }
+ drm_encoder_helper_add(&dsi->encoder, &mtk_dsi_encoder_helper_funcs);
+
+ /*
+ * Currently display data paths are statically assigned to a crtc each.
+ * crtc 0 is OVL0 -> COLOR0 -> AAL -> OD -> RDMA0 -> UFOE -> DSI0
+ */
+ dsi->encoder.possible_crtcs = 1;
+
+ /* If there's a bridge, attach to it and let it create the connector */
+ ret = mtk_drm_attach_bridge(dsi->bridge, &dsi->encoder);
+ if (ret) {
+ /* Otherwise create our own connector and attach to a panel */
+ ret = mtk_dsi_create_connector(drm, dsi);
+ if (ret)
+ goto err_encoder_cleanup;
+ }
+
+ return 0;
+
+err_encoder_cleanup:
+ drm_encoder_cleanup(&dsi->encoder);
+ return ret;
+}
+
+static void mtk_dsi_destroy_conn_enc(struct mtk_dsi *dsi)
+{
+ drm_encoder_cleanup(&dsi->encoder);
+ /* Skip connector cleanup if creation was delegated to the bridge */
+ if (dsi->conn.dev) {
+ drm_connector_unregister(&dsi->conn);
+ drm_connector_cleanup(&dsi->conn);
+ }
+}
+
+static void mtk_dsi_ddp_start(struct mtk_ddp_comp *comp)
+{
+ struct mtk_dsi *dsi = container_of(comp, struct mtk_dsi, ddp_comp);
+
+ mtk_dsi_poweron(dsi);
+}
+
+static void mtk_dsi_ddp_stop(struct mtk_ddp_comp *comp)
+{
+ struct mtk_dsi *dsi = container_of(comp, struct mtk_dsi, ddp_comp);
+
+ mtk_dsi_poweroff(dsi);
+}
+
+static const struct mtk_ddp_comp_funcs mtk_dsi_funcs = {
+ .start = mtk_dsi_ddp_start,
+ .stop = mtk_dsi_ddp_stop,
+};
+
+static int mtk_dsi_host_attach(struct mipi_dsi_host *host,
+ struct mipi_dsi_device *device)
+{
+ struct mtk_dsi *dsi = host_to_dsi(host);
+
+ dsi->lanes = device->lanes;
+ dsi->format = device->format;
+ dsi->mode_flags = device->mode_flags;
+
+ if (dsi->conn.dev)
+ drm_helper_hpd_irq_event(dsi->conn.dev);
+
+ return 0;
+}
+
+static int mtk_dsi_host_detach(struct mipi_dsi_host *host,
+ struct mipi_dsi_device *device)
+{
+ struct mtk_dsi *dsi = host_to_dsi(host);
+
+ if (dsi->conn.dev)
+ drm_helper_hpd_irq_event(dsi->conn.dev);
+
+ return 0;
+}
+
+static const struct mipi_dsi_host_ops mtk_dsi_ops = {
+ .attach = mtk_dsi_host_attach,
+ .detach = mtk_dsi_host_detach,
+};
+
+static int mtk_dsi_bind(struct device *dev, struct device *master, void *data)
+{
+ int ret;
+ struct drm_device *drm = data;
+ struct mtk_dsi *dsi = dev_get_drvdata(dev);
+
+ ret = mtk_ddp_comp_register(drm, &dsi->ddp_comp);
+ if (ret < 0) {
+ dev_err(dev, "Failed to register component %s: %d\n",
+ dev->of_node->full_name, ret);
+ return ret;
+ }
+
+ ret = mipi_dsi_host_register(&dsi->host);
+ if (ret < 0) {
+ dev_err(dev, "failed to register DSI host: %d\n", ret);
+ goto err_ddp_comp_unregister;
+ }
+
+ ret = mtk_dsi_create_conn_enc(drm, dsi);
+ if (ret) {
+ DRM_ERROR("Encoder create failed with %d\n", ret);
+ goto err_unregister;
+ }
+
+ return 0;
+
+err_unregister:
+ mipi_dsi_host_unregister(&dsi->host);
+err_ddp_comp_unregister:
+ mtk_ddp_comp_unregister(drm, &dsi->ddp_comp);
+ return ret;
+}
+
+static void mtk_dsi_unbind(struct device *dev, struct device *master,
+ void *data)
+{
+ struct drm_device *drm = data;
+ struct mtk_dsi *dsi = dev_get_drvdata(dev);
+
+ mtk_dsi_destroy_conn_enc(dsi);
+ mipi_dsi_host_unregister(&dsi->host);
+ mtk_ddp_comp_unregister(drm, &dsi->ddp_comp);
+}
+
+static const struct component_ops mtk_dsi_component_ops = {
+ .bind = mtk_dsi_bind,
+ .unbind = mtk_dsi_unbind,
+};
+
+static int mtk_dsi_probe(struct platform_device *pdev)
+{
+ struct mtk_dsi *dsi;
+ struct device *dev = &pdev->dev;
+ struct device_node *remote_node, *endpoint;
+ struct resource *regs;
+ int comp_id;
+ int ret;
+
+ dsi = devm_kzalloc(dev, sizeof(*dsi), GFP_KERNEL);
+ if (!dsi)
+ return -ENOMEM;
+
+ dsi->host.ops = &mtk_dsi_ops;
+ dsi->host.dev = dev;
+
+ endpoint = of_graph_get_next_endpoint(dev->of_node, NULL);
+ if (endpoint) {
+ remote_node = of_graph_get_remote_port_parent(endpoint);
+ if (!remote_node) {
+ dev_err(dev, "No panel connected\n");
+ return -ENODEV;
+ }
+
+ dsi->bridge = of_drm_find_bridge(remote_node);
+ dsi->panel = of_drm_find_panel(remote_node);
+ of_node_put(remote_node);
+ if (!dsi->bridge && !dsi->panel) {
+ dev_info(dev, "Waiting for bridge or panel driver\n");
+ return -EPROBE_DEFER;
+ }
+ }
+
+ dsi->engine_clk = devm_clk_get(dev, "engine");
+ if (IS_ERR(dsi->engine_clk)) {
+ ret = PTR_ERR(dsi->engine_clk);
+ dev_err(dev, "Failed to get engine clock: %d\n", ret);
+ return ret;
+ }
+
+ dsi->digital_clk = devm_clk_get(dev, "digital");
+ if (IS_ERR(dsi->digital_clk)) {
+ ret = PTR_ERR(dsi->digital_clk);
+ dev_err(dev, "Failed to get digital clock: %d\n", ret);
+ return ret;
+ }
+
+ dsi->hs_clk = devm_clk_get(dev, "hs");
+ if (IS_ERR(dsi->hs_clk)) {
+ ret = PTR_ERR(dsi->hs_clk);
+ dev_err(dev, "Failed to get hs clock: %d\n", ret);
+ return ret;
+ }
+
+ regs = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ dsi->regs = devm_ioremap_resource(dev, regs);
+ if (IS_ERR(dsi->regs)) {
+ ret = PTR_ERR(dsi->regs);
+ dev_err(dev, "Failed to ioremap memory: %d\n", ret);
+ return ret;
+ }
+
+ dsi->phy = devm_phy_get(dev, "dphy");
+ if (IS_ERR(dsi->phy)) {
+ ret = PTR_ERR(dsi->phy);
+ dev_err(dev, "Failed to get MIPI-DPHY: %d\n", ret);
+ return ret;
+ }
+
+ comp_id = mtk_ddp_comp_get_id(dev->of_node, MTK_DSI);
+ if (comp_id < 0) {
+ dev_err(dev, "Failed to identify by alias: %d\n", comp_id);
+ return comp_id;
+ }
+
+ ret = mtk_ddp_comp_init(dev, dev->of_node, &dsi->ddp_comp, comp_id,
+ &mtk_dsi_funcs);
+ if (ret) {
+ dev_err(dev, "Failed to initialize component: %d\n", ret);
+ return ret;
+ }
+
+ platform_set_drvdata(pdev, dsi);
+
+ return component_add(&pdev->dev, &mtk_dsi_component_ops);
+}
+
+static int mtk_dsi_remove(struct platform_device *pdev)
+{
+ struct mtk_dsi *dsi = platform_get_drvdata(pdev);
+
+ mtk_output_dsi_disable(dsi);
+ component_del(&pdev->dev, &mtk_dsi_component_ops);
+
+ return 0;
+}
+
+static const struct of_device_id mtk_dsi_of_match[] = {
+ { .compatible = "mediatek,mt8173-dsi" },
+ { },
+};
+
+struct platform_driver mtk_dsi_driver = {
+ .probe = mtk_dsi_probe,
+ .remove = mtk_dsi_remove,
+ .driver = {
+ .name = "mtk-dsi",
+ .of_match_table = mtk_dsi_of_match,
+ },
+};
diff --git a/drivers/gpu/drm/mediatek/mtk_mipi_tx.c b/drivers/gpu/drm/mediatek/mtk_mipi_tx.c
new file mode 100644
index 0000000..cf8f38d
--- /dev/null
+++ b/drivers/gpu/drm/mediatek/mtk_mipi_tx.c
@@ -0,0 +1,463 @@
+/*
+ * Copyright (c) 2015 MediaTek Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/clk.h>
+#include <linux/clk-provider.h>
+#include <linux/delay.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/phy/phy.h>
+
+#define MIPITX_DSI_CON 0x00
+#define RG_DSI_LDOCORE_EN BIT(0)
+#define RG_DSI_CKG_LDOOUT_EN BIT(1)
+#define RG_DSI_BCLK_SEL (3 << 2)
+#define RG_DSI_LD_IDX_SEL (7 << 4)
+#define RG_DSI_PHYCLK_SEL (2 << 8)
+#define RG_DSI_DSICLK_FREQ_SEL BIT(10)
+#define RG_DSI_LPTX_CLMP_EN BIT(11)
+
+#define MIPITX_DSI_CLOCK_LANE 0x04
+#define MIPITX_DSI_DATA_LANE0 0x08
+#define MIPITX_DSI_DATA_LANE1 0x0c
+#define MIPITX_DSI_DATA_LANE2 0x10
+#define MIPITX_DSI_DATA_LANE3 0x14
+#define RG_DSI_LNTx_LDOOUT_EN BIT(0)
+#define RG_DSI_LNTx_CKLANE_EN BIT(1)
+#define RG_DSI_LNTx_LPTX_IPLUS1 BIT(2)
+#define RG_DSI_LNTx_LPTX_IPLUS2 BIT(3)
+#define RG_DSI_LNTx_LPTX_IMINUS BIT(4)
+#define RG_DSI_LNTx_LPCD_IPLUS BIT(5)
+#define RG_DSI_LNTx_LPCD_IMINUS BIT(6)
+#define RG_DSI_LNTx_RT_CODE (0xf << 8)
+
+#define MIPITX_DSI_TOP_CON 0x40
+#define RG_DSI_LNT_INTR_EN BIT(0)
+#define RG_DSI_LNT_HS_BIAS_EN BIT(1)
+#define RG_DSI_LNT_IMP_CAL_EN BIT(2)
+#define RG_DSI_LNT_TESTMODE_EN BIT(3)
+#define RG_DSI_LNT_IMP_CAL_CODE (0xf << 4)
+#define RG_DSI_LNT_AIO_SEL (7 << 8)
+#define RG_DSI_PAD_TIE_LOW_EN BIT(11)
+#define RG_DSI_DEBUG_INPUT_EN BIT(12)
+#define RG_DSI_PRESERVE (7 << 13)
+
+#define MIPITX_DSI_BG_CON 0x44
+#define RG_DSI_BG_CORE_EN BIT(0)
+#define RG_DSI_BG_CKEN BIT(1)
+#define RG_DSI_BG_DIV (0x3 << 2)
+#define RG_DSI_BG_FAST_CHARGE BIT(4)
+#define RG_DSI_VOUT_MSK (0x3ffff << 5)
+#define RG_DSI_V12_SEL (7 << 5)
+#define RG_DSI_V10_SEL (7 << 8)
+#define RG_DSI_V072_SEL (7 << 11)
+#define RG_DSI_V04_SEL (7 << 14)
+#define RG_DSI_V032_SEL (7 << 17)
+#define RG_DSI_V02_SEL (7 << 20)
+#define RG_DSI_BG_R1_TRIM (0xf << 24)
+#define RG_DSI_BG_R2_TRIM (0xf << 28)
+
+#define MIPITX_DSI_PLL_CON0 0x50
+#define RG_DSI_MPPLL_PLL_EN BIT(0)
+#define RG_DSI_MPPLL_DIV_MSK (0x1ff << 1)
+#define RG_DSI_MPPLL_PREDIV (3 << 1)
+#define RG_DSI_MPPLL_TXDIV0 (3 << 3)
+#define RG_DSI_MPPLL_TXDIV1 (3 << 5)
+#define RG_DSI_MPPLL_POSDIV (7 << 7)
+#define RG_DSI_MPPLL_MONVC_EN BIT(10)
+#define RG_DSI_MPPLL_MONREF_EN BIT(11)
+#define RG_DSI_MPPLL_VOD_EN BIT(12)
+
+#define MIPITX_DSI_PLL_CON1 0x54
+#define RG_DSI_MPPLL_SDM_FRA_EN BIT(0)
+#define RG_DSI_MPPLL_SDM_SSC_PH_INIT BIT(1)
+#define RG_DSI_MPPLL_SDM_SSC_EN BIT(2)
+#define RG_DSI_MPPLL_SDM_SSC_PRD (0xffff << 16)
+
+#define MIPITX_DSI_PLL_CON2 0x58
+
+#define MIPITX_DSI_PLL_PWR 0x68
+#define RG_DSI_MPPLL_SDM_PWR_ON BIT(0)
+#define RG_DSI_MPPLL_SDM_ISO_EN BIT(1)
+#define RG_DSI_MPPLL_SDM_PWR_ACK BIT(8)
+
+#define MIPITX_DSI_SW_CTRL 0x80
+#define SW_CTRL_EN BIT(0)
+
+#define MIPITX_DSI_SW_CTRL_CON0 0x84
+#define SW_LNTC_LPTX_PRE_OE BIT(0)
+#define SW_LNTC_LPTX_OE BIT(1)
+#define SW_LNTC_LPTX_P BIT(2)
+#define SW_LNTC_LPTX_N BIT(3)
+#define SW_LNTC_HSTX_PRE_OE BIT(4)
+#define SW_LNTC_HSTX_OE BIT(5)
+#define SW_LNTC_HSTX_ZEROCLK BIT(6)
+#define SW_LNT0_LPTX_PRE_OE BIT(7)
+#define SW_LNT0_LPTX_OE BIT(8)
+#define SW_LNT0_LPTX_P BIT(9)
+#define SW_LNT0_LPTX_N BIT(10)
+#define SW_LNT0_HSTX_PRE_OE BIT(11)
+#define SW_LNT0_HSTX_OE BIT(12)
+#define SW_LNT0_LPRX_EN BIT(13)
+#define SW_LNT1_LPTX_PRE_OE BIT(14)
+#define SW_LNT1_LPTX_OE BIT(15)
+#define SW_LNT1_LPTX_P BIT(16)
+#define SW_LNT1_LPTX_N BIT(17)
+#define SW_LNT1_HSTX_PRE_OE BIT(18)
+#define SW_LNT1_HSTX_OE BIT(19)
+#define SW_LNT2_LPTX_PRE_OE BIT(20)
+#define SW_LNT2_LPTX_OE BIT(21)
+#define SW_LNT2_LPTX_P BIT(22)
+#define SW_LNT2_LPTX_N BIT(23)
+#define SW_LNT2_HSTX_PRE_OE BIT(24)
+#define SW_LNT2_HSTX_OE BIT(25)
+
+struct mtk_mipi_tx {
+ struct device *dev;
+ void __iomem *regs;
+ unsigned int data_rate;
+ struct clk_hw pll_hw;
+ struct clk *pll;
+};
+
+static inline struct mtk_mipi_tx *mtk_mipi_tx_from_clk_hw(struct clk_hw *hw)
+{
+ return container_of(hw, struct mtk_mipi_tx, pll_hw);
+}
+
+static void mtk_mipi_tx_clear_bits(struct mtk_mipi_tx *mipi_tx, u32 offset,
+ u32 bits)
+{
+ u32 temp = readl(mipi_tx->regs + offset);
+
+ writel(temp & ~bits, mipi_tx->regs + offset);
+}
+
+static void mtk_mipi_tx_set_bits(struct mtk_mipi_tx *mipi_tx, u32 offset,
+ u32 bits)
+{
+ u32 temp = readl(mipi_tx->regs + offset);
+
+ writel(temp | bits, mipi_tx->regs + offset);
+}
+
+static void mtk_mipi_tx_update_bits(struct mtk_mipi_tx *mipi_tx, u32 offset,
+ u32 mask, u32 data)
+{
+ u32 temp = readl(mipi_tx->regs + offset);
+
+ writel((temp & ~mask) | (data & mask), mipi_tx->regs + offset);
+}
+
+static int mtk_mipi_tx_pll_prepare(struct clk_hw *hw)
+{
+ struct mtk_mipi_tx *mipi_tx = mtk_mipi_tx_from_clk_hw(hw);
+ unsigned int txdiv, txdiv0, txdiv1;
+ u64 pcw;
+
+ dev_dbg(mipi_tx->dev, "prepare: %u Hz\n", mipi_tx->data_rate);
+
+ if (mipi_tx->data_rate >= 500000000) {
+ txdiv = 1;
+ txdiv0 = 0;
+ txdiv1 = 0;
+ } else if (mipi_tx->data_rate >= 250000000) {
+ txdiv = 2;
+ txdiv0 = 1;
+ txdiv1 = 0;
+ } else if (mipi_tx->data_rate >= 125000000) {
+ txdiv = 4;
+ txdiv0 = 2;
+ txdiv1 = 0;
+ } else if (mipi_tx->data_rate > 62000000) {
+ txdiv = 8;
+ txdiv0 = 2;
+ txdiv1 = 1;
+ } else if (mipi_tx->data_rate >= 50000000) {
+ txdiv = 16;
+ txdiv0 = 2;
+ txdiv1 = 2;
+ } else {
+ return -EINVAL;
+ }
+
+ mtk_mipi_tx_update_bits(mipi_tx, MIPITX_DSI_BG_CON,
+ RG_DSI_VOUT_MSK |
+ RG_DSI_BG_CKEN | RG_DSI_BG_CORE_EN,
+ (4 << 20) | (4 << 17) | (4 << 14) |
+ (4 << 11) | (4 << 8) | (4 << 5) |
+ RG_DSI_BG_CKEN | RG_DSI_BG_CORE_EN);
+
+ usleep_range(30, 100);
+
+ mtk_mipi_tx_update_bits(mipi_tx, MIPITX_DSI_TOP_CON,
+ RG_DSI_LNT_IMP_CAL_CODE | RG_DSI_LNT_HS_BIAS_EN,
+ (8 << 4) | RG_DSI_LNT_HS_BIAS_EN);
+
+ mtk_mipi_tx_set_bits(mipi_tx, MIPITX_DSI_CON,
+ RG_DSI_CKG_LDOOUT_EN | RG_DSI_LDOCORE_EN);
+
+ mtk_mipi_tx_update_bits(mipi_tx, MIPITX_DSI_PLL_PWR,
+ RG_DSI_MPPLL_SDM_PWR_ON |
+ RG_DSI_MPPLL_SDM_ISO_EN,
+ RG_DSI_MPPLL_SDM_PWR_ON);
+
+ mtk_mipi_tx_clear_bits(mipi_tx, MIPITX_DSI_PLL_CON0,
+ RG_DSI_MPPLL_PLL_EN);
+
+ mtk_mipi_tx_update_bits(mipi_tx, MIPITX_DSI_PLL_CON0,
+ RG_DSI_MPPLL_TXDIV0 | RG_DSI_MPPLL_TXDIV1 |
+ RG_DSI_MPPLL_PREDIV,
+ (txdiv0 << 3) | (txdiv1 << 5));
+
+ /*
+ * PLL PCW config
+ * PCW bit 24~30 = integer part of pcw
+ * PCW bit 0~23 = fractional part of pcw
+ * pcw = data_Rate*4*txdiv/(Ref_clk*2);
+ * Post DIV =4, so need data_Rate*4
+ * Ref_clk is 26MHz
+ */
+ pcw = div_u64(((u64)mipi_tx->data_rate * 2 * txdiv) << 24,
+ 26000000);
+ writel(pcw, mipi_tx->regs + MIPITX_DSI_PLL_CON2);
+
+ mtk_mipi_tx_set_bits(mipi_tx, MIPITX_DSI_PLL_CON1,
+ RG_DSI_MPPLL_SDM_FRA_EN);
+
+ mtk_mipi_tx_set_bits(mipi_tx, MIPITX_DSI_PLL_CON0, RG_DSI_MPPLL_PLL_EN);
+
+ usleep_range(20, 100);
+
+ mtk_mipi_tx_clear_bits(mipi_tx, MIPITX_DSI_PLL_CON1,
+ RG_DSI_MPPLL_SDM_SSC_EN);
+
+ return 0;
+}
+
+static void mtk_mipi_tx_pll_unprepare(struct clk_hw *hw)
+{
+ struct mtk_mipi_tx *mipi_tx = mtk_mipi_tx_from_clk_hw(hw);
+
+ dev_dbg(mipi_tx->dev, "unprepare\n");
+
+ mtk_mipi_tx_clear_bits(mipi_tx, MIPITX_DSI_PLL_CON0,
+ RG_DSI_MPPLL_PLL_EN);
+
+ mtk_mipi_tx_update_bits(mipi_tx, MIPITX_DSI_PLL_PWR,
+ RG_DSI_MPPLL_SDM_ISO_EN |
+ RG_DSI_MPPLL_SDM_PWR_ON,
+ RG_DSI_MPPLL_SDM_ISO_EN);
+
+ mtk_mipi_tx_clear_bits(mipi_tx, MIPITX_DSI_TOP_CON,
+ RG_DSI_LNT_HS_BIAS_EN);
+
+ mtk_mipi_tx_clear_bits(mipi_tx, MIPITX_DSI_CON,
+ RG_DSI_CKG_LDOOUT_EN | RG_DSI_LDOCORE_EN);
+
+ mtk_mipi_tx_clear_bits(mipi_tx, MIPITX_DSI_BG_CON,
+ RG_DSI_BG_CKEN | RG_DSI_BG_CORE_EN);
+
+ mtk_mipi_tx_clear_bits(mipi_tx, MIPITX_DSI_PLL_CON0,
+ RG_DSI_MPPLL_DIV_MSK);
+}
+
+static long mtk_mipi_tx_pll_round_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long *prate)
+{
+ return clamp_val(rate, 50000000, 1250000000);
+}
+
+static int mtk_mipi_tx_pll_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct mtk_mipi_tx *mipi_tx = mtk_mipi_tx_from_clk_hw(hw);
+
+ dev_dbg(mipi_tx->dev, "set rate: %lu Hz\n", rate);
+
+ mipi_tx->data_rate = rate;
+
+ return 0;
+}
+
+static unsigned long mtk_mipi_tx_pll_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct mtk_mipi_tx *mipi_tx = mtk_mipi_tx_from_clk_hw(hw);
+
+ return mipi_tx->data_rate;
+}
+
+static const struct clk_ops mtk_mipi_tx_pll_ops = {
+ .prepare = mtk_mipi_tx_pll_prepare,
+ .unprepare = mtk_mipi_tx_pll_unprepare,
+ .round_rate = mtk_mipi_tx_pll_round_rate,
+ .set_rate = mtk_mipi_tx_pll_set_rate,
+ .recalc_rate = mtk_mipi_tx_pll_recalc_rate,
+};
+
+static int mtk_mipi_tx_power_on_signal(struct phy *phy)
+{
+ struct mtk_mipi_tx *mipi_tx = phy_get_drvdata(phy);
+ unsigned int reg;
+
+ for (reg = MIPITX_DSI_CLOCK_LANE;
+ reg <= MIPITX_DSI_DATA_LANE3; reg += 4)
+ mtk_mipi_tx_set_bits(mipi_tx, reg, RG_DSI_LNTx_LDOOUT_EN);
+
+ mtk_mipi_tx_clear_bits(mipi_tx, MIPITX_DSI_TOP_CON,
+ RG_DSI_PAD_TIE_LOW_EN);
+
+ return 0;
+}
+
+static int mtk_mipi_tx_power_on(struct phy *phy)
+{
+ struct mtk_mipi_tx *mipi_tx = phy_get_drvdata(phy);
+ int ret;
+
+ /* Power up core and enable PLL */
+ ret = clk_prepare_enable(mipi_tx->pll);
+ if (ret < 0)
+ return ret;
+
+ /* Enable DSI Lane LDO outputs, disable pad tie low */
+ mtk_mipi_tx_power_on_signal(phy);
+
+ return 0;
+}
+
+static void mtk_mipi_tx_power_off_signal(struct phy *phy)
+{
+ struct mtk_mipi_tx *mipi_tx = phy_get_drvdata(phy);
+ unsigned int reg;
+
+ mtk_mipi_tx_set_bits(mipi_tx, MIPITX_DSI_TOP_CON,
+ RG_DSI_PAD_TIE_LOW_EN);
+
+ for (reg = MIPITX_DSI_CLOCK_LANE;
+ reg <= MIPITX_DSI_DATA_LANE3; reg += 4)
+ mtk_mipi_tx_clear_bits(mipi_tx, reg, RG_DSI_LNTx_LDOOUT_EN);
+}
+
+static int mtk_mipi_tx_power_off(struct phy *phy)
+{
+ struct mtk_mipi_tx *mipi_tx = phy_get_drvdata(phy);
+
+ /* Enable pad tie low, disable DSI Lane LDO outputs */
+ mtk_mipi_tx_power_off_signal(phy);
+
+ /* Disable PLL and power down core */
+ clk_disable_unprepare(mipi_tx->pll);
+
+ return 0;
+}
+
+static const struct phy_ops mtk_mipi_tx_ops = {
+ .power_on = mtk_mipi_tx_power_on,
+ .power_off = mtk_mipi_tx_power_off,
+ .owner = THIS_MODULE,
+};
+
+static int mtk_mipi_tx_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct mtk_mipi_tx *mipi_tx;
+ struct resource *mem;
+ struct clk *ref_clk;
+ const char *ref_clk_name;
+ struct clk_init_data clk_init = {
+ .ops = &mtk_mipi_tx_pll_ops,
+ .num_parents = 1,
+ .parent_names = (const char * const *)&ref_clk_name,
+ .flags = CLK_SET_RATE_GATE,
+ };
+ struct phy *phy;
+ struct phy_provider *phy_provider;
+ int ret;
+
+ mipi_tx = devm_kzalloc(dev, sizeof(*mipi_tx), GFP_KERNEL);
+ if (!mipi_tx)
+ return -ENOMEM;
+
+ mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ mipi_tx->regs = devm_ioremap_resource(dev, mem);
+ if (IS_ERR(mipi_tx->regs)) {
+ ret = PTR_ERR(mipi_tx->regs);
+ dev_err(dev, "Failed to get memory resource: %d\n", ret);
+ return ret;
+ }
+
+ ref_clk = devm_clk_get(dev, NULL);
+ if (IS_ERR(ref_clk)) {
+ ret = PTR_ERR(ref_clk);
+ dev_err(dev, "Failed to get reference clock: %d\n", ret);
+ return ret;
+ }
+ ref_clk_name = __clk_get_name(ref_clk);
+
+ ret = of_property_read_string(dev->of_node, "clock-output-names",
+ &clk_init.name);
+ if (ret < 0) {
+ dev_err(dev, "Failed to read clock-output-names: %d\n", ret);
+ return ret;
+ }
+
+ mipi_tx->pll_hw.init = &clk_init;
+ mipi_tx->pll = devm_clk_register(dev, &mipi_tx->pll_hw);
+ if (IS_ERR(mipi_tx->pll)) {
+ ret = PTR_ERR(mipi_tx->pll);
+ dev_err(dev, "Failed to register PLL: %d\n", ret);
+ return ret;
+ }
+
+ phy = devm_phy_create(dev, NULL, &mtk_mipi_tx_ops);
+ if (IS_ERR(phy)) {
+ ret = PTR_ERR(phy);
+ dev_err(dev, "Failed to create MIPI D-PHY: %d\n", ret);
+ return ret;
+ }
+ phy_set_drvdata(phy, mipi_tx);
+
+ phy_provider = devm_of_phy_provider_register(dev, of_phy_simple_xlate);
+ if (IS_ERR(phy)) {
+ ret = PTR_ERR(phy_provider);
+ return ret;
+ }
+
+ mipi_tx->dev = dev;
+
+ return of_clk_add_provider(dev->of_node, of_clk_src_simple_get,
+ mipi_tx->pll);
+}
+
+static int mtk_mipi_tx_remove(struct platform_device *pdev)
+{
+ of_clk_del_provider(pdev->dev.of_node);
+ return 0;
+}
+
+static const struct of_device_id mtk_mipi_tx_match[] = {
+ { .compatible = "mediatek,mt8173-mipi-tx", },
+ {},
+};
+
+struct platform_driver mtk_mipi_tx_driver = {
+ .probe = mtk_mipi_tx_probe,
+ .remove = mtk_mipi_tx_remove,
+ .driver = {
+ .name = "mediatek-mipi-tx",
+ .of_match_table = mtk_mipi_tx_match,
+ },
+};
diff --git a/drivers/gpu/drm/mgag200/mgag200_cursor.c b/drivers/gpu/drm/mgag200/mgag200_cursor.c
index a7bf6a9..2ac3fcb 100644
--- a/drivers/gpu/drm/mgag200/mgag200_cursor.c
+++ b/drivers/gpu/drm/mgag200/mgag200_cursor.c
@@ -75,7 +75,7 @@
return 0;
}
- obj = drm_gem_object_lookup(dev, file_priv, handle);
+ obj = drm_gem_object_lookup(file_priv, handle);
if (!obj)
return -ENOENT;
diff --git a/drivers/gpu/drm/mgag200/mgag200_drv.c b/drivers/gpu/drm/mgag200/mgag200_drv.c
index b0af774..ebb470f 100644
--- a/drivers/gpu/drm/mgag200/mgag200_drv.c
+++ b/drivers/gpu/drm/mgag200/mgag200_drv.c
@@ -116,10 +116,8 @@
static int __init mgag200_init(void)
{
-#ifdef CONFIG_VGA_CONSOLE
if (vgacon_text_force() && mgag200_modeset == -1)
return -EINVAL;
-#endif
if (mgag200_modeset == 0)
return -EINVAL;
diff --git a/drivers/gpu/drm/mgag200/mgag200_drv.h b/drivers/gpu/drm/mgag200/mgag200_drv.h
index 205b280..3e02ac2 100644
--- a/drivers/gpu/drm/mgag200/mgag200_drv.h
+++ b/drivers/gpu/drm/mgag200/mgag200_drv.h
@@ -281,7 +281,7 @@
{
int ret;
- ret = ttm_bo_reserve(&bo->bo, true, no_wait, false, NULL);
+ ret = ttm_bo_reserve(&bo->bo, true, no_wait, NULL);
if (ret) {
if (ret != -ERESTARTSYS && ret != -EBUSY)
DRM_ERROR("reserve failed %p\n", bo);
diff --git a/drivers/gpu/drm/mgag200/mgag200_main.c b/drivers/gpu/drm/mgag200/mgag200_main.c
index 9147444..615cbb0 100644
--- a/drivers/gpu/drm/mgag200/mgag200_main.c
+++ b/drivers/gpu/drm/mgag200/mgag200_main.c
@@ -53,7 +53,7 @@
struct mga_framebuffer *mga_fb;
int ret;
- obj = drm_gem_object_lookup(dev, filp, mode_cmd->handles[0]);
+ obj = drm_gem_object_lookup(filp, mode_cmd->handles[0]);
if (obj == NULL)
return ERR_PTR(-ENOENT);
@@ -358,7 +358,7 @@
struct drm_gem_object *obj;
struct mgag200_bo *bo;
- obj = drm_gem_object_lookup(dev, file, handle);
+ obj = drm_gem_object_lookup(file, handle);
if (obj == NULL)
return -ENOENT;
diff --git a/drivers/gpu/drm/mgag200/mgag200_ttm.c b/drivers/gpu/drm/mgag200/mgag200_ttm.c
index 05108b5..9d5083d 100644
--- a/drivers/gpu/drm/mgag200/mgag200_ttm.c
+++ b/drivers/gpu/drm/mgag200/mgag200_ttm.c
@@ -245,6 +245,8 @@
.verify_access = mgag200_bo_verify_access,
.io_mem_reserve = &mgag200_ttm_io_mem_reserve,
.io_mem_free = &mgag200_ttm_io_mem_free,
+ .lru_tail = &ttm_bo_default_lru_tail,
+ .swap_lru_tail = &ttm_bo_default_swap_lru_tail,
};
int mgag200_mm_init(struct mga_device *mdev)
diff --git a/drivers/gpu/drm/msm/Kconfig b/drivers/gpu/drm/msm/Kconfig
index 215495c..167a497 100644
--- a/drivers/gpu/drm/msm/Kconfig
+++ b/drivers/gpu/drm/msm/Kconfig
@@ -23,6 +23,13 @@
that can be parsed by envytools demsm tool. If enabled, register
logging can be switched on via msm.reglog=y module param.
+config DRM_MSM_HDMI_HDCP
+ bool "Enable HDMI HDCP support in MSM DRM driver"
+ depends on DRM_MSM && QCOM_SCM
+ default y
+ help
+ Choose this option to enable HDCP state machine
+
config DRM_MSM_DSI
bool "Enable DSI support in MSM DRM driver"
depends on DRM_MSM
diff --git a/drivers/gpu/drm/msm/Makefile b/drivers/gpu/drm/msm/Makefile
index ddb4c9d..60cb026 100644
--- a/drivers/gpu/drm/msm/Makefile
+++ b/drivers/gpu/drm/msm/Makefile
@@ -10,7 +10,6 @@
hdmi/hdmi_audio.o \
hdmi/hdmi_bridge.o \
hdmi/hdmi_connector.o \
- hdmi/hdmi_hdcp.o \
hdmi/hdmi_i2c.o \
hdmi/hdmi_phy.o \
hdmi/hdmi_phy_8960.o \
@@ -40,8 +39,10 @@
mdp/mdp5/mdp5_plane.o \
mdp/mdp5/mdp5_smp.o \
msm_atomic.o \
+ msm_debugfs.o \
msm_drv.o \
msm_fb.o \
+ msm_fence.o \
msm_gem.o \
msm_gem_prime.o \
msm_gem_submit.o \
@@ -56,6 +57,8 @@
msm-$(CONFIG_COMMON_CLK) += hdmi/hdmi_pll_8960.o
msm-$(CONFIG_COMMON_CLK) += hdmi/hdmi_phy_8996.o
+msm-$(CONFIG_DRM_MSM_HDMI_HDCP) += hdmi/hdmi_hdcp.o
+
msm-$(CONFIG_DRM_MSM_DSI) += dsi/dsi.o \
mdp/mdp4/mdp4_dsi_encoder.o \
dsi/dsi_cfg.o \
diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.c b/drivers/gpu/drm/msm/adreno/adreno_gpu.c
index 4951172..fbe304e 100644
--- a/drivers/gpu/drm/msm/adreno/adreno_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.c
@@ -120,8 +120,8 @@
/* reset ringbuffer: */
gpu->rb->cur = gpu->rb->start;
- /* reset completed fence seqno, just discard anything pending: */
- adreno_gpu->memptrs->fence = gpu->submitted_fence;
+ /* reset completed fence seqno: */
+ adreno_gpu->memptrs->fence = gpu->fctx->completed_fence;
adreno_gpu->memptrs->rptr = 0;
adreno_gpu->memptrs->wptr = 0;
@@ -133,7 +133,7 @@
}
}
-int adreno_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit,
+void adreno_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit,
struct msm_file_private *ctx)
{
struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
@@ -168,7 +168,7 @@
OUT_PKT2(ring);
OUT_PKT0(ring, REG_AXXX_CP_SCRATCH_REG2, 1);
- OUT_RING(ring, submit->fence);
+ OUT_RING(ring, submit->fence->seqno);
if (adreno_is_a3xx(adreno_gpu) || adreno_is_a4xx(adreno_gpu)) {
/* Flush HLSQ lazy updates to make sure there is nothing
@@ -185,7 +185,7 @@
OUT_PKT3(ring, CP_EVENT_WRITE, 3);
OUT_RING(ring, CACHE_FLUSH_TS);
OUT_RING(ring, rbmemptr(adreno_gpu, fence));
- OUT_RING(ring, submit->fence);
+ OUT_RING(ring, submit->fence->seqno);
/* we could maybe be clever and only CP_COND_EXEC the interrupt: */
OUT_PKT3(ring, CP_INTERRUPT, 1);
@@ -212,8 +212,6 @@
#endif
gpu->funcs->flush(gpu);
-
- return 0;
}
void adreno_flush(struct msm_gpu *gpu)
@@ -254,7 +252,7 @@
adreno_gpu->rev.patchid);
seq_printf(m, "fence: %d/%d\n", adreno_gpu->memptrs->fence,
- gpu->submitted_fence);
+ gpu->fctx->last_fence);
seq_printf(m, "rptr: %d\n", get_rptr(adreno_gpu));
seq_printf(m, "wptr: %d\n", adreno_gpu->memptrs->wptr);
seq_printf(m, "rb wptr: %d\n", get_wptr(gpu->rb));
@@ -295,7 +293,7 @@
adreno_gpu->rev.patchid);
printk("fence: %d/%d\n", adreno_gpu->memptrs->fence,
- gpu->submitted_fence);
+ gpu->fctx->last_fence);
printk("rptr: %d\n", get_rptr(adreno_gpu));
printk("wptr: %d\n", adreno_gpu->memptrs->wptr);
printk("rb wptr: %d\n", get_wptr(gpu->rb));
diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.h b/drivers/gpu/drm/msm/adreno/adreno_gpu.h
index 1d07511..a54f6e0 100644
--- a/drivers/gpu/drm/msm/adreno/adreno_gpu.h
+++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.h
@@ -238,7 +238,7 @@
int adreno_hw_init(struct msm_gpu *gpu);
uint32_t adreno_last_fence(struct msm_gpu *gpu);
void adreno_recover(struct msm_gpu *gpu);
-int adreno_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit,
+void adreno_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit,
struct msm_file_private *ctx);
void adreno_flush(struct msm_gpu *gpu);
void adreno_idle(struct msm_gpu *gpu);
diff --git a/drivers/gpu/drm/msm/dsi/dsi.h b/drivers/gpu/drm/msm/dsi/dsi.h
index 749fbb2..03f115f 100644
--- a/drivers/gpu/drm/msm/dsi/dsi.h
+++ b/drivers/gpu/drm/msm/dsi/dsi.h
@@ -41,8 +41,6 @@
/* Regulators for DSI devices */
struct dsi_reg_entry {
char name[32];
- int min_voltage;
- int max_voltage;
int enable_load;
int disable_load;
};
diff --git a/drivers/gpu/drm/msm/dsi/dsi_cfg.c b/drivers/gpu/drm/msm/dsi/dsi_cfg.c
index e58e9b9..93c1ee0 100644
--- a/drivers/gpu/drm/msm/dsi/dsi_cfg.c
+++ b/drivers/gpu/drm/msm/dsi/dsi_cfg.c
@@ -22,9 +22,9 @@
.reg_cfg = {
.num = 3,
.regs = {
- {"vdda", 1200000, 1200000, 100000, 100},
- {"avdd", 3000000, 3000000, 110000, 100},
- {"vddio", 1800000, 1800000, 100000, 100},
+ {"vdda", 100000, 100}, /* 1.2 V */
+ {"avdd", 10000, 100}, /* 3.0 V */
+ {"vddio", 100000, 100}, /* 1.8 V */
},
},
.bus_clk_names = dsi_v2_bus_clk_names,
@@ -40,10 +40,10 @@
.reg_cfg = {
.num = 4,
.regs = {
- {"gdsc", -1, -1, -1, -1},
- {"vdd", 3000000, 3000000, 150000, 100},
- {"vdda", 1200000, 1200000, 100000, 100},
- {"vddio", 1800000, 1800000, 100000, 100},
+ {"gdsc", -1, -1},
+ {"vdd", 150000, 100}, /* 3.0 V */
+ {"vdda", 100000, 100}, /* 1.2 V */
+ {"vddio", 100000, 100}, /* 1.8 V */
},
},
.bus_clk_names = dsi_6g_bus_clk_names,
@@ -59,9 +59,9 @@
.reg_cfg = {
.num = 3,
.regs = {
- {"gdsc", -1, -1, -1, -1},
- {"vdda", 1200000, 1200000, 100000, 100},
- {"vddio", 1800000, 1800000, 100000, 100},
+ {"gdsc", -1, -1},
+ {"vdda", 100000, 100}, /* 1.2 V */
+ {"vddio", 100000, 100}, /* 1.8 V */
},
},
.bus_clk_names = dsi_8916_bus_clk_names,
@@ -73,13 +73,13 @@
.reg_cfg = {
.num = 7,
.regs = {
- {"gdsc", -1, -1, -1, -1},
- {"vdda", 1250000, 1250000, 100000, 100},
- {"vddio", 1800000, 1800000, 100000, 100},
- {"vcca", 1000000, 1000000, 10000, 100},
- {"vdd", 1800000, 1800000, 100000, 100},
- {"lab_reg", -1, -1, -1, -1},
- {"ibb_reg", -1, -1, -1, -1},
+ {"gdsc", -1, -1},
+ {"vdda", 100000, 100}, /* 1.25 V */
+ {"vddio", 100000, 100}, /* 1.8 V */
+ {"vcca", 10000, 100}, /* 1.0 V */
+ {"vdd", 100000, 100}, /* 1.8 V */
+ {"lab_reg", -1, -1},
+ {"ibb_reg", -1, -1},
},
},
.bus_clk_names = dsi_6g_bus_clk_names,
diff --git a/drivers/gpu/drm/msm/dsi/dsi_host.c b/drivers/gpu/drm/msm/dsi/dsi_host.c
index 4282ec6..a3e47ad8 100644
--- a/drivers/gpu/drm/msm/dsi/dsi_host.c
+++ b/drivers/gpu/drm/msm/dsi/dsi_host.c
@@ -325,18 +325,6 @@
return ret;
}
- for (i = 0; i < num; i++) {
- if (regulator_can_change_voltage(s[i].consumer)) {
- ret = regulator_set_voltage(s[i].consumer,
- regs[i].min_voltage, regs[i].max_voltage);
- if (ret < 0) {
- pr_err("regulator %d set voltage failed, %d\n",
- i, ret);
- return ret;
- }
- }
- }
-
return 0;
}
diff --git a/drivers/gpu/drm/msm/dsi/dsi_manager.c b/drivers/gpu/drm/msm/dsi/dsi_manager.c
index 58ba7ec..c8d1f19 100644
--- a/drivers/gpu/drm/msm/dsi/dsi_manager.c
+++ b/drivers/gpu/drm/msm/dsi/dsi_manager.c
@@ -198,9 +198,13 @@
static void dsi_mgr_connector_destroy(struct drm_connector *connector)
{
+ struct dsi_connector *dsi_connector = to_dsi_connector(connector);
+
DBG("");
- drm_connector_unregister(connector);
+
drm_connector_cleanup(connector);
+
+ kfree(dsi_connector);
}
static void dsi_dual_connector_fix_modes(struct drm_connector *connector)
@@ -538,12 +542,9 @@
struct dsi_connector *dsi_connector;
int ret, i;
- dsi_connector = devm_kzalloc(msm_dsi->dev->dev,
- sizeof(*dsi_connector), GFP_KERNEL);
- if (!dsi_connector) {
- ret = -ENOMEM;
- goto fail;
- }
+ dsi_connector = kzalloc(sizeof(*dsi_connector), GFP_KERNEL);
+ if (!dsi_connector)
+ return ERR_PTR(-ENOMEM);
dsi_connector->id = id;
@@ -552,7 +553,7 @@
ret = drm_connector_init(msm_dsi->dev, connector,
&dsi_mgr_connector_funcs, DRM_MODE_CONNECTOR_DSI);
if (ret)
- goto fail;
+ return ERR_PTR(ret);
drm_connector_helper_add(connector, &dsi_mgr_conn_helper_funcs);
@@ -565,21 +566,11 @@
connector->interlace_allowed = 0;
connector->doublescan_allowed = 0;
- ret = drm_connector_register(connector);
- if (ret)
- goto fail;
-
for (i = 0; i < MSM_DSI_ENCODER_NUM; i++)
drm_mode_connector_attach_encoder(connector,
msm_dsi->encoders[i]);
return connector;
-
-fail:
- if (connector)
- dsi_mgr_connector_destroy(connector);
-
- return ERR_PTR(ret);
}
/* initialize bridge */
diff --git a/drivers/gpu/drm/msm/dsi/phy/dsi_phy.c b/drivers/gpu/drm/msm/dsi/phy/dsi_phy.c
index 91a95fb..e2f42d8 100644
--- a/drivers/gpu/drm/msm/dsi/phy/dsi_phy.c
+++ b/drivers/gpu/drm/msm/dsi/phy/dsi_phy.c
@@ -177,19 +177,6 @@
return ret;
}
- for (i = 0; i < num; i++) {
- if (regulator_can_change_voltage(s[i].consumer)) {
- ret = regulator_set_voltage(s[i].consumer,
- regs[i].min_voltage, regs[i].max_voltage);
- if (ret < 0) {
- dev_err(dev,
- "regulator %d set voltage failed, %d\n",
- i, ret);
- return ret;
- }
- }
- }
-
return 0;
}
diff --git a/drivers/gpu/drm/msm/dsi/phy/dsi_phy_20nm.c b/drivers/gpu/drm/msm/dsi/phy/dsi_phy_20nm.c
index 2e9ba11..f4bc11a 100644
--- a/drivers/gpu/drm/msm/dsi/phy/dsi_phy_20nm.c
+++ b/drivers/gpu/drm/msm/dsi/phy/dsi_phy_20nm.c
@@ -138,8 +138,8 @@
.reg_cfg = {
.num = 2,
.regs = {
- {"vddio", 1800000, 1800000, 100000, 100},
- {"vcca", 1000000, 1000000, 10000, 100},
+ {"vddio", 100000, 100}, /* 1.8 V */
+ {"vcca", 10000, 100}, /* 1.0 V */
},
},
.ops = {
diff --git a/drivers/gpu/drm/msm/dsi/phy/dsi_phy_28nm.c b/drivers/gpu/drm/msm/dsi/phy/dsi_phy_28nm.c
index edf7411..96d1852 100644
--- a/drivers/gpu/drm/msm/dsi/phy/dsi_phy_28nm.c
+++ b/drivers/gpu/drm/msm/dsi/phy/dsi_phy_28nm.c
@@ -138,7 +138,7 @@
.reg_cfg = {
.num = 1,
.regs = {
- {"vddio", 1800000, 1800000, 100000, 100},
+ {"vddio", 100000, 100},
},
},
.ops = {
@@ -153,7 +153,7 @@
.reg_cfg = {
.num = 1,
.regs = {
- {"vddio", 1800000, 1800000, 100000, 100},
+ {"vddio", 100000, 100}, /* 1.8 V */
},
},
.ops = {
diff --git a/drivers/gpu/drm/msm/dsi/phy/dsi_phy_28nm_8960.c b/drivers/gpu/drm/msm/dsi/phy/dsi_phy_28nm_8960.c
index 197b039..213355a 100644
--- a/drivers/gpu/drm/msm/dsi/phy/dsi_phy_28nm_8960.c
+++ b/drivers/gpu/drm/msm/dsi/phy/dsi_phy_28nm_8960.c
@@ -185,7 +185,7 @@
.reg_cfg = {
.num = 1,
.regs = {
- {"vddio", 1800000, 1800000, 100000, 100},
+ {"vddio", 100000, 100}, /* 1.8 V */
},
},
.ops = {
diff --git a/drivers/gpu/drm/msm/edp/edp_connector.c b/drivers/gpu/drm/msm/edp/edp_connector.c
index b4d1b46..72360cd 100644
--- a/drivers/gpu/drm/msm/edp/edp_connector.c
+++ b/drivers/gpu/drm/msm/edp/edp_connector.c
@@ -37,7 +37,7 @@
struct edp_connector *edp_connector = to_edp_connector(connector);
DBG("");
- drm_connector_unregister(connector);
+
drm_connector_cleanup(connector);
kfree(edp_connector);
@@ -124,10 +124,8 @@
int ret;
edp_connector = kzalloc(sizeof(*edp_connector), GFP_KERNEL);
- if (!edp_connector) {
- ret = -ENOMEM;
- goto fail;
- }
+ if (!edp_connector)
+ return ERR_PTR(-ENOMEM);
edp_connector->edp = edp;
@@ -136,7 +134,7 @@
ret = drm_connector_init(edp->dev, connector, &edp_connector_funcs,
DRM_MODE_CONNECTOR_eDP);
if (ret)
- goto fail;
+ return ERR_PTR(ret);
drm_connector_helper_add(connector, &edp_connector_helper_funcs);
@@ -147,17 +145,7 @@
connector->interlace_allowed = false;
connector->doublescan_allowed = false;
- ret = drm_connector_register(connector);
- if (ret)
- goto fail;
-
drm_mode_connector_attach_encoder(connector, edp->encoder);
return connector;
-
-fail:
- if (connector)
- edp_connector_destroy(connector);
-
- return ERR_PTR(ret);
}
diff --git a/drivers/gpu/drm/msm/edp/edp_ctrl.c b/drivers/gpu/drm/msm/edp/edp_ctrl.c
index 81200e9..149bfe7 100644
--- a/drivers/gpu/drm/msm/edp/edp_ctrl.c
+++ b/drivers/gpu/drm/msm/edp/edp_ctrl.c
@@ -21,8 +21,6 @@
#include "edp.h"
#include "edp.xml.h"
-#define VDDA_MIN_UV 1800000 /* uV units */
-#define VDDA_MAX_UV 1800000 /* uV units */
#define VDDA_UA_ON_LOAD 100000 /* uA units */
#define VDDA_UA_OFF_LOAD 100 /* uA units */
@@ -67,7 +65,7 @@
void __iomem *base;
/* regulators */
- struct regulator *vdda_vreg;
+ struct regulator *vdda_vreg; /* 1.8 V */
struct regulator *lvl_vreg;
/* clocks */
@@ -302,21 +300,24 @@
static int edp_regulator_init(struct edp_ctrl *ctrl)
{
struct device *dev = &ctrl->pdev->dev;
+ int ret;
DBG("");
ctrl->vdda_vreg = devm_regulator_get(dev, "vdda");
- if (IS_ERR(ctrl->vdda_vreg)) {
- pr_err("%s: Could not get vdda reg, ret = %ld\n", __func__,
- PTR_ERR(ctrl->vdda_vreg));
+ ret = PTR_ERR_OR_ZERO(ctrl->vdda_vreg);
+ if (ret) {
+ pr_err("%s: Could not get vdda reg, ret = %d\n", __func__,
+ ret);
ctrl->vdda_vreg = NULL;
- return PTR_ERR(ctrl->vdda_vreg);
+ return ret;
}
ctrl->lvl_vreg = devm_regulator_get(dev, "lvl-vdd");
- if (IS_ERR(ctrl->lvl_vreg)) {
- pr_err("Could not get lvl-vdd reg, %ld",
- PTR_ERR(ctrl->lvl_vreg));
+ ret = PTR_ERR_OR_ZERO(ctrl->lvl_vreg);
+ if (ret) {
+ pr_err("%s: Could not get lvl-vdd reg, ret = %d\n", __func__,
+ ret);
ctrl->lvl_vreg = NULL;
- return PTR_ERR(ctrl->lvl_vreg);
+ return ret;
}
return 0;
@@ -326,12 +327,6 @@
{
int ret;
- ret = regulator_set_voltage(ctrl->vdda_vreg, VDDA_MIN_UV, VDDA_MAX_UV);
- if (ret) {
- pr_err("%s:vdda_vreg set_voltage failed, %d\n", __func__, ret);
- goto vdda_set_fail;
- }
-
ret = regulator_set_load(ctrl->vdda_vreg, VDDA_UA_ON_LOAD);
if (ret < 0) {
pr_err("%s: vdda_vreg set regulator mode failed.\n", __func__);
diff --git a/drivers/gpu/drm/msm/hdmi/hdmi.h b/drivers/gpu/drm/msm/hdmi/hdmi.h
index 65428cf..bc7ba0b 100644
--- a/drivers/gpu/drm/msm/hdmi/hdmi.h
+++ b/drivers/gpu/drm/msm/hdmi/hdmi.h
@@ -243,10 +243,21 @@
/*
* hdcp
*/
+#ifdef CONFIG_DRM_MSM_HDMI_HDCP
struct hdmi_hdcp_ctrl *msm_hdmi_hdcp_init(struct hdmi *hdmi);
void msm_hdmi_hdcp_destroy(struct hdmi *hdmi);
void msm_hdmi_hdcp_on(struct hdmi_hdcp_ctrl *hdcp_ctrl);
void msm_hdmi_hdcp_off(struct hdmi_hdcp_ctrl *hdcp_ctrl);
void msm_hdmi_hdcp_irq(struct hdmi_hdcp_ctrl *hdcp_ctrl);
+#else
+static inline struct hdmi_hdcp_ctrl *msm_hdmi_hdcp_init(struct hdmi *hdmi)
+{
+ return ERR_PTR(-ENXIO);
+}
+static inline void msm_hdmi_hdcp_destroy(struct hdmi *hdmi) {}
+static inline void msm_hdmi_hdcp_on(struct hdmi_hdcp_ctrl *hdcp_ctrl) {}
+static inline void msm_hdmi_hdcp_off(struct hdmi_hdcp_ctrl *hdcp_ctrl) {}
+static inline void msm_hdmi_hdcp_irq(struct hdmi_hdcp_ctrl *hdcp_ctrl) {}
+#endif
#endif /* __HDMI_CONNECTOR_H__ */
diff --git a/drivers/gpu/drm/msm/hdmi/hdmi_connector.c b/drivers/gpu/drm/msm/hdmi/hdmi_connector.c
index 26129bf..b15d726 100644
--- a/drivers/gpu/drm/msm/hdmi/hdmi_connector.c
+++ b/drivers/gpu/drm/msm/hdmi/hdmi_connector.c
@@ -112,6 +112,9 @@
for (i = 0; i < HDMI_MAX_NUM_GPIO; i++) {
struct hdmi_gpio_data gpio = config->gpios[i];
+ if (gpio.num == -1)
+ continue;
+
if (gpio.output) {
int value = gpio.value ? 0 : 1;
@@ -126,8 +129,10 @@
return 0;
err:
- while (i--)
- gpio_free(config->gpios[i].num);
+ while (i--) {
+ if (config->gpios[i].num != -1)
+ gpio_free(config->gpios[i].num);
+ }
return ret;
}
@@ -341,7 +346,6 @@
hdp_disable(hdmi_connector);
- drm_connector_unregister(connector);
drm_connector_cleanup(connector);
kfree(hdmi_connector);
@@ -433,10 +437,8 @@
int ret;
hdmi_connector = kzalloc(sizeof(*hdmi_connector), GFP_KERNEL);
- if (!hdmi_connector) {
- ret = -ENOMEM;
- goto fail;
- }
+ if (!hdmi_connector)
+ return ERR_PTR(-ENOMEM);
hdmi_connector->hdmi = hdmi;
INIT_WORK(&hdmi_connector->hpd_work, msm_hdmi_hotplug_work);
@@ -453,21 +455,13 @@
connector->interlace_allowed = 0;
connector->doublescan_allowed = 0;
- drm_connector_register(connector);
-
ret = hpd_enable(hdmi_connector);
if (ret) {
dev_err(&hdmi->pdev->dev, "failed to enable HPD: %d\n", ret);
- goto fail;
+ return ERR_PTR(ret);
}
drm_mode_connector_attach_encoder(connector, hdmi->encoder);
return connector;
-
-fail:
- if (connector)
- hdmi_connector_destroy(connector);
-
- return ERR_PTR(ret);
}
diff --git a/drivers/gpu/drm/msm/mdp/mdp4/mdp4_crtc.c b/drivers/gpu/drm/msm/mdp/mdp4/mdp4_crtc.c
index e233acf..9527daf 100644
--- a/drivers/gpu/drm/msm/mdp/mdp4/mdp4_crtc.c
+++ b/drivers/gpu/drm/msm/mdp/mdp4/mdp4_crtc.c
@@ -121,7 +121,7 @@
if (!file || (event->base.file_priv == file)) {
mdp4_crtc->event = NULL;
DBG("%s: send event: %p", mdp4_crtc->name, event);
- drm_send_vblank_event(dev, mdp4_crtc->id, event);
+ drm_crtc_send_vblank_event(crtc, event);
}
}
spin_unlock_irqrestore(&dev->event_lock, flags);
@@ -427,7 +427,7 @@
}
if (handle) {
- cursor_bo = drm_gem_object_lookup(dev, file_priv, handle);
+ cursor_bo = drm_gem_object_lookup(file_priv, handle);
if (!cursor_bo)
return -ENOENT;
} else {
diff --git a/drivers/gpu/drm/msm/mdp/mdp4/mdp4_kms.c b/drivers/gpu/drm/msm/mdp/mdp4/mdp4_kms.c
index 76e1dfb..67442d5 100644
--- a/drivers/gpu/drm/msm/mdp/mdp4/mdp4_kms.c
+++ b/drivers/gpu/drm/msm/mdp/mdp4/mdp4_kms.c
@@ -50,30 +50,6 @@
mdp4_kms->rev = minor;
- if (mdp4_kms->dsi_pll_vdda) {
- if ((mdp4_kms->rev == 2) || (mdp4_kms->rev == 4)) {
- ret = regulator_set_voltage(mdp4_kms->dsi_pll_vdda,
- 1200000, 1200000);
- if (ret) {
- dev_err(dev->dev,
- "failed to set dsi_pll_vdda voltage: %d\n", ret);
- goto out;
- }
- }
- }
-
- if (mdp4_kms->dsi_pll_vddio) {
- if (mdp4_kms->rev == 2) {
- ret = regulator_set_voltage(mdp4_kms->dsi_pll_vddio,
- 1800000, 1800000);
- if (ret) {
- dev_err(dev->dev,
- "failed to set dsi_pll_vddio voltage: %d\n", ret);
- goto out;
- }
- }
- }
-
if (mdp4_kms->rev > 1) {
mdp4_write(mdp4_kms, REG_MDP4_CS_CONTROLLER0, 0x0707ffff);
mdp4_write(mdp4_kms, REG_MDP4_CS_CONTROLLER1, 0x03073f3f);
@@ -485,16 +461,6 @@
goto fail;
}
- mdp4_kms->dsi_pll_vdda =
- devm_regulator_get_optional(&pdev->dev, "dsi_pll_vdda");
- if (IS_ERR(mdp4_kms->dsi_pll_vdda))
- mdp4_kms->dsi_pll_vdda = NULL;
-
- mdp4_kms->dsi_pll_vddio =
- devm_regulator_get_optional(&pdev->dev, "dsi_pll_vddio");
- if (IS_ERR(mdp4_kms->dsi_pll_vddio))
- mdp4_kms->dsi_pll_vddio = NULL;
-
/* NOTE: driver for this regulator still missing upstream.. use
* _get_exclusive() and ignore the error if it does not exist
* (and hope that the bootloader left it on for us)
diff --git a/drivers/gpu/drm/msm/mdp/mdp4/mdp4_kms.h b/drivers/gpu/drm/msm/mdp/mdp4/mdp4_kms.h
index b282871..c5d045d 100644
--- a/drivers/gpu/drm/msm/mdp/mdp4/mdp4_kms.h
+++ b/drivers/gpu/drm/msm/mdp/mdp4/mdp4_kms.h
@@ -37,8 +37,6 @@
void __iomem *mmio;
- struct regulator *dsi_pll_vdda;
- struct regulator *dsi_pll_vddio;
struct regulator *vdd;
struct clk *clk;
diff --git a/drivers/gpu/drm/msm/mdp/mdp4/mdp4_lvds_connector.c b/drivers/gpu/drm/msm/mdp/mdp4/mdp4_lvds_connector.c
index e73e174..2648cd7 100644
--- a/drivers/gpu/drm/msm/mdp/mdp4/mdp4_lvds_connector.c
+++ b/drivers/gpu/drm/msm/mdp/mdp4/mdp4_lvds_connector.c
@@ -48,7 +48,6 @@
struct mdp4_lvds_connector *mdp4_lvds_connector =
to_mdp4_lvds_connector(connector);
- drm_connector_unregister(connector);
drm_connector_cleanup(connector);
kfree(mdp4_lvds_connector);
@@ -121,13 +120,10 @@
{
struct drm_connector *connector = NULL;
struct mdp4_lvds_connector *mdp4_lvds_connector;
- int ret;
mdp4_lvds_connector = kzalloc(sizeof(*mdp4_lvds_connector), GFP_KERNEL);
- if (!mdp4_lvds_connector) {
- ret = -ENOMEM;
- goto fail;
- }
+ if (!mdp4_lvds_connector)
+ return ERR_PTR(-ENOMEM);
mdp4_lvds_connector->encoder = encoder;
mdp4_lvds_connector->panel_node = panel_node;
@@ -143,15 +139,7 @@
connector->interlace_allowed = 0;
connector->doublescan_allowed = 0;
- drm_connector_register(connector);
-
drm_mode_connector_attach_encoder(connector, encoder);
return connector;
-
-fail:
- if (connector)
- mdp4_lvds_connector_destroy(connector);
-
- return ERR_PTR(ret);
}
diff --git a/drivers/gpu/drm/msm/mdp/mdp5/mdp5_crtc.c b/drivers/gpu/drm/msm/mdp/mdp5/mdp5_crtc.c
index 9673b95..88fe256 100644
--- a/drivers/gpu/drm/msm/mdp/mdp5/mdp5_crtc.c
+++ b/drivers/gpu/drm/msm/mdp/mdp5/mdp5_crtc.c
@@ -149,7 +149,7 @@
if (!file || (event->base.file_priv == file)) {
mdp5_crtc->event = NULL;
DBG("%s: send event: %p", mdp5_crtc->name, event);
- drm_send_vblank_event(dev, mdp5_crtc->id, event);
+ drm_crtc_send_vblank_event(crtc, event);
}
}
spin_unlock_irqrestore(&dev->event_lock, flags);
@@ -518,7 +518,7 @@
goto set_cursor;
}
- cursor_bo = drm_gem_object_lookup(dev, file, handle);
+ cursor_bo = drm_gem_object_lookup(file, handle);
if (!cursor_bo)
return -ENOENT;
diff --git a/drivers/gpu/drm/msm/mdp/mdp_format.c b/drivers/gpu/drm/msm/mdp/mdp_format.c
index 1c2caff..b4a8aa4 100644
--- a/drivers/gpu/drm/msm/mdp/mdp_format.c
+++ b/drivers/gpu/drm/msm/mdp/mdp_format.c
@@ -105,6 +105,12 @@
MDP_PLANE_INTERLEAVED, CHROMA_FULL, false),
FMT(XRGB8888, 8, 8, 8, 8, 1, 0, 2, 3, false, true, 4, 4,
MDP_PLANE_INTERLEAVED, CHROMA_FULL, false),
+ FMT(XBGR8888, 8, 8, 8, 8, 2, 0, 1, 3, false, true, 4, 4,
+ MDP_PLANE_INTERLEAVED, CHROMA_FULL, false),
+ FMT(RGBX8888, 8, 8, 8, 8, 3, 1, 0, 2, false, true, 4, 4,
+ MDP_PLANE_INTERLEAVED, CHROMA_FULL, false),
+ FMT(BGRX8888, 8, 8, 8, 8, 3, 2, 0, 1, false, true, 4, 4,
+ MDP_PLANE_INTERLEAVED, CHROMA_FULL, false),
FMT(RGB888, 0, 8, 8, 8, 1, 0, 2, 0, false, true, 3, 3,
MDP_PLANE_INTERLEAVED, CHROMA_FULL, false),
FMT(BGR888, 0, 8, 8, 8, 2, 0, 1, 0, false, true, 3, 3,
diff --git a/drivers/gpu/drm/msm/msm_atomic.c b/drivers/gpu/drm/msm/msm_atomic.c
index 7eb253b..e3892c2 100644
--- a/drivers/gpu/drm/msm/msm_atomic.c
+++ b/drivers/gpu/drm/msm/msm_atomic.c
@@ -18,16 +18,16 @@
#include "msm_drv.h"
#include "msm_kms.h"
#include "msm_gem.h"
+#include "msm_fence.h"
struct msm_commit {
struct drm_device *dev;
struct drm_atomic_state *state;
- uint32_t fence;
- struct msm_fence_cb fence_cb;
+ struct work_struct work;
uint32_t crtc_mask;
};
-static void fence_cb(struct msm_fence_cb *cb);
+static void commit_worker(struct work_struct *work);
/* block until specified crtcs are no longer pending update, and
* atomically mark them as pending update
@@ -69,11 +69,7 @@
c->dev = state->dev;
c->state = state;
- /* TODO we might need a way to indicate to run the cb on a
- * different wq so wait_for_vblanks() doesn't block retiring
- * bo's..
- */
- INIT_FENCE_CB(&c->fence_cb, fence_cb);
+ INIT_WORK(&c->work, commit_worker);
return c;
}
@@ -114,13 +110,15 @@
/* The (potentially) asynchronous part of the commit. At this point
* nothing can fail short of armageddon.
*/
-static void complete_commit(struct msm_commit *c)
+static void complete_commit(struct msm_commit *c, bool async)
{
struct drm_atomic_state *state = c->state;
struct drm_device *dev = state->dev;
struct msm_drm_private *priv = dev->dev_private;
struct msm_kms *kms = priv->kms;
+ drm_atomic_helper_wait_for_fences(dev, state);
+
kms->funcs->prepare_commit(kms, state);
drm_atomic_helper_commit_modeset_disables(dev, state);
@@ -153,17 +151,9 @@
commit_destroy(c);
}
-static void fence_cb(struct msm_fence_cb *cb)
+static void commit_worker(struct work_struct *work)
{
- struct msm_commit *c =
- container_of(cb, struct msm_commit, fence_cb);
- complete_commit(c);
-}
-
-static void add_fb(struct msm_commit *c, struct drm_framebuffer *fb)
-{
- struct drm_gem_object *obj = msm_framebuffer_bo(fb, 0);
- c->fence = max(c->fence, msm_gem_fence(to_msm_bo(obj), MSM_PREP_READ));
+ complete_commit(container_of(work, struct msm_commit, work), true);
}
int msm_atomic_check(struct drm_device *dev,
@@ -190,21 +180,20 @@
* drm_atomic_helper_commit - commit validated state object
* @dev: DRM device
* @state: the driver state object
- * @async: asynchronous commit
+ * @nonblock: nonblocking commit
*
* This function commits a with drm_atomic_helper_check() pre-validated state
- * object. This can still fail when e.g. the framebuffer reservation fails. For
- * now this doesn't implement asynchronous commits.
+ * object. This can still fail when e.g. the framebuffer reservation fails.
*
* RETURNS
* Zero for success or -errno.
*/
int msm_atomic_commit(struct drm_device *dev,
- struct drm_atomic_state *state, bool async)
+ struct drm_atomic_state *state, bool nonblock)
{
+ struct msm_drm_private *priv = dev->dev_private;
int nplanes = dev->mode_config.num_total_plane;
int ncrtcs = dev->mode_config.num_crtc;
- ktime_t timeout;
struct msm_commit *c;
int i, ret;
@@ -238,8 +227,12 @@
if (!plane)
continue;
- if ((plane->state->fb != new_state->fb) && new_state->fb)
- add_fb(c, new_state->fb);
+ if ((plane->state->fb != new_state->fb) && new_state->fb) {
+ struct drm_gem_object *obj = msm_framebuffer_bo(new_state->fb, 0);
+ struct msm_gem_object *msm_obj = to_msm_bo(obj);
+
+ new_state->fence = reservation_object_get_excl_rcu(msm_obj->resv);
+ }
}
/*
@@ -276,17 +269,12 @@
* current layout.
*/
- if (async) {
- msm_queue_fence_cb(dev, &c->fence_cb, c->fence);
+ if (nonblock) {
+ queue_work(priv->atomic_wq, &c->work);
return 0;
}
- timeout = ktime_add_ms(ktime_get(), 1000);
-
- /* uninterruptible wait */
- msm_wait_fence(dev, c->fence, &timeout, false);
-
- complete_commit(c);
+ complete_commit(c, false);
return 0;
diff --git a/drivers/gpu/drm/msm/msm_debugfs.c b/drivers/gpu/drm/msm/msm_debugfs.c
new file mode 100644
index 0000000..663f2b6
--- /dev/null
+++ b/drivers/gpu/drm/msm/msm_debugfs.c
@@ -0,0 +1,168 @@
+/*
+ * Copyright (C) 2013-2016 Red Hat
+ * Author: Rob Clark <robdclark@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifdef CONFIG_DEBUG_FS
+#include "msm_drv.h"
+#include "msm_gpu.h"
+
+static int msm_gpu_show(struct drm_device *dev, struct seq_file *m)
+{
+ struct msm_drm_private *priv = dev->dev_private;
+ struct msm_gpu *gpu = priv->gpu;
+
+ if (gpu) {
+ seq_printf(m, "%s Status:\n", gpu->name);
+ gpu->funcs->show(gpu, m);
+ }
+
+ return 0;
+}
+
+static int msm_gem_show(struct drm_device *dev, struct seq_file *m)
+{
+ struct msm_drm_private *priv = dev->dev_private;
+ struct msm_gpu *gpu = priv->gpu;
+
+ if (gpu) {
+ seq_printf(m, "Active Objects (%s):\n", gpu->name);
+ msm_gem_describe_objects(&gpu->active_list, m);
+ }
+
+ seq_printf(m, "Inactive Objects:\n");
+ msm_gem_describe_objects(&priv->inactive_list, m);
+
+ return 0;
+}
+
+static int msm_mm_show(struct drm_device *dev, struct seq_file *m)
+{
+ return drm_mm_dump_table(m, &dev->vma_offset_manager->vm_addr_space_mm);
+}
+
+static int msm_fb_show(struct drm_device *dev, struct seq_file *m)
+{
+ struct msm_drm_private *priv = dev->dev_private;
+ struct drm_framebuffer *fb, *fbdev_fb = NULL;
+
+ if (priv->fbdev) {
+ seq_printf(m, "fbcon ");
+ fbdev_fb = priv->fbdev->fb;
+ msm_framebuffer_describe(fbdev_fb, m);
+ }
+
+ mutex_lock(&dev->mode_config.fb_lock);
+ list_for_each_entry(fb, &dev->mode_config.fb_list, head) {
+ if (fb == fbdev_fb)
+ continue;
+
+ seq_printf(m, "user ");
+ msm_framebuffer_describe(fb, m);
+ }
+ mutex_unlock(&dev->mode_config.fb_lock);
+
+ return 0;
+}
+
+static int show_locked(struct seq_file *m, void *arg)
+{
+ struct drm_info_node *node = (struct drm_info_node *) m->private;
+ struct drm_device *dev = node->minor->dev;
+ int (*show)(struct drm_device *dev, struct seq_file *m) =
+ node->info_ent->data;
+ int ret;
+
+ ret = mutex_lock_interruptible(&dev->struct_mutex);
+ if (ret)
+ return ret;
+
+ ret = show(dev, m);
+
+ mutex_unlock(&dev->struct_mutex);
+
+ return ret;
+}
+
+static struct drm_info_list msm_debugfs_list[] = {
+ {"gpu", show_locked, 0, msm_gpu_show},
+ {"gem", show_locked, 0, msm_gem_show},
+ { "mm", show_locked, 0, msm_mm_show },
+ { "fb", show_locked, 0, msm_fb_show },
+};
+
+static int late_init_minor(struct drm_minor *minor)
+{
+ int ret;
+
+ if (!minor)
+ return 0;
+
+ ret = msm_rd_debugfs_init(minor);
+ if (ret) {
+ dev_err(minor->dev->dev, "could not install rd debugfs\n");
+ return ret;
+ }
+
+ ret = msm_perf_debugfs_init(minor);
+ if (ret) {
+ dev_err(minor->dev->dev, "could not install perf debugfs\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+int msm_debugfs_late_init(struct drm_device *dev)
+{
+ int ret;
+ ret = late_init_minor(dev->primary);
+ if (ret)
+ return ret;
+ ret = late_init_minor(dev->render);
+ if (ret)
+ return ret;
+ ret = late_init_minor(dev->control);
+ return ret;
+}
+
+int msm_debugfs_init(struct drm_minor *minor)
+{
+ struct drm_device *dev = minor->dev;
+ int ret;
+
+ ret = drm_debugfs_create_files(msm_debugfs_list,
+ ARRAY_SIZE(msm_debugfs_list),
+ minor->debugfs_root, minor);
+
+ if (ret) {
+ dev_err(dev->dev, "could not install msm_debugfs_list\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+void msm_debugfs_cleanup(struct drm_minor *minor)
+{
+ drm_debugfs_remove_files(msm_debugfs_list,
+ ARRAY_SIZE(msm_debugfs_list), minor);
+ if (!minor->dev->dev_private)
+ return;
+ msm_rd_debugfs_cleanup(minor);
+ msm_perf_debugfs_cleanup(minor);
+}
+#endif
+
diff --git a/drivers/gpu/drm/msm/msm_debugfs.h b/drivers/gpu/drm/msm/msm_debugfs.h
new file mode 100644
index 0000000..6110c97
--- /dev/null
+++ b/drivers/gpu/drm/msm/msm_debugfs.h
@@ -0,0 +1,26 @@
+/*
+ * Copyright (C) 2016 Red Hat
+ * Author: Rob Clark <robdclark@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __MSM_DEBUGFS_H__
+#define __MSM_DEBUGFS_H__
+
+#ifdef CONFIG_DEBUG_FS
+int msm_debugfs_init(struct drm_minor *minor);
+void msm_debugfs_cleanup(struct drm_minor *minor);
+#endif
+
+#endif /* __MSM_DEBUGFS_H__ */
diff --git a/drivers/gpu/drm/msm/msm_drv.c b/drivers/gpu/drm/msm/msm_drv.c
index c03b967..9c65409 100644
--- a/drivers/gpu/drm/msm/msm_drv.c
+++ b/drivers/gpu/drm/msm/msm_drv.c
@@ -16,6 +16,8 @@
*/
#include "msm_drv.h"
+#include "msm_debugfs.h"
+#include "msm_fence.h"
#include "msm_gpu.h"
#include "msm_kms.h"
@@ -173,13 +175,11 @@
return 0;
}
-/*
- * DRM operations:
- */
-
-static int msm_unload(struct drm_device *dev)
+static int msm_drm_uninit(struct device *dev)
{
- struct msm_drm_private *priv = dev->dev_private;
+ struct platform_device *pdev = to_platform_device(dev);
+ struct drm_device *ddev = platform_get_drvdata(pdev);
+ struct msm_drm_private *priv = ddev->dev_private;
struct msm_kms *kms = priv->kms;
struct msm_gpu *gpu = priv->gpu;
struct msm_vblank_ctrl *vbl_ctrl = &priv->vblank_ctrl;
@@ -195,31 +195,37 @@
kfree(vbl_ev);
}
- drm_kms_helper_poll_fini(dev);
+ drm_kms_helper_poll_fini(ddev);
+
+ drm_connector_unregister_all(ddev);
+
+ drm_dev_unregister(ddev);
#ifdef CONFIG_DRM_FBDEV_EMULATION
if (fbdev && priv->fbdev)
- msm_fbdev_free(dev);
+ msm_fbdev_free(ddev);
#endif
- drm_mode_config_cleanup(dev);
- drm_vblank_cleanup(dev);
+ drm_mode_config_cleanup(ddev);
- pm_runtime_get_sync(dev->dev);
- drm_irq_uninstall(dev);
- pm_runtime_put_sync(dev->dev);
+ pm_runtime_get_sync(dev);
+ drm_irq_uninstall(ddev);
+ pm_runtime_put_sync(dev);
flush_workqueue(priv->wq);
destroy_workqueue(priv->wq);
+ flush_workqueue(priv->atomic_wq);
+ destroy_workqueue(priv->atomic_wq);
+
if (kms) {
- pm_runtime_disable(dev->dev);
+ pm_runtime_disable(dev);
kms->funcs->destroy(kms);
}
if (gpu) {
- mutex_lock(&dev->struct_mutex);
+ mutex_lock(&ddev->struct_mutex);
gpu->funcs->pm_suspend(gpu);
- mutex_unlock(&dev->struct_mutex);
+ mutex_unlock(&ddev->struct_mutex);
gpu->funcs->destroy(gpu);
}
@@ -227,13 +233,14 @@
DEFINE_DMA_ATTRS(attrs);
dma_set_attr(DMA_ATTR_NO_KERNEL_MAPPING, &attrs);
drm_mm_takedown(&priv->vram.mm);
- dma_free_attrs(dev->dev, priv->vram.size, NULL,
- priv->vram.paddr, &attrs);
+ dma_free_attrs(dev, priv->vram.size, NULL,
+ priv->vram.paddr, &attrs);
}
- component_unbind_all(dev->dev, dev);
+ component_unbind_all(dev, ddev);
- dev->dev_private = NULL;
+ ddev->dev_private = NULL;
+ drm_dev_unref(ddev);
kfree(priv);
@@ -321,50 +328,60 @@
return ret;
}
-static int msm_load(struct drm_device *dev, unsigned long flags)
+static int msm_drm_init(struct device *dev, struct drm_driver *drv)
{
- struct platform_device *pdev = dev->platformdev;
+ struct platform_device *pdev = to_platform_device(dev);
+ struct drm_device *ddev;
struct msm_drm_private *priv;
struct msm_kms *kms;
int ret;
- priv = kzalloc(sizeof(*priv), GFP_KERNEL);
- if (!priv) {
- dev_err(dev->dev, "failed to allocate private data\n");
+ ddev = drm_dev_alloc(drv, dev);
+ if (!ddev) {
+ dev_err(dev, "failed to allocate drm_device\n");
return -ENOMEM;
}
- dev->dev_private = priv;
+ platform_set_drvdata(pdev, ddev);
+ ddev->platformdev = pdev;
+
+ priv = kzalloc(sizeof(*priv), GFP_KERNEL);
+ if (!priv) {
+ drm_dev_unref(ddev);
+ return -ENOMEM;
+ }
+
+ ddev->dev_private = priv;
priv->wq = alloc_ordered_workqueue("msm", 0);
- init_waitqueue_head(&priv->fence_event);
+ priv->atomic_wq = alloc_ordered_workqueue("msm:atomic", 0);
init_waitqueue_head(&priv->pending_crtcs_event);
INIT_LIST_HEAD(&priv->inactive_list);
- INIT_LIST_HEAD(&priv->fence_cbs);
INIT_LIST_HEAD(&priv->vblank_ctrl.event_list);
INIT_WORK(&priv->vblank_ctrl.work, vblank_ctrl_worker);
spin_lock_init(&priv->vblank_ctrl.lock);
- drm_mode_config_init(dev);
-
- platform_set_drvdata(pdev, dev);
+ drm_mode_config_init(ddev);
/* Bind all our sub-components: */
- ret = component_bind_all(dev->dev, dev);
- if (ret)
+ ret = component_bind_all(dev, ddev);
+ if (ret) {
+ kfree(priv);
+ drm_dev_unref(ddev);
return ret;
+ }
- ret = msm_init_vram(dev);
+ ret = msm_init_vram(ddev);
if (ret)
goto fail;
switch (get_mdp_ver(pdev)) {
case 4:
- kms = mdp4_kms_init(dev);
+ kms = mdp4_kms_init(ddev);
break;
case 5:
- kms = mdp5_kms_init(dev);
+ kms = mdp5_kms_init(ddev);
break;
default:
kms = ERR_PTR(-ENODEV);
@@ -378,7 +395,7 @@
* and (for example) use dmabuf/prime to share buffers with
* imx drm driver on iMX5
*/
- dev_err(dev->dev, "failed to load kms\n");
+ dev_err(dev, "failed to load kms\n");
ret = PTR_ERR(kms);
goto fail;
}
@@ -386,50 +403,64 @@
priv->kms = kms;
if (kms) {
- pm_runtime_enable(dev->dev);
+ pm_runtime_enable(dev);
ret = kms->funcs->hw_init(kms);
if (ret) {
- dev_err(dev->dev, "kms hw init failed: %d\n", ret);
+ dev_err(dev, "kms hw init failed: %d\n", ret);
goto fail;
}
}
- dev->mode_config.funcs = &mode_config_funcs;
+ ddev->mode_config.funcs = &mode_config_funcs;
- ret = drm_vblank_init(dev, priv->num_crtcs);
+ ret = drm_vblank_init(ddev, priv->num_crtcs);
if (ret < 0) {
- dev_err(dev->dev, "failed to initialize vblank\n");
+ dev_err(dev, "failed to initialize vblank\n");
goto fail;
}
- pm_runtime_get_sync(dev->dev);
- ret = drm_irq_install(dev, platform_get_irq(dev->platformdev, 0));
- pm_runtime_put_sync(dev->dev);
+ pm_runtime_get_sync(dev);
+ ret = drm_irq_install(ddev, platform_get_irq(pdev, 0));
+ pm_runtime_put_sync(dev);
if (ret < 0) {
- dev_err(dev->dev, "failed to install IRQ handler\n");
+ dev_err(dev, "failed to install IRQ handler\n");
goto fail;
}
- drm_mode_config_reset(dev);
-
-#ifdef CONFIG_DRM_FBDEV_EMULATION
- if (fbdev)
- priv->fbdev = msm_fbdev_init(dev);
-#endif
-
- ret = msm_debugfs_late_init(dev);
+ ret = drm_dev_register(ddev, 0);
if (ret)
goto fail;
- drm_kms_helper_poll_init(dev);
+ ret = drm_connector_register_all(ddev);
+ if (ret) {
+ dev_err(dev, "failed to register connectors\n");
+ goto fail;
+ }
+
+ drm_mode_config_reset(ddev);
+
+#ifdef CONFIG_DRM_FBDEV_EMULATION
+ if (fbdev)
+ priv->fbdev = msm_fbdev_init(ddev);
+#endif
+
+ ret = msm_debugfs_late_init(ddev);
+ if (ret)
+ goto fail;
+
+ drm_kms_helper_poll_init(ddev);
return 0;
fail:
- msm_unload(dev);
+ msm_drm_uninit(dev);
return ret;
}
+/*
+ * DRM operations:
+ */
+
static void load_gpu(struct drm_device *dev)
{
static DEFINE_MUTEX(init_lock);
@@ -465,7 +496,6 @@
{
struct msm_drm_private *priv = dev->dev_private;
struct msm_file_private *ctx = file->driver_priv;
- struct msm_kms *kms = priv->kms;
mutex_lock(&dev->struct_mutex);
if (ctx == priv->lastctx)
@@ -536,265 +566,6 @@
}
/*
- * DRM debugfs:
- */
-
-#ifdef CONFIG_DEBUG_FS
-static int msm_gpu_show(struct drm_device *dev, struct seq_file *m)
-{
- struct msm_drm_private *priv = dev->dev_private;
- struct msm_gpu *gpu = priv->gpu;
-
- if (gpu) {
- seq_printf(m, "%s Status:\n", gpu->name);
- gpu->funcs->show(gpu, m);
- }
-
- return 0;
-}
-
-static int msm_gem_show(struct drm_device *dev, struct seq_file *m)
-{
- struct msm_drm_private *priv = dev->dev_private;
- struct msm_gpu *gpu = priv->gpu;
-
- if (gpu) {
- seq_printf(m, "Active Objects (%s):\n", gpu->name);
- msm_gem_describe_objects(&gpu->active_list, m);
- }
-
- seq_printf(m, "Inactive Objects:\n");
- msm_gem_describe_objects(&priv->inactive_list, m);
-
- return 0;
-}
-
-static int msm_mm_show(struct drm_device *dev, struct seq_file *m)
-{
- return drm_mm_dump_table(m, &dev->vma_offset_manager->vm_addr_space_mm);
-}
-
-static int msm_fb_show(struct drm_device *dev, struct seq_file *m)
-{
- struct msm_drm_private *priv = dev->dev_private;
- struct drm_framebuffer *fb, *fbdev_fb = NULL;
-
- if (priv->fbdev) {
- seq_printf(m, "fbcon ");
- fbdev_fb = priv->fbdev->fb;
- msm_framebuffer_describe(fbdev_fb, m);
- }
-
- mutex_lock(&dev->mode_config.fb_lock);
- list_for_each_entry(fb, &dev->mode_config.fb_list, head) {
- if (fb == fbdev_fb)
- continue;
-
- seq_printf(m, "user ");
- msm_framebuffer_describe(fb, m);
- }
- mutex_unlock(&dev->mode_config.fb_lock);
-
- return 0;
-}
-
-static int show_locked(struct seq_file *m, void *arg)
-{
- struct drm_info_node *node = (struct drm_info_node *) m->private;
- struct drm_device *dev = node->minor->dev;
- int (*show)(struct drm_device *dev, struct seq_file *m) =
- node->info_ent->data;
- int ret;
-
- ret = mutex_lock_interruptible(&dev->struct_mutex);
- if (ret)
- return ret;
-
- ret = show(dev, m);
-
- mutex_unlock(&dev->struct_mutex);
-
- return ret;
-}
-
-static struct drm_info_list msm_debugfs_list[] = {
- {"gpu", show_locked, 0, msm_gpu_show},
- {"gem", show_locked, 0, msm_gem_show},
- { "mm", show_locked, 0, msm_mm_show },
- { "fb", show_locked, 0, msm_fb_show },
-};
-
-static int late_init_minor(struct drm_minor *minor)
-{
- int ret;
-
- if (!minor)
- return 0;
-
- ret = msm_rd_debugfs_init(minor);
- if (ret) {
- dev_err(minor->dev->dev, "could not install rd debugfs\n");
- return ret;
- }
-
- ret = msm_perf_debugfs_init(minor);
- if (ret) {
- dev_err(minor->dev->dev, "could not install perf debugfs\n");
- return ret;
- }
-
- return 0;
-}
-
-int msm_debugfs_late_init(struct drm_device *dev)
-{
- int ret;
- ret = late_init_minor(dev->primary);
- if (ret)
- return ret;
- ret = late_init_minor(dev->render);
- if (ret)
- return ret;
- ret = late_init_minor(dev->control);
- return ret;
-}
-
-static int msm_debugfs_init(struct drm_minor *minor)
-{
- struct drm_device *dev = minor->dev;
- int ret;
-
- ret = drm_debugfs_create_files(msm_debugfs_list,
- ARRAY_SIZE(msm_debugfs_list),
- minor->debugfs_root, minor);
-
- if (ret) {
- dev_err(dev->dev, "could not install msm_debugfs_list\n");
- return ret;
- }
-
- return 0;
-}
-
-static void msm_debugfs_cleanup(struct drm_minor *minor)
-{
- drm_debugfs_remove_files(msm_debugfs_list,
- ARRAY_SIZE(msm_debugfs_list), minor);
- if (!minor->dev->dev_private)
- return;
- msm_rd_debugfs_cleanup(minor);
- msm_perf_debugfs_cleanup(minor);
-}
-#endif
-
-/*
- * Fences:
- */
-
-int msm_wait_fence(struct drm_device *dev, uint32_t fence,
- ktime_t *timeout , bool interruptible)
-{
- struct msm_drm_private *priv = dev->dev_private;
- int ret;
-
- if (!priv->gpu)
- return 0;
-
- if (fence > priv->gpu->submitted_fence) {
- DRM_ERROR("waiting on invalid fence: %u (of %u)\n",
- fence, priv->gpu->submitted_fence);
- return -EINVAL;
- }
-
- if (!timeout) {
- /* no-wait: */
- ret = fence_completed(dev, fence) ? 0 : -EBUSY;
- } else {
- ktime_t now = ktime_get();
- unsigned long remaining_jiffies;
-
- if (ktime_compare(*timeout, now) < 0) {
- remaining_jiffies = 0;
- } else {
- ktime_t rem = ktime_sub(*timeout, now);
- struct timespec ts = ktime_to_timespec(rem);
- remaining_jiffies = timespec_to_jiffies(&ts);
- }
-
- if (interruptible)
- ret = wait_event_interruptible_timeout(priv->fence_event,
- fence_completed(dev, fence),
- remaining_jiffies);
- else
- ret = wait_event_timeout(priv->fence_event,
- fence_completed(dev, fence),
- remaining_jiffies);
-
- if (ret == 0) {
- DBG("timeout waiting for fence: %u (completed: %u)",
- fence, priv->completed_fence);
- ret = -ETIMEDOUT;
- } else if (ret != -ERESTARTSYS) {
- ret = 0;
- }
- }
-
- return ret;
-}
-
-int msm_queue_fence_cb(struct drm_device *dev,
- struct msm_fence_cb *cb, uint32_t fence)
-{
- struct msm_drm_private *priv = dev->dev_private;
- int ret = 0;
-
- mutex_lock(&dev->struct_mutex);
- if (!list_empty(&cb->work.entry)) {
- ret = -EINVAL;
- } else if (fence > priv->completed_fence) {
- cb->fence = fence;
- list_add_tail(&cb->work.entry, &priv->fence_cbs);
- } else {
- queue_work(priv->wq, &cb->work);
- }
- mutex_unlock(&dev->struct_mutex);
-
- return ret;
-}
-
-/* called from workqueue */
-void msm_update_fence(struct drm_device *dev, uint32_t fence)
-{
- struct msm_drm_private *priv = dev->dev_private;
-
- mutex_lock(&dev->struct_mutex);
- priv->completed_fence = max(fence, priv->completed_fence);
-
- while (!list_empty(&priv->fence_cbs)) {
- struct msm_fence_cb *cb;
-
- cb = list_first_entry(&priv->fence_cbs,
- struct msm_fence_cb, work.entry);
-
- if (cb->fence > priv->completed_fence)
- break;
-
- list_del_init(&cb->work.entry);
- queue_work(priv->wq, &cb->work);
- }
-
- mutex_unlock(&dev->struct_mutex);
-
- wake_up_all(&priv->fence_event);
-}
-
-void __msm_fence_worker(struct work_struct *work)
-{
- struct msm_fence_cb *cb = container_of(work, struct msm_fence_cb, work);
- cb->func(cb);
-}
-
-/*
* DRM ioctls:
*/
@@ -851,7 +622,7 @@
return -EINVAL;
}
- obj = drm_gem_object_lookup(dev, file, args->handle);
+ obj = drm_gem_object_lookup(file, args->handle);
if (!obj)
return -ENOENT;
@@ -869,7 +640,7 @@
struct drm_gem_object *obj;
int ret;
- obj = drm_gem_object_lookup(dev, file, args->handle);
+ obj = drm_gem_object_lookup(file, args->handle);
if (!obj)
return -ENOENT;
@@ -890,7 +661,7 @@
if (args->pad)
return -EINVAL;
- obj = drm_gem_object_lookup(dev, file, args->handle);
+ obj = drm_gem_object_lookup(file, args->handle);
if (!obj)
return -ENOENT;
@@ -904,6 +675,7 @@
static int msm_ioctl_wait_fence(struct drm_device *dev, void *data,
struct drm_file *file)
{
+ struct msm_drm_private *priv = dev->dev_private;
struct drm_msm_wait_fence *args = data;
ktime_t timeout = to_ktime(args->timeout);
@@ -912,7 +684,10 @@
return -EINVAL;
}
- return msm_wait_fence(dev, args->fence, &timeout, true);
+ if (!priv->gpu)
+ return 0;
+
+ return msm_wait_fence(priv->gpu->fctx, args->fence, &timeout, true);
}
static const struct drm_ioctl_desc msm_ioctls[] = {
@@ -952,8 +727,6 @@
DRIVER_RENDER |
DRIVER_ATOMIC |
DRIVER_MODESET,
- .load = msm_load,
- .unload = msm_unload,
.open = msm_open,
.preclose = msm_preclose,
.lastclose = msm_lastclose,
@@ -1053,12 +826,12 @@
static int msm_drm_bind(struct device *dev)
{
- return drm_platform_init(&msm_driver, to_platform_device(dev));
+ return msm_drm_init(dev, &msm_driver);
}
static void msm_drm_unbind(struct device *dev)
{
- drm_put_dev(platform_get_drvdata(to_platform_device(dev)));
+ msm_drm_uninit(dev);
}
static const struct component_master_ops msm_drm_ops = {
diff --git a/drivers/gpu/drm/msm/msm_drv.h b/drivers/gpu/drm/msm/msm_drv.h
index 870dbe5..5b2963f 100644
--- a/drivers/gpu/drm/msm/msm_drv.h
+++ b/drivers/gpu/drm/msm/msm_drv.h
@@ -49,6 +49,8 @@
struct msm_rd_state;
struct msm_perf_state;
struct msm_gem_submit;
+struct msm_fence_context;
+struct msm_fence_cb;
#define NUM_DOMAINS 2 /* one for KMS, then one per gpu core (?) */
@@ -100,9 +102,6 @@
struct drm_fb_helper *fbdev;
- uint32_t next_fence, completed_fence;
- wait_queue_head_t fence_event;
-
struct msm_rd_state *rd;
struct msm_perf_state *perf;
@@ -110,9 +109,7 @@
struct list_head inactive_list;
struct workqueue_struct *wq;
-
- /* callbacks deferred until bo is inactive: */
- struct list_head fence_cbs;
+ struct workqueue_struct *atomic_wq;
/* crtcs pending async atomic updates: */
uint32_t pending_crtcs;
@@ -157,33 +154,14 @@
uint32_t pixel_format;
};
-/* callback from wq once fence has passed: */
-struct msm_fence_cb {
- struct work_struct work;
- uint32_t fence;
- void (*func)(struct msm_fence_cb *cb);
-};
-
-void __msm_fence_worker(struct work_struct *work);
-
-#define INIT_FENCE_CB(_cb, _func) do { \
- INIT_WORK(&(_cb)->work, __msm_fence_worker); \
- (_cb)->func = _func; \
- } while (0)
-
int msm_atomic_check(struct drm_device *dev,
struct drm_atomic_state *state);
int msm_atomic_commit(struct drm_device *dev,
- struct drm_atomic_state *state, bool async);
+ struct drm_atomic_state *state, bool nonblock);
int msm_register_mmu(struct drm_device *dev, struct msm_mmu *mmu);
-int msm_wait_fence(struct drm_device *dev, uint32_t fence,
- ktime_t *timeout, bool interruptible);
-int msm_queue_fence_cb(struct drm_device *dev,
- struct msm_fence_cb *cb, uint32_t fence);
-void msm_update_fence(struct drm_device *dev, uint32_t fence);
-
+void msm_gem_submit_free(struct msm_gem_submit *submit);
int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
struct drm_file *file);
@@ -213,13 +191,12 @@
void msm_gem_prime_unpin(struct drm_gem_object *obj);
void *msm_gem_vaddr_locked(struct drm_gem_object *obj);
void *msm_gem_vaddr(struct drm_gem_object *obj);
-int msm_gem_queue_inactive_cb(struct drm_gem_object *obj,
- struct msm_fence_cb *cb);
+int msm_gem_sync_object(struct drm_gem_object *obj,
+ struct msm_fence_context *fctx, bool exclusive);
void msm_gem_move_to_active(struct drm_gem_object *obj,
- struct msm_gpu *gpu, bool write, uint32_t fence);
+ struct msm_gpu *gpu, bool exclusive, struct fence *fence);
void msm_gem_move_to_inactive(struct drm_gem_object *obj);
-int msm_gem_cpu_prep(struct drm_gem_object *obj, uint32_t op,
- ktime_t *timeout);
+int msm_gem_cpu_prep(struct drm_gem_object *obj, uint32_t op, ktime_t *timeout);
int msm_gem_cpu_fini(struct drm_gem_object *obj);
void msm_gem_free_object(struct drm_gem_object *obj);
int msm_gem_new_handle(struct drm_device *dev, struct drm_file *file,
@@ -227,7 +204,7 @@
struct drm_gem_object *msm_gem_new(struct drm_device *dev,
uint32_t size, uint32_t flags);
struct drm_gem_object *msm_gem_import(struct drm_device *dev,
- uint32_t size, struct sg_table *sgt);
+ struct dma_buf *dmabuf, struct sg_table *sgt);
int msm_framebuffer_prepare(struct drm_framebuffer *fb, int id);
void msm_framebuffer_cleanup(struct drm_framebuffer *fb, int id);
@@ -303,12 +280,6 @@
#define DBG(fmt, ...) DRM_DEBUG(fmt"\n", ##__VA_ARGS__)
#define VERB(fmt, ...) if (0) DRM_DEBUG(fmt"\n", ##__VA_ARGS__)
-static inline bool fence_completed(struct drm_device *dev, uint32_t fence)
-{
- struct msm_drm_private *priv = dev->dev_private;
- return priv->completed_fence >= fence;
-}
-
static inline int align_pitch(int width, int bpp)
{
int bytespp = (bpp + 7) / 8;
@@ -327,5 +298,20 @@
/* for conditionally setting boolean flag(s): */
#define COND(bool, val) ((bool) ? (val) : 0)
+static inline unsigned long timeout_to_jiffies(const ktime_t *timeout)
+{
+ ktime_t now = ktime_get();
+ unsigned long remaining_jiffies;
+
+ if (ktime_compare(*timeout, now) < 0) {
+ remaining_jiffies = 0;
+ } else {
+ ktime_t rem = ktime_sub(*timeout, now);
+ struct timespec ts = ktime_to_timespec(rem);
+ remaining_jiffies = timespec_to_jiffies(&ts);
+ }
+
+ return remaining_jiffies;
+}
#endif /* __MSM_DRV_H__ */
diff --git a/drivers/gpu/drm/msm/msm_fb.c b/drivers/gpu/drm/msm/msm_fb.c
index a474d6c..461dc8b 100644
--- a/drivers/gpu/drm/msm/msm_fb.c
+++ b/drivers/gpu/drm/msm/msm_fb.c
@@ -77,7 +77,7 @@
seq_printf(m, "fb: %dx%d@%4.4s (%2d, ID:%d)\n",
fb->width, fb->height, (char *)&fb->pixel_format,
- fb->refcount.refcount.counter, fb->base.id);
+ drm_framebuffer_read_refcount(fb), fb->base.id);
for (i = 0; i < n; i++) {
seq_printf(m, " %d: offset=%d pitch=%d, obj: ",
@@ -145,8 +145,7 @@
int ret, i, n = drm_format_num_planes(mode_cmd->pixel_format);
for (i = 0; i < n; i++) {
- bos[i] = drm_gem_object_lookup(dev, file,
- mode_cmd->handles[i]);
+ bos[i] = drm_gem_object_lookup(file, mode_cmd->handles[i]);
if (!bos[i]) {
ret = -ENXIO;
goto out_unref;
diff --git a/drivers/gpu/drm/msm/msm_fence.c b/drivers/gpu/drm/msm/msm_fence.c
new file mode 100644
index 0000000..a9b9b1c
--- /dev/null
+++ b/drivers/gpu/drm/msm/msm_fence.c
@@ -0,0 +1,163 @@
+/*
+ * Copyright (C) 2013-2016 Red Hat
+ * Author: Rob Clark <robdclark@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/fence.h>
+
+#include "msm_drv.h"
+#include "msm_fence.h"
+
+
+struct msm_fence_context *
+msm_fence_context_alloc(struct drm_device *dev, const char *name)
+{
+ struct msm_fence_context *fctx;
+
+ fctx = kzalloc(sizeof(*fctx), GFP_KERNEL);
+ if (!fctx)
+ return ERR_PTR(-ENOMEM);
+
+ fctx->dev = dev;
+ fctx->name = name;
+ fctx->context = fence_context_alloc(1);
+ init_waitqueue_head(&fctx->event);
+ spin_lock_init(&fctx->spinlock);
+
+ return fctx;
+}
+
+void msm_fence_context_free(struct msm_fence_context *fctx)
+{
+ kfree(fctx);
+}
+
+static inline bool fence_completed(struct msm_fence_context *fctx, uint32_t fence)
+{
+ return (int32_t)(fctx->completed_fence - fence) >= 0;
+}
+
+/* legacy path for WAIT_FENCE ioctl: */
+int msm_wait_fence(struct msm_fence_context *fctx, uint32_t fence,
+ ktime_t *timeout, bool interruptible)
+{
+ int ret;
+
+ if (fence > fctx->last_fence) {
+ DRM_ERROR("%s: waiting on invalid fence: %u (of %u)\n",
+ fctx->name, fence, fctx->last_fence);
+ return -EINVAL;
+ }
+
+ if (!timeout) {
+ /* no-wait: */
+ ret = fence_completed(fctx, fence) ? 0 : -EBUSY;
+ } else {
+ unsigned long remaining_jiffies = timeout_to_jiffies(timeout);
+
+ if (interruptible)
+ ret = wait_event_interruptible_timeout(fctx->event,
+ fence_completed(fctx, fence),
+ remaining_jiffies);
+ else
+ ret = wait_event_timeout(fctx->event,
+ fence_completed(fctx, fence),
+ remaining_jiffies);
+
+ if (ret == 0) {
+ DBG("timeout waiting for fence: %u (completed: %u)",
+ fence, fctx->completed_fence);
+ ret = -ETIMEDOUT;
+ } else if (ret != -ERESTARTSYS) {
+ ret = 0;
+ }
+ }
+
+ return ret;
+}
+
+/* called from workqueue */
+void msm_update_fence(struct msm_fence_context *fctx, uint32_t fence)
+{
+ spin_lock(&fctx->spinlock);
+ fctx->completed_fence = max(fence, fctx->completed_fence);
+ spin_unlock(&fctx->spinlock);
+
+ wake_up_all(&fctx->event);
+}
+
+struct msm_fence {
+ struct msm_fence_context *fctx;
+ struct fence base;
+};
+
+static inline struct msm_fence *to_msm_fence(struct fence *fence)
+{
+ return container_of(fence, struct msm_fence, base);
+}
+
+static const char *msm_fence_get_driver_name(struct fence *fence)
+{
+ return "msm";
+}
+
+static const char *msm_fence_get_timeline_name(struct fence *fence)
+{
+ struct msm_fence *f = to_msm_fence(fence);
+ return f->fctx->name;
+}
+
+static bool msm_fence_enable_signaling(struct fence *fence)
+{
+ return true;
+}
+
+static bool msm_fence_signaled(struct fence *fence)
+{
+ struct msm_fence *f = to_msm_fence(fence);
+ return fence_completed(f->fctx, f->base.seqno);
+}
+
+static void msm_fence_release(struct fence *fence)
+{
+ struct msm_fence *f = to_msm_fence(fence);
+ kfree_rcu(f, base.rcu);
+}
+
+static const struct fence_ops msm_fence_ops = {
+ .get_driver_name = msm_fence_get_driver_name,
+ .get_timeline_name = msm_fence_get_timeline_name,
+ .enable_signaling = msm_fence_enable_signaling,
+ .signaled = msm_fence_signaled,
+ .wait = fence_default_wait,
+ .release = msm_fence_release,
+};
+
+struct fence *
+msm_fence_alloc(struct msm_fence_context *fctx)
+{
+ struct msm_fence *f;
+
+ f = kzalloc(sizeof(*f), GFP_KERNEL);
+ if (!f)
+ return ERR_PTR(-ENOMEM);
+
+ f->fctx = fctx;
+
+ fence_init(&f->base, &msm_fence_ops, &fctx->spinlock,
+ fctx->context, ++fctx->last_fence);
+
+ return &f->base;
+}
diff --git a/drivers/gpu/drm/msm/msm_fence.h b/drivers/gpu/drm/msm/msm_fence.h
new file mode 100644
index 0000000..ceb5b3d
--- /dev/null
+++ b/drivers/gpu/drm/msm/msm_fence.h
@@ -0,0 +1,46 @@
+/*
+ * Copyright (C) 2013-2016 Red Hat
+ * Author: Rob Clark <robdclark@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __MSM_FENCE_H__
+#define __MSM_FENCE_H__
+
+#include "msm_drv.h"
+
+struct msm_fence_context {
+ struct drm_device *dev;
+ const char *name;
+ unsigned context;
+ /* last_fence == completed_fence --> no pending work */
+ uint32_t last_fence; /* last assigned fence */
+ uint32_t completed_fence; /* last completed fence */
+ wait_queue_head_t event;
+ spinlock_t spinlock;
+};
+
+struct msm_fence_context * msm_fence_context_alloc(struct drm_device *dev,
+ const char *name);
+void msm_fence_context_free(struct msm_fence_context *fctx);
+
+int msm_wait_fence(struct msm_fence_context *fctx, uint32_t fence,
+ ktime_t *timeout, bool interruptible);
+int msm_queue_fence_cb(struct msm_fence_context *fctx,
+ struct msm_fence_cb *cb, uint32_t fence);
+void msm_update_fence(struct msm_fence_context *fctx, uint32_t fence);
+
+struct fence * msm_fence_alloc(struct msm_fence_context *fctx);
+
+#endif
diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c
index 3cedb8d..7daf4054 100644
--- a/drivers/gpu/drm/msm/msm_gem.c
+++ b/drivers/gpu/drm/msm/msm_gem.c
@@ -21,6 +21,7 @@
#include <linux/pfn_t.h>
#include "msm_drv.h"
+#include "msm_fence.h"
#include "msm_gem.h"
#include "msm_gpu.h"
#include "msm_mmu.h"
@@ -373,7 +374,7 @@
int ret = 0;
/* GEM does all our handle to object mapping */
- obj = drm_gem_object_lookup(dev, file, handle);
+ obj = drm_gem_object_lookup(file, handle);
if (obj == NULL) {
ret = -ENOENT;
goto fail;
@@ -410,27 +411,62 @@
return ret;
}
-/* setup callback for when bo is no longer busy..
- * TODO probably want to differentiate read vs write..
- */
-int msm_gem_queue_inactive_cb(struct drm_gem_object *obj,
- struct msm_fence_cb *cb)
+/* must be called before _move_to_active().. */
+int msm_gem_sync_object(struct drm_gem_object *obj,
+ struct msm_fence_context *fctx, bool exclusive)
{
struct msm_gem_object *msm_obj = to_msm_bo(obj);
- uint32_t fence = msm_gem_fence(msm_obj,
- MSM_PREP_READ | MSM_PREP_WRITE);
- return msm_queue_fence_cb(obj->dev, cb, fence);
+ struct reservation_object_list *fobj;
+ struct fence *fence;
+ int i, ret;
+
+ if (!exclusive) {
+ /* NOTE: _reserve_shared() must happen before _add_shared_fence(),
+ * which makes this a slightly strange place to call it. OTOH this
+ * is a convenient can-fail point to hook it in. (And similar to
+ * how etnaviv and nouveau handle this.)
+ */
+ ret = reservation_object_reserve_shared(msm_obj->resv);
+ if (ret)
+ return ret;
+ }
+
+ fobj = reservation_object_get_list(msm_obj->resv);
+ if (!fobj || (fobj->shared_count == 0)) {
+ fence = reservation_object_get_excl(msm_obj->resv);
+ /* don't need to wait on our own fences, since ring is fifo */
+ if (fence && (fence->context != fctx->context)) {
+ ret = fence_wait(fence, true);
+ if (ret)
+ return ret;
+ }
+ }
+
+ if (!exclusive || !fobj)
+ return 0;
+
+ for (i = 0; i < fobj->shared_count; i++) {
+ fence = rcu_dereference_protected(fobj->shared[i],
+ reservation_object_held(msm_obj->resv));
+ if (fence->context != fctx->context) {
+ ret = fence_wait(fence, true);
+ if (ret)
+ return ret;
+ }
+ }
+
+ return 0;
}
void msm_gem_move_to_active(struct drm_gem_object *obj,
- struct msm_gpu *gpu, bool write, uint32_t fence)
+ struct msm_gpu *gpu, bool exclusive, struct fence *fence)
{
struct msm_gem_object *msm_obj = to_msm_bo(obj);
msm_obj->gpu = gpu;
- if (write)
- msm_obj->write_fence = fence;
+ if (exclusive)
+ reservation_object_add_excl_fence(msm_obj->resv, fence);
else
- msm_obj->read_fence = fence;
+ reservation_object_add_shared_fence(msm_obj->resv, fence);
list_del_init(&msm_obj->mm_list);
list_add_tail(&msm_obj->mm_list, &gpu->active_list);
}
@@ -444,30 +480,30 @@
WARN_ON(!mutex_is_locked(&dev->struct_mutex));
msm_obj->gpu = NULL;
- msm_obj->read_fence = 0;
- msm_obj->write_fence = 0;
list_del_init(&msm_obj->mm_list);
list_add_tail(&msm_obj->mm_list, &priv->inactive_list);
}
int msm_gem_cpu_prep(struct drm_gem_object *obj, uint32_t op, ktime_t *timeout)
{
- struct drm_device *dev = obj->dev;
struct msm_gem_object *msm_obj = to_msm_bo(obj);
- int ret = 0;
+ bool write = !!(op & MSM_PREP_WRITE);
- if (is_active(msm_obj)) {
- uint32_t fence = msm_gem_fence(msm_obj, op);
+ if (op & MSM_PREP_NOSYNC) {
+ if (!reservation_object_test_signaled_rcu(msm_obj->resv, write))
+ return -EBUSY;
+ } else {
+ int ret;
- if (op & MSM_PREP_NOSYNC)
- timeout = NULL;
-
- ret = msm_wait_fence(dev, fence, timeout, true);
+ ret = reservation_object_wait_timeout_rcu(msm_obj->resv, write,
+ true, timeout_to_jiffies(timeout));
+ if (ret <= 0)
+ return ret == 0 ? -ETIMEDOUT : ret;
}
/* TODO cache maintenance */
- return ret;
+ return 0;
}
int msm_gem_cpu_fini(struct drm_gem_object *obj)
@@ -477,18 +513,46 @@
}
#ifdef CONFIG_DEBUG_FS
+static void describe_fence(struct fence *fence, const char *type,
+ struct seq_file *m)
+{
+ if (!fence_is_signaled(fence))
+ seq_printf(m, "\t%9s: %s %s seq %u\n", type,
+ fence->ops->get_driver_name(fence),
+ fence->ops->get_timeline_name(fence),
+ fence->seqno);
+}
+
void msm_gem_describe(struct drm_gem_object *obj, struct seq_file *m)
{
- struct drm_device *dev = obj->dev;
struct msm_gem_object *msm_obj = to_msm_bo(obj);
+ struct reservation_object *robj = msm_obj->resv;
+ struct reservation_object_list *fobj;
+ struct fence *fence;
uint64_t off = drm_vma_node_start(&obj->vma_node);
- WARN_ON(!mutex_is_locked(&dev->struct_mutex));
- seq_printf(m, "%08x: %c(r=%u,w=%u) %2d (%2d) %08llx %p %zu\n",
+ WARN_ON(!mutex_is_locked(&obj->dev->struct_mutex));
+
+ seq_printf(m, "%08x: %c %2d (%2d) %08llx %p %zu\n",
msm_obj->flags, is_active(msm_obj) ? 'A' : 'I',
- msm_obj->read_fence, msm_obj->write_fence,
obj->name, obj->refcount.refcount.counter,
off, msm_obj->vaddr, obj->size);
+
+ rcu_read_lock();
+ fobj = rcu_dereference(robj->fence);
+ if (fobj) {
+ unsigned int i, shared_count = fobj->shared_count;
+
+ for (i = 0; i < shared_count; i++) {
+ fence = rcu_dereference(fobj->shared[i]);
+ describe_fence(fence, "Shared", m);
+ }
+ }
+
+ fence = rcu_dereference(robj->fence_excl);
+ if (fence)
+ describe_fence(fence, "Exclusive", m);
+ rcu_read_unlock();
}
void msm_gem_describe_objects(struct list_head *list, struct seq_file *m)
@@ -583,6 +647,7 @@
static int msm_gem_new_impl(struct drm_device *dev,
uint32_t size, uint32_t flags,
+ struct reservation_object *resv,
struct drm_gem_object **obj)
{
struct msm_drm_private *priv = dev->dev_private;
@@ -622,8 +687,12 @@
msm_obj->flags = flags;
- msm_obj->resv = &msm_obj->_resv;
- reservation_object_init(msm_obj->resv);
+ if (resv) {
+ msm_obj->resv = resv;
+ } else {
+ msm_obj->resv = &msm_obj->_resv;
+ reservation_object_init(msm_obj->resv);
+ }
INIT_LIST_HEAD(&msm_obj->submit_entry);
list_add_tail(&msm_obj->mm_list, &priv->inactive_list);
@@ -643,7 +712,7 @@
size = PAGE_ALIGN(size);
- ret = msm_gem_new_impl(dev, size, flags, &obj);
+ ret = msm_gem_new_impl(dev, size, flags, NULL, &obj);
if (ret)
goto fail;
@@ -665,10 +734,11 @@
}
struct drm_gem_object *msm_gem_import(struct drm_device *dev,
- uint32_t size, struct sg_table *sgt)
+ struct dma_buf *dmabuf, struct sg_table *sgt)
{
struct msm_gem_object *msm_obj;
struct drm_gem_object *obj;
+ uint32_t size;
int ret, npages;
/* if we don't have IOMMU, don't bother pretending we can import: */
@@ -677,9 +747,9 @@
return ERR_PTR(-EINVAL);
}
- size = PAGE_ALIGN(size);
+ size = PAGE_ALIGN(dmabuf->size);
- ret = msm_gem_new_impl(dev, size, MSM_BO_WC, &obj);
+ ret = msm_gem_new_impl(dev, size, MSM_BO_WC, dmabuf->resv, &obj);
if (ret)
goto fail;
diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h
index 6fc59bf..9facd4b 100644
--- a/drivers/gpu/drm/msm/msm_gem.h
+++ b/drivers/gpu/drm/msm/msm_gem.h
@@ -39,7 +39,6 @@
*/
struct list_head mm_list;
struct msm_gpu *gpu; /* non-null if active */
- uint32_t read_fence, write_fence;
/* Transiently in the process of submit ioctl, objects associated
* with the submit are on submit->bo_list.. this only lasts for
@@ -73,19 +72,6 @@
return msm_obj->gpu != NULL;
}
-static inline uint32_t msm_gem_fence(struct msm_gem_object *msm_obj,
- uint32_t op)
-{
- uint32_t fence = 0;
-
- if (op & MSM_PREP_READ)
- fence = msm_obj->write_fence;
- if (op & MSM_PREP_WRITE)
- fence = max(fence, msm_obj->read_fence);
-
- return fence;
-}
-
#define MAX_CMDS 4
/* Created per submit-ioctl, to track bo's and cmdstream bufs, etc,
@@ -99,8 +85,9 @@
struct list_head node; /* node in gpu submit_list */
struct list_head bo_list;
struct ww_acquire_ctx ticket;
- uint32_t fence;
- bool valid;
+ struct fence *fence;
+ struct pid *pid; /* submitting process */
+ bool valid; /* true if no cmdstream patching needed */
unsigned int nr_cmds;
unsigned int nr_bos;
struct {
diff --git a/drivers/gpu/drm/msm/msm_gem_prime.c b/drivers/gpu/drm/msm/msm_gem_prime.c
index 121975b..6b90890 100644
--- a/drivers/gpu/drm/msm/msm_gem_prime.c
+++ b/drivers/gpu/drm/msm/msm_gem_prime.c
@@ -55,7 +55,7 @@
struct drm_gem_object *msm_gem_prime_import_sg_table(struct drm_device *dev,
struct dma_buf_attachment *attach, struct sg_table *sg)
{
- return msm_gem_import(dev, attach->dmabuf->size, sg);
+ return msm_gem_import(dev, attach->dmabuf, sg);
}
int msm_gem_prime_pin(struct drm_gem_object *obj)
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index 43d2181..b89ca51 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -24,15 +24,10 @@
*/
/* make sure these don't conflict w/ MSM_SUBMIT_BO_x */
-#define BO_VALID 0x8000
+#define BO_VALID 0x8000 /* is current addr in cmdstream correct/valid? */
#define BO_LOCKED 0x4000
#define BO_PINNED 0x2000
-static inline void __user *to_user_ptr(u64 address)
-{
- return (void __user *)(uintptr_t)address;
-}
-
static struct msm_gem_submit *submit_create(struct drm_device *dev,
struct msm_gpu *gpu, int nr)
{
@@ -40,21 +35,31 @@
int sz = sizeof(*submit) + (nr * sizeof(submit->bos[0]));
submit = kmalloc(sz, GFP_TEMPORARY | __GFP_NOWARN | __GFP_NORETRY);
- if (submit) {
- submit->dev = dev;
- submit->gpu = gpu;
+ if (!submit)
+ return NULL;
- /* initially, until copy_from_user() and bo lookup succeeds: */
- submit->nr_bos = 0;
- submit->nr_cmds = 0;
+ submit->dev = dev;
+ submit->gpu = gpu;
+ submit->pid = get_pid(task_pid(current));
- INIT_LIST_HEAD(&submit->bo_list);
- ww_acquire_init(&submit->ticket, &reservation_ww_class);
- }
+ /* initially, until copy_from_user() and bo lookup succeeds: */
+ submit->nr_bos = 0;
+ submit->nr_cmds = 0;
+
+ INIT_LIST_HEAD(&submit->bo_list);
+ ww_acquire_init(&submit->ticket, &reservation_ww_class);
return submit;
}
+void msm_gem_submit_free(struct msm_gem_submit *submit)
+{
+ fence_put(submit->fence);
+ list_del(&submit->node);
+ put_pid(submit->pid);
+ kfree(submit);
+}
+
static int submit_lookup_objects(struct msm_gem_submit *submit,
struct drm_msm_gem_submit *args, struct drm_file *file)
{
@@ -68,7 +73,7 @@
struct drm_gem_object *obj;
struct msm_gem_object *msm_obj;
void __user *userptr =
- to_user_ptr(args->bos + (i * sizeof(submit_bo)));
+ u64_to_user_ptr(args->bos + (i * sizeof(submit_bo)));
ret = copy_from_user(&submit_bo, userptr, sizeof(submit_bo));
if (ret) {
@@ -136,16 +141,13 @@
}
/* This is where we make sure all the bo's are reserved and pin'd: */
-static int submit_validate_objects(struct msm_gem_submit *submit)
+static int submit_lock_objects(struct msm_gem_submit *submit)
{
int contended, slow_locked = -1, i, ret = 0;
retry:
- submit->valid = true;
-
for (i = 0; i < submit->nr_bos; i++) {
struct msm_gem_object *msm_obj = submit->bos[i].obj;
- uint32_t iova;
if (slow_locked == i)
slow_locked = -1;
@@ -159,30 +161,6 @@
goto fail;
submit->bos[i].flags |= BO_LOCKED;
}
-
-
- /* if locking succeeded, pin bo: */
- ret = msm_gem_get_iova_locked(&msm_obj->base,
- submit->gpu->id, &iova);
-
- /* this would break the logic in the fail path.. there is no
- * reason for this to happen, but just to be on the safe side
- * let's notice if this starts happening in the future:
- */
- WARN_ON(ret == -EDEADLK);
-
- if (ret)
- goto fail;
-
- submit->bos[i].flags |= BO_PINNED;
-
- if (iova == submit->bos[i].iova) {
- submit->bos[i].flags |= BO_VALID;
- } else {
- submit->bos[i].iova = iova;
- submit->bos[i].flags &= ~BO_VALID;
- submit->valid = false;
- }
}
ww_acquire_done(&submit->ticket);
@@ -211,6 +189,54 @@
return ret;
}
+static int submit_fence_sync(struct msm_gem_submit *submit)
+{
+ int i, ret = 0;
+
+ for (i = 0; i < submit->nr_bos; i++) {
+ struct msm_gem_object *msm_obj = submit->bos[i].obj;
+ bool write = submit->bos[i].flags & MSM_SUBMIT_BO_WRITE;
+
+ ret = msm_gem_sync_object(&msm_obj->base, submit->gpu->fctx, write);
+ if (ret)
+ break;
+ }
+
+ return ret;
+}
+
+static int submit_pin_objects(struct msm_gem_submit *submit)
+{
+ int i, ret = 0;
+
+ submit->valid = true;
+
+ for (i = 0; i < submit->nr_bos; i++) {
+ struct msm_gem_object *msm_obj = submit->bos[i].obj;
+ uint32_t iova;
+
+ /* if locking succeeded, pin bo: */
+ ret = msm_gem_get_iova_locked(&msm_obj->base,
+ submit->gpu->id, &iova);
+
+ if (ret)
+ break;
+
+ submit->bos[i].flags |= BO_PINNED;
+
+ if (iova == submit->bos[i].iova) {
+ submit->bos[i].flags |= BO_VALID;
+ } else {
+ submit->bos[i].iova = iova;
+ /* iova changed, so address in cmdstream is not valid: */
+ submit->bos[i].flags &= ~BO_VALID;
+ submit->valid = false;
+ }
+ }
+
+ return ret;
+}
+
static int submit_bo(struct msm_gem_submit *submit, uint32_t idx,
struct msm_gem_object **obj, uint32_t *iova, bool *valid)
{
@@ -257,7 +283,7 @@
for (i = 0; i < nr_relocs; i++) {
struct drm_msm_gem_submit_reloc submit_reloc;
void __user *userptr =
- to_user_ptr(relocs + (i * sizeof(submit_reloc)));
+ u64_to_user_ptr(relocs + (i * sizeof(submit_reloc)));
uint32_t iova, off;
bool valid;
@@ -302,7 +328,7 @@
return 0;
}
-static void submit_cleanup(struct msm_gem_submit *submit, bool fail)
+static void submit_cleanup(struct msm_gem_submit *submit)
{
unsigned i;
@@ -349,14 +375,22 @@
if (ret)
goto out;
- ret = submit_validate_objects(submit);
+ ret = submit_lock_objects(submit);
+ if (ret)
+ goto out;
+
+ ret = submit_fence_sync(submit);
+ if (ret)
+ goto out;
+
+ ret = submit_pin_objects(submit);
if (ret)
goto out;
for (i = 0; i < args->nr_cmds; i++) {
struct drm_msm_gem_submit_cmd submit_cmd;
void __user *userptr =
- to_user_ptr(args->cmds + (i * sizeof(submit_cmd)));
+ u64_to_user_ptr(args->cmds + (i * sizeof(submit_cmd)));
struct msm_gem_object *msm_obj;
uint32_t iova;
@@ -415,10 +449,12 @@
ret = msm_gpu_submit(gpu, submit, ctx);
- args->fence = submit->fence;
+ args->fence = submit->fence->seqno;
out:
- submit_cleanup(submit, !!ret);
+ submit_cleanup(submit);
+ if (ret)
+ msm_gem_submit_free(submit);
mutex_unlock(&dev->struct_mutex);
return ret;
}
diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c
index 6b02ada..36ed53e 100644
--- a/drivers/gpu/drm/msm/msm_gpu.c
+++ b/drivers/gpu/drm/msm/msm_gpu.c
@@ -18,6 +18,7 @@
#include "msm_gpu.h"
#include "msm_gem.h"
#include "msm_mmu.h"
+#include "msm_fence.h"
/*
@@ -265,22 +266,38 @@
* Hangcheck detection for locked gpu:
*/
-static void retire_submits(struct msm_gpu *gpu, uint32_t fence);
+static void retire_submits(struct msm_gpu *gpu);
static void recover_worker(struct work_struct *work)
{
struct msm_gpu *gpu = container_of(work, struct msm_gpu, recover_work);
struct drm_device *dev = gpu->dev;
+ struct msm_gem_submit *submit;
+ uint32_t fence = gpu->funcs->last_fence(gpu);
- dev_err(dev->dev, "%s: hangcheck recover!\n", gpu->name);
+ msm_update_fence(gpu->fctx, fence + 1);
mutex_lock(&dev->struct_mutex);
- if (msm_gpu_active(gpu)) {
- struct msm_gem_submit *submit;
- uint32_t fence = gpu->funcs->last_fence(gpu);
+ dev_err(dev->dev, "%s: hangcheck recover!\n", gpu->name);
+ list_for_each_entry(submit, &gpu->submit_list, node) {
+ if (submit->fence->seqno == (fence + 1)) {
+ struct task_struct *task;
+
+ rcu_read_lock();
+ task = pid_task(submit->pid, PIDTYPE_PID);
+ if (task) {
+ dev_err(dev->dev, "%s: offending task: %s\n",
+ gpu->name, task->comm);
+ }
+ rcu_read_unlock();
+ break;
+ }
+ }
+
+ if (msm_gpu_active(gpu)) {
/* retire completed submits, plus the one that hung: */
- retire_submits(gpu, fence + 1);
+ retire_submits(gpu);
inactive_cancel(gpu);
gpu->funcs->recover(gpu);
@@ -290,6 +307,7 @@
gpu->funcs->submit(gpu, submit, NULL);
}
}
+
mutex_unlock(&dev->struct_mutex);
msm_gpu_retire(gpu);
@@ -312,7 +330,7 @@
if (fence != gpu->hangcheck_fence) {
/* some progress has been made.. ya! */
gpu->hangcheck_fence = fence;
- } else if (fence < gpu->submitted_fence) {
+ } else if (fence < gpu->fctx->last_fence) {
/* no progress and not done.. hung! */
gpu->hangcheck_fence = fence;
dev_err(dev->dev, "%s: hangcheck detected gpu lockup!\n",
@@ -320,12 +338,12 @@
dev_err(dev->dev, "%s: completed fence: %u\n",
gpu->name, fence);
dev_err(dev->dev, "%s: submitted fence: %u\n",
- gpu->name, gpu->submitted_fence);
+ gpu->name, gpu->fctx->last_fence);
queue_work(priv->wq, &gpu->recover_work);
}
/* if still more pending work, reset the hangcheck timer: */
- if (gpu->submitted_fence > gpu->hangcheck_fence)
+ if (gpu->fctx->last_fence > gpu->hangcheck_fence)
hangcheck_timer_reset(gpu);
/* workaround for missing irq: */
@@ -431,7 +449,22 @@
* Cmdstream submission/retirement:
*/
-static void retire_submits(struct msm_gpu *gpu, uint32_t fence)
+static void retire_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit)
+{
+ int i;
+
+ for (i = 0; i < submit->nr_bos; i++) {
+ struct msm_gem_object *msm_obj = submit->bos[i].obj;
+ /* move to inactive: */
+ msm_gem_move_to_inactive(&msm_obj->base);
+ msm_gem_put_iova(&msm_obj->base, gpu->id);
+ drm_gem_object_unreference(&msm_obj->base);
+ }
+
+ msm_gem_submit_free(submit);
+}
+
+static void retire_submits(struct msm_gpu *gpu)
{
struct drm_device *dev = gpu->dev;
@@ -443,9 +476,8 @@
submit = list_first_entry(&gpu->submit_list,
struct msm_gem_submit, node);
- if (submit->fence <= fence) {
- list_del(&submit->node);
- kfree(submit);
+ if (fence_is_signaled(submit->fence)) {
+ retire_submit(gpu, submit);
} else {
break;
}
@@ -458,29 +490,10 @@
struct drm_device *dev = gpu->dev;
uint32_t fence = gpu->funcs->last_fence(gpu);
- msm_update_fence(gpu->dev, fence);
+ msm_update_fence(gpu->fctx, fence);
mutex_lock(&dev->struct_mutex);
-
- retire_submits(gpu, fence);
-
- while (!list_empty(&gpu->active_list)) {
- struct msm_gem_object *obj;
-
- obj = list_first_entry(&gpu->active_list,
- struct msm_gem_object, mm_list);
-
- if ((obj->read_fence <= fence) &&
- (obj->write_fence <= fence)) {
- /* move to inactive: */
- msm_gem_move_to_inactive(&obj->base);
- msm_gem_put_iova(&obj->base, gpu->id);
- drm_gem_object_unreference(&obj->base);
- } else {
- break;
- }
- }
-
+ retire_submits(gpu);
mutex_unlock(&dev->struct_mutex);
if (!msm_gpu_active(gpu))
@@ -505,9 +518,12 @@
WARN_ON(!mutex_is_locked(&dev->struct_mutex));
- submit->fence = ++priv->next_fence;
-
- gpu->submitted_fence = submit->fence;
+ submit->fence = msm_fence_alloc(gpu->fctx);
+ if (IS_ERR(submit->fence)) {
+ ret = PTR_ERR(submit->fence);
+ submit->fence = NULL;
+ return ret;
+ }
inactive_cancel(gpu);
@@ -515,40 +531,34 @@
msm_rd_dump_submit(submit);
- gpu->submitted_fence = submit->fence;
-
update_sw_cntrs(gpu);
for (i = 0; i < submit->nr_bos; i++) {
struct msm_gem_object *msm_obj = submit->bos[i].obj;
+ uint32_t iova;
/* can't happen yet.. but when we add 2d support we'll have
* to deal w/ cross-ring synchronization:
*/
WARN_ON(is_active(msm_obj) && (msm_obj->gpu != gpu));
- if (!is_active(msm_obj)) {
- uint32_t iova;
-
- /* ring takes a reference to the bo and iova: */
- drm_gem_object_reference(&msm_obj->base);
- msm_gem_get_iova_locked(&msm_obj->base,
- submit->gpu->id, &iova);
- }
-
- if (submit->bos[i].flags & MSM_SUBMIT_BO_READ)
- msm_gem_move_to_active(&msm_obj->base, gpu, false, submit->fence);
+ /* submit takes a reference to the bo and iova until retired: */
+ drm_gem_object_reference(&msm_obj->base);
+ msm_gem_get_iova_locked(&msm_obj->base,
+ submit->gpu->id, &iova);
if (submit->bos[i].flags & MSM_SUBMIT_BO_WRITE)
msm_gem_move_to_active(&msm_obj->base, gpu, true, submit->fence);
+ else if (submit->bos[i].flags & MSM_SUBMIT_BO_READ)
+ msm_gem_move_to_active(&msm_obj->base, gpu, false, submit->fence);
}
- ret = gpu->funcs->submit(gpu, submit, ctx);
+ gpu->funcs->submit(gpu, submit, ctx);
priv->lastctx = ctx;
hangcheck_timer_reset(gpu);
- return ret;
+ return 0;
}
/*
@@ -580,6 +590,12 @@
gpu->funcs = funcs;
gpu->name = name;
gpu->inactive = true;
+ gpu->fctx = msm_fence_context_alloc(drm, name);
+ if (IS_ERR(gpu->fctx)) {
+ ret = PTR_ERR(gpu->fctx);
+ gpu->fctx = NULL;
+ goto fail;
+ }
INIT_LIST_HEAD(&gpu->active_list);
INIT_WORK(&gpu->retire_work, retire_worker);
@@ -700,4 +716,7 @@
if (gpu->mmu)
gpu->mmu->funcs->destroy(gpu->mmu);
+
+ if (gpu->fctx)
+ msm_fence_context_free(gpu->fctx);
}
diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h
index 2bbe85a..c902283 100644
--- a/drivers/gpu/drm/msm/msm_gpu.h
+++ b/drivers/gpu/drm/msm/msm_gpu.h
@@ -22,6 +22,7 @@
#include <linux/regulator/consumer.h>
#include "msm_drv.h"
+#include "msm_fence.h"
#include "msm_ringbuffer.h"
struct msm_gem_submit;
@@ -46,7 +47,7 @@
int (*hw_init)(struct msm_gpu *gpu);
int (*pm_suspend)(struct msm_gpu *gpu);
int (*pm_resume)(struct msm_gpu *gpu);
- int (*submit)(struct msm_gpu *gpu, struct msm_gem_submit *submit,
+ void (*submit)(struct msm_gpu *gpu, struct msm_gem_submit *submit,
struct msm_file_private *ctx);
void (*flush)(struct msm_gpu *gpu);
void (*idle)(struct msm_gpu *gpu);
@@ -77,13 +78,15 @@
const struct msm_gpu_perfcntr *perfcntrs;
uint32_t num_perfcntrs;
+ /* ringbuffer: */
struct msm_ringbuffer *rb;
uint32_t rb_iova;
/* list of GEM active objects: */
struct list_head active_list;
- uint32_t submitted_fence;
+ /* fencing: */
+ struct msm_fence_context *fctx;
/* is gpu powered/active? */
int active_cnt;
@@ -125,7 +128,7 @@
static inline bool msm_gpu_active(struct msm_gpu *gpu)
{
- return gpu->submitted_fence > gpu->funcs->last_fence(gpu);
+ return gpu->fctx->last_fence > gpu->funcs->last_fence(gpu);
}
/* Perf-Counters:
diff --git a/drivers/gpu/drm/msm/msm_rd.c b/drivers/gpu/drm/msm/msm_rd.c
index 9a78c48..b48f73a 100644
--- a/drivers/gpu/drm/msm/msm_rd.c
+++ b/drivers/gpu/drm/msm/msm_rd.c
@@ -296,7 +296,7 @@
n = snprintf(msg, sizeof(msg), "%.*s/%d: fence=%u",
TASK_COMM_LEN, current->comm, task_pid_nr(current),
- submit->fence);
+ submit->fence->seqno);
rd_write_section(rd, RD_CMD, msg, ALIGN(n, 4));
diff --git a/drivers/gpu/drm/nouveau/dispnv04/arb.c b/drivers/gpu/drm/nouveau/dispnv04/arb.c
index 82bd465..a555681 100644
--- a/drivers/gpu/drm/nouveau/dispnv04/arb.c
+++ b/drivers/gpu/drm/nouveau/dispnv04/arb.c
@@ -23,7 +23,7 @@
#include <drm/drmP.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_reg.h"
#include "hw.h"
diff --git a/drivers/gpu/drm/nouveau/dispnv04/crtc.c b/drivers/gpu/drm/nouveau/dispnv04/crtc.c
index 55ccbf0..6f318c5 100644
--- a/drivers/gpu/drm/nouveau/dispnv04/crtc.c
+++ b/drivers/gpu/drm/nouveau/dispnv04/crtc.c
@@ -28,8 +28,9 @@
#include <drm/drm_crtc_helper.h>
#include <drm/drm_plane_helper.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_reg.h"
+#include "nouveau_ttm.h"
#include "nouveau_bo.h"
#include "nouveau_gem.h"
#include "nouveau_encoder.h"
@@ -995,7 +996,7 @@
if (width != 64 || height != 64)
return -EINVAL;
- gem = drm_gem_object_lookup(dev, file_priv, buffer_handle);
+ gem = drm_gem_object_lookup(file_priv, buffer_handle);
if (!gem)
return -ENOENT;
cursor = nouveau_gem_object(gem);
diff --git a/drivers/gpu/drm/nouveau/dispnv04/cursor.c b/drivers/gpu/drm/nouveau/dispnv04/cursor.c
index 4e61173..c83116a 100644
--- a/drivers/gpu/drm/nouveau/dispnv04/cursor.c
+++ b/drivers/gpu/drm/nouveau/dispnv04/cursor.c
@@ -1,6 +1,6 @@
#include <drm/drmP.h>
#include <drm/drm_mode.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_reg.h"
#include "nouveau_crtc.h"
#include "hw.h"
diff --git a/drivers/gpu/drm/nouveau/dispnv04/dac.c b/drivers/gpu/drm/nouveau/dispnv04/dac.c
index b48eec3..b6cc776 100644
--- a/drivers/gpu/drm/nouveau/dispnv04/dac.c
+++ b/drivers/gpu/drm/nouveau/dispnv04/dac.c
@@ -27,7 +27,7 @@
#include <drm/drmP.h>
#include <drm/drm_crtc_helper.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_encoder.h"
#include "nouveau_connector.h"
#include "nouveau_crtc.h"
diff --git a/drivers/gpu/drm/nouveau/dispnv04/dfp.c b/drivers/gpu/drm/nouveau/dispnv04/dfp.c
index 05bfd15..c2947ef 100644
--- a/drivers/gpu/drm/nouveau/dispnv04/dfp.c
+++ b/drivers/gpu/drm/nouveau/dispnv04/dfp.c
@@ -27,7 +27,7 @@
#include <drm/drmP.h>
#include <drm/drm_crtc_helper.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_reg.h"
#include "nouveau_encoder.h"
#include "nouveau_connector.h"
diff --git a/drivers/gpu/drm/nouveau/dispnv04/disp.c b/drivers/gpu/drm/nouveau/dispnv04/disp.c
index b4a6bc4..aea81a5 100644
--- a/drivers/gpu/drm/nouveau/dispnv04/disp.c
+++ b/drivers/gpu/drm/nouveau/dispnv04/disp.c
@@ -25,7 +25,7 @@
#include <drm/drmP.h>
#include <drm/drm_crtc_helper.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_reg.h"
#include "hw.h"
#include "nouveau_encoder.h"
diff --git a/drivers/gpu/drm/nouveau/dispnv04/disp.h b/drivers/gpu/drm/nouveau/dispnv04/disp.h
index 6c9a1e8..7030307 100644
--- a/drivers/gpu/drm/nouveau/dispnv04/disp.h
+++ b/drivers/gpu/drm/nouveau/dispnv04/disp.h
@@ -1,6 +1,6 @@
#ifndef __NV04_DISPLAY_H__
#define __NV04_DISPLAY_H__
-
+#include <subdev/bios.h>
#include <subdev/bios/pll.h>
#include "nouveau_display.h"
diff --git a/drivers/gpu/drm/nouveau/dispnv04/hw.c b/drivers/gpu/drm/nouveau/dispnv04/hw.c
index 956a833..74856a8 100644
--- a/drivers/gpu/drm/nouveau/dispnv04/hw.c
+++ b/drivers/gpu/drm/nouveau/dispnv04/hw.c
@@ -23,7 +23,7 @@
*/
#include <drm/drmP.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "hw.h"
#include <subdev/bios/pll.h>
diff --git a/drivers/gpu/drm/nouveau/dispnv04/overlay.c b/drivers/gpu/drm/nouveau/dispnv04/overlay.c
index aeebdd4..ec444ea 100644
--- a/drivers/gpu/drm/nouveau/dispnv04/overlay.c
+++ b/drivers/gpu/drm/nouveau/dispnv04/overlay.c
@@ -27,7 +27,7 @@
#include <drm/drm_crtc.h>
#include <drm/drm_fourcc.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_bo.h"
#include "nouveau_connector.h"
diff --git a/drivers/gpu/drm/nouveau/dispnv04/tvmodesnv17.c b/drivers/gpu/drm/nouveau/dispnv04/tvmodesnv17.c
index 903c473..2b83b2c 100644
--- a/drivers/gpu/drm/nouveau/dispnv04/tvmodesnv17.c
+++ b/drivers/gpu/drm/nouveau/dispnv04/tvmodesnv17.c
@@ -26,7 +26,7 @@
#include <drm/drmP.h>
#include <drm/drm_crtc_helper.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_encoder.h"
#include "nouveau_crtc.h"
#include "hw.h"
diff --git a/drivers/gpu/drm/nouveau/dispnv04/tvnv04.c b/drivers/gpu/drm/nouveau/dispnv04/tvnv04.c
index 54e9fb9e..477a8d0 100644
--- a/drivers/gpu/drm/nouveau/dispnv04/tvnv04.c
+++ b/drivers/gpu/drm/nouveau/dispnv04/tvnv04.c
@@ -25,7 +25,7 @@
*/
#include <drm/drmP.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_reg.h"
#include "nouveau_encoder.h"
#include "nouveau_connector.h"
diff --git a/drivers/gpu/drm/nouveau/dispnv04/tvnv17.c b/drivers/gpu/drm/nouveau/dispnv04/tvnv17.c
index 163317d..a665b78 100644
--- a/drivers/gpu/drm/nouveau/dispnv04/tvnv17.c
+++ b/drivers/gpu/drm/nouveau/dispnv04/tvnv17.c
@@ -26,7 +26,7 @@
#include <drm/drmP.h>
#include <drm/drm_crtc_helper.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_reg.h"
#include "nouveau_encoder.h"
#include "nouveau_connector.h"
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/core/device.h b/drivers/gpu/drm/nouveau/include/nvkm/core/device.h
index 4993a86..c612dc1 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/core/device.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/core/device.h
@@ -7,6 +7,7 @@
NVKM_SUBDEV_PCI,
NVKM_SUBDEV_VBIOS,
NVKM_SUBDEV_DEVINIT,
+ NVKM_SUBDEV_TOP,
NVKM_SUBDEV_IBUS,
NVKM_SUBDEV_GPIO,
NVKM_SUBDEV_I2C,
@@ -131,6 +132,7 @@
struct nvkm_secboot *secboot;
struct nvkm_therm *therm;
struct nvkm_timer *timer;
+ struct nvkm_top *top;
struct nvkm_volt *volt;
struct nvkm_engine *bsp;
@@ -200,6 +202,7 @@
int (*secboot )(struct nvkm_device *, int idx, struct nvkm_secboot **);
int (*therm )(struct nvkm_device *, int idx, struct nvkm_therm **);
int (*timer )(struct nvkm_device *, int idx, struct nvkm_timer **);
+ int (*top )(struct nvkm_device *, int idx, struct nvkm_top **);
int (*volt )(struct nvkm_device *, int idx, struct nvkm_volt **);
int (*bsp )(struct nvkm_device *, int idx, struct nvkm_engine **);
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/core/engine.h b/drivers/gpu/drm/nouveau/include/nvkm/core/engine.h
index 48bf128..9ebfd87 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/core/engine.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/core/engine.h
@@ -38,11 +38,9 @@
};
int nvkm_engine_ctor(const struct nvkm_engine_func *, struct nvkm_device *,
- int index, u32 pmc_enable, bool enable,
- struct nvkm_engine *);
+ int index, bool enable, struct nvkm_engine *);
int nvkm_engine_new_(const struct nvkm_engine_func *, struct nvkm_device *,
- int index, u32 pmc_enable, bool enable,
- struct nvkm_engine **);
+ int index, bool enable, struct nvkm_engine **);
struct nvkm_engine *nvkm_engine_ref(struct nvkm_engine *);
void nvkm_engine_unref(struct nvkm_engine **);
void nvkm_engine_tile(struct nvkm_engine *, int region);
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/core/subdev.h b/drivers/gpu/drm/nouveau/include/nvkm/core/subdev.h
index 3b5dc9c6..57adefa 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/core/subdev.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/core/subdev.h
@@ -6,7 +6,6 @@
const struct nvkm_subdev_func *func;
struct nvkm_device *device;
enum nvkm_devidx index;
- u32 pmc_enable;
struct mutex mutex;
u32 debug;
@@ -24,7 +23,7 @@
extern const char *nvkm_subdev_name[NVKM_SUBDEV_NR];
void nvkm_subdev_ctor(const struct nvkm_subdev_func *, struct nvkm_device *,
- int index, u32 pmc_enable, struct nvkm_subdev *);
+ int index, struct nvkm_subdev *);
void nvkm_subdev_del(struct nvkm_subdev **);
int nvkm_subdev_preinit(struct nvkm_subdev *);
int nvkm_subdev_init(struct nvkm_subdev *);
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/engine/falcon.h b/drivers/gpu/drm/nouveau/include/nvkm/engine/falcon.h
index 81c0bc6..e6baf03 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/engine/falcon.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/engine/falcon.h
@@ -40,7 +40,6 @@
u32 *data;
u32 size;
} data;
- u32 pmc_enable;
void (*init)(struct nvkm_falcon *);
void (*intr)(struct nvkm_falcon *, struct nvkm_fifo_chan *);
struct nvkm_sclass sclass[];
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/engine/xtensa.h b/drivers/gpu/drm/nouveau/include/nvkm/engine/xtensa.h
index 3128d21..b1fcc41 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/engine/xtensa.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/engine/xtensa.h
@@ -15,7 +15,6 @@
int index, bool enable, u32 addr, struct nvkm_engine **);
struct nvkm_xtensa_func {
- u32 pmc_enable;
u32 fifo_val;
u32 unkd28;
struct nvkm_sclass sclass[];
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/subdev/devinit.h b/drivers/gpu/drm/nouveau/include/nvkm/subdev/devinit.h
index 193626c..709d786 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/subdev/devinit.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/subdev/devinit.h
@@ -7,6 +7,7 @@
const struct nvkm_devinit_func *func;
struct nvkm_subdev subdev;
bool post;
+ bool force_post;
};
u32 nvkm_devinit_mmio(struct nvkm_devinit *, u32 addr);
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/subdev/fb.h b/drivers/gpu/drm/nouveau/include/nvkm/subdev/fb.h
index 85ab72c..0a734fd 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/subdev/fb.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/subdev/fb.h
@@ -55,6 +55,9 @@
struct nvkm_fb_tile region[16];
int regions;
} tile;
+
+ struct nvkm_memory *mmu_rd;
+ struct nvkm_memory *mmu_wr;
};
bool nvkm_fb_memtype_valid(struct nvkm_fb *, u32 memtype);
@@ -87,6 +90,7 @@
int gk104_fb_new(struct nvkm_device *, int, struct nvkm_fb **);
int gk20a_fb_new(struct nvkm_device *, int, struct nvkm_fb **);
int gm107_fb_new(struct nvkm_device *, int, struct nvkm_fb **);
+int gm200_fb_new(struct nvkm_device *, int, struct nvkm_fb **);
#include <subdev/bios.h>
#include <subdev/bios/ramcfg.h>
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/subdev/iccsense.h b/drivers/gpu/drm/nouveau/include/nvkm/subdev/iccsense.h
index 530c621..3c2ddd9 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/subdev/iccsense.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/subdev/iccsense.h
@@ -3,15 +3,13 @@
#include <core/subdev.h>
-struct nkvm_iccsense_rail;
struct nvkm_iccsense {
struct nvkm_subdev subdev;
- u8 rail_count;
bool data_valid;
- struct nvkm_iccsense_rail *rails;
+ struct list_head sensors;
+ struct list_head rails;
};
int gf100_iccsense_new(struct nvkm_device *, int index, struct nvkm_iccsense **);
-int nvkm_iccsense_read(struct nvkm_iccsense *iccsense, u8 idx);
int nvkm_iccsense_read_all(struct nvkm_iccsense *iccsense);
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/subdev/mc.h b/drivers/gpu/drm/nouveau/include/nvkm/subdev/mc.h
index 4de05e7..2e80682 100644
--- a/drivers/gpu/drm/nouveau/include/nvkm/subdev/mc.h
+++ b/drivers/gpu/drm/nouveau/include/nvkm/subdev/mc.h
@@ -10,12 +10,18 @@
void nvkm_mc_intr(struct nvkm_mc *, bool *handled);
void nvkm_mc_intr_unarm(struct nvkm_mc *);
void nvkm_mc_intr_rearm(struct nvkm_mc *);
+void nvkm_mc_reset(struct nvkm_mc *, enum nvkm_devidx);
void nvkm_mc_unk260(struct nvkm_mc *, u32 data);
int nv04_mc_new(struct nvkm_device *, int, struct nvkm_mc **);
+int nv11_mc_new(struct nvkm_device *, int, struct nvkm_mc **);
+int nv17_mc_new(struct nvkm_device *, int, struct nvkm_mc **);
int nv44_mc_new(struct nvkm_device *, int, struct nvkm_mc **);
int nv50_mc_new(struct nvkm_device *, int, struct nvkm_mc **);
+int g84_mc_new(struct nvkm_device *, int, struct nvkm_mc **);
int g98_mc_new(struct nvkm_device *, int, struct nvkm_mc **);
+int gt215_mc_new(struct nvkm_device *, int, struct nvkm_mc **);
int gf100_mc_new(struct nvkm_device *, int, struct nvkm_mc **);
+int gk104_mc_new(struct nvkm_device *, int, struct nvkm_mc **);
int gk20a_mc_new(struct nvkm_device *, int, struct nvkm_mc **);
#endif
diff --git a/drivers/gpu/drm/nouveau/include/nvkm/subdev/top.h b/drivers/gpu/drm/nouveau/include/nvkm/subdev/top.h
new file mode 100644
index 0000000..8fb575a
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/include/nvkm/subdev/top.h
@@ -0,0 +1,17 @@
+#ifndef __NVKM_TOP_H__
+#define __NVKM_TOP_H__
+#include <core/subdev.h>
+
+struct nvkm_top {
+ const struct nvkm_top_func *func;
+ struct nvkm_subdev subdev;
+ struct list_head device;
+};
+
+u32 nvkm_top_reset(struct nvkm_top *, enum nvkm_devidx);
+u32 nvkm_top_intr(struct nvkm_top *, u32 intr, u64 *subdevs);
+enum nvkm_devidx nvkm_top_fault(struct nvkm_top *, int fault);
+enum nvkm_devidx nvkm_top_engine(struct nvkm_top *, int, int *runl, int *engn);
+
+int gk104_top_new(struct nvkm_device *, int, struct nvkm_top **);
+#endif
diff --git a/drivers/gpu/drm/nouveau/nouveau_abi16.c b/drivers/gpu/drm/nouveau/nouveau_abi16.c
index a59e524..eb7de48 100644
--- a/drivers/gpu/drm/nouveau/nouveau_abi16.c
+++ b/drivers/gpu/drm/nouveau/nouveau_abi16.c
@@ -29,7 +29,7 @@
#include <nvif/cla06f.h>
#include <nvif/unpack.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_dma.h"
#include "nouveau_gem.h"
#include "nouveau_chan.h"
diff --git a/drivers/gpu/drm/nouveau/nouveau_acpi.c b/drivers/gpu/drm/nouveau/nouveau_acpi.c
index cdf5227..db76b94 100644
--- a/drivers/gpu/drm/nouveau/nouveau_acpi.c
+++ b/drivers/gpu/drm/nouveau/nouveau_acpi.c
@@ -6,7 +6,7 @@
#include <drm/drm_edid.h>
#include <acpi/video.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_acpi.h"
#define NOUVEAU_DSM_LED 0x02
diff --git a/drivers/gpu/drm/nouveau/nouveau_backlight.c b/drivers/gpu/drm/nouveau/nouveau_backlight.c
index 89eb460..f5101be 100644
--- a/drivers/gpu/drm/nouveau/nouveau_backlight.c
+++ b/drivers/gpu/drm/nouveau/nouveau_backlight.c
@@ -32,7 +32,7 @@
#include <linux/backlight.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_reg.h"
#include "nouveau_encoder.h"
diff --git a/drivers/gpu/drm/nouveau/nouveau_bios.c b/drivers/gpu/drm/nouveau/nouveau_bios.c
index 4dca65a..a1570b1 100644
--- a/drivers/gpu/drm/nouveau/nouveau_bios.c
+++ b/drivers/gpu/drm/nouveau/nouveau_bios.c
@@ -24,7 +24,7 @@
#include <drm/drmP.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_reg.h"
#include "dispnv04/hw.h"
#include "nouveau_encoder.h"
diff --git a/drivers/gpu/drm/nouveau/nouveau_bo.c b/drivers/gpu/drm/nouveau/nouveau_bo.c
index 2cdaea5..5e3f3e8 100644
--- a/drivers/gpu/drm/nouveau/nouveau_bo.c
+++ b/drivers/gpu/drm/nouveau/nouveau_bo.c
@@ -30,7 +30,7 @@
#include <linux/dma-mapping.h>
#include <linux/swiotlb.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_dma.h"
#include "nouveau_fence.h"
@@ -312,7 +312,7 @@
bool force = false, evict = false;
int ret;
- ret = ttm_bo_reserve(bo, false, false, false, NULL);
+ ret = ttm_bo_reserve(bo, false, false, NULL);
if (ret)
return ret;
@@ -385,7 +385,7 @@
struct ttm_buffer_object *bo = &nvbo->bo;
int ret, ref;
- ret = ttm_bo_reserve(bo, false, false, false, NULL);
+ ret = ttm_bo_reserve(bo, false, false, NULL);
if (ret)
return ret;
@@ -420,7 +420,7 @@
{
int ret;
- ret = ttm_bo_reserve(&nvbo->bo, false, false, false, NULL);
+ ret = ttm_bo_reserve(&nvbo->bo, false, false, NULL);
if (ret)
return ret;
@@ -1322,7 +1322,7 @@
}
/* Fallback to software copy. */
- ret = ttm_bo_wait(bo, true, intr, no_wait_gpu);
+ ret = ttm_bo_wait(bo, intr, no_wait_gpu);
if (ret == 0)
ret = ttm_bo_move_memcpy(bo, evict, no_wait_gpu, new_mem);
@@ -1611,6 +1611,8 @@
.fault_reserve_notify = &nouveau_ttm_fault_reserve_notify,
.io_mem_reserve = &nouveau_ttm_io_mem_reserve,
.io_mem_free = &nouveau_ttm_io_mem_free,
+ .lru_tail = &ttm_bo_default_lru_tail,
+ .swap_lru_tail = &ttm_bo_default_swap_lru_tail,
};
struct nvkm_vma *
diff --git a/drivers/gpu/drm/nouveau/nouveau_chan.c b/drivers/gpu/drm/nouveau/nouveau_chan.c
index 879655c..b1d2527 100644
--- a/drivers/gpu/drm/nouveau/nouveau_chan.c
+++ b/drivers/gpu/drm/nouveau/nouveau_chan.c
@@ -34,7 +34,7 @@
/*XXX*/
#include <core/client.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_dma.h"
#include "nouveau_bo.h"
#include "nouveau_chan.h"
diff --git a/drivers/gpu/drm/nouveau/nouveau_connector.c b/drivers/gpu/drm/nouveau/nouveau_connector.c
index e81aefe..c108408 100644
--- a/drivers/gpu/drm/nouveau/nouveau_connector.c
+++ b/drivers/gpu/drm/nouveau/nouveau_connector.c
@@ -34,7 +34,7 @@
#include <drm/drm_crtc_helper.h>
#include "nouveau_reg.h"
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "dispnv04/hw.h"
#include "nouveau_acpi.h"
diff --git a/drivers/gpu/drm/nouveau/nouveau_debugfs.c b/drivers/gpu/drm/nouveau/nouveau_debugfs.c
index 3d0dc19..411c12c 100644
--- a/drivers/gpu/drm/nouveau/nouveau_debugfs.c
+++ b/drivers/gpu/drm/nouveau/nouveau_debugfs.c
@@ -32,7 +32,7 @@
#include <nvif/class.h>
#include <nvif/if0001.h>
#include "nouveau_debugfs.h"
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
static int
nouveau_debugfs_vbios_image(struct seq_file *m, void *data)
diff --git a/drivers/gpu/drm/nouveau/nouveau_debugfs.h b/drivers/gpu/drm/nouveau/nouveau_debugfs.h
index b8c03ff..eab5881 100644
--- a/drivers/gpu/drm/nouveau/nouveau_debugfs.h
+++ b/drivers/gpu/drm/nouveau/nouveau_debugfs.h
@@ -5,7 +5,7 @@
#if defined(CONFIG_DEBUG_FS)
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
struct nouveau_debugfs {
struct nvif_object ctrl;
diff --git a/drivers/gpu/drm/nouveau/nouveau_display.c b/drivers/gpu/drm/nouveau/nouveau_display.c
index 7ce7fa5..7c77f96 100644
--- a/drivers/gpu/drm/nouveau/nouveau_display.c
+++ b/drivers/gpu/drm/nouveau/nouveau_display.c
@@ -279,7 +279,7 @@
struct drm_gem_object *gem;
int ret = -ENOMEM;
- gem = drm_gem_object_lookup(dev, file_priv, mode_cmd->handles[0]);
+ gem = drm_gem_object_lookup(file_priv, mode_cmd->handles[0]);
if (!gem)
return ERR_PTR(-ENOENT);
@@ -296,7 +296,7 @@
err:
kfree(nouveau_fb);
err_unref:
- drm_gem_object_unreference(gem);
+ drm_gem_object_unreference_unlocked(gem);
return ERR_PTR(ret);
}
@@ -739,7 +739,7 @@
}
mutex_lock(&cli->mutex);
- ret = ttm_bo_reserve(&new_bo->bo, true, false, false, NULL);
+ ret = ttm_bo_reserve(&new_bo->bo, true, false, NULL);
if (ret)
goto fail_unpin;
@@ -753,7 +753,7 @@
if (new_bo != old_bo) {
ttm_bo_unreserve(&new_bo->bo);
- ret = ttm_bo_reserve(&old_bo->bo, true, false, false, NULL);
+ ret = ttm_bo_reserve(&old_bo->bo, true, false, NULL);
if (ret)
goto fail_unpin;
}
@@ -916,7 +916,7 @@
{
struct drm_gem_object *gem;
- gem = drm_gem_object_lookup(dev, file_priv, handle);
+ gem = drm_gem_object_lookup(file_priv, handle);
if (gem) {
struct nouveau_bo *bo = nouveau_gem_object(gem);
*poffset = drm_vma_node_offset_addr(&bo->bo.vma_node);
diff --git a/drivers/gpu/drm/nouveau/nouveau_display.h b/drivers/gpu/drm/nouveau/nouveau_display.h
index 5a57d8b..24273ba 100644
--- a/drivers/gpu/drm/nouveau/nouveau_display.h
+++ b/drivers/gpu/drm/nouveau/nouveau_display.h
@@ -3,7 +3,7 @@
#include <subdev/mmu.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
struct nouveau_framebuffer {
struct drm_framebuffer base;
diff --git a/drivers/gpu/drm/nouveau/nouveau_dma.c b/drivers/gpu/drm/nouveau/nouveau_dma.c
index d168c63..2634a1a 100644
--- a/drivers/gpu/drm/nouveau/nouveau_dma.c
+++ b/drivers/gpu/drm/nouveau/nouveau_dma.c
@@ -24,7 +24,7 @@
*
*/
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_dma.h"
void
diff --git a/drivers/gpu/drm/nouveau/nouveau_dp.c b/drivers/gpu/drm/nouveau/nouveau_dp.c
index e17e15e..87d52d3 100644
--- a/drivers/gpu/drm/nouveau/nouveau_dp.c
+++ b/drivers/gpu/drm/nouveau/nouveau_dp.c
@@ -25,7 +25,7 @@
#include <drm/drmP.h>
#include <drm/drm_dp_helper.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_connector.h"
#include "nouveau_encoder.h"
#include "nouveau_crtc.h"
diff --git a/drivers/gpu/drm/nouveau/nouveau_drm.c b/drivers/gpu/drm/nouveau/nouveau_drm.c
index d06877d..11f8dd9 100644
--- a/drivers/gpu/drm/nouveau/nouveau_drm.c
+++ b/drivers/gpu/drm/nouveau/nouveau_drm.c
@@ -44,7 +44,7 @@
#include <nvif/cla06f.h>
#include <nvif/if0004.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_dma.h"
#include "nouveau_ttm.h"
#include "nouveau_gem.h"
@@ -1083,10 +1083,8 @@
nouveau_display_options();
if (nouveau_modeset == -1) {
-#ifdef CONFIG_VGA_CONSOLE
if (vgacon_text_force())
nouveau_modeset = 0;
-#endif
}
if (!nouveau_modeset)
diff --git a/drivers/gpu/drm/nouveau/nouveau_drm.h b/drivers/gpu/drm/nouveau/nouveau_drm.h
deleted file mode 100644
index 5c363ed..0000000
--- a/drivers/gpu/drm/nouveau/nouveau_drm.h
+++ /dev/null
@@ -1,207 +0,0 @@
-#ifndef __NOUVEAU_DRMCLI_H__
-#define __NOUVEAU_DRMCLI_H__
-
-#define DRIVER_AUTHOR "Nouveau Project"
-#define DRIVER_EMAIL "nouveau@lists.freedesktop.org"
-
-#define DRIVER_NAME "nouveau"
-#define DRIVER_DESC "nVidia Riva/TNT/GeForce/Quadro/Tesla"
-#define DRIVER_DATE "20120801"
-
-#define DRIVER_MAJOR 1
-#define DRIVER_MINOR 3
-#define DRIVER_PATCHLEVEL 1
-
-/*
- * 1.1.1:
- * - added support for tiled system memory buffer objects
- * - added support for NOUVEAU_GETPARAM_GRAPH_UNITS on [nvc0,nve0].
- * - added support for compressed memory storage types on [nvc0,nve0].
- * - added support for software methods 0x600,0x644,0x6ac on nvc0
- * to control registers on the MPs to enable performance counters,
- * and to control the warp error enable mask (OpenGL requires out of
- * bounds access to local memory to be silently ignored / return 0).
- * 1.1.2:
- * - fixes multiple bugs in flip completion events and timestamping
- * 1.2.0:
- * - object api exposed to userspace
- * - fermi,kepler,maxwell zbc
- * 1.2.1:
- * - allow concurrent access to bo's mapped read/write.
- * 1.2.2:
- * - add NOUVEAU_GEM_DOMAIN_COHERENT flag
- * 1.3.0:
- * - NVIF ABI modified, safe because only (current) users are test
- * programs that get directly linked with NVKM.
- * 1.3.1:
- * - implemented limited ABI16/NVIF interop
- */
-
-#include <nvif/client.h>
-#include <nvif/device.h>
-#include <nvif/ioctl.h>
-
-#include <drmP.h>
-
-#include <drm/ttm/ttm_bo_api.h>
-#include <drm/ttm/ttm_bo_driver.h>
-#include <drm/ttm/ttm_placement.h>
-#include <drm/ttm/ttm_memory.h>
-#include <drm/ttm/ttm_module.h>
-#include <drm/ttm/ttm_page_alloc.h>
-
-#include "uapi/drm/nouveau_drm.h"
-
-struct nouveau_channel;
-struct platform_device;
-
-#define DRM_FILE_PAGE_OFFSET (0x100000000ULL >> PAGE_SHIFT)
-
-#include "nouveau_fence.h"
-#include "nouveau_bios.h"
-
-struct nouveau_drm_tile {
- struct nouveau_fence *fence;
- bool used;
-};
-
-enum nouveau_drm_object_route {
- NVDRM_OBJECT_NVIF = NVIF_IOCTL_V0_OWNER_NVIF,
- NVDRM_OBJECT_USIF,
- NVDRM_OBJECT_ABI16,
- NVDRM_OBJECT_ANY = NVIF_IOCTL_V0_OWNER_ANY,
-};
-
-enum nouveau_drm_notify_route {
- NVDRM_NOTIFY_NVIF = 0,
- NVDRM_NOTIFY_USIF
-};
-
-enum nouveau_drm_handle {
- NVDRM_CHAN = 0xcccc0000, /* |= client chid */
- NVDRM_NVSW = 0x55550000,
-};
-
-struct nouveau_cli {
- struct nvif_client base;
- struct nvkm_vm *vm; /*XXX*/
- struct list_head head;
- struct mutex mutex;
- void *abi16;
- struct list_head objects;
- struct list_head notifys;
- char name[32];
- struct drm_device *dev;
-};
-
-static inline struct nouveau_cli *
-nouveau_cli(struct drm_file *fpriv)
-{
- return fpriv ? fpriv->driver_priv : NULL;
-}
-
-#include <nvif/object.h>
-#include <nvif/device.h>
-
-extern int nouveau_runtime_pm;
-
-struct nouveau_drm {
- struct nouveau_cli client;
- struct drm_device *dev;
-
- struct nvif_device device;
- struct list_head clients;
-
- struct {
- struct agp_bridge_data *bridge;
- u32 base;
- u32 size;
- bool cma;
- } agp;
-
- /* TTM interface support */
- struct {
- struct drm_global_reference mem_global_ref;
- struct ttm_bo_global_ref bo_global_ref;
- struct ttm_bo_device bdev;
- atomic_t validate_sequence;
- int (*move)(struct nouveau_channel *,
- struct ttm_buffer_object *,
- struct ttm_mem_reg *, struct ttm_mem_reg *);
- struct nouveau_channel *chan;
- struct nvif_object copy;
- int mtrr;
- } ttm;
-
- /* GEM interface support */
- struct {
- u64 vram_available;
- u64 gart_available;
- } gem;
-
- /* synchronisation */
- void *fence;
-
- /* context for accelerated drm-internal operations */
- struct nouveau_channel *cechan;
- struct nouveau_channel *channel;
- struct nvkm_gpuobj *notify;
- struct nouveau_fbdev *fbcon;
- struct nvif_object nvsw;
- struct nvif_object ntfy;
- struct nvif_notify flip;
-
- /* nv10-nv40 tiling regions */
- struct {
- struct nouveau_drm_tile reg[15];
- spinlock_t lock;
- } tile;
-
- /* modesetting */
- struct nvbios vbios;
- struct nouveau_display *display;
- struct backlight_device *backlight;
-
- /* power management */
- struct nouveau_hwmon *hwmon;
- struct nouveau_debugfs *debugfs;
-
- /* display power reference */
- bool have_disp_power_ref;
-
- struct dev_pm_domain vga_pm_domain;
- struct pci_dev *hdmi_device;
-};
-
-static inline struct nouveau_drm *
-nouveau_drm(struct drm_device *dev)
-{
- return dev->dev_private;
-}
-
-int nouveau_pmops_suspend(struct device *);
-int nouveau_pmops_resume(struct device *);
-
-#include <nvkm/core/tegra.h>
-
-struct drm_device *
-nouveau_platform_device_create(const struct nvkm_device_tegra_func *,
- struct platform_device *, struct nvkm_device **);
-void nouveau_drm_device_remove(struct drm_device *dev);
-
-#define NV_PRINTK(l,c,f,a...) do { \
- struct nouveau_cli *_cli = (c); \
- dev_##l(_cli->dev->dev, "%s: "f, _cli->name, ##a); \
-} while(0)
-#define NV_FATAL(drm,f,a...) NV_PRINTK(crit, &(drm)->client, f, ##a)
-#define NV_ERROR(drm,f,a...) NV_PRINTK(err, &(drm)->client, f, ##a)
-#define NV_WARN(drm,f,a...) NV_PRINTK(warn, &(drm)->client, f, ##a)
-#define NV_INFO(drm,f,a...) NV_PRINTK(info, &(drm)->client, f, ##a)
-#define NV_DEBUG(drm,f,a...) do { \
- if (unlikely(drm_debug & DRM_UT_DRIVER)) \
- NV_PRINTK(info, &(drm)->client, f, ##a); \
-} while(0)
-
-extern int nouveau_modeset;
-
-#endif
diff --git a/drivers/gpu/drm/nouveau/nouveau_drv.h b/drivers/gpu/drm/nouveau/nouveau_drv.h
new file mode 100644
index 0000000..822a021
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nouveau_drv.h
@@ -0,0 +1,207 @@
+#ifndef __NOUVEAU_DRV_H__
+#define __NOUVEAU_DRV_H__
+
+#define DRIVER_AUTHOR "Nouveau Project"
+#define DRIVER_EMAIL "nouveau@lists.freedesktop.org"
+
+#define DRIVER_NAME "nouveau"
+#define DRIVER_DESC "nVidia Riva/TNT/GeForce/Quadro/Tesla"
+#define DRIVER_DATE "20120801"
+
+#define DRIVER_MAJOR 1
+#define DRIVER_MINOR 3
+#define DRIVER_PATCHLEVEL 1
+
+/*
+ * 1.1.1:
+ * - added support for tiled system memory buffer objects
+ * - added support for NOUVEAU_GETPARAM_GRAPH_UNITS on [nvc0,nve0].
+ * - added support for compressed memory storage types on [nvc0,nve0].
+ * - added support for software methods 0x600,0x644,0x6ac on nvc0
+ * to control registers on the MPs to enable performance counters,
+ * and to control the warp error enable mask (OpenGL requires out of
+ * bounds access to local memory to be silently ignored / return 0).
+ * 1.1.2:
+ * - fixes multiple bugs in flip completion events and timestamping
+ * 1.2.0:
+ * - object api exposed to userspace
+ * - fermi,kepler,maxwell zbc
+ * 1.2.1:
+ * - allow concurrent access to bo's mapped read/write.
+ * 1.2.2:
+ * - add NOUVEAU_GEM_DOMAIN_COHERENT flag
+ * 1.3.0:
+ * - NVIF ABI modified, safe because only (current) users are test
+ * programs that get directly linked with NVKM.
+ * 1.3.1:
+ * - implemented limited ABI16/NVIF interop
+ */
+
+#include <nvif/client.h>
+#include <nvif/device.h>
+#include <nvif/ioctl.h>
+
+#include <drmP.h>
+
+#include <drm/ttm/ttm_bo_api.h>
+#include <drm/ttm/ttm_bo_driver.h>
+#include <drm/ttm/ttm_placement.h>
+#include <drm/ttm/ttm_memory.h>
+#include <drm/ttm/ttm_module.h>
+#include <drm/ttm/ttm_page_alloc.h>
+
+#include "uapi/drm/nouveau_drm.h"
+
+struct nouveau_channel;
+struct platform_device;
+
+#define DRM_FILE_PAGE_OFFSET (0x100000000ULL >> PAGE_SHIFT)
+
+#include "nouveau_fence.h"
+#include "nouveau_bios.h"
+
+struct nouveau_drm_tile {
+ struct nouveau_fence *fence;
+ bool used;
+};
+
+enum nouveau_drm_object_route {
+ NVDRM_OBJECT_NVIF = NVIF_IOCTL_V0_OWNER_NVIF,
+ NVDRM_OBJECT_USIF,
+ NVDRM_OBJECT_ABI16,
+ NVDRM_OBJECT_ANY = NVIF_IOCTL_V0_OWNER_ANY,
+};
+
+enum nouveau_drm_notify_route {
+ NVDRM_NOTIFY_NVIF = 0,
+ NVDRM_NOTIFY_USIF
+};
+
+enum nouveau_drm_handle {
+ NVDRM_CHAN = 0xcccc0000, /* |= client chid */
+ NVDRM_NVSW = 0x55550000,
+};
+
+struct nouveau_cli {
+ struct nvif_client base;
+ struct nvkm_vm *vm; /*XXX*/
+ struct list_head head;
+ struct mutex mutex;
+ void *abi16;
+ struct list_head objects;
+ struct list_head notifys;
+ char name[32];
+ struct drm_device *dev;
+};
+
+static inline struct nouveau_cli *
+nouveau_cli(struct drm_file *fpriv)
+{
+ return fpriv ? fpriv->driver_priv : NULL;
+}
+
+#include <nvif/object.h>
+#include <nvif/device.h>
+
+extern int nouveau_runtime_pm;
+
+struct nouveau_drm {
+ struct nouveau_cli client;
+ struct drm_device *dev;
+
+ struct nvif_device device;
+ struct list_head clients;
+
+ struct {
+ struct agp_bridge_data *bridge;
+ u32 base;
+ u32 size;
+ bool cma;
+ } agp;
+
+ /* TTM interface support */
+ struct {
+ struct drm_global_reference mem_global_ref;
+ struct ttm_bo_global_ref bo_global_ref;
+ struct ttm_bo_device bdev;
+ atomic_t validate_sequence;
+ int (*move)(struct nouveau_channel *,
+ struct ttm_buffer_object *,
+ struct ttm_mem_reg *, struct ttm_mem_reg *);
+ struct nouveau_channel *chan;
+ struct nvif_object copy;
+ int mtrr;
+ } ttm;
+
+ /* GEM interface support */
+ struct {
+ u64 vram_available;
+ u64 gart_available;
+ } gem;
+
+ /* synchronisation */
+ void *fence;
+
+ /* context for accelerated drm-internal operations */
+ struct nouveau_channel *cechan;
+ struct nouveau_channel *channel;
+ struct nvkm_gpuobj *notify;
+ struct nouveau_fbdev *fbcon;
+ struct nvif_object nvsw;
+ struct nvif_object ntfy;
+ struct nvif_notify flip;
+
+ /* nv10-nv40 tiling regions */
+ struct {
+ struct nouveau_drm_tile reg[15];
+ spinlock_t lock;
+ } tile;
+
+ /* modesetting */
+ struct nvbios vbios;
+ struct nouveau_display *display;
+ struct backlight_device *backlight;
+
+ /* power management */
+ struct nouveau_hwmon *hwmon;
+ struct nouveau_debugfs *debugfs;
+
+ /* display power reference */
+ bool have_disp_power_ref;
+
+ struct dev_pm_domain vga_pm_domain;
+ struct pci_dev *hdmi_device;
+};
+
+static inline struct nouveau_drm *
+nouveau_drm(struct drm_device *dev)
+{
+ return dev->dev_private;
+}
+
+int nouveau_pmops_suspend(struct device *);
+int nouveau_pmops_resume(struct device *);
+
+#include <nvkm/core/tegra.h>
+
+struct drm_device *
+nouveau_platform_device_create(const struct nvkm_device_tegra_func *,
+ struct platform_device *, struct nvkm_device **);
+void nouveau_drm_device_remove(struct drm_device *dev);
+
+#define NV_PRINTK(l,c,f,a...) do { \
+ struct nouveau_cli *_cli = (c); \
+ dev_##l(_cli->dev->dev, "%s: "f, _cli->name, ##a); \
+} while(0)
+#define NV_FATAL(drm,f,a...) NV_PRINTK(crit, &(drm)->client, f, ##a)
+#define NV_ERROR(drm,f,a...) NV_PRINTK(err, &(drm)->client, f, ##a)
+#define NV_WARN(drm,f,a...) NV_PRINTK(warn, &(drm)->client, f, ##a)
+#define NV_INFO(drm,f,a...) NV_PRINTK(info, &(drm)->client, f, ##a)
+#define NV_DEBUG(drm,f,a...) do { \
+ if (unlikely(drm_debug & DRM_UT_DRIVER)) \
+ NV_PRINTK(info, &(drm)->client, f, ##a); \
+} while(0)
+
+extern int nouveau_modeset;
+
+#endif
diff --git a/drivers/gpu/drm/nouveau/nouveau_fbcon.c b/drivers/gpu/drm/nouveau/nouveau_fbcon.c
index 59f27e7..57aaf98 100644
--- a/drivers/gpu/drm/nouveau/nouveau_fbcon.c
+++ b/drivers/gpu/drm/nouveau/nouveau_fbcon.c
@@ -43,7 +43,7 @@
#include <drm/drm_crtc_helper.h>
#include <drm/drm_fb_helper.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_gem.h"
#include "nouveau_bo.h"
#include "nouveau_fbcon.h"
@@ -386,8 +386,6 @@
}
}
- mutex_lock(&dev->struct_mutex);
-
info = drm_fb_helper_alloc_fbi(helper);
if (IS_ERR(info)) {
ret = PTR_ERR(info);
@@ -426,8 +424,6 @@
/* Use default scratch pixmap (info->pixmap.flags = FB_PIXMAP_SYSTEM) */
- mutex_unlock(&dev->struct_mutex);
-
if (chan)
nouveau_fbcon_accel_init(dev);
nouveau_fbcon_zfill(dev, fbcon);
@@ -441,7 +437,6 @@
return 0;
out_unlock:
- mutex_unlock(&dev->struct_mutex);
if (chan)
nouveau_bo_vma_del(nvbo, &fbcon->nouveau_fb.vma);
nouveau_bo_unmap(nvbo);
diff --git a/drivers/gpu/drm/nouveau/nouveau_fence.c b/drivers/gpu/drm/nouveau/nouveau_fence.c
index 9a8c5b7..4bb9ab8 100644
--- a/drivers/gpu/drm/nouveau/nouveau_fence.c
+++ b/drivers/gpu/drm/nouveau/nouveau_fence.c
@@ -34,7 +34,7 @@
#include <nvif/notify.h>
#include <nvif/event.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_dma.h"
#include "nouveau_fence.h"
diff --git a/drivers/gpu/drm/nouveau/nouveau_gem.c b/drivers/gpu/drm/nouveau/nouveau_gem.c
index a0865c4..72e2399 100644
--- a/drivers/gpu/drm/nouveau/nouveau_gem.c
+++ b/drivers/gpu/drm/nouveau/nouveau_gem.c
@@ -24,7 +24,7 @@
*
*/
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_dma.h"
#include "nouveau_fence.h"
#include "nouveau_abi16.h"
@@ -71,7 +71,7 @@
if (!cli->vm)
return 0;
- ret = ttm_bo_reserve(&nvbo->bo, false, false, false, NULL);
+ ret = ttm_bo_reserve(&nvbo->bo, false, false, NULL);
if (ret)
return ret;
@@ -126,7 +126,7 @@
list_del(&vma->head);
if (fobj && fobj->shared_count > 1)
- ttm_bo_wait(&nvbo->bo, true, false, false);
+ ttm_bo_wait(&nvbo->bo, false, false);
else if (fobj && fobj->shared_count == 1)
fence = rcu_dereference_protected(fobj->shared[0],
reservation_object_held(resv));
@@ -156,7 +156,7 @@
if (!cli->vm)
return;
- ret = ttm_bo_reserve(&nvbo->bo, false, false, false, NULL);
+ ret = ttm_bo_reserve(&nvbo->bo, false, false, NULL);
if (ret)
return;
@@ -368,7 +368,6 @@
int nr_buffers, struct validate_op *op)
{
struct nouveau_cli *cli = nouveau_cli(file_priv);
- struct drm_device *dev = chan->drm->dev;
int trycnt = 0;
int ret, i;
struct nouveau_bo *res_bo = NULL;
@@ -388,7 +387,7 @@
struct drm_gem_object *gem;
struct nouveau_bo *nvbo;
- gem = drm_gem_object_lookup(dev, file_priv, b->handle);
+ gem = drm_gem_object_lookup(file_priv, b->handle);
if (!gem) {
NV_PRINTK(err, cli, "Unknown handle 0x%08x\n", b->handle);
ret = -ENOENT;
@@ -409,7 +408,7 @@
break;
}
- ret = ttm_bo_reserve(&nvbo->bo, true, false, true, &op->ticket);
+ ret = ttm_bo_reserve(&nvbo->bo, true, false, &op->ticket);
if (ret) {
list_splice_tail_init(&vram_list, &op->list);
list_splice_tail_init(&gart_list, &op->list);
@@ -651,7 +650,7 @@
data |= r->vor;
}
- ret = ttm_bo_wait(&nvbo->bo, true, false, false);
+ ret = ttm_bo_wait(&nvbo->bo, false, false);
if (ret) {
NV_PRINTK(err, cli, "reloc wait_idle failed: %d\n", ret);
break;
@@ -864,7 +863,7 @@
bool write = !!(req->flags & NOUVEAU_GEM_CPU_PREP_WRITE);
int ret;
- gem = drm_gem_object_lookup(dev, file_priv, req->handle);
+ gem = drm_gem_object_lookup(file_priv, req->handle);
if (!gem)
return -ENOENT;
nvbo = nouveau_gem_object(gem);
@@ -896,7 +895,7 @@
struct drm_gem_object *gem;
struct nouveau_bo *nvbo;
- gem = drm_gem_object_lookup(dev, file_priv, req->handle);
+ gem = drm_gem_object_lookup(file_priv, req->handle);
if (!gem)
return -ENOENT;
nvbo = nouveau_gem_object(gem);
@@ -914,7 +913,7 @@
struct drm_gem_object *gem;
int ret;
- gem = drm_gem_object_lookup(dev, file_priv, req->handle);
+ gem = drm_gem_object_lookup(file_priv, req->handle);
if (!gem)
return -ENOENT;
diff --git a/drivers/gpu/drm/nouveau/nouveau_gem.h b/drivers/gpu/drm/nouveau/nouveau_gem.h
index e4049fa..7e32da2 100644
--- a/drivers/gpu/drm/nouveau/nouveau_gem.h
+++ b/drivers/gpu/drm/nouveau/nouveau_gem.h
@@ -3,7 +3,7 @@
#include <drm/drmP.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_bo.h"
#define nouveau_bo_tile_layout(nvbo) \
diff --git a/drivers/gpu/drm/nouveau/nouveau_hwmon.c b/drivers/gpu/drm/nouveau/nouveau_hwmon.c
index 67edd2f5..1ff4166 100644
--- a/drivers/gpu/drm/nouveau/nouveau_hwmon.c
+++ b/drivers/gpu/drm/nouveau/nouveau_hwmon.c
@@ -31,7 +31,7 @@
#include <drm/drmP.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_hwmon.h"
#include <nvkm/subdev/iccsense.h>
@@ -689,7 +689,7 @@
goto error;
}
- if (iccsense && iccsense->data_valid && iccsense->rail_count) {
+ if (iccsense && iccsense->data_valid && !list_empty(&iccsense->rails)) {
ret = sysfs_create_group(&hwmon_dev->kobj,
&hwmon_power_attrgroup);
if (ret)
diff --git a/drivers/gpu/drm/nouveau/nouveau_nvif.c b/drivers/gpu/drm/nouveau/nouveau_nvif.c
index 55eb9428..15f0925 100644
--- a/drivers/gpu/drm/nouveau/nouveau_nvif.c
+++ b/drivers/gpu/drm/nouveau/nouveau_nvif.c
@@ -36,7 +36,7 @@
#include <nvif/event.h>
#include <nvif/ioctl.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_usif.h"
static void
diff --git a/drivers/gpu/drm/nouveau/nouveau_platform.h b/drivers/gpu/drm/nouveau/nouveau_platform.h
index f41056d..a90d727 100644
--- a/drivers/gpu/drm/nouveau/nouveau_platform.h
+++ b/drivers/gpu/drm/nouveau/nouveau_platform.h
@@ -21,7 +21,7 @@
*/
#ifndef __NOUVEAU_PLATFORM_H__
#define __NOUVEAU_PLATFORM_H__
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
extern struct platform_driver nouveau_platform_driver;
#endif
diff --git a/drivers/gpu/drm/nouveau/nouveau_prime.c b/drivers/gpu/drm/nouveau/nouveau_prime.c
index dd32ad6..a0a9704 100644
--- a/drivers/gpu/drm/nouveau/nouveau_prime.c
+++ b/drivers/gpu/drm/nouveau/nouveau_prime.c
@@ -25,7 +25,7 @@
#include <drm/drmP.h>
#include <linux/dma-buf.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_gem.h"
struct sg_table *nouveau_gem_prime_get_sg_table(struct drm_gem_object *obj)
diff --git a/drivers/gpu/drm/nouveau/nouveau_sgdma.c b/drivers/gpu/drm/nouveau/nouveau_sgdma.c
index 8c3053a..db35ab5 100644
--- a/drivers/gpu/drm/nouveau/nouveau_sgdma.c
+++ b/drivers/gpu/drm/nouveau/nouveau_sgdma.c
@@ -1,7 +1,7 @@
#include <linux/pagemap.h>
#include <linux/slab.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_ttm.h"
struct nouveau_sgdma_be {
diff --git a/drivers/gpu/drm/nouveau/nouveau_ttm.c b/drivers/gpu/drm/nouveau/nouveau_ttm.c
index d2e7d20..bcee914 100644
--- a/drivers/gpu/drm/nouveau/nouveau_ttm.c
+++ b/drivers/gpu/drm/nouveau/nouveau_ttm.c
@@ -24,7 +24,7 @@
* USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_ttm.h"
#include "nouveau_gem.h"
diff --git a/drivers/gpu/drm/nouveau/nouveau_usif.c b/drivers/gpu/drm/nouveau/nouveau_usif.c
index e9f52ef..675e9e0 100644
--- a/drivers/gpu/drm/nouveau/nouveau_usif.c
+++ b/drivers/gpu/drm/nouveau/nouveau_usif.c
@@ -22,7 +22,7 @@
* Authors: Ben Skeggs <bskeggs@redhat.com>
*/
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_usif.h"
#include "nouveau_abi16.h"
diff --git a/drivers/gpu/drm/nouveau/nouveau_vga.c b/drivers/gpu/drm/nouveau/nouveau_vga.c
index af89c36..c6a180a 100644
--- a/drivers/gpu/drm/nouveau/nouveau_vga.c
+++ b/drivers/gpu/drm/nouveau/nouveau_vga.c
@@ -4,7 +4,7 @@
#include <drm/drmP.h>
#include <drm/drm_crtc_helper.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_acpi.h"
#include "nouveau_fbcon.h"
#include "nouveau_vga.h"
diff --git a/drivers/gpu/drm/nouveau/nv04_fbcon.c b/drivers/gpu/drm/nouveau/nv04_fbcon.c
index 789dc29..0f3e4bb 100644
--- a/drivers/gpu/drm/nouveau/nv04_fbcon.c
+++ b/drivers/gpu/drm/nouveau/nv04_fbcon.c
@@ -22,7 +22,7 @@
* DEALINGS IN THE SOFTWARE.
*/
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_dma.h"
#include "nouveau_fbcon.h"
diff --git a/drivers/gpu/drm/nouveau/nv04_fence.c b/drivers/gpu/drm/nouveau/nv04_fence.c
index 3022d24..1915b7b 100644
--- a/drivers/gpu/drm/nouveau/nv04_fence.c
+++ b/drivers/gpu/drm/nouveau/nv04_fence.c
@@ -22,7 +22,7 @@
* Authors: Ben Skeggs
*/
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_dma.h"
#include "nouveau_fence.h"
diff --git a/drivers/gpu/drm/nouveau/nv10_fence.c b/drivers/gpu/drm/nouveau/nv10_fence.c
index 2c35213..4e3de34 100644
--- a/drivers/gpu/drm/nouveau/nv10_fence.c
+++ b/drivers/gpu/drm/nouveau/nv10_fence.c
@@ -22,7 +22,7 @@
* Authors: Ben Skeggs <bskeggs@redhat.com>
*/
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_dma.h"
#include "nv10_fence.h"
diff --git a/drivers/gpu/drm/nouveau/nv17_fence.c b/drivers/gpu/drm/nouveau/nv17_fence.c
index 6a141c9..7d5e562 100644
--- a/drivers/gpu/drm/nouveau/nv17_fence.c
+++ b/drivers/gpu/drm/nouveau/nv17_fence.c
@@ -26,7 +26,7 @@
#include <nvif/class.h>
#include <nvif/cl0002.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_dma.h"
#include "nv10_fence.h"
diff --git a/drivers/gpu/drm/nouveau/nv50_display.c b/drivers/gpu/drm/nouveau/nv50_display.c
index a43445c..3ffc2b0 100644
--- a/drivers/gpu/drm/nouveau/nv50_display.c
+++ b/drivers/gpu/drm/nouveau/nv50_display.c
@@ -39,7 +39,7 @@
#include <nvif/cl507d.h>
#include <nvif/cl507e.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_dma.h"
#include "nouveau_gem.h"
#include "nouveau_connector.h"
@@ -1305,7 +1305,6 @@
uint32_t handle, uint32_t width, uint32_t height)
{
struct nouveau_crtc *nv_crtc = nouveau_crtc(crtc);
- struct drm_device *dev = crtc->dev;
struct drm_gem_object *gem = NULL;
struct nouveau_bo *nvbo = NULL;
int ret = 0;
@@ -1314,7 +1313,7 @@
if (width != 64 || height != 64)
return -EINVAL;
- gem = drm_gem_object_lookup(dev, file_priv, handle);
+ gem = drm_gem_object_lookup(file_priv, handle);
if (unlikely(!gem))
return -ENOENT;
nvbo = nouveau_gem_object(gem);
diff --git a/drivers/gpu/drm/nouveau/nv50_fbcon.c b/drivers/gpu/drm/nouveau/nv50_fbcon.c
index e05499d..33d9ee0 100644
--- a/drivers/gpu/drm/nouveau/nv50_fbcon.c
+++ b/drivers/gpu/drm/nouveau/nv50_fbcon.c
@@ -22,7 +22,7 @@
* Authors: Ben Skeggs
*/
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_dma.h"
#include "nouveau_fbcon.h"
diff --git a/drivers/gpu/drm/nouveau/nv50_fence.c b/drivers/gpu/drm/nouveau/nv50_fence.c
index 3695ccce..4d6f202 100644
--- a/drivers/gpu/drm/nouveau/nv50_fence.c
+++ b/drivers/gpu/drm/nouveau/nv50_fence.c
@@ -26,7 +26,7 @@
#include <nvif/class.h>
#include <nvif/cl0002.h>
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_dma.h"
#include "nv10_fence.h"
diff --git a/drivers/gpu/drm/nouveau/nv84_fence.c b/drivers/gpu/drm/nouveau/nv84_fence.c
index 412c5be..18bde9d 100644
--- a/drivers/gpu/drm/nouveau/nv84_fence.c
+++ b/drivers/gpu/drm/nouveau/nv84_fence.c
@@ -22,7 +22,7 @@
* Authors: Ben Skeggs
*/
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_dma.h"
#include "nouveau_fence.h"
diff --git a/drivers/gpu/drm/nouveau/nvc0_fbcon.c b/drivers/gpu/drm/nouveau/nvc0_fbcon.c
index c97395b..a091335 100644
--- a/drivers/gpu/drm/nouveau/nvc0_fbcon.c
+++ b/drivers/gpu/drm/nouveau/nvc0_fbcon.c
@@ -22,7 +22,7 @@
* Authors: Ben Skeggs
*/
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_dma.h"
#include "nouveau_fbcon.h"
diff --git a/drivers/gpu/drm/nouveau/nvc0_fence.c b/drivers/gpu/drm/nouveau/nvc0_fence.c
index becf19a..b797757 100644
--- a/drivers/gpu/drm/nouveau/nvc0_fence.c
+++ b/drivers/gpu/drm/nouveau/nvc0_fence.c
@@ -22,7 +22,7 @@
* Authors: Ben Skeggs
*/
-#include "nouveau_drm.h"
+#include "nouveau_drv.h"
#include "nouveau_dma.h"
#include "nouveau_fence.h"
diff --git a/drivers/gpu/drm/nouveau/nvkm/core/engine.c b/drivers/gpu/drm/nouveau/nvkm/core/engine.c
index 8a7bae7..ee8e583 100644
--- a/drivers/gpu/drm/nouveau/nvkm/core/engine.c
+++ b/drivers/gpu/drm/nouveau/nvkm/core/engine.c
@@ -137,11 +137,10 @@
int
nvkm_engine_ctor(const struct nvkm_engine_func *func,
- struct nvkm_device *device, int index, u32 pmc_enable,
- bool enable, struct nvkm_engine *engine)
+ struct nvkm_device *device, int index, bool enable,
+ struct nvkm_engine *engine)
{
- nvkm_subdev_ctor(&nvkm_engine_func, device, index,
- pmc_enable, &engine->subdev);
+ nvkm_subdev_ctor(&nvkm_engine_func, device, index, &engine->subdev);
engine->func = func;
if (!nvkm_boolopt(device->cfgopt, nvkm_subdev_name[index], enable)) {
@@ -155,11 +154,10 @@
int
nvkm_engine_new_(const struct nvkm_engine_func *func,
- struct nvkm_device *device, int index, u32 pmc_enable,
- bool enable, struct nvkm_engine **pengine)
+ struct nvkm_device *device, int index, bool enable,
+ struct nvkm_engine **pengine)
{
if (!(*pengine = kzalloc(sizeof(**pengine), GFP_KERNEL)))
return -ENOMEM;
- return nvkm_engine_ctor(func, device, index, pmc_enable,
- enable, *pengine);
+ return nvkm_engine_ctor(func, device, index, enable, *pengine);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/core/subdev.c b/drivers/gpu/drm/nouveau/nvkm/core/subdev.c
index 3bf08cb..b185578 100644
--- a/drivers/gpu/drm/nouveau/nvkm/core/subdev.c
+++ b/drivers/gpu/drm/nouveau/nvkm/core/subdev.c
@@ -24,6 +24,7 @@
#include <core/subdev.h>
#include <core/device.h>
#include <core/option.h>
+#include <subdev/mc.h>
static struct lock_class_key nvkm_subdev_lock_class[NVKM_SUBDEV_NR];
@@ -50,6 +51,7 @@
[NVKM_SUBDEV_SECBOOT ] = "secboot",
[NVKM_SUBDEV_THERM ] = "therm",
[NVKM_SUBDEV_TIMER ] = "tmr",
+ [NVKM_SUBDEV_TOP ] = "top",
[NVKM_SUBDEV_VOLT ] = "volt",
[NVKM_ENGINE_BSP ] = "bsp",
[NVKM_ENGINE_CE0 ] = "ce0",
@@ -89,7 +91,6 @@
{
struct nvkm_device *device = subdev->device;
const char *action = suspend ? "suspend" : "fini";
- u32 pmc_enable = subdev->pmc_enable;
s64 time;
nvkm_trace(subdev, "%s running...\n", action);
@@ -104,11 +105,7 @@
}
}
- if (pmc_enable) {
- nvkm_mask(device, 0x000200, pmc_enable, 0x00000000);
- nvkm_mask(device, 0x000200, pmc_enable, pmc_enable);
- nvkm_rd32(device, 0x000200);
- }
+ nvkm_mc_reset(device->mc, subdev->index);
time = ktime_to_us(ktime_get()) - time;
nvkm_trace(subdev, "%s completed in %lldus\n", action, time);
@@ -193,14 +190,13 @@
void
nvkm_subdev_ctor(const struct nvkm_subdev_func *func,
- struct nvkm_device *device, int index, u32 pmc_enable,
+ struct nvkm_device *device, int index,
struct nvkm_subdev *subdev)
{
const char *name = nvkm_subdev_name[index];
subdev->func = func;
subdev->device = device;
subdev->index = index;
- subdev->pmc_enable = pmc_enable;
__mutex_init(&subdev->mutex, name, &nvkm_subdev_lock_class[index]);
subdev->debug = nvkm_dbgopt(device->dbgopt, name);
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/bsp/g84.c b/drivers/gpu/drm/nouveau/nvkm/engine/bsp/g84.c
index 3ef0107..8e2e24a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/bsp/g84.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/bsp/g84.c
@@ -27,7 +27,6 @@
static const struct nvkm_xtensa_func
g84_bsp = {
- .pmc_enable = 0x04008000,
.fifo_val = 0x1111,
.unkd28 = 0x90044,
.sclass = {
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/ce/gf100.c b/drivers/gpu/drm/nouveau/nvkm/engine/ce/gf100.c
index 92a9f35..ad9f855 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/ce/gf100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/ce/gf100.c
@@ -40,7 +40,6 @@
.code.size = sizeof(gf100_ce_code),
.data.data = gf100_ce_data,
.data.size = sizeof(gf100_ce_data),
- .pmc_enable = 0x00000040,
.init = gf100_ce_init,
.intr = gt215_ce_intr,
.sclass = {
@@ -55,7 +54,6 @@
.code.size = sizeof(gf100_ce_code),
.data.data = gf100_ce_data,
.data.size = sizeof(gf100_ce_data),
- .pmc_enable = 0x00000080,
.init = gf100_ce_init,
.intr = gt215_ce_intr,
.sclass = {
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/ce/gk104.c b/drivers/gpu/drm/nouveau/nvkm/engine/ce/gk104.c
index e2b944d..9e0b53a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/ce/gk104.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/ce/gk104.c
@@ -97,17 +97,5 @@
gk104_ce_new(struct nvkm_device *device, int index,
struct nvkm_engine **pengine)
{
- if (index == NVKM_ENGINE_CE0) {
- return nvkm_engine_new_(&gk104_ce, device, index,
- 0x00000040, true, pengine);
- } else
- if (index == NVKM_ENGINE_CE1) {
- return nvkm_engine_new_(&gk104_ce, device, index,
- 0x00000080, true, pengine);
- } else
- if (index == NVKM_ENGINE_CE2) {
- return nvkm_engine_new_(&gk104_ce, device, index,
- 0x00200000, true, pengine);
- }
- return -ENODEV;
+ return nvkm_engine_new_(&gk104_ce, device, index, true, pengine);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/ce/gm107.c b/drivers/gpu/drm/nouveau/nvkm/engine/ce/gm107.c
index 4c2f429..c0df7da 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/ce/gm107.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/ce/gm107.c
@@ -39,17 +39,5 @@
gm107_ce_new(struct nvkm_device *device, int index,
struct nvkm_engine **pengine)
{
- if (index == NVKM_ENGINE_CE0) {
- return nvkm_engine_new_(&gm107_ce, device, index,
- 0x00000040, true, pengine);
- } else
- if (index == NVKM_ENGINE_CE1) {
- return nvkm_engine_new_(&gm107_ce, device, index,
- 0x00000080, true, pengine);
- } else
- if (index == NVKM_ENGINE_CE2) {
- return nvkm_engine_new_(&gm107_ce, device, index,
- 0x00200000, true, pengine);
- }
- return -ENODEV;
+ return nvkm_engine_new_(&gm107_ce, device, index, true, pengine);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/ce/gm200.c b/drivers/gpu/drm/nouveau/nvkm/engine/ce/gm200.c
index 13f07b3..c6fa8b2 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/ce/gm200.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/ce/gm200.c
@@ -38,17 +38,5 @@
gm200_ce_new(struct nvkm_device *device, int index,
struct nvkm_engine **pengine)
{
- if (index == NVKM_ENGINE_CE0) {
- return nvkm_engine_new_(&gm200_ce, device, index,
- 0x00000040, true, pengine);
- } else
- if (index == NVKM_ENGINE_CE1) {
- return nvkm_engine_new_(&gm200_ce, device, index,
- 0x00000080, true, pengine);
- } else
- if (index == NVKM_ENGINE_CE2) {
- return nvkm_engine_new_(&gm200_ce, device, index,
- 0x00200000, true, pengine);
- }
- return -ENODEV;
+ return nvkm_engine_new_(&gm200_ce, device, index, true, pengine);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/ce/gt215.c b/drivers/gpu/drm/nouveau/nvkm/engine/ce/gt215.c
index 402dcbc..63ac51a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/ce/gt215.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/ce/gt215.c
@@ -67,7 +67,6 @@
.code.size = sizeof(gt215_ce_code),
.data.data = gt215_ce_data,
.data.size = sizeof(gt215_ce_data),
- .pmc_enable = 0x00802000,
.intr = gt215_ce_intr,
.sclass = {
{ -1, -1, GT212_DMA },
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/cipher/g84.c b/drivers/gpu/drm/nouveau/nvkm/engine/cipher/g84.c
index bfd0162..68ffb52 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/cipher/g84.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/cipher/g84.c
@@ -130,6 +130,5 @@
g84_cipher_new(struct nvkm_device *device, int index,
struct nvkm_engine **pengine)
{
- return nvkm_engine_new_(&g84_cipher, device, index,
- 0x00004000, true, pengine);
+ return nvkm_engine_new_(&g84_cipher, device, index, true, pengine);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/device/base.c b/drivers/gpu/drm/nouveau/nvkm/engine/device/base.c
index 9f32c87..4572deb 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/device/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/device/base.c
@@ -146,7 +146,7 @@
.gpio = nv10_gpio_new,
.i2c = nv04_i2c_new,
.imem = nv04_instmem_new,
- .mc = nv04_mc_new,
+ .mc = nv11_mc_new,
.mmu = nv04_mmu_new,
.pci = nv04_pci_new,
.timer = nv04_timer_new,
@@ -190,7 +190,7 @@
.gpio = nv10_gpio_new,
.i2c = nv04_i2c_new,
.imem = nv04_instmem_new,
- .mc = nv04_mc_new,
+ .mc = nv17_mc_new,
.mmu = nv04_mmu_new,
.pci = nv04_pci_new,
.timer = nv04_timer_new,
@@ -212,7 +212,7 @@
.gpio = nv10_gpio_new,
.i2c = nv04_i2c_new,
.imem = nv04_instmem_new,
- .mc = nv04_mc_new,
+ .mc = nv17_mc_new,
.mmu = nv04_mmu_new,
.pci = nv04_pci_new,
.timer = nv04_timer_new,
@@ -256,7 +256,7 @@
.gpio = nv10_gpio_new,
.i2c = nv04_i2c_new,
.imem = nv04_instmem_new,
- .mc = nv04_mc_new,
+ .mc = nv17_mc_new,
.mmu = nv04_mmu_new,
.pci = nv04_pci_new,
.timer = nv04_timer_new,
@@ -278,7 +278,7 @@
.gpio = nv10_gpio_new,
.i2c = nv04_i2c_new,
.imem = nv04_instmem_new,
- .mc = nv04_mc_new,
+ .mc = nv17_mc_new,
.mmu = nv04_mmu_new,
.pci = nv04_pci_new,
.timer = nv04_timer_new,
@@ -300,7 +300,7 @@
.gpio = nv10_gpio_new,
.i2c = nv04_i2c_new,
.imem = nv04_instmem_new,
- .mc = nv04_mc_new,
+ .mc = nv17_mc_new,
.mmu = nv04_mmu_new,
.pci = nv04_pci_new,
.timer = nv04_timer_new,
@@ -322,7 +322,7 @@
.gpio = nv10_gpio_new,
.i2c = nv04_i2c_new,
.imem = nv04_instmem_new,
- .mc = nv04_mc_new,
+ .mc = nv17_mc_new,
.mmu = nv04_mmu_new,
.pci = nv04_pci_new,
.timer = nv04_timer_new,
@@ -344,7 +344,7 @@
.gpio = nv10_gpio_new,
.i2c = nv04_i2c_new,
.imem = nv04_instmem_new,
- .mc = nv04_mc_new,
+ .mc = nv17_mc_new,
.mmu = nv04_mmu_new,
.pci = nv04_pci_new,
.timer = nv04_timer_new,
@@ -366,7 +366,7 @@
.gpio = nv10_gpio_new,
.i2c = nv04_i2c_new,
.imem = nv04_instmem_new,
- .mc = nv04_mc_new,
+ .mc = nv17_mc_new,
.mmu = nv04_mmu_new,
.pci = nv04_pci_new,
.timer = nv04_timer_new,
@@ -388,7 +388,7 @@
.gpio = nv10_gpio_new,
.i2c = nv04_i2c_new,
.imem = nv04_instmem_new,
- .mc = nv04_mc_new,
+ .mc = nv17_mc_new,
.mmu = nv04_mmu_new,
.pci = nv04_pci_new,
.timer = nv04_timer_new,
@@ -411,7 +411,7 @@
.gpio = nv10_gpio_new,
.i2c = nv04_i2c_new,
.imem = nv04_instmem_new,
- .mc = nv04_mc_new,
+ .mc = nv17_mc_new,
.mmu = nv04_mmu_new,
.pci = nv04_pci_new,
.timer = nv04_timer_new,
@@ -434,7 +434,7 @@
.gpio = nv10_gpio_new,
.i2c = nv04_i2c_new,
.imem = nv04_instmem_new,
- .mc = nv04_mc_new,
+ .mc = nv17_mc_new,
.mmu = nv04_mmu_new,
.pci = nv04_pci_new,
.timer = nv04_timer_new,
@@ -456,7 +456,7 @@
.gpio = nv10_gpio_new,
.i2c = nv04_i2c_new,
.imem = nv04_instmem_new,
- .mc = nv04_mc_new,
+ .mc = nv17_mc_new,
.mmu = nv04_mmu_new,
.pci = nv04_pci_new,
.timer = nv04_timer_new,
@@ -479,7 +479,7 @@
.gpio = nv10_gpio_new,
.i2c = nv04_i2c_new,
.imem = nv40_instmem_new,
- .mc = nv04_mc_new,
+ .mc = nv17_mc_new,
.mmu = nv04_mmu_new,
.pci = nv40_pci_new,
.therm = nv40_therm_new,
@@ -505,7 +505,7 @@
.gpio = nv10_gpio_new,
.i2c = nv04_i2c_new,
.imem = nv40_instmem_new,
- .mc = nv04_mc_new,
+ .mc = nv17_mc_new,
.mmu = nv41_mmu_new,
.pci = nv40_pci_new,
.therm = nv40_therm_new,
@@ -531,7 +531,7 @@
.gpio = nv10_gpio_new,
.i2c = nv04_i2c_new,
.imem = nv40_instmem_new,
- .mc = nv04_mc_new,
+ .mc = nv17_mc_new,
.mmu = nv41_mmu_new,
.pci = nv40_pci_new,
.therm = nv40_therm_new,
@@ -557,7 +557,7 @@
.gpio = nv10_gpio_new,
.i2c = nv04_i2c_new,
.imem = nv40_instmem_new,
- .mc = nv04_mc_new,
+ .mc = nv17_mc_new,
.mmu = nv41_mmu_new,
.pci = nv40_pci_new,
.therm = nv40_therm_new,
@@ -609,7 +609,7 @@
.gpio = nv10_gpio_new,
.i2c = nv04_i2c_new,
.imem = nv40_instmem_new,
- .mc = nv04_mc_new,
+ .mc = nv17_mc_new,
.mmu = nv04_mmu_new,
.pci = nv40_pci_new,
.therm = nv40_therm_new,
@@ -661,7 +661,7 @@
.gpio = nv10_gpio_new,
.i2c = nv04_i2c_new,
.imem = nv40_instmem_new,
- .mc = nv04_mc_new,
+ .mc = nv17_mc_new,
.mmu = nv41_mmu_new,
.pci = nv40_pci_new,
.therm = nv40_therm_new,
@@ -687,7 +687,7 @@
.gpio = nv10_gpio_new,
.i2c = nv04_i2c_new,
.imem = nv40_instmem_new,
- .mc = nv04_mc_new,
+ .mc = nv17_mc_new,
.mmu = nv41_mmu_new,
.pci = nv40_pci_new,
.therm = nv40_therm_new,
@@ -739,7 +739,7 @@
.gpio = nv10_gpio_new,
.i2c = nv04_i2c_new,
.imem = nv40_instmem_new,
- .mc = nv04_mc_new,
+ .mc = nv17_mc_new,
.mmu = nv41_mmu_new,
.pci = nv40_pci_new,
.therm = nv40_therm_new,
@@ -926,7 +926,7 @@
.gpio = nv50_gpio_new,
.i2c = nv50_i2c_new,
.imem = nv50_instmem_new,
- .mc = nv50_mc_new,
+ .mc = g84_mc_new,
.mmu = nv50_mmu_new,
.mxm = nv50_mxm_new,
.pci = g84_pci_new,
@@ -958,7 +958,7 @@
.gpio = nv50_gpio_new,
.i2c = nv50_i2c_new,
.imem = nv50_instmem_new,
- .mc = nv50_mc_new,
+ .mc = g84_mc_new,
.mmu = nv50_mmu_new,
.mxm = nv50_mxm_new,
.pci = g84_pci_new,
@@ -990,7 +990,7 @@
.gpio = nv50_gpio_new,
.i2c = nv50_i2c_new,
.imem = nv50_instmem_new,
- .mc = nv50_mc_new,
+ .mc = g84_mc_new,
.mmu = nv50_mmu_new,
.mxm = nv50_mxm_new,
.pci = g84_pci_new,
@@ -1022,7 +1022,7 @@
.gpio = g94_gpio_new,
.i2c = g94_i2c_new,
.imem = nv50_instmem_new,
- .mc = nv50_mc_new,
+ .mc = g84_mc_new,
.mmu = nv50_mmu_new,
.mxm = nv50_mxm_new,
.pci = g94_pci_new,
@@ -1054,7 +1054,7 @@
.gpio = g94_gpio_new,
.i2c = g94_i2c_new,
.imem = nv50_instmem_new,
- .mc = nv50_mc_new,
+ .mc = g84_mc_new,
.mmu = nv50_mmu_new,
.mxm = nv50_mxm_new,
.pci = g94_pci_new,
@@ -1118,7 +1118,7 @@
.gpio = g94_gpio_new,
.i2c = nv50_i2c_new,
.imem = nv50_instmem_new,
- .mc = g98_mc_new,
+ .mc = g84_mc_new,
.mmu = nv50_mmu_new,
.mxm = nv50_mxm_new,
.pci = g94_pci_new,
@@ -1150,7 +1150,7 @@
.gpio = g94_gpio_new,
.i2c = g94_i2c_new,
.imem = nv50_instmem_new,
- .mc = g98_mc_new,
+ .mc = gt215_mc_new,
.mmu = nv50_mmu_new,
.mxm = nv50_mxm_new,
.pci = g94_pci_new,
@@ -1184,7 +1184,7 @@
.gpio = g94_gpio_new,
.i2c = g94_i2c_new,
.imem = nv50_instmem_new,
- .mc = g98_mc_new,
+ .mc = gt215_mc_new,
.mmu = nv50_mmu_new,
.mxm = nv50_mxm_new,
.pci = g94_pci_new,
@@ -1217,7 +1217,7 @@
.gpio = g94_gpio_new,
.i2c = g94_i2c_new,
.imem = nv50_instmem_new,
- .mc = g98_mc_new,
+ .mc = gt215_mc_new,
.mmu = nv50_mmu_new,
.mxm = nv50_mxm_new,
.pci = g94_pci_new,
@@ -1314,7 +1314,7 @@
.gpio = g94_gpio_new,
.i2c = g94_i2c_new,
.imem = nv50_instmem_new,
- .mc = g98_mc_new,
+ .mc = gt215_mc_new,
.mmu = nv50_mmu_new,
.mxm = nv50_mxm_new,
.pci = g94_pci_new,
@@ -1676,13 +1676,14 @@
.iccsense = gf100_iccsense_new,
.imem = nv50_instmem_new,
.ltc = gk104_ltc_new,
- .mc = gf100_mc_new,
+ .mc = gk104_mc_new,
.mmu = gf100_mmu_new,
.mxm = nv50_mxm_new,
.pci = gk104_pci_new,
.pmu = gk104_pmu_new,
.therm = gf119_therm_new,
.timer = nv41_timer_new,
+ .top = gk104_top_new,
.volt = gk104_volt_new,
.ce[0] = gk104_ce_new,
.ce[1] = gk104_ce_new,
@@ -1714,13 +1715,14 @@
.iccsense = gf100_iccsense_new,
.imem = nv50_instmem_new,
.ltc = gk104_ltc_new,
- .mc = gf100_mc_new,
+ .mc = gk104_mc_new,
.mmu = gf100_mmu_new,
.mxm = nv50_mxm_new,
.pci = gk104_pci_new,
.pmu = gk104_pmu_new,
.therm = gf119_therm_new,
.timer = nv41_timer_new,
+ .top = gk104_top_new,
.volt = gk104_volt_new,
.ce[0] = gk104_ce_new,
.ce[1] = gk104_ce_new,
@@ -1752,13 +1754,14 @@
.iccsense = gf100_iccsense_new,
.imem = nv50_instmem_new,
.ltc = gk104_ltc_new,
- .mc = gf100_mc_new,
+ .mc = gk104_mc_new,
.mmu = gf100_mmu_new,
.mxm = nv50_mxm_new,
.pci = gk104_pci_new,
.pmu = gk104_pmu_new,
.therm = gf119_therm_new,
.timer = nv41_timer_new,
+ .top = gk104_top_new,
.volt = gk104_volt_new,
.ce[0] = gk104_ce_new,
.ce[1] = gk104_ce_new,
@@ -1789,6 +1792,7 @@
.mmu = gf100_mmu_new,
.pmu = gk20a_pmu_new,
.timer = gk20a_timer_new,
+ .top = gk104_top_new,
.volt = gk20a_volt_new,
.ce[2] = gk104_ce_new,
.dma = gf119_dma_new,
@@ -1814,13 +1818,14 @@
.iccsense = gf100_iccsense_new,
.imem = nv50_instmem_new,
.ltc = gk104_ltc_new,
- .mc = gf100_mc_new,
+ .mc = gk104_mc_new,
.mmu = gf100_mmu_new,
.mxm = nv50_mxm_new,
.pci = gk104_pci_new,
.pmu = gk110_pmu_new,
.therm = gf119_therm_new,
.timer = nv41_timer_new,
+ .top = gk104_top_new,
.volt = gk104_volt_new,
.ce[0] = gk104_ce_new,
.ce[1] = gk104_ce_new,
@@ -1851,13 +1856,14 @@
.iccsense = gf100_iccsense_new,
.imem = nv50_instmem_new,
.ltc = gk104_ltc_new,
- .mc = gf100_mc_new,
+ .mc = gk104_mc_new,
.mmu = gf100_mmu_new,
.mxm = nv50_mxm_new,
.pci = gk104_pci_new,
.pmu = gk110_pmu_new,
.therm = gf119_therm_new,
.timer = nv41_timer_new,
+ .top = gk104_top_new,
.volt = gk104_volt_new,
.ce[0] = gk104_ce_new,
.ce[1] = gk104_ce_new,
@@ -1895,6 +1901,7 @@
.pmu = gk208_pmu_new,
.therm = gf119_therm_new,
.timer = nv41_timer_new,
+ .top = gk104_top_new,
.volt = gk104_volt_new,
.ce[0] = gk104_ce_new,
.ce[1] = gk104_ce_new,
@@ -1932,6 +1939,7 @@
.pmu = gk208_pmu_new,
.therm = gf119_therm_new,
.timer = nv41_timer_new,
+ .top = gk104_top_new,
.volt = gk104_volt_new,
.ce[0] = gk104_ce_new,
.ce[1] = gk104_ce_new,
@@ -1969,6 +1977,41 @@
.pmu = gm107_pmu_new,
.therm = gm107_therm_new,
.timer = gk20a_timer_new,
+ .top = gk104_top_new,
+ .volt = gk104_volt_new,
+ .ce[0] = gm107_ce_new,
+ .ce[2] = gm107_ce_new,
+ .disp = gm107_disp_new,
+ .dma = gf119_dma_new,
+ .fifo = gm107_fifo_new,
+ .gr = gm107_gr_new,
+ .sw = gf100_sw_new,
+};
+
+static const struct nvkm_device_chip
+nv118_chipset = {
+ .name = "GM108",
+ .bar = gf100_bar_new,
+ .bios = nvkm_bios_new,
+ .bus = gf100_bus_new,
+ .clk = gk104_clk_new,
+ .devinit = gm107_devinit_new,
+ .fb = gm107_fb_new,
+ .fuse = gm107_fuse_new,
+ .gpio = gk104_gpio_new,
+ .i2c = gf119_i2c_new,
+ .ibus = gk104_ibus_new,
+ .iccsense = gf100_iccsense_new,
+ .imem = nv50_instmem_new,
+ .ltc = gm107_ltc_new,
+ .mc = gk20a_mc_new,
+ .mmu = gf100_mmu_new,
+ .mxm = nv50_mxm_new,
+ .pci = gk104_pci_new,
+ .pmu = gm107_pmu_new,
+ .therm = gm107_therm_new,
+ .timer = gk20a_timer_new,
+ .top = gk104_top_new,
.volt = gk104_volt_new,
.ce[0] = gm107_ce_new,
.ce[2] = gm107_ce_new,
@@ -1986,7 +2029,7 @@
.bios = nvkm_bios_new,
.bus = gf100_bus_new,
.devinit = gm200_devinit_new,
- .fb = gm107_fb_new,
+ .fb = gm200_fb_new,
.fuse = gm107_fuse_new,
.gpio = gk104_gpio_new,
.i2c = gm200_i2c_new,
@@ -2001,6 +2044,7 @@
.pmu = gm107_pmu_new,
.secboot = gm200_secboot_new,
.timer = gk20a_timer_new,
+ .top = gk104_top_new,
.volt = gk104_volt_new,
.ce[0] = gm200_ce_new,
.ce[1] = gm200_ce_new,
@@ -2019,7 +2063,7 @@
.bios = nvkm_bios_new,
.bus = gf100_bus_new,
.devinit = gm200_devinit_new,
- .fb = gm107_fb_new,
+ .fb = gm200_fb_new,
.fuse = gm107_fuse_new,
.gpio = gk104_gpio_new,
.i2c = gm200_i2c_new,
@@ -2034,6 +2078,7 @@
.pmu = gm107_pmu_new,
.secboot = gm200_secboot_new,
.timer = gk20a_timer_new,
+ .top = gk104_top_new,
.volt = gk104_volt_new,
.ce[0] = gm200_ce_new,
.ce[1] = gm200_ce_new,
@@ -2052,7 +2097,7 @@
.bios = nvkm_bios_new,
.bus = gf100_bus_new,
.devinit = gm200_devinit_new,
- .fb = gm107_fb_new,
+ .fb = gm200_fb_new,
.fuse = gm107_fuse_new,
.gpio = gk104_gpio_new,
.i2c = gm200_i2c_new,
@@ -2067,6 +2112,7 @@
.pmu = gm107_pmu_new,
.secboot = gm200_secboot_new,
.timer = gk20a_timer_new,
+ .top = gk104_top_new,
.volt = gk104_volt_new,
.ce[0] = gm200_ce_new,
.ce[1] = gm200_ce_new,
@@ -2093,6 +2139,7 @@
.mmu = gf100_mmu_new,
.secboot = gm20b_secboot_new,
.timer = gk20a_timer_new,
+ .top = gk104_top_new,
.ce[2] = gm200_ce_new,
.volt = gm20b_volt_new,
.dma = gf119_dma_new,
@@ -2150,6 +2197,7 @@
_(SECBOOT , device->secboot , &device->secboot->subdev);
_(THERM , device->therm , &device->therm->subdev);
_(TIMER , device->timer , &device->timer->subdev);
+ _(TOP , device->top , &device->top->subdev);
_(VOLT , device->volt , &device->volt->subdev);
#undef _
default:
@@ -2523,6 +2571,7 @@
case 0x106: device->chip = &nv106_chipset; break;
case 0x108: device->chip = &nv108_chipset; break;
case 0x117: device->chip = &nv117_chipset; break;
+ case 0x118: device->chip = &nv118_chipset; break;
case 0x120: device->chip = &nv120_chipset; break;
case 0x124: device->chip = &nv124_chipset; break;
case 0x126: device->chip = &nv126_chipset; break;
@@ -2604,6 +2653,7 @@
_(NVKM_SUBDEV_SECBOOT , secboot);
_(NVKM_SUBDEV_THERM , therm);
_(NVKM_SUBDEV_TIMER , timer);
+ _(NVKM_SUBDEV_TOP , top);
_(NVKM_SUBDEV_VOLT , volt);
_(NVKM_ENGINE_BSP , bsp);
_(NVKM_ENGINE_CE0 , ce[0]);
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/device/priv.h b/drivers/gpu/drm/nouveau/nvkm/engine/device/priv.h
index e80f6ab..1a06ac1 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/device/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/device/priv.h
@@ -22,6 +22,7 @@
#include <subdev/pmu.h>
#include <subdev/therm.h>
#include <subdev/timer.h>
+#include <subdev/top.h>
#include <subdev/volt.h>
#include <subdev/secboot.h>
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/base.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/base.c
index 785fa76..1efe91b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/base.c
@@ -298,8 +298,7 @@
disp->func = func;
disp->head.nr = heads;
- ret = nvkm_engine_ctor(&nvkm_disp, device, index, 0,
- true, &disp->engine);
+ ret = nvkm_engine_ctor(&nvkm_disp, device, index, true, &disp->engine);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/dma/base.c b/drivers/gpu/drm/nouveau/nvkm/engine/dma/base.c
index 9769fc0..f11ebdd 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/dma/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/dma/base.c
@@ -152,6 +152,5 @@
return -ENOMEM;
dma->func = func;
- return nvkm_engine_ctor(&nvkm_dma, device, index,
- 0, true, &dma->engine);
+ return nvkm_engine_ctor(&nvkm_dma, device, index, true, &dma->engine);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/falcon.c b/drivers/gpu/drm/nouveau/nvkm/engine/falcon.c
index 7400060..2e7b4e2 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/falcon.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/falcon.c
@@ -348,6 +348,6 @@
falcon->data.size = func->data.size;
*pengine = &falcon->engine;
- return nvkm_engine_ctor(&nvkm_falcon, device, index, func->pmc_enable,
+ return nvkm_engine_ctor(&nvkm_falcon, device, index,
enable, &falcon->engine);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/base.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/base.c
index cfc7d57..1c9682a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/base.c
@@ -178,6 +178,17 @@
const struct nvkm_fifo_chan_oclass *sclass;
int c = 0;
+ if (fifo->func->class_get) {
+ int ret = fifo->func->class_get(fifo, index, &sclass);
+ if (ret == 0) {
+ oclass->base = sclass->base;
+ oclass->engn = sclass;
+ *class = &nvkm_fifo_class;
+ return 0;
+ }
+ return ret;
+ }
+
while ((sclass = fifo->func->chan[c])) {
if (c++ == index) {
oclass->base = sclass->base;
@@ -261,8 +272,7 @@
fifo->nr = nr;
bitmap_clear(fifo->mask, 0, fifo->nr);
- ret = nvkm_engine_ctor(&nvkm_fifo, device, index, 0x00000100,
- true, &fifo->engine);
+ ret = nvkm_engine_ctor(&nvkm_fifo, device, index, true, &fifo->engine);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.c
index 68acb36..743f3a1 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.c
@@ -25,21 +25,36 @@
#include "changk104.h"
#include <core/client.h>
-#include <core/enum.h>
#include <core/gpuobj.h>
#include <subdev/bar.h>
+#include <subdev/top.h>
#include <engine/sw.h>
#include <nvif/class.h>
-void
+static int
+gk104_fifo_class_get(struct nvkm_fifo *base, int index,
+ const struct nvkm_fifo_chan_oclass **psclass)
+{
+ struct gk104_fifo *fifo = gk104_fifo(base);
+ int c = 0;
+
+ while ((*psclass = fifo->func->chan[c])) {
+ if (c++ == index)
+ return 0;
+ }
+
+ return c;
+}
+
+static void
gk104_fifo_uevent_fini(struct nvkm_fifo *fifo)
{
struct nvkm_device *device = fifo->engine.subdev.device;
nvkm_mask(device, 0x002140, 0x80000000, 0x00000000);
}
-void
+static void
gk104_fifo_uevent_init(struct nvkm_fifo *fifo)
{
struct nvkm_device *device = fifo->engine.subdev.device;
@@ -267,111 +282,6 @@
nvkm_error(subdev, "DROPPED_MMU_FAULT %08x\n", stat);
}
-static const struct nvkm_enum
-gk104_fifo_fault_engine[] = {
- { 0x00, "GR", NULL, NVKM_ENGINE_GR },
- { 0x03, "IFB", NULL, NVKM_ENGINE_IFB },
- { 0x04, "BAR1", NULL, NVKM_SUBDEV_BAR },
- { 0x05, "BAR3", NULL, NVKM_SUBDEV_INSTMEM },
- { 0x07, "PBDMA0", NULL, NVKM_ENGINE_FIFO },
- { 0x08, "PBDMA1", NULL, NVKM_ENGINE_FIFO },
- { 0x09, "PBDMA2", NULL, NVKM_ENGINE_FIFO },
- { 0x10, "MSVLD", NULL, NVKM_ENGINE_MSVLD },
- { 0x11, "MSPPP", NULL, NVKM_ENGINE_MSPPP },
- { 0x13, "PERF" },
- { 0x14, "MSPDEC", NULL, NVKM_ENGINE_MSPDEC },
- { 0x15, "CE0", NULL, NVKM_ENGINE_CE0 },
- { 0x16, "CE1", NULL, NVKM_ENGINE_CE1 },
- { 0x17, "PMU" },
- { 0x19, "MSENC", NULL, NVKM_ENGINE_MSENC },
- { 0x1b, "CE2", NULL, NVKM_ENGINE_CE2 },
- {}
-};
-
-static const struct nvkm_enum
-gk104_fifo_fault_reason[] = {
- { 0x00, "PDE" },
- { 0x01, "PDE_SIZE" },
- { 0x02, "PTE" },
- { 0x03, "VA_LIMIT_VIOLATION" },
- { 0x04, "UNBOUND_INST_BLOCK" },
- { 0x05, "PRIV_VIOLATION" },
- { 0x06, "RO_VIOLATION" },
- { 0x07, "WO_VIOLATION" },
- { 0x08, "PITCH_MASK_VIOLATION" },
- { 0x09, "WORK_CREATION" },
- { 0x0a, "UNSUPPORTED_APERTURE" },
- { 0x0b, "COMPRESSION_FAILURE" },
- { 0x0c, "UNSUPPORTED_KIND" },
- { 0x0d, "REGION_VIOLATION" },
- { 0x0e, "BOTH_PTES_VALID" },
- { 0x0f, "INFO_TYPE_POISONED" },
- {}
-};
-
-static const struct nvkm_enum
-gk104_fifo_fault_hubclient[] = {
- { 0x00, "VIP" },
- { 0x01, "CE0" },
- { 0x02, "CE1" },
- { 0x03, "DNISO" },
- { 0x04, "FE" },
- { 0x05, "FECS" },
- { 0x06, "HOST" },
- { 0x07, "HOST_CPU" },
- { 0x08, "HOST_CPU_NB" },
- { 0x09, "ISO" },
- { 0x0a, "MMU" },
- { 0x0b, "MSPDEC" },
- { 0x0c, "MSPPP" },
- { 0x0d, "MSVLD" },
- { 0x0e, "NISO" },
- { 0x0f, "P2P" },
- { 0x10, "PD" },
- { 0x11, "PERF" },
- { 0x12, "PMU" },
- { 0x13, "RASTERTWOD" },
- { 0x14, "SCC" },
- { 0x15, "SCC_NB" },
- { 0x16, "SEC" },
- { 0x17, "SSYNC" },
- { 0x18, "GR_CE" },
- { 0x19, "CE2" },
- { 0x1a, "XV" },
- { 0x1b, "MMU_NB" },
- { 0x1c, "MSENC" },
- { 0x1d, "DFALCON" },
- { 0x1e, "SKED" },
- { 0x1f, "AFALCON" },
- {}
-};
-
-static const struct nvkm_enum
-gk104_fifo_fault_gpcclient[] = {
- { 0x00, "L1_0" }, { 0x01, "T1_0" }, { 0x02, "PE_0" },
- { 0x03, "L1_1" }, { 0x04, "T1_1" }, { 0x05, "PE_1" },
- { 0x06, "L1_2" }, { 0x07, "T1_2" }, { 0x08, "PE_2" },
- { 0x09, "L1_3" }, { 0x0a, "T1_3" }, { 0x0b, "PE_3" },
- { 0x0c, "RAST" },
- { 0x0d, "GCC" },
- { 0x0e, "GPCCS" },
- { 0x0f, "PROP_0" },
- { 0x10, "PROP_1" },
- { 0x11, "PROP_2" },
- { 0x12, "PROP_3" },
- { 0x13, "L1_4" }, { 0x14, "T1_4" }, { 0x15, "PE_4" },
- { 0x16, "L1_5" }, { 0x17, "T1_5" }, { 0x18, "PE_5" },
- { 0x19, "L1_6" }, { 0x1a, "T1_6" }, { 0x1b, "PE_6" },
- { 0x1c, "L1_7" }, { 0x1d, "T1_7" }, { 0x1e, "PE_7" },
- { 0x1f, "GPM" },
- { 0x20, "LTP_UTLB_0" },
- { 0x21, "LTP_UTLB_1" },
- { 0x22, "LTP_UTLB_2" },
- { 0x23, "LTP_UTLB_3" },
- { 0x24, "GPC_RGG_UTLB" },
- {}
-};
-
static void
gk104_fifo_intr_fault(struct gk104_fifo *fifo, int unit)
{
@@ -390,14 +300,14 @@
struct nvkm_engine *engine = NULL;
struct nvkm_fifo_chan *chan;
unsigned long flags;
- char gpcid[8] = "";
+ char gpcid[8] = "", en[16] = "";
- er = nvkm_enum_find(gk104_fifo_fault_reason, reason);
- eu = nvkm_enum_find(gk104_fifo_fault_engine, unit);
+ er = nvkm_enum_find(fifo->func->fault.reason, reason);
+ eu = nvkm_enum_find(fifo->func->fault.engine, unit);
if (hub) {
- ec = nvkm_enum_find(gk104_fifo_fault_hubclient, client);
+ ec = nvkm_enum_find(fifo->func->fault.hubclient, client);
} else {
- ec = nvkm_enum_find(gk104_fifo_fault_gpcclient, client);
+ ec = nvkm_enum_find(fifo->func->fault.gpcclient, client);
snprintf(gpcid, sizeof(gpcid), "GPC%d/", gpc);
}
@@ -418,13 +328,27 @@
}
}
+ if (eu == NULL) {
+ enum nvkm_devidx engidx = nvkm_top_fault(device->top, unit);
+ if (engidx < NVKM_SUBDEV_NR) {
+ const char *src = nvkm_subdev_name[engidx];
+ char *dst = en;
+ do {
+ *dst++ = toupper(*src++);
+ } while(*src);
+ engine = nvkm_device_engine(device, engidx);
+ }
+ } else {
+ snprintf(en, sizeof(en), "%s", eu->name);
+ }
+
chan = nvkm_fifo_chan_inst(&fifo->base, (u64)inst << 12, &flags);
nvkm_error(subdev,
"%s fault at %010llx engine %02x [%s] client %02x [%s%s] "
"reason %02x [%s] on channel %d [%010llx %s]\n",
write ? "write" : "read", (u64)vahi << 32 | valo,
- unit, eu ? eu->name : "", client, gpcid, ec ? ec->name : "",
+ unit, en, client, gpcid, ec ? ec->name : "",
reason, er ? er->name : "", chan ? chan->chid : -1,
(u64)inst << 12,
chan ? chan->object.client->name : "unknown");
@@ -557,7 +481,7 @@
nvkm_fifo_uevent(&fifo->base);
}
-void
+static void
gk104_fifo_intr(struct nvkm_fifo *base)
{
struct gk104_fifo *fifo = gk104_fifo(base);
@@ -649,7 +573,7 @@
}
}
-void
+static void
gk104_fifo_fini(struct nvkm_fifo *base)
{
struct gk104_fifo *fifo = gk104_fifo(base);
@@ -659,13 +583,15 @@
nvkm_mask(device, 0x002140, 0x10000000, 0x10000000);
}
-int
+static int
gk104_fifo_oneinit(struct nvkm_fifo *base)
{
struct gk104_fifo *fifo = gk104_fifo(base);
struct nvkm_subdev *subdev = &fifo->base.engine.subdev;
struct nvkm_device *device = subdev->device;
- int ret, i;
+ struct nvkm_top *top = device->top;
+ int engn, runl, pbid, ret, i, j;
+ enum nvkm_devidx engidx;
u32 *map;
/* Determine number of PBDMAs by checking valid enable bits. */
@@ -680,86 +606,26 @@
for (i = 0; i < fifo->pbdma_nr; i++)
map[i] = nvkm_rd32(device, 0x002390 + (i * 0x04));
- /* Read device topology from HW. */
- for (i = 0; i < 64; i++) {
- int type = -1, pbid = -1, engidx = -1;
- int engn = -1, runl = -1, intr = -1, mcen = -1;
- int fault = -1, j;
- u32 data, addr = 0;
-
- do {
- data = nvkm_rd32(device, 0x022700 + (i * 0x04));
- nvkm_trace(subdev, "%02x: %08x\n", i, data);
- switch (data & 0x00000003) {
- case 0x00000000: /* NOT_VALID */
- continue;
- case 0x00000001: /* DATA */
- addr = (data & 0x00fff000);
- fault = (data & 0x000000f8) >> 3;
- break;
- case 0x00000002: /* ENUM */
- if (data & 0x00000020)
- engn = (data & 0x3c000000) >> 26;
- if (data & 0x00000010)
- runl = (data & 0x01e00000) >> 21;
- if (data & 0x00000008)
- intr = (data & 0x000f8000) >> 15;
- if (data & 0x00000004)
- mcen = (data & 0x00003e00) >> 9;
- break;
- case 0x00000003: /* ENGINE_TYPE */
- type = (data & 0x7ffffffc) >> 2;
- break;
- }
- } while ((data & 0x80000000) && ++i < 64);
-
- if (!data)
- continue;
-
+ /* Determine runlist configuration from topology device info. */
+ i = 0;
+ while ((int)(engidx = nvkm_top_engine(top, i++, &runl, &engn)) >= 0) {
/* Determine which PBDMA handles requests for this engine. */
- for (j = 0; runl >= 0 && j < fifo->pbdma_nr; j++) {
+ for (j = 0, pbid = -1; j < fifo->pbdma_nr; j++) {
if (map[j] & (1 << runl)) {
pbid = j;
break;
}
}
- /* Translate engine type to NVKM engine identifier. */
- switch (type) {
- case 0x00000000: engidx = NVKM_ENGINE_GR; break;
- case 0x00000001: engidx = NVKM_ENGINE_CE0; break;
- case 0x00000002: engidx = NVKM_ENGINE_CE1; break;
- case 0x00000003: engidx = NVKM_ENGINE_CE2; break;
- case 0x00000008: engidx = NVKM_ENGINE_MSPDEC; break;
- case 0x00000009: engidx = NVKM_ENGINE_MSPPP; break;
- case 0x0000000a: engidx = NVKM_ENGINE_MSVLD; break;
- case 0x0000000b: engidx = NVKM_ENGINE_MSENC; break;
- case 0x0000000c: engidx = NVKM_ENGINE_VIC; break;
- case 0x0000000d: engidx = NVKM_ENGINE_SEC; break;
- case 0x0000000e: engidx = NVKM_ENGINE_NVENC0; break;
- case 0x0000000f: engidx = NVKM_ENGINE_NVENC1; break;
- case 0x00000010: engidx = NVKM_ENGINE_NVDEC; break;
- break;
- default:
- break;
- }
+ nvkm_debug(subdev, "engine %2d: runlist %2d pbdma %2d\n",
+ engn, runl, pbid);
- nvkm_debug(subdev, "%02x (%8s): engine %2d runlist %2d "
- "pbdma %2d intr %2d reset %2d "
- "fault %2d addr %06x\n", type,
- engidx < 0 ? NULL : nvkm_subdev_name[engidx],
- engn, runl, pbid, intr, mcen, fault, addr);
-
- /* Mark the engine as supported if everything checks out. */
- if (engn >= 0 && runl >= 0) {
- fifo->engine[engn].engine = engidx < 0 ? NULL :
- nvkm_device_engine(device, engidx);
- fifo->engine[engn].runl = runl;
- fifo->engine[engn].pbid = pbid;
- fifo->engine_nr = max(fifo->engine_nr, engn + 1);
- fifo->runlist[runl].engm |= 1 << engn;
- fifo->runlist_nr = max(fifo->runlist_nr, runl + 1);
- }
+ fifo->engine[engn].engine = nvkm_device_engine(device, engidx);
+ fifo->engine[engn].runl = runl;
+ fifo->engine[engn].pbid = pbid;
+ fifo->engine_nr = max(fifo->engine_nr, engn + 1);
+ fifo->runlist[runl].engm |= 1 << engn;
+ fifo->runlist_nr = max(fifo->runlist_nr, runl + 1);
}
kfree(map);
@@ -796,7 +662,7 @@
return 0;
}
-void
+static void
gk104_fifo_init(struct nvkm_fifo *base)
{
struct gk104_fifo *fifo = gk104_fifo(base);
@@ -825,7 +691,7 @@
nvkm_wr32(device, 0x002140, 0x7fffffff);
}
-void *
+static void *
gk104_fifo_dtor(struct nvkm_fifo *base)
{
struct gk104_fifo *fifo = gk104_fifo(base);
@@ -842,22 +708,8 @@
return fifo;
}
-int
-gk104_fifo_new_(const struct nvkm_fifo_func *func, struct nvkm_device *device,
- int index, int nr, struct nvkm_fifo **pfifo)
-{
- struct gk104_fifo *fifo;
-
- if (!(fifo = kzalloc(sizeof(*fifo), GFP_KERNEL)))
- return -ENOMEM;
- INIT_WORK(&fifo->recover.work, gk104_fifo_recover_work);
- *pfifo = &fifo->base;
-
- return nvkm_fifo_ctor(func, device, index, nr, &fifo->base);
-}
-
static const struct nvkm_fifo_func
-gk104_fifo = {
+gk104_fifo_ = {
.dtor = gk104_fifo_dtor,
.oneinit = gk104_fifo_oneinit,
.init = gk104_fifo_init,
@@ -865,6 +717,145 @@
.intr = gk104_fifo_intr,
.uevent_init = gk104_fifo_uevent_init,
.uevent_fini = gk104_fifo_uevent_fini,
+ .class_get = gk104_fifo_class_get,
+};
+
+int
+gk104_fifo_new_(const struct gk104_fifo_func *func, struct nvkm_device *device,
+ int index, int nr, struct nvkm_fifo **pfifo)
+{
+ struct gk104_fifo *fifo;
+
+ if (!(fifo = kzalloc(sizeof(*fifo), GFP_KERNEL)))
+ return -ENOMEM;
+ fifo->func = func;
+ INIT_WORK(&fifo->recover.work, gk104_fifo_recover_work);
+ *pfifo = &fifo->base;
+
+ return nvkm_fifo_ctor(&gk104_fifo_, device, index, nr, &fifo->base);
+}
+
+const struct nvkm_enum
+gk104_fifo_fault_engine[] = {
+ { 0x00, "GR", NULL, NVKM_ENGINE_GR },
+ { 0x01, "DISPLAY" },
+ { 0x02, "CAPTURE" },
+ { 0x03, "IFB", NULL, NVKM_ENGINE_IFB },
+ { 0x04, "BAR1", NULL, NVKM_SUBDEV_BAR },
+ { 0x05, "BAR2", NULL, NVKM_SUBDEV_INSTMEM },
+ { 0x06, "SCHED" },
+ { 0x07, "HOST0" },
+ { 0x08, "HOST1" },
+ { 0x09, "HOST2" },
+ { 0x0a, "HOST3" },
+ { 0x0b, "HOST4" },
+ { 0x0c, "HOST5" },
+ { 0x0d, "HOST6" },
+ { 0x0e, "HOST7" },
+ { 0x0f, "HOSTSR" },
+ { 0x10, "MSVLD", NULL, NVKM_ENGINE_MSVLD },
+ { 0x11, "MSPPP", NULL, NVKM_ENGINE_MSPPP },
+ { 0x13, "PERF" },
+ { 0x14, "MSPDEC", NULL, NVKM_ENGINE_MSPDEC },
+ { 0x15, "CE0", NULL, NVKM_ENGINE_CE0 },
+ { 0x16, "CE1", NULL, NVKM_ENGINE_CE1 },
+ { 0x17, "PMU" },
+ { 0x18, "PTP" },
+ { 0x19, "MSENC", NULL, NVKM_ENGINE_MSENC },
+ { 0x1b, "CE2", NULL, NVKM_ENGINE_CE2 },
+ {}
+};
+
+const struct nvkm_enum
+gk104_fifo_fault_reason[] = {
+ { 0x00, "PDE" },
+ { 0x01, "PDE_SIZE" },
+ { 0x02, "PTE" },
+ { 0x03, "VA_LIMIT_VIOLATION" },
+ { 0x04, "UNBOUND_INST_BLOCK" },
+ { 0x05, "PRIV_VIOLATION" },
+ { 0x06, "RO_VIOLATION" },
+ { 0x07, "WO_VIOLATION" },
+ { 0x08, "PITCH_MASK_VIOLATION" },
+ { 0x09, "WORK_CREATION" },
+ { 0x0a, "UNSUPPORTED_APERTURE" },
+ { 0x0b, "COMPRESSION_FAILURE" },
+ { 0x0c, "UNSUPPORTED_KIND" },
+ { 0x0d, "REGION_VIOLATION" },
+ { 0x0e, "BOTH_PTES_VALID" },
+ { 0x0f, "INFO_TYPE_POISONED" },
+ {}
+};
+
+const struct nvkm_enum
+gk104_fifo_fault_hubclient[] = {
+ { 0x00, "VIP" },
+ { 0x01, "CE0" },
+ { 0x02, "CE1" },
+ { 0x03, "DNISO" },
+ { 0x04, "FE" },
+ { 0x05, "FECS" },
+ { 0x06, "HOST" },
+ { 0x07, "HOST_CPU" },
+ { 0x08, "HOST_CPU_NB" },
+ { 0x09, "ISO" },
+ { 0x0a, "MMU" },
+ { 0x0b, "MSPDEC" },
+ { 0x0c, "MSPPP" },
+ { 0x0d, "MSVLD" },
+ { 0x0e, "NISO" },
+ { 0x0f, "P2P" },
+ { 0x10, "PD" },
+ { 0x11, "PERF" },
+ { 0x12, "PMU" },
+ { 0x13, "RASTERTWOD" },
+ { 0x14, "SCC" },
+ { 0x15, "SCC_NB" },
+ { 0x16, "SEC" },
+ { 0x17, "SSYNC" },
+ { 0x18, "GR_CE" },
+ { 0x19, "CE2" },
+ { 0x1a, "XV" },
+ { 0x1b, "MMU_NB" },
+ { 0x1c, "MSENC" },
+ { 0x1d, "DFALCON" },
+ { 0x1e, "SKED" },
+ { 0x1f, "AFALCON" },
+ {}
+};
+
+const struct nvkm_enum
+gk104_fifo_fault_gpcclient[] = {
+ { 0x00, "L1_0" }, { 0x01, "T1_0" }, { 0x02, "PE_0" },
+ { 0x03, "L1_1" }, { 0x04, "T1_1" }, { 0x05, "PE_1" },
+ { 0x06, "L1_2" }, { 0x07, "T1_2" }, { 0x08, "PE_2" },
+ { 0x09, "L1_3" }, { 0x0a, "T1_3" }, { 0x0b, "PE_3" },
+ { 0x0c, "RAST" },
+ { 0x0d, "GCC" },
+ { 0x0e, "GPCCS" },
+ { 0x0f, "PROP_0" },
+ { 0x10, "PROP_1" },
+ { 0x11, "PROP_2" },
+ { 0x12, "PROP_3" },
+ { 0x13, "L1_4" }, { 0x14, "T1_4" }, { 0x15, "PE_4" },
+ { 0x16, "L1_5" }, { 0x17, "T1_5" }, { 0x18, "PE_5" },
+ { 0x19, "L1_6" }, { 0x1a, "T1_6" }, { 0x1b, "PE_6" },
+ { 0x1c, "L1_7" }, { 0x1d, "T1_7" }, { 0x1e, "PE_7" },
+ { 0x1f, "GPM" },
+ { 0x20, "LTP_UTLB_0" },
+ { 0x21, "LTP_UTLB_1" },
+ { 0x22, "LTP_UTLB_2" },
+ { 0x23, "LTP_UTLB_3" },
+ { 0x24, "GPC_RGG_UTLB" },
+ {}
+};
+
+static const struct gk104_fifo_func
+gk104_fifo = {
+ .fault.engine = gk104_fifo_fault_engine,
+ .fault.reason = gk104_fifo_fault_reason,
+ .fault.hubclient = gk104_fifo_fault_hubclient,
+ .fault.gpcclient = gk104_fifo_fault_gpcclient,
.chan = {
&gk104_fifo_gpfifo_oclass,
NULL
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.h b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.h
index 9e5d00b..679f3ec3 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.h
@@ -3,10 +3,12 @@
#define gk104_fifo(p) container_of((p), struct gk104_fifo, base)
#include "priv.h"
+#include <core/enum.h>
#include <subdev/mmu.h>
struct gk104_fifo_chan;
struct gk104_fifo {
+ const struct gk104_fifo_func *func;
struct nvkm_fifo base;
struct {
@@ -39,15 +41,19 @@
} user;
};
-int gk104_fifo_new_(const struct nvkm_fifo_func *, struct nvkm_device *,
+struct gk104_fifo_func {
+ struct {
+ const struct nvkm_enum *engine;
+ const struct nvkm_enum *reason;
+ const struct nvkm_enum *hubclient;
+ const struct nvkm_enum *gpcclient;
+ } fault;
+
+ const struct nvkm_fifo_chan_oclass *chan[];
+};
+
+int gk104_fifo_new_(const struct gk104_fifo_func *, struct nvkm_device *,
int index, int nr, struct nvkm_fifo **);
-void *gk104_fifo_dtor(struct nvkm_fifo *);
-int gk104_fifo_oneinit(struct nvkm_fifo *);
-void gk104_fifo_init(struct nvkm_fifo *);
-void gk104_fifo_fini(struct nvkm_fifo *);
-void gk104_fifo_intr(struct nvkm_fifo *);
-void gk104_fifo_uevent_init(struct nvkm_fifo *);
-void gk104_fifo_uevent_fini(struct nvkm_fifo *);
void gk104_fifo_runlist_insert(struct gk104_fifo *, struct gk104_fifo_chan *);
void gk104_fifo_runlist_remove(struct gk104_fifo *, struct gk104_fifo_chan *);
void gk104_fifo_runlist_commit(struct gk104_fifo *, int runl);
@@ -70,4 +76,11 @@
return 0;
}
}
+
+extern const struct nvkm_enum gk104_fifo_fault_engine[];
+extern const struct nvkm_enum gk104_fifo_fault_reason[];
+extern const struct nvkm_enum gk104_fifo_fault_hubclient[];
+extern const struct nvkm_enum gk104_fifo_fault_gpcclient[];
+
+extern const struct nvkm_enum gm107_fifo_fault_engine[];
#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk110.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk110.c
index 41307fc..b2f8ab7 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk110.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk110.c
@@ -24,15 +24,12 @@
#include "gk104.h"
#include "changk104.h"
-static const struct nvkm_fifo_func
+static const struct gk104_fifo_func
gk110_fifo = {
- .dtor = gk104_fifo_dtor,
- .oneinit = gk104_fifo_oneinit,
- .init = gk104_fifo_init,
- .fini = gk104_fifo_fini,
- .intr = gk104_fifo_intr,
- .uevent_init = gk104_fifo_uevent_init,
- .uevent_fini = gk104_fifo_uevent_fini,
+ .fault.engine = gk104_fifo_fault_engine,
+ .fault.reason = gk104_fifo_fault_reason,
+ .fault.hubclient = gk104_fifo_fault_hubclient,
+ .fault.gpcclient = gk104_fifo_fault_gpcclient,
.chan = {
&gk110_fifo_gpfifo_oclass,
NULL
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk208.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk208.c
index ce01c1a7..160617d 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk208.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk208.c
@@ -24,15 +24,12 @@
#include "gk104.h"
#include "changk104.h"
-static const struct nvkm_fifo_func
+static const struct gk104_fifo_func
gk208_fifo = {
- .dtor = gk104_fifo_dtor,
- .oneinit = gk104_fifo_oneinit,
- .init = gk104_fifo_init,
- .fini = gk104_fifo_fini,
- .intr = gk104_fifo_intr,
- .uevent_init = gk104_fifo_uevent_init,
- .uevent_fini = gk104_fifo_uevent_fini,
+ .fault.engine = gk104_fifo_fault_engine,
+ .fault.reason = gk104_fifo_fault_reason,
+ .fault.hubclient = gk104_fifo_fault_hubclient,
+ .fault.gpcclient = gk104_fifo_fault_gpcclient,
.chan = {
&gk104_fifo_gpfifo_oclass,
NULL
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk20a.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk20a.c
index b47fe98..be9f5c1 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk20a.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk20a.c
@@ -22,15 +22,12 @@
#include "gk104.h"
#include "changk104.h"
-static const struct nvkm_fifo_func
+static const struct gk104_fifo_func
gk20a_fifo = {
- .dtor = gk104_fifo_dtor,
- .oneinit = gk104_fifo_oneinit,
- .init = gk104_fifo_init,
- .fini = gk104_fifo_fini,
- .intr = gk104_fifo_intr,
- .uevent_init = gk104_fifo_uevent_init,
- .uevent_fini = gk104_fifo_uevent_fini,
+ .fault.engine = gk104_fifo_fault_engine,
+ .fault.reason = gk104_fifo_fault_reason,
+ .fault.hubclient = gk104_fifo_fault_hubclient,
+ .fault.gpcclient = gk104_fifo_fault_gpcclient,
.chan = {
&gk104_fifo_gpfifo_oclass,
NULL
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm107.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm107.c
index 6d59d65..bd1ff87 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm107.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm107.c
@@ -24,15 +24,35 @@
#include "gk104.h"
#include "changk104.h"
-static const struct nvkm_fifo_func
+const struct nvkm_enum
+gm107_fifo_fault_engine[] = {
+ { 0x01, "DISPLAY" },
+ { 0x02, "CAPTURE" },
+ { 0x03, "IFB", NULL, NVKM_ENGINE_IFB },
+ { 0x04, "BAR1", NULL, NVKM_SUBDEV_BAR },
+ { 0x05, "BAR2", NULL, NVKM_SUBDEV_INSTMEM },
+ { 0x06, "SCHED" },
+ { 0x07, "HOST0" },
+ { 0x08, "HOST1" },
+ { 0x09, "HOST2" },
+ { 0x0a, "HOST3" },
+ { 0x0b, "HOST4" },
+ { 0x0c, "HOST5" },
+ { 0x0d, "HOST6" },
+ { 0x0e, "HOST7" },
+ { 0x0f, "HOSTSR" },
+ { 0x13, "PERF" },
+ { 0x17, "PMU" },
+ { 0x18, "PTP" },
+ {}
+};
+
+static const struct gk104_fifo_func
gm107_fifo = {
- .dtor = gk104_fifo_dtor,
- .oneinit = gk104_fifo_oneinit,
- .init = gk104_fifo_init,
- .fini = gk104_fifo_fini,
- .intr = gk104_fifo_intr,
- .uevent_init = gk104_fifo_uevent_init,
- .uevent_fini = gk104_fifo_uevent_fini,
+ .fault.engine = gm107_fifo_fault_engine,
+ .fault.reason = gk104_fifo_fault_reason,
+ .fault.hubclient = gk104_fifo_fault_hubclient,
+ .fault.gpcclient = gk104_fifo_fault_gpcclient,
.chan = {
&gk110_fifo_gpfifo_oclass,
NULL
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm200.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm200.c
index 4bdd430..b069f78 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm200.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm200.c
@@ -24,15 +24,12 @@
#include "gk104.h"
#include "changk104.h"
-static const struct nvkm_fifo_func
+static const struct gk104_fifo_func
gm200_fifo = {
- .dtor = gk104_fifo_dtor,
- .oneinit = gk104_fifo_oneinit,
- .init = gk104_fifo_init,
- .fini = gk104_fifo_fini,
- .intr = gk104_fifo_intr,
- .uevent_init = gk104_fifo_uevent_init,
- .uevent_fini = gk104_fifo_uevent_fini,
+ .fault.engine = gm107_fifo_fault_engine,
+ .fault.reason = gk104_fifo_fault_reason,
+ .fault.hubclient = gk104_fifo_fault_hubclient,
+ .fault.gpcclient = gk104_fifo_fault_gpcclient,
.chan = {
&gm200_fifo_gpfifo_oclass,
NULL
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm20b.c b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm20b.c
index 4c91d4a..2ed87c2 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm20b.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm20b.c
@@ -22,15 +22,12 @@
#include "gk104.h"
#include "changk104.h"
-static const struct nvkm_fifo_func
+static const struct gk104_fifo_func
gm20b_fifo = {
- .dtor = gk104_fifo_dtor,
- .oneinit = gk104_fifo_oneinit,
- .init = gk104_fifo_init,
- .fini = gk104_fifo_fini,
- .intr = gk104_fifo_intr,
- .uevent_init = gk104_fifo_uevent_init,
- .uevent_fini = gk104_fifo_uevent_fini,
+ .fault.engine = gm107_fifo_fault_engine,
+ .fault.reason = gk104_fifo_fault_reason,
+ .fault.hubclient = gk104_fifo_fault_hubclient,
+ .fault.gpcclient = gk104_fifo_fault_gpcclient,
.chan = {
&gm200_fifo_gpfifo_oclass,
NULL
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/priv.h b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/priv.h
index cb1432e..f6dfb37 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/fifo/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/fifo/priv.h
@@ -7,6 +7,7 @@
int index, int nr, struct nvkm_fifo *);
void nvkm_fifo_uevent(struct nvkm_fifo *);
+struct nvkm_fifo_chan_oclass;
struct nvkm_fifo_func {
void *(*dtor)(struct nvkm_fifo *);
int (*oneinit)(struct nvkm_fifo *);
@@ -17,6 +18,8 @@
void (*start)(struct nvkm_fifo *, unsigned long *);
void (*uevent_init)(struct nvkm_fifo *);
void (*uevent_fini)(struct nvkm_fifo *);
+ int (*class_get)(struct nvkm_fifo *, int index,
+ const struct nvkm_fifo_chan_oclass **);
const struct nvkm_fifo_chan_oclass *chan[];
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/base.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/base.c
index 090765f..467065d 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/base.c
@@ -128,9 +128,8 @@
int
nvkm_gr_ctor(const struct nvkm_gr_func *func, struct nvkm_device *device,
- int index, u32 pmc_enable, bool enable, struct nvkm_gr *gr)
+ int index, bool enable, struct nvkm_gr *gr)
{
gr->func = func;
- return nvkm_engine_ctor(&nvkm_gr, device, index, pmc_enable,
- enable, &gr->engine);
+ return nvkm_engine_ctor(&nvkm_gr, device, index, enable, &gr->engine);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf100.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf100.c
index 56f392d..b02d8f5 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf100.c
@@ -1181,20 +1181,20 @@
/* GPC_BROADCAST */
nvkm_wr32(device, 0x418bb8, (gr->tpc_total << 8) |
- gr->magic_not_rop_nr);
+ gr->screen_tile_row_offset);
for (i = 0; i < 6; i++)
nvkm_wr32(device, 0x418b08 + (i * 4), data[i]);
/* GPC_BROADCAST.TP_BROADCAST */
nvkm_wr32(device, 0x419bd0, (gr->tpc_total << 8) |
- gr->magic_not_rop_nr | data2[0]);
+ gr->screen_tile_row_offset | data2[0]);
nvkm_wr32(device, 0x419be4, data2[1]);
for (i = 0; i < 6; i++)
nvkm_wr32(device, 0x419b00 + (i * 4), data[i]);
/* UNK78xx */
nvkm_wr32(device, 0x4078bc, (gr->tpc_total << 8) |
- gr->magic_not_rop_nr);
+ gr->screen_tile_row_offset);
for (i = 0; i < 6; i++)
nvkm_wr32(device, 0x40780c + (i * 4), data[i]);
}
@@ -1238,6 +1238,7 @@
{
struct nvkm_device *device = gr->base.engine.subdev.device;
const struct gf100_grctx_func *grctx = gr->func->grctx;
+ u32 idle_timeout;
nvkm_mc_unk260(device->mc, 0);
@@ -1247,7 +1248,7 @@
gf100_gr_mmio(gr, grctx->tpc);
gf100_gr_mmio(gr, grctx->ppc);
- nvkm_wr32(device, 0x404154, 0x00000000);
+ idle_timeout = nvkm_mask(device, 0x404154, 0xffffffff, 0x00000000);
grctx->bundle(info);
grctx->pagepool(info);
@@ -1261,7 +1262,7 @@
gf100_grctx_generate_r406800(gr);
gf100_gr_icmd(gr, grctx->icmd);
- nvkm_wr32(device, 0x404154, 0x00000400);
+ nvkm_wr32(device, 0x404154, idle_timeout);
gf100_gr_mthd(gr, grctx->mthd);
nvkm_mc_unk260(device->mc, 1);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf100.h b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf100.h
index 3c86739..ac895ed 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf100.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf100.h
@@ -81,8 +81,6 @@
void gk104_grctx_generate_pagepool(struct gf100_grctx *);
void gk104_grctx_generate_unkn(struct gf100_gr *);
void gk104_grctx_generate_r418bb8(struct gf100_gr *);
-void gk104_grctx_generate_rop_active_fbps(struct gf100_gr *);
-
void gm107_grctx_generate_bundle(struct gf100_grctx *);
void gm107_grctx_generate_pagepool(struct gf100_grctx *);
@@ -98,7 +96,6 @@
void gm107_grctx_generate_attrib(struct gf100_grctx *);
extern const struct gf100_grctx_func gm200_grctx;
-void gm200_grctx_generate_main(struct gf100_gr *, struct gf100_grctx *);
void gm200_grctx_generate_tpcid(struct gf100_gr *);
void gm200_grctx_generate_405b60(struct gf100_gr *);
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf117.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf117.c
index 74de7a9..f521de11 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf117.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf117.c
@@ -223,6 +223,7 @@
{
struct nvkm_device *device = gr->base.engine.subdev.device;
const struct gf100_grctx_func *grctx = gr->func->grctx;
+ u32 idle_timeout;
int i;
nvkm_mc_unk260(device->mc, 0);
@@ -233,7 +234,7 @@
gf100_gr_mmio(gr, grctx->tpc);
gf100_gr_mmio(gr, grctx->ppc);
- nvkm_wr32(device, 0x404154, 0x00000000);
+ idle_timeout = nvkm_mask(device, 0x404154, 0xffffffff, 0x00000000);
grctx->bundle(info);
grctx->pagepool(info);
@@ -250,7 +251,7 @@
nvkm_wr32(device, 0x4064d0 + (i * 0x04), 0x00000000);
gf100_gr_icmd(gr, grctx->icmd);
- nvkm_wr32(device, 0x404154, 0x00000400);
+ nvkm_wr32(device, 0x404154, idle_timeout);
gf100_gr_mthd(gr, grctx->mthd);
nvkm_mc_unk260(device->mc, 1);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk104.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk104.c
index a843e36..9ba3377 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk104.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk104.c
@@ -924,38 +924,30 @@
/* GPC_BROADCAST */
nvkm_wr32(device, 0x418bb8, (gr->tpc_total << 8) |
- gr->magic_not_rop_nr);
+ gr->screen_tile_row_offset);
for (i = 0; i < 6; i++)
nvkm_wr32(device, 0x418b08 + (i * 4), data[i]);
/* GPC_BROADCAST.TP_BROADCAST */
nvkm_wr32(device, 0x41bfd0, (gr->tpc_total << 8) |
- gr->magic_not_rop_nr | data2[0]);
+ gr->screen_tile_row_offset | data2[0]);
nvkm_wr32(device, 0x41bfe4, data2[1]);
for (i = 0; i < 6; i++)
nvkm_wr32(device, 0x41bf00 + (i * 4), data[i]);
/* UNK78xx */
nvkm_wr32(device, 0x4078bc, (gr->tpc_total << 8) |
- gr->magic_not_rop_nr);
+ gr->screen_tile_row_offset);
for (i = 0; i < 6; i++)
nvkm_wr32(device, 0x40780c + (i * 4), data[i]);
}
void
-gk104_grctx_generate_rop_active_fbps(struct gf100_gr *gr)
-{
- struct nvkm_device *device = gr->base.engine.subdev.device;
- const u32 fbp_count = nvkm_rd32(device, 0x120074);
- nvkm_mask(device, 0x408850, 0x0000000f, fbp_count); /* zrop */
- nvkm_mask(device, 0x408958, 0x0000000f, fbp_count); /* crop */
-}
-
-void
gk104_grctx_generate_main(struct gf100_gr *gr, struct gf100_grctx *info)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
const struct gf100_grctx_func *grctx = gr->func->grctx;
+ u32 idle_timeout;
int i;
nvkm_mc_unk260(device->mc, 0);
@@ -966,7 +958,7 @@
gf100_gr_mmio(gr, grctx->tpc);
gf100_gr_mmio(gr, grctx->ppc);
- nvkm_wr32(device, 0x404154, 0x00000000);
+ idle_timeout = nvkm_mask(device, 0x404154, 0xffffffff, 0x00000000);
grctx->bundle(info);
grctx->pagepool(info);
@@ -982,11 +974,10 @@
nvkm_wr32(device, 0x4064d0 + (i * 0x04), 0x00000000);
nvkm_wr32(device, 0x405b00, (gr->tpc_total << 8) | gr->gpc_nr);
- gk104_grctx_generate_rop_active_fbps(gr);
nvkm_mask(device, 0x419f78, 0x00000001, 0x00000000);
gf100_gr_icmd(gr, grctx->icmd);
- nvkm_wr32(device, 0x404154, 0x00000400);
+ nvkm_wr32(device, 0x404154, idle_timeout);
gf100_gr_mthd(gr, grctx->mthd);
nvkm_mc_unk260(device->mc, 1);
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk20a.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk20a.c
index ad0a6cf..da7c35a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk20a.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk20a.c
@@ -29,15 +29,14 @@
{
struct nvkm_device *device = gr->base.engine.subdev.device;
const struct gf100_grctx_func *grctx = gr->func->grctx;
- int idle_timeout_save;
+ u32 idle_timeout;
int i;
gf100_gr_mmio(gr, gr->fuc_sw_ctx);
gf100_gr_wait_idle(gr);
- idle_timeout_save = nvkm_rd32(device, 0x404154);
- nvkm_wr32(device, 0x404154, 0x00000000);
+ idle_timeout = nvkm_mask(device, 0x404154, 0xffffffff, 0x00000000);
grctx->attrib(info);
@@ -53,13 +52,11 @@
nvkm_wr32(device, 0x405b00, (gr->tpc_total << 8) | gr->gpc_nr);
- gk104_grctx_generate_rop_active_fbps(gr);
-
nvkm_mask(device, 0x5044b0, 0x08000000, 0x08000000);
gf100_gr_wait_idle(gr);
- nvkm_wr32(device, 0x404154, idle_timeout_save);
+ nvkm_wr32(device, 0x404154, idle_timeout);
gf100_gr_wait_idle(gr);
gf100_gr_mthd(gr, gr->fuc_method);
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm107.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm107.c
index 95f59e3..6d3c501 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm107.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm107.c
@@ -920,13 +920,15 @@
const u32 bs = attrib * gr->ppc_tpc_nr[gpc][ppc];
const u32 u = 0x418ea0 + (n * 0x04);
const u32 o = PPC_UNIT(gpc, ppc, 0);
+ if (!(gr->ppc_mask[gpc] & (1 << ppc)))
+ continue;
mmio_wr32(info, o + 0xc0, bs);
mmio_wr32(info, o + 0xf4, bo);
bo += grctx->attrib_nr_max * gr->ppc_tpc_nr[gpc][ppc];
mmio_wr32(info, o + 0xe4, as);
mmio_wr32(info, o + 0xf8, ao);
ao += grctx->alpha_nr_max * gr->ppc_tpc_nr[gpc][ppc];
- mmio_wr32(info, u, ((bs / 3 /*XXX*/) << 16) | bs);
+ mmio_wr32(info, u, ((bs / 3) << 16) | bs);
}
}
}
@@ -957,6 +959,7 @@
{
struct nvkm_device *device = gr->base.engine.subdev.device;
const struct gf100_grctx_func *grctx = gr->func->grctx;
+ u32 idle_timeout;
int i;
gf100_gr_mmio(gr, grctx->hub);
@@ -965,7 +968,7 @@
gf100_gr_mmio(gr, grctx->tpc);
gf100_gr_mmio(gr, grctx->ppc);
- nvkm_wr32(device, 0x404154, 0x00000000);
+ idle_timeout = nvkm_mask(device, 0x404154, 0xffffffff, 0x00000000);
grctx->bundle(info);
grctx->pagepool(info);
@@ -984,10 +987,8 @@
nvkm_wr32(device, 0x405b00, (gr->tpc_total << 8) | gr->gpc_nr);
- gk104_grctx_generate_rop_active_fbps(gr);
-
gf100_gr_icmd(gr, grctx->icmd);
- nvkm_wr32(device, 0x404154, 0x00000400);
+ nvkm_wr32(device, 0x404154, idle_timeout);
gf100_gr_mthd(gr, grctx->mthd);
nvkm_mask(device, 0x419e00, 0x00808080, 0x00808080);
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm200.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm200.c
index e586699..db209d3 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm200.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm200.c
@@ -33,7 +33,7 @@
struct nvkm_device *device = gr->base.engine.subdev.device;
int gpc, tpc, id;
- for (tpc = 0, id = 0; tpc < 4; tpc++) {
+ for (tpc = 0, id = 0; tpc < TPC_MAX_PER_GPC; tpc++) {
for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
if (tpc < gr->tpc_nr[gpc]) {
nvkm_wr32(device, TPC_UNIT(gpc, tpc, 0x698), id);
@@ -45,15 +45,6 @@
}
}
-static void
-gm200_grctx_generate_rop_active_fbps(struct gf100_gr *gr)
-{
- struct nvkm_device *device = gr->base.engine.subdev.device;
- const u32 fbp_count = nvkm_rd32(device, 0x12006c);
- nvkm_mask(device, 0x408850, 0x0000000f, fbp_count); /* zrop */
- nvkm_mask(device, 0x408958, 0x0000000f, fbp_count); /* crop */
-}
-
void
gm200_grctx_generate_405b60(struct gf100_gr *gr)
{
@@ -86,17 +77,17 @@
nvkm_wr32(device, 0x405ba0 + (i * 4), gpcs[i]);
}
-void
+static void
gm200_grctx_generate_main(struct gf100_gr *gr, struct gf100_grctx *info)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
const struct gf100_grctx_func *grctx = gr->func->grctx;
- u32 tmp;
+ u32 idle_timeout, tmp;
int i;
gf100_gr_mmio(gr, gr->fuc_sw_ctx);
- nvkm_wr32(device, 0x404154, 0x00000000);
+ idle_timeout = nvkm_mask(device, 0x404154, 0xffffffff, 0x00000000);
grctx->bundle(info);
grctx->pagepool(info);
@@ -113,8 +104,6 @@
nvkm_wr32(device, 0x405b00, (gr->tpc_total << 8) | gr->gpc_nr);
- gm200_grctx_generate_rop_active_fbps(gr);
-
for (tmp = 0, i = 0; i < gr->gpc_nr; i++)
tmp |= ((1 << gr->tpc_nr[i]) - 1) << (i * 4);
nvkm_wr32(device, 0x4041c4, tmp);
@@ -122,7 +111,7 @@
gm200_grctx_generate_405b60(gr);
gf100_gr_icmd(gr, gr->fuc_bundle);
- nvkm_wr32(device, 0x404154, 0x00000800);
+ nvkm_wr32(device, 0x404154, idle_timeout);
gf100_gr_mthd(gr, gr->fuc_method);
nvkm_mask(device, 0x418e94, 0xffffffff, 0xc4230000);
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm20b.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm20b.c
index a8827ef..e5702e3 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm20b.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm20b.c
@@ -40,15 +40,14 @@
{
struct nvkm_device *device = gr->base.engine.subdev.device;
const struct gf100_grctx_func *grctx = gr->func->grctx;
- int idle_timeout_save;
+ u32 idle_timeout;
int i, tmp;
gf100_gr_mmio(gr, gr->fuc_sw_ctx);
gf100_gr_wait_idle(gr);
- idle_timeout_save = nvkm_rd32(device, 0x404154);
- nvkm_wr32(device, 0x404154, 0x00000000);
+ idle_timeout = nvkm_mask(device, 0x404154, 0xffffffff, 0x00000000);
grctx->attrib(info);
@@ -63,7 +62,6 @@
nvkm_wr32(device, 0x405b00, (gr->tpc_total << 8) | gr->gpc_nr);
- gk104_grctx_generate_rop_active_fbps(gr);
nvkm_wr32(device, 0x408908, nvkm_rd32(device, 0x410108) | 0x80000000);
for (tmp = 0, i = 0; i < gr->gpc_nr; i++)
@@ -74,7 +72,7 @@
gf100_gr_wait_idle(gr);
- nvkm_wr32(device, 0x404154, idle_timeout_save);
+ nvkm_wr32(device, 0x404154, idle_timeout);
gf100_gr_wait_idle(gr);
gf100_gr_mthd(gr, gr->fuc_method);
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpc.fuc b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpc.fuc
index dc60509..4984b00 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpc.fuc
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpc.fuc
@@ -291,12 +291,13 @@
// Main program loop, very simple, sleeps until woken up by the interrupt
// handler, pulls a command from the queue and executes its handler
//
-main:
- bset $flags $p0
+wait:
sleep $p0
+ bset $flags $p0
+main:
mov $r13 #cmd_queue
call(queue_get)
- bra $p1 #main
+ bra $p1 #wait
// 0x0000-0x0003 are all context transfers
cmpu b32 $r14 0x04
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgf100.fuc3.h b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgf100.fuc3.h
index 5f4ddfe..8cb240b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgf100.fuc3.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgf100.fuc3.h
@@ -370,9 +370,10 @@
0xf11f29f0,
0xf0080007,
0x02d00203,
-/* 0x04bb: main */
+/* 0x04bb: wait */
0xf404bd00,
- 0x28f40031,
+ 0x31f40028,
+/* 0x04c1: main */
0x1cd7f000,
0xf43921f4,
0xe4b0f401,
@@ -384,10 +385,10 @@
0x0018fe05,
0x05b421f5,
/* 0x04eb: main_not_ctx_xfer */
- 0x94d30ef4,
+ 0x94d90ef4,
0xf5f010ef,
0x7e21f501,
- 0xc60ef403,
+ 0xcc0ef403,
/* 0x04f8: ih */
0x80f900f9,
0xf90188fe,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgf117.fuc3.h b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgf117.fuc3.h
index 03381b1..550d6ba 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgf117.fuc3.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgf117.fuc3.h
@@ -397,9 +397,10 @@
0x080007f1,
0xd00203f0,
0x04bd0002,
-/* 0x0508: main */
- 0xf40031f4,
- 0xd7f00028,
+/* 0x0508: wait */
+ 0xf40028f4,
+/* 0x050e: main */
+ 0xd7f00031,
0x3921f424,
0xb0f401f4,
0x18f404e4,
@@ -409,13 +410,13 @@
0xfd01e4b6,
0x18fe051e,
0x0121f500,
- 0xd30ef406,
+ 0xd90ef406,
/* 0x0538: main_not_ctx_xfer */
0xf010ef94,
0x21f501f5,
0x0ef4037e,
/* 0x0545: ih */
- 0xf900f9c6,
+ 0xf900f9cc,
0x0188fe80,
0x90f980f9,
0xb0f9a0f9,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgk104.fuc3.h b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgk104.fuc3.h
index 99d9b48..271b59d 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgk104.fuc3.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgk104.fuc3.h
@@ -397,9 +397,10 @@
0x080007f1,
0xd00203f0,
0x04bd0002,
-/* 0x0508: main */
- 0xf40031f4,
- 0xd7f00028,
+/* 0x0508: wait */
+ 0xf40028f4,
+/* 0x050e: main */
+ 0xd7f00031,
0x3921f424,
0xb0f401f4,
0x18f404e4,
@@ -409,13 +410,13 @@
0xfd01e4b6,
0x18fe051e,
0x0121f500,
- 0xd30ef406,
+ 0xd90ef406,
/* 0x0538: main_not_ctx_xfer */
0xf010ef94,
0x21f501f5,
0x0ef4037e,
/* 0x0545: ih */
- 0xf900f9c6,
+ 0xf900f9cc,
0x0188fe80,
0x90f980f9,
0xb0f9a0f9,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgk110.fuc3.h b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgk110.fuc3.h
index f726769..73b4a32 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgk110.fuc3.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgk110.fuc3.h
@@ -397,9 +397,10 @@
0x300007f1,
0xd00203f0,
0x04bd0002,
-/* 0x0508: main */
- 0xf40031f4,
- 0xd7f00028,
+/* 0x0508: wait */
+ 0xf40028f4,
+/* 0x050e: main */
+ 0xd7f00031,
0x3921f424,
0xb0f401f4,
0x18f404e4,
@@ -409,13 +410,13 @@
0xfd01e4b6,
0x18fe051e,
0x0121f500,
- 0xd30ef406,
+ 0xd90ef406,
/* 0x0538: main_not_ctx_xfer */
0xf010ef94,
0x21f501f5,
0x0ef4037e,
/* 0x0545: ih */
- 0xf900f9c6,
+ 0xf900f9cc,
0x0188fe80,
0x90f980f9,
0xb0f9a0f9,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgk208.fuc5.h b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgk208.fuc5.h
index 387d1fa..0181698 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgk208.fuc5.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgk208.fuc5.h
@@ -349,9 +349,10 @@
0x801f29f0,
0xf6023000,
0x04bd0002,
-/* 0x0448: main */
- 0xf40031f4,
- 0x240d0028,
+/* 0x0448: wait */
+ 0xf40028f4,
+/* 0x044e: main */
+ 0x240d0031,
0x0000377e,
0xb0f401f4,
0x18f404e4,
@@ -362,10 +363,10 @@
0x0018fe05,
0x00051f7e,
/* 0x0477: main_not_ctx_xfer */
- 0x94d40ef4,
+ 0x94da0ef4,
0xf5f010ef,
0x02f87e01,
- 0xc70ef400,
+ 0xcd0ef400,
/* 0x0484: ih */
0x80f900f9,
0xf90188fe,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgm107.fuc5.h b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgm107.fuc5.h
index fa9f3c0..eca007f 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgm107.fuc5.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/gpcgm107.fuc5.h
@@ -427,9 +427,10 @@
0x1f29f024,
0x02300080,
0xbd0002f6,
-/* 0x0571: main */
- 0x0031f404,
- 0x0d0028f4,
+/* 0x0571: wait */
+ 0x0028f404,
+/* 0x0577: main */
+ 0x0d0031f4,
0x00377e24,
0xf401f400,
0xf404e4b0,
@@ -439,13 +440,13 @@
0xfd01e4b6,
0x18fe051e,
0x06487e00,
- 0xd40ef400,
+ 0xda0ef400,
/* 0x05a0: main_not_ctx_xfer */
0xf010ef94,
0xf87e01f5,
0x0ef40002,
/* 0x05ad: ih */
- 0xf900f9c7,
+ 0xf900f9cd,
0x0188fe80,
0x90f980f9,
0xb0f9a0f9,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hub.fuc b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hub.fuc
index e3a2fb3..4d416d4 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hub.fuc
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hub.fuc
@@ -218,13 +218,14 @@
// Main program loop, very simple, sleeps until woken up by the interrupt
// handler, pulls a command from the queue and executes its handler
//
-main:
+wait:
// sleep until we have something to do
- bset $flags $p0
sleep $p0
+ bset $flags $p0
+main:
mov $r13 #cmd_queue
call(queue_get)
- bra $p1 #main
+ bra $p1 #wait
// context switch, requested by GPU?
cmpu b32 $r14 0x4001
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgf100.fuc3.h b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgf100.fuc3.h
index 397921a..8015b40 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgf100.fuc3.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgf100.fuc3.h
@@ -584,9 +584,10 @@
0x080007f1,
0xd00203f0,
0x04bd0001,
-/* 0x0564: main */
- 0xf40031f4,
- 0xd7f00028,
+/* 0x0564: wait */
+ 0xf40028f4,
+/* 0x056a: main */
+ 0xd7f00031,
0x3921f410,
0xb1f401f4,
0xf54001e4,
@@ -650,7 +651,7 @@
0x170007f1,
0xd00203f0,
0x04bd0009,
- 0xff080ef5,
+ 0xff0e0ef5,
/* 0x0660: main_not_ctx_switch */
0xf401e4b0,
0xf2b90d1b,
@@ -675,12 +676,12 @@
0xf501f5f0,
0xf5037e21,
/* 0x06b3: main_done */
- 0xbdfeb50e,
+ 0xbdfebb0e,
0x1f29f024,
0x080007f1,
0xd00203f0,
0x04bd0002,
- 0xfea00ef5,
+ 0xfea60ef5,
/* 0x06c8: ih */
0x80f900f9,
0xf90188fe,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgf117.fuc3.h b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgf117.fuc3.h
index 50c9716..2af90ec 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgf117.fuc3.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgf117.fuc3.h
@@ -584,9 +584,10 @@
0x080007f1,
0xd00203f0,
0x04bd0001,
-/* 0x0564: main */
- 0xf40031f4,
- 0xd7f00028,
+/* 0x0564: wait */
+ 0xf40028f4,
+/* 0x056a: main */
+ 0xd7f00031,
0x3921f410,
0xb1f401f4,
0xf54001e4,
@@ -650,7 +651,7 @@
0x170007f1,
0xd00203f0,
0x04bd0009,
- 0xff080ef5,
+ 0xff0e0ef5,
/* 0x0660: main_not_ctx_switch */
0xf401e4b0,
0xf2b90d1b,
@@ -675,12 +676,12 @@
0xf501f5f0,
0xf5037e21,
/* 0x06b3: main_done */
- 0xbdfeb50e,
+ 0xbdfebb0e,
0x1f29f024,
0x080007f1,
0xd00203f0,
0x04bd0002,
- 0xfea00ef5,
+ 0xfea60ef5,
/* 0x06c8: ih */
0x80f900f9,
0xf90188fe,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgk104.fuc3.h b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgk104.fuc3.h
index 125824b..e8b8c1c 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgk104.fuc3.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgk104.fuc3.h
@@ -584,9 +584,10 @@
0x080007f1,
0xd00203f0,
0x04bd0001,
-/* 0x0564: main */
- 0xf40031f4,
- 0xd7f00028,
+/* 0x0564: wait */
+ 0xf40028f4,
+/* 0x056a: main */
+ 0xd7f00031,
0x3921f410,
0xb1f401f4,
0xf54001e4,
@@ -650,7 +651,7 @@
0x170007f1,
0xd00203f0,
0x04bd0009,
- 0xff080ef5,
+ 0xff0e0ef5,
/* 0x0660: main_not_ctx_switch */
0xf401e4b0,
0xf2b90d1b,
@@ -675,12 +676,12 @@
0xf501f5f0,
0xf5037e21,
/* 0x06b3: main_done */
- 0xbdfeb50e,
+ 0xbdfebb0e,
0x1f29f024,
0x080007f1,
0xd00203f0,
0x04bd0002,
- 0xfea00ef5,
+ 0xfea60ef5,
/* 0x06c8: ih */
0x80f900f9,
0xf90188fe,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgk110.fuc3.h b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgk110.fuc3.h
index 0a1b8c0..f4ed2fb 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgk110.fuc3.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgk110.fuc3.h
@@ -584,9 +584,10 @@
0x300007f1,
0xd00203f0,
0x04bd0001,
-/* 0x0564: main */
- 0xf40031f4,
- 0xd7f00028,
+/* 0x0564: wait */
+ 0xf40028f4,
+/* 0x056a: main */
+ 0xd7f00031,
0x3921f410,
0xb1f401f4,
0xf54001e4,
@@ -650,7 +651,7 @@
0x170007f1,
0xd00203f0,
0x04bd0009,
- 0xff080ef5,
+ 0xff0e0ef5,
/* 0x0660: main_not_ctx_switch */
0xf401e4b0,
0xf2b90d1b,
@@ -675,12 +676,12 @@
0xf501f5f0,
0xf5037e21,
/* 0x06b3: main_done */
- 0xbdfeb50e,
+ 0xbdfebb0e,
0x1f29f024,
0x300007f1,
0xd00203f0,
0x04bd0002,
- 0xfea00ef5,
+ 0xfea60ef5,
/* 0x06c8: ih */
0x80f900f9,
0xf90188fe,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgk208.fuc5.h b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgk208.fuc5.h
index 16869d0..ed48897 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgk208.fuc5.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgk208.fuc5.h
@@ -531,9 +531,10 @@
0x1f19f014,
0x02300080,
0xbd0001f6,
-/* 0x0491: main */
- 0x0031f404,
- 0x0d0028f4,
+/* 0x0491: wait */
+ 0x0028f404,
+/* 0x0497: main */
+ 0x0d0031f4,
0x00377e10,
0xf401f400,
0x4001e4b1,
@@ -590,7 +591,7 @@
0x09f60217,
0xf504bd00,
/* 0x056b: main_not_ctx_switch */
- 0xb0ff2a0e,
+ 0xb0ff300e,
0x1bf401e4,
0x7ef2b20c,
0xf4000820,
@@ -612,11 +613,11 @@
0x7e01f5f0,
0xf50002f8,
/* 0x05b7: main_done */
- 0xbdfede0e,
+ 0xbdfee40e,
0x1f29f024,
0x02300080,
0xbd0002f6,
- 0xcc0ef504,
+ 0xd20ef504,
/* 0x05c9: ih */
0xf900f9fe,
0x0188fe80,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgm107.fuc5.h b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgm107.fuc5.h
index d6343d2..5c90518 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgm107.fuc5.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/fuc/hubgm107.fuc5.h
@@ -531,9 +531,10 @@
0x1f19f014,
0x02300080,
0xbd0001f6,
-/* 0x0491: main */
- 0x0031f404,
- 0x0d0028f4,
+/* 0x0491: wait */
+ 0x0028f404,
+/* 0x0497: main */
+ 0x0d0031f4,
0x00377e10,
0xf401f400,
0x4001e4b1,
@@ -590,7 +591,7 @@
0x09f60217,
0xf504bd00,
/* 0x056b: main_not_ctx_switch */
- 0xb0ff2a0e,
+ 0xb0ff300e,
0x1bf401e4,
0x7ef2b20c,
0xf4000820,
@@ -612,11 +613,11 @@
0x7e01f5f0,
0xf50002f8,
/* 0x05b7: main_done */
- 0xbdfede0e,
+ 0xbdfee40e,
0x1f29f024,
0x02300080,
0xbd0002f6,
- 0xcc0ef504,
+ 0xd20ef504,
/* 0x05c9: ih */
0xf900f9fe,
0x0188fe80,
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.c
index b2de290..9513bad 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.c
@@ -702,6 +702,13 @@
* PGRAPH engine/subdev functions
******************************************************************************/
+int
+gf100_gr_rops(struct gf100_gr *gr)
+{
+ struct nvkm_device *device = gr->base.engine.subdev.device;
+ return (nvkm_rd32(device, 0x409604) & 0x001f0000) >> 16;
+}
+
void
gf100_gr_zbc_init(struct gf100_gr *gr)
{
@@ -1609,32 +1616,12 @@
{
struct gf100_gr *gr = gf100_gr(base);
struct nvkm_device *device = gr->base.engine.subdev.device;
- int ret, i, j;
+ int i, j;
nvkm_pmu_pgob(device->pmu, false);
- ret = nvkm_memory_new(device, NVKM_MEM_TARGET_INST, 0x1000, 256, false,
- &gr->unk4188b4);
- if (ret)
- return ret;
-
- ret = nvkm_memory_new(device, NVKM_MEM_TARGET_INST, 0x1000, 256, false,
- &gr->unk4188b8);
- if (ret)
- return ret;
-
- nvkm_kmap(gr->unk4188b4);
- for (i = 0; i < 0x1000; i += 4)
- nvkm_wo32(gr->unk4188b4, i, 0x00000010);
- nvkm_done(gr->unk4188b4);
-
- nvkm_kmap(gr->unk4188b8);
- for (i = 0; i < 0x1000; i += 4)
- nvkm_wo32(gr->unk4188b8, i, 0x00000010);
- nvkm_done(gr->unk4188b8);
-
- gr->rop_nr = (nvkm_rd32(device, 0x409604) & 0x001f0000) >> 16;
- gr->gpc_nr = nvkm_rd32(device, 0x409604) & 0x0000001f;
+ gr->rop_nr = gr->func->rops(gr);
+ gr->gpc_nr = nvkm_rd32(device, 0x409604) & 0x0000001f;
for (i = 0; i < gr->gpc_nr; i++) {
gr->tpc_nr[i] = nvkm_rd32(device, GPC_UNIT(i, 0x2608));
gr->tpc_total += gr->tpc_nr[i];
@@ -1651,38 +1638,38 @@
switch (device->chipset) {
case 0xc0:
if (gr->tpc_total == 11) { /* 465, 3/4/4/0, 4 */
- gr->magic_not_rop_nr = 0x07;
+ gr->screen_tile_row_offset = 0x07;
} else
if (gr->tpc_total == 14) { /* 470, 3/3/4/4, 5 */
- gr->magic_not_rop_nr = 0x05;
+ gr->screen_tile_row_offset = 0x05;
} else
if (gr->tpc_total == 15) { /* 480, 3/4/4/4, 6 */
- gr->magic_not_rop_nr = 0x06;
+ gr->screen_tile_row_offset = 0x06;
}
break;
case 0xc3: /* 450, 4/0/0/0, 2 */
- gr->magic_not_rop_nr = 0x03;
+ gr->screen_tile_row_offset = 0x03;
break;
case 0xc4: /* 460, 3/4/0/0, 4 */
- gr->magic_not_rop_nr = 0x01;
+ gr->screen_tile_row_offset = 0x01;
break;
case 0xc1: /* 2/0/0/0, 1 */
- gr->magic_not_rop_nr = 0x01;
+ gr->screen_tile_row_offset = 0x01;
break;
case 0xc8: /* 4/4/3/4, 5 */
- gr->magic_not_rop_nr = 0x06;
+ gr->screen_tile_row_offset = 0x06;
break;
case 0xce: /* 4/4/0/0, 4 */
- gr->magic_not_rop_nr = 0x03;
+ gr->screen_tile_row_offset = 0x03;
break;
case 0xcf: /* 4/0/0/0, 3 */
- gr->magic_not_rop_nr = 0x03;
+ gr->screen_tile_row_offset = 0x03;
break;
case 0xd7:
case 0xd9: /* 1/0/0/0, 1 */
case 0xea: /* gk20a */
case 0x12b: /* gm20b */
- gr->magic_not_rop_nr = 0x01;
+ gr->screen_tile_row_offset = 0x01;
break;
}
@@ -1729,8 +1716,6 @@
gf100_gr_dtor_init(gr->fuc_sw_ctx);
gf100_gr_dtor_init(gr->fuc_sw_nonctx);
- nvkm_memory_del(&gr->unk4188b8);
- nvkm_memory_del(&gr->unk4188b4);
return gr;
}
@@ -1776,7 +1761,7 @@
gr->firmware = nvkm_boolopt(device->cfgopt, "NvGrUseFW",
func->fecs.ucode == NULL);
- ret = nvkm_gr_ctor(&gf100_gr_, device, index, 0x08001000,
+ ret = nvkm_gr_ctor(&gf100_gr_, device, index,
gr->firmware || func->fecs.ucode != NULL,
&gr->base);
if (ret)
@@ -1815,6 +1800,7 @@
gf100_gr_init(struct gf100_gr *gr)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
+ struct nvkm_fb *fb = device->fb;
const u32 magicgpc918 = DIV_ROUND_UP(0x00800000, gr->tpc_total);
u32 data[TPC_MAX / 8] = {};
u8 tpcnr[GPC_MAX];
@@ -1827,8 +1813,8 @@
nvkm_wr32(device, GPC_BCAST(0x088c), 0x00000000);
nvkm_wr32(device, GPC_BCAST(0x0890), 0x00000000);
nvkm_wr32(device, GPC_BCAST(0x0894), 0x00000000);
- nvkm_wr32(device, GPC_BCAST(0x08b4), nvkm_memory_addr(gr->unk4188b4) >> 8);
- nvkm_wr32(device, GPC_BCAST(0x08b8), nvkm_memory_addr(gr->unk4188b8) >> 8);
+ nvkm_wr32(device, GPC_BCAST(0x08b4), nvkm_memory_addr(fb->mmu_wr) >> 8);
+ nvkm_wr32(device, GPC_BCAST(0x08b8), nvkm_memory_addr(fb->mmu_rd) >> 8);
gf100_gr_mmio(gr, gr->func->mmio);
@@ -1851,9 +1837,9 @@
for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
nvkm_wr32(device, GPC_UNIT(gpc, 0x0914),
- gr->magic_not_rop_nr << 8 | gr->tpc_nr[gpc]);
+ gr->screen_tile_row_offset << 8 | gr->tpc_nr[gpc]);
nvkm_wr32(device, GPC_UNIT(gpc, 0x0910), 0x00040000 |
- gr->tpc_total);
+ gr->tpc_total);
nvkm_wr32(device, GPC_UNIT(gpc, 0x0918), magicgpc918);
}
@@ -1946,6 +1932,7 @@
.mmio = gf100_gr_pack_mmio,
.fecs.ucode = &gf100_gr_fecs_ucode,
.gpccs.ucode = &gf100_gr_gpccs_ucode,
+ .rops = gf100_gr_rops,
.grctx = &gf100_grctx,
.sclass = {
{ -1, -1, FERMI_TWOD_A },
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.h b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.h
index f0c6acb..2b98abd 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.h
@@ -31,7 +31,8 @@
#include <subdev/mmu.h>
#define GPC_MAX 32
-#define TPC_MAX (GPC_MAX * 8)
+#define TPC_MAX_PER_GPC 8
+#define TPC_MAX (GPC_MAX * TPC_MAX_PER_GPC)
#define ROP_BCAST(r) (0x408800 + (r))
#define ROP_UNIT(u, r) (0x410000 + (u) * 0x400 + (r))
@@ -100,15 +101,12 @@
u8 ppc_mask[GPC_MAX];
u8 ppc_tpc_nr[GPC_MAX][4];
- struct nvkm_memory *unk4188b4;
- struct nvkm_memory *unk4188b8;
-
struct gf100_gr_data mmio_data[4];
struct gf100_gr_mmio mmio_list[4096/8];
u32 size;
u32 *data;
- u8 magic_not_rop_nr;
+ u8 screen_tile_row_offset;
};
int gf100_gr_ctor(const struct gf100_gr_func *, struct nvkm_device *,
@@ -121,6 +119,8 @@
void (*dtor)(struct gf100_gr *);
int (*init)(struct gf100_gr *);
void (*init_gpc_mmu)(struct gf100_gr *);
+ void (*init_rop_active_fbps)(struct gf100_gr *);
+ void (*init_ppc_exceptions)(struct gf100_gr *);
void (*set_hww_esr_report_mask)(struct gf100_gr *);
const struct gf100_gr_pack *mmio;
struct {
@@ -129,18 +129,23 @@
struct {
struct gf100_gr_ucode *ucode;
} gpccs;
+ int (*rops)(struct gf100_gr *);
int ppc_nr;
const struct gf100_grctx_func *grctx;
struct nvkm_sclass sclass[];
};
int gf100_gr_init(struct gf100_gr *);
+int gf100_gr_rops(struct gf100_gr *);
int gk104_gr_init(struct gf100_gr *);
+void gk104_gr_init_rop_active_fbps(struct gf100_gr *);
+void gk104_gr_init_ppc_exceptions(struct gf100_gr *);
int gk20a_gr_init(struct gf100_gr *);
int gm200_gr_init(struct gf100_gr *);
+int gm200_gr_rops(struct gf100_gr *);
#define gf100_gr_chan(p) container_of((p), struct gf100_gr_chan, object)
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf104.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf104.c
index 8f253e0..d736dcd 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf104.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf104.c
@@ -118,6 +118,7 @@
.mmio = gf104_gr_pack_mmio,
.fecs.ucode = &gf100_gr_fecs_ucode,
.gpccs.ucode = &gf100_gr_gpccs_ucode,
+ .rops = gf100_gr_rops,
.grctx = &gf104_grctx,
.sclass = {
{ -1, -1, FERMI_TWOD_A },
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf108.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf108.c
index 815a5aa..2f0d244 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf108.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf108.c
@@ -109,6 +109,7 @@
.mmio = gf108_gr_pack_mmio,
.fecs.ucode = &gf100_gr_fecs_ucode,
.gpccs.ucode = &gf100_gr_gpccs_ucode,
+ .rops = gf100_gr_rops,
.grctx = &gf108_grctx,
.sclass = {
{ -1, -1, FERMI_TWOD_A },
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf110.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf110.c
index d081ee4..d1d942e 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf110.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf110.c
@@ -90,6 +90,7 @@
.mmio = gf110_gr_pack_mmio,
.fecs.ucode = &gf100_gr_fecs_ucode,
.gpccs.ucode = &gf100_gr_gpccs_ucode,
+ .rops = gf100_gr_rops,
.grctx = &gf110_grctx,
.sclass = {
{ -1, -1, FERMI_TWOD_A },
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf117.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf117.c
index d8e8af4..70335f6 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf117.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf117.c
@@ -126,6 +126,7 @@
.mmio = gf117_gr_pack_mmio,
.fecs.ucode = &gf117_gr_fecs_ucode,
.gpccs.ucode = &gf117_gr_gpccs_ucode,
+ .rops = gf100_gr_rops,
.ppc_nr = 1,
.grctx = &gf117_grctx,
.sclass = {
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf119.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf119.c
index 01faf9a..8d8e4ca 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf119.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gf119.c
@@ -181,6 +181,7 @@
.mmio = gf119_gr_pack_mmio,
.fecs.ucode = &gf100_gr_fecs_ucode,
.gpccs.ucode = &gf100_gr_gpccs_ucode,
+ .rops = gf100_gr_rops,
.grctx = &gf119_grctx,
.sclass = {
{ -1, -1, FERMI_TWOD_A },
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.c
index abf5492..ec22da6 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.c
@@ -24,6 +24,8 @@
#include "gf100.h"
#include "ctxgf100.h"
+#include <subdev/fb.h>
+
#include <nvif/class.h>
/*******************************************************************************
@@ -177,10 +179,35 @@
* PGRAPH engine/subdev functions
******************************************************************************/
+void
+gk104_gr_init_rop_active_fbps(struct gf100_gr *gr)
+{
+ struct nvkm_device *device = gr->base.engine.subdev.device;
+ const u32 fbp_count = nvkm_rd32(device, 0x120074);
+ nvkm_mask(device, 0x408850, 0x0000000f, fbp_count); /* zrop */
+ nvkm_mask(device, 0x408958, 0x0000000f, fbp_count); /* crop */
+}
+
+void
+gk104_gr_init_ppc_exceptions(struct gf100_gr *gr)
+{
+ struct nvkm_device *device = gr->base.engine.subdev.device;
+ int gpc, ppc;
+
+ for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
+ for (ppc = 0; ppc < gr->ppc_nr[gpc]; ppc++) {
+ if (!(gr->ppc_mask[gpc] & (1 << ppc)))
+ continue;
+ nvkm_wr32(device, PPC_UNIT(gpc, ppc, 0x038), 0xc0000000);
+ }
+ }
+}
+
int
gk104_gr_init(struct gf100_gr *gr)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
+ struct nvkm_fb *fb = device->fb;
const u32 magicgpc918 = DIV_ROUND_UP(0x00800000, gr->tpc_total);
u32 data[TPC_MAX / 8] = {};
u8 tpcnr[GPC_MAX];
@@ -193,8 +220,8 @@
nvkm_wr32(device, GPC_BCAST(0x088c), 0x00000000);
nvkm_wr32(device, GPC_BCAST(0x0890), 0x00000000);
nvkm_wr32(device, GPC_BCAST(0x0894), 0x00000000);
- nvkm_wr32(device, GPC_BCAST(0x08b4), nvkm_memory_addr(gr->unk4188b4) >> 8);
- nvkm_wr32(device, GPC_BCAST(0x08b8), nvkm_memory_addr(gr->unk4188b8) >> 8);
+ nvkm_wr32(device, GPC_BCAST(0x08b4), nvkm_memory_addr(fb->mmu_wr) >> 8);
+ nvkm_wr32(device, GPC_BCAST(0x08b8), nvkm_memory_addr(fb->mmu_rd) >> 8);
gf100_gr_mmio(gr, gr->func->mmio);
@@ -218,15 +245,17 @@
for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
nvkm_wr32(device, GPC_UNIT(gpc, 0x0914),
- gr->magic_not_rop_nr << 8 | gr->tpc_nr[gpc]);
+ gr->screen_tile_row_offset << 8 | gr->tpc_nr[gpc]);
nvkm_wr32(device, GPC_UNIT(gpc, 0x0910), 0x00040000 |
- gr->tpc_total);
+ gr->tpc_total);
nvkm_wr32(device, GPC_UNIT(gpc, 0x0918), magicgpc918);
}
nvkm_wr32(device, GPC_BCAST(0x3fd4), magicgpc918);
nvkm_wr32(device, GPC_BCAST(0x08ac), nvkm_rd32(device, 0x100800));
+ gr->func->init_rop_active_fbps(gr);
+
nvkm_wr32(device, 0x400500, 0x00010001);
nvkm_wr32(device, 0x400100, 0xffffffff);
@@ -246,8 +275,9 @@
nvkm_mask(device, 0x419cc0, 0x00000008, 0x00000008);
nvkm_mask(device, 0x419eb4, 0x00001000, 0x00001000);
+ gr->func->init_ppc_exceptions(gr);
+
for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
- nvkm_wr32(device, GPC_UNIT(gpc, 0x3038), 0xc0000000);
nvkm_wr32(device, GPC_UNIT(gpc, 0x0420), 0xc0000000);
nvkm_wr32(device, GPC_UNIT(gpc, 0x0900), 0xc0000000);
nvkm_wr32(device, GPC_UNIT(gpc, 0x1028), 0xc0000000);
@@ -309,9 +339,12 @@
static const struct gf100_gr_func
gk104_gr = {
.init = gk104_gr_init,
+ .init_rop_active_fbps = gk104_gr_init_rop_active_fbps,
+ .init_ppc_exceptions = gk104_gr_init_ppc_exceptions,
.mmio = gk104_gr_pack_mmio,
.fecs.ucode = &gk104_gr_fecs_ucode,
.gpccs.ucode = &gk104_gr_gpccs_ucode,
+ .rops = gf100_gr_rops,
.ppc_nr = 1,
.grctx = &gk104_grctx,
.sclass = {
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110.c
index 32aa294..f31b171 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110.c
@@ -183,9 +183,12 @@
static const struct gf100_gr_func
gk110_gr = {
.init = gk104_gr_init,
+ .init_rop_active_fbps = gk104_gr_init_rop_active_fbps,
+ .init_ppc_exceptions = gk104_gr_init_ppc_exceptions,
.mmio = gk110_gr_pack_mmio,
.fecs.ucode = &gk110_gr_fecs_ucode,
.gpccs.ucode = &gk110_gr_gpccs_ucode,
+ .rops = gf100_gr_rops,
.ppc_nr = 2,
.grctx = &gk110_grctx,
.sclass = {
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110b.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110b.c
index 22f88af..d76dd178 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110b.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110b.c
@@ -103,9 +103,12 @@
static const struct gf100_gr_func
gk110b_gr = {
.init = gk104_gr_init,
+ .init_rop_active_fbps = gk104_gr_init_rop_active_fbps,
+ .init_ppc_exceptions = gk104_gr_init_ppc_exceptions,
.mmio = gk110b_gr_pack_mmio,
.fecs.ucode = &gk110_gr_fecs_ucode,
.gpccs.ucode = &gk110_gr_gpccs_ucode,
+ .rops = gf100_gr_rops,
.ppc_nr = 2,
.grctx = &gk110b_grctx,
.sclass = {
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk208.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk208.c
index ee7554f..14bbe6e 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk208.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk208.c
@@ -162,9 +162,12 @@
static const struct gf100_gr_func
gk208_gr = {
.init = gk104_gr_init,
+ .init_rop_active_fbps = gk104_gr_init_rop_active_fbps,
+ .init_ppc_exceptions = gk104_gr_init_ppc_exceptions,
.mmio = gk208_gr_pack_mmio,
.fecs.ucode = &gk208_gr_fecs_ucode,
.gpccs.ucode = &gk208_gr_gpccs_ucode,
+ .rops = gf100_gr_rops,
.ppc_nr = 1,
.grctx = &gk208_grctx,
.sclass = {
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk20a.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk20a.c
index 7ffb8a6..4ca8ed1 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk20a.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gk20a.c
@@ -239,9 +239,6 @@
return ret;
/* MMU debug buffer */
- nvkm_wr32(device, 0x100cc8, nvkm_memory_addr(gr->unk4188b4) >> 8);
- nvkm_wr32(device, 0x100ccc, nvkm_memory_addr(gr->unk4188b8) >> 8);
-
if (gr->func->init_gpc_mmu)
gr->func->init_gpc_mmu(gr);
@@ -267,7 +264,7 @@
for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
nvkm_wr32(device, GPC_UNIT(gpc, 0x0914),
- gr->magic_not_rop_nr << 8 | gr->tpc_nr[gpc]);
+ gr->screen_tile_row_offset << 8 | gr->tpc_nr[gpc]);
nvkm_wr32(device, GPC_UNIT(gpc, 0x0910), 0x00040000 |
gr->tpc_total);
nvkm_wr32(device, GPC_UNIT(gpc, 0x0918), magicgpc918);
@@ -275,6 +272,8 @@
nvkm_wr32(device, GPC_BCAST(0x3fd4), magicgpc918);
+ gr->func->init_rop_active_fbps(gr);
+
/* Enable FIFO access */
nvkm_wr32(device, 0x400500, 0x00010001);
@@ -312,7 +311,9 @@
static const struct gf100_gr_func
gk20a_gr = {
.init = gk20a_gr_init,
+ .init_rop_active_fbps = gk104_gr_init_rop_active_fbps,
.set_hww_esr_report_mask = gk20a_gr_set_hww_esr_report_mask,
+ .rops = gf100_gr_rops,
.ppc_nr = 1,
.grctx = &gk20a_grctx,
.sclass = {
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm107.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm107.c
index 56e9602..45f965f 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm107.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm107.c
@@ -26,6 +26,7 @@
#include <subdev/bios.h>
#include <subdev/bios/P0260.h>
+#include <subdev/fb.h>
#include <nvif/class.h>
@@ -311,17 +312,18 @@
gm107_gr_init(struct gf100_gr *gr)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
+ struct nvkm_fb *fb = device->fb;
const u32 magicgpc918 = DIV_ROUND_UP(0x00800000, gr->tpc_total);
u32 data[TPC_MAX / 8] = {};
u8 tpcnr[GPC_MAX];
- int gpc, tpc, ppc, rop;
+ int gpc, tpc, rop;
int i;
nvkm_wr32(device, GPC_BCAST(0x0880), 0x00000000);
nvkm_wr32(device, GPC_BCAST(0x0890), 0x00000000);
nvkm_wr32(device, GPC_BCAST(0x0894), 0x00000000);
- nvkm_wr32(device, GPC_BCAST(0x08b4), nvkm_memory_addr(gr->unk4188b4) >> 8);
- nvkm_wr32(device, GPC_BCAST(0x08b8), nvkm_memory_addr(gr->unk4188b8) >> 8);
+ nvkm_wr32(device, GPC_BCAST(0x08b4), nvkm_memory_addr(fb->mmu_wr) >> 8);
+ nvkm_wr32(device, GPC_BCAST(0x08b8), nvkm_memory_addr(fb->mmu_rd) >> 8);
gf100_gr_mmio(gr, gr->func->mmio);
@@ -347,15 +349,17 @@
for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
nvkm_wr32(device, GPC_UNIT(gpc, 0x0914),
- gr->magic_not_rop_nr << 8 | gr->tpc_nr[gpc]);
+ gr->screen_tile_row_offset << 8 | gr->tpc_nr[gpc]);
nvkm_wr32(device, GPC_UNIT(gpc, 0x0910), 0x00040000 |
- gr->tpc_total);
+ gr->tpc_total);
nvkm_wr32(device, GPC_UNIT(gpc, 0x0918), magicgpc918);
}
nvkm_wr32(device, GPC_BCAST(0x3fd4), magicgpc918);
nvkm_wr32(device, GPC_BCAST(0x08ac), nvkm_rd32(device, 0x100800));
+ gr->func->init_rop_active_fbps(gr);
+
nvkm_wr32(device, 0x400500, 0x00010001);
nvkm_wr32(device, 0x400100, 0xffffffff);
@@ -373,9 +377,9 @@
nvkm_wr32(device, 0x405844, 0x00ffffff);
nvkm_mask(device, 0x419cc0, 0x00000008, 0x00000008);
+ gr->func->init_ppc_exceptions(gr);
+
for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
- for (ppc = 0; ppc < 2 /* gr->ppc_nr[gpc] */; ppc++)
- nvkm_wr32(device, PPC_UNIT(gpc, ppc, 0x038), 0xc0000000);
nvkm_wr32(device, GPC_UNIT(gpc, 0x0420), 0xc0000000);
nvkm_wr32(device, GPC_UNIT(gpc, 0x0900), 0xc0000000);
nvkm_wr32(device, GPC_UNIT(gpc, 0x1028), 0xc0000000);
@@ -438,9 +442,12 @@
static const struct gf100_gr_func
gm107_gr = {
.init = gm107_gr_init,
+ .init_rop_active_fbps = gk104_gr_init_rop_active_fbps,
+ .init_ppc_exceptions = gk104_gr_init_ppc_exceptions,
.mmio = gm107_gr_pack_mmio,
.fecs.ucode = &gm107_gr_fecs_ucode,
.gpccs.ucode = &gm107_gr_gpccs_ucode,
+ .rops = gf100_gr_rops,
.ppc_nr = 2,
.grctx = &gm107_grctx,
.sclass = {
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm200.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm200.c
index 058fc1d..4dfa451 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm200.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm200.c
@@ -33,27 +33,45 @@
******************************************************************************/
int
+gm200_gr_rops(struct gf100_gr *gr)
+{
+ return nvkm_rd32(gr->base.engine.subdev.device, 0x12006c);
+}
+
+static void
+gm200_gr_init_gpc_mmu(struct gf100_gr *gr)
+{
+ struct nvkm_device *device = gr->base.engine.subdev.device;
+
+ nvkm_wr32(device, 0x418880, nvkm_rd32(device, 0x100c80) & 0xf0001fff);
+ nvkm_wr32(device, 0x418890, 0x00000000);
+ nvkm_wr32(device, 0x418894, 0x00000000);
+
+ nvkm_wr32(device, 0x4188b4, nvkm_rd32(device, 0x100cc8));
+ nvkm_wr32(device, 0x4188b8, nvkm_rd32(device, 0x100ccc));
+ nvkm_wr32(device, 0x4188b0, nvkm_rd32(device, 0x100cc4));
+}
+
+static void
+gm200_gr_init_rop_active_fbps(struct gf100_gr *gr)
+{
+ struct nvkm_device *device = gr->base.engine.subdev.device;
+ const u32 fbp_count = nvkm_rd32(device, 0x12006c);
+ nvkm_mask(device, 0x408850, 0x0000000f, fbp_count); /* zrop */
+ nvkm_mask(device, 0x408958, 0x0000000f, fbp_count); /* crop */
+}
+
+int
gm200_gr_init(struct gf100_gr *gr)
{
struct nvkm_device *device = gr->base.engine.subdev.device;
const u32 magicgpc918 = DIV_ROUND_UP(0x00800000, gr->tpc_total);
- u32 data[TPC_MAX / 8] = {}, tmp;
+ u32 data[TPC_MAX / 8] = {};
u8 tpcnr[GPC_MAX];
- int gpc, tpc, ppc, rop;
+ int gpc, tpc, rop;
int i;
- tmp = nvkm_rd32(device, 0x100c80); /*XXX: mask? */
- nvkm_wr32(device, 0x418880, 0x00001000 | (tmp & 0x00000fff));
- nvkm_wr32(device, 0x418890, 0x00000000);
- nvkm_wr32(device, 0x418894, 0x00000000);
- nvkm_wr32(device, 0x4188b4, nvkm_memory_addr(gr->unk4188b4) >> 8);
- nvkm_wr32(device, 0x4188b8, nvkm_memory_addr(gr->unk4188b8) >> 8);
- nvkm_mask(device, 0x4188b0, 0x00040000, 0x00040000);
-
- /*XXX: belongs in fb */
- nvkm_wr32(device, 0x100cc8, nvkm_memory_addr(gr->unk4188b4) >> 8);
- nvkm_wr32(device, 0x100ccc, nvkm_memory_addr(gr->unk4188b8) >> 8);
- nvkm_mask(device, 0x100cc4, 0x00040000, 0x00040000);
+ gr->func->init_gpc_mmu(gr);
gf100_gr_mmio(gr, gr->fuc_sw_nonctx);
@@ -79,9 +97,9 @@
for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
nvkm_wr32(device, GPC_UNIT(gpc, 0x0914),
- gr->magic_not_rop_nr << 8 | gr->tpc_nr[gpc]);
+ gr->screen_tile_row_offset << 8 | gr->tpc_nr[gpc]);
nvkm_wr32(device, GPC_UNIT(gpc, 0x0910), 0x00040000 |
- gr->tpc_total);
+ gr->tpc_total);
nvkm_wr32(device, GPC_UNIT(gpc, 0x0918), magicgpc918);
}
@@ -89,6 +107,8 @@
nvkm_wr32(device, GPC_BCAST(0x08ac), nvkm_rd32(device, 0x100800));
nvkm_wr32(device, GPC_BCAST(0x033c), nvkm_rd32(device, 0x100804));
+ gr->func->init_rop_active_fbps(gr);
+
nvkm_wr32(device, 0x400500, 0x00010001);
nvkm_wr32(device, 0x400100, 0xffffffff);
nvkm_wr32(device, 0x40013c, 0xffffffff);
@@ -106,9 +126,9 @@
nvkm_wr32(device, 0x405844, 0x00ffffff);
nvkm_mask(device, 0x419cc0, 0x00000008, 0x00000008);
+ gr->func->init_ppc_exceptions(gr);
+
for (gpc = 0; gpc < gr->gpc_nr; gpc++) {
- for (ppc = 0; ppc < gr->ppc_nr[gpc]; ppc++)
- nvkm_wr32(device, PPC_UNIT(gpc, ppc, 0x038), 0xc0000000);
nvkm_wr32(device, GPC_UNIT(gpc, 0x0420), 0xc0000000);
nvkm_wr32(device, GPC_UNIT(gpc, 0x0900), 0xc0000000);
nvkm_wr32(device, GPC_UNIT(gpc, 0x1028), 0xc0000000);
@@ -189,6 +209,10 @@
static const struct gf100_gr_func
gm200_gr = {
.init = gm200_gr_init,
+ .init_gpc_mmu = gm200_gr_init_gpc_mmu,
+ .init_rop_active_fbps = gm200_gr_init_rop_active_fbps,
+ .init_ppc_exceptions = gk104_gr_init_ppc_exceptions,
+ .rops = gm200_gr_rops,
.ppc_nr = 2,
.grctx = &gm200_grctx,
.sclass = {
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm20b.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm20b.c
index 29732bc..69479af 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm20b.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/gm20b.c
@@ -42,7 +42,7 @@
}
val = nvkm_rd32(device, 0x100c80);
- val &= 0xf000087f;
+ val &= 0xf000187f;
nvkm_wr32(device, 0x418880, val);
nvkm_wr32(device, 0x418890, 0);
nvkm_wr32(device, 0x418894, 0);
@@ -66,7 +66,9 @@
gm20b_gr = {
.init = gk20a_gr_init,
.init_gpc_mmu = gm20b_gr_init_gpc_mmu,
+ .init_rop_active_fbps = gk104_gr_init_rop_active_fbps,
.set_hww_esr_report_mask = gm20b_gr_set_hww_esr_report_mask,
+ .rops = gm200_gr_rops,
.ppc_nr = 1,
.grctx = &gm20b_grctx,
.sclass = {
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv04.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv04.c
index 85c5b7f..9c2e985 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv04.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv04.c
@@ -1422,6 +1422,5 @@
spin_lock_init(&gr->lock);
*pgr = &gr->base;
- return nvkm_gr_ctor(&nv04_gr, device, index, 0x00001000,
- true, &gr->base);
+ return nvkm_gr_ctor(&nv04_gr, device, index, true, &gr->base);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv10.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv10.c
index 4542867..4ebbfbd 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv10.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv10.c
@@ -1182,7 +1182,7 @@
spin_lock_init(&gr->lock);
*pgr = &gr->base;
- return nvkm_gr_ctor(func, device, index, 0x00001000, true, &gr->base);
+ return nvkm_gr_ctor(func, device, index, true, &gr->base);
}
static const struct nvkm_gr_func
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv20.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv20.c
index 5caef65..d1dc929 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv20.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv20.c
@@ -337,7 +337,7 @@
return -ENOMEM;
*pgr = &gr->base;
- return nvkm_gr_ctor(func, device, index, 0x00001000, true, &gr->base);
+ return nvkm_gr_ctor(func, device, index, true, &gr->base);
}
static const struct nvkm_gr_func
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv40.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv40.c
index 05a8954..5f1ad83 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv40.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv40.c
@@ -438,7 +438,7 @@
*pgr = &gr->base;
INIT_LIST_HEAD(&gr->chan);
- return nvkm_gr_ctor(func, device, index, 0x00001000, true, &gr->base);
+ return nvkm_gr_ctor(func, device, index, true, &gr->base);
}
static const struct nvkm_gr_func
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv50.c b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv50.c
index b19b912..fca67de 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv50.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/nv50.c
@@ -768,7 +768,7 @@
spin_lock_init(&gr->lock);
*pgr = &gr->base;
- return nvkm_gr_ctor(func, device, index, 0x00201000, true, &gr->base);
+ return nvkm_gr_ctor(func, device, index, true, &gr->base);
}
static const struct nvkm_gr_func
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/gr/priv.h b/drivers/gpu/drm/nouveau/nvkm/engine/gr/priv.h
index a234590..d8adcdf 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/gr/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/gr/priv.h
@@ -7,8 +7,7 @@
struct nvkm_fifo_chan;
int nvkm_gr_ctor(const struct nvkm_gr_func *, struct nvkm_device *,
- int index, u32 pmc_enable, bool enable,
- struct nvkm_gr *);
+ int index, bool enable, struct nvkm_gr *);
bool nv04_gr_idle(struct nvkm_gr *);
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/g84.c b/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/g84.c
index 34ff001..c0e11a0 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/g84.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/g84.c
@@ -39,6 +39,5 @@
int
g84_mpeg_new(struct nvkm_device *device, int index, struct nvkm_engine **pmpeg)
{
- return nvkm_engine_new_(&g84_mpeg, device, index, 0x00000002,
- true, pmpeg);
+ return nvkm_engine_new_(&g84_mpeg, device, index, true, pmpeg);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/nv31.c b/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/nv31.c
index d4d8942..003ac91 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/nv31.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/nv31.c
@@ -278,7 +278,7 @@
mpeg->func = func;
*pmpeg = &mpeg->engine;
- return nvkm_engine_ctor(&nv31_mpeg_, device, index, 0x00000002,
+ return nvkm_engine_ctor(&nv31_mpeg_, device, index,
true, &mpeg->engine);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/nv44.c b/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/nv44.c
index d433cfa..e536f37 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/nv44.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/nv44.c
@@ -212,6 +212,5 @@
INIT_LIST_HEAD(&mpeg->chan);
*pmpeg = &mpeg->engine;
- return nvkm_engine_ctor(&nv44_mpeg, device, index, 0x00000002,
- true, &mpeg->engine);
+ return nvkm_engine_ctor(&nv44_mpeg, device, index, true, &mpeg->engine);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/nv50.c b/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/nv50.c
index c3a85df..4e52885 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/nv50.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/mpeg/nv50.c
@@ -130,6 +130,5 @@
int
nv50_mpeg_new(struct nvkm_device *device, int index, struct nvkm_engine **pmpeg)
{
- return nvkm_engine_new_(&nv50_mpeg, device, index, 0x00400002,
- true, pmpeg);
+ return nvkm_engine_new_(&nv50_mpeg, device, index, true, pmpeg);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/mspdec/g98.c b/drivers/gpu/drm/nouveau/nvkm/engine/mspdec/g98.c
index 1f1a99e..f30cf1d 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/mspdec/g98.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/mspdec/g98.c
@@ -35,7 +35,6 @@
static const struct nvkm_falcon_func
g98_mspdec = {
- .pmc_enable = 0x01020000,
.init = g98_mspdec_init,
.sclass = {
{ -1, -1, G98_MSPDEC },
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/mspdec/gf100.c b/drivers/gpu/drm/nouveau/nvkm/engine/mspdec/gf100.c
index 371fd6c..cfe1aa8 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/mspdec/gf100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/mspdec/gf100.c
@@ -35,7 +35,6 @@
static const struct nvkm_falcon_func
gf100_mspdec = {
- .pmc_enable = 0x00020000,
.init = gf100_mspdec_init,
.sclass = {
{ -1, -1, GF100_MSPDEC },
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/mspdec/gk104.c b/drivers/gpu/drm/nouveau/nvkm/engine/mspdec/gk104.c
index de804a1..24272b4 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/mspdec/gk104.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/mspdec/gk104.c
@@ -27,7 +27,6 @@
static const struct nvkm_falcon_func
gk104_mspdec = {
- .pmc_enable = 0x00020000,
.init = gf100_mspdec_init,
.sclass = {
{ -1, -1, GK104_MSPDEC },
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/mspdec/gt215.c b/drivers/gpu/drm/nouveau/nvkm/engine/mspdec/gt215.c
index 8356317..cf6e59a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/mspdec/gt215.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/mspdec/gt215.c
@@ -27,7 +27,6 @@
static const struct nvkm_falcon_func
gt215_mspdec = {
- .pmc_enable = 0x01020000,
.init = g98_mspdec_init,
.sclass = {
{ -1, -1, GT212_MSPDEC },
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/msppp/g98.c b/drivers/gpu/drm/nouveau/nvkm/engine/msppp/g98.c
index 73f633a..c45dbf7 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/msppp/g98.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/msppp/g98.c
@@ -35,7 +35,6 @@
static const struct nvkm_falcon_func
g98_msppp = {
- .pmc_enable = 0x00400002,
.init = g98_msppp_init,
.sclass = {
{ -1, -1, G98_MSPPP },
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/msppp/gf100.c b/drivers/gpu/drm/nouveau/nvkm/engine/msppp/gf100.c
index c42c0c0..803c62a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/msppp/gf100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/msppp/gf100.c
@@ -35,7 +35,6 @@
static const struct nvkm_falcon_func
gf100_msppp = {
- .pmc_enable = 0x00000002,
.init = gf100_msppp_init,
.sclass = {
{ -1, -1, GF100_MSPPP },
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/msppp/gt215.c b/drivers/gpu/drm/nouveau/nvkm/engine/msppp/gt215.c
index 00e7795..49cbf72 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/msppp/gt215.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/msppp/gt215.c
@@ -27,7 +27,6 @@
static const struct nvkm_falcon_func
gt215_msppp = {
- .pmc_enable = 0x00400002,
.init = g98_msppp_init,
.sclass = {
{ -1, -1, GT212_MSPPP },
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/msvld/g98.c b/drivers/gpu/drm/nouveau/nvkm/engine/msvld/g98.c
index 47e2929..4a2a9f0 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/msvld/g98.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/msvld/g98.c
@@ -35,7 +35,6 @@
static const struct nvkm_falcon_func
g98_msvld = {
- .pmc_enable = 0x04008000,
.init = g98_msvld_init,
.sclass = {
{ -1, -1, G98_MSVLD },
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/msvld/gf100.c b/drivers/gpu/drm/nouveau/nvkm/engine/msvld/gf100.c
index 1ac581b..1695e53 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/msvld/gf100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/msvld/gf100.c
@@ -35,7 +35,6 @@
static const struct nvkm_falcon_func
gf100_msvld = {
- .pmc_enable = 0x00008000,
.init = gf100_msvld_init,
.sclass = {
{ -1, -1, GF100_MSVLD },
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/msvld/gk104.c b/drivers/gpu/drm/nouveau/nvkm/engine/msvld/gk104.c
index 4bba16e..b640cd6 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/msvld/gk104.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/msvld/gk104.c
@@ -27,7 +27,6 @@
static const struct nvkm_falcon_func
gk104_msvld = {
- .pmc_enable = 0x00008000,
.init = gf100_msvld_init,
.sclass = {
{ -1, -1, GK104_MSVLD },
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/msvld/gt215.c b/drivers/gpu/drm/nouveau/nvkm/engine/msvld/gt215.c
index e17cb56..201e8ef 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/msvld/gt215.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/msvld/gt215.c
@@ -27,7 +27,6 @@
static const struct nvkm_falcon_func
gt215_msvld = {
- .pmc_enable = 0x04008000,
.init = g98_msvld_init,
.sclass = {
{ -1, -1, GT212_MSVLD },
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/msvld/mcp89.c b/drivers/gpu/drm/nouveau/nvkm/engine/msvld/mcp89.c
index 511800f..a0f540e 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/msvld/mcp89.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/msvld/mcp89.c
@@ -27,7 +27,6 @@
static const struct nvkm_falcon_func
mcp89_msvld = {
- .pmc_enable = 0x04008000,
.init = g98_msvld_init,
.sclass = {
{ -1, -1, IGT21A_MSVLD },
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/pm/base.c b/drivers/gpu/drm/nouveau/nvkm/engine/pm/base.c
index f19fabe..8616636 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/pm/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/pm/base.c
@@ -863,5 +863,5 @@
pm->func = func;
INIT_LIST_HEAD(&pm->domains);
INIT_LIST_HEAD(&pm->sources);
- return nvkm_engine_ctor(&nvkm_pm, device, index, 0, true, &pm->engine);
+ return nvkm_engine_ctor(&nvkm_pm, device, index, true, &pm->engine);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/sec/g98.c b/drivers/gpu/drm/nouveau/nvkm/engine/sec/g98.c
index 995c2c5..6d2a7f0 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/sec/g98.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/sec/g98.c
@@ -66,7 +66,6 @@
.code.size = sizeof(g98_sec_code),
.data.data = g98_sec_data,
.data.size = sizeof(g98_sec_data),
- .pmc_enable = 0x00004000,
.intr = g98_sec_intr,
.sclass = {
{ -1, -1, G98_SEC },
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/sw/base.c b/drivers/gpu/drm/nouveau/nvkm/engine/sw/base.c
index 53c1f7e..7be3198 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/sw/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/sw/base.c
@@ -106,5 +106,5 @@
INIT_LIST_HEAD(&sw->chan);
sw->func = func;
- return nvkm_engine_ctor(&nvkm_sw, device, index, 0, true, &sw->engine);
+ return nvkm_engine_ctor(&nvkm_sw, device, index, true, &sw->engine);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/vp/g84.c b/drivers/gpu/drm/nouveau/nvkm/engine/vp/g84.c
index 4188c77..7a96178 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/vp/g84.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/vp/g84.c
@@ -27,7 +27,6 @@
static const struct nvkm_xtensa_func
g84_vp = {
- .pmc_enable = 0x01020000,
.fifo_val = 0x111,
.unkd28 = 0x9c544,
.sclass = {
diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/xtensa.c b/drivers/gpu/drm/nouveau/nvkm/engine/xtensa.c
index a3d4f5b..06bdb67 100644
--- a/drivers/gpu/drm/nouveau/nvkm/engine/xtensa.c
+++ b/drivers/gpu/drm/nouveau/nvkm/engine/xtensa.c
@@ -187,6 +187,6 @@
xtensa->addr = addr;
*pengine = &xtensa->engine;
- return nvkm_engine_ctor(&nvkm_xtensa, device, index, func->pmc_enable,
+ return nvkm_engine_ctor(&nvkm_xtensa, device, index,
enable, &xtensa->engine);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/Kbuild b/drivers/gpu/drm/nouveau/nvkm/subdev/Kbuild
index 642d27d..3f5d38d 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/Kbuild
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/Kbuild
@@ -19,4 +19,5 @@
include $(src)/nvkm/subdev/secboot/Kbuild
include $(src)/nvkm/subdev/therm/Kbuild
include $(src)/nvkm/subdev/timer/Kbuild
+include $(src)/nvkm/subdev/top/Kbuild
include $(src)/nvkm/subdev/volt/Kbuild
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/bar/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/bar/base.c
index a9433ad..c561d148 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/bar/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/bar/base.c
@@ -77,7 +77,7 @@
nvkm_bar_ctor(const struct nvkm_bar_func *func, struct nvkm_device *device,
int index, struct nvkm_bar *bar)
{
- nvkm_subdev_ctor(&nvkm_bar, device, index, 0, &bar->subdev);
+ nvkm_subdev_ctor(&nvkm_bar, device, index, &bar->subdev);
bar->func = func;
spin_lock_init(&bar->lock);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/bios/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/bios/base.c
index 7953689..e15b962 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/bios/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/bios/base.c
@@ -105,7 +105,7 @@
if (!(bios = *pbios = kzalloc(sizeof(*bios), GFP_KERNEL)))
return -ENOMEM;
- nvkm_subdev_ctor(&nvkm_bios, device, index, 0, &bios->subdev);
+ nvkm_subdev_ctor(&nvkm_bios, device, index, &bios->subdev);
ret = nvbios_shadow(bios);
if (ret)
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/bios/pll.c b/drivers/gpu/drm/nouveau/nvkm/subdev/bios/pll.c
index 125ec2e..91a7dc5 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/bios/pll.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/bios/pll.c
@@ -81,9 +81,11 @@
pll_limits_table(struct nvkm_bios *bios, u8 *ver, u8 *hdr, u8 *cnt, u8 *len)
{
struct bit_entry bit_C;
+ u16 data = 0x0000;
- if (!bit_entry(bios, 'C', &bit_C) && bit_C.length >= 10) {
- u16 data = nvbios_rd16(bios, bit_C.offset + 8);
+ if (!bit_entry(bios, 'C', &bit_C)) {
+ if (bit_C.version == 1 && bit_C.length >= 10)
+ data = nvbios_rd16(bios, bit_C.offset + 8);
if (data) {
*ver = nvbios_rd08(bios, data + 0);
*hdr = nvbios_rd08(bios, data + 1);
@@ -94,7 +96,7 @@
}
if (bmp_version(bios) >= 0x0524) {
- u16 data = nvbios_rd16(bios, bios->bmp_offset + 142);
+ data = nvbios_rd16(bios, bios->bmp_offset + 142);
if (data) {
*ver = nvbios_rd08(bios, data + 0);
*hdr = 1;
@@ -105,7 +107,7 @@
}
*ver = 0x00;
- return 0x0000;
+ return data;
}
static struct pll_mapping *
@@ -156,7 +158,7 @@
}
map = pll_map(bios);
- while (map->reg) {
+ while (map && map->reg) {
if (map->reg == reg && *ver >= 0x20) {
u16 addr = (data += hdr);
*type = map->type;
@@ -198,7 +200,7 @@
}
map = pll_map(bios);
- while (map->reg) {
+ while (map && map->reg) {
if (map->type == type && *ver >= 0x20) {
u16 addr = (data += hdr);
*reg = map->reg;
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/bus/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/bus/base.c
index dc5a10f..52ad73b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/bus/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/bus/base.c
@@ -58,7 +58,7 @@
struct nvkm_bus *bus;
if (!(bus = *pbus = kzalloc(sizeof(*bus), GFP_KERNEL)))
return -ENOMEM;
- nvkm_subdev_ctor(&nvkm_bus, device, index, 0, &bus->subdev);
+ nvkm_subdev_ctor(&nvkm_bus, device, index, &bus->subdev);
bus->func = func;
return 0;
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/clk/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/clk/base.c
index 889cce2..7102c25 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/clk/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/clk/base.c
@@ -564,7 +564,7 @@
int ret, idx, arglen;
const char *mode;
- nvkm_subdev_ctor(&nvkm_clk, device, index, 0, &clk->subdev);
+ nvkm_subdev_ctor(&nvkm_clk, device, index, &clk->subdev);
clk->func = func;
INIT_LIST_HEAD(&clk->states);
clk->domains = func->domains;
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/base.c
index 5f25402..4756019 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/base.c
@@ -83,6 +83,12 @@
if (init->func->preinit)
init->func->preinit(init);
+ /* Override the post flag during the first call if NvForcePost is set */
+ if (init->force_post) {
+ init->post = init->force_post;
+ init->force_post = false;
+ }
+
/* unlock the extended vga crtc regs */
nvkm_lockvgac(subdev->device, false);
return 0;
@@ -124,7 +130,7 @@
struct nvkm_device *device, int index,
struct nvkm_devinit *init)
{
- nvkm_subdev_ctor(&nvkm_devinit, device, index, 0, &init->subdev);
+ nvkm_subdev_ctor(&nvkm_devinit, device, index, &init->subdev);
init->func = func;
- init->post = nvkm_boolopt(device->cfgopt, "NvForcePost", false);
+ init->force_post = nvkm_boolopt(device->cfgopt, "NvForcePost", false);
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c
index 2923598..8b1b34c 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.c
@@ -97,9 +97,11 @@
struct nvkm_subdev *subdev = &init->base.subdev;
struct nvkm_device *device = subdev->device;
- /* This bit is set by devinit, and flips back to 0 on suspend */
- if (!base->post)
- base->post = ((nvkm_rd32(device, 0x2240c) & BIT(1)) == 0);
+ /*
+ * This bit is set by devinit, and flips back to 0 on suspend. We
+ * can use it as a reliable way to know whether we should run devinit.
+ */
+ base->post = ((nvkm_rd32(device, 0x2240c) & BIT(1)) == 0);
}
static const struct nvkm_devinit_func
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/Kbuild b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/Kbuild
index 0810570..842d5de 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/Kbuild
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/Kbuild
@@ -23,6 +23,7 @@
nvkm-y += nvkm/subdev/fb/gk104.o
nvkm-y += nvkm/subdev/fb/gk20a.o
nvkm-y += nvkm/subdev/fb/gm107.o
+nvkm-y += nvkm/subdev/fb/gm200.o
nvkm-y += nvkm/subdev/fb/ram.o
nvkm-y += nvkm/subdev/fb/ramnv04.o
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/base.c
index a719b9b..ce90242b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/base.c
@@ -24,6 +24,7 @@
#include "priv.h"
#include "ram.h"
+#include <core/memory.h>
#include <subdev/bios.h>
#include <subdev/bios/M0203.h>
#include <engine/gr.h>
@@ -98,6 +99,7 @@
nvkm_fb_oneinit(struct nvkm_subdev *subdev)
{
struct nvkm_fb *fb = nvkm_fb(subdev);
+
if (fb->func->ram_new) {
int ret = fb->func->ram_new(fb, &fb->ram);
if (ret) {
@@ -105,6 +107,13 @@
return ret;
}
}
+
+ if (fb->func->oneinit) {
+ int ret = fb->func->oneinit(fb);
+ if (ret)
+ return ret;
+ }
+
return 0;
}
@@ -134,6 +143,9 @@
struct nvkm_fb *fb = nvkm_fb(subdev);
int i;
+ nvkm_memory_del(&fb->mmu_wr);
+ nvkm_memory_del(&fb->mmu_rd);
+
for (i = 0; i < fb->tile.regions; i++)
fb->func->tile.fini(fb, i, &fb->tile.region[i]);
@@ -156,7 +168,7 @@
nvkm_fb_ctor(const struct nvkm_fb_func *func, struct nvkm_device *device,
int index, struct nvkm_fb *fb)
{
- nvkm_subdev_ctor(&nvkm_fb, device, index, 0, &fb->subdev);
+ nvkm_subdev_ctor(&nvkm_fb, device, index, &fb->subdev);
fb->func = func;
fb->tile.regions = fb->func->tile.regions;
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gf100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gf100.c
index 008bb98..e649ead 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gf100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gf100.c
@@ -24,6 +24,9 @@
#include "gf100.h"
#include "ram.h"
+#include <core/memory.h>
+#include <core/option.h>
+
extern const u8 gf100_pte_storage_type_map[256];
bool
@@ -46,6 +49,28 @@
nvkm_debug(subdev, "PBFB intr\n");
}
+int
+gf100_fb_oneinit(struct nvkm_fb *fb)
+{
+ struct nvkm_device *device = fb->subdev.device;
+ int ret, size = 0x1000;
+
+ size = nvkm_longopt(device->cfgopt, "MmuDebugBufferSize", size);
+ size = min(size, 0x1000);
+
+ ret = nvkm_memory_new(device, NVKM_MEM_TARGET_INST, size, 0x1000,
+ false, &fb->mmu_rd);
+ if (ret)
+ return ret;
+
+ ret = nvkm_memory_new(device, NVKM_MEM_TARGET_INST, size, 0x1000,
+ false, &fb->mmu_wr);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
void
gf100_fb_init(struct nvkm_fb *base)
{
@@ -98,6 +123,7 @@
static const struct nvkm_fb_func
gf100_fb = {
.dtor = gf100_fb_dtor,
+ .oneinit = gf100_fb_oneinit,
.init = gf100_fb_init,
.intr = gf100_fb_intr,
.ram_new = gf100_ram_new,
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk104.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk104.c
index 0edb3c3..b41f0f7 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk104.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk104.c
@@ -27,6 +27,7 @@
static const struct nvkm_fb_func
gk104_fb = {
.dtor = gf100_fb_dtor,
+ .oneinit = gf100_fb_oneinit,
.init = gf100_fb_init,
.intr = gf100_fb_intr,
.ram_new = gk104_ram_new,
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk20a.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk20a.c
index 81447eb..7306f7d 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk20a.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk20a.c
@@ -21,15 +21,20 @@
*/
#include "priv.h"
+#include <core/memory.h>
+
static void
gk20a_fb_init(struct nvkm_fb *fb)
{
struct nvkm_device *device = fb->subdev.device;
nvkm_mask(device, 0x100c80, 0x00000001, 0x00000000); /* 128KiB lpg */
+ nvkm_wr32(device, 0x100cc8, nvkm_memory_addr(fb->mmu_wr) >> 8);
+ nvkm_wr32(device, 0x100ccc, nvkm_memory_addr(fb->mmu_rd) >> 8);
}
static const struct nvkm_fb_func
gk20a_fb = {
+ .oneinit = gf100_fb_oneinit,
.init = gk20a_fb_init,
.memtype_valid = gf100_fb_memtype_valid,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gm107.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gm107.c
index 2a91df8..4869fdb 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gm107.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gm107.c
@@ -27,6 +27,7 @@
static const struct nvkm_fb_func
gm107_fb = {
.dtor = gf100_fb_dtor,
+ .oneinit = gf100_fb_oneinit,
.init = gf100_fb_init,
.intr = gf100_fb_intr,
.ram_new = gm107_ram_new,
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gm200.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gm200.c
new file mode 100644
index 0000000..44f5716
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/gm200.c
@@ -0,0 +1,60 @@
+/*
+ * Copyright 2012 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: Ben Skeggs
+ */
+#include "gf100.h"
+#include "ram.h"
+
+#include <core/memory.h>
+
+static void
+gm200_fb_init(struct nvkm_fb *base)
+{
+ struct gf100_fb *fb = gf100_fb(base);
+ struct nvkm_device *device = fb->base.subdev.device;
+
+ if (fb->r100c10_page)
+ nvkm_wr32(device, 0x100c10, fb->r100c10 >> 8);
+
+ nvkm_mask(device, 0x100c80, 0x00000001, 0x00000000); /* 128KiB lpg */
+
+ nvkm_wr32(device, 0x100cc8, nvkm_memory_addr(fb->base.mmu_wr) >> 8);
+ nvkm_wr32(device, 0x100ccc, nvkm_memory_addr(fb->base.mmu_rd) >> 8);
+ nvkm_mask(device, 0x100cc4, 0x00060000,
+ min(nvkm_memory_size(fb->base.mmu_rd) >> 16, (u64)2) << 17);
+}
+
+static const struct nvkm_fb_func
+gm200_fb = {
+ .dtor = gf100_fb_dtor,
+ .oneinit = gf100_fb_oneinit,
+ .init = gm200_fb_init,
+ .intr = gf100_fb_intr,
+ .ram_new = gm107_ram_new,
+ .memtype_valid = gf100_fb_memtype_valid,
+};
+
+int
+gm200_fb_new(struct nvkm_device *device, int index, struct nvkm_fb **pfb)
+{
+ return gf100_fb_new_(&gm200_fb, device, index, pfb);
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/priv.h b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/priv.h
index 62b9feb5..d97d640 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fb/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fb/priv.h
@@ -6,6 +6,7 @@
struct nvkm_fb_func {
void *(*dtor)(struct nvkm_fb *);
+ int (*oneinit)(struct nvkm_fb *);
void (*init)(struct nvkm_fb *);
void (*intr)(struct nvkm_fb *);
@@ -58,5 +59,6 @@
void nv46_fb_tile_init(struct nvkm_fb *, int i, u32 addr, u32 size,
u32 pitch, u32 flags, struct nvkm_fb_tile *);
+int gf100_fb_oneinit(struct nvkm_fb *);
bool gf100_fb_memtype_valid(struct nvkm_fb *, u32);
#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/fuse/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/fuse/base.c
index f414497..1c3c18e 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/fuse/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/fuse/base.c
@@ -47,7 +47,7 @@
struct nvkm_fuse *fuse;
if (!(fuse = *pfuse = kzalloc(sizeof(*fuse), GFP_KERNEL)))
return -ENOMEM;
- nvkm_subdev_ctor(&nvkm_fuse, device, index, 0, &fuse->subdev);
+ nvkm_subdev_ctor(&nvkm_fuse, device, index, &fuse->subdev);
fuse->func = func;
spin_lock_init(&fuse->lock);
return 0;
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/gpio/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/gpio/base.c
index d45ec99..77c64972 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/gpio/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/gpio/base.c
@@ -216,7 +216,7 @@
if (!(gpio = *pgpio = kzalloc(sizeof(*gpio), GFP_KERNEL)))
return -ENOMEM;
- nvkm_subdev_ctor(&nvkm_gpio, device, index, 0, &gpio->subdev);
+ nvkm_subdev_ctor(&nvkm_gpio, device, index, &gpio->subdev);
gpio->func = func;
return nvkm_event_init(&nvkm_gpio_intr_func, 2, func->lines,
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/i2c/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/i2c/base.c
index 243a71f..4f197b1 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/i2c/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/i2c/base.c
@@ -254,7 +254,7 @@
if (!(i2c = *pi2c = kzalloc(sizeof(*i2c), GFP_KERNEL)))
return -ENOMEM;
- nvkm_subdev_ctor(&nvkm_i2c, device, index, 0, &i2c->subdev);
+ nvkm_subdev_ctor(&nvkm_i2c, device, index, &i2c->subdev);
i2c->func = func;
INIT_LIST_HEAD(&i2c->pad);
INIT_LIST_HEAD(&i2c->bus);
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/ibus/gf100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/ibus/gf100.c
index 72d6330..2c6b374 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/ibus/gf100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/ibus/gf100.c
@@ -117,6 +117,6 @@
struct nvkm_subdev *ibus;
if (!(ibus = *pibus = kzalloc(sizeof(*ibus), GFP_KERNEL)))
return -ENOMEM;
- nvkm_subdev_ctor(&gf100_ibus, device, index, 0, ibus);
+ nvkm_subdev_ctor(&gf100_ibus, device, index, ibus);
return 0;
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/ibus/gf117.c b/drivers/gpu/drm/nouveau/nvkm/subdev/ibus/gf117.c
index f69f263..3905a80 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/ibus/gf117.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/ibus/gf117.c
@@ -46,6 +46,6 @@
struct nvkm_subdev *ibus;
if (!(ibus = *pibus = kzalloc(sizeof(*ibus), GFP_KERNEL)))
return -ENOMEM;
- nvkm_subdev_ctor(&gf117_ibus, device, index, 0, ibus);
+ nvkm_subdev_ctor(&gf117_ibus, device, index, ibus);
return 0;
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/ibus/gk104.c b/drivers/gpu/drm/nouveau/nvkm/subdev/ibus/gk104.c
index b5cee3f..c673853 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/ibus/gk104.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/ibus/gk104.c
@@ -120,6 +120,6 @@
struct nvkm_subdev *ibus;
if (!(ibus = *pibus = kzalloc(sizeof(*ibus), GFP_KERNEL)))
return -ENOMEM;
- nvkm_subdev_ctor(&gk104_ibus, device, index, 0, ibus);
+ nvkm_subdev_ctor(&gk104_ibus, device, index, ibus);
return 0;
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/ibus/gk20a.c b/drivers/gpu/drm/nouveau/nvkm/subdev/ibus/gk20a.c
index 3484079..b7159b3 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/ibus/gk20a.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/ibus/gk20a.c
@@ -84,6 +84,6 @@
struct nvkm_subdev *ibus;
if (!(ibus = *pibus = kzalloc(sizeof(*ibus), GFP_KERNEL)))
return -ENOMEM;
- nvkm_subdev_ctor(&gk20a_ibus, device, index, 0, ibus);
+ nvkm_subdev_ctor(&gk20a_ibus, device, index, ibus);
return 0;
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/ibus/gm200.c b/drivers/gpu/drm/nouveau/nvkm/subdev/ibus/gm200.c
index ef0b7f3..c633281 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/ibus/gm200.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/ibus/gm200.c
@@ -35,6 +35,6 @@
struct nvkm_subdev *ibus;
if (!(ibus = *pibus = kzalloc(sizeof(*ibus), GFP_KERNEL)))
return -ENOMEM;
- nvkm_subdev_ctor(&gm200_ibus, device, index, 0, ibus);
+ nvkm_subdev_ctor(&gm200_ibus, device, index, ibus);
return 0;
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/iccsense/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/iccsense/base.c
index c44a852..323c79a 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/iccsense/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/iccsense/base.c
@@ -30,15 +30,14 @@
static bool
nvkm_iccsense_validate_device(struct i2c_adapter *i2c, u8 addr,
- enum nvbios_extdev_type type, u8 rail)
+ enum nvbios_extdev_type type)
{
switch (type) {
case NVBIOS_EXTDEV_INA209:
case NVBIOS_EXTDEV_INA219:
- return rail == 0 && nv_rd16i2cr(i2c, addr, 0x0) >= 0;
+ return nv_rd16i2cr(i2c, addr, 0x0) >= 0;
case NVBIOS_EXTDEV_INA3221:
- return rail <= 3 &&
- nv_rd16i2cr(i2c, addr, 0xff) == 0x3220 &&
+ return nv_rd16i2cr(i2c, addr, 0xff) == 0x3220 &&
nv_rd16i2cr(i2c, addr, 0xfe) == 0x5449;
default:
return false;
@@ -67,8 +66,9 @@
struct nvkm_iccsense_rail *rail,
u8 shunt_reg, u8 bus_reg)
{
- return nvkm_iccsense_poll_lane(rail->i2c, rail->addr, shunt_reg, 0,
- bus_reg, 3, rail->mohm, 10 * 4);
+ return nvkm_iccsense_poll_lane(rail->sensor->i2c, rail->sensor->addr,
+ shunt_reg, 0, bus_reg, 3, rail->mohm,
+ 10 * 4);
}
static int
@@ -89,37 +89,87 @@
nvkm_iccsense_ina3221_read(struct nvkm_iccsense *iccsense,
struct nvkm_iccsense_rail *rail)
{
- return nvkm_iccsense_poll_lane(rail->i2c, rail->addr,
- 1 + (rail->rail * 2), 3,
- 2 + (rail->rail * 2), 3, rail->mohm,
+ return nvkm_iccsense_poll_lane(rail->sensor->i2c, rail->sensor->addr,
+ 1 + (rail->idx * 2), 3,
+ 2 + (rail->idx * 2), 3, rail->mohm,
40 * 8);
}
-int
-nvkm_iccsense_read(struct nvkm_iccsense *iccsense, u8 idx)
+static void
+nvkm_iccsense_ina209_config(struct nvkm_iccsense *iccsense,
+ struct nvkm_iccsense_sensor *sensor)
{
- struct nvkm_iccsense_rail *rail;
+ struct nvkm_subdev *subdev = &iccsense->subdev;
+ /* configuration:
+ * 0x0007: 0x0007 shunt and bus continous
+ * 0x0078: 0x0078 128 samples shunt
+ * 0x0780: 0x0780 128 samples bus
+ * 0x1800: 0x0000 +-40 mV shunt range
+ * 0x2000: 0x0000 16V FSR
+ */
+ u16 value = 0x07ff;
+ nvkm_debug(subdev, "config for sensor id %i: 0x%x\n", sensor->id, value);
+ nv_wr16i2cr(sensor->i2c, sensor->addr, 0x00, value);
+}
- if (!iccsense || idx >= iccsense->rail_count)
- return -EINVAL;
+static void
+nvkm_iccsense_ina3221_config(struct nvkm_iccsense *iccsense,
+ struct nvkm_iccsense_sensor *sensor)
+{
+ struct nvkm_subdev *subdev = &iccsense->subdev;
+ /* configuration:
+ * 0x0007: 0x0007 shunt and bus continous
+ * 0x0031: 0x0000 140 us conversion time shunt
+ * 0x01c0: 0x0000 140 us conversion time bus
+ * 0x0f00: 0x0f00 1024 samples
+ * 0x7000: 0x?000 channels
+ */
+ u16 value = 0x0e07;
+ if (sensor->rail_mask & 0x1)
+ value |= 0x1 << 14;
+ if (sensor->rail_mask & 0x2)
+ value |= 0x1 << 13;
+ if (sensor->rail_mask & 0x4)
+ value |= 0x1 << 12;
+ nvkm_debug(subdev, "config for sensor id %i: 0x%x\n", sensor->id, value);
+ nv_wr16i2cr(sensor->i2c, sensor->addr, 0x00, value);
+}
- rail = &iccsense->rails[idx];
- if (!rail->read)
- return -ENODEV;
-
- return rail->read(iccsense, rail);
+static void
+nvkm_iccsense_sensor_config(struct nvkm_iccsense *iccsense,
+ struct nvkm_iccsense_sensor *sensor)
+{
+ switch (sensor->type) {
+ case NVBIOS_EXTDEV_INA209:
+ case NVBIOS_EXTDEV_INA219:
+ nvkm_iccsense_ina209_config(iccsense, sensor);
+ break;
+ case NVBIOS_EXTDEV_INA3221:
+ nvkm_iccsense_ina3221_config(iccsense, sensor);
+ break;
+ default:
+ break;
+ }
}
int
nvkm_iccsense_read_all(struct nvkm_iccsense *iccsense)
{
- int result = 0, i;
- for (i = 0; i < iccsense->rail_count; ++i) {
- int res = nvkm_iccsense_read(iccsense, i);
- if (res >= 0)
- result += res;
- else
+ int result = 0;
+ struct nvkm_iccsense_rail *rail;
+
+ if (!iccsense)
+ return -EINVAL;
+
+ list_for_each_entry(rail, &iccsense->rails, head) {
+ int res;
+ if (!rail->read)
+ return -ENODEV;
+
+ res = rail->read(iccsense, rail);
+ if (res < 0)
return res;
+ result += res;
}
return result;
}
@@ -128,89 +178,158 @@
nvkm_iccsense_dtor(struct nvkm_subdev *subdev)
{
struct nvkm_iccsense *iccsense = nvkm_iccsense(subdev);
+ struct nvkm_iccsense_sensor *sensor, *tmps;
+ struct nvkm_iccsense_rail *rail, *tmpr;
- if (iccsense->rails)
- kfree(iccsense->rails);
+ list_for_each_entry_safe(sensor, tmps, &iccsense->sensors, head) {
+ list_del(&sensor->head);
+ kfree(sensor);
+ }
+ list_for_each_entry_safe(rail, tmpr, &iccsense->rails, head) {
+ list_del(&rail->head);
+ kfree(rail);
+ }
return iccsense;
}
+static struct nvkm_iccsense_sensor*
+nvkm_iccsense_create_sensor(struct nvkm_iccsense *iccsense, u8 id)
+{
+
+ struct nvkm_subdev *subdev = &iccsense->subdev;
+ struct nvkm_bios *bios = subdev->device->bios;
+ struct nvkm_i2c *i2c = subdev->device->i2c;
+ struct nvbios_extdev_func extdev;
+ struct nvkm_i2c_bus *i2c_bus;
+ struct nvkm_iccsense_sensor *sensor;
+ u8 addr;
+
+ if (!i2c || !bios || nvbios_extdev_parse(bios, id, &extdev))
+ return NULL;
+
+ if (extdev.type == 0xff)
+ return NULL;
+
+ if (extdev.type != NVBIOS_EXTDEV_INA209 &&
+ extdev.type != NVBIOS_EXTDEV_INA219 &&
+ extdev.type != NVBIOS_EXTDEV_INA3221) {
+ iccsense->data_valid = false;
+ nvkm_error(subdev, "Unknown sensor type %x, power reading "
+ "disabled\n", extdev.type);
+ return NULL;
+ }
+
+ if (extdev.bus)
+ i2c_bus = nvkm_i2c_bus_find(i2c, NVKM_I2C_BUS_SEC);
+ else
+ i2c_bus = nvkm_i2c_bus_find(i2c, NVKM_I2C_BUS_PRI);
+ if (!i2c_bus)
+ return NULL;
+
+ addr = extdev.addr >> 1;
+ if (!nvkm_iccsense_validate_device(&i2c_bus->i2c, addr,
+ extdev.type)) {
+ iccsense->data_valid = false;
+ nvkm_warn(subdev, "found invalid sensor id: %i, power reading"
+ "might be invalid\n", id);
+ return NULL;
+ }
+
+ sensor = kmalloc(sizeof(*sensor), GFP_KERNEL);
+ if (!sensor)
+ return NULL;
+
+ list_add_tail(&sensor->head, &iccsense->sensors);
+ sensor->id = id;
+ sensor->type = extdev.type;
+ sensor->i2c = &i2c_bus->i2c;
+ sensor->addr = addr;
+ sensor->rail_mask = 0x0;
+ return sensor;
+}
+
+static struct nvkm_iccsense_sensor*
+nvkm_iccsense_get_sensor(struct nvkm_iccsense *iccsense, u8 id)
+{
+ struct nvkm_iccsense_sensor *sensor;
+ list_for_each_entry(sensor, &iccsense->sensors, head) {
+ if (sensor->id == id)
+ return sensor;
+ }
+ return nvkm_iccsense_create_sensor(iccsense, id);
+}
+
static int
nvkm_iccsense_oneinit(struct nvkm_subdev *subdev)
{
struct nvkm_iccsense *iccsense = nvkm_iccsense(subdev);
struct nvkm_bios *bios = subdev->device->bios;
- struct nvkm_i2c *i2c = subdev->device->i2c;
struct nvbios_iccsense stbl;
int i;
- if (!i2c || !bios || nvbios_iccsense_parse(bios, &stbl)
- || !stbl.nr_entry)
+ if (!bios || nvbios_iccsense_parse(bios, &stbl) || !stbl.nr_entry)
return 0;
- iccsense->rails = kmalloc(sizeof(*iccsense->rails) * stbl.nr_entry,
- GFP_KERNEL);
- if (!iccsense->rails)
- return -ENOMEM;
-
iccsense->data_valid = true;
for (i = 0; i < stbl.nr_entry; ++i) {
struct pwr_rail_t *r = &stbl.rail[i];
- struct nvbios_extdev_func extdev;
struct nvkm_iccsense_rail *rail;
- struct nvkm_i2c_bus *i2c_bus;
- u8 addr;
+ struct nvkm_iccsense_sensor *sensor;
if (!r->mode || r->resistor_mohm == 0)
continue;
- if (nvbios_extdev_parse(bios, r->extdev_id, &extdev))
+ sensor = nvkm_iccsense_get_sensor(iccsense, r->extdev_id);
+ if (!sensor)
continue;
- if (extdev.type == 0xff)
- continue;
+ rail = kmalloc(sizeof(*rail), GFP_KERNEL);
+ if (!rail)
+ return -ENOMEM;
- if (extdev.bus)
- i2c_bus = nvkm_i2c_bus_find(i2c, NVKM_I2C_BUS_SEC);
- else
- i2c_bus = nvkm_i2c_bus_find(i2c, NVKM_I2C_BUS_PRI);
- if (!i2c_bus)
- continue;
-
- addr = extdev.addr >> 1;
- if (!nvkm_iccsense_validate_device(&i2c_bus->i2c, addr,
- extdev.type, r->rail)) {
- iccsense->data_valid = false;
- nvkm_warn(subdev, "found unknown or invalid rail entry"
- " type 0x%x rail %i, power reading might be"
- " invalid\n", extdev.type, r->rail);
- continue;
- }
-
- rail = &iccsense->rails[iccsense->rail_count];
- switch (extdev.type) {
+ switch (sensor->type) {
case NVBIOS_EXTDEV_INA209:
+ if (r->rail != 0)
+ continue;
rail->read = nvkm_iccsense_ina209_read;
break;
case NVBIOS_EXTDEV_INA219:
+ if (r->rail != 0)
+ continue;
rail->read = nvkm_iccsense_ina219_read;
break;
case NVBIOS_EXTDEV_INA3221:
+ if (r->rail >= 3)
+ continue;
rail->read = nvkm_iccsense_ina3221_read;
break;
+ default:
+ continue;
}
- rail->addr = addr;
- rail->rail = r->rail;
+ sensor->rail_mask |= 1 << r->rail;
+ rail->sensor = sensor;
+ rail->idx = r->rail;
rail->mohm = r->resistor_mohm;
- rail->i2c = &i2c_bus->i2c;
- ++iccsense->rail_count;
+ list_add_tail(&rail->head, &iccsense->rails);
}
return 0;
}
+static int
+nvkm_iccsense_init(struct nvkm_subdev *subdev)
+{
+ struct nvkm_iccsense *iccsense = nvkm_iccsense(subdev);
+ struct nvkm_iccsense_sensor *sensor;
+ list_for_each_entry(sensor, &iccsense->sensors, head)
+ nvkm_iccsense_sensor_config(iccsense, sensor);
+ return 0;
+}
+
struct nvkm_subdev_func iccsense_func = {
.oneinit = nvkm_iccsense_oneinit,
+ .init = nvkm_iccsense_init,
.dtor = nvkm_iccsense_dtor,
};
@@ -218,7 +337,7 @@
nvkm_iccsense_ctor(struct nvkm_device *device, int index,
struct nvkm_iccsense *iccsense)
{
- nvkm_subdev_ctor(&iccsense_func, device, index, 0, &iccsense->subdev);
+ nvkm_subdev_ctor(&iccsense_func, device, index, &iccsense->subdev);
}
int
@@ -227,6 +346,8 @@
{
if (!(*iccsense = kzalloc(sizeof(**iccsense), GFP_KERNEL)))
return -ENOMEM;
+ INIT_LIST_HEAD(&(*iccsense)->sensors);
+ INIT_LIST_HEAD(&(*iccsense)->rails);
nvkm_iccsense_ctor(device, index, *iccsense);
return 0;
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/iccsense/priv.h b/drivers/gpu/drm/nouveau/nvkm/subdev/iccsense/priv.h
index ed398b8..b72c31d 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/iccsense/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/iccsense/priv.h
@@ -2,12 +2,22 @@
#define __NVKM_ICCSENSE_PRIV_H__
#define nvkm_iccsense(p) container_of((p), struct nvkm_iccsense, subdev)
#include <subdev/iccsense.h>
+#include <subdev/bios/extdev.h>
-struct nvkm_iccsense_rail {
- int (*read)(struct nvkm_iccsense *, struct nvkm_iccsense_rail *);
+struct nvkm_iccsense_sensor {
+ struct list_head head;
+ int id;
+ enum nvbios_extdev_type type;
struct i2c_adapter *i2c;
u8 addr;
- u8 rail;
+ u8 rail_mask;
+};
+
+struct nvkm_iccsense_rail {
+ struct list_head head;
+ int (*read)(struct nvkm_iccsense *, struct nvkm_iccsense_rail *);
+ struct nvkm_iccsense_sensor *sensor;
+ u8 idx;
u8 mohm;
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/instmem/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/instmem/base.c
index 1d7dd38..8ed8f65 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/instmem/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/instmem/base.c
@@ -311,7 +311,7 @@
struct nvkm_device *device, int index,
struct nvkm_instmem *imem)
{
- nvkm_subdev_ctor(&nvkm_instmem, device, index, 0, &imem->subdev);
+ nvkm_subdev_ctor(&nvkm_instmem, device, index, &imem->subdev);
imem->func = func;
spin_lock_init(&imem->lock);
INIT_LIST_HEAD(&imem->list);
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/base.c
index 85b1464..39c2a38 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/ltc/base.c
@@ -138,7 +138,7 @@
if (!(ltc = *pltc = kzalloc(sizeof(*ltc), GFP_KERNEL)))
return -ENOMEM;
- nvkm_subdev_ctor(&nvkm_ltc, device, index, 0, <c->subdev);
+ nvkm_subdev_ctor(&nvkm_ltc, device, index, <c->subdev);
ltc->func = func;
ltc->zbc_min = 1; /* reserve 0 for disabled */
ltc->zbc_max = min(func->zbc, NVKM_LTC_MAX_ZBC_CNT) - 1;
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/Kbuild b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/Kbuild
index bef325d..49695ac7 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/Kbuild
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/Kbuild
@@ -1,7 +1,12 @@
nvkm-y += nvkm/subdev/mc/base.o
nvkm-y += nvkm/subdev/mc/nv04.o
+nvkm-y += nvkm/subdev/mc/nv11.o
+nvkm-y += nvkm/subdev/mc/nv17.o
nvkm-y += nvkm/subdev/mc/nv44.o
nvkm-y += nvkm/subdev/mc/nv50.o
+nvkm-y += nvkm/subdev/mc/g84.o
nvkm-y += nvkm/subdev/mc/g98.o
+nvkm-y += nvkm/subdev/mc/gt215.o
nvkm-y += nvkm/subdev/mc/gf100.o
+nvkm-y += nvkm/subdev/mc/gk104.o
nvkm-y += nvkm/subdev/mc/gk20a.o
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/base.c
index 954fbbe..350a8ca 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/base.c
@@ -24,6 +24,7 @@
#include "priv.h"
#include <core/option.h>
+#include <subdev/top.h>
void
nvkm_mc_unk260(struct nvkm_mc *mc, u32 data)
@@ -58,10 +59,19 @@
{
struct nvkm_device *device = mc->subdev.device;
struct nvkm_subdev *subdev;
- const struct nvkm_mc_intr *map = mc->func->intr;
- u32 stat, intr;
+ const struct nvkm_mc_map *map = mc->func->intr;
+ u32 stat, intr = nvkm_mc_intr_mask(mc);
+ u64 subdevs;
- stat = intr = nvkm_mc_intr_mask(mc);
+ stat = nvkm_top_intr(device->top, intr, &subdevs);
+ while (subdevs) {
+ enum nvkm_devidx subidx = __ffs64(subdevs);
+ subdev = nvkm_device_subdev(device, subidx);
+ if (subdev)
+ nvkm_subdev_intr(subdev);
+ subdevs &= ~BIT_ULL(subidx);
+ }
+
while (map->stat) {
if (intr & map->stat) {
subdev = nvkm_device_subdev(device, map->unit);
@@ -77,6 +87,36 @@
*handled = intr != 0;
}
+static void
+nvkm_mc_reset_(struct nvkm_mc *mc, enum nvkm_devidx devidx)
+{
+ struct nvkm_device *device = mc->subdev.device;
+ const struct nvkm_mc_map *map;
+ u64 pmc_enable;
+
+ if (!(pmc_enable = nvkm_top_reset(device->top, devidx))) {
+ for (map = mc->func->reset; map && map->stat; map++) {
+ if (map->unit == devidx) {
+ pmc_enable = map->stat;
+ break;
+ }
+ }
+ }
+
+ if (pmc_enable) {
+ nvkm_mask(device, 0x000200, pmc_enable, 0x00000000);
+ nvkm_mask(device, 0x000200, pmc_enable, pmc_enable);
+ nvkm_rd32(device, 0x000200);
+ }
+}
+
+void
+nvkm_mc_reset(struct nvkm_mc *mc, enum nvkm_devidx devidx)
+{
+ if (likely(mc))
+ nvkm_mc_reset_(mc, devidx);
+}
+
static int
nvkm_mc_fini(struct nvkm_subdev *subdev, bool suspend)
{
@@ -117,7 +157,7 @@
if (!(mc = *pmc = kzalloc(sizeof(*mc), GFP_KERNEL)))
return -ENOMEM;
- nvkm_subdev_ctor(&nvkm_mc, device, index, 0, &mc->subdev);
+ nvkm_subdev_ctor(&nvkm_mc, device, index, &mc->subdev);
mc->func = func;
return 0;
}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/g84.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/g84.c
new file mode 100644
index 0000000..5c85b47
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/g84.c
@@ -0,0 +1,68 @@
+/*
+ * Copyright 2012 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: Ben Skeggs
+ */
+#include "priv.h"
+
+static const struct nvkm_mc_map
+g84_mc_reset[] = {
+ { 0x04008000, NVKM_ENGINE_BSP },
+ { 0x02004000, NVKM_ENGINE_CIPHER },
+ { 0x01020000, NVKM_ENGINE_VP },
+ { 0x00400002, NVKM_ENGINE_MPEG },
+ { 0x00201000, NVKM_ENGINE_GR },
+ { 0x00000100, NVKM_ENGINE_FIFO },
+ {}
+};
+
+const struct nvkm_mc_map
+g84_mc_intr[] = {
+ { 0x04000000, NVKM_ENGINE_DISP },
+ { 0x00020000, NVKM_ENGINE_VP },
+ { 0x00008000, NVKM_ENGINE_BSP },
+ { 0x00004000, NVKM_ENGINE_CIPHER },
+ { 0x00001000, NVKM_ENGINE_GR },
+ { 0x00000100, NVKM_ENGINE_FIFO },
+ { 0x00000001, NVKM_ENGINE_MPEG },
+ { 0x0002d101, NVKM_SUBDEV_FB },
+ { 0x10000000, NVKM_SUBDEV_BUS },
+ { 0x00200000, NVKM_SUBDEV_GPIO },
+ { 0x00200000, NVKM_SUBDEV_I2C },
+ { 0x00100000, NVKM_SUBDEV_TIMER },
+ {},
+};
+
+static const struct nvkm_mc_func
+g84_mc = {
+ .init = nv50_mc_init,
+ .intr = g84_mc_intr,
+ .intr_unarm = nv04_mc_intr_unarm,
+ .intr_rearm = nv04_mc_intr_rearm,
+ .intr_mask = nv04_mc_intr_mask,
+ .reset = g84_mc_reset,
+};
+
+int
+g84_mc_new(struct nvkm_device *device, int index, struct nvkm_mc **pmc)
+{
+ return nvkm_mc_new_(&g84_mc, device, index, pmc);
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/g98.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/g98.c
index 7344ad6..0280b43 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/g98.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/g98.c
@@ -23,24 +23,31 @@
*/
#include "priv.h"
-static const struct nvkm_mc_intr
-g98_mc_intr[] = {
- { 0x04000000, NVKM_ENGINE_DISP }, /* DISP first, so pageflip timestamps work */
- { 0x00000001, NVKM_ENGINE_MSPPP },
+static const struct nvkm_mc_map
+g98_mc_reset[] = {
+ { 0x04008000, NVKM_ENGINE_MSVLD },
+ { 0x02004000, NVKM_ENGINE_SEC },
+ { 0x01020000, NVKM_ENGINE_MSPDEC },
+ { 0x00400002, NVKM_ENGINE_MSPPP },
+ { 0x00201000, NVKM_ENGINE_GR },
{ 0x00000100, NVKM_ENGINE_FIFO },
- { 0x00001000, NVKM_ENGINE_GR },
- { 0x00004000, NVKM_ENGINE_SEC }, /* NV84:NVA3 */
- { 0x00008000, NVKM_ENGINE_MSVLD },
+ {}
+};
+
+static const struct nvkm_mc_map
+g98_mc_intr[] = {
+ { 0x04000000, NVKM_ENGINE_DISP },
{ 0x00020000, NVKM_ENGINE_MSPDEC },
- { 0x00040000, NVKM_SUBDEV_PMU }, /* NVA3:NVC0 */
- { 0x00080000, NVKM_SUBDEV_THERM }, /* NVA3:NVC0 */
- { 0x00100000, NVKM_SUBDEV_TIMER },
- { 0x00200000, NVKM_SUBDEV_GPIO }, /* PMGR->GPIO */
- { 0x00200000, NVKM_SUBDEV_I2C }, /* PMGR->I2C/AUX */
- { 0x00400000, NVKM_ENGINE_CE0 }, /* NVA3- */
+ { 0x00008000, NVKM_ENGINE_MSVLD },
+ { 0x00004000, NVKM_ENGINE_SEC },
+ { 0x00001000, NVKM_ENGINE_GR },
+ { 0x00000100, NVKM_ENGINE_FIFO },
+ { 0x00000001, NVKM_ENGINE_MSPPP },
+ { 0x0002d101, NVKM_SUBDEV_FB },
{ 0x10000000, NVKM_SUBDEV_BUS },
- { 0x80000000, NVKM_ENGINE_SW },
- { 0x0042d101, NVKM_SUBDEV_FB },
+ { 0x00200000, NVKM_SUBDEV_GPIO },
+ { 0x00200000, NVKM_SUBDEV_I2C },
+ { 0x00100000, NVKM_SUBDEV_TIMER },
{},
};
@@ -51,6 +58,7 @@
.intr_unarm = nv04_mc_intr_unarm,
.intr_rearm = nv04_mc_intr_rearm,
.intr_mask = nv04_mc_intr_mask,
+ .reset = g98_mc_reset,
};
int
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gf100.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gf100.c
index 122fe69..8397e22 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gf100.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gf100.c
@@ -23,28 +23,38 @@
*/
#include "priv.h"
-const struct nvkm_mc_intr
-gf100_mc_intr[] = {
- { 0x04000000, NVKM_ENGINE_DISP }, /* DISP first, so pageflip timestamps work. */
- { 0x00000001, NVKM_ENGINE_MSPPP },
- { 0x00000020, NVKM_ENGINE_CE0 },
- { 0x00000040, NVKM_ENGINE_CE1 },
- { 0x00000080, NVKM_ENGINE_CE2 },
- { 0x00000100, NVKM_ENGINE_FIFO },
- { 0x00001000, NVKM_ENGINE_GR },
- { 0x00002000, NVKM_SUBDEV_FB },
- { 0x00008000, NVKM_ENGINE_MSVLD },
- { 0x00040000, NVKM_SUBDEV_THERM },
+static const struct nvkm_mc_map
+gf100_mc_reset[] = {
{ 0x00020000, NVKM_ENGINE_MSPDEC },
- { 0x00100000, NVKM_SUBDEV_TIMER },
- { 0x00200000, NVKM_SUBDEV_GPIO }, /* PMGR->GPIO */
- { 0x00200000, NVKM_SUBDEV_I2C }, /* PMGR->I2C/AUX */
- { 0x01000000, NVKM_SUBDEV_PMU },
- { 0x02000000, NVKM_SUBDEV_LTC },
- { 0x08000000, NVKM_SUBDEV_FB },
- { 0x10000000, NVKM_SUBDEV_BUS },
+ { 0x00008000, NVKM_ENGINE_MSVLD },
+ { 0x00001000, NVKM_ENGINE_GR },
+ { 0x00000100, NVKM_ENGINE_FIFO },
+ { 0x00000080, NVKM_ENGINE_CE1 },
+ { 0x00000040, NVKM_ENGINE_CE0 },
+ { 0x00000002, NVKM_ENGINE_MSPPP },
+ {}
+};
+
+static const struct nvkm_mc_map
+gf100_mc_intr[] = {
+ { 0x04000000, NVKM_ENGINE_DISP },
+ { 0x00020000, NVKM_ENGINE_MSPDEC },
+ { 0x00008000, NVKM_ENGINE_MSVLD },
+ { 0x00001000, NVKM_ENGINE_GR },
+ { 0x00000100, NVKM_ENGINE_FIFO },
+ { 0x00000040, NVKM_ENGINE_CE1 },
+ { 0x00000020, NVKM_ENGINE_CE0 },
+ { 0x00000001, NVKM_ENGINE_MSPPP },
{ 0x40000000, NVKM_SUBDEV_IBUS },
- { 0x80000000, NVKM_ENGINE_SW },
+ { 0x10000000, NVKM_SUBDEV_BUS },
+ { 0x08000000, NVKM_SUBDEV_FB },
+ { 0x02000000, NVKM_SUBDEV_LTC },
+ { 0x01000000, NVKM_SUBDEV_PMU },
+ { 0x00200000, NVKM_SUBDEV_GPIO },
+ { 0x00200000, NVKM_SUBDEV_I2C },
+ { 0x00100000, NVKM_SUBDEV_TIMER },
+ { 0x00040000, NVKM_SUBDEV_THERM },
+ { 0x00002000, NVKM_SUBDEV_FB },
{},
};
@@ -87,6 +97,7 @@
.intr_unarm = gf100_mc_intr_unarm,
.intr_rearm = gf100_mc_intr_rearm,
.intr_mask = gf100_mc_intr_mask,
+ .reset = gf100_mc_reset,
.unk260 = gf100_mc_unk260,
};
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gk104.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gk104.c
new file mode 100644
index 0000000..3174642
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gk104.c
@@ -0,0 +1,64 @@
+/*
+ * Copyright 2016 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: Ben Skeggs
+ */
+#include "priv.h"
+
+const struct nvkm_mc_map
+gk104_mc_reset[] = {
+ { 0x00000100, NVKM_ENGINE_FIFO },
+ {}
+};
+
+const struct nvkm_mc_map
+gk104_mc_intr[] = {
+ { 0x04000000, NVKM_ENGINE_DISP },
+ { 0x00000100, NVKM_ENGINE_FIFO },
+ { 0x40000000, NVKM_SUBDEV_IBUS },
+ { 0x10000000, NVKM_SUBDEV_BUS },
+ { 0x08000000, NVKM_SUBDEV_FB },
+ { 0x02000000, NVKM_SUBDEV_LTC },
+ { 0x01000000, NVKM_SUBDEV_PMU },
+ { 0x00200000, NVKM_SUBDEV_GPIO },
+ { 0x00200000, NVKM_SUBDEV_I2C },
+ { 0x00100000, NVKM_SUBDEV_TIMER },
+ { 0x00040000, NVKM_SUBDEV_THERM },
+ { 0x00002000, NVKM_SUBDEV_FB },
+ {},
+};
+
+static const struct nvkm_mc_func
+gk104_mc = {
+ .init = nv50_mc_init,
+ .intr = gk104_mc_intr,
+ .intr_unarm = gf100_mc_intr_unarm,
+ .intr_rearm = gf100_mc_intr_rearm,
+ .intr_mask = gf100_mc_intr_mask,
+ .reset = gk104_mc_reset,
+ .unk260 = gf100_mc_unk260,
+};
+
+int
+gk104_mc_new(struct nvkm_device *device, int index, struct nvkm_mc **pmc)
+{
+ return nvkm_mc_new_(&gk104_mc, device, index, pmc);
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gk20a.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gk20a.c
index d92efb3..60b044f 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gk20a.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gk20a.c
@@ -26,10 +26,11 @@
static const struct nvkm_mc_func
gk20a_mc = {
.init = nv50_mc_init,
- .intr = gf100_mc_intr,
+ .intr = gk104_mc_intr,
.intr_unarm = gf100_mc_intr_unarm,
.intr_rearm = gf100_mc_intr_rearm,
.intr_mask = gf100_mc_intr_mask,
+ .reset = gk104_mc_reset,
};
int
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gt215.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gt215.c
new file mode 100644
index 0000000..aad0ba9
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/gt215.c
@@ -0,0 +1,70 @@
+/*
+ * Copyright 2016 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: Ben Skeggs
+ */
+#include "priv.h"
+
+static const struct nvkm_mc_map
+gt215_mc_reset[] = {
+ { 0x04008000, NVKM_ENGINE_MSVLD },
+ { 0x01020000, NVKM_ENGINE_MSPDEC },
+ { 0x00802000, NVKM_ENGINE_CE0 },
+ { 0x00400002, NVKM_ENGINE_MSPPP },
+ { 0x00201000, NVKM_ENGINE_GR },
+ { 0x00000100, NVKM_ENGINE_FIFO },
+ {}
+};
+
+static const struct nvkm_mc_map
+gt215_mc_intr[] = {
+ { 0x04000000, NVKM_ENGINE_DISP },
+ { 0x00400000, NVKM_ENGINE_CE0 },
+ { 0x00020000, NVKM_ENGINE_MSPDEC },
+ { 0x00008000, NVKM_ENGINE_MSVLD },
+ { 0x00001000, NVKM_ENGINE_GR },
+ { 0x00000100, NVKM_ENGINE_FIFO },
+ { 0x00000001, NVKM_ENGINE_MSPPP },
+ { 0x00429101, NVKM_SUBDEV_FB },
+ { 0x10000000, NVKM_SUBDEV_BUS },
+ { 0x00200000, NVKM_SUBDEV_GPIO },
+ { 0x00200000, NVKM_SUBDEV_I2C },
+ { 0x00100000, NVKM_SUBDEV_TIMER },
+ { 0x00080000, NVKM_SUBDEV_THERM },
+ { 0x00040000, NVKM_SUBDEV_PMU },
+ {},
+};
+
+static const struct nvkm_mc_func
+gt215_mc = {
+ .init = nv50_mc_init,
+ .intr = gt215_mc_intr,
+ .intr_unarm = nv04_mc_intr_unarm,
+ .intr_rearm = nv04_mc_intr_rearm,
+ .intr_mask = nv04_mc_intr_mask,
+ .reset = gt215_mc_reset,
+};
+
+int
+gt215_mc_new(struct nvkm_device *device, int index, struct nvkm_mc **pmc)
+{
+ return nvkm_mc_new_(>215_mc, device, index, pmc);
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv04.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv04.c
index d282ec1..a062624 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv04.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv04.c
@@ -23,18 +23,20 @@
*/
#include "priv.h"
-const struct nvkm_mc_intr
-nv04_mc_intr[] = {
- { 0x00000001, NVKM_ENGINE_MPEG }, /* NV17- MPEG/ME */
- { 0x00000100, NVKM_ENGINE_FIFO },
+const struct nvkm_mc_map
+nv04_mc_reset[] = {
{ 0x00001000, NVKM_ENGINE_GR },
- { 0x00010000, NVKM_ENGINE_DISP },
- { 0x00020000, NVKM_ENGINE_VP }, /* NV40- */
- { 0x00100000, NVKM_SUBDEV_TIMER },
- { 0x01000000, NVKM_ENGINE_DISP }, /* NV04- PCRTC0 */
- { 0x02000000, NVKM_ENGINE_DISP }, /* NV11- PCRTC1 */
+ { 0x00000100, NVKM_ENGINE_FIFO },
+ {}
+};
+
+static const struct nvkm_mc_map
+nv04_mc_intr[] = {
+ { 0x01010000, NVKM_ENGINE_DISP },
+ { 0x00001000, NVKM_ENGINE_GR },
+ { 0x00000100, NVKM_ENGINE_FIFO },
{ 0x10000000, NVKM_SUBDEV_BUS },
- { 0x80000000, NVKM_ENGINE_SW },
+ { 0x00100000, NVKM_SUBDEV_TIMER },
{}
};
@@ -74,6 +76,7 @@
.intr_unarm = nv04_mc_intr_unarm,
.intr_rearm = nv04_mc_intr_rearm,
.intr_mask = nv04_mc_intr_mask,
+ .reset = nv04_mc_reset,
};
int
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv11.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv11.c
new file mode 100644
index 0000000..55f0b91
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv11.c
@@ -0,0 +1,50 @@
+/*
+ * Copyright 2016 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: Ben Skeggs <bskeggs@redhat.com>
+ */
+#include "priv.h"
+
+static const struct nvkm_mc_map
+nv11_mc_intr[] = {
+ { 0x03010000, NVKM_ENGINE_DISP },
+ { 0x00001000, NVKM_ENGINE_GR },
+ { 0x00000100, NVKM_ENGINE_FIFO },
+ { 0x10000000, NVKM_SUBDEV_BUS },
+ { 0x00100000, NVKM_SUBDEV_TIMER },
+ {}
+};
+
+static const struct nvkm_mc_func
+nv11_mc = {
+ .init = nv04_mc_init,
+ .intr = nv11_mc_intr,
+ .intr_unarm = nv04_mc_intr_unarm,
+ .intr_rearm = nv04_mc_intr_rearm,
+ .intr_mask = nv04_mc_intr_mask,
+ .reset = nv04_mc_reset,
+};
+
+int
+nv11_mc_new(struct nvkm_device *device, int index, struct nvkm_mc **pmc)
+{
+ return nvkm_mc_new_(&nv11_mc, device, index, pmc);
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv17.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv17.c
new file mode 100644
index 0000000..c40fa67
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv17.c
@@ -0,0 +1,59 @@
+/*
+ * Copyright 2016 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: Ben Skeggs <bskeggs@redhat.com>
+ */
+#include "priv.h"
+
+const struct nvkm_mc_map
+nv17_mc_reset[] = {
+ { 0x00001000, NVKM_ENGINE_GR },
+ { 0x00000100, NVKM_ENGINE_FIFO },
+ { 0x00000002, NVKM_ENGINE_MPEG },
+ {}
+};
+
+const struct nvkm_mc_map
+nv17_mc_intr[] = {
+ { 0x03010000, NVKM_ENGINE_DISP },
+ { 0x00001000, NVKM_ENGINE_GR },
+ { 0x00000100, NVKM_ENGINE_FIFO },
+ { 0x00000001, NVKM_ENGINE_MPEG },
+ { 0x10000000, NVKM_SUBDEV_BUS },
+ { 0x00100000, NVKM_SUBDEV_TIMER },
+ {}
+};
+
+static const struct nvkm_mc_func
+nv17_mc = {
+ .init = nv04_mc_init,
+ .intr = nv17_mc_intr,
+ .intr_unarm = nv04_mc_intr_unarm,
+ .intr_rearm = nv04_mc_intr_rearm,
+ .intr_mask = nv04_mc_intr_mask,
+ .reset = nv17_mc_reset,
+};
+
+int
+nv17_mc_new(struct nvkm_device *device, int index, struct nvkm_mc **pmc)
+{
+ return nvkm_mc_new_(&nv17_mc, device, index, pmc);
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv44.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv44.c
index 9a3ac99..cc56271 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv44.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv44.c
@@ -40,10 +40,11 @@
static const struct nvkm_mc_func
nv44_mc = {
.init = nv44_mc_init,
- .intr = nv04_mc_intr,
+ .intr = nv17_mc_intr,
.intr_unarm = nv04_mc_intr_unarm,
.intr_rearm = nv04_mc_intr_rearm,
.intr_mask = nv04_mc_intr_mask,
+ .reset = nv17_mc_reset,
};
int
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv50.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv50.c
index 5f27d7b..343b607 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv50.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv50.c
@@ -23,21 +23,17 @@
*/
#include "priv.h"
-const struct nvkm_mc_intr
+static const struct nvkm_mc_map
nv50_mc_intr[] = {
- { 0x04000000, NVKM_ENGINE_DISP }, /* DISP before FIFO, so pageflip-timestamping works! */
- { 0x00000001, NVKM_ENGINE_MPEG },
- { 0x00000100, NVKM_ENGINE_FIFO },
+ { 0x04000000, NVKM_ENGINE_DISP },
{ 0x00001000, NVKM_ENGINE_GR },
- { 0x00004000, NVKM_ENGINE_CIPHER }, /* NV84- */
- { 0x00008000, NVKM_ENGINE_BSP }, /* NV84- */
- { 0x00020000, NVKM_ENGINE_VP }, /* NV84- */
- { 0x00100000, NVKM_SUBDEV_TIMER },
- { 0x00200000, NVKM_SUBDEV_GPIO }, /* PMGR->GPIO */
- { 0x00200000, NVKM_SUBDEV_I2C }, /* PMGR->I2C/AUX */
+ { 0x00000100, NVKM_ENGINE_FIFO },
+ { 0x00000001, NVKM_ENGINE_MPEG },
+ { 0x00001101, NVKM_SUBDEV_FB },
{ 0x10000000, NVKM_SUBDEV_BUS },
- { 0x80000000, NVKM_ENGINE_SW },
- { 0x0002d101, NVKM_SUBDEV_FB },
+ { 0x00200000, NVKM_SUBDEV_GPIO },
+ { 0x00200000, NVKM_SUBDEV_I2C },
+ { 0x00100000, NVKM_SUBDEV_TIMER },
{},
};
@@ -55,6 +51,7 @@
.intr_unarm = nv04_mc_intr_unarm,
.intr_rearm = nv04_mc_intr_rearm,
.intr_mask = nv04_mc_intr_mask,
+ .reset = nv17_mc_reset,
};
int
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/priv.h b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/priv.h
index 307f6c6..a120381 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mc/priv.h
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mc/priv.h
@@ -6,37 +6,42 @@
int nvkm_mc_new_(const struct nvkm_mc_func *, struct nvkm_device *,
int index, struct nvkm_mc **);
-struct nvkm_mc_intr {
+struct nvkm_mc_map {
u32 stat;
u32 unit;
};
struct nvkm_mc_func {
void (*init)(struct nvkm_mc *);
- const struct nvkm_mc_intr *intr;
+ const struct nvkm_mc_map *intr;
/* disable reporting of interrupts to host */
void (*intr_unarm)(struct nvkm_mc *);
/* enable reporting of interrupts to host */
void (*intr_rearm)(struct nvkm_mc *);
/* retrieve pending interrupt mask (NV_PMC_INTR) */
u32 (*intr_mask)(struct nvkm_mc *);
+ const struct nvkm_mc_map *reset;
void (*unk260)(struct nvkm_mc *, u32);
};
void nv04_mc_init(struct nvkm_mc *);
-extern const struct nvkm_mc_intr nv04_mc_intr[];
void nv04_mc_intr_unarm(struct nvkm_mc *);
void nv04_mc_intr_rearm(struct nvkm_mc *);
u32 nv04_mc_intr_mask(struct nvkm_mc *);
+extern const struct nvkm_mc_map nv04_mc_reset[];
+
+extern const struct nvkm_mc_map nv17_mc_intr[];
+extern const struct nvkm_mc_map nv17_mc_reset[];
void nv44_mc_init(struct nvkm_mc *);
void nv50_mc_init(struct nvkm_mc *);
-extern const struct nvkm_mc_intr nv50_mc_intr[];
-extern const struct nvkm_mc_intr gf100_mc_intr[];
void gf100_mc_intr_unarm(struct nvkm_mc *);
void gf100_mc_intr_rearm(struct nvkm_mc *);
u32 gf100_mc_intr_mask(struct nvkm_mc *);
void gf100_mc_unk260(struct nvkm_mc *, u32);
+
+extern const struct nvkm_mc_map gk104_mc_intr[];
+extern const struct nvkm_mc_map gk104_mc_reset[];
#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mmu/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mmu/base.c
index e04a229..5df9669 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mmu/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mmu/base.c
@@ -524,7 +524,7 @@
nvkm_mmu_ctor(const struct nvkm_mmu_func *func, struct nvkm_device *device,
int index, struct nvkm_mmu *mmu)
{
- nvkm_subdev_ctor(&nvkm_mmu, device, index, 0, &mmu->subdev);
+ nvkm_subdev_ctor(&nvkm_mmu, device, index, &mmu->subdev);
mmu->func = func;
mmu->limit = func->limit;
mmu->dma_bits = func->dma_bits;
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mxm/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mxm/base.c
index 9700a76..21b65ee 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/mxm/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mxm/base.c
@@ -241,7 +241,7 @@
if (!(mxm = *pmxm = kzalloc(sizeof(*mxm), GFP_KERNEL)))
return -ENOMEM;
- nvkm_subdev_ctor(&nvkm_mxm, device, index, 0, &mxm->subdev);
+ nvkm_subdev_ctor(&nvkm_mxm, device, index, &mxm->subdev);
data = mxm_table(bios, &ver, &len);
if (!data || !(ver = nvbios_rd08(bios, data))) {
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pci/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/pci/base.c
index 65057c8..6b0328b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/pci/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pci/base.c
@@ -168,7 +168,7 @@
if (!(pci = *ppci = kzalloc(sizeof(**ppci), GFP_KERNEL)))
return -ENOMEM;
- nvkm_subdev_ctor(&nvkm_pci_func, device, index, 0, &pci->subdev);
+ nvkm_subdev_ctor(&nvkm_pci_func, device, index, &pci->subdev);
pci->func = func;
pci->pdev = device->func->pci(device)->pdev;
pci->irq = -1;
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/base.c
index d95eb86..8dd164d13 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/base.c
@@ -40,21 +40,23 @@
struct nvkm_device *device = subdev->device;
u32 addr;
+ mutex_lock(&subdev->mutex);
/* wait for a free slot in the fifo */
addr = nvkm_rd32(device, 0x10a4a0);
if (nvkm_msec(device, 2000,
u32 tmp = nvkm_rd32(device, 0x10a4b0);
if (tmp != (addr ^ 8))
break;
- ) < 0)
+ ) < 0) {
+ mutex_unlock(&subdev->mutex);
return -EBUSY;
+ }
/* we currently only support a single process at a time waiting
* on a synchronous reply, take the PMU mutex and tell the
* receive handler what we're waiting for
*/
if (reply) {
- mutex_lock(&subdev->mutex);
pmu->recv.message = message;
pmu->recv.process = process;
}
@@ -81,9 +83,9 @@
wait_event(pmu->recv.wait, (pmu->recv.process == 0));
reply[0] = pmu->recv.data[0];
reply[1] = pmu->recv.data[1];
- mutex_unlock(&subdev->mutex);
}
+ mutex_unlock(&subdev->mutex);
return 0;
}
@@ -272,7 +274,7 @@
struct nvkm_pmu *pmu;
if (!(pmu = *ppmu = kzalloc(sizeof(*pmu), GFP_KERNEL)))
return -ENOMEM;
- nvkm_subdev_ctor(&nvkm_pmu, device, index, 0, &pmu->subdev);
+ nvkm_subdev_ctor(&nvkm_pmu, device, index, &pmu->subdev);
pmu->func = func;
INIT_WORK(&pmu->recv.work, nvkm_pmu_recv);
init_waitqueue_head(&pmu->recv.wait);
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gk20a.c b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gk20a.c
index 6689d02..f996d90 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gk20a.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gk20a.c
@@ -220,7 +220,7 @@
pmu->base.func = &func;
*ppmu = &pmu->base;
- nvkm_subdev_ctor(&gk20a_pmu, device, index, 0, &pmu->base.subdev);
+ nvkm_subdev_ctor(&gk20a_pmu, device, index, &pmu->base.subdev);
pmu->data = &gk20a_dvfs_data;
nvkm_alarm_init(&pmu->alarm, gk20a_pmu_dvfs_work);
return 0;
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/secboot/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/secboot/base.c
index 520facf..213fdba 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/secboot/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/secboot/base.c
@@ -264,7 +264,7 @@
{
unsigned long fid;
- nvkm_subdev_ctor(&nvkm_secboot, device, index, 0, &sb->subdev);
+ nvkm_subdev_ctor(&nvkm_secboot, device, index, &sb->subdev);
sb->func = func;
/* setup the performing falcon's base address and masks */
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/therm/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/therm/base.c
index 949dc61..8894fee 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/therm/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/therm/base.c
@@ -366,7 +366,7 @@
if (!(therm = *ptherm = kzalloc(sizeof(*therm), GFP_KERNEL)))
return -ENOMEM;
- nvkm_subdev_ctor(&nvkm_therm, device, index, 0, &therm->subdev);
+ nvkm_subdev_ctor(&nvkm_therm, device, index, &therm->subdev);
therm->func = func;
nvkm_alarm_init(&therm->alarm, nvkm_therm_alarm);
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/timer/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/timer/base.c
index d4dae1f..07dc82b 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/timer/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/timer/base.c
@@ -143,7 +143,7 @@
if (!(tmr = *ptmr = kzalloc(sizeof(*tmr), GFP_KERNEL)))
return -ENOMEM;
- nvkm_subdev_ctor(&nvkm_timer, device, index, 0, &tmr->subdev);
+ nvkm_subdev_ctor(&nvkm_timer, device, index, &tmr->subdev);
tmr->func = func;
INIT_LIST_HEAD(&tmr->alarms);
spin_lock_init(&tmr->lock);
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/top/Kbuild b/drivers/gpu/drm/nouveau/nvkm/subdev/top/Kbuild
new file mode 100644
index 0000000..1078401
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/top/Kbuild
@@ -0,0 +1,2 @@
+nvkm-y += nvkm/subdev/top/base.o
+nvkm-y += nvkm/subdev/top/gk104.o
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/top/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/top/base.c
new file mode 100644
index 0000000..a1b2646
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/top/base.c
@@ -0,0 +1,148 @@
+/*
+ * Copyright 2016 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: Ben Skeggs <bskeggs@redhat.com>
+ */
+#include "priv.h"
+
+struct nvkm_top_device *
+nvkm_top_device_new(struct nvkm_top *top)
+{
+ struct nvkm_top_device *info = kmalloc(sizeof(*info), GFP_KERNEL);
+ if (info) {
+ info->index = NVKM_SUBDEV_NR;
+ info->addr = 0;
+ info->fault = -1;
+ info->engine = -1;
+ info->runlist = -1;
+ info->reset = -1;
+ info->intr = -1;
+ list_add_tail(&info->head, &top->device);
+ }
+ return info;
+}
+
+u32
+nvkm_top_reset(struct nvkm_top *top, enum nvkm_devidx index)
+{
+ struct nvkm_top_device *info;
+
+ if (top) {
+ list_for_each_entry(info, &top->device, head) {
+ if (info->index == index && info->reset >= 0)
+ return BIT(info->reset);
+ }
+ }
+
+ return 0;
+}
+
+u32
+nvkm_top_intr(struct nvkm_top *top, u32 intr, u64 *psubdevs)
+{
+ struct nvkm_top_device *info;
+ u64 subdevs = 0;
+ u32 handled = 0;
+
+ if (top) {
+ list_for_each_entry(info, &top->device, head) {
+ if (info->index != NVKM_SUBDEV_NR && info->intr >= 0) {
+ if (intr & BIT(info->intr)) {
+ subdevs |= BIT_ULL(info->index);
+ handled |= BIT(info->intr);
+ }
+ }
+ }
+ }
+
+ *psubdevs = subdevs;
+ return intr & ~handled;
+}
+
+enum nvkm_devidx
+nvkm_top_fault(struct nvkm_top *top, int fault)
+{
+ struct nvkm_top_device *info;
+
+ list_for_each_entry(info, &top->device, head) {
+ if (info->fault == fault)
+ return info->index;
+ }
+
+ return NVKM_SUBDEV_NR;
+}
+
+enum nvkm_devidx
+nvkm_top_engine(struct nvkm_top *top, int index, int *runl, int *engn)
+{
+ struct nvkm_top_device *info;
+ int n = 0;
+
+ list_for_each_entry(info, &top->device, head) {
+ if (info->engine >= 0 && info->runlist >= 0 && n++ == index) {
+ *runl = info->runlist;
+ *engn = info->engine;
+ return info->index;
+ }
+ }
+
+ return -ENODEV;
+}
+
+static int
+nvkm_top_oneinit(struct nvkm_subdev *subdev)
+{
+ struct nvkm_top *top = nvkm_top(subdev);
+ return top->func->oneinit(top);
+}
+
+static void *
+nvkm_top_dtor(struct nvkm_subdev *subdev)
+{
+ struct nvkm_top *top = nvkm_top(subdev);
+ struct nvkm_top_device *info, *temp;
+
+ list_for_each_entry_safe(info, temp, &top->device, head) {
+ list_del(&info->head);
+ kfree(info);
+ }
+
+ return top;
+}
+
+static const struct nvkm_subdev_func
+nvkm_top = {
+ .dtor = nvkm_top_dtor,
+ .oneinit = nvkm_top_oneinit,
+};
+
+int
+nvkm_top_new_(const struct nvkm_top_func *func, struct nvkm_device *device,
+ int index, struct nvkm_top **ptop)
+{
+ struct nvkm_top *top;
+ if (!(top = *ptop = kzalloc(sizeof(*top), GFP_KERNEL)))
+ return -ENOMEM;
+ nvkm_subdev_ctor(&nvkm_top, device, index, &top->subdev);
+ top->func = func;
+ INIT_LIST_HEAD(&top->device);
+ return 0;
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/top/gk104.c b/drivers/gpu/drm/nouveau/nvkm/subdev/top/gk104.c
new file mode 100644
index 0000000..e06acc3
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/top/gk104.c
@@ -0,0 +1,110 @@
+/*
+ * Copyright 2016 Red Hat Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: Ben Skeggs <bskeggs@redhat.com>
+ */
+#include "priv.h"
+
+static int
+gk104_top_oneinit(struct nvkm_top *top)
+{
+ struct nvkm_subdev *subdev = &top->subdev;
+ struct nvkm_device *device = subdev->device;
+ struct nvkm_top_device *info = NULL;
+ u32 data, type;
+ int i;
+
+ for (i = 0; i < 64; i++) {
+ if (!info) {
+ if (!(info = nvkm_top_device_new(top)))
+ return -ENOMEM;
+ type = ~0;
+ }
+
+ data = nvkm_rd32(device, 0x022700 + (i * 0x04));
+ nvkm_trace(subdev, "%02x: %08x\n", i, data);
+ switch (data & 0x00000003) {
+ case 0x00000000: /* NOT_VALID */
+ continue;
+ case 0x00000001: /* DATA */
+ info->addr = (data & 0x00fff000);
+ info->fault = (data & 0x000000f8) >> 3;
+ break;
+ case 0x00000002: /* ENUM */
+ if (data & 0x00000020)
+ info->engine = (data & 0x3c000000) >> 26;
+ if (data & 0x00000010)
+ info->runlist = (data & 0x01e00000) >> 21;
+ if (data & 0x00000008)
+ info->intr = (data & 0x000f8000) >> 15;
+ if (data & 0x00000004)
+ info->reset = (data & 0x00003e00) >> 9;
+ break;
+ case 0x00000003: /* ENGINE_TYPE */
+ type = (data & 0x7ffffffc) >> 2;
+ break;
+ }
+
+ if (data & 0x80000000)
+ continue;
+
+ /* Translate engine type to NVKM engine identifier. */
+ switch (type) {
+ case 0x00000000: info->index = NVKM_ENGINE_GR; break;
+ case 0x00000001: info->index = NVKM_ENGINE_CE0; break;
+ case 0x00000002: info->index = NVKM_ENGINE_CE1; break;
+ case 0x00000003: info->index = NVKM_ENGINE_CE2; break;
+ case 0x00000008: info->index = NVKM_ENGINE_MSPDEC; break;
+ case 0x00000009: info->index = NVKM_ENGINE_MSPPP; break;
+ case 0x0000000a: info->index = NVKM_ENGINE_MSVLD; break;
+ case 0x0000000b: info->index = NVKM_ENGINE_MSENC; break;
+ case 0x0000000c: info->index = NVKM_ENGINE_VIC; break;
+ case 0x0000000d: info->index = NVKM_ENGINE_SEC; break;
+ case 0x0000000e: info->index = NVKM_ENGINE_NVENC0; break;
+ case 0x0000000f: info->index = NVKM_ENGINE_NVENC1; break;
+ case 0x00000010: info->index = NVKM_ENGINE_NVDEC; break;
+ break;
+ default:
+ break;
+ }
+
+ nvkm_debug(subdev, "%02x (%8s): addr %06x fault %2d engine %2d "
+ "runlist %2d intr %2d reset %2d\n", type,
+ info->index == NVKM_SUBDEV_NR ? NULL :
+ nvkm_subdev_name[info->index],
+ info->addr, info->fault, info->engine, info->runlist,
+ info->intr, info->reset);
+ info = NULL;
+ }
+
+ return 0;
+}
+
+static const struct nvkm_top_func
+gk104_top = {
+ .oneinit = gk104_top_oneinit,
+};
+
+int
+gk104_top_new(struct nvkm_device *device, int index, struct nvkm_top **ptop)
+{
+ return nvkm_top_new_(&gk104_top, device, index, ptop);
+}
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/top/priv.h b/drivers/gpu/drm/nouveau/nvkm/subdev/top/priv.h
new file mode 100644
index 0000000..adb3ed0
--- /dev/null
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/top/priv.h
@@ -0,0 +1,25 @@
+#ifndef __NVKM_TOP_PRIV_H__
+#define __NVKM_TOP_PRIV_H__
+#define nvkm_top(p) container_of((p), struct nvkm_top, subdev)
+#include <subdev/top.h>
+
+struct nvkm_top_func {
+ int (*oneinit)(struct nvkm_top *);
+};
+
+int nvkm_top_new_(const struct nvkm_top_func *, struct nvkm_device *,
+ int, struct nvkm_top **);
+
+struct nvkm_top_device {
+ enum nvkm_devidx index;
+ u32 addr;
+ int fault;
+ int engine;
+ int runlist;
+ int reset;
+ int intr;
+ struct list_head head;
+};
+
+struct nvkm_top_device *nvkm_top_device_new(struct nvkm_top *);
+#endif
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/volt/base.c b/drivers/gpu/drm/nouveau/nvkm/subdev/volt/base.c
index 50b5649..6b2d753 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/volt/base.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/volt/base.c
@@ -177,7 +177,7 @@
struct nvkm_bios *bios = device->bios;
int i;
- nvkm_subdev_ctor(&nvkm_volt, device, index, 0, &volt->subdev);
+ nvkm_subdev_ctor(&nvkm_volt, device, index, &volt->subdev);
volt->func = func;
/* Assuming the non-bios device should build the voltage table later */
diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/volt/gk104.c b/drivers/gpu/drm/nouveau/nvkm/subdev/volt/gk104.c
index b735173..420bd84d 100644
--- a/drivers/gpu/drm/nouveau/nvkm/subdev/volt/gk104.c
+++ b/drivers/gpu/drm/nouveau/nvkm/subdev/volt/gk104.c
@@ -56,7 +56,7 @@
/* the blob uses this crystal frequency, let's use it too. */
div = 27648000 / bios->pwm_freq;
- duty = (uv - bios->base) * div / bios->pwm_range;
+ duty = DIV_ROUND_UP((uv - bios->base) * div, bios->pwm_range);
nvkm_wr32(device, 0x20340, div);
nvkm_wr32(device, 0x20344, 0x80000000 | duty);
diff --git a/drivers/gpu/drm/omapdrm/omap_drv.c b/drivers/gpu/drm/omapdrm/omap_drv.c
index 80398a6..d86f547 100644
--- a/drivers/gpu/drm/omapdrm/omap_drv.c
+++ b/drivers/gpu/drm/omapdrm/omap_drv.c
@@ -138,7 +138,7 @@
}
static int omap_atomic_commit(struct drm_device *dev,
- struct drm_atomic_state *state, bool async)
+ struct drm_atomic_state *state, bool nonblock)
{
struct omap_drm_private *priv = dev->dev_private;
struct omap_atomic_state_commit *commit;
@@ -177,7 +177,7 @@
/* Swap the state, this is the point of no return. */
drm_atomic_helper_swap_state(dev, state);
- if (async)
+ if (nonblock)
schedule_work(&commit->work);
else
omap_atomic_complete(commit);
@@ -561,7 +561,7 @@
VERB("%p:%p: handle=%d, op=%x", dev, file_priv, args->handle, args->op);
- obj = drm_gem_object_lookup(dev, file_priv, args->handle);
+ obj = drm_gem_object_lookup(file_priv, args->handle);
if (!obj)
return -ENOENT;
@@ -584,7 +584,7 @@
VERB("%p:%p: handle=%d", dev, file_priv, args->handle);
- obj = drm_gem_object_lookup(dev, file_priv, args->handle);
+ obj = drm_gem_object_lookup(file_priv, args->handle);
if (!obj)
return -ENOENT;
@@ -608,7 +608,7 @@
VERB("%p:%p: handle=%d", dev, file_priv, args->handle);
- obj = drm_gem_object_lookup(dev, file_priv, args->handle);
+ obj = drm_gem_object_lookup(file_priv, args->handle);
if (!obj)
return -ENOENT;
diff --git a/drivers/gpu/drm/omapdrm/omap_drv.h b/drivers/gpu/drm/omapdrm/omap_drv.h
index 0fbe17d..3f823c3 100644
--- a/drivers/gpu/drm/omapdrm/omap_drv.h
+++ b/drivers/gpu/drm/omapdrm/omap_drv.h
@@ -257,14 +257,14 @@
/* should these be made into common util helpers?
*/
-static inline int objects_lookup(struct drm_device *dev,
+static inline int objects_lookup(
struct drm_file *filp, uint32_t pixel_format,
struct drm_gem_object **bos, const uint32_t *handles)
{
int i, n = drm_format_num_planes(pixel_format);
for (i = 0; i < n; i++) {
- bos[i] = drm_gem_object_lookup(dev, filp, handles[i]);
+ bos[i] = drm_gem_object_lookup(filp, handles[i]);
if (!bos[i])
goto fail;
diff --git a/drivers/gpu/drm/omapdrm/omap_fb.c b/drivers/gpu/drm/omapdrm/omap_fb.c
index 6109623..94ec06d 100644
--- a/drivers/gpu/drm/omapdrm/omap_fb.c
+++ b/drivers/gpu/drm/omapdrm/omap_fb.c
@@ -378,7 +378,7 @@
struct drm_framebuffer *fb;
int ret;
- ret = objects_lookup(dev, file, mode_cmd->pixel_format,
+ ret = objects_lookup(file, mode_cmd->pixel_format,
bos, mode_cmd->handles);
if (ret)
return ERR_PTR(ret);
diff --git a/drivers/gpu/drm/omapdrm/omap_fbdev.c b/drivers/gpu/drm/omapdrm/omap_fbdev.c
index 3cb16f0..89da41a 100644
--- a/drivers/gpu/drm/omapdrm/omap_fbdev.c
+++ b/drivers/gpu/drm/omapdrm/omap_fbdev.c
@@ -153,7 +153,7 @@
/* note: if fb creation failed, we can't rely on fb destroy
* to unref the bo:
*/
- drm_gem_object_unreference(fbdev->bo);
+ drm_gem_object_unreference_unlocked(fbdev->bo);
ret = PTR_ERR(fb);
goto fail;
}
diff --git a/drivers/gpu/drm/omapdrm/omap_gem.c b/drivers/gpu/drm/omapdrm/omap_gem.c
index 907154f..b97afc2 100644
--- a/drivers/gpu/drm/omapdrm/omap_gem.c
+++ b/drivers/gpu/drm/omapdrm/omap_gem.c
@@ -687,7 +687,7 @@
int ret = 0;
/* GEM does all our handle to object mapping */
- obj = drm_gem_object_lookup(dev, file, handle);
+ obj = drm_gem_object_lookup(file, handle);
if (obj == NULL) {
ret = -ENOENT;
goto fail;
diff --git a/drivers/gpu/drm/omapdrm/omap_plane.c b/drivers/gpu/drm/omapdrm/omap_plane.c
index 93ee538..5252ab7 100644
--- a/drivers/gpu/drm/omapdrm/omap_plane.c
+++ b/drivers/gpu/drm/omapdrm/omap_plane.c
@@ -245,7 +245,7 @@
static void omap_plane_atomic_destroy_state(struct drm_plane *plane,
struct drm_plane_state *state)
{
- __drm_atomic_helper_plane_destroy_state(plane, state);
+ __drm_atomic_helper_plane_destroy_state(state);
kfree(to_omap_plane_state(state));
}
diff --git a/drivers/gpu/drm/panel/panel-simple.c b/drivers/gpu/drm/panel/panel-simple.c
index ceb2048..3a7bdf1 100644
--- a/drivers/gpu/drm/panel/panel-simple.c
+++ b/drivers/gpu/drm/panel/panel-simple.c
@@ -72,6 +72,7 @@
} delay;
u32 bus_format;
+ u32 bus_flags;
};
struct panel_simple {
@@ -116,7 +117,11 @@
}
drm_display_mode_from_videomode(&vm, mode);
- drm_mode_set_name(mode);
+
+ mode->type |= DRM_MODE_TYPE_DRIVER;
+
+ if (panel->desc->num_modes == 1)
+ mode->type |= DRM_MODE_TYPE_PREFERRED;
drm_mode_probed_add(connector, mode);
num++;
@@ -132,6 +137,11 @@
continue;
}
+ mode->type |= DRM_MODE_TYPE_DRIVER;
+
+ if (panel->desc->num_modes == 1)
+ mode->type |= DRM_MODE_TYPE_PREFERRED;
+
drm_mode_set_name(mode);
drm_mode_probed_add(connector, mode);
@@ -144,6 +154,7 @@
if (panel->desc->bus_format)
drm_display_info_set_bus_formats(&connector->display_info,
&panel->desc->bus_format, 1);
+ connector->display_info.bus_flags = panel->desc->bus_flags;
return num;
}
@@ -813,6 +824,29 @@
.bus_format = MEDIA_BUS_FMT_RGB888_1X24,
};
+static const struct drm_display_mode innolux_at070tn92_mode = {
+ .clock = 33333,
+ .hdisplay = 800,
+ .hsync_start = 800 + 210,
+ .hsync_end = 800 + 210 + 20,
+ .htotal = 800 + 210 + 20 + 46,
+ .vdisplay = 480,
+ .vsync_start = 480 + 22,
+ .vsync_end = 480 + 22 + 10,
+ .vtotal = 480 + 22 + 23 + 10,
+ .vrefresh = 60,
+};
+
+static const struct panel_desc innolux_at070tn92 = {
+ .modes = &innolux_at070tn92_mode,
+ .num_modes = 1,
+ .size = {
+ .width = 154,
+ .height = 86,
+ },
+ .bus_format = MEDIA_BUS_FMT_RGB888_1X24,
+};
+
static const struct drm_display_mode innolux_g121i1_l01_mode = {
.clock = 71000,
.hdisplay = 1280,
@@ -1051,7 +1085,8 @@
.width = 95,
.height = 54,
},
- .bus_format = MEDIA_BUS_FMT_RGB888_1X24
+ .bus_format = MEDIA_BUS_FMT_RGB888_1X24,
+ .bus_flags = DRM_BUS_FLAG_PIXDATA_POSEDGE,
};
static const struct display_timing okaya_rs800480t_7x0gp_timing = {
@@ -1084,6 +1119,63 @@
.bus_format = MEDIA_BUS_FMT_RGB666_1X18,
};
+static const struct drm_display_mode olimex_lcd_olinuxino_43ts_mode = {
+ .clock = 9000,
+ .hdisplay = 480,
+ .hsync_start = 480 + 5,
+ .hsync_end = 480 + 5 + 30,
+ .htotal = 480 + 5 + 30 + 10,
+ .vdisplay = 272,
+ .vsync_start = 272 + 8,
+ .vsync_end = 272 + 8 + 5,
+ .vtotal = 272 + 8 + 5 + 3,
+ .vrefresh = 60,
+};
+
+static const struct panel_desc olimex_lcd_olinuxino_43ts = {
+ .modes = &olimex_lcd_olinuxino_43ts_mode,
+ .num_modes = 1,
+ .size = {
+ .width = 105,
+ .height = 67,
+ },
+ .bus_format = MEDIA_BUS_FMT_RGB666_1X18,
+};
+
+/*
+ * 800x480 CVT. The panel appears to be quite accepting, at least as far as
+ * pixel clocks, but this is the timing that was being used in the Adafruit
+ * installation instructions.
+ */
+static const struct drm_display_mode ontat_yx700wv03_mode = {
+ .clock = 29500,
+ .hdisplay = 800,
+ .hsync_start = 824,
+ .hsync_end = 896,
+ .htotal = 992,
+ .vdisplay = 480,
+ .vsync_start = 483,
+ .vsync_end = 493,
+ .vtotal = 500,
+ .vrefresh = 60,
+ .flags = DRM_MODE_FLAG_NVSYNC | DRM_MODE_FLAG_NHSYNC,
+};
+
+/*
+ * Specification at:
+ * https://www.adafruit.com/images/product-files/2406/c3163.pdf
+ */
+static const struct panel_desc ontat_yx700wv03 = {
+ .modes = &ontat_yx700wv03_mode,
+ .num_modes = 1,
+ .bpc = 8,
+ .size = {
+ .width = 154,
+ .height = 83,
+ },
+ .bus_format = MEDIA_BUS_FMT_RGB888_1X24,
+};
+
static const struct drm_display_mode ortustech_com43h4m85ulc_mode = {
.clock = 25000,
.hdisplay = 480,
@@ -1201,6 +1293,51 @@
.bus_format = MEDIA_BUS_FMT_RGB666_1X18,
};
+static const struct drm_display_mode tpk_f07a_0102_mode = {
+ .clock = 33260,
+ .hdisplay = 800,
+ .hsync_start = 800 + 40,
+ .hsync_end = 800 + 40 + 128,
+ .htotal = 800 + 40 + 128 + 88,
+ .vdisplay = 480,
+ .vsync_start = 480 + 10,
+ .vsync_end = 480 + 10 + 2,
+ .vtotal = 480 + 10 + 2 + 33,
+ .vrefresh = 60,
+};
+
+static const struct panel_desc tpk_f07a_0102 = {
+ .modes = &tpk_f07a_0102_mode,
+ .num_modes = 1,
+ .size = {
+ .width = 152,
+ .height = 91,
+ },
+ .bus_flags = DRM_BUS_FLAG_PIXDATA_POSEDGE,
+};
+
+static const struct drm_display_mode tpk_f10a_0102_mode = {
+ .clock = 45000,
+ .hdisplay = 1024,
+ .hsync_start = 1024 + 176,
+ .hsync_end = 1024 + 176 + 5,
+ .htotal = 1024 + 176 + 5 + 88,
+ .vdisplay = 600,
+ .vsync_start = 600 + 20,
+ .vsync_end = 600 + 20 + 5,
+ .vtotal = 600 + 20 + 5 + 25,
+ .vrefresh = 60,
+};
+
+static const struct panel_desc tpk_f10a_0102 = {
+ .modes = &tpk_f10a_0102_mode,
+ .num_modes = 1,
+ .size = {
+ .width = 223,
+ .height = 125,
+ },
+};
+
static const struct display_timing urt_umsh_8596md_timing = {
.pixelclock = { 33260000, 33260000, 33260000 },
.hactive = { 800, 800, 800 },
@@ -1296,6 +1433,9 @@
.compatible = "innolux,at043tn24",
.data = &innolux_at043tn24,
}, {
+ .compatible = "innolux,at070tn92",
+ .data = &innolux_at070tn92,
+ }, {
.compatible ="innolux,g121i1-l01",
.data = &innolux_g121i1_l01
}, {
@@ -1329,6 +1469,12 @@
.compatible = "okaya,rs800480t-7x0gp",
.data = &okaya_rs800480t_7x0gp,
}, {
+ .compatible = "olimex,lcd-olinuxino-43-ts",
+ .data = &olimex_lcd_olinuxino_43ts,
+ }, {
+ .compatible = "ontat,yx700wv03",
+ .data = &ontat_yx700wv03,
+ }, {
.compatible = "ortustech,com43h4m85ulc",
.data = &ortustech_com43h4m85ulc,
}, {
@@ -1344,6 +1490,12 @@
.compatible = "shelly,sca07010-bfn-lnn",
.data = &shelly_sca07010_bfn_lnn,
}, {
+ .compatible = "tpk,f07a-0102",
+ .data = &tpk_f07a_0102,
+ }, {
+ .compatible = "tpk,f10a-0102",
+ .data = &tpk_f10a_0102,
+ }, {
.compatible = "urt,umsh-8596md-t",
.data = &urt_umsh_8596md_parallel,
}, {
diff --git a/drivers/gpu/drm/qxl/qxl_cmd.c b/drivers/gpu/drm/qxl/qxl_cmd.c
index fdc1833..b5d4b41 100644
--- a/drivers/gpu/drm/qxl/qxl_cmd.c
+++ b/drivers/gpu/drm/qxl/qxl_cmd.c
@@ -624,7 +624,7 @@
if (stall)
mutex_unlock(&qdev->surf_evict_mutex);
- ret = ttm_bo_wait(&surf->tbo, true, true, !stall);
+ ret = ttm_bo_wait(&surf->tbo, true, !stall);
if (stall)
mutex_lock(&qdev->surf_evict_mutex);
diff --git a/drivers/gpu/drm/qxl/qxl_display.c b/drivers/gpu/drm/qxl/qxl_display.c
index 030409a..8b5d543 100644
--- a/drivers/gpu/drm/qxl/qxl_display.c
+++ b/drivers/gpu/drm/qxl/qxl_display.c
@@ -318,7 +318,7 @@
if (!handle)
return qxl_hide_cursor(qdev);
- obj = drm_gem_object_lookup(crtc->dev, file_priv, handle);
+ obj = drm_gem_object_lookup(file_priv, handle);
if (!obj) {
DRM_ERROR("cannot find cursor object\n");
return -ENOENT;
@@ -465,7 +465,7 @@
.page_flip = qxl_crtc_page_flip,
};
-static void qxl_user_framebuffer_destroy(struct drm_framebuffer *fb)
+void qxl_user_framebuffer_destroy(struct drm_framebuffer *fb)
{
struct qxl_framebuffer *qxl_fb = to_qxl_framebuffer(fb);
@@ -527,12 +527,13 @@
qxl_framebuffer_init(struct drm_device *dev,
struct qxl_framebuffer *qfb,
const struct drm_mode_fb_cmd2 *mode_cmd,
- struct drm_gem_object *obj)
+ struct drm_gem_object *obj,
+ const struct drm_framebuffer_funcs *funcs)
{
int ret;
qfb->obj = obj;
- ret = drm_framebuffer_init(dev, &qfb->base, &qxl_fb_funcs);
+ ret = drm_framebuffer_init(dev, &qfb->base, funcs);
if (ret) {
qfb->obj = NULL;
return ret;
@@ -993,13 +994,15 @@
struct qxl_framebuffer *qxl_fb;
int ret;
- obj = drm_gem_object_lookup(dev, file_priv, mode_cmd->handles[0]);
+ obj = drm_gem_object_lookup(file_priv, mode_cmd->handles[0]);
+ if (!obj)
+ return NULL;
qxl_fb = kzalloc(sizeof(*qxl_fb), GFP_KERNEL);
if (qxl_fb == NULL)
return NULL;
- ret = qxl_framebuffer_init(dev, qxl_fb, mode_cmd, obj);
+ ret = qxl_framebuffer_init(dev, qxl_fb, mode_cmd, obj, &qxl_fb_funcs);
if (ret) {
kfree(qxl_fb);
drm_gem_object_unreference_unlocked(obj);
diff --git a/drivers/gpu/drm/qxl/qxl_drv.c b/drivers/gpu/drm/qxl/qxl_drv.c
index 7307b07..dc9df5f 100644
--- a/drivers/gpu/drm/qxl/qxl_drv.c
+++ b/drivers/gpu/drm/qxl/qxl_drv.c
@@ -272,10 +272,8 @@
static int __init qxl_init(void)
{
-#ifdef CONFIG_VGA_CONSOLE
if (vgacon_text_force() && qxl_modeset == -1)
return -EINVAL;
-#endif
if (qxl_modeset == 0)
return -EINVAL;
diff --git a/drivers/gpu/drm/qxl/qxl_drv.h b/drivers/gpu/drm/qxl/qxl_drv.h
index 3f3897e..3ad6604 100644
--- a/drivers/gpu/drm/qxl/qxl_drv.h
+++ b/drivers/gpu/drm/qxl/qxl_drv.h
@@ -324,8 +324,6 @@
struct workqueue_struct *gc_queue;
struct work_struct gc_work;
- struct work_struct fb_work;
-
struct drm_property *hotplug_mode_update_property;
int monitors_config_width;
int monitors_config_height;
@@ -389,11 +387,13 @@
void qxl_fbdev_set_suspend(struct qxl_device *qdev, int state);
/* qxl_display.c */
+void qxl_user_framebuffer_destroy(struct drm_framebuffer *fb);
int
qxl_framebuffer_init(struct drm_device *dev,
struct qxl_framebuffer *rfb,
const struct drm_mode_fb_cmd2 *mode_cmd,
- struct drm_gem_object *obj);
+ struct drm_gem_object *obj,
+ const struct drm_framebuffer_funcs *funcs);
void qxl_display_read_client_monitors_config(struct qxl_device *qdev);
void qxl_send_monitors_config(struct qxl_device *qdev);
int qxl_create_monitors_object(struct qxl_device *qdev);
@@ -553,7 +553,6 @@
irqreturn_t qxl_irq_handler(int irq, void *arg);
/* qxl_fb.c */
-int qxl_fb_init(struct qxl_device *qdev);
bool qxl_fbdev_qobj_is_fb(struct qxl_device *qdev, struct qxl_bo *qobj);
int qxl_debugfs_add_files(struct qxl_device *qdev,
diff --git a/drivers/gpu/drm/qxl/qxl_dumb.c b/drivers/gpu/drm/qxl/qxl_dumb.c
index d34bb41..5e65d5d 100644
--- a/drivers/gpu/drm/qxl/qxl_dumb.c
+++ b/drivers/gpu/drm/qxl/qxl_dumb.c
@@ -76,7 +76,7 @@
struct qxl_bo *qobj;
BUG_ON(!offset_p);
- gobj = drm_gem_object_lookup(dev, file_priv, handle);
+ gobj = drm_gem_object_lookup(file_priv, handle);
if (gobj == NULL)
return -ENOENT;
qobj = gem_to_qxl_bo(gobj);
diff --git a/drivers/gpu/drm/qxl/qxl_fb.c b/drivers/gpu/drm/qxl/qxl_fb.c
index 7136e52..5ea57f6 100644
--- a/drivers/gpu/drm/qxl/qxl_fb.c
+++ b/drivers/gpu/drm/qxl/qxl_fb.c
@@ -46,15 +46,6 @@
struct list_head delayed_ops;
void *shadow;
int size;
-
- /* dirty memory logging */
- struct {
- spinlock_t lock;
- unsigned x1;
- unsigned y1;
- unsigned x2;
- unsigned y2;
- } dirty;
};
static void qxl_fb_image_init(struct qxl_fb_image *qxl_fb_image,
@@ -82,169 +73,18 @@
}
}
-static void qxl_fb_dirty_flush(struct fb_info *info)
-{
- struct qxl_fbdev *qfbdev = info->par;
- struct qxl_device *qdev = qfbdev->qdev;
- struct qxl_fb_image qxl_fb_image;
- struct fb_image *image = &qxl_fb_image.fb_image;
- unsigned long flags;
- u32 x1, x2, y1, y2;
-
- /* TODO: hard coding 32 bpp */
- int stride = qfbdev->qfb.base.pitches[0];
-
- spin_lock_irqsave(&qfbdev->dirty.lock, flags);
-
- x1 = qfbdev->dirty.x1;
- x2 = qfbdev->dirty.x2;
- y1 = qfbdev->dirty.y1;
- y2 = qfbdev->dirty.y2;
- qfbdev->dirty.x1 = 0;
- qfbdev->dirty.x2 = 0;
- qfbdev->dirty.y1 = 0;
- qfbdev->dirty.y2 = 0;
-
- spin_unlock_irqrestore(&qfbdev->dirty.lock, flags);
-
- /*
- * we are using a shadow draw buffer, at qdev->surface0_shadow
- */
- qxl_io_log(qdev, "dirty x[%d, %d], y[%d, %d]", x1, x2, y1, y2);
- image->dx = x1;
- image->dy = y1;
- image->width = x2 - x1 + 1;
- image->height = y2 - y1 + 1;
- image->fg_color = 0xffffffff; /* unused, just to avoid uninitialized
- warnings */
- image->bg_color = 0;
- image->depth = 32; /* TODO: take from somewhere? */
- image->cmap.start = 0;
- image->cmap.len = 0;
- image->cmap.red = NULL;
- image->cmap.green = NULL;
- image->cmap.blue = NULL;
- image->cmap.transp = NULL;
- image->data = qfbdev->shadow + (x1 * 4) + (stride * y1);
-
- qxl_fb_image_init(&qxl_fb_image, qdev, info, NULL);
- qxl_draw_opaque_fb(&qxl_fb_image, stride);
-}
-
-static void qxl_dirty_update(struct qxl_fbdev *qfbdev,
- int x, int y, int width, int height)
-{
- struct qxl_device *qdev = qfbdev->qdev;
- unsigned long flags;
- int x2, y2;
-
- x2 = x + width - 1;
- y2 = y + height - 1;
-
- spin_lock_irqsave(&qfbdev->dirty.lock, flags);
-
- if ((qfbdev->dirty.y2 - qfbdev->dirty.y1) &&
- (qfbdev->dirty.x2 - qfbdev->dirty.x1)) {
- if (qfbdev->dirty.y1 < y)
- y = qfbdev->dirty.y1;
- if (qfbdev->dirty.y2 > y2)
- y2 = qfbdev->dirty.y2;
- if (qfbdev->dirty.x1 < x)
- x = qfbdev->dirty.x1;
- if (qfbdev->dirty.x2 > x2)
- x2 = qfbdev->dirty.x2;
- }
-
- qfbdev->dirty.x1 = x;
- qfbdev->dirty.x2 = x2;
- qfbdev->dirty.y1 = y;
- qfbdev->dirty.y2 = y2;
-
- spin_unlock_irqrestore(&qfbdev->dirty.lock, flags);
-
- schedule_work(&qdev->fb_work);
-}
-
-static void qxl_deferred_io(struct fb_info *info,
- struct list_head *pagelist)
-{
- struct qxl_fbdev *qfbdev = info->par;
- unsigned long start, end, min, max;
- struct page *page;
- int y1, y2;
-
- min = ULONG_MAX;
- max = 0;
- list_for_each_entry(page, pagelist, lru) {
- start = page->index << PAGE_SHIFT;
- end = start + PAGE_SIZE - 1;
- min = min(min, start);
- max = max(max, end);
- }
-
- if (min < max) {
- y1 = min / info->fix.line_length;
- y2 = (max / info->fix.line_length) + 1;
- qxl_dirty_update(qfbdev, 0, y1, info->var.xres, y2 - y1);
- }
-};
-
static struct fb_deferred_io qxl_defio = {
.delay = QXL_DIRTY_DELAY,
- .deferred_io = qxl_deferred_io,
+ .deferred_io = drm_fb_helper_deferred_io,
};
-static void qxl_fb_fillrect(struct fb_info *info,
- const struct fb_fillrect *rect)
-{
- struct qxl_fbdev *qfbdev = info->par;
-
- drm_fb_helper_sys_fillrect(info, rect);
- qxl_dirty_update(qfbdev, rect->dx, rect->dy, rect->width,
- rect->height);
-}
-
-static void qxl_fb_copyarea(struct fb_info *info,
- const struct fb_copyarea *area)
-{
- struct qxl_fbdev *qfbdev = info->par;
-
- drm_fb_helper_sys_copyarea(info, area);
- qxl_dirty_update(qfbdev, area->dx, area->dy, area->width,
- area->height);
-}
-
-static void qxl_fb_imageblit(struct fb_info *info,
- const struct fb_image *image)
-{
- struct qxl_fbdev *qfbdev = info->par;
-
- drm_fb_helper_sys_imageblit(info, image);
- qxl_dirty_update(qfbdev, image->dx, image->dy, image->width,
- image->height);
-}
-
-static void qxl_fb_work(struct work_struct *work)
-{
- struct qxl_device *qdev = container_of(work, struct qxl_device, fb_work);
- struct qxl_fbdev *qfbdev = qdev->mode_info.qfbdev;
-
- qxl_fb_dirty_flush(qfbdev->helper.fbdev);
-}
-
-int qxl_fb_init(struct qxl_device *qdev)
-{
- INIT_WORK(&qdev->fb_work, qxl_fb_work);
- return 0;
-}
-
static struct fb_ops qxlfb_ops = {
.owner = THIS_MODULE,
.fb_check_var = drm_fb_helper_check_var,
.fb_set_par = drm_fb_helper_set_par, /* TODO: copy vmwgfx */
- .fb_fillrect = qxl_fb_fillrect,
- .fb_copyarea = qxl_fb_copyarea,
- .fb_imageblit = qxl_fb_imageblit,
+ .fb_fillrect = drm_fb_helper_sys_fillrect,
+ .fb_copyarea = drm_fb_helper_sys_copyarea,
+ .fb_imageblit = drm_fb_helper_sys_imageblit,
.fb_pan_display = drm_fb_helper_pan_display,
.fb_blank = drm_fb_helper_blank,
.fb_setcmap = drm_fb_helper_setcmap,
@@ -338,6 +178,57 @@
return ret;
}
+/*
+ * FIXME
+ * It should not be necessary to have a special dirty() callback for fbdev.
+ */
+static int qxlfb_framebuffer_dirty(struct drm_framebuffer *fb,
+ struct drm_file *file_priv,
+ unsigned flags, unsigned color,
+ struct drm_clip_rect *clips,
+ unsigned num_clips)
+{
+ struct qxl_device *qdev = fb->dev->dev_private;
+ struct fb_info *info = qdev->fbdev_info;
+ struct qxl_fbdev *qfbdev = info->par;
+ struct qxl_fb_image qxl_fb_image;
+ struct fb_image *image = &qxl_fb_image.fb_image;
+
+ /* TODO: hard coding 32 bpp */
+ int stride = qfbdev->qfb.base.pitches[0];
+
+ /*
+ * we are using a shadow draw buffer, at qdev->surface0_shadow
+ */
+ qxl_io_log(qdev, "dirty x[%d, %d], y[%d, %d]", clips->x1, clips->x2,
+ clips->y1, clips->y2);
+ image->dx = clips->x1;
+ image->dy = clips->y1;
+ image->width = clips->x2 - clips->x1;
+ image->height = clips->y2 - clips->y1;
+ image->fg_color = 0xffffffff; /* unused, just to avoid uninitialized
+ warnings */
+ image->bg_color = 0;
+ image->depth = 32; /* TODO: take from somewhere? */
+ image->cmap.start = 0;
+ image->cmap.len = 0;
+ image->cmap.red = NULL;
+ image->cmap.green = NULL;
+ image->cmap.blue = NULL;
+ image->cmap.transp = NULL;
+ image->data = qfbdev->shadow + (clips->x1 * 4) + (stride * clips->y1);
+
+ qxl_fb_image_init(&qxl_fb_image, qdev, info, NULL);
+ qxl_draw_opaque_fb(&qxl_fb_image, stride);
+
+ return 0;
+}
+
+static const struct drm_framebuffer_funcs qxlfb_fb_funcs = {
+ .destroy = qxl_user_framebuffer_destroy,
+ .dirty = qxlfb_framebuffer_dirty,
+};
+
static int qxlfb_create(struct qxl_fbdev *qfbdev,
struct drm_fb_helper_surface_size *sizes)
{
@@ -360,6 +251,9 @@
mode_cmd.pixel_format = drm_mode_legacy_fb_format(bpp, depth);
ret = qxlfb_create_pinned_object(qfbdev, &mode_cmd, &gobj);
+ if (ret < 0)
+ return ret;
+
qbo = gem_to_qxl_bo(gobj);
QXL_INFO(qdev, "%s: %dx%d %d\n", __func__, mode_cmd.width,
mode_cmd.height, mode_cmd.pitches[0]);
@@ -383,7 +277,8 @@
info->par = qfbdev;
- qxl_framebuffer_init(qdev->ddev, &qfbdev->qfb, &mode_cmd, gobj);
+ qxl_framebuffer_init(qdev->ddev, &qfbdev->qfb, &mode_cmd, gobj,
+ &qxlfb_fb_funcs);
fb = &qfbdev->qfb.base;
@@ -443,11 +338,11 @@
}
}
if (fb && ret) {
- drm_gem_object_unreference(gobj);
+ drm_gem_object_unreference_unlocked(gobj);
drm_framebuffer_cleanup(fb);
kfree(fb);
}
- drm_gem_object_unreference(gobj);
+ drm_gem_object_unreference_unlocked(gobj);
return ret;
}
@@ -504,7 +399,6 @@
qfbdev->qdev = qdev;
qdev->mode_info.qfbdev = qfbdev;
spin_lock_init(&qfbdev->delayed_ops_lock);
- spin_lock_init(&qfbdev->dirty.lock);
INIT_LIST_HEAD(&qfbdev->delayed_ops);
drm_fb_helper_prepare(qdev->ddev, &qfbdev->helper,
diff --git a/drivers/gpu/drm/qxl/qxl_ioctl.c b/drivers/gpu/drm/qxl/qxl_ioctl.c
index 7c2e782..5a4c8c4 100644
--- a/drivers/gpu/drm/qxl/qxl_ioctl.c
+++ b/drivers/gpu/drm/qxl/qxl_ioctl.c
@@ -107,15 +107,14 @@
}
/* return holding the reference to this object */
-static int qxlhw_handle_to_bo(struct qxl_device *qdev,
- struct drm_file *file_priv, uint64_t handle,
+static int qxlhw_handle_to_bo(struct drm_file *file_priv, uint64_t handle,
struct qxl_release *release, struct qxl_bo **qbo_p)
{
struct drm_gem_object *gobj;
struct qxl_bo *qobj;
int ret;
- gobj = drm_gem_object_lookup(qdev->ddev, file_priv, handle);
+ gobj = drm_gem_object_lookup(file_priv, handle);
if (!gobj)
return -EINVAL;
@@ -221,7 +220,7 @@
reloc_info[i].type = reloc.reloc_type;
if (reloc.dst_handle) {
- ret = qxlhw_handle_to_bo(qdev, file_priv, reloc.dst_handle, release,
+ ret = qxlhw_handle_to_bo(file_priv, reloc.dst_handle, release,
&reloc_info[i].dst_bo);
if (ret)
goto out_free_bos;
@@ -234,7 +233,7 @@
/* reserve and validate the reloc dst bo */
if (reloc.reloc_type == QXL_RELOC_TYPE_BO || reloc.src_handle) {
- ret = qxlhw_handle_to_bo(qdev, file_priv, reloc.src_handle, release,
+ ret = qxlhw_handle_to_bo(file_priv, reloc.src_handle, release,
&reloc_info[i].src_bo);
if (ret)
goto out_free_bos;
@@ -314,7 +313,7 @@
update_area->top >= update_area->bottom)
return -EINVAL;
- gobj = drm_gem_object_lookup(dev, file, update_area->handle);
+ gobj = drm_gem_object_lookup(file, update_area->handle);
if (gobj == NULL)
return -ENOENT;
diff --git a/drivers/gpu/drm/qxl/qxl_kms.c b/drivers/gpu/drm/qxl/qxl_kms.c
index b2977a1..2319800 100644
--- a/drivers/gpu/drm/qxl/qxl_kms.c
+++ b/drivers/gpu/drm/qxl/qxl_kms.c
@@ -261,10 +261,6 @@
qdev->gc_queue = create_singlethread_workqueue("qxl_gc");
INIT_WORK(&qdev->gc_work, qxl_gc_work);
- r = qxl_fb_init(qdev);
- if (r)
- return r;
-
return 0;
}
diff --git a/drivers/gpu/drm/qxl/qxl_object.h b/drivers/gpu/drm/qxl/qxl_object.h
index 37af1bc..4d83113 100644
--- a/drivers/gpu/drm/qxl/qxl_object.h
+++ b/drivers/gpu/drm/qxl/qxl_object.h
@@ -31,7 +31,7 @@
{
int r;
- r = ttm_bo_reserve(&bo->tbo, true, no_wait, false, NULL);
+ r = ttm_bo_reserve(&bo->tbo, true, no_wait, NULL);
if (unlikely(r != 0)) {
if (r != -ERESTARTSYS) {
struct qxl_device *qdev = (struct qxl_device *)bo->gem_base.dev->dev_private;
@@ -67,7 +67,7 @@
{
int r;
- r = ttm_bo_reserve(&bo->tbo, true, no_wait, false, NULL);
+ r = ttm_bo_reserve(&bo->tbo, true, no_wait, NULL);
if (unlikely(r != 0)) {
if (r != -ERESTARTSYS) {
struct qxl_device *qdev = (struct qxl_device *)bo->gem_base.dev->dev_private;
@@ -79,7 +79,7 @@
if (mem_type)
*mem_type = bo->tbo.mem.mem_type;
- r = ttm_bo_wait(&bo->tbo, true, true, no_wait);
+ r = ttm_bo_wait(&bo->tbo, true, no_wait);
ttm_bo_unreserve(&bo->tbo);
return r;
}
diff --git a/drivers/gpu/drm/qxl/qxl_ttm.c b/drivers/gpu/drm/qxl/qxl_ttm.c
index 9534127..0738d74 100644
--- a/drivers/gpu/drm/qxl/qxl_ttm.c
+++ b/drivers/gpu/drm/qxl/qxl_ttm.c
@@ -384,6 +384,8 @@
.io_mem_reserve = &qxl_ttm_io_mem_reserve,
.io_mem_free = &qxl_ttm_io_mem_free,
.move_notify = &qxl_bo_move_notify,
+ .lru_tail = &ttm_bo_default_lru_tail,
+ .swap_lru_tail = &ttm_bo_default_swap_lru_tail,
};
int qxl_ttm_init(struct qxl_device *qdev)
diff --git a/drivers/gpu/drm/radeon/atombios_crtc.c b/drivers/gpu/drm/radeon/atombios_crtc.c
index 532127c..2e216e2 100644
--- a/drivers/gpu/drm/radeon/atombios_crtc.c
+++ b/drivers/gpu/drm/radeon/atombios_crtc.c
@@ -1375,6 +1375,11 @@
break;
}
+ /* Make sure surface address is updated at vertical blank rather than
+ * horizontal blank
+ */
+ WREG32(EVERGREEN_GRPH_FLIP_CONTROL + radeon_crtc->crtc_offset, 0);
+
WREG32(EVERGREEN_GRPH_PRIMARY_SURFACE_ADDRESS_HIGH + radeon_crtc->crtc_offset,
upper_32_bits(fb_location));
WREG32(EVERGREEN_GRPH_SECONDARY_SURFACE_ADDRESS_HIGH + radeon_crtc->crtc_offset,
@@ -1427,12 +1432,6 @@
WREG32(EVERGREEN_VIEWPORT_SIZE + radeon_crtc->crtc_offset,
(viewport_w << 16) | viewport_h);
- /* pageflip setup */
- /* make sure flip is at vb rather than hb */
- tmp = RREG32(EVERGREEN_GRPH_FLIP_CONTROL + radeon_crtc->crtc_offset);
- tmp &= ~EVERGREEN_GRPH_SURFACE_UPDATE_H_RETRACE_EN;
- WREG32(EVERGREEN_GRPH_FLIP_CONTROL + radeon_crtc->crtc_offset, tmp);
-
/* set pageflip to happen only at start of vblank interval (front porch) */
WREG32(EVERGREEN_MASTER_UPDATE_MODE + radeon_crtc->crtc_offset, 3);
@@ -1466,7 +1465,7 @@
uint64_t fb_location;
uint32_t fb_format, fb_pitch_pixels, tiling_flags;
u32 fb_swap = R600_D1GRPH_SWAP_ENDIAN_NONE;
- u32 tmp, viewport_w, viewport_h;
+ u32 viewport_w, viewport_h;
int r;
bool bypass_lut = false;
@@ -1581,6 +1580,11 @@
else
WREG32(AVIVO_D2VGA_CONTROL, 0);
+ /* Make sure surface address is update at vertical blank rather than
+ * horizontal blank
+ */
+ WREG32(AVIVO_D1GRPH_FLIP_CONTROL + radeon_crtc->crtc_offset, 0);
+
if (rdev->family >= CHIP_RV770) {
if (radeon_crtc->crtc_id) {
WREG32(R700_D2GRPH_PRIMARY_SURFACE_ADDRESS_HIGH, upper_32_bits(fb_location));
@@ -1627,12 +1631,6 @@
WREG32(AVIVO_D1MODE_VIEWPORT_SIZE + radeon_crtc->crtc_offset,
(viewport_w << 16) | viewport_h);
- /* pageflip setup */
- /* make sure flip is at vb rather than hb */
- tmp = RREG32(AVIVO_D1GRPH_FLIP_CONTROL + radeon_crtc->crtc_offset);
- tmp &= ~AVIVO_D1GRPH_SURFACE_UPDATE_H_RETRACE_EN;
- WREG32(AVIVO_D1GRPH_FLIP_CONTROL + radeon_crtc->crtc_offset, tmp);
-
/* set pageflip to happen only at start of vblank interval (front porch) */
WREG32(AVIVO_D1MODE_MASTER_UPDATE_MODE + radeon_crtc->crtc_offset, 3);
diff --git a/drivers/gpu/drm/radeon/cik.c b/drivers/gpu/drm/radeon/cik.c
index 8ac82df..ba192a3 100644
--- a/drivers/gpu/drm/radeon/cik.c
+++ b/drivers/gpu/drm/radeon/cik.c
@@ -5261,15 +5261,21 @@
* cik_asic_reset - soft reset GPU
*
* @rdev: radeon_device pointer
+ * @hard: force hard reset
*
* Look up which blocks are hung and attempt
* to reset them.
* Returns 0 for success.
*/
-int cik_asic_reset(struct radeon_device *rdev)
+int cik_asic_reset(struct radeon_device *rdev, bool hard)
{
u32 reset_mask;
+ if (hard) {
+ cik_gpu_pci_config_reset(rdev);
+ return 0;
+ }
+
reset_mask = cik_gpu_check_soft_reset(rdev);
if (reset_mask)
@@ -8137,6 +8143,164 @@
/*
* startup/shutdown callbacks
*/
+static void cik_uvd_init(struct radeon_device *rdev)
+{
+ int r;
+
+ if (!rdev->has_uvd)
+ return;
+
+ r = radeon_uvd_init(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed UVD (%d) init.\n", r);
+ /*
+ * At this point rdev->uvd.vcpu_bo is NULL which trickles down
+ * to early fails cik_uvd_start() and thus nothing happens
+ * there. So it is pointless to try to go through that code
+ * hence why we disable uvd here.
+ */
+ rdev->has_uvd = 0;
+ return;
+ }
+ rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_obj = NULL;
+ r600_ring_init(rdev, &rdev->ring[R600_RING_TYPE_UVD_INDEX], 4096);
+}
+
+static void cik_uvd_start(struct radeon_device *rdev)
+{
+ int r;
+
+ if (!rdev->has_uvd)
+ return;
+
+ r = radeon_uvd_resume(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed UVD resume (%d).\n", r);
+ goto error;
+ }
+ r = uvd_v4_2_resume(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed UVD 4.2 resume (%d).\n", r);
+ goto error;
+ }
+ r = radeon_fence_driver_start_ring(rdev, R600_RING_TYPE_UVD_INDEX);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing UVD fences (%d).\n", r);
+ goto error;
+ }
+ return;
+
+error:
+ rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_size = 0;
+}
+
+static void cik_uvd_resume(struct radeon_device *rdev)
+{
+ struct radeon_ring *ring;
+ int r;
+
+ if (!rdev->has_uvd || !rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_size)
+ return;
+
+ ring = &rdev->ring[R600_RING_TYPE_UVD_INDEX];
+ r = radeon_ring_init(rdev, ring, ring->ring_size, 0, RADEON_CP_PACKET2);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing UVD ring (%d).\n", r);
+ return;
+ }
+ r = uvd_v1_0_init(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing UVD (%d).\n", r);
+ return;
+ }
+}
+
+static void cik_vce_init(struct radeon_device *rdev)
+{
+ int r;
+
+ if (!rdev->has_vce)
+ return;
+
+ r = radeon_vce_init(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed VCE (%d) init.\n", r);
+ /*
+ * At this point rdev->vce.vcpu_bo is NULL which trickles down
+ * to early fails cik_vce_start() and thus nothing happens
+ * there. So it is pointless to try to go through that code
+ * hence why we disable vce here.
+ */
+ rdev->has_vce = 0;
+ return;
+ }
+ rdev->ring[TN_RING_TYPE_VCE1_INDEX].ring_obj = NULL;
+ r600_ring_init(rdev, &rdev->ring[TN_RING_TYPE_VCE1_INDEX], 4096);
+ rdev->ring[TN_RING_TYPE_VCE2_INDEX].ring_obj = NULL;
+ r600_ring_init(rdev, &rdev->ring[TN_RING_TYPE_VCE2_INDEX], 4096);
+}
+
+static void cik_vce_start(struct radeon_device *rdev)
+{
+ int r;
+
+ if (!rdev->has_vce)
+ return;
+
+ r = radeon_vce_resume(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed VCE resume (%d).\n", r);
+ goto error;
+ }
+ r = vce_v2_0_resume(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed VCE resume (%d).\n", r);
+ goto error;
+ }
+ r = radeon_fence_driver_start_ring(rdev, TN_RING_TYPE_VCE1_INDEX);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing VCE1 fences (%d).\n", r);
+ goto error;
+ }
+ r = radeon_fence_driver_start_ring(rdev, TN_RING_TYPE_VCE2_INDEX);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing VCE2 fences (%d).\n", r);
+ goto error;
+ }
+ return;
+
+error:
+ rdev->ring[TN_RING_TYPE_VCE1_INDEX].ring_size = 0;
+ rdev->ring[TN_RING_TYPE_VCE2_INDEX].ring_size = 0;
+}
+
+static void cik_vce_resume(struct radeon_device *rdev)
+{
+ struct radeon_ring *ring;
+ int r;
+
+ if (!rdev->has_vce || !rdev->ring[TN_RING_TYPE_VCE1_INDEX].ring_size)
+ return;
+
+ ring = &rdev->ring[TN_RING_TYPE_VCE1_INDEX];
+ r = radeon_ring_init(rdev, ring, ring->ring_size, 0, VCE_CMD_NO_OP);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing VCE1 ring (%d).\n", r);
+ return;
+ }
+ ring = &rdev->ring[TN_RING_TYPE_VCE2_INDEX];
+ r = radeon_ring_init(rdev, ring, ring->ring_size, 0, VCE_CMD_NO_OP);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing VCE1 ring (%d).\n", r);
+ return;
+ }
+ r = vce_v1_0_init(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing VCE (%d).\n", r);
+ return;
+ }
+}
+
/**
* cik_startup - program the asic to a functional state
*
@@ -8239,34 +8403,8 @@
return r;
}
- r = radeon_uvd_resume(rdev);
- if (!r) {
- r = uvd_v4_2_resume(rdev);
- if (!r) {
- r = radeon_fence_driver_start_ring(rdev,
- R600_RING_TYPE_UVD_INDEX);
- if (r)
- dev_err(rdev->dev, "UVD fences init error (%d).\n", r);
- }
- }
- if (r)
- rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_size = 0;
-
- r = radeon_vce_resume(rdev);
- if (!r) {
- r = vce_v2_0_resume(rdev);
- if (!r)
- r = radeon_fence_driver_start_ring(rdev,
- TN_RING_TYPE_VCE1_INDEX);
- if (!r)
- r = radeon_fence_driver_start_ring(rdev,
- TN_RING_TYPE_VCE2_INDEX);
- }
- if (r) {
- dev_err(rdev->dev, "VCE init error (%d).\n", r);
- rdev->ring[TN_RING_TYPE_VCE1_INDEX].ring_size = 0;
- rdev->ring[TN_RING_TYPE_VCE2_INDEX].ring_size = 0;
- }
+ cik_uvd_start(rdev);
+ cik_vce_start(rdev);
/* Enable IRQ */
if (!rdev->irq.installed) {
@@ -8342,32 +8480,8 @@
if (r)
return r;
- ring = &rdev->ring[R600_RING_TYPE_UVD_INDEX];
- if (ring->ring_size) {
- r = radeon_ring_init(rdev, ring, ring->ring_size, 0,
- RADEON_CP_PACKET2);
- if (!r)
- r = uvd_v1_0_init(rdev);
- if (r)
- DRM_ERROR("radeon: failed initializing UVD (%d).\n", r);
- }
-
- r = -ENOENT;
-
- ring = &rdev->ring[TN_RING_TYPE_VCE1_INDEX];
- if (ring->ring_size)
- r = radeon_ring_init(rdev, ring, ring->ring_size, 0,
- VCE_CMD_NO_OP);
-
- ring = &rdev->ring[TN_RING_TYPE_VCE2_INDEX];
- if (ring->ring_size)
- r = radeon_ring_init(rdev, ring, ring->ring_size, 0,
- VCE_CMD_NO_OP);
-
- if (!r)
- r = vce_v1_0_init(rdev);
- else if (r != -ENOENT)
- DRM_ERROR("radeon: failed initializing VCE (%d).\n", r);
+ cik_uvd_resume(rdev);
+ cik_vce_resume(rdev);
r = radeon_ib_pool_init(rdev);
if (r) {
@@ -8443,9 +8557,12 @@
radeon_vm_manager_fini(rdev);
cik_cp_enable(rdev, false);
cik_sdma_enable(rdev, false);
- uvd_v1_0_fini(rdev);
- radeon_uvd_suspend(rdev);
- radeon_vce_suspend(rdev);
+ if (rdev->has_uvd) {
+ uvd_v1_0_fini(rdev);
+ radeon_uvd_suspend(rdev);
+ }
+ if (rdev->has_vce)
+ radeon_vce_suspend(rdev);
cik_fini_pg(rdev);
cik_fini_cg(rdev);
cik_irq_suspend(rdev);
@@ -8571,23 +8688,8 @@
ring->ring_obj = NULL;
r600_ring_init(rdev, ring, 256 * 1024);
- r = radeon_uvd_init(rdev);
- if (!r) {
- ring = &rdev->ring[R600_RING_TYPE_UVD_INDEX];
- ring->ring_obj = NULL;
- r600_ring_init(rdev, ring, 4096);
- }
-
- r = radeon_vce_init(rdev);
- if (!r) {
- ring = &rdev->ring[TN_RING_TYPE_VCE1_INDEX];
- ring->ring_obj = NULL;
- r600_ring_init(rdev, ring, 4096);
-
- ring = &rdev->ring[TN_RING_TYPE_VCE2_INDEX];
- ring->ring_obj = NULL;
- r600_ring_init(rdev, ring, 4096);
- }
+ cik_uvd_init(rdev);
+ cik_vce_init(rdev);
rdev->ih.ring_obj = NULL;
r600_ih_ring_init(rdev, 64 * 1024);
diff --git a/drivers/gpu/drm/radeon/cikd.h b/drivers/gpu/drm/radeon/cikd.h
index 391ff9d..cead228 100644
--- a/drivers/gpu/drm/radeon/cikd.h
+++ b/drivers/gpu/drm/radeon/cikd.h
@@ -2071,6 +2071,7 @@
#define UVD_UDEC_DBW_ADDR_CONFIG 0xef54
#define UVD_LMI_EXT40_ADDR 0xf498
+#define UVD_GP_SCRATCH4 0xf4e0
#define UVD_LMI_ADDR_EXT 0xf594
#define UVD_VCPU_CACHE_OFFSET0 0xf608
#define UVD_VCPU_CACHE_SIZE0 0xf60c
diff --git a/drivers/gpu/drm/radeon/evergreen.c b/drivers/gpu/drm/radeon/evergreen.c
index 34f7a29..db275b7 100644
--- a/drivers/gpu/drm/radeon/evergreen.c
+++ b/drivers/gpu/drm/radeon/evergreen.c
@@ -1407,11 +1407,14 @@
* Triggers the actual pageflip by updating the primary
* surface base address (evergreen+).
*/
-void evergreen_page_flip(struct radeon_device *rdev, int crtc_id, u64 crtc_base)
+void evergreen_page_flip(struct radeon_device *rdev, int crtc_id, u64 crtc_base,
+ bool async)
{
struct radeon_crtc *radeon_crtc = rdev->mode_info.crtcs[crtc_id];
/* update the scanout addresses */
+ WREG32(EVERGREEN_GRPH_FLIP_CONTROL + radeon_crtc->crtc_offset,
+ async ? EVERGREEN_GRPH_SURFACE_UPDATE_H_RETRACE_EN : 0);
WREG32(EVERGREEN_GRPH_PRIMARY_SURFACE_ADDRESS_HIGH + radeon_crtc->crtc_offset,
upper_32_bits(crtc_base));
WREG32(EVERGREEN_GRPH_PRIMARY_SURFACE_ADDRESS + radeon_crtc->crtc_offset,
@@ -1864,7 +1867,8 @@
break;
}
radeon_hpd_set_polarity(rdev, radeon_connector->hpd.hpd);
- enabled |= 1 << radeon_connector->hpd.hpd;
+ if (radeon_connector->hpd.hpd != RADEON_HPD_NONE)
+ enabled |= 1 << radeon_connector->hpd.hpd;
}
radeon_irq_kms_enable_hpd(rdev, enabled);
}
@@ -1907,7 +1911,8 @@
default:
break;
}
- disabled |= 1 << radeon_connector->hpd.hpd;
+ if (radeon_connector->hpd.hpd != RADEON_HPD_NONE)
+ disabled |= 1 << radeon_connector->hpd.hpd;
}
radeon_irq_kms_disable_hpd(rdev, disabled);
}
@@ -4136,10 +4141,15 @@
}
}
-int evergreen_asic_reset(struct radeon_device *rdev)
+int evergreen_asic_reset(struct radeon_device *rdev, bool hard)
{
u32 reset_mask;
+ if (hard) {
+ evergreen_gpu_pci_config_reset(rdev);
+ return 0;
+ }
+
reset_mask = evergreen_gpu_check_soft_reset(rdev);
if (reset_mask)
@@ -5515,6 +5525,73 @@
return IRQ_HANDLED;
}
+static void evergreen_uvd_init(struct radeon_device *rdev)
+{
+ int r;
+
+ if (!rdev->has_uvd)
+ return;
+
+ r = radeon_uvd_init(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed UVD (%d) init.\n", r);
+ /*
+ * At this point rdev->uvd.vcpu_bo is NULL which trickles down
+ * to early fails uvd_v2_2_resume() and thus nothing happens
+ * there. So it is pointless to try to go through that code
+ * hence why we disable uvd here.
+ */
+ rdev->has_uvd = 0;
+ return;
+ }
+ rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_obj = NULL;
+ r600_ring_init(rdev, &rdev->ring[R600_RING_TYPE_UVD_INDEX], 4096);
+}
+
+static void evergreen_uvd_start(struct radeon_device *rdev)
+{
+ int r;
+
+ if (!rdev->has_uvd)
+ return;
+
+ r = uvd_v2_2_resume(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed UVD resume (%d).\n", r);
+ goto error;
+ }
+ r = radeon_fence_driver_start_ring(rdev, R600_RING_TYPE_UVD_INDEX);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing UVD fences (%d).\n", r);
+ goto error;
+ }
+ return;
+
+error:
+ rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_size = 0;
+}
+
+static void evergreen_uvd_resume(struct radeon_device *rdev)
+{
+ struct radeon_ring *ring;
+ int r;
+
+ if (!rdev->has_uvd || !rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_size)
+ return;
+
+ ring = &rdev->ring[R600_RING_TYPE_UVD_INDEX];
+ r = radeon_ring_init(rdev, ring, ring->ring_size, 0, RADEON_CP_PACKET2);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing UVD ring (%d).\n", r);
+ return;
+ }
+ r = uvd_v1_0_init(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing UVD (%d).\n", r);
+ return;
+ }
+}
+
static int evergreen_startup(struct radeon_device *rdev)
{
struct radeon_ring *ring;
@@ -5579,16 +5656,7 @@
return r;
}
- r = uvd_v2_2_resume(rdev);
- if (!r) {
- r = radeon_fence_driver_start_ring(rdev,
- R600_RING_TYPE_UVD_INDEX);
- if (r)
- dev_err(rdev->dev, "UVD fences init error (%d).\n", r);
- }
-
- if (r)
- rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_size = 0;
+ evergreen_uvd_start(rdev);
/* Enable IRQ */
if (!rdev->irq.installed) {
@@ -5627,16 +5695,7 @@
if (r)
return r;
- ring = &rdev->ring[R600_RING_TYPE_UVD_INDEX];
- if (ring->ring_size) {
- r = radeon_ring_init(rdev, ring, ring->ring_size, 0,
- RADEON_CP_PACKET2);
- if (!r)
- r = uvd_v1_0_init(rdev);
-
- if (r)
- DRM_ERROR("radeon: error initializing UVD (%d).\n", r);
- }
+ evergreen_uvd_resume(rdev);
r = radeon_ib_pool_init(rdev);
if (r) {
@@ -5691,8 +5750,10 @@
{
radeon_pm_suspend(rdev);
radeon_audio_fini(rdev);
- uvd_v1_0_fini(rdev);
- radeon_uvd_suspend(rdev);
+ if (rdev->has_uvd) {
+ uvd_v1_0_fini(rdev);
+ radeon_uvd_suspend(rdev);
+ }
r700_cp_stop(rdev);
r600_dma_stop(rdev);
evergreen_irq_suspend(rdev);
@@ -5793,12 +5854,7 @@
rdev->ring[R600_RING_TYPE_DMA_INDEX].ring_obj = NULL;
r600_ring_init(rdev, &rdev->ring[R600_RING_TYPE_DMA_INDEX], 64 * 1024);
- r = radeon_uvd_init(rdev);
- if (!r) {
- rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_obj = NULL;
- r600_ring_init(rdev, &rdev->ring[R600_RING_TYPE_UVD_INDEX],
- 4096);
- }
+ evergreen_uvd_init(rdev);
rdev->ih.ring_obj = NULL;
r600_ih_ring_init(rdev, 64 * 1024);
diff --git a/drivers/gpu/drm/radeon/evergreen_cs.c b/drivers/gpu/drm/radeon/evergreen_cs.c
index 9e93205..0d3f744 100644
--- a/drivers/gpu/drm/radeon/evergreen_cs.c
+++ b/drivers/gpu/drm/radeon/evergreen_cs.c
@@ -2608,6 +2608,51 @@
}
}
break;
+ case PACKET3_SET_APPEND_CNT:
+ {
+ uint32_t areg;
+ uint32_t allowed_reg_base;
+ uint32_t source_sel;
+ if (pkt->count != 2) {
+ DRM_ERROR("bad SET_APPEND_CNT (invalid count)\n");
+ return -EINVAL;
+ }
+
+ allowed_reg_base = GDS_APPEND_COUNT_0;
+ allowed_reg_base -= PACKET3_SET_CONTEXT_REG_START;
+ allowed_reg_base >>= 2;
+
+ areg = idx_value >> 16;
+ if (areg < allowed_reg_base || areg > (allowed_reg_base + 11)) {
+ dev_warn(p->dev, "forbidden register for append cnt 0x%08x at %d\n",
+ areg, idx);
+ return -EINVAL;
+ }
+
+ source_sel = G_PACKET3_SET_APPEND_CNT_SRC_SELECT(idx_value);
+ if (source_sel == PACKET3_SAC_SRC_SEL_MEM) {
+ uint64_t offset;
+ uint32_t swap;
+ r = radeon_cs_packet_next_reloc(p, &reloc, 0);
+ if (r) {
+ DRM_ERROR("bad SET_APPEND_CNT (missing reloc)\n");
+ return -EINVAL;
+ }
+ offset = radeon_get_ib_value(p, idx + 1);
+ swap = offset & 0x3;
+ offset &= ~0x3;
+
+ offset += ((u64)(radeon_get_ib_value(p, idx + 2) & 0xff)) << 32;
+
+ offset += reloc->gpu_offset;
+ ib[idx+1] = (offset & 0xfffffffc) | swap;
+ ib[idx+2] = upper_32_bits(offset) & 0xff;
+ } else {
+ DRM_ERROR("bad SET_APPEND_CNT (unsupported operation)\n");
+ return -EINVAL;
+ }
+ break;
+ }
case PACKET3_NOP:
break;
default:
@@ -3438,6 +3483,27 @@
}
}
break;
+ case PACKET3_SET_APPEND_CNT: {
+ uint32_t areg;
+ uint32_t allowed_reg_base;
+
+ if (pkt->count != 2) {
+ DRM_ERROR("bad SET_APPEND_CNT (invalid count)\n");
+ return -EINVAL;
+ }
+
+ allowed_reg_base = GDS_APPEND_COUNT_0;
+ allowed_reg_base -= PACKET3_SET_CONTEXT_REG_START;
+ allowed_reg_base >>= 2;
+
+ areg = idx_value >> 16;
+ if (areg < allowed_reg_base || areg > (allowed_reg_base + 11)) {
+ DRM_ERROR("forbidden register for append cnt 0x%08x at %d\n",
+ areg, idx);
+ return -EINVAL;
+ }
+ break;
+ }
default:
return -EINVAL;
}
diff --git a/drivers/gpu/drm/radeon/evergreend.h b/drivers/gpu/drm/radeon/evergreend.h
index 13b6029..0b174e1 100644
--- a/drivers/gpu/drm/radeon/evergreend.h
+++ b/drivers/gpu/drm/radeon/evergreend.h
@@ -1689,6 +1689,36 @@
#define PACKET3_SET_CONTEXT_REG_INDIRECT 0x73
#define PACKET3_SET_RESOURCE_INDIRECT 0x74
#define PACKET3_SET_APPEND_CNT 0x75
+/* SET_APPEND_CNT - documentation
+ * 1. header
+ * 2. COMMAND
+ * 1:0 - SOURCE SEL
+ * 15:2 - Reserved
+ * 31:16 - WR_REG_OFFSET - context register to write source data to.
+ * (one of R_02872C_GDS_APPEND_COUNT_0-11)
+ * 3. CONTROL
+ * (for source == mem)
+ * 31:2 SRC_ADDRESS_LO
+ * 0:1 SWAP
+ * (for source == GDS)
+ * 31:0 GDS offset
+ * (for source == DATA)
+ * 31:0 DATA
+ * (for source == REG)
+ * 31:0 REG
+ * 4. SRC_ADDRESS_HI[7:0]
+ * kernel driver 2.44 only supports SRC == MEM.
+ */
+#define PACKET3_SET_APPEND_CNT_SRC_SELECT(x) ((x) << 0)
+#define G_PACKET3_SET_APPEND_CNT_SRC_SELECT(x) ((x & 0x3) >> 0)
+/* source is from the data in CONTROL */
+#define PACKET3_SAC_SRC_SEL_DATA 0x0
+/* source is from register */
+#define PACKET3_SAC_SRC_SEL_REG 0x1
+/* source is from GDS offset in CONTROL */
+#define PACKET3_SAC_SRC_SEL_GDS 0x2
+/* source is from memory address */
+#define PACKET3_SAC_SRC_SEL_MEM 0x3
#define SQ_RESOURCE_CONSTANT_WORD7_0 0x3001c
#define S__SQ_CONSTANT_TYPE(x) (((x) & 3) << 30)
@@ -2005,6 +2035,19 @@
#define GDS_ADDR_BASE 0x28720
+#define GDS_APPEND_COUNT_0 0x2872C
+#define GDS_APPEND_COUNT_1 0x28730
+#define GDS_APPEND_COUNT_2 0x28734
+#define GDS_APPEND_COUNT_3 0x28738
+#define GDS_APPEND_COUNT_4 0x2873C
+#define GDS_APPEND_COUNT_5 0x28740
+#define GDS_APPEND_COUNT_6 0x28744
+#define GDS_APPEND_COUNT_7 0x28748
+#define GDS_APPEND_COUNT_8 0x2874c
+#define GDS_APPEND_COUNT_9 0x28750
+#define GDS_APPEND_COUNT_10 0x28754
+#define GDS_APPEND_COUNT_11 0x28758
+
#define CB_IMMED0_BASE 0x28b9c
#define CB_IMMED1_BASE 0x28ba0
#define CB_IMMED2_BASE 0x28ba4
diff --git a/drivers/gpu/drm/radeon/ni.c b/drivers/gpu/drm/radeon/ni.c
index b88d63c9..4a3d7ca 100644
--- a/drivers/gpu/drm/radeon/ni.c
+++ b/drivers/gpu/drm/radeon/ni.c
@@ -1959,10 +1959,15 @@
evergreen_print_gpu_status_regs(rdev);
}
-int cayman_asic_reset(struct radeon_device *rdev)
+int cayman_asic_reset(struct radeon_device *rdev, bool hard)
{
u32 reset_mask;
+ if (hard) {
+ evergreen_gpu_pci_config_reset(rdev);
+ return 0;
+ }
+
reset_mask = cayman_gpu_check_soft_reset(rdev);
if (reset_mask)
@@ -2002,6 +2007,160 @@
return radeon_ring_test_lockup(rdev, ring);
}
+static void cayman_uvd_init(struct radeon_device *rdev)
+{
+ int r;
+
+ if (!rdev->has_uvd)
+ return;
+
+ r = radeon_uvd_init(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed UVD (%d) init.\n", r);
+ /*
+ * At this point rdev->uvd.vcpu_bo is NULL which trickles down
+ * to early fails uvd_v2_2_resume() and thus nothing happens
+ * there. So it is pointless to try to go through that code
+ * hence why we disable uvd here.
+ */
+ rdev->has_uvd = 0;
+ return;
+ }
+ rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_obj = NULL;
+ r600_ring_init(rdev, &rdev->ring[R600_RING_TYPE_UVD_INDEX], 4096);
+}
+
+static void cayman_uvd_start(struct radeon_device *rdev)
+{
+ int r;
+
+ if (!rdev->has_uvd)
+ return;
+
+ r = uvd_v2_2_resume(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed UVD resume (%d).\n", r);
+ goto error;
+ }
+ r = radeon_fence_driver_start_ring(rdev, R600_RING_TYPE_UVD_INDEX);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing UVD fences (%d).\n", r);
+ goto error;
+ }
+ return;
+
+error:
+ rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_size = 0;
+}
+
+static void cayman_uvd_resume(struct radeon_device *rdev)
+{
+ struct radeon_ring *ring;
+ int r;
+
+ if (!rdev->has_uvd || !rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_size)
+ return;
+
+ ring = &rdev->ring[R600_RING_TYPE_UVD_INDEX];
+ r = radeon_ring_init(rdev, ring, ring->ring_size, 0, RADEON_CP_PACKET2);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing UVD ring (%d).\n", r);
+ return;
+ }
+ r = uvd_v1_0_init(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing UVD (%d).\n", r);
+ return;
+ }
+}
+
+static void cayman_vce_init(struct radeon_device *rdev)
+{
+ int r;
+
+ /* Only set for CHIP_ARUBA */
+ if (!rdev->has_vce)
+ return;
+
+ r = radeon_vce_init(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed VCE (%d) init.\n", r);
+ /*
+ * At this point rdev->vce.vcpu_bo is NULL which trickles down
+ * to early fails cayman_vce_start() and thus nothing happens
+ * there. So it is pointless to try to go through that code
+ * hence why we disable vce here.
+ */
+ rdev->has_vce = 0;
+ return;
+ }
+ rdev->ring[TN_RING_TYPE_VCE1_INDEX].ring_obj = NULL;
+ r600_ring_init(rdev, &rdev->ring[TN_RING_TYPE_VCE1_INDEX], 4096);
+ rdev->ring[TN_RING_TYPE_VCE2_INDEX].ring_obj = NULL;
+ r600_ring_init(rdev, &rdev->ring[TN_RING_TYPE_VCE2_INDEX], 4096);
+}
+
+static void cayman_vce_start(struct radeon_device *rdev)
+{
+ int r;
+
+ if (!rdev->has_vce)
+ return;
+
+ r = radeon_vce_resume(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed VCE resume (%d).\n", r);
+ goto error;
+ }
+ r = vce_v1_0_resume(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed VCE resume (%d).\n", r);
+ goto error;
+ }
+ r = radeon_fence_driver_start_ring(rdev, TN_RING_TYPE_VCE1_INDEX);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing VCE1 fences (%d).\n", r);
+ goto error;
+ }
+ r = radeon_fence_driver_start_ring(rdev, TN_RING_TYPE_VCE2_INDEX);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing VCE2 fences (%d).\n", r);
+ goto error;
+ }
+ return;
+
+error:
+ rdev->ring[TN_RING_TYPE_VCE1_INDEX].ring_size = 0;
+ rdev->ring[TN_RING_TYPE_VCE2_INDEX].ring_size = 0;
+}
+
+static void cayman_vce_resume(struct radeon_device *rdev)
+{
+ struct radeon_ring *ring;
+ int r;
+
+ if (!rdev->has_vce || !rdev->ring[TN_RING_TYPE_VCE1_INDEX].ring_size)
+ return;
+
+ ring = &rdev->ring[TN_RING_TYPE_VCE1_INDEX];
+ r = radeon_ring_init(rdev, ring, ring->ring_size, 0, 0x0);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing VCE1 ring (%d).\n", r);
+ return;
+ }
+ ring = &rdev->ring[TN_RING_TYPE_VCE2_INDEX];
+ r = radeon_ring_init(rdev, ring, ring->ring_size, 0, 0x0);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing VCE1 ring (%d).\n", r);
+ return;
+ }
+ r = vce_v1_0_init(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing VCE (%d).\n", r);
+ return;
+ }
+}
+
static int cayman_startup(struct radeon_device *rdev)
{
struct radeon_ring *ring = &rdev->ring[RADEON_RING_TYPE_GFX_INDEX];
@@ -2056,34 +2215,8 @@
return r;
}
- r = uvd_v2_2_resume(rdev);
- if (!r) {
- r = radeon_fence_driver_start_ring(rdev,
- R600_RING_TYPE_UVD_INDEX);
- if (r)
- dev_err(rdev->dev, "UVD fences init error (%d).\n", r);
- }
- if (r)
- rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_size = 0;
-
- if (rdev->family == CHIP_ARUBA) {
- r = radeon_vce_resume(rdev);
- if (!r)
- r = vce_v1_0_resume(rdev);
-
- if (!r)
- r = radeon_fence_driver_start_ring(rdev,
- TN_RING_TYPE_VCE1_INDEX);
- if (!r)
- r = radeon_fence_driver_start_ring(rdev,
- TN_RING_TYPE_VCE2_INDEX);
-
- if (r) {
- dev_err(rdev->dev, "VCE init error (%d).\n", r);
- rdev->ring[TN_RING_TYPE_VCE1_INDEX].ring_size = 0;
- rdev->ring[TN_RING_TYPE_VCE2_INDEX].ring_size = 0;
- }
- }
+ cayman_uvd_start(rdev);
+ cayman_vce_start(rdev);
r = radeon_fence_driver_start_ring(rdev, CAYMAN_RING_TYPE_CP1_INDEX);
if (r) {
@@ -2152,30 +2285,8 @@
if (r)
return r;
- ring = &rdev->ring[R600_RING_TYPE_UVD_INDEX];
- if (ring->ring_size) {
- r = radeon_ring_init(rdev, ring, ring->ring_size, 0,
- RADEON_CP_PACKET2);
- if (!r)
- r = uvd_v1_0_init(rdev);
- if (r)
- DRM_ERROR("radeon: failed initializing UVD (%d).\n", r);
- }
-
- if (rdev->family == CHIP_ARUBA) {
- ring = &rdev->ring[TN_RING_TYPE_VCE1_INDEX];
- if (ring->ring_size)
- r = radeon_ring_init(rdev, ring, ring->ring_size, 0, 0x0);
-
- ring = &rdev->ring[TN_RING_TYPE_VCE2_INDEX];
- if (ring->ring_size)
- r = radeon_ring_init(rdev, ring, ring->ring_size, 0, 0x0);
-
- if (!r)
- r = vce_v1_0_init(rdev);
- if (r)
- DRM_ERROR("radeon: failed initializing VCE (%d).\n", r);
- }
+ cayman_uvd_resume(rdev);
+ cayman_vce_resume(rdev);
r = radeon_ib_pool_init(rdev);
if (r) {
@@ -2230,8 +2341,10 @@
radeon_vm_manager_fini(rdev);
cayman_cp_enable(rdev, false);
cayman_dma_stop(rdev);
- uvd_v1_0_fini(rdev);
- radeon_uvd_suspend(rdev);
+ if (rdev->has_uvd) {
+ uvd_v1_0_fini(rdev);
+ radeon_uvd_suspend(rdev);
+ }
evergreen_irq_suspend(rdev);
radeon_wb_disable(rdev);
cayman_pcie_gart_disable(rdev);
@@ -2325,25 +2438,8 @@
ring->ring_obj = NULL;
r600_ring_init(rdev, ring, 64 * 1024);
- r = radeon_uvd_init(rdev);
- if (!r) {
- ring = &rdev->ring[R600_RING_TYPE_UVD_INDEX];
- ring->ring_obj = NULL;
- r600_ring_init(rdev, ring, 4096);
- }
-
- if (rdev->family == CHIP_ARUBA) {
- r = radeon_vce_init(rdev);
- if (!r) {
- ring = &rdev->ring[TN_RING_TYPE_VCE1_INDEX];
- ring->ring_obj = NULL;
- r600_ring_init(rdev, ring, 4096);
-
- ring = &rdev->ring[TN_RING_TYPE_VCE2_INDEX];
- ring->ring_obj = NULL;
- r600_ring_init(rdev, ring, 4096);
- }
- }
+ cayman_uvd_init(rdev);
+ cayman_vce_init(rdev);
rdev->ih.ring_obj = NULL;
r600_ih_ring_init(rdev, 64 * 1024);
@@ -2398,7 +2494,7 @@
radeon_irq_kms_fini(rdev);
uvd_v1_0_fini(rdev);
radeon_uvd_fini(rdev);
- if (rdev->family == CHIP_ARUBA)
+ if (rdev->has_vce)
radeon_vce_fini(rdev);
cayman_pcie_gart_fini(rdev);
r600_vram_scratch_fini(rdev);
diff --git a/drivers/gpu/drm/radeon/r100.c b/drivers/gpu/drm/radeon/r100.c
index 6e478a2..f25994b 100644
--- a/drivers/gpu/drm/radeon/r100.c
+++ b/drivers/gpu/drm/radeon/r100.c
@@ -153,7 +153,7 @@
* bit to go high, when it does, we release the lock, and allow the
* double buffered update to take place.
*/
-void r100_page_flip(struct radeon_device *rdev, int crtc_id, u64 crtc_base)
+void r100_page_flip(struct radeon_device *rdev, int crtc_id, u64 crtc_base, bool async)
{
struct radeon_crtc *radeon_crtc = rdev->mode_info.crtcs[crtc_id];
u32 tmp = ((u32)crtc_base) | RADEON_CRTC_OFFSET__OFFSET_LOCK;
@@ -592,7 +592,8 @@
list_for_each_entry(connector, &dev->mode_config.connector_list, head) {
struct radeon_connector *radeon_connector = to_radeon_connector(connector);
- enable |= 1 << radeon_connector->hpd.hpd;
+ if (radeon_connector->hpd.hpd != RADEON_HPD_NONE)
+ enable |= 1 << radeon_connector->hpd.hpd;
radeon_hpd_set_polarity(rdev, radeon_connector->hpd.hpd);
}
radeon_irq_kms_enable_hpd(rdev, enable);
@@ -614,7 +615,8 @@
list_for_each_entry(connector, &dev->mode_config.connector_list, head) {
struct radeon_connector *radeon_connector = to_radeon_connector(connector);
- disable |= 1 << radeon_connector->hpd.hpd;
+ if (radeon_connector->hpd.hpd != RADEON_HPD_NONE)
+ disable |= 1 << radeon_connector->hpd.hpd;
}
radeon_irq_kms_disable_hpd(rdev, disable);
}
@@ -2555,7 +2557,7 @@
mdelay(1);
}
-int r100_asic_reset(struct radeon_device *rdev)
+int r100_asic_reset(struct radeon_device *rdev, bool hard)
{
struct r100_mc_save save;
u32 status, tmp;
diff --git a/drivers/gpu/drm/radeon/r300.c b/drivers/gpu/drm/radeon/r300.c
index 718b12b..7e417d8 100644
--- a/drivers/gpu/drm/radeon/r300.c
+++ b/drivers/gpu/drm/radeon/r300.c
@@ -410,7 +410,7 @@
rdev->num_gb_pipes, rdev->num_z_pipes);
}
-int r300_asic_reset(struct radeon_device *rdev)
+int r300_asic_reset(struct radeon_device *rdev, bool hard)
{
struct r100_mc_save save;
u32 status, tmp;
diff --git a/drivers/gpu/drm/radeon/r600.c b/drivers/gpu/drm/radeon/r600.c
index f86ab69..9247e7d 100644
--- a/drivers/gpu/drm/radeon/r600.c
+++ b/drivers/gpu/drm/radeon/r600.c
@@ -1002,7 +1002,8 @@
break;
}
}
- enable |= 1 << radeon_connector->hpd.hpd;
+ if (radeon_connector->hpd.hpd != RADEON_HPD_NONE)
+ enable |= 1 << radeon_connector->hpd.hpd;
radeon_hpd_set_polarity(rdev, radeon_connector->hpd.hpd);
}
radeon_irq_kms_enable_hpd(rdev, enable);
@@ -1055,7 +1056,8 @@
break;
}
}
- disable |= 1 << radeon_connector->hpd.hpd;
+ if (radeon_connector->hpd.hpd != RADEON_HPD_NONE)
+ disable |= 1 << radeon_connector->hpd.hpd;
}
radeon_irq_kms_disable_hpd(rdev, disable);
}
@@ -1871,10 +1873,15 @@
}
}
-int r600_asic_reset(struct radeon_device *rdev)
+int r600_asic_reset(struct radeon_device *rdev, bool hard)
{
u32 reset_mask;
+ if (hard) {
+ r600_gpu_pci_config_reset(rdev);
+ return 0;
+ }
+
reset_mask = r600_gpu_check_soft_reset(rdev);
if (reset_mask)
@@ -3035,6 +3042,73 @@
/* FIXME: implement */
}
+static void r600_uvd_init(struct radeon_device *rdev)
+{
+ int r;
+
+ if (!rdev->has_uvd)
+ return;
+
+ r = radeon_uvd_init(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed UVD (%d) init.\n", r);
+ /*
+ * At this point rdev->uvd.vcpu_bo is NULL which trickles down
+ * to early fails uvd_v1_0_resume() and thus nothing happens
+ * there. So it is pointless to try to go through that code
+ * hence why we disable uvd here.
+ */
+ rdev->has_uvd = 0;
+ return;
+ }
+ rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_obj = NULL;
+ r600_ring_init(rdev, &rdev->ring[R600_RING_TYPE_UVD_INDEX], 4096);
+}
+
+static void r600_uvd_start(struct radeon_device *rdev)
+{
+ int r;
+
+ if (!rdev->has_uvd)
+ return;
+
+ r = uvd_v1_0_resume(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed UVD resume (%d).\n", r);
+ goto error;
+ }
+ r = radeon_fence_driver_start_ring(rdev, R600_RING_TYPE_UVD_INDEX);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing UVD fences (%d).\n", r);
+ goto error;
+ }
+ return;
+
+error:
+ rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_size = 0;
+}
+
+static void r600_uvd_resume(struct radeon_device *rdev)
+{
+ struct radeon_ring *ring;
+ int r;
+
+ if (!rdev->has_uvd || !rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_size)
+ return;
+
+ ring = &rdev->ring[R600_RING_TYPE_UVD_INDEX];
+ r = radeon_ring_init(rdev, ring, ring->ring_size, 0, RADEON_CP_PACKET2);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing UVD ring (%d).\n", r);
+ return;
+ }
+ r = uvd_v1_0_init(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing UVD (%d).\n", r);
+ return;
+ }
+}
+
static int r600_startup(struct radeon_device *rdev)
{
struct radeon_ring *ring;
@@ -3070,17 +3144,7 @@
return r;
}
- if (rdev->has_uvd) {
- r = uvd_v1_0_resume(rdev);
- if (!r) {
- r = radeon_fence_driver_start_ring(rdev, R600_RING_TYPE_UVD_INDEX);
- if (r) {
- dev_err(rdev->dev, "failed initializing UVD fences (%d).\n", r);
- }
- }
- if (r)
- rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_size = 0;
- }
+ r600_uvd_start(rdev);
/* Enable IRQ */
if (!rdev->irq.installed) {
@@ -3110,17 +3174,7 @@
if (r)
return r;
- if (rdev->has_uvd) {
- ring = &rdev->ring[R600_RING_TYPE_UVD_INDEX];
- if (ring->ring_size) {
- r = radeon_ring_init(rdev, ring, ring->ring_size, 0,
- RADEON_CP_PACKET2);
- if (!r)
- r = uvd_v1_0_init(rdev);
- if (r)
- DRM_ERROR("radeon: failed initializing UVD (%d).\n", r);
- }
- }
+ r600_uvd_resume(rdev);
r = radeon_ib_pool_init(rdev);
if (r) {
@@ -3264,13 +3318,7 @@
rdev->ring[RADEON_RING_TYPE_GFX_INDEX].ring_obj = NULL;
r600_ring_init(rdev, &rdev->ring[RADEON_RING_TYPE_GFX_INDEX], 1024 * 1024);
- if (rdev->has_uvd) {
- r = radeon_uvd_init(rdev);
- if (!r) {
- rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_obj = NULL;
- r600_ring_init(rdev, &rdev->ring[R600_RING_TYPE_UVD_INDEX], 4096);
- }
- }
+ r600_uvd_init(rdev);
rdev->ih.ring_obj = NULL;
r600_ih_ring_init(rdev, 64 * 1024);
diff --git a/drivers/gpu/drm/radeon/radeon.h b/drivers/gpu/drm/radeon/radeon.h
index 007be29..80b24a4 100644
--- a/drivers/gpu/drm/radeon/radeon.h
+++ b/drivers/gpu/drm/radeon/radeon.h
@@ -113,6 +113,8 @@
extern int radeon_backlight;
extern int radeon_auxch;
extern int radeon_mst;
+extern int radeon_uvd;
+extern int radeon_vce;
/*
* Copy from radeon_drv.h so we don't have to include both and have conflicting
@@ -744,6 +746,7 @@
struct drm_pending_vblank_event *event;
struct radeon_bo *old_rbo;
struct fence *fence;
+ bool async;
};
struct r500_irq_stat_regs {
@@ -1671,14 +1674,18 @@
/*
* UVD
*/
-#define RADEON_MAX_UVD_HANDLES 10
-#define RADEON_UVD_STACK_SIZE (1024*1024)
-#define RADEON_UVD_HEAP_SIZE (1024*1024)
+#define RADEON_DEFAULT_UVD_HANDLES 10
+#define RADEON_MAX_UVD_HANDLES 30
+#define RADEON_UVD_STACK_SIZE (200*1024)
+#define RADEON_UVD_HEAP_SIZE (256*1024)
+#define RADEON_UVD_SESSION_SIZE (50*1024)
struct radeon_uvd {
+ bool fw_header_present;
struct radeon_bo *vcpu_bo;
void *cpu_addr;
uint64_t gpu_addr;
+ unsigned max_handles;
atomic_t handles[RADEON_MAX_UVD_HANDLES];
struct drm_file *filp[RADEON_MAX_UVD_HANDLES];
unsigned img_size[RADEON_MAX_UVD_HANDLES];
@@ -1852,7 +1859,7 @@
int (*resume)(struct radeon_device *rdev);
int (*suspend)(struct radeon_device *rdev);
void (*vga_set_state)(struct radeon_device *rdev, bool state);
- int (*asic_reset)(struct radeon_device *rdev);
+ int (*asic_reset)(struct radeon_device *rdev, bool hard);
/* Flush the HDP cache via MMIO */
void (*mmio_hdp_flush)(struct radeon_device *rdev);
/* check if 3D engine is idle */
@@ -1998,7 +2005,7 @@
} dpm;
/* pageflipping */
struct {
- void (*page_flip)(struct radeon_device *rdev, int crtc, u64 crtc_base);
+ void (*page_flip)(struct radeon_device *rdev, int crtc, u64 crtc_base, bool async);
bool (*page_flip_pending)(struct radeon_device *rdev, int crtc);
} pflip;
};
@@ -2394,7 +2401,6 @@
struct radeon_wb wb;
struct radeon_dummy_page dummy_page;
bool shutdown;
- bool suspend;
bool need_dma32;
bool accel_working;
bool fastfb_working; /* IGP feature*/
@@ -2423,6 +2429,7 @@
int num_crtc; /* number of crtcs */
struct mutex dc_hw_i2c_mutex; /* display controller hw i2c mutex */
bool has_uvd;
+ bool has_vce;
struct r600_audio audio; /* audio stuff */
struct notifier_block acpi_nb;
/* only one userspace can use Hyperz features or CMASK at a time */
@@ -2717,7 +2724,7 @@
#define radeon_suspend(rdev) (rdev)->asic->suspend((rdev))
#define radeon_cs_parse(rdev, r, p) (rdev)->asic->ring[(r)]->cs_parse((p))
#define radeon_vga_set_state(rdev, state) (rdev)->asic->vga_set_state((rdev), (state))
-#define radeon_asic_reset(rdev) (rdev)->asic->asic_reset((rdev))
+#define radeon_asic_reset(rdev) (rdev)->asic->asic_reset((rdev), false)
#define radeon_gart_tlb_flush(rdev) (rdev)->asic->gart.tlb_flush((rdev))
#define radeon_gart_get_page_entry(a, f) (rdev)->asic->gart.get_page_entry((a), (f))
#define radeon_gart_set_page(rdev, i, e) (rdev)->asic->gart.set_page((rdev), (i), (e))
@@ -2775,7 +2782,7 @@
#define radeon_pm_finish(rdev) (rdev)->asic->pm.finish((rdev))
#define radeon_pm_init_profile(rdev) (rdev)->asic->pm.init_profile((rdev))
#define radeon_pm_get_dynpm_state(rdev) (rdev)->asic->pm.get_dynpm_state((rdev))
-#define radeon_page_flip(rdev, crtc, base) (rdev)->asic->pflip.page_flip((rdev), (crtc), (base))
+#define radeon_page_flip(rdev, crtc, base, async) (rdev)->asic->pflip.page_flip((rdev), (crtc), (base), (async))
#define radeon_page_flip_pending(rdev, crtc) (rdev)->asic->pflip.page_flip_pending((rdev), (crtc))
#define radeon_wait_for_vblank(rdev, crtc) (rdev)->asic->display.wait_for_vblank((rdev), (crtc))
#define radeon_mc_wait_for_idle(rdev) (rdev)->asic->mc_wait_for_idle((rdev))
@@ -2832,7 +2839,8 @@
extern void radeon_vram_location(struct radeon_device *rdev, struct radeon_mc *mc, u64 base);
extern void radeon_gtt_location(struct radeon_device *rdev, struct radeon_mc *mc);
extern int radeon_resume_kms(struct drm_device *dev, bool resume, bool fbcon);
-extern int radeon_suspend_kms(struct drm_device *dev, bool suspend, bool fbcon);
+extern int radeon_suspend_kms(struct drm_device *dev, bool suspend,
+ bool fbcon, bool freeze);
extern void radeon_ttm_set_active_vram_size(struct radeon_device *rdev, u64 size);
extern void radeon_program_register_sequence(struct radeon_device *rdev,
const u32 *registers,
diff --git a/drivers/gpu/drm/radeon/radeon_asic.c b/drivers/gpu/drm/radeon/radeon_asic.c
index 7d5a36d..bc5121d 100644
--- a/drivers/gpu/drm/radeon/radeon_asic.c
+++ b/drivers/gpu/drm/radeon/radeon_asic.c
@@ -2324,6 +2324,7 @@
rdev->num_crtc = 2;
rdev->has_uvd = false;
+ rdev->has_vce = false;
switch (rdev->family) {
case CHIP_R100:
@@ -2454,6 +2455,7 @@
/* set num crtcs */
rdev->num_crtc = 4;
rdev->has_uvd = true;
+ rdev->has_vce = true;
rdev->cg_flags =
RADEON_CG_SUPPORT_VCE_MGCG;
break;
@@ -2470,10 +2472,13 @@
rdev->num_crtc = 2;
else
rdev->num_crtc = 6;
- if (rdev->family == CHIP_HAINAN)
+ if (rdev->family == CHIP_HAINAN) {
rdev->has_uvd = false;
- else
+ rdev->has_vce = false;
+ } else {
rdev->has_uvd = true;
+ rdev->has_vce = true;
+ }
switch (rdev->family) {
case CHIP_TAHITI:
rdev->cg_flags =
@@ -2578,6 +2583,7 @@
rdev->asic = &ci_asic;
rdev->num_crtc = 6;
rdev->has_uvd = true;
+ rdev->has_vce = true;
if (rdev->family == CHIP_BONAIRE) {
rdev->cg_flags =
RADEON_CG_SUPPORT_GFX_MGCG |
@@ -2678,6 +2684,7 @@
RADEON_PG_SUPPORT_SAMU;*/
}
rdev->has_uvd = true;
+ rdev->has_vce = true;
break;
default:
/* FIXME: not supported yet */
@@ -2689,6 +2696,11 @@
rdev->asic->pm.set_memory_clock = NULL;
}
+ if (!radeon_uvd)
+ rdev->has_uvd = false;
+ if (!radeon_vce)
+ rdev->has_vce = false;
+
return 0;
}
diff --git a/drivers/gpu/drm/radeon/radeon_asic.h b/drivers/gpu/drm/radeon/radeon_asic.h
index e0aa332..e3f036c 100644
--- a/drivers/gpu/drm/radeon/radeon_asic.h
+++ b/drivers/gpu/drm/radeon/radeon_asic.h
@@ -64,7 +64,7 @@
int r100_resume(struct radeon_device *rdev);
void r100_vga_set_state(struct radeon_device *rdev, bool state);
bool r100_gpu_is_lockup(struct radeon_device *rdev, struct radeon_ring *cp);
-int r100_asic_reset(struct radeon_device *rdev);
+int r100_asic_reset(struct radeon_device *rdev, bool hard);
u32 r100_get_vblank_counter(struct radeon_device *rdev, int crtc);
void r100_pci_gart_tlb_flush(struct radeon_device *rdev);
uint64_t r100_pci_gart_get_page_entry(uint64_t addr, uint32_t flags);
@@ -138,7 +138,7 @@
extern void r100_pm_init_profile(struct radeon_device *rdev);
extern void r100_pm_get_dynpm_state(struct radeon_device *rdev);
extern void r100_page_flip(struct radeon_device *rdev, int crtc,
- u64 crtc_base);
+ u64 crtc_base, bool async);
extern bool r100_page_flip_pending(struct radeon_device *rdev, int crtc);
extern void r100_wait_for_vblank(struct radeon_device *rdev, int crtc);
extern int r100_mc_wait_for_idle(struct radeon_device *rdev);
@@ -167,7 +167,7 @@
extern void r300_fini(struct radeon_device *rdev);
extern int r300_suspend(struct radeon_device *rdev);
extern int r300_resume(struct radeon_device *rdev);
-extern int r300_asic_reset(struct radeon_device *rdev);
+extern int r300_asic_reset(struct radeon_device *rdev, bool hard);
extern void r300_ring_start(struct radeon_device *rdev, struct radeon_ring *ring);
extern void r300_fence_ring_emit(struct radeon_device *rdev,
struct radeon_fence *fence);
@@ -225,7 +225,7 @@
/*
* rs600.
*/
-extern int rs600_asic_reset(struct radeon_device *rdev);
+extern int rs600_asic_reset(struct radeon_device *rdev, bool hard);
extern int rs600_init(struct radeon_device *rdev);
extern void rs600_fini(struct radeon_device *rdev);
extern int rs600_suspend(struct radeon_device *rdev);
@@ -250,7 +250,7 @@
extern void rs600_pm_prepare(struct radeon_device *rdev);
extern void rs600_pm_finish(struct radeon_device *rdev);
extern void rs600_page_flip(struct radeon_device *rdev, int crtc,
- u64 crtc_base);
+ u64 crtc_base, bool async);
extern bool rs600_page_flip_pending(struct radeon_device *rdev, int crtc);
void rs600_set_safe_registers(struct radeon_device *rdev);
extern void avivo_wait_for_vblank(struct radeon_device *rdev, int crtc);
@@ -334,7 +334,7 @@
void r600_dma_ring_ib_execute(struct radeon_device *rdev, struct radeon_ib *ib);
bool r600_dma_is_lockup(struct radeon_device *rdev, struct radeon_ring *ring);
bool r600_gfx_is_lockup(struct radeon_device *rdev, struct radeon_ring *cp);
-int r600_asic_reset(struct radeon_device *rdev);
+int r600_asic_reset(struct radeon_device *rdev, bool hard);
int r600_set_surface_reg(struct radeon_device *rdev, int reg,
uint32_t tiling_flags, uint32_t pitch,
uint32_t offset, uint32_t obj_size);
@@ -464,7 +464,8 @@
int rv770_suspend(struct radeon_device *rdev);
int rv770_resume(struct radeon_device *rdev);
void rv770_pm_misc(struct radeon_device *rdev);
-void rv770_page_flip(struct radeon_device *rdev, int crtc, u64 crtc_base);
+void rv770_page_flip(struct radeon_device *rdev, int crtc, u64 crtc_base,
+ bool async);
bool rv770_page_flip_pending(struct radeon_device *rdev, int crtc);
void r700_vram_gtt_location(struct radeon_device *rdev, struct radeon_mc *mc);
void r700_cp_stop(struct radeon_device *rdev);
@@ -513,7 +514,7 @@
int evergreen_resume(struct radeon_device *rdev);
bool evergreen_gfx_is_lockup(struct radeon_device *rdev, struct radeon_ring *cp);
bool evergreen_dma_is_lockup(struct radeon_device *rdev, struct radeon_ring *cp);
-int evergreen_asic_reset(struct radeon_device *rdev);
+int evergreen_asic_reset(struct radeon_device *rdev, bool hard);
void evergreen_bandwidth_update(struct radeon_device *rdev);
void evergreen_ring_ib_execute(struct radeon_device *rdev, struct radeon_ib *ib);
void evergreen_hpd_init(struct radeon_device *rdev);
@@ -534,7 +535,7 @@
int sumo_set_uvd_clocks(struct radeon_device *rdev, u32 vclk, u32 dclk);
int evergreen_set_uvd_clocks(struct radeon_device *rdev, u32 vclk, u32 dclk);
extern void evergreen_page_flip(struct radeon_device *rdev, int crtc,
- u64 crtc_base);
+ u64 crtc_base, bool async);
extern bool evergreen_page_flip_pending(struct radeon_device *rdev, int crtc);
extern void dce4_wait_for_vblank(struct radeon_device *rdev, int crtc);
void evergreen_disable_interrupt_state(struct radeon_device *rdev);
@@ -606,7 +607,7 @@
void cayman_fini(struct radeon_device *rdev);
int cayman_suspend(struct radeon_device *rdev);
int cayman_resume(struct radeon_device *rdev);
-int cayman_asic_reset(struct radeon_device *rdev);
+int cayman_asic_reset(struct radeon_device *rdev, bool hard);
void cayman_ring_ib_execute(struct radeon_device *rdev, struct radeon_ib *ib);
int cayman_vm_init(struct radeon_device *rdev);
void cayman_vm_fini(struct radeon_device *rdev);
@@ -712,7 +713,7 @@
int si_resume(struct radeon_device *rdev);
bool si_gfx_is_lockup(struct radeon_device *rdev, struct radeon_ring *cp);
bool si_dma_is_lockup(struct radeon_device *rdev, struct radeon_ring *cp);
-int si_asic_reset(struct radeon_device *rdev);
+int si_asic_reset(struct radeon_device *rdev, bool hard);
void si_ring_ib_execute(struct radeon_device *rdev, struct radeon_ib *ib);
int si_irq_set(struct radeon_device *rdev);
int si_irq_process(struct radeon_device *rdev);
@@ -817,7 +818,7 @@
int cik_suspend(struct radeon_device *rdev);
int cik_resume(struct radeon_device *rdev);
bool cik_gfx_is_lockup(struct radeon_device *rdev, struct radeon_ring *cp);
-int cik_asic_reset(struct radeon_device *rdev);
+int cik_asic_reset(struct radeon_device *rdev, bool hard);
void cik_ring_ib_execute(struct radeon_device *rdev, struct radeon_ib *ib);
int cik_ring_test(struct radeon_device *rdev, struct radeon_ring *ring);
int cik_ib_test(struct radeon_device *rdev, struct radeon_ring *ring);
diff --git a/drivers/gpu/drm/radeon/radeon_cs.c b/drivers/gpu/drm/radeon/radeon_cs.c
index ab39b85..510ea37 100644
--- a/drivers/gpu/drm/radeon/radeon_cs.c
+++ b/drivers/gpu/drm/radeon/radeon_cs.c
@@ -74,7 +74,6 @@
static int radeon_cs_parser_relocs(struct radeon_cs_parser *p)
{
- struct drm_device *ddev = p->rdev->ddev;
struct radeon_cs_chunk *chunk;
struct radeon_cs_buckets buckets;
unsigned i;
@@ -101,7 +100,7 @@
unsigned priority;
r = (struct drm_radeon_cs_reloc *)&chunk->kdata[i*4];
- gobj = drm_gem_object_lookup(ddev, p->filp, r->handle);
+ gobj = drm_gem_object_lookup(p->filp, r->handle);
if (gobj == NULL) {
DRM_ERROR("gem object lookup failed 0x%x\n",
r->handle);
diff --git a/drivers/gpu/drm/radeon/radeon_cursor.c b/drivers/gpu/drm/radeon/radeon_cursor.c
index afaf346..2a10e24 100644
--- a/drivers/gpu/drm/radeon/radeon_cursor.c
+++ b/drivers/gpu/drm/radeon/radeon_cursor.c
@@ -274,7 +274,7 @@
return -EINVAL;
}
- obj = drm_gem_object_lookup(crtc->dev, file_priv, handle);
+ obj = drm_gem_object_lookup(file_priv, handle);
if (!obj) {
DRM_ERROR("Cannot find cursor object %x for crtc %d\n", handle, radeon_crtc->crtc_id);
return -ENOENT;
diff --git a/drivers/gpu/drm/radeon/radeon_device.c b/drivers/gpu/drm/radeon/radeon_device.c
index d0826fb..e721e6b 100644
--- a/drivers/gpu/drm/radeon/radeon_device.c
+++ b/drivers/gpu/drm/radeon/radeon_device.c
@@ -1230,7 +1230,7 @@
printk(KERN_INFO "radeon: switched off\n");
drm_kms_helper_poll_disable(dev);
dev->switch_power_state = DRM_SWITCH_POWER_CHANGING;
- radeon_suspend_kms(dev, true, true);
+ radeon_suspend_kms(dev, true, true, false);
dev->switch_power_state = DRM_SWITCH_POWER_OFF;
}
}
@@ -1555,7 +1555,8 @@
* Returns 0 for success or an error on failure.
* Called at driver suspend.
*/
-int radeon_suspend_kms(struct drm_device *dev, bool suspend, bool fbcon)
+int radeon_suspend_kms(struct drm_device *dev, bool suspend,
+ bool fbcon, bool freeze)
{
struct radeon_device *rdev;
struct drm_crtc *crtc;
@@ -1630,7 +1631,10 @@
radeon_agp_suspend(rdev);
pci_save_state(dev->pdev);
- if (suspend) {
+ if (freeze && rdev->family >= CHIP_R600) {
+ rdev->asic->asic_reset(rdev, true);
+ pci_restore_state(dev->pdev);
+ } else if (suspend) {
/* Shut down the device */
pci_disable_device(dev->pdev);
pci_set_power_state(dev->pdev, PCI_D3hot);
diff --git a/drivers/gpu/drm/radeon/radeon_display.c b/drivers/gpu/drm/radeon/radeon_display.c
index fcc7483..6a41b49 100644
--- a/drivers/gpu/drm/radeon/radeon_display.c
+++ b/drivers/gpu/drm/radeon/radeon_display.c
@@ -377,7 +377,7 @@
/* wakeup userspace */
if (work->event)
- drm_send_vblank_event(rdev->ddev, crtc_id, work->event);
+ drm_crtc_send_vblank_event(&radeon_crtc->base, work->event);
spin_unlock_irqrestore(&rdev->ddev->event_lock, flags);
@@ -490,7 +490,7 @@
vblank->linedur_ns / 1000, stat, vpos, hpos);
/* do the flip (mmio) */
- radeon_page_flip(rdev, radeon_crtc->crtc_id, work->base);
+ radeon_page_flip(rdev, radeon_crtc->crtc_id, work->base, work->async);
radeon_crtc->flip_status = RADEON_FLIP_SUBMITTED;
spin_unlock_irqrestore(&crtc->dev->event_lock, flags);
@@ -525,6 +525,7 @@
work->rdev = rdev;
work->crtc_id = radeon_crtc->crtc_id;
work->event = event;
+ work->async = (page_flip_flags & DRM_MODE_PAGE_FLIP_ASYNC) != 0;
/* schedule unpin of the old buffer */
old_radeon_fb = to_radeon_framebuffer(crtc->primary->fb);
@@ -1367,7 +1368,7 @@
struct radeon_framebuffer *radeon_fb;
int ret;
- obj = drm_gem_object_lookup(dev, file_priv, mode_cmd->handles[0]);
+ obj = drm_gem_object_lookup(file_priv, mode_cmd->handles[0]);
if (obj == NULL) {
dev_err(&dev->pdev->dev, "No GEM object associated to handle 0x%08X, "
"can't create framebuffer\n", mode_cmd->handles[0]);
@@ -1630,6 +1631,9 @@
rdev->ddev->mode_config.funcs = &radeon_mode_funcs;
+ if (radeon_use_pflipirq == 2 && rdev->family >= CHIP_R600)
+ rdev->ddev->mode_config.async_page_flip = true;
+
if (ASIC_IS_DCE5(rdev)) {
rdev->ddev->mode_config.max_width = 16384;
rdev->ddev->mode_config.max_height = 16384;
diff --git a/drivers/gpu/drm/radeon/radeon_drv.c b/drivers/gpu/drm/radeon/radeon_drv.c
index ccd4ad4..b55aa74 100644
--- a/drivers/gpu/drm/radeon/radeon_drv.c
+++ b/drivers/gpu/drm/radeon/radeon_drv.c
@@ -93,9 +93,11 @@
* 2.41.0 - evergreen/cayman: Add SET_BASE/DRAW_INDIRECT command parsing support
* 2.42.0 - Add VCE/VUI (Video Usability Information) support
* 2.43.0 - RADEON_INFO_GPU_RESET_COUNTER
+ * 2.44.0 - SET_APPEND_CNT packet3 support
+ * 2.45.0 - Allow setting shader registers using DMA/COPY packet3 on SI
*/
#define KMS_DRIVER_MAJOR 2
-#define KMS_DRIVER_MINOR 43
+#define KMS_DRIVER_MINOR 45
#define KMS_DRIVER_PATCHLEVEL 0
int radeon_driver_load_kms(struct drm_device *dev, unsigned long flags);
int radeon_driver_unload_kms(struct drm_device *dev);
@@ -105,7 +107,8 @@
struct drm_file *file_priv);
void radeon_driver_preclose_kms(struct drm_device *dev,
struct drm_file *file_priv);
-int radeon_suspend_kms(struct drm_device *dev, bool suspend, bool fbcon);
+int radeon_suspend_kms(struct drm_device *dev, bool suspend,
+ bool fbcon, bool freeze);
int radeon_resume_kms(struct drm_device *dev, bool resume, bool fbcon);
u32 radeon_get_vblank_counter_kms(struct drm_device *dev, unsigned int pipe);
int radeon_enable_vblank_kms(struct drm_device *dev, unsigned int pipe);
@@ -196,6 +199,8 @@
int radeon_backlight = -1;
int radeon_auxch = -1;
int radeon_mst = 0;
+int radeon_uvd = 1;
+int radeon_vce = 1;
MODULE_PARM_DESC(no_wb, "Disable AGP writeback for scratch registers");
module_param_named(no_wb, radeon_no_wb, int, 0444);
@@ -287,6 +292,12 @@
MODULE_PARM_DESC(mst, "DisplayPort MST experimental support (1 = enable, 0 = disable)");
module_param_named(mst, radeon_mst, int, 0444);
+MODULE_PARM_DESC(uvd, "uvd enable/disable uvd support (1 = enable, 0 = disable)");
+module_param_named(uvd, radeon_uvd, int, 0444);
+
+MODULE_PARM_DESC(vce, "vce enable/disable vce support (1 = enable, 0 = disable)");
+module_param_named(vce, radeon_vce, int, 0444);
+
static struct pci_device_id pciidlist[] = {
radeon_PCI_IDS
};
@@ -358,7 +369,7 @@
{
struct pci_dev *pdev = to_pci_dev(dev);
struct drm_device *drm_dev = pci_get_drvdata(pdev);
- return radeon_suspend_kms(drm_dev, true, true);
+ return radeon_suspend_kms(drm_dev, true, true, false);
}
static int radeon_pmops_resume(struct device *dev)
@@ -372,7 +383,7 @@
{
struct pci_dev *pdev = to_pci_dev(dev);
struct drm_device *drm_dev = pci_get_drvdata(pdev);
- return radeon_suspend_kms(drm_dev, false, true);
+ return radeon_suspend_kms(drm_dev, false, true, true);
}
static int radeon_pmops_thaw(struct device *dev)
@@ -397,7 +408,7 @@
drm_kms_helper_poll_disable(drm_dev);
vga_switcheroo_set_dynamic_switch(pdev, VGA_SWITCHEROO_OFF);
- ret = radeon_suspend_kms(drm_dev, false, false);
+ ret = radeon_suspend_kms(drm_dev, false, false, false);
pci_save_state(pdev);
pci_disable_device(pdev);
pci_ignore_hotplug(pdev);
@@ -525,7 +536,7 @@
.irq_uninstall = radeon_driver_irq_uninstall_kms,
.irq_handler = radeon_driver_irq_handler_kms,
.ioctls = radeon_ioctls_kms,
- .gem_free_object = radeon_gem_object_free,
+ .gem_free_object_unlocked = radeon_gem_object_free,
.gem_open_object = radeon_gem_object_open,
.gem_close_object = radeon_gem_object_close,
.dumb_create = radeon_mode_dumb_create,
@@ -566,12 +577,10 @@
static int __init radeon_init(void)
{
-#ifdef CONFIG_VGA_CONSOLE
if (vgacon_text_force() && radeon_modeset == -1) {
DRM_INFO("VGACON disable radeon kernel modesetting.\n");
radeon_modeset = 0;
}
-#endif
/* set to modesetting by default if not nomodeset */
if (radeon_modeset == -1)
radeon_modeset = 1;
diff --git a/drivers/gpu/drm/radeon/radeon_gem.c b/drivers/gpu/drm/radeon/radeon_gem.c
index e26c963..deb9511 100644
--- a/drivers/gpu/drm/radeon/radeon_gem.c
+++ b/drivers/gpu/drm/radeon/radeon_gem.c
@@ -382,7 +382,7 @@
down_read(&rdev->exclusive_lock);
/* just do a BO wait for now */
- gobj = drm_gem_object_lookup(dev, filp, args->handle);
+ gobj = drm_gem_object_lookup(filp, args->handle);
if (gobj == NULL) {
up_read(&rdev->exclusive_lock);
return -ENOENT;
@@ -404,7 +404,7 @@
struct drm_gem_object *gobj;
struct radeon_bo *robj;
- gobj = drm_gem_object_lookup(dev, filp, handle);
+ gobj = drm_gem_object_lookup(filp, handle);
if (gobj == NULL) {
return -ENOENT;
}
@@ -435,7 +435,7 @@
int r;
uint32_t cur_placement = 0;
- gobj = drm_gem_object_lookup(dev, filp, args->handle);
+ gobj = drm_gem_object_lookup(filp, args->handle);
if (gobj == NULL) {
return -ENOENT;
}
@@ -464,7 +464,7 @@
uint32_t cur_placement = 0;
long ret;
- gobj = drm_gem_object_lookup(dev, filp, args->handle);
+ gobj = drm_gem_object_lookup(filp, args->handle);
if (gobj == NULL) {
return -ENOENT;
}
@@ -495,7 +495,7 @@
int r = 0;
DRM_DEBUG("%d \n", args->handle);
- gobj = drm_gem_object_lookup(dev, filp, args->handle);
+ gobj = drm_gem_object_lookup(filp, args->handle);
if (gobj == NULL)
return -ENOENT;
robj = gem_to_radeon_bo(gobj);
@@ -513,7 +513,7 @@
int r = 0;
DRM_DEBUG("\n");
- gobj = drm_gem_object_lookup(dev, filp, args->handle);
+ gobj = drm_gem_object_lookup(filp, args->handle);
if (gobj == NULL)
return -ENOENT;
rbo = gem_to_radeon_bo(gobj);
@@ -648,7 +648,7 @@
return -EINVAL;
}
- gobj = drm_gem_object_lookup(dev, filp, args->handle);
+ gobj = drm_gem_object_lookup(filp, args->handle);
if (gobj == NULL) {
args->operation = RADEON_VA_RESULT_ERROR;
return -ENOENT;
@@ -703,7 +703,7 @@
struct radeon_bo *robj;
int r;
- gobj = drm_gem_object_lookup(dev, filp, args->handle);
+ gobj = drm_gem_object_lookup(filp, args->handle);
if (gobj == NULL) {
return -ENOENT;
}
diff --git a/drivers/gpu/drm/radeon/radeon_irq_kms.c b/drivers/gpu/drm/radeon/radeon_irq_kms.c
index 1e9304d..c084cad 100644
--- a/drivers/gpu/drm/radeon/radeon_irq_kms.c
+++ b/drivers/gpu/drm/radeon/radeon_irq_kms.c
@@ -291,7 +291,6 @@
if (r) {
return r;
}
- rdev->ddev->vblank_disable_allowed = true;
/* enable msi */
rdev->msi_enabled = 0;
diff --git a/drivers/gpu/drm/radeon/radeon_mn.c b/drivers/gpu/drm/radeon/radeon_mn.c
index eef006c..896f2cf 100644
--- a/drivers/gpu/drm/radeon/radeon_mn.c
+++ b/drivers/gpu/drm/radeon/radeon_mn.c
@@ -186,7 +186,9 @@
struct radeon_mn *rmn;
int r;
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem))
+ return ERR_PTR(-EINTR);
+
mutex_lock(&rdev->mn_lock);
hash_for_each_possible(rdev->mn_hash, rmn, node, (unsigned long)mm)
diff --git a/drivers/gpu/drm/radeon/radeon_object.c b/drivers/gpu/drm/radeon/radeon_object.c
index 2d901bf..be30861 100644
--- a/drivers/gpu/drm/radeon/radeon_object.c
+++ b/drivers/gpu/drm/radeon/radeon_object.c
@@ -832,13 +832,13 @@
{
int r;
- r = ttm_bo_reserve(&bo->tbo, true, no_wait, false, NULL);
+ r = ttm_bo_reserve(&bo->tbo, true, no_wait, NULL);
if (unlikely(r != 0))
return r;
if (mem_type)
*mem_type = bo->tbo.mem.mem_type;
- r = ttm_bo_wait(&bo->tbo, true, true, no_wait);
+ r = ttm_bo_wait(&bo->tbo, true, no_wait);
ttm_bo_unreserve(&bo->tbo);
return r;
}
diff --git a/drivers/gpu/drm/radeon/radeon_object.h b/drivers/gpu/drm/radeon/radeon_object.h
index d8d295e..a10bb3d 100644
--- a/drivers/gpu/drm/radeon/radeon_object.h
+++ b/drivers/gpu/drm/radeon/radeon_object.h
@@ -65,7 +65,7 @@
{
int r;
- r = ttm_bo_reserve(&bo->tbo, !no_intr, false, false, NULL);
+ r = ttm_bo_reserve(&bo->tbo, !no_intr, false, NULL);
if (unlikely(r != 0)) {
if (r != -ERESTARTSYS)
dev_err(bo->rdev->dev, "%p reserve failed\n", bo);
diff --git a/drivers/gpu/drm/radeon/radeon_ttm.c b/drivers/gpu/drm/radeon/radeon_ttm.c
index 90f7394..590b037 100644
--- a/drivers/gpu/drm/radeon/radeon_ttm.c
+++ b/drivers/gpu/drm/radeon/radeon_ttm.c
@@ -865,6 +865,8 @@
.fault_reserve_notify = &radeon_bo_fault_reserve_notify,
.io_mem_reserve = &radeon_ttm_io_mem_reserve,
.io_mem_free = &radeon_ttm_io_mem_free,
+ .lru_tail = &ttm_bo_default_lru_tail,
+ .swap_lru_tail = &ttm_bo_default_swap_lru_tail,
};
int radeon_ttm_init(struct radeon_device *rdev)
diff --git a/drivers/gpu/drm/radeon/radeon_uvd.c b/drivers/gpu/drm/radeon/radeon_uvd.c
index 6fe9e4e..73dfe01 100644
--- a/drivers/gpu/drm/radeon/radeon_uvd.c
+++ b/drivers/gpu/drm/radeon/radeon_uvd.c
@@ -34,6 +34,7 @@
#include <drm/drm.h>
#include "radeon.h"
+#include "radeon_ucode.h"
#include "r600d.h"
/* 1 second timeout */
@@ -47,7 +48,8 @@
#define FIRMWARE_CYPRESS "radeon/CYPRESS_uvd.bin"
#define FIRMWARE_SUMO "radeon/SUMO_uvd.bin"
#define FIRMWARE_TAHITI "radeon/TAHITI_uvd.bin"
-#define FIRMWARE_BONAIRE "radeon/BONAIRE_uvd.bin"
+#define FIRMWARE_BONAIRE_LEGACY "radeon/BONAIRE_uvd.bin"
+#define FIRMWARE_BONAIRE "radeon/bonaire_uvd.bin"
MODULE_FIRMWARE(FIRMWARE_R600);
MODULE_FIRMWARE(FIRMWARE_RS780);
@@ -56,6 +58,7 @@
MODULE_FIRMWARE(FIRMWARE_CYPRESS);
MODULE_FIRMWARE(FIRMWARE_SUMO);
MODULE_FIRMWARE(FIRMWARE_TAHITI);
+MODULE_FIRMWARE(FIRMWARE_BONAIRE_LEGACY);
MODULE_FIRMWARE(FIRMWARE_BONAIRE);
static void radeon_uvd_idle_work_handler(struct work_struct *work);
@@ -63,7 +66,7 @@
int radeon_uvd_init(struct radeon_device *rdev)
{
unsigned long bo_size;
- const char *fw_name;
+ const char *fw_name = NULL, *legacy_fw_name = NULL;
int i, r;
INIT_DELAYED_WORK(&rdev->uvd.idle_work, radeon_uvd_idle_work_handler);
@@ -74,22 +77,22 @@
case CHIP_RV670:
case CHIP_RV620:
case CHIP_RV635:
- fw_name = FIRMWARE_R600;
+ legacy_fw_name = FIRMWARE_R600;
break;
case CHIP_RS780:
case CHIP_RS880:
- fw_name = FIRMWARE_RS780;
+ legacy_fw_name = FIRMWARE_RS780;
break;
case CHIP_RV770:
- fw_name = FIRMWARE_RV770;
+ legacy_fw_name = FIRMWARE_RV770;
break;
case CHIP_RV710:
case CHIP_RV730:
case CHIP_RV740:
- fw_name = FIRMWARE_RV710;
+ legacy_fw_name = FIRMWARE_RV710;
break;
case CHIP_CYPRESS:
@@ -97,7 +100,7 @@
case CHIP_JUNIPER:
case CHIP_REDWOOD:
case CHIP_CEDAR:
- fw_name = FIRMWARE_CYPRESS;
+ legacy_fw_name = FIRMWARE_CYPRESS;
break;
case CHIP_SUMO:
@@ -107,7 +110,7 @@
case CHIP_BARTS:
case CHIP_TURKS:
case CHIP_CAICOS:
- fw_name = FIRMWARE_SUMO;
+ legacy_fw_name = FIRMWARE_SUMO;
break;
case CHIP_TAHITI:
@@ -115,7 +118,7 @@
case CHIP_PITCAIRN:
case CHIP_ARUBA:
case CHIP_OLAND:
- fw_name = FIRMWARE_TAHITI;
+ legacy_fw_name = FIRMWARE_TAHITI;
break;
case CHIP_BONAIRE:
@@ -123,6 +126,7 @@
case CHIP_KAVERI:
case CHIP_HAWAII:
case CHIP_MULLINS:
+ legacy_fw_name = FIRMWARE_BONAIRE_LEGACY;
fw_name = FIRMWARE_BONAIRE;
break;
@@ -130,16 +134,56 @@
return -EINVAL;
}
- r = request_firmware(&rdev->uvd_fw, fw_name, rdev->dev);
- if (r) {
- dev_err(rdev->dev, "radeon_uvd: Can't load firmware \"%s\"\n",
- fw_name);
- return r;
+ rdev->uvd.fw_header_present = false;
+ rdev->uvd.max_handles = RADEON_DEFAULT_UVD_HANDLES;
+ if (fw_name) {
+ /* Let's try to load the newer firmware first */
+ r = request_firmware(&rdev->uvd_fw, fw_name, rdev->dev);
+ if (r) {
+ dev_err(rdev->dev, "radeon_uvd: Can't load firmware \"%s\"\n",
+ fw_name);
+ } else {
+ struct common_firmware_header *hdr = (void *)rdev->uvd_fw->data;
+ unsigned version_major, version_minor, family_id;
+
+ r = radeon_ucode_validate(rdev->uvd_fw);
+ if (r)
+ return r;
+
+ rdev->uvd.fw_header_present = true;
+
+ family_id = le32_to_cpu(hdr->ucode_version) & 0xff;
+ version_major = (le32_to_cpu(hdr->ucode_version) >> 24) & 0xff;
+ version_minor = (le32_to_cpu(hdr->ucode_version) >> 8) & 0xff;
+ DRM_INFO("Found UVD firmware Version: %hu.%hu Family ID: %hu\n",
+ version_major, version_minor, family_id);
+
+ /*
+ * Limit the number of UVD handles depending on
+ * microcode major and minor versions.
+ */
+ if ((version_major >= 0x01) && (version_minor >= 0x37))
+ rdev->uvd.max_handles = RADEON_MAX_UVD_HANDLES;
+ }
+ }
+
+ /*
+ * In case there is only legacy firmware, or we encounter an error
+ * while loading the new firmware, we fall back to loading the legacy
+ * firmware now.
+ */
+ if (!fw_name || r) {
+ r = request_firmware(&rdev->uvd_fw, legacy_fw_name, rdev->dev);
+ if (r) {
+ dev_err(rdev->dev, "radeon_uvd: Can't load firmware \"%s\"\n",
+ legacy_fw_name);
+ return r;
+ }
}
bo_size = RADEON_GPU_PAGE_ALIGN(rdev->uvd_fw->size + 8) +
RADEON_UVD_STACK_SIZE + RADEON_UVD_HEAP_SIZE +
- RADEON_GPU_PAGE_SIZE;
+ RADEON_UVD_SESSION_SIZE * rdev->uvd.max_handles;
r = radeon_bo_create(rdev, bo_size, PAGE_SIZE, true,
RADEON_GEM_DOMAIN_VRAM, 0, NULL,
NULL, &rdev->uvd.vcpu_bo);
@@ -172,7 +216,7 @@
radeon_bo_unreserve(rdev->uvd.vcpu_bo);
- for (i = 0; i < RADEON_MAX_UVD_HANDLES; ++i) {
+ for (i = 0; i < rdev->uvd.max_handles; ++i) {
atomic_set(&rdev->uvd.handles[i], 0);
rdev->uvd.filp[i] = NULL;
rdev->uvd.img_size[i] = 0;
@@ -209,7 +253,7 @@
if (rdev->uvd.vcpu_bo == NULL)
return 0;
- for (i = 0; i < RADEON_MAX_UVD_HANDLES; ++i) {
+ for (i = 0; i < rdev->uvd.max_handles; ++i) {
uint32_t handle = atomic_read(&rdev->uvd.handles[i]);
if (handle != 0) {
struct radeon_fence *fence;
@@ -284,7 +328,7 @@
void radeon_uvd_free_handles(struct radeon_device *rdev, struct drm_file *filp)
{
int i, r;
- for (i = 0; i < RADEON_MAX_UVD_HANDLES; ++i) {
+ for (i = 0; i < rdev->uvd.max_handles; ++i) {
uint32_t handle = atomic_read(&rdev->uvd.handles[i]);
if (handle != 0 && rdev->uvd.filp[i] == filp) {
struct radeon_fence *fence;
@@ -469,7 +513,7 @@
return r;
/* try to alloc a new handle */
- for (i = 0; i < RADEON_MAX_UVD_HANDLES; ++i) {
+ for (i = 0; i < p->rdev->uvd.max_handles; ++i) {
if (atomic_read(&p->rdev->uvd.handles[i]) == handle) {
DRM_ERROR("Handle 0x%x already in use!\n", handle);
return -EINVAL;
@@ -495,7 +539,7 @@
return r;
/* validate the handle */
- for (i = 0; i < RADEON_MAX_UVD_HANDLES; ++i) {
+ for (i = 0; i < p->rdev->uvd.max_handles; ++i) {
if (atomic_read(&p->rdev->uvd.handles[i]) == handle) {
if (p->rdev->uvd.filp[i] != p->filp) {
DRM_ERROR("UVD handle collision detected!\n");
@@ -510,7 +554,7 @@
case 2:
/* it's a destroy msg, free the handle */
- for (i = 0; i < RADEON_MAX_UVD_HANDLES; ++i)
+ for (i = 0; i < p->rdev->uvd.max_handles; ++i)
atomic_cmpxchg(&p->rdev->uvd.handles[i], handle, 0);
radeon_bo_kunmap(bo);
return 0;
@@ -809,7 +853,7 @@
*sd = 0;
*hd = 0;
- for (i = 0; i < RADEON_MAX_UVD_HANDLES; ++i) {
+ for (i = 0; i < rdev->uvd.max_handles; ++i) {
if (!atomic_read(&rdev->uvd.handles[i]))
continue;
diff --git a/drivers/gpu/drm/radeon/rs600.c b/drivers/gpu/drm/radeon/rs600.c
index 6244f4e..f16af11 100644
--- a/drivers/gpu/drm/radeon/rs600.c
+++ b/drivers/gpu/drm/radeon/rs600.c
@@ -110,7 +110,7 @@
}
}
-void rs600_page_flip(struct radeon_device *rdev, int crtc_id, u64 crtc_base)
+void rs600_page_flip(struct radeon_device *rdev, int crtc_id, u64 crtc_base, bool async)
{
struct radeon_crtc *radeon_crtc = rdev->mode_info.crtcs[crtc_id];
u32 tmp = RREG32(AVIVO_D1GRPH_UPDATE + radeon_crtc->crtc_offset);
@@ -121,6 +121,8 @@
WREG32(AVIVO_D1GRPH_UPDATE + radeon_crtc->crtc_offset, tmp);
/* update the scanout addresses */
+ WREG32(AVIVO_D1GRPH_FLIP_CONTROL + radeon_crtc->crtc_offset,
+ async ? AVIVO_D1GRPH_SURFACE_UPDATE_H_RETRACE_EN : 0);
WREG32(AVIVO_D1GRPH_SECONDARY_SURFACE_ADDRESS + radeon_crtc->crtc_offset,
(u32)crtc_base);
WREG32(AVIVO_D1GRPH_PRIMARY_SURFACE_ADDRESS + radeon_crtc->crtc_offset,
@@ -413,7 +415,8 @@
default:
break;
}
- enable |= 1 << radeon_connector->hpd.hpd;
+ if (radeon_connector->hpd.hpd != RADEON_HPD_NONE)
+ enable |= 1 << radeon_connector->hpd.hpd;
radeon_hpd_set_polarity(rdev, radeon_connector->hpd.hpd);
}
radeon_irq_kms_enable_hpd(rdev, enable);
@@ -439,12 +442,13 @@
default:
break;
}
- disable |= 1 << radeon_connector->hpd.hpd;
+ if (radeon_connector->hpd.hpd != RADEON_HPD_NONE)
+ disable |= 1 << radeon_connector->hpd.hpd;
}
radeon_irq_kms_disable_hpd(rdev, disable);
}
-int rs600_asic_reset(struct radeon_device *rdev)
+int rs600_asic_reset(struct radeon_device *rdev, bool hard)
{
struct rv515_mc_save save;
u32 status, tmp;
diff --git a/drivers/gpu/drm/radeon/rv770.c b/drivers/gpu/drm/radeon/rv770.c
index 01ee96ac..1c120a4 100644
--- a/drivers/gpu/drm/radeon/rv770.c
+++ b/drivers/gpu/drm/radeon/rv770.c
@@ -801,7 +801,7 @@
return reference_clock;
}
-void rv770_page_flip(struct radeon_device *rdev, int crtc_id, u64 crtc_base)
+void rv770_page_flip(struct radeon_device *rdev, int crtc_id, u64 crtc_base, bool async)
{
struct radeon_crtc *radeon_crtc = rdev->mode_info.crtcs[crtc_id];
u32 tmp = RREG32(AVIVO_D1GRPH_UPDATE + radeon_crtc->crtc_offset);
@@ -812,6 +812,8 @@
WREG32(AVIVO_D1GRPH_UPDATE + radeon_crtc->crtc_offset, tmp);
/* update the scanout addresses */
+ WREG32(AVIVO_D1GRPH_FLIP_CONTROL + radeon_crtc->crtc_offset,
+ async ? AVIVO_D1GRPH_SURFACE_UPDATE_H_RETRACE_EN : 0);
if (radeon_crtc->crtc_id) {
WREG32(D2GRPH_SECONDARY_SURFACE_ADDRESS_HIGH, upper_32_bits(crtc_base));
WREG32(D2GRPH_PRIMARY_SURFACE_ADDRESS_HIGH, upper_32_bits(crtc_base));
@@ -1681,6 +1683,73 @@
return 0;
}
+static void rv770_uvd_init(struct radeon_device *rdev)
+{
+ int r;
+
+ if (!rdev->has_uvd)
+ return;
+
+ r = radeon_uvd_init(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed UVD (%d) init.\n", r);
+ /*
+ * At this point rdev->uvd.vcpu_bo is NULL which trickles down
+ * to early fails uvd_v2_2_resume() and thus nothing happens
+ * there. So it is pointless to try to go through that code
+ * hence why we disable uvd here.
+ */
+ rdev->has_uvd = 0;
+ return;
+ }
+ rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_obj = NULL;
+ r600_ring_init(rdev, &rdev->ring[R600_RING_TYPE_UVD_INDEX], 4096);
+}
+
+static void rv770_uvd_start(struct radeon_device *rdev)
+{
+ int r;
+
+ if (!rdev->has_uvd)
+ return;
+
+ r = uvd_v2_2_resume(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed UVD resume (%d).\n", r);
+ goto error;
+ }
+ r = radeon_fence_driver_start_ring(rdev, R600_RING_TYPE_UVD_INDEX);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing UVD fences (%d).\n", r);
+ goto error;
+ }
+ return;
+
+error:
+ rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_size = 0;
+}
+
+static void rv770_uvd_resume(struct radeon_device *rdev)
+{
+ struct radeon_ring *ring;
+ int r;
+
+ if (!rdev->has_uvd || !rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_size)
+ return;
+
+ ring = &rdev->ring[R600_RING_TYPE_UVD_INDEX];
+ r = radeon_ring_init(rdev, ring, ring->ring_size, 0, RADEON_CP_PACKET2);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing UVD ring (%d).\n", r);
+ return;
+ }
+ r = uvd_v1_0_init(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing UVD (%d).\n", r);
+ return;
+ }
+}
+
static int rv770_startup(struct radeon_device *rdev)
{
struct radeon_ring *ring;
@@ -1723,16 +1792,7 @@
return r;
}
- r = uvd_v2_2_resume(rdev);
- if (!r) {
- r = radeon_fence_driver_start_ring(rdev,
- R600_RING_TYPE_UVD_INDEX);
- if (r)
- dev_err(rdev->dev, "UVD fences init error (%d).\n", r);
- }
-
- if (r)
- rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_size = 0;
+ rv770_uvd_start(rdev);
/* Enable IRQ */
if (!rdev->irq.installed) {
@@ -1772,16 +1832,7 @@
if (r)
return r;
- ring = &rdev->ring[R600_RING_TYPE_UVD_INDEX];
- if (ring->ring_size) {
- r = radeon_ring_init(rdev, ring, ring->ring_size, 0,
- RADEON_CP_PACKET2);
- if (!r)
- r = uvd_v1_0_init(rdev);
-
- if (r)
- DRM_ERROR("radeon: failed initializing UVD (%d).\n", r);
- }
+ rv770_uvd_resume(rdev);
r = radeon_ib_pool_init(rdev);
if (r) {
@@ -1831,8 +1882,10 @@
{
radeon_pm_suspend(rdev);
radeon_audio_fini(rdev);
- uvd_v1_0_fini(rdev);
- radeon_uvd_suspend(rdev);
+ if (rdev->has_uvd) {
+ uvd_v1_0_fini(rdev);
+ radeon_uvd_suspend(rdev);
+ }
r700_cp_stop(rdev);
r600_dma_stop(rdev);
r600_irq_suspend(rdev);
@@ -1917,12 +1970,7 @@
rdev->ring[R600_RING_TYPE_DMA_INDEX].ring_obj = NULL;
r600_ring_init(rdev, &rdev->ring[R600_RING_TYPE_DMA_INDEX], 64 * 1024);
- r = radeon_uvd_init(rdev);
- if (!r) {
- rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_obj = NULL;
- r600_ring_init(rdev, &rdev->ring[R600_RING_TYPE_UVD_INDEX],
- 4096);
- }
+ rv770_uvd_init(rdev);
rdev->ih.ring_obj = NULL;
r600_ih_ring_init(rdev, 64 * 1024);
diff --git a/drivers/gpu/drm/radeon/si.c b/drivers/gpu/drm/radeon/si.c
index ae21550..b30e719 100644
--- a/drivers/gpu/drm/radeon/si.c
+++ b/drivers/gpu/drm/radeon/si.c
@@ -4034,10 +4034,15 @@
}
}
-int si_asic_reset(struct radeon_device *rdev)
+int si_asic_reset(struct radeon_device *rdev, bool hard)
{
u32 reset_mask;
+ if (hard) {
+ si_gpu_pci_config_reset(rdev);
+ return 0;
+ }
+
reset_mask = si_gpu_check_soft_reset(rdev);
if (reset_mask)
@@ -4359,6 +4364,10 @@
if (reg >= 0x28000)
return true;
+ /* shader regs are also fine */
+ if (reg >= 0xB000 && reg < 0xC000)
+ return true;
+
/* check config regs */
switch (reg) {
case GRBM_GFX_INDEX:
@@ -6821,6 +6830,159 @@
/*
* startup/shutdown callbacks
*/
+static void si_uvd_init(struct radeon_device *rdev)
+{
+ int r;
+
+ if (!rdev->has_uvd)
+ return;
+
+ r = radeon_uvd_init(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed UVD (%d) init.\n", r);
+ /*
+ * At this point rdev->uvd.vcpu_bo is NULL which trickles down
+ * to early fails uvd_v2_2_resume() and thus nothing happens
+ * there. So it is pointless to try to go through that code
+ * hence why we disable uvd here.
+ */
+ rdev->has_uvd = 0;
+ return;
+ }
+ rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_obj = NULL;
+ r600_ring_init(rdev, &rdev->ring[R600_RING_TYPE_UVD_INDEX], 4096);
+}
+
+static void si_uvd_start(struct radeon_device *rdev)
+{
+ int r;
+
+ if (!rdev->has_uvd)
+ return;
+
+ r = uvd_v2_2_resume(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed UVD resume (%d).\n", r);
+ goto error;
+ }
+ r = radeon_fence_driver_start_ring(rdev, R600_RING_TYPE_UVD_INDEX);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing UVD fences (%d).\n", r);
+ goto error;
+ }
+ return;
+
+error:
+ rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_size = 0;
+}
+
+static void si_uvd_resume(struct radeon_device *rdev)
+{
+ struct radeon_ring *ring;
+ int r;
+
+ if (!rdev->has_uvd || !rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_size)
+ return;
+
+ ring = &rdev->ring[R600_RING_TYPE_UVD_INDEX];
+ r = radeon_ring_init(rdev, ring, ring->ring_size, 0, RADEON_CP_PACKET2);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing UVD ring (%d).\n", r);
+ return;
+ }
+ r = uvd_v1_0_init(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing UVD (%d).\n", r);
+ return;
+ }
+}
+
+static void si_vce_init(struct radeon_device *rdev)
+{
+ int r;
+
+ if (!rdev->has_vce)
+ return;
+
+ r = radeon_vce_init(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed VCE (%d) init.\n", r);
+ /*
+ * At this point rdev->vce.vcpu_bo is NULL which trickles down
+ * to early fails si_vce_start() and thus nothing happens
+ * there. So it is pointless to try to go through that code
+ * hence why we disable vce here.
+ */
+ rdev->has_vce = 0;
+ return;
+ }
+ rdev->ring[TN_RING_TYPE_VCE1_INDEX].ring_obj = NULL;
+ r600_ring_init(rdev, &rdev->ring[TN_RING_TYPE_VCE1_INDEX], 4096);
+ rdev->ring[TN_RING_TYPE_VCE2_INDEX].ring_obj = NULL;
+ r600_ring_init(rdev, &rdev->ring[TN_RING_TYPE_VCE2_INDEX], 4096);
+}
+
+static void si_vce_start(struct radeon_device *rdev)
+{
+ int r;
+
+ if (!rdev->has_vce)
+ return;
+
+ r = radeon_vce_resume(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed VCE resume (%d).\n", r);
+ goto error;
+ }
+ r = vce_v1_0_resume(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed VCE resume (%d).\n", r);
+ goto error;
+ }
+ r = radeon_fence_driver_start_ring(rdev, TN_RING_TYPE_VCE1_INDEX);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing VCE1 fences (%d).\n", r);
+ goto error;
+ }
+ r = radeon_fence_driver_start_ring(rdev, TN_RING_TYPE_VCE2_INDEX);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing VCE2 fences (%d).\n", r);
+ goto error;
+ }
+ return;
+
+error:
+ rdev->ring[TN_RING_TYPE_VCE1_INDEX].ring_size = 0;
+ rdev->ring[TN_RING_TYPE_VCE2_INDEX].ring_size = 0;
+}
+
+static void si_vce_resume(struct radeon_device *rdev)
+{
+ struct radeon_ring *ring;
+ int r;
+
+ if (!rdev->has_vce || !rdev->ring[TN_RING_TYPE_VCE1_INDEX].ring_size)
+ return;
+
+ ring = &rdev->ring[TN_RING_TYPE_VCE1_INDEX];
+ r = radeon_ring_init(rdev, ring, ring->ring_size, 0, VCE_CMD_NO_OP);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing VCE1 ring (%d).\n", r);
+ return;
+ }
+ ring = &rdev->ring[TN_RING_TYPE_VCE2_INDEX];
+ r = radeon_ring_init(rdev, ring, ring->ring_size, 0, VCE_CMD_NO_OP);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing VCE1 ring (%d).\n", r);
+ return;
+ }
+ r = vce_v1_0_init(rdev);
+ if (r) {
+ dev_err(rdev->dev, "failed initializing VCE (%d).\n", r);
+ return;
+ }
+}
+
static int si_startup(struct radeon_device *rdev)
{
struct radeon_ring *ring;
@@ -6899,33 +7061,8 @@
return r;
}
- if (rdev->has_uvd) {
- r = uvd_v2_2_resume(rdev);
- if (!r) {
- r = radeon_fence_driver_start_ring(rdev,
- R600_RING_TYPE_UVD_INDEX);
- if (r)
- dev_err(rdev->dev, "UVD fences init error (%d).\n", r);
- }
- if (r)
- rdev->ring[R600_RING_TYPE_UVD_INDEX].ring_size = 0;
- }
-
- r = radeon_vce_resume(rdev);
- if (!r) {
- r = vce_v1_0_resume(rdev);
- if (!r)
- r = radeon_fence_driver_start_ring(rdev,
- TN_RING_TYPE_VCE1_INDEX);
- if (!r)
- r = radeon_fence_driver_start_ring(rdev,
- TN_RING_TYPE_VCE2_INDEX);
- }
- if (r) {
- dev_err(rdev->dev, "VCE init error (%d).\n", r);
- rdev->ring[TN_RING_TYPE_VCE1_INDEX].ring_size = 0;
- rdev->ring[TN_RING_TYPE_VCE2_INDEX].ring_size = 0;
- }
+ si_uvd_start(rdev);
+ si_vce_start(rdev);
/* Enable IRQ */
if (!rdev->irq.installed) {
@@ -6983,34 +7120,8 @@
if (r)
return r;
- if (rdev->has_uvd) {
- ring = &rdev->ring[R600_RING_TYPE_UVD_INDEX];
- if (ring->ring_size) {
- r = radeon_ring_init(rdev, ring, ring->ring_size, 0,
- RADEON_CP_PACKET2);
- if (!r)
- r = uvd_v1_0_init(rdev);
- if (r)
- DRM_ERROR("radeon: failed initializing UVD (%d).\n", r);
- }
- }
-
- r = -ENOENT;
-
- ring = &rdev->ring[TN_RING_TYPE_VCE1_INDEX];
- if (ring->ring_size)
- r = radeon_ring_init(rdev, ring, ring->ring_size, 0,
- VCE_CMD_NO_OP);
-
- ring = &rdev->ring[TN_RING_TYPE_VCE2_INDEX];
- if (ring->ring_size)
- r = radeon_ring_init(rdev, ring, ring->ring_size, 0,
- VCE_CMD_NO_OP);
-
- if (!r)
- r = vce_v1_0_init(rdev);
- else if (r != -ENOENT)
- DRM_ERROR("radeon: failed initializing VCE (%d).\n", r);
+ si_uvd_resume(rdev);
+ si_vce_resume(rdev);
r = radeon_ib_pool_init(rdev);
if (r) {
@@ -7070,8 +7181,9 @@
if (rdev->has_uvd) {
uvd_v1_0_fini(rdev);
radeon_uvd_suspend(rdev);
- radeon_vce_suspend(rdev);
}
+ if (rdev->has_vce)
+ radeon_vce_suspend(rdev);
si_fini_pg(rdev);
si_fini_cg(rdev);
si_irq_suspend(rdev);
@@ -7169,25 +7281,8 @@
ring->ring_obj = NULL;
r600_ring_init(rdev, ring, 64 * 1024);
- if (rdev->has_uvd) {
- r = radeon_uvd_init(rdev);
- if (!r) {
- ring = &rdev->ring[R600_RING_TYPE_UVD_INDEX];
- ring->ring_obj = NULL;
- r600_ring_init(rdev, ring, 4096);
- }
- }
-
- r = radeon_vce_init(rdev);
- if (!r) {
- ring = &rdev->ring[TN_RING_TYPE_VCE1_INDEX];
- ring->ring_obj = NULL;
- r600_ring_init(rdev, ring, 4096);
-
- ring = &rdev->ring[TN_RING_TYPE_VCE2_INDEX];
- ring->ring_obj = NULL;
- r600_ring_init(rdev, ring, 4096);
- }
+ si_uvd_init(rdev);
+ si_vce_init(rdev);
rdev->ih.ring_obj = NULL;
r600_ih_ring_init(rdev, 64 * 1024);
@@ -7240,8 +7335,9 @@
if (rdev->has_uvd) {
uvd_v1_0_fini(rdev);
radeon_uvd_fini(rdev);
- radeon_vce_fini(rdev);
}
+ if (rdev->has_vce)
+ radeon_vce_fini(rdev);
si_pcie_gart_fini(rdev);
r600_vram_scratch_fini(rdev);
radeon_gem_fini(rdev);
diff --git a/drivers/gpu/drm/radeon/uvd_v1_0.c b/drivers/gpu/drm/radeon/uvd_v1_0.c
index 12ddcfa..0dbeb50 100644
--- a/drivers/gpu/drm/radeon/uvd_v1_0.c
+++ b/drivers/gpu/drm/radeon/uvd_v1_0.c
@@ -124,12 +124,13 @@
WREG32(UVD_VCPU_CACHE_SIZE0, size);
addr += size;
- size = RADEON_UVD_STACK_SIZE >> 3;
+ size = RADEON_UVD_HEAP_SIZE >> 3;
WREG32(UVD_VCPU_CACHE_OFFSET1, addr);
WREG32(UVD_VCPU_CACHE_SIZE1, size);
addr += size;
- size = RADEON_UVD_HEAP_SIZE >> 3;
+ size = (RADEON_UVD_STACK_SIZE +
+ (RADEON_UVD_SESSION_SIZE * rdev->uvd.max_handles)) >> 3;
WREG32(UVD_VCPU_CACHE_OFFSET2, addr);
WREG32(UVD_VCPU_CACHE_SIZE2, size);
diff --git a/drivers/gpu/drm/radeon/uvd_v2_2.c b/drivers/gpu/drm/radeon/uvd_v2_2.c
index 7ed778c..9071e65 100644
--- a/drivers/gpu/drm/radeon/uvd_v2_2.c
+++ b/drivers/gpu/drm/radeon/uvd_v2_2.c
@@ -116,12 +116,13 @@
WREG32(UVD_VCPU_CACHE_SIZE0, size);
addr += size;
- size = RADEON_UVD_STACK_SIZE >> 3;
+ size = RADEON_UVD_HEAP_SIZE >> 3;
WREG32(UVD_VCPU_CACHE_OFFSET1, addr);
WREG32(UVD_VCPU_CACHE_SIZE1, size);
addr += size;
- size = RADEON_UVD_HEAP_SIZE >> 3;
+ size = (RADEON_UVD_STACK_SIZE +
+ (RADEON_UVD_SESSION_SIZE * rdev->uvd.max_handles)) >> 3;
WREG32(UVD_VCPU_CACHE_OFFSET2, addr);
WREG32(UVD_VCPU_CACHE_SIZE2, size);
diff --git a/drivers/gpu/drm/radeon/uvd_v4_2.c b/drivers/gpu/drm/radeon/uvd_v4_2.c
index d04d507..91613b8 100644
--- a/drivers/gpu/drm/radeon/uvd_v4_2.c
+++ b/drivers/gpu/drm/radeon/uvd_v4_2.c
@@ -41,18 +41,25 @@
uint32_t size;
/* programm the VCPU memory controller bits 0-27 */
- addr = rdev->uvd.gpu_addr >> 3;
+
+ /* skip over the header of the new firmware format */
+ if (rdev->uvd.fw_header_present)
+ addr = (rdev->uvd.gpu_addr + 0x200) >> 3;
+ else
+ addr = rdev->uvd.gpu_addr >> 3;
+
size = RADEON_GPU_PAGE_ALIGN(rdev->uvd_fw->size + 4) >> 3;
WREG32(UVD_VCPU_CACHE_OFFSET0, addr);
WREG32(UVD_VCPU_CACHE_SIZE0, size);
addr += size;
- size = RADEON_UVD_STACK_SIZE >> 3;
+ size = RADEON_UVD_HEAP_SIZE >> 3;
WREG32(UVD_VCPU_CACHE_OFFSET1, addr);
WREG32(UVD_VCPU_CACHE_SIZE1, size);
addr += size;
- size = RADEON_UVD_HEAP_SIZE >> 3;
+ size = (RADEON_UVD_STACK_SIZE +
+ (RADEON_UVD_SESSION_SIZE * rdev->uvd.max_handles)) >> 3;
WREG32(UVD_VCPU_CACHE_OFFSET2, addr);
WREG32(UVD_VCPU_CACHE_SIZE2, size);
@@ -64,5 +71,8 @@
addr = (rdev->uvd.gpu_addr >> 32) & 0xFF;
WREG32(UVD_LMI_EXT40_ADDR, addr | (0x9 << 16) | (0x1 << 31));
+ if (rdev->uvd.fw_header_present)
+ WREG32(UVD_GP_SCRATCH4, rdev->uvd.max_handles);
+
return 0;
}
diff --git a/drivers/gpu/drm/rcar-du/Kconfig b/drivers/gpu/drm/rcar-du/Kconfig
index 1f10fa0..7fc3ca5 100644
--- a/drivers/gpu/drm/rcar-du/Kconfig
+++ b/drivers/gpu/drm/rcar-du/Kconfig
@@ -2,7 +2,7 @@
tristate "DRM Support for R-Car Display Unit"
depends on DRM && OF
depends on ARM || ARM64
- depends on ARCH_SHMOBILE || COMPILE_TEST
+ depends on ARCH_RENESAS || COMPILE_TEST
select DRM_KMS_HELPER
select DRM_KMS_CMA_HELPER
select DRM_GEM_CMA_HELPER
@@ -27,6 +27,6 @@
config DRM_RCAR_VSP
bool "R-Car DU VSP Compositor Support"
depends on DRM_RCAR_DU
- depends on VIDEO_RENESAS_VSP1
+ depends on VIDEO_RENESAS_VSP1=y || (VIDEO_RENESAS_VSP1 && DRM_RCAR_DU=m)
help
Enable support to expose the R-Car VSP Compositor as KMS planes.
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_crtc.c b/drivers/gpu/drm/rcar-du/rcar_du_crtc.c
index d9f06cc..0d8bdda 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_crtc.c
+++ b/drivers/gpu/drm/rcar-du/rcar_du_crtc.c
@@ -314,7 +314,7 @@
return;
spin_lock_irqsave(&dev->event_lock, flags);
- drm_send_vblank_event(dev, rcrtc->index, event);
+ drm_crtc_send_vblank_event(&rcrtc->crtc, event);
wake_up(&rcrtc->flip_wait);
spin_unlock_irqrestore(&dev->event_lock, flags);
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_drv.c b/drivers/gpu/drm/rcar-du/rcar_du_drv.c
index ed6006b..fb9242d 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_drv.c
+++ b/drivers/gpu/drm/rcar-du/rcar_du_drv.c
@@ -278,10 +278,7 @@
struct rcar_du_device *rcdu = platform_get_drvdata(pdev);
struct drm_device *ddev = rcdu->ddev;
- mutex_lock(&ddev->mode_config.mutex);
- drm_connector_unplug_all(ddev);
- mutex_unlock(&ddev->mode_config.mutex);
-
+ drm_connector_unregister_all(ddev);
drm_dev_unregister(ddev);
if (rcdu->fbdev)
@@ -300,7 +297,6 @@
{
struct device_node *np = pdev->dev.of_node;
struct rcar_du_device *rcdu;
- struct drm_connector *connector;
struct drm_device *ddev;
struct resource *mem;
int ret;
@@ -364,14 +360,7 @@
if (ret)
goto error;
- mutex_lock(&ddev->mode_config.mutex);
- drm_for_each_connector(connector, ddev) {
- ret = drm_connector_register(connector);
- if (ret < 0)
- break;
- }
- mutex_unlock(&ddev->mode_config.mutex);
-
+ ret = drm_connector_register_all(ddev);
if (ret < 0)
goto error;
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_kms.c b/drivers/gpu/drm/rcar-du/rcar_du_kms.c
index 24725bf..e70a4f3 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_kms.c
+++ b/drivers/gpu/drm/rcar-du/rcar_du_kms.c
@@ -283,7 +283,8 @@
}
static int rcar_du_atomic_commit(struct drm_device *dev,
- struct drm_atomic_state *state, bool async)
+ struct drm_atomic_state *state,
+ bool nonblock)
{
struct rcar_du_device *rcdu = dev->dev_private;
struct rcar_du_commit *commit;
@@ -328,7 +329,7 @@
/* Swap the state, this is the point of no return. */
drm_atomic_helper_swap_state(dev, state);
- if (async)
+ if (nonblock)
schedule_work(&commit->work);
else
rcar_du_atomic_complete(commit);
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_plane.c b/drivers/gpu/drm/rcar-du/rcar_du_plane.c
index 8460ae1..d445e67 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_plane.c
+++ b/drivers/gpu/drm/rcar-du/rcar_du_plane.c
@@ -635,7 +635,7 @@
static void rcar_du_plane_atomic_destroy_state(struct drm_plane *plane,
struct drm_plane_state *state)
{
- __drm_atomic_helper_plane_destroy_state(plane, state);
+ __drm_atomic_helper_plane_destroy_state(state);
kfree(to_rcar_plane_state(state));
}
diff --git a/drivers/gpu/drm/rcar-du/rcar_du_vsp.c b/drivers/gpu/drm/rcar-du/rcar_du_vsp.c
index de7ef04..e671a7c 100644
--- a/drivers/gpu/drm/rcar-du/rcar_du_vsp.c
+++ b/drivers/gpu/drm/rcar-du/rcar_du_vsp.c
@@ -251,7 +251,7 @@
static void rcar_du_vsp_plane_atomic_destroy_state(struct drm_plane *plane,
struct drm_plane_state *state)
{
- __drm_atomic_helper_plane_destroy_state(plane, state);
+ __drm_atomic_helper_plane_destroy_state(state);
kfree(to_rcar_vsp_plane_state(state));
}
diff --git a/drivers/gpu/drm/rockchip/Kconfig b/drivers/gpu/drm/rockchip/Kconfig
index 76b3362..d30bdc3 100644
--- a/drivers/gpu/drm/rockchip/Kconfig
+++ b/drivers/gpu/drm/rockchip/Kconfig
@@ -16,6 +16,15 @@
2D or 3D acceleration; acceleration is performed by other
IP found on the SoC.
+config ROCKCHIP_ANALOGIX_DP
+ tristate "Rockchip specific extensions for Analogix DP driver"
+ depends on DRM_ROCKCHIP
+ select DRM_ANALOGIX_DP
+ help
+ This selects support for Rockchip SoC specific extensions
+ for the Analogix Core DP driver. If you want to enable DP
+ on RK3288 based SoC, you should selet this option.
+
config ROCKCHIP_DW_HDMI
tristate "Rockchip specific extensions for Synopsys DW HDMI"
depends on DRM_ROCKCHIP
diff --git a/drivers/gpu/drm/rockchip/Makefile b/drivers/gpu/drm/rockchip/Makefile
index df8fbef..05d0713 100644
--- a/drivers/gpu/drm/rockchip/Makefile
+++ b/drivers/gpu/drm/rockchip/Makefile
@@ -6,6 +6,7 @@
rockchip_drm_gem.o rockchip_drm_vop.o
rockchipdrm-$(CONFIG_DRM_FBDEV_EMULATION) += rockchip_drm_fbdev.o
+obj-$(CONFIG_ROCKCHIP_ANALOGIX_DP) += analogix_dp-rockchip.o
obj-$(CONFIG_ROCKCHIP_DW_HDMI) += dw_hdmi-rockchip.o
obj-$(CONFIG_ROCKCHIP_DW_MIPI_DSI) += dw-mipi-dsi.o
obj-$(CONFIG_ROCKCHIP_INNO_HDMI) += inno_hdmi.o
diff --git a/drivers/gpu/drm/rockchip/analogix_dp-rockchip.c b/drivers/gpu/drm/rockchip/analogix_dp-rockchip.c
new file mode 100644
index 0000000..7f6a55c
--- /dev/null
+++ b/drivers/gpu/drm/rockchip/analogix_dp-rockchip.c
@@ -0,0 +1,390 @@
+/*
+ * Rockchip SoC DP (Display Port) interface driver.
+ *
+ * Copyright (C) Fuzhou Rockchip Electronics Co., Ltd.
+ * Author: Andy Yan <andy.yan@rock-chips.com>
+ * Yakir Yang <ykk@rock-chips.com>
+ * Jeff Chen <jeff.chen@rock-chips.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#include <linux/component.h>
+#include <linux/mfd/syscon.h>
+#include <linux/of_graph.h>
+#include <linux/regmap.h>
+#include <linux/reset.h>
+#include <linux/clk.h>
+
+#include <drm/drmP.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_dp_helper.h>
+#include <drm/drm_of.h>
+#include <drm/drm_panel.h>
+
+#include <video/of_videomode.h>
+#include <video/videomode.h>
+
+#include <drm/bridge/analogix_dp.h>
+
+#include "rockchip_drm_drv.h"
+#include "rockchip_drm_vop.h"
+
+#define to_dp(nm) container_of(nm, struct rockchip_dp_device, nm)
+
+/* dp grf register offset */
+#define GRF_SOC_CON6 0x025c
+#define GRF_EDP_LCD_SEL_MASK BIT(5)
+#define GRF_EDP_SEL_VOP_LIT BIT(5)
+#define GRF_EDP_SEL_VOP_BIG 0
+
+struct rockchip_dp_device {
+ struct drm_device *drm_dev;
+ struct device *dev;
+ struct drm_encoder encoder;
+ struct drm_display_mode mode;
+
+ struct clk *pclk;
+ struct regmap *grf;
+ struct reset_control *rst;
+
+ struct analogix_dp_plat_data plat_data;
+};
+
+static int rockchip_dp_pre_init(struct rockchip_dp_device *dp)
+{
+ reset_control_assert(dp->rst);
+ usleep_range(10, 20);
+ reset_control_deassert(dp->rst);
+
+ return 0;
+}
+
+static int rockchip_dp_poweron(struct analogix_dp_plat_data *plat_data)
+{
+ struct rockchip_dp_device *dp = to_dp(plat_data);
+ int ret;
+
+ ret = clk_prepare_enable(dp->pclk);
+ if (ret < 0) {
+ dev_err(dp->dev, "failed to enable pclk %d\n", ret);
+ return ret;
+ }
+
+ ret = rockchip_dp_pre_init(dp);
+ if (ret < 0) {
+ dev_err(dp->dev, "failed to dp pre init %d\n", ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int rockchip_dp_powerdown(struct analogix_dp_plat_data *plat_data)
+{
+ struct rockchip_dp_device *dp = to_dp(plat_data);
+
+ clk_disable_unprepare(dp->pclk);
+
+ return 0;
+}
+
+static bool
+rockchip_dp_drm_encoder_mode_fixup(struct drm_encoder *encoder,
+ const struct drm_display_mode *mode,
+ struct drm_display_mode *adjusted_mode)
+{
+ /* do nothing */
+ return true;
+}
+
+static void rockchip_dp_drm_encoder_mode_set(struct drm_encoder *encoder,
+ struct drm_display_mode *mode,
+ struct drm_display_mode *adjusted)
+{
+ /* do nothing */
+}
+
+static void rockchip_dp_drm_encoder_enable(struct drm_encoder *encoder)
+{
+ struct rockchip_dp_device *dp = to_dp(encoder);
+ int ret;
+ u32 val;
+
+ ret = drm_of_encoder_active_endpoint_id(dp->dev->of_node, encoder);
+ if (ret < 0)
+ return;
+
+ if (ret)
+ val = GRF_EDP_SEL_VOP_LIT | (GRF_EDP_LCD_SEL_MASK << 16);
+ else
+ val = GRF_EDP_SEL_VOP_BIG | (GRF_EDP_LCD_SEL_MASK << 16);
+
+ dev_dbg(dp->dev, "vop %s output to dp\n", (ret) ? "LIT" : "BIG");
+
+ ret = regmap_write(dp->grf, GRF_SOC_CON6, val);
+ if (ret != 0) {
+ dev_err(dp->dev, "Could not write to GRF: %d\n", ret);
+ return;
+ }
+}
+
+static void rockchip_dp_drm_encoder_nop(struct drm_encoder *encoder)
+{
+ /* do nothing */
+}
+
+static int
+rockchip_dp_drm_encoder_atomic_check(struct drm_encoder *encoder,
+ struct drm_crtc_state *crtc_state,
+ struct drm_connector_state *conn_state)
+{
+ struct rockchip_crtc_state *s = to_rockchip_crtc_state(crtc_state);
+
+ /*
+ * FIXME(Yakir): driver should configure the CRTC output video
+ * mode with the display information which indicated the monitor
+ * support colorimetry.
+ *
+ * But don't know why the CRTC driver seems could only output the
+ * RGBaaa rightly. For example, if connect the "innolux,n116bge"
+ * eDP screen, EDID would indicated that screen only accepted the
+ * 6bpc mode. But if I configure CRTC to RGB666 output, then eDP
+ * screen would show a blue picture (RGB888 show a green picture).
+ * But if I configure CTRC to RGBaaa, and eDP driver still keep
+ * RGB666 input video mode, then screen would works prefect.
+ */
+ s->output_mode = ROCKCHIP_OUT_MODE_AAAA;
+ s->output_type = DRM_MODE_CONNECTOR_eDP;
+
+ return 0;
+}
+
+static struct drm_encoder_helper_funcs rockchip_dp_encoder_helper_funcs = {
+ .mode_fixup = rockchip_dp_drm_encoder_mode_fixup,
+ .mode_set = rockchip_dp_drm_encoder_mode_set,
+ .enable = rockchip_dp_drm_encoder_enable,
+ .disable = rockchip_dp_drm_encoder_nop,
+ .atomic_check = rockchip_dp_drm_encoder_atomic_check,
+};
+
+static void rockchip_dp_drm_encoder_destroy(struct drm_encoder *encoder)
+{
+ drm_encoder_cleanup(encoder);
+}
+
+static struct drm_encoder_funcs rockchip_dp_encoder_funcs = {
+ .destroy = rockchip_dp_drm_encoder_destroy,
+};
+
+static int rockchip_dp_init(struct rockchip_dp_device *dp)
+{
+ struct device *dev = dp->dev;
+ struct device_node *np = dev->of_node;
+ int ret;
+
+ dp->grf = syscon_regmap_lookup_by_phandle(np, "rockchip,grf");
+ if (IS_ERR(dp->grf)) {
+ dev_err(dev, "failed to get rockchip,grf property\n");
+ return PTR_ERR(dp->grf);
+ }
+
+ dp->pclk = devm_clk_get(dev, "pclk");
+ if (IS_ERR(dp->pclk)) {
+ dev_err(dev, "failed to get pclk property\n");
+ return PTR_ERR(dp->pclk);
+ }
+
+ dp->rst = devm_reset_control_get(dev, "dp");
+ if (IS_ERR(dp->rst)) {
+ dev_err(dev, "failed to get dp reset control\n");
+ return PTR_ERR(dp->rst);
+ }
+
+ ret = clk_prepare_enable(dp->pclk);
+ if (ret < 0) {
+ dev_err(dp->dev, "failed to enable pclk %d\n", ret);
+ return ret;
+ }
+
+ ret = rockchip_dp_pre_init(dp);
+ if (ret < 0) {
+ dev_err(dp->dev, "failed to pre init %d\n", ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int rockchip_dp_drm_create_encoder(struct rockchip_dp_device *dp)
+{
+ struct drm_encoder *encoder = &dp->encoder;
+ struct drm_device *drm_dev = dp->drm_dev;
+ struct device *dev = dp->dev;
+ int ret;
+
+ encoder->possible_crtcs = drm_of_find_possible_crtcs(drm_dev,
+ dev->of_node);
+ DRM_DEBUG_KMS("possible_crtcs = 0x%x\n", encoder->possible_crtcs);
+
+ ret = drm_encoder_init(drm_dev, encoder, &rockchip_dp_encoder_funcs,
+ DRM_MODE_ENCODER_TMDS, NULL);
+ if (ret) {
+ DRM_ERROR("failed to initialize encoder with drm\n");
+ return ret;
+ }
+
+ drm_encoder_helper_add(encoder, &rockchip_dp_encoder_helper_funcs);
+
+ return 0;
+}
+
+static int rockchip_dp_bind(struct device *dev, struct device *master,
+ void *data)
+{
+ struct rockchip_dp_device *dp = dev_get_drvdata(dev);
+ struct drm_device *drm_dev = data;
+ int ret;
+
+ /*
+ * Just like the probe function said, we don't need the
+ * device drvrate anymore, we should leave the charge to
+ * analogix dp driver, set the device drvdata to NULL.
+ */
+ dev_set_drvdata(dev, NULL);
+
+ ret = rockchip_dp_init(dp);
+ if (ret < 0)
+ return ret;
+
+ dp->drm_dev = drm_dev;
+
+ ret = rockchip_dp_drm_create_encoder(dp);
+ if (ret) {
+ DRM_ERROR("failed to create drm encoder\n");
+ return ret;
+ }
+
+ dp->plat_data.encoder = &dp->encoder;
+
+ dp->plat_data.dev_type = RK3288_DP;
+ dp->plat_data.power_on = rockchip_dp_poweron;
+ dp->plat_data.power_off = rockchip_dp_powerdown;
+
+ return analogix_dp_bind(dev, dp->drm_dev, &dp->plat_data);
+}
+
+static void rockchip_dp_unbind(struct device *dev, struct device *master,
+ void *data)
+{
+ return analogix_dp_unbind(dev, master, data);
+}
+
+static const struct component_ops rockchip_dp_component_ops = {
+ .bind = rockchip_dp_bind,
+ .unbind = rockchip_dp_unbind,
+};
+
+static int rockchip_dp_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *panel_node, *port, *endpoint;
+ struct rockchip_dp_device *dp;
+ struct drm_panel *panel;
+
+ port = of_graph_get_port_by_id(dev->of_node, 1);
+ if (!port) {
+ dev_err(dev, "can't find output port\n");
+ return -EINVAL;
+ }
+
+ endpoint = of_get_child_by_name(port, "endpoint");
+ of_node_put(port);
+ if (!endpoint) {
+ dev_err(dev, "no output endpoint found\n");
+ return -EINVAL;
+ }
+
+ panel_node = of_graph_get_remote_port_parent(endpoint);
+ of_node_put(endpoint);
+ if (!panel_node) {
+ dev_err(dev, "no output node found\n");
+ return -EINVAL;
+ }
+
+ panel = of_drm_find_panel(panel_node);
+ if (!panel) {
+ DRM_ERROR("failed to find panel\n");
+ of_node_put(panel_node);
+ return -EPROBE_DEFER;
+ }
+
+ of_node_put(panel_node);
+
+ dp = devm_kzalloc(dev, sizeof(*dp), GFP_KERNEL);
+ if (!dp)
+ return -ENOMEM;
+
+ dp->dev = dev;
+
+ dp->plat_data.panel = panel;
+
+ /*
+ * We just use the drvdata until driver run into component
+ * add function, and then we would set drvdata to null, so
+ * that analogix dp driver could take charge of the drvdata.
+ */
+ platform_set_drvdata(pdev, dp);
+
+ return component_add(dev, &rockchip_dp_component_ops);
+}
+
+static int rockchip_dp_remove(struct platform_device *pdev)
+{
+ component_del(&pdev->dev, &rockchip_dp_component_ops);
+
+ return 0;
+}
+
+#ifdef CONFIG_PM_SLEEP
+static int rockchip_dp_suspend(struct device *dev)
+{
+ return analogix_dp_suspend(dev);
+}
+
+static int rockchip_dp_resume(struct device *dev)
+{
+ return analogix_dp_resume(dev);
+}
+#endif
+
+static const struct dev_pm_ops rockchip_dp_pm_ops = {
+ SET_SYSTEM_SLEEP_PM_OPS(rockchip_dp_suspend, rockchip_dp_resume)
+};
+
+static const struct of_device_id rockchip_dp_dt_ids[] = {
+ {.compatible = "rockchip,rk3288-dp",},
+ {}
+};
+MODULE_DEVICE_TABLE(of, rockchip_dp_dt_ids);
+
+static struct platform_driver rockchip_dp_driver = {
+ .probe = rockchip_dp_probe,
+ .remove = rockchip_dp_remove,
+ .driver = {
+ .name = "rockchip-dp",
+ .owner = THIS_MODULE,
+ .pm = &rockchip_dp_pm_ops,
+ .of_match_table = of_match_ptr(rockchip_dp_dt_ids),
+ },
+};
+
+module_platform_driver(rockchip_dp_driver);
+
+MODULE_AUTHOR("Yakir Yang <ykk@rock-chips.com>");
+MODULE_AUTHOR("Jeff chen <jeff.chen@rock-chips.com>");
+MODULE_DESCRIPTION("Rockchip Specific Analogix-DP Driver Extension");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/gpu/drm/rockchip/dw-mipi-dsi.c b/drivers/gpu/drm/rockchip/dw-mipi-dsi.c
index 7975158..dedc65b 100644
--- a/drivers/gpu/drm/rockchip/dw-mipi-dsi.c
+++ b/drivers/gpu/drm/rockchip/dw-mipi-dsi.c
@@ -879,7 +879,6 @@
{
struct dw_mipi_dsi *dsi = encoder_to_dsi(encoder);
int mux = drm_of_encoder_active_endpoint_id(dsi->dev->of_node, encoder);
- u32 interface_pix_fmt;
u32 val;
if (clk_prepare_enable(dsi->pclk)) {
@@ -895,24 +894,6 @@
clk_disable_unprepare(dsi->pclk);
- switch (dsi->format) {
- case MIPI_DSI_FMT_RGB888:
- interface_pix_fmt = ROCKCHIP_OUT_MODE_P888;
- break;
- case MIPI_DSI_FMT_RGB666:
- interface_pix_fmt = ROCKCHIP_OUT_MODE_P666;
- break;
- case MIPI_DSI_FMT_RGB565:
- interface_pix_fmt = ROCKCHIP_OUT_MODE_P565;
- break;
- default:
- WARN_ON(1);
- return;
- }
-
- rockchip_drm_crtc_mode_config(encoder->crtc, DRM_MODE_CONNECTOR_DSI,
- interface_pix_fmt);
-
if (mux)
val = DSI0_SEL_VOP_LIT | (DSI0_SEL_VOP_LIT << 16);
else
@@ -922,11 +903,40 @@
dev_dbg(dsi->dev, "vop %s output to dsi0\n", (mux) ? "LIT" : "BIG");
}
+static int
+dw_mipi_dsi_encoder_atomic_check(struct drm_encoder *encoder,
+ struct drm_crtc_state *crtc_state,
+ struct drm_connector_state *conn_state)
+{
+ struct rockchip_crtc_state *s = to_rockchip_crtc_state(crtc_state);
+ struct dw_mipi_dsi *dsi = encoder_to_dsi(encoder);
+
+ switch (dsi->format) {
+ case MIPI_DSI_FMT_RGB888:
+ s->output_mode = ROCKCHIP_OUT_MODE_P888;
+ break;
+ case MIPI_DSI_FMT_RGB666:
+ s->output_mode = ROCKCHIP_OUT_MODE_P666;
+ break;
+ case MIPI_DSI_FMT_RGB565:
+ s->output_mode = ROCKCHIP_OUT_MODE_P565;
+ break;
+ default:
+ WARN_ON(1);
+ return -EINVAL;
+ }
+
+ s->output_type = DRM_MODE_CONNECTOR_DSI;
+
+ return 0;
+}
+
static struct drm_encoder_helper_funcs
dw_mipi_dsi_encoder_helper_funcs = {
.commit = dw_mipi_dsi_encoder_commit,
.mode_set = dw_mipi_dsi_encoder_mode_set,
.disable = dw_mipi_dsi_encoder_disable,
+ .atomic_check = dw_mipi_dsi_encoder_atomic_check,
};
static struct drm_encoder_funcs dw_mipi_dsi_encoder_funcs = {
diff --git a/drivers/gpu/drm/rockchip/dw_hdmi-rockchip.c b/drivers/gpu/drm/rockchip/dw_hdmi-rockchip.c
index d5cfef7..801110f 100644
--- a/drivers/gpu/drm/rockchip/dw_hdmi-rockchip.c
+++ b/drivers/gpu/drm/rockchip/dw_hdmi-rockchip.c
@@ -201,9 +201,6 @@
u32 val;
int mux;
- rockchip_drm_crtc_mode_config(encoder->crtc, DRM_MODE_CONNECTOR_HDMIA,
- ROCKCHIP_OUT_MODE_AAAA);
-
mux = drm_of_encoder_active_endpoint_id(hdmi->dev->of_node, encoder);
if (mux)
val = HDMI_SEL_VOP_LIT | (HDMI_SEL_VOP_LIT << 16);
@@ -215,11 +212,25 @@
(mux) ? "LIT" : "BIG");
}
+static int
+dw_hdmi_rockchip_encoder_atomic_check(struct drm_encoder *encoder,
+ struct drm_crtc_state *crtc_state,
+ struct drm_connector_state *conn_state)
+{
+ struct rockchip_crtc_state *s = to_rockchip_crtc_state(crtc_state);
+
+ s->output_mode = ROCKCHIP_OUT_MODE_AAAA;
+ s->output_type = DRM_MODE_CONNECTOR_HDMIA;
+
+ return 0;
+}
+
static const struct drm_encoder_helper_funcs dw_hdmi_rockchip_encoder_helper_funcs = {
.mode_fixup = dw_hdmi_rockchip_encoder_mode_fixup,
.mode_set = dw_hdmi_rockchip_encoder_mode_set,
.enable = dw_hdmi_rockchip_encoder_enable,
.disable = dw_hdmi_rockchip_encoder_disable,
+ .atomic_check = dw_hdmi_rockchip_encoder_atomic_check,
};
static const struct dw_hdmi_plat_data rockchip_hdmi_drv_data = {
diff --git a/drivers/gpu/drm/rockchip/inno_hdmi.c b/drivers/gpu/drm/rockchip/inno_hdmi.c
index 10d62ff..f8b4feb 100644
--- a/drivers/gpu/drm/rockchip/inno_hdmi.c
+++ b/drivers/gpu/drm/rockchip/inno_hdmi.c
@@ -500,9 +500,6 @@
{
struct inno_hdmi *hdmi = to_inno_hdmi(encoder);
- rockchip_drm_crtc_mode_config(encoder->crtc, DRM_MODE_CONNECTOR_HDMIA,
- ROCKCHIP_OUT_MODE_P888);
-
inno_hdmi_set_pwr_mode(hdmi, NORMAL);
}
@@ -520,11 +517,25 @@
return true;
}
+static int
+inno_hdmi_encoder_atomic_check(struct drm_encoder *encoder,
+ struct drm_crtc_state *crtc_state,
+ struct drm_connector_state *conn_state)
+{
+ struct rockchip_crtc_state *s = to_rockchip_crtc_state(crtc_state);
+
+ s->output_mode = ROCKCHIP_OUT_MODE_P888;
+ s->output_type = DRM_MODE_CONNECTOR_HDMIA;
+
+ return 0;
+}
+
static struct drm_encoder_helper_funcs inno_hdmi_encoder_helper_funcs = {
.enable = inno_hdmi_encoder_enable,
.disable = inno_hdmi_encoder_disable,
.mode_fixup = inno_hdmi_encoder_mode_fixup,
.mode_set = inno_hdmi_encoder_mode_set,
+ .atomic_check = inno_hdmi_encoder_atomic_check,
};
static struct drm_encoder_funcs inno_hdmi_encoder_funcs = {
@@ -855,8 +866,9 @@
hdmi->ddc = inno_hdmi_i2c_adapter(hdmi);
if (IS_ERR(hdmi->ddc)) {
+ ret = PTR_ERR(hdmi->ddc);
hdmi->ddc = NULL;
- return PTR_ERR(hdmi->ddc);
+ return ret;
}
/*
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_drv.c b/drivers/gpu/drm/rockchip/rockchip_drm_drv.c
index f556a8f..a409d1f 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_drv.c
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_drv.c
@@ -36,6 +36,8 @@
#define DRIVER_MAJOR 1
#define DRIVER_MINOR 0
+static bool is_support_iommu = true;
+
/*
* Attach a (component) device to the shared drm dma mapping from master drm
* device. This is used by the VOPs to map GEM buffers to a common DMA
@@ -47,6 +49,9 @@
struct dma_iommu_mapping *mapping = drm_dev->dev->archdata.mapping;
int ret;
+ if (!is_support_iommu)
+ return 0;
+
ret = dma_set_coherent_mask(dev, DMA_BIT_MASK(32));
if (ret)
return ret;
@@ -59,6 +64,9 @@
void rockchip_drm_dma_detach_device(struct drm_device *drm_dev,
struct device *dev)
{
+ if (!is_support_iommu)
+ return;
+
arm_iommu_detach_device(dev);
}
@@ -127,7 +135,7 @@
static int rockchip_drm_load(struct drm_device *drm_dev, unsigned long flags)
{
struct rockchip_drm_private *private;
- struct dma_iommu_mapping *mapping;
+ struct dma_iommu_mapping *mapping = NULL;
struct device *dev = drm_dev->dev;
struct drm_connector *connector;
int ret;
@@ -152,24 +160,27 @@
goto err_config_cleanup;
}
- /* TODO(djkurtz): fetch the mapping start/size from somewhere */
- mapping = arm_iommu_create_mapping(&platform_bus_type, 0x00000000,
- SZ_2G);
- if (IS_ERR(mapping)) {
- ret = PTR_ERR(mapping);
- goto err_config_cleanup;
+ if (is_support_iommu) {
+ /* TODO(djkurtz): fetch the mapping start/size from somewhere */
+ mapping = arm_iommu_create_mapping(&platform_bus_type,
+ 0x00000000,
+ SZ_2G);
+ if (IS_ERR(mapping)) {
+ ret = PTR_ERR(mapping);
+ goto err_config_cleanup;
+ }
+
+ ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32));
+ if (ret)
+ goto err_release_mapping;
+
+ dma_set_max_seg_size(dev, DMA_BIT_MASK(32));
+
+ ret = arm_iommu_attach_device(dev, mapping);
+ if (ret)
+ goto err_release_mapping;
}
- ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32));
- if (ret)
- goto err_release_mapping;
-
- dma_set_max_seg_size(dev, DMA_BIT_MASK(32));
-
- ret = arm_iommu_attach_device(dev, mapping);
- if (ret)
- goto err_release_mapping;
-
/* Try to bind all sub drivers. */
ret = component_bind_all(dev, drm_dev);
if (ret)
@@ -205,19 +216,14 @@
if (ret)
goto err_kms_helper_poll_fini;
- /*
- * with vblank_disable_allowed = true, vblank interrupt will be disabled
- * by drm timer once a current process gives up ownership of
- * vblank event.(after drm_vblank_put function is called)
- */
- drm_dev->vblank_disable_allowed = true;
-
drm_mode_config_reset(drm_dev);
ret = rockchip_drm_fbdev_init(drm_dev);
if (ret)
goto err_vblank_cleanup;
+ if (is_support_iommu)
+ arm_iommu_release_mapping(mapping);
return 0;
err_vblank_cleanup:
drm_vblank_cleanup(drm_dev);
@@ -226,9 +232,11 @@
err_unbind:
component_unbind_all(dev, drm_dev);
err_detach_device:
- arm_iommu_detach_device(dev);
+ if (is_support_iommu)
+ arm_iommu_detach_device(dev);
err_release_mapping:
- arm_iommu_release_mapping(dev->archdata.mapping);
+ if (is_support_iommu)
+ arm_iommu_release_mapping(mapping);
err_config_cleanup:
drm_mode_config_cleanup(drm_dev);
drm_dev->dev_private = NULL;
@@ -243,8 +251,8 @@
drm_vblank_cleanup(drm_dev);
drm_kms_helper_poll_fini(drm_dev);
component_unbind_all(dev, drm_dev);
- arm_iommu_detach_device(dev);
- arm_iommu_release_mapping(dev->archdata.mapping);
+ if (is_support_iommu)
+ arm_iommu_detach_device(dev);
drm_mode_config_cleanup(drm_dev);
drm_dev->dev_private = NULL;
@@ -488,6 +496,8 @@
* works as expected.
*/
for (i = 0;; i++) {
+ struct device_node *iommu;
+
port = of_parse_phandle(np, "ports", i);
if (!port)
break;
@@ -497,6 +507,17 @@
continue;
}
+ iommu = of_parse_phandle(port->parent, "iommus", 0);
+ if (!iommu || !of_device_is_available(iommu->parent)) {
+ dev_dbg(dev, "no iommu attached for %s, using non-iommu buffers\n",
+ port->parent->full_name);
+ /*
+ * if there is a crtc not support iommu, force set all
+ * crtc use non-iommu buffer.
+ */
+ is_support_iommu = false;
+ }
+
component_match_add(dev, &match, compare_of, port->parent);
of_node_put(port);
}
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_drv.h b/drivers/gpu/drm/rockchip/rockchip_drm_drv.h
index 00d17d7..56f43a3 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_drv.h
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_drv.h
@@ -50,6 +50,14 @@
struct mutex lock;
};
+struct rockchip_crtc_state {
+ struct drm_crtc_state base;
+ int output_type;
+ int output_mode;
+};
+#define to_rockchip_crtc_state(s) \
+ container_of(s, struct rockchip_crtc_state, base)
+
/*
* Rockchip drm private structure.
*
@@ -68,8 +76,6 @@
int rockchip_register_crtc_funcs(struct drm_crtc *crtc,
const struct rockchip_crtc_funcs *crtc_funcs);
void rockchip_unregister_crtc_funcs(struct drm_crtc *crtc);
-int rockchip_drm_crtc_mode_config(struct drm_crtc *crtc, int connector_type,
- int out_mode);
int rockchip_drm_dma_attach_device(struct drm_device *drm_dev,
struct device *dev);
void rockchip_drm_dma_detach_device(struct drm_device *drm_dev,
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_fb.c b/drivers/gpu/drm/rockchip/rockchip_drm_fb.c
index 3b8f652..755cfdb 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_fb.c
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_fb.c
@@ -123,8 +123,7 @@
unsigned int height = mode_cmd->height / (i ? vsub : 1);
unsigned int min_size;
- obj = drm_gem_object_lookup(dev, file_priv,
- mode_cmd->handles[i]);
+ obj = drm_gem_object_lookup(file_priv, mode_cmd->handles[i]);
if (!obj) {
dev_err(dev->dev, "Failed to lookup GEM object\n");
ret = -ENXIO;
@@ -276,7 +275,7 @@
int rockchip_drm_atomic_commit(struct drm_device *dev,
struct drm_atomic_state *state,
- bool async)
+ bool nonblock)
{
struct rockchip_drm_private *private = dev->dev_private;
struct rockchip_atomic_commit *commit = &private->commit;
@@ -286,7 +285,7 @@
if (ret)
return ret;
- /* serialize outstanding asynchronous commits */
+ /* serialize outstanding nonblocking commits */
mutex_lock(&commit->lock);
flush_work(&commit->work);
@@ -295,7 +294,7 @@
commit->dev = dev;
commit->state = state;
- if (async)
+ if (nonblock)
schedule_work(&commit->work);
else
rockchip_atomic_commit_complete(commit);
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_gem.c b/drivers/gpu/drm/rockchip/rockchip_drm_gem.c
index 18e0733..9c2d8a8 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_gem.c
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_gem.c
@@ -198,7 +198,7 @@
struct drm_gem_object *obj;
int ret;
- obj = drm_gem_object_lookup(dev, file_priv, handle);
+ obj = drm_gem_object_lookup(file_priv, handle);
if (!obj) {
DRM_ERROR("failed to lookup gem object.\n");
return -EINVAL;
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_vop.c b/drivers/gpu/drm/rockchip/rockchip_drm_vop.c
index a619f12..1c4d5b5 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_vop.c
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_vop.c
@@ -310,7 +310,7 @@
uint16_t vsu_mode;
uint16_t lb_mode;
uint32_t val;
- int vskiplines;
+ int vskiplines = 0;
if (dst_w > 3840) {
DRM_ERROR("Maximum destination width (3840) exceeded\n");
@@ -560,6 +560,22 @@
drm_plane_cleanup(plane);
}
+static int vop_plane_prepare_fb(struct drm_plane *plane,
+ const struct drm_plane_state *new_state)
+{
+ if (plane->state->fb)
+ drm_framebuffer_reference(plane->state->fb);
+
+ return 0;
+}
+
+static void vop_plane_cleanup_fb(struct drm_plane *plane,
+ const struct drm_plane_state *old_state)
+{
+ if (old_state->fb)
+ drm_framebuffer_unreference(old_state->fb);
+}
+
static int vop_plane_atomic_check(struct drm_plane *plane,
struct drm_plane_state *state)
{
@@ -756,6 +772,8 @@
}
static const struct drm_plane_helper_funcs plane_helper_funcs = {
+ .prepare_fb = vop_plane_prepare_fb,
+ .cleanup_fb = vop_plane_cleanup_fb,
.atomic_check = vop_plane_atomic_check,
.atomic_update = vop_plane_atomic_update,
.atomic_disable = vop_plane_atomic_disable,
@@ -804,7 +822,7 @@
{
struct vop_plane_state *vop_state = to_vop_plane_state(state);
- __drm_atomic_helper_plane_destroy_state(plane, state);
+ __drm_atomic_helper_plane_destroy_state(state);
kfree(vop_state);
}
@@ -818,38 +836,6 @@
.atomic_destroy_state = vop_atomic_plane_destroy_state,
};
-int rockchip_drm_crtc_mode_config(struct drm_crtc *crtc,
- int connector_type,
- int out_mode)
-{
- struct vop *vop = to_vop(crtc);
-
- if (WARN_ON(!vop->is_enabled))
- return -EINVAL;
-
- switch (connector_type) {
- case DRM_MODE_CONNECTOR_LVDS:
- VOP_CTRL_SET(vop, rgb_en, 1);
- break;
- case DRM_MODE_CONNECTOR_eDP:
- VOP_CTRL_SET(vop, edp_en, 1);
- break;
- case DRM_MODE_CONNECTOR_HDMIA:
- VOP_CTRL_SET(vop, hdmi_en, 1);
- break;
- case DRM_MODE_CONNECTOR_DSI:
- VOP_CTRL_SET(vop, mipi_en, 1);
- break;
- default:
- DRM_ERROR("unsupport connector_type[%d]\n", connector_type);
- return -EINVAL;
- };
- VOP_CTRL_SET(vop, out_mode, out_mode);
-
- return 0;
-}
-EXPORT_SYMBOL_GPL(rockchip_drm_crtc_mode_config);
-
static int vop_crtc_enable_vblank(struct drm_crtc *crtc)
{
struct vop *vop = to_vop(crtc);
@@ -931,6 +917,7 @@
static void vop_crtc_enable(struct drm_crtc *crtc)
{
struct vop *vop = to_vop(crtc);
+ struct rockchip_crtc_state *s = to_rockchip_crtc_state(crtc->state);
struct drm_display_mode *adjusted_mode = &crtc->state->adjusted_mode;
u16 hsync_len = adjusted_mode->hsync_end - adjusted_mode->hsync_start;
u16 hdisplay = adjusted_mode->hdisplay;
@@ -985,6 +972,23 @@
val |= (adjusted_mode->flags & DRM_MODE_FLAG_NHSYNC) ? 0 : 1;
val |= (adjusted_mode->flags & DRM_MODE_FLAG_NVSYNC) ? 0 : (1 << 1);
VOP_CTRL_SET(vop, pin_pol, val);
+ switch (s->output_type) {
+ case DRM_MODE_CONNECTOR_LVDS:
+ VOP_CTRL_SET(vop, rgb_en, 1);
+ break;
+ case DRM_MODE_CONNECTOR_eDP:
+ VOP_CTRL_SET(vop, edp_en, 1);
+ break;
+ case DRM_MODE_CONNECTOR_HDMIA:
+ VOP_CTRL_SET(vop, hdmi_en, 1);
+ break;
+ case DRM_MODE_CONNECTOR_DSI:
+ VOP_CTRL_SET(vop, mipi_en, 1);
+ break;
+ default:
+ DRM_ERROR("unsupport connector_type[%d]\n", s->output_type);
+ }
+ VOP_CTRL_SET(vop, out_mode, s->output_mode);
VOP_CTRL_SET(vop, htotal_pw, (htotal << 16) | hsync_len);
val = hact_st << 16;
@@ -1044,13 +1048,34 @@
drm_crtc_cleanup(crtc);
}
+static struct drm_crtc_state *vop_crtc_duplicate_state(struct drm_crtc *crtc)
+{
+ struct rockchip_crtc_state *rockchip_state;
+
+ rockchip_state = kzalloc(sizeof(*rockchip_state), GFP_KERNEL);
+ if (!rockchip_state)
+ return NULL;
+
+ __drm_atomic_helper_crtc_duplicate_state(crtc, &rockchip_state->base);
+ return &rockchip_state->base;
+}
+
+static void vop_crtc_destroy_state(struct drm_crtc *crtc,
+ struct drm_crtc_state *state)
+{
+ struct rockchip_crtc_state *s = to_rockchip_crtc_state(state);
+
+ __drm_atomic_helper_crtc_destroy_state(&s->base);
+ kfree(s);
+}
+
static const struct drm_crtc_funcs vop_crtc_funcs = {
.set_config = drm_atomic_helper_set_config,
.page_flip = drm_atomic_helper_page_flip,
.destroy = vop_crtc_destroy,
.reset = drm_atomic_helper_crtc_reset,
- .atomic_duplicate_state = drm_atomic_helper_crtc_duplicate_state,
- .atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
+ .atomic_duplicate_state = vop_crtc_duplicate_state,
+ .atomic_destroy_state = vop_crtc_destroy_state,
};
static bool vop_win_pending_is_complete(struct vop_win *vop_win)
diff --git a/drivers/gpu/drm/shmobile/shmob_drm_crtc.c b/drivers/gpu/drm/shmobile/shmob_drm_crtc.c
index 88643ab..1e154fc 100644
--- a/drivers/gpu/drm/shmobile/shmob_drm_crtc.c
+++ b/drivers/gpu/drm/shmobile/shmob_drm_crtc.c
@@ -440,7 +440,7 @@
event = scrtc->event;
scrtc->event = NULL;
if (event) {
- drm_send_vblank_event(dev, 0, event);
+ drm_crtc_send_vblank_event(&scrtc->crtc, event);
drm_vblank_put(dev, 0);
}
spin_unlock_irqrestore(&dev->event_lock, flags);
diff --git a/drivers/gpu/drm/sti/sti_cursor.c b/drivers/gpu/drm/sti/sti_cursor.c
index 3abb400..4e99029 100644
--- a/drivers/gpu/drm/sti/sti_cursor.c
+++ b/drivers/gpu/drm/sti/sti_cursor.c
@@ -6,6 +6,8 @@
* License terms: GNU General Public License (GPL), version 2
*/
+#include <linux/seq_file.h>
+
#include <drm/drm_atomic.h>
#include <drm/drm_fb_cma_helper.h>
#include <drm/drm_gem_cma_helper.h>
diff --git a/drivers/gpu/drm/sti/sti_drv.c b/drivers/gpu/drm/sti/sti_drv.c
index 6bd6aba..872495e 100644
--- a/drivers/gpu/drm/sti/sti_drv.c
+++ b/drivers/gpu/drm/sti/sti_drv.c
@@ -202,7 +202,7 @@
}
static int sti_atomic_commit(struct drm_device *drm,
- struct drm_atomic_state *state, bool async)
+ struct drm_atomic_state *state, bool nonblock)
{
struct sti_private *private = drm->dev_private;
int err;
@@ -211,7 +211,7 @@
if (err)
return err;
- /* serialize outstanding asynchronous commits */
+ /* serialize outstanding nonblocking commits */
mutex_lock(&private->commit.lock);
flush_work(&private->commit.work);
@@ -223,7 +223,7 @@
drm_atomic_helper_swap_state(drm, state);
- if (async)
+ if (nonblock)
sti_atomic_schedule(private, state);
else
sti_atomic_complete(private, state);
diff --git a/drivers/gpu/drm/sti/sti_gdp.c b/drivers/gpu/drm/sti/sti_gdp.c
index ff3d3e7..ff33c38 100644
--- a/drivers/gpu/drm/sti/sti_gdp.c
+++ b/drivers/gpu/drm/sti/sti_gdp.c
@@ -5,6 +5,7 @@
* for STMicroelectronics.
* License terms: GNU General Public License (GPL), version 2
*/
+#include <linux/seq_file.h>
#include <drm/drm_atomic.h>
#include <drm/drm_fb_cma_helper.h>
diff --git a/drivers/gpu/drm/sti/sti_hda.c b/drivers/gpu/drm/sti/sti_hda.c
index ec0d017..f7d3464 100644
--- a/drivers/gpu/drm/sti/sti_hda.c
+++ b/drivers/gpu/drm/sti/sti_hda.c
@@ -8,6 +8,7 @@
#include <linux/component.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/seq_file.h>
#include <drm/drmP.h>
#include <drm/drm_atomic_helper.h>
diff --git a/drivers/gpu/drm/sti/sti_hqvdp.c b/drivers/gpu/drm/sti/sti_hqvdp.c
index e05b0dc..1edec29 100644
--- a/drivers/gpu/drm/sti/sti_hqvdp.c
+++ b/drivers/gpu/drm/sti/sti_hqvdp.c
@@ -7,6 +7,7 @@
#include <linux/component.h>
#include <linux/firmware.h>
#include <linux/reset.h>
+#include <linux/seq_file.h>
#include <drm/drm_atomic.h>
#include <drm/drm_fb_cma_helper.h>
diff --git a/drivers/gpu/drm/sti/sti_mixer.c b/drivers/gpu/drm/sti/sti_mixer.c
index e7425c3..aed7801 100644
--- a/drivers/gpu/drm/sti/sti_mixer.c
+++ b/drivers/gpu/drm/sti/sti_mixer.c
@@ -5,6 +5,7 @@
* for STMicroelectronics.
* License terms: GNU General Public License (GPL), version 2
*/
+#include <linux/seq_file.h>
#include "sti_compositor.h"
#include "sti_mixer.h"
diff --git a/drivers/gpu/drm/sti/sti_tvout.c b/drivers/gpu/drm/sti/sti_tvout.c
index 2c99016..f983db5 100644
--- a/drivers/gpu/drm/sti/sti_tvout.c
+++ b/drivers/gpu/drm/sti/sti_tvout.c
@@ -12,6 +12,7 @@
#include <linux/of_platform.h>
#include <linux/platform_device.h>
#include <linux/reset.h>
+#include <linux/seq_file.h>
#include <drm/drmP.h>
#include <drm/drm_crtc_helper.h>
diff --git a/drivers/gpu/drm/sti/sti_vid.c b/drivers/gpu/drm/sti/sti_vid.c
index 5a2c5dc..523ed19 100644
--- a/drivers/gpu/drm/sti/sti_vid.c
+++ b/drivers/gpu/drm/sti/sti_vid.c
@@ -3,6 +3,7 @@
* Author: Fabien Dessenne <fabien.dessenne@st.com> for STMicroelectronics.
* License terms: GNU General Public License (GPL), version 2
*/
+#include <linux/seq_file.h>
#include <drm/drmP.h>
diff --git a/drivers/gpu/drm/sun4i/Kconfig b/drivers/gpu/drm/sun4i/Kconfig
new file mode 100644
index 0000000..99510e6
--- /dev/null
+++ b/drivers/gpu/drm/sun4i/Kconfig
@@ -0,0 +1,14 @@
+config DRM_SUN4I
+ tristate "DRM Support for Allwinner A10 Display Engine"
+ depends on DRM && ARM
+ depends on ARCH_SUNXI || COMPILE_TEST
+ select DRM_GEM_CMA_HELPER
+ select DRM_KMS_HELPER
+ select DRM_KMS_CMA_HELPER
+ select DRM_PANEL
+ select REGMAP_MMIO
+ select VIDEOMODE_HELPERS
+ help
+ Choose this option if you have an Allwinner SoC with a
+ Display Engine. If M is selected the module will be called
+ sun4i-drm.
diff --git a/drivers/gpu/drm/sun4i/Makefile b/drivers/gpu/drm/sun4i/Makefile
new file mode 100644
index 0000000..58cd551
--- /dev/null
+++ b/drivers/gpu/drm/sun4i/Makefile
@@ -0,0 +1,13 @@
+sun4i-drm-y += sun4i_crtc.o
+sun4i-drm-y += sun4i_drv.o
+sun4i-drm-y += sun4i_framebuffer.o
+sun4i-drm-y += sun4i_layer.o
+
+sun4i-tcon-y += sun4i_tcon.o
+sun4i-tcon-y += sun4i_rgb.o
+sun4i-tcon-y += sun4i_dotclock.o
+
+obj-$(CONFIG_DRM_SUN4I) += sun4i-drm.o sun4i-tcon.o
+obj-$(CONFIG_DRM_SUN4I) += sun4i_backend.o
+
+obj-$(CONFIG_DRM_SUN4I) += sun4i_tv.o
diff --git a/drivers/gpu/drm/sun4i/sun4i_backend.c b/drivers/gpu/drm/sun4i/sun4i_backend.c
new file mode 100644
index 0000000..f7a15c1
--- /dev/null
+++ b/drivers/gpu/drm/sun4i/sun4i_backend.c
@@ -0,0 +1,364 @@
+/*
+ * Copyright (C) 2015 Free Electrons
+ * Copyright (C) 2015 NextThing Co
+ *
+ * Maxime Ripard <maxime.ripard@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ */
+
+#include <drm/drmP.h>
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_crtc.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_fb_cma_helper.h>
+#include <drm/drm_gem_cma_helper.h>
+#include <drm/drm_plane_helper.h>
+
+#include <linux/component.h>
+#include <linux/reset.h>
+
+#include "sun4i_backend.h"
+#include "sun4i_drv.h"
+
+static u32 sunxi_rgb2yuv_coef[12] = {
+ 0x00000107, 0x00000204, 0x00000064, 0x00000108,
+ 0x00003f69, 0x00003ed6, 0x000001c1, 0x00000808,
+ 0x000001c1, 0x00003e88, 0x00003fb8, 0x00000808
+};
+
+void sun4i_backend_apply_color_correction(struct sun4i_backend *backend)
+{
+ int i;
+
+ DRM_DEBUG_DRIVER("Applying RGB to YUV color correction\n");
+
+ /* Set color correction */
+ regmap_write(backend->regs, SUN4I_BACKEND_OCCTL_REG,
+ SUN4I_BACKEND_OCCTL_ENABLE);
+
+ for (i = 0; i < 12; i++)
+ regmap_write(backend->regs, SUN4I_BACKEND_OCRCOEF_REG(i),
+ sunxi_rgb2yuv_coef[i]);
+}
+EXPORT_SYMBOL(sun4i_backend_apply_color_correction);
+
+void sun4i_backend_disable_color_correction(struct sun4i_backend *backend)
+{
+ DRM_DEBUG_DRIVER("Disabling color correction\n");
+
+ /* Disable color correction */
+ regmap_update_bits(backend->regs, SUN4I_BACKEND_OCCTL_REG,
+ SUN4I_BACKEND_OCCTL_ENABLE, 0);
+}
+EXPORT_SYMBOL(sun4i_backend_disable_color_correction);
+
+void sun4i_backend_commit(struct sun4i_backend *backend)
+{
+ DRM_DEBUG_DRIVER("Committing changes\n");
+
+ regmap_write(backend->regs, SUN4I_BACKEND_REGBUFFCTL_REG,
+ SUN4I_BACKEND_REGBUFFCTL_AUTOLOAD_DIS |
+ SUN4I_BACKEND_REGBUFFCTL_LOADCTL);
+}
+EXPORT_SYMBOL(sun4i_backend_commit);
+
+void sun4i_backend_layer_enable(struct sun4i_backend *backend,
+ int layer, bool enable)
+{
+ u32 val;
+
+ DRM_DEBUG_DRIVER("Enabling layer %d\n", layer);
+
+ if (enable)
+ val = SUN4I_BACKEND_MODCTL_LAY_EN(layer);
+ else
+ val = 0;
+
+ regmap_update_bits(backend->regs, SUN4I_BACKEND_MODCTL_REG,
+ SUN4I_BACKEND_MODCTL_LAY_EN(layer), val);
+}
+EXPORT_SYMBOL(sun4i_backend_layer_enable);
+
+static int sun4i_backend_drm_format_to_layer(u32 format, u32 *mode)
+{
+ switch (format) {
+ case DRM_FORMAT_ARGB8888:
+ *mode = SUN4I_BACKEND_LAY_FBFMT_ARGB8888;
+ break;
+
+ case DRM_FORMAT_XRGB8888:
+ *mode = SUN4I_BACKEND_LAY_FBFMT_XRGB8888;
+ break;
+
+ case DRM_FORMAT_RGB888:
+ *mode = SUN4I_BACKEND_LAY_FBFMT_RGB888;
+ break;
+
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+int sun4i_backend_update_layer_coord(struct sun4i_backend *backend,
+ int layer, struct drm_plane *plane)
+{
+ struct drm_plane_state *state = plane->state;
+ struct drm_framebuffer *fb = state->fb;
+
+ DRM_DEBUG_DRIVER("Updating layer %d\n", layer);
+
+ if (plane->type == DRM_PLANE_TYPE_PRIMARY) {
+ DRM_DEBUG_DRIVER("Primary layer, updating global size W: %u H: %u\n",
+ state->crtc_w, state->crtc_h);
+ regmap_write(backend->regs, SUN4I_BACKEND_DISSIZE_REG,
+ SUN4I_BACKEND_DISSIZE(state->crtc_w,
+ state->crtc_h));
+ }
+
+ /* Set the line width */
+ DRM_DEBUG_DRIVER("Layer line width: %d bits\n", fb->pitches[0] * 8);
+ regmap_write(backend->regs, SUN4I_BACKEND_LAYLINEWIDTH_REG(layer),
+ fb->pitches[0] * 8);
+
+ /* Set height and width */
+ DRM_DEBUG_DRIVER("Layer size W: %u H: %u\n",
+ state->crtc_w, state->crtc_h);
+ regmap_write(backend->regs, SUN4I_BACKEND_LAYSIZE_REG(layer),
+ SUN4I_BACKEND_LAYSIZE(state->crtc_w,
+ state->crtc_h));
+
+ /* Set base coordinates */
+ DRM_DEBUG_DRIVER("Layer coordinates X: %d Y: %d\n",
+ state->crtc_x, state->crtc_y);
+ regmap_write(backend->regs, SUN4I_BACKEND_LAYCOOR_REG(layer),
+ SUN4I_BACKEND_LAYCOOR(state->crtc_x,
+ state->crtc_y));
+
+ return 0;
+}
+EXPORT_SYMBOL(sun4i_backend_update_layer_coord);
+
+int sun4i_backend_update_layer_formats(struct sun4i_backend *backend,
+ int layer, struct drm_plane *plane)
+{
+ struct drm_plane_state *state = plane->state;
+ struct drm_framebuffer *fb = state->fb;
+ bool interlaced = false;
+ u32 val;
+ int ret;
+
+ if (plane->state->crtc)
+ interlaced = plane->state->crtc->state->adjusted_mode.flags
+ & DRM_MODE_FLAG_INTERLACE;
+
+ regmap_update_bits(backend->regs, SUN4I_BACKEND_MODCTL_REG,
+ SUN4I_BACKEND_MODCTL_ITLMOD_EN,
+ interlaced ? SUN4I_BACKEND_MODCTL_ITLMOD_EN : 0);
+
+ DRM_DEBUG_DRIVER("Switching display backend interlaced mode %s\n",
+ interlaced ? "on" : "off");
+
+ ret = sun4i_backend_drm_format_to_layer(fb->pixel_format, &val);
+ if (ret) {
+ DRM_DEBUG_DRIVER("Invalid format\n");
+ return val;
+ }
+
+ regmap_update_bits(backend->regs, SUN4I_BACKEND_ATTCTL_REG1(layer),
+ SUN4I_BACKEND_ATTCTL_REG1_LAY_FBFMT, val);
+
+ return 0;
+}
+EXPORT_SYMBOL(sun4i_backend_update_layer_formats);
+
+int sun4i_backend_update_layer_buffer(struct sun4i_backend *backend,
+ int layer, struct drm_plane *plane)
+{
+ struct drm_plane_state *state = plane->state;
+ struct drm_framebuffer *fb = state->fb;
+ struct drm_gem_cma_object *gem;
+ u32 lo_paddr, hi_paddr;
+ dma_addr_t paddr;
+ int bpp;
+
+ /* Get the physical address of the buffer in memory */
+ gem = drm_fb_cma_get_gem_obj(fb, 0);
+
+ DRM_DEBUG_DRIVER("Using GEM @ 0x%x\n", gem->paddr);
+
+ /* Compute the start of the displayed memory */
+ bpp = drm_format_plane_cpp(fb->pixel_format, 0);
+ paddr = gem->paddr + fb->offsets[0];
+ paddr += (state->src_x >> 16) * bpp;
+ paddr += (state->src_y >> 16) * fb->pitches[0];
+
+ DRM_DEBUG_DRIVER("Setting buffer address to 0x%x\n", paddr);
+
+ /* Write the 32 lower bits of the address (in bits) */
+ lo_paddr = paddr << 3;
+ DRM_DEBUG_DRIVER("Setting address lower bits to 0x%x\n", lo_paddr);
+ regmap_write(backend->regs, SUN4I_BACKEND_LAYFB_L32ADD_REG(layer),
+ lo_paddr);
+
+ /* And the upper bits */
+ hi_paddr = paddr >> 29;
+ DRM_DEBUG_DRIVER("Setting address high bits to 0x%x\n", hi_paddr);
+ regmap_update_bits(backend->regs, SUN4I_BACKEND_LAYFB_H4ADD_REG,
+ SUN4I_BACKEND_LAYFB_H4ADD_MSK(layer),
+ SUN4I_BACKEND_LAYFB_H4ADD(layer, hi_paddr));
+
+ return 0;
+}
+EXPORT_SYMBOL(sun4i_backend_update_layer_buffer);
+
+static struct regmap_config sun4i_backend_regmap_config = {
+ .reg_bits = 32,
+ .val_bits = 32,
+ .reg_stride = 4,
+ .max_register = 0x5800,
+};
+
+static int sun4i_backend_bind(struct device *dev, struct device *master,
+ void *data)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct drm_device *drm = data;
+ struct sun4i_drv *drv = drm->dev_private;
+ struct sun4i_backend *backend;
+ struct resource *res;
+ void __iomem *regs;
+ int i, ret;
+
+ backend = devm_kzalloc(dev, sizeof(*backend), GFP_KERNEL);
+ if (!backend)
+ return -ENOMEM;
+ dev_set_drvdata(dev, backend);
+ drv->backend = backend;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ regs = devm_ioremap_resource(dev, res);
+ if (IS_ERR(regs)) {
+ dev_err(dev, "Couldn't map the backend registers\n");
+ return PTR_ERR(regs);
+ }
+
+ backend->regs = devm_regmap_init_mmio(dev, regs,
+ &sun4i_backend_regmap_config);
+ if (IS_ERR(backend->regs)) {
+ dev_err(dev, "Couldn't create the backend0 regmap\n");
+ return PTR_ERR(backend->regs);
+ }
+
+ backend->reset = devm_reset_control_get(dev, NULL);
+ if (IS_ERR(backend->reset)) {
+ dev_err(dev, "Couldn't get our reset line\n");
+ return PTR_ERR(backend->reset);
+ }
+
+ ret = reset_control_deassert(backend->reset);
+ if (ret) {
+ dev_err(dev, "Couldn't deassert our reset line\n");
+ return ret;
+ }
+
+ backend->bus_clk = devm_clk_get(dev, "ahb");
+ if (IS_ERR(backend->bus_clk)) {
+ dev_err(dev, "Couldn't get the backend bus clock\n");
+ ret = PTR_ERR(backend->bus_clk);
+ goto err_assert_reset;
+ }
+ clk_prepare_enable(backend->bus_clk);
+
+ backend->mod_clk = devm_clk_get(dev, "mod");
+ if (IS_ERR(backend->mod_clk)) {
+ dev_err(dev, "Couldn't get the backend module clock\n");
+ ret = PTR_ERR(backend->mod_clk);
+ goto err_disable_bus_clk;
+ }
+ clk_prepare_enable(backend->mod_clk);
+
+ backend->ram_clk = devm_clk_get(dev, "ram");
+ if (IS_ERR(backend->ram_clk)) {
+ dev_err(dev, "Couldn't get the backend RAM clock\n");
+ ret = PTR_ERR(backend->ram_clk);
+ goto err_disable_mod_clk;
+ }
+ clk_prepare_enable(backend->ram_clk);
+
+ /* Reset the registers */
+ for (i = 0x800; i < 0x1000; i += 4)
+ regmap_write(backend->regs, i, 0);
+
+ /* Disable registers autoloading */
+ regmap_write(backend->regs, SUN4I_BACKEND_REGBUFFCTL_REG,
+ SUN4I_BACKEND_REGBUFFCTL_AUTOLOAD_DIS);
+
+ /* Enable the backend */
+ regmap_write(backend->regs, SUN4I_BACKEND_MODCTL_REG,
+ SUN4I_BACKEND_MODCTL_DEBE_EN |
+ SUN4I_BACKEND_MODCTL_START_CTL);
+
+ return 0;
+
+err_disable_mod_clk:
+ clk_disable_unprepare(backend->mod_clk);
+err_disable_bus_clk:
+ clk_disable_unprepare(backend->bus_clk);
+err_assert_reset:
+ reset_control_assert(backend->reset);
+ return ret;
+}
+
+static void sun4i_backend_unbind(struct device *dev, struct device *master,
+ void *data)
+{
+ struct sun4i_backend *backend = dev_get_drvdata(dev);
+
+ clk_disable_unprepare(backend->ram_clk);
+ clk_disable_unprepare(backend->mod_clk);
+ clk_disable_unprepare(backend->bus_clk);
+ reset_control_assert(backend->reset);
+}
+
+static struct component_ops sun4i_backend_ops = {
+ .bind = sun4i_backend_bind,
+ .unbind = sun4i_backend_unbind,
+};
+
+static int sun4i_backend_probe(struct platform_device *pdev)
+{
+ return component_add(&pdev->dev, &sun4i_backend_ops);
+}
+
+static int sun4i_backend_remove(struct platform_device *pdev)
+{
+ component_del(&pdev->dev, &sun4i_backend_ops);
+
+ return 0;
+}
+
+static const struct of_device_id sun4i_backend_of_table[] = {
+ { .compatible = "allwinner,sun5i-a13-display-backend" },
+ { }
+};
+MODULE_DEVICE_TABLE(of, sun4i_backend_of_table);
+
+static struct platform_driver sun4i_backend_platform_driver = {
+ .probe = sun4i_backend_probe,
+ .remove = sun4i_backend_remove,
+ .driver = {
+ .name = "sun4i-backend",
+ .of_match_table = sun4i_backend_of_table,
+ },
+};
+module_platform_driver(sun4i_backend_platform_driver);
+
+MODULE_AUTHOR("Maxime Ripard <maxime.ripard@free-electrons.com>");
+MODULE_DESCRIPTION("Allwinner A10 Display Backend Driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/gpu/drm/sun4i/sun4i_backend.h b/drivers/gpu/drm/sun4i/sun4i_backend.h
new file mode 100644
index 0000000..7070bb3
--- /dev/null
+++ b/drivers/gpu/drm/sun4i/sun4i_backend.h
@@ -0,0 +1,165 @@
+/*
+ * Copyright (C) 2015 Free Electrons
+ * Copyright (C) 2015 NextThing Co
+ *
+ * Maxime Ripard <maxime.ripard@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ */
+
+#ifndef _SUN4I_BACKEND_H_
+#define _SUN4I_BACKEND_H_
+
+#include <linux/clk.h>
+#include <linux/regmap.h>
+#include <linux/reset.h>
+
+#define SUN4I_BACKEND_MODCTL_REG 0x800
+#define SUN4I_BACKEND_MODCTL_LINE_SEL BIT(29)
+#define SUN4I_BACKEND_MODCTL_ITLMOD_EN BIT(28)
+#define SUN4I_BACKEND_MODCTL_OUT_SEL GENMASK(22, 20)
+#define SUN4I_BACKEND_MODCTL_OUT_LCD (0 << 20)
+#define SUN4I_BACKEND_MODCTL_OUT_FE0 (6 << 20)
+#define SUN4I_BACKEND_MODCTL_OUT_FE1 (7 << 20)
+#define SUN4I_BACKEND_MODCTL_HWC_EN BIT(16)
+#define SUN4I_BACKEND_MODCTL_LAY_EN(l) BIT(8 + l)
+#define SUN4I_BACKEND_MODCTL_OCSC_EN BIT(5)
+#define SUN4I_BACKEND_MODCTL_DFLK_EN BIT(4)
+#define SUN4I_BACKEND_MODCTL_DLP_START_CTL BIT(2)
+#define SUN4I_BACKEND_MODCTL_START_CTL BIT(1)
+#define SUN4I_BACKEND_MODCTL_DEBE_EN BIT(0)
+
+#define SUN4I_BACKEND_BACKCOLOR_REG 0x804
+#define SUN4I_BACKEND_BACKCOLOR(r, g, b) (((r) << 16) | ((g) << 8) | (b))
+
+#define SUN4I_BACKEND_DISSIZE_REG 0x808
+#define SUN4I_BACKEND_DISSIZE(w, h) (((((h) - 1) & 0xffff) << 16) | \
+ (((w) - 1) & 0xffff))
+
+#define SUN4I_BACKEND_LAYSIZE_REG(l) (0x810 + (0x4 * (l)))
+#define SUN4I_BACKEND_LAYSIZE(w, h) (((((h) - 1) & 0x1fff) << 16) | \
+ (((w) - 1) & 0x1fff))
+
+#define SUN4I_BACKEND_LAYCOOR_REG(l) (0x820 + (0x4 * (l)))
+#define SUN4I_BACKEND_LAYCOOR(x, y) ((((u32)(y) & 0xffff) << 16) | \
+ ((u32)(x) & 0xffff))
+
+#define SUN4I_BACKEND_LAYLINEWIDTH_REG(l) (0x840 + (0x4 * (l)))
+
+#define SUN4I_BACKEND_LAYFB_L32ADD_REG(l) (0x850 + (0x4 * (l)))
+
+#define SUN4I_BACKEND_LAYFB_H4ADD_REG 0x860
+#define SUN4I_BACKEND_LAYFB_H4ADD_MSK(l) GENMASK(3 + ((l) * 8), 0)
+#define SUN4I_BACKEND_LAYFB_H4ADD(l, val) ((val) << ((l) * 8))
+
+#define SUN4I_BACKEND_REGBUFFCTL_REG 0x870
+#define SUN4I_BACKEND_REGBUFFCTL_AUTOLOAD_DIS BIT(1)
+#define SUN4I_BACKEND_REGBUFFCTL_LOADCTL BIT(0)
+
+#define SUN4I_BACKEND_CKMAX_REG 0x880
+#define SUN4I_BACKEND_CKMIN_REG 0x884
+#define SUN4I_BACKEND_CKCFG_REG 0x888
+#define SUN4I_BACKEND_ATTCTL_REG0(l) (0x890 + (0x4 * (l)))
+#define SUN4I_BACKEND_ATTCTL_REG0_LAY_PIPESEL_MASK BIT(15)
+#define SUN4I_BACKEND_ATTCTL_REG0_LAY_PIPESEL(x) ((x) << 15)
+#define SUN4I_BACKEND_ATTCTL_REG0_LAY_PRISEL_MASK GENMASK(11, 10)
+#define SUN4I_BACKEND_ATTCTL_REG0_LAY_PRISEL(x) ((x) << 10)
+
+#define SUN4I_BACKEND_ATTCTL_REG1(l) (0x8a0 + (0x4 * (l)))
+#define SUN4I_BACKEND_ATTCTL_REG1_LAY_HSCAFCT GENMASK(15, 14)
+#define SUN4I_BACKEND_ATTCTL_REG1_LAY_WSCAFCT GENMASK(13, 12)
+#define SUN4I_BACKEND_ATTCTL_REG1_LAY_FBFMT GENMASK(11, 8)
+#define SUN4I_BACKEND_LAY_FBFMT_1BPP (0 << 8)
+#define SUN4I_BACKEND_LAY_FBFMT_2BPP (1 << 8)
+#define SUN4I_BACKEND_LAY_FBFMT_4BPP (2 << 8)
+#define SUN4I_BACKEND_LAY_FBFMT_8BPP (3 << 8)
+#define SUN4I_BACKEND_LAY_FBFMT_RGB655 (4 << 8)
+#define SUN4I_BACKEND_LAY_FBFMT_RGB565 (5 << 8)
+#define SUN4I_BACKEND_LAY_FBFMT_RGB556 (6 << 8)
+#define SUN4I_BACKEND_LAY_FBFMT_ARGB1555 (7 << 8)
+#define SUN4I_BACKEND_LAY_FBFMT_RGBA5551 (8 << 8)
+#define SUN4I_BACKEND_LAY_FBFMT_XRGB8888 (9 << 8)
+#define SUN4I_BACKEND_LAY_FBFMT_ARGB8888 (10 << 8)
+#define SUN4I_BACKEND_LAY_FBFMT_RGB888 (11 << 8)
+#define SUN4I_BACKEND_LAY_FBFMT_ARGB4444 (12 << 8)
+#define SUN4I_BACKEND_LAY_FBFMT_RGBA4444 (13 << 8)
+
+#define SUN4I_BACKEND_DLCDPCTL_REG 0x8b0
+#define SUN4I_BACKEND_DLCDPFRMBUF_ADDRCTL_REG 0x8b4
+#define SUN4I_BACKEND_DLCDPCOOR_REG0 0x8b8
+#define SUN4I_BACKEND_DLCDPCOOR_REG1 0x8bc
+
+#define SUN4I_BACKEND_INT_EN_REG 0x8c0
+#define SUN4I_BACKEND_INT_FLAG_REG 0x8c4
+#define SUN4I_BACKEND_REG_LOAD_FINISHED BIT(1)
+
+#define SUN4I_BACKEND_HWCCTL_REG 0x8d8
+#define SUN4I_BACKEND_HWCFBCTL_REG 0x8e0
+#define SUN4I_BACKEND_WBCTL_REG 0x8f0
+#define SUN4I_BACKEND_WBADD_REG 0x8f4
+#define SUN4I_BACKEND_WBLINEWIDTH_REG 0x8f8
+#define SUN4I_BACKEND_SPREN_REG 0x900
+#define SUN4I_BACKEND_SPRFMTCTL_REG 0x908
+#define SUN4I_BACKEND_SPRALPHACTL_REG 0x90c
+#define SUN4I_BACKEND_IYUVCTL_REG 0x920
+#define SUN4I_BACKEND_IYUVADD_REG(c) (0x930 + (0x4 * (c)))
+#define SUN4I_BACKEND_IYUVLINEWITDTH_REG(c) (0x940 + (0x4 * (c)))
+#define SUN4I_BACKEND_YGCOEF_REG(c) (0x950 + (0x4 * (c)))
+#define SUN4I_BACKEND_YGCONS_REG 0x95c
+#define SUN4I_BACKEND_URCOEF_REG(c) (0x960 + (0x4 * (c)))
+#define SUN4I_BACKEND_URCONS_REG 0x96c
+#define SUN4I_BACKEND_VBCOEF_REG(c) (0x970 + (0x4 * (c)))
+#define SUN4I_BACKEND_VBCONS_REG 0x97c
+#define SUN4I_BACKEND_KSCTL_REG 0x980
+#define SUN4I_BACKEND_KSBKCOLOR_REG 0x984
+#define SUN4I_BACKEND_KSFSTLINEWIDTH_REG 0x988
+#define SUN4I_BACKEND_KSVSCAFCT_REG 0x98c
+#define SUN4I_BACKEND_KSHSCACOEF_REG(x) (0x9a0 + (0x4 * (x)))
+#define SUN4I_BACKEND_OCCTL_REG 0x9c0
+#define SUN4I_BACKEND_OCCTL_ENABLE BIT(0)
+
+#define SUN4I_BACKEND_OCRCOEF_REG(x) (0x9d0 + (0x4 * (x)))
+#define SUN4I_BACKEND_OCRCONS_REG 0x9dc
+#define SUN4I_BACKEND_OCGCOEF_REG(x) (0x9e0 + (0x4 * (x)))
+#define SUN4I_BACKEND_OCGCONS_REG 0x9ec
+#define SUN4I_BACKEND_OCBCOEF_REG(x) (0x9f0 + (0x4 * (x)))
+#define SUN4I_BACKEND_OCBCONS_REG 0x9fc
+#define SUN4I_BACKEND_SPRCOORCTL_REG(s) (0xa00 + (0x4 * (s)))
+#define SUN4I_BACKEND_SPRATTCTL_REG(s) (0xb00 + (0x4 * (s)))
+#define SUN4I_BACKEND_SPRADD_REG(s) (0xc00 + (0x4 * (s)))
+#define SUN4I_BACKEND_SPRLINEWIDTH_REG(s) (0xd00 + (0x4 * (s)))
+
+#define SUN4I_BACKEND_SPRPALTAB_OFF 0x4000
+#define SUN4I_BACKEND_GAMMATAB_OFF 0x4400
+#define SUN4I_BACKEND_HWCPATTERN_OFF 0x4800
+#define SUN4I_BACKEND_HWCCOLORTAB_OFF 0x4c00
+#define SUN4I_BACKEND_PIPE_OFF(p) (0x5000 + (0x400 * (p)))
+
+struct sun4i_backend {
+ struct regmap *regs;
+
+ struct reset_control *reset;
+
+ struct clk *bus_clk;
+ struct clk *mod_clk;
+ struct clk *ram_clk;
+};
+
+void sun4i_backend_apply_color_correction(struct sun4i_backend *backend);
+void sun4i_backend_disable_color_correction(struct sun4i_backend *backend);
+
+void sun4i_backend_commit(struct sun4i_backend *backend);
+
+void sun4i_backend_layer_enable(struct sun4i_backend *backend,
+ int layer, bool enable);
+int sun4i_backend_update_layer_coord(struct sun4i_backend *backend,
+ int layer, struct drm_plane *plane);
+int sun4i_backend_update_layer_formats(struct sun4i_backend *backend,
+ int layer, struct drm_plane *plane);
+int sun4i_backend_update_layer_buffer(struct sun4i_backend *backend,
+ int layer, struct drm_plane *plane);
+
+#endif /* _SUN4I_BACKEND_H_ */
diff --git a/drivers/gpu/drm/sun4i/sun4i_crtc.c b/drivers/gpu/drm/sun4i/sun4i_crtc.c
new file mode 100644
index 0000000..4182a21
--- /dev/null
+++ b/drivers/gpu/drm/sun4i/sun4i_crtc.c
@@ -0,0 +1,120 @@
+/*
+ * Copyright (C) 2015 Free Electrons
+ * Copyright (C) 2015 NextThing Co
+ *
+ * Maxime Ripard <maxime.ripard@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ */
+
+#include <drm/drmP.h>
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_crtc.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_modes.h>
+
+#include <linux/clk-provider.h>
+#include <linux/ioport.h>
+#include <linux/of_address.h>
+#include <linux/of_irq.h>
+#include <linux/regmap.h>
+
+#include <video/videomode.h>
+
+#include "sun4i_backend.h"
+#include "sun4i_crtc.h"
+#include "sun4i_drv.h"
+#include "sun4i_tcon.h"
+
+static void sun4i_crtc_atomic_begin(struct drm_crtc *crtc,
+ struct drm_crtc_state *old_state)
+{
+ struct sun4i_crtc *scrtc = drm_crtc_to_sun4i_crtc(crtc);
+ struct drm_device *dev = crtc->dev;
+ unsigned long flags;
+
+ if (crtc->state->event) {
+ WARN_ON(drm_crtc_vblank_get(crtc) != 0);
+
+ spin_lock_irqsave(&dev->event_lock, flags);
+ scrtc->event = crtc->state->event;
+ spin_unlock_irqrestore(&dev->event_lock, flags);
+ crtc->state->event = NULL;
+ }
+}
+
+static void sun4i_crtc_atomic_flush(struct drm_crtc *crtc,
+ struct drm_crtc_state *old_state)
+{
+ struct sun4i_crtc *scrtc = drm_crtc_to_sun4i_crtc(crtc);
+ struct sun4i_drv *drv = scrtc->drv;
+
+ DRM_DEBUG_DRIVER("Committing plane changes\n");
+
+ sun4i_backend_commit(drv->backend);
+}
+
+static void sun4i_crtc_disable(struct drm_crtc *crtc)
+{
+ struct sun4i_crtc *scrtc = drm_crtc_to_sun4i_crtc(crtc);
+ struct sun4i_drv *drv = scrtc->drv;
+
+ DRM_DEBUG_DRIVER("Disabling the CRTC\n");
+
+ sun4i_tcon_disable(drv->tcon);
+}
+
+static void sun4i_crtc_enable(struct drm_crtc *crtc)
+{
+ struct sun4i_crtc *scrtc = drm_crtc_to_sun4i_crtc(crtc);
+ struct sun4i_drv *drv = scrtc->drv;
+
+ DRM_DEBUG_DRIVER("Enabling the CRTC\n");
+
+ sun4i_tcon_enable(drv->tcon);
+}
+
+static const struct drm_crtc_helper_funcs sun4i_crtc_helper_funcs = {
+ .atomic_begin = sun4i_crtc_atomic_begin,
+ .atomic_flush = sun4i_crtc_atomic_flush,
+ .disable = sun4i_crtc_disable,
+ .enable = sun4i_crtc_enable,
+};
+
+static const struct drm_crtc_funcs sun4i_crtc_funcs = {
+ .atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
+ .atomic_duplicate_state = drm_atomic_helper_crtc_duplicate_state,
+ .destroy = drm_crtc_cleanup,
+ .page_flip = drm_atomic_helper_page_flip,
+ .reset = drm_atomic_helper_crtc_reset,
+ .set_config = drm_atomic_helper_set_config,
+};
+
+struct sun4i_crtc *sun4i_crtc_init(struct drm_device *drm)
+{
+ struct sun4i_drv *drv = drm->dev_private;
+ struct sun4i_crtc *scrtc;
+ int ret;
+
+ scrtc = devm_kzalloc(drm->dev, sizeof(*scrtc), GFP_KERNEL);
+ if (!scrtc)
+ return NULL;
+ scrtc->drv = drv;
+
+ ret = drm_crtc_init_with_planes(drm, &scrtc->crtc,
+ drv->primary,
+ NULL,
+ &sun4i_crtc_funcs,
+ NULL);
+ if (ret) {
+ dev_err(drm->dev, "Couldn't init DRM CRTC\n");
+ return NULL;
+ }
+
+ drm_crtc_helper_add(&scrtc->crtc, &sun4i_crtc_helper_funcs);
+
+ return scrtc;
+}
diff --git a/drivers/gpu/drm/sun4i/sun4i_crtc.h b/drivers/gpu/drm/sun4i/sun4i_crtc.h
new file mode 100644
index 0000000..dec8ce4
--- /dev/null
+++ b/drivers/gpu/drm/sun4i/sun4i_crtc.h
@@ -0,0 +1,30 @@
+/*
+ * Copyright (C) 2015 Free Electrons
+ * Copyright (C) 2015 NextThing Co
+ *
+ * Maxime Ripard <maxime.ripard@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ */
+
+#ifndef _SUN4I_CRTC_H_
+#define _SUN4I_CRTC_H_
+
+struct sun4i_crtc {
+ struct drm_crtc crtc;
+ struct drm_pending_vblank_event *event;
+
+ struct sun4i_drv *drv;
+};
+
+static inline struct sun4i_crtc *drm_crtc_to_sun4i_crtc(struct drm_crtc *crtc)
+{
+ return container_of(crtc, struct sun4i_crtc, crtc);
+}
+
+struct sun4i_crtc *sun4i_crtc_init(struct drm_device *drm);
+
+#endif /* _SUN4I_CRTC_H_ */
diff --git a/drivers/gpu/drm/sun4i/sun4i_dotclock.c b/drivers/gpu/drm/sun4i/sun4i_dotclock.c
new file mode 100644
index 0000000..3ff668c
--- /dev/null
+++ b/drivers/gpu/drm/sun4i/sun4i_dotclock.c
@@ -0,0 +1,160 @@
+/*
+ * Copyright (C) 2016 Free Electrons
+ * Copyright (C) 2016 NextThing Co
+ *
+ * Maxime Ripard <maxime.ripard@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ */
+
+#include <linux/clk-provider.h>
+#include <linux/regmap.h>
+
+#include "sun4i_tcon.h"
+
+struct sun4i_dclk {
+ struct clk_hw hw;
+ struct regmap *regmap;
+};
+
+static inline struct sun4i_dclk *hw_to_dclk(struct clk_hw *hw)
+{
+ return container_of(hw, struct sun4i_dclk, hw);
+}
+
+static void sun4i_dclk_disable(struct clk_hw *hw)
+{
+ struct sun4i_dclk *dclk = hw_to_dclk(hw);
+
+ regmap_update_bits(dclk->regmap, SUN4I_TCON0_DCLK_REG,
+ BIT(SUN4I_TCON0_DCLK_GATE_BIT), 0);
+}
+
+static int sun4i_dclk_enable(struct clk_hw *hw)
+{
+ struct sun4i_dclk *dclk = hw_to_dclk(hw);
+
+ return regmap_update_bits(dclk->regmap, SUN4I_TCON0_DCLK_REG,
+ BIT(SUN4I_TCON0_DCLK_GATE_BIT),
+ BIT(SUN4I_TCON0_DCLK_GATE_BIT));
+}
+
+static int sun4i_dclk_is_enabled(struct clk_hw *hw)
+{
+ struct sun4i_dclk *dclk = hw_to_dclk(hw);
+ u32 val;
+
+ regmap_read(dclk->regmap, SUN4I_TCON0_DCLK_REG, &val);
+
+ return val & BIT(SUN4I_TCON0_DCLK_GATE_BIT);
+}
+
+static unsigned long sun4i_dclk_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct sun4i_dclk *dclk = hw_to_dclk(hw);
+ u32 val;
+
+ regmap_read(dclk->regmap, SUN4I_TCON0_DCLK_REG, &val);
+
+ val >>= SUN4I_TCON0_DCLK_DIV_SHIFT;
+ val &= SUN4I_TCON0_DCLK_DIV_WIDTH;
+
+ if (!val)
+ val = 1;
+
+ return parent_rate / val;
+}
+
+static long sun4i_dclk_round_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long *parent_rate)
+{
+ return *parent_rate / DIV_ROUND_CLOSEST(*parent_rate, rate);
+}
+
+static int sun4i_dclk_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct sun4i_dclk *dclk = hw_to_dclk(hw);
+ int div = DIV_ROUND_CLOSEST(parent_rate, rate);
+
+ return regmap_update_bits(dclk->regmap, SUN4I_TCON0_DCLK_REG,
+ GENMASK(6, 0), div);
+}
+
+static int sun4i_dclk_get_phase(struct clk_hw *hw)
+{
+ struct sun4i_dclk *dclk = hw_to_dclk(hw);
+ u32 val;
+
+ regmap_read(dclk->regmap, SUN4I_TCON0_IO_POL_REG, &val);
+
+ val >>= 28;
+ val &= 3;
+
+ return val * 120;
+}
+
+static int sun4i_dclk_set_phase(struct clk_hw *hw, int degrees)
+{
+ struct sun4i_dclk *dclk = hw_to_dclk(hw);
+
+ regmap_update_bits(dclk->regmap, SUN4I_TCON0_IO_POL_REG,
+ GENMASK(29, 28),
+ degrees / 120);
+
+ return 0;
+}
+
+static const struct clk_ops sun4i_dclk_ops = {
+ .disable = sun4i_dclk_disable,
+ .enable = sun4i_dclk_enable,
+ .is_enabled = sun4i_dclk_is_enabled,
+
+ .recalc_rate = sun4i_dclk_recalc_rate,
+ .round_rate = sun4i_dclk_round_rate,
+ .set_rate = sun4i_dclk_set_rate,
+
+ .get_phase = sun4i_dclk_get_phase,
+ .set_phase = sun4i_dclk_set_phase,
+};
+
+int sun4i_dclk_create(struct device *dev, struct sun4i_tcon *tcon)
+{
+ const char *clk_name, *parent_name;
+ struct clk_init_data init;
+ struct sun4i_dclk *dclk;
+
+ parent_name = __clk_get_name(tcon->sclk0);
+ of_property_read_string_index(dev->of_node, "clock-output-names", 0,
+ &clk_name);
+
+ dclk = devm_kzalloc(dev, sizeof(*dclk), GFP_KERNEL);
+ if (!dclk)
+ return -ENOMEM;
+
+ init.name = clk_name;
+ init.ops = &sun4i_dclk_ops;
+ init.parent_names = &parent_name;
+ init.num_parents = 1;
+
+ dclk->regmap = tcon->regs;
+ dclk->hw.init = &init;
+
+ tcon->dclk = clk_register(dev, &dclk->hw);
+ if (IS_ERR(tcon->dclk))
+ return PTR_ERR(tcon->dclk);
+
+ return 0;
+}
+EXPORT_SYMBOL(sun4i_dclk_create);
+
+int sun4i_dclk_free(struct sun4i_tcon *tcon)
+{
+ clk_unregister(tcon->dclk);
+ return 0;
+}
+EXPORT_SYMBOL(sun4i_dclk_free);
diff --git a/drivers/gpu/drm/sun4i/sun4i_dotclock.h b/drivers/gpu/drm/sun4i/sun4i_dotclock.h
new file mode 100644
index 0000000..d5e25fa
--- /dev/null
+++ b/drivers/gpu/drm/sun4i/sun4i_dotclock.h
@@ -0,0 +1,21 @@
+/*
+ * Copyright (C) 2015 Free Electrons
+ * Copyright (C) 2015 NextThing Co
+ *
+ * Maxime Ripard <maxime.ripard@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ */
+
+#ifndef _SUN4I_DOTCLOCK_H_
+#define _SUN4I_DOTCLOCK_H_
+
+struct sun4i_tcon;
+
+int sun4i_dclk_create(struct device *dev, struct sun4i_tcon *tcon);
+int sun4i_dclk_free(struct sun4i_tcon *tcon);
+
+#endif /* _SUN4I_DOTCLOCK_H_ */
diff --git a/drivers/gpu/drm/sun4i/sun4i_drv.c b/drivers/gpu/drm/sun4i/sun4i_drv.c
new file mode 100644
index 0000000..76e922b
--- /dev/null
+++ b/drivers/gpu/drm/sun4i/sun4i_drv.c
@@ -0,0 +1,358 @@
+/*
+ * Copyright (C) 2015 Free Electrons
+ * Copyright (C) 2015 NextThing Co
+ *
+ * Maxime Ripard <maxime.ripard@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ */
+
+#include <linux/component.h>
+#include <linux/of_graph.h>
+
+#include <drm/drmP.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_fb_cma_helper.h>
+#include <drm/drm_gem_cma_helper.h>
+
+#include "sun4i_crtc.h"
+#include "sun4i_drv.h"
+#include "sun4i_framebuffer.h"
+#include "sun4i_layer.h"
+#include "sun4i_tcon.h"
+
+static int sun4i_drv_connector_plug_all(struct drm_device *drm)
+{
+ struct drm_connector *connector, *failed;
+ int ret;
+
+ mutex_lock(&drm->mode_config.mutex);
+ list_for_each_entry(connector, &drm->mode_config.connector_list, head) {
+ ret = drm_connector_register(connector);
+ if (ret) {
+ failed = connector;
+ goto err;
+ }
+ }
+ mutex_unlock(&drm->mode_config.mutex);
+ return 0;
+
+err:
+ list_for_each_entry(connector, &drm->mode_config.connector_list, head) {
+ if (failed == connector)
+ break;
+
+ drm_connector_unregister(connector);
+ }
+ mutex_unlock(&drm->mode_config.mutex);
+
+ return ret;
+}
+
+static int sun4i_drv_enable_vblank(struct drm_device *drm, unsigned int pipe)
+{
+ struct sun4i_drv *drv = drm->dev_private;
+ struct sun4i_tcon *tcon = drv->tcon;
+
+ DRM_DEBUG_DRIVER("Enabling VBLANK on pipe %d\n", pipe);
+
+ sun4i_tcon_enable_vblank(tcon, true);
+
+ return 0;
+}
+
+static void sun4i_drv_disable_vblank(struct drm_device *drm, unsigned int pipe)
+{
+ struct sun4i_drv *drv = drm->dev_private;
+ struct sun4i_tcon *tcon = drv->tcon;
+
+ DRM_DEBUG_DRIVER("Disabling VBLANK on pipe %d\n", pipe);
+
+ sun4i_tcon_enable_vblank(tcon, false);
+}
+
+static const struct file_operations sun4i_drv_fops = {
+ .owner = THIS_MODULE,
+ .open = drm_open,
+ .release = drm_release,
+ .unlocked_ioctl = drm_ioctl,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = drm_compat_ioctl,
+#endif
+ .poll = drm_poll,
+ .read = drm_read,
+ .llseek = no_llseek,
+ .mmap = drm_gem_cma_mmap,
+};
+
+static struct drm_driver sun4i_drv_driver = {
+ .driver_features = DRIVER_GEM | DRIVER_MODESET | DRIVER_PRIME | DRIVER_ATOMIC,
+
+ /* Generic Operations */
+ .fops = &sun4i_drv_fops,
+ .name = "sun4i-drm",
+ .desc = "Allwinner sun4i Display Engine",
+ .date = "20150629",
+ .major = 1,
+ .minor = 0,
+
+ /* GEM Operations */
+ .dumb_create = drm_gem_cma_dumb_create,
+ .dumb_destroy = drm_gem_dumb_destroy,
+ .dumb_map_offset = drm_gem_cma_dumb_map_offset,
+ .gem_free_object = drm_gem_cma_free_object,
+ .gem_vm_ops = &drm_gem_cma_vm_ops,
+
+ /* PRIME Operations */
+ .prime_handle_to_fd = drm_gem_prime_handle_to_fd,
+ .prime_fd_to_handle = drm_gem_prime_fd_to_handle,
+ .gem_prime_import = drm_gem_prime_import,
+ .gem_prime_export = drm_gem_prime_export,
+ .gem_prime_get_sg_table = drm_gem_cma_prime_get_sg_table,
+ .gem_prime_import_sg_table = drm_gem_cma_prime_import_sg_table,
+ .gem_prime_vmap = drm_gem_cma_prime_vmap,
+ .gem_prime_vunmap = drm_gem_cma_prime_vunmap,
+ .gem_prime_mmap = drm_gem_cma_prime_mmap,
+
+ /* Frame Buffer Operations */
+
+ /* VBlank Operations */
+ .get_vblank_counter = drm_vblank_count,
+ .enable_vblank = sun4i_drv_enable_vblank,
+ .disable_vblank = sun4i_drv_disable_vblank,
+};
+
+static int sun4i_drv_bind(struct device *dev)
+{
+ struct drm_device *drm;
+ struct sun4i_drv *drv;
+ int ret;
+
+ drm = drm_dev_alloc(&sun4i_drv_driver, dev);
+ if (!drm)
+ return -ENOMEM;
+
+ ret = drm_dev_set_unique(drm, dev_name(drm->dev));
+ if (ret)
+ goto free_drm;
+
+ drv = devm_kzalloc(dev, sizeof(*drv), GFP_KERNEL);
+ if (!drv) {
+ ret = -ENOMEM;
+ goto free_drm;
+ }
+ drm->dev_private = drv;
+
+ drm_vblank_init(drm, 1);
+ drm_mode_config_init(drm);
+
+ ret = component_bind_all(drm->dev, drm);
+ if (ret) {
+ dev_err(drm->dev, "Couldn't bind all pipelines components\n");
+ goto free_drm;
+ }
+
+ /* Create our layers */
+ drv->layers = sun4i_layers_init(drm);
+ if (!drv->layers) {
+ dev_err(drm->dev, "Couldn't create the planes\n");
+ ret = -EINVAL;
+ goto free_drm;
+ }
+
+ /* Create our CRTC */
+ drv->crtc = sun4i_crtc_init(drm);
+ if (!drv->crtc) {
+ dev_err(drm->dev, "Couldn't create the CRTC\n");
+ ret = -EINVAL;
+ goto free_drm;
+ }
+ drm->irq_enabled = true;
+
+ /* Create our framebuffer */
+ drv->fbdev = sun4i_framebuffer_init(drm);
+ if (IS_ERR(drv->fbdev)) {
+ dev_err(drm->dev, "Couldn't create our framebuffer\n");
+ ret = PTR_ERR(drv->fbdev);
+ goto free_drm;
+ }
+
+ /* Enable connectors polling */
+ drm_kms_helper_poll_init(drm);
+
+ ret = drm_dev_register(drm, 0);
+ if (ret)
+ goto free_drm;
+
+ ret = sun4i_drv_connector_plug_all(drm);
+ if (ret)
+ goto unregister_drm;
+
+ return 0;
+
+unregister_drm:
+ drm_dev_unregister(drm);
+free_drm:
+ drm_dev_unref(drm);
+ return ret;
+}
+
+static void sun4i_drv_unbind(struct device *dev)
+{
+ struct drm_device *drm = dev_get_drvdata(dev);
+
+ drm_dev_unregister(drm);
+ drm_kms_helper_poll_fini(drm);
+ sun4i_framebuffer_free(drm);
+ drm_vblank_cleanup(drm);
+ drm_dev_unref(drm);
+}
+
+static const struct component_master_ops sun4i_drv_master_ops = {
+ .bind = sun4i_drv_bind,
+ .unbind = sun4i_drv_unbind,
+};
+
+static bool sun4i_drv_node_is_frontend(struct device_node *node)
+{
+ return of_device_is_compatible(node,
+ "allwinner,sun5i-a13-display-frontend");
+}
+
+static bool sun4i_drv_node_is_tcon(struct device_node *node)
+{
+ return of_device_is_compatible(node, "allwinner,sun5i-a13-tcon");
+}
+
+static int compare_of(struct device *dev, void *data)
+{
+ DRM_DEBUG_DRIVER("Comparing of node %s with %s\n",
+ of_node_full_name(dev->of_node),
+ of_node_full_name(data));
+
+ return dev->of_node == data;
+}
+
+static int sun4i_drv_add_endpoints(struct device *dev,
+ struct component_match **match,
+ struct device_node *node)
+{
+ struct device_node *port, *ep, *remote;
+ int count = 0;
+
+ /*
+ * We don't support the frontend for now, so we will never
+ * have a device bound. Just skip over it, but we still want
+ * the rest our pipeline to be added.
+ */
+ if (!sun4i_drv_node_is_frontend(node) &&
+ !of_device_is_available(node))
+ return 0;
+
+ if (!sun4i_drv_node_is_frontend(node)) {
+ /* Add current component */
+ DRM_DEBUG_DRIVER("Adding component %s\n",
+ of_node_full_name(node));
+ component_match_add(dev, match, compare_of, node);
+ count++;
+ }
+
+ /* Inputs are listed first, then outputs */
+ port = of_graph_get_port_by_id(node, 1);
+ if (!port) {
+ DRM_DEBUG_DRIVER("No output to bind\n");
+ return count;
+ }
+
+ for_each_available_child_of_node(port, ep) {
+ remote = of_graph_get_remote_port_parent(ep);
+ if (!remote) {
+ DRM_DEBUG_DRIVER("Error retrieving the output node\n");
+ of_node_put(remote);
+ continue;
+ }
+
+ /*
+ * If the node is our TCON, the first port is used for our
+ * panel, and will not be part of the
+ * component framework.
+ */
+ if (sun4i_drv_node_is_tcon(node)) {
+ struct of_endpoint endpoint;
+
+ if (of_graph_parse_endpoint(ep, &endpoint)) {
+ DRM_DEBUG_DRIVER("Couldn't parse endpoint\n");
+ continue;
+ }
+
+ if (!endpoint.id) {
+ DRM_DEBUG_DRIVER("Endpoint is our panel... skipping\n");
+ continue;
+ }
+ }
+
+ /* Walk down our tree */
+ count += sun4i_drv_add_endpoints(dev, match, remote);
+
+ of_node_put(remote);
+ }
+
+ return count;
+}
+
+static int sun4i_drv_probe(struct platform_device *pdev)
+{
+ struct component_match *match = NULL;
+ struct device_node *np = pdev->dev.of_node;
+ int i, count = 0;
+
+ for (i = 0;; i++) {
+ struct device_node *pipeline = of_parse_phandle(np,
+ "allwinner,pipelines",
+ i);
+ if (!pipeline)
+ break;
+
+ count += sun4i_drv_add_endpoints(&pdev->dev, &match,
+ pipeline);
+
+ DRM_DEBUG_DRIVER("Queued %d outputs on pipeline %d\n",
+ count, i);
+ }
+
+ if (count)
+ return component_master_add_with_match(&pdev->dev,
+ &sun4i_drv_master_ops,
+ match);
+ else
+ return 0;
+}
+
+static int sun4i_drv_remove(struct platform_device *pdev)
+{
+ return 0;
+}
+
+static const struct of_device_id sun4i_drv_of_table[] = {
+ { .compatible = "allwinner,sun5i-a13-display-engine" },
+ { }
+};
+MODULE_DEVICE_TABLE(of, sun4i_drv_of_table);
+
+static struct platform_driver sun4i_drv_platform_driver = {
+ .probe = sun4i_drv_probe,
+ .remove = sun4i_drv_remove,
+ .driver = {
+ .name = "sun4i-drm",
+ .of_match_table = sun4i_drv_of_table,
+ },
+};
+module_platform_driver(sun4i_drv_platform_driver);
+
+MODULE_AUTHOR("Boris Brezillon <boris.brezillon@free-electrons.com>");
+MODULE_AUTHOR("Maxime Ripard <maxime.ripard@free-electrons.com>");
+MODULE_DESCRIPTION("Allwinner A10 Display Engine DRM/KMS Driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/gpu/drm/sun4i/sun4i_drv.h b/drivers/gpu/drm/sun4i/sun4i_drv.h
new file mode 100644
index 0000000..597353e
--- /dev/null
+++ b/drivers/gpu/drm/sun4i/sun4i_drv.h
@@ -0,0 +1,30 @@
+/*
+ * Copyright (C) 2015 Free Electrons
+ * Copyright (C) 2015 NextThing Co
+ *
+ * Maxime Ripard <maxime.ripard@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ */
+
+#ifndef _SUN4I_DRV_H_
+#define _SUN4I_DRV_H_
+
+#include <linux/clk.h>
+#include <linux/regmap.h>
+
+struct sun4i_drv {
+ struct sun4i_backend *backend;
+ struct sun4i_crtc *crtc;
+ struct sun4i_tcon *tcon;
+
+ struct drm_plane *primary;
+ struct drm_fbdev_cma *fbdev;
+
+ struct sun4i_layer **layers;
+};
+
+#endif /* _SUN4I_DRV_H_ */
diff --git a/drivers/gpu/drm/sun4i/sun4i_framebuffer.c b/drivers/gpu/drm/sun4i/sun4i_framebuffer.c
new file mode 100644
index 0000000..a0b30c2
--- /dev/null
+++ b/drivers/gpu/drm/sun4i/sun4i_framebuffer.c
@@ -0,0 +1,54 @@
+/*
+ * Copyright (C) 2015 Free Electrons
+ * Copyright (C) 2015 NextThing Co
+ *
+ * Maxime Ripard <maxime.ripard@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ */
+
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_fb_cma_helper.h>
+#include <drm/drmP.h>
+
+#include "sun4i_drv.h"
+
+static void sun4i_de_output_poll_changed(struct drm_device *drm)
+{
+ struct sun4i_drv *drv = drm->dev_private;
+
+ if (drv->fbdev)
+ drm_fbdev_cma_hotplug_event(drv->fbdev);
+}
+
+static const struct drm_mode_config_funcs sun4i_de_mode_config_funcs = {
+ .output_poll_changed = sun4i_de_output_poll_changed,
+ .atomic_check = drm_atomic_helper_check,
+ .atomic_commit = drm_atomic_helper_commit,
+ .fb_create = drm_fb_cma_create,
+};
+
+struct drm_fbdev_cma *sun4i_framebuffer_init(struct drm_device *drm)
+{
+ drm_mode_config_reset(drm);
+
+ drm->mode_config.max_width = 8192;
+ drm->mode_config.max_height = 8192;
+
+ drm->mode_config.funcs = &sun4i_de_mode_config_funcs;
+
+ return drm_fbdev_cma_init(drm, 32,
+ drm->mode_config.num_crtc,
+ drm->mode_config.num_connector);
+}
+
+void sun4i_framebuffer_free(struct drm_device *drm)
+{
+ struct sun4i_drv *drv = drm->dev_private;
+
+ drm_fbdev_cma_fini(drv->fbdev);
+ drm_mode_config_cleanup(drm);
+}
diff --git a/drivers/gpu/drm/sun4i/sun4i_framebuffer.h b/drivers/gpu/drm/sun4i/sun4i_framebuffer.h
new file mode 100644
index 0000000..3afd652
--- /dev/null
+++ b/drivers/gpu/drm/sun4i/sun4i_framebuffer.h
@@ -0,0 +1,19 @@
+/*
+ * Copyright (C) 2015 Free Electrons
+ * Copyright (C) 2015 NextThing Co
+ *
+ * Maxime Ripard <maxime.ripard@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ */
+
+#ifndef _SUN4I_FRAMEBUFFER_H_
+#define _SUN4I_FRAMEBUFFER_H_
+
+struct drm_fbdev_cma *sun4i_framebuffer_init(struct drm_device *drm);
+void sun4i_framebuffer_free(struct drm_device *drm);
+
+#endif /* _SUN4I_FRAMEBUFFER_H_ */
diff --git a/drivers/gpu/drm/sun4i/sun4i_layer.c b/drivers/gpu/drm/sun4i/sun4i_layer.c
new file mode 100644
index 0000000..068ab80
--- /dev/null
+++ b/drivers/gpu/drm/sun4i/sun4i_layer.c
@@ -0,0 +1,161 @@
+/*
+ * Copyright (C) 2015 Free Electrons
+ * Copyright (C) 2015 NextThing Co
+ *
+ * Maxime Ripard <maxime.ripard@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ */
+
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_crtc.h>
+#include <drm/drm_plane_helper.h>
+#include <drm/drmP.h>
+
+#include "sun4i_backend.h"
+#include "sun4i_drv.h"
+#include "sun4i_layer.h"
+
+#define SUN4I_NUM_LAYERS 2
+
+static int sun4i_backend_layer_atomic_check(struct drm_plane *plane,
+ struct drm_plane_state *state)
+{
+ return 0;
+}
+
+static void sun4i_backend_layer_atomic_disable(struct drm_plane *plane,
+ struct drm_plane_state *old_state)
+{
+ struct sun4i_layer *layer = plane_to_sun4i_layer(plane);
+ struct sun4i_drv *drv = layer->drv;
+ struct sun4i_backend *backend = drv->backend;
+
+ sun4i_backend_layer_enable(backend, layer->id, false);
+}
+
+static void sun4i_backend_layer_atomic_update(struct drm_plane *plane,
+ struct drm_plane_state *old_state)
+{
+ struct sun4i_layer *layer = plane_to_sun4i_layer(plane);
+ struct sun4i_drv *drv = layer->drv;
+ struct sun4i_backend *backend = drv->backend;
+
+ sun4i_backend_update_layer_coord(backend, layer->id, plane);
+ sun4i_backend_update_layer_formats(backend, layer->id, plane);
+ sun4i_backend_update_layer_buffer(backend, layer->id, plane);
+ sun4i_backend_layer_enable(backend, layer->id, true);
+}
+
+static struct drm_plane_helper_funcs sun4i_backend_layer_helper_funcs = {
+ .atomic_check = sun4i_backend_layer_atomic_check,
+ .atomic_disable = sun4i_backend_layer_atomic_disable,
+ .atomic_update = sun4i_backend_layer_atomic_update,
+};
+
+static const struct drm_plane_funcs sun4i_backend_layer_funcs = {
+ .atomic_destroy_state = drm_atomic_helper_plane_destroy_state,
+ .atomic_duplicate_state = drm_atomic_helper_plane_duplicate_state,
+ .destroy = drm_plane_cleanup,
+ .disable_plane = drm_atomic_helper_disable_plane,
+ .reset = drm_atomic_helper_plane_reset,
+ .update_plane = drm_atomic_helper_update_plane,
+};
+
+static const uint32_t sun4i_backend_layer_formats[] = {
+ DRM_FORMAT_ARGB8888,
+ DRM_FORMAT_XRGB8888,
+ DRM_FORMAT_RGB888,
+};
+
+static struct sun4i_layer *sun4i_layer_init_one(struct drm_device *drm,
+ enum drm_plane_type type)
+{
+ struct sun4i_drv *drv = drm->dev_private;
+ struct sun4i_layer *layer;
+ int ret;
+
+ layer = devm_kzalloc(drm->dev, sizeof(*layer), GFP_KERNEL);
+ if (!layer)
+ return ERR_PTR(-ENOMEM);
+
+ ret = drm_universal_plane_init(drm, &layer->plane, BIT(0),
+ &sun4i_backend_layer_funcs,
+ sun4i_backend_layer_formats,
+ ARRAY_SIZE(sun4i_backend_layer_formats),
+ type,
+ NULL);
+ if (ret) {
+ dev_err(drm->dev, "Couldn't initialize layer\n");
+ return ERR_PTR(ret);
+ }
+
+ drm_plane_helper_add(&layer->plane,
+ &sun4i_backend_layer_helper_funcs);
+ layer->drv = drv;
+
+ if (type == DRM_PLANE_TYPE_PRIMARY)
+ drv->primary = &layer->plane;
+
+ return layer;
+}
+
+struct sun4i_layer **sun4i_layers_init(struct drm_device *drm)
+{
+ struct sun4i_drv *drv = drm->dev_private;
+ struct sun4i_layer **layers;
+ int i;
+
+ layers = devm_kcalloc(drm->dev, SUN4I_NUM_LAYERS, sizeof(**layers),
+ GFP_KERNEL);
+ if (!layers)
+ return ERR_PTR(-ENOMEM);
+
+ /*
+ * The hardware is a bit unusual here.
+ *
+ * Even though it supports 4 layers, it does the composition
+ * in two separate steps.
+ *
+ * The first one is assigning a layer to one of its two
+ * pipes. If more that 1 layer is assigned to the same pipe,
+ * and if pixels overlaps, the pipe will take the pixel from
+ * the layer with the highest priority.
+ *
+ * The second step is the actual alpha blending, that takes
+ * the two pipes as input, and uses the eventual alpha
+ * component to do the transparency between the two.
+ *
+ * This two steps scenario makes us unable to guarantee a
+ * robust alpha blending between the 4 layers in all
+ * situations. So we just expose two layers, one per pipe. On
+ * SoCs that support it, sprites could fill the need for more
+ * layers.
+ */
+ for (i = 0; i < SUN4I_NUM_LAYERS; i++) {
+ enum drm_plane_type type = (i == 0)
+ ? DRM_PLANE_TYPE_PRIMARY
+ : DRM_PLANE_TYPE_OVERLAY;
+ struct sun4i_layer *layer = layers[i];
+
+ layer = sun4i_layer_init_one(drm, type);
+ if (IS_ERR(layer)) {
+ dev_err(drm->dev, "Couldn't initialize %s plane\n",
+ i ? "overlay" : "primary");
+ return ERR_CAST(layer);
+ };
+
+ DRM_DEBUG_DRIVER("Assigning %s plane to pipe %d\n",
+ i ? "overlay" : "primary", i);
+ regmap_update_bits(drv->backend->regs, SUN4I_BACKEND_ATTCTL_REG0(i),
+ SUN4I_BACKEND_ATTCTL_REG0_LAY_PIPESEL_MASK,
+ SUN4I_BACKEND_ATTCTL_REG0_LAY_PIPESEL(i));
+
+ layer->id = i;
+ };
+
+ return layers;
+}
diff --git a/drivers/gpu/drm/sun4i/sun4i_layer.h b/drivers/gpu/drm/sun4i/sun4i_layer.h
new file mode 100644
index 0000000..a2f65d7
--- /dev/null
+++ b/drivers/gpu/drm/sun4i/sun4i_layer.h
@@ -0,0 +1,30 @@
+/*
+ * Copyright (C) 2015 Free Electrons
+ * Copyright (C) 2015 NextThing Co
+ *
+ * Maxime Ripard <maxime.ripard@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ */
+
+#ifndef _SUN4I_LAYER_H_
+#define _SUN4I_LAYER_H_
+
+struct sun4i_layer {
+ struct drm_plane plane;
+ struct sun4i_drv *drv;
+ int id;
+};
+
+static inline struct sun4i_layer *
+plane_to_sun4i_layer(struct drm_plane *plane)
+{
+ return container_of(plane, struct sun4i_layer, plane);
+}
+
+struct sun4i_layer **sun4i_layers_init(struct drm_device *drm);
+
+#endif /* _SUN4I_LAYER_H_ */
diff --git a/drivers/gpu/drm/sun4i/sun4i_rgb.c b/drivers/gpu/drm/sun4i/sun4i_rgb.c
new file mode 100644
index 0000000..ab64948
--- /dev/null
+++ b/drivers/gpu/drm/sun4i/sun4i_rgb.c
@@ -0,0 +1,250 @@
+/*
+ * Copyright (C) 2015 Free Electrons
+ * Copyright (C) 2015 NextThing Co
+ *
+ * Maxime Ripard <maxime.ripard@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ */
+
+#include <linux/clk.h>
+
+#include <drm/drmP.h>
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_panel.h>
+
+#include "sun4i_drv.h"
+#include "sun4i_tcon.h"
+
+struct sun4i_rgb {
+ struct drm_connector connector;
+ struct drm_encoder encoder;
+
+ struct sun4i_drv *drv;
+};
+
+static inline struct sun4i_rgb *
+drm_connector_to_sun4i_rgb(struct drm_connector *connector)
+{
+ return container_of(connector, struct sun4i_rgb,
+ connector);
+}
+
+static inline struct sun4i_rgb *
+drm_encoder_to_sun4i_rgb(struct drm_encoder *encoder)
+{
+ return container_of(encoder, struct sun4i_rgb,
+ encoder);
+}
+
+static int sun4i_rgb_get_modes(struct drm_connector *connector)
+{
+ struct sun4i_rgb *rgb =
+ drm_connector_to_sun4i_rgb(connector);
+ struct sun4i_drv *drv = rgb->drv;
+ struct sun4i_tcon *tcon = drv->tcon;
+
+ return drm_panel_get_modes(tcon->panel);
+}
+
+static int sun4i_rgb_mode_valid(struct drm_connector *connector,
+ struct drm_display_mode *mode)
+{
+ u32 hsync = mode->hsync_end - mode->hsync_start;
+ u32 vsync = mode->vsync_end - mode->vsync_start;
+
+ DRM_DEBUG_DRIVER("Validating modes...\n");
+
+ if (hsync < 1)
+ return MODE_HSYNC_NARROW;
+
+ if (hsync > 0x3ff)
+ return MODE_HSYNC_WIDE;
+
+ if ((mode->hdisplay < 1) || (mode->htotal < 1))
+ return MODE_H_ILLEGAL;
+
+ if ((mode->hdisplay > 0x7ff) || (mode->htotal > 0xfff))
+ return MODE_BAD_HVALUE;
+
+ DRM_DEBUG_DRIVER("Horizontal parameters OK\n");
+
+ if (vsync < 1)
+ return MODE_VSYNC_NARROW;
+
+ if (vsync > 0x3ff)
+ return MODE_VSYNC_WIDE;
+
+ if ((mode->vdisplay < 1) || (mode->vtotal < 1))
+ return MODE_V_ILLEGAL;
+
+ if ((mode->vdisplay > 0x7ff) || (mode->vtotal > 0xfff))
+ return MODE_BAD_VVALUE;
+
+ DRM_DEBUG_DRIVER("Vertical parameters OK\n");
+
+ return MODE_OK;
+}
+
+static struct drm_encoder *
+sun4i_rgb_best_encoder(struct drm_connector *connector)
+{
+ struct sun4i_rgb *rgb =
+ drm_connector_to_sun4i_rgb(connector);
+
+ return &rgb->encoder;
+}
+
+static struct drm_connector_helper_funcs sun4i_rgb_con_helper_funcs = {
+ .get_modes = sun4i_rgb_get_modes,
+ .mode_valid = sun4i_rgb_mode_valid,
+ .best_encoder = sun4i_rgb_best_encoder,
+};
+
+static enum drm_connector_status
+sun4i_rgb_connector_detect(struct drm_connector *connector, bool force)
+{
+ return connector_status_connected;
+}
+
+static void
+sun4i_rgb_connector_destroy(struct drm_connector *connector)
+{
+ struct sun4i_rgb *rgb = drm_connector_to_sun4i_rgb(connector);
+ struct sun4i_drv *drv = rgb->drv;
+ struct sun4i_tcon *tcon = drv->tcon;
+
+ drm_panel_detach(tcon->panel);
+ drm_connector_cleanup(connector);
+}
+
+static struct drm_connector_funcs sun4i_rgb_con_funcs = {
+ .dpms = drm_atomic_helper_connector_dpms,
+ .detect = sun4i_rgb_connector_detect,
+ .fill_modes = drm_helper_probe_single_connector_modes,
+ .destroy = sun4i_rgb_connector_destroy,
+ .reset = drm_atomic_helper_connector_reset,
+ .atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
+};
+
+static int sun4i_rgb_atomic_check(struct drm_encoder *encoder,
+ struct drm_crtc_state *crtc_state,
+ struct drm_connector_state *conn_state)
+{
+ return 0;
+}
+
+static void sun4i_rgb_encoder_enable(struct drm_encoder *encoder)
+{
+ struct sun4i_rgb *rgb = drm_encoder_to_sun4i_rgb(encoder);
+ struct sun4i_drv *drv = rgb->drv;
+ struct sun4i_tcon *tcon = drv->tcon;
+
+ DRM_DEBUG_DRIVER("Enabling RGB output\n");
+
+ drm_panel_enable(tcon->panel);
+ sun4i_tcon_channel_enable(tcon, 0);
+}
+
+static void sun4i_rgb_encoder_disable(struct drm_encoder *encoder)
+{
+ struct sun4i_rgb *rgb = drm_encoder_to_sun4i_rgb(encoder);
+ struct sun4i_drv *drv = rgb->drv;
+ struct sun4i_tcon *tcon = drv->tcon;
+
+ DRM_DEBUG_DRIVER("Disabling RGB output\n");
+
+ sun4i_tcon_channel_disable(tcon, 0);
+ drm_panel_disable(tcon->panel);
+}
+
+static void sun4i_rgb_encoder_mode_set(struct drm_encoder *encoder,
+ struct drm_display_mode *mode,
+ struct drm_display_mode *adjusted_mode)
+{
+ struct sun4i_rgb *rgb = drm_encoder_to_sun4i_rgb(encoder);
+ struct sun4i_drv *drv = rgb->drv;
+ struct sun4i_tcon *tcon = drv->tcon;
+
+ sun4i_tcon0_mode_set(tcon, mode);
+
+ clk_set_rate(tcon->dclk, mode->crtc_clock * 1000);
+
+ /* FIXME: This seems to be board specific */
+ clk_set_phase(tcon->dclk, 120);
+}
+
+static struct drm_encoder_helper_funcs sun4i_rgb_enc_helper_funcs = {
+ .atomic_check = sun4i_rgb_atomic_check,
+ .mode_set = sun4i_rgb_encoder_mode_set,
+ .disable = sun4i_rgb_encoder_disable,
+ .enable = sun4i_rgb_encoder_enable,
+};
+
+static void sun4i_rgb_enc_destroy(struct drm_encoder *encoder)
+{
+ drm_encoder_cleanup(encoder);
+}
+
+static struct drm_encoder_funcs sun4i_rgb_enc_funcs = {
+ .destroy = sun4i_rgb_enc_destroy,
+};
+
+int sun4i_rgb_init(struct drm_device *drm)
+{
+ struct sun4i_drv *drv = drm->dev_private;
+ struct sun4i_tcon *tcon = drv->tcon;
+ struct sun4i_rgb *rgb;
+ int ret;
+
+ /* If we don't have a panel, there's no point in going on */
+ if (!tcon->panel)
+ return -ENODEV;
+
+ rgb = devm_kzalloc(drm->dev, sizeof(*rgb), GFP_KERNEL);
+ if (!rgb)
+ return -ENOMEM;
+ rgb->drv = drv;
+
+ drm_encoder_helper_add(&rgb->encoder,
+ &sun4i_rgb_enc_helper_funcs);
+ ret = drm_encoder_init(drm,
+ &rgb->encoder,
+ &sun4i_rgb_enc_funcs,
+ DRM_MODE_ENCODER_NONE,
+ NULL);
+ if (ret) {
+ dev_err(drm->dev, "Couldn't initialise the rgb encoder\n");
+ goto err_out;
+ }
+
+ /* The RGB encoder can only work with the TCON channel 0 */
+ rgb->encoder.possible_crtcs = BIT(0);
+
+ drm_connector_helper_add(&rgb->connector,
+ &sun4i_rgb_con_helper_funcs);
+ ret = drm_connector_init(drm, &rgb->connector,
+ &sun4i_rgb_con_funcs,
+ DRM_MODE_CONNECTOR_Unknown);
+ if (ret) {
+ dev_err(drm->dev, "Couldn't initialise the rgb connector\n");
+ goto err_cleanup_connector;
+ }
+
+ drm_mode_connector_attach_encoder(&rgb->connector, &rgb->encoder);
+
+ drm_panel_attach(tcon->panel, &rgb->connector);
+
+ return 0;
+
+err_cleanup_connector:
+ drm_encoder_cleanup(&rgb->encoder);
+err_out:
+ return ret;
+}
+EXPORT_SYMBOL(sun4i_rgb_init);
diff --git a/drivers/gpu/drm/sun4i/sun4i_rgb.h b/drivers/gpu/drm/sun4i/sun4i_rgb.h
new file mode 100644
index 0000000..7c4da4c
--- /dev/null
+++ b/drivers/gpu/drm/sun4i/sun4i_rgb.h
@@ -0,0 +1,18 @@
+/*
+ * Copyright (C) 2015 Free Electrons
+ * Copyright (C) 2015 NextThing Co
+ *
+ * Maxime Ripard <maxime.ripard@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ */
+
+#ifndef _SUN4I_RGB_H_
+#define _SUN4I_RGB_H_
+
+int sun4i_rgb_init(struct drm_device *drm);
+
+#endif /* _SUN4I_RGB_H_ */
diff --git a/drivers/gpu/drm/sun4i/sun4i_tcon.c b/drivers/gpu/drm/sun4i/sun4i_tcon.c
new file mode 100644
index 0000000..9f19b0e
--- /dev/null
+++ b/drivers/gpu/drm/sun4i/sun4i_tcon.c
@@ -0,0 +1,561 @@
+/*
+ * Copyright (C) 2015 Free Electrons
+ * Copyright (C) 2015 NextThing Co
+ *
+ * Maxime Ripard <maxime.ripard@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ */
+
+#include <drm/drmP.h>
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_crtc.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_modes.h>
+#include <drm/drm_panel.h>
+
+#include <linux/component.h>
+#include <linux/ioport.h>
+#include <linux/of_address.h>
+#include <linux/of_graph.h>
+#include <linux/of_irq.h>
+#include <linux/regmap.h>
+#include <linux/reset.h>
+
+#include "sun4i_crtc.h"
+#include "sun4i_dotclock.h"
+#include "sun4i_drv.h"
+#include "sun4i_rgb.h"
+#include "sun4i_tcon.h"
+
+void sun4i_tcon_disable(struct sun4i_tcon *tcon)
+{
+ DRM_DEBUG_DRIVER("Disabling TCON\n");
+
+ /* Disable the TCON */
+ regmap_update_bits(tcon->regs, SUN4I_TCON_GCTL_REG,
+ SUN4I_TCON_GCTL_TCON_ENABLE, 0);
+}
+EXPORT_SYMBOL(sun4i_tcon_disable);
+
+void sun4i_tcon_enable(struct sun4i_tcon *tcon)
+{
+ DRM_DEBUG_DRIVER("Enabling TCON\n");
+
+ /* Enable the TCON */
+ regmap_update_bits(tcon->regs, SUN4I_TCON_GCTL_REG,
+ SUN4I_TCON_GCTL_TCON_ENABLE,
+ SUN4I_TCON_GCTL_TCON_ENABLE);
+}
+EXPORT_SYMBOL(sun4i_tcon_enable);
+
+void sun4i_tcon_channel_disable(struct sun4i_tcon *tcon, int channel)
+{
+ /* Disable the TCON's channel */
+ if (channel == 0) {
+ regmap_update_bits(tcon->regs, SUN4I_TCON0_CTL_REG,
+ SUN4I_TCON0_CTL_TCON_ENABLE, 0);
+ clk_disable_unprepare(tcon->dclk);
+ } else if (channel == 1) {
+ regmap_update_bits(tcon->regs, SUN4I_TCON1_CTL_REG,
+ SUN4I_TCON1_CTL_TCON_ENABLE, 0);
+ clk_disable_unprepare(tcon->sclk1);
+ }
+}
+EXPORT_SYMBOL(sun4i_tcon_channel_disable);
+
+void sun4i_tcon_channel_enable(struct sun4i_tcon *tcon, int channel)
+{
+ /* Enable the TCON's channel */
+ if (channel == 0) {
+ regmap_update_bits(tcon->regs, SUN4I_TCON0_CTL_REG,
+ SUN4I_TCON0_CTL_TCON_ENABLE,
+ SUN4I_TCON0_CTL_TCON_ENABLE);
+ clk_prepare_enable(tcon->dclk);
+ } else if (channel == 1) {
+ regmap_update_bits(tcon->regs, SUN4I_TCON1_CTL_REG,
+ SUN4I_TCON1_CTL_TCON_ENABLE,
+ SUN4I_TCON1_CTL_TCON_ENABLE);
+ clk_prepare_enable(tcon->sclk1);
+ }
+}
+EXPORT_SYMBOL(sun4i_tcon_channel_enable);
+
+void sun4i_tcon_enable_vblank(struct sun4i_tcon *tcon, bool enable)
+{
+ u32 mask, val = 0;
+
+ DRM_DEBUG_DRIVER("%sabling VBLANK interrupt\n", enable ? "En" : "Dis");
+
+ mask = SUN4I_TCON_GINT0_VBLANK_ENABLE(0) |
+ SUN4I_TCON_GINT0_VBLANK_ENABLE(1);
+
+ if (enable)
+ val = mask;
+
+ regmap_update_bits(tcon->regs, SUN4I_TCON_GINT0_REG, mask, val);
+}
+EXPORT_SYMBOL(sun4i_tcon_enable_vblank);
+
+static int sun4i_tcon_get_clk_delay(struct drm_display_mode *mode,
+ int channel)
+{
+ int delay = mode->vtotal - mode->vdisplay;
+
+ if (mode->flags & DRM_MODE_FLAG_INTERLACE)
+ delay /= 2;
+
+ if (channel == 1)
+ delay -= 2;
+
+ delay = min(delay, 30);
+
+ DRM_DEBUG_DRIVER("TCON %d clock delay %u\n", channel, delay);
+
+ return delay;
+}
+
+void sun4i_tcon0_mode_set(struct sun4i_tcon *tcon,
+ struct drm_display_mode *mode)
+{
+ unsigned int bp, hsync, vsync;
+ u8 clk_delay;
+ u32 val = 0;
+
+ /* Adjust clock delay */
+ clk_delay = sun4i_tcon_get_clk_delay(mode, 0);
+ regmap_update_bits(tcon->regs, SUN4I_TCON0_CTL_REG,
+ SUN4I_TCON0_CTL_CLK_DELAY_MASK,
+ SUN4I_TCON0_CTL_CLK_DELAY(clk_delay));
+
+ /* Set the resolution */
+ regmap_write(tcon->regs, SUN4I_TCON0_BASIC0_REG,
+ SUN4I_TCON0_BASIC0_X(mode->crtc_hdisplay) |
+ SUN4I_TCON0_BASIC0_Y(mode->crtc_vdisplay));
+
+ /*
+ * This is called a backporch in the register documentation,
+ * but it really is the front porch + hsync
+ */
+ bp = mode->crtc_htotal - mode->crtc_hsync_start;
+ DRM_DEBUG_DRIVER("Setting horizontal total %d, backporch %d\n",
+ mode->crtc_htotal, bp);
+
+ /* Set horizontal display timings */
+ regmap_write(tcon->regs, SUN4I_TCON0_BASIC1_REG,
+ SUN4I_TCON0_BASIC1_H_TOTAL(mode->crtc_htotal) |
+ SUN4I_TCON0_BASIC1_H_BACKPORCH(bp));
+
+ /*
+ * This is called a backporch in the register documentation,
+ * but it really is the front porch + hsync
+ */
+ bp = mode->crtc_vtotal - mode->crtc_vsync_start;
+ DRM_DEBUG_DRIVER("Setting vertical total %d, backporch %d\n",
+ mode->crtc_vtotal, bp);
+
+ /* Set vertical display timings */
+ regmap_write(tcon->regs, SUN4I_TCON0_BASIC2_REG,
+ SUN4I_TCON0_BASIC2_V_TOTAL(mode->crtc_vtotal) |
+ SUN4I_TCON0_BASIC2_V_BACKPORCH(bp));
+
+ /* Set Hsync and Vsync length */
+ hsync = mode->crtc_hsync_end - mode->crtc_hsync_start;
+ vsync = mode->crtc_vsync_end - mode->crtc_vsync_start;
+ DRM_DEBUG_DRIVER("Setting HSYNC %d, VSYNC %d\n", hsync, vsync);
+ regmap_write(tcon->regs, SUN4I_TCON0_BASIC3_REG,
+ SUN4I_TCON0_BASIC3_V_SYNC(vsync) |
+ SUN4I_TCON0_BASIC3_H_SYNC(hsync));
+
+ /* Setup the polarity of the various signals */
+ if (!(mode->flags & DRM_MODE_FLAG_PHSYNC))
+ val |= SUN4I_TCON0_IO_POL_HSYNC_POSITIVE;
+
+ if (!(mode->flags & DRM_MODE_FLAG_PVSYNC))
+ val |= SUN4I_TCON0_IO_POL_VSYNC_POSITIVE;
+
+ regmap_update_bits(tcon->regs, SUN4I_TCON0_IO_POL_REG,
+ SUN4I_TCON0_IO_POL_HSYNC_POSITIVE | SUN4I_TCON0_IO_POL_VSYNC_POSITIVE,
+ val);
+
+ /* Map output pins to channel 0 */
+ regmap_update_bits(tcon->regs, SUN4I_TCON_GCTL_REG,
+ SUN4I_TCON_GCTL_IOMAP_MASK,
+ SUN4I_TCON_GCTL_IOMAP_TCON0);
+
+ /* Enable the output on the pins */
+ regmap_write(tcon->regs, SUN4I_TCON0_IO_TRI_REG, 0);
+}
+EXPORT_SYMBOL(sun4i_tcon0_mode_set);
+
+void sun4i_tcon1_mode_set(struct sun4i_tcon *tcon,
+ struct drm_display_mode *mode)
+{
+ unsigned int bp, hsync, vsync;
+ u8 clk_delay;
+ u32 val;
+
+ /* Adjust clock delay */
+ clk_delay = sun4i_tcon_get_clk_delay(mode, 1);
+ regmap_update_bits(tcon->regs, SUN4I_TCON1_CTL_REG,
+ SUN4I_TCON1_CTL_CLK_DELAY_MASK,
+ SUN4I_TCON1_CTL_CLK_DELAY(clk_delay));
+
+ /* Set interlaced mode */
+ if (mode->flags & DRM_MODE_FLAG_INTERLACE)
+ val = SUN4I_TCON1_CTL_INTERLACE_ENABLE;
+ else
+ val = 0;
+ regmap_update_bits(tcon->regs, SUN4I_TCON1_CTL_REG,
+ SUN4I_TCON1_CTL_INTERLACE_ENABLE,
+ val);
+
+ /* Set the input resolution */
+ regmap_write(tcon->regs, SUN4I_TCON1_BASIC0_REG,
+ SUN4I_TCON1_BASIC0_X(mode->crtc_hdisplay) |
+ SUN4I_TCON1_BASIC0_Y(mode->crtc_vdisplay));
+
+ /* Set the upscaling resolution */
+ regmap_write(tcon->regs, SUN4I_TCON1_BASIC1_REG,
+ SUN4I_TCON1_BASIC1_X(mode->crtc_hdisplay) |
+ SUN4I_TCON1_BASIC1_Y(mode->crtc_vdisplay));
+
+ /* Set the output resolution */
+ regmap_write(tcon->regs, SUN4I_TCON1_BASIC2_REG,
+ SUN4I_TCON1_BASIC2_X(mode->crtc_hdisplay) |
+ SUN4I_TCON1_BASIC2_Y(mode->crtc_vdisplay));
+
+ /* Set horizontal display timings */
+ bp = mode->crtc_htotal - mode->crtc_hsync_end;
+ DRM_DEBUG_DRIVER("Setting horizontal total %d, backporch %d\n",
+ mode->htotal, bp);
+ regmap_write(tcon->regs, SUN4I_TCON1_BASIC3_REG,
+ SUN4I_TCON1_BASIC3_H_TOTAL(mode->crtc_htotal) |
+ SUN4I_TCON1_BASIC3_H_BACKPORCH(bp));
+
+ /* Set vertical display timings */
+ bp = mode->crtc_vtotal - mode->crtc_vsync_end;
+ DRM_DEBUG_DRIVER("Setting vertical total %d, backporch %d\n",
+ mode->vtotal, bp);
+ regmap_write(tcon->regs, SUN4I_TCON1_BASIC4_REG,
+ SUN4I_TCON1_BASIC4_V_TOTAL(mode->vtotal) |
+ SUN4I_TCON1_BASIC4_V_BACKPORCH(bp));
+
+ /* Set Hsync and Vsync length */
+ hsync = mode->crtc_hsync_end - mode->crtc_hsync_start;
+ vsync = mode->crtc_vsync_end - mode->crtc_vsync_start;
+ DRM_DEBUG_DRIVER("Setting HSYNC %d, VSYNC %d\n", hsync, vsync);
+ regmap_write(tcon->regs, SUN4I_TCON1_BASIC5_REG,
+ SUN4I_TCON1_BASIC5_V_SYNC(vsync) |
+ SUN4I_TCON1_BASIC5_H_SYNC(hsync));
+
+ /* Map output pins to channel 1 */
+ regmap_update_bits(tcon->regs, SUN4I_TCON_GCTL_REG,
+ SUN4I_TCON_GCTL_IOMAP_MASK,
+ SUN4I_TCON_GCTL_IOMAP_TCON1);
+
+ /*
+ * FIXME: Undocumented bits
+ */
+ if (tcon->has_mux)
+ regmap_write(tcon->regs, SUN4I_TCON_MUX_CTRL_REG, 1);
+}
+EXPORT_SYMBOL(sun4i_tcon1_mode_set);
+
+static void sun4i_tcon_finish_page_flip(struct drm_device *dev,
+ struct sun4i_crtc *scrtc)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&dev->event_lock, flags);
+ if (scrtc->event) {
+ drm_crtc_send_vblank_event(&scrtc->crtc, scrtc->event);
+ drm_crtc_vblank_put(&scrtc->crtc);
+ scrtc->event = NULL;
+ }
+ spin_unlock_irqrestore(&dev->event_lock, flags);
+}
+
+static irqreturn_t sun4i_tcon_handler(int irq, void *private)
+{
+ struct sun4i_tcon *tcon = private;
+ struct drm_device *drm = tcon->drm;
+ struct sun4i_drv *drv = drm->dev_private;
+ struct sun4i_crtc *scrtc = drv->crtc;
+ unsigned int status;
+
+ regmap_read(tcon->regs, SUN4I_TCON_GINT0_REG, &status);
+
+ if (!(status & (SUN4I_TCON_GINT0_VBLANK_INT(0) |
+ SUN4I_TCON_GINT0_VBLANK_INT(1))))
+ return IRQ_NONE;
+
+ drm_crtc_handle_vblank(&scrtc->crtc);
+ sun4i_tcon_finish_page_flip(drm, scrtc);
+
+ /* Acknowledge the interrupt */
+ regmap_update_bits(tcon->regs, SUN4I_TCON_GINT0_REG,
+ SUN4I_TCON_GINT0_VBLANK_INT(0) |
+ SUN4I_TCON_GINT0_VBLANK_INT(1),
+ 0);
+
+ return IRQ_HANDLED;
+}
+
+static int sun4i_tcon_init_clocks(struct device *dev,
+ struct sun4i_tcon *tcon)
+{
+ tcon->clk = devm_clk_get(dev, "ahb");
+ if (IS_ERR(tcon->clk)) {
+ dev_err(dev, "Couldn't get the TCON bus clock\n");
+ return PTR_ERR(tcon->clk);
+ }
+ clk_prepare_enable(tcon->clk);
+
+ tcon->sclk0 = devm_clk_get(dev, "tcon-ch0");
+ if (IS_ERR(tcon->sclk0)) {
+ dev_err(dev, "Couldn't get the TCON channel 0 clock\n");
+ return PTR_ERR(tcon->sclk0);
+ }
+
+ tcon->sclk1 = devm_clk_get(dev, "tcon-ch1");
+ if (IS_ERR(tcon->sclk1)) {
+ dev_err(dev, "Couldn't get the TCON channel 1 clock\n");
+ return PTR_ERR(tcon->sclk1);
+ }
+
+ return sun4i_dclk_create(dev, tcon);
+}
+
+static void sun4i_tcon_free_clocks(struct sun4i_tcon *tcon)
+{
+ sun4i_dclk_free(tcon);
+ clk_disable_unprepare(tcon->clk);
+}
+
+static int sun4i_tcon_init_irq(struct device *dev,
+ struct sun4i_tcon *tcon)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ int irq, ret;
+
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0) {
+ dev_err(dev, "Couldn't retrieve the TCON interrupt\n");
+ return irq;
+ }
+
+ ret = devm_request_irq(dev, irq, sun4i_tcon_handler, 0,
+ dev_name(dev), tcon);
+ if (ret) {
+ dev_err(dev, "Couldn't request the IRQ\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+static struct regmap_config sun4i_tcon_regmap_config = {
+ .reg_bits = 32,
+ .val_bits = 32,
+ .reg_stride = 4,
+ .max_register = 0x800,
+};
+
+static int sun4i_tcon_init_regmap(struct device *dev,
+ struct sun4i_tcon *tcon)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct resource *res;
+ void __iomem *regs;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ regs = devm_ioremap_resource(dev, res);
+ if (IS_ERR(regs)) {
+ dev_err(dev, "Couldn't map the TCON registers\n");
+ return PTR_ERR(regs);
+ }
+
+ tcon->regs = devm_regmap_init_mmio(dev, regs,
+ &sun4i_tcon_regmap_config);
+ if (IS_ERR(tcon->regs)) {
+ dev_err(dev, "Couldn't create the TCON regmap\n");
+ return PTR_ERR(tcon->regs);
+ }
+
+ /* Make sure the TCON is disabled and all IRQs are off */
+ regmap_write(tcon->regs, SUN4I_TCON_GCTL_REG, 0);
+ regmap_write(tcon->regs, SUN4I_TCON_GINT0_REG, 0);
+ regmap_write(tcon->regs, SUN4I_TCON_GINT1_REG, 0);
+
+ /* Disable IO lines and set them to tristate */
+ regmap_write(tcon->regs, SUN4I_TCON0_IO_TRI_REG, ~0);
+ regmap_write(tcon->regs, SUN4I_TCON1_IO_TRI_REG, ~0);
+
+ return 0;
+}
+
+static struct drm_panel *sun4i_tcon_find_panel(struct device_node *node)
+{
+ struct device_node *port, *remote, *child;
+ struct device_node *end_node = NULL;
+
+ /* Inputs are listed first, then outputs */
+ port = of_graph_get_port_by_id(node, 1);
+
+ /*
+ * Our first output is the RGB interface where the panel will
+ * be connected.
+ */
+ for_each_child_of_node(port, child) {
+ u32 reg;
+
+ of_property_read_u32(child, "reg", ®);
+ if (reg == 0)
+ end_node = child;
+ }
+
+ if (!end_node) {
+ DRM_DEBUG_DRIVER("Missing panel endpoint\n");
+ return ERR_PTR(-ENODEV);
+ }
+
+ remote = of_graph_get_remote_port_parent(end_node);
+ if (!remote) {
+ DRM_DEBUG_DRIVER("Enable to parse remote node\n");
+ return ERR_PTR(-EINVAL);
+ }
+
+ return of_drm_find_panel(remote);
+}
+
+static int sun4i_tcon_bind(struct device *dev, struct device *master,
+ void *data)
+{
+ struct drm_device *drm = data;
+ struct sun4i_drv *drv = drm->dev_private;
+ struct sun4i_tcon *tcon;
+ int ret;
+
+ tcon = devm_kzalloc(dev, sizeof(*tcon), GFP_KERNEL);
+ if (!tcon)
+ return -ENOMEM;
+ dev_set_drvdata(dev, tcon);
+ drv->tcon = tcon;
+ tcon->drm = drm;
+
+ if (of_device_is_compatible(dev->of_node, "allwinner,sun5i-a13-tcon"))
+ tcon->has_mux = true;
+
+ tcon->lcd_rst = devm_reset_control_get(dev, "lcd");
+ if (IS_ERR(tcon->lcd_rst)) {
+ dev_err(dev, "Couldn't get our reset line\n");
+ return PTR_ERR(tcon->lcd_rst);
+ }
+
+ /* Make sure our TCON is reset */
+ if (!reset_control_status(tcon->lcd_rst))
+ reset_control_assert(tcon->lcd_rst);
+
+ ret = reset_control_deassert(tcon->lcd_rst);
+ if (ret) {
+ dev_err(dev, "Couldn't deassert our reset line\n");
+ return ret;
+ }
+
+ ret = sun4i_tcon_init_regmap(dev, tcon);
+ if (ret) {
+ dev_err(dev, "Couldn't init our TCON regmap\n");
+ goto err_assert_reset;
+ }
+
+ ret = sun4i_tcon_init_clocks(dev, tcon);
+ if (ret) {
+ dev_err(dev, "Couldn't init our TCON clocks\n");
+ goto err_assert_reset;
+ }
+
+ ret = sun4i_tcon_init_irq(dev, tcon);
+ if (ret) {
+ dev_err(dev, "Couldn't init our TCON interrupts\n");
+ goto err_free_clocks;
+ }
+
+ tcon->panel = sun4i_tcon_find_panel(dev->of_node);
+ if (IS_ERR(tcon->panel)) {
+ dev_info(dev, "No panel found... RGB output disabled\n");
+ return 0;
+ }
+
+ return sun4i_rgb_init(drm);
+
+err_free_clocks:
+ sun4i_tcon_free_clocks(tcon);
+err_assert_reset:
+ reset_control_assert(tcon->lcd_rst);
+ return ret;
+}
+
+static void sun4i_tcon_unbind(struct device *dev, struct device *master,
+ void *data)
+{
+ struct sun4i_tcon *tcon = dev_get_drvdata(dev);
+
+ sun4i_tcon_free_clocks(tcon);
+}
+
+static struct component_ops sun4i_tcon_ops = {
+ .bind = sun4i_tcon_bind,
+ .unbind = sun4i_tcon_unbind,
+};
+
+static int sun4i_tcon_probe(struct platform_device *pdev)
+{
+ struct device_node *node = pdev->dev.of_node;
+ struct drm_panel *panel;
+
+ /*
+ * The panel is not ready.
+ * Defer the probe.
+ */
+ panel = sun4i_tcon_find_panel(node);
+ if (IS_ERR(panel)) {
+ /*
+ * If we don't have a panel endpoint, just go on
+ */
+ if (PTR_ERR(panel) != -ENODEV)
+ return -EPROBE_DEFER;
+ }
+
+ return component_add(&pdev->dev, &sun4i_tcon_ops);
+}
+
+static int sun4i_tcon_remove(struct platform_device *pdev)
+{
+ component_del(&pdev->dev, &sun4i_tcon_ops);
+
+ return 0;
+}
+
+static const struct of_device_id sun4i_tcon_of_table[] = {
+ { .compatible = "allwinner,sun5i-a13-tcon" },
+ { }
+};
+MODULE_DEVICE_TABLE(of, sun4i_tcon_of_table);
+
+static struct platform_driver sun4i_tcon_platform_driver = {
+ .probe = sun4i_tcon_probe,
+ .remove = sun4i_tcon_remove,
+ .driver = {
+ .name = "sun4i-tcon",
+ .of_match_table = sun4i_tcon_of_table,
+ },
+};
+module_platform_driver(sun4i_tcon_platform_driver);
+
+MODULE_AUTHOR("Maxime Ripard <maxime.ripard@free-electrons.com>");
+MODULE_DESCRIPTION("Allwinner A10 Timing Controller Driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/gpu/drm/sun4i/sun4i_tcon.h b/drivers/gpu/drm/sun4i/sun4i_tcon.h
new file mode 100644
index 0000000..0e0b11d
--- /dev/null
+++ b/drivers/gpu/drm/sun4i/sun4i_tcon.h
@@ -0,0 +1,186 @@
+/*
+ * Copyright (C) 2015 Free Electrons
+ * Copyright (C) 2015 NextThing Co
+ *
+ * Boris Brezillon <boris.brezillon@free-electrons.com>
+ * Maxime Ripard <maxime.ripard@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ */
+
+#ifndef __SUN4I_TCON_H__
+#define __SUN4I_TCON_H__
+
+#include <drm/drm_crtc.h>
+
+#include <linux/kernel.h>
+#include <linux/reset.h>
+
+#define SUN4I_TCON_GCTL_REG 0x0
+#define SUN4I_TCON_GCTL_TCON_ENABLE BIT(31)
+#define SUN4I_TCON_GCTL_IOMAP_MASK BIT(0)
+#define SUN4I_TCON_GCTL_IOMAP_TCON1 (1 << 0)
+#define SUN4I_TCON_GCTL_IOMAP_TCON0 (0 << 0)
+
+#define SUN4I_TCON_GINT0_REG 0x4
+#define SUN4I_TCON_GINT0_VBLANK_ENABLE(pipe) BIT(31 - (pipe))
+#define SUN4I_TCON_GINT0_VBLANK_INT(pipe) BIT(15 - (pipe))
+
+#define SUN4I_TCON_GINT1_REG 0x8
+#define SUN4I_TCON_FRM_CTL_REG 0x10
+
+#define SUN4I_TCON0_CTL_REG 0x40
+#define SUN4I_TCON0_CTL_TCON_ENABLE BIT(31)
+#define SUN4I_TCON0_CTL_CLK_DELAY_MASK GENMASK(8, 4)
+#define SUN4I_TCON0_CTL_CLK_DELAY(delay) ((delay << 4) & SUN4I_TCON0_CTL_CLK_DELAY_MASK)
+
+#define SUN4I_TCON0_DCLK_REG 0x44
+#define SUN4I_TCON0_DCLK_GATE_BIT (31)
+#define SUN4I_TCON0_DCLK_DIV_SHIFT (0)
+#define SUN4I_TCON0_DCLK_DIV_WIDTH (7)
+
+#define SUN4I_TCON0_BASIC0_REG 0x48
+#define SUN4I_TCON0_BASIC0_X(width) ((((width) - 1) & 0xfff) << 16)
+#define SUN4I_TCON0_BASIC0_Y(height) (((height) - 1) & 0xfff)
+
+#define SUN4I_TCON0_BASIC1_REG 0x4c
+#define SUN4I_TCON0_BASIC1_H_TOTAL(total) ((((total) - 1) & 0x1fff) << 16)
+#define SUN4I_TCON0_BASIC1_H_BACKPORCH(bp) (((bp) - 1) & 0xfff)
+
+#define SUN4I_TCON0_BASIC2_REG 0x50
+#define SUN4I_TCON0_BASIC2_V_TOTAL(total) ((((total) * 2) & 0x1fff) << 16)
+#define SUN4I_TCON0_BASIC2_V_BACKPORCH(bp) (((bp) - 1) & 0xfff)
+
+#define SUN4I_TCON0_BASIC3_REG 0x54
+#define SUN4I_TCON0_BASIC3_H_SYNC(width) ((((width) - 1) & 0x7ff) << 16)
+#define SUN4I_TCON0_BASIC3_V_SYNC(height) (((height) - 1) & 0x7ff)
+
+#define SUN4I_TCON0_HV_IF_REG 0x58
+#define SUN4I_TCON0_CPU_IF_REG 0x60
+#define SUN4I_TCON0_CPU_WR_REG 0x64
+#define SUN4I_TCON0_CPU_RD0_REG 0x68
+#define SUN4I_TCON0_CPU_RDA_REG 0x6c
+#define SUN4I_TCON0_TTL0_REG 0x70
+#define SUN4I_TCON0_TTL1_REG 0x74
+#define SUN4I_TCON0_TTL2_REG 0x78
+#define SUN4I_TCON0_TTL3_REG 0x7c
+#define SUN4I_TCON0_TTL4_REG 0x80
+#define SUN4I_TCON0_LVDS_IF_REG 0x84
+#define SUN4I_TCON0_IO_POL_REG 0x88
+#define SUN4I_TCON0_IO_POL_DCLK_PHASE(phase) ((phase & 3) << 28)
+#define SUN4I_TCON0_IO_POL_HSYNC_POSITIVE BIT(25)
+#define SUN4I_TCON0_IO_POL_VSYNC_POSITIVE BIT(24)
+
+#define SUN4I_TCON0_IO_TRI_REG 0x8c
+#define SUN4I_TCON0_IO_TRI_HSYNC_DISABLE BIT(25)
+#define SUN4I_TCON0_IO_TRI_VSYNC_DISABLE BIT(24)
+#define SUN4I_TCON0_IO_TRI_DATA_PINS_DISABLE(pins) GENMASK(pins, 0)
+
+#define SUN4I_TCON1_CTL_REG 0x90
+#define SUN4I_TCON1_CTL_TCON_ENABLE BIT(31)
+#define SUN4I_TCON1_CTL_INTERLACE_ENABLE BIT(20)
+#define SUN4I_TCON1_CTL_CLK_DELAY_MASK GENMASK(8, 4)
+#define SUN4I_TCON1_CTL_CLK_DELAY(delay) ((delay << 4) & SUN4I_TCON1_CTL_CLK_DELAY_MASK)
+
+#define SUN4I_TCON1_BASIC0_REG 0x94
+#define SUN4I_TCON1_BASIC0_X(width) ((((width) - 1) & 0xfff) << 16)
+#define SUN4I_TCON1_BASIC0_Y(height) (((height) - 1) & 0xfff)
+
+#define SUN4I_TCON1_BASIC1_REG 0x98
+#define SUN4I_TCON1_BASIC1_X(width) ((((width) - 1) & 0xfff) << 16)
+#define SUN4I_TCON1_BASIC1_Y(height) (((height) - 1) & 0xfff)
+
+#define SUN4I_TCON1_BASIC2_REG 0x9c
+#define SUN4I_TCON1_BASIC2_X(width) ((((width) - 1) & 0xfff) << 16)
+#define SUN4I_TCON1_BASIC2_Y(height) (((height) - 1) & 0xfff)
+
+#define SUN4I_TCON1_BASIC3_REG 0xa0
+#define SUN4I_TCON1_BASIC3_H_TOTAL(total) ((((total) - 1) & 0x1fff) << 16)
+#define SUN4I_TCON1_BASIC3_H_BACKPORCH(bp) (((bp) - 1) & 0xfff)
+
+#define SUN4I_TCON1_BASIC4_REG 0xa4
+#define SUN4I_TCON1_BASIC4_V_TOTAL(total) (((total) & 0x1fff) << 16)
+#define SUN4I_TCON1_BASIC4_V_BACKPORCH(bp) (((bp) - 1) & 0xfff)
+
+#define SUN4I_TCON1_BASIC5_REG 0xa8
+#define SUN4I_TCON1_BASIC5_H_SYNC(width) ((((width) - 1) & 0x3ff) << 16)
+#define SUN4I_TCON1_BASIC5_V_SYNC(height) (((height) - 1) & 0x3ff)
+
+#define SUN4I_TCON1_IO_POL_REG 0xf0
+#define SUN4I_TCON1_IO_TRI_REG 0xf4
+#define SUN4I_TCON_CEU_CTL_REG 0x100
+#define SUN4I_TCON_CEU_MUL_RR_REG 0x110
+#define SUN4I_TCON_CEU_MUL_RG_REG 0x114
+#define SUN4I_TCON_CEU_MUL_RB_REG 0x118
+#define SUN4I_TCON_CEU_ADD_RC_REG 0x11c
+#define SUN4I_TCON_CEU_MUL_GR_REG 0x120
+#define SUN4I_TCON_CEU_MUL_GG_REG 0x124
+#define SUN4I_TCON_CEU_MUL_GB_REG 0x128
+#define SUN4I_TCON_CEU_ADD_GC_REG 0x12c
+#define SUN4I_TCON_CEU_MUL_BR_REG 0x130
+#define SUN4I_TCON_CEU_MUL_BG_REG 0x134
+#define SUN4I_TCON_CEU_MUL_BB_REG 0x138
+#define SUN4I_TCON_CEU_ADD_BC_REG 0x13c
+#define SUN4I_TCON_CEU_RANGE_R_REG 0x140
+#define SUN4I_TCON_CEU_RANGE_G_REG 0x144
+#define SUN4I_TCON_CEU_RANGE_B_REG 0x148
+#define SUN4I_TCON_MUX_CTRL_REG 0x200
+#define SUN4I_TCON1_FILL_CTL_REG 0x300
+#define SUN4I_TCON1_FILL_BEG0_REG 0x304
+#define SUN4I_TCON1_FILL_END0_REG 0x308
+#define SUN4I_TCON1_FILL_DATA0_REG 0x30c
+#define SUN4I_TCON1_FILL_BEG1_REG 0x310
+#define SUN4I_TCON1_FILL_END1_REG 0x314
+#define SUN4I_TCON1_FILL_DATA1_REG 0x318
+#define SUN4I_TCON1_FILL_BEG2_REG 0x31c
+#define SUN4I_TCON1_FILL_END2_REG 0x320
+#define SUN4I_TCON1_FILL_DATA2_REG 0x324
+#define SUN4I_TCON1_GAMMA_TABLE_REG 0x400
+
+#define SUN4I_TCON_MAX_CHANNELS 2
+
+struct sun4i_tcon {
+ struct drm_device *drm;
+ struct regmap *regs;
+
+ /* Main bus clock */
+ struct clk *clk;
+
+ /* Clocks for the TCON channels */
+ struct clk *sclk0;
+ struct clk *sclk1;
+
+ /* Pixel clock */
+ struct clk *dclk;
+
+ /* Reset control */
+ struct reset_control *lcd_rst;
+
+ /* Platform adjustments */
+ bool has_mux;
+
+ struct drm_panel *panel;
+};
+
+/* Global Control */
+void sun4i_tcon_disable(struct sun4i_tcon *tcon);
+void sun4i_tcon_enable(struct sun4i_tcon *tcon);
+
+/* Channel Control */
+void sun4i_tcon_channel_disable(struct sun4i_tcon *tcon, int channel);
+void sun4i_tcon_channel_enable(struct sun4i_tcon *tcon, int channel);
+
+void sun4i_tcon_enable_vblank(struct sun4i_tcon *tcon, bool enable);
+
+/* Mode Related Controls */
+void sun4i_tcon_switch_interlace(struct sun4i_tcon *tcon,
+ bool enable);
+void sun4i_tcon0_mode_set(struct sun4i_tcon *tcon,
+ struct drm_display_mode *mode);
+void sun4i_tcon1_mode_set(struct sun4i_tcon *tcon,
+ struct drm_display_mode *mode);
+
+#endif /* __SUN4I_TCON_H__ */
diff --git a/drivers/gpu/drm/sun4i/sun4i_tv.c b/drivers/gpu/drm/sun4i/sun4i_tv.c
new file mode 100644
index 0000000..bc047f9
--- /dev/null
+++ b/drivers/gpu/drm/sun4i/sun4i_tv.c
@@ -0,0 +1,708 @@
+/*
+ * Copyright (C) 2015 Free Electrons
+ * Copyright (C) 2015 NextThing Co
+ *
+ * Maxime Ripard <maxime.ripard@free-electrons.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ */
+
+#include <linux/clk.h>
+#include <linux/component.h>
+#include <linux/of_address.h>
+#include <linux/regmap.h>
+#include <linux/reset.h>
+
+#include <drm/drmP.h>
+#include <drm/drm_atomic_helper.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/drm_panel.h>
+
+#include "sun4i_backend.h"
+#include "sun4i_drv.h"
+#include "sun4i_tcon.h"
+
+#define SUN4I_TVE_EN_REG 0x000
+#define SUN4I_TVE_EN_DAC_MAP_MASK GENMASK(19, 4)
+#define SUN4I_TVE_EN_DAC_MAP(dac, out) (((out) & 0xf) << (dac + 1) * 4)
+#define SUN4I_TVE_EN_ENABLE BIT(0)
+
+#define SUN4I_TVE_CFG0_REG 0x004
+#define SUN4I_TVE_CFG0_DAC_CONTROL_54M BIT(26)
+#define SUN4I_TVE_CFG0_CORE_DATAPATH_54M BIT(25)
+#define SUN4I_TVE_CFG0_CORE_CONTROL_54M BIT(24)
+#define SUN4I_TVE_CFG0_YC_EN BIT(17)
+#define SUN4I_TVE_CFG0_COMP_EN BIT(16)
+#define SUN4I_TVE_CFG0_RES(x) ((x) & 0xf)
+#define SUN4I_TVE_CFG0_RES_480i SUN4I_TVE_CFG0_RES(0)
+#define SUN4I_TVE_CFG0_RES_576i SUN4I_TVE_CFG0_RES(1)
+
+#define SUN4I_TVE_DAC0_REG 0x008
+#define SUN4I_TVE_DAC0_CLOCK_INVERT BIT(24)
+#define SUN4I_TVE_DAC0_LUMA(x) (((x) & 3) << 20)
+#define SUN4I_TVE_DAC0_LUMA_0_4 SUN4I_TVE_DAC0_LUMA(3)
+#define SUN4I_TVE_DAC0_CHROMA(x) (((x) & 3) << 18)
+#define SUN4I_TVE_DAC0_CHROMA_0_75 SUN4I_TVE_DAC0_CHROMA(3)
+#define SUN4I_TVE_DAC0_INTERNAL_DAC(x) (((x) & 3) << 16)
+#define SUN4I_TVE_DAC0_INTERNAL_DAC_37_5_OHMS SUN4I_TVE_DAC0_INTERNAL_DAC(3)
+#define SUN4I_TVE_DAC0_DAC_EN(dac) BIT(dac)
+
+#define SUN4I_TVE_NOTCH_REG 0x00c
+#define SUN4I_TVE_NOTCH_DAC0_TO_DAC_DLY(dac, x) ((4 - (x)) << (dac * 3))
+
+#define SUN4I_TVE_CHROMA_FREQ_REG 0x010
+
+#define SUN4I_TVE_PORCH_REG 0x014
+#define SUN4I_TVE_PORCH_BACK(x) ((x) << 16)
+#define SUN4I_TVE_PORCH_FRONT(x) (x)
+
+#define SUN4I_TVE_LINE_REG 0x01c
+#define SUN4I_TVE_LINE_FIRST(x) ((x) << 16)
+#define SUN4I_TVE_LINE_NUMBER(x) (x)
+
+#define SUN4I_TVE_LEVEL_REG 0x020
+#define SUN4I_TVE_LEVEL_BLANK(x) ((x) << 16)
+#define SUN4I_TVE_LEVEL_BLACK(x) (x)
+
+#define SUN4I_TVE_DAC1_REG 0x024
+#define SUN4I_TVE_DAC1_AMPLITUDE(dac, x) ((x) << (dac * 8))
+
+#define SUN4I_TVE_DETECT_STA_REG 0x038
+#define SUN4I_TVE_DETECT_STA_DAC(dac) BIT((dac * 8))
+#define SUN4I_TVE_DETECT_STA_UNCONNECTED 0
+#define SUN4I_TVE_DETECT_STA_CONNECTED 1
+#define SUN4I_TVE_DETECT_STA_GROUND 2
+
+#define SUN4I_TVE_CB_CR_LVL_REG 0x10c
+#define SUN4I_TVE_CB_CR_LVL_CR_BURST(x) ((x) << 8)
+#define SUN4I_TVE_CB_CR_LVL_CB_BURST(x) (x)
+
+#define SUN4I_TVE_TINT_BURST_PHASE_REG 0x110
+#define SUN4I_TVE_TINT_BURST_PHASE_CHROMA(x) (x)
+
+#define SUN4I_TVE_BURST_WIDTH_REG 0x114
+#define SUN4I_TVE_BURST_WIDTH_BREEZEWAY(x) ((x) << 16)
+#define SUN4I_TVE_BURST_WIDTH_BURST_WIDTH(x) ((x) << 8)
+#define SUN4I_TVE_BURST_WIDTH_HSYNC_WIDTH(x) (x)
+
+#define SUN4I_TVE_CB_CR_GAIN_REG 0x118
+#define SUN4I_TVE_CB_CR_GAIN_CR(x) ((x) << 8)
+#define SUN4I_TVE_CB_CR_GAIN_CB(x) (x)
+
+#define SUN4I_TVE_SYNC_VBI_REG 0x11c
+#define SUN4I_TVE_SYNC_VBI_SYNC(x) ((x) << 16)
+#define SUN4I_TVE_SYNC_VBI_VBLANK(x) (x)
+
+#define SUN4I_TVE_ACTIVE_LINE_REG 0x124
+#define SUN4I_TVE_ACTIVE_LINE(x) (x)
+
+#define SUN4I_TVE_CHROMA_REG 0x128
+#define SUN4I_TVE_CHROMA_COMP_GAIN(x) ((x) & 3)
+#define SUN4I_TVE_CHROMA_COMP_GAIN_50 SUN4I_TVE_CHROMA_COMP_GAIN(2)
+
+#define SUN4I_TVE_12C_REG 0x12c
+#define SUN4I_TVE_12C_NOTCH_WIDTH_WIDE BIT(8)
+#define SUN4I_TVE_12C_COMP_YUV_EN BIT(0)
+
+#define SUN4I_TVE_RESYNC_REG 0x130
+#define SUN4I_TVE_RESYNC_FIELD BIT(31)
+#define SUN4I_TVE_RESYNC_LINE(x) ((x) << 16)
+#define SUN4I_TVE_RESYNC_PIXEL(x) (x)
+
+#define SUN4I_TVE_SLAVE_REG 0x134
+
+#define SUN4I_TVE_WSS_DATA2_REG 0x244
+
+struct color_gains {
+ u16 cb;
+ u16 cr;
+};
+
+struct burst_levels {
+ u16 cb;
+ u16 cr;
+};
+
+struct video_levels {
+ u16 black;
+ u16 blank;
+};
+
+struct resync_parameters {
+ bool field;
+ u16 line;
+ u16 pixel;
+};
+
+struct tv_mode {
+ char *name;
+
+ u32 mode;
+ u32 chroma_freq;
+ u16 back_porch;
+ u16 front_porch;
+ u16 line_number;
+ u16 vblank_level;
+
+ u32 hdisplay;
+ u16 hfront_porch;
+ u16 hsync_len;
+ u16 hback_porch;
+
+ u32 vdisplay;
+ u16 vfront_porch;
+ u16 vsync_len;
+ u16 vback_porch;
+
+ bool yc_en;
+ bool dac3_en;
+ bool dac_bit25_en;
+
+ struct color_gains *color_gains;
+ struct burst_levels *burst_levels;
+ struct video_levels *video_levels;
+ struct resync_parameters *resync_params;
+};
+
+struct sun4i_tv {
+ struct drm_connector connector;
+ struct drm_encoder encoder;
+
+ struct clk *clk;
+ struct regmap *regs;
+ struct reset_control *reset;
+
+ struct sun4i_drv *drv;
+};
+
+struct video_levels ntsc_video_levels = {
+ .black = 282, .blank = 240,
+};
+
+struct video_levels pal_video_levels = {
+ .black = 252, .blank = 252,
+};
+
+struct burst_levels ntsc_burst_levels = {
+ .cb = 79, .cr = 0,
+};
+
+struct burst_levels pal_burst_levels = {
+ .cb = 40, .cr = 40,
+};
+
+struct color_gains ntsc_color_gains = {
+ .cb = 160, .cr = 160,
+};
+
+struct color_gains pal_color_gains = {
+ .cb = 224, .cr = 224,
+};
+
+struct resync_parameters ntsc_resync_parameters = {
+ .field = false, .line = 14, .pixel = 12,
+};
+
+struct resync_parameters pal_resync_parameters = {
+ .field = true, .line = 13, .pixel = 12,
+};
+
+struct tv_mode tv_modes[] = {
+ {
+ .name = "NTSC",
+ .mode = SUN4I_TVE_CFG0_RES_480i,
+ .chroma_freq = 0x21f07c1f,
+ .yc_en = true,
+ .dac3_en = true,
+ .dac_bit25_en = true,
+
+ .back_porch = 118,
+ .front_porch = 32,
+ .line_number = 525,
+
+ .hdisplay = 720,
+ .hfront_porch = 18,
+ .hsync_len = 2,
+ .hback_porch = 118,
+
+ .vdisplay = 480,
+ .vfront_porch = 26,
+ .vsync_len = 2,
+ .vback_porch = 17,
+
+ .vblank_level = 240,
+
+ .color_gains = &ntsc_color_gains,
+ .burst_levels = &ntsc_burst_levels,
+ .video_levels = &ntsc_video_levels,
+ .resync_params = &ntsc_resync_parameters,
+ },
+ {
+ .name = "PAL",
+ .mode = SUN4I_TVE_CFG0_RES_576i,
+ .chroma_freq = 0x2a098acb,
+
+ .back_porch = 138,
+ .front_porch = 24,
+ .line_number = 625,
+
+ .hdisplay = 720,
+ .hfront_porch = 3,
+ .hsync_len = 2,
+ .hback_porch = 139,
+
+ .vdisplay = 576,
+ .vfront_porch = 28,
+ .vsync_len = 2,
+ .vback_porch = 19,
+
+ .vblank_level = 252,
+
+ .color_gains = &pal_color_gains,
+ .burst_levels = &pal_burst_levels,
+ .video_levels = &pal_video_levels,
+ .resync_params = &pal_resync_parameters,
+ },
+};
+
+static inline struct sun4i_tv *
+drm_encoder_to_sun4i_tv(struct drm_encoder *encoder)
+{
+ return container_of(encoder, struct sun4i_tv,
+ encoder);
+}
+
+static inline struct sun4i_tv *
+drm_connector_to_sun4i_tv(struct drm_connector *connector)
+{
+ return container_of(connector, struct sun4i_tv,
+ connector);
+}
+
+/*
+ * FIXME: If only the drm_display_mode private field was usable, this
+ * could go away...
+ *
+ * So far, it doesn't seem to be preserved when the mode is passed by
+ * to mode_set for some reason.
+ */
+static struct tv_mode *sun4i_tv_find_tv_by_mode(struct drm_display_mode *mode)
+{
+ int i;
+
+ /* First try to identify the mode by name */
+ for (i = 0; i < ARRAY_SIZE(tv_modes); i++) {
+ struct tv_mode *tv_mode = &tv_modes[i];
+
+ DRM_DEBUG_DRIVER("Comparing mode %s vs %s",
+ mode->name, tv_mode->name);
+
+ if (!strcmp(mode->name, tv_mode->name))
+ return tv_mode;
+ }
+
+ /* Then by number of lines */
+ for (i = 0; i < ARRAY_SIZE(tv_modes); i++) {
+ struct tv_mode *tv_mode = &tv_modes[i];
+
+ DRM_DEBUG_DRIVER("Comparing mode %s vs %s (X: %d vs %d)",
+ mode->name, tv_mode->name,
+ mode->vdisplay, tv_mode->vdisplay);
+
+ if (mode->vdisplay == tv_mode->vdisplay)
+ return tv_mode;
+ }
+
+ return NULL;
+}
+
+static void sun4i_tv_mode_to_drm_mode(struct tv_mode *tv_mode,
+ struct drm_display_mode *mode)
+{
+ DRM_DEBUG_DRIVER("Creating mode %s\n", mode->name);
+
+ mode->type = DRM_MODE_TYPE_DRIVER;
+ mode->clock = 13500;
+ mode->flags = DRM_MODE_FLAG_INTERLACE;
+
+ mode->hdisplay = tv_mode->hdisplay;
+ mode->hsync_start = mode->hdisplay + tv_mode->hfront_porch;
+ mode->hsync_end = mode->hsync_start + tv_mode->hsync_len;
+ mode->htotal = mode->hsync_end + tv_mode->hback_porch;
+
+ mode->vdisplay = tv_mode->vdisplay;
+ mode->vsync_start = mode->vdisplay + tv_mode->vfront_porch;
+ mode->vsync_end = mode->vsync_start + tv_mode->vsync_len;
+ mode->vtotal = mode->vsync_end + tv_mode->vback_porch;
+}
+
+static int sun4i_tv_atomic_check(struct drm_encoder *encoder,
+ struct drm_crtc_state *crtc_state,
+ struct drm_connector_state *conn_state)
+{
+ return 0;
+}
+
+static void sun4i_tv_disable(struct drm_encoder *encoder)
+{
+ struct sun4i_tv *tv = drm_encoder_to_sun4i_tv(encoder);
+ struct sun4i_drv *drv = tv->drv;
+ struct sun4i_tcon *tcon = drv->tcon;
+
+ DRM_DEBUG_DRIVER("Disabling the TV Output\n");
+
+ sun4i_tcon_channel_disable(tcon, 1);
+
+ regmap_update_bits(tv->regs, SUN4I_TVE_EN_REG,
+ SUN4I_TVE_EN_ENABLE,
+ 0);
+ sun4i_backend_disable_color_correction(drv->backend);
+}
+
+static void sun4i_tv_enable(struct drm_encoder *encoder)
+{
+ struct sun4i_tv *tv = drm_encoder_to_sun4i_tv(encoder);
+ struct sun4i_drv *drv = tv->drv;
+ struct sun4i_tcon *tcon = drv->tcon;
+
+ DRM_DEBUG_DRIVER("Enabling the TV Output\n");
+
+ sun4i_backend_apply_color_correction(drv->backend);
+
+ regmap_update_bits(tv->regs, SUN4I_TVE_EN_REG,
+ SUN4I_TVE_EN_ENABLE,
+ SUN4I_TVE_EN_ENABLE);
+
+ sun4i_tcon_channel_enable(tcon, 1);
+}
+
+static void sun4i_tv_mode_set(struct drm_encoder *encoder,
+ struct drm_display_mode *mode,
+ struct drm_display_mode *adjusted_mode)
+{
+ struct sun4i_tv *tv = drm_encoder_to_sun4i_tv(encoder);
+ struct sun4i_drv *drv = tv->drv;
+ struct sun4i_tcon *tcon = drv->tcon;
+ struct tv_mode *tv_mode = sun4i_tv_find_tv_by_mode(mode);
+
+ sun4i_tcon1_mode_set(tcon, mode);
+
+ /* Enable and map the DAC to the output */
+ regmap_update_bits(tv->regs, SUN4I_TVE_EN_REG,
+ SUN4I_TVE_EN_DAC_MAP_MASK,
+ SUN4I_TVE_EN_DAC_MAP(0, 1) |
+ SUN4I_TVE_EN_DAC_MAP(1, 2) |
+ SUN4I_TVE_EN_DAC_MAP(2, 3) |
+ SUN4I_TVE_EN_DAC_MAP(3, 4));
+
+ /* Set PAL settings */
+ regmap_write(tv->regs, SUN4I_TVE_CFG0_REG,
+ tv_mode->mode |
+ (tv_mode->yc_en ? SUN4I_TVE_CFG0_YC_EN : 0) |
+ SUN4I_TVE_CFG0_COMP_EN |
+ SUN4I_TVE_CFG0_DAC_CONTROL_54M |
+ SUN4I_TVE_CFG0_CORE_DATAPATH_54M |
+ SUN4I_TVE_CFG0_CORE_CONTROL_54M);
+
+ /* Configure the DAC for a composite output */
+ regmap_write(tv->regs, SUN4I_TVE_DAC0_REG,
+ SUN4I_TVE_DAC0_DAC_EN(0) |
+ (tv_mode->dac3_en ? SUN4I_TVE_DAC0_DAC_EN(3) : 0) |
+ SUN4I_TVE_DAC0_INTERNAL_DAC_37_5_OHMS |
+ SUN4I_TVE_DAC0_CHROMA_0_75 |
+ SUN4I_TVE_DAC0_LUMA_0_4 |
+ SUN4I_TVE_DAC0_CLOCK_INVERT |
+ (tv_mode->dac_bit25_en ? BIT(25) : 0) |
+ BIT(30));
+
+ /* Configure the sample delay between DAC0 and the other DAC */
+ regmap_write(tv->regs, SUN4I_TVE_NOTCH_REG,
+ SUN4I_TVE_NOTCH_DAC0_TO_DAC_DLY(1, 0) |
+ SUN4I_TVE_NOTCH_DAC0_TO_DAC_DLY(2, 0));
+
+ regmap_write(tv->regs, SUN4I_TVE_CHROMA_FREQ_REG,
+ tv_mode->chroma_freq);
+
+ /* Set the front and back porch */
+ regmap_write(tv->regs, SUN4I_TVE_PORCH_REG,
+ SUN4I_TVE_PORCH_BACK(tv_mode->back_porch) |
+ SUN4I_TVE_PORCH_FRONT(tv_mode->front_porch));
+
+ /* Set the lines setup */
+ regmap_write(tv->regs, SUN4I_TVE_LINE_REG,
+ SUN4I_TVE_LINE_FIRST(22) |
+ SUN4I_TVE_LINE_NUMBER(tv_mode->line_number));
+
+ regmap_write(tv->regs, SUN4I_TVE_LEVEL_REG,
+ SUN4I_TVE_LEVEL_BLANK(tv_mode->video_levels->blank) |
+ SUN4I_TVE_LEVEL_BLACK(tv_mode->video_levels->black));
+
+ regmap_write(tv->regs, SUN4I_TVE_DAC1_REG,
+ SUN4I_TVE_DAC1_AMPLITUDE(0, 0x18) |
+ SUN4I_TVE_DAC1_AMPLITUDE(1, 0x18) |
+ SUN4I_TVE_DAC1_AMPLITUDE(2, 0x18) |
+ SUN4I_TVE_DAC1_AMPLITUDE(3, 0x18));
+
+ regmap_write(tv->regs, SUN4I_TVE_CB_CR_LVL_REG,
+ SUN4I_TVE_CB_CR_LVL_CB_BURST(tv_mode->burst_levels->cb) |
+ SUN4I_TVE_CB_CR_LVL_CR_BURST(tv_mode->burst_levels->cr));
+
+ /* Set burst width for a composite output */
+ regmap_write(tv->regs, SUN4I_TVE_BURST_WIDTH_REG,
+ SUN4I_TVE_BURST_WIDTH_HSYNC_WIDTH(126) |
+ SUN4I_TVE_BURST_WIDTH_BURST_WIDTH(68) |
+ SUN4I_TVE_BURST_WIDTH_BREEZEWAY(22));
+
+ regmap_write(tv->regs, SUN4I_TVE_CB_CR_GAIN_REG,
+ SUN4I_TVE_CB_CR_GAIN_CB(tv_mode->color_gains->cb) |
+ SUN4I_TVE_CB_CR_GAIN_CR(tv_mode->color_gains->cr));
+
+ regmap_write(tv->regs, SUN4I_TVE_SYNC_VBI_REG,
+ SUN4I_TVE_SYNC_VBI_SYNC(0x10) |
+ SUN4I_TVE_SYNC_VBI_VBLANK(tv_mode->vblank_level));
+
+ regmap_write(tv->regs, SUN4I_TVE_ACTIVE_LINE_REG,
+ SUN4I_TVE_ACTIVE_LINE(1440));
+
+ /* Set composite chroma gain to 50 % */
+ regmap_write(tv->regs, SUN4I_TVE_CHROMA_REG,
+ SUN4I_TVE_CHROMA_COMP_GAIN_50);
+
+ regmap_write(tv->regs, SUN4I_TVE_12C_REG,
+ SUN4I_TVE_12C_COMP_YUV_EN |
+ SUN4I_TVE_12C_NOTCH_WIDTH_WIDE);
+
+ regmap_write(tv->regs, SUN4I_TVE_RESYNC_REG,
+ SUN4I_TVE_RESYNC_PIXEL(tv_mode->resync_params->pixel) |
+ SUN4I_TVE_RESYNC_LINE(tv_mode->resync_params->line) |
+ (tv_mode->resync_params->field ?
+ SUN4I_TVE_RESYNC_FIELD : 0));
+
+ regmap_write(tv->regs, SUN4I_TVE_SLAVE_REG, 0);
+
+ clk_set_rate(tcon->sclk1, mode->crtc_clock * 1000);
+}
+
+static struct drm_encoder_helper_funcs sun4i_tv_helper_funcs = {
+ .atomic_check = sun4i_tv_atomic_check,
+ .disable = sun4i_tv_disable,
+ .enable = sun4i_tv_enable,
+ .mode_set = sun4i_tv_mode_set,
+};
+
+static void sun4i_tv_destroy(struct drm_encoder *encoder)
+{
+ drm_encoder_cleanup(encoder);
+}
+
+static struct drm_encoder_funcs sun4i_tv_funcs = {
+ .destroy = sun4i_tv_destroy,
+};
+
+static int sun4i_tv_comp_get_modes(struct drm_connector *connector)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(tv_modes); i++) {
+ struct drm_display_mode *mode = drm_mode_create(connector->dev);
+ struct tv_mode *tv_mode = &tv_modes[i];
+
+ strcpy(mode->name, tv_mode->name);
+
+ sun4i_tv_mode_to_drm_mode(tv_mode, mode);
+ drm_mode_probed_add(connector, mode);
+ }
+
+ return i;
+}
+
+static int sun4i_tv_comp_mode_valid(struct drm_connector *connector,
+ struct drm_display_mode *mode)
+{
+ /* TODO */
+ return MODE_OK;
+}
+
+static struct drm_encoder *
+sun4i_tv_comp_best_encoder(struct drm_connector *connector)
+{
+ struct sun4i_tv *tv = drm_connector_to_sun4i_tv(connector);
+
+ return &tv->encoder;
+}
+
+static struct drm_connector_helper_funcs sun4i_tv_comp_connector_helper_funcs = {
+ .get_modes = sun4i_tv_comp_get_modes,
+ .mode_valid = sun4i_tv_comp_mode_valid,
+ .best_encoder = sun4i_tv_comp_best_encoder,
+};
+
+static enum drm_connector_status
+sun4i_tv_comp_connector_detect(struct drm_connector *connector, bool force)
+{
+ return connector_status_connected;
+}
+
+static void
+sun4i_tv_comp_connector_destroy(struct drm_connector *connector)
+{
+ drm_connector_cleanup(connector);
+}
+
+static struct drm_connector_funcs sun4i_tv_comp_connector_funcs = {
+ .dpms = drm_atomic_helper_connector_dpms,
+ .detect = sun4i_tv_comp_connector_detect,
+ .fill_modes = drm_helper_probe_single_connector_modes,
+ .destroy = sun4i_tv_comp_connector_destroy,
+ .reset = drm_atomic_helper_connector_reset,
+ .atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
+};
+
+static struct regmap_config sun4i_tv_regmap_config = {
+ .reg_bits = 32,
+ .val_bits = 32,
+ .reg_stride = 4,
+ .max_register = SUN4I_TVE_WSS_DATA2_REG,
+ .name = "tv-encoder",
+};
+
+static int sun4i_tv_bind(struct device *dev, struct device *master,
+ void *data)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct drm_device *drm = data;
+ struct sun4i_drv *drv = drm->dev_private;
+ struct sun4i_tv *tv;
+ struct resource *res;
+ void __iomem *regs;
+ int ret;
+
+ tv = devm_kzalloc(dev, sizeof(*tv), GFP_KERNEL);
+ if (!tv)
+ return -ENOMEM;
+ tv->drv = drv;
+ dev_set_drvdata(dev, tv);
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ regs = devm_ioremap_resource(dev, res);
+ if (IS_ERR(regs)) {
+ dev_err(dev, "Couldn't map the TV encoder registers\n");
+ return PTR_ERR(regs);
+ }
+
+ tv->regs = devm_regmap_init_mmio(dev, regs,
+ &sun4i_tv_regmap_config);
+ if (IS_ERR(tv->regs)) {
+ dev_err(dev, "Couldn't create the TV encoder regmap\n");
+ return PTR_ERR(tv->regs);
+ }
+
+ tv->reset = devm_reset_control_get(dev, NULL);
+ if (IS_ERR(tv->reset)) {
+ dev_err(dev, "Couldn't get our reset line\n");
+ return PTR_ERR(tv->reset);
+ }
+
+ ret = reset_control_deassert(tv->reset);
+ if (ret) {
+ dev_err(dev, "Couldn't deassert our reset line\n");
+ return ret;
+ }
+
+ tv->clk = devm_clk_get(dev, NULL);
+ if (IS_ERR(tv->clk)) {
+ dev_err(dev, "Couldn't get the TV encoder clock\n");
+ ret = PTR_ERR(tv->clk);
+ goto err_assert_reset;
+ }
+ clk_prepare_enable(tv->clk);
+
+ drm_encoder_helper_add(&tv->encoder,
+ &sun4i_tv_helper_funcs);
+ ret = drm_encoder_init(drm,
+ &tv->encoder,
+ &sun4i_tv_funcs,
+ DRM_MODE_ENCODER_TVDAC,
+ NULL);
+ if (ret) {
+ dev_err(dev, "Couldn't initialise the TV encoder\n");
+ goto err_disable_clk;
+ }
+
+ tv->encoder.possible_crtcs = BIT(0);
+
+ drm_connector_helper_add(&tv->connector,
+ &sun4i_tv_comp_connector_helper_funcs);
+ ret = drm_connector_init(drm, &tv->connector,
+ &sun4i_tv_comp_connector_funcs,
+ DRM_MODE_CONNECTOR_Composite);
+ if (ret) {
+ dev_err(dev,
+ "Couldn't initialise the Composite connector\n");
+ goto err_cleanup_connector;
+ }
+ tv->connector.interlace_allowed = true;
+
+ drm_mode_connector_attach_encoder(&tv->connector, &tv->encoder);
+
+ return 0;
+
+err_cleanup_connector:
+ drm_encoder_cleanup(&tv->encoder);
+err_disable_clk:
+ clk_disable_unprepare(tv->clk);
+err_assert_reset:
+ reset_control_assert(tv->reset);
+ return ret;
+}
+
+static void sun4i_tv_unbind(struct device *dev, struct device *master,
+ void *data)
+{
+ struct sun4i_tv *tv = dev_get_drvdata(dev);
+
+ drm_connector_cleanup(&tv->connector);
+ drm_encoder_cleanup(&tv->encoder);
+ clk_disable_unprepare(tv->clk);
+}
+
+static struct component_ops sun4i_tv_ops = {
+ .bind = sun4i_tv_bind,
+ .unbind = sun4i_tv_unbind,
+};
+
+static int sun4i_tv_probe(struct platform_device *pdev)
+{
+ return component_add(&pdev->dev, &sun4i_tv_ops);
+}
+
+static int sun4i_tv_remove(struct platform_device *pdev)
+{
+ component_del(&pdev->dev, &sun4i_tv_ops);
+
+ return 0;
+}
+
+static const struct of_device_id sun4i_tv_of_table[] = {
+ { .compatible = "allwinner,sun4i-a10-tv-encoder" },
+ { }
+};
+MODULE_DEVICE_TABLE(of, sun4i_tv_of_table);
+
+static struct platform_driver sun4i_tv_platform_driver = {
+ .probe = sun4i_tv_probe,
+ .remove = sun4i_tv_remove,
+ .driver = {
+ .name = "sun4i-tve",
+ .of_match_table = sun4i_tv_of_table,
+ },
+};
+module_platform_driver(sun4i_tv_platform_driver);
+
+MODULE_AUTHOR("Maxime Ripard <maxime.ripard@free-electrons.com>");
+MODULE_DESCRIPTION("Allwinner A10 TV Encoder Driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/gpu/drm/tegra/dc.c b/drivers/gpu/drm/tegra/dc.c
index fb2b4b0..39940f5 100644
--- a/drivers/gpu/drm/tegra/dc.c
+++ b/drivers/gpu/drm/tegra/dc.c
@@ -434,7 +434,7 @@
struct tegra_plane_state *state;
if (plane->state)
- __drm_atomic_helper_plane_destroy_state(plane, plane->state);
+ __drm_atomic_helper_plane_destroy_state(plane->state);
kfree(plane->state);
plane->state = NULL;
@@ -466,7 +466,7 @@
static void tegra_plane_atomic_destroy_state(struct drm_plane *plane,
struct drm_plane_state *state)
{
- __drm_atomic_helper_plane_destroy_state(plane, state);
+ __drm_atomic_helper_plane_destroy_state(state);
kfree(state);
}
@@ -998,7 +998,7 @@
struct tegra_dc_state *state;
if (crtc->state)
- __drm_atomic_helper_crtc_destroy_state(crtc, crtc->state);
+ __drm_atomic_helper_crtc_destroy_state(crtc->state);
kfree(crtc->state);
crtc->state = NULL;
@@ -1034,7 +1034,7 @@
static void tegra_crtc_atomic_destroy_state(struct drm_crtc *crtc,
struct drm_crtc_state *state)
{
- __drm_atomic_helper_crtc_destroy_state(crtc, state);
+ __drm_atomic_helper_crtc_destroy_state(state);
kfree(state);
}
@@ -1722,7 +1722,6 @@
if (err < 0)
goto cleanup;
- drm_mode_crtc_set_gamma_size(&dc->base, 256);
drm_crtc_helper_add(&dc->base, &tegra_crtc_helper_funcs);
/*
diff --git a/drivers/gpu/drm/tegra/drm.c b/drivers/gpu/drm/tegra/drm.c
index 8e6b18c..b59c3bf0 100644
--- a/drivers/gpu/drm/tegra/drm.c
+++ b/drivers/gpu/drm/tegra/drm.c
@@ -74,7 +74,7 @@
}
static int tegra_atomic_commit(struct drm_device *drm,
- struct drm_atomic_state *state, bool async)
+ struct drm_atomic_state *state, bool nonblock)
{
struct tegra_drm *tegra = drm->dev_private;
int err;
@@ -83,7 +83,7 @@
if (err)
return err;
- /* serialize outstanding asynchronous commits */
+ /* serialize outstanding nonblocking commits */
mutex_lock(&tegra->commit.lock);
flush_work(&tegra->commit.work);
@@ -95,7 +95,7 @@
drm_atomic_helper_swap_state(drm, state);
- if (async)
+ if (nonblock)
tegra_atomic_schedule(tegra, state);
else
tegra_atomic_complete(tegra, state);
@@ -180,7 +180,6 @@
/* syncpoints are used for full 32-bit hardware VBLANK counters */
drm->max_vblank_count = 0xffffffff;
- drm->vblank_disable_allowed = true;
err = drm_vblank_init(drm, drm->mode_config.num_crtc);
if (err < 0)
@@ -268,12 +267,12 @@
}
static struct host1x_bo *
-host1x_bo_lookup(struct drm_device *drm, struct drm_file *file, u32 handle)
+host1x_bo_lookup(struct drm_file *file, u32 handle)
{
struct drm_gem_object *gem;
struct tegra_bo *bo;
- gem = drm_gem_object_lookup(drm, file, handle);
+ gem = drm_gem_object_lookup(file, handle);
if (!gem)
return NULL;
@@ -311,11 +310,11 @@
if (err < 0)
return err;
- dest->cmdbuf.bo = host1x_bo_lookup(drm, file, cmdbuf);
+ dest->cmdbuf.bo = host1x_bo_lookup(file, cmdbuf);
if (!dest->cmdbuf.bo)
return -ENOENT;
- dest->target.bo = host1x_bo_lookup(drm, file, target);
+ dest->target.bo = host1x_bo_lookup(file, target);
if (!dest->target.bo)
return -ENOENT;
@@ -363,7 +362,7 @@
goto fail;
}
- bo = host1x_bo_lookup(drm, file, cmdbuf.handle);
+ bo = host1x_bo_lookup(file, cmdbuf.handle);
if (!bo) {
err = -ENOENT;
goto fail;
@@ -463,7 +462,7 @@
struct drm_gem_object *gem;
struct tegra_bo *bo;
- gem = drm_gem_object_lookup(drm, file, args->handle);
+ gem = drm_gem_object_lookup(file, args->handle);
if (!gem)
return -EINVAL;
@@ -672,7 +671,7 @@
return -EINVAL;
}
- gem = drm_gem_object_lookup(drm, file, args->handle);
+ gem = drm_gem_object_lookup(file, args->handle);
if (!gem)
return -ENOENT;
@@ -694,7 +693,7 @@
struct tegra_bo *bo;
int err = 0;
- gem = drm_gem_object_lookup(drm, file, args->handle);
+ gem = drm_gem_object_lookup(file, args->handle);
if (!gem)
return -ENOENT;
@@ -736,7 +735,7 @@
if (args->flags & ~DRM_TEGRA_GEM_FLAGS)
return -EINVAL;
- gem = drm_gem_object_lookup(drm, file, args->handle);
+ gem = drm_gem_object_lookup(file, args->handle);
if (!gem)
return -ENOENT;
@@ -758,7 +757,7 @@
struct drm_gem_object *gem;
struct tegra_bo *bo;
- gem = drm_gem_object_lookup(drm, file, args->handle);
+ gem = drm_gem_object_lookup(file, args->handle);
if (!gem)
return -ENOENT;
@@ -878,7 +877,7 @@
seq_printf(s, "%3d: user size: %d x %d, depth %d, %d bpp, refcount %d\n",
fb->base.id, fb->width, fb->height, fb->depth,
fb->bits_per_pixel,
- atomic_read(&fb->refcount.refcount));
+ drm_framebuffer_read_refcount(fb));
}
mutex_unlock(&drm->mode_config.fb_lock);
@@ -932,7 +931,7 @@
.debugfs_cleanup = tegra_debugfs_cleanup,
#endif
- .gem_free_object = tegra_bo_free_object,
+ .gem_free_object_unlocked = tegra_bo_free_object,
.gem_vm_ops = &tegra_bo_vm_ops,
.prime_handle_to_fd = drm_gem_prime_handle_to_fd,
diff --git a/drivers/gpu/drm/tegra/dsi.c b/drivers/gpu/drm/tegra/dsi.c
index 44e1027..d1239eb 100644
--- a/drivers/gpu/drm/tegra/dsi.c
+++ b/drivers/gpu/drm/tegra/dsi.c
@@ -745,13 +745,17 @@
static void tegra_dsi_connector_reset(struct drm_connector *connector)
{
- struct tegra_dsi_state *state =
- kzalloc(sizeof(*state), GFP_KERNEL);
+ struct tegra_dsi_state *state = kzalloc(sizeof(*state), GFP_KERNEL);
- if (state) {
+ if (!state)
+ return;
+
+ if (connector->state) {
+ __drm_atomic_helper_connector_destroy_state(connector->state);
kfree(connector->state);
- __drm_atomic_helper_connector_reset(connector, &state->base);
}
+
+ __drm_atomic_helper_connector_reset(connector, &state->base);
}
static struct drm_connector_state *
@@ -764,6 +768,9 @@
if (!copy)
return NULL;
+ __drm_atomic_helper_connector_duplicate_state(connector,
+ ©->base);
+
return ©->base;
}
diff --git a/drivers/gpu/drm/tegra/fb.c b/drivers/gpu/drm/tegra/fb.c
index ca84de9..1b12aa7 100644
--- a/drivers/gpu/drm/tegra/fb.c
+++ b/drivers/gpu/drm/tegra/fb.c
@@ -149,7 +149,7 @@
unsigned int height = cmd->height / (i ? vsub : 1);
unsigned int size, bpp;
- gem = drm_gem_object_lookup(drm, file, cmd->handles[i]);
+ gem = drm_gem_object_lookup(file, cmd->handles[i]);
if (!gem) {
err = -ENXIO;
goto unreference;
diff --git a/drivers/gpu/drm/tegra/gem.c b/drivers/gpu/drm/tegra/gem.c
index 3b0d8c3..aa60d99 100644
--- a/drivers/gpu/drm/tegra/gem.c
+++ b/drivers/gpu/drm/tegra/gem.c
@@ -401,7 +401,7 @@
struct drm_gem_object *gem;
struct tegra_bo *bo;
- gem = drm_gem_object_lookup(drm, file, handle);
+ gem = drm_gem_object_lookup(file, handle);
if (!gem) {
dev_err(drm->dev, "failed to lookup GEM object\n");
return -EINVAL;
diff --git a/drivers/gpu/drm/tilcdc/tilcdc_crtc.c b/drivers/gpu/drm/tilcdc/tilcdc_crtc.c
index 051e5e1..79027b1 100644
--- a/drivers/gpu/drm/tilcdc/tilcdc_crtc.c
+++ b/drivers/gpu/drm/tilcdc/tilcdc_crtc.c
@@ -707,7 +707,7 @@
event = tilcdc_crtc->event;
tilcdc_crtc->event = NULL;
if (event)
- drm_send_vblank_event(dev, 0, event);
+ drm_crtc_send_vblank_event(crtc, event);
spin_unlock_irqrestore(&dev->event_lock, flags);
}
diff --git a/drivers/gpu/drm/tilcdc/tilcdc_slave_compat.c b/drivers/gpu/drm/tilcdc/tilcdc_slave_compat.c
index 106679b..f9c79da 100644
--- a/drivers/gpu/drm/tilcdc/tilcdc_slave_compat.c
+++ b/drivers/gpu/drm/tilcdc/tilcdc_slave_compat.c
@@ -157,7 +157,7 @@
if (!overlay_data || kfree_table_add(kft, overlay_data))
return NULL;
- of_fdt_unflatten_tree(overlay_data, &overlay);
+ of_fdt_unflatten_tree(overlay_data, NULL, &overlay);
if (!overlay) {
pr_warn("%s: Unfattening overlay tree failed\n", __func__);
return NULL;
diff --git a/drivers/gpu/drm/ttm/Makefile b/drivers/gpu/drm/ttm/Makefile
index b433b9f..f923258 100644
--- a/drivers/gpu/drm/ttm/Makefile
+++ b/drivers/gpu/drm/ttm/Makefile
@@ -2,9 +2,10 @@
# Makefile for the drm device driver. This driver provides support for the
ccflags-y := -Iinclude/drm
-ttm-y := ttm_agp_backend.o ttm_memory.o ttm_tt.o ttm_bo.o \
+ttm-y := ttm_memory.o ttm_tt.o ttm_bo.o \
ttm_bo_util.o ttm_bo_vm.o ttm_module.o \
ttm_object.o ttm_lock.o ttm_execbuf_util.o ttm_page_alloc.o \
ttm_bo_manager.o ttm_page_alloc_dma.o
+ttm-$(CONFIG_AGP) += ttm_agp_backend.o
obj-$(CONFIG_DRM_TTM) += ttm.o
diff --git a/drivers/gpu/drm/ttm/ttm_agp_backend.c b/drivers/gpu/drm/ttm/ttm_agp_backend.c
index 764be36..028ab60 100644
--- a/drivers/gpu/drm/ttm/ttm_agp_backend.c
+++ b/drivers/gpu/drm/ttm/ttm_agp_backend.c
@@ -34,7 +34,6 @@
#include <drm/ttm/ttm_module.h>
#include <drm/ttm/ttm_bo_driver.h>
#include <drm/ttm/ttm_page_alloc.h>
-#ifdef TTM_HAS_AGP
#include <drm/ttm/ttm_placement.h>
#include <linux/agp_backend.h>
#include <linux/module.h>
@@ -148,5 +147,3 @@
ttm_pool_unpopulate(ttm);
}
EXPORT_SYMBOL(ttm_agp_tt_unpopulate);
-
-#endif
diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c
index e3daafa..39386f5 100644
--- a/drivers/gpu/drm/ttm/ttm_bo.c
+++ b/drivers/gpu/drm/ttm/ttm_bo.c
@@ -164,7 +164,6 @@
void ttm_bo_add_to_lru(struct ttm_buffer_object *bo)
{
struct ttm_bo_device *bdev = bo->bdev;
- struct ttm_mem_type_manager *man;
lockdep_assert_held(&bo->resv->lock.base);
@@ -172,12 +171,11 @@
BUG_ON(!list_empty(&bo->lru));
- man = &bdev->man[bo->mem.mem_type];
- list_add_tail(&bo->lru, &man->lru);
+ list_add(&bo->lru, bdev->driver->lru_tail(bo));
kref_get(&bo->list_kref);
if (bo->ttm && !(bo->ttm->page_flags & TTM_PAGE_FLAG_SG)) {
- list_add_tail(&bo->swap, &bo->glob->swap_lru);
+ list_add(&bo->swap, bdev->driver->swap_lru_tail(bo));
kref_get(&bo->list_kref);
}
}
@@ -186,8 +184,12 @@
int ttm_bo_del_from_lru(struct ttm_buffer_object *bo)
{
+ struct ttm_bo_device *bdev = bo->bdev;
int put_count = 0;
+ if (bdev->driver->lru_removal)
+ bdev->driver->lru_removal(bo);
+
if (!list_empty(&bo->swap)) {
list_del_init(&bo->swap);
++put_count;
@@ -197,11 +199,6 @@
++put_count;
}
- /*
- * TODO: Add a driver hook to delete from
- * driver-specific LRU's here.
- */
-
return put_count;
}
@@ -230,16 +227,32 @@
void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo)
{
+ struct ttm_bo_device *bdev = bo->bdev;
int put_count = 0;
lockdep_assert_held(&bo->resv->lock.base);
+ if (bdev->driver->lru_removal)
+ bdev->driver->lru_removal(bo);
+
put_count = ttm_bo_del_from_lru(bo);
ttm_bo_list_ref_sub(bo, put_count, true);
ttm_bo_add_to_lru(bo);
}
EXPORT_SYMBOL(ttm_bo_move_to_lru_tail);
+struct list_head *ttm_bo_default_lru_tail(struct ttm_buffer_object *bo)
+{
+ return bo->bdev->man[bo->mem.mem_type].lru.prev;
+}
+EXPORT_SYMBOL(ttm_bo_default_lru_tail);
+
+struct list_head *ttm_bo_default_swap_lru_tail(struct ttm_buffer_object *bo)
+{
+ return bo->glob->swap_lru.prev;
+}
+EXPORT_SYMBOL(ttm_bo_default_swap_lru_tail);
+
/*
* Call bo->mutex locked.
*/
@@ -443,10 +456,10 @@
int ret;
spin_lock(&glob->lru_lock);
- ret = __ttm_bo_reserve(bo, false, true, false, NULL);
+ ret = __ttm_bo_reserve(bo, false, true, NULL);
if (!ret) {
- if (!ttm_bo_wait(bo, false, false, true)) {
+ if (!ttm_bo_wait(bo, false, true)) {
put_count = ttm_bo_del_from_lru(bo);
spin_unlock(&glob->lru_lock);
@@ -499,7 +512,7 @@
int put_count;
int ret;
- ret = ttm_bo_wait(bo, false, false, true);
+ ret = ttm_bo_wait(bo, false, true);
if (ret && !no_wait_gpu) {
long lret;
@@ -517,7 +530,7 @@
return -EBUSY;
spin_lock(&glob->lru_lock);
- ret = __ttm_bo_reserve(bo, false, true, false, NULL);
+ ret = __ttm_bo_reserve(bo, false, true, NULL);
/*
* We raced, and lost, someone else holds the reservation now,
@@ -536,7 +549,7 @@
* remove sync_obj with ttm_bo_wait, the wait should be
* finished, and no new wait object should have been added.
*/
- ret = ttm_bo_wait(bo, false, false, true);
+ ret = ttm_bo_wait(bo, false, true);
WARN_ON(ret);
}
@@ -586,11 +599,10 @@
kref_get(&nentry->list_kref);
}
- ret = __ttm_bo_reserve(entry, false, true, false, NULL);
+ ret = __ttm_bo_reserve(entry, false, true, NULL);
if (remove_all && ret) {
spin_unlock(&glob->lru_lock);
- ret = __ttm_bo_reserve(entry, false, false,
- false, NULL);
+ ret = __ttm_bo_reserve(entry, false, false, NULL);
spin_lock(&glob->lru_lock);
}
@@ -676,7 +688,7 @@
struct ttm_placement placement;
int ret = 0;
- ret = ttm_bo_wait(bo, false, interruptible, no_wait_gpu);
+ ret = ttm_bo_wait(bo, interruptible, no_wait_gpu);
if (unlikely(ret != 0)) {
if (ret != -ERESTARTSYS) {
@@ -732,7 +744,7 @@
spin_lock(&glob->lru_lock);
list_for_each_entry(bo, &man->lru, lru) {
- ret = __ttm_bo_reserve(bo, false, true, false, NULL);
+ ret = __ttm_bo_reserve(bo, false, true, NULL);
if (!ret) {
if (place && (place->fpfn || place->lpfn)) {
/* Don't evict this BO if it's outside of the
@@ -989,13 +1001,19 @@
lockdep_assert_held(&bo->resv->lock.base);
/*
- * FIXME: It's possible to pipeline buffer moves.
- * Have the driver move function wait for idle when necessary,
- * instead of doing it here.
+ * Don't wait for the BO on initial allocation. This is important when
+ * the BO has an imported reservation object.
*/
- ret = ttm_bo_wait(bo, false, interruptible, no_wait_gpu);
- if (ret)
- return ret;
+ if (bo->mem.mem_type != TTM_PL_SYSTEM || bo->ttm != NULL) {
+ /*
+ * FIXME: It's possible to pipeline buffer moves.
+ * Have the driver move function wait for idle when necessary,
+ * instead of doing it here.
+ */
+ ret = ttm_bo_wait(bo, interruptible, no_wait_gpu);
+ if (ret)
+ return ret;
+ }
mem.num_pages = bo->num_pages;
mem.size = mem.num_pages << PAGE_SHIFT;
mem.page_alignment = bo->mem.page_alignment;
@@ -1206,7 +1224,7 @@
size_t size = 0;
size += ttm_round_pot(struct_size);
- size += PAGE_ALIGN(npages * sizeof(void *));
+ size += ttm_round_pot(npages * sizeof(void *));
size += ttm_round_pot(sizeof(struct ttm_tt));
return size;
}
@@ -1220,8 +1238,7 @@
size_t size = 0;
size += ttm_round_pot(struct_size);
- size += PAGE_ALIGN(npages * sizeof(void *));
- size += PAGE_ALIGN(npages * sizeof(dma_addr_t));
+ size += ttm_round_pot(npages * (2*sizeof(void *) + sizeof(dma_addr_t)));
size += ttm_round_pot(sizeof(struct ttm_dma_tt));
return size;
}
@@ -1500,7 +1517,6 @@
bdev->dev_mapping = mapping;
bdev->glob = glob;
bdev->need_dma32 = need_dma32;
- bdev->val_seq = 0;
mutex_lock(&glob->device_list_mutex);
list_add_tail(&bdev->device_list, &glob->device_list);
mutex_unlock(&glob->device_list_mutex);
@@ -1554,7 +1570,7 @@
EXPORT_SYMBOL(ttm_bo_unmap_virtual);
int ttm_bo_wait(struct ttm_buffer_object *bo,
- bool lazy, bool interruptible, bool no_wait)
+ bool interruptible, bool no_wait)
{
struct reservation_object_list *fobj;
struct reservation_object *resv;
@@ -1609,10 +1625,10 @@
* Using ttm_bo_reserve makes sure the lru lists are updated.
*/
- ret = ttm_bo_reserve(bo, true, no_wait, false, NULL);
+ ret = ttm_bo_reserve(bo, true, no_wait, NULL);
if (unlikely(ret != 0))
return ret;
- ret = ttm_bo_wait(bo, false, true, no_wait);
+ ret = ttm_bo_wait(bo, true, no_wait);
if (likely(ret == 0))
atomic_inc(&bo->cpu_writers);
ttm_bo_unreserve(bo);
@@ -1642,7 +1658,7 @@
spin_lock(&glob->lru_lock);
list_for_each_entry(bo, &glob->swap_lru, swap) {
- ret = __ttm_bo_reserve(bo, false, true, false, NULL);
+ ret = __ttm_bo_reserve(bo, false, true, NULL);
if (!ret)
break;
}
@@ -1669,7 +1685,7 @@
* Wait for GPU, then move to system cached.
*/
- ret = ttm_bo_wait(bo, false, false, false);
+ ret = ttm_bo_wait(bo, false, false);
if (unlikely(ret != 0))
goto out;
@@ -1741,7 +1757,7 @@
return -ERESTARTSYS;
if (!ww_mutex_is_locked(&bo->resv->lock))
goto out_unlock;
- ret = __ttm_bo_reserve(bo, true, false, false, NULL);
+ ret = __ttm_bo_reserve(bo, true, false, NULL);
if (unlikely(ret != 0))
goto out_unlock;
__ttm_bo_unreserve(bo);
diff --git a/drivers/gpu/drm/ttm/ttm_bo_util.c b/drivers/gpu/drm/ttm/ttm_bo_util.c
index ac6fe40..d983155 100644
--- a/drivers/gpu/drm/ttm/ttm_bo_util.c
+++ b/drivers/gpu/drm/ttm/ttm_bo_util.c
@@ -645,7 +645,7 @@
reservation_object_add_excl_fence(bo->resv, fence);
if (evict) {
- ret = ttm_bo_wait(bo, false, false, false);
+ ret = ttm_bo_wait(bo, false, false);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c
index 06d26dc..3216878 100644
--- a/drivers/gpu/drm/ttm/ttm_bo_vm.c
+++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c
@@ -54,7 +54,7 @@
/*
* Quick non-stalling check for idle.
*/
- ret = ttm_bo_wait(bo, false, false, true);
+ ret = ttm_bo_wait(bo, false, true);
if (likely(ret == 0))
goto out_unlock;
@@ -68,14 +68,14 @@
goto out_unlock;
up_read(&vma->vm_mm->mmap_sem);
- (void) ttm_bo_wait(bo, false, true, false);
+ (void) ttm_bo_wait(bo, true, false);
goto out_unlock;
}
/*
* Ordinary wait.
*/
- ret = ttm_bo_wait(bo, false, true, false);
+ ret = ttm_bo_wait(bo, true, false);
if (unlikely(ret != 0))
ret = (ret != -ERESTARTSYS) ? VM_FAULT_SIGBUS :
VM_FAULT_NOPAGE;
@@ -108,7 +108,7 @@
* for reserve, and if it fails, retry the fault after waiting
* for the buffer to become unreserved.
*/
- ret = ttm_bo_reserve(bo, true, true, false, NULL);
+ ret = ttm_bo_reserve(bo, true, true, NULL);
if (unlikely(ret != 0)) {
if (ret != -EBUSY)
return VM_FAULT_NOPAGE;
diff --git a/drivers/gpu/drm/ttm/ttm_execbuf_util.c b/drivers/gpu/drm/ttm/ttm_execbuf_util.c
index 3820ae9..a80717b 100644
--- a/drivers/gpu/drm/ttm/ttm_execbuf_util.c
+++ b/drivers/gpu/drm/ttm/ttm_execbuf_util.c
@@ -112,8 +112,7 @@
list_for_each_entry(entry, list, head) {
struct ttm_buffer_object *bo = entry->bo;
- ret = __ttm_bo_reserve(bo, intr, (ticket == NULL), true,
- ticket);
+ ret = __ttm_bo_reserve(bo, intr, (ticket == NULL), ticket);
if (!ret && unlikely(atomic_read(&bo->cpu_writers) > 0)) {
__ttm_bo_unreserve(bo);
diff --git a/drivers/gpu/drm/ttm/ttm_page_alloc.c b/drivers/gpu/drm/ttm/ttm_page_alloc.c
index 025c429..a37de5d 100644
--- a/drivers/gpu/drm/ttm/ttm_page_alloc.c
+++ b/drivers/gpu/drm/ttm/ttm_page_alloc.c
@@ -48,7 +48,7 @@
#include <drm/ttm/ttm_bo_driver.h>
#include <drm/ttm/ttm_page_alloc.h>
-#ifdef TTM_HAS_AGP
+#if IS_ENABLED(CONFIG_AGP)
#include <asm/agp.h>
#endif
@@ -219,7 +219,7 @@
#ifndef CONFIG_X86
static int set_pages_array_wb(struct page **pages, int addrinarray)
{
-#ifdef TTM_HAS_AGP
+#if IS_ENABLED(CONFIG_AGP)
int i;
for (i = 0; i < addrinarray; i++)
@@ -230,7 +230,7 @@
static int set_pages_array_wc(struct page **pages, int addrinarray)
{
-#ifdef TTM_HAS_AGP
+#if IS_ENABLED(CONFIG_AGP)
int i;
for (i = 0; i < addrinarray; i++)
@@ -241,7 +241,7 @@
static int set_pages_array_uc(struct page **pages, int addrinarray)
{
-#ifdef TTM_HAS_AGP
+#if IS_ENABLED(CONFIG_AGP)
int i;
for (i = 0; i < addrinarray; i++)
diff --git a/drivers/gpu/drm/ttm/ttm_page_alloc_dma.c b/drivers/gpu/drm/ttm/ttm_page_alloc_dma.c
index 624d941..bef9f6f 100644
--- a/drivers/gpu/drm/ttm/ttm_page_alloc_dma.c
+++ b/drivers/gpu/drm/ttm/ttm_page_alloc_dma.c
@@ -50,7 +50,7 @@
#include <linux/kthread.h>
#include <drm/ttm/ttm_bo_driver.h>
#include <drm/ttm/ttm_page_alloc.h>
-#ifdef TTM_HAS_AGP
+#if IS_ENABLED(CONFIG_AGP)
#include <asm/agp.h>
#endif
@@ -271,7 +271,7 @@
#ifndef CONFIG_X86
static int set_pages_array_wb(struct page **pages, int addrinarray)
{
-#ifdef TTM_HAS_AGP
+#if IS_ENABLED(CONFIG_AGP)
int i;
for (i = 0; i < addrinarray; i++)
@@ -282,7 +282,7 @@
static int set_pages_array_wc(struct page **pages, int addrinarray)
{
-#ifdef TTM_HAS_AGP
+#if IS_ENABLED(CONFIG_AGP)
int i;
for (i = 0; i < addrinarray; i++)
@@ -293,7 +293,7 @@
static int set_pages_array_uc(struct page **pages, int addrinarray)
{
-#ifdef TTM_HAS_AGP
+#if IS_ENABLED(CONFIG_AGP)
int i;
for (i = 0; i < addrinarray; i++)
diff --git a/drivers/gpu/drm/udl/udl_drv.c b/drivers/gpu/drm/udl/udl_drv.c
index 772ec9e..c204089 100644
--- a/drivers/gpu/drm/udl/udl_drv.c
+++ b/drivers/gpu/drm/udl/udl_drv.c
@@ -94,7 +94,7 @@
struct drm_device *dev = usb_get_intfdata(interface);
drm_kms_helper_poll_disable(dev);
- drm_connector_unplug_all(dev);
+ drm_connector_unregister_all(dev);
udl_fbdev_unplug(dev);
udl_drop_usb(dev);
drm_unplug_dev(dev);
diff --git a/drivers/gpu/drm/udl/udl_drv.h b/drivers/gpu/drm/udl/udl_drv.h
index 4a064ef..0b03d34 100644
--- a/drivers/gpu/drm/udl/udl_drv.h
+++ b/drivers/gpu/drm/udl/udl_drv.h
@@ -81,8 +81,6 @@
struct drm_framebuffer base;
struct udl_gem_object *obj;
bool active_16; /* active on the 16-bit channel */
- int x1, y1, x2, y2; /* dirty rect */
- spinlock_t dirty_lock;
};
#define to_udl_fb(x) container_of(x, struct udl_framebuffer, base)
diff --git a/drivers/gpu/drm/udl/udl_fb.c b/drivers/gpu/drm/udl/udl_fb.c
index fd1eb9d..d5df555 100644
--- a/drivers/gpu/drm/udl/udl_fb.c
+++ b/drivers/gpu/drm/udl/udl_fb.c
@@ -77,68 +77,6 @@
}
#endif
-/*
- * NOTE: fb_defio.c is holding info->fbdefio.mutex
- * Touching ANY framebuffer memory that triggers a page fault
- * in fb_defio will cause a deadlock, when it also tries to
- * grab the same mutex.
- */
-static void udlfb_dpy_deferred_io(struct fb_info *info,
- struct list_head *pagelist)
-{
- struct page *cur;
- struct fb_deferred_io *fbdefio = info->fbdefio;
- struct udl_fbdev *ufbdev = info->par;
- struct drm_device *dev = ufbdev->ufb.base.dev;
- struct udl_device *udl = dev->dev_private;
- struct urb *urb;
- char *cmd;
- cycles_t start_cycles, end_cycles;
- int bytes_sent = 0;
- int bytes_identical = 0;
- int bytes_rendered = 0;
-
- if (!fb_defio)
- return;
-
- start_cycles = get_cycles();
-
- urb = udl_get_urb(dev);
- if (!urb)
- return;
-
- cmd = urb->transfer_buffer;
-
- /* walk the written page list and render each to device */
- list_for_each_entry(cur, &fbdefio->pagelist, lru) {
-
- if (udl_render_hline(dev, (ufbdev->ufb.base.bits_per_pixel / 8),
- &urb, (char *) info->fix.smem_start,
- &cmd, cur->index << PAGE_SHIFT,
- cur->index << PAGE_SHIFT,
- PAGE_SIZE, &bytes_identical, &bytes_sent))
- goto error;
- bytes_rendered += PAGE_SIZE;
- }
-
- if (cmd > (char *) urb->transfer_buffer) {
- /* Send partial buffer remaining before exiting */
- int len = cmd - (char *) urb->transfer_buffer;
- udl_submit_urb(dev, urb, len);
- bytes_sent += len;
- } else
- udl_urb_completion(urb);
-
-error:
- atomic_add(bytes_sent, &udl->bytes_sent);
- atomic_add(bytes_identical, &udl->bytes_identical);
- atomic_add(bytes_rendered, &udl->bytes_rendered);
- end_cycles = get_cycles();
- atomic_add(((unsigned int) ((end_cycles - start_cycles)
- >> 10)), /* Kcycles */
- &udl->cpu_kcycles_used);
-}
-
int udl_handle_damage(struct udl_framebuffer *fb, int x, int y,
int width, int height)
{
@@ -152,9 +90,6 @@
struct urb *urb;
int aligned_x;
int bpp = (fb->base.bits_per_pixel / 8);
- int x2, y2;
- bool store_for_later = false;
- unsigned long flags;
if (!fb->active_16)
return 0;
@@ -180,38 +115,6 @@
(y + height > fb->base.height))
return -EINVAL;
- /* if we are in atomic just store the info
- can't test inside spin lock */
- if (in_atomic())
- store_for_later = true;
-
- x2 = x + width - 1;
- y2 = y + height - 1;
-
- spin_lock_irqsave(&fb->dirty_lock, flags);
-
- if (fb->y1 < y)
- y = fb->y1;
- if (fb->y2 > y2)
- y2 = fb->y2;
- if (fb->x1 < x)
- x = fb->x1;
- if (fb->x2 > x2)
- x2 = fb->x2;
-
- if (store_for_later) {
- fb->x1 = x;
- fb->x2 = x2;
- fb->y1 = y;
- fb->y2 = y2;
- spin_unlock_irqrestore(&fb->dirty_lock, flags);
- return 0;
- }
-
- fb->x1 = fb->y1 = INT_MAX;
- fb->x2 = fb->y2 = 0;
-
- spin_unlock_irqrestore(&fb->dirty_lock, flags);
start_cycles = get_cycles();
urb = udl_get_urb(dev);
@@ -219,14 +122,14 @@
return 0;
cmd = urb->transfer_buffer;
- for (i = y; i <= y2 ; i++) {
+ for (i = y; i < height ; i++) {
const int line_offset = fb->base.pitches[0] * i;
const int byte_offset = line_offset + (x * bpp);
const int dev_byte_offset = (fb->base.width * bpp * i) + (x * bpp);
if (udl_render_hline(dev, bpp, &urb,
(char *) fb->obj->vmapping,
&cmd, byte_offset, dev_byte_offset,
- (x2 - x + 1) * bpp,
+ width * bpp,
&bytes_identical, &bytes_sent))
goto error;
}
@@ -283,36 +186,6 @@
return 0;
}
-static void udl_fb_fillrect(struct fb_info *info, const struct fb_fillrect *rect)
-{
- struct udl_fbdev *ufbdev = info->par;
-
- drm_fb_helper_sys_fillrect(info, rect);
-
- udl_handle_damage(&ufbdev->ufb, rect->dx, rect->dy, rect->width,
- rect->height);
-}
-
-static void udl_fb_copyarea(struct fb_info *info, const struct fb_copyarea *region)
-{
- struct udl_fbdev *ufbdev = info->par;
-
- drm_fb_helper_sys_copyarea(info, region);
-
- udl_handle_damage(&ufbdev->ufb, region->dx, region->dy, region->width,
- region->height);
-}
-
-static void udl_fb_imageblit(struct fb_info *info, const struct fb_image *image)
-{
- struct udl_fbdev *ufbdev = info->par;
-
- drm_fb_helper_sys_imageblit(info, image);
-
- udl_handle_damage(&ufbdev->ufb, image->dx, image->dy, image->width,
- image->height);
-}
-
/*
* It's common for several clients to have framebuffer open simultaneously.
* e.g. both fbcon and X. Makes things interesting.
@@ -339,7 +212,7 @@
if (fbdefio) {
fbdefio->delay = DL_DEFIO_WRITE_DELAY;
- fbdefio->deferred_io = udlfb_dpy_deferred_io;
+ fbdefio->deferred_io = drm_fb_helper_deferred_io;
}
info->fbdefio = fbdefio;
@@ -379,9 +252,9 @@
.owner = THIS_MODULE,
.fb_check_var = drm_fb_helper_check_var,
.fb_set_par = drm_fb_helper_set_par,
- .fb_fillrect = udl_fb_fillrect,
- .fb_copyarea = udl_fb_copyarea,
- .fb_imageblit = udl_fb_imageblit,
+ .fb_fillrect = drm_fb_helper_sys_fillrect,
+ .fb_copyarea = drm_fb_helper_sys_copyarea,
+ .fb_imageblit = drm_fb_helper_sys_imageblit,
.fb_pan_display = drm_fb_helper_pan_display,
.fb_blank = drm_fb_helper_blank,
.fb_setcmap = drm_fb_helper_setcmap,
@@ -458,7 +331,6 @@
{
int ret;
- spin_lock_init(&ufb->dirty_lock);
ufb->obj = obj;
drm_helper_mode_fill_fb_struct(&ufb->base, mode_cmd);
ret = drm_framebuffer_init(dev, &ufb->base, &udlfb_funcs);
@@ -628,7 +500,7 @@
int ret;
uint32_t size;
- obj = drm_gem_object_lookup(dev, file, mode_cmd->handles[0]);
+ obj = drm_gem_object_lookup(file, mode_cmd->handles[0]);
if (obj == NULL)
return ERR_PTR(-ENOENT);
diff --git a/drivers/gpu/drm/udl/udl_gem.c b/drivers/gpu/drm/udl/udl_gem.c
index d7528e0..818e707 100644
--- a/drivers/gpu/drm/udl/udl_gem.c
+++ b/drivers/gpu/drm/udl/udl_gem.c
@@ -217,7 +217,7 @@
int ret = 0;
mutex_lock(&dev->struct_mutex);
- obj = drm_gem_object_lookup(dev, file, handle);
+ obj = drm_gem_object_lookup(file, handle);
if (obj == NULL) {
ret = -ENOENT;
goto unlock;
diff --git a/drivers/gpu/drm/vc4/Kconfig b/drivers/gpu/drm/vc4/Kconfig
index 5848104..e53df59 100644
--- a/drivers/gpu/drm/vc4/Kconfig
+++ b/drivers/gpu/drm/vc4/Kconfig
@@ -5,6 +5,7 @@
select DRM_KMS_HELPER
select DRM_KMS_CMA_HELPER
select DRM_GEM_CMA_HELPER
+ select DRM_PANEL
help
Choose this option if you have a system that has a Broadcom
VC4 GPU, such as the Raspberry Pi or other BCM2708/BCM2835.
diff --git a/drivers/gpu/drm/vc4/Makefile b/drivers/gpu/drm/vc4/Makefile
index 4c6a99f..fb77db7 100644
--- a/drivers/gpu/drm/vc4/Makefile
+++ b/drivers/gpu/drm/vc4/Makefile
@@ -7,6 +7,7 @@
vc4_bo.o \
vc4_crtc.o \
vc4_drv.o \
+ vc4_dpi.o \
vc4_kms.o \
vc4_gem.o \
vc4_hdmi.o \
diff --git a/drivers/gpu/drm/vc4/vc4_bo.c b/drivers/gpu/drm/vc4/vc4_bo.c
index 9807bc9..e5a9d3a 100644
--- a/drivers/gpu/drm/vc4/vc4_bo.c
+++ b/drivers/gpu/drm/vc4/vc4_bo.c
@@ -457,7 +457,7 @@
struct drm_vc4_mmap_bo *args = data;
struct drm_gem_object *gem_obj;
- gem_obj = drm_gem_object_lookup(dev, file_priv, args->handle);
+ gem_obj = drm_gem_object_lookup(file_priv, args->handle);
if (!gem_obj) {
DRM_ERROR("Failed to look up GEM BO %d\n", args->handle);
return -EINVAL;
diff --git a/drivers/gpu/drm/vc4/vc4_crtc.c b/drivers/gpu/drm/vc4/vc4_crtc.c
index 355ee4b..904d075 100644
--- a/drivers/gpu/drm/vc4/vc4_crtc.c
+++ b/drivers/gpu/drm/vc4/vc4_crtc.c
@@ -49,6 +49,10 @@
/* Which HVS channel we're using for our CRTC. */
int channel;
+ u8 lut_r[256];
+ u8 lut_g[256];
+ u8 lut_b[256];
+
struct drm_pending_vblank_event *event;
};
@@ -147,6 +151,46 @@
drm_crtc_cleanup(crtc);
}
+static void
+vc4_crtc_lut_load(struct drm_crtc *crtc)
+{
+ struct drm_device *dev = crtc->dev;
+ struct vc4_dev *vc4 = to_vc4_dev(dev);
+ struct vc4_crtc *vc4_crtc = to_vc4_crtc(crtc);
+ u32 i;
+
+ /* The LUT memory is laid out with each HVS channel in order,
+ * each of which takes 256 writes for R, 256 for G, then 256
+ * for B.
+ */
+ HVS_WRITE(SCALER_GAMADDR,
+ SCALER_GAMADDR_AUTOINC |
+ (vc4_crtc->channel * 3 * crtc->gamma_size));
+
+ for (i = 0; i < crtc->gamma_size; i++)
+ HVS_WRITE(SCALER_GAMDATA, vc4_crtc->lut_r[i]);
+ for (i = 0; i < crtc->gamma_size; i++)
+ HVS_WRITE(SCALER_GAMDATA, vc4_crtc->lut_g[i]);
+ for (i = 0; i < crtc->gamma_size; i++)
+ HVS_WRITE(SCALER_GAMDATA, vc4_crtc->lut_b[i]);
+}
+
+static void
+vc4_crtc_gamma_set(struct drm_crtc *crtc, u16 *r, u16 *g, u16 *b,
+ uint32_t start, uint32_t size)
+{
+ struct vc4_crtc *vc4_crtc = to_vc4_crtc(crtc);
+ u32 i;
+
+ for (i = start; i < start + size; i++) {
+ vc4_crtc->lut_r[i] = r[i] >> 8;
+ vc4_crtc->lut_g[i] = g[i] >> 8;
+ vc4_crtc->lut_b[i] = b[i] >> 8;
+ }
+
+ vc4_crtc_lut_load(crtc);
+}
+
static u32 vc4_get_fifo_full_level(u32 format)
{
static const u32 fifo_len_bytes = 64;
@@ -260,8 +304,14 @@
HVS_WRITE(SCALER_DISPBKGNDX(vc4_crtc->channel),
SCALER_DISPBKGND_AUTOHS |
+ SCALER_DISPBKGND_GAMMA |
(interlace ? SCALER_DISPBKGND_INTERLACE : 0));
+ /* Reload the LUT, since the SRAMs would have been disabled if
+ * all CRTCs had SCALER_DISPBKGND_GAMMA unset at once.
+ */
+ vc4_crtc_lut_load(crtc);
+
if (debug_dump_regs) {
DRM_INFO("CRTC %d regs after:\n", drm_crtc_index(crtc));
vc4_crtc_dump_regs(vc4_crtc);
@@ -600,7 +650,7 @@
}
- __drm_atomic_helper_crtc_destroy_state(crtc, state);
+ __drm_atomic_helper_crtc_destroy_state(state);
}
static const struct drm_crtc_funcs vc4_crtc_funcs = {
@@ -613,6 +663,7 @@
.reset = drm_atomic_helper_crtc_reset,
.atomic_duplicate_state = vc4_crtc_duplicate_state,
.atomic_destroy_state = vc4_crtc_destroy_state,
+ .gamma_set = vc4_crtc_gamma_set,
};
static const struct drm_crtc_helper_funcs vc4_crtc_helper_funcs = {
@@ -711,6 +762,7 @@
primary_plane->crtc = crtc;
vc4->crtc[drm_crtc_index(crtc)] = vc4_crtc;
vc4_crtc->channel = vc4_crtc->data->hvs_channel;
+ drm_mode_crtc_set_gamma_size(crtc, ARRAY_SIZE(vc4_crtc->lut_r));
/* Set up some arbitrary number of planes. We're not limited
* by a set number of physical registers, just the space in
@@ -751,6 +803,12 @@
vc4_set_crtc_possible_masks(drm, crtc);
+ for (i = 0; i < crtc->gamma_size; i++) {
+ vc4_crtc->lut_r[i] = i;
+ vc4_crtc->lut_g[i] = i;
+ vc4_crtc->lut_b[i] = i;
+ }
+
platform_set_drvdata(pdev, vc4_crtc);
return 0;
diff --git a/drivers/gpu/drm/vc4/vc4_debugfs.c b/drivers/gpu/drm/vc4/vc4_debugfs.c
index d76ad10..245115d 100644
--- a/drivers/gpu/drm/vc4/vc4_debugfs.c
+++ b/drivers/gpu/drm/vc4/vc4_debugfs.c
@@ -17,6 +17,7 @@
static const struct drm_info_list vc4_debugfs_list[] = {
{"bo_stats", vc4_bo_stats_debugfs, 0},
+ {"dpi_regs", vc4_dpi_debugfs_regs, 0},
{"hdmi_regs", vc4_hdmi_debugfs_regs, 0},
{"hvs_regs", vc4_hvs_debugfs_regs, 0},
{"crtc0_regs", vc4_crtc_debugfs_regs, 0, (void *)(uintptr_t)0},
diff --git a/drivers/gpu/drm/vc4/vc4_dpi.c b/drivers/gpu/drm/vc4/vc4_dpi.c
new file mode 100644
index 0000000..9817dbf
--- /dev/null
+++ b/drivers/gpu/drm/vc4/vc4_dpi.c
@@ -0,0 +1,520 @@
+/*
+ * Copyright (C) 2016 Broadcom Limited
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+/**
+ * DOC: VC4 DPI module
+ *
+ * The VC4 DPI hardware supports MIPI DPI type 4 and Nokia ViSSI
+ * signals, which are routed out to GPIO0-27 with the ALT2 function.
+ */
+
+#include "drm_atomic_helper.h"
+#include "drm_crtc_helper.h"
+#include "drm_edid.h"
+#include "drm_panel.h"
+#include "linux/clk.h"
+#include "linux/component.h"
+#include "linux/of_graph.h"
+#include "linux/of_platform.h"
+#include "vc4_drv.h"
+#include "vc4_regs.h"
+
+#define DPI_C 0x00
+# define DPI_OUTPUT_ENABLE_MODE BIT(16)
+
+/* The order field takes the incoming 24 bit RGB from the pixel valve
+ * and shuffles the 3 channels.
+ */
+# define DPI_ORDER_MASK VC4_MASK(15, 14)
+# define DPI_ORDER_SHIFT 14
+# define DPI_ORDER_RGB 0
+# define DPI_ORDER_BGR 1
+# define DPI_ORDER_GRB 2
+# define DPI_ORDER_BRG 3
+
+/* The format field takes the ORDER-shuffled pixel valve data and
+ * formats it onto the output lines.
+ */
+# define DPI_FORMAT_MASK VC4_MASK(13, 11)
+# define DPI_FORMAT_SHIFT 11
+/* This define is named in the hardware, but actually just outputs 0. */
+# define DPI_FORMAT_9BIT_666_RGB 0
+/* Outputs 00000000rrrrrggggggbbbbb */
+# define DPI_FORMAT_16BIT_565_RGB_1 1
+/* Outputs 000rrrrr00gggggg000bbbbb */
+# define DPI_FORMAT_16BIT_565_RGB_2 2
+/* Outputs 00rrrrr000gggggg00bbbbb0 */
+# define DPI_FORMAT_16BIT_565_RGB_3 3
+/* Outputs 000000rrrrrrggggggbbbbbb */
+# define DPI_FORMAT_18BIT_666_RGB_1 4
+/* Outputs 00rrrrrr00gggggg00bbbbbb */
+# define DPI_FORMAT_18BIT_666_RGB_2 5
+/* Outputs rrrrrrrrggggggggbbbbbbbb */
+# define DPI_FORMAT_24BIT_888_RGB 6
+
+/* Reverses the polarity of the corresponding signal */
+# define DPI_PIXEL_CLK_INVERT BIT(10)
+# define DPI_HSYNC_INVERT BIT(9)
+# define DPI_VSYNC_INVERT BIT(8)
+# define DPI_OUTPUT_ENABLE_INVERT BIT(7)
+
+/* Outputs the signal the falling clock edge instead of rising. */
+# define DPI_HSYNC_NEGATE BIT(6)
+# define DPI_VSYNC_NEGATE BIT(5)
+# define DPI_OUTPUT_ENABLE_NEGATE BIT(4)
+
+/* Disables the signal */
+# define DPI_HSYNC_DISABLE BIT(3)
+# define DPI_VSYNC_DISABLE BIT(2)
+# define DPI_OUTPUT_ENABLE_DISABLE BIT(1)
+
+/* Power gate to the device, full reset at 0 -> 1 transition */
+# define DPI_ENABLE BIT(0)
+
+/* All other registers besides DPI_C return the ID */
+#define DPI_ID 0x04
+# define DPI_ID_VALUE 0x00647069
+
+/* General DPI hardware state. */
+struct vc4_dpi {
+ struct platform_device *pdev;
+
+ struct drm_encoder *encoder;
+ struct drm_connector *connector;
+ struct drm_panel *panel;
+
+ void __iomem *regs;
+
+ struct clk *pixel_clock;
+ struct clk *core_clock;
+};
+
+#define DPI_READ(offset) readl(dpi->regs + (offset))
+#define DPI_WRITE(offset, val) writel(val, dpi->regs + (offset))
+
+/* VC4 DPI encoder KMS struct */
+struct vc4_dpi_encoder {
+ struct vc4_encoder base;
+ struct vc4_dpi *dpi;
+};
+
+static inline struct vc4_dpi_encoder *
+to_vc4_dpi_encoder(struct drm_encoder *encoder)
+{
+ return container_of(encoder, struct vc4_dpi_encoder, base.base);
+}
+
+/* VC4 DPI connector KMS struct */
+struct vc4_dpi_connector {
+ struct drm_connector base;
+ struct vc4_dpi *dpi;
+
+ /* Since the connector is attached to just the one encoder,
+ * this is the reference to it so we can do the best_encoder()
+ * hook.
+ */
+ struct drm_encoder *encoder;
+};
+
+static inline struct vc4_dpi_connector *
+to_vc4_dpi_connector(struct drm_connector *connector)
+{
+ return container_of(connector, struct vc4_dpi_connector, base);
+}
+
+#define DPI_REG(reg) { reg, #reg }
+static const struct {
+ u32 reg;
+ const char *name;
+} dpi_regs[] = {
+ DPI_REG(DPI_C),
+ DPI_REG(DPI_ID),
+};
+
+static void vc4_dpi_dump_regs(struct vc4_dpi *dpi)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(dpi_regs); i++) {
+ DRM_INFO("0x%04x (%s): 0x%08x\n",
+ dpi_regs[i].reg, dpi_regs[i].name,
+ DPI_READ(dpi_regs[i].reg));
+ }
+}
+
+#ifdef CONFIG_DEBUG_FS
+int vc4_dpi_debugfs_regs(struct seq_file *m, void *unused)
+{
+ struct drm_info_node *node = (struct drm_info_node *)m->private;
+ struct drm_device *dev = node->minor->dev;
+ struct vc4_dev *vc4 = to_vc4_dev(dev);
+ struct vc4_dpi *dpi = vc4->dpi;
+ int i;
+
+ if (!dpi)
+ return 0;
+
+ for (i = 0; i < ARRAY_SIZE(dpi_regs); i++) {
+ seq_printf(m, "%s (0x%04x): 0x%08x\n",
+ dpi_regs[i].name, dpi_regs[i].reg,
+ DPI_READ(dpi_regs[i].reg));
+ }
+
+ return 0;
+}
+#endif
+
+static enum drm_connector_status
+vc4_dpi_connector_detect(struct drm_connector *connector, bool force)
+{
+ struct vc4_dpi_connector *vc4_connector =
+ to_vc4_dpi_connector(connector);
+ struct vc4_dpi *dpi = vc4_connector->dpi;
+
+ if (dpi->panel)
+ return connector_status_connected;
+ else
+ return connector_status_disconnected;
+}
+
+static void vc4_dpi_connector_destroy(struct drm_connector *connector)
+{
+ drm_connector_unregister(connector);
+ drm_connector_cleanup(connector);
+}
+
+static int vc4_dpi_connector_get_modes(struct drm_connector *connector)
+{
+ struct vc4_dpi_connector *vc4_connector =
+ to_vc4_dpi_connector(connector);
+ struct vc4_dpi *dpi = vc4_connector->dpi;
+
+ if (dpi->panel)
+ return drm_panel_get_modes(dpi->panel);
+
+ return 0;
+}
+
+static struct drm_encoder *
+vc4_dpi_connector_best_encoder(struct drm_connector *connector)
+{
+ struct vc4_dpi_connector *dpi_connector =
+ to_vc4_dpi_connector(connector);
+ return dpi_connector->encoder;
+}
+
+static const struct drm_connector_funcs vc4_dpi_connector_funcs = {
+ .dpms = drm_atomic_helper_connector_dpms,
+ .detect = vc4_dpi_connector_detect,
+ .fill_modes = drm_helper_probe_single_connector_modes,
+ .destroy = vc4_dpi_connector_destroy,
+ .reset = drm_atomic_helper_connector_reset,
+ .atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
+ .atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
+};
+
+static const struct drm_connector_helper_funcs vc4_dpi_connector_helper_funcs = {
+ .get_modes = vc4_dpi_connector_get_modes,
+ .best_encoder = vc4_dpi_connector_best_encoder,
+};
+
+static struct drm_connector *vc4_dpi_connector_init(struct drm_device *dev,
+ struct vc4_dpi *dpi)
+{
+ struct drm_connector *connector = NULL;
+ struct vc4_dpi_connector *dpi_connector;
+ int ret = 0;
+
+ dpi_connector = devm_kzalloc(dev->dev, sizeof(*dpi_connector),
+ GFP_KERNEL);
+ if (!dpi_connector) {
+ ret = -ENOMEM;
+ goto fail;
+ }
+ connector = &dpi_connector->base;
+
+ dpi_connector->encoder = dpi->encoder;
+ dpi_connector->dpi = dpi;
+
+ drm_connector_init(dev, connector, &vc4_dpi_connector_funcs,
+ DRM_MODE_CONNECTOR_DPI);
+ drm_connector_helper_add(connector, &vc4_dpi_connector_helper_funcs);
+
+ connector->polled = 0;
+ connector->interlace_allowed = 0;
+ connector->doublescan_allowed = 0;
+
+ drm_mode_connector_attach_encoder(connector, dpi->encoder);
+
+ return connector;
+
+ fail:
+ if (connector)
+ vc4_dpi_connector_destroy(connector);
+
+ return ERR_PTR(ret);
+}
+
+static const struct drm_encoder_funcs vc4_dpi_encoder_funcs = {
+ .destroy = drm_encoder_cleanup,
+};
+
+static void vc4_dpi_encoder_disable(struct drm_encoder *encoder)
+{
+ struct vc4_dpi_encoder *vc4_encoder = to_vc4_dpi_encoder(encoder);
+ struct vc4_dpi *dpi = vc4_encoder->dpi;
+
+ drm_panel_disable(dpi->panel);
+
+ clk_disable_unprepare(dpi->pixel_clock);
+
+ drm_panel_unprepare(dpi->panel);
+}
+
+static void vc4_dpi_encoder_enable(struct drm_encoder *encoder)
+{
+ struct drm_display_mode *mode = &encoder->crtc->mode;
+ struct vc4_dpi_encoder *vc4_encoder = to_vc4_dpi_encoder(encoder);
+ struct vc4_dpi *dpi = vc4_encoder->dpi;
+ u32 dpi_c = DPI_ENABLE | DPI_OUTPUT_ENABLE_MODE;
+ int ret;
+
+ ret = drm_panel_prepare(dpi->panel);
+ if (ret) {
+ DRM_ERROR("Panel failed to prepare\n");
+ return;
+ }
+
+ if (dpi->connector->display_info.num_bus_formats) {
+ u32 bus_format = dpi->connector->display_info.bus_formats[0];
+
+ switch (bus_format) {
+ case MEDIA_BUS_FMT_RGB888_1X24:
+ dpi_c |= VC4_SET_FIELD(DPI_FORMAT_24BIT_888_RGB,
+ DPI_FORMAT);
+ break;
+ case MEDIA_BUS_FMT_BGR888_1X24:
+ dpi_c |= VC4_SET_FIELD(DPI_FORMAT_24BIT_888_RGB,
+ DPI_FORMAT);
+ dpi_c |= VC4_SET_FIELD(DPI_ORDER_BGR, DPI_ORDER);
+ break;
+ case MEDIA_BUS_FMT_RGB666_1X24_CPADHI:
+ dpi_c |= VC4_SET_FIELD(DPI_FORMAT_18BIT_666_RGB_2,
+ DPI_FORMAT);
+ break;
+ case MEDIA_BUS_FMT_RGB666_1X18:
+ dpi_c |= VC4_SET_FIELD(DPI_FORMAT_18BIT_666_RGB_1,
+ DPI_FORMAT);
+ break;
+ case MEDIA_BUS_FMT_RGB565_1X16:
+ dpi_c |= VC4_SET_FIELD(DPI_FORMAT_16BIT_565_RGB_3,
+ DPI_FORMAT);
+ break;
+ default:
+ DRM_ERROR("Unknown media bus format %d\n", bus_format);
+ break;
+ }
+ }
+
+ if (mode->flags & DRM_MODE_FLAG_NHSYNC)
+ dpi_c |= DPI_HSYNC_INVERT;
+ else if (!(mode->flags & DRM_MODE_FLAG_PHSYNC))
+ dpi_c |= DPI_HSYNC_DISABLE;
+
+ if (mode->flags & DRM_MODE_FLAG_NVSYNC)
+ dpi_c |= DPI_VSYNC_INVERT;
+ else if (!(mode->flags & DRM_MODE_FLAG_PVSYNC))
+ dpi_c |= DPI_VSYNC_DISABLE;
+
+ DPI_WRITE(DPI_C, dpi_c);
+
+ ret = clk_set_rate(dpi->pixel_clock, mode->clock * 1000);
+ if (ret)
+ DRM_ERROR("Failed to set clock rate: %d\n", ret);
+
+ ret = clk_prepare_enable(dpi->pixel_clock);
+ if (ret)
+ DRM_ERROR("Failed to set clock rate: %d\n", ret);
+
+ ret = drm_panel_enable(dpi->panel);
+ if (ret) {
+ DRM_ERROR("Panel failed to enable\n");
+ drm_panel_unprepare(dpi->panel);
+ return;
+ }
+}
+
+static const struct drm_encoder_helper_funcs vc4_dpi_encoder_helper_funcs = {
+ .disable = vc4_dpi_encoder_disable,
+ .enable = vc4_dpi_encoder_enable,
+};
+
+static const struct of_device_id vc4_dpi_dt_match[] = {
+ { .compatible = "brcm,bcm2835-dpi", .data = NULL },
+ {}
+};
+
+/* Walks the OF graph to find the panel node and then asks DRM to look
+ * up the panel.
+ */
+static struct drm_panel *vc4_dpi_get_panel(struct device *dev)
+{
+ struct device_node *endpoint, *panel_node;
+ struct device_node *np = dev->of_node;
+ struct drm_panel *panel;
+
+ endpoint = of_graph_get_next_endpoint(np, NULL);
+ if (!endpoint) {
+ dev_err(dev, "no endpoint to fetch DPI panel\n");
+ return NULL;
+ }
+
+ /* don't proceed if we have an endpoint but no panel_node tied to it */
+ panel_node = of_graph_get_remote_port_parent(endpoint);
+ of_node_put(endpoint);
+ if (!panel_node) {
+ dev_err(dev, "no valid panel node\n");
+ return NULL;
+ }
+
+ panel = of_drm_find_panel(panel_node);
+ of_node_put(panel_node);
+
+ return panel;
+}
+
+static int vc4_dpi_bind(struct device *dev, struct device *master, void *data)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct drm_device *drm = dev_get_drvdata(master);
+ struct vc4_dev *vc4 = to_vc4_dev(drm);
+ struct vc4_dpi *dpi;
+ struct vc4_dpi_encoder *vc4_dpi_encoder;
+ int ret;
+
+ dpi = devm_kzalloc(dev, sizeof(*dpi), GFP_KERNEL);
+ if (!dpi)
+ return -ENOMEM;
+
+ vc4_dpi_encoder = devm_kzalloc(dev, sizeof(*vc4_dpi_encoder),
+ GFP_KERNEL);
+ if (!vc4_dpi_encoder)
+ return -ENOMEM;
+ vc4_dpi_encoder->base.type = VC4_ENCODER_TYPE_DPI;
+ vc4_dpi_encoder->dpi = dpi;
+ dpi->encoder = &vc4_dpi_encoder->base.base;
+
+ dpi->pdev = pdev;
+ dpi->regs = vc4_ioremap_regs(pdev, 0);
+ if (IS_ERR(dpi->regs))
+ return PTR_ERR(dpi->regs);
+
+ vc4_dpi_dump_regs(dpi);
+
+ if (DPI_READ(DPI_ID) != DPI_ID_VALUE) {
+ dev_err(dev, "Port returned 0x%08x for ID instead of 0x%08x\n",
+ DPI_READ(DPI_ID), DPI_ID_VALUE);
+ return -ENODEV;
+ }
+
+ dpi->core_clock = devm_clk_get(dev, "core");
+ if (IS_ERR(dpi->core_clock)) {
+ ret = PTR_ERR(dpi->core_clock);
+ if (ret != -EPROBE_DEFER)
+ DRM_ERROR("Failed to get core clock: %d\n", ret);
+ return ret;
+ }
+ dpi->pixel_clock = devm_clk_get(dev, "pixel");
+ if (IS_ERR(dpi->pixel_clock)) {
+ ret = PTR_ERR(dpi->pixel_clock);
+ if (ret != -EPROBE_DEFER)
+ DRM_ERROR("Failed to get pixel clock: %d\n", ret);
+ return ret;
+ }
+
+ ret = clk_prepare_enable(dpi->core_clock);
+ if (ret)
+ DRM_ERROR("Failed to turn on core clock: %d\n", ret);
+
+ dpi->panel = vc4_dpi_get_panel(dev);
+
+ drm_encoder_init(drm, dpi->encoder, &vc4_dpi_encoder_funcs,
+ DRM_MODE_ENCODER_DPI, NULL);
+ drm_encoder_helper_add(dpi->encoder, &vc4_dpi_encoder_helper_funcs);
+
+ dpi->connector = vc4_dpi_connector_init(drm, dpi);
+ if (IS_ERR(dpi->connector)) {
+ ret = PTR_ERR(dpi->connector);
+ goto err_destroy_encoder;
+ }
+
+ if (dpi->panel)
+ drm_panel_attach(dpi->panel, dpi->connector);
+
+ dev_set_drvdata(dev, dpi);
+
+ vc4->dpi = dpi;
+
+ return 0;
+
+err_destroy_encoder:
+ drm_encoder_cleanup(dpi->encoder);
+ clk_disable_unprepare(dpi->core_clock);
+ return ret;
+}
+
+static void vc4_dpi_unbind(struct device *dev, struct device *master,
+ void *data)
+{
+ struct drm_device *drm = dev_get_drvdata(master);
+ struct vc4_dev *vc4 = to_vc4_dev(drm);
+ struct vc4_dpi *dpi = dev_get_drvdata(dev);
+
+ if (dpi->panel)
+ drm_panel_detach(dpi->panel);
+
+ vc4_dpi_connector_destroy(dpi->connector);
+ drm_encoder_cleanup(dpi->encoder);
+
+ clk_disable_unprepare(dpi->core_clock);
+
+ vc4->dpi = NULL;
+}
+
+static const struct component_ops vc4_dpi_ops = {
+ .bind = vc4_dpi_bind,
+ .unbind = vc4_dpi_unbind,
+};
+
+static int vc4_dpi_dev_probe(struct platform_device *pdev)
+{
+ return component_add(&pdev->dev, &vc4_dpi_ops);
+}
+
+static int vc4_dpi_dev_remove(struct platform_device *pdev)
+{
+ component_del(&pdev->dev, &vc4_dpi_ops);
+ return 0;
+}
+
+struct platform_driver vc4_dpi_driver = {
+ .probe = vc4_dpi_dev_probe,
+ .remove = vc4_dpi_dev_remove,
+ .driver = {
+ .name = "vc4_dpi",
+ .of_match_table = vc4_dpi_dt_match,
+ },
+};
diff --git a/drivers/gpu/drm/vc4/vc4_drv.c b/drivers/gpu/drm/vc4/vc4_drv.c
index 109b106..3446ece 100644
--- a/drivers/gpu/drm/vc4/vc4_drv.c
+++ b/drivers/gpu/drm/vc4/vc4_drv.c
@@ -81,6 +81,7 @@
DRIVER_ATOMIC |
DRIVER_GEM |
DRIVER_HAVE_IRQ |
+ DRIVER_RENDER |
DRIVER_PRIME),
.lastclose = vc4_lastclose,
.irq_handler = vc4_irq,
@@ -257,6 +258,7 @@
static struct platform_driver *const component_drivers[] = {
&vc4_hdmi_driver,
+ &vc4_dpi_driver,
&vc4_crtc_driver,
&vc4_hvs_driver,
&vc4_v3d_driver,
diff --git a/drivers/gpu/drm/vc4/vc4_drv.h b/drivers/gpu/drm/vc4/vc4_drv.h
index fa2ad15..37cac59 100644
--- a/drivers/gpu/drm/vc4/vc4_drv.h
+++ b/drivers/gpu/drm/vc4/vc4_drv.h
@@ -16,6 +16,7 @@
struct vc4_hvs *hvs;
struct vc4_crtc *crtc[3];
struct vc4_v3d *v3d;
+ struct vc4_dpi *dpi;
struct drm_fbdev_cma *fbdev;
@@ -422,6 +423,10 @@
/* vc4_drv.c */
void __iomem *vc4_ioremap_regs(struct platform_device *dev, int index);
+/* vc4_dpi.c */
+extern struct platform_driver vc4_dpi_driver;
+int vc4_dpi_debugfs_regs(struct seq_file *m, void *unused);
+
/* vc4_gem.c */
void vc4_gem_init(struct drm_device *dev);
void vc4_gem_destroy(struct drm_device *dev);
diff --git a/drivers/gpu/drm/vc4/vc4_gem.c b/drivers/gpu/drm/vc4/vc4_gem.c
index 8d4384f..46899d6 100644
--- a/drivers/gpu/drm/vc4/vc4_gem.c
+++ b/drivers/gpu/drm/vc4/vc4_gem.c
@@ -822,7 +822,7 @@
if (args->pad != 0)
return -EINVAL;
- gem_obj = drm_gem_object_lookup(dev, file_priv, args->handle);
+ gem_obj = drm_gem_object_lookup(file_priv, args->handle);
if (!gem_obj) {
DRM_ERROR("Failed to look up GEM BO %d\n", args->handle);
return -EINVAL;
diff --git a/drivers/gpu/drm/vc4/vc4_hdmi.c b/drivers/gpu/drm/vc4/vc4_hdmi.c
index d8b8649..fd2644d 100644
--- a/drivers/gpu/drm/vc4/vc4_hdmi.c
+++ b/drivers/gpu/drm/vc4/vc4_hdmi.c
@@ -573,7 +573,7 @@
err_unprepare_pix:
clk_disable_unprepare(hdmi->pixel_clock);
err_put_i2c:
- put_device(&vc4->hdmi->ddc->dev);
+ put_device(&hdmi->ddc->dev);
return ret;
}
diff --git a/drivers/gpu/drm/vc4/vc4_kms.c b/drivers/gpu/drm/vc4/vc4_kms.c
index 4718ae5..cb37751 100644
--- a/drivers/gpu/drm/vc4/vc4_kms.c
+++ b/drivers/gpu/drm/vc4/vc4_kms.c
@@ -93,7 +93,7 @@
* vc4_atomic_commit - commit validated state object
* @dev: DRM device
* @state: the driver state object
- * @async: asynchronous commit
+ * @nonblock: nonblocking commit
*
* This function commits a with drm_atomic_helper_check() pre-validated state
* object. This can still fail when e.g. the framebuffer reservation fails. For
@@ -104,7 +104,7 @@
*/
static int vc4_atomic_commit(struct drm_device *dev,
struct drm_atomic_state *state,
- bool async)
+ bool nonblock)
{
struct vc4_dev *vc4 = to_vc4_dev(dev);
int ret;
@@ -170,7 +170,7 @@
* current layout.
*/
- if (async) {
+ if (nonblock) {
vc4_queue_seqno_cb(dev, &c->cb, wait_seqno,
vc4_atomic_complete_commit_seqno_cb);
} else {
@@ -207,8 +207,6 @@
dev->mode_config.preferred_depth = 24;
dev->mode_config.async_page_flip = true;
- dev->vblank_disable_allowed = true;
-
drm_mode_config_reset(dev);
vc4->fbdev = drm_fbdev_cma_init(dev, 32,
diff --git a/drivers/gpu/drm/vc4/vc4_plane.c b/drivers/gpu/drm/vc4/vc4_plane.c
index 7b0c72a..4037b52 100644
--- a/drivers/gpu/drm/vc4/vc4_plane.c
+++ b/drivers/gpu/drm/vc4/vc4_plane.c
@@ -208,7 +208,7 @@
}
kfree(vc4_state->dlist);
- __drm_atomic_helper_plane_destroy_state(plane, &vc4_state->base);
+ __drm_atomic_helper_plane_destroy_state(&vc4_state->base);
kfree(state);
}
diff --git a/drivers/gpu/drm/vc4/vc4_regs.h b/drivers/gpu/drm/vc4/vc4_regs.h
index bf42a8e..6163b95 100644
--- a/drivers/gpu/drm/vc4/vc4_regs.h
+++ b/drivers/gpu/drm/vc4/vc4_regs.h
@@ -390,6 +390,12 @@
#define SCALER_DISPBASE2 0x0000006c
#define SCALER_DISPALPHA2 0x00000070
#define SCALER_GAMADDR 0x00000078
+# define SCALER_GAMADDR_AUTOINC BIT(31)
+/* Enables all gamma ramp SRAMs, not just those of CRTCs with gamma
+ * enabled.
+ */
+# define SCALER_GAMADDR_SRAMENB BIT(30)
+
#define SCALER_GAMDATA 0x000000e0
#define SCALER_DLIST_START 0x00002000
#define SCALER_DLIST_SIZE 0x00004000
diff --git a/drivers/gpu/drm/vgem/vgem_drv.c b/drivers/gpu/drm/vgem/vgem_drv.c
index c503a84..341f9be 100644
--- a/drivers/gpu/drm/vgem/vgem_drv.c
+++ b/drivers/gpu/drm/vgem/vgem_drv.c
@@ -89,7 +89,6 @@
static int vgem_gem_fault(struct vm_area_struct *vma, struct vm_fault *vmf)
{
struct drm_vgem_gem_object *obj = vma->vm_private_data;
- struct drm_device *dev = obj->base.dev;
loff_t num_pages;
pgoff_t page_offset;
int ret;
@@ -103,12 +102,8 @@
if (page_offset > num_pages)
return VM_FAULT_SIGBUS;
- mutex_lock(&dev->struct_mutex);
-
ret = vm_insert_page(vma, (unsigned long)vmf->virtual_address,
obj->pages[page_offset]);
-
- mutex_unlock(&dev->struct_mutex);
switch (ret) {
case 0:
return VM_FAULT_NOPAGE;
@@ -154,6 +149,10 @@
if (err)
goto out;
+ err = vgem_gem_get_pages(obj);
+ if (err)
+ goto out;
+
err = drm_gem_handle_create(file, gem_object, handle);
if (err)
goto handle_out;
@@ -201,37 +200,23 @@
int ret = 0;
struct drm_gem_object *obj;
- mutex_lock(&dev->struct_mutex);
- obj = drm_gem_object_lookup(dev, file, handle);
- if (!obj) {
- ret = -ENOENT;
- goto unlock;
- }
+ obj = drm_gem_object_lookup(file, handle);
+ if (!obj)
+ return -ENOENT;
- if (!drm_vma_node_has_offset(&obj->vma_node)) {
- ret = drm_gem_create_mmap_offset(obj);
- if (ret)
- goto unref;
- }
+ ret = drm_gem_create_mmap_offset(obj);
+ if (ret)
+ goto unref;
BUG_ON(!obj->filp);
obj->filp->private_data = obj;
- ret = vgem_gem_get_pages(to_vgem_bo(obj));
- if (ret)
- goto fail_get_pages;
-
*offset = drm_vma_node_offset_addr(&obj->vma_node);
- goto unref;
-
-fail_get_pages:
- drm_gem_free_mmap_offset(obj);
unref:
- drm_gem_object_unreference(obj);
-unlock:
- mutex_unlock(&dev->struct_mutex);
+ drm_gem_object_unreference_unlocked(obj);
+
return ret;
}
diff --git a/drivers/gpu/drm/virtio/virtgpu_display.c b/drivers/gpu/drm/virtio/virtgpu_display.c
index 5fd1fd0..d4305da 100644
--- a/drivers/gpu/drm/virtio/virtgpu_display.c
+++ b/drivers/gpu/drm/virtio/virtgpu_display.c
@@ -38,13 +38,6 @@
#define XRES_MAX 8192
#define YRES_MAX 8192
-static void virtio_gpu_crtc_gamma_set(struct drm_crtc *crtc,
- u16 *red, u16 *green, u16 *blue,
- uint32_t start, uint32_t size)
-{
- /* TODO */
-}
-
static void
virtio_gpu_hide_cursor(struct virtio_gpu_device *vgdev,
struct virtio_gpu_output *output)
@@ -75,7 +68,7 @@
}
/* lookup the cursor */
- gobj = drm_gem_object_lookup(crtc->dev, file_priv, handle);
+ gobj = drm_gem_object_lookup(file_priv, handle);
if (gobj == NULL)
return -ENOENT;
@@ -173,7 +166,6 @@
static const struct drm_crtc_funcs virtio_gpu_crtc_funcs = {
.cursor_set2 = virtio_gpu_crtc_cursor_set,
.cursor_move = virtio_gpu_crtc_cursor_move,
- .gamma_set = virtio_gpu_crtc_gamma_set,
.set_config = drm_atomic_helper_set_config,
.destroy = drm_crtc_cleanup,
@@ -428,7 +420,6 @@
return PTR_ERR(plane);
drm_crtc_init_with_planes(dev, crtc, plane, NULL,
&virtio_gpu_crtc_funcs, NULL);
- drm_mode_crtc_set_gamma_size(crtc, 256);
drm_crtc_helper_add(crtc, &virtio_gpu_crtc_helper_funcs);
plane->crtc = crtc;
@@ -456,7 +447,7 @@
int ret;
/* lookup object associated with res handle */
- obj = drm_gem_object_lookup(dev, file_priv, mode_cmd->handles[0]);
+ obj = drm_gem_object_lookup(file_priv, mode_cmd->handles[0]);
if (!obj)
return ERR_PTR(-EINVAL);
diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.c b/drivers/gpu/drm/virtio/virtgpu_drv.c
index 7f898cf..3cc7afa 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.c
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.c
@@ -42,10 +42,8 @@
static int virtio_gpu_probe(struct virtio_device *vdev)
{
-#ifdef CONFIG_VGA_CONSOLE
if (vgacon_text_force() && virtio_gpu_modeset == -1)
return -EINVAL;
-#endif
if (virtio_gpu_modeset == 0)
return -EINVAL;
diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
index 8f486f4..0a54f43 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.h
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
@@ -400,7 +400,7 @@
{
int r;
- r = ttm_bo_reserve(&bo->tbo, true, no_wait, false, NULL);
+ r = ttm_bo_reserve(&bo->tbo, true, no_wait, NULL);
if (unlikely(r != 0)) {
if (r != -ERESTARTSYS) {
struct virtio_gpu_device *qdev =
diff --git a/drivers/gpu/drm/virtio/virtgpu_gem.c b/drivers/gpu/drm/virtio/virtgpu_gem.c
index 1feb7ce..336a57f 100644
--- a/drivers/gpu/drm/virtio/virtgpu_gem.c
+++ b/drivers/gpu/drm/virtio/virtgpu_gem.c
@@ -130,7 +130,7 @@
struct drm_gem_object *gobj;
struct virtio_gpu_object *obj;
BUG_ON(!offset_p);
- gobj = drm_gem_object_lookup(dev, file_priv, handle);
+ gobj = drm_gem_object_lookup(file_priv, handle);
if (gobj == NULL)
return -ENOENT;
obj = gem_to_virtio_gpu_obj(gobj);
diff --git a/drivers/gpu/drm/virtio/virtgpu_ioctl.c b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
index b4de18e..c046903 100644
--- a/drivers/gpu/drm/virtio/virtgpu_ioctl.c
+++ b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
@@ -133,8 +133,7 @@
}
for (i = 0; i < exbuf->num_bo_handles; i++) {
- gobj = drm_gem_object_lookup(dev,
- drm_file, bo_handles[i]);
+ gobj = drm_gem_object_lookup(drm_file, bo_handles[i]);
if (!gobj) {
drm_free_large(bo_handles);
drm_free_large(buflist);
@@ -345,7 +344,7 @@
struct drm_gem_object *gobj = NULL;
struct virtio_gpu_object *qobj = NULL;
- gobj = drm_gem_object_lookup(dev, file_priv, ri->bo_handle);
+ gobj = drm_gem_object_lookup(file_priv, ri->bo_handle);
if (gobj == NULL)
return -ENOENT;
@@ -374,7 +373,7 @@
if (vgdev->has_virgl_3d == false)
return -ENOSYS;
- gobj = drm_gem_object_lookup(dev, file, args->bo_handle);
+ gobj = drm_gem_object_lookup(file, args->bo_handle);
if (gobj == NULL)
return -ENOENT;
@@ -418,7 +417,7 @@
int ret;
u32 offset = args->offset;
- gobj = drm_gem_object_lookup(dev, file, args->bo_handle);
+ gobj = drm_gem_object_lookup(file, args->bo_handle);
if (gobj == NULL)
return -ENOENT;
@@ -464,7 +463,7 @@
int ret;
bool nowait = false;
- gobj = drm_gem_object_lookup(dev, file, args->handle);
+ gobj = drm_gem_object_lookup(file, args->handle);
if (gobj == NULL)
return -ENOENT;
diff --git a/drivers/gpu/drm/virtio/virtgpu_object.c b/drivers/gpu/drm/virtio/virtgpu_object.c
index f300eba..1483dae 100644
--- a/drivers/gpu/drm/virtio/virtgpu_object.c
+++ b/drivers/gpu/drm/virtio/virtgpu_object.c
@@ -155,10 +155,10 @@
{
int r;
- r = ttm_bo_reserve(&bo->tbo, true, no_wait, false, NULL);
+ r = ttm_bo_reserve(&bo->tbo, true, no_wait, NULL);
if (unlikely(r != 0))
return r;
- r = ttm_bo_wait(&bo->tbo, true, true, no_wait);
+ r = ttm_bo_wait(&bo->tbo, true, no_wait);
ttm_bo_unreserve(&bo->tbo);
return r;
}
diff --git a/drivers/gpu/drm/virtio/virtgpu_ttm.c b/drivers/gpu/drm/virtio/virtgpu_ttm.c
index 9fd924cd..a058081 100644
--- a/drivers/gpu/drm/virtio/virtgpu_ttm.c
+++ b/drivers/gpu/drm/virtio/virtgpu_ttm.c
@@ -426,6 +426,8 @@
.io_mem_free = &virtio_gpu_ttm_io_mem_free,
.move_notify = &virtio_gpu_bo_move_notify,
.swap_notify = &virtio_gpu_bo_swap_notify,
+ .lru_tail = &ttm_bo_default_lru_tail,
+ .swap_lru_tail = &ttm_bo_default_swap_lru_tail,
};
int virtio_gpu_ttm_init(struct virtio_gpu_device *vgdev)
diff --git a/drivers/gpu/drm/vmwgfx/Makefile b/drivers/gpu/drm/vmwgfx/Makefile
index d281575..473d004 100644
--- a/drivers/gpu/drm/vmwgfx/Makefile
+++ b/drivers/gpu/drm/vmwgfx/Makefile
@@ -8,6 +8,6 @@
vmwgfx_fence.o vmwgfx_dmabuf.o vmwgfx_scrn.o vmwgfx_context.o \
vmwgfx_surface.o vmwgfx_prime.o vmwgfx_mob.o vmwgfx_shader.o \
vmwgfx_cmdbuf_res.o vmwgfx_cmdbuf.o vmwgfx_stdu.o \
- vmwgfx_cotable.o vmwgfx_so.o vmwgfx_binding.o
+ vmwgfx_cotable.o vmwgfx_so.o vmwgfx_binding.o vmwgfx_msg.o
obj-$(CONFIG_DRM_VMWGFX) := vmwgfx.o
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_buffer.c b/drivers/gpu/drm/vmwgfx/vmwgfx_buffer.c
index 3329f62..78b75ee 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_buffer.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_buffer.c
@@ -839,7 +839,7 @@
*/
static void vmw_swap_notify(struct ttm_buffer_object *bo)
{
- ttm_bo_wait(bo, false, false, false);
+ ttm_bo_wait(bo, false, false);
}
@@ -857,4 +857,6 @@
.fault_reserve_notify = &vmw_ttm_fault_reserve_notify,
.io_mem_reserve = &vmw_ttm_io_mem_reserve,
.io_mem_free = &vmw_ttm_io_mem_free,
+ .lru_tail = &ttm_bo_default_lru_tail,
+ .swap_lru_tail = &ttm_bo_default_swap_lru_tail,
};
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_cotable.c b/drivers/gpu/drm/vmwgfx/vmwgfx_cotable.c
index 092ea81..265c81e 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_cotable.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_cotable.c
@@ -421,9 +421,9 @@
}
bo = &buf->base;
- WARN_ON_ONCE(ttm_bo_reserve(bo, false, true, false, NULL));
+ WARN_ON_ONCE(ttm_bo_reserve(bo, false, true, NULL));
- ret = ttm_bo_wait(old_bo, false, false, false);
+ ret = ttm_bo_wait(old_bo, false, false);
if (unlikely(ret != 0)) {
DRM_ERROR("Failed waiting for cotable unbind.\n");
goto out_wait;
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_dmabuf.c b/drivers/gpu/drm/vmwgfx/vmwgfx_dmabuf.c
index 299925a..9b078a4 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_dmabuf.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_dmabuf.c
@@ -56,7 +56,7 @@
vmw_execbuf_release_pinned_bo(dev_priv);
- ret = ttm_bo_reserve(bo, interruptible, false, false, NULL);
+ ret = ttm_bo_reserve(bo, interruptible, false, NULL);
if (unlikely(ret != 0))
goto err;
@@ -98,7 +98,7 @@
vmw_execbuf_release_pinned_bo(dev_priv);
- ret = ttm_bo_reserve(bo, interruptible, false, false, NULL);
+ ret = ttm_bo_reserve(bo, interruptible, false, NULL);
if (unlikely(ret != 0))
goto err;
@@ -174,7 +174,7 @@
return ret;
vmw_execbuf_release_pinned_bo(dev_priv);
- ret = ttm_bo_reserve(bo, interruptible, false, false, NULL);
+ ret = ttm_bo_reserve(bo, interruptible, false, NULL);
if (unlikely(ret != 0))
goto err_unlock;
@@ -225,7 +225,7 @@
if (unlikely(ret != 0))
return ret;
- ret = ttm_bo_reserve(bo, interruptible, false, false, NULL);
+ ret = ttm_bo_reserve(bo, interruptible, false, NULL);
if (unlikely(ret != 0))
goto err;
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c
index 6cbb7d4b..9fcd820 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c
@@ -1,6 +1,6 @@
/**************************************************************************
*
- * Copyright © 2009-2015 VMware, Inc., Palo Alto, CA., USA
+ * Copyright © 2009-2016 VMware, Inc., Palo Alto, CA., USA
* All Rights Reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining a
@@ -44,6 +44,12 @@
#define VMW_MIN_INITIAL_WIDTH 800
#define VMW_MIN_INITIAL_HEIGHT 600
+#ifndef VMWGFX_GIT_VERSION
+#define VMWGFX_GIT_VERSION "Unknown"
+#endif
+
+#define VMWGFX_REPO "In Tree"
+
/**
* Fully encoded drm commands. Might move to vmw_drm.h
@@ -326,7 +332,7 @@
if (unlikely(ret != 0))
return ret;
- ret = ttm_bo_reserve(&vbo->base, false, true, false, NULL);
+ ret = ttm_bo_reserve(&vbo->base, false, true, NULL);
BUG_ON(ret != 0);
vmw_bo_pin_reserved(vbo, true);
@@ -613,6 +619,7 @@
uint32_t svga_id;
enum vmw_res_type i;
bool refuse_dma = false;
+ char host_log[100] = {0};
dev_priv = kzalloc(sizeof(*dev_priv), GFP_KERNEL);
if (unlikely(dev_priv == NULL)) {
@@ -628,6 +635,7 @@
mutex_init(&dev_priv->cmdbuf_mutex);
mutex_init(&dev_priv->release_mutex);
mutex_init(&dev_priv->binding_mutex);
+ mutex_init(&dev_priv->global_kms_state_mutex);
rwlock_init(&dev_priv->resource_lock);
ttm_lock_init(&dev_priv->reservation_sem);
spin_lock_init(&dev_priv->hw_lock);
@@ -873,6 +881,16 @@
DRM_INFO("DX: %s\n", dev_priv->has_dx ? "yes." : "no.");
+ snprintf(host_log, sizeof(host_log), "vmwgfx: %s-%s",
+ VMWGFX_REPO, VMWGFX_GIT_VERSION);
+ vmw_host_log(host_log);
+
+ memset(host_log, 0, sizeof(host_log));
+ snprintf(host_log, sizeof(host_log), "vmwgfx: Module Version: %d.%d.%d",
+ VMWGFX_DRIVER_MAJOR, VMWGFX_DRIVER_MINOR,
+ VMWGFX_DRIVER_PATCHLEVEL);
+ vmw_host_log(host_log);
+
if (dev_priv->enable_fb) {
vmw_fifo_resource_inc(dev_priv);
vmw_svga_enable(dev_priv);
@@ -1530,10 +1548,8 @@
{
int ret;
-#ifdef CONFIG_VGA_CONSOLE
if (vgacon_text_force())
return -EINVAL;
-#endif
ret = drm_pci_init(&driver, &vmw_pci_driver);
if (ret)
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h
index 019a6ca..1980e2a 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h
@@ -412,6 +412,7 @@
struct drm_property *implicit_placement_property;
unsigned num_implicit;
struct vmw_framebuffer *implicit_fb;
+ struct mutex global_kms_state_mutex;
/*
* Context and surface management.
@@ -1234,4 +1235,10 @@
{
WRITE_ONCE(*addr, value);
}
+
+/**
+ * Add vmw_msg module function
+ */
+extern int vmw_host_log(const char *log);
+
#endif
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c b/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c
index 4742ec4..55231cc 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c
@@ -98,7 +98,7 @@
kmap_offset = 0;
kmap_num = (width*height*4 + PAGE_SIZE - 1) >> PAGE_SHIFT;
- ret = ttm_bo_reserve(&dmabuf->base, true, false, false, NULL);
+ ret = ttm_bo_reserve(&dmabuf->base, true, false, NULL);
if (unlikely(ret != 0)) {
DRM_ERROR("reserve failed\n");
return -EINVAL;
@@ -318,7 +318,7 @@
kmap_offset = cmd->dma.guest.ptr.offset >> PAGE_SHIFT;
kmap_num = (64*64*4) >> PAGE_SHIFT;
- ret = ttm_bo_reserve(bo, true, false, false, NULL);
+ ret = ttm_bo_reserve(bo, true, false, NULL);
if (unlikely(ret != 0)) {
DRM_ERROR("reserve failed\n");
return;
@@ -1859,7 +1859,7 @@
struct ttm_buffer_object *bo = &buf->base;
int ret;
- ttm_bo_reserve(bo, false, false, interruptible, NULL);
+ ttm_bo_reserve(bo, false, false, NULL);
ret = vmw_validate_single_buffer(dev_priv, bo, interruptible,
validate_as_mob);
if (ret)
@@ -2143,13 +2143,13 @@
void vmw_kms_del_active(struct vmw_private *dev_priv,
struct vmw_display_unit *du)
{
- lockdep_assert_held_once(&dev_priv->dev->mode_config.mutex);
-
+ mutex_lock(&dev_priv->global_kms_state_mutex);
if (du->active_implicit) {
if (--(dev_priv->num_implicit) == 0)
dev_priv->implicit_fb = NULL;
du->active_implicit = false;
}
+ mutex_unlock(&dev_priv->global_kms_state_mutex);
}
/**
@@ -2165,8 +2165,7 @@
struct vmw_display_unit *du,
struct vmw_framebuffer *vfb)
{
- lockdep_assert_held_once(&dev_priv->dev->mode_config.mutex);
-
+ mutex_lock(&dev_priv->global_kms_state_mutex);
WARN_ON_ONCE(!dev_priv->num_implicit && dev_priv->implicit_fb);
if (!du->active_implicit && du->is_implicit) {
@@ -2174,6 +2173,7 @@
du->active_implicit = true;
dev_priv->num_implicit++;
}
+ mutex_unlock(&dev_priv->global_kms_state_mutex);
}
/**
@@ -2190,16 +2190,13 @@
struct drm_crtc *crtc)
{
struct vmw_display_unit *du = vmw_crtc_to_du(crtc);
+ bool ret;
- lockdep_assert_held_once(&dev_priv->dev->mode_config.mutex);
+ mutex_lock(&dev_priv->global_kms_state_mutex);
+ ret = !du->is_implicit || dev_priv->num_implicit == 1;
+ mutex_unlock(&dev_priv->global_kms_state_mutex);
- if (!du->is_implicit)
- return true;
-
- if (dev_priv->num_implicit != 1)
- return false;
-
- return true;
+ return ret;
}
/**
@@ -2214,16 +2211,18 @@
struct vmw_display_unit *du = vmw_crtc_to_du(crtc);
struct vmw_framebuffer *vfb;
- lockdep_assert_held_once(&dev_priv->dev->mode_config.mutex);
+ mutex_lock(&dev_priv->global_kms_state_mutex);
if (!du->is_implicit)
- return;
+ goto out_unlock;
vfb = vmw_framebuffer_to_vfb(crtc->primary->fb);
WARN_ON_ONCE(dev_priv->num_implicit != 1 &&
dev_priv->implicit_fb != vfb);
dev_priv->implicit_fb = vfb;
+out_unlock:
+ mutex_unlock(&dev_priv->global_kms_state_mutex);
}
/**
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_mob.c b/drivers/gpu/drm/vmwgfx/vmwgfx_mob.c
index 23db160..b6126a5 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_mob.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_mob.c
@@ -222,7 +222,7 @@
if (bo) {
int ret;
- ret = ttm_bo_reserve(bo, false, true, false, NULL);
+ ret = ttm_bo_reserve(bo, false, true, NULL);
BUG_ON(ret != 0);
vmw_fence_single_bo(bo, NULL);
@@ -262,7 +262,7 @@
if (unlikely(ret != 0))
goto out_no_bo;
- ret = ttm_bo_reserve(batch->otable_bo, false, true, false, NULL);
+ ret = ttm_bo_reserve(batch->otable_bo, false, true, NULL);
BUG_ON(ret != 0);
ret = vmw_bo_driver.ttm_tt_populate(batch->otable_bo->ttm);
if (unlikely(ret != 0))
@@ -357,7 +357,7 @@
vmw_takedown_otable_base(dev_priv, i,
&batch->otables[i]);
- ret = ttm_bo_reserve(bo, false, true, false, NULL);
+ ret = ttm_bo_reserve(bo, false, true, NULL);
BUG_ON(ret != 0);
vmw_fence_single_bo(bo, NULL);
@@ -440,7 +440,7 @@
if (unlikely(ret != 0))
return ret;
- ret = ttm_bo_reserve(mob->pt_bo, false, true, false, NULL);
+ ret = ttm_bo_reserve(mob->pt_bo, false, true, NULL);
BUG_ON(ret != 0);
ret = vmw_bo_driver.ttm_tt_populate(mob->pt_bo->ttm);
@@ -545,7 +545,7 @@
const struct vmw_sg_table *vsgt;
int ret;
- ret = ttm_bo_reserve(bo, false, true, false, NULL);
+ ret = ttm_bo_reserve(bo, false, true, NULL);
BUG_ON(ret != 0);
vsgt = vmw_bo_sg_table(bo);
@@ -595,7 +595,7 @@
struct ttm_buffer_object *bo = mob->pt_bo;
if (bo) {
- ret = ttm_bo_reserve(bo, false, true, false, NULL);
+ ret = ttm_bo_reserve(bo, false, true, NULL);
/*
* Noone else should be using this buffer.
*/
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_msg.c b/drivers/gpu/drm/vmwgfx/vmwgfx_msg.c
new file mode 100644
index 0000000..6de283c
--- /dev/null
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_msg.c
@@ -0,0 +1,416 @@
+/*
+ * Copyright © 2016 VMware, Inc., Palo Alto, CA., USA
+ * All Rights Reserved.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sub license, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the
+ * next paragraph) shall be included in all copies or substantial portions
+ * of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDERS, AUTHORS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM,
+ * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
+ * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
+ * USE OR OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+
+#include <linux/slab.h>
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <asm/hypervisor.h>
+#include "drmP.h"
+#include "vmwgfx_msg.h"
+
+
+#define MESSAGE_STATUS_SUCCESS 0x0001
+#define MESSAGE_STATUS_DORECV 0x0002
+#define MESSAGE_STATUS_CPT 0x0010
+#define MESSAGE_STATUS_HB 0x0080
+
+#define RPCI_PROTOCOL_NUM 0x49435052
+#define GUESTMSG_FLAG_COOKIE 0x80000000
+
+#define RETRIES 3
+
+#define VMW_HYPERVISOR_MAGIC 0x564D5868
+#define VMW_HYPERVISOR_PORT 0x5658
+#define VMW_HYPERVISOR_HB_PORT 0x5659
+
+#define VMW_PORT_CMD_MSG 30
+#define VMW_PORT_CMD_HB_MSG 0
+#define VMW_PORT_CMD_OPEN_CHANNEL (MSG_TYPE_OPEN << 16 | VMW_PORT_CMD_MSG)
+#define VMW_PORT_CMD_CLOSE_CHANNEL (MSG_TYPE_CLOSE << 16 | VMW_PORT_CMD_MSG)
+#define VMW_PORT_CMD_SENDSIZE (MSG_TYPE_SENDSIZE << 16 | VMW_PORT_CMD_MSG)
+#define VMW_PORT_CMD_RECVSIZE (MSG_TYPE_RECVSIZE << 16 | VMW_PORT_CMD_MSG)
+#define VMW_PORT_CMD_RECVSTATUS (MSG_TYPE_RECVSTATUS << 16 | VMW_PORT_CMD_MSG)
+
+#define HIGH_WORD(X) ((X & 0xFFFF0000) >> 16)
+
+static u32 vmw_msg_enabled = 1;
+
+enum rpc_msg_type {
+ MSG_TYPE_OPEN,
+ MSG_TYPE_SENDSIZE,
+ MSG_TYPE_SENDPAYLOAD,
+ MSG_TYPE_RECVSIZE,
+ MSG_TYPE_RECVPAYLOAD,
+ MSG_TYPE_RECVSTATUS,
+ MSG_TYPE_CLOSE,
+};
+
+struct rpc_channel {
+ u16 channel_id;
+ u32 cookie_high;
+ u32 cookie_low;
+};
+
+
+
+/**
+ * vmw_open_channel
+ *
+ * @channel: RPC channel
+ * @protocol:
+ *
+ * Returns: 0 on success
+ */
+static int vmw_open_channel(struct rpc_channel *channel, unsigned int protocol)
+{
+ unsigned long eax, ebx, ecx, edx, si = 0, di = 0;
+
+ VMW_PORT(VMW_PORT_CMD_OPEN_CHANNEL,
+ (protocol | GUESTMSG_FLAG_COOKIE), si, di,
+ VMW_HYPERVISOR_PORT,
+ VMW_HYPERVISOR_MAGIC,
+ eax, ebx, ecx, edx, si, di);
+
+ if ((HIGH_WORD(ecx) & MESSAGE_STATUS_SUCCESS) == 0)
+ return -EINVAL;
+
+ channel->channel_id = HIGH_WORD(edx);
+ channel->cookie_high = si;
+ channel->cookie_low = di;
+
+ return 0;
+}
+
+
+
+/**
+ * vmw_close_channel
+ *
+ * @channel: RPC channel
+ *
+ * Returns: 0 on success
+ */
+static int vmw_close_channel(struct rpc_channel *channel)
+{
+ unsigned long eax, ebx, ecx, edx, si, di;
+
+ /* Set up additional parameters */
+ si = channel->cookie_high;
+ di = channel->cookie_low;
+
+ VMW_PORT(VMW_PORT_CMD_CLOSE_CHANNEL,
+ 0, si, di,
+ (VMW_HYPERVISOR_PORT | (channel->channel_id << 16)),
+ VMW_HYPERVISOR_MAGIC,
+ eax, ebx, ecx, edx, si, di);
+
+ if ((HIGH_WORD(ecx) & MESSAGE_STATUS_SUCCESS) == 0)
+ return -EINVAL;
+
+ return 0;
+}
+
+
+
+/**
+ * vmw_send_msg: Sends a message to the host
+ *
+ * @channel: RPC channel
+ * @logmsg: NULL terminated string
+ *
+ * Returns: 0 on success
+ */
+static int vmw_send_msg(struct rpc_channel *channel, const char *msg)
+{
+ unsigned long eax, ebx, ecx, edx, si, di, bp;
+ size_t msg_len = strlen(msg);
+ int retries = 0;
+
+
+ while (retries < RETRIES) {
+ retries++;
+
+ /* Set up additional parameters */
+ si = channel->cookie_high;
+ di = channel->cookie_low;
+
+ VMW_PORT(VMW_PORT_CMD_SENDSIZE,
+ msg_len, si, di,
+ VMW_HYPERVISOR_PORT | (channel->channel_id << 16),
+ VMW_HYPERVISOR_MAGIC,
+ eax, ebx, ecx, edx, si, di);
+
+ if ((HIGH_WORD(ecx) & MESSAGE_STATUS_SUCCESS) == 0 ||
+ (HIGH_WORD(ecx) & MESSAGE_STATUS_HB) == 0) {
+ /* Expected success + high-bandwidth. Give up. */
+ return -EINVAL;
+ }
+
+ /* Send msg */
+ si = (uintptr_t) msg;
+ di = channel->cookie_low;
+ bp = channel->cookie_high;
+
+ VMW_PORT_HB_OUT(
+ (MESSAGE_STATUS_SUCCESS << 16) | VMW_PORT_CMD_HB_MSG,
+ msg_len, si, di,
+ VMW_HYPERVISOR_HB_PORT | (channel->channel_id << 16),
+ VMW_HYPERVISOR_MAGIC, bp,
+ eax, ebx, ecx, edx, si, di);
+
+ if ((HIGH_WORD(ebx) & MESSAGE_STATUS_SUCCESS) != 0) {
+ return 0;
+ } else if ((HIGH_WORD(ebx) & MESSAGE_STATUS_CPT) != 0) {
+ /* A checkpoint occurred. Retry. */
+ continue;
+ } else {
+ break;
+ }
+ }
+
+ return -EINVAL;
+}
+
+
+
+/**
+ * vmw_recv_msg: Receives a message from the host
+ *
+ * Note: It is the caller's responsibility to call kfree() on msg.
+ *
+ * @channel: channel opened by vmw_open_channel
+ * @msg: [OUT] message received from the host
+ * @msg_len: message length
+ */
+static int vmw_recv_msg(struct rpc_channel *channel, void **msg,
+ size_t *msg_len)
+{
+ unsigned long eax, ebx, ecx, edx, si, di, bp;
+ char *reply;
+ size_t reply_len;
+ int retries = 0;
+
+
+ *msg_len = 0;
+ *msg = NULL;
+
+ while (retries < RETRIES) {
+ retries++;
+
+ /* Set up additional parameters */
+ si = channel->cookie_high;
+ di = channel->cookie_low;
+
+ VMW_PORT(VMW_PORT_CMD_RECVSIZE,
+ 0, si, di,
+ (VMW_HYPERVISOR_PORT | (channel->channel_id << 16)),
+ VMW_HYPERVISOR_MAGIC,
+ eax, ebx, ecx, edx, si, di);
+
+ if ((HIGH_WORD(ecx) & MESSAGE_STATUS_SUCCESS) == 0 ||
+ (HIGH_WORD(ecx) & MESSAGE_STATUS_HB) == 0) {
+ DRM_ERROR("Failed to get reply size\n");
+ return -EINVAL;
+ }
+
+ /* No reply available. This is okay. */
+ if ((HIGH_WORD(ecx) & MESSAGE_STATUS_DORECV) == 0)
+ return 0;
+
+ reply_len = ebx;
+ reply = kzalloc(reply_len + 1, GFP_KERNEL);
+ if (reply == NULL) {
+ DRM_ERROR("Cannot allocate memory for reply\n");
+ return -ENOMEM;
+ }
+
+
+ /* Receive buffer */
+ si = channel->cookie_high;
+ di = (uintptr_t) reply;
+ bp = channel->cookie_low;
+
+ VMW_PORT_HB_IN(
+ (MESSAGE_STATUS_SUCCESS << 16) | VMW_PORT_CMD_HB_MSG,
+ reply_len, si, di,
+ VMW_HYPERVISOR_HB_PORT | (channel->channel_id << 16),
+ VMW_HYPERVISOR_MAGIC, bp,
+ eax, ebx, ecx, edx, si, di);
+
+ if ((HIGH_WORD(ebx) & MESSAGE_STATUS_SUCCESS) == 0) {
+ kfree(reply);
+
+ if ((HIGH_WORD(ebx) & MESSAGE_STATUS_CPT) != 0) {
+ /* A checkpoint occurred. Retry. */
+ continue;
+ }
+
+ return -EINVAL;
+ }
+
+ reply[reply_len] = '\0';
+
+
+ /* Ack buffer */
+ si = channel->cookie_high;
+ di = channel->cookie_low;
+
+ VMW_PORT(VMW_PORT_CMD_RECVSTATUS,
+ MESSAGE_STATUS_SUCCESS, si, di,
+ (VMW_HYPERVISOR_PORT | (channel->channel_id << 16)),
+ VMW_HYPERVISOR_MAGIC,
+ eax, ebx, ecx, edx, si, di);
+
+ if ((HIGH_WORD(ecx) & MESSAGE_STATUS_SUCCESS) == 0) {
+ kfree(reply);
+
+ if ((HIGH_WORD(ecx) & MESSAGE_STATUS_CPT) != 0) {
+ /* A checkpoint occurred. Retry. */
+ continue;
+ }
+
+ return -EINVAL;
+ }
+
+ break;
+ }
+
+ *msg_len = reply_len;
+ *msg = reply;
+
+ return 0;
+}
+
+
+/**
+ * vmw_host_get_guestinfo: Gets a GuestInfo parameter
+ *
+ * Gets the value of a GuestInfo.* parameter. The value returned will be in
+ * a string, and it is up to the caller to post-process.
+ *
+ * @guest_info_param: Parameter to get, e.g. GuestInfo.svga.gl3
+ * @buffer: if NULL, *reply_len will contain reply size.
+ * @length: size of the reply_buf. Set to size of reply upon return
+ *
+ * Returns: 0 on success
+ */
+int vmw_host_get_guestinfo(const char *guest_info_param,
+ char *buffer, size_t *length)
+{
+ struct rpc_channel channel;
+ char *msg, *reply = NULL;
+ size_t msg_len, reply_len = 0;
+ int ret = 0;
+
+
+ if (!vmw_msg_enabled)
+ return -ENODEV;
+
+ if (!guest_info_param || !length)
+ return -EINVAL;
+
+ msg_len = strlen(guest_info_param) + strlen("info-get ") + 1;
+ msg = kzalloc(msg_len, GFP_KERNEL);
+ if (msg == NULL) {
+ DRM_ERROR("Cannot allocate memory to get %s", guest_info_param);
+ return -ENOMEM;
+ }
+
+ sprintf(msg, "info-get %s", guest_info_param);
+
+ if (vmw_open_channel(&channel, RPCI_PROTOCOL_NUM) ||
+ vmw_send_msg(&channel, msg) ||
+ vmw_recv_msg(&channel, (void *) &reply, &reply_len) ||
+ vmw_close_channel(&channel)) {
+ DRM_ERROR("Failed to get %s", guest_info_param);
+
+ ret = -EINVAL;
+ }
+
+ if (buffer && reply && reply_len > 0) {
+ /* Remove reply code, which are the first 2 characters of
+ * the reply
+ */
+ reply_len = max(reply_len - 2, (size_t) 0);
+ reply_len = min(reply_len, *length);
+
+ if (reply_len > 0)
+ memcpy(buffer, reply + 2, reply_len);
+ }
+
+ *length = reply_len;
+
+ kfree(reply);
+ kfree(msg);
+
+ return ret;
+}
+
+
+
+/**
+ * vmw_host_log: Sends a log message to the host
+ *
+ * @log: NULL terminated string
+ *
+ * Returns: 0 on success
+ */
+int vmw_host_log(const char *log)
+{
+ struct rpc_channel channel;
+ char *msg;
+ int msg_len;
+ int ret = 0;
+
+
+ if (!vmw_msg_enabled)
+ return -ENODEV;
+
+ if (!log)
+ return ret;
+
+ msg_len = strlen(log) + strlen("log ") + 1;
+ msg = kzalloc(msg_len, GFP_KERNEL);
+ if (msg == NULL) {
+ DRM_ERROR("Cannot allocate memory for log message\n");
+ return -ENOMEM;
+ }
+
+ sprintf(msg, "log %s", log);
+
+ if (vmw_open_channel(&channel, RPCI_PROTOCOL_NUM) ||
+ vmw_send_msg(&channel, msg) ||
+ vmw_close_channel(&channel)) {
+ DRM_ERROR("Failed to send log\n");
+
+ ret = -EINVAL;
+ }
+
+ kfree(msg);
+
+ return ret;
+}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_msg.h b/drivers/gpu/drm/vmwgfx/vmwgfx_msg.h
new file mode 100644
index 0000000..557a033
--- /dev/null
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_msg.h
@@ -0,0 +1,191 @@
+/*
+ * Copyright (C) 2016, VMware, Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, GOOD TITLE or
+ * NON INFRINGEMENT. See the GNU General Public License for more
+ * details.
+ *
+ * Based on code from vmware.c and vmmouse.c.
+ * Author:
+ * Sinclair Yeh <syeh@vmware.com>
+ */
+#ifndef _VMWGFX_MSG_H
+#define _VMWGFX_MSG_H
+
+
+/**
+ * Hypervisor-specific bi-directional communication channel. Should never
+ * execute on bare metal hardware. The caller must make sure to check for
+ * supported hypervisor before using these macros.
+ *
+ * The last two parameters are both input and output and must be initialized.
+ *
+ * @cmd: [IN] Message Cmd
+ * @in_ebx: [IN] Message Len, through EBX
+ * @in_si: [IN] Input argument through SI, set to 0 if not used
+ * @in_di: [IN] Input argument through DI, set ot 0 if not used
+ * @port_num: [IN] port number + [channel id]
+ * @magic: [IN] hypervisor magic value
+ * @eax: [OUT] value of EAX register
+ * @ebx: [OUT] e.g. status from an HB message status command
+ * @ecx: [OUT] e.g. status from a non-HB message status command
+ * @edx: [OUT] e.g. channel id
+ * @si: [OUT]
+ * @di: [OUT]
+ */
+#define VMW_PORT(cmd, in_ebx, in_si, in_di, \
+ port_num, magic, \
+ eax, ebx, ecx, edx, si, di) \
+({ \
+ asm volatile ("inl %%dx, %%eax;" : \
+ "=a"(eax), \
+ "=b"(ebx), \
+ "=c"(ecx), \
+ "=d"(edx), \
+ "=S"(si), \
+ "=D"(di) : \
+ "a"(magic), \
+ "b"(in_ebx), \
+ "c"(cmd), \
+ "d"(port_num), \
+ "S"(in_si), \
+ "D"(in_di) : \
+ "memory"); \
+})
+
+
+/**
+ * Hypervisor-specific bi-directional communication channel. Should never
+ * execute on bare metal hardware. The caller must make sure to check for
+ * supported hypervisor before using these macros.
+ *
+ * The last 3 parameters are both input and output and must be initialized.
+ *
+ * @cmd: [IN] Message Cmd
+ * @in_ecx: [IN] Message Len, through ECX
+ * @in_si: [IN] Input argument through SI, set to 0 if not used
+ * @in_di: [IN] Input argument through DI, set to 0 if not used
+ * @port_num: [IN] port number + [channel id]
+ * @magic: [IN] hypervisor magic value
+ * @bp: [IN]
+ * @eax: [OUT] value of EAX register
+ * @ebx: [OUT] e.g. status from an HB message status command
+ * @ecx: [OUT] e.g. status from a non-HB message status command
+ * @edx: [OUT] e.g. channel id
+ * @si: [OUT]
+ * @di: [OUT]
+ */
+#ifdef __x86_64__
+
+#define VMW_PORT_HB_OUT(cmd, in_ecx, in_si, in_di, \
+ port_num, magic, bp, \
+ eax, ebx, ecx, edx, si, di) \
+({ \
+ asm volatile ("push %%rbp;" \
+ "mov %12, %%rbp;" \
+ "rep outsb;" \
+ "pop %%rbp;" : \
+ "=a"(eax), \
+ "=b"(ebx), \
+ "=c"(ecx), \
+ "=d"(edx), \
+ "=S"(si), \
+ "=D"(di) : \
+ "a"(magic), \
+ "b"(cmd), \
+ "c"(in_ecx), \
+ "d"(port_num), \
+ "S"(in_si), \
+ "D"(in_di), \
+ "r"(bp) : \
+ "memory", "cc"); \
+})
+
+
+#define VMW_PORT_HB_IN(cmd, in_ecx, in_si, in_di, \
+ port_num, magic, bp, \
+ eax, ebx, ecx, edx, si, di) \
+({ \
+ asm volatile ("push %%rbp;" \
+ "mov %12, %%rbp;" \
+ "rep insb;" \
+ "pop %%rbp" : \
+ "=a"(eax), \
+ "=b"(ebx), \
+ "=c"(ecx), \
+ "=d"(edx), \
+ "=S"(si), \
+ "=D"(di) : \
+ "a"(magic), \
+ "b"(cmd), \
+ "c"(in_ecx), \
+ "d"(port_num), \
+ "S"(in_si), \
+ "D"(in_di), \
+ "r"(bp) : \
+ "memory", "cc"); \
+})
+
+#else
+
+/* In the 32-bit version of this macro, we use "m" because there is no
+ * more register left for bp
+ */
+#define VMW_PORT_HB_OUT(cmd, in_ecx, in_si, in_di, \
+ port_num, magic, bp, \
+ eax, ebx, ecx, edx, si, di) \
+({ \
+ asm volatile ("push %%ebp;" \
+ "mov %12, %%ebp;" \
+ "rep outsb;" \
+ "pop %%ebp;" : \
+ "=a"(eax), \
+ "=b"(ebx), \
+ "=c"(ecx), \
+ "=d"(edx), \
+ "=S"(si), \
+ "=D"(di) : \
+ "a"(magic), \
+ "b"(cmd), \
+ "c"(in_ecx), \
+ "d"(port_num), \
+ "S"(in_si), \
+ "D"(in_di), \
+ "m"(bp) : \
+ "memory", "cc"); \
+})
+
+
+#define VMW_PORT_HB_IN(cmd, in_ecx, in_si, in_di, \
+ port_num, magic, bp, \
+ eax, ebx, ecx, edx, si, di) \
+({ \
+ asm volatile ("push %%ebp;" \
+ "mov %12, %%ebp;" \
+ "rep insb;" \
+ "pop %%ebp" : \
+ "=a"(eax), \
+ "=b"(ebx), \
+ "=c"(ecx), \
+ "=d"(edx), \
+ "=S"(si), \
+ "=D"(di) : \
+ "a"(magic), \
+ "b"(cmd), \
+ "c"(in_ecx), \
+ "d"(port_num), \
+ "S"(in_si), \
+ "D"(in_di), \
+ "m"(bp) : \
+ "memory", "cc"); \
+})
+#endif /* #if __x86_64__ */
+
+#endif
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c b/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c
index e57667c..6a328d5 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c
@@ -129,7 +129,7 @@
if (res->backup) {
struct ttm_buffer_object *bo = &res->backup->base;
- ttm_bo_reserve(bo, false, false, false, NULL);
+ ttm_bo_reserve(bo, false, false, NULL);
if (!list_empty(&res->mob_head) &&
res->func->unbind != NULL) {
struct ttm_validate_buffer val_buf;
@@ -1512,7 +1512,7 @@
list_del_init(&res->mob_head);
}
- (void) ttm_bo_wait(bo, false, false, false);
+ (void) ttm_bo_wait(bo, false, false);
}
}
@@ -1605,7 +1605,7 @@
if (fence != NULL)
vmw_fence_obj_unreference(&fence);
- (void) ttm_bo_wait(bo, false, false, false);
+ (void) ttm_bo_wait(bo, false, false);
} else
mutex_unlock(&dev_priv->binding_mutex);
@@ -1717,8 +1717,7 @@
if (res->backup) {
vbo = res->backup;
- ttm_bo_reserve(&vbo->base, interruptible, false, false,
- NULL);
+ ttm_bo_reserve(&vbo->base, interruptible, false, NULL);
if (!vbo->pin_count) {
ret = ttm_bo_validate
(&vbo->base,
@@ -1773,7 +1772,7 @@
if (--res->pin_count == 0 && res->backup) {
struct vmw_dma_buffer *vbo = res->backup;
- ttm_bo_reserve(&vbo->base, false, false, false, NULL);
+ ttm_bo_reserve(&vbo->base, false, false, NULL);
vmw_bo_pin_reserved(vbo, false);
ttm_bo_unreserve(&vbo->base);
}
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_scrn.c b/drivers/gpu/drm/vmwgfx/vmwgfx_scrn.c
index 0ea22fd..b74eae2 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_scrn.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_scrn.c
@@ -285,14 +285,17 @@
}
/* Only one active implicit frame-buffer at a time. */
+ mutex_lock(&dev_priv->global_kms_state_mutex);
if (sou->base.is_implicit &&
dev_priv->implicit_fb && vfb &&
!(dev_priv->num_implicit == 1 &&
sou->base.active_implicit) &&
dev_priv->implicit_fb != vfb) {
+ mutex_unlock(&dev_priv->global_kms_state_mutex);
DRM_ERROR("Multiple implicit framebuffers not supported.\n");
return -EINVAL;
}
+ mutex_unlock(&dev_priv->global_kms_state_mutex);
/* since they always map one to one these are safe */
connector = &sou->base.connector;
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_shader.c b/drivers/gpu/drm/vmwgfx/vmwgfx_shader.c
index fd47547..92f8b1d 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_shader.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_shader.c
@@ -988,7 +988,7 @@
if (unlikely(ret != 0))
goto out;
- ret = ttm_bo_reserve(&buf->base, false, true, false, NULL);
+ ret = ttm_bo_reserve(&buf->base, false, true, NULL);
if (unlikely(ret != 0))
goto no_reserve;
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_stdu.c b/drivers/gpu/drm/vmwgfx/vmwgfx_stdu.c
index b949102..9ca818f 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_stdu.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_stdu.c
@@ -553,12 +553,15 @@
}
/* Only one active implicit frame-buffer at a time. */
+ mutex_lock(&dev_priv->global_kms_state_mutex);
if (!turning_off && stdu->base.is_implicit && dev_priv->implicit_fb &&
!(dev_priv->num_implicit == 1 && stdu->base.active_implicit)
&& dev_priv->implicit_fb != vfb) {
+ mutex_unlock(&dev_priv->global_kms_state_mutex);
DRM_ERROR("Multiple implicit framebuffers not supported.\n");
return -EINVAL;
}
+ mutex_unlock(&dev_priv->global_kms_state_mutex);
/* Since they always map one to one these are safe */
connector = &stdu->base.connector;
diff --git a/drivers/hsi/controllers/Kconfig b/drivers/hsi/controllers/Kconfig
index 6aba278..48e4eda 100644
--- a/drivers/hsi/controllers/Kconfig
+++ b/drivers/hsi/controllers/Kconfig
@@ -5,15 +5,11 @@
config OMAP_SSI
tristate "OMAP SSI hardware driver"
- depends on HSI && OF && (ARCH_OMAP3 || (ARM && COMPILE_TEST))
+ depends on HSI && OF && ARM && COMMON_CLK
+ depends on ARCH_OMAP3 || COMPILE_TEST
---help---
SSI is a legacy version of HSI. It is usually used to connect
an application engine with a cellular modem.
If you say Y here, you will enable the OMAP SSI hardware driver.
If unsure, say N.
-
-config OMAP_SSI_PORT
- tristate
- default m if OMAP_SSI=m
- default y if OMAP_SSI=y
diff --git a/drivers/hsi/controllers/Makefile b/drivers/hsi/controllers/Makefile
index d2665cf..7aba9c7 100644
--- a/drivers/hsi/controllers/Makefile
+++ b/drivers/hsi/controllers/Makefile
@@ -2,5 +2,5 @@
# Makefile for HSI controllers drivers
#
-obj-$(CONFIG_OMAP_SSI) += omap_ssi.o
-obj-$(CONFIG_OMAP_SSI_PORT) += omap_ssi_port.o
+omap_ssi-objs += omap_ssi_core.o omap_ssi_port.o
+obj-$(CONFIG_OMAP_SSI) += omap_ssi.o
diff --git a/drivers/hsi/controllers/omap_ssi.c b/drivers/hsi/controllers/omap_ssi.c
deleted file mode 100644
index 27b91f1..0000000
--- a/drivers/hsi/controllers/omap_ssi.c
+++ /dev/null
@@ -1,610 +0,0 @@
-/* OMAP SSI driver.
- *
- * Copyright (C) 2010 Nokia Corporation. All rights reserved.
- * Copyright (C) 2014 Sebastian Reichel <sre@kernel.org>
- *
- * Contact: Carlos Chinea <carlos.chinea@nokia.com>
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * version 2 as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
- * 02110-1301 USA
- */
-
-#include <linux/compiler.h>
-#include <linux/err.h>
-#include <linux/ioport.h>
-#include <linux/io.h>
-#include <linux/gpio.h>
-#include <linux/clk.h>
-#include <linux/device.h>
-#include <linux/platform_device.h>
-#include <linux/dma-mapping.h>
-#include <linux/dmaengine.h>
-#include <linux/delay.h>
-#include <linux/seq_file.h>
-#include <linux/scatterlist.h>
-#include <linux/interrupt.h>
-#include <linux/spinlock.h>
-#include <linux/debugfs.h>
-#include <linux/pm_runtime.h>
-#include <linux/of_platform.h>
-#include <linux/hsi/hsi.h>
-#include <linux/idr.h>
-
-#include "omap_ssi_regs.h"
-#include "omap_ssi.h"
-
-/* For automatically allocated device IDs */
-static DEFINE_IDA(platform_omap_ssi_ida);
-
-#ifdef CONFIG_DEBUG_FS
-static int ssi_debug_show(struct seq_file *m, void *p __maybe_unused)
-{
- struct hsi_controller *ssi = m->private;
- struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
- void __iomem *sys = omap_ssi->sys;
-
- pm_runtime_get_sync(ssi->device.parent);
- seq_printf(m, "REVISION\t: 0x%08x\n", readl(sys + SSI_REVISION_REG));
- seq_printf(m, "SYSCONFIG\t: 0x%08x\n", readl(sys + SSI_SYSCONFIG_REG));
- seq_printf(m, "SYSSTATUS\t: 0x%08x\n", readl(sys + SSI_SYSSTATUS_REG));
- pm_runtime_put_sync(ssi->device.parent);
-
- return 0;
-}
-
-static int ssi_debug_gdd_show(struct seq_file *m, void *p __maybe_unused)
-{
- struct hsi_controller *ssi = m->private;
- struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
- void __iomem *gdd = omap_ssi->gdd;
- void __iomem *sys = omap_ssi->sys;
- int lch;
-
- pm_runtime_get_sync(ssi->device.parent);
-
- seq_printf(m, "GDD_MPU_STATUS\t: 0x%08x\n",
- readl(sys + SSI_GDD_MPU_IRQ_STATUS_REG));
- seq_printf(m, "GDD_MPU_ENABLE\t: 0x%08x\n\n",
- readl(sys + SSI_GDD_MPU_IRQ_ENABLE_REG));
- seq_printf(m, "HW_ID\t\t: 0x%08x\n",
- readl(gdd + SSI_GDD_HW_ID_REG));
- seq_printf(m, "PPORT_ID\t: 0x%08x\n",
- readl(gdd + SSI_GDD_PPORT_ID_REG));
- seq_printf(m, "MPORT_ID\t: 0x%08x\n",
- readl(gdd + SSI_GDD_MPORT_ID_REG));
- seq_printf(m, "TEST\t\t: 0x%08x\n",
- readl(gdd + SSI_GDD_TEST_REG));
- seq_printf(m, "GCR\t\t: 0x%08x\n",
- readl(gdd + SSI_GDD_GCR_REG));
-
- for (lch = 0; lch < SSI_MAX_GDD_LCH; lch++) {
- seq_printf(m, "\nGDD LCH %d\n=========\n", lch);
- seq_printf(m, "CSDP\t\t: 0x%04x\n",
- readw(gdd + SSI_GDD_CSDP_REG(lch)));
- seq_printf(m, "CCR\t\t: 0x%04x\n",
- readw(gdd + SSI_GDD_CCR_REG(lch)));
- seq_printf(m, "CICR\t\t: 0x%04x\n",
- readw(gdd + SSI_GDD_CICR_REG(lch)));
- seq_printf(m, "CSR\t\t: 0x%04x\n",
- readw(gdd + SSI_GDD_CSR_REG(lch)));
- seq_printf(m, "CSSA\t\t: 0x%08x\n",
- readl(gdd + SSI_GDD_CSSA_REG(lch)));
- seq_printf(m, "CDSA\t\t: 0x%08x\n",
- readl(gdd + SSI_GDD_CDSA_REG(lch)));
- seq_printf(m, "CEN\t\t: 0x%04x\n",
- readw(gdd + SSI_GDD_CEN_REG(lch)));
- seq_printf(m, "CSAC\t\t: 0x%04x\n",
- readw(gdd + SSI_GDD_CSAC_REG(lch)));
- seq_printf(m, "CDAC\t\t: 0x%04x\n",
- readw(gdd + SSI_GDD_CDAC_REG(lch)));
- seq_printf(m, "CLNK_CTRL\t: 0x%04x\n",
- readw(gdd + SSI_GDD_CLNK_CTRL_REG(lch)));
- }
-
- pm_runtime_put_sync(ssi->device.parent);
-
- return 0;
-}
-
-static int ssi_regs_open(struct inode *inode, struct file *file)
-{
- return single_open(file, ssi_debug_show, inode->i_private);
-}
-
-static int ssi_gdd_regs_open(struct inode *inode, struct file *file)
-{
- return single_open(file, ssi_debug_gdd_show, inode->i_private);
-}
-
-static const struct file_operations ssi_regs_fops = {
- .open = ssi_regs_open,
- .read = seq_read,
- .llseek = seq_lseek,
- .release = single_release,
-};
-
-static const struct file_operations ssi_gdd_regs_fops = {
- .open = ssi_gdd_regs_open,
- .read = seq_read,
- .llseek = seq_lseek,
- .release = single_release,
-};
-
-static int __init ssi_debug_add_ctrl(struct hsi_controller *ssi)
-{
- struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
- struct dentry *dir;
-
- /* SSI controller */
- omap_ssi->dir = debugfs_create_dir(dev_name(&ssi->device), NULL);
- if (!omap_ssi->dir)
- return -ENOMEM;
-
- debugfs_create_file("regs", S_IRUGO, omap_ssi->dir, ssi,
- &ssi_regs_fops);
- /* SSI GDD (DMA) */
- dir = debugfs_create_dir("gdd", omap_ssi->dir);
- if (!dir)
- goto rback;
- debugfs_create_file("regs", S_IRUGO, dir, ssi, &ssi_gdd_regs_fops);
-
- return 0;
-rback:
- debugfs_remove_recursive(omap_ssi->dir);
-
- return -ENOMEM;
-}
-
-static void ssi_debug_remove_ctrl(struct hsi_controller *ssi)
-{
- struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
-
- debugfs_remove_recursive(omap_ssi->dir);
-}
-#endif /* CONFIG_DEBUG_FS */
-
-/*
- * FIXME: Horrible HACK needed until we remove the useless wakeline test
- * in the CMT. To be removed !!!!
- */
-void ssi_waketest(struct hsi_client *cl, unsigned int enable)
-{
- struct hsi_port *port = hsi_get_port(cl);
- struct omap_ssi_port *omap_port = hsi_port_drvdata(port);
- struct hsi_controller *ssi = to_hsi_controller(port->device.parent);
- struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
-
- omap_port->wktest = !!enable;
- if (omap_port->wktest) {
- pm_runtime_get_sync(ssi->device.parent);
- writel_relaxed(SSI_WAKE(0),
- omap_ssi->sys + SSI_SET_WAKE_REG(port->num));
- } else {
- writel_relaxed(SSI_WAKE(0),
- omap_ssi->sys + SSI_CLEAR_WAKE_REG(port->num));
- pm_runtime_put_sync(ssi->device.parent);
- }
-}
-EXPORT_SYMBOL_GPL(ssi_waketest);
-
-static void ssi_gdd_complete(struct hsi_controller *ssi, unsigned int lch)
-{
- struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
- struct hsi_msg *msg = omap_ssi->gdd_trn[lch].msg;
- struct hsi_port *port = to_hsi_port(msg->cl->device.parent);
- struct omap_ssi_port *omap_port = hsi_port_drvdata(port);
- unsigned int dir;
- u32 csr;
- u32 val;
-
- spin_lock(&omap_ssi->lock);
-
- val = readl(omap_ssi->sys + SSI_GDD_MPU_IRQ_ENABLE_REG);
- val &= ~SSI_GDD_LCH(lch);
- writel_relaxed(val, omap_ssi->sys + SSI_GDD_MPU_IRQ_ENABLE_REG);
-
- if (msg->ttype == HSI_MSG_READ) {
- dir = DMA_FROM_DEVICE;
- val = SSI_DATAAVAILABLE(msg->channel);
- pm_runtime_put_sync(ssi->device.parent);
- } else {
- dir = DMA_TO_DEVICE;
- val = SSI_DATAACCEPT(msg->channel);
- /* Keep clocks reference for write pio event */
- }
- dma_unmap_sg(&ssi->device, msg->sgt.sgl, msg->sgt.nents, dir);
- csr = readw(omap_ssi->gdd + SSI_GDD_CSR_REG(lch));
- omap_ssi->gdd_trn[lch].msg = NULL; /* release GDD lch */
- dev_dbg(&port->device, "DMA completed ch %d ttype %d\n",
- msg->channel, msg->ttype);
- spin_unlock(&omap_ssi->lock);
- if (csr & SSI_CSR_TOUR) { /* Timeout error */
- msg->status = HSI_STATUS_ERROR;
- msg->actual_len = 0;
- spin_lock(&omap_port->lock);
- list_del(&msg->link); /* Dequeue msg */
- spin_unlock(&omap_port->lock);
- msg->complete(msg);
- return;
- }
- spin_lock(&omap_port->lock);
- val |= readl(omap_ssi->sys + SSI_MPU_ENABLE_REG(port->num, 0));
- writel_relaxed(val, omap_ssi->sys + SSI_MPU_ENABLE_REG(port->num, 0));
- spin_unlock(&omap_port->lock);
-
- msg->status = HSI_STATUS_COMPLETED;
- msg->actual_len = sg_dma_len(msg->sgt.sgl);
-}
-
-static void ssi_gdd_tasklet(unsigned long dev)
-{
- struct hsi_controller *ssi = (struct hsi_controller *)dev;
- struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
- void __iomem *sys = omap_ssi->sys;
- unsigned int lch;
- u32 status_reg;
-
- pm_runtime_get_sync(ssi->device.parent);
-
- status_reg = readl(sys + SSI_GDD_MPU_IRQ_STATUS_REG);
- for (lch = 0; lch < SSI_MAX_GDD_LCH; lch++) {
- if (status_reg & SSI_GDD_LCH(lch))
- ssi_gdd_complete(ssi, lch);
- }
- writel_relaxed(status_reg, sys + SSI_GDD_MPU_IRQ_STATUS_REG);
- status_reg = readl(sys + SSI_GDD_MPU_IRQ_STATUS_REG);
-
- pm_runtime_put_sync(ssi->device.parent);
-
- if (status_reg)
- tasklet_hi_schedule(&omap_ssi->gdd_tasklet);
- else
- enable_irq(omap_ssi->gdd_irq);
-
-}
-
-static irqreturn_t ssi_gdd_isr(int irq, void *ssi)
-{
- struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
-
- tasklet_hi_schedule(&omap_ssi->gdd_tasklet);
- disable_irq_nosync(irq);
-
- return IRQ_HANDLED;
-}
-
-static unsigned long ssi_get_clk_rate(struct hsi_controller *ssi)
-{
- struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
- unsigned long rate = clk_get_rate(omap_ssi->fck);
- return rate;
-}
-
-static int __init ssi_get_iomem(struct platform_device *pd,
- const char *name, void __iomem **pbase, dma_addr_t *phy)
-{
- struct resource *mem;
- void __iomem *base;
- struct hsi_controller *ssi = platform_get_drvdata(pd);
-
- mem = platform_get_resource_byname(pd, IORESOURCE_MEM, name);
- base = devm_ioremap_resource(&ssi->device, mem);
- if (IS_ERR(base))
- return PTR_ERR(base);
-
- *pbase = base;
-
- if (phy)
- *phy = mem->start;
-
- return 0;
-}
-
-static int __init ssi_add_controller(struct hsi_controller *ssi,
- struct platform_device *pd)
-{
- struct omap_ssi_controller *omap_ssi;
- int err;
-
- omap_ssi = devm_kzalloc(&ssi->device, sizeof(*omap_ssi), GFP_KERNEL);
- if (!omap_ssi) {
- dev_err(&pd->dev, "not enough memory for omap ssi\n");
- return -ENOMEM;
- }
-
- err = ida_simple_get(&platform_omap_ssi_ida, 0, 0, GFP_KERNEL);
- if (err < 0)
- goto out_err;
- ssi->id = err;
-
- ssi->owner = THIS_MODULE;
- ssi->device.parent = &pd->dev;
- dev_set_name(&ssi->device, "ssi%d", ssi->id);
- hsi_controller_set_drvdata(ssi, omap_ssi);
- omap_ssi->dev = &ssi->device;
- err = ssi_get_iomem(pd, "sys", &omap_ssi->sys, NULL);
- if (err < 0)
- goto out_err;
- err = ssi_get_iomem(pd, "gdd", &omap_ssi->gdd, NULL);
- if (err < 0)
- goto out_err;
- err = platform_get_irq_byname(pd, "gdd_mpu");
- if (err < 0) {
- dev_err(&pd->dev, "GDD IRQ resource missing\n");
- goto out_err;
- }
- omap_ssi->gdd_irq = err;
- tasklet_init(&omap_ssi->gdd_tasklet, ssi_gdd_tasklet,
- (unsigned long)ssi);
- err = devm_request_irq(&ssi->device, omap_ssi->gdd_irq, ssi_gdd_isr,
- 0, "gdd_mpu", ssi);
- if (err < 0) {
- dev_err(&ssi->device, "Request GDD IRQ %d failed (%d)",
- omap_ssi->gdd_irq, err);
- goto out_err;
- }
-
- omap_ssi->port = devm_kzalloc(&ssi->device,
- sizeof(struct omap_ssi_port *) * ssi->num_ports, GFP_KERNEL);
- if (!omap_ssi->port) {
- err = -ENOMEM;
- goto out_err;
- }
-
- omap_ssi->fck = devm_clk_get(&ssi->device, "ssi_ssr_fck");
- if (IS_ERR(omap_ssi->fck)) {
- dev_err(&pd->dev, "Could not acquire clock \"ssi_ssr_fck\": %li\n",
- PTR_ERR(omap_ssi->fck));
- err = -ENODEV;
- goto out_err;
- }
-
- /* TODO: find register, which can be used to detect context loss */
- omap_ssi->get_loss = NULL;
-
- omap_ssi->max_speed = UINT_MAX;
- spin_lock_init(&omap_ssi->lock);
- err = hsi_register_controller(ssi);
-
- if (err < 0)
- goto out_err;
-
- return 0;
-
-out_err:
- ida_simple_remove(&platform_omap_ssi_ida, ssi->id);
- return err;
-}
-
-static int __init ssi_hw_init(struct hsi_controller *ssi)
-{
- struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
- unsigned int i;
- u32 val;
- int err;
-
- err = pm_runtime_get_sync(ssi->device.parent);
- if (err < 0) {
- dev_err(&ssi->device, "runtime PM failed %d\n", err);
- return err;
- }
- /* Reseting SSI controller */
- writel_relaxed(SSI_SOFTRESET, omap_ssi->sys + SSI_SYSCONFIG_REG);
- val = readl(omap_ssi->sys + SSI_SYSSTATUS_REG);
- for (i = 0; ((i < 20) && !(val & SSI_RESETDONE)); i++) {
- msleep(20);
- val = readl(omap_ssi->sys + SSI_SYSSTATUS_REG);
- }
- if (!(val & SSI_RESETDONE)) {
- dev_err(&ssi->device, "SSI HW reset failed\n");
- pm_runtime_put_sync(ssi->device.parent);
- return -EIO;
- }
- /* Reseting GDD */
- writel_relaxed(SSI_SWRESET, omap_ssi->gdd + SSI_GDD_GRST_REG);
- /* Get FCK rate in KHz */
- omap_ssi->fck_rate = DIV_ROUND_CLOSEST(ssi_get_clk_rate(ssi), 1000);
- dev_dbg(&ssi->device, "SSI fck rate %lu KHz\n", omap_ssi->fck_rate);
- /* Set default PM settings */
- val = SSI_AUTOIDLE | SSI_SIDLEMODE_SMART | SSI_MIDLEMODE_SMART;
- writel_relaxed(val, omap_ssi->sys + SSI_SYSCONFIG_REG);
- omap_ssi->sysconfig = val;
- writel_relaxed(SSI_CLK_AUTOGATING_ON, omap_ssi->sys + SSI_GDD_GCR_REG);
- omap_ssi->gdd_gcr = SSI_CLK_AUTOGATING_ON;
- pm_runtime_put_sync(ssi->device.parent);
-
- return 0;
-}
-
-static void ssi_remove_controller(struct hsi_controller *ssi)
-{
- struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
- int id = ssi->id;
- tasklet_kill(&omap_ssi->gdd_tasklet);
- hsi_unregister_controller(ssi);
- ida_simple_remove(&platform_omap_ssi_ida, id);
-}
-
-static inline int ssi_of_get_available_ports_count(const struct device_node *np)
-{
- struct device_node *child;
- int num = 0;
-
- for_each_available_child_of_node(np, child)
- if (of_device_is_compatible(child, "ti,omap3-ssi-port"))
- num++;
-
- return num;
-}
-
-static int ssi_remove_ports(struct device *dev, void *c)
-{
- struct platform_device *pdev = to_platform_device(dev);
-
- of_device_unregister(pdev);
-
- return 0;
-}
-
-static int __init ssi_probe(struct platform_device *pd)
-{
- struct platform_device *childpdev;
- struct device_node *np = pd->dev.of_node;
- struct device_node *child;
- struct hsi_controller *ssi;
- int err;
- int num_ports;
-
- if (!np) {
- dev_err(&pd->dev, "missing device tree data\n");
- return -EINVAL;
- }
-
- num_ports = ssi_of_get_available_ports_count(np);
-
- ssi = hsi_alloc_controller(num_ports, GFP_KERNEL);
- if (!ssi) {
- dev_err(&pd->dev, "No memory for controller\n");
- return -ENOMEM;
- }
-
- platform_set_drvdata(pd, ssi);
-
- err = ssi_add_controller(ssi, pd);
- if (err < 0)
- goto out1;
-
- pm_runtime_irq_safe(&pd->dev);
- pm_runtime_enable(&pd->dev);
-
- err = ssi_hw_init(ssi);
- if (err < 0)
- goto out2;
-#ifdef CONFIG_DEBUG_FS
- err = ssi_debug_add_ctrl(ssi);
- if (err < 0)
- goto out2;
-#endif
-
- for_each_available_child_of_node(np, child) {
- if (!of_device_is_compatible(child, "ti,omap3-ssi-port"))
- continue;
-
- childpdev = of_platform_device_create(child, NULL, &pd->dev);
- if (!childpdev) {
- err = -ENODEV;
- dev_err(&pd->dev, "failed to create ssi controller port\n");
- goto out3;
- }
- }
-
- dev_info(&pd->dev, "ssi controller %d initialized (%d ports)!\n",
- ssi->id, num_ports);
- return err;
-out3:
- device_for_each_child(&pd->dev, NULL, ssi_remove_ports);
-out2:
- ssi_remove_controller(ssi);
-out1:
- platform_set_drvdata(pd, NULL);
- pm_runtime_disable(&pd->dev);
-
- return err;
-}
-
-static int __exit ssi_remove(struct platform_device *pd)
-{
- struct hsi_controller *ssi = platform_get_drvdata(pd);
-
-#ifdef CONFIG_DEBUG_FS
- ssi_debug_remove_ctrl(ssi);
-#endif
- ssi_remove_controller(ssi);
- platform_set_drvdata(pd, NULL);
-
- pm_runtime_disable(&pd->dev);
-
- /* cleanup of of_platform_populate() call */
- device_for_each_child(&pd->dev, NULL, ssi_remove_ports);
-
- return 0;
-}
-
-#ifdef CONFIG_PM
-static int omap_ssi_runtime_suspend(struct device *dev)
-{
- struct hsi_controller *ssi = dev_get_drvdata(dev);
- struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
-
- dev_dbg(dev, "runtime suspend!\n");
-
- if (omap_ssi->get_loss)
- omap_ssi->loss_count =
- omap_ssi->get_loss(ssi->device.parent);
-
- return 0;
-}
-
-static int omap_ssi_runtime_resume(struct device *dev)
-{
- struct hsi_controller *ssi = dev_get_drvdata(dev);
- struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
-
- dev_dbg(dev, "runtime resume!\n");
-
- if ((omap_ssi->get_loss) && (omap_ssi->loss_count ==
- omap_ssi->get_loss(ssi->device.parent)))
- return 0;
-
- writel_relaxed(omap_ssi->gdd_gcr, omap_ssi->gdd + SSI_GDD_GCR_REG);
-
- return 0;
-}
-
-static const struct dev_pm_ops omap_ssi_pm_ops = {
- SET_RUNTIME_PM_OPS(omap_ssi_runtime_suspend, omap_ssi_runtime_resume,
- NULL)
-};
-
-#define DEV_PM_OPS (&omap_ssi_pm_ops)
-#else
-#define DEV_PM_OPS NULL
-#endif
-
-#ifdef CONFIG_OF
-static const struct of_device_id omap_ssi_of_match[] = {
- { .compatible = "ti,omap3-ssi", },
- {},
-};
-MODULE_DEVICE_TABLE(of, omap_ssi_of_match);
-#else
-#define omap_ssi_of_match NULL
-#endif
-
-static struct platform_driver ssi_pdriver = {
- .remove = __exit_p(ssi_remove),
- .driver = {
- .name = "omap_ssi",
- .pm = DEV_PM_OPS,
- .of_match_table = omap_ssi_of_match,
- },
-};
-
-module_platform_driver_probe(ssi_pdriver, ssi_probe);
-
-MODULE_ALIAS("platform:omap_ssi");
-MODULE_AUTHOR("Carlos Chinea <carlos.chinea@nokia.com>");
-MODULE_AUTHOR("Sebastian Reichel <sre@kernel.org>");
-MODULE_DESCRIPTION("Synchronous Serial Interface Driver");
-MODULE_LICENSE("GPL v2");
diff --git a/drivers/hsi/controllers/omap_ssi.h b/drivers/hsi/controllers/omap_ssi.h
index f9aaf37..7b4dec2 100644
--- a/drivers/hsi/controllers/omap_ssi.h
+++ b/drivers/hsi/controllers/omap_ssi.h
@@ -27,7 +27,7 @@
#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/hsi/hsi.h>
-#include <linux/gpio.h>
+#include <linux/gpio/consumer.h>
#include <linux/interrupt.h>
#include <linux/io.h>
@@ -97,7 +97,7 @@
struct list_head brkqueue;
unsigned int irq;
int wake_irq;
- int wake_gpio;
+ struct gpio_desc *wake_gpio;
struct tasklet_struct pio_tasklet;
struct tasklet_struct wake_tasklet;
bool wktest:1; /* FIXME: HACK to be removed */
@@ -134,6 +134,8 @@
* @gdd_tasklet: bottom half for DMA transfers
* @gdd_trn: Array of GDD transaction data for ongoing GDD transfers
* @lock: lock to serialize access to GDD
+ * @fck_nb: DVFS notfifier block
+ * @fck_rate: clock rate
* @loss_count: To follow if we need to restore context or not
* @max_speed: Maximum TX speed (Kb/s) set by the clients.
* @sysconfig: SSI controller saved context
@@ -151,6 +153,7 @@
struct tasklet_struct gdd_tasklet;
struct gdd_trn gdd_trn[SSI_MAX_GDD_LCH];
spinlock_t lock;
+ struct notifier_block fck_nb;
unsigned long fck_rate;
u32 loss_count;
u32 max_speed;
@@ -164,4 +167,9 @@
#endif
};
+void omap_ssi_port_update_fclk(struct hsi_controller *ssi,
+ struct omap_ssi_port *omap_port);
+
+extern struct platform_driver ssi_port_pdriver;
+
#endif /* __LINUX_HSI_OMAP_SSI_H__ */
diff --git a/drivers/hsi/controllers/omap_ssi_core.c b/drivers/hsi/controllers/omap_ssi_core.c
new file mode 100644
index 0000000..a3e0feb
--- /dev/null
+++ b/drivers/hsi/controllers/omap_ssi_core.c
@@ -0,0 +1,693 @@
+/* OMAP SSI driver.
+ *
+ * Copyright (C) 2010 Nokia Corporation. All rights reserved.
+ * Copyright (C) 2014 Sebastian Reichel <sre@kernel.org>
+ *
+ * Contact: Carlos Chinea <carlos.chinea@nokia.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
+ * 02110-1301 USA
+ */
+
+#include <linux/compiler.h>
+#include <linux/err.h>
+#include <linux/ioport.h>
+#include <linux/io.h>
+#include <linux/clk.h>
+#include <linux/device.h>
+#include <linux/platform_device.h>
+#include <linux/dma-mapping.h>
+#include <linux/dmaengine.h>
+#include <linux/delay.h>
+#include <linux/seq_file.h>
+#include <linux/scatterlist.h>
+#include <linux/interrupt.h>
+#include <linux/spinlock.h>
+#include <linux/debugfs.h>
+#include <linux/pinctrl/consumer.h>
+#include <linux/pm_runtime.h>
+#include <linux/of_platform.h>
+#include <linux/hsi/hsi.h>
+#include <linux/idr.h>
+
+#include "omap_ssi_regs.h"
+#include "omap_ssi.h"
+
+/* For automatically allocated device IDs */
+static DEFINE_IDA(platform_omap_ssi_ida);
+
+#ifdef CONFIG_DEBUG_FS
+static int ssi_debug_show(struct seq_file *m, void *p __maybe_unused)
+{
+ struct hsi_controller *ssi = m->private;
+ struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
+ void __iomem *sys = omap_ssi->sys;
+
+ pm_runtime_get_sync(ssi->device.parent);
+ seq_printf(m, "REVISION\t: 0x%08x\n", readl(sys + SSI_REVISION_REG));
+ seq_printf(m, "SYSCONFIG\t: 0x%08x\n", readl(sys + SSI_SYSCONFIG_REG));
+ seq_printf(m, "SYSSTATUS\t: 0x%08x\n", readl(sys + SSI_SYSSTATUS_REG));
+ pm_runtime_put_sync(ssi->device.parent);
+
+ return 0;
+}
+
+static int ssi_debug_gdd_show(struct seq_file *m, void *p __maybe_unused)
+{
+ struct hsi_controller *ssi = m->private;
+ struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
+ void __iomem *gdd = omap_ssi->gdd;
+ void __iomem *sys = omap_ssi->sys;
+ int lch;
+
+ pm_runtime_get_sync(ssi->device.parent);
+
+ seq_printf(m, "GDD_MPU_STATUS\t: 0x%08x\n",
+ readl(sys + SSI_GDD_MPU_IRQ_STATUS_REG));
+ seq_printf(m, "GDD_MPU_ENABLE\t: 0x%08x\n\n",
+ readl(sys + SSI_GDD_MPU_IRQ_ENABLE_REG));
+ seq_printf(m, "HW_ID\t\t: 0x%08x\n",
+ readl(gdd + SSI_GDD_HW_ID_REG));
+ seq_printf(m, "PPORT_ID\t: 0x%08x\n",
+ readl(gdd + SSI_GDD_PPORT_ID_REG));
+ seq_printf(m, "MPORT_ID\t: 0x%08x\n",
+ readl(gdd + SSI_GDD_MPORT_ID_REG));
+ seq_printf(m, "TEST\t\t: 0x%08x\n",
+ readl(gdd + SSI_GDD_TEST_REG));
+ seq_printf(m, "GCR\t\t: 0x%08x\n",
+ readl(gdd + SSI_GDD_GCR_REG));
+
+ for (lch = 0; lch < SSI_MAX_GDD_LCH; lch++) {
+ seq_printf(m, "\nGDD LCH %d\n=========\n", lch);
+ seq_printf(m, "CSDP\t\t: 0x%04x\n",
+ readw(gdd + SSI_GDD_CSDP_REG(lch)));
+ seq_printf(m, "CCR\t\t: 0x%04x\n",
+ readw(gdd + SSI_GDD_CCR_REG(lch)));
+ seq_printf(m, "CICR\t\t: 0x%04x\n",
+ readw(gdd + SSI_GDD_CICR_REG(lch)));
+ seq_printf(m, "CSR\t\t: 0x%04x\n",
+ readw(gdd + SSI_GDD_CSR_REG(lch)));
+ seq_printf(m, "CSSA\t\t: 0x%08x\n",
+ readl(gdd + SSI_GDD_CSSA_REG(lch)));
+ seq_printf(m, "CDSA\t\t: 0x%08x\n",
+ readl(gdd + SSI_GDD_CDSA_REG(lch)));
+ seq_printf(m, "CEN\t\t: 0x%04x\n",
+ readw(gdd + SSI_GDD_CEN_REG(lch)));
+ seq_printf(m, "CSAC\t\t: 0x%04x\n",
+ readw(gdd + SSI_GDD_CSAC_REG(lch)));
+ seq_printf(m, "CDAC\t\t: 0x%04x\n",
+ readw(gdd + SSI_GDD_CDAC_REG(lch)));
+ seq_printf(m, "CLNK_CTRL\t: 0x%04x\n",
+ readw(gdd + SSI_GDD_CLNK_CTRL_REG(lch)));
+ }
+
+ pm_runtime_put_sync(ssi->device.parent);
+
+ return 0;
+}
+
+static int ssi_regs_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, ssi_debug_show, inode->i_private);
+}
+
+static int ssi_gdd_regs_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, ssi_debug_gdd_show, inode->i_private);
+}
+
+static const struct file_operations ssi_regs_fops = {
+ .open = ssi_regs_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
+static const struct file_operations ssi_gdd_regs_fops = {
+ .open = ssi_gdd_regs_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
+static int ssi_debug_add_ctrl(struct hsi_controller *ssi)
+{
+ struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
+ struct dentry *dir;
+
+ /* SSI controller */
+ omap_ssi->dir = debugfs_create_dir(dev_name(&ssi->device), NULL);
+ if (!omap_ssi->dir)
+ return -ENOMEM;
+
+ debugfs_create_file("regs", S_IRUGO, omap_ssi->dir, ssi,
+ &ssi_regs_fops);
+ /* SSI GDD (DMA) */
+ dir = debugfs_create_dir("gdd", omap_ssi->dir);
+ if (!dir)
+ goto rback;
+ debugfs_create_file("regs", S_IRUGO, dir, ssi, &ssi_gdd_regs_fops);
+
+ return 0;
+rback:
+ debugfs_remove_recursive(omap_ssi->dir);
+
+ return -ENOMEM;
+}
+
+static void ssi_debug_remove_ctrl(struct hsi_controller *ssi)
+{
+ struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
+
+ debugfs_remove_recursive(omap_ssi->dir);
+}
+#endif /* CONFIG_DEBUG_FS */
+
+/*
+ * FIXME: Horrible HACK needed until we remove the useless wakeline test
+ * in the CMT. To be removed !!!!
+ */
+void ssi_waketest(struct hsi_client *cl, unsigned int enable)
+{
+ struct hsi_port *port = hsi_get_port(cl);
+ struct omap_ssi_port *omap_port = hsi_port_drvdata(port);
+ struct hsi_controller *ssi = to_hsi_controller(port->device.parent);
+ struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
+
+ omap_port->wktest = !!enable;
+ if (omap_port->wktest) {
+ pm_runtime_get_sync(ssi->device.parent);
+ writel_relaxed(SSI_WAKE(0),
+ omap_ssi->sys + SSI_SET_WAKE_REG(port->num));
+ } else {
+ writel_relaxed(SSI_WAKE(0),
+ omap_ssi->sys + SSI_CLEAR_WAKE_REG(port->num));
+ pm_runtime_put_sync(ssi->device.parent);
+ }
+}
+EXPORT_SYMBOL_GPL(ssi_waketest);
+
+static void ssi_gdd_complete(struct hsi_controller *ssi, unsigned int lch)
+{
+ struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
+ struct hsi_msg *msg = omap_ssi->gdd_trn[lch].msg;
+ struct hsi_port *port = to_hsi_port(msg->cl->device.parent);
+ struct omap_ssi_port *omap_port = hsi_port_drvdata(port);
+ unsigned int dir;
+ u32 csr;
+ u32 val;
+
+ spin_lock(&omap_ssi->lock);
+
+ val = readl(omap_ssi->sys + SSI_GDD_MPU_IRQ_ENABLE_REG);
+ val &= ~SSI_GDD_LCH(lch);
+ writel_relaxed(val, omap_ssi->sys + SSI_GDD_MPU_IRQ_ENABLE_REG);
+
+ if (msg->ttype == HSI_MSG_READ) {
+ dir = DMA_FROM_DEVICE;
+ val = SSI_DATAAVAILABLE(msg->channel);
+ pm_runtime_put_sync(ssi->device.parent);
+ } else {
+ dir = DMA_TO_DEVICE;
+ val = SSI_DATAACCEPT(msg->channel);
+ /* Keep clocks reference for write pio event */
+ }
+ dma_unmap_sg(&ssi->device, msg->sgt.sgl, msg->sgt.nents, dir);
+ csr = readw(omap_ssi->gdd + SSI_GDD_CSR_REG(lch));
+ omap_ssi->gdd_trn[lch].msg = NULL; /* release GDD lch */
+ dev_dbg(&port->device, "DMA completed ch %d ttype %d\n",
+ msg->channel, msg->ttype);
+ spin_unlock(&omap_ssi->lock);
+ if (csr & SSI_CSR_TOUR) { /* Timeout error */
+ msg->status = HSI_STATUS_ERROR;
+ msg->actual_len = 0;
+ spin_lock(&omap_port->lock);
+ list_del(&msg->link); /* Dequeue msg */
+ spin_unlock(&omap_port->lock);
+ msg->complete(msg);
+ return;
+ }
+ spin_lock(&omap_port->lock);
+ val |= readl(omap_ssi->sys + SSI_MPU_ENABLE_REG(port->num, 0));
+ writel_relaxed(val, omap_ssi->sys + SSI_MPU_ENABLE_REG(port->num, 0));
+ spin_unlock(&omap_port->lock);
+
+ msg->status = HSI_STATUS_COMPLETED;
+ msg->actual_len = sg_dma_len(msg->sgt.sgl);
+}
+
+static void ssi_gdd_tasklet(unsigned long dev)
+{
+ struct hsi_controller *ssi = (struct hsi_controller *)dev;
+ struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
+ void __iomem *sys = omap_ssi->sys;
+ unsigned int lch;
+ u32 status_reg;
+
+ pm_runtime_get_sync(ssi->device.parent);
+
+ status_reg = readl(sys + SSI_GDD_MPU_IRQ_STATUS_REG);
+ for (lch = 0; lch < SSI_MAX_GDD_LCH; lch++) {
+ if (status_reg & SSI_GDD_LCH(lch))
+ ssi_gdd_complete(ssi, lch);
+ }
+ writel_relaxed(status_reg, sys + SSI_GDD_MPU_IRQ_STATUS_REG);
+ status_reg = readl(sys + SSI_GDD_MPU_IRQ_STATUS_REG);
+
+ pm_runtime_put_sync(ssi->device.parent);
+
+ if (status_reg)
+ tasklet_hi_schedule(&omap_ssi->gdd_tasklet);
+ else
+ enable_irq(omap_ssi->gdd_irq);
+
+}
+
+static irqreturn_t ssi_gdd_isr(int irq, void *ssi)
+{
+ struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
+
+ tasklet_hi_schedule(&omap_ssi->gdd_tasklet);
+ disable_irq_nosync(irq);
+
+ return IRQ_HANDLED;
+}
+
+static unsigned long ssi_get_clk_rate(struct hsi_controller *ssi)
+{
+ struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
+ unsigned long rate = clk_get_rate(omap_ssi->fck);
+ return rate;
+}
+
+static int ssi_clk_event(struct notifier_block *nb, unsigned long event,
+ void *data)
+{
+ struct omap_ssi_controller *omap_ssi = container_of(nb,
+ struct omap_ssi_controller, fck_nb);
+ struct hsi_controller *ssi = to_hsi_controller(omap_ssi->dev);
+ struct clk_notifier_data *clk_data = data;
+ struct omap_ssi_port *omap_port;
+ int i;
+
+ switch (event) {
+ case PRE_RATE_CHANGE:
+ dev_dbg(&ssi->device, "pre rate change\n");
+
+ for (i = 0; i < ssi->num_ports; i++) {
+ omap_port = omap_ssi->port[i];
+
+ if (!omap_port)
+ continue;
+
+ /* Workaround for SWBREAK + CAwake down race in CMT */
+ tasklet_disable(&omap_port->wake_tasklet);
+
+ /* stop all ssi communication */
+ pinctrl_pm_select_idle_state(omap_port->pdev);
+ udelay(1); /* wait for racing frames */
+ }
+
+ break;
+ case ABORT_RATE_CHANGE:
+ dev_dbg(&ssi->device, "abort rate change\n");
+ /* Fall through */
+ case POST_RATE_CHANGE:
+ dev_dbg(&ssi->device, "post rate change (%lu -> %lu)\n",
+ clk_data->old_rate, clk_data->new_rate);
+ omap_ssi->fck_rate = DIV_ROUND_CLOSEST(clk_data->new_rate, 1000); /* KHz */
+
+ for (i = 0; i < ssi->num_ports; i++) {
+ omap_port = omap_ssi->port[i];
+
+ if (!omap_port)
+ continue;
+
+ omap_ssi_port_update_fclk(ssi, omap_port);
+
+ /* resume ssi communication */
+ pinctrl_pm_select_default_state(omap_port->pdev);
+ tasklet_enable(&omap_port->wake_tasklet);
+ }
+
+ break;
+ default:
+ break;
+ }
+
+ return NOTIFY_DONE;
+}
+
+static int ssi_get_iomem(struct platform_device *pd,
+ const char *name, void __iomem **pbase, dma_addr_t *phy)
+{
+ struct resource *mem;
+ void __iomem *base;
+ struct hsi_controller *ssi = platform_get_drvdata(pd);
+
+ mem = platform_get_resource_byname(pd, IORESOURCE_MEM, name);
+ base = devm_ioremap_resource(&ssi->device, mem);
+ if (IS_ERR(base))
+ return PTR_ERR(base);
+
+ *pbase = base;
+
+ if (phy)
+ *phy = mem->start;
+
+ return 0;
+}
+
+static int ssi_add_controller(struct hsi_controller *ssi,
+ struct platform_device *pd)
+{
+ struct omap_ssi_controller *omap_ssi;
+ int err;
+
+ omap_ssi = devm_kzalloc(&ssi->device, sizeof(*omap_ssi), GFP_KERNEL);
+ if (!omap_ssi) {
+ dev_err(&pd->dev, "not enough memory for omap ssi\n");
+ return -ENOMEM;
+ }
+
+ err = ida_simple_get(&platform_omap_ssi_ida, 0, 0, GFP_KERNEL);
+ if (err < 0)
+ goto out_err;
+ ssi->id = err;
+
+ ssi->owner = THIS_MODULE;
+ ssi->device.parent = &pd->dev;
+ dev_set_name(&ssi->device, "ssi%d", ssi->id);
+ hsi_controller_set_drvdata(ssi, omap_ssi);
+ omap_ssi->dev = &ssi->device;
+ err = ssi_get_iomem(pd, "sys", &omap_ssi->sys, NULL);
+ if (err < 0)
+ goto out_err;
+ err = ssi_get_iomem(pd, "gdd", &omap_ssi->gdd, NULL);
+ if (err < 0)
+ goto out_err;
+ err = platform_get_irq_byname(pd, "gdd_mpu");
+ if (err < 0) {
+ dev_err(&pd->dev, "GDD IRQ resource missing\n");
+ goto out_err;
+ }
+ omap_ssi->gdd_irq = err;
+ tasklet_init(&omap_ssi->gdd_tasklet, ssi_gdd_tasklet,
+ (unsigned long)ssi);
+ err = devm_request_irq(&ssi->device, omap_ssi->gdd_irq, ssi_gdd_isr,
+ 0, "gdd_mpu", ssi);
+ if (err < 0) {
+ dev_err(&ssi->device, "Request GDD IRQ %d failed (%d)",
+ omap_ssi->gdd_irq, err);
+ goto out_err;
+ }
+
+ omap_ssi->port = devm_kzalloc(&ssi->device,
+ sizeof(struct omap_ssi_port *) * ssi->num_ports, GFP_KERNEL);
+ if (!omap_ssi->port) {
+ err = -ENOMEM;
+ goto out_err;
+ }
+
+ omap_ssi->fck = devm_clk_get(&ssi->device, "ssi_ssr_fck");
+ if (IS_ERR(omap_ssi->fck)) {
+ dev_err(&pd->dev, "Could not acquire clock \"ssi_ssr_fck\": %li\n",
+ PTR_ERR(omap_ssi->fck));
+ err = -ENODEV;
+ goto out_err;
+ }
+
+ omap_ssi->fck_nb.notifier_call = ssi_clk_event;
+ omap_ssi->fck_nb.priority = INT_MAX;
+ clk_notifier_register(omap_ssi->fck, &omap_ssi->fck_nb);
+
+ /* TODO: find register, which can be used to detect context loss */
+ omap_ssi->get_loss = NULL;
+
+ omap_ssi->max_speed = UINT_MAX;
+ spin_lock_init(&omap_ssi->lock);
+ err = hsi_register_controller(ssi);
+
+ if (err < 0)
+ goto out_err;
+
+ return 0;
+
+out_err:
+ ida_simple_remove(&platform_omap_ssi_ida, ssi->id);
+ return err;
+}
+
+static int ssi_hw_init(struct hsi_controller *ssi)
+{
+ struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
+ unsigned int i;
+ u32 val;
+ int err;
+
+ err = pm_runtime_get_sync(ssi->device.parent);
+ if (err < 0) {
+ dev_err(&ssi->device, "runtime PM failed %d\n", err);
+ return err;
+ }
+ /* Reseting SSI controller */
+ writel_relaxed(SSI_SOFTRESET, omap_ssi->sys + SSI_SYSCONFIG_REG);
+ val = readl(omap_ssi->sys + SSI_SYSSTATUS_REG);
+ for (i = 0; ((i < 20) && !(val & SSI_RESETDONE)); i++) {
+ msleep(20);
+ val = readl(omap_ssi->sys + SSI_SYSSTATUS_REG);
+ }
+ if (!(val & SSI_RESETDONE)) {
+ dev_err(&ssi->device, "SSI HW reset failed\n");
+ pm_runtime_put_sync(ssi->device.parent);
+ return -EIO;
+ }
+ /* Reseting GDD */
+ writel_relaxed(SSI_SWRESET, omap_ssi->gdd + SSI_GDD_GRST_REG);
+ /* Get FCK rate in KHz */
+ omap_ssi->fck_rate = DIV_ROUND_CLOSEST(ssi_get_clk_rate(ssi), 1000);
+ dev_dbg(&ssi->device, "SSI fck rate %lu KHz\n", omap_ssi->fck_rate);
+ /* Set default PM settings */
+ val = SSI_AUTOIDLE | SSI_SIDLEMODE_SMART | SSI_MIDLEMODE_SMART;
+ writel_relaxed(val, omap_ssi->sys + SSI_SYSCONFIG_REG);
+ omap_ssi->sysconfig = val;
+ writel_relaxed(SSI_CLK_AUTOGATING_ON, omap_ssi->sys + SSI_GDD_GCR_REG);
+ omap_ssi->gdd_gcr = SSI_CLK_AUTOGATING_ON;
+ pm_runtime_put_sync(ssi->device.parent);
+
+ return 0;
+}
+
+static void ssi_remove_controller(struct hsi_controller *ssi)
+{
+ struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
+ int id = ssi->id;
+ tasklet_kill(&omap_ssi->gdd_tasklet);
+ hsi_unregister_controller(ssi);
+ clk_notifier_unregister(omap_ssi->fck, &omap_ssi->fck_nb);
+ ida_simple_remove(&platform_omap_ssi_ida, id);
+}
+
+static inline int ssi_of_get_available_ports_count(const struct device_node *np)
+{
+ struct device_node *child;
+ int num = 0;
+
+ for_each_available_child_of_node(np, child)
+ if (of_device_is_compatible(child, "ti,omap3-ssi-port"))
+ num++;
+
+ return num;
+}
+
+static int ssi_remove_ports(struct device *dev, void *c)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+
+ if (!dev->of_node)
+ return 0;
+
+ of_node_clear_flag(dev->of_node, OF_POPULATED);
+ of_device_unregister(pdev);
+
+ return 0;
+}
+
+static int ssi_probe(struct platform_device *pd)
+{
+ struct platform_device *childpdev;
+ struct device_node *np = pd->dev.of_node;
+ struct device_node *child;
+ struct hsi_controller *ssi;
+ int err;
+ int num_ports;
+
+ if (!np) {
+ dev_err(&pd->dev, "missing device tree data\n");
+ return -EINVAL;
+ }
+
+ num_ports = ssi_of_get_available_ports_count(np);
+
+ ssi = hsi_alloc_controller(num_ports, GFP_KERNEL);
+ if (!ssi) {
+ dev_err(&pd->dev, "No memory for controller\n");
+ return -ENOMEM;
+ }
+
+ platform_set_drvdata(pd, ssi);
+
+ err = ssi_add_controller(ssi, pd);
+ if (err < 0)
+ goto out1;
+
+ pm_runtime_irq_safe(&pd->dev);
+ pm_runtime_enable(&pd->dev);
+
+ err = ssi_hw_init(ssi);
+ if (err < 0)
+ goto out2;
+#ifdef CONFIG_DEBUG_FS
+ err = ssi_debug_add_ctrl(ssi);
+ if (err < 0)
+ goto out2;
+#endif
+
+ for_each_available_child_of_node(np, child) {
+ if (!of_device_is_compatible(child, "ti,omap3-ssi-port"))
+ continue;
+
+ childpdev = of_platform_device_create(child, NULL, &pd->dev);
+ if (!childpdev) {
+ err = -ENODEV;
+ dev_err(&pd->dev, "failed to create ssi controller port\n");
+ goto out3;
+ }
+ }
+
+ dev_info(&pd->dev, "ssi controller %d initialized (%d ports)!\n",
+ ssi->id, num_ports);
+ return err;
+out3:
+ device_for_each_child(&pd->dev, NULL, ssi_remove_ports);
+out2:
+ ssi_remove_controller(ssi);
+out1:
+ platform_set_drvdata(pd, NULL);
+ pm_runtime_disable(&pd->dev);
+
+ return err;
+}
+
+static int ssi_remove(struct platform_device *pd)
+{
+ struct hsi_controller *ssi = platform_get_drvdata(pd);
+
+ /* cleanup of of_platform_populate() call */
+ device_for_each_child(&pd->dev, NULL, ssi_remove_ports);
+
+#ifdef CONFIG_DEBUG_FS
+ ssi_debug_remove_ctrl(ssi);
+#endif
+ ssi_remove_controller(ssi);
+ platform_set_drvdata(pd, NULL);
+
+ pm_runtime_disable(&pd->dev);
+
+ return 0;
+}
+
+#ifdef CONFIG_PM
+static int omap_ssi_runtime_suspend(struct device *dev)
+{
+ struct hsi_controller *ssi = dev_get_drvdata(dev);
+ struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
+
+ dev_dbg(dev, "runtime suspend!\n");
+
+ if (omap_ssi->get_loss)
+ omap_ssi->loss_count =
+ omap_ssi->get_loss(ssi->device.parent);
+
+ return 0;
+}
+
+static int omap_ssi_runtime_resume(struct device *dev)
+{
+ struct hsi_controller *ssi = dev_get_drvdata(dev);
+ struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
+
+ dev_dbg(dev, "runtime resume!\n");
+
+ if ((omap_ssi->get_loss) && (omap_ssi->loss_count ==
+ omap_ssi->get_loss(ssi->device.parent)))
+ return 0;
+
+ writel_relaxed(omap_ssi->gdd_gcr, omap_ssi->gdd + SSI_GDD_GCR_REG);
+
+ return 0;
+}
+
+static const struct dev_pm_ops omap_ssi_pm_ops = {
+ SET_RUNTIME_PM_OPS(omap_ssi_runtime_suspend, omap_ssi_runtime_resume,
+ NULL)
+};
+
+#define DEV_PM_OPS (&omap_ssi_pm_ops)
+#else
+#define DEV_PM_OPS NULL
+#endif
+
+#ifdef CONFIG_OF
+static const struct of_device_id omap_ssi_of_match[] = {
+ { .compatible = "ti,omap3-ssi", },
+ {},
+};
+MODULE_DEVICE_TABLE(of, omap_ssi_of_match);
+#else
+#define omap_ssi_of_match NULL
+#endif
+
+static struct platform_driver ssi_pdriver = {
+ .probe = ssi_probe,
+ .remove = ssi_remove,
+ .driver = {
+ .name = "omap_ssi",
+ .pm = DEV_PM_OPS,
+ .of_match_table = omap_ssi_of_match,
+ },
+};
+
+static int __init ssi_init(void) {
+ int ret;
+
+ ret = platform_driver_register(&ssi_pdriver);
+ if (ret)
+ return ret;
+
+ return platform_driver_register(&ssi_port_pdriver);
+}
+module_init(ssi_init);
+
+static void __exit ssi_exit(void) {
+ platform_driver_unregister(&ssi_port_pdriver);
+ platform_driver_unregister(&ssi_pdriver);
+}
+module_exit(ssi_exit);
+
+MODULE_ALIAS("platform:omap_ssi");
+MODULE_AUTHOR("Carlos Chinea <carlos.chinea@nokia.com>");
+MODULE_AUTHOR("Sebastian Reichel <sre@kernel.org>");
+MODULE_DESCRIPTION("Synchronous Serial Interface Driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/hsi/controllers/omap_ssi_port.c b/drivers/hsi/controllers/omap_ssi_port.c
index e80a66e..6b8f773 100644
--- a/drivers/hsi/controllers/omap_ssi_port.c
+++ b/drivers/hsi/controllers/omap_ssi_port.c
@@ -23,8 +23,10 @@
#include <linux/platform_device.h>
#include <linux/dma-mapping.h>
#include <linux/pm_runtime.h>
+#include <linux/delay.h>
-#include <linux/of_gpio.h>
+#include <linux/gpio/consumer.h>
+#include <linux/pinctrl/consumer.h>
#include <linux/debugfs.h>
#include "omap_ssi_regs.h"
@@ -43,7 +45,7 @@
static inline unsigned int ssi_wakein(struct hsi_port *port)
{
struct omap_ssi_port *omap_port = hsi_port_drvdata(port);
- return gpio_get_value(omap_port->wake_gpio);
+ return gpiod_get_value(omap_port->wake_gpio);
}
#ifdef CONFIG_DEBUG_FS
@@ -171,7 +173,7 @@
DEFINE_SIMPLE_ATTRIBUTE(ssi_sst_div_fops, ssi_div_get, ssi_div_set, "%llu\n");
-static int __init ssi_debug_add_port(struct omap_ssi_port *omap_port,
+static int ssi_debug_add_port(struct omap_ssi_port *omap_port,
struct dentry *dir)
{
struct hsi_port *port = to_hsi_port(omap_port->dev);
@@ -514,6 +516,11 @@
pm_runtime_get_sync(omap_port->pdev);
spin_lock_bh(&omap_port->lock);
+
+ /* stop all ssi communication */
+ pinctrl_pm_select_idle_state(omap_port->pdev);
+ udelay(1); /* wait for racing frames */
+
/* Stop all DMA transfers */
for (i = 0; i < SSI_MAX_GDD_LCH; i++) {
msg = omap_ssi->gdd_trn[i].msg;
@@ -550,6 +557,10 @@
ssi_flush_queue(&omap_port->rxqueue[i], NULL);
}
ssi_flush_queue(&omap_port->brkqueue, NULL);
+
+ /* Resume SSI communication */
+ pinctrl_pm_select_default_state(omap_port->pdev);
+
spin_unlock_bh(&omap_port->lock);
pm_runtime_put_sync(omap_port->pdev);
@@ -1007,7 +1018,7 @@
return IRQ_HANDLED;
}
-static int __init ssi_port_irq(struct hsi_port *port,
+static int ssi_port_irq(struct hsi_port *port,
struct platform_device *pd)
{
struct omap_ssi_port *omap_port = hsi_port_drvdata(port);
@@ -1029,19 +1040,19 @@
return err;
}
-static int __init ssi_wake_irq(struct hsi_port *port,
+static int ssi_wake_irq(struct hsi_port *port,
struct platform_device *pd)
{
struct omap_ssi_port *omap_port = hsi_port_drvdata(port);
int cawake_irq;
int err;
- if (omap_port->wake_gpio == -1) {
+ if (!omap_port->wake_gpio) {
omap_port->wake_irq = -1;
return 0;
}
- cawake_irq = gpio_to_irq(omap_port->wake_gpio);
+ cawake_irq = gpiod_to_irq(omap_port->wake_gpio);
omap_port->wake_irq = cawake_irq;
tasklet_init(&omap_port->wake_tasklet, ssi_wake_tasklet,
@@ -1060,7 +1071,7 @@
return err;
}
-static void __init ssi_queues_init(struct omap_ssi_port *omap_port)
+static void ssi_queues_init(struct omap_ssi_port *omap_port)
{
unsigned int ch;
@@ -1071,7 +1082,7 @@
INIT_LIST_HEAD(&omap_port->brkqueue);
}
-static int __init ssi_port_get_iomem(struct platform_device *pd,
+static int ssi_port_get_iomem(struct platform_device *pd,
const char *name, void __iomem **pbase, dma_addr_t *phy)
{
struct hsi_port *port = platform_get_drvdata(pd);
@@ -1104,24 +1115,19 @@
return 0;
}
-static int __init ssi_port_probe(struct platform_device *pd)
+static int ssi_port_probe(struct platform_device *pd)
{
struct device_node *np = pd->dev.of_node;
struct hsi_port *port;
struct omap_ssi_port *omap_port;
struct hsi_controller *ssi = dev_get_drvdata(pd->dev.parent);
struct omap_ssi_controller *omap_ssi = hsi_controller_drvdata(ssi);
- int cawake_gpio = 0;
+ struct gpio_desc *cawake_gpio = NULL;
u32 port_id;
int err;
dev_dbg(&pd->dev, "init ssi port...\n");
- if (!try_module_get(ssi->owner)) {
- dev_err(&pd->dev, "could not increment parent module refcount\n");
- return -ENODEV;
- }
-
if (!ssi->port || !omap_ssi->port) {
dev_err(&pd->dev, "ssi controller not initialized!\n");
err = -ENODEV;
@@ -1147,20 +1153,10 @@
goto error;
}
- err = of_get_named_gpio(np, "ti,ssi-cawake-gpio", 0);
- if (err < 0) {
- dev_err(&pd->dev, "DT data is missing cawake gpio (err=%d)\n",
- err);
- goto error;
- }
- cawake_gpio = err;
-
- err = devm_gpio_request_one(&port->device, cawake_gpio, GPIOF_DIR_IN,
- "cawake");
- if (err) {
- dev_err(&pd->dev, "could not request cawake gpio (err=%d)!\n",
- err);
- err = -ENXIO;
+ cawake_gpio = devm_gpiod_get(&pd->dev, "ti,ssi-cawake", GPIOD_IN);
+ if (IS_ERR(cawake_gpio)) {
+ err = PTR_ERR(cawake_gpio);
+ dev_err(&pd->dev, "couldn't get cawake gpio (err=%d)!\n", err);
goto error;
}
@@ -1219,8 +1215,7 @@
hsi_add_clients_from_dt(port, np);
- dev_info(&pd->dev, "ssi port %u successfully initialized (cawake=%d)\n",
- port_id, cawake_gpio);
+ dev_info(&pd->dev, "ssi port %u successfully initialized\n", port_id);
return 0;
@@ -1228,7 +1223,7 @@
return err;
}
-static int __exit ssi_port_remove(struct platform_device *pd)
+static int ssi_port_remove(struct platform_device *pd)
{
struct hsi_port *port = platform_get_drvdata(pd);
struct omap_ssi_port *omap_port = hsi_port_drvdata(port);
@@ -1253,12 +1248,28 @@
omap_ssi->port[omap_port->port_id] = NULL;
platform_set_drvdata(pd, NULL);
- module_put(ssi->owner);
pm_runtime_disable(&pd->dev);
return 0;
}
+static int ssi_restore_divisor(struct omap_ssi_port *omap_port)
+{
+ writel_relaxed(omap_port->sst.divisor,
+ omap_port->sst_base + SSI_SST_DIVISOR_REG);
+
+ return 0;
+}
+
+void omap_ssi_port_update_fclk(struct hsi_controller *ssi,
+ struct omap_ssi_port *omap_port)
+{
+ /* update divisor */
+ u32 div = ssi_calculate_div(ssi);
+ omap_port->sst.divisor = div;
+ ssi_restore_divisor(omap_port);
+}
+
#ifdef CONFIG_PM
static int ssi_save_port_ctx(struct omap_ssi_port *omap_port)
{
@@ -1311,14 +1322,6 @@
return 0;
}
-static int ssi_restore_divisor(struct omap_ssi_port *omap_port)
-{
- writel_relaxed(omap_port->sst.divisor,
- omap_port->sst_base + SSI_SST_DIVISOR_REG);
-
- return 0;
-}
-
static int omap_ssi_port_runtime_suspend(struct device *dev)
{
struct hsi_port *port = dev_get_drvdata(dev);
@@ -1380,19 +1383,12 @@
#define omap_ssi_port_of_match NULL
#endif
-static struct platform_driver ssi_port_pdriver = {
- .remove = __exit_p(ssi_port_remove),
+struct platform_driver ssi_port_pdriver = {
+ .probe = ssi_port_probe,
+ .remove = ssi_port_remove,
.driver = {
.name = "omap_ssi_port",
.of_match_table = omap_ssi_port_of_match,
.pm = DEV_PM_OPS,
},
};
-
-module_platform_driver_probe(ssi_port_pdriver, ssi_port_probe);
-
-MODULE_ALIAS("platform:omap_ssi_port");
-MODULE_AUTHOR("Carlos Chinea <carlos.chinea@nokia.com>");
-MODULE_AUTHOR("Sebastian Reichel <sre@kernel.org>");
-MODULE_DESCRIPTION("Synchronous Serial Interface Port Driver");
-MODULE_LICENSE("GPL v2");
diff --git a/drivers/hv/channel_mgmt.c b/drivers/hv/channel_mgmt.c
index 38b682ba..b6c1211 100644
--- a/drivers/hv/channel_mgmt.c
+++ b/drivers/hv/channel_mgmt.c
@@ -597,27 +597,55 @@
static void vmbus_wait_for_unload(void)
{
- int cpu = smp_processor_id();
- void *page_addr = hv_context.synic_message_page[cpu];
- struct hv_message *msg = (struct hv_message *)page_addr +
- VMBUS_MESSAGE_SINT;
+ int cpu;
+ void *page_addr;
+ struct hv_message *msg;
struct vmbus_channel_message_header *hdr;
- bool unloaded = false;
+ u32 message_type;
+ /*
+ * CHANNELMSG_UNLOAD_RESPONSE is always delivered to the CPU which was
+ * used for initial contact or to CPU0 depending on host version. When
+ * we're crashing on a different CPU let's hope that IRQ handler on
+ * the cpu which receives CHANNELMSG_UNLOAD_RESPONSE is still
+ * functional and vmbus_unload_response() will complete
+ * vmbus_connection.unload_event. If not, the last thing we can do is
+ * read message pages for all CPUs directly.
+ */
while (1) {
- if (READ_ONCE(msg->header.message_type) == HVMSG_NONE) {
- mdelay(10);
- continue;
+ if (completion_done(&vmbus_connection.unload_event))
+ break;
+
+ for_each_online_cpu(cpu) {
+ page_addr = hv_context.synic_message_page[cpu];
+ msg = (struct hv_message *)page_addr +
+ VMBUS_MESSAGE_SINT;
+
+ message_type = READ_ONCE(msg->header.message_type);
+ if (message_type == HVMSG_NONE)
+ continue;
+
+ hdr = (struct vmbus_channel_message_header *)
+ msg->u.payload;
+
+ if (hdr->msgtype == CHANNELMSG_UNLOAD_RESPONSE)
+ complete(&vmbus_connection.unload_event);
+
+ vmbus_signal_eom(msg, message_type);
}
- hdr = (struct vmbus_channel_message_header *)msg->u.payload;
- if (hdr->msgtype == CHANNELMSG_UNLOAD_RESPONSE)
- unloaded = true;
+ mdelay(10);
+ }
- vmbus_signal_eom(msg);
-
- if (unloaded)
- break;
+ /*
+ * We're crashing and already got the UNLOAD_RESPONSE, cleanup all
+ * maybe-pending messages on all CPUs to be able to receive new
+ * messages after we reconnect.
+ */
+ for_each_online_cpu(cpu) {
+ page_addr = hv_context.synic_message_page[cpu];
+ msg = (struct hv_message *)page_addr + VMBUS_MESSAGE_SINT;
+ msg->header.message_type = HVMSG_NONE;
}
}
diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c
index d02f137..fcf8a02 100644
--- a/drivers/hv/connection.c
+++ b/drivers/hv/connection.c
@@ -495,3 +495,4 @@
hv_do_hypercall(HVCALL_SIGNAL_EVENT, channel->sig_event, NULL);
}
+EXPORT_SYMBOL_GPL(vmbus_set_event);
diff --git a/drivers/hv/hv_balloon.c b/drivers/hv/hv_balloon.c
index b853b4b..df35fb7 100644
--- a/drivers/hv/hv_balloon.c
+++ b/drivers/hv/hv_balloon.c
@@ -714,7 +714,7 @@
* If the pfn range we are dealing with is not in the current
* "hot add block", move on.
*/
- if ((start_pfn >= has->end_pfn))
+ if (start_pfn < has->start_pfn || start_pfn >= has->end_pfn)
continue;
/*
* If the current hot add-request extends beyond
@@ -768,7 +768,7 @@
* If the pfn range we are dealing with is not in the current
* "hot add block", move on.
*/
- if ((start_pfn >= has->end_pfn))
+ if (start_pfn < has->start_pfn || start_pfn >= has->end_pfn)
continue;
old_covered_state = has->covered_end_pfn;
@@ -1400,6 +1400,7 @@
* This is a normal hot-add request specifying
* hot-add memory.
*/
+ dm->host_specified_ha_region = false;
ha_pg_range = &ha_msg->range;
dm->ha_wrk.ha_page_range = *ha_pg_range;
dm->ha_wrk.ha_region_range.page_range = 0;
diff --git a/drivers/hv/hv_kvp.c b/drivers/hv/hv_kvp.c
index 9b9b370..cb1a916 100644
--- a/drivers/hv/hv_kvp.c
+++ b/drivers/hv/hv_kvp.c
@@ -78,9 +78,11 @@
static void kvp_respond_to_host(struct hv_kvp_msg *msg, int error);
static void kvp_timeout_func(struct work_struct *dummy);
+static void kvp_host_handshake_func(struct work_struct *dummy);
static void kvp_register(int);
static DECLARE_DELAYED_WORK(kvp_timeout_work, kvp_timeout_func);
+static DECLARE_DELAYED_WORK(kvp_host_handshake_work, kvp_host_handshake_func);
static DECLARE_WORK(kvp_sendkey_work, kvp_send_key);
static const char kvp_devname[] = "vmbus/hv_kvp";
@@ -130,6 +132,11 @@
hv_poll_channel(kvp_transaction.recv_channel, kvp_poll_wrapper);
}
+static void kvp_host_handshake_func(struct work_struct *dummy)
+{
+ hv_poll_channel(kvp_transaction.recv_channel, hv_kvp_onchannelcallback);
+}
+
static int kvp_handle_handshake(struct hv_kvp_msg *msg)
{
switch (msg->kvp_hdr.operation) {
@@ -154,6 +161,12 @@
pr_debug("KVP: userspace daemon ver. %d registered\n",
KVP_OP_REGISTER);
kvp_register(dm_reg_value);
+
+ /*
+ * If we're still negotiating with the host cancel the timeout
+ * work to not poll the channel twice.
+ */
+ cancel_delayed_work_sync(&kvp_host_handshake_work);
hv_poll_channel(kvp_transaction.recv_channel, kvp_poll_wrapper);
return 0;
@@ -594,7 +607,22 @@
struct icmsg_negotiate *negop = NULL;
int util_fw_version;
int kvp_srv_version;
+ static enum {NEGO_NOT_STARTED,
+ NEGO_IN_PROGRESS,
+ NEGO_FINISHED} host_negotiatied = NEGO_NOT_STARTED;
+ if (host_negotiatied == NEGO_NOT_STARTED &&
+ kvp_transaction.state < HVUTIL_READY) {
+ /*
+ * If userspace daemon is not connected and host is asking
+ * us to negotiate we need to delay to not lose messages.
+ * This is important for Failover IP setting.
+ */
+ host_negotiatied = NEGO_IN_PROGRESS;
+ schedule_delayed_work(&kvp_host_handshake_work,
+ HV_UTIL_NEGO_TIMEOUT * HZ);
+ return;
+ }
if (kvp_transaction.state > HVUTIL_READY)
return;
@@ -672,6 +700,8 @@
vmbus_sendpacket(channel, recv_buffer,
recvlen, requestid,
VM_PKT_DATA_INBAND, 0);
+
+ host_negotiatied = NEGO_FINISHED;
}
}
@@ -708,6 +738,7 @@
void hv_kvp_deinit(void)
{
kvp_transaction.state = HVUTIL_DEVICE_DYING;
+ cancel_delayed_work_sync(&kvp_host_handshake_work);
cancel_delayed_work_sync(&kvp_timeout_work);
cancel_work_sync(&kvp_sendkey_work);
hvutil_transport_destroy(hvt);
diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
index 12321b9..718b5c7 100644
--- a/drivers/hv/hyperv_vmbus.h
+++ b/drivers/hv/hyperv_vmbus.h
@@ -36,6 +36,11 @@
#define HV_UTIL_TIMEOUT 30
/*
+ * Timeout for guest-host handshake for services.
+ */
+#define HV_UTIL_NEGO_TIMEOUT 60
+
+/*
* The below CPUID leaves are present if VersionAndFeatures.HypervisorPresent
* is set by CPUID(HVCPUID_VERSION_FEATURES).
*/
@@ -620,9 +625,21 @@
channel_message_table[CHANNELMSG_COUNT];
/* Free the message slot and signal end-of-message if required */
-static inline void vmbus_signal_eom(struct hv_message *msg)
+static inline void vmbus_signal_eom(struct hv_message *msg, u32 old_msg_type)
{
- msg->header.message_type = HVMSG_NONE;
+ /*
+ * On crash we're reading some other CPU's message page and we need
+ * to be careful: this other CPU may already had cleared the header
+ * and the host may already had delivered some other message there.
+ * In case we blindly write msg->header.message_type we're going
+ * to lose it. We can still lose a message of the same type but
+ * we count on the fact that there can only be one
+ * CHANNELMSG_UNLOAD_RESPONSE and we don't care about other messages
+ * on crash.
+ */
+ if (cmpxchg(&msg->header.message_type, old_msg_type,
+ HVMSG_NONE) != old_msg_type)
+ return;
/*
* Make sure the write to MessageType (ie set to
@@ -667,8 +684,6 @@
int vmbus_post_msg(void *buffer, size_t buflen);
-void vmbus_set_event(struct vmbus_channel *channel);
-
void vmbus_on_event(unsigned long data);
void vmbus_on_msg_dpc(unsigned long data);
diff --git a/drivers/hv/ring_buffer.c b/drivers/hv/ring_buffer.c
index a40a73a..fe586bf 100644
--- a/drivers/hv/ring_buffer.c
+++ b/drivers/hv/ring_buffer.c
@@ -33,25 +33,21 @@
void hv_begin_read(struct hv_ring_buffer_info *rbi)
{
rbi->ring_buffer->interrupt_mask = 1;
- mb();
+ virt_mb();
}
u32 hv_end_read(struct hv_ring_buffer_info *rbi)
{
- u32 read;
- u32 write;
rbi->ring_buffer->interrupt_mask = 0;
- mb();
+ virt_mb();
/*
* Now check to see if the ring buffer is still empty.
* If it is not, we raced and we need to process new
* incoming messages.
*/
- hv_get_ringbuffer_availbytes(rbi, &read, &write);
-
- return read;
+ return hv_get_bytes_to_read(rbi);
}
/*
@@ -72,69 +68,17 @@
static bool hv_need_to_signal(u32 old_write, struct hv_ring_buffer_info *rbi)
{
- mb();
- if (rbi->ring_buffer->interrupt_mask)
+ virt_mb();
+ if (READ_ONCE(rbi->ring_buffer->interrupt_mask))
return false;
/* check interrupt_mask before read_index */
- rmb();
+ virt_rmb();
/*
* This is the only case we need to signal when the
* ring transitions from being empty to non-empty.
*/
- if (old_write == rbi->ring_buffer->read_index)
- return true;
-
- return false;
-}
-
-/*
- * To optimize the flow management on the send-side,
- * when the sender is blocked because of lack of
- * sufficient space in the ring buffer, potential the
- * consumer of the ring buffer can signal the producer.
- * This is controlled by the following parameters:
- *
- * 1. pending_send_sz: This is the size in bytes that the
- * producer is trying to send.
- * 2. The feature bit feat_pending_send_sz set to indicate if
- * the consumer of the ring will signal when the ring
- * state transitions from being full to a state where
- * there is room for the producer to send the pending packet.
- */
-
-static bool hv_need_to_signal_on_read(struct hv_ring_buffer_info *rbi)
-{
- u32 cur_write_sz;
- u32 r_size;
- u32 write_loc;
- u32 read_loc = rbi->ring_buffer->read_index;
- u32 pending_sz;
-
- /*
- * Issue a full memory barrier before making the signaling decision.
- * Here is the reason for having this barrier:
- * If the reading of the pend_sz (in this function)
- * were to be reordered and read before we commit the new read
- * index (in the calling function) we could
- * have a problem. If the host were to set the pending_sz after we
- * have sampled pending_sz and go to sleep before we commit the
- * read index, we could miss sending the interrupt. Issue a full
- * memory barrier to address this.
- */
- mb();
-
- pending_sz = rbi->ring_buffer->pending_send_sz;
- write_loc = rbi->ring_buffer->write_index;
- /* If the other end is not blocked on write don't bother. */
- if (pending_sz == 0)
- return false;
-
- r_size = rbi->ring_datasize;
- cur_write_sz = write_loc >= read_loc ? r_size - (write_loc - read_loc) :
- read_loc - write_loc;
-
- if (cur_write_sz >= pending_sz)
+ if (old_write == READ_ONCE(rbi->ring_buffer->read_index))
return true;
return false;
@@ -188,17 +132,9 @@
u32 next_read_location)
{
ring_info->ring_buffer->read_index = next_read_location;
+ ring_info->priv_read_index = next_read_location;
}
-
-/* Get the start of the ring buffer. */
-static inline void *
-hv_get_ring_buffer(struct hv_ring_buffer_info *ring_info)
-{
- return (void *)ring_info->ring_buffer->buffer;
-}
-
-
/* Get the size of the ring buffer. */
static inline u32
hv_get_ring_buffersize(struct hv_ring_buffer_info *ring_info)
@@ -332,7 +268,6 @@
{
int i = 0;
u32 bytes_avail_towrite;
- u32 bytes_avail_toread;
u32 totalbytes_towrite = 0;
u32 next_write_location;
@@ -348,9 +283,7 @@
if (lock)
spin_lock_irqsave(&outring_info->ring_lock, flags);
- hv_get_ringbuffer_availbytes(outring_info,
- &bytes_avail_toread,
- &bytes_avail_towrite);
+ bytes_avail_towrite = hv_get_bytes_to_write(outring_info);
/*
* If there is only room for the packet, assume it is full.
@@ -384,7 +317,7 @@
sizeof(u64));
/* Issue a full memory barrier before updating the write index */
- mb();
+ virt_mb();
/* Now, update the write location */
hv_set_next_write_location(outring_info, next_write_location);
@@ -401,7 +334,6 @@
void *buffer, u32 buflen, u32 *buffer_actual_len,
u64 *requestid, bool *signal, bool raw)
{
- u32 bytes_avail_towrite;
u32 bytes_avail_toread;
u32 next_read_location = 0;
u64 prev_indices = 0;
@@ -417,10 +349,7 @@
*buffer_actual_len = 0;
*requestid = 0;
- hv_get_ringbuffer_availbytes(inring_info,
- &bytes_avail_toread,
- &bytes_avail_towrite);
-
+ bytes_avail_toread = hv_get_bytes_to_read(inring_info);
/* Make sure there is something to read */
if (bytes_avail_toread < sizeof(desc)) {
/*
@@ -464,7 +393,7 @@
* the writer may start writing to the read area once the read index
* is updated.
*/
- mb();
+ virt_mb();
/* Update the read index */
hv_set_next_read_location(inring_info, next_read_location);
diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
index 64713ff..952f20f 100644
--- a/drivers/hv/vmbus_drv.c
+++ b/drivers/hv/vmbus_drv.c
@@ -41,6 +41,7 @@
#include <linux/ptrace.h>
#include <linux/screen_info.h>
#include <linux/kdebug.h>
+#include <linux/efi.h>
#include "hyperv_vmbus.h"
static struct acpi_device *hv_acpi_dev;
@@ -101,7 +102,10 @@
.notifier_call = hyperv_panic_event,
};
+static const char *fb_mmio_name = "fb_range";
+static struct resource *fb_mmio;
struct resource *hyperv_mmio;
+DEFINE_SEMAPHORE(hyperv_mmio_lock);
static int vmbus_exists(void)
{
@@ -708,7 +712,7 @@
if (dev->event_handler)
dev->event_handler(dev);
- vmbus_signal_eom(msg);
+ vmbus_signal_eom(msg, HVMSG_TIMER_EXPIRED);
}
void vmbus_on_msg_dpc(unsigned long data)
@@ -720,8 +724,9 @@
struct vmbus_channel_message_header *hdr;
struct vmbus_channel_message_table_entry *entry;
struct onmessage_work_context *ctx;
+ u32 message_type = msg->header.message_type;
- if (msg->header.message_type == HVMSG_NONE)
+ if (message_type == HVMSG_NONE)
/* no msg */
return;
@@ -746,7 +751,7 @@
entry->message_handler(hdr);
msg_handled:
- vmbus_signal_eom(msg);
+ vmbus_signal_eom(msg, message_type);
}
static void vmbus_isr(void)
@@ -1048,7 +1053,6 @@
new_res->end = end;
/*
- * Stick ranges from higher in address space at the front of the list.
* If two ranges are adjacent, merge them.
*/
do {
@@ -1069,7 +1073,7 @@
break;
}
- if ((*old_res)->end < new_res->start) {
+ if ((*old_res)->start > new_res->end) {
new_res->sibling = *old_res;
if (prev_res)
(*prev_res)->sibling = new_res;
@@ -1091,6 +1095,12 @@
struct resource *next_res;
if (hyperv_mmio) {
+ if (fb_mmio) {
+ __release_region(hyperv_mmio, fb_mmio->start,
+ resource_size(fb_mmio));
+ fb_mmio = NULL;
+ }
+
for (cur_res = hyperv_mmio; cur_res; cur_res = next_res) {
next_res = cur_res->sibling;
kfree(cur_res);
@@ -1100,6 +1110,30 @@
return 0;
}
+static void vmbus_reserve_fb(void)
+{
+ int size;
+ /*
+ * Make a claim for the frame buffer in the resource tree under the
+ * first node, which will be the one below 4GB. The length seems to
+ * be underreported, particularly in a Generation 1 VM. So start out
+ * reserving a larger area and make it smaller until it succeeds.
+ */
+
+ if (screen_info.lfb_base) {
+ if (efi_enabled(EFI_BOOT))
+ size = max_t(__u32, screen_info.lfb_size, 0x800000);
+ else
+ size = max_t(__u32, screen_info.lfb_size, 0x4000000);
+
+ for (; !fb_mmio && (size >= 0x100000); size >>= 1) {
+ fb_mmio = __request_region(hyperv_mmio,
+ screen_info.lfb_base, size,
+ fb_mmio_name, 0);
+ }
+ }
+}
+
/**
* vmbus_allocate_mmio() - Pick a memory-mapped I/O range.
* @new: If successful, supplied a pointer to the
@@ -1128,11 +1162,33 @@
resource_size_t size, resource_size_t align,
bool fb_overlap_ok)
{
- struct resource *iter;
- resource_size_t range_min, range_max, start, local_min, local_max;
+ struct resource *iter, *shadow;
+ resource_size_t range_min, range_max, start;
const char *dev_n = dev_name(&device_obj->device);
- u32 fb_end = screen_info.lfb_base + (screen_info.lfb_size << 1);
- int i;
+ int retval;
+
+ retval = -ENXIO;
+ down(&hyperv_mmio_lock);
+
+ /*
+ * If overlaps with frame buffers are allowed, then first attempt to
+ * make the allocation from within the reserved region. Because it
+ * is already reserved, no shadow allocation is necessary.
+ */
+ if (fb_overlap_ok && fb_mmio && !(min > fb_mmio->end) &&
+ !(max < fb_mmio->start)) {
+
+ range_min = fb_mmio->start;
+ range_max = fb_mmio->end;
+ start = (range_min + align - 1) & ~(align - 1);
+ for (; start + size - 1 <= range_max; start += align) {
+ *new = request_mem_region_exclusive(start, size, dev_n);
+ if (*new) {
+ retval = 0;
+ goto exit;
+ }
+ }
+ }
for (iter = hyperv_mmio; iter; iter = iter->sibling) {
if ((iter->start >= max) || (iter->end <= min))
@@ -1140,46 +1196,56 @@
range_min = iter->start;
range_max = iter->end;
+ start = (range_min + align - 1) & ~(align - 1);
+ for (; start + size - 1 <= range_max; start += align) {
+ shadow = __request_region(iter, start, size, NULL,
+ IORESOURCE_BUSY);
+ if (!shadow)
+ continue;
- /* If this range overlaps the frame buffer, split it into
- two tries. */
- for (i = 0; i < 2; i++) {
- local_min = range_min;
- local_max = range_max;
- if (fb_overlap_ok || (range_min >= fb_end) ||
- (range_max <= screen_info.lfb_base)) {
- i++;
- } else {
- if ((range_min <= screen_info.lfb_base) &&
- (range_max >= screen_info.lfb_base)) {
- /*
- * The frame buffer is in this window,
- * so trim this into the part that
- * preceeds the frame buffer.
- */
- local_max = screen_info.lfb_base - 1;
- range_min = fb_end;
- } else {
- range_min = fb_end;
- continue;
- }
+ *new = request_mem_region_exclusive(start, size, dev_n);
+ if (*new) {
+ shadow->name = (char *)*new;
+ retval = 0;
+ goto exit;
}
- start = (local_min + align - 1) & ~(align - 1);
- for (; start + size - 1 <= local_max; start += align) {
- *new = request_mem_region_exclusive(start, size,
- dev_n);
- if (*new)
- return 0;
- }
+ __release_region(iter, start, size);
}
}
- return -ENXIO;
+exit:
+ up(&hyperv_mmio_lock);
+ return retval;
}
EXPORT_SYMBOL_GPL(vmbus_allocate_mmio);
/**
+ * vmbus_free_mmio() - Free a memory-mapped I/O range.
+ * @start: Base address of region to release.
+ * @size: Size of the range to be allocated
+ *
+ * This function releases anything requested by
+ * vmbus_mmio_allocate().
+ */
+void vmbus_free_mmio(resource_size_t start, resource_size_t size)
+{
+ struct resource *iter;
+
+ down(&hyperv_mmio_lock);
+ for (iter = hyperv_mmio; iter; iter = iter->sibling) {
+ if ((iter->start >= start + size) || (iter->end <= start))
+ continue;
+
+ __release_region(iter, start, size);
+ }
+ release_mem_region(start, size);
+ up(&hyperv_mmio_lock);
+
+}
+EXPORT_SYMBOL_GPL(vmbus_free_mmio);
+
+/**
* vmbus_cpu_number_to_vp_number() - Map CPU to VP.
* @cpu_number: CPU number in Linux terms
*
@@ -1219,8 +1285,10 @@
if (ACPI_FAILURE(result))
continue;
- if (hyperv_mmio)
+ if (hyperv_mmio) {
+ vmbus_reserve_fb();
break;
+ }
}
ret_val = 0;
diff --git a/drivers/hwspinlock/hwspinlock_core.c b/drivers/hwspinlock/hwspinlock_core.c
index d50c701..4074441 100644
--- a/drivers/hwspinlock/hwspinlock_core.c
+++ b/drivers/hwspinlock/hwspinlock_core.c
@@ -313,7 +313,7 @@
hwlock = radix_tree_deref_slot(slot);
if (unlikely(!hwlock))
continue;
- if (radix_tree_is_indirect_ptr(hwlock)) {
+ if (radix_tree_deref_retry(hwlock)) {
slot = radix_tree_iter_retry(&iter);
continue;
}
diff --git a/drivers/hwtracing/coresight/Kconfig b/drivers/hwtracing/coresight/Kconfig
index db05410..130cb21 100644
--- a/drivers/hwtracing/coresight/Kconfig
+++ b/drivers/hwtracing/coresight/Kconfig
@@ -78,4 +78,15 @@
programmable ATB replicator sends the ATB trace stream from the
ETB/ETF to the TPIUi and ETR.
+config CORESIGHT_STM
+ bool "CoreSight System Trace Macrocell driver"
+ depends on (ARM && !(CPU_32v3 || CPU_32v4 || CPU_32v4T)) || ARM64
+ select CORESIGHT_LINKS_AND_SINKS
+ select STM
+ help
+ This driver provides support for hardware assisted software
+ instrumentation based tracing. This is primarily used for
+ logging useful software events or data coming from various entities
+ in the system, possibly running different OSs
+
endif
diff --git a/drivers/hwtracing/coresight/Makefile b/drivers/hwtracing/coresight/Makefile
index cf8c6d6..af480d9 100644
--- a/drivers/hwtracing/coresight/Makefile
+++ b/drivers/hwtracing/coresight/Makefile
@@ -1,15 +1,18 @@
#
# Makefile for CoreSight drivers.
#
-obj-$(CONFIG_CORESIGHT) += coresight.o
+obj-$(CONFIG_CORESIGHT) += coresight.o coresight-etm-perf.o
obj-$(CONFIG_OF) += of_coresight.o
-obj-$(CONFIG_CORESIGHT_LINK_AND_SINK_TMC) += coresight-tmc.o
+obj-$(CONFIG_CORESIGHT_LINK_AND_SINK_TMC) += coresight-tmc.o \
+ coresight-tmc-etf.o \
+ coresight-tmc-etr.o
obj-$(CONFIG_CORESIGHT_SINK_TPIU) += coresight-tpiu.o
obj-$(CONFIG_CORESIGHT_SINK_ETBV10) += coresight-etb10.o
obj-$(CONFIG_CORESIGHT_LINKS_AND_SINKS) += coresight-funnel.o \
coresight-replicator.o
obj-$(CONFIG_CORESIGHT_SOURCE_ETM3X) += coresight-etm3x.o coresight-etm-cp14.o \
- coresight-etm3x-sysfs.o \
- coresight-etm-perf.o
-obj-$(CONFIG_CORESIGHT_SOURCE_ETM4X) += coresight-etm4x.o
+ coresight-etm3x-sysfs.o
+obj-$(CONFIG_CORESIGHT_SOURCE_ETM4X) += coresight-etm4x.o \
+ coresight-etm4x-sysfs.o
obj-$(CONFIG_CORESIGHT_QCOM_REPLICATOR) += coresight-replicator-qcom.o
+obj-$(CONFIG_CORESIGHT_STM) += coresight-stm.o
diff --git a/drivers/hwtracing/coresight/coresight-etb10.c b/drivers/hwtracing/coresight/coresight-etb10.c
index acbce79..4d20b0b 100644
--- a/drivers/hwtracing/coresight/coresight-etb10.c
+++ b/drivers/hwtracing/coresight/coresight-etb10.c
@@ -71,26 +71,6 @@
#define ETB_FRAME_SIZE_WORDS 4
/**
- * struct cs_buffer - keep track of a recording session' specifics
- * @cur: index of the current buffer
- * @nr_pages: max number of pages granted to us
- * @offset: offset within the current buffer
- * @data_size: how much we collected in this run
- * @lost: other than zero if we had a HW buffer wrap around
- * @snapshot: is this run in snapshot mode
- * @data_pages: a handle the ring buffer
- */
-struct cs_buffers {
- unsigned int cur;
- unsigned int nr_pages;
- unsigned long offset;
- local_t data_size;
- local_t lost;
- bool snapshot;
- void **data_pages;
-};
-
-/**
* struct etb_drvdata - specifics associated to an ETB component
* @base: memory mapped base address for this component.
* @dev: the device entity associated to this component.
@@ -440,7 +420,7 @@
u32 mask = ~(ETB_FRAME_SIZE_WORDS - 1);
/* The new read pointer must be frame size aligned */
- to_read -= handle->size & mask;
+ to_read = handle->size & mask;
/*
* Move the RAM read pointer up, keeping in mind that
* everything is in frame size units.
@@ -448,7 +428,8 @@
read_ptr = (write_ptr + drvdata->buffer_depth) -
to_read / ETB_FRAME_SIZE_WORDS;
/* Wrap around if need be*/
- read_ptr &= ~(drvdata->buffer_depth - 1);
+ if (read_ptr > (drvdata->buffer_depth - 1))
+ read_ptr -= drvdata->buffer_depth;
/* let the decoder know we've skipped ahead */
local_inc(&buf->lost);
}
@@ -579,47 +560,29 @@
.llseek = no_llseek,
};
-static ssize_t status_show(struct device *dev,
- struct device_attribute *attr, char *buf)
-{
- unsigned long flags;
- u32 etb_rdr, etb_sr, etb_rrp, etb_rwp;
- u32 etb_trg, etb_cr, etb_ffsr, etb_ffcr;
- struct etb_drvdata *drvdata = dev_get_drvdata(dev->parent);
+#define coresight_etb10_simple_func(name, offset) \
+ coresight_simple_func(struct etb_drvdata, name, offset)
- pm_runtime_get_sync(drvdata->dev);
- spin_lock_irqsave(&drvdata->spinlock, flags);
- CS_UNLOCK(drvdata->base);
+coresight_etb10_simple_func(rdp, ETB_RAM_DEPTH_REG);
+coresight_etb10_simple_func(sts, ETB_STATUS_REG);
+coresight_etb10_simple_func(rrp, ETB_RAM_READ_POINTER);
+coresight_etb10_simple_func(rwp, ETB_RAM_WRITE_POINTER);
+coresight_etb10_simple_func(trg, ETB_TRG);
+coresight_etb10_simple_func(ctl, ETB_CTL_REG);
+coresight_etb10_simple_func(ffsr, ETB_FFSR);
+coresight_etb10_simple_func(ffcr, ETB_FFCR);
- etb_rdr = readl_relaxed(drvdata->base + ETB_RAM_DEPTH_REG);
- etb_sr = readl_relaxed(drvdata->base + ETB_STATUS_REG);
- etb_rrp = readl_relaxed(drvdata->base + ETB_RAM_READ_POINTER);
- etb_rwp = readl_relaxed(drvdata->base + ETB_RAM_WRITE_POINTER);
- etb_trg = readl_relaxed(drvdata->base + ETB_TRG);
- etb_cr = readl_relaxed(drvdata->base + ETB_CTL_REG);
- etb_ffsr = readl_relaxed(drvdata->base + ETB_FFSR);
- etb_ffcr = readl_relaxed(drvdata->base + ETB_FFCR);
-
- CS_LOCK(drvdata->base);
- spin_unlock_irqrestore(&drvdata->spinlock, flags);
-
- pm_runtime_put(drvdata->dev);
-
- return sprintf(buf,
- "Depth:\t\t0x%x\n"
- "Status:\t\t0x%x\n"
- "RAM read ptr:\t0x%x\n"
- "RAM wrt ptr:\t0x%x\n"
- "Trigger cnt:\t0x%x\n"
- "Control:\t0x%x\n"
- "Flush status:\t0x%x\n"
- "Flush ctrl:\t0x%x\n",
- etb_rdr, etb_sr, etb_rrp, etb_rwp,
- etb_trg, etb_cr, etb_ffsr, etb_ffcr);
-
- return -EINVAL;
-}
-static DEVICE_ATTR_RO(status);
+static struct attribute *coresight_etb_mgmt_attrs[] = {
+ &dev_attr_rdp.attr,
+ &dev_attr_sts.attr,
+ &dev_attr_rrp.attr,
+ &dev_attr_rwp.attr,
+ &dev_attr_trg.attr,
+ &dev_attr_ctl.attr,
+ &dev_attr_ffsr.attr,
+ &dev_attr_ffcr.attr,
+ NULL,
+};
static ssize_t trigger_cntr_show(struct device *dev,
struct device_attribute *attr, char *buf)
@@ -649,10 +612,23 @@
static struct attribute *coresight_etb_attrs[] = {
&dev_attr_trigger_cntr.attr,
- &dev_attr_status.attr,
NULL,
};
-ATTRIBUTE_GROUPS(coresight_etb);
+
+static const struct attribute_group coresight_etb_group = {
+ .attrs = coresight_etb_attrs,
+};
+
+static const struct attribute_group coresight_etb_mgmt_group = {
+ .attrs = coresight_etb_mgmt_attrs,
+ .name = "mgmt",
+};
+
+const struct attribute_group *coresight_etb_groups[] = {
+ &coresight_etb_group,
+ &coresight_etb_mgmt_group,
+ NULL,
+};
static int etb_probe(struct amba_device *adev, const struct amba_id *id)
{
@@ -729,7 +705,6 @@
if (ret)
goto err_misc_register;
- dev_info(dev, "ETB initialized\n");
return 0;
err_misc_register:
diff --git a/drivers/hwtracing/coresight/coresight-etm3x-sysfs.c b/drivers/hwtracing/coresight/coresight-etm3x-sysfs.c
index cbb4046..02d4b62 100644
--- a/drivers/hwtracing/coresight/coresight-etm3x-sysfs.c
+++ b/drivers/hwtracing/coresight/coresight-etm3x-sysfs.c
@@ -1221,26 +1221,19 @@
NULL,
};
-#define coresight_simple_func(name, offset) \
-static ssize_t name##_show(struct device *_dev, \
- struct device_attribute *attr, char *buf) \
-{ \
- struct etm_drvdata *drvdata = dev_get_drvdata(_dev->parent); \
- return scnprintf(buf, PAGE_SIZE, "0x%x\n", \
- readl_relaxed(drvdata->base + offset)); \
-} \
-DEVICE_ATTR_RO(name)
+#define coresight_etm3x_simple_func(name, offset) \
+ coresight_simple_func(struct etm_drvdata, name, offset)
-coresight_simple_func(etmccr, ETMCCR);
-coresight_simple_func(etmccer, ETMCCER);
-coresight_simple_func(etmscr, ETMSCR);
-coresight_simple_func(etmidr, ETMIDR);
-coresight_simple_func(etmcr, ETMCR);
-coresight_simple_func(etmtraceidr, ETMTRACEIDR);
-coresight_simple_func(etmteevr, ETMTEEVR);
-coresight_simple_func(etmtssvr, ETMTSSCR);
-coresight_simple_func(etmtecr1, ETMTECR1);
-coresight_simple_func(etmtecr2, ETMTECR2);
+coresight_etm3x_simple_func(etmccr, ETMCCR);
+coresight_etm3x_simple_func(etmccer, ETMCCER);
+coresight_etm3x_simple_func(etmscr, ETMSCR);
+coresight_etm3x_simple_func(etmidr, ETMIDR);
+coresight_etm3x_simple_func(etmcr, ETMCR);
+coresight_etm3x_simple_func(etmtraceidr, ETMTRACEIDR);
+coresight_etm3x_simple_func(etmteevr, ETMTEEVR);
+coresight_etm3x_simple_func(etmtssvr, ETMTSSCR);
+coresight_etm3x_simple_func(etmtecr1, ETMTECR1);
+coresight_etm3x_simple_func(etmtecr2, ETMTECR2);
static struct attribute *coresight_etm_mgmt_attrs[] = {
&dev_attr_etmccr.attr,
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c b/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c
new file mode 100644
index 0000000..7c84308
--- /dev/null
+++ b/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c
@@ -0,0 +1,2126 @@
+/*
+ * Copyright(C) 2015 Linaro Limited. All rights reserved.
+ * Author: Mathieu Poirier <mathieu.poirier@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/pm_runtime.h>
+#include <linux/sysfs.h>
+#include "coresight-etm4x.h"
+
+static int etm4_set_mode_exclude(struct etmv4_drvdata *drvdata, bool exclude)
+{
+ u8 idx;
+ struct etmv4_config *config = &drvdata->config;
+
+ idx = config->addr_idx;
+
+ /*
+ * TRCACATRn.TYPE bit[1:0]: type of comparison
+ * the trace unit performs
+ */
+ if (BMVAL(config->addr_acc[idx], 0, 1) == ETM_INSTR_ADDR) {
+ if (idx % 2 != 0)
+ return -EINVAL;
+
+ /*
+ * We are performing instruction address comparison. Set the
+ * relevant bit of ViewInst Include/Exclude Control register
+ * for corresponding address comparator pair.
+ */
+ if (config->addr_type[idx] != ETM_ADDR_TYPE_RANGE ||
+ config->addr_type[idx + 1] != ETM_ADDR_TYPE_RANGE)
+ return -EINVAL;
+
+ if (exclude == true) {
+ /*
+ * Set exclude bit and unset the include bit
+ * corresponding to comparator pair
+ */
+ config->viiectlr |= BIT(idx / 2 + 16);
+ config->viiectlr &= ~BIT(idx / 2);
+ } else {
+ /*
+ * Set include bit and unset exclude bit
+ * corresponding to comparator pair
+ */
+ config->viiectlr |= BIT(idx / 2);
+ config->viiectlr &= ~BIT(idx / 2 + 16);
+ }
+ }
+ return 0;
+}
+
+static ssize_t nr_pe_cmp_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+
+ val = drvdata->nr_pe_cmp;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+static DEVICE_ATTR_RO(nr_pe_cmp);
+
+static ssize_t nr_addr_cmp_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+
+ val = drvdata->nr_addr_cmp;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+static DEVICE_ATTR_RO(nr_addr_cmp);
+
+static ssize_t nr_cntr_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+
+ val = drvdata->nr_cntr;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+static DEVICE_ATTR_RO(nr_cntr);
+
+static ssize_t nr_ext_inp_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+
+ val = drvdata->nr_ext_inp;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+static DEVICE_ATTR_RO(nr_ext_inp);
+
+static ssize_t numcidc_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+
+ val = drvdata->numcidc;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+static DEVICE_ATTR_RO(numcidc);
+
+static ssize_t numvmidc_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+
+ val = drvdata->numvmidc;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+static DEVICE_ATTR_RO(numvmidc);
+
+static ssize_t nrseqstate_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+
+ val = drvdata->nrseqstate;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+static DEVICE_ATTR_RO(nrseqstate);
+
+static ssize_t nr_resource_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+
+ val = drvdata->nr_resource;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+static DEVICE_ATTR_RO(nr_resource);
+
+static ssize_t nr_ss_cmp_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+
+ val = drvdata->nr_ss_cmp;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+static DEVICE_ATTR_RO(nr_ss_cmp);
+
+static ssize_t reset_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ int i;
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+ if (val)
+ config->mode = 0x0;
+
+ /* Disable data tracing: do not trace load and store data transfers */
+ config->mode &= ~(ETM_MODE_LOAD | ETM_MODE_STORE);
+ config->cfg &= ~(BIT(1) | BIT(2));
+
+ /* Disable data value and data address tracing */
+ config->mode &= ~(ETM_MODE_DATA_TRACE_ADDR |
+ ETM_MODE_DATA_TRACE_VAL);
+ config->cfg &= ~(BIT(16) | BIT(17));
+
+ /* Disable all events tracing */
+ config->eventctrl0 = 0x0;
+ config->eventctrl1 = 0x0;
+
+ /* Disable timestamp event */
+ config->ts_ctrl = 0x0;
+
+ /* Disable stalling */
+ config->stall_ctrl = 0x0;
+
+ /* Reset trace synchronization period to 2^8 = 256 bytes*/
+ if (drvdata->syncpr == false)
+ config->syncfreq = 0x8;
+
+ /*
+ * Enable ViewInst to trace everything with start-stop logic in
+ * started state. ARM recommends start-stop logic is set before
+ * each trace run.
+ */
+ config->vinst_ctrl |= BIT(0);
+ if (drvdata->nr_addr_cmp == true) {
+ config->mode |= ETM_MODE_VIEWINST_STARTSTOP;
+ /* SSSTATUS, bit[9] */
+ config->vinst_ctrl |= BIT(9);
+ }
+
+ /* No address range filtering for ViewInst */
+ config->viiectlr = 0x0;
+
+ /* No start-stop filtering for ViewInst */
+ config->vissctlr = 0x0;
+
+ /* Disable seq events */
+ for (i = 0; i < drvdata->nrseqstate-1; i++)
+ config->seq_ctrl[i] = 0x0;
+ config->seq_rst = 0x0;
+ config->seq_state = 0x0;
+
+ /* Disable external input events */
+ config->ext_inp = 0x0;
+
+ config->cntr_idx = 0x0;
+ for (i = 0; i < drvdata->nr_cntr; i++) {
+ config->cntrldvr[i] = 0x0;
+ config->cntr_ctrl[i] = 0x0;
+ config->cntr_val[i] = 0x0;
+ }
+
+ config->res_idx = 0x0;
+ for (i = 0; i < drvdata->nr_resource; i++)
+ config->res_ctrl[i] = 0x0;
+
+ for (i = 0; i < drvdata->nr_ss_cmp; i++) {
+ config->ss_ctrl[i] = 0x0;
+ config->ss_pe_cmp[i] = 0x0;
+ }
+
+ config->addr_idx = 0x0;
+ for (i = 0; i < drvdata->nr_addr_cmp * 2; i++) {
+ config->addr_val[i] = 0x0;
+ config->addr_acc[i] = 0x0;
+ config->addr_type[i] = ETM_ADDR_TYPE_NONE;
+ }
+
+ config->ctxid_idx = 0x0;
+ for (i = 0; i < drvdata->numcidc; i++) {
+ config->ctxid_pid[i] = 0x0;
+ config->ctxid_vpid[i] = 0x0;
+ }
+
+ config->ctxid_mask0 = 0x0;
+ config->ctxid_mask1 = 0x0;
+
+ config->vmid_idx = 0x0;
+ for (i = 0; i < drvdata->numvmidc; i++)
+ config->vmid_val[i] = 0x0;
+ config->vmid_mask0 = 0x0;
+ config->vmid_mask1 = 0x0;
+
+ drvdata->trcid = drvdata->cpu + 1;
+
+ spin_unlock(&drvdata->spinlock);
+
+ return size;
+}
+static DEVICE_ATTR_WO(reset);
+
+static ssize_t mode_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ val = config->mode;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t mode_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ unsigned long val, mode;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+ config->mode = val & ETMv4_MODE_ALL;
+
+ if (config->mode & ETM_MODE_EXCLUDE)
+ etm4_set_mode_exclude(drvdata, true);
+ else
+ etm4_set_mode_exclude(drvdata, false);
+
+ if (drvdata->instrp0 == true) {
+ /* start by clearing instruction P0 field */
+ config->cfg &= ~(BIT(1) | BIT(2));
+ if (config->mode & ETM_MODE_LOAD)
+ /* 0b01 Trace load instructions as P0 instructions */
+ config->cfg |= BIT(1);
+ if (config->mode & ETM_MODE_STORE)
+ /* 0b10 Trace store instructions as P0 instructions */
+ config->cfg |= BIT(2);
+ if (config->mode & ETM_MODE_LOAD_STORE)
+ /*
+ * 0b11 Trace load and store instructions
+ * as P0 instructions
+ */
+ config->cfg |= BIT(1) | BIT(2);
+ }
+
+ /* bit[3], Branch broadcast mode */
+ if ((config->mode & ETM_MODE_BB) && (drvdata->trcbb == true))
+ config->cfg |= BIT(3);
+ else
+ config->cfg &= ~BIT(3);
+
+ /* bit[4], Cycle counting instruction trace bit */
+ if ((config->mode & ETMv4_MODE_CYCACC) &&
+ (drvdata->trccci == true))
+ config->cfg |= BIT(4);
+ else
+ config->cfg &= ~BIT(4);
+
+ /* bit[6], Context ID tracing bit */
+ if ((config->mode & ETMv4_MODE_CTXID) && (drvdata->ctxid_size))
+ config->cfg |= BIT(6);
+ else
+ config->cfg &= ~BIT(6);
+
+ if ((config->mode & ETM_MODE_VMID) && (drvdata->vmid_size))
+ config->cfg |= BIT(7);
+ else
+ config->cfg &= ~BIT(7);
+
+ /* bits[10:8], Conditional instruction tracing bit */
+ mode = ETM_MODE_COND(config->mode);
+ if (drvdata->trccond == true) {
+ config->cfg &= ~(BIT(8) | BIT(9) | BIT(10));
+ config->cfg |= mode << 8;
+ }
+
+ /* bit[11], Global timestamp tracing bit */
+ if ((config->mode & ETMv4_MODE_TIMESTAMP) && (drvdata->ts_size))
+ config->cfg |= BIT(11);
+ else
+ config->cfg &= ~BIT(11);
+
+ /* bit[12], Return stack enable bit */
+ if ((config->mode & ETM_MODE_RETURNSTACK) &&
+ (drvdata->retstack == true))
+ config->cfg |= BIT(12);
+ else
+ config->cfg &= ~BIT(12);
+
+ /* bits[14:13], Q element enable field */
+ mode = ETM_MODE_QELEM(config->mode);
+ /* start by clearing QE bits */
+ config->cfg &= ~(BIT(13) | BIT(14));
+ /* if supported, Q elements with instruction counts are enabled */
+ if ((mode & BIT(0)) && (drvdata->q_support & BIT(0)))
+ config->cfg |= BIT(13);
+ /*
+ * if supported, Q elements with and without instruction
+ * counts are enabled
+ */
+ if ((mode & BIT(1)) && (drvdata->q_support & BIT(1)))
+ config->cfg |= BIT(14);
+
+ /* bit[11], AMBA Trace Bus (ATB) trigger enable bit */
+ if ((config->mode & ETM_MODE_ATB_TRIGGER) &&
+ (drvdata->atbtrig == true))
+ config->eventctrl1 |= BIT(11);
+ else
+ config->eventctrl1 &= ~BIT(11);
+
+ /* bit[12], Low-power state behavior override bit */
+ if ((config->mode & ETM_MODE_LPOVERRIDE) &&
+ (drvdata->lpoverride == true))
+ config->eventctrl1 |= BIT(12);
+ else
+ config->eventctrl1 &= ~BIT(12);
+
+ /* bit[8], Instruction stall bit */
+ if (config->mode & ETM_MODE_ISTALL_EN)
+ config->stall_ctrl |= BIT(8);
+ else
+ config->stall_ctrl &= ~BIT(8);
+
+ /* bit[10], Prioritize instruction trace bit */
+ if (config->mode & ETM_MODE_INSTPRIO)
+ config->stall_ctrl |= BIT(10);
+ else
+ config->stall_ctrl &= ~BIT(10);
+
+ /* bit[13], Trace overflow prevention bit */
+ if ((config->mode & ETM_MODE_NOOVERFLOW) &&
+ (drvdata->nooverflow == true))
+ config->stall_ctrl |= BIT(13);
+ else
+ config->stall_ctrl &= ~BIT(13);
+
+ /* bit[9] Start/stop logic control bit */
+ if (config->mode & ETM_MODE_VIEWINST_STARTSTOP)
+ config->vinst_ctrl |= BIT(9);
+ else
+ config->vinst_ctrl &= ~BIT(9);
+
+ /* bit[10], Whether a trace unit must trace a Reset exception */
+ if (config->mode & ETM_MODE_TRACE_RESET)
+ config->vinst_ctrl |= BIT(10);
+ else
+ config->vinst_ctrl &= ~BIT(10);
+
+ /* bit[11], Whether a trace unit must trace a system error exception */
+ if ((config->mode & ETM_MODE_TRACE_ERR) &&
+ (drvdata->trc_error == true))
+ config->vinst_ctrl |= BIT(11);
+ else
+ config->vinst_ctrl &= ~BIT(11);
+
+ if (config->mode & (ETM_MODE_EXCL_KERN | ETM_MODE_EXCL_USER))
+ etm4_config_trace_mode(config);
+
+ spin_unlock(&drvdata->spinlock);
+
+ return size;
+}
+static DEVICE_ATTR_RW(mode);
+
+static ssize_t pe_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ val = config->pe_sel;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t pe_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+ if (val > drvdata->nr_pe) {
+ spin_unlock(&drvdata->spinlock);
+ return -EINVAL;
+ }
+
+ config->pe_sel = val;
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(pe);
+
+static ssize_t event_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ val = config->eventctrl0;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t event_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+ switch (drvdata->nr_event) {
+ case 0x0:
+ /* EVENT0, bits[7:0] */
+ config->eventctrl0 = val & 0xFF;
+ break;
+ case 0x1:
+ /* EVENT1, bits[15:8] */
+ config->eventctrl0 = val & 0xFFFF;
+ break;
+ case 0x2:
+ /* EVENT2, bits[23:16] */
+ config->eventctrl0 = val & 0xFFFFFF;
+ break;
+ case 0x3:
+ /* EVENT3, bits[31:24] */
+ config->eventctrl0 = val;
+ break;
+ default:
+ break;
+ }
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(event);
+
+static ssize_t event_instren_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ val = BMVAL(config->eventctrl1, 0, 3);
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t event_instren_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+ /* start by clearing all instruction event enable bits */
+ config->eventctrl1 &= ~(BIT(0) | BIT(1) | BIT(2) | BIT(3));
+ switch (drvdata->nr_event) {
+ case 0x0:
+ /* generate Event element for event 1 */
+ config->eventctrl1 |= val & BIT(1);
+ break;
+ case 0x1:
+ /* generate Event element for event 1 and 2 */
+ config->eventctrl1 |= val & (BIT(0) | BIT(1));
+ break;
+ case 0x2:
+ /* generate Event element for event 1, 2 and 3 */
+ config->eventctrl1 |= val & (BIT(0) | BIT(1) | BIT(2));
+ break;
+ case 0x3:
+ /* generate Event element for all 4 events */
+ config->eventctrl1 |= val & 0xF;
+ break;
+ default:
+ break;
+ }
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(event_instren);
+
+static ssize_t event_ts_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ val = config->ts_ctrl;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t event_ts_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+ if (!drvdata->ts_size)
+ return -EINVAL;
+
+ config->ts_ctrl = val & ETMv4_EVENT_MASK;
+ return size;
+}
+static DEVICE_ATTR_RW(event_ts);
+
+static ssize_t syncfreq_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ val = config->syncfreq;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t syncfreq_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+ if (drvdata->syncpr == true)
+ return -EINVAL;
+
+ config->syncfreq = val & ETMv4_SYNC_MASK;
+ return size;
+}
+static DEVICE_ATTR_RW(syncfreq);
+
+static ssize_t cyc_threshold_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ val = config->ccctlr;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t cyc_threshold_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+ if (val < drvdata->ccitmin)
+ return -EINVAL;
+
+ config->ccctlr = val & ETM_CYC_THRESHOLD_MASK;
+ return size;
+}
+static DEVICE_ATTR_RW(cyc_threshold);
+
+static ssize_t bb_ctrl_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ val = config->bb_ctrl;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t bb_ctrl_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+ if (drvdata->trcbb == false)
+ return -EINVAL;
+ if (!drvdata->nr_addr_cmp)
+ return -EINVAL;
+ /*
+ * Bit[7:0] selects which address range comparator is used for
+ * branch broadcast control.
+ */
+ if (BMVAL(val, 0, 7) > drvdata->nr_addr_cmp)
+ return -EINVAL;
+
+ config->bb_ctrl = val;
+ return size;
+}
+static DEVICE_ATTR_RW(bb_ctrl);
+
+static ssize_t event_vinst_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ val = config->vinst_ctrl & ETMv4_EVENT_MASK;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t event_vinst_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+ val &= ETMv4_EVENT_MASK;
+ config->vinst_ctrl &= ~ETMv4_EVENT_MASK;
+ config->vinst_ctrl |= val;
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(event_vinst);
+
+static ssize_t s_exlevel_vinst_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ val = BMVAL(config->vinst_ctrl, 16, 19);
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t s_exlevel_vinst_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+ /* clear all EXLEVEL_S bits (bit[18] is never implemented) */
+ config->vinst_ctrl &= ~(BIT(16) | BIT(17) | BIT(19));
+ /* enable instruction tracing for corresponding exception level */
+ val &= drvdata->s_ex_level;
+ config->vinst_ctrl |= (val << 16);
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(s_exlevel_vinst);
+
+static ssize_t ns_exlevel_vinst_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ /* EXLEVEL_NS, bits[23:20] */
+ val = BMVAL(config->vinst_ctrl, 20, 23);
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t ns_exlevel_vinst_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+ /* clear EXLEVEL_NS bits (bit[23] is never implemented */
+ config->vinst_ctrl &= ~(BIT(20) | BIT(21) | BIT(22));
+ /* enable instruction tracing for corresponding exception level */
+ val &= drvdata->ns_ex_level;
+ config->vinst_ctrl |= (val << 20);
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(ns_exlevel_vinst);
+
+static ssize_t addr_idx_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ val = config->addr_idx;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t addr_idx_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+ if (val >= drvdata->nr_addr_cmp * 2)
+ return -EINVAL;
+
+ /*
+ * Use spinlock to ensure index doesn't change while it gets
+ * dereferenced multiple times within a spinlock block elsewhere.
+ */
+ spin_lock(&drvdata->spinlock);
+ config->addr_idx = val;
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(addr_idx);
+
+static ssize_t addr_instdatatype_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ ssize_t len;
+ u8 val, idx;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->addr_idx;
+ val = BMVAL(config->addr_acc[idx], 0, 1);
+ len = scnprintf(buf, PAGE_SIZE, "%s\n",
+ val == ETM_INSTR_ADDR ? "instr" :
+ (val == ETM_DATA_LOAD_ADDR ? "data_load" :
+ (val == ETM_DATA_STORE_ADDR ? "data_store" :
+ "data_load_store")));
+ spin_unlock(&drvdata->spinlock);
+ return len;
+}
+
+static ssize_t addr_instdatatype_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ u8 idx;
+ char str[20] = "";
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (strlen(buf) >= 20)
+ return -EINVAL;
+ if (sscanf(buf, "%s", str) != 1)
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->addr_idx;
+ if (!strcmp(str, "instr"))
+ /* TYPE, bits[1:0] */
+ config->addr_acc[idx] &= ~(BIT(0) | BIT(1));
+
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(addr_instdatatype);
+
+static ssize_t addr_single_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ u8 idx;
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ idx = config->addr_idx;
+ spin_lock(&drvdata->spinlock);
+ if (!(config->addr_type[idx] == ETM_ADDR_TYPE_NONE ||
+ config->addr_type[idx] == ETM_ADDR_TYPE_SINGLE)) {
+ spin_unlock(&drvdata->spinlock);
+ return -EPERM;
+ }
+ val = (unsigned long)config->addr_val[idx];
+ spin_unlock(&drvdata->spinlock);
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t addr_single_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ u8 idx;
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->addr_idx;
+ if (!(config->addr_type[idx] == ETM_ADDR_TYPE_NONE ||
+ config->addr_type[idx] == ETM_ADDR_TYPE_SINGLE)) {
+ spin_unlock(&drvdata->spinlock);
+ return -EPERM;
+ }
+
+ config->addr_val[idx] = (u64)val;
+ config->addr_type[idx] = ETM_ADDR_TYPE_SINGLE;
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(addr_single);
+
+static ssize_t addr_range_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ u8 idx;
+ unsigned long val1, val2;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->addr_idx;
+ if (idx % 2 != 0) {
+ spin_unlock(&drvdata->spinlock);
+ return -EPERM;
+ }
+ if (!((config->addr_type[idx] == ETM_ADDR_TYPE_NONE &&
+ config->addr_type[idx + 1] == ETM_ADDR_TYPE_NONE) ||
+ (config->addr_type[idx] == ETM_ADDR_TYPE_RANGE &&
+ config->addr_type[idx + 1] == ETM_ADDR_TYPE_RANGE))) {
+ spin_unlock(&drvdata->spinlock);
+ return -EPERM;
+ }
+
+ val1 = (unsigned long)config->addr_val[idx];
+ val2 = (unsigned long)config->addr_val[idx + 1];
+ spin_unlock(&drvdata->spinlock);
+ return scnprintf(buf, PAGE_SIZE, "%#lx %#lx\n", val1, val2);
+}
+
+static ssize_t addr_range_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ u8 idx;
+ unsigned long val1, val2;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (sscanf(buf, "%lx %lx", &val1, &val2) != 2)
+ return -EINVAL;
+ /* lower address comparator cannot have a higher address value */
+ if (val1 > val2)
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->addr_idx;
+ if (idx % 2 != 0) {
+ spin_unlock(&drvdata->spinlock);
+ return -EPERM;
+ }
+
+ if (!((config->addr_type[idx] == ETM_ADDR_TYPE_NONE &&
+ config->addr_type[idx + 1] == ETM_ADDR_TYPE_NONE) ||
+ (config->addr_type[idx] == ETM_ADDR_TYPE_RANGE &&
+ config->addr_type[idx + 1] == ETM_ADDR_TYPE_RANGE))) {
+ spin_unlock(&drvdata->spinlock);
+ return -EPERM;
+ }
+
+ config->addr_val[idx] = (u64)val1;
+ config->addr_type[idx] = ETM_ADDR_TYPE_RANGE;
+ config->addr_val[idx + 1] = (u64)val2;
+ config->addr_type[idx + 1] = ETM_ADDR_TYPE_RANGE;
+ /*
+ * Program include or exclude control bits for vinst or vdata
+ * whenever we change addr comparators to ETM_ADDR_TYPE_RANGE
+ */
+ if (config->mode & ETM_MODE_EXCLUDE)
+ etm4_set_mode_exclude(drvdata, true);
+ else
+ etm4_set_mode_exclude(drvdata, false);
+
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(addr_range);
+
+static ssize_t addr_start_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ u8 idx;
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->addr_idx;
+
+ if (!(config->addr_type[idx] == ETM_ADDR_TYPE_NONE ||
+ config->addr_type[idx] == ETM_ADDR_TYPE_START)) {
+ spin_unlock(&drvdata->spinlock);
+ return -EPERM;
+ }
+
+ val = (unsigned long)config->addr_val[idx];
+ spin_unlock(&drvdata->spinlock);
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t addr_start_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ u8 idx;
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->addr_idx;
+ if (!drvdata->nr_addr_cmp) {
+ spin_unlock(&drvdata->spinlock);
+ return -EINVAL;
+ }
+ if (!(config->addr_type[idx] == ETM_ADDR_TYPE_NONE ||
+ config->addr_type[idx] == ETM_ADDR_TYPE_START)) {
+ spin_unlock(&drvdata->spinlock);
+ return -EPERM;
+ }
+
+ config->addr_val[idx] = (u64)val;
+ config->addr_type[idx] = ETM_ADDR_TYPE_START;
+ config->vissctlr |= BIT(idx);
+ /* SSSTATUS, bit[9] - turn on start/stop logic */
+ config->vinst_ctrl |= BIT(9);
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(addr_start);
+
+static ssize_t addr_stop_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ u8 idx;
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->addr_idx;
+
+ if (!(config->addr_type[idx] == ETM_ADDR_TYPE_NONE ||
+ config->addr_type[idx] == ETM_ADDR_TYPE_STOP)) {
+ spin_unlock(&drvdata->spinlock);
+ return -EPERM;
+ }
+
+ val = (unsigned long)config->addr_val[idx];
+ spin_unlock(&drvdata->spinlock);
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t addr_stop_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ u8 idx;
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->addr_idx;
+ if (!drvdata->nr_addr_cmp) {
+ spin_unlock(&drvdata->spinlock);
+ return -EINVAL;
+ }
+ if (!(config->addr_type[idx] == ETM_ADDR_TYPE_NONE ||
+ config->addr_type[idx] == ETM_ADDR_TYPE_STOP)) {
+ spin_unlock(&drvdata->spinlock);
+ return -EPERM;
+ }
+
+ config->addr_val[idx] = (u64)val;
+ config->addr_type[idx] = ETM_ADDR_TYPE_STOP;
+ config->vissctlr |= BIT(idx + 16);
+ /* SSSTATUS, bit[9] - turn on start/stop logic */
+ config->vinst_ctrl |= BIT(9);
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(addr_stop);
+
+static ssize_t addr_ctxtype_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ ssize_t len;
+ u8 idx, val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->addr_idx;
+ /* CONTEXTTYPE, bits[3:2] */
+ val = BMVAL(config->addr_acc[idx], 2, 3);
+ len = scnprintf(buf, PAGE_SIZE, "%s\n", val == ETM_CTX_NONE ? "none" :
+ (val == ETM_CTX_CTXID ? "ctxid" :
+ (val == ETM_CTX_VMID ? "vmid" : "all")));
+ spin_unlock(&drvdata->spinlock);
+ return len;
+}
+
+static ssize_t addr_ctxtype_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ u8 idx;
+ char str[10] = "";
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (strlen(buf) >= 10)
+ return -EINVAL;
+ if (sscanf(buf, "%s", str) != 1)
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->addr_idx;
+ if (!strcmp(str, "none"))
+ /* start by clearing context type bits */
+ config->addr_acc[idx] &= ~(BIT(2) | BIT(3));
+ else if (!strcmp(str, "ctxid")) {
+ /* 0b01 The trace unit performs a Context ID */
+ if (drvdata->numcidc) {
+ config->addr_acc[idx] |= BIT(2);
+ config->addr_acc[idx] &= ~BIT(3);
+ }
+ } else if (!strcmp(str, "vmid")) {
+ /* 0b10 The trace unit performs a VMID */
+ if (drvdata->numvmidc) {
+ config->addr_acc[idx] &= ~BIT(2);
+ config->addr_acc[idx] |= BIT(3);
+ }
+ } else if (!strcmp(str, "all")) {
+ /*
+ * 0b11 The trace unit performs a Context ID
+ * comparison and a VMID
+ */
+ if (drvdata->numcidc)
+ config->addr_acc[idx] |= BIT(2);
+ if (drvdata->numvmidc)
+ config->addr_acc[idx] |= BIT(3);
+ }
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(addr_ctxtype);
+
+static ssize_t addr_context_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ u8 idx;
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->addr_idx;
+ /* context ID comparator bits[6:4] */
+ val = BMVAL(config->addr_acc[idx], 4, 6);
+ spin_unlock(&drvdata->spinlock);
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t addr_context_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ u8 idx;
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+ if ((drvdata->numcidc <= 1) && (drvdata->numvmidc <= 1))
+ return -EINVAL;
+ if (val >= (drvdata->numcidc >= drvdata->numvmidc ?
+ drvdata->numcidc : drvdata->numvmidc))
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->addr_idx;
+ /* clear context ID comparator bits[6:4] */
+ config->addr_acc[idx] &= ~(BIT(4) | BIT(5) | BIT(6));
+ config->addr_acc[idx] |= (val << 4);
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(addr_context);
+
+static ssize_t seq_idx_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ val = config->seq_idx;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t seq_idx_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+ if (val >= drvdata->nrseqstate - 1)
+ return -EINVAL;
+
+ /*
+ * Use spinlock to ensure index doesn't change while it gets
+ * dereferenced multiple times within a spinlock block elsewhere.
+ */
+ spin_lock(&drvdata->spinlock);
+ config->seq_idx = val;
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(seq_idx);
+
+static ssize_t seq_state_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ val = config->seq_state;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t seq_state_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+ if (val >= drvdata->nrseqstate)
+ return -EINVAL;
+
+ config->seq_state = val;
+ return size;
+}
+static DEVICE_ATTR_RW(seq_state);
+
+static ssize_t seq_event_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ u8 idx;
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->seq_idx;
+ val = config->seq_ctrl[idx];
+ spin_unlock(&drvdata->spinlock);
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t seq_event_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ u8 idx;
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->seq_idx;
+ /* RST, bits[7:0] */
+ config->seq_ctrl[idx] = val & 0xFF;
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(seq_event);
+
+static ssize_t seq_reset_event_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ val = config->seq_rst;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t seq_reset_event_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+ if (!(drvdata->nrseqstate))
+ return -EINVAL;
+
+ config->seq_rst = val & ETMv4_EVENT_MASK;
+ return size;
+}
+static DEVICE_ATTR_RW(seq_reset_event);
+
+static ssize_t cntr_idx_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ val = config->cntr_idx;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t cntr_idx_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+ if (val >= drvdata->nr_cntr)
+ return -EINVAL;
+
+ /*
+ * Use spinlock to ensure index doesn't change while it gets
+ * dereferenced multiple times within a spinlock block elsewhere.
+ */
+ spin_lock(&drvdata->spinlock);
+ config->cntr_idx = val;
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(cntr_idx);
+
+static ssize_t cntrldvr_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ u8 idx;
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->cntr_idx;
+ val = config->cntrldvr[idx];
+ spin_unlock(&drvdata->spinlock);
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t cntrldvr_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ u8 idx;
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+ if (val > ETM_CNTR_MAX_VAL)
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->cntr_idx;
+ config->cntrldvr[idx] = val;
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(cntrldvr);
+
+static ssize_t cntr_val_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ u8 idx;
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->cntr_idx;
+ val = config->cntr_val[idx];
+ spin_unlock(&drvdata->spinlock);
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t cntr_val_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ u8 idx;
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+ if (val > ETM_CNTR_MAX_VAL)
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->cntr_idx;
+ config->cntr_val[idx] = val;
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(cntr_val);
+
+static ssize_t cntr_ctrl_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ u8 idx;
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->cntr_idx;
+ val = config->cntr_ctrl[idx];
+ spin_unlock(&drvdata->spinlock);
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t cntr_ctrl_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ u8 idx;
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->cntr_idx;
+ config->cntr_ctrl[idx] = val;
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(cntr_ctrl);
+
+static ssize_t res_idx_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ val = config->res_idx;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t res_idx_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+ /* Resource selector pair 0 is always implemented and reserved */
+ if ((val == 0) || (val >= drvdata->nr_resource))
+ return -EINVAL;
+
+ /*
+ * Use spinlock to ensure index doesn't change while it gets
+ * dereferenced multiple times within a spinlock block elsewhere.
+ */
+ spin_lock(&drvdata->spinlock);
+ config->res_idx = val;
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(res_idx);
+
+static ssize_t res_ctrl_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ u8 idx;
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->res_idx;
+ val = config->res_ctrl[idx];
+ spin_unlock(&drvdata->spinlock);
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t res_ctrl_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ u8 idx;
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->res_idx;
+ /* For odd idx pair inversal bit is RES0 */
+ if (idx % 2 != 0)
+ /* PAIRINV, bit[21] */
+ val &= ~BIT(21);
+ config->res_ctrl[idx] = val;
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(res_ctrl);
+
+static ssize_t ctxid_idx_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ val = config->ctxid_idx;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t ctxid_idx_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+ if (val >= drvdata->numcidc)
+ return -EINVAL;
+
+ /*
+ * Use spinlock to ensure index doesn't change while it gets
+ * dereferenced multiple times within a spinlock block elsewhere.
+ */
+ spin_lock(&drvdata->spinlock);
+ config->ctxid_idx = val;
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(ctxid_idx);
+
+static ssize_t ctxid_pid_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ u8 idx;
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->ctxid_idx;
+ val = (unsigned long)config->ctxid_vpid[idx];
+ spin_unlock(&drvdata->spinlock);
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t ctxid_pid_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ u8 idx;
+ unsigned long vpid, pid;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ /*
+ * only implemented when ctxid tracing is enabled, i.e. at least one
+ * ctxid comparator is implemented and ctxid is greater than 0 bits
+ * in length
+ */
+ if (!drvdata->ctxid_size || !drvdata->numcidc)
+ return -EINVAL;
+ if (kstrtoul(buf, 16, &vpid))
+ return -EINVAL;
+
+ pid = coresight_vpid_to_pid(vpid);
+
+ spin_lock(&drvdata->spinlock);
+ idx = config->ctxid_idx;
+ config->ctxid_pid[idx] = (u64)pid;
+ config->ctxid_vpid[idx] = (u64)vpid;
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(ctxid_pid);
+
+static ssize_t ctxid_masks_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val1, val2;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ spin_lock(&drvdata->spinlock);
+ val1 = config->ctxid_mask0;
+ val2 = config->ctxid_mask1;
+ spin_unlock(&drvdata->spinlock);
+ return scnprintf(buf, PAGE_SIZE, "%#lx %#lx\n", val1, val2);
+}
+
+static ssize_t ctxid_masks_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ u8 i, j, maskbyte;
+ unsigned long val1, val2, mask;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ /*
+ * only implemented when ctxid tracing is enabled, i.e. at least one
+ * ctxid comparator is implemented and ctxid is greater than 0 bits
+ * in length
+ */
+ if (!drvdata->ctxid_size || !drvdata->numcidc)
+ return -EINVAL;
+ if (sscanf(buf, "%lx %lx", &val1, &val2) != 2)
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+ /*
+ * each byte[0..3] controls mask value applied to ctxid
+ * comparator[0..3]
+ */
+ switch (drvdata->numcidc) {
+ case 0x1:
+ /* COMP0, bits[7:0] */
+ config->ctxid_mask0 = val1 & 0xFF;
+ break;
+ case 0x2:
+ /* COMP1, bits[15:8] */
+ config->ctxid_mask0 = val1 & 0xFFFF;
+ break;
+ case 0x3:
+ /* COMP2, bits[23:16] */
+ config->ctxid_mask0 = val1 & 0xFFFFFF;
+ break;
+ case 0x4:
+ /* COMP3, bits[31:24] */
+ config->ctxid_mask0 = val1;
+ break;
+ case 0x5:
+ /* COMP4, bits[7:0] */
+ config->ctxid_mask0 = val1;
+ config->ctxid_mask1 = val2 & 0xFF;
+ break;
+ case 0x6:
+ /* COMP5, bits[15:8] */
+ config->ctxid_mask0 = val1;
+ config->ctxid_mask1 = val2 & 0xFFFF;
+ break;
+ case 0x7:
+ /* COMP6, bits[23:16] */
+ config->ctxid_mask0 = val1;
+ config->ctxid_mask1 = val2 & 0xFFFFFF;
+ break;
+ case 0x8:
+ /* COMP7, bits[31:24] */
+ config->ctxid_mask0 = val1;
+ config->ctxid_mask1 = val2;
+ break;
+ default:
+ break;
+ }
+ /*
+ * If software sets a mask bit to 1, it must program relevant byte
+ * of ctxid comparator value 0x0, otherwise behavior is unpredictable.
+ * For example, if bit[3] of ctxid_mask0 is 1, we must clear bits[31:24]
+ * of ctxid comparator0 value (corresponding to byte 0) register.
+ */
+ mask = config->ctxid_mask0;
+ for (i = 0; i < drvdata->numcidc; i++) {
+ /* mask value of corresponding ctxid comparator */
+ maskbyte = mask & ETMv4_EVENT_MASK;
+ /*
+ * each bit corresponds to a byte of respective ctxid comparator
+ * value register
+ */
+ for (j = 0; j < 8; j++) {
+ if (maskbyte & 1)
+ config->ctxid_pid[i] &= ~(0xFF << (j * 8));
+ maskbyte >>= 1;
+ }
+ /* Select the next ctxid comparator mask value */
+ if (i == 3)
+ /* ctxid comparators[4-7] */
+ mask = config->ctxid_mask1;
+ else
+ mask >>= 0x8;
+ }
+
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(ctxid_masks);
+
+static ssize_t vmid_idx_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ val = config->vmid_idx;
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t vmid_idx_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+ if (val >= drvdata->numvmidc)
+ return -EINVAL;
+
+ /*
+ * Use spinlock to ensure index doesn't change while it gets
+ * dereferenced multiple times within a spinlock block elsewhere.
+ */
+ spin_lock(&drvdata->spinlock);
+ config->vmid_idx = val;
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(vmid_idx);
+
+static ssize_t vmid_val_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ val = (unsigned long)config->vmid_val[config->vmid_idx];
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t vmid_val_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ unsigned long val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ /*
+ * only implemented when vmid tracing is enabled, i.e. at least one
+ * vmid comparator is implemented and at least 8 bit vmid size
+ */
+ if (!drvdata->vmid_size || !drvdata->numvmidc)
+ return -EINVAL;
+ if (kstrtoul(buf, 16, &val))
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+ config->vmid_val[config->vmid_idx] = (u64)val;
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(vmid_val);
+
+static ssize_t vmid_masks_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ unsigned long val1, val2;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ spin_lock(&drvdata->spinlock);
+ val1 = config->vmid_mask0;
+ val2 = config->vmid_mask1;
+ spin_unlock(&drvdata->spinlock);
+ return scnprintf(buf, PAGE_SIZE, "%#lx %#lx\n", val1, val2);
+}
+
+static ssize_t vmid_masks_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ u8 i, j, maskbyte;
+ unsigned long val1, val2, mask;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ struct etmv4_config *config = &drvdata->config;
+
+ /*
+ * only implemented when vmid tracing is enabled, i.e. at least one
+ * vmid comparator is implemented and at least 8 bit vmid size
+ */
+ if (!drvdata->vmid_size || !drvdata->numvmidc)
+ return -EINVAL;
+ if (sscanf(buf, "%lx %lx", &val1, &val2) != 2)
+ return -EINVAL;
+
+ spin_lock(&drvdata->spinlock);
+
+ /*
+ * each byte[0..3] controls mask value applied to vmid
+ * comparator[0..3]
+ */
+ switch (drvdata->numvmidc) {
+ case 0x1:
+ /* COMP0, bits[7:0] */
+ config->vmid_mask0 = val1 & 0xFF;
+ break;
+ case 0x2:
+ /* COMP1, bits[15:8] */
+ config->vmid_mask0 = val1 & 0xFFFF;
+ break;
+ case 0x3:
+ /* COMP2, bits[23:16] */
+ config->vmid_mask0 = val1 & 0xFFFFFF;
+ break;
+ case 0x4:
+ /* COMP3, bits[31:24] */
+ config->vmid_mask0 = val1;
+ break;
+ case 0x5:
+ /* COMP4, bits[7:0] */
+ config->vmid_mask0 = val1;
+ config->vmid_mask1 = val2 & 0xFF;
+ break;
+ case 0x6:
+ /* COMP5, bits[15:8] */
+ config->vmid_mask0 = val1;
+ config->vmid_mask1 = val2 & 0xFFFF;
+ break;
+ case 0x7:
+ /* COMP6, bits[23:16] */
+ config->vmid_mask0 = val1;
+ config->vmid_mask1 = val2 & 0xFFFFFF;
+ break;
+ case 0x8:
+ /* COMP7, bits[31:24] */
+ config->vmid_mask0 = val1;
+ config->vmid_mask1 = val2;
+ break;
+ default:
+ break;
+ }
+
+ /*
+ * If software sets a mask bit to 1, it must program relevant byte
+ * of vmid comparator value 0x0, otherwise behavior is unpredictable.
+ * For example, if bit[3] of vmid_mask0 is 1, we must clear bits[31:24]
+ * of vmid comparator0 value (corresponding to byte 0) register.
+ */
+ mask = config->vmid_mask0;
+ for (i = 0; i < drvdata->numvmidc; i++) {
+ /* mask value of corresponding vmid comparator */
+ maskbyte = mask & ETMv4_EVENT_MASK;
+ /*
+ * each bit corresponds to a byte of respective vmid comparator
+ * value register
+ */
+ for (j = 0; j < 8; j++) {
+ if (maskbyte & 1)
+ config->vmid_val[i] &= ~(0xFF << (j * 8));
+ maskbyte >>= 1;
+ }
+ /* Select the next vmid comparator mask value */
+ if (i == 3)
+ /* vmid comparators[4-7] */
+ mask = config->vmid_mask1;
+ else
+ mask >>= 0x8;
+ }
+ spin_unlock(&drvdata->spinlock);
+ return size;
+}
+static DEVICE_ATTR_RW(vmid_masks);
+
+static ssize_t cpu_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ int val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
+
+ val = drvdata->cpu;
+ return scnprintf(buf, PAGE_SIZE, "%d\n", val);
+
+}
+static DEVICE_ATTR_RO(cpu);
+
+static struct attribute *coresight_etmv4_attrs[] = {
+ &dev_attr_nr_pe_cmp.attr,
+ &dev_attr_nr_addr_cmp.attr,
+ &dev_attr_nr_cntr.attr,
+ &dev_attr_nr_ext_inp.attr,
+ &dev_attr_numcidc.attr,
+ &dev_attr_numvmidc.attr,
+ &dev_attr_nrseqstate.attr,
+ &dev_attr_nr_resource.attr,
+ &dev_attr_nr_ss_cmp.attr,
+ &dev_attr_reset.attr,
+ &dev_attr_mode.attr,
+ &dev_attr_pe.attr,
+ &dev_attr_event.attr,
+ &dev_attr_event_instren.attr,
+ &dev_attr_event_ts.attr,
+ &dev_attr_syncfreq.attr,
+ &dev_attr_cyc_threshold.attr,
+ &dev_attr_bb_ctrl.attr,
+ &dev_attr_event_vinst.attr,
+ &dev_attr_s_exlevel_vinst.attr,
+ &dev_attr_ns_exlevel_vinst.attr,
+ &dev_attr_addr_idx.attr,
+ &dev_attr_addr_instdatatype.attr,
+ &dev_attr_addr_single.attr,
+ &dev_attr_addr_range.attr,
+ &dev_attr_addr_start.attr,
+ &dev_attr_addr_stop.attr,
+ &dev_attr_addr_ctxtype.attr,
+ &dev_attr_addr_context.attr,
+ &dev_attr_seq_idx.attr,
+ &dev_attr_seq_state.attr,
+ &dev_attr_seq_event.attr,
+ &dev_attr_seq_reset_event.attr,
+ &dev_attr_cntr_idx.attr,
+ &dev_attr_cntrldvr.attr,
+ &dev_attr_cntr_val.attr,
+ &dev_attr_cntr_ctrl.attr,
+ &dev_attr_res_idx.attr,
+ &dev_attr_res_ctrl.attr,
+ &dev_attr_ctxid_idx.attr,
+ &dev_attr_ctxid_pid.attr,
+ &dev_attr_ctxid_masks.attr,
+ &dev_attr_vmid_idx.attr,
+ &dev_attr_vmid_val.attr,
+ &dev_attr_vmid_masks.attr,
+ &dev_attr_cpu.attr,
+ NULL,
+};
+
+#define coresight_etm4x_simple_func(name, offset) \
+ coresight_simple_func(struct etmv4_drvdata, name, offset)
+
+coresight_etm4x_simple_func(trcoslsr, TRCOSLSR);
+coresight_etm4x_simple_func(trcpdcr, TRCPDCR);
+coresight_etm4x_simple_func(trcpdsr, TRCPDSR);
+coresight_etm4x_simple_func(trclsr, TRCLSR);
+coresight_etm4x_simple_func(trcconfig, TRCCONFIGR);
+coresight_etm4x_simple_func(trctraceid, TRCTRACEIDR);
+coresight_etm4x_simple_func(trcauthstatus, TRCAUTHSTATUS);
+coresight_etm4x_simple_func(trcdevid, TRCDEVID);
+coresight_etm4x_simple_func(trcdevtype, TRCDEVTYPE);
+coresight_etm4x_simple_func(trcpidr0, TRCPIDR0);
+coresight_etm4x_simple_func(trcpidr1, TRCPIDR1);
+coresight_etm4x_simple_func(trcpidr2, TRCPIDR2);
+coresight_etm4x_simple_func(trcpidr3, TRCPIDR3);
+
+static struct attribute *coresight_etmv4_mgmt_attrs[] = {
+ &dev_attr_trcoslsr.attr,
+ &dev_attr_trcpdcr.attr,
+ &dev_attr_trcpdsr.attr,
+ &dev_attr_trclsr.attr,
+ &dev_attr_trcconfig.attr,
+ &dev_attr_trctraceid.attr,
+ &dev_attr_trcauthstatus.attr,
+ &dev_attr_trcdevid.attr,
+ &dev_attr_trcdevtype.attr,
+ &dev_attr_trcpidr0.attr,
+ &dev_attr_trcpidr1.attr,
+ &dev_attr_trcpidr2.attr,
+ &dev_attr_trcpidr3.attr,
+ NULL,
+};
+
+coresight_etm4x_simple_func(trcidr0, TRCIDR0);
+coresight_etm4x_simple_func(trcidr1, TRCIDR1);
+coresight_etm4x_simple_func(trcidr2, TRCIDR2);
+coresight_etm4x_simple_func(trcidr3, TRCIDR3);
+coresight_etm4x_simple_func(trcidr4, TRCIDR4);
+coresight_etm4x_simple_func(trcidr5, TRCIDR5);
+/* trcidr[6,7] are reserved */
+coresight_etm4x_simple_func(trcidr8, TRCIDR8);
+coresight_etm4x_simple_func(trcidr9, TRCIDR9);
+coresight_etm4x_simple_func(trcidr10, TRCIDR10);
+coresight_etm4x_simple_func(trcidr11, TRCIDR11);
+coresight_etm4x_simple_func(trcidr12, TRCIDR12);
+coresight_etm4x_simple_func(trcidr13, TRCIDR13);
+
+static struct attribute *coresight_etmv4_trcidr_attrs[] = {
+ &dev_attr_trcidr0.attr,
+ &dev_attr_trcidr1.attr,
+ &dev_attr_trcidr2.attr,
+ &dev_attr_trcidr3.attr,
+ &dev_attr_trcidr4.attr,
+ &dev_attr_trcidr5.attr,
+ /* trcidr[6,7] are reserved */
+ &dev_attr_trcidr8.attr,
+ &dev_attr_trcidr9.attr,
+ &dev_attr_trcidr10.attr,
+ &dev_attr_trcidr11.attr,
+ &dev_attr_trcidr12.attr,
+ &dev_attr_trcidr13.attr,
+ NULL,
+};
+
+static const struct attribute_group coresight_etmv4_group = {
+ .attrs = coresight_etmv4_attrs,
+};
+
+static const struct attribute_group coresight_etmv4_mgmt_group = {
+ .attrs = coresight_etmv4_mgmt_attrs,
+ .name = "mgmt",
+};
+
+static const struct attribute_group coresight_etmv4_trcidr_group = {
+ .attrs = coresight_etmv4_trcidr_attrs,
+ .name = "trcidr",
+};
+
+const struct attribute_group *coresight_etmv4_groups[] = {
+ &coresight_etmv4_group,
+ &coresight_etmv4_mgmt_group,
+ &coresight_etmv4_trcidr_group,
+ NULL,
+};
diff --git a/drivers/hwtracing/coresight/coresight-etm4x.c b/drivers/hwtracing/coresight/coresight-etm4x.c
index 1c59bd3..462f0dc 100644
--- a/drivers/hwtracing/coresight/coresight-etm4x.c
+++ b/drivers/hwtracing/coresight/coresight-etm4x.c
@@ -26,15 +26,19 @@
#include <linux/clk.h>
#include <linux/cpu.h>
#include <linux/coresight.h>
+#include <linux/coresight-pmu.h>
#include <linux/pm_wakeup.h>
#include <linux/amba/bus.h>
#include <linux/seq_file.h>
#include <linux/uaccess.h>
+#include <linux/perf_event.h>
#include <linux/pm_runtime.h>
#include <linux/perf_event.h>
#include <asm/sections.h>
+#include <asm/local.h>
#include "coresight-etm4x.h"
+#include "coresight-etm-perf.h"
static int boot_enable;
module_param_named(boot_enable, boot_enable, int, S_IRUGO);
@@ -42,13 +46,13 @@
/* The number of ETMv4 currently registered */
static int etm4_count;
static struct etmv4_drvdata *etmdrvdata[NR_CPUS];
+static void etm4_set_default(struct etmv4_config *config);
-static void etm4_os_unlock(void *info)
+static void etm4_os_unlock(struct etmv4_drvdata *drvdata)
{
- struct etmv4_drvdata *drvdata = (struct etmv4_drvdata *)info;
-
/* Writing any value to ETMOSLAR unlocks the trace registers */
writel_relaxed(0x0, drvdata->base + TRCOSLAR);
+ drvdata->os_unlock = true;
isb();
}
@@ -76,7 +80,7 @@
unsigned long flags;
int trace_id = -1;
- if (!drvdata->enable)
+ if (!local_read(&drvdata->mode))
return drvdata->trcid;
spin_lock_irqsave(&drvdata->spinlock, flags);
@@ -95,6 +99,7 @@
{
int i;
struct etmv4_drvdata *drvdata = info;
+ struct etmv4_config *config = &drvdata->config;
CS_UNLOCK(drvdata->base);
@@ -109,69 +114,69 @@
"timeout observed when probing at offset %#x\n",
TRCSTATR);
- writel_relaxed(drvdata->pe_sel, drvdata->base + TRCPROCSELR);
- writel_relaxed(drvdata->cfg, drvdata->base + TRCCONFIGR);
+ writel_relaxed(config->pe_sel, drvdata->base + TRCPROCSELR);
+ writel_relaxed(config->cfg, drvdata->base + TRCCONFIGR);
/* nothing specific implemented */
writel_relaxed(0x0, drvdata->base + TRCAUXCTLR);
- writel_relaxed(drvdata->eventctrl0, drvdata->base + TRCEVENTCTL0R);
- writel_relaxed(drvdata->eventctrl1, drvdata->base + TRCEVENTCTL1R);
- writel_relaxed(drvdata->stall_ctrl, drvdata->base + TRCSTALLCTLR);
- writel_relaxed(drvdata->ts_ctrl, drvdata->base + TRCTSCTLR);
- writel_relaxed(drvdata->syncfreq, drvdata->base + TRCSYNCPR);
- writel_relaxed(drvdata->ccctlr, drvdata->base + TRCCCCTLR);
- writel_relaxed(drvdata->bb_ctrl, drvdata->base + TRCBBCTLR);
+ writel_relaxed(config->eventctrl0, drvdata->base + TRCEVENTCTL0R);
+ writel_relaxed(config->eventctrl1, drvdata->base + TRCEVENTCTL1R);
+ writel_relaxed(config->stall_ctrl, drvdata->base + TRCSTALLCTLR);
+ writel_relaxed(config->ts_ctrl, drvdata->base + TRCTSCTLR);
+ writel_relaxed(config->syncfreq, drvdata->base + TRCSYNCPR);
+ writel_relaxed(config->ccctlr, drvdata->base + TRCCCCTLR);
+ writel_relaxed(config->bb_ctrl, drvdata->base + TRCBBCTLR);
writel_relaxed(drvdata->trcid, drvdata->base + TRCTRACEIDR);
- writel_relaxed(drvdata->vinst_ctrl, drvdata->base + TRCVICTLR);
- writel_relaxed(drvdata->viiectlr, drvdata->base + TRCVIIECTLR);
- writel_relaxed(drvdata->vissctlr,
+ writel_relaxed(config->vinst_ctrl, drvdata->base + TRCVICTLR);
+ writel_relaxed(config->viiectlr, drvdata->base + TRCVIIECTLR);
+ writel_relaxed(config->vissctlr,
drvdata->base + TRCVISSCTLR);
- writel_relaxed(drvdata->vipcssctlr,
+ writel_relaxed(config->vipcssctlr,
drvdata->base + TRCVIPCSSCTLR);
for (i = 0; i < drvdata->nrseqstate - 1; i++)
- writel_relaxed(drvdata->seq_ctrl[i],
+ writel_relaxed(config->seq_ctrl[i],
drvdata->base + TRCSEQEVRn(i));
- writel_relaxed(drvdata->seq_rst, drvdata->base + TRCSEQRSTEVR);
- writel_relaxed(drvdata->seq_state, drvdata->base + TRCSEQSTR);
- writel_relaxed(drvdata->ext_inp, drvdata->base + TRCEXTINSELR);
+ writel_relaxed(config->seq_rst, drvdata->base + TRCSEQRSTEVR);
+ writel_relaxed(config->seq_state, drvdata->base + TRCSEQSTR);
+ writel_relaxed(config->ext_inp, drvdata->base + TRCEXTINSELR);
for (i = 0; i < drvdata->nr_cntr; i++) {
- writel_relaxed(drvdata->cntrldvr[i],
+ writel_relaxed(config->cntrldvr[i],
drvdata->base + TRCCNTRLDVRn(i));
- writel_relaxed(drvdata->cntr_ctrl[i],
+ writel_relaxed(config->cntr_ctrl[i],
drvdata->base + TRCCNTCTLRn(i));
- writel_relaxed(drvdata->cntr_val[i],
+ writel_relaxed(config->cntr_val[i],
drvdata->base + TRCCNTVRn(i));
}
/* Resource selector pair 0 is always implemented and reserved */
- for (i = 2; i < drvdata->nr_resource * 2; i++)
- writel_relaxed(drvdata->res_ctrl[i],
+ for (i = 0; i < drvdata->nr_resource * 2; i++)
+ writel_relaxed(config->res_ctrl[i],
drvdata->base + TRCRSCTLRn(i));
for (i = 0; i < drvdata->nr_ss_cmp; i++) {
- writel_relaxed(drvdata->ss_ctrl[i],
+ writel_relaxed(config->ss_ctrl[i],
drvdata->base + TRCSSCCRn(i));
- writel_relaxed(drvdata->ss_status[i],
+ writel_relaxed(config->ss_status[i],
drvdata->base + TRCSSCSRn(i));
- writel_relaxed(drvdata->ss_pe_cmp[i],
+ writel_relaxed(config->ss_pe_cmp[i],
drvdata->base + TRCSSPCICRn(i));
}
for (i = 0; i < drvdata->nr_addr_cmp; i++) {
- writeq_relaxed(drvdata->addr_val[i],
+ writeq_relaxed(config->addr_val[i],
drvdata->base + TRCACVRn(i));
- writeq_relaxed(drvdata->addr_acc[i],
+ writeq_relaxed(config->addr_acc[i],
drvdata->base + TRCACATRn(i));
}
for (i = 0; i < drvdata->numcidc; i++)
- writeq_relaxed(drvdata->ctxid_pid[i],
+ writeq_relaxed(config->ctxid_pid[i],
drvdata->base + TRCCIDCVRn(i));
- writel_relaxed(drvdata->ctxid_mask0, drvdata->base + TRCCIDCCTLR0);
- writel_relaxed(drvdata->ctxid_mask1, drvdata->base + TRCCIDCCTLR1);
+ writel_relaxed(config->ctxid_mask0, drvdata->base + TRCCIDCCTLR0);
+ writel_relaxed(config->ctxid_mask1, drvdata->base + TRCCIDCCTLR1);
for (i = 0; i < drvdata->numvmidc; i++)
- writeq_relaxed(drvdata->vmid_val[i],
+ writeq_relaxed(config->vmid_val[i],
drvdata->base + TRCVMIDCVRn(i));
- writel_relaxed(drvdata->vmid_mask0, drvdata->base + TRCVMIDCCTLR0);
- writel_relaxed(drvdata->vmid_mask1, drvdata->base + TRCVMIDCCTLR1);
+ writel_relaxed(config->vmid_mask0, drvdata->base + TRCVMIDCCTLR0);
+ writel_relaxed(config->vmid_mask1, drvdata->base + TRCVMIDCCTLR1);
/* Enable the trace unit */
writel_relaxed(1, drvdata->base + TRCPRGCTLR);
@@ -187,8 +192,59 @@
dev_dbg(drvdata->dev, "cpu: %d enable smp call done\n", drvdata->cpu);
}
-static int etm4_enable(struct coresight_device *csdev,
- struct perf_event_attr *attr, u32 mode)
+static int etm4_parse_event_config(struct etmv4_drvdata *drvdata,
+ struct perf_event_attr *attr)
+{
+ struct etmv4_config *config = &drvdata->config;
+
+ if (!attr)
+ return -EINVAL;
+
+ /* Clear configuration from previous run */
+ memset(config, 0, sizeof(struct etmv4_config));
+
+ if (attr->exclude_kernel)
+ config->mode = ETM_MODE_EXCL_KERN;
+
+ if (attr->exclude_user)
+ config->mode = ETM_MODE_EXCL_USER;
+
+ /* Always start from the default config */
+ etm4_set_default(config);
+
+ /*
+ * By default the tracers are configured to trace the whole address
+ * range. Narrow the field only if requested by user space.
+ */
+ if (config->mode)
+ etm4_config_trace_mode(config);
+
+ /* Go from generic option to ETMv4 specifics */
+ if (attr->config & BIT(ETM_OPT_CYCACC))
+ config->cfg |= ETMv4_MODE_CYCACC;
+ if (attr->config & BIT(ETM_OPT_TS))
+ config->cfg |= ETMv4_MODE_TIMESTAMP;
+
+ return 0;
+}
+
+static int etm4_enable_perf(struct coresight_device *csdev,
+ struct perf_event_attr *attr)
+{
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
+
+ if (WARN_ON_ONCE(drvdata->cpu != smp_processor_id()))
+ return -EINVAL;
+
+ /* Configure the tracer based on the session's specifics */
+ etm4_parse_event_config(drvdata, attr);
+ /* And enable it */
+ etm4_enable_hw(drvdata);
+
+ return 0;
+}
+
+static int etm4_enable_sysfs(struct coresight_device *csdev)
{
struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
int ret;
@@ -203,18 +259,49 @@
etm4_enable_hw, drvdata, 1);
if (ret)
goto err;
- drvdata->enable = true;
- drvdata->sticky_enable = true;
+ drvdata->sticky_enable = true;
spin_unlock(&drvdata->spinlock);
dev_info(drvdata->dev, "ETM tracing enabled\n");
return 0;
+
err:
spin_unlock(&drvdata->spinlock);
return ret;
}
+static int etm4_enable(struct coresight_device *csdev,
+ struct perf_event_attr *attr, u32 mode)
+{
+ int ret;
+ u32 val;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
+
+ val = local_cmpxchg(&drvdata->mode, CS_MODE_DISABLED, mode);
+
+ /* Someone is already using the tracer */
+ if (val)
+ return -EBUSY;
+
+ switch (mode) {
+ case CS_MODE_SYSFS:
+ ret = etm4_enable_sysfs(csdev);
+ break;
+ case CS_MODE_PERF:
+ ret = etm4_enable_perf(csdev, attr);
+ break;
+ default:
+ ret = -EINVAL;
+ }
+
+ /* The tracer didn't start */
+ if (ret)
+ local_set(&drvdata->mode, CS_MODE_DISABLED);
+
+ return ret;
+}
+
static void etm4_disable_hw(void *info)
{
u32 control;
@@ -237,7 +324,18 @@
dev_dbg(drvdata->dev, "cpu: %d disable smp call done\n", drvdata->cpu);
}
-static void etm4_disable(struct coresight_device *csdev)
+static int etm4_disable_perf(struct coresight_device *csdev)
+{
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
+
+ if (WARN_ON_ONCE(drvdata->cpu != smp_processor_id()))
+ return -EINVAL;
+
+ etm4_disable_hw(drvdata);
+ return 0;
+}
+
+static void etm4_disable_sysfs(struct coresight_device *csdev)
{
struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
@@ -255,7 +353,6 @@
* ensures that register writes occur when cpu is powered.
*/
smp_call_function_single(drvdata->cpu, etm4_disable_hw, drvdata, 1);
- drvdata->enable = false;
spin_unlock(&drvdata->spinlock);
put_online_cpus();
@@ -263,6 +360,33 @@
dev_info(drvdata->dev, "ETM tracing disabled\n");
}
+static void etm4_disable(struct coresight_device *csdev)
+{
+ u32 mode;
+ struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
+
+ /*
+ * For as long as the tracer isn't disabled another entity can't
+ * change its status. As such we can read the status here without
+ * fearing it will change under us.
+ */
+ mode = local_read(&drvdata->mode);
+
+ switch (mode) {
+ case CS_MODE_DISABLED:
+ break;
+ case CS_MODE_SYSFS:
+ etm4_disable_sysfs(csdev);
+ break;
+ case CS_MODE_PERF:
+ etm4_disable_perf(csdev);
+ break;
+ }
+
+ if (mode)
+ local_set(&drvdata->mode, CS_MODE_DISABLED);
+}
+
static const struct coresight_ops_source etm4_source_ops = {
.cpu_id = etm4_cpu_id,
.trace_id = etm4_trace_id,
@@ -274,2035 +398,6 @@
.source_ops = &etm4_source_ops,
};
-static int etm4_set_mode_exclude(struct etmv4_drvdata *drvdata, bool exclude)
-{
- u8 idx = drvdata->addr_idx;
-
- /*
- * TRCACATRn.TYPE bit[1:0]: type of comparison
- * the trace unit performs
- */
- if (BMVAL(drvdata->addr_acc[idx], 0, 1) == ETM_INSTR_ADDR) {
- if (idx % 2 != 0)
- return -EINVAL;
-
- /*
- * We are performing instruction address comparison. Set the
- * relevant bit of ViewInst Include/Exclude Control register
- * for corresponding address comparator pair.
- */
- if (drvdata->addr_type[idx] != ETM_ADDR_TYPE_RANGE ||
- drvdata->addr_type[idx + 1] != ETM_ADDR_TYPE_RANGE)
- return -EINVAL;
-
- if (exclude == true) {
- /*
- * Set exclude bit and unset the include bit
- * corresponding to comparator pair
- */
- drvdata->viiectlr |= BIT(idx / 2 + 16);
- drvdata->viiectlr &= ~BIT(idx / 2);
- } else {
- /*
- * Set include bit and unset exclude bit
- * corresponding to comparator pair
- */
- drvdata->viiectlr |= BIT(idx / 2);
- drvdata->viiectlr &= ~BIT(idx / 2 + 16);
- }
- }
- return 0;
-}
-
-static ssize_t nr_pe_cmp_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->nr_pe_cmp;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-static DEVICE_ATTR_RO(nr_pe_cmp);
-
-static ssize_t nr_addr_cmp_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->nr_addr_cmp;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-static DEVICE_ATTR_RO(nr_addr_cmp);
-
-static ssize_t nr_cntr_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->nr_cntr;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-static DEVICE_ATTR_RO(nr_cntr);
-
-static ssize_t nr_ext_inp_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->nr_ext_inp;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-static DEVICE_ATTR_RO(nr_ext_inp);
-
-static ssize_t numcidc_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->numcidc;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-static DEVICE_ATTR_RO(numcidc);
-
-static ssize_t numvmidc_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->numvmidc;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-static DEVICE_ATTR_RO(numvmidc);
-
-static ssize_t nrseqstate_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->nrseqstate;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-static DEVICE_ATTR_RO(nrseqstate);
-
-static ssize_t nr_resource_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->nr_resource;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-static DEVICE_ATTR_RO(nr_resource);
-
-static ssize_t nr_ss_cmp_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->nr_ss_cmp;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-static DEVICE_ATTR_RO(nr_ss_cmp);
-
-static ssize_t reset_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- int i;
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
- if (val)
- drvdata->mode = 0x0;
-
- /* Disable data tracing: do not trace load and store data transfers */
- drvdata->mode &= ~(ETM_MODE_LOAD | ETM_MODE_STORE);
- drvdata->cfg &= ~(BIT(1) | BIT(2));
-
- /* Disable data value and data address tracing */
- drvdata->mode &= ~(ETM_MODE_DATA_TRACE_ADDR |
- ETM_MODE_DATA_TRACE_VAL);
- drvdata->cfg &= ~(BIT(16) | BIT(17));
-
- /* Disable all events tracing */
- drvdata->eventctrl0 = 0x0;
- drvdata->eventctrl1 = 0x0;
-
- /* Disable timestamp event */
- drvdata->ts_ctrl = 0x0;
-
- /* Disable stalling */
- drvdata->stall_ctrl = 0x0;
-
- /* Reset trace synchronization period to 2^8 = 256 bytes*/
- if (drvdata->syncpr == false)
- drvdata->syncfreq = 0x8;
-
- /*
- * Enable ViewInst to trace everything with start-stop logic in
- * started state. ARM recommends start-stop logic is set before
- * each trace run.
- */
- drvdata->vinst_ctrl |= BIT(0);
- if (drvdata->nr_addr_cmp == true) {
- drvdata->mode |= ETM_MODE_VIEWINST_STARTSTOP;
- /* SSSTATUS, bit[9] */
- drvdata->vinst_ctrl |= BIT(9);
- }
-
- /* No address range filtering for ViewInst */
- drvdata->viiectlr = 0x0;
-
- /* No start-stop filtering for ViewInst */
- drvdata->vissctlr = 0x0;
-
- /* Disable seq events */
- for (i = 0; i < drvdata->nrseqstate-1; i++)
- drvdata->seq_ctrl[i] = 0x0;
- drvdata->seq_rst = 0x0;
- drvdata->seq_state = 0x0;
-
- /* Disable external input events */
- drvdata->ext_inp = 0x0;
-
- drvdata->cntr_idx = 0x0;
- for (i = 0; i < drvdata->nr_cntr; i++) {
- drvdata->cntrldvr[i] = 0x0;
- drvdata->cntr_ctrl[i] = 0x0;
- drvdata->cntr_val[i] = 0x0;
- }
-
- /* Resource selector pair 0 is always implemented and reserved */
- drvdata->res_idx = 0x2;
- for (i = 2; i < drvdata->nr_resource * 2; i++)
- drvdata->res_ctrl[i] = 0x0;
-
- for (i = 0; i < drvdata->nr_ss_cmp; i++) {
- drvdata->ss_ctrl[i] = 0x0;
- drvdata->ss_pe_cmp[i] = 0x0;
- }
-
- drvdata->addr_idx = 0x0;
- for (i = 0; i < drvdata->nr_addr_cmp * 2; i++) {
- drvdata->addr_val[i] = 0x0;
- drvdata->addr_acc[i] = 0x0;
- drvdata->addr_type[i] = ETM_ADDR_TYPE_NONE;
- }
-
- drvdata->ctxid_idx = 0x0;
- for (i = 0; i < drvdata->numcidc; i++) {
- drvdata->ctxid_pid[i] = 0x0;
- drvdata->ctxid_vpid[i] = 0x0;
- }
-
- drvdata->ctxid_mask0 = 0x0;
- drvdata->ctxid_mask1 = 0x0;
-
- drvdata->vmid_idx = 0x0;
- for (i = 0; i < drvdata->numvmidc; i++)
- drvdata->vmid_val[i] = 0x0;
- drvdata->vmid_mask0 = 0x0;
- drvdata->vmid_mask1 = 0x0;
-
- drvdata->trcid = drvdata->cpu + 1;
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_WO(reset);
-
-static ssize_t mode_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->mode;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t mode_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- unsigned long val, mode;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
- drvdata->mode = val & ETMv4_MODE_ALL;
-
- if (drvdata->mode & ETM_MODE_EXCLUDE)
- etm4_set_mode_exclude(drvdata, true);
- else
- etm4_set_mode_exclude(drvdata, false);
-
- if (drvdata->instrp0 == true) {
- /* start by clearing instruction P0 field */
- drvdata->cfg &= ~(BIT(1) | BIT(2));
- if (drvdata->mode & ETM_MODE_LOAD)
- /* 0b01 Trace load instructions as P0 instructions */
- drvdata->cfg |= BIT(1);
- if (drvdata->mode & ETM_MODE_STORE)
- /* 0b10 Trace store instructions as P0 instructions */
- drvdata->cfg |= BIT(2);
- if (drvdata->mode & ETM_MODE_LOAD_STORE)
- /*
- * 0b11 Trace load and store instructions
- * as P0 instructions
- */
- drvdata->cfg |= BIT(1) | BIT(2);
- }
-
- /* bit[3], Branch broadcast mode */
- if ((drvdata->mode & ETM_MODE_BB) && (drvdata->trcbb == true))
- drvdata->cfg |= BIT(3);
- else
- drvdata->cfg &= ~BIT(3);
-
- /* bit[4], Cycle counting instruction trace bit */
- if ((drvdata->mode & ETMv4_MODE_CYCACC) &&
- (drvdata->trccci == true))
- drvdata->cfg |= BIT(4);
- else
- drvdata->cfg &= ~BIT(4);
-
- /* bit[6], Context ID tracing bit */
- if ((drvdata->mode & ETMv4_MODE_CTXID) && (drvdata->ctxid_size))
- drvdata->cfg |= BIT(6);
- else
- drvdata->cfg &= ~BIT(6);
-
- if ((drvdata->mode & ETM_MODE_VMID) && (drvdata->vmid_size))
- drvdata->cfg |= BIT(7);
- else
- drvdata->cfg &= ~BIT(7);
-
- /* bits[10:8], Conditional instruction tracing bit */
- mode = ETM_MODE_COND(drvdata->mode);
- if (drvdata->trccond == true) {
- drvdata->cfg &= ~(BIT(8) | BIT(9) | BIT(10));
- drvdata->cfg |= mode << 8;
- }
-
- /* bit[11], Global timestamp tracing bit */
- if ((drvdata->mode & ETMv4_MODE_TIMESTAMP) && (drvdata->ts_size))
- drvdata->cfg |= BIT(11);
- else
- drvdata->cfg &= ~BIT(11);
-
- /* bit[12], Return stack enable bit */
- if ((drvdata->mode & ETM_MODE_RETURNSTACK) &&
- (drvdata->retstack == true))
- drvdata->cfg |= BIT(12);
- else
- drvdata->cfg &= ~BIT(12);
-
- /* bits[14:13], Q element enable field */
- mode = ETM_MODE_QELEM(drvdata->mode);
- /* start by clearing QE bits */
- drvdata->cfg &= ~(BIT(13) | BIT(14));
- /* if supported, Q elements with instruction counts are enabled */
- if ((mode & BIT(0)) && (drvdata->q_support & BIT(0)))
- drvdata->cfg |= BIT(13);
- /*
- * if supported, Q elements with and without instruction
- * counts are enabled
- */
- if ((mode & BIT(1)) && (drvdata->q_support & BIT(1)))
- drvdata->cfg |= BIT(14);
-
- /* bit[11], AMBA Trace Bus (ATB) trigger enable bit */
- if ((drvdata->mode & ETM_MODE_ATB_TRIGGER) &&
- (drvdata->atbtrig == true))
- drvdata->eventctrl1 |= BIT(11);
- else
- drvdata->eventctrl1 &= ~BIT(11);
-
- /* bit[12], Low-power state behavior override bit */
- if ((drvdata->mode & ETM_MODE_LPOVERRIDE) &&
- (drvdata->lpoverride == true))
- drvdata->eventctrl1 |= BIT(12);
- else
- drvdata->eventctrl1 &= ~BIT(12);
-
- /* bit[8], Instruction stall bit */
- if (drvdata->mode & ETM_MODE_ISTALL_EN)
- drvdata->stall_ctrl |= BIT(8);
- else
- drvdata->stall_ctrl &= ~BIT(8);
-
- /* bit[10], Prioritize instruction trace bit */
- if (drvdata->mode & ETM_MODE_INSTPRIO)
- drvdata->stall_ctrl |= BIT(10);
- else
- drvdata->stall_ctrl &= ~BIT(10);
-
- /* bit[13], Trace overflow prevention bit */
- if ((drvdata->mode & ETM_MODE_NOOVERFLOW) &&
- (drvdata->nooverflow == true))
- drvdata->stall_ctrl |= BIT(13);
- else
- drvdata->stall_ctrl &= ~BIT(13);
-
- /* bit[9] Start/stop logic control bit */
- if (drvdata->mode & ETM_MODE_VIEWINST_STARTSTOP)
- drvdata->vinst_ctrl |= BIT(9);
- else
- drvdata->vinst_ctrl &= ~BIT(9);
-
- /* bit[10], Whether a trace unit must trace a Reset exception */
- if (drvdata->mode & ETM_MODE_TRACE_RESET)
- drvdata->vinst_ctrl |= BIT(10);
- else
- drvdata->vinst_ctrl &= ~BIT(10);
-
- /* bit[11], Whether a trace unit must trace a system error exception */
- if ((drvdata->mode & ETM_MODE_TRACE_ERR) &&
- (drvdata->trc_error == true))
- drvdata->vinst_ctrl |= BIT(11);
- else
- drvdata->vinst_ctrl &= ~BIT(11);
-
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(mode);
-
-static ssize_t pe_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->pe_sel;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t pe_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
- if (val > drvdata->nr_pe) {
- spin_unlock(&drvdata->spinlock);
- return -EINVAL;
- }
-
- drvdata->pe_sel = val;
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(pe);
-
-static ssize_t event_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->eventctrl0;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t event_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
- switch (drvdata->nr_event) {
- case 0x0:
- /* EVENT0, bits[7:0] */
- drvdata->eventctrl0 = val & 0xFF;
- break;
- case 0x1:
- /* EVENT1, bits[15:8] */
- drvdata->eventctrl0 = val & 0xFFFF;
- break;
- case 0x2:
- /* EVENT2, bits[23:16] */
- drvdata->eventctrl0 = val & 0xFFFFFF;
- break;
- case 0x3:
- /* EVENT3, bits[31:24] */
- drvdata->eventctrl0 = val;
- break;
- default:
- break;
- }
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(event);
-
-static ssize_t event_instren_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = BMVAL(drvdata->eventctrl1, 0, 3);
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t event_instren_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
- /* start by clearing all instruction event enable bits */
- drvdata->eventctrl1 &= ~(BIT(0) | BIT(1) | BIT(2) | BIT(3));
- switch (drvdata->nr_event) {
- case 0x0:
- /* generate Event element for event 1 */
- drvdata->eventctrl1 |= val & BIT(1);
- break;
- case 0x1:
- /* generate Event element for event 1 and 2 */
- drvdata->eventctrl1 |= val & (BIT(0) | BIT(1));
- break;
- case 0x2:
- /* generate Event element for event 1, 2 and 3 */
- drvdata->eventctrl1 |= val & (BIT(0) | BIT(1) | BIT(2));
- break;
- case 0x3:
- /* generate Event element for all 4 events */
- drvdata->eventctrl1 |= val & 0xF;
- break;
- default:
- break;
- }
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(event_instren);
-
-static ssize_t event_ts_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->ts_ctrl;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t event_ts_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
- if (!drvdata->ts_size)
- return -EINVAL;
-
- drvdata->ts_ctrl = val & ETMv4_EVENT_MASK;
- return size;
-}
-static DEVICE_ATTR_RW(event_ts);
-
-static ssize_t syncfreq_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->syncfreq;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t syncfreq_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
- if (drvdata->syncpr == true)
- return -EINVAL;
-
- drvdata->syncfreq = val & ETMv4_SYNC_MASK;
- return size;
-}
-static DEVICE_ATTR_RW(syncfreq);
-
-static ssize_t cyc_threshold_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->ccctlr;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t cyc_threshold_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
- if (val < drvdata->ccitmin)
- return -EINVAL;
-
- drvdata->ccctlr = val & ETM_CYC_THRESHOLD_MASK;
- return size;
-}
-static DEVICE_ATTR_RW(cyc_threshold);
-
-static ssize_t bb_ctrl_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->bb_ctrl;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t bb_ctrl_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
- if (drvdata->trcbb == false)
- return -EINVAL;
- if (!drvdata->nr_addr_cmp)
- return -EINVAL;
- /*
- * Bit[7:0] selects which address range comparator is used for
- * branch broadcast control.
- */
- if (BMVAL(val, 0, 7) > drvdata->nr_addr_cmp)
- return -EINVAL;
-
- drvdata->bb_ctrl = val;
- return size;
-}
-static DEVICE_ATTR_RW(bb_ctrl);
-
-static ssize_t event_vinst_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->vinst_ctrl & ETMv4_EVENT_MASK;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t event_vinst_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
- val &= ETMv4_EVENT_MASK;
- drvdata->vinst_ctrl &= ~ETMv4_EVENT_MASK;
- drvdata->vinst_ctrl |= val;
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(event_vinst);
-
-static ssize_t s_exlevel_vinst_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = BMVAL(drvdata->vinst_ctrl, 16, 19);
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t s_exlevel_vinst_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
- /* clear all EXLEVEL_S bits (bit[18] is never implemented) */
- drvdata->vinst_ctrl &= ~(BIT(16) | BIT(17) | BIT(19));
- /* enable instruction tracing for corresponding exception level */
- val &= drvdata->s_ex_level;
- drvdata->vinst_ctrl |= (val << 16);
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(s_exlevel_vinst);
-
-static ssize_t ns_exlevel_vinst_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- /* EXLEVEL_NS, bits[23:20] */
- val = BMVAL(drvdata->vinst_ctrl, 20, 23);
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t ns_exlevel_vinst_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
- /* clear EXLEVEL_NS bits (bit[23] is never implemented */
- drvdata->vinst_ctrl &= ~(BIT(20) | BIT(21) | BIT(22));
- /* enable instruction tracing for corresponding exception level */
- val &= drvdata->ns_ex_level;
- drvdata->vinst_ctrl |= (val << 20);
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(ns_exlevel_vinst);
-
-static ssize_t addr_idx_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->addr_idx;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t addr_idx_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
- if (val >= drvdata->nr_addr_cmp * 2)
- return -EINVAL;
-
- /*
- * Use spinlock to ensure index doesn't change while it gets
- * dereferenced multiple times within a spinlock block elsewhere.
- */
- spin_lock(&drvdata->spinlock);
- drvdata->addr_idx = val;
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(addr_idx);
-
-static ssize_t addr_instdatatype_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- ssize_t len;
- u8 val, idx;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->addr_idx;
- val = BMVAL(drvdata->addr_acc[idx], 0, 1);
- len = scnprintf(buf, PAGE_SIZE, "%s\n",
- val == ETM_INSTR_ADDR ? "instr" :
- (val == ETM_DATA_LOAD_ADDR ? "data_load" :
- (val == ETM_DATA_STORE_ADDR ? "data_store" :
- "data_load_store")));
- spin_unlock(&drvdata->spinlock);
- return len;
-}
-
-static ssize_t addr_instdatatype_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- u8 idx;
- char str[20] = "";
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (strlen(buf) >= 20)
- return -EINVAL;
- if (sscanf(buf, "%s", str) != 1)
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->addr_idx;
- if (!strcmp(str, "instr"))
- /* TYPE, bits[1:0] */
- drvdata->addr_acc[idx] &= ~(BIT(0) | BIT(1));
-
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(addr_instdatatype);
-
-static ssize_t addr_single_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- u8 idx;
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- idx = drvdata->addr_idx;
- spin_lock(&drvdata->spinlock);
- if (!(drvdata->addr_type[idx] == ETM_ADDR_TYPE_NONE ||
- drvdata->addr_type[idx] == ETM_ADDR_TYPE_SINGLE)) {
- spin_unlock(&drvdata->spinlock);
- return -EPERM;
- }
- val = (unsigned long)drvdata->addr_val[idx];
- spin_unlock(&drvdata->spinlock);
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t addr_single_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- u8 idx;
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->addr_idx;
- if (!(drvdata->addr_type[idx] == ETM_ADDR_TYPE_NONE ||
- drvdata->addr_type[idx] == ETM_ADDR_TYPE_SINGLE)) {
- spin_unlock(&drvdata->spinlock);
- return -EPERM;
- }
-
- drvdata->addr_val[idx] = (u64)val;
- drvdata->addr_type[idx] = ETM_ADDR_TYPE_SINGLE;
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(addr_single);
-
-static ssize_t addr_range_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- u8 idx;
- unsigned long val1, val2;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->addr_idx;
- if (idx % 2 != 0) {
- spin_unlock(&drvdata->spinlock);
- return -EPERM;
- }
- if (!((drvdata->addr_type[idx] == ETM_ADDR_TYPE_NONE &&
- drvdata->addr_type[idx + 1] == ETM_ADDR_TYPE_NONE) ||
- (drvdata->addr_type[idx] == ETM_ADDR_TYPE_RANGE &&
- drvdata->addr_type[idx + 1] == ETM_ADDR_TYPE_RANGE))) {
- spin_unlock(&drvdata->spinlock);
- return -EPERM;
- }
-
- val1 = (unsigned long)drvdata->addr_val[idx];
- val2 = (unsigned long)drvdata->addr_val[idx + 1];
- spin_unlock(&drvdata->spinlock);
- return scnprintf(buf, PAGE_SIZE, "%#lx %#lx\n", val1, val2);
-}
-
-static ssize_t addr_range_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- u8 idx;
- unsigned long val1, val2;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (sscanf(buf, "%lx %lx", &val1, &val2) != 2)
- return -EINVAL;
- /* lower address comparator cannot have a higher address value */
- if (val1 > val2)
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->addr_idx;
- if (idx % 2 != 0) {
- spin_unlock(&drvdata->spinlock);
- return -EPERM;
- }
-
- if (!((drvdata->addr_type[idx] == ETM_ADDR_TYPE_NONE &&
- drvdata->addr_type[idx + 1] == ETM_ADDR_TYPE_NONE) ||
- (drvdata->addr_type[idx] == ETM_ADDR_TYPE_RANGE &&
- drvdata->addr_type[idx + 1] == ETM_ADDR_TYPE_RANGE))) {
- spin_unlock(&drvdata->spinlock);
- return -EPERM;
- }
-
- drvdata->addr_val[idx] = (u64)val1;
- drvdata->addr_type[idx] = ETM_ADDR_TYPE_RANGE;
- drvdata->addr_val[idx + 1] = (u64)val2;
- drvdata->addr_type[idx + 1] = ETM_ADDR_TYPE_RANGE;
- /*
- * Program include or exclude control bits for vinst or vdata
- * whenever we change addr comparators to ETM_ADDR_TYPE_RANGE
- */
- if (drvdata->mode & ETM_MODE_EXCLUDE)
- etm4_set_mode_exclude(drvdata, true);
- else
- etm4_set_mode_exclude(drvdata, false);
-
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(addr_range);
-
-static ssize_t addr_start_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- u8 idx;
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->addr_idx;
-
- if (!(drvdata->addr_type[idx] == ETM_ADDR_TYPE_NONE ||
- drvdata->addr_type[idx] == ETM_ADDR_TYPE_START)) {
- spin_unlock(&drvdata->spinlock);
- return -EPERM;
- }
-
- val = (unsigned long)drvdata->addr_val[idx];
- spin_unlock(&drvdata->spinlock);
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t addr_start_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- u8 idx;
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->addr_idx;
- if (!drvdata->nr_addr_cmp) {
- spin_unlock(&drvdata->spinlock);
- return -EINVAL;
- }
- if (!(drvdata->addr_type[idx] == ETM_ADDR_TYPE_NONE ||
- drvdata->addr_type[idx] == ETM_ADDR_TYPE_START)) {
- spin_unlock(&drvdata->spinlock);
- return -EPERM;
- }
-
- drvdata->addr_val[idx] = (u64)val;
- drvdata->addr_type[idx] = ETM_ADDR_TYPE_START;
- drvdata->vissctlr |= BIT(idx);
- /* SSSTATUS, bit[9] - turn on start/stop logic */
- drvdata->vinst_ctrl |= BIT(9);
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(addr_start);
-
-static ssize_t addr_stop_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- u8 idx;
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->addr_idx;
-
- if (!(drvdata->addr_type[idx] == ETM_ADDR_TYPE_NONE ||
- drvdata->addr_type[idx] == ETM_ADDR_TYPE_STOP)) {
- spin_unlock(&drvdata->spinlock);
- return -EPERM;
- }
-
- val = (unsigned long)drvdata->addr_val[idx];
- spin_unlock(&drvdata->spinlock);
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t addr_stop_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- u8 idx;
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->addr_idx;
- if (!drvdata->nr_addr_cmp) {
- spin_unlock(&drvdata->spinlock);
- return -EINVAL;
- }
- if (!(drvdata->addr_type[idx] == ETM_ADDR_TYPE_NONE ||
- drvdata->addr_type[idx] == ETM_ADDR_TYPE_STOP)) {
- spin_unlock(&drvdata->spinlock);
- return -EPERM;
- }
-
- drvdata->addr_val[idx] = (u64)val;
- drvdata->addr_type[idx] = ETM_ADDR_TYPE_STOP;
- drvdata->vissctlr |= BIT(idx + 16);
- /* SSSTATUS, bit[9] - turn on start/stop logic */
- drvdata->vinst_ctrl |= BIT(9);
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(addr_stop);
-
-static ssize_t addr_ctxtype_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- ssize_t len;
- u8 idx, val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->addr_idx;
- /* CONTEXTTYPE, bits[3:2] */
- val = BMVAL(drvdata->addr_acc[idx], 2, 3);
- len = scnprintf(buf, PAGE_SIZE, "%s\n", val == ETM_CTX_NONE ? "none" :
- (val == ETM_CTX_CTXID ? "ctxid" :
- (val == ETM_CTX_VMID ? "vmid" : "all")));
- spin_unlock(&drvdata->spinlock);
- return len;
-}
-
-static ssize_t addr_ctxtype_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- u8 idx;
- char str[10] = "";
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (strlen(buf) >= 10)
- return -EINVAL;
- if (sscanf(buf, "%s", str) != 1)
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->addr_idx;
- if (!strcmp(str, "none"))
- /* start by clearing context type bits */
- drvdata->addr_acc[idx] &= ~(BIT(2) | BIT(3));
- else if (!strcmp(str, "ctxid")) {
- /* 0b01 The trace unit performs a Context ID */
- if (drvdata->numcidc) {
- drvdata->addr_acc[idx] |= BIT(2);
- drvdata->addr_acc[idx] &= ~BIT(3);
- }
- } else if (!strcmp(str, "vmid")) {
- /* 0b10 The trace unit performs a VMID */
- if (drvdata->numvmidc) {
- drvdata->addr_acc[idx] &= ~BIT(2);
- drvdata->addr_acc[idx] |= BIT(3);
- }
- } else if (!strcmp(str, "all")) {
- /*
- * 0b11 The trace unit performs a Context ID
- * comparison and a VMID
- */
- if (drvdata->numcidc)
- drvdata->addr_acc[idx] |= BIT(2);
- if (drvdata->numvmidc)
- drvdata->addr_acc[idx] |= BIT(3);
- }
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(addr_ctxtype);
-
-static ssize_t addr_context_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- u8 idx;
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->addr_idx;
- /* context ID comparator bits[6:4] */
- val = BMVAL(drvdata->addr_acc[idx], 4, 6);
- spin_unlock(&drvdata->spinlock);
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t addr_context_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- u8 idx;
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
- if ((drvdata->numcidc <= 1) && (drvdata->numvmidc <= 1))
- return -EINVAL;
- if (val >= (drvdata->numcidc >= drvdata->numvmidc ?
- drvdata->numcidc : drvdata->numvmidc))
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->addr_idx;
- /* clear context ID comparator bits[6:4] */
- drvdata->addr_acc[idx] &= ~(BIT(4) | BIT(5) | BIT(6));
- drvdata->addr_acc[idx] |= (val << 4);
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(addr_context);
-
-static ssize_t seq_idx_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->seq_idx;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t seq_idx_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
- if (val >= drvdata->nrseqstate - 1)
- return -EINVAL;
-
- /*
- * Use spinlock to ensure index doesn't change while it gets
- * dereferenced multiple times within a spinlock block elsewhere.
- */
- spin_lock(&drvdata->spinlock);
- drvdata->seq_idx = val;
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(seq_idx);
-
-static ssize_t seq_state_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->seq_state;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t seq_state_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
- if (val >= drvdata->nrseqstate)
- return -EINVAL;
-
- drvdata->seq_state = val;
- return size;
-}
-static DEVICE_ATTR_RW(seq_state);
-
-static ssize_t seq_event_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- u8 idx;
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->seq_idx;
- val = drvdata->seq_ctrl[idx];
- spin_unlock(&drvdata->spinlock);
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t seq_event_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- u8 idx;
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->seq_idx;
- /* RST, bits[7:0] */
- drvdata->seq_ctrl[idx] = val & 0xFF;
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(seq_event);
-
-static ssize_t seq_reset_event_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->seq_rst;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t seq_reset_event_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
- if (!(drvdata->nrseqstate))
- return -EINVAL;
-
- drvdata->seq_rst = val & ETMv4_EVENT_MASK;
- return size;
-}
-static DEVICE_ATTR_RW(seq_reset_event);
-
-static ssize_t cntr_idx_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->cntr_idx;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t cntr_idx_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
- if (val >= drvdata->nr_cntr)
- return -EINVAL;
-
- /*
- * Use spinlock to ensure index doesn't change while it gets
- * dereferenced multiple times within a spinlock block elsewhere.
- */
- spin_lock(&drvdata->spinlock);
- drvdata->cntr_idx = val;
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(cntr_idx);
-
-static ssize_t cntrldvr_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- u8 idx;
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->cntr_idx;
- val = drvdata->cntrldvr[idx];
- spin_unlock(&drvdata->spinlock);
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t cntrldvr_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- u8 idx;
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
- if (val > ETM_CNTR_MAX_VAL)
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->cntr_idx;
- drvdata->cntrldvr[idx] = val;
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(cntrldvr);
-
-static ssize_t cntr_val_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- u8 idx;
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->cntr_idx;
- val = drvdata->cntr_val[idx];
- spin_unlock(&drvdata->spinlock);
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t cntr_val_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- u8 idx;
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
- if (val > ETM_CNTR_MAX_VAL)
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->cntr_idx;
- drvdata->cntr_val[idx] = val;
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(cntr_val);
-
-static ssize_t cntr_ctrl_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- u8 idx;
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->cntr_idx;
- val = drvdata->cntr_ctrl[idx];
- spin_unlock(&drvdata->spinlock);
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t cntr_ctrl_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- u8 idx;
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->cntr_idx;
- drvdata->cntr_ctrl[idx] = val;
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(cntr_ctrl);
-
-static ssize_t res_idx_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->res_idx;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t res_idx_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
- /* Resource selector pair 0 is always implemented and reserved */
- if (val < 2 || val >= drvdata->nr_resource * 2)
- return -EINVAL;
-
- /*
- * Use spinlock to ensure index doesn't change while it gets
- * dereferenced multiple times within a spinlock block elsewhere.
- */
- spin_lock(&drvdata->spinlock);
- drvdata->res_idx = val;
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(res_idx);
-
-static ssize_t res_ctrl_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- u8 idx;
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->res_idx;
- val = drvdata->res_ctrl[idx];
- spin_unlock(&drvdata->spinlock);
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t res_ctrl_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- u8 idx;
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->res_idx;
- /* For odd idx pair inversal bit is RES0 */
- if (idx % 2 != 0)
- /* PAIRINV, bit[21] */
- val &= ~BIT(21);
- drvdata->res_ctrl[idx] = val;
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(res_ctrl);
-
-static ssize_t ctxid_idx_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->ctxid_idx;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t ctxid_idx_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
- if (val >= drvdata->numcidc)
- return -EINVAL;
-
- /*
- * Use spinlock to ensure index doesn't change while it gets
- * dereferenced multiple times within a spinlock block elsewhere.
- */
- spin_lock(&drvdata->spinlock);
- drvdata->ctxid_idx = val;
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(ctxid_idx);
-
-static ssize_t ctxid_pid_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- u8 idx;
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->ctxid_idx;
- val = (unsigned long)drvdata->ctxid_vpid[idx];
- spin_unlock(&drvdata->spinlock);
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t ctxid_pid_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- u8 idx;
- unsigned long vpid, pid;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- /*
- * only implemented when ctxid tracing is enabled, i.e. at least one
- * ctxid comparator is implemented and ctxid is greater than 0 bits
- * in length
- */
- if (!drvdata->ctxid_size || !drvdata->numcidc)
- return -EINVAL;
- if (kstrtoul(buf, 16, &vpid))
- return -EINVAL;
-
- pid = coresight_vpid_to_pid(vpid);
-
- spin_lock(&drvdata->spinlock);
- idx = drvdata->ctxid_idx;
- drvdata->ctxid_pid[idx] = (u64)pid;
- drvdata->ctxid_vpid[idx] = (u64)vpid;
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(ctxid_pid);
-
-static ssize_t ctxid_masks_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val1, val2;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- spin_lock(&drvdata->spinlock);
- val1 = drvdata->ctxid_mask0;
- val2 = drvdata->ctxid_mask1;
- spin_unlock(&drvdata->spinlock);
- return scnprintf(buf, PAGE_SIZE, "%#lx %#lx\n", val1, val2);
-}
-
-static ssize_t ctxid_masks_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- u8 i, j, maskbyte;
- unsigned long val1, val2, mask;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- /*
- * only implemented when ctxid tracing is enabled, i.e. at least one
- * ctxid comparator is implemented and ctxid is greater than 0 bits
- * in length
- */
- if (!drvdata->ctxid_size || !drvdata->numcidc)
- return -EINVAL;
- if (sscanf(buf, "%lx %lx", &val1, &val2) != 2)
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
- /*
- * each byte[0..3] controls mask value applied to ctxid
- * comparator[0..3]
- */
- switch (drvdata->numcidc) {
- case 0x1:
- /* COMP0, bits[7:0] */
- drvdata->ctxid_mask0 = val1 & 0xFF;
- break;
- case 0x2:
- /* COMP1, bits[15:8] */
- drvdata->ctxid_mask0 = val1 & 0xFFFF;
- break;
- case 0x3:
- /* COMP2, bits[23:16] */
- drvdata->ctxid_mask0 = val1 & 0xFFFFFF;
- break;
- case 0x4:
- /* COMP3, bits[31:24] */
- drvdata->ctxid_mask0 = val1;
- break;
- case 0x5:
- /* COMP4, bits[7:0] */
- drvdata->ctxid_mask0 = val1;
- drvdata->ctxid_mask1 = val2 & 0xFF;
- break;
- case 0x6:
- /* COMP5, bits[15:8] */
- drvdata->ctxid_mask0 = val1;
- drvdata->ctxid_mask1 = val2 & 0xFFFF;
- break;
- case 0x7:
- /* COMP6, bits[23:16] */
- drvdata->ctxid_mask0 = val1;
- drvdata->ctxid_mask1 = val2 & 0xFFFFFF;
- break;
- case 0x8:
- /* COMP7, bits[31:24] */
- drvdata->ctxid_mask0 = val1;
- drvdata->ctxid_mask1 = val2;
- break;
- default:
- break;
- }
- /*
- * If software sets a mask bit to 1, it must program relevant byte
- * of ctxid comparator value 0x0, otherwise behavior is unpredictable.
- * For example, if bit[3] of ctxid_mask0 is 1, we must clear bits[31:24]
- * of ctxid comparator0 value (corresponding to byte 0) register.
- */
- mask = drvdata->ctxid_mask0;
- for (i = 0; i < drvdata->numcidc; i++) {
- /* mask value of corresponding ctxid comparator */
- maskbyte = mask & ETMv4_EVENT_MASK;
- /*
- * each bit corresponds to a byte of respective ctxid comparator
- * value register
- */
- for (j = 0; j < 8; j++) {
- if (maskbyte & 1)
- drvdata->ctxid_pid[i] &= ~(0xFF << (j * 8));
- maskbyte >>= 1;
- }
- /* Select the next ctxid comparator mask value */
- if (i == 3)
- /* ctxid comparators[4-7] */
- mask = drvdata->ctxid_mask1;
- else
- mask >>= 0x8;
- }
-
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(ctxid_masks);
-
-static ssize_t vmid_idx_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->vmid_idx;
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t vmid_idx_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
- if (val >= drvdata->numvmidc)
- return -EINVAL;
-
- /*
- * Use spinlock to ensure index doesn't change while it gets
- * dereferenced multiple times within a spinlock block elsewhere.
- */
- spin_lock(&drvdata->spinlock);
- drvdata->vmid_idx = val;
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(vmid_idx);
-
-static ssize_t vmid_val_show(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = (unsigned long)drvdata->vmid_val[drvdata->vmid_idx];
- return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
-}
-
-static ssize_t vmid_val_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- unsigned long val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- /*
- * only implemented when vmid tracing is enabled, i.e. at least one
- * vmid comparator is implemented and at least 8 bit vmid size
- */
- if (!drvdata->vmid_size || !drvdata->numvmidc)
- return -EINVAL;
- if (kstrtoul(buf, 16, &val))
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
- drvdata->vmid_val[drvdata->vmid_idx] = (u64)val;
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(vmid_val);
-
-static ssize_t vmid_masks_show(struct device *dev,
- struct device_attribute *attr, char *buf)
-{
- unsigned long val1, val2;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- spin_lock(&drvdata->spinlock);
- val1 = drvdata->vmid_mask0;
- val2 = drvdata->vmid_mask1;
- spin_unlock(&drvdata->spinlock);
- return scnprintf(buf, PAGE_SIZE, "%#lx %#lx\n", val1, val2);
-}
-
-static ssize_t vmid_masks_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t size)
-{
- u8 i, j, maskbyte;
- unsigned long val1, val2, mask;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
- /*
- * only implemented when vmid tracing is enabled, i.e. at least one
- * vmid comparator is implemented and at least 8 bit vmid size
- */
- if (!drvdata->vmid_size || !drvdata->numvmidc)
- return -EINVAL;
- if (sscanf(buf, "%lx %lx", &val1, &val2) != 2)
- return -EINVAL;
-
- spin_lock(&drvdata->spinlock);
-
- /*
- * each byte[0..3] controls mask value applied to vmid
- * comparator[0..3]
- */
- switch (drvdata->numvmidc) {
- case 0x1:
- /* COMP0, bits[7:0] */
- drvdata->vmid_mask0 = val1 & 0xFF;
- break;
- case 0x2:
- /* COMP1, bits[15:8] */
- drvdata->vmid_mask0 = val1 & 0xFFFF;
- break;
- case 0x3:
- /* COMP2, bits[23:16] */
- drvdata->vmid_mask0 = val1 & 0xFFFFFF;
- break;
- case 0x4:
- /* COMP3, bits[31:24] */
- drvdata->vmid_mask0 = val1;
- break;
- case 0x5:
- /* COMP4, bits[7:0] */
- drvdata->vmid_mask0 = val1;
- drvdata->vmid_mask1 = val2 & 0xFF;
- break;
- case 0x6:
- /* COMP5, bits[15:8] */
- drvdata->vmid_mask0 = val1;
- drvdata->vmid_mask1 = val2 & 0xFFFF;
- break;
- case 0x7:
- /* COMP6, bits[23:16] */
- drvdata->vmid_mask0 = val1;
- drvdata->vmid_mask1 = val2 & 0xFFFFFF;
- break;
- case 0x8:
- /* COMP7, bits[31:24] */
- drvdata->vmid_mask0 = val1;
- drvdata->vmid_mask1 = val2;
- break;
- default:
- break;
- }
-
- /*
- * If software sets a mask bit to 1, it must program relevant byte
- * of vmid comparator value 0x0, otherwise behavior is unpredictable.
- * For example, if bit[3] of vmid_mask0 is 1, we must clear bits[31:24]
- * of vmid comparator0 value (corresponding to byte 0) register.
- */
- mask = drvdata->vmid_mask0;
- for (i = 0; i < drvdata->numvmidc; i++) {
- /* mask value of corresponding vmid comparator */
- maskbyte = mask & ETMv4_EVENT_MASK;
- /*
- * each bit corresponds to a byte of respective vmid comparator
- * value register
- */
- for (j = 0; j < 8; j++) {
- if (maskbyte & 1)
- drvdata->vmid_val[i] &= ~(0xFF << (j * 8));
- maskbyte >>= 1;
- }
- /* Select the next vmid comparator mask value */
- if (i == 3)
- /* vmid comparators[4-7] */
- mask = drvdata->vmid_mask1;
- else
- mask >>= 0x8;
- }
- spin_unlock(&drvdata->spinlock);
- return size;
-}
-static DEVICE_ATTR_RW(vmid_masks);
-
-static ssize_t cpu_show(struct device *dev,
- struct device_attribute *attr, char *buf)
-{
- int val;
- struct etmv4_drvdata *drvdata = dev_get_drvdata(dev->parent);
-
- val = drvdata->cpu;
- return scnprintf(buf, PAGE_SIZE, "%d\n", val);
-
-}
-static DEVICE_ATTR_RO(cpu);
-
-static struct attribute *coresight_etmv4_attrs[] = {
- &dev_attr_nr_pe_cmp.attr,
- &dev_attr_nr_addr_cmp.attr,
- &dev_attr_nr_cntr.attr,
- &dev_attr_nr_ext_inp.attr,
- &dev_attr_numcidc.attr,
- &dev_attr_numvmidc.attr,
- &dev_attr_nrseqstate.attr,
- &dev_attr_nr_resource.attr,
- &dev_attr_nr_ss_cmp.attr,
- &dev_attr_reset.attr,
- &dev_attr_mode.attr,
- &dev_attr_pe.attr,
- &dev_attr_event.attr,
- &dev_attr_event_instren.attr,
- &dev_attr_event_ts.attr,
- &dev_attr_syncfreq.attr,
- &dev_attr_cyc_threshold.attr,
- &dev_attr_bb_ctrl.attr,
- &dev_attr_event_vinst.attr,
- &dev_attr_s_exlevel_vinst.attr,
- &dev_attr_ns_exlevel_vinst.attr,
- &dev_attr_addr_idx.attr,
- &dev_attr_addr_instdatatype.attr,
- &dev_attr_addr_single.attr,
- &dev_attr_addr_range.attr,
- &dev_attr_addr_start.attr,
- &dev_attr_addr_stop.attr,
- &dev_attr_addr_ctxtype.attr,
- &dev_attr_addr_context.attr,
- &dev_attr_seq_idx.attr,
- &dev_attr_seq_state.attr,
- &dev_attr_seq_event.attr,
- &dev_attr_seq_reset_event.attr,
- &dev_attr_cntr_idx.attr,
- &dev_attr_cntrldvr.attr,
- &dev_attr_cntr_val.attr,
- &dev_attr_cntr_ctrl.attr,
- &dev_attr_res_idx.attr,
- &dev_attr_res_ctrl.attr,
- &dev_attr_ctxid_idx.attr,
- &dev_attr_ctxid_pid.attr,
- &dev_attr_ctxid_masks.attr,
- &dev_attr_vmid_idx.attr,
- &dev_attr_vmid_val.attr,
- &dev_attr_vmid_masks.attr,
- &dev_attr_cpu.attr,
- NULL,
-};
-
-#define coresight_simple_func(name, offset) \
-static ssize_t name##_show(struct device *_dev, \
- struct device_attribute *attr, char *buf) \
-{ \
- struct etmv4_drvdata *drvdata = dev_get_drvdata(_dev->parent); \
- return scnprintf(buf, PAGE_SIZE, "0x%x\n", \
- readl_relaxed(drvdata->base + offset)); \
-} \
-static DEVICE_ATTR_RO(name)
-
-coresight_simple_func(trcoslsr, TRCOSLSR);
-coresight_simple_func(trcpdcr, TRCPDCR);
-coresight_simple_func(trcpdsr, TRCPDSR);
-coresight_simple_func(trclsr, TRCLSR);
-coresight_simple_func(trcauthstatus, TRCAUTHSTATUS);
-coresight_simple_func(trcdevid, TRCDEVID);
-coresight_simple_func(trcdevtype, TRCDEVTYPE);
-coresight_simple_func(trcpidr0, TRCPIDR0);
-coresight_simple_func(trcpidr1, TRCPIDR1);
-coresight_simple_func(trcpidr2, TRCPIDR2);
-coresight_simple_func(trcpidr3, TRCPIDR3);
-
-static struct attribute *coresight_etmv4_mgmt_attrs[] = {
- &dev_attr_trcoslsr.attr,
- &dev_attr_trcpdcr.attr,
- &dev_attr_trcpdsr.attr,
- &dev_attr_trclsr.attr,
- &dev_attr_trcauthstatus.attr,
- &dev_attr_trcdevid.attr,
- &dev_attr_trcdevtype.attr,
- &dev_attr_trcpidr0.attr,
- &dev_attr_trcpidr1.attr,
- &dev_attr_trcpidr2.attr,
- &dev_attr_trcpidr3.attr,
- NULL,
-};
-
-coresight_simple_func(trcidr0, TRCIDR0);
-coresight_simple_func(trcidr1, TRCIDR1);
-coresight_simple_func(trcidr2, TRCIDR2);
-coresight_simple_func(trcidr3, TRCIDR3);
-coresight_simple_func(trcidr4, TRCIDR4);
-coresight_simple_func(trcidr5, TRCIDR5);
-/* trcidr[6,7] are reserved */
-coresight_simple_func(trcidr8, TRCIDR8);
-coresight_simple_func(trcidr9, TRCIDR9);
-coresight_simple_func(trcidr10, TRCIDR10);
-coresight_simple_func(trcidr11, TRCIDR11);
-coresight_simple_func(trcidr12, TRCIDR12);
-coresight_simple_func(trcidr13, TRCIDR13);
-
-static struct attribute *coresight_etmv4_trcidr_attrs[] = {
- &dev_attr_trcidr0.attr,
- &dev_attr_trcidr1.attr,
- &dev_attr_trcidr2.attr,
- &dev_attr_trcidr3.attr,
- &dev_attr_trcidr4.attr,
- &dev_attr_trcidr5.attr,
- /* trcidr[6,7] are reserved */
- &dev_attr_trcidr8.attr,
- &dev_attr_trcidr9.attr,
- &dev_attr_trcidr10.attr,
- &dev_attr_trcidr11.attr,
- &dev_attr_trcidr12.attr,
- &dev_attr_trcidr13.attr,
- NULL,
-};
-
-static const struct attribute_group coresight_etmv4_group = {
- .attrs = coresight_etmv4_attrs,
-};
-
-static const struct attribute_group coresight_etmv4_mgmt_group = {
- .attrs = coresight_etmv4_mgmt_attrs,
- .name = "mgmt",
-};
-
-static const struct attribute_group coresight_etmv4_trcidr_group = {
- .attrs = coresight_etmv4_trcidr_attrs,
- .name = "trcidr",
-};
-
-static const struct attribute_group *coresight_etmv4_groups[] = {
- &coresight_etmv4_group,
- &coresight_etmv4_mgmt_group,
- &coresight_etmv4_trcidr_group,
- NULL,
-};
-
static void etm4_init_arch_data(void *info)
{
u32 etmidr0;
@@ -2313,6 +408,9 @@
u32 etmidr5;
struct etmv4_drvdata *drvdata = info;
+ /* Make sure all registers are accessible */
+ etm4_os_unlock(drvdata);
+
CS_UNLOCK(drvdata->base);
/* find all capabilities of the tracing unit */
@@ -2464,93 +562,115 @@
CS_LOCK(drvdata->base);
}
-static void etm4_init_default_data(struct etmv4_drvdata *drvdata)
+static void etm4_set_default(struct etmv4_config *config)
{
- int i;
+ if (WARN_ON_ONCE(!config))
+ return;
- drvdata->pe_sel = 0x0;
- drvdata->cfg = (ETMv4_MODE_CTXID | ETM_MODE_VMID |
- ETMv4_MODE_TIMESTAMP | ETM_MODE_RETURNSTACK);
+ /*
+ * Make default initialisation trace everything
+ *
+ * Select the "always true" resource selector on the
+ * "Enablign Event" line and configure address range comparator
+ * '0' to trace all the possible address range. From there
+ * configure the "include/exclude" engine to include address
+ * range comparator '0'.
+ */
/* disable all events tracing */
- drvdata->eventctrl0 = 0x0;
- drvdata->eventctrl1 = 0x0;
+ config->eventctrl0 = 0x0;
+ config->eventctrl1 = 0x0;
/* disable stalling */
- drvdata->stall_ctrl = 0x0;
+ config->stall_ctrl = 0x0;
+
+ /* enable trace synchronization every 4096 bytes, if available */
+ config->syncfreq = 0xC;
/* disable timestamp event */
- drvdata->ts_ctrl = 0x0;
+ config->ts_ctrl = 0x0;
- /* enable trace synchronization every 4096 bytes for trace */
- if (drvdata->syncpr == false)
- drvdata->syncfreq = 0xC;
+ /* TRCVICTLR::EVENT = 0x01, select the always on logic */
+ config->vinst_ctrl |= BIT(0);
/*
- * enable viewInst to trace everything with start-stop logic in
- * started state
+ * TRCVICTLR::SSSTATUS == 1, the start-stop logic is
+ * in the started state
*/
- drvdata->vinst_ctrl |= BIT(0);
- /* set initial state of start-stop logic */
- if (drvdata->nr_addr_cmp)
- drvdata->vinst_ctrl |= BIT(9);
+ config->vinst_ctrl |= BIT(9);
- /* no address range filtering for ViewInst */
- drvdata->viiectlr = 0x0;
+ /*
+ * Configure address range comparator '0' to encompass all
+ * possible addresses.
+ */
+
+ /* First half of default address comparator: start at address 0 */
+ config->addr_val[ETM_DEFAULT_ADDR_COMP] = 0x0;
+ /* trace instruction addresses */
+ config->addr_acc[ETM_DEFAULT_ADDR_COMP] &= ~(BIT(0) | BIT(1));
+ /* EXLEVEL_NS, bits[12:15], only trace application and kernel space */
+ config->addr_acc[ETM_DEFAULT_ADDR_COMP] |= ETM_EXLEVEL_NS_HYP;
+ /* EXLEVEL_S, bits[11:8], don't trace anything in secure state */
+ config->addr_acc[ETM_DEFAULT_ADDR_COMP] |= (ETM_EXLEVEL_S_APP |
+ ETM_EXLEVEL_S_OS |
+ ETM_EXLEVEL_S_HYP);
+ config->addr_type[ETM_DEFAULT_ADDR_COMP] = ETM_ADDR_TYPE_RANGE;
+
+ /*
+ * Second half of default address comparator: go all
+ * the way to the top.
+ */
+ config->addr_val[ETM_DEFAULT_ADDR_COMP + 1] = ~0x0;
+ /* trace instruction addresses */
+ config->addr_acc[ETM_DEFAULT_ADDR_COMP + 1] &= ~(BIT(0) | BIT(1));
+ /* Address comparator type must be equal for both halves */
+ config->addr_acc[ETM_DEFAULT_ADDR_COMP + 1] =
+ config->addr_acc[ETM_DEFAULT_ADDR_COMP];
+ config->addr_type[ETM_DEFAULT_ADDR_COMP + 1] = ETM_ADDR_TYPE_RANGE;
+
+ /*
+ * Configure the ViewInst function to filter on address range
+ * comparator '0'.
+ */
+ config->viiectlr = BIT(0);
+
/* no start-stop filtering for ViewInst */
- drvdata->vissctlr = 0x0;
+ config->vissctlr = 0x0;
+}
- /* disable seq events */
- for (i = 0; i < drvdata->nrseqstate-1; i++)
- drvdata->seq_ctrl[i] = 0x0;
- drvdata->seq_rst = 0x0;
- drvdata->seq_state = 0x0;
+void etm4_config_trace_mode(struct etmv4_config *config)
+{
+ u32 addr_acc, mode;
- /* disable external input events */
- drvdata->ext_inp = 0x0;
+ mode = config->mode;
+ mode &= (ETM_MODE_EXCL_KERN | ETM_MODE_EXCL_USER);
- for (i = 0; i < drvdata->nr_cntr; i++) {
- drvdata->cntrldvr[i] = 0x0;
- drvdata->cntr_ctrl[i] = 0x0;
- drvdata->cntr_val[i] = 0x0;
- }
+ /* excluding kernel AND user space doesn't make sense */
+ WARN_ON_ONCE(mode == (ETM_MODE_EXCL_KERN | ETM_MODE_EXCL_USER));
- /* Resource selector pair 0 is always implemented and reserved */
- drvdata->res_idx = 0x2;
- for (i = 2; i < drvdata->nr_resource * 2; i++)
- drvdata->res_ctrl[i] = 0x0;
+ /* nothing to do if neither flags are set */
+ if (!(mode & ETM_MODE_EXCL_KERN) && !(mode & ETM_MODE_EXCL_USER))
+ return;
- for (i = 0; i < drvdata->nr_ss_cmp; i++) {
- drvdata->ss_ctrl[i] = 0x0;
- drvdata->ss_pe_cmp[i] = 0x0;
- }
-
- if (drvdata->nr_addr_cmp >= 1) {
- drvdata->addr_val[0] = (unsigned long)_stext;
- drvdata->addr_val[1] = (unsigned long)_etext;
- drvdata->addr_type[0] = ETM_ADDR_TYPE_RANGE;
- drvdata->addr_type[1] = ETM_ADDR_TYPE_RANGE;
- }
-
- for (i = 0; i < drvdata->numcidc; i++) {
- drvdata->ctxid_pid[i] = 0x0;
- drvdata->ctxid_vpid[i] = 0x0;
- }
-
- drvdata->ctxid_mask0 = 0x0;
- drvdata->ctxid_mask1 = 0x0;
-
- for (i = 0; i < drvdata->numvmidc; i++)
- drvdata->vmid_val[i] = 0x0;
- drvdata->vmid_mask0 = 0x0;
- drvdata->vmid_mask1 = 0x0;
+ addr_acc = config->addr_acc[ETM_DEFAULT_ADDR_COMP];
+ /* clear default config */
+ addr_acc &= ~(ETM_EXLEVEL_NS_APP | ETM_EXLEVEL_NS_OS);
/*
- * A trace ID value of 0 is invalid, so let's start at some
- * random value that fits in 7 bits. ETMv3.x has 0x10 so let's
- * start at 0x20.
+ * EXLEVEL_NS, bits[15:12]
+ * The Exception levels are:
+ * Bit[12] Exception level 0 - Application
+ * Bit[13] Exception level 1 - OS
+ * Bit[14] Exception level 2 - Hypervisor
+ * Bit[15] Never implemented
*/
- drvdata->trcid = 0x20 + drvdata->cpu;
+ if (mode & ETM_MODE_EXCL_KERN)
+ addr_acc |= ETM_EXLEVEL_NS_OS;
+ else
+ addr_acc |= ETM_EXLEVEL_NS_APP;
+
+ config->addr_acc[ETM_DEFAULT_ADDR_COMP] = addr_acc;
+ config->addr_acc[ETM_DEFAULT_ADDR_COMP + 1] = addr_acc;
}
static int etm4_cpu_callback(struct notifier_block *nfb, unsigned long action,
@@ -2569,7 +689,7 @@
etmdrvdata[cpu]->os_unlock = true;
}
- if (etmdrvdata[cpu]->enable)
+ if (local_read(&etmdrvdata[cpu]->mode))
etm4_enable_hw(etmdrvdata[cpu]);
spin_unlock(&etmdrvdata[cpu]->spinlock);
break;
@@ -2582,7 +702,7 @@
case CPU_DYING:
spin_lock(&etmdrvdata[cpu]->spinlock);
- if (etmdrvdata[cpu]->enable)
+ if (local_read(&etmdrvdata[cpu]->mode))
etm4_disable_hw(etmdrvdata[cpu]);
spin_unlock(&etmdrvdata[cpu]->spinlock);
break;
@@ -2595,6 +715,11 @@
.notifier_call = etm4_cpu_callback,
};
+static void etm4_init_trace_id(struct etmv4_drvdata *drvdata)
+{
+ drvdata->trcid = coresight_get_trace_id(drvdata->cpu);
+}
+
static int etm4_probe(struct amba_device *adev, const struct amba_id *id)
{
int ret;
@@ -2638,9 +763,6 @@
get_online_cpus();
etmdrvdata[drvdata->cpu] = drvdata;
- if (!smp_call_function_single(drvdata->cpu, etm4_os_unlock, drvdata, 1))
- drvdata->os_unlock = true;
-
if (smp_call_function_single(drvdata->cpu,
etm4_init_arch_data, drvdata, 1))
dev_err(dev, "ETM arch init failed\n");
@@ -2654,9 +776,9 @@
ret = -EINVAL;
goto err_arch_supported;
}
- etm4_init_default_data(drvdata);
- pm_runtime_put(&adev->dev);
+ etm4_init_trace_id(drvdata);
+ etm4_set_default(&drvdata->config);
desc->type = CORESIGHT_DEV_TYPE_SOURCE;
desc->subtype.source_subtype = CORESIGHT_DEV_SUBTYPE_SOURCE_PROC;
@@ -2667,9 +789,16 @@
drvdata->csdev = coresight_register(desc);
if (IS_ERR(drvdata->csdev)) {
ret = PTR_ERR(drvdata->csdev);
- goto err_coresight_register;
+ goto err_arch_supported;
}
+ ret = etm_perf_symlink(drvdata->csdev, true);
+ if (ret) {
+ coresight_unregister(drvdata->csdev);
+ goto err_arch_supported;
+ }
+
+ pm_runtime_put(&adev->dev);
dev_info(dev, "%s initialized\n", (char *)id->data);
if (boot_enable) {
@@ -2680,8 +809,6 @@
return 0;
err_arch_supported:
- pm_runtime_put(&adev->dev);
-err_coresight_register:
if (--etm4_count == 0)
unregister_hotcpu_notifier(&etm4_cpu_notifier);
return ret;
@@ -2698,6 +825,11 @@
.mask = 0x000fffff,
.data = "ETM 4.0",
},
+ { /* ETM 4.0 - A72, Maia, HiSilicon */
+ .id = 0x000bb95a,
+ .mask = 0x000fffff,
+ .data = "ETM 4.0",
+ },
{ 0, 0},
};
diff --git a/drivers/hwtracing/coresight/coresight-etm4x.h b/drivers/hwtracing/coresight/coresight-etm4x.h
index c341002..5359c51 100644
--- a/drivers/hwtracing/coresight/coresight-etm4x.h
+++ b/drivers/hwtracing/coresight/coresight-etm4x.h
@@ -13,6 +13,7 @@
#ifndef _CORESIGHT_CORESIGHT_ETM_H
#define _CORESIGHT_CORESIGHT_ETM_H
+#include <asm/local.h>
#include <linux/spinlock.h>
#include "coresight-priv.h"
@@ -175,71 +176,38 @@
#define ETM_MODE_TRACE_RESET BIT(25)
#define ETM_MODE_TRACE_ERR BIT(26)
#define ETM_MODE_VIEWINST_STARTSTOP BIT(27)
-#define ETMv4_MODE_ALL 0xFFFFFFF
+#define ETMv4_MODE_ALL (GENMASK(27, 0) | \
+ ETM_MODE_EXCL_KERN | \
+ ETM_MODE_EXCL_USER)
#define TRCSTATR_IDLE_BIT 0
+#define ETM_DEFAULT_ADDR_COMP 0
+
+/* secure state access levels */
+#define ETM_EXLEVEL_S_APP BIT(8)
+#define ETM_EXLEVEL_S_OS BIT(9)
+#define ETM_EXLEVEL_S_NA BIT(10)
+#define ETM_EXLEVEL_S_HYP BIT(11)
+/* non-secure state access levels */
+#define ETM_EXLEVEL_NS_APP BIT(12)
+#define ETM_EXLEVEL_NS_OS BIT(13)
+#define ETM_EXLEVEL_NS_HYP BIT(14)
+#define ETM_EXLEVEL_NS_NA BIT(15)
/**
- * struct etm4_drvdata - specifics associated to an ETM component
- * @base: Memory mapped base address for this component.
- * @dev: The device entity associated to this component.
- * @csdev: Component vitals needed by the framework.
- * @spinlock: Only one at a time pls.
- * @cpu: The cpu this component is affined to.
- * @arch: ETM version number.
- * @enable: Is this ETM currently tracing.
- * @sticky_enable: true if ETM base configuration has been done.
- * @boot_enable:True if we should start tracing at boot time.
- * @os_unlock: True if access to management registers is allowed.
- * @nr_pe: The number of processing entity available for tracing.
- * @nr_pe_cmp: The number of processing entity comparator inputs that are
- * available for tracing.
- * @nr_addr_cmp:Number of pairs of address comparators available
- * as found in ETMIDR4 0-3.
- * @nr_cntr: Number of counters as found in ETMIDR5 bit 28-30.
- * @nr_ext_inp: Number of external input.
- * @numcidc: Number of contextID comparators.
- * @numvmidc: Number of VMID comparators.
- * @nrseqstate: The number of sequencer states that are implemented.
- * @nr_event: Indicates how many events the trace unit support.
- * @nr_resource:The number of resource selection pairs available for tracing.
- * @nr_ss_cmp: Number of single-shot comparator controls that are available.
+ * struct etmv4_config - configuration information related to an ETMv4
* @mode: Controls various modes supported by this ETM.
- * @trcid: value of the current ID for this component.
- * @trcid_size: Indicates the trace ID width.
- * @instrp0: Tracing of load and store instructions
- * as P0 elements is supported.
- * @trccond: If the trace unit supports conditional
- * instruction tracing.
- * @retstack: Indicates if the implementation supports a return stack.
- * @trc_error: Whether a trace unit can trace a system
- * error exception.
- * @atbtrig: If the implementation can support ATB triggers
- * @lpoverride: If the implementation can support low-power state over.
* @pe_sel: Controls which PE to trace.
* @cfg: Controls the tracing options.
* @eventctrl0: Controls the tracing of arbitrary events.
* @eventctrl1: Controls the behavior of the events that @event_ctrl0 selects.
* @stallctl: If functionality that prevents trace unit buffer overflows
* is available.
- * @sysstall: Does the system support stall control of the PE?
- * @nooverflow: Indicate if overflow prevention is supported.
- * @stall_ctrl: Enables trace unit functionality that prevents trace
- * unit buffer overflows.
- * @ts_size: Global timestamp size field.
* @ts_ctrl: Controls the insertion of global timestamps in the
* trace streams.
- * @syncpr: Indicates if an implementation has a fixed
- * synchronization period.
* @syncfreq: Controls how often trace synchronization requests occur.
- * @trccci: Indicates if the trace unit supports cycle counting
- * for instruction.
- * @ccsize: Indicates the size of the cycle counter in bits.
- * @ccitmin: minimum value that can be programmed in
* the TRCCCCTLR register.
* @ccctlr: Sets the threshold value for cycle counting.
- * @trcbb: Indicates if the trace unit supports branch broadcast tracing.
- * @q_support: Q element support characteristics.
* @vinst_ctrl: Controls instruction trace filtering.
* @viiectlr: Set or read, the address range comparators.
* @vissctlr: Set, or read, the single address comparators that control the
@@ -264,73 +232,28 @@
* @addr_acc: Address comparator access type.
* @addr_type: Current status of the comparator register.
* @ctxid_idx: Context ID index selector.
- * @ctxid_size: Size of the context ID field to consider.
* @ctxid_pid: Value of the context ID comparator.
* @ctxid_vpid: Virtual PID seen by users if PID namespace is enabled, otherwise
* the same value of ctxid_pid.
* @ctxid_mask0:Context ID comparator mask for comparator 0-3.
* @ctxid_mask1:Context ID comparator mask for comparator 4-7.
* @vmid_idx: VM ID index selector.
- * @vmid_size: Size of the VM ID comparator to consider.
* @vmid_val: Value of the VM ID comparator.
* @vmid_mask0: VM ID comparator mask for comparator 0-3.
* @vmid_mask1: VM ID comparator mask for comparator 4-7.
- * @s_ex_level: In secure state, indicates whether instruction tracing is
- * supported for the corresponding Exception level.
- * @ns_ex_level:In non-secure state, indicates whether instruction tracing is
- * supported for the corresponding Exception level.
* @ext_inp: External input selection.
*/
-struct etmv4_drvdata {
- void __iomem *base;
- struct device *dev;
- struct coresight_device *csdev;
- spinlock_t spinlock;
- int cpu;
- u8 arch;
- bool enable;
- bool sticky_enable;
- bool boot_enable;
- bool os_unlock;
- u8 nr_pe;
- u8 nr_pe_cmp;
- u8 nr_addr_cmp;
- u8 nr_cntr;
- u8 nr_ext_inp;
- u8 numcidc;
- u8 numvmidc;
- u8 nrseqstate;
- u8 nr_event;
- u8 nr_resource;
- u8 nr_ss_cmp;
+struct etmv4_config {
u32 mode;
- u8 trcid;
- u8 trcid_size;
- bool instrp0;
- bool trccond;
- bool retstack;
- bool trc_error;
- bool atbtrig;
- bool lpoverride;
u32 pe_sel;
u32 cfg;
u32 eventctrl0;
u32 eventctrl1;
- bool stallctl;
- bool sysstall;
- bool nooverflow;
u32 stall_ctrl;
- u8 ts_size;
u32 ts_ctrl;
- bool syncpr;
u32 syncfreq;
- bool trccci;
- u8 ccsize;
- u8 ccitmin;
u32 ccctlr;
- bool trcbb;
u32 bb_ctrl;
- bool q_support;
u32 vinst_ctrl;
u32 viiectlr;
u32 vissctlr;
@@ -353,19 +276,119 @@
u64 addr_acc[ETM_MAX_SINGLE_ADDR_CMP];
u8 addr_type[ETM_MAX_SINGLE_ADDR_CMP];
u8 ctxid_idx;
- u8 ctxid_size;
u64 ctxid_pid[ETMv4_MAX_CTXID_CMP];
u64 ctxid_vpid[ETMv4_MAX_CTXID_CMP];
u32 ctxid_mask0;
u32 ctxid_mask1;
u8 vmid_idx;
- u8 vmid_size;
u64 vmid_val[ETM_MAX_VMID_CMP];
u32 vmid_mask0;
u32 vmid_mask1;
+ u32 ext_inp;
+};
+
+/**
+ * struct etm4_drvdata - specifics associated to an ETM component
+ * @base: Memory mapped base address for this component.
+ * @dev: The device entity associated to this component.
+ * @csdev: Component vitals needed by the framework.
+ * @spinlock: Only one at a time pls.
+ * @mode: This tracer's mode, i.e sysFS, Perf or disabled.
+ * @cpu: The cpu this component is affined to.
+ * @arch: ETM version number.
+ * @nr_pe: The number of processing entity available for tracing.
+ * @nr_pe_cmp: The number of processing entity comparator inputs that are
+ * available for tracing.
+ * @nr_addr_cmp:Number of pairs of address comparators available
+ * as found in ETMIDR4 0-3.
+ * @nr_cntr: Number of counters as found in ETMIDR5 bit 28-30.
+ * @nr_ext_inp: Number of external input.
+ * @numcidc: Number of contextID comparators.
+ * @numvmidc: Number of VMID comparators.
+ * @nrseqstate: The number of sequencer states that are implemented.
+ * @nr_event: Indicates how many events the trace unit support.
+ * @nr_resource:The number of resource selection pairs available for tracing.
+ * @nr_ss_cmp: Number of single-shot comparator controls that are available.
+ * @trcid: value of the current ID for this component.
+ * @trcid_size: Indicates the trace ID width.
+ * @ts_size: Global timestamp size field.
+ * @ctxid_size: Size of the context ID field to consider.
+ * @vmid_size: Size of the VM ID comparator to consider.
+ * @ccsize: Indicates the size of the cycle counter in bits.
+ * @ccitmin: minimum value that can be programmed in
+ * @s_ex_level: In secure state, indicates whether instruction tracing is
+ * supported for the corresponding Exception level.
+ * @ns_ex_level:In non-secure state, indicates whether instruction tracing is
+ * supported for the corresponding Exception level.
+ * @sticky_enable: true if ETM base configuration has been done.
+ * @boot_enable:True if we should start tracing at boot time.
+ * @os_unlock: True if access to management registers is allowed.
+ * @instrp0: Tracing of load and store instructions
+ * as P0 elements is supported.
+ * @trcbb: Indicates if the trace unit supports branch broadcast tracing.
+ * @trccond: If the trace unit supports conditional
+ * instruction tracing.
+ * @retstack: Indicates if the implementation supports a return stack.
+ * @trccci: Indicates if the trace unit supports cycle counting
+ * for instruction.
+ * @q_support: Q element support characteristics.
+ * @trc_error: Whether a trace unit can trace a system
+ * error exception.
+ * @syncpr: Indicates if an implementation has a fixed
+ * synchronization period.
+ * @stall_ctrl: Enables trace unit functionality that prevents trace
+ * unit buffer overflows.
+ * @sysstall: Does the system support stall control of the PE?
+ * @nooverflow: Indicate if overflow prevention is supported.
+ * @atbtrig: If the implementation can support ATB triggers
+ * @lpoverride: If the implementation can support low-power state over.
+ * @config: structure holding configuration parameters.
+ */
+struct etmv4_drvdata {
+ void __iomem *base;
+ struct device *dev;
+ struct coresight_device *csdev;
+ spinlock_t spinlock;
+ local_t mode;
+ int cpu;
+ u8 arch;
+ u8 nr_pe;
+ u8 nr_pe_cmp;
+ u8 nr_addr_cmp;
+ u8 nr_cntr;
+ u8 nr_ext_inp;
+ u8 numcidc;
+ u8 numvmidc;
+ u8 nrseqstate;
+ u8 nr_event;
+ u8 nr_resource;
+ u8 nr_ss_cmp;
+ u8 trcid;
+ u8 trcid_size;
+ u8 ts_size;
+ u8 ctxid_size;
+ u8 vmid_size;
+ u8 ccsize;
+ u8 ccitmin;
u8 s_ex_level;
u8 ns_ex_level;
- u32 ext_inp;
+ u8 q_support;
+ bool sticky_enable;
+ bool boot_enable;
+ bool os_unlock;
+ bool instrp0;
+ bool trcbb;
+ bool trccond;
+ bool retstack;
+ bool trccci;
+ bool trc_error;
+ bool syncpr;
+ bool stallctl;
+ bool sysstall;
+ bool nooverflow;
+ bool atbtrig;
+ bool lpoverride;
+ struct etmv4_config config;
};
/* Address comparator access types */
@@ -391,4 +414,7 @@
ETM_ADDR_TYPE_START,
ETM_ADDR_TYPE_STOP,
};
+
+extern const struct attribute_group *coresight_etmv4_groups[];
+void etm4_config_trace_mode(struct etmv4_config *config);
#endif
diff --git a/drivers/hwtracing/coresight/coresight-funnel.c b/drivers/hwtracing/coresight/coresight-funnel.c
index 0600ca3..05df789 100644
--- a/drivers/hwtracing/coresight/coresight-funnel.c
+++ b/drivers/hwtracing/coresight/coresight-funnel.c
@@ -221,7 +221,6 @@
if (IS_ERR(drvdata->csdev))
return PTR_ERR(drvdata->csdev);
- dev_info(dev, "FUNNEL initialized\n");
return 0;
}
diff --git a/drivers/hwtracing/coresight/coresight-priv.h b/drivers/hwtracing/coresight/coresight-priv.h
index 333edda..ad975c5 100644
--- a/drivers/hwtracing/coresight/coresight-priv.h
+++ b/drivers/hwtracing/coresight/coresight-priv.h
@@ -37,12 +37,42 @@
#define ETM_MODE_EXCL_KERN BIT(30)
#define ETM_MODE_EXCL_USER BIT(31)
+#define coresight_simple_func(type, name, offset) \
+static ssize_t name##_show(struct device *_dev, \
+ struct device_attribute *attr, char *buf) \
+{ \
+ type *drvdata = dev_get_drvdata(_dev->parent); \
+ return scnprintf(buf, PAGE_SIZE, "0x%x\n", \
+ readl_relaxed(drvdata->base + offset)); \
+} \
+static DEVICE_ATTR_RO(name)
+
enum cs_mode {
CS_MODE_DISABLED,
CS_MODE_SYSFS,
CS_MODE_PERF,
};
+/**
+ * struct cs_buffer - keep track of a recording session' specifics
+ * @cur: index of the current buffer
+ * @nr_pages: max number of pages granted to us
+ * @offset: offset within the current buffer
+ * @data_size: how much we collected in this run
+ * @lost: other than zero if we had a HW buffer wrap around
+ * @snapshot: is this run in snapshot mode
+ * @data_pages: a handle the ring buffer
+ */
+struct cs_buffers {
+ unsigned int cur;
+ unsigned int nr_pages;
+ unsigned long offset;
+ local_t data_size;
+ local_t lost;
+ bool snapshot;
+ void **data_pages;
+};
+
static inline void CS_LOCK(void __iomem *addr)
{
do {
diff --git a/drivers/hwtracing/coresight/coresight-replicator.c b/drivers/hwtracing/coresight/coresight-replicator.c
index 4299c05..c6982e3 100644
--- a/drivers/hwtracing/coresight/coresight-replicator.c
+++ b/drivers/hwtracing/coresight/coresight-replicator.c
@@ -114,7 +114,6 @@
pm_runtime_put(&pdev->dev);
- dev_info(dev, "REPLICATOR initialized\n");
return 0;
out_disable_pm:
diff --git a/drivers/hwtracing/coresight/coresight-stm.c b/drivers/hwtracing/coresight/coresight-stm.c
new file mode 100644
index 0000000..73be58a
--- /dev/null
+++ b/drivers/hwtracing/coresight/coresight-stm.c
@@ -0,0 +1,920 @@
+/* Copyright (c) 2015-2016, The Linux Foundation. All rights reserved.
+ *
+ * Description: CoreSight System Trace Macrocell driver
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * Initial implementation by Pratik Patel
+ * (C) 2014-2015 Pratik Patel <pratikp@codeaurora.org>
+ *
+ * Serious refactoring, code cleanup and upgrading to the Coresight upstream
+ * framework by Mathieu Poirier
+ * (C) 2015-2016 Mathieu Poirier <mathieu.poirier@linaro.org>
+ *
+ * Guaranteed timing and support for various packet type coming from the
+ * generic STM API by Chunyan Zhang
+ * (C) 2015-2016 Chunyan Zhang <zhang.chunyan@linaro.org>
+ */
+#include <asm/local.h>
+#include <linux/amba/bus.h>
+#include <linux/bitmap.h>
+#include <linux/clk.h>
+#include <linux/coresight.h>
+#include <linux/coresight-stm.h>
+#include <linux/err.h>
+#include <linux/kernel.h>
+#include <linux/moduleparam.h>
+#include <linux/of_address.h>
+#include <linux/perf_event.h>
+#include <linux/pm_runtime.h>
+#include <linux/stm.h>
+
+#include "coresight-priv.h"
+
+#define STMDMASTARTR 0xc04
+#define STMDMASTOPR 0xc08
+#define STMDMASTATR 0xc0c
+#define STMDMACTLR 0xc10
+#define STMDMAIDR 0xcfc
+#define STMHEER 0xd00
+#define STMHETER 0xd20
+#define STMHEBSR 0xd60
+#define STMHEMCR 0xd64
+#define STMHEMASTR 0xdf4
+#define STMHEFEAT1R 0xdf8
+#define STMHEIDR 0xdfc
+#define STMSPER 0xe00
+#define STMSPTER 0xe20
+#define STMPRIVMASKR 0xe40
+#define STMSPSCR 0xe60
+#define STMSPMSCR 0xe64
+#define STMSPOVERRIDER 0xe68
+#define STMSPMOVERRIDER 0xe6c
+#define STMSPTRIGCSR 0xe70
+#define STMTCSR 0xe80
+#define STMTSSTIMR 0xe84
+#define STMTSFREQR 0xe8c
+#define STMSYNCR 0xe90
+#define STMAUXCR 0xe94
+#define STMSPFEAT1R 0xea0
+#define STMSPFEAT2R 0xea4
+#define STMSPFEAT3R 0xea8
+#define STMITTRIGGER 0xee8
+#define STMITATBDATA0 0xeec
+#define STMITATBCTR2 0xef0
+#define STMITATBID 0xef4
+#define STMITATBCTR0 0xef8
+
+#define STM_32_CHANNEL 32
+#define BYTES_PER_CHANNEL 256
+#define STM_TRACE_BUF_SIZE 4096
+#define STM_SW_MASTER_END 127
+
+/* Register bit definition */
+#define STMTCSR_BUSY_BIT 23
+/* Reserve the first 10 channels for kernel usage */
+#define STM_CHANNEL_OFFSET 0
+
+enum stm_pkt_type {
+ STM_PKT_TYPE_DATA = 0x98,
+ STM_PKT_TYPE_FLAG = 0xE8,
+ STM_PKT_TYPE_TRIG = 0xF8,
+};
+
+#define stm_channel_addr(drvdata, ch) (drvdata->chs.base + \
+ (ch * BYTES_PER_CHANNEL))
+#define stm_channel_off(type, opts) (type & ~opts)
+
+static int boot_nr_channel;
+
+/*
+ * Not really modular but using module_param is the easiest way to
+ * remain consistent with existing use cases for now.
+ */
+module_param_named(
+ boot_nr_channel, boot_nr_channel, int, S_IRUGO
+);
+
+/**
+ * struct channel_space - central management entity for extended ports
+ * @base: memory mapped base address where channels start.
+ * @guaraneed: is the channel delivery guaranteed.
+ */
+struct channel_space {
+ void __iomem *base;
+ unsigned long *guaranteed;
+};
+
+/**
+ * struct stm_drvdata - specifics associated to an STM component
+ * @base: memory mapped base address for this component.
+ * @dev: the device entity associated to this component.
+ * @atclk: optional clock for the core parts of the STM.
+ * @csdev: component vitals needed by the framework.
+ * @spinlock: only one at a time pls.
+ * @chs: the channels accociated to this STM.
+ * @stm: structure associated to the generic STM interface.
+ * @mode: this tracer's mode, i.e sysFS, or disabled.
+ * @traceid: value of the current ID for this component.
+ * @write_bytes: Maximus bytes this STM can write at a time.
+ * @stmsper: settings for register STMSPER.
+ * @stmspscr: settings for register STMSPSCR.
+ * @numsp: the total number of stimulus port support by this STM.
+ * @stmheer: settings for register STMHEER.
+ * @stmheter: settings for register STMHETER.
+ * @stmhebsr: settings for register STMHEBSR.
+ */
+struct stm_drvdata {
+ void __iomem *base;
+ struct device *dev;
+ struct clk *atclk;
+ struct coresight_device *csdev;
+ spinlock_t spinlock;
+ struct channel_space chs;
+ struct stm_data stm;
+ local_t mode;
+ u8 traceid;
+ u32 write_bytes;
+ u32 stmsper;
+ u32 stmspscr;
+ u32 numsp;
+ u32 stmheer;
+ u32 stmheter;
+ u32 stmhebsr;
+};
+
+static void stm_hwevent_enable_hw(struct stm_drvdata *drvdata)
+{
+ CS_UNLOCK(drvdata->base);
+
+ writel_relaxed(drvdata->stmhebsr, drvdata->base + STMHEBSR);
+ writel_relaxed(drvdata->stmheter, drvdata->base + STMHETER);
+ writel_relaxed(drvdata->stmheer, drvdata->base + STMHEER);
+ writel_relaxed(0x01 | /* Enable HW event tracing */
+ 0x04, /* Error detection on event tracing */
+ drvdata->base + STMHEMCR);
+
+ CS_LOCK(drvdata->base);
+}
+
+static void stm_port_enable_hw(struct stm_drvdata *drvdata)
+{
+ CS_UNLOCK(drvdata->base);
+ /* ATB trigger enable on direct writes to TRIG locations */
+ writel_relaxed(0x10,
+ drvdata->base + STMSPTRIGCSR);
+ writel_relaxed(drvdata->stmspscr, drvdata->base + STMSPSCR);
+ writel_relaxed(drvdata->stmsper, drvdata->base + STMSPER);
+
+ CS_LOCK(drvdata->base);
+}
+
+static void stm_enable_hw(struct stm_drvdata *drvdata)
+{
+ if (drvdata->stmheer)
+ stm_hwevent_enable_hw(drvdata);
+
+ stm_port_enable_hw(drvdata);
+
+ CS_UNLOCK(drvdata->base);
+
+ /* 4096 byte between synchronisation packets */
+ writel_relaxed(0xFFF, drvdata->base + STMSYNCR);
+ writel_relaxed((drvdata->traceid << 16 | /* trace id */
+ 0x02 | /* timestamp enable */
+ 0x01), /* global STM enable */
+ drvdata->base + STMTCSR);
+
+ CS_LOCK(drvdata->base);
+}
+
+static int stm_enable(struct coresight_device *csdev,
+ struct perf_event_attr *attr, u32 mode)
+{
+ u32 val;
+ struct stm_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
+
+ if (mode != CS_MODE_SYSFS)
+ return -EINVAL;
+
+ val = local_cmpxchg(&drvdata->mode, CS_MODE_DISABLED, mode);
+
+ /* Someone is already using the tracer */
+ if (val)
+ return -EBUSY;
+
+ pm_runtime_get_sync(drvdata->dev);
+
+ spin_lock(&drvdata->spinlock);
+ stm_enable_hw(drvdata);
+ spin_unlock(&drvdata->spinlock);
+
+ dev_info(drvdata->dev, "STM tracing enabled\n");
+ return 0;
+}
+
+static void stm_hwevent_disable_hw(struct stm_drvdata *drvdata)
+{
+ CS_UNLOCK(drvdata->base);
+
+ writel_relaxed(0x0, drvdata->base + STMHEMCR);
+ writel_relaxed(0x0, drvdata->base + STMHEER);
+ writel_relaxed(0x0, drvdata->base + STMHETER);
+
+ CS_LOCK(drvdata->base);
+}
+
+static void stm_port_disable_hw(struct stm_drvdata *drvdata)
+{
+ CS_UNLOCK(drvdata->base);
+
+ writel_relaxed(0x0, drvdata->base + STMSPER);
+ writel_relaxed(0x0, drvdata->base + STMSPTRIGCSR);
+
+ CS_LOCK(drvdata->base);
+}
+
+static void stm_disable_hw(struct stm_drvdata *drvdata)
+{
+ u32 val;
+
+ CS_UNLOCK(drvdata->base);
+
+ val = readl_relaxed(drvdata->base + STMTCSR);
+ val &= ~0x1; /* clear global STM enable [0] */
+ writel_relaxed(val, drvdata->base + STMTCSR);
+
+ CS_LOCK(drvdata->base);
+
+ stm_port_disable_hw(drvdata);
+ if (drvdata->stmheer)
+ stm_hwevent_disable_hw(drvdata);
+}
+
+static void stm_disable(struct coresight_device *csdev)
+{
+ struct stm_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
+
+ /*
+ * For as long as the tracer isn't disabled another entity can't
+ * change its status. As such we can read the status here without
+ * fearing it will change under us.
+ */
+ if (local_read(&drvdata->mode) == CS_MODE_SYSFS) {
+ spin_lock(&drvdata->spinlock);
+ stm_disable_hw(drvdata);
+ spin_unlock(&drvdata->spinlock);
+
+ /* Wait until the engine has completely stopped */
+ coresight_timeout(drvdata, STMTCSR, STMTCSR_BUSY_BIT, 0);
+
+ pm_runtime_put(drvdata->dev);
+
+ local_set(&drvdata->mode, CS_MODE_DISABLED);
+ dev_info(drvdata->dev, "STM tracing disabled\n");
+ }
+}
+
+static int stm_trace_id(struct coresight_device *csdev)
+{
+ struct stm_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
+
+ return drvdata->traceid;
+}
+
+static const struct coresight_ops_source stm_source_ops = {
+ .trace_id = stm_trace_id,
+ .enable = stm_enable,
+ .disable = stm_disable,
+};
+
+static const struct coresight_ops stm_cs_ops = {
+ .source_ops = &stm_source_ops,
+};
+
+static inline bool stm_addr_unaligned(const void *addr, u8 write_bytes)
+{
+ return ((unsigned long)addr & (write_bytes - 1));
+}
+
+static void stm_send(void *addr, const void *data, u32 size, u8 write_bytes)
+{
+ u8 paload[8];
+
+ if (stm_addr_unaligned(data, write_bytes)) {
+ memcpy(paload, data, size);
+ data = paload;
+ }
+
+ /* now we are 64bit/32bit aligned */
+ switch (size) {
+#ifdef CONFIG_64BIT
+ case 8:
+ writeq_relaxed(*(u64 *)data, addr);
+ break;
+#endif
+ case 4:
+ writel_relaxed(*(u32 *)data, addr);
+ break;
+ case 2:
+ writew_relaxed(*(u16 *)data, addr);
+ break;
+ case 1:
+ writeb_relaxed(*(u8 *)data, addr);
+ break;
+ default:
+ break;
+ }
+}
+
+static int stm_generic_link(struct stm_data *stm_data,
+ unsigned int master, unsigned int channel)
+{
+ struct stm_drvdata *drvdata = container_of(stm_data,
+ struct stm_drvdata, stm);
+ if (!drvdata || !drvdata->csdev)
+ return -EINVAL;
+
+ return coresight_enable(drvdata->csdev);
+}
+
+static void stm_generic_unlink(struct stm_data *stm_data,
+ unsigned int master, unsigned int channel)
+{
+ struct stm_drvdata *drvdata = container_of(stm_data,
+ struct stm_drvdata, stm);
+ if (!drvdata || !drvdata->csdev)
+ return;
+
+ stm_disable(drvdata->csdev);
+}
+
+static long stm_generic_set_options(struct stm_data *stm_data,
+ unsigned int master,
+ unsigned int channel,
+ unsigned int nr_chans,
+ unsigned long options)
+{
+ struct stm_drvdata *drvdata = container_of(stm_data,
+ struct stm_drvdata, stm);
+ if (!(drvdata && local_read(&drvdata->mode)))
+ return -EINVAL;
+
+ if (channel >= drvdata->numsp)
+ return -EINVAL;
+
+ switch (options) {
+ case STM_OPTION_GUARANTEED:
+ set_bit(channel, drvdata->chs.guaranteed);
+ break;
+
+ case STM_OPTION_INVARIANT:
+ clear_bit(channel, drvdata->chs.guaranteed);
+ break;
+
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static ssize_t stm_generic_packet(struct stm_data *stm_data,
+ unsigned int master,
+ unsigned int channel,
+ unsigned int packet,
+ unsigned int flags,
+ unsigned int size,
+ const unsigned char *payload)
+{
+ unsigned long ch_addr;
+ struct stm_drvdata *drvdata = container_of(stm_data,
+ struct stm_drvdata, stm);
+
+ if (!(drvdata && local_read(&drvdata->mode)))
+ return 0;
+
+ if (channel >= drvdata->numsp)
+ return 0;
+
+ ch_addr = (unsigned long)stm_channel_addr(drvdata, channel);
+
+ flags = (flags == STP_PACKET_TIMESTAMPED) ? STM_FLAG_TIMESTAMPED : 0;
+ flags |= test_bit(channel, drvdata->chs.guaranteed) ?
+ STM_FLAG_GUARANTEED : 0;
+
+ if (size > drvdata->write_bytes)
+ size = drvdata->write_bytes;
+ else
+ size = rounddown_pow_of_two(size);
+
+ switch (packet) {
+ case STP_PACKET_FLAG:
+ ch_addr |= stm_channel_off(STM_PKT_TYPE_FLAG, flags);
+
+ /*
+ * The generic STM core sets a size of '0' on flag packets.
+ * As such send a flag packet of size '1' and tell the
+ * core we did so.
+ */
+ stm_send((void *)ch_addr, payload, 1, drvdata->write_bytes);
+ size = 1;
+ break;
+
+ case STP_PACKET_DATA:
+ ch_addr |= stm_channel_off(STM_PKT_TYPE_DATA, flags);
+ stm_send((void *)ch_addr, payload, size,
+ drvdata->write_bytes);
+ break;
+
+ default:
+ return -ENOTSUPP;
+ }
+
+ return size;
+}
+
+static ssize_t hwevent_enable_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct stm_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ unsigned long val = drvdata->stmheer;
+
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t hwevent_enable_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ struct stm_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ unsigned long val;
+ int ret = 0;
+
+ ret = kstrtoul(buf, 16, &val);
+ if (ret)
+ return -EINVAL;
+
+ drvdata->stmheer = val;
+ /* HW event enable and trigger go hand in hand */
+ drvdata->stmheter = val;
+
+ return size;
+}
+static DEVICE_ATTR_RW(hwevent_enable);
+
+static ssize_t hwevent_select_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct stm_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ unsigned long val = drvdata->stmhebsr;
+
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t hwevent_select_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ struct stm_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ unsigned long val;
+ int ret = 0;
+
+ ret = kstrtoul(buf, 16, &val);
+ if (ret)
+ return -EINVAL;
+
+ drvdata->stmhebsr = val;
+
+ return size;
+}
+static DEVICE_ATTR_RW(hwevent_select);
+
+static ssize_t port_select_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct stm_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ unsigned long val;
+
+ if (!local_read(&drvdata->mode)) {
+ val = drvdata->stmspscr;
+ } else {
+ spin_lock(&drvdata->spinlock);
+ val = readl_relaxed(drvdata->base + STMSPSCR);
+ spin_unlock(&drvdata->spinlock);
+ }
+
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t port_select_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ struct stm_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ unsigned long val, stmsper;
+ int ret = 0;
+
+ ret = kstrtoul(buf, 16, &val);
+ if (ret)
+ return ret;
+
+ spin_lock(&drvdata->spinlock);
+ drvdata->stmspscr = val;
+
+ if (local_read(&drvdata->mode)) {
+ CS_UNLOCK(drvdata->base);
+ /* Process as per ARM's TRM recommendation */
+ stmsper = readl_relaxed(drvdata->base + STMSPER);
+ writel_relaxed(0x0, drvdata->base + STMSPER);
+ writel_relaxed(drvdata->stmspscr, drvdata->base + STMSPSCR);
+ writel_relaxed(stmsper, drvdata->base + STMSPER);
+ CS_LOCK(drvdata->base);
+ }
+ spin_unlock(&drvdata->spinlock);
+
+ return size;
+}
+static DEVICE_ATTR_RW(port_select);
+
+static ssize_t port_enable_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct stm_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ unsigned long val;
+
+ if (!local_read(&drvdata->mode)) {
+ val = drvdata->stmsper;
+ } else {
+ spin_lock(&drvdata->spinlock);
+ val = readl_relaxed(drvdata->base + STMSPER);
+ spin_unlock(&drvdata->spinlock);
+ }
+
+ return scnprintf(buf, PAGE_SIZE, "%#lx\n", val);
+}
+
+static ssize_t port_enable_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ struct stm_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ unsigned long val;
+ int ret = 0;
+
+ ret = kstrtoul(buf, 16, &val);
+ if (ret)
+ return ret;
+
+ spin_lock(&drvdata->spinlock);
+ drvdata->stmsper = val;
+
+ if (local_read(&drvdata->mode)) {
+ CS_UNLOCK(drvdata->base);
+ writel_relaxed(drvdata->stmsper, drvdata->base + STMSPER);
+ CS_LOCK(drvdata->base);
+ }
+ spin_unlock(&drvdata->spinlock);
+
+ return size;
+}
+static DEVICE_ATTR_RW(port_enable);
+
+static ssize_t traceid_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ unsigned long val;
+ struct stm_drvdata *drvdata = dev_get_drvdata(dev->parent);
+
+ val = drvdata->traceid;
+ return sprintf(buf, "%#lx\n", val);
+}
+
+static ssize_t traceid_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t size)
+{
+ int ret;
+ unsigned long val;
+ struct stm_drvdata *drvdata = dev_get_drvdata(dev->parent);
+
+ ret = kstrtoul(buf, 16, &val);
+ if (ret)
+ return ret;
+
+ /* traceid field is 7bit wide on STM32 */
+ drvdata->traceid = val & 0x7f;
+ return size;
+}
+static DEVICE_ATTR_RW(traceid);
+
+#define coresight_stm_simple_func(name, offset) \
+ coresight_simple_func(struct stm_drvdata, name, offset)
+
+coresight_stm_simple_func(tcsr, STMTCSR);
+coresight_stm_simple_func(tsfreqr, STMTSFREQR);
+coresight_stm_simple_func(syncr, STMSYNCR);
+coresight_stm_simple_func(sper, STMSPER);
+coresight_stm_simple_func(spter, STMSPTER);
+coresight_stm_simple_func(privmaskr, STMPRIVMASKR);
+coresight_stm_simple_func(spscr, STMSPSCR);
+coresight_stm_simple_func(spmscr, STMSPMSCR);
+coresight_stm_simple_func(spfeat1r, STMSPFEAT1R);
+coresight_stm_simple_func(spfeat2r, STMSPFEAT2R);
+coresight_stm_simple_func(spfeat3r, STMSPFEAT3R);
+coresight_stm_simple_func(devid, CORESIGHT_DEVID);
+
+static struct attribute *coresight_stm_attrs[] = {
+ &dev_attr_hwevent_enable.attr,
+ &dev_attr_hwevent_select.attr,
+ &dev_attr_port_enable.attr,
+ &dev_attr_port_select.attr,
+ &dev_attr_traceid.attr,
+ NULL,
+};
+
+static struct attribute *coresight_stm_mgmt_attrs[] = {
+ &dev_attr_tcsr.attr,
+ &dev_attr_tsfreqr.attr,
+ &dev_attr_syncr.attr,
+ &dev_attr_sper.attr,
+ &dev_attr_spter.attr,
+ &dev_attr_privmaskr.attr,
+ &dev_attr_spscr.attr,
+ &dev_attr_spmscr.attr,
+ &dev_attr_spfeat1r.attr,
+ &dev_attr_spfeat2r.attr,
+ &dev_attr_spfeat3r.attr,
+ &dev_attr_devid.attr,
+ NULL,
+};
+
+static const struct attribute_group coresight_stm_group = {
+ .attrs = coresight_stm_attrs,
+};
+
+static const struct attribute_group coresight_stm_mgmt_group = {
+ .attrs = coresight_stm_mgmt_attrs,
+ .name = "mgmt",
+};
+
+static const struct attribute_group *coresight_stm_groups[] = {
+ &coresight_stm_group,
+ &coresight_stm_mgmt_group,
+ NULL,
+};
+
+static int stm_get_resource_byname(struct device_node *np,
+ char *ch_base, struct resource *res)
+{
+ const char *name = NULL;
+ int index = 0, found = 0;
+
+ while (!of_property_read_string_index(np, "reg-names", index, &name)) {
+ if (strcmp(ch_base, name)) {
+ index++;
+ continue;
+ }
+
+ /* We have a match and @index is where it's at */
+ found = 1;
+ break;
+ }
+
+ if (!found)
+ return -EINVAL;
+
+ return of_address_to_resource(np, index, res);
+}
+
+static u32 stm_fundamental_data_size(struct stm_drvdata *drvdata)
+{
+ u32 stmspfeat2r;
+
+ if (!IS_ENABLED(CONFIG_64BIT))
+ return 4;
+
+ stmspfeat2r = readl_relaxed(drvdata->base + STMSPFEAT2R);
+
+ /*
+ * bit[15:12] represents the fundamental data size
+ * 0 - 32-bit data
+ * 1 - 64-bit data
+ */
+ return BMVAL(stmspfeat2r, 12, 15) ? 8 : 4;
+}
+
+static u32 stm_num_stimulus_port(struct stm_drvdata *drvdata)
+{
+ u32 numsp;
+
+ numsp = readl_relaxed(drvdata->base + CORESIGHT_DEVID);
+ /*
+ * NUMPS in STMDEVID is 17 bit long and if equal to 0x0,
+ * 32 stimulus ports are supported.
+ */
+ numsp &= 0x1ffff;
+ if (!numsp)
+ numsp = STM_32_CHANNEL;
+ return numsp;
+}
+
+static void stm_init_default_data(struct stm_drvdata *drvdata)
+{
+ /* Don't use port selection */
+ drvdata->stmspscr = 0x0;
+ /*
+ * Enable all channel regardless of their number. When port
+ * selection isn't used (see above) STMSPER applies to all
+ * 32 channel group available, hence setting all 32 bits to 1
+ */
+ drvdata->stmsper = ~0x0;
+
+ /*
+ * The trace ID value for *ETM* tracers start at CPU_ID * 2 + 0x10 and
+ * anything equal to or higher than 0x70 is reserved. Since 0x00 is
+ * also reserved the STM trace ID needs to be higher than 0x00 and
+ * lowner than 0x10.
+ */
+ drvdata->traceid = 0x1;
+
+ /* Set invariant transaction timing on all channels */
+ bitmap_clear(drvdata->chs.guaranteed, 0, drvdata->numsp);
+}
+
+static void stm_init_generic_data(struct stm_drvdata *drvdata)
+{
+ drvdata->stm.name = dev_name(drvdata->dev);
+
+ /*
+ * MasterIDs are assigned at HW design phase. As such the core is
+ * using a single master for interaction with this device.
+ */
+ drvdata->stm.sw_start = 1;
+ drvdata->stm.sw_end = 1;
+ drvdata->stm.hw_override = true;
+ drvdata->stm.sw_nchannels = drvdata->numsp;
+ drvdata->stm.packet = stm_generic_packet;
+ drvdata->stm.link = stm_generic_link;
+ drvdata->stm.unlink = stm_generic_unlink;
+ drvdata->stm.set_options = stm_generic_set_options;
+}
+
+static int stm_probe(struct amba_device *adev, const struct amba_id *id)
+{
+ int ret;
+ void __iomem *base;
+ unsigned long *guaranteed;
+ struct device *dev = &adev->dev;
+ struct coresight_platform_data *pdata = NULL;
+ struct stm_drvdata *drvdata;
+ struct resource *res = &adev->res;
+ struct resource ch_res;
+ size_t res_size, bitmap_size;
+ struct coresight_desc *desc;
+ struct device_node *np = adev->dev.of_node;
+
+ if (np) {
+ pdata = of_get_coresight_platform_data(dev, np);
+ if (IS_ERR(pdata))
+ return PTR_ERR(pdata);
+ adev->dev.platform_data = pdata;
+ }
+ drvdata = devm_kzalloc(dev, sizeof(*drvdata), GFP_KERNEL);
+ if (!drvdata)
+ return -ENOMEM;
+
+ drvdata->dev = &adev->dev;
+ drvdata->atclk = devm_clk_get(&adev->dev, "atclk"); /* optional */
+ if (!IS_ERR(drvdata->atclk)) {
+ ret = clk_prepare_enable(drvdata->atclk);
+ if (ret)
+ return ret;
+ }
+ dev_set_drvdata(dev, drvdata);
+
+ base = devm_ioremap_resource(dev, res);
+ if (IS_ERR(base))
+ return PTR_ERR(base);
+ drvdata->base = base;
+
+ ret = stm_get_resource_byname(np, "stm-stimulus-base", &ch_res);
+ if (ret)
+ return ret;
+
+ base = devm_ioremap_resource(dev, &ch_res);
+ if (IS_ERR(base))
+ return PTR_ERR(base);
+ drvdata->chs.base = base;
+
+ drvdata->write_bytes = stm_fundamental_data_size(drvdata);
+
+ if (boot_nr_channel) {
+ drvdata->numsp = boot_nr_channel;
+ res_size = min((resource_size_t)(boot_nr_channel *
+ BYTES_PER_CHANNEL), resource_size(res));
+ } else {
+ drvdata->numsp = stm_num_stimulus_port(drvdata);
+ res_size = min((resource_size_t)(drvdata->numsp *
+ BYTES_PER_CHANNEL), resource_size(res));
+ }
+ bitmap_size = BITS_TO_LONGS(drvdata->numsp) * sizeof(long);
+
+ guaranteed = devm_kzalloc(dev, bitmap_size, GFP_KERNEL);
+ if (!guaranteed)
+ return -ENOMEM;
+ drvdata->chs.guaranteed = guaranteed;
+
+ spin_lock_init(&drvdata->spinlock);
+
+ stm_init_default_data(drvdata);
+ stm_init_generic_data(drvdata);
+
+ if (stm_register_device(dev, &drvdata->stm, THIS_MODULE)) {
+ dev_info(dev,
+ "stm_register_device failed, probing deffered\n");
+ return -EPROBE_DEFER;
+ }
+
+ desc = devm_kzalloc(dev, sizeof(*desc), GFP_KERNEL);
+ if (!desc) {
+ ret = -ENOMEM;
+ goto stm_unregister;
+ }
+
+ desc->type = CORESIGHT_DEV_TYPE_SOURCE;
+ desc->subtype.source_subtype = CORESIGHT_DEV_SUBTYPE_SOURCE_SOFTWARE;
+ desc->ops = &stm_cs_ops;
+ desc->pdata = pdata;
+ desc->dev = dev;
+ desc->groups = coresight_stm_groups;
+ drvdata->csdev = coresight_register(desc);
+ if (IS_ERR(drvdata->csdev)) {
+ ret = PTR_ERR(drvdata->csdev);
+ goto stm_unregister;
+ }
+
+ pm_runtime_put(&adev->dev);
+
+ dev_info(dev, "%s initialized\n", (char *)id->data);
+ return 0;
+
+stm_unregister:
+ stm_unregister_device(&drvdata->stm);
+ return ret;
+}
+
+#ifdef CONFIG_PM
+static int stm_runtime_suspend(struct device *dev)
+{
+ struct stm_drvdata *drvdata = dev_get_drvdata(dev);
+
+ if (drvdata && !IS_ERR(drvdata->atclk))
+ clk_disable_unprepare(drvdata->atclk);
+
+ return 0;
+}
+
+static int stm_runtime_resume(struct device *dev)
+{
+ struct stm_drvdata *drvdata = dev_get_drvdata(dev);
+
+ if (drvdata && !IS_ERR(drvdata->atclk))
+ clk_prepare_enable(drvdata->atclk);
+
+ return 0;
+}
+#endif
+
+static const struct dev_pm_ops stm_dev_pm_ops = {
+ SET_RUNTIME_PM_OPS(stm_runtime_suspend, stm_runtime_resume, NULL)
+};
+
+static struct amba_id stm_ids[] = {
+ {
+ .id = 0x0003b962,
+ .mask = 0x0003ffff,
+ .data = "STM32",
+ },
+ { 0, 0},
+};
+
+static struct amba_driver stm_driver = {
+ .drv = {
+ .name = "coresight-stm",
+ .owner = THIS_MODULE,
+ .pm = &stm_dev_pm_ops,
+ .suppress_bind_attrs = true,
+ },
+ .probe = stm_probe,
+ .id_table = stm_ids,
+};
+
+builtin_amba_driver(stm_driver);
diff --git a/drivers/hwtracing/coresight/coresight-tmc-etf.c b/drivers/hwtracing/coresight/coresight-tmc-etf.c
new file mode 100644
index 0000000..466af86
--- /dev/null
+++ b/drivers/hwtracing/coresight/coresight-tmc-etf.c
@@ -0,0 +1,604 @@
+/*
+ * Copyright(C) 2016 Linaro Limited. All rights reserved.
+ * Author: Mathieu Poirier <mathieu.poirier@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/circ_buf.h>
+#include <linux/coresight.h>
+#include <linux/perf_event.h>
+#include <linux/slab.h>
+#include "coresight-priv.h"
+#include "coresight-tmc.h"
+
+void tmc_etb_enable_hw(struct tmc_drvdata *drvdata)
+{
+ CS_UNLOCK(drvdata->base);
+
+ /* Wait for TMCSReady bit to be set */
+ tmc_wait_for_tmcready(drvdata);
+
+ writel_relaxed(TMC_MODE_CIRCULAR_BUFFER, drvdata->base + TMC_MODE);
+ writel_relaxed(TMC_FFCR_EN_FMT | TMC_FFCR_EN_TI |
+ TMC_FFCR_FON_FLIN | TMC_FFCR_FON_TRIG_EVT |
+ TMC_FFCR_TRIGON_TRIGIN,
+ drvdata->base + TMC_FFCR);
+
+ writel_relaxed(drvdata->trigger_cntr, drvdata->base + TMC_TRG);
+ tmc_enable_hw(drvdata);
+
+ CS_LOCK(drvdata->base);
+}
+
+static void tmc_etb_dump_hw(struct tmc_drvdata *drvdata)
+{
+ char *bufp;
+ u32 read_data;
+ int i;
+
+ bufp = drvdata->buf;
+ while (1) {
+ for (i = 0; i < drvdata->memwidth; i++) {
+ read_data = readl_relaxed(drvdata->base + TMC_RRD);
+ if (read_data == 0xFFFFFFFF)
+ return;
+ memcpy(bufp, &read_data, 4);
+ bufp += 4;
+ }
+ }
+}
+
+static void tmc_etb_disable_hw(struct tmc_drvdata *drvdata)
+{
+ CS_UNLOCK(drvdata->base);
+
+ tmc_flush_and_stop(drvdata);
+ /*
+ * When operating in sysFS mode the content of the buffer needs to be
+ * read before the TMC is disabled.
+ */
+ if (local_read(&drvdata->mode) == CS_MODE_SYSFS)
+ tmc_etb_dump_hw(drvdata);
+ tmc_disable_hw(drvdata);
+
+ CS_LOCK(drvdata->base);
+}
+
+static void tmc_etf_enable_hw(struct tmc_drvdata *drvdata)
+{
+ CS_UNLOCK(drvdata->base);
+
+ /* Wait for TMCSReady bit to be set */
+ tmc_wait_for_tmcready(drvdata);
+
+ writel_relaxed(TMC_MODE_HARDWARE_FIFO, drvdata->base + TMC_MODE);
+ writel_relaxed(TMC_FFCR_EN_FMT | TMC_FFCR_EN_TI,
+ drvdata->base + TMC_FFCR);
+ writel_relaxed(0x0, drvdata->base + TMC_BUFWM);
+ tmc_enable_hw(drvdata);
+
+ CS_LOCK(drvdata->base);
+}
+
+static void tmc_etf_disable_hw(struct tmc_drvdata *drvdata)
+{
+ CS_UNLOCK(drvdata->base);
+
+ tmc_flush_and_stop(drvdata);
+ tmc_disable_hw(drvdata);
+
+ CS_LOCK(drvdata->base);
+}
+
+static int tmc_enable_etf_sink_sysfs(struct coresight_device *csdev, u32 mode)
+{
+ int ret = 0;
+ bool used = false;
+ char *buf = NULL;
+ long val;
+ unsigned long flags;
+ struct tmc_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
+
+ /* This shouldn't be happening */
+ if (WARN_ON(mode != CS_MODE_SYSFS))
+ return -EINVAL;
+
+ /*
+ * If we don't have a buffer release the lock and allocate memory.
+ * Otherwise keep the lock and move along.
+ */
+ spin_lock_irqsave(&drvdata->spinlock, flags);
+ if (!drvdata->buf) {
+ spin_unlock_irqrestore(&drvdata->spinlock, flags);
+
+ /* Allocating the memory here while outside of the spinlock */
+ buf = kzalloc(drvdata->size, GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+
+ /* Let's try again */
+ spin_lock_irqsave(&drvdata->spinlock, flags);
+ }
+
+ if (drvdata->reading) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ val = local_xchg(&drvdata->mode, mode);
+ /*
+ * In sysFS mode we can have multiple writers per sink. Since this
+ * sink is already enabled no memory is needed and the HW need not be
+ * touched.
+ */
+ if (val == CS_MODE_SYSFS)
+ goto out;
+
+ /*
+ * If drvdata::buf isn't NULL, memory was allocated for a previous
+ * trace run but wasn't read. If so simply zero-out the memory.
+ * Otherwise use the memory allocated above.
+ *
+ * The memory is freed when users read the buffer using the
+ * /dev/xyz.{etf|etb} interface. See tmc_read_unprepare_etf() for
+ * details.
+ */
+ if (drvdata->buf) {
+ memset(drvdata->buf, 0, drvdata->size);
+ } else {
+ used = true;
+ drvdata->buf = buf;
+ }
+
+ tmc_etb_enable_hw(drvdata);
+out:
+ spin_unlock_irqrestore(&drvdata->spinlock, flags);
+
+ /* Free memory outside the spinlock if need be */
+ if (!used && buf)
+ kfree(buf);
+
+ if (!ret)
+ dev_info(drvdata->dev, "TMC-ETB/ETF enabled\n");
+
+ return ret;
+}
+
+static int tmc_enable_etf_sink_perf(struct coresight_device *csdev, u32 mode)
+{
+ int ret = 0;
+ long val;
+ unsigned long flags;
+ struct tmc_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
+
+ /* This shouldn't be happening */
+ if (WARN_ON(mode != CS_MODE_PERF))
+ return -EINVAL;
+
+ spin_lock_irqsave(&drvdata->spinlock, flags);
+ if (drvdata->reading) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ val = local_xchg(&drvdata->mode, mode);
+ /*
+ * In Perf mode there can be only one writer per sink. There
+ * is also no need to continue if the ETB/ETR is already operated
+ * from sysFS.
+ */
+ if (val != CS_MODE_DISABLED) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ tmc_etb_enable_hw(drvdata);
+out:
+ spin_unlock_irqrestore(&drvdata->spinlock, flags);
+
+ return ret;
+}
+
+static int tmc_enable_etf_sink(struct coresight_device *csdev, u32 mode)
+{
+ switch (mode) {
+ case CS_MODE_SYSFS:
+ return tmc_enable_etf_sink_sysfs(csdev, mode);
+ case CS_MODE_PERF:
+ return tmc_enable_etf_sink_perf(csdev, mode);
+ }
+
+ /* We shouldn't be here */
+ return -EINVAL;
+}
+
+static void tmc_disable_etf_sink(struct coresight_device *csdev)
+{
+ long val;
+ unsigned long flags;
+ struct tmc_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
+
+ spin_lock_irqsave(&drvdata->spinlock, flags);
+ if (drvdata->reading) {
+ spin_unlock_irqrestore(&drvdata->spinlock, flags);
+ return;
+ }
+
+ val = local_xchg(&drvdata->mode, CS_MODE_DISABLED);
+ /* Disable the TMC only if it needs to */
+ if (val != CS_MODE_DISABLED)
+ tmc_etb_disable_hw(drvdata);
+
+ spin_unlock_irqrestore(&drvdata->spinlock, flags);
+
+ dev_info(drvdata->dev, "TMC-ETB/ETF disabled\n");
+}
+
+static int tmc_enable_etf_link(struct coresight_device *csdev,
+ int inport, int outport)
+{
+ unsigned long flags;
+ struct tmc_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
+
+ spin_lock_irqsave(&drvdata->spinlock, flags);
+ if (drvdata->reading) {
+ spin_unlock_irqrestore(&drvdata->spinlock, flags);
+ return -EBUSY;
+ }
+
+ tmc_etf_enable_hw(drvdata);
+ local_set(&drvdata->mode, CS_MODE_SYSFS);
+ spin_unlock_irqrestore(&drvdata->spinlock, flags);
+
+ dev_info(drvdata->dev, "TMC-ETF enabled\n");
+ return 0;
+}
+
+static void tmc_disable_etf_link(struct coresight_device *csdev,
+ int inport, int outport)
+{
+ unsigned long flags;
+ struct tmc_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
+
+ spin_lock_irqsave(&drvdata->spinlock, flags);
+ if (drvdata->reading) {
+ spin_unlock_irqrestore(&drvdata->spinlock, flags);
+ return;
+ }
+
+ tmc_etf_disable_hw(drvdata);
+ local_set(&drvdata->mode, CS_MODE_DISABLED);
+ spin_unlock_irqrestore(&drvdata->spinlock, flags);
+
+ dev_info(drvdata->dev, "TMC disabled\n");
+}
+
+static void *tmc_alloc_etf_buffer(struct coresight_device *csdev, int cpu,
+ void **pages, int nr_pages, bool overwrite)
+{
+ int node;
+ struct cs_buffers *buf;
+
+ if (cpu == -1)
+ cpu = smp_processor_id();
+ node = cpu_to_node(cpu);
+
+ /* Allocate memory structure for interaction with Perf */
+ buf = kzalloc_node(sizeof(struct cs_buffers), GFP_KERNEL, node);
+ if (!buf)
+ return NULL;
+
+ buf->snapshot = overwrite;
+ buf->nr_pages = nr_pages;
+ buf->data_pages = pages;
+
+ return buf;
+}
+
+static void tmc_free_etf_buffer(void *config)
+{
+ struct cs_buffers *buf = config;
+
+ kfree(buf);
+}
+
+static int tmc_set_etf_buffer(struct coresight_device *csdev,
+ struct perf_output_handle *handle,
+ void *sink_config)
+{
+ int ret = 0;
+ unsigned long head;
+ struct cs_buffers *buf = sink_config;
+
+ /* wrap head around to the amount of space we have */
+ head = handle->head & ((buf->nr_pages << PAGE_SHIFT) - 1);
+
+ /* find the page to write to */
+ buf->cur = head / PAGE_SIZE;
+
+ /* and offset within that page */
+ buf->offset = head % PAGE_SIZE;
+
+ local_set(&buf->data_size, 0);
+
+ return ret;
+}
+
+static unsigned long tmc_reset_etf_buffer(struct coresight_device *csdev,
+ struct perf_output_handle *handle,
+ void *sink_config, bool *lost)
+{
+ long size = 0;
+ struct cs_buffers *buf = sink_config;
+
+ if (buf) {
+ /*
+ * In snapshot mode ->data_size holds the new address of the
+ * ring buffer's head. The size itself is the whole address
+ * range since we want the latest information.
+ */
+ if (buf->snapshot)
+ handle->head = local_xchg(&buf->data_size,
+ buf->nr_pages << PAGE_SHIFT);
+ /*
+ * Tell the tracer PMU how much we got in this run and if
+ * something went wrong along the way. Nobody else can use
+ * this cs_buffers instance until we are done. As such
+ * resetting parameters here and squaring off with the ring
+ * buffer API in the tracer PMU is fine.
+ */
+ *lost = !!local_xchg(&buf->lost, 0);
+ size = local_xchg(&buf->data_size, 0);
+ }
+
+ return size;
+}
+
+static void tmc_update_etf_buffer(struct coresight_device *csdev,
+ struct perf_output_handle *handle,
+ void *sink_config)
+{
+ int i, cur;
+ u32 *buf_ptr;
+ u32 read_ptr, write_ptr;
+ u32 status, to_read;
+ unsigned long offset;
+ struct cs_buffers *buf = sink_config;
+ struct tmc_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
+
+ if (!buf)
+ return;
+
+ /* This shouldn't happen */
+ if (WARN_ON_ONCE(local_read(&drvdata->mode) != CS_MODE_PERF))
+ return;
+
+ CS_UNLOCK(drvdata->base);
+
+ tmc_flush_and_stop(drvdata);
+
+ read_ptr = readl_relaxed(drvdata->base + TMC_RRP);
+ write_ptr = readl_relaxed(drvdata->base + TMC_RWP);
+
+ /*
+ * Get a hold of the status register and see if a wrap around
+ * has occurred. If so adjust things accordingly.
+ */
+ status = readl_relaxed(drvdata->base + TMC_STS);
+ if (status & TMC_STS_FULL) {
+ local_inc(&buf->lost);
+ to_read = drvdata->size;
+ } else {
+ to_read = CIRC_CNT(write_ptr, read_ptr, drvdata->size);
+ }
+
+ /*
+ * The TMC RAM buffer may be bigger than the space available in the
+ * perf ring buffer (handle->size). If so advance the RRP so that we
+ * get the latest trace data.
+ */
+ if (to_read > handle->size) {
+ u32 mask = 0;
+
+ /*
+ * The value written to RRP must be byte-address aligned to
+ * the width of the trace memory databus _and_ to a frame
+ * boundary (16 byte), whichever is the biggest. For example,
+ * for 32-bit, 64-bit and 128-bit wide trace memory, the four
+ * LSBs must be 0s. For 256-bit wide trace memory, the five
+ * LSBs must be 0s.
+ */
+ switch (drvdata->memwidth) {
+ case TMC_MEM_INTF_WIDTH_32BITS:
+ case TMC_MEM_INTF_WIDTH_64BITS:
+ case TMC_MEM_INTF_WIDTH_128BITS:
+ mask = GENMASK(31, 5);
+ break;
+ case TMC_MEM_INTF_WIDTH_256BITS:
+ mask = GENMASK(31, 6);
+ break;
+ }
+
+ /*
+ * Make sure the new size is aligned in accordance with the
+ * requirement explained above.
+ */
+ to_read = handle->size & mask;
+ /* Move the RAM read pointer up */
+ read_ptr = (write_ptr + drvdata->size) - to_read;
+ /* Make sure we are still within our limits */
+ if (read_ptr > (drvdata->size - 1))
+ read_ptr -= drvdata->size;
+ /* Tell the HW */
+ writel_relaxed(read_ptr, drvdata->base + TMC_RRP);
+ local_inc(&buf->lost);
+ }
+
+ cur = buf->cur;
+ offset = buf->offset;
+
+ /* for every byte to read */
+ for (i = 0; i < to_read; i += 4) {
+ buf_ptr = buf->data_pages[cur] + offset;
+ *buf_ptr = readl_relaxed(drvdata->base + TMC_RRD);
+
+ offset += 4;
+ if (offset >= PAGE_SIZE) {
+ offset = 0;
+ cur++;
+ /* wrap around at the end of the buffer */
+ cur &= buf->nr_pages - 1;
+ }
+ }
+
+ /*
+ * In snapshot mode all we have to do is communicate to
+ * perf_aux_output_end() the address of the current head. In full
+ * trace mode the same function expects a size to move rb->aux_head
+ * forward.
+ */
+ if (buf->snapshot)
+ local_set(&buf->data_size, (cur * PAGE_SIZE) + offset);
+ else
+ local_add(to_read, &buf->data_size);
+
+ CS_LOCK(drvdata->base);
+}
+
+static const struct coresight_ops_sink tmc_etf_sink_ops = {
+ .enable = tmc_enable_etf_sink,
+ .disable = tmc_disable_etf_sink,
+ .alloc_buffer = tmc_alloc_etf_buffer,
+ .free_buffer = tmc_free_etf_buffer,
+ .set_buffer = tmc_set_etf_buffer,
+ .reset_buffer = tmc_reset_etf_buffer,
+ .update_buffer = tmc_update_etf_buffer,
+};
+
+static const struct coresight_ops_link tmc_etf_link_ops = {
+ .enable = tmc_enable_etf_link,
+ .disable = tmc_disable_etf_link,
+};
+
+const struct coresight_ops tmc_etb_cs_ops = {
+ .sink_ops = &tmc_etf_sink_ops,
+};
+
+const struct coresight_ops tmc_etf_cs_ops = {
+ .sink_ops = &tmc_etf_sink_ops,
+ .link_ops = &tmc_etf_link_ops,
+};
+
+int tmc_read_prepare_etb(struct tmc_drvdata *drvdata)
+{
+ long val;
+ enum tmc_mode mode;
+ int ret = 0;
+ unsigned long flags;
+
+ /* config types are set a boot time and never change */
+ if (WARN_ON_ONCE(drvdata->config_type != TMC_CONFIG_TYPE_ETB &&
+ drvdata->config_type != TMC_CONFIG_TYPE_ETF))
+ return -EINVAL;
+
+ spin_lock_irqsave(&drvdata->spinlock, flags);
+
+ if (drvdata->reading) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ /* There is no point in reading a TMC in HW FIFO mode */
+ mode = readl_relaxed(drvdata->base + TMC_MODE);
+ if (mode != TMC_MODE_CIRCULAR_BUFFER) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ val = local_read(&drvdata->mode);
+ /* Don't interfere if operated from Perf */
+ if (val == CS_MODE_PERF) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ /* If drvdata::buf is NULL the trace data has been read already */
+ if (drvdata->buf == NULL) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ /* Disable the TMC if need be */
+ if (val == CS_MODE_SYSFS)
+ tmc_etb_disable_hw(drvdata);
+
+ drvdata->reading = true;
+out:
+ spin_unlock_irqrestore(&drvdata->spinlock, flags);
+
+ return ret;
+}
+
+int tmc_read_unprepare_etb(struct tmc_drvdata *drvdata)
+{
+ char *buf = NULL;
+ enum tmc_mode mode;
+ unsigned long flags;
+
+ /* config types are set a boot time and never change */
+ if (WARN_ON_ONCE(drvdata->config_type != TMC_CONFIG_TYPE_ETB &&
+ drvdata->config_type != TMC_CONFIG_TYPE_ETF))
+ return -EINVAL;
+
+ spin_lock_irqsave(&drvdata->spinlock, flags);
+
+ /* There is no point in reading a TMC in HW FIFO mode */
+ mode = readl_relaxed(drvdata->base + TMC_MODE);
+ if (mode != TMC_MODE_CIRCULAR_BUFFER) {
+ spin_unlock_irqrestore(&drvdata->spinlock, flags);
+ return -EINVAL;
+ }
+
+ /* Re-enable the TMC if need be */
+ if (local_read(&drvdata->mode) == CS_MODE_SYSFS) {
+ /*
+ * The trace run will continue with the same allocated trace
+ * buffer. As such zero-out the buffer so that we don't end
+ * up with stale data.
+ *
+ * Since the tracer is still enabled drvdata::buf
+ * can't be NULL.
+ */
+ memset(drvdata->buf, 0, drvdata->size);
+ tmc_etb_enable_hw(drvdata);
+ } else {
+ /*
+ * The ETB/ETF is not tracing and the buffer was just read.
+ * As such prepare to free the trace buffer.
+ */
+ buf = drvdata->buf;
+ drvdata->buf = NULL;
+ }
+
+ drvdata->reading = false;
+ spin_unlock_irqrestore(&drvdata->spinlock, flags);
+
+ /*
+ * Free allocated memory outside of the spinlock. There is no need
+ * to assert the validity of 'buf' since calling kfree(NULL) is safe.
+ */
+ kfree(buf);
+
+ return 0;
+}
diff --git a/drivers/hwtracing/coresight/coresight-tmc-etr.c b/drivers/hwtracing/coresight/coresight-tmc-etr.c
new file mode 100644
index 0000000..847d1b5
--- /dev/null
+++ b/drivers/hwtracing/coresight/coresight-tmc-etr.c
@@ -0,0 +1,329 @@
+/*
+ * Copyright(C) 2016 Linaro Limited. All rights reserved.
+ * Author: Mathieu Poirier <mathieu.poirier@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/coresight.h>
+#include <linux/dma-mapping.h>
+#include "coresight-priv.h"
+#include "coresight-tmc.h"
+
+void tmc_etr_enable_hw(struct tmc_drvdata *drvdata)
+{
+ u32 axictl;
+
+ /* Zero out the memory to help with debug */
+ memset(drvdata->vaddr, 0, drvdata->size);
+
+ CS_UNLOCK(drvdata->base);
+
+ /* Wait for TMCSReady bit to be set */
+ tmc_wait_for_tmcready(drvdata);
+
+ writel_relaxed(drvdata->size / 4, drvdata->base + TMC_RSZ);
+ writel_relaxed(TMC_MODE_CIRCULAR_BUFFER, drvdata->base + TMC_MODE);
+
+ axictl = readl_relaxed(drvdata->base + TMC_AXICTL);
+ axictl |= TMC_AXICTL_WR_BURST_16;
+ writel_relaxed(axictl, drvdata->base + TMC_AXICTL);
+ axictl &= ~TMC_AXICTL_SCT_GAT_MODE;
+ writel_relaxed(axictl, drvdata->base + TMC_AXICTL);
+ axictl = (axictl &
+ ~(TMC_AXICTL_PROT_CTL_B0 | TMC_AXICTL_PROT_CTL_B1)) |
+ TMC_AXICTL_PROT_CTL_B1;
+ writel_relaxed(axictl, drvdata->base + TMC_AXICTL);
+
+ writel_relaxed(drvdata->paddr, drvdata->base + TMC_DBALO);
+ writel_relaxed(0x0, drvdata->base + TMC_DBAHI);
+ writel_relaxed(TMC_FFCR_EN_FMT | TMC_FFCR_EN_TI |
+ TMC_FFCR_FON_FLIN | TMC_FFCR_FON_TRIG_EVT |
+ TMC_FFCR_TRIGON_TRIGIN,
+ drvdata->base + TMC_FFCR);
+ writel_relaxed(drvdata->trigger_cntr, drvdata->base + TMC_TRG);
+ tmc_enable_hw(drvdata);
+
+ CS_LOCK(drvdata->base);
+}
+
+static void tmc_etr_dump_hw(struct tmc_drvdata *drvdata)
+{
+ u32 rwp, val;
+
+ rwp = readl_relaxed(drvdata->base + TMC_RWP);
+ val = readl_relaxed(drvdata->base + TMC_STS);
+
+ /* How much memory do we still have */
+ if (val & BIT(0))
+ drvdata->buf = drvdata->vaddr + rwp - drvdata->paddr;
+ else
+ drvdata->buf = drvdata->vaddr;
+}
+
+static void tmc_etr_disable_hw(struct tmc_drvdata *drvdata)
+{
+ CS_UNLOCK(drvdata->base);
+
+ tmc_flush_and_stop(drvdata);
+ /*
+ * When operating in sysFS mode the content of the buffer needs to be
+ * read before the TMC is disabled.
+ */
+ if (local_read(&drvdata->mode) == CS_MODE_SYSFS)
+ tmc_etr_dump_hw(drvdata);
+ tmc_disable_hw(drvdata);
+
+ CS_LOCK(drvdata->base);
+}
+
+static int tmc_enable_etr_sink_sysfs(struct coresight_device *csdev, u32 mode)
+{
+ int ret = 0;
+ bool used = false;
+ long val;
+ unsigned long flags;
+ void __iomem *vaddr = NULL;
+ dma_addr_t paddr;
+ struct tmc_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
+
+ /* This shouldn't be happening */
+ if (WARN_ON(mode != CS_MODE_SYSFS))
+ return -EINVAL;
+
+ /*
+ * If we don't have a buffer release the lock and allocate memory.
+ * Otherwise keep the lock and move along.
+ */
+ spin_lock_irqsave(&drvdata->spinlock, flags);
+ if (!drvdata->vaddr) {
+ spin_unlock_irqrestore(&drvdata->spinlock, flags);
+
+ /*
+ * Contiguous memory can't be allocated while a spinlock is
+ * held. As such allocate memory here and free it if a buffer
+ * has already been allocated (from a previous session).
+ */
+ vaddr = dma_alloc_coherent(drvdata->dev, drvdata->size,
+ &paddr, GFP_KERNEL);
+ if (!vaddr)
+ return -ENOMEM;
+
+ /* Let's try again */
+ spin_lock_irqsave(&drvdata->spinlock, flags);
+ }
+
+ if (drvdata->reading) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ val = local_xchg(&drvdata->mode, mode);
+ /*
+ * In sysFS mode we can have multiple writers per sink. Since this
+ * sink is already enabled no memory is needed and the HW need not be
+ * touched.
+ */
+ if (val == CS_MODE_SYSFS)
+ goto out;
+
+ /*
+ * If drvdata::buf == NULL, use the memory allocated above.
+ * Otherwise a buffer still exists from a previous session, so
+ * simply use that.
+ */
+ if (drvdata->buf == NULL) {
+ used = true;
+ drvdata->vaddr = vaddr;
+ drvdata->paddr = paddr;
+ drvdata->buf = drvdata->vaddr;
+ }
+
+ memset(drvdata->vaddr, 0, drvdata->size);
+
+ tmc_etr_enable_hw(drvdata);
+out:
+ spin_unlock_irqrestore(&drvdata->spinlock, flags);
+
+ /* Free memory outside the spinlock if need be */
+ if (!used && vaddr)
+ dma_free_coherent(drvdata->dev, drvdata->size, vaddr, paddr);
+
+ if (!ret)
+ dev_info(drvdata->dev, "TMC-ETR enabled\n");
+
+ return ret;
+}
+
+static int tmc_enable_etr_sink_perf(struct coresight_device *csdev, u32 mode)
+{
+ int ret = 0;
+ long val;
+ unsigned long flags;
+ struct tmc_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
+
+ /* This shouldn't be happening */
+ if (WARN_ON(mode != CS_MODE_PERF))
+ return -EINVAL;
+
+ spin_lock_irqsave(&drvdata->spinlock, flags);
+ if (drvdata->reading) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ val = local_xchg(&drvdata->mode, mode);
+ /*
+ * In Perf mode there can be only one writer per sink. There
+ * is also no need to continue if the ETR is already operated
+ * from sysFS.
+ */
+ if (val != CS_MODE_DISABLED) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ tmc_etr_enable_hw(drvdata);
+out:
+ spin_unlock_irqrestore(&drvdata->spinlock, flags);
+
+ return ret;
+}
+
+static int tmc_enable_etr_sink(struct coresight_device *csdev, u32 mode)
+{
+ switch (mode) {
+ case CS_MODE_SYSFS:
+ return tmc_enable_etr_sink_sysfs(csdev, mode);
+ case CS_MODE_PERF:
+ return tmc_enable_etr_sink_perf(csdev, mode);
+ }
+
+ /* We shouldn't be here */
+ return -EINVAL;
+}
+
+static void tmc_disable_etr_sink(struct coresight_device *csdev)
+{
+ long val;
+ unsigned long flags;
+ struct tmc_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
+
+ spin_lock_irqsave(&drvdata->spinlock, flags);
+ if (drvdata->reading) {
+ spin_unlock_irqrestore(&drvdata->spinlock, flags);
+ return;
+ }
+
+ val = local_xchg(&drvdata->mode, CS_MODE_DISABLED);
+ /* Disable the TMC only if it needs to */
+ if (val != CS_MODE_DISABLED)
+ tmc_etr_disable_hw(drvdata);
+
+ spin_unlock_irqrestore(&drvdata->spinlock, flags);
+
+ dev_info(drvdata->dev, "TMC-ETR disabled\n");
+}
+
+static const struct coresight_ops_sink tmc_etr_sink_ops = {
+ .enable = tmc_enable_etr_sink,
+ .disable = tmc_disable_etr_sink,
+};
+
+const struct coresight_ops tmc_etr_cs_ops = {
+ .sink_ops = &tmc_etr_sink_ops,
+};
+
+int tmc_read_prepare_etr(struct tmc_drvdata *drvdata)
+{
+ int ret = 0;
+ long val;
+ unsigned long flags;
+
+ /* config types are set a boot time and never change */
+ if (WARN_ON_ONCE(drvdata->config_type != TMC_CONFIG_TYPE_ETR))
+ return -EINVAL;
+
+ spin_lock_irqsave(&drvdata->spinlock, flags);
+ if (drvdata->reading) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ val = local_read(&drvdata->mode);
+ /* Don't interfere if operated from Perf */
+ if (val == CS_MODE_PERF) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ /* If drvdata::buf is NULL the trace data has been read already */
+ if (drvdata->buf == NULL) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ /* Disable the TMC if need be */
+ if (val == CS_MODE_SYSFS)
+ tmc_etr_disable_hw(drvdata);
+
+ drvdata->reading = true;
+out:
+ spin_unlock_irqrestore(&drvdata->spinlock, flags);
+
+ return ret;
+}
+
+int tmc_read_unprepare_etr(struct tmc_drvdata *drvdata)
+{
+ unsigned long flags;
+ dma_addr_t paddr;
+ void __iomem *vaddr = NULL;
+
+ /* config types are set a boot time and never change */
+ if (WARN_ON_ONCE(drvdata->config_type != TMC_CONFIG_TYPE_ETR))
+ return -EINVAL;
+
+ spin_lock_irqsave(&drvdata->spinlock, flags);
+
+ /* RE-enable the TMC if need be */
+ if (local_read(&drvdata->mode) == CS_MODE_SYSFS) {
+ /*
+ * The trace run will continue with the same allocated trace
+ * buffer. As such zero-out the buffer so that we don't end
+ * up with stale data.
+ *
+ * Since the tracer is still enabled drvdata::buf
+ * can't be NULL.
+ */
+ memset(drvdata->buf, 0, drvdata->size);
+ tmc_etr_enable_hw(drvdata);
+ } else {
+ /*
+ * The ETR is not tracing and the buffer was just read.
+ * As such prepare to free the trace buffer.
+ */
+ vaddr = drvdata->vaddr;
+ paddr = drvdata->paddr;
+ drvdata->buf = NULL;
+ }
+
+ drvdata->reading = false;
+ spin_unlock_irqrestore(&drvdata->spinlock, flags);
+
+ /* Free allocated memory out side of the spinlock */
+ if (vaddr)
+ dma_free_coherent(drvdata->dev, drvdata->size, vaddr, paddr);
+
+ return 0;
+}
diff --git a/drivers/hwtracing/coresight/coresight-tmc.c b/drivers/hwtracing/coresight/coresight-tmc.c
index 1be191f..9e02ac9 100644
--- a/drivers/hwtracing/coresight/coresight-tmc.c
+++ b/drivers/hwtracing/coresight/coresight-tmc.c
@@ -30,127 +30,27 @@
#include <linux/amba/bus.h>
#include "coresight-priv.h"
+#include "coresight-tmc.h"
-#define TMC_RSZ 0x004
-#define TMC_STS 0x00c
-#define TMC_RRD 0x010
-#define TMC_RRP 0x014
-#define TMC_RWP 0x018
-#define TMC_TRG 0x01c
-#define TMC_CTL 0x020
-#define TMC_RWD 0x024
-#define TMC_MODE 0x028
-#define TMC_LBUFLEVEL 0x02c
-#define TMC_CBUFLEVEL 0x030
-#define TMC_BUFWM 0x034
-#define TMC_RRPHI 0x038
-#define TMC_RWPHI 0x03c
-#define TMC_AXICTL 0x110
-#define TMC_DBALO 0x118
-#define TMC_DBAHI 0x11c
-#define TMC_FFSR 0x300
-#define TMC_FFCR 0x304
-#define TMC_PSCR 0x308
-#define TMC_ITMISCOP0 0xee0
-#define TMC_ITTRFLIN 0xee8
-#define TMC_ITATBDATA0 0xeec
-#define TMC_ITATBCTR2 0xef0
-#define TMC_ITATBCTR1 0xef4
-#define TMC_ITATBCTR0 0xef8
-
-/* register description */
-/* TMC_CTL - 0x020 */
-#define TMC_CTL_CAPT_EN BIT(0)
-/* TMC_STS - 0x00C */
-#define TMC_STS_TRIGGERED BIT(1)
-/* TMC_AXICTL - 0x110 */
-#define TMC_AXICTL_PROT_CTL_B0 BIT(0)
-#define TMC_AXICTL_PROT_CTL_B1 BIT(1)
-#define TMC_AXICTL_SCT_GAT_MODE BIT(7)
-#define TMC_AXICTL_WR_BURST_LEN 0xF00
-/* TMC_FFCR - 0x304 */
-#define TMC_FFCR_EN_FMT BIT(0)
-#define TMC_FFCR_EN_TI BIT(1)
-#define TMC_FFCR_FON_FLIN BIT(4)
-#define TMC_FFCR_FON_TRIG_EVT BIT(5)
-#define TMC_FFCR_FLUSHMAN BIT(6)
-#define TMC_FFCR_TRIGON_TRIGIN BIT(8)
-#define TMC_FFCR_STOP_ON_FLUSH BIT(12)
-
-#define TMC_STS_TRIGGERED_BIT 2
-#define TMC_FFCR_FLUSHMAN_BIT 6
-
-enum tmc_config_type {
- TMC_CONFIG_TYPE_ETB,
- TMC_CONFIG_TYPE_ETR,
- TMC_CONFIG_TYPE_ETF,
-};
-
-enum tmc_mode {
- TMC_MODE_CIRCULAR_BUFFER,
- TMC_MODE_SOFTWARE_FIFO,
- TMC_MODE_HARDWARE_FIFO,
-};
-
-enum tmc_mem_intf_width {
- TMC_MEM_INTF_WIDTH_32BITS = 0x2,
- TMC_MEM_INTF_WIDTH_64BITS = 0x3,
- TMC_MEM_INTF_WIDTH_128BITS = 0x4,
- TMC_MEM_INTF_WIDTH_256BITS = 0x5,
-};
-
-/**
- * struct tmc_drvdata - specifics associated to an TMC component
- * @base: memory mapped base address for this component.
- * @dev: the device entity associated to this component.
- * @csdev: component vitals needed by the framework.
- * @miscdev: specifics to handle "/dev/xyz.tmc" entry.
- * @spinlock: only one at a time pls.
- * @read_count: manages preparation of buffer for reading.
- * @buf: area of memory where trace data get sent.
- * @paddr: DMA start location in RAM.
- * @vaddr: virtual representation of @paddr.
- * @size: @buf size.
- * @enable: this TMC is being used.
- * @config_type: TMC variant, must be of type @tmc_config_type.
- * @trigger_cntr: amount of words to store after a trigger.
- */
-struct tmc_drvdata {
- void __iomem *base;
- struct device *dev;
- struct coresight_device *csdev;
- struct miscdevice miscdev;
- spinlock_t spinlock;
- int read_count;
- bool reading;
- char *buf;
- dma_addr_t paddr;
- void *vaddr;
- u32 size;
- bool enable;
- enum tmc_config_type config_type;
- u32 trigger_cntr;
-};
-
-static void tmc_wait_for_ready(struct tmc_drvdata *drvdata)
+void tmc_wait_for_tmcready(struct tmc_drvdata *drvdata)
{
/* Ensure formatter, unformatter and hardware fifo are empty */
if (coresight_timeout(drvdata->base,
- TMC_STS, TMC_STS_TRIGGERED_BIT, 1)) {
+ TMC_STS, TMC_STS_TMCREADY_BIT, 1)) {
dev_err(drvdata->dev,
"timeout observed when probing at offset %#x\n",
TMC_STS);
}
}
-static void tmc_flush_and_stop(struct tmc_drvdata *drvdata)
+void tmc_flush_and_stop(struct tmc_drvdata *drvdata)
{
u32 ffcr;
ffcr = readl_relaxed(drvdata->base + TMC_FFCR);
ffcr |= TMC_FFCR_STOP_ON_FLUSH;
writel_relaxed(ffcr, drvdata->base + TMC_FFCR);
- ffcr |= TMC_FFCR_FLUSHMAN;
+ ffcr |= BIT(TMC_FFCR_FLUSHMAN_BIT);
writel_relaxed(ffcr, drvdata->base + TMC_FFCR);
/* Ensure flush completes */
if (coresight_timeout(drvdata->base,
@@ -160,338 +60,73 @@
TMC_FFCR);
}
- tmc_wait_for_ready(drvdata);
+ tmc_wait_for_tmcready(drvdata);
}
-static void tmc_enable_hw(struct tmc_drvdata *drvdata)
+void tmc_enable_hw(struct tmc_drvdata *drvdata)
{
writel_relaxed(TMC_CTL_CAPT_EN, drvdata->base + TMC_CTL);
}
-static void tmc_disable_hw(struct tmc_drvdata *drvdata)
+void tmc_disable_hw(struct tmc_drvdata *drvdata)
{
writel_relaxed(0x0, drvdata->base + TMC_CTL);
}
-static void tmc_etb_enable_hw(struct tmc_drvdata *drvdata)
-{
- /* Zero out the memory to help with debug */
- memset(drvdata->buf, 0, drvdata->size);
-
- CS_UNLOCK(drvdata->base);
-
- writel_relaxed(TMC_MODE_CIRCULAR_BUFFER, drvdata->base + TMC_MODE);
- writel_relaxed(TMC_FFCR_EN_FMT | TMC_FFCR_EN_TI |
- TMC_FFCR_FON_FLIN | TMC_FFCR_FON_TRIG_EVT |
- TMC_FFCR_TRIGON_TRIGIN,
- drvdata->base + TMC_FFCR);
-
- writel_relaxed(drvdata->trigger_cntr, drvdata->base + TMC_TRG);
- tmc_enable_hw(drvdata);
-
- CS_LOCK(drvdata->base);
-}
-
-static void tmc_etr_enable_hw(struct tmc_drvdata *drvdata)
-{
- u32 axictl;
-
- /* Zero out the memory to help with debug */
- memset(drvdata->vaddr, 0, drvdata->size);
-
- CS_UNLOCK(drvdata->base);
-
- writel_relaxed(drvdata->size / 4, drvdata->base + TMC_RSZ);
- writel_relaxed(TMC_MODE_CIRCULAR_BUFFER, drvdata->base + TMC_MODE);
-
- axictl = readl_relaxed(drvdata->base + TMC_AXICTL);
- axictl |= TMC_AXICTL_WR_BURST_LEN;
- writel_relaxed(axictl, drvdata->base + TMC_AXICTL);
- axictl &= ~TMC_AXICTL_SCT_GAT_MODE;
- writel_relaxed(axictl, drvdata->base + TMC_AXICTL);
- axictl = (axictl &
- ~(TMC_AXICTL_PROT_CTL_B0 | TMC_AXICTL_PROT_CTL_B1)) |
- TMC_AXICTL_PROT_CTL_B1;
- writel_relaxed(axictl, drvdata->base + TMC_AXICTL);
-
- writel_relaxed(drvdata->paddr, drvdata->base + TMC_DBALO);
- writel_relaxed(0x0, drvdata->base + TMC_DBAHI);
- writel_relaxed(TMC_FFCR_EN_FMT | TMC_FFCR_EN_TI |
- TMC_FFCR_FON_FLIN | TMC_FFCR_FON_TRIG_EVT |
- TMC_FFCR_TRIGON_TRIGIN,
- drvdata->base + TMC_FFCR);
- writel_relaxed(drvdata->trigger_cntr, drvdata->base + TMC_TRG);
- tmc_enable_hw(drvdata);
-
- CS_LOCK(drvdata->base);
-}
-
-static void tmc_etf_enable_hw(struct tmc_drvdata *drvdata)
-{
- CS_UNLOCK(drvdata->base);
-
- writel_relaxed(TMC_MODE_HARDWARE_FIFO, drvdata->base + TMC_MODE);
- writel_relaxed(TMC_FFCR_EN_FMT | TMC_FFCR_EN_TI,
- drvdata->base + TMC_FFCR);
- writel_relaxed(0x0, drvdata->base + TMC_BUFWM);
- tmc_enable_hw(drvdata);
-
- CS_LOCK(drvdata->base);
-}
-
-static int tmc_enable(struct tmc_drvdata *drvdata, enum tmc_mode mode)
-{
- unsigned long flags;
-
- spin_lock_irqsave(&drvdata->spinlock, flags);
- if (drvdata->reading) {
- spin_unlock_irqrestore(&drvdata->spinlock, flags);
- return -EBUSY;
- }
-
- if (drvdata->config_type == TMC_CONFIG_TYPE_ETB) {
- tmc_etb_enable_hw(drvdata);
- } else if (drvdata->config_type == TMC_CONFIG_TYPE_ETR) {
- tmc_etr_enable_hw(drvdata);
- } else {
- if (mode == TMC_MODE_CIRCULAR_BUFFER)
- tmc_etb_enable_hw(drvdata);
- else
- tmc_etf_enable_hw(drvdata);
- }
- drvdata->enable = true;
- spin_unlock_irqrestore(&drvdata->spinlock, flags);
-
- dev_info(drvdata->dev, "TMC enabled\n");
- return 0;
-}
-
-static int tmc_enable_sink(struct coresight_device *csdev, u32 mode)
-{
- struct tmc_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
-
- return tmc_enable(drvdata, TMC_MODE_CIRCULAR_BUFFER);
-}
-
-static int tmc_enable_link(struct coresight_device *csdev, int inport,
- int outport)
-{
- struct tmc_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
-
- return tmc_enable(drvdata, TMC_MODE_HARDWARE_FIFO);
-}
-
-static void tmc_etb_dump_hw(struct tmc_drvdata *drvdata)
-{
- enum tmc_mem_intf_width memwidth;
- u8 memwords;
- char *bufp;
- u32 read_data;
- int i;
-
- memwidth = BMVAL(readl_relaxed(drvdata->base + CORESIGHT_DEVID), 8, 10);
- if (memwidth == TMC_MEM_INTF_WIDTH_32BITS)
- memwords = 1;
- else if (memwidth == TMC_MEM_INTF_WIDTH_64BITS)
- memwords = 2;
- else if (memwidth == TMC_MEM_INTF_WIDTH_128BITS)
- memwords = 4;
- else
- memwords = 8;
-
- bufp = drvdata->buf;
- while (1) {
- for (i = 0; i < memwords; i++) {
- read_data = readl_relaxed(drvdata->base + TMC_RRD);
- if (read_data == 0xFFFFFFFF)
- return;
- memcpy(bufp, &read_data, 4);
- bufp += 4;
- }
- }
-}
-
-static void tmc_etb_disable_hw(struct tmc_drvdata *drvdata)
-{
- CS_UNLOCK(drvdata->base);
-
- tmc_flush_and_stop(drvdata);
- tmc_etb_dump_hw(drvdata);
- tmc_disable_hw(drvdata);
-
- CS_LOCK(drvdata->base);
-}
-
-static void tmc_etr_dump_hw(struct tmc_drvdata *drvdata)
-{
- u32 rwp, val;
-
- rwp = readl_relaxed(drvdata->base + TMC_RWP);
- val = readl_relaxed(drvdata->base + TMC_STS);
-
- /* How much memory do we still have */
- if (val & BIT(0))
- drvdata->buf = drvdata->vaddr + rwp - drvdata->paddr;
- else
- drvdata->buf = drvdata->vaddr;
-}
-
-static void tmc_etr_disable_hw(struct tmc_drvdata *drvdata)
-{
- CS_UNLOCK(drvdata->base);
-
- tmc_flush_and_stop(drvdata);
- tmc_etr_dump_hw(drvdata);
- tmc_disable_hw(drvdata);
-
- CS_LOCK(drvdata->base);
-}
-
-static void tmc_etf_disable_hw(struct tmc_drvdata *drvdata)
-{
- CS_UNLOCK(drvdata->base);
-
- tmc_flush_and_stop(drvdata);
- tmc_disable_hw(drvdata);
-
- CS_LOCK(drvdata->base);
-}
-
-static void tmc_disable(struct tmc_drvdata *drvdata, enum tmc_mode mode)
-{
- unsigned long flags;
-
- spin_lock_irqsave(&drvdata->spinlock, flags);
- if (drvdata->reading)
- goto out;
-
- if (drvdata->config_type == TMC_CONFIG_TYPE_ETB) {
- tmc_etb_disable_hw(drvdata);
- } else if (drvdata->config_type == TMC_CONFIG_TYPE_ETR) {
- tmc_etr_disable_hw(drvdata);
- } else {
- if (mode == TMC_MODE_CIRCULAR_BUFFER)
- tmc_etb_disable_hw(drvdata);
- else
- tmc_etf_disable_hw(drvdata);
- }
-out:
- drvdata->enable = false;
- spin_unlock_irqrestore(&drvdata->spinlock, flags);
-
- dev_info(drvdata->dev, "TMC disabled\n");
-}
-
-static void tmc_disable_sink(struct coresight_device *csdev)
-{
- struct tmc_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
-
- tmc_disable(drvdata, TMC_MODE_CIRCULAR_BUFFER);
-}
-
-static void tmc_disable_link(struct coresight_device *csdev, int inport,
- int outport)
-{
- struct tmc_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
-
- tmc_disable(drvdata, TMC_MODE_HARDWARE_FIFO);
-}
-
-static const struct coresight_ops_sink tmc_sink_ops = {
- .enable = tmc_enable_sink,
- .disable = tmc_disable_sink,
-};
-
-static const struct coresight_ops_link tmc_link_ops = {
- .enable = tmc_enable_link,
- .disable = tmc_disable_link,
-};
-
-static const struct coresight_ops tmc_etb_cs_ops = {
- .sink_ops = &tmc_sink_ops,
-};
-
-static const struct coresight_ops tmc_etr_cs_ops = {
- .sink_ops = &tmc_sink_ops,
-};
-
-static const struct coresight_ops tmc_etf_cs_ops = {
- .sink_ops = &tmc_sink_ops,
- .link_ops = &tmc_link_ops,
-};
-
static int tmc_read_prepare(struct tmc_drvdata *drvdata)
{
- int ret;
- unsigned long flags;
- enum tmc_mode mode;
+ int ret = 0;
- spin_lock_irqsave(&drvdata->spinlock, flags);
- if (!drvdata->enable)
- goto out;
-
- if (drvdata->config_type == TMC_CONFIG_TYPE_ETB) {
- tmc_etb_disable_hw(drvdata);
- } else if (drvdata->config_type == TMC_CONFIG_TYPE_ETR) {
- tmc_etr_disable_hw(drvdata);
- } else {
- mode = readl_relaxed(drvdata->base + TMC_MODE);
- if (mode == TMC_MODE_CIRCULAR_BUFFER) {
- tmc_etb_disable_hw(drvdata);
- } else {
- ret = -ENODEV;
- goto err;
- }
+ switch (drvdata->config_type) {
+ case TMC_CONFIG_TYPE_ETB:
+ case TMC_CONFIG_TYPE_ETF:
+ ret = tmc_read_prepare_etb(drvdata);
+ break;
+ case TMC_CONFIG_TYPE_ETR:
+ ret = tmc_read_prepare_etr(drvdata);
+ break;
+ default:
+ ret = -EINVAL;
}
-out:
- drvdata->reading = true;
- spin_unlock_irqrestore(&drvdata->spinlock, flags);
- dev_info(drvdata->dev, "TMC read start\n");
- return 0;
-err:
- spin_unlock_irqrestore(&drvdata->spinlock, flags);
+ if (!ret)
+ dev_info(drvdata->dev, "TMC read start\n");
+
return ret;
}
-static void tmc_read_unprepare(struct tmc_drvdata *drvdata)
+static int tmc_read_unprepare(struct tmc_drvdata *drvdata)
{
- unsigned long flags;
- enum tmc_mode mode;
+ int ret = 0;
- spin_lock_irqsave(&drvdata->spinlock, flags);
- if (!drvdata->enable)
- goto out;
-
- if (drvdata->config_type == TMC_CONFIG_TYPE_ETB) {
- tmc_etb_enable_hw(drvdata);
- } else if (drvdata->config_type == TMC_CONFIG_TYPE_ETR) {
- tmc_etr_enable_hw(drvdata);
- } else {
- mode = readl_relaxed(drvdata->base + TMC_MODE);
- if (mode == TMC_MODE_CIRCULAR_BUFFER)
- tmc_etb_enable_hw(drvdata);
+ switch (drvdata->config_type) {
+ case TMC_CONFIG_TYPE_ETB:
+ case TMC_CONFIG_TYPE_ETF:
+ ret = tmc_read_unprepare_etb(drvdata);
+ break;
+ case TMC_CONFIG_TYPE_ETR:
+ ret = tmc_read_unprepare_etr(drvdata);
+ break;
+ default:
+ ret = -EINVAL;
}
-out:
- drvdata->reading = false;
- spin_unlock_irqrestore(&drvdata->spinlock, flags);
- dev_info(drvdata->dev, "TMC read end\n");
+ if (!ret)
+ dev_info(drvdata->dev, "TMC read end\n");
+
+ return ret;
}
static int tmc_open(struct inode *inode, struct file *file)
{
+ int ret;
struct tmc_drvdata *drvdata = container_of(file->private_data,
struct tmc_drvdata, miscdev);
- int ret = 0;
-
- if (drvdata->read_count++)
- goto out;
ret = tmc_read_prepare(drvdata);
if (ret)
return ret;
-out:
+
nonseekable_open(inode, file);
dev_dbg(drvdata->dev, "%s: successfully opened\n", __func__);
@@ -531,19 +166,14 @@
static int tmc_release(struct inode *inode, struct file *file)
{
+ int ret;
struct tmc_drvdata *drvdata = container_of(file->private_data,
struct tmc_drvdata, miscdev);
- if (--drvdata->read_count) {
- if (drvdata->read_count < 0) {
- dev_err(drvdata->dev, "mismatched close\n");
- drvdata->read_count = 0;
- }
- goto out;
- }
+ ret = tmc_read_unprepare(drvdata);
+ if (ret)
+ return ret;
- tmc_read_unprepare(drvdata);
-out:
dev_dbg(drvdata->dev, "%s: released\n", __func__);
return 0;
}
@@ -556,56 +186,71 @@
.llseek = no_llseek,
};
-static ssize_t status_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+static enum tmc_mem_intf_width tmc_get_memwidth(u32 devid)
{
- unsigned long flags;
- u32 tmc_rsz, tmc_sts, tmc_rrp, tmc_rwp, tmc_trg;
- u32 tmc_ctl, tmc_ffsr, tmc_ffcr, tmc_mode, tmc_pscr;
- u32 devid;
- struct tmc_drvdata *drvdata = dev_get_drvdata(dev->parent);
+ enum tmc_mem_intf_width memwidth;
- pm_runtime_get_sync(drvdata->dev);
- spin_lock_irqsave(&drvdata->spinlock, flags);
- CS_UNLOCK(drvdata->base);
+ /*
+ * Excerpt from the TRM:
+ *
+ * DEVID::MEMWIDTH[10:8]
+ * 0x2 Memory interface databus is 32 bits wide.
+ * 0x3 Memory interface databus is 64 bits wide.
+ * 0x4 Memory interface databus is 128 bits wide.
+ * 0x5 Memory interface databus is 256 bits wide.
+ */
+ switch (BMVAL(devid, 8, 10)) {
+ case 0x2:
+ memwidth = TMC_MEM_INTF_WIDTH_32BITS;
+ break;
+ case 0x3:
+ memwidth = TMC_MEM_INTF_WIDTH_64BITS;
+ break;
+ case 0x4:
+ memwidth = TMC_MEM_INTF_WIDTH_128BITS;
+ break;
+ case 0x5:
+ memwidth = TMC_MEM_INTF_WIDTH_256BITS;
+ break;
+ default:
+ memwidth = 0;
+ }
- tmc_rsz = readl_relaxed(drvdata->base + TMC_RSZ);
- tmc_sts = readl_relaxed(drvdata->base + TMC_STS);
- tmc_rrp = readl_relaxed(drvdata->base + TMC_RRP);
- tmc_rwp = readl_relaxed(drvdata->base + TMC_RWP);
- tmc_trg = readl_relaxed(drvdata->base + TMC_TRG);
- tmc_ctl = readl_relaxed(drvdata->base + TMC_CTL);
- tmc_ffsr = readl_relaxed(drvdata->base + TMC_FFSR);
- tmc_ffcr = readl_relaxed(drvdata->base + TMC_FFCR);
- tmc_mode = readl_relaxed(drvdata->base + TMC_MODE);
- tmc_pscr = readl_relaxed(drvdata->base + TMC_PSCR);
- devid = readl_relaxed(drvdata->base + CORESIGHT_DEVID);
-
- CS_LOCK(drvdata->base);
- spin_unlock_irqrestore(&drvdata->spinlock, flags);
- pm_runtime_put(drvdata->dev);
-
- return sprintf(buf,
- "Depth:\t\t0x%x\n"
- "Status:\t\t0x%x\n"
- "RAM read ptr:\t0x%x\n"
- "RAM wrt ptr:\t0x%x\n"
- "Trigger cnt:\t0x%x\n"
- "Control:\t0x%x\n"
- "Flush status:\t0x%x\n"
- "Flush ctrl:\t0x%x\n"
- "Mode:\t\t0x%x\n"
- "PSRC:\t\t0x%x\n"
- "DEVID:\t\t0x%x\n",
- tmc_rsz, tmc_sts, tmc_rrp, tmc_rwp, tmc_trg,
- tmc_ctl, tmc_ffsr, tmc_ffcr, tmc_mode, tmc_pscr, devid);
-
- return -EINVAL;
+ return memwidth;
}
-static DEVICE_ATTR_RO(status);
-static ssize_t trigger_cntr_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+#define coresight_tmc_simple_func(name, offset) \
+ coresight_simple_func(struct tmc_drvdata, name, offset)
+
+coresight_tmc_simple_func(rsz, TMC_RSZ);
+coresight_tmc_simple_func(sts, TMC_STS);
+coresight_tmc_simple_func(rrp, TMC_RRP);
+coresight_tmc_simple_func(rwp, TMC_RWP);
+coresight_tmc_simple_func(trg, TMC_TRG);
+coresight_tmc_simple_func(ctl, TMC_CTL);
+coresight_tmc_simple_func(ffsr, TMC_FFSR);
+coresight_tmc_simple_func(ffcr, TMC_FFCR);
+coresight_tmc_simple_func(mode, TMC_MODE);
+coresight_tmc_simple_func(pscr, TMC_PSCR);
+coresight_tmc_simple_func(devid, CORESIGHT_DEVID);
+
+static struct attribute *coresight_tmc_mgmt_attrs[] = {
+ &dev_attr_rsz.attr,
+ &dev_attr_sts.attr,
+ &dev_attr_rrp.attr,
+ &dev_attr_rwp.attr,
+ &dev_attr_trg.attr,
+ &dev_attr_ctl.attr,
+ &dev_attr_ffsr.attr,
+ &dev_attr_ffcr.attr,
+ &dev_attr_mode.attr,
+ &dev_attr_pscr.attr,
+ &dev_attr_devid.attr,
+ NULL,
+};
+
+ssize_t trigger_cntr_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
{
struct tmc_drvdata *drvdata = dev_get_drvdata(dev->parent);
unsigned long val = drvdata->trigger_cntr;
@@ -630,26 +275,25 @@
}
static DEVICE_ATTR_RW(trigger_cntr);
-static struct attribute *coresight_etb_attrs[] = {
+static struct attribute *coresight_tmc_attrs[] = {
&dev_attr_trigger_cntr.attr,
- &dev_attr_status.attr,
NULL,
};
-ATTRIBUTE_GROUPS(coresight_etb);
-static struct attribute *coresight_etr_attrs[] = {
- &dev_attr_trigger_cntr.attr,
- &dev_attr_status.attr,
- NULL,
+static const struct attribute_group coresight_tmc_group = {
+ .attrs = coresight_tmc_attrs,
};
-ATTRIBUTE_GROUPS(coresight_etr);
-static struct attribute *coresight_etf_attrs[] = {
- &dev_attr_trigger_cntr.attr,
- &dev_attr_status.attr,
+static const struct attribute_group coresight_tmc_mgmt_group = {
+ .attrs = coresight_tmc_mgmt_attrs,
+ .name = "mgmt",
+};
+
+const struct attribute_group *coresight_tmc_groups[] = {
+ &coresight_tmc_group,
+ &coresight_tmc_mgmt_group,
NULL,
};
-ATTRIBUTE_GROUPS(coresight_etf);
static int tmc_probe(struct amba_device *adev, const struct amba_id *id)
{
@@ -688,6 +332,7 @@
devid = readl_relaxed(drvdata->base + CORESIGHT_DEVID);
drvdata->config_type = BMVAL(devid, 6, 7);
+ drvdata->memwidth = tmc_get_memwidth(devid);
if (drvdata->config_type == TMC_CONFIG_TYPE_ETR) {
if (np)
@@ -702,20 +347,6 @@
pm_runtime_put(&adev->dev);
- if (drvdata->config_type == TMC_CONFIG_TYPE_ETR) {
- drvdata->vaddr = dma_alloc_coherent(dev, drvdata->size,
- &drvdata->paddr, GFP_KERNEL);
- if (!drvdata->vaddr)
- return -ENOMEM;
-
- memset(drvdata->vaddr, 0, drvdata->size);
- drvdata->buf = drvdata->vaddr;
- } else {
- drvdata->buf = devm_kzalloc(dev, drvdata->size, GFP_KERNEL);
- if (!drvdata->buf)
- return -ENOMEM;
- }
-
desc = devm_kzalloc(dev, sizeof(*desc), GFP_KERNEL);
if (!desc) {
ret = -ENOMEM;
@@ -725,20 +356,18 @@
desc->pdata = pdata;
desc->dev = dev;
desc->subtype.sink_subtype = CORESIGHT_DEV_SUBTYPE_SINK_BUFFER;
+ desc->groups = coresight_tmc_groups;
if (drvdata->config_type == TMC_CONFIG_TYPE_ETB) {
desc->type = CORESIGHT_DEV_TYPE_SINK;
desc->ops = &tmc_etb_cs_ops;
- desc->groups = coresight_etb_groups;
} else if (drvdata->config_type == TMC_CONFIG_TYPE_ETR) {
desc->type = CORESIGHT_DEV_TYPE_SINK;
desc->ops = &tmc_etr_cs_ops;
- desc->groups = coresight_etr_groups;
} else {
desc->type = CORESIGHT_DEV_TYPE_LINKSINK;
desc->subtype.link_subtype = CORESIGHT_DEV_SUBTYPE_LINK_FIFO;
desc->ops = &tmc_etf_cs_ops;
- desc->groups = coresight_etf_groups;
}
drvdata->csdev = coresight_register(desc);
@@ -754,7 +383,6 @@
if (ret)
goto err_misc_register;
- dev_info(dev, "TMC initialized\n");
return 0;
err_misc_register:
diff --git a/drivers/hwtracing/coresight/coresight-tmc.h b/drivers/hwtracing/coresight/coresight-tmc.h
new file mode 100644
index 0000000..5c5fe2a
--- /dev/null
+++ b/drivers/hwtracing/coresight/coresight-tmc.h
@@ -0,0 +1,140 @@
+/*
+ * Copyright(C) 2015 Linaro Limited. All rights reserved.
+ * Author: Mathieu Poirier <mathieu.poirier@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef _CORESIGHT_TMC_H
+#define _CORESIGHT_TMC_H
+
+#include <linux/miscdevice.h>
+
+#define TMC_RSZ 0x004
+#define TMC_STS 0x00c
+#define TMC_RRD 0x010
+#define TMC_RRP 0x014
+#define TMC_RWP 0x018
+#define TMC_TRG 0x01c
+#define TMC_CTL 0x020
+#define TMC_RWD 0x024
+#define TMC_MODE 0x028
+#define TMC_LBUFLEVEL 0x02c
+#define TMC_CBUFLEVEL 0x030
+#define TMC_BUFWM 0x034
+#define TMC_RRPHI 0x038
+#define TMC_RWPHI 0x03c
+#define TMC_AXICTL 0x110
+#define TMC_DBALO 0x118
+#define TMC_DBAHI 0x11c
+#define TMC_FFSR 0x300
+#define TMC_FFCR 0x304
+#define TMC_PSCR 0x308
+#define TMC_ITMISCOP0 0xee0
+#define TMC_ITTRFLIN 0xee8
+#define TMC_ITATBDATA0 0xeec
+#define TMC_ITATBCTR2 0xef0
+#define TMC_ITATBCTR1 0xef4
+#define TMC_ITATBCTR0 0xef8
+
+/* register description */
+/* TMC_CTL - 0x020 */
+#define TMC_CTL_CAPT_EN BIT(0)
+/* TMC_STS - 0x00C */
+#define TMC_STS_TMCREADY_BIT 2
+#define TMC_STS_FULL BIT(0)
+#define TMC_STS_TRIGGERED BIT(1)
+/* TMC_AXICTL - 0x110 */
+#define TMC_AXICTL_PROT_CTL_B0 BIT(0)
+#define TMC_AXICTL_PROT_CTL_B1 BIT(1)
+#define TMC_AXICTL_SCT_GAT_MODE BIT(7)
+#define TMC_AXICTL_WR_BURST_16 0xF00
+/* TMC_FFCR - 0x304 */
+#define TMC_FFCR_FLUSHMAN_BIT 6
+#define TMC_FFCR_EN_FMT BIT(0)
+#define TMC_FFCR_EN_TI BIT(1)
+#define TMC_FFCR_FON_FLIN BIT(4)
+#define TMC_FFCR_FON_TRIG_EVT BIT(5)
+#define TMC_FFCR_TRIGON_TRIGIN BIT(8)
+#define TMC_FFCR_STOP_ON_FLUSH BIT(12)
+
+
+enum tmc_config_type {
+ TMC_CONFIG_TYPE_ETB,
+ TMC_CONFIG_TYPE_ETR,
+ TMC_CONFIG_TYPE_ETF,
+};
+
+enum tmc_mode {
+ TMC_MODE_CIRCULAR_BUFFER,
+ TMC_MODE_SOFTWARE_FIFO,
+ TMC_MODE_HARDWARE_FIFO,
+};
+
+enum tmc_mem_intf_width {
+ TMC_MEM_INTF_WIDTH_32BITS = 1,
+ TMC_MEM_INTF_WIDTH_64BITS = 2,
+ TMC_MEM_INTF_WIDTH_128BITS = 4,
+ TMC_MEM_INTF_WIDTH_256BITS = 8,
+};
+
+/**
+ * struct tmc_drvdata - specifics associated to an TMC component
+ * @base: memory mapped base address for this component.
+ * @dev: the device entity associated to this component.
+ * @csdev: component vitals needed by the framework.
+ * @miscdev: specifics to handle "/dev/xyz.tmc" entry.
+ * @spinlock: only one at a time pls.
+ * @buf: area of memory where trace data get sent.
+ * @paddr: DMA start location in RAM.
+ * @vaddr: virtual representation of @paddr.
+ * @size: @buf size.
+ * @mode: how this TMC is being used.
+ * @config_type: TMC variant, must be of type @tmc_config_type.
+ * @memwidth: width of the memory interface databus, in bytes.
+ * @trigger_cntr: amount of words to store after a trigger.
+ */
+struct tmc_drvdata {
+ void __iomem *base;
+ struct device *dev;
+ struct coresight_device *csdev;
+ struct miscdevice miscdev;
+ spinlock_t spinlock;
+ bool reading;
+ char *buf;
+ dma_addr_t paddr;
+ void __iomem *vaddr;
+ u32 size;
+ local_t mode;
+ enum tmc_config_type config_type;
+ enum tmc_mem_intf_width memwidth;
+ u32 trigger_cntr;
+};
+
+/* Generic functions */
+void tmc_wait_for_tmcready(struct tmc_drvdata *drvdata);
+void tmc_flush_and_stop(struct tmc_drvdata *drvdata);
+void tmc_enable_hw(struct tmc_drvdata *drvdata);
+void tmc_disable_hw(struct tmc_drvdata *drvdata);
+
+/* ETB/ETF functions */
+int tmc_read_prepare_etb(struct tmc_drvdata *drvdata);
+int tmc_read_unprepare_etb(struct tmc_drvdata *drvdata);
+extern const struct coresight_ops tmc_etb_cs_ops;
+extern const struct coresight_ops tmc_etf_cs_ops;
+
+/* ETR functions */
+int tmc_read_prepare_etr(struct tmc_drvdata *drvdata);
+int tmc_read_unprepare_etr(struct tmc_drvdata *drvdata);
+extern const struct coresight_ops tmc_etr_cs_ops;
+#endif
diff --git a/drivers/hwtracing/coresight/coresight-tpiu.c b/drivers/hwtracing/coresight/coresight-tpiu.c
index 8fb09d9..4e471e2 100644
--- a/drivers/hwtracing/coresight/coresight-tpiu.c
+++ b/drivers/hwtracing/coresight/coresight-tpiu.c
@@ -167,7 +167,6 @@
if (IS_ERR(drvdata->csdev))
return PTR_ERR(drvdata->csdev);
- dev_info(dev, "TPIU initialized\n");
return 0;
}
diff --git a/drivers/hwtracing/coresight/coresight.c b/drivers/hwtracing/coresight/coresight.c
index 2ea5961..5443d03 100644
--- a/drivers/hwtracing/coresight/coresight.c
+++ b/drivers/hwtracing/coresight/coresight.c
@@ -43,7 +43,15 @@
* When operating Coresight drivers from the sysFS interface, only a single
* path can exist from a tracer (associated to a CPU) to a sink.
*/
-static DEFINE_PER_CPU(struct list_head *, sysfs_path);
+static DEFINE_PER_CPU(struct list_head *, tracer_path);
+
+/*
+ * As of this writing only a single STM can be found in CS topologies. Since
+ * there is no way to know if we'll ever see more and what kind of
+ * configuration they will enact, for the time being only define a single path
+ * for STM.
+ */
+static struct list_head *stm_path;
static int coresight_id_match(struct device *dev, void *data)
{
@@ -257,15 +265,27 @@
void coresight_disable_path(struct list_head *path)
{
+ u32 type;
struct coresight_node *nd;
struct coresight_device *csdev, *parent, *child;
list_for_each_entry(nd, path, link) {
csdev = nd->csdev;
+ type = csdev->type;
- switch (csdev->type) {
+ /*
+ * ETF devices are tricky... They can be a link or a sink,
+ * depending on how they are configured. If an ETF has been
+ * "activated" it will be configured as a sink, otherwise
+ * go ahead with the link configuration.
+ */
+ if (type == CORESIGHT_DEV_TYPE_LINKSINK)
+ type = (csdev == coresight_get_sink(path)) ?
+ CORESIGHT_DEV_TYPE_SINK :
+ CORESIGHT_DEV_TYPE_LINK;
+
+ switch (type) {
case CORESIGHT_DEV_TYPE_SINK:
- case CORESIGHT_DEV_TYPE_LINKSINK:
coresight_disable_sink(csdev);
break;
case CORESIGHT_DEV_TYPE_SOURCE:
@@ -286,15 +306,27 @@
{
int ret = 0;
+ u32 type;
struct coresight_node *nd;
struct coresight_device *csdev, *parent, *child;
list_for_each_entry_reverse(nd, path, link) {
csdev = nd->csdev;
+ type = csdev->type;
- switch (csdev->type) {
+ /*
+ * ETF devices are tricky... They can be a link or a sink,
+ * depending on how they are configured. If an ETF has been
+ * "activated" it will be configured as a sink, otherwise
+ * go ahead with the link configuration.
+ */
+ if (type == CORESIGHT_DEV_TYPE_LINKSINK)
+ type = (csdev == coresight_get_sink(path)) ?
+ CORESIGHT_DEV_TYPE_SINK :
+ CORESIGHT_DEV_TYPE_LINK;
+
+ switch (type) {
case CORESIGHT_DEV_TYPE_SINK:
- case CORESIGHT_DEV_TYPE_LINKSINK:
ret = coresight_enable_sink(csdev, mode);
if (ret)
goto err;
@@ -432,18 +464,45 @@
path = NULL;
}
+/** coresight_validate_source - make sure a source has the right credentials
+ * @csdev: the device structure for a source.
+ * @function: the function this was called from.
+ *
+ * Assumes the coresight_mutex is held.
+ */
+static int coresight_validate_source(struct coresight_device *csdev,
+ const char *function)
+{
+ u32 type, subtype;
+
+ type = csdev->type;
+ subtype = csdev->subtype.source_subtype;
+
+ if (type != CORESIGHT_DEV_TYPE_SOURCE) {
+ dev_err(&csdev->dev, "wrong device type in %s\n", function);
+ return -EINVAL;
+ }
+
+ if (subtype != CORESIGHT_DEV_SUBTYPE_SOURCE_PROC &&
+ subtype != CORESIGHT_DEV_SUBTYPE_SOURCE_SOFTWARE) {
+ dev_err(&csdev->dev, "wrong device subtype in %s\n", function);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
int coresight_enable(struct coresight_device *csdev)
{
- int ret = 0;
- int cpu;
+ int cpu, ret = 0;
struct list_head *path;
mutex_lock(&coresight_mutex);
- if (csdev->type != CORESIGHT_DEV_TYPE_SOURCE) {
- ret = -EINVAL;
- dev_err(&csdev->dev, "wrong device type in %s\n", __func__);
+
+ ret = coresight_validate_source(csdev, __func__);
+ if (ret)
goto out;
- }
+
if (csdev->enable)
goto out;
@@ -461,15 +520,25 @@
if (ret)
goto err_source;
- /*
- * When working from sysFS it is important to keep track
- * of the paths that were created so that they can be
- * undone in 'coresight_disable()'. Since there can only
- * be a single session per tracer (when working from sysFS)
- * a per-cpu variable will do just fine.
- */
- cpu = source_ops(csdev)->cpu_id(csdev);
- per_cpu(sysfs_path, cpu) = path;
+ switch (csdev->subtype.source_subtype) {
+ case CORESIGHT_DEV_SUBTYPE_SOURCE_PROC:
+ /*
+ * When working from sysFS it is important to keep track
+ * of the paths that were created so that they can be
+ * undone in 'coresight_disable()'. Since there can only
+ * be a single session per tracer (when working from sysFS)
+ * a per-cpu variable will do just fine.
+ */
+ cpu = source_ops(csdev)->cpu_id(csdev);
+ per_cpu(tracer_path, cpu) = path;
+ break;
+ case CORESIGHT_DEV_SUBTYPE_SOURCE_SOFTWARE:
+ stm_path = path;
+ break;
+ default:
+ /* We can't be here */
+ break;
+ }
out:
mutex_unlock(&coresight_mutex);
@@ -486,23 +555,36 @@
void coresight_disable(struct coresight_device *csdev)
{
- int cpu;
- struct list_head *path;
+ int cpu, ret;
+ struct list_head *path = NULL;
mutex_lock(&coresight_mutex);
- if (csdev->type != CORESIGHT_DEV_TYPE_SOURCE) {
- dev_err(&csdev->dev, "wrong device type in %s\n", __func__);
+
+ ret = coresight_validate_source(csdev, __func__);
+ if (ret)
goto out;
- }
+
if (!csdev->enable)
goto out;
- cpu = source_ops(csdev)->cpu_id(csdev);
- path = per_cpu(sysfs_path, cpu);
+ switch (csdev->subtype.source_subtype) {
+ case CORESIGHT_DEV_SUBTYPE_SOURCE_PROC:
+ cpu = source_ops(csdev)->cpu_id(csdev);
+ path = per_cpu(tracer_path, cpu);
+ per_cpu(tracer_path, cpu) = NULL;
+ break;
+ case CORESIGHT_DEV_SUBTYPE_SOURCE_SOFTWARE:
+ path = stm_path;
+ stm_path = NULL;
+ break;
+ default:
+ /* We can't be here */
+ break;
+ }
+
coresight_disable_source(csdev);
coresight_disable_path(path);
coresight_release_path(path);
- per_cpu(sysfs_path, cpu) = NULL;
out:
mutex_unlock(&coresight_mutex);
@@ -514,7 +596,7 @@
{
struct coresight_device *csdev = to_coresight_device(dev);
- return scnprintf(buf, PAGE_SIZE, "%u\n", (unsigned)csdev->activated);
+ return scnprintf(buf, PAGE_SIZE, "%u\n", csdev->activated);
}
static ssize_t enable_sink_store(struct device *dev,
@@ -544,7 +626,7 @@
{
struct coresight_device *csdev = to_coresight_device(dev);
- return scnprintf(buf, PAGE_SIZE, "%u\n", (unsigned)csdev->enable);
+ return scnprintf(buf, PAGE_SIZE, "%u\n", csdev->enable);
}
static ssize_t enable_source_store(struct device *dev,
diff --git a/drivers/hwtracing/intel_th/core.c b/drivers/hwtracing/intel_th/core.c
index 4272f2c..1be543e 100644
--- a/drivers/hwtracing/intel_th/core.c
+++ b/drivers/hwtracing/intel_th/core.c
@@ -71,6 +71,15 @@
if (ret)
return ret;
+ if (thdrv->attr_group) {
+ ret = sysfs_create_group(&thdev->dev.kobj, thdrv->attr_group);
+ if (ret) {
+ thdrv->remove(thdev);
+
+ return ret;
+ }
+ }
+
if (thdev->type == INTEL_TH_OUTPUT &&
!intel_th_output_assigned(thdev))
ret = hubdrv->assign(hub, thdev);
@@ -91,6 +100,9 @@
return err;
}
+ if (thdrv->attr_group)
+ sysfs_remove_group(&thdev->dev.kobj, thdrv->attr_group);
+
thdrv->remove(thdev);
if (intel_th_output_assigned(thdev)) {
@@ -171,7 +183,14 @@
static int intel_th_output_activate(struct intel_th_device *thdev)
{
- struct intel_th_driver *thdrv = to_intel_th_driver(thdev->dev.driver);
+ struct intel_th_driver *thdrv =
+ to_intel_th_driver_or_null(thdev->dev.driver);
+
+ if (!thdrv)
+ return -ENODEV;
+
+ if (!try_module_get(thdrv->driver.owner))
+ return -ENODEV;
if (thdrv->activate)
return thdrv->activate(thdev);
@@ -183,12 +202,18 @@
static void intel_th_output_deactivate(struct intel_th_device *thdev)
{
- struct intel_th_driver *thdrv = to_intel_th_driver(thdev->dev.driver);
+ struct intel_th_driver *thdrv =
+ to_intel_th_driver_or_null(thdev->dev.driver);
+
+ if (!thdrv)
+ return;
if (thdrv->deactivate)
thdrv->deactivate(thdev);
else
intel_th_trace_disable(thdev);
+
+ module_put(thdrv->driver.owner);
}
static ssize_t active_show(struct device *dev, struct device_attribute *attr,
diff --git a/drivers/hwtracing/intel_th/intel_th.h b/drivers/hwtracing/intel_th/intel_th.h
index eedd093..0df22e3 100644
--- a/drivers/hwtracing/intel_th/intel_th.h
+++ b/drivers/hwtracing/intel_th/intel_th.h
@@ -115,6 +115,7 @@
* @enable: enable tracing for a given output device
* @disable: disable tracing for a given output device
* @fops: file operations for device nodes
+ * @attr_group: attributes provided by the driver
*
* Callbacks @probe and @remove are required for all device types.
* Switch device driver needs to fill in @assign, @enable and @disable
@@ -139,6 +140,8 @@
void (*deactivate)(struct intel_th_device *thdev);
/* file_operations for those who want a device node */
const struct file_operations *fops;
+ /* optional attributes */
+ struct attribute_group *attr_group;
/* source ops */
int (*set_output)(struct intel_th_device *thdev,
@@ -148,6 +151,9 @@
#define to_intel_th_driver(_d) \
container_of((_d), struct intel_th_driver, driver)
+#define to_intel_th_driver_or_null(_d) \
+ ((_d) ? to_intel_th_driver(_d) : NULL)
+
static inline struct intel_th_device *
to_intel_th_hub(struct intel_th_device *thdev)
{
diff --git a/drivers/hwtracing/intel_th/msu.c b/drivers/hwtracing/intel_th/msu.c
index d9d6022..e8d55a1 100644
--- a/drivers/hwtracing/intel_th/msu.c
+++ b/drivers/hwtracing/intel_th/msu.c
@@ -122,7 +122,6 @@
atomic_t mmap_count;
struct mutex buf_mutex;
- struct mutex iter_mutex;
struct list_head iter_list;
/* config */
@@ -257,23 +256,37 @@
iter = kzalloc(sizeof(*iter), GFP_KERNEL);
if (!iter)
- return NULL;
+ return ERR_PTR(-ENOMEM);
+
+ mutex_lock(&msc->buf_mutex);
+
+ /*
+ * Reading and tracing are mutually exclusive; if msc is
+ * enabled, open() will fail; otherwise existing readers
+ * will prevent enabling the msc and the rest of fops don't
+ * need to worry about it.
+ */
+ if (msc->enabled) {
+ kfree(iter);
+ iter = ERR_PTR(-EBUSY);
+ goto unlock;
+ }
msc_iter_init(iter);
iter->msc = msc;
- mutex_lock(&msc->iter_mutex);
list_add_tail(&iter->entry, &msc->iter_list);
- mutex_unlock(&msc->iter_mutex);
+unlock:
+ mutex_unlock(&msc->buf_mutex);
return iter;
}
static void msc_iter_remove(struct msc_iter *iter, struct msc *msc)
{
- mutex_lock(&msc->iter_mutex);
+ mutex_lock(&msc->buf_mutex);
list_del(&iter->entry);
- mutex_unlock(&msc->iter_mutex);
+ mutex_unlock(&msc->buf_mutex);
kfree(iter);
}
@@ -454,7 +467,6 @@
{
struct msc_window *win;
- mutex_lock(&msc->buf_mutex);
list_for_each_entry(win, &msc->win_list, entry) {
unsigned int blk;
size_t hw_sz = sizeof(struct msc_block_desc) -
@@ -466,7 +478,6 @@
memset(&bdesc->hw_tag, 0, hw_sz);
}
}
- mutex_unlock(&msc->buf_mutex);
}
/**
@@ -474,12 +485,15 @@
* @msc: the MSC device to configure
*
* Program storage mode, wrapping, burst length and trace buffer address
- * into a given MSC. If msc::enabled is set, enable the trace, too.
+ * into a given MSC. Then, enable tracing and set msc::enabled.
+ * The latter is serialized on msc::buf_mutex, so make sure to hold it.
*/
static int msc_configure(struct msc *msc)
{
u32 reg;
+ lockdep_assert_held(&msc->buf_mutex);
+
if (msc->mode > MSC_MODE_MULTI)
return -ENOTSUPP;
@@ -497,21 +511,19 @@
reg = ioread32(msc->reg_base + REG_MSU_MSC0CTL);
reg &= ~(MSC_MODE | MSC_WRAPEN | MSC_EN | MSC_RD_HDR_OVRD);
+ reg |= MSC_EN;
reg |= msc->mode << __ffs(MSC_MODE);
reg |= msc->burst_len << __ffs(MSC_LEN);
- /*if (msc->mode == MSC_MODE_MULTI)
- reg |= MSC_RD_HDR_OVRD; */
+
if (msc->wrap)
reg |= MSC_WRAPEN;
- if (msc->enabled)
- reg |= MSC_EN;
iowrite32(reg, msc->reg_base + REG_MSU_MSC0CTL);
- if (msc->enabled) {
- msc->thdev->output.multiblock = msc->mode == MSC_MODE_MULTI;
- intel_th_trace_enable(msc->thdev);
- }
+ msc->thdev->output.multiblock = msc->mode == MSC_MODE_MULTI;
+ intel_th_trace_enable(msc->thdev);
+ msc->enabled = 1;
+
return 0;
}
@@ -521,15 +533,14 @@
* @msc: MSC device to disable
*
* If @msc is enabled, disable tracing on the switch and then disable MSC
- * storage.
+ * storage. Caller must hold msc::buf_mutex.
*/
static void msc_disable(struct msc *msc)
{
unsigned long count;
u32 reg;
- if (!msc->enabled)
- return;
+ lockdep_assert_held(&msc->buf_mutex);
intel_th_trace_disable(msc->thdev);
@@ -569,33 +580,35 @@
static int intel_th_msc_activate(struct intel_th_device *thdev)
{
struct msc *msc = dev_get_drvdata(&thdev->dev);
- int ret = 0;
+ int ret = -EBUSY;
if (!atomic_inc_unless_negative(&msc->user_count))
return -ENODEV;
- mutex_lock(&msc->iter_mutex);
- if (!list_empty(&msc->iter_list))
- ret = -EBUSY;
- mutex_unlock(&msc->iter_mutex);
+ mutex_lock(&msc->buf_mutex);
- if (ret) {
+ /* if there are readers, refuse */
+ if (list_empty(&msc->iter_list))
+ ret = msc_configure(msc);
+
+ mutex_unlock(&msc->buf_mutex);
+
+ if (ret)
atomic_dec(&msc->user_count);
- return ret;
- }
- msc->enabled = 1;
-
- return msc_configure(msc);
+ return ret;
}
static void intel_th_msc_deactivate(struct intel_th_device *thdev)
{
struct msc *msc = dev_get_drvdata(&thdev->dev);
- msc_disable(msc);
-
- atomic_dec(&msc->user_count);
+ mutex_lock(&msc->buf_mutex);
+ if (msc->enabled) {
+ msc_disable(msc);
+ atomic_dec(&msc->user_count);
+ }
+ mutex_unlock(&msc->buf_mutex);
}
/**
@@ -1035,8 +1048,8 @@
return -EPERM;
iter = msc_iter_install(msc);
- if (!iter)
- return -ENOMEM;
+ if (IS_ERR(iter))
+ return PTR_ERR(iter);
file->private_data = iter;
@@ -1101,11 +1114,6 @@
if (!atomic_inc_unless_negative(&msc->user_count))
return 0;
- if (msc->enabled) {
- ret = -EBUSY;
- goto put_count;
- }
-
if (msc->mode == MSC_MODE_SINGLE && !msc->single_wrap)
size = msc->single_sz;
else
@@ -1164,7 +1172,7 @@
if (!atomic_dec_and_mutex_lock(&msc->mmap_count, &msc->buf_mutex))
return;
- /* drop page _counts */
+ /* drop page _refcounts */
for (pg = 0; pg < msc->nr_pages; pg++) {
struct page *page = msc_buffer_get_page(msc, pg);
@@ -1245,6 +1253,7 @@
.read = intel_th_msc_read,
.mmap = intel_th_msc_mmap,
.llseek = no_llseek,
+ .owner = THIS_MODULE,
};
static int intel_th_msc_init(struct msc *msc)
@@ -1254,8 +1263,6 @@
msc->mode = MSC_MODE_MULTI;
mutex_init(&msc->buf_mutex);
INIT_LIST_HEAD(&msc->win_list);
-
- mutex_init(&msc->iter_mutex);
INIT_LIST_HEAD(&msc->iter_list);
msc->burst_len =
@@ -1393,6 +1400,11 @@
do {
end = memchr(p, ',', len);
s = kstrndup(p, end ? end - p : len, GFP_KERNEL);
+ if (!s) {
+ ret = -ENOMEM;
+ goto free_win;
+ }
+
ret = kstrtoul(s, 10, &val);
kfree(s);
@@ -1473,10 +1485,6 @@
if (err)
return err;
- err = sysfs_create_group(&dev->kobj, &msc_output_group);
- if (err)
- return err;
-
dev_set_drvdata(dev, msc);
return 0;
@@ -1484,7 +1492,18 @@
static void intel_th_msc_remove(struct intel_th_device *thdev)
{
- sysfs_remove_group(&thdev->dev.kobj, &msc_output_group);
+ struct msc *msc = dev_get_drvdata(&thdev->dev);
+ int ret;
+
+ intel_th_msc_deactivate(thdev);
+
+ /*
+ * Buffers should not be used at this point except if the
+ * output character device is still open and the parent
+ * device gets detached from its bus, which is a FIXME.
+ */
+ ret = msc_buffer_free_unless_used(msc);
+ WARN_ON_ONCE(ret);
}
static struct intel_th_driver intel_th_msc_driver = {
@@ -1493,6 +1512,7 @@
.activate = intel_th_msc_activate,
.deactivate = intel_th_msc_deactivate,
.fops = &intel_th_msc_fops,
+ .attr_group = &msc_output_group,
.driver = {
.name = "msc",
.owner = THIS_MODULE,
diff --git a/drivers/hwtracing/intel_th/pci.c b/drivers/hwtracing/intel_th/pci.c
index bca7a2a..5e25c7e 100644
--- a/drivers/hwtracing/intel_th/pci.c
+++ b/drivers/hwtracing/intel_th/pci.c
@@ -75,6 +75,11 @@
PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x0a80),
.driver_data = (kernel_ulong_t)0,
},
+ {
+ /* Broxton B-step */
+ PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x1a8e),
+ .driver_data = (kernel_ulong_t)0,
+ },
{ 0 },
};
diff --git a/drivers/hwtracing/intel_th/pti.c b/drivers/hwtracing/intel_th/pti.c
index 57cbfdc..35738b5 100644
--- a/drivers/hwtracing/intel_th/pti.c
+++ b/drivers/hwtracing/intel_th/pti.c
@@ -200,7 +200,6 @@
struct resource *res;
struct pti_device *pti;
void __iomem *base;
- int ret;
res = intel_th_device_get_resource(thdev, IORESOURCE_MEM, 0);
if (!res)
@@ -219,10 +218,6 @@
read_hw_config(pti);
- ret = sysfs_create_group(&dev->kobj, &pti_output_group);
- if (ret)
- return ret;
-
dev_set_drvdata(dev, pti);
return 0;
@@ -237,6 +232,7 @@
.remove = intel_th_pti_remove,
.activate = intel_th_pti_activate,
.deactivate = intel_th_pti_deactivate,
+ .attr_group = &pti_output_group,
.driver = {
.name = "pti",
.owner = THIS_MODULE,
diff --git a/drivers/hwtracing/stm/core.c b/drivers/hwtracing/stm/core.c
index de80d45..ff31108 100644
--- a/drivers/hwtracing/stm/core.c
+++ b/drivers/hwtracing/stm/core.c
@@ -67,9 +67,24 @@
static DEVICE_ATTR_RO(channels);
+static ssize_t hw_override_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ struct stm_device *stm = to_stm_device(dev);
+ int ret;
+
+ ret = sprintf(buf, "%u\n", stm->data->hw_override);
+
+ return ret;
+}
+
+static DEVICE_ATTR_RO(hw_override);
+
static struct attribute *stm_attrs[] = {
&dev_attr_masters.attr,
&dev_attr_channels.attr,
+ &dev_attr_hw_override.attr,
NULL,
};
@@ -546,8 +561,6 @@
if (ret)
goto err_free;
- ret = 0;
-
if (stm->data->link)
ret = stm->data->link(stm->data, stmf->output.master,
stmf->output.channel);
@@ -668,6 +681,18 @@
stm->dev.parent = parent;
stm->dev.release = stm_device_release;
+ mutex_init(&stm->link_mutex);
+ spin_lock_init(&stm->link_lock);
+ INIT_LIST_HEAD(&stm->link_list);
+
+ /* initialize the object before it is accessible via sysfs */
+ spin_lock_init(&stm->mc_lock);
+ mutex_init(&stm->policy_mutex);
+ stm->sw_nmasters = nmasters;
+ stm->owner = owner;
+ stm->data = stm_data;
+ stm_data->stm = stm;
+
err = kobject_set_name(&stm->dev.kobj, "%s", stm_data->name);
if (err)
goto err_device;
@@ -676,20 +701,11 @@
if (err)
goto err_device;
- mutex_init(&stm->link_mutex);
- spin_lock_init(&stm->link_lock);
- INIT_LIST_HEAD(&stm->link_list);
-
- spin_lock_init(&stm->mc_lock);
- mutex_init(&stm->policy_mutex);
- stm->sw_nmasters = nmasters;
- stm->owner = owner;
- stm->data = stm_data;
- stm_data->stm = stm;
-
return 0;
err_device:
+ unregister_chrdev(stm->major, stm_data->name);
+
/* matches device_initialize() above */
put_device(&stm->dev);
err_free:
diff --git a/drivers/hwtracing/stm/dummy_stm.c b/drivers/hwtracing/stm/dummy_stm.c
index 310adf5..a86612d 100644
--- a/drivers/hwtracing/stm/dummy_stm.c
+++ b/drivers/hwtracing/stm/dummy_stm.c
@@ -46,9 +46,7 @@
static int nr_dummies = 4;
-module_param(nr_dummies, int, 0600);
-
-static unsigned int dummy_stm_nr;
+module_param(nr_dummies, int, 0400);
static unsigned int fail_mode;
@@ -65,12 +63,12 @@
static int dummy_stm_init(void)
{
- int i, ret = -ENOMEM, __nr_dummies = ACCESS_ONCE(nr_dummies);
+ int i, ret = -ENOMEM;
- if (__nr_dummies < 0 || __nr_dummies > DUMMY_STM_MAX)
+ if (nr_dummies < 0 || nr_dummies > DUMMY_STM_MAX)
return -EINVAL;
- for (i = 0; i < __nr_dummies; i++) {
+ for (i = 0; i < nr_dummies; i++) {
dummy_stm[i].name = kasprintf(GFP_KERNEL, "dummy_stm.%d", i);
if (!dummy_stm[i].name)
goto fail_unregister;
@@ -86,8 +84,6 @@
goto fail_free;
}
- dummy_stm_nr = __nr_dummies;
-
return 0;
fail_unregister:
@@ -105,7 +101,7 @@
{
int i;
- for (i = 0; i < dummy_stm_nr; i++) {
+ for (i = 0; i < nr_dummies; i++) {
stm_unregister_device(&dummy_stm[i]);
kfree(dummy_stm[i].name);
}
diff --git a/drivers/hwtracing/stm/heartbeat.c b/drivers/hwtracing/stm/heartbeat.c
index 0133571..3da7b67 100644
--- a/drivers/hwtracing/stm/heartbeat.c
+++ b/drivers/hwtracing/stm/heartbeat.c
@@ -26,7 +26,7 @@
static int nr_devs = 4;
static int interval_ms = 10;
-module_param(nr_devs, int, 0600);
+module_param(nr_devs, int, 0400);
module_param(interval_ms, int, 0600);
static struct stm_heartbeat {
@@ -35,8 +35,6 @@
unsigned int active;
} stm_heartbeat[STM_HEARTBEAT_MAX];
-static unsigned int nr_instances;
-
static const char str[] = "heartbeat stm source driver is here to serve you";
static enum hrtimer_restart stm_heartbeat_hrtimer_handler(struct hrtimer *hr)
@@ -74,12 +72,12 @@
static int stm_heartbeat_init(void)
{
- int i, ret = -ENOMEM, __nr_instances = ACCESS_ONCE(nr_devs);
+ int i, ret = -ENOMEM;
- if (__nr_instances < 0 || __nr_instances > STM_HEARTBEAT_MAX)
+ if (nr_devs < 0 || nr_devs > STM_HEARTBEAT_MAX)
return -EINVAL;
- for (i = 0; i < __nr_instances; i++) {
+ for (i = 0; i < nr_devs; i++) {
stm_heartbeat[i].data.name =
kasprintf(GFP_KERNEL, "heartbeat.%d", i);
if (!stm_heartbeat[i].data.name)
@@ -98,8 +96,6 @@
goto fail_free;
}
- nr_instances = __nr_instances;
-
return 0;
fail_unregister:
@@ -116,7 +112,7 @@
{
int i;
- for (i = 0; i < nr_instances; i++) {
+ for (i = 0; i < nr_devs; i++) {
stm_source_unregister_device(&stm_heartbeat[i].data);
kfree(stm_heartbeat[i].data.name);
}
diff --git a/drivers/hwtracing/stm/policy.c b/drivers/hwtracing/stm/policy.c
index 1db1896..6c0ae29 100644
--- a/drivers/hwtracing/stm/policy.c
+++ b/drivers/hwtracing/stm/policy.c
@@ -107,8 +107,7 @@
goto unlock;
/* must be within [sw_start..sw_end], which is an inclusive range */
- if (first > INT_MAX || last > INT_MAX || first > last ||
- first < stm->data->sw_start ||
+ if (first > last || first < stm->data->sw_start ||
last > stm->data->sw_end) {
ret = -ERANGE;
goto unlock;
@@ -342,7 +341,7 @@
return ERR_PTR(-EINVAL);
}
- *p++ = '\0';
+ *p = '\0';
stm = stm_find_device(devname);
kfree(devname);
diff --git a/drivers/i2c/algos/i2c-algo-bit.c b/drivers/i2c/algos/i2c-algo-bit.c
index 9d233bb..a8e89df 100644
--- a/drivers/i2c/algos/i2c-algo-bit.c
+++ b/drivers/i2c/algos/i2c-algo-bit.c
@@ -617,7 +617,7 @@
};
EXPORT_SYMBOL(i2c_bit_algo);
-const struct i2c_adapter_quirks i2c_bit_quirk_no_clk_stretch = {
+static const struct i2c_adapter_quirks i2c_bit_quirk_no_clk_stretch = {
.flags = I2C_AQ_NO_CLK_STRETCH,
};
diff --git a/drivers/i2c/busses/Kconfig b/drivers/i2c/busses/Kconfig
index 0967e1a..2dd40dd 100644
--- a/drivers/i2c/busses/Kconfig
+++ b/drivers/i2c/busses/Kconfig
@@ -663,7 +663,7 @@
config I2C_MV64XXX
tristate "Marvell mv64xxx I2C Controller"
- depends on MV64X60 || PLAT_ORION || ARCH_SUNXI
+ depends on MV64X60 || PLAT_ORION || ARCH_SUNXI || ARCH_MVEBU
help
If you say yes to this option, support will be included for the
built-in I2C interface on the Marvell 64xxx line of host bridges.
diff --git a/drivers/i2c/busses/i2c-bcm-iproc.c b/drivers/i2c/busses/i2c-bcm-iproc.c
index b9f0fff..19c8438 100644
--- a/drivers/i2c/busses/i2c-bcm-iproc.c
+++ b/drivers/i2c/busses/i2c-bcm-iproc.c
@@ -267,7 +267,7 @@
iproc_i2c->msg = msg;
/* format and load slave address into the TX FIFO */
- addr = msg->addr << 1 | (msg->flags & I2C_M_RD ? 1 : 0);
+ addr = i2c_8bit_addr_from_msg(msg);
writel(addr, iproc_i2c->base + M_TX_OFFSET);
/*
diff --git a/drivers/i2c/busses/i2c-bcm-kona.c b/drivers/i2c/busses/i2c-bcm-kona.c
index 2c9d9b1..ac9f476 100644
--- a/drivers/i2c/busses/i2c-bcm-kona.c
+++ b/drivers/i2c/busses/i2c-bcm-kona.c
@@ -501,10 +501,7 @@
return -EREMOTEIO;
}
} else {
- addr = msg->addr << 1;
-
- if (msg->flags & I2C_M_RD)
- addr |= 1;
+ addr = i2c_8bit_addr_from_msg(msg);
if (bcm_kona_i2c_write_byte(dev, addr, 0) < 0)
return -EREMOTEIO;
diff --git a/drivers/i2c/busses/i2c-brcmstb.c b/drivers/i2c/busses/i2c-brcmstb.c
index 4a45408..6a8cfc1 100644
--- a/drivers/i2c/busses/i2c-brcmstb.c
+++ b/drivers/i2c/busses/i2c-brcmstb.c
@@ -446,9 +446,7 @@
}
} else {
- addr = msg->addr << 1;
- if (msg->flags & I2C_M_RD)
- addr |= 1;
+ addr = i2c_8bit_addr_from_msg(msg);
bsc_writel(dev, addr, chip_address);
}
diff --git a/drivers/i2c/busses/i2c-cpm.c b/drivers/i2c/busses/i2c-cpm.c
index b167ab2..ee57c1e 100644
--- a/drivers/i2c/busses/i2c-cpm.c
+++ b/drivers/i2c/busses/i2c-cpm.c
@@ -197,9 +197,7 @@
tbdf = cpm->tbase + tx;
rbdf = cpm->rbase + rx;
- addr = pmsg->addr << 1;
- if (pmsg->flags & I2C_M_RD)
- addr |= 1;
+ addr = i2c_8bit_addr_from_msg(pmsg);
tb = cpm->txbuf[tx];
rb = cpm->rxbuf[rx];
diff --git a/drivers/i2c/busses/i2c-dln2.c b/drivers/i2c/busses/i2c-dln2.c
index 1600edd..f2eb4f7 100644
--- a/drivers/i2c/busses/i2c-dln2.c
+++ b/drivers/i2c/busses/i2c-dln2.c
@@ -19,6 +19,7 @@
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/mfd/dln2.h>
+#include <linux/acpi.h>
#define DLN2_I2C_MODULE_ID 0x03
#define DLN2_I2C_CMD(cmd) DLN2_CMD(cmd, DLN2_I2C_MODULE_ID)
@@ -210,6 +211,7 @@
dln2->adapter.algo = &dln2_i2c_usb_algorithm;
dln2->adapter.quirks = &dln2_i2c_quirks;
dln2->adapter.dev.parent = dev;
+ ACPI_COMPANION_SET(&dln2->adapter.dev, ACPI_COMPANION(&pdev->dev));
dln2->adapter.dev.of_node = dev->of_node;
i2c_set_adapdata(&dln2->adapter, dln2);
snprintf(dln2->adapter.name, sizeof(dln2->adapter.name), "%s-%s-%d",
diff --git a/drivers/i2c/busses/i2c-exynos5.c b/drivers/i2c/busses/i2c-exynos5.c
index f54ece8..c0e3ada 100644
--- a/drivers/i2c/busses/i2c-exynos5.c
+++ b/drivers/i2c/busses/i2c-exynos5.c
@@ -861,14 +861,8 @@
#endif
static const struct dev_pm_ops exynos5_i2c_dev_pm_ops = {
-#ifdef CONFIG_PM_SLEEP
- .suspend_noirq = exynos5_i2c_suspend_noirq,
- .resume_noirq = exynos5_i2c_resume_noirq,
- .freeze_noirq = exynos5_i2c_suspend_noirq,
- .thaw_noirq = exynos5_i2c_resume_noirq,
- .poweroff_noirq = exynos5_i2c_suspend_noirq,
- .restore_noirq = exynos5_i2c_resume_noirq,
-#endif
+ SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(exynos5_i2c_suspend_noirq,
+ exynos5_i2c_resume_noirq)
};
static struct platform_driver exynos5_i2c_driver = {
diff --git a/drivers/i2c/busses/i2c-i801.c b/drivers/i2c/busses/i2c-i801.c
index 585a3b7..64b1208b 100644
--- a/drivers/i2c/busses/i2c-i801.c
+++ b/drivers/i2c/busses/i2c-i801.c
@@ -94,6 +94,7 @@
#include <linux/err.h>
#include <linux/platform_device.h>
#include <linux/platform_data/itco_wdt.h>
+#include <linux/pm_runtime.h>
#if (defined CONFIG_I2C_MUX_GPIO || defined CONFIG_I2C_MUX_GPIO_MODULE) && \
defined CONFIG_DMI
@@ -714,9 +715,11 @@
{
int hwpec;
int block = 0;
- int ret, xact = 0;
+ int ret = 0, xact = 0;
struct i801_priv *priv = i2c_get_adapdata(adap);
+ pm_runtime_get_sync(&priv->pci_dev->dev);
+
hwpec = (priv->features & FEATURE_SMBUS_PEC) && (flags & I2C_CLIENT_PEC)
&& size != I2C_SMBUS_QUICK
&& size != I2C_SMBUS_I2C_BLOCK_DATA;
@@ -773,7 +776,8 @@
default:
dev_err(&priv->pci_dev->dev, "Unsupported transaction %d\n",
size);
- return -EOPNOTSUPP;
+ ret = -EOPNOTSUPP;
+ goto out;
}
if (hwpec) /* enable/disable hardware PEC */
@@ -796,11 +800,11 @@
~(SMBAUXCTL_CRC | SMBAUXCTL_E32B), SMBAUXCTL(priv));
if (block)
- return ret;
+ goto out;
if (ret)
- return ret;
+ goto out;
if ((read_write == I2C_SMBUS_WRITE) || (xact == I801_QUICK))
- return 0;
+ goto out;
switch (xact & 0x7f) {
case I801_BYTE: /* Result put in SMBHSTDAT0 */
@@ -812,7 +816,11 @@
(inb_p(SMBHSTDAT1(priv)) << 8);
break;
}
- return 0;
+
+out:
+ pm_runtime_mark_last_busy(&priv->pci_dev->dev);
+ pm_runtime_put_autosuspend(&priv->pci_dev->dev);
+ return ret;
}
@@ -1413,6 +1421,11 @@
pci_set_drvdata(dev, priv);
+ pm_runtime_set_autosuspend_delay(&dev->dev, 1000);
+ pm_runtime_use_autosuspend(&dev->dev);
+ pm_runtime_put_autosuspend(&dev->dev);
+ pm_runtime_allow(&dev->dev);
+
return 0;
}
@@ -1420,6 +1433,9 @@
{
struct i801_priv *priv = pci_get_drvdata(dev);
+ pm_runtime_forbid(&dev->dev);
+ pm_runtime_get_noresume(&dev->dev);
+
i801_del_mux(priv);
i2c_del_adapter(&priv->adapter);
pci_write_config_byte(dev, SMBHSTCFG, priv->original_hstcfg);
@@ -1433,34 +1449,32 @@
}
#ifdef CONFIG_PM
-static int i801_suspend(struct pci_dev *dev, pm_message_t mesg)
+static int i801_suspend(struct device *dev)
{
- struct i801_priv *priv = pci_get_drvdata(dev);
+ struct pci_dev *pci_dev = to_pci_dev(dev);
+ struct i801_priv *priv = pci_get_drvdata(pci_dev);
- pci_save_state(dev);
- pci_write_config_byte(dev, SMBHSTCFG, priv->original_hstcfg);
- pci_set_power_state(dev, pci_choose_state(dev, mesg));
+ pci_write_config_byte(pci_dev, SMBHSTCFG, priv->original_hstcfg);
return 0;
}
-static int i801_resume(struct pci_dev *dev)
+static int i801_resume(struct device *dev)
{
- pci_set_power_state(dev, PCI_D0);
- pci_restore_state(dev);
return 0;
}
-#else
-#define i801_suspend NULL
-#define i801_resume NULL
#endif
+static UNIVERSAL_DEV_PM_OPS(i801_pm_ops, i801_suspend,
+ i801_resume, NULL);
+
static struct pci_driver i801_driver = {
.name = "i801_smbus",
.id_table = i801_ids,
.probe = i801_probe,
.remove = i801_remove,
- .suspend = i801_suspend,
- .resume = i801_resume,
+ .driver = {
+ .pm = &i801_pm_ops,
+ },
};
static int __init i2c_i801_init(void)
diff --git a/drivers/i2c/busses/i2c-ibm_iic.c b/drivers/i2c/busses/i2c-ibm_iic.c
index b6c0803..cdaa7be 100644
--- a/drivers/i2c/busses/i2c-ibm_iic.c
+++ b/drivers/i2c/busses/i2c-ibm_iic.c
@@ -269,7 +269,7 @@
ndelay(t->hd_sta);
/* Send address */
- v = (u8)((p->addr << 1) | ((p->flags & I2C_M_RD) ? 1 : 0));
+ v = i2c_8bit_addr_from_msg(p);
for (i = 0, mask = 0x80; i < 8; ++i, mask >>= 1){
out_8(&iic->directcntl, sda);
ndelay(t->low / 2);
diff --git a/drivers/i2c/busses/i2c-img-scb.c b/drivers/i2c/busses/i2c-img-scb.c
index 379ef9c..ea20425b 100644
--- a/drivers/i2c/busses/i2c-img-scb.c
+++ b/drivers/i2c/busses/i2c-img-scb.c
@@ -751,9 +751,7 @@
switch (i2c->at_cur_cmd) {
case CMD_GEN_START:
next_cmd = CMD_GEN_DATA;
- next_data = (i2c->msg.addr << 1);
- if (i2c->msg.flags & I2C_M_RD)
- next_data |= 0x1;
+ next_data = i2c_8bit_addr_from_msg(&i2c->msg);
break;
case CMD_GEN_DATA:
if (i2c->line_status & LINESTAT_INPUT_HELD_V)
diff --git a/drivers/i2c/busses/i2c-imx.c b/drivers/i2c/busses/i2c-imx.c
index 1ca7ef2..1844bc9 100644
--- a/drivers/i2c/busses/i2c-imx.c
+++ b/drivers/i2c/busses/i2c-imx.c
@@ -525,7 +525,7 @@
imx_i2c_write_reg(i2c_imx->hwdata->i2cr_ien_opcode, i2c_imx, IMX_I2C_I2CR);
/* Wait controller to be stable */
- udelay(50);
+ usleep_range(50, 150);
/* Start I2C transaction */
temp = imx_i2c_read_reg(i2c_imx, IMX_I2C_I2CR);
diff --git a/drivers/i2c/busses/i2c-iop3xx.c b/drivers/i2c/busses/i2c-iop3xx.c
index 72d6161..85cbe4b 100644
--- a/drivers/i2c/busses/i2c-iop3xx.c
+++ b/drivers/i2c/busses/i2c-iop3xx.c
@@ -50,10 +50,7 @@
{
unsigned char addr;
- addr = (msg->addr << 1);
-
- if (msg->flags & I2C_M_RD)
- addr |= 1;
+ addr = i2c_8bit_addr_from_msg(msg);
return addr;
}
diff --git a/drivers/i2c/busses/i2c-lpc2k.c b/drivers/i2c/busses/i2c-lpc2k.c
index 8560a13..586a152 100644
--- a/drivers/i2c/busses/i2c-lpc2k.c
+++ b/drivers/i2c/busses/i2c-lpc2k.c
@@ -133,9 +133,7 @@
case M_START:
case M_REPSTART:
/* Start bit was just sent out, send out addr and dir */
- data = i2c->msg->addr << 1;
- if (i2c->msg->flags & I2C_M_RD)
- data |= 1;
+ data = i2c_8bit_addr_from_msg(i2c->msg);
writel(data, i2c->base + LPC24XX_I2DAT);
writel(LPC24XX_STA, i2c->base + LPC24XX_I2CONCLR);
diff --git a/drivers/i2c/busses/i2c-mt65xx.c b/drivers/i2c/busses/i2c-mt65xx.c
index 453358b..d9373e6 100644
--- a/drivers/i2c/busses/i2c-mt65xx.c
+++ b/drivers/i2c/busses/i2c-mt65xx.c
@@ -413,10 +413,7 @@
else
writew(I2C_FS_START_CON, i2c->base + OFFSET_EXT_CONF);
- addr_reg = msgs->addr << 1;
- if (i2c->op == I2C_MASTER_RD)
- addr_reg |= 0x1;
-
+ addr_reg = i2c_8bit_addr_from_msg(msgs);
writew(addr_reg, i2c->base + OFFSET_SLAVE_ADDR);
/* Clear interrupt status */
diff --git a/drivers/i2c/busses/i2c-mv64xxx.c b/drivers/i2c/busses/i2c-mv64xxx.c
index 43207f5..b4dec08 100644
--- a/drivers/i2c/busses/i2c-mv64xxx.c
+++ b/drivers/i2c/busses/i2c-mv64xxx.c
@@ -134,9 +134,7 @@
int rc;
u32 freq_m;
u32 freq_n;
-#if defined(CONFIG_HAVE_CLK)
struct clk *clk;
-#endif
wait_queue_head_t waitq;
spinlock_t lock;
struct i2c_msg *msg;
@@ -757,7 +755,6 @@
MODULE_DEVICE_TABLE(of, mv64xxx_i2c_of_match_table);
#ifdef CONFIG_OF
-#ifdef CONFIG_HAVE_CLK
static int
mv64xxx_calc_freq(struct mv64xxx_i2c_data *drv_data,
const int tclk, const int n, const int m)
@@ -791,25 +788,20 @@
return false;
return true;
}
-#endif /* CONFIG_HAVE_CLK */
static int
mv64xxx_of_config(struct mv64xxx_i2c_data *drv_data,
struct device *dev)
{
- /* CLK is mandatory when using DT to describe the i2c bus. We
- * need to know tclk in order to calculate bus clock
- * factors.
- */
-#if !defined(CONFIG_HAVE_CLK)
- /* Have OF but no CLK */
- return -ENODEV;
-#else
const struct of_device_id *device;
struct device_node *np = dev->of_node;
u32 bus_freq, tclk;
int rc = 0;
+ /* CLK is mandatory when using DT to describe the i2c bus. We
+ * need to know tclk in order to calculate bus clock
+ * factors.
+ */
if (IS_ERR(drv_data->clk)) {
rc = -ENODEV;
goto out;
@@ -869,7 +861,6 @@
out:
return rc;
-#endif
}
#else /* CONFIG_OF */
static int
@@ -907,14 +898,13 @@
init_waitqueue_head(&drv_data->waitq);
spin_lock_init(&drv_data->lock);
-#if defined(CONFIG_HAVE_CLK)
/* Not all platforms have a clk */
drv_data->clk = devm_clk_get(&pd->dev, NULL);
- if (!IS_ERR(drv_data->clk)) {
- clk_prepare(drv_data->clk);
- clk_enable(drv_data->clk);
- }
-#endif
+ if (IS_ERR(drv_data->clk) && PTR_ERR(drv_data->clk) == -EPROBE_DEFER)
+ return -EPROBE_DEFER;
+ if (!IS_ERR(drv_data->clk))
+ clk_prepare_enable(drv_data->clk);
+
if (pdata) {
drv_data->freq_m = pdata->freq_m;
drv_data->freq_n = pdata->freq_n;
@@ -964,13 +954,10 @@
if (!IS_ERR_OR_NULL(drv_data->rstc))
reset_control_assert(drv_data->rstc);
exit_clk:
-#if defined(CONFIG_HAVE_CLK)
/* Not all platforms have a clk */
- if (!IS_ERR(drv_data->clk)) {
- clk_disable(drv_data->clk);
- clk_unprepare(drv_data->clk);
- }
-#endif
+ if (!IS_ERR(drv_data->clk))
+ clk_disable_unprepare(drv_data->clk);
+
return rc;
}
@@ -983,13 +970,9 @@
free_irq(drv_data->irq, drv_data);
if (!IS_ERR_OR_NULL(drv_data->rstc))
reset_control_assert(drv_data->rstc);
-#if defined(CONFIG_HAVE_CLK)
/* Not all platforms have a clk */
- if (!IS_ERR(drv_data->clk)) {
- clk_disable(drv_data->clk);
- clk_unprepare(drv_data->clk);
- }
-#endif
+ if (!IS_ERR(drv_data->clk))
+ clk_disable_unprepare(drv_data->clk);
return 0;
}
diff --git a/drivers/i2c/busses/i2c-nforce2.c b/drivers/i2c/busses/i2c-nforce2.c
index 70b3c91..42fcc94 100644
--- a/drivers/i2c/busses/i2c-nforce2.c
+++ b/drivers/i2c/busses/i2c-nforce2.c
@@ -127,7 +127,7 @@
/* For multiplexing support, we need a global reference to the 1st
SMBus channel */
-#if defined CONFIG_I2C_NFORCE2_S4985 || defined CONFIG_I2C_NFORCE2_S4985_MODULE
+#if IS_ENABLED(CONFIG_I2C_NFORCE2_S4985)
struct i2c_adapter *nforce2_smbus;
EXPORT_SYMBOL_GPL(nforce2_smbus);
diff --git a/drivers/i2c/busses/i2c-ocores.c b/drivers/i2c/busses/i2c-ocores.c
index 11b7b87..dfa7a4b 100644
--- a/drivers/i2c/busses/i2c-ocores.c
+++ b/drivers/i2c/busses/i2c-ocores.c
@@ -178,10 +178,7 @@
if (i2c->nmsgs) { /* end? */
/* send start? */
if (!(msg->flags & I2C_M_NOSTART)) {
- u8 addr = (msg->addr << 1);
-
- if (msg->flags & I2C_M_RD)
- addr |= 1;
+ u8 addr = i2c_8bit_addr_from_msg(msg);
i2c->state = STATE_START;
diff --git a/drivers/i2c/busses/i2c-octeon.c b/drivers/i2c/busses/i2c-octeon.c
index 46fb6c4..aa5f01e 100644
--- a/drivers/i2c/busses/i2c-octeon.c
+++ b/drivers/i2c/busses/i2c-octeon.c
@@ -11,6 +11,7 @@
* warranty of any kind, whether express or implied.
*/
+#include <linux/atomic.h>
#include <linux/platform_device.h>
#include <linux/interrupt.h>
#include <linux/kernel.h>
@@ -29,13 +30,23 @@
/* Register offsets */
#define SW_TWSI 0x00
#define TWSI_INT 0x10
+#define SW_TWSI_EXT 0x18
/* Controller command patterns */
#define SW_TWSI_V BIT_ULL(63) /* Valid bit */
+#define SW_TWSI_EIA BIT_ULL(61) /* Extended internal address */
#define SW_TWSI_R BIT_ULL(56) /* Result or read bit */
+#define SW_TWSI_SOVR BIT_ULL(55) /* Size override */
+#define SW_TWSI_SIZE_SHIFT 52
+#define SW_TWSI_ADDR_SHIFT 40
+#define SW_TWSI_IA_SHIFT 32 /* Internal address */
/* Controller opcode word (bits 60:57) */
#define SW_TWSI_OP_SHIFT 57
+#define SW_TWSI_OP_7 (0ULL << SW_TWSI_OP_SHIFT)
+#define SW_TWSI_OP_7_IA (1ULL << SW_TWSI_OP_SHIFT)
+#define SW_TWSI_OP_10 (2ULL << SW_TWSI_OP_SHIFT)
+#define SW_TWSI_OP_10_IA (3ULL << SW_TWSI_OP_SHIFT)
#define SW_TWSI_OP_TWSI_CLK (4ULL << SW_TWSI_OP_SHIFT)
#define SW_TWSI_OP_EOP (6ULL << SW_TWSI_OP_SHIFT) /* Extended opcode */
@@ -48,46 +59,93 @@
#define SW_TWSI_EOP_TWSI_RST (SW_TWSI_OP_EOP | 7ULL << SW_TWSI_EOP_SHIFT)
/* Controller command and status bits */
-#define TWSI_CTL_CE 0x80
+#define TWSI_CTL_CE 0x80 /* High level controller enable */
#define TWSI_CTL_ENAB 0x40 /* Bus enable */
#define TWSI_CTL_STA 0x20 /* Master-mode start, HW clears when done */
#define TWSI_CTL_STP 0x10 /* Master-mode stop, HW clears when done */
#define TWSI_CTL_IFLG 0x08 /* HW event, SW writes 0 to ACK */
#define TWSI_CTL_AAK 0x04 /* Assert ACK */
-/* Some status values */
+/* Status values */
+#define STAT_ERROR 0x00
#define STAT_START 0x08
-#define STAT_RSTART 0x10
+#define STAT_REP_START 0x10
#define STAT_TXADDR_ACK 0x18
+#define STAT_TXADDR_NAK 0x20
#define STAT_TXDATA_ACK 0x28
+#define STAT_TXDATA_NAK 0x30
+#define STAT_LOST_ARB_38 0x38
#define STAT_RXADDR_ACK 0x40
+#define STAT_RXADDR_NAK 0x48
#define STAT_RXDATA_ACK 0x50
+#define STAT_RXDATA_NAK 0x58
+#define STAT_SLAVE_60 0x60
+#define STAT_LOST_ARB_68 0x68
+#define STAT_SLAVE_70 0x70
+#define STAT_LOST_ARB_78 0x78
+#define STAT_SLAVE_80 0x80
+#define STAT_SLAVE_88 0x88
+#define STAT_GENDATA_ACK 0x90
+#define STAT_GENDATA_NAK 0x98
+#define STAT_SLAVE_A0 0xA0
+#define STAT_SLAVE_A8 0xA8
+#define STAT_LOST_ARB_B0 0xB0
+#define STAT_SLAVE_LOST 0xB8
+#define STAT_SLAVE_NAK 0xC0
+#define STAT_SLAVE_ACK 0xC8
+#define STAT_AD2W_ACK 0xD0
+#define STAT_AD2W_NAK 0xD8
#define STAT_IDLE 0xF8
/* TWSI_INT values */
+#define TWSI_INT_ST_INT BIT_ULL(0)
+#define TWSI_INT_TS_INT BIT_ULL(1)
+#define TWSI_INT_CORE_INT BIT_ULL(2)
+#define TWSI_INT_ST_EN BIT_ULL(4)
+#define TWSI_INT_TS_EN BIT_ULL(5)
#define TWSI_INT_CORE_EN BIT_ULL(6)
#define TWSI_INT_SDA_OVR BIT_ULL(8)
#define TWSI_INT_SCL_OVR BIT_ULL(9)
+#define TWSI_INT_SDA BIT_ULL(10)
+#define TWSI_INT_SCL BIT_ULL(11)
+
+#define I2C_OCTEON_EVENT_WAIT 80 /* microseconds */
struct octeon_i2c {
wait_queue_head_t queue;
struct i2c_adapter adap;
int irq;
+ int hlc_irq; /* For cn7890 only */
u32 twsi_freq;
int sys_freq;
void __iomem *twsi_base;
struct device *dev;
+ bool hlc_enabled;
+ bool broken_irq_mode;
+ bool broken_irq_check;
+ void (*int_enable)(struct octeon_i2c *);
+ void (*int_disable)(struct octeon_i2c *);
+ void (*hlc_int_enable)(struct octeon_i2c *);
+ void (*hlc_int_disable)(struct octeon_i2c *);
+ atomic_t int_enable_cnt;
+ atomic_t hlc_int_enable_cnt;
};
+static void octeon_i2c_writeq_flush(u64 val, void __iomem *addr)
+{
+ __raw_writeq(val, addr);
+ __raw_readq(addr); /* wait for write to land */
+}
+
/**
- * octeon_i2c_write_sw - write an I2C core register
+ * octeon_i2c_reg_write - write an I2C core register
* @i2c: The struct octeon_i2c
* @eop_reg: Register selector
* @data: Value to be written
*
* The I2C core registers are accessed indirectly via the SW_TWSI CSR.
*/
-static void octeon_i2c_write_sw(struct octeon_i2c *i2c, u64 eop_reg, u8 data)
+static void octeon_i2c_reg_write(struct octeon_i2c *i2c, u64 eop_reg, u8 data)
{
u64 tmp;
@@ -97,8 +155,13 @@
} while ((tmp & SW_TWSI_V) != 0);
}
+#define octeon_i2c_ctl_write(i2c, val) \
+ octeon_i2c_reg_write(i2c, SW_TWSI_EOP_TWSI_CTL, val)
+#define octeon_i2c_data_write(i2c, val) \
+ octeon_i2c_reg_write(i2c, SW_TWSI_EOP_TWSI_DATA, val)
+
/**
- * octeon_i2c_read_sw - read lower bits of an I2C core register
+ * octeon_i2c_reg_read - read lower bits of an I2C core register
* @i2c: The struct octeon_i2c
* @eop_reg: Register selector
*
@@ -106,7 +169,7 @@
*
* The I2C core registers are accessed indirectly via the SW_TWSI CSR.
*/
-static u8 octeon_i2c_read_sw(struct octeon_i2c *i2c, u64 eop_reg)
+static u8 octeon_i2c_reg_read(struct octeon_i2c *i2c, u64 eop_reg)
{
u64 tmp;
@@ -118,6 +181,24 @@
return tmp & 0xFF;
}
+#define octeon_i2c_ctl_read(i2c) \
+ octeon_i2c_reg_read(i2c, SW_TWSI_EOP_TWSI_CTL)
+#define octeon_i2c_data_read(i2c) \
+ octeon_i2c_reg_read(i2c, SW_TWSI_EOP_TWSI_DATA)
+#define octeon_i2c_stat_read(i2c) \
+ octeon_i2c_reg_read(i2c, SW_TWSI_EOP_TWSI_STAT)
+
+/**
+ * octeon_i2c_read_int - read the TWSI_INT register
+ * @i2c: The struct octeon_i2c
+ *
+ * Returns the value of the register.
+ */
+static u64 octeon_i2c_read_int(struct octeon_i2c *i2c)
+{
+ return __raw_readq(i2c->twsi_base + TWSI_INT);
+}
+
/**
* octeon_i2c_write_int - write the TWSI_INT register
* @i2c: The struct octeon_i2c
@@ -125,8 +206,7 @@
*/
static void octeon_i2c_write_int(struct octeon_i2c *i2c, u64 data)
{
- __raw_writeq(data, i2c->twsi_base + TWSI_INT);
- __raw_readq(i2c->twsi_base + TWSI_INT);
+ octeon_i2c_writeq_flush(data, i2c->twsi_base + TWSI_INT);
}
/**
@@ -149,30 +229,96 @@
}
/**
- * octeon_i2c_unblock - unblock the bus
+ * octeon_i2c_int_enable78 - enable the CORE interrupt
* @i2c: The struct octeon_i2c
*
- * If there was a reset while a device was driving 0 to bus, bus is blocked.
- * We toggle it free manually by some clock cycles and send a stop.
+ * The interrupt will be asserted when there is non-STAT_IDLE state in the
+ * SW_TWSI_EOP_TWSI_STAT register.
*/
-static void octeon_i2c_unblock(struct octeon_i2c *i2c)
+static void octeon_i2c_int_enable78(struct octeon_i2c *i2c)
{
- int i;
+ atomic_inc_return(&i2c->int_enable_cnt);
+ enable_irq(i2c->irq);
+}
- dev_dbg(i2c->dev, "%s\n", __func__);
+static void __octeon_i2c_irq_disable(atomic_t *cnt, int irq)
+{
+ int count;
- for (i = 0; i < 9; i++) {
- octeon_i2c_write_int(i2c, 0);
- udelay(5);
- octeon_i2c_write_int(i2c, TWSI_INT_SCL_OVR);
- udelay(5);
+ /*
+ * The interrupt can be disabled in two places, but we only
+ * want to make the disable_irq_nosync() call once, so keep
+ * track with the atomic variable.
+ */
+ count = atomic_dec_if_positive(cnt);
+ if (count >= 0)
+ disable_irq_nosync(irq);
+}
+
+/* disable the CORE interrupt */
+static void octeon_i2c_int_disable78(struct octeon_i2c *i2c)
+{
+ __octeon_i2c_irq_disable(&i2c->int_enable_cnt, i2c->irq);
+}
+
+/**
+ * octeon_i2c_hlc_int_enable78 - enable the ST interrupt
+ * @i2c: The struct octeon_i2c
+ *
+ * The interrupt will be asserted when there is non-STAT_IDLE state in
+ * the SW_TWSI_EOP_TWSI_STAT register.
+ */
+static void octeon_i2c_hlc_int_enable78(struct octeon_i2c *i2c)
+{
+ atomic_inc_return(&i2c->hlc_int_enable_cnt);
+ enable_irq(i2c->hlc_irq);
+}
+
+/* disable the ST interrupt */
+static void octeon_i2c_hlc_int_disable78(struct octeon_i2c *i2c)
+{
+ __octeon_i2c_irq_disable(&i2c->hlc_int_enable_cnt, i2c->hlc_irq);
+}
+
+/*
+ * Cleanup low-level state & enable high-level controller.
+ */
+static void octeon_i2c_hlc_enable(struct octeon_i2c *i2c)
+{
+ int try = 0;
+ u64 val;
+
+ if (i2c->hlc_enabled)
+ return;
+ i2c->hlc_enabled = true;
+
+ while (1) {
+ val = octeon_i2c_ctl_read(i2c);
+ if (!(val & (TWSI_CTL_STA | TWSI_CTL_STP)))
+ break;
+
+ /* clear IFLG event */
+ if (val & TWSI_CTL_IFLG)
+ octeon_i2c_ctl_write(i2c, TWSI_CTL_ENAB);
+
+ if (try++ > 100) {
+ pr_err("%s: giving up\n", __func__);
+ break;
+ }
+
+ /* spin until any start/stop has finished */
+ udelay(10);
}
- /* hand-crank a STOP */
- octeon_i2c_write_int(i2c, TWSI_INT_SDA_OVR | TWSI_INT_SCL_OVR);
- udelay(5);
- octeon_i2c_write_int(i2c, TWSI_INT_SDA_OVR);
- udelay(5);
- octeon_i2c_write_int(i2c, 0);
+ octeon_i2c_ctl_write(i2c, TWSI_CTL_CE | TWSI_CTL_AAK | TWSI_CTL_ENAB);
+}
+
+static void octeon_i2c_hlc_disable(struct octeon_i2c *i2c)
+{
+ if (!i2c->hlc_enabled)
+ return;
+
+ i2c->hlc_enabled = false;
+ octeon_i2c_ctl_write(i2c, TWSI_CTL_ENAB);
}
/* interrupt service routine */
@@ -180,16 +326,44 @@
{
struct octeon_i2c *i2c = dev_id;
- octeon_i2c_int_disable(i2c);
+ i2c->int_disable(i2c);
wake_up(&i2c->queue);
return IRQ_HANDLED;
}
-
-static int octeon_i2c_test_iflg(struct octeon_i2c *i2c)
+/* HLC interrupt service routine */
+static irqreturn_t octeon_i2c_hlc_isr78(int irq, void *dev_id)
{
- return (octeon_i2c_read_sw(i2c, SW_TWSI_EOP_TWSI_CTL) & TWSI_CTL_IFLG) != 0;
+ struct octeon_i2c *i2c = dev_id;
+
+ i2c->hlc_int_disable(i2c);
+ wake_up(&i2c->queue);
+
+ return IRQ_HANDLED;
+}
+
+static bool octeon_i2c_test_iflg(struct octeon_i2c *i2c)
+{
+ return (octeon_i2c_ctl_read(i2c) & TWSI_CTL_IFLG);
+}
+
+static bool octeon_i2c_test_ready(struct octeon_i2c *i2c, bool *first)
+{
+ if (octeon_i2c_test_iflg(i2c))
+ return true;
+
+ if (*first) {
+ *first = false;
+ return false;
+ }
+
+ /*
+ * IRQ has signaled an event but IFLG hasn't changed.
+ * Sleep and retry once.
+ */
+ usleep_range(I2C_OCTEON_EVENT_WAIT, 2 * I2C_OCTEON_EVENT_WAIT);
+ return octeon_i2c_test_iflg(i2c);
}
/**
@@ -201,233 +375,379 @@
static int octeon_i2c_wait(struct octeon_i2c *i2c)
{
long time_left;
+ bool first = 1;
- octeon_i2c_int_enable(i2c);
- time_left = wait_event_timeout(i2c->queue, octeon_i2c_test_iflg(i2c),
- i2c->adap.timeout);
- octeon_i2c_int_disable(i2c);
- if (!time_left) {
- dev_dbg(i2c->dev, "%s: timeout\n", __func__);
- return -ETIMEDOUT;
+ /*
+ * Some chip revisions don't assert the irq in the interrupt
+ * controller. So we must poll for the IFLG change.
+ */
+ if (i2c->broken_irq_mode) {
+ u64 end = get_jiffies_64() + i2c->adap.timeout;
+
+ while (!octeon_i2c_test_iflg(i2c) &&
+ time_before64(get_jiffies_64(), end))
+ usleep_range(I2C_OCTEON_EVENT_WAIT / 2, I2C_OCTEON_EVENT_WAIT);
+
+ return octeon_i2c_test_iflg(i2c) ? 0 : -ETIMEDOUT;
}
+ i2c->int_enable(i2c);
+ time_left = wait_event_timeout(i2c->queue, octeon_i2c_test_ready(i2c, &first),
+ i2c->adap.timeout);
+ i2c->int_disable(i2c);
+
+ if (i2c->broken_irq_check && !time_left &&
+ octeon_i2c_test_iflg(i2c)) {
+ dev_err(i2c->dev, "broken irq connection detected, switching to polling mode.\n");
+ i2c->broken_irq_mode = true;
+ return 0;
+ }
+
+ if (!time_left)
+ return -ETIMEDOUT;
+
return 0;
}
-/**
- * octeon_i2c_start - send START to the bus
- * @i2c: The struct octeon_i2c
- *
- * Returns 0 on success, otherwise a negative errno.
- */
-static int octeon_i2c_start(struct octeon_i2c *i2c)
+static int octeon_i2c_check_status(struct octeon_i2c *i2c, int final_read)
{
- int result;
- u8 data;
+ u8 stat = octeon_i2c_stat_read(i2c);
- octeon_i2c_write_sw(i2c, SW_TWSI_EOP_TWSI_CTL,
- TWSI_CTL_ENAB | TWSI_CTL_STA);
+ switch (stat) {
+ /* Everything is fine */
+ case STAT_IDLE:
+ case STAT_AD2W_ACK:
+ case STAT_RXADDR_ACK:
+ case STAT_TXADDR_ACK:
+ case STAT_TXDATA_ACK:
+ return 0;
- result = octeon_i2c_wait(i2c);
- if (result) {
- if (octeon_i2c_read_sw(i2c, SW_TWSI_EOP_TWSI_STAT) == STAT_IDLE) {
- /*
- * Controller refused to send start flag May
- * be a client is holding SDA low - let's try
- * to free it.
- */
- octeon_i2c_unblock(i2c);
- octeon_i2c_write_sw(i2c, SW_TWSI_EOP_TWSI_CTL,
- TWSI_CTL_ENAB | TWSI_CTL_STA);
- result = octeon_i2c_wait(i2c);
- }
- if (result)
- return result;
- }
+ /* ACK allowed on pre-terminal bytes only */
+ case STAT_RXDATA_ACK:
+ if (!final_read)
+ return 0;
+ return -EIO;
- data = octeon_i2c_read_sw(i2c, SW_TWSI_EOP_TWSI_STAT);
- if ((data != STAT_START) && (data != STAT_RSTART)) {
- dev_err(i2c->dev, "%s: bad status (0x%x)\n", __func__, data);
+ /* NAK allowed on terminal byte only */
+ case STAT_RXDATA_NAK:
+ if (final_read)
+ return 0;
+ return -EIO;
+
+ /* Arbitration lost */
+ case STAT_LOST_ARB_38:
+ case STAT_LOST_ARB_68:
+ case STAT_LOST_ARB_78:
+ case STAT_LOST_ARB_B0:
+ return -EAGAIN;
+
+ /* Being addressed as slave, should back off & listen */
+ case STAT_SLAVE_60:
+ case STAT_SLAVE_70:
+ case STAT_GENDATA_ACK:
+ case STAT_GENDATA_NAK:
+ return -EOPNOTSUPP;
+
+ /* Core busy as slave */
+ case STAT_SLAVE_80:
+ case STAT_SLAVE_88:
+ case STAT_SLAVE_A0:
+ case STAT_SLAVE_A8:
+ case STAT_SLAVE_LOST:
+ case STAT_SLAVE_NAK:
+ case STAT_SLAVE_ACK:
+ return -EOPNOTSUPP;
+
+ case STAT_TXDATA_NAK:
+ return -EIO;
+ case STAT_TXADDR_NAK:
+ case STAT_RXADDR_NAK:
+ case STAT_AD2W_NAK:
+ return -ENXIO;
+ default:
+ dev_err(i2c->dev, "unhandled state: %d\n", stat);
return -EIO;
}
-
- return 0;
}
-/* send STOP to the bus */
-static void octeon_i2c_stop(struct octeon_i2c *i2c)
+static bool octeon_i2c_hlc_test_valid(struct octeon_i2c *i2c)
{
- octeon_i2c_write_sw(i2c, SW_TWSI_EOP_TWSI_CTL,
- TWSI_CTL_ENAB | TWSI_CTL_STP);
+ return (__raw_readq(i2c->twsi_base + SW_TWSI) & SW_TWSI_V) == 0;
+}
+
+static bool octeon_i2c_hlc_test_ready(struct octeon_i2c *i2c, bool *first)
+{
+ /* check if valid bit is cleared */
+ if (octeon_i2c_hlc_test_valid(i2c))
+ return true;
+
+ if (*first) {
+ *first = false;
+ return false;
+ }
+
+ /*
+ * IRQ has signaled an event but valid bit isn't cleared.
+ * Sleep and retry once.
+ */
+ usleep_range(I2C_OCTEON_EVENT_WAIT, 2 * I2C_OCTEON_EVENT_WAIT);
+ return octeon_i2c_hlc_test_valid(i2c);
+}
+
+static void octeon_i2c_hlc_int_enable(struct octeon_i2c *i2c)
+{
+ octeon_i2c_write_int(i2c, TWSI_INT_ST_EN);
+}
+
+static void octeon_i2c_hlc_int_clear(struct octeon_i2c *i2c)
+{
+ /* clear ST/TS events, listen for neither */
+ octeon_i2c_write_int(i2c, TWSI_INT_ST_INT | TWSI_INT_TS_INT);
}
/**
- * octeon_i2c_write - send data to the bus via low-level controller
+ * octeon_i2c_hlc_wait - wait for an HLC operation to complete
* @i2c: The struct octeon_i2c
- * @target: Target address
- * @data: Pointer to the data to be sent
- * @length: Length of the data
*
- * The address is sent over the bus, then the data.
- *
- * Returns 0 on success, otherwise a negative errno.
+ * Returns 0 on success, otherwise -ETIMEDOUT.
*/
-static int octeon_i2c_write(struct octeon_i2c *i2c, int target,
- const u8 *data, int length)
+static int octeon_i2c_hlc_wait(struct octeon_i2c *i2c)
{
- int i, result;
- u8 tmp;
+ bool first = 1;
+ int time_left;
- result = octeon_i2c_start(i2c);
- if (result)
- return result;
+ /*
+ * Some cn38xx boards don't assert the irq in the interrupt
+ * controller. So we must poll for the valid bit change.
+ */
+ if (i2c->broken_irq_mode) {
+ u64 end = get_jiffies_64() + i2c->adap.timeout;
- octeon_i2c_write_sw(i2c, SW_TWSI_EOP_TWSI_DATA, target << 1);
- octeon_i2c_write_sw(i2c, SW_TWSI_EOP_TWSI_CTL, TWSI_CTL_ENAB);
+ while (!octeon_i2c_hlc_test_valid(i2c) &&
+ time_before64(get_jiffies_64(), end))
+ usleep_range(I2C_OCTEON_EVENT_WAIT / 2, I2C_OCTEON_EVENT_WAIT);
- result = octeon_i2c_wait(i2c);
- if (result)
- return result;
-
- for (i = 0; i < length; i++) {
- tmp = octeon_i2c_read_sw(i2c, SW_TWSI_EOP_TWSI_STAT);
-
- if ((tmp != STAT_TXADDR_ACK) && (tmp != STAT_TXDATA_ACK)) {
- dev_err(i2c->dev,
- "%s: bad status before write (0x%x)\n",
- __func__, tmp);
- return -EIO;
- }
-
- octeon_i2c_write_sw(i2c, SW_TWSI_EOP_TWSI_DATA, data[i]);
- octeon_i2c_write_sw(i2c, SW_TWSI_EOP_TWSI_CTL, TWSI_CTL_ENAB);
-
- result = octeon_i2c_wait(i2c);
- if (result)
- return result;
+ return octeon_i2c_hlc_test_valid(i2c) ? 0 : -ETIMEDOUT;
}
+ i2c->hlc_int_enable(i2c);
+ time_left = wait_event_timeout(i2c->queue,
+ octeon_i2c_hlc_test_ready(i2c, &first),
+ i2c->adap.timeout);
+ i2c->hlc_int_disable(i2c);
+ if (!time_left)
+ octeon_i2c_hlc_int_clear(i2c);
+
+ if (i2c->broken_irq_check && !time_left &&
+ octeon_i2c_hlc_test_valid(i2c)) {
+ dev_err(i2c->dev, "broken irq connection detected, switching to polling mode.\n");
+ i2c->broken_irq_mode = true;
+ return 0;
+ }
+
+ if (!time_left)
+ return -ETIMEDOUT;
return 0;
}
-/**
- * octeon_i2c_read - receive data from the bus via low-level controller
- * @i2c: The struct octeon_i2c
- * @target: Target address
- * @data: Pointer to the location to store the data
- * @rlength: Length of the data
- * @recv_len: flag for length byte
- *
- * The address is sent over the bus, then the data is read.
- *
- * Returns 0 on success, otherwise a negative errno.
- */
-static int octeon_i2c_read(struct octeon_i2c *i2c, int target,
- u8 *data, u16 *rlength, bool recv_len)
+/* high-level-controller pure read of up to 8 bytes */
+static int octeon_i2c_hlc_read(struct octeon_i2c *i2c, struct i2c_msg *msgs)
{
- int i, result, length = *rlength;
- u8 tmp;
+ int i, j, ret = 0;
+ u64 cmd;
- if (length < 1)
- return -EINVAL;
+ octeon_i2c_hlc_enable(i2c);
+ octeon_i2c_hlc_int_clear(i2c);
- result = octeon_i2c_start(i2c);
- if (result)
- return result;
+ cmd = SW_TWSI_V | SW_TWSI_R | SW_TWSI_SOVR;
+ /* SIZE */
+ cmd |= (u64)(msgs[0].len - 1) << SW_TWSI_SIZE_SHIFT;
+ /* A */
+ cmd |= (u64)(msgs[0].addr & 0x7full) << SW_TWSI_ADDR_SHIFT;
- octeon_i2c_write_sw(i2c, SW_TWSI_EOP_TWSI_DATA, (target << 1) | 1);
- octeon_i2c_write_sw(i2c, SW_TWSI_EOP_TWSI_CTL, TWSI_CTL_ENAB);
+ if (msgs[0].flags & I2C_M_TEN)
+ cmd |= SW_TWSI_OP_10;
+ else
+ cmd |= SW_TWSI_OP_7;
- result = octeon_i2c_wait(i2c);
- if (result)
- return result;
+ octeon_i2c_writeq_flush(cmd, i2c->twsi_base + SW_TWSI);
+ ret = octeon_i2c_hlc_wait(i2c);
+ if (ret)
+ goto err;
- for (i = 0; i < length; i++) {
- tmp = octeon_i2c_read_sw(i2c, SW_TWSI_EOP_TWSI_STAT);
+ cmd = __raw_readq(i2c->twsi_base + SW_TWSI);
+ if ((cmd & SW_TWSI_R) == 0)
+ return -EAGAIN;
- if ((tmp != STAT_RXDATA_ACK) && (tmp != STAT_RXADDR_ACK)) {
- dev_err(i2c->dev,
- "%s: bad status before read (0x%x)\n",
- __func__, tmp);
- return -EIO;
- }
+ for (i = 0, j = msgs[0].len - 1; i < msgs[0].len && i < 4; i++, j--)
+ msgs[0].buf[j] = (cmd >> (8 * i)) & 0xff;
- if (i + 1 < length)
- octeon_i2c_write_sw(i2c, SW_TWSI_EOP_TWSI_CTL,
- TWSI_CTL_ENAB | TWSI_CTL_AAK);
- else
- octeon_i2c_write_sw(i2c, SW_TWSI_EOP_TWSI_CTL,
- TWSI_CTL_ENAB);
-
- result = octeon_i2c_wait(i2c);
- if (result)
- return result;
-
- data[i] = octeon_i2c_read_sw(i2c, SW_TWSI_EOP_TWSI_DATA);
- if (recv_len && i == 0) {
- if (data[i] > I2C_SMBUS_BLOCK_MAX + 1) {
- dev_err(i2c->dev,
- "%s: read len > I2C_SMBUS_BLOCK_MAX %d\n",
- __func__, data[i]);
- return -EPROTO;
- }
- length += data[i];
- }
+ if (msgs[0].len > 4) {
+ cmd = __raw_readq(i2c->twsi_base + SW_TWSI_EXT);
+ for (i = 0; i < msgs[0].len - 4 && i < 4; i++, j--)
+ msgs[0].buf[j] = (cmd >> (8 * i)) & 0xff;
}
- *rlength = length;
- return 0;
+
+err:
+ return ret;
}
-/**
- * octeon_i2c_xfer - The driver's master_xfer function
- * @adap: Pointer to the i2c_adapter structure
- * @msgs: Pointer to the messages to be processed
- * @num: Length of the MSGS array
- *
- * Returns the number of messages processed, or a negative errno on failure.
- */
-static int octeon_i2c_xfer(struct i2c_adapter *adap, struct i2c_msg *msgs,
- int num)
+/* high-level-controller pure write of up to 8 bytes */
+static int octeon_i2c_hlc_write(struct octeon_i2c *i2c, struct i2c_msg *msgs)
{
- struct octeon_i2c *i2c = i2c_get_adapdata(adap);
- int i, ret = 0;
+ int i, j, ret = 0;
+ u64 cmd;
- for (i = 0; ret == 0 && i < num; i++) {
- struct i2c_msg *pmsg = &msgs[i];
+ octeon_i2c_hlc_enable(i2c);
+ octeon_i2c_hlc_int_clear(i2c);
- dev_dbg(i2c->dev,
- "Doing %s %d byte(s) to/from 0x%02x - %d of %d messages\n",
- pmsg->flags & I2C_M_RD ? "read" : "write",
- pmsg->len, pmsg->addr, i + 1, num);
- if (pmsg->flags & I2C_M_RD)
- ret = octeon_i2c_read(i2c, pmsg->addr, pmsg->buf,
- &pmsg->len, pmsg->flags & I2C_M_RECV_LEN);
- else
- ret = octeon_i2c_write(i2c, pmsg->addr, pmsg->buf,
- pmsg->len);
+ cmd = SW_TWSI_V | SW_TWSI_SOVR;
+ /* SIZE */
+ cmd |= (u64)(msgs[0].len - 1) << SW_TWSI_SIZE_SHIFT;
+ /* A */
+ cmd |= (u64)(msgs[0].addr & 0x7full) << SW_TWSI_ADDR_SHIFT;
+
+ if (msgs[0].flags & I2C_M_TEN)
+ cmd |= SW_TWSI_OP_10;
+ else
+ cmd |= SW_TWSI_OP_7;
+
+ for (i = 0, j = msgs[0].len - 1; i < msgs[0].len && i < 4; i++, j--)
+ cmd |= (u64)msgs[0].buf[j] << (8 * i);
+
+ if (msgs[0].len > 4) {
+ u64 ext = 0;
+
+ for (i = 0; i < msgs[0].len - 4 && i < 4; i++, j--)
+ ext |= (u64)msgs[0].buf[j] << (8 * i);
+ octeon_i2c_writeq_flush(ext, i2c->twsi_base + SW_TWSI_EXT);
}
- octeon_i2c_stop(i2c);
- return (ret != 0) ? ret : num;
+ octeon_i2c_writeq_flush(cmd, i2c->twsi_base + SW_TWSI);
+ ret = octeon_i2c_hlc_wait(i2c);
+ if (ret)
+ goto err;
+
+ cmd = __raw_readq(i2c->twsi_base + SW_TWSI);
+ if ((cmd & SW_TWSI_R) == 0)
+ return -EAGAIN;
+
+ ret = octeon_i2c_check_status(i2c, false);
+
+err:
+ return ret;
}
-static u32 octeon_i2c_functionality(struct i2c_adapter *adap)
+/* high-level-controller composite write+read, msg0=addr, msg1=data */
+static int octeon_i2c_hlc_comp_read(struct octeon_i2c *i2c, struct i2c_msg *msgs)
{
- return I2C_FUNC_I2C | I2C_FUNC_SMBUS_EMUL |
- I2C_FUNC_SMBUS_READ_BLOCK_DATA | I2C_SMBUS_BLOCK_PROC_CALL;
+ int i, j, ret = 0;
+ u64 cmd;
+
+ octeon_i2c_hlc_enable(i2c);
+
+ cmd = SW_TWSI_V | SW_TWSI_R | SW_TWSI_SOVR;
+ /* SIZE */
+ cmd |= (u64)(msgs[1].len - 1) << SW_TWSI_SIZE_SHIFT;
+ /* A */
+ cmd |= (u64)(msgs[0].addr & 0x7full) << SW_TWSI_ADDR_SHIFT;
+
+ if (msgs[0].flags & I2C_M_TEN)
+ cmd |= SW_TWSI_OP_10_IA;
+ else
+ cmd |= SW_TWSI_OP_7_IA;
+
+ if (msgs[0].len == 2) {
+ u64 ext = 0;
+
+ cmd |= SW_TWSI_EIA;
+ ext = (u64)msgs[0].buf[0] << SW_TWSI_IA_SHIFT;
+ cmd |= (u64)msgs[0].buf[1] << SW_TWSI_IA_SHIFT;
+ octeon_i2c_writeq_flush(ext, i2c->twsi_base + SW_TWSI_EXT);
+ } else {
+ cmd |= (u64)msgs[0].buf[0] << SW_TWSI_IA_SHIFT;
+ }
+
+ octeon_i2c_hlc_int_clear(i2c);
+ octeon_i2c_writeq_flush(cmd, i2c->twsi_base + SW_TWSI);
+
+ ret = octeon_i2c_hlc_wait(i2c);
+ if (ret)
+ goto err;
+
+ cmd = __raw_readq(i2c->twsi_base + SW_TWSI);
+ if ((cmd & SW_TWSI_R) == 0)
+ return -EAGAIN;
+
+ for (i = 0, j = msgs[1].len - 1; i < msgs[1].len && i < 4; i++, j--)
+ msgs[1].buf[j] = (cmd >> (8 * i)) & 0xff;
+
+ if (msgs[1].len > 4) {
+ cmd = __raw_readq(i2c->twsi_base + SW_TWSI_EXT);
+ for (i = 0; i < msgs[1].len - 4 && i < 4; i++, j--)
+ msgs[1].buf[j] = (cmd >> (8 * i)) & 0xff;
+ }
+
+err:
+ return ret;
}
-static const struct i2c_algorithm octeon_i2c_algo = {
- .master_xfer = octeon_i2c_xfer,
- .functionality = octeon_i2c_functionality,
-};
+/* high-level-controller composite write+write, m[0]len<=2, m[1]len<=8 */
+static int octeon_i2c_hlc_comp_write(struct octeon_i2c *i2c, struct i2c_msg *msgs)
+{
+ bool set_ext = false;
+ int i, j, ret = 0;
+ u64 cmd, ext = 0;
-static struct i2c_adapter octeon_i2c_ops = {
- .owner = THIS_MODULE,
- .name = "OCTEON adapter",
- .algo = &octeon_i2c_algo,
- .timeout = HZ / 50,
-};
+ octeon_i2c_hlc_enable(i2c);
+
+ cmd = SW_TWSI_V | SW_TWSI_SOVR;
+ /* SIZE */
+ cmd |= (u64)(msgs[1].len - 1) << SW_TWSI_SIZE_SHIFT;
+ /* A */
+ cmd |= (u64)(msgs[0].addr & 0x7full) << SW_TWSI_ADDR_SHIFT;
+
+ if (msgs[0].flags & I2C_M_TEN)
+ cmd |= SW_TWSI_OP_10_IA;
+ else
+ cmd |= SW_TWSI_OP_7_IA;
+
+ if (msgs[0].len == 2) {
+ cmd |= SW_TWSI_EIA;
+ ext |= (u64)msgs[0].buf[0] << SW_TWSI_IA_SHIFT;
+ set_ext = true;
+ cmd |= (u64)msgs[0].buf[1] << SW_TWSI_IA_SHIFT;
+ } else {
+ cmd |= (u64)msgs[0].buf[0] << SW_TWSI_IA_SHIFT;
+ }
+
+ for (i = 0, j = msgs[1].len - 1; i < msgs[1].len && i < 4; i++, j--)
+ cmd |= (u64)msgs[1].buf[j] << (8 * i);
+
+ if (msgs[1].len > 4) {
+ for (i = 0; i < msgs[1].len - 4 && i < 4; i++, j--)
+ ext |= (u64)msgs[1].buf[j] << (8 * i);
+ set_ext = true;
+ }
+ if (set_ext)
+ octeon_i2c_writeq_flush(ext, i2c->twsi_base + SW_TWSI_EXT);
+
+ octeon_i2c_hlc_int_clear(i2c);
+ octeon_i2c_writeq_flush(cmd, i2c->twsi_base + SW_TWSI);
+
+ ret = octeon_i2c_hlc_wait(i2c);
+ if (ret)
+ goto err;
+
+ cmd = __raw_readq(i2c->twsi_base + SW_TWSI);
+ if ((cmd & SW_TWSI_R) == 0)
+ return -EAGAIN;
+
+ ret = octeon_i2c_check_status(i2c, false);
+
+err:
+ return ret;
+}
/* calculate and set clock divisors */
static void octeon_i2c_set_clock(struct octeon_i2c *i2c)
@@ -467,42 +787,342 @@
}
}
}
- octeon_i2c_write_sw(i2c, SW_TWSI_OP_TWSI_CLK, thp);
- octeon_i2c_write_sw(i2c, SW_TWSI_EOP_TWSI_CLKCTL, (mdiv << 3) | ndiv);
+ octeon_i2c_reg_write(i2c, SW_TWSI_OP_TWSI_CLK, thp);
+ octeon_i2c_reg_write(i2c, SW_TWSI_EOP_TWSI_CLKCTL, (mdiv << 3) | ndiv);
}
static int octeon_i2c_init_lowlevel(struct octeon_i2c *i2c)
{
- u8 status;
+ u8 status = 0;
int tries;
- /* disable high level controller, enable bus access */
- octeon_i2c_write_sw(i2c, SW_TWSI_EOP_TWSI_CTL, TWSI_CTL_ENAB);
-
/* reset controller */
- octeon_i2c_write_sw(i2c, SW_TWSI_EOP_TWSI_RST, 0);
+ octeon_i2c_reg_write(i2c, SW_TWSI_EOP_TWSI_RST, 0);
- for (tries = 10; tries; tries--) {
+ for (tries = 10; tries && status != STAT_IDLE; tries--) {
udelay(1);
- status = octeon_i2c_read_sw(i2c, SW_TWSI_EOP_TWSI_STAT);
+ status = octeon_i2c_stat_read(i2c);
if (status == STAT_IDLE)
- return 0;
+ break;
}
- dev_err(i2c->dev, "%s: TWSI_RST failed! (0x%x)\n", __func__, status);
- return -EIO;
+
+ if (status != STAT_IDLE) {
+ dev_err(i2c->dev, "%s: TWSI_RST failed! (0x%x)\n",
+ __func__, status);
+ return -EIO;
+ }
+
+ /* toggle twice to force both teardowns */
+ octeon_i2c_hlc_enable(i2c);
+ octeon_i2c_hlc_disable(i2c);
+ return 0;
}
+static int octeon_i2c_recovery(struct octeon_i2c *i2c)
+{
+ int ret;
+
+ ret = i2c_recover_bus(&i2c->adap);
+ if (ret)
+ /* recover failed, try hardware re-init */
+ ret = octeon_i2c_init_lowlevel(i2c);
+ return ret;
+}
+
+/**
+ * octeon_i2c_start - send START to the bus
+ * @i2c: The struct octeon_i2c
+ *
+ * Returns 0 on success, otherwise a negative errno.
+ */
+static int octeon_i2c_start(struct octeon_i2c *i2c)
+{
+ int ret;
+ u8 stat;
+
+ octeon_i2c_hlc_disable(i2c);
+
+ octeon_i2c_ctl_write(i2c, TWSI_CTL_ENAB | TWSI_CTL_STA);
+ ret = octeon_i2c_wait(i2c);
+ if (ret)
+ goto error;
+
+ stat = octeon_i2c_stat_read(i2c);
+ if (stat == STAT_START || stat == STAT_REP_START)
+ /* START successful, bail out */
+ return 0;
+
+error:
+ /* START failed, try to recover */
+ ret = octeon_i2c_recovery(i2c);
+ return (ret) ? ret : -EAGAIN;
+}
+
+/* send STOP to the bus */
+static void octeon_i2c_stop(struct octeon_i2c *i2c)
+{
+ octeon_i2c_ctl_write(i2c, TWSI_CTL_ENAB | TWSI_CTL_STP);
+}
+
+/**
+ * octeon_i2c_write - send data to the bus via low-level controller
+ * @i2c: The struct octeon_i2c
+ * @target: Target address
+ * @data: Pointer to the data to be sent
+ * @length: Length of the data
+ *
+ * The address is sent over the bus, then the data.
+ *
+ * Returns 0 on success, otherwise a negative errno.
+ */
+static int octeon_i2c_write(struct octeon_i2c *i2c, int target,
+ const u8 *data, int length)
+{
+ int i, result;
+
+ octeon_i2c_data_write(i2c, target << 1);
+ octeon_i2c_ctl_write(i2c, TWSI_CTL_ENAB);
+
+ result = octeon_i2c_wait(i2c);
+ if (result)
+ return result;
+
+ for (i = 0; i < length; i++) {
+ result = octeon_i2c_check_status(i2c, false);
+ if (result)
+ return result;
+
+ octeon_i2c_data_write(i2c, data[i]);
+ octeon_i2c_ctl_write(i2c, TWSI_CTL_ENAB);
+
+ result = octeon_i2c_wait(i2c);
+ if (result)
+ return result;
+ }
+
+ return 0;
+}
+
+/**
+ * octeon_i2c_read - receive data from the bus via low-level controller
+ * @i2c: The struct octeon_i2c
+ * @target: Target address
+ * @data: Pointer to the location to store the data
+ * @rlength: Length of the data
+ * @recv_len: flag for length byte
+ *
+ * The address is sent over the bus, then the data is read.
+ *
+ * Returns 0 on success, otherwise a negative errno.
+ */
+static int octeon_i2c_read(struct octeon_i2c *i2c, int target,
+ u8 *data, u16 *rlength, bool recv_len)
+{
+ int i, result, length = *rlength;
+ bool final_read = false;
+
+ octeon_i2c_data_write(i2c, (target << 1) | 1);
+ octeon_i2c_ctl_write(i2c, TWSI_CTL_ENAB);
+
+ result = octeon_i2c_wait(i2c);
+ if (result)
+ return result;
+
+ /* address OK ? */
+ result = octeon_i2c_check_status(i2c, false);
+ if (result)
+ return result;
+
+ for (i = 0; i < length; i++) {
+ /* for the last byte TWSI_CTL_AAK must not be set */
+ if (i + 1 == length)
+ final_read = true;
+
+ /* clear iflg to allow next event */
+ if (final_read)
+ octeon_i2c_ctl_write(i2c, TWSI_CTL_ENAB);
+ else
+ octeon_i2c_ctl_write(i2c, TWSI_CTL_ENAB | TWSI_CTL_AAK);
+
+ result = octeon_i2c_wait(i2c);
+ if (result)
+ return result;
+
+ data[i] = octeon_i2c_data_read(i2c);
+ if (recv_len && i == 0) {
+ if (data[i] > I2C_SMBUS_BLOCK_MAX + 1) {
+ dev_err(i2c->dev,
+ "%s: read len > I2C_SMBUS_BLOCK_MAX %d\n",
+ __func__, data[i]);
+ return -EPROTO;
+ }
+ length += data[i];
+ }
+
+ result = octeon_i2c_check_status(i2c, final_read);
+ if (result)
+ return result;
+ }
+ *rlength = length;
+ return 0;
+}
+
+/**
+ * octeon_i2c_xfer - The driver's master_xfer function
+ * @adap: Pointer to the i2c_adapter structure
+ * @msgs: Pointer to the messages to be processed
+ * @num: Length of the MSGS array
+ *
+ * Returns the number of messages processed, or a negative errno on failure.
+ */
+static int octeon_i2c_xfer(struct i2c_adapter *adap, struct i2c_msg *msgs,
+ int num)
+{
+ struct octeon_i2c *i2c = i2c_get_adapdata(adap);
+ int i, ret = 0;
+
+ if (num == 1) {
+ if (msgs[0].len > 0 && msgs[0].len <= 8) {
+ if (msgs[0].flags & I2C_M_RD)
+ ret = octeon_i2c_hlc_read(i2c, msgs);
+ else
+ ret = octeon_i2c_hlc_write(i2c, msgs);
+ goto out;
+ }
+ } else if (num == 2) {
+ if ((msgs[0].flags & I2C_M_RD) == 0 &&
+ (msgs[1].flags & I2C_M_RECV_LEN) == 0 &&
+ msgs[0].len > 0 && msgs[0].len <= 2 &&
+ msgs[1].len > 0 && msgs[1].len <= 8 &&
+ msgs[0].addr == msgs[1].addr) {
+ if (msgs[1].flags & I2C_M_RD)
+ ret = octeon_i2c_hlc_comp_read(i2c, msgs);
+ else
+ ret = octeon_i2c_hlc_comp_write(i2c, msgs);
+ goto out;
+ }
+ }
+
+ for (i = 0; ret == 0 && i < num; i++) {
+ struct i2c_msg *pmsg = &msgs[i];
+
+ /* zero-length messages are not supported */
+ if (!pmsg->len) {
+ ret = -EOPNOTSUPP;
+ break;
+ }
+
+ ret = octeon_i2c_start(i2c);
+ if (ret)
+ return ret;
+
+ if (pmsg->flags & I2C_M_RD)
+ ret = octeon_i2c_read(i2c, pmsg->addr, pmsg->buf,
+ &pmsg->len, pmsg->flags & I2C_M_RECV_LEN);
+ else
+ ret = octeon_i2c_write(i2c, pmsg->addr, pmsg->buf,
+ pmsg->len);
+ }
+ octeon_i2c_stop(i2c);
+out:
+ return (ret != 0) ? ret : num;
+}
+
+static int octeon_i2c_get_scl(struct i2c_adapter *adap)
+{
+ struct octeon_i2c *i2c = i2c_get_adapdata(adap);
+ u64 state;
+
+ state = octeon_i2c_read_int(i2c);
+ return state & TWSI_INT_SCL;
+}
+
+static void octeon_i2c_set_scl(struct i2c_adapter *adap, int val)
+{
+ struct octeon_i2c *i2c = i2c_get_adapdata(adap);
+
+ octeon_i2c_write_int(i2c, TWSI_INT_SCL_OVR);
+}
+
+static int octeon_i2c_get_sda(struct i2c_adapter *adap)
+{
+ struct octeon_i2c *i2c = i2c_get_adapdata(adap);
+ u64 state;
+
+ state = octeon_i2c_read_int(i2c);
+ return state & TWSI_INT_SDA;
+}
+
+static void octeon_i2c_prepare_recovery(struct i2c_adapter *adap)
+{
+ struct octeon_i2c *i2c = i2c_get_adapdata(adap);
+
+ /*
+ * The stop resets the state machine, does not _transmit_ STOP unless
+ * engine was active.
+ */
+ octeon_i2c_stop(i2c);
+
+ octeon_i2c_hlc_disable(i2c);
+ octeon_i2c_write_int(i2c, 0);
+}
+
+static void octeon_i2c_unprepare_recovery(struct i2c_adapter *adap)
+{
+ struct octeon_i2c *i2c = i2c_get_adapdata(adap);
+
+ octeon_i2c_write_int(i2c, 0);
+}
+
+static struct i2c_bus_recovery_info octeon_i2c_recovery_info = {
+ .recover_bus = i2c_generic_scl_recovery,
+ .get_scl = octeon_i2c_get_scl,
+ .set_scl = octeon_i2c_set_scl,
+ .get_sda = octeon_i2c_get_sda,
+ .prepare_recovery = octeon_i2c_prepare_recovery,
+ .unprepare_recovery = octeon_i2c_unprepare_recovery,
+};
+
+static u32 octeon_i2c_functionality(struct i2c_adapter *adap)
+{
+ return I2C_FUNC_I2C | (I2C_FUNC_SMBUS_EMUL & ~I2C_FUNC_SMBUS_QUICK) |
+ I2C_FUNC_SMBUS_READ_BLOCK_DATA | I2C_SMBUS_BLOCK_PROC_CALL;
+}
+
+static const struct i2c_algorithm octeon_i2c_algo = {
+ .master_xfer = octeon_i2c_xfer,
+ .functionality = octeon_i2c_functionality,
+};
+
+static struct i2c_adapter octeon_i2c_ops = {
+ .owner = THIS_MODULE,
+ .name = "OCTEON adapter",
+ .algo = &octeon_i2c_algo,
+};
+
static int octeon_i2c_probe(struct platform_device *pdev)
{
struct device_node *node = pdev->dev.of_node;
+ int irq, result = 0, hlc_irq = 0;
struct resource *res_mem;
struct octeon_i2c *i2c;
- int irq, result = 0;
+ bool cn78xx_style;
- /* All adaptors have an irq. */
- irq = platform_get_irq(pdev, 0);
- if (irq < 0)
- return irq;
+ cn78xx_style = of_device_is_compatible(node, "cavium,octeon-7890-twsi");
+ if (cn78xx_style) {
+ hlc_irq = platform_get_irq(pdev, 0);
+ if (hlc_irq < 0)
+ return hlc_irq;
+
+ irq = platform_get_irq(pdev, 2);
+ if (irq < 0)
+ return irq;
+ } else {
+ /* All adaptors have an irq. */
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0)
+ return irq;
+ }
i2c = devm_kzalloc(&pdev->dev, sizeof(*i2c), GFP_KERNEL);
if (!i2c) {
@@ -537,6 +1157,31 @@
i2c->irq = irq;
+ if (cn78xx_style) {
+ i2c->hlc_irq = hlc_irq;
+
+ i2c->int_enable = octeon_i2c_int_enable78;
+ i2c->int_disable = octeon_i2c_int_disable78;
+ i2c->hlc_int_enable = octeon_i2c_hlc_int_enable78;
+ i2c->hlc_int_disable = octeon_i2c_hlc_int_disable78;
+
+ irq_set_status_flags(i2c->irq, IRQ_NOAUTOEN);
+ irq_set_status_flags(i2c->hlc_irq, IRQ_NOAUTOEN);
+
+ result = devm_request_irq(&pdev->dev, i2c->hlc_irq,
+ octeon_i2c_hlc_isr78, 0,
+ DRV_NAME, i2c);
+ if (result < 0) {
+ dev_err(i2c->dev, "failed to attach interrupt\n");
+ goto out;
+ }
+ } else {
+ i2c->int_enable = octeon_i2c_int_enable;
+ i2c->int_disable = octeon_i2c_int_disable;
+ i2c->hlc_int_enable = octeon_i2c_hlc_int_enable;
+ i2c->hlc_int_disable = octeon_i2c_int_disable;
+ }
+
result = devm_request_irq(&pdev->dev, i2c->irq,
octeon_i2c_isr, 0, DRV_NAME, i2c);
if (result < 0) {
@@ -544,6 +1189,9 @@
goto out;
}
+ if (OCTEON_IS_MODEL(OCTEON_CN38XX))
+ i2c->broken_irq_check = true;
+
result = octeon_i2c_init_lowlevel(i2c);
if (result) {
dev_err(i2c->dev, "init low level failed\n");
@@ -553,6 +1201,9 @@
octeon_i2c_set_clock(i2c);
i2c->adap = octeon_i2c_ops;
+ i2c->adap.timeout = msecs_to_jiffies(2);
+ i2c->adap.retries = 5;
+ i2c->adap.bus_recovery_info = &octeon_i2c_recovery_info;
i2c->adap.dev.parent = &pdev->dev;
i2c->adap.dev.of_node = node;
i2c_set_adapdata(&i2c->adap, i2c);
@@ -580,6 +1231,7 @@
static const struct of_device_id octeon_i2c_match[] = {
{ .compatible = "cavium,octeon-3860-twsi", },
+ { .compatible = "cavium,octeon-7890-twsi", },
{},
};
MODULE_DEVICE_TABLE(of, octeon_i2c_match);
diff --git a/drivers/i2c/busses/i2c-omap.c b/drivers/i2c/busses/i2c-omap.c
index 13c4529..ab1279b 100644
--- a/drivers/i2c/busses/i2c-omap.c
+++ b/drivers/i2c/busses/i2c-omap.c
@@ -185,7 +185,6 @@
#define OMAP_I2C_IP_V2_INTERRUPTS_MASK 0x6FFF
struct omap_i2c_dev {
- spinlock_t lock; /* IRQ synchronization */
struct device *dev;
void __iomem *base; /* virtual */
int irq;
@@ -995,15 +994,12 @@
u16 mask;
u16 stat;
- spin_lock(&omap->lock);
- mask = omap_i2c_read_reg(omap, OMAP_I2C_IE_REG);
stat = omap_i2c_read_reg(omap, OMAP_I2C_STAT_REG);
+ mask = omap_i2c_read_reg(omap, OMAP_I2C_IE_REG);
if (stat & mask)
ret = IRQ_WAKE_THREAD;
- spin_unlock(&omap->lock);
-
return ret;
}
@@ -1011,12 +1007,10 @@
omap_i2c_isr_thread(int this_irq, void *dev_id)
{
struct omap_i2c_dev *omap = dev_id;
- unsigned long flags;
u16 bits;
u16 stat;
int err = 0, count = 0;
- spin_lock_irqsave(&omap->lock, flags);
do {
bits = omap_i2c_read_reg(omap, OMAP_I2C_IE_REG);
stat = omap_i2c_read_reg(omap, OMAP_I2C_STAT_REG);
@@ -1142,8 +1136,6 @@
omap_i2c_complete_cmd(omap, err);
out:
- spin_unlock_irqrestore(&omap->lock, flags);
-
return IRQ_HANDLED;
}
@@ -1330,8 +1322,6 @@
omap->dev = &pdev->dev;
omap->irq = irq;
- spin_lock_init(&omap->lock);
-
platform_set_drvdata(pdev, omap);
init_completion(&omap->cmd_complete);
diff --git a/drivers/i2c/busses/i2c-powermac.c b/drivers/i2c/busses/i2c-powermac.c
index 6abcf69..b0d9dee 100644
--- a/drivers/i2c/busses/i2c-powermac.c
+++ b/drivers/i2c/busses/i2c-powermac.c
@@ -150,13 +150,11 @@
{
struct pmac_i2c_bus *bus = i2c_get_adapdata(adap);
int rc = 0;
- int read;
int addrdir;
if (msgs->flags & I2C_M_TEN)
return -EINVAL;
- read = (msgs->flags & I2C_M_RD) != 0;
- addrdir = (msgs->addr << 1) | read;
+ addrdir = i2c_8bit_addr_from_msg(msgs);
rc = pmac_i2c_open(bus, 0);
if (rc) {
diff --git a/drivers/i2c/busses/i2c-qup.c b/drivers/i2c/busses/i2c-qup.c
index 23eaabb..cc6439a 100644
--- a/drivers/i2c/busses/i2c-qup.c
+++ b/drivers/i2c/busses/i2c-qup.c
@@ -515,7 +515,7 @@
static int qup_i2c_set_tags(u8 *tags, struct qup_i2c_dev *qup,
struct i2c_msg *msg, int is_dma)
{
- u16 addr = (msg->addr << 1) | ((msg->flags & I2C_M_RD) == I2C_M_RD);
+ u16 addr = i2c_8bit_addr_from_msg(msg);
int len = 0;
int data_len;
diff --git a/drivers/i2c/busses/i2c-rcar.c b/drivers/i2c/busses/i2c-rcar.c
index 68ecb56..9aca1b4 100644
--- a/drivers/i2c/busses/i2c-rcar.c
+++ b/drivers/i2c/busses/i2c-rcar.c
@@ -21,6 +21,8 @@
*/
#include <linux/clk.h>
#include <linux/delay.h>
+#include <linux/dmaengine.h>
+#include <linux/dma-mapping.h>
#include <linux/err.h>
#include <linux/interrupt.h>
#include <linux/io.h>
@@ -43,6 +45,8 @@
#define ICSAR 0x1C /* slave address */
#define ICMAR 0x20 /* master address */
#define ICRXTX 0x24 /* data port */
+#define ICDMAER 0x3c /* DMA enable */
+#define ICFBSCR 0x38 /* first bit setup cycle */
/* ICSCR */
#define SDBS (1 << 3) /* slave data buffer select */
@@ -78,6 +82,16 @@
#define MDR (1 << 1)
#define MAT (1 << 0) /* slave addr xfer done */
+/* ICDMAER */
+#define RSDMAE (1 << 3) /* DMA Slave Received Enable */
+#define TSDMAE (1 << 2) /* DMA Slave Transmitted Enable */
+#define RMDMAE (1 << 1) /* DMA Master Received Enable */
+#define TMDMAE (1 << 0) /* DMA Master Transmitted Enable */
+
+/* ICFBSCR */
+#define TCYC06 0x04 /* 6*Tcyc delay 1st bit between SDA and SCL */
+#define TCYC17 0x0f /* 17*Tcyc delay 1st bit between SDA and SCL */
+
#define RCAR_BUS_PHASE_START (MDBS | MIE | ESG)
#define RCAR_BUS_PHASE_DATA (MDBS | MIE)
@@ -120,6 +134,12 @@
u32 flags;
enum rcar_i2c_type devtype;
struct i2c_client *slave;
+
+ struct resource *res;
+ struct dma_chan *dma_tx;
+ struct dma_chan *dma_rx;
+ struct scatterlist sg;
+ enum dma_data_direction dma_direction;
};
#define rcar_i2c_priv_to_dev(p) ((p)->adap.dev.parent)
@@ -287,6 +307,118 @@
/*
* interrupt functions
*/
+static void rcar_i2c_dma_unmap(struct rcar_i2c_priv *priv)
+{
+ struct dma_chan *chan = priv->dma_direction == DMA_FROM_DEVICE
+ ? priv->dma_rx : priv->dma_tx;
+
+ /* Disable DMA Master Received/Transmitted */
+ rcar_i2c_write(priv, ICDMAER, 0);
+
+ /* Reset default delay */
+ rcar_i2c_write(priv, ICFBSCR, TCYC06);
+
+ dma_unmap_single(chan->device->dev, sg_dma_address(&priv->sg),
+ priv->msg->len, priv->dma_direction);
+
+ priv->dma_direction = DMA_NONE;
+}
+
+static void rcar_i2c_cleanup_dma(struct rcar_i2c_priv *priv)
+{
+ if (priv->dma_direction == DMA_NONE)
+ return;
+ else if (priv->dma_direction == DMA_FROM_DEVICE)
+ dmaengine_terminate_all(priv->dma_rx);
+ else if (priv->dma_direction == DMA_TO_DEVICE)
+ dmaengine_terminate_all(priv->dma_tx);
+
+ rcar_i2c_dma_unmap(priv);
+}
+
+static void rcar_i2c_dma_callback(void *data)
+{
+ struct rcar_i2c_priv *priv = data;
+
+ priv->pos += sg_dma_len(&priv->sg);
+
+ rcar_i2c_dma_unmap(priv);
+}
+
+static void rcar_i2c_dma(struct rcar_i2c_priv *priv)
+{
+ struct device *dev = rcar_i2c_priv_to_dev(priv);
+ struct i2c_msg *msg = priv->msg;
+ bool read = msg->flags & I2C_M_RD;
+ enum dma_data_direction dir = read ? DMA_FROM_DEVICE : DMA_TO_DEVICE;
+ struct dma_chan *chan = read ? priv->dma_rx : priv->dma_tx;
+ struct dma_async_tx_descriptor *txdesc;
+ dma_addr_t dma_addr;
+ dma_cookie_t cookie;
+ unsigned char *buf;
+ int len;
+
+ /* Do not use DMA if it's not available or for messages < 8 bytes */
+ if (IS_ERR(chan) || msg->len < 8)
+ return;
+
+ if (read) {
+ /*
+ * The last two bytes needs to be fetched using PIO in
+ * order for the STOP phase to work.
+ */
+ buf = priv->msg->buf;
+ len = priv->msg->len - 2;
+ } else {
+ /*
+ * First byte in message was sent using PIO.
+ */
+ buf = priv->msg->buf + 1;
+ len = priv->msg->len - 1;
+ }
+
+ dma_addr = dma_map_single(chan->device->dev, buf, len, dir);
+ if (dma_mapping_error(dev, dma_addr)) {
+ dev_dbg(dev, "dma map failed, using PIO\n");
+ return;
+ }
+
+ sg_dma_len(&priv->sg) = len;
+ sg_dma_address(&priv->sg) = dma_addr;
+
+ priv->dma_direction = dir;
+
+ txdesc = dmaengine_prep_slave_sg(chan, &priv->sg, 1,
+ read ? DMA_DEV_TO_MEM : DMA_MEM_TO_DEV,
+ DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
+ if (!txdesc) {
+ dev_dbg(dev, "dma prep slave sg failed, using PIO\n");
+ rcar_i2c_cleanup_dma(priv);
+ return;
+ }
+
+ txdesc->callback = rcar_i2c_dma_callback;
+ txdesc->callback_param = priv;
+
+ cookie = dmaengine_submit(txdesc);
+ if (dma_submit_error(cookie)) {
+ dev_dbg(dev, "submitting dma failed, using PIO\n");
+ rcar_i2c_cleanup_dma(priv);
+ return;
+ }
+
+ /* Set delay for DMA operations */
+ rcar_i2c_write(priv, ICFBSCR, TCYC17);
+
+ /* Enable DMA Master Received/Transmitted */
+ if (read)
+ rcar_i2c_write(priv, ICDMAER, RMDMAE);
+ else
+ rcar_i2c_write(priv, ICDMAER, TMDMAE);
+
+ dma_async_issue_pending(chan);
+}
+
static void rcar_i2c_irq_send(struct rcar_i2c_priv *priv, u32 msr)
{
struct i2c_msg *msg = priv->msg;
@@ -306,6 +438,12 @@
rcar_i2c_write(priv, ICRXTX, msg->buf[priv->pos]);
priv->pos++;
+ /*
+ * Try to use DMA to transmit the rest of the data if
+ * address transfer pashe just finished.
+ */
+ if (msr & MAT)
+ rcar_i2c_dma(priv);
} else {
/*
* The last data was pushed to ICRXTX on _PREV_ empty irq.
@@ -340,7 +478,11 @@
return;
if (msr & MAT) {
- /* Address transfer phase finished, but no data at this point. */
+ /*
+ * Address transfer phase finished, but no data at this point.
+ * Try to use DMA to receive data.
+ */
+ rcar_i2c_dma(priv);
} else if (priv->pos < msg->len) {
/* get received data */
msg->buf[priv->pos] = rcar_i2c_read(priv, ICRXTX);
@@ -472,6 +614,81 @@
return IRQ_HANDLED;
}
+static struct dma_chan *rcar_i2c_request_dma_chan(struct device *dev,
+ enum dma_transfer_direction dir,
+ dma_addr_t port_addr)
+{
+ struct dma_chan *chan;
+ struct dma_slave_config cfg;
+ char *chan_name = dir == DMA_MEM_TO_DEV ? "tx" : "rx";
+ int ret;
+
+ chan = dma_request_slave_channel_reason(dev, chan_name);
+ if (IS_ERR(chan)) {
+ ret = PTR_ERR(chan);
+ dev_dbg(dev, "request_channel failed for %s (%d)\n",
+ chan_name, ret);
+ return chan;
+ }
+
+ memset(&cfg, 0, sizeof(cfg));
+ cfg.direction = dir;
+ if (dir == DMA_MEM_TO_DEV) {
+ cfg.dst_addr = port_addr;
+ cfg.dst_addr_width = DMA_SLAVE_BUSWIDTH_1_BYTE;
+ } else {
+ cfg.src_addr = port_addr;
+ cfg.src_addr_width = DMA_SLAVE_BUSWIDTH_1_BYTE;
+ }
+
+ ret = dmaengine_slave_config(chan, &cfg);
+ if (ret) {
+ dev_dbg(dev, "slave_config failed for %s (%d)\n",
+ chan_name, ret);
+ dma_release_channel(chan);
+ return ERR_PTR(ret);
+ }
+
+ dev_dbg(dev, "got DMA channel for %s\n", chan_name);
+ return chan;
+}
+
+static void rcar_i2c_request_dma(struct rcar_i2c_priv *priv,
+ struct i2c_msg *msg)
+{
+ struct device *dev = rcar_i2c_priv_to_dev(priv);
+ bool read;
+ struct dma_chan *chan;
+ enum dma_transfer_direction dir;
+
+ read = msg->flags & I2C_M_RD;
+
+ chan = read ? priv->dma_rx : priv->dma_tx;
+ if (PTR_ERR(chan) != -EPROBE_DEFER)
+ return;
+
+ dir = read ? DMA_DEV_TO_MEM : DMA_MEM_TO_DEV;
+ chan = rcar_i2c_request_dma_chan(dev, dir, priv->res->start + ICRXTX);
+
+ if (read)
+ priv->dma_rx = chan;
+ else
+ priv->dma_tx = chan;
+}
+
+static void rcar_i2c_release_dma(struct rcar_i2c_priv *priv)
+{
+ if (!IS_ERR(priv->dma_tx)) {
+ dma_release_channel(priv->dma_tx);
+ priv->dma_tx = ERR_PTR(-EPROBE_DEFER);
+ }
+
+ if (!IS_ERR(priv->dma_rx)) {
+ dma_release_channel(priv->dma_rx);
+ priv->dma_rx = ERR_PTR(-EPROBE_DEFER);
+ }
+}
+
static int rcar_i2c_master_xfer(struct i2c_adapter *adap,
struct i2c_msg *msgs,
int num)
@@ -493,6 +710,7 @@
ret = -EOPNOTSUPP;
goto out;
}
+ rcar_i2c_request_dma(priv, msgs + i);
}
/* init first message */
@@ -504,6 +722,7 @@
time_left = wait_event_timeout(priv->wait, priv->flags & ID_DONE,
num * adap->timeout);
if (!time_left) {
+ rcar_i2c_cleanup_dma(priv);
rcar_i2c_init(priv);
ret = -ETIMEDOUT;
} else if (priv->flags & ID_NACK) {
@@ -591,7 +810,6 @@
{
struct rcar_i2c_priv *priv;
struct i2c_adapter *adap;
- struct resource *res;
struct device *dev = &pdev->dev;
struct i2c_timings i2c_t;
int irq, ret;
@@ -606,8 +824,9 @@
return PTR_ERR(priv->clk);
}
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- priv->io = devm_ioremap_resource(dev, res);
+ priv->res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+
+ priv->io = devm_ioremap_resource(dev, priv->res);
if (IS_ERR(priv->io))
return PTR_ERR(priv->io);
@@ -626,6 +845,11 @@
i2c_parse_fw_timings(dev, &i2c_t, false);
+ /* Init DMA */
+ sg_init_table(&priv->sg, 1);
+ priv->dma_direction = DMA_NONE;
+ priv->dma_rx = priv->dma_tx = ERR_PTR(-EPROBE_DEFER);
+
pm_runtime_enable(dev);
pm_runtime_get_sync(dev);
ret = rcar_i2c_clock_calculate(priv, &i2c_t);
@@ -673,6 +897,7 @@
struct device *dev = &pdev->dev;
i2c_del_adapter(&priv->adap);
+ rcar_i2c_release_dma(priv);
if (priv->flags & ID_P_PM_BLOCKED)
pm_runtime_put(dev);
pm_runtime_disable(dev);
diff --git a/drivers/i2c/busses/i2c-rk3x.c b/drivers/i2c/busses/i2c-rk3x.c
index 3dcc5f3..80bed02 100644
--- a/drivers/i2c/busses/i2c-rk3x.c
+++ b/drivers/i2c/busses/i2c-rk3x.c
@@ -101,10 +101,7 @@
struct notifier_block clk_rate_nb;
/* Settings */
- unsigned int scl_frequency;
- unsigned int scl_rise_ns;
- unsigned int scl_fall_ns;
- unsigned int sda_fall_ns;
+ struct i2c_timings t;
/* Synchronization & notification */
spinlock_t lock;
@@ -437,10 +434,7 @@
* Calculate divider values for desired SCL frequency
*
* @clk_rate: I2C input clock rate
- * @scl_rate: Desired SCL rate
- * @scl_rise_ns: How many ns it takes for SCL to rise.
- * @scl_fall_ns: How many ns it takes for SCL to fall.
- * @sda_fall_ns: How many ns it takes for SDA to fall.
+ * @t: Known I2C timing information.
* @div_low: Divider output for low
* @div_high: Divider output for high
*
@@ -448,11 +442,10 @@
* a best-effort divider value is returned in divs. If the target rate is
* too high, we silently use the highest possible rate.
*/
-static int rk3x_i2c_calc_divs(unsigned long clk_rate, unsigned long scl_rate,
- unsigned long scl_rise_ns,
- unsigned long scl_fall_ns,
- unsigned long sda_fall_ns,
- unsigned long *div_low, unsigned long *div_high)
+static int rk3x_i2c_calc_divs(unsigned long clk_rate,
+ struct i2c_timings *t,
+ unsigned long *div_low,
+ unsigned long *div_high)
{
unsigned long spec_min_low_ns, spec_min_high_ns;
unsigned long spec_setup_start, spec_max_data_hold_ns;
@@ -472,12 +465,12 @@
int ret = 0;
/* Only support standard-mode and fast-mode */
- if (WARN_ON(scl_rate > 400000))
- scl_rate = 400000;
+ if (WARN_ON(t->bus_freq_hz > 400000))
+ t->bus_freq_hz = 400000;
/* prevent scl_rate_khz from becoming 0 */
- if (WARN_ON(scl_rate < 1000))
- scl_rate = 1000;
+ if (WARN_ON(t->bus_freq_hz < 1000))
+ t->bus_freq_hz = 1000;
/*
* min_low_ns: The minimum number of ns we need to hold low to
@@ -491,7 +484,7 @@
* This is because the i2c host on Rockchip holds the data line
* for half the low time.
*/
- if (scl_rate <= 100000) {
+ if (t->bus_freq_hz <= 100000) {
/* Standard-mode */
spec_min_low_ns = 4700;
spec_setup_start = 4700;
@@ -506,7 +499,7 @@
spec_max_data_hold_ns = 900;
data_hold_buffer_ns = 50;
}
- min_high_ns = scl_rise_ns + spec_min_high_ns;
+ min_high_ns = t->scl_rise_ns + spec_min_high_ns;
/*
* Timings for repeated start:
@@ -517,18 +510,18 @@
* we meet tSU;STA and tHD;STA times.
*/
min_high_ns = max(min_high_ns,
- DIV_ROUND_UP((scl_rise_ns + spec_setup_start) * 1000, 875));
+ DIV_ROUND_UP((t->scl_rise_ns + spec_setup_start) * 1000, 875));
min_high_ns = max(min_high_ns,
- DIV_ROUND_UP((scl_rise_ns + spec_setup_start +
- sda_fall_ns + spec_min_high_ns), 2));
+ DIV_ROUND_UP((t->scl_rise_ns + spec_setup_start +
+ t->sda_fall_ns + spec_min_high_ns), 2));
- min_low_ns = scl_fall_ns + spec_min_low_ns;
+ min_low_ns = t->scl_fall_ns + spec_min_low_ns;
max_low_ns = spec_max_data_hold_ns * 2 - data_hold_buffer_ns;
min_total_ns = min_low_ns + min_high_ns;
/* Adjust to avoid overflow */
clk_rate_khz = DIV_ROUND_UP(clk_rate, 1000);
- scl_rate_khz = scl_rate / 1000;
+ scl_rate_khz = t->bus_freq_hz / 1000;
/*
* We need the total div to be >= this number
@@ -616,14 +609,13 @@
static void rk3x_i2c_adapt_div(struct rk3x_i2c *i2c, unsigned long clk_rate)
{
+ struct i2c_timings *t = &i2c->t;
unsigned long div_low, div_high;
u64 t_low_ns, t_high_ns;
int ret;
- ret = rk3x_i2c_calc_divs(clk_rate, i2c->scl_frequency, i2c->scl_rise_ns,
- i2c->scl_fall_ns, i2c->sda_fall_ns,
- &div_low, &div_high);
- WARN_ONCE(ret != 0, "Could not reach SCL freq %u", i2c->scl_frequency);
+ ret = rk3x_i2c_calc_divs(clk_rate, t, &div_low, &div_high);
+ WARN_ONCE(ret != 0, "Could not reach SCL freq %u", t->bus_freq_hz);
clk_enable(i2c->clk);
i2c_writel(i2c, (div_high << 16) | (div_low & 0xffff), REG_CLKDIV);
@@ -634,7 +626,7 @@
dev_dbg(i2c->dev,
"CLK %lukhz, Req %uns, Act low %lluns high %lluns\n",
clk_rate / 1000,
- 1000000000 / i2c->scl_frequency,
+ 1000000000 / t->bus_freq_hz,
t_low_ns, t_high_ns);
}
@@ -664,9 +656,7 @@
switch (event) {
case PRE_RATE_CHANGE:
- if (rk3x_i2c_calc_divs(ndata->new_rate, i2c->scl_frequency,
- i2c->scl_rise_ns, i2c->scl_fall_ns,
- i2c->sda_fall_ns,
+ if (rk3x_i2c_calc_divs(ndata->new_rate, &i2c->t,
&div_low, &div_high) != 0)
return NOTIFY_STOP;
@@ -880,37 +870,8 @@
match = of_match_node(rk3x_i2c_match, np);
i2c->soc_data = (struct rk3x_i2c_soc_data *)match->data;
- if (of_property_read_u32(pdev->dev.of_node, "clock-frequency",
- &i2c->scl_frequency)) {
- dev_info(&pdev->dev, "using default SCL frequency: %d\n",
- DEFAULT_SCL_RATE);
- i2c->scl_frequency = DEFAULT_SCL_RATE;
- }
-
- if (i2c->scl_frequency == 0 || i2c->scl_frequency > 400 * 1000) {
- dev_warn(&pdev->dev, "invalid SCL frequency specified.\n");
- dev_warn(&pdev->dev, "using default SCL frequency: %d\n",
- DEFAULT_SCL_RATE);
- i2c->scl_frequency = DEFAULT_SCL_RATE;
- }
-
- /*
- * Read rise and fall time from device tree. If not available use
- * the default maximum timing from the specification.
- */
- if (of_property_read_u32(pdev->dev.of_node, "i2c-scl-rising-time-ns",
- &i2c->scl_rise_ns)) {
- if (i2c->scl_frequency <= 100000)
- i2c->scl_rise_ns = 1000;
- else
- i2c->scl_rise_ns = 300;
- }
- if (of_property_read_u32(pdev->dev.of_node, "i2c-scl-falling-time-ns",
- &i2c->scl_fall_ns))
- i2c->scl_fall_ns = 300;
- if (of_property_read_u32(pdev->dev.of_node, "i2c-sda-falling-time-ns",
- &i2c->sda_fall_ns))
- i2c->sda_fall_ns = i2c->scl_fall_ns;
+ /* use common interface to get I2C timing properties */
+ i2c_parse_fw_timings(&pdev->dev, &i2c->t, true);
strlcpy(i2c->adap.name, "rk3x-i2c", sizeof(i2c->adap.name));
i2c->adap.owner = THIS_MODULE;
diff --git a/drivers/i2c/busses/i2c-s3c2410.c b/drivers/i2c/busses/i2c-s3c2410.c
index 362a6de5..38dc1ca 100644
--- a/drivers/i2c/busses/i2c-s3c2410.c
+++ b/drivers/i2c/busses/i2c-s3c2410.c
@@ -163,15 +163,14 @@
MODULE_DEVICE_TABLE(of, s3c24xx_i2c_match);
#endif
-/* s3c24xx_get_device_quirks
- *
+/*
* Get controller type either from device tree or platform device variant.
-*/
-
+ */
static inline kernel_ulong_t s3c24xx_get_device_quirks(struct platform_device *pdev)
{
if (pdev->dev.of_node) {
const struct of_device_id *match;
+
match = of_match_node(s3c24xx_i2c_match, pdev->dev.of_node);
return (kernel_ulong_t)match->data;
}
@@ -179,12 +178,10 @@
return platform_get_device_id(pdev)->driver_data;
}
-/* s3c24xx_i2c_master_complete
- *
- * complete the message and wake up the caller, using the given return code,
+/*
+ * Complete the message and wake up the caller, using the given return code,
* or zero to mean ok.
-*/
-
+ */
static inline void s3c24xx_i2c_master_complete(struct s3c24xx_i2c *i2c, int ret)
{
dev_dbg(i2c->dev, "master_complete %d\n", ret);
@@ -217,7 +214,6 @@
}
/* irq enable/disable functions */
-
static inline void s3c24xx_i2c_disable_irq(struct s3c24xx_i2c *i2c)
{
unsigned long tmp;
@@ -251,11 +247,9 @@
return false;
}
-/* s3c24xx_i2c_message_start
- *
+/*
* put the start of a message onto the bus
-*/
-
+ */
static void s3c24xx_i2c_message_start(struct s3c24xx_i2c *i2c,
struct i2c_msg *msg)
{
@@ -284,9 +278,10 @@
dev_dbg(i2c->dev, "START: %08lx to IICSTAT, %02x to DS\n", stat, addr);
writeb(addr, i2c->regs + S3C2410_IICDS);
- /* delay here to ensure the data byte has gotten onto the bus
- * before the transaction is started */
-
+ /*
+ * delay here to ensure the data byte has gotten onto the bus
+ * before the transaction is started
+ */
ndelay(i2c->tx_setup);
dev_dbg(i2c->dev, "iiccon, %08lx\n", iiccon);
@@ -361,50 +356,46 @@
s3c24xx_i2c_disable_irq(i2c);
}
-/* helper functions to determine the current state in the set of
- * messages we are sending */
+/*
+ * helper functions to determine the current state in the set of
+ * messages we are sending
+ */
-/* is_lastmsg()
- *
+/*
* returns TRUE if the current message is the last in the set
-*/
-
+ */
static inline int is_lastmsg(struct s3c24xx_i2c *i2c)
{
return i2c->msg_idx >= (i2c->msg_num - 1);
}
-/* is_msglast
- *
+/*
* returns TRUE if we this is the last byte in the current message
-*/
-
+ */
static inline int is_msglast(struct s3c24xx_i2c *i2c)
{
- /* msg->len is always 1 for the first byte of smbus block read.
+ /*
+ * msg->len is always 1 for the first byte of smbus block read.
* Actual length will be read from slave. More bytes will be
- * read according to the length then. */
+ * read according to the length then.
+ */
if (i2c->msg->flags & I2C_M_RECV_LEN && i2c->msg->len == 1)
return 0;
return i2c->msg_ptr == i2c->msg->len-1;
}
-/* is_msgend
- *
+/*
* returns TRUE if we reached the end of the current message
-*/
-
+ */
static inline int is_msgend(struct s3c24xx_i2c *i2c)
{
return i2c->msg_ptr >= i2c->msg->len;
}
-/* i2c_s3c_irq_nextbyte
- *
+/*
* process an interrupt and work out what to do
*/
-
static int i2c_s3c_irq_nextbyte(struct s3c24xx_i2c *i2c, unsigned long iicstat)
{
unsigned long tmp;
@@ -423,14 +414,13 @@
goto out_ack;
case STATE_START:
- /* last thing we did was send a start condition on the
+ /*
+ * last thing we did was send a start condition on the
* bus, or started a new i2c message
*/
-
if (iicstat & S3C2410_IICSTAT_LASTBIT &&
!(i2c->msg->flags & I2C_M_IGNORE_NAK)) {
/* ack was not received... */
-
dev_dbg(i2c->dev, "ack was not received\n");
s3c24xx_i2c_stop(i2c, -ENXIO);
goto out_ack;
@@ -441,9 +431,10 @@
else
i2c->state = STATE_WRITE;
- /* terminate the transfer if there is nothing to do
- * as this is used by the i2c probe to find devices. */
-
+ /*
+ * Terminate the transfer if there is nothing to do
+ * as this is used by the i2c probe to find devices.
+ */
if (is_lastmsg(i2c) && i2c->msg->len == 0) {
s3c24xx_i2c_stop(i2c, 0);
goto out_ack;
@@ -452,14 +443,16 @@
if (i2c->state == STATE_READ)
goto prepare_read;
- /* fall through to the write state, as we will need to
- * send a byte as well */
-
- case STATE_WRITE:
- /* we are writing data to the device... check for the
- * end of the message, and if so, work out what to do
+ /*
+ * fall through to the write state, as we will need to
+ * send a byte as well
*/
+ case STATE_WRITE:
+ /*
+ * we are writing data to the device... check for the
+ * end of the message, and if so, work out what to do
+ */
if (!(i2c->msg->flags & I2C_M_IGNORE_NAK)) {
if (iicstat & S3C2410_IICSTAT_LASTBIT) {
dev_dbg(i2c->dev, "WRITE: No Ack\n");
@@ -475,12 +468,13 @@
byte = i2c->msg->buf[i2c->msg_ptr++];
writeb(byte, i2c->regs + S3C2410_IICDS);
- /* delay after writing the byte to allow the
+ /*
+ * delay after writing the byte to allow the
* data setup time on the bus, as writing the
* data to the register causes the first bit
* to appear on SDA, and SCL will change as
- * soon as the interrupt is acknowledged */
-
+ * soon as the interrupt is acknowledged
+ */
ndelay(i2c->tx_setup);
} else if (!is_lastmsg(i2c)) {
@@ -496,10 +490,11 @@
if (i2c->msg->flags & I2C_M_NOSTART) {
if (i2c->msg->flags & I2C_M_RD) {
- /* cannot do this, the controller
+ /*
+ * cannot do this, the controller
* forces us to send a new START
- * when we change direction */
-
+ * when we change direction
+ */
s3c24xx_i2c_stop(i2c, -EINVAL);
}
@@ -512,17 +507,16 @@
} else {
/* send stop */
-
s3c24xx_i2c_stop(i2c, 0);
}
break;
case STATE_READ:
- /* we have a byte of data in the data register, do
+ /*
+ * we have a byte of data in the data register, do
* something with it, and then work out whether we are
* going to do any more read/write
*/
-
byte = readb(i2c->regs + S3C2410_IICDS);
i2c->msg->buf[i2c->msg_ptr++] = byte;
@@ -537,9 +531,10 @@
s3c24xx_i2c_disable_ack(i2c);
} else if (is_msgend(i2c)) {
- /* ok, we've read the entire buffer, see if there
- * is anything else we need to do */
-
+ /*
+ * ok, we've read the entire buffer, see if there
+ * is anything else we need to do
+ */
if (is_lastmsg(i2c)) {
/* last message, send stop and complete */
dev_dbg(i2c->dev, "READ: Send Stop\n");
@@ -568,11 +563,9 @@
return ret;
}
-/* s3c24xx_i2c_irq
- *
+/*
* top level IRQ servicing routine
-*/
-
+ */
static irqreturn_t s3c24xx_i2c_irq(int irqno, void *dev_id)
{
struct s3c24xx_i2c *i2c = dev_id;
@@ -595,9 +588,10 @@
goto out;
}
- /* pretty much this leaves us with the fact that we've
- * transmitted or received whatever byte we last sent */
-
+ /*
+ * pretty much this leaves us with the fact that we've
+ * transmitted or received whatever byte we last sent
+ */
i2c_s3c_irq_nextbyte(i2c, status);
out:
@@ -630,11 +624,9 @@
}
-/* s3c24xx_i2c_set_master
- *
+/*
* get the i2c bus for a master transaction
-*/
-
+ */
static int s3c24xx_i2c_set_master(struct s3c24xx_i2c *i2c)
{
unsigned long iicstat;
@@ -652,11 +644,9 @@
return -ETIMEDOUT;
}
-/* s3c24xx_i2c_wait_idle
- *
+/*
* wait for the i2c bus to become idle.
-*/
-
+ */
static void s3c24xx_i2c_wait_idle(struct s3c24xx_i2c *i2c)
{
unsigned long iicstat;
@@ -706,11 +696,9 @@
dev_warn(i2c->dev, "timeout waiting for bus idle\n");
}
-/* s3c24xx_i2c_doxfer
- *
+/*
* this starts an i2c transfer
-*/
-
+ */
static int s3c24xx_i2c_doxfer(struct s3c24xx_i2c *i2c,
struct i2c_msg *msgs, int num)
{
@@ -749,9 +737,10 @@
ret = i2c->msg_idx;
- /* having these next two as dev_err() makes life very
- * noisy when doing an i2cdetect */
-
+ /*
+ * Having these next two as dev_err() makes life very
+ * noisy when doing an i2cdetect
+ */
if (timeout == 0)
dev_dbg(i2c->dev, "timeout\n");
else if (ret != num)
@@ -771,12 +760,10 @@
return ret;
}
-/* s3c24xx_i2c_xfer
- *
+/*
* first port of call from the i2c bus code when an message needs
* transferring across the i2c bus.
-*/
-
+ */
static int s3c24xx_i2c_xfer(struct i2c_adapter *adap,
struct i2c_msg *msgs, int num)
{
@@ -814,17 +801,14 @@
}
/* i2c bus registration info */
-
static const struct i2c_algorithm s3c24xx_i2c_algorithm = {
.master_xfer = s3c24xx_i2c_xfer,
.functionality = s3c24xx_i2c_func,
};
-/* s3c24xx_i2c_calcdivisor
- *
+/*
* return the divisor settings for a given frequency
-*/
-
+ */
static int s3c24xx_i2c_calcdivisor(unsigned long clkin, unsigned int wanted,
unsigned int *div1, unsigned int *divs)
{
@@ -850,13 +834,11 @@
return clkin / (calc_divs * calc_div1);
}
-/* s3c24xx_i2c_clockrate
- *
+/*
* work out a divisor for the user requested frequency setting,
* either by the requested frequency, or scanning the acceptable
* range of frequencies until something is found
-*/
-
+ */
static int s3c24xx_i2c_clockrate(struct s3c24xx_i2c *i2c, unsigned int *got)
{
struct s3c2410_platform_i2c *pdata = i2c->pdata;
@@ -944,7 +926,7 @@
i2c_unlock_adapter(&i2c->adap);
if (ret < 0)
- dev_err(i2c->dev, "cannot find frequency\n");
+ dev_err(i2c->dev, "cannot find frequency (%d)\n", ret);
else
dev_info(i2c->dev, "setting freq %d\n", got);
}
@@ -995,7 +977,8 @@
ret = gpio_request(gpio, "i2c-bus");
if (ret) {
- dev_err(i2c->dev, "gpio [%d] request failed\n", gpio);
+ dev_err(i2c->dev, "gpio [%d] request failed (%d)\n",
+ gpio, ret);
goto free_gpio;
}
}
@@ -1028,11 +1011,9 @@
}
#endif
-/* s3c24xx_i2c_init
- *
+/*
* initialise the controller, set the IO lines and frequency
-*/
-
+ */
static int s3c24xx_i2c_init(struct s3c24xx_i2c *i2c)
{
struct s3c2410_platform_i2c *pdata;
@@ -1068,11 +1049,9 @@
}
#ifdef CONFIG_OF
-/* s3c24xx_i2c_parse_dt
- *
+/*
* Parse the device tree node and retreive the platform data.
-*/
-
+ */
static void
s3c24xx_i2c_parse_dt(struct device_node *np, struct s3c24xx_i2c *i2c)
{
@@ -1105,17 +1084,9 @@
}
#else
static void
-s3c24xx_i2c_parse_dt(struct device_node *np, struct s3c24xx_i2c *i2c)
-{
- return;
-}
+s3c24xx_i2c_parse_dt(struct device_node *np, struct s3c24xx_i2c *i2c) { }
#endif
-/* s3c24xx_i2c_probe
- *
- * called by the bus driver when a suitable device is found
-*/
-
static int s3c24xx_i2c_probe(struct platform_device *pdev)
{
struct s3c24xx_i2c *i2c;
@@ -1156,7 +1127,6 @@
init_waitqueue_head(&i2c->wait);
/* find the clock and enable it */
-
i2c->dev = &pdev->dev;
i2c->clk = devm_clk_get(&pdev->dev, "i2c");
if (IS_ERR(i2c->clk)) {
@@ -1166,9 +1136,7 @@
dev_dbg(&pdev->dev, "clock source %p\n", i2c->clk);
-
/* map the registers */
-
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
i2c->regs = devm_ioremap_resource(&pdev->dev, res);
@@ -1179,33 +1147,35 @@
i2c->regs, res);
/* setup info block for the i2c core */
-
i2c->adap.algo_data = i2c;
i2c->adap.dev.parent = &pdev->dev;
-
i2c->pctrl = devm_pinctrl_get_select_default(i2c->dev);
/* inititalise the i2c gpio lines */
-
- if (i2c->pdata->cfg_gpio) {
+ if (i2c->pdata->cfg_gpio)
i2c->pdata->cfg_gpio(to_platform_device(i2c->dev));
- } else if (IS_ERR(i2c->pctrl) && s3c24xx_i2c_parse_dt_gpio(i2c)) {
+ else if (IS_ERR(i2c->pctrl) && s3c24xx_i2c_parse_dt_gpio(i2c))
return -EINVAL;
- }
/* initialise the i2c controller */
+ ret = clk_prepare_enable(i2c->clk);
+ if (ret) {
+ dev_err(&pdev->dev, "I2C clock enable failed\n");
+ return ret;
+ }
- clk_prepare_enable(i2c->clk);
ret = s3c24xx_i2c_init(i2c);
clk_disable(i2c->clk);
if (ret != 0) {
dev_err(&pdev->dev, "I2C controller init failed\n");
+ clk_unprepare(i2c->clk);
return ret;
}
- /* find the IRQ for this unit (note, this relies on the init call to
+
+ /*
+ * find the IRQ for this unit (note, this relies on the init call to
* ensure no current IRQs pending
*/
-
if (!(i2c->quirks & QUIRK_POLL)) {
i2c->irq = ret = platform_get_irq(pdev, 0);
if (ret <= 0) {
@@ -1214,9 +1184,8 @@
return ret;
}
- ret = devm_request_irq(&pdev->dev, i2c->irq, s3c24xx_i2c_irq, 0,
- dev_name(&pdev->dev), i2c);
-
+ ret = devm_request_irq(&pdev->dev, i2c->irq, s3c24xx_i2c_irq,
+ 0, dev_name(&pdev->dev), i2c);
if (ret != 0) {
dev_err(&pdev->dev, "cannot claim IRQ %d\n", i2c->irq);
clk_unprepare(i2c->clk);
@@ -1231,12 +1200,12 @@
return ret;
}
- /* Note, previous versions of the driver used i2c_add_adapter()
+ /*
+ * Note, previous versions of the driver used i2c_add_adapter()
* to add the bus at any number. We now pass the bus number via
* the platform data, so if unset it will now default to always
* being bus 0.
*/
-
i2c->adap.nr = i2c->pdata->bus_num;
i2c->adap.dev.of_node = pdev->dev.of_node;
@@ -1257,11 +1226,6 @@
return 0;
}
-/* s3c24xx_i2c_remove
- *
- * called when device is removed from the bus
-*/
-
static int s3c24xx_i2c_remove(struct platform_device *pdev)
{
struct s3c24xx_i2c *i2c = platform_get_drvdata(pdev);
@@ -1316,14 +1280,8 @@
#ifdef CONFIG_PM
static const struct dev_pm_ops s3c24xx_i2c_dev_pm_ops = {
-#ifdef CONFIG_PM_SLEEP
- .suspend_noirq = s3c24xx_i2c_suspend_noirq,
- .resume_noirq = s3c24xx_i2c_resume_noirq,
- .freeze_noirq = s3c24xx_i2c_suspend_noirq,
- .thaw_noirq = s3c24xx_i2c_resume_noirq,
- .poweroff_noirq = s3c24xx_i2c_suspend_noirq,
- .restore_noirq = s3c24xx_i2c_resume_noirq,
-#endif
+ SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(s3c24xx_i2c_suspend_noirq,
+ s3c24xx_i2c_resume_noirq)
};
#define S3C24XX_DEV_PM_OPS (&s3c24xx_i2c_dev_pm_ops)
@@ -1331,8 +1289,6 @@
#define S3C24XX_DEV_PM_OPS NULL
#endif
-/* device driver for platform bus bits */
-
static struct platform_driver s3c24xx_i2c_driver = {
.probe = s3c24xx_i2c_probe,
.remove = s3c24xx_i2c_remove,
diff --git a/drivers/i2c/busses/i2c-sh_mobile.c b/drivers/i2c/busses/i2c-sh_mobile.c
index 7d2bd3e..6fb3e26 100644
--- a/drivers/i2c/busses/i2c-sh_mobile.c
+++ b/drivers/i2c/busses/i2c-sh_mobile.c
@@ -398,8 +398,7 @@
{
switch (pd->pos) {
case -1:
- *buf = (pd->msg->addr & 0x7f) << 1;
- *buf |= (pd->msg->flags & I2C_M_RD) ? 1 : 0;
+ *buf = i2c_8bit_addr_from_msg(pd->msg);
break;
default:
*buf = pd->msg->buf[pd->pos];
diff --git a/drivers/i2c/busses/i2c-sirf.c b/drivers/i2c/busses/i2c-sirf.c
index 13e51ef..792a42b 100644
--- a/drivers/i2c/busses/i2c-sirf.c
+++ b/drivers/i2c/busses/i2c-sirf.c
@@ -190,9 +190,7 @@
writel(regval, siic->base + SIRFSOC_I2C_CMD(siic->cmd_ptr++));
- addr = msg->addr << 1; /* Generate address */
- if (msg->flags & I2C_M_RD)
- addr |= 1;
+ addr = i2c_8bit_addr_from_msg(msg);
/* Reverse direction bit */
if (msg->flags & I2C_M_REV_DIR_ADDR)
diff --git a/drivers/i2c/busses/i2c-st.c b/drivers/i2c/busses/i2c-st.c
index 6ee7715..944ec420 100644
--- a/drivers/i2c/busses/i2c-st.c
+++ b/drivers/i2c/busses/i2c-st.c
@@ -337,10 +337,42 @@
writel_relaxed(val, i2c_dev->base + SSC_NOISE_SUPP_WIDTH_DATAOUT);
}
+static int st_i2c_recover_bus(struct i2c_adapter *i2c_adap)
+{
+ struct st_i2c_dev *i2c_dev = i2c_get_adapdata(i2c_adap);
+ u32 ctl;
+
+ dev_dbg(i2c_dev->dev, "Trying to recover bus\n");
+
+ /*
+ * SSP IP is dual role SPI/I2C to generate 9 clock pulses
+ * we switch to SPI node, 9 bit words and write a 0. This
+ * has been validate with a oscilloscope and is easier
+ * than switching to GPIO mode.
+ */
+
+ /* Disable interrupts */
+ writel_relaxed(0, i2c_dev->base + SSC_IEN);
+
+ st_i2c_hw_config(i2c_dev);
+
+ ctl = SSC_CTL_EN | SSC_CTL_MS | SSC_CTL_EN_RX_FIFO | SSC_CTL_EN_TX_FIFO;
+ st_i2c_set_bits(i2c_dev->base + SSC_CTL, ctl);
+
+ st_i2c_clr_bits(i2c_dev->base + SSC_I2C, SSC_I2C_I2CM);
+ usleep_range(8000, 10000);
+
+ writel_relaxed(0, i2c_dev->base + SSC_TBUF);
+ usleep_range(2000, 4000);
+ st_i2c_set_bits(i2c_dev->base + SSC_I2C, SSC_I2C_I2CM);
+
+ return 0;
+}
+
static int st_i2c_wait_free_bus(struct st_i2c_dev *i2c_dev)
{
u32 sta;
- int i;
+ int i, ret;
for (i = 0; i < 10; i++) {
sta = readl_relaxed(i2c_dev->base + SSC_STA);
@@ -352,6 +384,12 @@
dev_err(i2c_dev->dev, "bus not free (status = 0x%08x)\n", sta);
+ ret = i2c_recover_bus(&i2c_dev->adap);
+ if (ret) {
+ dev_err(i2c_dev->dev, "Failed to recover the bus (%d)\n", ret);
+ return ret;
+ }
+
return -EBUSY;
}
@@ -614,8 +652,7 @@
unsigned long timeout;
int ret;
- c->addr = (u8)(msg->addr << 1);
- c->addr |= (msg->flags & I2C_M_RD);
+ c->addr = i2c_8bit_addr_from_msg(msg);
c->buf = msg->buf;
c->count = msg->len;
c->xfered = 0;
@@ -744,6 +781,10 @@
.functionality = st_i2c_func,
};
+static struct i2c_bus_recovery_info st_i2c_recovery_info = {
+ .recover_bus = st_i2c_recover_bus,
+};
+
static int st_i2c_of_get_deglitch(struct device_node *np,
struct st_i2c_dev *i2c_dev)
{
@@ -826,6 +867,7 @@
adap->timeout = 2 * HZ;
adap->retries = 0;
adap->algo = &st_i2c_algo;
+ adap->bus_recovery_info = &st_i2c_recovery_info;
adap->dev.parent = &pdev->dev;
adap->dev.of_node = pdev->dev.of_node;
diff --git a/drivers/i2c/busses/i2c-tegra.c b/drivers/i2c/busses/i2c-tegra.c
index 929185a..445398c3 100644
--- a/drivers/i2c/busses/i2c-tegra.c
+++ b/drivers/i2c/busses/i2c-tegra.c
@@ -38,6 +38,7 @@
#define I2C_CNFG_DEBOUNCE_CNT_SHIFT 12
#define I2C_CNFG_PACKET_MODE_EN (1<<10)
#define I2C_CNFG_NEW_MASTER_FSM (1<<11)
+#define I2C_CNFG_MULTI_MASTER_MODE (1<<17)
#define I2C_STATUS 0x01C
#define I2C_SL_CNFG 0x020
#define I2C_SL_CNFG_NACK (1<<1)
@@ -106,6 +107,9 @@
#define I2C_SLV_CONFIG_LOAD (1 << 1)
#define I2C_TIMEOUT_CONFIG_LOAD (1 << 2)
+#define I2C_CLKEN_OVERRIDE 0x090
+#define I2C_MST_CORE_CLKEN_OVR (1 << 0)
+
/*
* msg_end_type: The bus control which need to be send at end of transfer.
* @MSG_END_STOP: Send stop pulse at end of transfer.
@@ -143,6 +147,8 @@
int clk_divisor_hs_mode;
int clk_divisor_std_fast_mode;
u16 clk_divisor_fast_plus_mode;
+ bool has_multi_master_mode;
+ bool has_slcg_override_reg;
};
/**
@@ -184,6 +190,7 @@
u32 bus_clk_rate;
u16 clk_divisor_non_hs_mode;
bool is_suspended;
+ bool is_multimaster_mode;
};
static void dvc_writel(struct tegra_i2c_dev *i2c_dev, u32 val, unsigned long reg)
@@ -438,6 +445,10 @@
val = I2C_CNFG_NEW_MASTER_FSM | I2C_CNFG_PACKET_MODE_EN |
(0x2 << I2C_CNFG_DEBOUNCE_CNT_SHIFT);
+
+ if (i2c_dev->hw->has_multi_master_mode)
+ val |= I2C_CNFG_MULTI_MASTER_MODE;
+
i2c_writel(i2c_dev, val, I2C_CNFG);
i2c_writel(i2c_dev, 0, I2C_INT_MASK);
@@ -463,25 +474,29 @@
if (tegra_i2c_flush_fifos(i2c_dev))
err = -ETIMEDOUT;
+ if (i2c_dev->is_multimaster_mode && i2c_dev->hw->has_slcg_override_reg)
+ i2c_writel(i2c_dev, I2C_MST_CORE_CLKEN_OVR, I2C_CLKEN_OVERRIDE);
+
if (i2c_dev->hw->has_config_load_reg) {
i2c_writel(i2c_dev, I2C_MSTR_CONFIG_LOAD, I2C_CONFIG_LOAD);
while (i2c_readl(i2c_dev, I2C_CONFIG_LOAD) != 0) {
if (time_after(jiffies, timeout)) {
dev_warn(i2c_dev->dev,
"timeout waiting for config load\n");
- return -ETIMEDOUT;
+ err = -ETIMEDOUT;
+ goto err;
}
msleep(1);
}
}
- tegra_i2c_clock_disable(i2c_dev);
-
if (i2c_dev->irq_disabled) {
i2c_dev->irq_disabled = 0;
enable_irq(i2c_dev->irq);
}
+err:
+ tegra_i2c_clock_disable(i2c_dev);
return err;
}
@@ -688,6 +703,20 @@
return ret;
}
+static void tegra_i2c_parse_dt(struct tegra_i2c_dev *i2c_dev)
+{
+ struct device_node *np = i2c_dev->dev->of_node;
+ int ret;
+
+ ret = of_property_read_u32(np, "clock-frequency",
+ &i2c_dev->bus_clk_rate);
+ if (ret)
+ i2c_dev->bus_clk_rate = 100000; /* default clock rate */
+
+ i2c_dev->is_multimaster_mode = of_property_read_bool(np,
+ "multi-master");
+}
+
static const struct i2c_algorithm tegra_i2c_algo = {
.master_xfer = tegra_i2c_xfer,
.functionality = tegra_i2c_func,
@@ -707,6 +736,8 @@
.clk_divisor_std_fast_mode = 0,
.clk_divisor_fast_plus_mode = 0,
.has_config_load_reg = false,
+ .has_multi_master_mode = false,
+ .has_slcg_override_reg = false,
};
static const struct tegra_i2c_hw_feature tegra30_i2c_hw = {
@@ -717,6 +748,8 @@
.clk_divisor_std_fast_mode = 0,
.clk_divisor_fast_plus_mode = 0,
.has_config_load_reg = false,
+ .has_multi_master_mode = false,
+ .has_slcg_override_reg = false,
};
static const struct tegra_i2c_hw_feature tegra114_i2c_hw = {
@@ -727,6 +760,8 @@
.clk_divisor_std_fast_mode = 0x19,
.clk_divisor_fast_plus_mode = 0x10,
.has_config_load_reg = false,
+ .has_multi_master_mode = false,
+ .has_slcg_override_reg = false,
};
static const struct tegra_i2c_hw_feature tegra124_i2c_hw = {
@@ -737,10 +772,25 @@
.clk_divisor_std_fast_mode = 0x19,
.clk_divisor_fast_plus_mode = 0x10,
.has_config_load_reg = true,
+ .has_multi_master_mode = false,
+ .has_slcg_override_reg = true,
+};
+
+static const struct tegra_i2c_hw_feature tegra210_i2c_hw = {
+ .has_continue_xfer_support = true,
+ .has_per_pkt_xfer_complete_irq = true,
+ .has_single_clk_source = true,
+ .clk_divisor_hs_mode = 1,
+ .clk_divisor_std_fast_mode = 0x19,
+ .clk_divisor_fast_plus_mode = 0x10,
+ .has_config_load_reg = true,
+ .has_multi_master_mode = true,
+ .has_slcg_override_reg = true,
};
/* Match table for of_platform binding */
static const struct of_device_id tegra_i2c_of_match[] = {
+ { .compatible = "nvidia,tegra210-i2c", .data = &tegra210_i2c_hw, },
{ .compatible = "nvidia,tegra124-i2c", .data = &tegra124_i2c_hw, },
{ .compatible = "nvidia,tegra114-i2c", .data = &tegra114_i2c_hw, },
{ .compatible = "nvidia,tegra30-i2c", .data = &tegra30_i2c_hw, },
@@ -797,10 +847,7 @@
return PTR_ERR(i2c_dev->rst);
}
- ret = of_property_read_u32(i2c_dev->dev->of_node, "clock-frequency",
- &i2c_dev->bus_clk_rate);
- if (ret)
- i2c_dev->bus_clk_rate = 100000; /* default clock rate */
+ tegra_i2c_parse_dt(i2c_dev);
i2c_dev->hw = &tegra20_i2c_hw;
@@ -853,6 +900,15 @@
goto unprepare_fast_clk;
}
+ if (i2c_dev->is_multimaster_mode) {
+ ret = clk_enable(i2c_dev->div_clk);
+ if (ret < 0) {
+ dev_err(i2c_dev->dev, "div_clk enable failed %d\n",
+ ret);
+ goto unprepare_div_clk;
+ }
+ }
+
ret = tegra_i2c_init(i2c_dev);
if (ret) {
dev_err(&pdev->dev, "Failed to initialize i2c controller");
@@ -863,7 +919,7 @@
tegra_i2c_isr, 0, dev_name(&pdev->dev), i2c_dev);
if (ret) {
dev_err(&pdev->dev, "Failed to request irq %i\n", i2c_dev->irq);
- goto unprepare_div_clk;
+ goto disable_div_clk;
}
i2c_set_adapdata(&i2c_dev->adapter, i2c_dev);
@@ -878,11 +934,15 @@
ret = i2c_add_numbered_adapter(&i2c_dev->adapter);
if (ret) {
dev_err(&pdev->dev, "Failed to add I2C adapter\n");
- goto unprepare_div_clk;
+ goto disable_div_clk;
}
return 0;
+disable_div_clk:
+ if (i2c_dev->is_multimaster_mode)
+ clk_disable(i2c_dev->div_clk);
+
unprepare_div_clk:
clk_unprepare(i2c_dev->div_clk);
@@ -898,6 +958,9 @@
struct tegra_i2c_dev *i2c_dev = platform_get_drvdata(pdev);
i2c_del_adapter(&i2c_dev->adapter);
+ if (i2c_dev->is_multimaster_mode)
+ clk_disable(i2c_dev->div_clk);
+
clk_unprepare(i2c_dev->div_clk);
if (!i2c_dev->hw->has_single_clk_source)
clk_unprepare(i2c_dev->fast_clk);
diff --git a/drivers/i2c/busses/i2c-uniphier-f.c b/drivers/i2c/busses/i2c-uniphier-f.c
index 213ba55..aeead0d 100644
--- a/drivers/i2c/busses/i2c-uniphier-f.c
+++ b/drivers/i2c/busses/i2c-uniphier-f.c
@@ -524,7 +524,7 @@
irq = platform_get_irq(pdev, 0);
if (irq < 0) {
- dev_err(dev, "failed to get IRQ number");
+ dev_err(dev, "failed to get IRQ number\n");
return irq;
}
diff --git a/drivers/i2c/busses/i2c-uniphier.c b/drivers/i2c/busses/i2c-uniphier.c
index 89eaa8a7..475a5eb 100644
--- a/drivers/i2c/busses/i2c-uniphier.c
+++ b/drivers/i2c/busses/i2c-uniphier.c
@@ -381,7 +381,7 @@
irq = platform_get_irq(pdev, 0);
if (irq < 0) {
- dev_err(dev, "failed to get IRQ number");
+ dev_err(dev, "failed to get IRQ number\n");
return irq;
}
diff --git a/drivers/i2c/i2c-core.c b/drivers/i2c/i2c-core.c
index e584d88..af11b65 100644
--- a/drivers/i2c/i2c-core.c
+++ b/drivers/i2c/i2c-core.c
@@ -954,48 +954,40 @@
}
/**
- * i2c_lock_adapter - Get exclusive access to an I2C bus segment
+ * i2c_adapter_lock_bus - Get exclusive access to an I2C bus segment
* @adapter: Target I2C bus segment
+ * @flags: I2C_LOCK_ROOT_ADAPTER locks the root i2c adapter, I2C_LOCK_SEGMENT
+ * locks only this branch in the adapter tree
*/
-void i2c_lock_adapter(struct i2c_adapter *adapter)
+static void i2c_adapter_lock_bus(struct i2c_adapter *adapter,
+ unsigned int flags)
{
- struct i2c_adapter *parent = i2c_parent_is_i2c_adapter(adapter);
-
- if (parent)
- i2c_lock_adapter(parent);
- else
- rt_mutex_lock(&adapter->bus_lock);
-}
-EXPORT_SYMBOL_GPL(i2c_lock_adapter);
-
-/**
- * i2c_trylock_adapter - Try to get exclusive access to an I2C bus segment
- * @adapter: Target I2C bus segment
- */
-static int i2c_trylock_adapter(struct i2c_adapter *adapter)
-{
- struct i2c_adapter *parent = i2c_parent_is_i2c_adapter(adapter);
-
- if (parent)
- return i2c_trylock_adapter(parent);
- else
- return rt_mutex_trylock(&adapter->bus_lock);
+ rt_mutex_lock(&adapter->bus_lock);
}
/**
- * i2c_unlock_adapter - Release exclusive access to an I2C bus segment
+ * i2c_adapter_trylock_bus - Try to get exclusive access to an I2C bus segment
* @adapter: Target I2C bus segment
+ * @flags: I2C_LOCK_ROOT_ADAPTER trylocks the root i2c adapter, I2C_LOCK_SEGMENT
+ * trylocks only this branch in the adapter tree
*/
-void i2c_unlock_adapter(struct i2c_adapter *adapter)
+static int i2c_adapter_trylock_bus(struct i2c_adapter *adapter,
+ unsigned int flags)
{
- struct i2c_adapter *parent = i2c_parent_is_i2c_adapter(adapter);
-
- if (parent)
- i2c_unlock_adapter(parent);
- else
- rt_mutex_unlock(&adapter->bus_lock);
+ return rt_mutex_trylock(&adapter->bus_lock);
}
-EXPORT_SYMBOL_GPL(i2c_unlock_adapter);
+
+/**
+ * i2c_adapter_unlock_bus - Release exclusive access to an I2C bus segment
+ * @adapter: Target I2C bus segment
+ * @flags: I2C_LOCK_ROOT_ADAPTER unlocks the root i2c adapter, I2C_LOCK_SEGMENT
+ * unlocks only this branch in the adapter tree
+ */
+static void i2c_adapter_unlock_bus(struct i2c_adapter *adapter,
+ unsigned int flags)
+{
+ rt_mutex_unlock(&adapter->bus_lock);
+}
static void i2c_dev_set_name(struct i2c_adapter *adap,
struct i2c_client *client)
@@ -1541,7 +1533,14 @@
return -EINVAL;
}
+ if (!adap->lock_bus) {
+ adap->lock_bus = i2c_adapter_lock_bus;
+ adap->trylock_bus = i2c_adapter_trylock_bus;
+ adap->unlock_bus = i2c_adapter_unlock_bus;
+ }
+
rt_mutex_init(&adap->bus_lock);
+ rt_mutex_init(&adap->mux_lock);
mutex_init(&adap->userspace_clients_lock);
INIT_LIST_HEAD(&adap->userspace_clients);
@@ -1559,6 +1558,7 @@
dev_dbg(&adap->dev, "adapter [%s] registered\n", adap->name);
pm_runtime_no_callbacks(&adap->dev);
+ pm_suspend_ignore_children(&adap->dev, true);
pm_runtime_enable(&adap->dev);
#ifdef CONFIG_I2C_COMPAT
@@ -1594,10 +1594,12 @@
bri->get_scl = get_scl_gpio_value;
bri->set_scl = set_scl_gpio_value;
- } else if (!bri->set_scl || !bri->get_scl) {
+ } else if (bri->recover_bus == i2c_generic_scl_recovery) {
/* Generic SCL recovery */
- dev_err(&adap->dev, "No {get|set}_gpio() found, not using recovery\n");
- adap->bus_recovery_info = NULL;
+ if (!bri->set_scl || !bri->get_scl) {
+ dev_err(&adap->dev, "No {get|set}_scl() found, not using recovery\n");
+ adap->bus_recovery_info = NULL;
+ }
}
}
@@ -2309,16 +2311,16 @@
#endif
if (in_atomic() || irqs_disabled()) {
- ret = i2c_trylock_adapter(adap);
+ ret = adap->trylock_bus(adap, I2C_LOCK_SEGMENT);
if (!ret)
/* I2C activity is ongoing. */
return -EAGAIN;
} else {
- i2c_lock_adapter(adap);
+ i2c_lock_bus(adap, I2C_LOCK_SEGMENT);
}
ret = __i2c_transfer(adap, msgs, num);
- i2c_unlock_adapter(adap);
+ i2c_unlock_bus(adap, I2C_LOCK_SEGMENT);
return ret;
} else {
@@ -2646,7 +2648,7 @@
static u8 i2c_smbus_msg_pec(u8 pec, struct i2c_msg *msg)
{
/* The address will be sent first */
- u8 addr = (msg->addr << 1) | !!(msg->flags & I2C_M_RD);
+ u8 addr = i2c_8bit_addr_from_msg(msg);
pec = i2c_smbus_pec(pec, &addr, 1);
/* The data buffer follows */
@@ -3093,7 +3095,7 @@
flags &= I2C_M_TEN | I2C_CLIENT_PEC | I2C_CLIENT_SCCB;
if (adapter->algo->smbus_xfer) {
- i2c_lock_adapter(adapter);
+ i2c_lock_bus(adapter, I2C_LOCK_SEGMENT);
/* Retry automatically on arbitration loss */
orig_jiffies = jiffies;
@@ -3107,7 +3109,7 @@
orig_jiffies + adapter->timeout))
break;
}
- i2c_unlock_adapter(adapter);
+ i2c_unlock_bus(adapter, I2C_LOCK_SEGMENT);
if (res != -EOPNOTSUPP || !adapter->algo->master_xfer)
goto trace;
diff --git a/drivers/i2c/i2c-mux.c b/drivers/i2c/i2c-mux.c
index d402287..8eee986 100644
--- a/drivers/i2c/i2c-mux.c
+++ b/drivers/i2c/i2c-mux.c
@@ -31,30 +31,66 @@
struct i2c_mux_priv {
struct i2c_adapter adap;
struct i2c_algorithm algo;
-
- struct i2c_adapter *parent;
- struct device *mux_dev;
- void *mux_priv;
+ struct i2c_mux_core *muxc;
u32 chan_id;
-
- int (*select)(struct i2c_adapter *, void *mux_priv, u32 chan_id);
- int (*deselect)(struct i2c_adapter *, void *mux_priv, u32 chan_id);
};
+static int __i2c_mux_master_xfer(struct i2c_adapter *adap,
+ struct i2c_msg msgs[], int num)
+{
+ struct i2c_mux_priv *priv = adap->algo_data;
+ struct i2c_mux_core *muxc = priv->muxc;
+ struct i2c_adapter *parent = muxc->parent;
+ int ret;
+
+ /* Switch to the right mux port and perform the transfer. */
+
+ ret = muxc->select(muxc, priv->chan_id);
+ if (ret >= 0)
+ ret = __i2c_transfer(parent, msgs, num);
+ if (muxc->deselect)
+ muxc->deselect(muxc, priv->chan_id);
+
+ return ret;
+}
+
static int i2c_mux_master_xfer(struct i2c_adapter *adap,
struct i2c_msg msgs[], int num)
{
struct i2c_mux_priv *priv = adap->algo_data;
- struct i2c_adapter *parent = priv->parent;
+ struct i2c_mux_core *muxc = priv->muxc;
+ struct i2c_adapter *parent = muxc->parent;
int ret;
/* Switch to the right mux port and perform the transfer. */
- ret = priv->select(parent, priv->mux_priv, priv->chan_id);
+ ret = muxc->select(muxc, priv->chan_id);
if (ret >= 0)
- ret = __i2c_transfer(parent, msgs, num);
- if (priv->deselect)
- priv->deselect(parent, priv->mux_priv, priv->chan_id);
+ ret = i2c_transfer(parent, msgs, num);
+ if (muxc->deselect)
+ muxc->deselect(muxc, priv->chan_id);
+
+ return ret;
+}
+
+static int __i2c_mux_smbus_xfer(struct i2c_adapter *adap,
+ u16 addr, unsigned short flags,
+ char read_write, u8 command,
+ int size, union i2c_smbus_data *data)
+{
+ struct i2c_mux_priv *priv = adap->algo_data;
+ struct i2c_mux_core *muxc = priv->muxc;
+ struct i2c_adapter *parent = muxc->parent;
+ int ret;
+
+ /* Select the right mux port and perform the transfer. */
+
+ ret = muxc->select(muxc, priv->chan_id);
+ if (ret >= 0)
+ ret = parent->algo->smbus_xfer(parent, addr, flags,
+ read_write, command, size, data);
+ if (muxc->deselect)
+ muxc->deselect(muxc, priv->chan_id);
return ret;
}
@@ -65,17 +101,18 @@
int size, union i2c_smbus_data *data)
{
struct i2c_mux_priv *priv = adap->algo_data;
- struct i2c_adapter *parent = priv->parent;
+ struct i2c_mux_core *muxc = priv->muxc;
+ struct i2c_adapter *parent = muxc->parent;
int ret;
/* Select the right mux port and perform the transfer. */
- ret = priv->select(parent, priv->mux_priv, priv->chan_id);
+ ret = muxc->select(muxc, priv->chan_id);
if (ret >= 0)
- ret = parent->algo->smbus_xfer(parent, addr, flags,
- read_write, command, size, data);
- if (priv->deselect)
- priv->deselect(parent, priv->mux_priv, priv->chan_id);
+ ret = i2c_smbus_xfer(parent, addr, flags,
+ read_write, command, size, data);
+ if (muxc->deselect)
+ muxc->deselect(muxc, priv->chan_id);
return ret;
}
@@ -84,7 +121,7 @@
static u32 i2c_mux_functionality(struct i2c_adapter *adap)
{
struct i2c_mux_priv *priv = adap->algo_data;
- struct i2c_adapter *parent = priv->parent;
+ struct i2c_adapter *parent = priv->muxc->parent;
return parent->algo->functionality(parent);
}
@@ -102,38 +139,167 @@
return class;
}
-struct i2c_adapter *i2c_add_mux_adapter(struct i2c_adapter *parent,
- struct device *mux_dev,
- void *mux_priv, u32 force_nr, u32 chan_id,
- unsigned int class,
- int (*select) (struct i2c_adapter *,
- void *, u32),
- int (*deselect) (struct i2c_adapter *,
- void *, u32))
+static void i2c_mux_lock_bus(struct i2c_adapter *adapter, unsigned int flags)
{
+ struct i2c_mux_priv *priv = adapter->algo_data;
+ struct i2c_adapter *parent = priv->muxc->parent;
+
+ rt_mutex_lock(&parent->mux_lock);
+ if (!(flags & I2C_LOCK_ROOT_ADAPTER))
+ return;
+ i2c_lock_bus(parent, flags);
+}
+
+static int i2c_mux_trylock_bus(struct i2c_adapter *adapter, unsigned int flags)
+{
+ struct i2c_mux_priv *priv = adapter->algo_data;
+ struct i2c_adapter *parent = priv->muxc->parent;
+
+ if (!rt_mutex_trylock(&parent->mux_lock))
+ return 0; /* mux_lock not locked, failure */
+ if (!(flags & I2C_LOCK_ROOT_ADAPTER))
+ return 1; /* we only want mux_lock, success */
+ if (parent->trylock_bus(parent, flags))
+ return 1; /* parent locked too, success */
+ rt_mutex_unlock(&parent->mux_lock);
+ return 0; /* parent not locked, failure */
+}
+
+static void i2c_mux_unlock_bus(struct i2c_adapter *adapter, unsigned int flags)
+{
+ struct i2c_mux_priv *priv = adapter->algo_data;
+ struct i2c_adapter *parent = priv->muxc->parent;
+
+ if (flags & I2C_LOCK_ROOT_ADAPTER)
+ i2c_unlock_bus(parent, flags);
+ rt_mutex_unlock(&parent->mux_lock);
+}
+
+static void i2c_parent_lock_bus(struct i2c_adapter *adapter,
+ unsigned int flags)
+{
+ struct i2c_mux_priv *priv = adapter->algo_data;
+ struct i2c_adapter *parent = priv->muxc->parent;
+
+ rt_mutex_lock(&parent->mux_lock);
+ i2c_lock_bus(parent, flags);
+}
+
+static int i2c_parent_trylock_bus(struct i2c_adapter *adapter,
+ unsigned int flags)
+{
+ struct i2c_mux_priv *priv = adapter->algo_data;
+ struct i2c_adapter *parent = priv->muxc->parent;
+
+ if (!rt_mutex_trylock(&parent->mux_lock))
+ return 0; /* mux_lock not locked, failure */
+ if (parent->trylock_bus(parent, flags))
+ return 1; /* parent locked too, success */
+ rt_mutex_unlock(&parent->mux_lock);
+ return 0; /* parent not locked, failure */
+}
+
+static void i2c_parent_unlock_bus(struct i2c_adapter *adapter,
+ unsigned int flags)
+{
+ struct i2c_mux_priv *priv = adapter->algo_data;
+ struct i2c_adapter *parent = priv->muxc->parent;
+
+ i2c_unlock_bus(parent, flags);
+ rt_mutex_unlock(&parent->mux_lock);
+}
+
+struct i2c_adapter *i2c_root_adapter(struct device *dev)
+{
+ struct device *i2c;
+ struct i2c_adapter *i2c_root;
+
+ /*
+ * Walk up the device tree to find an i2c adapter, indicating
+ * that this is an i2c client device. Check all ancestors to
+ * handle mfd devices etc.
+ */
+ for (i2c = dev; i2c; i2c = i2c->parent) {
+ if (i2c->type == &i2c_adapter_type)
+ break;
+ }
+ if (!i2c)
+ return NULL;
+
+ /* Continue up the tree to find the root i2c adapter */
+ i2c_root = to_i2c_adapter(i2c);
+ while (i2c_parent_is_i2c_adapter(i2c_root))
+ i2c_root = i2c_parent_is_i2c_adapter(i2c_root);
+
+ return i2c_root;
+}
+EXPORT_SYMBOL_GPL(i2c_root_adapter);
+
+struct i2c_mux_core *i2c_mux_alloc(struct i2c_adapter *parent,
+ struct device *dev, int max_adapters,
+ int sizeof_priv, u32 flags,
+ int (*select)(struct i2c_mux_core *, u32),
+ int (*deselect)(struct i2c_mux_core *, u32))
+{
+ struct i2c_mux_core *muxc;
+
+ muxc = devm_kzalloc(dev, sizeof(*muxc)
+ + max_adapters * sizeof(muxc->adapter[0])
+ + sizeof_priv, GFP_KERNEL);
+ if (!muxc)
+ return NULL;
+ if (sizeof_priv)
+ muxc->priv = &muxc->adapter[max_adapters];
+
+ muxc->parent = parent;
+ muxc->dev = dev;
+ if (flags & I2C_MUX_LOCKED)
+ muxc->mux_locked = true;
+ muxc->select = select;
+ muxc->deselect = deselect;
+ muxc->max_adapters = max_adapters;
+
+ return muxc;
+}
+EXPORT_SYMBOL_GPL(i2c_mux_alloc);
+
+int i2c_mux_add_adapter(struct i2c_mux_core *muxc,
+ u32 force_nr, u32 chan_id,
+ unsigned int class)
+{
+ struct i2c_adapter *parent = muxc->parent;
struct i2c_mux_priv *priv;
char symlink_name[20];
int ret;
- priv = kzalloc(sizeof(struct i2c_mux_priv), GFP_KERNEL);
+ if (muxc->num_adapters >= muxc->max_adapters) {
+ dev_err(muxc->dev, "No room for more i2c-mux adapters\n");
+ return -EINVAL;
+ }
+
+ priv = kzalloc(sizeof(*priv), GFP_KERNEL);
if (!priv)
- return NULL;
+ return -ENOMEM;
/* Set up private adapter data */
- priv->parent = parent;
- priv->mux_dev = mux_dev;
- priv->mux_priv = mux_priv;
+ priv->muxc = muxc;
priv->chan_id = chan_id;
- priv->select = select;
- priv->deselect = deselect;
/* Need to do algo dynamically because we don't know ahead
* of time what sort of physical adapter we'll be dealing with.
*/
- if (parent->algo->master_xfer)
- priv->algo.master_xfer = i2c_mux_master_xfer;
- if (parent->algo->smbus_xfer)
- priv->algo.smbus_xfer = i2c_mux_smbus_xfer;
+ if (parent->algo->master_xfer) {
+ if (muxc->mux_locked)
+ priv->algo.master_xfer = i2c_mux_master_xfer;
+ else
+ priv->algo.master_xfer = __i2c_mux_master_xfer;
+ }
+ if (parent->algo->smbus_xfer) {
+ if (muxc->mux_locked)
+ priv->algo.smbus_xfer = i2c_mux_smbus_xfer;
+ else
+ priv->algo.smbus_xfer = __i2c_mux_smbus_xfer;
+ }
priv->algo.functionality = i2c_mux_functionality;
/* Now fill out new adapter structure */
@@ -146,6 +312,15 @@
priv->adap.retries = parent->retries;
priv->adap.timeout = parent->timeout;
priv->adap.quirks = parent->quirks;
+ if (muxc->mux_locked) {
+ priv->adap.lock_bus = i2c_mux_lock_bus;
+ priv->adap.trylock_bus = i2c_mux_trylock_bus;
+ priv->adap.unlock_bus = i2c_mux_unlock_bus;
+ } else {
+ priv->adap.lock_bus = i2c_parent_lock_bus;
+ priv->adap.trylock_bus = i2c_parent_trylock_bus;
+ priv->adap.unlock_bus = i2c_parent_unlock_bus;
+ }
/* Sanity check on class */
if (i2c_mux_parent_classes(parent) & class)
@@ -159,11 +334,11 @@
* Try to populate the mux adapter's of_node, expands to
* nothing if !CONFIG_OF.
*/
- if (mux_dev->of_node) {
+ if (muxc->dev->of_node) {
struct device_node *child;
u32 reg;
- for_each_child_of_node(mux_dev->of_node, child) {
+ for_each_child_of_node(muxc->dev->of_node, child) {
ret = of_property_read_u32(child, "reg", ®);
if (ret)
continue;
@@ -177,8 +352,9 @@
/*
* Associate the mux channel with an ACPI node.
*/
- if (has_acpi_companion(mux_dev))
- acpi_preset_companion(&priv->adap.dev, ACPI_COMPANION(mux_dev),
+ if (has_acpi_companion(muxc->dev))
+ acpi_preset_companion(&priv->adap.dev,
+ ACPI_COMPANION(muxc->dev),
chan_id);
if (force_nr) {
@@ -192,35 +368,45 @@
"failed to add mux-adapter (error=%d)\n",
ret);
kfree(priv);
- return NULL;
+ return ret;
}
- WARN(sysfs_create_link(&priv->adap.dev.kobj, &mux_dev->kobj, "mux_device"),
- "can't create symlink to mux device\n");
+ WARN(sysfs_create_link(&priv->adap.dev.kobj, &muxc->dev->kobj,
+ "mux_device"),
+ "can't create symlink to mux device\n");
snprintf(symlink_name, sizeof(symlink_name), "channel-%u", chan_id);
- WARN(sysfs_create_link(&mux_dev->kobj, &priv->adap.dev.kobj, symlink_name),
- "can't create symlink for channel %u\n", chan_id);
+ WARN(sysfs_create_link(&muxc->dev->kobj, &priv->adap.dev.kobj,
+ symlink_name),
+ "can't create symlink for channel %u\n", chan_id);
dev_info(&parent->dev, "Added multiplexed i2c bus %d\n",
i2c_adapter_id(&priv->adap));
- return &priv->adap;
+ muxc->adapter[muxc->num_adapters++] = &priv->adap;
+ return 0;
}
-EXPORT_SYMBOL_GPL(i2c_add_mux_adapter);
+EXPORT_SYMBOL_GPL(i2c_mux_add_adapter);
-void i2c_del_mux_adapter(struct i2c_adapter *adap)
+void i2c_mux_del_adapters(struct i2c_mux_core *muxc)
{
- struct i2c_mux_priv *priv = adap->algo_data;
char symlink_name[20];
- snprintf(symlink_name, sizeof(symlink_name), "channel-%u", priv->chan_id);
- sysfs_remove_link(&priv->mux_dev->kobj, symlink_name);
+ while (muxc->num_adapters) {
+ struct i2c_adapter *adap = muxc->adapter[--muxc->num_adapters];
+ struct i2c_mux_priv *priv = adap->algo_data;
- sysfs_remove_link(&priv->adap.dev.kobj, "mux_device");
- i2c_del_adapter(adap);
- kfree(priv);
+ muxc->adapter[muxc->num_adapters] = NULL;
+
+ snprintf(symlink_name, sizeof(symlink_name),
+ "channel-%u", priv->chan_id);
+ sysfs_remove_link(&muxc->dev->kobj, symlink_name);
+
+ sysfs_remove_link(&priv->adap.dev.kobj, "mux_device");
+ i2c_del_adapter(adap);
+ kfree(priv);
+ }
}
-EXPORT_SYMBOL_GPL(i2c_del_mux_adapter);
+EXPORT_SYMBOL_GPL(i2c_mux_del_adapters);
MODULE_AUTHOR("Rodolfo Giometti <giometti@linux.it>");
MODULE_DESCRIPTION("I2C driver for multiplexed I2C busses");
diff --git a/drivers/i2c/muxes/i2c-arb-gpio-challenge.c b/drivers/i2c/muxes/i2c-arb-gpio-challenge.c
index 402e3a6..a90bbc4 100644
--- a/drivers/i2c/muxes/i2c-arb-gpio-challenge.c
+++ b/drivers/i2c/muxes/i2c-arb-gpio-challenge.c
@@ -28,8 +28,6 @@
/**
* struct i2c_arbitrator_data - Driver data for I2C arbitrator
*
- * @parent: Parent adapter
- * @child: Child bus
* @our_gpio: GPIO we'll use to claim.
* @our_gpio_release: 0 if active high; 1 if active low; AKA if the GPIO ==
* this then consider it released.
@@ -42,8 +40,6 @@
*/
struct i2c_arbitrator_data {
- struct i2c_adapter *parent;
- struct i2c_adapter *child;
int our_gpio;
int our_gpio_release;
int their_gpio;
@@ -59,9 +55,9 @@
*
* Use the GPIO-based signalling protocol; return -EBUSY if we fail.
*/
-static int i2c_arbitrator_select(struct i2c_adapter *adap, void *data, u32 chan)
+static int i2c_arbitrator_select(struct i2c_mux_core *muxc, u32 chan)
{
- const struct i2c_arbitrator_data *arb = data;
+ const struct i2c_arbitrator_data *arb = i2c_mux_priv(muxc);
unsigned long stop_retry, stop_time;
/* Start a round of trying to claim the bus */
@@ -93,7 +89,7 @@
/* Give up, release our claim */
gpio_set_value(arb->our_gpio, arb->our_gpio_release);
udelay(arb->slew_delay_us);
- dev_err(&adap->dev, "Could not claim bus, timeout\n");
+ dev_err(muxc->dev, "Could not claim bus, timeout\n");
return -EBUSY;
}
@@ -102,10 +98,9 @@
*
* Release the I2C bus using the GPIO-based signalling protocol.
*/
-static int i2c_arbitrator_deselect(struct i2c_adapter *adap, void *data,
- u32 chan)
+static int i2c_arbitrator_deselect(struct i2c_mux_core *muxc, u32 chan)
{
- const struct i2c_arbitrator_data *arb = data;
+ const struct i2c_arbitrator_data *arb = i2c_mux_priv(muxc);
/* Release the bus and wait for the other master to notice */
gpio_set_value(arb->our_gpio, arb->our_gpio_release);
@@ -119,6 +114,7 @@
struct device *dev = &pdev->dev;
struct device_node *np = dev->of_node;
struct device_node *parent_np;
+ struct i2c_mux_core *muxc;
struct i2c_arbitrator_data *arb;
enum of_gpio_flags gpio_flags;
unsigned long out_init;
@@ -134,12 +130,13 @@
return -EINVAL;
}
- arb = devm_kzalloc(dev, sizeof(*arb), GFP_KERNEL);
- if (!arb) {
- dev_err(dev, "Cannot allocate i2c_arbitrator_data\n");
+ muxc = i2c_mux_alloc(NULL, dev, 1, sizeof(*arb), 0,
+ i2c_arbitrator_select, i2c_arbitrator_deselect);
+ if (!muxc)
return -ENOMEM;
- }
- platform_set_drvdata(pdev, arb);
+ arb = i2c_mux_priv(muxc);
+
+ platform_set_drvdata(pdev, muxc);
/* Request GPIOs */
ret = of_get_named_gpio_flags(np, "our-claim-gpio", 0, &gpio_flags);
@@ -196,21 +193,18 @@
dev_err(dev, "Cannot parse i2c-parent\n");
return -EINVAL;
}
- arb->parent = of_get_i2c_adapter_by_node(parent_np);
+ muxc->parent = of_get_i2c_adapter_by_node(parent_np);
of_node_put(parent_np);
- if (!arb->parent) {
+ if (!muxc->parent) {
dev_err(dev, "Cannot find parent bus\n");
return -EPROBE_DEFER;
}
/* Actually add the mux adapter */
- arb->child = i2c_add_mux_adapter(arb->parent, dev, arb, 0, 0, 0,
- i2c_arbitrator_select,
- i2c_arbitrator_deselect);
- if (!arb->child) {
+ ret = i2c_mux_add_adapter(muxc, 0, 0, 0);
+ if (ret) {
dev_err(dev, "Failed to add adapter\n");
- ret = -ENODEV;
- i2c_put_adapter(arb->parent);
+ i2c_put_adapter(muxc->parent);
}
return ret;
@@ -218,11 +212,10 @@
static int i2c_arbitrator_remove(struct platform_device *pdev)
{
- struct i2c_arbitrator_data *arb = platform_get_drvdata(pdev);
+ struct i2c_mux_core *muxc = platform_get_drvdata(pdev);
- i2c_del_mux_adapter(arb->child);
- i2c_put_adapter(arb->parent);
-
+ i2c_mux_del_adapters(muxc);
+ i2c_put_adapter(muxc->parent);
return 0;
}
diff --git a/drivers/i2c/muxes/i2c-mux-gpio.c b/drivers/i2c/muxes/i2c-mux-gpio.c
index b8e11c1..e5cf26e 100644
--- a/drivers/i2c/muxes/i2c-mux-gpio.c
+++ b/drivers/i2c/muxes/i2c-mux-gpio.c
@@ -15,11 +15,10 @@
#include <linux/module.h>
#include <linux/slab.h>
#include <linux/gpio.h>
+#include "../../gpio/gpiolib.h"
#include <linux/of_gpio.h>
struct gpiomux {
- struct i2c_adapter *parent;
- struct i2c_adapter **adap; /* child busses */
struct i2c_mux_gpio_platform_data data;
unsigned gpio_base;
};
@@ -33,18 +32,18 @@
val & (1 << i));
}
-static int i2c_mux_gpio_select(struct i2c_adapter *adap, void *data, u32 chan)
+static int i2c_mux_gpio_select(struct i2c_mux_core *muxc, u32 chan)
{
- struct gpiomux *mux = data;
+ struct gpiomux *mux = i2c_mux_priv(muxc);
i2c_mux_gpio_set(mux, chan);
return 0;
}
-static int i2c_mux_gpio_deselect(struct i2c_adapter *adap, void *data, u32 chan)
+static int i2c_mux_gpio_deselect(struct i2c_mux_core *muxc, u32 chan)
{
- struct gpiomux *mux = data;
+ struct gpiomux *mux = i2c_mux_priv(muxc);
i2c_mux_gpio_set(mux, mux->data.idle);
@@ -136,19 +135,16 @@
static int i2c_mux_gpio_probe(struct platform_device *pdev)
{
+ struct i2c_mux_core *muxc;
struct gpiomux *mux;
struct i2c_adapter *parent;
- int (*deselect) (struct i2c_adapter *, void *, u32);
+ struct i2c_adapter *root;
unsigned initial_state, gpio_base;
int i, ret;
mux = devm_kzalloc(&pdev->dev, sizeof(*mux), GFP_KERNEL);
- if (!mux) {
- dev_err(&pdev->dev, "Cannot allocate gpiomux structure");
+ if (!mux)
return -ENOMEM;
- }
-
- platform_set_drvdata(pdev, mux);
if (!dev_get_platdata(&pdev->dev)) {
ret = i2c_mux_gpio_probe_dt(mux, pdev);
@@ -180,27 +176,32 @@
if (!parent)
return -EPROBE_DEFER;
- mux->parent = parent;
- mux->gpio_base = gpio_base;
-
- mux->adap = devm_kzalloc(&pdev->dev,
- sizeof(*mux->adap) * mux->data.n_values,
- GFP_KERNEL);
- if (!mux->adap) {
- dev_err(&pdev->dev, "Cannot allocate i2c_adapter structure");
+ muxc = i2c_mux_alloc(parent, &pdev->dev, mux->data.n_values, 0, 0,
+ i2c_mux_gpio_select, NULL);
+ if (!muxc) {
ret = -ENOMEM;
goto alloc_failed;
}
+ muxc->priv = mux;
+
+ platform_set_drvdata(pdev, muxc);
+
+ root = i2c_root_adapter(&parent->dev);
+
+ muxc->mux_locked = true;
+ mux->gpio_base = gpio_base;
if (mux->data.idle != I2C_MUX_GPIO_NO_IDLE) {
initial_state = mux->data.idle;
- deselect = i2c_mux_gpio_deselect;
+ muxc->deselect = i2c_mux_gpio_deselect;
} else {
initial_state = mux->data.values[0];
- deselect = NULL;
}
for (i = 0; i < mux->data.n_gpios; i++) {
+ struct device *gpio_dev;
+ struct gpio_desc *gpio_desc;
+
ret = gpio_request(gpio_base + mux->data.gpios[i], "i2c-mux-gpio");
if (ret) {
dev_err(&pdev->dev, "Failed to request GPIO %d\n",
@@ -217,17 +218,24 @@
i++; /* gpio_request above succeeded, so must free */
goto err_request_gpio;
}
+
+ if (!muxc->mux_locked)
+ continue;
+
+ gpio_desc = gpio_to_desc(gpio_base + mux->data.gpios[i]);
+ gpio_dev = &gpio_desc->gdev->dev;
+ muxc->mux_locked = i2c_root_adapter(gpio_dev) == root;
}
+ if (muxc->mux_locked)
+ dev_info(&pdev->dev, "mux-locked i2c mux\n");
+
for (i = 0; i < mux->data.n_values; i++) {
u32 nr = mux->data.base_nr ? (mux->data.base_nr + i) : 0;
unsigned int class = mux->data.classes ? mux->data.classes[i] : 0;
- mux->adap[i] = i2c_add_mux_adapter(parent, &pdev->dev, mux, nr,
- mux->data.values[i], class,
- i2c_mux_gpio_select, deselect);
- if (!mux->adap[i]) {
- ret = -ENODEV;
+ ret = i2c_mux_add_adapter(muxc, nr, mux->data.values[i], class);
+ if (ret) {
dev_err(&pdev->dev, "Failed to add adapter %d\n", i);
goto add_adapter_failed;
}
@@ -239,8 +247,7 @@
return 0;
add_adapter_failed:
- for (; i > 0; i--)
- i2c_del_mux_adapter(mux->adap[i - 1]);
+ i2c_mux_del_adapters(muxc);
i = mux->data.n_gpios;
err_request_gpio:
for (; i > 0; i--)
@@ -253,16 +260,16 @@
static int i2c_mux_gpio_remove(struct platform_device *pdev)
{
- struct gpiomux *mux = platform_get_drvdata(pdev);
+ struct i2c_mux_core *muxc = platform_get_drvdata(pdev);
+ struct gpiomux *mux = i2c_mux_priv(muxc);
int i;
- for (i = 0; i < mux->data.n_values; i++)
- i2c_del_mux_adapter(mux->adap[i]);
+ i2c_mux_del_adapters(muxc);
for (i = 0; i < mux->data.n_gpios; i++)
gpio_free(mux->gpio_base + mux->data.gpios[i]);
- i2c_put_adapter(mux->parent);
+ i2c_put_adapter(muxc->parent);
return 0;
}
diff --git a/drivers/i2c/muxes/i2c-mux-pca9541.c b/drivers/i2c/muxes/i2c-mux-pca9541.c
index d0ba424..3cb8af6 100644
--- a/drivers/i2c/muxes/i2c-mux-pca9541.c
+++ b/drivers/i2c/muxes/i2c-mux-pca9541.c
@@ -73,7 +73,7 @@
#define SELECT_DELAY_LONG 1000
struct pca9541 {
- struct i2c_adapter *mux_adap;
+ struct i2c_client *client;
unsigned long select_timeout;
unsigned long arb_timeout;
};
@@ -217,7 +217,8 @@
*/
static int pca9541_arbitrate(struct i2c_client *client)
{
- struct pca9541 *data = i2c_get_clientdata(client);
+ struct i2c_mux_core *muxc = i2c_get_clientdata(client);
+ struct pca9541 *data = i2c_mux_priv(muxc);
int reg;
reg = pca9541_reg_read(client, PCA9541_CONTROL);
@@ -285,9 +286,10 @@
return 0;
}
-static int pca9541_select_chan(struct i2c_adapter *adap, void *client, u32 chan)
+static int pca9541_select_chan(struct i2c_mux_core *muxc, u32 chan)
{
- struct pca9541 *data = i2c_get_clientdata(client);
+ struct pca9541 *data = i2c_mux_priv(muxc);
+ struct i2c_client *client = data->client;
int ret;
unsigned long timeout = jiffies + ARB2_TIMEOUT;
/* give up after this time */
@@ -309,9 +311,11 @@
return -ETIMEDOUT;
}
-static int pca9541_release_chan(struct i2c_adapter *adap,
- void *client, u32 chan)
+static int pca9541_release_chan(struct i2c_mux_core *muxc, u32 chan)
{
+ struct pca9541 *data = i2c_mux_priv(muxc);
+ struct i2c_client *client = data->client;
+
pca9541_release_bus(client);
return 0;
}
@@ -324,20 +328,13 @@
{
struct i2c_adapter *adap = client->adapter;
struct pca954x_platform_data *pdata = dev_get_platdata(&client->dev);
+ struct i2c_mux_core *muxc;
struct pca9541 *data;
int force;
- int ret = -ENODEV;
+ int ret;
if (!i2c_check_functionality(adap, I2C_FUNC_SMBUS_BYTE_DATA))
- goto err;
-
- data = kzalloc(sizeof(struct pca9541), GFP_KERNEL);
- if (!data) {
- ret = -ENOMEM;
- goto err;
- }
-
- i2c_set_clientdata(client, data);
+ return -ENODEV;
/*
* I2C accesses are unprotected here.
@@ -352,34 +349,33 @@
force = 0;
if (pdata)
force = pdata->modes[0].adap_id;
- data->mux_adap = i2c_add_mux_adapter(adap, &client->dev, client,
- force, 0, 0,
- pca9541_select_chan,
- pca9541_release_chan);
+ muxc = i2c_mux_alloc(adap, &client->dev, 1, sizeof(*data), 0,
+ pca9541_select_chan, pca9541_release_chan);
+ if (!muxc)
+ return -ENOMEM;
- if (data->mux_adap == NULL) {
+ data = i2c_mux_priv(muxc);
+ data->client = client;
+
+ i2c_set_clientdata(client, muxc);
+
+ ret = i2c_mux_add_adapter(muxc, force, 0, 0);
+ if (ret) {
dev_err(&client->dev, "failed to register master selector\n");
- goto exit_free;
+ return ret;
}
dev_info(&client->dev, "registered master selector for I2C %s\n",
client->name);
return 0;
-
-exit_free:
- kfree(data);
-err:
- return ret;
}
static int pca9541_remove(struct i2c_client *client)
{
- struct pca9541 *data = i2c_get_clientdata(client);
+ struct i2c_mux_core *muxc = i2c_get_clientdata(client);
- i2c_del_mux_adapter(data->mux_adap);
-
- kfree(data);
+ i2c_mux_del_adapters(muxc);
return 0;
}
diff --git a/drivers/i2c/muxes/i2c-mux-pca954x.c b/drivers/i2c/muxes/i2c-mux-pca954x.c
index acfcef3..528e755 100644
--- a/drivers/i2c/muxes/i2c-mux-pca954x.c
+++ b/drivers/i2c/muxes/i2c-mux-pca954x.c
@@ -60,9 +60,10 @@
struct pca954x {
enum pca_type type;
- struct i2c_adapter *virt_adaps[PCA954X_MAX_NCHANS];
u8 last_chan; /* last register value */
+ u8 deselect;
+ struct i2c_client *client;
};
struct chip_desc {
@@ -146,10 +147,10 @@
return ret;
}
-static int pca954x_select_chan(struct i2c_adapter *adap,
- void *client, u32 chan)
+static int pca954x_select_chan(struct i2c_mux_core *muxc, u32 chan)
{
- struct pca954x *data = i2c_get_clientdata(client);
+ struct pca954x *data = i2c_mux_priv(muxc);
+ struct i2c_client *client = data->client;
const struct chip_desc *chip = &chips[data->type];
u8 regval;
int ret = 0;
@@ -162,21 +163,24 @@
/* Only select the channel if its different from the last channel */
if (data->last_chan != regval) {
- ret = pca954x_reg_write(adap, client, regval);
+ ret = pca954x_reg_write(muxc->parent, client, regval);
data->last_chan = regval;
}
return ret;
}
-static int pca954x_deselect_mux(struct i2c_adapter *adap,
- void *client, u32 chan)
+static int pca954x_deselect_mux(struct i2c_mux_core *muxc, u32 chan)
{
- struct pca954x *data = i2c_get_clientdata(client);
+ struct pca954x *data = i2c_mux_priv(muxc);
+ struct i2c_client *client = data->client;
+
+ if (!(data->deselect & (1 << chan)))
+ return 0;
/* Deselect active channel */
data->last_chan = 0;
- return pca954x_reg_write(adap, client, data->last_chan);
+ return pca954x_reg_write(muxc->parent, client, data->last_chan);
}
/*
@@ -191,17 +195,22 @@
bool idle_disconnect_dt;
struct gpio_desc *gpio;
int num, force, class;
+ struct i2c_mux_core *muxc;
struct pca954x *data;
int ret;
if (!i2c_check_functionality(adap, I2C_FUNC_SMBUS_BYTE))
return -ENODEV;
- data = devm_kzalloc(&client->dev, sizeof(struct pca954x), GFP_KERNEL);
- if (!data)
+ muxc = i2c_mux_alloc(adap, &client->dev,
+ PCA954X_MAX_NCHANS, sizeof(*data), 0,
+ pca954x_select_chan, pca954x_deselect_mux);
+ if (!muxc)
return -ENOMEM;
+ data = i2c_mux_priv(muxc);
- i2c_set_clientdata(client, data);
+ i2c_set_clientdata(client, muxc);
+ data->client = client;
/* Get the mux out of reset if a reset GPIO is specified. */
gpio = devm_gpiod_get_optional(&client->dev, "reset", GPIOD_OUT_LOW);
@@ -238,16 +247,13 @@
/* discard unconfigured channels */
break;
idle_disconnect_pd = pdata->modes[num].deselect_on_exit;
+ data->deselect |= (idle_disconnect_pd
+ || idle_disconnect_dt) << num;
}
- data->virt_adaps[num] =
- i2c_add_mux_adapter(adap, &client->dev, client,
- force, num, class, pca954x_select_chan,
- (idle_disconnect_pd || idle_disconnect_dt)
- ? pca954x_deselect_mux : NULL);
+ ret = i2c_mux_add_adapter(muxc, force, num, class);
- if (data->virt_adaps[num] == NULL) {
- ret = -ENODEV;
+ if (ret) {
dev_err(&client->dev,
"failed to register multiplexed adapter"
" %d as bus %d\n", num, force);
@@ -263,23 +269,15 @@
return 0;
virt_reg_failed:
- for (num--; num >= 0; num--)
- i2c_del_mux_adapter(data->virt_adaps[num]);
+ i2c_mux_del_adapters(muxc);
return ret;
}
static int pca954x_remove(struct i2c_client *client)
{
- struct pca954x *data = i2c_get_clientdata(client);
- const struct chip_desc *chip = &chips[data->type];
- int i;
+ struct i2c_mux_core *muxc = i2c_get_clientdata(client);
- for (i = 0; i < chip->nchans; ++i)
- if (data->virt_adaps[i]) {
- i2c_del_mux_adapter(data->virt_adaps[i]);
- data->virt_adaps[i] = NULL;
- }
-
+ i2c_mux_del_adapters(muxc);
return 0;
}
@@ -287,7 +285,8 @@
static int pca954x_resume(struct device *dev)
{
struct i2c_client *client = to_i2c_client(dev);
- struct pca954x *data = i2c_get_clientdata(client);
+ struct i2c_mux_core *muxc = i2c_get_clientdata(client);
+ struct pca954x *data = i2c_mux_priv(muxc);
data->last_chan = 0;
return i2c_smbus_write_byte(client, 0);
diff --git a/drivers/i2c/muxes/i2c-mux-pinctrl.c b/drivers/i2c/muxes/i2c-mux-pinctrl.c
index b5a982b..35bb775 100644
--- a/drivers/i2c/muxes/i2c-mux-pinctrl.c
+++ b/drivers/i2c/muxes/i2c-mux-pinctrl.c
@@ -24,36 +24,32 @@
#include <linux/platform_device.h>
#include <linux/slab.h>
#include <linux/of.h>
+#include "../../pinctrl/core.h"
struct i2c_mux_pinctrl {
- struct device *dev;
struct i2c_mux_pinctrl_platform_data *pdata;
struct pinctrl *pinctrl;
struct pinctrl_state **states;
struct pinctrl_state *state_idle;
- struct i2c_adapter *parent;
- struct i2c_adapter **busses;
};
-static int i2c_mux_pinctrl_select(struct i2c_adapter *adap, void *data,
- u32 chan)
+static int i2c_mux_pinctrl_select(struct i2c_mux_core *muxc, u32 chan)
{
- struct i2c_mux_pinctrl *mux = data;
+ struct i2c_mux_pinctrl *mux = i2c_mux_priv(muxc);
return pinctrl_select_state(mux->pinctrl, mux->states[chan]);
}
-static int i2c_mux_pinctrl_deselect(struct i2c_adapter *adap, void *data,
- u32 chan)
+static int i2c_mux_pinctrl_deselect(struct i2c_mux_core *muxc, u32 chan)
{
- struct i2c_mux_pinctrl *mux = data;
+ struct i2c_mux_pinctrl *mux = i2c_mux_priv(muxc);
return pinctrl_select_state(mux->pinctrl, mux->state_idle);
}
#ifdef CONFIG_OF
static int i2c_mux_pinctrl_parse_dt(struct i2c_mux_pinctrl *mux,
- struct platform_device *pdev)
+ struct platform_device *pdev)
{
struct device_node *np = pdev->dev.of_node;
int num_names, i, ret;
@@ -64,15 +60,12 @@
return 0;
mux->pdata = devm_kzalloc(&pdev->dev, sizeof(*mux->pdata), GFP_KERNEL);
- if (!mux->pdata) {
- dev_err(mux->dev,
- "Cannot allocate i2c_mux_pinctrl_platform_data\n");
+ if (!mux->pdata)
return -ENOMEM;
- }
num_names = of_property_count_strings(np, "pinctrl-names");
if (num_names < 0) {
- dev_err(mux->dev, "Cannot parse pinctrl-names: %d\n",
+ dev_err(&pdev->dev, "Cannot parse pinctrl-names: %d\n",
num_names);
return num_names;
}
@@ -80,23 +73,22 @@
mux->pdata->pinctrl_states = devm_kzalloc(&pdev->dev,
sizeof(*mux->pdata->pinctrl_states) * num_names,
GFP_KERNEL);
- if (!mux->pdata->pinctrl_states) {
- dev_err(mux->dev, "Cannot allocate pinctrl_states\n");
+ if (!mux->pdata->pinctrl_states)
return -ENOMEM;
- }
for (i = 0; i < num_names; i++) {
ret = of_property_read_string_index(np, "pinctrl-names", i,
&mux->pdata->pinctrl_states[mux->pdata->bus_count]);
if (ret < 0) {
- dev_err(mux->dev, "Cannot parse pinctrl-names: %d\n",
+ dev_err(&pdev->dev, "Cannot parse pinctrl-names: %d\n",
ret);
return ret;
}
if (!strcmp(mux->pdata->pinctrl_states[mux->pdata->bus_count],
"idle")) {
if (i != num_names - 1) {
- dev_err(mux->dev, "idle state must be last\n");
+ dev_err(&pdev->dev,
+ "idle state must be last\n");
return -EINVAL;
}
mux->pdata->pinctrl_state_idle = "idle";
@@ -107,13 +99,13 @@
adapter_np = of_parse_phandle(np, "i2c-parent", 0);
if (!adapter_np) {
- dev_err(mux->dev, "Cannot parse i2c-parent\n");
+ dev_err(&pdev->dev, "Cannot parse i2c-parent\n");
return -ENODEV;
}
adapter = of_find_i2c_adapter_by_node(adapter_np);
of_node_put(adapter_np);
if (!adapter) {
- dev_err(mux->dev, "Cannot find parent bus\n");
+ dev_err(&pdev->dev, "Cannot find parent bus\n");
return -EPROBE_DEFER;
}
mux->pdata->parent_bus_num = i2c_adapter_id(adapter);
@@ -129,21 +121,38 @@
}
#endif
+static struct i2c_adapter *i2c_mux_pinctrl_root_adapter(
+ struct pinctrl_state *state)
+{
+ struct i2c_adapter *root = NULL;
+ struct pinctrl_setting *setting;
+ struct i2c_adapter *pin_root;
+
+ list_for_each_entry(setting, &state->settings, node) {
+ pin_root = i2c_root_adapter(setting->pctldev->dev);
+ if (!pin_root)
+ return NULL;
+ if (!root)
+ root = pin_root;
+ else if (root != pin_root)
+ return NULL;
+ }
+
+ return root;
+}
+
static int i2c_mux_pinctrl_probe(struct platform_device *pdev)
{
+ struct i2c_mux_core *muxc;
struct i2c_mux_pinctrl *mux;
- int (*deselect)(struct i2c_adapter *, void *, u32);
+ struct i2c_adapter *root;
int i, ret;
mux = devm_kzalloc(&pdev->dev, sizeof(*mux), GFP_KERNEL);
if (!mux) {
- dev_err(&pdev->dev, "Cannot allocate i2c_mux_pinctrl\n");
ret = -ENOMEM;
goto err;
}
- platform_set_drvdata(pdev, mux);
-
- mux->dev = &pdev->dev;
mux->pdata = dev_get_platdata(&pdev->dev);
if (!mux->pdata) {
@@ -166,14 +175,15 @@
goto err;
}
- mux->busses = devm_kzalloc(&pdev->dev,
- sizeof(*mux->busses) * mux->pdata->bus_count,
- GFP_KERNEL);
- if (!mux->busses) {
- dev_err(&pdev->dev, "Cannot allocate busses\n");
+ muxc = i2c_mux_alloc(NULL, &pdev->dev, mux->pdata->bus_count, 0, 0,
+ i2c_mux_pinctrl_select, NULL);
+ if (!muxc) {
ret = -ENOMEM;
goto err;
}
+ muxc->priv = mux;
+
+ platform_set_drvdata(pdev, muxc);
mux->pinctrl = devm_pinctrl_get(&pdev->dev);
if (IS_ERR(mux->pinctrl)) {
@@ -184,13 +194,13 @@
for (i = 0; i < mux->pdata->bus_count; i++) {
mux->states[i] = pinctrl_lookup_state(mux->pinctrl,
mux->pdata->pinctrl_states[i]);
- if (IS_ERR(mux->states[i])) {
- ret = PTR_ERR(mux->states[i]);
- dev_err(&pdev->dev,
- "Cannot look up pinctrl state %s: %d\n",
- mux->pdata->pinctrl_states[i], ret);
- goto err;
- }
+ if (IS_ERR(mux->states[i])) {
+ ret = PTR_ERR(mux->states[i]);
+ dev_err(&pdev->dev,
+ "Cannot look up pinctrl state %s: %d\n",
+ mux->pdata->pinctrl_states[i], ret);
+ goto err;
+ }
}
if (mux->pdata->pinctrl_state_idle) {
mux->state_idle = pinctrl_lookup_state(mux->pinctrl,
@@ -203,29 +213,39 @@
goto err;
}
- deselect = i2c_mux_pinctrl_deselect;
- } else {
- deselect = NULL;
+ muxc->deselect = i2c_mux_pinctrl_deselect;
}
- mux->parent = i2c_get_adapter(mux->pdata->parent_bus_num);
- if (!mux->parent) {
+ muxc->parent = i2c_get_adapter(mux->pdata->parent_bus_num);
+ if (!muxc->parent) {
dev_err(&pdev->dev, "Parent adapter (%d) not found\n",
mux->pdata->parent_bus_num);
ret = -EPROBE_DEFER;
goto err;
}
+ root = i2c_root_adapter(&muxc->parent->dev);
+
+ muxc->mux_locked = true;
+ for (i = 0; i < mux->pdata->bus_count; i++) {
+ if (root != i2c_mux_pinctrl_root_adapter(mux->states[i])) {
+ muxc->mux_locked = false;
+ break;
+ }
+ }
+ if (muxc->mux_locked && mux->pdata->pinctrl_state_idle &&
+ root != i2c_mux_pinctrl_root_adapter(mux->state_idle))
+ muxc->mux_locked = false;
+
+ if (muxc->mux_locked)
+ dev_info(&pdev->dev, "mux-locked i2c mux\n");
+
for (i = 0; i < mux->pdata->bus_count; i++) {
u32 bus = mux->pdata->base_bus_num ?
(mux->pdata->base_bus_num + i) : 0;
- mux->busses[i] = i2c_add_mux_adapter(mux->parent, &pdev->dev,
- mux, bus, i, 0,
- i2c_mux_pinctrl_select,
- deselect);
- if (!mux->busses[i]) {
- ret = -ENODEV;
+ ret = i2c_mux_add_adapter(muxc, bus, i, 0);
+ if (ret) {
dev_err(&pdev->dev, "Failed to add adapter %d\n", i);
goto err_del_adapter;
}
@@ -234,23 +254,18 @@
return 0;
err_del_adapter:
- for (; i > 0; i--)
- i2c_del_mux_adapter(mux->busses[i - 1]);
- i2c_put_adapter(mux->parent);
+ i2c_mux_del_adapters(muxc);
+ i2c_put_adapter(muxc->parent);
err:
return ret;
}
static int i2c_mux_pinctrl_remove(struct platform_device *pdev)
{
- struct i2c_mux_pinctrl *mux = platform_get_drvdata(pdev);
- int i;
+ struct i2c_mux_core *muxc = platform_get_drvdata(pdev);
- for (i = 0; i < mux->pdata->bus_count; i++)
- i2c_del_mux_adapter(mux->busses[i]);
-
- i2c_put_adapter(mux->parent);
-
+ i2c_mux_del_adapters(muxc);
+ i2c_put_adapter(muxc->parent);
return 0;
}
diff --git a/drivers/i2c/muxes/i2c-mux-reg.c b/drivers/i2c/muxes/i2c-mux-reg.c
index 5fbd5bd..6773cad 100644
--- a/drivers/i2c/muxes/i2c-mux-reg.c
+++ b/drivers/i2c/muxes/i2c-mux-reg.c
@@ -21,8 +21,6 @@
#include <linux/slab.h>
struct regmux {
- struct i2c_adapter *parent;
- struct i2c_adapter **adap; /* child busses */
struct i2c_mux_reg_platform_data data;
};
@@ -64,18 +62,16 @@
return 0;
}
-static int i2c_mux_reg_select(struct i2c_adapter *adap, void *data,
- unsigned int chan)
+static int i2c_mux_reg_select(struct i2c_mux_core *muxc, u32 chan)
{
- struct regmux *mux = data;
+ struct regmux *mux = i2c_mux_priv(muxc);
return i2c_mux_reg_set(mux, chan);
}
-static int i2c_mux_reg_deselect(struct i2c_adapter *adap, void *data,
- unsigned int chan)
+static int i2c_mux_reg_deselect(struct i2c_mux_core *muxc, u32 chan)
{
- struct regmux *mux = data;
+ struct regmux *mux = i2c_mux_priv(muxc);
if (mux->data.idle_in_use)
return i2c_mux_reg_set(mux, mux->data.idle);
@@ -85,7 +81,7 @@
#ifdef CONFIG_OF
static int i2c_mux_reg_probe_dt(struct regmux *mux,
- struct platform_device *pdev)
+ struct platform_device *pdev)
{
struct device_node *np = pdev->dev.of_node;
struct device_node *adapter_np, *child;
@@ -107,7 +103,6 @@
if (!adapter)
return -EPROBE_DEFER;
- mux->parent = adapter;
mux->data.parent = i2c_adapter_id(adapter);
put_device(&adapter->dev);
@@ -161,7 +156,7 @@
}
#else
static int i2c_mux_reg_probe_dt(struct regmux *mux,
- struct platform_device *pdev)
+ struct platform_device *pdev)
{
return 0;
}
@@ -169,10 +164,10 @@
static int i2c_mux_reg_probe(struct platform_device *pdev)
{
+ struct i2c_mux_core *muxc;
struct regmux *mux;
struct i2c_adapter *parent;
struct resource *res;
- int (*deselect)(struct i2c_adapter *, void *, u32);
unsigned int class;
int i, ret, nr;
@@ -180,17 +175,9 @@
if (!mux)
return -ENOMEM;
- platform_set_drvdata(pdev, mux);
-
if (dev_get_platdata(&pdev->dev)) {
memcpy(&mux->data, dev_get_platdata(&pdev->dev),
sizeof(mux->data));
-
- parent = i2c_get_adapter(mux->data.parent);
- if (!parent)
- return -EPROBE_DEFER;
-
- mux->parent = parent;
} else {
ret = i2c_mux_reg_probe_dt(mux, pdev);
if (ret < 0) {
@@ -199,6 +186,10 @@
}
}
+ parent = i2c_get_adapter(mux->data.parent);
+ if (!parent)
+ return -EPROBE_DEFER;
+
if (!mux->data.reg) {
dev_info(&pdev->dev,
"Register not set, using platform resource\n");
@@ -215,55 +206,45 @@
return -EINVAL;
}
- mux->adap = devm_kzalloc(&pdev->dev,
- sizeof(*mux->adap) * mux->data.n_values,
- GFP_KERNEL);
- if (!mux->adap) {
- dev_err(&pdev->dev, "Cannot allocate i2c_adapter structure");
+ muxc = i2c_mux_alloc(parent, &pdev->dev, mux->data.n_values, 0, 0,
+ i2c_mux_reg_select, NULL);
+ if (!muxc)
return -ENOMEM;
- }
+ muxc->priv = mux;
+
+ platform_set_drvdata(pdev, muxc);
if (mux->data.idle_in_use)
- deselect = i2c_mux_reg_deselect;
- else
- deselect = NULL;
+ muxc->deselect = i2c_mux_reg_deselect;
for (i = 0; i < mux->data.n_values; i++) {
nr = mux->data.base_nr ? (mux->data.base_nr + i) : 0;
class = mux->data.classes ? mux->data.classes[i] : 0;
- mux->adap[i] = i2c_add_mux_adapter(mux->parent, &pdev->dev, mux,
- nr, mux->data.values[i],
- class, i2c_mux_reg_select,
- deselect);
- if (!mux->adap[i]) {
- ret = -ENODEV;
+ ret = i2c_mux_add_adapter(muxc, nr, mux->data.values[i], class);
+ if (ret) {
dev_err(&pdev->dev, "Failed to add adapter %d\n", i);
goto add_adapter_failed;
}
}
dev_dbg(&pdev->dev, "%d port mux on %s adapter\n",
- mux->data.n_values, mux->parent->name);
+ mux->data.n_values, muxc->parent->name);
return 0;
add_adapter_failed:
- for (; i > 0; i--)
- i2c_del_mux_adapter(mux->adap[i - 1]);
+ i2c_mux_del_adapters(muxc);
return ret;
}
static int i2c_mux_reg_remove(struct platform_device *pdev)
{
- struct regmux *mux = platform_get_drvdata(pdev);
- int i;
+ struct i2c_mux_core *muxc = platform_get_drvdata(pdev);
- for (i = 0; i < mux->data.n_values; i++)
- i2c_del_mux_adapter(mux->adap[i]);
-
- i2c_put_adapter(mux->parent);
+ i2c_mux_del_adapters(muxc);
+ i2c_put_adapter(muxc->parent);
return 0;
}
diff --git a/drivers/iio/accel/Kconfig b/drivers/iio/accel/Kconfig
index b0d3ecf..e4a758c 100644
--- a/drivers/iio/accel/Kconfig
+++ b/drivers/iio/accel/Kconfig
@@ -64,7 +64,7 @@
help
Say yes here to build support for STMicroelectronics accelerometers:
LSM303DLH, LSM303DLHC, LIS3DH, LSM330D, LSM330DL, LSM330DLC,
- LIS331DLH, LSM303DL, LSM303DLM, LSM330, LIS2DH12.
+ LIS331DLH, LSM303DL, LSM303DLM, LSM330, LIS2DH12, H3LIS331DL.
This driver can also be built as a module. If so, these modules
will be created:
@@ -143,7 +143,8 @@
select IIO_TRIGGERED_BUFFER
help
Say yes here to build support for the following Freescale 3-axis
- accelerometers: MMA8451Q, MMA8452Q, MMA8453Q, MMA8652FC, MMA8653FC.
+ accelerometers: MMA8451Q, MMA8452Q, MMA8453Q, MMA8652FC, MMA8653FC,
+ FXLS8471Q.
To compile this driver as a module, choose M here: the module
will be called mma8452.
diff --git a/drivers/iio/accel/bmc150-accel-core.c b/drivers/iio/accel/bmc150-accel-core.c
index 2072a31..197e693 100644
--- a/drivers/iio/accel/bmc150-accel-core.c
+++ b/drivers/iio/accel/bmc150-accel-core.c
@@ -25,7 +25,6 @@
#include <linux/delay.h>
#include <linux/slab.h>
#include <linux/acpi.h>
-#include <linux/gpio/consumer.h>
#include <linux/pm.h>
#include <linux/pm_runtime.h>
#include <linux/iio/iio.h>
@@ -138,6 +137,7 @@
AXIS_X,
AXIS_Y,
AXIS_Z,
+ AXIS_MAX,
};
enum bmc150_power_modes {
@@ -188,7 +188,6 @@
struct bmc150_accel_data {
struct regmap *regmap;
- struct device *dev;
int irq;
struct bmc150_accel_interrupt interrupts[BMC150_ACCEL_INTERRUPTS];
atomic_t active_intr;
@@ -246,16 +245,18 @@
{500000, BMC150_ACCEL_SLEEP_500_MS},
{1000000, BMC150_ACCEL_SLEEP_1_SEC} };
-static const struct regmap_config bmc150_i2c_regmap_conf = {
+const struct regmap_config bmc150_regmap_conf = {
.reg_bits = 8,
.val_bits = 8,
.max_register = 0x3f,
};
+EXPORT_SYMBOL_GPL(bmc150_regmap_conf);
static int bmc150_accel_set_mode(struct bmc150_accel_data *data,
enum bmc150_power_modes mode,
int dur_us)
{
+ struct device *dev = regmap_get_device(data->regmap);
int i;
int ret;
u8 lpw_bits;
@@ -279,11 +280,11 @@
lpw_bits = mode << BMC150_ACCEL_PMU_MODE_SHIFT;
lpw_bits |= (dur_val << BMC150_ACCEL_PMU_BIT_SLEEP_DUR_SHIFT);
- dev_dbg(data->dev, "Set Mode bits %x\n", lpw_bits);
+ dev_dbg(dev, "Set Mode bits %x\n", lpw_bits);
ret = regmap_write(data->regmap, BMC150_ACCEL_REG_PMU_LPW, lpw_bits);
if (ret < 0) {
- dev_err(data->dev, "Error writing reg_pmu_lpw\n");
+ dev_err(dev, "Error writing reg_pmu_lpw\n");
return ret;
}
@@ -316,23 +317,24 @@
static int bmc150_accel_update_slope(struct bmc150_accel_data *data)
{
+ struct device *dev = regmap_get_device(data->regmap);
int ret;
ret = regmap_write(data->regmap, BMC150_ACCEL_REG_INT_6,
data->slope_thres);
if (ret < 0) {
- dev_err(data->dev, "Error writing reg_int_6\n");
+ dev_err(dev, "Error writing reg_int_6\n");
return ret;
}
ret = regmap_update_bits(data->regmap, BMC150_ACCEL_REG_INT_5,
BMC150_ACCEL_SLOPE_DUR_MASK, data->slope_dur);
if (ret < 0) {
- dev_err(data->dev, "Error updating reg_int_5\n");
+ dev_err(dev, "Error updating reg_int_5\n");
return ret;
}
- dev_dbg(data->dev, "%s: %x %x\n", __func__, data->slope_thres,
+ dev_dbg(dev, "%s: %x %x\n", __func__, data->slope_thres,
data->slope_dur);
return ret;
@@ -378,20 +380,21 @@
static int bmc150_accel_set_power_state(struct bmc150_accel_data *data, bool on)
{
+ struct device *dev = regmap_get_device(data->regmap);
int ret;
if (on) {
- ret = pm_runtime_get_sync(data->dev);
+ ret = pm_runtime_get_sync(dev);
} else {
- pm_runtime_mark_last_busy(data->dev);
- ret = pm_runtime_put_autosuspend(data->dev);
+ pm_runtime_mark_last_busy(dev);
+ ret = pm_runtime_put_autosuspend(dev);
}
if (ret < 0) {
- dev_err(data->dev,
+ dev_err(dev,
"Failed: bmc150_accel_set_power_state for %d\n", on);
if (on)
- pm_runtime_put_noidle(data->dev);
+ pm_runtime_put_noidle(dev);
return ret;
}
@@ -445,6 +448,7 @@
static int bmc150_accel_set_interrupt(struct bmc150_accel_data *data, int i,
bool state)
{
+ struct device *dev = regmap_get_device(data->regmap);
struct bmc150_accel_interrupt *intr = &data->interrupts[i];
const struct bmc150_accel_interrupt_info *info = intr->info;
int ret;
@@ -474,7 +478,7 @@
ret = regmap_update_bits(data->regmap, info->map_reg, info->map_bitmask,
(state ? info->map_bitmask : 0));
if (ret < 0) {
- dev_err(data->dev, "Error updating reg_int_map\n");
+ dev_err(dev, "Error updating reg_int_map\n");
goto out_fix_power_state;
}
@@ -482,7 +486,7 @@
ret = regmap_update_bits(data->regmap, info->en_reg, info->en_bitmask,
(state ? info->en_bitmask : 0));
if (ret < 0) {
- dev_err(data->dev, "Error updating reg_int_en\n");
+ dev_err(dev, "Error updating reg_int_en\n");
goto out_fix_power_state;
}
@@ -500,6 +504,7 @@
static int bmc150_accel_set_scale(struct bmc150_accel_data *data, int val)
{
+ struct device *dev = regmap_get_device(data->regmap);
int ret, i;
for (i = 0; i < ARRAY_SIZE(data->chip_info->scale_table); ++i) {
@@ -508,8 +513,7 @@
BMC150_ACCEL_REG_PMU_RANGE,
data->chip_info->scale_table[i].reg_range);
if (ret < 0) {
- dev_err(data->dev,
- "Error writing pmu_range\n");
+ dev_err(dev, "Error writing pmu_range\n");
return ret;
}
@@ -523,6 +527,7 @@
static int bmc150_accel_get_temp(struct bmc150_accel_data *data, int *val)
{
+ struct device *dev = regmap_get_device(data->regmap);
int ret;
unsigned int value;
@@ -530,7 +535,7 @@
ret = regmap_read(data->regmap, BMC150_ACCEL_REG_TEMP, &value);
if (ret < 0) {
- dev_err(data->dev, "Error reading reg_temp\n");
+ dev_err(dev, "Error reading reg_temp\n");
mutex_unlock(&data->mutex);
return ret;
}
@@ -545,6 +550,7 @@
struct iio_chan_spec const *chan,
int *val)
{
+ struct device *dev = regmap_get_device(data->regmap);
int ret;
int axis = chan->scan_index;
__le16 raw_val;
@@ -559,7 +565,7 @@
ret = regmap_bulk_read(data->regmap, BMC150_ACCEL_AXIS_TO_REG(axis),
&raw_val, sizeof(raw_val));
if (ret < 0) {
- dev_err(data->dev, "Error reading axis %d\n", axis);
+ dev_err(dev, "Error reading axis %d\n", axis);
bmc150_accel_set_power_state(data, false);
mutex_unlock(&data->mutex);
return ret;
@@ -831,6 +837,7 @@
static int bmc150_accel_fifo_transfer(struct bmc150_accel_data *data,
char *buffer, int samples)
{
+ struct device *dev = regmap_get_device(data->regmap);
int sample_length = 3 * 2;
int ret;
int total_length = samples * sample_length;
@@ -854,7 +861,8 @@
}
if (ret)
- dev_err(data->dev, "Error transferring data from fifo in single steps of %zu\n",
+ dev_err(dev,
+ "Error transferring data from fifo in single steps of %zu\n",
step);
return ret;
@@ -864,6 +872,7 @@
unsigned samples, bool irq)
{
struct bmc150_accel_data *data = iio_priv(indio_dev);
+ struct device *dev = regmap_get_device(data->regmap);
int ret, i;
u8 count;
u16 buffer[BMC150_ACCEL_FIFO_LENGTH * 3];
@@ -873,7 +882,7 @@
ret = regmap_read(data->regmap, BMC150_ACCEL_REG_FIFO_STATUS, &val);
if (ret < 0) {
- dev_err(data->dev, "Error reading reg_fifo_status\n");
+ dev_err(dev, "Error reading reg_fifo_status\n");
return ret;
}
@@ -1105,27 +1114,23 @@
.driver_module = THIS_MODULE,
};
+static const unsigned long bmc150_accel_scan_masks[] = {
+ BIT(AXIS_X) | BIT(AXIS_Y) | BIT(AXIS_Z),
+ 0};
+
static irqreturn_t bmc150_accel_trigger_handler(int irq, void *p)
{
struct iio_poll_func *pf = p;
struct iio_dev *indio_dev = pf->indio_dev;
struct bmc150_accel_data *data = iio_priv(indio_dev);
- int bit, ret, i = 0;
- unsigned int raw_val;
+ int ret;
mutex_lock(&data->mutex);
- for_each_set_bit(bit, indio_dev->active_scan_mask,
- indio_dev->masklength) {
- ret = regmap_bulk_read(data->regmap,
- BMC150_ACCEL_AXIS_TO_REG(bit), &raw_val,
- 2);
- if (ret < 0) {
- mutex_unlock(&data->mutex);
- goto err_read;
- }
- data->buffer[i++] = raw_val;
- }
+ ret = regmap_bulk_read(data->regmap, BMC150_ACCEL_REG_XOUT_L,
+ data->buffer, AXIS_MAX * 2);
mutex_unlock(&data->mutex);
+ if (ret < 0)
+ goto err_read;
iio_push_to_buffers_with_timestamp(indio_dev, data->buffer,
pf->timestamp);
@@ -1139,6 +1144,7 @@
{
struct bmc150_accel_trigger *t = iio_trigger_get_drvdata(trig);
struct bmc150_accel_data *data = t->data;
+ struct device *dev = regmap_get_device(data->regmap);
int ret;
/* new data interrupts don't need ack */
@@ -1152,8 +1158,7 @@
BMC150_ACCEL_INT_MODE_LATCH_RESET);
mutex_unlock(&data->mutex);
if (ret < 0) {
- dev_err(data->dev,
- "Error writing reg_int_rst_latch\n");
+ dev_err(dev, "Error writing reg_int_rst_latch\n");
return ret;
}
@@ -1204,13 +1209,14 @@
static int bmc150_accel_handle_roc_event(struct iio_dev *indio_dev)
{
struct bmc150_accel_data *data = iio_priv(indio_dev);
+ struct device *dev = regmap_get_device(data->regmap);
int dir;
int ret;
unsigned int val;
ret = regmap_read(data->regmap, BMC150_ACCEL_REG_INT_STATUS_2, &val);
if (ret < 0) {
- dev_err(data->dev, "Error reading reg_int_status_2\n");
+ dev_err(dev, "Error reading reg_int_status_2\n");
return ret;
}
@@ -1253,6 +1259,7 @@
{
struct iio_dev *indio_dev = private;
struct bmc150_accel_data *data = iio_priv(indio_dev);
+ struct device *dev = regmap_get_device(data->regmap);
bool ack = false;
int ret;
@@ -1276,7 +1283,7 @@
BMC150_ACCEL_INT_MODE_LATCH_INT |
BMC150_ACCEL_INT_MODE_LATCH_RESET);
if (ret)
- dev_err(data->dev, "Error writing reg_int_rst_latch\n");
+ dev_err(dev, "Error writing reg_int_rst_latch\n");
ret = IRQ_HANDLED;
} else {
@@ -1347,13 +1354,14 @@
static int bmc150_accel_triggers_setup(struct iio_dev *indio_dev,
struct bmc150_accel_data *data)
{
+ struct device *dev = regmap_get_device(data->regmap);
int i, ret;
for (i = 0; i < BMC150_ACCEL_TRIGGERS; i++) {
struct bmc150_accel_trigger *t = &data->triggers[i];
- t->indio_trig = devm_iio_trigger_alloc(data->dev,
- bmc150_accel_triggers[i].name,
+ t->indio_trig = devm_iio_trigger_alloc(dev,
+ bmc150_accel_triggers[i].name,
indio_dev->name,
indio_dev->id);
if (!t->indio_trig) {
@@ -1361,7 +1369,7 @@
break;
}
- t->indio_trig->dev.parent = data->dev;
+ t->indio_trig->dev.parent = dev;
t->indio_trig->ops = &bmc150_accel_trigger_ops;
t->intr = bmc150_accel_triggers[i].intr;
t->data = data;
@@ -1385,12 +1393,13 @@
static int bmc150_accel_fifo_set_mode(struct bmc150_accel_data *data)
{
+ struct device *dev = regmap_get_device(data->regmap);
u8 reg = BMC150_ACCEL_REG_FIFO_CONFIG1;
int ret;
ret = regmap_write(data->regmap, reg, data->fifo_mode);
if (ret < 0) {
- dev_err(data->dev, "Error writing reg_fifo_config1\n");
+ dev_err(dev, "Error writing reg_fifo_config1\n");
return ret;
}
@@ -1400,7 +1409,7 @@
ret = regmap_write(data->regmap, BMC150_ACCEL_REG_FIFO_CONFIG0,
data->watermark);
if (ret < 0)
- dev_err(data->dev, "Error writing reg_fifo_config0\n");
+ dev_err(dev, "Error writing reg_fifo_config0\n");
return ret;
}
@@ -1484,17 +1493,17 @@
static int bmc150_accel_chip_init(struct bmc150_accel_data *data)
{
+ struct device *dev = regmap_get_device(data->regmap);
int ret, i;
unsigned int val;
ret = regmap_read(data->regmap, BMC150_ACCEL_REG_CHIP_ID, &val);
if (ret < 0) {
- dev_err(data->dev,
- "Error: Reading chip id\n");
+ dev_err(dev, "Error: Reading chip id\n");
return ret;
}
- dev_dbg(data->dev, "Chip Id %x\n", val);
+ dev_dbg(dev, "Chip Id %x\n", val);
for (i = 0; i < ARRAY_SIZE(bmc150_accel_chip_info_tbl); i++) {
if (bmc150_accel_chip_info_tbl[i].chip_id == val) {
data->chip_info = &bmc150_accel_chip_info_tbl[i];
@@ -1503,7 +1512,7 @@
}
if (!data->chip_info) {
- dev_err(data->dev, "Invalid chip %x\n", val);
+ dev_err(dev, "Invalid chip %x\n", val);
return -ENODEV;
}
@@ -1520,8 +1529,7 @@
ret = regmap_write(data->regmap, BMC150_ACCEL_REG_PMU_RANGE,
BMC150_ACCEL_DEF_RANGE_4G);
if (ret < 0) {
- dev_err(data->dev,
- "Error writing reg_pmu_range\n");
+ dev_err(dev, "Error writing reg_pmu_range\n");
return ret;
}
@@ -1539,8 +1547,7 @@
BMC150_ACCEL_INT_MODE_LATCH_INT |
BMC150_ACCEL_INT_MODE_LATCH_RESET);
if (ret < 0) {
- dev_err(data->dev,
- "Error writing reg_int_rst_latch\n");
+ dev_err(dev, "Error writing reg_int_rst_latch\n");
return ret;
}
@@ -1560,7 +1567,6 @@
data = iio_priv(indio_dev);
dev_set_drvdata(dev, indio_dev);
- data->dev = dev;
data->irq = irq;
data->regmap = regmap;
@@ -1575,6 +1581,7 @@
indio_dev->channels = data->chip_info->channels;
indio_dev->num_channels = data->chip_info->num_channels;
indio_dev->name = name ? name : data->chip_info->name;
+ indio_dev->available_scan_masks = bmc150_accel_scan_masks;
indio_dev->modes = INDIO_DIRECT_MODE;
indio_dev->info = &bmc150_accel_info;
@@ -1583,13 +1590,13 @@
bmc150_accel_trigger_handler,
&bmc150_accel_buffer_ops);
if (ret < 0) {
- dev_err(data->dev, "Failed: iio triggered buffer setup\n");
+ dev_err(dev, "Failed: iio triggered buffer setup\n");
return ret;
}
if (data->irq > 0) {
ret = devm_request_threaded_irq(
- data->dev, data->irq,
+ dev, data->irq,
bmc150_accel_irq_handler,
bmc150_accel_irq_thread_handler,
IRQF_TRIGGER_RISING,
@@ -1607,7 +1614,7 @@
ret = regmap_write(data->regmap, BMC150_ACCEL_REG_INT_RST_LATCH,
BMC150_ACCEL_INT_MODE_LATCH_RESET);
if (ret < 0) {
- dev_err(data->dev, "Error writing reg_int_rst_latch\n");
+ dev_err(dev, "Error writing reg_int_rst_latch\n");
goto err_buffer_cleanup;
}
@@ -1656,9 +1663,9 @@
iio_device_unregister(indio_dev);
- pm_runtime_disable(data->dev);
- pm_runtime_set_suspended(data->dev);
- pm_runtime_put_noidle(data->dev);
+ pm_runtime_disable(dev);
+ pm_runtime_set_suspended(dev);
+ pm_runtime_put_noidle(dev);
bmc150_accel_unregister_triggers(data, BMC150_ACCEL_TRIGGERS - 1);
@@ -1707,7 +1714,7 @@
struct bmc150_accel_data *data = iio_priv(indio_dev);
int ret;
- dev_dbg(data->dev, __func__);
+ dev_dbg(dev, __func__);
ret = bmc150_accel_set_mode(data, BMC150_ACCEL_SLEEP_MODE_SUSPEND, 0);
if (ret < 0)
return -EAGAIN;
@@ -1722,7 +1729,7 @@
int ret;
int sleep_val;
- dev_dbg(data->dev, __func__);
+ dev_dbg(dev, __func__);
ret = bmc150_accel_set_mode(data, BMC150_ACCEL_SLEEP_MODE_NORMAL, 0);
if (ret < 0)
diff --git a/drivers/iio/accel/bmc150-accel-i2c.c b/drivers/iio/accel/bmc150-accel-i2c.c
index b41404b..8ca8041 100644
--- a/drivers/iio/accel/bmc150-accel-i2c.c
+++ b/drivers/iio/accel/bmc150-accel-i2c.c
@@ -28,11 +28,6 @@
#include "bmc150-accel.h"
-static const struct regmap_config bmc150_i2c_regmap_conf = {
- .reg_bits = 8,
- .val_bits = 8,
-};
-
static int bmc150_accel_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
@@ -43,7 +38,7 @@
i2c_check_functionality(client->adapter,
I2C_FUNC_SMBUS_READ_I2C_BLOCK);
- regmap = devm_regmap_init_i2c(client, &bmc150_i2c_regmap_conf);
+ regmap = devm_regmap_init_i2c(client, &bmc150_regmap_conf);
if (IS_ERR(regmap)) {
dev_err(&client->dev, "Failed to initialize i2c regmap\n");
return PTR_ERR(regmap);
diff --git a/drivers/iio/accel/bmc150-accel-spi.c b/drivers/iio/accel/bmc150-accel-spi.c
index 16b66f2..006794a7 100644
--- a/drivers/iio/accel/bmc150-accel-spi.c
+++ b/drivers/iio/accel/bmc150-accel-spi.c
@@ -25,18 +25,12 @@
#include "bmc150-accel.h"
-static const struct regmap_config bmc150_spi_regmap_conf = {
- .reg_bits = 8,
- .val_bits = 8,
- .max_register = 0x3f,
-};
-
static int bmc150_accel_probe(struct spi_device *spi)
{
struct regmap *regmap;
const struct spi_device_id *id = spi_get_device_id(spi);
- regmap = devm_regmap_init_spi(spi, &bmc150_spi_regmap_conf);
+ regmap = devm_regmap_init_spi(spi, &bmc150_regmap_conf);
if (IS_ERR(regmap)) {
dev_err(&spi->dev, "Failed to initialize spi regmap\n");
return PTR_ERR(regmap);
diff --git a/drivers/iio/accel/bmc150-accel.h b/drivers/iio/accel/bmc150-accel.h
index ba03359..38a8b11 100644
--- a/drivers/iio/accel/bmc150-accel.h
+++ b/drivers/iio/accel/bmc150-accel.h
@@ -16,5 +16,6 @@
const char *name, bool block_supported);
int bmc150_accel_core_remove(struct device *dev);
extern const struct dev_pm_ops bmc150_accel_pm_ops;
+extern const struct regmap_config bmc150_regmap_conf;
#endif /* _BMC150_ACCEL_H_ */
diff --git a/drivers/iio/accel/kxcjk-1013.c b/drivers/iio/accel/kxcjk-1013.c
index edec1d0..bfe219a 100644
--- a/drivers/iio/accel/kxcjk-1013.c
+++ b/drivers/iio/accel/kxcjk-1013.c
@@ -20,7 +20,6 @@
#include <linux/slab.h>
#include <linux/string.h>
#include <linux/acpi.h>
-#include <linux/gpio/consumer.h>
#include <linux/pm.h>
#include <linux/pm_runtime.h>
#include <linux/iio/iio.h>
@@ -115,6 +114,7 @@
AXIS_X,
AXIS_Y,
AXIS_Z,
+ AXIS_MAX,
};
enum kxcjk1013_mode {
@@ -922,7 +922,7 @@
.realbits = 12, \
.storagebits = 16, \
.shift = 4, \
- .endianness = IIO_CPU, \
+ .endianness = IIO_LE, \
}, \
.event_spec = &kxcjk1013_event, \
.num_event_specs = 1 \
@@ -953,25 +953,23 @@
.driver_module = THIS_MODULE,
};
+static const unsigned long kxcjk1013_scan_masks[] = {0x7, 0};
+
static irqreturn_t kxcjk1013_trigger_handler(int irq, void *p)
{
struct iio_poll_func *pf = p;
struct iio_dev *indio_dev = pf->indio_dev;
struct kxcjk1013_data *data = iio_priv(indio_dev);
- int bit, ret, i = 0;
+ int ret;
mutex_lock(&data->mutex);
-
- for_each_set_bit(bit, indio_dev->active_scan_mask,
- indio_dev->masklength) {
- ret = kxcjk1013_get_acc_reg(data, bit);
- if (ret < 0) {
- mutex_unlock(&data->mutex);
- goto err;
- }
- data->buffer[i++] = ret;
- }
+ ret = i2c_smbus_read_i2c_block_data_or_emulated(data->client,
+ KXCJK1013_REG_XOUT_L,
+ AXIS_MAX * 2,
+ (u8 *)data->buffer);
mutex_unlock(&data->mutex);
+ if (ret < 0)
+ goto err;
iio_push_to_buffers_with_timestamp(indio_dev, data->buffer,
data->timestamp);
@@ -1204,6 +1202,7 @@
indio_dev->dev.parent = &client->dev;
indio_dev->channels = kxcjk1013_channels;
indio_dev->num_channels = ARRAY_SIZE(kxcjk1013_channels);
+ indio_dev->available_scan_masks = kxcjk1013_scan_masks;
indio_dev->name = name;
indio_dev->modes = INDIO_DIRECT_MODE;
indio_dev->info = &kxcjk1013_info;
diff --git a/drivers/iio/accel/mma7455_core.c b/drivers/iio/accel/mma7455_core.c
index c633cc2..c902f54 100644
--- a/drivers/iio/accel/mma7455_core.c
+++ b/drivers/iio/accel/mma7455_core.c
@@ -55,11 +55,11 @@
struct mma7455_data {
struct regmap *regmap;
- struct device *dev;
};
static int mma7455_drdy(struct mma7455_data *mma7455)
{
+ struct device *dev = regmap_get_device(mma7455->regmap);
unsigned int reg;
int tries = 3;
int ret;
@@ -75,7 +75,7 @@
msleep(20);
}
- dev_warn(mma7455->dev, "data not ready\n");
+ dev_warn(dev, "data not ready\n");
return -EIO;
}
@@ -260,7 +260,6 @@
dev_set_drvdata(dev, indio_dev);
mma7455 = iio_priv(indio_dev);
mma7455->regmap = regmap;
- mma7455->dev = dev;
indio_dev->info = &mma7455_info;
indio_dev->name = name;
diff --git a/drivers/iio/accel/mma8452.c b/drivers/iio/accel/mma8452.c
index 7f4994f..e225d3c 100644
--- a/drivers/iio/accel/mma8452.c
+++ b/drivers/iio/accel/mma8452.c
@@ -6,6 +6,7 @@
* MMA8453Q (10 bit)
* MMA8652FC (12 bit)
* MMA8653FC (10 bit)
+ * FXLS8471Q (14 bit)
*
* Copyright 2015 Martin Kepplinger <martin.kepplinger@theobroma-systems.com>
* Copyright 2014 Peter Meerwald <pmeerw@pmeerw.net>
@@ -16,7 +17,7 @@
*
* 7-bit I2C slave address 0x1c/0x1d (pin selectable)
*
- * TODO: orientation events, autosleep
+ * TODO: orientation events
*/
#include <linux/module.h>
@@ -31,6 +32,7 @@
#include <linux/delay.h>
#include <linux/of_device.h>
#include <linux/of_irq.h>
+#include <linux/pm_runtime.h>
#define MMA8452_STATUS 0x00
#define MMA8452_STATUS_DRDY (BIT(2) | BIT(1) | BIT(0))
@@ -91,6 +93,9 @@
#define MMA8453_DEVICE_ID 0x3a
#define MMA8652_DEVICE_ID 0x4a
#define MMA8653_DEVICE_ID 0x5a
+#define FXLS8471_DEVICE_ID 0x6a
+
+#define MMA8452_AUTO_SUSPEND_DELAY_MS 2000
struct mma8452_data {
struct i2c_client *client;
@@ -172,6 +177,31 @@
return -EIO;
}
+static int mma8452_set_runtime_pm_state(struct i2c_client *client, bool on)
+{
+#ifdef CONFIG_PM
+ int ret;
+
+ if (on) {
+ ret = pm_runtime_get_sync(&client->dev);
+ } else {
+ pm_runtime_mark_last_busy(&client->dev);
+ ret = pm_runtime_put_autosuspend(&client->dev);
+ }
+
+ if (ret < 0) {
+ dev_err(&client->dev,
+ "failed to change power state to %d\n", on);
+ if (on)
+ pm_runtime_put_noidle(&client->dev);
+
+ return ret;
+ }
+#endif
+
+ return 0;
+}
+
static int mma8452_read(struct mma8452_data *data, __be16 buf[3])
{
int ret = mma8452_drdy(data);
@@ -179,8 +209,16 @@
if (ret < 0)
return ret;
- return i2c_smbus_read_i2c_block_data(data->client, MMA8452_OUT_X,
- 3 * sizeof(__be16), (u8 *)buf);
+ ret = mma8452_set_runtime_pm_state(data->client, true);
+ if (ret)
+ return ret;
+
+ ret = i2c_smbus_read_i2c_block_data(data->client, MMA8452_OUT_X,
+ 3 * sizeof(__be16), (u8 *)buf);
+
+ ret = mma8452_set_runtime_pm_state(data->client, false);
+
+ return ret;
}
static ssize_t mma8452_show_int_plus_micros(char *buf, const int (*vals)[2],
@@ -357,7 +395,8 @@
return IIO_VAL_INT_PLUS_MICRO;
case IIO_CHAN_INFO_CALIBBIAS:
ret = i2c_smbus_read_byte_data(data->client,
- MMA8452_OFF_X + chan->scan_index);
+ MMA8452_OFF_X +
+ chan->scan_index);
if (ret < 0)
return ret;
@@ -392,24 +431,47 @@
data->ctrl_reg1);
}
+/* returns >0 if active, 0 if in standby and <0 on error */
+static int mma8452_is_active(struct mma8452_data *data)
+{
+ int reg;
+
+ reg = i2c_smbus_read_byte_data(data->client, MMA8452_CTRL_REG1);
+ if (reg < 0)
+ return reg;
+
+ return reg & MMA8452_CTRL_ACTIVE;
+}
+
static int mma8452_change_config(struct mma8452_data *data, u8 reg, u8 val)
{
int ret;
+ int is_active;
mutex_lock(&data->lock);
- /* config can only be changed when in standby */
- ret = mma8452_standby(data);
- if (ret < 0)
+ is_active = mma8452_is_active(data);
+ if (is_active < 0) {
+ ret = is_active;
goto fail;
+ }
+
+ /* config can only be changed when in standby */
+ if (is_active > 0) {
+ ret = mma8452_standby(data);
+ if (ret < 0)
+ goto fail;
+ }
ret = i2c_smbus_write_byte_data(data->client, reg, val);
if (ret < 0)
goto fail;
- ret = mma8452_active(data);
- if (ret < 0)
- goto fail;
+ if (is_active > 0) {
+ ret = mma8452_active(data);
+ if (ret < 0)
+ goto fail;
+ }
ret = 0;
fail:
@@ -418,7 +480,7 @@
return ret;
}
-/* returns >0 if in freefall mode, 0 if not or <0 if an error occured */
+/* returns >0 if in freefall mode, 0 if not or <0 if an error occurred */
static int mma8452_freefall_mode_enabled(struct mma8452_data *data)
{
int val;
@@ -668,7 +730,8 @@
if (ret < 0)
return ret;
- return !!(ret & BIT(chan->scan_index + chip->ev_cfg_chan_shift));
+ return !!(ret & BIT(chan->scan_index +
+ chip->ev_cfg_chan_shift));
default:
return -EINVAL;
}
@@ -682,7 +745,11 @@
{
struct mma8452_data *data = iio_priv(indio_dev);
const struct mma_chip_info *chip = data->chip_info;
- int val;
+ int val, ret;
+
+ ret = mma8452_set_runtime_pm_state(data->client, state);
+ if (ret)
+ return ret;
switch (dir) {
case IIO_EV_DIR_FALLING:
@@ -990,6 +1057,7 @@
mma8453,
mma8652,
mma8653,
+ fxls8471,
};
static const struct mma_chip_info mma_chip_info_table[] = {
@@ -1003,7 +1071,7 @@
* bit.
* The userspace interface uses m/s^2 and we declare micro units
* So scale factor for 12 bit here is given by:
- * g * N * 1000000 / 2048 for N = 2, 4, 8 and g=9.80665
+ * g * N * 1000000 / 2048 for N = 2, 4, 8 and g=9.80665
*/
.mma_scales = { {0, 2394}, {0, 4788}, {0, 9577} },
.ev_cfg = MMA8452_TRANSIENT_CFG,
@@ -1081,6 +1149,22 @@
.ev_ths_mask = MMA8452_FF_MT_THS_MASK,
.ev_count = MMA8452_FF_MT_COUNT,
},
+ [fxls8471] = {
+ .chip_id = FXLS8471_DEVICE_ID,
+ .channels = mma8451_channels,
+ .num_channels = ARRAY_SIZE(mma8451_channels),
+ .mma_scales = { {0, 2394}, {0, 4788}, {0, 9577} },
+ .ev_cfg = MMA8452_TRANSIENT_CFG,
+ .ev_cfg_ele = MMA8452_TRANSIENT_CFG_ELE,
+ .ev_cfg_chan_shift = 1,
+ .ev_src = MMA8452_TRANSIENT_SRC,
+ .ev_src_xe = MMA8452_TRANSIENT_SRC_XTRANSE,
+ .ev_src_ye = MMA8452_TRANSIENT_SRC_YTRANSE,
+ .ev_src_ze = MMA8452_TRANSIENT_SRC_ZTRANSE,
+ .ev_ths = MMA8452_TRANSIENT_THS,
+ .ev_ths_mask = MMA8452_TRANSIENT_THS_MASK,
+ .ev_count = MMA8452_TRANSIENT_COUNT,
+ },
};
static struct attribute *mma8452_attributes[] = {
@@ -1114,7 +1198,11 @@
{
struct iio_dev *indio_dev = iio_trigger_get_drvdata(trig);
struct mma8452_data *data = iio_priv(indio_dev);
- int reg;
+ int reg, ret;
+
+ ret = mma8452_set_runtime_pm_state(data->client, state);
+ if (ret)
+ return ret;
reg = i2c_smbus_read_byte_data(data->client, MMA8452_CTRL_REG4);
if (reg < 0)
@@ -1206,6 +1294,7 @@
{ .compatible = "fsl,mma8453", .data = &mma_chip_info_table[mma8453] },
{ .compatible = "fsl,mma8652", .data = &mma_chip_info_table[mma8652] },
{ .compatible = "fsl,mma8653", .data = &mma_chip_info_table[mma8653] },
+ { .compatible = "fsl,fxls8471", .data = &mma_chip_info_table[fxls8471] },
{ }
};
MODULE_DEVICE_TABLE(of, mma8452_dt_ids);
@@ -1243,6 +1332,7 @@
case MMA8453_DEVICE_ID:
case MMA8652_DEVICE_ID:
case MMA8653_DEVICE_ID:
+ case FXLS8471_DEVICE_ID:
if (ret == data->chip_info->chip_id)
break;
default:
@@ -1340,6 +1430,15 @@
goto buffer_cleanup;
}
+ ret = pm_runtime_set_active(&client->dev);
+ if (ret < 0)
+ goto buffer_cleanup;
+
+ pm_runtime_enable(&client->dev);
+ pm_runtime_set_autosuspend_delay(&client->dev,
+ MMA8452_AUTO_SUSPEND_DELAY_MS);
+ pm_runtime_use_autosuspend(&client->dev);
+
ret = iio_device_register(indio_dev);
if (ret < 0)
goto buffer_cleanup;
@@ -1364,6 +1463,11 @@
struct iio_dev *indio_dev = i2c_get_clientdata(client);
iio_device_unregister(indio_dev);
+
+ pm_runtime_disable(&client->dev);
+ pm_runtime_set_suspended(&client->dev);
+ pm_runtime_put_noidle(&client->dev);
+
iio_triggered_buffer_cleanup(indio_dev);
mma8452_trigger_cleanup(indio_dev);
mma8452_standby(iio_priv(indio_dev));
@@ -1371,6 +1475,45 @@
return 0;
}
+#ifdef CONFIG_PM
+static int mma8452_runtime_suspend(struct device *dev)
+{
+ struct iio_dev *indio_dev = i2c_get_clientdata(to_i2c_client(dev));
+ struct mma8452_data *data = iio_priv(indio_dev);
+ int ret;
+
+ mutex_lock(&data->lock);
+ ret = mma8452_standby(data);
+ mutex_unlock(&data->lock);
+ if (ret < 0) {
+ dev_err(&data->client->dev, "powering off device failed\n");
+ return -EAGAIN;
+ }
+
+ return 0;
+}
+
+static int mma8452_runtime_resume(struct device *dev)
+{
+ struct iio_dev *indio_dev = i2c_get_clientdata(to_i2c_client(dev));
+ struct mma8452_data *data = iio_priv(indio_dev);
+ int ret, sleep_val;
+
+ ret = mma8452_active(data);
+ if (ret < 0)
+ return ret;
+
+ ret = mma8452_get_odr_index(data);
+ sleep_val = 1000 / mma8452_samp_freq[ret][0];
+ if (sleep_val < 20)
+ usleep_range(sleep_val * 1000, 20000);
+ else
+ msleep_interruptible(sleep_val);
+
+ return 0;
+}
+#endif
+
#ifdef CONFIG_PM_SLEEP
static int mma8452_suspend(struct device *dev)
{
@@ -1383,18 +1526,21 @@
return mma8452_active(iio_priv(i2c_get_clientdata(
to_i2c_client(dev))));
}
-
-static SIMPLE_DEV_PM_OPS(mma8452_pm_ops, mma8452_suspend, mma8452_resume);
-#define MMA8452_PM_OPS (&mma8452_pm_ops)
-#else
-#define MMA8452_PM_OPS NULL
#endif
+static const struct dev_pm_ops mma8452_pm_ops = {
+ SET_SYSTEM_SLEEP_PM_OPS(mma8452_suspend, mma8452_resume)
+ SET_RUNTIME_PM_OPS(mma8452_runtime_suspend,
+ mma8452_runtime_resume, NULL)
+};
+
static const struct i2c_device_id mma8452_id[] = {
+ { "mma8451", mma8451 },
{ "mma8452", mma8452 },
{ "mma8453", mma8453 },
{ "mma8652", mma8652 },
{ "mma8653", mma8653 },
+ { "fxls8471", fxls8471 },
{ }
};
MODULE_DEVICE_TABLE(i2c, mma8452_id);
@@ -1403,7 +1549,7 @@
.driver = {
.name = "mma8452",
.of_match_table = of_match_ptr(mma8452_dt_ids),
- .pm = MMA8452_PM_OPS,
+ .pm = &mma8452_pm_ops,
},
.probe = mma8452_probe,
.remove = mma8452_remove,
diff --git a/drivers/iio/accel/mma9553.c b/drivers/iio/accel/mma9553.c
index fa7d362..bb05f3e 100644
--- a/drivers/iio/accel/mma9553.c
+++ b/drivers/iio/accel/mma9553.c
@@ -17,7 +17,6 @@
#include <linux/interrupt.h>
#include <linux/slab.h>
#include <linux/acpi.h>
-#include <linux/gpio/consumer.h>
#include <linux/iio/iio.h>
#include <linux/iio/sysfs.h>
#include <linux/iio/events.h>
diff --git a/drivers/iio/accel/mxc4005.c b/drivers/iio/accel/mxc4005.c
index e72e218..c23f47a 100644
--- a/drivers/iio/accel/mxc4005.c
+++ b/drivers/iio/accel/mxc4005.c
@@ -17,7 +17,6 @@
#include <linux/i2c.h>
#include <linux/iio/iio.h>
#include <linux/acpi.h>
-#include <linux/gpio/consumer.h>
#include <linux/regmap.h>
#include <linux/iio/sysfs.h>
#include <linux/iio/trigger.h>
@@ -380,31 +379,6 @@
.owner = THIS_MODULE,
};
-static int mxc4005_gpio_probe(struct i2c_client *client,
- struct mxc4005_data *data)
-{
- struct device *dev;
- struct gpio_desc *gpio;
- int ret;
-
- if (!client)
- return -EINVAL;
-
- dev = &client->dev;
-
- gpio = devm_gpiod_get_index(dev, "mxc4005_int", 0, GPIOD_IN);
- if (IS_ERR(gpio)) {
- dev_err(dev, "failed to get acpi gpio index\n");
- return PTR_ERR(gpio);
- }
-
- ret = gpiod_to_irq(gpio);
-
- dev_dbg(dev, "GPIO resource, no:%d irq:%d\n", desc_to_gpio(gpio), ret);
-
- return ret;
-}
-
static int mxc4005_chip_init(struct mxc4005_data *data)
{
int ret;
@@ -470,9 +444,6 @@
return ret;
}
- if (client->irq < 0)
- client->irq = mxc4005_gpio_probe(client, data);
-
if (client->irq > 0) {
data->dready_trig = devm_iio_trigger_alloc(&client->dev,
"%s-dev%d",
diff --git a/drivers/iio/accel/st_accel.h b/drivers/iio/accel/st_accel.h
index 5d4a189..57f83a6 100644
--- a/drivers/iio/accel/st_accel.h
+++ b/drivers/iio/accel/st_accel.h
@@ -14,6 +14,7 @@
#include <linux/types.h>
#include <linux/iio/common/st_sensors.h>
+#define H3LIS331DL_DRIVER_NAME "h3lis331dl_accel"
#define LIS3LV02DL_ACCEL_DEV_NAME "lis3lv02dl_accel"
#define LSM303DLHC_ACCEL_DEV_NAME "lsm303dlhc_accel"
#define LIS3DH_ACCEL_DEV_NAME "lis3dh"
diff --git a/drivers/iio/accel/st_accel_core.c b/drivers/iio/accel/st_accel_core.c
index a03a141..dc73f2d 100644
--- a/drivers/iio/accel/st_accel_core.c
+++ b/drivers/iio/accel/st_accel_core.c
@@ -39,6 +39,9 @@
#define ST_ACCEL_FS_AVL_6G 6
#define ST_ACCEL_FS_AVL_8G 8
#define ST_ACCEL_FS_AVL_16G 16
+#define ST_ACCEL_FS_AVL_100G 100
+#define ST_ACCEL_FS_AVL_200G 200
+#define ST_ACCEL_FS_AVL_400G 400
/* CUSTOM VALUES FOR SENSOR 1 */
#define ST_ACCEL_1_WAI_EXP 0x33
@@ -96,6 +99,8 @@
#define ST_ACCEL_2_DRDY_IRQ_INT2_MASK 0x10
#define ST_ACCEL_2_IHL_IRQ_ADDR 0x22
#define ST_ACCEL_2_IHL_IRQ_MASK 0x80
+#define ST_ACCEL_2_OD_IRQ_ADDR 0x22
+#define ST_ACCEL_2_OD_IRQ_MASK 0x40
#define ST_ACCEL_2_MULTIREAD_BIT true
/* CUSTOM VALUES FOR SENSOR 3 */
@@ -177,10 +182,39 @@
#define ST_ACCEL_5_DRDY_IRQ_INT2_MASK 0x20
#define ST_ACCEL_5_IHL_IRQ_ADDR 0x22
#define ST_ACCEL_5_IHL_IRQ_MASK 0x80
+#define ST_ACCEL_5_OD_IRQ_ADDR 0x22
+#define ST_ACCEL_5_OD_IRQ_MASK 0x40
#define ST_ACCEL_5_IG1_EN_ADDR 0x21
#define ST_ACCEL_5_IG1_EN_MASK 0x08
#define ST_ACCEL_5_MULTIREAD_BIT false
+/* CUSTOM VALUES FOR SENSOR 6 */
+#define ST_ACCEL_6_WAI_EXP 0x32
+#define ST_ACCEL_6_ODR_ADDR 0x20
+#define ST_ACCEL_6_ODR_MASK 0x18
+#define ST_ACCEL_6_ODR_AVL_50HZ_VAL 0x00
+#define ST_ACCEL_6_ODR_AVL_100HZ_VAL 0x01
+#define ST_ACCEL_6_ODR_AVL_400HZ_VAL 0x02
+#define ST_ACCEL_6_ODR_AVL_1000HZ_VAL 0x03
+#define ST_ACCEL_6_PW_ADDR 0x20
+#define ST_ACCEL_6_PW_MASK 0x20
+#define ST_ACCEL_6_FS_ADDR 0x23
+#define ST_ACCEL_6_FS_MASK 0x30
+#define ST_ACCEL_6_FS_AVL_100_VAL 0x00
+#define ST_ACCEL_6_FS_AVL_200_VAL 0x01
+#define ST_ACCEL_6_FS_AVL_400_VAL 0x03
+#define ST_ACCEL_6_FS_AVL_100_GAIN IIO_G_TO_M_S_2(49000)
+#define ST_ACCEL_6_FS_AVL_200_GAIN IIO_G_TO_M_S_2(98000)
+#define ST_ACCEL_6_FS_AVL_400_GAIN IIO_G_TO_M_S_2(195000)
+#define ST_ACCEL_6_BDU_ADDR 0x23
+#define ST_ACCEL_6_BDU_MASK 0x80
+#define ST_ACCEL_6_DRDY_IRQ_ADDR 0x22
+#define ST_ACCEL_6_DRDY_IRQ_INT1_MASK 0x02
+#define ST_ACCEL_6_DRDY_IRQ_INT2_MASK 0x10
+#define ST_ACCEL_6_IHL_IRQ_ADDR 0x22
+#define ST_ACCEL_6_IHL_IRQ_MASK 0x80
+#define ST_ACCEL_6_MULTIREAD_BIT true
+
static const struct iio_chan_spec st_accel_8bit_channels[] = {
ST_SENSORS_LSM_CHANNELS(IIO_ACCEL,
BIT(IIO_CHAN_INFO_RAW) | BIT(IIO_CHAN_INFO_SCALE),
@@ -302,6 +336,7 @@
.mask_int2 = ST_ACCEL_1_DRDY_IRQ_INT2_MASK,
.addr_ihl = ST_ACCEL_1_IHL_IRQ_ADDR,
.mask_ihl = ST_ACCEL_1_IHL_IRQ_MASK,
+ .addr_stat_drdy = ST_SENSORS_DEFAULT_STAT_ADDR,
},
.multi_read_bit = ST_ACCEL_1_MULTIREAD_BIT,
.bootime = 2,
@@ -367,6 +402,9 @@
.mask_int2 = ST_ACCEL_2_DRDY_IRQ_INT2_MASK,
.addr_ihl = ST_ACCEL_2_IHL_IRQ_ADDR,
.mask_ihl = ST_ACCEL_2_IHL_IRQ_MASK,
+ .addr_od = ST_ACCEL_2_OD_IRQ_ADDR,
+ .mask_od = ST_ACCEL_2_OD_IRQ_MASK,
+ .addr_stat_drdy = ST_SENSORS_DEFAULT_STAT_ADDR,
},
.multi_read_bit = ST_ACCEL_2_MULTIREAD_BIT,
.bootime = 2,
@@ -444,6 +482,7 @@
.mask_int2 = ST_ACCEL_3_DRDY_IRQ_INT2_MASK,
.addr_ihl = ST_ACCEL_3_IHL_IRQ_ADDR,
.mask_ihl = ST_ACCEL_3_IHL_IRQ_MASK,
+ .addr_stat_drdy = ST_SENSORS_DEFAULT_STAT_ADDR,
.ig1 = {
.en_addr = ST_ACCEL_3_IG1_EN_ADDR,
.en_mask = ST_ACCEL_3_IG1_EN_MASK,
@@ -502,6 +541,7 @@
.drdy_irq = {
.addr = ST_ACCEL_4_DRDY_IRQ_ADDR,
.mask_int1 = ST_ACCEL_4_DRDY_IRQ_INT1_MASK,
+ .addr_stat_drdy = ST_SENSORS_DEFAULT_STAT_ADDR,
},
.multi_read_bit = ST_ACCEL_4_MULTIREAD_BIT,
.bootime = 2, /* guess */
@@ -553,10 +593,75 @@
.mask_int2 = ST_ACCEL_5_DRDY_IRQ_INT2_MASK,
.addr_ihl = ST_ACCEL_5_IHL_IRQ_ADDR,
.mask_ihl = ST_ACCEL_5_IHL_IRQ_MASK,
+ .addr_od = ST_ACCEL_5_OD_IRQ_ADDR,
+ .mask_od = ST_ACCEL_5_OD_IRQ_MASK,
+ .addr_stat_drdy = ST_SENSORS_DEFAULT_STAT_ADDR,
},
.multi_read_bit = ST_ACCEL_5_MULTIREAD_BIT,
.bootime = 2, /* guess */
},
+ {
+ .wai = ST_ACCEL_6_WAI_EXP,
+ .wai_addr = ST_SENSORS_DEFAULT_WAI_ADDRESS,
+ .sensors_supported = {
+ [0] = H3LIS331DL_DRIVER_NAME,
+ },
+ .ch = (struct iio_chan_spec *)st_accel_12bit_channels,
+ .odr = {
+ .addr = ST_ACCEL_6_ODR_ADDR,
+ .mask = ST_ACCEL_6_ODR_MASK,
+ .odr_avl = {
+ { 50, ST_ACCEL_6_ODR_AVL_50HZ_VAL },
+ { 100, ST_ACCEL_6_ODR_AVL_100HZ_VAL, },
+ { 400, ST_ACCEL_6_ODR_AVL_400HZ_VAL, },
+ { 1000, ST_ACCEL_6_ODR_AVL_1000HZ_VAL, },
+ },
+ },
+ .pw = {
+ .addr = ST_ACCEL_6_PW_ADDR,
+ .mask = ST_ACCEL_6_PW_MASK,
+ .value_on = ST_SENSORS_DEFAULT_POWER_ON_VALUE,
+ .value_off = ST_SENSORS_DEFAULT_POWER_OFF_VALUE,
+ },
+ .enable_axis = {
+ .addr = ST_SENSORS_DEFAULT_AXIS_ADDR,
+ .mask = ST_SENSORS_DEFAULT_AXIS_MASK,
+ },
+ .fs = {
+ .addr = ST_ACCEL_6_FS_ADDR,
+ .mask = ST_ACCEL_6_FS_MASK,
+ .fs_avl = {
+ [0] = {
+ .num = ST_ACCEL_FS_AVL_100G,
+ .value = ST_ACCEL_6_FS_AVL_100_VAL,
+ .gain = ST_ACCEL_6_FS_AVL_100_GAIN,
+ },
+ [1] = {
+ .num = ST_ACCEL_FS_AVL_200G,
+ .value = ST_ACCEL_6_FS_AVL_200_VAL,
+ .gain = ST_ACCEL_6_FS_AVL_200_GAIN,
+ },
+ [2] = {
+ .num = ST_ACCEL_FS_AVL_400G,
+ .value = ST_ACCEL_6_FS_AVL_400_VAL,
+ .gain = ST_ACCEL_6_FS_AVL_400_GAIN,
+ },
+ },
+ },
+ .bdu = {
+ .addr = ST_ACCEL_6_BDU_ADDR,
+ .mask = ST_ACCEL_6_BDU_MASK,
+ },
+ .drdy_irq = {
+ .addr = ST_ACCEL_6_DRDY_IRQ_ADDR,
+ .mask_int1 = ST_ACCEL_6_DRDY_IRQ_INT1_MASK,
+ .mask_int2 = ST_ACCEL_6_DRDY_IRQ_INT2_MASK,
+ .addr_ihl = ST_ACCEL_6_IHL_IRQ_ADDR,
+ .mask_ihl = ST_ACCEL_6_IHL_IRQ_MASK,
+ },
+ .multi_read_bit = ST_ACCEL_6_MULTIREAD_BIT,
+ .bootime = 2,
+ },
};
static int st_accel_read_raw(struct iio_dev *indio_dev,
diff --git a/drivers/iio/accel/st_accel_i2c.c b/drivers/iio/accel/st_accel_i2c.c
index 294a32f..7333ee9 100644
--- a/drivers/iio/accel/st_accel_i2c.c
+++ b/drivers/iio/accel/st_accel_i2c.c
@@ -76,6 +76,10 @@
.compatible = "st,lis2dh12-accel",
.data = LIS2DH12_ACCEL_DEV_NAME,
},
+ {
+ .compatible = "st,h3lis331dl-accel",
+ .data = H3LIS331DL_DRIVER_NAME,
+ },
{},
};
MODULE_DEVICE_TABLE(of, st_accel_of_match);
diff --git a/drivers/iio/accel/stk8312.c b/drivers/iio/accel/stk8312.c
index 85fe7f7..e31023d 100644
--- a/drivers/iio/accel/stk8312.c
+++ b/drivers/iio/accel/stk8312.c
@@ -11,7 +11,6 @@
*/
#include <linux/acpi.h>
-#include <linux/gpio/consumer.h>
#include <linux/i2c.h>
#include <linux/interrupt.h>
#include <linux/kernel.h>
diff --git a/drivers/iio/accel/stk8ba50.c b/drivers/iio/accel/stk8ba50.c
index 5709d9e..300d955 100644
--- a/drivers/iio/accel/stk8ba50.c
+++ b/drivers/iio/accel/stk8ba50.c
@@ -11,7 +11,6 @@
*/
#include <linux/acpi.h>
-#include <linux/gpio/consumer.h>
#include <linux/i2c.h>
#include <linux/interrupt.h>
#include <linux/kernel.h>
diff --git a/drivers/iio/adc/Kconfig b/drivers/iio/adc/Kconfig
index 82c718c..25378c5 100644
--- a/drivers/iio/adc/Kconfig
+++ b/drivers/iio/adc/Kconfig
@@ -242,6 +242,16 @@
To compile this driver as a module, choose M here: the module will be
called lp8788_adc.
+config LPC18XX_ADC
+ tristate "NXP LPC18xx ADC driver"
+ depends on ARCH_LPC18XX || COMPILE_TEST
+ depends on OF && HAS_IOMEM
+ help
+ Say yes here to build support for NXP LPC18XX ADC.
+
+ To compile this driver as a module, choose M here: the module will be
+ called lpc18xx_adc.
+
config MAX1027
tristate "Maxim max1027 ADC driver"
depends on SPI
@@ -375,11 +385,11 @@
module will be called rockchip_saradc.
config TI_ADC081C
- tristate "Texas Instruments ADC081C021/027"
+ tristate "Texas Instruments ADC081C/ADC101C/ADC121C family"
depends on I2C
help
- If you say yes here you get support for Texas Instruments ADC081C021
- and ADC081C027 ADC chips.
+ If you say yes here you get support for Texas Instruments ADC081C,
+ ADC101C and ADC121C ADC chips.
This driver can also be built as a module. If so, the module will be
called ti-adc081c.
diff --git a/drivers/iio/adc/Makefile b/drivers/iio/adc/Makefile
index 0cb7921..38638d4 100644
--- a/drivers/iio/adc/Makefile
+++ b/drivers/iio/adc/Makefile
@@ -25,6 +25,7 @@
obj-$(CONFIG_IMX7D_ADC) += imx7d_adc.o
obj-$(CONFIG_INA2XX_ADC) += ina2xx-adc.o
obj-$(CONFIG_LP8788_ADC) += lp8788_adc.o
+obj-$(CONFIG_LPC18XX_ADC) += lpc18xx_adc.o
obj-$(CONFIG_MAX1027) += max1027.o
obj-$(CONFIG_MAX1363) += max1363.o
obj-$(CONFIG_MCP320X) += mcp320x.o
diff --git a/drivers/iio/adc/ad799x.c b/drivers/iio/adc/ad799x.c
index 01d7158..a3f5254 100644
--- a/drivers/iio/adc/ad799x.c
+++ b/drivers/iio/adc/ad799x.c
@@ -477,7 +477,7 @@
if (ret < 0)
return ret;
*val = (ret >> chan->scan_type.shift) &
- GENMASK(chan->scan_type.realbits - 1 , 0);
+ GENMASK(chan->scan_type.realbits - 1, 0);
return IIO_VAL_INT;
}
diff --git a/drivers/iio/adc/at91-sama5d2_adc.c b/drivers/iio/adc/at91-sama5d2_adc.c
index 2e154cb..e10dca3 100644
--- a/drivers/iio/adc/at91-sama5d2_adc.c
+++ b/drivers/iio/adc/at91-sama5d2_adc.c
@@ -66,8 +66,10 @@
#define AT91_SAMA5D2_MR_PRESCAL(v) ((v) << AT91_SAMA5D2_MR_PRESCAL_OFFSET)
#define AT91_SAMA5D2_MR_PRESCAL_OFFSET 8
#define AT91_SAMA5D2_MR_PRESCAL_MAX 0xff
+#define AT91_SAMA5D2_MR_PRESCAL_MASK GENMASK(15, 8)
/* Startup Time */
#define AT91_SAMA5D2_MR_STARTUP(v) ((v) << 16)
+#define AT91_SAMA5D2_MR_STARTUP_MASK GENMASK(19, 16)
/* Analog Change */
#define AT91_SAMA5D2_MR_ANACH BIT(23)
/* Tracking Time */
@@ -92,13 +94,13 @@
/* Last Converted Data Register */
#define AT91_SAMA5D2_LCDR 0x20
/* Interrupt Enable Register */
-#define AT91_SAMA5D2_IER 0x24
+#define AT91_SAMA5D2_IER 0x24
/* Interrupt Disable Register */
-#define AT91_SAMA5D2_IDR 0x28
+#define AT91_SAMA5D2_IDR 0x28
/* Interrupt Mask Register */
-#define AT91_SAMA5D2_IMR 0x2c
+#define AT91_SAMA5D2_IMR 0x2c
/* Interrupt Status Register */
-#define AT91_SAMA5D2_ISR 0x30
+#define AT91_SAMA5D2_ISR 0x30
/* Last Channel Trigger Mode Register */
#define AT91_SAMA5D2_LCTMR 0x34
/* Last Channel Compare Window Register */
@@ -106,17 +108,20 @@
/* Overrun Status Register */
#define AT91_SAMA5D2_OVER 0x3c
/* Extended Mode Register */
-#define AT91_SAMA5D2_EMR 0x40
+#define AT91_SAMA5D2_EMR 0x40
/* Compare Window Register */
-#define AT91_SAMA5D2_CWR 0x44
+#define AT91_SAMA5D2_CWR 0x44
/* Channel Gain Register */
-#define AT91_SAMA5D2_CGR 0x48
+#define AT91_SAMA5D2_CGR 0x48
+
/* Channel Offset Register */
-#define AT91_SAMA5D2_COR 0x4c
+#define AT91_SAMA5D2_COR 0x4c
+#define AT91_SAMA5D2_COR_DIFF_OFFSET 16
+
/* Channel Data Register 0 */
#define AT91_SAMA5D2_CDR0 0x50
/* Analog Control Register */
-#define AT91_SAMA5D2_ACR 0x94
+#define AT91_SAMA5D2_ACR 0x94
/* Touchscreen Mode Register */
#define AT91_SAMA5D2_TSMR 0xb0
/* Touchscreen X Position Register */
@@ -130,7 +135,7 @@
/* Correction Select Register */
#define AT91_SAMA5D2_COSR 0xd0
/* Correction Value Register */
-#define AT91_SAMA5D2_CVR 0xd4
+#define AT91_SAMA5D2_CVR 0xd4
/* Channel Error Correction Register */
#define AT91_SAMA5D2_CECR 0xd8
/* Write Protection Mode Register */
@@ -140,7 +145,7 @@
/* Version Register */
#define AT91_SAMA5D2_VERSION 0xfc
-#define AT91_AT91_SAMA5D2_CHAN(num, addr) \
+#define AT91_SAMA5D2_CHAN_SINGLE(num, addr) \
{ \
.type = IIO_VOLTAGE, \
.channel = num, \
@@ -156,6 +161,24 @@
.indexed = 1, \
}
+#define AT91_SAMA5D2_CHAN_DIFF(num, num2, addr) \
+ { \
+ .type = IIO_VOLTAGE, \
+ .differential = 1, \
+ .channel = num, \
+ .channel2 = num2, \
+ .address = addr, \
+ .scan_type = { \
+ .sign = 's', \
+ .realbits = 12, \
+ }, \
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW), \
+ .info_mask_shared_by_type = BIT(IIO_CHAN_INFO_SCALE), \
+ .info_mask_shared_by_all = BIT(IIO_CHAN_INFO_SAMP_FREQ),\
+ .datasheet_name = "CH"#num"-CH"#num2, \
+ .indexed = 1, \
+ }
+
#define at91_adc_readl(st, reg) readl_relaxed(st->base + reg)
#define at91_adc_writel(st, reg, val) writel_relaxed(val, st->base + reg)
@@ -185,18 +208,24 @@
};
static const struct iio_chan_spec at91_adc_channels[] = {
- AT91_AT91_SAMA5D2_CHAN(0, 0x50),
- AT91_AT91_SAMA5D2_CHAN(1, 0x54),
- AT91_AT91_SAMA5D2_CHAN(2, 0x58),
- AT91_AT91_SAMA5D2_CHAN(3, 0x5c),
- AT91_AT91_SAMA5D2_CHAN(4, 0x60),
- AT91_AT91_SAMA5D2_CHAN(5, 0x64),
- AT91_AT91_SAMA5D2_CHAN(6, 0x68),
- AT91_AT91_SAMA5D2_CHAN(7, 0x6c),
- AT91_AT91_SAMA5D2_CHAN(8, 0x70),
- AT91_AT91_SAMA5D2_CHAN(9, 0x74),
- AT91_AT91_SAMA5D2_CHAN(10, 0x78),
- AT91_AT91_SAMA5D2_CHAN(11, 0x7c),
+ AT91_SAMA5D2_CHAN_SINGLE(0, 0x50),
+ AT91_SAMA5D2_CHAN_SINGLE(1, 0x54),
+ AT91_SAMA5D2_CHAN_SINGLE(2, 0x58),
+ AT91_SAMA5D2_CHAN_SINGLE(3, 0x5c),
+ AT91_SAMA5D2_CHAN_SINGLE(4, 0x60),
+ AT91_SAMA5D2_CHAN_SINGLE(5, 0x64),
+ AT91_SAMA5D2_CHAN_SINGLE(6, 0x68),
+ AT91_SAMA5D2_CHAN_SINGLE(7, 0x6c),
+ AT91_SAMA5D2_CHAN_SINGLE(8, 0x70),
+ AT91_SAMA5D2_CHAN_SINGLE(9, 0x74),
+ AT91_SAMA5D2_CHAN_SINGLE(10, 0x78),
+ AT91_SAMA5D2_CHAN_SINGLE(11, 0x7c),
+ AT91_SAMA5D2_CHAN_DIFF(0, 1, 0x50),
+ AT91_SAMA5D2_CHAN_DIFF(2, 3, 0x58),
+ AT91_SAMA5D2_CHAN_DIFF(4, 5, 0x60),
+ AT91_SAMA5D2_CHAN_DIFF(6, 7, 0x68),
+ AT91_SAMA5D2_CHAN_DIFF(8, 9, 0x70),
+ AT91_SAMA5D2_CHAN_DIFF(10, 11, 0x78),
};
static unsigned at91_adc_startup_time(unsigned startup_time_min,
@@ -226,7 +255,7 @@
static void at91_adc_setup_samp_freq(struct at91_adc_state *st, unsigned freq)
{
struct iio_dev *indio_dev = iio_priv_to_dev(st);
- unsigned f_per, prescal, startup;
+ unsigned f_per, prescal, startup, mr;
f_per = clk_get_rate(st->per_clk);
prescal = (f_per / (2 * freq)) - 1;
@@ -234,10 +263,11 @@
startup = at91_adc_startup_time(st->soc_info.startup_time,
freq / 1000);
- at91_adc_writel(st, AT91_SAMA5D2_MR,
- AT91_SAMA5D2_MR_TRANSFER(2)
- | AT91_SAMA5D2_MR_STARTUP(startup)
- | AT91_SAMA5D2_MR_PRESCAL(prescal));
+ mr = at91_adc_readl(st, AT91_SAMA5D2_MR);
+ mr &= ~(AT91_SAMA5D2_MR_STARTUP_MASK | AT91_SAMA5D2_MR_PRESCAL_MASK);
+ mr |= AT91_SAMA5D2_MR_STARTUP(startup);
+ mr |= AT91_SAMA5D2_MR_PRESCAL(prescal);
+ at91_adc_writel(st, AT91_SAMA5D2_MR, mr);
dev_dbg(&indio_dev->dev, "freq: %u, startup: %u, prescal: %u\n",
freq, startup, prescal);
@@ -278,6 +308,7 @@
int *val, int *val2, long mask)
{
struct at91_adc_state *st = iio_priv(indio_dev);
+ u32 cor = 0;
int ret;
switch (mask) {
@@ -286,6 +317,11 @@
st->chan = chan;
+ if (chan->differential)
+ cor = (BIT(chan->channel) | BIT(chan->channel2)) <<
+ AT91_SAMA5D2_COR_DIFF_OFFSET;
+
+ at91_adc_writel(st, AT91_SAMA5D2_COR, cor);
at91_adc_writel(st, AT91_SAMA5D2_CHER, BIT(chan->channel));
at91_adc_writel(st, AT91_SAMA5D2_IER, BIT(chan->channel));
at91_adc_writel(st, AT91_SAMA5D2_CR, AT91_SAMA5D2_CR_START);
@@ -298,6 +334,8 @@
if (ret > 0) {
*val = st->conversion_value;
+ if (chan->scan_type.sign == 's')
+ *val = sign_extend32(*val, 11);
ret = IIO_VAL_INT;
st->conversion_done = false;
}
@@ -310,6 +348,8 @@
case IIO_CHAN_INFO_SCALE:
*val = st->vref_uv / 1000;
+ if (chan->differential)
+ *val *= 2;
*val2 = chan->scan_type.realbits;
return IIO_VAL_FRACTIONAL_LOG2;
@@ -444,6 +484,12 @@
at91_adc_writel(st, AT91_SAMA5D2_CR, AT91_SAMA5D2_CR_SWRST);
at91_adc_writel(st, AT91_SAMA5D2_IDR, 0xffffffff);
+ /*
+ * Transfer field must be set to 2 according to the datasheet and
+ * allows different analog settings for each channel.
+ */
+ at91_adc_writel(st, AT91_SAMA5D2_MR,
+ AT91_SAMA5D2_MR_TRANSFER(2) | AT91_SAMA5D2_MR_ANACH);
at91_adc_setup_samp_freq(st, st->soc_info.min_sample_rate);
diff --git a/drivers/iio/adc/at91_adc.c b/drivers/iio/adc/at91_adc.c
index f284cd6..52430ba 100644
--- a/drivers/iio/adc/at91_adc.c
+++ b/drivers/iio/adc/at91_adc.c
@@ -797,8 +797,8 @@
* Startup Time = <lookup_table_value> / ADC Clock
*/
const int startup_lookup[] = {
- 0 , 8 , 16 , 24 ,
- 64 , 80 , 96 , 112,
+ 0, 8, 16, 24,
+ 64, 80, 96, 112,
512, 576, 640, 704,
768, 832, 896, 960
};
@@ -924,14 +924,14 @@
ret = -EINVAL;
goto error_ret;
}
- trig->name = name;
+ trig->name = name;
if (of_property_read_u32(trig_node, "trigger-value", &prop)) {
dev_err(&idev->dev, "Missing trigger-value property in the DT.\n");
ret = -EINVAL;
goto error_ret;
}
- trig->value = prop;
+ trig->value = prop;
trig->is_external = of_property_read_bool(trig_node, "trigger-external");
i++;
}
diff --git a/drivers/iio/adc/ina2xx-adc.c b/drivers/iio/adc/ina2xx-adc.c
index 65909d5..502f2fb 100644
--- a/drivers/iio/adc/ina2xx-adc.c
+++ b/drivers/iio/adc/ina2xx-adc.c
@@ -185,9 +185,9 @@
case IIO_CHAN_INFO_SCALE:
switch (chan->address) {
case INA2XX_SHUNT_VOLTAGE:
- /* processed (mV) = raw*1000/shunt_div */
+ /* processed (mV) = raw/shunt_div */
*val2 = chip->config->shunt_div;
- *val = 1000;
+ *val = 1;
return IIO_VAL_FRACTIONAL;
case INA2XX_BUS_VOLTAGE:
@@ -350,6 +350,23 @@
return len;
}
+/*
+ * Set current LSB to 1mA, shunt is in uOhms
+ * (equation 13 in datasheet). We hardcode a Current_LSB
+ * of 1.0 x10-6. The only remaining parameter is RShunt.
+ * There is no need to expose the CALIBRATION register
+ * to the user for now. But we need to reset this register
+ * if the user updates RShunt after driver init, e.g upon
+ * reading an EEPROM/Probe-type value.
+ */
+static int ina2xx_set_calibration(struct ina2xx_chip_info *chip)
+{
+ u16 regval = DIV_ROUND_CLOSEST(chip->config->calibration_factor,
+ chip->shunt_resistor);
+
+ return regmap_write(chip->regmap, INA2XX_CALIBRATION, regval);
+}
+
static int set_shunt_resistor(struct ina2xx_chip_info *chip, unsigned int val)
{
if (val <= 0 || val > chip->config->calibration_factor)
@@ -385,6 +402,11 @@
if (ret)
return ret;
+ /* Update the Calibration register */
+ ret = ina2xx_set_calibration(chip);
+ if (ret)
+ return ret;
+
return len;
}
@@ -602,24 +624,11 @@
/* Initialize the configuration and calibration registers. */
static int ina2xx_init(struct ina2xx_chip_info *chip, unsigned int config)
{
- u16 regval;
- int ret;
-
- ret = regmap_write(chip->regmap, INA2XX_CONFIG, config);
+ int ret = regmap_write(chip->regmap, INA2XX_CONFIG, config);
if (ret)
return ret;
- /*
- * Set current LSB to 1mA, shunt is in uOhms
- * (equation 13 in datasheet). We hardcode a Current_LSB
- * of 1.0 x10-6. The only remaining parameter is RShunt.
- * There is no need to expose the CALIBRATION register
- * to the user for now.
- */
- regval = DIV_ROUND_CLOSEST(chip->config->calibration_factor,
- chip->shunt_resistor);
-
- return regmap_write(chip->regmap, INA2XX_CALIBRATION, regval);
+ return ina2xx_set_calibration(chip);
}
static int ina2xx_probe(struct i2c_client *client,
diff --git a/drivers/iio/adc/lpc18xx_adc.c b/drivers/iio/adc/lpc18xx_adc.c
new file mode 100644
index 0000000..3ef18f4
--- /dev/null
+++ b/drivers/iio/adc/lpc18xx_adc.c
@@ -0,0 +1,231 @@
+/*
+ * IIO ADC driver for NXP LPC18xx ADC
+ *
+ * Copyright (C) 2016 Joachim Eastwood <manabian@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * UNSUPPORTED hardware features:
+ * - Hardware triggers
+ * - Burst mode
+ * - Interrupts
+ * - DMA
+ */
+
+#include <linux/clk.h>
+#include <linux/err.h>
+#include <linux/iio/iio.h>
+#include <linux/iio/driver.h>
+#include <linux/io.h>
+#include <linux/iopoll.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+#include <linux/regulator/consumer.h>
+
+/* LPC18XX ADC registers and bits */
+#define LPC18XX_ADC_CR 0x000
+#define LPC18XX_ADC_CR_CLKDIV_SHIFT 8
+#define LPC18XX_ADC_CR_PDN BIT(21)
+#define LPC18XX_ADC_CR_START_NOW (0x1 << 24)
+#define LPC18XX_ADC_GDR 0x004
+
+/* Data register bits */
+#define LPC18XX_ADC_SAMPLE_SHIFT 6
+#define LPC18XX_ADC_SAMPLE_MASK 0x3ff
+#define LPC18XX_ADC_CONV_DONE BIT(31)
+
+/* Clock should be 4.5 MHz or less */
+#define LPC18XX_ADC_CLK_TARGET 4500000
+
+struct lpc18xx_adc {
+ struct regulator *vref;
+ void __iomem *base;
+ struct device *dev;
+ struct mutex lock;
+ struct clk *clk;
+ u32 cr_reg;
+};
+
+#define LPC18XX_ADC_CHAN(_idx) { \
+ .type = IIO_VOLTAGE, \
+ .indexed = 1, \
+ .channel = _idx, \
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW), \
+ .info_mask_shared_by_type = BIT(IIO_CHAN_INFO_SCALE), \
+}
+
+static const struct iio_chan_spec lpc18xx_adc_iio_channels[] = {
+ LPC18XX_ADC_CHAN(0),
+ LPC18XX_ADC_CHAN(1),
+ LPC18XX_ADC_CHAN(2),
+ LPC18XX_ADC_CHAN(3),
+ LPC18XX_ADC_CHAN(4),
+ LPC18XX_ADC_CHAN(5),
+ LPC18XX_ADC_CHAN(6),
+ LPC18XX_ADC_CHAN(7),
+};
+
+static int lpc18xx_adc_read_chan(struct lpc18xx_adc *adc, unsigned int ch)
+{
+ int ret;
+ u32 reg;
+
+ reg = adc->cr_reg | BIT(ch) | LPC18XX_ADC_CR_START_NOW;
+ writel(reg, adc->base + LPC18XX_ADC_CR);
+
+ ret = readl_poll_timeout(adc->base + LPC18XX_ADC_GDR, reg,
+ reg & LPC18XX_ADC_CONV_DONE, 3, 9);
+ if (ret) {
+ dev_warn(adc->dev, "adc read timed out\n");
+ return ret;
+ }
+
+ return (reg >> LPC18XX_ADC_SAMPLE_SHIFT) & LPC18XX_ADC_SAMPLE_MASK;
+}
+
+static int lpc18xx_adc_read_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int *val, int *val2, long mask)
+{
+ struct lpc18xx_adc *adc = iio_priv(indio_dev);
+
+ switch (mask) {
+ case IIO_CHAN_INFO_RAW:
+ mutex_lock(&adc->lock);
+ *val = lpc18xx_adc_read_chan(adc, chan->channel);
+ mutex_unlock(&adc->lock);
+ if (*val < 0)
+ return *val;
+
+ return IIO_VAL_INT;
+
+ case IIO_CHAN_INFO_SCALE:
+ *val = regulator_get_voltage(adc->vref) / 1000;
+ *val2 = 10;
+
+ return IIO_VAL_FRACTIONAL_LOG2;
+ }
+
+ return -EINVAL;
+}
+
+static const struct iio_info lpc18xx_adc_info = {
+ .read_raw = lpc18xx_adc_read_raw,
+ .driver_module = THIS_MODULE,
+};
+
+static int lpc18xx_adc_probe(struct platform_device *pdev)
+{
+ struct iio_dev *indio_dev;
+ struct lpc18xx_adc *adc;
+ struct resource *res;
+ unsigned int clkdiv;
+ unsigned long rate;
+ int ret;
+
+ indio_dev = devm_iio_device_alloc(&pdev->dev, sizeof(*adc));
+ if (!indio_dev)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, indio_dev);
+ adc = iio_priv(indio_dev);
+ adc->dev = &pdev->dev;
+ mutex_init(&adc->lock);
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ adc->base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(adc->base))
+ return PTR_ERR(adc->base);
+
+ adc->clk = devm_clk_get(&pdev->dev, NULL);
+ if (IS_ERR(adc->clk)) {
+ dev_err(&pdev->dev, "error getting clock\n");
+ return PTR_ERR(adc->clk);
+ }
+
+ rate = clk_get_rate(adc->clk);
+ clkdiv = DIV_ROUND_UP(rate, LPC18XX_ADC_CLK_TARGET);
+
+ adc->vref = devm_regulator_get(&pdev->dev, "vref");
+ if (IS_ERR(adc->vref)) {
+ dev_err(&pdev->dev, "error getting regulator\n");
+ return PTR_ERR(adc->vref);
+ }
+
+ indio_dev->name = dev_name(&pdev->dev);
+ indio_dev->dev.parent = &pdev->dev;
+ indio_dev->info = &lpc18xx_adc_info;
+ indio_dev->modes = INDIO_DIRECT_MODE;
+ indio_dev->channels = lpc18xx_adc_iio_channels;
+ indio_dev->num_channels = ARRAY_SIZE(lpc18xx_adc_iio_channels);
+
+ ret = regulator_enable(adc->vref);
+ if (ret) {
+ dev_err(&pdev->dev, "unable to enable regulator\n");
+ return ret;
+ }
+
+ ret = clk_prepare_enable(adc->clk);
+ if (ret) {
+ dev_err(&pdev->dev, "unable to enable clock\n");
+ goto dis_reg;
+ }
+
+ adc->cr_reg = (clkdiv << LPC18XX_ADC_CR_CLKDIV_SHIFT) |
+ LPC18XX_ADC_CR_PDN;
+ writel(adc->cr_reg, adc->base + LPC18XX_ADC_CR);
+
+ ret = iio_device_register(indio_dev);
+ if (ret) {
+ dev_err(&pdev->dev, "unable to register device\n");
+ goto dis_clk;
+ }
+
+ return 0;
+
+dis_clk:
+ writel(0, adc->base + LPC18XX_ADC_CR);
+ clk_disable_unprepare(adc->clk);
+dis_reg:
+ regulator_disable(adc->vref);
+ return ret;
+}
+
+static int lpc18xx_adc_remove(struct platform_device *pdev)
+{
+ struct iio_dev *indio_dev = platform_get_drvdata(pdev);
+ struct lpc18xx_adc *adc = iio_priv(indio_dev);
+
+ iio_device_unregister(indio_dev);
+
+ writel(0, adc->base + LPC18XX_ADC_CR);
+ clk_disable_unprepare(adc->clk);
+ regulator_disable(adc->vref);
+
+ return 0;
+}
+
+static const struct of_device_id lpc18xx_adc_match[] = {
+ { .compatible = "nxp,lpc1850-adc" },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, lpc18xx_adc_match);
+
+static struct platform_driver lpc18xx_adc_driver = {
+ .probe = lpc18xx_adc_probe,
+ .remove = lpc18xx_adc_remove,
+ .driver = {
+ .name = "lpc18xx-adc",
+ .of_match_table = lpc18xx_adc_match,
+ },
+};
+module_platform_driver(lpc18xx_adc_driver);
+
+MODULE_DESCRIPTION("LPC18xx ADC driver");
+MODULE_AUTHOR("Joachim Eastwood <manabian@gmail.com>");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/iio/adc/mcp3422.c b/drivers/iio/adc/mcp3422.c
index d7b36ef..d1172dc1 100644
--- a/drivers/iio/adc/mcp3422.c
+++ b/drivers/iio/adc/mcp3422.c
@@ -61,9 +61,9 @@
static const int mcp3422_scales[4][4] = {
{ 1000000, 500000, 250000, 125000 },
- { 250000 , 125000, 62500 , 31250 },
- { 62500 , 31250 , 15625 , 7812 },
- { 15625 , 7812 , 3906 , 1953 } };
+ { 250000, 125000, 62500, 31250 },
+ { 62500, 31250, 15625, 7812 },
+ { 15625, 7812, 3906, 1953 } };
/* Constant msleep times for data acquisitions */
static const int mcp3422_read_times[4] = {
diff --git a/drivers/iio/adc/mxs-lradc.c b/drivers/iio/adc/mxs-lradc.c
index 33051b8..ad26da1 100644
--- a/drivers/iio/adc/mxs-lradc.c
+++ b/drivers/iio/adc/mxs-lradc.c
@@ -686,6 +686,17 @@
static void mxs_lradc_enable_touch_detection(struct mxs_lradc *lradc)
{
+ /* Configure the touchscreen type */
+ if (lradc->soc == IMX28_LRADC) {
+ mxs_lradc_reg_clear(lradc, LRADC_CTRL0_MX28_TOUCH_SCREEN_TYPE,
+ LRADC_CTRL0);
+
+ if (lradc->use_touchscreen == MXS_LRADC_TOUCHSCREEN_5WIRE)
+ mxs_lradc_reg_set(lradc,
+ LRADC_CTRL0_MX28_TOUCH_SCREEN_TYPE,
+ LRADC_CTRL0);
+ }
+
mxs_lradc_setup_touch_detection(lradc);
lradc->cur_plate = LRADC_TOUCH;
@@ -1127,6 +1138,7 @@
__set_bit(EV_ABS, input->evbit);
__set_bit(EV_KEY, input->evbit);
__set_bit(BTN_TOUCH, input->keybit);
+ __set_bit(INPUT_PROP_DIRECT, input->propbit);
input_set_abs_params(input, ABS_X, 0, LRADC_SINGLE_SAMPLE_MASK, 0, 0);
input_set_abs_params(input, ABS_Y, 0, LRADC_SINGLE_SAMPLE_MASK, 0, 0);
input_set_abs_params(input, ABS_PRESSURE, 0, LRADC_SINGLE_SAMPLE_MASK,
@@ -1475,18 +1487,13 @@
MXS_ADC_CHAN(15, IIO_VOLTAGE, "VDD5V"),
};
-static int mxs_lradc_hw_init(struct mxs_lradc *lradc)
+static void mxs_lradc_hw_init(struct mxs_lradc *lradc)
{
/* The ADC always uses DELAY CHANNEL 0. */
const u32 adc_cfg =
(1 << (LRADC_DELAY_TRIGGER_DELAYS_OFFSET + 0)) |
(LRADC_DELAY_TIMER_PER << LRADC_DELAY_DELAY_OFFSET);
- int ret = stmp_reset_block(lradc->base);
-
- if (ret)
- return ret;
-
/* Configure DELAY CHANNEL 0 for generic ADC sampling. */
mxs_lradc_reg_wrt(lradc, adc_cfg, LRADC_DELAY(0));
@@ -1495,20 +1502,8 @@
mxs_lradc_reg_wrt(lradc, 0, LRADC_DELAY(2));
mxs_lradc_reg_wrt(lradc, 0, LRADC_DELAY(3));
- /* Configure the touchscreen type */
- if (lradc->soc == IMX28_LRADC) {
- mxs_lradc_reg_clear(lradc, LRADC_CTRL0_MX28_TOUCH_SCREEN_TYPE,
- LRADC_CTRL0);
-
- if (lradc->use_touchscreen == MXS_LRADC_TOUCHSCREEN_5WIRE)
- mxs_lradc_reg_set(lradc, LRADC_CTRL0_MX28_TOUCH_SCREEN_TYPE,
- LRADC_CTRL0);
- }
-
/* Start internal temperature sensing. */
mxs_lradc_reg_wrt(lradc, 0, LRADC_CTRL2);
-
- return 0;
}
static void mxs_lradc_hw_stop(struct mxs_lradc *lradc)
@@ -1708,11 +1703,13 @@
}
}
- /* Configure the hardware. */
- ret = mxs_lradc_hw_init(lradc);
+ ret = stmp_reset_block(lradc->base);
if (ret)
goto err_dev;
+ /* Configure the hardware. */
+ mxs_lradc_hw_init(lradc);
+
/* Register the touchscreen input device. */
if (touch_ret == 0) {
ret = mxs_lradc_ts_register(lradc);
diff --git a/drivers/iio/adc/rockchip_saradc.c b/drivers/iio/adc/rockchip_saradc.c
index 9c311c1..f9ad6c2 100644
--- a/drivers/iio/adc/rockchip_saradc.c
+++ b/drivers/iio/adc/rockchip_saradc.c
@@ -159,6 +159,22 @@
.clk_rate = 50000,
};
+static const struct iio_chan_spec rockchip_rk3399_saradc_iio_channels[] = {
+ ADC_CHANNEL(0, "adc0"),
+ ADC_CHANNEL(1, "adc1"),
+ ADC_CHANNEL(2, "adc2"),
+ ADC_CHANNEL(3, "adc3"),
+ ADC_CHANNEL(4, "adc4"),
+ ADC_CHANNEL(5, "adc5"),
+};
+
+static const struct rockchip_saradc_data rk3399_saradc_data = {
+ .num_bits = 10,
+ .channels = rockchip_rk3399_saradc_iio_channels,
+ .num_channels = ARRAY_SIZE(rockchip_rk3399_saradc_iio_channels),
+ .clk_rate = 1000000,
+};
+
static const struct of_device_id rockchip_saradc_match[] = {
{
.compatible = "rockchip,saradc",
@@ -166,6 +182,9 @@
}, {
.compatible = "rockchip,rk3066-tsadc",
.data = &rk3066_tsadc_data,
+ }, {
+ .compatible = "rockchip,rk3399-saradc",
+ .data = &rk3399_saradc_data,
},
{},
};
diff --git a/drivers/iio/adc/ti-adc081c.c b/drivers/iio/adc/ti-adc081c.c
index ecbc121..9fd032d 100644
--- a/drivers/iio/adc/ti-adc081c.c
+++ b/drivers/iio/adc/ti-adc081c.c
@@ -1,9 +1,21 @@
/*
+ * TI ADC081C/ADC101C/ADC121C 8/10/12-bit ADC driver
+ *
* Copyright (C) 2012 Avionic Design GmbH
+ * Copyright (C) 2016 Intel
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
+ *
+ * Datasheets:
+ * http://www.ti.com/lit/ds/symlink/adc081c021.pdf
+ * http://www.ti.com/lit/ds/symlink/adc101c021.pdf
+ * http://www.ti.com/lit/ds/symlink/adc121c021.pdf
+ *
+ * The devices have a very similar interface and differ mostly in the number of
+ * bits handled. For the 8-bit and 10-bit models the least-significant 4 or 2
+ * bits of value registers are reserved.
*/
#include <linux/err.h>
@@ -12,11 +24,17 @@
#include <linux/of.h>
#include <linux/iio/iio.h>
+#include <linux/iio/buffer.h>
+#include <linux/iio/trigger_consumer.h>
+#include <linux/iio/triggered_buffer.h>
#include <linux/regulator/consumer.h>
struct adc081c {
struct i2c_client *i2c;
struct regulator *ref;
+
+ /* 8, 10 or 12 */
+ int bits;
};
#define REG_CONV_RES 0x00
@@ -34,7 +52,7 @@
if (err < 0)
return err;
- *value = (err >> 4) & 0xff;
+ *value = (err & 0xFFF) >> (12 - adc->bits);
return IIO_VAL_INT;
case IIO_CHAN_INFO_SCALE:
@@ -43,7 +61,7 @@
return err;
*value = err / 1000;
- *shift = 8;
+ *shift = adc->bits;
return IIO_VAL_FRACTIONAL_LOG2;
@@ -54,10 +72,53 @@
return -EINVAL;
}
-static const struct iio_chan_spec adc081c_channel = {
- .type = IIO_VOLTAGE,
- .info_mask_shared_by_type = BIT(IIO_CHAN_INFO_SCALE),
- .info_mask_separate = BIT(IIO_CHAN_INFO_RAW),
+#define ADCxx1C_CHAN(_bits) { \
+ .type = IIO_VOLTAGE, \
+ .info_mask_shared_by_type = BIT(IIO_CHAN_INFO_SCALE), \
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW), \
+ .scan_type = { \
+ .sign = 'u', \
+ .realbits = (_bits), \
+ .storagebits = 16, \
+ .shift = 12 - (_bits), \
+ .endianness = IIO_CPU, \
+ }, \
+}
+
+#define DEFINE_ADCxx1C_CHANNELS(_name, _bits) \
+ static const struct iio_chan_spec _name ## _channels[] = { \
+ ADCxx1C_CHAN((_bits)), \
+ IIO_CHAN_SOFT_TIMESTAMP(1), \
+ }; \
+
+#define ADC081C_NUM_CHANNELS 2
+
+struct adcxx1c_model {
+ const struct iio_chan_spec* channels;
+ int bits;
+};
+
+#define ADCxx1C_MODEL(_name, _bits) \
+ { \
+ .channels = _name ## _channels, \
+ .bits = (_bits), \
+ }
+
+DEFINE_ADCxx1C_CHANNELS(adc081c, 8);
+DEFINE_ADCxx1C_CHANNELS(adc101c, 10);
+DEFINE_ADCxx1C_CHANNELS(adc121c, 12);
+
+/* Model ids are indexes in _models array */
+enum adcxx1c_model_id {
+ ADC081C = 0,
+ ADC101C = 1,
+ ADC121C = 2,
+};
+
+static struct adcxx1c_model adcxx1c_models[] = {
+ ADCxx1C_MODEL(adc081c, 8),
+ ADCxx1C_MODEL(adc101c, 10),
+ ADCxx1C_MODEL(adc121c, 12),
};
static const struct iio_info adc081c_info = {
@@ -65,11 +126,30 @@
.driver_module = THIS_MODULE,
};
+static irqreturn_t adc081c_trigger_handler(int irq, void *p)
+{
+ struct iio_poll_func *pf = p;
+ struct iio_dev *indio_dev = pf->indio_dev;
+ struct adc081c *data = iio_priv(indio_dev);
+ u16 buf[8]; /* 2 bytes data + 6 bytes padding + 8 bytes timestamp */
+ int ret;
+
+ ret = i2c_smbus_read_word_swapped(data->i2c, REG_CONV_RES);
+ if (ret < 0)
+ goto out;
+ buf[0] = ret;
+ iio_push_to_buffers_with_timestamp(indio_dev, buf, iio_get_time_ns());
+out:
+ iio_trigger_notify_done(indio_dev->trig);
+ return IRQ_HANDLED;
+}
+
static int adc081c_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
struct iio_dev *iio;
struct adc081c *adc;
+ struct adcxx1c_model *model = &adcxx1c_models[id->driver_data];
int err;
if (!i2c_check_functionality(client->adapter, I2C_FUNC_SMBUS_WORD_DATA))
@@ -81,6 +161,7 @@
adc = iio_priv(iio);
adc->i2c = client;
+ adc->bits = model->bits;
adc->ref = devm_regulator_get(&client->dev, "vref");
if (IS_ERR(adc->ref))
@@ -95,18 +176,26 @@
iio->modes = INDIO_DIRECT_MODE;
iio->info = &adc081c_info;
- iio->channels = &adc081c_channel;
- iio->num_channels = 1;
+ iio->channels = model->channels;
+ iio->num_channels = ADC081C_NUM_CHANNELS;
+
+ err = iio_triggered_buffer_setup(iio, NULL, adc081c_trigger_handler, NULL);
+ if (err < 0) {
+ dev_err(&client->dev, "iio triggered buffer setup failed\n");
+ goto err_regulator_disable;
+ }
err = iio_device_register(iio);
if (err < 0)
- goto regulator_disable;
+ goto err_buffer_cleanup;
i2c_set_clientdata(client, iio);
return 0;
-regulator_disable:
+err_buffer_cleanup:
+ iio_triggered_buffer_cleanup(iio);
+err_regulator_disable:
regulator_disable(adc->ref);
return err;
@@ -118,13 +207,16 @@
struct adc081c *adc = iio_priv(iio);
iio_device_unregister(iio);
+ iio_triggered_buffer_cleanup(iio);
regulator_disable(adc->ref);
return 0;
}
static const struct i2c_device_id adc081c_id[] = {
- { "adc081c", 0 },
+ { "adc081c", ADC081C },
+ { "adc101c", ADC101C },
+ { "adc121c", ADC121C },
{ }
};
MODULE_DEVICE_TABLE(i2c, adc081c_id);
@@ -132,6 +224,8 @@
#ifdef CONFIG_OF
static const struct of_device_id adc081c_of_match[] = {
{ .compatible = "ti,adc081c" },
+ { .compatible = "ti,adc101c" },
+ { .compatible = "ti,adc121c" },
{ }
};
MODULE_DEVICE_TABLE(of, adc081c_of_match);
@@ -149,5 +243,5 @@
module_i2c_driver(adc081c_driver);
MODULE_AUTHOR("Thierry Reding <thierry.reding@avionic-design.de>");
-MODULE_DESCRIPTION("Texas Instruments ADC081C021/027 driver");
+MODULE_DESCRIPTION("Texas Instruments ADC081C/ADC101C/ADC121C driver");
MODULE_LICENSE("GPL v2");
diff --git a/drivers/iio/adc/vf610_adc.c b/drivers/iio/adc/vf610_adc.c
index b10f629..653bf13 100644
--- a/drivers/iio/adc/vf610_adc.c
+++ b/drivers/iio/adc/vf610_adc.c
@@ -714,19 +714,19 @@
int i;
switch (mask) {
- case IIO_CHAN_INFO_SAMP_FREQ:
- for (i = 0;
- i < ARRAY_SIZE(info->sample_freq_avail);
- i++)
- if (val == info->sample_freq_avail[i]) {
- info->adc_feature.sample_rate = i;
- vf610_adc_sample_set(info);
- return 0;
- }
- break;
+ case IIO_CHAN_INFO_SAMP_FREQ:
+ for (i = 0;
+ i < ARRAY_SIZE(info->sample_freq_avail);
+ i++)
+ if (val == info->sample_freq_avail[i]) {
+ info->adc_feature.sample_rate = i;
+ vf610_adc_sample_set(info);
+ return 0;
+ }
+ break;
- default:
- break;
+ default:
+ break;
}
return -EINVAL;
diff --git a/drivers/iio/common/hid-sensors/hid-sensor-trigger.c b/drivers/iio/common/hid-sensors/hid-sensor-trigger.c
index 5955110..5b41f9d 100644
--- a/drivers/iio/common/hid-sensors/hid-sensor-trigger.c
+++ b/drivers/iio/common/hid-sensors/hid-sensor-trigger.c
@@ -115,7 +115,7 @@
return ret;
}
- return 0;
+ return 0;
#else
atomic_set(&st->user_requested_state, state);
return _hid_sensor_power_state(st, state);
diff --git a/drivers/iio/common/ms_sensors/ms_sensors_i2c.c b/drivers/iio/common/ms_sensors/ms_sensors_i2c.c
index 669dc7c..ecf7721 100644
--- a/drivers/iio/common/ms_sensors/ms_sensors_i2c.c
+++ b/drivers/iio/common/ms_sensors/ms_sensors_i2c.c
@@ -106,7 +106,7 @@
unsigned int delay, u32 *adc)
{
int ret;
- __be32 buf = 0;
+ __be32 buf = 0;
struct i2c_client *client = (struct i2c_client *)cli;
/* Trigger conversion */
diff --git a/drivers/iio/common/st_sensors/st_sensors_buffer.c b/drivers/iio/common/st_sensors/st_sensors_buffer.c
index e18bc67..c558985 100644
--- a/drivers/iio/common/st_sensors/st_sensors_buffer.c
+++ b/drivers/iio/common/st_sensors/st_sensors_buffer.c
@@ -24,81 +24,30 @@
int st_sensors_get_buffer_element(struct iio_dev *indio_dev, u8 *buf)
{
- u8 *addr;
- int i, n = 0, len;
+ int i, len;
+ int total = 0;
struct st_sensor_data *sdata = iio_priv(indio_dev);
unsigned int num_data_channels = sdata->num_data_channels;
- unsigned int byte_for_channel =
- indio_dev->channels[0].scan_type.storagebits >> 3;
-
- addr = kmalloc(num_data_channels, GFP_KERNEL);
- if (!addr) {
- len = -ENOMEM;
- goto st_sensors_get_buffer_element_error;
- }
for (i = 0; i < num_data_channels; i++) {
+ unsigned int bytes_to_read;
+
if (test_bit(i, indio_dev->active_scan_mask)) {
- addr[n] = indio_dev->channels[i].address;
- n++;
+ bytes_to_read = indio_dev->channels[i].scan_type.storagebits >> 3;
+ len = sdata->tf->read_multiple_byte(&sdata->tb,
+ sdata->dev, indio_dev->channels[i].address,
+ bytes_to_read,
+ buf + total, sdata->multiread_bit);
+
+ if (len < bytes_to_read)
+ return -EIO;
+
+ /* Advance the buffer pointer */
+ total += len;
}
}
- switch (n) {
- case 1:
- len = sdata->tf->read_multiple_byte(&sdata->tb, sdata->dev,
- addr[0], byte_for_channel, buf, sdata->multiread_bit);
- break;
- case 2:
- if ((addr[1] - addr[0]) == byte_for_channel) {
- len = sdata->tf->read_multiple_byte(&sdata->tb,
- sdata->dev, addr[0], byte_for_channel * n,
- buf, sdata->multiread_bit);
- } else {
- u8 *rx_array;
- rx_array = kmalloc(byte_for_channel * num_data_channels,
- GFP_KERNEL);
- if (!rx_array) {
- len = -ENOMEM;
- goto st_sensors_free_memory;
- }
- len = sdata->tf->read_multiple_byte(&sdata->tb,
- sdata->dev, addr[0],
- byte_for_channel * num_data_channels,
- rx_array, sdata->multiread_bit);
- if (len < 0) {
- kfree(rx_array);
- goto st_sensors_free_memory;
- }
-
- for (i = 0; i < n * byte_for_channel; i++) {
- if (i < n)
- buf[i] = rx_array[i];
- else
- buf[i] = rx_array[n + i];
- }
- kfree(rx_array);
- len = byte_for_channel * n;
- }
- break;
- case 3:
- len = sdata->tf->read_multiple_byte(&sdata->tb, sdata->dev,
- addr[0], byte_for_channel * num_data_channels,
- buf, sdata->multiread_bit);
- break;
- default:
- len = -EINVAL;
- goto st_sensors_free_memory;
- }
- if (len != byte_for_channel * n) {
- len = -EIO;
- goto st_sensors_free_memory;
- }
-
-st_sensors_free_memory:
- kfree(addr);
-st_sensors_get_buffer_element_error:
- return len;
+ return total;
}
EXPORT_SYMBOL(st_sensors_get_buffer_element);
@@ -109,6 +58,24 @@
struct iio_dev *indio_dev = pf->indio_dev;
struct st_sensor_data *sdata = iio_priv(indio_dev);
+ /* If we have a status register, check if this IRQ came from us */
+ if (sdata->sensor_settings->drdy_irq.addr_stat_drdy) {
+ u8 status;
+
+ len = sdata->tf->read_byte(&sdata->tb, sdata->dev,
+ sdata->sensor_settings->drdy_irq.addr_stat_drdy,
+ &status);
+ if (len < 0)
+ dev_err(sdata->dev, "could not read channel status\n");
+
+ /*
+ * If this was not caused by any channels on this sensor,
+ * return IRQ_NONE
+ */
+ if (!(status & (u8)indio_dev->active_scan_mask[0]))
+ return IRQ_NONE;
+ }
+
len = st_sensors_get_buffer_element(indio_dev, sdata->buffer_data);
if (len < 0)
goto st_sensors_get_buffer_element_error;
diff --git a/drivers/iio/common/st_sensors/st_sensors_core.c b/drivers/iio/common/st_sensors/st_sensors_core.c
index f5a2d44..dffe006 100644
--- a/drivers/iio/common/st_sensors/st_sensors_core.c
+++ b/drivers/iio/common/st_sensors/st_sensors_core.c
@@ -301,6 +301,14 @@
return -EINVAL;
}
+ if (pdata->open_drain) {
+ if (!sdata->sensor_settings->drdy_irq.addr_od)
+ dev_err(&indio_dev->dev,
+ "open drain requested but unsupported.\n");
+ else
+ sdata->int_pin_open_drain = true;
+ }
+
return 0;
}
@@ -321,6 +329,8 @@
else
pdata->drdy_int_pin = defdata ? defdata->drdy_int_pin : 0;
+ pdata->open_drain = of_property_read_bool(np, "drive-open-drain");
+
return pdata;
}
#else
@@ -374,6 +384,16 @@
return err;
}
+ if (sdata->int_pin_open_drain) {
+ dev_info(&indio_dev->dev,
+ "set interrupt line to open drain mode\n");
+ err = st_sensors_write_data_with_mask(indio_dev,
+ sdata->sensor_settings->drdy_irq.addr_od,
+ sdata->sensor_settings->drdy_irq.mask_od, 1);
+ if (err < 0)
+ return err;
+ }
+
err = st_sensors_set_axis_enable(indio_dev, ST_SENSORS_ENABLE_ALL_AXIS);
return err;
diff --git a/drivers/iio/common/st_sensors/st_sensors_trigger.c b/drivers/iio/common/st_sensors/st_sensors_trigger.c
index 6a8c983..da72279 100644
--- a/drivers/iio/common/st_sensors/st_sensors_trigger.c
+++ b/drivers/iio/common/st_sensors/st_sensors_trigger.c
@@ -64,6 +64,19 @@
"rising edge\n", irq_trig);
irq_trig = IRQF_TRIGGER_RISING;
}
+
+ /*
+ * If the interrupt pin is Open Drain, by definition this
+ * means that the interrupt line may be shared with other
+ * peripherals. But to do this we also need to have a status
+ * register and mask to figure out if this sensor was firing
+ * the IRQ or not, so we can tell the interrupt handle that
+ * it was "our" interrupt.
+ */
+ if (sdata->int_pin_open_drain &&
+ sdata->sensor_settings->drdy_irq.addr_stat_drdy)
+ irq_trig |= IRQF_SHARED;
+
err = request_threaded_irq(irq,
iio_trigger_generic_data_rdy_poll,
NULL,
diff --git a/drivers/iio/dac/Kconfig b/drivers/iio/dac/Kconfig
index a995139..e63b957 100644
--- a/drivers/iio/dac/Kconfig
+++ b/drivers/iio/dac/Kconfig
@@ -74,6 +74,33 @@
To compile this driver as a module, choose M here: the
module will be called ad5449.
+config AD5592R_BASE
+ tristate
+
+config AD5592R
+ tristate "Analog Devices AD5592R ADC/DAC driver"
+ depends on SPI_MASTER
+ select GPIOLIB
+ select AD5592R_BASE
+ help
+ Say yes here to build support for Analog Devices AD5592R
+ Digital to Analog / Analog to Digital Converter.
+
+ To compile this driver as a module, choose M here: the
+ module will be called ad5592r.
+
+config AD5593R
+ tristate "Analog Devices AD5593R ADC/DAC driver"
+ depends on I2C
+ select GPIOLIB
+ select AD5592R_BASE
+ help
+ Say yes here to build support for Analog Devices AD5593R
+ Digital to Analog / Analog to Digital Converter.
+
+ To compile this driver as a module, choose M here: the
+ module will be called ad5593r.
+
config AD5504
tristate "Analog Devices AD5504/AD5501 DAC SPI driver"
depends on SPI
@@ -154,6 +181,16 @@
To compile this driver as module choose M here: the module will be called
ad7303.
+config LPC18XX_DAC
+ tristate "NXP LPC18xx DAC driver"
+ depends on ARCH_LPC18XX || COMPILE_TEST
+ depends on OF && HAS_IOMEM
+ help
+ Say yes here to build support for NXP LPC18XX DAC.
+
+ To compile this driver as a module, choose M here: the module will be
+ called lpc18xx_dac.
+
config M62332
tristate "Mitsubishi M62332 DAC driver"
depends on I2C
@@ -210,7 +247,7 @@
config STX104
tristate "Apex Embedded Systems STX104 DAC driver"
- depends on ISA
+ depends on X86 && ISA
help
Say yes here to build support for the 2-channel DAC on the Apex
Embedded Systems STX104 integrated analog PC/104 card. The base port
diff --git a/drivers/iio/dac/Makefile b/drivers/iio/dac/Makefile
index 67b4842..8b78d5c 100644
--- a/drivers/iio/dac/Makefile
+++ b/drivers/iio/dac/Makefile
@@ -11,12 +11,16 @@
obj-$(CONFIG_AD5504) += ad5504.o
obj-$(CONFIG_AD5446) += ad5446.o
obj-$(CONFIG_AD5449) += ad5449.o
+obj-$(CONFIG_AD5592R_BASE) += ad5592r-base.o
+obj-$(CONFIG_AD5592R) += ad5592r.o
+obj-$(CONFIG_AD5593R) += ad5593r.o
obj-$(CONFIG_AD5755) += ad5755.o
obj-$(CONFIG_AD5761) += ad5761.o
obj-$(CONFIG_AD5764) += ad5764.o
obj-$(CONFIG_AD5791) += ad5791.o
obj-$(CONFIG_AD5686) += ad5686.o
obj-$(CONFIG_AD7303) += ad7303.o
+obj-$(CONFIG_LPC18XX_DAC) += lpc18xx_dac.o
obj-$(CONFIG_M62332) += m62332.o
obj-$(CONFIG_MAX517) += max517.o
obj-$(CONFIG_MAX5821) += max5821.o
diff --git a/drivers/iio/dac/ad5592r-base.c b/drivers/iio/dac/ad5592r-base.c
new file mode 100644
index 0000000..948f600
--- /dev/null
+++ b/drivers/iio/dac/ad5592r-base.c
@@ -0,0 +1,691 @@
+/*
+ * AD5592R Digital <-> Analog converters driver
+ *
+ * Copyright 2014-2016 Analog Devices Inc.
+ * Author: Paul Cercueil <paul.cercueil@analog.com>
+ *
+ * Licensed under the GPL-2.
+ */
+
+#include <linux/bitops.h>
+#include <linux/delay.h>
+#include <linux/iio/iio.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/of.h>
+#include <linux/regulator/consumer.h>
+#include <linux/gpio/consumer.h>
+#include <linux/gpio/driver.h>
+#include <linux/gpio.h>
+#include <linux/property.h>
+
+#include <dt-bindings/iio/adi,ad5592r.h>
+
+#include "ad5592r-base.h"
+
+static int ad5592r_gpio_get(struct gpio_chip *chip, unsigned offset)
+{
+ struct ad5592r_state *st = gpiochip_get_data(chip);
+ int ret = 0;
+ u8 val;
+
+ mutex_lock(&st->gpio_lock);
+
+ if (st->gpio_out & BIT(offset))
+ val = st->gpio_val;
+ else
+ ret = st->ops->gpio_read(st, &val);
+
+ mutex_unlock(&st->gpio_lock);
+
+ if (ret < 0)
+ return ret;
+
+ return !!(val & BIT(offset));
+}
+
+static void ad5592r_gpio_set(struct gpio_chip *chip, unsigned offset, int value)
+{
+ struct ad5592r_state *st = gpiochip_get_data(chip);
+
+ mutex_lock(&st->gpio_lock);
+
+ if (value)
+ st->gpio_val |= BIT(offset);
+ else
+ st->gpio_val &= ~BIT(offset);
+
+ st->ops->reg_write(st, AD5592R_REG_GPIO_SET, st->gpio_val);
+
+ mutex_unlock(&st->gpio_lock);
+}
+
+static int ad5592r_gpio_direction_input(struct gpio_chip *chip, unsigned offset)
+{
+ struct ad5592r_state *st = gpiochip_get_data(chip);
+ int ret;
+
+ mutex_lock(&st->gpio_lock);
+
+ st->gpio_out &= ~BIT(offset);
+ st->gpio_in |= BIT(offset);
+
+ ret = st->ops->reg_write(st, AD5592R_REG_GPIO_OUT_EN, st->gpio_out);
+ if (ret < 0)
+ goto err_unlock;
+
+ ret = st->ops->reg_write(st, AD5592R_REG_GPIO_IN_EN, st->gpio_in);
+
+err_unlock:
+ mutex_unlock(&st->gpio_lock);
+
+ return ret;
+}
+
+static int ad5592r_gpio_direction_output(struct gpio_chip *chip,
+ unsigned offset, int value)
+{
+ struct ad5592r_state *st = gpiochip_get_data(chip);
+ int ret;
+
+ mutex_lock(&st->gpio_lock);
+
+ if (value)
+ st->gpio_val |= BIT(offset);
+ else
+ st->gpio_val &= ~BIT(offset);
+
+ st->gpio_in &= ~BIT(offset);
+ st->gpio_out |= BIT(offset);
+
+ ret = st->ops->reg_write(st, AD5592R_REG_GPIO_SET, st->gpio_val);
+ if (ret < 0)
+ goto err_unlock;
+
+ ret = st->ops->reg_write(st, AD5592R_REG_GPIO_OUT_EN, st->gpio_out);
+ if (ret < 0)
+ goto err_unlock;
+
+ ret = st->ops->reg_write(st, AD5592R_REG_GPIO_IN_EN, st->gpio_in);
+
+err_unlock:
+ mutex_unlock(&st->gpio_lock);
+
+ return ret;
+}
+
+static int ad5592r_gpio_request(struct gpio_chip *chip, unsigned offset)
+{
+ struct ad5592r_state *st = gpiochip_get_data(chip);
+
+ if (!(st->gpio_map & BIT(offset))) {
+ dev_err(st->dev, "GPIO %d is reserved by alternate function\n",
+ offset);
+ return -ENODEV;
+ }
+
+ return 0;
+}
+
+static int ad5592r_gpio_init(struct ad5592r_state *st)
+{
+ if (!st->gpio_map)
+ return 0;
+
+ st->gpiochip.label = dev_name(st->dev);
+ st->gpiochip.base = -1;
+ st->gpiochip.ngpio = 8;
+ st->gpiochip.parent = st->dev;
+ st->gpiochip.can_sleep = true;
+ st->gpiochip.direction_input = ad5592r_gpio_direction_input;
+ st->gpiochip.direction_output = ad5592r_gpio_direction_output;
+ st->gpiochip.get = ad5592r_gpio_get;
+ st->gpiochip.set = ad5592r_gpio_set;
+ st->gpiochip.request = ad5592r_gpio_request;
+ st->gpiochip.owner = THIS_MODULE;
+
+ mutex_init(&st->gpio_lock);
+
+ return gpiochip_add_data(&st->gpiochip, st);
+}
+
+static void ad5592r_gpio_cleanup(struct ad5592r_state *st)
+{
+ if (st->gpio_map)
+ gpiochip_remove(&st->gpiochip);
+}
+
+static int ad5592r_reset(struct ad5592r_state *st)
+{
+ struct gpio_desc *gpio;
+ struct iio_dev *iio_dev = iio_priv_to_dev(st);
+
+ gpio = devm_gpiod_get_optional(st->dev, "reset", GPIOD_OUT_LOW);
+ if (IS_ERR(gpio))
+ return PTR_ERR(gpio);
+
+ if (gpio) {
+ udelay(1);
+ gpiod_set_value(gpio, 1);
+ } else {
+ mutex_lock(&iio_dev->mlock);
+ /* Writing this magic value resets the device */
+ st->ops->reg_write(st, AD5592R_REG_RESET, 0xdac);
+ mutex_unlock(&iio_dev->mlock);
+ }
+
+ udelay(250);
+
+ return 0;
+}
+
+static int ad5592r_get_vref(struct ad5592r_state *st)
+{
+ int ret;
+
+ if (st->reg) {
+ ret = regulator_get_voltage(st->reg);
+ if (ret < 0)
+ return ret;
+
+ return ret / 1000;
+ } else {
+ return 2500;
+ }
+}
+
+static int ad5592r_set_channel_modes(struct ad5592r_state *st)
+{
+ const struct ad5592r_rw_ops *ops = st->ops;
+ int ret;
+ unsigned i;
+ struct iio_dev *iio_dev = iio_priv_to_dev(st);
+ u8 pulldown = 0, tristate = 0, dac = 0, adc = 0;
+ u16 read_back;
+
+ for (i = 0; i < st->num_channels; i++) {
+ switch (st->channel_modes[i]) {
+ case CH_MODE_DAC:
+ dac |= BIT(i);
+ break;
+
+ case CH_MODE_ADC:
+ adc |= BIT(i);
+ break;
+
+ case CH_MODE_DAC_AND_ADC:
+ dac |= BIT(i);
+ adc |= BIT(i);
+ break;
+
+ case CH_MODE_GPIO:
+ st->gpio_map |= BIT(i);
+ st->gpio_in |= BIT(i); /* Default to input */
+ break;
+
+ case CH_MODE_UNUSED:
+ /* fall-through */
+ default:
+ switch (st->channel_offstate[i]) {
+ case CH_OFFSTATE_OUT_TRISTATE:
+ tristate |= BIT(i);
+ break;
+
+ case CH_OFFSTATE_OUT_LOW:
+ st->gpio_out |= BIT(i);
+ break;
+
+ case CH_OFFSTATE_OUT_HIGH:
+ st->gpio_out |= BIT(i);
+ st->gpio_val |= BIT(i);
+ break;
+
+ case CH_OFFSTATE_PULLDOWN:
+ /* fall-through */
+ default:
+ pulldown |= BIT(i);
+ break;
+ }
+ }
+ }
+
+ mutex_lock(&iio_dev->mlock);
+
+ /* Pull down unused pins to GND */
+ ret = ops->reg_write(st, AD5592R_REG_PULLDOWN, pulldown);
+ if (ret)
+ goto err_unlock;
+
+ ret = ops->reg_write(st, AD5592R_REG_TRISTATE, tristate);
+ if (ret)
+ goto err_unlock;
+
+ /* Configure pins that we use */
+ ret = ops->reg_write(st, AD5592R_REG_DAC_EN, dac);
+ if (ret)
+ goto err_unlock;
+
+ ret = ops->reg_write(st, AD5592R_REG_ADC_EN, adc);
+ if (ret)
+ goto err_unlock;
+
+ ret = ops->reg_write(st, AD5592R_REG_GPIO_SET, st->gpio_val);
+ if (ret)
+ goto err_unlock;
+
+ ret = ops->reg_write(st, AD5592R_REG_GPIO_OUT_EN, st->gpio_out);
+ if (ret)
+ goto err_unlock;
+
+ ret = ops->reg_write(st, AD5592R_REG_GPIO_IN_EN, st->gpio_in);
+ if (ret)
+ goto err_unlock;
+
+ /* Verify that we can read back at least one register */
+ ret = ops->reg_read(st, AD5592R_REG_ADC_EN, &read_back);
+ if (!ret && (read_back & 0xff) != adc)
+ ret = -EIO;
+
+err_unlock:
+ mutex_unlock(&iio_dev->mlock);
+ return ret;
+}
+
+static int ad5592r_reset_channel_modes(struct ad5592r_state *st)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(st->channel_modes); i++)
+ st->channel_modes[i] = CH_MODE_UNUSED;
+
+ return ad5592r_set_channel_modes(st);
+}
+
+static int ad5592r_write_raw(struct iio_dev *iio_dev,
+ struct iio_chan_spec const *chan, int val, int val2, long mask)
+{
+ struct ad5592r_state *st = iio_priv(iio_dev);
+ int ret;
+
+ switch (mask) {
+ case IIO_CHAN_INFO_RAW:
+
+ if (val >= (1 << chan->scan_type.realbits) || val < 0)
+ return -EINVAL;
+
+ if (!chan->output)
+ return -EINVAL;
+
+ mutex_lock(&iio_dev->mlock);
+ ret = st->ops->write_dac(st, chan->channel, val);
+ if (!ret)
+ st->cached_dac[chan->channel] = val;
+ mutex_unlock(&iio_dev->mlock);
+ return ret;
+ case IIO_CHAN_INFO_SCALE:
+ if (chan->type == IIO_VOLTAGE) {
+ bool gain;
+
+ if (val == st->scale_avail[0][0] &&
+ val2 == st->scale_avail[0][1])
+ gain = false;
+ else if (val == st->scale_avail[1][0] &&
+ val2 == st->scale_avail[1][1])
+ gain = true;
+ else
+ return -EINVAL;
+
+ mutex_lock(&iio_dev->mlock);
+
+ ret = st->ops->reg_read(st, AD5592R_REG_CTRL,
+ &st->cached_gp_ctrl);
+ if (ret < 0) {
+ mutex_unlock(&iio_dev->mlock);
+ return ret;
+ }
+
+ if (chan->output) {
+ if (gain)
+ st->cached_gp_ctrl |=
+ AD5592R_REG_CTRL_DAC_RANGE;
+ else
+ st->cached_gp_ctrl &=
+ ~AD5592R_REG_CTRL_DAC_RANGE;
+ } else {
+ if (gain)
+ st->cached_gp_ctrl |=
+ AD5592R_REG_CTRL_ADC_RANGE;
+ else
+ st->cached_gp_ctrl &=
+ ~AD5592R_REG_CTRL_ADC_RANGE;
+ }
+
+ ret = st->ops->reg_write(st, AD5592R_REG_CTRL,
+ st->cached_gp_ctrl);
+ mutex_unlock(&iio_dev->mlock);
+
+ return ret;
+ }
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int ad5592r_read_raw(struct iio_dev *iio_dev,
+ struct iio_chan_spec const *chan,
+ int *val, int *val2, long m)
+{
+ struct ad5592r_state *st = iio_priv(iio_dev);
+ u16 read_val;
+ int ret;
+
+ switch (m) {
+ case IIO_CHAN_INFO_RAW:
+ mutex_lock(&iio_dev->mlock);
+
+ if (!chan->output) {
+ ret = st->ops->read_adc(st, chan->channel, &read_val);
+ if (ret)
+ goto unlock;
+
+ if ((read_val >> 12 & 0x7) != (chan->channel & 0x7)) {
+ dev_err(st->dev, "Error while reading channel %u\n",
+ chan->channel);
+ ret = -EIO;
+ goto unlock;
+ }
+
+ read_val &= GENMASK(11, 0);
+
+ } else {
+ read_val = st->cached_dac[chan->channel];
+ }
+
+ dev_dbg(st->dev, "Channel %u read: 0x%04hX\n",
+ chan->channel, read_val);
+
+ *val = (int) read_val;
+ ret = IIO_VAL_INT;
+ break;
+ case IIO_CHAN_INFO_SCALE:
+ *val = ad5592r_get_vref(st);
+
+ if (chan->type == IIO_TEMP) {
+ s64 tmp = *val * (3767897513LL / 25LL);
+ *val = div_s64_rem(tmp, 1000000000LL, val2);
+
+ ret = IIO_VAL_INT_PLUS_MICRO;
+ } else {
+ int mult;
+
+ mutex_lock(&iio_dev->mlock);
+
+ if (chan->output)
+ mult = !!(st->cached_gp_ctrl &
+ AD5592R_REG_CTRL_DAC_RANGE);
+ else
+ mult = !!(st->cached_gp_ctrl &
+ AD5592R_REG_CTRL_ADC_RANGE);
+
+ *val *= ++mult;
+
+ *val2 = chan->scan_type.realbits;
+ ret = IIO_VAL_FRACTIONAL_LOG2;
+ }
+ break;
+ case IIO_CHAN_INFO_OFFSET:
+ ret = ad5592r_get_vref(st);
+
+ mutex_lock(&iio_dev->mlock);
+
+ if (st->cached_gp_ctrl & AD5592R_REG_CTRL_ADC_RANGE)
+ *val = (-34365 * 25) / ret;
+ else
+ *val = (-75365 * 25) / ret;
+ ret = IIO_VAL_INT;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+
+unlock:
+ mutex_unlock(&iio_dev->mlock);
+ return ret;
+}
+
+static int ad5592r_write_raw_get_fmt(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan, long mask)
+{
+ switch (mask) {
+ case IIO_CHAN_INFO_SCALE:
+ return IIO_VAL_INT_PLUS_NANO;
+
+ default:
+ return IIO_VAL_INT_PLUS_MICRO;
+ }
+
+ return -EINVAL;
+}
+
+static const struct iio_info ad5592r_info = {
+ .read_raw = ad5592r_read_raw,
+ .write_raw = ad5592r_write_raw,
+ .write_raw_get_fmt = ad5592r_write_raw_get_fmt,
+ .driver_module = THIS_MODULE,
+};
+
+static ssize_t ad5592r_show_scale_available(struct iio_dev *iio_dev,
+ uintptr_t private,
+ const struct iio_chan_spec *chan,
+ char *buf)
+{
+ struct ad5592r_state *st = iio_priv(iio_dev);
+
+ return sprintf(buf, "%d.%09u %d.%09u\n",
+ st->scale_avail[0][0], st->scale_avail[0][1],
+ st->scale_avail[1][0], st->scale_avail[1][1]);
+}
+
+static struct iio_chan_spec_ext_info ad5592r_ext_info[] = {
+ {
+ .name = "scale_available",
+ .read = ad5592r_show_scale_available,
+ .shared = true,
+ },
+ {},
+};
+
+static void ad5592r_setup_channel(struct iio_dev *iio_dev,
+ struct iio_chan_spec *chan, bool output, unsigned id)
+{
+ chan->type = IIO_VOLTAGE;
+ chan->indexed = 1;
+ chan->output = output;
+ chan->channel = id;
+ chan->info_mask_separate = BIT(IIO_CHAN_INFO_RAW);
+ chan->info_mask_shared_by_type = BIT(IIO_CHAN_INFO_SCALE);
+ chan->scan_type.sign = 'u';
+ chan->scan_type.realbits = 12;
+ chan->scan_type.storagebits = 16;
+ chan->ext_info = ad5592r_ext_info;
+}
+
+static int ad5592r_alloc_channels(struct ad5592r_state *st)
+{
+ unsigned i, curr_channel = 0,
+ num_channels = st->num_channels;
+ struct iio_dev *iio_dev = iio_priv_to_dev(st);
+ struct iio_chan_spec *channels;
+ struct fwnode_handle *child;
+ u32 reg, tmp;
+ int ret;
+
+ device_for_each_child_node(st->dev, child) {
+ ret = fwnode_property_read_u32(child, "reg", ®);
+ if (ret || reg > ARRAY_SIZE(st->channel_modes))
+ continue;
+
+ ret = fwnode_property_read_u32(child, "adi,mode", &tmp);
+ if (!ret)
+ st->channel_modes[reg] = tmp;
+
+ fwnode_property_read_u32(child, "adi,off-state", &tmp);
+ if (!ret)
+ st->channel_offstate[reg] = tmp;
+ }
+
+ channels = devm_kzalloc(st->dev,
+ (1 + 2 * num_channels) * sizeof(*channels), GFP_KERNEL);
+ if (!channels)
+ return -ENOMEM;
+
+ for (i = 0; i < num_channels; i++) {
+ switch (st->channel_modes[i]) {
+ case CH_MODE_DAC:
+ ad5592r_setup_channel(iio_dev, &channels[curr_channel],
+ true, i);
+ curr_channel++;
+ break;
+
+ case CH_MODE_ADC:
+ ad5592r_setup_channel(iio_dev, &channels[curr_channel],
+ false, i);
+ curr_channel++;
+ break;
+
+ case CH_MODE_DAC_AND_ADC:
+ ad5592r_setup_channel(iio_dev, &channels[curr_channel],
+ true, i);
+ curr_channel++;
+ ad5592r_setup_channel(iio_dev, &channels[curr_channel],
+ false, i);
+ curr_channel++;
+ break;
+
+ default:
+ continue;
+ }
+ }
+
+ channels[curr_channel].type = IIO_TEMP;
+ channels[curr_channel].channel = 8;
+ channels[curr_channel].info_mask_separate = BIT(IIO_CHAN_INFO_RAW) |
+ BIT(IIO_CHAN_INFO_SCALE) |
+ BIT(IIO_CHAN_INFO_OFFSET);
+ curr_channel++;
+
+ iio_dev->num_channels = curr_channel;
+ iio_dev->channels = channels;
+
+ return 0;
+}
+
+static void ad5592r_init_scales(struct ad5592r_state *st, int vref_mV)
+{
+ s64 tmp = (s64)vref_mV * 1000000000LL >> 12;
+
+ st->scale_avail[0][0] =
+ div_s64_rem(tmp, 1000000000LL, &st->scale_avail[0][1]);
+ st->scale_avail[1][0] =
+ div_s64_rem(tmp * 2, 1000000000LL, &st->scale_avail[1][1]);
+}
+
+int ad5592r_probe(struct device *dev, const char *name,
+ const struct ad5592r_rw_ops *ops)
+{
+ struct iio_dev *iio_dev;
+ struct ad5592r_state *st;
+ int ret;
+
+ iio_dev = devm_iio_device_alloc(dev, sizeof(*st));
+ if (!iio_dev)
+ return -ENOMEM;
+
+ st = iio_priv(iio_dev);
+ st->dev = dev;
+ st->ops = ops;
+ st->num_channels = 8;
+ dev_set_drvdata(dev, iio_dev);
+
+ st->reg = devm_regulator_get_optional(dev, "vref");
+ if (IS_ERR(st->reg)) {
+ if ((PTR_ERR(st->reg) != -ENODEV) && dev->of_node)
+ return PTR_ERR(st->reg);
+
+ st->reg = NULL;
+ } else {
+ ret = regulator_enable(st->reg);
+ if (ret)
+ return ret;
+ }
+
+ iio_dev->dev.parent = dev;
+ iio_dev->name = name;
+ iio_dev->info = &ad5592r_info;
+ iio_dev->modes = INDIO_DIRECT_MODE;
+
+ ad5592r_init_scales(st, ad5592r_get_vref(st));
+
+ ret = ad5592r_reset(st);
+ if (ret)
+ goto error_disable_reg;
+
+ ret = ops->reg_write(st, AD5592R_REG_PD,
+ (st->reg == NULL) ? AD5592R_REG_PD_EN_REF : 0);
+ if (ret)
+ goto error_disable_reg;
+
+ ret = ad5592r_alloc_channels(st);
+ if (ret)
+ goto error_disable_reg;
+
+ ret = ad5592r_set_channel_modes(st);
+ if (ret)
+ goto error_reset_ch_modes;
+
+ ret = iio_device_register(iio_dev);
+ if (ret)
+ goto error_reset_ch_modes;
+
+ ret = ad5592r_gpio_init(st);
+ if (ret)
+ goto error_dev_unregister;
+
+ return 0;
+
+error_dev_unregister:
+ iio_device_unregister(iio_dev);
+
+error_reset_ch_modes:
+ ad5592r_reset_channel_modes(st);
+
+error_disable_reg:
+ if (st->reg)
+ regulator_disable(st->reg);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(ad5592r_probe);
+
+int ad5592r_remove(struct device *dev)
+{
+ struct iio_dev *iio_dev = dev_get_drvdata(dev);
+ struct ad5592r_state *st = iio_priv(iio_dev);
+
+ iio_device_unregister(iio_dev);
+ ad5592r_reset_channel_modes(st);
+ ad5592r_gpio_cleanup(st);
+
+ if (st->reg)
+ regulator_disable(st->reg);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(ad5592r_remove);
+
+MODULE_AUTHOR("Paul Cercueil <paul.cercueil@analog.com>");
+MODULE_DESCRIPTION("Analog Devices AD5592R multi-channel converters");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/iio/dac/ad5592r-base.h b/drivers/iio/dac/ad5592r-base.h
new file mode 100644
index 0000000..841457e
--- /dev/null
+++ b/drivers/iio/dac/ad5592r-base.h
@@ -0,0 +1,76 @@
+/*
+ * AD5592R / AD5593R Digital <-> Analog converters driver
+ *
+ * Copyright 2015-2016 Analog Devices Inc.
+ * Author: Paul Cercueil <paul.cercueil@analog.com>
+ *
+ * Licensed under the GPL-2.
+ */
+
+#ifndef __DRIVERS_IIO_DAC_AD5592R_BASE_H__
+#define __DRIVERS_IIO_DAC_AD5592R_BASE_H__
+
+#include <linux/types.h>
+#include <linux/cache.h>
+#include <linux/mutex.h>
+#include <linux/gpio/driver.h>
+
+struct device;
+struct ad5592r_state;
+
+enum ad5592r_registers {
+ AD5592R_REG_NOOP = 0x0,
+ AD5592R_REG_DAC_READBACK = 0x1,
+ AD5592R_REG_ADC_SEQ = 0x2,
+ AD5592R_REG_CTRL = 0x3,
+ AD5592R_REG_ADC_EN = 0x4,
+ AD5592R_REG_DAC_EN = 0x5,
+ AD5592R_REG_PULLDOWN = 0x6,
+ AD5592R_REG_LDAC = 0x7,
+ AD5592R_REG_GPIO_OUT_EN = 0x8,
+ AD5592R_REG_GPIO_SET = 0x9,
+ AD5592R_REG_GPIO_IN_EN = 0xA,
+ AD5592R_REG_PD = 0xB,
+ AD5592R_REG_OPEN_DRAIN = 0xC,
+ AD5592R_REG_TRISTATE = 0xD,
+ AD5592R_REG_RESET = 0xF,
+};
+
+#define AD5592R_REG_PD_EN_REF BIT(9)
+#define AD5592R_REG_CTRL_ADC_RANGE BIT(5)
+#define AD5592R_REG_CTRL_DAC_RANGE BIT(4)
+
+struct ad5592r_rw_ops {
+ int (*write_dac)(struct ad5592r_state *st, unsigned chan, u16 value);
+ int (*read_adc)(struct ad5592r_state *st, unsigned chan, u16 *value);
+ int (*reg_write)(struct ad5592r_state *st, u8 reg, u16 value);
+ int (*reg_read)(struct ad5592r_state *st, u8 reg, u16 *value);
+ int (*gpio_read)(struct ad5592r_state *st, u8 *value);
+};
+
+struct ad5592r_state {
+ struct device *dev;
+ struct regulator *reg;
+ struct gpio_chip gpiochip;
+ struct mutex gpio_lock; /* Protect cached gpio_out, gpio_val, etc. */
+ unsigned int num_channels;
+ const struct ad5592r_rw_ops *ops;
+ int scale_avail[2][2];
+ u16 cached_dac[8];
+ u16 cached_gp_ctrl;
+ u8 channel_modes[8];
+ u8 channel_offstate[8];
+ u8 gpio_map;
+ u8 gpio_out;
+ u8 gpio_in;
+ u8 gpio_val;
+
+ __be16 spi_msg ____cacheline_aligned;
+ __be16 spi_msg_nop;
+};
+
+int ad5592r_probe(struct device *dev, const char *name,
+ const struct ad5592r_rw_ops *ops);
+int ad5592r_remove(struct device *dev);
+
+#endif /* __DRIVERS_IIO_DAC_AD5592R_BASE_H__ */
diff --git a/drivers/iio/dac/ad5592r.c b/drivers/iio/dac/ad5592r.c
new file mode 100644
index 0000000..0b235a2
--- /dev/null
+++ b/drivers/iio/dac/ad5592r.c
@@ -0,0 +1,164 @@
+/*
+ * AD5592R Digital <-> Analog converters driver
+ *
+ * Copyright 2015-2016 Analog Devices Inc.
+ * Author: Paul Cercueil <paul.cercueil@analog.com>
+ *
+ * Licensed under the GPL-2.
+ */
+
+#include "ad5592r-base.h"
+
+#include <linux/bitops.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/spi/spi.h>
+
+#define AD5592R_GPIO_READBACK_EN BIT(10)
+#define AD5592R_LDAC_READBACK_EN BIT(6)
+
+static int ad5592r_spi_wnop_r16(struct ad5592r_state *st, u16 *buf)
+{
+ struct spi_device *spi = container_of(st->dev, struct spi_device, dev);
+ struct spi_transfer t = {
+ .tx_buf = &st->spi_msg_nop,
+ .rx_buf = buf,
+ .len = 2
+ };
+
+ st->spi_msg_nop = 0; /* NOP */
+
+ return spi_sync_transfer(spi, &t, 1);
+}
+
+static int ad5592r_write_dac(struct ad5592r_state *st, unsigned chan, u16 value)
+{
+ struct spi_device *spi = container_of(st->dev, struct spi_device, dev);
+
+ st->spi_msg = cpu_to_be16(BIT(15) | (chan << 12) | value);
+
+ return spi_write(spi, &st->spi_msg, sizeof(st->spi_msg));
+}
+
+static int ad5592r_read_adc(struct ad5592r_state *st, unsigned chan, u16 *value)
+{
+ struct spi_device *spi = container_of(st->dev, struct spi_device, dev);
+ int ret;
+
+ st->spi_msg = cpu_to_be16((AD5592R_REG_ADC_SEQ << 11) | BIT(chan));
+
+ ret = spi_write(spi, &st->spi_msg, sizeof(st->spi_msg));
+ if (ret)
+ return ret;
+
+ /*
+ * Invalid data:
+ * See Figure 40. Single-Channel ADC Conversion Sequence
+ */
+ ret = ad5592r_spi_wnop_r16(st, &st->spi_msg);
+ if (ret)
+ return ret;
+
+ ret = ad5592r_spi_wnop_r16(st, &st->spi_msg);
+ if (ret)
+ return ret;
+
+ *value = be16_to_cpu(st->spi_msg);
+
+ return 0;
+}
+
+static int ad5592r_reg_write(struct ad5592r_state *st, u8 reg, u16 value)
+{
+ struct spi_device *spi = container_of(st->dev, struct spi_device, dev);
+
+ st->spi_msg = cpu_to_be16((reg << 11) | value);
+
+ return spi_write(spi, &st->spi_msg, sizeof(st->spi_msg));
+}
+
+static int ad5592r_reg_read(struct ad5592r_state *st, u8 reg, u16 *value)
+{
+ struct spi_device *spi = container_of(st->dev, struct spi_device, dev);
+ int ret;
+
+ st->spi_msg = cpu_to_be16((AD5592R_REG_LDAC << 11) |
+ AD5592R_LDAC_READBACK_EN | (reg << 2));
+
+ ret = spi_write(spi, &st->spi_msg, sizeof(st->spi_msg));
+ if (ret)
+ return ret;
+
+ ret = ad5592r_spi_wnop_r16(st, &st->spi_msg);
+ if (ret)
+ return ret;
+
+ *value = be16_to_cpu(st->spi_msg);
+
+ return 0;
+}
+
+static int ad5593r_gpio_read(struct ad5592r_state *st, u8 *value)
+{
+ int ret;
+
+ ret = ad5592r_reg_write(st, AD5592R_REG_GPIO_IN_EN,
+ AD5592R_GPIO_READBACK_EN | st->gpio_in);
+ if (ret)
+ return ret;
+
+ ret = ad5592r_spi_wnop_r16(st, &st->spi_msg);
+ if (ret)
+ return ret;
+
+ *value = (u8) be16_to_cpu(st->spi_msg);
+
+ return 0;
+}
+
+static const struct ad5592r_rw_ops ad5592r_rw_ops = {
+ .write_dac = ad5592r_write_dac,
+ .read_adc = ad5592r_read_adc,
+ .reg_write = ad5592r_reg_write,
+ .reg_read = ad5592r_reg_read,
+ .gpio_read = ad5593r_gpio_read,
+};
+
+static int ad5592r_spi_probe(struct spi_device *spi)
+{
+ const struct spi_device_id *id = spi_get_device_id(spi);
+
+ return ad5592r_probe(&spi->dev, id->name, &ad5592r_rw_ops);
+}
+
+static int ad5592r_spi_remove(struct spi_device *spi)
+{
+ return ad5592r_remove(&spi->dev);
+}
+
+static const struct spi_device_id ad5592r_spi_ids[] = {
+ { .name = "ad5592r", },
+ {}
+};
+MODULE_DEVICE_TABLE(spi, ad5592r_spi_ids);
+
+static const struct of_device_id ad5592r_of_match[] = {
+ { .compatible = "adi,ad5592r", },
+ {},
+};
+MODULE_DEVICE_TABLE(of, ad5592r_of_match);
+
+static struct spi_driver ad5592r_spi_driver = {
+ .driver = {
+ .name = "ad5592r",
+ .of_match_table = of_match_ptr(ad5592r_of_match),
+ },
+ .probe = ad5592r_spi_probe,
+ .remove = ad5592r_spi_remove,
+ .id_table = ad5592r_spi_ids,
+};
+module_spi_driver(ad5592r_spi_driver);
+
+MODULE_AUTHOR("Paul Cercueil <paul.cercueil@analog.com>");
+MODULE_DESCRIPTION("Analog Devices AD5592R multi-channel converters");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/iio/dac/ad5593r.c b/drivers/iio/dac/ad5593r.c
new file mode 100644
index 0000000..dca158a
--- /dev/null
+++ b/drivers/iio/dac/ad5593r.c
@@ -0,0 +1,131 @@
+/*
+ * AD5593R Digital <-> Analog converters driver
+ *
+ * Copyright 2015-2016 Analog Devices Inc.
+ * Author: Paul Cercueil <paul.cercueil@analog.com>
+ *
+ * Licensed under the GPL-2.
+ */
+
+#include "ad5592r-base.h"
+
+#include <linux/bitops.h>
+#include <linux/i2c.h>
+#include <linux/module.h>
+#include <linux/of.h>
+
+#define AD5593R_MODE_CONF (0 << 4)
+#define AD5593R_MODE_DAC_WRITE (1 << 4)
+#define AD5593R_MODE_ADC_READBACK (4 << 4)
+#define AD5593R_MODE_DAC_READBACK (5 << 4)
+#define AD5593R_MODE_GPIO_READBACK (6 << 4)
+#define AD5593R_MODE_REG_READBACK (7 << 4)
+
+static int ad5593r_write_dac(struct ad5592r_state *st, unsigned chan, u16 value)
+{
+ struct i2c_client *i2c = to_i2c_client(st->dev);
+
+ return i2c_smbus_write_word_swapped(i2c,
+ AD5593R_MODE_DAC_WRITE | chan, value);
+}
+
+static int ad5593r_read_adc(struct ad5592r_state *st, unsigned chan, u16 *value)
+{
+ struct i2c_client *i2c = to_i2c_client(st->dev);
+ s32 val;
+
+ val = i2c_smbus_write_word_swapped(i2c,
+ AD5593R_MODE_CONF | AD5592R_REG_ADC_SEQ, BIT(chan));
+ if (val < 0)
+ return (int) val;
+
+ val = i2c_smbus_read_word_swapped(i2c, AD5593R_MODE_ADC_READBACK);
+ if (val < 0)
+ return (int) val;
+
+ *value = (u16) val;
+
+ return 0;
+}
+
+static int ad5593r_reg_write(struct ad5592r_state *st, u8 reg, u16 value)
+{
+ struct i2c_client *i2c = to_i2c_client(st->dev);
+
+ return i2c_smbus_write_word_swapped(i2c,
+ AD5593R_MODE_CONF | reg, value);
+}
+
+static int ad5593r_reg_read(struct ad5592r_state *st, u8 reg, u16 *value)
+{
+ struct i2c_client *i2c = to_i2c_client(st->dev);
+ s32 val;
+
+ val = i2c_smbus_read_word_swapped(i2c, AD5593R_MODE_REG_READBACK | reg);
+ if (val < 0)
+ return (int) val;
+
+ *value = (u16) val;
+
+ return 0;
+}
+
+static int ad5593r_gpio_read(struct ad5592r_state *st, u8 *value)
+{
+ struct i2c_client *i2c = to_i2c_client(st->dev);
+ s32 val;
+
+ val = i2c_smbus_read_word_swapped(i2c, AD5593R_MODE_GPIO_READBACK);
+ if (val < 0)
+ return (int) val;
+
+ *value = (u8) val;
+
+ return 0;
+}
+
+static const struct ad5592r_rw_ops ad5593r_rw_ops = {
+ .write_dac = ad5593r_write_dac,
+ .read_adc = ad5593r_read_adc,
+ .reg_write = ad5593r_reg_write,
+ .reg_read = ad5593r_reg_read,
+ .gpio_read = ad5593r_gpio_read,
+};
+
+static int ad5593r_i2c_probe(struct i2c_client *i2c,
+ const struct i2c_device_id *id)
+{
+ return ad5592r_probe(&i2c->dev, id->name, &ad5593r_rw_ops);
+}
+
+static int ad5593r_i2c_remove(struct i2c_client *i2c)
+{
+ return ad5592r_remove(&i2c->dev);
+}
+
+static const struct i2c_device_id ad5593r_i2c_ids[] = {
+ { .name = "ad5593r", },
+ {},
+};
+MODULE_DEVICE_TABLE(i2c, ad5593r_i2c_ids);
+
+static const struct of_device_id ad5593r_of_match[] = {
+ { .compatible = "adi,ad5593r", },
+ {},
+};
+MODULE_DEVICE_TABLE(of, ad5593r_of_match);
+
+static struct i2c_driver ad5593r_driver = {
+ .driver = {
+ .name = "ad5593r",
+ .of_match_table = of_match_ptr(ad5593r_of_match),
+ },
+ .probe = ad5593r_i2c_probe,
+ .remove = ad5593r_i2c_remove,
+ .id_table = ad5593r_i2c_ids,
+};
+module_i2c_driver(ad5593r_driver);
+
+MODULE_AUTHOR("Paul Cercueil <paul.cercueil@analog.com>");
+MODULE_DESCRIPTION("Analog Devices AD5592R multi-channel converters");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/iio/dac/lpc18xx_dac.c b/drivers/iio/dac/lpc18xx_dac.c
new file mode 100644
index 0000000..55d1456
--- /dev/null
+++ b/drivers/iio/dac/lpc18xx_dac.c
@@ -0,0 +1,210 @@
+/*
+ * IIO DAC driver for NXP LPC18xx DAC
+ *
+ * Copyright (C) 2016 Joachim Eastwood <manabian@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * UNSUPPORTED hardware features:
+ * - Interrupts
+ * - DMA
+ */
+
+#include <linux/clk.h>
+#include <linux/err.h>
+#include <linux/iio/iio.h>
+#include <linux/iio/driver.h>
+#include <linux/io.h>
+#include <linux/iopoll.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+#include <linux/regulator/consumer.h>
+
+/* LPC18XX DAC registers and bits */
+#define LPC18XX_DAC_CR 0x000
+#define LPC18XX_DAC_CR_VALUE_SHIFT 6
+#define LPC18XX_DAC_CR_VALUE_MASK 0x3ff
+#define LPC18XX_DAC_CR_BIAS BIT(16)
+#define LPC18XX_DAC_CTRL 0x004
+#define LPC18XX_DAC_CTRL_DMA_ENA BIT(3)
+
+struct lpc18xx_dac {
+ struct regulator *vref;
+ void __iomem *base;
+ struct mutex lock;
+ struct clk *clk;
+};
+
+static const struct iio_chan_spec lpc18xx_dac_iio_channels[] = {
+ {
+ .type = IIO_VOLTAGE,
+ .output = 1,
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) |
+ BIT(IIO_CHAN_INFO_SCALE),
+ },
+};
+
+static int lpc18xx_dac_read_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int *val, int *val2, long mask)
+{
+ struct lpc18xx_dac *dac = iio_priv(indio_dev);
+ u32 reg;
+
+ switch (mask) {
+ case IIO_CHAN_INFO_RAW:
+ reg = readl(dac->base + LPC18XX_DAC_CR);
+ *val = reg >> LPC18XX_DAC_CR_VALUE_SHIFT;
+ *val &= LPC18XX_DAC_CR_VALUE_MASK;
+
+ return IIO_VAL_INT;
+
+ case IIO_CHAN_INFO_SCALE:
+ *val = regulator_get_voltage(dac->vref) / 1000;
+ *val2 = 10;
+
+ return IIO_VAL_FRACTIONAL_LOG2;
+ }
+
+ return -EINVAL;
+}
+
+static int lpc18xx_dac_write_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int val, int val2, long mask)
+{
+ struct lpc18xx_dac *dac = iio_priv(indio_dev);
+ u32 reg;
+
+ switch (mask) {
+ case IIO_CHAN_INFO_RAW:
+ if (val < 0 || val > LPC18XX_DAC_CR_VALUE_MASK)
+ return -EINVAL;
+
+ reg = LPC18XX_DAC_CR_BIAS;
+ reg |= val << LPC18XX_DAC_CR_VALUE_SHIFT;
+
+ mutex_lock(&dac->lock);
+ writel(reg, dac->base + LPC18XX_DAC_CR);
+ writel(LPC18XX_DAC_CTRL_DMA_ENA, dac->base + LPC18XX_DAC_CTRL);
+ mutex_unlock(&dac->lock);
+
+ return 0;
+ }
+
+ return -EINVAL;
+}
+
+static const struct iio_info lpc18xx_dac_info = {
+ .read_raw = lpc18xx_dac_read_raw,
+ .write_raw = lpc18xx_dac_write_raw,
+ .driver_module = THIS_MODULE,
+};
+
+static int lpc18xx_dac_probe(struct platform_device *pdev)
+{
+ struct iio_dev *indio_dev;
+ struct lpc18xx_dac *dac;
+ struct resource *res;
+ int ret;
+
+ indio_dev = devm_iio_device_alloc(&pdev->dev, sizeof(*dac));
+ if (!indio_dev)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, indio_dev);
+ dac = iio_priv(indio_dev);
+ mutex_init(&dac->lock);
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ dac->base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(dac->base))
+ return PTR_ERR(dac->base);
+
+ dac->clk = devm_clk_get(&pdev->dev, NULL);
+ if (IS_ERR(dac->clk)) {
+ dev_err(&pdev->dev, "error getting clock\n");
+ return PTR_ERR(dac->clk);
+ }
+
+ dac->vref = devm_regulator_get(&pdev->dev, "vref");
+ if (IS_ERR(dac->vref)) {
+ dev_err(&pdev->dev, "error getting regulator\n");
+ return PTR_ERR(dac->vref);
+ }
+
+ indio_dev->name = dev_name(&pdev->dev);
+ indio_dev->dev.parent = &pdev->dev;
+ indio_dev->info = &lpc18xx_dac_info;
+ indio_dev->modes = INDIO_DIRECT_MODE;
+ indio_dev->channels = lpc18xx_dac_iio_channels;
+ indio_dev->num_channels = ARRAY_SIZE(lpc18xx_dac_iio_channels);
+
+ ret = regulator_enable(dac->vref);
+ if (ret) {
+ dev_err(&pdev->dev, "unable to enable regulator\n");
+ return ret;
+ }
+
+ ret = clk_prepare_enable(dac->clk);
+ if (ret) {
+ dev_err(&pdev->dev, "unable to enable clock\n");
+ goto dis_reg;
+ }
+
+ writel(0, dac->base + LPC18XX_DAC_CTRL);
+ writel(0, dac->base + LPC18XX_DAC_CR);
+
+ ret = iio_device_register(indio_dev);
+ if (ret) {
+ dev_err(&pdev->dev, "unable to register device\n");
+ goto dis_clk;
+ }
+
+ return 0;
+
+dis_clk:
+ clk_disable_unprepare(dac->clk);
+dis_reg:
+ regulator_disable(dac->vref);
+ return ret;
+}
+
+static int lpc18xx_dac_remove(struct platform_device *pdev)
+{
+ struct iio_dev *indio_dev = platform_get_drvdata(pdev);
+ struct lpc18xx_dac *dac = iio_priv(indio_dev);
+
+ iio_device_unregister(indio_dev);
+
+ writel(0, dac->base + LPC18XX_DAC_CTRL);
+ clk_disable_unprepare(dac->clk);
+ regulator_disable(dac->vref);
+
+ return 0;
+}
+
+static const struct of_device_id lpc18xx_dac_match[] = {
+ { .compatible = "nxp,lpc1850-dac" },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, lpc18xx_dac_match);
+
+static struct platform_driver lpc18xx_dac_driver = {
+ .probe = lpc18xx_dac_probe,
+ .remove = lpc18xx_dac_remove,
+ .driver = {
+ .name = "lpc18xx-dac",
+ .of_match_table = lpc18xx_dac_match,
+ },
+};
+module_platform_driver(lpc18xx_dac_driver);
+
+MODULE_DESCRIPTION("LPC18xx DAC driver");
+MODULE_AUTHOR("Joachim Eastwood <manabian@gmail.com>");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/iio/dac/stx104.c b/drivers/iio/dac/stx104.c
index 174f4b7..2794122 100644
--- a/drivers/iio/dac/stx104.c
+++ b/drivers/iio/dac/stx104.c
@@ -33,16 +33,9 @@
}
#define STX104_EXTENT 16
-/**
- * The highest base address possible for an ISA device is 0x3FF; this results in
- * 1024 possible base addresses. Dividing the number of possible base addresses
- * by the address extent taken by each device results in the maximum number of
- * devices on a system.
- */
-#define MAX_NUM_STX104 (1024 / STX104_EXTENT)
-static unsigned base[MAX_NUM_STX104];
-static unsigned num_stx104;
+static unsigned int base[max_num_isa_dev(STX104_EXTENT)];
+static unsigned int num_stx104;
module_param_array(base, uint, &num_stx104, 0);
MODULE_PARM_DESC(base, "Apex Embedded Systems STX104 base addresses");
@@ -134,18 +127,7 @@
}
};
-static void __exit stx104_exit(void)
-{
- isa_unregister_driver(&stx104_driver);
-}
-
-static int __init stx104_init(void)
-{
- return isa_register_driver(&stx104_driver, num_stx104);
-}
-
-module_init(stx104_init);
-module_exit(stx104_exit);
+module_isa_driver(stx104_driver, num_stx104);
MODULE_AUTHOR("William Breathitt Gray <vilhelm.gray@gmail.com>");
MODULE_DESCRIPTION("Apex Embedded Systems STX104 DAC driver");
diff --git a/drivers/iio/frequency/ad9523.c b/drivers/iio/frequency/ad9523.c
index 44a30f2..99eba52 100644
--- a/drivers/iio/frequency/ad9523.c
+++ b/drivers/iio/frequency/ad9523.c
@@ -284,7 +284,7 @@
} data[2] ____cacheline_aligned;
};
-static int ad9523_read(struct iio_dev *indio_dev, unsigned addr)
+static int ad9523_read(struct iio_dev *indio_dev, unsigned int addr)
{
struct ad9523_state *st = iio_priv(indio_dev);
int ret;
@@ -318,7 +318,8 @@
return ret;
};
-static int ad9523_write(struct iio_dev *indio_dev, unsigned addr, unsigned val)
+static int ad9523_write(struct iio_dev *indio_dev,
+ unsigned int addr, unsigned int val)
{
struct ad9523_state *st = iio_priv(indio_dev);
int ret;
@@ -351,11 +352,11 @@
}
static int ad9523_vco_out_map(struct iio_dev *indio_dev,
- unsigned ch, unsigned out)
+ unsigned int ch, unsigned int out)
{
struct ad9523_state *st = iio_priv(indio_dev);
int ret;
- unsigned mask;
+ unsigned int mask;
switch (ch) {
case 0 ... 3:
@@ -405,7 +406,7 @@
}
static int ad9523_set_clock_provider(struct iio_dev *indio_dev,
- unsigned ch, unsigned long freq)
+ unsigned int ch, unsigned long freq)
{
struct ad9523_state *st = iio_priv(indio_dev);
long tmp1, tmp2;
@@ -619,7 +620,7 @@
long m)
{
struct ad9523_state *st = iio_priv(indio_dev);
- unsigned code;
+ unsigned int code;
int ret;
mutex_lock(&indio_dev->mlock);
@@ -655,7 +656,7 @@
long mask)
{
struct ad9523_state *st = iio_priv(indio_dev);
- unsigned reg;
+ unsigned int reg;
int ret, tmp, code;
mutex_lock(&indio_dev->mlock);
@@ -709,8 +710,8 @@
}
static int ad9523_reg_access(struct iio_dev *indio_dev,
- unsigned reg, unsigned writeval,
- unsigned *readval)
+ unsigned int reg, unsigned int writeval,
+ unsigned int *readval)
{
int ret;
diff --git a/drivers/iio/gyro/Kconfig b/drivers/iio/gyro/Kconfig
index e816d29..205a844 100644
--- a/drivers/iio/gyro/Kconfig
+++ b/drivers/iio/gyro/Kconfig
@@ -93,7 +93,7 @@
select IIO_TRIGGERED_BUFFER if (IIO_BUFFER)
help
Say yes here to build support for STMicroelectronics gyroscopes:
- L3G4200D, LSM330DL, L3GD20, LSM330DLC, L3G4IS, LSM330.
+ L3G4200D, LSM330DL, L3GD20, LSM330DLC, L3G4IS, LSM330, LSM9DS0.
This driver can also be built as a module. If so, these modules
will be created:
diff --git a/drivers/iio/gyro/bmg160_core.c b/drivers/iio/gyro/bmg160_core.c
index 4dac567..7ccc044 100644
--- a/drivers/iio/gyro/bmg160_core.c
+++ b/drivers/iio/gyro/bmg160_core.c
@@ -17,7 +17,6 @@
#include <linux/delay.h>
#include <linux/slab.h>
#include <linux/acpi.h>
-#include <linux/gpio/consumer.h>
#include <linux/pm.h>
#include <linux/pm_runtime.h>
#include <linux/iio/iio.h>
@@ -31,7 +30,6 @@
#include "bmg160.h"
#define BMG160_IRQ_NAME "bmg160_event"
-#define BMG160_GPIO_NAME "gpio_int"
#define BMG160_REG_CHIP_ID 0x00
#define BMG160_CHIP_ID_VAL 0x0F
@@ -97,7 +95,6 @@
#define BMG160_AUTO_SUSPEND_DELAY_MS 2000
struct bmg160_data {
- struct device *dev;
struct regmap *regmap;
struct iio_trigger *dready_trig;
struct iio_trigger *motion_trig;
@@ -116,6 +113,7 @@
AXIS_X,
AXIS_Y,
AXIS_Z,
+ AXIS_MAX,
};
static const struct {
@@ -138,11 +136,12 @@
static int bmg160_set_mode(struct bmg160_data *data, u8 mode)
{
+ struct device *dev = regmap_get_device(data->regmap);
int ret;
ret = regmap_write(data->regmap, BMG160_REG_PMU_LPW, mode);
if (ret < 0) {
- dev_err(data->dev, "Error writing reg_pmu_lpw\n");
+ dev_err(dev, "Error writing reg_pmu_lpw\n");
return ret;
}
@@ -163,6 +162,7 @@
static int bmg160_set_bw(struct bmg160_data *data, int val)
{
+ struct device *dev = regmap_get_device(data->regmap);
int ret;
int bw_bits;
@@ -172,7 +172,7 @@
ret = regmap_write(data->regmap, BMG160_REG_PMU_BW, bw_bits);
if (ret < 0) {
- dev_err(data->dev, "Error writing reg_pmu_bw\n");
+ dev_err(dev, "Error writing reg_pmu_bw\n");
return ret;
}
@@ -183,18 +183,19 @@
static int bmg160_chip_init(struct bmg160_data *data)
{
+ struct device *dev = regmap_get_device(data->regmap);
int ret;
unsigned int val;
ret = regmap_read(data->regmap, BMG160_REG_CHIP_ID, &val);
if (ret < 0) {
- dev_err(data->dev, "Error reading reg_chip_id\n");
+ dev_err(dev, "Error reading reg_chip_id\n");
return ret;
}
- dev_dbg(data->dev, "Chip Id %x\n", val);
+ dev_dbg(dev, "Chip Id %x\n", val);
if (val != BMG160_CHIP_ID_VAL) {
- dev_err(data->dev, "invalid chip %x\n", val);
+ dev_err(dev, "invalid chip %x\n", val);
return -ENODEV;
}
@@ -213,14 +214,14 @@
/* Set Default Range */
ret = regmap_write(data->regmap, BMG160_REG_RANGE, BMG160_RANGE_500DPS);
if (ret < 0) {
- dev_err(data->dev, "Error writing reg_range\n");
+ dev_err(dev, "Error writing reg_range\n");
return ret;
}
data->dps_range = BMG160_RANGE_500DPS;
ret = regmap_read(data->regmap, BMG160_REG_SLOPE_THRES, &val);
if (ret < 0) {
- dev_err(data->dev, "Error reading reg_slope_thres\n");
+ dev_err(dev, "Error reading reg_slope_thres\n");
return ret;
}
data->slope_thres = val;
@@ -229,7 +230,7 @@
ret = regmap_update_bits(data->regmap, BMG160_REG_INT_EN_1,
BMG160_INT1_BIT_OD, 0);
if (ret < 0) {
- dev_err(data->dev, "Error updating bits in reg_int_en_1\n");
+ dev_err(dev, "Error updating bits in reg_int_en_1\n");
return ret;
}
@@ -237,7 +238,7 @@
BMG160_INT_MODE_LATCH_INT |
BMG160_INT_MODE_LATCH_RESET);
if (ret < 0) {
- dev_err(data->dev,
+ dev_err(dev,
"Error writing reg_motion_intr\n");
return ret;
}
@@ -248,20 +249,21 @@
static int bmg160_set_power_state(struct bmg160_data *data, bool on)
{
#ifdef CONFIG_PM
+ struct device *dev = regmap_get_device(data->regmap);
int ret;
if (on)
- ret = pm_runtime_get_sync(data->dev);
+ ret = pm_runtime_get_sync(dev);
else {
- pm_runtime_mark_last_busy(data->dev);
- ret = pm_runtime_put_autosuspend(data->dev);
+ pm_runtime_mark_last_busy(dev);
+ ret = pm_runtime_put_autosuspend(dev);
}
if (ret < 0) {
- dev_err(data->dev,
- "Failed: bmg160_set_power_state for %d\n", on);
+ dev_err(dev, "Failed: bmg160_set_power_state for %d\n", on);
+
if (on)
- pm_runtime_put_noidle(data->dev);
+ pm_runtime_put_noidle(dev);
return ret;
}
@@ -273,6 +275,7 @@
static int bmg160_setup_any_motion_interrupt(struct bmg160_data *data,
bool status)
{
+ struct device *dev = regmap_get_device(data->regmap);
int ret;
/* Enable/Disable INT_MAP0 mapping */
@@ -280,7 +283,7 @@
BMG160_INT_MAP_0_BIT_ANY,
(status ? BMG160_INT_MAP_0_BIT_ANY : 0));
if (ret < 0) {
- dev_err(data->dev, "Error updating bits reg_int_map0\n");
+ dev_err(dev, "Error updating bits reg_int_map0\n");
return ret;
}
@@ -290,8 +293,7 @@
ret = regmap_write(data->regmap, BMG160_REG_SLOPE_THRES,
data->slope_thres);
if (ret < 0) {
- dev_err(data->dev,
- "Error writing reg_slope_thres\n");
+ dev_err(dev, "Error writing reg_slope_thres\n");
return ret;
}
@@ -299,8 +301,7 @@
BMG160_INT_MOTION_X | BMG160_INT_MOTION_Y |
BMG160_INT_MOTION_Z);
if (ret < 0) {
- dev_err(data->dev,
- "Error writing reg_motion_intr\n");
+ dev_err(dev, "Error writing reg_motion_intr\n");
return ret;
}
@@ -315,8 +316,7 @@
BMG160_INT_MODE_LATCH_INT |
BMG160_INT_MODE_LATCH_RESET);
if (ret < 0) {
- dev_err(data->dev,
- "Error writing reg_rst_latch\n");
+ dev_err(dev, "Error writing reg_rst_latch\n");
return ret;
}
}
@@ -329,7 +329,7 @@
}
if (ret < 0) {
- dev_err(data->dev, "Error writing reg_int_en0\n");
+ dev_err(dev, "Error writing reg_int_en0\n");
return ret;
}
@@ -339,6 +339,7 @@
static int bmg160_setup_new_data_interrupt(struct bmg160_data *data,
bool status)
{
+ struct device *dev = regmap_get_device(data->regmap);
int ret;
/* Enable/Disable INT_MAP1 mapping */
@@ -346,7 +347,7 @@
BMG160_INT_MAP_1_BIT_NEW_DATA,
(status ? BMG160_INT_MAP_1_BIT_NEW_DATA : 0));
if (ret < 0) {
- dev_err(data->dev, "Error updating bits in reg_int_map1\n");
+ dev_err(dev, "Error updating bits in reg_int_map1\n");
return ret;
}
@@ -355,9 +356,8 @@
BMG160_INT_MODE_NON_LATCH_INT |
BMG160_INT_MODE_LATCH_RESET);
if (ret < 0) {
- dev_err(data->dev,
- "Error writing reg_rst_latch\n");
- return ret;
+ dev_err(dev, "Error writing reg_rst_latch\n");
+ return ret;
}
ret = regmap_write(data->regmap, BMG160_REG_INT_EN_0,
@@ -369,16 +369,15 @@
BMG160_INT_MODE_LATCH_INT |
BMG160_INT_MODE_LATCH_RESET);
if (ret < 0) {
- dev_err(data->dev,
- "Error writing reg_rst_latch\n");
- return ret;
+ dev_err(dev, "Error writing reg_rst_latch\n");
+ return ret;
}
ret = regmap_write(data->regmap, BMG160_REG_INT_EN_0, 0);
}
if (ret < 0) {
- dev_err(data->dev, "Error writing reg_int_en0\n");
+ dev_err(dev, "Error writing reg_int_en0\n");
return ret;
}
@@ -401,6 +400,7 @@
static int bmg160_set_scale(struct bmg160_data *data, int val)
{
+ struct device *dev = regmap_get_device(data->regmap);
int ret, i;
for (i = 0; i < ARRAY_SIZE(bmg160_scale_table); ++i) {
@@ -408,8 +408,7 @@
ret = regmap_write(data->regmap, BMG160_REG_RANGE,
bmg160_scale_table[i].dps_range);
if (ret < 0) {
- dev_err(data->dev,
- "Error writing reg_range\n");
+ dev_err(dev, "Error writing reg_range\n");
return ret;
}
data->dps_range = bmg160_scale_table[i].dps_range;
@@ -422,6 +421,7 @@
static int bmg160_get_temp(struct bmg160_data *data, int *val)
{
+ struct device *dev = regmap_get_device(data->regmap);
int ret;
unsigned int raw_val;
@@ -434,7 +434,7 @@
ret = regmap_read(data->regmap, BMG160_REG_TEMP, &raw_val);
if (ret < 0) {
- dev_err(data->dev, "Error reading reg_temp\n");
+ dev_err(dev, "Error reading reg_temp\n");
bmg160_set_power_state(data, false);
mutex_unlock(&data->mutex);
return ret;
@@ -451,6 +451,7 @@
static int bmg160_get_axis(struct bmg160_data *data, int axis, int *val)
{
+ struct device *dev = regmap_get_device(data->regmap);
int ret;
__le16 raw_val;
@@ -464,7 +465,7 @@
ret = regmap_bulk_read(data->regmap, BMG160_AXIS_TO_REG(axis), &raw_val,
sizeof(raw_val));
if (ret < 0) {
- dev_err(data->dev, "Error reading axis %d\n", axis);
+ dev_err(dev, "Error reading axis %d\n", axis);
bmg160_set_power_state(data, false);
mutex_unlock(&data->mutex);
return ret;
@@ -764,26 +765,23 @@
.driver_module = THIS_MODULE,
};
+static const unsigned long bmg160_accel_scan_masks[] = {
+ BIT(AXIS_X) | BIT(AXIS_Y) | BIT(AXIS_Z),
+ 0};
+
static irqreturn_t bmg160_trigger_handler(int irq, void *p)
{
struct iio_poll_func *pf = p;
struct iio_dev *indio_dev = pf->indio_dev;
struct bmg160_data *data = iio_priv(indio_dev);
- int bit, ret, i = 0;
- unsigned int val;
+ int ret;
mutex_lock(&data->mutex);
- for_each_set_bit(bit, indio_dev->active_scan_mask,
- indio_dev->masklength) {
- ret = regmap_bulk_read(data->regmap, BMG160_AXIS_TO_REG(bit),
- &val, 2);
- if (ret < 0) {
- mutex_unlock(&data->mutex);
- goto err;
- }
- data->buffer[i++] = val;
- }
+ ret = regmap_bulk_read(data->regmap, BMG160_REG_XOUT_L,
+ data->buffer, AXIS_MAX * 2);
mutex_unlock(&data->mutex);
+ if (ret < 0)
+ goto err;
iio_push_to_buffers_with_timestamp(indio_dev, data->buffer,
pf->timestamp);
@@ -797,6 +795,7 @@
{
struct iio_dev *indio_dev = iio_trigger_get_drvdata(trig);
struct bmg160_data *data = iio_priv(indio_dev);
+ struct device *dev = regmap_get_device(data->regmap);
int ret;
/* new data interrupts don't need ack */
@@ -808,7 +807,7 @@
BMG160_INT_MODE_LATCH_INT |
BMG160_INT_MODE_LATCH_RESET);
if (ret < 0) {
- dev_err(data->dev, "Error writing reg_rst_latch\n");
+ dev_err(dev, "Error writing reg_rst_latch\n");
return ret;
}
@@ -868,13 +867,14 @@
{
struct iio_dev *indio_dev = private;
struct bmg160_data *data = iio_priv(indio_dev);
+ struct device *dev = regmap_get_device(data->regmap);
int ret;
int dir;
unsigned int val;
ret = regmap_read(data->regmap, BMG160_REG_INT_STATUS_2, &val);
if (ret < 0) {
- dev_err(data->dev, "Error reading reg_int_status2\n");
+ dev_err(dev, "Error reading reg_int_status2\n");
goto ack_intr_status;
}
@@ -911,8 +911,7 @@
BMG160_INT_MODE_LATCH_INT |
BMG160_INT_MODE_LATCH_RESET);
if (ret < 0)
- dev_err(data->dev,
- "Error writing reg_rst_latch\n");
+ dev_err(dev, "Error writing reg_rst_latch\n");
}
return IRQ_HANDLED;
@@ -956,29 +955,6 @@
.postdisable = bmg160_buffer_postdisable,
};
-static int bmg160_gpio_probe(struct bmg160_data *data)
-
-{
- struct device *dev;
- struct gpio_desc *gpio;
-
- dev = data->dev;
-
- /* data ready gpio interrupt pin */
- gpio = devm_gpiod_get_index(dev, BMG160_GPIO_NAME, 0, GPIOD_IN);
- if (IS_ERR(gpio)) {
- dev_err(dev, "acpi gpio get index failed\n");
- return PTR_ERR(gpio);
- }
-
- data->irq = gpiod_to_irq(gpio);
-
- dev_dbg(dev, "GPIO resource, no:%d irq:%d\n", desc_to_gpio(gpio),
- data->irq);
-
- return 0;
-}
-
static const char *bmg160_match_acpi_device(struct device *dev)
{
const struct acpi_device_id *id;
@@ -1003,7 +979,6 @@
data = iio_priv(indio_dev);
dev_set_drvdata(dev, indio_dev);
- data->dev = dev;
data->irq = irq;
data->regmap = regmap;
@@ -1020,12 +995,10 @@
indio_dev->channels = bmg160_channels;
indio_dev->num_channels = ARRAY_SIZE(bmg160_channels);
indio_dev->name = name;
+ indio_dev->available_scan_masks = bmg160_accel_scan_masks;
indio_dev->modes = INDIO_DIRECT_MODE;
indio_dev->info = &bmg160_info;
- if (data->irq <= 0)
- bmg160_gpio_probe(data);
-
if (data->irq > 0) {
ret = devm_request_threaded_irq(dev,
data->irq,
@@ -1168,7 +1141,7 @@
ret = bmg160_set_mode(data, BMG160_MODE_SUSPEND);
if (ret < 0) {
- dev_err(data->dev, "set mode failed\n");
+ dev_err(dev, "set mode failed\n");
return -EAGAIN;
}
diff --git a/drivers/iio/gyro/st_gyro.h b/drivers/iio/gyro/st_gyro.h
index 5353d63..a5c5c4e 100644
--- a/drivers/iio/gyro/st_gyro.h
+++ b/drivers/iio/gyro/st_gyro.h
@@ -21,6 +21,7 @@
#define L3GD20_GYRO_DEV_NAME "l3gd20"
#define L3G4IS_GYRO_DEV_NAME "l3g4is_ui"
#define LSM330_GYRO_DEV_NAME "lsm330_gyro"
+#define LSM9DS0_GYRO_DEV_NAME "lsm9ds0_gyro"
/**
* struct st_sensors_platform_data - gyro platform data
diff --git a/drivers/iio/gyro/st_gyro_core.c b/drivers/iio/gyro/st_gyro_core.c
index 110f95b..52a3c87 100644
--- a/drivers/iio/gyro/st_gyro_core.c
+++ b/drivers/iio/gyro/st_gyro_core.c
@@ -190,6 +190,7 @@
* drain settings, but only for INT1 and not
* for the DRDY line on INT2.
*/
+ .addr_stat_drdy = ST_SENSORS_DEFAULT_STAT_ADDR,
},
.multi_read_bit = ST_GYRO_1_MULTIREAD_BIT,
.bootime = 2,
@@ -203,6 +204,7 @@
[2] = LSM330DLC_GYRO_DEV_NAME,
[3] = L3G4IS_GYRO_DEV_NAME,
[4] = LSM330_GYRO_DEV_NAME,
+ [5] = LSM9DS0_GYRO_DEV_NAME,
},
.ch = (struct iio_chan_spec *)st_gyro_16bit_channels,
.odr = {
@@ -258,6 +260,7 @@
* drain settings, but only for INT1 and not
* for the DRDY line on INT2.
*/
+ .addr_stat_drdy = ST_SENSORS_DEFAULT_STAT_ADDR,
},
.multi_read_bit = ST_GYRO_2_MULTIREAD_BIT,
.bootime = 2,
@@ -322,6 +325,7 @@
* drain settings, but only for INT1 and not
* for the DRDY line on INT2.
*/
+ .addr_stat_drdy = ST_SENSORS_DEFAULT_STAT_ADDR,
},
.multi_read_bit = ST_GYRO_3_MULTIREAD_BIT,
.bootime = 2,
diff --git a/drivers/iio/gyro/st_gyro_i2c.c b/drivers/iio/gyro/st_gyro_i2c.c
index 6848451..40056b8 100644
--- a/drivers/iio/gyro/st_gyro_i2c.c
+++ b/drivers/iio/gyro/st_gyro_i2c.c
@@ -48,6 +48,10 @@
.compatible = "st,lsm330-gyro",
.data = LSM330_GYRO_DEV_NAME,
},
+ {
+ .compatible = "st,lsm9ds0-gyro",
+ .data = LSM9DS0_GYRO_DEV_NAME,
+ },
{},
};
MODULE_DEVICE_TABLE(of, st_gyro_of_match);
@@ -93,6 +97,7 @@
{ L3GD20_GYRO_DEV_NAME },
{ L3G4IS_GYRO_DEV_NAME },
{ LSM330_GYRO_DEV_NAME },
+ { LSM9DS0_GYRO_DEV_NAME },
{},
};
MODULE_DEVICE_TABLE(i2c, st_gyro_id_table);
diff --git a/drivers/iio/gyro/st_gyro_spi.c b/drivers/iio/gyro/st_gyro_spi.c
index d2b7a5f..fbf2fae 100644
--- a/drivers/iio/gyro/st_gyro_spi.c
+++ b/drivers/iio/gyro/st_gyro_spi.c
@@ -54,6 +54,7 @@
{ L3GD20_GYRO_DEV_NAME },
{ L3G4IS_GYRO_DEV_NAME },
{ LSM330_GYRO_DEV_NAME },
+ { LSM9DS0_GYRO_DEV_NAME },
{},
};
MODULE_DEVICE_TABLE(spi, st_gyro_id_table);
diff --git a/drivers/iio/humidity/Kconfig b/drivers/iio/humidity/Kconfig
index 866dda1..738a86d 100644
--- a/drivers/iio/humidity/Kconfig
+++ b/drivers/iio/humidity/Kconfig
@@ -3,6 +3,16 @@
#
menu "Humidity sensors"
+config AM2315
+ tristate "Aosong AM2315 relative humidity and temperature sensor"
+ depends on I2C
+ help
+ If you say yes here you get support for the Aosong AM2315
+ relative humidity and ambient temperature sensor.
+
+ This driver can also be built as a module. If so, the module will
+ be called am2315.
+
config DHT11
tristate "DHT11 (and compatible sensors) driver"
depends on GPIOLIB || COMPILE_TEST
diff --git a/drivers/iio/humidity/Makefile b/drivers/iio/humidity/Makefile
index c9f089a..4a73442 100644
--- a/drivers/iio/humidity/Makefile
+++ b/drivers/iio/humidity/Makefile
@@ -2,6 +2,7 @@
# Makefile for IIO humidity sensor drivers
#
+obj-$(CONFIG_AM2315) += am2315.o
obj-$(CONFIG_DHT11) += dht11.o
obj-$(CONFIG_HDC100X) += hdc100x.o
obj-$(CONFIG_HTU21) += htu21.o
diff --git a/drivers/iio/humidity/am2315.c b/drivers/iio/humidity/am2315.c
new file mode 100644
index 0000000..3be6d20
--- /dev/null
+++ b/drivers/iio/humidity/am2315.c
@@ -0,0 +1,303 @@
+/**
+ * Aosong AM2315 relative humidity and temperature
+ *
+ * Copyright (c) 2016, Intel Corporation.
+ *
+ * This file is subject to the terms and conditions of version 2 of
+ * the GNU General Public License. See the file COPYING in the main
+ * directory of this archive for more details.
+ *
+ * 7-bit I2C address: 0x5C.
+ */
+
+#include <linux/acpi.h>
+#include <linux/delay.h>
+#include <linux/i2c.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/iio/buffer.h>
+#include <linux/iio/iio.h>
+#include <linux/iio/sysfs.h>
+#include <linux/iio/trigger_consumer.h>
+#include <linux/iio/triggered_buffer.h>
+
+#define AM2315_REG_HUM_MSB 0x00
+#define AM2315_REG_HUM_LSB 0x01
+#define AM2315_REG_TEMP_MSB 0x02
+#define AM2315_REG_TEMP_LSB 0x03
+
+#define AM2315_FUNCTION_READ 0x03
+#define AM2315_HUM_OFFSET 2
+#define AM2315_TEMP_OFFSET 4
+#define AM2315_ALL_CHANNEL_MASK GENMASK(1, 0)
+
+#define AM2315_DRIVER_NAME "am2315"
+
+struct am2315_data {
+ struct i2c_client *client;
+ struct mutex lock;
+ s16 buffer[8]; /* 2x16-bit channels + 2x16 padding + 4x16 timestamp */
+};
+
+struct am2315_sensor_data {
+ s16 hum_data;
+ s16 temp_data;
+};
+
+static const struct iio_chan_spec am2315_channels[] = {
+ {
+ .type = IIO_HUMIDITYRELATIVE,
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) |
+ BIT(IIO_CHAN_INFO_SCALE),
+ .scan_index = 0,
+ .scan_type = {
+ .sign = 's',
+ .realbits = 16,
+ .storagebits = 16,
+ .endianness = IIO_CPU,
+ },
+ },
+ {
+ .type = IIO_TEMP,
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) |
+ BIT(IIO_CHAN_INFO_SCALE),
+ .scan_index = 1,
+ .scan_type = {
+ .sign = 's',
+ .realbits = 16,
+ .storagebits = 16,
+ .endianness = IIO_CPU,
+ },
+ },
+ IIO_CHAN_SOFT_TIMESTAMP(2),
+};
+
+/* CRC calculation algorithm, as specified in the datasheet (page 13). */
+static u16 am2315_crc(u8 *data, u8 nr_bytes)
+{
+ int i;
+ u16 crc = 0xffff;
+
+ while (nr_bytes--) {
+ crc ^= *data++;
+ for (i = 0; i < 8; i++) {
+ if (crc & 0x01) {
+ crc >>= 1;
+ crc ^= 0xA001;
+ } else {
+ crc >>= 1;
+ }
+ }
+ }
+
+ return crc;
+}
+
+/* Simple function that sends a few bytes to the device to wake it up. */
+static void am2315_ping(struct i2c_client *client)
+{
+ i2c_smbus_read_byte_data(client, AM2315_REG_HUM_MSB);
+}
+
+static int am2315_read_data(struct am2315_data *data,
+ struct am2315_sensor_data *sensor_data)
+{
+ int ret;
+ /* tx_buf format: <function code> <start addr> <nr of regs to read> */
+ u8 tx_buf[3] = { AM2315_FUNCTION_READ, AM2315_REG_HUM_MSB, 4 };
+ /*
+ * rx_buf format:
+ * <function code> <number of registers read>
+ * <humidity MSB> <humidity LSB> <temp MSB> <temp LSB>
+ * <CRC LSB> <CRC MSB>
+ */
+ u8 rx_buf[8];
+ u16 crc;
+
+ /* First wake up the device. */
+ am2315_ping(data->client);
+
+ mutex_lock(&data->lock);
+ ret = i2c_master_send(data->client, tx_buf, sizeof(tx_buf));
+ if (ret < 0) {
+ dev_err(&data->client->dev, "failed to send read request\n");
+ goto exit_unlock;
+ }
+ /* Wait 2-3 ms, then read back the data sent by the device. */
+ usleep_range(2000, 3000);
+ /* Do a bulk data read, then pick out what we need. */
+ ret = i2c_master_recv(data->client, rx_buf, sizeof(rx_buf));
+ if (ret < 0) {
+ dev_err(&data->client->dev, "failed to read sensor data\n");
+ goto exit_unlock;
+ }
+ mutex_unlock(&data->lock);
+ /*
+ * Do a CRC check on the data and compare it to the value
+ * calculated by the device.
+ */
+ crc = am2315_crc(rx_buf, sizeof(rx_buf) - 2);
+ if ((crc & 0xff) != rx_buf[6] || (crc >> 8) != rx_buf[7]) {
+ dev_err(&data->client->dev, "failed to verify sensor data\n");
+ return -EIO;
+ }
+
+ sensor_data->hum_data = (rx_buf[AM2315_HUM_OFFSET] << 8) |
+ rx_buf[AM2315_HUM_OFFSET + 1];
+ sensor_data->temp_data = (rx_buf[AM2315_TEMP_OFFSET] << 8) |
+ rx_buf[AM2315_TEMP_OFFSET + 1];
+
+ return ret;
+
+exit_unlock:
+ mutex_unlock(&data->lock);
+ return ret;
+}
+
+static irqreturn_t am2315_trigger_handler(int irq, void *p)
+{
+ int i;
+ int ret;
+ int bit;
+ struct iio_poll_func *pf = p;
+ struct iio_dev *indio_dev = pf->indio_dev;
+ struct am2315_data *data = iio_priv(indio_dev);
+ struct am2315_sensor_data sensor_data;
+
+ ret = am2315_read_data(data, &sensor_data);
+ if (ret < 0) {
+ mutex_unlock(&data->lock);
+ goto err;
+ }
+
+ mutex_lock(&data->lock);
+ if (*(indio_dev->active_scan_mask) == AM2315_ALL_CHANNEL_MASK) {
+ data->buffer[0] = sensor_data.hum_data;
+ data->buffer[1] = sensor_data.temp_data;
+ } else {
+ i = 0;
+ for_each_set_bit(bit, indio_dev->active_scan_mask,
+ indio_dev->masklength) {
+ data->buffer[i] = (bit ? sensor_data.temp_data :
+ sensor_data.hum_data);
+ i++;
+ }
+ }
+ mutex_unlock(&data->lock);
+
+ iio_push_to_buffers_with_timestamp(indio_dev, data->buffer,
+ pf->timestamp);
+err:
+ iio_trigger_notify_done(indio_dev->trig);
+ return IRQ_HANDLED;
+}
+
+static int am2315_read_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int *val, int *val2, long mask)
+{
+ int ret;
+ struct am2315_sensor_data sensor_data;
+ struct am2315_data *data = iio_priv(indio_dev);
+
+ switch (mask) {
+ case IIO_CHAN_INFO_RAW:
+ ret = am2315_read_data(data, &sensor_data);
+ if (ret < 0)
+ return ret;
+ *val = (chan->type == IIO_HUMIDITYRELATIVE) ?
+ sensor_data.hum_data : sensor_data.temp_data;
+ return IIO_VAL_INT;
+ case IIO_CHAN_INFO_SCALE:
+ *val = 100;
+ return IIO_VAL_INT;
+ }
+
+ return -EINVAL;
+}
+
+static const struct iio_info am2315_info = {
+ .driver_module = THIS_MODULE,
+ .read_raw = am2315_read_raw,
+};
+
+static int am2315_probe(struct i2c_client *client,
+ const struct i2c_device_id *id)
+{
+ int ret;
+ struct iio_dev *indio_dev;
+ struct am2315_data *data;
+
+ indio_dev = devm_iio_device_alloc(&client->dev, sizeof(*data));
+ if (!indio_dev) {
+ dev_err(&client->dev, "iio allocation failed!\n");
+ return -ENOMEM;
+ }
+
+ data = iio_priv(indio_dev);
+ data->client = client;
+ i2c_set_clientdata(client, indio_dev);
+ mutex_init(&data->lock);
+
+ indio_dev->dev.parent = &client->dev;
+ indio_dev->info = &am2315_info;
+ indio_dev->name = AM2315_DRIVER_NAME;
+ indio_dev->modes = INDIO_DIRECT_MODE;
+ indio_dev->channels = am2315_channels;
+ indio_dev->num_channels = ARRAY_SIZE(am2315_channels);
+
+ ret = iio_triggered_buffer_setup(indio_dev, NULL,
+ am2315_trigger_handler, NULL);
+ if (ret < 0) {
+ dev_err(&client->dev, "iio triggered buffer setup failed\n");
+ return ret;
+ }
+
+ ret = iio_device_register(indio_dev);
+ if (ret < 0)
+ goto err_buffer_cleanup;
+
+ return 0;
+
+err_buffer_cleanup:
+ iio_triggered_buffer_cleanup(indio_dev);
+ return ret;
+}
+
+static int am2315_remove(struct i2c_client *client)
+{
+ struct iio_dev *indio_dev = i2c_get_clientdata(client);
+
+ iio_device_unregister(indio_dev);
+ iio_triggered_buffer_cleanup(indio_dev);
+
+ return 0;
+}
+
+static const struct i2c_device_id am2315_i2c_id[] = {
+ {"am2315", 0},
+ {}
+};
+
+static const struct acpi_device_id am2315_acpi_id[] = {
+ {"AOS2315", 0},
+ {}
+};
+
+MODULE_DEVICE_TABLE(acpi, am2315_acpi_id);
+
+static struct i2c_driver am2315_driver = {
+ .driver = {
+ .name = "am2315",
+ .acpi_match_table = ACPI_PTR(am2315_acpi_id),
+ },
+ .probe = am2315_probe,
+ .remove = am2315_remove,
+ .id_table = am2315_i2c_id,
+};
+
+module_i2c_driver(am2315_driver);
+
+MODULE_AUTHOR("Tiberiu Breana <tiberiu.a.breana@intel.com>");
+MODULE_DESCRIPTION("Aosong AM2315 relative humidity and temperature");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/iio/humidity/dht11.c b/drivers/iio/humidity/dht11.c
index 20b500d..9c47bc9 100644
--- a/drivers/iio/humidity/dht11.c
+++ b/drivers/iio/humidity/dht11.c
@@ -96,6 +96,24 @@
struct {s64 ts; int value; } edges[DHT11_EDGES_PER_READ];
};
+#ifdef CONFIG_DYNAMIC_DEBUG
+/*
+ * dht11_edges_print: show the data as actually received by the
+ * driver.
+ */
+static void dht11_edges_print(struct dht11 *dht11)
+{
+ int i;
+
+ dev_dbg(dht11->dev, "%d edges detected:\n", dht11->num_edges);
+ for (i = 1; i < dht11->num_edges; ++i) {
+ dev_dbg(dht11->dev, "%d: %lld ns %s\n", i,
+ dht11->edges[i].ts - dht11->edges[i - 1].ts,
+ dht11->edges[i - 1].value ? "high" : "low");
+ }
+}
+#endif /* CONFIG_DYNAMIC_DEBUG */
+
static unsigned char dht11_decode_byte(char *bits)
{
unsigned char ret = 0;
@@ -119,8 +137,12 @@
for (i = 0; i < DHT11_BITS_PER_READ; ++i) {
t = dht11->edges[offset + 2 * i + 2].ts -
dht11->edges[offset + 2 * i + 1].ts;
- if (!dht11->edges[offset + 2 * i + 1].value)
- return -EIO; /* lost synchronisation */
+ if (!dht11->edges[offset + 2 * i + 1].value) {
+ dev_dbg(dht11->dev,
+ "lost synchronisation at edge %d\n",
+ offset + 2 * i + 1);
+ return -EIO;
+ }
bits[i] = t > DHT11_THRESHOLD;
}
@@ -130,8 +152,10 @@
temp_dec = dht11_decode_byte(&bits[24]);
checksum = dht11_decode_byte(&bits[32]);
- if (((hum_int + hum_dec + temp_int + temp_dec) & 0xff) != checksum)
+ if (((hum_int + hum_dec + temp_int + temp_dec) & 0xff) != checksum) {
+ dev_dbg(dht11->dev, "invalid checksum\n");
return -EIO;
+ }
dht11->timestamp = ktime_get_boot_ns();
if (hum_int < 20) { /* DHT22 */
@@ -182,6 +206,7 @@
mutex_lock(&dht11->lock);
if (dht11->timestamp + DHT11_DATA_VALID_TIME < ktime_get_boot_ns()) {
timeres = ktime_get_resolution_ns();
+ dev_dbg(dht11->dev, "current timeresolution: %dns\n", timeres);
if (timeres > DHT11_MIN_TIMERES) {
dev_err(dht11->dev, "timeresolution %dns too low\n",
timeres);
@@ -219,10 +244,13 @@
free_irq(dht11->irq, iio_dev);
+#ifdef CONFIG_DYNAMIC_DEBUG
+ dht11_edges_print(dht11);
+#endif
+
if (ret == 0 && dht11->num_edges < DHT11_EDGES_PER_READ - 1) {
- dev_err(&iio_dev->dev,
- "Only %d signal edges detected\n",
- dht11->num_edges);
+ dev_err(dht11->dev, "Only %d signal edges detected\n",
+ dht11->num_edges);
ret = -ETIMEDOUT;
}
if (ret < 0)
diff --git a/drivers/iio/imu/Kconfig b/drivers/iio/imu/Kconfig
index 5e610f7..1f1ad41 100644
--- a/drivers/iio/imu/Kconfig
+++ b/drivers/iio/imu/Kconfig
@@ -25,6 +25,8 @@
Say yes here to build support for Analog Devices ADIS16375, ADIS16480,
ADIS16485, ADIS16488 inertial sensors.
+source "drivers/iio/imu/bmi160/Kconfig"
+
config KMX61
tristate "Kionix KMX61 6-axis accelerometer and magnetometer"
depends on I2C
diff --git a/drivers/iio/imu/Makefile b/drivers/iio/imu/Makefile
index e1e6e3d..c71bcd3 100644
--- a/drivers/iio/imu/Makefile
+++ b/drivers/iio/imu/Makefile
@@ -13,6 +13,7 @@
adis_lib-$(CONFIG_IIO_ADIS_LIB_BUFFER) += adis_buffer.o
obj-$(CONFIG_IIO_ADIS_LIB) += adis_lib.o
+obj-y += bmi160/
obj-y += inv_mpu6050/
obj-$(CONFIG_KMX61) += kmx61.o
diff --git a/drivers/iio/imu/adis.c b/drivers/iio/imu/adis.c
index 911255d..ad6f91d 100644
--- a/drivers/iio/imu/adis.c
+++ b/drivers/iio/imu/adis.c
@@ -324,7 +324,12 @@
msleep(adis->data->startup_delay);
- return adis_check_status(adis);
+ ret = adis_check_status(adis);
+
+ if (adis->data->self_test_no_autoclear)
+ adis_write_reg_16(adis, adis->data->msc_ctrl_reg, 0x00);
+
+ return ret;
}
/**
diff --git a/drivers/iio/imu/bmi160/Kconfig b/drivers/iio/imu/bmi160/Kconfig
new file mode 100644
index 0000000..005c17c
--- /dev/null
+++ b/drivers/iio/imu/bmi160/Kconfig
@@ -0,0 +1,32 @@
+#
+# BMI160 IMU driver
+#
+
+config BMI160
+ tristate
+ select IIO_BUFFER
+ select IIO_TRIGGERED_BUFFER
+
+config BMI160_I2C
+ tristate "Bosch BMI160 I2C driver"
+ depends on I2C
+ select BMI160
+ select REGMAP_I2C
+ help
+ If you say yes here you get support for BMI160 IMU on I2C with
+ accelerometer, gyroscope and external BMG160 magnetometer.
+
+ This driver can also be built as a module. If so, the module will be
+ called bmi160_i2c.
+
+config BMI160_SPI
+ tristate "Bosch BMI160 SPI driver"
+ depends on SPI
+ select BMI160
+ select REGMAP_SPI
+ help
+ If you say yes here you get support for BMI160 IMU on SPI with
+ accelerometer, gyroscope and external BMG160 magnetometer.
+
+ This driver can also be built as a module. If so, the module will be
+ called bmi160_spi.
diff --git a/drivers/iio/imu/bmi160/Makefile b/drivers/iio/imu/bmi160/Makefile
new file mode 100644
index 0000000..10365e4
--- /dev/null
+++ b/drivers/iio/imu/bmi160/Makefile
@@ -0,0 +1,6 @@
+#
+# Makefile for Bosch BMI160 IMU
+#
+obj-$(CONFIG_BMI160) += bmi160_core.o
+obj-$(CONFIG_BMI160_I2C) += bmi160_i2c.o
+obj-$(CONFIG_BMI160_SPI) += bmi160_spi.o
diff --git a/drivers/iio/imu/bmi160/bmi160.h b/drivers/iio/imu/bmi160/bmi160.h
new file mode 100644
index 0000000..d2ae6ed
--- /dev/null
+++ b/drivers/iio/imu/bmi160/bmi160.h
@@ -0,0 +1,10 @@
+#ifndef BMI160_H_
+#define BMI160_H_
+
+extern const struct regmap_config bmi160_regmap_config;
+
+int bmi160_core_probe(struct device *dev, struct regmap *regmap,
+ const char *name, bool use_spi);
+void bmi160_core_remove(struct device *dev);
+
+#endif /* BMI160_H_ */
diff --git a/drivers/iio/imu/bmi160/bmi160_core.c b/drivers/iio/imu/bmi160/bmi160_core.c
new file mode 100644
index 0000000..0bf92b0
--- /dev/null
+++ b/drivers/iio/imu/bmi160/bmi160_core.c
@@ -0,0 +1,596 @@
+/*
+ * BMI160 - Bosch IMU (accel, gyro plus external magnetometer)
+ *
+ * Copyright (c) 2016, Intel Corporation.
+ *
+ * This file is subject to the terms and conditions of version 2 of
+ * the GNU General Public License. See the file COPYING in the main
+ * directory of this archive for more details.
+ *
+ * IIO core driver for BMI160, with support for I2C/SPI busses
+ *
+ * TODO: magnetometer, interrupts, hardware FIFO
+ */
+#include <linux/module.h>
+#include <linux/regmap.h>
+#include <linux/acpi.h>
+#include <linux/delay.h>
+
+#include <linux/iio/iio.h>
+#include <linux/iio/triggered_buffer.h>
+#include <linux/iio/trigger_consumer.h>
+#include <linux/iio/buffer.h>
+
+#include "bmi160.h"
+
+#define BMI160_REG_CHIP_ID 0x00
+#define BMI160_CHIP_ID_VAL 0xD1
+
+#define BMI160_REG_PMU_STATUS 0x03
+
+/* X axis data low byte address, the rest can be obtained using axis offset */
+#define BMI160_REG_DATA_MAGN_XOUT_L 0x04
+#define BMI160_REG_DATA_GYRO_XOUT_L 0x0C
+#define BMI160_REG_DATA_ACCEL_XOUT_L 0x12
+
+#define BMI160_REG_ACCEL_CONFIG 0x40
+#define BMI160_ACCEL_CONFIG_ODR_MASK GENMASK(3, 0)
+#define BMI160_ACCEL_CONFIG_BWP_MASK GENMASK(6, 4)
+
+#define BMI160_REG_ACCEL_RANGE 0x41
+#define BMI160_ACCEL_RANGE_2G 0x03
+#define BMI160_ACCEL_RANGE_4G 0x05
+#define BMI160_ACCEL_RANGE_8G 0x08
+#define BMI160_ACCEL_RANGE_16G 0x0C
+
+#define BMI160_REG_GYRO_CONFIG 0x42
+#define BMI160_GYRO_CONFIG_ODR_MASK GENMASK(3, 0)
+#define BMI160_GYRO_CONFIG_BWP_MASK GENMASK(5, 4)
+
+#define BMI160_REG_GYRO_RANGE 0x43
+#define BMI160_GYRO_RANGE_2000DPS 0x00
+#define BMI160_GYRO_RANGE_1000DPS 0x01
+#define BMI160_GYRO_RANGE_500DPS 0x02
+#define BMI160_GYRO_RANGE_250DPS 0x03
+#define BMI160_GYRO_RANGE_125DPS 0x04
+
+#define BMI160_REG_CMD 0x7E
+#define BMI160_CMD_ACCEL_PM_SUSPEND 0x10
+#define BMI160_CMD_ACCEL_PM_NORMAL 0x11
+#define BMI160_CMD_ACCEL_PM_LOW_POWER 0x12
+#define BMI160_CMD_GYRO_PM_SUSPEND 0x14
+#define BMI160_CMD_GYRO_PM_NORMAL 0x15
+#define BMI160_CMD_GYRO_PM_FAST_STARTUP 0x17
+#define BMI160_CMD_SOFTRESET 0xB6
+
+#define BMI160_REG_DUMMY 0x7F
+
+#define BMI160_ACCEL_PMU_MIN_USLEEP 3200
+#define BMI160_ACCEL_PMU_MAX_USLEEP 3800
+#define BMI160_GYRO_PMU_MIN_USLEEP 55000
+#define BMI160_GYRO_PMU_MAX_USLEEP 80000
+#define BMI160_SOFTRESET_USLEEP 1000
+
+#define BMI160_CHANNEL(_type, _axis, _index) { \
+ .type = _type, \
+ .modified = 1, \
+ .channel2 = IIO_MOD_##_axis, \
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW), \
+ .info_mask_shared_by_type = BIT(IIO_CHAN_INFO_SCALE) | \
+ BIT(IIO_CHAN_INFO_SAMP_FREQ), \
+ .scan_index = _index, \
+ .scan_type = { \
+ .sign = 's', \
+ .realbits = 16, \
+ .storagebits = 16, \
+ .endianness = IIO_LE, \
+ }, \
+}
+
+/* scan indexes follow DATA register order */
+enum bmi160_scan_axis {
+ BMI160_SCAN_EXT_MAGN_X = 0,
+ BMI160_SCAN_EXT_MAGN_Y,
+ BMI160_SCAN_EXT_MAGN_Z,
+ BMI160_SCAN_RHALL,
+ BMI160_SCAN_GYRO_X,
+ BMI160_SCAN_GYRO_Y,
+ BMI160_SCAN_GYRO_Z,
+ BMI160_SCAN_ACCEL_X,
+ BMI160_SCAN_ACCEL_Y,
+ BMI160_SCAN_ACCEL_Z,
+ BMI160_SCAN_TIMESTAMP,
+};
+
+enum bmi160_sensor_type {
+ BMI160_ACCEL = 0,
+ BMI160_GYRO,
+ BMI160_EXT_MAGN,
+ BMI160_NUM_SENSORS /* must be last */
+};
+
+struct bmi160_data {
+ struct regmap *regmap;
+};
+
+const struct regmap_config bmi160_regmap_config = {
+ .reg_bits = 8,
+ .val_bits = 8,
+};
+EXPORT_SYMBOL(bmi160_regmap_config);
+
+struct bmi160_regs {
+ u8 data; /* LSB byte register for X-axis */
+ u8 config;
+ u8 config_odr_mask;
+ u8 config_bwp_mask;
+ u8 range;
+ u8 pmu_cmd_normal;
+ u8 pmu_cmd_suspend;
+};
+
+static struct bmi160_regs bmi160_regs[] = {
+ [BMI160_ACCEL] = {
+ .data = BMI160_REG_DATA_ACCEL_XOUT_L,
+ .config = BMI160_REG_ACCEL_CONFIG,
+ .config_odr_mask = BMI160_ACCEL_CONFIG_ODR_MASK,
+ .config_bwp_mask = BMI160_ACCEL_CONFIG_BWP_MASK,
+ .range = BMI160_REG_ACCEL_RANGE,
+ .pmu_cmd_normal = BMI160_CMD_ACCEL_PM_NORMAL,
+ .pmu_cmd_suspend = BMI160_CMD_ACCEL_PM_SUSPEND,
+ },
+ [BMI160_GYRO] = {
+ .data = BMI160_REG_DATA_GYRO_XOUT_L,
+ .config = BMI160_REG_GYRO_CONFIG,
+ .config_odr_mask = BMI160_GYRO_CONFIG_ODR_MASK,
+ .config_bwp_mask = BMI160_GYRO_CONFIG_BWP_MASK,
+ .range = BMI160_REG_GYRO_RANGE,
+ .pmu_cmd_normal = BMI160_CMD_GYRO_PM_NORMAL,
+ .pmu_cmd_suspend = BMI160_CMD_GYRO_PM_SUSPEND,
+ },
+};
+
+struct bmi160_pmu_time {
+ unsigned long min;
+ unsigned long max;
+};
+
+static struct bmi160_pmu_time bmi160_pmu_time[] = {
+ [BMI160_ACCEL] = {
+ .min = BMI160_ACCEL_PMU_MIN_USLEEP,
+ .max = BMI160_ACCEL_PMU_MAX_USLEEP
+ },
+ [BMI160_GYRO] = {
+ .min = BMI160_GYRO_PMU_MIN_USLEEP,
+ .max = BMI160_GYRO_PMU_MIN_USLEEP,
+ },
+};
+
+struct bmi160_scale {
+ u8 bits;
+ int uscale;
+};
+
+struct bmi160_odr {
+ u8 bits;
+ int odr;
+ int uodr;
+};
+
+static const struct bmi160_scale bmi160_accel_scale[] = {
+ { BMI160_ACCEL_RANGE_2G, 598},
+ { BMI160_ACCEL_RANGE_4G, 1197},
+ { BMI160_ACCEL_RANGE_8G, 2394},
+ { BMI160_ACCEL_RANGE_16G, 4788},
+};
+
+static const struct bmi160_scale bmi160_gyro_scale[] = {
+ { BMI160_GYRO_RANGE_2000DPS, 1065},
+ { BMI160_GYRO_RANGE_1000DPS, 532},
+ { BMI160_GYRO_RANGE_500DPS, 266},
+ { BMI160_GYRO_RANGE_250DPS, 133},
+ { BMI160_GYRO_RANGE_125DPS, 66},
+};
+
+struct bmi160_scale_item {
+ const struct bmi160_scale *tbl;
+ int num;
+};
+
+static const struct bmi160_scale_item bmi160_scale_table[] = {
+ [BMI160_ACCEL] = {
+ .tbl = bmi160_accel_scale,
+ .num = ARRAY_SIZE(bmi160_accel_scale),
+ },
+ [BMI160_GYRO] = {
+ .tbl = bmi160_gyro_scale,
+ .num = ARRAY_SIZE(bmi160_gyro_scale),
+ },
+};
+
+static const struct bmi160_odr bmi160_accel_odr[] = {
+ {0x01, 0, 78125},
+ {0x02, 1, 5625},
+ {0x03, 3, 125},
+ {0x04, 6, 25},
+ {0x05, 12, 5},
+ {0x06, 25, 0},
+ {0x07, 50, 0},
+ {0x08, 100, 0},
+ {0x09, 200, 0},
+ {0x0A, 400, 0},
+ {0x0B, 800, 0},
+ {0x0C, 1600, 0},
+};
+
+static const struct bmi160_odr bmi160_gyro_odr[] = {
+ {0x06, 25, 0},
+ {0x07, 50, 0},
+ {0x08, 100, 0},
+ {0x09, 200, 0},
+ {0x0A, 400, 0},
+ {0x0B, 8000, 0},
+ {0x0C, 1600, 0},
+ {0x0D, 3200, 0},
+};
+
+struct bmi160_odr_item {
+ const struct bmi160_odr *tbl;
+ int num;
+};
+
+static const struct bmi160_odr_item bmi160_odr_table[] = {
+ [BMI160_ACCEL] = {
+ .tbl = bmi160_accel_odr,
+ .num = ARRAY_SIZE(bmi160_accel_odr),
+ },
+ [BMI160_GYRO] = {
+ .tbl = bmi160_gyro_odr,
+ .num = ARRAY_SIZE(bmi160_gyro_odr),
+ },
+};
+
+static const struct iio_chan_spec bmi160_channels[] = {
+ BMI160_CHANNEL(IIO_ACCEL, X, BMI160_SCAN_ACCEL_X),
+ BMI160_CHANNEL(IIO_ACCEL, Y, BMI160_SCAN_ACCEL_Y),
+ BMI160_CHANNEL(IIO_ACCEL, Z, BMI160_SCAN_ACCEL_Z),
+ BMI160_CHANNEL(IIO_ANGL_VEL, X, BMI160_SCAN_GYRO_X),
+ BMI160_CHANNEL(IIO_ANGL_VEL, Y, BMI160_SCAN_GYRO_Y),
+ BMI160_CHANNEL(IIO_ANGL_VEL, Z, BMI160_SCAN_GYRO_Z),
+ IIO_CHAN_SOFT_TIMESTAMP(BMI160_SCAN_TIMESTAMP),
+};
+
+static enum bmi160_sensor_type bmi160_to_sensor(enum iio_chan_type iio_type)
+{
+ switch (iio_type) {
+ case IIO_ACCEL:
+ return BMI160_ACCEL;
+ case IIO_ANGL_VEL:
+ return BMI160_GYRO;
+ default:
+ return -EINVAL;
+ }
+}
+
+static
+int bmi160_set_mode(struct bmi160_data *data, enum bmi160_sensor_type t,
+ bool mode)
+{
+ int ret;
+ u8 cmd;
+
+ if (mode)
+ cmd = bmi160_regs[t].pmu_cmd_normal;
+ else
+ cmd = bmi160_regs[t].pmu_cmd_suspend;
+
+ ret = regmap_write(data->regmap, BMI160_REG_CMD, cmd);
+ if (ret < 0)
+ return ret;
+
+ usleep_range(bmi160_pmu_time[t].min, bmi160_pmu_time[t].max);
+
+ return 0;
+}
+
+static
+int bmi160_set_scale(struct bmi160_data *data, enum bmi160_sensor_type t,
+ int uscale)
+{
+ int i;
+
+ for (i = 0; i < bmi160_scale_table[t].num; i++)
+ if (bmi160_scale_table[t].tbl[i].uscale == uscale)
+ break;
+
+ if (i == bmi160_scale_table[t].num)
+ return -EINVAL;
+
+ return regmap_write(data->regmap, bmi160_regs[t].range,
+ bmi160_scale_table[t].tbl[i].bits);
+}
+
+static
+int bmi160_get_scale(struct bmi160_data *data, enum bmi160_sensor_type t,
+ int *uscale)
+{
+ int i, ret, val;
+
+ ret = regmap_read(data->regmap, bmi160_regs[t].range, &val);
+ if (ret < 0)
+ return ret;
+
+ for (i = 0; i < bmi160_scale_table[t].num; i++)
+ if (bmi160_scale_table[t].tbl[i].bits == val) {
+ *uscale = bmi160_scale_table[t].tbl[i].uscale;
+ return 0;
+ }
+
+ return -EINVAL;
+}
+
+static int bmi160_get_data(struct bmi160_data *data, int chan_type,
+ int axis, int *val)
+{
+ u8 reg;
+ int ret;
+ __le16 sample;
+ enum bmi160_sensor_type t = bmi160_to_sensor(chan_type);
+
+ reg = bmi160_regs[t].data + (axis - IIO_MOD_X) * sizeof(__le16);
+
+ ret = regmap_bulk_read(data->regmap, reg, &sample, sizeof(__le16));
+ if (ret < 0)
+ return ret;
+
+ *val = sign_extend32(le16_to_cpu(sample), 15);
+
+ return 0;
+}
+
+static
+int bmi160_set_odr(struct bmi160_data *data, enum bmi160_sensor_type t,
+ int odr, int uodr)
+{
+ int i;
+
+ for (i = 0; i < bmi160_odr_table[t].num; i++)
+ if (bmi160_odr_table[t].tbl[i].odr == odr &&
+ bmi160_odr_table[t].tbl[i].uodr == uodr)
+ break;
+
+ if (i >= bmi160_odr_table[t].num)
+ return -EINVAL;
+
+ return regmap_update_bits(data->regmap,
+ bmi160_regs[t].config,
+ bmi160_odr_table[t].tbl[i].bits,
+ bmi160_regs[t].config_odr_mask);
+}
+
+static int bmi160_get_odr(struct bmi160_data *data, enum bmi160_sensor_type t,
+ int *odr, int *uodr)
+{
+ int i, val, ret;
+
+ ret = regmap_read(data->regmap, bmi160_regs[t].config, &val);
+ if (ret < 0)
+ return ret;
+
+ val &= bmi160_regs[t].config_odr_mask;
+
+ for (i = 0; i < bmi160_odr_table[t].num; i++)
+ if (val == bmi160_odr_table[t].tbl[i].bits)
+ break;
+
+ if (i >= bmi160_odr_table[t].num)
+ return -EINVAL;
+
+ *odr = bmi160_odr_table[t].tbl[i].odr;
+ *uodr = bmi160_odr_table[t].tbl[i].uodr;
+
+ return 0;
+}
+
+static irqreturn_t bmi160_trigger_handler(int irq, void *p)
+{
+ struct iio_poll_func *pf = p;
+ struct iio_dev *indio_dev = pf->indio_dev;
+ struct bmi160_data *data = iio_priv(indio_dev);
+ s16 buf[16]; /* 3 sens x 3 axis x s16 + 3 x s16 pad + 4 x s16 tstamp */
+ int i, ret, j = 0, base = BMI160_REG_DATA_MAGN_XOUT_L;
+ __le16 sample;
+
+ for_each_set_bit(i, indio_dev->active_scan_mask,
+ indio_dev->masklength) {
+ ret = regmap_bulk_read(data->regmap, base + i * sizeof(__le16),
+ &sample, sizeof(__le16));
+ if (ret < 0)
+ goto done;
+ buf[j++] = sample;
+ }
+
+ iio_push_to_buffers_with_timestamp(indio_dev, buf, iio_get_time_ns());
+done:
+ iio_trigger_notify_done(indio_dev->trig);
+ return IRQ_HANDLED;
+}
+
+static int bmi160_read_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int *val, int *val2, long mask)
+{
+ int ret;
+ struct bmi160_data *data = iio_priv(indio_dev);
+
+ switch (mask) {
+ case IIO_CHAN_INFO_RAW:
+ ret = bmi160_get_data(data, chan->type, chan->channel2, val);
+ if (ret < 0)
+ return ret;
+ return IIO_VAL_INT;
+ case IIO_CHAN_INFO_SCALE:
+ *val = 0;
+ ret = bmi160_get_scale(data,
+ bmi160_to_sensor(chan->type), val2);
+ return ret < 0 ? ret : IIO_VAL_INT_PLUS_MICRO;
+ case IIO_CHAN_INFO_SAMP_FREQ:
+ ret = bmi160_get_odr(data, bmi160_to_sensor(chan->type),
+ val, val2);
+ return ret < 0 ? ret : IIO_VAL_INT_PLUS_MICRO;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int bmi160_write_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int val, int val2, long mask)
+{
+ struct bmi160_data *data = iio_priv(indio_dev);
+
+ switch (mask) {
+ case IIO_CHAN_INFO_SCALE:
+ return bmi160_set_scale(data,
+ bmi160_to_sensor(chan->type), val2);
+ break;
+ case IIO_CHAN_INFO_SAMP_FREQ:
+ return bmi160_set_odr(data, bmi160_to_sensor(chan->type),
+ val, val2);
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static const struct iio_info bmi160_info = {
+ .driver_module = THIS_MODULE,
+ .read_raw = bmi160_read_raw,
+ .write_raw = bmi160_write_raw,
+};
+
+static const char *bmi160_match_acpi_device(struct device *dev)
+{
+ const struct acpi_device_id *id;
+
+ id = acpi_match_device(dev->driver->acpi_match_table, dev);
+ if (!id)
+ return NULL;
+
+ return dev_name(dev);
+}
+
+static int bmi160_chip_init(struct bmi160_data *data, bool use_spi)
+{
+ int ret;
+ unsigned int val;
+ struct device *dev = regmap_get_device(data->regmap);
+
+ ret = regmap_write(data->regmap, BMI160_REG_CMD, BMI160_CMD_SOFTRESET);
+ if (ret < 0)
+ return ret;
+
+ usleep_range(BMI160_SOFTRESET_USLEEP, BMI160_SOFTRESET_USLEEP + 1);
+
+ /*
+ * CS rising edge is needed before starting SPI, so do a dummy read
+ * See Section 3.2.1, page 86 of the datasheet
+ */
+ if (use_spi) {
+ ret = regmap_read(data->regmap, BMI160_REG_DUMMY, &val);
+ if (ret < 0)
+ return ret;
+ }
+
+ ret = regmap_read(data->regmap, BMI160_REG_CHIP_ID, &val);
+ if (ret < 0) {
+ dev_err(dev, "Error reading chip id\n");
+ return ret;
+ }
+ if (val != BMI160_CHIP_ID_VAL) {
+ dev_err(dev, "Wrong chip id, got %x expected %x\n",
+ val, BMI160_CHIP_ID_VAL);
+ return -ENODEV;
+ }
+
+ ret = bmi160_set_mode(data, BMI160_ACCEL, true);
+ if (ret < 0)
+ return ret;
+
+ ret = bmi160_set_mode(data, BMI160_GYRO, true);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
+static void bmi160_chip_uninit(struct bmi160_data *data)
+{
+ bmi160_set_mode(data, BMI160_GYRO, false);
+ bmi160_set_mode(data, BMI160_ACCEL, false);
+}
+
+int bmi160_core_probe(struct device *dev, struct regmap *regmap,
+ const char *name, bool use_spi)
+{
+ struct iio_dev *indio_dev;
+ struct bmi160_data *data;
+ int ret;
+
+ indio_dev = devm_iio_device_alloc(dev, sizeof(*data));
+ if (!indio_dev)
+ return -ENOMEM;
+
+ data = iio_priv(indio_dev);
+ dev_set_drvdata(dev, indio_dev);
+ data->regmap = regmap;
+
+ ret = bmi160_chip_init(data, use_spi);
+ if (ret < 0)
+ return ret;
+
+ if (!name && ACPI_HANDLE(dev))
+ name = bmi160_match_acpi_device(dev);
+
+ indio_dev->dev.parent = dev;
+ indio_dev->channels = bmi160_channels;
+ indio_dev->num_channels = ARRAY_SIZE(bmi160_channels);
+ indio_dev->name = name;
+ indio_dev->modes = INDIO_DIRECT_MODE;
+ indio_dev->info = &bmi160_info;
+
+ ret = iio_triggered_buffer_setup(indio_dev, NULL,
+ bmi160_trigger_handler, NULL);
+ if (ret < 0)
+ goto uninit;
+
+ ret = iio_device_register(indio_dev);
+ if (ret < 0)
+ goto buffer_cleanup;
+
+ return 0;
+buffer_cleanup:
+ iio_triggered_buffer_cleanup(indio_dev);
+uninit:
+ bmi160_chip_uninit(data);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(bmi160_core_probe);
+
+void bmi160_core_remove(struct device *dev)
+{
+ struct iio_dev *indio_dev = dev_get_drvdata(dev);
+ struct bmi160_data *data = iio_priv(indio_dev);
+
+ iio_device_unregister(indio_dev);
+ iio_triggered_buffer_cleanup(indio_dev);
+ bmi160_chip_uninit(data);
+}
+EXPORT_SYMBOL_GPL(bmi160_core_remove);
+
+MODULE_AUTHOR("Daniel Baluta <daniel.baluta@intel.com");
+MODULE_DESCRIPTION("Bosch BMI160 driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/iio/imu/bmi160/bmi160_i2c.c b/drivers/iio/imu/bmi160/bmi160_i2c.c
new file mode 100644
index 0000000..07a179d
--- /dev/null
+++ b/drivers/iio/imu/bmi160/bmi160_i2c.c
@@ -0,0 +1,72 @@
+/*
+ * BMI160 - Bosch IMU, I2C bits
+ *
+ * Copyright (c) 2016, Intel Corporation.
+ *
+ * This file is subject to the terms and conditions of version 2 of
+ * the GNU General Public License. See the file COPYING in the main
+ * directory of this archive for more details.
+ *
+ * 7-bit I2C slave address is:
+ * - 0x68 if SDO is pulled to GND
+ * - 0x69 if SDO is pulled to VDDIO
+ */
+#include <linux/module.h>
+#include <linux/i2c.h>
+#include <linux/regmap.h>
+#include <linux/acpi.h>
+
+#include "bmi160.h"
+
+static int bmi160_i2c_probe(struct i2c_client *client,
+ const struct i2c_device_id *id)
+{
+ struct regmap *regmap;
+ const char *name = NULL;
+
+ regmap = devm_regmap_init_i2c(client, &bmi160_regmap_config);
+ if (IS_ERR(regmap)) {
+ dev_err(&client->dev, "Failed to register i2c regmap %d\n",
+ (int)PTR_ERR(regmap));
+ return PTR_ERR(regmap);
+ }
+
+ if (id)
+ name = id->name;
+
+ return bmi160_core_probe(&client->dev, regmap, name, false);
+}
+
+static int bmi160_i2c_remove(struct i2c_client *client)
+{
+ bmi160_core_remove(&client->dev);
+
+ return 0;
+}
+
+static const struct i2c_device_id bmi160_i2c_id[] = {
+ {"bmi160", 0},
+ {}
+};
+MODULE_DEVICE_TABLE(i2c, bmi160_i2c_id);
+
+static const struct acpi_device_id bmi160_acpi_match[] = {
+ {"BMI0160", 0},
+ { },
+};
+MODULE_DEVICE_TABLE(acpi, bmi160_acpi_match);
+
+static struct i2c_driver bmi160_i2c_driver = {
+ .driver = {
+ .name = "bmi160_i2c",
+ .acpi_match_table = ACPI_PTR(bmi160_acpi_match),
+ },
+ .probe = bmi160_i2c_probe,
+ .remove = bmi160_i2c_remove,
+ .id_table = bmi160_i2c_id,
+};
+module_i2c_driver(bmi160_i2c_driver);
+
+MODULE_AUTHOR("Daniel Baluta <daniel.baluta@intel.com>");
+MODULE_DESCRIPTION("BMI160 I2C driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/iio/imu/bmi160/bmi160_spi.c b/drivers/iio/imu/bmi160/bmi160_spi.c
new file mode 100644
index 0000000..1ec8b12
--- /dev/null
+++ b/drivers/iio/imu/bmi160/bmi160_spi.c
@@ -0,0 +1,63 @@
+/*
+ * BMI160 - Bosch IMU, SPI bits
+ *
+ * Copyright (c) 2016, Intel Corporation.
+ *
+ * This file is subject to the terms and conditions of version 2 of
+ * the GNU General Public License. See the file COPYING in the main
+ * directory of this archive for more details.
+ */
+#include <linux/module.h>
+#include <linux/spi/spi.h>
+#include <linux/regmap.h>
+#include <linux/acpi.h>
+
+#include "bmi160.h"
+
+static int bmi160_spi_probe(struct spi_device *spi)
+{
+ struct regmap *regmap;
+ const struct spi_device_id *id = spi_get_device_id(spi);
+
+ regmap = devm_regmap_init_spi(spi, &bmi160_regmap_config);
+ if (IS_ERR(regmap)) {
+ dev_err(&spi->dev, "Failed to register spi regmap %d\n",
+ (int)PTR_ERR(regmap));
+ return PTR_ERR(regmap);
+ }
+ return bmi160_core_probe(&spi->dev, regmap, id->name, true);
+}
+
+static int bmi160_spi_remove(struct spi_device *spi)
+{
+ bmi160_core_remove(&spi->dev);
+
+ return 0;
+}
+
+static const struct spi_device_id bmi160_spi_id[] = {
+ {"bmi160", 0},
+ {}
+};
+MODULE_DEVICE_TABLE(spi, bmi160_spi_id);
+
+static const struct acpi_device_id bmi160_acpi_match[] = {
+ {"BMI0160", 0},
+ { },
+};
+MODULE_DEVICE_TABLE(acpi, bmi160_acpi_match);
+
+static struct spi_driver bmi160_spi_driver = {
+ .probe = bmi160_spi_probe,
+ .remove = bmi160_spi_remove,
+ .id_table = bmi160_spi_id,
+ .driver = {
+ .acpi_match_table = ACPI_PTR(bmi160_acpi_match),
+ .name = "bmi160_spi",
+ },
+};
+module_spi_driver(bmi160_spi_driver);
+
+MODULE_AUTHOR("Daniel Baluta <daniel.baluta@intel.com");
+MODULE_DESCRIPTION("Bosch BMI160 SPI driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/iio/imu/inv_mpu6050/Kconfig b/drivers/iio/imu/inv_mpu6050/Kconfig
index 847455a..f756fee 100644
--- a/drivers/iio/imu/inv_mpu6050/Kconfig
+++ b/drivers/iio/imu/inv_mpu6050/Kconfig
@@ -13,10 +13,8 @@
select INV_MPU6050_IIO
select REGMAP_I2C
help
- This driver supports the Invensense MPU6050 devices.
- This driver can also support MPU6500 in MPU6050 compatibility mode
- and also in MPU6500 mode with some limitations.
- It is a gyroscope/accelerometer combo device.
+ This driver supports the Invensense MPU6050/6500/9150 motion tracking
+ devices over I2C.
This driver can be built as a module. The module will be called
inv-mpu6050-i2c.
@@ -26,7 +24,7 @@
select INV_MPU6050_IIO
select REGMAP_SPI
help
- This driver supports the Invensense MPU6050 devices.
- It is a gyroscope/accelerometer combo device.
+ This driver supports the Invensense MPU6000/6500/9150 motion tracking
+ devices over SPI.
This driver can be built as a module. The module will be called
inv-mpu6050-spi.
diff --git a/drivers/iio/imu/inv_mpu6050/inv_mpu_acpi.c b/drivers/iio/imu/inv_mpu6050/inv_mpu_acpi.c
index 2771106..f62b8bd 100644
--- a/drivers/iio/imu/inv_mpu6050/inv_mpu_acpi.c
+++ b/drivers/iio/imu/inv_mpu6050/inv_mpu_acpi.c
@@ -183,7 +183,7 @@
} else
return 0; /* no secondary addr, which is OK */
}
- st->mux_client = i2c_new_device(st->mux_adapter, &info);
+ st->mux_client = i2c_new_device(st->muxc->adapter[0], &info);
if (!st->mux_client)
return -ENODEV;
}
diff --git a/drivers/iio/imu/inv_mpu6050/inv_mpu_core.c b/drivers/iio/imu/inv_mpu6050/inv_mpu_core.c
index d192953..ee40dae 100644
--- a/drivers/iio/imu/inv_mpu6050/inv_mpu_core.c
+++ b/drivers/iio/imu/inv_mpu6050/inv_mpu_core.c
@@ -23,7 +23,6 @@
#include <linux/kfifo.h>
#include <linux/spinlock.h>
#include <linux/iio/iio.h>
-#include <linux/i2c-mux.h>
#include <linux/acpi.h>
#include "inv_mpu_iio.h"
@@ -88,16 +87,29 @@
.accl_fs = INV_MPU6050_FS_02G,
};
+/* Indexed by enum inv_devices */
static const struct inv_mpu6050_hw hw_info[] = {
{
- .num_reg = 117,
+ .whoami = INV_MPU6050_WHOAMI_VALUE,
+ .name = "MPU6050",
+ .reg = ®_set_6050,
+ .config = &chip_config_6050,
+ },
+ {
+ .whoami = INV_MPU6500_WHOAMI_VALUE,
.name = "MPU6500",
.reg = ®_set_6500,
.config = &chip_config_6050,
},
{
- .num_reg = 117,
- .name = "MPU6050",
+ .whoami = INV_MPU6000_WHOAMI_VALUE,
+ .name = "MPU6000",
+ .reg = ®_set_6050,
+ .config = &chip_config_6050,
+ },
+ {
+ .whoami = INV_MPU9150_WHOAMI_VALUE,
+ .name = "MPU9150",
.reg = ®_set_6050,
.config = &chip_config_6050,
},
@@ -600,6 +612,10 @@
/**
* inv_attr_show() - calling this function will show current
* parameters.
+ *
+ * Deprecated in favor of IIO mounting matrix API.
+ *
+ * See inv_get_mount_matrix()
*/
static ssize_t inv_attr_show(struct device *dev, struct device_attribute *attr,
char *buf)
@@ -644,6 +660,18 @@
return 0;
}
+static const struct iio_mount_matrix *
+inv_get_mount_matrix(const struct iio_dev *indio_dev,
+ const struct iio_chan_spec *chan)
+{
+ return &((struct inv_mpu6050_state *)iio_priv(indio_dev))->orientation;
+}
+
+static const struct iio_chan_spec_ext_info inv_ext_info[] = {
+ IIO_MOUNT_MATRIX(IIO_SHARED_BY_TYPE, inv_get_mount_matrix),
+ { },
+};
+
#define INV_MPU6050_CHAN(_type, _channel2, _index) \
{ \
.type = _type, \
@@ -660,6 +688,7 @@
.shift = 0, \
.endianness = IIO_BE, \
}, \
+ .ext_info = inv_ext_info, \
}
static const struct iio_chan_spec inv_mpu_channels[] = {
@@ -692,14 +721,16 @@
"0.000598 0.001196 0.002392 0.004785");
static IIO_DEV_ATTR_SAMP_FREQ(S_IRUGO | S_IWUSR, inv_fifo_rate_show,
inv_mpu6050_fifo_rate_store);
+
+/* Deprecated: kept for userspace backward compatibility. */
static IIO_DEVICE_ATTR(in_gyro_matrix, S_IRUGO, inv_attr_show, NULL,
ATTR_GYRO_MATRIX);
static IIO_DEVICE_ATTR(in_accel_matrix, S_IRUGO, inv_attr_show, NULL,
ATTR_ACCL_MATRIX);
static struct attribute *inv_attributes[] = {
- &iio_dev_attr_in_gyro_matrix.dev_attr.attr,
- &iio_dev_attr_in_accel_matrix.dev_attr.attr,
+ &iio_dev_attr_in_gyro_matrix.dev_attr.attr, /* deprecated */
+ &iio_dev_attr_in_accel_matrix.dev_attr.attr, /* deprecated */
&iio_dev_attr_sampling_frequency.dev_attr.attr,
&iio_const_attr_sampling_frequency_available.dev_attr.attr,
&iio_const_attr_in_accel_scale_available.dev_attr.attr,
@@ -726,6 +757,7 @@
static int inv_check_and_setup_chip(struct inv_mpu6050_state *st)
{
int result;
+ unsigned int regval;
st->hw = &hw_info[st->chip_type];
st->reg = hw_info[st->chip_type].reg;
@@ -736,6 +768,17 @@
if (result)
return result;
msleep(INV_MPU6050_POWER_UP_TIME);
+
+ /* check chip self-identification */
+ result = regmap_read(st->map, INV_MPU6050_REG_WHOAMI, ®val);
+ if (result)
+ return result;
+ if (regval != st->hw->whoami) {
+ dev_warn(regmap_get_device(st->map),
+ "whoami mismatch got %#02x expected %#02hhx for %s\n",
+ regval, st->hw->whoami, st->hw->name);
+ }
+
/*
* toggle power state. After reset, the sleep bit could be on
* or off depending on the OTP settings. Toggling power would
@@ -774,14 +817,31 @@
if (!indio_dev)
return -ENOMEM;
+ BUILD_BUG_ON(ARRAY_SIZE(hw_info) != INV_NUM_PARTS);
+ if (chip_type < 0 || chip_type >= INV_NUM_PARTS) {
+ dev_err(dev, "Bad invensense chip_type=%d name=%s\n",
+ chip_type, name);
+ return -ENODEV;
+ }
st = iio_priv(indio_dev);
st->chip_type = chip_type;
st->powerup_count = 0;
st->irq = irq;
st->map = regmap;
+
pdata = dev_get_platdata(dev);
- if (pdata)
+ if (!pdata) {
+ result = of_iio_read_mount_matrix(dev, "mount-matrix",
+ &st->orientation);
+ if (result) {
+ dev_err(dev, "Failed to retrieve mounting matrix %d\n",
+ result);
+ return result;
+ }
+ } else {
st->plat_data = *pdata;
+ }
+
/* power is turned on inside check chip type*/
result = inv_check_and_setup_chip(st);
if (result)
diff --git a/drivers/iio/imu/inv_mpu6050/inv_mpu_i2c.c b/drivers/iio/imu/inv_mpu6050/inv_mpu_i2c.c
index 5ee4e0d..e1fd7fa 100644
--- a/drivers/iio/imu/inv_mpu6050/inv_mpu_i2c.c
+++ b/drivers/iio/imu/inv_mpu6050/inv_mpu_i2c.c
@@ -15,7 +15,6 @@
#include <linux/delay.h>
#include <linux/err.h>
#include <linux/i2c.h>
-#include <linux/i2c-mux.h>
#include <linux/iio/iio.h>
#include <linux/module.h>
#include "inv_mpu_iio.h"
@@ -25,46 +24,16 @@
.val_bits = 8,
};
-/*
- * The i2c read/write needs to happen in unlocked mode. As the parent
- * adapter is common. If we use locked versions, it will fail as
- * the mux adapter will lock the parent i2c adapter, while calling
- * select/deselect functions.
- */
-static int inv_mpu6050_write_reg_unlocked(struct i2c_client *client,
- u8 reg, u8 d)
+static int inv_mpu6050_select_bypass(struct i2c_mux_core *muxc, u32 chan_id)
{
- int ret;
- u8 buf[2] = {reg, d};
- struct i2c_msg msg[1] = {
- {
- .addr = client->addr,
- .flags = 0,
- .len = sizeof(buf),
- .buf = buf,
- }
- };
-
- ret = __i2c_transfer(client->adapter, msg, 1);
- if (ret != 1)
- return ret;
-
- return 0;
-}
-
-static int inv_mpu6050_select_bypass(struct i2c_adapter *adap, void *mux_priv,
- u32 chan_id)
-{
- struct i2c_client *client = mux_priv;
- struct iio_dev *indio_dev = dev_get_drvdata(&client->dev);
+ struct iio_dev *indio_dev = i2c_mux_priv(muxc);
struct inv_mpu6050_state *st = iio_priv(indio_dev);
int ret = 0;
/* Use the same mutex which was used everywhere to protect power-op */
mutex_lock(&indio_dev->mlock);
if (!st->powerup_count) {
- ret = inv_mpu6050_write_reg_unlocked(client,
- st->reg->pwr_mgmt_1, 0);
+ ret = regmap_write(st->map, st->reg->pwr_mgmt_1, 0);
if (ret)
goto write_error;
@@ -73,10 +42,9 @@
}
if (!ret) {
st->powerup_count++;
- ret = inv_mpu6050_write_reg_unlocked(client,
- st->reg->int_pin_cfg,
- INV_MPU6050_INT_PIN_CFG |
- INV_MPU6050_BIT_BYPASS_EN);
+ ret = regmap_write(st->map, st->reg->int_pin_cfg,
+ INV_MPU6050_INT_PIN_CFG |
+ INV_MPU6050_BIT_BYPASS_EN);
}
write_error:
mutex_unlock(&indio_dev->mlock);
@@ -84,21 +52,18 @@
return ret;
}
-static int inv_mpu6050_deselect_bypass(struct i2c_adapter *adap,
- void *mux_priv, u32 chan_id)
+static int inv_mpu6050_deselect_bypass(struct i2c_mux_core *muxc, u32 chan_id)
{
- struct i2c_client *client = mux_priv;
- struct iio_dev *indio_dev = dev_get_drvdata(&client->dev);
+ struct iio_dev *indio_dev = i2c_mux_priv(muxc);
struct inv_mpu6050_state *st = iio_priv(indio_dev);
mutex_lock(&indio_dev->mlock);
/* It doesn't really mattter, if any of the calls fails */
- inv_mpu6050_write_reg_unlocked(client, st->reg->int_pin_cfg,
- INV_MPU6050_INT_PIN_CFG);
+ regmap_write(st->map, st->reg->int_pin_cfg, INV_MPU6050_INT_PIN_CFG);
st->powerup_count--;
if (!st->powerup_count)
- inv_mpu6050_write_reg_unlocked(client, st->reg->pwr_mgmt_1,
- INV_MPU6050_BIT_SLEEP);
+ regmap_write(st->map, st->reg->pwr_mgmt_1,
+ INV_MPU6050_BIT_SLEEP);
mutex_unlock(&indio_dev->mlock);
return 0;
@@ -160,16 +125,18 @@
return result;
st = iio_priv(dev_get_drvdata(&client->dev));
- st->mux_adapter = i2c_add_mux_adapter(client->adapter,
- &client->dev,
- client,
- 0, 0, 0,
- inv_mpu6050_select_bypass,
- inv_mpu6050_deselect_bypass);
- if (!st->mux_adapter) {
- result = -ENODEV;
+ st->muxc = i2c_mux_alloc(client->adapter, &client->dev,
+ 1, 0, I2C_MUX_LOCKED,
+ inv_mpu6050_select_bypass,
+ inv_mpu6050_deselect_bypass);
+ if (!st->muxc) {
+ result = -ENOMEM;
goto out_unreg_device;
}
+ st->muxc->priv = dev_get_drvdata(&client->dev);
+ result = i2c_mux_add_adapter(st->muxc, 0, 0, 0);
+ if (result)
+ goto out_unreg_device;
result = inv_mpu_acpi_create_mux_client(client);
if (result)
@@ -178,7 +145,7 @@
return 0;
out_del_mux:
- i2c_del_mux_adapter(st->mux_adapter);
+ i2c_mux_del_adapters(st->muxc);
out_unreg_device:
inv_mpu_core_remove(&client->dev);
return result;
@@ -190,7 +157,7 @@
struct inv_mpu6050_state *st = iio_priv(indio_dev);
inv_mpu_acpi_delete_mux_client(client);
- i2c_del_mux_adapter(st->mux_adapter);
+ i2c_mux_del_adapters(st->muxc);
return inv_mpu_core_remove(&client->dev);
}
@@ -202,13 +169,14 @@
static const struct i2c_device_id inv_mpu_id[] = {
{"mpu6050", INV_MPU6050},
{"mpu6500", INV_MPU6500},
+ {"mpu9150", INV_MPU9150},
{}
};
MODULE_DEVICE_TABLE(i2c, inv_mpu_id);
static const struct acpi_device_id inv_acpi_match[] = {
- {"INVN6500", 0},
+ {"INVN6500", INV_MPU6500},
{ },
};
diff --git a/drivers/iio/imu/inv_mpu6050/inv_mpu_iio.h b/drivers/iio/imu/inv_mpu6050/inv_mpu_iio.h
index e302a49..3bf8544c 100644
--- a/drivers/iio/imu/inv_mpu6050/inv_mpu_iio.h
+++ b/drivers/iio/imu/inv_mpu6050/inv_mpu_iio.h
@@ -11,6 +11,7 @@
* GNU General Public License for more details.
*/
#include <linux/i2c.h>
+#include <linux/i2c-mux.h>
#include <linux/kfifo.h>
#include <linux/spinlock.h>
#include <linux/iio/iio.h>
@@ -68,6 +69,7 @@
INV_MPU6050,
INV_MPU6500,
INV_MPU6000,
+ INV_MPU9150,
INV_NUM_PARTS
};
@@ -93,13 +95,13 @@
/**
* struct inv_mpu6050_hw - Other important hardware information.
- * @num_reg: Number of registers on device.
+ * @whoami: Self identification byte from WHO_AM_I register
* @name: name of the chip.
* @reg: register map of the chip.
* @config: configuration of the chip.
*/
struct inv_mpu6050_hw {
- u8 num_reg;
+ u8 whoami;
u8 *name;
const struct inv_mpu6050_reg_map *reg;
const struct inv_mpu6050_chip_config *config;
@@ -114,7 +116,8 @@
* @hw: Other hardware-specific information.
* @chip_type: chip type.
* @time_stamp_lock: spin lock to time stamp.
- * @plat_data: platform data.
+ * @plat_data: platform data (deprecated in favor of @orientation).
+ * @orientation: sensor chip orientation relative to main hardware.
* @timestamps: kfifo queue to store time stamp.
* @map regmap pointer.
* @irq interrupt number.
@@ -127,10 +130,11 @@
const struct inv_mpu6050_hw *hw;
enum inv_devices chip_type;
spinlock_t time_stamp_lock;
- struct i2c_adapter *mux_adapter;
+ struct i2c_mux_core *muxc;
struct i2c_client *mux_client;
unsigned int powerup_count;
struct inv_mpu6050_platform_data plat_data;
+ struct iio_mount_matrix orientation;
DECLARE_KFIFO(timestamps, long long, TIMESTAMP_FIFO_SIZE);
struct regmap *map;
int irq;
@@ -215,6 +219,13 @@
#define INV_MPU6050_MIN_FIFO_RATE 4
#define INV_MPU6050_ONE_K_HZ 1000
+#define INV_MPU6050_REG_WHOAMI 117
+
+#define INV_MPU6000_WHOAMI_VALUE 0x68
+#define INV_MPU6050_WHOAMI_VALUE 0x68
+#define INV_MPU6500_WHOAMI_VALUE 0x70
+#define INV_MPU9150_WHOAMI_VALUE 0x68
+
/* scan element definition */
enum inv_mpu6050_scan {
INV_MPU6050_SCAN_ACCL_X,
diff --git a/drivers/iio/imu/inv_mpu6050/inv_mpu_spi.c b/drivers/iio/imu/inv_mpu6050/inv_mpu_spi.c
index 7bcb8d8..190a4a5 100644
--- a/drivers/iio/imu/inv_mpu6050/inv_mpu_spi.c
+++ b/drivers/iio/imu/inv_mpu6050/inv_mpu_spi.c
@@ -44,9 +44,19 @@
static int inv_mpu_probe(struct spi_device *spi)
{
struct regmap *regmap;
- const struct spi_device_id *id = spi_get_device_id(spi);
- const char *name = id ? id->name : NULL;
- const int chip_type = id ? id->driver_data : 0;
+ const struct spi_device_id *spi_id;
+ const struct acpi_device_id *acpi_id;
+ const char *name = NULL;
+ enum inv_devices chip_type;
+
+ if ((spi_id = spi_get_device_id(spi))) {
+ chip_type = (enum inv_devices)spi_id->driver_data;
+ name = spi_id->name;
+ } else if ((acpi_id = acpi_match_device(spi->dev.driver->acpi_match_table, &spi->dev))) {
+ chip_type = (enum inv_devices)acpi_id->driver_data;
+ } else {
+ return -ENODEV;
+ }
regmap = devm_regmap_init_spi(spi, &inv_mpu_regmap_config);
if (IS_ERR(regmap)) {
@@ -70,13 +80,15 @@
*/
static const struct spi_device_id inv_mpu_id[] = {
{"mpu6000", INV_MPU6000},
+ {"mpu6500", INV_MPU6500},
+ {"mpu9150", INV_MPU9150},
{}
};
MODULE_DEVICE_TABLE(spi, inv_mpu_id);
static const struct acpi_device_id inv_acpi_match[] = {
- {"INVN6000", 0},
+ {"INVN6000", INV_MPU6000},
{ },
};
MODULE_DEVICE_TABLE(acpi, inv_acpi_match);
diff --git a/drivers/iio/imu/kmx61.c b/drivers/iio/imu/kmx61.c
index e5306b4..2e7dd57 100644
--- a/drivers/iio/imu/kmx61.c
+++ b/drivers/iio/imu/kmx61.c
@@ -14,7 +14,6 @@
#include <linux/module.h>
#include <linux/i2c.h>
#include <linux/acpi.h>
-#include <linux/gpio/consumer.h>
#include <linux/interrupt.h>
#include <linux/pm.h>
#include <linux/pm_runtime.h>
diff --git a/drivers/iio/industrialio-core.c b/drivers/iio/industrialio-core.c
index 70cb7eb..e6319a9 100644
--- a/drivers/iio/industrialio-core.c
+++ b/drivers/iio/industrialio-core.c
@@ -25,6 +25,7 @@
#include <linux/slab.h>
#include <linux/anon_inodes.h>
#include <linux/debugfs.h>
+#include <linux/mutex.h>
#include <linux/iio/iio.h>
#include "iio_core.h"
#include "iio_core_trigger.h"
@@ -78,6 +79,7 @@
[IIO_CONCENTRATION] = "concentration",
[IIO_RESISTANCE] = "resistance",
[IIO_PH] = "ph",
+ [IIO_UVINDEX] = "uvindex",
};
static const char * const iio_modifier_names[] = {
@@ -100,6 +102,7 @@
[IIO_MOD_LIGHT_RED] = "red",
[IIO_MOD_LIGHT_GREEN] = "green",
[IIO_MOD_LIGHT_BLUE] = "blue",
+ [IIO_MOD_LIGHT_UV] = "uv",
[IIO_MOD_QUATERNION] = "quaternion",
[IIO_MOD_TEMP_AMBIENT] = "ambient",
[IIO_MOD_TEMP_OBJECT] = "object",
@@ -409,6 +412,88 @@
}
EXPORT_SYMBOL_GPL(iio_enum_write);
+static const struct iio_mount_matrix iio_mount_idmatrix = {
+ .rotation = {
+ "1", "0", "0",
+ "0", "1", "0",
+ "0", "0", "1"
+ }
+};
+
+static int iio_setup_mount_idmatrix(const struct device *dev,
+ struct iio_mount_matrix *matrix)
+{
+ *matrix = iio_mount_idmatrix;
+ dev_info(dev, "mounting matrix not found: using identity...\n");
+ return 0;
+}
+
+ssize_t iio_show_mount_matrix(struct iio_dev *indio_dev, uintptr_t priv,
+ const struct iio_chan_spec *chan, char *buf)
+{
+ const struct iio_mount_matrix *mtx = ((iio_get_mount_matrix_t *)
+ priv)(indio_dev, chan);
+
+ if (IS_ERR(mtx))
+ return PTR_ERR(mtx);
+
+ if (!mtx)
+ mtx = &iio_mount_idmatrix;
+
+ return snprintf(buf, PAGE_SIZE, "%s, %s, %s; %s, %s, %s; %s, %s, %s\n",
+ mtx->rotation[0], mtx->rotation[1], mtx->rotation[2],
+ mtx->rotation[3], mtx->rotation[4], mtx->rotation[5],
+ mtx->rotation[6], mtx->rotation[7], mtx->rotation[8]);
+}
+EXPORT_SYMBOL_GPL(iio_show_mount_matrix);
+
+/**
+ * of_iio_read_mount_matrix() - retrieve iio device mounting matrix from
+ * device-tree "mount-matrix" property
+ * @dev: device the mounting matrix property is assigned to
+ * @propname: device specific mounting matrix property name
+ * @matrix: where to store retrieved matrix
+ *
+ * If device is assigned no mounting matrix property, a default 3x3 identity
+ * matrix will be filled in.
+ *
+ * Return: 0 if success, or a negative error code on failure.
+ */
+#ifdef CONFIG_OF
+int of_iio_read_mount_matrix(const struct device *dev,
+ const char *propname,
+ struct iio_mount_matrix *matrix)
+{
+ if (dev->of_node) {
+ int err = of_property_read_string_array(dev->of_node,
+ propname, matrix->rotation,
+ ARRAY_SIZE(iio_mount_idmatrix.rotation));
+
+ if (err == ARRAY_SIZE(iio_mount_idmatrix.rotation))
+ return 0;
+
+ if (err >= 0)
+ /* Invalid number of matrix entries. */
+ return -EINVAL;
+
+ if (err != -EINVAL)
+ /* Invalid matrix declaration format. */
+ return err;
+ }
+
+ /* Matrix was not declared at all: fallback to identity. */
+ return iio_setup_mount_idmatrix(dev, matrix);
+}
+#else
+int of_iio_read_mount_matrix(const struct device *dev,
+ const char *propname,
+ struct iio_mount_matrix *matrix)
+{
+ return iio_setup_mount_idmatrix(dev, matrix);
+}
+#endif
+EXPORT_SYMBOL(of_iio_read_mount_matrix);
+
/**
* iio_format_value() - Formats a IIO value into its string representation
* @buf: The buffer to which the formatted value gets written
@@ -1375,6 +1460,44 @@
}
EXPORT_SYMBOL_GPL(devm_iio_device_unregister);
+/**
+ * iio_device_claim_direct_mode - Keep device in direct mode
+ * @indio_dev: the iio_dev associated with the device
+ *
+ * If the device is in direct mode it is guaranteed to stay
+ * that way until iio_device_release_direct_mode() is called.
+ *
+ * Use with iio_device_release_direct_mode()
+ *
+ * Returns: 0 on success, -EBUSY on failure
+ */
+int iio_device_claim_direct_mode(struct iio_dev *indio_dev)
+{
+ mutex_lock(&indio_dev->mlock);
+
+ if (iio_buffer_enabled(indio_dev)) {
+ mutex_unlock(&indio_dev->mlock);
+ return -EBUSY;
+ }
+ return 0;
+}
+EXPORT_SYMBOL_GPL(iio_device_claim_direct_mode);
+
+/**
+ * iio_device_release_direct_mode - releases claim on direct mode
+ * @indio_dev: the iio_dev associated with the device
+ *
+ * Release the claim. Device is no longer guaranteed to stay
+ * in direct mode.
+ *
+ * Use with iio_device_claim_direct_mode()
+ */
+void iio_device_release_direct_mode(struct iio_dev *indio_dev)
+{
+ mutex_unlock(&indio_dev->mlock);
+}
+EXPORT_SYMBOL_GPL(iio_device_release_direct_mode);
+
subsys_initcall(iio_init);
module_exit(iio_exit);
diff --git a/drivers/iio/inkern.c b/drivers/iio/inkern.c
index 734a004..c4757e6 100644
--- a/drivers/iio/inkern.c
+++ b/drivers/iio/inkern.c
@@ -356,6 +356,54 @@
}
EXPORT_SYMBOL_GPL(iio_channel_release);
+static void devm_iio_channel_free(struct device *dev, void *res)
+{
+ struct iio_channel *channel = *(struct iio_channel **)res;
+
+ iio_channel_release(channel);
+}
+
+static int devm_iio_channel_match(struct device *dev, void *res, void *data)
+{
+ struct iio_channel **r = res;
+
+ if (!r || !*r) {
+ WARN_ON(!r || !*r);
+ return 0;
+ }
+
+ return *r == data;
+}
+
+struct iio_channel *devm_iio_channel_get(struct device *dev,
+ const char *channel_name)
+{
+ struct iio_channel **ptr, *channel;
+
+ ptr = devres_alloc(devm_iio_channel_free, sizeof(*ptr), GFP_KERNEL);
+ if (!ptr)
+ return ERR_PTR(-ENOMEM);
+
+ channel = iio_channel_get(dev, channel_name);
+ if (IS_ERR(channel)) {
+ devres_free(ptr);
+ return channel;
+ }
+
+ *ptr = channel;
+ devres_add(dev, ptr);
+
+ return channel;
+}
+EXPORT_SYMBOL_GPL(devm_iio_channel_get);
+
+void devm_iio_channel_release(struct device *dev, struct iio_channel *channel)
+{
+ WARN_ON(devres_release(dev, devm_iio_channel_free,
+ devm_iio_channel_match, channel));
+}
+EXPORT_SYMBOL_GPL(devm_iio_channel_release);
+
struct iio_channel *iio_channel_get_all(struct device *dev)
{
const char *name;
@@ -441,6 +489,42 @@
}
EXPORT_SYMBOL_GPL(iio_channel_release_all);
+static void devm_iio_channel_free_all(struct device *dev, void *res)
+{
+ struct iio_channel *channels = *(struct iio_channel **)res;
+
+ iio_channel_release_all(channels);
+}
+
+struct iio_channel *devm_iio_channel_get_all(struct device *dev)
+{
+ struct iio_channel **ptr, *channels;
+
+ ptr = devres_alloc(devm_iio_channel_free_all, sizeof(*ptr), GFP_KERNEL);
+ if (!ptr)
+ return ERR_PTR(-ENOMEM);
+
+ channels = iio_channel_get_all(dev);
+ if (IS_ERR(channels)) {
+ devres_free(ptr);
+ return channels;
+ }
+
+ *ptr = channels;
+ devres_add(dev, ptr);
+
+ return channels;
+}
+EXPORT_SYMBOL_GPL(devm_iio_channel_get_all);
+
+void devm_iio_channel_release_all(struct device *dev,
+ struct iio_channel *channels)
+{
+ WARN_ON(devres_release(dev, devm_iio_channel_free_all,
+ devm_iio_channel_match, channels));
+}
+EXPORT_SYMBOL_GPL(devm_iio_channel_release_all);
+
static int iio_channel_read(struct iio_channel *chan, int *val, int *val2,
enum iio_chan_info_enum info)
{
@@ -452,7 +536,7 @@
if (val2 == NULL)
val2 = &unused;
- if(!iio_channel_has_info(chan->channel, info))
+ if (!iio_channel_has_info(chan->channel, info))
return -EINVAL;
if (chan->indio_dev->info->read_raw_multi) {
diff --git a/drivers/iio/light/Kconfig b/drivers/iio/light/Kconfig
index cfd3df8..7c566f5 100644
--- a/drivers/iio/light/Kconfig
+++ b/drivers/iio/light/Kconfig
@@ -73,6 +73,17 @@
To compile this driver as a module, choose M here: the module will
be called bh1750.
+config BH1780
+ tristate "ROHM BH1780 ambient light sensor"
+ depends on I2C
+ depends on !SENSORS_BH1780
+ help
+ Say Y here to build support for the ROHM BH1780GLI ambient
+ light sensor.
+
+ To compile this driver as a module, choose M here: the module will
+ be called bh1780.
+
config CM32181
depends on I2C
tristate "CM32181 driver"
@@ -223,6 +234,17 @@
This driver can also be built as a module. If so, the module
will be called ltr501.
+config MAX44000
+ tristate "MAX44000 Ambient and Infrared Proximity Sensor"
+ depends on I2C
+ select REGMAP_I2C
+ help
+ Say Y here if you want to build support for Maxim Integrated's
+ MAX44000 ambient and infrared proximity sensor device.
+
+ To compile this driver as a module, choose M here:
+ the module will be called max44000.
+
config OPT3001
tristate "Texas Instruments OPT3001 Light Sensor"
depends on I2C
@@ -320,4 +342,14 @@
To compile this driver as a module, choose M here: the
module will be called vcnl4000.
+config VEML6070
+ tristate "VEML6070 UV A light sensor"
+ depends on I2C
+ help
+ Say Y here if you want to build a driver for the Vishay VEML6070 UV A
+ light sensor.
+
+ To compile this driver as a module, choose M here: the
+ module will be called veml6070.
+
endmenu
diff --git a/drivers/iio/light/Makefile b/drivers/iio/light/Makefile
index b2c3105..6f2a3c6 100644
--- a/drivers/iio/light/Makefile
+++ b/drivers/iio/light/Makefile
@@ -9,6 +9,7 @@
obj-$(CONFIG_APDS9300) += apds9300.o
obj-$(CONFIG_APDS9960) += apds9960.o
obj-$(CONFIG_BH1750) += bh1750.o
+obj-$(CONFIG_BH1780) += bh1780.o
obj-$(CONFIG_CM32181) += cm32181.o
obj-$(CONFIG_CM3232) += cm3232.o
obj-$(CONFIG_CM3323) += cm3323.o
@@ -20,6 +21,7 @@
obj-$(CONFIG_JSA1212) += jsa1212.o
obj-$(CONFIG_SENSORS_LM3533) += lm3533-als.o
obj-$(CONFIG_LTR501) += ltr501.o
+obj-$(CONFIG_MAX44000) += max44000.o
obj-$(CONFIG_OPT3001) += opt3001.o
obj-$(CONFIG_PA12203001) += pa12203001.o
obj-$(CONFIG_RPR0521) += rpr0521.o
@@ -30,3 +32,4 @@
obj-$(CONFIG_TSL4531) += tsl4531.o
obj-$(CONFIG_US5182D) += us5182d.o
obj-$(CONFIG_VCNL4000) += vcnl4000.o
+obj-$(CONFIG_VEML6070) += veml6070.o
diff --git a/drivers/iio/light/apds9960.c b/drivers/iio/light/apds9960.c
index a6af56a..b4dbb39 100644
--- a/drivers/iio/light/apds9960.c
+++ b/drivers/iio/light/apds9960.c
@@ -321,8 +321,12 @@
};
/* integration time in us */
-static const int apds9960_int_time[][2] =
- { {28000, 246}, {100000, 219}, {200000, 182}, {700000, 0} };
+static const int apds9960_int_time[][2] = {
+ { 28000, 246},
+ {100000, 219},
+ {200000, 182},
+ {700000, 0}
+};
/* gain mapping */
static const int apds9960_pxs_gain_map[] = {1, 2, 4, 8};
@@ -491,9 +495,10 @@
case IIO_INTENSITY:
ret = regmap_bulk_read(data->regmap, chan->address,
&buf, 2);
- if (!ret)
+ if (!ret) {
ret = IIO_VAL_INT;
- *val = le16_to_cpu(buf);
+ *val = le16_to_cpu(buf);
+ }
break;
default:
ret = -EINVAL;
diff --git a/drivers/iio/light/bh1780.c b/drivers/iio/light/bh1780.c
new file mode 100644
index 0000000..72b364e
--- /dev/null
+++ b/drivers/iio/light/bh1780.c
@@ -0,0 +1,297 @@
+/*
+ * ROHM 1780GLI Ambient Light Sensor Driver
+ *
+ * Copyright (C) 2016 Linaro Ltd.
+ * Author: Linus Walleij <linus.walleij@linaro.org>
+ * Loosely based on the previous BH1780 ALS misc driver
+ * Copyright (C) 2010 Texas Instruments
+ * Author: Hemanth V <hemanthv@ti.com>
+ */
+#include <linux/i2c.h>
+#include <linux/slab.h>
+#include <linux/platform_device.h>
+#include <linux/delay.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/pm_runtime.h>
+#include <linux/iio/iio.h>
+#include <linux/iio/sysfs.h>
+#include <linux/bitops.h>
+
+#define BH1780_CMD_BIT BIT(7)
+#define BH1780_REG_CONTROL 0x00
+#define BH1780_REG_PARTID 0x0A
+#define BH1780_REG_MANFID 0x0B
+#define BH1780_REG_DLOW 0x0C
+#define BH1780_REG_DHIGH 0x0D
+
+#define BH1780_REVMASK GENMASK(3,0)
+#define BH1780_POWMASK GENMASK(1,0)
+#define BH1780_POFF (0x0)
+#define BH1780_PON (0x3)
+
+/* power on settling time in ms */
+#define BH1780_PON_DELAY 2
+/* max time before value available in ms */
+#define BH1780_INTERVAL 250
+
+struct bh1780_data {
+ struct i2c_client *client;
+};
+
+static int bh1780_write(struct bh1780_data *bh1780, u8 reg, u8 val)
+{
+ int ret = i2c_smbus_write_byte_data(bh1780->client,
+ BH1780_CMD_BIT | reg,
+ val);
+ if (ret < 0)
+ dev_err(&bh1780->client->dev,
+ "i2c_smbus_write_byte_data failed error "
+ "%d, register %01x\n",
+ ret, reg);
+ return ret;
+}
+
+static int bh1780_read(struct bh1780_data *bh1780, u8 reg)
+{
+ int ret = i2c_smbus_read_byte_data(bh1780->client,
+ BH1780_CMD_BIT | reg);
+ if (ret < 0)
+ dev_err(&bh1780->client->dev,
+ "i2c_smbus_read_byte_data failed error "
+ "%d, register %01x\n",
+ ret, reg);
+ return ret;
+}
+
+static int bh1780_read_word(struct bh1780_data *bh1780, u8 reg)
+{
+ int ret = i2c_smbus_read_word_data(bh1780->client,
+ BH1780_CMD_BIT | reg);
+ if (ret < 0)
+ dev_err(&bh1780->client->dev,
+ "i2c_smbus_read_word_data failed error "
+ "%d, register %01x\n",
+ ret, reg);
+ return ret;
+}
+
+static int bh1780_debugfs_reg_access(struct iio_dev *indio_dev,
+ unsigned int reg, unsigned int writeval,
+ unsigned int *readval)
+{
+ struct bh1780_data *bh1780 = iio_priv(indio_dev);
+ int ret;
+
+ if (!readval)
+ bh1780_write(bh1780, (u8)reg, (u8)writeval);
+
+ ret = bh1780_read(bh1780, (u8)reg);
+ if (ret < 0)
+ return ret;
+
+ *readval = ret;
+
+ return 0;
+}
+
+static int bh1780_read_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int *val, int *val2, long mask)
+{
+ struct bh1780_data *bh1780 = iio_priv(indio_dev);
+ int value;
+
+ switch (mask) {
+ case IIO_CHAN_INFO_RAW:
+ switch (chan->type) {
+ case IIO_LIGHT:
+ pm_runtime_get_sync(&bh1780->client->dev);
+ value = bh1780_read_word(bh1780, BH1780_REG_DLOW);
+ if (value < 0)
+ return value;
+ pm_runtime_mark_last_busy(&bh1780->client->dev);
+ pm_runtime_put_autosuspend(&bh1780->client->dev);
+ *val = value;
+
+ return IIO_VAL_INT;
+ default:
+ return -EINVAL;
+ }
+ case IIO_CHAN_INFO_INT_TIME:
+ *val = 0;
+ *val2 = BH1780_INTERVAL * 1000;
+ return IIO_VAL_INT_PLUS_MICRO;
+ default:
+ return -EINVAL;
+ }
+}
+
+static const struct iio_info bh1780_info = {
+ .driver_module = THIS_MODULE,
+ .read_raw = bh1780_read_raw,
+ .debugfs_reg_access = bh1780_debugfs_reg_access,
+};
+
+static const struct iio_chan_spec bh1780_channels[] = {
+ {
+ .type = IIO_LIGHT,
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) |
+ BIT(IIO_CHAN_INFO_INT_TIME)
+ }
+};
+
+static int bh1780_probe(struct i2c_client *client,
+ const struct i2c_device_id *id)
+{
+ int ret;
+ struct bh1780_data *bh1780;
+ struct i2c_adapter *adapter = to_i2c_adapter(client->dev.parent);
+ struct iio_dev *indio_dev;
+
+ if (!i2c_check_functionality(adapter, I2C_FUNC_SMBUS_BYTE))
+ return -EIO;
+
+ indio_dev = devm_iio_device_alloc(&client->dev, sizeof(*bh1780));
+ if (!indio_dev)
+ return -ENOMEM;
+
+ bh1780 = iio_priv(indio_dev);
+ bh1780->client = client;
+ i2c_set_clientdata(client, indio_dev);
+
+ /* Power up the device */
+ ret = bh1780_write(bh1780, BH1780_REG_CONTROL, BH1780_PON);
+ if (ret < 0)
+ return ret;
+ msleep(BH1780_PON_DELAY);
+ pm_runtime_get_noresume(&client->dev);
+ pm_runtime_set_active(&client->dev);
+ pm_runtime_enable(&client->dev);
+
+ ret = bh1780_read(bh1780, BH1780_REG_PARTID);
+ if (ret < 0)
+ goto out_disable_pm;
+ dev_info(&client->dev,
+ "Ambient Light Sensor, Rev : %lu\n",
+ (ret & BH1780_REVMASK));
+
+ /*
+ * As the device takes 250 ms to even come up with a fresh
+ * measurement after power-on, do not shut it down unnecessarily.
+ * Set autosuspend to a five seconds.
+ */
+ pm_runtime_set_autosuspend_delay(&client->dev, 5000);
+ pm_runtime_use_autosuspend(&client->dev);
+ pm_runtime_put(&client->dev);
+
+ indio_dev->dev.parent = &client->dev;
+ indio_dev->info = &bh1780_info;
+ indio_dev->name = id->name;
+ indio_dev->channels = bh1780_channels;
+ indio_dev->num_channels = ARRAY_SIZE(bh1780_channels);
+ indio_dev->modes = INDIO_DIRECT_MODE;
+
+ ret = iio_device_register(indio_dev);
+ if (ret)
+ goto out_disable_pm;
+ return 0;
+
+out_disable_pm:
+ pm_runtime_put_noidle(&client->dev);
+ pm_runtime_disable(&client->dev);
+ return ret;
+}
+
+static int bh1780_remove(struct i2c_client *client)
+{
+ struct iio_dev *indio_dev = i2c_get_clientdata(client);
+ struct bh1780_data *bh1780 = iio_priv(indio_dev);
+ int ret;
+
+ iio_device_unregister(indio_dev);
+ pm_runtime_get_sync(&client->dev);
+ pm_runtime_put_noidle(&client->dev);
+ pm_runtime_disable(&client->dev);
+ ret = bh1780_write(bh1780, BH1780_REG_CONTROL, BH1780_POFF);
+ if (ret < 0) {
+ dev_err(&client->dev, "failed to power off\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+#ifdef CONFIG_PM
+static int bh1780_runtime_suspend(struct device *dev)
+{
+ struct i2c_client *client = to_i2c_client(dev);
+ struct bh1780_data *bh1780 = i2c_get_clientdata(client);
+ int ret;
+
+ ret = bh1780_write(bh1780, BH1780_REG_CONTROL, BH1780_POFF);
+ if (ret < 0) {
+ dev_err(dev, "failed to runtime suspend\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+static int bh1780_runtime_resume(struct device *dev)
+{
+ struct i2c_client *client = to_i2c_client(dev);
+ struct bh1780_data *bh1780 = i2c_get_clientdata(client);
+ int ret;
+
+ ret = bh1780_write(bh1780, BH1780_REG_CONTROL, BH1780_PON);
+ if (ret < 0) {
+ dev_err(dev, "failed to runtime resume\n");
+ return ret;
+ }
+
+ /* Wait for power on, then for a value to be available */
+ msleep(BH1780_PON_DELAY + BH1780_INTERVAL);
+
+ return 0;
+}
+#endif /* CONFIG_PM */
+
+static const struct dev_pm_ops bh1780_dev_pm_ops = {
+ SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend,
+ pm_runtime_force_resume)
+ SET_RUNTIME_PM_OPS(bh1780_runtime_suspend,
+ bh1780_runtime_resume, NULL)
+};
+
+static const struct i2c_device_id bh1780_id[] = {
+ { "bh1780", 0 },
+ { },
+};
+
+MODULE_DEVICE_TABLE(i2c, bh1780_id);
+
+#ifdef CONFIG_OF
+static const struct of_device_id of_bh1780_match[] = {
+ { .compatible = "rohm,bh1780gli", },
+ {},
+};
+MODULE_DEVICE_TABLE(of, of_bh1780_match);
+#endif
+
+static struct i2c_driver bh1780_driver = {
+ .probe = bh1780_probe,
+ .remove = bh1780_remove,
+ .id_table = bh1780_id,
+ .driver = {
+ .name = "bh1780",
+ .pm = &bh1780_dev_pm_ops,
+ .of_match_table = of_match_ptr(of_bh1780_match),
+ },
+};
+
+module_i2c_driver(bh1780_driver);
+
+MODULE_DESCRIPTION("ROHM BH1780GLI Ambient Light Sensor Driver");
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Linus Walleij <linus.walleij@linaro.org>");
diff --git a/drivers/iio/light/max44000.c b/drivers/iio/light/max44000.c
new file mode 100644
index 0000000..e01e58a
--- /dev/null
+++ b/drivers/iio/light/max44000.c
@@ -0,0 +1,639 @@
+/*
+ * MAX44000 Ambient and Infrared Proximity Sensor
+ *
+ * Copyright (c) 2016, Intel Corporation.
+ *
+ * This file is subject to the terms and conditions of version 2 of
+ * the GNU General Public License. See the file COPYING in the main
+ * directory of this archive for more details.
+ *
+ * Data sheet: https://datasheets.maximintegrated.com/en/ds/MAX44000.pdf
+ *
+ * 7-bit I2C slave address 0x4a
+ */
+
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/i2c.h>
+#include <linux/regmap.h>
+#include <linux/util_macros.h>
+#include <linux/iio/iio.h>
+#include <linux/iio/sysfs.h>
+#include <linux/iio/buffer.h>
+#include <linux/iio/trigger_consumer.h>
+#include <linux/iio/triggered_buffer.h>
+#include <linux/acpi.h>
+
+#define MAX44000_DRV_NAME "max44000"
+
+/* Registers in datasheet order */
+#define MAX44000_REG_STATUS 0x00
+#define MAX44000_REG_CFG_MAIN 0x01
+#define MAX44000_REG_CFG_RX 0x02
+#define MAX44000_REG_CFG_TX 0x03
+#define MAX44000_REG_ALS_DATA_HI 0x04
+#define MAX44000_REG_ALS_DATA_LO 0x05
+#define MAX44000_REG_PRX_DATA 0x16
+#define MAX44000_REG_ALS_UPTHR_HI 0x06
+#define MAX44000_REG_ALS_UPTHR_LO 0x07
+#define MAX44000_REG_ALS_LOTHR_HI 0x08
+#define MAX44000_REG_ALS_LOTHR_LO 0x09
+#define MAX44000_REG_PST 0x0a
+#define MAX44000_REG_PRX_IND 0x0b
+#define MAX44000_REG_PRX_THR 0x0c
+#define MAX44000_REG_TRIM_GAIN_GREEN 0x0f
+#define MAX44000_REG_TRIM_GAIN_IR 0x10
+
+/* REG_CFG bits */
+#define MAX44000_CFG_ALSINTE 0x01
+#define MAX44000_CFG_PRXINTE 0x02
+#define MAX44000_CFG_MASK 0x1c
+#define MAX44000_CFG_MODE_SHUTDOWN 0x00
+#define MAX44000_CFG_MODE_ALS_GIR 0x04
+#define MAX44000_CFG_MODE_ALS_G 0x08
+#define MAX44000_CFG_MODE_ALS_IR 0x0c
+#define MAX44000_CFG_MODE_ALS_PRX 0x10
+#define MAX44000_CFG_MODE_PRX 0x14
+#define MAX44000_CFG_TRIM 0x20
+
+/*
+ * Upper 4 bits are not documented but start as 1 on powerup
+ * Setting them to 0 causes proximity to misbehave so set them to 1
+ */
+#define MAX44000_REG_CFG_RX_DEFAULT 0xf0
+
+/* REG_RX bits */
+#define MAX44000_CFG_RX_ALSTIM_MASK 0x0c
+#define MAX44000_CFG_RX_ALSTIM_SHIFT 2
+#define MAX44000_CFG_RX_ALSPGA_MASK 0x03
+#define MAX44000_CFG_RX_ALSPGA_SHIFT 0
+
+/* REG_TX bits */
+#define MAX44000_LED_CURRENT_MASK 0xf
+#define MAX44000_LED_CURRENT_MAX 11
+#define MAX44000_LED_CURRENT_DEFAULT 6
+
+#define MAX44000_ALSDATA_OVERFLOW 0x4000
+
+struct max44000_data {
+ struct mutex lock;
+ struct regmap *regmap;
+};
+
+/* Default scale is set to the minimum of 0.03125 or 1 / (1 << 5) lux */
+#define MAX44000_ALS_TO_LUX_DEFAULT_FRACTION_LOG2 5
+
+/* Scale can be multiplied by up to 128x via ALSPGA for measurement gain */
+static const int max44000_alspga_shift[] = {0, 2, 4, 7};
+#define MAX44000_ALSPGA_MAX_SHIFT 7
+
+/*
+ * Scale can be multiplied by up to 64x via ALSTIM because of lost resolution
+ *
+ * This scaling factor is hidden from userspace and instead accounted for when
+ * reading raw values from the device.
+ *
+ * This makes it possible to cleanly expose ALSPGA as IIO_CHAN_INFO_SCALE and
+ * ALSTIM as IIO_CHAN_INFO_INT_TIME without the values affecting each other.
+ *
+ * Handling this internally is also required for buffer support because the
+ * channel's scan_type can't be modified dynamically.
+ */
+static const int max44000_alstim_shift[] = {0, 2, 4, 6};
+#define MAX44000_ALSTIM_SHIFT(alstim) (2 * (alstim))
+
+/* Available integration times with pretty manual alignment: */
+static const int max44000_int_time_avail_ns_array[] = {
+ 100000000,
+ 25000000,
+ 6250000,
+ 1562500,
+};
+static const char max44000_int_time_avail_str[] =
+ "0.100 "
+ "0.025 "
+ "0.00625 "
+ "0.001625";
+
+/* Available scales (internal to ulux) with pretty manual alignment: */
+static const int max44000_scale_avail_ulux_array[] = {
+ 31250,
+ 125000,
+ 500000,
+ 4000000,
+};
+static const char max44000_scale_avail_str[] =
+ "0.03125 "
+ "0.125 "
+ "0.5 "
+ "4";
+
+#define MAX44000_SCAN_INDEX_ALS 0
+#define MAX44000_SCAN_INDEX_PRX 1
+
+static const struct iio_chan_spec max44000_channels[] = {
+ {
+ .type = IIO_LIGHT,
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW),
+ .info_mask_shared_by_type = BIT(IIO_CHAN_INFO_SCALE) |
+ BIT(IIO_CHAN_INFO_INT_TIME),
+ .scan_index = MAX44000_SCAN_INDEX_ALS,
+ .scan_type = {
+ .sign = 'u',
+ .realbits = 14,
+ .storagebits = 16,
+ }
+ },
+ {
+ .type = IIO_PROXIMITY,
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW),
+ .info_mask_shared_by_type = BIT(IIO_CHAN_INFO_SCALE),
+ .scan_index = MAX44000_SCAN_INDEX_PRX,
+ .scan_type = {
+ .sign = 'u',
+ .realbits = 8,
+ .storagebits = 16,
+ }
+ },
+ IIO_CHAN_SOFT_TIMESTAMP(2),
+ {
+ .type = IIO_CURRENT,
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) |
+ BIT(IIO_CHAN_INFO_SCALE),
+ .extend_name = "led",
+ .output = 1,
+ .scan_index = -1,
+ },
+};
+
+static int max44000_read_alstim(struct max44000_data *data)
+{
+ unsigned int val;
+ int ret;
+
+ ret = regmap_read(data->regmap, MAX44000_REG_CFG_RX, &val);
+ if (ret < 0)
+ return ret;
+ return (val & MAX44000_CFG_RX_ALSTIM_MASK) >> MAX44000_CFG_RX_ALSTIM_SHIFT;
+}
+
+static int max44000_write_alstim(struct max44000_data *data, int val)
+{
+ return regmap_write_bits(data->regmap, MAX44000_REG_CFG_RX,
+ MAX44000_CFG_RX_ALSTIM_MASK,
+ val << MAX44000_CFG_RX_ALSTIM_SHIFT);
+}
+
+static int max44000_read_alspga(struct max44000_data *data)
+{
+ unsigned int val;
+ int ret;
+
+ ret = regmap_read(data->regmap, MAX44000_REG_CFG_RX, &val);
+ if (ret < 0)
+ return ret;
+ return (val & MAX44000_CFG_RX_ALSPGA_MASK) >> MAX44000_CFG_RX_ALSPGA_SHIFT;
+}
+
+static int max44000_write_alspga(struct max44000_data *data, int val)
+{
+ return regmap_write_bits(data->regmap, MAX44000_REG_CFG_RX,
+ MAX44000_CFG_RX_ALSPGA_MASK,
+ val << MAX44000_CFG_RX_ALSPGA_SHIFT);
+}
+
+static int max44000_read_alsval(struct max44000_data *data)
+{
+ u16 regval;
+ int alstim, ret;
+
+ ret = regmap_bulk_read(data->regmap, MAX44000_REG_ALS_DATA_HI,
+ ®val, sizeof(regval));
+ if (ret < 0)
+ return ret;
+ alstim = ret = max44000_read_alstim(data);
+ if (ret < 0)
+ return ret;
+
+ regval = be16_to_cpu(regval);
+
+ /*
+ * Overflow is explained on datasheet page 17.
+ *
+ * It's a warning that either the G or IR channel has become saturated
+ * and that the value in the register is likely incorrect.
+ *
+ * The recommendation is to change the scale (ALSPGA).
+ * The driver just returns the max representable value.
+ */
+ if (regval & MAX44000_ALSDATA_OVERFLOW)
+ return 0x3FFF;
+
+ return regval << MAX44000_ALSTIM_SHIFT(alstim);
+}
+
+static int max44000_write_led_current_raw(struct max44000_data *data, int val)
+{
+ /* Maybe we should clamp the value instead? */
+ if (val < 0 || val > MAX44000_LED_CURRENT_MAX)
+ return -ERANGE;
+ if (val >= 8)
+ val += 4;
+ return regmap_write_bits(data->regmap, MAX44000_REG_CFG_TX,
+ MAX44000_LED_CURRENT_MASK, val);
+}
+
+static int max44000_read_led_current_raw(struct max44000_data *data)
+{
+ unsigned int regval;
+ int ret;
+
+ ret = regmap_read(data->regmap, MAX44000_REG_CFG_TX, ®val);
+ if (ret < 0)
+ return ret;
+ regval &= MAX44000_LED_CURRENT_MASK;
+ if (regval >= 8)
+ regval -= 4;
+ return regval;
+}
+
+static int max44000_read_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int *val, int *val2, long mask)
+{
+ struct max44000_data *data = iio_priv(indio_dev);
+ int alstim, alspga;
+ unsigned int regval;
+ int ret;
+
+ switch (mask) {
+ case IIO_CHAN_INFO_RAW:
+ switch (chan->type) {
+ case IIO_LIGHT:
+ mutex_lock(&data->lock);
+ ret = max44000_read_alsval(data);
+ mutex_unlock(&data->lock);
+ if (ret < 0)
+ return ret;
+ *val = ret;
+ return IIO_VAL_INT;
+
+ case IIO_PROXIMITY:
+ mutex_lock(&data->lock);
+ ret = regmap_read(data->regmap, MAX44000_REG_PRX_DATA, ®val);
+ mutex_unlock(&data->lock);
+ if (ret < 0)
+ return ret;
+ *val = regval;
+ return IIO_VAL_INT;
+
+ case IIO_CURRENT:
+ mutex_lock(&data->lock);
+ ret = max44000_read_led_current_raw(data);
+ mutex_unlock(&data->lock);
+ if (ret < 0)
+ return ret;
+ *val = ret;
+ return IIO_VAL_INT;
+
+ default:
+ return -EINVAL;
+ }
+
+ case IIO_CHAN_INFO_SCALE:
+ switch (chan->type) {
+ case IIO_CURRENT:
+ /* Output register is in 10s of miliamps */
+ *val = 10;
+ return IIO_VAL_INT;
+
+ case IIO_LIGHT:
+ mutex_lock(&data->lock);
+ alspga = ret = max44000_read_alspga(data);
+ mutex_unlock(&data->lock);
+ if (ret < 0)
+ return ret;
+
+ /* Avoid negative shifts */
+ *val = (1 << MAX44000_ALSPGA_MAX_SHIFT);
+ *val2 = MAX44000_ALS_TO_LUX_DEFAULT_FRACTION_LOG2
+ + MAX44000_ALSPGA_MAX_SHIFT
+ - max44000_alspga_shift[alspga];
+ return IIO_VAL_FRACTIONAL_LOG2;
+
+ default:
+ return -EINVAL;
+ }
+
+ case IIO_CHAN_INFO_INT_TIME:
+ mutex_lock(&data->lock);
+ alstim = ret = max44000_read_alstim(data);
+ mutex_unlock(&data->lock);
+
+ if (ret < 0)
+ return ret;
+ *val = 0;
+ *val2 = max44000_int_time_avail_ns_array[alstim];
+ return IIO_VAL_INT_PLUS_NANO;
+
+ default:
+ return -EINVAL;
+ }
+}
+
+static int max44000_write_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int val, int val2, long mask)
+{
+ struct max44000_data *data = iio_priv(indio_dev);
+ int ret;
+
+ if (mask == IIO_CHAN_INFO_RAW && chan->type == IIO_CURRENT) {
+ mutex_lock(&data->lock);
+ ret = max44000_write_led_current_raw(data, val);
+ mutex_unlock(&data->lock);
+ return ret;
+ } else if (mask == IIO_CHAN_INFO_INT_TIME && chan->type == IIO_LIGHT) {
+ s64 valns = val * NSEC_PER_SEC + val2;
+ int alstim = find_closest_descending(valns,
+ max44000_int_time_avail_ns_array,
+ ARRAY_SIZE(max44000_int_time_avail_ns_array));
+ mutex_lock(&data->lock);
+ ret = max44000_write_alstim(data, alstim);
+ mutex_unlock(&data->lock);
+ return ret;
+ } else if (mask == IIO_CHAN_INFO_SCALE && chan->type == IIO_LIGHT) {
+ s64 valus = val * USEC_PER_SEC + val2;
+ int alspga = find_closest(valus,
+ max44000_scale_avail_ulux_array,
+ ARRAY_SIZE(max44000_scale_avail_ulux_array));
+ mutex_lock(&data->lock);
+ ret = max44000_write_alspga(data, alspga);
+ mutex_unlock(&data->lock);
+ return ret;
+ }
+
+ return -EINVAL;
+}
+
+static int max44000_write_raw_get_fmt(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ long mask)
+{
+ if (mask == IIO_CHAN_INFO_INT_TIME && chan->type == IIO_LIGHT)
+ return IIO_VAL_INT_PLUS_NANO;
+ else if (mask == IIO_CHAN_INFO_SCALE && chan->type == IIO_LIGHT)
+ return IIO_VAL_INT_PLUS_MICRO;
+ else
+ return IIO_VAL_INT;
+}
+
+static IIO_CONST_ATTR(illuminance_integration_time_available, max44000_int_time_avail_str);
+static IIO_CONST_ATTR(illuminance_scale_available, max44000_scale_avail_str);
+
+static struct attribute *max44000_attributes[] = {
+ &iio_const_attr_illuminance_integration_time_available.dev_attr.attr,
+ &iio_const_attr_illuminance_scale_available.dev_attr.attr,
+ NULL
+};
+
+static const struct attribute_group max44000_attribute_group = {
+ .attrs = max44000_attributes,
+};
+
+static const struct iio_info max44000_info = {
+ .driver_module = THIS_MODULE,
+ .read_raw = max44000_read_raw,
+ .write_raw = max44000_write_raw,
+ .write_raw_get_fmt = max44000_write_raw_get_fmt,
+ .attrs = &max44000_attribute_group,
+};
+
+static bool max44000_readable_reg(struct device *dev, unsigned int reg)
+{
+ switch (reg) {
+ case MAX44000_REG_STATUS:
+ case MAX44000_REG_CFG_MAIN:
+ case MAX44000_REG_CFG_RX:
+ case MAX44000_REG_CFG_TX:
+ case MAX44000_REG_ALS_DATA_HI:
+ case MAX44000_REG_ALS_DATA_LO:
+ case MAX44000_REG_PRX_DATA:
+ case MAX44000_REG_ALS_UPTHR_HI:
+ case MAX44000_REG_ALS_UPTHR_LO:
+ case MAX44000_REG_ALS_LOTHR_HI:
+ case MAX44000_REG_ALS_LOTHR_LO:
+ case MAX44000_REG_PST:
+ case MAX44000_REG_PRX_IND:
+ case MAX44000_REG_PRX_THR:
+ case MAX44000_REG_TRIM_GAIN_GREEN:
+ case MAX44000_REG_TRIM_GAIN_IR:
+ return true;
+ default:
+ return false;
+ }
+}
+
+static bool max44000_writeable_reg(struct device *dev, unsigned int reg)
+{
+ switch (reg) {
+ case MAX44000_REG_CFG_MAIN:
+ case MAX44000_REG_CFG_RX:
+ case MAX44000_REG_CFG_TX:
+ case MAX44000_REG_ALS_UPTHR_HI:
+ case MAX44000_REG_ALS_UPTHR_LO:
+ case MAX44000_REG_ALS_LOTHR_HI:
+ case MAX44000_REG_ALS_LOTHR_LO:
+ case MAX44000_REG_PST:
+ case MAX44000_REG_PRX_IND:
+ case MAX44000_REG_PRX_THR:
+ case MAX44000_REG_TRIM_GAIN_GREEN:
+ case MAX44000_REG_TRIM_GAIN_IR:
+ return true;
+ default:
+ return false;
+ }
+}
+
+static bool max44000_volatile_reg(struct device *dev, unsigned int reg)
+{
+ switch (reg) {
+ case MAX44000_REG_STATUS:
+ case MAX44000_REG_ALS_DATA_HI:
+ case MAX44000_REG_ALS_DATA_LO:
+ case MAX44000_REG_PRX_DATA:
+ return true;
+ default:
+ return false;
+ }
+}
+
+static bool max44000_precious_reg(struct device *dev, unsigned int reg)
+{
+ return reg == MAX44000_REG_STATUS;
+}
+
+static const struct regmap_config max44000_regmap_config = {
+ .reg_bits = 8,
+ .val_bits = 8,
+
+ .max_register = MAX44000_REG_PRX_DATA,
+ .readable_reg = max44000_readable_reg,
+ .writeable_reg = max44000_writeable_reg,
+ .volatile_reg = max44000_volatile_reg,
+ .precious_reg = max44000_precious_reg,
+
+ .use_single_rw = 1,
+ .cache_type = REGCACHE_RBTREE,
+};
+
+static irqreturn_t max44000_trigger_handler(int irq, void *p)
+{
+ struct iio_poll_func *pf = p;
+ struct iio_dev *indio_dev = pf->indio_dev;
+ struct max44000_data *data = iio_priv(indio_dev);
+ u16 buf[8]; /* 2x u16 + padding + 8 bytes timestamp */
+ int index = 0;
+ unsigned int regval;
+ int ret;
+
+ mutex_lock(&data->lock);
+ if (test_bit(MAX44000_SCAN_INDEX_ALS, indio_dev->active_scan_mask)) {
+ ret = max44000_read_alsval(data);
+ if (ret < 0)
+ goto out_unlock;
+ buf[index++] = ret;
+ }
+ if (test_bit(MAX44000_SCAN_INDEX_PRX, indio_dev->active_scan_mask)) {
+ ret = regmap_read(data->regmap, MAX44000_REG_PRX_DATA, ®val);
+ if (ret < 0)
+ goto out_unlock;
+ buf[index] = regval;
+ }
+ mutex_unlock(&data->lock);
+
+ iio_push_to_buffers_with_timestamp(indio_dev, buf, iio_get_time_ns());
+ iio_trigger_notify_done(indio_dev->trig);
+ return IRQ_HANDLED;
+
+out_unlock:
+ mutex_unlock(&data->lock);
+ iio_trigger_notify_done(indio_dev->trig);
+ return IRQ_HANDLED;
+}
+
+static int max44000_probe(struct i2c_client *client,
+ const struct i2c_device_id *id)
+{
+ struct max44000_data *data;
+ struct iio_dev *indio_dev;
+ int ret, reg;
+
+ indio_dev = devm_iio_device_alloc(&client->dev, sizeof(*data));
+ if (!indio_dev)
+ return -ENOMEM;
+ data = iio_priv(indio_dev);
+ data->regmap = devm_regmap_init_i2c(client, &max44000_regmap_config);
+ if (IS_ERR(data->regmap)) {
+ dev_err(&client->dev, "regmap_init failed!\n");
+ return PTR_ERR(data->regmap);
+ }
+
+ i2c_set_clientdata(client, indio_dev);
+ mutex_init(&data->lock);
+ indio_dev->dev.parent = &client->dev;
+ indio_dev->info = &max44000_info;
+ indio_dev->name = MAX44000_DRV_NAME;
+ indio_dev->channels = max44000_channels;
+ indio_dev->num_channels = ARRAY_SIZE(max44000_channels);
+
+ /*
+ * The device doesn't have a reset function so we just clear some
+ * important bits at probe time to ensure sane operation.
+ *
+ * Since we don't support interrupts/events the threshold values are
+ * not important. We also don't touch trim values.
+ */
+
+ /* Reset ALS scaling bits */
+ ret = regmap_write(data->regmap, MAX44000_REG_CFG_RX,
+ MAX44000_REG_CFG_RX_DEFAULT);
+ if (ret < 0) {
+ dev_err(&client->dev, "failed to write default CFG_RX: %d\n",
+ ret);
+ return ret;
+ }
+
+ /*
+ * By default the LED pulse used for the proximity sensor is disabled.
+ * Set a middle value so that we get some sort of valid data by default.
+ */
+ ret = max44000_write_led_current_raw(data, MAX44000_LED_CURRENT_DEFAULT);
+ if (ret < 0) {
+ dev_err(&client->dev, "failed to write init config: %d\n", ret);
+ return ret;
+ }
+
+ /* Reset CFG bits to ALS_PRX mode which allows easy reading of both values. */
+ reg = MAX44000_CFG_TRIM | MAX44000_CFG_MODE_ALS_PRX;
+ ret = regmap_write(data->regmap, MAX44000_REG_CFG_MAIN, reg);
+ if (ret < 0) {
+ dev_err(&client->dev, "failed to write init config: %d\n", ret);
+ return ret;
+ }
+
+ /* Read status at least once to clear any stale interrupt bits. */
+ ret = regmap_read(data->regmap, MAX44000_REG_STATUS, ®);
+ if (ret < 0) {
+ dev_err(&client->dev, "failed to read init status: %d\n", ret);
+ return ret;
+ }
+
+ ret = iio_triggered_buffer_setup(indio_dev, NULL, max44000_trigger_handler, NULL);
+ if (ret < 0) {
+ dev_err(&client->dev, "iio triggered buffer setup failed\n");
+ return ret;
+ }
+
+ return iio_device_register(indio_dev);
+}
+
+static int max44000_remove(struct i2c_client *client)
+{
+ struct iio_dev *indio_dev = i2c_get_clientdata(client);
+
+ iio_device_unregister(indio_dev);
+ iio_triggered_buffer_cleanup(indio_dev);
+
+ return 0;
+}
+
+static const struct i2c_device_id max44000_id[] = {
+ {"max44000", 0},
+ { }
+};
+MODULE_DEVICE_TABLE(i2c, max44000_id);
+
+#ifdef CONFIG_ACPI
+static const struct acpi_device_id max44000_acpi_match[] = {
+ {"MAX44000", 0},
+ { }
+};
+MODULE_DEVICE_TABLE(acpi, max44000_acpi_match);
+#endif
+
+static struct i2c_driver max44000_driver = {
+ .driver = {
+ .name = MAX44000_DRV_NAME,
+ .acpi_match_table = ACPI_PTR(max44000_acpi_match),
+ },
+ .probe = max44000_probe,
+ .remove = max44000_remove,
+ .id_table = max44000_id,
+};
+
+module_i2c_driver(max44000_driver);
+
+MODULE_AUTHOR("Crestez Dan Leonard <leonard.crestez@intel.com>");
+MODULE_DESCRIPTION("MAX44000 Ambient and Infrared Proximity Sensor");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/iio/light/stk3310.c b/drivers/iio/light/stk3310.c
index 42d334b..9e847f8 100644
--- a/drivers/iio/light/stk3310.c
+++ b/drivers/iio/light/stk3310.c
@@ -16,7 +16,6 @@
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/regmap.h>
-#include <linux/gpio/consumer.h>
#include <linux/iio/events.h>
#include <linux/iio/iio.h>
#include <linux/iio/sysfs.h>
diff --git a/drivers/iio/light/tsl2563.c b/drivers/iio/light/tsl2563.c
index 12731d6..57b108c 100644
--- a/drivers/iio/light/tsl2563.c
+++ b/drivers/iio/light/tsl2563.c
@@ -806,8 +806,7 @@
return 0;
fail:
- cancel_delayed_work(&chip->poweroff_work);
- flush_scheduled_work();
+ cancel_delayed_work_sync(&chip->poweroff_work);
return err;
}
diff --git a/drivers/iio/light/veml6070.c b/drivers/iio/light/veml6070.c
new file mode 100644
index 0000000..bc1c4cb
--- /dev/null
+++ b/drivers/iio/light/veml6070.c
@@ -0,0 +1,218 @@
+/*
+ * veml6070.c - Support for Vishay VEML6070 UV A light sensor
+ *
+ * Copyright 2016 Peter Meerwald-Stadler <pmeerw@pmeerw.net>
+ *
+ * This file is subject to the terms and conditions of version 2 of
+ * the GNU General Public License. See the file COPYING in the main
+ * directory of this archive for more details.
+ *
+ * IIO driver for VEML6070 (7-bit I2C slave addresses 0x38 and 0x39)
+ *
+ * TODO: integration time, ACK signal
+ */
+
+#include <linux/module.h>
+#include <linux/i2c.h>
+#include <linux/mutex.h>
+#include <linux/err.h>
+#include <linux/delay.h>
+
+#include <linux/iio/iio.h>
+#include <linux/iio/sysfs.h>
+
+#define VEML6070_DRV_NAME "veml6070"
+
+#define VEML6070_ADDR_CONFIG_DATA_MSB 0x38 /* read: MSB data, write: config */
+#define VEML6070_ADDR_DATA_LSB 0x39 /* LSB data */
+
+#define VEML6070_COMMAND_ACK BIT(5) /* raise interrupt when over threshold */
+#define VEML6070_COMMAND_IT GENMASK(3, 2) /* bit mask integration time */
+#define VEML6070_COMMAND_RSRVD BIT(1) /* reserved, set to 1 */
+#define VEML6070_COMMAND_SD BIT(0) /* shutdown mode when set */
+
+#define VEML6070_IT_10 0x04 /* integration time 1x */
+
+struct veml6070_data {
+ struct i2c_client *client1;
+ struct i2c_client *client2;
+ u8 config;
+ struct mutex lock;
+};
+
+static int veml6070_read(struct veml6070_data *data)
+{
+ int ret;
+ u8 msb, lsb;
+
+ mutex_lock(&data->lock);
+
+ /* disable shutdown */
+ ret = i2c_smbus_write_byte(data->client1,
+ data->config & ~VEML6070_COMMAND_SD);
+ if (ret < 0)
+ goto out;
+
+ msleep(125 + 10); /* measurement takes up to 125 ms for IT 1x */
+
+ ret = i2c_smbus_read_byte(data->client2); /* read MSB, address 0x39 */
+ if (ret < 0)
+ goto out;
+ msb = ret;
+
+ ret = i2c_smbus_read_byte(data->client1); /* read LSB, address 0x38 */
+ if (ret < 0)
+ goto out;
+ lsb = ret;
+
+ /* shutdown again */
+ ret = i2c_smbus_write_byte(data->client1, data->config);
+ if (ret < 0)
+ goto out;
+
+ ret = (msb << 8) | lsb;
+
+out:
+ mutex_unlock(&data->lock);
+ return ret;
+}
+
+static const struct iio_chan_spec veml6070_channels[] = {
+ {
+ .type = IIO_INTENSITY,
+ .modified = 1,
+ .channel2 = IIO_MOD_LIGHT_UV,
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW),
+ },
+ {
+ .type = IIO_UVINDEX,
+ .info_mask_separate = BIT(IIO_CHAN_INFO_PROCESSED),
+ }
+};
+
+static int veml6070_to_uv_index(unsigned val)
+{
+ /*
+ * conversion of raw UV intensity values to UV index depends on
+ * integration time (IT) and value of the resistor connected to
+ * the RSET pin (default: 270 KOhm)
+ */
+ unsigned uvi[11] = {
+ 187, 373, 560, /* low */
+ 746, 933, 1120, /* moderate */
+ 1308, 1494, /* high */
+ 1681, 1868, 2054}; /* very high */
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(uvi); i++)
+ if (val <= uvi[i])
+ return i;
+
+ return 11; /* extreme */
+}
+
+static int veml6070_read_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int *val, int *val2, long mask)
+{
+ struct veml6070_data *data = iio_priv(indio_dev);
+ int ret;
+
+ switch (mask) {
+ case IIO_CHAN_INFO_RAW:
+ case IIO_CHAN_INFO_PROCESSED:
+ ret = veml6070_read(data);
+ if (ret < 0)
+ return ret;
+ if (mask == IIO_CHAN_INFO_PROCESSED)
+ *val = veml6070_to_uv_index(ret);
+ else
+ *val = ret;
+ return IIO_VAL_INT;
+ default:
+ return -EINVAL;
+ }
+}
+
+static const struct iio_info veml6070_info = {
+ .read_raw = veml6070_read_raw,
+ .driver_module = THIS_MODULE,
+};
+
+static int veml6070_probe(struct i2c_client *client,
+ const struct i2c_device_id *id)
+{
+ struct veml6070_data *data;
+ struct iio_dev *indio_dev;
+ int ret;
+
+ indio_dev = devm_iio_device_alloc(&client->dev, sizeof(*data));
+ if (!indio_dev)
+ return -ENOMEM;
+
+ data = iio_priv(indio_dev);
+ i2c_set_clientdata(client, indio_dev);
+ data->client1 = client;
+ mutex_init(&data->lock);
+
+ indio_dev->dev.parent = &client->dev;
+ indio_dev->info = &veml6070_info;
+ indio_dev->channels = veml6070_channels;
+ indio_dev->num_channels = ARRAY_SIZE(veml6070_channels);
+ indio_dev->name = VEML6070_DRV_NAME;
+ indio_dev->modes = INDIO_DIRECT_MODE;
+
+ data->client2 = i2c_new_dummy(client->adapter, VEML6070_ADDR_DATA_LSB);
+ if (!data->client2) {
+ dev_err(&client->dev, "i2c device for second chip address failed\n");
+ return -ENODEV;
+ }
+
+ data->config = VEML6070_IT_10 | VEML6070_COMMAND_RSRVD |
+ VEML6070_COMMAND_SD;
+ ret = i2c_smbus_write_byte(data->client1, data->config);
+ if (ret < 0)
+ goto fail;
+
+ ret = iio_device_register(indio_dev);
+ if (ret < 0)
+ goto fail;
+
+ return ret;
+
+fail:
+ i2c_unregister_device(data->client2);
+ return ret;
+}
+
+static int veml6070_remove(struct i2c_client *client)
+{
+ struct iio_dev *indio_dev = i2c_get_clientdata(client);
+ struct veml6070_data *data = iio_priv(indio_dev);
+
+ iio_device_unregister(indio_dev);
+ i2c_unregister_device(data->client2);
+
+ return 0;
+}
+
+static const struct i2c_device_id veml6070_id[] = {
+ { "veml6070", 0 },
+ { }
+};
+MODULE_DEVICE_TABLE(i2c, veml6070_id);
+
+static struct i2c_driver veml6070_driver = {
+ .driver = {
+ .name = VEML6070_DRV_NAME,
+ },
+ .probe = veml6070_probe,
+ .remove = veml6070_remove,
+ .id_table = veml6070_id,
+};
+
+module_i2c_driver(veml6070_driver);
+
+MODULE_AUTHOR("Peter Meerwald-Stadler <pmeerw@pmeerw.net>");
+MODULE_DESCRIPTION("Vishay VEML6070 UV A light sensor driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/iio/magnetometer/Kconfig b/drivers/iio/magnetometer/Kconfig
index 021dc53..84e6559 100644
--- a/drivers/iio/magnetometer/Kconfig
+++ b/drivers/iio/magnetometer/Kconfig
@@ -9,6 +9,8 @@
tristate "Asahi Kasei AK 3-Axis Magnetometer"
depends on I2C
depends on GPIOLIB || COMPILE_TEST
+ select IIO_BUFFER
+ select IIO_TRIGGERED_BUFFER
help
Say yes here to build support for Asahi Kasei AK8975, AK8963,
AK09911 or AK09912 3-Axis Magnetometer.
@@ -25,22 +27,41 @@
Deprecated: AK09911 is now supported by AK8975 driver.
config BMC150_MAGN
- tristate "Bosch BMC150 Magnetometer Driver"
- depends on I2C
- select REGMAP_I2C
+ tristate
select IIO_BUFFER
select IIO_TRIGGERED_BUFFER
- help
- Say yes here to build support for the BMC150 magnetometer.
- Currently this only supports the device via an i2c interface.
+config BMC150_MAGN_I2C
+ tristate "Bosch BMC150 I2C Magnetometer Driver"
+ depends on I2C
+ select BMC150_MAGN
+ select REGMAP_I2C
+ help
+ Say yes here to build support for the BMC150 magnetometer with
+ I2C interface.
This is a combo module with both accelerometer and magnetometer.
This driver is only implementing magnetometer part, which has
its own address and register map.
To compile this driver as a module, choose M here: the module will be
- called bmc150_magn.
+ called bmc150_magn_i2c.
+
+config BMC150_MAGN_SPI
+ tristate "Bosch BMC150 SPI Magnetometer Driver"
+ depends on SPI
+ select BMC150_MAGN
+ select REGMAP_SPI
+ help
+ Say yes here to build support for the BMC150 magnetometer with
+ SPI interface.
+
+ This is a combo module with both accelerometer and magnetometer.
+ This driver is only implementing magnetometer part, which has
+ its own address and register map.
+
+ To compile this driver as a module, choose M here: the module will be
+ called bmc150_magn_spi.
config MAG3110
tristate "Freescale MAG3110 3-Axis Magnetometer"
diff --git a/drivers/iio/magnetometer/Makefile b/drivers/iio/magnetometer/Makefile
index dd03fe5..92a745c 100644
--- a/drivers/iio/magnetometer/Makefile
+++ b/drivers/iio/magnetometer/Makefile
@@ -5,6 +5,9 @@
# When adding new entries keep the list in alphabetical order
obj-$(CONFIG_AK8975) += ak8975.o
obj-$(CONFIG_BMC150_MAGN) += bmc150_magn.o
+obj-$(CONFIG_BMC150_MAGN_I2C) += bmc150_magn_i2c.o
+obj-$(CONFIG_BMC150_MAGN_SPI) += bmc150_magn_spi.o
+
obj-$(CONFIG_MAG3110) += mag3110.o
obj-$(CONFIG_HID_SENSOR_MAGNETOMETER_3D) += hid-sensor-magn-3d.o
obj-$(CONFIG_MMC35240) += mmc35240.o
diff --git a/drivers/iio/magnetometer/ak8975.c b/drivers/iio/magnetometer/ak8975.c
index 0e931a9..609a2c4 100644
--- a/drivers/iio/magnetometer/ak8975.c
+++ b/drivers/iio/magnetometer/ak8975.c
@@ -32,9 +32,17 @@
#include <linux/gpio.h>
#include <linux/of_gpio.h>
#include <linux/acpi.h>
+#include <linux/regulator/consumer.h>
#include <linux/iio/iio.h>
#include <linux/iio/sysfs.h>
+#include <linux/iio/buffer.h>
+#include <linux/iio/trigger.h>
+#include <linux/iio/trigger_consumer.h>
+#include <linux/iio/triggered_buffer.h>
+
+#include <linux/iio/magnetometer/ak8975.h>
+
/*
* Register definitions, as well as various shifts and masks to get at the
* individual fields of the registers.
@@ -361,7 +369,6 @@
struct ak8975_data {
struct i2c_client *client;
const struct ak_def *def;
- struct attribute_group attrs;
struct mutex lock;
u8 asa[3];
long raw_to_gauss[3];
@@ -370,8 +377,41 @@
wait_queue_head_t data_ready_queue;
unsigned long flags;
u8 cntl_cache;
+ struct iio_mount_matrix orientation;
+ struct regulator *vdd;
};
+/* Enable attached power regulator if any. */
+static int ak8975_power_on(struct i2c_client *client)
+{
+ const struct iio_dev *indio_dev = i2c_get_clientdata(client);
+ struct ak8975_data *data = iio_priv(indio_dev);
+ int ret;
+
+ data->vdd = devm_regulator_get(&client->dev, "vdd");
+ if (IS_ERR_OR_NULL(data->vdd)) {
+ ret = PTR_ERR(data->vdd);
+ if (ret == -ENODEV)
+ ret = 0;
+ } else {
+ ret = regulator_enable(data->vdd);
+ }
+
+ if (ret)
+ dev_err(&client->dev, "failed to enable Vdd supply: %d\n", ret);
+ return ret;
+}
+
+/* Disable attached power regulator if any. */
+static void ak8975_power_off(const struct i2c_client *client)
+{
+ const struct iio_dev *indio_dev = i2c_get_clientdata(client);
+ const struct ak8975_data *data = iio_priv(indio_dev);
+
+ if (!IS_ERR_OR_NULL(data->vdd))
+ regulator_disable(data->vdd);
+}
+
/*
* Return 0 if the i2c device is the one we expect.
* return a negative error number otherwise
@@ -601,22 +641,15 @@
return ret > 0 ? 0 : -ETIME;
}
-/*
- * Emits the raw flux value for the x, y, or z axis.
- */
-static int ak8975_read_axis(struct iio_dev *indio_dev, int index, int *val)
+static int ak8975_start_read_axis(struct ak8975_data *data,
+ const struct i2c_client *client)
{
- struct ak8975_data *data = iio_priv(indio_dev);
- struct i2c_client *client = data->client;
- int ret;
-
- mutex_lock(&data->lock);
-
/* Set up the device for taking a sample. */
- ret = ak8975_set_mode(data, MODE_ONCE);
+ int ret = ak8975_set_mode(data, MODE_ONCE);
+
if (ret < 0) {
dev_err(&client->dev, "Error in setting operating mode\n");
- goto exit;
+ return ret;
}
/* Wait for the conversion to complete. */
@@ -627,7 +660,7 @@
else
ret = wait_conversion_complete_polled(data);
if (ret < 0)
- goto exit;
+ return ret;
/* This will be executed only for non-interrupt based waiting case */
if (ret & data->def->ctrl_masks[ST1_DRDY]) {
@@ -635,32 +668,45 @@
data->def->ctrl_regs[ST2]);
if (ret < 0) {
dev_err(&client->dev, "Error in reading ST2\n");
- goto exit;
+ return ret;
}
if (ret & (data->def->ctrl_masks[ST2_DERR] |
data->def->ctrl_masks[ST2_HOFL])) {
dev_err(&client->dev, "ST2 status error 0x%x\n", ret);
- ret = -EINVAL;
- goto exit;
+ return -EINVAL;
}
}
- /* Read the flux value from the appropriate register
- (the register is specified in the iio device attributes). */
- ret = i2c_smbus_read_word_data(client, data->def->data_regs[index]);
- if (ret < 0) {
- dev_err(&client->dev, "Read axis data fails\n");
+ return 0;
+}
+
+/* Retrieve raw flux value for one of the x, y, or z axis. */
+static int ak8975_read_axis(struct iio_dev *indio_dev, int index, int *val)
+{
+ struct ak8975_data *data = iio_priv(indio_dev);
+ const struct i2c_client *client = data->client;
+ const struct ak_def *def = data->def;
+ int ret;
+
+ mutex_lock(&data->lock);
+
+ ret = ak8975_start_read_axis(data, client);
+ if (ret)
goto exit;
- }
+
+ ret = i2c_smbus_read_word_data(client, def->data_regs[index]);
+ if (ret < 0)
+ goto exit;
mutex_unlock(&data->lock);
/* Clamp to valid range. */
- *val = clamp_t(s16, ret, -data->def->range, data->def->range);
+ *val = clamp_t(s16, ret, -def->range, def->range);
return IIO_VAL_INT;
exit:
mutex_unlock(&data->lock);
+ dev_err(&client->dev, "Error in reading axis\n");
return ret;
}
@@ -682,6 +728,18 @@
return -EINVAL;
}
+static const struct iio_mount_matrix *
+ak8975_get_mount_matrix(const struct iio_dev *indio_dev,
+ const struct iio_chan_spec *chan)
+{
+ return &((struct ak8975_data *)iio_priv(indio_dev))->orientation;
+}
+
+static const struct iio_chan_spec_ext_info ak8975_ext_info[] = {
+ IIO_MOUNT_MATRIX(IIO_SHARED_BY_DIR, ak8975_get_mount_matrix),
+ { },
+};
+
#define AK8975_CHANNEL(axis, index) \
{ \
.type = IIO_MAGN, \
@@ -690,12 +748,23 @@
.info_mask_separate = BIT(IIO_CHAN_INFO_RAW) | \
BIT(IIO_CHAN_INFO_SCALE), \
.address = index, \
+ .scan_index = index, \
+ .scan_type = { \
+ .sign = 's', \
+ .realbits = 16, \
+ .storagebits = 16, \
+ .endianness = IIO_CPU \
+ }, \
+ .ext_info = ak8975_ext_info, \
}
static const struct iio_chan_spec ak8975_channels[] = {
AK8975_CHANNEL(X, 0), AK8975_CHANNEL(Y, 1), AK8975_CHANNEL(Z, 2),
+ IIO_CHAN_SOFT_TIMESTAMP(3),
};
+static const unsigned long ak8975_scan_masks[] = { 0x7, 0 };
+
static const struct iio_info ak8975_info = {
.read_raw = &ak8975_read_raw,
.driver_module = THIS_MODULE,
@@ -724,6 +793,56 @@
return dev_name(dev);
}
+static void ak8975_fill_buffer(struct iio_dev *indio_dev)
+{
+ struct ak8975_data *data = iio_priv(indio_dev);
+ const struct i2c_client *client = data->client;
+ const struct ak_def *def = data->def;
+ int ret;
+ s16 buff[8]; /* 3 x 16 bits axis values + 1 aligned 64 bits timestamp */
+
+ mutex_lock(&data->lock);
+
+ ret = ak8975_start_read_axis(data, client);
+ if (ret)
+ goto unlock;
+
+ /*
+ * For each axis, read the flux value from the appropriate register
+ * (the register is specified in the iio device attributes).
+ */
+ ret = i2c_smbus_read_i2c_block_data_or_emulated(client,
+ def->data_regs[0],
+ 3 * sizeof(buff[0]),
+ (u8 *)buff);
+ if (ret < 0)
+ goto unlock;
+
+ mutex_unlock(&data->lock);
+
+ /* Clamp to valid range. */
+ buff[0] = clamp_t(s16, le16_to_cpu(buff[0]), -def->range, def->range);
+ buff[1] = clamp_t(s16, le16_to_cpu(buff[1]), -def->range, def->range);
+ buff[2] = clamp_t(s16, le16_to_cpu(buff[2]), -def->range, def->range);
+
+ iio_push_to_buffers_with_timestamp(indio_dev, buff, iio_get_time_ns());
+ return;
+
+unlock:
+ mutex_unlock(&data->lock);
+ dev_err(&client->dev, "Error in reading axes block\n");
+}
+
+static irqreturn_t ak8975_handle_trigger(int irq, void *p)
+{
+ const struct iio_poll_func *pf = p;
+ struct iio_dev *indio_dev = pf->indio_dev;
+
+ ak8975_fill_buffer(indio_dev);
+ iio_trigger_notify_done(indio_dev->trig);
+ return IRQ_HANDLED;
+}
+
static int ak8975_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
@@ -733,10 +852,12 @@
int err;
const char *name = NULL;
enum asahi_compass_chipset chipset = AK_MAX_TYPE;
+ const struct ak8975_platform_data *pdata =
+ dev_get_platdata(&client->dev);
/* Grab and set up the supplied GPIO. */
- if (client->dev.platform_data)
- eoc_gpio = *(int *)(client->dev.platform_data);
+ if (pdata)
+ eoc_gpio = pdata->eoc_gpio;
else if (client->dev.of_node)
eoc_gpio = of_get_gpio(client->dev.of_node, 0);
else
@@ -770,13 +891,24 @@
data->eoc_gpio = eoc_gpio;
data->eoc_irq = 0;
+ if (!pdata) {
+ err = of_iio_read_mount_matrix(&client->dev,
+ "mount-matrix",
+ &data->orientation);
+ if (err)
+ return err;
+ } else
+ data->orientation = pdata->orientation;
+
/* id will be NULL when enumerated via ACPI */
if (id) {
chipset = (enum asahi_compass_chipset)(id->driver_data);
name = id->name;
- } else if (ACPI_HANDLE(&client->dev))
+ } else if (ACPI_HANDLE(&client->dev)) {
name = ak8975_match_acpi_device(&client->dev, &chipset);
- else
+ if (!name)
+ return -ENODEV;
+ } else
return -ENOSYS;
if (chipset >= AK_MAX_TYPE) {
@@ -786,10 +918,15 @@
}
data->def = &ak_def_array[chipset];
+
+ err = ak8975_power_on(client);
+ if (err)
+ return err;
+
err = ak8975_who_i_am(client, data->def->type);
if (err < 0) {
dev_err(&client->dev, "Unexpected device\n");
- return err;
+ goto power_off;
}
dev_dbg(&client->dev, "Asahi compass chip %s\n", name);
@@ -797,7 +934,7 @@
err = ak8975_setup(client);
if (err < 0) {
dev_err(&client->dev, "%s initialization fails\n", name);
- return err;
+ goto power_off;
}
mutex_init(&data->lock);
@@ -805,9 +942,41 @@
indio_dev->channels = ak8975_channels;
indio_dev->num_channels = ARRAY_SIZE(ak8975_channels);
indio_dev->info = &ak8975_info;
+ indio_dev->available_scan_masks = ak8975_scan_masks;
indio_dev->modes = INDIO_DIRECT_MODE;
indio_dev->name = name;
- return devm_iio_device_register(&client->dev, indio_dev);
+
+ err = iio_triggered_buffer_setup(indio_dev, NULL, ak8975_handle_trigger,
+ NULL);
+ if (err) {
+ dev_err(&client->dev, "triggered buffer setup failed\n");
+ goto power_off;
+ }
+
+ err = iio_device_register(indio_dev);
+ if (err) {
+ dev_err(&client->dev, "device register failed\n");
+ goto cleanup_buffer;
+ }
+
+ return 0;
+
+cleanup_buffer:
+ iio_triggered_buffer_cleanup(indio_dev);
+power_off:
+ ak8975_power_off(client);
+ return err;
+}
+
+static int ak8975_remove(struct i2c_client *client)
+{
+ struct iio_dev *indio_dev = i2c_get_clientdata(client);
+
+ iio_device_unregister(indio_dev);
+ iio_triggered_buffer_cleanup(indio_dev);
+ ak8975_power_off(client);
+
+ return 0;
}
static const struct i2c_device_id ak8975_id[] = {
@@ -841,6 +1010,7 @@
.acpi_match_table = ACPI_PTR(ak_acpi_match),
},
.probe = ak8975_probe,
+ .remove = ak8975_remove,
.id_table = ak8975_id,
};
module_i2c_driver(ak8975_driver);
diff --git a/drivers/iio/magnetometer/bmc150_magn.c b/drivers/iio/magnetometer/bmc150_magn.c
index ffcb75e..d104fb8 100644
--- a/drivers/iio/magnetometer/bmc150_magn.c
+++ b/drivers/iio/magnetometer/bmc150_magn.c
@@ -23,7 +23,6 @@
#include <linux/delay.h>
#include <linux/slab.h>
#include <linux/acpi.h>
-#include <linux/gpio/consumer.h>
#include <linux/pm.h>
#include <linux/pm_runtime.h>
#include <linux/iio/iio.h>
@@ -35,6 +34,8 @@
#include <linux/iio/triggered_buffer.h>
#include <linux/regmap.h>
+#include "bmc150_magn.h"
+
#define BMC150_MAGN_DRV_NAME "bmc150_magn"
#define BMC150_MAGN_IRQ_NAME "bmc150_magn_event"
@@ -135,7 +136,7 @@
} __packed;
struct bmc150_magn_data {
- struct i2c_client *client;
+ struct device *dev;
/*
* 1. Protect this structure.
* 2. Serialize sequences that power on/off the device and access HW.
@@ -147,6 +148,7 @@
struct iio_trigger *dready_trig;
bool dready_trigger_on;
int max_odr;
+ int irq;
};
static const struct {
@@ -216,7 +218,7 @@
}
}
-static const struct regmap_config bmc150_magn_regmap_config = {
+const struct regmap_config bmc150_magn_regmap_config = {
.reg_bits = 8,
.val_bits = 8,
@@ -226,6 +228,7 @@
.writeable_reg = bmc150_magn_is_writeable_reg,
.volatile_reg = bmc150_magn_is_volatile_reg,
};
+EXPORT_SYMBOL(bmc150_magn_regmap_config);
static int bmc150_magn_set_power_mode(struct bmc150_magn_data *data,
enum bmc150_magn_power_modes mode,
@@ -264,17 +267,17 @@
int ret;
if (on) {
- ret = pm_runtime_get_sync(&data->client->dev);
+ ret = pm_runtime_get_sync(data->dev);
} else {
- pm_runtime_mark_last_busy(&data->client->dev);
- ret = pm_runtime_put_autosuspend(&data->client->dev);
+ pm_runtime_mark_last_busy(data->dev);
+ ret = pm_runtime_put_autosuspend(data->dev);
}
if (ret < 0) {
- dev_err(&data->client->dev,
+ dev_err(data->dev,
"failed to change power state to %d\n", on);
if (on)
- pm_runtime_put_noidle(&data->client->dev);
+ pm_runtime_put_noidle(data->dev);
return ret;
}
@@ -351,7 +354,7 @@
/* the maximum selectable read-out frequency from datasheet */
max_odr = 1000000 / (145 * rep_xy + 500 * rep_z + 980);
if (odr > max_odr) {
- dev_err(&data->client->dev,
+ dev_err(data->dev,
"Can't set oversampling with sampling freq %d\n",
odr);
return -EINVAL;
@@ -685,27 +688,27 @@
ret = bmc150_magn_set_power_mode(data, BMC150_MAGN_POWER_MODE_SUSPEND,
false);
if (ret < 0) {
- dev_err(&data->client->dev,
+ dev_err(data->dev,
"Failed to bring up device from suspend mode\n");
return ret;
}
ret = regmap_read(data->regmap, BMC150_MAGN_REG_CHIP_ID, &chip_id);
if (ret < 0) {
- dev_err(&data->client->dev, "Failed reading chip id\n");
+ dev_err(data->dev, "Failed reading chip id\n");
goto err_poweroff;
}
if (chip_id != BMC150_MAGN_CHIP_ID_VAL) {
- dev_err(&data->client->dev, "Invalid chip id 0x%x\n", chip_id);
+ dev_err(data->dev, "Invalid chip id 0x%x\n", chip_id);
ret = -ENODEV;
goto err_poweroff;
}
- dev_dbg(&data->client->dev, "Chip id %x\n", chip_id);
+ dev_dbg(data->dev, "Chip id %x\n", chip_id);
preset = bmc150_magn_presets_table[BMC150_MAGN_DEFAULT_PRESET];
ret = bmc150_magn_set_odr(data, preset.odr);
if (ret < 0) {
- dev_err(&data->client->dev, "Failed to set ODR to %d\n",
+ dev_err(data->dev, "Failed to set ODR to %d\n",
preset.odr);
goto err_poweroff;
}
@@ -713,7 +716,7 @@
ret = regmap_write(data->regmap, BMC150_MAGN_REG_REP_XY,
BMC150_MAGN_REPXY_TO_REGVAL(preset.rep_xy));
if (ret < 0) {
- dev_err(&data->client->dev, "Failed to set REP XY to %d\n",
+ dev_err(data->dev, "Failed to set REP XY to %d\n",
preset.rep_xy);
goto err_poweroff;
}
@@ -721,7 +724,7 @@
ret = regmap_write(data->regmap, BMC150_MAGN_REG_REP_Z,
BMC150_MAGN_REPZ_TO_REGVAL(preset.rep_z));
if (ret < 0) {
- dev_err(&data->client->dev, "Failed to set REP Z to %d\n",
+ dev_err(data->dev, "Failed to set REP Z to %d\n",
preset.rep_z);
goto err_poweroff;
}
@@ -734,7 +737,7 @@
ret = bmc150_magn_set_power_mode(data, BMC150_MAGN_POWER_MODE_NORMAL,
true);
if (ret < 0) {
- dev_err(&data->client->dev, "Failed to power on device\n");
+ dev_err(data->dev, "Failed to power on device\n");
goto err_poweroff;
}
@@ -843,41 +846,33 @@
return dev_name(dev);
}
-static int bmc150_magn_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
+int bmc150_magn_probe(struct device *dev, struct regmap *regmap,
+ int irq, const char *name)
{
struct bmc150_magn_data *data;
struct iio_dev *indio_dev;
- const char *name = NULL;
int ret;
- indio_dev = devm_iio_device_alloc(&client->dev, sizeof(*data));
+ indio_dev = devm_iio_device_alloc(dev, sizeof(*data));
if (!indio_dev)
return -ENOMEM;
data = iio_priv(indio_dev);
- i2c_set_clientdata(client, indio_dev);
- data->client = client;
+ dev_set_drvdata(dev, indio_dev);
+ data->regmap = regmap;
+ data->irq = irq;
+ data->dev = dev;
- if (id)
- name = id->name;
- else if (ACPI_HANDLE(&client->dev))
- name = bmc150_magn_match_acpi_device(&client->dev);
- else
- return -ENOSYS;
+ if (!name && ACPI_HANDLE(dev))
+ name = bmc150_magn_match_acpi_device(dev);
mutex_init(&data->mutex);
- data->regmap = devm_regmap_init_i2c(client, &bmc150_magn_regmap_config);
- if (IS_ERR(data->regmap)) {
- dev_err(&client->dev, "Failed to allocate register map\n");
- return PTR_ERR(data->regmap);
- }
ret = bmc150_magn_init(data);
if (ret < 0)
return ret;
- indio_dev->dev.parent = &client->dev;
+ indio_dev->dev.parent = dev;
indio_dev->channels = bmc150_magn_channels;
indio_dev->num_channels = ARRAY_SIZE(bmc150_magn_channels);
indio_dev->available_scan_masks = bmc150_magn_scan_masks;
@@ -885,35 +880,34 @@
indio_dev->modes = INDIO_DIRECT_MODE;
indio_dev->info = &bmc150_magn_info;
- if (client->irq > 0) {
- data->dready_trig = devm_iio_trigger_alloc(&client->dev,
+ if (irq > 0) {
+ data->dready_trig = devm_iio_trigger_alloc(dev,
"%s-dev%d",
indio_dev->name,
indio_dev->id);
if (!data->dready_trig) {
ret = -ENOMEM;
- dev_err(&client->dev, "iio trigger alloc failed\n");
+ dev_err(dev, "iio trigger alloc failed\n");
goto err_poweroff;
}
- data->dready_trig->dev.parent = &client->dev;
+ data->dready_trig->dev.parent = dev;
data->dready_trig->ops = &bmc150_magn_trigger_ops;
iio_trigger_set_drvdata(data->dready_trig, indio_dev);
ret = iio_trigger_register(data->dready_trig);
if (ret) {
- dev_err(&client->dev, "iio trigger register failed\n");
+ dev_err(dev, "iio trigger register failed\n");
goto err_poweroff;
}
- ret = request_threaded_irq(client->irq,
+ ret = request_threaded_irq(irq,
iio_trigger_generic_data_rdy_poll,
NULL,
IRQF_TRIGGER_RISING | IRQF_ONESHOT,
BMC150_MAGN_IRQ_NAME,
data->dready_trig);
if (ret < 0) {
- dev_err(&client->dev, "request irq %d failed\n",
- client->irq);
+ dev_err(dev, "request irq %d failed\n", irq);
goto err_trigger_unregister;
}
}
@@ -923,34 +917,33 @@
bmc150_magn_trigger_handler,
&bmc150_magn_buffer_setup_ops);
if (ret < 0) {
- dev_err(&client->dev,
- "iio triggered buffer setup failed\n");
+ dev_err(dev, "iio triggered buffer setup failed\n");
goto err_free_irq;
}
- ret = pm_runtime_set_active(&client->dev);
+ ret = pm_runtime_set_active(dev);
if (ret)
goto err_buffer_cleanup;
- pm_runtime_enable(&client->dev);
- pm_runtime_set_autosuspend_delay(&client->dev,
+ pm_runtime_enable(dev);
+ pm_runtime_set_autosuspend_delay(dev,
BMC150_MAGN_AUTO_SUSPEND_DELAY_MS);
- pm_runtime_use_autosuspend(&client->dev);
+ pm_runtime_use_autosuspend(dev);
ret = iio_device_register(indio_dev);
if (ret < 0) {
- dev_err(&client->dev, "unable to register iio device\n");
+ dev_err(dev, "unable to register iio device\n");
goto err_buffer_cleanup;
}
- dev_dbg(&indio_dev->dev, "Registered device %s\n", name);
+ dev_dbg(dev, "Registered device %s\n", name);
return 0;
err_buffer_cleanup:
iio_triggered_buffer_cleanup(indio_dev);
err_free_irq:
- if (client->irq > 0)
- free_irq(client->irq, data->dready_trig);
+ if (irq > 0)
+ free_irq(irq, data->dready_trig);
err_trigger_unregister:
if (data->dready_trig)
iio_trigger_unregister(data->dready_trig);
@@ -958,22 +951,23 @@
bmc150_magn_set_power_mode(data, BMC150_MAGN_POWER_MODE_SUSPEND, true);
return ret;
}
+EXPORT_SYMBOL(bmc150_magn_probe);
-static int bmc150_magn_remove(struct i2c_client *client)
+int bmc150_magn_remove(struct device *dev)
{
- struct iio_dev *indio_dev = i2c_get_clientdata(client);
+ struct iio_dev *indio_dev = dev_get_drvdata(dev);
struct bmc150_magn_data *data = iio_priv(indio_dev);
iio_device_unregister(indio_dev);
- pm_runtime_disable(&client->dev);
- pm_runtime_set_suspended(&client->dev);
- pm_runtime_put_noidle(&client->dev);
+ pm_runtime_disable(dev);
+ pm_runtime_set_suspended(dev);
+ pm_runtime_put_noidle(dev);
iio_triggered_buffer_cleanup(indio_dev);
- if (client->irq > 0)
- free_irq(data->client->irq, data->dready_trig);
+ if (data->irq > 0)
+ free_irq(data->irq, data->dready_trig);
if (data->dready_trig)
iio_trigger_unregister(data->dready_trig);
@@ -984,11 +978,12 @@
return 0;
}
+EXPORT_SYMBOL(bmc150_magn_remove);
#ifdef CONFIG_PM
static int bmc150_magn_runtime_suspend(struct device *dev)
{
- struct iio_dev *indio_dev = i2c_get_clientdata(to_i2c_client(dev));
+ struct iio_dev *indio_dev = dev_get_drvdata(dev);
struct bmc150_magn_data *data = iio_priv(indio_dev);
int ret;
@@ -997,7 +992,7 @@
true);
mutex_unlock(&data->mutex);
if (ret < 0) {
- dev_err(&data->client->dev, "powering off device failed\n");
+ dev_err(dev, "powering off device failed\n");
return ret;
}
return 0;
@@ -1008,7 +1003,7 @@
*/
static int bmc150_magn_runtime_resume(struct device *dev)
{
- struct iio_dev *indio_dev = i2c_get_clientdata(to_i2c_client(dev));
+ struct iio_dev *indio_dev = dev_get_drvdata(dev);
struct bmc150_magn_data *data = iio_priv(indio_dev);
return bmc150_magn_set_power_mode(data, BMC150_MAGN_POWER_MODE_NORMAL,
@@ -1019,7 +1014,7 @@
#ifdef CONFIG_PM_SLEEP
static int bmc150_magn_suspend(struct device *dev)
{
- struct iio_dev *indio_dev = i2c_get_clientdata(to_i2c_client(dev));
+ struct iio_dev *indio_dev = dev_get_drvdata(dev);
struct bmc150_magn_data *data = iio_priv(indio_dev);
int ret;
@@ -1033,7 +1028,7 @@
static int bmc150_magn_resume(struct device *dev)
{
- struct iio_dev *indio_dev = i2c_get_clientdata(to_i2c_client(dev));
+ struct iio_dev *indio_dev = dev_get_drvdata(dev);
struct bmc150_magn_data *data = iio_priv(indio_dev);
int ret;
@@ -1046,38 +1041,13 @@
}
#endif
-static const struct dev_pm_ops bmc150_magn_pm_ops = {
+const struct dev_pm_ops bmc150_magn_pm_ops = {
SET_SYSTEM_SLEEP_PM_OPS(bmc150_magn_suspend, bmc150_magn_resume)
SET_RUNTIME_PM_OPS(bmc150_magn_runtime_suspend,
bmc150_magn_runtime_resume, NULL)
};
-
-static const struct acpi_device_id bmc150_magn_acpi_match[] = {
- {"BMC150B", 0},
- {"BMC156B", 0},
- {},
-};
-MODULE_DEVICE_TABLE(acpi, bmc150_magn_acpi_match);
-
-static const struct i2c_device_id bmc150_magn_id[] = {
- {"bmc150_magn", 0},
- {"bmc156_magn", 0},
- {},
-};
-MODULE_DEVICE_TABLE(i2c, bmc150_magn_id);
-
-static struct i2c_driver bmc150_magn_driver = {
- .driver = {
- .name = BMC150_MAGN_DRV_NAME,
- .acpi_match_table = ACPI_PTR(bmc150_magn_acpi_match),
- .pm = &bmc150_magn_pm_ops,
- },
- .probe = bmc150_magn_probe,
- .remove = bmc150_magn_remove,
- .id_table = bmc150_magn_id,
-};
-module_i2c_driver(bmc150_magn_driver);
+EXPORT_SYMBOL(bmc150_magn_pm_ops);
MODULE_AUTHOR("Irina Tirdea <irina.tirdea@intel.com>");
MODULE_LICENSE("GPL v2");
-MODULE_DESCRIPTION("BMC150 magnetometer driver");
+MODULE_DESCRIPTION("BMC150 magnetometer core driver");
diff --git a/drivers/iio/magnetometer/bmc150_magn.h b/drivers/iio/magnetometer/bmc150_magn.h
new file mode 100644
index 0000000..9a8e268
--- /dev/null
+++ b/drivers/iio/magnetometer/bmc150_magn.h
@@ -0,0 +1,11 @@
+#ifndef _BMC150_MAGN_H_
+#define _BMC150_MAGN_H_
+
+extern const struct regmap_config bmc150_magn_regmap_config;
+extern const struct dev_pm_ops bmc150_magn_pm_ops;
+
+int bmc150_magn_probe(struct device *dev, struct regmap *regmap, int irq,
+ const char *name);
+int bmc150_magn_remove(struct device *dev);
+
+#endif /* _BMC150_MAGN_H_ */
diff --git a/drivers/iio/magnetometer/bmc150_magn_i2c.c b/drivers/iio/magnetometer/bmc150_magn_i2c.c
new file mode 100644
index 0000000..eddc7f0
--- /dev/null
+++ b/drivers/iio/magnetometer/bmc150_magn_i2c.c
@@ -0,0 +1,77 @@
+/*
+ * 3-axis magnetometer driver supporting following I2C Bosch-Sensortec chips:
+ * - BMC150
+ * - BMC156
+ *
+ * Copyright (c) 2016, Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ */
+#include <linux/device.h>
+#include <linux/mod_devicetable.h>
+#include <linux/i2c.h>
+#include <linux/module.h>
+#include <linux/acpi.h>
+#include <linux/regmap.h>
+
+#include "bmc150_magn.h"
+
+static int bmc150_magn_i2c_probe(struct i2c_client *client,
+ const struct i2c_device_id *id)
+{
+ struct regmap *regmap;
+ const char *name = NULL;
+
+ regmap = devm_regmap_init_i2c(client, &bmc150_magn_regmap_config);
+ if (IS_ERR(regmap)) {
+ dev_err(&client->dev, "Failed to initialize i2c regmap\n");
+ return PTR_ERR(regmap);
+ }
+
+ if (id)
+ name = id->name;
+
+ return bmc150_magn_probe(&client->dev, regmap, client->irq, name);
+}
+
+static int bmc150_magn_i2c_remove(struct i2c_client *client)
+{
+ return bmc150_magn_remove(&client->dev);
+}
+
+static const struct acpi_device_id bmc150_magn_acpi_match[] = {
+ {"BMC150B", 0},
+ {"BMC156B", 0},
+ {},
+};
+MODULE_DEVICE_TABLE(acpi, bmc150_magn_acpi_match);
+
+static const struct i2c_device_id bmc150_magn_i2c_id[] = {
+ {"bmc150_magn", 0},
+ {"bmc156_magn", 0},
+ {}
+};
+MODULE_DEVICE_TABLE(i2c, bmc150_magn_i2c_id);
+
+static struct i2c_driver bmc150_magn_driver = {
+ .driver = {
+ .name = "bmc150_magn_i2c",
+ .acpi_match_table = ACPI_PTR(bmc150_magn_acpi_match),
+ .pm = &bmc150_magn_pm_ops,
+ },
+ .probe = bmc150_magn_i2c_probe,
+ .remove = bmc150_magn_i2c_remove,
+ .id_table = bmc150_magn_i2c_id,
+};
+module_i2c_driver(bmc150_magn_driver);
+
+MODULE_AUTHOR("Daniel Baluta <daniel.baluta@intel.com");
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("BMC150 I2C magnetometer driver");
diff --git a/drivers/iio/magnetometer/bmc150_magn_spi.c b/drivers/iio/magnetometer/bmc150_magn_spi.c
new file mode 100644
index 0000000..c4c738a
--- /dev/null
+++ b/drivers/iio/magnetometer/bmc150_magn_spi.c
@@ -0,0 +1,68 @@
+/*
+ * 3-axis magnetometer driver support following SPI Bosch-Sensortec chips:
+ * - BMC150
+ * - BMC156
+ *
+ * Copyright (c) 2016, Intel Corporation.
+ *
+ * This file is subject to the terms and conditions of version 2 of
+ * the GNU General Public License. See the file COPYING in the main
+ * directory of this archive for more details.
+ */
+#include <linux/module.h>
+#include <linux/mod_devicetable.h>
+#include <linux/spi/spi.h>
+#include <linux/acpi.h>
+#include <linux/regmap.h>
+
+#include "bmc150_magn.h"
+
+static int bmc150_magn_spi_probe(struct spi_device *spi)
+{
+ struct regmap *regmap;
+ const struct spi_device_id *id = spi_get_device_id(spi);
+
+ regmap = devm_regmap_init_spi(spi, &bmc150_magn_regmap_config);
+ if (IS_ERR(regmap)) {
+ dev_err(&spi->dev, "Failed to register spi regmap %d\n",
+ (int)PTR_ERR(regmap));
+ return PTR_ERR(regmap);
+ }
+ return bmc150_magn_probe(&spi->dev, regmap, spi->irq, id->name);
+}
+
+static int bmc150_magn_spi_remove(struct spi_device *spi)
+{
+ bmc150_magn_remove(&spi->dev);
+
+ return 0;
+}
+
+static const struct spi_device_id bmc150_magn_spi_id[] = {
+ {"bmc150_magn", 0},
+ {"bmc156_magn", 0},
+ {}
+};
+MODULE_DEVICE_TABLE(spi, bmc150_magn_spi_id);
+
+static const struct acpi_device_id bmc150_magn_acpi_match[] = {
+ {"BMC150B", 0},
+ {"BMC156B", 0},
+ {},
+};
+MODULE_DEVICE_TABLE(acpi, bmc150_magn_acpi_match);
+
+static struct spi_driver bmc150_magn_spi_driver = {
+ .probe = bmc150_magn_spi_probe,
+ .remove = bmc150_magn_spi_remove,
+ .id_table = bmc150_magn_spi_id,
+ .driver = {
+ .acpi_match_table = ACPI_PTR(bmc150_magn_acpi_match),
+ .name = "bmc150_magn_spi",
+ },
+};
+module_spi_driver(bmc150_magn_spi_driver);
+
+MODULE_AUTHOR("Daniel Baluta <daniel.baluta@intel.com");
+MODULE_DESCRIPTION("BMC150 magnetometer SPI driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/iio/magnetometer/st_magn_core.c b/drivers/iio/magnetometer/st_magn_core.c
index 501f858..62036d2 100644
--- a/drivers/iio/magnetometer/st_magn_core.c
+++ b/drivers/iio/magnetometer/st_magn_core.c
@@ -484,6 +484,7 @@
.mask_int1 = ST_MAGN_3_DRDY_INT_MASK,
.addr_ihl = ST_MAGN_3_IHL_IRQ_ADDR,
.mask_ihl = ST_MAGN_3_IHL_IRQ_MASK,
+ .addr_stat_drdy = ST_SENSORS_DEFAULT_STAT_ADDR,
},
.multi_read_bit = ST_MAGN_3_MULTIREAD_BIT,
.bootime = 2,
diff --git a/drivers/iio/potentiometer/Kconfig b/drivers/iio/potentiometer/Kconfig
index ffc735c..6acb238 100644
--- a/drivers/iio/potentiometer/Kconfig
+++ b/drivers/iio/potentiometer/Kconfig
@@ -5,6 +5,34 @@
menu "Digital potentiometers"
+config DS1803
+ tristate "Maxim Integrated DS1803 Digital Potentiometer driver"
+ depends on I2C
+ help
+ Say yes here to build support for the Maxim Integrated DS1803
+ digital potentiomenter chip.
+
+ To compile this driver as a module, choose M here: the
+ module will be called ds1803.
+
+config MCP4131
+ tristate "Microchip MCP413X/414X/415X/416X/423X/424X/425X/426X Digital Potentiometer driver"
+ depends on SPI
+ help
+ Say yes here to build support for the Microchip
+ MCP4131, MCP4132,
+ MCP4141, MCP4142,
+ MCP4151, MCP4152,
+ MCP4161, MCP4162,
+ MCP4231, MCP4232,
+ MCP4241, MCP4242,
+ MCP4251, MCP4252,
+ MCP4261, MCP4262,
+ digital potentiomenter chips.
+
+ To compile this driver as a module, choose M here: the
+ module will be called mcp4131.
+
config MCP4531
tristate "Microchip MCP45xx/MCP46xx Digital Potentiometer driver"
depends on I2C
diff --git a/drivers/iio/potentiometer/Makefile b/drivers/iio/potentiometer/Makefile
index b563b49..6007faa 100644
--- a/drivers/iio/potentiometer/Makefile
+++ b/drivers/iio/potentiometer/Makefile
@@ -3,5 +3,7 @@
#
# When adding new entries keep the list in alphabetical order
+obj-$(CONFIG_DS1803) += ds1803.o
+obj-$(CONFIG_MCP4131) += mcp4131.o
obj-$(CONFIG_MCP4531) += mcp4531.o
obj-$(CONFIG_TPL0102) += tpl0102.o
diff --git a/drivers/iio/potentiometer/ds1803.c b/drivers/iio/potentiometer/ds1803.c
new file mode 100644
index 0000000..fb9e2a3
--- /dev/null
+++ b/drivers/iio/potentiometer/ds1803.c
@@ -0,0 +1,173 @@
+/*
+ * Maxim Integrated DS1803 digital potentiometer driver
+ * Copyright (c) 2016 Slawomir Stepien
+ *
+ * Datasheet: https://datasheets.maximintegrated.com/en/ds/DS1803.pdf
+ *
+ * DEVID #Wipers #Positions Resistor Opts (kOhm) i2c address
+ * ds1803 2 256 10, 50, 100 0101xxx
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ */
+
+#include <linux/err.h>
+#include <linux/export.h>
+#include <linux/i2c.h>
+#include <linux/iio/iio.h>
+#include <linux/module.h>
+#include <linux/of.h>
+
+#define DS1803_MAX_POS 255
+#define DS1803_WRITE(chan) (0xa8 | ((chan) + 1))
+
+enum ds1803_type {
+ DS1803_010,
+ DS1803_050,
+ DS1803_100,
+};
+
+struct ds1803_cfg {
+ int kohms;
+};
+
+static const struct ds1803_cfg ds1803_cfg[] = {
+ [DS1803_010] = { .kohms = 10, },
+ [DS1803_050] = { .kohms = 50, },
+ [DS1803_100] = { .kohms = 100, },
+};
+
+struct ds1803_data {
+ struct i2c_client *client;
+ const struct ds1803_cfg *cfg;
+};
+
+#define DS1803_CHANNEL(ch) { \
+ .type = IIO_RESISTANCE, \
+ .indexed = 1, \
+ .output = 1, \
+ .channel = (ch), \
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW), \
+ .info_mask_shared_by_type = BIT(IIO_CHAN_INFO_SCALE), \
+}
+
+static const struct iio_chan_spec ds1803_channels[] = {
+ DS1803_CHANNEL(0),
+ DS1803_CHANNEL(1),
+};
+
+static int ds1803_read_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int *val, int *val2, long mask)
+{
+ struct ds1803_data *data = iio_priv(indio_dev);
+ int pot = chan->channel;
+ int ret;
+ u8 result[indio_dev->num_channels];
+
+ switch (mask) {
+ case IIO_CHAN_INFO_RAW:
+ ret = i2c_master_recv(data->client, result,
+ indio_dev->num_channels);
+ if (ret < 0)
+ return ret;
+
+ *val = result[pot];
+ return IIO_VAL_INT;
+
+ case IIO_CHAN_INFO_SCALE:
+ *val = 1000 * data->cfg->kohms;
+ *val2 = DS1803_MAX_POS;
+ return IIO_VAL_FRACTIONAL;
+ }
+
+ return -EINVAL;
+}
+
+static int ds1803_write_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int val, int val2, long mask)
+{
+ struct ds1803_data *data = iio_priv(indio_dev);
+ int pot = chan->channel;
+
+ if (val2 != 0)
+ return -EINVAL;
+
+ switch (mask) {
+ case IIO_CHAN_INFO_RAW:
+ if (val > DS1803_MAX_POS || val < 0)
+ return -EINVAL;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return i2c_smbus_write_byte_data(data->client, DS1803_WRITE(pot), val);
+}
+
+static const struct iio_info ds1803_info = {
+ .read_raw = ds1803_read_raw,
+ .write_raw = ds1803_write_raw,
+ .driver_module = THIS_MODULE,
+};
+
+static int ds1803_probe(struct i2c_client *client,
+ const struct i2c_device_id *id)
+{
+ struct device *dev = &client->dev;
+ struct ds1803_data *data;
+ struct iio_dev *indio_dev;
+
+ indio_dev = devm_iio_device_alloc(dev, sizeof(*data));
+ if (!indio_dev)
+ return -ENOMEM;
+
+ i2c_set_clientdata(client, indio_dev);
+
+ data = iio_priv(indio_dev);
+ data->client = client;
+ data->cfg = &ds1803_cfg[id->driver_data];
+
+ indio_dev->dev.parent = dev;
+ indio_dev->info = &ds1803_info;
+ indio_dev->channels = ds1803_channels;
+ indio_dev->num_channels = ARRAY_SIZE(ds1803_channels);
+ indio_dev->name = client->name;
+
+ return devm_iio_device_register(dev, indio_dev);
+}
+
+#if defined(CONFIG_OF)
+static const struct of_device_id ds1803_dt_ids[] = {
+ { .compatible = "maxim,ds1803-010", .data = &ds1803_cfg[DS1803_010] },
+ { .compatible = "maxim,ds1803-050", .data = &ds1803_cfg[DS1803_050] },
+ { .compatible = "maxim,ds1803-100", .data = &ds1803_cfg[DS1803_100] },
+ {}
+};
+MODULE_DEVICE_TABLE(of, ds1803_dt_ids);
+#endif /* CONFIG_OF */
+
+static const struct i2c_device_id ds1803_id[] = {
+ { "ds1803-010", DS1803_010 },
+ { "ds1803-050", DS1803_050 },
+ { "ds1803-100", DS1803_100 },
+ {}
+};
+MODULE_DEVICE_TABLE(i2c, ds1803_id);
+
+static struct i2c_driver ds1803_driver = {
+ .driver = {
+ .name = "ds1803",
+ .of_match_table = of_match_ptr(ds1803_dt_ids),
+ },
+ .probe = ds1803_probe,
+ .id_table = ds1803_id,
+};
+
+module_i2c_driver(ds1803_driver);
+
+MODULE_AUTHOR("Slawomir Stepien <sst@poczta.fm>");
+MODULE_DESCRIPTION("DS1803 digital potentiometer");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/iio/potentiometer/mcp4131.c b/drivers/iio/potentiometer/mcp4131.c
new file mode 100644
index 0000000..4e7e2c6
--- /dev/null
+++ b/drivers/iio/potentiometer/mcp4131.c
@@ -0,0 +1,494 @@
+/*
+ * Industrial I/O driver for Microchip digital potentiometers
+ *
+ * Copyright (c) 2016 Slawomir Stepien
+ * Based on: Peter Rosin's code from mcp4531.c
+ *
+ * Datasheet: http://ww1.microchip.com/downloads/en/DeviceDoc/22060b.pdf
+ *
+ * DEVID #Wipers #Positions Resistor Opts (kOhm)
+ * mcp4131 1 129 5, 10, 50, 100
+ * mcp4132 1 129 5, 10, 50, 100
+ * mcp4141 1 129 5, 10, 50, 100
+ * mcp4142 1 129 5, 10, 50, 100
+ * mcp4151 1 257 5, 10, 50, 100
+ * mcp4152 1 257 5, 10, 50, 100
+ * mcp4161 1 257 5, 10, 50, 100
+ * mcp4162 1 257 5, 10, 50, 100
+ * mcp4231 2 129 5, 10, 50, 100
+ * mcp4232 2 129 5, 10, 50, 100
+ * mcp4241 2 129 5, 10, 50, 100
+ * mcp4242 2 129 5, 10, 50, 100
+ * mcp4251 2 257 5, 10, 50, 100
+ * mcp4252 2 257 5, 10, 50, 100
+ * mcp4261 2 257 5, 10, 50, 100
+ * mcp4262 2 257 5, 10, 50, 100
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ */
+
+/*
+ * TODO:
+ * 1. Write wiper setting to EEPROM for EEPROM capable models.
+ */
+
+#include <linux/cache.h>
+#include <linux/err.h>
+#include <linux/export.h>
+#include <linux/iio/iio.h>
+#include <linux/iio/types.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/of.h>
+#include <linux/spi/spi.h>
+
+#define MCP4131_WRITE (0x00 << 2)
+#define MCP4131_READ (0x03 << 2)
+
+#define MCP4131_WIPER_SHIFT 4
+#define MCP4131_CMDERR(r) ((r[0]) & 0x02)
+#define MCP4131_RAW(r) ((r[0]) == 0xff ? 0x100 : (r[1]))
+
+struct mcp4131_cfg {
+ int wipers;
+ int max_pos;
+ int kohms;
+};
+
+enum mcp4131_type {
+ MCP413x_502 = 0,
+ MCP413x_103,
+ MCP413x_503,
+ MCP413x_104,
+ MCP414x_502,
+ MCP414x_103,
+ MCP414x_503,
+ MCP414x_104,
+ MCP415x_502,
+ MCP415x_103,
+ MCP415x_503,
+ MCP415x_104,
+ MCP416x_502,
+ MCP416x_103,
+ MCP416x_503,
+ MCP416x_104,
+ MCP423x_502,
+ MCP423x_103,
+ MCP423x_503,
+ MCP423x_104,
+ MCP424x_502,
+ MCP424x_103,
+ MCP424x_503,
+ MCP424x_104,
+ MCP425x_502,
+ MCP425x_103,
+ MCP425x_503,
+ MCP425x_104,
+ MCP426x_502,
+ MCP426x_103,
+ MCP426x_503,
+ MCP426x_104,
+};
+
+static const struct mcp4131_cfg mcp4131_cfg[] = {
+ [MCP413x_502] = { .wipers = 1, .max_pos = 128, .kohms = 5, },
+ [MCP413x_103] = { .wipers = 1, .max_pos = 128, .kohms = 10, },
+ [MCP413x_503] = { .wipers = 1, .max_pos = 128, .kohms = 50, },
+ [MCP413x_104] = { .wipers = 1, .max_pos = 128, .kohms = 100, },
+ [MCP414x_502] = { .wipers = 1, .max_pos = 128, .kohms = 5, },
+ [MCP414x_103] = { .wipers = 1, .max_pos = 128, .kohms = 10, },
+ [MCP414x_503] = { .wipers = 1, .max_pos = 128, .kohms = 50, },
+ [MCP414x_104] = { .wipers = 1, .max_pos = 128, .kohms = 100, },
+ [MCP415x_502] = { .wipers = 1, .max_pos = 256, .kohms = 5, },
+ [MCP415x_103] = { .wipers = 1, .max_pos = 256, .kohms = 10, },
+ [MCP415x_503] = { .wipers = 1, .max_pos = 256, .kohms = 50, },
+ [MCP415x_104] = { .wipers = 1, .max_pos = 256, .kohms = 100, },
+ [MCP416x_502] = { .wipers = 1, .max_pos = 256, .kohms = 5, },
+ [MCP416x_103] = { .wipers = 1, .max_pos = 256, .kohms = 10, },
+ [MCP416x_503] = { .wipers = 1, .max_pos = 256, .kohms = 50, },
+ [MCP416x_104] = { .wipers = 1, .max_pos = 256, .kohms = 100, },
+ [MCP423x_502] = { .wipers = 2, .max_pos = 128, .kohms = 5, },
+ [MCP423x_103] = { .wipers = 2, .max_pos = 128, .kohms = 10, },
+ [MCP423x_503] = { .wipers = 2, .max_pos = 128, .kohms = 50, },
+ [MCP423x_104] = { .wipers = 2, .max_pos = 128, .kohms = 100, },
+ [MCP424x_502] = { .wipers = 2, .max_pos = 128, .kohms = 5, },
+ [MCP424x_103] = { .wipers = 2, .max_pos = 128, .kohms = 10, },
+ [MCP424x_503] = { .wipers = 2, .max_pos = 128, .kohms = 50, },
+ [MCP424x_104] = { .wipers = 2, .max_pos = 128, .kohms = 100, },
+ [MCP425x_502] = { .wipers = 2, .max_pos = 256, .kohms = 5, },
+ [MCP425x_103] = { .wipers = 2, .max_pos = 256, .kohms = 10, },
+ [MCP425x_503] = { .wipers = 2, .max_pos = 256, .kohms = 50, },
+ [MCP425x_104] = { .wipers = 2, .max_pos = 256, .kohms = 100, },
+ [MCP426x_502] = { .wipers = 2, .max_pos = 256, .kohms = 5, },
+ [MCP426x_103] = { .wipers = 2, .max_pos = 256, .kohms = 10, },
+ [MCP426x_503] = { .wipers = 2, .max_pos = 256, .kohms = 50, },
+ [MCP426x_104] = { .wipers = 2, .max_pos = 256, .kohms = 100, },
+};
+
+struct mcp4131_data {
+ struct spi_device *spi;
+ const struct mcp4131_cfg *cfg;
+ struct mutex lock;
+ u8 buf[2] ____cacheline_aligned;
+};
+
+#define MCP4131_CHANNEL(ch) { \
+ .type = IIO_RESISTANCE, \
+ .indexed = 1, \
+ .output = 1, \
+ .channel = (ch), \
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW), \
+ .info_mask_shared_by_type = BIT(IIO_CHAN_INFO_SCALE), \
+}
+
+static const struct iio_chan_spec mcp4131_channels[] = {
+ MCP4131_CHANNEL(0),
+ MCP4131_CHANNEL(1),
+};
+
+static int mcp4131_read(struct spi_device *spi, void *buf, size_t len)
+{
+ struct spi_transfer t = {
+ .tx_buf = buf, /* We need to send addr, cmd and 12 bits */
+ .rx_buf = buf,
+ .len = len,
+ };
+ struct spi_message m;
+
+ spi_message_init(&m);
+ spi_message_add_tail(&t, &m);
+
+ return spi_sync(spi, &m);
+}
+
+static int mcp4131_read_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int *val, int *val2, long mask)
+{
+ int err;
+ struct mcp4131_data *data = iio_priv(indio_dev);
+ int address = chan->channel;
+
+ switch (mask) {
+ case IIO_CHAN_INFO_RAW:
+ mutex_lock(&data->lock);
+
+ data->buf[0] = (address << MCP4131_WIPER_SHIFT) | MCP4131_READ;
+ data->buf[1] = 0;
+
+ err = mcp4131_read(data->spi, data->buf, 2);
+ if (err) {
+ mutex_unlock(&data->lock);
+ return err;
+ }
+
+ /* Error, bad address/command combination */
+ if (!MCP4131_CMDERR(data->buf)) {
+ mutex_unlock(&data->lock);
+ return -EIO;
+ }
+
+ *val = MCP4131_RAW(data->buf);
+ mutex_unlock(&data->lock);
+
+ return IIO_VAL_INT;
+
+ case IIO_CHAN_INFO_SCALE:
+ *val = 1000 * data->cfg->kohms;
+ *val2 = data->cfg->max_pos;
+ return IIO_VAL_FRACTIONAL;
+ }
+
+ return -EINVAL;
+}
+
+static int mcp4131_write_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int val, int val2, long mask)
+{
+ int err;
+ struct mcp4131_data *data = iio_priv(indio_dev);
+ int address = chan->channel << MCP4131_WIPER_SHIFT;
+
+ switch (mask) {
+ case IIO_CHAN_INFO_RAW:
+ if (val > data->cfg->max_pos || val < 0)
+ return -EINVAL;
+ break;
+
+ default:
+ return -EINVAL;
+ }
+
+ mutex_lock(&data->lock);
+
+ data->buf[0] = address << MCP4131_WIPER_SHIFT;
+ data->buf[0] |= MCP4131_WRITE | (val >> 8);
+ data->buf[1] = val & 0xFF; /* 8 bits here */
+
+ err = spi_write(data->spi, data->buf, 2);
+ mutex_unlock(&data->lock);
+
+ return err;
+}
+
+static const struct iio_info mcp4131_info = {
+ .read_raw = mcp4131_read_raw,
+ .write_raw = mcp4131_write_raw,
+ .driver_module = THIS_MODULE,
+};
+
+static int mcp4131_probe(struct spi_device *spi)
+{
+ int err;
+ struct device *dev = &spi->dev;
+ unsigned long devid = spi_get_device_id(spi)->driver_data;
+ struct mcp4131_data *data;
+ struct iio_dev *indio_dev;
+
+ indio_dev = devm_iio_device_alloc(dev, sizeof(*data));
+ if (!indio_dev)
+ return -ENOMEM;
+
+ data = iio_priv(indio_dev);
+ spi_set_drvdata(spi, indio_dev);
+ data->spi = spi;
+ data->cfg = &mcp4131_cfg[devid];
+
+ mutex_init(&data->lock);
+
+ indio_dev->dev.parent = dev;
+ indio_dev->info = &mcp4131_info;
+ indio_dev->channels = mcp4131_channels;
+ indio_dev->num_channels = data->cfg->wipers;
+ indio_dev->name = spi_get_device_id(spi)->name;
+
+ err = devm_iio_device_register(dev, indio_dev);
+ if (err) {
+ dev_info(&spi->dev, "Unable to register %s\n", indio_dev->name);
+ return err;
+ }
+
+ return 0;
+}
+
+#if defined(CONFIG_OF)
+static const struct of_device_id mcp4131_dt_ids[] = {
+ { .compatible = "microchip,mcp4131-502",
+ .data = &mcp4131_cfg[MCP413x_502] },
+ { .compatible = "microchip,mcp4131-103",
+ .data = &mcp4131_cfg[MCP413x_103] },
+ { .compatible = "microchip,mcp4131-503",
+ .data = &mcp4131_cfg[MCP413x_503] },
+ { .compatible = "microchip,mcp4131-104",
+ .data = &mcp4131_cfg[MCP413x_104] },
+ { .compatible = "microchip,mcp4132-502",
+ .data = &mcp4131_cfg[MCP413x_502] },
+ { .compatible = "microchip,mcp4132-103",
+ .data = &mcp4131_cfg[MCP413x_103] },
+ { .compatible = "microchip,mcp4132-503",
+ .data = &mcp4131_cfg[MCP413x_503] },
+ { .compatible = "microchip,mcp4132-104",
+ .data = &mcp4131_cfg[MCP413x_104] },
+ { .compatible = "microchip,mcp4141-502",
+ .data = &mcp4131_cfg[MCP414x_502] },
+ { .compatible = "microchip,mcp4141-103",
+ .data = &mcp4131_cfg[MCP414x_103] },
+ { .compatible = "microchip,mcp4141-503",
+ .data = &mcp4131_cfg[MCP414x_503] },
+ { .compatible = "microchip,mcp4141-104",
+ .data = &mcp4131_cfg[MCP414x_104] },
+ { .compatible = "microchip,mcp4142-502",
+ .data = &mcp4131_cfg[MCP414x_502] },
+ { .compatible = "microchip,mcp4142-103",
+ .data = &mcp4131_cfg[MCP414x_103] },
+ { .compatible = "microchip,mcp4142-503",
+ .data = &mcp4131_cfg[MCP414x_503] },
+ { .compatible = "microchip,mcp4142-104",
+ .data = &mcp4131_cfg[MCP414x_104] },
+ { .compatible = "microchip,mcp4151-502",
+ .data = &mcp4131_cfg[MCP415x_502] },
+ { .compatible = "microchip,mcp4151-103",
+ .data = &mcp4131_cfg[MCP415x_103] },
+ { .compatible = "microchip,mcp4151-503",
+ .data = &mcp4131_cfg[MCP415x_503] },
+ { .compatible = "microchip,mcp4151-104",
+ .data = &mcp4131_cfg[MCP415x_104] },
+ { .compatible = "microchip,mcp4152-502",
+ .data = &mcp4131_cfg[MCP415x_502] },
+ { .compatible = "microchip,mcp4152-103",
+ .data = &mcp4131_cfg[MCP415x_103] },
+ { .compatible = "microchip,mcp4152-503",
+ .data = &mcp4131_cfg[MCP415x_503] },
+ { .compatible = "microchip,mcp4152-104",
+ .data = &mcp4131_cfg[MCP415x_104] },
+ { .compatible = "microchip,mcp4161-502",
+ .data = &mcp4131_cfg[MCP416x_502] },
+ { .compatible = "microchip,mcp4161-103",
+ .data = &mcp4131_cfg[MCP416x_103] },
+ { .compatible = "microchip,mcp4161-503",
+ .data = &mcp4131_cfg[MCP416x_503] },
+ { .compatible = "microchip,mcp4161-104",
+ .data = &mcp4131_cfg[MCP416x_104] },
+ { .compatible = "microchip,mcp4162-502",
+ .data = &mcp4131_cfg[MCP416x_502] },
+ { .compatible = "microchip,mcp4162-103",
+ .data = &mcp4131_cfg[MCP416x_103] },
+ { .compatible = "microchip,mcp4162-503",
+ .data = &mcp4131_cfg[MCP416x_503] },
+ { .compatible = "microchip,mcp4162-104",
+ .data = &mcp4131_cfg[MCP416x_104] },
+ { .compatible = "microchip,mcp4231-502",
+ .data = &mcp4131_cfg[MCP423x_502] },
+ { .compatible = "microchip,mcp4231-103",
+ .data = &mcp4131_cfg[MCP423x_103] },
+ { .compatible = "microchip,mcp4231-503",
+ .data = &mcp4131_cfg[MCP423x_503] },
+ { .compatible = "microchip,mcp4231-104",
+ .data = &mcp4131_cfg[MCP423x_104] },
+ { .compatible = "microchip,mcp4232-502",
+ .data = &mcp4131_cfg[MCP423x_502] },
+ { .compatible = "microchip,mcp4232-103",
+ .data = &mcp4131_cfg[MCP423x_103] },
+ { .compatible = "microchip,mcp4232-503",
+ .data = &mcp4131_cfg[MCP423x_503] },
+ { .compatible = "microchip,mcp4232-104",
+ .data = &mcp4131_cfg[MCP423x_104] },
+ { .compatible = "microchip,mcp4241-502",
+ .data = &mcp4131_cfg[MCP424x_502] },
+ { .compatible = "microchip,mcp4241-103",
+ .data = &mcp4131_cfg[MCP424x_103] },
+ { .compatible = "microchip,mcp4241-503",
+ .data = &mcp4131_cfg[MCP424x_503] },
+ { .compatible = "microchip,mcp4241-104",
+ .data = &mcp4131_cfg[MCP424x_104] },
+ { .compatible = "microchip,mcp4242-502",
+ .data = &mcp4131_cfg[MCP424x_502] },
+ { .compatible = "microchip,mcp4242-103",
+ .data = &mcp4131_cfg[MCP424x_103] },
+ { .compatible = "microchip,mcp4242-503",
+ .data = &mcp4131_cfg[MCP424x_503] },
+ { .compatible = "microchip,mcp4242-104",
+ .data = &mcp4131_cfg[MCP424x_104] },
+ { .compatible = "microchip,mcp4251-502",
+ .data = &mcp4131_cfg[MCP425x_502] },
+ { .compatible = "microchip,mcp4251-103",
+ .data = &mcp4131_cfg[MCP425x_103] },
+ { .compatible = "microchip,mcp4251-503",
+ .data = &mcp4131_cfg[MCP425x_503] },
+ { .compatible = "microchip,mcp4251-104",
+ .data = &mcp4131_cfg[MCP425x_104] },
+ { .compatible = "microchip,mcp4252-502",
+ .data = &mcp4131_cfg[MCP425x_502] },
+ { .compatible = "microchip,mcp4252-103",
+ .data = &mcp4131_cfg[MCP425x_103] },
+ { .compatible = "microchip,mcp4252-503",
+ .data = &mcp4131_cfg[MCP425x_503] },
+ { .compatible = "microchip,mcp4252-104",
+ .data = &mcp4131_cfg[MCP425x_104] },
+ { .compatible = "microchip,mcp4261-502",
+ .data = &mcp4131_cfg[MCP426x_502] },
+ { .compatible = "microchip,mcp4261-103",
+ .data = &mcp4131_cfg[MCP426x_103] },
+ { .compatible = "microchip,mcp4261-503",
+ .data = &mcp4131_cfg[MCP426x_503] },
+ { .compatible = "microchip,mcp4261-104",
+ .data = &mcp4131_cfg[MCP426x_104] },
+ { .compatible = "microchip,mcp4262-502",
+ .data = &mcp4131_cfg[MCP426x_502] },
+ { .compatible = "microchip,mcp4262-103",
+ .data = &mcp4131_cfg[MCP426x_103] },
+ { .compatible = "microchip,mcp4262-503",
+ .data = &mcp4131_cfg[MCP426x_503] },
+ { .compatible = "microchip,mcp4262-104",
+ .data = &mcp4131_cfg[MCP426x_104] },
+ {}
+};
+MODULE_DEVICE_TABLE(of, mcp4131_dt_ids);
+#endif /* CONFIG_OF */
+
+static const struct spi_device_id mcp4131_id[] = {
+ { "mcp4131-502", MCP413x_502 },
+ { "mcp4131-103", MCP413x_103 },
+ { "mcp4131-503", MCP413x_503 },
+ { "mcp4131-104", MCP413x_104 },
+ { "mcp4132-502", MCP413x_502 },
+ { "mcp4132-103", MCP413x_103 },
+ { "mcp4132-503", MCP413x_503 },
+ { "mcp4132-104", MCP413x_104 },
+ { "mcp4141-502", MCP414x_502 },
+ { "mcp4141-103", MCP414x_103 },
+ { "mcp4141-503", MCP414x_503 },
+ { "mcp4141-104", MCP414x_104 },
+ { "mcp4142-502", MCP414x_502 },
+ { "mcp4142-103", MCP414x_103 },
+ { "mcp4142-503", MCP414x_503 },
+ { "mcp4142-104", MCP414x_104 },
+ { "mcp4151-502", MCP415x_502 },
+ { "mcp4151-103", MCP415x_103 },
+ { "mcp4151-503", MCP415x_503 },
+ { "mcp4151-104", MCP415x_104 },
+ { "mcp4152-502", MCP415x_502 },
+ { "mcp4152-103", MCP415x_103 },
+ { "mcp4152-503", MCP415x_503 },
+ { "mcp4152-104", MCP415x_104 },
+ { "mcp4161-502", MCP416x_502 },
+ { "mcp4161-103", MCP416x_103 },
+ { "mcp4161-503", MCP416x_503 },
+ { "mcp4161-104", MCP416x_104 },
+ { "mcp4162-502", MCP416x_502 },
+ { "mcp4162-103", MCP416x_103 },
+ { "mcp4162-503", MCP416x_503 },
+ { "mcp4162-104", MCP416x_104 },
+ { "mcp4231-502", MCP423x_502 },
+ { "mcp4231-103", MCP423x_103 },
+ { "mcp4231-503", MCP423x_503 },
+ { "mcp4231-104", MCP423x_104 },
+ { "mcp4232-502", MCP423x_502 },
+ { "mcp4232-103", MCP423x_103 },
+ { "mcp4232-503", MCP423x_503 },
+ { "mcp4232-104", MCP423x_104 },
+ { "mcp4241-502", MCP424x_502 },
+ { "mcp4241-103", MCP424x_103 },
+ { "mcp4241-503", MCP424x_503 },
+ { "mcp4241-104", MCP424x_104 },
+ { "mcp4242-502", MCP424x_502 },
+ { "mcp4242-103", MCP424x_103 },
+ { "mcp4242-503", MCP424x_503 },
+ { "mcp4242-104", MCP424x_104 },
+ { "mcp4251-502", MCP425x_502 },
+ { "mcp4251-103", MCP425x_103 },
+ { "mcp4251-503", MCP425x_503 },
+ { "mcp4251-104", MCP425x_104 },
+ { "mcp4252-502", MCP425x_502 },
+ { "mcp4252-103", MCP425x_103 },
+ { "mcp4252-503", MCP425x_503 },
+ { "mcp4252-104", MCP425x_104 },
+ { "mcp4261-502", MCP426x_502 },
+ { "mcp4261-103", MCP426x_103 },
+ { "mcp4261-503", MCP426x_503 },
+ { "mcp4261-104", MCP426x_104 },
+ { "mcp4262-502", MCP426x_502 },
+ { "mcp4262-103", MCP426x_103 },
+ { "mcp4262-503", MCP426x_503 },
+ { "mcp4262-104", MCP426x_104 },
+ {}
+};
+MODULE_DEVICE_TABLE(spi, mcp4131_id);
+
+static struct spi_driver mcp4131_driver = {
+ .driver = {
+ .name = "mcp4131",
+ .of_match_table = of_match_ptr(mcp4131_dt_ids),
+ },
+ .probe = mcp4131_probe,
+ .id_table = mcp4131_id,
+};
+
+module_spi_driver(mcp4131_driver);
+
+MODULE_AUTHOR("Slawomir Stepien <sst@poczta.fm>");
+MODULE_DESCRIPTION("MCP4131 digital potentiometer");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/iio/potentiometer/mcp4531.c b/drivers/iio/potentiometer/mcp4531.c
index 0db67fe..3b72e1a 100644
--- a/drivers/iio/potentiometer/mcp4531.c
+++ b/drivers/iio/potentiometer/mcp4531.c
@@ -79,7 +79,7 @@
struct mcp4531_data {
struct i2c_client *client;
- unsigned long devid;
+ const struct mcp4531_cfg *cfg;
};
#define MCP4531_CHANNEL(ch) { \
@@ -113,8 +113,8 @@
*val = ret;
return IIO_VAL_INT;
case IIO_CHAN_INFO_SCALE:
- *val = 1000 * mcp4531_cfg[data->devid].kohms;
- *val2 = mcp4531_cfg[data->devid].max_pos;
+ *val = 1000 * data->cfg->kohms;
+ *val2 = data->cfg->max_pos;
return IIO_VAL_FRACTIONAL;
}
@@ -130,7 +130,7 @@
switch (mask) {
case IIO_CHAN_INFO_RAW:
- if (val > mcp4531_cfg[data->devid].max_pos || val < 0)
+ if (val > data->cfg->max_pos || val < 0)
return -EINVAL;
break;
default:
@@ -152,7 +152,6 @@
const struct i2c_device_id *id)
{
struct device *dev = &client->dev;
- unsigned long devid = id->driver_data;
struct mcp4531_data *data;
struct iio_dev *indio_dev;
@@ -168,12 +167,12 @@
data = iio_priv(indio_dev);
i2c_set_clientdata(client, indio_dev);
data->client = client;
- data->devid = devid;
+ data->cfg = &mcp4531_cfg[id->driver_data];
indio_dev->dev.parent = dev;
indio_dev->info = &mcp4531_info;
indio_dev->channels = mcp4531_channels;
- indio_dev->num_channels = mcp4531_cfg[devid].wipers;
+ indio_dev->num_channels = data->cfg->wipers;
indio_dev->name = client->name;
return devm_iio_device_register(dev, indio_dev);
diff --git a/drivers/iio/potentiometer/tpl0102.c b/drivers/iio/potentiometer/tpl0102.c
index 313124b..5c304d4 100644
--- a/drivers/iio/potentiometer/tpl0102.c
+++ b/drivers/iio/potentiometer/tpl0102.c
@@ -118,7 +118,7 @@
if (!i2c_check_functionality(client->adapter,
I2C_FUNC_SMBUS_WORD_DATA))
- return -ENOTSUPP;
+ return -EOPNOTSUPP;
indio_dev = devm_iio_device_alloc(dev, sizeof(*data));
if (!indio_dev)
diff --git a/drivers/iio/pressure/Kconfig b/drivers/iio/pressure/Kconfig
index 31c0e1f..cda9f12 100644
--- a/drivers/iio/pressure/Kconfig
+++ b/drivers/iio/pressure/Kconfig
@@ -6,12 +6,13 @@
menu "Pressure sensors"
config BMP280
- tristate "Bosch Sensortec BMP280 pressure sensor driver"
+ tristate "Bosch Sensortec BMP180 and BMP280 pressure sensor driver"
depends on I2C
+ depends on !(BMP085_I2C=y || BMP085_I2C=m)
select REGMAP_I2C
help
- Say yes here to build support for Bosch Sensortec BMP280
- pressure and temperature sensor.
+ Say yes here to build support for Bosch Sensortec BMP180 and BMP280
+ pressure and temperature sensors.
To compile this driver as a module, choose M here: the module
will be called bmp280.
@@ -30,6 +31,17 @@
To compile this driver as a module, choose M here: the module
will be called hid-sensor-press.
+config HP03
+ tristate "Hope RF HP03 temperature and pressure sensor driver"
+ depends on I2C
+ select REGMAP_I2C
+ help
+ Say yes here to build support for Hope RF HP03 pressure and
+ temperature sensor.
+
+ To compile this driver as a module, choose M here: the module
+ will be called hp03.
+
config MPL115
tristate
@@ -148,4 +160,14 @@
To compile this driver as a module, choose M here: the module
will be called t5403.
+config HP206C
+ tristate "HOPERF HP206C precision barometer and altimeter sensor"
+ depends on I2C
+ help
+ Say yes here to build support for the HOPREF HP206C precision
+ barometer and altimeter sensor.
+
+ This driver can also be built as a module. If so, the module will
+ be called hp206c.
+
endmenu
diff --git a/drivers/iio/pressure/Makefile b/drivers/iio/pressure/Makefile
index d336af1..17d6e7a 100644
--- a/drivers/iio/pressure/Makefile
+++ b/drivers/iio/pressure/Makefile
@@ -5,6 +5,7 @@
# When adding new entries keep the list in alphabetical order
obj-$(CONFIG_BMP280) += bmp280.o
obj-$(CONFIG_HID_SENSOR_PRESS) += hid-sensor-press.o
+obj-$(CONFIG_HP03) += hp03.o
obj-$(CONFIG_MPL115) += mpl115.o
obj-$(CONFIG_MPL115_I2C) += mpl115_i2c.o
obj-$(CONFIG_MPL115_SPI) += mpl115_spi.o
@@ -17,6 +18,7 @@
st_pressure-y := st_pressure_core.o
st_pressure-$(CONFIG_IIO_BUFFER) += st_pressure_buffer.o
obj-$(CONFIG_T5403) += t5403.o
+obj-$(CONFIG_HP206C) += hp206c.o
obj-$(CONFIG_IIO_ST_PRESS_I2C) += st_pressure_i2c.o
obj-$(CONFIG_IIO_ST_PRESS_SPI) += st_pressure_spi.o
diff --git a/drivers/iio/pressure/bmp280.c b/drivers/iio/pressure/bmp280.c
index a2602d8..2f1498e 100644
--- a/drivers/iio/pressure/bmp280.c
+++ b/drivers/iio/pressure/bmp280.c
@@ -1,12 +1,15 @@
/*
* Copyright (c) 2014 Intel Corporation
*
- * Driver for Bosch Sensortec BMP280 digital pressure sensor.
+ * Driver for Bosch Sensortec BMP180 and BMP280 digital pressure sensor.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*
+ * Datasheet:
+ * https://ae-bst.resource.bosch.com/media/_tech/media/datasheets/BST-BMP180-DS000-121.pdf
+ * https://ae-bst.resource.bosch.com/media/_tech/media/datasheets/BST-BMP280-DS001-12.pdf
*/
#define pr_fmt(fmt) "bmp280: " fmt
@@ -15,9 +18,11 @@
#include <linux/i2c.h>
#include <linux/acpi.h>
#include <linux/regmap.h>
+#include <linux/delay.h>
#include <linux/iio/iio.h>
#include <linux/iio/sysfs.h>
+/* BMP280 specific registers */
#define BMP280_REG_TEMP_XLSB 0xFC
#define BMP280_REG_TEMP_LSB 0xFB
#define BMP280_REG_TEMP_MSB 0xFA
@@ -26,10 +31,7 @@
#define BMP280_REG_PRESS_MSB 0xF7
#define BMP280_REG_CONFIG 0xF5
-#define BMP280_REG_CTRL_MEAS 0xF4
#define BMP280_REG_STATUS 0xF3
-#define BMP280_REG_RESET 0xE0
-#define BMP280_REG_ID 0xD0
#define BMP280_REG_COMP_TEMP_START 0x88
#define BMP280_COMP_TEMP_REG_COUNT 6
@@ -46,25 +48,49 @@
#define BMP280_OSRS_TEMP_MASK (BIT(7) | BIT(6) | BIT(5))
#define BMP280_OSRS_TEMP_SKIP 0
-#define BMP280_OSRS_TEMP_1X BIT(5)
-#define BMP280_OSRS_TEMP_2X BIT(6)
-#define BMP280_OSRS_TEMP_4X (BIT(6) | BIT(5))
-#define BMP280_OSRS_TEMP_8X BIT(7)
-#define BMP280_OSRS_TEMP_16X (BIT(7) | BIT(5))
+#define BMP280_OSRS_TEMP_X(osrs_t) ((osrs_t) << 5)
+#define BMP280_OSRS_TEMP_1X BMP280_OSRS_TEMP_X(1)
+#define BMP280_OSRS_TEMP_2X BMP280_OSRS_TEMP_X(2)
+#define BMP280_OSRS_TEMP_4X BMP280_OSRS_TEMP_X(3)
+#define BMP280_OSRS_TEMP_8X BMP280_OSRS_TEMP_X(4)
+#define BMP280_OSRS_TEMP_16X BMP280_OSRS_TEMP_X(5)
#define BMP280_OSRS_PRESS_MASK (BIT(4) | BIT(3) | BIT(2))
#define BMP280_OSRS_PRESS_SKIP 0
-#define BMP280_OSRS_PRESS_1X BIT(2)
-#define BMP280_OSRS_PRESS_2X BIT(3)
-#define BMP280_OSRS_PRESS_4X (BIT(3) | BIT(2))
-#define BMP280_OSRS_PRESS_8X BIT(4)
-#define BMP280_OSRS_PRESS_16X (BIT(4) | BIT(2))
+#define BMP280_OSRS_PRESS_X(osrs_p) ((osrs_p) << 2)
+#define BMP280_OSRS_PRESS_1X BMP280_OSRS_PRESS_X(1)
+#define BMP280_OSRS_PRESS_2X BMP280_OSRS_PRESS_X(2)
+#define BMP280_OSRS_PRESS_4X BMP280_OSRS_PRESS_X(3)
+#define BMP280_OSRS_PRESS_8X BMP280_OSRS_PRESS_X(4)
+#define BMP280_OSRS_PRESS_16X BMP280_OSRS_PRESS_X(5)
#define BMP280_MODE_MASK (BIT(1) | BIT(0))
#define BMP280_MODE_SLEEP 0
#define BMP280_MODE_FORCED BIT(0)
#define BMP280_MODE_NORMAL (BIT(1) | BIT(0))
+/* BMP180 specific registers */
+#define BMP180_REG_OUT_XLSB 0xF8
+#define BMP180_REG_OUT_LSB 0xF7
+#define BMP180_REG_OUT_MSB 0xF6
+
+#define BMP180_REG_CALIB_START 0xAA
+#define BMP180_REG_CALIB_COUNT 22
+
+#define BMP180_MEAS_SCO BIT(5)
+#define BMP180_MEAS_TEMP (0x0E | BMP180_MEAS_SCO)
+#define BMP180_MEAS_PRESS_X(oss) ((oss) << 6 | 0x14 | BMP180_MEAS_SCO)
+#define BMP180_MEAS_PRESS_1X BMP180_MEAS_PRESS_X(0)
+#define BMP180_MEAS_PRESS_2X BMP180_MEAS_PRESS_X(1)
+#define BMP180_MEAS_PRESS_4X BMP180_MEAS_PRESS_X(2)
+#define BMP180_MEAS_PRESS_8X BMP180_MEAS_PRESS_X(3)
+
+/* BMP180 and BMP280 common registers */
+#define BMP280_REG_CTRL_MEAS 0xF4
+#define BMP280_REG_RESET 0xE0
+#define BMP280_REG_ID 0xD0
+
+#define BMP180_CHIP_ID 0x55
#define BMP280_CHIP_ID 0x58
#define BMP280_SOFT_RESET_VAL 0xB6
@@ -72,6 +98,11 @@
struct i2c_client *client;
struct mutex lock;
struct regmap *regmap;
+ const struct bmp280_chip_info *chip_info;
+
+ /* log of base 2 of oversampling rate */
+ u8 oversampling_press;
+ u8 oversampling_temp;
/*
* Carryover value from temperature conversion, used in pressure
@@ -80,9 +111,23 @@
s32 t_fine;
};
+struct bmp280_chip_info {
+ const struct regmap_config *regmap_config;
+
+ const int *oversampling_temp_avail;
+ int num_oversampling_temp_avail;
+
+ const int *oversampling_press_avail;
+ int num_oversampling_press_avail;
+
+ int (*chip_config)(struct bmp280_data *);
+ int (*read_temp)(struct bmp280_data *, int *);
+ int (*read_press)(struct bmp280_data *, int *, int *);
+};
+
/*
* These enums are used for indexing into the array of compensation
- * parameters.
+ * parameters for BMP280.
*/
enum { T1, T2, T3 };
enum { P1, P2, P3, P4, P5, P6, P7, P8, P9 };
@@ -90,11 +135,13 @@
static const struct iio_chan_spec bmp280_channels[] = {
{
.type = IIO_PRESSURE,
- .info_mask_separate = BIT(IIO_CHAN_INFO_PROCESSED),
+ .info_mask_separate = BIT(IIO_CHAN_INFO_PROCESSED) |
+ BIT(IIO_CHAN_INFO_OVERSAMPLING_RATIO),
},
{
.type = IIO_TEMP,
- .info_mask_separate = BIT(IIO_CHAN_INFO_PROCESSED),
+ .info_mask_separate = BIT(IIO_CHAN_INFO_PROCESSED) |
+ BIT(IIO_CHAN_INFO_OVERSAMPLING_RATIO),
},
};
@@ -290,10 +337,25 @@
case IIO_CHAN_INFO_PROCESSED:
switch (chan->type) {
case IIO_PRESSURE:
- ret = bmp280_read_press(data, val, val2);
+ ret = data->chip_info->read_press(data, val, val2);
break;
case IIO_TEMP:
- ret = bmp280_read_temp(data, val);
+ ret = data->chip_info->read_temp(data, val);
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+ break;
+ case IIO_CHAN_INFO_OVERSAMPLING_RATIO:
+ switch (chan->type) {
+ case IIO_PRESSURE:
+ *val = 1 << data->oversampling_press;
+ ret = IIO_VAL_INT;
+ break;
+ case IIO_TEMP:
+ *val = 1 << data->oversampling_temp;
+ ret = IIO_VAL_INT;
break;
default:
ret = -EINVAL;
@@ -310,22 +372,135 @@
return ret;
}
+static int bmp280_write_oversampling_ratio_temp(struct bmp280_data *data,
+ int val)
+{
+ int i;
+ const int *avail = data->chip_info->oversampling_temp_avail;
+ const int n = data->chip_info->num_oversampling_temp_avail;
+
+ for (i = 0; i < n; i++) {
+ if (avail[i] == val) {
+ data->oversampling_temp = ilog2(val);
+
+ return data->chip_info->chip_config(data);
+ }
+ }
+ return -EINVAL;
+}
+
+static int bmp280_write_oversampling_ratio_press(struct bmp280_data *data,
+ int val)
+{
+ int i;
+ const int *avail = data->chip_info->oversampling_press_avail;
+ const int n = data->chip_info->num_oversampling_press_avail;
+
+ for (i = 0; i < n; i++) {
+ if (avail[i] == val) {
+ data->oversampling_press = ilog2(val);
+
+ return data->chip_info->chip_config(data);
+ }
+ }
+ return -EINVAL;
+}
+
+static int bmp280_write_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int val, int val2, long mask)
+{
+ int ret = 0;
+ struct bmp280_data *data = iio_priv(indio_dev);
+
+ switch (mask) {
+ case IIO_CHAN_INFO_OVERSAMPLING_RATIO:
+ mutex_lock(&data->lock);
+ switch (chan->type) {
+ case IIO_PRESSURE:
+ ret = bmp280_write_oversampling_ratio_press(data, val);
+ break;
+ case IIO_TEMP:
+ ret = bmp280_write_oversampling_ratio_temp(data, val);
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+ mutex_unlock(&data->lock);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return ret;
+}
+
+static ssize_t bmp280_show_avail(char *buf, const int *vals, const int n)
+{
+ size_t len = 0;
+ int i;
+
+ for (i = 0; i < n; i++)
+ len += scnprintf(buf + len, PAGE_SIZE - len, "%d ", vals[i]);
+
+ buf[len - 1] = '\n';
+
+ return len;
+}
+
+static ssize_t bmp280_show_temp_oversampling_avail(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct bmp280_data *data = iio_priv(dev_to_iio_dev(dev));
+
+ return bmp280_show_avail(buf, data->chip_info->oversampling_temp_avail,
+ data->chip_info->num_oversampling_temp_avail);
+}
+
+static ssize_t bmp280_show_press_oversampling_avail(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct bmp280_data *data = iio_priv(dev_to_iio_dev(dev));
+
+ return bmp280_show_avail(buf, data->chip_info->oversampling_press_avail,
+ data->chip_info->num_oversampling_press_avail);
+}
+
+static IIO_DEVICE_ATTR(in_temp_oversampling_ratio_available,
+ S_IRUGO, bmp280_show_temp_oversampling_avail, NULL, 0);
+
+static IIO_DEVICE_ATTR(in_pressure_oversampling_ratio_available,
+ S_IRUGO, bmp280_show_press_oversampling_avail, NULL, 0);
+
+static struct attribute *bmp280_attributes[] = {
+ &iio_dev_attr_in_temp_oversampling_ratio_available.dev_attr.attr,
+ &iio_dev_attr_in_pressure_oversampling_ratio_available.dev_attr.attr,
+ NULL,
+};
+
+static const struct attribute_group bmp280_attrs_group = {
+ .attrs = bmp280_attributes,
+};
+
static const struct iio_info bmp280_info = {
.driver_module = THIS_MODULE,
.read_raw = &bmp280_read_raw,
+ .write_raw = &bmp280_write_raw,
+ .attrs = &bmp280_attrs_group,
};
-static int bmp280_chip_init(struct bmp280_data *data)
+static int bmp280_chip_config(struct bmp280_data *data)
{
int ret;
+ u8 osrs = BMP280_OSRS_TEMP_X(data->oversampling_temp + 1) |
+ BMP280_OSRS_PRESS_X(data->oversampling_press + 1);
ret = regmap_update_bits(data->regmap, BMP280_REG_CTRL_MEAS,
BMP280_OSRS_TEMP_MASK |
BMP280_OSRS_PRESS_MASK |
BMP280_MODE_MASK,
- BMP280_OSRS_TEMP_2X |
- BMP280_OSRS_PRESS_16X |
- BMP280_MODE_NORMAL);
+ osrs | BMP280_MODE_NORMAL);
if (ret < 0) {
dev_err(&data->client->dev,
"failed to write ctrl_meas register\n");
@@ -344,6 +519,317 @@
return ret;
}
+static const int bmp280_oversampling_avail[] = { 1, 2, 4, 8, 16 };
+
+static const struct bmp280_chip_info bmp280_chip_info = {
+ .regmap_config = &bmp280_regmap_config,
+
+ .oversampling_temp_avail = bmp280_oversampling_avail,
+ .num_oversampling_temp_avail = ARRAY_SIZE(bmp280_oversampling_avail),
+
+ .oversampling_press_avail = bmp280_oversampling_avail,
+ .num_oversampling_press_avail = ARRAY_SIZE(bmp280_oversampling_avail),
+
+ .chip_config = bmp280_chip_config,
+ .read_temp = bmp280_read_temp,
+ .read_press = bmp280_read_press,
+};
+
+static bool bmp180_is_writeable_reg(struct device *dev, unsigned int reg)
+{
+ switch (reg) {
+ case BMP280_REG_CTRL_MEAS:
+ case BMP280_REG_RESET:
+ return true;
+ default:
+ return false;
+ };
+}
+
+static bool bmp180_is_volatile_reg(struct device *dev, unsigned int reg)
+{
+ switch (reg) {
+ case BMP180_REG_OUT_XLSB:
+ case BMP180_REG_OUT_LSB:
+ case BMP180_REG_OUT_MSB:
+ case BMP280_REG_CTRL_MEAS:
+ return true;
+ default:
+ return false;
+ }
+}
+
+static const struct regmap_config bmp180_regmap_config = {
+ .reg_bits = 8,
+ .val_bits = 8,
+
+ .max_register = BMP180_REG_OUT_XLSB,
+ .cache_type = REGCACHE_RBTREE,
+
+ .writeable_reg = bmp180_is_writeable_reg,
+ .volatile_reg = bmp180_is_volatile_reg,
+};
+
+static int bmp180_measure(struct bmp280_data *data, u8 ctrl_meas)
+{
+ int ret;
+ const int conversion_time_max[] = { 4500, 7500, 13500, 25500 };
+ unsigned int delay_us;
+ unsigned int ctrl;
+
+ ret = regmap_write(data->regmap, BMP280_REG_CTRL_MEAS, ctrl_meas);
+ if (ret)
+ return ret;
+
+ if (ctrl_meas == BMP180_MEAS_TEMP)
+ delay_us = 4500;
+ else
+ delay_us = conversion_time_max[data->oversampling_press];
+
+ usleep_range(delay_us, delay_us + 1000);
+
+ ret = regmap_read(data->regmap, BMP280_REG_CTRL_MEAS, &ctrl);
+ if (ret)
+ return ret;
+
+ /* The value of this bit reset to "0" after conversion is complete */
+ if (ctrl & BMP180_MEAS_SCO)
+ return -EIO;
+
+ return 0;
+}
+
+static int bmp180_read_adc_temp(struct bmp280_data *data, int *val)
+{
+ int ret;
+ __be16 tmp = 0;
+
+ ret = bmp180_measure(data, BMP180_MEAS_TEMP);
+ if (ret)
+ return ret;
+
+ ret = regmap_bulk_read(data->regmap, BMP180_REG_OUT_MSB, (u8 *)&tmp, 2);
+ if (ret)
+ return ret;
+
+ *val = be16_to_cpu(tmp);
+
+ return 0;
+}
+
+/*
+ * These enums are used for indexing into the array of calibration
+ * coefficients for BMP180.
+ */
+enum { AC1, AC2, AC3, AC4, AC5, AC6, B1, B2, MB, MC, MD };
+
+struct bmp180_calib {
+ s16 AC1;
+ s16 AC2;
+ s16 AC3;
+ u16 AC4;
+ u16 AC5;
+ u16 AC6;
+ s16 B1;
+ s16 B2;
+ s16 MB;
+ s16 MC;
+ s16 MD;
+};
+
+static int bmp180_read_calib(struct bmp280_data *data,
+ struct bmp180_calib *calib)
+{
+ int ret;
+ int i;
+ __be16 buf[BMP180_REG_CALIB_COUNT / 2];
+
+ ret = regmap_bulk_read(data->regmap, BMP180_REG_CALIB_START, buf,
+ sizeof(buf));
+
+ if (ret < 0)
+ return ret;
+
+ /* None of the words has the value 0 or 0xFFFF */
+ for (i = 0; i < ARRAY_SIZE(buf); i++) {
+ if (buf[i] == cpu_to_be16(0) || buf[i] == cpu_to_be16(0xffff))
+ return -EIO;
+ }
+
+ calib->AC1 = be16_to_cpu(buf[AC1]);
+ calib->AC2 = be16_to_cpu(buf[AC2]);
+ calib->AC3 = be16_to_cpu(buf[AC3]);
+ calib->AC4 = be16_to_cpu(buf[AC4]);
+ calib->AC5 = be16_to_cpu(buf[AC5]);
+ calib->AC6 = be16_to_cpu(buf[AC6]);
+ calib->B1 = be16_to_cpu(buf[B1]);
+ calib->B2 = be16_to_cpu(buf[B2]);
+ calib->MB = be16_to_cpu(buf[MB]);
+ calib->MC = be16_to_cpu(buf[MC]);
+ calib->MD = be16_to_cpu(buf[MD]);
+
+ return 0;
+}
+
+/*
+ * Returns temperature in DegC, resolution is 0.1 DegC.
+ * t_fine carries fine temperature as global value.
+ *
+ * Taken from datasheet, Section 3.5, "Calculating pressure and temperature".
+ */
+static s32 bmp180_compensate_temp(struct bmp280_data *data, s32 adc_temp)
+{
+ int ret;
+ s32 x1, x2;
+ struct bmp180_calib calib;
+
+ ret = bmp180_read_calib(data, &calib);
+ if (ret < 0) {
+ dev_err(&data->client->dev,
+ "failed to read calibration coefficients\n");
+ return ret;
+ }
+
+ x1 = ((adc_temp - calib.AC6) * calib.AC5) >> 15;
+ x2 = (calib.MC << 11) / (x1 + calib.MD);
+ data->t_fine = x1 + x2;
+
+ return (data->t_fine + 8) >> 4;
+}
+
+static int bmp180_read_temp(struct bmp280_data *data, int *val)
+{
+ int ret;
+ s32 adc_temp, comp_temp;
+
+ ret = bmp180_read_adc_temp(data, &adc_temp);
+ if (ret)
+ return ret;
+
+ comp_temp = bmp180_compensate_temp(data, adc_temp);
+
+ /*
+ * val might be NULL if we're called by the read_press routine,
+ * who only cares about the carry over t_fine value.
+ */
+ if (val) {
+ *val = comp_temp * 100;
+ return IIO_VAL_INT;
+ }
+
+ return 0;
+}
+
+static int bmp180_read_adc_press(struct bmp280_data *data, int *val)
+{
+ int ret;
+ __be32 tmp = 0;
+ u8 oss = data->oversampling_press;
+
+ ret = bmp180_measure(data, BMP180_MEAS_PRESS_X(oss));
+ if (ret)
+ return ret;
+
+ ret = regmap_bulk_read(data->regmap, BMP180_REG_OUT_MSB, (u8 *)&tmp, 3);
+ if (ret)
+ return ret;
+
+ *val = (be32_to_cpu(tmp) >> 8) >> (8 - oss);
+
+ return 0;
+}
+
+/*
+ * Returns pressure in Pa, resolution is 1 Pa.
+ *
+ * Taken from datasheet, Section 3.5, "Calculating pressure and temperature".
+ */
+static u32 bmp180_compensate_press(struct bmp280_data *data, s32 adc_press)
+{
+ int ret;
+ s32 x1, x2, x3, p;
+ s32 b3, b6;
+ u32 b4, b7;
+ s32 oss = data->oversampling_press;
+ struct bmp180_calib calib;
+
+ ret = bmp180_read_calib(data, &calib);
+ if (ret < 0) {
+ dev_err(&data->client->dev,
+ "failed to read calibration coefficients\n");
+ return ret;
+ }
+
+ b6 = data->t_fine - 4000;
+ x1 = (calib.B2 * (b6 * b6 >> 12)) >> 11;
+ x2 = calib.AC2 * b6 >> 11;
+ x3 = x1 + x2;
+ b3 = ((((s32)calib.AC1 * 4 + x3) << oss) + 2) / 4;
+ x1 = calib.AC3 * b6 >> 13;
+ x2 = (calib.B1 * ((b6 * b6) >> 12)) >> 16;
+ x3 = (x1 + x2 + 2) >> 2;
+ b4 = calib.AC4 * (u32)(x3 + 32768) >> 15;
+ b7 = ((u32)adc_press - b3) * (50000 >> oss);
+ if (b7 < 0x80000000)
+ p = (b7 * 2) / b4;
+ else
+ p = (b7 / b4) * 2;
+
+ x1 = (p >> 8) * (p >> 8);
+ x1 = (x1 * 3038) >> 16;
+ x2 = (-7357 * p) >> 16;
+
+ return p + ((x1 + x2 + 3791) >> 4);
+}
+
+static int bmp180_read_press(struct bmp280_data *data,
+ int *val, int *val2)
+{
+ int ret;
+ s32 adc_press;
+ u32 comp_press;
+
+ /* Read and compensate temperature so we get a reading of t_fine. */
+ ret = bmp180_read_temp(data, NULL);
+ if (ret)
+ return ret;
+
+ ret = bmp180_read_adc_press(data, &adc_press);
+ if (ret)
+ return ret;
+
+ comp_press = bmp180_compensate_press(data, adc_press);
+
+ *val = comp_press;
+ *val2 = 1000;
+
+ return IIO_VAL_FRACTIONAL;
+}
+
+static int bmp180_chip_config(struct bmp280_data *data)
+{
+ return 0;
+}
+
+static const int bmp180_oversampling_temp_avail[] = { 1 };
+static const int bmp180_oversampling_press_avail[] = { 1, 2, 4, 8 };
+
+static const struct bmp280_chip_info bmp180_chip_info = {
+ .regmap_config = &bmp180_regmap_config,
+
+ .oversampling_temp_avail = bmp180_oversampling_temp_avail,
+ .num_oversampling_temp_avail =
+ ARRAY_SIZE(bmp180_oversampling_temp_avail),
+
+ .oversampling_press_avail = bmp180_oversampling_press_avail,
+ .num_oversampling_press_avail =
+ ARRAY_SIZE(bmp180_oversampling_press_avail),
+
+ .chip_config = bmp180_chip_config,
+ .read_temp = bmp180_read_temp,
+ .read_press = bmp180_read_press,
+};
+
static int bmp280_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
@@ -367,7 +853,23 @@
indio_dev->info = &bmp280_info;
indio_dev->modes = INDIO_DIRECT_MODE;
- data->regmap = devm_regmap_init_i2c(client, &bmp280_regmap_config);
+ switch (id->driver_data) {
+ case BMP180_CHIP_ID:
+ data->chip_info = &bmp180_chip_info;
+ data->oversampling_press = ilog2(8);
+ data->oversampling_temp = ilog2(1);
+ break;
+ case BMP280_CHIP_ID:
+ data->chip_info = &bmp280_chip_info;
+ data->oversampling_press = ilog2(16);
+ data->oversampling_temp = ilog2(2);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ data->regmap = devm_regmap_init_i2c(client,
+ data->chip_info->regmap_config);
if (IS_ERR(data->regmap)) {
dev_err(&client->dev, "failed to allocate register map\n");
return PTR_ERR(data->regmap);
@@ -376,13 +878,13 @@
ret = regmap_read(data->regmap, BMP280_REG_ID, &chip_id);
if (ret < 0)
return ret;
- if (chip_id != BMP280_CHIP_ID) {
+ if (chip_id != id->driver_data) {
dev_err(&client->dev, "bad chip id. expected %x got %x\n",
BMP280_CHIP_ID, chip_id);
return -EINVAL;
}
- ret = bmp280_chip_init(data);
+ ret = data->chip_info->chip_config(data);
if (ret < 0)
return ret;
@@ -390,13 +892,17 @@
}
static const struct acpi_device_id bmp280_acpi_match[] = {
- {"BMP0280", 0},
+ {"BMP0280", BMP280_CHIP_ID },
+ {"BMP0180", BMP180_CHIP_ID },
+ {"BMP0085", BMP180_CHIP_ID },
{ },
};
MODULE_DEVICE_TABLE(acpi, bmp280_acpi_match);
static const struct i2c_device_id bmp280_id[] = {
- {"bmp280", 0},
+ {"bmp280", BMP280_CHIP_ID },
+ {"bmp180", BMP180_CHIP_ID },
+ {"bmp085", BMP180_CHIP_ID },
{ },
};
MODULE_DEVICE_TABLE(i2c, bmp280_id);
@@ -412,5 +918,5 @@
module_i2c_driver(bmp280_driver);
MODULE_AUTHOR("Vlad Dogaru <vlad.dogaru@intel.com>");
-MODULE_DESCRIPTION("Driver for Bosch Sensortec BMP280 pressure and temperature sensor");
+MODULE_DESCRIPTION("Driver for Bosch Sensortec BMP180/BMP280 pressure and temperature sensor");
MODULE_LICENSE("GPL v2");
diff --git a/drivers/iio/pressure/hp03.c b/drivers/iio/pressure/hp03.c
new file mode 100644
index 0000000..ac76515
--- /dev/null
+++ b/drivers/iio/pressure/hp03.c
@@ -0,0 +1,312 @@
+/*
+ * Copyright (c) 2016 Marek Vasut <marex@denx.de>
+ *
+ * Driver for Hope RF HP03 digital temperature and pressure sensor.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#define pr_fmt(fmt) "hp03: " fmt
+
+#include <linux/module.h>
+#include <linux/delay.h>
+#include <linux/gpio/consumer.h>
+#include <linux/i2c.h>
+#include <linux/regmap.h>
+#include <linux/iio/iio.h>
+#include <linux/iio/sysfs.h>
+
+/*
+ * The HP03 sensor occupies two fixed I2C addresses:
+ * 0x50 ... read-only EEPROM with calibration data
+ * 0x77 ... read-write ADC for pressure and temperature
+ */
+#define HP03_EEPROM_ADDR 0x50
+#define HP03_ADC_ADDR 0x77
+
+#define HP03_EEPROM_CX_OFFSET 0x10
+#define HP03_EEPROM_AB_OFFSET 0x1e
+#define HP03_EEPROM_CD_OFFSET 0x20
+
+#define HP03_ADC_WRITE_REG 0xff
+#define HP03_ADC_READ_REG 0xfd
+#define HP03_ADC_READ_PRESSURE 0xf0 /* D1 in datasheet */
+#define HP03_ADC_READ_TEMP 0xe8 /* D2 in datasheet */
+
+struct hp03_priv {
+ struct i2c_client *client;
+ struct mutex lock;
+ struct gpio_desc *xclr_gpio;
+
+ struct i2c_client *eeprom_client;
+ struct regmap *eeprom_regmap;
+
+ s32 pressure; /* kPa */
+ s32 temp; /* Deg. C */
+};
+
+static const struct iio_chan_spec hp03_channels[] = {
+ {
+ .type = IIO_PRESSURE,
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW),
+ .info_mask_shared_by_type = BIT(IIO_CHAN_INFO_SCALE),
+ },
+ {
+ .type = IIO_TEMP,
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW),
+ .info_mask_shared_by_type = BIT(IIO_CHAN_INFO_SCALE),
+ },
+};
+
+static bool hp03_is_writeable_reg(struct device *dev, unsigned int reg)
+{
+ return false;
+}
+
+static bool hp03_is_volatile_reg(struct device *dev, unsigned int reg)
+{
+ return false;
+}
+
+static const struct regmap_config hp03_regmap_config = {
+ .reg_bits = 8,
+ .val_bits = 8,
+
+ .max_register = HP03_EEPROM_CD_OFFSET + 1,
+ .cache_type = REGCACHE_RBTREE,
+
+ .writeable_reg = hp03_is_writeable_reg,
+ .volatile_reg = hp03_is_volatile_reg,
+};
+
+static int hp03_get_temp_pressure(struct hp03_priv *priv, const u8 reg)
+{
+ int ret;
+
+ ret = i2c_smbus_write_byte_data(priv->client, HP03_ADC_WRITE_REG, reg);
+ if (ret < 0)
+ return ret;
+
+ msleep(50); /* Wait for conversion to finish */
+
+ return i2c_smbus_read_word_data(priv->client, HP03_ADC_READ_REG);
+}
+
+static int hp03_update_temp_pressure(struct hp03_priv *priv)
+{
+ struct device *dev = &priv->client->dev;
+ u8 coefs[18];
+ u16 cx_val[7];
+ int ab_val, d1_val, d2_val, diff_val, dut, off, sens, x;
+ int i, ret;
+
+ /* Sample coefficients from EEPROM */
+ ret = regmap_bulk_read(priv->eeprom_regmap, HP03_EEPROM_CX_OFFSET,
+ coefs, sizeof(coefs));
+ if (ret < 0) {
+ dev_err(dev, "Failed to read EEPROM (reg=%02x)\n",
+ HP03_EEPROM_CX_OFFSET);
+ return ret;
+ }
+
+ /* Sample Temperature and Pressure */
+ gpiod_set_value_cansleep(priv->xclr_gpio, 1);
+
+ ret = hp03_get_temp_pressure(priv, HP03_ADC_READ_PRESSURE);
+ if (ret < 0) {
+ dev_err(dev, "Failed to read pressure\n");
+ goto err_adc;
+ }
+ d1_val = ret;
+
+ ret = hp03_get_temp_pressure(priv, HP03_ADC_READ_TEMP);
+ if (ret < 0) {
+ dev_err(dev, "Failed to read temperature\n");
+ goto err_adc;
+ }
+ d2_val = ret;
+
+ gpiod_set_value_cansleep(priv->xclr_gpio, 0);
+
+ /* The Cx coefficients and Temp/Pressure values are MSB first. */
+ for (i = 0; i < 7; i++)
+ cx_val[i] = (coefs[2 * i] << 8) | (coefs[(2 * i) + 1] << 0);
+ d1_val = ((d1_val >> 8) & 0xff) | ((d1_val & 0xff) << 8);
+ d2_val = ((d2_val >> 8) & 0xff) | ((d2_val & 0xff) << 8);
+
+ /* Coefficient voodoo from the HP03 datasheet. */
+ if (d2_val >= cx_val[4])
+ ab_val = coefs[14]; /* A-value */
+ else
+ ab_val = coefs[15]; /* B-value */
+
+ diff_val = d2_val - cx_val[4];
+ dut = (ab_val * (diff_val >> 7) * (diff_val >> 7)) >> coefs[16];
+ dut = diff_val - dut;
+
+ off = (cx_val[1] + (((cx_val[3] - 1024) * dut) >> 14)) * 4;
+ sens = cx_val[0] + ((cx_val[2] * dut) >> 10);
+ x = ((sens * (d1_val - 7168)) >> 14) - off;
+
+ priv->pressure = ((x * 100) >> 5) + (cx_val[6] * 10);
+ priv->temp = 250 + ((dut * cx_val[5]) >> 16) - (dut >> coefs[17]);
+
+ return 0;
+
+err_adc:
+ gpiod_set_value_cansleep(priv->xclr_gpio, 0);
+ return ret;
+}
+
+static int hp03_read_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int *val, int *val2, long mask)
+{
+ struct hp03_priv *priv = iio_priv(indio_dev);
+ int ret;
+
+ mutex_lock(&priv->lock);
+ ret = hp03_update_temp_pressure(priv);
+ mutex_unlock(&priv->lock);
+
+ if (ret)
+ return ret;
+
+ switch (mask) {
+ case IIO_CHAN_INFO_RAW:
+ switch (chan->type) {
+ case IIO_PRESSURE:
+ *val = priv->pressure;
+ return IIO_VAL_INT;
+ case IIO_TEMP:
+ *val = priv->temp;
+ return IIO_VAL_INT;
+ default:
+ return -EINVAL;
+ }
+ break;
+ case IIO_CHAN_INFO_SCALE:
+ switch (chan->type) {
+ case IIO_PRESSURE:
+ *val = 0;
+ *val2 = 1000;
+ return IIO_VAL_INT_PLUS_MICRO;
+ case IIO_TEMP:
+ *val = 10;
+ return IIO_VAL_INT;
+ default:
+ return -EINVAL;
+ }
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return -EINVAL;
+}
+
+static const struct iio_info hp03_info = {
+ .driver_module = THIS_MODULE,
+ .read_raw = &hp03_read_raw,
+};
+
+static int hp03_probe(struct i2c_client *client,
+ const struct i2c_device_id *id)
+{
+ struct device *dev = &client->dev;
+ struct iio_dev *indio_dev;
+ struct hp03_priv *priv;
+ int ret;
+
+ indio_dev = devm_iio_device_alloc(dev, sizeof(*priv));
+ if (!indio_dev)
+ return -ENOMEM;
+
+ priv = iio_priv(indio_dev);
+ priv->client = client;
+ mutex_init(&priv->lock);
+
+ indio_dev->dev.parent = dev;
+ indio_dev->name = id->name;
+ indio_dev->channels = hp03_channels;
+ indio_dev->num_channels = ARRAY_SIZE(hp03_channels);
+ indio_dev->info = &hp03_info;
+ indio_dev->modes = INDIO_DIRECT_MODE;
+
+ priv->xclr_gpio = devm_gpiod_get_index(dev, "xclr", 0, GPIOD_OUT_HIGH);
+ if (IS_ERR(priv->xclr_gpio)) {
+ dev_err(dev, "Failed to claim XCLR GPIO\n");
+ ret = PTR_ERR(priv->xclr_gpio);
+ return ret;
+ }
+
+ /*
+ * Allocate another device for the on-sensor EEPROM,
+ * which has it's dedicated I2C address and contains
+ * the calibration constants for the sensor.
+ */
+ priv->eeprom_client = i2c_new_dummy(client->adapter, HP03_EEPROM_ADDR);
+ if (!priv->eeprom_client) {
+ dev_err(dev, "New EEPROM I2C device failed\n");
+ return -ENODEV;
+ }
+
+ priv->eeprom_regmap = regmap_init_i2c(priv->eeprom_client,
+ &hp03_regmap_config);
+ if (IS_ERR(priv->eeprom_regmap)) {
+ dev_err(dev, "Failed to allocate EEPROM regmap\n");
+ ret = PTR_ERR(priv->eeprom_regmap);
+ goto err_cleanup_eeprom_client;
+ }
+
+ ret = iio_device_register(indio_dev);
+ if (ret) {
+ dev_err(dev, "Failed to register IIO device\n");
+ goto err_cleanup_eeprom_regmap;
+ }
+
+ i2c_set_clientdata(client, indio_dev);
+
+ return 0;
+
+err_cleanup_eeprom_regmap:
+ regmap_exit(priv->eeprom_regmap);
+
+err_cleanup_eeprom_client:
+ i2c_unregister_device(priv->eeprom_client);
+ return ret;
+}
+
+static int hp03_remove(struct i2c_client *client)
+{
+ struct iio_dev *indio_dev = i2c_get_clientdata(client);
+ struct hp03_priv *priv = iio_priv(indio_dev);
+
+ iio_device_unregister(indio_dev);
+ regmap_exit(priv->eeprom_regmap);
+ i2c_unregister_device(priv->eeprom_client);
+
+ return 0;
+}
+
+static const struct i2c_device_id hp03_id[] = {
+ { "hp03", 0 },
+ { },
+};
+MODULE_DEVICE_TABLE(i2c, hp03_id);
+
+static struct i2c_driver hp03_driver = {
+ .driver = {
+ .name = "hp03",
+ },
+ .probe = hp03_probe,
+ .remove = hp03_remove,
+ .id_table = hp03_id,
+};
+module_i2c_driver(hp03_driver);
+
+MODULE_AUTHOR("Marek Vasut <marex@denx.de>");
+MODULE_DESCRIPTION("Driver for Hope RF HP03 pressure and temperature sensor");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/iio/pressure/hp206c.c b/drivers/iio/pressure/hp206c.c
new file mode 100644
index 0000000..90f2b6e
--- /dev/null
+++ b/drivers/iio/pressure/hp206c.c
@@ -0,0 +1,426 @@
+/*
+ * hp206c.c - HOPERF HP206C precision barometer and altimeter sensor
+ *
+ * Copyright (c) 2016, Intel Corporation.
+ *
+ * This file is subject to the terms and conditions of version 2 of
+ * the GNU General Public License. See the file COPYING in the main
+ * directory of this archive for more details.
+ *
+ * (7-bit I2C slave address 0x76)
+ *
+ * Datasheet:
+ * http://www.hoperf.com/upload/sensor/HP206C_DataSheet_EN_V2.0.pdf
+ */
+
+#include <linux/module.h>
+#include <linux/i2c.h>
+#include <linux/iio/iio.h>
+#include <linux/iio/sysfs.h>
+#include <linux/delay.h>
+#include <linux/util_macros.h>
+#include <linux/acpi.h>
+
+/* I2C commands: */
+#define HP206C_CMD_SOFT_RST 0x06
+
+#define HP206C_CMD_ADC_CVT 0x40
+
+#define HP206C_CMD_ADC_CVT_OSR_4096 0x00
+#define HP206C_CMD_ADC_CVT_OSR_2048 0x04
+#define HP206C_CMD_ADC_CVT_OSR_1024 0x08
+#define HP206C_CMD_ADC_CVT_OSR_512 0x0c
+#define HP206C_CMD_ADC_CVT_OSR_256 0x10
+#define HP206C_CMD_ADC_CVT_OSR_128 0x14
+
+#define HP206C_CMD_ADC_CVT_CHNL_PT 0x00
+#define HP206C_CMD_ADC_CVT_CHNL_T 0x02
+
+#define HP206C_CMD_READ_P 0x30
+#define HP206C_CMD_READ_T 0x32
+
+#define HP206C_CMD_READ_REG 0x80
+#define HP206C_CMD_WRITE_REG 0xc0
+
+#define HP206C_REG_INT_EN 0x0b
+#define HP206C_REG_INT_CFG 0x0c
+
+#define HP206C_REG_INT_SRC 0x0d
+#define HP206C_FLAG_DEV_RDY 0x40
+
+#define HP206C_REG_PARA 0x0f
+#define HP206C_FLAG_CMPS_EN 0x80
+
+/* Maximum spin for DEV_RDY */
+#define HP206C_MAX_DEV_RDY_WAIT_COUNT 20
+#define HP206C_DEV_RDY_WAIT_US 20000
+
+struct hp206c_data {
+ struct mutex mutex;
+ struct i2c_client *client;
+ int temp_osr_index;
+ int pres_osr_index;
+};
+
+struct hp206c_osr_setting {
+ u8 osr_mask;
+ unsigned int temp_conv_time_us;
+ unsigned int pres_conv_time_us;
+};
+
+/* Data from Table 5 in datasheet. */
+static const struct hp206c_osr_setting hp206c_osr_settings[] = {
+ { HP206C_CMD_ADC_CVT_OSR_4096, 65600, 131100 },
+ { HP206C_CMD_ADC_CVT_OSR_2048, 32800, 65600 },
+ { HP206C_CMD_ADC_CVT_OSR_1024, 16400, 32800 },
+ { HP206C_CMD_ADC_CVT_OSR_512, 8200, 16400 },
+ { HP206C_CMD_ADC_CVT_OSR_256, 4100, 8200 },
+ { HP206C_CMD_ADC_CVT_OSR_128, 2100, 4100 },
+};
+static const int hp206c_osr_rates[] = { 4096, 2048, 1024, 512, 256, 128 };
+static const char hp206c_osr_rates_str[] = "4096 2048 1024 512 256 128";
+
+static inline int hp206c_read_reg(struct i2c_client *client, u8 reg)
+{
+ return i2c_smbus_read_byte_data(client, HP206C_CMD_READ_REG | reg);
+}
+
+static inline int hp206c_write_reg(struct i2c_client *client, u8 reg, u8 val)
+{
+ return i2c_smbus_write_byte_data(client,
+ HP206C_CMD_WRITE_REG | reg, val);
+}
+
+static int hp206c_read_20bit(struct i2c_client *client, u8 cmd)
+{
+ int ret;
+ u8 values[3];
+
+ ret = i2c_smbus_read_i2c_block_data(client, cmd, 3, values);
+ if (ret < 0)
+ return ret;
+ if (ret != 3)
+ return -EIO;
+ return ((values[0] & 0xF) << 16) | (values[1] << 8) | (values[2]);
+}
+
+/* Spin for max 160ms until DEV_RDY is 1, or return error. */
+static int hp206c_wait_dev_rdy(struct iio_dev *indio_dev)
+{
+ int ret;
+ int count = 0;
+ struct hp206c_data *data = iio_priv(indio_dev);
+ struct i2c_client *client = data->client;
+
+ while (++count <= HP206C_MAX_DEV_RDY_WAIT_COUNT) {
+ ret = hp206c_read_reg(client, HP206C_REG_INT_SRC);
+ if (ret < 0) {
+ dev_err(&indio_dev->dev, "Failed READ_REG INT_SRC: %d\n", ret);
+ return ret;
+ }
+ if (ret & HP206C_FLAG_DEV_RDY)
+ return 0;
+ usleep_range(HP206C_DEV_RDY_WAIT_US, HP206C_DEV_RDY_WAIT_US * 3 / 2);
+ }
+ return -ETIMEDOUT;
+}
+
+static int hp206c_set_compensation(struct i2c_client *client, bool enabled)
+{
+ int val;
+
+ val = hp206c_read_reg(client, HP206C_REG_PARA);
+ if (val < 0)
+ return val;
+ if (enabled)
+ val |= HP206C_FLAG_CMPS_EN;
+ else
+ val &= ~HP206C_FLAG_CMPS_EN;
+
+ return hp206c_write_reg(client, HP206C_REG_PARA, val);
+}
+
+/* Do a soft reset */
+static int hp206c_soft_reset(struct iio_dev *indio_dev)
+{
+ int ret;
+ struct hp206c_data *data = iio_priv(indio_dev);
+ struct i2c_client *client = data->client;
+
+ ret = i2c_smbus_write_byte(client, HP206C_CMD_SOFT_RST);
+ if (ret) {
+ dev_err(&client->dev, "Failed to reset device: %d\n", ret);
+ return ret;
+ }
+
+ usleep_range(400, 600);
+
+ ret = hp206c_wait_dev_rdy(indio_dev);
+ if (ret) {
+ dev_err(&client->dev, "Device not ready after soft reset: %d\n", ret);
+ return ret;
+ }
+
+ ret = hp206c_set_compensation(client, true);
+ if (ret)
+ dev_err(&client->dev, "Failed to enable compensation: %d\n", ret);
+ return ret;
+}
+
+static int hp206c_conv_and_read(struct iio_dev *indio_dev,
+ u8 conv_cmd, u8 read_cmd,
+ unsigned int sleep_us)
+{
+ int ret;
+ struct hp206c_data *data = iio_priv(indio_dev);
+ struct i2c_client *client = data->client;
+
+ ret = hp206c_wait_dev_rdy(indio_dev);
+ if (ret < 0) {
+ dev_err(&indio_dev->dev, "Device not ready: %d\n", ret);
+ return ret;
+ }
+
+ ret = i2c_smbus_write_byte(client, conv_cmd);
+ if (ret < 0) {
+ dev_err(&indio_dev->dev, "Failed convert: %d\n", ret);
+ return ret;
+ }
+
+ usleep_range(sleep_us, sleep_us * 3 / 2);
+
+ ret = hp206c_wait_dev_rdy(indio_dev);
+ if (ret < 0) {
+ dev_err(&indio_dev->dev, "Device not ready: %d\n", ret);
+ return ret;
+ }
+
+ ret = hp206c_read_20bit(client, read_cmd);
+ if (ret < 0)
+ dev_err(&indio_dev->dev, "Failed read: %d\n", ret);
+
+ return ret;
+}
+
+static int hp206c_read_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan, int *val,
+ int *val2, long mask)
+{
+ int ret;
+ struct hp206c_data *data = iio_priv(indio_dev);
+ const struct hp206c_osr_setting *osr_setting;
+ u8 conv_cmd;
+
+ mutex_lock(&data->mutex);
+
+ switch (mask) {
+ case IIO_CHAN_INFO_OVERSAMPLING_RATIO:
+ switch (chan->type) {
+ case IIO_TEMP:
+ *val = hp206c_osr_rates[data->temp_osr_index];
+ ret = IIO_VAL_INT;
+ break;
+
+ case IIO_PRESSURE:
+ *val = hp206c_osr_rates[data->pres_osr_index];
+ ret = IIO_VAL_INT;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+ break;
+
+ case IIO_CHAN_INFO_RAW:
+ switch (chan->type) {
+ case IIO_TEMP:
+ osr_setting = &hp206c_osr_settings[data->temp_osr_index];
+ conv_cmd = HP206C_CMD_ADC_CVT |
+ osr_setting->osr_mask |
+ HP206C_CMD_ADC_CVT_CHNL_T;
+ ret = hp206c_conv_and_read(indio_dev,
+ conv_cmd,
+ HP206C_CMD_READ_T,
+ osr_setting->temp_conv_time_us);
+ if (ret >= 0) {
+ /* 20 significant bits are provided.
+ * Extend sign over the rest.
+ */
+ *val = sign_extend32(ret, 19);
+ ret = IIO_VAL_INT;
+ }
+ break;
+
+ case IIO_PRESSURE:
+ osr_setting = &hp206c_osr_settings[data->pres_osr_index];
+ conv_cmd = HP206C_CMD_ADC_CVT |
+ osr_setting->osr_mask |
+ HP206C_CMD_ADC_CVT_CHNL_PT;
+ ret = hp206c_conv_and_read(indio_dev,
+ conv_cmd,
+ HP206C_CMD_READ_P,
+ osr_setting->pres_conv_time_us);
+ if (ret >= 0) {
+ *val = ret;
+ ret = IIO_VAL_INT;
+ }
+ break;
+ default:
+ ret = -EINVAL;
+ }
+ break;
+
+ case IIO_CHAN_INFO_SCALE:
+ switch (chan->type) {
+ case IIO_TEMP:
+ *val = 0;
+ *val2 = 10000;
+ ret = IIO_VAL_INT_PLUS_MICRO;
+ break;
+
+ case IIO_PRESSURE:
+ *val = 0;
+ *val2 = 1000;
+ ret = IIO_VAL_INT_PLUS_MICRO;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+ break;
+
+ default:
+ ret = -EINVAL;
+ }
+
+ mutex_unlock(&data->mutex);
+ return ret;
+}
+
+static int hp206c_write_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int val, int val2, long mask)
+{
+ int ret = 0;
+ struct hp206c_data *data = iio_priv(indio_dev);
+
+ if (mask != IIO_CHAN_INFO_OVERSAMPLING_RATIO)
+ return -EINVAL;
+ mutex_lock(&data->mutex);
+ switch (chan->type) {
+ case IIO_TEMP:
+ data->temp_osr_index = find_closest_descending(val,
+ hp206c_osr_rates, ARRAY_SIZE(hp206c_osr_rates));
+ break;
+ case IIO_PRESSURE:
+ data->pres_osr_index = find_closest_descending(val,
+ hp206c_osr_rates, ARRAY_SIZE(hp206c_osr_rates));
+ break;
+ default:
+ ret = -EINVAL;
+ }
+ mutex_unlock(&data->mutex);
+ return ret;
+}
+
+static const struct iio_chan_spec hp206c_channels[] = {
+ {
+ .type = IIO_TEMP,
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) |
+ BIT(IIO_CHAN_INFO_SCALE) |
+ BIT(IIO_CHAN_INFO_OVERSAMPLING_RATIO),
+ },
+ {
+ .type = IIO_PRESSURE,
+ .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) |
+ BIT(IIO_CHAN_INFO_SCALE) |
+ BIT(IIO_CHAN_INFO_OVERSAMPLING_RATIO),
+ }
+};
+
+static IIO_CONST_ATTR_SAMP_FREQ_AVAIL(hp206c_osr_rates_str);
+
+static struct attribute *hp206c_attributes[] = {
+ &iio_const_attr_sampling_frequency_available.dev_attr.attr,
+ NULL,
+};
+
+static const struct attribute_group hp206c_attribute_group = {
+ .attrs = hp206c_attributes,
+};
+
+static const struct iio_info hp206c_info = {
+ .attrs = &hp206c_attribute_group,
+ .read_raw = hp206c_read_raw,
+ .write_raw = hp206c_write_raw,
+ .driver_module = THIS_MODULE,
+};
+
+static int hp206c_probe(struct i2c_client *client,
+ const struct i2c_device_id *id)
+{
+ struct iio_dev *indio_dev;
+ struct hp206c_data *data;
+ int ret;
+
+ if (!i2c_check_functionality(client->adapter,
+ I2C_FUNC_SMBUS_BYTE |
+ I2C_FUNC_SMBUS_BYTE_DATA |
+ I2C_FUNC_SMBUS_READ_I2C_BLOCK)) {
+ dev_err(&client->dev, "Adapter does not support "
+ "all required i2c functionality\n");
+ return -ENODEV;
+ }
+
+ indio_dev = devm_iio_device_alloc(&client->dev, sizeof(*data));
+ if (!indio_dev)
+ return -ENOMEM;
+
+ data = iio_priv(indio_dev);
+ data->client = client;
+ mutex_init(&data->mutex);
+
+ indio_dev->info = &hp206c_info;
+ indio_dev->name = id->name;
+ indio_dev->dev.parent = &client->dev;
+ indio_dev->modes = INDIO_DIRECT_MODE;
+ indio_dev->channels = hp206c_channels;
+ indio_dev->num_channels = ARRAY_SIZE(hp206c_channels);
+
+ i2c_set_clientdata(client, indio_dev);
+
+ /* Do a soft reset on probe */
+ ret = hp206c_soft_reset(indio_dev);
+ if (ret) {
+ dev_err(&client->dev, "Failed to reset on startup: %d\n", ret);
+ return -ENODEV;
+ }
+
+ return devm_iio_device_register(&client->dev, indio_dev);
+}
+
+static const struct i2c_device_id hp206c_id[] = {
+ {"hp206c"},
+ {}
+};
+
+#ifdef CONFIG_ACPI
+static const struct acpi_device_id hp206c_acpi_match[] = {
+ {"HOP206C", 0},
+ { },
+};
+MODULE_DEVICE_TABLE(acpi, hp206c_acpi_match);
+#endif
+
+static struct i2c_driver hp206c_driver = {
+ .probe = hp206c_probe,
+ .id_table = hp206c_id,
+ .driver = {
+ .name = "hp206c",
+ .acpi_match_table = ACPI_PTR(hp206c_acpi_match),
+ },
+};
+
+module_i2c_driver(hp206c_driver);
+
+MODULE_DESCRIPTION("HOPERF HP206C precision barometer and altimeter sensor");
+MODULE_AUTHOR("Leonard Crestez <leonard.crestez@intel.com>");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/iio/pressure/ms5611.h b/drivers/iio/pressure/ms5611.h
index 8b08e4b..ccda63c 100644
--- a/drivers/iio/pressure/ms5611.h
+++ b/drivers/iio/pressure/ms5611.h
@@ -16,15 +16,11 @@
#include <linux/iio/iio.h>
#include <linux/mutex.h>
+struct regulator;
+
#define MS5611_RESET 0x1e
#define MS5611_READ_ADC 0x00
#define MS5611_READ_PROM_WORD 0xA0
-#define MS5611_START_TEMP_CONV 0x58
-#define MS5611_START_PRESSURE_CONV 0x48
-
-#define MS5611_CONV_TIME_MIN 9040
-#define MS5611_CONV_TIME_MAX 10000
-
#define MS5611_PROM_WORDS_NB 8
enum {
@@ -39,16 +35,31 @@
s32 *temp, s32 *pressure);
};
+/*
+ * OverSampling Rate descriptor.
+ * Warning: cmd MUST be kept aligned on a word boundary (see
+ * m5611_spi_read_adc_temp_and_pressure in ms5611_spi.c).
+ */
+struct ms5611_osr {
+ unsigned long conv_usec;
+ u8 cmd;
+ unsigned short rate;
+};
+
struct ms5611_state {
void *client;
struct mutex lock;
+ const struct ms5611_osr *pressure_osr;
+ const struct ms5611_osr *temp_osr;
+
int (*reset)(struct device *dev);
int (*read_prom_word)(struct device *dev, int index, u16 *word);
int (*read_adc_temp_and_pressure)(struct device *dev,
s32 *temp, s32 *pressure);
struct ms5611_chip_info *chip_info;
+ struct regulator *vdd;
};
int ms5611_probe(struct iio_dev *indio_dev, struct device *dev,
diff --git a/drivers/iio/pressure/ms5611_core.c b/drivers/iio/pressure/ms5611_core.c
index 992ad8d..76578b0 100644
--- a/drivers/iio/pressure/ms5611_core.c
+++ b/drivers/iio/pressure/ms5611_core.c
@@ -18,11 +18,44 @@
#include <linux/delay.h>
#include <linux/regulator/consumer.h>
+#include <linux/iio/sysfs.h>
#include <linux/iio/buffer.h>
#include <linux/iio/triggered_buffer.h>
#include <linux/iio/trigger_consumer.h>
#include "ms5611.h"
+#define MS5611_INIT_OSR(_cmd, _conv_usec, _rate) \
+ { .cmd = _cmd, .conv_usec = _conv_usec, .rate = _rate }
+
+static const struct ms5611_osr ms5611_avail_pressure_osr[] = {
+ MS5611_INIT_OSR(0x40, 600, 256),
+ MS5611_INIT_OSR(0x42, 1170, 512),
+ MS5611_INIT_OSR(0x44, 2280, 1024),
+ MS5611_INIT_OSR(0x46, 4540, 2048),
+ MS5611_INIT_OSR(0x48, 9040, 4096)
+};
+
+static const struct ms5611_osr ms5611_avail_temp_osr[] = {
+ MS5611_INIT_OSR(0x50, 600, 256),
+ MS5611_INIT_OSR(0x52, 1170, 512),
+ MS5611_INIT_OSR(0x54, 2280, 1024),
+ MS5611_INIT_OSR(0x56, 4540, 2048),
+ MS5611_INIT_OSR(0x58, 9040, 4096)
+};
+
+static const char ms5611_show_osr[] = "256 512 1024 2048 4096";
+
+static IIO_CONST_ATTR(oversampling_ratio_available, ms5611_show_osr);
+
+static struct attribute *ms5611_attributes[] = {
+ &iio_const_attr_oversampling_ratio_available.dev_attr.attr,
+ NULL,
+};
+
+static const struct attribute_group ms5611_attribute_group = {
+ .attrs = ms5611_attributes,
+};
+
static bool ms5611_prom_is_valid(u16 *prom, size_t len)
{
int i, j;
@@ -239,11 +272,70 @@
default:
return -EINVAL;
}
+ case IIO_CHAN_INFO_OVERSAMPLING_RATIO:
+ if (chan->type != IIO_TEMP && chan->type != IIO_PRESSURE)
+ break;
+ mutex_lock(&st->lock);
+ if (chan->type == IIO_TEMP)
+ *val = (int)st->temp_osr->rate;
+ else
+ *val = (int)st->pressure_osr->rate;
+ mutex_unlock(&st->lock);
+ return IIO_VAL_INT;
}
return -EINVAL;
}
+static const struct ms5611_osr *ms5611_find_osr(int rate,
+ const struct ms5611_osr *osr,
+ size_t count)
+{
+ unsigned int r;
+
+ for (r = 0; r < count; r++)
+ if ((unsigned short)rate == osr[r].rate)
+ break;
+ if (r >= count)
+ return NULL;
+ return &osr[r];
+}
+
+static int ms5611_write_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int val, int val2, long mask)
+{
+ struct ms5611_state *st = iio_priv(indio_dev);
+ const struct ms5611_osr *osr = NULL;
+
+ if (mask != IIO_CHAN_INFO_OVERSAMPLING_RATIO)
+ return -EINVAL;
+
+ if (chan->type == IIO_TEMP)
+ osr = ms5611_find_osr(val, ms5611_avail_temp_osr,
+ ARRAY_SIZE(ms5611_avail_temp_osr));
+ else if (chan->type == IIO_PRESSURE)
+ osr = ms5611_find_osr(val, ms5611_avail_pressure_osr,
+ ARRAY_SIZE(ms5611_avail_pressure_osr));
+ if (!osr)
+ return -EINVAL;
+
+ mutex_lock(&st->lock);
+
+ if (iio_buffer_enabled(indio_dev)) {
+ mutex_unlock(&st->lock);
+ return -EBUSY;
+ }
+
+ if (chan->type == IIO_TEMP)
+ st->temp_osr = osr;
+ else
+ st->pressure_osr = osr;
+
+ mutex_unlock(&st->lock);
+ return 0;
+}
+
static const unsigned long ms5611_scan_masks[] = {0x3, 0};
static struct ms5611_chip_info chip_info_tbl[] = {
@@ -259,7 +351,8 @@
{
.type = IIO_PRESSURE,
.info_mask_separate = BIT(IIO_CHAN_INFO_PROCESSED) |
- BIT(IIO_CHAN_INFO_SCALE),
+ BIT(IIO_CHAN_INFO_SCALE) |
+ BIT(IIO_CHAN_INFO_OVERSAMPLING_RATIO),
.scan_index = 0,
.scan_type = {
.sign = 's',
@@ -271,7 +364,8 @@
{
.type = IIO_TEMP,
.info_mask_separate = BIT(IIO_CHAN_INFO_PROCESSED) |
- BIT(IIO_CHAN_INFO_SCALE),
+ BIT(IIO_CHAN_INFO_SCALE) |
+ BIT(IIO_CHAN_INFO_OVERSAMPLING_RATIO),
.scan_index = 1,
.scan_type = {
.sign = 's',
@@ -285,40 +379,68 @@
static const struct iio_info ms5611_info = {
.read_raw = &ms5611_read_raw,
+ .write_raw = &ms5611_write_raw,
+ .attrs = &ms5611_attribute_group,
.driver_module = THIS_MODULE,
};
static int ms5611_init(struct iio_dev *indio_dev)
{
int ret;
- struct regulator *vdd = devm_regulator_get(indio_dev->dev.parent,
- "vdd");
+ struct ms5611_state *st = iio_priv(indio_dev);
/* Enable attached regulator if any. */
- if (!IS_ERR(vdd)) {
- ret = regulator_enable(vdd);
+ st->vdd = devm_regulator_get(indio_dev->dev.parent, "vdd");
+ if (!IS_ERR(st->vdd)) {
+ ret = regulator_enable(st->vdd);
if (ret) {
dev_err(indio_dev->dev.parent,
- "failed to enable Vdd supply: %d\n", ret);
+ "failed to enable Vdd supply: %d\n", ret);
return ret;
}
+ } else {
+ ret = PTR_ERR(st->vdd);
+ if (ret != -ENODEV)
+ return ret;
}
ret = ms5611_reset(indio_dev);
if (ret < 0)
- return ret;
+ goto err_regulator_disable;
- return ms5611_read_prom(indio_dev);
+ ret = ms5611_read_prom(indio_dev);
+ if (ret < 0)
+ goto err_regulator_disable;
+
+ return 0;
+
+err_regulator_disable:
+ if (!IS_ERR_OR_NULL(st->vdd))
+ regulator_disable(st->vdd);
+ return ret;
+}
+
+static void ms5611_fini(const struct iio_dev *indio_dev)
+{
+ const struct ms5611_state *st = iio_priv(indio_dev);
+
+ if (!IS_ERR_OR_NULL(st->vdd))
+ regulator_disable(st->vdd);
}
int ms5611_probe(struct iio_dev *indio_dev, struct device *dev,
- const char *name, int type)
+ const char *name, int type)
{
int ret;
struct ms5611_state *st = iio_priv(indio_dev);
mutex_init(&st->lock);
st->chip_info = &chip_info_tbl[type];
+ st->temp_osr =
+ &ms5611_avail_temp_osr[ARRAY_SIZE(ms5611_avail_temp_osr) - 1];
+ st->pressure_osr =
+ &ms5611_avail_pressure_osr[ARRAY_SIZE(ms5611_avail_pressure_osr)
+ - 1];
indio_dev->dev.parent = dev;
indio_dev->name = name;
indio_dev->info = &ms5611_info;
@@ -335,7 +457,7 @@
ms5611_trigger_handler, NULL);
if (ret < 0) {
dev_err(dev, "iio triggered buffer setup failed\n");
- return ret;
+ goto err_fini;
}
ret = iio_device_register(indio_dev);
@@ -348,7 +470,8 @@
err_buffer_cleanup:
iio_triggered_buffer_cleanup(indio_dev);
-
+err_fini:
+ ms5611_fini(indio_dev);
return ret;
}
EXPORT_SYMBOL(ms5611_probe);
@@ -357,6 +480,7 @@
{
iio_device_unregister(indio_dev);
iio_triggered_buffer_cleanup(indio_dev);
+ ms5611_fini(indio_dev);
return 0;
}
diff --git a/drivers/iio/pressure/ms5611_i2c.c b/drivers/iio/pressure/ms5611_i2c.c
index 7f6fc8e..55fb5fc 100644
--- a/drivers/iio/pressure/ms5611_i2c.c
+++ b/drivers/iio/pressure/ms5611_i2c.c
@@ -17,6 +17,7 @@
#include <linux/delay.h>
#include <linux/i2c.h>
#include <linux/module.h>
+#include <linux/of_device.h>
#include "ms5611.h"
@@ -62,23 +63,23 @@
{
int ret;
struct ms5611_state *st = iio_priv(dev_to_iio_dev(dev));
+ const struct ms5611_osr *osr = st->temp_osr;
- ret = i2c_smbus_write_byte(st->client, MS5611_START_TEMP_CONV);
+ ret = i2c_smbus_write_byte(st->client, osr->cmd);
if (ret < 0)
return ret;
- usleep_range(MS5611_CONV_TIME_MIN, MS5611_CONV_TIME_MAX);
-
+ usleep_range(osr->conv_usec, osr->conv_usec + (osr->conv_usec / 10UL));
ret = ms5611_i2c_read_adc(st, temp);
if (ret < 0)
return ret;
- ret = i2c_smbus_write_byte(st->client, MS5611_START_PRESSURE_CONV);
+ osr = st->pressure_osr;
+ ret = i2c_smbus_write_byte(st->client, osr->cmd);
if (ret < 0)
return ret;
- usleep_range(MS5611_CONV_TIME_MIN, MS5611_CONV_TIME_MAX);
-
+ usleep_range(osr->conv_usec, osr->conv_usec + (osr->conv_usec / 10UL));
return ms5611_i2c_read_adc(st, pressure);
}
@@ -113,6 +114,17 @@
return ms5611_remove(i2c_get_clientdata(client));
}
+#if defined(CONFIG_OF)
+static const struct of_device_id ms5611_i2c_matches[] = {
+ { .compatible = "meas,ms5611" },
+ { .compatible = "ms5611" },
+ { .compatible = "meas,ms5607" },
+ { .compatible = "ms5607" },
+ { }
+};
+MODULE_DEVICE_TABLE(of, ms5611_i2c_matches);
+#endif
+
static const struct i2c_device_id ms5611_id[] = {
{ "ms5611", MS5611 },
{ "ms5607", MS5607 },
@@ -123,6 +135,7 @@
static struct i2c_driver ms5611_driver = {
.driver = {
.name = "ms5611",
+ .of_match_table = of_match_ptr(ms5611_i2c_matches)
},
.id_table = ms5611_id,
.probe = ms5611_i2c_probe,
diff --git a/drivers/iio/pressure/ms5611_spi.c b/drivers/iio/pressure/ms5611_spi.c
index 5cc009e..932e050 100644
--- a/drivers/iio/pressure/ms5611_spi.c
+++ b/drivers/iio/pressure/ms5611_spi.c
@@ -12,6 +12,7 @@
#include <linux/delay.h>
#include <linux/module.h>
#include <linux/spi/spi.h>
+#include <linux/of_device.h>
#include "ms5611.h"
@@ -55,28 +56,29 @@
static int ms5611_spi_read_adc_temp_and_pressure(struct device *dev,
s32 *temp, s32 *pressure)
{
- u8 cmd;
int ret;
struct ms5611_state *st = iio_priv(dev_to_iio_dev(dev));
+ const struct ms5611_osr *osr = st->temp_osr;
- cmd = MS5611_START_TEMP_CONV;
- ret = spi_write_then_read(st->client, &cmd, 1, NULL, 0);
+ /*
+ * Warning: &osr->cmd MUST be aligned on a word boundary since used as
+ * 2nd argument (void*) of spi_write_then_read.
+ */
+ ret = spi_write_then_read(st->client, &osr->cmd, 1, NULL, 0);
if (ret < 0)
return ret;
- usleep_range(MS5611_CONV_TIME_MIN, MS5611_CONV_TIME_MAX);
-
+ usleep_range(osr->conv_usec, osr->conv_usec + (osr->conv_usec / 10UL));
ret = ms5611_spi_read_adc(dev, temp);
if (ret < 0)
return ret;
- cmd = MS5611_START_PRESSURE_CONV;
- ret = spi_write_then_read(st->client, &cmd, 1, NULL, 0);
+ osr = st->pressure_osr;
+ ret = spi_write_then_read(st->client, &osr->cmd, 1, NULL, 0);
if (ret < 0)
return ret;
- usleep_range(MS5611_CONV_TIME_MIN, MS5611_CONV_TIME_MAX);
-
+ usleep_range(osr->conv_usec, osr->conv_usec + (osr->conv_usec / 10UL));
return ms5611_spi_read_adc(dev, pressure);
}
@@ -106,7 +108,7 @@
st->client = spi;
return ms5611_probe(indio_dev, &spi->dev, spi_get_device_id(spi)->name,
- spi_get_device_id(spi)->driver_data);
+ spi_get_device_id(spi)->driver_data);
}
static int ms5611_spi_remove(struct spi_device *spi)
@@ -114,6 +116,17 @@
return ms5611_remove(spi_get_drvdata(spi));
}
+#if defined(CONFIG_OF)
+static const struct of_device_id ms5611_spi_matches[] = {
+ { .compatible = "meas,ms5611" },
+ { .compatible = "ms5611" },
+ { .compatible = "meas,ms5607" },
+ { .compatible = "ms5607" },
+ { }
+};
+MODULE_DEVICE_TABLE(of, ms5611_spi_matches);
+#endif
+
static const struct spi_device_id ms5611_id[] = {
{ "ms5611", MS5611 },
{ "ms5607", MS5607 },
@@ -124,6 +137,7 @@
static struct spi_driver ms5611_driver = {
.driver = {
.name = "ms5611",
+ .of_match_table = of_match_ptr(ms5611_spi_matches)
},
.id_table = ms5611_id,
.probe = ms5611_spi_probe,
diff --git a/drivers/iio/pressure/st_pressure_core.c b/drivers/iio/pressure/st_pressure_core.c
index 172393a..9e9b72a 100644
--- a/drivers/iio/pressure/st_pressure_core.c
+++ b/drivers/iio/pressure/st_pressure_core.c
@@ -64,6 +64,8 @@
#define ST_PRESS_LPS331AP_DRDY_IRQ_INT2_MASK 0x20
#define ST_PRESS_LPS331AP_IHL_IRQ_ADDR 0x22
#define ST_PRESS_LPS331AP_IHL_IRQ_MASK 0x80
+#define ST_PRESS_LPS331AP_OD_IRQ_ADDR 0x22
+#define ST_PRESS_LPS331AP_OD_IRQ_MASK 0x40
#define ST_PRESS_LPS331AP_MULTIREAD_BIT true
#define ST_PRESS_LPS331AP_TEMP_OFFSET 42500
@@ -104,6 +106,8 @@
#define ST_PRESS_LPS25H_DRDY_IRQ_INT2_MASK 0x10
#define ST_PRESS_LPS25H_IHL_IRQ_ADDR 0x22
#define ST_PRESS_LPS25H_IHL_IRQ_MASK 0x80
+#define ST_PRESS_LPS25H_OD_IRQ_ADDR 0x22
+#define ST_PRESS_LPS25H_OD_IRQ_MASK 0x40
#define ST_PRESS_LPS25H_MULTIREAD_BIT true
#define ST_PRESS_LPS25H_TEMP_OFFSET 42500
#define ST_PRESS_LPS25H_OUT_XL_ADDR 0x28
@@ -226,6 +230,9 @@
.mask_int2 = ST_PRESS_LPS331AP_DRDY_IRQ_INT2_MASK,
.addr_ihl = ST_PRESS_LPS331AP_IHL_IRQ_ADDR,
.mask_ihl = ST_PRESS_LPS331AP_IHL_IRQ_MASK,
+ .addr_od = ST_PRESS_LPS331AP_OD_IRQ_ADDR,
+ .mask_od = ST_PRESS_LPS331AP_OD_IRQ_MASK,
+ .addr_stat_drdy = ST_SENSORS_DEFAULT_STAT_ADDR,
},
.multi_read_bit = ST_PRESS_LPS331AP_MULTIREAD_BIT,
.bootime = 2,
@@ -312,6 +319,9 @@
.mask_int2 = ST_PRESS_LPS25H_DRDY_IRQ_INT2_MASK,
.addr_ihl = ST_PRESS_LPS25H_IHL_IRQ_ADDR,
.mask_ihl = ST_PRESS_LPS25H_IHL_IRQ_MASK,
+ .addr_od = ST_PRESS_LPS25H_OD_IRQ_ADDR,
+ .mask_od = ST_PRESS_LPS25H_OD_IRQ_MASK,
+ .addr_stat_drdy = ST_SENSORS_DEFAULT_STAT_ADDR,
},
.multi_read_bit = ST_PRESS_LPS25H_MULTIREAD_BIT,
.bootime = 2,
diff --git a/drivers/infiniband/core/Makefile b/drivers/infiniband/core/Makefile
index f818538..26987d9 100644
--- a/drivers/infiniband/core/Makefile
+++ b/drivers/infiniband/core/Makefile
@@ -8,9 +8,9 @@
obj-$(CONFIG_INFINIBAND_USER_ACCESS) += ib_uverbs.o ib_ucm.o \
$(user_access-y)
-ib_core-y := packer.o ud_header.o verbs.o cq.o sysfs.o \
+ib_core-y := packer.o ud_header.o verbs.o cq.o rw.o sysfs.o \
device.o fmr_pool.o cache.o netlink.o \
- roce_gid_mgmt.o
+ roce_gid_mgmt.o mr_pool.o
ib_core-$(CONFIG_INFINIBAND_USER_MEM) += umem.o
ib_core-$(CONFIG_INFINIBAND_ON_DEMAND_PAGING) += umem_odp.o umem_rbtree.o
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 93ab0ae..f0c91ba 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -800,6 +800,7 @@
if (id->device != pd->device)
return -EINVAL;
+ qp_init_attr->port_num = id->port_num;
qp = ib_create_qp(pd, qp_init_attr);
if (IS_ERR(qp))
return PTR_ERR(qp);
@@ -4294,7 +4295,8 @@
if (ret)
goto err;
- if (ibnl_add_client(RDMA_NL_RDMA_CM, RDMA_NL_RDMA_CM_NUM_OPS, cma_cb_table))
+ if (ibnl_add_client(RDMA_NL_RDMA_CM, ARRAY_SIZE(cma_cb_table),
+ cma_cb_table))
pr_warn("RDMA CMA: failed to add netlink callback\n");
cma_configfs_init();
diff --git a/drivers/infiniband/core/iwcm.c b/drivers/infiniband/core/iwcm.c
index e28a160..f057204 100644
--- a/drivers/infiniband/core/iwcm.c
+++ b/drivers/infiniband/core/iwcm.c
@@ -459,7 +459,7 @@
if (pm_addr->ss_family == AF_INET) {
struct sockaddr_in *pm4_addr = (struct sockaddr_in *)pm_addr;
- if (pm4_addr->sin_addr.s_addr == INADDR_ANY) {
+ if (pm4_addr->sin_addr.s_addr == htonl(INADDR_ANY)) {
struct sockaddr_in *cm4_addr =
(struct sockaddr_in *)cm_addr;
struct sockaddr_in *cm4_outaddr =
@@ -1175,7 +1175,7 @@
if (ret)
pr_err("iw_cm: couldn't init iwpm\n");
- ret = ibnl_add_client(RDMA_NL_IWCM, RDMA_NL_IWPM_NUM_OPS,
+ ret = ibnl_add_client(RDMA_NL_IWCM, ARRAY_SIZE(iwcm_nl_cb_table),
iwcm_nl_cb_table);
if (ret)
pr_err("iw_cm: couldn't register netlink callbacks\n");
diff --git a/drivers/infiniband/core/iwpm_util.c b/drivers/infiniband/core/iwpm_util.c
index 9b2bf2f..b65e06c 100644
--- a/drivers/infiniband/core/iwpm_util.c
+++ b/drivers/infiniband/core/iwpm_util.c
@@ -634,6 +634,7 @@
if (!(ibnl_put_msg(skb, &nlh, 0, 0, nl_client,
RDMA_NL_IWPM_MAPINFO, NLM_F_MULTI))) {
pr_warn("%s Unable to put NLMSG_DONE\n", __func__);
+ dev_kfree_skb(skb);
return -ENOMEM;
}
nlh->nlmsg_type = NLMSG_DONE;
diff --git a/drivers/infiniband/core/mr_pool.c b/drivers/infiniband/core/mr_pool.c
new file mode 100644
index 0000000..49d478b
--- /dev/null
+++ b/drivers/infiniband/core/mr_pool.c
@@ -0,0 +1,86 @@
+/*
+ * Copyright (c) 2016 HGST, a Western Digital Company.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ */
+#include <rdma/ib_verbs.h>
+#include <rdma/mr_pool.h>
+
+struct ib_mr *ib_mr_pool_get(struct ib_qp *qp, struct list_head *list)
+{
+ struct ib_mr *mr;
+ unsigned long flags;
+
+ spin_lock_irqsave(&qp->mr_lock, flags);
+ mr = list_first_entry_or_null(list, struct ib_mr, qp_entry);
+ if (mr) {
+ list_del(&mr->qp_entry);
+ qp->mrs_used++;
+ }
+ spin_unlock_irqrestore(&qp->mr_lock, flags);
+
+ return mr;
+}
+EXPORT_SYMBOL(ib_mr_pool_get);
+
+void ib_mr_pool_put(struct ib_qp *qp, struct list_head *list, struct ib_mr *mr)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&qp->mr_lock, flags);
+ list_add(&mr->qp_entry, list);
+ qp->mrs_used--;
+ spin_unlock_irqrestore(&qp->mr_lock, flags);
+}
+EXPORT_SYMBOL(ib_mr_pool_put);
+
+int ib_mr_pool_init(struct ib_qp *qp, struct list_head *list, int nr,
+ enum ib_mr_type type, u32 max_num_sg)
+{
+ struct ib_mr *mr;
+ unsigned long flags;
+ int ret, i;
+
+ for (i = 0; i < nr; i++) {
+ mr = ib_alloc_mr(qp->pd, type, max_num_sg);
+ if (IS_ERR(mr)) {
+ ret = PTR_ERR(mr);
+ goto out;
+ }
+
+ spin_lock_irqsave(&qp->mr_lock, flags);
+ list_add_tail(&mr->qp_entry, list);
+ spin_unlock_irqrestore(&qp->mr_lock, flags);
+ }
+
+ return 0;
+out:
+ ib_mr_pool_destroy(qp, list);
+ return ret;
+}
+EXPORT_SYMBOL(ib_mr_pool_init);
+
+void ib_mr_pool_destroy(struct ib_qp *qp, struct list_head *list)
+{
+ struct ib_mr *mr;
+ unsigned long flags;
+
+ spin_lock_irqsave(&qp->mr_lock, flags);
+ while (!list_empty(list)) {
+ mr = list_first_entry(list, struct ib_mr, qp_entry);
+ list_del(&mr->qp_entry);
+
+ spin_unlock_irqrestore(&qp->mr_lock, flags);
+ ib_dereg_mr(mr);
+ spin_lock_irqsave(&qp->mr_lock, flags);
+ }
+ spin_unlock_irqrestore(&qp->mr_lock, flags);
+}
+EXPORT_SYMBOL(ib_mr_pool_destroy);
diff --git a/drivers/infiniband/core/netlink.c b/drivers/infiniband/core/netlink.c
index d47df93..9b8c20c 100644
--- a/drivers/infiniband/core/netlink.c
+++ b/drivers/infiniband/core/netlink.c
@@ -151,12 +151,11 @@
struct ibnl_client *client;
int type = nlh->nlmsg_type;
int index = RDMA_NL_GET_CLIENT(type);
- int op = RDMA_NL_GET_OP(type);
+ unsigned int op = RDMA_NL_GET_OP(type);
list_for_each_entry(client, &client_list, list) {
if (client->index == index) {
- if (op < 0 || op >= client->nops ||
- !client->cb_table[op].dump)
+ if (op >= client->nops || !client->cb_table[op].dump)
return -EINVAL;
/*
diff --git a/drivers/infiniband/core/rw.c b/drivers/infiniband/core/rw.c
new file mode 100644
index 0000000..1eb9b12
--- /dev/null
+++ b/drivers/infiniband/core/rw.c
@@ -0,0 +1,727 @@
+/*
+ * Copyright (c) 2016 HGST, a Western Digital Company.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ */
+#include <linux/moduleparam.h>
+#include <linux/slab.h>
+#include <rdma/mr_pool.h>
+#include <rdma/rw.h>
+
+enum {
+ RDMA_RW_SINGLE_WR,
+ RDMA_RW_MULTI_WR,
+ RDMA_RW_MR,
+ RDMA_RW_SIG_MR,
+};
+
+static bool rdma_rw_force_mr;
+module_param_named(force_mr, rdma_rw_force_mr, bool, 0);
+MODULE_PARM_DESC(force_mr, "Force usage of MRs for RDMA READ/WRITE operations");
+
+/*
+ * Check if the device might use memory registration. This is currently only
+ * true for iWarp devices. In the future we can hopefully fine tune this based
+ * on HCA driver input.
+ */
+static inline bool rdma_rw_can_use_mr(struct ib_device *dev, u8 port_num)
+{
+ if (rdma_protocol_iwarp(dev, port_num))
+ return true;
+ if (unlikely(rdma_rw_force_mr))
+ return true;
+ return false;
+}
+
+/*
+ * Check if the device will use memory registration for this RW operation.
+ * We currently always use memory registrations for iWarp RDMA READs, and
+ * have a debug option to force usage of MRs.
+ *
+ * XXX: In the future we can hopefully fine tune this based on HCA driver
+ * input.
+ */
+static inline bool rdma_rw_io_needs_mr(struct ib_device *dev, u8 port_num,
+ enum dma_data_direction dir, int dma_nents)
+{
+ if (rdma_protocol_iwarp(dev, port_num) && dir == DMA_FROM_DEVICE)
+ return true;
+ if (unlikely(rdma_rw_force_mr))
+ return true;
+ return false;
+}
+
+static inline u32 rdma_rw_max_sge(struct ib_device *dev,
+ enum dma_data_direction dir)
+{
+ return dir == DMA_TO_DEVICE ?
+ dev->attrs.max_sge : dev->attrs.max_sge_rd;
+}
+
+static inline u32 rdma_rw_fr_page_list_len(struct ib_device *dev)
+{
+ /* arbitrary limit to avoid allocating gigantic resources */
+ return min_t(u32, dev->attrs.max_fast_reg_page_list_len, 256);
+}
+
+static int rdma_rw_init_one_mr(struct ib_qp *qp, u8 port_num,
+ struct rdma_rw_reg_ctx *reg, struct scatterlist *sg,
+ u32 sg_cnt, u32 offset)
+{
+ u32 pages_per_mr = rdma_rw_fr_page_list_len(qp->pd->device);
+ u32 nents = min(sg_cnt, pages_per_mr);
+ int count = 0, ret;
+
+ reg->mr = ib_mr_pool_get(qp, &qp->rdma_mrs);
+ if (!reg->mr)
+ return -EAGAIN;
+
+ if (reg->mr->need_inval) {
+ reg->inv_wr.opcode = IB_WR_LOCAL_INV;
+ reg->inv_wr.ex.invalidate_rkey = reg->mr->lkey;
+ reg->inv_wr.next = ®->reg_wr.wr;
+ count++;
+ } else {
+ reg->inv_wr.next = NULL;
+ }
+
+ ret = ib_map_mr_sg(reg->mr, sg, nents, &offset, PAGE_SIZE);
+ if (ret < nents) {
+ ib_mr_pool_put(qp, &qp->rdma_mrs, reg->mr);
+ return -EINVAL;
+ }
+
+ reg->reg_wr.wr.opcode = IB_WR_REG_MR;
+ reg->reg_wr.mr = reg->mr;
+ reg->reg_wr.access = IB_ACCESS_LOCAL_WRITE;
+ if (rdma_protocol_iwarp(qp->device, port_num))
+ reg->reg_wr.access |= IB_ACCESS_REMOTE_WRITE;
+ count++;
+
+ reg->sge.addr = reg->mr->iova;
+ reg->sge.length = reg->mr->length;
+ return count;
+}
+
+static int rdma_rw_init_mr_wrs(struct rdma_rw_ctx *ctx, struct ib_qp *qp,
+ u8 port_num, struct scatterlist *sg, u32 sg_cnt, u32 offset,
+ u64 remote_addr, u32 rkey, enum dma_data_direction dir)
+{
+ u32 pages_per_mr = rdma_rw_fr_page_list_len(qp->pd->device);
+ int i, j, ret = 0, count = 0;
+
+ ctx->nr_ops = (sg_cnt + pages_per_mr - 1) / pages_per_mr;
+ ctx->reg = kcalloc(ctx->nr_ops, sizeof(*ctx->reg), GFP_KERNEL);
+ if (!ctx->reg) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ for (i = 0; i < ctx->nr_ops; i++) {
+ struct rdma_rw_reg_ctx *prev = i ? &ctx->reg[i - 1] : NULL;
+ struct rdma_rw_reg_ctx *reg = &ctx->reg[i];
+ u32 nents = min(sg_cnt, pages_per_mr);
+
+ ret = rdma_rw_init_one_mr(qp, port_num, reg, sg, sg_cnt,
+ offset);
+ if (ret < 0)
+ goto out_free;
+ count += ret;
+
+ if (prev) {
+ if (reg->mr->need_inval)
+ prev->wr.wr.next = ®->inv_wr;
+ else
+ prev->wr.wr.next = ®->reg_wr.wr;
+ }
+
+ reg->reg_wr.wr.next = ®->wr.wr;
+
+ reg->wr.wr.sg_list = ®->sge;
+ reg->wr.wr.num_sge = 1;
+ reg->wr.remote_addr = remote_addr;
+ reg->wr.rkey = rkey;
+ if (dir == DMA_TO_DEVICE) {
+ reg->wr.wr.opcode = IB_WR_RDMA_WRITE;
+ } else if (!rdma_cap_read_inv(qp->device, port_num)) {
+ reg->wr.wr.opcode = IB_WR_RDMA_READ;
+ } else {
+ reg->wr.wr.opcode = IB_WR_RDMA_READ_WITH_INV;
+ reg->wr.wr.ex.invalidate_rkey = reg->mr->lkey;
+ }
+ count++;
+
+ remote_addr += reg->sge.length;
+ sg_cnt -= nents;
+ for (j = 0; j < nents; j++)
+ sg = sg_next(sg);
+ offset = 0;
+ }
+
+ ctx->type = RDMA_RW_MR;
+ return count;
+
+out_free:
+ while (--i >= 0)
+ ib_mr_pool_put(qp, &qp->rdma_mrs, ctx->reg[i].mr);
+ kfree(ctx->reg);
+out:
+ return ret;
+}
+
+static int rdma_rw_init_map_wrs(struct rdma_rw_ctx *ctx, struct ib_qp *qp,
+ struct scatterlist *sg, u32 sg_cnt, u32 offset,
+ u64 remote_addr, u32 rkey, enum dma_data_direction dir)
+{
+ struct ib_device *dev = qp->pd->device;
+ u32 max_sge = rdma_rw_max_sge(dev, dir);
+ struct ib_sge *sge;
+ u32 total_len = 0, i, j;
+
+ ctx->nr_ops = DIV_ROUND_UP(sg_cnt, max_sge);
+
+ ctx->map.sges = sge = kcalloc(sg_cnt, sizeof(*sge), GFP_KERNEL);
+ if (!ctx->map.sges)
+ goto out;
+
+ ctx->map.wrs = kcalloc(ctx->nr_ops, sizeof(*ctx->map.wrs), GFP_KERNEL);
+ if (!ctx->map.wrs)
+ goto out_free_sges;
+
+ for (i = 0; i < ctx->nr_ops; i++) {
+ struct ib_rdma_wr *rdma_wr = &ctx->map.wrs[i];
+ u32 nr_sge = min(sg_cnt, max_sge);
+
+ if (dir == DMA_TO_DEVICE)
+ rdma_wr->wr.opcode = IB_WR_RDMA_WRITE;
+ else
+ rdma_wr->wr.opcode = IB_WR_RDMA_READ;
+ rdma_wr->remote_addr = remote_addr + total_len;
+ rdma_wr->rkey = rkey;
+ rdma_wr->wr.sg_list = sge;
+
+ for (j = 0; j < nr_sge; j++, sg = sg_next(sg)) {
+ rdma_wr->wr.num_sge++;
+
+ sge->addr = ib_sg_dma_address(dev, sg) + offset;
+ sge->length = ib_sg_dma_len(dev, sg) - offset;
+ sge->lkey = qp->pd->local_dma_lkey;
+
+ total_len += sge->length;
+ sge++;
+ sg_cnt--;
+ offset = 0;
+ }
+
+ if (i + 1 < ctx->nr_ops)
+ rdma_wr->wr.next = &ctx->map.wrs[i + 1].wr;
+ }
+
+ ctx->type = RDMA_RW_MULTI_WR;
+ return ctx->nr_ops;
+
+out_free_sges:
+ kfree(ctx->map.sges);
+out:
+ return -ENOMEM;
+}
+
+static int rdma_rw_init_single_wr(struct rdma_rw_ctx *ctx, struct ib_qp *qp,
+ struct scatterlist *sg, u32 offset, u64 remote_addr, u32 rkey,
+ enum dma_data_direction dir)
+{
+ struct ib_device *dev = qp->pd->device;
+ struct ib_rdma_wr *rdma_wr = &ctx->single.wr;
+
+ ctx->nr_ops = 1;
+
+ ctx->single.sge.lkey = qp->pd->local_dma_lkey;
+ ctx->single.sge.addr = ib_sg_dma_address(dev, sg) + offset;
+ ctx->single.sge.length = ib_sg_dma_len(dev, sg) - offset;
+
+ memset(rdma_wr, 0, sizeof(*rdma_wr));
+ if (dir == DMA_TO_DEVICE)
+ rdma_wr->wr.opcode = IB_WR_RDMA_WRITE;
+ else
+ rdma_wr->wr.opcode = IB_WR_RDMA_READ;
+ rdma_wr->wr.sg_list = &ctx->single.sge;
+ rdma_wr->wr.num_sge = 1;
+ rdma_wr->remote_addr = remote_addr;
+ rdma_wr->rkey = rkey;
+
+ ctx->type = RDMA_RW_SINGLE_WR;
+ return 1;
+}
+
+/**
+ * rdma_rw_ctx_init - initialize a RDMA READ/WRITE context
+ * @ctx: context to initialize
+ * @qp: queue pair to operate on
+ * @port_num: port num to which the connection is bound
+ * @sg: scatterlist to READ/WRITE from/to
+ * @sg_cnt: number of entries in @sg
+ * @sg_offset: current byte offset into @sg
+ * @remote_addr:remote address to read/write (relative to @rkey)
+ * @rkey: remote key to operate on
+ * @dir: %DMA_TO_DEVICE for RDMA WRITE, %DMA_FROM_DEVICE for RDMA READ
+ *
+ * Returns the number of WQEs that will be needed on the workqueue if
+ * successful, or a negative error code.
+ */
+int rdma_rw_ctx_init(struct rdma_rw_ctx *ctx, struct ib_qp *qp, u8 port_num,
+ struct scatterlist *sg, u32 sg_cnt, u32 sg_offset,
+ u64 remote_addr, u32 rkey, enum dma_data_direction dir)
+{
+ struct ib_device *dev = qp->pd->device;
+ int ret;
+
+ ret = ib_dma_map_sg(dev, sg, sg_cnt, dir);
+ if (!ret)
+ return -ENOMEM;
+ sg_cnt = ret;
+
+ /*
+ * Skip to the S/G entry that sg_offset falls into:
+ */
+ for (;;) {
+ u32 len = ib_sg_dma_len(dev, sg);
+
+ if (sg_offset < len)
+ break;
+
+ sg = sg_next(sg);
+ sg_offset -= len;
+ sg_cnt--;
+ }
+
+ ret = -EIO;
+ if (WARN_ON_ONCE(sg_cnt == 0))
+ goto out_unmap_sg;
+
+ if (rdma_rw_io_needs_mr(qp->device, port_num, dir, sg_cnt)) {
+ ret = rdma_rw_init_mr_wrs(ctx, qp, port_num, sg, sg_cnt,
+ sg_offset, remote_addr, rkey, dir);
+ } else if (sg_cnt > 1) {
+ ret = rdma_rw_init_map_wrs(ctx, qp, sg, sg_cnt, sg_offset,
+ remote_addr, rkey, dir);
+ } else {
+ ret = rdma_rw_init_single_wr(ctx, qp, sg, sg_offset,
+ remote_addr, rkey, dir);
+ }
+
+ if (ret < 0)
+ goto out_unmap_sg;
+ return ret;
+
+out_unmap_sg:
+ ib_dma_unmap_sg(dev, sg, sg_cnt, dir);
+ return ret;
+}
+EXPORT_SYMBOL(rdma_rw_ctx_init);
+
+/**
+ * rdma_rw_ctx_signature init - initialize a RW context with signature offload
+ * @ctx: context to initialize
+ * @qp: queue pair to operate on
+ * @port_num: port num to which the connection is bound
+ * @sg: scatterlist to READ/WRITE from/to
+ * @sg_cnt: number of entries in @sg
+ * @prot_sg: scatterlist to READ/WRITE protection information from/to
+ * @prot_sg_cnt: number of entries in @prot_sg
+ * @sig_attrs: signature offloading algorithms
+ * @remote_addr:remote address to read/write (relative to @rkey)
+ * @rkey: remote key to operate on
+ * @dir: %DMA_TO_DEVICE for RDMA WRITE, %DMA_FROM_DEVICE for RDMA READ
+ *
+ * Returns the number of WQEs that will be needed on the workqueue if
+ * successful, or a negative error code.
+ */
+int rdma_rw_ctx_signature_init(struct rdma_rw_ctx *ctx, struct ib_qp *qp,
+ u8 port_num, struct scatterlist *sg, u32 sg_cnt,
+ struct scatterlist *prot_sg, u32 prot_sg_cnt,
+ struct ib_sig_attrs *sig_attrs,
+ u64 remote_addr, u32 rkey, enum dma_data_direction dir)
+{
+ struct ib_device *dev = qp->pd->device;
+ u32 pages_per_mr = rdma_rw_fr_page_list_len(qp->pd->device);
+ struct ib_rdma_wr *rdma_wr;
+ struct ib_send_wr *prev_wr = NULL;
+ int count = 0, ret;
+
+ if (sg_cnt > pages_per_mr || prot_sg_cnt > pages_per_mr) {
+ pr_err("SG count too large\n");
+ return -EINVAL;
+ }
+
+ ret = ib_dma_map_sg(dev, sg, sg_cnt, dir);
+ if (!ret)
+ return -ENOMEM;
+ sg_cnt = ret;
+
+ ret = ib_dma_map_sg(dev, prot_sg, prot_sg_cnt, dir);
+ if (!ret) {
+ ret = -ENOMEM;
+ goto out_unmap_sg;
+ }
+ prot_sg_cnt = ret;
+
+ ctx->type = RDMA_RW_SIG_MR;
+ ctx->nr_ops = 1;
+ ctx->sig = kcalloc(1, sizeof(*ctx->sig), GFP_KERNEL);
+ if (!ctx->sig) {
+ ret = -ENOMEM;
+ goto out_unmap_prot_sg;
+ }
+
+ ret = rdma_rw_init_one_mr(qp, port_num, &ctx->sig->data, sg, sg_cnt, 0);
+ if (ret < 0)
+ goto out_free_ctx;
+ count += ret;
+ prev_wr = &ctx->sig->data.reg_wr.wr;
+
+ if (prot_sg_cnt) {
+ ret = rdma_rw_init_one_mr(qp, port_num, &ctx->sig->prot,
+ prot_sg, prot_sg_cnt, 0);
+ if (ret < 0)
+ goto out_destroy_data_mr;
+ count += ret;
+
+ if (ctx->sig->prot.inv_wr.next)
+ prev_wr->next = &ctx->sig->prot.inv_wr;
+ else
+ prev_wr->next = &ctx->sig->prot.reg_wr.wr;
+ prev_wr = &ctx->sig->prot.reg_wr.wr;
+ } else {
+ ctx->sig->prot.mr = NULL;
+ }
+
+ ctx->sig->sig_mr = ib_mr_pool_get(qp, &qp->sig_mrs);
+ if (!ctx->sig->sig_mr) {
+ ret = -EAGAIN;
+ goto out_destroy_prot_mr;
+ }
+
+ if (ctx->sig->sig_mr->need_inval) {
+ memset(&ctx->sig->sig_inv_wr, 0, sizeof(ctx->sig->sig_inv_wr));
+
+ ctx->sig->sig_inv_wr.opcode = IB_WR_LOCAL_INV;
+ ctx->sig->sig_inv_wr.ex.invalidate_rkey = ctx->sig->sig_mr->rkey;
+
+ prev_wr->next = &ctx->sig->sig_inv_wr;
+ prev_wr = &ctx->sig->sig_inv_wr;
+ }
+
+ ctx->sig->sig_wr.wr.opcode = IB_WR_REG_SIG_MR;
+ ctx->sig->sig_wr.wr.wr_cqe = NULL;
+ ctx->sig->sig_wr.wr.sg_list = &ctx->sig->data.sge;
+ ctx->sig->sig_wr.wr.num_sge = 1;
+ ctx->sig->sig_wr.access_flags = IB_ACCESS_LOCAL_WRITE;
+ ctx->sig->sig_wr.sig_attrs = sig_attrs;
+ ctx->sig->sig_wr.sig_mr = ctx->sig->sig_mr;
+ if (prot_sg_cnt)
+ ctx->sig->sig_wr.prot = &ctx->sig->prot.sge;
+ prev_wr->next = &ctx->sig->sig_wr.wr;
+ prev_wr = &ctx->sig->sig_wr.wr;
+ count++;
+
+ ctx->sig->sig_sge.addr = 0;
+ ctx->sig->sig_sge.length = ctx->sig->data.sge.length;
+ if (sig_attrs->wire.sig_type != IB_SIG_TYPE_NONE)
+ ctx->sig->sig_sge.length += ctx->sig->prot.sge.length;
+
+ rdma_wr = &ctx->sig->data.wr;
+ rdma_wr->wr.sg_list = &ctx->sig->sig_sge;
+ rdma_wr->wr.num_sge = 1;
+ rdma_wr->remote_addr = remote_addr;
+ rdma_wr->rkey = rkey;
+ if (dir == DMA_TO_DEVICE)
+ rdma_wr->wr.opcode = IB_WR_RDMA_WRITE;
+ else
+ rdma_wr->wr.opcode = IB_WR_RDMA_READ;
+ prev_wr->next = &rdma_wr->wr;
+ prev_wr = &rdma_wr->wr;
+ count++;
+
+ return count;
+
+out_destroy_prot_mr:
+ if (prot_sg_cnt)
+ ib_mr_pool_put(qp, &qp->rdma_mrs, ctx->sig->prot.mr);
+out_destroy_data_mr:
+ ib_mr_pool_put(qp, &qp->rdma_mrs, ctx->sig->data.mr);
+out_free_ctx:
+ kfree(ctx->sig);
+out_unmap_prot_sg:
+ ib_dma_unmap_sg(dev, prot_sg, prot_sg_cnt, dir);
+out_unmap_sg:
+ ib_dma_unmap_sg(dev, sg, sg_cnt, dir);
+ return ret;
+}
+EXPORT_SYMBOL(rdma_rw_ctx_signature_init);
+
+/*
+ * Now that we are going to post the WRs we can update the lkey and need_inval
+ * state on the MRs. If we were doing this at init time, we would get double
+ * or missing invalidations if a context was initialized but not actually
+ * posted.
+ */
+static void rdma_rw_update_lkey(struct rdma_rw_reg_ctx *reg, bool need_inval)
+{
+ reg->mr->need_inval = need_inval;
+ ib_update_fast_reg_key(reg->mr, ib_inc_rkey(reg->mr->lkey));
+ reg->reg_wr.key = reg->mr->lkey;
+ reg->sge.lkey = reg->mr->lkey;
+}
+
+/**
+ * rdma_rw_ctx_wrs - return chain of WRs for a RDMA READ or WRITE operation
+ * @ctx: context to operate on
+ * @qp: queue pair to operate on
+ * @port_num: port num to which the connection is bound
+ * @cqe: completion queue entry for the last WR
+ * @chain_wr: WR to append to the posted chain
+ *
+ * Return the WR chain for the set of RDMA READ/WRITE operations described by
+ * @ctx, as well as any memory registration operations needed. If @chain_wr
+ * is non-NULL the WR it points to will be appended to the chain of WRs posted.
+ * If @chain_wr is not set @cqe must be set so that the caller gets a
+ * completion notification.
+ */
+struct ib_send_wr *rdma_rw_ctx_wrs(struct rdma_rw_ctx *ctx, struct ib_qp *qp,
+ u8 port_num, struct ib_cqe *cqe, struct ib_send_wr *chain_wr)
+{
+ struct ib_send_wr *first_wr, *last_wr;
+ int i;
+
+ switch (ctx->type) {
+ case RDMA_RW_SIG_MR:
+ rdma_rw_update_lkey(&ctx->sig->data, true);
+ if (ctx->sig->prot.mr)
+ rdma_rw_update_lkey(&ctx->sig->prot, true);
+
+ ctx->sig->sig_mr->need_inval = true;
+ ib_update_fast_reg_key(ctx->sig->sig_mr,
+ ib_inc_rkey(ctx->sig->sig_mr->lkey));
+ ctx->sig->sig_sge.lkey = ctx->sig->sig_mr->lkey;
+
+ if (ctx->sig->data.inv_wr.next)
+ first_wr = &ctx->sig->data.inv_wr;
+ else
+ first_wr = &ctx->sig->data.reg_wr.wr;
+ last_wr = &ctx->sig->data.wr.wr;
+ break;
+ case RDMA_RW_MR:
+ for (i = 0; i < ctx->nr_ops; i++) {
+ rdma_rw_update_lkey(&ctx->reg[i],
+ ctx->reg[i].wr.wr.opcode !=
+ IB_WR_RDMA_READ_WITH_INV);
+ }
+
+ if (ctx->reg[0].inv_wr.next)
+ first_wr = &ctx->reg[0].inv_wr;
+ else
+ first_wr = &ctx->reg[0].reg_wr.wr;
+ last_wr = &ctx->reg[ctx->nr_ops - 1].wr.wr;
+ break;
+ case RDMA_RW_MULTI_WR:
+ first_wr = &ctx->map.wrs[0].wr;
+ last_wr = &ctx->map.wrs[ctx->nr_ops - 1].wr;
+ break;
+ case RDMA_RW_SINGLE_WR:
+ first_wr = &ctx->single.wr.wr;
+ last_wr = &ctx->single.wr.wr;
+ break;
+ default:
+ BUG();
+ }
+
+ if (chain_wr) {
+ last_wr->next = chain_wr;
+ } else {
+ last_wr->wr_cqe = cqe;
+ last_wr->send_flags |= IB_SEND_SIGNALED;
+ }
+
+ return first_wr;
+}
+EXPORT_SYMBOL(rdma_rw_ctx_wrs);
+
+/**
+ * rdma_rw_ctx_post - post a RDMA READ or RDMA WRITE operation
+ * @ctx: context to operate on
+ * @qp: queue pair to operate on
+ * @port_num: port num to which the connection is bound
+ * @cqe: completion queue entry for the last WR
+ * @chain_wr: WR to append to the posted chain
+ *
+ * Post the set of RDMA READ/WRITE operations described by @ctx, as well as
+ * any memory registration operations needed. If @chain_wr is non-NULL the
+ * WR it points to will be appended to the chain of WRs posted. If @chain_wr
+ * is not set @cqe must be set so that the caller gets a completion
+ * notification.
+ */
+int rdma_rw_ctx_post(struct rdma_rw_ctx *ctx, struct ib_qp *qp, u8 port_num,
+ struct ib_cqe *cqe, struct ib_send_wr *chain_wr)
+{
+ struct ib_send_wr *first_wr, *bad_wr;
+
+ first_wr = rdma_rw_ctx_wrs(ctx, qp, port_num, cqe, chain_wr);
+ return ib_post_send(qp, first_wr, &bad_wr);
+}
+EXPORT_SYMBOL(rdma_rw_ctx_post);
+
+/**
+ * rdma_rw_ctx_destroy - release all resources allocated by rdma_rw_ctx_init
+ * @ctx: context to release
+ * @qp: queue pair to operate on
+ * @port_num: port num to which the connection is bound
+ * @sg: scatterlist that was used for the READ/WRITE
+ * @sg_cnt: number of entries in @sg
+ * @dir: %DMA_TO_DEVICE for RDMA WRITE, %DMA_FROM_DEVICE for RDMA READ
+ */
+void rdma_rw_ctx_destroy(struct rdma_rw_ctx *ctx, struct ib_qp *qp, u8 port_num,
+ struct scatterlist *sg, u32 sg_cnt, enum dma_data_direction dir)
+{
+ int i;
+
+ switch (ctx->type) {
+ case RDMA_RW_MR:
+ for (i = 0; i < ctx->nr_ops; i++)
+ ib_mr_pool_put(qp, &qp->rdma_mrs, ctx->reg[i].mr);
+ kfree(ctx->reg);
+ break;
+ case RDMA_RW_MULTI_WR:
+ kfree(ctx->map.wrs);
+ kfree(ctx->map.sges);
+ break;
+ case RDMA_RW_SINGLE_WR:
+ break;
+ default:
+ BUG();
+ break;
+ }
+
+ ib_dma_unmap_sg(qp->pd->device, sg, sg_cnt, dir);
+}
+EXPORT_SYMBOL(rdma_rw_ctx_destroy);
+
+/**
+ * rdma_rw_ctx_destroy_signature - release all resources allocated by
+ * rdma_rw_ctx_init_signature
+ * @ctx: context to release
+ * @qp: queue pair to operate on
+ * @port_num: port num to which the connection is bound
+ * @sg: scatterlist that was used for the READ/WRITE
+ * @sg_cnt: number of entries in @sg
+ * @prot_sg: scatterlist that was used for the READ/WRITE of the PI
+ * @prot_sg_cnt: number of entries in @prot_sg
+ * @dir: %DMA_TO_DEVICE for RDMA WRITE, %DMA_FROM_DEVICE for RDMA READ
+ */
+void rdma_rw_ctx_destroy_signature(struct rdma_rw_ctx *ctx, struct ib_qp *qp,
+ u8 port_num, struct scatterlist *sg, u32 sg_cnt,
+ struct scatterlist *prot_sg, u32 prot_sg_cnt,
+ enum dma_data_direction dir)
+{
+ if (WARN_ON_ONCE(ctx->type != RDMA_RW_SIG_MR))
+ return;
+
+ ib_mr_pool_put(qp, &qp->rdma_mrs, ctx->sig->data.mr);
+ ib_dma_unmap_sg(qp->pd->device, sg, sg_cnt, dir);
+
+ if (ctx->sig->prot.mr) {
+ ib_mr_pool_put(qp, &qp->rdma_mrs, ctx->sig->prot.mr);
+ ib_dma_unmap_sg(qp->pd->device, prot_sg, prot_sg_cnt, dir);
+ }
+
+ ib_mr_pool_put(qp, &qp->sig_mrs, ctx->sig->sig_mr);
+ kfree(ctx->sig);
+}
+EXPORT_SYMBOL(rdma_rw_ctx_destroy_signature);
+
+void rdma_rw_init_qp(struct ib_device *dev, struct ib_qp_init_attr *attr)
+{
+ u32 factor;
+
+ WARN_ON_ONCE(attr->port_num == 0);
+
+ /*
+ * Each context needs at least one RDMA READ or WRITE WR.
+ *
+ * For some hardware we might need more, eventually we should ask the
+ * HCA driver for a multiplier here.
+ */
+ factor = 1;
+
+ /*
+ * If the devices needs MRs to perform RDMA READ or WRITE operations,
+ * we'll need two additional MRs for the registrations and the
+ * invalidation.
+ */
+ if (attr->create_flags & IB_QP_CREATE_SIGNATURE_EN)
+ factor += 6; /* (inv + reg) * (data + prot + sig) */
+ else if (rdma_rw_can_use_mr(dev, attr->port_num))
+ factor += 2; /* inv + reg */
+
+ attr->cap.max_send_wr += factor * attr->cap.max_rdma_ctxs;
+
+ /*
+ * But maybe we were just too high in the sky and the device doesn't
+ * even support all we need, and we'll have to live with what we get..
+ */
+ attr->cap.max_send_wr =
+ min_t(u32, attr->cap.max_send_wr, dev->attrs.max_qp_wr);
+}
+
+int rdma_rw_init_mrs(struct ib_qp *qp, struct ib_qp_init_attr *attr)
+{
+ struct ib_device *dev = qp->pd->device;
+ u32 nr_mrs = 0, nr_sig_mrs = 0;
+ int ret = 0;
+
+ if (attr->create_flags & IB_QP_CREATE_SIGNATURE_EN) {
+ nr_sig_mrs = attr->cap.max_rdma_ctxs;
+ nr_mrs = attr->cap.max_rdma_ctxs * 2;
+ } else if (rdma_rw_can_use_mr(dev, attr->port_num)) {
+ nr_mrs = attr->cap.max_rdma_ctxs;
+ }
+
+ if (nr_mrs) {
+ ret = ib_mr_pool_init(qp, &qp->rdma_mrs, nr_mrs,
+ IB_MR_TYPE_MEM_REG,
+ rdma_rw_fr_page_list_len(dev));
+ if (ret) {
+ pr_err("%s: failed to allocated %d MRs\n",
+ __func__, nr_mrs);
+ return ret;
+ }
+ }
+
+ if (nr_sig_mrs) {
+ ret = ib_mr_pool_init(qp, &qp->sig_mrs, nr_sig_mrs,
+ IB_MR_TYPE_SIGNATURE, 2);
+ if (ret) {
+ pr_err("%s: failed to allocated %d SIG MRs\n",
+ __func__, nr_mrs);
+ goto out_free_rdma_mrs;
+ }
+ }
+
+ return 0;
+
+out_free_rdma_mrs:
+ ib_mr_pool_destroy(qp, &qp->rdma_mrs);
+ return ret;
+}
+
+void rdma_rw_cleanup_mrs(struct ib_qp *qp)
+{
+ ib_mr_pool_destroy(qp, &qp->sig_mrs);
+ ib_mr_pool_destroy(qp, &qp->rdma_mrs);
+}
diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c
index 8a09c0f..3ebd108 100644
--- a/drivers/infiniband/core/sa_query.c
+++ b/drivers/infiniband/core/sa_query.c
@@ -536,7 +536,7 @@
data = ibnl_put_msg(skb, &nlh, query->seq, 0, RDMA_NL_LS,
RDMA_NL_LS_OP_RESOLVE, NLM_F_REQUEST);
if (!data) {
- kfree_skb(skb);
+ nlmsg_free(skb);
return -EMSGSIZE;
}
@@ -1820,7 +1820,7 @@
goto err3;
}
- if (ibnl_add_client(RDMA_NL_LS, RDMA_NL_LS_NUM_OPS,
+ if (ibnl_add_client(RDMA_NL_LS, ARRAY_SIZE(ib_sa_cb_table),
ib_sa_cb_table)) {
pr_err("Failed to add netlink callback\n");
ret = -EINVAL;
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index 6fdc7ec..1a8babb 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -1833,7 +1833,8 @@
if (attr.create_flags & ~(IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK |
IB_QP_CREATE_CROSS_CHANNEL |
IB_QP_CREATE_MANAGED_SEND |
- IB_QP_CREATE_MANAGED_RECV)) {
+ IB_QP_CREATE_MANAGED_RECV |
+ IB_QP_CREATE_SCATTER_FCS)) {
ret = -EINVAL;
goto err_put;
}
@@ -3088,8 +3089,7 @@
if (cmd.comp_mask)
return -EINVAL;
- if ((cmd.flow_attr.type == IB_FLOW_ATTR_SNIFFER &&
- !capable(CAP_NET_ADMIN)) || !capable(CAP_NET_RAW))
+ if (!capable(CAP_NET_RAW))
return -EPERM;
if (cmd.flow_attr.flags >= IB_FLOW_ATTR_FLAGS_RESERVED)
@@ -3655,6 +3655,11 @@
resp.hca_core_clock = attr.hca_core_clock;
resp.response_length += sizeof(resp.hca_core_clock);
+ if (ucore->outlen < resp.response_length + sizeof(resp.device_cap_flags_ex))
+ goto end;
+
+ resp.device_cap_flags_ex = attr.device_cap_flags;
+ resp.response_length += sizeof(resp.device_cap_flags_ex);
end:
err = ib_copy_to_udata(ucore, &resp, resp.response_length);
return err;
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index b65b354..1d7d4cf 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -48,6 +48,7 @@
#include <rdma/ib_verbs.h>
#include <rdma/ib_cache.h>
#include <rdma/ib_addr.h>
+#include <rdma/rw.h>
#include "core_priv.h"
@@ -723,59 +724,89 @@
}
EXPORT_SYMBOL(ib_open_qp);
+static struct ib_qp *ib_create_xrc_qp(struct ib_qp *qp,
+ struct ib_qp_init_attr *qp_init_attr)
+{
+ struct ib_qp *real_qp = qp;
+
+ qp->event_handler = __ib_shared_qp_event_handler;
+ qp->qp_context = qp;
+ qp->pd = NULL;
+ qp->send_cq = qp->recv_cq = NULL;
+ qp->srq = NULL;
+ qp->xrcd = qp_init_attr->xrcd;
+ atomic_inc(&qp_init_attr->xrcd->usecnt);
+ INIT_LIST_HEAD(&qp->open_list);
+
+ qp = __ib_open_qp(real_qp, qp_init_attr->event_handler,
+ qp_init_attr->qp_context);
+ if (!IS_ERR(qp))
+ __ib_insert_xrcd_qp(qp_init_attr->xrcd, real_qp);
+ else
+ real_qp->device->destroy_qp(real_qp);
+ return qp;
+}
+
struct ib_qp *ib_create_qp(struct ib_pd *pd,
struct ib_qp_init_attr *qp_init_attr)
{
- struct ib_qp *qp, *real_qp;
- struct ib_device *device;
+ struct ib_device *device = pd ? pd->device : qp_init_attr->xrcd->device;
+ struct ib_qp *qp;
+ int ret;
- device = pd ? pd->device : qp_init_attr->xrcd->device;
+ /*
+ * If the callers is using the RDMA API calculate the resources
+ * needed for the RDMA READ/WRITE operations.
+ *
+ * Note that these callers need to pass in a port number.
+ */
+ if (qp_init_attr->cap.max_rdma_ctxs)
+ rdma_rw_init_qp(device, qp_init_attr);
+
qp = device->create_qp(pd, qp_init_attr, NULL);
+ if (IS_ERR(qp))
+ return qp;
- if (!IS_ERR(qp)) {
- qp->device = device;
- qp->real_qp = qp;
- qp->uobject = NULL;
- qp->qp_type = qp_init_attr->qp_type;
+ qp->device = device;
+ qp->real_qp = qp;
+ qp->uobject = NULL;
+ qp->qp_type = qp_init_attr->qp_type;
- atomic_set(&qp->usecnt, 0);
- if (qp_init_attr->qp_type == IB_QPT_XRC_TGT) {
- qp->event_handler = __ib_shared_qp_event_handler;
- qp->qp_context = qp;
- qp->pd = NULL;
- qp->send_cq = qp->recv_cq = NULL;
- qp->srq = NULL;
- qp->xrcd = qp_init_attr->xrcd;
- atomic_inc(&qp_init_attr->xrcd->usecnt);
- INIT_LIST_HEAD(&qp->open_list);
+ atomic_set(&qp->usecnt, 0);
+ qp->mrs_used = 0;
+ spin_lock_init(&qp->mr_lock);
+ INIT_LIST_HEAD(&qp->rdma_mrs);
+ INIT_LIST_HEAD(&qp->sig_mrs);
- real_qp = qp;
- qp = __ib_open_qp(real_qp, qp_init_attr->event_handler,
- qp_init_attr->qp_context);
- if (!IS_ERR(qp))
- __ib_insert_xrcd_qp(qp_init_attr->xrcd, real_qp);
- else
- real_qp->device->destroy_qp(real_qp);
- } else {
- qp->event_handler = qp_init_attr->event_handler;
- qp->qp_context = qp_init_attr->qp_context;
- if (qp_init_attr->qp_type == IB_QPT_XRC_INI) {
- qp->recv_cq = NULL;
- qp->srq = NULL;
- } else {
- qp->recv_cq = qp_init_attr->recv_cq;
- atomic_inc(&qp_init_attr->recv_cq->usecnt);
- qp->srq = qp_init_attr->srq;
- if (qp->srq)
- atomic_inc(&qp_init_attr->srq->usecnt);
- }
+ if (qp_init_attr->qp_type == IB_QPT_XRC_TGT)
+ return ib_create_xrc_qp(qp, qp_init_attr);
- qp->pd = pd;
- qp->send_cq = qp_init_attr->send_cq;
- qp->xrcd = NULL;
+ qp->event_handler = qp_init_attr->event_handler;
+ qp->qp_context = qp_init_attr->qp_context;
+ if (qp_init_attr->qp_type == IB_QPT_XRC_INI) {
+ qp->recv_cq = NULL;
+ qp->srq = NULL;
+ } else {
+ qp->recv_cq = qp_init_attr->recv_cq;
+ atomic_inc(&qp_init_attr->recv_cq->usecnt);
+ qp->srq = qp_init_attr->srq;
+ if (qp->srq)
+ atomic_inc(&qp_init_attr->srq->usecnt);
+ }
- atomic_inc(&pd->usecnt);
- atomic_inc(&qp_init_attr->send_cq->usecnt);
+ qp->pd = pd;
+ qp->send_cq = qp_init_attr->send_cq;
+ qp->xrcd = NULL;
+
+ atomic_inc(&pd->usecnt);
+ atomic_inc(&qp_init_attr->send_cq->usecnt);
+
+ if (qp_init_attr->cap.max_rdma_ctxs) {
+ ret = rdma_rw_init_mrs(qp, qp_init_attr);
+ if (ret) {
+ pr_err("failed to init MR pool ret= %d\n", ret);
+ ib_destroy_qp(qp);
+ qp = ERR_PTR(ret);
}
}
@@ -1250,6 +1281,8 @@
struct ib_srq *srq;
int ret;
+ WARN_ON_ONCE(qp->mrs_used > 0);
+
if (atomic_read(&qp->usecnt))
return -EBUSY;
@@ -1261,6 +1294,9 @@
rcq = qp->recv_cq;
srq = qp->srq;
+ if (!qp->uobject)
+ rdma_rw_cleanup_mrs(qp);
+
ret = qp->device->destroy_qp(qp);
if (!ret) {
if (pd)
@@ -1343,6 +1379,7 @@
mr->pd = pd;
mr->uobject = NULL;
atomic_inc(&pd->usecnt);
+ mr->need_inval = false;
}
return mr;
@@ -1389,6 +1426,7 @@
mr->pd = pd;
mr->uobject = NULL;
atomic_inc(&pd->usecnt);
+ mr->need_inval = false;
}
return mr;
@@ -1597,6 +1635,7 @@
* @mr: memory region
* @sg: dma mapped scatterlist
* @sg_nents: number of entries in sg
+ * @sg_offset: offset in bytes into sg
* @page_size: page vector desired page size
*
* Constraints:
@@ -1615,17 +1654,15 @@
* After this completes successfully, the memory region
* is ready for registration.
*/
-int ib_map_mr_sg(struct ib_mr *mr,
- struct scatterlist *sg,
- int sg_nents,
- unsigned int page_size)
+int ib_map_mr_sg(struct ib_mr *mr, struct scatterlist *sg, int sg_nents,
+ unsigned int *sg_offset, unsigned int page_size)
{
if (unlikely(!mr->device->map_mr_sg))
return -ENOSYS;
mr->page_size = page_size;
- return mr->device->map_mr_sg(mr, sg, sg_nents);
+ return mr->device->map_mr_sg(mr, sg, sg_nents, sg_offset);
}
EXPORT_SYMBOL(ib_map_mr_sg);
@@ -1635,6 +1672,10 @@
* @mr: memory region
* @sgl: dma mapped scatterlist
* @sg_nents: number of entries in sg
+ * @sg_offset_p: IN: start offset in bytes into sg
+ * OUT: offset in bytes for element n of the sg of the first
+ * byte that has not been processed where n is the return
+ * value of this function.
* @set_page: driver page assignment function pointer
*
* Core service helper for drivers to convert the largest
@@ -1645,23 +1686,26 @@
* Returns the number of sg elements that were assigned to
* a page vector.
*/
-int ib_sg_to_pages(struct ib_mr *mr,
- struct scatterlist *sgl,
- int sg_nents,
- int (*set_page)(struct ib_mr *, u64))
+int ib_sg_to_pages(struct ib_mr *mr, struct scatterlist *sgl, int sg_nents,
+ unsigned int *sg_offset_p, int (*set_page)(struct ib_mr *, u64))
{
struct scatterlist *sg;
u64 last_end_dma_addr = 0;
+ unsigned int sg_offset = sg_offset_p ? *sg_offset_p : 0;
unsigned int last_page_off = 0;
u64 page_mask = ~((u64)mr->page_size - 1);
int i, ret;
- mr->iova = sg_dma_address(&sgl[0]);
+ if (unlikely(sg_nents <= 0 || sg_offset > sg_dma_len(&sgl[0])))
+ return -EINVAL;
+
+ mr->iova = sg_dma_address(&sgl[0]) + sg_offset;
mr->length = 0;
for_each_sg(sgl, sg, sg_nents, i) {
- u64 dma_addr = sg_dma_address(sg);
- unsigned int dma_len = sg_dma_len(sg);
+ u64 dma_addr = sg_dma_address(sg) + sg_offset;
+ u64 prev_addr = dma_addr;
+ unsigned int dma_len = sg_dma_len(sg) - sg_offset;
u64 end_dma_addr = dma_addr + dma_len;
u64 page_addr = dma_addr & page_mask;
@@ -1685,8 +1729,14 @@
do {
ret = set_page(mr, page_addr);
- if (unlikely(ret < 0))
- return i ? : ret;
+ if (unlikely(ret < 0)) {
+ sg_offset = prev_addr - sg_dma_address(sg);
+ mr->length += prev_addr - dma_addr;
+ if (sg_offset_p)
+ *sg_offset_p = sg_offset;
+ return i || sg_offset ? i : ret;
+ }
+ prev_addr = page_addr;
next_page:
page_addr += mr->page_size;
} while (page_addr < end_dma_addr);
@@ -1694,8 +1744,12 @@
mr->length += dma_len;
last_end_dma_addr = end_dma_addr;
last_page_off = end_dma_addr & ~page_mask;
+
+ sg_offset = 0;
}
+ if (sg_offset_p)
+ *sg_offset_p = 0;
return i;
}
EXPORT_SYMBOL(ib_sg_to_pages);
diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c b/drivers/infiniband/hw/cxgb3/iwch_provider.c
index 3234a8b..47cb927 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_provider.c
+++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c
@@ -783,15 +783,14 @@
return 0;
}
-static int iwch_map_mr_sg(struct ib_mr *ibmr,
- struct scatterlist *sg,
- int sg_nents)
+static int iwch_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg,
+ int sg_nents, unsigned int *sg_offset)
{
struct iwch_mr *mhp = to_iwch_mr(ibmr);
mhp->npages = 0;
- return ib_sg_to_pages(ibmr, sg, sg_nents, iwch_set_page);
+ return ib_sg_to_pages(ibmr, sg, sg_nents, sg_offset, iwch_set_page);
}
static int iwch_destroy_qp(struct ib_qp *ib_qp)
diff --git a/drivers/infiniband/hw/cxgb4/cm.c b/drivers/infiniband/hw/cxgb4/cm.c
index 6517113..a3a6721 100644
--- a/drivers/infiniband/hw/cxgb4/cm.c
+++ b/drivers/infiniband/hw/cxgb4/cm.c
@@ -119,7 +119,7 @@
static int mpa_rev = 2;
module_param(mpa_rev, int, 0644);
MODULE_PARM_DESC(mpa_rev, "MPA Revision, 0 supports amso1100, "
- "1 is RFC0544 spec compliant, 2 is IETF MPA Peer Connect Draft"
+ "1 is RFC5044 spec compliant, 2 is IETF MPA Peer Connect Draft"
" compliant (default=2)");
static int markers_enabled;
@@ -145,19 +145,35 @@
static struct sk_buff *get_skb(struct sk_buff *skb, int len, gfp_t gfp);
static void ep_timeout(unsigned long arg);
static void connect_reply_upcall(struct c4iw_ep *ep, int status);
+static int sched(struct c4iw_dev *dev, struct sk_buff *skb);
static LIST_HEAD(timeout_list);
static spinlock_t timeout_lock;
+static void deref_cm_id(struct c4iw_ep_common *epc)
+{
+ epc->cm_id->rem_ref(epc->cm_id);
+ epc->cm_id = NULL;
+ set_bit(CM_ID_DEREFED, &epc->history);
+}
+
+static void ref_cm_id(struct c4iw_ep_common *epc)
+{
+ set_bit(CM_ID_REFED, &epc->history);
+ epc->cm_id->add_ref(epc->cm_id);
+}
+
static void deref_qp(struct c4iw_ep *ep)
{
c4iw_qp_rem_ref(&ep->com.qp->ibqp);
clear_bit(QP_REFERENCED, &ep->com.flags);
+ set_bit(QP_DEREFED, &ep->com.history);
}
static void ref_qp(struct c4iw_ep *ep)
{
set_bit(QP_REFERENCED, &ep->com.flags);
+ set_bit(QP_REFED, &ep->com.history);
c4iw_qp_add_ref(&ep->com.qp->ibqp);
}
@@ -201,6 +217,8 @@
error = cxgb4_l2t_send(rdev->lldi.ports[0], skb, l2e);
if (error < 0)
kfree_skb(skb);
+ else if (error == NET_XMIT_DROP)
+ return -ENOMEM;
return error < 0 ? error : 0;
}
@@ -290,12 +308,63 @@
return epc;
}
+static void remove_ep_tid(struct c4iw_ep *ep)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&ep->com.dev->lock, flags);
+ _remove_handle(ep->com.dev, &ep->com.dev->hwtid_idr, ep->hwtid, 0);
+ spin_unlock_irqrestore(&ep->com.dev->lock, flags);
+}
+
+static void insert_ep_tid(struct c4iw_ep *ep)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&ep->com.dev->lock, flags);
+ _insert_handle(ep->com.dev, &ep->com.dev->hwtid_idr, ep, ep->hwtid, 0);
+ spin_unlock_irqrestore(&ep->com.dev->lock, flags);
+}
+
+/*
+ * Atomically lookup the ep ptr given the tid and grab a reference on the ep.
+ */
+static struct c4iw_ep *get_ep_from_tid(struct c4iw_dev *dev, unsigned int tid)
+{
+ struct c4iw_ep *ep;
+ unsigned long flags;
+
+ spin_lock_irqsave(&dev->lock, flags);
+ ep = idr_find(&dev->hwtid_idr, tid);
+ if (ep)
+ c4iw_get_ep(&ep->com);
+ spin_unlock_irqrestore(&dev->lock, flags);
+ return ep;
+}
+
+/*
+ * Atomically lookup the ep ptr given the stid and grab a reference on the ep.
+ */
+static struct c4iw_listen_ep *get_ep_from_stid(struct c4iw_dev *dev,
+ unsigned int stid)
+{
+ struct c4iw_listen_ep *ep;
+ unsigned long flags;
+
+ spin_lock_irqsave(&dev->lock, flags);
+ ep = idr_find(&dev->stid_idr, stid);
+ if (ep)
+ c4iw_get_ep(&ep->com);
+ spin_unlock_irqrestore(&dev->lock, flags);
+ return ep;
+}
+
void _c4iw_free_ep(struct kref *kref)
{
struct c4iw_ep *ep;
ep = container_of(kref, struct c4iw_ep, com.kref);
- PDBG("%s ep %p state %s\n", __func__, ep, states[state_read(&ep->com)]);
+ PDBG("%s ep %p state %s\n", __func__, ep, states[ep->com.state]);
if (test_bit(QP_REFERENCED, &ep->com.flags))
deref_qp(ep);
if (test_bit(RELEASE_RESOURCES, &ep->com.flags)) {
@@ -309,10 +378,11 @@
(const u32 *)&sin6->sin6_addr.s6_addr,
1);
}
- remove_handle(ep->com.dev, &ep->com.dev->hwtid_idr, ep->hwtid);
cxgb4_remove_tid(ep->com.dev->rdev.lldi.tids, 0, ep->hwtid);
dst_release(ep->dst);
cxgb4_l2t_release(ep->l2t);
+ if (ep->mpa_skb)
+ kfree_skb(ep->mpa_skb);
}
kfree(ep);
}
@@ -320,6 +390,15 @@
static void release_ep_resources(struct c4iw_ep *ep)
{
set_bit(RELEASE_RESOURCES, &ep->com.flags);
+
+ /*
+ * If we have a hwtid, then remove it from the idr table
+ * so lookups will no longer find this endpoint. Otherwise
+ * we have a race where one thread finds the ep ptr just
+ * before the other thread is freeing the ep memory.
+ */
+ if (ep->hwtid != -1)
+ remove_ep_tid(ep);
c4iw_put_ep(&ep->com);
}
@@ -432,10 +511,74 @@
static void arp_failure_discard(void *handle, struct sk_buff *skb)
{
- PDBG("%s c4iw_dev %p\n", __func__, handle);
+ pr_err(MOD "ARP failure\n");
kfree_skb(skb);
}
+static void mpa_start_arp_failure(void *handle, struct sk_buff *skb)
+{
+ pr_err("ARP failure during MPA Negotiation - Closing Connection\n");
+}
+
+enum {
+ NUM_FAKE_CPLS = 2,
+ FAKE_CPL_PUT_EP_SAFE = NUM_CPL_CMDS + 0,
+ FAKE_CPL_PASS_PUT_EP_SAFE = NUM_CPL_CMDS + 1,
+};
+
+static int _put_ep_safe(struct c4iw_dev *dev, struct sk_buff *skb)
+{
+ struct c4iw_ep *ep;
+
+ ep = *((struct c4iw_ep **)(skb->cb + 2 * sizeof(void *)));
+ release_ep_resources(ep);
+ return 0;
+}
+
+static int _put_pass_ep_safe(struct c4iw_dev *dev, struct sk_buff *skb)
+{
+ struct c4iw_ep *ep;
+
+ ep = *((struct c4iw_ep **)(skb->cb + 2 * sizeof(void *)));
+ c4iw_put_ep(&ep->parent_ep->com);
+ release_ep_resources(ep);
+ return 0;
+}
+
+/*
+ * Fake up a special CPL opcode and call sched() so process_work() will call
+ * _put_ep_safe() in a safe context to free the ep resources. This is needed
+ * because ARP error handlers are called in an ATOMIC context, and
+ * _c4iw_free_ep() needs to block.
+ */
+static void queue_arp_failure_cpl(struct c4iw_ep *ep, struct sk_buff *skb,
+ int cpl)
+{
+ struct cpl_act_establish *rpl = cplhdr(skb);
+
+ /* Set our special ARP_FAILURE opcode */
+ rpl->ot.opcode = cpl;
+
+ /*
+ * Save ep in the skb->cb area, after where sched() will save the dev
+ * ptr.
+ */
+ *((struct c4iw_ep **)(skb->cb + 2 * sizeof(void *))) = ep;
+ sched(ep->com.dev, skb);
+}
+
+/* Handle an ARP failure for an accept */
+static void pass_accept_rpl_arp_failure(void *handle, struct sk_buff *skb)
+{
+ struct c4iw_ep *ep = handle;
+
+ pr_err(MOD "ARP failure during accept - tid %u -dropping connection\n",
+ ep->hwtid);
+
+ __state_set(&ep->com, DEAD);
+ queue_arp_failure_cpl(ep, skb, FAKE_CPL_PASS_PUT_EP_SAFE);
+}
+
/*
* Handle an ARP failure for an active open.
*/
@@ -444,9 +587,8 @@
struct c4iw_ep *ep = handle;
printk(KERN_ERR MOD "ARP failure during connect\n");
- kfree_skb(skb);
connect_reply_upcall(ep, -EHOSTUNREACH);
- state_set(&ep->com, DEAD);
+ __state_set(&ep->com, DEAD);
if (ep->com.remote_addr.ss_family == AF_INET6) {
struct sockaddr_in6 *sin6 =
(struct sockaddr_in6 *)&ep->com.local_addr;
@@ -455,9 +597,7 @@
}
remove_handle(ep->com.dev, &ep->com.dev->atid_idr, ep->atid);
cxgb4_free_atid(ep->com.dev->rdev.lldi.tids, ep->atid);
- dst_release(ep->dst);
- cxgb4_l2t_release(ep->l2t);
- c4iw_put_ep(&ep->com);
+ queue_arp_failure_cpl(ep, skb, FAKE_CPL_PUT_EP_SAFE);
}
/*
@@ -466,15 +606,21 @@
*/
static void abort_arp_failure(void *handle, struct sk_buff *skb)
{
- struct c4iw_rdev *rdev = handle;
+ int ret;
+ struct c4iw_ep *ep = handle;
+ struct c4iw_rdev *rdev = &ep->com.dev->rdev;
struct cpl_abort_req *req = cplhdr(skb);
PDBG("%s rdev %p\n", __func__, rdev);
req->cmd = CPL_ABORT_NO_RST;
- c4iw_ofld_send(rdev, skb);
+ ret = c4iw_ofld_send(rdev, skb);
+ if (ret) {
+ __state_set(&ep->com, DEAD);
+ queue_arp_failure_cpl(ep, skb, FAKE_CPL_PUT_EP_SAFE);
+ }
}
-static void send_flowc(struct c4iw_ep *ep, struct sk_buff *skb)
+static int send_flowc(struct c4iw_ep *ep, struct sk_buff *skb)
{
unsigned int flowclen = 80;
struct fw_flowc_wr *flowc;
@@ -530,7 +676,7 @@
}
set_wr_txq(skb, CPL_PRIORITY_DATA, ep->txq_idx);
- c4iw_ofld_send(&ep->com.dev->rdev, skb);
+ return c4iw_ofld_send(&ep->com.dev->rdev, skb);
}
static int send_halfclose(struct c4iw_ep *ep, gfp_t gfp)
@@ -568,7 +714,7 @@
return -ENOMEM;
}
set_wr_txq(skb, CPL_PRIORITY_DATA, ep->txq_idx);
- t4_set_arp_err_handler(skb, &ep->com.dev->rdev, abort_arp_failure);
+ t4_set_arp_err_handler(skb, ep, abort_arp_failure);
req = (struct cpl_abort_req *) skb_put(skb, wrlen);
memset(req, 0, wrlen);
INIT_TP_WR(req, ep->hwtid);
@@ -807,10 +953,10 @@
return ret;
}
-static void send_mpa_req(struct c4iw_ep *ep, struct sk_buff *skb,
- u8 mpa_rev_to_use)
+static int send_mpa_req(struct c4iw_ep *ep, struct sk_buff *skb,
+ u8 mpa_rev_to_use)
{
- int mpalen, wrlen;
+ int mpalen, wrlen, ret;
struct fw_ofld_tx_data_wr *req;
struct mpa_message *mpa;
struct mpa_v2_conn_params mpa_v2_params;
@@ -826,7 +972,7 @@
skb = get_skb(skb, wrlen, GFP_KERNEL);
if (!skb) {
connect_reply_upcall(ep, -ENOMEM);
- return;
+ return -ENOMEM;
}
set_wr_txq(skb, CPL_PRIORITY_DATA, ep->txq_idx);
@@ -894,12 +1040,14 @@
t4_set_arp_err_handler(skb, NULL, arp_failure_discard);
BUG_ON(ep->mpa_skb);
ep->mpa_skb = skb;
- c4iw_l2t_send(&ep->com.dev->rdev, skb, ep->l2t);
+ ret = c4iw_l2t_send(&ep->com.dev->rdev, skb, ep->l2t);
+ if (ret)
+ return ret;
start_ep_timer(ep);
__state_set(&ep->com, MPA_REQ_SENT);
ep->mpa_attr.initiator = 1;
ep->snd_seq += mpalen;
- return;
+ return ret;
}
static int send_mpa_reject(struct c4iw_ep *ep, const void *pdata, u8 plen)
@@ -975,7 +1123,7 @@
*/
skb_get(skb);
set_wr_txq(skb, CPL_PRIORITY_DATA, ep->txq_idx);
- t4_set_arp_err_handler(skb, NULL, arp_failure_discard);
+ t4_set_arp_err_handler(skb, NULL, mpa_start_arp_failure);
BUG_ON(ep->mpa_skb);
ep->mpa_skb = skb;
ep->snd_seq += mpalen;
@@ -1060,7 +1208,7 @@
* Function fw4_ack() will deref it.
*/
skb_get(skb);
- t4_set_arp_err_handler(skb, NULL, arp_failure_discard);
+ t4_set_arp_err_handler(skb, NULL, mpa_start_arp_failure);
ep->mpa_skb = skb;
__state_set(&ep->com, MPA_REP_SENT);
ep->snd_seq += mpalen;
@@ -1074,6 +1222,7 @@
unsigned int tid = GET_TID(req);
unsigned int atid = TID_TID_G(ntohl(req->tos_atid));
struct tid_info *t = dev->rdev.lldi.tids;
+ int ret;
ep = lookup_atid(t, atid);
@@ -1086,7 +1235,7 @@
/* setup the hwtid for this connection */
ep->hwtid = tid;
cxgb4_insert_tid(t, ep, tid);
- insert_handle(dev, &dev->hwtid_idr, ep, ep->hwtid);
+ insert_ep_tid(ep);
ep->snd_seq = be32_to_cpu(req->snd_isn);
ep->rcv_seq = be32_to_cpu(req->rcv_isn);
@@ -1099,13 +1248,22 @@
set_bit(ACT_ESTAB, &ep->com.history);
/* start MPA negotiation */
- send_flowc(ep, NULL);
+ ret = send_flowc(ep, NULL);
+ if (ret)
+ goto err;
if (ep->retry_with_mpa_v1)
- send_mpa_req(ep, skb, 1);
+ ret = send_mpa_req(ep, skb, 1);
else
- send_mpa_req(ep, skb, mpa_rev);
+ ret = send_mpa_req(ep, skb, mpa_rev);
+ if (ret)
+ goto err;
mutex_unlock(&ep->com.mutex);
return 0;
+err:
+ mutex_unlock(&ep->com.mutex);
+ connect_reply_upcall(ep, -ENOMEM);
+ c4iw_ep_disconnect(ep, 0, GFP_KERNEL);
+ return 0;
}
static void close_complete_upcall(struct c4iw_ep *ep, int status)
@@ -1120,20 +1278,11 @@
PDBG("close complete delivered ep %p cm_id %p tid %u\n",
ep, ep->com.cm_id, ep->hwtid);
ep->com.cm_id->event_handler(ep->com.cm_id, &event);
- ep->com.cm_id->rem_ref(ep->com.cm_id);
- ep->com.cm_id = NULL;
+ deref_cm_id(&ep->com);
set_bit(CLOSE_UPCALL, &ep->com.history);
}
}
-static int abort_connection(struct c4iw_ep *ep, struct sk_buff *skb, gfp_t gfp)
-{
- PDBG("%s ep %p tid %u\n", __func__, ep, ep->hwtid);
- __state_set(&ep->com, ABORTING);
- set_bit(ABORT_CONN, &ep->com.history);
- return send_abort(ep, skb, gfp);
-}
-
static void peer_close_upcall(struct c4iw_ep *ep)
{
struct iw_cm_event event;
@@ -1161,8 +1310,7 @@
PDBG("abort delivered ep %p cm_id %p tid %u\n", ep,
ep->com.cm_id, ep->hwtid);
ep->com.cm_id->event_handler(ep->com.cm_id, &event);
- ep->com.cm_id->rem_ref(ep->com.cm_id);
- ep->com.cm_id = NULL;
+ deref_cm_id(&ep->com);
set_bit(ABORT_UPCALL, &ep->com.history);
}
}
@@ -1205,10 +1353,8 @@
set_bit(CONN_RPL_UPCALL, &ep->com.history);
ep->com.cm_id->event_handler(ep->com.cm_id, &event);
- if (status < 0) {
- ep->com.cm_id->rem_ref(ep->com.cm_id);
- ep->com.cm_id = NULL;
- }
+ if (status < 0)
+ deref_cm_id(&ep->com);
}
static int connect_request_upcall(struct c4iw_ep *ep)
@@ -1301,6 +1447,18 @@
#define RELAXED_IRD_NEGOTIATION 1
+/*
+ * process_mpa_reply - process streaming mode MPA reply
+ *
+ * Returns:
+ *
+ * 0 upon success indicating a connect request was delivered to the ULP
+ * or the mpa request is incomplete but valid so far.
+ *
+ * 1 if a failure requires the caller to close the connection.
+ *
+ * 2 if a failure requires the caller to abort the connection.
+ */
static int process_mpa_reply(struct c4iw_ep *ep, struct sk_buff *skb)
{
struct mpa_message *mpa;
@@ -1316,20 +1474,12 @@
PDBG("%s ep %p tid %u\n", __func__, ep, ep->hwtid);
/*
- * Stop mpa timer. If it expired, then
- * we ignore the MPA reply. process_timeout()
- * will abort the connection.
- */
- if (stop_ep_timer(ep))
- return 0;
-
- /*
* If we get more than the supported amount of private data
* then we must fail this connection.
*/
if (ep->mpa_pkt_len + skb->len > sizeof(ep->mpa_pkt)) {
err = -EINVAL;
- goto err;
+ goto err_stop_timer;
}
/*
@@ -1351,11 +1501,11 @@
printk(KERN_ERR MOD "%s MPA version mismatch. Local = %d,"
" Received = %d\n", __func__, mpa_rev, mpa->revision);
err = -EPROTO;
- goto err;
+ goto err_stop_timer;
}
if (memcmp(mpa->key, MPA_KEY_REP, sizeof(mpa->key))) {
err = -EPROTO;
- goto err;
+ goto err_stop_timer;
}
plen = ntohs(mpa->private_data_size);
@@ -1365,7 +1515,7 @@
*/
if (plen > MPA_MAX_PRIVATE_DATA) {
err = -EPROTO;
- goto err;
+ goto err_stop_timer;
}
/*
@@ -1373,7 +1523,7 @@
*/
if (ep->mpa_pkt_len > (sizeof(*mpa) + plen)) {
err = -EPROTO;
- goto err;
+ goto err_stop_timer;
}
ep->plen = (u8) plen;
@@ -1387,10 +1537,18 @@
if (mpa->flags & MPA_REJECT) {
err = -ECONNREFUSED;
- goto err;
+ goto err_stop_timer;
}
/*
+ * Stop mpa timer. If it expired, then
+ * we ignore the MPA reply. process_timeout()
+ * will abort the connection.
+ */
+ if (stop_ep_timer(ep))
+ return 0;
+
+ /*
* If we get here we have accumulated the entire mpa
* start reply message including private data. And
* the MPA header is valid.
@@ -1529,15 +1687,28 @@
goto out;
}
goto out;
+err_stop_timer:
+ stop_ep_timer(ep);
err:
- __state_set(&ep->com, ABORTING);
- send_abort(ep, skb, GFP_KERNEL);
+ disconnect = 2;
out:
connect_reply_upcall(ep, err);
return disconnect;
}
-static void process_mpa_request(struct c4iw_ep *ep, struct sk_buff *skb)
+/*
+ * process_mpa_request - process streaming mode MPA request
+ *
+ * Returns:
+ *
+ * 0 upon success indicating a connect request was delivered to the ULP
+ * or the mpa request is incomplete but valid so far.
+ *
+ * 1 if a failure requires the caller to close the connection.
+ *
+ * 2 if a failure requires the caller to abort the connection.
+ */
+static int process_mpa_request(struct c4iw_ep *ep, struct sk_buff *skb)
{
struct mpa_message *mpa;
struct mpa_v2_conn_params *mpa_v2_params;
@@ -1549,11 +1720,8 @@
* If we get more than the supported amount of private data
* then we must fail this connection.
*/
- if (ep->mpa_pkt_len + skb->len > sizeof(ep->mpa_pkt)) {
- (void)stop_ep_timer(ep);
- abort_connection(ep, skb, GFP_KERNEL);
- return;
- }
+ if (ep->mpa_pkt_len + skb->len > sizeof(ep->mpa_pkt))
+ goto err_stop_timer;
PDBG("%s enter (%s line %u)\n", __func__, __FILE__, __LINE__);
@@ -1569,7 +1737,7 @@
* We'll continue process when more data arrives.
*/
if (ep->mpa_pkt_len < sizeof(*mpa))
- return;
+ return 0;
PDBG("%s enter (%s line %u)\n", __func__, __FILE__, __LINE__);
mpa = (struct mpa_message *) ep->mpa_pkt;
@@ -1580,43 +1748,32 @@
if (mpa->revision > mpa_rev) {
printk(KERN_ERR MOD "%s MPA version mismatch. Local = %d,"
" Received = %d\n", __func__, mpa_rev, mpa->revision);
- (void)stop_ep_timer(ep);
- abort_connection(ep, skb, GFP_KERNEL);
- return;
+ goto err_stop_timer;
}
- if (memcmp(mpa->key, MPA_KEY_REQ, sizeof(mpa->key))) {
- (void)stop_ep_timer(ep);
- abort_connection(ep, skb, GFP_KERNEL);
- return;
- }
+ if (memcmp(mpa->key, MPA_KEY_REQ, sizeof(mpa->key)))
+ goto err_stop_timer;
plen = ntohs(mpa->private_data_size);
/*
* Fail if there's too much private data.
*/
- if (plen > MPA_MAX_PRIVATE_DATA) {
- (void)stop_ep_timer(ep);
- abort_connection(ep, skb, GFP_KERNEL);
- return;
- }
+ if (plen > MPA_MAX_PRIVATE_DATA)
+ goto err_stop_timer;
/*
* If plen does not account for pkt size
*/
- if (ep->mpa_pkt_len > (sizeof(*mpa) + plen)) {
- (void)stop_ep_timer(ep);
- abort_connection(ep, skb, GFP_KERNEL);
- return;
- }
+ if (ep->mpa_pkt_len > (sizeof(*mpa) + plen))
+ goto err_stop_timer;
ep->plen = (u8) plen;
/*
* If we don't have all the pdata yet, then bail.
*/
if (ep->mpa_pkt_len < (sizeof(*mpa) + plen))
- return;
+ return 0;
/*
* If we get here we have accumulated the entire mpa
@@ -1665,26 +1822,26 @@
ep->mpa_attr.xmit_marker_enabled, ep->mpa_attr.version,
ep->mpa_attr.p2p_type);
- /*
- * If the endpoint timer already expired, then we ignore
- * the start request. process_timeout() will abort
- * the connection.
- */
- if (!stop_ep_timer(ep)) {
- __state_set(&ep->com, MPA_REQ_RCVD);
+ __state_set(&ep->com, MPA_REQ_RCVD);
- /* drive upcall */
- mutex_lock_nested(&ep->parent_ep->com.mutex,
- SINGLE_DEPTH_NESTING);
- if (ep->parent_ep->com.state != DEAD) {
- if (connect_request_upcall(ep))
- abort_connection(ep, skb, GFP_KERNEL);
- } else {
- abort_connection(ep, skb, GFP_KERNEL);
- }
- mutex_unlock(&ep->parent_ep->com.mutex);
+ /* drive upcall */
+ mutex_lock_nested(&ep->parent_ep->com.mutex, SINGLE_DEPTH_NESTING);
+ if (ep->parent_ep->com.state != DEAD) {
+ if (connect_request_upcall(ep))
+ goto err_unlock_parent;
+ } else {
+ goto err_unlock_parent;
}
- return;
+ mutex_unlock(&ep->parent_ep->com.mutex);
+ return 0;
+
+err_unlock_parent:
+ mutex_unlock(&ep->parent_ep->com.mutex);
+ goto err_out;
+err_stop_timer:
+ (void)stop_ep_timer(ep);
+err_out:
+ return 2;
}
static int rx_data(struct c4iw_dev *dev, struct sk_buff *skb)
@@ -1693,11 +1850,10 @@
struct cpl_rx_data *hdr = cplhdr(skb);
unsigned int dlen = ntohs(hdr->len);
unsigned int tid = GET_TID(hdr);
- struct tid_info *t = dev->rdev.lldi.tids;
__u8 status = hdr->status;
int disconnect = 0;
- ep = lookup_tid(t, tid);
+ ep = get_ep_from_tid(dev, tid);
if (!ep)
return 0;
PDBG("%s ep %p tid %u dlen %u\n", __func__, ep, ep->hwtid, dlen);
@@ -1715,7 +1871,7 @@
break;
case MPA_REQ_WAIT:
ep->rcv_seq += dlen;
- process_mpa_request(ep, skb);
+ disconnect = process_mpa_request(ep, skb);
break;
case FPDU_MODE: {
struct c4iw_qp_attributes attrs;
@@ -1736,7 +1892,8 @@
}
mutex_unlock(&ep->com.mutex);
if (disconnect)
- c4iw_ep_disconnect(ep, 0, GFP_KERNEL);
+ c4iw_ep_disconnect(ep, disconnect == 2, GFP_KERNEL);
+ c4iw_put_ep(&ep->com);
return 0;
}
@@ -1746,9 +1903,8 @@
struct cpl_abort_rpl_rss *rpl = cplhdr(skb);
int release = 0;
unsigned int tid = GET_TID(rpl);
- struct tid_info *t = dev->rdev.lldi.tids;
- ep = lookup_tid(t, tid);
+ ep = get_ep_from_tid(dev, tid);
if (!ep) {
printk(KERN_WARNING MOD "Abort rpl to freed endpoint\n");
return 0;
@@ -1770,10 +1926,11 @@
if (release)
release_ep_resources(ep);
+ c4iw_put_ep(&ep->com);
return 0;
}
-static void send_fw_act_open_req(struct c4iw_ep *ep, unsigned int atid)
+static int send_fw_act_open_req(struct c4iw_ep *ep, unsigned int atid)
{
struct sk_buff *skb;
struct fw_ofld_connection_wr *req;
@@ -1843,7 +2000,7 @@
req->tcb.opt2 = cpu_to_be32((__force u32)req->tcb.opt2);
set_wr_txq(skb, CPL_PRIORITY_CONTROL, ep->ctrlq_idx);
set_bit(ACT_OFLD_CONN, &ep->com.history);
- c4iw_l2t_send(&ep->com.dev->rdev, skb, ep->l2t);
+ return c4iw_l2t_send(&ep->com.dev->rdev, skb, ep->l2t);
}
/*
@@ -1986,6 +2143,7 @@
PDBG("%s qp %p cm_id %p\n", __func__, ep->com.qp, ep->com.cm_id);
init_timer(&ep->timer);
+ c4iw_init_wr_wait(&ep->com.wr_wait);
/*
* Allocate an active TID to initiate a TCP connection.
@@ -2069,6 +2227,7 @@
struct sockaddr_in *ra;
struct sockaddr_in6 *la6;
struct sockaddr_in6 *ra6;
+ int ret = 0;
ep = lookup_atid(t, atid);
la = (struct sockaddr_in *)&ep->com.local_addr;
@@ -2104,9 +2263,10 @@
mutex_unlock(&dev->rdev.stats.lock);
if (ep->com.local_addr.ss_family == AF_INET &&
dev->rdev.lldi.enable_fw_ofld_conn) {
- send_fw_act_open_req(ep,
- TID_TID_G(AOPEN_ATID_G(
- ntohl(rpl->atid_status))));
+ ret = send_fw_act_open_req(ep, TID_TID_G(AOPEN_ATID_G(
+ ntohl(rpl->atid_status))));
+ if (ret)
+ goto fail;
return 0;
}
break;
@@ -2146,6 +2306,7 @@
break;
}
+fail:
connect_reply_upcall(ep, status2errno(status));
state_set(&ep->com, DEAD);
@@ -2170,9 +2331,8 @@
static int pass_open_rpl(struct c4iw_dev *dev, struct sk_buff *skb)
{
struct cpl_pass_open_rpl *rpl = cplhdr(skb);
- struct tid_info *t = dev->rdev.lldi.tids;
unsigned int stid = GET_TID(rpl);
- struct c4iw_listen_ep *ep = lookup_stid(t, stid);
+ struct c4iw_listen_ep *ep = get_ep_from_stid(dev, stid);
if (!ep) {
PDBG("%s stid %d lookup failure!\n", __func__, stid);
@@ -2181,7 +2341,7 @@
PDBG("%s ep %p status %d error %d\n", __func__, ep,
rpl->status, status2errno(rpl->status));
c4iw_wake_up(&ep->com.wr_wait, status2errno(rpl->status));
-
+ c4iw_put_ep(&ep->com);
out:
return 0;
}
@@ -2189,17 +2349,17 @@
static int close_listsrv_rpl(struct c4iw_dev *dev, struct sk_buff *skb)
{
struct cpl_close_listsvr_rpl *rpl = cplhdr(skb);
- struct tid_info *t = dev->rdev.lldi.tids;
unsigned int stid = GET_TID(rpl);
- struct c4iw_listen_ep *ep = lookup_stid(t, stid);
+ struct c4iw_listen_ep *ep = get_ep_from_stid(dev, stid);
PDBG("%s ep %p\n", __func__, ep);
c4iw_wake_up(&ep->com.wr_wait, status2errno(rpl->status));
+ c4iw_put_ep(&ep->com);
return 0;
}
-static void accept_cr(struct c4iw_ep *ep, struct sk_buff *skb,
- struct cpl_pass_accept_req *req)
+static int accept_cr(struct c4iw_ep *ep, struct sk_buff *skb,
+ struct cpl_pass_accept_req *req)
{
struct cpl_pass_accept_rpl *rpl;
unsigned int mtu_idx;
@@ -2287,10 +2447,9 @@
rpl->opt0 = cpu_to_be64(opt0);
rpl->opt2 = cpu_to_be32(opt2);
set_wr_txq(skb, CPL_PRIORITY_SETUP, ep->ctrlq_idx);
- t4_set_arp_err_handler(skb, NULL, arp_failure_discard);
- c4iw_l2t_send(&ep->com.dev->rdev, skb, ep->l2t);
+ t4_set_arp_err_handler(skb, ep, pass_accept_rpl_arp_failure);
- return;
+ return c4iw_l2t_send(&ep->com.dev->rdev, skb, ep->l2t);
}
static void reject_cr(struct c4iw_dev *dev, u32 hwtid, struct sk_buff *skb)
@@ -2355,7 +2514,7 @@
unsigned short hdrs;
u8 tos = PASS_OPEN_TOS_G(ntohl(req->tos_stid));
- parent_ep = lookup_stid(t, stid);
+ parent_ep = (struct c4iw_ep *)get_ep_from_stid(dev, stid);
if (!parent_ep) {
PDBG("%s connect request on invalid stid %d\n", __func__, stid);
goto reject;
@@ -2468,9 +2627,13 @@
init_timer(&child_ep->timer);
cxgb4_insert_tid(t, child_ep, hwtid);
- insert_handle(dev, &dev->hwtid_idr, child_ep, child_ep->hwtid);
- accept_cr(child_ep, skb, req);
- set_bit(PASS_ACCEPT_REQ, &child_ep->com.history);
+ insert_ep_tid(child_ep);
+ if (accept_cr(child_ep, skb, req)) {
+ c4iw_put_ep(&parent_ep->com);
+ release_ep_resources(child_ep);
+ } else {
+ set_bit(PASS_ACCEPT_REQ, &child_ep->com.history);
+ }
if (iptype == 6) {
sin6 = (struct sockaddr_in6 *)&child_ep->com.local_addr;
cxgb4_clip_get(child_ep->com.dev->rdev.lldi.ports[0],
@@ -2479,6 +2642,8 @@
goto out;
reject:
reject_cr(dev, hwtid, skb);
+ if (parent_ep)
+ c4iw_put_ep(&parent_ep->com);
out:
return 0;
}
@@ -2487,10 +2652,10 @@
{
struct c4iw_ep *ep;
struct cpl_pass_establish *req = cplhdr(skb);
- struct tid_info *t = dev->rdev.lldi.tids;
unsigned int tid = GET_TID(req);
+ int ret;
- ep = lookup_tid(t, tid);
+ ep = get_ep_from_tid(dev, tid);
PDBG("%s ep %p tid %u\n", __func__, ep, ep->hwtid);
ep->snd_seq = be32_to_cpu(req->snd_isn);
ep->rcv_seq = be32_to_cpu(req->rcv_isn);
@@ -2501,10 +2666,15 @@
set_emss(ep, ntohs(req->tcp_opt));
dst_confirm(ep->dst);
- state_set(&ep->com, MPA_REQ_WAIT);
+ mutex_lock(&ep->com.mutex);
+ ep->com.state = MPA_REQ_WAIT;
start_ep_timer(ep);
- send_flowc(ep, skb);
set_bit(PASS_ESTAB, &ep->com.history);
+ ret = send_flowc(ep, skb);
+ mutex_unlock(&ep->com.mutex);
+ if (ret)
+ c4iw_ep_disconnect(ep, 1, GFP_KERNEL);
+ c4iw_put_ep(&ep->com);
return 0;
}
@@ -2516,11 +2686,13 @@
struct c4iw_qp_attributes attrs;
int disconnect = 1;
int release = 0;
- struct tid_info *t = dev->rdev.lldi.tids;
unsigned int tid = GET_TID(hdr);
int ret;
- ep = lookup_tid(t, tid);
+ ep = get_ep_from_tid(dev, tid);
+ if (!ep)
+ return 0;
+
PDBG("%s ep %p tid %u\n", __func__, ep, ep->hwtid);
dst_confirm(ep->dst);
@@ -2592,6 +2764,7 @@
c4iw_ep_disconnect(ep, 0, GFP_KERNEL);
if (release)
release_ep_resources(ep);
+ c4iw_put_ep(&ep->com);
return 0;
}
@@ -2604,10 +2777,12 @@
struct c4iw_qp_attributes attrs;
int ret;
int release = 0;
- struct tid_info *t = dev->rdev.lldi.tids;
unsigned int tid = GET_TID(req);
- ep = lookup_tid(t, tid);
+ ep = get_ep_from_tid(dev, tid);
+ if (!ep)
+ return 0;
+
if (is_neg_adv(req->status)) {
PDBG("%s Negative advice on abort- tid %u status %d (%s)\n",
__func__, ep->hwtid, req->status,
@@ -2616,7 +2791,7 @@
mutex_lock(&dev->rdev.stats.lock);
dev->rdev.stats.neg_adv++;
mutex_unlock(&dev->rdev.stats.lock);
- return 0;
+ goto deref_ep;
}
PDBG("%s ep %p tid %u state %u\n", __func__, ep, ep->hwtid,
ep->com.state);
@@ -2633,6 +2808,7 @@
mutex_lock(&ep->com.mutex);
switch (ep->com.state) {
case CONNECTING:
+ c4iw_put_ep(&ep->parent_ep->com);
break;
case MPA_REQ_WAIT:
(void)stop_ep_timer(ep);
@@ -2681,7 +2857,7 @@
case DEAD:
PDBG("%s PEER_ABORT IN DEAD STATE!!!!\n", __func__);
mutex_unlock(&ep->com.mutex);
- return 0;
+ goto deref_ep;
default:
BUG_ON(1);
break;
@@ -2728,6 +2904,10 @@
c4iw_reconnect(ep);
}
+deref_ep:
+ c4iw_put_ep(&ep->com);
+ /* Dereferencing ep, referenced in peer_abort_intr() */
+ c4iw_put_ep(&ep->com);
return 0;
}
@@ -2737,16 +2917,18 @@
struct c4iw_qp_attributes attrs;
struct cpl_close_con_rpl *rpl = cplhdr(skb);
int release = 0;
- struct tid_info *t = dev->rdev.lldi.tids;
unsigned int tid = GET_TID(rpl);
- ep = lookup_tid(t, tid);
+ ep = get_ep_from_tid(dev, tid);
+ if (!ep)
+ return 0;
PDBG("%s ep %p tid %u\n", __func__, ep, ep->hwtid);
BUG_ON(!ep);
/* The cm_id may be null if we failed to connect */
mutex_lock(&ep->com.mutex);
+ set_bit(CLOSE_CON_RPL, &ep->com.history);
switch (ep->com.state) {
case CLOSING:
__state_set(&ep->com, MORIBUND);
@@ -2774,18 +2956,18 @@
mutex_unlock(&ep->com.mutex);
if (release)
release_ep_resources(ep);
+ c4iw_put_ep(&ep->com);
return 0;
}
static int terminate(struct c4iw_dev *dev, struct sk_buff *skb)
{
struct cpl_rdma_terminate *rpl = cplhdr(skb);
- struct tid_info *t = dev->rdev.lldi.tids;
unsigned int tid = GET_TID(rpl);
struct c4iw_ep *ep;
struct c4iw_qp_attributes attrs;
- ep = lookup_tid(t, tid);
+ ep = get_ep_from_tid(dev, tid);
BUG_ON(!ep);
if (ep && ep->com.qp) {
@@ -2796,6 +2978,7 @@
C4IW_QP_ATTR_NEXT_STATE, &attrs, 1);
} else
printk(KERN_WARNING MOD "TERM received tid %u no ep/qp\n", tid);
+ c4iw_put_ep(&ep->com);
return 0;
}
@@ -2811,15 +2994,16 @@
struct cpl_fw4_ack *hdr = cplhdr(skb);
u8 credits = hdr->credits;
unsigned int tid = GET_TID(hdr);
- struct tid_info *t = dev->rdev.lldi.tids;
- ep = lookup_tid(t, tid);
+ ep = get_ep_from_tid(dev, tid);
+ if (!ep)
+ return 0;
PDBG("%s ep %p tid %u credits %u\n", __func__, ep, ep->hwtid, credits);
if (credits == 0) {
PDBG("%s 0 credit ack ep %p tid %u state %u\n",
__func__, ep, ep->hwtid, state_read(&ep->com));
- return 0;
+ goto out;
}
dst_confirm(ep->dst);
@@ -2829,7 +3013,13 @@
state_read(&ep->com), ep->mpa_attr.initiator ? 1 : 0);
kfree_skb(ep->mpa_skb);
ep->mpa_skb = NULL;
+ mutex_lock(&ep->com.mutex);
+ if (test_bit(STOP_MPA_TIMER, &ep->com.flags))
+ stop_ep_timer(ep);
+ mutex_unlock(&ep->com.mutex);
}
+out:
+ c4iw_put_ep(&ep->com);
return 0;
}
@@ -2841,22 +3031,23 @@
PDBG("%s ep %p tid %u\n", __func__, ep, ep->hwtid);
mutex_lock(&ep->com.mutex);
- if (ep->com.state == DEAD) {
+ if (ep->com.state != MPA_REQ_RCVD) {
mutex_unlock(&ep->com.mutex);
c4iw_put_ep(&ep->com);
return -ECONNRESET;
}
set_bit(ULP_REJECT, &ep->com.history);
- BUG_ON(ep->com.state != MPA_REQ_RCVD);
if (mpa_rev == 0)
- abort_connection(ep, NULL, GFP_KERNEL);
+ disconnect = 2;
else {
err = send_mpa_reject(ep, pdata, pdata_len);
disconnect = 1;
}
mutex_unlock(&ep->com.mutex);
- if (disconnect)
- err = c4iw_ep_disconnect(ep, 0, GFP_KERNEL);
+ if (disconnect) {
+ stop_ep_timer(ep);
+ err = c4iw_ep_disconnect(ep, disconnect == 2, GFP_KERNEL);
+ }
c4iw_put_ep(&ep->com);
return 0;
}
@@ -2869,24 +3060,23 @@
struct c4iw_ep *ep = to_ep(cm_id);
struct c4iw_dev *h = to_c4iw_dev(cm_id->device);
struct c4iw_qp *qp = get_qhp(h, conn_param->qpn);
+ int abort = 0;
PDBG("%s ep %p tid %u\n", __func__, ep, ep->hwtid);
mutex_lock(&ep->com.mutex);
- if (ep->com.state == DEAD) {
+ if (ep->com.state != MPA_REQ_RCVD) {
err = -ECONNRESET;
- goto err;
+ goto err_out;
}
- BUG_ON(ep->com.state != MPA_REQ_RCVD);
BUG_ON(!qp);
set_bit(ULP_ACCEPT, &ep->com.history);
if ((conn_param->ord > cur_max_read_depth(ep->com.dev)) ||
(conn_param->ird > cur_max_read_depth(ep->com.dev))) {
- abort_connection(ep, NULL, GFP_KERNEL);
err = -EINVAL;
- goto err;
+ goto err_abort;
}
if (ep->mpa_attr.version == 2 && ep->mpa_attr.enhanced_rdma_conn) {
@@ -2898,9 +3088,8 @@
ep->ord = conn_param->ord;
send_mpa_reject(ep, conn_param->private_data,
conn_param->private_data_len);
- abort_connection(ep, NULL, GFP_KERNEL);
err = -ENOMEM;
- goto err;
+ goto err_abort;
}
}
if (conn_param->ird < ep->ord) {
@@ -2908,9 +3097,8 @@
ep->ord <= h->rdev.lldi.max_ordird_qp) {
conn_param->ird = ep->ord;
} else {
- abort_connection(ep, NULL, GFP_KERNEL);
err = -ENOMEM;
- goto err;
+ goto err_abort;
}
}
}
@@ -2929,8 +3117,8 @@
PDBG("%s %d ird %d ord %d\n", __func__, __LINE__, ep->ird, ep->ord);
- cm_id->add_ref(cm_id);
ep->com.cm_id = cm_id;
+ ref_cm_id(&ep->com);
ep->com.qp = qp;
ref_qp(ep);
@@ -2951,23 +3139,27 @@
err = c4iw_modify_qp(ep->com.qp->rhp,
ep->com.qp, mask, &attrs, 1);
if (err)
- goto err1;
+ goto err_deref_cm_id;
+
+ set_bit(STOP_MPA_TIMER, &ep->com.flags);
err = send_mpa_reply(ep, conn_param->private_data,
conn_param->private_data_len);
if (err)
- goto err1;
+ goto err_deref_cm_id;
__state_set(&ep->com, FPDU_MODE);
established_upcall(ep);
mutex_unlock(&ep->com.mutex);
c4iw_put_ep(&ep->com);
return 0;
-err1:
- ep->com.cm_id = NULL;
- abort_connection(ep, NULL, GFP_KERNEL);
- cm_id->rem_ref(cm_id);
-err:
+err_deref_cm_id:
+ deref_cm_id(&ep->com);
+err_abort:
+ abort = 1;
+err_out:
mutex_unlock(&ep->com.mutex);
+ if (abort)
+ c4iw_ep_disconnect(ep, 1, GFP_KERNEL);
c4iw_put_ep(&ep->com);
return err;
}
@@ -3067,9 +3259,9 @@
if (peer2peer && ep->ord == 0)
ep->ord = 1;
- cm_id->add_ref(cm_id);
- ep->com.dev = dev;
ep->com.cm_id = cm_id;
+ ref_cm_id(&ep->com);
+ ep->com.dev = dev;
ep->com.qp = get_qhp(dev, conn_param->qpn);
if (!ep->com.qp) {
PDBG("%s qpn 0x%x not found!\n", __func__, conn_param->qpn);
@@ -3108,7 +3300,7 @@
/*
* Handle loopback requests to INADDR_ANY.
*/
- if ((__force int)raddr->sin_addr.s_addr == INADDR_ANY) {
+ if (raddr->sin_addr.s_addr == htonl(INADDR_ANY)) {
err = pick_local_ipaddrs(dev, cm_id);
if (err)
goto fail1;
@@ -3176,7 +3368,7 @@
remove_handle(ep->com.dev, &ep->com.dev->atid_idr, ep->atid);
cxgb4_free_atid(ep->com.dev->rdev.lldi.tids, ep->atid);
fail1:
- cm_id->rem_ref(cm_id);
+ deref_cm_id(&ep->com);
c4iw_put_ep(&ep->com);
out:
return err;
@@ -3270,8 +3462,8 @@
goto fail1;
}
PDBG("%s ep %p\n", __func__, ep);
- cm_id->add_ref(cm_id);
ep->com.cm_id = cm_id;
+ ref_cm_id(&ep->com);
ep->com.dev = dev;
ep->backlog = backlog;
memcpy(&ep->com.local_addr, &cm_id->m_local_addr,
@@ -3311,7 +3503,7 @@
cxgb4_free_stid(ep->com.dev->rdev.lldi.tids, ep->stid,
ep->com.local_addr.ss_family);
fail2:
- cm_id->rem_ref(cm_id);
+ deref_cm_id(&ep->com);
c4iw_put_ep(&ep->com);
fail1:
out:
@@ -3350,7 +3542,7 @@
cxgb4_free_stid(ep->com.dev->rdev.lldi.tids, ep->stid,
ep->com.local_addr.ss_family);
done:
- cm_id->rem_ref(cm_id);
+ deref_cm_id(&ep->com);
c4iw_put_ep(&ep->com);
return err;
}
@@ -3367,6 +3559,12 @@
PDBG("%s ep %p state %s, abrupt %d\n", __func__, ep,
states[ep->com.state], abrupt);
+ /*
+ * Ref the ep here in case we have fatal errors causing the
+ * ep to be released and freed.
+ */
+ c4iw_get_ep(&ep->com);
+
rdev = &ep->com.dev->rdev;
if (c4iw_fatal_error(rdev)) {
fatal = 1;
@@ -3418,10 +3616,30 @@
set_bit(EP_DISC_CLOSE, &ep->com.history);
ret = send_halfclose(ep, gfp);
}
- if (ret)
+ if (ret) {
+ set_bit(EP_DISC_FAIL, &ep->com.history);
+ if (!abrupt) {
+ stop_ep_timer(ep);
+ close_complete_upcall(ep, -EIO);
+ }
+ if (ep->com.qp) {
+ struct c4iw_qp_attributes attrs;
+
+ attrs.next_state = C4IW_QP_STATE_ERROR;
+ ret = c4iw_modify_qp(ep->com.qp->rhp,
+ ep->com.qp,
+ C4IW_QP_ATTR_NEXT_STATE,
+ &attrs, 1);
+ if (ret)
+ pr_err(MOD
+ "%s - qp <- error failed!\n",
+ __func__);
+ }
fatal = 1;
+ }
}
mutex_unlock(&ep->com.mutex);
+ c4iw_put_ep(&ep->com);
if (fatal)
release_ep_resources(ep);
return ret;
@@ -3676,7 +3894,7 @@
struct cpl_pass_accept_req *req = (void *)(rss + 1);
struct l2t_entry *e;
struct dst_entry *dst;
- struct c4iw_ep *lep;
+ struct c4iw_ep *lep = NULL;
u16 window;
struct port_info *pi;
struct net_device *pdev;
@@ -3701,7 +3919,7 @@
*/
stid = (__force int) cpu_to_be32((__force u32) rss->hash_val);
- lep = (struct c4iw_ep *)lookup_stid(dev->rdev.lldi.tids, stid);
+ lep = (struct c4iw_ep *)get_ep_from_stid(dev, stid);
if (!lep) {
PDBG("%s connect request on invalid stid %d\n", __func__, stid);
goto reject;
@@ -3802,6 +4020,8 @@
free_dst:
dst_release(dst);
reject:
+ if (lep)
+ c4iw_put_ep(&lep->com);
return 0;
}
@@ -3809,7 +4029,7 @@
* These are the real handlers that are called from a
* work queue.
*/
-static c4iw_handler_func work_handlers[NUM_CPL_CMDS] = {
+static c4iw_handler_func work_handlers[NUM_CPL_CMDS + NUM_FAKE_CPLS] = {
[CPL_ACT_ESTABLISH] = act_establish,
[CPL_ACT_OPEN_RPL] = act_open_rpl,
[CPL_RX_DATA] = rx_data,
@@ -3825,7 +4045,9 @@
[CPL_RDMA_TERMINATE] = terminate,
[CPL_FW4_ACK] = fw4_ack,
[CPL_FW6_MSG] = deferred_fw6_msg,
- [CPL_RX_PKT] = rx_pkt
+ [CPL_RX_PKT] = rx_pkt,
+ [FAKE_CPL_PUT_EP_SAFE] = _put_ep_safe,
+ [FAKE_CPL_PASS_PUT_EP_SAFE] = _put_pass_ep_safe
};
static void process_timeout(struct c4iw_ep *ep)
@@ -3839,11 +4061,12 @@
set_bit(TIMEDOUT, &ep->com.history);
switch (ep->com.state) {
case MPA_REQ_SENT:
- __state_set(&ep->com, ABORTING);
connect_reply_upcall(ep, -ETIMEDOUT);
break;
case MPA_REQ_WAIT:
- __state_set(&ep->com, ABORTING);
+ case MPA_REQ_RCVD:
+ case MPA_REP_SENT:
+ case FPDU_MODE:
break;
case CLOSING:
case MORIBUND:
@@ -3853,7 +4076,6 @@
ep->com.qp, C4IW_QP_ATTR_NEXT_STATE,
&attrs, 1);
}
- __state_set(&ep->com, ABORTING);
close_complete_upcall(ep, -ETIMEDOUT);
break;
case ABORTING:
@@ -3871,9 +4093,9 @@
__func__, ep, ep->hwtid, ep->com.state);
abort = 0;
}
- if (abort)
- abort_connection(ep, NULL, GFP_KERNEL);
mutex_unlock(&ep->com.mutex);
+ if (abort)
+ c4iw_ep_disconnect(ep, 1, GFP_KERNEL);
c4iw_put_ep(&ep->com);
}
@@ -4006,10 +4228,10 @@
{
struct cpl_abort_req_rss *req = cplhdr(skb);
struct c4iw_ep *ep;
- struct tid_info *t = dev->rdev.lldi.tids;
unsigned int tid = GET_TID(req);
- ep = lookup_tid(t, tid);
+ ep = get_ep_from_tid(dev, tid);
+ /* This EP will be dereferenced in peer_abort() */
if (!ep) {
printk(KERN_WARNING MOD
"Abort on non-existent endpoint, tid %d\n", tid);
@@ -4020,24 +4242,13 @@
PDBG("%s Negative advice on abort- tid %u status %d (%s)\n",
__func__, ep->hwtid, req->status,
neg_adv_str(req->status));
- ep->stats.abort_neg_adv++;
- dev->rdev.stats.neg_adv++;
- kfree_skb(skb);
- return 0;
+ goto out;
}
PDBG("%s ep %p tid %u state %u\n", __func__, ep, ep->hwtid,
ep->com.state);
- /*
- * Wake up any threads in rdma_init() or rdma_fini().
- * However, if we are on MPAv2 and want to retry with MPAv1
- * then, don't wake up yet.
- */
- if (mpa_rev == 2 && !ep->tried_with_mpa_v1) {
- if (ep->com.state != MPA_REQ_SENT)
- c4iw_wake_up(&ep->com.wr_wait, -ECONNRESET);
- } else
- c4iw_wake_up(&ep->com.wr_wait, -ECONNRESET);
+ c4iw_wake_up(&ep->com.wr_wait, -ECONNRESET);
+out:
sched(dev, skb);
return 0;
}
diff --git a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
index df43f87..f6f34a7 100644
--- a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
+++ b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
@@ -755,6 +755,7 @@
CLOSE_SENT = 3,
TIMEOUT = 4,
QP_REFERENCED = 5,
+ STOP_MPA_TIMER = 7,
};
enum c4iw_ep_history {
@@ -779,7 +780,13 @@
EP_DISC_ABORT = 18,
CONN_RPL_UPCALL = 19,
ACT_RETRY_NOMEM = 20,
- ACT_RETRY_INUSE = 21
+ ACT_RETRY_INUSE = 21,
+ CLOSE_CON_RPL = 22,
+ EP_DISC_FAIL = 24,
+ QP_REFED = 25,
+ QP_DEREFED = 26,
+ CM_ID_REFED = 27,
+ CM_ID_DEREFED = 28,
};
struct c4iw_ep_common {
@@ -917,9 +924,8 @@
struct ib_mr *c4iw_alloc_mr(struct ib_pd *pd,
enum ib_mr_type mr_type,
u32 max_num_sg);
-int c4iw_map_mr_sg(struct ib_mr *ibmr,
- struct scatterlist *sg,
- int sg_nents);
+int c4iw_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, int sg_nents,
+ unsigned int *sg_offset);
int c4iw_dealloc_mw(struct ib_mw *mw);
struct ib_mw *c4iw_alloc_mw(struct ib_pd *pd, enum ib_mw_type type,
struct ib_udata *udata);
diff --git a/drivers/infiniband/hw/cxgb4/mem.c b/drivers/infiniband/hw/cxgb4/mem.c
index 008be07..55d0651 100644
--- a/drivers/infiniband/hw/cxgb4/mem.c
+++ b/drivers/infiniband/hw/cxgb4/mem.c
@@ -86,8 +86,9 @@
(wait ? FW_WR_COMPL_F : 0));
req->wr.wr_lo = wait ? (__force __be64)(unsigned long) &wr_wait : 0L;
req->wr.wr_mid = cpu_to_be32(FW_WR_LEN16_V(DIV_ROUND_UP(wr_len, 16)));
- req->cmd = cpu_to_be32(ULPTX_CMD_V(ULP_TX_MEM_WRITE));
- req->cmd |= cpu_to_be32(T5_ULP_MEMIO_ORDER_V(1));
+ req->cmd = cpu_to_be32(ULPTX_CMD_V(ULP_TX_MEM_WRITE) |
+ T5_ULP_MEMIO_ORDER_V(1) |
+ T5_ULP_MEMIO_FID_V(rdev->lldi.rxq_ids[0]));
req->dlen = cpu_to_be32(ULP_MEMIO_DATA_LEN_V(len>>5));
req->len16 = cpu_to_be32(DIV_ROUND_UP(wr_len-sizeof(req->wr), 16));
req->lock_addr = cpu_to_be32(ULP_MEMIO_ADDR_V(addr));
@@ -690,15 +691,14 @@
return 0;
}
-int c4iw_map_mr_sg(struct ib_mr *ibmr,
- struct scatterlist *sg,
- int sg_nents)
+int c4iw_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, int sg_nents,
+ unsigned int *sg_offset)
{
struct c4iw_mr *mhp = to_c4iw_mr(ibmr);
mhp->mpl_len = 0;
- return ib_sg_to_pages(ibmr, sg, sg_nents, c4iw_set_page);
+ return ib_sg_to_pages(ibmr, sg, sg_nents, sg_offset, c4iw_set_page);
}
int c4iw_dereg_mr(struct ib_mr *ib_mr)
diff --git a/drivers/infiniband/hw/i40iw/i40iw.h b/drivers/infiniband/hw/i40iw/i40iw.h
index 8197676..8b95320 100644
--- a/drivers/infiniband/hw/i40iw/i40iw.h
+++ b/drivers/infiniband/hw/i40iw/i40iw.h
@@ -50,8 +50,6 @@
#include <rdma/ib_pack.h>
#include <rdma/rdma_cm.h>
#include <rdma/iw_cm.h>
-#include <rdma/iw_portmap.h>
-#include <rdma/rdma_netlink.h>
#include <crypto/hash.h>
#include "i40iw_status.h"
@@ -254,6 +252,7 @@
u32 arp_table_size;
u32 next_arp_index;
spinlock_t resource_lock; /* hw resource access */
+ spinlock_t qptable_lock;
u32 vendor_id;
u32 vendor_part_id;
u32 of_device_registered;
@@ -392,7 +391,7 @@
void i40iw_manage_arp_cache(struct i40iw_device *iwdev,
unsigned char *mac_addr,
- __be32 *ip_addr,
+ u32 *ip_addr,
bool ipv4,
u32 action);
@@ -550,7 +549,7 @@
struct i40iw_qp_flush_info *info,
bool wait);
-void i40iw_copy_ip_ntohl(u32 *dst, u32 *src);
+void i40iw_copy_ip_ntohl(u32 *dst, __be32 *src);
struct ib_mr *i40iw_reg_phys_mr(struct ib_pd *ib_pd,
u64 addr,
u64 size,
diff --git a/drivers/infiniband/hw/i40iw/i40iw_cm.c b/drivers/infiniband/hw/i40iw/i40iw_cm.c
index 38f917a..d2fa725 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_cm.c
+++ b/drivers/infiniband/hw/i40iw/i40iw_cm.c
@@ -771,6 +771,7 @@
{
struct ietf_mpa_v2 *mpa_frame = (struct ietf_mpa_v2 *)start_addr;
struct ietf_rtr_msg *rtr_msg = &mpa_frame->rtr_msg;
+ u16 ctrl_ird, ctrl_ord;
/* initialize the upper 5 bytes of the frame */
i40iw_build_mpa_v1(cm_node, start_addr, mpa_key);
@@ -779,38 +780,38 @@
/* initialize RTR msg */
if (cm_node->mpav2_ird_ord == IETF_NO_IRD_ORD) {
- rtr_msg->ctrl_ird = IETF_NO_IRD_ORD;
- rtr_msg->ctrl_ord = IETF_NO_IRD_ORD;
+ ctrl_ird = IETF_NO_IRD_ORD;
+ ctrl_ord = IETF_NO_IRD_ORD;
} else {
- rtr_msg->ctrl_ird = (cm_node->ird_size > IETF_NO_IRD_ORD) ?
+ ctrl_ird = (cm_node->ird_size > IETF_NO_IRD_ORD) ?
IETF_NO_IRD_ORD : cm_node->ird_size;
- rtr_msg->ctrl_ord = (cm_node->ord_size > IETF_NO_IRD_ORD) ?
+ ctrl_ord = (cm_node->ord_size > IETF_NO_IRD_ORD) ?
IETF_NO_IRD_ORD : cm_node->ord_size;
}
- rtr_msg->ctrl_ird |= IETF_PEER_TO_PEER;
- rtr_msg->ctrl_ird |= IETF_FLPDU_ZERO_LEN;
+ ctrl_ird |= IETF_PEER_TO_PEER;
+ ctrl_ird |= IETF_FLPDU_ZERO_LEN;
switch (mpa_key) {
case MPA_KEY_REQUEST:
- rtr_msg->ctrl_ord |= IETF_RDMA0_WRITE;
- rtr_msg->ctrl_ord |= IETF_RDMA0_READ;
+ ctrl_ord |= IETF_RDMA0_WRITE;
+ ctrl_ord |= IETF_RDMA0_READ;
break;
case MPA_KEY_REPLY:
switch (cm_node->send_rdma0_op) {
case SEND_RDMA_WRITE_ZERO:
- rtr_msg->ctrl_ord |= IETF_RDMA0_WRITE;
+ ctrl_ord |= IETF_RDMA0_WRITE;
break;
case SEND_RDMA_READ_ZERO:
- rtr_msg->ctrl_ord |= IETF_RDMA0_READ;
+ ctrl_ord |= IETF_RDMA0_READ;
break;
}
break;
default:
break;
}
- rtr_msg->ctrl_ird = htons(rtr_msg->ctrl_ird);
- rtr_msg->ctrl_ord = htons(rtr_msg->ctrl_ord);
+ rtr_msg->ctrl_ird = htons(ctrl_ird);
+ rtr_msg->ctrl_ord = htons(ctrl_ord);
}
/**
@@ -2107,7 +2108,7 @@
struct in6_addr raddr6;
i40iw_copy_ip_htonl(raddr6.in6_u.u6_addr32, rem_addr);
- return (!memcmp(loc_addr, rem_addr, 16) || ipv6_addr_loopback(&raddr6));
+ return !memcmp(loc_addr, rem_addr, 16) || ipv6_addr_loopback(&raddr6);
}
/**
@@ -2160,7 +2161,7 @@
cm_node->tcp_cntxt.rcv_wnd =
I40IW_CM_DEFAULT_RCV_WND_SCALED >> I40IW_CM_DEFAULT_RCV_WND_SCALE;
ts = current_kernel_time();
- cm_node->tcp_cntxt.loc_seq_num = htonl(ts.tv_nsec);
+ cm_node->tcp_cntxt.loc_seq_num = ts.tv_nsec;
cm_node->tcp_cntxt.mss = iwdev->mss;
cm_node->iwdev = iwdev;
@@ -2234,7 +2235,7 @@
if (cm_node->listener) {
i40iw_dec_refcnt_listen(cm_core, cm_node->listener, 0, true);
} else {
- if (!i40iw_listen_port_in_use(cm_core, htons(cm_node->loc_port)) &&
+ if (!i40iw_listen_port_in_use(cm_core, cm_node->loc_port) &&
cm_node->apbvt_set && cm_node->iwdev) {
i40iw_manage_apbvt(cm_node->iwdev,
cm_node->loc_port,
@@ -2852,7 +2853,6 @@
void *private_data,
struct i40iw_cm_info *cm_info)
{
- int ret;
struct i40iw_cm_node *cm_node;
struct i40iw_cm_listener *loopback_remotelistener;
struct i40iw_cm_node *loopback_remotenode;
@@ -2922,30 +2922,6 @@
memcpy(cm_node->pdata_buf, private_data, private_data_len);
cm_node->state = I40IW_CM_STATE_SYN_SENT;
- ret = i40iw_send_syn(cm_node, 0);
-
- if (ret) {
- if (cm_node->ipv4)
- i40iw_debug(cm_node->dev,
- I40IW_DEBUG_CM,
- "Api - connect() FAILED: dest addr=%pI4",
- cm_node->rem_addr);
- else
- i40iw_debug(cm_node->dev, I40IW_DEBUG_CM,
- "Api - connect() FAILED: dest addr=%pI6",
- cm_node->rem_addr);
- i40iw_rem_ref_cm_node(cm_node);
- cm_node = NULL;
- }
-
- if (cm_node)
- i40iw_debug(cm_node->dev,
- I40IW_DEBUG_CM,
- "Api - connect(): port=0x%04x, cm_node=%p, cm_id = %p.\n",
- cm_node->rem_port,
- cm_node,
- cm_node->cm_id);
-
return cm_node;
}
@@ -3266,11 +3242,13 @@
tcp_info->dest_ip_addr3 = cpu_to_le32(cm_node->rem_addr[0]);
tcp_info->local_ipaddr3 = cpu_to_le32(cm_node->loc_addr[0]);
- tcp_info->arp_idx = cpu_to_le32(i40iw_arp_table(iwqp->iwdev,
- &tcp_info->dest_ip_addr3,
- true,
- NULL,
- I40IW_ARP_RESOLVE));
+ tcp_info->arp_idx =
+ cpu_to_le16((u16)i40iw_arp_table(
+ iwqp->iwdev,
+ &tcp_info->dest_ip_addr3,
+ true,
+ NULL,
+ I40IW_ARP_RESOLVE));
} else {
tcp_info->src_port = cpu_to_le16(cm_node->loc_port);
tcp_info->dst_port = cpu_to_le16(cm_node->rem_port);
@@ -3282,12 +3260,13 @@
tcp_info->local_ipaddr1 = cpu_to_le32(cm_node->loc_addr[1]);
tcp_info->local_ipaddr2 = cpu_to_le32(cm_node->loc_addr[2]);
tcp_info->local_ipaddr3 = cpu_to_le32(cm_node->loc_addr[3]);
- tcp_info->arp_idx = cpu_to_le32(i40iw_arp_table(
- iwqp->iwdev,
- &tcp_info->dest_ip_addr0,
- false,
- NULL,
- I40IW_ARP_RESOLVE));
+ tcp_info->arp_idx =
+ cpu_to_le16((u16)i40iw_arp_table(
+ iwqp->iwdev,
+ &tcp_info->dest_ip_addr0,
+ false,
+ NULL,
+ I40IW_ARP_RESOLVE));
}
}
@@ -3564,7 +3543,6 @@
struct i40iw_cm_node *cm_node;
struct ib_qp_attr attr;
int passive_state;
- struct i40iw_ib_device *iwibdev;
struct ib_mr *ibmr;
struct i40iw_pd *iwpd;
u16 buf_len = 0;
@@ -3627,7 +3605,6 @@
!i40iw_ipv4_is_loopback(cm_node->loc_addr[0], cm_node->rem_addr[0])) ||
(!cm_node->ipv4 &&
!i40iw_ipv6_is_loopback(cm_node->loc_addr, cm_node->rem_addr))) {
- iwibdev = iwdev->iwibdev;
iwpd = iwqp->iwpd;
tagged_offset = (uintptr_t)iwqp->ietf_mem.va;
ibmr = i40iw_reg_phys_mr(&iwpd->ibpd,
@@ -3752,6 +3729,7 @@
struct sockaddr_in *raddr;
struct sockaddr_in6 *laddr6;
struct sockaddr_in6 *raddr6;
+ bool qhash_set = false;
int apbvt_set = 0;
enum i40iw_status_code status;
@@ -3810,6 +3788,7 @@
true);
if (status)
return -EINVAL;
+ qhash_set = true;
}
status = i40iw_manage_apbvt(iwdev, cm_info.loc_port, I40IW_MANAGE_APBVT_ADD);
if (status) {
@@ -3828,23 +3807,8 @@
conn_param->private_data_len,
(void *)conn_param->private_data,
&cm_info);
- if (!cm_node) {
- i40iw_manage_qhash(iwdev,
- &cm_info,
- I40IW_QHASH_TYPE_TCP_ESTABLISHED,
- I40IW_QHASH_MANAGE_TYPE_DELETE,
- NULL,
- false);
-
- if (apbvt_set && !i40iw_listen_port_in_use(&iwdev->cm_core,
- cm_info.loc_port))
- i40iw_manage_apbvt(iwdev,
- cm_info.loc_port,
- I40IW_MANAGE_APBVT_DEL);
- cm_id->rem_ref(cm_id);
- iwdev->cm_core.stats_connect_errs++;
- return -ENOMEM;
- }
+ if (!cm_node)
+ goto err;
i40iw_record_ird_ord(cm_node, (u16)conn_param->ird, (u16)conn_param->ord);
if (cm_node->send_rdma0_op == SEND_RDMA_READ_ZERO &&
@@ -3852,12 +3816,54 @@
cm_node->ord_size = 1;
cm_node->apbvt_set = apbvt_set;
- cm_node->qhash_set = true;
+ cm_node->qhash_set = qhash_set;
iwqp->cm_node = cm_node;
cm_node->iwqp = iwqp;
iwqp->cm_id = cm_id;
i40iw_add_ref(&iwqp->ibqp);
+
+ if (cm_node->state == I40IW_CM_STATE_SYN_SENT) {
+ if (i40iw_send_syn(cm_node, 0)) {
+ i40iw_rem_ref_cm_node(cm_node);
+ goto err;
+ }
+ }
+
+ i40iw_debug(cm_node->dev,
+ I40IW_DEBUG_CM,
+ "Api - connect(): port=0x%04x, cm_node=%p, cm_id = %p.\n",
+ cm_node->rem_port,
+ cm_node,
+ cm_node->cm_id);
return 0;
+
+err:
+ if (cm_node) {
+ if (cm_node->ipv4)
+ i40iw_debug(cm_node->dev,
+ I40IW_DEBUG_CM,
+ "Api - connect() FAILED: dest addr=%pI4",
+ cm_node->rem_addr);
+ else
+ i40iw_debug(cm_node->dev, I40IW_DEBUG_CM,
+ "Api - connect() FAILED: dest addr=%pI6",
+ cm_node->rem_addr);
+ }
+ i40iw_manage_qhash(iwdev,
+ &cm_info,
+ I40IW_QHASH_TYPE_TCP_ESTABLISHED,
+ I40IW_QHASH_MANAGE_TYPE_DELETE,
+ NULL,
+ false);
+
+ if (apbvt_set && !i40iw_listen_port_in_use(&iwdev->cm_core,
+ cm_info.loc_port))
+ i40iw_manage_apbvt(iwdev,
+ cm_info.loc_port,
+ I40IW_MANAGE_APBVT_DEL);
+ cm_id->rem_ref(cm_id);
+ iwdev->cm_core.stats_connect_errs++;
+ return -ENOMEM;
}
/**
diff --git a/drivers/infiniband/hw/i40iw/i40iw_cm.h b/drivers/infiniband/hw/i40iw/i40iw_cm.h
index 5f8ceb4..e9046d9 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_cm.h
+++ b/drivers/infiniband/hw/i40iw/i40iw_cm.h
@@ -1,6 +1,6 @@
/*******************************************************************************
*
-* Copyright (c) 2015 Intel Corporation. All rights reserved.
+* Copyright (c) 2015-2016 Intel Corporation. All rights reserved.
*
* This software is available to you under a choice of one of two
* licenses. You may choose to be licensed under the terms of the GNU
@@ -291,8 +291,6 @@
u8 loc_mac[ETH_ALEN];
u32 loc_addr[4];
u16 loc_port;
- u32 map_loc_addr[4];
- u16 map_loc_port;
struct iw_cm_id *cm_id;
atomic_t ref_count;
struct i40iw_device *iwdev;
@@ -317,8 +315,6 @@
struct i40iw_cm_node {
u32 loc_addr[4], rem_addr[4];
u16 loc_port, rem_port;
- u32 map_loc_addr[4], map_rem_addr[4];
- u16 map_loc_port, map_rem_port;
u16 vlan_id;
enum i40iw_cm_node_state state;
u8 loc_mac[ETH_ALEN];
@@ -370,10 +366,6 @@
u16 rem_port;
u32 loc_addr[4];
u32 rem_addr[4];
- u16 map_loc_port;
- u16 map_rem_port;
- u32 map_loc_addr[4];
- u32 map_rem_addr[4];
u16 vlan_id;
int backlog;
u16 user_pri;
diff --git a/drivers/infiniband/hw/i40iw/i40iw_ctrl.c b/drivers/infiniband/hw/i40iw/i40iw_ctrl.c
index f05802b..2c4b4d0 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_ctrl.c
+++ b/drivers/infiniband/hw/i40iw/i40iw_ctrl.c
@@ -114,16 +114,21 @@
* i40iw_sc_parse_fpm_commit_buf - parse fpm commit buffer
* @buf: ptr to fpm commit buffer
* @info: ptr to i40iw_hmc_obj_info struct
+ * @sd: number of SDs for HMC objects
*
* parses fpm commit info and copy base value
* of hmc objects in hmc_info
*/
static enum i40iw_status_code i40iw_sc_parse_fpm_commit_buf(
u64 *buf,
- struct i40iw_hmc_obj_info *info)
+ struct i40iw_hmc_obj_info *info,
+ u32 *sd)
{
u64 temp;
+ u64 size;
+ u64 base = 0;
u32 i, j;
+ u32 k = 0;
u32 low;
/* copy base values in obj_info */
@@ -131,10 +136,20 @@
i <= I40IW_HMC_IW_PBLE; i++, j += 8) {
get_64bit_val(buf, j, &temp);
info[i].base = RS_64_1(temp, 32) * 512;
+ if (info[i].base > base) {
+ base = info[i].base;
+ k = i;
+ }
low = (u32)(temp);
if (low)
info[i].cnt = low;
}
+ size = info[k].cnt * info[k].size + info[k].base;
+ if (size & 0x1FFFFF)
+ *sd = (u32)((size >> 21) + 1); /* add 1 for remainder */
+ else
+ *sd = (u32)(size >> 21);
+
return 0;
}
@@ -2909,6 +2924,65 @@
}
/**
+ * i40iw_sc_mr_fast_register - Posts RDMA fast register mr WR to iwarp qp
+ * @qp: sc qp struct
+ * @info: fast mr info
+ * @post_sq: flag for cqp db to ring
+ */
+enum i40iw_status_code i40iw_sc_mr_fast_register(
+ struct i40iw_sc_qp *qp,
+ struct i40iw_fast_reg_stag_info *info,
+ bool post_sq)
+{
+ u64 temp, header;
+ u64 *wqe;
+ u32 wqe_idx;
+
+ wqe = i40iw_qp_get_next_send_wqe(&qp->qp_uk, &wqe_idx, I40IW_QP_WQE_MIN_SIZE,
+ 0, info->wr_id);
+ if (!wqe)
+ return I40IW_ERR_QP_TOOMANY_WRS_POSTED;
+
+ i40iw_debug(qp->dev, I40IW_DEBUG_MR, "%s: wr_id[%llxh] wqe_idx[%04d] location[%p]\n",
+ __func__, info->wr_id, wqe_idx,
+ &qp->qp_uk.sq_wrtrk_array[wqe_idx].wrid);
+ temp = (info->addr_type == I40IW_ADDR_TYPE_VA_BASED) ? (uintptr_t)info->va : info->fbo;
+ set_64bit_val(wqe, 0, temp);
+
+ temp = RS_64(info->first_pm_pbl_index >> 16, I40IWQPSQ_FIRSTPMPBLIDXHI);
+ set_64bit_val(wqe,
+ 8,
+ LS_64(temp, I40IWQPSQ_FIRSTPMPBLIDXHI) |
+ LS_64(info->reg_addr_pa >> I40IWQPSQ_PBLADDR_SHIFT, I40IWQPSQ_PBLADDR));
+
+ set_64bit_val(wqe,
+ 16,
+ info->total_len |
+ LS_64(info->first_pm_pbl_index, I40IWQPSQ_FIRSTPMPBLIDXLO));
+
+ header = LS_64(info->stag_key, I40IWQPSQ_STAGKEY) |
+ LS_64(info->stag_idx, I40IWQPSQ_STAGINDEX) |
+ LS_64(I40IWQP_OP_FAST_REGISTER, I40IWQPSQ_OPCODE) |
+ LS_64(info->chunk_size, I40IWQPSQ_LPBLSIZE) |
+ LS_64(info->page_size, I40IWQPSQ_HPAGESIZE) |
+ LS_64(info->access_rights, I40IWQPSQ_STAGRIGHTS) |
+ LS_64(info->addr_type, I40IWQPSQ_VABASEDTO) |
+ LS_64(info->read_fence, I40IWQPSQ_READFENCE) |
+ LS_64(info->local_fence, I40IWQPSQ_LOCALFENCE) |
+ LS_64(info->signaled, I40IWQPSQ_SIGCOMPL) |
+ LS_64(qp->qp_uk.swqe_polarity, I40IWQPSQ_VALID);
+
+ i40iw_insert_wqe_hdr(wqe, header);
+
+ i40iw_debug_buf(qp->dev, I40IW_DEBUG_WQE, "FAST_REG WQE",
+ wqe, I40IW_QP_WQE_MIN_SIZE);
+
+ if (post_sq)
+ i40iw_qp_post_wr(&qp->qp_uk);
+ return 0;
+}
+
+/**
* i40iw_sc_send_lsmm - send last streaming mode message
* @qp: sc qp struct
* @lsmm_buf: buffer with lsmm message
@@ -3147,7 +3221,7 @@
i40iw_cqp_commit_fpm_values_cmd(dev, &query_fpm_mem, hmc_fn_id);
/* parse the fpm_commit_buf and fill hmc obj info */
- i40iw_sc_parse_fpm_commit_buf((u64 *)query_fpm_mem.va, hmc_info->hmc_obj);
+ i40iw_sc_parse_fpm_commit_buf((u64 *)query_fpm_mem.va, hmc_info->hmc_obj, &hmc_info->sd_table.sd_cnt);
mem_size = sizeof(struct i40iw_hmc_sd_entry) *
(hmc_info->sd_table.sd_cnt + hmc_info->first_sd_index);
ret_code = i40iw_allocate_virt_mem(dev->hw, &virt_mem, mem_size);
@@ -3221,7 +3295,9 @@
/* parse the fpm_commit_buf and fill hmc obj info */
if (!ret_code)
- ret_code = i40iw_sc_parse_fpm_commit_buf(dev->fpm_commit_buf, hmc_info->hmc_obj);
+ ret_code = i40iw_sc_parse_fpm_commit_buf(dev->fpm_commit_buf,
+ hmc_info->hmc_obj,
+ &hmc_info->sd_table.sd_cnt);
i40iw_debug_buf(dev, I40IW_DEBUG_HMC, "COMMIT FPM BUFFER",
commit_fpm_mem.va, I40IW_COMMIT_FPM_BUF_SIZE);
@@ -3469,6 +3545,40 @@
}
/**
+ * i40iw_est_sd - returns approximate number of SDs for HMC
+ * @dev: sc device struct
+ * @hmc_info: hmc structure, size and count for HMC objects
+ */
+static u64 i40iw_est_sd(struct i40iw_sc_dev *dev, struct i40iw_hmc_info *hmc_info)
+{
+ int i;
+ u64 size = 0;
+ u64 sd;
+
+ for (i = I40IW_HMC_IW_QP; i < I40IW_HMC_IW_PBLE; i++)
+ size += hmc_info->hmc_obj[i].cnt * hmc_info->hmc_obj[i].size;
+
+ if (dev->is_pf)
+ size += hmc_info->hmc_obj[I40IW_HMC_IW_PBLE].cnt * hmc_info->hmc_obj[I40IW_HMC_IW_PBLE].size;
+
+ if (size & 0x1FFFFF)
+ sd = (size >> 21) + 1; /* add 1 for remainder */
+ else
+ sd = size >> 21;
+
+ if (!dev->is_pf) {
+ /* 2MB alignment for VF PBLE HMC */
+ size = hmc_info->hmc_obj[I40IW_HMC_IW_PBLE].cnt * hmc_info->hmc_obj[I40IW_HMC_IW_PBLE].size;
+ if (size & 0x1FFFFF)
+ sd += (size >> 21) + 1; /* add 1 for remainder */
+ else
+ sd += size >> 21;
+ }
+
+ return sd;
+}
+
+/**
* i40iw_config_fpm_values - configure HMC objects
* @dev: sc device struct
* @qp_count: desired qp count
@@ -3479,7 +3589,7 @@
u32 i, mem_size;
u32 qpwantedoriginal, qpwanted, mrwanted, pblewanted;
u32 powerof2;
- u64 sd_needed, bytes_needed;
+ u64 sd_needed;
u32 loop_count = 0;
struct i40iw_hmc_info *hmc_info;
@@ -3497,23 +3607,15 @@
return ret_code;
}
- bytes_needed = 0;
- for (i = I40IW_HMC_IW_QP; i < I40IW_HMC_IW_MAX; i++) {
+ for (i = I40IW_HMC_IW_QP; i < I40IW_HMC_IW_MAX; i++)
hmc_info->hmc_obj[i].cnt = hmc_info->hmc_obj[i].max_cnt;
- bytes_needed +=
- (hmc_info->hmc_obj[i].max_cnt) * (hmc_info->hmc_obj[i].size);
- i40iw_debug(dev, I40IW_DEBUG_HMC,
- "%s i[%04d] max_cnt[0x%04X] size[0x%04llx]\n",
- __func__, i, hmc_info->hmc_obj[i].max_cnt,
- hmc_info->hmc_obj[i].size);
- }
- sd_needed = (bytes_needed / I40IW_HMC_DIRECT_BP_SIZE) + 1; /* round up */
+ sd_needed = i40iw_est_sd(dev, hmc_info);
i40iw_debug(dev, I40IW_DEBUG_HMC,
"%s: FW initial max sd_count[%08lld] first_sd_index[%04d]\n",
__func__, sd_needed, hmc_info->first_sd_index);
i40iw_debug(dev, I40IW_DEBUG_HMC,
- "%s: bytes_needed=0x%llx sd count %d where max sd is %d\n",
- __func__, bytes_needed, hmc_info->sd_table.sd_cnt,
+ "%s: sd count %d where max sd is %d\n",
+ __func__, hmc_info->sd_table.sd_cnt,
hmc_fpm_misc->max_sds);
qpwanted = min(qp_count, hmc_info->hmc_obj[I40IW_HMC_IW_QP].max_cnt);
@@ -3555,11 +3657,7 @@
hmc_info->hmc_obj[I40IW_HMC_IW_PBLE].cnt = pblewanted;
/* How much memory is needed for all the objects. */
- bytes_needed = 0;
- for (i = I40IW_HMC_IW_QP; i < I40IW_HMC_IW_MAX; i++)
- bytes_needed +=
- (hmc_info->hmc_obj[i].cnt) * (hmc_info->hmc_obj[i].size);
- sd_needed = (bytes_needed / I40IW_HMC_DIRECT_BP_SIZE) + 1;
+ sd_needed = i40iw_est_sd(dev, hmc_info);
if ((loop_count > 1000) ||
((!(loop_count % 10)) &&
(qpwanted > qpwantedoriginal * 2 / 3))) {
@@ -3580,15 +3678,7 @@
pblewanted -= FPM_MULTIPLIER * 1000;
} while (sd_needed > hmc_fpm_misc->max_sds && loop_count < 2000);
- bytes_needed = 0;
- for (i = I40IW_HMC_IW_QP; i < I40IW_HMC_IW_MAX; i++) {
- bytes_needed += (hmc_info->hmc_obj[i].cnt) * (hmc_info->hmc_obj[i].size);
- i40iw_debug(dev, I40IW_DEBUG_HMC,
- "%s i[%04d] cnt[0x%04x] size[0x%04llx]\n",
- __func__, i, hmc_info->hmc_obj[i].cnt,
- hmc_info->hmc_obj[i].size);
- }
- sd_needed = (bytes_needed / I40IW_HMC_DIRECT_BP_SIZE) + 1; /* round up not truncate. */
+ sd_needed = i40iw_est_sd(dev, hmc_info);
i40iw_debug(dev, I40IW_DEBUG_HMC,
"loop_cnt=%d, sd_needed=%lld, qpcnt = %d, cqcnt=%d, mrcnt=%d, pblecnt=%d\n",
@@ -3606,8 +3696,6 @@
return ret_code;
}
- hmc_info->sd_table.sd_cnt = (u32)sd_needed;
-
mem_size = sizeof(struct i40iw_hmc_sd_entry) *
(hmc_info->sd_table.sd_cnt + hmc_info->first_sd_index + 1);
ret_code = i40iw_allocate_virt_mem(dev->hw, &virt_mem, mem_size);
@@ -3911,11 +3999,11 @@
*/
static u32 i40iw_iwarp_opcode(struct i40iw_aeqe_info *info, u8 *pkt)
{
- u16 *mpa;
+ __be16 *mpa;
u32 opcode = 0xffffffff;
if (info->q2_data_written) {
- mpa = (u16 *)pkt;
+ mpa = (__be16 *)pkt;
opcode = ntohs(mpa[1]) & 0xf;
}
return opcode;
@@ -3977,7 +4065,7 @@
if (info->q2_data_written) {
/* Use data from offending packet to fill in ddp & rdma hdrs */
pkt = i40iw_locate_mpa(pkt);
- ddp_seg_len = ntohs(*(u16 *)pkt);
+ ddp_seg_len = ntohs(*(__be16 *)pkt);
if (ddp_seg_len) {
copy_len = 2;
termhdr->hdrct = DDP_LEN_FLAG;
@@ -4188,13 +4276,13 @@
void i40iw_terminate_received(struct i40iw_sc_qp *qp, struct i40iw_aeqe_info *info)
{
u8 *pkt = qp->q2_buf + Q2_BAD_FRAME_OFFSET;
- u32 *mpa;
+ __be32 *mpa;
u8 ddp_ctl;
u8 rdma_ctl;
u16 aeq_id = 0;
struct i40iw_terminate_hdr *termhdr;
- mpa = (u32 *)i40iw_locate_mpa(pkt);
+ mpa = (__be32 *)i40iw_locate_mpa(pkt);
if (info->q2_data_written) {
/* did not validate the frame - do it now */
ddp_ctl = (ntohl(mpa[0]) >> 8) & 0xff;
@@ -4559,17 +4647,18 @@
};
static struct i40iw_priv_qp_ops iw_priv_qp_ops = {
- i40iw_sc_qp_init,
- i40iw_sc_qp_create,
- i40iw_sc_qp_modify,
- i40iw_sc_qp_destroy,
- i40iw_sc_qp_flush_wqes,
- i40iw_sc_qp_upload_context,
- i40iw_sc_qp_setctx,
- i40iw_sc_send_lsmm,
- i40iw_sc_send_lsmm_nostag,
- i40iw_sc_send_rtt,
- i40iw_sc_post_wqe0,
+ .qp_init = i40iw_sc_qp_init,
+ .qp_create = i40iw_sc_qp_create,
+ .qp_modify = i40iw_sc_qp_modify,
+ .qp_destroy = i40iw_sc_qp_destroy,
+ .qp_flush_wqes = i40iw_sc_qp_flush_wqes,
+ .qp_upload_context = i40iw_sc_qp_upload_context,
+ .qp_setctx = i40iw_sc_qp_setctx,
+ .qp_send_lsmm = i40iw_sc_send_lsmm,
+ .qp_send_lsmm_nostag = i40iw_sc_send_lsmm_nostag,
+ .qp_send_rtt = i40iw_sc_send_rtt,
+ .qp_post_wqe0 = i40iw_sc_post_wqe0,
+ .iw_mr_fast_register = i40iw_sc_mr_fast_register
};
static struct i40iw_priv_cq_ops iw_priv_cq_ops = {
diff --git a/drivers/infiniband/hw/i40iw/i40iw_d.h b/drivers/infiniband/hw/i40iw/i40iw_d.h
index aab88d6..bd942da 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_d.h
+++ b/drivers/infiniband/hw/i40iw/i40iw_d.h
@@ -1290,7 +1290,7 @@
/* wqe size considering 32 bytes per wqe*/
#define I40IWQP_SW_MIN_WQSIZE 4 /* 128 bytes */
-#define I40IWQP_SW_MAX_WQSIZE 16384 /* 524288 bytes */
+#define I40IWQP_SW_MAX_WQSIZE 2048 /* 2048 bytes */
#define I40IWQP_OP_RDMA_WRITE 0
#define I40IWQP_OP_RDMA_READ 1
@@ -1512,6 +1512,8 @@
I40IW_SD_BUF_ALIGNMENT = 0x100
};
+#define I40IW_WQE_SIZE_64 64
+
#define I40IW_QP_WQE_MIN_SIZE 32
#define I40IW_QP_WQE_MAX_SIZE 128
diff --git a/drivers/infiniband/hw/i40iw/i40iw_hw.c b/drivers/infiniband/hw/i40iw/i40iw_hw.c
index 9fd3024..3ee0cad 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_hw.c
+++ b/drivers/infiniband/hw/i40iw/i40iw_hw.c
@@ -106,7 +106,9 @@
set_bit(2, iwdev->allocated_pds);
spin_lock_init(&iwdev->resource_lock);
- mrdrvbits = 24 - get_count_order(iwdev->max_mr);
+ spin_lock_init(&iwdev->qptable_lock);
+ /* stag index mask has a minimum of 14 bits */
+ mrdrvbits = 24 - max(get_count_order(iwdev->max_mr), 14);
iwdev->mr_stagmask = ~(((1 << mrdrvbits) - 1) << (32 - mrdrvbits));
return 0;
}
@@ -301,11 +303,15 @@
"%s ae_id = 0x%x bool qp=%d qp_id = %d\n",
__func__, info->ae_id, info->qp, info->qp_cq_id);
if (info->qp) {
+ spin_lock_irqsave(&iwdev->qptable_lock, flags);
iwqp = iwdev->qp_table[info->qp_cq_id];
if (!iwqp) {
+ spin_unlock_irqrestore(&iwdev->qptable_lock, flags);
i40iw_pr_err("qp_id %d is already freed\n", info->qp_cq_id);
continue;
}
+ i40iw_add_ref(&iwqp->ibqp);
+ spin_unlock_irqrestore(&iwdev->qptable_lock, flags);
qp = &iwqp->sc_qp;
spin_lock_irqsave(&iwqp->lock, flags);
iwqp->hw_tcp_state = info->tcp_state;
@@ -411,6 +417,8 @@
i40iw_terminate_connection(qp, info);
break;
}
+ if (info->qp)
+ i40iw_rem_ref(&iwqp->ibqp);
} while (1);
if (aeqcnt)
@@ -460,7 +468,7 @@
*/
void i40iw_manage_arp_cache(struct i40iw_device *iwdev,
unsigned char *mac_addr,
- __be32 *ip_addr,
+ u32 *ip_addr,
bool ipv4,
u32 action)
{
@@ -481,7 +489,7 @@
cqp_info->cqp_cmd = OP_ADD_ARP_CACHE_ENTRY;
info = &cqp_info->in.u.add_arp_cache_entry.info;
memset(info, 0, sizeof(*info));
- info->arp_index = cpu_to_le32(arp_index);
+ info->arp_index = cpu_to_le16((u16)arp_index);
info->permanent = true;
ether_addr_copy(info->mac_addr, mac_addr);
cqp_info->in.u.add_arp_cache_entry.scratch = (uintptr_t)cqp_request;
diff --git a/drivers/infiniband/hw/i40iw/i40iw_main.c b/drivers/infiniband/hw/i40iw/i40iw_main.c
index e41fae24..c963cad 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_main.c
+++ b/drivers/infiniband/hw/i40iw/i40iw_main.c
@@ -270,7 +270,6 @@
i40iw_wr32(dev->hw, I40E_PFINT_DYN_CTLN(msix_vec->idx - 1), 0);
else
i40iw_wr32(dev->hw, I40E_VFINT_DYN_CTLN1(msix_vec->idx - 1), 0);
- synchronize_irq(msix_vec->irq);
free_irq(msix_vec->irq, dev_id);
}
@@ -1147,10 +1146,7 @@
if (!status) {
status = i40iw_add_mac_ipaddr_entry(iwdev, macaddr,
(u8)iwdev->mac_ip_table_idx);
- if (!status)
- status = i40iw_add_mac_ipaddr_entry(iwdev, macaddr,
- (u8)iwdev->mac_ip_table_idx);
- else
+ if (status)
i40iw_del_macip_entry(iwdev, (u8)iwdev->mac_ip_table_idx);
}
return status;
@@ -1165,7 +1161,7 @@
struct net_device *ip_dev;
struct inet6_dev *idev;
struct inet6_ifaddr *ifp;
- __be32 local_ipaddr6[4];
+ u32 local_ipaddr6[4];
rcu_read_lock();
for_each_netdev_rcu(&init_net, ip_dev) {
@@ -1512,6 +1508,7 @@
I40IW_HMC_PROFILE_DEFAULT;
iwdev->max_rdma_vfs =
(iwdev->resource_profile != I40IW_HMC_PROFILE_DEFAULT) ? max_rdma_vfs : 0;
+ iwdev->max_enabled_vfs = iwdev->max_rdma_vfs;
iwdev->netdev = ldev->netdev;
hdl->client = client;
iwdev->mss = (!ldev->params.mtu) ? I40IW_DEFAULT_MSS : ldev->params.mtu - I40IW_MTU_TO_MSS;
@@ -1531,7 +1528,10 @@
goto exit;
iwdev->obj_next = iwdev->obj_mem;
iwdev->push_mode = push_mode;
+
init_waitqueue_head(&iwdev->vchnl_waitq);
+ init_waitqueue_head(&dev->vf_reqs);
+
status = i40iw_initialize_dev(iwdev, ldev);
exit:
if (status) {
@@ -1710,7 +1710,6 @@
for (i = 0; i < I40IW_MAX_PE_ENABLED_VF_COUNT; i++) {
if (!dev->vf_dev[i] || (dev->vf_dev[i]->vf_id != vf_id))
continue;
-
/* free all resources allocated on behalf of vf */
tmp_vfdev = dev->vf_dev[i];
spin_lock_irqsave(&dev->dev_pestat.stats_lock, flags);
@@ -1819,8 +1818,6 @@
dev = &hdl->device.sc_dev;
iwdev = dev->back_dev;
- i40iw_debug(dev, I40IW_DEBUG_VIRT, "msg %p, message length %u\n", msg, len);
-
if (dev->vchnl_if.vchnl_recv) {
ret_code = dev->vchnl_if.vchnl_recv(dev, vf_id, msg, len);
if (!dev->is_pf) {
@@ -1832,6 +1829,39 @@
}
/**
+ * i40iw_vf_clear_to_send - wait to send virtual channel message
+ * @dev: iwarp device *
+ * Wait for until virtual channel is clear
+ * before sending the next message
+ *
+ * Returns false if error
+ * Returns true if clear to send
+ */
+bool i40iw_vf_clear_to_send(struct i40iw_sc_dev *dev)
+{
+ struct i40iw_device *iwdev;
+ wait_queue_t wait;
+
+ iwdev = dev->back_dev;
+
+ if (!wq_has_sleeper(&dev->vf_reqs) &&
+ (atomic_read(&iwdev->vchnl_msgs) == 0))
+ return true; /* virtual channel is clear */
+
+ init_wait(&wait);
+ add_wait_queue_exclusive(&dev->vf_reqs, &wait);
+
+ if (!wait_event_timeout(dev->vf_reqs,
+ (atomic_read(&iwdev->vchnl_msgs) == 0),
+ I40IW_VCHNL_EVENT_TIMEOUT))
+ dev->vchnl_up = false;
+
+ remove_wait_queue(&dev->vf_reqs, &wait);
+
+ return dev->vchnl_up;
+}
+
+/**
* i40iw_virtchnl_send - send a message through the virtual channel
* @dev: iwarp device
* @vf_id: virtual function id associated with the message
@@ -1848,18 +1878,16 @@
{
struct i40iw_device *iwdev;
struct i40e_info *ldev;
- enum i40iw_status_code ret_code = I40IW_ERR_BAD_PTR;
if (!dev || !dev->back_dev)
- return ret_code;
+ return I40IW_ERR_BAD_PTR;
iwdev = dev->back_dev;
ldev = iwdev->ldev;
if (ldev && ldev->ops && ldev->ops->virtchnl_send)
- ret_code = ldev->ops->virtchnl_send(ldev, &i40iw_client, vf_id, msg, len);
-
- return ret_code;
+ return ldev->ops->virtchnl_send(ldev, &i40iw_client, vf_id, msg, len);
+ return I40IW_ERR_BAD_PTR;
}
/* client interface functions */
diff --git a/drivers/infiniband/hw/i40iw/i40iw_osdep.h b/drivers/infiniband/hw/i40iw/i40iw_osdep.h
index 7e20493..80f422b 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_osdep.h
+++ b/drivers/infiniband/hw/i40iw/i40iw_osdep.h
@@ -172,6 +172,7 @@
u8 __iomem *i40iw_get_hw_addr(void *dev);
void i40iw_ieq_mpa_crc_ae(struct i40iw_sc_dev *dev, struct i40iw_sc_qp *qp);
enum i40iw_status_code i40iw_vf_wait_vchnl_resp(struct i40iw_sc_dev *dev);
+bool i40iw_vf_clear_to_send(struct i40iw_sc_dev *dev);
enum i40iw_status_code i40iw_ieq_check_mpacrc(struct shash_desc *desc, void *addr,
u32 length, u32 value);
struct i40iw_sc_qp *i40iw_ieq_get_qp(struct i40iw_sc_dev *dev, struct i40iw_puda_buf *buf);
diff --git a/drivers/infiniband/hw/i40iw/i40iw_pble.c b/drivers/infiniband/hw/i40iw/i40iw_pble.c
index ded853d..85993dc 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_pble.c
+++ b/drivers/infiniband/hw/i40iw/i40iw_pble.c
@@ -404,13 +404,14 @@
sd_entry->u.pd_table.pd_page_addr.pa : sd_entry->u.bp.addr.pa;
if (sd_entry->valid)
return 0;
- if (dev->is_pf)
+ if (dev->is_pf) {
ret_code = i40iw_hmc_sd_one(dev, hmc_info->hmc_fn_id,
sd_reg_val, idx->sd_idx,
sd_entry->entry_type, true);
- if (ret_code) {
- i40iw_pr_err("cqp cmd failed for sd (pbles)\n");
- goto error;
+ if (ret_code) {
+ i40iw_pr_err("cqp cmd failed for sd (pbles)\n");
+ goto error;
+ }
}
sd_entry->valid = true;
diff --git a/drivers/infiniband/hw/i40iw/i40iw_puda.c b/drivers/infiniband/hw/i40iw/i40iw_puda.c
index 8eb400d8..e9c6e82 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_puda.c
+++ b/drivers/infiniband/hw/i40iw/i40iw_puda.c
@@ -1194,7 +1194,7 @@
ioffset = (u16)(buf->data - (u8 *)buf->mem.va);
while (datalen) {
- fpdu_len = i40iw_ieq_get_fpdu_length(ntohs(*(u16 *)datap));
+ fpdu_len = i40iw_ieq_get_fpdu_length(ntohs(*(__be16 *)datap));
if (fpdu_len > pfpdu->max_fpdu_data) {
i40iw_debug(ieq->dev, I40IW_DEBUG_IEQ,
"%s: error bad fpdu_len\n", __func__);
diff --git a/drivers/infiniband/hw/i40iw/i40iw_status.h b/drivers/infiniband/hw/i40iw/i40iw_status.h
index b0110c1..91c4217 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_status.h
+++ b/drivers/infiniband/hw/i40iw/i40iw_status.h
@@ -95,6 +95,7 @@
I40IW_ERR_INVALID_MAC_ADDR = -65,
I40IW_ERR_BAD_STAG = -66,
I40IW_ERR_CQ_COMPL_ERROR = -67,
+ I40IW_ERR_QUEUE_DESTROYED = -68
};
#endif
diff --git a/drivers/infiniband/hw/i40iw/i40iw_type.h b/drivers/infiniband/hw/i40iw/i40iw_type.h
index edb3a8c..16cc617 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_type.h
+++ b/drivers/infiniband/hw/i40iw/i40iw_type.h
@@ -479,16 +479,17 @@
struct i40iw_virt_mem ieq_mem;
struct i40iw_puda_rsrc *ieq;
- struct i40iw_vf_cqp_ops *iw_vf_cqp_ops;
+ const struct i40iw_vf_cqp_ops *iw_vf_cqp_ops;
struct i40iw_hmc_fpm_misc hmc_fpm_misc;
u16 qs_handle;
- u32 debug_mask;
+ u32 debug_mask;
u16 exception_lan_queue;
u8 hmc_fn_id;
bool is_pf;
bool vchnl_up;
u8 vf_id;
+ wait_queue_head_t vf_reqs;
u64 cqp_cmd_stats[OP_SIZE_CQP_STAT_ARRAY];
struct i40iw_vchnl_vf_msg_buffer vchnl_vf_msg_buf;
u8 hw_rev;
@@ -889,8 +890,8 @@
u32 qp_num;
u32 dest_ip[4];
u32 src_ip[4];
- u32 dest_port;
- u32 src_port;
+ u16 dest_port;
+ u16 src_port;
};
struct i40iw_local_mac_ipaddr_entry_info {
@@ -1040,6 +1041,9 @@
void (*qp_send_lsmm_nostag)(struct i40iw_sc_qp *, void *, u32);
void (*qp_send_rtt)(struct i40iw_sc_qp *, bool);
enum i40iw_status_code (*qp_post_wqe0)(struct i40iw_sc_qp *, u8);
+ enum i40iw_status_code (*iw_mr_fast_register)(struct i40iw_sc_qp *,
+ struct i40iw_fast_reg_stag_info *,
+ bool);
};
struct i40iw_priv_cq_ops {
@@ -1108,7 +1112,7 @@
enum i40iw_status_code (*parse_fpm_query_buf)(u64 *, struct i40iw_hmc_info *,
struct i40iw_hmc_fpm_misc *);
enum i40iw_status_code (*configure_iw_fpm)(struct i40iw_sc_dev *, u8);
- enum i40iw_status_code (*parse_fpm_commit_buf)(u64 *, struct i40iw_hmc_obj_info *);
+ enum i40iw_status_code (*parse_fpm_commit_buf)(u64 *, struct i40iw_hmc_obj_info *, u32 *sd);
enum i40iw_status_code (*create_hmc_object)(struct i40iw_sc_dev *dev,
struct i40iw_hmc_create_obj_info *);
enum i40iw_status_code (*del_hmc_object)(struct i40iw_sc_dev *dev,
diff --git a/drivers/infiniband/hw/i40iw/i40iw_uk.c b/drivers/infiniband/hw/i40iw/i40iw_uk.c
index f78c3dc..e35faea8 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_uk.c
+++ b/drivers/infiniband/hw/i40iw/i40iw_uk.c
@@ -56,6 +56,9 @@
wqe_idx = I40IW_RING_GETCURRENT_HEAD(qp->sq_ring);
wqe = qp->sq_base[wqe_idx].elem;
+
+ qp->sq_wrtrk_array[wqe_idx].wqe_size = I40IW_QP_WQE_MIN_SIZE;
+
peek_head = (qp->sq_ring.head + 1) % qp->sq_ring.size;
wqe_0 = qp->sq_base[peek_head].elem;
if (peek_head)
@@ -130,7 +133,10 @@
*/
u64 *i40iw_qp_get_next_send_wqe(struct i40iw_qp_uk *qp,
u32 *wqe_idx,
- u8 wqe_size)
+ u8 wqe_size,
+ u32 total_size,
+ u64 wr_id
+ )
{
u64 *wqe = NULL;
u64 wqe_ptr;
@@ -159,6 +165,17 @@
if (!*wqe_idx)
qp->swqe_polarity = !qp->swqe_polarity;
}
+
+ if (((*wqe_idx & 3) == 1) && (wqe_size == I40IW_WQE_SIZE_64)) {
+ i40iw_nop_1(qp);
+ I40IW_RING_MOVE_HEAD(qp->sq_ring, ret_code);
+ if (ret_code)
+ return NULL;
+ *wqe_idx = I40IW_RING_GETCURRENT_HEAD(qp->sq_ring);
+ if (!*wqe_idx)
+ qp->swqe_polarity = !qp->swqe_polarity;
+ }
+
for (i = 0; i < wqe_size / I40IW_QP_WQE_MIN_SIZE; i++) {
I40IW_RING_MOVE_HEAD(qp->sq_ring, ret_code);
if (ret_code)
@@ -169,8 +186,15 @@
peek_head = I40IW_RING_GETCURRENT_HEAD(qp->sq_ring);
wqe_0 = qp->sq_base[peek_head].elem;
- if (peek_head & 0x3)
- wqe_0[3] = LS_64(!qp->swqe_polarity, I40IWQPSQ_VALID);
+
+ if (((peek_head & 3) == 1) || ((peek_head & 3) == 3)) {
+ if (RS_64(wqe_0[3], I40IWQPSQ_VALID) != !qp->swqe_polarity)
+ wqe_0[3] = LS_64(!qp->swqe_polarity, I40IWQPSQ_VALID);
+ }
+
+ qp->sq_wrtrk_array[*wqe_idx].wrid = wr_id;
+ qp->sq_wrtrk_array[*wqe_idx].wr_len = total_size;
+ qp->sq_wrtrk_array[*wqe_idx].wqe_size = wqe_size;
return wqe;
}
@@ -249,12 +273,9 @@
if (ret_code)
return ret_code;
- wqe = i40iw_qp_get_next_send_wqe(qp, &wqe_idx, wqe_size);
+ wqe = i40iw_qp_get_next_send_wqe(qp, &wqe_idx, wqe_size, total_size, info->wr_id);
if (!wqe)
return I40IW_ERR_QP_TOOMANY_WRS_POSTED;
-
- qp->sq_wrtrk_array[wqe_idx].wrid = info->wr_id;
- qp->sq_wrtrk_array[wqe_idx].wr_len = total_size;
set_64bit_val(wqe, 16,
LS_64(op_info->rem_addr.tag_off, I40IWQPSQ_FRAG_TO));
if (!op_info->rem_addr.stag)
@@ -309,12 +330,9 @@
ret_code = i40iw_fragcnt_to_wqesize_sq(1, &wqe_size);
if (ret_code)
return ret_code;
- wqe = i40iw_qp_get_next_send_wqe(qp, &wqe_idx, wqe_size);
+ wqe = i40iw_qp_get_next_send_wqe(qp, &wqe_idx, wqe_size, op_info->lo_addr.len, info->wr_id);
if (!wqe)
return I40IW_ERR_QP_TOOMANY_WRS_POSTED;
-
- qp->sq_wrtrk_array[wqe_idx].wrid = info->wr_id;
- qp->sq_wrtrk_array[wqe_idx].wr_len = op_info->lo_addr.len;
local_fence |= info->local_fence;
set_64bit_val(wqe, 16, LS_64(op_info->rem_addr.tag_off, I40IWQPSQ_FRAG_TO));
@@ -366,13 +384,11 @@
if (ret_code)
return ret_code;
- wqe = i40iw_qp_get_next_send_wqe(qp, &wqe_idx, wqe_size);
+ wqe = i40iw_qp_get_next_send_wqe(qp, &wqe_idx, wqe_size, total_size, info->wr_id);
if (!wqe)
return I40IW_ERR_QP_TOOMANY_WRS_POSTED;
read_fence |= info->read_fence;
- qp->sq_wrtrk_array[wqe_idx].wrid = info->wr_id;
- qp->sq_wrtrk_array[wqe_idx].wr_len = total_size;
set_64bit_val(wqe, 16, 0);
header = LS_64(stag_to_inv, I40IWQPSQ_REMSTAG) |
LS_64(info->op_type, I40IWQPSQ_OPCODE) |
@@ -427,13 +443,11 @@
if (ret_code)
return ret_code;
- wqe = i40iw_qp_get_next_send_wqe(qp, &wqe_idx, wqe_size);
+ wqe = i40iw_qp_get_next_send_wqe(qp, &wqe_idx, wqe_size, op_info->len, info->wr_id);
if (!wqe)
return I40IW_ERR_QP_TOOMANY_WRS_POSTED;
read_fence |= info->read_fence;
- qp->sq_wrtrk_array[wqe_idx].wrid = info->wr_id;
- qp->sq_wrtrk_array[wqe_idx].wr_len = op_info->len;
set_64bit_val(wqe, 16,
LS_64(op_info->rem_addr.tag_off, I40IWQPSQ_FRAG_TO));
@@ -507,14 +521,11 @@
if (ret_code)
return ret_code;
- wqe = i40iw_qp_get_next_send_wqe(qp, &wqe_idx, wqe_size);
+ wqe = i40iw_qp_get_next_send_wqe(qp, &wqe_idx, wqe_size, op_info->len, info->wr_id);
if (!wqe)
return I40IW_ERR_QP_TOOMANY_WRS_POSTED;
read_fence |= info->read_fence;
-
- qp->sq_wrtrk_array[wqe_idx].wrid = info->wr_id;
- qp->sq_wrtrk_array[wqe_idx].wr_len = op_info->len;
header = LS_64(stag_to_inv, I40IWQPSQ_REMSTAG) |
LS_64(info->op_type, I40IWQPSQ_OPCODE) |
LS_64(op_info->len, I40IWQPSQ_INLINEDATALEN) |
@@ -574,12 +585,9 @@
op_info = &info->op.inv_local_stag;
local_fence = info->local_fence;
- wqe = i40iw_qp_get_next_send_wqe(qp, &wqe_idx, I40IW_QP_WQE_MIN_SIZE);
+ wqe = i40iw_qp_get_next_send_wqe(qp, &wqe_idx, I40IW_QP_WQE_MIN_SIZE, 0, info->wr_id);
if (!wqe)
return I40IW_ERR_QP_TOOMANY_WRS_POSTED;
-
- qp->sq_wrtrk_array[wqe_idx].wrid = info->wr_id;
- qp->sq_wrtrk_array[wqe_idx].wr_len = 0;
set_64bit_val(wqe, 0, 0);
set_64bit_val(wqe, 8,
LS_64(op_info->target_stag, I40IWQPSQ_LOCSTAG));
@@ -619,12 +627,9 @@
op_info = &info->op.bind_window;
local_fence |= info->local_fence;
- wqe = i40iw_qp_get_next_send_wqe(qp, &wqe_idx, I40IW_QP_WQE_MIN_SIZE);
+ wqe = i40iw_qp_get_next_send_wqe(qp, &wqe_idx, I40IW_QP_WQE_MIN_SIZE, 0, info->wr_id);
if (!wqe)
return I40IW_ERR_QP_TOOMANY_WRS_POSTED;
-
- qp->sq_wrtrk_array[wqe_idx].wrid = info->wr_id;
- qp->sq_wrtrk_array[wqe_idx].wr_len = 0;
set_64bit_val(wqe, 0, (uintptr_t)op_info->va);
set_64bit_val(wqe, 8,
LS_64(op_info->mr_stag, I40IWQPSQ_PARENTMRSTAG) |
@@ -760,7 +765,7 @@
enum i40iw_status_code ret_code2 = 0;
bool move_cq_head = true;
u8 polarity;
- u8 addl_frag_cnt, addl_wqes = 0;
+ u8 addl_wqes = 0;
if (cq->avoid_mem_cflct)
cqe = (u64 *)I40IW_GET_CURRENT_EXTENDED_CQ_ELEMENT(cq);
@@ -797,6 +802,10 @@
info->is_srq = (bool)RS_64(qword3, I40IWCQ_SRQ);
qp = (struct i40iw_qp_uk *)(unsigned long)comp_ctx;
+ if (!qp) {
+ ret_code = I40IW_ERR_QUEUE_DESTROYED;
+ goto exit;
+ }
wqe_idx = (u32)RS_64(qword3, I40IW_CQ_WQEIDX);
info->qp_handle = (i40iw_qp_handle)(unsigned long)qp;
@@ -827,11 +836,8 @@
info->op_type = (u8)RS_64(qword3, I40IWCQ_OP);
sw_wqe = qp->sq_base[wqe_idx].elem;
get_64bit_val(sw_wqe, 24, &wqe_qword);
- addl_frag_cnt =
- (u8)RS_64(wqe_qword, I40IWQPSQ_ADDFRAGCNT);
- i40iw_fragcnt_to_wqesize_sq(addl_frag_cnt + 1, &addl_wqes);
- addl_wqes = (addl_wqes / I40IW_QP_WQE_MIN_SIZE);
+ addl_wqes = qp->sq_wrtrk_array[wqe_idx].wqe_size / I40IW_QP_WQE_MIN_SIZE;
I40IW_RING_SET_TAIL(qp->sq_ring, (wqe_idx + addl_wqes));
} else {
do {
@@ -843,9 +849,7 @@
get_64bit_val(sw_wqe, 24, &wqe_qword);
op_type = (u8)RS_64(wqe_qword, I40IWQPSQ_OPCODE);
info->op_type = op_type;
- addl_frag_cnt = (u8)RS_64(wqe_qword, I40IWQPSQ_ADDFRAGCNT);
- i40iw_fragcnt_to_wqesize_sq(addl_frag_cnt + 1, &addl_wqes);
- addl_wqes = (addl_wqes / I40IW_QP_WQE_MIN_SIZE);
+ addl_wqes = qp->sq_wrtrk_array[tail].wqe_size / I40IW_QP_WQE_MIN_SIZE;
I40IW_RING_SET_TAIL(qp->sq_ring, (tail + addl_wqes));
if (op_type != I40IWQP_OP_NOP) {
info->wr_id = qp->sq_wrtrk_array[tail].wrid;
@@ -859,6 +863,7 @@
ret_code = 0;
+exit:
if (!ret_code &&
(info->comp_status == I40IW_COMPL_STATUS_FLUSHED))
if (pring && (I40IW_RING_MORE_WORK(*pring)))
@@ -893,19 +898,21 @@
* i40iw_get_wqe_shift - get shift count for maximum wqe size
* @wqdepth: depth of wq required.
* @sge: Maximum Scatter Gather Elements wqe
+ * @inline_data: Maximum inline data size
* @shift: Returns the shift needed based on sge
*
- * Shift can be used to left shift the wqe size based on sge.
- * If sge, == 1, shift =0 (wqe_size of 32 bytes), for sge=2 and 3, shift =1
- * (64 bytes wqes) and 2 otherwise (128 bytes wqe).
+ * Shift can be used to left shift the wqe size based on number of SGEs and inlind data size.
+ * For 1 SGE or inline data <= 16, shift = 0 (wqe size of 32 bytes).
+ * For 2 or 3 SGEs or inline data <= 48, shift = 1 (wqe size of 64 bytes).
+ * Shift of 2 otherwise (wqe size of 128 bytes).
*/
-enum i40iw_status_code i40iw_get_wqe_shift(u32 wqdepth, u8 sge, u8 *shift)
+enum i40iw_status_code i40iw_get_wqe_shift(u32 wqdepth, u32 sge, u32 inline_data, u8 *shift)
{
u32 size;
*shift = 0;
- if (sge > 1)
- *shift = (sge < 4) ? 1 : 2;
+ if (sge > 1 || inline_data > 16)
+ *shift = (sge < 4 && inline_data <= 48) ? 1 : 2;
/* check if wqdepth is multiple of 2 or not */
@@ -968,11 +975,11 @@
if (info->max_rq_frag_cnt > I40IW_MAX_WQ_FRAGMENT_COUNT)
return I40IW_ERR_INVALID_FRAG_COUNT;
- ret_code = i40iw_get_wqe_shift(info->sq_size, info->max_sq_frag_cnt, &sqshift);
+ ret_code = i40iw_get_wqe_shift(info->sq_size, info->max_sq_frag_cnt, info->max_inline_data, &sqshift);
if (ret_code)
return ret_code;
- ret_code = i40iw_get_wqe_shift(info->rq_size, info->max_rq_frag_cnt, &rqshift);
+ ret_code = i40iw_get_wqe_shift(info->rq_size, info->max_rq_frag_cnt, 0, &rqshift);
if (ret_code)
return ret_code;
@@ -1097,12 +1104,9 @@
u64 header, *wqe;
u32 wqe_idx;
- wqe = i40iw_qp_get_next_send_wqe(qp, &wqe_idx, I40IW_QP_WQE_MIN_SIZE);
+ wqe = i40iw_qp_get_next_send_wqe(qp, &wqe_idx, I40IW_QP_WQE_MIN_SIZE, 0, wr_id);
if (!wqe)
return I40IW_ERR_QP_TOOMANY_WRS_POSTED;
-
- qp->sq_wrtrk_array[wqe_idx].wrid = wr_id;
- qp->sq_wrtrk_array[wqe_idx].wr_len = 0;
set_64bit_val(wqe, 0, 0);
set_64bit_val(wqe, 8, 0);
set_64bit_val(wqe, 16, 0);
@@ -1125,7 +1129,7 @@
* @frag_cnt: number of fragments
* @wqe_size: size of sq wqe returned
*/
-enum i40iw_status_code i40iw_fragcnt_to_wqesize_sq(u8 frag_cnt, u8 *wqe_size)
+enum i40iw_status_code i40iw_fragcnt_to_wqesize_sq(u32 frag_cnt, u8 *wqe_size)
{
switch (frag_cnt) {
case 0:
@@ -1156,7 +1160,7 @@
* @frag_cnt: number of fragments
* @wqe_size: size of rq wqe returned
*/
-enum i40iw_status_code i40iw_fragcnt_to_wqesize_rq(u8 frag_cnt, u8 *wqe_size)
+enum i40iw_status_code i40iw_fragcnt_to_wqesize_rq(u32 frag_cnt, u8 *wqe_size)
{
switch (frag_cnt) {
case 0:
diff --git a/drivers/infiniband/hw/i40iw/i40iw_user.h b/drivers/infiniband/hw/i40iw/i40iw_user.h
index 5cd971b..4627646 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_user.h
+++ b/drivers/infiniband/hw/i40iw/i40iw_user.h
@@ -61,7 +61,7 @@
I40IW_MAX_CQ_SIZE = 1048575,
I40IW_MAX_AEQ_ALLOCATE_COUNT = 255,
I40IW_DB_ID_ZERO = 0,
- I40IW_MAX_WQ_FRAGMENT_COUNT = 6,
+ I40IW_MAX_WQ_FRAGMENT_COUNT = 3,
I40IW_MAX_SGE_RD = 1,
I40IW_MAX_OUTBOUND_MESSAGE_SIZE = 2147483647,
I40IW_MAX_INBOUND_MESSAGE_SIZE = 2147483647,
@@ -70,8 +70,8 @@
I40IW_MAX_VF_FPM_ID = 47,
I40IW_MAX_VF_PER_PF = 127,
I40IW_MAX_SQ_PAYLOAD_SIZE = 2145386496,
- I40IW_MAX_INLINE_DATA_SIZE = 112,
- I40IW_MAX_PUSHMODE_INLINE_DATA_SIZE = 112,
+ I40IW_MAX_INLINE_DATA_SIZE = 48,
+ I40IW_MAX_PUSHMODE_INLINE_DATA_SIZE = 48,
I40IW_MAX_IRD_SIZE = 32,
I40IW_QPCTX_ENCD_MAXIRD = 3,
I40IW_MAX_WQ_ENTRIES = 2048,
@@ -102,6 +102,8 @@
#define I40IW_STAG_INDEX_FROM_STAG(stag) (((stag) && 0xFFFFFF00) >> 8)
+#define I40IW_MAX_MR_SIZE 0x10000000000L
+
struct i40iw_qp_uk;
struct i40iw_cq_uk;
struct i40iw_srq_uk;
@@ -198,7 +200,7 @@
struct i40iw_post_send {
i40iw_sgl sg_list;
- u8 num_sges;
+ u32 num_sges;
};
struct i40iw_post_inline_send {
@@ -220,7 +222,7 @@
struct i40iw_rdma_write {
i40iw_sgl lo_sg_list;
- u8 num_lo_sges;
+ u32 num_lo_sges;
struct i40iw_sge rem_addr;
};
@@ -345,7 +347,9 @@
struct i40iw_sq_uk_wr_trk_info {
u64 wrid;
- u64 wr_len;
+ u32 wr_len;
+ u8 wqe_size;
+ u8 reserved[3];
};
struct i40iw_qp_quanta {
@@ -367,6 +371,8 @@
u32 qp_id;
u32 sq_size;
u32 rq_size;
+ u32 max_sq_frag_cnt;
+ u32 max_rq_frag_cnt;
struct i40iw_qp_uk_ops ops;
bool use_srq;
u8 swqe_polarity;
@@ -374,8 +380,6 @@
u8 rwqe_polarity;
u8 rq_wqe_size;
u8 rq_wqe_size_multiplier;
- u8 max_sq_frag_cnt;
- u8 max_rq_frag_cnt;
bool deferred_flag;
};
@@ -404,8 +408,9 @@
u32 qp_id;
u32 sq_size;
u32 rq_size;
- u8 max_sq_frag_cnt;
- u8 max_rq_frag_cnt;
+ u32 max_sq_frag_cnt;
+ u32 max_rq_frag_cnt;
+ u32 max_inline_data;
};
@@ -422,7 +427,10 @@
void i40iw_qp_post_wr(struct i40iw_qp_uk *qp);
u64 *i40iw_qp_get_next_send_wqe(struct i40iw_qp_uk *qp, u32 *wqe_idx,
- u8 wqe_size);
+ u8 wqe_size,
+ u32 total_size,
+ u64 wr_id
+ );
u64 *i40iw_qp_get_next_recv_wqe(struct i40iw_qp_uk *qp, u32 *wqe_idx);
u64 *i40iw_qp_get_next_srq_wqe(struct i40iw_srq_uk *srq, u32 *wqe_idx);
@@ -434,9 +442,9 @@
void i40iw_clean_cq(void *queue, struct i40iw_cq_uk *cq);
enum i40iw_status_code i40iw_nop(struct i40iw_qp_uk *qp, u64 wr_id,
bool signaled, bool post_sq);
-enum i40iw_status_code i40iw_fragcnt_to_wqesize_sq(u8 frag_cnt, u8 *wqe_size);
-enum i40iw_status_code i40iw_fragcnt_to_wqesize_rq(u8 frag_cnt, u8 *wqe_size);
+enum i40iw_status_code i40iw_fragcnt_to_wqesize_sq(u32 frag_cnt, u8 *wqe_size);
+enum i40iw_status_code i40iw_fragcnt_to_wqesize_rq(u32 frag_cnt, u8 *wqe_size);
enum i40iw_status_code i40iw_inline_data_size_to_wqesize(u32 data_size,
u8 *wqe_size);
-enum i40iw_status_code i40iw_get_wqe_shift(u32 wqdepth, u8 sge, u8 *shift);
+enum i40iw_status_code i40iw_get_wqe_shift(u32 wqdepth, u32 sge, u32 inline_data, u8 *shift);
#endif
diff --git a/drivers/infiniband/hw/i40iw/i40iw_utils.c b/drivers/infiniband/hw/i40iw/i40iw_utils.c
index 1ceec81..0e8db0a 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_utils.c
+++ b/drivers/infiniband/hw/i40iw/i40iw_utils.c
@@ -59,7 +59,7 @@
* @action: modify, delete or add
*/
int i40iw_arp_table(struct i40iw_device *iwdev,
- __be32 *ip_addr,
+ u32 *ip_addr,
bool ipv4,
u8 *mac_addr,
u32 action)
@@ -152,7 +152,7 @@
struct net_device *upper_dev;
struct i40iw_device *iwdev;
struct i40iw_handler *hdl;
- __be32 local_ipaddr;
+ u32 local_ipaddr;
hdl = i40iw_find_netdev(event_netdev);
if (!hdl)
@@ -167,11 +167,10 @@
switch (event) {
case NETDEV_DOWN:
if (upper_dev)
- local_ipaddr =
- ((struct in_device *)upper_dev->ip_ptr)->ifa_list->ifa_address;
+ local_ipaddr = ntohl(
+ ((struct in_device *)upper_dev->ip_ptr)->ifa_list->ifa_address);
else
- local_ipaddr = ifa->ifa_address;
- local_ipaddr = ntohl(local_ipaddr);
+ local_ipaddr = ntohl(ifa->ifa_address);
i40iw_manage_arp_cache(iwdev,
netdev->dev_addr,
&local_ipaddr,
@@ -180,11 +179,10 @@
return NOTIFY_OK;
case NETDEV_UP:
if (upper_dev)
- local_ipaddr =
- ((struct in_device *)upper_dev->ip_ptr)->ifa_list->ifa_address;
+ local_ipaddr = ntohl(
+ ((struct in_device *)upper_dev->ip_ptr)->ifa_list->ifa_address);
else
- local_ipaddr = ifa->ifa_address;
- local_ipaddr = ntohl(local_ipaddr);
+ local_ipaddr = ntohl(ifa->ifa_address);
i40iw_manage_arp_cache(iwdev,
netdev->dev_addr,
&local_ipaddr,
@@ -194,12 +192,11 @@
case NETDEV_CHANGEADDR:
/* Add the address to the IP table */
if (upper_dev)
- local_ipaddr =
- ((struct in_device *)upper_dev->ip_ptr)->ifa_list->ifa_address;
+ local_ipaddr = ntohl(
+ ((struct in_device *)upper_dev->ip_ptr)->ifa_list->ifa_address);
else
- local_ipaddr = ifa->ifa_address;
+ local_ipaddr = ntohl(ifa->ifa_address);
- local_ipaddr = ntohl(local_ipaddr);
i40iw_manage_arp_cache(iwdev,
netdev->dev_addr,
&local_ipaddr,
@@ -227,7 +224,7 @@
struct net_device *netdev;
struct i40iw_device *iwdev;
struct i40iw_handler *hdl;
- __be32 local_ipaddr6[4];
+ u32 local_ipaddr6[4];
hdl = i40iw_find_netdev(event_netdev);
if (!hdl)
@@ -506,14 +503,19 @@
struct cqp_commands_info *cqp_info;
struct i40iw_device *iwdev;
u32 qp_num;
+ unsigned long flags;
iwqp = to_iwqp(ibqp);
- if (!atomic_dec_and_test(&iwqp->refcount))
- return;
-
iwdev = iwqp->iwdev;
+ spin_lock_irqsave(&iwdev->qptable_lock, flags);
+ if (!atomic_dec_and_test(&iwqp->refcount)) {
+ spin_unlock_irqrestore(&iwdev->qptable_lock, flags);
+ return;
+ }
+
qp_num = iwqp->ibqp.qp_num;
iwdev->qp_table[qp_num] = NULL;
+ spin_unlock_irqrestore(&iwdev->qptable_lock, flags);
cqp_request = i40iw_get_cqp_request(&iwdev->cqp, false);
if (!cqp_request)
return;
@@ -985,21 +987,24 @@
enum i40iw_status_code i40iw_vf_wait_vchnl_resp(struct i40iw_sc_dev *dev)
{
struct i40iw_device *iwdev = dev->back_dev;
- enum i40iw_status_code err_code = 0;
int timeout_ret;
i40iw_debug(dev, I40IW_DEBUG_VIRT, "%s[%u] dev %p, iwdev %p\n",
__func__, __LINE__, dev, iwdev);
- atomic_add(2, &iwdev->vchnl_msgs);
+
+ atomic_set(&iwdev->vchnl_msgs, 2);
timeout_ret = wait_event_timeout(iwdev->vchnl_waitq,
(atomic_read(&iwdev->vchnl_msgs) == 1),
I40IW_VCHNL_EVENT_TIMEOUT);
atomic_dec(&iwdev->vchnl_msgs);
if (!timeout_ret) {
i40iw_pr_err("virt channel completion timeout = 0x%x\n", timeout_ret);
- err_code = I40IW_ERR_TIMEOUT;
+ atomic_set(&iwdev->vchnl_msgs, 0);
+ dev->vchnl_up = false;
+ return I40IW_ERR_TIMEOUT;
}
- return err_code;
+ wake_up(&dev->vf_reqs);
+ return 0;
}
/**
diff --git a/drivers/infiniband/hw/i40iw/i40iw_verbs.c b/drivers/infiniband/hw/i40iw/i40iw_verbs.c
index 1fe3b84..4a740f7 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_verbs.c
+++ b/drivers/infiniband/hw/i40iw/i40iw_verbs.c
@@ -63,8 +63,8 @@
ether_addr_copy((u8 *)&props->sys_image_guid, iwdev->netdev->dev_addr);
props->fw_ver = I40IW_FW_VERSION;
props->device_cap_flags = iwdev->device_cap_flags;
- props->vendor_id = iwdev->vendor_id;
- props->vendor_part_id = iwdev->vendor_part_id;
+ props->vendor_id = iwdev->ldev->pcidev->vendor;
+ props->vendor_part_id = iwdev->ldev->pcidev->device;
props->hw_ver = (u32)iwdev->sc_dev.hw_rev;
props->max_mr_size = I40IW_MAX_OUTBOUND_MESSAGE_SIZE;
props->max_qp = iwdev->max_qp;
@@ -74,7 +74,7 @@
props->max_cqe = iwdev->max_cqe;
props->max_mr = iwdev->max_mr;
props->max_pd = iwdev->max_pd;
- props->max_sge_rd = 1;
+ props->max_sge_rd = I40IW_MAX_SGE_RD;
props->max_qp_rd_atom = I40IW_MAX_IRD_SIZE;
props->max_qp_init_rd_atom = props->max_qp_rd_atom;
props->atomic_cap = IB_ATOMIC_NONE;
@@ -120,7 +120,7 @@
props->pkey_tbl_len = 1;
props->active_width = IB_WIDTH_4X;
props->active_speed = 1;
- props->max_msg_sz = 0x80000000;
+ props->max_msg_sz = I40IW_MAX_OUTBOUND_MESSAGE_SIZE;
return 0;
}
@@ -437,7 +437,6 @@
kfree(iwqp->kqp.wrid_mem);
iwqp->kqp.wrid_mem = NULL;
kfree(iwqp->allocated_buffer);
- iwqp->allocated_buffer = NULL;
}
/**
@@ -521,14 +520,12 @@
enum i40iw_status_code status;
struct i40iw_qp_uk_init_info *ukinfo = &info->qp_uk_init_info;
- ukinfo->max_sq_frag_cnt = I40IW_MAX_WQ_FRAGMENT_COUNT;
-
sq_size = i40iw_qp_roundup(ukinfo->sq_size + 1);
rq_size = i40iw_qp_roundup(ukinfo->rq_size + 1);
- status = i40iw_get_wqe_shift(sq_size, ukinfo->max_sq_frag_cnt, &sqshift);
+ status = i40iw_get_wqe_shift(sq_size, ukinfo->max_sq_frag_cnt, ukinfo->max_inline_data, &sqshift);
if (!status)
- status = i40iw_get_wqe_shift(rq_size, ukinfo->max_rq_frag_cnt, &rqshift);
+ status = i40iw_get_wqe_shift(rq_size, ukinfo->max_rq_frag_cnt, 0, &rqshift);
if (status)
return -ENOSYS;
@@ -609,6 +606,9 @@
if (init_attr->cap.max_inline_data > I40IW_MAX_INLINE_DATA_SIZE)
init_attr->cap.max_inline_data = I40IW_MAX_INLINE_DATA_SIZE;
+ if (init_attr->cap.max_send_sge > I40IW_MAX_WQ_FRAGMENT_COUNT)
+ init_attr->cap.max_send_sge = I40IW_MAX_WQ_FRAGMENT_COUNT;
+
memset(&init_info, 0, sizeof(init_info));
sq_size = init_attr->cap.max_send_wr;
@@ -618,6 +618,7 @@
init_info.qp_uk_init_info.rq_size = rq_size;
init_info.qp_uk_init_info.max_sq_frag_cnt = init_attr->cap.max_send_sge;
init_info.qp_uk_init_info.max_rq_frag_cnt = init_attr->cap.max_recv_sge;
+ init_info.qp_uk_init_info.max_inline_data = init_attr->cap.max_inline_data;
mem = kzalloc(sizeof(*iwqp), GFP_KERNEL);
if (!mem)
@@ -722,8 +723,10 @@
iwarp_info = &iwqp->iwarp_info;
iwarp_info->rd_enable = true;
iwarp_info->wr_rdresp_en = true;
- if (!iwqp->user_mode)
+ if (!iwqp->user_mode) {
+ iwarp_info->fast_reg_en = true;
iwarp_info->priv_mode_en = true;
+ }
iwarp_info->ddp_ver = 1;
iwarp_info->rdmap_ver = 1;
@@ -784,6 +787,8 @@
return ERR_PTR(err_code);
}
}
+ init_completion(&iwqp->sq_drained);
+ init_completion(&iwqp->rq_drained);
return &iwqp->ibqp;
error:
@@ -1444,6 +1449,166 @@
}
/**
+ * i40iw_hw_alloc_stag - cqp command to allocate stag
+ * @iwdev: iwarp device
+ * @iwmr: iwarp mr pointer
+ */
+static int i40iw_hw_alloc_stag(struct i40iw_device *iwdev, struct i40iw_mr *iwmr)
+{
+ struct i40iw_allocate_stag_info *info;
+ struct i40iw_pd *iwpd = to_iwpd(iwmr->ibmr.pd);
+ enum i40iw_status_code status;
+ int err = 0;
+ struct i40iw_cqp_request *cqp_request;
+ struct cqp_commands_info *cqp_info;
+
+ cqp_request = i40iw_get_cqp_request(&iwdev->cqp, true);
+ if (!cqp_request)
+ return -ENOMEM;
+
+ cqp_info = &cqp_request->info;
+ info = &cqp_info->in.u.alloc_stag.info;
+ memset(info, 0, sizeof(*info));
+ info->page_size = PAGE_SIZE;
+ info->stag_idx = iwmr->stag >> I40IW_CQPSQ_STAG_IDX_SHIFT;
+ info->pd_id = iwpd->sc_pd.pd_id;
+ info->total_len = iwmr->length;
+ cqp_info->cqp_cmd = OP_ALLOC_STAG;
+ cqp_info->post_sq = 1;
+ cqp_info->in.u.alloc_stag.dev = &iwdev->sc_dev;
+ cqp_info->in.u.alloc_stag.scratch = (uintptr_t)cqp_request;
+
+ status = i40iw_handle_cqp_op(iwdev, cqp_request);
+ if (status) {
+ err = -ENOMEM;
+ i40iw_pr_err("CQP-OP MR Reg fail");
+ }
+ return err;
+}
+
+/**
+ * i40iw_alloc_mr - register stag for fast memory registration
+ * @pd: ibpd pointer
+ * @mr_type: memory for stag registrion
+ * @max_num_sg: man number of pages
+ */
+static struct ib_mr *i40iw_alloc_mr(struct ib_pd *pd,
+ enum ib_mr_type mr_type,
+ u32 max_num_sg)
+{
+ struct i40iw_pd *iwpd = to_iwpd(pd);
+ struct i40iw_device *iwdev = to_iwdev(pd->device);
+ struct i40iw_pble_alloc *palloc;
+ struct i40iw_pbl *iwpbl;
+ struct i40iw_mr *iwmr;
+ enum i40iw_status_code status;
+ u32 stag;
+ int err_code = -ENOMEM;
+
+ iwmr = kzalloc(sizeof(*iwmr), GFP_KERNEL);
+ if (!iwmr)
+ return ERR_PTR(-ENOMEM);
+
+ stag = i40iw_create_stag(iwdev);
+ if (!stag) {
+ err_code = -EOVERFLOW;
+ goto err;
+ }
+ iwmr->stag = stag;
+ iwmr->ibmr.rkey = stag;
+ iwmr->ibmr.lkey = stag;
+ iwmr->ibmr.pd = pd;
+ iwmr->ibmr.device = pd->device;
+ iwpbl = &iwmr->iwpbl;
+ iwpbl->iwmr = iwmr;
+ iwmr->type = IW_MEMREG_TYPE_MEM;
+ palloc = &iwpbl->pble_alloc;
+ iwmr->page_cnt = max_num_sg;
+ mutex_lock(&iwdev->pbl_mutex);
+ status = i40iw_get_pble(&iwdev->sc_dev, iwdev->pble_rsrc, palloc, iwmr->page_cnt);
+ mutex_unlock(&iwdev->pbl_mutex);
+ if (!status)
+ goto err1;
+
+ if (palloc->level != I40IW_LEVEL_1)
+ goto err2;
+ err_code = i40iw_hw_alloc_stag(iwdev, iwmr);
+ if (err_code)
+ goto err2;
+ iwpbl->pbl_allocated = true;
+ i40iw_add_pdusecount(iwpd);
+ return &iwmr->ibmr;
+err2:
+ i40iw_free_pble(iwdev->pble_rsrc, palloc);
+err1:
+ i40iw_free_stag(iwdev, stag);
+err:
+ kfree(iwmr);
+ return ERR_PTR(err_code);
+}
+
+/**
+ * i40iw_set_page - populate pbl list for fmr
+ * @ibmr: ib mem to access iwarp mr pointer
+ * @addr: page dma address fro pbl list
+ */
+static int i40iw_set_page(struct ib_mr *ibmr, u64 addr)
+{
+ struct i40iw_mr *iwmr = to_iwmr(ibmr);
+ struct i40iw_pbl *iwpbl = &iwmr->iwpbl;
+ struct i40iw_pble_alloc *palloc = &iwpbl->pble_alloc;
+ u64 *pbl;
+
+ if (unlikely(iwmr->npages == iwmr->page_cnt))
+ return -ENOMEM;
+
+ pbl = (u64 *)palloc->level1.addr;
+ pbl[iwmr->npages++] = cpu_to_le64(addr);
+ return 0;
+}
+
+/**
+ * i40iw_map_mr_sg - map of sg list for fmr
+ * @ibmr: ib mem to access iwarp mr pointer
+ * @sg: scatter gather list for fmr
+ * @sg_nents: number of sg pages
+ */
+static int i40iw_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg,
+ int sg_nents, unsigned int *sg_offset)
+{
+ struct i40iw_mr *iwmr = to_iwmr(ibmr);
+
+ iwmr->npages = 0;
+ return ib_sg_to_pages(ibmr, sg, sg_nents, sg_offset, i40iw_set_page);
+}
+
+/**
+ * i40iw_drain_sq - drain the send queue
+ * @ibqp: ib qp pointer
+ */
+static void i40iw_drain_sq(struct ib_qp *ibqp)
+{
+ struct i40iw_qp *iwqp = to_iwqp(ibqp);
+ struct i40iw_sc_qp *qp = &iwqp->sc_qp;
+
+ if (I40IW_RING_MORE_WORK(qp->qp_uk.sq_ring))
+ wait_for_completion(&iwqp->sq_drained);
+}
+
+/**
+ * i40iw_drain_rq - drain the receive queue
+ * @ibqp: ib qp pointer
+ */
+static void i40iw_drain_rq(struct ib_qp *ibqp)
+{
+ struct i40iw_qp *iwqp = to_iwqp(ibqp);
+ struct i40iw_sc_qp *qp = &iwqp->sc_qp;
+
+ if (I40IW_RING_MORE_WORK(qp->qp_uk.rq_ring))
+ wait_for_completion(&iwqp->rq_drained);
+}
+
+/**
* i40iw_hwreg_mr - send cqp command for memory registration
* @iwdev: iwarp device
* @iwmr: iwarp mr pointer
@@ -1526,14 +1691,16 @@
struct i40iw_mr *iwmr;
struct ib_umem *region;
struct i40iw_mem_reg_req req;
- u32 pbl_depth = 0;
+ u64 pbl_depth = 0;
u32 stag = 0;
u16 access;
- u32 region_length;
+ u64 region_length;
bool use_pbles = false;
unsigned long flags;
int err = -ENOSYS;
+ if (length > I40IW_MAX_MR_SIZE)
+ return ERR_PTR(-EINVAL);
region = ib_umem_get(pd->uobject->context, start, length, acc, 0);
if (IS_ERR(region))
return (struct ib_mr *)region;
@@ -1564,7 +1731,7 @@
palloc = &iwpbl->pble_alloc;
iwmr->type = req.reg_type;
- iwmr->page_cnt = pbl_depth;
+ iwmr->page_cnt = (u32)pbl_depth;
switch (req.reg_type) {
case IW_MEMREG_TYPE_QP:
@@ -1881,12 +2048,14 @@
enum i40iw_status_code ret;
int err = 0;
unsigned long flags;
+ bool inv_stag;
iwqp = (struct i40iw_qp *)ibqp;
ukqp = &iwqp->sc_qp.qp_uk;
spin_lock_irqsave(&iwqp->lock, flags);
while (ib_wr) {
+ inv_stag = false;
memset(&info, 0, sizeof(info));
info.wr_id = (u64)(ib_wr->wr_id);
if ((ib_wr->send_flags & IB_SEND_SIGNALED) || iwqp->sig_all)
@@ -1896,19 +2065,28 @@
switch (ib_wr->opcode) {
case IB_WR_SEND:
- if (ib_wr->send_flags & IB_SEND_SOLICITED)
- info.op_type = I40IW_OP_TYPE_SEND_SOL;
- else
- info.op_type = I40IW_OP_TYPE_SEND;
+ /* fall-through */
+ case IB_WR_SEND_WITH_INV:
+ if (ib_wr->opcode == IB_WR_SEND) {
+ if (ib_wr->send_flags & IB_SEND_SOLICITED)
+ info.op_type = I40IW_OP_TYPE_SEND_SOL;
+ else
+ info.op_type = I40IW_OP_TYPE_SEND;
+ } else {
+ if (ib_wr->send_flags & IB_SEND_SOLICITED)
+ info.op_type = I40IW_OP_TYPE_SEND_SOL_INV;
+ else
+ info.op_type = I40IW_OP_TYPE_SEND_INV;
+ }
if (ib_wr->send_flags & IB_SEND_INLINE) {
info.op.inline_send.data = (void *)(unsigned long)ib_wr->sg_list[0].addr;
info.op.inline_send.len = ib_wr->sg_list[0].length;
- ret = ukqp->ops.iw_inline_send(ukqp, &info, rdma_wr(ib_wr)->rkey, false);
+ ret = ukqp->ops.iw_inline_send(ukqp, &info, ib_wr->ex.invalidate_rkey, false);
} else {
info.op.send.num_sges = ib_wr->num_sge;
info.op.send.sg_list = (struct i40iw_sge *)ib_wr->sg_list;
- ret = ukqp->ops.iw_send(ukqp, &info, rdma_wr(ib_wr)->rkey, false);
+ ret = ukqp->ops.iw_send(ukqp, &info, ib_wr->ex.invalidate_rkey, false);
}
if (ret)
@@ -1936,7 +2114,14 @@
if (ret)
err = -EIO;
break;
+ case IB_WR_RDMA_READ_WITH_INV:
+ inv_stag = true;
+ /* fall-through*/
case IB_WR_RDMA_READ:
+ if (ib_wr->num_sge > I40IW_MAX_SGE_RD) {
+ err = -EINVAL;
+ break;
+ }
info.op_type = I40IW_OP_TYPE_RDMA_READ;
info.op.rdma_read.rem_addr.tag_off = rdma_wr(ib_wr)->remote_addr;
info.op.rdma_read.rem_addr.stag = rdma_wr(ib_wr)->rkey;
@@ -1944,10 +2129,47 @@
info.op.rdma_read.lo_addr.tag_off = ib_wr->sg_list->addr;
info.op.rdma_read.lo_addr.stag = ib_wr->sg_list->lkey;
info.op.rdma_read.lo_addr.len = ib_wr->sg_list->length;
- ret = ukqp->ops.iw_rdma_read(ukqp, &info, false, false);
+ ret = ukqp->ops.iw_rdma_read(ukqp, &info, inv_stag, false);
if (ret)
err = -EIO;
break;
+ case IB_WR_LOCAL_INV:
+ info.op_type = I40IW_OP_TYPE_INV_STAG;
+ info.op.inv_local_stag.target_stag = ib_wr->ex.invalidate_rkey;
+ ret = ukqp->ops.iw_stag_local_invalidate(ukqp, &info, true);
+ if (ret)
+ err = -EIO;
+ break;
+ case IB_WR_REG_MR:
+ {
+ struct i40iw_mr *iwmr = to_iwmr(reg_wr(ib_wr)->mr);
+ int page_shift = ilog2(reg_wr(ib_wr)->mr->page_size);
+ int flags = reg_wr(ib_wr)->access;
+ struct i40iw_pble_alloc *palloc = &iwmr->iwpbl.pble_alloc;
+ struct i40iw_sc_dev *dev = &iwqp->iwdev->sc_dev;
+ struct i40iw_fast_reg_stag_info info;
+
+ info.access_rights = I40IW_ACCESS_FLAGS_LOCALREAD;
+ info.access_rights |= i40iw_get_user_access(flags);
+ info.stag_key = reg_wr(ib_wr)->key & 0xff;
+ info.stag_idx = reg_wr(ib_wr)->key >> 8;
+ info.wr_id = ib_wr->wr_id;
+
+ info.addr_type = I40IW_ADDR_TYPE_VA_BASED;
+ info.va = (void *)(uintptr_t)iwmr->ibmr.iova;
+ info.total_len = iwmr->ibmr.length;
+ info.first_pm_pbl_index = palloc->level1.idx;
+ info.local_fence = ib_wr->send_flags & IB_SEND_FENCE;
+ info.signaled = ib_wr->send_flags & IB_SEND_SIGNALED;
+
+ if (page_shift == 21)
+ info.page_size = 1; /* 2M page */
+
+ ret = dev->iw_priv_qp_ops->iw_mr_fast_register(&iwqp->sc_qp, &info, true);
+ if (ret)
+ err = -EIO;
+ break;
+ }
default:
err = -EINVAL;
i40iw_pr_err(" upost_send bad opcode = 0x%x\n",
@@ -2027,6 +2249,7 @@
enum i40iw_status_code ret;
struct i40iw_cq_uk *ukcq;
struct i40iw_sc_qp *qp;
+ struct i40iw_qp *iwqp;
unsigned long flags;
iwcq = (struct i40iw_cq *)ibcq;
@@ -2037,6 +2260,8 @@
ret = ukcq->ops.iw_cq_poll_completion(ukcq, &cq_poll_info, true);
if (ret == I40IW_ERR_QUEUE_EMPTY) {
break;
+ } else if (ret == I40IW_ERR_QUEUE_DESTROYED) {
+ continue;
} else if (ret) {
if (!cqe_count)
cqe_count = -1;
@@ -2044,10 +2269,12 @@
}
entry->wc_flags = 0;
entry->wr_id = cq_poll_info.wr_id;
- if (!cq_poll_info.error)
- entry->status = IB_WC_SUCCESS;
- else
+ if (cq_poll_info.error) {
entry->status = IB_WC_WR_FLUSH_ERR;
+ entry->vendor_err = cq_poll_info.major_err << 16 | cq_poll_info.minor_err;
+ } else {
+ entry->status = IB_WC_SUCCESS;
+ }
switch (cq_poll_info.op_type) {
case I40IW_OP_TYPE_RDMA_WRITE:
@@ -2071,12 +2298,17 @@
break;
}
- entry->vendor_err =
- cq_poll_info.major_err << 16 | cq_poll_info.minor_err;
entry->ex.imm_data = 0;
qp = (struct i40iw_sc_qp *)cq_poll_info.qp_handle;
entry->qp = (struct ib_qp *)qp->back_qp;
entry->src_qp = cq_poll_info.qp_id;
+ iwqp = (struct i40iw_qp *)qp->back_qp;
+ if (iwqp->iwarp_state > I40IW_QP_STATE_RTS) {
+ if (!I40IW_RING_MORE_WORK(qp->qp_uk.sq_ring))
+ complete(&iwqp->sq_drained);
+ if (!I40IW_RING_MORE_WORK(qp->qp_uk.rq_ring))
+ complete(&iwqp->rq_drained);
+ }
entry->byte_len = cq_poll_info.bytes_xfered;
entry++;
cqe_count++;
@@ -2143,7 +2375,6 @@
struct i40iw_dev_hw_stats *hw_stats = &devstat->hw_stats;
struct timespec curr_time;
static struct timespec last_rd_time = {0, 0};
- enum i40iw_status_code status = 0;
unsigned long flags;
curr_time = current_kernel_time();
@@ -2156,11 +2387,8 @@
spin_unlock_irqrestore(&devstat->stats_lock, flags);
} else {
if (((u64)curr_time.tv_sec - (u64)last_rd_time.tv_sec) > 1)
- status = i40iw_vchnl_vf_get_pe_stats(dev,
- &devstat->hw_stats);
-
- if (status)
- return -ENOSYS;
+ if (i40iw_vchnl_vf_get_pe_stats(dev, &devstat->hw_stats))
+ return -ENOSYS;
}
stats->iw.ipInReceives = hw_stats->stat_value_64[I40IW_HW_STAT_INDEX_IP4RXPKTS] +
@@ -2327,6 +2555,10 @@
iwibdev->ibdev.query_device = i40iw_query_device;
iwibdev->ibdev.create_ah = i40iw_create_ah;
iwibdev->ibdev.destroy_ah = i40iw_destroy_ah;
+ iwibdev->ibdev.drain_sq = i40iw_drain_sq;
+ iwibdev->ibdev.drain_rq = i40iw_drain_rq;
+ iwibdev->ibdev.alloc_mr = i40iw_alloc_mr;
+ iwibdev->ibdev.map_mr_sg = i40iw_map_mr_sg;
iwibdev->ibdev.iwcm = kzalloc(sizeof(*iwibdev->ibdev.iwcm), GFP_KERNEL);
if (!iwibdev->ibdev.iwcm) {
ib_dealloc_device(&iwibdev->ibdev);
diff --git a/drivers/infiniband/hw/i40iw/i40iw_verbs.h b/drivers/infiniband/hw/i40iw/i40iw_verbs.h
index 1101f77..0069be8 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_verbs.h
+++ b/drivers/infiniband/hw/i40iw/i40iw_verbs.h
@@ -92,6 +92,7 @@
struct ib_umem *region;
u16 type;
u32 page_cnt;
+ u32 npages;
u32 stag;
u64 length;
u64 pgaddrmem[MAX_SAVE_PAGE_ADDRS];
@@ -169,5 +170,7 @@
struct i40iw_pbl *iwpbl;
struct i40iw_dma_mem q2_ctx_mem;
struct i40iw_dma_mem ietf_mem;
+ struct completion sq_drained;
+ struct completion rq_drained;
};
#endif
diff --git a/drivers/infiniband/hw/i40iw/i40iw_vf.c b/drivers/infiniband/hw/i40iw/i40iw_vf.c
index cb0f183..e33d481 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_vf.c
+++ b/drivers/infiniband/hw/i40iw/i40iw_vf.c
@@ -80,6 +80,6 @@
return 0;
}
-struct i40iw_vf_cqp_ops iw_vf_cqp_ops = {
+const struct i40iw_vf_cqp_ops iw_vf_cqp_ops = {
i40iw_manage_vf_pble_bp
};
diff --git a/drivers/infiniband/hw/i40iw/i40iw_vf.h b/drivers/infiniband/hw/i40iw/i40iw_vf.h
index f649f3a..4359559 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_vf.h
+++ b/drivers/infiniband/hw/i40iw/i40iw_vf.h
@@ -57,6 +57,6 @@
u64 scratch,
bool post_sq);
-extern struct i40iw_vf_cqp_ops iw_vf_cqp_ops;
+extern const struct i40iw_vf_cqp_ops iw_vf_cqp_ops;
#endif
diff --git a/drivers/infiniband/hw/i40iw/i40iw_virtchnl.c b/drivers/infiniband/hw/i40iw/i40iw_virtchnl.c
index 6b68f78..3041003 100644
--- a/drivers/infiniband/hw/i40iw/i40iw_virtchnl.c
+++ b/drivers/infiniband/hw/i40iw/i40iw_virtchnl.c
@@ -254,7 +254,7 @@
static void vchnl_pf_send_get_pe_stats_resp(struct i40iw_sc_dev *dev,
u32 vf_id,
struct i40iw_virtchnl_op_buf *vchnl_msg,
- struct i40iw_dev_hw_stats hw_stats)
+ struct i40iw_dev_hw_stats *hw_stats)
{
enum i40iw_status_code ret_code;
u8 resp_buffer[sizeof(struct i40iw_virtchnl_resp_buf) + sizeof(struct i40iw_dev_hw_stats) - 1];
@@ -264,7 +264,7 @@
vchnl_msg_resp->iw_chnl_op_ctx = vchnl_msg->iw_chnl_op_ctx;
vchnl_msg_resp->iw_chnl_buf_len = sizeof(resp_buffer);
vchnl_msg_resp->iw_op_ret_code = I40IW_SUCCESS;
- *((struct i40iw_dev_hw_stats *)vchnl_msg_resp->iw_chnl_buf) = hw_stats;
+ *((struct i40iw_dev_hw_stats *)vchnl_msg_resp->iw_chnl_buf) = *hw_stats;
ret_code = dev->vchnl_if.vchnl_send(dev, vf_id, resp_buffer, sizeof(resp_buffer));
if (ret_code)
i40iw_debug(dev, I40IW_DEBUG_VIRT,
@@ -437,11 +437,9 @@
vchnl_pf_send_get_ver_resp(dev, vf_id, vchnl_msg);
return I40IW_SUCCESS;
}
- for (iw_vf_idx = 0; iw_vf_idx < I40IW_MAX_PE_ENABLED_VF_COUNT;
- iw_vf_idx++) {
+ for (iw_vf_idx = 0; iw_vf_idx < I40IW_MAX_PE_ENABLED_VF_COUNT; iw_vf_idx++) {
if (!dev->vf_dev[iw_vf_idx]) {
- if (first_avail_iw_vf ==
- I40IW_MAX_PE_ENABLED_VF_COUNT)
+ if (first_avail_iw_vf == I40IW_MAX_PE_ENABLED_VF_COUNT)
first_avail_iw_vf = iw_vf_idx;
continue;
}
@@ -541,7 +539,7 @@
devstat->ops.iw_hw_stat_read_all(devstat, &devstat->hw_stats);
spin_unlock_irqrestore(&dev->dev_pestat.stats_lock, flags);
vf_dev->msg_count--;
- vchnl_pf_send_get_pe_stats_resp(dev, vf_id, vchnl_msg, devstat->hw_stats);
+ vchnl_pf_send_get_pe_stats_resp(dev, vf_id, vchnl_msg, &devstat->hw_stats);
break;
default:
i40iw_debug(dev, I40IW_DEBUG_VIRT,
@@ -596,23 +594,25 @@
struct i40iw_virtchnl_req vchnl_req;
enum i40iw_status_code ret_code;
+ if (!i40iw_vf_clear_to_send(dev))
+ return I40IW_ERR_TIMEOUT;
memset(&vchnl_req, 0, sizeof(vchnl_req));
vchnl_req.dev = dev;
vchnl_req.parm = vchnl_ver;
vchnl_req.parm_len = sizeof(*vchnl_ver);
vchnl_req.vchnl_msg = &dev->vchnl_vf_msg_buf.vchnl_msg;
+
ret_code = vchnl_vf_send_get_ver_req(dev, &vchnl_req);
- if (!ret_code) {
- ret_code = i40iw_vf_wait_vchnl_resp(dev);
- if (!ret_code)
- ret_code = vchnl_req.ret_code;
- else
- dev->vchnl_up = false;
- } else {
+ if (ret_code) {
i40iw_debug(dev, I40IW_DEBUG_VIRT,
"%s Send message failed 0x%0x\n", __func__, ret_code);
+ return ret_code;
}
- return ret_code;
+ ret_code = i40iw_vf_wait_vchnl_resp(dev);
+ if (ret_code)
+ return ret_code;
+ else
+ return vchnl_req.ret_code;
}
/**
@@ -626,23 +626,25 @@
struct i40iw_virtchnl_req vchnl_req;
enum i40iw_status_code ret_code;
+ if (!i40iw_vf_clear_to_send(dev))
+ return I40IW_ERR_TIMEOUT;
memset(&vchnl_req, 0, sizeof(vchnl_req));
vchnl_req.dev = dev;
vchnl_req.parm = hmc_fcn;
vchnl_req.parm_len = sizeof(*hmc_fcn);
vchnl_req.vchnl_msg = &dev->vchnl_vf_msg_buf.vchnl_msg;
+
ret_code = vchnl_vf_send_get_hmc_fcn_req(dev, &vchnl_req);
- if (!ret_code) {
- ret_code = i40iw_vf_wait_vchnl_resp(dev);
- if (!ret_code)
- ret_code = vchnl_req.ret_code;
- else
- dev->vchnl_up = false;
- } else {
+ if (ret_code) {
i40iw_debug(dev, I40IW_DEBUG_VIRT,
"%s Send message failed 0x%0x\n", __func__, ret_code);
+ return ret_code;
}
- return ret_code;
+ ret_code = i40iw_vf_wait_vchnl_resp(dev);
+ if (ret_code)
+ return ret_code;
+ else
+ return vchnl_req.ret_code;
}
/**
@@ -660,25 +662,27 @@
struct i40iw_virtchnl_req vchnl_req;
enum i40iw_status_code ret_code;
+ if (!i40iw_vf_clear_to_send(dev))
+ return I40IW_ERR_TIMEOUT;
memset(&vchnl_req, 0, sizeof(vchnl_req));
vchnl_req.dev = dev;
vchnl_req.vchnl_msg = &dev->vchnl_vf_msg_buf.vchnl_msg;
+
ret_code = vchnl_vf_send_add_hmc_objs_req(dev,
&vchnl_req,
rsrc_type,
start_index,
rsrc_count);
- if (!ret_code) {
- ret_code = i40iw_vf_wait_vchnl_resp(dev);
- if (!ret_code)
- ret_code = vchnl_req.ret_code;
- else
- dev->vchnl_up = false;
- } else {
+ if (ret_code) {
i40iw_debug(dev, I40IW_DEBUG_VIRT,
"%s Send message failed 0x%0x\n", __func__, ret_code);
+ return ret_code;
}
- return ret_code;
+ ret_code = i40iw_vf_wait_vchnl_resp(dev);
+ if (ret_code)
+ return ret_code;
+ else
+ return vchnl_req.ret_code;
}
/**
@@ -696,25 +700,27 @@
struct i40iw_virtchnl_req vchnl_req;
enum i40iw_status_code ret_code;
+ if (!i40iw_vf_clear_to_send(dev))
+ return I40IW_ERR_TIMEOUT;
memset(&vchnl_req, 0, sizeof(vchnl_req));
vchnl_req.dev = dev;
vchnl_req.vchnl_msg = &dev->vchnl_vf_msg_buf.vchnl_msg;
+
ret_code = vchnl_vf_send_del_hmc_objs_req(dev,
&vchnl_req,
rsrc_type,
start_index,
rsrc_count);
- if (!ret_code) {
- ret_code = i40iw_vf_wait_vchnl_resp(dev);
- if (!ret_code)
- ret_code = vchnl_req.ret_code;
- else
- dev->vchnl_up = false;
- } else {
+ if (ret_code) {
i40iw_debug(dev, I40IW_DEBUG_VIRT,
"%s Send message failed 0x%0x\n", __func__, ret_code);
+ return ret_code;
}
- return ret_code;
+ ret_code = i40iw_vf_wait_vchnl_resp(dev);
+ if (ret_code)
+ return ret_code;
+ else
+ return vchnl_req.ret_code;
}
/**
@@ -728,21 +734,23 @@
struct i40iw_virtchnl_req vchnl_req;
enum i40iw_status_code ret_code;
+ if (!i40iw_vf_clear_to_send(dev))
+ return I40IW_ERR_TIMEOUT;
memset(&vchnl_req, 0, sizeof(vchnl_req));
vchnl_req.dev = dev;
vchnl_req.parm = hw_stats;
vchnl_req.parm_len = sizeof(*hw_stats);
vchnl_req.vchnl_msg = &dev->vchnl_vf_msg_buf.vchnl_msg;
+
ret_code = vchnl_vf_send_get_pe_stats_req(dev, &vchnl_req);
- if (!ret_code) {
- ret_code = i40iw_vf_wait_vchnl_resp(dev);
- if (!ret_code)
- ret_code = vchnl_req.ret_code;
- else
- dev->vchnl_up = false;
- } else {
+ if (ret_code) {
i40iw_debug(dev, I40IW_DEBUG_VIRT,
"%s Send message failed 0x%0x\n", __func__, ret_code);
+ return ret_code;
}
- return ret_code;
+ ret_code = i40iw_vf_wait_vchnl_resp(dev);
+ if (ret_code)
+ return ret_code;
+ else
+ return vchnl_req.ret_code;
}
diff --git a/drivers/infiniband/hw/mlx4/mcg.c b/drivers/infiniband/hw/mlx4/mcg.c
index 99451d8..8f7ad07 100644
--- a/drivers/infiniband/hw/mlx4/mcg.c
+++ b/drivers/infiniband/hw/mlx4/mcg.c
@@ -96,7 +96,7 @@
u8 scope_join_state;
u8 proxy_join;
u8 reserved[2];
-};
+} __packed __aligned(4);
struct mcast_group {
struct ib_sa_mcmember_data rec;
@@ -747,14 +747,11 @@
__be64 tid,
union ib_gid *new_mgid)
{
- struct mcast_group *group = NULL, *cur_group;
+ struct mcast_group *group = NULL, *cur_group, *n;
struct mcast_req *req;
- struct list_head *pos;
- struct list_head *n;
mutex_lock(&ctx->mcg_table_lock);
- list_for_each_safe(pos, n, &ctx->mcg_mgid0_list) {
- group = list_entry(pos, struct mcast_group, mgid0_list);
+ list_for_each_entry_safe(group, n, &ctx->mcg_mgid0_list, mgid0_list) {
mutex_lock(&group->lock);
if (group->last_req_tid == tid) {
if (memcmp(new_mgid, &mgid0, sizeof mgid0)) {
diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h
index 1eca01c..6c5ac5d 100644
--- a/drivers/infiniband/hw/mlx4/mlx4_ib.h
+++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h
@@ -717,9 +717,8 @@
struct ib_mr *mlx4_ib_alloc_mr(struct ib_pd *pd,
enum ib_mr_type mr_type,
u32 max_num_sg);
-int mlx4_ib_map_mr_sg(struct ib_mr *ibmr,
- struct scatterlist *sg,
- int sg_nents);
+int mlx4_ib_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, int sg_nents,
+ unsigned int *sg_offset);
int mlx4_ib_modify_cq(struct ib_cq *cq, u16 cq_count, u16 cq_period);
int mlx4_ib_resize_cq(struct ib_cq *ibcq, int entries, struct ib_udata *udata);
struct ib_cq *mlx4_ib_create_cq(struct ib_device *ibdev,
diff --git a/drivers/infiniband/hw/mlx4/mr.c b/drivers/infiniband/hw/mlx4/mr.c
index ce0b5aa..6312721 100644
--- a/drivers/infiniband/hw/mlx4/mr.c
+++ b/drivers/infiniband/hw/mlx4/mr.c
@@ -528,9 +528,8 @@
return 0;
}
-int mlx4_ib_map_mr_sg(struct ib_mr *ibmr,
- struct scatterlist *sg,
- int sg_nents)
+int mlx4_ib_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, int sg_nents,
+ unsigned int *sg_offset)
{
struct mlx4_ib_mr *mr = to_mmr(ibmr);
int rc;
@@ -541,7 +540,7 @@
sizeof(u64) * mr->max_pages,
DMA_TO_DEVICE);
- rc = ib_sg_to_pages(ibmr, sg, sg_nents, mlx4_set_page);
+ rc = ib_sg_to_pages(ibmr, sg, sg_nents, sg_offset, mlx4_set_page);
ib_dma_sync_single_for_device(ibmr->device, mr->page_map,
sizeof(u64) * mr->max_pages,
diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c
index a00ba44..dabcc65 100644
--- a/drivers/infiniband/hw/mlx5/cq.c
+++ b/drivers/infiniband/hw/mlx5/cq.c
@@ -879,7 +879,10 @@
mlx5_ib_dbg(dev, "cqn 0x%x\n", cq->mcq.cqn);
cq->mcq.irqn = irqn;
- cq->mcq.comp = mlx5_ib_cq_comp;
+ if (context)
+ cq->mcq.tasklet_ctx.comp = mlx5_ib_cq_comp;
+ else
+ cq->mcq.comp = mlx5_ib_cq_comp;
cq->mcq.event = mlx5_ib_cq_event;
INIT_LIST_HEAD(&cq->wc_list);
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index 4cb81f6..c72797c 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -38,6 +38,9 @@
#include <linux/dma-mapping.h>
#include <linux/slab.h>
#include <linux/io-mapping.h>
+#if defined(CONFIG_X86)
+#include <asm/pat.h>
+#endif
#include <linux/sched.h>
#include <rdma/ib_user_verbs.h>
#include <rdma/ib_addr.h>
@@ -517,6 +520,10 @@
props->device_cap_flags |= IB_DEVICE_UD_TSO;
}
+ if (MLX5_CAP_GEN(dev->mdev, eth_net_offloads) &&
+ MLX5_CAP_ETH(dev->mdev, scatter_fcs))
+ props->device_cap_flags |= IB_DEVICE_RAW_SCATTER_FCS;
+
props->vendor_part_id = mdev->pdev->device;
props->hw_ver = mdev->pdev->revision;
@@ -1068,38 +1075,89 @@
return get_arg(offset);
}
+static inline char *mmap_cmd2str(enum mlx5_ib_mmap_cmd cmd)
+{
+ switch (cmd) {
+ case MLX5_IB_MMAP_WC_PAGE:
+ return "WC";
+ case MLX5_IB_MMAP_REGULAR_PAGE:
+ return "best effort WC";
+ case MLX5_IB_MMAP_NC_PAGE:
+ return "NC";
+ default:
+ return NULL;
+ }
+}
+
+static int uar_mmap(struct mlx5_ib_dev *dev, enum mlx5_ib_mmap_cmd cmd,
+ struct vm_area_struct *vma, struct mlx5_uuar_info *uuari)
+{
+ int err;
+ unsigned long idx;
+ phys_addr_t pfn, pa;
+ pgprot_t prot;
+
+ switch (cmd) {
+ case MLX5_IB_MMAP_WC_PAGE:
+/* Some architectures don't support WC memory */
+#if defined(CONFIG_X86)
+ if (!pat_enabled())
+ return -EPERM;
+#elif !(defined(CONFIG_PPC) || (defined(CONFIG_ARM) && defined(CONFIG_MMU)))
+ return -EPERM;
+#endif
+ /* fall through */
+ case MLX5_IB_MMAP_REGULAR_PAGE:
+ /* For MLX5_IB_MMAP_REGULAR_PAGE do the best effort to get WC */
+ prot = pgprot_writecombine(vma->vm_page_prot);
+ break;
+ case MLX5_IB_MMAP_NC_PAGE:
+ prot = pgprot_noncached(vma->vm_page_prot);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ if (vma->vm_end - vma->vm_start != PAGE_SIZE)
+ return -EINVAL;
+
+ idx = get_index(vma->vm_pgoff);
+ if (idx >= uuari->num_uars)
+ return -EINVAL;
+
+ pfn = uar_index2pfn(dev, uuari->uars[idx].index);
+ mlx5_ib_dbg(dev, "uar idx 0x%lx, pfn %pa\n", idx, &pfn);
+
+ vma->vm_page_prot = prot;
+ err = io_remap_pfn_range(vma, vma->vm_start, pfn,
+ PAGE_SIZE, vma->vm_page_prot);
+ if (err) {
+ mlx5_ib_err(dev, "io_remap_pfn_range failed with error=%d, vm_start=0x%lx, pfn=%pa, mmap_cmd=%s\n",
+ err, vma->vm_start, &pfn, mmap_cmd2str(cmd));
+ return -EAGAIN;
+ }
+
+ pa = pfn << PAGE_SHIFT;
+ mlx5_ib_dbg(dev, "mapped %s at 0x%lx, PA %pa\n", mmap_cmd2str(cmd),
+ vma->vm_start, &pa);
+
+ return 0;
+}
+
static int mlx5_ib_mmap(struct ib_ucontext *ibcontext, struct vm_area_struct *vma)
{
struct mlx5_ib_ucontext *context = to_mucontext(ibcontext);
struct mlx5_ib_dev *dev = to_mdev(ibcontext->device);
struct mlx5_uuar_info *uuari = &context->uuari;
unsigned long command;
- unsigned long idx;
phys_addr_t pfn;
command = get_command(vma->vm_pgoff);
switch (command) {
+ case MLX5_IB_MMAP_WC_PAGE:
+ case MLX5_IB_MMAP_NC_PAGE:
case MLX5_IB_MMAP_REGULAR_PAGE:
- if (vma->vm_end - vma->vm_start != PAGE_SIZE)
- return -EINVAL;
-
- idx = get_index(vma->vm_pgoff);
- if (idx >= uuari->num_uars)
- return -EINVAL;
-
- pfn = uar_index2pfn(dev, uuari->uars[idx].index);
- mlx5_ib_dbg(dev, "uar idx 0x%lx, pfn 0x%llx\n", idx,
- (unsigned long long)pfn);
-
- vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
- if (io_remap_pfn_range(vma, vma->vm_start, pfn,
- PAGE_SIZE, vma->vm_page_prot))
- return -EAGAIN;
-
- mlx5_ib_dbg(dev, "mapped WC at 0x%lx, PA 0x%llx\n",
- vma->vm_start,
- (unsigned long long)pfn << PAGE_SHIFT);
- break;
+ return uar_mmap(dev, command, vma, uuari);
case MLX5_IB_MMAP_GET_CONTIGUOUS_PAGES:
return -ENOSYS;
@@ -1108,7 +1166,7 @@
if (vma->vm_end - vma->vm_start != PAGE_SIZE)
return -EINVAL;
- if (vma->vm_flags & (VM_WRITE | VM_EXEC))
+ if (vma->vm_flags & VM_WRITE)
return -EPERM;
/* Don't expose to user-space information it shouldn't have */
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index b46c255..c4a9825 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -70,6 +70,8 @@
enum mlx5_ib_mmap_cmd {
MLX5_IB_MMAP_REGULAR_PAGE = 0,
MLX5_IB_MMAP_GET_CONTIGUOUS_PAGES = 1,
+ MLX5_IB_MMAP_WC_PAGE = 2,
+ MLX5_IB_MMAP_NC_PAGE = 3,
/* 5 is chosen in order to be compatible with old versions of libmlx5 */
MLX5_IB_MMAP_CORE_CLOCK = 5,
};
@@ -356,6 +358,7 @@
MLX5_IB_QP_SIGNATURE_HANDLING = 1 << 5,
/* QP uses 1 as its source QP number */
MLX5_IB_QP_SQPN_QP1 = 1 << 6,
+ MLX5_IB_QP_CAP_SCATTER_FCS = 1 << 7,
};
struct mlx5_umr_wr {
@@ -712,9 +715,8 @@
struct ib_mr *mlx5_ib_alloc_mr(struct ib_pd *pd,
enum ib_mr_type mr_type,
u32 max_num_sg);
-int mlx5_ib_map_mr_sg(struct ib_mr *ibmr,
- struct scatterlist *sg,
- int sg_nents);
+int mlx5_ib_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, int sg_nents,
+ unsigned int *sg_offset);
int mlx5_ib_process_mad(struct ib_device *ibdev, int mad_flags, u8 port_num,
const struct ib_wc *in_wc, const struct ib_grh *in_grh,
const struct ib_mad_hdr *in, size_t in_mad_size,
diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
index 4d5bff1..8cf2ce5 100644
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@ -1751,26 +1751,33 @@
static int
mlx5_ib_sg_to_klms(struct mlx5_ib_mr *mr,
struct scatterlist *sgl,
- unsigned short sg_nents)
+ unsigned short sg_nents,
+ unsigned int *sg_offset_p)
{
struct scatterlist *sg = sgl;
struct mlx5_klm *klms = mr->descs;
+ unsigned int sg_offset = sg_offset_p ? *sg_offset_p : 0;
u32 lkey = mr->ibmr.pd->local_dma_lkey;
int i;
- mr->ibmr.iova = sg_dma_address(sg);
+ mr->ibmr.iova = sg_dma_address(sg) + sg_offset;
mr->ibmr.length = 0;
mr->ndescs = sg_nents;
for_each_sg(sgl, sg, sg_nents, i) {
if (unlikely(i > mr->max_descs))
break;
- klms[i].va = cpu_to_be64(sg_dma_address(sg));
- klms[i].bcount = cpu_to_be32(sg_dma_len(sg));
+ klms[i].va = cpu_to_be64(sg_dma_address(sg) + sg_offset);
+ klms[i].bcount = cpu_to_be32(sg_dma_len(sg) - sg_offset);
klms[i].key = cpu_to_be32(lkey);
mr->ibmr.length += sg_dma_len(sg);
+
+ sg_offset = 0;
}
+ if (sg_offset_p)
+ *sg_offset_p = sg_offset;
+
return i;
}
@@ -1788,9 +1795,8 @@
return 0;
}
-int mlx5_ib_map_mr_sg(struct ib_mr *ibmr,
- struct scatterlist *sg,
- int sg_nents)
+int mlx5_ib_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, int sg_nents,
+ unsigned int *sg_offset)
{
struct mlx5_ib_mr *mr = to_mmr(ibmr);
int n;
@@ -1802,9 +1808,10 @@
DMA_TO_DEVICE);
if (mr->access_mode == MLX5_ACCESS_MODE_KLM)
- n = mlx5_ib_sg_to_klms(mr, sg, sg_nents);
+ n = mlx5_ib_sg_to_klms(mr, sg, sg_nents, sg_offset);
else
- n = ib_sg_to_pages(ibmr, sg, sg_nents, mlx5_set_page);
+ n = ib_sg_to_pages(ibmr, sg, sg_nents, sg_offset,
+ mlx5_set_page);
ib_dma_sync_single_for_device(ibmr->device, mr->desc_map,
mr->desc_size * mr->max_descs,
diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index 8dee8bc..5041176 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -1028,6 +1028,7 @@
static int create_raw_packet_qp_rq(struct mlx5_ib_dev *dev,
struct mlx5_ib_rq *rq, void *qpin)
{
+ struct mlx5_ib_qp *mqp = rq->base.container_mibqp;
__be64 *pas;
__be64 *qp_pas;
void *in;
@@ -1051,6 +1052,9 @@
MLX5_SET(rqc, rqc, user_index, MLX5_GET(qpc, qpc, user_index));
MLX5_SET(rqc, rqc, cqn, MLX5_GET(qpc, qpc, cqn_rcv));
+ if (mqp->flags & MLX5_IB_QP_CAP_SCATTER_FCS)
+ MLX5_SET(rqc, rqc, scatter_fcs, 1);
+
wq = MLX5_ADDR_OF(rqc, rqc, wq);
MLX5_SET(wq, wq, wq_type, MLX5_WQ_TYPE_CYCLIC);
MLX5_SET(wq, wq, end_padding_mode,
@@ -1136,11 +1140,12 @@
}
if (qp->rq.wqe_cnt) {
+ rq->base.container_mibqp = qp;
+
err = create_raw_packet_qp_rq(dev, rq, in);
if (err)
goto err_destroy_sq;
- rq->base.container_mibqp = qp;
err = create_raw_packet_qp_tir(dev, rq, tdn);
if (err)
@@ -1252,6 +1257,19 @@
return -EOPNOTSUPP;
}
+ if (init_attr->create_flags & IB_QP_CREATE_SCATTER_FCS) {
+ if (init_attr->qp_type != IB_QPT_RAW_PACKET) {
+ mlx5_ib_dbg(dev, "Scatter FCS is supported only for Raw Packet QPs");
+ return -EOPNOTSUPP;
+ }
+ if (!MLX5_CAP_GEN(dev->mdev, eth_net_offloads) ||
+ !MLX5_CAP_ETH(dev->mdev, scatter_fcs)) {
+ mlx5_ib_dbg(dev, "Scatter FCS isn't supported\n");
+ return -EOPNOTSUPP;
+ }
+ qp->flags |= MLX5_IB_QP_CAP_SCATTER_FCS;
+ }
+
if (init_attr->sq_sig_type == IB_SIGNAL_ALL_WR)
qp->sq_signal_bits = MLX5_WQE_CTRL_CQ_UPDATE;
diff --git a/drivers/infiniband/hw/nes/nes_utils.c b/drivers/infiniband/hw/nes/nes_utils.c
index 6d3a169..37331e2 100644
--- a/drivers/infiniband/hw/nes/nes_utils.c
+++ b/drivers/infiniband/hw/nes/nes_utils.c
@@ -44,6 +44,7 @@
#include <linux/ip.h>
#include <linux/tcp.h>
#include <linux/init.h>
+#include <linux/kernel.h>
#include <asm/io.h>
#include <asm/irq.h>
@@ -903,70 +904,15 @@
*/
void nes_dump_mem(unsigned int dump_debug_level, void *addr, int length)
{
- char xlate[] = {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9',
- 'a', 'b', 'c', 'd', 'e', 'f'};
- char *ptr;
- char hex_buf[80];
- char ascii_buf[20];
- int num_char;
- int num_ascii;
- int num_hex;
-
if (!(nes_debug_level & dump_debug_level)) {
return;
}
- ptr = addr;
if (length > 0x100) {
nes_debug(dump_debug_level, "Length truncated from %x to %x\n", length, 0x100);
length = 0x100;
}
- nes_debug(dump_debug_level, "Address=0x%p, length=0x%x (%d)\n", ptr, length, length);
+ nes_debug(dump_debug_level, "Address=0x%p, length=0x%x (%d)\n", addr, length, length);
- memset(ascii_buf, 0, 20);
- memset(hex_buf, 0, 80);
-
- num_ascii = 0;
- num_hex = 0;
- for (num_char = 0; num_char < length; num_char++) {
- if (num_ascii == 8) {
- ascii_buf[num_ascii++] = ' ';
- hex_buf[num_hex++] = '-';
- hex_buf[num_hex++] = ' ';
- }
-
- if (*ptr < 0x20 || *ptr > 0x7e)
- ascii_buf[num_ascii++] = '.';
- else
- ascii_buf[num_ascii++] = *ptr;
- hex_buf[num_hex++] = xlate[((*ptr & 0xf0) >> 4)];
- hex_buf[num_hex++] = xlate[*ptr & 0x0f];
- hex_buf[num_hex++] = ' ';
- ptr++;
-
- if (num_ascii >= 17) {
- /* output line and reset */
- nes_debug(dump_debug_level, " %s | %s\n", hex_buf, ascii_buf);
- memset(ascii_buf, 0, 20);
- memset(hex_buf, 0, 80);
- num_ascii = 0;
- num_hex = 0;
- }
- }
-
- /* output the rest */
- if (num_ascii) {
- while (num_ascii < 17) {
- if (num_ascii == 8) {
- hex_buf[num_hex++] = ' ';
- hex_buf[num_hex++] = ' ';
- }
- hex_buf[num_hex++] = ' ';
- hex_buf[num_hex++] = ' ';
- hex_buf[num_hex++] = ' ';
- num_ascii++;
- }
-
- nes_debug(dump_debug_level, " %s | %s\n", hex_buf, ascii_buf);
- }
+ print_hex_dump(KERN_ERR, PFX, DUMP_PREFIX_NONE, 16, 1, addr, length, true);
}
diff --git a/drivers/infiniband/hw/nes/nes_verbs.c b/drivers/infiniband/hw/nes/nes_verbs.c
index fba69a3..464d6da 100644
--- a/drivers/infiniband/hw/nes/nes_verbs.c
+++ b/drivers/infiniband/hw/nes/nes_verbs.c
@@ -402,15 +402,14 @@
return 0;
}
-static int nes_map_mr_sg(struct ib_mr *ibmr,
- struct scatterlist *sg,
- int sg_nents)
+static int nes_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg,
+ int sg_nents, unsigned int *sg_offset)
{
struct nes_mr *nesmr = to_nesmr(ibmr);
nesmr->npages = 0;
- return ib_sg_to_pages(ibmr, sg, sg_nents, nes_set_page);
+ return ib_sg_to_pages(ibmr, sg, sg_nents, sg_offset, nes_set_page);
}
/**
@@ -981,7 +980,7 @@
/**
* nes_free_qp_mem() is to free up the qp's pci_alloc_consistent() memory.
*/
-static inline void nes_free_qp_mem(struct nes_device *nesdev,
+static void nes_free_qp_mem(struct nes_device *nesdev,
struct nes_qp *nesqp, int virt_wqs)
{
unsigned long flags;
@@ -1315,6 +1314,8 @@
nes_debug(NES_DBG_QP, "Invalid QP type: %d\n", init_attr->qp_type);
return ERR_PTR(-EINVAL);
}
+ init_completion(&nesqp->sq_drained);
+ init_completion(&nesqp->rq_drained);
nesqp->sig_all = (init_attr->sq_sig_type == IB_SIGNAL_ALL_WR);
init_timer(&nesqp->terminate_timer);
@@ -3452,6 +3453,29 @@
return err;
}
+/**
+ * nes_drain_sq - drain sq
+ * @ibqp: pointer to ibqp
+ */
+static void nes_drain_sq(struct ib_qp *ibqp)
+{
+ struct nes_qp *nesqp = to_nesqp(ibqp);
+
+ if (nesqp->hwqp.sq_tail != nesqp->hwqp.sq_head)
+ wait_for_completion(&nesqp->sq_drained);
+}
+
+/**
+ * nes_drain_rq - drain rq
+ * @ibqp: pointer to ibqp
+ */
+static void nes_drain_rq(struct ib_qp *ibqp)
+{
+ struct nes_qp *nesqp = to_nesqp(ibqp);
+
+ if (nesqp->hwqp.rq_tail != nesqp->hwqp.rq_head)
+ wait_for_completion(&nesqp->rq_drained);
+}
/**
* nes_poll_cq
@@ -3582,6 +3606,13 @@
}
}
+ if (nesqp->iwarp_state > NES_CQP_QP_IWARP_STATE_RTS) {
+ if (nesqp->hwqp.sq_tail == nesqp->hwqp.sq_head)
+ complete(&nesqp->sq_drained);
+ if (nesqp->hwqp.rq_tail == nesqp->hwqp.rq_head)
+ complete(&nesqp->rq_drained);
+ }
+
entry->wr_id = wrid;
entry++;
cqe_count++;
@@ -3754,6 +3785,8 @@
nesibdev->ibdev.req_notify_cq = nes_req_notify_cq;
nesibdev->ibdev.post_send = nes_post_send;
nesibdev->ibdev.post_recv = nes_post_recv;
+ nesibdev->ibdev.drain_sq = nes_drain_sq;
+ nesibdev->ibdev.drain_rq = nes_drain_rq;
nesibdev->ibdev.iwcm = kzalloc(sizeof(*nesibdev->ibdev.iwcm), GFP_KERNEL);
if (nesibdev->ibdev.iwcm == NULL) {
diff --git a/drivers/infiniband/hw/nes/nes_verbs.h b/drivers/infiniband/hw/nes/nes_verbs.h
index 7029088..e02a566 100644
--- a/drivers/infiniband/hw/nes/nes_verbs.h
+++ b/drivers/infiniband/hw/nes/nes_verbs.h
@@ -189,6 +189,8 @@
u8 pau_pending;
u8 pau_state;
__u64 nesuqp_addr;
+ struct completion sq_drained;
+ struct completion rq_drained;
};
struct ib_mr *nes_reg_phys_mr(struct ib_pd *ib_pd,
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c b/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
index a8496a1..b1a3d91 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
@@ -3081,13 +3081,12 @@
return 0;
}
-int ocrdma_map_mr_sg(struct ib_mr *ibmr,
- struct scatterlist *sg,
- int sg_nents)
+int ocrdma_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, int sg_nents,
+ unsigned int *sg_offset)
{
struct ocrdma_mr *mr = get_ocrdma_mr(ibmr);
mr->npages = 0;
- return ib_sg_to_pages(ibmr, sg, sg_nents, ocrdma_set_page);
+ return ib_sg_to_pages(ibmr, sg, sg_nents, sg_offset, ocrdma_set_page);
}
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_verbs.h b/drivers/infiniband/hw/ocrdma/ocrdma_verbs.h
index 8b517fd..704ef1e 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_verbs.h
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_verbs.h
@@ -122,8 +122,7 @@
struct ib_mr *ocrdma_alloc_mr(struct ib_pd *pd,
enum ib_mr_type mr_type,
u32 max_num_sg);
-int ocrdma_map_mr_sg(struct ib_mr *ibmr,
- struct scatterlist *sg,
- int sg_nents);
+int ocrdma_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, int sg_nents,
+ unsigned int *sg_offset);
#endif /* __OCRDMA_VERBS_H__ */
diff --git a/drivers/infiniband/hw/qib/qib_file_ops.c b/drivers/infiniband/hw/qib/qib_file_ops.c
index 24f4a78..ff946d5 100644
--- a/drivers/infiniband/hw/qib/qib_file_ops.c
+++ b/drivers/infiniband/hw/qib/qib_file_ops.c
@@ -824,10 +824,7 @@
phys = dd->physaddr + piobufs;
#if defined(__powerpc__)
- /* There isn't a generic way to specify writethrough mappings */
- pgprot_val(vma->vm_page_prot) |= _PAGE_NO_CACHE;
- pgprot_val(vma->vm_page_prot) |= _PAGE_WRITETHRU;
- pgprot_val(vma->vm_page_prot) &= ~_PAGE_GUARDED;
+ vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
#endif
/*
diff --git a/drivers/infiniband/hw/qib/qib_init.c b/drivers/infiniband/hw/qib/qib_init.c
index 3f062f0..f253111 100644
--- a/drivers/infiniband/hw/qib/qib_init.c
+++ b/drivers/infiniband/hw/qib/qib_init.c
@@ -1090,7 +1090,7 @@
qib_dbg_ibdev_exit(&dd->verbs_dev);
#endif
free_percpu(dd->int_counter);
- ib_dealloc_device(&dd->verbs_dev.rdi.ibdev);
+ rvt_dealloc_device(&dd->verbs_dev.rdi);
}
u64 qib_int_counter(struct qib_devdata *dd)
@@ -1183,7 +1183,7 @@
bail:
if (!list_empty(&dd->list))
list_del_init(&dd->list);
- ib_dealloc_device(&dd->verbs_dev.rdi.ibdev);
+ rvt_dealloc_device(&dd->verbs_dev.rdi);
return ERR_PTR(ret);
}
diff --git a/drivers/infiniband/hw/qib/qib_pcie.c b/drivers/infiniband/hw/qib/qib_pcie.c
index 4758a38..6abe1c6 100644
--- a/drivers/infiniband/hw/qib/qib_pcie.c
+++ b/drivers/infiniband/hw/qib/qib_pcie.c
@@ -144,13 +144,7 @@
addr = pci_resource_start(pdev, 0);
len = pci_resource_len(pdev, 0);
-#if defined(__powerpc__)
- /* There isn't a generic way to specify writethrough mappings */
- dd->kregbase = __ioremap(addr, len, _PAGE_NO_CACHE | _PAGE_WRITETHRU);
-#else
dd->kregbase = ioremap_nocache(addr, len);
-#endif
-
if (!dd->kregbase)
return -ENOMEM;
diff --git a/drivers/infiniband/hw/qib/qib_rc.c b/drivers/infiniband/hw/qib/qib_rc.c
index 9088e26..444028a 100644
--- a/drivers/infiniband/hw/qib/qib_rc.c
+++ b/drivers/infiniband/hw/qib/qib_rc.c
@@ -230,7 +230,7 @@
*
* Return 1 if constructed; otherwise, return 0.
*/
-int qib_make_rc_req(struct rvt_qp *qp)
+int qib_make_rc_req(struct rvt_qp *qp, unsigned long *flags)
{
struct qib_qp_priv *priv = qp->priv;
struct qib_ibdev *dev = to_idev(qp->ibqp.device);
diff --git a/drivers/infiniband/hw/qib/qib_ruc.c b/drivers/infiniband/hw/qib/qib_ruc.c
index a5f07a6..b677792 100644
--- a/drivers/infiniband/hw/qib/qib_ruc.c
+++ b/drivers/infiniband/hw/qib/qib_ruc.c
@@ -739,7 +739,7 @@
struct qib_qp_priv *priv = qp->priv;
struct qib_ibport *ibp = to_iport(qp->ibqp.device, qp->port_num);
struct qib_pportdata *ppd = ppd_from_ibp(ibp);
- int (*make_req)(struct rvt_qp *qp);
+ int (*make_req)(struct rvt_qp *qp, unsigned long *flags);
unsigned long flags;
if ((qp->ibqp.qp_type == IB_QPT_RC ||
@@ -781,7 +781,7 @@
qp->s_hdrwords = 0;
spin_lock_irqsave(&qp->s_lock, flags);
}
- } while (make_req(qp));
+ } while (make_req(qp, &flags));
spin_unlock_irqrestore(&qp->s_lock, flags);
}
diff --git a/drivers/infiniband/hw/qib/qib_uc.c b/drivers/infiniband/hw/qib/qib_uc.c
index 7bdbc79..1d61bd0 100644
--- a/drivers/infiniband/hw/qib/qib_uc.c
+++ b/drivers/infiniband/hw/qib/qib_uc.c
@@ -45,7 +45,7 @@
*
* Return 1 if constructed; otherwise, return 0.
*/
-int qib_make_uc_req(struct rvt_qp *qp)
+int qib_make_uc_req(struct rvt_qp *qp, unsigned long *flags)
{
struct qib_qp_priv *priv = qp->priv;
struct qib_other_headers *ohdr;
diff --git a/drivers/infiniband/hw/qib/qib_ud.c b/drivers/infiniband/hw/qib/qib_ud.c
index d950213..846e6c7 100644
--- a/drivers/infiniband/hw/qib/qib_ud.c
+++ b/drivers/infiniband/hw/qib/qib_ud.c
@@ -238,7 +238,7 @@
*
* Return 1 if constructed; otherwise, return 0.
*/
-int qib_make_ud_req(struct rvt_qp *qp)
+int qib_make_ud_req(struct rvt_qp *qp, unsigned long *flags)
{
struct qib_qp_priv *priv = qp->priv;
struct qib_other_headers *ohdr;
@@ -294,7 +294,7 @@
this_cpu_inc(ibp->pmastats->n_unicast_xmit);
lid = ah_attr->dlid & ~((1 << ppd->lmc) - 1);
if (unlikely(lid == ppd->lid)) {
- unsigned long flags;
+ unsigned long tflags = *flags;
/*
* If DMAs are in progress, we can't generate
* a completion for the loopback packet since
@@ -307,10 +307,10 @@
goto bail;
}
qp->s_cur = next_cur;
- local_irq_save(flags);
- spin_unlock_irqrestore(&qp->s_lock, flags);
+ spin_unlock_irqrestore(&qp->s_lock, tflags);
qib_ud_loopback(qp, wqe);
- spin_lock_irqsave(&qp->s_lock, flags);
+ spin_lock_irqsave(&qp->s_lock, tflags);
+ *flags = tflags;
qib_send_complete(qp, wqe, IB_WC_SUCCESS);
goto done;
}
diff --git a/drivers/infiniband/hw/qib/qib_verbs.h b/drivers/infiniband/hw/qib/qib_verbs.h
index 4b76a8d..6888f03 100644
--- a/drivers/infiniband/hw/qib/qib_verbs.h
+++ b/drivers/infiniband/hw/qib/qib_verbs.h
@@ -430,11 +430,11 @@
void qib_send_rc_ack(struct rvt_qp *qp);
-int qib_make_rc_req(struct rvt_qp *qp);
+int qib_make_rc_req(struct rvt_qp *qp, unsigned long *flags);
-int qib_make_uc_req(struct rvt_qp *qp);
+int qib_make_uc_req(struct rvt_qp *qp, unsigned long *flags);
-int qib_make_ud_req(struct rvt_qp *qp);
+int qib_make_ud_req(struct rvt_qp *qp, unsigned long *flags);
int qib_register_ib_device(struct qib_devdata *);
diff --git a/drivers/infiniband/sw/rdmavt/qp.c b/drivers/infiniband/sw/rdmavt/qp.c
index a9e3bcc..0f12c21 100644
--- a/drivers/infiniband/sw/rdmavt/qp.c
+++ b/drivers/infiniband/sw/rdmavt/qp.c
@@ -829,13 +829,13 @@
case IB_QPT_SMI:
case IB_QPT_GSI:
case IB_QPT_UD:
- qp->allowed_ops = IB_OPCODE_UD_SEND_ONLY & RVT_OPCODE_QP_MASK;
+ qp->allowed_ops = IB_OPCODE_UD;
break;
case IB_QPT_RC:
- qp->allowed_ops = IB_OPCODE_RC_SEND_ONLY & RVT_OPCODE_QP_MASK;
+ qp->allowed_ops = IB_OPCODE_RC;
break;
case IB_QPT_UC:
- qp->allowed_ops = IB_OPCODE_UC_SEND_ONLY & RVT_OPCODE_QP_MASK;
+ qp->allowed_ops = IB_OPCODE_UC;
break;
default:
ret = ERR_PTR(-EINVAL);
diff --git a/drivers/infiniband/sw/rdmavt/vt.c b/drivers/infiniband/sw/rdmavt/vt.c
index 6caf527..e1cc2cc 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -106,6 +106,19 @@
}
EXPORT_SYMBOL(rvt_alloc_device);
+/**
+ * rvt_dealloc_device - deallocate rdi
+ * @rdi: structure to free
+ *
+ * Free a structure allocated with rvt_alloc_device()
+ */
+void rvt_dealloc_device(struct rvt_dev_info *rdi)
+{
+ kfree(rdi->ports);
+ ib_dealloc_device(&rdi->ibdev);
+}
+EXPORT_SYMBOL(rvt_dealloc_device);
+
static int rvt_query_device(struct ib_device *ibdev,
struct ib_device_attr *props,
struct ib_udata *uhw)
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ethtool.c b/drivers/infiniband/ulp/ipoib/ipoib_ethtool.c
index a53fa5f..1502199 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_ethtool.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_ethtool.c
@@ -36,6 +36,27 @@
#include "ipoib.h"
+struct ipoib_stats {
+ char stat_string[ETH_GSTRING_LEN];
+ int stat_offset;
+};
+
+#define IPOIB_NETDEV_STAT(m) { \
+ .stat_string = #m, \
+ .stat_offset = offsetof(struct rtnl_link_stats64, m) }
+
+static const struct ipoib_stats ipoib_gstrings_stats[] = {
+ IPOIB_NETDEV_STAT(rx_packets),
+ IPOIB_NETDEV_STAT(tx_packets),
+ IPOIB_NETDEV_STAT(rx_bytes),
+ IPOIB_NETDEV_STAT(tx_bytes),
+ IPOIB_NETDEV_STAT(tx_errors),
+ IPOIB_NETDEV_STAT(rx_dropped),
+ IPOIB_NETDEV_STAT(tx_dropped)
+};
+
+#define IPOIB_GLOBAL_STATS_LEN ARRAY_SIZE(ipoib_gstrings_stats)
+
static void ipoib_get_drvinfo(struct net_device *netdev,
struct ethtool_drvinfo *drvinfo)
{
@@ -92,11 +113,57 @@
return 0;
}
+static void ipoib_get_ethtool_stats(struct net_device *dev,
+ struct ethtool_stats __always_unused *stats,
+ u64 *data)
+{
+ int i;
+ struct net_device_stats *net_stats = &dev->stats;
+ u8 *p = (u8 *)net_stats;
+
+ for (i = 0; i < IPOIB_GLOBAL_STATS_LEN; i++)
+ data[i] = *(u64 *)(p + ipoib_gstrings_stats[i].stat_offset);
+
+}
+static void ipoib_get_strings(struct net_device __always_unused *dev,
+ u32 stringset, u8 *data)
+{
+ u8 *p = data;
+ int i;
+
+ switch (stringset) {
+ case ETH_SS_STATS:
+ for (i = 0; i < IPOIB_GLOBAL_STATS_LEN; i++) {
+ memcpy(p, ipoib_gstrings_stats[i].stat_string,
+ ETH_GSTRING_LEN);
+ p += ETH_GSTRING_LEN;
+ }
+ break;
+ case ETH_SS_TEST:
+ default:
+ break;
+ }
+}
+static int ipoib_get_sset_count(struct net_device __always_unused *dev,
+ int sset)
+{
+ switch (sset) {
+ case ETH_SS_STATS:
+ return IPOIB_GLOBAL_STATS_LEN;
+ case ETH_SS_TEST:
+ default:
+ break;
+ }
+ return -EOPNOTSUPP;
+}
static const struct ethtool_ops ipoib_ethtool_ops = {
.get_drvinfo = ipoib_get_drvinfo,
.get_coalesce = ipoib_get_coalesce,
.set_coalesce = ipoib_set_coalesce,
+ .get_strings = ipoib_get_strings,
+ .get_ethtool_stats = ipoib_get_ethtool_stats,
+ .get_sset_count = ipoib_get_sset_count,
};
void ipoib_set_ethtool_ops(struct net_device *dev)
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
index 3643d55..418e5a1 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
@@ -51,8 +51,6 @@
"Enable data path debug tracing if > 0");
#endif
-static DEFINE_MUTEX(pkey_mutex);
-
struct ipoib_ah *ipoib_create_ah(struct net_device *dev,
struct ib_pd *pd, struct ib_ah_attr *attr)
{
diff --git a/drivers/infiniband/ulp/iser/iser_memory.c b/drivers/infiniband/ulp/iser/iser_memory.c
index 9a391cc..90be568 100644
--- a/drivers/infiniband/ulp/iser/iser_memory.c
+++ b/drivers/infiniband/ulp/iser/iser_memory.c
@@ -236,7 +236,7 @@
page_vec->npages = 0;
page_vec->fake_mr.page_size = SIZE_4K;
plen = ib_sg_to_pages(&page_vec->fake_mr, mem->sg,
- mem->size, iser_set_page);
+ mem->size, NULL, iser_set_page);
if (unlikely(plen < mem->size)) {
iser_err("page vec too short to hold this SG\n");
iser_data_buf_dump(mem, device->ib_device);
@@ -446,7 +446,7 @@
ib_update_fast_reg_key(mr, ib_inc_rkey(mr->rkey));
- n = ib_map_mr_sg(mr, mem->sg, mem->size, SIZE_4K);
+ n = ib_map_mr_sg(mr, mem->sg, mem->size, NULL, SIZE_4K);
if (unlikely(n != mem->size)) {
iser_err("failed to map sg (%d/%d)\n",
n, mem->size);
diff --git a/drivers/infiniband/ulp/isert/ib_isert.c b/drivers/infiniband/ulp/isert/ib_isert.c
index 411e446..897b5a4 100644
--- a/drivers/infiniband/ulp/isert/ib_isert.c
+++ b/drivers/infiniband/ulp/isert/ib_isert.c
@@ -33,7 +33,8 @@
#define ISERT_MAX_CONN 8
#define ISER_MAX_RX_CQ_LEN (ISERT_QP_MAX_RECV_DTOS * ISERT_MAX_CONN)
-#define ISER_MAX_TX_CQ_LEN (ISERT_QP_MAX_REQ_DTOS * ISERT_MAX_CONN)
+#define ISER_MAX_TX_CQ_LEN \
+ ((ISERT_QP_MAX_REQ_DTOS + ISCSI_DEF_XMIT_CMDS_MAX) * ISERT_MAX_CONN)
#define ISER_MAX_CQ_LEN (ISER_MAX_RX_CQ_LEN + ISER_MAX_TX_CQ_LEN + \
ISERT_MAX_CONN)
@@ -46,14 +47,6 @@
static struct workqueue_struct *isert_comp_wq;
static struct workqueue_struct *isert_release_wq;
-static void
-isert_unmap_cmd(struct isert_cmd *isert_cmd, struct isert_conn *isert_conn);
-static int
-isert_map_rdma(struct isert_cmd *isert_cmd, struct iscsi_conn *conn);
-static void
-isert_unreg_rdma(struct isert_cmd *isert_cmd, struct isert_conn *isert_conn);
-static int
-isert_reg_rdma(struct isert_cmd *isert_cmd, struct iscsi_conn *conn);
static int
isert_put_response(struct iscsi_conn *conn, struct iscsi_cmd *cmd);
static int
@@ -142,6 +135,7 @@
attr.recv_cq = comp->cq;
attr.cap.max_send_wr = ISERT_QP_MAX_REQ_DTOS + 1;
attr.cap.max_recv_wr = ISERT_QP_MAX_RECV_DTOS + 1;
+ attr.cap.max_rdma_ctxs = ISCSI_DEF_XMIT_CMDS_MAX;
attr.cap.max_send_sge = device->ib_device->attrs.max_sge;
isert_conn->max_sge = min(device->ib_device->attrs.max_sge,
device->ib_device->attrs.max_sge_rd);
@@ -270,9 +264,9 @@
device->ib_device->num_comp_vectors));
isert_info("Using %d CQs, %s supports %d vectors support "
- "Fast registration %d pi_capable %d\n",
+ "pi_capable %d\n",
device->comps_used, device->ib_device->name,
- device->ib_device->num_comp_vectors, device->use_fastreg,
+ device->ib_device->num_comp_vectors,
device->pi_capable);
device->comps = kcalloc(device->comps_used, sizeof(struct isert_comp),
@@ -313,18 +307,6 @@
isert_dbg("devattr->max_sge: %d\n", ib_dev->attrs.max_sge);
isert_dbg("devattr->max_sge_rd: %d\n", ib_dev->attrs.max_sge_rd);
- /* asign function handlers */
- if (ib_dev->attrs.device_cap_flags & IB_DEVICE_MEM_MGT_EXTENSIONS &&
- ib_dev->attrs.device_cap_flags & IB_DEVICE_SIGNATURE_HANDOVER) {
- device->use_fastreg = 1;
- device->reg_rdma_mem = isert_reg_rdma;
- device->unreg_rdma_mem = isert_unreg_rdma;
- } else {
- device->use_fastreg = 0;
- device->reg_rdma_mem = isert_map_rdma;
- device->unreg_rdma_mem = isert_unmap_cmd;
- }
-
ret = isert_alloc_comps(device);
if (ret)
goto out;
@@ -417,146 +399,6 @@
}
static void
-isert_conn_free_fastreg_pool(struct isert_conn *isert_conn)
-{
- struct fast_reg_descriptor *fr_desc, *tmp;
- int i = 0;
-
- if (list_empty(&isert_conn->fr_pool))
- return;
-
- isert_info("Freeing conn %p fastreg pool", isert_conn);
-
- list_for_each_entry_safe(fr_desc, tmp,
- &isert_conn->fr_pool, list) {
- list_del(&fr_desc->list);
- ib_dereg_mr(fr_desc->data_mr);
- if (fr_desc->pi_ctx) {
- ib_dereg_mr(fr_desc->pi_ctx->prot_mr);
- ib_dereg_mr(fr_desc->pi_ctx->sig_mr);
- kfree(fr_desc->pi_ctx);
- }
- kfree(fr_desc);
- ++i;
- }
-
- if (i < isert_conn->fr_pool_size)
- isert_warn("Pool still has %d regions registered\n",
- isert_conn->fr_pool_size - i);
-}
-
-static int
-isert_create_pi_ctx(struct fast_reg_descriptor *desc,
- struct ib_device *device,
- struct ib_pd *pd)
-{
- struct pi_context *pi_ctx;
- int ret;
-
- pi_ctx = kzalloc(sizeof(*desc->pi_ctx), GFP_KERNEL);
- if (!pi_ctx) {
- isert_err("Failed to allocate pi context\n");
- return -ENOMEM;
- }
-
- pi_ctx->prot_mr = ib_alloc_mr(pd, IB_MR_TYPE_MEM_REG,
- ISCSI_ISER_SG_TABLESIZE);
- if (IS_ERR(pi_ctx->prot_mr)) {
- isert_err("Failed to allocate prot frmr err=%ld\n",
- PTR_ERR(pi_ctx->prot_mr));
- ret = PTR_ERR(pi_ctx->prot_mr);
- goto err_pi_ctx;
- }
- desc->ind |= ISERT_PROT_KEY_VALID;
-
- pi_ctx->sig_mr = ib_alloc_mr(pd, IB_MR_TYPE_SIGNATURE, 2);
- if (IS_ERR(pi_ctx->sig_mr)) {
- isert_err("Failed to allocate signature enabled mr err=%ld\n",
- PTR_ERR(pi_ctx->sig_mr));
- ret = PTR_ERR(pi_ctx->sig_mr);
- goto err_prot_mr;
- }
-
- desc->pi_ctx = pi_ctx;
- desc->ind |= ISERT_SIG_KEY_VALID;
- desc->ind &= ~ISERT_PROTECTED;
-
- return 0;
-
-err_prot_mr:
- ib_dereg_mr(pi_ctx->prot_mr);
-err_pi_ctx:
- kfree(pi_ctx);
-
- return ret;
-}
-
-static int
-isert_create_fr_desc(struct ib_device *ib_device, struct ib_pd *pd,
- struct fast_reg_descriptor *fr_desc)
-{
- fr_desc->data_mr = ib_alloc_mr(pd, IB_MR_TYPE_MEM_REG,
- ISCSI_ISER_SG_TABLESIZE);
- if (IS_ERR(fr_desc->data_mr)) {
- isert_err("Failed to allocate data frmr err=%ld\n",
- PTR_ERR(fr_desc->data_mr));
- return PTR_ERR(fr_desc->data_mr);
- }
- fr_desc->ind |= ISERT_DATA_KEY_VALID;
-
- isert_dbg("Created fr_desc %p\n", fr_desc);
-
- return 0;
-}
-
-static int
-isert_conn_create_fastreg_pool(struct isert_conn *isert_conn)
-{
- struct fast_reg_descriptor *fr_desc;
- struct isert_device *device = isert_conn->device;
- struct se_session *se_sess = isert_conn->conn->sess->se_sess;
- struct se_node_acl *se_nacl = se_sess->se_node_acl;
- int i, ret, tag_num;
- /*
- * Setup the number of FRMRs based upon the number of tags
- * available to session in iscsi_target_locate_portal().
- */
- tag_num = max_t(u32, ISCSIT_MIN_TAGS, se_nacl->queue_depth);
- tag_num = (tag_num * 2) + ISCSIT_EXTRA_TAGS;
-
- isert_conn->fr_pool_size = 0;
- for (i = 0; i < tag_num; i++) {
- fr_desc = kzalloc(sizeof(*fr_desc), GFP_KERNEL);
- if (!fr_desc) {
- isert_err("Failed to allocate fast_reg descriptor\n");
- ret = -ENOMEM;
- goto err;
- }
-
- ret = isert_create_fr_desc(device->ib_device,
- device->pd, fr_desc);
- if (ret) {
- isert_err("Failed to create fastreg descriptor err=%d\n",
- ret);
- kfree(fr_desc);
- goto err;
- }
-
- list_add_tail(&fr_desc->list, &isert_conn->fr_pool);
- isert_conn->fr_pool_size++;
- }
-
- isert_dbg("Creating conn %p fastreg pool size=%d",
- isert_conn, isert_conn->fr_pool_size);
-
- return 0;
-
-err:
- isert_conn_free_fastreg_pool(isert_conn);
- return ret;
-}
-
-static void
isert_init_conn(struct isert_conn *isert_conn)
{
isert_conn->state = ISER_CONN_INIT;
@@ -565,8 +407,6 @@
init_completion(&isert_conn->login_req_comp);
kref_init(&isert_conn->kref);
mutex_init(&isert_conn->mutex);
- spin_lock_init(&isert_conn->pool_lock);
- INIT_LIST_HEAD(&isert_conn->fr_pool);
INIT_WORK(&isert_conn->release_work, isert_release_work);
}
@@ -739,9 +579,6 @@
BUG_ON(!device);
- if (device->use_fastreg)
- isert_conn_free_fastreg_pool(isert_conn);
-
isert_free_rx_descriptors(isert_conn);
if (isert_conn->cm_id)
rdma_destroy_id(isert_conn->cm_id);
@@ -1080,7 +917,6 @@
{
struct iser_tx_desc *tx_desc = &isert_cmd->tx_desc;
- isert_cmd->iser_ib_op = ISER_IB_SEND;
tx_desc->tx_cqe.done = isert_send_done;
send_wr->wr_cqe = &tx_desc->tx_cqe;
@@ -1160,16 +996,6 @@
}
if (!login->login_failed) {
if (login->login_complete) {
- if (!conn->sess->sess_ops->SessionType &&
- isert_conn->device->use_fastreg) {
- ret = isert_conn_create_fastreg_pool(isert_conn);
- if (ret) {
- isert_err("Conn: %p failed to create"
- " fastreg pool\n", isert_conn);
- return ret;
- }
- }
-
ret = isert_alloc_rx_descriptors(isert_conn);
if (ret)
return ret;
@@ -1633,97 +1459,26 @@
ISER_RX_PAYLOAD_SIZE, DMA_FROM_DEVICE);
}
-static int
-isert_map_data_buf(struct isert_conn *isert_conn, struct isert_cmd *isert_cmd,
- struct scatterlist *sg, u32 nents, u32 length, u32 offset,
- enum iser_ib_op_code op, struct isert_data_buf *data)
-{
- struct ib_device *ib_dev = isert_conn->cm_id->device;
-
- data->dma_dir = op == ISER_IB_RDMA_WRITE ?
- DMA_TO_DEVICE : DMA_FROM_DEVICE;
-
- data->len = length - offset;
- data->offset = offset;
- data->sg_off = data->offset / PAGE_SIZE;
-
- data->sg = &sg[data->sg_off];
- data->nents = min_t(unsigned int, nents - data->sg_off,
- ISCSI_ISER_SG_TABLESIZE);
- data->len = min_t(unsigned int, data->len, ISCSI_ISER_SG_TABLESIZE *
- PAGE_SIZE);
-
- data->dma_nents = ib_dma_map_sg(ib_dev, data->sg, data->nents,
- data->dma_dir);
- if (unlikely(!data->dma_nents)) {
- isert_err("Cmd: unable to dma map SGs %p\n", sg);
- return -EINVAL;
- }
-
- isert_dbg("Mapped cmd: %p count: %u sg: %p sg_nents: %u rdma_len %d\n",
- isert_cmd, data->dma_nents, data->sg, data->nents, data->len);
-
- return 0;
-}
-
static void
-isert_unmap_data_buf(struct isert_conn *isert_conn, struct isert_data_buf *data)
+isert_rdma_rw_ctx_destroy(struct isert_cmd *cmd, struct isert_conn *conn)
{
- struct ib_device *ib_dev = isert_conn->cm_id->device;
+ struct se_cmd *se_cmd = &cmd->iscsi_cmd->se_cmd;
+ enum dma_data_direction dir = target_reverse_dma_direction(se_cmd);
- ib_dma_unmap_sg(ib_dev, data->sg, data->nents, data->dma_dir);
- memset(data, 0, sizeof(*data));
-}
+ if (!cmd->rw.nr_ops)
+ return;
-
-
-static void
-isert_unmap_cmd(struct isert_cmd *isert_cmd, struct isert_conn *isert_conn)
-{
- isert_dbg("Cmd %p\n", isert_cmd);
-
- if (isert_cmd->data.sg) {
- isert_dbg("Cmd %p unmap_sg op\n", isert_cmd);
- isert_unmap_data_buf(isert_conn, &isert_cmd->data);
+ if (isert_prot_cmd(conn, se_cmd)) {
+ rdma_rw_ctx_destroy_signature(&cmd->rw, conn->qp,
+ conn->cm_id->port_num, se_cmd->t_data_sg,
+ se_cmd->t_data_nents, se_cmd->t_prot_sg,
+ se_cmd->t_prot_nents, dir);
+ } else {
+ rdma_rw_ctx_destroy(&cmd->rw, conn->qp, conn->cm_id->port_num,
+ se_cmd->t_data_sg, se_cmd->t_data_nents, dir);
}
- if (isert_cmd->rdma_wr) {
- isert_dbg("Cmd %p free send_wr\n", isert_cmd);
- kfree(isert_cmd->rdma_wr);
- isert_cmd->rdma_wr = NULL;
- }
-
- if (isert_cmd->ib_sge) {
- isert_dbg("Cmd %p free ib_sge\n", isert_cmd);
- kfree(isert_cmd->ib_sge);
- isert_cmd->ib_sge = NULL;
- }
-}
-
-static void
-isert_unreg_rdma(struct isert_cmd *isert_cmd, struct isert_conn *isert_conn)
-{
- isert_dbg("Cmd %p\n", isert_cmd);
-
- if (isert_cmd->fr_desc) {
- isert_dbg("Cmd %p free fr_desc %p\n", isert_cmd, isert_cmd->fr_desc);
- if (isert_cmd->fr_desc->ind & ISERT_PROTECTED) {
- isert_unmap_data_buf(isert_conn, &isert_cmd->prot);
- isert_cmd->fr_desc->ind &= ~ISERT_PROTECTED;
- }
- spin_lock_bh(&isert_conn->pool_lock);
- list_add_tail(&isert_cmd->fr_desc->list, &isert_conn->fr_pool);
- spin_unlock_bh(&isert_conn->pool_lock);
- isert_cmd->fr_desc = NULL;
- }
-
- if (isert_cmd->data.sg) {
- isert_dbg("Cmd %p unmap_sg op\n", isert_cmd);
- isert_unmap_data_buf(isert_conn, &isert_cmd->data);
- }
-
- isert_cmd->ib_sge = NULL;
- isert_cmd->rdma_wr = NULL;
+ cmd->rw.nr_ops = 0;
}
static void
@@ -1732,7 +1487,6 @@
struct iscsi_cmd *cmd = isert_cmd->iscsi_cmd;
struct isert_conn *isert_conn = isert_cmd->conn;
struct iscsi_conn *conn = isert_conn->conn;
- struct isert_device *device = isert_conn->device;
struct iscsi_text_rsp *hdr;
isert_dbg("Cmd %p\n", isert_cmd);
@@ -1760,7 +1514,7 @@
}
}
- device->unreg_rdma_mem(isert_cmd, isert_conn);
+ isert_rdma_rw_ctx_destroy(isert_cmd, isert_conn);
transport_generic_free_cmd(&cmd->se_cmd, 0);
break;
case ISCSI_OP_SCSI_TMFUNC:
@@ -1894,14 +1648,9 @@
isert_dbg("Cmd %p\n", isert_cmd);
- if (isert_cmd->fr_desc && isert_cmd->fr_desc->ind & ISERT_PROTECTED) {
- ret = isert_check_pi_status(cmd,
- isert_cmd->fr_desc->pi_ctx->sig_mr);
- isert_cmd->fr_desc->ind &= ~ISERT_PROTECTED;
- }
+ ret = isert_check_pi_status(cmd, isert_cmd->rw.sig->sig_mr);
+ isert_rdma_rw_ctx_destroy(isert_cmd, isert_conn);
- device->unreg_rdma_mem(isert_cmd, isert_conn);
- isert_cmd->rdma_wr_num = 0;
if (ret)
transport_send_check_condition_and_sense(cmd, cmd->pi_err, 0);
else
@@ -1929,16 +1678,12 @@
isert_dbg("Cmd %p\n", isert_cmd);
- if (isert_cmd->fr_desc && isert_cmd->fr_desc->ind & ISERT_PROTECTED) {
- ret = isert_check_pi_status(se_cmd,
- isert_cmd->fr_desc->pi_ctx->sig_mr);
- isert_cmd->fr_desc->ind &= ~ISERT_PROTECTED;
- }
-
iscsit_stop_dataout_timer(cmd);
- device->unreg_rdma_mem(isert_cmd, isert_conn);
- cmd->write_data_done = isert_cmd->data.len;
- isert_cmd->rdma_wr_num = 0;
+
+ if (isert_prot_cmd(isert_conn, se_cmd))
+ ret = isert_check_pi_status(se_cmd, isert_cmd->rw.sig->sig_mr);
+ isert_rdma_rw_ctx_destroy(isert_cmd, isert_conn);
+ cmd->write_data_done = 0;
isert_dbg("Cmd: %p RDMA_READ comp calling execute_cmd\n", isert_cmd);
spin_lock_bh(&cmd->istate_lock);
@@ -2111,7 +1856,6 @@
{
struct isert_cmd *isert_cmd = iscsit_priv_cmd(cmd);
struct isert_conn *isert_conn = conn->context;
- struct isert_device *device = isert_conn->device;
spin_lock_bh(&conn->cmd_lock);
if (!list_empty(&cmd->i_conn_node))
@@ -2120,8 +1864,7 @@
if (cmd->data_direction == DMA_TO_DEVICE)
iscsit_stop_dataout_timer(cmd);
-
- device->unreg_rdma_mem(isert_cmd, isert_conn);
+ isert_rdma_rw_ctx_destroy(isert_cmd, isert_conn);
}
static enum target_prot_op
@@ -2274,234 +2017,6 @@
return isert_post_response(isert_conn, isert_cmd);
}
-static int
-isert_build_rdma_wr(struct isert_conn *isert_conn, struct isert_cmd *isert_cmd,
- struct ib_sge *ib_sge, struct ib_rdma_wr *rdma_wr,
- u32 data_left, u32 offset)
-{
- struct iscsi_cmd *cmd = isert_cmd->iscsi_cmd;
- struct scatterlist *sg_start, *tmp_sg;
- struct isert_device *device = isert_conn->device;
- struct ib_device *ib_dev = device->ib_device;
- u32 sg_off, page_off;
- int i = 0, sg_nents;
-
- sg_off = offset / PAGE_SIZE;
- sg_start = &cmd->se_cmd.t_data_sg[sg_off];
- sg_nents = min(cmd->se_cmd.t_data_nents - sg_off, isert_conn->max_sge);
- page_off = offset % PAGE_SIZE;
-
- rdma_wr->wr.sg_list = ib_sge;
- rdma_wr->wr.wr_cqe = &isert_cmd->tx_desc.tx_cqe;
-
- /*
- * Perform mapping of TCM scatterlist memory ib_sge dma_addr.
- */
- for_each_sg(sg_start, tmp_sg, sg_nents, i) {
- isert_dbg("RDMA from SGL dma_addr: 0x%llx dma_len: %u, "
- "page_off: %u\n",
- (unsigned long long)tmp_sg->dma_address,
- tmp_sg->length, page_off);
-
- ib_sge->addr = ib_sg_dma_address(ib_dev, tmp_sg) + page_off;
- ib_sge->length = min_t(u32, data_left,
- ib_sg_dma_len(ib_dev, tmp_sg) - page_off);
- ib_sge->lkey = device->pd->local_dma_lkey;
-
- isert_dbg("RDMA ib_sge: addr: 0x%llx length: %u lkey: %x\n",
- ib_sge->addr, ib_sge->length, ib_sge->lkey);
- page_off = 0;
- data_left -= ib_sge->length;
- if (!data_left)
- break;
- ib_sge++;
- isert_dbg("Incrementing ib_sge pointer to %p\n", ib_sge);
- }
-
- rdma_wr->wr.num_sge = ++i;
- isert_dbg("Set outgoing sg_list: %p num_sg: %u from TCM SGLs\n",
- rdma_wr->wr.sg_list, rdma_wr->wr.num_sge);
-
- return rdma_wr->wr.num_sge;
-}
-
-static int
-isert_map_rdma(struct isert_cmd *isert_cmd, struct iscsi_conn *conn)
-{
- struct iscsi_cmd *cmd = isert_cmd->iscsi_cmd;
- struct se_cmd *se_cmd = &cmd->se_cmd;
- struct isert_conn *isert_conn = conn->context;
- struct isert_data_buf *data = &isert_cmd->data;
- struct ib_rdma_wr *rdma_wr;
- struct ib_sge *ib_sge;
- u32 offset, data_len, data_left, rdma_write_max, va_offset = 0;
- int ret = 0, i, ib_sge_cnt;
-
- offset = isert_cmd->iser_ib_op == ISER_IB_RDMA_READ ?
- cmd->write_data_done : 0;
- ret = isert_map_data_buf(isert_conn, isert_cmd, se_cmd->t_data_sg,
- se_cmd->t_data_nents, se_cmd->data_length,
- offset, isert_cmd->iser_ib_op,
- &isert_cmd->data);
- if (ret)
- return ret;
-
- data_left = data->len;
- offset = data->offset;
-
- ib_sge = kzalloc(sizeof(struct ib_sge) * data->nents, GFP_KERNEL);
- if (!ib_sge) {
- isert_warn("Unable to allocate ib_sge\n");
- ret = -ENOMEM;
- goto unmap_cmd;
- }
- isert_cmd->ib_sge = ib_sge;
-
- isert_cmd->rdma_wr_num = DIV_ROUND_UP(data->nents, isert_conn->max_sge);
- isert_cmd->rdma_wr = kzalloc(sizeof(struct ib_rdma_wr) *
- isert_cmd->rdma_wr_num, GFP_KERNEL);
- if (!isert_cmd->rdma_wr) {
- isert_dbg("Unable to allocate isert_cmd->rdma_wr\n");
- ret = -ENOMEM;
- goto unmap_cmd;
- }
-
- rdma_write_max = isert_conn->max_sge * PAGE_SIZE;
-
- for (i = 0; i < isert_cmd->rdma_wr_num; i++) {
- rdma_wr = &isert_cmd->rdma_wr[i];
- data_len = min(data_left, rdma_write_max);
-
- rdma_wr->wr.send_flags = 0;
- if (isert_cmd->iser_ib_op == ISER_IB_RDMA_WRITE) {
- isert_cmd->tx_desc.tx_cqe.done = isert_rdma_write_done;
-
- rdma_wr->wr.opcode = IB_WR_RDMA_WRITE;
- rdma_wr->remote_addr = isert_cmd->read_va + offset;
- rdma_wr->rkey = isert_cmd->read_stag;
- if (i + 1 == isert_cmd->rdma_wr_num)
- rdma_wr->wr.next = &isert_cmd->tx_desc.send_wr;
- else
- rdma_wr->wr.next = &isert_cmd->rdma_wr[i + 1].wr;
- } else {
- isert_cmd->tx_desc.tx_cqe.done = isert_rdma_read_done;
-
- rdma_wr->wr.opcode = IB_WR_RDMA_READ;
- rdma_wr->remote_addr = isert_cmd->write_va + va_offset;
- rdma_wr->rkey = isert_cmd->write_stag;
- if (i + 1 == isert_cmd->rdma_wr_num)
- rdma_wr->wr.send_flags = IB_SEND_SIGNALED;
- else
- rdma_wr->wr.next = &isert_cmd->rdma_wr[i + 1].wr;
- }
-
- ib_sge_cnt = isert_build_rdma_wr(isert_conn, isert_cmd, ib_sge,
- rdma_wr, data_len, offset);
- ib_sge += ib_sge_cnt;
-
- offset += data_len;
- va_offset += data_len;
- data_left -= data_len;
- }
-
- return 0;
-unmap_cmd:
- isert_unmap_data_buf(isert_conn, data);
-
- return ret;
-}
-
-static inline void
-isert_inv_rkey(struct ib_send_wr *inv_wr, struct ib_mr *mr)
-{
- u32 rkey;
-
- memset(inv_wr, 0, sizeof(*inv_wr));
- inv_wr->wr_cqe = NULL;
- inv_wr->opcode = IB_WR_LOCAL_INV;
- inv_wr->ex.invalidate_rkey = mr->rkey;
-
- /* Bump the key */
- rkey = ib_inc_rkey(mr->rkey);
- ib_update_fast_reg_key(mr, rkey);
-}
-
-static int
-isert_fast_reg_mr(struct isert_conn *isert_conn,
- struct fast_reg_descriptor *fr_desc,
- struct isert_data_buf *mem,
- enum isert_indicator ind,
- struct ib_sge *sge)
-{
- struct isert_device *device = isert_conn->device;
- struct ib_device *ib_dev = device->ib_device;
- struct ib_mr *mr;
- struct ib_reg_wr reg_wr;
- struct ib_send_wr inv_wr, *bad_wr, *wr = NULL;
- int ret, n;
-
- if (mem->dma_nents == 1) {
- sge->lkey = device->pd->local_dma_lkey;
- sge->addr = ib_sg_dma_address(ib_dev, &mem->sg[0]);
- sge->length = ib_sg_dma_len(ib_dev, &mem->sg[0]);
- isert_dbg("sge: addr: 0x%llx length: %u lkey: %x\n",
- sge->addr, sge->length, sge->lkey);
- return 0;
- }
-
- if (ind == ISERT_DATA_KEY_VALID)
- /* Registering data buffer */
- mr = fr_desc->data_mr;
- else
- /* Registering protection buffer */
- mr = fr_desc->pi_ctx->prot_mr;
-
- if (!(fr_desc->ind & ind)) {
- isert_inv_rkey(&inv_wr, mr);
- wr = &inv_wr;
- }
-
- n = ib_map_mr_sg(mr, mem->sg, mem->nents, PAGE_SIZE);
- if (unlikely(n != mem->nents)) {
- isert_err("failed to map mr sg (%d/%d)\n",
- n, mem->nents);
- return n < 0 ? n : -EINVAL;
- }
-
- isert_dbg("Use fr_desc %p sg_nents %d offset %u\n",
- fr_desc, mem->nents, mem->offset);
-
- reg_wr.wr.next = NULL;
- reg_wr.wr.opcode = IB_WR_REG_MR;
- reg_wr.wr.wr_cqe = NULL;
- reg_wr.wr.send_flags = 0;
- reg_wr.wr.num_sge = 0;
- reg_wr.mr = mr;
- reg_wr.key = mr->lkey;
- reg_wr.access = IB_ACCESS_LOCAL_WRITE;
-
- if (!wr)
- wr = ®_wr.wr;
- else
- wr->next = ®_wr.wr;
-
- ret = ib_post_send(isert_conn->qp, wr, &bad_wr);
- if (ret) {
- isert_err("fast registration failed, ret:%d\n", ret);
- return ret;
- }
- fr_desc->ind &= ~ind;
-
- sge->lkey = mr->lkey;
- sge->addr = mr->iova;
- sge->length = mr->length;
-
- isert_dbg("sge: addr: 0x%llx length: %u lkey: %x\n",
- sge->addr, sge->length, sge->lkey);
-
- return ret;
-}
-
static inline void
isert_set_dif_domain(struct se_cmd *se_cmd, struct ib_sig_attrs *sig_attrs,
struct ib_sig_domain *domain)
@@ -2526,6 +2041,8 @@
static int
isert_set_sig_attrs(struct se_cmd *se_cmd, struct ib_sig_attrs *sig_attrs)
{
+ memset(sig_attrs, 0, sizeof(*sig_attrs));
+
switch (se_cmd->prot_op) {
case TARGET_PROT_DIN_INSERT:
case TARGET_PROT_DOUT_STRIP:
@@ -2547,228 +2064,59 @@
return -EINVAL;
}
+ sig_attrs->check_mask =
+ (se_cmd->prot_checks & TARGET_DIF_CHECK_GUARD ? 0xc0 : 0) |
+ (se_cmd->prot_checks & TARGET_DIF_CHECK_REFTAG ? 0x30 : 0) |
+ (se_cmd->prot_checks & TARGET_DIF_CHECK_REFTAG ? 0x0f : 0);
return 0;
}
-static inline u8
-isert_set_prot_checks(u8 prot_checks)
-{
- return (prot_checks & TARGET_DIF_CHECK_GUARD ? 0xc0 : 0) |
- (prot_checks & TARGET_DIF_CHECK_REFTAG ? 0x30 : 0) |
- (prot_checks & TARGET_DIF_CHECK_REFTAG ? 0x0f : 0);
-}
-
static int
-isert_reg_sig_mr(struct isert_conn *isert_conn,
- struct isert_cmd *isert_cmd,
- struct fast_reg_descriptor *fr_desc)
+isert_rdma_rw_ctx_post(struct isert_cmd *cmd, struct isert_conn *conn,
+ struct ib_cqe *cqe, struct ib_send_wr *chain_wr)
{
- struct se_cmd *se_cmd = &isert_cmd->iscsi_cmd->se_cmd;
- struct ib_sig_handover_wr sig_wr;
- struct ib_send_wr inv_wr, *bad_wr, *wr = NULL;
- struct pi_context *pi_ctx = fr_desc->pi_ctx;
- struct ib_sig_attrs sig_attrs;
+ struct se_cmd *se_cmd = &cmd->iscsi_cmd->se_cmd;
+ enum dma_data_direction dir = target_reverse_dma_direction(se_cmd);
+ u8 port_num = conn->cm_id->port_num;
+ u64 addr;
+ u32 rkey, offset;
int ret;
- memset(&sig_attrs, 0, sizeof(sig_attrs));
- ret = isert_set_sig_attrs(se_cmd, &sig_attrs);
- if (ret)
- goto err;
-
- sig_attrs.check_mask = isert_set_prot_checks(se_cmd->prot_checks);
-
- if (!(fr_desc->ind & ISERT_SIG_KEY_VALID)) {
- isert_inv_rkey(&inv_wr, pi_ctx->sig_mr);
- wr = &inv_wr;
+ if (dir == DMA_FROM_DEVICE) {
+ addr = cmd->write_va;
+ rkey = cmd->write_stag;
+ offset = cmd->iscsi_cmd->write_data_done;
+ } else {
+ addr = cmd->read_va;
+ rkey = cmd->read_stag;
+ offset = 0;
}
- memset(&sig_wr, 0, sizeof(sig_wr));
- sig_wr.wr.opcode = IB_WR_REG_SIG_MR;
- sig_wr.wr.wr_cqe = NULL;
- sig_wr.wr.sg_list = &isert_cmd->ib_sg[DATA];
- sig_wr.wr.num_sge = 1;
- sig_wr.access_flags = IB_ACCESS_LOCAL_WRITE;
- sig_wr.sig_attrs = &sig_attrs;
- sig_wr.sig_mr = pi_ctx->sig_mr;
- if (se_cmd->t_prot_sg)
- sig_wr.prot = &isert_cmd->ib_sg[PROT];
+ if (isert_prot_cmd(conn, se_cmd)) {
+ struct ib_sig_attrs sig_attrs;
- if (!wr)
- wr = &sig_wr.wr;
- else
- wr->next = &sig_wr.wr;
-
- ret = ib_post_send(isert_conn->qp, wr, &bad_wr);
- if (ret) {
- isert_err("fast registration failed, ret:%d\n", ret);
- goto err;
- }
- fr_desc->ind &= ~ISERT_SIG_KEY_VALID;
-
- isert_cmd->ib_sg[SIG].lkey = pi_ctx->sig_mr->lkey;
- isert_cmd->ib_sg[SIG].addr = 0;
- isert_cmd->ib_sg[SIG].length = se_cmd->data_length;
- if (se_cmd->prot_op != TARGET_PROT_DIN_STRIP &&
- se_cmd->prot_op != TARGET_PROT_DOUT_INSERT)
- /*
- * We have protection guards on the wire
- * so we need to set a larget transfer
- */
- isert_cmd->ib_sg[SIG].length += se_cmd->prot_length;
-
- isert_dbg("sig_sge: addr: 0x%llx length: %u lkey: %x\n",
- isert_cmd->ib_sg[SIG].addr, isert_cmd->ib_sg[SIG].length,
- isert_cmd->ib_sg[SIG].lkey);
-err:
- return ret;
-}
-
-static int
-isert_handle_prot_cmd(struct isert_conn *isert_conn,
- struct isert_cmd *isert_cmd)
-{
- struct isert_device *device = isert_conn->device;
- struct se_cmd *se_cmd = &isert_cmd->iscsi_cmd->se_cmd;
- int ret;
-
- if (!isert_cmd->fr_desc->pi_ctx) {
- ret = isert_create_pi_ctx(isert_cmd->fr_desc,
- device->ib_device,
- device->pd);
- if (ret) {
- isert_err("conn %p failed to allocate pi_ctx\n",
- isert_conn);
- return ret;
- }
- }
-
- if (se_cmd->t_prot_sg) {
- ret = isert_map_data_buf(isert_conn, isert_cmd,
- se_cmd->t_prot_sg,
- se_cmd->t_prot_nents,
- se_cmd->prot_length,
- 0,
- isert_cmd->iser_ib_op,
- &isert_cmd->prot);
- if (ret) {
- isert_err("conn %p failed to map protection buffer\n",
- isert_conn);
- return ret;
- }
-
- memset(&isert_cmd->ib_sg[PROT], 0, sizeof(isert_cmd->ib_sg[PROT]));
- ret = isert_fast_reg_mr(isert_conn, isert_cmd->fr_desc,
- &isert_cmd->prot,
- ISERT_PROT_KEY_VALID,
- &isert_cmd->ib_sg[PROT]);
- if (ret) {
- isert_err("conn %p failed to fast reg mr\n",
- isert_conn);
- goto unmap_prot_cmd;
- }
- }
-
- ret = isert_reg_sig_mr(isert_conn, isert_cmd, isert_cmd->fr_desc);
- if (ret) {
- isert_err("conn %p failed to fast reg mr\n",
- isert_conn);
- goto unmap_prot_cmd;
- }
- isert_cmd->fr_desc->ind |= ISERT_PROTECTED;
-
- return 0;
-
-unmap_prot_cmd:
- if (se_cmd->t_prot_sg)
- isert_unmap_data_buf(isert_conn, &isert_cmd->prot);
-
- return ret;
-}
-
-static int
-isert_reg_rdma(struct isert_cmd *isert_cmd, struct iscsi_conn *conn)
-{
- struct iscsi_cmd *cmd = isert_cmd->iscsi_cmd;
- struct se_cmd *se_cmd = &cmd->se_cmd;
- struct isert_conn *isert_conn = conn->context;
- struct fast_reg_descriptor *fr_desc = NULL;
- struct ib_rdma_wr *rdma_wr;
- struct ib_sge *ib_sg;
- u32 offset;
- int ret = 0;
- unsigned long flags;
-
- offset = isert_cmd->iser_ib_op == ISER_IB_RDMA_READ ?
- cmd->write_data_done : 0;
- ret = isert_map_data_buf(isert_conn, isert_cmd, se_cmd->t_data_sg,
- se_cmd->t_data_nents, se_cmd->data_length,
- offset, isert_cmd->iser_ib_op,
- &isert_cmd->data);
- if (ret)
- return ret;
-
- if (isert_cmd->data.dma_nents != 1 ||
- isert_prot_cmd(isert_conn, se_cmd)) {
- spin_lock_irqsave(&isert_conn->pool_lock, flags);
- fr_desc = list_first_entry(&isert_conn->fr_pool,
- struct fast_reg_descriptor, list);
- list_del(&fr_desc->list);
- spin_unlock_irqrestore(&isert_conn->pool_lock, flags);
- isert_cmd->fr_desc = fr_desc;
- }
-
- ret = isert_fast_reg_mr(isert_conn, fr_desc, &isert_cmd->data,
- ISERT_DATA_KEY_VALID, &isert_cmd->ib_sg[DATA]);
- if (ret)
- goto unmap_cmd;
-
- if (isert_prot_cmd(isert_conn, se_cmd)) {
- ret = isert_handle_prot_cmd(isert_conn, isert_cmd);
+ ret = isert_set_sig_attrs(se_cmd, &sig_attrs);
if (ret)
- goto unmap_cmd;
+ return ret;
- ib_sg = &isert_cmd->ib_sg[SIG];
+ WARN_ON_ONCE(offset);
+ ret = rdma_rw_ctx_signature_init(&cmd->rw, conn->qp, port_num,
+ se_cmd->t_data_sg, se_cmd->t_data_nents,
+ se_cmd->t_prot_sg, se_cmd->t_prot_nents,
+ &sig_attrs, addr, rkey, dir);
} else {
- ib_sg = &isert_cmd->ib_sg[DATA];
+ ret = rdma_rw_ctx_init(&cmd->rw, conn->qp, port_num,
+ se_cmd->t_data_sg, se_cmd->t_data_nents,
+ offset, addr, rkey, dir);
+ }
+ if (ret < 0) {
+ isert_err("Cmd: %p failed to prepare RDMA res\n", cmd);
+ return ret;
}
- memcpy(&isert_cmd->s_ib_sge, ib_sg, sizeof(*ib_sg));
- isert_cmd->ib_sge = &isert_cmd->s_ib_sge;
- isert_cmd->rdma_wr_num = 1;
- memset(&isert_cmd->s_rdma_wr, 0, sizeof(isert_cmd->s_rdma_wr));
- isert_cmd->rdma_wr = &isert_cmd->s_rdma_wr;
-
- rdma_wr = &isert_cmd->s_rdma_wr;
- rdma_wr->wr.sg_list = &isert_cmd->s_ib_sge;
- rdma_wr->wr.num_sge = 1;
- rdma_wr->wr.wr_cqe = &isert_cmd->tx_desc.tx_cqe;
- if (isert_cmd->iser_ib_op == ISER_IB_RDMA_WRITE) {
- isert_cmd->tx_desc.tx_cqe.done = isert_rdma_write_done;
-
- rdma_wr->wr.opcode = IB_WR_RDMA_WRITE;
- rdma_wr->remote_addr = isert_cmd->read_va;
- rdma_wr->rkey = isert_cmd->read_stag;
- rdma_wr->wr.send_flags = !isert_prot_cmd(isert_conn, se_cmd) ?
- 0 : IB_SEND_SIGNALED;
- } else {
- isert_cmd->tx_desc.tx_cqe.done = isert_rdma_read_done;
-
- rdma_wr->wr.opcode = IB_WR_RDMA_READ;
- rdma_wr->remote_addr = isert_cmd->write_va;
- rdma_wr->rkey = isert_cmd->write_stag;
- rdma_wr->wr.send_flags = IB_SEND_SIGNALED;
- }
-
- return 0;
-
-unmap_cmd:
- if (fr_desc) {
- spin_lock_irqsave(&isert_conn->pool_lock, flags);
- list_add_tail(&fr_desc->list, &isert_conn->fr_pool);
- spin_unlock_irqrestore(&isert_conn->pool_lock, flags);
- }
- isert_unmap_data_buf(isert_conn, &isert_cmd->data);
-
+ ret = rdma_rw_ctx_post(&cmd->rw, conn->qp, port_num, cqe, chain_wr);
+ if (ret < 0)
+ isert_err("Cmd: %p failed to post RDMA res\n", cmd);
return ret;
}
@@ -2778,21 +2126,17 @@
struct se_cmd *se_cmd = &cmd->se_cmd;
struct isert_cmd *isert_cmd = iscsit_priv_cmd(cmd);
struct isert_conn *isert_conn = conn->context;
- struct isert_device *device = isert_conn->device;
- struct ib_send_wr *wr_failed;
+ struct ib_cqe *cqe = NULL;
+ struct ib_send_wr *chain_wr = NULL;
int rc;
isert_dbg("Cmd: %p RDMA_WRITE data_length: %u\n",
isert_cmd, se_cmd->data_length);
- isert_cmd->iser_ib_op = ISER_IB_RDMA_WRITE;
- rc = device->reg_rdma_mem(isert_cmd, conn);
- if (rc) {
- isert_err("Cmd: %p failed to prepare RDMA res\n", isert_cmd);
- return rc;
- }
-
- if (!isert_prot_cmd(isert_conn, se_cmd)) {
+ if (isert_prot_cmd(isert_conn, se_cmd)) {
+ isert_cmd->tx_desc.tx_cqe.done = isert_rdma_write_done;
+ cqe = &isert_cmd->tx_desc.tx_cqe;
+ } else {
/*
* Build isert_conn->tx_desc for iSCSI response PDU and attach
*/
@@ -2803,56 +2147,35 @@
isert_init_tx_hdrs(isert_conn, &isert_cmd->tx_desc);
isert_init_send_wr(isert_conn, isert_cmd,
&isert_cmd->tx_desc.send_wr);
- isert_cmd->s_rdma_wr.wr.next = &isert_cmd->tx_desc.send_wr;
- isert_cmd->rdma_wr_num += 1;
rc = isert_post_recv(isert_conn, isert_cmd->rx_desc);
if (rc) {
isert_err("ib_post_recv failed with %d\n", rc);
return rc;
}
+
+ chain_wr = &isert_cmd->tx_desc.send_wr;
}
- rc = ib_post_send(isert_conn->qp, &isert_cmd->rdma_wr->wr, &wr_failed);
- if (rc)
- isert_warn("ib_post_send() failed for IB_WR_RDMA_WRITE\n");
-
- if (!isert_prot_cmd(isert_conn, se_cmd))
- isert_dbg("Cmd: %p posted RDMA_WRITE + Response for iSER Data "
- "READ\n", isert_cmd);
- else
- isert_dbg("Cmd: %p posted RDMA_WRITE for iSER Data READ\n",
- isert_cmd);
-
+ isert_rdma_rw_ctx_post(isert_cmd, isert_conn, cqe, chain_wr);
+ isert_dbg("Cmd: %p posted RDMA_WRITE for iSER Data READ\n", isert_cmd);
return 1;
}
static int
isert_get_dataout(struct iscsi_conn *conn, struct iscsi_cmd *cmd, bool recovery)
{
- struct se_cmd *se_cmd = &cmd->se_cmd;
struct isert_cmd *isert_cmd = iscsit_priv_cmd(cmd);
- struct isert_conn *isert_conn = conn->context;
- struct isert_device *device = isert_conn->device;
- struct ib_send_wr *wr_failed;
- int rc;
isert_dbg("Cmd: %p RDMA_READ data_length: %u write_data_done: %u\n",
- isert_cmd, se_cmd->data_length, cmd->write_data_done);
- isert_cmd->iser_ib_op = ISER_IB_RDMA_READ;
- rc = device->reg_rdma_mem(isert_cmd, conn);
- if (rc) {
- isert_err("Cmd: %p failed to prepare RDMA res\n", isert_cmd);
- return rc;
- }
+ isert_cmd, cmd->se_cmd.data_length, cmd->write_data_done);
- rc = ib_post_send(isert_conn->qp, &isert_cmd->rdma_wr->wr, &wr_failed);
- if (rc)
- isert_warn("ib_post_send() failed for IB_WR_RDMA_READ\n");
+ isert_cmd->tx_desc.tx_cqe.done = isert_rdma_read_done;
+ isert_rdma_rw_ctx_post(isert_cmd, conn->context,
+ &isert_cmd->tx_desc.tx_cqe, NULL);
isert_dbg("Cmd: %p posted RDMA_READ memory for ISER Data WRITE\n",
isert_cmd);
-
return 0;
}
diff --git a/drivers/infiniband/ulp/isert/ib_isert.h b/drivers/infiniband/ulp/isert/ib_isert.h
index 147900c..e512ba9 100644
--- a/drivers/infiniband/ulp/isert/ib_isert.h
+++ b/drivers/infiniband/ulp/isert/ib_isert.h
@@ -3,6 +3,7 @@
#include <linux/in6.h>
#include <rdma/ib_verbs.h>
#include <rdma/rdma_cm.h>
+#include <rdma/rw.h>
#include <scsi/iser.h>
@@ -53,10 +54,7 @@
#define ISERT_MIN_POSTED_RX (ISCSI_DEF_XMIT_CMDS_MAX >> 2)
-#define ISERT_INFLIGHT_DATAOUTS 8
-
-#define ISERT_QP_MAX_REQ_DTOS (ISCSI_DEF_XMIT_CMDS_MAX * \
- (1 + ISERT_INFLIGHT_DATAOUTS) + \
+#define ISERT_QP_MAX_REQ_DTOS (ISCSI_DEF_XMIT_CMDS_MAX + \
ISERT_MAX_TX_MISC_PDUS + \
ISERT_MAX_RX_MISC_PDUS)
@@ -71,13 +69,6 @@
ISCSI_TX_DATAIN
};
-enum iser_ib_op_code {
- ISER_IB_RECV,
- ISER_IB_SEND,
- ISER_IB_RDMA_WRITE,
- ISER_IB_RDMA_READ,
-};
-
enum iser_conn_state {
ISER_CONN_INIT,
ISER_CONN_UP,
@@ -118,42 +109,6 @@
return container_of(cqe, struct iser_tx_desc, tx_cqe);
}
-
-enum isert_indicator {
- ISERT_PROTECTED = 1 << 0,
- ISERT_DATA_KEY_VALID = 1 << 1,
- ISERT_PROT_KEY_VALID = 1 << 2,
- ISERT_SIG_KEY_VALID = 1 << 3,
-};
-
-struct pi_context {
- struct ib_mr *prot_mr;
- struct ib_mr *sig_mr;
-};
-
-struct fast_reg_descriptor {
- struct list_head list;
- struct ib_mr *data_mr;
- u8 ind;
- struct pi_context *pi_ctx;
-};
-
-struct isert_data_buf {
- struct scatterlist *sg;
- int nents;
- u32 sg_off;
- u32 len; /* cur_rdma_length */
- u32 offset;
- unsigned int dma_nents;
- enum dma_data_direction dma_dir;
-};
-
-enum {
- DATA = 0,
- PROT = 1,
- SIG = 2,
-};
-
struct isert_cmd {
uint32_t read_stag;
uint32_t write_stag;
@@ -166,16 +121,7 @@
struct iscsi_cmd *iscsi_cmd;
struct iser_tx_desc tx_desc;
struct iser_rx_desc *rx_desc;
- enum iser_ib_op_code iser_ib_op;
- struct ib_sge *ib_sge;
- struct ib_sge s_ib_sge;
- int rdma_wr_num;
- struct ib_rdma_wr *rdma_wr;
- struct ib_rdma_wr s_rdma_wr;
- struct ib_sge ib_sg[3];
- struct isert_data_buf data;
- struct isert_data_buf prot;
- struct fast_reg_descriptor *fr_desc;
+ struct rdma_rw_ctx rw;
struct work_struct comp_work;
struct scatterlist sg;
};
@@ -210,10 +156,6 @@
struct isert_device *device;
struct mutex mutex;
struct kref kref;
- struct list_head fr_pool;
- int fr_pool_size;
- /* lock to protect fastreg pool */
- spinlock_t pool_lock;
struct work_struct release_work;
bool logout_posted;
bool snd_w_inv;
@@ -236,7 +178,6 @@
};
struct isert_device {
- int use_fastreg;
bool pi_capable;
int refcount;
struct ib_device *ib_device;
@@ -244,10 +185,6 @@
struct isert_comp *comps;
int comps_used;
struct list_head dev_node;
- int (*reg_rdma_mem)(struct isert_cmd *isert_cmd,
- struct iscsi_conn *conn);
- void (*unreg_rdma_mem)(struct isert_cmd *isert_cmd,
- struct isert_conn *isert_conn);
};
struct isert_np {
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
index b6bf204..646de17 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -70,6 +70,7 @@
static bool allow_ext_sg;
static bool prefer_fr = true;
static bool register_always = true;
+static bool never_register;
static int topspin_workarounds = 1;
module_param(srp_sg_tablesize, uint, 0444);
@@ -81,7 +82,7 @@
module_param(indirect_sg_entries, uint, 0444);
MODULE_PARM_DESC(indirect_sg_entries,
- "Default max number of gather/scatter entries (default is 12, max is " __stringify(SCSI_MAX_SG_CHAIN_SEGMENTS) ")");
+ "Default max number of gather/scatter entries (default is 12, max is " __stringify(SG_MAX_SEGMENTS) ")");
module_param(allow_ext_sg, bool, 0444);
MODULE_PARM_DESC(allow_ext_sg,
@@ -99,6 +100,9 @@
MODULE_PARM_DESC(register_always,
"Use memory registration even for contiguous memory regions");
+module_param(never_register, bool, 0444);
+MODULE_PARM_DESC(never_register, "Never register memory");
+
static const struct kernel_param_ops srp_tmo_ops;
static int srp_reconnect_delay = 10;
@@ -316,7 +320,7 @@
struct ib_fmr_pool_param fmr_param;
memset(&fmr_param, 0, sizeof(fmr_param));
- fmr_param.pool_size = target->scsi_host->can_queue;
+ fmr_param.pool_size = target->mr_pool_size;
fmr_param.dirty_watermark = fmr_param.pool_size / 4;
fmr_param.cache = 1;
fmr_param.max_pages_per_fmr = dev->max_pages_per_mr;
@@ -441,23 +445,22 @@
{
struct srp_device *dev = target->srp_host->srp_dev;
- return srp_create_fr_pool(dev->dev, dev->pd,
- target->scsi_host->can_queue,
+ return srp_create_fr_pool(dev->dev, dev->pd, target->mr_pool_size,
dev->max_pages_per_mr);
}
/**
* srp_destroy_qp() - destroy an RDMA queue pair
- * @ch: SRP RDMA channel.
+ * @qp: RDMA queue pair.
*
* Drain the qp before destroying it. This avoids that the receive
* completion handler can access the queue pair while it is
* being destroyed.
*/
-static void srp_destroy_qp(struct srp_rdma_ch *ch)
+static void srp_destroy_qp(struct ib_qp *qp)
{
- ib_drain_rq(ch->qp);
- ib_destroy_qp(ch->qp);
+ ib_drain_rq(qp);
+ ib_destroy_qp(qp);
}
static int srp_create_ch_ib(struct srp_rdma_ch *ch)
@@ -469,7 +472,7 @@
struct ib_qp *qp;
struct ib_fmr_pool *fmr_pool = NULL;
struct srp_fr_pool *fr_pool = NULL;
- const int m = dev->use_fast_reg ? 3 : 1;
+ const int m = 1 + dev->use_fast_reg * target->mr_per_cmd * 2;
int ret;
init_attr = kzalloc(sizeof *init_attr, GFP_KERNEL);
@@ -530,7 +533,7 @@
}
if (ch->qp)
- srp_destroy_qp(ch);
+ srp_destroy_qp(ch->qp);
if (ch->recv_cq)
ib_free_cq(ch->recv_cq);
if (ch->send_cq)
@@ -554,7 +557,7 @@
return 0;
err_qp:
- srp_destroy_qp(ch);
+ srp_destroy_qp(qp);
err_send_cq:
ib_free_cq(send_cq);
@@ -597,7 +600,7 @@
ib_destroy_fmr_pool(ch->fmr_pool);
}
- srp_destroy_qp(ch);
+ srp_destroy_qp(ch->qp);
ib_free_cq(ch->send_cq);
ib_free_cq(ch->recv_cq);
@@ -850,7 +853,7 @@
for (i = 0; i < target->req_ring_size; ++i) {
req = &ch->req_ring[i];
- mr_list = kmalloc(target->cmd_sg_cnt * sizeof(void *),
+ mr_list = kmalloc(target->mr_per_cmd * sizeof(void *),
GFP_KERNEL);
if (!mr_list)
goto out;
@@ -1112,7 +1115,7 @@
}
/**
- * srp_free_req() - Unmap data and add request to the free request list.
+ * srp_free_req() - Unmap data and adjust ch->req_lim.
* @ch: SRP RDMA channel.
* @req: Request to be freed.
* @scmnd: SCSI command associated with @req.
@@ -1299,9 +1302,16 @@
srp_handle_qp_err(cq, wc, "FAST REG");
}
+/*
+ * Map up to sg_nents elements of state->sg where *sg_offset_p is the offset
+ * where to start in the first element. If sg_offset_p != NULL then
+ * *sg_offset_p is updated to the offset in state->sg[retval] of the first
+ * byte that has not yet been mapped.
+ */
static int srp_map_finish_fr(struct srp_map_state *state,
struct srp_request *req,
- struct srp_rdma_ch *ch, int sg_nents)
+ struct srp_rdma_ch *ch, int sg_nents,
+ unsigned int *sg_offset_p)
{
struct srp_target_port *target = ch->target;
struct srp_device *dev = target->srp_host->srp_dev;
@@ -1316,13 +1326,14 @@
WARN_ON_ONCE(!dev->use_fast_reg);
- if (sg_nents == 0)
- return 0;
-
if (sg_nents == 1 && target->global_mr) {
- srp_map_desc(state, sg_dma_address(state->sg),
- sg_dma_len(state->sg),
+ unsigned int sg_offset = sg_offset_p ? *sg_offset_p : 0;
+
+ srp_map_desc(state, sg_dma_address(state->sg) + sg_offset,
+ sg_dma_len(state->sg) - sg_offset,
target->global_mr->rkey);
+ if (sg_offset_p)
+ *sg_offset_p = 0;
return 1;
}
@@ -1333,9 +1344,17 @@
rkey = ib_inc_rkey(desc->mr->rkey);
ib_update_fast_reg_key(desc->mr, rkey);
- n = ib_map_mr_sg(desc->mr, state->sg, sg_nents, dev->mr_page_size);
- if (unlikely(n < 0))
+ n = ib_map_mr_sg(desc->mr, state->sg, sg_nents, sg_offset_p,
+ dev->mr_page_size);
+ if (unlikely(n < 0)) {
+ srp_fr_pool_put(ch->fr_pool, &desc, 1);
+ pr_debug("%s: ib_map_mr_sg(%d, %d) returned %d.\n",
+ dev_name(&req->scmnd->device->sdev_gendev), sg_nents,
+ sg_offset_p ? *sg_offset_p : -1, n);
return n;
+ }
+
+ WARN_ON_ONCE(desc->mr->length == 0);
req->reg_cqe.done = srp_reg_mr_err_done;
@@ -1357,8 +1376,10 @@
desc->mr->length, desc->mr->rkey);
err = ib_post_send(ch->qp, &wr.wr, &bad_wr);
- if (unlikely(err))
+ if (unlikely(err)) {
+ WARN_ON_ONCE(err == -ENOMEM);
return err;
+ }
return n;
}
@@ -1398,7 +1419,7 @@
/*
* If the last entry of the MR wasn't a full page, then we need to
* close it out and start a new one -- we can only merge at page
- * boundries.
+ * boundaries.
*/
ret = 0;
if (len != dev->mr_page_size)
@@ -1413,10 +1434,9 @@
struct scatterlist *sg;
int i, ret;
- state->desc = req->indirect_desc;
state->pages = req->map_page;
state->fmr.next = req->fmr_list;
- state->fmr.end = req->fmr_list + ch->target->cmd_sg_cnt;
+ state->fmr.end = req->fmr_list + ch->target->mr_per_cmd;
for_each_sg(scat, sg, count, i) {
ret = srp_map_sg_entry(state, ch, sg, i);
@@ -1428,8 +1448,6 @@
if (ret)
return ret;
- req->nmdesc = state->nmdesc;
-
return 0;
}
@@ -1437,15 +1455,20 @@
struct srp_request *req, struct scatterlist *scat,
int count)
{
+ unsigned int sg_offset = 0;
+
state->desc = req->indirect_desc;
state->fr.next = req->fr_list;
- state->fr.end = req->fr_list + ch->target->cmd_sg_cnt;
+ state->fr.end = req->fr_list + ch->target->mr_per_cmd;
state->sg = scat;
+ if (count == 0)
+ return 0;
+
while (count) {
int i, n;
- n = srp_map_finish_fr(state, req, ch, count);
+ n = srp_map_finish_fr(state, req, ch, count, &sg_offset);
if (unlikely(n < 0))
return n;
@@ -1454,8 +1477,6 @@
state->sg = sg_next(state->sg);
}
- req->nmdesc = state->nmdesc;
-
return 0;
}
@@ -1475,8 +1496,6 @@
target->global_mr->rkey);
}
- req->nmdesc = state->nmdesc;
-
return 0;
}
@@ -1509,14 +1528,15 @@
if (dev->use_fast_reg) {
state.sg = idb_sg;
- sg_set_buf(idb_sg, req->indirect_desc, idb_len);
+ sg_init_one(idb_sg, req->indirect_desc, idb_len);
idb_sg->dma_address = req->indirect_dma_addr; /* hack! */
#ifdef CONFIG_NEED_SG_DMA_LENGTH
idb_sg->dma_length = idb_sg->length; /* hack^2 */
#endif
- ret = srp_map_finish_fr(&state, req, ch, 1);
+ ret = srp_map_finish_fr(&state, req, ch, 1, NULL);
if (ret < 0)
return ret;
+ WARN_ON_ONCE(ret < 1);
} else if (dev->use_fmr) {
state.pages = idb_pages;
state.pages[0] = (req->indirect_dma_addr &
@@ -1534,6 +1554,41 @@
return 0;
}
+#if defined(DYNAMIC_DATA_DEBUG)
+static void srp_check_mapping(struct srp_map_state *state,
+ struct srp_rdma_ch *ch, struct srp_request *req,
+ struct scatterlist *scat, int count)
+{
+ struct srp_device *dev = ch->target->srp_host->srp_dev;
+ struct srp_fr_desc **pfr;
+ u64 desc_len = 0, mr_len = 0;
+ int i;
+
+ for (i = 0; i < state->ndesc; i++)
+ desc_len += be32_to_cpu(req->indirect_desc[i].len);
+ if (dev->use_fast_reg)
+ for (i = 0, pfr = req->fr_list; i < state->nmdesc; i++, pfr++)
+ mr_len += (*pfr)->mr->length;
+ else if (dev->use_fmr)
+ for (i = 0; i < state->nmdesc; i++)
+ mr_len += be32_to_cpu(req->indirect_desc[i].len);
+ if (desc_len != scsi_bufflen(req->scmnd) ||
+ mr_len > scsi_bufflen(req->scmnd))
+ pr_err("Inconsistent: scsi len %d <> desc len %lld <> mr len %lld; ndesc %d; nmdesc = %d\n",
+ scsi_bufflen(req->scmnd), desc_len, mr_len,
+ state->ndesc, state->nmdesc);
+}
+#endif
+
+/**
+ * srp_map_data() - map SCSI data buffer onto an SRP request
+ * @scmnd: SCSI command to map
+ * @ch: SRP RDMA channel
+ * @req: SRP request
+ *
+ * Returns the length in bytes of the SRP_CMD IU or a negative value if
+ * mapping failed.
+ */
static int srp_map_data(struct scsi_cmnd *scmnd, struct srp_rdma_ch *ch,
struct srp_request *req)
{
@@ -1601,11 +1656,23 @@
memset(&state, 0, sizeof(state));
if (dev->use_fast_reg)
- srp_map_sg_fr(&state, ch, req, scat, count);
+ ret = srp_map_sg_fr(&state, ch, req, scat, count);
else if (dev->use_fmr)
- srp_map_sg_fmr(&state, ch, req, scat, count);
+ ret = srp_map_sg_fmr(&state, ch, req, scat, count);
else
- srp_map_sg_dma(&state, ch, req, scat, count);
+ ret = srp_map_sg_dma(&state, ch, req, scat, count);
+ req->nmdesc = state.nmdesc;
+ if (ret < 0)
+ goto unmap;
+
+#if defined(DYNAMIC_DEBUG)
+ {
+ DEFINE_DYNAMIC_DEBUG_METADATA(ddm,
+ "Memory mapping consistency check");
+ if (unlikely(ddm.flags & _DPRINTK_FLAGS_PRINT))
+ srp_check_mapping(&state, ch, req, scat, count);
+ }
+#endif
/* We've mapped the request, now pull as much of the indirect
* descriptor table as we can into the command buffer. If this
@@ -1628,7 +1695,8 @@
!target->allow_ext_sg)) {
shost_printk(KERN_ERR, target->scsi_host,
"Could not fit S/G list into SRP_CMD\n");
- return -EIO;
+ ret = -EIO;
+ goto unmap;
}
count = min(state.ndesc, target->cmd_sg_cnt);
@@ -1646,7 +1714,7 @@
ret = srp_map_idb(ch, req, state.gen.next, state.gen.end,
idb_len, &idb_rkey);
if (ret < 0)
- return ret;
+ goto unmap;
req->nmdesc++;
} else {
idb_rkey = cpu_to_be32(target->global_mr->rkey);
@@ -1672,6 +1740,12 @@
cmd->buf_fmt = fmt;
return len;
+
+unmap:
+ srp_unmap_data(scmnd, ch, req);
+ if (ret == -ENOMEM && req->nmdesc >= target->mr_pool_size)
+ ret = -E2BIG;
+ return ret;
}
/*
@@ -2564,6 +2638,20 @@
return srp_reconnect_rport(target->rport) == 0 ? SUCCESS : FAILED;
}
+static int srp_slave_alloc(struct scsi_device *sdev)
+{
+ struct Scsi_Host *shost = sdev->host;
+ struct srp_target_port *target = host_to_target(shost);
+ struct srp_device *srp_dev = target->srp_host->srp_dev;
+ struct ib_device *ibdev = srp_dev->dev;
+
+ if (!(ibdev->attrs.device_cap_flags & IB_DEVICE_SG_GAPS_REG))
+ blk_queue_virt_boundary(sdev->request_queue,
+ ~srp_dev->mr_page_mask);
+
+ return 0;
+}
+
static int srp_slave_configure(struct scsi_device *sdev)
{
struct Scsi_Host *shost = sdev->host;
@@ -2755,6 +2843,7 @@
.module = THIS_MODULE,
.name = "InfiniBand SRP initiator",
.proc_name = DRV_NAME,
+ .slave_alloc = srp_slave_alloc,
.slave_configure = srp_slave_configure,
.info = srp_target_info,
.queuecommand = srp_queuecommand,
@@ -2819,7 +2908,7 @@
spin_unlock(&host->target_lock);
scsi_scan_target(&target->scsi_host->shost_gendev,
- 0, target->scsi_id, SCAN_WILD_CARD, 0);
+ 0, target->scsi_id, SCAN_WILD_CARD, SCSI_SCAN_INITIAL);
if (srp_connected_ch(target) < target->ch_count ||
target->qp_in_error) {
@@ -2829,7 +2918,7 @@
goto out;
}
- pr_debug(PFX "%s: SCSI scan succeeded - detected %d LUNs\n",
+ pr_debug("%s: SCSI scan succeeded - detected %d LUNs\n",
dev_name(&target->scsi_host->shost_gendev),
srp_sdev_count(target->scsi_host));
@@ -3097,7 +3186,7 @@
case SRP_OPT_SG_TABLESIZE:
if (match_int(args, &token) || token < 1 ||
- token > SCSI_MAX_SG_CHAIN_SEGMENTS) {
+ token > SG_MAX_SEGMENTS) {
pr_warn("bad max sg_tablesize parameter '%s'\n",
p);
goto out;
@@ -3161,6 +3250,7 @@
struct srp_device *srp_dev = host->srp_dev;
struct ib_device *ibdev = srp_dev->dev;
int ret, node_idx, node, cpu, i;
+ unsigned int max_sectors_per_mr, mr_per_cmd = 0;
bool multich = false;
target_host = scsi_host_alloc(&srp_template,
@@ -3217,7 +3307,33 @@
target->sg_tablesize = target->cmd_sg_cnt;
}
+ if (srp_dev->use_fast_reg || srp_dev->use_fmr) {
+ /*
+ * FR and FMR can only map one HCA page per entry. If the
+ * start address is not aligned on a HCA page boundary two
+ * entries will be used for the head and the tail although
+ * these two entries combined contain at most one HCA page of
+ * data. Hence the "+ 1" in the calculation below.
+ *
+ * The indirect data buffer descriptor is contiguous so the
+ * memory for that buffer will only be registered if
+ * register_always is true. Hence add one to mr_per_cmd if
+ * register_always has been set.
+ */
+ max_sectors_per_mr = srp_dev->max_pages_per_mr <<
+ (ilog2(srp_dev->mr_page_size) - 9);
+ mr_per_cmd = register_always +
+ (target->scsi_host->max_sectors + 1 +
+ max_sectors_per_mr - 1) / max_sectors_per_mr;
+ pr_debug("max_sectors = %u; max_pages_per_mr = %u; mr_page_size = %u; max_sectors_per_mr = %u; mr_per_cmd = %u\n",
+ target->scsi_host->max_sectors,
+ srp_dev->max_pages_per_mr, srp_dev->mr_page_size,
+ max_sectors_per_mr, mr_per_cmd);
+ }
+
target_host->sg_tablesize = target->sg_tablesize;
+ target->mr_pool_size = target->scsi_host->can_queue * mr_per_cmd;
+ target->mr_per_cmd = mr_per_cmd;
target->indirect_size = target->sg_tablesize *
sizeof (struct srp_direct_buf);
target->max_iu_len = sizeof (struct srp_cmd) +
@@ -3414,17 +3530,6 @@
if (!srp_dev)
return;
- srp_dev->has_fmr = (device->alloc_fmr && device->dealloc_fmr &&
- device->map_phys_fmr && device->unmap_fmr);
- srp_dev->has_fr = (device->attrs.device_cap_flags &
- IB_DEVICE_MEM_MGT_EXTENSIONS);
- if (!srp_dev->has_fmr && !srp_dev->has_fr)
- dev_warn(&device->dev, "neither FMR nor FR is supported\n");
-
- srp_dev->use_fast_reg = (srp_dev->has_fr &&
- (!srp_dev->has_fmr || prefer_fr));
- srp_dev->use_fmr = !srp_dev->use_fast_reg && srp_dev->has_fmr;
-
/*
* Use the smallest page size supported by the HCA, down to a
* minimum of 4096 bytes. We're unlikely to build large sglists
@@ -3435,8 +3540,25 @@
srp_dev->mr_page_mask = ~((u64) srp_dev->mr_page_size - 1);
max_pages_per_mr = device->attrs.max_mr_size;
do_div(max_pages_per_mr, srp_dev->mr_page_size);
+ pr_debug("%s: %llu / %u = %llu <> %u\n", __func__,
+ device->attrs.max_mr_size, srp_dev->mr_page_size,
+ max_pages_per_mr, SRP_MAX_PAGES_PER_MR);
srp_dev->max_pages_per_mr = min_t(u64, SRP_MAX_PAGES_PER_MR,
max_pages_per_mr);
+
+ srp_dev->has_fmr = (device->alloc_fmr && device->dealloc_fmr &&
+ device->map_phys_fmr && device->unmap_fmr);
+ srp_dev->has_fr = (device->attrs.device_cap_flags &
+ IB_DEVICE_MEM_MGT_EXTENSIONS);
+ if (!never_register && !srp_dev->has_fmr && !srp_dev->has_fr) {
+ dev_warn(&device->dev, "neither FMR nor FR is supported\n");
+ } else if (!never_register &&
+ device->attrs.max_mr_size >= 2 * srp_dev->mr_page_size) {
+ srp_dev->use_fast_reg = (srp_dev->has_fr &&
+ (!srp_dev->has_fmr || prefer_fr));
+ srp_dev->use_fmr = !srp_dev->use_fast_reg && srp_dev->has_fmr;
+ }
+
if (srp_dev->use_fast_reg) {
srp_dev->max_pages_per_mr =
min_t(u32, srp_dev->max_pages_per_mr,
@@ -3456,7 +3578,8 @@
if (IS_ERR(srp_dev->pd))
goto free_dev;
- if (!register_always || (!srp_dev->has_fmr && !srp_dev->has_fr)) {
+ if (never_register || !register_always ||
+ (!srp_dev->has_fmr && !srp_dev->has_fr)) {
srp_dev->global_mr = ib_get_dma_mr(srp_dev->pd,
IB_ACCESS_LOCAL_WRITE |
IB_ACCESS_REMOTE_READ |
diff --git a/drivers/infiniband/ulp/srp/ib_srp.h b/drivers/infiniband/ulp/srp/ib_srp.h
index 9e05ce4..26bb9b0 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.h
+++ b/drivers/infiniband/ulp/srp/ib_srp.h
@@ -202,6 +202,8 @@
char target_name[32];
unsigned int scsi_id;
unsigned int sg_tablesize;
+ int mr_pool_size;
+ int mr_per_cmd;
int queue_size;
int req_ring_size;
int comp_vector;
diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c
index 8b42401..2843f1a 100644
--- a/drivers/infiniband/ulp/srpt/ib_srpt.c
+++ b/drivers/infiniband/ulp/srpt/ib_srpt.c
@@ -765,52 +765,6 @@
}
/**
- * srpt_post_send() - Post an IB send request.
- *
- * Returns zero upon success and a non-zero value upon failure.
- */
-static int srpt_post_send(struct srpt_rdma_ch *ch,
- struct srpt_send_ioctx *ioctx, int len)
-{
- struct ib_sge list;
- struct ib_send_wr wr, *bad_wr;
- struct srpt_device *sdev = ch->sport->sdev;
- int ret;
-
- atomic_inc(&ch->req_lim);
-
- ret = -ENOMEM;
- if (unlikely(atomic_dec_return(&ch->sq_wr_avail) < 0)) {
- pr_warn("IB send queue full (needed 1)\n");
- goto out;
- }
-
- ib_dma_sync_single_for_device(sdev->device, ioctx->ioctx.dma, len,
- DMA_TO_DEVICE);
-
- list.addr = ioctx->ioctx.dma;
- list.length = len;
- list.lkey = sdev->pd->local_dma_lkey;
-
- ioctx->ioctx.cqe.done = srpt_send_done;
- wr.next = NULL;
- wr.wr_cqe = &ioctx->ioctx.cqe;
- wr.sg_list = &list;
- wr.num_sge = 1;
- wr.opcode = IB_WR_SEND;
- wr.send_flags = IB_SEND_SIGNALED;
-
- ret = ib_post_send(ch->qp, &wr, &bad_wr);
-
-out:
- if (ret < 0) {
- atomic_inc(&ch->sq_wr_avail);
- atomic_dec(&ch->req_lim);
- }
- return ret;
-}
-
-/**
* srpt_zerolength_write() - Perform a zero-length RDMA write.
*
* A quote from the InfiniBand specification: C9-88: For an HCA responder
@@ -843,6 +797,110 @@
}
}
+static int srpt_alloc_rw_ctxs(struct srpt_send_ioctx *ioctx,
+ struct srp_direct_buf *db, int nbufs, struct scatterlist **sg,
+ unsigned *sg_cnt)
+{
+ enum dma_data_direction dir = target_reverse_dma_direction(&ioctx->cmd);
+ struct srpt_rdma_ch *ch = ioctx->ch;
+ struct scatterlist *prev = NULL;
+ unsigned prev_nents;
+ int ret, i;
+
+ if (nbufs == 1) {
+ ioctx->rw_ctxs = &ioctx->s_rw_ctx;
+ } else {
+ ioctx->rw_ctxs = kmalloc_array(nbufs, sizeof(*ioctx->rw_ctxs),
+ GFP_KERNEL);
+ if (!ioctx->rw_ctxs)
+ return -ENOMEM;
+ }
+
+ for (i = ioctx->n_rw_ctx; i < nbufs; i++, db++) {
+ struct srpt_rw_ctx *ctx = &ioctx->rw_ctxs[i];
+ u64 remote_addr = be64_to_cpu(db->va);
+ u32 size = be32_to_cpu(db->len);
+ u32 rkey = be32_to_cpu(db->key);
+
+ ret = target_alloc_sgl(&ctx->sg, &ctx->nents, size, false,
+ i < nbufs - 1);
+ if (ret)
+ goto unwind;
+
+ ret = rdma_rw_ctx_init(&ctx->rw, ch->qp, ch->sport->port,
+ ctx->sg, ctx->nents, 0, remote_addr, rkey, dir);
+ if (ret < 0) {
+ target_free_sgl(ctx->sg, ctx->nents);
+ goto unwind;
+ }
+
+ ioctx->n_rdma += ret;
+ ioctx->n_rw_ctx++;
+
+ if (prev) {
+ sg_unmark_end(&prev[prev_nents - 1]);
+ sg_chain(prev, prev_nents + 1, ctx->sg);
+ } else {
+ *sg = ctx->sg;
+ }
+
+ prev = ctx->sg;
+ prev_nents = ctx->nents;
+
+ *sg_cnt += ctx->nents;
+ }
+
+ return 0;
+
+unwind:
+ while (--i >= 0) {
+ struct srpt_rw_ctx *ctx = &ioctx->rw_ctxs[i];
+
+ rdma_rw_ctx_destroy(&ctx->rw, ch->qp, ch->sport->port,
+ ctx->sg, ctx->nents, dir);
+ target_free_sgl(ctx->sg, ctx->nents);
+ }
+ if (ioctx->rw_ctxs != &ioctx->s_rw_ctx)
+ kfree(ioctx->rw_ctxs);
+ return ret;
+}
+
+static void srpt_free_rw_ctxs(struct srpt_rdma_ch *ch,
+ struct srpt_send_ioctx *ioctx)
+{
+ enum dma_data_direction dir = target_reverse_dma_direction(&ioctx->cmd);
+ int i;
+
+ for (i = 0; i < ioctx->n_rw_ctx; i++) {
+ struct srpt_rw_ctx *ctx = &ioctx->rw_ctxs[i];
+
+ rdma_rw_ctx_destroy(&ctx->rw, ch->qp, ch->sport->port,
+ ctx->sg, ctx->nents, dir);
+ target_free_sgl(ctx->sg, ctx->nents);
+ }
+
+ if (ioctx->rw_ctxs != &ioctx->s_rw_ctx)
+ kfree(ioctx->rw_ctxs);
+}
+
+static inline void *srpt_get_desc_buf(struct srp_cmd *srp_cmd)
+{
+ /*
+ * The pointer computations below will only be compiled correctly
+ * if srp_cmd::add_data is declared as s8*, u8*, s8[] or u8[], so check
+ * whether srp_cmd::add_data has been declared as a byte pointer.
+ */
+ BUILD_BUG_ON(!__same_type(srp_cmd->add_data[0], (s8)0) &&
+ !__same_type(srp_cmd->add_data[0], (u8)0));
+
+ /*
+ * According to the SRP spec, the lower two bits of the 'ADDITIONAL
+ * CDB LENGTH' field are reserved and the size in bytes of this field
+ * is four times the value specified in bits 3..7. Hence the "& ~3".
+ */
+ return srp_cmd->add_data + (srp_cmd->add_cdb_len & ~3);
+}
+
/**
* srpt_get_desc_tbl() - Parse the data descriptors of an SRP_CMD request.
* @ioctx: Pointer to the I/O context associated with the request.
@@ -858,94 +916,59 @@
* -ENOMEM when memory allocation fails and zero upon success.
*/
static int srpt_get_desc_tbl(struct srpt_send_ioctx *ioctx,
- struct srp_cmd *srp_cmd,
- enum dma_data_direction *dir, u64 *data_len)
+ struct srp_cmd *srp_cmd, enum dma_data_direction *dir,
+ struct scatterlist **sg, unsigned *sg_cnt, u64 *data_len)
{
- struct srp_indirect_buf *idb;
- struct srp_direct_buf *db;
- unsigned add_cdb_offset;
- int ret;
-
- /*
- * The pointer computations below will only be compiled correctly
- * if srp_cmd::add_data is declared as s8*, u8*, s8[] or u8[], so check
- * whether srp_cmd::add_data has been declared as a byte pointer.
- */
- BUILD_BUG_ON(!__same_type(srp_cmd->add_data[0], (s8)0)
- && !__same_type(srp_cmd->add_data[0], (u8)0));
-
BUG_ON(!dir);
BUG_ON(!data_len);
- ret = 0;
- *data_len = 0;
-
/*
* The lower four bits of the buffer format field contain the DATA-IN
* buffer descriptor format, and the highest four bits contain the
* DATA-OUT buffer descriptor format.
*/
- *dir = DMA_NONE;
if (srp_cmd->buf_fmt & 0xf)
/* DATA-IN: transfer data from target to initiator (read). */
*dir = DMA_FROM_DEVICE;
else if (srp_cmd->buf_fmt >> 4)
/* DATA-OUT: transfer data from initiator to target (write). */
*dir = DMA_TO_DEVICE;
+ else
+ *dir = DMA_NONE;
- /*
- * According to the SRP spec, the lower two bits of the 'ADDITIONAL
- * CDB LENGTH' field are reserved and the size in bytes of this field
- * is four times the value specified in bits 3..7. Hence the "& ~3".
- */
- add_cdb_offset = srp_cmd->add_cdb_len & ~3;
+ /* initialize data_direction early as srpt_alloc_rw_ctxs needs it */
+ ioctx->cmd.data_direction = *dir;
+
if (((srp_cmd->buf_fmt & 0xf) == SRP_DATA_DESC_DIRECT) ||
((srp_cmd->buf_fmt >> 4) == SRP_DATA_DESC_DIRECT)) {
- ioctx->n_rbuf = 1;
- ioctx->rbufs = &ioctx->single_rbuf;
+ struct srp_direct_buf *db = srpt_get_desc_buf(srp_cmd);
- db = (struct srp_direct_buf *)(srp_cmd->add_data
- + add_cdb_offset);
- memcpy(ioctx->rbufs, db, sizeof(*db));
*data_len = be32_to_cpu(db->len);
+ return srpt_alloc_rw_ctxs(ioctx, db, 1, sg, sg_cnt);
} else if (((srp_cmd->buf_fmt & 0xf) == SRP_DATA_DESC_INDIRECT) ||
((srp_cmd->buf_fmt >> 4) == SRP_DATA_DESC_INDIRECT)) {
- idb = (struct srp_indirect_buf *)(srp_cmd->add_data
- + add_cdb_offset);
+ struct srp_indirect_buf *idb = srpt_get_desc_buf(srp_cmd);
+ int nbufs = be32_to_cpu(idb->table_desc.len) /
+ sizeof(struct srp_direct_buf);
- ioctx->n_rbuf = be32_to_cpu(idb->table_desc.len) / sizeof(*db);
-
- if (ioctx->n_rbuf >
+ if (nbufs >
(srp_cmd->data_out_desc_cnt + srp_cmd->data_in_desc_cnt)) {
pr_err("received unsupported SRP_CMD request"
" type (%u out + %u in != %u / %zu)\n",
srp_cmd->data_out_desc_cnt,
srp_cmd->data_in_desc_cnt,
be32_to_cpu(idb->table_desc.len),
- sizeof(*db));
- ioctx->n_rbuf = 0;
- ret = -EINVAL;
- goto out;
+ sizeof(struct srp_direct_buf));
+ return -EINVAL;
}
- if (ioctx->n_rbuf == 1)
- ioctx->rbufs = &ioctx->single_rbuf;
- else {
- ioctx->rbufs =
- kmalloc(ioctx->n_rbuf * sizeof(*db), GFP_ATOMIC);
- if (!ioctx->rbufs) {
- ioctx->n_rbuf = 0;
- ret = -ENOMEM;
- goto out;
- }
- }
-
- db = idb->desc_list;
- memcpy(ioctx->rbufs, db, ioctx->n_rbuf * sizeof(*db));
*data_len = be32_to_cpu(idb->len);
+ return srpt_alloc_rw_ctxs(ioctx, idb->desc_list, nbufs,
+ sg, sg_cnt);
+ } else {
+ *data_len = 0;
+ return 0;
}
-out:
- return ret;
}
/**
@@ -1049,217 +1072,6 @@
}
/**
- * srpt_unmap_sg_to_ib_sge() - Unmap an IB SGE list.
- */
-static void srpt_unmap_sg_to_ib_sge(struct srpt_rdma_ch *ch,
- struct srpt_send_ioctx *ioctx)
-{
- struct scatterlist *sg;
- enum dma_data_direction dir;
-
- BUG_ON(!ch);
- BUG_ON(!ioctx);
- BUG_ON(ioctx->n_rdma && !ioctx->rdma_wrs);
-
- while (ioctx->n_rdma)
- kfree(ioctx->rdma_wrs[--ioctx->n_rdma].wr.sg_list);
-
- kfree(ioctx->rdma_wrs);
- ioctx->rdma_wrs = NULL;
-
- if (ioctx->mapped_sg_count) {
- sg = ioctx->sg;
- WARN_ON(!sg);
- dir = ioctx->cmd.data_direction;
- BUG_ON(dir == DMA_NONE);
- ib_dma_unmap_sg(ch->sport->sdev->device, sg, ioctx->sg_cnt,
- target_reverse_dma_direction(&ioctx->cmd));
- ioctx->mapped_sg_count = 0;
- }
-}
-
-/**
- * srpt_map_sg_to_ib_sge() - Map an SG list to an IB SGE list.
- */
-static int srpt_map_sg_to_ib_sge(struct srpt_rdma_ch *ch,
- struct srpt_send_ioctx *ioctx)
-{
- struct ib_device *dev = ch->sport->sdev->device;
- struct se_cmd *cmd;
- struct scatterlist *sg, *sg_orig;
- int sg_cnt;
- enum dma_data_direction dir;
- struct ib_rdma_wr *riu;
- struct srp_direct_buf *db;
- dma_addr_t dma_addr;
- struct ib_sge *sge;
- u64 raddr;
- u32 rsize;
- u32 tsize;
- u32 dma_len;
- int count, nrdma;
- int i, j, k;
-
- BUG_ON(!ch);
- BUG_ON(!ioctx);
- cmd = &ioctx->cmd;
- dir = cmd->data_direction;
- BUG_ON(dir == DMA_NONE);
-
- ioctx->sg = sg = sg_orig = cmd->t_data_sg;
- ioctx->sg_cnt = sg_cnt = cmd->t_data_nents;
-
- count = ib_dma_map_sg(ch->sport->sdev->device, sg, sg_cnt,
- target_reverse_dma_direction(cmd));
- if (unlikely(!count))
- return -EAGAIN;
-
- ioctx->mapped_sg_count = count;
-
- if (ioctx->rdma_wrs && ioctx->n_rdma_wrs)
- nrdma = ioctx->n_rdma_wrs;
- else {
- nrdma = (count + SRPT_DEF_SG_PER_WQE - 1) / SRPT_DEF_SG_PER_WQE
- + ioctx->n_rbuf;
-
- ioctx->rdma_wrs = kcalloc(nrdma, sizeof(*ioctx->rdma_wrs),
- GFP_KERNEL);
- if (!ioctx->rdma_wrs)
- goto free_mem;
-
- ioctx->n_rdma_wrs = nrdma;
- }
-
- db = ioctx->rbufs;
- tsize = cmd->data_length;
- dma_len = ib_sg_dma_len(dev, &sg[0]);
- riu = ioctx->rdma_wrs;
-
- /*
- * For each remote desc - calculate the #ib_sge.
- * If #ib_sge < SRPT_DEF_SG_PER_WQE per rdma operation then
- * each remote desc rdma_iu is required a rdma wr;
- * else
- * we need to allocate extra rdma_iu to carry extra #ib_sge in
- * another rdma wr
- */
- for (i = 0, j = 0;
- j < count && i < ioctx->n_rbuf && tsize > 0; ++i, ++riu, ++db) {
- rsize = be32_to_cpu(db->len);
- raddr = be64_to_cpu(db->va);
- riu->remote_addr = raddr;
- riu->rkey = be32_to_cpu(db->key);
- riu->wr.num_sge = 0;
-
- /* calculate how many sge required for this remote_buf */
- while (rsize > 0 && tsize > 0) {
-
- if (rsize >= dma_len) {
- tsize -= dma_len;
- rsize -= dma_len;
- raddr += dma_len;
-
- if (tsize > 0) {
- ++j;
- if (j < count) {
- sg = sg_next(sg);
- dma_len = ib_sg_dma_len(
- dev, sg);
- }
- }
- } else {
- tsize -= rsize;
- dma_len -= rsize;
- rsize = 0;
- }
-
- ++riu->wr.num_sge;
-
- if (rsize > 0 &&
- riu->wr.num_sge == SRPT_DEF_SG_PER_WQE) {
- ++ioctx->n_rdma;
- riu->wr.sg_list = kmalloc_array(riu->wr.num_sge,
- sizeof(*riu->wr.sg_list),
- GFP_KERNEL);
- if (!riu->wr.sg_list)
- goto free_mem;
-
- ++riu;
- riu->wr.num_sge = 0;
- riu->remote_addr = raddr;
- riu->rkey = be32_to_cpu(db->key);
- }
- }
-
- ++ioctx->n_rdma;
- riu->wr.sg_list = kmalloc_array(riu->wr.num_sge,
- sizeof(*riu->wr.sg_list),
- GFP_KERNEL);
- if (!riu->wr.sg_list)
- goto free_mem;
- }
-
- db = ioctx->rbufs;
- tsize = cmd->data_length;
- riu = ioctx->rdma_wrs;
- sg = sg_orig;
- dma_len = ib_sg_dma_len(dev, &sg[0]);
- dma_addr = ib_sg_dma_address(dev, &sg[0]);
-
- /* this second loop is really mapped sg_addres to rdma_iu->ib_sge */
- for (i = 0, j = 0;
- j < count && i < ioctx->n_rbuf && tsize > 0; ++i, ++riu, ++db) {
- rsize = be32_to_cpu(db->len);
- sge = riu->wr.sg_list;
- k = 0;
-
- while (rsize > 0 && tsize > 0) {
- sge->addr = dma_addr;
- sge->lkey = ch->sport->sdev->pd->local_dma_lkey;
-
- if (rsize >= dma_len) {
- sge->length =
- (tsize < dma_len) ? tsize : dma_len;
- tsize -= dma_len;
- rsize -= dma_len;
-
- if (tsize > 0) {
- ++j;
- if (j < count) {
- sg = sg_next(sg);
- dma_len = ib_sg_dma_len(
- dev, sg);
- dma_addr = ib_sg_dma_address(
- dev, sg);
- }
- }
- } else {
- sge->length = (tsize < rsize) ? tsize : rsize;
- tsize -= rsize;
- dma_len -= rsize;
- dma_addr += rsize;
- rsize = 0;
- }
-
- ++k;
- if (k == riu->wr.num_sge && rsize > 0 && tsize > 0) {
- ++riu;
- sge = riu->wr.sg_list;
- k = 0;
- } else if (rsize > 0 && tsize > 0)
- ++sge;
- }
- }
-
- return 0;
-
-free_mem:
- srpt_unmap_sg_to_ib_sge(ch, ioctx);
-
- return -ENOMEM;
-}
-
-/**
* srpt_get_send_ioctx() - Obtain an I/O context for sending to the initiator.
*/
static struct srpt_send_ioctx *srpt_get_send_ioctx(struct srpt_rdma_ch *ch)
@@ -1284,12 +1096,8 @@
BUG_ON(ioctx->ch != ch);
spin_lock_init(&ioctx->spinlock);
ioctx->state = SRPT_STATE_NEW;
- ioctx->n_rbuf = 0;
- ioctx->rbufs = NULL;
ioctx->n_rdma = 0;
- ioctx->n_rdma_wrs = 0;
- ioctx->rdma_wrs = NULL;
- ioctx->mapped_sg_count = 0;
+ ioctx->n_rw_ctx = 0;
init_completion(&ioctx->tx_done);
ioctx->queue_status_only = false;
/*
@@ -1359,7 +1167,6 @@
* SRP_RSP sending failed or the SRP_RSP send completion has
* not been received in time.
*/
- srpt_unmap_sg_to_ib_sge(ioctx->ch, ioctx);
transport_generic_free_cmd(&ioctx->cmd, 0);
break;
case SRPT_STATE_MGMT_RSP_SENT:
@@ -1387,6 +1194,7 @@
WARN_ON(ioctx->n_rdma <= 0);
atomic_add(ioctx->n_rdma, &ch->sq_wr_avail);
+ ioctx->n_rdma = 0;
if (unlikely(wc->status != IB_WC_SUCCESS)) {
pr_info("RDMA_READ for ioctx 0x%p failed with status %d\n",
@@ -1403,23 +1211,6 @@
__LINE__, srpt_get_cmd_state(ioctx));
}
-static void srpt_rdma_write_done(struct ib_cq *cq, struct ib_wc *wc)
-{
- struct srpt_send_ioctx *ioctx =
- container_of(wc->wr_cqe, struct srpt_send_ioctx, rdma_cqe);
-
- if (unlikely(wc->status != IB_WC_SUCCESS)) {
- /*
- * Note: if an RDMA write error completion is received that
- * means that a SEND also has been posted. Defer further
- * processing of the associated command until the send error
- * completion has been received.
- */
- pr_info("RDMA_WRITE for ioctx 0x%p failed with status %d\n",
- ioctx, wc->status);
- }
-}
-
/**
* srpt_build_cmd_rsp() - Build an SRP_RSP response.
* @ch: RDMA channel through which the request has been received.
@@ -1537,6 +1328,8 @@
{
struct se_cmd *cmd;
struct srp_cmd *srp_cmd;
+ struct scatterlist *sg = NULL;
+ unsigned sg_cnt = 0;
u64 data_len;
enum dma_data_direction dir;
int rc;
@@ -1563,16 +1356,21 @@
break;
}
- if (srpt_get_desc_tbl(send_ioctx, srp_cmd, &dir, &data_len)) {
- pr_err("0x%llx: parsing SRP descriptor table failed.\n",
- srp_cmd->tag);
+ rc = srpt_get_desc_tbl(send_ioctx, srp_cmd, &dir, &sg, &sg_cnt,
+ &data_len);
+ if (rc) {
+ if (rc != -EAGAIN) {
+ pr_err("0x%llx: parsing SRP descriptor table failed.\n",
+ srp_cmd->tag);
+ }
goto release_ioctx;
}
- rc = target_submit_cmd(cmd, ch->sess, srp_cmd->cdb,
+ rc = target_submit_cmd_map_sgls(cmd, ch->sess, srp_cmd->cdb,
&send_ioctx->sense_data[0],
scsilun_to_int(&srp_cmd->lun), data_len,
- TCM_SIMPLE_TAG, dir, TARGET_SCF_ACK_KREF);
+ TCM_SIMPLE_TAG, dir, TARGET_SCF_ACK_KREF,
+ sg, sg_cnt, NULL, 0, NULL, 0);
if (rc != 0) {
pr_debug("target_submit_cmd() returned %d for tag %#llx\n", rc,
srp_cmd->tag);
@@ -1664,23 +1462,21 @@
recv_ioctx->ioctx.dma, srp_max_req_size,
DMA_FROM_DEVICE);
- if (unlikely(ch->state == CH_CONNECTING)) {
- list_add_tail(&recv_ioctx->wait_list, &ch->cmd_wait_list);
- goto out;
- }
+ if (unlikely(ch->state == CH_CONNECTING))
+ goto out_wait;
if (unlikely(ch->state != CH_LIVE))
- goto out;
+ return;
srp_cmd = recv_ioctx->ioctx.buf;
if (srp_cmd->opcode == SRP_CMD || srp_cmd->opcode == SRP_TSK_MGMT) {
- if (!send_ioctx)
+ if (!send_ioctx) {
+ if (!list_empty(&ch->cmd_wait_list))
+ goto out_wait;
send_ioctx = srpt_get_send_ioctx(ch);
- if (unlikely(!send_ioctx)) {
- list_add_tail(&recv_ioctx->wait_list,
- &ch->cmd_wait_list);
- goto out;
}
+ if (unlikely(!send_ioctx))
+ goto out_wait;
}
switch (srp_cmd->opcode) {
@@ -1709,8 +1505,10 @@
}
srpt_post_recv(ch->sport->sdev, recv_ioctx);
-out:
return;
+
+out_wait:
+ list_add_tail(&recv_ioctx->wait_list, &ch->cmd_wait_list);
}
static void srpt_recv_done(struct ib_cq *cq, struct ib_wc *wc)
@@ -1779,14 +1577,13 @@
WARN_ON(state != SRPT_STATE_CMD_RSP_SENT &&
state != SRPT_STATE_MGMT_RSP_SENT);
- atomic_inc(&ch->sq_wr_avail);
+ atomic_add(1 + ioctx->n_rdma, &ch->sq_wr_avail);
if (wc->status != IB_WC_SUCCESS)
pr_info("sending response for ioctx 0x%p failed"
" with status %d\n", ioctx, wc->status);
if (state != SRPT_STATE_DONE) {
- srpt_unmap_sg_to_ib_sge(ch, ioctx);
transport_generic_free_cmd(&ioctx->cmd, 0);
} else {
pr_err("IB completion has been received too late for"
@@ -1832,8 +1629,18 @@
qp_init->srq = sdev->srq;
qp_init->sq_sig_type = IB_SIGNAL_REQ_WR;
qp_init->qp_type = IB_QPT_RC;
- qp_init->cap.max_send_wr = srp_sq_size;
- qp_init->cap.max_send_sge = SRPT_DEF_SG_PER_WQE;
+ /*
+ * We divide up our send queue size into half SEND WRs to send the
+ * completions, and half R/W contexts to actually do the RDMA
+ * READ/WRITE transfers. Note that we need to allocate CQ slots for
+ * both both, as RDMA contexts will also post completions for the
+ * RDMA READ case.
+ */
+ qp_init->cap.max_send_wr = srp_sq_size / 2;
+ qp_init->cap.max_rdma_ctxs = srp_sq_size / 2;
+ qp_init->cap.max_send_sge = max(sdev->device->attrs.max_sge_rd,
+ sdev->device->attrs.max_sge);
+ qp_init->port_num = ch->sport->port;
ch->qp = ib_create_qp(sdev->pd, qp_init);
if (IS_ERR(ch->qp)) {
@@ -2386,95 +2193,6 @@
return ret;
}
-/**
- * srpt_perform_rdmas() - Perform IB RDMA.
- *
- * Returns zero upon success or a negative number upon failure.
- */
-static int srpt_perform_rdmas(struct srpt_rdma_ch *ch,
- struct srpt_send_ioctx *ioctx)
-{
- struct ib_send_wr *bad_wr;
- int sq_wr_avail, ret, i;
- enum dma_data_direction dir;
- const int n_rdma = ioctx->n_rdma;
-
- dir = ioctx->cmd.data_direction;
- if (dir == DMA_TO_DEVICE) {
- /* write */
- ret = -ENOMEM;
- sq_wr_avail = atomic_sub_return(n_rdma, &ch->sq_wr_avail);
- if (sq_wr_avail < 0) {
- pr_warn("IB send queue full (needed %d)\n",
- n_rdma);
- goto out;
- }
- }
-
- for (i = 0; i < n_rdma; i++) {
- struct ib_send_wr *wr = &ioctx->rdma_wrs[i].wr;
-
- wr->opcode = (dir == DMA_FROM_DEVICE) ?
- IB_WR_RDMA_WRITE : IB_WR_RDMA_READ;
-
- if (i == n_rdma - 1) {
- /* only get completion event for the last rdma read */
- if (dir == DMA_TO_DEVICE) {
- wr->send_flags = IB_SEND_SIGNALED;
- ioctx->rdma_cqe.done = srpt_rdma_read_done;
- } else {
- ioctx->rdma_cqe.done = srpt_rdma_write_done;
- }
- wr->wr_cqe = &ioctx->rdma_cqe;
- wr->next = NULL;
- } else {
- wr->wr_cqe = NULL;
- wr->next = &ioctx->rdma_wrs[i + 1].wr;
- }
- }
-
- ret = ib_post_send(ch->qp, &ioctx->rdma_wrs->wr, &bad_wr);
- if (ret)
- pr_err("%s[%d]: ib_post_send() returned %d for %d/%d\n",
- __func__, __LINE__, ret, i, n_rdma);
-out:
- if (unlikely(dir == DMA_TO_DEVICE && ret < 0))
- atomic_add(n_rdma, &ch->sq_wr_avail);
- return ret;
-}
-
-/**
- * srpt_xfer_data() - Start data transfer from initiator to target.
- */
-static int srpt_xfer_data(struct srpt_rdma_ch *ch,
- struct srpt_send_ioctx *ioctx)
-{
- int ret;
-
- ret = srpt_map_sg_to_ib_sge(ch, ioctx);
- if (ret) {
- pr_err("%s[%d] ret=%d\n", __func__, __LINE__, ret);
- goto out;
- }
-
- ret = srpt_perform_rdmas(ch, ioctx);
- if (ret) {
- if (ret == -EAGAIN || ret == -ENOMEM)
- pr_info("%s[%d] queue full -- ret=%d\n",
- __func__, __LINE__, ret);
- else
- pr_err("%s[%d] fatal error -- ret=%d\n",
- __func__, __LINE__, ret);
- goto out_unmap;
- }
-
-out:
- return ret;
-out_unmap:
- srpt_unmap_sg_to_ib_sge(ch, ioctx);
- goto out;
-}
-
static int srpt_write_pending_status(struct se_cmd *se_cmd)
{
struct srpt_send_ioctx *ioctx;
@@ -2491,11 +2209,42 @@
struct srpt_send_ioctx *ioctx =
container_of(se_cmd, struct srpt_send_ioctx, cmd);
struct srpt_rdma_ch *ch = ioctx->ch;
+ struct ib_send_wr *first_wr = NULL, *bad_wr;
+ struct ib_cqe *cqe = &ioctx->rdma_cqe;
enum srpt_command_state new_state;
+ int ret, i;
new_state = srpt_set_cmd_state(ioctx, SRPT_STATE_NEED_DATA);
WARN_ON(new_state == SRPT_STATE_DONE);
- return srpt_xfer_data(ch, ioctx);
+
+ if (atomic_sub_return(ioctx->n_rdma, &ch->sq_wr_avail) < 0) {
+ pr_warn("%s: IB send queue full (needed %d)\n",
+ __func__, ioctx->n_rdma);
+ ret = -ENOMEM;
+ goto out_undo;
+ }
+
+ cqe->done = srpt_rdma_read_done;
+ for (i = ioctx->n_rw_ctx - 1; i >= 0; i--) {
+ struct srpt_rw_ctx *ctx = &ioctx->rw_ctxs[i];
+
+ first_wr = rdma_rw_ctx_wrs(&ctx->rw, ch->qp, ch->sport->port,
+ cqe, first_wr);
+ cqe = NULL;
+ }
+
+ ret = ib_post_send(ch->qp, first_wr, &bad_wr);
+ if (ret) {
+ pr_err("%s: ib_post_send() returned %d for %d (avail: %d)\n",
+ __func__, ret, ioctx->n_rdma,
+ atomic_read(&ch->sq_wr_avail));
+ goto out_undo;
+ }
+
+ return 0;
+out_undo:
+ atomic_add(ioctx->n_rdma, &ch->sq_wr_avail);
+ return ret;
}
static u8 tcm_to_srp_tsk_mgmt_status(const int tcm_mgmt_status)
@@ -2517,17 +2266,17 @@
*/
static void srpt_queue_response(struct se_cmd *cmd)
{
- struct srpt_rdma_ch *ch;
- struct srpt_send_ioctx *ioctx;
+ struct srpt_send_ioctx *ioctx =
+ container_of(cmd, struct srpt_send_ioctx, cmd);
+ struct srpt_rdma_ch *ch = ioctx->ch;
+ struct srpt_device *sdev = ch->sport->sdev;
+ struct ib_send_wr send_wr, *first_wr = NULL, *bad_wr;
+ struct ib_sge sge;
enum srpt_command_state state;
unsigned long flags;
- int ret;
- enum dma_data_direction dir;
- int resp_len;
+ int resp_len, ret, i;
u8 srp_tm_status;
- ioctx = container_of(cmd, struct srpt_send_ioctx, cmd);
- ch = ioctx->ch;
BUG_ON(!ch);
spin_lock_irqsave(&ioctx->spinlock, flags);
@@ -2554,17 +2303,19 @@
return;
}
- dir = ioctx->cmd.data_direction;
-
/* For read commands, transfer the data to the initiator. */
- if (dir == DMA_FROM_DEVICE && ioctx->cmd.data_length &&
+ if (ioctx->cmd.data_direction == DMA_FROM_DEVICE &&
+ ioctx->cmd.data_length &&
!ioctx->queue_status_only) {
- ret = srpt_xfer_data(ch, ioctx);
- if (ret) {
- pr_err("xfer_data failed for tag %llu\n",
- ioctx->cmd.tag);
- return;
+ for (i = ioctx->n_rw_ctx - 1; i >= 0; i--) {
+ struct srpt_rw_ctx *ctx = &ioctx->rw_ctxs[i];
+
+ first_wr = rdma_rw_ctx_wrs(&ctx->rw, ch->qp,
+ ch->sport->port, NULL,
+ first_wr ? first_wr : &send_wr);
}
+ } else {
+ first_wr = &send_wr;
}
if (state != SRPT_STATE_MGMT)
@@ -2576,14 +2327,46 @@
resp_len = srpt_build_tskmgmt_rsp(ch, ioctx, srp_tm_status,
ioctx->cmd.tag);
}
- ret = srpt_post_send(ch, ioctx, resp_len);
- if (ret) {
- pr_err("sending cmd response failed for tag %llu\n",
- ioctx->cmd.tag);
- srpt_unmap_sg_to_ib_sge(ch, ioctx);
- srpt_set_cmd_state(ioctx, SRPT_STATE_DONE);
- target_put_sess_cmd(&ioctx->cmd);
+
+ atomic_inc(&ch->req_lim);
+
+ if (unlikely(atomic_sub_return(1 + ioctx->n_rdma,
+ &ch->sq_wr_avail) < 0)) {
+ pr_warn("%s: IB send queue full (needed %d)\n",
+ __func__, ioctx->n_rdma);
+ ret = -ENOMEM;
+ goto out;
}
+
+ ib_dma_sync_single_for_device(sdev->device, ioctx->ioctx.dma, resp_len,
+ DMA_TO_DEVICE);
+
+ sge.addr = ioctx->ioctx.dma;
+ sge.length = resp_len;
+ sge.lkey = sdev->pd->local_dma_lkey;
+
+ ioctx->ioctx.cqe.done = srpt_send_done;
+ send_wr.next = NULL;
+ send_wr.wr_cqe = &ioctx->ioctx.cqe;
+ send_wr.sg_list = &sge;
+ send_wr.num_sge = 1;
+ send_wr.opcode = IB_WR_SEND;
+ send_wr.send_flags = IB_SEND_SIGNALED;
+
+ ret = ib_post_send(ch->qp, first_wr, &bad_wr);
+ if (ret < 0) {
+ pr_err("%s: sending cmd response failed for tag %llu (%d)\n",
+ __func__, ioctx->cmd.tag, ret);
+ goto out;
+ }
+
+ return;
+
+out:
+ atomic_add(1 + ioctx->n_rdma, &ch->sq_wr_avail);
+ atomic_dec(&ch->req_lim);
+ srpt_set_cmd_state(ioctx, SRPT_STATE_DONE);
+ target_put_sess_cmd(&ioctx->cmd);
}
static int srpt_queue_data_in(struct se_cmd *cmd)
@@ -2599,10 +2382,6 @@
static void srpt_aborted_task(struct se_cmd *cmd)
{
- struct srpt_send_ioctx *ioctx = container_of(cmd,
- struct srpt_send_ioctx, cmd);
-
- srpt_unmap_sg_to_ib_sge(ioctx->ch, ioctx);
}
static int srpt_queue_status(struct se_cmd *cmd)
@@ -2903,12 +2682,10 @@
unsigned long flags;
WARN_ON(ioctx->state != SRPT_STATE_DONE);
- WARN_ON(ioctx->mapped_sg_count != 0);
- if (ioctx->n_rbuf > 1) {
- kfree(ioctx->rbufs);
- ioctx->rbufs = NULL;
- ioctx->n_rbuf = 0;
+ if (ioctx->n_rw_ctx) {
+ srpt_free_rw_ctxs(ch, ioctx);
+ ioctx->n_rw_ctx = 0;
}
spin_lock_irqsave(&ch->spinlock, flags);
diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.h b/drivers/infiniband/ulp/srpt/ib_srpt.h
index af9b8b5..fee6bfd 100644
--- a/drivers/infiniband/ulp/srpt/ib_srpt.h
+++ b/drivers/infiniband/ulp/srpt/ib_srpt.h
@@ -42,6 +42,7 @@
#include <rdma/ib_verbs.h>
#include <rdma/ib_sa.h>
#include <rdma/ib_cm.h>
+#include <rdma/rw.h>
#include <scsi/srp.h>
@@ -105,7 +106,6 @@
SRP_LOGIN_RSP_MULTICHAN_MAINTAINED = 0x2,
SRPT_DEF_SG_TABLESIZE = 128,
- SRPT_DEF_SG_PER_WQE = 16,
MIN_SRPT_SQ_SIZE = 16,
DEF_SRPT_SQ_SIZE = 4096,
@@ -174,21 +174,17 @@
struct srpt_ioctx ioctx;
struct list_head wait_list;
};
+
+struct srpt_rw_ctx {
+ struct rdma_rw_ctx rw;
+ struct scatterlist *sg;
+ unsigned int nents;
+};
/**
* struct srpt_send_ioctx - SRPT send I/O context.
* @ioctx: See above.
* @ch: Channel pointer.
- * @free_list: Node in srpt_rdma_ch.free_list.
- * @n_rbuf: Number of data buffers in the received SRP command.
- * @rbufs: Pointer to SRP data buffer array.
- * @single_rbuf: SRP data buffer if the command has only a single buffer.
- * @sg: Pointer to sg-list associated with this I/O context.
- * @sg_cnt: SG-list size.
- * @mapped_sg_count: ib_dma_map_sg() return value.
- * @n_rdma_wrs: Number of elements in the rdma_wrs array.
- * @rdma_wrs: Array with information about the RDMA mapping.
- * @tag: Tag of the received SRP information unit.
* @spinlock: Protects 'state'.
* @state: I/O context state.
* @cmd: Target core command data structure.
@@ -197,21 +193,18 @@
struct srpt_send_ioctx {
struct srpt_ioctx ioctx;
struct srpt_rdma_ch *ch;
- struct ib_rdma_wr *rdma_wrs;
+
+ struct srpt_rw_ctx s_rw_ctx;
+ struct srpt_rw_ctx *rw_ctxs;
+
struct ib_cqe rdma_cqe;
- struct srp_direct_buf *rbufs;
- struct srp_direct_buf single_rbuf;
- struct scatterlist *sg;
struct list_head free_list;
spinlock_t spinlock;
enum srpt_command_state state;
struct se_cmd cmd;
struct completion tx_done;
- int sg_cnt;
- int mapped_sg_count;
- u16 n_rdma_wrs;
u8 n_rdma;
- u8 n_rbuf;
+ u8 n_rw_ctx;
bool queue_status_only;
u8 sense_data[TRANSPORT_SENSE_BUFFER];
};
diff --git a/drivers/input/keyboard/omap-keypad.c b/drivers/input/keyboard/omap-keypad.c
index e0d72c8..146b26f 100644
--- a/drivers/input/keyboard/omap-keypad.c
+++ b/drivers/input/keyboard/omap-keypad.c
@@ -64,31 +64,6 @@
static unsigned int *row_gpios;
static unsigned int *col_gpios;
-#ifdef CONFIG_ARCH_OMAP2
-static void set_col_gpio_val(struct omap_kp *omap_kp, u8 value)
-{
- int col;
-
- for (col = 0; col < omap_kp->cols; col++)
- gpio_set_value(col_gpios[col], value & (1 << col));
-}
-
-static u8 get_row_gpio_val(struct omap_kp *omap_kp)
-{
- int row;
- u8 value = 0;
-
- for (row = 0; row < omap_kp->rows; row++) {
- if (gpio_get_value(row_gpios[row]))
- value |= (1 << row);
- }
- return value;
-}
-#else
-#define set_col_gpio_val(x, y) do {} while (0)
-#define get_row_gpio_val(x) 0
-#endif
-
static irqreturn_t omap_kp_interrupt(int irq, void *dev_id)
{
/* disable keyboard interrupt and schedule for handling */
@@ -133,7 +108,6 @@
unsigned int row_shift = get_count_order(omap_kp_data->cols);
unsigned char new_state[8], changed, key_down = 0;
int col, row;
- int spurious = 0;
/* check for any changes */
omap_kp_scan_keypad(omap_kp_data, new_state);
@@ -170,12 +144,9 @@
memcpy(keypad_state, new_state, sizeof(keypad_state));
if (key_down) {
- int delay = HZ / 20;
/* some key is pressed - keep irq disabled and use timer
* to poll the keypad */
- if (spurious)
- delay = 2 * HZ;
- mod_timer(&omap_kp_data->timer, jiffies + delay);
+ mod_timer(&omap_kp_data->timer, jiffies + HZ / 20);
} else {
/* enable interrupts */
omap_writew(0, OMAP1_MPUIO_BASE + OMAP_MPUIO_KBD_MASKIT);
@@ -216,25 +187,6 @@
static DEVICE_ATTR(enable, S_IRUGO | S_IWUSR, omap_kp_enable_show, omap_kp_enable_store);
-#ifdef CONFIG_PM
-static int omap_kp_suspend(struct platform_device *dev, pm_message_t state)
-{
- /* Nothing yet */
-
- return 0;
-}
-
-static int omap_kp_resume(struct platform_device *dev)
-{
- /* Nothing yet */
-
- return 0;
-}
-#else
-#define omap_kp_suspend NULL
-#define omap_kp_resume NULL
-#endif
-
static int omap_kp_probe(struct platform_device *pdev)
{
struct omap_kp *omap_kp;
@@ -371,8 +323,6 @@
static struct platform_driver omap_kp_driver = {
.probe = omap_kp_probe,
.remove = omap_kp_remove,
- .suspend = omap_kp_suspend,
- .resume = omap_kp_resume,
.driver = {
.name = "omap-keypad",
},
diff --git a/drivers/input/keyboard/twl4030_keypad.c b/drivers/input/keyboard/twl4030_keypad.c
index bbcccd6..323a0fb 100644
--- a/drivers/input/keyboard/twl4030_keypad.c
+++ b/drivers/input/keyboard/twl4030_keypad.c
@@ -61,9 +61,9 @@
unsigned short keymap[TWL4030_KEYMAP_SIZE];
u16 kp_state[TWL4030_MAX_ROWS];
bool autorepeat;
- unsigned n_rows;
- unsigned n_cols;
- unsigned irq;
+ unsigned int n_rows;
+ unsigned int n_cols;
+ unsigned int irq;
struct device *dbg_dev;
struct input_dev *input;
@@ -110,7 +110,7 @@
#define KEYP_CTRL_KBD_ON BIT(6)
/* KEYP_DEB, KEYP_LONG_KEY, KEYP_TIMEOUT_x*/
-#define KEYP_PERIOD_US(t, prescale) ((t) / (31 << (prescale + 1)) - 1)
+#define KEYP_PERIOD_US(t, prescale) ((t) / (31 << ((prescale) + 1)) - 1)
/* KEYP_LK_PTV_REG Fields */
#define KEYP_LK_PTV_PTV_SHIFT 5
@@ -162,9 +162,10 @@
static inline u16 twl4030_col_xlate(struct twl4030_keypad *kp, u8 col)
{
- /* If all bits in a row are active for all coloumns then
+ /*
+ * If all bits in a row are active for all columns then
* we have that row line connected to gnd. Mark this
- * key on as if it was on matrix position n_cols (ie
+ * key on as if it was on matrix position n_cols (i.e.
* one higher than the size of the matrix).
*/
if (col == 0xFF)
@@ -209,9 +210,9 @@
u16 new_state[TWL4030_MAX_ROWS];
int col, row;
- if (release_all)
+ if (release_all) {
memset(new_state, 0, sizeof(new_state));
- else {
+ } else {
/* check for any changes */
int ret = twl4030_read_kp_matrix_state(kp, new_state);
@@ -262,8 +263,10 @@
/* Read & Clear TWL4030 pending interrupt */
ret = twl4030_kpread(kp, ®, KEYP_ISR1, 1);
- /* Release all keys if I2C has gone bad or
- * the KEYP has gone to idle state */
+ /*
+ * Release all keys if I2C has gone bad or
+ * the KEYP has gone to idle state.
+ */
if (ret >= 0 && (reg & KEYP_IMR1_KP))
twl4030_kp_scan(kp, false);
else
@@ -283,7 +286,8 @@
if (twl4030_kpwrite_u8(kp, reg, KEYP_CTRL) < 0)
return -EIO;
- /* NOTE: we could use sih_setup() here to package keypad
+ /*
+ * NOTE: we could use sih_setup() here to package keypad
* event sources as four different IRQs ... but we don't.
*/
@@ -312,7 +316,7 @@
/*
* Enable Clear-on-Read; disable remembering events that fire
- * after the IRQ but before our handler acks (reads) them,
+ * after the IRQ but before our handler acks (reads) them.
*/
reg = TWL4030_SIH_CTRL_COR_MASK | TWL4030_SIH_CTRL_PENDDIS_MASK;
if (twl4030_kpwrite_u8(kp, reg, KEYP_SIH_CTRL) < 0)
diff --git a/drivers/input/misc/cm109.c b/drivers/input/misc/cm109.c
index 9365535..9cc6d05 100644
--- a/drivers/input/misc/cm109.c
+++ b/drivers/input/misc/cm109.c
@@ -76,8 +76,8 @@
BUZZER_ON = 1 << 5,
- /* up to 256 normal keys, up to 16 special keys */
- KEYMAP_SIZE = 256 + 16,
+ /* up to 256 normal keys, up to 15 special key combinations */
+ KEYMAP_SIZE = 256 + 15,
};
/* CM109 protocol packet */
@@ -139,7 +139,7 @@
{
if (code > 0xff) {
switch (code - 0xff) {
- case RECORD_MUTE: return KEY_MUTE;
+ case RECORD_MUTE: return KEY_MICMUTE;
case PLAYBACK_MUTE: return KEY_MUTE;
case VOLUME_DOWN: return KEY_VOLUMEDOWN;
case VOLUME_UP: return KEY_VOLUMEUP;
@@ -312,6 +312,32 @@
input_sync(idev);
}
+/*
+ * Converts data of special key presses (volume, mute) into events
+ * for the input subsystem, sends press-n-release for mute keys.
+ */
+static void cm109_report_special(struct cm109_dev *dev)
+{
+ static const u8 autorelease = RECORD_MUTE | PLAYBACK_MUTE;
+ struct input_dev *idev = dev->idev;
+ u8 data = dev->irq_data->byte[HID_IR0];
+ unsigned short keycode;
+ int i;
+
+ for (i = 0; i < 4; i++) {
+ keycode = dev->keymap[0xff + BIT(i)];
+ if (keycode == KEY_RESERVED)
+ continue;
+
+ input_report_key(idev, keycode, data & BIT(i));
+ if (data & autorelease & BIT(i)) {
+ input_sync(idev);
+ input_report_key(idev, keycode, 0);
+ }
+ }
+ input_sync(idev);
+}
+
/******************************************************************************
* CM109 usb communication interface
*****************************************************************************/
@@ -340,6 +366,7 @@
struct cm109_dev *dev = urb->context;
const int status = urb->status;
int error;
+ unsigned long flags;
dev_dbg(&dev->intf->dev, "### URB IRQ: [0x%02x 0x%02x 0x%02x 0x%02x] keybit=0x%02x\n",
dev->irq_data->byte[0],
@@ -357,10 +384,7 @@
}
/* Special keys */
- if (dev->irq_data->byte[HID_IR0] & 0x0f) {
- const int code = (dev->irq_data->byte[HID_IR0] & 0x0f);
- report_key(dev, dev->keymap[0xff + code]);
- }
+ cm109_report_special(dev);
/* Scan key column */
if (dev->keybit == 0xf) {
@@ -381,7 +405,7 @@
out:
- spin_lock(&dev->ctl_submit_lock);
+ spin_lock_irqsave(&dev->ctl_submit_lock, flags);
dev->irq_urb_pending = 0;
@@ -405,7 +429,7 @@
__func__, error);
}
- spin_unlock(&dev->ctl_submit_lock);
+ spin_unlock_irqrestore(&dev->ctl_submit_lock, flags);
}
static void cm109_urb_ctl_callback(struct urb *urb)
@@ -413,6 +437,7 @@
struct cm109_dev *dev = urb->context;
const int status = urb->status;
int error;
+ unsigned long flags;
dev_dbg(&dev->intf->dev, "### URB CTL: [0x%02x 0x%02x 0x%02x 0x%02x]\n",
dev->ctl_data->byte[0],
@@ -427,7 +452,7 @@
__func__, status);
}
- spin_lock(&dev->ctl_submit_lock);
+ spin_lock_irqsave(&dev->ctl_submit_lock, flags);
dev->ctl_urb_pending = 0;
@@ -448,7 +473,7 @@
}
}
- spin_unlock(&dev->ctl_submit_lock);
+ spin_unlock_irqrestore(&dev->ctl_submit_lock, flags);
}
static void cm109_toggle_buzzer_async(struct cm109_dev *dev)
diff --git a/drivers/input/misc/rotary_encoder.c b/drivers/input/misc/rotary_encoder.c
index 96c486d..c7fc8d4 100644
--- a/drivers/input/misc/rotary_encoder.c
+++ b/drivers/input/misc/rotary_encoder.c
@@ -47,13 +47,13 @@
bool armed;
signed char dir; /* 1 - clockwise, -1 - CCW */
- unsigned last_stable;
+ unsigned int last_stable;
};
-static unsigned rotary_encoder_get_state(struct rotary_encoder *encoder)
+static unsigned int rotary_encoder_get_state(struct rotary_encoder *encoder)
{
int i;
- unsigned ret = 0;
+ unsigned int ret = 0;
for (i = 0; i < encoder->gpios->ndescs; ++i) {
int val = gpiod_get_value_cansleep(encoder->gpios->desc[i]);
@@ -100,7 +100,7 @@
static irqreturn_t rotary_encoder_irq(int irq, void *dev_id)
{
struct rotary_encoder *encoder = dev_id;
- unsigned state;
+ unsigned int state;
mutex_lock(&encoder->access_mutex);
diff --git a/drivers/input/misc/twl6040-vibra.c b/drivers/input/misc/twl6040-vibra.c
index 42de34b..5690eb7 100644
--- a/drivers/input/misc/twl6040-vibra.c
+++ b/drivers/input/misc/twl6040-vibra.c
@@ -46,7 +46,7 @@
struct device *dev;
struct input_dev *input_dev;
struct work_struct play_work;
- struct mutex mutex;
+
int irq;
bool enabled;
@@ -190,8 +190,6 @@
return;
}
- mutex_lock(&info->mutex);
-
if (info->weak_speed || info->strong_speed) {
if (!info->enabled)
twl6040_vibra_enable(info);
@@ -200,7 +198,6 @@
} else if (info->enabled)
twl6040_vibra_disable(info);
- mutex_unlock(&info->mutex);
}
static int vibra_play(struct input_dev *input, void *data,
@@ -223,12 +220,8 @@
cancel_work_sync(&info->play_work);
- mutex_lock(&info->mutex);
-
if (info->enabled)
twl6040_vibra_disable(info);
-
- mutex_unlock(&info->mutex);
}
static int __maybe_unused twl6040_vibra_suspend(struct device *dev)
@@ -236,13 +229,11 @@
struct platform_device *pdev = to_platform_device(dev);
struct vibra_info *info = platform_get_drvdata(pdev);
- mutex_lock(&info->mutex);
+ cancel_work_sync(&info->play_work);
if (info->enabled)
twl6040_vibra_disable(info);
- mutex_unlock(&info->mutex);
-
return 0;
}
@@ -301,8 +292,6 @@
return -EINVAL;
}
- mutex_init(&info->mutex);
-
error = devm_request_threaded_irq(&pdev->dev, info->irq, NULL,
twl6040_vib_irq_handler,
IRQF_ONESHOT,
diff --git a/drivers/input/mouse/byd.c b/drivers/input/mouse/byd.c
index e583f8b..b27aa63 100644
--- a/drivers/input/mouse/byd.c
+++ b/drivers/input/mouse/byd.c
@@ -478,7 +478,6 @@
if (!priv)
return -ENOMEM;
- memset(priv, 0, sizeof(*priv));
setup_timer(&priv->timer, byd_clear_touch, (unsigned long) psmouse);
psmouse->private = priv;
diff --git a/drivers/input/tablet/acecad.c b/drivers/input/tablet/acecad.c
index 889f6b7..e86e377 100644
--- a/drivers/input/tablet/acecad.c
+++ b/drivers/input/tablet/acecad.c
@@ -49,7 +49,6 @@
struct usb_acecad {
char name[128];
char phys[64];
- struct usb_device *usbdev;
struct usb_interface *intf;
struct input_dev *input;
struct urb *irq;
@@ -64,6 +63,7 @@
unsigned char *data = acecad->data;
struct input_dev *dev = acecad->input;
struct usb_interface *intf = acecad->intf;
+ struct usb_device *udev = interface_to_usbdev(intf);
int prox, status;
switch (urb->status) {
@@ -110,15 +110,15 @@
if (status)
dev_err(&intf->dev,
"can't resubmit intr, %s-%s/input0, status %d\n",
- acecad->usbdev->bus->bus_name,
- acecad->usbdev->devpath, status);
+ udev->bus->bus_name,
+ udev->devpath, status);
}
static int usb_acecad_open(struct input_dev *dev)
{
struct usb_acecad *acecad = input_get_drvdata(dev);
- acecad->irq->dev = acecad->usbdev;
+ acecad->irq->dev = interface_to_usbdev(acecad->intf);
if (usb_submit_urb(acecad->irq, GFP_KERNEL))
return -EIO;
@@ -172,7 +172,6 @@
goto fail2;
}
- acecad->usbdev = dev;
acecad->intf = intf;
acecad->input = input_dev;
@@ -251,12 +250,13 @@
static void usb_acecad_disconnect(struct usb_interface *intf)
{
struct usb_acecad *acecad = usb_get_intfdata(intf);
+ struct usb_device *udev = interface_to_usbdev(intf);
usb_set_intfdata(intf, NULL);
input_unregister_device(acecad->input);
usb_free_urb(acecad->irq);
- usb_free_coherent(acecad->usbdev, 8, acecad->data, acecad->data_dma);
+ usb_free_coherent(udev, 8, acecad->data, acecad->data_dma);
kfree(acecad);
}
diff --git a/drivers/input/tablet/aiptek.c b/drivers/input/tablet/aiptek.c
index 78ca4484..4613f0a 100644
--- a/drivers/input/tablet/aiptek.c
+++ b/drivers/input/tablet/aiptek.c
@@ -307,7 +307,6 @@
struct aiptek {
struct input_dev *inputdev; /* input device struct */
- struct usb_device *usbdev; /* usb device struct */
struct usb_interface *intf; /* usb interface struct */
struct urb *urb; /* urb for incoming reports */
dma_addr_t data_dma; /* our dma stuffage */
@@ -847,7 +846,7 @@
{
struct aiptek *aiptek = input_get_drvdata(inputdev);
- aiptek->urb->dev = aiptek->usbdev;
+ aiptek->urb->dev = interface_to_usbdev(aiptek->intf);
if (usb_submit_urb(aiptek->urb, GFP_KERNEL) != 0)
return -EIO;
@@ -873,8 +872,10 @@
unsigned char report_type,
unsigned char report_id, void *buffer, int size)
{
- return usb_control_msg(aiptek->usbdev,
- usb_sndctrlpipe(aiptek->usbdev, 0),
+ struct usb_device *udev = interface_to_usbdev(aiptek->intf);
+
+ return usb_control_msg(udev,
+ usb_sndctrlpipe(udev, 0),
USB_REQ_SET_REPORT,
USB_TYPE_CLASS | USB_RECIP_INTERFACE |
USB_DIR_OUT, (report_type << 8) + report_id,
@@ -886,8 +887,10 @@
unsigned char report_type,
unsigned char report_id, void *buffer, int size)
{
- return usb_control_msg(aiptek->usbdev,
- usb_rcvctrlpipe(aiptek->usbdev, 0),
+ struct usb_device *udev = interface_to_usbdev(aiptek->intf);
+
+ return usb_control_msg(udev,
+ usb_rcvctrlpipe(udev, 0),
USB_REQ_GET_REPORT,
USB_TYPE_CLASS | USB_RECIP_INTERFACE |
USB_DIR_IN, (report_type << 8) + report_id,
@@ -1729,7 +1732,6 @@
}
aiptek->inputdev = inputdev;
- aiptek->usbdev = usbdev;
aiptek->intf = intf;
aiptek->ifnum = intf->altsetting[0].desc.bInterfaceNumber;
aiptek->inDelay = 0;
@@ -1833,8 +1835,8 @@
* input.
*/
usb_fill_int_urb(aiptek->urb,
- aiptek->usbdev,
- usb_rcvintpipe(aiptek->usbdev,
+ usbdev,
+ usb_rcvintpipe(usbdev,
endpoint->bEndpointAddress),
aiptek->data, 8, aiptek_irq, aiptek,
endpoint->bInterval);
diff --git a/drivers/input/tablet/gtco.c b/drivers/input/tablet/gtco.c
index 7c18249..abf09ac 100644
--- a/drivers/input/tablet/gtco.c
+++ b/drivers/input/tablet/gtco.c
@@ -104,7 +104,6 @@
struct gtco {
struct input_dev *inputdevice; /* input device struct pointer */
- struct usb_device *usbdev; /* the usb device for this device */
struct usb_interface *intf; /* the usb interface for this device */
struct urb *urbinfo; /* urb for incoming reports */
dma_addr_t buf_dma; /* dma addr of the data buffer*/
@@ -540,7 +539,7 @@
{
struct gtco *device = input_get_drvdata(inputdev);
- device->urbinfo->dev = device->usbdev;
+ device->urbinfo->dev = interface_to_usbdev(device->intf);
if (usb_submit_urb(device->urbinfo, GFP_KERNEL))
return -EIO;
@@ -824,6 +823,7 @@
int result = 0, retry;
int error;
struct usb_endpoint_descriptor *endpoint;
+ struct usb_device *udev = interface_to_usbdev(usbinterface);
/* Allocate memory for device structure */
gtco = kzalloc(sizeof(struct gtco), GFP_KERNEL);
@@ -838,11 +838,10 @@
gtco->inputdevice = input_dev;
/* Save interface information */
- gtco->usbdev = interface_to_usbdev(usbinterface);
gtco->intf = usbinterface;
/* Allocate some data for incoming reports */
- gtco->buffer = usb_alloc_coherent(gtco->usbdev, REPORT_MAX_SIZE,
+ gtco->buffer = usb_alloc_coherent(udev, REPORT_MAX_SIZE,
GFP_KERNEL, >co->buf_dma);
if (!gtco->buffer) {
dev_err(&usbinterface->dev, "No more memory for us buffers\n");
@@ -907,8 +906,8 @@
/* Couple of tries to get reply */
for (retry = 0; retry < 3; retry++) {
- result = usb_control_msg(gtco->usbdev,
- usb_rcvctrlpipe(gtco->usbdev, 0),
+ result = usb_control_msg(udev,
+ usb_rcvctrlpipe(udev, 0),
USB_REQ_GET_DESCRIPTOR,
USB_RECIP_INTERFACE | USB_DIR_IN,
REPORT_DEVICE_TYPE << 8,
@@ -936,7 +935,7 @@
}
/* Create a device file node */
- usb_make_path(gtco->usbdev, gtco->usbpath, sizeof(gtco->usbpath));
+ usb_make_path(udev, gtco->usbpath, sizeof(gtco->usbpath));
strlcat(gtco->usbpath, "/input0", sizeof(gtco->usbpath));
/* Set Input device functions */
@@ -953,15 +952,15 @@
gtco_setup_caps(input_dev);
/* Set input device required ID information */
- usb_to_input_id(gtco->usbdev, &input_dev->id);
+ usb_to_input_id(udev, &input_dev->id);
input_dev->dev.parent = &usbinterface->dev;
/* Setup the URB, it will be posted later on open of input device */
endpoint = &usbinterface->altsetting[0].endpoint[0].desc;
usb_fill_int_urb(gtco->urbinfo,
- gtco->usbdev,
- usb_rcvintpipe(gtco->usbdev,
+ udev,
+ usb_rcvintpipe(udev,
endpoint->bEndpointAddress),
gtco->buffer,
REPORT_MAX_SIZE,
@@ -985,7 +984,7 @@
err_free_urb:
usb_free_urb(gtco->urbinfo);
err_free_buf:
- usb_free_coherent(gtco->usbdev, REPORT_MAX_SIZE,
+ usb_free_coherent(udev, REPORT_MAX_SIZE,
gtco->buffer, gtco->buf_dma);
err_free_devs:
input_free_device(input_dev);
@@ -1002,13 +1001,14 @@
{
/* Grab private device ptr */
struct gtco *gtco = usb_get_intfdata(interface);
+ struct usb_device *udev = interface_to_usbdev(interface);
/* Now reverse all the registration stuff */
if (gtco) {
input_unregister_device(gtco->inputdevice);
usb_kill_urb(gtco->urbinfo);
usb_free_urb(gtco->urbinfo);
- usb_free_coherent(gtco->usbdev, REPORT_MAX_SIZE,
+ usb_free_coherent(udev, REPORT_MAX_SIZE,
gtco->buffer, gtco->buf_dma);
kfree(gtco);
}
diff --git a/drivers/input/tablet/kbtab.c b/drivers/input/tablet/kbtab.c
index d2ac7c2..e850d7e 100644
--- a/drivers/input/tablet/kbtab.c
+++ b/drivers/input/tablet/kbtab.c
@@ -31,7 +31,6 @@
unsigned char *data;
dma_addr_t data_dma;
struct input_dev *dev;
- struct usb_device *usbdev;
struct usb_interface *intf;
struct urb *irq;
char phys[32];
@@ -99,8 +98,9 @@
static int kbtab_open(struct input_dev *dev)
{
struct kbtab *kbtab = input_get_drvdata(dev);
+ struct usb_device *udev = interface_to_usbdev(kbtab->intf);
- kbtab->irq->dev = kbtab->usbdev;
+ kbtab->irq->dev = udev;
if (usb_submit_urb(kbtab->irq, GFP_KERNEL))
return -EIO;
@@ -135,7 +135,6 @@
if (!kbtab->irq)
goto fail2;
- kbtab->usbdev = dev;
kbtab->intf = intf;
kbtab->dev = input_dev;
@@ -188,12 +187,13 @@
static void kbtab_disconnect(struct usb_interface *intf)
{
struct kbtab *kbtab = usb_get_intfdata(intf);
+ struct usb_device *udev = interface_to_usbdev(intf);
usb_set_intfdata(intf, NULL);
input_unregister_device(kbtab->dev);
usb_free_urb(kbtab->irq);
- usb_free_coherent(kbtab->usbdev, 8, kbtab->data, kbtab->data_dma);
+ usb_free_coherent(udev, 8, kbtab->data, kbtab->data_dma);
kfree(kbtab);
}
diff --git a/drivers/input/touchscreen/bcm_iproc_tsc.c b/drivers/input/touchscreen/bcm_iproc_tsc.c
index ae460a5c..4d11b27 100644
--- a/drivers/input/touchscreen/bcm_iproc_tsc.c
+++ b/drivers/input/touchscreen/bcm_iproc_tsc.c
@@ -23,6 +23,8 @@
#include <linux/io.h>
#include <linux/clk.h>
#include <linux/serio.h>
+#include <linux/mfd/syscon.h>
+#include <linux/regmap.h>
#define IPROC_TS_NAME "iproc-ts"
@@ -88,7 +90,11 @@
#define TS_WIRE_MODE_BIT BIT(1)
#define dbg_reg(dev, priv, reg) \
- dev_dbg(dev, "%20s= 0x%08x\n", #reg, readl((priv)->regs + reg))
+do { \
+ u32 val; \
+ regmap_read(priv->regmap, reg, &val); \
+ dev_dbg(dev, "%20s= 0x%08x\n", #reg, val); \
+} while (0)
struct tsc_param {
/* Each step is 1024 us. Valid 1-256 */
@@ -141,7 +147,7 @@
struct platform_device *pdev;
struct input_dev *idev;
- void __iomem *regs;
+ struct regmap *regmap;
struct clk *tsc_clk;
int pen_status;
@@ -196,22 +202,22 @@
int i;
bool needs_sync = false;
- intr_status = readl(priv->regs + INTERRUPT_STATUS);
+ regmap_read(priv->regmap, INTERRUPT_STATUS, &intr_status);
intr_status &= TS_PEN_INTR_MASK | TS_FIFO_INTR_MASK;
if (intr_status == 0)
return IRQ_NONE;
/* Clear all interrupt status bits, write-1-clear */
- writel(intr_status, priv->regs + INTERRUPT_STATUS);
-
+ regmap_write(priv->regmap, INTERRUPT_STATUS, intr_status);
/* Pen up/down */
if (intr_status & TS_PEN_INTR_MASK) {
- if (readl(priv->regs + CONTROLLER_STATUS) & TS_PEN_DOWN)
+ regmap_read(priv->regmap, CONTROLLER_STATUS, &priv->pen_status);
+ if (priv->pen_status & TS_PEN_DOWN)
priv->pen_status = PEN_DOWN_STATUS;
else
priv->pen_status = PEN_UP_STATUS;
- input_report_key(priv->idev, BTN_TOUCH, priv->pen_status);
+ input_report_key(priv->idev, BTN_TOUCH, priv->pen_status);
needs_sync = true;
dev_dbg(&priv->pdev->dev,
@@ -221,7 +227,7 @@
/* coordinates in FIFO exceed the theshold */
if (intr_status & TS_FIFO_INTR_MASK) {
for (i = 0; i < priv->cfg_params.fifo_threshold; i++) {
- raw_coordinate = readl(priv->regs + FIFO_DATA);
+ regmap_read(priv->regmap, FIFO_DATA, &raw_coordinate);
if (raw_coordinate == INVALID_COORD)
continue;
@@ -239,7 +245,7 @@
x = (x >> 4) & 0x0FFF;
y = (y >> 4) & 0x0FFF;
- /* adjust x y according to lcd tsc mount angle */
+ /* Adjust x y according to LCD tsc mount angle. */
if (priv->cfg_params.invert_x)
x = priv->cfg_params.max_x - x;
@@ -262,9 +268,10 @@
static int iproc_ts_start(struct input_dev *idev)
{
- struct iproc_ts_priv *priv = input_get_drvdata(idev);
u32 val;
+ u32 mask;
int error;
+ struct iproc_ts_priv *priv = input_get_drvdata(idev);
/* Enable clock */
error = clk_prepare_enable(priv->tsc_clk);
@@ -279,9 +286,10 @@
* FIFO reaches the int_th value, and pen event(up/down)
*/
val = TS_PEN_INTR_MASK | TS_FIFO_INTR_MASK;
- writel(val, priv->regs + INTERRUPT_MASK);
+ regmap_update_bits(priv->regmap, INTERRUPT_MASK, val, val);
- writel(priv->cfg_params.fifo_threshold, priv->regs + INTERRUPT_THRES);
+ val = priv->cfg_params.fifo_threshold;
+ regmap_write(priv->regmap, INTERRUPT_THRES, val);
/* Initialize control reg1 */
val = 0;
@@ -289,26 +297,23 @@
val |= priv->cfg_params.debounce_timeout << DEBOUNCE_TIMEOUT_SHIFT;
val |= priv->cfg_params.settling_timeout << SETTLING_TIMEOUT_SHIFT;
val |= priv->cfg_params.touch_timeout << TOUCH_TIMEOUT_SHIFT;
- writel(val, priv->regs + REGCTL1);
+ regmap_write(priv->regmap, REGCTL1, val);
/* Try to clear all interrupt status */
- val = readl(priv->regs + INTERRUPT_STATUS);
- val |= TS_FIFO_INTR_MASK | TS_PEN_INTR_MASK;
- writel(val, priv->regs + INTERRUPT_STATUS);
+ val = TS_FIFO_INTR_MASK | TS_PEN_INTR_MASK;
+ regmap_update_bits(priv->regmap, INTERRUPT_STATUS, val, val);
/* Initialize control reg2 */
- val = readl(priv->regs + REGCTL2);
- val |= TS_CONTROLLER_EN_BIT | TS_WIRE_MODE_BIT;
-
- val &= ~TS_CONTROLLER_AVGDATA_MASK;
+ val = TS_CONTROLLER_EN_BIT | TS_WIRE_MODE_BIT;
val |= priv->cfg_params.average_data << TS_CONTROLLER_AVGDATA_SHIFT;
- val &= ~(TS_CONTROLLER_PWR_LDO | /* PWR up LDO */
+ mask = (TS_CONTROLLER_AVGDATA_MASK);
+ mask |= (TS_CONTROLLER_PWR_LDO | /* PWR up LDO */
TS_CONTROLLER_PWR_ADC | /* PWR up ADC */
TS_CONTROLLER_PWR_BGP | /* PWR up BGP */
TS_CONTROLLER_PWR_TS); /* PWR up TS */
-
- writel(val, priv->regs + REGCTL2);
+ mask |= val;
+ regmap_update_bits(priv->regmap, REGCTL2, mask, val);
ts_reg_dump(priv);
@@ -320,12 +325,17 @@
u32 val;
struct iproc_ts_priv *priv = input_get_drvdata(dev);
- writel(0, priv->regs + INTERRUPT_MASK); /* Disable all interrupts */
+ /*
+ * Disable FIFO int_th and pen event(up/down)Interrupts only
+ * as the interrupt mask register is shared between ADC, TS and
+ * flextimer.
+ */
+ val = TS_PEN_INTR_MASK | TS_FIFO_INTR_MASK;
+ regmap_update_bits(priv->regmap, INTERRUPT_MASK, val, 0);
/* Only power down touch screen controller */
- val = readl(priv->regs + REGCTL2);
- val |= TS_CONTROLLER_PWR_TS;
- writel(val, priv->regs + REGCTL2);
+ val = TS_CONTROLLER_PWR_TS;
+ regmap_update_bits(priv->regmap, REGCTL2, val, val);
clk_disable(priv->tsc_clk);
}
@@ -414,7 +424,6 @@
{
struct iproc_ts_priv *priv;
struct input_dev *idev;
- struct resource *res;
int irq;
int error;
@@ -422,12 +431,12 @@
if (!priv)
return -ENOMEM;
- /* touchscreen controller memory mapped regs */
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- priv->regs = devm_ioremap_resource(&pdev->dev, res);
- if (IS_ERR(priv->regs)) {
- error = PTR_ERR(priv->regs);
- dev_err(&pdev->dev, "unable to map I/O memory: %d\n", error);
+ /* touchscreen controller memory mapped regs via syscon*/
+ priv->regmap = syscon_regmap_lookup_by_phandle(pdev->dev.of_node,
+ "ts_syscon");
+ if (IS_ERR(priv->regmap)) {
+ error = PTR_ERR(priv->regmap);
+ dev_err(&pdev->dev, "unable to map I/O memory:%d\n", error);
return error;
}
diff --git a/drivers/input/touchscreen/ti_am335x_tsc.c b/drivers/input/touchscreen/ti_am335x_tsc.c
index a21a07c..8b3f15c 100644
--- a/drivers/input/touchscreen/ti_am335x_tsc.c
+++ b/drivers/input/touchscreen/ti_am335x_tsc.c
@@ -487,8 +487,7 @@
return 0;
}
-#ifdef CONFIG_PM
-static int titsc_suspend(struct device *dev)
+static int __maybe_unused titsc_suspend(struct device *dev)
{
struct titsc *ts_dev = dev_get_drvdata(dev);
struct ti_tscadc_dev *tscadc_dev;
@@ -504,7 +503,7 @@
return 0;
}
-static int titsc_resume(struct device *dev)
+static int __maybe_unused titsc_resume(struct device *dev)
{
struct titsc *ts_dev = dev_get_drvdata(dev);
struct ti_tscadc_dev *tscadc_dev;
@@ -521,14 +520,7 @@
return 0;
}
-static const struct dev_pm_ops titsc_pm_ops = {
- .suspend = titsc_suspend,
- .resume = titsc_resume,
-};
-#define TITSC_PM_OPS (&titsc_pm_ops)
-#else
-#define TITSC_PM_OPS NULL
-#endif
+static SIMPLE_DEV_PM_OPS(titsc_pm_ops, titsc_suspend, titsc_resume);
static const struct of_device_id ti_tsc_dt_ids[] = {
{ .compatible = "ti,am3359-tsc", },
@@ -541,7 +533,7 @@
.remove = titsc_remove,
.driver = {
.name = "TI-am335x-tsc",
- .pm = TITSC_PM_OPS,
+ .pm = &titsc_pm_ops,
.of_match_table = ti_tsc_dt_ids,
},
};
diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
index dd1dc39..ad08603 100644
--- a/drivers/iommu/Kconfig
+++ b/drivers/iommu/Kconfig
@@ -76,8 +76,7 @@
config FSL_PAMU
bool "Freescale IOMMU support"
- depends on PPC32
- depends on PPC_E500MC || COMPILE_TEST
+ depends on PPC_E500MC || (COMPILE_TEST && PPC)
select IOMMU_API
select GENERIC_ALLOCATOR
help
@@ -124,16 +123,6 @@
your BIOS for an option to enable it or if you have an IVRS ACPI
table.
-config AMD_IOMMU_STATS
- bool "Export AMD IOMMU statistics to debugfs"
- depends on AMD_IOMMU
- select DEBUG_FS
- ---help---
- This option enables code in the AMD IOMMU driver to collect various
- statistics about whats happening in the driver and exports that
- information to userspace via debugfs.
- If unsure, say N.
-
config AMD_IOMMU_V2
tristate "AMD IOMMU Version 2 driver"
depends on AMD_IOMMU
diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index 5efadad..634f636 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -19,6 +19,8 @@
#include <linux/ratelimit.h>
#include <linux/pci.h>
+#include <linux/acpi.h>
+#include <linux/amba/bus.h>
#include <linux/pci-ats.h>
#include <linux/bitmap.h>
#include <linux/slab.h>
@@ -72,6 +74,7 @@
LIST_HEAD(ioapic_map);
LIST_HEAD(hpet_map);
+LIST_HEAD(acpihid_map);
/*
* Domain for untranslated devices - only allocated
@@ -162,18 +165,65 @@
*
****************************************************************************/
-static struct protection_domain *to_pdomain(struct iommu_domain *dom)
+static inline int match_hid_uid(struct device *dev,
+ struct acpihid_map_entry *entry)
{
- return container_of(dom, struct protection_domain, domain);
+ const char *hid, *uid;
+
+ hid = acpi_device_hid(ACPI_COMPANION(dev));
+ uid = acpi_device_uid(ACPI_COMPANION(dev));
+
+ if (!hid || !(*hid))
+ return -ENODEV;
+
+ if (!uid || !(*uid))
+ return strcmp(hid, entry->hid);
+
+ if (!(*entry->uid))
+ return strcmp(hid, entry->hid);
+
+ return (strcmp(hid, entry->hid) || strcmp(uid, entry->uid));
}
-static inline u16 get_device_id(struct device *dev)
+static inline u16 get_pci_device_id(struct device *dev)
{
struct pci_dev *pdev = to_pci_dev(dev);
return PCI_DEVID(pdev->bus->number, pdev->devfn);
}
+static inline int get_acpihid_device_id(struct device *dev,
+ struct acpihid_map_entry **entry)
+{
+ struct acpihid_map_entry *p;
+
+ list_for_each_entry(p, &acpihid_map, list) {
+ if (!match_hid_uid(dev, p)) {
+ if (entry)
+ *entry = p;
+ return p->devid;
+ }
+ }
+ return -EINVAL;
+}
+
+static inline int get_device_id(struct device *dev)
+{
+ int devid;
+
+ if (dev_is_pci(dev))
+ devid = get_pci_device_id(dev);
+ else
+ devid = get_acpihid_device_id(dev, NULL);
+
+ return devid;
+}
+
+static struct protection_domain *to_pdomain(struct iommu_domain *dom)
+{
+ return container_of(dom, struct protection_domain, domain);
+}
+
static struct iommu_dev_data *alloc_dev_data(u16 devid)
{
struct iommu_dev_data *dev_data;
@@ -222,6 +272,7 @@
struct pci_dev *pdev = to_pci_dev(dev);
u16 devid, ivrs_alias, pci_alias;
+ /* The callers make sure that get_device_id() does not fail here */
devid = get_device_id(dev);
ivrs_alias = amd_iommu_alias_table[devid];
pci_for_each_dma_alias(pdev, __last_alias, &pci_alias);
@@ -263,8 +314,7 @@
*/
if (pci_alias == devid &&
PCI_BUS_NUM(ivrs_alias) == pdev->bus->number) {
- pdev->dev_flags |= PCI_DEV_FLAGS_DMA_ALIAS_DEVFN;
- pdev->dma_alias_devfn = ivrs_alias & 0xff;
+ pci_add_dma_alias(pdev, ivrs_alias & 0xff);
pr_info("AMD-Vi: Added PCI DMA alias %02x.%d for %s\n",
PCI_SLOT(ivrs_alias), PCI_FUNC(ivrs_alias),
dev_name(dev));
@@ -290,6 +340,29 @@
return dev->archdata.iommu;
}
+/*
+* Find or create an IOMMU group for a acpihid device.
+*/
+static struct iommu_group *acpihid_device_group(struct device *dev)
+{
+ struct acpihid_map_entry *p, *entry = NULL;
+ int devid;
+
+ devid = get_acpihid_device_id(dev, &entry);
+ if (devid < 0)
+ return ERR_PTR(devid);
+
+ list_for_each_entry(p, &acpihid_map, list) {
+ if ((devid == p->devid) && p->group)
+ entry->group = p->group;
+ }
+
+ if (!entry->group)
+ entry->group = generic_device_group(dev);
+
+ return entry->group;
+}
+
static bool pci_iommuv2_capable(struct pci_dev *pdev)
{
static const int caps[] = {
@@ -341,9 +414,11 @@
struct dma_ops_domain *dma_dom)
{
struct unity_map_entry *e;
- u16 devid;
+ int devid;
devid = get_device_id(dev);
+ if (devid < 0)
+ return;
list_for_each_entry(e, &amd_iommu_unity_map, list) {
if (!(devid >= e->devid_start && devid <= e->devid_end))
@@ -358,16 +433,14 @@
*/
static bool check_device(struct device *dev)
{
- u16 devid;
+ int devid;
if (!dev || !dev->dma_mask)
return false;
- /* No PCI device */
- if (!dev_is_pci(dev))
- return false;
-
devid = get_device_id(dev);
+ if (devid < 0)
+ return false;
/* Out of our scope? */
if (devid > amd_iommu_last_bdf)
@@ -402,22 +475,26 @@
static int iommu_init_device(struct device *dev)
{
- struct pci_dev *pdev = to_pci_dev(dev);
struct iommu_dev_data *dev_data;
+ int devid;
if (dev->archdata.iommu)
return 0;
- dev_data = find_dev_data(get_device_id(dev));
+ devid = get_device_id(dev);
+ if (devid < 0)
+ return devid;
+
+ dev_data = find_dev_data(devid);
if (!dev_data)
return -ENOMEM;
dev_data->alias = get_alias(dev);
- if (pci_iommuv2_capable(pdev)) {
+ if (dev_is_pci(dev) && pci_iommuv2_capable(to_pci_dev(dev))) {
struct amd_iommu *iommu;
- iommu = amd_iommu_rlookup_table[dev_data->devid];
+ iommu = amd_iommu_rlookup_table[dev_data->devid];
dev_data->iommu_v2 = iommu->is_iommu_v2;
}
@@ -431,9 +508,13 @@
static void iommu_ignore_device(struct device *dev)
{
- u16 devid, alias;
+ u16 alias;
+ int devid;
devid = get_device_id(dev);
+ if (devid < 0)
+ return;
+
alias = get_alias(dev);
memset(&amd_iommu_dev_table[devid], 0, sizeof(struct dev_table_entry));
@@ -445,8 +526,14 @@
static void iommu_uninit_device(struct device *dev)
{
- struct iommu_dev_data *dev_data = search_dev_data(get_device_id(dev));
+ int devid;
+ struct iommu_dev_data *dev_data;
+ devid = get_device_id(dev);
+ if (devid < 0)
+ return;
+
+ dev_data = search_dev_data(devid);
if (!dev_data)
return;
@@ -467,70 +554,6 @@
*/
}
-#ifdef CONFIG_AMD_IOMMU_STATS
-
-/*
- * Initialization code for statistics collection
- */
-
-DECLARE_STATS_COUNTER(compl_wait);
-DECLARE_STATS_COUNTER(cnt_map_single);
-DECLARE_STATS_COUNTER(cnt_unmap_single);
-DECLARE_STATS_COUNTER(cnt_map_sg);
-DECLARE_STATS_COUNTER(cnt_unmap_sg);
-DECLARE_STATS_COUNTER(cnt_alloc_coherent);
-DECLARE_STATS_COUNTER(cnt_free_coherent);
-DECLARE_STATS_COUNTER(cross_page);
-DECLARE_STATS_COUNTER(domain_flush_single);
-DECLARE_STATS_COUNTER(domain_flush_all);
-DECLARE_STATS_COUNTER(alloced_io_mem);
-DECLARE_STATS_COUNTER(total_map_requests);
-DECLARE_STATS_COUNTER(complete_ppr);
-DECLARE_STATS_COUNTER(invalidate_iotlb);
-DECLARE_STATS_COUNTER(invalidate_iotlb_all);
-DECLARE_STATS_COUNTER(pri_requests);
-
-static struct dentry *stats_dir;
-static struct dentry *de_fflush;
-
-static void amd_iommu_stats_add(struct __iommu_counter *cnt)
-{
- if (stats_dir == NULL)
- return;
-
- cnt->dent = debugfs_create_u64(cnt->name, 0444, stats_dir,
- &cnt->value);
-}
-
-static void amd_iommu_stats_init(void)
-{
- stats_dir = debugfs_create_dir("amd-iommu", NULL);
- if (stats_dir == NULL)
- return;
-
- de_fflush = debugfs_create_bool("fullflush", 0444, stats_dir,
- &amd_iommu_unmap_flush);
-
- amd_iommu_stats_add(&compl_wait);
- amd_iommu_stats_add(&cnt_map_single);
- amd_iommu_stats_add(&cnt_unmap_single);
- amd_iommu_stats_add(&cnt_map_sg);
- amd_iommu_stats_add(&cnt_unmap_sg);
- amd_iommu_stats_add(&cnt_alloc_coherent);
- amd_iommu_stats_add(&cnt_free_coherent);
- amd_iommu_stats_add(&cross_page);
- amd_iommu_stats_add(&domain_flush_single);
- amd_iommu_stats_add(&domain_flush_all);
- amd_iommu_stats_add(&alloced_io_mem);
- amd_iommu_stats_add(&total_map_requests);
- amd_iommu_stats_add(&complete_ppr);
- amd_iommu_stats_add(&invalidate_iotlb);
- amd_iommu_stats_add(&invalidate_iotlb_all);
- amd_iommu_stats_add(&pri_requests);
-}
-
-#endif
-
/****************************************************************************
*
* Interrupt handling functions
@@ -653,8 +676,6 @@
{
struct amd_iommu_fault fault;
- INC_STATS_COUNTER(pri_requests);
-
if (PPR_REQ_TYPE(raw[0]) != PPR_REQ_FAULT) {
pr_err_ratelimited("AMD-Vi: Unknown PPR request received\n");
return;
@@ -2284,13 +2305,17 @@
static int attach_device(struct device *dev,
struct protection_domain *domain)
{
- struct pci_dev *pdev = to_pci_dev(dev);
+ struct pci_dev *pdev;
struct iommu_dev_data *dev_data;
unsigned long flags;
int ret;
dev_data = get_dev_data(dev);
+ if (!dev_is_pci(dev))
+ goto skip_ats_check;
+
+ pdev = to_pci_dev(dev);
if (domain->flags & PD_IOMMUV2_MASK) {
if (!dev_data->passthrough)
return -EINVAL;
@@ -2309,6 +2334,7 @@
dev_data->ats.qdep = pci_ats_queue_depth(pdev);
}
+skip_ats_check:
write_lock_irqsave(&amd_iommu_devtable_lock, flags);
ret = __attach_device(dev_data, domain);
write_unlock_irqrestore(&amd_iommu_devtable_lock, flags);
@@ -2365,6 +2391,9 @@
__detach_device(dev_data);
write_unlock_irqrestore(&amd_iommu_devtable_lock, flags);
+ if (!dev_is_pci(dev))
+ return;
+
if (domain->flags & PD_IOMMUV2_MASK && dev_data->iommu_v2)
pdev_iommuv2_disable(to_pci_dev(dev));
else if (dev_data->ats.enabled)
@@ -2378,13 +2407,15 @@
struct iommu_dev_data *dev_data;
struct iommu_domain *domain;
struct amd_iommu *iommu;
- u16 devid;
- int ret;
+ int ret, devid;
if (!check_device(dev) || get_dev_data(dev))
return 0;
devid = get_device_id(dev);
+ if (devid < 0)
+ return devid;
+
iommu = amd_iommu_rlookup_table[devid];
ret = iommu_init_device(dev);
@@ -2422,18 +2453,29 @@
static void amd_iommu_remove_device(struct device *dev)
{
struct amd_iommu *iommu;
- u16 devid;
+ int devid;
if (!check_device(dev))
return;
devid = get_device_id(dev);
+ if (devid < 0)
+ return;
+
iommu = amd_iommu_rlookup_table[devid];
iommu_uninit_device(dev);
iommu_completion_wait(iommu);
}
+static struct iommu_group *amd_iommu_device_group(struct device *dev)
+{
+ if (dev_is_pci(dev))
+ return pci_device_group(dev);
+
+ return acpihid_device_group(dev);
+}
+
/*****************************************************************************
*
* The next functions belong to the dma_ops mapping/unmapping code.
@@ -2598,11 +2640,6 @@
pages = iommu_num_pages(paddr, size, PAGE_SIZE);
paddr &= PAGE_MASK;
- INC_STATS_COUNTER(total_map_requests);
-
- if (pages > 1)
- INC_STATS_COUNTER(cross_page);
-
if (align)
align_mask = (1UL << get_order(size)) - 1;
@@ -2623,8 +2660,6 @@
}
address += offset;
- ADD_STATS_COUNTER(alloced_io_mem, size);
-
if (unlikely(amd_iommu_np_cache)) {
domain_flush_pages(&dma_dom->domain, address, size);
domain_flush_complete(&dma_dom->domain);
@@ -2672,8 +2707,6 @@
start += PAGE_SIZE;
}
- SUB_STATS_COUNTER(alloced_io_mem, size);
-
dma_ops_free_addresses(dma_dom, dma_addr, pages);
}
@@ -2689,8 +2722,6 @@
struct protection_domain *domain;
u64 dma_mask;
- INC_STATS_COUNTER(cnt_map_single);
-
domain = get_domain(dev);
if (PTR_ERR(domain) == -EINVAL)
return (dma_addr_t)paddr;
@@ -2711,8 +2742,6 @@
{
struct protection_domain *domain;
- INC_STATS_COUNTER(cnt_unmap_single);
-
domain = get_domain(dev);
if (IS_ERR(domain))
return;
@@ -2735,8 +2764,6 @@
int mapped_elems = 0;
u64 dma_mask;
- INC_STATS_COUNTER(cnt_map_sg);
-
domain = get_domain(dev);
if (IS_ERR(domain))
return 0;
@@ -2782,8 +2809,6 @@
struct scatterlist *s;
int i;
- INC_STATS_COUNTER(cnt_unmap_sg);
-
domain = get_domain(dev);
if (IS_ERR(domain))
return;
@@ -2806,8 +2831,6 @@
struct protection_domain *domain;
struct page *page;
- INC_STATS_COUNTER(cnt_alloc_coherent);
-
domain = get_domain(dev);
if (PTR_ERR(domain) == -EINVAL) {
page = alloc_pages(flag, get_order(size));
@@ -2861,8 +2884,6 @@
struct protection_domain *domain;
struct page *page;
- INC_STATS_COUNTER(cnt_free_coherent);
-
page = virt_to_page(virt_addr);
size = PAGE_ALIGN(size);
@@ -2927,7 +2948,17 @@
int __init amd_iommu_init_api(void)
{
- return bus_set_iommu(&pci_bus_type, &amd_iommu_ops);
+ int err = 0;
+
+ err = bus_set_iommu(&pci_bus_type, &amd_iommu_ops);
+ if (err)
+ return err;
+#ifdef CONFIG_ARM_AMBA
+ err = bus_set_iommu(&amba_bustype, &amd_iommu_ops);
+ if (err)
+ return err;
+#endif
+ return 0;
}
int __init amd_iommu_init_dma_ops(void)
@@ -2944,8 +2975,6 @@
if (!swiotlb)
dma_ops = &nommu_dma_ops;
- amd_iommu_stats_init();
-
if (amd_iommu_unmap_flush)
pr_info("AMD-Vi: IO/TLB flush on unmap enabled\n");
else
@@ -3099,12 +3128,14 @@
{
struct iommu_dev_data *dev_data = dev->archdata.iommu;
struct amd_iommu *iommu;
- u16 devid;
+ int devid;
if (!check_device(dev))
return;
devid = get_device_id(dev);
+ if (devid < 0)
+ return;
if (dev_data->domain != NULL)
detach_device(dev);
@@ -3222,9 +3253,11 @@
struct list_head *head)
{
struct unity_map_entry *entry;
- u16 devid;
+ int devid;
devid = get_device_id(dev);
+ if (devid < 0)
+ return;
list_for_each_entry(entry, &amd_iommu_unity_map, list) {
struct iommu_dm_region *region;
@@ -3271,7 +3304,7 @@
.iova_to_phys = amd_iommu_iova_to_phys,
.add_device = amd_iommu_add_device,
.remove_device = amd_iommu_remove_device,
- .device_group = pci_device_group,
+ .device_group = amd_iommu_device_group,
.get_dm_regions = amd_iommu_get_dm_regions,
.put_dm_regions = amd_iommu_put_dm_regions,
.pgsize_bitmap = AMD_IOMMU_PGSIZES,
@@ -3432,8 +3465,6 @@
static int __amd_iommu_flush_page(struct protection_domain *domain, int pasid,
u64 address)
{
- INC_STATS_COUNTER(invalidate_iotlb);
-
return __flush_pasid(domain, pasid, address, false);
}
@@ -3454,8 +3485,6 @@
static int __amd_iommu_flush_tlb(struct protection_domain *domain, int pasid)
{
- INC_STATS_COUNTER(invalidate_iotlb_all);
-
return __flush_pasid(domain, pasid, CMD_INV_IOMMU_ALL_PAGES_ADDRESS,
true);
}
@@ -3575,8 +3604,6 @@
struct amd_iommu *iommu;
struct iommu_cmd cmd;
- INC_STATS_COUNTER(complete_ppr);
-
dev_data = get_dev_data(&pdev->dev);
iommu = amd_iommu_rlookup_table[dev_data->devid];
@@ -3926,6 +3953,9 @@
case X86_IRQ_ALLOC_TYPE_MSI:
case X86_IRQ_ALLOC_TYPE_MSIX:
devid = get_device_id(&info->msi_dev->dev);
+ if (devid < 0)
+ return NULL;
+
iommu = amd_iommu_rlookup_table[devid];
if (iommu)
return iommu->msi_domain;
diff --git a/drivers/iommu/amd_iommu_init.c b/drivers/iommu/amd_iommu_init.c
index bf4959f..9e00341 100644
--- a/drivers/iommu/amd_iommu_init.c
+++ b/drivers/iommu/amd_iommu_init.c
@@ -44,7 +44,7 @@
*/
#define IVRS_HEADER_LENGTH 48
-#define ACPI_IVHD_TYPE 0x10
+#define ACPI_IVHD_TYPE_MAX_SUPPORTED 0x40
#define ACPI_IVMD_TYPE_ALL 0x20
#define ACPI_IVMD_TYPE 0x21
#define ACPI_IVMD_TYPE_RANGE 0x22
@@ -58,6 +58,11 @@
#define IVHD_DEV_EXT_SELECT 0x46
#define IVHD_DEV_EXT_SELECT_RANGE 0x47
#define IVHD_DEV_SPECIAL 0x48
+#define IVHD_DEV_ACPI_HID 0xf0
+
+#define UID_NOT_PRESENT 0
+#define UID_IS_INTEGER 1
+#define UID_IS_CHARACTER 2
#define IVHD_SPECIAL_IOAPIC 1
#define IVHD_SPECIAL_HPET 2
@@ -99,7 +104,11 @@
u64 mmio_phys;
u16 pci_seg;
u16 info;
- u32 efr;
+ u32 efr_attr;
+
+ /* Following only valid on IVHD type 11h and 40h */
+ u64 efr_reg; /* Exact copy of MMIO_EXT_FEATURES */
+ u64 res;
} __attribute__((packed));
/*
@@ -111,6 +120,11 @@
u16 devid;
u8 flags;
u32 ext;
+ u32 hidh;
+ u64 cid;
+ u8 uidf;
+ u8 uidl;
+ u8 uid;
} __attribute__((packed));
/*
@@ -133,6 +147,7 @@
static bool amd_iommu_detected;
static bool __initdata amd_iommu_disabled;
+static int amd_iommu_target_ivhd_type;
u16 amd_iommu_last_bdf; /* largest PCI device id we have
to handle */
@@ -218,8 +233,12 @@
#define EARLY_MAP_SIZE 4
static struct devid_map __initdata early_ioapic_map[EARLY_MAP_SIZE];
static struct devid_map __initdata early_hpet_map[EARLY_MAP_SIZE];
+static struct acpihid_map_entry __initdata early_acpihid_map[EARLY_MAP_SIZE];
+
static int __initdata early_ioapic_map_size;
static int __initdata early_hpet_map_size;
+static int __initdata early_acpihid_map_size;
+
static bool __initdata cmdline_maps;
static enum iommu_init_state init_state = IOMMU_START_STATE;
@@ -394,6 +413,22 @@
release_mem_region(iommu->mmio_phys, iommu->mmio_phys_end);
}
+static inline u32 get_ivhd_header_size(struct ivhd_header *h)
+{
+ u32 size = 0;
+
+ switch (h->type) {
+ case 0x10:
+ size = 24;
+ break;
+ case 0x11:
+ case 0x40:
+ size = 40;
+ break;
+ }
+ return size;
+}
+
/****************************************************************************
*
* The functions below belong to the first pass of AMD IOMMU ACPI table
@@ -408,7 +443,15 @@
*/
static inline int ivhd_entry_length(u8 *ivhd)
{
- return 0x04 << (*ivhd >> 6);
+ u32 type = ((struct ivhd_entry *)ivhd)->type;
+
+ if (type < 0x80) {
+ return 0x04 << (*ivhd >> 6);
+ } else if (type == IVHD_DEV_ACPI_HID) {
+ /* For ACPI_HID, offset 21 is uid len */
+ return *((u8 *)ivhd + 21) + 22;
+ }
+ return 0;
}
/*
@@ -420,7 +463,14 @@
u8 *p = (void *)h, *end = (void *)h;
struct ivhd_entry *dev;
- p += sizeof(*h);
+ u32 ivhd_size = get_ivhd_header_size(h);
+
+ if (!ivhd_size) {
+ pr_err("AMD-Vi: Unsupported IVHD type %#x\n", h->type);
+ return -EINVAL;
+ }
+
+ p += ivhd_size;
end += h->length;
while (p < end) {
@@ -448,6 +498,22 @@
return 0;
}
+static int __init check_ivrs_checksum(struct acpi_table_header *table)
+{
+ int i;
+ u8 checksum = 0, *p = (u8 *)table;
+
+ for (i = 0; i < table->length; ++i)
+ checksum += p[i];
+ if (checksum != 0) {
+ /* ACPI table corrupt */
+ pr_err(FW_BUG "AMD-Vi: IVRS invalid checksum\n");
+ return -ENODEV;
+ }
+
+ return 0;
+}
+
/*
* Iterate over all IVHD entries in the ACPI table and find the highest device
* id which we need to handle. This is the first of three functions which parse
@@ -455,31 +521,19 @@
*/
static int __init find_last_devid_acpi(struct acpi_table_header *table)
{
- int i;
- u8 checksum = 0, *p = (u8 *)table, *end = (u8 *)table;
+ u8 *p = (u8 *)table, *end = (u8 *)table;
struct ivhd_header *h;
- /*
- * Validate checksum here so we don't need to do it when
- * we actually parse the table
- */
- for (i = 0; i < table->length; ++i)
- checksum += p[i];
- if (checksum != 0)
- /* ACPI table corrupt */
- return -ENODEV;
-
p += IVRS_HEADER_LENGTH;
end += table->length;
while (p < end) {
h = (struct ivhd_header *)p;
- switch (h->type) {
- case ACPI_IVHD_TYPE:
- find_last_devid_from_ivhd(h);
- break;
- default:
- break;
+ if (h->type == amd_iommu_target_ivhd_type) {
+ int ret = find_last_devid_from_ivhd(h);
+
+ if (ret)
+ return ret;
}
p += h->length;
}
@@ -724,6 +778,42 @@
return 0;
}
+static int __init add_acpi_hid_device(u8 *hid, u8 *uid, u16 *devid,
+ bool cmd_line)
+{
+ struct acpihid_map_entry *entry;
+ struct list_head *list = &acpihid_map;
+
+ list_for_each_entry(entry, list, list) {
+ if (strcmp(entry->hid, hid) ||
+ (*uid && *entry->uid && strcmp(entry->uid, uid)) ||
+ !entry->cmd_line)
+ continue;
+
+ pr_info("AMD-Vi: Command-line override for hid:%s uid:%s\n",
+ hid, uid);
+ *devid = entry->devid;
+ return 0;
+ }
+
+ entry = kzalloc(sizeof(*entry), GFP_KERNEL);
+ if (!entry)
+ return -ENOMEM;
+
+ memcpy(entry->uid, uid, strlen(uid));
+ memcpy(entry->hid, hid, strlen(hid));
+ entry->devid = *devid;
+ entry->cmd_line = cmd_line;
+ entry->root_devid = (entry->devid & (~0x7));
+
+ pr_info("AMD-Vi:%s, add hid:%s, uid:%s, rdevid:%d\n",
+ entry->cmd_line ? "cmd" : "ivrs",
+ entry->hid, entry->uid, entry->root_devid);
+
+ list_add_tail(&entry->list, list);
+ return 0;
+}
+
static int __init add_early_maps(void)
{
int i, ret;
@@ -746,6 +836,15 @@
return ret;
}
+ for (i = 0; i < early_acpihid_map_size; ++i) {
+ ret = add_acpi_hid_device(early_acpihid_map[i].hid,
+ early_acpihid_map[i].uid,
+ &early_acpihid_map[i].devid,
+ early_acpihid_map[i].cmd_line);
+ if (ret)
+ return ret;
+ }
+
return 0;
}
@@ -785,6 +884,7 @@
u32 dev_i, ext_flags = 0;
bool alias = false;
struct ivhd_entry *e;
+ u32 ivhd_size;
int ret;
@@ -800,7 +900,14 @@
/*
* Done. Now parse the device entries
*/
- p += sizeof(struct ivhd_header);
+ ivhd_size = get_ivhd_header_size(h);
+ if (!ivhd_size) {
+ pr_err("AMD-Vi: Unsupported IVHD type %#x\n", h->type);
+ return -EINVAL;
+ }
+
+ p += ivhd_size;
+
end += h->length;
@@ -958,6 +1065,70 @@
break;
}
+ case IVHD_DEV_ACPI_HID: {
+ u16 devid;
+ u8 hid[ACPIHID_HID_LEN] = {0};
+ u8 uid[ACPIHID_UID_LEN] = {0};
+ int ret;
+
+ if (h->type != 0x40) {
+ pr_err(FW_BUG "Invalid IVHD device type %#x\n",
+ e->type);
+ break;
+ }
+
+ memcpy(hid, (u8 *)(&e->ext), ACPIHID_HID_LEN - 1);
+ hid[ACPIHID_HID_LEN - 1] = '\0';
+
+ if (!(*hid)) {
+ pr_err(FW_BUG "Invalid HID.\n");
+ break;
+ }
+
+ switch (e->uidf) {
+ case UID_NOT_PRESENT:
+
+ if (e->uidl != 0)
+ pr_warn(FW_BUG "Invalid UID length.\n");
+
+ break;
+ case UID_IS_INTEGER:
+
+ sprintf(uid, "%d", e->uid);
+
+ break;
+ case UID_IS_CHARACTER:
+
+ memcpy(uid, (u8 *)(&e->uid), ACPIHID_UID_LEN - 1);
+ uid[ACPIHID_UID_LEN - 1] = '\0';
+
+ break;
+ default:
+ break;
+ }
+
+ DUMP_printk(" DEV_ACPI_HID(%s[%s])\t\tdevid: %02x:%02x.%x\n",
+ hid, uid,
+ PCI_BUS_NUM(devid),
+ PCI_SLOT(devid),
+ PCI_FUNC(devid));
+
+ devid = e->devid;
+ flags = e->flags;
+
+ ret = add_acpi_hid_device(hid, uid, &devid, false);
+ if (ret)
+ return ret;
+
+ /*
+ * add_special_device might update the devid in case a
+ * command-line override is present. So call
+ * set_dev_entry_from_acpi after add_special_device.
+ */
+ set_dev_entry_from_acpi(iommu, devid, e->flags, 0);
+
+ break;
+ }
default:
break;
}
@@ -1078,13 +1249,25 @@
iommu->pci_seg = h->pci_seg;
iommu->mmio_phys = h->mmio_phys;
- /* Check if IVHD EFR contains proper max banks/counters */
- if ((h->efr != 0) &&
- ((h->efr & (0xF << 13)) != 0) &&
- ((h->efr & (0x3F << 17)) != 0)) {
- iommu->mmio_phys_end = MMIO_REG_END_OFFSET;
- } else {
- iommu->mmio_phys_end = MMIO_CNTR_CONF_OFFSET;
+ switch (h->type) {
+ case 0x10:
+ /* Check if IVHD EFR contains proper max banks/counters */
+ if ((h->efr_attr != 0) &&
+ ((h->efr_attr & (0xF << 13)) != 0) &&
+ ((h->efr_attr & (0x3F << 17)) != 0))
+ iommu->mmio_phys_end = MMIO_REG_END_OFFSET;
+ else
+ iommu->mmio_phys_end = MMIO_CNTR_CONF_OFFSET;
+ break;
+ case 0x11:
+ case 0x40:
+ if (h->efr_reg & (1 << 9))
+ iommu->mmio_phys_end = MMIO_REG_END_OFFSET;
+ else
+ iommu->mmio_phys_end = MMIO_CNTR_CONF_OFFSET;
+ break;
+ default:
+ return -EINVAL;
}
iommu->mmio_base = iommu_map_mmio_space(iommu->mmio_phys,
@@ -1117,6 +1300,32 @@
return 0;
}
+/**
+ * get_highest_supported_ivhd_type - Look up the appropriate IVHD type
+ * @ivrs Pointer to the IVRS header
+ *
+ * This function search through all IVDB of the maximum supported IVHD
+ */
+static u8 get_highest_supported_ivhd_type(struct acpi_table_header *ivrs)
+{
+ u8 *base = (u8 *)ivrs;
+ struct ivhd_header *ivhd = (struct ivhd_header *)
+ (base + IVRS_HEADER_LENGTH);
+ u8 last_type = ivhd->type;
+ u16 devid = ivhd->devid;
+
+ while (((u8 *)ivhd - base < ivrs->length) &&
+ (ivhd->type <= ACPI_IVHD_TYPE_MAX_SUPPORTED)) {
+ u8 *p = (u8 *) ivhd;
+
+ if (ivhd->devid == devid)
+ last_type = ivhd->type;
+ ivhd = (struct ivhd_header *)(p + ivhd->length);
+ }
+
+ return last_type;
+}
+
/*
* Iterates over all IOMMU entries in the ACPI table, allocates the
* IOMMU structure and initializes it with init_iommu_one()
@@ -1133,8 +1342,7 @@
while (p < end) {
h = (struct ivhd_header *)p;
- switch (*p) {
- case ACPI_IVHD_TYPE:
+ if (*p == amd_iommu_target_ivhd_type) {
DUMP_printk("device: %02x:%02x.%01x cap: %04x "
"seg: %d flags: %01x info %04x\n",
@@ -1151,9 +1359,6 @@
ret = init_iommu_one(iommu, h);
if (ret)
return ret;
- break;
- default:
- break;
}
p += h->length;
@@ -1818,18 +2023,20 @@
* remapping setup code.
*
* This function basically parses the ACPI table for AMD IOMMU (IVRS)
- * three times:
+ * four times:
*
- * 1 pass) Find the highest PCI device id the driver has to handle.
+ * 1 pass) Discover the most comprehensive IVHD type to use.
+ *
+ * 2 pass) Find the highest PCI device id the driver has to handle.
* Upon this information the size of the data structures is
* determined that needs to be allocated.
*
- * 2 pass) Initialize the data structures just allocated with the
+ * 3 pass) Initialize the data structures just allocated with the
* information in the ACPI table about available AMD IOMMUs
* in the system. It also maps the PCI devices in the
* system to specific IOMMUs
*
- * 3 pass) After the basic data structures are allocated and
+ * 4 pass) After the basic data structures are allocated and
* initialized we update them with information about memory
* remapping requirements parsed out of the ACPI table in
* this last pass.
@@ -1857,6 +2064,17 @@
}
/*
+ * Validate checksum here so we don't need to do it when
+ * we actually parse the table
+ */
+ ret = check_ivrs_checksum(ivrs_base);
+ if (ret)
+ return ret;
+
+ amd_iommu_target_ivhd_type = get_highest_supported_ivhd_type(ivrs_base);
+ DUMP_printk("Using IVHD type %#x\n", amd_iommu_target_ivhd_type);
+
+ /*
* First parse ACPI tables to find the largest Bus/Dev/Func
* we need to handle. Upon this information the shared data
* structures for the IOMMUs in the system will be allocated
@@ -2259,10 +2477,43 @@
return 1;
}
+static int __init parse_ivrs_acpihid(char *str)
+{
+ u32 bus, dev, fn;
+ char *hid, *uid, *p;
+ char acpiid[ACPIHID_UID_LEN + ACPIHID_HID_LEN] = {0};
+ int ret, i;
+
+ ret = sscanf(str, "[%x:%x.%x]=%s", &bus, &dev, &fn, acpiid);
+ if (ret != 4) {
+ pr_err("AMD-Vi: Invalid command line: ivrs_acpihid(%s)\n", str);
+ return 1;
+ }
+
+ p = acpiid;
+ hid = strsep(&p, ":");
+ uid = p;
+
+ if (!hid || !(*hid) || !uid) {
+ pr_err("AMD-Vi: Invalid command line: hid or uid\n");
+ return 1;
+ }
+
+ i = early_acpihid_map_size++;
+ memcpy(early_acpihid_map[i].hid, hid, strlen(hid));
+ memcpy(early_acpihid_map[i].uid, uid, strlen(uid));
+ early_acpihid_map[i].devid =
+ ((bus & 0xff) << 8) | ((dev & 0x1f) << 3) | (fn & 0x7);
+ early_acpihid_map[i].cmd_line = true;
+
+ return 1;
+}
+
__setup("amd_iommu_dump", parse_amd_iommu_dump);
__setup("amd_iommu=", parse_amd_iommu_options);
__setup("ivrs_ioapic", parse_ivrs_ioapic);
__setup("ivrs_hpet", parse_ivrs_hpet);
+__setup("ivrs_acpihid", parse_ivrs_acpihid);
IOMMU_INIT_FINISH(amd_iommu_detect,
gart_iommu_hole_init,
diff --git a/drivers/iommu/amd_iommu_types.h b/drivers/iommu/amd_iommu_types.h
index 9d32b20..590956a 100644
--- a/drivers/iommu/amd_iommu_types.h
+++ b/drivers/iommu/amd_iommu_types.h
@@ -527,6 +527,19 @@
#endif
};
+#define ACPIHID_UID_LEN 256
+#define ACPIHID_HID_LEN 9
+
+struct acpihid_map_entry {
+ struct list_head list;
+ u8 uid[ACPIHID_UID_LEN];
+ u8 hid[ACPIHID_HID_LEN];
+ u16 devid;
+ u16 root_devid;
+ bool cmd_line;
+ struct iommu_group *group;
+};
+
struct devid_map {
struct list_head list;
u8 id;
@@ -537,6 +550,7 @@
/* Map HPET and IOAPIC ids to the devid used by the IOMMU */
extern struct list_head ioapic_map;
extern struct list_head hpet_map;
+extern struct list_head acpihid_map;
/*
* List with all IOMMUs in the system. This list is not locked because it is
@@ -668,30 +682,4 @@
return -EINVAL;
}
-#ifdef CONFIG_AMD_IOMMU_STATS
-
-struct __iommu_counter {
- char *name;
- struct dentry *dent;
- u64 value;
-};
-
-#define DECLARE_STATS_COUNTER(nm) \
- static struct __iommu_counter nm = { \
- .name = #nm, \
- }
-
-#define INC_STATS_COUNTER(name) name.value += 1
-#define ADD_STATS_COUNTER(name, x) name.value += (x)
-#define SUB_STATS_COUNTER(name, x) name.value -= (x)
-
-#else /* CONFIG_AMD_IOMMU_STATS */
-
-#define DECLARE_STATS_COUNTER(name)
-#define INC_STATS_COUNTER(name)
-#define ADD_STATS_COUNTER(name, x)
-#define SUB_STATS_COUNTER(name, x)
-
-#endif /* CONFIG_AMD_IOMMU_STATS */
-
#endif /* _ASM_X86_AMD_IOMMU_TYPES_H */
diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index 4ff73ff..ebab33e 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -590,6 +590,7 @@
unsigned long ias; /* IPA */
unsigned long oas; /* PA */
+ unsigned long pgsize_bitmap;
#define ARM_SMMU_MAX_ASIDS (1 << 16)
unsigned int asid_bits;
@@ -1516,8 +1517,6 @@
return 0;
}
-static struct iommu_ops arm_smmu_ops;
-
static int arm_smmu_domain_finalise(struct iommu_domain *domain)
{
int ret;
@@ -1555,7 +1554,7 @@
}
pgtbl_cfg = (struct io_pgtable_cfg) {
- .pgsize_bitmap = arm_smmu_ops.pgsize_bitmap,
+ .pgsize_bitmap = smmu->pgsize_bitmap,
.ias = ias,
.oas = oas,
.tlb = &arm_smmu_gather_ops,
@@ -1566,7 +1565,7 @@
if (!pgtbl_ops)
return -ENOMEM;
- arm_smmu_ops.pgsize_bitmap = pgtbl_cfg.pgsize_bitmap;
+ domain->pgsize_bitmap = pgtbl_cfg.pgsize_bitmap;
smmu_domain->pgtbl_ops = pgtbl_ops;
ret = finalise_stage_fn(smmu_domain, &pgtbl_cfg);
@@ -2410,7 +2409,6 @@
{
u32 reg;
bool coherent;
- unsigned long pgsize_bitmap = 0;
/* IDR0 */
reg = readl_relaxed(smmu->base + ARM_SMMU_IDR0);
@@ -2541,13 +2539,16 @@
/* Page sizes */
if (reg & IDR5_GRAN64K)
- pgsize_bitmap |= SZ_64K | SZ_512M;
+ smmu->pgsize_bitmap |= SZ_64K | SZ_512M;
if (reg & IDR5_GRAN16K)
- pgsize_bitmap |= SZ_16K | SZ_32M;
+ smmu->pgsize_bitmap |= SZ_16K | SZ_32M;
if (reg & IDR5_GRAN4K)
- pgsize_bitmap |= SZ_4K | SZ_2M | SZ_1G;
+ smmu->pgsize_bitmap |= SZ_4K | SZ_2M | SZ_1G;
- arm_smmu_ops.pgsize_bitmap &= pgsize_bitmap;
+ if (arm_smmu_ops.pgsize_bitmap == -1UL)
+ arm_smmu_ops.pgsize_bitmap = smmu->pgsize_bitmap;
+ else
+ arm_smmu_ops.pgsize_bitmap |= smmu->pgsize_bitmap;
/* Output address size */
switch (reg & IDR5_OAS_MASK << IDR5_OAS_SHIFT) {
diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c
index 7c39ac4..e206ce7 100644
--- a/drivers/iommu/arm-smmu.c
+++ b/drivers/iommu/arm-smmu.c
@@ -34,6 +34,7 @@
#include <linux/err.h>
#include <linux/interrupt.h>
#include <linux/io.h>
+#include <linux/io-64-nonatomic-hi-lo.h>
#include <linux/iommu.h>
#include <linux/iopoll.h>
#include <linux/module.h>
@@ -49,7 +50,7 @@
#include "io-pgtable.h"
/* Maximum number of stream IDs assigned to a single device */
-#define MAX_MASTER_STREAMIDS MAX_PHANDLE_ARGS
+#define MAX_MASTER_STREAMIDS 128
/* Maximum number of context banks per SMMU */
#define ARM_SMMU_MAX_CBS 128
@@ -71,16 +72,15 @@
((smmu->options & ARM_SMMU_OPT_SECURE_CFG_ACCESS) \
? 0x400 : 0))
+/*
+ * Some 64-bit registers only make sense to write atomically, but in such
+ * cases all the data relevant to AArch32 formats lies within the lower word,
+ * therefore this actually makes more sense than it might first appear.
+ */
#ifdef CONFIG_64BIT
-#define smmu_writeq writeq_relaxed
+#define smmu_write_atomic_lq writeq_relaxed
#else
-#define smmu_writeq(reg64, addr) \
- do { \
- u64 __val = (reg64); \
- void __iomem *__addr = (addr); \
- writel_relaxed(__val >> 32, __addr + 4); \
- writel_relaxed(__val, __addr); \
- } while (0)
+#define smmu_write_atomic_lq writel_relaxed
#endif
/* Configuration registers */
@@ -94,9 +94,13 @@
#define sCR0_VMIDPNE (1 << 11)
#define sCR0_PTM (1 << 12)
#define sCR0_FB (1 << 13)
+#define sCR0_VMID16EN (1 << 31)
#define sCR0_BSU_SHIFT 14
#define sCR0_BSU_MASK 0x3
+/* Auxiliary Configuration register */
+#define ARM_SMMU_GR0_sACR 0x10
+
/* Identification registers */
#define ARM_SMMU_GR0_ID0 0x20
#define ARM_SMMU_GR0_ID1 0x24
@@ -116,6 +120,8 @@
#define ID0_NTS (1 << 28)
#define ID0_SMS (1 << 27)
#define ID0_ATOSNS (1 << 26)
+#define ID0_PTFS_NO_AARCH32 (1 << 25)
+#define ID0_PTFS_NO_AARCH32S (1 << 24)
#define ID0_CTTW (1 << 14)
#define ID0_NUMIRPT_SHIFT 16
#define ID0_NUMIRPT_MASK 0xff
@@ -141,6 +147,10 @@
#define ID2_PTFS_4K (1 << 12)
#define ID2_PTFS_16K (1 << 13)
#define ID2_PTFS_64K (1 << 14)
+#define ID2_VMID16 (1 << 15)
+
+#define ID7_MAJOR_SHIFT 4
+#define ID7_MAJOR_MASK 0xf
/* Global TLB invalidation */
#define ARM_SMMU_GR0_TLBIVMID 0x64
@@ -193,12 +203,15 @@
#define ARM_SMMU_GR1_CBA2R(n) (0x800 + ((n) << 2))
#define CBA2R_RW64_32BIT (0 << 0)
#define CBA2R_RW64_64BIT (1 << 0)
+#define CBA2R_VMID_SHIFT 16
+#define CBA2R_VMID_MASK 0xffff
/* Translation context bank */
#define ARM_SMMU_CB_BASE(smmu) ((smmu)->base + ((smmu)->size >> 1))
#define ARM_SMMU_CB(smmu, n) ((n) * (1 << (smmu)->pgshift))
#define ARM_SMMU_CB_SCTLR 0x0
+#define ARM_SMMU_CB_ACTLR 0x4
#define ARM_SMMU_CB_RESUME 0x8
#define ARM_SMMU_CB_TTBCR2 0x10
#define ARM_SMMU_CB_TTBR0 0x20
@@ -206,11 +219,9 @@
#define ARM_SMMU_CB_TTBCR 0x30
#define ARM_SMMU_CB_S1_MAIR0 0x38
#define ARM_SMMU_CB_S1_MAIR1 0x3c
-#define ARM_SMMU_CB_PAR_LO 0x50
-#define ARM_SMMU_CB_PAR_HI 0x54
+#define ARM_SMMU_CB_PAR 0x50
#define ARM_SMMU_CB_FSR 0x58
-#define ARM_SMMU_CB_FAR_LO 0x60
-#define ARM_SMMU_CB_FAR_HI 0x64
+#define ARM_SMMU_CB_FAR 0x60
#define ARM_SMMU_CB_FSYNR0 0x68
#define ARM_SMMU_CB_S1_TLBIVA 0x600
#define ARM_SMMU_CB_S1_TLBIASID 0x610
@@ -230,6 +241,10 @@
#define SCTLR_M (1 << 0)
#define SCTLR_EAE_SBOP (SCTLR_AFE | SCTLR_TRE)
+#define ARM_MMU500_ACTLR_CPRE (1 << 1)
+
+#define ARM_MMU500_ACR_CACHE_LOCK (1 << 26)
+
#define CB_PAR_F (1 << 0)
#define ATSR_ACTIVE (1 << 0)
@@ -270,10 +285,17 @@
"Disable bypass streams such that incoming transactions from devices that are not attached to an iommu domain will report an abort back to the device and will not be allowed to pass through the SMMU.");
enum arm_smmu_arch_version {
- ARM_SMMU_V1 = 1,
+ ARM_SMMU_V1,
+ ARM_SMMU_V1_64K,
ARM_SMMU_V2,
};
+enum arm_smmu_implementation {
+ GENERIC_SMMU,
+ ARM_MMU500,
+ CAVIUM_SMMUV2,
+};
+
struct arm_smmu_smr {
u8 idx;
u16 mask;
@@ -305,11 +327,18 @@
#define ARM_SMMU_FEAT_TRANS_S2 (1 << 3)
#define ARM_SMMU_FEAT_TRANS_NESTED (1 << 4)
#define ARM_SMMU_FEAT_TRANS_OPS (1 << 5)
+#define ARM_SMMU_FEAT_VMID16 (1 << 6)
+#define ARM_SMMU_FEAT_FMT_AARCH64_4K (1 << 7)
+#define ARM_SMMU_FEAT_FMT_AARCH64_16K (1 << 8)
+#define ARM_SMMU_FEAT_FMT_AARCH64_64K (1 << 9)
+#define ARM_SMMU_FEAT_FMT_AARCH32_L (1 << 10)
+#define ARM_SMMU_FEAT_FMT_AARCH32_S (1 << 11)
u32 features;
#define ARM_SMMU_OPT_SECURE_CFG_ACCESS (1 << 0)
u32 options;
enum arm_smmu_arch_version version;
+ enum arm_smmu_implementation model;
u32 num_context_banks;
u32 num_s2_context_banks;
@@ -322,6 +351,7 @@
unsigned long va_size;
unsigned long ipa_size;
unsigned long pa_size;
+ unsigned long pgsize_bitmap;
u32 num_global_irqs;
u32 num_context_irqs;
@@ -329,17 +359,27 @@
struct list_head list;
struct rb_root masters;
+
+ u32 cavium_id_base; /* Specific to Cavium */
+};
+
+enum arm_smmu_context_fmt {
+ ARM_SMMU_CTX_FMT_NONE,
+ ARM_SMMU_CTX_FMT_AARCH64,
+ ARM_SMMU_CTX_FMT_AARCH32_L,
+ ARM_SMMU_CTX_FMT_AARCH32_S,
};
struct arm_smmu_cfg {
u8 cbndx;
u8 irptndx;
u32 cbar;
+ enum arm_smmu_context_fmt fmt;
};
#define INVALID_IRPTNDX 0xff
-#define ARM_SMMU_CB_ASID(cfg) ((cfg)->cbndx)
-#define ARM_SMMU_CB_VMID(cfg) ((cfg)->cbndx + 1)
+#define ARM_SMMU_CB_ASID(smmu, cfg) ((u16)(smmu)->cavium_id_base + (cfg)->cbndx)
+#define ARM_SMMU_CB_VMID(smmu, cfg) ((u16)(smmu)->cavium_id_base + (cfg)->cbndx + 1)
enum arm_smmu_domain_stage {
ARM_SMMU_DOMAIN_S1 = 0,
@@ -357,7 +397,11 @@
struct iommu_domain domain;
};
-static struct iommu_ops arm_smmu_ops;
+struct arm_smmu_phandle_args {
+ struct device_node *np;
+ int args_count;
+ uint32_t args[MAX_MASTER_STREAMIDS];
+};
static DEFINE_SPINLOCK(arm_smmu_devices_lock);
static LIST_HEAD(arm_smmu_devices);
@@ -367,6 +411,8 @@
const char *prop;
};
+static atomic_t cavium_smmu_context_count = ATOMIC_INIT(0);
+
static struct arm_smmu_option_prop arm_smmu_options[] = {
{ ARM_SMMU_OPT_SECURE_CFG_ACCESS, "calxeda,smmu-secure-config-access" },
{ 0, NULL},
@@ -466,7 +512,7 @@
static int register_smmu_master(struct arm_smmu_device *smmu,
struct device *dev,
- struct of_phandle_args *masterspec)
+ struct arm_smmu_phandle_args *masterspec)
{
int i;
struct arm_smmu_master *master;
@@ -578,11 +624,11 @@
if (stage1) {
base = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB(smmu, cfg->cbndx);
- writel_relaxed(ARM_SMMU_CB_ASID(cfg),
+ writel_relaxed(ARM_SMMU_CB_ASID(smmu, cfg),
base + ARM_SMMU_CB_S1_TLBIASID);
} else {
base = ARM_SMMU_GR0(smmu);
- writel_relaxed(ARM_SMMU_CB_VMID(cfg),
+ writel_relaxed(ARM_SMMU_CB_VMID(smmu, cfg),
base + ARM_SMMU_GR0_TLBIVMID);
}
@@ -602,37 +648,33 @@
reg = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB(smmu, cfg->cbndx);
reg += leaf ? ARM_SMMU_CB_S1_TLBIVAL : ARM_SMMU_CB_S1_TLBIVA;
- if (!IS_ENABLED(CONFIG_64BIT) || smmu->version == ARM_SMMU_V1) {
+ if (cfg->fmt != ARM_SMMU_CTX_FMT_AARCH64) {
iova &= ~12UL;
- iova |= ARM_SMMU_CB_ASID(cfg);
+ iova |= ARM_SMMU_CB_ASID(smmu, cfg);
do {
writel_relaxed(iova, reg);
iova += granule;
} while (size -= granule);
-#ifdef CONFIG_64BIT
} else {
iova >>= 12;
- iova |= (u64)ARM_SMMU_CB_ASID(cfg) << 48;
+ iova |= (u64)ARM_SMMU_CB_ASID(smmu, cfg) << 48;
do {
writeq_relaxed(iova, reg);
iova += granule >> 12;
} while (size -= granule);
-#endif
}
-#ifdef CONFIG_64BIT
} else if (smmu->version == ARM_SMMU_V2) {
reg = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB(smmu, cfg->cbndx);
reg += leaf ? ARM_SMMU_CB_S2_TLBIIPAS2L :
ARM_SMMU_CB_S2_TLBIIPAS2;
iova >>= 12;
do {
- writeq_relaxed(iova, reg);
+ smmu_write_atomic_lq(iova, reg);
iova += granule >> 12;
} while (size -= granule);
-#endif
} else {
reg = ARM_SMMU_GR0(smmu) + ARM_SMMU_GR0_TLBIVMID;
- writel_relaxed(ARM_SMMU_CB_VMID(cfg), reg);
+ writel_relaxed(ARM_SMMU_CB_VMID(smmu, cfg), reg);
}
}
@@ -645,7 +687,7 @@
static irqreturn_t arm_smmu_context_fault(int irq, void *dev)
{
int flags, ret;
- u32 fsr, far, fsynr, resume;
+ u32 fsr, fsynr, resume;
unsigned long iova;
struct iommu_domain *domain = dev;
struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
@@ -667,13 +709,7 @@
fsynr = readl_relaxed(cb_base + ARM_SMMU_CB_FSYNR0);
flags = fsynr & FSYNR0_WNR ? IOMMU_FAULT_WRITE : IOMMU_FAULT_READ;
- far = readl_relaxed(cb_base + ARM_SMMU_CB_FAR_LO);
- iova = far;
-#ifdef CONFIG_64BIT
- far = readl_relaxed(cb_base + ARM_SMMU_CB_FAR_HI);
- iova |= ((unsigned long)far << 32);
-#endif
-
+ iova = readq_relaxed(cb_base + ARM_SMMU_CB_FAR);
if (!report_iommu_fault(domain, smmu->dev, iova, flags)) {
ret = IRQ_HANDLED;
resume = RESUME_RETRY;
@@ -734,22 +770,20 @@
cb_base = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB(smmu, cfg->cbndx);
if (smmu->version > ARM_SMMU_V1) {
- /*
- * CBA2R.
- * *Must* be initialised before CBAR thanks to VMID16
- * architectural oversight affected some implementations.
- */
-#ifdef CONFIG_64BIT
- reg = CBA2R_RW64_64BIT;
-#else
- reg = CBA2R_RW64_32BIT;
-#endif
+ if (cfg->fmt == ARM_SMMU_CTX_FMT_AARCH64)
+ reg = CBA2R_RW64_64BIT;
+ else
+ reg = CBA2R_RW64_32BIT;
+ /* 16-bit VMIDs live in CBA2R */
+ if (smmu->features & ARM_SMMU_FEAT_VMID16)
+ reg |= ARM_SMMU_CB_VMID(smmu, cfg) << CBA2R_VMID_SHIFT;
+
writel_relaxed(reg, gr1_base + ARM_SMMU_GR1_CBA2R(cfg->cbndx));
}
/* CBAR */
reg = cfg->cbar;
- if (smmu->version == ARM_SMMU_V1)
+ if (smmu->version < ARM_SMMU_V2)
reg |= cfg->irptndx << CBAR_IRPTNDX_SHIFT;
/*
@@ -759,8 +793,9 @@
if (stage1) {
reg |= (CBAR_S1_BPSHCFG_NSH << CBAR_S1_BPSHCFG_SHIFT) |
(CBAR_S1_MEMATTR_WB << CBAR_S1_MEMATTR_SHIFT);
- } else {
- reg |= ARM_SMMU_CB_VMID(cfg) << CBAR_VMID_SHIFT;
+ } else if (!(smmu->features & ARM_SMMU_FEAT_VMID16)) {
+ /* 8-bit VMIDs live in CBAR */
+ reg |= ARM_SMMU_CB_VMID(smmu, cfg) << CBAR_VMID_SHIFT;
}
writel_relaxed(reg, gr1_base + ARM_SMMU_GR1_CBAR(cfg->cbndx));
@@ -768,15 +803,15 @@
if (stage1) {
reg64 = pgtbl_cfg->arm_lpae_s1_cfg.ttbr[0];
- reg64 |= ((u64)ARM_SMMU_CB_ASID(cfg)) << TTBRn_ASID_SHIFT;
- smmu_writeq(reg64, cb_base + ARM_SMMU_CB_TTBR0);
+ reg64 |= ((u64)ARM_SMMU_CB_ASID(smmu, cfg)) << TTBRn_ASID_SHIFT;
+ writeq_relaxed(reg64, cb_base + ARM_SMMU_CB_TTBR0);
reg64 = pgtbl_cfg->arm_lpae_s1_cfg.ttbr[1];
- reg64 |= ((u64)ARM_SMMU_CB_ASID(cfg)) << TTBRn_ASID_SHIFT;
- smmu_writeq(reg64, cb_base + ARM_SMMU_CB_TTBR1);
+ reg64 |= ((u64)ARM_SMMU_CB_ASID(smmu, cfg)) << TTBRn_ASID_SHIFT;
+ writeq_relaxed(reg64, cb_base + ARM_SMMU_CB_TTBR1);
} else {
reg64 = pgtbl_cfg->arm_lpae_s2_cfg.vttbr;
- smmu_writeq(reg64, cb_base + ARM_SMMU_CB_TTBR0);
+ writeq_relaxed(reg64, cb_base + ARM_SMMU_CB_TTBR0);
}
/* TTBCR */
@@ -855,16 +890,40 @@
if (!(smmu->features & ARM_SMMU_FEAT_TRANS_S2))
smmu_domain->stage = ARM_SMMU_DOMAIN_S1;
+ /*
+ * Choosing a suitable context format is even more fiddly. Until we
+ * grow some way for the caller to express a preference, and/or move
+ * the decision into the io-pgtable code where it arguably belongs,
+ * just aim for the closest thing to the rest of the system, and hope
+ * that the hardware isn't esoteric enough that we can't assume AArch64
+ * support to be a superset of AArch32 support...
+ */
+ if (smmu->features & ARM_SMMU_FEAT_FMT_AARCH32_L)
+ cfg->fmt = ARM_SMMU_CTX_FMT_AARCH32_L;
+ if ((IS_ENABLED(CONFIG_64BIT) || cfg->fmt == ARM_SMMU_CTX_FMT_NONE) &&
+ (smmu->features & (ARM_SMMU_FEAT_FMT_AARCH64_64K |
+ ARM_SMMU_FEAT_FMT_AARCH64_16K |
+ ARM_SMMU_FEAT_FMT_AARCH64_4K)))
+ cfg->fmt = ARM_SMMU_CTX_FMT_AARCH64;
+
+ if (cfg->fmt == ARM_SMMU_CTX_FMT_NONE) {
+ ret = -EINVAL;
+ goto out_unlock;
+ }
+
switch (smmu_domain->stage) {
case ARM_SMMU_DOMAIN_S1:
cfg->cbar = CBAR_TYPE_S1_TRANS_S2_BYPASS;
start = smmu->num_s2_context_banks;
ias = smmu->va_size;
oas = smmu->ipa_size;
- if (IS_ENABLED(CONFIG_64BIT))
+ if (cfg->fmt == ARM_SMMU_CTX_FMT_AARCH64) {
fmt = ARM_64_LPAE_S1;
- else
+ } else {
fmt = ARM_32_LPAE_S1;
+ ias = min(ias, 32UL);
+ oas = min(oas, 40UL);
+ }
break;
case ARM_SMMU_DOMAIN_NESTED:
/*
@@ -876,10 +935,13 @@
start = 0;
ias = smmu->ipa_size;
oas = smmu->pa_size;
- if (IS_ENABLED(CONFIG_64BIT))
+ if (cfg->fmt == ARM_SMMU_CTX_FMT_AARCH64) {
fmt = ARM_64_LPAE_S2;
- else
+ } else {
fmt = ARM_32_LPAE_S2;
+ ias = min(ias, 40UL);
+ oas = min(oas, 40UL);
+ }
break;
default:
ret = -EINVAL;
@@ -892,7 +954,7 @@
goto out_unlock;
cfg->cbndx = ret;
- if (smmu->version == ARM_SMMU_V1) {
+ if (smmu->version < ARM_SMMU_V2) {
cfg->irptndx = atomic_inc_return(&smmu->irptndx);
cfg->irptndx %= smmu->num_context_irqs;
} else {
@@ -900,7 +962,7 @@
}
pgtbl_cfg = (struct io_pgtable_cfg) {
- .pgsize_bitmap = arm_smmu_ops.pgsize_bitmap,
+ .pgsize_bitmap = smmu->pgsize_bitmap,
.ias = ias,
.oas = oas,
.tlb = &arm_smmu_gather_ops,
@@ -914,8 +976,8 @@
goto out_clear_smmu;
}
- /* Update our support page sizes to reflect the page table format */
- arm_smmu_ops.pgsize_bitmap = pgtbl_cfg.pgsize_bitmap;
+ /* Update the domain's page sizes to reflect the page table format */
+ domain->pgsize_bitmap = pgtbl_cfg.pgsize_bitmap;
/* Initialise the context bank with our page table cfg */
arm_smmu_init_context_bank(smmu_domain, &pgtbl_cfg);
@@ -1252,8 +1314,8 @@
/* ATS1 registers can only be written atomically */
va = iova & ~0xfffUL;
if (smmu->version == ARM_SMMU_V2)
- smmu_writeq(va, cb_base + ARM_SMMU_CB_ATS1PR);
- else
+ smmu_write_atomic_lq(va, cb_base + ARM_SMMU_CB_ATS1PR);
+ else /* Register is only 32-bit in v1 */
writel_relaxed(va, cb_base + ARM_SMMU_CB_ATS1PR);
if (readl_poll_timeout_atomic(cb_base + ARM_SMMU_CB_ATSR, tmp,
@@ -1264,9 +1326,7 @@
return ops->iova_to_phys(ops, iova);
}
- phys = readl_relaxed(cb_base + ARM_SMMU_CB_PAR_LO);
- phys |= ((u64)readl_relaxed(cb_base + ARM_SMMU_CB_PAR_HI)) << 32;
-
+ phys = readq_relaxed(cb_base + ARM_SMMU_CB_PAR);
if (phys & CB_PAR_F) {
dev_err(dev, "translation fault!\n");
dev_err(dev, "PAR = 0x%llx\n", phys);
@@ -1492,7 +1552,7 @@
void __iomem *gr0_base = ARM_SMMU_GR0(smmu);
void __iomem *cb_base;
int i = 0;
- u32 reg;
+ u32 reg, major;
/* clear global FSR */
reg = readl_relaxed(ARM_SMMU_GR0_NS(smmu) + ARM_SMMU_GR0_sGFSR);
@@ -1505,11 +1565,33 @@
writel_relaxed(reg, gr0_base + ARM_SMMU_GR0_S2CR(i));
}
+ /*
+ * Before clearing ARM_MMU500_ACTLR_CPRE, need to
+ * clear CACHE_LOCK bit of ACR first. And, CACHE_LOCK
+ * bit is only present in MMU-500r2 onwards.
+ */
+ reg = readl_relaxed(gr0_base + ARM_SMMU_GR0_ID7);
+ major = (reg >> ID7_MAJOR_SHIFT) & ID7_MAJOR_MASK;
+ if ((smmu->model == ARM_MMU500) && (major >= 2)) {
+ reg = readl_relaxed(gr0_base + ARM_SMMU_GR0_sACR);
+ reg &= ~ARM_MMU500_ACR_CACHE_LOCK;
+ writel_relaxed(reg, gr0_base + ARM_SMMU_GR0_sACR);
+ }
+
/* Make sure all context banks are disabled and clear CB_FSR */
for (i = 0; i < smmu->num_context_banks; ++i) {
cb_base = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB(smmu, i);
writel_relaxed(0, cb_base + ARM_SMMU_CB_SCTLR);
writel_relaxed(FSR_FAULT, cb_base + ARM_SMMU_CB_FSR);
+ /*
+ * Disable MMU-500's not-particularly-beneficial next-page
+ * prefetcher for the sake of errata #841119 and #826419.
+ */
+ if (smmu->model == ARM_MMU500) {
+ reg = readl_relaxed(cb_base + ARM_SMMU_CB_ACTLR);
+ reg &= ~ARM_MMU500_ACTLR_CPRE;
+ writel_relaxed(reg, cb_base + ARM_SMMU_CB_ACTLR);
+ }
}
/* Invalidate the TLB, just in case */
@@ -1537,6 +1619,9 @@
/* Don't upgrade barriers */
reg &= ~(sCR0_BSU_MASK << sCR0_BSU_SHIFT);
+ if (smmu->features & ARM_SMMU_FEAT_VMID16)
+ reg |= sCR0_VMID16EN;
+
/* Push the button */
__arm_smmu_tlb_sync(smmu);
writel(reg, ARM_SMMU_GR0_NS(smmu) + ARM_SMMU_GR0_sCR0);
@@ -1569,7 +1654,8 @@
bool cttw_dt, cttw_reg;
dev_notice(smmu->dev, "probing hardware configuration...\n");
- dev_notice(smmu->dev, "SMMUv%d with:\n", smmu->version);
+ dev_notice(smmu->dev, "SMMUv%d with:\n",
+ smmu->version == ARM_SMMU_V2 ? 2 : 1);
/* ID0 */
id = readl_relaxed(gr0_base + ARM_SMMU_GR0_ID0);
@@ -1601,7 +1687,8 @@
return -ENODEV;
}
- if ((id & ID0_S1TS) && ((smmu->version == 1) || !(id & ID0_ATOSNS))) {
+ if ((id & ID0_S1TS) &&
+ ((smmu->version < ARM_SMMU_V2) || !(id & ID0_ATOSNS))) {
smmu->features |= ARM_SMMU_FEAT_TRANS_OPS;
dev_notice(smmu->dev, "\taddress translation ops\n");
}
@@ -1657,6 +1744,12 @@
ID0_NUMSIDB_MASK;
}
+ if (smmu->version < ARM_SMMU_V2 || !(id & ID0_PTFS_NO_AARCH32)) {
+ smmu->features |= ARM_SMMU_FEAT_FMT_AARCH32_L;
+ if (!(id & ID0_PTFS_NO_AARCH32S))
+ smmu->features |= ARM_SMMU_FEAT_FMT_AARCH32_S;
+ }
+
/* ID1 */
id = readl_relaxed(gr0_base + ARM_SMMU_GR0_ID1);
smmu->pgshift = (id & ID1_PAGESIZE) ? 16 : 12;
@@ -1677,6 +1770,17 @@
}
dev_notice(smmu->dev, "\t%u context banks (%u stage-2 only)\n",
smmu->num_context_banks, smmu->num_s2_context_banks);
+ /*
+ * Cavium CN88xx erratum #27704.
+ * Ensure ASID and VMID allocation is unique across all SMMUs in
+ * the system.
+ */
+ if (smmu->model == CAVIUM_SMMUV2) {
+ smmu->cavium_id_base =
+ atomic_add_return(smmu->num_context_banks,
+ &cavium_smmu_context_count);
+ smmu->cavium_id_base -= smmu->num_context_banks;
+ }
/* ID2 */
id = readl_relaxed(gr0_base + ARM_SMMU_GR0_ID2);
@@ -1687,6 +1791,9 @@
size = arm_smmu_id_size_to_bits((id >> ID2_OAS_SHIFT) & ID2_OAS_MASK);
smmu->pa_size = size;
+ if (id & ID2_VMID16)
+ smmu->features |= ARM_SMMU_FEAT_VMID16;
+
/*
* What the page table walker can address actually depends on which
* descriptor format is in use, but since a) we don't know that yet,
@@ -1696,26 +1803,39 @@
dev_warn(smmu->dev,
"failed to set DMA mask for table walker\n");
- if (smmu->version == ARM_SMMU_V1) {
+ if (smmu->version < ARM_SMMU_V2) {
smmu->va_size = smmu->ipa_size;
- size = SZ_4K | SZ_2M | SZ_1G;
+ if (smmu->version == ARM_SMMU_V1_64K)
+ smmu->features |= ARM_SMMU_FEAT_FMT_AARCH64_64K;
} else {
size = (id >> ID2_UBS_SHIFT) & ID2_UBS_MASK;
smmu->va_size = arm_smmu_id_size_to_bits(size);
-#ifndef CONFIG_64BIT
- smmu->va_size = min(32UL, smmu->va_size);
-#endif
- size = 0;
if (id & ID2_PTFS_4K)
- size |= SZ_4K | SZ_2M | SZ_1G;
+ smmu->features |= ARM_SMMU_FEAT_FMT_AARCH64_4K;
if (id & ID2_PTFS_16K)
- size |= SZ_16K | SZ_32M;
+ smmu->features |= ARM_SMMU_FEAT_FMT_AARCH64_16K;
if (id & ID2_PTFS_64K)
- size |= SZ_64K | SZ_512M;
+ smmu->features |= ARM_SMMU_FEAT_FMT_AARCH64_64K;
}
- arm_smmu_ops.pgsize_bitmap &= size;
- dev_notice(smmu->dev, "\tSupported page sizes: 0x%08lx\n", size);
+ /* Now we've corralled the various formats, what'll it do? */
+ if (smmu->features & ARM_SMMU_FEAT_FMT_AARCH32_S)
+ smmu->pgsize_bitmap |= SZ_4K | SZ_64K | SZ_1M | SZ_16M;
+ if (smmu->features &
+ (ARM_SMMU_FEAT_FMT_AARCH32_L | ARM_SMMU_FEAT_FMT_AARCH64_4K))
+ smmu->pgsize_bitmap |= SZ_4K | SZ_2M | SZ_1G;
+ if (smmu->features & ARM_SMMU_FEAT_FMT_AARCH64_16K)
+ smmu->pgsize_bitmap |= SZ_16K | SZ_32M;
+ if (smmu->features & ARM_SMMU_FEAT_FMT_AARCH64_64K)
+ smmu->pgsize_bitmap |= SZ_64K | SZ_512M;
+
+ if (arm_smmu_ops.pgsize_bitmap == -1UL)
+ arm_smmu_ops.pgsize_bitmap = smmu->pgsize_bitmap;
+ else
+ arm_smmu_ops.pgsize_bitmap |= smmu->pgsize_bitmap;
+ dev_notice(smmu->dev, "\tSupported page sizes: 0x%08lx\n",
+ smmu->pgsize_bitmap);
+
if (smmu->features & ARM_SMMU_FEAT_TRANS_S1)
dev_notice(smmu->dev, "\tStage-1: %lu-bit VA -> %lu-bit IPA\n",
@@ -1728,12 +1848,27 @@
return 0;
}
+struct arm_smmu_match_data {
+ enum arm_smmu_arch_version version;
+ enum arm_smmu_implementation model;
+};
+
+#define ARM_SMMU_MATCH_DATA(name, ver, imp) \
+static struct arm_smmu_match_data name = { .version = ver, .model = imp }
+
+ARM_SMMU_MATCH_DATA(smmu_generic_v1, ARM_SMMU_V1, GENERIC_SMMU);
+ARM_SMMU_MATCH_DATA(smmu_generic_v2, ARM_SMMU_V2, GENERIC_SMMU);
+ARM_SMMU_MATCH_DATA(arm_mmu401, ARM_SMMU_V1_64K, GENERIC_SMMU);
+ARM_SMMU_MATCH_DATA(arm_mmu500, ARM_SMMU_V2, ARM_MMU500);
+ARM_SMMU_MATCH_DATA(cavium_smmuv2, ARM_SMMU_V2, CAVIUM_SMMUV2);
+
static const struct of_device_id arm_smmu_of_match[] = {
- { .compatible = "arm,smmu-v1", .data = (void *)ARM_SMMU_V1 },
- { .compatible = "arm,smmu-v2", .data = (void *)ARM_SMMU_V2 },
- { .compatible = "arm,mmu-400", .data = (void *)ARM_SMMU_V1 },
- { .compatible = "arm,mmu-401", .data = (void *)ARM_SMMU_V1 },
- { .compatible = "arm,mmu-500", .data = (void *)ARM_SMMU_V2 },
+ { .compatible = "arm,smmu-v1", .data = &smmu_generic_v1 },
+ { .compatible = "arm,smmu-v2", .data = &smmu_generic_v2 },
+ { .compatible = "arm,mmu-400", .data = &smmu_generic_v1 },
+ { .compatible = "arm,mmu-401", .data = &arm_mmu401 },
+ { .compatible = "arm,mmu-500", .data = &arm_mmu500 },
+ { .compatible = "cavium,smmu-v2", .data = &cavium_smmuv2 },
{ },
};
MODULE_DEVICE_TABLE(of, arm_smmu_of_match);
@@ -1741,11 +1876,13 @@
static int arm_smmu_device_dt_probe(struct platform_device *pdev)
{
const struct of_device_id *of_id;
+ const struct arm_smmu_match_data *data;
struct resource *res;
struct arm_smmu_device *smmu;
struct device *dev = &pdev->dev;
struct rb_node *node;
- struct of_phandle_args masterspec;
+ struct of_phandle_iterator it;
+ struct arm_smmu_phandle_args *masterspec;
int num_irqs, i, err;
smmu = devm_kzalloc(dev, sizeof(*smmu), GFP_KERNEL);
@@ -1756,7 +1893,9 @@
smmu->dev = dev;
of_id = of_match_node(arm_smmu_of_match, dev->of_node);
- smmu->version = (enum arm_smmu_arch_version)of_id->data;
+ data = of_id->data;
+ smmu->version = data->version;
+ smmu->model = data->model;
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
smmu->base = devm_ioremap_resource(dev, res);
@@ -1806,23 +1945,38 @@
i = 0;
smmu->masters = RB_ROOT;
- while (!of_parse_phandle_with_args(dev->of_node, "mmu-masters",
- "#stream-id-cells", i,
- &masterspec)) {
- err = register_smmu_master(smmu, dev, &masterspec);
+
+ err = -ENOMEM;
+ /* No need to zero the memory for masterspec */
+ masterspec = kmalloc(sizeof(*masterspec), GFP_KERNEL);
+ if (!masterspec)
+ goto out_put_masters;
+
+ of_for_each_phandle(&it, err, dev->of_node,
+ "mmu-masters", "#stream-id-cells", 0) {
+ int count = of_phandle_iterator_args(&it, masterspec->args,
+ MAX_MASTER_STREAMIDS);
+ masterspec->np = of_node_get(it.node);
+ masterspec->args_count = count;
+
+ err = register_smmu_master(smmu, dev, masterspec);
if (err) {
dev_err(dev, "failed to add master %s\n",
- masterspec.np->name);
+ masterspec->np->name);
+ kfree(masterspec);
goto out_put_masters;
}
i++;
}
+
dev_notice(dev, "registered %d master devices\n", i);
+ kfree(masterspec);
+
parse_driver_options(smmu);
- if (smmu->version > ARM_SMMU_V1 &&
+ if (smmu->version == ARM_SMMU_V2 &&
smmu->num_context_banks != smmu->num_context_irqs) {
dev_err(dev,
"found only %d context interrupt(s) but %d required\n",
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 58f2fe6..ea5a9eb 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -94,7 +94,7 @@
return -ENODEV;
/* Use the smallest supported page size for IOVA granularity */
- order = __ffs(domain->ops->pgsize_bitmap);
+ order = __ffs(domain->pgsize_bitmap);
base_pfn = max_t(unsigned long, 1, base >> order);
end_pfn = (base + size - 1) >> order;
@@ -190,11 +190,15 @@
kvfree(pages);
}
-static struct page **__iommu_dma_alloc_pages(unsigned int count, gfp_t gfp)
+static struct page **__iommu_dma_alloc_pages(unsigned int count,
+ unsigned long order_mask, gfp_t gfp)
{
struct page **pages;
unsigned int i = 0, array_size = count * sizeof(*pages);
- unsigned int order = MAX_ORDER;
+
+ order_mask &= (2U << MAX_ORDER) - 1;
+ if (!order_mask)
+ return NULL;
if (array_size <= PAGE_SIZE)
pages = kzalloc(array_size, GFP_KERNEL);
@@ -208,36 +212,38 @@
while (count) {
struct page *page = NULL;
- int j;
+ unsigned int order_size;
/*
* Higher-order allocations are a convenience rather
* than a necessity, hence using __GFP_NORETRY until
- * falling back to single-page allocations.
+ * falling back to minimum-order allocations.
*/
- for (order = min_t(unsigned int, order, __fls(count));
- order > 0; order--) {
- page = alloc_pages(gfp | __GFP_NORETRY, order);
+ for (order_mask &= (2U << __fls(count)) - 1;
+ order_mask; order_mask &= ~order_size) {
+ unsigned int order = __fls(order_mask);
+
+ order_size = 1U << order;
+ page = alloc_pages((order_mask - order_size) ?
+ gfp | __GFP_NORETRY : gfp, order);
if (!page)
continue;
- if (PageCompound(page)) {
- if (!split_huge_page(page))
- break;
- __free_pages(page, order);
- } else {
+ if (!order)
+ break;
+ if (!PageCompound(page)) {
split_page(page, order);
break;
+ } else if (!split_huge_page(page)) {
+ break;
}
+ __free_pages(page, order);
}
- if (!page)
- page = alloc_page(gfp);
if (!page) {
__iommu_dma_free_pages(pages, i);
return NULL;
}
- j = 1 << order;
- count -= j;
- while (j--)
+ count -= order_size;
+ while (order_size--)
pages[i++] = page++;
}
return pages;
@@ -267,6 +273,7 @@
* attached to an iommu_dma_domain
* @size: Size of buffer in bytes
* @gfp: Allocation flags
+ * @attrs: DMA attributes for this allocation
* @prot: IOMMU mapping flags
* @handle: Out argument for allocated DMA handle
* @flush_page: Arch callback which must ensure PAGE_SIZE bytes from the
@@ -278,8 +285,8 @@
* Return: Array of struct page pointers describing the buffer,
* or NULL on failure.
*/
-struct page **iommu_dma_alloc(struct device *dev, size_t size,
- gfp_t gfp, int prot, dma_addr_t *handle,
+struct page **iommu_dma_alloc(struct device *dev, size_t size, gfp_t gfp,
+ struct dma_attrs *attrs, int prot, dma_addr_t *handle,
void (*flush_page)(struct device *, const void *, phys_addr_t))
{
struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
@@ -288,11 +295,22 @@
struct page **pages;
struct sg_table sgt;
dma_addr_t dma_addr;
- unsigned int count = PAGE_ALIGN(size) >> PAGE_SHIFT;
+ unsigned int count, min_size, alloc_sizes = domain->pgsize_bitmap;
*handle = DMA_ERROR_CODE;
- pages = __iommu_dma_alloc_pages(count, gfp);
+ min_size = alloc_sizes & -alloc_sizes;
+ if (min_size < PAGE_SIZE) {
+ min_size = PAGE_SIZE;
+ alloc_sizes |= PAGE_SIZE;
+ } else {
+ size = ALIGN(size, min_size);
+ }
+ if (dma_get_attr(DMA_ATTR_ALLOC_SINGLE_PAGES, attrs))
+ alloc_sizes = min_size;
+
+ count = PAGE_ALIGN(size) >> PAGE_SHIFT;
+ pages = __iommu_dma_alloc_pages(count, alloc_sizes >> PAGE_SHIFT, gfp);
if (!pages)
return NULL;
@@ -389,26 +407,58 @@
/*
* Prepare a successfully-mapped scatterlist to give back to the caller.
- * Handling IOVA concatenation can come later, if needed
+ *
+ * At this point the segments are already laid out by iommu_dma_map_sg() to
+ * avoid individually crossing any boundaries, so we merely need to check a
+ * segment's start address to avoid concatenating across one.
*/
static int __finalise_sg(struct device *dev, struct scatterlist *sg, int nents,
dma_addr_t dma_addr)
{
- struct scatterlist *s;
- int i;
+ struct scatterlist *s, *cur = sg;
+ unsigned long seg_mask = dma_get_seg_boundary(dev);
+ unsigned int cur_len = 0, max_len = dma_get_max_seg_size(dev);
+ int i, count = 0;
for_each_sg(sg, s, nents, i) {
- /* Un-swizzling the fields here, hence the naming mismatch */
- unsigned int s_offset = sg_dma_address(s);
+ /* Restore this segment's original unaligned fields first */
+ unsigned int s_iova_off = sg_dma_address(s);
unsigned int s_length = sg_dma_len(s);
- unsigned int s_dma_len = s->length;
+ unsigned int s_iova_len = s->length;
- s->offset += s_offset;
+ s->offset += s_iova_off;
s->length = s_length;
- sg_dma_address(s) = dma_addr + s_offset;
- dma_addr += s_dma_len;
+ sg_dma_address(s) = DMA_ERROR_CODE;
+ sg_dma_len(s) = 0;
+
+ /*
+ * Now fill in the real DMA data. If...
+ * - there is a valid output segment to append to
+ * - and this segment starts on an IOVA page boundary
+ * - but doesn't fall at a segment boundary
+ * - and wouldn't make the resulting output segment too long
+ */
+ if (cur_len && !s_iova_off && (dma_addr & seg_mask) &&
+ (cur_len + s_length <= max_len)) {
+ /* ...then concatenate it with the previous one */
+ cur_len += s_length;
+ } else {
+ /* Otherwise start the next output segment */
+ if (i > 0)
+ cur = sg_next(cur);
+ cur_len = s_length;
+ count++;
+
+ sg_dma_address(cur) = dma_addr + s_iova_off;
+ }
+
+ sg_dma_len(cur) = cur_len;
+ dma_addr += s_iova_len;
+
+ if (s_length + s_iova_off < s_iova_len)
+ cur_len = 0;
}
- return i;
+ return count;
}
/*
@@ -446,34 +496,40 @@
struct scatterlist *s, *prev = NULL;
dma_addr_t dma_addr;
size_t iova_len = 0;
+ unsigned long mask = dma_get_seg_boundary(dev);
int i;
/*
* Work out how much IOVA space we need, and align the segments to
* IOVA granules for the IOMMU driver to handle. With some clever
* trickery we can modify the list in-place, but reversibly, by
- * hiding the original data in the as-yet-unused DMA fields.
+ * stashing the unaligned parts in the as-yet-unused DMA fields.
*/
for_each_sg(sg, s, nents, i) {
- size_t s_offset = iova_offset(iovad, s->offset);
+ size_t s_iova_off = iova_offset(iovad, s->offset);
size_t s_length = s->length;
+ size_t pad_len = (mask - iova_len + 1) & mask;
- sg_dma_address(s) = s_offset;
+ sg_dma_address(s) = s_iova_off;
sg_dma_len(s) = s_length;
- s->offset -= s_offset;
- s_length = iova_align(iovad, s_length + s_offset);
+ s->offset -= s_iova_off;
+ s_length = iova_align(iovad, s_length + s_iova_off);
s->length = s_length;
/*
- * The simple way to avoid the rare case of a segment
- * crossing the boundary mask is to pad the previous one
- * to end at a naturally-aligned IOVA for this one's size,
- * at the cost of potentially over-allocating a little.
+ * Due to the alignment of our single IOVA allocation, we can
+ * depend on these assumptions about the segment boundary mask:
+ * - If mask size >= IOVA size, then the IOVA range cannot
+ * possibly fall across a boundary, so we don't care.
+ * - If mask size < IOVA size, then the IOVA range must start
+ * exactly on a boundary, therefore we can lay things out
+ * based purely on segment lengths without needing to know
+ * the actual addresses beforehand.
+ * - The mask must be a power of 2, so pad_len == 0 if
+ * iova_len == 0, thus we cannot dereference prev the first
+ * time through here (i.e. before it has a meaningful value).
*/
- if (prev) {
- size_t pad_len = roundup_pow_of_two(s_length);
-
- pad_len = (pad_len - iova_len) & (pad_len - 1);
+ if (pad_len && pad_len < s_length - 1) {
prev->length += pad_len;
iova_len += pad_len;
}
diff --git a/drivers/iommu/dmar.c b/drivers/iommu/dmar.c
index 8ffd756..6a86b5d 100644
--- a/drivers/iommu/dmar.c
+++ b/drivers/iommu/dmar.c
@@ -1579,18 +1579,14 @@
reason = dmar_get_fault_reason(fault_reason, &fault_type);
if (fault_type == INTR_REMAP)
- pr_err("INTR-REMAP: Request device [[%02x:%02x.%d] "
- "fault index %llx\n"
- "INTR-REMAP:[fault reason %02d] %s\n",
- (source_id >> 8), PCI_SLOT(source_id & 0xFF),
+ pr_err("[INTR-REMAP] Request device [%02x:%02x.%d] fault index %llx [fault reason %02d] %s\n",
+ source_id >> 8, PCI_SLOT(source_id & 0xFF),
PCI_FUNC(source_id & 0xFF), addr >> 48,
fault_reason, reason);
else
- pr_err("DMAR:[%s] Request device [%02x:%02x.%d] "
- "fault addr %llx \n"
- "DMAR:[fault reason %02d] %s\n",
- (type ? "DMA Read" : "DMA Write"),
- (source_id >> 8), PCI_SLOT(source_id & 0xFF),
+ pr_err("[%s] Request device [%02x:%02x.%d] fault addr %llx [fault reason %02d] %s\n",
+ type ? "DMA Read" : "DMA Write",
+ source_id >> 8, PCI_SLOT(source_id & 0xFF),
PCI_FUNC(source_id & 0xFF), addr, fault_reason, reason);
return 0;
}
@@ -1602,10 +1598,17 @@
int reg, fault_index;
u32 fault_status;
unsigned long flag;
+ bool ratelimited;
+ static DEFINE_RATELIMIT_STATE(rs,
+ DEFAULT_RATELIMIT_INTERVAL,
+ DEFAULT_RATELIMIT_BURST);
+
+ /* Disable printing, simply clear the fault when ratelimited */
+ ratelimited = !__ratelimit(&rs);
raw_spin_lock_irqsave(&iommu->register_lock, flag);
fault_status = readl(iommu->reg + DMAR_FSTS_REG);
- if (fault_status)
+ if (fault_status && !ratelimited)
pr_err("DRHD: handling fault status reg %x\n", fault_status);
/* TBD: ignore advanced fault log currently */
@@ -1627,24 +1630,28 @@
if (!(data & DMA_FRCD_F))
break;
- fault_reason = dma_frcd_fault_reason(data);
- type = dma_frcd_type(data);
+ if (!ratelimited) {
+ fault_reason = dma_frcd_fault_reason(data);
+ type = dma_frcd_type(data);
- data = readl(iommu->reg + reg +
- fault_index * PRIMARY_FAULT_REG_LEN + 8);
- source_id = dma_frcd_source_id(data);
+ data = readl(iommu->reg + reg +
+ fault_index * PRIMARY_FAULT_REG_LEN + 8);
+ source_id = dma_frcd_source_id(data);
- guest_addr = dmar_readq(iommu->reg + reg +
- fault_index * PRIMARY_FAULT_REG_LEN);
- guest_addr = dma_frcd_page_addr(guest_addr);
+ guest_addr = dmar_readq(iommu->reg + reg +
+ fault_index * PRIMARY_FAULT_REG_LEN);
+ guest_addr = dma_frcd_page_addr(guest_addr);
+ }
+
/* clear the fault */
writel(DMA_FRCD_F, iommu->reg + reg +
fault_index * PRIMARY_FAULT_REG_LEN + 12);
raw_spin_unlock_irqrestore(&iommu->register_lock, flag);
- dmar_fault_do_one(iommu, type, fault_reason,
- source_id, guest_addr);
+ if (!ratelimited)
+ dmar_fault_do_one(iommu, type, fault_reason,
+ source_id, guest_addr);
fault_index++;
if (fault_index >= cap_num_fault_regs(iommu->cap))
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index e1852e8..b2bfb95 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1143,7 +1143,7 @@
} while (!first_pte_in_page(++pte) && pfn <= last_pfn);
}
-/* free page table pages. last level pte should already be cleared */
+/* clear last level (leaf) ptes and free page table pages. */
static void dma_pte_free_pagetable(struct dmar_domain *domain,
unsigned long start_pfn,
unsigned long last_pfn)
diff --git a/drivers/iommu/io-pgtable-arm-v7s.c b/drivers/iommu/io-pgtable-arm-v7s.c
index 9488e3c..8c61399 100644
--- a/drivers/iommu/io-pgtable-arm-v7s.c
+++ b/drivers/iommu/io-pgtable-arm-v7s.c
@@ -121,6 +121,8 @@
#define ARM_V7S_TEX_MASK 0x7
#define ARM_V7S_ATTR_TEX(val) (((val) & ARM_V7S_TEX_MASK) << ARM_V7S_TEX_SHIFT)
+#define ARM_V7S_ATTR_MTK_4GB BIT(9) /* MTK extend it for 4GB mode */
+
/* *well, except for TEX on level 2 large pages, of course :( */
#define ARM_V7S_CONT_PAGE_TEX_SHIFT 6
#define ARM_V7S_CONT_PAGE_TEX_MASK (ARM_V7S_TEX_MASK << ARM_V7S_CONT_PAGE_TEX_SHIFT)
@@ -258,9 +260,10 @@
struct io_pgtable_cfg *cfg)
{
bool ap = !(cfg->quirks & IO_PGTABLE_QUIRK_NO_PERMS);
- arm_v7s_iopte pte = ARM_V7S_ATTR_NG | ARM_V7S_ATTR_S |
- ARM_V7S_ATTR_TEX(1);
+ arm_v7s_iopte pte = ARM_V7S_ATTR_NG | ARM_V7S_ATTR_S;
+ if (!(prot & IOMMU_MMIO))
+ pte |= ARM_V7S_ATTR_TEX(1);
if (ap) {
pte |= ARM_V7S_PTE_AF | ARM_V7S_PTE_AP_UNPRIV;
if (!(prot & IOMMU_WRITE))
@@ -270,7 +273,9 @@
if ((prot & IOMMU_NOEXEC) && ap)
pte |= ARM_V7S_ATTR_XN(lvl);
- if (prot & IOMMU_CACHE)
+ if (prot & IOMMU_MMIO)
+ pte |= ARM_V7S_ATTR_B;
+ else if (prot & IOMMU_CACHE)
pte |= ARM_V7S_ATTR_B | ARM_V7S_ATTR_C;
return pte;
@@ -279,10 +284,13 @@
static int arm_v7s_pte_to_prot(arm_v7s_iopte pte, int lvl)
{
int prot = IOMMU_READ;
+ arm_v7s_iopte attr = pte >> ARM_V7S_ATTR_SHIFT(lvl);
- if (pte & (ARM_V7S_PTE_AP_RDONLY << ARM_V7S_ATTR_SHIFT(lvl)))
+ if (attr & ARM_V7S_PTE_AP_RDONLY)
prot |= IOMMU_WRITE;
- if (pte & ARM_V7S_ATTR_C)
+ if ((attr & (ARM_V7S_TEX_MASK << ARM_V7S_TEX_SHIFT)) == 0)
+ prot |= IOMMU_MMIO;
+ else if (pte & ARM_V7S_ATTR_C)
prot |= IOMMU_CACHE;
return prot;
@@ -364,6 +372,9 @@
if (lvl == 1 && (cfg->quirks & IO_PGTABLE_QUIRK_ARM_NS))
pte |= ARM_V7S_ATTR_NS_SECTION;
+ if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_MTK_4GB)
+ pte |= ARM_V7S_ATTR_MTK_4GB;
+
if (num_entries > 1)
pte = arm_v7s_pte_to_cont(pte, lvl);
@@ -625,9 +636,15 @@
if (cfg->quirks & ~(IO_PGTABLE_QUIRK_ARM_NS |
IO_PGTABLE_QUIRK_NO_PERMS |
- IO_PGTABLE_QUIRK_TLBI_ON_MAP))
+ IO_PGTABLE_QUIRK_TLBI_ON_MAP |
+ IO_PGTABLE_QUIRK_ARM_MTK_4GB))
return NULL;
+ /* If ARM_MTK_4GB is enabled, the NO_PERMS is also expected. */
+ if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_MTK_4GB &&
+ !(cfg->quirks & IO_PGTABLE_QUIRK_NO_PERMS))
+ return NULL;
+
data = kmalloc(sizeof(*data), GFP_KERNEL);
if (!data)
return NULL;
diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c
index f433b51..a1ed1b7 100644
--- a/drivers/iommu/io-pgtable-arm.c
+++ b/drivers/iommu/io-pgtable-arm.c
@@ -355,7 +355,10 @@
if (!(prot & IOMMU_WRITE) && (prot & IOMMU_READ))
pte |= ARM_LPAE_PTE_AP_RDONLY;
- if (prot & IOMMU_CACHE)
+ if (prot & IOMMU_MMIO)
+ pte |= (ARM_LPAE_MAIR_ATTR_IDX_DEV
+ << ARM_LPAE_PTE_ATTRINDX_SHIFT);
+ else if (prot & IOMMU_CACHE)
pte |= (ARM_LPAE_MAIR_ATTR_IDX_CACHE
<< ARM_LPAE_PTE_ATTRINDX_SHIFT);
} else {
@@ -364,7 +367,9 @@
pte |= ARM_LPAE_PTE_HAP_READ;
if (prot & IOMMU_WRITE)
pte |= ARM_LPAE_PTE_HAP_WRITE;
- if (prot & IOMMU_CACHE)
+ if (prot & IOMMU_MMIO)
+ pte |= ARM_LPAE_PTE_MEMATTR_DEV;
+ else if (prot & IOMMU_CACHE)
pte |= ARM_LPAE_PTE_MEMATTR_OIWB;
else
pte |= ARM_LPAE_PTE_MEMATTR_NC;
diff --git a/drivers/iommu/io-pgtable.c b/drivers/iommu/io-pgtable.c
index 876f6a7..127558d 100644
--- a/drivers/iommu/io-pgtable.c
+++ b/drivers/iommu/io-pgtable.c
@@ -25,8 +25,7 @@
#include "io-pgtable.h"
static const struct io_pgtable_init_fns *
-io_pgtable_init_table[IO_PGTABLE_NUM_FMTS] =
-{
+io_pgtable_init_table[IO_PGTABLE_NUM_FMTS] = {
#ifdef CONFIG_IOMMU_IO_PGTABLE_LPAE
[ARM_32_LPAE_S1] = &io_pgtable_arm_32_lpae_s1_init_fns,
[ARM_32_LPAE_S2] = &io_pgtable_arm_32_lpae_s2_init_fns,
diff --git a/drivers/iommu/io-pgtable.h b/drivers/iommu/io-pgtable.h
index d4f5027..969d82c 100644
--- a/drivers/iommu/io-pgtable.h
+++ b/drivers/iommu/io-pgtable.h
@@ -60,10 +60,16 @@
* IO_PGTABLE_QUIRK_TLBI_ON_MAP: If the format forbids caching invalid
* (unmapped) entries but the hardware might do so anyway, perform
* TLB maintenance when mapping as well as when unmapping.
+ *
+ * IO_PGTABLE_QUIRK_ARM_MTK_4GB: (ARM v7s format) Set bit 9 in all
+ * PTEs, for Mediatek IOMMUs which treat it as a 33rd address bit
+ * when the SoC is in "4GB mode" and they can only access the high
+ * remap of DRAM (0x1_00000000 to 0x1_ffffffff).
*/
#define IO_PGTABLE_QUIRK_ARM_NS BIT(0)
#define IO_PGTABLE_QUIRK_NO_PERMS BIT(1)
#define IO_PGTABLE_QUIRK_TLBI_ON_MAP BIT(2)
+ #define IO_PGTABLE_QUIRK_ARM_MTK_4GB BIT(3)
unsigned long quirks;
unsigned long pgsize_bitmap;
unsigned int ias;
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index b9df141..3000051 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -337,9 +337,9 @@
if (!domain || domain->type != IOMMU_DOMAIN_DMA)
return 0;
- BUG_ON(!domain->ops->pgsize_bitmap);
+ BUG_ON(!domain->pgsize_bitmap);
- pg_size = 1UL << __ffs(domain->ops->pgsize_bitmap);
+ pg_size = 1UL << __ffs(domain->pgsize_bitmap);
INIT_LIST_HEAD(&mappings);
iommu_get_dm_regions(dev, &mappings);
@@ -660,8 +660,8 @@
}
/*
- * Look for aliases to or from the given device for exisiting groups. The
- * dma_alias_devfn only supports aliases on the same bus, therefore the search
+ * Look for aliases to or from the given device for existing groups. DMA
+ * aliases are only supported on the same bus, therefore the search
* space is quite small (especially since we're really only looking at pcie
* device, and therefore only expect multiple slots on the root complex or
* downstream switch ports). It's conceivable though that a pair of
@@ -686,11 +686,7 @@
continue;
/* We alias them or they alias us */
- if (((pdev->dev_flags & PCI_DEV_FLAGS_DMA_ALIAS_DEVFN) &&
- pdev->dma_alias_devfn == tmp->devfn) ||
- ((tmp->dev_flags & PCI_DEV_FLAGS_DMA_ALIAS_DEVFN) &&
- tmp->dma_alias_devfn == pdev->devfn)) {
-
+ if (pci_devs_are_dma_aliases(pdev, tmp)) {
group = get_pci_alias_group(tmp, devfns);
if (group) {
pci_dev_put(tmp);
@@ -1073,6 +1069,8 @@
domain->ops = bus->iommu_ops;
domain->type = type;
+ /* Assume all sizes by default; the driver may override this later */
+ domain->pgsize_bitmap = bus->iommu_ops->pgsize_bitmap;
return domain;
}
@@ -1297,7 +1295,7 @@
pgsize = (1UL << (pgsize_idx + 1)) - 1;
/* throw away page sizes not supported by the hardware */
- pgsize &= domain->ops->pgsize_bitmap;
+ pgsize &= domain->pgsize_bitmap;
/* make sure we're still sane */
BUG_ON(!pgsize);
@@ -1319,14 +1317,14 @@
int ret = 0;
if (unlikely(domain->ops->map == NULL ||
- domain->ops->pgsize_bitmap == 0UL))
+ domain->pgsize_bitmap == 0UL))
return -ENODEV;
if (unlikely(!(domain->type & __IOMMU_DOMAIN_PAGING)))
return -EINVAL;
/* find out the minimum page size supported */
- min_pagesz = 1 << __ffs(domain->ops->pgsize_bitmap);
+ min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
/*
* both the virtual address and the physical one, as well as
@@ -1373,14 +1371,14 @@
unsigned long orig_iova = iova;
if (unlikely(domain->ops->unmap == NULL ||
- domain->ops->pgsize_bitmap == 0UL))
+ domain->pgsize_bitmap == 0UL))
return -ENODEV;
if (unlikely(!(domain->type & __IOMMU_DOMAIN_PAGING)))
return -EINVAL;
/* find out the minimum page size supported */
- min_pagesz = 1 << __ffs(domain->ops->pgsize_bitmap);
+ min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
/*
* The virtual address, as well as the size of the mapping, must be
@@ -1426,10 +1424,10 @@
unsigned int i, min_pagesz;
int ret;
- if (unlikely(domain->ops->pgsize_bitmap == 0UL))
+ if (unlikely(domain->pgsize_bitmap == 0UL))
return 0;
- min_pagesz = 1 << __ffs(domain->ops->pgsize_bitmap);
+ min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
for_each_sg(sg, s, nents, i) {
phys_addr_t phys = page_to_phys(sg_page(s)) + s->offset;
@@ -1510,7 +1508,7 @@
break;
case DOMAIN_ATTR_PAGING:
paging = data;
- *paging = (domain->ops->pgsize_bitmap != 0UL);
+ *paging = (domain->pgsize_bitmap != 0UL);
break;
case DOMAIN_ATTR_WINDOWS:
count = data;
diff --git a/drivers/iommu/mtk_iommu.c b/drivers/iommu/mtk_iommu.c
index 929a66a..c3043d8 100644
--- a/drivers/iommu/mtk_iommu.c
+++ b/drivers/iommu/mtk_iommu.c
@@ -11,6 +11,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*/
+#include <linux/bootmem.h>
#include <linux/bug.h>
#include <linux/clk.h>
#include <linux/component.h>
@@ -56,7 +57,7 @@
#define F_MMU_TF_PROTECT_SEL(prot) (((prot) & 0x3) << 5)
#define REG_MMU_IVRP_PADDR 0x114
-#define F_MMU_IVRP_PA_SET(pa) ((pa) >> 1)
+#define F_MMU_IVRP_PA_SET(pa, ext) (((pa) >> 1) | ((!!(ext)) << 31))
#define REG_MMU_INT_CONTROL0 0x120
#define F_L2_MULIT_HIT_EN BIT(0)
@@ -125,6 +126,7 @@
struct mtk_iommu_domain *m4u_dom;
struct iommu_group *m4u_group;
struct mtk_smi_iommu smi_imu; /* SMI larb iommu info */
+ bool enable_4GB;
};
static struct iommu_ops mtk_iommu_ops;
@@ -257,6 +259,9 @@
.iommu_dev = data->dev,
};
+ if (data->enable_4GB)
+ dom->cfg.quirks |= IO_PGTABLE_QUIRK_ARM_MTK_4GB;
+
dom->iop = alloc_io_pgtable_ops(ARM_V7S, &dom->cfg, data);
if (!dom->iop) {
dev_err(data->dev, "Failed to alloc io pgtable\n");
@@ -264,7 +269,7 @@
}
/* Update our support page sizes bitmap */
- mtk_iommu_ops.pgsize_bitmap = dom->cfg.pgsize_bitmap;
+ dom->domain.pgsize_bitmap = dom->cfg.pgsize_bitmap;
writel(data->m4u_dom->cfg.arm_v7s_cfg.ttbr[0],
data->base + REG_MMU_PT_BASE_ADDR);
@@ -530,7 +535,7 @@
F_INT_PRETETCH_TRANSATION_FIFO_FAULT;
writel_relaxed(regval, data->base + REG_MMU_INT_MAIN_CONTROL);
- writel_relaxed(F_MMU_IVRP_PA_SET(data->protect_base),
+ writel_relaxed(F_MMU_IVRP_PA_SET(data->protect_base, data->enable_4GB),
data->base + REG_MMU_IVRP_PADDR);
writel_relaxed(0, data->base + REG_MMU_DCM_DIS);
@@ -591,6 +596,9 @@
return -ENOMEM;
data->protect_base = ALIGN(virt_to_phys(protect), MTK_PROTECT_PA_ALIGN);
+ /* Whether the current dram is over 4GB */
+ data->enable_4GB = !!(max_pfn > (0xffffffffUL >> PAGE_SHIFT));
+
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
data->base = devm_ioremap_resource(dev, res);
if (IS_ERR(data->base))
@@ -690,7 +698,7 @@
writel_relaxed(reg->ctrl_reg, base + REG_MMU_CTRL_REG);
writel_relaxed(reg->int_control0, base + REG_MMU_INT_CONTROL0);
writel_relaxed(reg->int_main_control, base + REG_MMU_INT_MAIN_CONTROL);
- writel_relaxed(F_MMU_IVRP_PA_SET(data->protect_base),
+ writel_relaxed(F_MMU_IVRP_PA_SET(data->protect_base, data->enable_4GB),
base + REG_MMU_IVRP_PADDR);
return 0;
}
diff --git a/drivers/iommu/of_iommu.c b/drivers/iommu/of_iommu.c
index 5fea665..af499aea 100644
--- a/drivers/iommu/of_iommu.c
+++ b/drivers/iommu/of_iommu.c
@@ -98,12 +98,12 @@
struct of_iommu_node {
struct list_head list;
struct device_node *np;
- struct iommu_ops *ops;
+ const struct iommu_ops *ops;
};
static LIST_HEAD(of_iommu_list);
static DEFINE_SPINLOCK(of_iommu_lock);
-void of_iommu_set_ops(struct device_node *np, struct iommu_ops *ops)
+void of_iommu_set_ops(struct device_node *np, const struct iommu_ops *ops)
{
struct of_iommu_node *iommu = kzalloc(sizeof(*iommu), GFP_KERNEL);
@@ -119,10 +119,10 @@
spin_unlock(&of_iommu_lock);
}
-struct iommu_ops *of_iommu_get_ops(struct device_node *np)
+const struct iommu_ops *of_iommu_get_ops(struct device_node *np)
{
struct of_iommu_node *node;
- struct iommu_ops *ops = NULL;
+ const struct iommu_ops *ops = NULL;
spin_lock(&of_iommu_lock);
list_for_each_entry(node, &of_iommu_list, list)
@@ -134,12 +134,12 @@
return ops;
}
-struct iommu_ops *of_iommu_configure(struct device *dev,
- struct device_node *master_np)
+const struct iommu_ops *of_iommu_configure(struct device *dev,
+ struct device_node *master_np)
{
struct of_phandle_args iommu_spec;
struct device_node *np;
- struct iommu_ops *ops = NULL;
+ const struct iommu_ops *ops = NULL;
int idx = 0;
/*
diff --git a/drivers/iommu/omap-iommu-debug.c b/drivers/iommu/omap-iommu-debug.c
index 9bc20e2..505548a 100644
--- a/drivers/iommu/omap-iommu-debug.c
+++ b/drivers/iommu/omap-iommu-debug.c
@@ -136,7 +136,7 @@
struct seq_file *s)
{
seq_printf(s, "%08x %08x %01x\n", cr->cam, cr->ram,
- (cr->cam & MMU_CAM_P) ? 1 : 0);
+ (cr->cam & MMU_CAM_P) ? 1 : 0);
return 0;
}
diff --git a/drivers/iommu/omap-iommu.c b/drivers/iommu/omap-iommu.c
index 3dc5b65..e2583cc 100644
--- a/drivers/iommu/omap-iommu.c
+++ b/drivers/iommu/omap-iommu.c
@@ -628,10 +628,12 @@
break;
default:
fn = NULL;
- BUG();
break;
}
+ if (WARN_ON(!fn))
+ return -EINVAL;
+
prot = get_iopte_attr(e);
spin_lock(&obj->page_table_lock);
@@ -987,7 +989,6 @@
{
struct omap_iommu *obj = platform_get_drvdata(pdev);
- iopgtable_clear_entry_all(obj);
omap_iommu_debugfs_remove(obj);
pm_runtime_disable(obj->dev);
@@ -1161,7 +1162,8 @@
* should never fail, but please keep this around to ensure
* we keep the hardware happy
*/
- BUG_ON(!IS_ALIGNED((long)omap_domain->pgtable, IOPGD_TABLE_SIZE));
+ if (WARN_ON(!IS_ALIGNED((long)omap_domain->pgtable, IOPGD_TABLE_SIZE)))
+ goto fail_align;
clean_dcache_area(omap_domain->pgtable, IOPGD_TABLE_SIZE);
spin_lock_init(&omap_domain->lock);
@@ -1172,6 +1174,8 @@
return &omap_domain->domain;
+fail_align:
+ kfree(omap_domain->pgtable);
fail_nomem:
kfree(omap_domain);
out:
diff --git a/drivers/iommu/rockchip-iommu.c b/drivers/iommu/rockchip-iommu.c
index 5710a06..c7d6156 100644
--- a/drivers/iommu/rockchip-iommu.c
+++ b/drivers/iommu/rockchip-iommu.c
@@ -1049,6 +1049,8 @@
for (i = 0; i < pdev->num_resources; i++) {
res = platform_get_resource(pdev, IORESOURCE_MEM, i);
+ if (!res)
+ continue;
iommu->bases[i] = devm_ioremap_resource(&pdev->dev, res);
if (IS_ERR(iommu->bases[i]))
continue;
diff --git a/drivers/ipack/devices/ipoctal.c b/drivers/ipack/devices/ipoctal.c
index 035d544..75dd15d 100644
--- a/drivers/ipack/devices/ipoctal.c
+++ b/drivers/ipack/devices/ipoctal.c
@@ -629,8 +629,7 @@
tty_port_hangup(&channel->tty_port);
ipoctal_reset_channel(channel);
-
- clear_bit(ASYNCB_INITIALIZED, &channel->tty_port.flags);
+ tty_port_set_initialized(&channel->tty_port, 0);
wake_up_interruptible(&channel->tty_port.open_wait);
}
@@ -642,7 +641,7 @@
return;
ipoctal_reset_channel(channel);
- clear_bit(ASYNCB_INITIALIZED, &channel->tty_port.flags);
+ tty_port_set_initialized(&channel->tty_port, 0);
}
static void ipoctal_cleanup(struct tty_struct *tty)
diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
index 81f88ad..fa33c50 100644
--- a/drivers/irqchip/Kconfig
+++ b/drivers/irqchip/Kconfig
@@ -253,3 +253,10 @@
config PARTITION_PERCPU
bool
+
+config EZNPS_GIC
+ bool "NPS400 Global Interrupt Manager (GIM)"
+ depends on ARC || (COMPILE_TEST && !64BIT)
+ select IRQ_DOMAIN
+ help
+ Support the EZchip NPS400 global interrupt controller
diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
index f828244..38853a1 100644
--- a/drivers/irqchip/Makefile
+++ b/drivers/irqchip/Makefile
@@ -68,3 +68,4 @@
obj-$(CONFIG_PIC32_EVIC) += irq-pic32-evic.o
obj-$(CONFIG_MVEBU_ODMI) += irq-mvebu-odmi.o
obj-$(CONFIG_LS_SCFG_MSI) += irq-ls-scfg-msi.o
+obj-$(CONFIG_EZNPS_GIC) += irq-eznps.o
diff --git a/drivers/irqchip/irq-eznps.c b/drivers/irqchip/irq-eznps.c
new file mode 100644
index 0000000..efbf0e4
--- /dev/null
+++ b/drivers/irqchip/irq-eznps.c
@@ -0,0 +1,165 @@
+/*
+ * Copyright (c) 2016, Mellanox Technologies. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/irq.h>
+#include <linux/irqdomain.h>
+#include <linux/irqchip.h>
+#include <soc/nps/common.h>
+
+#define NPS_NR_CPU_IRQS 8 /* number of interrupt lines of NPS400 CPU */
+#define NPS_TIMER0_IRQ 3
+
+/*
+ * NPS400 core includes an Interrupt Controller (IC) support.
+ * All cores can deactivate level irqs at first level control
+ * at cores mesh layer called MTM.
+ * For devices out side chip e.g. uart, network there is another
+ * level called Global Interrupt Manager (GIM).
+ * This second level can control level and edge interrupt.
+ *
+ * NOTE: AUX_IENABLE and CTOP_AUX_IACK are auxiliary registers
+ * with private HW copy per CPU.
+ */
+
+static void nps400_irq_mask(struct irq_data *irqd)
+{
+ unsigned int ienb;
+ unsigned int irq = irqd_to_hwirq(irqd);
+
+ ienb = read_aux_reg(AUX_IENABLE);
+ ienb &= ~(1 << irq);
+ write_aux_reg(AUX_IENABLE, ienb);
+}
+
+static void nps400_irq_unmask(struct irq_data *irqd)
+{
+ unsigned int ienb;
+ unsigned int irq = irqd_to_hwirq(irqd);
+
+ ienb = read_aux_reg(AUX_IENABLE);
+ ienb |= (1 << irq);
+ write_aux_reg(AUX_IENABLE, ienb);
+}
+
+static void nps400_irq_eoi_global(struct irq_data *irqd)
+{
+ unsigned int __maybe_unused irq = irqd_to_hwirq(irqd);
+
+ write_aux_reg(CTOP_AUX_IACK, 1 << irq);
+
+ /* Don't ack GIC before all device access attempts are done */
+ mb();
+
+ nps_ack_gic();
+}
+
+static void nps400_irq_eoi(struct irq_data *irqd)
+{
+ unsigned int __maybe_unused irq = irqd_to_hwirq(irqd);
+
+ write_aux_reg(CTOP_AUX_IACK, 1 << irq);
+}
+
+static struct irq_chip nps400_irq_chip_fasteoi = {
+ .name = "NPS400 IC Global",
+ .irq_mask = nps400_irq_mask,
+ .irq_unmask = nps400_irq_unmask,
+ .irq_eoi = nps400_irq_eoi_global,
+};
+
+static struct irq_chip nps400_irq_chip_percpu = {
+ .name = "NPS400 IC",
+ .irq_mask = nps400_irq_mask,
+ .irq_unmask = nps400_irq_unmask,
+ .irq_eoi = nps400_irq_eoi,
+};
+
+static int nps400_irq_map(struct irq_domain *d, unsigned int virq,
+ irq_hw_number_t hw)
+{
+ switch (hw) {
+ case NPS_TIMER0_IRQ:
+#ifdef CONFIG_SMP
+ case NPS_IPI_IRQ:
+#endif
+ irq_set_percpu_devid(virq);
+ irq_set_chip_and_handler(virq, &nps400_irq_chip_percpu,
+ handle_percpu_devid_irq);
+ break;
+ default:
+ irq_set_chip_and_handler(virq, &nps400_irq_chip_fasteoi,
+ handle_fasteoi_irq);
+ break;
+ }
+
+ return 0;
+}
+
+static const struct irq_domain_ops nps400_irq_ops = {
+ .xlate = irq_domain_xlate_onecell,
+ .map = nps400_irq_map,
+};
+
+static int __init nps400_of_init(struct device_node *node,
+ struct device_node *parent)
+{
+ static struct irq_domain *nps400_root_domain;
+
+ if (parent) {
+ pr_err("DeviceTree incore ic not a root irq controller\n");
+ return -EINVAL;
+ }
+
+ nps400_root_domain = irq_domain_add_linear(node, NPS_NR_CPU_IRQS,
+ &nps400_irq_ops, NULL);
+
+ if (!nps400_root_domain) {
+ pr_err("nps400 root irq domain not avail\n");
+ return -ENOMEM;
+ }
+
+ /*
+ * Needed for primary domain lookup to succeed
+ * This is a primary irqchip, and can never have a parent
+ */
+ irq_set_default_host(nps400_root_domain);
+
+#ifdef CONFIG_SMP
+ irq_create_mapping(nps400_root_domain, NPS_IPI_IRQ);
+#endif
+
+ return 0;
+}
+IRQCHIP_DECLARE(ezchip_nps400_ic, "ezchip,nps400-ic", nps400_of_init);
diff --git a/drivers/irqchip/irq-gic-common.c b/drivers/irqchip/irq-gic-common.c
index 97c0028..89e7423 100644
--- a/drivers/irqchip/irq-gic-common.c
+++ b/drivers/irqchip/irq-gic-common.c
@@ -21,6 +21,19 @@
#include "irq-gic-common.h"
+static const struct gic_kvm_info *gic_kvm_info;
+
+const struct gic_kvm_info *gic_get_kvm_info(void)
+{
+ return gic_kvm_info;
+}
+
+void gic_set_kvm_info(const struct gic_kvm_info *info)
+{
+ BUG_ON(gic_kvm_info != NULL);
+ gic_kvm_info = info;
+}
+
void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,
void *data)
{
diff --git a/drivers/irqchip/irq-gic-common.h b/drivers/irqchip/irq-gic-common.h
index fff697db..205e5fd 100644
--- a/drivers/irqchip/irq-gic-common.h
+++ b/drivers/irqchip/irq-gic-common.h
@@ -19,6 +19,7 @@
#include <linux/of.h>
#include <linux/irqdomain.h>
+#include <linux/irqchip/arm-gic-common.h>
struct gic_quirk {
const char *desc;
@@ -35,4 +36,6 @@
void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,
void *data);
+void gic_set_kvm_info(const struct gic_kvm_info *info);
+
#endif /* _IRQ_GIC_COMMON_H */
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 1a1ea4f..fb042ba 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -15,6 +15,8 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
+#define pr_fmt(fmt) "GICv3: " fmt
+
#include <linux/acpi.h>
#include <linux/cpu.h>
#include <linux/cpu_pm.h>
@@ -28,6 +30,7 @@
#include <linux/slab.h>
#include <linux/irqchip.h>
+#include <linux/irqchip/arm-gic-common.h>
#include <linux/irqchip/arm-gic-v3.h>
#include <linux/irqchip/irq-partition-percpu.h>
@@ -59,6 +62,8 @@
static struct gic_chip_data gic_data __read_mostly;
static struct static_key supports_deactivate = STATIC_KEY_INIT_TRUE;
+static struct gic_kvm_info gic_v3_kvm_info;
+
#define gic_data_rdist() (this_cpu_ptr(gic_data.rdists.rdist))
#define gic_data_rdist_rd_base() (gic_data_rdist()->rd_base)
#define gic_data_rdist_sgi_base() (gic_data_rdist_rd_base() + SZ_64K)
@@ -1002,7 +1007,7 @@
}
/* Create all possible partitions at boot time */
-static void gic_populate_ppi_partitions(struct device_node *gic_node)
+static void __init gic_populate_ppi_partitions(struct device_node *gic_node)
{
struct device_node *parts_node, *child_part;
int part_idx = 0, i;
@@ -1089,6 +1094,30 @@
}
}
+static void __init gic_of_setup_kvm_info(struct device_node *node)
+{
+ int ret;
+ struct resource r;
+ u32 gicv_idx;
+
+ gic_v3_kvm_info.type = GIC_V3;
+
+ gic_v3_kvm_info.maint_irq = irq_of_parse_and_map(node, 0);
+ if (!gic_v3_kvm_info.maint_irq)
+ return;
+
+ if (of_property_read_u32(node, "#redistributor-regions",
+ &gicv_idx))
+ gicv_idx = 1;
+
+ gicv_idx += 3; /* Also skip GICD, GICC, GICH */
+ ret = of_address_to_resource(node, gicv_idx, &r);
+ if (!ret)
+ gic_v3_kvm_info.vcpu = r;
+
+ gic_set_kvm_info(&gic_v3_kvm_info);
+}
+
static int __init gic_of_init(struct device_node *node, struct device_node *parent)
{
void __iomem *dist_base;
@@ -1144,6 +1173,7 @@
goto out_unmap_rdist;
gic_populate_ppi_partitions(node);
+ gic_of_setup_kvm_info(node);
return 0;
out_unmap_rdist:
@@ -1159,19 +1189,25 @@
IRQCHIP_DECLARE(gic_v3, "arm,gic-v3", gic_of_init);
#ifdef CONFIG_ACPI
-static void __iomem *dist_base;
-static struct redist_region *redist_regs __initdata;
-static u32 nr_redist_regions __initdata;
-static bool single_redist;
+static struct
+{
+ void __iomem *dist_base;
+ struct redist_region *redist_regs;
+ u32 nr_redist_regions;
+ bool single_redist;
+ u32 maint_irq;
+ int maint_irq_mode;
+ phys_addr_t vcpu_base;
+} acpi_data __initdata;
static void __init
gic_acpi_register_redist(phys_addr_t phys_base, void __iomem *redist_base)
{
static int count = 0;
- redist_regs[count].phys_base = phys_base;
- redist_regs[count].redist_base = redist_base;
- redist_regs[count].single_redist = single_redist;
+ acpi_data.redist_regs[count].phys_base = phys_base;
+ acpi_data.redist_regs[count].redist_base = redist_base;
+ acpi_data.redist_regs[count].single_redist = acpi_data.single_redist;
count++;
}
@@ -1199,7 +1235,7 @@
{
struct acpi_madt_generic_interrupt *gicc =
(struct acpi_madt_generic_interrupt *)header;
- u32 reg = readl_relaxed(dist_base + GICD_PIDR2) & GIC_PIDR2_ARCH_MASK;
+ u32 reg = readl_relaxed(acpi_data.dist_base + GICD_PIDR2) & GIC_PIDR2_ARCH_MASK;
u32 size = reg == GIC_PIDR2_ARCH_GICv4 ? SZ_64K * 4 : SZ_64K * 2;
void __iomem *redist_base;
@@ -1216,7 +1252,7 @@
acpi_tbl_entry_handler redist_parser;
enum acpi_madt_type type;
- if (single_redist) {
+ if (acpi_data.single_redist) {
type = ACPI_MADT_TYPE_GENERIC_INTERRUPT;
redist_parser = gic_acpi_parse_madt_gicc;
} else {
@@ -1267,14 +1303,14 @@
count = acpi_table_parse_madt(ACPI_MADT_TYPE_GENERIC_REDISTRIBUTOR,
gic_acpi_match_gicr, 0);
if (count > 0) {
- single_redist = false;
+ acpi_data.single_redist = false;
return count;
}
count = acpi_table_parse_madt(ACPI_MADT_TYPE_GENERIC_INTERRUPT,
gic_acpi_match_gicc, 0);
if (count > 0)
- single_redist = true;
+ acpi_data.single_redist = true;
return count;
}
@@ -1294,36 +1330,117 @@
if (count <= 0)
return false;
- nr_redist_regions = count;
+ acpi_data.nr_redist_regions = count;
return true;
}
+static int __init gic_acpi_parse_virt_madt_gicc(struct acpi_subtable_header *header,
+ const unsigned long end)
+{
+ struct acpi_madt_generic_interrupt *gicc =
+ (struct acpi_madt_generic_interrupt *)header;
+ int maint_irq_mode;
+ static int first_madt = true;
+
+ /* Skip unusable CPUs */
+ if (!(gicc->flags & ACPI_MADT_ENABLED))
+ return 0;
+
+ maint_irq_mode = (gicc->flags & ACPI_MADT_VGIC_IRQ_MODE) ?
+ ACPI_EDGE_SENSITIVE : ACPI_LEVEL_SENSITIVE;
+
+ if (first_madt) {
+ first_madt = false;
+
+ acpi_data.maint_irq = gicc->vgic_interrupt;
+ acpi_data.maint_irq_mode = maint_irq_mode;
+ acpi_data.vcpu_base = gicc->gicv_base_address;
+
+ return 0;
+ }
+
+ /*
+ * The maintenance interrupt and GICV should be the same for every CPU
+ */
+ if ((acpi_data.maint_irq != gicc->vgic_interrupt) ||
+ (acpi_data.maint_irq_mode != maint_irq_mode) ||
+ (acpi_data.vcpu_base != gicc->gicv_base_address))
+ return -EINVAL;
+
+ return 0;
+}
+
+static bool __init gic_acpi_collect_virt_info(void)
+{
+ int count;
+
+ count = acpi_table_parse_madt(ACPI_MADT_TYPE_GENERIC_INTERRUPT,
+ gic_acpi_parse_virt_madt_gicc, 0);
+
+ return (count > 0);
+}
+
#define ACPI_GICV3_DIST_MEM_SIZE (SZ_64K)
+#define ACPI_GICV2_VCTRL_MEM_SIZE (SZ_4K)
+#define ACPI_GICV2_VCPU_MEM_SIZE (SZ_8K)
+
+static void __init gic_acpi_setup_kvm_info(void)
+{
+ int irq;
+
+ if (!gic_acpi_collect_virt_info()) {
+ pr_warn("Unable to get hardware information used for virtualization\n");
+ return;
+ }
+
+ gic_v3_kvm_info.type = GIC_V3;
+
+ irq = acpi_register_gsi(NULL, acpi_data.maint_irq,
+ acpi_data.maint_irq_mode,
+ ACPI_ACTIVE_HIGH);
+ if (irq <= 0)
+ return;
+
+ gic_v3_kvm_info.maint_irq = irq;
+
+ if (acpi_data.vcpu_base) {
+ struct resource *vcpu = &gic_v3_kvm_info.vcpu;
+
+ vcpu->flags = IORESOURCE_MEM;
+ vcpu->start = acpi_data.vcpu_base;
+ vcpu->end = vcpu->start + ACPI_GICV2_VCPU_MEM_SIZE - 1;
+ }
+
+ gic_set_kvm_info(&gic_v3_kvm_info);
+}
static int __init
gic_acpi_init(struct acpi_subtable_header *header, const unsigned long end)
{
struct acpi_madt_generic_distributor *dist;
struct fwnode_handle *domain_handle;
+ size_t size;
int i, err;
/* Get distributor base address */
dist = (struct acpi_madt_generic_distributor *)header;
- dist_base = ioremap(dist->base_address, ACPI_GICV3_DIST_MEM_SIZE);
- if (!dist_base) {
+ acpi_data.dist_base = ioremap(dist->base_address,
+ ACPI_GICV3_DIST_MEM_SIZE);
+ if (!acpi_data.dist_base) {
pr_err("Unable to map GICD registers\n");
return -ENOMEM;
}
- err = gic_validate_dist_version(dist_base);
+ err = gic_validate_dist_version(acpi_data.dist_base);
if (err) {
- pr_err("No distributor detected at @%p, giving up", dist_base);
+ pr_err("No distributor detected at @%p, giving up",
+ acpi_data.dist_base);
goto out_dist_unmap;
}
- redist_regs = kzalloc(sizeof(*redist_regs) * nr_redist_regions,
- GFP_KERNEL);
- if (!redist_regs) {
+ size = sizeof(*acpi_data.redist_regs) * acpi_data.nr_redist_regions;
+ acpi_data.redist_regs = kzalloc(size, GFP_KERNEL);
+ if (!acpi_data.redist_regs) {
err = -ENOMEM;
goto out_dist_unmap;
}
@@ -1332,29 +1449,31 @@
if (err)
goto out_redist_unmap;
- domain_handle = irq_domain_alloc_fwnode(dist_base);
+ domain_handle = irq_domain_alloc_fwnode(acpi_data.dist_base);
if (!domain_handle) {
err = -ENOMEM;
goto out_redist_unmap;
}
- err = gic_init_bases(dist_base, redist_regs, nr_redist_regions, 0,
- domain_handle);
+ err = gic_init_bases(acpi_data.dist_base, acpi_data.redist_regs,
+ acpi_data.nr_redist_regions, 0, domain_handle);
if (err)
goto out_fwhandle_free;
acpi_set_irq_model(ACPI_IRQ_MODEL_GIC, domain_handle);
+ gic_acpi_setup_kvm_info();
+
return 0;
out_fwhandle_free:
irq_domain_free_fwnode(domain_handle);
out_redist_unmap:
- for (i = 0; i < nr_redist_regions; i++)
- if (redist_regs[i].redist_base)
- iounmap(redist_regs[i].redist_base);
- kfree(redist_regs);
+ for (i = 0; i < acpi_data.nr_redist_regions; i++)
+ if (acpi_data.redist_regs[i].redist_base)
+ iounmap(acpi_data.redist_regs[i].redist_base);
+ kfree(acpi_data.redist_regs);
out_dist_unmap:
- iounmap(dist_base);
+ iounmap(acpi_data.dist_base);
return err;
}
IRQCHIP_ACPI_DECLARE(gic_v3, ACPI_MADT_TYPE_GENERIC_DISTRIBUTOR,
diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
index 1de20e1..b4e6471 100644
--- a/drivers/irqchip/irq-gic.c
+++ b/drivers/irqchip/irq-gic.c
@@ -105,6 +105,8 @@
static struct gic_chip_data gic_data[CONFIG_ARM_GIC_MAX_NR] __read_mostly;
+static struct gic_kvm_info gic_v2_kvm_info;
+
#ifdef CONFIG_GIC_NON_BANKED
static void __iomem *gic_get_percpu_base(union gic_base *base)
{
@@ -1248,7 +1250,7 @@
return true;
}
-static int gic_of_setup(struct gic_chip_data *gic, struct device_node *node)
+static int __init gic_of_setup(struct gic_chip_data *gic, struct device_node *node)
{
if (!gic || !node)
return -EINVAL;
@@ -1272,6 +1274,29 @@
return -ENOMEM;
}
+static void __init gic_of_setup_kvm_info(struct device_node *node)
+{
+ int ret;
+ struct resource *vctrl_res = &gic_v2_kvm_info.vctrl;
+ struct resource *vcpu_res = &gic_v2_kvm_info.vcpu;
+
+ gic_v2_kvm_info.type = GIC_V2;
+
+ gic_v2_kvm_info.maint_irq = irq_of_parse_and_map(node, 0);
+ if (!gic_v2_kvm_info.maint_irq)
+ return;
+
+ ret = of_address_to_resource(node, 2, vctrl_res);
+ if (ret)
+ return;
+
+ ret = of_address_to_resource(node, 3, vcpu_res);
+ if (ret)
+ return;
+
+ gic_set_kvm_info(&gic_v2_kvm_info);
+}
+
int __init
gic_of_init(struct device_node *node, struct device_node *parent)
{
@@ -1303,8 +1328,10 @@
return ret;
}
- if (!gic_cnt)
+ if (!gic_cnt) {
gic_init_physaddr(node);
+ gic_of_setup_kvm_info(node);
+ }
if (parent) {
irq = irq_of_parse_and_map(node, 0);
@@ -1330,7 +1357,14 @@
#endif
#ifdef CONFIG_ACPI
-static phys_addr_t cpu_phy_base __initdata;
+static struct
+{
+ phys_addr_t cpu_phys_base;
+ u32 maint_irq;
+ int maint_irq_mode;
+ phys_addr_t vctrl_base;
+ phys_addr_t vcpu_base;
+} acpi_data __initdata;
static int __init
gic_acpi_parse_madt_cpu(struct acpi_subtable_header *header,
@@ -1350,10 +1384,16 @@
* All CPU interface addresses have to be the same.
*/
gic_cpu_base = processor->base_address;
- if (cpu_base_assigned && gic_cpu_base != cpu_phy_base)
+ if (cpu_base_assigned && gic_cpu_base != acpi_data.cpu_phys_base)
return -EINVAL;
- cpu_phy_base = gic_cpu_base;
+ acpi_data.cpu_phys_base = gic_cpu_base;
+ acpi_data.maint_irq = processor->vgic_interrupt;
+ acpi_data.maint_irq_mode = (processor->flags & ACPI_MADT_VGIC_IRQ_MODE) ?
+ ACPI_EDGE_SENSITIVE : ACPI_LEVEL_SENSITIVE;
+ acpi_data.vctrl_base = processor->gich_base_address;
+ acpi_data.vcpu_base = processor->gicv_base_address;
+
cpu_base_assigned = 1;
return 0;
}
@@ -1384,6 +1424,41 @@
#define ACPI_GICV2_DIST_MEM_SIZE (SZ_4K)
#define ACPI_GIC_CPU_IF_MEM_SIZE (SZ_8K)
+#define ACPI_GICV2_VCTRL_MEM_SIZE (SZ_4K)
+#define ACPI_GICV2_VCPU_MEM_SIZE (SZ_8K)
+
+static void __init gic_acpi_setup_kvm_info(void)
+{
+ int irq;
+ struct resource *vctrl_res = &gic_v2_kvm_info.vctrl;
+ struct resource *vcpu_res = &gic_v2_kvm_info.vcpu;
+
+ gic_v2_kvm_info.type = GIC_V2;
+
+ if (!acpi_data.vctrl_base)
+ return;
+
+ vctrl_res->flags = IORESOURCE_MEM;
+ vctrl_res->start = acpi_data.vctrl_base;
+ vctrl_res->end = vctrl_res->start + ACPI_GICV2_VCTRL_MEM_SIZE - 1;
+
+ if (!acpi_data.vcpu_base)
+ return;
+
+ vcpu_res->flags = IORESOURCE_MEM;
+ vcpu_res->start = acpi_data.vcpu_base;
+ vcpu_res->end = vcpu_res->start + ACPI_GICV2_VCPU_MEM_SIZE - 1;
+
+ irq = acpi_register_gsi(NULL, acpi_data.maint_irq,
+ acpi_data.maint_irq_mode,
+ ACPI_ACTIVE_HIGH);
+ if (irq <= 0)
+ return;
+
+ gic_v2_kvm_info.maint_irq = irq;
+
+ gic_set_kvm_info(&gic_v2_kvm_info);
+}
static int __init gic_v2_acpi_init(struct acpi_subtable_header *header,
const unsigned long end)
@@ -1401,7 +1476,7 @@
return -EINVAL;
}
- gic->raw_cpu_base = ioremap(cpu_phy_base, ACPI_GIC_CPU_IF_MEM_SIZE);
+ gic->raw_cpu_base = ioremap(acpi_data.cpu_phys_base, ACPI_GIC_CPU_IF_MEM_SIZE);
if (!gic->raw_cpu_base) {
pr_err("Unable to map GICC registers\n");
return -ENOMEM;
@@ -1447,6 +1522,8 @@
if (IS_ENABLED(CONFIG_ARM_GIC_V2M))
gicv2m_init(NULL, gic_data[0].domain);
+ gic_acpi_setup_kvm_info();
+
return 0;
}
IRQCHIP_ACPI_DECLARE(gic_v2, ACPI_MADT_TYPE_GENERIC_DISTRIBUTOR,
diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c
index 4dffccf..c089f49 100644
--- a/drivers/irqchip/irq-mips-gic.c
+++ b/drivers/irqchip/irq-mips-gic.c
@@ -197,7 +197,7 @@
local_irq_save(flags);
- gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR), cpu);
+ gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR), mips_cm_vp_id(cpu));
if (mips_cm_is64) {
gic_write(GIC_REG(VPE_OTHER, GIC_VPE_COMPARE), cnt);
@@ -246,6 +246,14 @@
#endif
+unsigned gic_read_local_vp_id(void)
+{
+ unsigned long ident;
+
+ ident = gic_read(GIC_REG(VPE_LOCAL, GIC_VP_IDENT));
+ return ident & GIC_VP_IDENT_VCNUM_MSK;
+}
+
static bool gic_local_irq_is_routable(int intr)
{
u32 vpe_ctl;
@@ -553,7 +561,8 @@
spin_lock_irqsave(&gic_lock, flags);
for (i = 0; i < gic_vpes; i++) {
- gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR), i);
+ gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR),
+ mips_cm_vp_id(i));
gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_RMASK), 1 << intr);
}
spin_unlock_irqrestore(&gic_lock, flags);
@@ -567,7 +576,8 @@
spin_lock_irqsave(&gic_lock, flags);
for (i = 0; i < gic_vpes; i++) {
- gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR), i);
+ gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR),
+ mips_cm_vp_id(i));
gic_write32(GIC_REG(VPE_OTHER, GIC_VPE_SMASK), 1 << intr);
}
spin_unlock_irqrestore(&gic_lock, flags);
@@ -607,7 +617,8 @@
for (i = 0; i < gic_vpes; i++) {
unsigned int j;
- gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR), i);
+ gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR),
+ mips_cm_vp_id(i));
for (j = 0; j < GIC_NUM_LOCAL_INTRS; j++) {
if (!gic_local_irq_is_routable(j))
continue;
@@ -652,7 +663,8 @@
for (i = 0; i < gic_vpes; i++) {
u32 val = GIC_MAP_TO_PIN_MSK | gic_cpu_pin;
- gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR), i);
+ gic_write(GIC_REG(VPE_LOCAL, GIC_VPE_OTHER_ADDR),
+ mips_cm_vp_id(i));
switch (intr) {
case GIC_LOCAL_INT_WD:
diff --git a/drivers/isdn/i4l/isdn_tty.c b/drivers/isdn/i4l/isdn_tty.c
index 947d5c9..63eaa0a 100644
--- a/drivers/isdn/i4l/isdn_tty.c
+++ b/drivers/isdn/i4l/isdn_tty.c
@@ -1043,17 +1043,13 @@
if (!(cflag & PARODD))
cval |= UART_LCR_EPAR;
- if (cflag & CLOCAL)
- port->flags &= ~ASYNC_CHECK_CD;
- else {
- port->flags |= ASYNC_CHECK_CD;
- }
+ tty_port_set_check_carrier(port, ~cflag & CLOCAL);
}
static int
isdn_tty_startup(modem_info *info)
{
- if (info->port.flags & ASYNC_INITIALIZED)
+ if (tty_port_initialized(&info->port))
return 0;
isdn_lock_drivers();
#ifdef ISDN_DEBUG_MODEM_OPEN
@@ -1070,7 +1066,7 @@
*/
isdn_tty_change_speed(info);
- info->port.flags |= ASYNC_INITIALIZED;
+ tty_port_set_initialized(&info->port, 1);
info->msr |= (UART_MSR_DSR | UART_MSR_CTS);
info->send_outstanding = 0;
return 0;
@@ -1083,7 +1079,7 @@
static void
isdn_tty_shutdown(modem_info *info)
{
- if (!(info->port.flags & ASYNC_INITIALIZED))
+ if (!tty_port_initialized(&info->port))
return;
#ifdef ISDN_DEBUG_MODEM_OPEN
printk(KERN_DEBUG "Shutting down isdnmodem port %d ....\n", info->line);
@@ -1103,7 +1099,7 @@
if (info->port.tty)
set_bit(TTY_IO_ERROR, &info->port.tty->flags);
- info->port.flags &= ~ASYNC_INITIALIZED;
+ tty_port_set_initialized(&info->port, 0);
}
/* isdn_tty_write() is the main send-routine. It is called from the upper
@@ -1351,7 +1347,7 @@
if (isdn_tty_paranoia_check(info, tty->name, __func__))
return -ENODEV;
- if (tty->flags & (1 << TTY_IO_ERROR))
+ if (tty_io_error(tty))
return -EIO;
mutex_lock(&modem_info_mutex);
@@ -1378,7 +1374,7 @@
if (isdn_tty_paranoia_check(info, tty->name, __func__))
return -ENODEV;
- if (tty->flags & (1 << TTY_IO_ERROR))
+ if (tty_io_error(tty))
return -EIO;
#ifdef ISDN_DEBUG_MODEM_IOCTL
@@ -1419,7 +1415,7 @@
if (isdn_tty_paranoia_check(info, tty->name, "isdn_tty_ioctl"))
return -ENODEV;
- if (tty->flags & (1 << TTY_IO_ERROR))
+ if (tty_io_error(tty))
return -EIO;
switch (cmd) {
case TCSBRK: /* SVID version: non-zero arg --> no break */
@@ -1581,7 +1577,7 @@
* interrupt driver to stop checking the data ready bit in the
* line status register.
*/
- if (port->flags & ASYNC_INITIALIZED) {
+ if (tty_port_initialized(port)) {
tty_wait_until_sent(tty, 3000); /* 30 seconds timeout */
/*
* Before we drop DTR, make sure the UART transmitter
@@ -1622,7 +1618,7 @@
return;
isdn_tty_shutdown(info);
port->count = 0;
- port->flags &= ~ASYNC_NORMAL_ACTIVE;
+ tty_port_set_active(port, 0);
port->tty = NULL;
wake_up_interruptible(&port->open_wait);
}
@@ -1979,7 +1975,7 @@
#endif
if (
#ifndef FIX_FILE_TRANSFER
- (info->port.flags & ASYNC_NORMAL_ACTIVE) &&
+ tty_port_active(&info->port) &&
#endif
(info->isdn_driver == -1) &&
(info->isdn_channel == -1) &&
@@ -2018,8 +2014,6 @@
return (wret == 2) ? 3 : 0;
}
-#define TTY_IS_ACTIVE(info) (info->port.flags & ASYNC_NORMAL_ACTIVE)
-
int
isdn_tty_stat_callback(int i, isdn_ctrl *c)
{
@@ -2077,7 +2071,7 @@
#ifdef ISDN_TTY_STAT_DEBUG
printk(KERN_DEBUG "tty_STAT_DCONN ttyI%d\n", info->line);
#endif
- if (TTY_IS_ACTIVE(info)) {
+ if (tty_port_active(&info->port)) {
if (info->dialing == 1) {
info->dialing = 2;
return 1;
@@ -2088,7 +2082,7 @@
#ifdef ISDN_TTY_STAT_DEBUG
printk(KERN_DEBUG "tty_STAT_DHUP ttyI%d\n", info->line);
#endif
- if (TTY_IS_ACTIVE(info)) {
+ if (tty_port_active(&info->port)) {
if (info->dialing == 1)
isdn_tty_modem_result(RESULT_BUSY, info);
if (info->dialing > 1)
@@ -2118,7 +2112,7 @@
* waiting for it and
* set DCD-bit of its modem-status.
*/
- if (TTY_IS_ACTIVE(info) ||
+ if (tty_port_active(&info->port) ||
(info->port.blocked_open &&
(info->emu.mdmreg[REG_DCD] & BIT_DCD))) {
info->msr |= UART_MSR_DCD;
@@ -2145,7 +2139,7 @@
#ifdef ISDN_TTY_STAT_DEBUG
printk(KERN_DEBUG "tty_STAT_BHUP ttyI%d\n", info->line);
#endif
- if (TTY_IS_ACTIVE(info)) {
+ if (tty_port_active(&info->port)) {
#ifdef ISDN_DEBUG_MODEM_HUP
printk(KERN_DEBUG "Mhup in ISDN_STAT_BHUP\n");
#endif
@@ -2157,7 +2151,7 @@
#ifdef ISDN_TTY_STAT_DEBUG
printk(KERN_DEBUG "tty_STAT_NODCH ttyI%d\n", info->line);
#endif
- if (TTY_IS_ACTIVE(info)) {
+ if (tty_port_active(&info->port)) {
if (info->dialing) {
info->dialing = 0;
info->last_l2 = -1;
@@ -2183,14 +2177,14 @@
return 1;
#ifdef CONFIG_ISDN_TTY_FAX
case ISDN_STAT_FAXIND:
- if (TTY_IS_ACTIVE(info)) {
+ if (tty_port_active(&info->port)) {
isdn_tty_fax_command(info, c);
}
break;
#endif
#ifdef CONFIG_ISDN_AUDIO
case ISDN_STAT_AUDIO:
- if (TTY_IS_ACTIVE(info)) {
+ if (tty_port_active(&info->port)) {
switch (c->parm.num[0]) {
case ISDN_AUDIO_DTMF:
if (info->vonline) {
@@ -2528,7 +2522,7 @@
if (info->closing || (!info->port.tty))
return;
- if (info->port.flags & ASYNC_CHECK_CD)
+ if (tty_port_check_carrier(&info->port))
tty_hangup(info->port.tty);
}
}
diff --git a/drivers/macintosh/rack-meter.c b/drivers/macintosh/rack-meter.c
index caaec65..465c522 100644
--- a/drivers/macintosh/rack-meter.c
+++ b/drivers/macintosh/rack-meter.c
@@ -154,8 +154,8 @@
DBDMA_DO_STOP(rm->dma_regs);
return;
}
- memset(rdma->buf1, 0, SAMPLE_COUNT & sizeof(u32));
- memset(rdma->buf2, 0, SAMPLE_COUNT & sizeof(u32));
+ memset(rdma->buf1, 0, ARRAY_SIZE(rdma->buf1));
+ memset(rdma->buf2, 0, ARRAY_SIZE(rdma->buf2));
rm->dma_buf_v->mark = 0;
@@ -227,6 +227,7 @@
total_idle_ticks = get_cpu_idle_time(cpu);
idle_ticks = (unsigned int) (total_idle_ticks - rcpu->prev_idle);
+ idle_ticks = min(idle_ticks, total_ticks);
rcpu->prev_idle = total_idle_ticks;
/* We do a very dumb calculation to update the LEDs for now,
diff --git a/drivers/macintosh/via-pmu.c b/drivers/macintosh/via-pmu.c
index 01ee736..f8b6d14 100644
--- a/drivers/macintosh/via-pmu.c
+++ b/drivers/macintosh/via-pmu.c
@@ -1851,7 +1851,7 @@
_set_L2CR(save_l2cr);
/* Restore userland MMU context */
- switch_mmu_context(NULL, current->active_mm);
+ switch_mmu_context(NULL, current->active_mm, NULL);
/* Power things up */
pmu_unlock();
@@ -1940,7 +1940,7 @@
_set_L3CR(save_l3cr);
/* Restore userland MMU context */
- switch_mmu_context(NULL, current->active_mm);
+ switch_mmu_context(NULL, current->active_mm, NULL);
/* Tell PMU we are ready */
pmu_unlock();
diff --git a/drivers/mailbox/mailbox-sti.c b/drivers/mailbox/mailbox-sti.c
index 2394cfe..a334db5 100644
--- a/drivers/mailbox/mailbox-sti.c
+++ b/drivers/mailbox/mailbox-sti.c
@@ -430,8 +430,8 @@
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
mdev->base = devm_ioremap_resource(&pdev->dev, res);
- if (!mdev->base)
- return -ENOMEM;
+ if (IS_ERR(mdev->base))
+ return PTR_ERR(mdev->base);
ret = of_property_read_string(np, "mbox-name", &mdev->name);
if (ret)
diff --git a/drivers/mailbox/omap-mailbox.c b/drivers/mailbox/omap-mailbox.c
index b7f636f..c5e8b9c 100644
--- a/drivers/mailbox/omap-mailbox.c
+++ b/drivers/mailbox/omap-mailbox.c
@@ -2,7 +2,7 @@
* OMAP mailbox driver
*
* Copyright (C) 2006-2009 Nokia Corporation. All rights reserved.
- * Copyright (C) 2013-2014 Texas Instruments Inc.
+ * Copyright (C) 2013-2016 Texas Instruments Incorporated - http://www.ti.com
*
* Contact: Hiroshi DOYU <Hiroshi.DOYU@nokia.com>
* Suman Anna <s-anna@ti.com>
@@ -15,12 +15,6 @@
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
- * 02110-1301 USA
- *
*/
#include <linux/interrupt.h>
@@ -33,7 +27,6 @@
#include <linux/of_device.h>
#include <linux/platform_device.h>
#include <linux/pm_runtime.h>
-#include <linux/platform_data/mailbox-omap.h>
#include <linux/omap-mailbox.h>
#include <linux/mailbox_controller.h>
#include <linux/mailbox_client.h>
@@ -62,12 +55,9 @@
#define MAILBOX_IRQ_NEWMSG(m) (1 << (2 * (m)))
#define MAILBOX_IRQ_NOTFULL(m) (1 << (2 * (m) + 1))
-#define MBOX_REG_SIZE 0x120
-
-#define OMAP4_MBOX_REG_SIZE 0x130
-
-#define MBOX_NR_REGS (MBOX_REG_SIZE / sizeof(u32))
-#define OMAP4_MBOX_NR_REGS (OMAP4_MBOX_REG_SIZE / sizeof(u32))
+/* Interrupt register configuration types */
+#define MBOX_INTR_CFG_TYPE1 0
+#define MBOX_INTR_CFG_TYPE2 1
struct omap_mbox_fifo {
unsigned long msg;
@@ -91,8 +81,10 @@
struct device *dev;
struct mutex cfg_lock;
void __iomem *mbox_base;
+ u32 *irq_ctx;
u32 num_users;
u32 num_fifos;
+ u32 intr_type;
struct omap_mbox **mboxes;
struct mbox_controller controller;
struct list_head elem;
@@ -119,7 +111,6 @@
struct omap_mbox_device *parent;
struct omap_mbox_fifo tx_fifo;
struct omap_mbox_fifo rx_fifo;
- u32 ctx[OMAP4_MBOX_NR_REGS];
u32 intr_type;
struct mbox_chan *chan;
bool send_no_irq;
@@ -157,24 +148,28 @@
static mbox_msg_t mbox_fifo_read(struct omap_mbox *mbox)
{
struct omap_mbox_fifo *fifo = &mbox->rx_fifo;
- return (mbox_msg_t) mbox_read_reg(mbox->parent, fifo->msg);
+
+ return (mbox_msg_t)mbox_read_reg(mbox->parent, fifo->msg);
}
static void mbox_fifo_write(struct omap_mbox *mbox, mbox_msg_t msg)
{
struct omap_mbox_fifo *fifo = &mbox->tx_fifo;
+
mbox_write_reg(mbox->parent, msg, fifo->msg);
}
static int mbox_fifo_empty(struct omap_mbox *mbox)
{
struct omap_mbox_fifo *fifo = &mbox->rx_fifo;
+
return (mbox_read_reg(mbox->parent, fifo->msg_stat) == 0);
}
static int mbox_fifo_full(struct omap_mbox *mbox)
{
struct omap_mbox_fifo *fifo = &mbox->tx_fifo;
+
return mbox_read_reg(mbox->parent, fifo->fifo_stat);
}
@@ -206,49 +201,6 @@
return (int)(enable & status & bit);
}
-void omap_mbox_save_ctx(struct mbox_chan *chan)
-{
- int i;
- int nr_regs;
- struct omap_mbox *mbox = mbox_chan_to_omap_mbox(chan);
-
- if (WARN_ON(!mbox))
- return;
-
- if (mbox->intr_type)
- nr_regs = OMAP4_MBOX_NR_REGS;
- else
- nr_regs = MBOX_NR_REGS;
- for (i = 0; i < nr_regs; i++) {
- mbox->ctx[i] = mbox_read_reg(mbox->parent, i * sizeof(u32));
-
- dev_dbg(mbox->dev, "%s: [%02x] %08x\n", __func__,
- i, mbox->ctx[i]);
- }
-}
-EXPORT_SYMBOL(omap_mbox_save_ctx);
-
-void omap_mbox_restore_ctx(struct mbox_chan *chan)
-{
- int i;
- int nr_regs;
- struct omap_mbox *mbox = mbox_chan_to_omap_mbox(chan);
-
- if (WARN_ON(!mbox))
- return;
-
- if (mbox->intr_type)
- nr_regs = OMAP4_MBOX_NR_REGS;
- else
- nr_regs = MBOX_NR_REGS;
- for (i = 0; i < nr_regs; i++) {
- mbox_write_reg(mbox->parent, mbox->ctx[i], i * sizeof(u32));
- dev_dbg(mbox->dev, "%s: [%02x] %08x\n", __func__,
- i, mbox->ctx[i]);
- }
-}
-EXPORT_SYMBOL(omap_mbox_restore_ctx);
-
static void _omap_mbox_enable_irq(struct omap_mbox *mbox, omap_mbox_irq_t irq)
{
u32 l;
@@ -381,7 +333,7 @@
if (!work)
return NULL;
- mq = kzalloc(sizeof(struct omap_mbox_queue), GFP_KERNEL);
+ mq = kzalloc(sizeof(*mq), GFP_KERNEL);
if (!mq)
return NULL;
@@ -525,6 +477,7 @@
mboxes = mdev->mboxes;
for (i = 0; mboxes[i]; i++) {
struct omap_mbox *mbox = mboxes[i];
+
mbox->dev = device_create(&omap_mbox_class, mdev->dev,
0, mbox, "%s", mbox->name);
if (IS_ERR(mbox->dev)) {
@@ -647,6 +600,52 @@
.shutdown = omap_mbox_chan_shutdown,
};
+#ifdef CONFIG_PM_SLEEP
+static int omap_mbox_suspend(struct device *dev)
+{
+ struct omap_mbox_device *mdev = dev_get_drvdata(dev);
+ u32 usr, fifo, reg;
+
+ if (pm_runtime_status_suspended(dev))
+ return 0;
+
+ for (fifo = 0; fifo < mdev->num_fifos; fifo++) {
+ if (mbox_read_reg(mdev, MAILBOX_MSGSTATUS(fifo))) {
+ dev_err(mdev->dev, "fifo %d has unexpected unread messages\n",
+ fifo);
+ return -EBUSY;
+ }
+ }
+
+ for (usr = 0; usr < mdev->num_users; usr++) {
+ reg = MAILBOX_IRQENABLE(mdev->intr_type, usr);
+ mdev->irq_ctx[usr] = mbox_read_reg(mdev, reg);
+ }
+
+ return 0;
+}
+
+static int omap_mbox_resume(struct device *dev)
+{
+ struct omap_mbox_device *mdev = dev_get_drvdata(dev);
+ u32 usr, reg;
+
+ if (pm_runtime_status_suspended(dev))
+ return 0;
+
+ for (usr = 0; usr < mdev->num_users; usr++) {
+ reg = MAILBOX_IRQENABLE(mdev->intr_type, usr);
+ mbox_write_reg(mdev, mdev->irq_ctx[usr], reg);
+ }
+
+ return 0;
+}
+#endif
+
+static const struct dev_pm_ops omap_mbox_pm_ops = {
+ SET_SYSTEM_SLEEP_PM_OPS(omap_mbox_suspend, omap_mbox_resume)
+};
+
static const struct of_device_id omap_mailbox_of_match[] = {
{
.compatible = "ti,omap2-mailbox",
@@ -696,8 +695,6 @@
int ret;
struct mbox_chan *chnls;
struct omap_mbox **list, *mbox, *mboxblk;
- struct omap_mbox_pdata *pdata = pdev->dev.platform_data;
- struct omap_mbox_dev_info *info = NULL;
struct omap_mbox_fifo_info *finfo, *finfoblk;
struct omap_mbox_device *mdev;
struct omap_mbox_fifo *fifo;
@@ -710,36 +707,26 @@
u32 l;
int i;
- if (!node && (!pdata || !pdata->info_cnt || !pdata->info)) {
- pr_err("%s: platform not supported\n", __func__);
+ if (!node) {
+ pr_err("%s: only DT-based devices are supported\n", __func__);
return -ENODEV;
}
- if (node) {
- match = of_match_device(omap_mailbox_of_match, &pdev->dev);
- if (!match)
- return -ENODEV;
- intr_type = (u32)match->data;
+ match = of_match_device(omap_mailbox_of_match, &pdev->dev);
+ if (!match)
+ return -ENODEV;
+ intr_type = (u32)match->data;
- if (of_property_read_u32(node, "ti,mbox-num-users",
- &num_users))
- return -ENODEV;
+ if (of_property_read_u32(node, "ti,mbox-num-users", &num_users))
+ return -ENODEV;
- if (of_property_read_u32(node, "ti,mbox-num-fifos",
- &num_fifos))
- return -ENODEV;
+ if (of_property_read_u32(node, "ti,mbox-num-fifos", &num_fifos))
+ return -ENODEV;
- info_count = of_get_available_child_count(node);
- if (!info_count) {
- dev_err(&pdev->dev, "no available mbox devices found\n");
- return -ENODEV;
- }
- } else { /* non-DT device creation */
- info_count = pdata->info_cnt;
- info = pdata->info;
- intr_type = pdata->intr_type;
- num_users = pdata->num_users;
- num_fifos = pdata->num_fifos;
+ info_count = of_get_available_child_count(node);
+ if (!info_count) {
+ dev_err(&pdev->dev, "no available mbox devices found\n");
+ return -ENODEV;
}
finfoblk = devm_kzalloc(&pdev->dev, info_count * sizeof(*finfoblk),
@@ -750,38 +737,28 @@
finfo = finfoblk;
child = NULL;
for (i = 0; i < info_count; i++, finfo++) {
- if (node) {
- child = of_get_next_available_child(node, child);
- ret = of_property_read_u32_array(child, "ti,mbox-tx",
- tmp, ARRAY_SIZE(tmp));
- if (ret)
- return ret;
- finfo->tx_id = tmp[0];
- finfo->tx_irq = tmp[1];
- finfo->tx_usr = tmp[2];
+ child = of_get_next_available_child(node, child);
+ ret = of_property_read_u32_array(child, "ti,mbox-tx", tmp,
+ ARRAY_SIZE(tmp));
+ if (ret)
+ return ret;
+ finfo->tx_id = tmp[0];
+ finfo->tx_irq = tmp[1];
+ finfo->tx_usr = tmp[2];
- ret = of_property_read_u32_array(child, "ti,mbox-rx",
- tmp, ARRAY_SIZE(tmp));
- if (ret)
- return ret;
- finfo->rx_id = tmp[0];
- finfo->rx_irq = tmp[1];
- finfo->rx_usr = tmp[2];
+ ret = of_property_read_u32_array(child, "ti,mbox-rx", tmp,
+ ARRAY_SIZE(tmp));
+ if (ret)
+ return ret;
+ finfo->rx_id = tmp[0];
+ finfo->rx_irq = tmp[1];
+ finfo->rx_usr = tmp[2];
- finfo->name = child->name;
+ finfo->name = child->name;
- if (of_find_property(child, "ti,mbox-send-noirq", NULL))
- finfo->send_no_irq = true;
- } else {
- finfo->tx_id = info->tx_id;
- finfo->rx_id = info->rx_id;
- finfo->tx_usr = info->usr_id;
- finfo->tx_irq = info->irq_id;
- finfo->rx_usr = info->usr_id;
- finfo->rx_irq = info->irq_id;
- finfo->name = info->name;
- info++;
- }
+ if (of_find_property(child, "ti,mbox-send-noirq", NULL))
+ finfo->send_no_irq = true;
+
if (finfo->tx_id >= num_fifos || finfo->rx_id >= num_fifos ||
finfo->tx_usr >= num_users || finfo->rx_usr >= num_users)
return -EINVAL;
@@ -796,6 +773,11 @@
if (IS_ERR(mdev->mbox_base))
return PTR_ERR(mdev->mbox_base);
+ mdev->irq_ctx = devm_kzalloc(&pdev->dev, num_users * sizeof(u32),
+ GFP_KERNEL);
+ if (!mdev->irq_ctx)
+ return -ENOMEM;
+
/* allocate one extra for marking end of list */
list = devm_kzalloc(&pdev->dev, (info_count + 1) * sizeof(*list),
GFP_KERNEL);
@@ -848,6 +830,7 @@
mdev->dev = &pdev->dev;
mdev->num_users = num_users;
mdev->num_fifos = num_fifos;
+ mdev->intr_type = intr_type;
mdev->mboxes = list;
/* OMAP does not have a Tx-Done IRQ, but rather a Tx-Ready IRQ */
@@ -905,6 +888,7 @@
.remove = omap_mbox_remove,
.driver = {
.name = "omap-mailbox",
+ .pm = &omap_mbox_pm_ops,
.of_match_table = of_match_ptr(omap_mailbox_of_match),
},
};
diff --git a/drivers/mcb/mcb-core.c b/drivers/mcb/mcb-core.c
index a4be451..b73c6e7 100644
--- a/drivers/mcb/mcb-core.c
+++ b/drivers/mcb/mcb-core.c
@@ -83,13 +83,67 @@
static void mcb_shutdown(struct device *dev)
{
+ struct mcb_driver *mdrv = to_mcb_driver(dev->driver);
struct mcb_device *mdev = to_mcb_device(dev);
- struct mcb_driver *mdrv = mdev->driver;
if (mdrv && mdrv->shutdown)
mdrv->shutdown(mdev);
}
+static ssize_t revision_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct mcb_bus *bus = to_mcb_bus(dev);
+
+ return scnprintf(buf, PAGE_SIZE, "%d\n", bus->revision);
+}
+static DEVICE_ATTR_RO(revision);
+
+static ssize_t model_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct mcb_bus *bus = to_mcb_bus(dev);
+
+ return scnprintf(buf, PAGE_SIZE, "%c\n", bus->model);
+}
+static DEVICE_ATTR_RO(model);
+
+static ssize_t minor_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct mcb_bus *bus = to_mcb_bus(dev);
+
+ return scnprintf(buf, PAGE_SIZE, "%d\n", bus->minor);
+}
+static DEVICE_ATTR_RO(minor);
+
+static ssize_t name_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct mcb_bus *bus = to_mcb_bus(dev);
+
+ return scnprintf(buf, PAGE_SIZE, "%s\n", bus->name);
+}
+static DEVICE_ATTR_RO(name);
+
+static struct attribute *mcb_bus_attrs[] = {
+ &dev_attr_revision.attr,
+ &dev_attr_model.attr,
+ &dev_attr_minor.attr,
+ &dev_attr_name.attr,
+ NULL,
+};
+
+static const struct attribute_group mcb_carrier_group = {
+ .attrs = mcb_bus_attrs,
+};
+
+static const struct attribute_group *mcb_carrier_groups[] = {
+ &mcb_carrier_group,
+ NULL,
+};
+
+
static struct bus_type mcb_bus_type = {
.name = "mcb",
.match = mcb_match,
@@ -99,6 +153,11 @@
.shutdown = mcb_shutdown,
};
+static struct device_type mcb_carrier_device_type = {
+ .name = "mcb-carrier",
+ .groups = mcb_carrier_groups,
+};
+
/**
* __mcb_register_driver() - Register a @mcb_driver at the system
* @drv: The @mcb_driver
@@ -155,6 +214,7 @@
int device_id;
device_initialize(&dev->dev);
+ mcb_bus_get(bus);
dev->dev.bus = &mcb_bus_type;
dev->dev.parent = bus->dev.parent;
dev->dev.release = mcb_release_dev;
@@ -178,6 +238,15 @@
}
EXPORT_SYMBOL_GPL(mcb_device_register);
+static void mcb_free_bus(struct device *dev)
+{
+ struct mcb_bus *bus = to_mcb_bus(dev);
+
+ put_device(bus->carrier);
+ ida_simple_remove(&mcb_ida, bus->bus_nr);
+ kfree(bus);
+}
+
/**
* mcb_alloc_bus() - Allocate a new @mcb_bus
*
@@ -187,6 +256,7 @@
{
struct mcb_bus *bus;
int bus_nr;
+ int rc;
bus = kzalloc(sizeof(struct mcb_bus), GFP_KERNEL);
if (!bus)
@@ -194,14 +264,29 @@
bus_nr = ida_simple_get(&mcb_ida, 0, 0, GFP_KERNEL);
if (bus_nr < 0) {
- kfree(bus);
- return ERR_PTR(bus_nr);
+ rc = bus_nr;
+ goto err_free;
}
- INIT_LIST_HEAD(&bus->children);
bus->bus_nr = bus_nr;
- bus->carrier = carrier;
+ bus->carrier = get_device(carrier);
+
+ device_initialize(&bus->dev);
+ bus->dev.parent = carrier;
+ bus->dev.bus = &mcb_bus_type;
+ bus->dev.type = &mcb_carrier_device_type;
+ bus->dev.release = &mcb_free_bus;
+
+ dev_set_name(&bus->dev, "mcb:%d", bus_nr);
+ rc = device_add(&bus->dev);
+ if (rc)
+ goto err_free;
+
return bus;
+err_free:
+ put_device(carrier);
+ kfree(bus);
+ return ERR_PTR(rc);
}
EXPORT_SYMBOL_GPL(mcb_alloc_bus);
@@ -224,10 +309,6 @@
void mcb_release_bus(struct mcb_bus *bus)
{
mcb_devices_unregister(bus);
-
- ida_simple_remove(&mcb_ida, bus->bus_nr);
-
- kfree(bus);
}
EXPORT_SYMBOL_GPL(mcb_release_bus);
diff --git a/drivers/mcb/mcb-internal.h b/drivers/mcb/mcb-internal.h
index fb7493d..5254e02 100644
--- a/drivers/mcb/mcb-internal.h
+++ b/drivers/mcb/mcb-internal.h
@@ -5,7 +5,6 @@
#define PCI_VENDOR_ID_MEN 0x1a88
#define PCI_DEVICE_ID_MEN_CHAMELEON 0x4d45
-#define CHAMELEON_FILENAME_LEN 12
#define CHAMELEONV2_MAGIC 0xabce
#define CHAM_HEADER_SIZE 0x200
diff --git a/drivers/mcb/mcb-parse.c b/drivers/mcb/mcb-parse.c
index 0049269..dbecbed 100644
--- a/drivers/mcb/mcb-parse.c
+++ b/drivers/mcb/mcb-parse.c
@@ -57,7 +57,7 @@
mdev->id = GDD_DEV(reg1);
mdev->rev = GDD_REV(reg1);
mdev->var = GDD_VAR(reg1);
- mdev->bar = GDD_BAR(reg1);
+ mdev->bar = GDD_BAR(reg2);
mdev->group = GDD_GRP(reg2);
mdev->inst = GDD_INS(reg2);
@@ -113,16 +113,11 @@
}
p += hsize;
- pr_debug("header->revision = %d\n", header->revision);
- pr_debug("header->model = 0x%x ('%c')\n", header->model,
- header->model);
- pr_debug("header->minor = %d\n", header->minor);
- pr_debug("header->bus_type = 0x%x\n", header->bus_type);
-
-
- pr_debug("header->magic = 0x%x\n", header->magic);
- pr_debug("header->filename = \"%.*s\"\n", CHAMELEON_FILENAME_LEN,
- header->filename);
+ bus->revision = header->revision;
+ bus->model = header->model;
+ bus->minor = header->minor;
+ snprintf(bus->name, CHAMELEON_FILENAME_LEN + 1, "%s",
+ header->filename);
for_each_chameleon_cell(dtype, p) {
switch (dtype) {
diff --git a/drivers/mcb/mcb-pci.c b/drivers/mcb/mcb-pci.c
index 67d5e7d..b15a034 100644
--- a/drivers/mcb/mcb-pci.c
+++ b/drivers/mcb/mcb-pci.c
@@ -35,7 +35,6 @@
struct resource *res;
struct priv *priv;
int ret;
- int num_cells;
unsigned long flags;
priv = devm_kzalloc(&pdev->dev, sizeof(struct priv), GFP_KERNEL);
@@ -55,19 +54,20 @@
goto out_disable;
}
- res = request_mem_region(priv->mapbase, CHAM_HEADER_SIZE,
- KBUILD_MODNAME);
+ res = devm_request_mem_region(&pdev->dev, priv->mapbase,
+ CHAM_HEADER_SIZE,
+ KBUILD_MODNAME);
if (!res) {
dev_err(&pdev->dev, "Failed to request PCI memory\n");
ret = -EBUSY;
goto out_disable;
}
- priv->base = ioremap(priv->mapbase, CHAM_HEADER_SIZE);
+ priv->base = devm_ioremap(&pdev->dev, priv->mapbase, CHAM_HEADER_SIZE);
if (!priv->base) {
dev_err(&pdev->dev, "Cannot ioremap\n");
ret = -ENOMEM;
- goto out_release;
+ goto out_disable;
}
flags = pci_resource_flags(pdev, 0);
@@ -75,7 +75,7 @@
ret = -ENOTSUPP;
dev_err(&pdev->dev,
"IO mapped PCI devices are not supported\n");
- goto out_iounmap;
+ goto out_disable;
}
pci_set_drvdata(pdev, priv);
@@ -83,7 +83,7 @@
priv->bus = mcb_alloc_bus(&pdev->dev);
if (IS_ERR(priv->bus)) {
ret = PTR_ERR(priv->bus);
- goto out_iounmap;
+ goto out_disable;
}
priv->bus->get_irq = mcb_pci_get_irq;
@@ -91,9 +91,8 @@
ret = chameleon_parse_cells(priv->bus, priv->mapbase, priv->base);
if (ret < 0)
goto out_mcb_bus;
- num_cells = ret;
- dev_dbg(&pdev->dev, "Found %d cells\n", num_cells);
+ dev_dbg(&pdev->dev, "Found %d cells\n", ret);
mcb_bus_add_devices(priv->bus);
@@ -101,10 +100,6 @@
out_mcb_bus:
mcb_release_bus(priv->bus);
-out_iounmap:
- iounmap(priv->base);
-out_release:
- pci_release_region(pdev, 0);
out_disable:
pci_disable_device(pdev);
return ret;
@@ -116,8 +111,6 @@
mcb_release_bus(priv->bus);
- iounmap(priv->base);
- release_region(priv->mapbase, CHAM_HEADER_SIZE);
pci_disable_device(pdev);
}
diff --git a/drivers/md/bitmap.c b/drivers/md/bitmap.c
index 3fe86b5..d8129ec 100644
--- a/drivers/md/bitmap.c
+++ b/drivers/md/bitmap.c
@@ -46,7 +46,7 @@
* allocated while we're using it
*/
static int bitmap_checkpage(struct bitmap_counts *bitmap,
- unsigned long page, int create)
+ unsigned long page, int create, int no_hijack)
__releases(bitmap->lock)
__acquires(bitmap->lock)
{
@@ -90,6 +90,9 @@
if (mappage == NULL) {
pr_debug("md/bitmap: map page allocation failed, hijacking\n");
+ /* We don't support hijack for cluster raid */
+ if (no_hijack)
+ return -ENOMEM;
/* failed - set the hijacked flag so that we can use the
* pointer as a counter */
if (!bitmap->bp[page].map)
@@ -756,7 +759,7 @@
bytes += sizeof(bitmap_super_t);
num_pages = DIV_ROUND_UP(bytes, PAGE_SIZE);
- offset = slot_number * (num_pages - 1);
+ offset = slot_number * num_pages;
store->filemap = kmalloc(sizeof(struct page *)
* num_pages, GFP_KERNEL);
@@ -900,6 +903,11 @@
struct page *page;
void *kaddr;
unsigned long chunk = block >> bitmap->counts.chunkshift;
+ struct bitmap_storage *store = &bitmap->storage;
+ unsigned long node_offset = 0;
+
+ if (mddev_is_clustered(bitmap->mddev))
+ node_offset = bitmap->cluster_slot * store->file_pages;
page = filemap_get_page(&bitmap->storage, chunk);
if (!page)
@@ -915,7 +923,7 @@
kunmap_atomic(kaddr);
pr_debug("set file bit %lu page %lu\n", bit, page->index);
/* record page number so it gets flushed to disk when unplug occurs */
- set_page_attr(bitmap, page->index, BITMAP_PAGE_DIRTY);
+ set_page_attr(bitmap, page->index - node_offset, BITMAP_PAGE_DIRTY);
}
static void bitmap_file_clear_bit(struct bitmap *bitmap, sector_t block)
@@ -924,6 +932,11 @@
struct page *page;
void *paddr;
unsigned long chunk = block >> bitmap->counts.chunkshift;
+ struct bitmap_storage *store = &bitmap->storage;
+ unsigned long node_offset = 0;
+
+ if (mddev_is_clustered(bitmap->mddev))
+ node_offset = bitmap->cluster_slot * store->file_pages;
page = filemap_get_page(&bitmap->storage, chunk);
if (!page)
@@ -935,8 +948,8 @@
else
clear_bit_le(bit, paddr);
kunmap_atomic(paddr);
- if (!test_page_attr(bitmap, page->index, BITMAP_PAGE_NEEDWRITE)) {
- set_page_attr(bitmap, page->index, BITMAP_PAGE_PENDING);
+ if (!test_page_attr(bitmap, page->index - node_offset, BITMAP_PAGE_NEEDWRITE)) {
+ set_page_attr(bitmap, page->index - node_offset, BITMAP_PAGE_PENDING);
bitmap->allclean = 0;
}
}
@@ -1321,7 +1334,7 @@
sector_t csize;
int err;
- err = bitmap_checkpage(bitmap, page, create);
+ err = bitmap_checkpage(bitmap, page, create, 0);
if (bitmap->bp[page].hijacked ||
bitmap->bp[page].map == NULL)
@@ -1594,6 +1607,27 @@
}
EXPORT_SYMBOL(bitmap_cond_end_sync);
+void bitmap_sync_with_cluster(struct mddev *mddev,
+ sector_t old_lo, sector_t old_hi,
+ sector_t new_lo, sector_t new_hi)
+{
+ struct bitmap *bitmap = mddev->bitmap;
+ sector_t sector, blocks = 0;
+
+ for (sector = old_lo; sector < new_lo; ) {
+ bitmap_end_sync(bitmap, sector, &blocks, 0);
+ sector += blocks;
+ }
+ WARN((blocks > new_lo) && old_lo, "alignment is not correct for lo\n");
+
+ for (sector = old_hi; sector < new_hi; ) {
+ bitmap_start_sync(bitmap, sector, &blocks, 0);
+ sector += blocks;
+ }
+ WARN((blocks > new_hi) && old_hi, "alignment is not correct for hi\n");
+}
+EXPORT_SYMBOL(bitmap_sync_with_cluster);
+
static void bitmap_set_memory_bits(struct bitmap *bitmap, sector_t offset, int needed)
{
/* For each chunk covered by any of these sectors, set the
@@ -1814,6 +1848,9 @@
if (!bitmap)
goto out;
+ if (mddev_is_clustered(mddev))
+ md_cluster_ops->load_bitmaps(mddev, mddev->bitmap_info.nodes);
+
/* Clear out old bitmap info first: Either there is none, or we
* are resuming after someone else has possibly changed things,
* so we should forget old cached info.
@@ -1890,14 +1927,14 @@
if (clear_bits) {
bitmap_update_sb(bitmap);
- /* Setting this for the ev_page should be enough.
- * And we do not require both write_all and PAGE_DIRT either
- */
+ /* BITMAP_PAGE_PENDING is set, but bitmap_unplug needs
+ * BITMAP_PAGE_DIRTY or _NEEDWRITE to write ... */
for (i = 0; i < bitmap->storage.file_pages; i++)
- set_page_attr(bitmap, i, BITMAP_PAGE_DIRTY);
- bitmap_write_all(bitmap);
+ if (test_page_attr(bitmap, i, BITMAP_PAGE_PENDING))
+ set_page_attr(bitmap, i, BITMAP_PAGE_NEEDWRITE);
bitmap_unplug(bitmap);
}
+ bitmap_unplug(mddev->bitmap);
*low = lo;
*high = hi;
err:
@@ -2032,6 +2069,35 @@
chunks << chunkshift);
spin_lock_irq(&bitmap->counts.lock);
+ /* For cluster raid, need to pre-allocate bitmap */
+ if (mddev_is_clustered(bitmap->mddev)) {
+ unsigned long page;
+ for (page = 0; page < pages; page++) {
+ ret = bitmap_checkpage(&bitmap->counts, page, 1, 1);
+ if (ret) {
+ unsigned long k;
+
+ /* deallocate the page memory */
+ for (k = 0; k < page; k++) {
+ kfree(new_bp[k].map);
+ }
+
+ /* restore some fields from old_counts */
+ bitmap->counts.bp = old_counts.bp;
+ bitmap->counts.pages = old_counts.pages;
+ bitmap->counts.missing_pages = old_counts.pages;
+ bitmap->counts.chunkshift = old_counts.chunkshift;
+ bitmap->counts.chunks = old_counts.chunks;
+ bitmap->mddev->bitmap_info.chunksize = 1 << (old_counts.chunkshift +
+ BITMAP_BLOCK_SHIFT);
+ blocks = old_counts.chunks << old_counts.chunkshift;
+ pr_err("Could not pre-allocate in-memory bitmap for cluster raid\n");
+ break;
+ } else
+ bitmap->counts.bp[page].count += 1;
+ }
+ }
+
for (block = 0; block < blocks; ) {
bitmap_counter_t *bmc_old, *bmc_new;
int set;
diff --git a/drivers/md/bitmap.h b/drivers/md/bitmap.h
index 5e3fcd6..5b6dd63 100644
--- a/drivers/md/bitmap.h
+++ b/drivers/md/bitmap.h
@@ -258,6 +258,9 @@
void bitmap_end_sync(struct bitmap *bitmap, sector_t offset, sector_t *blocks, int aborted);
void bitmap_close_sync(struct bitmap *bitmap);
void bitmap_cond_end_sync(struct bitmap *bitmap, sector_t sector, bool force);
+void bitmap_sync_with_cluster(struct mddev *mddev,
+ sector_t old_lo, sector_t old_hi,
+ sector_t new_lo, sector_t new_hi);
void bitmap_unplug(struct bitmap *bitmap);
void bitmap_daemon_work(struct mddev *mddev);
diff --git a/drivers/md/md-cluster.c b/drivers/md/md-cluster.c
index dd97d42..41573f1 100644
--- a/drivers/md/md-cluster.c
+++ b/drivers/md/md-cluster.c
@@ -61,6 +61,10 @@
* the lock.
*/
#define MD_CLUSTER_SEND_LOCKED_ALREADY 5
+/* We should receive message after node joined cluster and
+ * set up all the related infos such as bitmap and personality */
+#define MD_CLUSTER_ALREADY_IN_CLUSTER 6
+#define MD_CLUSTER_PENDING_RECV_EVENT 7
struct md_cluster_info {
@@ -85,6 +89,9 @@
struct completion newdisk_completion;
wait_queue_head_t wait;
unsigned long state;
+ /* record the region in RESYNCING message */
+ sector_t sync_low;
+ sector_t sync_hi;
};
enum msg_type {
@@ -284,11 +291,14 @@
goto dlm_unlock;
}
if (hi > 0) {
- /* TODO:Wait for current resync to get over */
- set_bit(MD_RECOVERY_NEEDED, &mddev->recovery);
if (lo < mddev->recovery_cp)
mddev->recovery_cp = lo;
- md_check_recovery(mddev);
+ /* wake up thread to continue resync in case resync
+ * is not finished */
+ if (mddev->recovery_cp != MaxSector) {
+ set_bit(MD_RECOVERY_NEEDED, &mddev->recovery);
+ md_wakeup_thread(mddev->thread);
+ }
}
dlm_unlock:
dlm_unlock_sync(bm_lockres);
@@ -370,8 +380,12 @@
struct dlm_lock_resource *res = arg;
struct md_cluster_info *cinfo = res->mddev->cluster_info;
- if (mode == DLM_LOCK_EX)
- md_wakeup_thread(cinfo->recv_thread);
+ if (mode == DLM_LOCK_EX) {
+ if (test_bit(MD_CLUSTER_ALREADY_IN_CLUSTER, &cinfo->state))
+ md_wakeup_thread(cinfo->recv_thread);
+ else
+ set_bit(MD_CLUSTER_PENDING_RECV_EVENT, &cinfo->state);
+ }
}
static void __remove_suspend_info(struct md_cluster_info *cinfo, int slot)
@@ -408,6 +422,30 @@
md_wakeup_thread(mddev->thread);
return;
}
+
+ /*
+ * The bitmaps are not same for different nodes
+ * if RESYNCING is happening in one node, then
+ * the node which received the RESYNCING message
+ * probably will perform resync with the region
+ * [lo, hi] again, so we could reduce resync time
+ * a lot if we can ensure that the bitmaps among
+ * different nodes are match up well.
+ *
+ * sync_low/hi is used to record the region which
+ * arrived in the previous RESYNCING message,
+ *
+ * Call bitmap_sync_with_cluster to clear
+ * NEEDED_MASK and set RESYNC_MASK since
+ * resync thread is running in another node,
+ * so we don't need to do the resync again
+ * with the same section */
+ bitmap_sync_with_cluster(mddev, cinfo->sync_low,
+ cinfo->sync_hi,
+ lo, hi);
+ cinfo->sync_low = lo;
+ cinfo->sync_hi = hi;
+
s = kzalloc(sizeof(struct suspend_info), GFP_KERNEL);
if (!s)
return;
@@ -482,11 +520,13 @@
__func__, __LINE__, le32_to_cpu(msg->raid_slot));
}
-static void process_recvd_msg(struct mddev *mddev, struct cluster_msg *msg)
+static int process_recvd_msg(struct mddev *mddev, struct cluster_msg *msg)
{
+ int ret = 0;
+
if (WARN(mddev->cluster_info->slot_number - 1 == le32_to_cpu(msg->slot),
"node %d received it's own msg\n", le32_to_cpu(msg->slot)))
- return;
+ return -1;
switch (le32_to_cpu(msg->type)) {
case METADATA_UPDATED:
process_metadata_update(mddev, msg);
@@ -509,9 +549,11 @@
__recover_slot(mddev, le32_to_cpu(msg->slot));
break;
default:
+ ret = -1;
pr_warn("%s:%d Received unknown message from %d\n",
__func__, __LINE__, msg->slot);
}
+ return ret;
}
/*
@@ -535,7 +577,9 @@
/* read lvb and wake up thread to process this message_lockres */
memcpy(&msg, message_lockres->lksb.sb_lvbptr, sizeof(struct cluster_msg));
- process_recvd_msg(thread->mddev, &msg);
+ ret = process_recvd_msg(thread->mddev, &msg);
+ if (ret)
+ goto out;
/*release CR on ack_lockres*/
ret = dlm_unlock_sync(ack_lockres);
@@ -549,6 +593,7 @@
ret = dlm_lock_sync(ack_lockres, DLM_LOCK_CR);
if (unlikely(ret != 0))
pr_info("lock CR on ack failed return %d\n", ret);
+out:
/*release CR on message_lockres*/
ret = dlm_unlock_sync(message_lockres);
if (unlikely(ret != 0))
@@ -778,17 +823,24 @@
cinfo->token_lockres = lockres_init(mddev, "token", NULL, 0);
if (!cinfo->token_lockres)
goto err;
- cinfo->ack_lockres = lockres_init(mddev, "ack", ack_bast, 0);
- if (!cinfo->ack_lockres)
- goto err;
cinfo->no_new_dev_lockres = lockres_init(mddev, "no-new-dev", NULL, 0);
if (!cinfo->no_new_dev_lockres)
goto err;
+ ret = dlm_lock_sync(cinfo->token_lockres, DLM_LOCK_EX);
+ if (ret) {
+ ret = -EAGAIN;
+ pr_err("md-cluster: can't join cluster to avoid lock issue\n");
+ goto err;
+ }
+ cinfo->ack_lockres = lockres_init(mddev, "ack", ack_bast, 0);
+ if (!cinfo->ack_lockres)
+ goto err;
/* get sync CR lock on ACK. */
if (dlm_lock_sync(cinfo->ack_lockres, DLM_LOCK_CR))
pr_err("md-cluster: failed to get a sync CR lock on ACK!(%d)\n",
ret);
+ dlm_unlock_sync(cinfo->token_lockres);
/* get sync CR lock on no-new-dev. */
if (dlm_lock_sync(cinfo->no_new_dev_lockres, DLM_LOCK_CR))
pr_err("md-cluster: failed to get a sync CR lock on no-new-dev!(%d)\n", ret);
@@ -809,12 +861,10 @@
if (!cinfo->resync_lockres)
goto err;
- ret = gather_all_resync_info(mddev, nodes);
- if (ret)
- goto err;
-
return 0;
err:
+ md_unregister_thread(&cinfo->recovery_thread);
+ md_unregister_thread(&cinfo->recv_thread);
lockres_free(cinfo->message_lockres);
lockres_free(cinfo->token_lockres);
lockres_free(cinfo->ack_lockres);
@@ -828,6 +878,19 @@
return ret;
}
+static void load_bitmaps(struct mddev *mddev, int total_slots)
+{
+ struct md_cluster_info *cinfo = mddev->cluster_info;
+
+ /* load all the node's bitmap info for resync */
+ if (gather_all_resync_info(mddev, total_slots))
+ pr_err("md-cluster: failed to gather all resyn infos\n");
+ set_bit(MD_CLUSTER_ALREADY_IN_CLUSTER, &cinfo->state);
+ /* wake up recv thread in case something need to be handled */
+ if (test_and_clear_bit(MD_CLUSTER_PENDING_RECV_EVENT, &cinfo->state))
+ md_wakeup_thread(cinfo->recv_thread);
+}
+
static void resync_bitmap(struct mddev *mddev)
{
struct md_cluster_info *cinfo = mddev->cluster_info;
@@ -937,7 +1000,6 @@
static int resync_start(struct mddev *mddev)
{
struct md_cluster_info *cinfo = mddev->cluster_info;
- cinfo->resync_lockres->flags |= DLM_LKF_NOQUEUE;
return dlm_lock_sync(cinfo->resync_lockres, DLM_LOCK_EX);
}
@@ -967,7 +1029,6 @@
static int resync_finish(struct mddev *mddev)
{
struct md_cluster_info *cinfo = mddev->cluster_info;
- cinfo->resync_lockres->flags &= ~DLM_LKF_NOQUEUE;
dlm_unlock_sync(cinfo->resync_lockres);
return resync_info_update(mddev, 0, 0);
}
@@ -1171,6 +1232,7 @@
.add_new_disk_cancel = add_new_disk_cancel,
.new_disk_ack = new_disk_ack,
.remove_disk = remove_disk,
+ .load_bitmaps = load_bitmaps,
.gather_bitmaps = gather_bitmaps,
.lock_all_bitmaps = lock_all_bitmaps,
.unlock_all_bitmaps = unlock_all_bitmaps,
diff --git a/drivers/md/md-cluster.h b/drivers/md/md-cluster.h
index 45ce6c9..e765499 100644
--- a/drivers/md/md-cluster.h
+++ b/drivers/md/md-cluster.h
@@ -23,6 +23,7 @@
void (*add_new_disk_cancel)(struct mddev *mddev);
int (*new_disk_ack)(struct mddev *mddev, bool ack);
int (*remove_disk)(struct mddev *mddev, struct md_rdev *rdev);
+ void (*load_bitmaps)(struct mddev *mddev, int total_slots);
int (*gather_bitmaps)(struct md_rdev *rdev);
int (*lock_all_bitmaps)(struct mddev *mddev);
void (*unlock_all_bitmaps)(struct mddev *mddev);
diff --git a/drivers/md/md.c b/drivers/md/md.c
index c9a475c..866825f 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -307,7 +307,7 @@
*/
void mddev_suspend(struct mddev *mddev)
{
- WARN_ON_ONCE(current == mddev->thread->tsk);
+ WARN_ON_ONCE(mddev->thread && current == mddev->thread->tsk);
if (mddev->suspended++)
return;
synchronize_rcu();
@@ -2291,19 +2291,24 @@
return;
}
+repeat:
if (mddev_is_clustered(mddev)) {
if (test_and_clear_bit(MD_CHANGE_DEVS, &mddev->flags))
force_change = 1;
+ if (test_and_clear_bit(MD_CHANGE_CLEAN, &mddev->flags))
+ nospares = 1;
ret = md_cluster_ops->metadata_update_start(mddev);
/* Has someone else has updated the sb */
if (!does_sb_need_changing(mddev)) {
if (ret == 0)
md_cluster_ops->metadata_update_cancel(mddev);
- clear_bit(MD_CHANGE_PENDING, &mddev->flags);
+ bit_clear_unless(&mddev->flags, BIT(MD_CHANGE_PENDING),
+ BIT(MD_CHANGE_DEVS) |
+ BIT(MD_CHANGE_CLEAN));
return;
}
}
-repeat:
+
/* First make sure individual recovery_offsets are correct */
rdev_for_each(rdev, mddev) {
if (rdev->raid_disk >= 0 &&
@@ -2430,15 +2435,14 @@
md_super_wait(mddev);
/* if there was a failure, MD_CHANGE_DEVS was set, and we re-write super */
- spin_lock(&mddev->lock);
+ if (mddev_is_clustered(mddev) && ret == 0)
+ md_cluster_ops->metadata_update_finish(mddev);
+
if (mddev->in_sync != sync_req ||
- test_bit(MD_CHANGE_DEVS, &mddev->flags)) {
+ !bit_clear_unless(&mddev->flags, BIT(MD_CHANGE_PENDING),
+ BIT(MD_CHANGE_DEVS) | BIT(MD_CHANGE_CLEAN)))
/* have to write it out again */
- spin_unlock(&mddev->lock);
goto repeat;
- }
- clear_bit(MD_CHANGE_PENDING, &mddev->flags);
- spin_unlock(&mddev->lock);
wake_up(&mddev->sb_wait);
if (test_bit(MD_RECOVERY_RUNNING, &mddev->recovery))
sysfs_notify(&mddev->kobj, NULL, "sync_completed");
@@ -2452,9 +2456,6 @@
clear_bit(BlockedBadBlocks, &rdev->flags);
wake_up(&rdev->blocked_wait);
}
-
- if (mddev_is_clustered(mddev) && ret == 0)
- md_cluster_ops->metadata_update_finish(mddev);
}
EXPORT_SYMBOL(md_update_sb);
@@ -4816,6 +4817,10 @@
if (err)
return err;
+ /* cluster raid doesn't support change array_sectors */
+ if (mddev_is_clustered(mddev))
+ return -EINVAL;
+
if (strncmp(buf, "default", 7) == 0) {
if (mddev->pers)
sectors = mddev->pers->size(mddev, 0, 0);
@@ -6437,6 +6442,10 @@
int rv;
int fit = (num_sectors == 0);
+ /* cluster raid doesn't support update size */
+ if (mddev_is_clustered(mddev))
+ return -EINVAL;
+
if (mddev->pers->resize == NULL)
return -EINVAL;
/* The "num_sectors" is the number of sectors of each device that
@@ -7785,7 +7794,7 @@
struct md_rdev *rdev;
char *desc, *action = NULL;
struct blk_plug plug;
- bool cluster_resync_finished = false;
+ int ret;
/* just incase thread restarts... */
if (test_bit(MD_RECOVERY_DONE, &mddev->recovery))
@@ -7795,6 +7804,19 @@
return;
}
+ if (mddev_is_clustered(mddev)) {
+ ret = md_cluster_ops->resync_start(mddev);
+ if (ret)
+ goto skip;
+
+ if (!(test_bit(MD_RECOVERY_SYNC, &mddev->recovery) ||
+ test_bit(MD_RECOVERY_RESHAPE, &mddev->recovery) ||
+ test_bit(MD_RECOVERY_RECOVER, &mddev->recovery))
+ && ((unsigned long long)mddev->curr_resync_completed
+ < (unsigned long long)mddev->resync_max_sectors))
+ goto skip;
+ }
+
if (test_bit(MD_RECOVERY_SYNC, &mddev->recovery)) {
if (test_bit(MD_RECOVERY_CHECK, &mddev->recovery)) {
desc = "data-check";
@@ -8089,11 +8111,6 @@
mddev->curr_resync_completed = mddev->curr_resync;
sysfs_notify(&mddev->kobj, NULL, "sync_completed");
}
- /* tell personality and other nodes that we are finished */
- if (mddev_is_clustered(mddev)) {
- md_cluster_ops->resync_finish(mddev);
- cluster_resync_finished = true;
- }
mddev->pers->sync_request(mddev, max_sectors, &skipped);
if (!test_bit(MD_RECOVERY_CHECK, &mddev->recovery) &&
@@ -8130,12 +8147,18 @@
}
}
skip:
- set_bit(MD_CHANGE_DEVS, &mddev->flags);
-
if (mddev_is_clustered(mddev) &&
- test_bit(MD_RECOVERY_INTR, &mddev->recovery) &&
- !cluster_resync_finished)
+ ret == 0) {
+ /* set CHANGE_PENDING here since maybe another
+ * update is needed, so other nodes are informed */
+ set_mask_bits(&mddev->flags, 0,
+ BIT(MD_CHANGE_PENDING) | BIT(MD_CHANGE_DEVS));
+ md_wakeup_thread(mddev->thread);
+ wait_event(mddev->sb_wait,
+ !test_bit(MD_CHANGE_PENDING, &mddev->flags));
md_cluster_ops->resync_finish(mddev);
+ } else
+ set_bit(MD_CHANGE_DEVS, &mddev->flags);
spin_lock(&mddev->lock);
if (!test_bit(MD_RECOVERY_INTR, &mddev->recovery)) {
@@ -8226,18 +8249,9 @@
struct mddev *mddev = container_of(ws, struct mddev, del_work);
int ret = 0;
- if (mddev_is_clustered(mddev)) {
- ret = md_cluster_ops->resync_start(mddev);
- if (ret) {
- mddev->sync_thread = NULL;
- goto out;
- }
- }
-
mddev->sync_thread = md_register_thread(md_do_sync,
mddev,
"resync");
-out:
if (!mddev->sync_thread) {
if (!(mddev_is_clustered(mddev) && ret == -EAGAIN))
printk(KERN_ERR "%s: could not start resync"
@@ -8536,6 +8550,7 @@
int rdev_set_badblocks(struct md_rdev *rdev, sector_t s, int sectors,
int is_new)
{
+ struct mddev *mddev = rdev->mddev;
int rv;
if (is_new)
s += rdev->new_data_offset;
@@ -8545,8 +8560,8 @@
if (rv == 0) {
/* Make sure they get written out promptly */
sysfs_notify_dirent_safe(rdev->sysfs_state);
- set_bit(MD_CHANGE_CLEAN, &rdev->mddev->flags);
- set_bit(MD_CHANGE_PENDING, &rdev->mddev->flags);
+ set_mask_bits(&mddev->flags, 0,
+ BIT(MD_CHANGE_CLEAN) | BIT(MD_CHANGE_PENDING));
md_wakeup_thread(rdev->mddev->thread);
return 1;
} else
@@ -8680,6 +8695,11 @@
ret = remove_and_add_spares(mddev, rdev2);
pr_info("Activated spare: %s\n",
bdevname(rdev2->bdev,b));
+ /* wakeup mddev->thread here, so array could
+ * perform resync with the new activated disk */
+ set_bit(MD_RECOVERY_NEEDED, &mddev->recovery);
+ md_wakeup_thread(mddev->thread);
+
}
/* device faulty
* We just want to do the minimum to mark the disk
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index a7f2b9c..c7c8cde 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -1474,8 +1474,8 @@
* if recovery is running, make sure it aborts.
*/
set_bit(MD_RECOVERY_INTR, &mddev->recovery);
- set_bit(MD_CHANGE_DEVS, &mddev->flags);
- set_bit(MD_CHANGE_PENDING, &mddev->flags);
+ set_mask_bits(&mddev->flags, 0,
+ BIT(MD_CHANGE_DEVS) | BIT(MD_CHANGE_PENDING));
printk(KERN_ALERT
"md/raid1:%s: Disk failure on %s, disabling device.\n"
"md/raid1:%s: Operation continuing on %d devices.\n",
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index e3fd725..c7de2a5 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -1102,8 +1102,8 @@
bio->bi_iter.bi_sector < conf->reshape_progress))) {
/* Need to update reshape_position in metadata */
mddev->reshape_position = conf->reshape_progress;
- set_bit(MD_CHANGE_DEVS, &mddev->flags);
- set_bit(MD_CHANGE_PENDING, &mddev->flags);
+ set_mask_bits(&mddev->flags, 0,
+ BIT(MD_CHANGE_DEVS) | BIT(MD_CHANGE_PENDING));
md_wakeup_thread(mddev->thread);
wait_event(mddev->sb_wait,
!test_bit(MD_CHANGE_PENDING, &mddev->flags));
@@ -1591,8 +1591,8 @@
set_bit(MD_RECOVERY_INTR, &mddev->recovery);
set_bit(Blocked, &rdev->flags);
set_bit(Faulty, &rdev->flags);
- set_bit(MD_CHANGE_DEVS, &mddev->flags);
- set_bit(MD_CHANGE_PENDING, &mddev->flags);
+ set_mask_bits(&mddev->flags, 0,
+ BIT(MD_CHANGE_DEVS) | BIT(MD_CHANGE_PENDING));
spin_unlock_irqrestore(&conf->device_lock, flags);
printk(KERN_ALERT
"md/raid10:%s: Disk failure on %s, disabling device.\n"
@@ -3782,8 +3782,10 @@
return ret;
}
md_set_array_sectors(mddev, size);
- set_capacity(mddev->gendisk, mddev->array_sectors);
- revalidate_disk(mddev->gendisk);
+ if (mddev->queue) {
+ set_capacity(mddev->gendisk, mddev->array_sectors);
+ revalidate_disk(mddev->gendisk);
+ }
if (sectors > mddev->dev_sectors &&
mddev->recovery_cp > oldsize) {
mddev->recovery_cp = oldsize;
@@ -4593,8 +4595,10 @@
set_bit(MD_RECOVERY_NEEDED, &mddev->recovery);
}
mddev->resync_max_sectors = size;
- set_capacity(mddev->gendisk, mddev->array_sectors);
- revalidate_disk(mddev->gendisk);
+ if (mddev->queue) {
+ set_capacity(mddev->gendisk, mddev->array_sectors);
+ revalidate_disk(mddev->gendisk);
+ }
} else {
int d;
for (d = conf->geo.raid_disks ;
diff --git a/drivers/md/raid5-cache.c b/drivers/md/raid5-cache.c
index 26f1497..e889e2d 100644
--- a/drivers/md/raid5-cache.c
+++ b/drivers/md/raid5-cache.c
@@ -712,8 +712,8 @@
* in_teardown check workaround this issue.
*/
if (!log->in_teardown) {
- set_bit(MD_CHANGE_DEVS, &mddev->flags);
- set_bit(MD_CHANGE_PENDING, &mddev->flags);
+ set_mask_bits(&mddev->flags, 0,
+ BIT(MD_CHANGE_DEVS) | BIT(MD_CHANGE_PENDING));
md_wakeup_thread(mddev->thread);
wait_event(mddev->sb_wait,
!test_bit(MD_CHANGE_PENDING, &mddev->flags) ||
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index e48c262..8959e6d 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -2514,8 +2514,8 @@
set_bit(Blocked, &rdev->flags);
set_bit(Faulty, &rdev->flags);
- set_bit(MD_CHANGE_DEVS, &mddev->flags);
- set_bit(MD_CHANGE_PENDING, &mddev->flags);
+ set_mask_bits(&mddev->flags, 0,
+ BIT(MD_CHANGE_DEVS) | BIT(MD_CHANGE_PENDING));
printk(KERN_ALERT
"md/raid:%s: Disk failure on %s, disabling device.\n"
"md/raid:%s: Operation continuing on %d devices.\n",
@@ -7572,8 +7572,10 @@
if (mddev->delta_disks > 0) {
md_set_array_sectors(mddev, raid5_size(mddev, 0, 0));
- set_capacity(mddev->gendisk, mddev->array_sectors);
- revalidate_disk(mddev->gendisk);
+ if (mddev->queue) {
+ set_capacity(mddev->gendisk, mddev->array_sectors);
+ revalidate_disk(mddev->gendisk);
+ }
} else {
int d;
spin_lock_irq(&conf->device_lock);
diff --git a/drivers/media/common/Kconfig b/drivers/media/common/Kconfig
index 21154dd..326df0a 100644
--- a/drivers/media/common/Kconfig
+++ b/drivers/media/common/Kconfig
@@ -19,3 +19,4 @@
source "drivers/media/common/b2c2/Kconfig"
source "drivers/media/common/saa7146/Kconfig"
source "drivers/media/common/siano/Kconfig"
+source "drivers/media/common/v4l2-tpg/Kconfig"
diff --git a/drivers/media/common/Makefile b/drivers/media/common/Makefile
index 89b795d..2d1b0a0 100644
--- a/drivers/media/common/Makefile
+++ b/drivers/media/common/Makefile
@@ -1,4 +1,4 @@
-obj-y += b2c2/ saa7146/ siano/
+obj-y += b2c2/ saa7146/ siano/ v4l2-tpg/
obj-$(CONFIG_VIDEO_CX2341X) += cx2341x.o
obj-$(CONFIG_VIDEO_TVEEPROM) += tveeprom.o
obj-$(CONFIG_CYPRESS_FIRMWARE) += cypress_firmware.o
diff --git a/drivers/media/common/v4l2-tpg/Kconfig b/drivers/media/common/v4l2-tpg/Kconfig
new file mode 100644
index 0000000..7456fc1
--- /dev/null
+++ b/drivers/media/common/v4l2-tpg/Kconfig
@@ -0,0 +1,2 @@
+config VIDEO_V4L2_TPG
+ tristate
diff --git a/drivers/media/common/v4l2-tpg/Makefile b/drivers/media/common/v4l2-tpg/Makefile
new file mode 100644
index 0000000..f588df4
--- /dev/null
+++ b/drivers/media/common/v4l2-tpg/Makefile
@@ -0,0 +1,3 @@
+v4l2-tpg-objs := v4l2-tpg-core.o v4l2-tpg-colors.o
+
+obj-$(CONFIG_VIDEO_V4L2_TPG) += v4l2-tpg.o
diff --git a/drivers/media/common/v4l2-tpg/v4l2-tpg-colors.c b/drivers/media/common/v4l2-tpg/v4l2-tpg-colors.c
new file mode 100644
index 0000000..9bcbd31
--- /dev/null
+++ b/drivers/media/common/v4l2-tpg/v4l2-tpg-colors.c
@@ -0,0 +1,1415 @@
+/*
+ * v4l2-tpg-colors.c - A table that converts colors to various colorspaces
+ *
+ * The test pattern generator uses the tpg_colors for its test patterns.
+ * For testing colorspaces the first 8 colors of that table need to be
+ * converted to their equivalent in the target colorspace.
+ *
+ * The tpg_csc_colors[] table is the result of that conversion and since
+ * it is precalculated the colorspace conversion is just a simple table
+ * lookup.
+ *
+ * This source also contains the code used to generate the tpg_csc_colors
+ * table. Run the following command to compile it:
+ *
+ * gcc v4l2-tpg-colors.c -DCOMPILE_APP -o gen-colors -lm
+ *
+ * and run the utility.
+ *
+ * Note that the converted colors are in the range 0x000-0xff0 (so times 16)
+ * in order to preserve precision.
+ *
+ * Copyright 2014 Cisco Systems, Inc. and/or its affiliates. All rights reserved.
+ *
+ * This program is free software; you may redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <linux/videodev2.h>
+#include <media/v4l2-tpg-colors.h>
+
+/* sRGB colors with range [0-255] */
+const struct color tpg_colors[TPG_COLOR_MAX] = {
+ /*
+ * Colors to test colorspace conversion: converting these colors
+ * to other colorspaces will never lead to out-of-gamut colors.
+ */
+ { 191, 191, 191 }, /* TPG_COLOR_CSC_WHITE */
+ { 191, 191, 50 }, /* TPG_COLOR_CSC_YELLOW */
+ { 50, 191, 191 }, /* TPG_COLOR_CSC_CYAN */
+ { 50, 191, 50 }, /* TPG_COLOR_CSC_GREEN */
+ { 191, 50, 191 }, /* TPG_COLOR_CSC_MAGENTA */
+ { 191, 50, 50 }, /* TPG_COLOR_CSC_RED */
+ { 50, 50, 191 }, /* TPG_COLOR_CSC_BLUE */
+ { 50, 50, 50 }, /* TPG_COLOR_CSC_BLACK */
+
+ /* 75% colors */
+ { 191, 191, 0 }, /* TPG_COLOR_75_YELLOW */
+ { 0, 191, 191 }, /* TPG_COLOR_75_CYAN */
+ { 0, 191, 0 }, /* TPG_COLOR_75_GREEN */
+ { 191, 0, 191 }, /* TPG_COLOR_75_MAGENTA */
+ { 191, 0, 0 }, /* TPG_COLOR_75_RED */
+ { 0, 0, 191 }, /* TPG_COLOR_75_BLUE */
+
+ /* 100% colors */
+ { 255, 255, 255 }, /* TPG_COLOR_100_WHITE */
+ { 255, 255, 0 }, /* TPG_COLOR_100_YELLOW */
+ { 0, 255, 255 }, /* TPG_COLOR_100_CYAN */
+ { 0, 255, 0 }, /* TPG_COLOR_100_GREEN */
+ { 255, 0, 255 }, /* TPG_COLOR_100_MAGENTA */
+ { 255, 0, 0 }, /* TPG_COLOR_100_RED */
+ { 0, 0, 255 }, /* TPG_COLOR_100_BLUE */
+ { 0, 0, 0 }, /* TPG_COLOR_100_BLACK */
+
+ { 0, 0, 0 }, /* TPG_COLOR_RANDOM placeholder */
+};
+
+#ifndef COMPILE_APP
+
+/* Generated table */
+const unsigned short tpg_rec709_to_linear[255 * 16 + 1] = {
+ 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 2, 3, 3, 3, 3,
+ 4, 4, 4, 4, 4, 5, 5, 5, 5, 6, 6, 6, 6, 6, 7, 7,
+ 7, 7, 8, 8, 8, 8, 8, 9, 9, 9, 9, 10, 10, 10, 10, 10,
+ 11, 11, 11, 11, 12, 12, 12, 12, 12, 13, 13, 13, 13, 14, 14, 14,
+ 14, 14, 15, 15, 15, 15, 16, 16, 16, 16, 16, 17, 17, 17, 17, 18,
+ 18, 18, 18, 18, 19, 19, 19, 19, 20, 20, 20, 20, 20, 21, 21, 21,
+ 21, 22, 22, 22, 22, 22, 23, 23, 23, 23, 24, 24, 24, 24, 24, 25,
+ 25, 25, 25, 26, 26, 26, 26, 26, 27, 27, 27, 27, 28, 28, 28, 28,
+ 28, 29, 29, 29, 29, 30, 30, 30, 30, 30, 31, 31, 31, 31, 32, 32,
+ 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 34, 34, 35, 35, 35, 35,
+ 36, 36, 36, 36, 36, 37, 37, 37, 37, 38, 38, 38, 38, 38, 39, 39,
+ 39, 39, 40, 40, 40, 40, 40, 41, 41, 41, 41, 42, 42, 42, 42, 42,
+ 43, 43, 43, 43, 44, 44, 44, 44, 44, 45, 45, 45, 45, 46, 46, 46,
+ 46, 46, 47, 47, 47, 47, 48, 48, 48, 48, 48, 49, 49, 49, 49, 50,
+ 50, 50, 50, 50, 51, 51, 51, 51, 52, 52, 52, 52, 52, 53, 53, 53,
+ 53, 54, 54, 54, 54, 54, 55, 55, 55, 55, 56, 56, 56, 56, 56, 57,
+ 57, 57, 57, 58, 58, 58, 58, 58, 59, 59, 59, 59, 60, 60, 60, 60,
+ 60, 61, 61, 61, 61, 62, 62, 62, 62, 62, 63, 63, 63, 63, 64, 64,
+ 64, 64, 64, 65, 65, 65, 65, 66, 66, 66, 66, 66, 67, 67, 67, 67,
+ 68, 68, 68, 68, 68, 69, 69, 69, 69, 70, 70, 70, 70, 70, 71, 71,
+ 71, 71, 72, 72, 72, 72, 72, 73, 73, 73, 73, 73, 74, 74, 74, 74,
+ 74, 75, 75, 75, 75, 76, 76, 76, 76, 76, 77, 77, 77, 77, 78, 78,
+ 78, 78, 79, 79, 79, 79, 79, 80, 80, 80, 80, 81, 81, 81, 81, 82,
+ 82, 82, 82, 82, 83, 83, 83, 83, 84, 84, 84, 84, 85, 85, 85, 85,
+ 86, 86, 86, 86, 87, 87, 87, 87, 88, 88, 88, 88, 89, 89, 89, 89,
+ 90, 90, 90, 90, 91, 91, 91, 91, 92, 92, 92, 92, 93, 93, 93, 93,
+ 94, 94, 94, 94, 95, 95, 95, 95, 96, 96, 96, 96, 97, 97, 97, 97,
+ 98, 98, 98, 98, 99, 99, 99, 99, 100, 100, 100, 101, 101, 101, 101, 102,
+ 102, 102, 102, 103, 103, 103, 103, 104, 104, 104, 105, 105, 105, 105, 106, 106,
+ 106, 106, 107, 107, 107, 107, 108, 108, 108, 109, 109, 109, 109, 110, 110, 110,
+ 111, 111, 111, 111, 112, 112, 112, 112, 113, 113, 113, 114, 114, 114, 114, 115,
+ 115, 115, 116, 116, 116, 116, 117, 117, 117, 118, 118, 118, 118, 119, 119, 119,
+ 120, 120, 120, 120, 121, 121, 121, 122, 122, 122, 123, 123, 123, 123, 124, 124,
+ 124, 125, 125, 125, 125, 126, 126, 126, 127, 127, 127, 128, 128, 128, 128, 129,
+ 129, 129, 130, 130, 130, 131, 131, 131, 132, 132, 132, 132, 133, 133, 133, 134,
+ 134, 134, 135, 135, 135, 136, 136, 136, 136, 137, 137, 137, 138, 138, 138, 139,
+ 139, 139, 140, 140, 140, 141, 141, 141, 142, 142, 142, 142, 143, 143, 143, 144,
+ 144, 144, 145, 145, 145, 146, 146, 146, 147, 147, 147, 148, 148, 148, 149, 149,
+ 149, 150, 150, 150, 151, 151, 151, 152, 152, 152, 153, 153, 153, 154, 154, 154,
+ 155, 155, 155, 156, 156, 156, 157, 157, 157, 158, 158, 158, 159, 159, 159, 160,
+ 160, 160, 161, 161, 161, 162, 162, 162, 163, 163, 163, 164, 164, 164, 165, 165,
+ 165, 166, 166, 167, 167, 167, 168, 168, 168, 169, 169, 169, 170, 170, 170, 171,
+ 171, 171, 172, 172, 172, 173, 173, 174, 174, 174, 175, 175, 175, 176, 176, 176,
+ 177, 177, 177, 178, 178, 179, 179, 179, 180, 180, 180, 181, 181, 181, 182, 182,
+ 183, 183, 183, 184, 184, 184, 185, 185, 186, 186, 186, 187, 187, 187, 188, 188,
+ 188, 189, 189, 190, 190, 190, 191, 191, 191, 192, 192, 193, 193, 193, 194, 194,
+ 194, 195, 195, 196, 196, 196, 197, 197, 198, 198, 198, 199, 199, 199, 200, 200,
+ 201, 201, 201, 202, 202, 203, 203, 203, 204, 204, 204, 205, 205, 206, 206, 206,
+ 207, 207, 208, 208, 208, 209, 209, 210, 210, 210, 211, 211, 212, 212, 212, 213,
+ 213, 214, 214, 214, 215, 215, 216, 216, 216, 217, 217, 218, 218, 218, 219, 219,
+ 220, 220, 220, 221, 221, 222, 222, 222, 223, 223, 224, 224, 224, 225, 225, 226,
+ 226, 227, 227, 227, 228, 228, 229, 229, 229, 230, 230, 231, 231, 232, 232, 232,
+ 233, 233, 234, 234, 234, 235, 235, 236, 236, 237, 237, 237, 238, 238, 239, 239,
+ 240, 240, 240, 241, 241, 242, 242, 243, 243, 243, 244, 244, 245, 245, 246, 246,
+ 246, 247, 247, 248, 248, 249, 249, 249, 250, 250, 251, 251, 252, 252, 252, 253,
+ 253, 254, 254, 255, 255, 256, 256, 256, 257, 257, 258, 258, 259, 259, 260, 260,
+ 260, 261, 261, 262, 262, 263, 263, 264, 264, 264, 265, 265, 266, 266, 267, 267,
+ 268, 268, 269, 269, 269, 270, 270, 271, 271, 272, 272, 273, 273, 274, 274, 274,
+ 275, 275, 276, 276, 277, 277, 278, 278, 279, 279, 279, 280, 280, 281, 281, 282,
+ 282, 283, 283, 284, 284, 285, 285, 286, 286, 286, 287, 287, 288, 288, 289, 289,
+ 290, 290, 291, 291, 292, 292, 293, 293, 294, 294, 295, 295, 295, 296, 296, 297,
+ 297, 298, 298, 299, 299, 300, 300, 301, 301, 302, 302, 303, 303, 304, 304, 305,
+ 305, 306, 306, 307, 307, 308, 308, 309, 309, 309, 310, 310, 311, 311, 312, 312,
+ 313, 313, 314, 314, 315, 315, 316, 316, 317, 317, 318, 318, 319, 319, 320, 320,
+ 321, 321, 322, 322, 323, 323, 324, 324, 325, 325, 326, 326, 327, 327, 328, 328,
+ 329, 329, 330, 330, 331, 331, 332, 332, 333, 333, 334, 335, 335, 336, 336, 337,
+ 337, 338, 338, 339, 339, 340, 340, 341, 341, 342, 342, 343, 343, 344, 344, 345,
+ 345, 346, 346, 347, 347, 348, 348, 349, 349, 350, 351, 351, 352, 352, 353, 353,
+ 354, 354, 355, 355, 356, 356, 357, 357, 358, 358, 359, 360, 360, 361, 361, 362,
+ 362, 363, 363, 364, 364, 365, 365, 366, 366, 367, 368, 368, 369, 369, 370, 370,
+ 371, 371, 372, 372, 373, 373, 374, 375, 375, 376, 376, 377, 377, 378, 378, 379,
+ 379, 380, 381, 381, 382, 382, 383, 383, 384, 384, 385, 386, 386, 387, 387, 388,
+ 388, 389, 389, 390, 391, 391, 392, 392, 393, 393, 394, 394, 395, 396, 396, 397,
+ 397, 398, 398, 399, 399, 400, 401, 401, 402, 402, 403, 403, 404, 405, 405, 406,
+ 406, 407, 407, 408, 409, 409, 410, 410, 411, 411, 412, 413, 413, 414, 414, 415,
+ 415, 416, 417, 417, 418, 418, 419, 419, 420, 421, 421, 422, 422, 423, 424, 424,
+ 425, 425, 426, 426, 427, 428, 428, 429, 429, 430, 431, 431, 432, 432, 433, 433,
+ 434, 435, 435, 436, 436, 437, 438, 438, 439, 439, 440, 441, 441, 442, 442, 443,
+ 444, 444, 445, 445, 446, 447, 447, 448, 448, 449, 450, 450, 451, 451, 452, 453,
+ 453, 454, 454, 455, 456, 456, 457, 457, 458, 459, 459, 460, 460, 461, 462, 462,
+ 463, 463, 464, 465, 465, 466, 467, 467, 468, 468, 469, 470, 470, 471, 471, 472,
+ 473, 473, 474, 475, 475, 476, 476, 477, 478, 478, 479, 480, 480, 481, 481, 482,
+ 483, 483, 484, 485, 485, 486, 486, 487, 488, 488, 489, 490, 490, 491, 491, 492,
+ 493, 493, 494, 495, 495, 496, 497, 497, 498, 498, 499, 500, 500, 501, 502, 502,
+ 503, 504, 504, 505, 505, 506, 507, 507, 508, 509, 509, 510, 511, 511, 512, 513,
+ 513, 514, 514, 515, 516, 516, 517, 518, 518, 519, 520, 520, 521, 522, 522, 523,
+ 524, 524, 525, 526, 526, 527, 528, 528, 529, 529, 530, 531, 531, 532, 533, 533,
+ 534, 535, 535, 536, 537, 537, 538, 539, 539, 540, 541, 541, 542, 543, 543, 544,
+ 545, 545, 546, 547, 547, 548, 549, 549, 550, 551, 551, 552, 553, 553, 554, 555,
+ 555, 556, 557, 557, 558, 559, 560, 560, 561, 562, 562, 563, 564, 564, 565, 566,
+ 566, 567, 568, 568, 569, 570, 570, 571, 572, 572, 573, 574, 575, 575, 576, 577,
+ 577, 578, 579, 579, 580, 581, 581, 582, 583, 584, 584, 585, 586, 586, 587, 588,
+ 588, 589, 590, 590, 591, 592, 593, 593, 594, 595, 595, 596, 597, 598, 598, 599,
+ 600, 600, 601, 602, 602, 603, 604, 605, 605, 606, 607, 607, 608, 609, 610, 610,
+ 611, 612, 612, 613, 614, 615, 615, 616, 617, 617, 618, 619, 620, 620, 621, 622,
+ 622, 623, 624, 625, 625, 626, 627, 627, 628, 629, 630, 630, 631, 632, 632, 633,
+ 634, 635, 635, 636, 637, 638, 638, 639, 640, 640, 641, 642, 643, 643, 644, 645,
+ 646, 646, 647, 648, 649, 649, 650, 651, 652, 652, 653, 654, 654, 655, 656, 657,
+ 657, 658, 659, 660, 660, 661, 662, 663, 663, 664, 665, 666, 666, 667, 668, 669,
+ 669, 670, 671, 672, 672, 673, 674, 675, 675, 676, 677, 678, 678, 679, 680, 681,
+ 681, 682, 683, 684, 684, 685, 686, 687, 687, 688, 689, 690, 690, 691, 692, 693,
+ 694, 694, 695, 696, 697, 697, 698, 699, 700, 700, 701, 702, 703, 703, 704, 705,
+ 706, 707, 707, 708, 709, 710, 710, 711, 712, 713, 714, 714, 715, 716, 717, 717,
+ 718, 719, 720, 720, 721, 722, 723, 724, 724, 725, 726, 727, 728, 728, 729, 730,
+ 731, 731, 732, 733, 734, 735, 735, 736, 737, 738, 739, 739, 740, 741, 742, 742,
+ 743, 744, 745, 746, 746, 747, 748, 749, 750, 750, 751, 752, 753, 754, 754, 755,
+ 756, 757, 758, 758, 759, 760, 761, 762, 762, 763, 764, 765, 766, 766, 767, 768,
+ 769, 770, 771, 771, 772, 773, 774, 775, 775, 776, 777, 778, 779, 779, 780, 781,
+ 782, 783, 783, 784, 785, 786, 787, 788, 788, 789, 790, 791, 792, 793, 793, 794,
+ 795, 796, 797, 797, 798, 799, 800, 801, 802, 802, 803, 804, 805, 806, 807, 807,
+ 808, 809, 810, 811, 812, 812, 813, 814, 815, 816, 817, 817, 818, 819, 820, 821,
+ 822, 822, 823, 824, 825, 826, 827, 827, 828, 829, 830, 831, 832, 832, 833, 834,
+ 835, 836, 837, 838, 838, 839, 840, 841, 842, 843, 843, 844, 845, 846, 847, 848,
+ 849, 849, 850, 851, 852, 853, 854, 855, 855, 856, 857, 858, 859, 860, 861, 861,
+ 862, 863, 864, 865, 866, 867, 867, 868, 869, 870, 871, 872, 873, 873, 874, 875,
+ 876, 877, 878, 879, 880, 880, 881, 882, 883, 884, 885, 886, 887, 887, 888, 889,
+ 890, 891, 892, 893, 894, 894, 895, 896, 897, 898, 899, 900, 901, 901, 902, 903,
+ 904, 905, 906, 907, 908, 909, 909, 910, 911, 912, 913, 914, 915, 916, 916, 917,
+ 918, 919, 920, 921, 922, 923, 924, 925, 925, 926, 927, 928, 929, 930, 931, 932,
+ 933, 933, 934, 935, 936, 937, 938, 939, 940, 941, 942, 942, 943, 944, 945, 946,
+ 947, 948, 949, 950, 951, 952, 952, 953, 954, 955, 956, 957, 958, 959, 960, 961,
+ 962, 962, 963, 964, 965, 966, 967, 968, 969, 970, 971, 972, 973, 973, 974, 975,
+ 976, 977, 978, 979, 980, 981, 982, 983, 984, 985, 985, 986, 987, 988, 989, 990,
+ 991, 992, 993, 994, 995, 996, 997, 998, 998, 999, 1000, 1001, 1002, 1003, 1004, 1005,
+ 1006, 1007, 1008, 1009, 1010, 1011, 1012, 1013, 1013, 1014, 1015, 1016, 1017, 1018, 1019, 1020,
+ 1021, 1022, 1023, 1024, 1025, 1026, 1027, 1028, 1029, 1030, 1030, 1031, 1032, 1033, 1034, 1035,
+ 1036, 1037, 1038, 1039, 1040, 1041, 1042, 1043, 1044, 1045, 1046, 1047, 1048, 1049, 1050, 1050,
+ 1051, 1052, 1053, 1054, 1055, 1056, 1057, 1058, 1059, 1060, 1061, 1062, 1063, 1064, 1065, 1066,
+ 1067, 1068, 1069, 1070, 1071, 1072, 1073, 1074, 1075, 1076, 1077, 1078, 1078, 1079, 1080, 1081,
+ 1082, 1083, 1084, 1085, 1086, 1087, 1088, 1089, 1090, 1091, 1092, 1093, 1094, 1095, 1096, 1097,
+ 1098, 1099, 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112, 1113,
+ 1114, 1115, 1116, 1117, 1118, 1119, 1120, 1121, 1122, 1123, 1124, 1125, 1126, 1127, 1128, 1129,
+ 1130, 1131, 1132, 1133, 1134, 1135, 1136, 1137, 1138, 1139, 1140, 1141, 1142, 1143, 1144, 1145,
+ 1146, 1147, 1148, 1149, 1150, 1151, 1152, 1153, 1154, 1155, 1156, 1157, 1158, 1159, 1160, 1161,
+ 1162, 1163, 1164, 1165, 1166, 1167, 1168, 1169, 1170, 1171, 1172, 1173, 1174, 1175, 1176, 1177,
+ 1178, 1179, 1180, 1181, 1182, 1183, 1184, 1185, 1186, 1187, 1188, 1189, 1190, 1191, 1193, 1194,
+ 1195, 1196, 1197, 1198, 1199, 1200, 1201, 1202, 1203, 1204, 1205, 1206, 1207, 1208, 1209, 1210,
+ 1211, 1212, 1213, 1214, 1215, 1216, 1217, 1218, 1219, 1220, 1221, 1223, 1224, 1225, 1226, 1227,
+ 1228, 1229, 1230, 1231, 1232, 1233, 1234, 1235, 1236, 1237, 1238, 1239, 1240, 1241, 1242, 1243,
+ 1245, 1246, 1247, 1248, 1249, 1250, 1251, 1252, 1253, 1254, 1255, 1256, 1257, 1258, 1259, 1260,
+ 1261, 1262, 1264, 1265, 1266, 1267, 1268, 1269, 1270, 1271, 1272, 1273, 1274, 1275, 1276, 1277,
+ 1278, 1280, 1281, 1282, 1283, 1284, 1285, 1286, 1287, 1288, 1289, 1290, 1291, 1292, 1293, 1295,
+ 1296, 1297, 1298, 1299, 1300, 1301, 1302, 1303, 1304, 1305, 1306, 1307, 1309, 1310, 1311, 1312,
+ 1313, 1314, 1315, 1316, 1317, 1318, 1319, 1320, 1322, 1323, 1324, 1325, 1326, 1327, 1328, 1329,
+ 1330, 1331, 1332, 1334, 1335, 1336, 1337, 1338, 1339, 1340, 1341, 1342, 1343, 1345, 1346, 1347,
+ 1348, 1349, 1350, 1351, 1352, 1353, 1354, 1356, 1357, 1358, 1359, 1360, 1361, 1362, 1363, 1364,
+ 1365, 1367, 1368, 1369, 1370, 1371, 1372, 1373, 1374, 1375, 1377, 1378, 1379, 1380, 1381, 1382,
+ 1383, 1384, 1385, 1387, 1388, 1389, 1390, 1391, 1392, 1393, 1394, 1396, 1397, 1398, 1399, 1400,
+ 1401, 1402, 1403, 1405, 1406, 1407, 1408, 1409, 1410, 1411, 1412, 1414, 1415, 1416, 1417, 1418,
+ 1419, 1420, 1421, 1423, 1424, 1425, 1426, 1427, 1428, 1429, 1431, 1432, 1433, 1434, 1435, 1436,
+ 1437, 1439, 1440, 1441, 1442, 1443, 1444, 1445, 1446, 1448, 1449, 1450, 1451, 1452, 1453, 1455,
+ 1456, 1457, 1458, 1459, 1460, 1461, 1463, 1464, 1465, 1466, 1467, 1468, 1469, 1471, 1472, 1473,
+ 1474, 1475, 1476, 1478, 1479, 1480, 1481, 1482, 1483, 1484, 1486, 1487, 1488, 1489, 1490, 1491,
+ 1493, 1494, 1495, 1496, 1497, 1498, 1500, 1501, 1502, 1503, 1504, 1505, 1507, 1508, 1509, 1510,
+ 1511, 1512, 1514, 1515, 1516, 1517, 1518, 1519, 1521, 1522, 1523, 1524, 1525, 1527, 1528, 1529,
+ 1530, 1531, 1532, 1534, 1535, 1536, 1537, 1538, 1540, 1541, 1542, 1543, 1544, 1545, 1547, 1548,
+ 1549, 1550, 1551, 1553, 1554, 1555, 1556, 1557, 1559, 1560, 1561, 1562, 1563, 1564, 1566, 1567,
+ 1568, 1569, 1570, 1572, 1573, 1574, 1575, 1576, 1578, 1579, 1580, 1581, 1582, 1584, 1585, 1586,
+ 1587, 1588, 1590, 1591, 1592, 1593, 1594, 1596, 1597, 1598, 1599, 1601, 1602, 1603, 1604, 1605,
+ 1607, 1608, 1609, 1610, 1611, 1613, 1614, 1615, 1616, 1617, 1619, 1620, 1621, 1622, 1624, 1625,
+ 1626, 1627, 1628, 1630, 1631, 1632, 1633, 1635, 1636, 1637, 1638, 1639, 1641, 1642, 1643, 1644,
+ 1646, 1647, 1648, 1649, 1650, 1652, 1653, 1654, 1655, 1657, 1658, 1659, 1660, 1662, 1663, 1664,
+ 1665, 1667, 1668, 1669, 1670, 1671, 1673, 1674, 1675, 1676, 1678, 1679, 1680, 1681, 1683, 1684,
+ 1685, 1686, 1688, 1689, 1690, 1691, 1693, 1694, 1695, 1696, 1698, 1699, 1700, 1701, 1703, 1704,
+ 1705, 1706, 1708, 1709, 1710, 1711, 1713, 1714, 1715, 1716, 1718, 1719, 1720, 1721, 1723, 1724,
+ 1725, 1726, 1728, 1729, 1730, 1731, 1733, 1734, 1735, 1737, 1738, 1739, 1740, 1742, 1743, 1744,
+ 1745, 1747, 1748, 1749, 1750, 1752, 1753, 1754, 1756, 1757, 1758, 1759, 1761, 1762, 1763, 1764,
+ 1766, 1767, 1768, 1770, 1771, 1772, 1773, 1775, 1776, 1777, 1778, 1780, 1781, 1782, 1784, 1785,
+ 1786, 1787, 1789, 1790, 1791, 1793, 1794, 1795, 1796, 1798, 1799, 1800, 1802, 1803, 1804, 1806,
+ 1807, 1808, 1809, 1811, 1812, 1813, 1815, 1816, 1817, 1818, 1820, 1821, 1822, 1824, 1825, 1826,
+ 1828, 1829, 1830, 1831, 1833, 1834, 1835, 1837, 1838, 1839, 1841, 1842, 1843, 1844, 1846, 1847,
+ 1848, 1850, 1851, 1852, 1854, 1855, 1856, 1858, 1859, 1860, 1862, 1863, 1864, 1865, 1867, 1868,
+ 1869, 1871, 1872, 1873, 1875, 1876, 1877, 1879, 1880, 1881, 1883, 1884, 1885, 1887, 1888, 1889,
+ 1891, 1892, 1893, 1894, 1896, 1897, 1898, 1900, 1901, 1902, 1904, 1905, 1906, 1908, 1909, 1910,
+ 1912, 1913, 1914, 1916, 1917, 1918, 1920, 1921, 1922, 1924, 1925, 1926, 1928, 1929, 1930, 1932,
+ 1933, 1935, 1936, 1937, 1939, 1940, 1941, 1943, 1944, 1945, 1947, 1948, 1949, 1951, 1952, 1953,
+ 1955, 1956, 1957, 1959, 1960, 1961, 1963, 1964, 1965, 1967, 1968, 1970, 1971, 1972, 1974, 1975,
+ 1976, 1978, 1979, 1980, 1982, 1983, 1984, 1986, 1987, 1989, 1990, 1991, 1993, 1994, 1995, 1997,
+ 1998, 1999, 2001, 2002, 2004, 2005, 2006, 2008, 2009, 2010, 2012, 2013, 2015, 2016, 2017, 2019,
+ 2020, 2021, 2023, 2024, 2026, 2027, 2028, 2030, 2031, 2032, 2034, 2035, 2037, 2038, 2039, 2041,
+ 2042, 2043, 2045, 2046, 2048, 2049, 2050, 2052, 2053, 2055, 2056, 2057, 2059, 2060, 2061, 2063,
+ 2064, 2066, 2067, 2068, 2070, 2071, 2073, 2074, 2075, 2077, 2078, 2080, 2081, 2082, 2084, 2085,
+ 2087, 2088, 2089, 2091, 2092, 2094, 2095, 2096, 2098, 2099, 2101, 2102, 2103, 2105, 2106, 2108,
+ 2109, 2110, 2112, 2113, 2115, 2116, 2117, 2119, 2120, 2122, 2123, 2124, 2126, 2127, 2129, 2130,
+ 2132, 2133, 2134, 2136, 2137, 2139, 2140, 2141, 2143, 2144, 2146, 2147, 2149, 2150, 2151, 2153,
+ 2154, 2156, 2157, 2159, 2160, 2161, 2163, 2164, 2166, 2167, 2169, 2170, 2171, 2173, 2174, 2176,
+ 2177, 2179, 2180, 2181, 2183, 2184, 2186, 2187, 2189, 2190, 2191, 2193, 2194, 2196, 2197, 2199,
+ 2200, 2202, 2203, 2204, 2206, 2207, 2209, 2210, 2212, 2213, 2214, 2216, 2217, 2219, 2220, 2222,
+ 2223, 2225, 2226, 2228, 2229, 2230, 2232, 2233, 2235, 2236, 2238, 2239, 2241, 2242, 2243, 2245,
+ 2246, 2248, 2249, 2251, 2252, 2254, 2255, 2257, 2258, 2260, 2261, 2262, 2264, 2265, 2267, 2268,
+ 2270, 2271, 2273, 2274, 2276, 2277, 2279, 2280, 2282, 2283, 2284, 2286, 2287, 2289, 2290, 2292,
+ 2293, 2295, 2296, 2298, 2299, 2301, 2302, 2304, 2305, 2307, 2308, 2310, 2311, 2312, 2314, 2315,
+ 2317, 2318, 2320, 2321, 2323, 2324, 2326, 2327, 2329, 2330, 2332, 2333, 2335, 2336, 2338, 2339,
+ 2341, 2342, 2344, 2345, 2347, 2348, 2350, 2351, 2353, 2354, 2356, 2357, 2359, 2360, 2362, 2363,
+ 2365, 2366, 2368, 2369, 2371, 2372, 2374, 2375, 2377, 2378, 2380, 2381, 2383, 2384, 2386, 2387,
+ 2389, 2390, 2392, 2393, 2395, 2396, 2398, 2399, 2401, 2402, 2404, 2405, 2407, 2408, 2410, 2411,
+ 2413, 2414, 2416, 2417, 2419, 2420, 2422, 2423, 2425, 2426, 2428, 2429, 2431, 2433, 2434, 2436,
+ 2437, 2439, 2440, 2442, 2443, 2445, 2446, 2448, 2449, 2451, 2452, 2454, 2455, 2457, 2458, 2460,
+ 2462, 2463, 2465, 2466, 2468, 2469, 2471, 2472, 2474, 2475, 2477, 2478, 2480, 2481, 2483, 2485,
+ 2486, 2488, 2489, 2491, 2492, 2494, 2495, 2497, 2498, 2500, 2502, 2503, 2505, 2506, 2508, 2509,
+ 2511, 2512, 2514, 2515, 2517, 2519, 2520, 2522, 2523, 2525, 2526, 2528, 2529, 2531, 2533, 2534,
+ 2536, 2537, 2539, 2540, 2542, 2543, 2545, 2547, 2548, 2550, 2551, 2553, 2554, 2556, 2557, 2559,
+ 2561, 2562, 2564, 2565, 2567, 2568, 2570, 2572, 2573, 2575, 2576, 2578, 2579, 2581, 2583, 2584,
+ 2586, 2587, 2589, 2590, 2592, 2594, 2595, 2597, 2598, 2600, 2601, 2603, 2605, 2606, 2608, 2609,
+ 2611, 2613, 2614, 2616, 2617, 2619, 2620, 2622, 2624, 2625, 2627, 2628, 2630, 2632, 2633, 2635,
+ 2636, 2638, 2640, 2641, 2643, 2644, 2646, 2647, 2649, 2651, 2652, 2654, 2655, 2657, 2659, 2660,
+ 2662, 2663, 2665, 2667, 2668, 2670, 2671, 2673, 2675, 2676, 2678, 2679, 2681, 2683, 2684, 2686,
+ 2687, 2689, 2691, 2692, 2694, 2696, 2697, 2699, 2700, 2702, 2704, 2705, 2707, 2708, 2710, 2712,
+ 2713, 2715, 2716, 2718, 2720, 2721, 2723, 2725, 2726, 2728, 2729, 2731, 2733, 2734, 2736, 2738,
+ 2739, 2741, 2742, 2744, 2746, 2747, 2749, 2751, 2752, 2754, 2755, 2757, 2759, 2760, 2762, 2764,
+ 2765, 2767, 2769, 2770, 2772, 2773, 2775, 2777, 2778, 2780, 2782, 2783, 2785, 2787, 2788, 2790,
+ 2791, 2793, 2795, 2796, 2798, 2800, 2801, 2803, 2805, 2806, 2808, 2810, 2811, 2813, 2814, 2816,
+ 2818, 2819, 2821, 2823, 2824, 2826, 2828, 2829, 2831, 2833, 2834, 2836, 2838, 2839, 2841, 2843,
+ 2844, 2846, 2848, 2849, 2851, 2853, 2854, 2856, 2857, 2859, 2861, 2862, 2864, 2866, 2867, 2869,
+ 2871, 2872, 2874, 2876, 2877, 2879, 2881, 2882, 2884, 2886, 2888, 2889, 2891, 2893, 2894, 2896,
+ 2898, 2899, 2901, 2903, 2904, 2906, 2908, 2909, 2911, 2913, 2914, 2916, 2918, 2919, 2921, 2923,
+ 2924, 2926, 2928, 2929, 2931, 2933, 2935, 2936, 2938, 2940, 2941, 2943, 2945, 2946, 2948, 2950,
+ 2951, 2953, 2955, 2956, 2958, 2960, 2962, 2963, 2965, 2967, 2968, 2970, 2972, 2973, 2975, 2977,
+ 2979, 2980, 2982, 2984, 2985, 2987, 2989, 2990, 2992, 2994, 2996, 2997, 2999, 3001, 3002, 3004,
+ 3006, 3008, 3009, 3011, 3013, 3014, 3016, 3018, 3020, 3021, 3023, 3025, 3026, 3028, 3030, 3032,
+ 3033, 3035, 3037, 3038, 3040, 3042, 3044, 3045, 3047, 3049, 3050, 3052, 3054, 3056, 3057, 3059,
+ 3061, 3063, 3064, 3066, 3068, 3069, 3071, 3073, 3075, 3076, 3078, 3080, 3082, 3083, 3085, 3087,
+ 3089, 3090, 3092, 3094, 3095, 3097, 3099, 3101, 3102, 3104, 3106, 3108, 3109, 3111, 3113, 3115,
+ 3116, 3118, 3120, 3122, 3123, 3125, 3127, 3129, 3130, 3132, 3134, 3136, 3137, 3139, 3141, 3143,
+ 3144, 3146, 3148, 3150, 3151, 3153, 3155, 3157, 3158, 3160, 3162, 3164, 3165, 3167, 3169, 3171,
+ 3172, 3174, 3176, 3178, 3179, 3181, 3183, 3185, 3187, 3188, 3190, 3192, 3194, 3195, 3197, 3199,
+ 3201, 3202, 3204, 3206, 3208, 3209, 3211, 3213, 3215, 3217, 3218, 3220, 3222, 3224, 3225, 3227,
+ 3229, 3231, 3233, 3234, 3236, 3238, 3240, 3241, 3243, 3245, 3247, 3249, 3250, 3252, 3254, 3256,
+ 3258, 3259, 3261, 3263, 3265, 3266, 3268, 3270, 3272, 3274, 3275, 3277, 3279, 3281, 3283, 3284,
+ 3286, 3288, 3290, 3292, 3293, 3295, 3297, 3299, 3301, 3302, 3304, 3306, 3308, 3310, 3311, 3313,
+ 3315, 3317, 3319, 3320, 3322, 3324, 3326, 3328, 3329, 3331, 3333, 3335, 3337, 3338, 3340, 3342,
+ 3344, 3346, 3348, 3349, 3351, 3353, 3355, 3357, 3358, 3360, 3362, 3364, 3366, 3368, 3369, 3371,
+ 3373, 3375, 3377, 3378, 3380, 3382, 3384, 3386, 3388, 3389, 3391, 3393, 3395, 3397, 3399, 3400,
+ 3402, 3404, 3406, 3408, 3410, 3411, 3413, 3415, 3417, 3419, 3421, 3422, 3424, 3426, 3428, 3430,
+ 3432, 3433, 3435, 3437, 3439, 3441, 3443, 3444, 3446, 3448, 3450, 3452, 3454, 3455, 3457, 3459,
+ 3461, 3463, 3465, 3467, 3468, 3470, 3472, 3474, 3476, 3478, 3480, 3481, 3483, 3485, 3487, 3489,
+ 3491, 3492, 3494, 3496, 3498, 3500, 3502, 3504, 3506, 3507, 3509, 3511, 3513, 3515, 3517, 3519,
+ 3520, 3522, 3524, 3526, 3528, 3530, 3532, 3533, 3535, 3537, 3539, 3541, 3543, 3545, 3547, 3548,
+ 3550, 3552, 3554, 3556, 3558, 3560, 3562, 3563, 3565, 3567, 3569, 3571, 3573, 3575, 3577, 3578,
+ 3580, 3582, 3584, 3586, 3588, 3590, 3592, 3594, 3595, 3597, 3599, 3601, 3603, 3605, 3607, 3609,
+ 3611, 3612, 3614, 3616, 3618, 3620, 3622, 3624, 3626, 3628, 3629, 3631, 3633, 3635, 3637, 3639,
+ 3641, 3643, 3645, 3647, 3648, 3650, 3652, 3654, 3656, 3658, 3660, 3662, 3664, 3666, 3667, 3669,
+ 3671, 3673, 3675, 3677, 3679, 3681, 3683, 3685, 3687, 3688, 3690, 3692, 3694, 3696, 3698, 3700,
+ 3702, 3704, 3706, 3708, 3710, 3711, 3713, 3715, 3717, 3719, 3721, 3723, 3725, 3727, 3729, 3731,
+ 3733, 3735, 3736, 3738, 3740, 3742, 3744, 3746, 3748, 3750, 3752, 3754, 3756, 3758, 3760, 3762,
+ 3764, 3765, 3767, 3769, 3771, 3773, 3775, 3777, 3779, 3781, 3783, 3785, 3787, 3789, 3791, 3793,
+ 3795, 3796, 3798, 3800, 3802, 3804, 3806, 3808, 3810, 3812, 3814, 3816, 3818, 3820, 3822, 3824,
+ 3826, 3828, 3830, 3832, 3833, 3835, 3837, 3839, 3841, 3843, 3845, 3847, 3849, 3851, 3853, 3855,
+ 3857, 3859, 3861, 3863, 3865, 3867, 3869, 3871, 3873, 3875, 3877, 3879, 3881, 3883, 3884, 3886,
+ 3888, 3890, 3892, 3894, 3896, 3898, 3900, 3902, 3904, 3906, 3908, 3910, 3912, 3914, 3916, 3918,
+ 3920, 3922, 3924, 3926, 3928, 3930, 3932, 3934, 3936, 3938, 3940, 3942, 3944, 3946, 3948, 3950,
+ 3952, 3954, 3956, 3958, 3960, 3962, 3964, 3966, 3968, 3970, 3972, 3974, 3976, 3978, 3980, 3982,
+ 3984, 3986, 3988, 3990, 3992, 3994, 3996, 3998, 4000, 4002, 4004, 4006, 4008, 4010, 4012, 4014,
+ 4016, 4018, 4020, 4022, 4024, 4026, 4028, 4030, 4032, 4034, 4036, 4038, 4040, 4042, 4044, 4046,
+ 4048, 4050, 4052, 4054, 4056, 4058, 4060, 4062, 4064, 4066, 4068, 4070, 4072, 4074, 4076, 4078,
+ 4080,
+};
+
+/* Generated table */
+const unsigned short tpg_linear_to_rec709[255 * 16 + 1] = {
+ 0, 5, 9, 14, 18, 22, 27, 32, 36, 41, 45, 50, 54, 59, 63, 68,
+ 72, 77, 81, 86, 90, 95, 99, 104, 108, 113, 117, 122, 126, 131, 135, 139,
+ 144, 149, 153, 158, 162, 167, 171, 176, 180, 185, 189, 194, 198, 203, 207, 212,
+ 216, 221, 225, 230, 234, 239, 243, 248, 252, 257, 261, 266, 270, 275, 279, 284,
+ 288, 293, 297, 302, 306, 311, 315, 320, 324, 328, 334, 338, 343, 347, 352, 356,
+ 360, 365, 369, 373, 377, 381, 386, 390, 394, 398, 402, 406, 410, 414, 418, 422,
+ 426, 430, 433, 437, 441, 445, 449, 452, 456, 460, 464, 467, 471, 475, 478, 482,
+ 485, 489, 492, 496, 499, 503, 506, 510, 513, 517, 520, 524, 527, 530, 534, 537,
+ 540, 544, 547, 550, 554, 557, 560, 563, 566, 570, 573, 576, 579, 582, 586, 589,
+ 592, 595, 598, 601, 604, 607, 610, 613, 616, 619, 622, 625, 628, 631, 634, 637,
+ 640, 643, 646, 649, 652, 655, 658, 660, 663, 666, 669, 672, 675, 677, 680, 683,
+ 686, 689, 691, 694, 697, 700, 702, 705, 708, 711, 713, 716, 719, 721, 724, 727,
+ 729, 732, 735, 737, 740, 743, 745, 748, 750, 753, 756, 758, 761, 763, 766, 768,
+ 771, 773, 776, 779, 781, 784, 786, 789, 791, 794, 796, 799, 801, 803, 806, 808,
+ 811, 813, 816, 818, 821, 823, 825, 828, 830, 833, 835, 837, 840, 842, 844, 847,
+ 849, 851, 854, 856, 858, 861, 863, 865, 868, 870, 872, 875, 877, 879, 881, 884,
+ 886, 888, 891, 893, 895, 897, 900, 902, 904, 906, 908, 911, 913, 915, 917, 919,
+ 922, 924, 926, 928, 930, 933, 935, 937, 939, 941, 943, 946, 948, 950, 952, 954,
+ 956, 958, 960, 963, 965, 967, 969, 971, 973, 975, 977, 979, 981, 984, 986, 988,
+ 990, 992, 994, 996, 998, 1000, 1002, 1004, 1006, 1008, 1010, 1012, 1014, 1016, 1018, 1020,
+ 1022, 1024, 1026, 1028, 1030, 1032, 1034, 1036, 1038, 1040, 1042, 1044, 1046, 1048, 1050, 1052,
+ 1054, 1056, 1058, 1060, 1062, 1064, 1066, 1068, 1069, 1071, 1073, 1075, 1077, 1079, 1081, 1083,
+ 1085, 1087, 1089, 1090, 1092, 1094, 1096, 1098, 1100, 1102, 1104, 1106, 1107, 1109, 1111, 1113,
+ 1115, 1117, 1119, 1120, 1122, 1124, 1126, 1128, 1130, 1131, 1133, 1135, 1137, 1139, 1141, 1142,
+ 1144, 1146, 1148, 1150, 1151, 1153, 1155, 1157, 1159, 1160, 1162, 1164, 1166, 1168, 1169, 1171,
+ 1173, 1175, 1176, 1178, 1180, 1182, 1184, 1185, 1187, 1189, 1191, 1192, 1194, 1196, 1198, 1199,
+ 1201, 1203, 1204, 1206, 1208, 1210, 1211, 1213, 1215, 1217, 1218, 1220, 1222, 1223, 1225, 1227,
+ 1228, 1230, 1232, 1234, 1235, 1237, 1239, 1240, 1242, 1244, 1245, 1247, 1249, 1250, 1252, 1254,
+ 1255, 1257, 1259, 1260, 1262, 1264, 1265, 1267, 1269, 1270, 1272, 1274, 1275, 1277, 1279, 1280,
+ 1282, 1283, 1285, 1287, 1288, 1290, 1292, 1293, 1295, 1296, 1298, 1300, 1301, 1303, 1305, 1306,
+ 1308, 1309, 1311, 1313, 1314, 1316, 1317, 1319, 1321, 1322, 1324, 1325, 1327, 1328, 1330, 1332,
+ 1333, 1335, 1336, 1338, 1339, 1341, 1343, 1344, 1346, 1347, 1349, 1350, 1352, 1354, 1355, 1357,
+ 1358, 1360, 1361, 1363, 1364, 1366, 1367, 1369, 1371, 1372, 1374, 1375, 1377, 1378, 1380, 1381,
+ 1383, 1384, 1386, 1387, 1389, 1390, 1392, 1393, 1395, 1396, 1398, 1399, 1401, 1402, 1404, 1405,
+ 1407, 1408, 1410, 1411, 1413, 1414, 1416, 1417, 1419, 1420, 1422, 1423, 1425, 1426, 1428, 1429,
+ 1431, 1432, 1434, 1435, 1437, 1438, 1440, 1441, 1442, 1444, 1445, 1447, 1448, 1450, 1451, 1453,
+ 1454, 1456, 1457, 1458, 1460, 1461, 1463, 1464, 1466, 1467, 1469, 1470, 1471, 1473, 1474, 1476,
+ 1477, 1479, 1480, 1481, 1483, 1484, 1486, 1487, 1489, 1490, 1491, 1493, 1494, 1496, 1497, 1498,
+ 1500, 1501, 1503, 1504, 1505, 1507, 1508, 1510, 1511, 1512, 1514, 1515, 1517, 1518, 1519, 1521,
+ 1522, 1524, 1525, 1526, 1528, 1529, 1531, 1532, 1533, 1535, 1536, 1537, 1539, 1540, 1542, 1543,
+ 1544, 1546, 1547, 1548, 1550, 1551, 1553, 1554, 1555, 1557, 1558, 1559, 1561, 1562, 1563, 1565,
+ 1566, 1567, 1569, 1570, 1571, 1573, 1574, 1576, 1577, 1578, 1580, 1581, 1582, 1584, 1585, 1586,
+ 1588, 1589, 1590, 1592, 1593, 1594, 1596, 1597, 1598, 1600, 1601, 1602, 1603, 1605, 1606, 1607,
+ 1609, 1610, 1611, 1613, 1614, 1615, 1617, 1618, 1619, 1621, 1622, 1623, 1624, 1626, 1627, 1628,
+ 1630, 1631, 1632, 1634, 1635, 1636, 1637, 1639, 1640, 1641, 1643, 1644, 1645, 1647, 1648, 1649,
+ 1650, 1652, 1653, 1654, 1655, 1657, 1658, 1659, 1661, 1662, 1663, 1664, 1666, 1667, 1668, 1670,
+ 1671, 1672, 1673, 1675, 1676, 1677, 1678, 1680, 1681, 1682, 1683, 1685, 1686, 1687, 1688, 1690,
+ 1691, 1692, 1693, 1695, 1696, 1697, 1698, 1700, 1701, 1702, 1703, 1705, 1706, 1707, 1708, 1710,
+ 1711, 1712, 1713, 1715, 1716, 1717, 1718, 1720, 1721, 1722, 1723, 1724, 1726, 1727, 1728, 1729,
+ 1731, 1732, 1733, 1734, 1736, 1737, 1738, 1739, 1740, 1742, 1743, 1744, 1745, 1746, 1748, 1749,
+ 1750, 1751, 1753, 1754, 1755, 1756, 1757, 1759, 1760, 1761, 1762, 1763, 1765, 1766, 1767, 1768,
+ 1769, 1771, 1772, 1773, 1774, 1775, 1777, 1778, 1779, 1780, 1781, 1783, 1784, 1785, 1786, 1787,
+ 1788, 1790, 1791, 1792, 1793, 1794, 1796, 1797, 1798, 1799, 1800, 1801, 1803, 1804, 1805, 1806,
+ 1807, 1809, 1810, 1811, 1812, 1813, 1814, 1816, 1817, 1818, 1819, 1820, 1821, 1823, 1824, 1825,
+ 1826, 1827, 1828, 1829, 1831, 1832, 1833, 1834, 1835, 1836, 1838, 1839, 1840, 1841, 1842, 1843,
+ 1845, 1846, 1847, 1848, 1849, 1850, 1851, 1853, 1854, 1855, 1856, 1857, 1858, 1859, 1861, 1862,
+ 1863, 1864, 1865, 1866, 1867, 1868, 1870, 1871, 1872, 1873, 1874, 1875, 1876, 1878, 1879, 1880,
+ 1881, 1882, 1883, 1884, 1885, 1887, 1888, 1889, 1890, 1891, 1892, 1893, 1894, 1896, 1897, 1898,
+ 1899, 1900, 1901, 1902, 1903, 1904, 1906, 1907, 1908, 1909, 1910, 1911, 1912, 1913, 1914, 1916,
+ 1917, 1918, 1919, 1920, 1921, 1922, 1923, 1924, 1925, 1927, 1928, 1929, 1930, 1931, 1932, 1933,
+ 1934, 1935, 1936, 1938, 1939, 1940, 1941, 1942, 1943, 1944, 1945, 1946, 1947, 1948, 1950, 1951,
+ 1952, 1953, 1954, 1955, 1956, 1957, 1958, 1959, 1960, 1961, 1963, 1964, 1965, 1966, 1967, 1968,
+ 1969, 1970, 1971, 1972, 1973, 1974, 1975, 1977, 1978, 1979, 1980, 1981, 1982, 1983, 1984, 1985,
+ 1986, 1987, 1988, 1989, 1990, 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002,
+ 2003, 2004, 2005, 2006, 2007, 2008, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019,
+ 2020, 2021, 2022, 2023, 2024, 2025, 2026, 2027, 2028, 2029, 2031, 2032, 2033, 2034, 2035, 2036,
+ 2037, 2038, 2039, 2040, 2041, 2042, 2043, 2044, 2045, 2046, 2047, 2048, 2049, 2050, 2051, 2052,
+ 2053, 2054, 2055, 2056, 2057, 2058, 2060, 2061, 2062, 2063, 2064, 2065, 2066, 2067, 2068, 2069,
+ 2070, 2071, 2072, 2073, 2074, 2075, 2076, 2077, 2078, 2079, 2080, 2081, 2082, 2083, 2084, 2085,
+ 2086, 2087, 2088, 2089, 2090, 2091, 2092, 2093, 2094, 2095, 2096, 2097, 2098, 2099, 2100, 2101,
+ 2102, 2103, 2104, 2105, 2106, 2107, 2108, 2109, 2110, 2111, 2112, 2113, 2114, 2115, 2116, 2117,
+ 2118, 2119, 2120, 2121, 2122, 2123, 2124, 2125, 2126, 2127, 2128, 2129, 2130, 2131, 2132, 2133,
+ 2134, 2135, 2136, 2137, 2138, 2139, 2140, 2141, 2142, 2143, 2144, 2145, 2146, 2147, 2148, 2149,
+ 2150, 2151, 2152, 2153, 2154, 2155, 2156, 2157, 2158, 2159, 2160, 2161, 2162, 2163, 2164, 2165,
+ 2166, 2167, 2168, 2169, 2170, 2171, 2172, 2173, 2173, 2174, 2175, 2176, 2177, 2178, 2179, 2180,
+ 2181, 2182, 2183, 2184, 2185, 2186, 2187, 2188, 2189, 2190, 2191, 2192, 2193, 2194, 2195, 2196,
+ 2197, 2198, 2199, 2200, 2201, 2202, 2202, 2203, 2204, 2205, 2206, 2207, 2208, 2209, 2210, 2211,
+ 2212, 2213, 2214, 2215, 2216, 2217, 2218, 2219, 2220, 2221, 2222, 2223, 2224, 2224, 2225, 2226,
+ 2227, 2228, 2229, 2230, 2231, 2232, 2233, 2234, 2235, 2236, 2237, 2238, 2239, 2240, 2241, 2241,
+ 2242, 2243, 2244, 2245, 2246, 2247, 2248, 2249, 2250, 2251, 2252, 2253, 2254, 2255, 2256, 2257,
+ 2257, 2258, 2259, 2260, 2261, 2262, 2263, 2264, 2265, 2266, 2267, 2268, 2269, 2270, 2271, 2271,
+ 2272, 2273, 2274, 2275, 2276, 2277, 2278, 2279, 2280, 2281, 2282, 2283, 2283, 2284, 2285, 2286,
+ 2287, 2288, 2289, 2290, 2291, 2292, 2293, 2294, 2295, 2295, 2296, 2297, 2298, 2299, 2300, 2301,
+ 2302, 2303, 2304, 2305, 2306, 2306, 2307, 2308, 2309, 2310, 2311, 2312, 2313, 2314, 2315, 2316,
+ 2317, 2317, 2318, 2319, 2320, 2321, 2322, 2323, 2324, 2325, 2326, 2327, 2327, 2328, 2329, 2330,
+ 2331, 2332, 2333, 2334, 2335, 2336, 2336, 2337, 2338, 2339, 2340, 2341, 2342, 2343, 2344, 2345,
+ 2345, 2346, 2347, 2348, 2349, 2350, 2351, 2352, 2353, 2354, 2354, 2355, 2356, 2357, 2358, 2359,
+ 2360, 2361, 2362, 2363, 2363, 2364, 2365, 2366, 2367, 2368, 2369, 2370, 2371, 2371, 2372, 2373,
+ 2374, 2375, 2376, 2377, 2378, 2379, 2379, 2380, 2381, 2382, 2383, 2384, 2385, 2386, 2386, 2387,
+ 2388, 2389, 2390, 2391, 2392, 2393, 2394, 2394, 2395, 2396, 2397, 2398, 2399, 2400, 2401, 2401,
+ 2402, 2403, 2404, 2405, 2406, 2407, 2408, 2408, 2409, 2410, 2411, 2412, 2413, 2414, 2415, 2415,
+ 2416, 2417, 2418, 2419, 2420, 2421, 2422, 2422, 2423, 2424, 2425, 2426, 2427, 2428, 2428, 2429,
+ 2430, 2431, 2432, 2433, 2434, 2435, 2435, 2436, 2437, 2438, 2439, 2440, 2441, 2441, 2442, 2443,
+ 2444, 2445, 2446, 2447, 2447, 2448, 2449, 2450, 2451, 2452, 2453, 2453, 2454, 2455, 2456, 2457,
+ 2458, 2459, 2459, 2460, 2461, 2462, 2463, 2464, 2465, 2465, 2466, 2467, 2468, 2469, 2470, 2471,
+ 2471, 2472, 2473, 2474, 2475, 2476, 2477, 2477, 2478, 2479, 2480, 2481, 2482, 2482, 2483, 2484,
+ 2485, 2486, 2487, 2488, 2488, 2489, 2490, 2491, 2492, 2493, 2493, 2494, 2495, 2496, 2497, 2498,
+ 2499, 2499, 2500, 2501, 2502, 2503, 2504, 2504, 2505, 2506, 2507, 2508, 2509, 2509, 2510, 2511,
+ 2512, 2513, 2514, 2514, 2515, 2516, 2517, 2518, 2519, 2519, 2520, 2521, 2522, 2523, 2524, 2524,
+ 2525, 2526, 2527, 2528, 2529, 2529, 2530, 2531, 2532, 2533, 2534, 2534, 2535, 2536, 2537, 2538,
+ 2539, 2539, 2540, 2541, 2542, 2543, 2544, 2544, 2545, 2546, 2547, 2548, 2548, 2549, 2550, 2551,
+ 2552, 2553, 2553, 2554, 2555, 2556, 2557, 2558, 2558, 2559, 2560, 2561, 2562, 2562, 2563, 2564,
+ 2565, 2566, 2567, 2567, 2568, 2569, 2570, 2571, 2571, 2572, 2573, 2574, 2575, 2576, 2576, 2577,
+ 2578, 2579, 2580, 2580, 2581, 2582, 2583, 2584, 2584, 2585, 2586, 2587, 2588, 2589, 2589, 2590,
+ 2591, 2592, 2593, 2593, 2594, 2595, 2596, 2597, 2597, 2598, 2599, 2600, 2601, 2601, 2602, 2603,
+ 2604, 2605, 2605, 2606, 2607, 2608, 2609, 2610, 2610, 2611, 2612, 2613, 2614, 2614, 2615, 2616,
+ 2617, 2618, 2618, 2619, 2620, 2621, 2622, 2622, 2623, 2624, 2625, 2626, 2626, 2627, 2628, 2629,
+ 2630, 2630, 2631, 2632, 2633, 2634, 2634, 2635, 2636, 2637, 2637, 2638, 2639, 2640, 2641, 2641,
+ 2642, 2643, 2644, 2645, 2645, 2646, 2647, 2648, 2649, 2649, 2650, 2651, 2652, 2653, 2653, 2654,
+ 2655, 2656, 2656, 2657, 2658, 2659, 2660, 2660, 2661, 2662, 2663, 2664, 2664, 2665, 2666, 2667,
+ 2668, 2668, 2669, 2670, 2671, 2671, 2672, 2673, 2674, 2675, 2675, 2676, 2677, 2678, 2678, 2679,
+ 2680, 2681, 2682, 2682, 2683, 2684, 2685, 2686, 2686, 2687, 2688, 2689, 2689, 2690, 2691, 2692,
+ 2693, 2693, 2694, 2695, 2696, 2696, 2697, 2698, 2699, 2700, 2700, 2701, 2702, 2703, 2703, 2704,
+ 2705, 2706, 2706, 2707, 2708, 2709, 2710, 2710, 2711, 2712, 2713, 2713, 2714, 2715, 2716, 2717,
+ 2717, 2718, 2719, 2720, 2720, 2721, 2722, 2723, 2723, 2724, 2725, 2726, 2727, 2727, 2728, 2729,
+ 2730, 2730, 2731, 2732, 2733, 2733, 2734, 2735, 2736, 2736, 2737, 2738, 2739, 2740, 2740, 2741,
+ 2742, 2743, 2743, 2744, 2745, 2746, 2746, 2747, 2748, 2749, 2749, 2750, 2751, 2752, 2752, 2753,
+ 2754, 2755, 2755, 2756, 2757, 2758, 2759, 2759, 2760, 2761, 2762, 2762, 2763, 2764, 2765, 2765,
+ 2766, 2767, 2768, 2768, 2769, 2770, 2771, 2771, 2772, 2773, 2774, 2774, 2775, 2776, 2777, 2777,
+ 2778, 2779, 2780, 2780, 2781, 2782, 2783, 2783, 2784, 2785, 2786, 2786, 2787, 2788, 2789, 2789,
+ 2790, 2791, 2792, 2792, 2793, 2794, 2795, 2795, 2796, 2797, 2798, 2798, 2799, 2800, 2801, 2801,
+ 2802, 2803, 2804, 2804, 2805, 2806, 2807, 2807, 2808, 2809, 2810, 2810, 2811, 2812, 2813, 2813,
+ 2814, 2815, 2815, 2816, 2817, 2818, 2818, 2819, 2820, 2821, 2821, 2822, 2823, 2824, 2824, 2825,
+ 2826, 2827, 2827, 2828, 2829, 2830, 2830, 2831, 2832, 2832, 2833, 2834, 2835, 2835, 2836, 2837,
+ 2838, 2838, 2839, 2840, 2841, 2841, 2842, 2843, 2844, 2844, 2845, 2846, 2846, 2847, 2848, 2849,
+ 2849, 2850, 2851, 2852, 2852, 2853, 2854, 2855, 2855, 2856, 2857, 2857, 2858, 2859, 2860, 2860,
+ 2861, 2862, 2863, 2863, 2864, 2865, 2865, 2866, 2867, 2868, 2868, 2869, 2870, 2871, 2871, 2872,
+ 2873, 2873, 2874, 2875, 2876, 2876, 2877, 2878, 2879, 2879, 2880, 2881, 2881, 2882, 2883, 2884,
+ 2884, 2885, 2886, 2886, 2887, 2888, 2889, 2889, 2890, 2891, 2892, 2892, 2893, 2894, 2894, 2895,
+ 2896, 2897, 2897, 2898, 2899, 2899, 2900, 2901, 2902, 2902, 2903, 2904, 2904, 2905, 2906, 2907,
+ 2907, 2908, 2909, 2909, 2910, 2911, 2912, 2912, 2913, 2914, 2914, 2915, 2916, 2917, 2917, 2918,
+ 2919, 2919, 2920, 2921, 2922, 2922, 2923, 2924, 2924, 2925, 2926, 2927, 2927, 2928, 2929, 2929,
+ 2930, 2931, 2932, 2932, 2933, 2934, 2934, 2935, 2936, 2937, 2937, 2938, 2939, 2939, 2940, 2941,
+ 2941, 2942, 2943, 2944, 2944, 2945, 2946, 2946, 2947, 2948, 2949, 2949, 2950, 2951, 2951, 2952,
+ 2953, 2953, 2954, 2955, 2956, 2956, 2957, 2958, 2958, 2959, 2960, 2961, 2961, 2962, 2963, 2963,
+ 2964, 2965, 2965, 2966, 2967, 2968, 2968, 2969, 2970, 2970, 2971, 2972, 2972, 2973, 2974, 2975,
+ 2975, 2976, 2977, 2977, 2978, 2979, 2979, 2980, 2981, 2982, 2982, 2983, 2984, 2984, 2985, 2986,
+ 2986, 2987, 2988, 2988, 2989, 2990, 2991, 2991, 2992, 2993, 2993, 2994, 2995, 2995, 2996, 2997,
+ 2998, 2998, 2999, 3000, 3000, 3001, 3002, 3002, 3003, 3004, 3004, 3005, 3006, 3006, 3007, 3008,
+ 3009, 3009, 3010, 3011, 3011, 3012, 3013, 3013, 3014, 3015, 3015, 3016, 3017, 3018, 3018, 3019,
+ 3020, 3020, 3021, 3022, 3022, 3023, 3024, 3024, 3025, 3026, 3026, 3027, 3028, 3029, 3029, 3030,
+ 3031, 3031, 3032, 3033, 3033, 3034, 3035, 3035, 3036, 3037, 3037, 3038, 3039, 3039, 3040, 3041,
+ 3042, 3042, 3043, 3044, 3044, 3045, 3046, 3046, 3047, 3048, 3048, 3049, 3050, 3050, 3051, 3052,
+ 3052, 3053, 3054, 3054, 3055, 3056, 3056, 3057, 3058, 3059, 3059, 3060, 3061, 3061, 3062, 3063,
+ 3063, 3064, 3065, 3065, 3066, 3067, 3067, 3068, 3069, 3069, 3070, 3071, 3071, 3072, 3073, 3073,
+ 3074, 3075, 3075, 3076, 3077, 3077, 3078, 3079, 3079, 3080, 3081, 3081, 3082, 3083, 3084, 3084,
+ 3085, 3086, 3086, 3087, 3088, 3088, 3089, 3090, 3090, 3091, 3092, 3092, 3093, 3094, 3094, 3095,
+ 3096, 3096, 3097, 3098, 3098, 3099, 3100, 3100, 3101, 3102, 3102, 3103, 3104, 3104, 3105, 3106,
+ 3106, 3107, 3108, 3108, 3109, 3110, 3110, 3111, 3112, 3112, 3113, 3114, 3114, 3115, 3116, 3116,
+ 3117, 3118, 3118, 3119, 3120, 3120, 3121, 3122, 3122, 3123, 3124, 3124, 3125, 3126, 3126, 3127,
+ 3128, 3128, 3129, 3130, 3130, 3131, 3132, 3132, 3133, 3134, 3134, 3135, 3135, 3136, 3137, 3137,
+ 3138, 3139, 3139, 3140, 3141, 3141, 3142, 3143, 3143, 3144, 3145, 3145, 3146, 3147, 3147, 3148,
+ 3149, 3149, 3150, 3151, 3151, 3152, 3153, 3153, 3154, 3155, 3155, 3156, 3157, 3157, 3158, 3159,
+ 3159, 3160, 3160, 3161, 3162, 3162, 3163, 3164, 3164, 3165, 3166, 3166, 3167, 3168, 3168, 3169,
+ 3170, 3170, 3171, 3172, 3172, 3173, 3174, 3174, 3175, 3175, 3176, 3177, 3177, 3178, 3179, 3179,
+ 3180, 3181, 3181, 3182, 3183, 3183, 3184, 3185, 3185, 3186, 3187, 3187, 3188, 3188, 3189, 3190,
+ 3190, 3191, 3192, 3192, 3193, 3194, 3194, 3195, 3196, 3196, 3197, 3198, 3198, 3199, 3199, 3200,
+ 3201, 3201, 3202, 3203, 3203, 3204, 3205, 3205, 3206, 3207, 3207, 3208, 3209, 3209, 3210, 3210,
+ 3211, 3212, 3212, 3213, 3214, 3214, 3215, 3216, 3216, 3217, 3218, 3218, 3219, 3219, 3220, 3221,
+ 3221, 3222, 3223, 3223, 3224, 3225, 3225, 3226, 3227, 3227, 3228, 3228, 3229, 3230, 3230, 3231,
+ 3232, 3232, 3233, 3234, 3234, 3235, 3235, 3236, 3237, 3237, 3238, 3239, 3239, 3240, 3241, 3241,
+ 3242, 3242, 3243, 3244, 3244, 3245, 3246, 3246, 3247, 3248, 3248, 3249, 3249, 3250, 3251, 3251,
+ 3252, 3253, 3253, 3254, 3255, 3255, 3256, 3256, 3257, 3258, 3258, 3259, 3260, 3260, 3261, 3262,
+ 3262, 3263, 3263, 3264, 3265, 3265, 3266, 3267, 3267, 3268, 3268, 3269, 3270, 3270, 3271, 3272,
+ 3272, 3273, 3274, 3274, 3275, 3275, 3276, 3277, 3277, 3278, 3279, 3279, 3280, 3280, 3281, 3282,
+ 3282, 3283, 3284, 3284, 3285, 3285, 3286, 3287, 3287, 3288, 3289, 3289, 3290, 3290, 3291, 3292,
+ 3292, 3293, 3294, 3294, 3295, 3295, 3296, 3297, 3297, 3298, 3299, 3299, 3300, 3300, 3301, 3302,
+ 3302, 3303, 3304, 3304, 3305, 3305, 3306, 3307, 3307, 3308, 3309, 3309, 3310, 3310, 3311, 3312,
+ 3312, 3313, 3314, 3314, 3315, 3315, 3316, 3317, 3317, 3318, 3319, 3319, 3320, 3320, 3321, 3322,
+ 3322, 3323, 3323, 3324, 3325, 3325, 3326, 3327, 3327, 3328, 3328, 3329, 3330, 3330, 3331, 3332,
+ 3332, 3333, 3333, 3334, 3335, 3335, 3336, 3336, 3337, 3338, 3338, 3339, 3340, 3340, 3341, 3341,
+ 3342, 3343, 3343, 3344, 3345, 3345, 3346, 3346, 3347, 3348, 3348, 3349, 3349, 3350, 3351, 3351,
+ 3352, 3352, 3353, 3354, 3354, 3355, 3356, 3356, 3357, 3357, 3358, 3359, 3359, 3360, 3360, 3361,
+ 3362, 3362, 3363, 3364, 3364, 3365, 3365, 3366, 3367, 3367, 3368, 3368, 3369, 3370, 3370, 3371,
+ 3371, 3372, 3373, 3373, 3374, 3375, 3375, 3376, 3376, 3377, 3378, 3378, 3379, 3379, 3380, 3381,
+ 3381, 3382, 3382, 3383, 3384, 3384, 3385, 3385, 3386, 3387, 3387, 3388, 3389, 3389, 3390, 3390,
+ 3391, 3392, 3392, 3393, 3393, 3394, 3395, 3395, 3396, 3396, 3397, 3398, 3398, 3399, 3399, 3400,
+ 3401, 3401, 3402, 3402, 3403, 3404, 3404, 3405, 3405, 3406, 3407, 3407, 3408, 3408, 3409, 3410,
+ 3410, 3411, 3411, 3412, 3413, 3413, 3414, 3414, 3415, 3416, 3416, 3417, 3418, 3418, 3419, 3419,
+ 3420, 3421, 3421, 3422, 3422, 3423, 3424, 3424, 3425, 3425, 3426, 3427, 3427, 3428, 3428, 3429,
+ 3430, 3430, 3431, 3431, 3432, 3433, 3433, 3434, 3434, 3435, 3435, 3436, 3437, 3437, 3438, 3438,
+ 3439, 3440, 3440, 3441, 3441, 3442, 3443, 3443, 3444, 3444, 3445, 3446, 3446, 3447, 3447, 3448,
+ 3449, 3449, 3450, 3450, 3451, 3452, 3452, 3453, 3453, 3454, 3455, 3455, 3456, 3456, 3457, 3458,
+ 3458, 3459, 3459, 3460, 3461, 3461, 3462, 3462, 3463, 3463, 3464, 3465, 3465, 3466, 3466, 3467,
+ 3468, 3468, 3469, 3469, 3470, 3471, 3471, 3472, 3472, 3473, 3474, 3474, 3475, 3475, 3476, 3476,
+ 3477, 3478, 3478, 3479, 3479, 3480, 3481, 3481, 3482, 3482, 3483, 3484, 3484, 3485, 3485, 3486,
+ 3486, 3487, 3488, 3488, 3489, 3489, 3490, 3491, 3491, 3492, 3492, 3493, 3494, 3494, 3495, 3495,
+ 3496, 3496, 3497, 3498, 3498, 3499, 3499, 3500, 3501, 3501, 3502, 3502, 3503, 3504, 3504, 3505,
+ 3505, 3506, 3506, 3507, 3508, 3508, 3509, 3509, 3510, 3511, 3511, 3512, 3512, 3513, 3513, 3514,
+ 3515, 3515, 3516, 3516, 3517, 3518, 3518, 3519, 3519, 3520, 3520, 3521, 3522, 3522, 3523, 3523,
+ 3524, 3525, 3525, 3526, 3526, 3527, 3527, 3528, 3529, 3529, 3530, 3530, 3531, 3531, 3532, 3533,
+ 3533, 3534, 3534, 3535, 3536, 3536, 3537, 3537, 3538, 3538, 3539, 3540, 3540, 3541, 3541, 3542,
+ 3542, 3543, 3544, 3544, 3545, 3545, 3546, 3547, 3547, 3548, 3548, 3549, 3549, 3550, 3551, 3551,
+ 3552, 3552, 3553, 3553, 3554, 3555, 3555, 3556, 3556, 3557, 3557, 3558, 3559, 3559, 3560, 3560,
+ 3561, 3561, 3562, 3563, 3563, 3564, 3564, 3565, 3566, 3566, 3567, 3567, 3568, 3568, 3569, 3570,
+ 3570, 3571, 3571, 3572, 3572, 3573, 3574, 3574, 3575, 3575, 3576, 3576, 3577, 3578, 3578, 3579,
+ 3579, 3580, 3580, 3581, 3582, 3582, 3583, 3583, 3584, 3584, 3585, 3586, 3586, 3587, 3587, 3588,
+ 3588, 3589, 3590, 3590, 3591, 3591, 3592, 3592, 3593, 3594, 3594, 3595, 3595, 3596, 3596, 3597,
+ 3597, 3598, 3599, 3599, 3600, 3600, 3601, 3601, 3602, 3603, 3603, 3604, 3604, 3605, 3605, 3606,
+ 3607, 3607, 3608, 3608, 3609, 3609, 3610, 3611, 3611, 3612, 3612, 3613, 3613, 3614, 3615, 3615,
+ 3616, 3616, 3617, 3617, 3618, 3618, 3619, 3620, 3620, 3621, 3621, 3622, 3622, 3623, 3624, 3624,
+ 3625, 3625, 3626, 3626, 3627, 3627, 3628, 3629, 3629, 3630, 3630, 3631, 3631, 3632, 3633, 3633,
+ 3634, 3634, 3635, 3635, 3636, 3636, 3637, 3638, 3638, 3639, 3639, 3640, 3640, 3641, 3642, 3642,
+ 3643, 3643, 3644, 3644, 3645, 3645, 3646, 3647, 3647, 3648, 3648, 3649, 3649, 3650, 3650, 3651,
+ 3652, 3652, 3653, 3653, 3654, 3654, 3655, 3656, 3656, 3657, 3657, 3658, 3658, 3659, 3659, 3660,
+ 3661, 3661, 3662, 3662, 3663, 3663, 3664, 3664, 3665, 3666, 3666, 3667, 3667, 3668, 3668, 3669,
+ 3669, 3670, 3671, 3671, 3672, 3672, 3673, 3673, 3674, 3674, 3675, 3676, 3676, 3677, 3677, 3678,
+ 3678, 3679, 3679, 3680, 3681, 3681, 3682, 3682, 3683, 3683, 3684, 3684, 3685, 3686, 3686, 3687,
+ 3687, 3688, 3688, 3689, 3689, 3690, 3691, 3691, 3692, 3692, 3693, 3693, 3694, 3694, 3695, 3695,
+ 3696, 3697, 3697, 3698, 3698, 3699, 3699, 3700, 3700, 3701, 3702, 3702, 3703, 3703, 3704, 3704,
+ 3705, 3705, 3706, 3707, 3707, 3708, 3708, 3709, 3709, 3710, 3710, 3711, 3711, 3712, 3713, 3713,
+ 3714, 3714, 3715, 3715, 3716, 3716, 3717, 3717, 3718, 3719, 3719, 3720, 3720, 3721, 3721, 3722,
+ 3722, 3723, 3724, 3724, 3725, 3725, 3726, 3726, 3727, 3727, 3728, 3728, 3729, 3730, 3730, 3731,
+ 3731, 3732, 3732, 3733, 3733, 3734, 3734, 3735, 3736, 3736, 3737, 3737, 3738, 3738, 3739, 3739,
+ 3740, 3740, 3741, 3742, 3742, 3743, 3743, 3744, 3744, 3745, 3745, 3746, 3746, 3747, 3748, 3748,
+ 3749, 3749, 3750, 3750, 3751, 3751, 3752, 3752, 3753, 3753, 3754, 3755, 3755, 3756, 3756, 3757,
+ 3757, 3758, 3758, 3759, 3759, 3760, 3761, 3761, 3762, 3762, 3763, 3763, 3764, 3764, 3765, 3765,
+ 3766, 3766, 3767, 3768, 3768, 3769, 3769, 3770, 3770, 3771, 3771, 3772, 3772, 3773, 3773, 3774,
+ 3775, 3775, 3776, 3776, 3777, 3777, 3778, 3778, 3779, 3779, 3780, 3781, 3781, 3782, 3782, 3783,
+ 3783, 3784, 3784, 3785, 3785, 3786, 3786, 3787, 3787, 3788, 3789, 3789, 3790, 3790, 3791, 3791,
+ 3792, 3792, 3793, 3793, 3794, 3794, 3795, 3796, 3796, 3797, 3797, 3798, 3798, 3799, 3799, 3800,
+ 3800, 3801, 3801, 3802, 3802, 3803, 3804, 3804, 3805, 3805, 3806, 3806, 3807, 3807, 3808, 3808,
+ 3809, 3809, 3810, 3811, 3811, 3812, 3812, 3813, 3813, 3814, 3814, 3815, 3815, 3816, 3816, 3817,
+ 3817, 3818, 3819, 3819, 3820, 3820, 3821, 3821, 3822, 3822, 3823, 3823, 3824, 3824, 3825, 3825,
+ 3826, 3826, 3827, 3828, 3828, 3829, 3829, 3830, 3830, 3831, 3831, 3832, 3832, 3833, 3833, 3834,
+ 3834, 3835, 3835, 3836, 3837, 3837, 3838, 3838, 3839, 3839, 3840, 3840, 3841, 3841, 3842, 3842,
+ 3843, 3843, 3844, 3844, 3845, 3846, 3846, 3847, 3847, 3848, 3848, 3849, 3849, 3850, 3850, 3851,
+ 3851, 3852, 3852, 3853, 3853, 3854, 3855, 3855, 3856, 3856, 3857, 3857, 3858, 3858, 3859, 3859,
+ 3860, 3860, 3861, 3861, 3862, 3862, 3863, 3863, 3864, 3864, 3865, 3866, 3866, 3867, 3867, 3868,
+ 3868, 3869, 3869, 3870, 3870, 3871, 3871, 3872, 3872, 3873, 3873, 3874, 3874, 3875, 3876, 3876,
+ 3877, 3877, 3878, 3878, 3879, 3879, 3880, 3880, 3881, 3881, 3882, 3882, 3883, 3883, 3884, 3884,
+ 3885, 3885, 3886, 3886, 3887, 3888, 3888, 3889, 3889, 3890, 3890, 3891, 3891, 3892, 3892, 3893,
+ 3893, 3894, 3894, 3895, 3895, 3896, 3896, 3897, 3897, 3898, 3898, 3899, 3900, 3900, 3901, 3901,
+ 3902, 3902, 3903, 3903, 3904, 3904, 3905, 3905, 3906, 3906, 3907, 3907, 3908, 3908, 3909, 3909,
+ 3910, 3910, 3911, 3911, 3912, 3912, 3913, 3914, 3914, 3915, 3915, 3916, 3916, 3917, 3917, 3918,
+ 3918, 3919, 3919, 3920, 3920, 3921, 3921, 3922, 3922, 3923, 3923, 3924, 3924, 3925, 3925, 3926,
+ 3926, 3927, 3927, 3928, 3929, 3929, 3930, 3930, 3931, 3931, 3932, 3932, 3933, 3933, 3934, 3934,
+ 3935, 3935, 3936, 3936, 3937, 3937, 3938, 3938, 3939, 3939, 3940, 3940, 3941, 3941, 3942, 3942,
+ 3943, 3943, 3944, 3944, 3945, 3945, 3946, 3947, 3947, 3948, 3948, 3949, 3949, 3950, 3950, 3951,
+ 3951, 3952, 3952, 3953, 3953, 3954, 3954, 3955, 3955, 3956, 3956, 3957, 3957, 3958, 3958, 3959,
+ 3959, 3960, 3960, 3961, 3961, 3962, 3962, 3963, 3963, 3964, 3964, 3965, 3965, 3966, 3966, 3967,
+ 3967, 3968, 3969, 3969, 3970, 3970, 3971, 3971, 3972, 3972, 3973, 3973, 3974, 3974, 3975, 3975,
+ 3976, 3976, 3977, 3977, 3978, 3978, 3979, 3979, 3980, 3980, 3981, 3981, 3982, 3982, 3983, 3983,
+ 3984, 3984, 3985, 3985, 3986, 3986, 3987, 3987, 3988, 3988, 3989, 3989, 3990, 3990, 3991, 3991,
+ 3992, 3992, 3993, 3993, 3994, 3994, 3995, 3995, 3996, 3996, 3997, 3997, 3998, 3998, 3999, 3999,
+ 4000, 4001, 4001, 4002, 4002, 4003, 4003, 4004, 4004, 4005, 4005, 4006, 4006, 4007, 4007, 4008,
+ 4008, 4009, 4009, 4010, 4010, 4011, 4011, 4012, 4012, 4013, 4013, 4014, 4014, 4015, 4015, 4016,
+ 4016, 4017, 4017, 4018, 4018, 4019, 4019, 4020, 4020, 4021, 4021, 4022, 4022, 4023, 4023, 4024,
+ 4024, 4025, 4025, 4026, 4026, 4027, 4027, 4028, 4028, 4029, 4029, 4030, 4030, 4031, 4031, 4032,
+ 4032, 4033, 4033, 4034, 4034, 4035, 4035, 4036, 4036, 4037, 4037, 4038, 4038, 4039, 4039, 4040,
+ 4040, 4041, 4041, 4042, 4042, 4043, 4043, 4044, 4044, 4045, 4045, 4046, 4046, 4047, 4047, 4048,
+ 4048, 4049, 4049, 4050, 4050, 4051, 4051, 4052, 4052, 4053, 4053, 4054, 4054, 4055, 4055, 4056,
+ 4056, 4057, 4057, 4058, 4058, 4059, 4059, 4060, 4060, 4061, 4061, 4062, 4062, 4063, 4063, 4064,
+ 4064, 4065, 4065, 4066, 4066, 4067, 4067, 4068, 4068, 4069, 4069, 4070, 4070, 4071, 4071, 4072,
+ 4072, 4073, 4073, 4074, 4074, 4075, 4075, 4076, 4076, 4077, 4077, 4078, 4078, 4079, 4079, 4080,
+ 4080,
+};
+
+/* Generated table */
+const struct color16 tpg_csc_colors[V4L2_COLORSPACE_DCI_P3 + 1][V4L2_XFER_FUNC_SMPTE2084 + 1][TPG_COLOR_CSC_BLACK + 1] = {
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_709][0] = { 2939, 2939, 2939 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_709][1] = { 2953, 2963, 586 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_709][2] = { 0, 2967, 2937 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_709][3] = { 88, 2990, 575 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_709][4] = { 3016, 259, 2933 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_709][5] = { 3030, 405, 558 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_709][6] = { 478, 428, 2931 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_709][7] = { 547, 547, 547 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SRGB][0] = { 3056, 3056, 3056 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SRGB][1] = { 3068, 3077, 838 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SRGB][2] = { 0, 3081, 3053 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SRGB][3] = { 241, 3102, 828 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SRGB][4] = { 3126, 504, 3050 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SRGB][5] = { 3138, 657, 810 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SRGB][6] = { 731, 680, 3048 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SRGB][7] = { 800, 799, 800 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_ADOBERGB][0] = { 3033, 3033, 3033 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_ADOBERGB][1] = { 3046, 3054, 886 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_ADOBERGB][2] = { 0, 3058, 3031 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_ADOBERGB][3] = { 360, 3079, 877 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_ADOBERGB][4] = { 3103, 587, 3027 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_ADOBERGB][5] = { 3116, 723, 861 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_ADOBERGB][6] = { 789, 744, 3025 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_ADOBERGB][7] = { 851, 851, 851 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE240M][0] = { 2926, 2926, 2926 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE240M][1] = { 2941, 2950, 546 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE240M][2] = { 0, 2954, 2924 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE240M][3] = { 78, 2978, 536 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE240M][4] = { 3004, 230, 2920 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE240M][5] = { 3018, 363, 518 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE240M][6] = { 437, 387, 2918 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE240M][7] = { 507, 507, 507 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_NONE][0] = { 2125, 2125, 2125 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_NONE][1] = { 2145, 2159, 142 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_NONE][2] = { 0, 2164, 2122 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_NONE][3] = { 19, 2198, 138 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_NONE][4] = { 2236, 57, 2116 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_NONE][5] = { 2256, 90, 133 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_NONE][6] = { 110, 96, 2113 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_NONE][7] = { 130, 130, 130 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_DCI_P3][0] = { 3175, 3175, 3175 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_DCI_P3][1] = { 3186, 3194, 1121 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_DCI_P3][2] = { 0, 3197, 3173 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_DCI_P3][3] = { 523, 3216, 1112 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_DCI_P3][4] = { 3237, 792, 3169 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_DCI_P3][5] = { 3248, 944, 1094 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_DCI_P3][6] = { 1017, 967, 3168 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_DCI_P3][7] = { 1084, 1084, 1084 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE2084][0] = { 3798, 3798, 3798 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE2084][1] = { 3802, 3805, 2602 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE2084][2] = { 0, 3806, 3797 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE2084][3] = { 1780, 3812, 2592 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE2084][4] = { 3820, 2215, 3796 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE2084][5] = { 3824, 2409, 2574 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE2084][6] = { 2491, 2435, 3795 },
+ [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE2084][7] = { 2563, 2563, 2563 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_709][0] = { 2939, 2939, 2939 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_709][1] = { 2953, 2963, 586 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_709][2] = { 0, 2967, 2937 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_709][3] = { 88, 2990, 575 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_709][4] = { 3016, 259, 2933 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_709][5] = { 3030, 405, 558 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_709][6] = { 478, 428, 2931 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_709][7] = { 547, 547, 547 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SRGB][0] = { 3056, 3056, 3056 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SRGB][1] = { 3068, 3077, 838 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SRGB][2] = { 0, 3081, 3053 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SRGB][3] = { 241, 3102, 828 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SRGB][4] = { 3126, 504, 3050 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SRGB][5] = { 3138, 657, 810 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SRGB][6] = { 731, 680, 3048 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SRGB][7] = { 800, 799, 800 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_ADOBERGB][0] = { 3033, 3033, 3033 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_ADOBERGB][1] = { 3046, 3054, 886 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_ADOBERGB][2] = { 0, 3058, 3031 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_ADOBERGB][3] = { 360, 3079, 877 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_ADOBERGB][4] = { 3103, 587, 3027 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_ADOBERGB][5] = { 3116, 723, 861 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_ADOBERGB][6] = { 789, 744, 3025 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_ADOBERGB][7] = { 851, 851, 851 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE240M][0] = { 2926, 2926, 2926 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE240M][1] = { 2941, 2950, 546 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE240M][2] = { 0, 2954, 2924 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE240M][3] = { 78, 2978, 536 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE240M][4] = { 3004, 230, 2920 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE240M][5] = { 3018, 363, 518 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE240M][6] = { 437, 387, 2918 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE240M][7] = { 507, 507, 507 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_NONE][0] = { 2125, 2125, 2125 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_NONE][1] = { 2145, 2159, 142 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_NONE][2] = { 0, 2164, 2122 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_NONE][3] = { 19, 2198, 138 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_NONE][4] = { 2236, 57, 2116 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_NONE][5] = { 2256, 90, 133 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_NONE][6] = { 110, 96, 2113 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_NONE][7] = { 130, 130, 130 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_DCI_P3][0] = { 3175, 3175, 3175 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_DCI_P3][1] = { 3186, 3194, 1121 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_DCI_P3][2] = { 0, 3197, 3173 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_DCI_P3][3] = { 523, 3216, 1112 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_DCI_P3][4] = { 3237, 792, 3169 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_DCI_P3][5] = { 3248, 944, 1094 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_DCI_P3][6] = { 1017, 967, 3168 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_DCI_P3][7] = { 1084, 1084, 1084 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE2084][0] = { 3798, 3798, 3798 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE2084][1] = { 3802, 3805, 2602 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE2084][2] = { 0, 3806, 3797 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE2084][3] = { 1780, 3812, 2592 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE2084][4] = { 3820, 2215, 3796 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE2084][5] = { 3824, 2409, 2574 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE2084][6] = { 2491, 2435, 3795 },
+ [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE2084][7] = { 2563, 2563, 2563 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_709][0] = { 2939, 2939, 2939 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_709][1] = { 2939, 2939, 547 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_709][2] = { 547, 2939, 2939 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_709][3] = { 547, 2939, 547 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_709][4] = { 2939, 547, 2939 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_709][5] = { 2939, 547, 547 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_709][6] = { 547, 547, 2939 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_709][7] = { 547, 547, 547 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SRGB][0] = { 3056, 3056, 3056 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SRGB][1] = { 3056, 3056, 800 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SRGB][2] = { 800, 3056, 3056 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SRGB][3] = { 800, 3056, 800 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SRGB][4] = { 3056, 800, 3056 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SRGB][5] = { 3056, 800, 800 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SRGB][6] = { 800, 800, 3056 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SRGB][7] = { 800, 800, 800 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_ADOBERGB][0] = { 3033, 3033, 3033 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_ADOBERGB][1] = { 3033, 3033, 851 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_ADOBERGB][2] = { 851, 3033, 3033 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_ADOBERGB][3] = { 851, 3033, 851 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_ADOBERGB][4] = { 3033, 851, 3033 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_ADOBERGB][5] = { 3033, 851, 851 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_ADOBERGB][6] = { 851, 851, 3033 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_ADOBERGB][7] = { 851, 851, 851 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE240M][0] = { 2926, 2926, 2926 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE240M][1] = { 2926, 2926, 507 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE240M][2] = { 507, 2926, 2926 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE240M][3] = { 507, 2926, 507 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE240M][4] = { 2926, 507, 2926 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE240M][5] = { 2926, 507, 507 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE240M][6] = { 507, 507, 2926 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE240M][7] = { 507, 507, 507 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_NONE][0] = { 2125, 2125, 2125 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_NONE][1] = { 2125, 2125, 130 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_NONE][2] = { 130, 2125, 2125 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_NONE][3] = { 130, 2125, 130 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_NONE][4] = { 2125, 130, 2125 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_NONE][5] = { 2125, 130, 130 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_NONE][6] = { 130, 130, 2125 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_NONE][7] = { 130, 130, 130 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_DCI_P3][0] = { 3175, 3175, 3175 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_DCI_P3][1] = { 3175, 3175, 1084 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_DCI_P3][2] = { 1084, 3175, 3175 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_DCI_P3][3] = { 1084, 3175, 1084 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_DCI_P3][4] = { 3175, 1084, 3175 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_DCI_P3][5] = { 3175, 1084, 1084 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_DCI_P3][6] = { 1084, 1084, 3175 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_DCI_P3][7] = { 1084, 1084, 1084 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE2084][0] = { 3798, 3798, 3798 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE2084][1] = { 3798, 3798, 2563 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE2084][2] = { 2563, 3798, 3798 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE2084][3] = { 2563, 3798, 2563 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE2084][4] = { 3798, 2563, 3798 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE2084][5] = { 3798, 2563, 2563 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE2084][6] = { 2563, 2563, 3798 },
+ [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE2084][7] = { 2563, 2563, 2563 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_709][0] = { 2939, 2939, 2939 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_709][1] = { 2892, 3034, 910 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_709][2] = { 1715, 2916, 2914 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_709][3] = { 1631, 3012, 828 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_709][4] = { 2497, 119, 2867 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_709][5] = { 2440, 649, 657 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_709][6] = { 740, 0, 2841 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_709][7] = { 547, 547, 547 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SRGB][0] = { 3056, 3055, 3056 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SRGB][1] = { 3013, 3142, 1157 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SRGB][2] = { 1926, 3034, 3032 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SRGB][3] = { 1847, 3121, 1076 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SRGB][4] = { 2651, 304, 2990 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SRGB][5] = { 2599, 901, 909 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SRGB][6] = { 991, 0, 2966 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SRGB][7] = { 800, 799, 800 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_ADOBERGB][0] = { 3033, 3033, 3033 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_ADOBERGB][1] = { 2989, 3120, 1180 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_ADOBERGB][2] = { 1913, 3011, 3009 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_ADOBERGB][3] = { 1836, 3099, 1105 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_ADOBERGB][4] = { 2627, 413, 2966 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_ADOBERGB][5] = { 2576, 943, 951 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_ADOBERGB][6] = { 1026, 0, 2942 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_ADOBERGB][7] = { 851, 851, 851 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE240M][0] = { 2926, 2926, 2926 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE240M][1] = { 2879, 3022, 874 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE240M][2] = { 1688, 2903, 2901 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE240M][3] = { 1603, 2999, 791 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE240M][4] = { 2479, 106, 2853 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE240M][5] = { 2422, 610, 618 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE240M][6] = { 702, 0, 2827 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE240M][7] = { 507, 507, 507 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_NONE][0] = { 2125, 2125, 2125 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_NONE][1] = { 2059, 2262, 266 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_NONE][2] = { 771, 2092, 2089 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_NONE][3] = { 705, 2229, 231 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_NONE][4] = { 1550, 26, 2024 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_NONE][5] = { 1484, 163, 165 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_NONE][6] = { 196, 0, 1988 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_NONE][7] = { 130, 130, 130 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_DCI_P3][0] = { 3175, 3175, 3175 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_DCI_P3][1] = { 3136, 3251, 1429 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_DCI_P3][2] = { 2150, 3156, 3154 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_DCI_P3][3] = { 2077, 3233, 1352 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_DCI_P3][4] = { 2812, 589, 3116 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_DCI_P3][5] = { 2765, 1182, 1190 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_DCI_P3][6] = { 1270, 0, 3094 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_DCI_P3][7] = { 1084, 1084, 1084 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE2084][0] = { 3798, 3798, 3798 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE2084][1] = { 3784, 3825, 2879 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE2084][2] = { 3351, 3791, 3790 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE2084][3] = { 3311, 3819, 2815 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE2084][4] = { 3659, 1900, 3777 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE2084][5] = { 3640, 2662, 2669 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE2084][6] = { 2743, 0, 3769 },
+ [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE2084][7] = { 2563, 2563, 2563 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_709][0] = { 2939, 2939, 2939 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_709][1] = { 2939, 2939, 464 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_709][2] = { 786, 2939, 2939 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_709][3] = { 786, 2939, 464 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_709][4] = { 2879, 547, 2956 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_709][5] = { 2879, 547, 547 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_709][6] = { 547, 547, 2956 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_709][7] = { 547, 547, 547 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SRGB][0] = { 3056, 3056, 3056 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SRGB][1] = { 3056, 3056, 717 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SRGB][2] = { 1036, 3056, 3056 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SRGB][3] = { 1036, 3056, 717 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SRGB][4] = { 3001, 800, 3071 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SRGB][5] = { 3001, 800, 799 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SRGB][6] = { 800, 800, 3071 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SRGB][7] = { 800, 800, 799 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_ADOBERGB][0] = { 3033, 3033, 3033 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_ADOBERGB][1] = { 3033, 3033, 776 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_ADOBERGB][2] = { 1068, 3033, 3033 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_ADOBERGB][3] = { 1068, 3033, 776 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_ADOBERGB][4] = { 2977, 851, 3048 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_ADOBERGB][5] = { 2977, 851, 851 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_ADOBERGB][6] = { 851, 851, 3048 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_ADOBERGB][7] = { 851, 851, 851 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE240M][0] = { 2926, 2926, 2926 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE240M][1] = { 2926, 2926, 423 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE240M][2] = { 749, 2926, 2926 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE240M][3] = { 749, 2926, 423 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE240M][4] = { 2865, 507, 2943 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE240M][5] = { 2865, 507, 507 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE240M][6] = { 507, 507, 2943 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE240M][7] = { 507, 507, 507 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_NONE][0] = { 2125, 2125, 2125 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_NONE][1] = { 2125, 2125, 106 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_NONE][2] = { 214, 2125, 2125 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_NONE][3] = { 214, 2125, 106 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_NONE][4] = { 2041, 130, 2149 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_NONE][5] = { 2041, 130, 130 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_NONE][6] = { 130, 130, 2149 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_NONE][7] = { 130, 130, 130 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_DCI_P3][0] = { 3175, 3175, 3175 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_DCI_P3][1] = { 3175, 3175, 1003 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_DCI_P3][2] = { 1313, 3175, 3175 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_DCI_P3][3] = { 1313, 3175, 1003 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_DCI_P3][4] = { 3126, 1084, 3188 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_DCI_P3][5] = { 3126, 1084, 1084 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_DCI_P3][6] = { 1084, 1084, 3188 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_DCI_P3][7] = { 1084, 1084, 1084 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE2084][0] = { 3798, 3798, 3798 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE2084][1] = { 3798, 3798, 2476 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE2084][2] = { 2782, 3798, 3798 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE2084][3] = { 2782, 3798, 2476 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE2084][4] = { 3780, 2563, 3803 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE2084][5] = { 3780, 2563, 2563 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE2084][6] = { 2563, 2563, 3803 },
+ [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE2084][7] = { 2563, 2563, 2563 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_709][0] = { 2939, 2939, 2939 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_709][1] = { 2939, 2939, 547 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_709][2] = { 547, 2939, 2939 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_709][3] = { 547, 2939, 547 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_709][4] = { 2939, 547, 2939 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_709][5] = { 2939, 547, 547 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_709][6] = { 547, 547, 2939 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_709][7] = { 547, 547, 547 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SRGB][0] = { 3056, 3056, 3056 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SRGB][1] = { 3056, 3056, 800 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SRGB][2] = { 800, 3056, 3056 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SRGB][3] = { 800, 3056, 800 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SRGB][4] = { 3056, 800, 3056 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SRGB][5] = { 3056, 800, 800 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SRGB][6] = { 800, 800, 3056 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SRGB][7] = { 800, 800, 800 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_ADOBERGB][0] = { 3033, 3033, 3033 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_ADOBERGB][1] = { 3033, 3033, 851 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_ADOBERGB][2] = { 851, 3033, 3033 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_ADOBERGB][3] = { 851, 3033, 851 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_ADOBERGB][4] = { 3033, 851, 3033 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_ADOBERGB][5] = { 3033, 851, 851 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_ADOBERGB][6] = { 851, 851, 3033 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_ADOBERGB][7] = { 851, 851, 851 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE240M][0] = { 2926, 2926, 2926 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE240M][1] = { 2926, 2926, 507 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE240M][2] = { 507, 2926, 2926 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE240M][3] = { 507, 2926, 507 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE240M][4] = { 2926, 507, 2926 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE240M][5] = { 2926, 507, 507 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE240M][6] = { 507, 507, 2926 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE240M][7] = { 507, 507, 507 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_NONE][0] = { 2125, 2125, 2125 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_NONE][1] = { 2125, 2125, 130 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_NONE][2] = { 130, 2125, 2125 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_NONE][3] = { 130, 2125, 130 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_NONE][4] = { 2125, 130, 2125 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_NONE][5] = { 2125, 130, 130 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_NONE][6] = { 130, 130, 2125 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_NONE][7] = { 130, 130, 130 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_DCI_P3][0] = { 3175, 3175, 3175 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_DCI_P3][1] = { 3175, 3175, 1084 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_DCI_P3][2] = { 1084, 3175, 3175 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_DCI_P3][3] = { 1084, 3175, 1084 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_DCI_P3][4] = { 3175, 1084, 3175 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_DCI_P3][5] = { 3175, 1084, 1084 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_DCI_P3][6] = { 1084, 1084, 3175 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_DCI_P3][7] = { 1084, 1084, 1084 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE2084][0] = { 3798, 3798, 3798 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE2084][1] = { 3798, 3798, 2563 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE2084][2] = { 2563, 3798, 3798 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE2084][3] = { 2563, 3798, 2563 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE2084][4] = { 3798, 2563, 3798 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE2084][5] = { 3798, 2563, 2563 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE2084][6] = { 2563, 2563, 3798 },
+ [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE2084][7] = { 2563, 2563, 2563 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_709][0] = { 2939, 2939, 2939 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_709][1] = { 2939, 2939, 781 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_709][2] = { 1622, 2939, 2939 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_709][3] = { 1622, 2939, 781 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_709][4] = { 2502, 547, 2881 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_709][5] = { 2502, 547, 547 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_709][6] = { 547, 547, 2881 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_709][7] = { 547, 547, 547 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SRGB][0] = { 3056, 3056, 3056 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SRGB][1] = { 3056, 3056, 1031 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SRGB][2] = { 1838, 3056, 3056 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SRGB][3] = { 1838, 3056, 1031 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SRGB][4] = { 2657, 800, 3002 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SRGB][5] = { 2657, 800, 800 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SRGB][6] = { 800, 800, 3002 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SRGB][7] = { 800, 800, 800 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_ADOBERGB][0] = { 3033, 3033, 3033 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_ADOBERGB][1] = { 3033, 3033, 1063 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_ADOBERGB][2] = { 1828, 3033, 3033 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_ADOBERGB][3] = { 1828, 3033, 1063 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_ADOBERGB][4] = { 2633, 851, 2979 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_ADOBERGB][5] = { 2633, 851, 851 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_ADOBERGB][6] = { 851, 851, 2979 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_ADOBERGB][7] = { 851, 851, 851 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE240M][0] = { 2926, 2926, 2926 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE240M][1] = { 2926, 2926, 744 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE240M][2] = { 1594, 2926, 2926 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE240M][3] = { 1594, 2926, 744 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE240M][4] = { 2484, 507, 2867 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE240M][5] = { 2484, 507, 507 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE240M][6] = { 507, 507, 2867 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE240M][7] = { 507, 507, 507 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_NONE][0] = { 2125, 2125, 2125 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_NONE][1] = { 2125, 2125, 212 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_NONE][2] = { 698, 2125, 2125 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_NONE][3] = { 698, 2125, 212 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_NONE][4] = { 1557, 130, 2043 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_NONE][5] = { 1557, 130, 130 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_NONE][6] = { 130, 130, 2043 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_NONE][7] = { 130, 130, 130 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_DCI_P3][0] = { 3175, 3175, 3175 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_DCI_P3][1] = { 3175, 3175, 1308 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_DCI_P3][2] = { 2069, 3175, 3175 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_DCI_P3][3] = { 2069, 3175, 1308 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_DCI_P3][4] = { 2816, 1084, 3127 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_DCI_P3][5] = { 2816, 1084, 1084 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_DCI_P3][6] = { 1084, 1084, 3127 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_DCI_P3][7] = { 1084, 1084, 1084 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE2084][0] = { 3798, 3798, 3798 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE2084][1] = { 3798, 3798, 2778 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE2084][2] = { 3306, 3798, 3798 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE2084][3] = { 3306, 3798, 2778 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE2084][4] = { 3661, 2563, 3781 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE2084][5] = { 3661, 2563, 2563 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE2084][6] = { 2563, 2563, 3781 },
+ [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE2084][7] = { 2563, 2563, 2563 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_709][0] = { 2939, 2939, 2939 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_709][1] = { 2877, 2923, 1058 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_709][2] = { 1837, 2840, 2916 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_709][3] = { 1734, 2823, 993 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_709][4] = { 2427, 961, 2812 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_709][5] = { 2351, 912, 648 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_709][6] = { 792, 618, 2788 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_709][7] = { 547, 547, 547 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SRGB][0] = { 3056, 3056, 3056 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SRGB][1] = { 2999, 3041, 1301 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SRGB][2] = { 2040, 2965, 3034 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SRGB][3] = { 1944, 2950, 1238 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SRGB][4] = { 2587, 1207, 2940 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SRGB][5] = { 2517, 1159, 900 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SRGB][6] = { 1042, 870, 2917 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SRGB][7] = { 800, 800, 800 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_ADOBERGB][0] = { 3033, 3033, 3033 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_ADOBERGB][1] = { 2976, 3018, 1315 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_ADOBERGB][2] = { 2024, 2942, 3011 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_ADOBERGB][3] = { 1930, 2926, 1256 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_ADOBERGB][4] = { 2563, 1227, 2916 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_ADOBERGB][5] = { 2494, 1183, 943 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_ADOBERGB][6] = { 1073, 916, 2894 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_ADOBERGB][7] = { 851, 851, 851 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE240M][0] = { 2926, 2926, 2926 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE240M][1] = { 2864, 2910, 1024 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE240M][2] = { 1811, 2826, 2903 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE240M][3] = { 1707, 2809, 958 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE240M][4] = { 2408, 926, 2798 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE240M][5] = { 2331, 876, 609 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE240M][6] = { 755, 579, 2773 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE240M][7] = { 507, 507, 507 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_NONE][0] = { 2125, 2125, 2125 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_NONE][1] = { 2039, 2102, 338 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_NONE][2] = { 873, 1987, 2092 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_NONE][3] = { 787, 1965, 305 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_NONE][4] = { 1468, 290, 1949 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_NONE][5] = { 1382, 268, 162 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_NONE][6] = { 216, 152, 1917 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_NONE][7] = { 130, 130, 130 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_DCI_P3][0] = { 3175, 3175, 3175 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_DCI_P3][1] = { 3124, 3161, 1566 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_DCI_P3][2] = { 2255, 3094, 3156 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_DCI_P3][3] = { 2166, 3080, 1506 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_DCI_P3][4] = { 2754, 1477, 3071 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_DCI_P3][5] = { 2690, 1431, 1182 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_DCI_P3][6] = { 1318, 1153, 3051 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_DCI_P3][7] = { 1084, 1084, 1084 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE2084][0] = { 3798, 3798, 3798 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE2084][1] = { 3780, 3793, 2984 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE2084][2] = { 3406, 3768, 3791 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE2084][3] = { 3359, 3763, 2939 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE2084][4] = { 3636, 2916, 3760 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE2084][5] = { 3609, 2880, 2661 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE2084][6] = { 2786, 2633, 3753 },
+ [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE2084][7] = { 2563, 2563, 2563 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_709][0] = { 2939, 2939, 2939 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_709][1] = { 2936, 2934, 992 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_709][2] = { 1159, 2890, 2916 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_709][3] = { 1150, 2885, 921 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_709][4] = { 2751, 766, 2837 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_709][5] = { 2747, 747, 650 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_709][6] = { 563, 570, 2812 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_709][7] = { 547, 547, 547 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SRGB][0] = { 3056, 3056, 3055 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SRGB][1] = { 3052, 3051, 1237 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SRGB][2] = { 1397, 3011, 3034 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SRGB][3] = { 1389, 3006, 1168 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SRGB][4] = { 2884, 1016, 2962 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SRGB][5] = { 2880, 998, 902 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SRGB][6] = { 816, 823, 2940 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SRGB][7] = { 800, 800, 799 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_ADOBERGB][0] = { 3033, 3033, 3033 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_ADOBERGB][1] = { 3029, 3028, 1255 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_ADOBERGB][2] = { 1406, 2988, 3011 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_ADOBERGB][3] = { 1398, 2983, 1190 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_ADOBERGB][4] = { 2860, 1050, 2939 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_ADOBERGB][5] = { 2857, 1033, 945 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_ADOBERGB][6] = { 866, 873, 2916 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_ADOBERGB][7] = { 851, 851, 851 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE240M][0] = { 2926, 2926, 2926 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE240M][1] = { 2923, 2921, 957 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE240M][2] = { 1125, 2877, 2902 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE240M][3] = { 1116, 2871, 885 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE240M][4] = { 2736, 729, 2823 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE240M][5] = { 2732, 710, 611 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE240M][6] = { 523, 531, 2798 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE240M][7] = { 507, 507, 507 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_NONE][0] = { 2125, 2125, 2125 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_NONE][1] = { 2120, 2118, 305 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_NONE][2] = { 392, 2056, 2092 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_NONE][3] = { 387, 2049, 271 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_NONE][4] = { 1868, 206, 1983 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_NONE][5] = { 1863, 199, 163 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_NONE][6] = { 135, 137, 1950 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_NONE][7] = { 130, 130, 130 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_DCI_P3][0] = { 3175, 3175, 3175 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_DCI_P3][1] = { 3172, 3170, 1505 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_DCI_P3][2] = { 1657, 3135, 3155 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_DCI_P3][3] = { 1649, 3130, 1439 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_DCI_P3][4] = { 3021, 1294, 3091 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_DCI_P3][5] = { 3018, 1276, 1184 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_DCI_P3][6] = { 1100, 1107, 3071 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_DCI_P3][7] = { 1084, 1084, 1084 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE2084][0] = { 3798, 3798, 3798 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE2084][1] = { 3797, 3796, 2938 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE2084][2] = { 3049, 3783, 3791 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE2084][3] = { 3044, 3782, 2887 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE2084][4] = { 3741, 2765, 3768 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE2084][5] = { 3740, 2749, 2663 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE2084][6] = { 2580, 2587, 3760 },
+ [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE2084][7] = { 2563, 2563, 2563 },
+};
+
+#else
+
+/* This code generates the table above */
+
+#include <math.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+static const double rec709_to_ntsc1953[3][3] = {
+ /*
+ * This transform uses the Bradford method to compensate for
+ * the different whitepoints.
+ */
+ { 0.6785011, 0.2883441, 0.0331548 },
+ { 0.0165284, 1.0518725, -0.0684009 },
+ { 0.0179230, 0.0506096, 0.9314674 }
+};
+
+static const double rec709_to_ebu[3][3] = {
+ { 0.9578221, 0.0421779, -0.0000000 },
+ { -0.0000000, 1.0000000, 0.0000000 },
+ { -0.0000000, -0.0119367, 1.0119367 }
+};
+
+static const double rec709_to_170m[3][3] = {
+ { 1.0653640, -0.0553900, -0.0099740 },
+ { -0.0196361, 1.0363630, -0.0167269 },
+ { 0.0016327, 0.0044133, 0.9939540 },
+};
+
+static const double rec709_to_240m[3][3] = {
+ { 1.0653640, -0.0553900, -0.0099740 },
+ { -0.0196361, 1.0363630, -0.0167269 },
+ { 0.0016327, 0.0044133, 0.9939540 },
+};
+
+static const double rec709_to_adobergb[3][3] = {
+ { 0.7151627, 0.2848373, -0.0000000 },
+ { 0.0000000, 1.0000000, 0.0000000 },
+ { -0.0000000, 0.0411705, 0.9588295 },
+};
+
+static const double rec709_to_bt2020[3][3] = {
+ { 0.6274524, 0.3292485, 0.0432991 },
+ { 0.0691092, 0.9195311, 0.0113597 },
+ { 0.0163976, 0.0880301, 0.8955723 },
+};
+
+static const double rec709_to_dcip3[3][3] = {
+ /*
+ * This transform uses the Bradford method to compensate for
+ * the different whitepoints.
+ */
+ { 0.8686648, 0.1288456, 0.0024896 },
+ { 0.0345479, 0.9618084, 0.0036437 },
+ { 0.0167785, 0.0710559, 0.9121655 }
+};
+
+static void mult_matrix(double *r, double *g, double *b, const double m[3][3])
+{
+ double ir, ig, ib;
+
+ ir = m[0][0] * (*r) + m[0][1] * (*g) + m[0][2] * (*b);
+ ig = m[1][0] * (*r) + m[1][1] * (*g) + m[1][2] * (*b);
+ ib = m[2][0] * (*r) + m[2][1] * (*g) + m[2][2] * (*b);
+ *r = ir;
+ *g = ig;
+ *b = ib;
+}
+
+static double transfer_srgb_to_rgb(double v)
+{
+ if (v < -0.04045)
+ return pow((-v + 0.055) / 1.055, 2.4);
+ return (v <= 0.04045) ? v / 12.92 : pow((v + 0.055) / 1.055, 2.4);
+}
+
+static double transfer_rgb_to_srgb(double v)
+{
+ if (v <= -0.0031308)
+ return -1.055 * pow(-v, 1.0 / 2.4) + 0.055;
+ if (v <= 0.0031308)
+ return v * 12.92;
+ return 1.055 * pow(v, 1.0 / 2.4) - 0.055;
+}
+
+static double transfer_rgb_to_smpte240m(double v)
+{
+ return (v <= 0.0228) ? v * 4.0 : 1.1115 * pow(v, 0.45) - 0.1115;
+}
+
+static double transfer_rgb_to_rec709(double v)
+{
+ if (v <= -0.018)
+ return -1.099 * pow(-v, 0.45) + 0.099;
+ return (v < 0.018) ? v * 4.5 : 1.099 * pow(v, 0.45) - 0.099;
+}
+
+static double transfer_rec709_to_rgb(double v)
+{
+ return (v < 0.081) ? v / 4.5 : pow((v + 0.099) / 1.099, 1.0 / 0.45);
+}
+
+static double transfer_rgb_to_adobergb(double v)
+{
+ return pow(v, 1.0 / 2.19921875);
+}
+
+static double transfer_rgb_to_dcip3(double v)
+{
+ return pow(v, 1.0 / 2.6);
+}
+
+static double transfer_rgb_to_smpte2084(double v)
+{
+ const double m1 = (2610.0 / 4096.0) / 4.0;
+ const double m2 = 128.0 * 2523.0 / 4096.0;
+ const double c1 = 3424.0 / 4096.0;
+ const double c2 = 32.0 * 2413.0 / 4096.0;
+ const double c3 = 32.0 * 2392.0 / 4096.0;
+
+ v = pow(v, m1);
+ return pow((c1 + c2 * v) / (1 + c3 * v), m2);
+}
+
+static double transfer_srgb_to_rec709(double v)
+{
+ return transfer_rgb_to_rec709(transfer_srgb_to_rgb(v));
+}
+
+static void csc(enum v4l2_colorspace colorspace, enum v4l2_xfer_func xfer_func,
+ double *r, double *g, double *b)
+{
+ int clamp = 1;
+
+ *r = transfer_srgb_to_rgb(*r);
+ *g = transfer_srgb_to_rgb(*g);
+ *b = transfer_srgb_to_rgb(*b);
+
+ /* Convert the primaries of Rec. 709 Linear RGB */
+ switch (colorspace) {
+ case V4L2_COLORSPACE_SMPTE240M:
+ mult_matrix(r, g, b, rec709_to_240m);
+ break;
+ case V4L2_COLORSPACE_SMPTE170M:
+ mult_matrix(r, g, b, rec709_to_170m);
+ break;
+ case V4L2_COLORSPACE_470_SYSTEM_BG:
+ mult_matrix(r, g, b, rec709_to_ebu);
+ break;
+ case V4L2_COLORSPACE_470_SYSTEM_M:
+ mult_matrix(r, g, b, rec709_to_ntsc1953);
+ break;
+ case V4L2_COLORSPACE_ADOBERGB:
+ mult_matrix(r, g, b, rec709_to_adobergb);
+ break;
+ case V4L2_COLORSPACE_BT2020:
+ mult_matrix(r, g, b, rec709_to_bt2020);
+ break;
+ case V4L2_COLORSPACE_DCI_P3:
+ mult_matrix(r, g, b, rec709_to_dcip3);
+ break;
+ case V4L2_COLORSPACE_SRGB:
+ case V4L2_COLORSPACE_REC709:
+ break;
+ default:
+ break;
+ }
+
+ if (clamp) {
+ *r = ((*r) < 0) ? 0 : (((*r) > 1) ? 1 : (*r));
+ *g = ((*g) < 0) ? 0 : (((*g) > 1) ? 1 : (*g));
+ *b = ((*b) < 0) ? 0 : (((*b) > 1) ? 1 : (*b));
+ }
+
+ switch (xfer_func) {
+ case V4L2_XFER_FUNC_709:
+ *r = transfer_rgb_to_rec709(*r);
+ *g = transfer_rgb_to_rec709(*g);
+ *b = transfer_rgb_to_rec709(*b);
+ break;
+ case V4L2_XFER_FUNC_SRGB:
+ *r = transfer_rgb_to_srgb(*r);
+ *g = transfer_rgb_to_srgb(*g);
+ *b = transfer_rgb_to_srgb(*b);
+ break;
+ case V4L2_XFER_FUNC_ADOBERGB:
+ *r = transfer_rgb_to_adobergb(*r);
+ *g = transfer_rgb_to_adobergb(*g);
+ *b = transfer_rgb_to_adobergb(*b);
+ break;
+ case V4L2_XFER_FUNC_DCI_P3:
+ *r = transfer_rgb_to_dcip3(*r);
+ *g = transfer_rgb_to_dcip3(*g);
+ *b = transfer_rgb_to_dcip3(*b);
+ break;
+ case V4L2_XFER_FUNC_SMPTE2084:
+ *r = transfer_rgb_to_smpte2084(*r);
+ *g = transfer_rgb_to_smpte2084(*g);
+ *b = transfer_rgb_to_smpte2084(*b);
+ break;
+ case V4L2_XFER_FUNC_SMPTE240M:
+ *r = transfer_rgb_to_smpte240m(*r);
+ *g = transfer_rgb_to_smpte240m(*g);
+ *b = transfer_rgb_to_smpte240m(*b);
+ break;
+ case V4L2_XFER_FUNC_NONE:
+ break;
+ }
+}
+
+int main(int argc, char **argv)
+{
+ static const unsigned colorspaces[] = {
+ 0,
+ V4L2_COLORSPACE_SMPTE170M,
+ V4L2_COLORSPACE_SMPTE240M,
+ V4L2_COLORSPACE_REC709,
+ 0,
+ V4L2_COLORSPACE_470_SYSTEM_M,
+ V4L2_COLORSPACE_470_SYSTEM_BG,
+ 0,
+ V4L2_COLORSPACE_SRGB,
+ V4L2_COLORSPACE_ADOBERGB,
+ V4L2_COLORSPACE_BT2020,
+ 0,
+ V4L2_COLORSPACE_DCI_P3,
+ };
+ static const char * const colorspace_names[] = {
+ "",
+ "V4L2_COLORSPACE_SMPTE170M",
+ "V4L2_COLORSPACE_SMPTE240M",
+ "V4L2_COLORSPACE_REC709",
+ "",
+ "V4L2_COLORSPACE_470_SYSTEM_M",
+ "V4L2_COLORSPACE_470_SYSTEM_BG",
+ "",
+ "V4L2_COLORSPACE_SRGB",
+ "V4L2_COLORSPACE_ADOBERGB",
+ "V4L2_COLORSPACE_BT2020",
+ "",
+ "V4L2_COLORSPACE_DCI_P3",
+ };
+ static const char * const xfer_func_names[] = {
+ "",
+ "V4L2_XFER_FUNC_709",
+ "V4L2_XFER_FUNC_SRGB",
+ "V4L2_XFER_FUNC_ADOBERGB",
+ "V4L2_XFER_FUNC_SMPTE240M",
+ "V4L2_XFER_FUNC_NONE",
+ "V4L2_XFER_FUNC_DCI_P3",
+ "V4L2_XFER_FUNC_SMPTE2084",
+ };
+ int i;
+ int x;
+ int c;
+
+ printf("/* Generated table */\n");
+ printf("const unsigned short tpg_rec709_to_linear[255 * 16 + 1] = {");
+ for (i = 0; i <= 255 * 16; i++) {
+ if (i % 16 == 0)
+ printf("\n\t");
+ printf("%4d,%s",
+ (int)(0.5 + 16.0 * 255.0 *
+ transfer_rec709_to_rgb(i / (16.0 * 255.0))),
+ i % 16 == 15 || i == 255 * 16 ? "" : " ");
+ }
+ printf("\n};\n\n");
+
+ printf("/* Generated table */\n");
+ printf("const unsigned short tpg_linear_to_rec709[255 * 16 + 1] = {");
+ for (i = 0; i <= 255 * 16; i++) {
+ if (i % 16 == 0)
+ printf("\n\t");
+ printf("%4d,%s",
+ (int)(0.5 + 16.0 * 255.0 *
+ transfer_rgb_to_rec709(i / (16.0 * 255.0))),
+ i % 16 == 15 || i == 255 * 16 ? "" : " ");
+ }
+ printf("\n};\n\n");
+
+ printf("/* Generated table */\n");
+ printf("const struct color16 tpg_csc_colors[V4L2_COLORSPACE_DCI_P3 + 1][V4L2_XFER_FUNC_SMPTE2084 + 1][TPG_COLOR_CSC_BLACK + 1] = {\n");
+ for (c = 0; c <= V4L2_COLORSPACE_DCI_P3; c++) {
+ for (x = 1; x <= V4L2_XFER_FUNC_SMPTE2084; x++) {
+ for (i = 0; i <= TPG_COLOR_CSC_BLACK; i++) {
+ double r, g, b;
+
+ if (colorspaces[c] == 0)
+ continue;
+
+ r = tpg_colors[i].r / 255.0;
+ g = tpg_colors[i].g / 255.0;
+ b = tpg_colors[i].b / 255.0;
+
+ csc(c, x, &r, &g, &b);
+
+ printf("\t[%s][%s][%d] = { %d, %d, %d },\n",
+ colorspace_names[c],
+ xfer_func_names[x], i,
+ (int)(r * 4080), (int)(g * 4080), (int)(b * 4080));
+ }
+ }
+ }
+ printf("};\n\n");
+ return 0;
+}
+
+#endif
diff --git a/drivers/media/common/v4l2-tpg/v4l2-tpg-core.c b/drivers/media/common/v4l2-tpg/v4l2-tpg-core.c
new file mode 100644
index 0000000..cf1dadd
--- /dev/null
+++ b/drivers/media/common/v4l2-tpg/v4l2-tpg-core.c
@@ -0,0 +1,2335 @@
+/*
+ * v4l2-tpg-core.c - Test Pattern Generator
+ *
+ * Note: gen_twopix and tpg_gen_text are based on code from vivi.c. See the
+ * vivi.c source for the copyright information of those functions.
+ *
+ * Copyright 2014 Cisco Systems, Inc. and/or its affiliates. All rights reserved.
+ *
+ * This program is free software; you may redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <linux/module.h>
+#include <media/v4l2-tpg.h>
+
+/* Must remain in sync with enum tpg_pattern */
+const char * const tpg_pattern_strings[] = {
+ "75% Colorbar",
+ "100% Colorbar",
+ "CSC Colorbar",
+ "Horizontal 100% Colorbar",
+ "100% Color Squares",
+ "100% Black",
+ "100% White",
+ "100% Red",
+ "100% Green",
+ "100% Blue",
+ "16x16 Checkers",
+ "2x2 Checkers",
+ "1x1 Checkers",
+ "2x2 Red/Green Checkers",
+ "1x1 Red/Green Checkers",
+ "Alternating Hor Lines",
+ "Alternating Vert Lines",
+ "One Pixel Wide Cross",
+ "Two Pixels Wide Cross",
+ "Ten Pixels Wide Cross",
+ "Gray Ramp",
+ "Noise",
+ NULL
+};
+EXPORT_SYMBOL_GPL(tpg_pattern_strings);
+
+/* Must remain in sync with enum tpg_aspect */
+const char * const tpg_aspect_strings[] = {
+ "Source Width x Height",
+ "4x3",
+ "14x9",
+ "16x9",
+ "16x9 Anamorphic",
+ NULL
+};
+EXPORT_SYMBOL_GPL(tpg_aspect_strings);
+
+/*
+ * Sine table: sin[0] = 127 * sin(-180 degrees)
+ * sin[128] = 127 * sin(0 degrees)
+ * sin[256] = 127 * sin(180 degrees)
+ */
+static const s8 sin[257] = {
+ 0, -4, -7, -11, -13, -18, -20, -22, -26, -29, -33, -35, -37, -41, -43, -48,
+ -50, -52, -56, -58, -62, -63, -65, -69, -71, -75, -76, -78, -82, -83, -87, -88,
+ -90, -93, -94, -97, -99, -101, -103, -104, -107, -108, -110, -111, -112, -114, -115, -117,
+ -118, -119, -120, -121, -122, -123, -123, -124, -125, -125, -126, -126, -127, -127, -127, -127,
+ -127, -127, -127, -127, -126, -126, -125, -125, -124, -124, -123, -122, -121, -120, -119, -118,
+ -117, -116, -114, -113, -111, -110, -109, -107, -105, -103, -101, -100, -97, -96, -93, -91,
+ -90, -87, -85, -82, -80, -76, -75, -73, -69, -67, -63, -62, -60, -56, -54, -50,
+ -48, -46, -41, -39, -35, -33, -31, -26, -24, -20, -18, -15, -11, -9, -4, -2,
+ 0, 2, 4, 9, 11, 15, 18, 20, 24, 26, 31, 33, 35, 39, 41, 46,
+ 48, 50, 54, 56, 60, 62, 64, 67, 69, 73, 75, 76, 80, 82, 85, 87,
+ 90, 91, 93, 96, 97, 100, 101, 103, 105, 107, 109, 110, 111, 113, 114, 116,
+ 117, 118, 119, 120, 121, 122, 123, 124, 124, 125, 125, 126, 126, 127, 127, 127,
+ 127, 127, 127, 127, 127, 126, 126, 125, 125, 124, 123, 123, 122, 121, 120, 119,
+ 118, 117, 115, 114, 112, 111, 110, 108, 107, 104, 103, 101, 99, 97, 94, 93,
+ 90, 88, 87, 83, 82, 78, 76, 75, 71, 69, 65, 64, 62, 58, 56, 52,
+ 50, 48, 43, 41, 37, 35, 33, 29, 26, 22, 20, 18, 13, 11, 7, 4,
+ 0,
+};
+
+#define cos(idx) sin[((idx) + 64) % sizeof(sin)]
+
+/* Global font descriptor */
+static const u8 *font8x16;
+
+void tpg_set_font(const u8 *f)
+{
+ font8x16 = f;
+}
+EXPORT_SYMBOL_GPL(tpg_set_font);
+
+void tpg_init(struct tpg_data *tpg, unsigned w, unsigned h)
+{
+ memset(tpg, 0, sizeof(*tpg));
+ tpg->scaled_width = tpg->src_width = w;
+ tpg->src_height = tpg->buf_height = h;
+ tpg->crop.width = tpg->compose.width = w;
+ tpg->crop.height = tpg->compose.height = h;
+ tpg->recalc_colors = true;
+ tpg->recalc_square_border = true;
+ tpg->brightness = 128;
+ tpg->contrast = 128;
+ tpg->saturation = 128;
+ tpg->hue = 0;
+ tpg->mv_hor_mode = TPG_MOVE_NONE;
+ tpg->mv_vert_mode = TPG_MOVE_NONE;
+ tpg->field = V4L2_FIELD_NONE;
+ tpg_s_fourcc(tpg, V4L2_PIX_FMT_RGB24);
+ tpg->colorspace = V4L2_COLORSPACE_SRGB;
+ tpg->perc_fill = 100;
+}
+EXPORT_SYMBOL_GPL(tpg_init);
+
+int tpg_alloc(struct tpg_data *tpg, unsigned max_w)
+{
+ unsigned pat;
+ unsigned plane;
+
+ tpg->max_line_width = max_w;
+ for (pat = 0; pat < TPG_MAX_PAT_LINES; pat++) {
+ for (plane = 0; plane < TPG_MAX_PLANES; plane++) {
+ unsigned pixelsz = plane ? 2 : 4;
+
+ tpg->lines[pat][plane] = vzalloc(max_w * 2 * pixelsz);
+ if (!tpg->lines[pat][plane])
+ return -ENOMEM;
+ if (plane == 0)
+ continue;
+ tpg->downsampled_lines[pat][plane] = vzalloc(max_w * 2 * pixelsz);
+ if (!tpg->downsampled_lines[pat][plane])
+ return -ENOMEM;
+ }
+ }
+ for (plane = 0; plane < TPG_MAX_PLANES; plane++) {
+ unsigned pixelsz = plane ? 2 : 4;
+
+ tpg->contrast_line[plane] = vzalloc(max_w * pixelsz);
+ if (!tpg->contrast_line[plane])
+ return -ENOMEM;
+ tpg->black_line[plane] = vzalloc(max_w * pixelsz);
+ if (!tpg->black_line[plane])
+ return -ENOMEM;
+ tpg->random_line[plane] = vzalloc(max_w * 2 * pixelsz);
+ if (!tpg->random_line[plane])
+ return -ENOMEM;
+ }
+ return 0;
+}
+EXPORT_SYMBOL_GPL(tpg_alloc);
+
+void tpg_free(struct tpg_data *tpg)
+{
+ unsigned pat;
+ unsigned plane;
+
+ for (pat = 0; pat < TPG_MAX_PAT_LINES; pat++)
+ for (plane = 0; plane < TPG_MAX_PLANES; plane++) {
+ vfree(tpg->lines[pat][plane]);
+ tpg->lines[pat][plane] = NULL;
+ if (plane == 0)
+ continue;
+ vfree(tpg->downsampled_lines[pat][plane]);
+ tpg->downsampled_lines[pat][plane] = NULL;
+ }
+ for (plane = 0; plane < TPG_MAX_PLANES; plane++) {
+ vfree(tpg->contrast_line[plane]);
+ vfree(tpg->black_line[plane]);
+ vfree(tpg->random_line[plane]);
+ tpg->contrast_line[plane] = NULL;
+ tpg->black_line[plane] = NULL;
+ tpg->random_line[plane] = NULL;
+ }
+}
+EXPORT_SYMBOL_GPL(tpg_free);
+
+bool tpg_s_fourcc(struct tpg_data *tpg, u32 fourcc)
+{
+ tpg->fourcc = fourcc;
+ tpg->planes = 1;
+ tpg->buffers = 1;
+ tpg->recalc_colors = true;
+ tpg->interleaved = false;
+ tpg->vdownsampling[0] = 1;
+ tpg->hdownsampling[0] = 1;
+ tpg->hmask[0] = ~0;
+ tpg->hmask[1] = ~0;
+ tpg->hmask[2] = ~0;
+
+ switch (fourcc) {
+ case V4L2_PIX_FMT_SBGGR8:
+ case V4L2_PIX_FMT_SGBRG8:
+ case V4L2_PIX_FMT_SGRBG8:
+ case V4L2_PIX_FMT_SRGGB8:
+ case V4L2_PIX_FMT_SBGGR10:
+ case V4L2_PIX_FMT_SGBRG10:
+ case V4L2_PIX_FMT_SGRBG10:
+ case V4L2_PIX_FMT_SRGGB10:
+ case V4L2_PIX_FMT_SBGGR12:
+ case V4L2_PIX_FMT_SGBRG12:
+ case V4L2_PIX_FMT_SGRBG12:
+ case V4L2_PIX_FMT_SRGGB12:
+ tpg->interleaved = true;
+ tpg->vdownsampling[1] = 1;
+ tpg->hdownsampling[1] = 1;
+ tpg->planes = 2;
+ /* fall through */
+ case V4L2_PIX_FMT_RGB332:
+ case V4L2_PIX_FMT_RGB565:
+ case V4L2_PIX_FMT_RGB565X:
+ case V4L2_PIX_FMT_RGB444:
+ case V4L2_PIX_FMT_XRGB444:
+ case V4L2_PIX_FMT_ARGB444:
+ case V4L2_PIX_FMT_RGB555:
+ case V4L2_PIX_FMT_XRGB555:
+ case V4L2_PIX_FMT_ARGB555:
+ case V4L2_PIX_FMT_RGB555X:
+ case V4L2_PIX_FMT_XRGB555X:
+ case V4L2_PIX_FMT_ARGB555X:
+ case V4L2_PIX_FMT_BGR666:
+ case V4L2_PIX_FMT_RGB24:
+ case V4L2_PIX_FMT_BGR24:
+ case V4L2_PIX_FMT_RGB32:
+ case V4L2_PIX_FMT_BGR32:
+ case V4L2_PIX_FMT_XRGB32:
+ case V4L2_PIX_FMT_XBGR32:
+ case V4L2_PIX_FMT_ARGB32:
+ case V4L2_PIX_FMT_ABGR32:
+ case V4L2_PIX_FMT_GREY:
+ case V4L2_PIX_FMT_Y16:
+ case V4L2_PIX_FMT_Y16_BE:
+ tpg->is_yuv = false;
+ break;
+ case V4L2_PIX_FMT_YUV444:
+ case V4L2_PIX_FMT_YUV555:
+ case V4L2_PIX_FMT_YUV565:
+ case V4L2_PIX_FMT_YUV32:
+ tpg->is_yuv = true;
+ break;
+ case V4L2_PIX_FMT_YUV420M:
+ case V4L2_PIX_FMT_YVU420M:
+ tpg->buffers = 3;
+ /* fall through */
+ case V4L2_PIX_FMT_YUV420:
+ case V4L2_PIX_FMT_YVU420:
+ tpg->vdownsampling[1] = 2;
+ tpg->vdownsampling[2] = 2;
+ tpg->hdownsampling[1] = 2;
+ tpg->hdownsampling[2] = 2;
+ tpg->planes = 3;
+ tpg->is_yuv = true;
+ break;
+ case V4L2_PIX_FMT_YUV422M:
+ case V4L2_PIX_FMT_YVU422M:
+ tpg->buffers = 3;
+ /* fall through */
+ case V4L2_PIX_FMT_YUV422P:
+ tpg->vdownsampling[1] = 1;
+ tpg->vdownsampling[2] = 1;
+ tpg->hdownsampling[1] = 2;
+ tpg->hdownsampling[2] = 2;
+ tpg->planes = 3;
+ tpg->is_yuv = true;
+ break;
+ case V4L2_PIX_FMT_NV16M:
+ case V4L2_PIX_FMT_NV61M:
+ tpg->buffers = 2;
+ /* fall through */
+ case V4L2_PIX_FMT_NV16:
+ case V4L2_PIX_FMT_NV61:
+ tpg->vdownsampling[1] = 1;
+ tpg->hdownsampling[1] = 1;
+ tpg->hmask[1] = ~1;
+ tpg->planes = 2;
+ tpg->is_yuv = true;
+ break;
+ case V4L2_PIX_FMT_NV12M:
+ case V4L2_PIX_FMT_NV21M:
+ tpg->buffers = 2;
+ /* fall through */
+ case V4L2_PIX_FMT_NV12:
+ case V4L2_PIX_FMT_NV21:
+ tpg->vdownsampling[1] = 2;
+ tpg->hdownsampling[1] = 1;
+ tpg->hmask[1] = ~1;
+ tpg->planes = 2;
+ tpg->is_yuv = true;
+ break;
+ case V4L2_PIX_FMT_YUV444M:
+ case V4L2_PIX_FMT_YVU444M:
+ tpg->buffers = 3;
+ tpg->planes = 3;
+ tpg->vdownsampling[1] = 1;
+ tpg->vdownsampling[2] = 1;
+ tpg->hdownsampling[1] = 1;
+ tpg->hdownsampling[2] = 1;
+ tpg->is_yuv = true;
+ break;
+ case V4L2_PIX_FMT_NV24:
+ case V4L2_PIX_FMT_NV42:
+ tpg->vdownsampling[1] = 1;
+ tpg->hdownsampling[1] = 1;
+ tpg->planes = 2;
+ tpg->is_yuv = true;
+ break;
+ case V4L2_PIX_FMT_YUYV:
+ case V4L2_PIX_FMT_UYVY:
+ case V4L2_PIX_FMT_YVYU:
+ case V4L2_PIX_FMT_VYUY:
+ tpg->hmask[0] = ~1;
+ tpg->is_yuv = true;
+ break;
+ default:
+ return false;
+ }
+
+ switch (fourcc) {
+ case V4L2_PIX_FMT_GREY:
+ case V4L2_PIX_FMT_RGB332:
+ tpg->twopixelsize[0] = 2;
+ break;
+ case V4L2_PIX_FMT_RGB565:
+ case V4L2_PIX_FMT_RGB565X:
+ case V4L2_PIX_FMT_RGB444:
+ case V4L2_PIX_FMT_XRGB444:
+ case V4L2_PIX_FMT_ARGB444:
+ case V4L2_PIX_FMT_RGB555:
+ case V4L2_PIX_FMT_XRGB555:
+ case V4L2_PIX_FMT_ARGB555:
+ case V4L2_PIX_FMT_RGB555X:
+ case V4L2_PIX_FMT_XRGB555X:
+ case V4L2_PIX_FMT_ARGB555X:
+ case V4L2_PIX_FMT_YUYV:
+ case V4L2_PIX_FMT_UYVY:
+ case V4L2_PIX_FMT_YVYU:
+ case V4L2_PIX_FMT_VYUY:
+ case V4L2_PIX_FMT_YUV444:
+ case V4L2_PIX_FMT_YUV555:
+ case V4L2_PIX_FMT_YUV565:
+ case V4L2_PIX_FMT_Y16:
+ case V4L2_PIX_FMT_Y16_BE:
+ tpg->twopixelsize[0] = 2 * 2;
+ break;
+ case V4L2_PIX_FMT_RGB24:
+ case V4L2_PIX_FMT_BGR24:
+ tpg->twopixelsize[0] = 2 * 3;
+ break;
+ case V4L2_PIX_FMT_BGR666:
+ case V4L2_PIX_FMT_RGB32:
+ case V4L2_PIX_FMT_BGR32:
+ case V4L2_PIX_FMT_XRGB32:
+ case V4L2_PIX_FMT_XBGR32:
+ case V4L2_PIX_FMT_ARGB32:
+ case V4L2_PIX_FMT_ABGR32:
+ case V4L2_PIX_FMT_YUV32:
+ tpg->twopixelsize[0] = 2 * 4;
+ break;
+ case V4L2_PIX_FMT_NV12:
+ case V4L2_PIX_FMT_NV21:
+ case V4L2_PIX_FMT_NV12M:
+ case V4L2_PIX_FMT_NV21M:
+ case V4L2_PIX_FMT_NV16:
+ case V4L2_PIX_FMT_NV61:
+ case V4L2_PIX_FMT_NV16M:
+ case V4L2_PIX_FMT_NV61M:
+ case V4L2_PIX_FMT_SBGGR8:
+ case V4L2_PIX_FMT_SGBRG8:
+ case V4L2_PIX_FMT_SGRBG8:
+ case V4L2_PIX_FMT_SRGGB8:
+ tpg->twopixelsize[0] = 2;
+ tpg->twopixelsize[1] = 2;
+ break;
+ case V4L2_PIX_FMT_SRGGB10:
+ case V4L2_PIX_FMT_SGRBG10:
+ case V4L2_PIX_FMT_SGBRG10:
+ case V4L2_PIX_FMT_SBGGR10:
+ case V4L2_PIX_FMT_SRGGB12:
+ case V4L2_PIX_FMT_SGRBG12:
+ case V4L2_PIX_FMT_SGBRG12:
+ case V4L2_PIX_FMT_SBGGR12:
+ tpg->twopixelsize[0] = 4;
+ tpg->twopixelsize[1] = 4;
+ break;
+ case V4L2_PIX_FMT_YUV444M:
+ case V4L2_PIX_FMT_YVU444M:
+ case V4L2_PIX_FMT_YUV422M:
+ case V4L2_PIX_FMT_YVU422M:
+ case V4L2_PIX_FMT_YUV422P:
+ case V4L2_PIX_FMT_YUV420:
+ case V4L2_PIX_FMT_YVU420:
+ case V4L2_PIX_FMT_YUV420M:
+ case V4L2_PIX_FMT_YVU420M:
+ tpg->twopixelsize[0] = 2;
+ tpg->twopixelsize[1] = 2;
+ tpg->twopixelsize[2] = 2;
+ break;
+ case V4L2_PIX_FMT_NV24:
+ case V4L2_PIX_FMT_NV42:
+ tpg->twopixelsize[0] = 2;
+ tpg->twopixelsize[1] = 4;
+ break;
+ }
+ return true;
+}
+EXPORT_SYMBOL_GPL(tpg_s_fourcc);
+
+void tpg_s_crop_compose(struct tpg_data *tpg, const struct v4l2_rect *crop,
+ const struct v4l2_rect *compose)
+{
+ tpg->crop = *crop;
+ tpg->compose = *compose;
+ tpg->scaled_width = (tpg->src_width * tpg->compose.width +
+ tpg->crop.width - 1) / tpg->crop.width;
+ tpg->scaled_width &= ~1;
+ if (tpg->scaled_width > tpg->max_line_width)
+ tpg->scaled_width = tpg->max_line_width;
+ if (tpg->scaled_width < 2)
+ tpg->scaled_width = 2;
+ tpg->recalc_lines = true;
+}
+EXPORT_SYMBOL_GPL(tpg_s_crop_compose);
+
+void tpg_reset_source(struct tpg_data *tpg, unsigned width, unsigned height,
+ u32 field)
+{
+ unsigned p;
+
+ tpg->src_width = width;
+ tpg->src_height = height;
+ tpg->field = field;
+ tpg->buf_height = height;
+ if (V4L2_FIELD_HAS_T_OR_B(field))
+ tpg->buf_height /= 2;
+ tpg->scaled_width = width;
+ tpg->crop.top = tpg->crop.left = 0;
+ tpg->crop.width = width;
+ tpg->crop.height = height;
+ tpg->compose.top = tpg->compose.left = 0;
+ tpg->compose.width = width;
+ tpg->compose.height = tpg->buf_height;
+ for (p = 0; p < tpg->planes; p++)
+ tpg->bytesperline[p] = (width * tpg->twopixelsize[p]) /
+ (2 * tpg->hdownsampling[p]);
+ tpg->recalc_square_border = true;
+}
+EXPORT_SYMBOL_GPL(tpg_reset_source);
+
+static enum tpg_color tpg_get_textbg_color(struct tpg_data *tpg)
+{
+ switch (tpg->pattern) {
+ case TPG_PAT_BLACK:
+ return TPG_COLOR_100_WHITE;
+ case TPG_PAT_CSC_COLORBAR:
+ return TPG_COLOR_CSC_BLACK;
+ default:
+ return TPG_COLOR_100_BLACK;
+ }
+}
+
+static enum tpg_color tpg_get_textfg_color(struct tpg_data *tpg)
+{
+ switch (tpg->pattern) {
+ case TPG_PAT_75_COLORBAR:
+ case TPG_PAT_CSC_COLORBAR:
+ return TPG_COLOR_CSC_WHITE;
+ case TPG_PAT_BLACK:
+ return TPG_COLOR_100_BLACK;
+ default:
+ return TPG_COLOR_100_WHITE;
+ }
+}
+
+static inline int rec709_to_linear(int v)
+{
+ v = clamp(v, 0, 0xff0);
+ return tpg_rec709_to_linear[v];
+}
+
+static inline int linear_to_rec709(int v)
+{
+ v = clamp(v, 0, 0xff0);
+ return tpg_linear_to_rec709[v];
+}
+
+static void rgb2ycbcr(const int m[3][3], int r, int g, int b,
+ int y_offset, int *y, int *cb, int *cr)
+{
+ *y = ((m[0][0] * r + m[0][1] * g + m[0][2] * b) >> 16) + (y_offset << 4);
+ *cb = ((m[1][0] * r + m[1][1] * g + m[1][2] * b) >> 16) + (128 << 4);
+ *cr = ((m[2][0] * r + m[2][1] * g + m[2][2] * b) >> 16) + (128 << 4);
+}
+
+static void color_to_ycbcr(struct tpg_data *tpg, int r, int g, int b,
+ int *y, int *cb, int *cr)
+{
+#define COEFF(v, r) ((int)(0.5 + (v) * (r) * 256.0))
+
+ static const int bt601[3][3] = {
+ { COEFF(0.299, 219), COEFF(0.587, 219), COEFF(0.114, 219) },
+ { COEFF(-0.169, 224), COEFF(-0.331, 224), COEFF(0.5, 224) },
+ { COEFF(0.5, 224), COEFF(-0.419, 224), COEFF(-0.081, 224) },
+ };
+ static const int bt601_full[3][3] = {
+ { COEFF(0.299, 255), COEFF(0.587, 255), COEFF(0.114, 255) },
+ { COEFF(-0.169, 255), COEFF(-0.331, 255), COEFF(0.5, 255) },
+ { COEFF(0.5, 255), COEFF(-0.419, 255), COEFF(-0.081, 255) },
+ };
+ static const int rec709[3][3] = {
+ { COEFF(0.2126, 219), COEFF(0.7152, 219), COEFF(0.0722, 219) },
+ { COEFF(-0.1146, 224), COEFF(-0.3854, 224), COEFF(0.5, 224) },
+ { COEFF(0.5, 224), COEFF(-0.4542, 224), COEFF(-0.0458, 224) },
+ };
+ static const int rec709_full[3][3] = {
+ { COEFF(0.2126, 255), COEFF(0.7152, 255), COEFF(0.0722, 255) },
+ { COEFF(-0.1146, 255), COEFF(-0.3854, 255), COEFF(0.5, 255) },
+ { COEFF(0.5, 255), COEFF(-0.4542, 255), COEFF(-0.0458, 255) },
+ };
+ static const int smpte240m[3][3] = {
+ { COEFF(0.212, 219), COEFF(0.701, 219), COEFF(0.087, 219) },
+ { COEFF(-0.116, 224), COEFF(-0.384, 224), COEFF(0.5, 224) },
+ { COEFF(0.5, 224), COEFF(-0.445, 224), COEFF(-0.055, 224) },
+ };
+ static const int smpte240m_full[3][3] = {
+ { COEFF(0.212, 255), COEFF(0.701, 255), COEFF(0.087, 255) },
+ { COEFF(-0.116, 255), COEFF(-0.384, 255), COEFF(0.5, 255) },
+ { COEFF(0.5, 255), COEFF(-0.445, 255), COEFF(-0.055, 255) },
+ };
+ static const int bt2020[3][3] = {
+ { COEFF(0.2627, 219), COEFF(0.6780, 219), COEFF(0.0593, 219) },
+ { COEFF(-0.1396, 224), COEFF(-0.3604, 224), COEFF(0.5, 224) },
+ { COEFF(0.5, 224), COEFF(-0.4598, 224), COEFF(-0.0402, 224) },
+ };
+ static const int bt2020_full[3][3] = {
+ { COEFF(0.2627, 255), COEFF(0.6780, 255), COEFF(0.0593, 255) },
+ { COEFF(-0.1396, 255), COEFF(-0.3604, 255), COEFF(0.5, 255) },
+ { COEFF(0.5, 255), COEFF(-0.4698, 255), COEFF(-0.0402, 255) },
+ };
+ static const int bt2020c[4] = {
+ COEFF(1.0 / 1.9404, 224), COEFF(1.0 / 1.5816, 224),
+ COEFF(1.0 / 1.7184, 224), COEFF(1.0 / 0.9936, 224),
+ };
+ static const int bt2020c_full[4] = {
+ COEFF(1.0 / 1.9404, 255), COEFF(1.0 / 1.5816, 255),
+ COEFF(1.0 / 1.7184, 255), COEFF(1.0 / 0.9936, 255),
+ };
+
+ bool full = tpg->real_quantization == V4L2_QUANTIZATION_FULL_RANGE;
+ unsigned y_offset = full ? 0 : 16;
+ int lin_y, yc;
+
+ switch (tpg->real_ycbcr_enc) {
+ case V4L2_YCBCR_ENC_601:
+ case V4L2_YCBCR_ENC_SYCC:
+ rgb2ycbcr(full ? bt601_full : bt601, r, g, b, y_offset, y, cb, cr);
+ break;
+ case V4L2_YCBCR_ENC_XV601:
+ /* Ignore quantization range, there is only one possible
+ * Y'CbCr encoding. */
+ rgb2ycbcr(bt601, r, g, b, 16, y, cb, cr);
+ break;
+ case V4L2_YCBCR_ENC_XV709:
+ /* Ignore quantization range, there is only one possible
+ * Y'CbCr encoding. */
+ rgb2ycbcr(rec709, r, g, b, 16, y, cb, cr);
+ break;
+ case V4L2_YCBCR_ENC_BT2020:
+ rgb2ycbcr(full ? bt2020_full : bt2020, r, g, b, y_offset, y, cb, cr);
+ break;
+ case V4L2_YCBCR_ENC_BT2020_CONST_LUM:
+ lin_y = (COEFF(0.2627, 255) * rec709_to_linear(r) +
+ COEFF(0.6780, 255) * rec709_to_linear(g) +
+ COEFF(0.0593, 255) * rec709_to_linear(b)) >> 16;
+ yc = linear_to_rec709(lin_y);
+ *y = full ? yc : (yc * 219) / 255 + (16 << 4);
+ if (b <= yc)
+ *cb = (((b - yc) * (full ? bt2020c_full[0] : bt2020c[0])) >> 16) + (128 << 4);
+ else
+ *cb = (((b - yc) * (full ? bt2020c_full[1] : bt2020c[1])) >> 16) + (128 << 4);
+ if (r <= yc)
+ *cr = (((r - yc) * (full ? bt2020c_full[2] : bt2020c[2])) >> 16) + (128 << 4);
+ else
+ *cr = (((r - yc) * (full ? bt2020c_full[3] : bt2020c[3])) >> 16) + (128 << 4);
+ break;
+ case V4L2_YCBCR_ENC_SMPTE240M:
+ rgb2ycbcr(full ? smpte240m_full : smpte240m, r, g, b, y_offset, y, cb, cr);
+ break;
+ case V4L2_YCBCR_ENC_709:
+ default:
+ rgb2ycbcr(full ? rec709_full : rec709, r, g, b, y_offset, y, cb, cr);
+ break;
+ }
+}
+
+static void ycbcr2rgb(const int m[3][3], int y, int cb, int cr,
+ int y_offset, int *r, int *g, int *b)
+{
+ y -= y_offset << 4;
+ cb -= 128 << 4;
+ cr -= 128 << 4;
+ *r = m[0][0] * y + m[0][1] * cb + m[0][2] * cr;
+ *g = m[1][0] * y + m[1][1] * cb + m[1][2] * cr;
+ *b = m[2][0] * y + m[2][1] * cb + m[2][2] * cr;
+ *r = clamp(*r >> 12, 0, 0xff0);
+ *g = clamp(*g >> 12, 0, 0xff0);
+ *b = clamp(*b >> 12, 0, 0xff0);
+}
+
+static void ycbcr_to_color(struct tpg_data *tpg, int y, int cb, int cr,
+ int *r, int *g, int *b)
+{
+#undef COEFF
+#define COEFF(v, r) ((int)(0.5 + (v) * ((255.0 * 255.0 * 16.0) / (r))))
+ static const int bt601[3][3] = {
+ { COEFF(1, 219), COEFF(0, 224), COEFF(1.4020, 224) },
+ { COEFF(1, 219), COEFF(-0.3441, 224), COEFF(-0.7141, 224) },
+ { COEFF(1, 219), COEFF(1.7720, 224), COEFF(0, 224) },
+ };
+ static const int bt601_full[3][3] = {
+ { COEFF(1, 255), COEFF(0, 255), COEFF(1.4020, 255) },
+ { COEFF(1, 255), COEFF(-0.3441, 255), COEFF(-0.7141, 255) },
+ { COEFF(1, 255), COEFF(1.7720, 255), COEFF(0, 255) },
+ };
+ static const int rec709[3][3] = {
+ { COEFF(1, 219), COEFF(0, 224), COEFF(1.5748, 224) },
+ { COEFF(1, 219), COEFF(-0.1873, 224), COEFF(-0.4681, 224) },
+ { COEFF(1, 219), COEFF(1.8556, 224), COEFF(0, 224) },
+ };
+ static const int rec709_full[3][3] = {
+ { COEFF(1, 255), COEFF(0, 255), COEFF(1.5748, 255) },
+ { COEFF(1, 255), COEFF(-0.1873, 255), COEFF(-0.4681, 255) },
+ { COEFF(1, 255), COEFF(1.8556, 255), COEFF(0, 255) },
+ };
+ static const int smpte240m[3][3] = {
+ { COEFF(1, 219), COEFF(0, 224), COEFF(1.5756, 224) },
+ { COEFF(1, 219), COEFF(-0.2253, 224), COEFF(-0.4767, 224) },
+ { COEFF(1, 219), COEFF(1.8270, 224), COEFF(0, 224) },
+ };
+ static const int smpte240m_full[3][3] = {
+ { COEFF(1, 255), COEFF(0, 255), COEFF(1.5756, 255) },
+ { COEFF(1, 255), COEFF(-0.2253, 255), COEFF(-0.4767, 255) },
+ { COEFF(1, 255), COEFF(1.8270, 255), COEFF(0, 255) },
+ };
+ static const int bt2020[3][3] = {
+ { COEFF(1, 219), COEFF(0, 224), COEFF(1.4746, 224) },
+ { COEFF(1, 219), COEFF(-0.1646, 224), COEFF(-0.5714, 224) },
+ { COEFF(1, 219), COEFF(1.8814, 224), COEFF(0, 224) },
+ };
+ static const int bt2020_full[3][3] = {
+ { COEFF(1, 255), COEFF(0, 255), COEFF(1.4746, 255) },
+ { COEFF(1, 255), COEFF(-0.1646, 255), COEFF(-0.5714, 255) },
+ { COEFF(1, 255), COEFF(1.8814, 255), COEFF(0, 255) },
+ };
+ static const int bt2020c[4] = {
+ COEFF(1.9404, 224), COEFF(1.5816, 224),
+ COEFF(1.7184, 224), COEFF(0.9936, 224),
+ };
+ static const int bt2020c_full[4] = {
+ COEFF(1.9404, 255), COEFF(1.5816, 255),
+ COEFF(1.7184, 255), COEFF(0.9936, 255),
+ };
+
+ bool full = tpg->real_quantization == V4L2_QUANTIZATION_FULL_RANGE;
+ unsigned y_offset = full ? 0 : 16;
+ int y_fac = full ? COEFF(1.0, 255) : COEFF(1.0, 219);
+ int lin_r, lin_g, lin_b, lin_y;
+
+ switch (tpg->real_ycbcr_enc) {
+ case V4L2_YCBCR_ENC_601:
+ case V4L2_YCBCR_ENC_SYCC:
+ ycbcr2rgb(full ? bt601_full : bt601, y, cb, cr, y_offset, r, g, b);
+ break;
+ case V4L2_YCBCR_ENC_XV601:
+ /* Ignore quantization range, there is only one possible
+ * Y'CbCr encoding. */
+ ycbcr2rgb(bt601, y, cb, cr, 16, r, g, b);
+ break;
+ case V4L2_YCBCR_ENC_XV709:
+ /* Ignore quantization range, there is only one possible
+ * Y'CbCr encoding. */
+ ycbcr2rgb(rec709, y, cb, cr, 16, r, g, b);
+ break;
+ case V4L2_YCBCR_ENC_BT2020:
+ ycbcr2rgb(full ? bt2020_full : bt2020, y, cb, cr, y_offset, r, g, b);
+ break;
+ case V4L2_YCBCR_ENC_BT2020_CONST_LUM:
+ y -= full ? 0 : 16 << 4;
+ cb -= 128 << 4;
+ cr -= 128 << 4;
+
+ if (cb <= 0)
+ *b = y_fac * y + (full ? bt2020c_full[0] : bt2020c[0]) * cb;
+ else
+ *b = y_fac * y + (full ? bt2020c_full[1] : bt2020c[1]) * cb;
+ *b = *b >> 12;
+ if (cr <= 0)
+ *r = y_fac * y + (full ? bt2020c_full[2] : bt2020c[2]) * cr;
+ else
+ *r = y_fac * y + (full ? bt2020c_full[3] : bt2020c[3]) * cr;
+ *r = *r >> 12;
+ lin_r = rec709_to_linear(*r);
+ lin_b = rec709_to_linear(*b);
+ lin_y = rec709_to_linear((y * 255) / (full ? 255 : 219));
+
+ lin_g = COEFF(1.0 / 0.6780, 255) * lin_y -
+ COEFF(0.2627 / 0.6780, 255) * lin_r -
+ COEFF(0.0593 / 0.6780, 255) * lin_b;
+ *g = linear_to_rec709(lin_g >> 12);
+ break;
+ case V4L2_YCBCR_ENC_SMPTE240M:
+ ycbcr2rgb(full ? smpte240m_full : smpte240m, y, cb, cr, y_offset, r, g, b);
+ break;
+ case V4L2_YCBCR_ENC_709:
+ default:
+ ycbcr2rgb(full ? rec709_full : rec709, y, cb, cr, y_offset, r, g, b);
+ break;
+ }
+}
+
+/* precalculate color bar values to speed up rendering */
+static void precalculate_color(struct tpg_data *tpg, int k)
+{
+ int col = k;
+ int r = tpg_colors[col].r;
+ int g = tpg_colors[col].g;
+ int b = tpg_colors[col].b;
+
+ if (k == TPG_COLOR_TEXTBG) {
+ col = tpg_get_textbg_color(tpg);
+
+ r = tpg_colors[col].r;
+ g = tpg_colors[col].g;
+ b = tpg_colors[col].b;
+ } else if (k == TPG_COLOR_TEXTFG) {
+ col = tpg_get_textfg_color(tpg);
+
+ r = tpg_colors[col].r;
+ g = tpg_colors[col].g;
+ b = tpg_colors[col].b;
+ } else if (tpg->pattern == TPG_PAT_NOISE) {
+ r = g = b = prandom_u32_max(256);
+ } else if (k == TPG_COLOR_RANDOM) {
+ r = g = b = tpg->qual_offset + prandom_u32_max(196);
+ } else if (k >= TPG_COLOR_RAMP) {
+ r = g = b = k - TPG_COLOR_RAMP;
+ }
+
+ if (tpg->pattern == TPG_PAT_CSC_COLORBAR && col <= TPG_COLOR_CSC_BLACK) {
+ r = tpg_csc_colors[tpg->colorspace][tpg->real_xfer_func][col].r;
+ g = tpg_csc_colors[tpg->colorspace][tpg->real_xfer_func][col].g;
+ b = tpg_csc_colors[tpg->colorspace][tpg->real_xfer_func][col].b;
+ } else {
+ r <<= 4;
+ g <<= 4;
+ b <<= 4;
+ }
+ if (tpg->qual == TPG_QUAL_GRAY || tpg->fourcc == V4L2_PIX_FMT_GREY ||
+ tpg->fourcc == V4L2_PIX_FMT_Y16 ||
+ tpg->fourcc == V4L2_PIX_FMT_Y16_BE) {
+ /* Rec. 709 Luma function */
+ /* (0.2126, 0.7152, 0.0722) * (255 * 256) */
+ r = g = b = (13879 * r + 46688 * g + 4713 * b) >> 16;
+ }
+
+ /*
+ * The assumption is that the RGB output is always full range,
+ * so only if the rgb_range overrides the 'real' rgb range do
+ * we need to convert the RGB values.
+ *
+ * Remember that r, g and b are still in the 0 - 0xff0 range.
+ */
+ if (tpg->real_rgb_range == V4L2_DV_RGB_RANGE_LIMITED &&
+ tpg->rgb_range == V4L2_DV_RGB_RANGE_FULL) {
+ /*
+ * Convert from full range (which is what r, g and b are)
+ * to limited range (which is the 'real' RGB range), which
+ * is then interpreted as full range.
+ */
+ r = (r * 219) / 255 + (16 << 4);
+ g = (g * 219) / 255 + (16 << 4);
+ b = (b * 219) / 255 + (16 << 4);
+ } else if (tpg->real_rgb_range != V4L2_DV_RGB_RANGE_LIMITED &&
+ tpg->rgb_range == V4L2_DV_RGB_RANGE_LIMITED) {
+ /*
+ * Clamp r, g and b to the limited range and convert to full
+ * range since that's what we deliver.
+ */
+ r = clamp(r, 16 << 4, 235 << 4);
+ g = clamp(g, 16 << 4, 235 << 4);
+ b = clamp(b, 16 << 4, 235 << 4);
+ r = (r - (16 << 4)) * 255 / 219;
+ g = (g - (16 << 4)) * 255 / 219;
+ b = (b - (16 << 4)) * 255 / 219;
+ }
+
+ if (tpg->brightness != 128 || tpg->contrast != 128 ||
+ tpg->saturation != 128 || tpg->hue) {
+ /* Implement these operations */
+ int y, cb, cr;
+ int tmp_cb, tmp_cr;
+
+ /* First convert to YCbCr */
+
+ color_to_ycbcr(tpg, r, g, b, &y, &cb, &cr);
+
+ y = (16 << 4) + ((y - (16 << 4)) * tpg->contrast) / 128;
+ y += (tpg->brightness << 4) - (128 << 4);
+
+ cb -= 128 << 4;
+ cr -= 128 << 4;
+ tmp_cb = (cb * cos(128 + tpg->hue)) / 127 + (cr * sin[128 + tpg->hue]) / 127;
+ tmp_cr = (cr * cos(128 + tpg->hue)) / 127 - (cb * sin[128 + tpg->hue]) / 127;
+
+ cb = (128 << 4) + (tmp_cb * tpg->contrast * tpg->saturation) / (128 * 128);
+ cr = (128 << 4) + (tmp_cr * tpg->contrast * tpg->saturation) / (128 * 128);
+ if (tpg->is_yuv) {
+ tpg->colors[k][0] = clamp(y >> 4, 1, 254);
+ tpg->colors[k][1] = clamp(cb >> 4, 1, 254);
+ tpg->colors[k][2] = clamp(cr >> 4, 1, 254);
+ return;
+ }
+ ycbcr_to_color(tpg, y, cb, cr, &r, &g, &b);
+ }
+
+ if (tpg->is_yuv) {
+ /* Convert to YCbCr */
+ int y, cb, cr;
+
+ color_to_ycbcr(tpg, r, g, b, &y, &cb, &cr);
+
+ if (tpg->real_quantization == V4L2_QUANTIZATION_LIM_RANGE) {
+ y = clamp(y, 16 << 4, 235 << 4);
+ cb = clamp(cb, 16 << 4, 240 << 4);
+ cr = clamp(cr, 16 << 4, 240 << 4);
+ }
+ y = clamp(y >> 4, 1, 254);
+ cb = clamp(cb >> 4, 1, 254);
+ cr = clamp(cr >> 4, 1, 254);
+ switch (tpg->fourcc) {
+ case V4L2_PIX_FMT_YUV444:
+ y >>= 4;
+ cb >>= 4;
+ cr >>= 4;
+ break;
+ case V4L2_PIX_FMT_YUV555:
+ y >>= 3;
+ cb >>= 3;
+ cr >>= 3;
+ break;
+ case V4L2_PIX_FMT_YUV565:
+ y >>= 3;
+ cb >>= 2;
+ cr >>= 3;
+ break;
+ }
+ tpg->colors[k][0] = y;
+ tpg->colors[k][1] = cb;
+ tpg->colors[k][2] = cr;
+ } else {
+ if (tpg->real_quantization == V4L2_QUANTIZATION_LIM_RANGE) {
+ r = (r * 219) / 255 + (16 << 4);
+ g = (g * 219) / 255 + (16 << 4);
+ b = (b * 219) / 255 + (16 << 4);
+ }
+ switch (tpg->fourcc) {
+ case V4L2_PIX_FMT_RGB332:
+ r >>= 9;
+ g >>= 9;
+ b >>= 10;
+ break;
+ case V4L2_PIX_FMT_RGB565:
+ case V4L2_PIX_FMT_RGB565X:
+ r >>= 7;
+ g >>= 6;
+ b >>= 7;
+ break;
+ case V4L2_PIX_FMT_RGB444:
+ case V4L2_PIX_FMT_XRGB444:
+ case V4L2_PIX_FMT_ARGB444:
+ r >>= 8;
+ g >>= 8;
+ b >>= 8;
+ break;
+ case V4L2_PIX_FMT_RGB555:
+ case V4L2_PIX_FMT_XRGB555:
+ case V4L2_PIX_FMT_ARGB555:
+ case V4L2_PIX_FMT_RGB555X:
+ case V4L2_PIX_FMT_XRGB555X:
+ case V4L2_PIX_FMT_ARGB555X:
+ r >>= 7;
+ g >>= 7;
+ b >>= 7;
+ break;
+ case V4L2_PIX_FMT_BGR666:
+ r >>= 6;
+ g >>= 6;
+ b >>= 6;
+ break;
+ default:
+ r >>= 4;
+ g >>= 4;
+ b >>= 4;
+ break;
+ }
+
+ tpg->colors[k][0] = r;
+ tpg->colors[k][1] = g;
+ tpg->colors[k][2] = b;
+ }
+}
+
+static void tpg_precalculate_colors(struct tpg_data *tpg)
+{
+ int k;
+
+ for (k = 0; k < TPG_COLOR_MAX; k++)
+ precalculate_color(tpg, k);
+}
+
+/* 'odd' is true for pixels 1, 3, 5, etc. and false for pixels 0, 2, 4, etc. */
+static void gen_twopix(struct tpg_data *tpg,
+ u8 buf[TPG_MAX_PLANES][8], int color, bool odd)
+{
+ unsigned offset = odd * tpg->twopixelsize[0] / 2;
+ u8 alpha = tpg->alpha_component;
+ u8 r_y, g_u, b_v;
+
+ if (tpg->alpha_red_only && color != TPG_COLOR_CSC_RED &&
+ color != TPG_COLOR_100_RED &&
+ color != TPG_COLOR_75_RED)
+ alpha = 0;
+ if (color == TPG_COLOR_RANDOM)
+ precalculate_color(tpg, color);
+ r_y = tpg->colors[color][0]; /* R or precalculated Y */
+ g_u = tpg->colors[color][1]; /* G or precalculated U */
+ b_v = tpg->colors[color][2]; /* B or precalculated V */
+
+ switch (tpg->fourcc) {
+ case V4L2_PIX_FMT_GREY:
+ buf[0][offset] = r_y;
+ break;
+ case V4L2_PIX_FMT_Y16:
+ /*
+ * Ideally both bytes should be set to r_y, but then you won't
+ * be able to detect endian problems. So keep it 0 except for
+ * the corner case where r_y is 0xff so white really will be
+ * white (0xffff).
+ */
+ buf[0][offset] = r_y == 0xff ? r_y : 0;
+ buf[0][offset+1] = r_y;
+ break;
+ case V4L2_PIX_FMT_Y16_BE:
+ /* See comment for V4L2_PIX_FMT_Y16 above */
+ buf[0][offset] = r_y;
+ buf[0][offset+1] = r_y == 0xff ? r_y : 0;
+ break;
+ case V4L2_PIX_FMT_YUV422M:
+ case V4L2_PIX_FMT_YUV422P:
+ case V4L2_PIX_FMT_YUV420:
+ case V4L2_PIX_FMT_YUV420M:
+ buf[0][offset] = r_y;
+ if (odd) {
+ buf[1][0] = (buf[1][0] + g_u) / 2;
+ buf[2][0] = (buf[2][0] + b_v) / 2;
+ buf[1][1] = buf[1][0];
+ buf[2][1] = buf[2][0];
+ break;
+ }
+ buf[1][0] = g_u;
+ buf[2][0] = b_v;
+ break;
+ case V4L2_PIX_FMT_YVU422M:
+ case V4L2_PIX_FMT_YVU420:
+ case V4L2_PIX_FMT_YVU420M:
+ buf[0][offset] = r_y;
+ if (odd) {
+ buf[1][0] = (buf[1][0] + b_v) / 2;
+ buf[2][0] = (buf[2][0] + g_u) / 2;
+ buf[1][1] = buf[1][0];
+ buf[2][1] = buf[2][0];
+ break;
+ }
+ buf[1][0] = b_v;
+ buf[2][0] = g_u;
+ break;
+
+ case V4L2_PIX_FMT_NV12:
+ case V4L2_PIX_FMT_NV12M:
+ case V4L2_PIX_FMT_NV16:
+ case V4L2_PIX_FMT_NV16M:
+ buf[0][offset] = r_y;
+ if (odd) {
+ buf[1][0] = (buf[1][0] + g_u) / 2;
+ buf[1][1] = (buf[1][1] + b_v) / 2;
+ break;
+ }
+ buf[1][0] = g_u;
+ buf[1][1] = b_v;
+ break;
+ case V4L2_PIX_FMT_NV21:
+ case V4L2_PIX_FMT_NV21M:
+ case V4L2_PIX_FMT_NV61:
+ case V4L2_PIX_FMT_NV61M:
+ buf[0][offset] = r_y;
+ if (odd) {
+ buf[1][0] = (buf[1][0] + b_v) / 2;
+ buf[1][1] = (buf[1][1] + g_u) / 2;
+ break;
+ }
+ buf[1][0] = b_v;
+ buf[1][1] = g_u;
+ break;
+
+ case V4L2_PIX_FMT_YUV444M:
+ buf[0][offset] = r_y;
+ buf[1][offset] = g_u;
+ buf[2][offset] = b_v;
+ break;
+
+ case V4L2_PIX_FMT_YVU444M:
+ buf[0][offset] = r_y;
+ buf[1][offset] = b_v;
+ buf[2][offset] = g_u;
+ break;
+
+ case V4L2_PIX_FMT_NV24:
+ buf[0][offset] = r_y;
+ buf[1][2 * offset] = g_u;
+ buf[1][2 * offset + 1] = b_v;
+ break;
+
+ case V4L2_PIX_FMT_NV42:
+ buf[0][offset] = r_y;
+ buf[1][2 * offset] = b_v;
+ buf[1][2 * offset + 1] = g_u;
+ break;
+
+ case V4L2_PIX_FMT_YUYV:
+ buf[0][offset] = r_y;
+ if (odd) {
+ buf[0][1] = (buf[0][1] + g_u) / 2;
+ buf[0][3] = (buf[0][3] + b_v) / 2;
+ break;
+ }
+ buf[0][1] = g_u;
+ buf[0][3] = b_v;
+ break;
+ case V4L2_PIX_FMT_UYVY:
+ buf[0][offset + 1] = r_y;
+ if (odd) {
+ buf[0][0] = (buf[0][0] + g_u) / 2;
+ buf[0][2] = (buf[0][2] + b_v) / 2;
+ break;
+ }
+ buf[0][0] = g_u;
+ buf[0][2] = b_v;
+ break;
+ case V4L2_PIX_FMT_YVYU:
+ buf[0][offset] = r_y;
+ if (odd) {
+ buf[0][1] = (buf[0][1] + b_v) / 2;
+ buf[0][3] = (buf[0][3] + g_u) / 2;
+ break;
+ }
+ buf[0][1] = b_v;
+ buf[0][3] = g_u;
+ break;
+ case V4L2_PIX_FMT_VYUY:
+ buf[0][offset + 1] = r_y;
+ if (odd) {
+ buf[0][0] = (buf[0][0] + b_v) / 2;
+ buf[0][2] = (buf[0][2] + g_u) / 2;
+ break;
+ }
+ buf[0][0] = b_v;
+ buf[0][2] = g_u;
+ break;
+ case V4L2_PIX_FMT_RGB332:
+ buf[0][offset] = (r_y << 5) | (g_u << 2) | b_v;
+ break;
+ case V4L2_PIX_FMT_YUV565:
+ case V4L2_PIX_FMT_RGB565:
+ buf[0][offset] = (g_u << 5) | b_v;
+ buf[0][offset + 1] = (r_y << 3) | (g_u >> 3);
+ break;
+ case V4L2_PIX_FMT_RGB565X:
+ buf[0][offset] = (r_y << 3) | (g_u >> 3);
+ buf[0][offset + 1] = (g_u << 5) | b_v;
+ break;
+ case V4L2_PIX_FMT_RGB444:
+ case V4L2_PIX_FMT_XRGB444:
+ alpha = 0;
+ /* fall through */
+ case V4L2_PIX_FMT_YUV444:
+ case V4L2_PIX_FMT_ARGB444:
+ buf[0][offset] = (g_u << 4) | b_v;
+ buf[0][offset + 1] = (alpha & 0xf0) | r_y;
+ break;
+ case V4L2_PIX_FMT_RGB555:
+ case V4L2_PIX_FMT_XRGB555:
+ alpha = 0;
+ /* fall through */
+ case V4L2_PIX_FMT_YUV555:
+ case V4L2_PIX_FMT_ARGB555:
+ buf[0][offset] = (g_u << 5) | b_v;
+ buf[0][offset + 1] = (alpha & 0x80) | (r_y << 2) | (g_u >> 3);
+ break;
+ case V4L2_PIX_FMT_RGB555X:
+ case V4L2_PIX_FMT_XRGB555X:
+ alpha = 0;
+ /* fall through */
+ case V4L2_PIX_FMT_ARGB555X:
+ buf[0][offset] = (alpha & 0x80) | (r_y << 2) | (g_u >> 3);
+ buf[0][offset + 1] = (g_u << 5) | b_v;
+ break;
+ case V4L2_PIX_FMT_RGB24:
+ buf[0][offset] = r_y;
+ buf[0][offset + 1] = g_u;
+ buf[0][offset + 2] = b_v;
+ break;
+ case V4L2_PIX_FMT_BGR24:
+ buf[0][offset] = b_v;
+ buf[0][offset + 1] = g_u;
+ buf[0][offset + 2] = r_y;
+ break;
+ case V4L2_PIX_FMT_BGR666:
+ buf[0][offset] = (b_v << 2) | (g_u >> 4);
+ buf[0][offset + 1] = (g_u << 4) | (r_y >> 2);
+ buf[0][offset + 2] = r_y << 6;
+ buf[0][offset + 3] = 0;
+ break;
+ case V4L2_PIX_FMT_RGB32:
+ case V4L2_PIX_FMT_XRGB32:
+ alpha = 0;
+ /* fall through */
+ case V4L2_PIX_FMT_YUV32:
+ case V4L2_PIX_FMT_ARGB32:
+ buf[0][offset] = alpha;
+ buf[0][offset + 1] = r_y;
+ buf[0][offset + 2] = g_u;
+ buf[0][offset + 3] = b_v;
+ break;
+ case V4L2_PIX_FMT_BGR32:
+ case V4L2_PIX_FMT_XBGR32:
+ alpha = 0;
+ /* fall through */
+ case V4L2_PIX_FMT_ABGR32:
+ buf[0][offset] = b_v;
+ buf[0][offset + 1] = g_u;
+ buf[0][offset + 2] = r_y;
+ buf[0][offset + 3] = alpha;
+ break;
+ case V4L2_PIX_FMT_SBGGR8:
+ buf[0][offset] = odd ? g_u : b_v;
+ buf[1][offset] = odd ? r_y : g_u;
+ break;
+ case V4L2_PIX_FMT_SGBRG8:
+ buf[0][offset] = odd ? b_v : g_u;
+ buf[1][offset] = odd ? g_u : r_y;
+ break;
+ case V4L2_PIX_FMT_SGRBG8:
+ buf[0][offset] = odd ? r_y : g_u;
+ buf[1][offset] = odd ? g_u : b_v;
+ break;
+ case V4L2_PIX_FMT_SRGGB8:
+ buf[0][offset] = odd ? g_u : r_y;
+ buf[1][offset] = odd ? b_v : g_u;
+ break;
+ case V4L2_PIX_FMT_SBGGR10:
+ buf[0][offset] = odd ? g_u << 2 : b_v << 2;
+ buf[0][offset + 1] = odd ? g_u >> 6 : b_v >> 6;
+ buf[1][offset] = odd ? r_y << 2 : g_u << 2;
+ buf[1][offset + 1] = odd ? r_y >> 6 : g_u >> 6;
+ buf[0][offset] |= (buf[0][offset] >> 2) & 3;
+ buf[1][offset] |= (buf[1][offset] >> 2) & 3;
+ break;
+ case V4L2_PIX_FMT_SGBRG10:
+ buf[0][offset] = odd ? b_v << 2 : g_u << 2;
+ buf[0][offset + 1] = odd ? b_v >> 6 : g_u >> 6;
+ buf[1][offset] = odd ? g_u << 2 : r_y << 2;
+ buf[1][offset + 1] = odd ? g_u >> 6 : r_y >> 6;
+ buf[0][offset] |= (buf[0][offset] >> 2) & 3;
+ buf[1][offset] |= (buf[1][offset] >> 2) & 3;
+ break;
+ case V4L2_PIX_FMT_SGRBG10:
+ buf[0][offset] = odd ? r_y << 2 : g_u << 2;
+ buf[0][offset + 1] = odd ? r_y >> 6 : g_u >> 6;
+ buf[1][offset] = odd ? g_u << 2 : b_v << 2;
+ buf[1][offset + 1] = odd ? g_u >> 6 : b_v >> 6;
+ buf[0][offset] |= (buf[0][offset] >> 2) & 3;
+ buf[1][offset] |= (buf[1][offset] >> 2) & 3;
+ break;
+ case V4L2_PIX_FMT_SRGGB10:
+ buf[0][offset] = odd ? g_u << 2 : r_y << 2;
+ buf[0][offset + 1] = odd ? g_u >> 6 : r_y >> 6;
+ buf[1][offset] = odd ? b_v << 2 : g_u << 2;
+ buf[1][offset + 1] = odd ? b_v >> 6 : g_u >> 6;
+ buf[0][offset] |= (buf[0][offset] >> 2) & 3;
+ buf[1][offset] |= (buf[1][offset] >> 2) & 3;
+ break;
+ case V4L2_PIX_FMT_SBGGR12:
+ buf[0][offset] = odd ? g_u << 4 : b_v << 4;
+ buf[0][offset + 1] = odd ? g_u >> 4 : b_v >> 4;
+ buf[1][offset] = odd ? r_y << 4 : g_u << 4;
+ buf[1][offset + 1] = odd ? r_y >> 4 : g_u >> 4;
+ buf[0][offset] |= (buf[0][offset] >> 4) & 0xf;
+ buf[1][offset] |= (buf[1][offset] >> 4) & 0xf;
+ break;
+ case V4L2_PIX_FMT_SGBRG12:
+ buf[0][offset] = odd ? b_v << 4 : g_u << 4;
+ buf[0][offset + 1] = odd ? b_v >> 4 : g_u >> 4;
+ buf[1][offset] = odd ? g_u << 4 : r_y << 4;
+ buf[1][offset + 1] = odd ? g_u >> 4 : r_y >> 4;
+ buf[0][offset] |= (buf[0][offset] >> 4) & 0xf;
+ buf[1][offset] |= (buf[1][offset] >> 4) & 0xf;
+ break;
+ case V4L2_PIX_FMT_SGRBG12:
+ buf[0][offset] = odd ? r_y << 4 : g_u << 4;
+ buf[0][offset + 1] = odd ? r_y >> 4 : g_u >> 4;
+ buf[1][offset] = odd ? g_u << 4 : b_v << 4;
+ buf[1][offset + 1] = odd ? g_u >> 4 : b_v >> 4;
+ buf[0][offset] |= (buf[0][offset] >> 4) & 0xf;
+ buf[1][offset] |= (buf[1][offset] >> 4) & 0xf;
+ break;
+ case V4L2_PIX_FMT_SRGGB12:
+ buf[0][offset] = odd ? g_u << 4 : r_y << 4;
+ buf[0][offset + 1] = odd ? g_u >> 4 : r_y >> 4;
+ buf[1][offset] = odd ? b_v << 4 : g_u << 4;
+ buf[1][offset + 1] = odd ? b_v >> 4 : g_u >> 4;
+ buf[0][offset] |= (buf[0][offset] >> 4) & 0xf;
+ buf[1][offset] |= (buf[1][offset] >> 4) & 0xf;
+ break;
+ }
+}
+
+unsigned tpg_g_interleaved_plane(const struct tpg_data *tpg, unsigned buf_line)
+{
+ switch (tpg->fourcc) {
+ case V4L2_PIX_FMT_SBGGR8:
+ case V4L2_PIX_FMT_SGBRG8:
+ case V4L2_PIX_FMT_SGRBG8:
+ case V4L2_PIX_FMT_SRGGB8:
+ case V4L2_PIX_FMT_SBGGR10:
+ case V4L2_PIX_FMT_SGBRG10:
+ case V4L2_PIX_FMT_SGRBG10:
+ case V4L2_PIX_FMT_SRGGB10:
+ case V4L2_PIX_FMT_SBGGR12:
+ case V4L2_PIX_FMT_SGBRG12:
+ case V4L2_PIX_FMT_SGRBG12:
+ case V4L2_PIX_FMT_SRGGB12:
+ return buf_line & 1;
+ default:
+ return 0;
+ }
+}
+EXPORT_SYMBOL_GPL(tpg_g_interleaved_plane);
+
+/* Return how many pattern lines are used by the current pattern. */
+static unsigned tpg_get_pat_lines(const struct tpg_data *tpg)
+{
+ switch (tpg->pattern) {
+ case TPG_PAT_CHECKERS_16X16:
+ case TPG_PAT_CHECKERS_2X2:
+ case TPG_PAT_CHECKERS_1X1:
+ case TPG_PAT_COLOR_CHECKERS_2X2:
+ case TPG_PAT_COLOR_CHECKERS_1X1:
+ case TPG_PAT_ALTERNATING_HLINES:
+ case TPG_PAT_CROSS_1_PIXEL:
+ case TPG_PAT_CROSS_2_PIXELS:
+ case TPG_PAT_CROSS_10_PIXELS:
+ return 2;
+ case TPG_PAT_100_COLORSQUARES:
+ case TPG_PAT_100_HCOLORBAR:
+ return 8;
+ default:
+ return 1;
+ }
+}
+
+/* Which pattern line should be used for the given frame line. */
+static unsigned tpg_get_pat_line(const struct tpg_data *tpg, unsigned line)
+{
+ switch (tpg->pattern) {
+ case TPG_PAT_CHECKERS_16X16:
+ return (line >> 4) & 1;
+ case TPG_PAT_CHECKERS_1X1:
+ case TPG_PAT_COLOR_CHECKERS_1X1:
+ case TPG_PAT_ALTERNATING_HLINES:
+ return line & 1;
+ case TPG_PAT_CHECKERS_2X2:
+ case TPG_PAT_COLOR_CHECKERS_2X2:
+ return (line & 2) >> 1;
+ case TPG_PAT_100_COLORSQUARES:
+ case TPG_PAT_100_HCOLORBAR:
+ return (line * 8) / tpg->src_height;
+ case TPG_PAT_CROSS_1_PIXEL:
+ return line == tpg->src_height / 2;
+ case TPG_PAT_CROSS_2_PIXELS:
+ return (line + 1) / 2 == tpg->src_height / 4;
+ case TPG_PAT_CROSS_10_PIXELS:
+ return (line + 10) / 20 == tpg->src_height / 40;
+ default:
+ return 0;
+ }
+}
+
+/*
+ * Which color should be used for the given pattern line and X coordinate.
+ * Note: x is in the range 0 to 2 * tpg->src_width.
+ */
+static enum tpg_color tpg_get_color(const struct tpg_data *tpg,
+ unsigned pat_line, unsigned x)
+{
+ /* Maximum number of bars are TPG_COLOR_MAX - otherwise, the input print code
+ should be modified */
+ static const enum tpg_color bars[3][8] = {
+ /* Standard ITU-R 75% color bar sequence */
+ { TPG_COLOR_CSC_WHITE, TPG_COLOR_75_YELLOW,
+ TPG_COLOR_75_CYAN, TPG_COLOR_75_GREEN,
+ TPG_COLOR_75_MAGENTA, TPG_COLOR_75_RED,
+ TPG_COLOR_75_BLUE, TPG_COLOR_100_BLACK, },
+ /* Standard ITU-R 100% color bar sequence */
+ { TPG_COLOR_100_WHITE, TPG_COLOR_100_YELLOW,
+ TPG_COLOR_100_CYAN, TPG_COLOR_100_GREEN,
+ TPG_COLOR_100_MAGENTA, TPG_COLOR_100_RED,
+ TPG_COLOR_100_BLUE, TPG_COLOR_100_BLACK, },
+ /* Color bar sequence suitable to test CSC */
+ { TPG_COLOR_CSC_WHITE, TPG_COLOR_CSC_YELLOW,
+ TPG_COLOR_CSC_CYAN, TPG_COLOR_CSC_GREEN,
+ TPG_COLOR_CSC_MAGENTA, TPG_COLOR_CSC_RED,
+ TPG_COLOR_CSC_BLUE, TPG_COLOR_CSC_BLACK, },
+ };
+
+ switch (tpg->pattern) {
+ case TPG_PAT_75_COLORBAR:
+ case TPG_PAT_100_COLORBAR:
+ case TPG_PAT_CSC_COLORBAR:
+ return bars[tpg->pattern][((x * 8) / tpg->src_width) % 8];
+ case TPG_PAT_100_COLORSQUARES:
+ return bars[1][(pat_line + (x * 8) / tpg->src_width) % 8];
+ case TPG_PAT_100_HCOLORBAR:
+ return bars[1][pat_line];
+ case TPG_PAT_BLACK:
+ return TPG_COLOR_100_BLACK;
+ case TPG_PAT_WHITE:
+ return TPG_COLOR_100_WHITE;
+ case TPG_PAT_RED:
+ return TPG_COLOR_100_RED;
+ case TPG_PAT_GREEN:
+ return TPG_COLOR_100_GREEN;
+ case TPG_PAT_BLUE:
+ return TPG_COLOR_100_BLUE;
+ case TPG_PAT_CHECKERS_16X16:
+ return (((x >> 4) & 1) ^ (pat_line & 1)) ?
+ TPG_COLOR_100_BLACK : TPG_COLOR_100_WHITE;
+ case TPG_PAT_CHECKERS_1X1:
+ return ((x & 1) ^ (pat_line & 1)) ?
+ TPG_COLOR_100_WHITE : TPG_COLOR_100_BLACK;
+ case TPG_PAT_COLOR_CHECKERS_1X1:
+ return ((x & 1) ^ (pat_line & 1)) ?
+ TPG_COLOR_100_RED : TPG_COLOR_100_BLUE;
+ case TPG_PAT_CHECKERS_2X2:
+ return (((x >> 1) & 1) ^ (pat_line & 1)) ?
+ TPG_COLOR_100_WHITE : TPG_COLOR_100_BLACK;
+ case TPG_PAT_COLOR_CHECKERS_2X2:
+ return (((x >> 1) & 1) ^ (pat_line & 1)) ?
+ TPG_COLOR_100_RED : TPG_COLOR_100_BLUE;
+ case TPG_PAT_ALTERNATING_HLINES:
+ return pat_line ? TPG_COLOR_100_WHITE : TPG_COLOR_100_BLACK;
+ case TPG_PAT_ALTERNATING_VLINES:
+ return (x & 1) ? TPG_COLOR_100_WHITE : TPG_COLOR_100_BLACK;
+ case TPG_PAT_CROSS_1_PIXEL:
+ if (pat_line || (x % tpg->src_width) == tpg->src_width / 2)
+ return TPG_COLOR_100_BLACK;
+ return TPG_COLOR_100_WHITE;
+ case TPG_PAT_CROSS_2_PIXELS:
+ if (pat_line || ((x % tpg->src_width) + 1) / 2 == tpg->src_width / 4)
+ return TPG_COLOR_100_BLACK;
+ return TPG_COLOR_100_WHITE;
+ case TPG_PAT_CROSS_10_PIXELS:
+ if (pat_line || ((x % tpg->src_width) + 10) / 20 == tpg->src_width / 40)
+ return TPG_COLOR_100_BLACK;
+ return TPG_COLOR_100_WHITE;
+ case TPG_PAT_GRAY_RAMP:
+ return TPG_COLOR_RAMP + ((x % tpg->src_width) * 256) / tpg->src_width;
+ default:
+ return TPG_COLOR_100_RED;
+ }
+}
+
+/*
+ * Given the pixel aspect ratio and video aspect ratio calculate the
+ * coordinates of a centered square and the coordinates of the border of
+ * the active video area. The coordinates are relative to the source
+ * frame rectangle.
+ */
+static void tpg_calculate_square_border(struct tpg_data *tpg)
+{
+ unsigned w = tpg->src_width;
+ unsigned h = tpg->src_height;
+ unsigned sq_w, sq_h;
+
+ sq_w = (w * 2 / 5) & ~1;
+ if (((w - sq_w) / 2) & 1)
+ sq_w += 2;
+ sq_h = sq_w;
+ tpg->square.width = sq_w;
+ if (tpg->vid_aspect == TPG_VIDEO_ASPECT_16X9_ANAMORPHIC) {
+ unsigned ana_sq_w = (sq_w / 4) * 3;
+
+ if (((w - ana_sq_w) / 2) & 1)
+ ana_sq_w += 2;
+ tpg->square.width = ana_sq_w;
+ }
+ tpg->square.left = (w - tpg->square.width) / 2;
+ if (tpg->pix_aspect == TPG_PIXEL_ASPECT_NTSC)
+ sq_h = sq_w * 10 / 11;
+ else if (tpg->pix_aspect == TPG_PIXEL_ASPECT_PAL)
+ sq_h = sq_w * 59 / 54;
+ tpg->square.height = sq_h;
+ tpg->square.top = (h - sq_h) / 2;
+ tpg->border.left = 0;
+ tpg->border.width = w;
+ tpg->border.top = 0;
+ tpg->border.height = h;
+ switch (tpg->vid_aspect) {
+ case TPG_VIDEO_ASPECT_4X3:
+ if (tpg->pix_aspect)
+ return;
+ if (3 * w >= 4 * h) {
+ tpg->border.width = ((4 * h) / 3) & ~1;
+ if (((w - tpg->border.width) / 2) & ~1)
+ tpg->border.width -= 2;
+ tpg->border.left = (w - tpg->border.width) / 2;
+ break;
+ }
+ tpg->border.height = ((3 * w) / 4) & ~1;
+ tpg->border.top = (h - tpg->border.height) / 2;
+ break;
+ case TPG_VIDEO_ASPECT_14X9_CENTRE:
+ if (tpg->pix_aspect) {
+ tpg->border.height = tpg->pix_aspect == TPG_PIXEL_ASPECT_NTSC ? 420 : 506;
+ tpg->border.top = (h - tpg->border.height) / 2;
+ break;
+ }
+ if (9 * w >= 14 * h) {
+ tpg->border.width = ((14 * h) / 9) & ~1;
+ if (((w - tpg->border.width) / 2) & ~1)
+ tpg->border.width -= 2;
+ tpg->border.left = (w - tpg->border.width) / 2;
+ break;
+ }
+ tpg->border.height = ((9 * w) / 14) & ~1;
+ tpg->border.top = (h - tpg->border.height) / 2;
+ break;
+ case TPG_VIDEO_ASPECT_16X9_CENTRE:
+ if (tpg->pix_aspect) {
+ tpg->border.height = tpg->pix_aspect == TPG_PIXEL_ASPECT_NTSC ? 368 : 442;
+ tpg->border.top = (h - tpg->border.height) / 2;
+ break;
+ }
+ if (9 * w >= 16 * h) {
+ tpg->border.width = ((16 * h) / 9) & ~1;
+ if (((w - tpg->border.width) / 2) & ~1)
+ tpg->border.width -= 2;
+ tpg->border.left = (w - tpg->border.width) / 2;
+ break;
+ }
+ tpg->border.height = ((9 * w) / 16) & ~1;
+ tpg->border.top = (h - tpg->border.height) / 2;
+ break;
+ default:
+ break;
+ }
+}
+
+static void tpg_precalculate_line(struct tpg_data *tpg)
+{
+ enum tpg_color contrast;
+ u8 pix[TPG_MAX_PLANES][8];
+ unsigned pat;
+ unsigned p;
+ unsigned x;
+
+ switch (tpg->pattern) {
+ case TPG_PAT_GREEN:
+ contrast = TPG_COLOR_100_RED;
+ break;
+ case TPG_PAT_CSC_COLORBAR:
+ contrast = TPG_COLOR_CSC_GREEN;
+ break;
+ default:
+ contrast = TPG_COLOR_100_GREEN;
+ break;
+ }
+
+ for (pat = 0; pat < tpg_get_pat_lines(tpg); pat++) {
+ /* Coarse scaling with Bresenham */
+ unsigned int_part = tpg->src_width / tpg->scaled_width;
+ unsigned fract_part = tpg->src_width % tpg->scaled_width;
+ unsigned src_x = 0;
+ unsigned error = 0;
+
+ for (x = 0; x < tpg->scaled_width * 2; x += 2) {
+ unsigned real_x = src_x;
+ enum tpg_color color1, color2;
+
+ real_x = tpg->hflip ? tpg->src_width * 2 - real_x - 2 : real_x;
+ color1 = tpg_get_color(tpg, pat, real_x);
+
+ src_x += int_part;
+ error += fract_part;
+ if (error >= tpg->scaled_width) {
+ error -= tpg->scaled_width;
+ src_x++;
+ }
+
+ real_x = src_x;
+ real_x = tpg->hflip ? tpg->src_width * 2 - real_x - 2 : real_x;
+ color2 = tpg_get_color(tpg, pat, real_x);
+
+ src_x += int_part;
+ error += fract_part;
+ if (error >= tpg->scaled_width) {
+ error -= tpg->scaled_width;
+ src_x++;
+ }
+
+ gen_twopix(tpg, pix, tpg->hflip ? color2 : color1, 0);
+ gen_twopix(tpg, pix, tpg->hflip ? color1 : color2, 1);
+ for (p = 0; p < tpg->planes; p++) {
+ unsigned twopixsize = tpg->twopixelsize[p];
+ unsigned hdiv = tpg->hdownsampling[p];
+ u8 *pos = tpg->lines[pat][p] + tpg_hdiv(tpg, p, x);
+
+ memcpy(pos, pix[p], twopixsize / hdiv);
+ }
+ }
+ }
+
+ if (tpg->vdownsampling[tpg->planes - 1] > 1) {
+ unsigned pat_lines = tpg_get_pat_lines(tpg);
+
+ for (pat = 0; pat < pat_lines; pat++) {
+ unsigned next_pat = (pat + 1) % pat_lines;
+
+ for (p = 1; p < tpg->planes; p++) {
+ unsigned w = tpg_hdiv(tpg, p, tpg->scaled_width * 2);
+ u8 *pos1 = tpg->lines[pat][p];
+ u8 *pos2 = tpg->lines[next_pat][p];
+ u8 *dest = tpg->downsampled_lines[pat][p];
+
+ for (x = 0; x < w; x++, pos1++, pos2++, dest++)
+ *dest = ((u16)*pos1 + (u16)*pos2) / 2;
+ }
+ }
+ }
+
+ gen_twopix(tpg, pix, contrast, 0);
+ gen_twopix(tpg, pix, contrast, 1);
+ for (p = 0; p < tpg->planes; p++) {
+ unsigned twopixsize = tpg->twopixelsize[p];
+ u8 *pos = tpg->contrast_line[p];
+
+ for (x = 0; x < tpg->scaled_width; x += 2, pos += twopixsize)
+ memcpy(pos, pix[p], twopixsize);
+ }
+
+ gen_twopix(tpg, pix, TPG_COLOR_100_BLACK, 0);
+ gen_twopix(tpg, pix, TPG_COLOR_100_BLACK, 1);
+ for (p = 0; p < tpg->planes; p++) {
+ unsigned twopixsize = tpg->twopixelsize[p];
+ u8 *pos = tpg->black_line[p];
+
+ for (x = 0; x < tpg->scaled_width; x += 2, pos += twopixsize)
+ memcpy(pos, pix[p], twopixsize);
+ }
+
+ for (x = 0; x < tpg->scaled_width * 2; x += 2) {
+ gen_twopix(tpg, pix, TPG_COLOR_RANDOM, 0);
+ gen_twopix(tpg, pix, TPG_COLOR_RANDOM, 1);
+ for (p = 0; p < tpg->planes; p++) {
+ unsigned twopixsize = tpg->twopixelsize[p];
+ u8 *pos = tpg->random_line[p] + x * twopixsize / 2;
+
+ memcpy(pos, pix[p], twopixsize);
+ }
+ }
+
+ gen_twopix(tpg, tpg->textbg, TPG_COLOR_TEXTBG, 0);
+ gen_twopix(tpg, tpg->textbg, TPG_COLOR_TEXTBG, 1);
+ gen_twopix(tpg, tpg->textfg, TPG_COLOR_TEXTFG, 0);
+ gen_twopix(tpg, tpg->textfg, TPG_COLOR_TEXTFG, 1);
+}
+
+/* need this to do rgb24 rendering */
+typedef struct { u16 __; u8 _; } __packed x24;
+
+#define PRINTSTR(PIXTYPE) do { \
+ unsigned vdiv = tpg->vdownsampling[p]; \
+ unsigned hdiv = tpg->hdownsampling[p]; \
+ int line; \
+ PIXTYPE fg; \
+ PIXTYPE bg; \
+ memcpy(&fg, tpg->textfg[p], sizeof(PIXTYPE)); \
+ memcpy(&bg, tpg->textbg[p], sizeof(PIXTYPE)); \
+ \
+ for (line = first; line < 16; line += vdiv * step) { \
+ int l = tpg->vflip ? 15 - line : line; \
+ PIXTYPE *pos = (PIXTYPE *)(basep[p][(line / vdiv) & 1] + \
+ ((y * step + l) / (vdiv * div)) * tpg->bytesperline[p] + \
+ (x / hdiv) * sizeof(PIXTYPE)); \
+ unsigned s; \
+ \
+ for (s = 0; s < len; s++) { \
+ u8 chr = font8x16[text[s] * 16 + line]; \
+ \
+ if (hdiv == 2 && tpg->hflip) { \
+ pos[3] = (chr & (0x01 << 6) ? fg : bg); \
+ pos[2] = (chr & (0x01 << 4) ? fg : bg); \
+ pos[1] = (chr & (0x01 << 2) ? fg : bg); \
+ pos[0] = (chr & (0x01 << 0) ? fg : bg); \
+ } else if (hdiv == 2) { \
+ pos[0] = (chr & (0x01 << 7) ? fg : bg); \
+ pos[1] = (chr & (0x01 << 5) ? fg : bg); \
+ pos[2] = (chr & (0x01 << 3) ? fg : bg); \
+ pos[3] = (chr & (0x01 << 1) ? fg : bg); \
+ } else if (tpg->hflip) { \
+ pos[7] = (chr & (0x01 << 7) ? fg : bg); \
+ pos[6] = (chr & (0x01 << 6) ? fg : bg); \
+ pos[5] = (chr & (0x01 << 5) ? fg : bg); \
+ pos[4] = (chr & (0x01 << 4) ? fg : bg); \
+ pos[3] = (chr & (0x01 << 3) ? fg : bg); \
+ pos[2] = (chr & (0x01 << 2) ? fg : bg); \
+ pos[1] = (chr & (0x01 << 1) ? fg : bg); \
+ pos[0] = (chr & (0x01 << 0) ? fg : bg); \
+ } else { \
+ pos[0] = (chr & (0x01 << 7) ? fg : bg); \
+ pos[1] = (chr & (0x01 << 6) ? fg : bg); \
+ pos[2] = (chr & (0x01 << 5) ? fg : bg); \
+ pos[3] = (chr & (0x01 << 4) ? fg : bg); \
+ pos[4] = (chr & (0x01 << 3) ? fg : bg); \
+ pos[5] = (chr & (0x01 << 2) ? fg : bg); \
+ pos[6] = (chr & (0x01 << 1) ? fg : bg); \
+ pos[7] = (chr & (0x01 << 0) ? fg : bg); \
+ } \
+ \
+ pos += (tpg->hflip ? -8 : 8) / hdiv; \
+ } \
+ } \
+} while (0)
+
+static noinline void tpg_print_str_2(const struct tpg_data *tpg, u8 *basep[TPG_MAX_PLANES][2],
+ unsigned p, unsigned first, unsigned div, unsigned step,
+ int y, int x, char *text, unsigned len)
+{
+ PRINTSTR(u8);
+}
+
+static noinline void tpg_print_str_4(const struct tpg_data *tpg, u8 *basep[TPG_MAX_PLANES][2],
+ unsigned p, unsigned first, unsigned div, unsigned step,
+ int y, int x, char *text, unsigned len)
+{
+ PRINTSTR(u16);
+}
+
+static noinline void tpg_print_str_6(const struct tpg_data *tpg, u8 *basep[TPG_MAX_PLANES][2],
+ unsigned p, unsigned first, unsigned div, unsigned step,
+ int y, int x, char *text, unsigned len)
+{
+ PRINTSTR(x24);
+}
+
+static noinline void tpg_print_str_8(const struct tpg_data *tpg, u8 *basep[TPG_MAX_PLANES][2],
+ unsigned p, unsigned first, unsigned div, unsigned step,
+ int y, int x, char *text, unsigned len)
+{
+ PRINTSTR(u32);
+}
+
+void tpg_gen_text(const struct tpg_data *tpg, u8 *basep[TPG_MAX_PLANES][2],
+ int y, int x, char *text)
+{
+ unsigned step = V4L2_FIELD_HAS_T_OR_B(tpg->field) ? 2 : 1;
+ unsigned div = step;
+ unsigned first = 0;
+ unsigned len = strlen(text);
+ unsigned p;
+
+ if (font8x16 == NULL || basep == NULL)
+ return;
+
+ /* Checks if it is possible to show string */
+ if (y + 16 >= tpg->compose.height || x + 8 >= tpg->compose.width)
+ return;
+
+ if (len > (tpg->compose.width - x) / 8)
+ len = (tpg->compose.width - x) / 8;
+ if (tpg->vflip)
+ y = tpg->compose.height - y - 16;
+ if (tpg->hflip)
+ x = tpg->compose.width - x - 8;
+ y += tpg->compose.top;
+ x += tpg->compose.left;
+ if (tpg->field == V4L2_FIELD_BOTTOM)
+ first = 1;
+ else if (tpg->field == V4L2_FIELD_SEQ_TB || tpg->field == V4L2_FIELD_SEQ_BT)
+ div = 2;
+
+ for (p = 0; p < tpg->planes; p++) {
+ /* Print text */
+ switch (tpg->twopixelsize[p]) {
+ case 2:
+ tpg_print_str_2(tpg, basep, p, first, div, step, y, x,
+ text, len);
+ break;
+ case 4:
+ tpg_print_str_4(tpg, basep, p, first, div, step, y, x,
+ text, len);
+ break;
+ case 6:
+ tpg_print_str_6(tpg, basep, p, first, div, step, y, x,
+ text, len);
+ break;
+ case 8:
+ tpg_print_str_8(tpg, basep, p, first, div, step, y, x,
+ text, len);
+ break;
+ }
+ }
+}
+EXPORT_SYMBOL_GPL(tpg_gen_text);
+
+void tpg_update_mv_step(struct tpg_data *tpg)
+{
+ int factor = tpg->mv_hor_mode > TPG_MOVE_NONE ? -1 : 1;
+
+ if (tpg->hflip)
+ factor = -factor;
+ switch (tpg->mv_hor_mode) {
+ case TPG_MOVE_NEG_FAST:
+ case TPG_MOVE_POS_FAST:
+ tpg->mv_hor_step = ((tpg->src_width + 319) / 320) * 4;
+ break;
+ case TPG_MOVE_NEG:
+ case TPG_MOVE_POS:
+ tpg->mv_hor_step = ((tpg->src_width + 639) / 640) * 4;
+ break;
+ case TPG_MOVE_NEG_SLOW:
+ case TPG_MOVE_POS_SLOW:
+ tpg->mv_hor_step = 2;
+ break;
+ case TPG_MOVE_NONE:
+ tpg->mv_hor_step = 0;
+ break;
+ }
+ if (factor < 0)
+ tpg->mv_hor_step = tpg->src_width - tpg->mv_hor_step;
+
+ factor = tpg->mv_vert_mode > TPG_MOVE_NONE ? -1 : 1;
+ switch (tpg->mv_vert_mode) {
+ case TPG_MOVE_NEG_FAST:
+ case TPG_MOVE_POS_FAST:
+ tpg->mv_vert_step = ((tpg->src_width + 319) / 320) * 4;
+ break;
+ case TPG_MOVE_NEG:
+ case TPG_MOVE_POS:
+ tpg->mv_vert_step = ((tpg->src_width + 639) / 640) * 4;
+ break;
+ case TPG_MOVE_NEG_SLOW:
+ case TPG_MOVE_POS_SLOW:
+ tpg->mv_vert_step = 1;
+ break;
+ case TPG_MOVE_NONE:
+ tpg->mv_vert_step = 0;
+ break;
+ }
+ if (factor < 0)
+ tpg->mv_vert_step = tpg->src_height - tpg->mv_vert_step;
+}
+EXPORT_SYMBOL_GPL(tpg_update_mv_step);
+
+/* Map the line number relative to the crop rectangle to a frame line number */
+static unsigned tpg_calc_frameline(const struct tpg_data *tpg, unsigned src_y,
+ unsigned field)
+{
+ switch (field) {
+ case V4L2_FIELD_TOP:
+ return tpg->crop.top + src_y * 2;
+ case V4L2_FIELD_BOTTOM:
+ return tpg->crop.top + src_y * 2 + 1;
+ default:
+ return src_y + tpg->crop.top;
+ }
+}
+
+/*
+ * Map the line number relative to the compose rectangle to a destination
+ * buffer line number.
+ */
+static unsigned tpg_calc_buffer_line(const struct tpg_data *tpg, unsigned y,
+ unsigned field)
+{
+ y += tpg->compose.top;
+ switch (field) {
+ case V4L2_FIELD_SEQ_TB:
+ if (y & 1)
+ return tpg->buf_height / 2 + y / 2;
+ return y / 2;
+ case V4L2_FIELD_SEQ_BT:
+ if (y & 1)
+ return y / 2;
+ return tpg->buf_height / 2 + y / 2;
+ default:
+ return y;
+ }
+}
+
+static void tpg_recalc(struct tpg_data *tpg)
+{
+ if (tpg->recalc_colors) {
+ tpg->recalc_colors = false;
+ tpg->recalc_lines = true;
+ tpg->real_xfer_func = tpg->xfer_func;
+ tpg->real_ycbcr_enc = tpg->ycbcr_enc;
+ tpg->real_quantization = tpg->quantization;
+
+ if (tpg->xfer_func == V4L2_XFER_FUNC_DEFAULT)
+ tpg->real_xfer_func =
+ V4L2_MAP_XFER_FUNC_DEFAULT(tpg->colorspace);
+
+ if (tpg->ycbcr_enc == V4L2_YCBCR_ENC_DEFAULT)
+ tpg->real_ycbcr_enc =
+ V4L2_MAP_YCBCR_ENC_DEFAULT(tpg->colorspace);
+
+ if (tpg->quantization == V4L2_QUANTIZATION_DEFAULT)
+ tpg->real_quantization =
+ V4L2_MAP_QUANTIZATION_DEFAULT(!tpg->is_yuv,
+ tpg->colorspace, tpg->real_ycbcr_enc);
+
+ tpg_precalculate_colors(tpg);
+ }
+ if (tpg->recalc_square_border) {
+ tpg->recalc_square_border = false;
+ tpg_calculate_square_border(tpg);
+ }
+ if (tpg->recalc_lines) {
+ tpg->recalc_lines = false;
+ tpg_precalculate_line(tpg);
+ }
+}
+
+void tpg_calc_text_basep(struct tpg_data *tpg,
+ u8 *basep[TPG_MAX_PLANES][2], unsigned p, u8 *vbuf)
+{
+ unsigned stride = tpg->bytesperline[p];
+ unsigned h = tpg->buf_height;
+
+ tpg_recalc(tpg);
+
+ basep[p][0] = vbuf;
+ basep[p][1] = vbuf;
+ h /= tpg->vdownsampling[p];
+ if (tpg->field == V4L2_FIELD_SEQ_TB)
+ basep[p][1] += h * stride / 2;
+ else if (tpg->field == V4L2_FIELD_SEQ_BT)
+ basep[p][0] += h * stride / 2;
+ if (p == 0 && tpg->interleaved)
+ tpg_calc_text_basep(tpg, basep, 1, vbuf);
+}
+EXPORT_SYMBOL_GPL(tpg_calc_text_basep);
+
+static int tpg_pattern_avg(const struct tpg_data *tpg,
+ unsigned pat1, unsigned pat2)
+{
+ unsigned pat_lines = tpg_get_pat_lines(tpg);
+
+ if (pat1 == (pat2 + 1) % pat_lines)
+ return pat2;
+ if (pat2 == (pat1 + 1) % pat_lines)
+ return pat1;
+ return -1;
+}
+
+void tpg_log_status(struct tpg_data *tpg)
+{
+ pr_info("tpg source WxH: %ux%u (%s)\n",
+ tpg->src_width, tpg->src_height,
+ tpg->is_yuv ? "YCbCr" : "RGB");
+ pr_info("tpg field: %u\n", tpg->field);
+ pr_info("tpg crop: %ux%u@%dx%d\n", tpg->crop.width, tpg->crop.height,
+ tpg->crop.left, tpg->crop.top);
+ pr_info("tpg compose: %ux%u@%dx%d\n", tpg->compose.width, tpg->compose.height,
+ tpg->compose.left, tpg->compose.top);
+ pr_info("tpg colorspace: %d\n", tpg->colorspace);
+ pr_info("tpg transfer function: %d/%d\n", tpg->xfer_func, tpg->real_xfer_func);
+ pr_info("tpg Y'CbCr encoding: %d/%d\n", tpg->ycbcr_enc, tpg->real_ycbcr_enc);
+ pr_info("tpg quantization: %d/%d\n", tpg->quantization, tpg->real_quantization);
+ pr_info("tpg RGB range: %d/%d\n", tpg->rgb_range, tpg->real_rgb_range);
+}
+EXPORT_SYMBOL_GPL(tpg_log_status);
+
+/*
+ * This struct contains common parameters used by both the drawing of the
+ * test pattern and the drawing of the extras (borders, square, etc.)
+ */
+struct tpg_draw_params {
+ /* common data */
+ bool is_tv;
+ bool is_60hz;
+ unsigned twopixsize;
+ unsigned img_width;
+ unsigned stride;
+ unsigned hmax;
+ unsigned frame_line;
+ unsigned frame_line_next;
+
+ /* test pattern */
+ unsigned mv_hor_old;
+ unsigned mv_hor_new;
+ unsigned mv_vert_old;
+ unsigned mv_vert_new;
+
+ /* extras */
+ unsigned wss_width;
+ unsigned wss_random_offset;
+ unsigned sav_eav_f;
+ unsigned left_pillar_width;
+ unsigned right_pillar_start;
+};
+
+static void tpg_fill_params_pattern(const struct tpg_data *tpg, unsigned p,
+ struct tpg_draw_params *params)
+{
+ params->mv_hor_old =
+ tpg_hscale_div(tpg, p, tpg->mv_hor_count % tpg->src_width);
+ params->mv_hor_new =
+ tpg_hscale_div(tpg, p, (tpg->mv_hor_count + tpg->mv_hor_step) %
+ tpg->src_width);
+ params->mv_vert_old = tpg->mv_vert_count % tpg->src_height;
+ params->mv_vert_new =
+ (tpg->mv_vert_count + tpg->mv_vert_step) % tpg->src_height;
+}
+
+static void tpg_fill_params_extras(const struct tpg_data *tpg,
+ unsigned p,
+ struct tpg_draw_params *params)
+{
+ unsigned left_pillar_width = 0;
+ unsigned right_pillar_start = params->img_width;
+
+ params->wss_width = tpg->crop.left < tpg->src_width / 2 ?
+ tpg->src_width / 2 - tpg->crop.left : 0;
+ if (params->wss_width > tpg->crop.width)
+ params->wss_width = tpg->crop.width;
+ params->wss_width = tpg_hscale_div(tpg, p, params->wss_width);
+ params->wss_random_offset =
+ params->twopixsize * prandom_u32_max(tpg->src_width / 2);
+
+ if (tpg->crop.left < tpg->border.left) {
+ left_pillar_width = tpg->border.left - tpg->crop.left;
+ if (left_pillar_width > tpg->crop.width)
+ left_pillar_width = tpg->crop.width;
+ left_pillar_width = tpg_hscale_div(tpg, p, left_pillar_width);
+ }
+ params->left_pillar_width = left_pillar_width;
+
+ if (tpg->crop.left + tpg->crop.width >
+ tpg->border.left + tpg->border.width) {
+ right_pillar_start =
+ tpg->border.left + tpg->border.width - tpg->crop.left;
+ right_pillar_start =
+ tpg_hscale_div(tpg, p, right_pillar_start);
+ if (right_pillar_start > params->img_width)
+ right_pillar_start = params->img_width;
+ }
+ params->right_pillar_start = right_pillar_start;
+
+ params->sav_eav_f = tpg->field ==
+ (params->is_60hz ? V4L2_FIELD_TOP : V4L2_FIELD_BOTTOM);
+}
+
+static void tpg_fill_plane_extras(const struct tpg_data *tpg,
+ const struct tpg_draw_params *params,
+ unsigned p, unsigned h, u8 *vbuf)
+{
+ unsigned twopixsize = params->twopixsize;
+ unsigned img_width = params->img_width;
+ unsigned frame_line = params->frame_line;
+ const struct v4l2_rect *sq = &tpg->square;
+ const struct v4l2_rect *b = &tpg->border;
+ const struct v4l2_rect *c = &tpg->crop;
+
+ if (params->is_tv && !params->is_60hz &&
+ frame_line == 0 && params->wss_width) {
+ /*
+ * Replace the first half of the top line of a 50 Hz frame
+ * with random data to simulate a WSS signal.
+ */
+ u8 *wss = tpg->random_line[p] + params->wss_random_offset;
+
+ memcpy(vbuf, wss, params->wss_width);
+ }
+
+ if (tpg->show_border && frame_line >= b->top &&
+ frame_line < b->top + b->height) {
+ unsigned bottom = b->top + b->height - 1;
+ unsigned left = params->left_pillar_width;
+ unsigned right = params->right_pillar_start;
+
+ if (frame_line == b->top || frame_line == b->top + 1 ||
+ frame_line == bottom || frame_line == bottom - 1) {
+ memcpy(vbuf + left, tpg->contrast_line[p],
+ right - left);
+ } else {
+ if (b->left >= c->left &&
+ b->left < c->left + c->width)
+ memcpy(vbuf + left,
+ tpg->contrast_line[p], twopixsize);
+ if (b->left + b->width > c->left &&
+ b->left + b->width <= c->left + c->width)
+ memcpy(vbuf + right - twopixsize,
+ tpg->contrast_line[p], twopixsize);
+ }
+ }
+ if (tpg->qual != TPG_QUAL_NOISE && frame_line >= b->top &&
+ frame_line < b->top + b->height) {
+ memcpy(vbuf, tpg->black_line[p], params->left_pillar_width);
+ memcpy(vbuf + params->right_pillar_start, tpg->black_line[p],
+ img_width - params->right_pillar_start);
+ }
+ if (tpg->show_square && frame_line >= sq->top &&
+ frame_line < sq->top + sq->height &&
+ sq->left < c->left + c->width &&
+ sq->left + sq->width >= c->left) {
+ unsigned left = sq->left;
+ unsigned width = sq->width;
+
+ if (c->left > left) {
+ width -= c->left - left;
+ left = c->left;
+ }
+ if (c->left + c->width < left + width)
+ width -= left + width - c->left - c->width;
+ left -= c->left;
+ left = tpg_hscale_div(tpg, p, left);
+ width = tpg_hscale_div(tpg, p, width);
+ memcpy(vbuf + left, tpg->contrast_line[p], width);
+ }
+ if (tpg->insert_sav) {
+ unsigned offset = tpg_hdiv(tpg, p, tpg->compose.width / 3);
+ u8 *p = vbuf + offset;
+ unsigned vact = 0, hact = 0;
+
+ p[0] = 0xff;
+ p[1] = 0;
+ p[2] = 0;
+ p[3] = 0x80 | (params->sav_eav_f << 6) |
+ (vact << 5) | (hact << 4) |
+ ((hact ^ vact) << 3) |
+ ((hact ^ params->sav_eav_f) << 2) |
+ ((params->sav_eav_f ^ vact) << 1) |
+ (hact ^ vact ^ params->sav_eav_f);
+ }
+ if (tpg->insert_eav) {
+ unsigned offset = tpg_hdiv(tpg, p, tpg->compose.width * 2 / 3);
+ u8 *p = vbuf + offset;
+ unsigned vact = 0, hact = 1;
+
+ p[0] = 0xff;
+ p[1] = 0;
+ p[2] = 0;
+ p[3] = 0x80 | (params->sav_eav_f << 6) |
+ (vact << 5) | (hact << 4) |
+ ((hact ^ vact) << 3) |
+ ((hact ^ params->sav_eav_f) << 2) |
+ ((params->sav_eav_f ^ vact) << 1) |
+ (hact ^ vact ^ params->sav_eav_f);
+ }
+}
+
+static void tpg_fill_plane_pattern(const struct tpg_data *tpg,
+ const struct tpg_draw_params *params,
+ unsigned p, unsigned h, u8 *vbuf)
+{
+ unsigned twopixsize = params->twopixsize;
+ unsigned img_width = params->img_width;
+ unsigned mv_hor_old = params->mv_hor_old;
+ unsigned mv_hor_new = params->mv_hor_new;
+ unsigned mv_vert_old = params->mv_vert_old;
+ unsigned mv_vert_new = params->mv_vert_new;
+ unsigned frame_line = params->frame_line;
+ unsigned frame_line_next = params->frame_line_next;
+ unsigned line_offset = tpg_hscale_div(tpg, p, tpg->crop.left);
+ bool even;
+ bool fill_blank = false;
+ unsigned pat_line_old;
+ unsigned pat_line_new;
+ u8 *linestart_older;
+ u8 *linestart_newer;
+ u8 *linestart_top;
+ u8 *linestart_bottom;
+
+ even = !(frame_line & 1);
+
+ if (h >= params->hmax) {
+ if (params->hmax == tpg->compose.height)
+ return;
+ if (!tpg->perc_fill_blank)
+ return;
+ fill_blank = true;
+ }
+
+ if (tpg->vflip) {
+ frame_line = tpg->src_height - frame_line - 1;
+ frame_line_next = tpg->src_height - frame_line_next - 1;
+ }
+
+ if (fill_blank) {
+ linestart_older = tpg->contrast_line[p];
+ linestart_newer = tpg->contrast_line[p];
+ } else if (tpg->qual != TPG_QUAL_NOISE &&
+ (frame_line < tpg->border.top ||
+ frame_line >= tpg->border.top + tpg->border.height)) {
+ linestart_older = tpg->black_line[p];
+ linestart_newer = tpg->black_line[p];
+ } else if (tpg->pattern == TPG_PAT_NOISE || tpg->qual == TPG_QUAL_NOISE) {
+ linestart_older = tpg->random_line[p] +
+ twopixsize * prandom_u32_max(tpg->src_width / 2);
+ linestart_newer = tpg->random_line[p] +
+ twopixsize * prandom_u32_max(tpg->src_width / 2);
+ } else {
+ unsigned frame_line_old =
+ (frame_line + mv_vert_old) % tpg->src_height;
+ unsigned frame_line_new =
+ (frame_line + mv_vert_new) % tpg->src_height;
+ unsigned pat_line_next_old;
+ unsigned pat_line_next_new;
+
+ pat_line_old = tpg_get_pat_line(tpg, frame_line_old);
+ pat_line_new = tpg_get_pat_line(tpg, frame_line_new);
+ linestart_older = tpg->lines[pat_line_old][p] + mv_hor_old;
+ linestart_newer = tpg->lines[pat_line_new][p] + mv_hor_new;
+
+ if (tpg->vdownsampling[p] > 1 && frame_line != frame_line_next) {
+ int avg_pat;
+
+ /*
+ * Now decide whether we need to use downsampled_lines[].
+ * That's necessary if the two lines use different patterns.
+ */
+ pat_line_next_old = tpg_get_pat_line(tpg,
+ (frame_line_next + mv_vert_old) % tpg->src_height);
+ pat_line_next_new = tpg_get_pat_line(tpg,
+ (frame_line_next + mv_vert_new) % tpg->src_height);
+
+ switch (tpg->field) {
+ case V4L2_FIELD_INTERLACED:
+ case V4L2_FIELD_INTERLACED_BT:
+ case V4L2_FIELD_INTERLACED_TB:
+ avg_pat = tpg_pattern_avg(tpg, pat_line_old, pat_line_new);
+ if (avg_pat < 0)
+ break;
+ linestart_older = tpg->downsampled_lines[avg_pat][p] + mv_hor_old;
+ linestart_newer = linestart_older;
+ break;
+ case V4L2_FIELD_NONE:
+ case V4L2_FIELD_TOP:
+ case V4L2_FIELD_BOTTOM:
+ case V4L2_FIELD_SEQ_BT:
+ case V4L2_FIELD_SEQ_TB:
+ avg_pat = tpg_pattern_avg(tpg, pat_line_old, pat_line_next_old);
+ if (avg_pat >= 0)
+ linestart_older = tpg->downsampled_lines[avg_pat][p] +
+ mv_hor_old;
+ avg_pat = tpg_pattern_avg(tpg, pat_line_new, pat_line_next_new);
+ if (avg_pat >= 0)
+ linestart_newer = tpg->downsampled_lines[avg_pat][p] +
+ mv_hor_new;
+ break;
+ }
+ }
+ linestart_older += line_offset;
+ linestart_newer += line_offset;
+ }
+ if (tpg->field_alternate) {
+ linestart_top = linestart_bottom = linestart_older;
+ } else if (params->is_60hz) {
+ linestart_top = linestart_newer;
+ linestart_bottom = linestart_older;
+ } else {
+ linestart_top = linestart_older;
+ linestart_bottom = linestart_newer;
+ }
+
+ switch (tpg->field) {
+ case V4L2_FIELD_INTERLACED:
+ case V4L2_FIELD_INTERLACED_TB:
+ case V4L2_FIELD_SEQ_TB:
+ case V4L2_FIELD_SEQ_BT:
+ if (even)
+ memcpy(vbuf, linestart_top, img_width);
+ else
+ memcpy(vbuf, linestart_bottom, img_width);
+ break;
+ case V4L2_FIELD_INTERLACED_BT:
+ if (even)
+ memcpy(vbuf, linestart_bottom, img_width);
+ else
+ memcpy(vbuf, linestart_top, img_width);
+ break;
+ case V4L2_FIELD_TOP:
+ memcpy(vbuf, linestart_top, img_width);
+ break;
+ case V4L2_FIELD_BOTTOM:
+ memcpy(vbuf, linestart_bottom, img_width);
+ break;
+ case V4L2_FIELD_NONE:
+ default:
+ memcpy(vbuf, linestart_older, img_width);
+ break;
+ }
+}
+
+void tpg_fill_plane_buffer(struct tpg_data *tpg, v4l2_std_id std,
+ unsigned p, u8 *vbuf)
+{
+ struct tpg_draw_params params;
+ unsigned factor = V4L2_FIELD_HAS_T_OR_B(tpg->field) ? 2 : 1;
+
+ /* Coarse scaling with Bresenham */
+ unsigned int_part = (tpg->crop.height / factor) / tpg->compose.height;
+ unsigned fract_part = (tpg->crop.height / factor) % tpg->compose.height;
+ unsigned src_y = 0;
+ unsigned error = 0;
+ unsigned h;
+
+ tpg_recalc(tpg);
+
+ params.is_tv = std;
+ params.is_60hz = std & V4L2_STD_525_60;
+ params.twopixsize = tpg->twopixelsize[p];
+ params.img_width = tpg_hdiv(tpg, p, tpg->compose.width);
+ params.stride = tpg->bytesperline[p];
+ params.hmax = (tpg->compose.height * tpg->perc_fill) / 100;
+
+ tpg_fill_params_pattern(tpg, p, ¶ms);
+ tpg_fill_params_extras(tpg, p, ¶ms);
+
+ vbuf += tpg_hdiv(tpg, p, tpg->compose.left);
+
+ for (h = 0; h < tpg->compose.height; h++) {
+ unsigned buf_line;
+
+ params.frame_line = tpg_calc_frameline(tpg, src_y, tpg->field);
+ params.frame_line_next = params.frame_line;
+ buf_line = tpg_calc_buffer_line(tpg, h, tpg->field);
+ src_y += int_part;
+ error += fract_part;
+ if (error >= tpg->compose.height) {
+ error -= tpg->compose.height;
+ src_y++;
+ }
+
+ /*
+ * For line-interleaved formats determine the 'plane'
+ * based on the buffer line.
+ */
+ if (tpg_g_interleaved(tpg))
+ p = tpg_g_interleaved_plane(tpg, buf_line);
+
+ if (tpg->vdownsampling[p] > 1) {
+ /*
+ * When doing vertical downsampling the field setting
+ * matters: for SEQ_BT/TB we downsample each field
+ * separately (i.e. lines 0+2 are combined, as are
+ * lines 1+3), for the other field settings we combine
+ * odd and even lines. Doing that for SEQ_BT/TB would
+ * be really weird.
+ */
+ if (tpg->field == V4L2_FIELD_SEQ_BT ||
+ tpg->field == V4L2_FIELD_SEQ_TB) {
+ unsigned next_src_y = src_y;
+
+ if ((h & 3) >= 2)
+ continue;
+ next_src_y += int_part;
+ if (error + fract_part >= tpg->compose.height)
+ next_src_y++;
+ params.frame_line_next =
+ tpg_calc_frameline(tpg, next_src_y, tpg->field);
+ } else {
+ if (h & 1)
+ continue;
+ params.frame_line_next =
+ tpg_calc_frameline(tpg, src_y, tpg->field);
+ }
+
+ buf_line /= tpg->vdownsampling[p];
+ }
+ tpg_fill_plane_pattern(tpg, ¶ms, p, h,
+ vbuf + buf_line * params.stride);
+ tpg_fill_plane_extras(tpg, ¶ms, p, h,
+ vbuf + buf_line * params.stride);
+ }
+}
+EXPORT_SYMBOL_GPL(tpg_fill_plane_buffer);
+
+void tpg_fillbuffer(struct tpg_data *tpg, v4l2_std_id std, unsigned p, u8 *vbuf)
+{
+ unsigned offset = 0;
+ unsigned i;
+
+ if (tpg->buffers > 1) {
+ tpg_fill_plane_buffer(tpg, std, p, vbuf);
+ return;
+ }
+
+ for (i = 0; i < tpg_g_planes(tpg); i++) {
+ tpg_fill_plane_buffer(tpg, std, i, vbuf + offset);
+ offset += tpg_calc_plane_size(tpg, i);
+ }
+}
+EXPORT_SYMBOL_GPL(tpg_fillbuffer);
+
+MODULE_DESCRIPTION("V4L2 Test Pattern Generator");
+MODULE_AUTHOR("Hans Verkuil");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/dvb-core/dvb-usb-ids.h b/drivers/media/dvb-core/dvb-usb-ids.h
index 0afad39..a7a4674 100644
--- a/drivers/media/dvb-core/dvb-usb-ids.h
+++ b/drivers/media/dvb-core/dvb-usb-ids.h
@@ -58,6 +58,14 @@
#define USB_VID_TELESTAR 0x10b9
#define USB_VID_VISIONPLUS 0x13d3
#define USB_VID_SONY 0x1415
+#define USB_PID_TEVII_S421 0xd421
+#define USB_PID_TEVII_S480_1 0xd481
+#define USB_PID_TEVII_S480_2 0xd482
+#define USB_PID_TEVII_S630 0xd630
+#define USB_PID_TEVII_S632 0xd632
+#define USB_PID_TEVII_S650 0xd650
+#define USB_PID_TEVII_S660 0xd660
+#define USB_PID_TEVII_S662 0xd662
#define USB_VID_TWINHAN 0x1822
#define USB_VID_ULTIMA_ELECTRONIC 0x05d8
#define USB_VID_UNIWILL 0x1584
@@ -141,6 +149,7 @@
#define USB_PID_GENIUS_TVGO_DVB_T03 0x4012
#define USB_PID_GRANDTEC_DVBT_USB_COLD 0x0fa0
#define USB_PID_GRANDTEC_DVBT_USB_WARM 0x0fa1
+#define USB_PID_GOTVIEW_SAT_HD 0x5456
#define USB_PID_INTEL_CE9500 0x9500
#define USB_PID_ITETECH_IT9135 0x9135
#define USB_PID_ITETECH_IT9135_9005 0x9005
@@ -159,6 +168,8 @@
#define USB_PID_KWORLD_UB499_2T_T09 0xe409
#define USB_PID_KWORLD_VSTREAM_COLD 0x17de
#define USB_PID_KWORLD_VSTREAM_WARM 0x17df
+#define USB_PID_PROF_1100 0xb012
+#define USB_PID_TERRATEC_CINERGY_S 0x0064
#define USB_PID_TERRATEC_CINERGY_T_USB_XE 0x0055
#define USB_PID_TERRATEC_CINERGY_T_USB_XE_REV2 0x0069
#define USB_PID_TERRATEC_CINERGY_T_STICK 0x0093
@@ -361,6 +372,8 @@
#define USB_PID_YUAN_STK7700D 0x1efc
#define USB_PID_YUAN_STK7700D_2 0x1e8c
#define USB_PID_DW2102 0x2102
+#define USB_PID_DW2104 0x2104
+#define USB_PID_DW3101 0x3101
#define USB_PID_XTENSIONS_XD_380 0x0381
#define USB_PID_TELESTAR_STARSTICK_2 0x8000
#define USB_PID_MSI_DIGI_VOX_MINI_III 0x8807
@@ -373,6 +386,7 @@
#define USB_PID_ELGATO_EYETV_DTT_Dlx 0x0020
#define USB_PID_ELGATO_EYETV_SAT 0x002a
#define USB_PID_ELGATO_EYETV_SAT_V2 0x0025
+#define USB_PID_ELGATO_EYETV_SAT_V3 0x0036
#define USB_PID_DVB_T_USB_STICK_HIGH_SPEED_COLD 0x5000
#define USB_PID_DVB_T_USB_STICK_HIGH_SPEED_WARM 0x5001
#define USB_PID_FRIIO_WHITE 0x0001
diff --git a/drivers/media/dvb-core/dvbdev.c b/drivers/media/dvb-core/dvbdev.c
index e1684c5..75a3f4b 100644
--- a/drivers/media/dvb-core/dvbdev.c
+++ b/drivers/media/dvb-core/dvbdev.c
@@ -676,13 +676,13 @@
demux, 0, MEDIA_LNK_FL_ENABLED,
false);
if (ret)
- return -ENOMEM;
+ return ret;
}
if (demux && ca) {
ret = media_create_pad_link(demux, 1, ca,
0, MEDIA_LNK_FL_ENABLED);
if (ret)
- return -ENOMEM;
+ return ret;
}
/* Create demux links for each ringbuffer/pad */
diff --git a/drivers/media/dvb-frontends/dib0090.c b/drivers/media/dvb-frontends/dib0090.c
index dc2d41e..d879dc0 100644
--- a/drivers/media/dvb-frontends/dib0090.c
+++ b/drivers/media/dvb-frontends/dib0090.c
@@ -1121,7 +1121,7 @@
(state->current_band == BAND_CBAND) ? "CBAND" : "NOT CBAND",
state->identity.version & 0x1f);
- if (rf_ramp && ((state->rf_ramp[0] == 0) ||
+ if (rf_ramp && ((state->rf_ramp && state->rf_ramp[0] == 0) ||
(state->current_band == BAND_CBAND &&
(state->identity.version & 0x1f) <= P1D_E_F))) {
dprintk("DE-Engage mux for direct gain reg control");
diff --git a/drivers/media/dvb-frontends/ds3000.c b/drivers/media/dvb-frontends/ds3000.c
index e8fc032..addffc3 100644
--- a/drivers/media/dvb-frontends/ds3000.c
+++ b/drivers/media/dvb-frontends/ds3000.c
@@ -458,7 +458,7 @@
break;
default:
- return 1;
+ return -EINVAL;
}
if (state->config->set_lock_led)
@@ -528,7 +528,7 @@
*ber = 0xffffffff;
break;
default:
- return 1;
+ return -EINVAL;
}
return 0;
@@ -623,7 +623,7 @@
snr_reading, *snr);
break;
default:
- return 1;
+ return -EINVAL;
}
return 0;
@@ -661,7 +661,7 @@
state->prevUCBS2 = _ucblocks;
break;
default:
- return 1;
+ return -EINVAL;
}
return 0;
@@ -754,7 +754,7 @@
data |= 0x80;
ds3000_writereg(state, 0xa2, data);
- return 1;
+ return -ETIMEDOUT;
}
data = ds3000_readreg(state, 0xa2);
@@ -808,7 +808,7 @@
data |= 0x80;
ds3000_writereg(state, 0xa2, data);
- return 1;
+ return -ETIMEDOUT;
}
data = ds3000_readreg(state, 0xa2);
@@ -951,7 +951,7 @@
ds3000_writereg(state, 0xfe, 0x98);
break;
default:
- return 1;
+ return -EINVAL;
}
/* enable 27MHz clock output */
diff --git a/drivers/media/dvb-frontends/m88ds3103.c b/drivers/media/dvb-frontends/m88ds3103.c
index 7688360..5557ef8 100644
--- a/drivers/media/dvb-frontends/m88ds3103.c
+++ b/drivers/media/dvb-frontends/m88ds3103.c
@@ -1251,9 +1251,9 @@
i2c_unregister_device(client);
}
-static int m88ds3103_select(struct i2c_adapter *adap, void *mux_priv, u32 chan)
+static int m88ds3103_select(struct i2c_mux_core *muxc, u32 chan)
{
- struct m88ds3103_dev *dev = mux_priv;
+ struct m88ds3103_dev *dev = i2c_mux_priv(muxc);
struct i2c_client *client = dev->client;
int ret;
struct i2c_msg msg = {
@@ -1374,7 +1374,7 @@
dev_dbg(&client->dev, "\n");
- return dev->i2c_adapter;
+ return dev->muxc->adapter[0];
}
static int m88ds3103_probe(struct i2c_client *client,
@@ -1467,13 +1467,16 @@
goto err_kfree;
/* create mux i2c adapter for tuner */
- dev->i2c_adapter = i2c_add_mux_adapter(client->adapter, &client->dev,
- dev, 0, 0, 0, m88ds3103_select,
- NULL);
- if (dev->i2c_adapter == NULL) {
+ dev->muxc = i2c_mux_alloc(client->adapter, &client->dev, 1, 0, 0,
+ m88ds3103_select, NULL);
+ if (!dev->muxc) {
ret = -ENOMEM;
goto err_kfree;
}
+ dev->muxc->priv = dev;
+ ret = i2c_mux_add_adapter(dev->muxc, 0, 0, 0);
+ if (ret)
+ goto err_kfree;
/* create dvb_frontend */
memcpy(&dev->fe.ops, &m88ds3103_ops, sizeof(struct dvb_frontend_ops));
@@ -1502,7 +1505,7 @@
dev_dbg(&client->dev, "\n");
- i2c_del_mux_adapter(dev->i2c_adapter);
+ i2c_mux_del_adapters(dev->muxc);
kfree(dev);
return 0;
diff --git a/drivers/media/dvb-frontends/m88ds3103_priv.h b/drivers/media/dvb-frontends/m88ds3103_priv.h
index eee8c22..d78e467 100644
--- a/drivers/media/dvb-frontends/m88ds3103_priv.h
+++ b/drivers/media/dvb-frontends/m88ds3103_priv.h
@@ -42,11 +42,11 @@
enum fe_status fe_status;
u32 dvbv3_ber; /* for old DVBv3 API read_ber */
bool warm; /* FW running */
- struct i2c_adapter *i2c_adapter;
+ struct i2c_mux_core *muxc;
/* auto detect chip id to do different config */
u8 chip_id;
/* main mclk is calculated for M88RS6000 dynamically */
- u32 mclk_khz;
+ s32 mclk_khz;
u64 post_bit_error;
u64 post_bit_count;
};
diff --git a/drivers/media/dvb-frontends/rtl2830.c b/drivers/media/dvb-frontends/rtl2830.c
index 3f96429..d25d1e0 100644
--- a/drivers/media/dvb-frontends/rtl2830.c
+++ b/drivers/media/dvb-frontends/rtl2830.c
@@ -677,9 +677,9 @@
* adapter lock is already taken by tuner driver.
* Gate is closed automatically after single I2C transfer.
*/
-static int rtl2830_select(struct i2c_adapter *adap, void *mux_priv, u32 chan_id)
+static int rtl2830_select(struct i2c_mux_core *muxc, u32 chan_id)
{
- struct i2c_client *client = mux_priv;
+ struct i2c_client *client = i2c_mux_priv(muxc);
struct rtl2830_dev *dev = i2c_get_clientdata(client);
int ret;
@@ -712,7 +712,7 @@
dev_dbg(&client->dev, "\n");
- return dev->adapter;
+ return dev->muxc->adapter[0];
}
/*
@@ -865,12 +865,16 @@
goto err_regmap_exit;
/* create muxed i2c adapter for tuner */
- dev->adapter = i2c_add_mux_adapter(client->adapter, &client->dev,
- client, 0, 0, 0, rtl2830_select, NULL);
- if (dev->adapter == NULL) {
- ret = -ENODEV;
+ dev->muxc = i2c_mux_alloc(client->adapter, &client->dev, 1, 0, 0,
+ rtl2830_select, NULL);
+ if (!dev->muxc) {
+ ret = -ENOMEM;
goto err_regmap_exit;
}
+ dev->muxc->priv = client;
+ ret = i2c_mux_add_adapter(dev->muxc, 0, 0, 0);
+ if (ret)
+ goto err_regmap_exit;
/* create dvb frontend */
memcpy(&dev->fe.ops, &rtl2830_ops, sizeof(dev->fe.ops));
@@ -903,7 +907,7 @@
/* stop statistics polling */
cancel_delayed_work_sync(&dev->stat_work);
- i2c_del_mux_adapter(dev->adapter);
+ i2c_mux_del_adapters(dev->muxc);
regmap_exit(dev->regmap);
kfree(dev);
diff --git a/drivers/media/dvb-frontends/rtl2830_priv.h b/drivers/media/dvb-frontends/rtl2830_priv.h
index cf793f3..da49095 100644
--- a/drivers/media/dvb-frontends/rtl2830_priv.h
+++ b/drivers/media/dvb-frontends/rtl2830_priv.h
@@ -29,7 +29,7 @@
struct rtl2830_platform_data *pdata;
struct i2c_client *client;
struct regmap *regmap;
- struct i2c_adapter *adapter;
+ struct i2c_mux_core *muxc;
struct dvb_frontend fe;
bool sleeping;
unsigned long filters;
diff --git a/drivers/media/dvb-frontends/rtl2832.c b/drivers/media/dvb-frontends/rtl2832.c
index 7c96f76..bfb6bee 100644
--- a/drivers/media/dvb-frontends/rtl2832.c
+++ b/drivers/media/dvb-frontends/rtl2832.c
@@ -153,43 +153,6 @@
[DVBT_REG_4MSEL] = {0x013, 0, 0},
};
-/* Our regmap is bypassing I2C adapter lock, thus we do it! */
-static int rtl2832_bulk_write(struct i2c_client *client, unsigned int reg,
- const void *val, size_t val_count)
-{
- struct rtl2832_dev *dev = i2c_get_clientdata(client);
- int ret;
-
- i2c_lock_adapter(client->adapter);
- ret = regmap_bulk_write(dev->regmap, reg, val, val_count);
- i2c_unlock_adapter(client->adapter);
- return ret;
-}
-
-static int rtl2832_update_bits(struct i2c_client *client, unsigned int reg,
- unsigned int mask, unsigned int val)
-{
- struct rtl2832_dev *dev = i2c_get_clientdata(client);
- int ret;
-
- i2c_lock_adapter(client->adapter);
- ret = regmap_update_bits(dev->regmap, reg, mask, val);
- i2c_unlock_adapter(client->adapter);
- return ret;
-}
-
-static int rtl2832_bulk_read(struct i2c_client *client, unsigned int reg,
- void *val, size_t val_count)
-{
- struct rtl2832_dev *dev = i2c_get_clientdata(client);
- int ret;
-
- i2c_lock_adapter(client->adapter);
- ret = regmap_bulk_read(dev->regmap, reg, val, val_count);
- i2c_unlock_adapter(client->adapter);
- return ret;
-}
-
static int rtl2832_rd_demod_reg(struct rtl2832_dev *dev, int reg, u32 *val)
{
struct i2c_client *client = dev->client;
@@ -204,7 +167,7 @@
len = (msb >> 3) + 1;
mask = REG_MASK(msb - lsb);
- ret = rtl2832_bulk_read(client, reg_start_addr, reading, len);
+ ret = regmap_bulk_read(dev->regmap, reg_start_addr, reading, len);
if (ret)
goto err;
@@ -234,7 +197,7 @@
len = (msb >> 3) + 1;
mask = REG_MASK(msb - lsb);
- ret = rtl2832_bulk_read(client, reg_start_addr, reading, len);
+ ret = regmap_bulk_read(dev->regmap, reg_start_addr, reading, len);
if (ret)
goto err;
@@ -248,7 +211,7 @@
for (i = 0; i < len; i++)
writing[i] = (writing_tmp >> ((len - 1 - i) * 8)) & 0xff;
- ret = rtl2832_bulk_write(client, reg_start_addr, writing, len);
+ ret = regmap_bulk_write(dev->regmap, reg_start_addr, writing, len);
if (ret)
goto err;
@@ -525,7 +488,8 @@
}
for (j = 0; j < sizeof(bw_params[0]); j++) {
- ret = rtl2832_bulk_write(client, 0x11c + j, &bw_params[i][j], 1);
+ ret = regmap_bulk_write(dev->regmap,
+ 0x11c + j, &bw_params[i][j], 1);
if (ret)
goto err;
}
@@ -581,11 +545,11 @@
if (dev->sleeping)
return 0;
- ret = rtl2832_bulk_read(client, 0x33c, buf, 2);
+ ret = regmap_bulk_read(dev->regmap, 0x33c, buf, 2);
if (ret)
goto err;
- ret = rtl2832_bulk_read(client, 0x351, &buf[2], 1);
+ ret = regmap_bulk_read(dev->regmap, 0x351, &buf[2], 1);
if (ret)
goto err;
@@ -716,7 +680,7 @@
/* signal strength */
if (dev->fe_status & FE_HAS_SIGNAL) {
/* read digital AGC */
- ret = rtl2832_bulk_read(client, 0x305, &u8tmp, 1);
+ ret = regmap_bulk_read(dev->regmap, 0x305, &u8tmp, 1);
if (ret)
goto err;
@@ -742,7 +706,7 @@
{87659938, 87659938, 87885178, 88241743},
};
- ret = rtl2832_bulk_read(client, 0x33c, &u8tmp, 1);
+ ret = regmap_bulk_read(dev->regmap, 0x33c, &u8tmp, 1);
if (ret)
goto err;
@@ -754,7 +718,7 @@
if (hierarchy > HIERARCHY_NUM - 1)
goto err;
- ret = rtl2832_bulk_read(client, 0x40c, buf, 2);
+ ret = regmap_bulk_read(dev->regmap, 0x40c, buf, 2);
if (ret)
goto err;
@@ -775,7 +739,7 @@
/* BER */
if (dev->fe_status & FE_HAS_LOCK) {
- ret = rtl2832_bulk_read(client, 0x34e, buf, 2);
+ ret = regmap_bulk_read(dev->regmap, 0x34e, buf, 2);
if (ret)
goto err;
@@ -825,8 +789,6 @@
/*
* I2C gate/mux/repeater logic
- * We must use unlocked __i2c_transfer() here (through regmap) because of I2C
- * adapter lock is already taken by tuner driver.
* There is delay mechanism to avoid unneeded I2C gate open / close. Gate close
* is delayed here a little bit in order to see if there is sequence of I2C
* messages sent to same I2C bus.
@@ -838,7 +800,7 @@
int ret;
/* close gate */
- ret = rtl2832_update_bits(dev->client, 0x101, 0x08, 0x00);
+ ret = regmap_update_bits(dev->regmap, 0x101, 0x08, 0x00);
if (ret)
goto err;
@@ -847,19 +809,16 @@
dev_dbg(&client->dev, "failed=%d\n", ret);
}
-static int rtl2832_select(struct i2c_adapter *adap, void *mux_priv, u32 chan_id)
+static int rtl2832_select(struct i2c_mux_core *muxc, u32 chan_id)
{
- struct rtl2832_dev *dev = mux_priv;
+ struct rtl2832_dev *dev = i2c_mux_priv(muxc);
struct i2c_client *client = dev->client;
int ret;
/* terminate possible gate closing */
cancel_delayed_work(&dev->i2c_gate_work);
- /*
- * I2C adapter lock is already taken and due to that we will use
- * regmap_update_bits() which does not lock again I2C adapter.
- */
+ /* open gate */
ret = regmap_update_bits(dev->regmap, 0x101, 0x08, 0x08);
if (ret)
goto err;
@@ -870,10 +829,9 @@
return ret;
}
-static int rtl2832_deselect(struct i2c_adapter *adap, void *mux_priv,
- u32 chan_id)
+static int rtl2832_deselect(struct i2c_mux_core *muxc, u32 chan_id)
{
- struct rtl2832_dev *dev = mux_priv;
+ struct rtl2832_dev *dev = i2c_mux_priv(muxc);
schedule_delayed_work(&dev->i2c_gate_work, usecs_to_jiffies(100));
return 0;
@@ -932,120 +890,6 @@
return false;
}
-/*
- * We implement own I2C access routines for regmap in order to get manual access
- * to I2C adapter lock, which is needed for I2C mux adapter.
- */
-static int rtl2832_regmap_read(void *context, const void *reg_buf,
- size_t reg_size, void *val_buf, size_t val_size)
-{
- struct i2c_client *client = context;
- int ret;
- struct i2c_msg msg[2] = {
- {
- .addr = client->addr,
- .flags = 0,
- .len = reg_size,
- .buf = (u8 *)reg_buf,
- }, {
- .addr = client->addr,
- .flags = I2C_M_RD,
- .len = val_size,
- .buf = val_buf,
- }
- };
-
- ret = __i2c_transfer(client->adapter, msg, 2);
- if (ret != 2) {
- dev_warn(&client->dev, "i2c reg read failed %d reg %02x\n",
- ret, *(u8 *)reg_buf);
- if (ret >= 0)
- ret = -EREMOTEIO;
- return ret;
- }
- return 0;
-}
-
-static int rtl2832_regmap_write(void *context, const void *data, size_t count)
-{
- struct i2c_client *client = context;
- int ret;
- struct i2c_msg msg[1] = {
- {
- .addr = client->addr,
- .flags = 0,
- .len = count,
- .buf = (u8 *)data,
- }
- };
-
- ret = __i2c_transfer(client->adapter, msg, 1);
- if (ret != 1) {
- dev_warn(&client->dev, "i2c reg write failed %d reg %02x\n",
- ret, *(u8 *)data);
- if (ret >= 0)
- ret = -EREMOTEIO;
- return ret;
- }
- return 0;
-}
-
-static int rtl2832_regmap_gather_write(void *context, const void *reg,
- size_t reg_len, const void *val,
- size_t val_len)
-{
- struct i2c_client *client = context;
- int ret;
- u8 buf[256];
- struct i2c_msg msg[1] = {
- {
- .addr = client->addr,
- .flags = 0,
- .len = 1 + val_len,
- .buf = buf,
- }
- };
-
- buf[0] = *(u8 const *)reg;
- memcpy(&buf[1], val, val_len);
-
- ret = __i2c_transfer(client->adapter, msg, 1);
- if (ret != 1) {
- dev_warn(&client->dev, "i2c reg write failed %d reg %02x\n",
- ret, *(u8 const *)reg);
- if (ret >= 0)
- ret = -EREMOTEIO;
- return ret;
- }
- return 0;
-}
-
-/*
- * FIXME: Hack. Implement own regmap locking in order to silence lockdep
- * recursive lock warning. That happens when regmap I2C client calls I2C mux
- * adapter, which leads demod I2C repeater enable via demod regmap. Operation
- * takes two regmap locks recursively - but those are different regmap instances
- * in a two different I2C drivers, so it is not deadlock. Proper fix is to make
- * regmap aware of lockdep.
- */
-static void rtl2832_regmap_lock(void *__dev)
-{
- struct rtl2832_dev *dev = __dev;
- struct i2c_client *client = dev->client;
-
- dev_dbg(&client->dev, "\n");
- mutex_lock(&dev->regmap_mutex);
-}
-
-static void rtl2832_regmap_unlock(void *__dev)
-{
- struct rtl2832_dev *dev = __dev;
- struct i2c_client *client = dev->client;
-
- dev_dbg(&client->dev, "\n");
- mutex_unlock(&dev->regmap_mutex);
-}
-
static struct dvb_frontend *rtl2832_get_dvb_frontend(struct i2c_client *client)
{
struct rtl2832_dev *dev = i2c_get_clientdata(client);
@@ -1059,7 +903,7 @@
struct rtl2832_dev *dev = i2c_get_clientdata(client);
dev_dbg(&client->dev, "\n");
- return dev->i2c_adapter_tuner;
+ return dev->muxc->adapter[0];
}
static int rtl2832_slave_ts_ctrl(struct i2c_client *client, bool enable)
@@ -1073,29 +917,29 @@
ret = rtl2832_wr_demod_reg(dev, DVBT_SOFT_RST, 0x0);
if (ret)
goto err;
- ret = rtl2832_bulk_write(client, 0x10c, "\x5f\xff", 2);
+ ret = regmap_bulk_write(dev->regmap, 0x10c, "\x5f\xff", 2);
if (ret)
goto err;
ret = rtl2832_wr_demod_reg(dev, DVBT_PIP_ON, 0x1);
if (ret)
goto err;
- ret = rtl2832_bulk_write(client, 0x0bc, "\x18", 1);
+ ret = regmap_bulk_write(dev->regmap, 0x0bc, "\x18", 1);
if (ret)
goto err;
- ret = rtl2832_bulk_write(client, 0x192, "\x7f\xf7\xff", 3);
+ ret = regmap_bulk_write(dev->regmap, 0x192, "\x7f\xf7\xff", 3);
if (ret)
goto err;
} else {
- ret = rtl2832_bulk_write(client, 0x192, "\x00\x0f\xff", 3);
+ ret = regmap_bulk_write(dev->regmap, 0x192, "\x00\x0f\xff", 3);
if (ret)
goto err;
- ret = rtl2832_bulk_write(client, 0x0bc, "\x08", 1);
+ ret = regmap_bulk_write(dev->regmap, 0x0bc, "\x08", 1);
if (ret)
goto err;
ret = rtl2832_wr_demod_reg(dev, DVBT_PIP_ON, 0x0);
if (ret)
goto err;
- ret = rtl2832_bulk_write(client, 0x10c, "\x00\x00", 2);
+ ret = regmap_bulk_write(dev->regmap, 0x10c, "\x00\x00", 2);
if (ret)
goto err;
ret = rtl2832_wr_demod_reg(dev, DVBT_SOFT_RST, 0x1);
@@ -1124,7 +968,7 @@
else
u8tmp = 0x00;
- ret = rtl2832_update_bits(client, 0x061, 0xc0, u8tmp);
+ ret = regmap_update_bits(dev->regmap, 0x061, 0xc0, u8tmp);
if (ret)
goto err;
@@ -1159,14 +1003,14 @@
buf[1] = (dev->filters >> 8) & 0xff;
buf[2] = (dev->filters >> 16) & 0xff;
buf[3] = (dev->filters >> 24) & 0xff;
- ret = rtl2832_bulk_write(client, 0x062, buf, 4);
+ ret = regmap_bulk_write(dev->regmap, 0x062, buf, 4);
if (ret)
goto err;
/* add PID */
buf[0] = (pid >> 8) & 0xff;
buf[1] = (pid >> 0) & 0xff;
- ret = rtl2832_bulk_write(client, 0x066 + 2 * index, buf, 2);
+ ret = regmap_bulk_write(dev->regmap, 0x066 + 2 * index, buf, 2);
if (ret)
goto err;
@@ -1184,12 +1028,6 @@
struct rtl2832_dev *dev;
int ret;
u8 tmp;
- static const struct regmap_bus regmap_bus = {
- .read = rtl2832_regmap_read,
- .write = rtl2832_regmap_write,
- .gather_write = rtl2832_regmap_gather_write,
- .val_format_endian_default = REGMAP_ENDIAN_NATIVE,
- };
static const struct regmap_range_cfg regmap_range_cfg[] = {
{
.selector_reg = 0x00,
@@ -1218,36 +1056,35 @@
dev->sleeping = true;
INIT_DELAYED_WORK(&dev->i2c_gate_work, rtl2832_i2c_gate_work);
/* create regmap */
- mutex_init(&dev->regmap_mutex);
dev->regmap_config.reg_bits = 8,
dev->regmap_config.val_bits = 8,
- dev->regmap_config.lock = rtl2832_regmap_lock,
- dev->regmap_config.unlock = rtl2832_regmap_unlock,
- dev->regmap_config.lock_arg = dev,
dev->regmap_config.volatile_reg = rtl2832_volatile_reg,
dev->regmap_config.max_register = 5 * 0x100,
dev->regmap_config.ranges = regmap_range_cfg,
dev->regmap_config.num_ranges = ARRAY_SIZE(regmap_range_cfg),
dev->regmap_config.cache_type = REGCACHE_NONE,
- dev->regmap = regmap_init(&client->dev, ®map_bus, client,
- &dev->regmap_config);
+ dev->regmap = regmap_init_i2c(client, &dev->regmap_config);
if (IS_ERR(dev->regmap)) {
ret = PTR_ERR(dev->regmap);
goto err_kfree;
}
/* check if the demod is there */
- ret = rtl2832_bulk_read(client, 0x000, &tmp, 1);
+ ret = regmap_bulk_read(dev->regmap, 0x000, &tmp, 1);
if (ret)
goto err_regmap_exit;
/* create muxed i2c adapter for demod tuner bus */
- dev->i2c_adapter_tuner = i2c_add_mux_adapter(i2c, &i2c->dev, dev,
- 0, 0, 0, rtl2832_select, rtl2832_deselect);
- if (dev->i2c_adapter_tuner == NULL) {
- ret = -ENODEV;
+ dev->muxc = i2c_mux_alloc(i2c, &i2c->dev, 1, 0, I2C_MUX_LOCKED,
+ rtl2832_select, rtl2832_deselect);
+ if (!dev->muxc) {
+ ret = -ENOMEM;
goto err_regmap_exit;
}
+ dev->muxc->priv = dev;
+ ret = i2c_mux_add_adapter(dev->muxc, 0, 0, 0);
+ if (ret)
+ goto err_regmap_exit;
/* create dvb_frontend */
memcpy(&dev->fe.ops, &rtl2832_ops, sizeof(struct dvb_frontend_ops));
@@ -1259,9 +1096,7 @@
pdata->slave_ts_ctrl = rtl2832_slave_ts_ctrl;
pdata->pid_filter = rtl2832_pid_filter;
pdata->pid_filter_ctrl = rtl2832_pid_filter_ctrl;
- pdata->bulk_read = rtl2832_bulk_read;
- pdata->bulk_write = rtl2832_bulk_write;
- pdata->update_bits = rtl2832_update_bits;
+ pdata->regmap = dev->regmap;
dev_info(&client->dev, "Realtek RTL2832 successfully attached\n");
return 0;
@@ -1282,7 +1117,7 @@
cancel_delayed_work_sync(&dev->i2c_gate_work);
- i2c_del_mux_adapter(dev->i2c_adapter_tuner);
+ i2c_mux_del_adapters(dev->muxc);
regmap_exit(dev->regmap);
diff --git a/drivers/media/dvb-frontends/rtl2832.h b/drivers/media/dvb-frontends/rtl2832.h
index 6390af6..03c0de0 100644
--- a/drivers/media/dvb-frontends/rtl2832.h
+++ b/drivers/media/dvb-frontends/rtl2832.h
@@ -57,9 +57,7 @@
int (*pid_filter)(struct dvb_frontend *, u8, u16, int);
int (*pid_filter_ctrl)(struct dvb_frontend *, int);
/* private: Register access for SDR module use only */
- int (*bulk_read)(struct i2c_client *, unsigned int, void *, size_t);
- int (*bulk_write)(struct i2c_client *, unsigned int, const void *, size_t);
- int (*update_bits)(struct i2c_client *, unsigned int, unsigned int, unsigned int);
+ struct regmap *regmap;
};
#endif /* RTL2832_H */
diff --git a/drivers/media/dvb-frontends/rtl2832_priv.h b/drivers/media/dvb-frontends/rtl2832_priv.h
index 6b875f4..c1a8a69 100644
--- a/drivers/media/dvb-frontends/rtl2832_priv.h
+++ b/drivers/media/dvb-frontends/rtl2832_priv.h
@@ -33,10 +33,9 @@
struct rtl2832_dev {
struct rtl2832_platform_data *pdata;
struct i2c_client *client;
- struct mutex regmap_mutex;
struct regmap_config regmap_config;
struct regmap *regmap;
- struct i2c_adapter *i2c_adapter_tuner;
+ struct i2c_mux_core *muxc;
struct dvb_frontend fe;
enum fe_status fe_status;
u64 post_bit_error_prev; /* for old DVBv3 read_ber() calculation */
diff --git a/drivers/media/dvb-frontends/rtl2832_sdr.c b/drivers/media/dvb-frontends/rtl2832_sdr.c
index b860f02..47a480a 100644
--- a/drivers/media/dvb-frontends/rtl2832_sdr.c
+++ b/drivers/media/dvb-frontends/rtl2832_sdr.c
@@ -35,6 +35,7 @@
#include <linux/platform_device.h>
#include <linux/jiffies.h>
#include <linux/math64.h>
+#include <linux/regmap.h>
static bool rtl2832_sdr_emulated_fmt;
module_param_named(emulated_formats, rtl2832_sdr_emulated_fmt, bool, 0644);
@@ -119,6 +120,7 @@
unsigned long flags;
struct platform_device *pdev;
+ struct regmap *regmap;
struct video_device vdev;
struct v4l2_device v4l2_dev;
@@ -163,47 +165,6 @@
unsigned long jiffies_next;
};
-/* write multiple registers */
-static int rtl2832_sdr_wr_regs(struct rtl2832_sdr_dev *dev, u16 reg,
- const u8 *val, int len)
-{
- struct platform_device *pdev = dev->pdev;
- struct rtl2832_sdr_platform_data *pdata = pdev->dev.platform_data;
- struct i2c_client *client = pdata->i2c_client;
-
- return pdata->bulk_write(client, reg, val, len);
-}
-
-#if 0
-/* read multiple registers */
-static int rtl2832_sdr_rd_regs(struct rtl2832_sdr_dev *dev, u16 reg, u8 *val,
- int len)
-{
- struct platform_device *pdev = dev->pdev;
- struct rtl2832_sdr_platform_data *pdata = pdev->dev.platform_data;
- struct i2c_client *client = pdata->i2c_client;
-
- return pdata->bulk_read(client, reg, val, len);
-}
-#endif
-
-/* write single register */
-static int rtl2832_sdr_wr_reg(struct rtl2832_sdr_dev *dev, u16 reg, u8 val)
-{
- return rtl2832_sdr_wr_regs(dev, reg, &val, 1);
-}
-
-/* write single register with mask */
-static int rtl2832_sdr_wr_reg_mask(struct rtl2832_sdr_dev *dev, u16 reg,
- u8 val, u8 mask)
-{
- struct platform_device *pdev = dev->pdev;
- struct rtl2832_sdr_platform_data *pdata = pdev->dev.platform_data;
- struct i2c_client *client = pdata->i2c_client;
-
- return pdata->update_bits(client, reg, mask, val);
-}
-
/* Private functions */
static struct rtl2832_sdr_frame_buf *rtl2832_sdr_get_next_fill_buf(
struct rtl2832_sdr_dev *dev)
@@ -558,11 +519,11 @@
f_sr = dev->f_adc;
- ret = rtl2832_sdr_wr_regs(dev, 0x13e, "\x00\x00", 2);
+ ret = regmap_bulk_write(dev->regmap, 0x13e, "\x00\x00", 2);
if (ret)
goto err;
- ret = rtl2832_sdr_wr_regs(dev, 0x115, "\x00\x00\x00\x00", 4);
+ ret = regmap_bulk_write(dev->regmap, 0x115, "\x00\x00\x00\x00", 4);
if (ret)
goto err;
@@ -588,7 +549,7 @@
buf[1] = (u32tmp >> 8) & 0xff;
buf[2] = (u32tmp >> 0) & 0xff;
- ret = rtl2832_sdr_wr_regs(dev, 0x119, buf, 3);
+ ret = regmap_bulk_write(dev->regmap, 0x119, buf, 3);
if (ret)
goto err;
@@ -602,15 +563,15 @@
u8tmp2 = 0xcd; /* enable ADC I, ADC Q */
}
- ret = rtl2832_sdr_wr_reg(dev, 0x1b1, u8tmp1);
+ ret = regmap_write(dev->regmap, 0x1b1, u8tmp1);
if (ret)
goto err;
- ret = rtl2832_sdr_wr_reg(dev, 0x008, u8tmp2);
+ ret = regmap_write(dev->regmap, 0x008, u8tmp2);
if (ret)
goto err;
- ret = rtl2832_sdr_wr_reg(dev, 0x006, 0x80);
+ ret = regmap_write(dev->regmap, 0x006, 0x80);
if (ret)
goto err;
@@ -621,168 +582,169 @@
buf[1] = (u32tmp >> 16) & 0xff;
buf[2] = (u32tmp >> 8) & 0xff;
buf[3] = (u32tmp >> 0) & 0xff;
- ret = rtl2832_sdr_wr_regs(dev, 0x19f, buf, 4);
+ ret = regmap_bulk_write(dev->regmap, 0x19f, buf, 4);
if (ret)
goto err;
/* low-pass filter */
- ret = rtl2832_sdr_wr_regs(dev, 0x11c,
- "\xca\xdc\xd7\xd8\xe0\xf2\x0e\x35\x06\x50\x9c\x0d\x71\x11\x14\x71\x74\x19\x41\xa5",
- 20);
+ ret = regmap_bulk_write(dev->regmap, 0x11c,
+ "\xca\xdc\xd7\xd8\xe0\xf2\x0e\x35\x06\x50\x9c\x0d\x71\x11\x14\x71\x74\x19\x41\xa5",
+ 20);
if (ret)
goto err;
- ret = rtl2832_sdr_wr_regs(dev, 0x017, "\x11\x10", 2);
+ ret = regmap_bulk_write(dev->regmap, 0x017, "\x11\x10", 2);
if (ret)
goto err;
/* mode */
- ret = rtl2832_sdr_wr_regs(dev, 0x019, "\x05", 1);
+ ret = regmap_write(dev->regmap, 0x019, 0x05);
if (ret)
goto err;
- ret = rtl2832_sdr_wr_regs(dev, 0x01a, "\x1b\x16\x0d\x06\x01\xff", 6);
+ ret = regmap_bulk_write(dev->regmap, 0x01a,
+ "\x1b\x16\x0d\x06\x01\xff", 6);
if (ret)
goto err;
/* FSM */
- ret = rtl2832_sdr_wr_regs(dev, 0x192, "\x00\xf0\x0f", 3);
+ ret = regmap_bulk_write(dev->regmap, 0x192, "\x00\xf0\x0f", 3);
if (ret)
goto err;
/* PID filter */
- ret = rtl2832_sdr_wr_regs(dev, 0x061, "\x60", 1);
+ ret = regmap_write(dev->regmap, 0x061, 0x60);
if (ret)
goto err;
/* used RF tuner based settings */
switch (pdata->tuner) {
case RTL2832_SDR_TUNER_E4000:
- ret = rtl2832_sdr_wr_regs(dev, 0x112, "\x5a", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x102, "\x40", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x103, "\x5a", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1c7, "\x30", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x104, "\xd0", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x105, "\xbe", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1c8, "\x18", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x106, "\x35", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1c9, "\x21", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1ca, "\x21", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1cb, "\x00", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x107, "\x40", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1cd, "\x10", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1ce, "\x10", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x108, "\x80", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x109, "\x7f", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x10a, "\x80", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x10b, "\x7f", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x00e, "\xfc", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x00e, "\xfc", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x011, "\xd4", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1e5, "\xf0", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1d9, "\x00", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1db, "\x00", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1dd, "\x14", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1de, "\xec", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1d8, "\x0c", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1e6, "\x02", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1d7, "\x09", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x00d, "\x83", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x010, "\x49", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x00d, "\x87", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x00d, "\x85", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x013, "\x02", 1);
+ ret = regmap_write(dev->regmap, 0x112, 0x5a);
+ ret = regmap_write(dev->regmap, 0x102, 0x40);
+ ret = regmap_write(dev->regmap, 0x103, 0x5a);
+ ret = regmap_write(dev->regmap, 0x1c7, 0x30);
+ ret = regmap_write(dev->regmap, 0x104, 0xd0);
+ ret = regmap_write(dev->regmap, 0x105, 0xbe);
+ ret = regmap_write(dev->regmap, 0x1c8, 0x18);
+ ret = regmap_write(dev->regmap, 0x106, 0x35);
+ ret = regmap_write(dev->regmap, 0x1c9, 0x21);
+ ret = regmap_write(dev->regmap, 0x1ca, 0x21);
+ ret = regmap_write(dev->regmap, 0x1cb, 0x00);
+ ret = regmap_write(dev->regmap, 0x107, 0x40);
+ ret = regmap_write(dev->regmap, 0x1cd, 0x10);
+ ret = regmap_write(dev->regmap, 0x1ce, 0x10);
+ ret = regmap_write(dev->regmap, 0x108, 0x80);
+ ret = regmap_write(dev->regmap, 0x109, 0x7f);
+ ret = regmap_write(dev->regmap, 0x10a, 0x80);
+ ret = regmap_write(dev->regmap, 0x10b, 0x7f);
+ ret = regmap_write(dev->regmap, 0x00e, 0xfc);
+ ret = regmap_write(dev->regmap, 0x00e, 0xfc);
+ ret = regmap_write(dev->regmap, 0x011, 0xd4);
+ ret = regmap_write(dev->regmap, 0x1e5, 0xf0);
+ ret = regmap_write(dev->regmap, 0x1d9, 0x00);
+ ret = regmap_write(dev->regmap, 0x1db, 0x00);
+ ret = regmap_write(dev->regmap, 0x1dd, 0x14);
+ ret = regmap_write(dev->regmap, 0x1de, 0xec);
+ ret = regmap_write(dev->regmap, 0x1d8, 0x0c);
+ ret = regmap_write(dev->regmap, 0x1e6, 0x02);
+ ret = regmap_write(dev->regmap, 0x1d7, 0x09);
+ ret = regmap_write(dev->regmap, 0x00d, 0x83);
+ ret = regmap_write(dev->regmap, 0x010, 0x49);
+ ret = regmap_write(dev->regmap, 0x00d, 0x87);
+ ret = regmap_write(dev->regmap, 0x00d, 0x85);
+ ret = regmap_write(dev->regmap, 0x013, 0x02);
break;
case RTL2832_SDR_TUNER_FC0012:
case RTL2832_SDR_TUNER_FC0013:
- ret = rtl2832_sdr_wr_regs(dev, 0x112, "\x5a", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x102, "\x40", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x103, "\x5a", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1c7, "\x2c", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x104, "\xcc", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x105, "\xbe", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1c8, "\x16", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x106, "\x35", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1c9, "\x21", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1ca, "\x21", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1cb, "\x00", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x107, "\x40", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1cd, "\x10", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1ce, "\x10", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x108, "\x80", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x109, "\x7f", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x10a, "\x80", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x10b, "\x7f", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x00e, "\xfc", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x00e, "\xfc", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x011, "\xe9\xbf", 2);
- ret = rtl2832_sdr_wr_regs(dev, 0x1e5, "\xf0", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1d9, "\x00", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1db, "\x00", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1dd, "\x11", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1de, "\xef", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1d8, "\x0c", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1e6, "\x02", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1d7, "\x09", 1);
+ ret = regmap_write(dev->regmap, 0x112, 0x5a);
+ ret = regmap_write(dev->regmap, 0x102, 0x40);
+ ret = regmap_write(dev->regmap, 0x103, 0x5a);
+ ret = regmap_write(dev->regmap, 0x1c7, 0x2c);
+ ret = regmap_write(dev->regmap, 0x104, 0xcc);
+ ret = regmap_write(dev->regmap, 0x105, 0xbe);
+ ret = regmap_write(dev->regmap, 0x1c8, 0x16);
+ ret = regmap_write(dev->regmap, 0x106, 0x35);
+ ret = regmap_write(dev->regmap, 0x1c9, 0x21);
+ ret = regmap_write(dev->regmap, 0x1ca, 0x21);
+ ret = regmap_write(dev->regmap, 0x1cb, 0x00);
+ ret = regmap_write(dev->regmap, 0x107, 0x40);
+ ret = regmap_write(dev->regmap, 0x1cd, 0x10);
+ ret = regmap_write(dev->regmap, 0x1ce, 0x10);
+ ret = regmap_write(dev->regmap, 0x108, 0x80);
+ ret = regmap_write(dev->regmap, 0x109, 0x7f);
+ ret = regmap_write(dev->regmap, 0x10a, 0x80);
+ ret = regmap_write(dev->regmap, 0x10b, 0x7f);
+ ret = regmap_write(dev->regmap, 0x00e, 0xfc);
+ ret = regmap_write(dev->regmap, 0x00e, 0xfc);
+ ret = regmap_bulk_write(dev->regmap, 0x011, "\xe9\xbf", 2);
+ ret = regmap_write(dev->regmap, 0x1e5, 0xf0);
+ ret = regmap_write(dev->regmap, 0x1d9, 0x00);
+ ret = regmap_write(dev->regmap, 0x1db, 0x00);
+ ret = regmap_write(dev->regmap, 0x1dd, 0x11);
+ ret = regmap_write(dev->regmap, 0x1de, 0xef);
+ ret = regmap_write(dev->regmap, 0x1d8, 0x0c);
+ ret = regmap_write(dev->regmap, 0x1e6, 0x02);
+ ret = regmap_write(dev->regmap, 0x1d7, 0x09);
break;
case RTL2832_SDR_TUNER_R820T:
case RTL2832_SDR_TUNER_R828D:
- ret = rtl2832_sdr_wr_regs(dev, 0x112, "\x5a", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x102, "\x40", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x115, "\x01", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x103, "\x80", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1c7, "\x24", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x104, "\xcc", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x105, "\xbe", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1c8, "\x14", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x106, "\x35", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1c9, "\x21", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1ca, "\x21", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1cb, "\x00", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x107, "\x40", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1cd, "\x10", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1ce, "\x10", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x108, "\x80", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x109, "\x7f", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x10a, "\x80", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x10b, "\x7f", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x00e, "\xfc", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x00e, "\xfc", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x011, "\xf4", 1);
+ ret = regmap_write(dev->regmap, 0x112, 0x5a);
+ ret = regmap_write(dev->regmap, 0x102, 0x40);
+ ret = regmap_write(dev->regmap, 0x115, 0x01);
+ ret = regmap_write(dev->regmap, 0x103, 0x80);
+ ret = regmap_write(dev->regmap, 0x1c7, 0x24);
+ ret = regmap_write(dev->regmap, 0x104, 0xcc);
+ ret = regmap_write(dev->regmap, 0x105, 0xbe);
+ ret = regmap_write(dev->regmap, 0x1c8, 0x14);
+ ret = regmap_write(dev->regmap, 0x106, 0x35);
+ ret = regmap_write(dev->regmap, 0x1c9, 0x21);
+ ret = regmap_write(dev->regmap, 0x1ca, 0x21);
+ ret = regmap_write(dev->regmap, 0x1cb, 0x00);
+ ret = regmap_write(dev->regmap, 0x107, 0x40);
+ ret = regmap_write(dev->regmap, 0x1cd, 0x10);
+ ret = regmap_write(dev->regmap, 0x1ce, 0x10);
+ ret = regmap_write(dev->regmap, 0x108, 0x80);
+ ret = regmap_write(dev->regmap, 0x109, 0x7f);
+ ret = regmap_write(dev->regmap, 0x10a, 0x80);
+ ret = regmap_write(dev->regmap, 0x10b, 0x7f);
+ ret = regmap_write(dev->regmap, 0x00e, 0xfc);
+ ret = regmap_write(dev->regmap, 0x00e, 0xfc);
+ ret = regmap_write(dev->regmap, 0x011, 0xf4);
break;
case RTL2832_SDR_TUNER_FC2580:
- ret = rtl2832_sdr_wr_regs(dev, 0x112, "\x39", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x102, "\x40", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x103, "\x5a", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1c7, "\x2c", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x104, "\xcc", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x105, "\xbe", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1c8, "\x16", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x106, "\x35", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1c9, "\x21", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1ca, "\x21", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1cb, "\x00", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x107, "\x40", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1cd, "\x10", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x1ce, "\x10", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x108, "\x80", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x109, "\x7f", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x10a, "\x9c", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x10b, "\x7f", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x00e, "\xfc", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x00e, "\xfc", 1);
- ret = rtl2832_sdr_wr_regs(dev, 0x011, "\xe9\xf4", 2);
+ ret = regmap_write(dev->regmap, 0x112, 0x39);
+ ret = regmap_write(dev->regmap, 0x102, 0x40);
+ ret = regmap_write(dev->regmap, 0x103, 0x5a);
+ ret = regmap_write(dev->regmap, 0x1c7, 0x2c);
+ ret = regmap_write(dev->regmap, 0x104, 0xcc);
+ ret = regmap_write(dev->regmap, 0x105, 0xbe);
+ ret = regmap_write(dev->regmap, 0x1c8, 0x16);
+ ret = regmap_write(dev->regmap, 0x106, 0x35);
+ ret = regmap_write(dev->regmap, 0x1c9, 0x21);
+ ret = regmap_write(dev->regmap, 0x1ca, 0x21);
+ ret = regmap_write(dev->regmap, 0x1cb, 0x00);
+ ret = regmap_write(dev->regmap, 0x107, 0x40);
+ ret = regmap_write(dev->regmap, 0x1cd, 0x10);
+ ret = regmap_write(dev->regmap, 0x1ce, 0x10);
+ ret = regmap_write(dev->regmap, 0x108, 0x80);
+ ret = regmap_write(dev->regmap, 0x109, 0x7f);
+ ret = regmap_write(dev->regmap, 0x10a, 0x9c);
+ ret = regmap_write(dev->regmap, 0x10b, 0x7f);
+ ret = regmap_write(dev->regmap, 0x00e, 0xfc);
+ ret = regmap_write(dev->regmap, 0x00e, 0xfc);
+ ret = regmap_bulk_write(dev->regmap, 0x011, "\xe9\xf4", 2);
break;
default:
dev_notice(&pdev->dev, "Unsupported tuner\n");
}
/* software reset */
- ret = rtl2832_sdr_wr_reg_mask(dev, 0x101, 0x04, 0x04);
+ ret = regmap_update_bits(dev->regmap, 0x101, 0x04, 0x04);
if (ret)
goto err;
- ret = rtl2832_sdr_wr_reg_mask(dev, 0x101, 0x00, 0x04);
+ ret = regmap_update_bits(dev->regmap, 0x101, 0x04, 0x00);
if (ret)
goto err;
err:
@@ -797,29 +759,29 @@
dev_dbg(&pdev->dev, "\n");
/* PID filter */
- ret = rtl2832_sdr_wr_regs(dev, 0x061, "\xe0", 1);
+ ret = regmap_write(dev->regmap, 0x061, 0xe0);
if (ret)
goto err;
/* mode */
- ret = rtl2832_sdr_wr_regs(dev, 0x019, "\x20", 1);
+ ret = regmap_write(dev->regmap, 0x019, 0x20);
if (ret)
goto err;
- ret = rtl2832_sdr_wr_regs(dev, 0x017, "\x11\x10", 2);
+ ret = regmap_bulk_write(dev->regmap, 0x017, "\x11\x10", 2);
if (ret)
goto err;
/* FSM */
- ret = rtl2832_sdr_wr_regs(dev, 0x192, "\x00\x0f\xff", 3);
+ ret = regmap_bulk_write(dev->regmap, 0x192, "\x00\x0f\xff", 3);
if (ret)
goto err;
- ret = rtl2832_sdr_wr_regs(dev, 0x13e, "\x40\x00", 2);
+ ret = regmap_bulk_write(dev->regmap, 0x13e, "\x40\x00", 2);
if (ret)
goto err;
- ret = rtl2832_sdr_wr_regs(dev, 0x115, "\x06\x3f\xce\xcc", 4);
+ ret = regmap_bulk_write(dev->regmap, 0x115, "\x06\x3f\xce\xcc", 4);
if (ret)
goto err;
err:
@@ -1399,6 +1361,7 @@
subdev = pdata->v4l2_subdev;
dev->v4l2_subdev = pdata->v4l2_subdev;
dev->pdev = pdev;
+ dev->regmap = pdata->regmap;
dev->udev = pdata->dvb_usb_device->udev;
dev->f_adc = bands_adc[0].rangelow;
dev->f_tuner = bands_fm[0].rangelow;
diff --git a/drivers/media/dvb-frontends/rtl2832_sdr.h b/drivers/media/dvb-frontends/rtl2832_sdr.h
index 342ea84..d8fc7e7 100644
--- a/drivers/media/dvb-frontends/rtl2832_sdr.h
+++ b/drivers/media/dvb-frontends/rtl2832_sdr.h
@@ -56,10 +56,7 @@
#define RTL2832_SDR_TUNER_R828D 0x2b
u8 tuner;
- struct i2c_client *i2c_client;
- int (*bulk_read)(struct i2c_client *, unsigned int, void *, size_t);
- int (*bulk_write)(struct i2c_client *, unsigned int, const void *, size_t);
- int (*update_bits)(struct i2c_client *, unsigned int, unsigned int, unsigned int);
+ struct regmap *regmap;
struct dvb_frontend *dvb_frontend;
struct v4l2_subdev *v4l2_subdev;
struct dvb_usb_device *dvb_usb_device;
diff --git a/drivers/media/dvb-frontends/si2168.c b/drivers/media/dvb-frontends/si2168.c
index 821a8f4..108a069 100644
--- a/drivers/media/dvb-frontends/si2168.c
+++ b/drivers/media/dvb-frontends/si2168.c
@@ -18,53 +18,23 @@
static const struct dvb_frontend_ops si2168_ops;
-/* Own I2C adapter locking is needed because of I2C gate logic. */
-static int si2168_i2c_master_send_unlocked(const struct i2c_client *client,
- const char *buf, int count)
-{
- int ret;
- struct i2c_msg msg = {
- .addr = client->addr,
- .flags = 0,
- .len = count,
- .buf = (char *)buf,
- };
-
- ret = __i2c_transfer(client->adapter, &msg, 1);
- return (ret == 1) ? count : ret;
-}
-
-static int si2168_i2c_master_recv_unlocked(const struct i2c_client *client,
- char *buf, int count)
-{
- int ret;
- struct i2c_msg msg = {
- .addr = client->addr,
- .flags = I2C_M_RD,
- .len = count,
- .buf = buf,
- };
-
- ret = __i2c_transfer(client->adapter, &msg, 1);
- return (ret == 1) ? count : ret;
-}
-
/* execute firmware command */
-static int si2168_cmd_execute_unlocked(struct i2c_client *client,
- struct si2168_cmd *cmd)
+static int si2168_cmd_execute(struct i2c_client *client, struct si2168_cmd *cmd)
{
+ struct si2168_dev *dev = i2c_get_clientdata(client);
int ret;
unsigned long timeout;
+ mutex_lock(&dev->i2c_mutex);
+
if (cmd->wlen) {
/* write cmd and args for firmware */
- ret = si2168_i2c_master_send_unlocked(client, cmd->args,
- cmd->wlen);
+ ret = i2c_master_send(client, cmd->args, cmd->wlen);
if (ret < 0) {
- goto err;
+ goto err_mutex_unlock;
} else if (ret != cmd->wlen) {
ret = -EREMOTEIO;
- goto err;
+ goto err_mutex_unlock;
}
}
@@ -73,13 +43,12 @@
#define TIMEOUT 70
timeout = jiffies + msecs_to_jiffies(TIMEOUT);
while (!time_after(jiffies, timeout)) {
- ret = si2168_i2c_master_recv_unlocked(client, cmd->args,
- cmd->rlen);
+ ret = i2c_master_recv(client, cmd->args, cmd->rlen);
if (ret < 0) {
- goto err;
+ goto err_mutex_unlock;
} else if (ret != cmd->rlen) {
ret = -EREMOTEIO;
- goto err;
+ goto err_mutex_unlock;
}
/* firmware ready? */
@@ -94,32 +63,23 @@
/* error bit set? */
if ((cmd->args[0] >> 6) & 0x01) {
ret = -EREMOTEIO;
- goto err;
+ goto err_mutex_unlock;
}
if (!((cmd->args[0] >> 7) & 0x01)) {
ret = -ETIMEDOUT;
- goto err;
+ goto err_mutex_unlock;
}
}
+ mutex_unlock(&dev->i2c_mutex);
return 0;
-err:
+err_mutex_unlock:
+ mutex_unlock(&dev->i2c_mutex);
dev_dbg(&client->dev, "failed=%d\n", ret);
return ret;
}
-static int si2168_cmd_execute(struct i2c_client *client, struct si2168_cmd *cmd)
-{
- int ret;
-
- i2c_lock_adapter(client->adapter);
- ret = si2168_cmd_execute_unlocked(client, cmd);
- i2c_unlock_adapter(client->adapter);
-
- return ret;
-}
-
static int si2168_read_status(struct dvb_frontend *fe, enum fe_status *status)
{
struct i2c_client *client = fe->demodulator_priv;
@@ -610,14 +570,9 @@
return 0;
}
-/*
- * I2C gate logic
- * We must use unlocked I2C I/O because I2C adapter lock is already taken
- * by the caller (usually tuner driver).
- */
-static int si2168_select(struct i2c_adapter *adap, void *mux_priv, u32 chan)
+static int si2168_select(struct i2c_mux_core *muxc, u32 chan)
{
- struct i2c_client *client = mux_priv;
+ struct i2c_client *client = i2c_mux_priv(muxc);
int ret;
struct si2168_cmd cmd;
@@ -625,7 +580,7 @@
memcpy(cmd.args, "\xc0\x0d\x01", 3);
cmd.wlen = 3;
cmd.rlen = 0;
- ret = si2168_cmd_execute_unlocked(client, &cmd);
+ ret = si2168_cmd_execute(client, &cmd);
if (ret)
goto err;
@@ -635,9 +590,9 @@
return ret;
}
-static int si2168_deselect(struct i2c_adapter *adap, void *mux_priv, u32 chan)
+static int si2168_deselect(struct i2c_mux_core *muxc, u32 chan)
{
- struct i2c_client *client = mux_priv;
+ struct i2c_client *client = i2c_mux_priv(muxc);
int ret;
struct si2168_cmd cmd;
@@ -645,7 +600,7 @@
memcpy(cmd.args, "\xc0\x0d\x00", 3);
cmd.wlen = 3;
cmd.rlen = 0;
- ret = si2168_cmd_execute_unlocked(client, &cmd);
+ ret = si2168_cmd_execute(client, &cmd);
if (ret)
goto err;
@@ -708,18 +663,25 @@
goto err;
}
+ mutex_init(&dev->i2c_mutex);
+
/* create mux i2c adapter for tuner */
- dev->adapter = i2c_add_mux_adapter(client->adapter, &client->dev,
- client, 0, 0, 0, si2168_select, si2168_deselect);
- if (dev->adapter == NULL) {
- ret = -ENODEV;
+ dev->muxc = i2c_mux_alloc(client->adapter, &client->dev,
+ 1, 0, I2C_MUX_LOCKED,
+ si2168_select, si2168_deselect);
+ if (!dev->muxc) {
+ ret = -ENOMEM;
goto err_kfree;
}
+ dev->muxc->priv = client;
+ ret = i2c_mux_add_adapter(dev->muxc, 0, 0, 0);
+ if (ret)
+ goto err_kfree;
/* create dvb_frontend */
memcpy(&dev->fe.ops, &si2168_ops, sizeof(struct dvb_frontend_ops));
dev->fe.demodulator_priv = client;
- *config->i2c_adapter = dev->adapter;
+ *config->i2c_adapter = dev->muxc->adapter[0];
*config->fe = &dev->fe;
dev->ts_mode = config->ts_mode;
dev->ts_clock_inv = config->ts_clock_inv;
@@ -743,7 +705,7 @@
dev_dbg(&client->dev, "\n");
- i2c_del_mux_adapter(dev->adapter);
+ i2c_mux_del_adapters(dev->muxc);
dev->fe.ops.release = NULL;
dev->fe.demodulator_priv = NULL;
diff --git a/drivers/media/dvb-frontends/si2168_priv.h b/drivers/media/dvb-frontends/si2168_priv.h
index c07e6fe..8a1f36d 100644
--- a/drivers/media/dvb-frontends/si2168_priv.h
+++ b/drivers/media/dvb-frontends/si2168_priv.h
@@ -29,7 +29,8 @@
/* state struct */
struct si2168_dev {
- struct i2c_adapter *adapter;
+ struct mutex i2c_mutex;
+ struct i2c_mux_core *muxc;
struct dvb_frontend fe;
enum fe_delivery_system delivery_system;
enum fe_status fe_status;
diff --git a/drivers/media/dvb-frontends/zl10353.c b/drivers/media/dvb-frontends/zl10353.c
index 1832c2f..3b08176 100644
--- a/drivers/media/dvb-frontends/zl10353.c
+++ b/drivers/media/dvb-frontends/zl10353.c
@@ -135,8 +135,7 @@
value = (u64)10 * (1 << 23) / 7 * 125;
value = (bw * value) + adc_clock / 2;
- do_div(value, adc_clock);
- *nominal_rate = value;
+ *nominal_rate = div_u64(value, adc_clock);
dprintk("%s: bw %d, adc_clock %d => 0x%x\n",
__func__, bw, adc_clock, *nominal_rate);
@@ -163,8 +162,7 @@
if (ife > adc_clock / 2)
ife = adc_clock - ife;
}
- value = (u64)65536 * ife + adc_clock / 2;
- do_div(value, adc_clock);
+ value = div_u64((u64)65536 * ife + adc_clock / 2, adc_clock);
*input_freq = -value;
dprintk("%s: if2 %d, ife %d, adc_clock %d => %d / 0x%x\n",
diff --git a/drivers/media/i2c/ad9389b.c b/drivers/media/i2c/ad9389b.c
index 788967d..0462f46 100644
--- a/drivers/media/i2c/ad9389b.c
+++ b/drivers/media/i2c/ad9389b.c
@@ -1130,8 +1130,6 @@
hdl = &state->hdl;
v4l2_ctrl_handler_init(hdl, 5);
- /* private controls */
-
state->hdmi_mode_ctrl = v4l2_ctrl_new_std_menu(hdl, &ad9389b_ctrl_ops,
V4L2_CID_DV_TX_MODE, V4L2_DV_TX_MODE_HDMI,
0, V4L2_DV_TX_MODE_DVI_D);
@@ -1151,12 +1149,6 @@
goto err_hdl;
}
- state->hdmi_mode_ctrl->is_private = true;
- state->hotplug_ctrl->is_private = true;
- state->rx_sense_ctrl->is_private = true;
- state->have_edid0_ctrl->is_private = true;
- state->rgb_quantization_range_ctrl->is_private = true;
-
state->pad.flags = MEDIA_PAD_FL_SINK;
err = media_entity_pads_init(&sd->entity, 1, &state->pad);
if (err)
diff --git a/drivers/media/i2c/adp1653.c b/drivers/media/i2c/adp1653.c
index fb7ed73..9e1731c 100644
--- a/drivers/media/i2c/adp1653.c
+++ b/drivers/media/i2c/adp1653.c
@@ -466,9 +466,9 @@
of_node_put(child);
pd->enable_gpio = devm_gpiod_get(&client->dev, "enable", GPIOD_OUT_LOW);
- if (!pd->enable_gpio) {
+ if (IS_ERR(pd->enable_gpio)) {
dev_err(&client->dev, "Error getting GPIO\n");
- return -EINVAL;
+ return PTR_ERR(pd->enable_gpio);
}
return 0;
diff --git a/drivers/media/i2c/adv7180.c b/drivers/media/i2c/adv7180.c
index ff57c1d..b77b0a4 100644
--- a/drivers/media/i2c/adv7180.c
+++ b/drivers/media/i2c/adv7180.c
@@ -26,8 +26,9 @@
#include <linux/i2c.h>
#include <linux/slab.h>
#include <linux/of.h>
-#include <media/v4l2-ioctl.h>
#include <linux/videodev2.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-event.h>
#include <media/v4l2-device.h>
#include <media/v4l2-ctrls.h>
#include <linux/mutex.h>
@@ -192,8 +193,8 @@
struct mutex mutex; /* mutual excl. when accessing chip */
int irq;
v4l2_std_id curr_norm;
- bool autodetect;
bool powered;
+ bool streaming;
u8 input;
struct i2c_client *client;
@@ -338,12 +339,26 @@
if (err)
return err;
- /* when we are interrupt driven we know the state */
- if (!state->autodetect || state->irq > 0)
- *std = state->curr_norm;
- else
- err = __adv7180_status(state, NULL, std);
+ if (state->streaming) {
+ err = -EBUSY;
+ goto unlock;
+ }
+ err = adv7180_set_video_standard(state,
+ ADV7180_STD_AD_PAL_BG_NTSC_J_SECAM);
+ if (err)
+ goto unlock;
+
+ msleep(100);
+ __adv7180_status(state, NULL, std);
+
+ err = v4l2_std_to_adv7180(state->curr_norm);
+ if (err < 0)
+ goto unlock;
+
+ err = adv7180_set_video_standard(state, err);
+
+unlock:
mutex_unlock(&state->mutex);
return err;
}
@@ -387,23 +402,13 @@
{
int ret;
- if (state->autodetect) {
- ret = adv7180_set_video_standard(state,
- ADV7180_STD_AD_PAL_BG_NTSC_J_SECAM);
- if (ret < 0)
- return ret;
+ ret = v4l2_std_to_adv7180(state->curr_norm);
+ if (ret < 0)
+ return ret;
- __adv7180_status(state, NULL, &state->curr_norm);
- } else {
- ret = v4l2_std_to_adv7180(state->curr_norm);
- if (ret < 0)
- return ret;
-
- ret = adv7180_set_video_standard(state, ret);
- if (ret < 0)
- return ret;
- }
-
+ ret = adv7180_set_video_standard(state, ret);
+ if (ret < 0)
+ return ret;
return 0;
}
@@ -415,18 +420,12 @@
if (ret)
return ret;
- /* all standards -> autodetect */
- if (std == V4L2_STD_ALL) {
- state->autodetect = true;
- } else {
- /* Make sure we can support this std */
- ret = v4l2_std_to_adv7180(std);
- if (ret < 0)
- goto out;
+ /* Make sure we can support this std */
+ ret = v4l2_std_to_adv7180(std);
+ if (ret < 0)
+ goto out;
- state->curr_norm = std;
- state->autodetect = false;
- }
+ state->curr_norm = std;
ret = adv7180_program_std(state);
out:
@@ -434,6 +433,15 @@
return ret;
}
+static int adv7180_g_std(struct v4l2_subdev *sd, v4l2_std_id *norm)
+{
+ struct adv7180_state *state = to_state(sd);
+
+ *norm = state->curr_norm;
+
+ return 0;
+}
+
static int adv7180_set_power(struct adv7180_state *state, bool on)
{
u8 val;
@@ -717,17 +725,77 @@
return 0;
}
+static int adv7180_cropcap(struct v4l2_subdev *sd, struct v4l2_cropcap *cropcap)
+{
+ struct adv7180_state *state = to_state(sd);
+
+ if (state->curr_norm & V4L2_STD_525_60) {
+ cropcap->pixelaspect.numerator = 11;
+ cropcap->pixelaspect.denominator = 10;
+ } else {
+ cropcap->pixelaspect.numerator = 54;
+ cropcap->pixelaspect.denominator = 59;
+ }
+
+ return 0;
+}
+
+static int adv7180_g_tvnorms(struct v4l2_subdev *sd, v4l2_std_id *norm)
+{
+ *norm = V4L2_STD_ALL;
+ return 0;
+}
+
+static int adv7180_s_stream(struct v4l2_subdev *sd, int enable)
+{
+ struct adv7180_state *state = to_state(sd);
+ int ret;
+
+ /* It's always safe to stop streaming, no need to take the lock */
+ if (!enable) {
+ state->streaming = enable;
+ return 0;
+ }
+
+ /* Must wait until querystd released the lock */
+ ret = mutex_lock_interruptible(&state->mutex);
+ if (ret)
+ return ret;
+ state->streaming = enable;
+ mutex_unlock(&state->mutex);
+ return 0;
+}
+
+static int adv7180_subscribe_event(struct v4l2_subdev *sd,
+ struct v4l2_fh *fh,
+ struct v4l2_event_subscription *sub)
+{
+ switch (sub->type) {
+ case V4L2_EVENT_SOURCE_CHANGE:
+ return v4l2_src_change_event_subdev_subscribe(sd, fh, sub);
+ case V4L2_EVENT_CTRL:
+ return v4l2_ctrl_subdev_subscribe_event(sd, fh, sub);
+ default:
+ return -EINVAL;
+ }
+}
+
static const struct v4l2_subdev_video_ops adv7180_video_ops = {
.s_std = adv7180_s_std,
+ .g_std = adv7180_g_std,
.querystd = adv7180_querystd,
.g_input_status = adv7180_g_input_status,
.s_routing = adv7180_s_routing,
.g_mbus_config = adv7180_g_mbus_config,
+ .cropcap = adv7180_cropcap,
+ .g_tvnorms = adv7180_g_tvnorms,
+ .s_stream = adv7180_s_stream,
};
-
static const struct v4l2_subdev_core_ops adv7180_core_ops = {
.s_power = adv7180_s_power,
+ .subscribe_event = adv7180_subscribe_event,
+ .unsubscribe_event = v4l2_event_subdev_unsubscribe,
};
static const struct v4l2_subdev_pad_ops adv7180_pad_ops = {
@@ -752,8 +820,14 @@
/* clear */
adv7180_write(state, ADV7180_REG_ICR3, isr3);
- if (isr3 & ADV7180_IRQ3_AD_CHANGE && state->autodetect)
- __adv7180_status(state, NULL, &state->curr_norm);
+ if (isr3 & ADV7180_IRQ3_AD_CHANGE) {
+ static const struct v4l2_event src_ch = {
+ .type = V4L2_EVENT_SOURCE_CHANGE,
+ .u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION,
+ };
+
+ v4l2_subdev_notify_event(&state->sd, &src_ch);
+ }
mutex_unlock(&state->mutex);
return IRQ_HANDLED;
@@ -1198,7 +1272,7 @@
state->irq = client->irq;
mutex_init(&state->mutex);
- state->autodetect = true;
+ state->curr_norm = V4L2_STD_NTSC;
if (state->chip_info->flags & ADV7180_FLAG_RESET_POWERED)
state->powered = true;
else
@@ -1206,7 +1280,7 @@
state->input = 0;
sd = &state->sd;
v4l2_i2c_subdev_init(sd, client, &adv7180_ops);
- sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+ sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_HAS_EVENTS;
ret = adv7180_init_controls(state);
if (ret)
@@ -1328,6 +1402,14 @@
#ifdef CONFIG_OF
static const struct of_device_id adv7180_of_id[] = {
{ .compatible = "adi,adv7180", },
+ { .compatible = "adi,adv7182", },
+ { .compatible = "adi,adv7280", },
+ { .compatible = "adi,adv7280-m", },
+ { .compatible = "adi,adv7281", },
+ { .compatible = "adi,adv7281-m", },
+ { .compatible = "adi,adv7281-ma", },
+ { .compatible = "adi,adv7282", },
+ { .compatible = "adi,adv7282-m", },
{ },
};
diff --git a/drivers/media/i2c/adv7511.c b/drivers/media/i2c/adv7511.c
index bd822f0..39271c3 100644
--- a/drivers/media/i2c/adv7511.c
+++ b/drivers/media/i2c/adv7511.c
@@ -1502,12 +1502,6 @@
err = hdl->error;
goto err_hdl;
}
- state->hdmi_mode_ctrl->is_private = true;
- state->hotplug_ctrl->is_private = true;
- state->rx_sense_ctrl->is_private = true;
- state->have_edid0_ctrl->is_private = true;
- state->rgb_quantization_range_ctrl->is_private = true;
-
state->pad.flags = MEDIA_PAD_FL_SINK;
err = media_entity_pads_init(&sd->entity, 1, &state->pad);
if (err)
diff --git a/drivers/media/i2c/adv7604.c b/drivers/media/i2c/adv7604.c
index 41a1bfc..beb2841 100644
--- a/drivers/media/i2c/adv7604.c
+++ b/drivers/media/i2c/adv7604.c
@@ -3141,7 +3141,6 @@
if (ctrl)
ctrl->flags |= V4L2_CTRL_FLAG_VOLATILE;
- /* private controls */
state->detect_tx_5v_ctrl = v4l2_ctrl_new_std(hdl, NULL,
V4L2_CID_DV_RX_POWER_PRESENT, 0,
(1 << state->info->num_dv_ports) - 1, 0, 0);
@@ -3164,13 +3163,6 @@
err = hdl->error;
goto err_hdl;
}
- state->detect_tx_5v_ctrl->is_private = true;
- state->rgb_quantization_range_ctrl->is_private = true;
- if (adv76xx_has_afe(state))
- state->analog_sampling_phase_ctrl->is_private = true;
- state->free_run_color_manual_ctrl->is_private = true;
- state->free_run_color_ctrl->is_private = true;
-
if (adv76xx_s_detect_tx_5v_ctrl(sd)) {
err = -ENODEV;
goto err_hdl;
diff --git a/drivers/media/i2c/adv7842.c b/drivers/media/i2c/adv7842.c
index 7ccb85d..ecaacb0 100644
--- a/drivers/media/i2c/adv7842.c
+++ b/drivers/media/i2c/adv7842.c
@@ -3300,12 +3300,6 @@
err = hdl->error;
goto err_hdl;
}
- state->detect_tx_5v_ctrl->is_private = true;
- state->rgb_quantization_range_ctrl->is_private = true;
- state->analog_sampling_phase_ctrl->is_private = true;
- state->free_run_color_ctrl_manual->is_private = true;
- state->free_run_color_ctrl->is_private = true;
-
if (adv7842_s_detect_tx_5v_ctrl(sd)) {
err = -ENODEV;
goto err_hdl;
diff --git a/drivers/media/i2c/m5mols/m5mols_controls.c b/drivers/media/i2c/m5mols/m5mols_controls.c
index a60931e..c2218c0 100644
--- a/drivers/media/i2c/m5mols/m5mols_controls.c
+++ b/drivers/media/i2c/m5mols/m5mols_controls.c
@@ -405,7 +405,7 @@
struct v4l2_subdev *sd = to_sd(ctrl);
struct m5mols_info *info = to_m5mols(sd);
int ret = 0;
- u8 status;
+ u8 status = REG_ISO_AUTO;
v4l2_dbg(1, m5mols_debug, sd, "%s: ctrl: %s (%d)\n",
__func__, ctrl->name, info->isp_ready);
diff --git a/drivers/media/i2c/saa7115.c b/drivers/media/i2c/saa7115.c
index d2a1ce2..bd3526b 100644
--- a/drivers/media/i2c/saa7115.c
+++ b/drivers/media/i2c/saa7115.c
@@ -1798,6 +1798,21 @@
return GM7113C;
}
+ /* Check if it is a CJC7113 */
+ if (!memcmp(name, "1111111111111111", CHIP_VER_SIZE)) {
+ strlcpy(name, "cjc7113", CHIP_VER_SIZE);
+
+ if (!autodetect && strcmp(name, id->name))
+ return -EINVAL;
+
+ v4l_dbg(1, debug, client,
+ "It seems to be a %s chip (%*ph) @ 0x%x.\n",
+ name, 16, chip_ver, client->addr << 1);
+
+ /* CJC7113 seems to be SAA7113-compatible */
+ return SAA7113;
+ }
+
/* Chip was not discovered. Return its ID and don't bind */
v4l_dbg(1, debug, client, "chip %*ph @ 0x%x is unknown.\n",
16, chip_ver, client->addr << 1);
diff --git a/drivers/media/i2c/smiapp/smiapp-core.c b/drivers/media/i2c/smiapp/smiapp-core.c
index a215efe..3dfe387 100644
--- a/drivers/media/i2c/smiapp/smiapp-core.c
+++ b/drivers/media/i2c/smiapp/smiapp-core.c
@@ -188,6 +188,8 @@
embedded_end = 0;
}
+ sensor->image_start = image_start;
+
dev_dbg(&client->dev, "embedded data from lines %d to %d\n",
embedded_start, embedded_end);
dev_dbg(&client->dev, "image data starts at line %d\n", image_start);
@@ -2280,6 +2282,15 @@
return 0;
}
+static int smiapp_get_skip_top_lines(struct v4l2_subdev *subdev, u32 *lines)
+{
+ struct smiapp_sensor *sensor = to_smiapp_sensor(subdev);
+
+ *lines = sensor->image_start;
+
+ return 0;
+}
+
/* -----------------------------------------------------------------------------
* sysfs attributes
*/
@@ -2890,6 +2901,7 @@
static const struct v4l2_subdev_sensor_ops smiapp_sensor_ops = {
.g_skip_frames = smiapp_get_skip_frames,
+ .g_skip_top_lines = smiapp_get_skip_top_lines,
};
static const struct v4l2_subdev_ops smiapp_ops = {
diff --git a/drivers/media/i2c/smiapp/smiapp.h b/drivers/media/i2c/smiapp/smiapp.h
index f6af0cc..2174f89 100644
--- a/drivers/media/i2c/smiapp/smiapp.h
+++ b/drivers/media/i2c/smiapp/smiapp.h
@@ -217,6 +217,7 @@
u8 hvflip_inv_mask; /* H/VFLIP inversion due to sensor orientation */
u8 frame_skip;
+ u16 image_start; /* Offset to first line after metadata lines */
int power_count;
diff --git a/drivers/media/i2c/tc358743.c b/drivers/media/i2c/tc358743.c
index 972e0d4..6cf6d06 100644
--- a/drivers/media/i2c/tc358743.c
+++ b/drivers/media/i2c/tc358743.c
@@ -1551,6 +1551,8 @@
{
struct tc358743_state *state = to_state(sd);
+ memset(edid->reserved, 0, sizeof(edid->reserved));
+
if (edid->pad != 0)
return -EINVAL;
@@ -1585,6 +1587,8 @@
v4l2_dbg(2, debug, sd, "%s, pad %d, start block %d, blocks %d\n",
__func__, edid->pad, edid->start_block, edid->blocks);
+ memset(edid->reserved, 0, sizeof(edid->reserved));
+
if (edid->pad != 0)
return -EINVAL;
@@ -1859,7 +1863,6 @@
/* control handlers */
v4l2_ctrl_handler_init(&state->hdl, 3);
- /* private controls */
state->detect_tx_5v_ctrl = v4l2_ctrl_new_std(&state->hdl, NULL,
V4L2_CID_DV_RX_POWER_PRESENT, 0, 1, 0, 0);
diff --git a/drivers/media/i2c/ths7303.c b/drivers/media/i2c/ths7303.c
index 5bbfcab..71a3135 100644
--- a/drivers/media/i2c/ths7303.c
+++ b/drivers/media/i2c/ths7303.c
@@ -285,7 +285,7 @@
v4l2_info(sd, "stream %s\n", state->stream_on ? "On" : "Off");
if (state->bt.pixelclock) {
- struct v4l2_bt_timings *bt = bt = &state->bt;
+ struct v4l2_bt_timings *bt = &state->bt;
u32 frame_width, frame_height;
frame_width = V4L2_DV_BT_FRAME_WIDTH(bt);
diff --git a/drivers/media/i2c/tvp5150.c b/drivers/media/i2c/tvp5150.c
index ff18444..0b6d46c 100644
--- a/drivers/media/i2c/tvp5150.c
+++ b/drivers/media/i2c/tvp5150.c
@@ -83,7 +83,7 @@
return rc;
}
-static inline void tvp5150_write(struct v4l2_subdev *sd, unsigned char addr,
+static int tvp5150_write(struct v4l2_subdev *sd, unsigned char addr,
unsigned char value)
{
struct i2c_client *c = v4l2_get_subdevdata(sd);
@@ -92,7 +92,9 @@
v4l2_dbg(2, debug, sd, "tvp5150: writing 0x%02x 0x%02x\n", addr, value);
rc = i2c_smbus_write_byte_data(c, addr, value);
if (rc < 0)
- v4l2_dbg(0, debug, sd, "i2c i/o error: rc == %d\n", rc);
+ v4l2_err(sd, "i2c i/o error: rc == %d\n", rc);
+
+ return rc;
}
static void dump_reg_range(struct v4l2_subdev *sd, char *s, u8 init,
@@ -1159,8 +1161,7 @@
static int tvp5150_s_register(struct v4l2_subdev *sd, const struct v4l2_dbg_register *reg)
{
- tvp5150_write(sd, reg->reg & 0xff, reg->val & 0xff);
- return 0;
+ return tvp5150_write(sd, reg->reg & 0xff, reg->val & 0xff);
}
#endif
diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
index 3cfd7af..a1cd50f 100644
--- a/drivers/media/media-device.c
+++ b/drivers/media/media-device.c
@@ -90,18 +90,13 @@
id &= ~MEDIA_ENT_ID_FLAG_NEXT;
- spin_lock(&mdev->lock);
-
media_device_for_each_entity(entity, mdev) {
if (((media_entity_id(entity) == id) && !next) ||
((media_entity_id(entity) > id) && next)) {
- spin_unlock(&mdev->lock);
return entity;
}
}
- spin_unlock(&mdev->lock);
-
return NULL;
}
@@ -431,6 +426,7 @@
struct media_device *dev = to_media_device(devnode);
long ret;
+ mutex_lock(&dev->graph_mutex);
switch (cmd) {
case MEDIA_IOC_DEVICE_INFO:
ret = media_device_get_info(dev,
@@ -443,29 +439,24 @@
break;
case MEDIA_IOC_ENUM_LINKS:
- mutex_lock(&dev->graph_mutex);
ret = media_device_enum_links(dev,
(struct media_links_enum __user *)arg);
- mutex_unlock(&dev->graph_mutex);
break;
case MEDIA_IOC_SETUP_LINK:
- mutex_lock(&dev->graph_mutex);
ret = media_device_setup_link(dev,
(struct media_link_desc __user *)arg);
- mutex_unlock(&dev->graph_mutex);
break;
case MEDIA_IOC_G_TOPOLOGY:
- mutex_lock(&dev->graph_mutex);
ret = media_device_get_topology(dev,
(struct media_v2_topology __user *)arg);
- mutex_unlock(&dev->graph_mutex);
break;
default:
ret = -ENOIOCTLCMD;
}
+ mutex_unlock(&dev->graph_mutex);
return ret;
}
@@ -508,12 +499,6 @@
long ret;
switch (cmd) {
- case MEDIA_IOC_DEVICE_INFO:
- case MEDIA_IOC_ENUM_ENTITIES:
- case MEDIA_IOC_SETUP_LINK:
- case MEDIA_IOC_G_TOPOLOGY:
- return media_device_ioctl(filp, cmd, arg);
-
case MEDIA_IOC_ENUM_LINKS32:
mutex_lock(&dev->graph_mutex);
ret = media_device_enum_links32(dev,
@@ -522,7 +507,7 @@
break;
default:
- ret = -ENOIOCTLCMD;
+ return media_device_ioctl(filp, cmd, arg);
}
return ret;
@@ -590,12 +575,12 @@
if (!ida_pre_get(&mdev->entity_internal_idx, GFP_KERNEL))
return -ENOMEM;
- spin_lock(&mdev->lock);
+ mutex_lock(&mdev->graph_mutex);
ret = ida_get_new_above(&mdev->entity_internal_idx, 1,
&entity->internal_idx);
if (ret < 0) {
- spin_unlock(&mdev->lock);
+ mutex_unlock(&mdev->graph_mutex);
return ret;
}
@@ -615,9 +600,6 @@
(notify)->notify(entity, notify->notify_data);
}
- spin_unlock(&mdev->lock);
-
- mutex_lock(&mdev->graph_mutex);
if (mdev->entity_internal_idx_max
>= mdev->pm_count_walk.ent_enum.idx_max) {
struct media_entity_graph new = { .top = 0 };
@@ -680,9 +662,9 @@
if (mdev == NULL)
return;
- spin_lock(&mdev->lock);
+ mutex_lock(&mdev->graph_mutex);
__media_device_unregister_entity(entity);
- spin_unlock(&mdev->lock);
+ mutex_unlock(&mdev->graph_mutex);
}
EXPORT_SYMBOL_GPL(media_device_unregister_entity);
@@ -703,7 +685,6 @@
INIT_LIST_HEAD(&mdev->pads);
INIT_LIST_HEAD(&mdev->links);
INIT_LIST_HEAD(&mdev->entity_notify);
- spin_lock_init(&mdev->lock);
mutex_init(&mdev->graph_mutex);
ida_init(&mdev->entity_internal_idx);
@@ -752,9 +733,9 @@
int __must_check media_device_register_entity_notify(struct media_device *mdev,
struct media_entity_notify *nptr)
{
- spin_lock(&mdev->lock);
+ mutex_lock(&mdev->graph_mutex);
list_add_tail(&nptr->list, &mdev->entity_notify);
- spin_unlock(&mdev->lock);
+ mutex_unlock(&mdev->graph_mutex);
return 0;
}
EXPORT_SYMBOL_GPL(media_device_register_entity_notify);
@@ -771,9 +752,9 @@
void media_device_unregister_entity_notify(struct media_device *mdev,
struct media_entity_notify *nptr)
{
- spin_lock(&mdev->lock);
+ mutex_lock(&mdev->graph_mutex);
__media_device_unregister_entity_notify(mdev, nptr);
- spin_unlock(&mdev->lock);
+ mutex_unlock(&mdev->graph_mutex);
}
EXPORT_SYMBOL_GPL(media_device_unregister_entity_notify);
@@ -787,11 +768,11 @@
if (mdev == NULL)
return;
- spin_lock(&mdev->lock);
+ mutex_lock(&mdev->graph_mutex);
/* Check if mdev was ever registered at all */
if (!media_devnode_is_registered(&mdev->devnode)) {
- spin_unlock(&mdev->lock);
+ mutex_unlock(&mdev->graph_mutex);
return;
}
@@ -811,12 +792,11 @@
kfree(intf);
}
- spin_unlock(&mdev->lock);
+ mutex_unlock(&mdev->graph_mutex);
device_remove_file(&mdev->devnode.dev, &dev_attr_model);
+ dev_dbg(mdev->dev, "Media device unregistering\n");
media_devnode_unregister(&mdev->devnode);
-
- dev_dbg(mdev->dev, "Media device unregistered\n");
}
EXPORT_SYMBOL_GPL(media_device_unregister);
diff --git a/drivers/media/media-devnode.c b/drivers/media/media-devnode.c
index 29409f4..b66dc9d 100644
--- a/drivers/media/media-devnode.c
+++ b/drivers/media/media-devnode.c
@@ -197,10 +197,11 @@
if (mdev->fops->release)
mdev->fops->release(filp);
+ filp->private_data = NULL;
+
/* decrease the refcount unconditionally since the release()
return value is ignored. */
put_device(&mdev->dev);
- filp->private_data = NULL;
return 0;
}
@@ -267,8 +268,11 @@
return 0;
error:
+ mutex_lock(&media_devnode_lock);
cdev_del(&mdev->cdev);
clear_bit(mdev->minor, media_devnode_nums);
+ mutex_unlock(&media_devnode_lock);
+
return ret;
}
diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
index e95070b..d8a2299 100644
--- a/drivers/media/media-entity.c
+++ b/drivers/media/media-entity.c
@@ -219,7 +219,7 @@
entity->pads = pads;
if (mdev)
- spin_lock(&mdev->lock);
+ mutex_lock(&mdev->graph_mutex);
for (i = 0; i < num_pads; i++) {
pads[i].entity = entity;
@@ -230,7 +230,7 @@
}
if (mdev)
- spin_unlock(&mdev->lock);
+ mutex_unlock(&mdev->graph_mutex);
return 0;
}
@@ -445,7 +445,7 @@
bitmap_or(active, active, has_no_links, entity->num_pads);
if (!bitmap_full(active, entity->num_pads)) {
- ret = -EPIPE;
+ ret = -ENOLINK;
dev_dbg(entity->graph_obj.mdev->dev,
"\"%s\":%u must be connected by an enabled link\n",
entity->name,
@@ -747,9 +747,9 @@
if (mdev == NULL)
return;
- spin_lock(&mdev->lock);
+ mutex_lock(&mdev->graph_mutex);
__media_entity_remove_links(entity);
- spin_unlock(&mdev->lock);
+ mutex_unlock(&mdev->graph_mutex);
}
EXPORT_SYMBOL_GPL(media_entity_remove_links);
@@ -951,9 +951,9 @@
if (mdev == NULL)
return;
- spin_lock(&mdev->lock);
+ mutex_lock(&mdev->graph_mutex);
__media_remove_intf_link(link);
- spin_unlock(&mdev->lock);
+ mutex_unlock(&mdev->graph_mutex);
}
EXPORT_SYMBOL_GPL(media_remove_intf_link);
@@ -975,8 +975,8 @@
if (mdev == NULL)
return;
- spin_lock(&mdev->lock);
+ mutex_lock(&mdev->graph_mutex);
__media_remove_intf_links(intf);
- spin_unlock(&mdev->lock);
+ mutex_unlock(&mdev->graph_mutex);
}
EXPORT_SYMBOL_GPL(media_remove_intf_links);
diff --git a/drivers/media/pci/Kconfig b/drivers/media/pci/Kconfig
index 48a611b..4f6467f 100644
--- a/drivers/media/pci/Kconfig
+++ b/drivers/media/pci/Kconfig
@@ -14,6 +14,7 @@
source "drivers/media/pci/solo6x10/Kconfig"
source "drivers/media/pci/sta2x11/Kconfig"
source "drivers/media/pci/tw68/Kconfig"
+source "drivers/media/pci/tw686x/Kconfig"
source "drivers/media/pci/zoran/Kconfig"
endif
diff --git a/drivers/media/pci/Makefile b/drivers/media/pci/Makefile
index 5f8aacb..2e54c36 100644
--- a/drivers/media/pci/Makefile
+++ b/drivers/media/pci/Makefile
@@ -25,6 +25,7 @@
obj-$(CONFIG_VIDEO_SAA7134) += saa7134/
obj-$(CONFIG_VIDEO_SAA7164) += saa7164/
obj-$(CONFIG_VIDEO_TW68) += tw68/
+obj-$(CONFIG_VIDEO_TW686X) += tw686x/
obj-$(CONFIG_VIDEO_DT3155) += dt3155/
obj-$(CONFIG_VIDEO_MEYE) += meye/
obj-$(CONFIG_STA2X11_VIP) += sta2x11/
diff --git a/drivers/media/pci/cobalt/Kconfig b/drivers/media/pci/cobalt/Kconfig
index a01f0cc..7034382 100644
--- a/drivers/media/pci/cobalt/Kconfig
+++ b/drivers/media/pci/cobalt/Kconfig
@@ -4,6 +4,7 @@
depends on PCI_MSI && MTD_COMPLEX_MAPPINGS
depends on GPIOLIB || COMPILE_TEST
depends on SND
+ depends on MTD
select I2C_ALGOBIT
select VIDEO_ADV7604
select VIDEO_ADV7511
diff --git a/drivers/media/pci/cx18/cx18-driver.h b/drivers/media/pci/cx18/cx18-driver.h
index 7e31f2a..47ce80f 100644
--- a/drivers/media/pci/cx18/cx18-driver.h
+++ b/drivers/media/pci/cx18/cx18-driver.h
@@ -707,11 +707,7 @@
/* Call the specified callback for all subdevs with a grp_id bit matching the
* mask in hw (if 0, then match them all). Ignore any errors. */
#define cx18_call_hw(cx, hw, o, f, args...) \
- do { \
- struct v4l2_subdev *__sd; \
- __v4l2_device_call_subdevs_p(&(cx)->v4l2_dev, __sd, \
- !(hw) || (__sd->grp_id & (hw)), o, f , ##args); \
- } while (0)
+ v4l2_device_mask_call_all(&(cx)->v4l2_dev, hw, o, f, ##args)
#define cx18_call_all(cx, o, f, args...) cx18_call_hw(cx, 0, o, f , ##args)
@@ -719,12 +715,7 @@
* mask in hw (if 0, then match them all). If the callback returns an error
* other than 0 or -ENOIOCTLCMD, then return with that error code. */
#define cx18_call_hw_err(cx, hw, o, f, args...) \
-({ \
- struct v4l2_subdev *__sd; \
- __v4l2_device_call_subdevs_until_err_p(&(cx)->v4l2_dev, \
- __sd, !(hw) || (__sd->grp_id & (hw)), o, f, \
- ##args); \
-})
+ v4l2_device_mask_call_until_err(&(cx)->v4l2_dev, hw, o, f, ##args)
#define cx18_call_all_err(cx, o, f, args...) \
cx18_call_hw_err(cx, 0, o, f , ##args)
diff --git a/drivers/media/pci/cx23885/cx23885-av.c b/drivers/media/pci/cx23885/cx23885-av.c
index 877dad8..e7d4406 100644
--- a/drivers/media/pci/cx23885/cx23885-av.c
+++ b/drivers/media/pci/cx23885/cx23885-av.c
@@ -24,7 +24,7 @@
{
struct cx23885_dev *dev =
container_of(work, struct cx23885_dev, cx25840_work);
- bool handled;
+ bool handled = false;
v4l2_subdev_call(dev->sd_cx25840, core, interrupt_service_routine,
PCI_MSK_AV_CORE, &handled);
diff --git a/drivers/media/pci/ivtv/ivtv-driver.h b/drivers/media/pci/ivtv/ivtv-driver.h
index 6c08dae..10cba305 100644
--- a/drivers/media/pci/ivtv/ivtv-driver.h
+++ b/drivers/media/pci/ivtv/ivtv-driver.h
@@ -827,12 +827,7 @@
/* Call the specified callback for all subdevs matching hw (if 0, then
match them all). Ignore any errors. */
#define ivtv_call_hw(itv, hw, o, f, args...) \
- do { \
- struct v4l2_subdev *__sd; \
- __v4l2_device_call_subdevs_p(&(itv)->v4l2_dev, __sd, \
- !(hw) ? true : (__sd->grp_id & (hw)), \
- o, f, ##args); \
- } while (0)
+ v4l2_device_mask_call_all(&(itv)->v4l2_dev, hw, o, f, ##args)
#define ivtv_call_all(itv, o, f, args...) ivtv_call_hw(itv, 0, o, f , ##args)
@@ -840,11 +835,7 @@
match them all). If the callback returns an error other than 0 or
-ENOIOCTLCMD, then return with that error code. */
#define ivtv_call_hw_err(itv, hw, o, f, args...) \
-({ \
- struct v4l2_subdev *__sd; \
- __v4l2_device_call_subdevs_until_err_p(&(itv)->v4l2_dev, __sd, \
- !(hw) || (__sd->grp_id & (hw)), o, f , ##args); \
-})
+ v4l2_device_mask_call_until_err(&(itv)->v4l2_dev, hw, o, f, ##args)
#define ivtv_call_all_err(itv, o, f, args...) ivtv_call_hw_err(itv, 0, o, f , ##args)
diff --git a/drivers/media/pci/smipcie/smipcie-ir.c b/drivers/media/pci/smipcie/smipcie-ir.c
index d018673..826c7c7 100644
--- a/drivers/media/pci/smipcie/smipcie-ir.c
+++ b/drivers/media/pci/smipcie/smipcie-ir.c
@@ -203,7 +203,7 @@
rc_dev->dev.parent = &dev->pci_dev->dev;
rc_dev->driver_type = RC_DRIVER_SCANCODE;
- rc_dev->map_name = RC_MAP_DVBSKY;
+ rc_dev->map_name = dev->info->rc_map;
ir->rc_dev = rc_dev;
ir->dev = dev;
diff --git a/drivers/media/pci/smipcie/smipcie-main.c b/drivers/media/pci/smipcie/smipcie-main.c
index b039a22..83981d61 100644
--- a/drivers/media/pci/smipcie/smipcie-main.c
+++ b/drivers/media/pci/smipcie/smipcie-main.c
@@ -716,7 +716,8 @@
/* init MAC.*/
ret = smi_read_eeprom(&dev->i2c_bus[0], 0xc0, mac_ee, 16);
dev_info(&port->dev->pci_dev->dev,
- "DVBSky SMI PCIe MAC= %pM\n", mac_ee + (port->idx)*8);
+ "%s port %d MAC: %pM\n", dev->info->name,
+ port->idx, mac_ee + (port->idx)*8);
memcpy(adap->proposed_mac, mac_ee + (port->idx)*8, 6);
return ret;
}
@@ -1066,6 +1067,7 @@
.ts_1 = SMI_TS_DMA_BOTH,
.fe_0 = DVBSKY_FE_NULL,
.fe_1 = DVBSKY_FE_M88DS3103,
+ .rc_map = RC_MAP_DVBSKY,
};
static struct smi_cfg_info dvbsky_s952_cfg = {
@@ -1075,6 +1077,7 @@
.ts_1 = SMI_TS_DMA_BOTH,
.fe_0 = DVBSKY_FE_M88RS6000,
.fe_1 = DVBSKY_FE_M88RS6000,
+ .rc_map = RC_MAP_DVBSKY,
};
static struct smi_cfg_info dvbsky_t9580_cfg = {
@@ -1084,6 +1087,17 @@
.ts_1 = SMI_TS_DMA_BOTH,
.fe_0 = DVBSKY_FE_SIT2,
.fe_1 = DVBSKY_FE_M88DS3103,
+ .rc_map = RC_MAP_DVBSKY,
+};
+
+static struct smi_cfg_info technotrend_s2_4200_cfg = {
+ .type = SMI_TECHNOTREND_S2_4200,
+ .name = "TechnoTrend TT-budget S2-4200 Twin",
+ .ts_0 = SMI_TS_DMA_BOTH,
+ .ts_1 = SMI_TS_DMA_BOTH,
+ .fe_0 = DVBSKY_FE_M88RS6000,
+ .fe_1 = DVBSKY_FE_M88RS6000,
+ .rc_map = RC_MAP_TT_1500,
};
/* PCI IDs */
@@ -1096,6 +1110,7 @@
SMI_ID(0x4254, 0x0550, dvbsky_s950_cfg),
SMI_ID(0x4254, 0x0552, dvbsky_s952_cfg),
SMI_ID(0x4254, 0x5580, dvbsky_t9580_cfg),
+ SMI_ID(0x13c2, 0x3016, technotrend_s2_4200_cfg),
{0}
};
MODULE_DEVICE_TABLE(pci, smi_id_table);
diff --git a/drivers/media/pci/smipcie/smipcie.h b/drivers/media/pci/smipcie/smipcie.h
index 68cdda2..611e4f0 100644
--- a/drivers/media/pci/smipcie/smipcie.h
+++ b/drivers/media/pci/smipcie/smipcie.h
@@ -216,6 +216,7 @@
#define SMI_DVBSKY_S950 1
#define SMI_DVBSKY_T9580 2
#define SMI_DVBSKY_T982 3
+#define SMI_TECHNOTREND_S2_4200 4
int type;
char *name;
#define SMI_TS_NULL 0
@@ -232,6 +233,7 @@
#define DVBSKY_FE_SIT2 3
int fe_0;
int fe_1;
+ char *rc_map;
};
struct smi_rc {
diff --git a/drivers/media/pci/sta2x11/sta2x11_vip.c b/drivers/media/pci/sta2x11/sta2x11_vip.c
index 753411c..1fc195f 100644
--- a/drivers/media/pci/sta2x11/sta2x11_vip.c
+++ b/drivers/media/pci/sta2x11/sta2x11_vip.c
@@ -444,27 +444,19 @@
static int vidioc_s_std(struct file *file, void *priv, v4l2_std_id std)
{
struct sta2x11_vip *vip = video_drvdata(file);
- v4l2_std_id oldstd = vip->std, newstd;
- int status;
- if (V4L2_STD_ALL == std) {
- v4l2_subdev_call(vip->decoder, video, s_std, std);
- ssleep(2);
- v4l2_subdev_call(vip->decoder, video, querystd, &newstd);
- v4l2_subdev_call(vip->decoder, video, g_input_status, &status);
- if (status & V4L2_IN_ST_NO_SIGNAL)
+ /*
+ * This is here for backwards compatibility only.
+ * The use of V4L2_STD_ALL to trigger a querystd is non-standard.
+ */
+ if (std == V4L2_STD_ALL) {
+ v4l2_subdev_call(vip->decoder, video, querystd, &std);
+ if (std == V4L2_STD_UNKNOWN)
return -EIO;
- std = vip->std = newstd;
- if (oldstd != std) {
- if (V4L2_STD_525_60 & std)
- vip->format = formats_60[0];
- else
- vip->format = formats_50[0];
- }
- return 0;
}
- if (oldstd != std) {
+ if (vip->std != std) {
+ vip->std = std;
if (V4L2_STD_525_60 & std)
vip->format = formats_60[0];
else
diff --git a/drivers/media/pci/tw686x/Kconfig b/drivers/media/pci/tw686x/Kconfig
new file mode 100644
index 0000000..fb85369
--- /dev/null
+++ b/drivers/media/pci/tw686x/Kconfig
@@ -0,0 +1,18 @@
+config VIDEO_TW686X
+ tristate "Intersil/Techwell TW686x video capture cards"
+ depends on PCI && VIDEO_DEV && VIDEO_V4L2 && SND
+ depends on HAS_DMA
+ select VIDEOBUF2_VMALLOC
+ select SND_PCM
+ help
+ Support for Intersil/Techwell TW686x-based frame grabber cards.
+
+ Currently supported chips:
+ - TW6864 (4 video channels),
+ - TW6865 (4 video channels, not tested, second generation chip),
+ - TW6868 (8 video channels but only 4 first channels using
+ built-in video decoder are supported, not tested),
+ - TW6869 (8 video channels, second generation chip).
+
+ To compile this driver as a module, choose M here: the module
+ will be named tw686x.
diff --git a/drivers/media/pci/tw686x/Makefile b/drivers/media/pci/tw686x/Makefile
new file mode 100644
index 0000000..9981954
--- /dev/null
+++ b/drivers/media/pci/tw686x/Makefile
@@ -0,0 +1,3 @@
+tw686x-objs := tw686x-core.o tw686x-video.o tw686x-audio.o
+
+obj-$(CONFIG_VIDEO_TW686X) += tw686x.o
diff --git a/drivers/media/pci/tw686x/tw686x-audio.c b/drivers/media/pci/tw686x/tw686x-audio.c
new file mode 100644
index 0000000..91459ab
--- /dev/null
+++ b/drivers/media/pci/tw686x/tw686x-audio.c
@@ -0,0 +1,386 @@
+/*
+ * Copyright (C) 2015 VanguardiaSur - www.vanguardiasur.com.ar
+ *
+ * Based on the audio support from the tw6869 driver:
+ * Copyright 2015 www.starterkit.ru <info@starterkit.ru>
+ *
+ * Based on:
+ * Driver for Intersil|Techwell TW6869 based DVR cards
+ * (c) 2011-12 liran <jli11@intersil.com> [Intersil|Techwell China]
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of version 2 of the GNU General Public License
+ * as published by the Free Software Foundation.
+ */
+
+#include <linux/types.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/kmod.h>
+#include <linux/mutex.h>
+#include <linux/pci.h>
+#include <linux/delay.h>
+
+#include <sound/core.h>
+#include <sound/initval.h>
+#include <sound/pcm.h>
+#include <sound/control.h>
+#include "tw686x.h"
+#include "tw686x-regs.h"
+
+#define AUDIO_CHANNEL_OFFSET 8
+
+void tw686x_audio_irq(struct tw686x_dev *dev, unsigned long requests,
+ unsigned int pb_status)
+{
+ unsigned long flags;
+ unsigned int ch, pb;
+
+ for_each_set_bit(ch, &requests, max_channels(dev)) {
+ struct tw686x_audio_channel *ac = &dev->audio_channels[ch];
+ struct tw686x_audio_buf *done = NULL;
+ struct tw686x_audio_buf *next = NULL;
+ struct tw686x_dma_desc *desc;
+
+ pb = !!(pb_status & BIT(AUDIO_CHANNEL_OFFSET + ch));
+
+ spin_lock_irqsave(&ac->lock, flags);
+
+ /* Sanity check */
+ if (!ac->ss || !ac->curr_bufs[0] || !ac->curr_bufs[1]) {
+ spin_unlock_irqrestore(&ac->lock, flags);
+ continue;
+ }
+
+ if (!list_empty(&ac->buf_list)) {
+ next = list_first_entry(&ac->buf_list,
+ struct tw686x_audio_buf, list);
+ list_move_tail(&next->list, &ac->buf_list);
+ done = ac->curr_bufs[!pb];
+ ac->curr_bufs[pb] = next;
+ }
+ spin_unlock_irqrestore(&ac->lock, flags);
+
+ desc = &ac->dma_descs[pb];
+ if (done && next && desc->virt) {
+ memcpy(done->virt, desc->virt, desc->size);
+ ac->ptr = done->dma - ac->buf[0].dma;
+ snd_pcm_period_elapsed(ac->ss);
+ }
+ }
+}
+
+static int tw686x_pcm_hw_params(struct snd_pcm_substream *ss,
+ struct snd_pcm_hw_params *hw_params)
+{
+ return snd_pcm_lib_malloc_pages(ss, params_buffer_bytes(hw_params));
+}
+
+static int tw686x_pcm_hw_free(struct snd_pcm_substream *ss)
+{
+ return snd_pcm_lib_free_pages(ss);
+}
+
+/*
+ * The audio device rate is global and shared among all
+ * capture channels. The driver makes no effort to prevent
+ * rate modifications. User is free change the rate, but it
+ * means changing the rate for all capture sub-devices.
+ */
+static const struct snd_pcm_hardware tw686x_capture_hw = {
+ .info = (SNDRV_PCM_INFO_MMAP |
+ SNDRV_PCM_INFO_INTERLEAVED |
+ SNDRV_PCM_INFO_BLOCK_TRANSFER |
+ SNDRV_PCM_INFO_MMAP_VALID),
+ .formats = SNDRV_PCM_FMTBIT_S16_LE,
+ .rates = SNDRV_PCM_RATE_8000_48000,
+ .rate_min = 8000,
+ .rate_max = 48000,
+ .channels_min = 1,
+ .channels_max = 1,
+ .buffer_bytes_max = TW686X_AUDIO_PAGE_MAX * TW686X_AUDIO_PAGE_SZ,
+ .period_bytes_min = TW686X_AUDIO_PAGE_SZ,
+ .period_bytes_max = TW686X_AUDIO_PAGE_SZ,
+ .periods_min = TW686X_AUDIO_PERIODS_MIN,
+ .periods_max = TW686X_AUDIO_PERIODS_MAX,
+};
+
+static int tw686x_pcm_open(struct snd_pcm_substream *ss)
+{
+ struct tw686x_dev *dev = snd_pcm_substream_chip(ss);
+ struct tw686x_audio_channel *ac = &dev->audio_channels[ss->number];
+ struct snd_pcm_runtime *rt = ss->runtime;
+ int err;
+
+ ac->ss = ss;
+ rt->hw = tw686x_capture_hw;
+
+ err = snd_pcm_hw_constraint_integer(rt, SNDRV_PCM_HW_PARAM_PERIODS);
+ if (err < 0)
+ return err;
+
+ return 0;
+}
+
+static int tw686x_pcm_close(struct snd_pcm_substream *ss)
+{
+ struct tw686x_dev *dev = snd_pcm_substream_chip(ss);
+ struct tw686x_audio_channel *ac = &dev->audio_channels[ss->number];
+
+ ac->ss = NULL;
+ return 0;
+}
+
+static int tw686x_pcm_prepare(struct snd_pcm_substream *ss)
+{
+ struct tw686x_dev *dev = snd_pcm_substream_chip(ss);
+ struct tw686x_audio_channel *ac = &dev->audio_channels[ss->number];
+ struct snd_pcm_runtime *rt = ss->runtime;
+ unsigned int period_size = snd_pcm_lib_period_bytes(ss);
+ struct tw686x_audio_buf *p_buf, *b_buf;
+ unsigned long flags;
+ int i;
+
+ spin_lock_irqsave(&dev->lock, flags);
+ tw686x_disable_channel(dev, AUDIO_CHANNEL_OFFSET + ac->ch);
+ spin_unlock_irqrestore(&dev->lock, flags);
+
+ if (dev->audio_rate != rt->rate) {
+ u32 reg;
+
+ dev->audio_rate = rt->rate;
+ reg = ((125000000 / rt->rate) << 16) +
+ ((125000000 % rt->rate) << 16) / rt->rate;
+
+ reg_write(dev, AUDIO_CONTROL2, reg);
+ }
+
+ if (period_size != TW686X_AUDIO_PAGE_SZ ||
+ rt->periods < TW686X_AUDIO_PERIODS_MIN ||
+ rt->periods > TW686X_AUDIO_PERIODS_MAX) {
+ return -EINVAL;
+ }
+
+ spin_lock_irqsave(&ac->lock, flags);
+ INIT_LIST_HEAD(&ac->buf_list);
+
+ for (i = 0; i < rt->periods; i++) {
+ ac->buf[i].dma = rt->dma_addr + period_size * i;
+ ac->buf[i].virt = rt->dma_area + period_size * i;
+ INIT_LIST_HEAD(&ac->buf[i].list);
+ list_add_tail(&ac->buf[i].list, &ac->buf_list);
+ }
+
+ p_buf = list_first_entry(&ac->buf_list, struct tw686x_audio_buf, list);
+ list_move_tail(&p_buf->list, &ac->buf_list);
+
+ b_buf = list_first_entry(&ac->buf_list, struct tw686x_audio_buf, list);
+ list_move_tail(&b_buf->list, &ac->buf_list);
+
+ ac->curr_bufs[0] = p_buf;
+ ac->curr_bufs[1] = b_buf;
+ ac->ptr = 0;
+ spin_unlock_irqrestore(&ac->lock, flags);
+
+ return 0;
+}
+
+static int tw686x_pcm_trigger(struct snd_pcm_substream *ss, int cmd)
+{
+ struct tw686x_dev *dev = snd_pcm_substream_chip(ss);
+ struct tw686x_audio_channel *ac = &dev->audio_channels[ss->number];
+ unsigned long flags;
+ int err = 0;
+
+ switch (cmd) {
+ case SNDRV_PCM_TRIGGER_START:
+ if (ac->curr_bufs[0] && ac->curr_bufs[1]) {
+ spin_lock_irqsave(&dev->lock, flags);
+ tw686x_enable_channel(dev,
+ AUDIO_CHANNEL_OFFSET + ac->ch);
+ spin_unlock_irqrestore(&dev->lock, flags);
+
+ mod_timer(&dev->dma_delay_timer,
+ jiffies + msecs_to_jiffies(100));
+ } else {
+ err = -EIO;
+ }
+ break;
+ case SNDRV_PCM_TRIGGER_STOP:
+ spin_lock_irqsave(&dev->lock, flags);
+ tw686x_disable_channel(dev, AUDIO_CHANNEL_OFFSET + ac->ch);
+ spin_unlock_irqrestore(&dev->lock, flags);
+
+ spin_lock_irqsave(&ac->lock, flags);
+ ac->curr_bufs[0] = NULL;
+ ac->curr_bufs[1] = NULL;
+ spin_unlock_irqrestore(&ac->lock, flags);
+ break;
+ default:
+ err = -EINVAL;
+ }
+ return err;
+}
+
+static snd_pcm_uframes_t tw686x_pcm_pointer(struct snd_pcm_substream *ss)
+{
+ struct tw686x_dev *dev = snd_pcm_substream_chip(ss);
+ struct tw686x_audio_channel *ac = &dev->audio_channels[ss->number];
+
+ return bytes_to_frames(ss->runtime, ac->ptr);
+}
+
+static struct snd_pcm_ops tw686x_pcm_ops = {
+ .open = tw686x_pcm_open,
+ .close = tw686x_pcm_close,
+ .ioctl = snd_pcm_lib_ioctl,
+ .hw_params = tw686x_pcm_hw_params,
+ .hw_free = tw686x_pcm_hw_free,
+ .prepare = tw686x_pcm_prepare,
+ .trigger = tw686x_pcm_trigger,
+ .pointer = tw686x_pcm_pointer,
+};
+
+static int tw686x_snd_pcm_init(struct tw686x_dev *dev)
+{
+ struct snd_card *card = dev->snd_card;
+ struct snd_pcm *pcm;
+ struct snd_pcm_substream *ss;
+ unsigned int i;
+ int err;
+
+ err = snd_pcm_new(card, card->driver, 0, 0, max_channels(dev), &pcm);
+ if (err < 0)
+ return err;
+
+ snd_pcm_set_ops(pcm, SNDRV_PCM_STREAM_CAPTURE, &tw686x_pcm_ops);
+ snd_pcm_chip(pcm) = dev;
+ pcm->info_flags = 0;
+ strlcpy(pcm->name, "tw686x PCM", sizeof(pcm->name));
+
+ for (i = 0, ss = pcm->streams[SNDRV_PCM_STREAM_CAPTURE].substream;
+ ss; ss = ss->next, i++)
+ snprintf(ss->name, sizeof(ss->name), "vch%u audio", i);
+
+ return snd_pcm_lib_preallocate_pages_for_all(pcm,
+ SNDRV_DMA_TYPE_DEV,
+ snd_dma_pci_data(dev->pci_dev),
+ TW686X_AUDIO_PAGE_MAX * TW686X_AUDIO_PAGE_SZ,
+ TW686X_AUDIO_PAGE_MAX * TW686X_AUDIO_PAGE_SZ);
+}
+
+static void tw686x_audio_dma_free(struct tw686x_dev *dev,
+ struct tw686x_audio_channel *ac)
+{
+ int pb;
+
+ for (pb = 0; pb < 2; pb++) {
+ if (!ac->dma_descs[pb].virt)
+ continue;
+ pci_free_consistent(dev->pci_dev, ac->dma_descs[pb].size,
+ ac->dma_descs[pb].virt,
+ ac->dma_descs[pb].phys);
+ ac->dma_descs[pb].virt = NULL;
+ }
+}
+
+static int tw686x_audio_dma_alloc(struct tw686x_dev *dev,
+ struct tw686x_audio_channel *ac)
+{
+ int pb;
+
+ for (pb = 0; pb < 2; pb++) {
+ u32 reg = pb ? ADMA_B_ADDR[ac->ch] : ADMA_P_ADDR[ac->ch];
+ void *virt;
+
+ virt = pci_alloc_consistent(dev->pci_dev, TW686X_AUDIO_PAGE_SZ,
+ &ac->dma_descs[pb].phys);
+ if (!virt) {
+ dev_err(&dev->pci_dev->dev,
+ "dma%d: unable to allocate audio DMA %s-buffer\n",
+ ac->ch, pb ? "B" : "P");
+ return -ENOMEM;
+ }
+ ac->dma_descs[pb].virt = virt;
+ ac->dma_descs[pb].size = TW686X_AUDIO_PAGE_SZ;
+ reg_write(dev, reg, ac->dma_descs[pb].phys);
+ }
+ return 0;
+}
+
+void tw686x_audio_free(struct tw686x_dev *dev)
+{
+ unsigned long flags;
+ u32 dma_ch_mask;
+ u32 dma_cmd;
+
+ spin_lock_irqsave(&dev->lock, flags);
+ dma_cmd = reg_read(dev, DMA_CMD);
+ dma_ch_mask = reg_read(dev, DMA_CHANNEL_ENABLE);
+ reg_write(dev, DMA_CMD, dma_cmd & ~0xff00);
+ reg_write(dev, DMA_CHANNEL_ENABLE, dma_ch_mask & ~0xff00);
+ spin_unlock_irqrestore(&dev->lock, flags);
+
+ if (!dev->snd_card)
+ return;
+ snd_card_free(dev->snd_card);
+ dev->snd_card = NULL;
+}
+
+int tw686x_audio_init(struct tw686x_dev *dev)
+{
+ struct pci_dev *pci_dev = dev->pci_dev;
+ struct snd_card *card;
+ int err, ch;
+
+ /*
+ * AUDIO_CONTROL1
+ * DMA byte length [31:19] = 4096 (i.e. ALSA period)
+ * External audio enable [0] = enabled
+ */
+ reg_write(dev, AUDIO_CONTROL1, 0x80000001);
+
+ err = snd_card_new(&pci_dev->dev, SNDRV_DEFAULT_IDX1,
+ SNDRV_DEFAULT_STR1,
+ THIS_MODULE, 0, &card);
+ if (err < 0)
+ return err;
+
+ dev->snd_card = card;
+ strlcpy(card->driver, "tw686x", sizeof(card->driver));
+ strlcpy(card->shortname, "tw686x", sizeof(card->shortname));
+ strlcpy(card->longname, pci_name(pci_dev), sizeof(card->longname));
+ snd_card_set_dev(card, &pci_dev->dev);
+
+ for (ch = 0; ch < max_channels(dev); ch++) {
+ struct tw686x_audio_channel *ac;
+
+ ac = &dev->audio_channels[ch];
+ spin_lock_init(&ac->lock);
+ ac->dev = dev;
+ ac->ch = ch;
+
+ err = tw686x_audio_dma_alloc(dev, ac);
+ if (err < 0)
+ goto err_cleanup;
+ }
+
+ err = tw686x_snd_pcm_init(dev);
+ if (err < 0)
+ goto err_cleanup;
+
+ err = snd_card_register(card);
+ if (!err)
+ return 0;
+
+err_cleanup:
+ for (ch = 0; ch < max_channels(dev); ch++) {
+ if (!dev->audio_channels[ch].dev)
+ continue;
+ tw686x_audio_dma_free(dev, &dev->audio_channels[ch]);
+ }
+ snd_card_free(card);
+ dev->snd_card = NULL;
+ return err;
+}
diff --git a/drivers/media/pci/tw686x/tw686x-core.c b/drivers/media/pci/tw686x/tw686x-core.c
new file mode 100644
index 0000000..cf53b0e
--- /dev/null
+++ b/drivers/media/pci/tw686x/tw686x-core.c
@@ -0,0 +1,415 @@
+/*
+ * Copyright (C) 2015 VanguardiaSur - www.vanguardiasur.com.ar
+ *
+ * Based on original driver by Krzysztof Ha?asa:
+ * Copyright (C) 2015 Industrial Research Institute for Automation
+ * and Measurements PIAP
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of version 2 of the GNU General Public License
+ * as published by the Free Software Foundation.
+ *
+ * Notes
+ * -----
+ *
+ * 1. Under stress-testing, it has been observed that the PCIe link
+ * goes down, without reason. Therefore, the driver takes special care
+ * to allow device hot-unplugging.
+ *
+ * 2. TW686X devices are capable of setting a few different DMA modes,
+ * including: scatter-gather, field and frame modes. However,
+ * under stress testings it has been found that the machine can
+ * freeze completely if DMA registers are programmed while streaming
+ * is active.
+ * This driver tries to access hardware registers as infrequently
+ * as possible by:
+ * i. allocating fixed DMA buffers and memcpy'ing into
+ * vmalloc'ed buffers
+ * ii. using a timer to mitigate the rate of DMA reset operations,
+ * on DMA channels error.
+ */
+
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/delay.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/pci_ids.h>
+#include <linux/slab.h>
+#include <linux/timer.h>
+
+#include "tw686x.h"
+#include "tw686x-regs.h"
+
+/*
+ * This module parameter allows to control the DMA_TIMER_INTERVAL value.
+ * The DMA_TIMER_INTERVAL register controls the minimum DMA interrupt
+ * time span (iow, the maximum DMA interrupt rate) thus allowing for
+ * IRQ coalescing.
+ *
+ * The chip datasheet does not mention a time unit for this value, so
+ * users wanting fine-grain control over the interrupt rate should
+ * determine the desired value through testing.
+ */
+static u32 dma_interval = 0x00098968;
+module_param(dma_interval, int, 0444);
+MODULE_PARM_DESC(dma_interval, "Minimum time span for DMA interrupting host");
+
+void tw686x_disable_channel(struct tw686x_dev *dev, unsigned int channel)
+{
+ u32 dma_en = reg_read(dev, DMA_CHANNEL_ENABLE);
+ u32 dma_cmd = reg_read(dev, DMA_CMD);
+
+ dma_en &= ~BIT(channel);
+ dma_cmd &= ~BIT(channel);
+
+ /* Must remove it from pending too */
+ dev->pending_dma_en &= ~BIT(channel);
+ dev->pending_dma_cmd &= ~BIT(channel);
+
+ /* Stop DMA if no channels are enabled */
+ if (!dma_en)
+ dma_cmd = 0;
+ reg_write(dev, DMA_CHANNEL_ENABLE, dma_en);
+ reg_write(dev, DMA_CMD, dma_cmd);
+}
+
+void tw686x_enable_channel(struct tw686x_dev *dev, unsigned int channel)
+{
+ u32 dma_en = reg_read(dev, DMA_CHANNEL_ENABLE);
+ u32 dma_cmd = reg_read(dev, DMA_CMD);
+
+ dev->pending_dma_en |= dma_en | BIT(channel);
+ dev->pending_dma_cmd |= dma_cmd | DMA_CMD_ENABLE | BIT(channel);
+}
+
+/*
+ * The purpose of this awful hack is to avoid enabling the DMA
+ * channels "too fast" which makes some TW686x devices very
+ * angry and freeze the CPU (see note 1).
+ */
+static void tw686x_dma_delay(unsigned long data)
+{
+ struct tw686x_dev *dev = (struct tw686x_dev *)data;
+ unsigned long flags;
+
+ spin_lock_irqsave(&dev->lock, flags);
+
+ reg_write(dev, DMA_CHANNEL_ENABLE, dev->pending_dma_en);
+ reg_write(dev, DMA_CMD, dev->pending_dma_cmd);
+ dev->pending_dma_en = 0;
+ dev->pending_dma_cmd = 0;
+
+ spin_unlock_irqrestore(&dev->lock, flags);
+}
+
+static void tw686x_reset_channels(struct tw686x_dev *dev, unsigned int ch_mask)
+{
+ u32 dma_en, dma_cmd;
+
+ dma_en = reg_read(dev, DMA_CHANNEL_ENABLE);
+ dma_cmd = reg_read(dev, DMA_CMD);
+
+ /*
+ * Save pending register status, the timer will
+ * restore them.
+ */
+ dev->pending_dma_en |= dma_en;
+ dev->pending_dma_cmd |= dma_cmd;
+
+ /* Disable the reset channels */
+ reg_write(dev, DMA_CHANNEL_ENABLE, dma_en & ~ch_mask);
+
+ if ((dma_en & ~ch_mask) == 0) {
+ dev_dbg(&dev->pci_dev->dev, "reset: stopping DMA\n");
+ dma_cmd &= ~DMA_CMD_ENABLE;
+ }
+ reg_write(dev, DMA_CMD, dma_cmd & ~ch_mask);
+}
+
+static irqreturn_t tw686x_irq(int irq, void *dev_id)
+{
+ struct tw686x_dev *dev = (struct tw686x_dev *)dev_id;
+ unsigned int video_requests, audio_requests, reset_ch;
+ u32 fifo_status, fifo_signal, fifo_ov, fifo_bad, fifo_errors;
+ u32 int_status, dma_en, video_en, pb_status;
+ unsigned long flags;
+
+ int_status = reg_read(dev, INT_STATUS); /* cleared on read */
+ fifo_status = reg_read(dev, VIDEO_FIFO_STATUS);
+
+ /* INT_STATUS does not include FIFO_STATUS errors! */
+ if (!int_status && !TW686X_FIFO_ERROR(fifo_status))
+ return IRQ_NONE;
+
+ if (int_status & INT_STATUS_DMA_TOUT) {
+ dev_dbg(&dev->pci_dev->dev,
+ "DMA timeout. Resetting DMA for all channels\n");
+ reset_ch = ~0;
+ goto reset_channels;
+ }
+
+ spin_lock_irqsave(&dev->lock, flags);
+ dma_en = reg_read(dev, DMA_CHANNEL_ENABLE);
+ spin_unlock_irqrestore(&dev->lock, flags);
+
+ video_en = dma_en & 0xff;
+ fifo_signal = ~(fifo_status & 0xff) & video_en;
+ fifo_ov = fifo_status >> 24;
+ fifo_bad = fifo_status >> 16;
+
+ /* Mask of channels with signal and FIFO errors */
+ fifo_errors = fifo_signal & (fifo_ov | fifo_bad);
+
+ reset_ch = 0;
+ pb_status = reg_read(dev, PB_STATUS);
+
+ /* Coalesce video frame/error events */
+ video_requests = (int_status & video_en) | fifo_errors;
+ audio_requests = (int_status & dma_en) >> 8;
+
+ if (video_requests)
+ tw686x_video_irq(dev, video_requests, pb_status,
+ fifo_status, &reset_ch);
+ if (audio_requests)
+ tw686x_audio_irq(dev, audio_requests, pb_status);
+
+reset_channels:
+ if (reset_ch) {
+ spin_lock_irqsave(&dev->lock, flags);
+ tw686x_reset_channels(dev, reset_ch);
+ spin_unlock_irqrestore(&dev->lock, flags);
+ mod_timer(&dev->dma_delay_timer,
+ jiffies + msecs_to_jiffies(100));
+ }
+
+ return IRQ_HANDLED;
+}
+
+static void tw686x_dev_release(struct v4l2_device *v4l2_dev)
+{
+ struct tw686x_dev *dev = container_of(v4l2_dev, struct tw686x_dev,
+ v4l2_dev);
+ unsigned int ch;
+
+ for (ch = 0; ch < max_channels(dev); ch++)
+ v4l2_ctrl_handler_free(&dev->video_channels[ch].ctrl_handler);
+
+ v4l2_device_unregister(&dev->v4l2_dev);
+
+ kfree(dev->audio_channels);
+ kfree(dev->video_channels);
+ kfree(dev);
+}
+
+static int tw686x_probe(struct pci_dev *pci_dev,
+ const struct pci_device_id *pci_id)
+{
+ struct tw686x_dev *dev;
+ int err;
+
+ dev = kzalloc(sizeof(*dev), GFP_KERNEL);
+ if (!dev)
+ return -ENOMEM;
+ dev->type = pci_id->driver_data;
+ sprintf(dev->name, "tw%04X", pci_dev->device);
+
+ dev->video_channels = kcalloc(max_channels(dev),
+ sizeof(*dev->video_channels), GFP_KERNEL);
+ if (!dev->video_channels) {
+ err = -ENOMEM;
+ goto free_dev;
+ }
+
+ dev->audio_channels = kcalloc(max_channels(dev),
+ sizeof(*dev->audio_channels), GFP_KERNEL);
+ if (!dev->audio_channels) {
+ err = -ENOMEM;
+ goto free_video;
+ }
+
+ pr_info("%s: PCI %s, IRQ %d, MMIO 0x%lx\n", dev->name,
+ pci_name(pci_dev), pci_dev->irq,
+ (unsigned long)pci_resource_start(pci_dev, 0));
+
+ dev->pci_dev = pci_dev;
+ if (pci_enable_device(pci_dev)) {
+ err = -EIO;
+ goto free_audio;
+ }
+
+ pci_set_master(pci_dev);
+ err = pci_set_dma_mask(pci_dev, DMA_BIT_MASK(32));
+ if (err) {
+ dev_err(&pci_dev->dev, "32-bit PCI DMA not supported\n");
+ err = -EIO;
+ goto disable_pci;
+ }
+
+ err = pci_request_regions(pci_dev, dev->name);
+ if (err) {
+ dev_err(&pci_dev->dev, "unable to request PCI region\n");
+ goto disable_pci;
+ }
+
+ dev->mmio = pci_ioremap_bar(pci_dev, 0);
+ if (!dev->mmio) {
+ dev_err(&pci_dev->dev, "unable to remap PCI region\n");
+ err = -ENOMEM;
+ goto free_region;
+ }
+
+ /* Reset all subsystems */
+ reg_write(dev, SYS_SOFT_RST, 0x0f);
+ mdelay(1);
+
+ reg_write(dev, SRST[0], 0x3f);
+ if (max_channels(dev) > 4)
+ reg_write(dev, SRST[1], 0x3f);
+
+ /* Disable the DMA engine */
+ reg_write(dev, DMA_CMD, 0);
+ reg_write(dev, DMA_CHANNEL_ENABLE, 0);
+
+ /* Enable DMA FIFO overflow and pointer check */
+ reg_write(dev, DMA_CONFIG, 0xffffff04);
+ reg_write(dev, DMA_CHANNEL_TIMEOUT, 0x140c8584);
+ reg_write(dev, DMA_TIMER_INTERVAL, dma_interval);
+
+ spin_lock_init(&dev->lock);
+
+ err = request_irq(pci_dev->irq, tw686x_irq, IRQF_SHARED,
+ dev->name, dev);
+ if (err < 0) {
+ dev_err(&pci_dev->dev, "unable to request interrupt\n");
+ goto iounmap;
+ }
+
+ setup_timer(&dev->dma_delay_timer,
+ tw686x_dma_delay, (unsigned long) dev);
+
+ /*
+ * This must be set right before initializing v4l2_dev.
+ * It's used to release resources after the last handle
+ * held is released.
+ */
+ dev->v4l2_dev.release = tw686x_dev_release;
+ err = tw686x_video_init(dev);
+ if (err) {
+ dev_err(&pci_dev->dev, "can't register video\n");
+ goto free_irq;
+ }
+
+ err = tw686x_audio_init(dev);
+ if (err)
+ dev_warn(&pci_dev->dev, "can't register audio\n");
+
+ pci_set_drvdata(pci_dev, dev);
+ return 0;
+
+free_irq:
+ free_irq(pci_dev->irq, dev);
+iounmap:
+ pci_iounmap(pci_dev, dev->mmio);
+free_region:
+ pci_release_regions(pci_dev);
+disable_pci:
+ pci_disable_device(pci_dev);
+free_audio:
+ kfree(dev->audio_channels);
+free_video:
+ kfree(dev->video_channels);
+free_dev:
+ kfree(dev);
+ return err;
+}
+
+static void tw686x_remove(struct pci_dev *pci_dev)
+{
+ struct tw686x_dev *dev = pci_get_drvdata(pci_dev);
+ unsigned long flags;
+
+ /* This guarantees the IRQ handler is no longer running,
+ * which means we can kiss good-bye some resources.
+ */
+ free_irq(pci_dev->irq, dev);
+
+ tw686x_video_free(dev);
+ tw686x_audio_free(dev);
+ del_timer_sync(&dev->dma_delay_timer);
+
+ pci_iounmap(pci_dev, dev->mmio);
+ pci_release_regions(pci_dev);
+ pci_disable_device(pci_dev);
+
+ /*
+ * Setting pci_dev to NULL allows to detect hardware is no longer
+ * available and will be used by vb2_ops. This is required because
+ * the device sometimes hot-unplugs itself as the result of a PCIe
+ * link down.
+ * The lock is really important here.
+ */
+ spin_lock_irqsave(&dev->lock, flags);
+ dev->pci_dev = NULL;
+ spin_unlock_irqrestore(&dev->lock, flags);
+
+ /*
+ * This calls tw686x_dev_release if it's the last reference.
+ * Otherwise, release is postponed until there are no users left.
+ */
+ v4l2_device_put(&dev->v4l2_dev);
+}
+
+/*
+ * On TW6864 and TW6868, all channels share the pair of video DMA SG tables,
+ * with 10-bit start_idx and end_idx determining start and end of frame buffer
+ * for particular channel.
+ * TW6868 with all its 8 channels would be problematic (only 127 SG entries per
+ * channel) but we support only 4 channels on this chip anyway (the first
+ * 4 channels are driven with internal video decoder, the other 4 would require
+ * an external TW286x part).
+ *
+ * On TW6865 and TW6869, each channel has its own DMA SG table, with indexes
+ * starting with 0. Both chips have complete sets of internal video decoders
+ * (respectively 4 or 8-channel).
+ *
+ * All chips have separate SG tables for two video frames.
+ */
+
+/* driver_data is number of A/V channels */
+static const struct pci_device_id tw686x_pci_tbl[] = {
+ {
+ PCI_DEVICE(PCI_VENDOR_ID_TECHWELL, 0x6864),
+ .driver_data = 4
+ },
+ {
+ PCI_DEVICE(PCI_VENDOR_ID_TECHWELL, 0x6865), /* not tested */
+ .driver_data = 4 | TYPE_SECOND_GEN
+ },
+ /*
+ * TW6868 supports 8 A/V channels with an external TW2865 chip;
+ * not supported by the driver.
+ */
+ {
+ PCI_DEVICE(PCI_VENDOR_ID_TECHWELL, 0x6868), /* not tested */
+ .driver_data = 4
+ },
+ {
+ PCI_DEVICE(PCI_VENDOR_ID_TECHWELL, 0x6869),
+ .driver_data = 8 | TYPE_SECOND_GEN},
+ {}
+};
+MODULE_DEVICE_TABLE(pci, tw686x_pci_tbl);
+
+static struct pci_driver tw686x_pci_driver = {
+ .name = "tw686x",
+ .id_table = tw686x_pci_tbl,
+ .probe = tw686x_probe,
+ .remove = tw686x_remove,
+};
+module_pci_driver(tw686x_pci_driver);
+
+MODULE_DESCRIPTION("Driver for video frame grabber cards based on Intersil/Techwell TW686[4589]");
+MODULE_AUTHOR("Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>");
+MODULE_AUTHOR("Krzysztof Ha?asa <khalasa@piap.pl>");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/media/pci/tw686x/tw686x-regs.h b/drivers/media/pci/tw686x/tw686x-regs.h
new file mode 100644
index 0000000..fcef586
--- /dev/null
+++ b/drivers/media/pci/tw686x/tw686x-regs.h
@@ -0,0 +1,122 @@
+/* DMA controller registers */
+#define REG8_1(a0) ((const u16[8]) { a0, a0 + 1, a0 + 2, a0 + 3, \
+ a0 + 4, a0 + 5, a0 + 6, a0 + 7})
+#define REG8_2(a0) ((const u16[8]) { a0, a0 + 2, a0 + 4, a0 + 6, \
+ a0 + 8, a0 + 0xa, a0 + 0xc, a0 + 0xe})
+#define REG8_8(a0) ((const u16[8]) { a0, a0 + 8, a0 + 0x10, a0 + 0x18, \
+ a0 + 0x20, a0 + 0x28, a0 + 0x30, \
+ a0 + 0x38})
+#define INT_STATUS 0x00
+#define PB_STATUS 0x01
+#define DMA_CMD 0x02
+#define VIDEO_FIFO_STATUS 0x03
+#define VIDEO_CHANNEL_ID 0x04
+#define VIDEO_PARSER_STATUS 0x05
+#define SYS_SOFT_RST 0x06
+#define DMA_PAGE_TABLE0_ADDR ((const u16[8]) { 0x08, 0xd0, 0xd2, 0xd4, \
+ 0xd6, 0xd8, 0xda, 0xdc })
+#define DMA_PAGE_TABLE1_ADDR ((const u16[8]) { 0x09, 0xd1, 0xd3, 0xd5, \
+ 0xd7, 0xd9, 0xdb, 0xdd })
+#define DMA_CHANNEL_ENABLE 0x0a
+#define DMA_CONFIG 0x0b
+#define DMA_TIMER_INTERVAL 0x0c
+#define DMA_CHANNEL_TIMEOUT 0x0d
+#define VDMA_CHANNEL_CONFIG REG8_1(0x10)
+#define ADMA_P_ADDR REG8_2(0x18)
+#define ADMA_B_ADDR REG8_2(0x19)
+#define DMA10_P_ADDR 0x28
+#define DMA10_B_ADDR 0x29
+#define VIDEO_CONTROL1 0x2a
+#define VIDEO_CONTROL2 0x2b
+#define AUDIO_CONTROL1 0x2c
+#define AUDIO_CONTROL2 0x2d
+#define PHASE_REF 0x2e
+#define GPIO_REG 0x2f
+#define INTL_HBAR_CTRL REG8_1(0x30)
+#define AUDIO_CONTROL3 0x38
+#define VIDEO_FIELD_CTRL REG8_1(0x39)
+#define HSCALER_CTRL REG8_1(0x42)
+#define VIDEO_SIZE REG8_1(0x4A)
+#define VIDEO_SIZE_F2 REG8_1(0x52)
+#define MD_CONF REG8_1(0x60)
+#define MD_INIT REG8_1(0x68)
+#define MD_MAP0 REG8_1(0x70)
+#define VDMA_P_ADDR REG8_8(0x80) /* not used in DMA SG mode */
+#define VDMA_WHP REG8_8(0x81)
+#define VDMA_B_ADDR REG8_8(0x82)
+#define VDMA_F2_P_ADDR REG8_8(0x84)
+#define VDMA_F2_WHP REG8_8(0x85)
+#define VDMA_F2_B_ADDR REG8_8(0x86)
+#define EP_REG_ADDR 0xfe
+#define EP_REG_DATA 0xff
+
+/* Video decoder registers */
+#define VDREG8(a0) ((const u16[8]) { \
+ a0 + 0x000, a0 + 0x010, a0 + 0x020, a0 + 0x030, \
+ a0 + 0x100, a0 + 0x110, a0 + 0x120, a0 + 0x130})
+#define VIDSTAT VDREG8(0x100)
+#define BRIGHT VDREG8(0x101)
+#define CONTRAST VDREG8(0x102)
+#define SHARPNESS VDREG8(0x103)
+#define SAT_U VDREG8(0x104)
+#define SAT_V VDREG8(0x105)
+#define HUE VDREG8(0x106)
+#define CROP_HI VDREG8(0x107)
+#define VDELAY_LO VDREG8(0x108)
+#define VACTIVE_LO VDREG8(0x109)
+#define HDELAY_LO VDREG8(0x10a)
+#define HACTIVE_LO VDREG8(0x10b)
+#define MVSN VDREG8(0x10c)
+#define STATUS2 VDREG8(0x10d)
+#define SDT VDREG8(0x10e)
+#define SDT_EN VDREG8(0x10f)
+
+#define VSCALE_LO VDREG8(0x144)
+#define SCALE_HI VDREG8(0x145)
+#define HSCALE_LO VDREG8(0x146)
+#define F2CROP_HI VDREG8(0x147)
+#define F2VDELAY_LO VDREG8(0x148)
+#define F2VACTIVE_LO VDREG8(0x149)
+#define F2HDELAY_LO VDREG8(0x14a)
+#define F2HACTIVE_LO VDREG8(0x14b)
+#define F2VSCALE_LO VDREG8(0x14c)
+#define F2SCALE_HI VDREG8(0x14d)
+#define F2HSCALE_LO VDREG8(0x14e)
+#define F2CNT VDREG8(0x14f)
+
+#define VDREG2(a0) ((const u16[2]) { a0, a0 + 0x100 })
+#define SRST VDREG2(0x180)
+#define ACNTL VDREG2(0x181)
+#define ACNTL2 VDREG2(0x182)
+#define CNTRL1 VDREG2(0x183)
+#define CKHY VDREG2(0x184)
+#define SHCOR VDREG2(0x185)
+#define CORING VDREG2(0x186)
+#define CLMPG VDREG2(0x187)
+#define IAGC VDREG2(0x188)
+#define VCTRL1 VDREG2(0x18f)
+#define MISC1 VDREG2(0x194)
+#define LOOP VDREG2(0x195)
+#define MISC2 VDREG2(0x196)
+
+#define CLMD VDREG2(0x197)
+#define ANPWRDOWN VDREG2(0x1ce)
+#define AIGAIN ((const u16[8]) { 0x1d0, 0x1d1, 0x1d2, 0x1d3, \
+ 0x2d0, 0x2d1, 0x2d2, 0x2d3 })
+
+#define SYS_MODE_DMA_SHIFT 13
+
+#define DMA_CMD_ENABLE BIT(31)
+#define INT_STATUS_DMA_TOUT BIT(17)
+#define TW686X_VIDSTAT_HLOCK BIT(6)
+#define TW686X_VIDSTAT_VDLOSS BIT(7)
+
+#define TW686X_STD_NTSC_M 0
+#define TW686X_STD_PAL 1
+#define TW686X_STD_SECAM 2
+#define TW686X_STD_NTSC_443 3
+#define TW686X_STD_PAL_M 4
+#define TW686X_STD_PAL_CN 5
+#define TW686X_STD_PAL_60 6
+
+#define TW686X_FIFO_ERROR(x) (x & ~(0xff))
diff --git a/drivers/media/pci/tw686x/tw686x-video.c b/drivers/media/pci/tw686x/tw686x-video.c
new file mode 100644
index 0000000..253e108
--- /dev/null
+++ b/drivers/media/pci/tw686x/tw686x-video.c
@@ -0,0 +1,937 @@
+/*
+ * Copyright (C) 2015 VanguardiaSur - www.vanguardiasur.com.ar
+ *
+ * Based on original driver by Krzysztof Ha?asa:
+ * Copyright (C) 2015 Industrial Research Institute for Automation
+ * and Measurements PIAP
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of version 2 of the GNU General Public License
+ * as published by the Free Software Foundation.
+ *
+ */
+
+#include <linux/init.h>
+#include <linux/delay.h>
+#include <linux/list.h>
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+#include <media/v4l2-common.h>
+#include <media/v4l2-event.h>
+#include <media/videobuf2-vmalloc.h>
+#include "tw686x.h"
+#include "tw686x-regs.h"
+
+#define TW686X_INPUTS_PER_CH 4
+#define TW686X_VIDEO_WIDTH 720
+#define TW686X_VIDEO_HEIGHT(id) ((id & V4L2_STD_525_60) ? 480 : 576)
+
+static const struct tw686x_format formats[] = {
+ {
+ .fourcc = V4L2_PIX_FMT_UYVY,
+ .mode = 0,
+ .depth = 16,
+ }, {
+ .fourcc = V4L2_PIX_FMT_RGB565,
+ .mode = 5,
+ .depth = 16,
+ }, {
+ .fourcc = V4L2_PIX_FMT_YUYV,
+ .mode = 6,
+ .depth = 16,
+ }
+};
+
+static unsigned int tw686x_fields_map(v4l2_std_id std, unsigned int fps)
+{
+ static const unsigned int map[15] = {
+ 0x00000000, 0x00000001, 0x00004001, 0x00104001, 0x00404041,
+ 0x01041041, 0x01104411, 0x01111111, 0x04444445, 0x04511445,
+ 0x05145145, 0x05151515, 0x05515455, 0x05551555, 0x05555555
+ };
+
+ static const unsigned int std_625_50[26] = {
+ 0, 1, 1, 2, 3, 3, 4, 4, 5, 5, 6, 7, 7,
+ 8, 8, 9, 10, 10, 11, 11, 12, 13, 13, 14, 14, 0
+ };
+
+ static const unsigned int std_525_60[31] = {
+ 0, 1, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7,
+ 8, 8, 9, 9, 10, 10, 11, 11, 12, 12, 13, 13, 14, 0, 0
+ };
+
+ unsigned int i;
+
+ if (std & V4L2_STD_525_60) {
+ if (fps >= ARRAY_SIZE(std_525_60))
+ fps = 30;
+ i = std_525_60[fps];
+ } else {
+ if (fps >= ARRAY_SIZE(std_625_50))
+ fps = 25;
+ i = std_625_50[fps];
+ }
+
+ return map[i];
+}
+
+static void tw686x_set_framerate(struct tw686x_video_channel *vc,
+ unsigned int fps)
+{
+ unsigned int map;
+
+ if (vc->fps == fps)
+ return;
+
+ map = tw686x_fields_map(vc->video_standard, fps) << 1;
+ map |= map << 1;
+ if (map > 0)
+ map |= BIT(31);
+ reg_write(vc->dev, VIDEO_FIELD_CTRL[vc->ch], map);
+ vc->fps = fps;
+}
+
+static const struct tw686x_format *format_by_fourcc(unsigned int fourcc)
+{
+ unsigned int cnt;
+
+ for (cnt = 0; cnt < ARRAY_SIZE(formats); cnt++)
+ if (formats[cnt].fourcc == fourcc)
+ return &formats[cnt];
+ return NULL;
+}
+
+static int tw686x_queue_setup(struct vb2_queue *vq,
+ unsigned int *nbuffers, unsigned int *nplanes,
+ unsigned int sizes[], void *alloc_ctxs[])
+{
+ struct tw686x_video_channel *vc = vb2_get_drv_priv(vq);
+ unsigned int szimage =
+ (vc->width * vc->height * vc->format->depth) >> 3;
+
+ /*
+ * Let's request at least three buffers: two for the
+ * DMA engine and one for userspace.
+ */
+ if (vq->num_buffers + *nbuffers < 3)
+ *nbuffers = 3 - vq->num_buffers;
+
+ if (*nplanes) {
+ if (*nplanes != 1 || sizes[0] < szimage)
+ return -EINVAL;
+ return 0;
+ }
+
+ sizes[0] = szimage;
+ *nplanes = 1;
+ return 0;
+}
+
+static void tw686x_buf_queue(struct vb2_buffer *vb)
+{
+ struct tw686x_video_channel *vc = vb2_get_drv_priv(vb->vb2_queue);
+ struct tw686x_dev *dev = vc->dev;
+ struct pci_dev *pci_dev;
+ unsigned long flags;
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct tw686x_v4l2_buf *buf =
+ container_of(vbuf, struct tw686x_v4l2_buf, vb);
+
+ /* Check device presence */
+ spin_lock_irqsave(&dev->lock, flags);
+ pci_dev = dev->pci_dev;
+ spin_unlock_irqrestore(&dev->lock, flags);
+ if (!pci_dev) {
+ vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
+ return;
+ }
+
+ spin_lock_irqsave(&vc->qlock, flags);
+ list_add_tail(&buf->list, &vc->vidq_queued);
+ spin_unlock_irqrestore(&vc->qlock, flags);
+}
+
+/*
+ * We can call this even when alloc_dma failed for the given channel
+ */
+static void tw686x_free_dma(struct tw686x_video_channel *vc, unsigned int pb)
+{
+ struct tw686x_dma_desc *desc = &vc->dma_descs[pb];
+ struct tw686x_dev *dev = vc->dev;
+ struct pci_dev *pci_dev;
+ unsigned long flags;
+
+ /* Check device presence. Shouldn't really happen! */
+ spin_lock_irqsave(&dev->lock, flags);
+ pci_dev = dev->pci_dev;
+ spin_unlock_irqrestore(&dev->lock, flags);
+ if (!pci_dev) {
+ WARN(1, "trying to deallocate on missing device\n");
+ return;
+ }
+
+ if (desc->virt) {
+ pci_free_consistent(dev->pci_dev, desc->size,
+ desc->virt, desc->phys);
+ desc->virt = NULL;
+ }
+}
+
+static int tw686x_alloc_dma(struct tw686x_video_channel *vc, unsigned int pb)
+{
+ struct tw686x_dev *dev = vc->dev;
+ u32 reg = pb ? VDMA_B_ADDR[vc->ch] : VDMA_P_ADDR[vc->ch];
+ unsigned int len;
+ void *virt;
+
+ WARN(vc->dma_descs[pb].virt,
+ "Allocating buffer but previous still here\n");
+
+ len = (vc->width * vc->height * vc->format->depth) >> 3;
+ virt = pci_alloc_consistent(dev->pci_dev, len,
+ &vc->dma_descs[pb].phys);
+ if (!virt) {
+ v4l2_err(&dev->v4l2_dev,
+ "dma%d: unable to allocate %s-buffer\n",
+ vc->ch, pb ? "B" : "P");
+ return -ENOMEM;
+ }
+ vc->dma_descs[pb].size = len;
+ vc->dma_descs[pb].virt = virt;
+ reg_write(dev, reg, vc->dma_descs[pb].phys);
+
+ return 0;
+}
+
+static void tw686x_buffer_refill(struct tw686x_video_channel *vc,
+ unsigned int pb)
+{
+ struct tw686x_v4l2_buf *buf;
+
+ while (!list_empty(&vc->vidq_queued)) {
+
+ buf = list_first_entry(&vc->vidq_queued,
+ struct tw686x_v4l2_buf, list);
+ list_del(&buf->list);
+
+ vc->curr_bufs[pb] = buf;
+ return;
+ }
+ vc->curr_bufs[pb] = NULL;
+}
+
+static void tw686x_clear_queue(struct tw686x_video_channel *vc,
+ enum vb2_buffer_state state)
+{
+ unsigned int pb;
+
+ while (!list_empty(&vc->vidq_queued)) {
+ struct tw686x_v4l2_buf *buf;
+
+ buf = list_first_entry(&vc->vidq_queued,
+ struct tw686x_v4l2_buf, list);
+ list_del(&buf->list);
+ vb2_buffer_done(&buf->vb.vb2_buf, state);
+ }
+
+ for (pb = 0; pb < 2; pb++) {
+ if (vc->curr_bufs[pb])
+ vb2_buffer_done(&vc->curr_bufs[pb]->vb.vb2_buf, state);
+ vc->curr_bufs[pb] = NULL;
+ }
+}
+
+static int tw686x_start_streaming(struct vb2_queue *vq, unsigned int count)
+{
+ struct tw686x_video_channel *vc = vb2_get_drv_priv(vq);
+ struct tw686x_dev *dev = vc->dev;
+ struct pci_dev *pci_dev;
+ unsigned long flags;
+ int pb, err;
+
+ /* Check device presence */
+ spin_lock_irqsave(&dev->lock, flags);
+ pci_dev = dev->pci_dev;
+ spin_unlock_irqrestore(&dev->lock, flags);
+ if (!pci_dev) {
+ err = -ENODEV;
+ goto err_clear_queue;
+ }
+
+ spin_lock_irqsave(&vc->qlock, flags);
+
+ /* Sanity check */
+ if (!vc->dma_descs[0].virt || !vc->dma_descs[1].virt) {
+ spin_unlock_irqrestore(&vc->qlock, flags);
+ v4l2_err(&dev->v4l2_dev,
+ "video%d: refusing to start without DMA buffers\n",
+ vc->num);
+ err = -ENOMEM;
+ goto err_clear_queue;
+ }
+
+ for (pb = 0; pb < 2; pb++)
+ tw686x_buffer_refill(vc, pb);
+ spin_unlock_irqrestore(&vc->qlock, flags);
+
+ vc->sequence = 0;
+ vc->pb = 0;
+
+ spin_lock_irqsave(&dev->lock, flags);
+ tw686x_enable_channel(dev, vc->ch);
+ spin_unlock_irqrestore(&dev->lock, flags);
+
+ mod_timer(&dev->dma_delay_timer, jiffies + msecs_to_jiffies(100));
+
+ return 0;
+
+err_clear_queue:
+ spin_lock_irqsave(&vc->qlock, flags);
+ tw686x_clear_queue(vc, VB2_BUF_STATE_QUEUED);
+ spin_unlock_irqrestore(&vc->qlock, flags);
+ return err;
+}
+
+static void tw686x_stop_streaming(struct vb2_queue *vq)
+{
+ struct tw686x_video_channel *vc = vb2_get_drv_priv(vq);
+ struct tw686x_dev *dev = vc->dev;
+ struct pci_dev *pci_dev;
+ unsigned long flags;
+
+ /* Check device presence */
+ spin_lock_irqsave(&dev->lock, flags);
+ pci_dev = dev->pci_dev;
+ spin_unlock_irqrestore(&dev->lock, flags);
+ if (pci_dev)
+ tw686x_disable_channel(dev, vc->ch);
+
+ spin_lock_irqsave(&vc->qlock, flags);
+ tw686x_clear_queue(vc, VB2_BUF_STATE_ERROR);
+ spin_unlock_irqrestore(&vc->qlock, flags);
+}
+
+static int tw686x_buf_prepare(struct vb2_buffer *vb)
+{
+ struct tw686x_video_channel *vc = vb2_get_drv_priv(vb->vb2_queue);
+ unsigned int size =
+ (vc->width * vc->height * vc->format->depth) >> 3;
+
+ if (vb2_plane_size(vb, 0) < size)
+ return -EINVAL;
+ vb2_set_plane_payload(vb, 0, size);
+ return 0;
+}
+
+static struct vb2_ops tw686x_video_qops = {
+ .queue_setup = tw686x_queue_setup,
+ .buf_queue = tw686x_buf_queue,
+ .buf_prepare = tw686x_buf_prepare,
+ .start_streaming = tw686x_start_streaming,
+ .stop_streaming = tw686x_stop_streaming,
+ .wait_prepare = vb2_ops_wait_prepare,
+ .wait_finish = vb2_ops_wait_finish,
+};
+
+static int tw686x_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct tw686x_video_channel *vc;
+ struct tw686x_dev *dev;
+ unsigned int ch;
+
+ vc = container_of(ctrl->handler, struct tw686x_video_channel,
+ ctrl_handler);
+ dev = vc->dev;
+ ch = vc->ch;
+
+ switch (ctrl->id) {
+ case V4L2_CID_BRIGHTNESS:
+ reg_write(dev, BRIGHT[ch], ctrl->val & 0xff);
+ return 0;
+
+ case V4L2_CID_CONTRAST:
+ reg_write(dev, CONTRAST[ch], ctrl->val);
+ return 0;
+
+ case V4L2_CID_SATURATION:
+ reg_write(dev, SAT_U[ch], ctrl->val);
+ reg_write(dev, SAT_V[ch], ctrl->val);
+ return 0;
+
+ case V4L2_CID_HUE:
+ reg_write(dev, HUE[ch], ctrl->val & 0xff);
+ return 0;
+ }
+
+ return -EINVAL;
+}
+
+static const struct v4l2_ctrl_ops ctrl_ops = {
+ .s_ctrl = tw686x_s_ctrl,
+};
+
+static int tw686x_g_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct tw686x_video_channel *vc = video_drvdata(file);
+
+ f->fmt.pix.width = vc->width;
+ f->fmt.pix.height = vc->height;
+ f->fmt.pix.field = V4L2_FIELD_INTERLACED;
+ f->fmt.pix.pixelformat = vc->format->fourcc;
+ f->fmt.pix.colorspace = V4L2_COLORSPACE_SMPTE170M;
+ f->fmt.pix.bytesperline = (f->fmt.pix.width * vc->format->depth) / 8;
+ f->fmt.pix.sizeimage = f->fmt.pix.height * f->fmt.pix.bytesperline;
+ return 0;
+}
+
+static int tw686x_try_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct tw686x_video_channel *vc = video_drvdata(file);
+ unsigned int video_height = TW686X_VIDEO_HEIGHT(vc->video_standard);
+ const struct tw686x_format *format;
+
+ format = format_by_fourcc(f->fmt.pix.pixelformat);
+ if (!format) {
+ format = &formats[0];
+ f->fmt.pix.pixelformat = format->fourcc;
+ }
+
+ if (f->fmt.pix.width <= TW686X_VIDEO_WIDTH / 2)
+ f->fmt.pix.width = TW686X_VIDEO_WIDTH / 2;
+ else
+ f->fmt.pix.width = TW686X_VIDEO_WIDTH;
+
+ if (f->fmt.pix.height <= video_height / 2)
+ f->fmt.pix.height = video_height / 2;
+ else
+ f->fmt.pix.height = video_height;
+
+ f->fmt.pix.bytesperline = (f->fmt.pix.width * format->depth) / 8;
+ f->fmt.pix.sizeimage = f->fmt.pix.height * f->fmt.pix.bytesperline;
+ f->fmt.pix.colorspace = V4L2_COLORSPACE_SMPTE170M;
+ f->fmt.pix.field = V4L2_FIELD_INTERLACED;
+
+ return 0;
+}
+
+static int tw686x_s_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct tw686x_video_channel *vc = video_drvdata(file);
+ u32 val, width, line_width, height;
+ unsigned long bitsperframe;
+ int err, pb;
+
+ if (vb2_is_busy(&vc->vidq))
+ return -EBUSY;
+
+ bitsperframe = vc->width * vc->height * vc->format->depth;
+ err = tw686x_try_fmt_vid_cap(file, priv, f);
+ if (err)
+ return err;
+
+ vc->format = format_by_fourcc(f->fmt.pix.pixelformat);
+ vc->width = f->fmt.pix.width;
+ vc->height = f->fmt.pix.height;
+
+ /* We need new DMA buffers if the framesize has changed */
+ if (bitsperframe != vc->width * vc->height * vc->format->depth) {
+ for (pb = 0; pb < 2; pb++)
+ tw686x_free_dma(vc, pb);
+
+ for (pb = 0; pb < 2; pb++) {
+ err = tw686x_alloc_dma(vc, pb);
+ if (err) {
+ if (pb > 0)
+ tw686x_free_dma(vc, 0);
+ return err;
+ }
+ }
+ }
+
+ val = reg_read(vc->dev, VDMA_CHANNEL_CONFIG[vc->ch]);
+
+ if (vc->width <= TW686X_VIDEO_WIDTH / 2)
+ val |= BIT(23);
+ else
+ val &= ~BIT(23);
+
+ if (vc->height <= TW686X_VIDEO_HEIGHT(vc->video_standard) / 2)
+ val |= BIT(24);
+ else
+ val &= ~BIT(24);
+
+ val &= ~(0x7 << 20);
+ val |= vc->format->mode << 20;
+ reg_write(vc->dev, VDMA_CHANNEL_CONFIG[vc->ch], val);
+
+ /* Program the DMA frame size */
+ width = (vc->width * 2) & 0x7ff;
+ height = vc->height / 2;
+ line_width = (vc->width * 2) & 0x7ff;
+ val = (height << 22) | (line_width << 11) | width;
+ reg_write(vc->dev, VDMA_WHP[vc->ch], val);
+ return 0;
+}
+
+static int tw686x_querycap(struct file *file, void *priv,
+ struct v4l2_capability *cap)
+{
+ struct tw686x_video_channel *vc = video_drvdata(file);
+ struct tw686x_dev *dev = vc->dev;
+
+ strlcpy(cap->driver, "tw686x", sizeof(cap->driver));
+ strlcpy(cap->card, dev->name, sizeof(cap->card));
+ snprintf(cap->bus_info, sizeof(cap->bus_info),
+ "PCI:%s", pci_name(dev->pci_dev));
+ cap->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING |
+ V4L2_CAP_READWRITE;
+ cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
+ return 0;
+}
+
+static int tw686x_s_std(struct file *file, void *priv, v4l2_std_id id)
+{
+ struct tw686x_video_channel *vc = video_drvdata(file);
+ struct v4l2_format f;
+ u32 val, ret;
+
+ if (vc->video_standard == id)
+ return 0;
+
+ if (vb2_is_busy(&vc->vidq))
+ return -EBUSY;
+
+ if (id & V4L2_STD_NTSC)
+ val = 0;
+ else if (id & V4L2_STD_PAL)
+ val = 1;
+ else if (id & V4L2_STD_SECAM)
+ val = 2;
+ else if (id & V4L2_STD_NTSC_443)
+ val = 3;
+ else if (id & V4L2_STD_PAL_M)
+ val = 4;
+ else if (id & V4L2_STD_PAL_Nc)
+ val = 5;
+ else if (id & V4L2_STD_PAL_60)
+ val = 6;
+ else
+ return -EINVAL;
+
+ vc->video_standard = id;
+ reg_write(vc->dev, SDT[vc->ch], val);
+
+ val = reg_read(vc->dev, VIDEO_CONTROL1);
+ if (id & V4L2_STD_525_60)
+ val &= ~(1 << (SYS_MODE_DMA_SHIFT + vc->ch));
+ else
+ val |= (1 << (SYS_MODE_DMA_SHIFT + vc->ch));
+ reg_write(vc->dev, VIDEO_CONTROL1, val);
+
+ /*
+ * Adjust format after V4L2_STD_525_60/V4L2_STD_625_50 change,
+ * calling g_fmt and s_fmt will sanitize the height
+ * according to the standard.
+ */
+ ret = tw686x_g_fmt_vid_cap(file, priv, &f);
+ if (!ret)
+ tw686x_s_fmt_vid_cap(file, priv, &f);
+ return 0;
+}
+
+static int tw686x_querystd(struct file *file, void *priv, v4l2_std_id *std)
+{
+ struct tw686x_video_channel *vc = video_drvdata(file);
+ struct tw686x_dev *dev = vc->dev;
+ unsigned int old_std, detected_std = 0;
+ unsigned long end;
+
+ if (vb2_is_streaming(&vc->vidq))
+ return -EBUSY;
+
+ /* Enable and start standard detection */
+ old_std = reg_read(dev, SDT[vc->ch]);
+ reg_write(dev, SDT[vc->ch], 0x7);
+ reg_write(dev, SDT_EN[vc->ch], 0xff);
+
+ end = jiffies + msecs_to_jiffies(500);
+ while (time_is_after_jiffies(end)) {
+
+ detected_std = reg_read(dev, SDT[vc->ch]);
+ if (!(detected_std & BIT(7)))
+ break;
+ msleep(100);
+ }
+ reg_write(dev, SDT[vc->ch], old_std);
+
+ /* Exit if still busy */
+ if (detected_std & BIT(7))
+ return 0;
+
+ detected_std = (detected_std >> 4) & 0x7;
+ switch (detected_std) {
+ case TW686X_STD_NTSC_M:
+ *std &= V4L2_STD_NTSC;
+ break;
+ case TW686X_STD_NTSC_443:
+ *std &= V4L2_STD_NTSC_443;
+ break;
+ case TW686X_STD_PAL_M:
+ *std &= V4L2_STD_PAL_M;
+ break;
+ case TW686X_STD_PAL_60:
+ *std &= V4L2_STD_PAL_60;
+ break;
+ case TW686X_STD_PAL:
+ *std &= V4L2_STD_PAL;
+ break;
+ case TW686X_STD_PAL_CN:
+ *std &= V4L2_STD_PAL_Nc;
+ break;
+ case TW686X_STD_SECAM:
+ *std &= V4L2_STD_SECAM;
+ break;
+ default:
+ *std = 0;
+ }
+ return 0;
+}
+
+static int tw686x_g_std(struct file *file, void *priv, v4l2_std_id *id)
+{
+ struct tw686x_video_channel *vc = video_drvdata(file);
+
+ *id = vc->video_standard;
+ return 0;
+}
+
+static int tw686x_enum_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_fmtdesc *f)
+{
+ if (f->index >= ARRAY_SIZE(formats))
+ return -EINVAL;
+ f->pixelformat = formats[f->index].fourcc;
+ return 0;
+}
+
+static int tw686x_s_input(struct file *file, void *priv, unsigned int i)
+{
+ struct tw686x_video_channel *vc = video_drvdata(file);
+ u32 val;
+
+ if (i >= TW686X_INPUTS_PER_CH)
+ return -EINVAL;
+ if (i == vc->input)
+ return 0;
+ /*
+ * Not sure we are able to support on the fly input change
+ */
+ if (vb2_is_busy(&vc->vidq))
+ return -EBUSY;
+
+ vc->input = i;
+
+ val = reg_read(vc->dev, VDMA_CHANNEL_CONFIG[vc->ch]);
+ val &= ~(0x3 << 30);
+ val |= i << 30;
+ reg_write(vc->dev, VDMA_CHANNEL_CONFIG[vc->ch], val);
+ return 0;
+}
+
+static int tw686x_g_input(struct file *file, void *priv, unsigned int *i)
+{
+ struct tw686x_video_channel *vc = video_drvdata(file);
+
+ *i = vc->input;
+ return 0;
+}
+
+static int tw686x_enum_input(struct file *file, void *priv,
+ struct v4l2_input *i)
+{
+ struct tw686x_video_channel *vc = video_drvdata(file);
+ unsigned int vidstat;
+
+ if (i->index >= TW686X_INPUTS_PER_CH)
+ return -EINVAL;
+
+ snprintf(i->name, sizeof(i->name), "Composite%d", i->index);
+ i->type = V4L2_INPUT_TYPE_CAMERA;
+ i->std = vc->device->tvnorms;
+ i->capabilities = V4L2_IN_CAP_STD;
+
+ vidstat = reg_read(vc->dev, VIDSTAT[vc->ch]);
+ i->status = 0;
+ if (vidstat & TW686X_VIDSTAT_VDLOSS)
+ i->status |= V4L2_IN_ST_NO_SIGNAL;
+ if (!(vidstat & TW686X_VIDSTAT_HLOCK))
+ i->status |= V4L2_IN_ST_NO_H_LOCK;
+
+ return 0;
+}
+
+static const struct v4l2_file_operations tw686x_video_fops = {
+ .owner = THIS_MODULE,
+ .open = v4l2_fh_open,
+ .unlocked_ioctl = video_ioctl2,
+ .release = vb2_fop_release,
+ .poll = vb2_fop_poll,
+ .read = vb2_fop_read,
+ .mmap = vb2_fop_mmap,
+};
+
+static const struct v4l2_ioctl_ops tw686x_video_ioctl_ops = {
+ .vidioc_querycap = tw686x_querycap,
+ .vidioc_g_fmt_vid_cap = tw686x_g_fmt_vid_cap,
+ .vidioc_s_fmt_vid_cap = tw686x_s_fmt_vid_cap,
+ .vidioc_enum_fmt_vid_cap = tw686x_enum_fmt_vid_cap,
+ .vidioc_try_fmt_vid_cap = tw686x_try_fmt_vid_cap,
+
+ .vidioc_querystd = tw686x_querystd,
+ .vidioc_g_std = tw686x_g_std,
+ .vidioc_s_std = tw686x_s_std,
+
+ .vidioc_enum_input = tw686x_enum_input,
+ .vidioc_g_input = tw686x_g_input,
+ .vidioc_s_input = tw686x_s_input,
+
+ .vidioc_reqbufs = vb2_ioctl_reqbufs,
+ .vidioc_querybuf = vb2_ioctl_querybuf,
+ .vidioc_qbuf = vb2_ioctl_qbuf,
+ .vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_create_bufs = vb2_ioctl_create_bufs,
+ .vidioc_streamon = vb2_ioctl_streamon,
+ .vidioc_streamoff = vb2_ioctl_streamoff,
+ .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+
+ .vidioc_log_status = v4l2_ctrl_log_status,
+ .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static void tw686x_buffer_copy(struct tw686x_video_channel *vc,
+ unsigned int pb, struct vb2_v4l2_buffer *vb)
+{
+ struct tw686x_dma_desc *desc = &vc->dma_descs[pb];
+ struct vb2_buffer *vb2_buf = &vb->vb2_buf;
+
+ vb->field = V4L2_FIELD_INTERLACED;
+ vb->sequence = vc->sequence++;
+
+ memcpy(vb2_plane_vaddr(vb2_buf, 0), desc->virt, desc->size);
+ vb2_buf->timestamp = ktime_get_ns();
+ vb2_buffer_done(vb2_buf, VB2_BUF_STATE_DONE);
+}
+
+void tw686x_video_irq(struct tw686x_dev *dev, unsigned long requests,
+ unsigned int pb_status, unsigned int fifo_status,
+ unsigned int *reset_ch)
+{
+ struct tw686x_video_channel *vc;
+ struct vb2_v4l2_buffer *vb;
+ unsigned long flags;
+ unsigned int ch, pb;
+
+ for_each_set_bit(ch, &requests, max_channels(dev)) {
+ vc = &dev->video_channels[ch];
+
+ /*
+ * This can either be a blue frame (with signal-lost bit set)
+ * or a good frame (with signal-lost bit clear). If we have just
+ * got signal, then this channel needs resetting.
+ */
+ if (vc->no_signal && !(fifo_status & BIT(ch))) {
+ v4l2_printk(KERN_DEBUG, &dev->v4l2_dev,
+ "video%d: signal recovered\n", vc->num);
+ vc->no_signal = false;
+ *reset_ch |= BIT(ch);
+ vc->pb = 0;
+ continue;
+ }
+ vc->no_signal = !!(fifo_status & BIT(ch));
+
+ /* Check FIFO errors only if there's signal */
+ if (!vc->no_signal) {
+ u32 fifo_ov, fifo_bad;
+
+ fifo_ov = (fifo_status >> 24) & BIT(ch);
+ fifo_bad = (fifo_status >> 16) & BIT(ch);
+ if (fifo_ov || fifo_bad) {
+ /* Mark this channel for reset */
+ v4l2_printk(KERN_DEBUG, &dev->v4l2_dev,
+ "video%d: FIFO error\n", vc->num);
+ *reset_ch |= BIT(ch);
+ vc->pb = 0;
+ continue;
+ }
+ }
+
+ pb = !!(pb_status & BIT(ch));
+ if (vc->pb != pb) {
+ /* Mark this channel for reset */
+ v4l2_printk(KERN_DEBUG, &dev->v4l2_dev,
+ "video%d: unexpected p-b buffer!\n",
+ vc->num);
+ *reset_ch |= BIT(ch);
+ vc->pb = 0;
+ continue;
+ }
+
+ /* handle video stream */
+ spin_lock_irqsave(&vc->qlock, flags);
+ if (vc->curr_bufs[pb]) {
+ vb = &vc->curr_bufs[pb]->vb;
+ tw686x_buffer_copy(vc, pb, vb);
+ }
+ vc->pb = !pb;
+ tw686x_buffer_refill(vc, pb);
+ spin_unlock_irqrestore(&vc->qlock, flags);
+ }
+}
+
+void tw686x_video_free(struct tw686x_dev *dev)
+{
+ unsigned int ch, pb;
+
+ for (ch = 0; ch < max_channels(dev); ch++) {
+ struct tw686x_video_channel *vc = &dev->video_channels[ch];
+
+ if (vc->device)
+ video_unregister_device(vc->device);
+
+ for (pb = 0; pb < 2; pb++)
+ tw686x_free_dma(vc, pb);
+ }
+}
+
+int tw686x_video_init(struct tw686x_dev *dev)
+{
+ unsigned int ch, val, pb;
+ int err;
+
+ err = v4l2_device_register(&dev->pci_dev->dev, &dev->v4l2_dev);
+ if (err)
+ return err;
+
+ for (ch = 0; ch < max_channels(dev); ch++) {
+ struct tw686x_video_channel *vc = &dev->video_channels[ch];
+ struct video_device *vdev;
+
+ mutex_init(&vc->vb_mutex);
+ spin_lock_init(&vc->qlock);
+ INIT_LIST_HEAD(&vc->vidq_queued);
+
+ vc->dev = dev;
+ vc->ch = ch;
+
+ /* default settings */
+ vc->format = &formats[0];
+ vc->video_standard = V4L2_STD_NTSC;
+ vc->width = TW686X_VIDEO_WIDTH;
+ vc->height = TW686X_VIDEO_HEIGHT(vc->video_standard);
+ vc->input = 0;
+
+ reg_write(vc->dev, SDT[ch], 0);
+ tw686x_set_framerate(vc, 30);
+
+ reg_write(dev, VDELAY_LO[ch], 0x14);
+ reg_write(dev, HACTIVE_LO[ch], 0xd0);
+ reg_write(dev, VIDEO_SIZE[ch], 0);
+
+ for (pb = 0; pb < 2; pb++) {
+ err = tw686x_alloc_dma(vc, pb);
+ if (err)
+ goto error;
+ }
+
+ vc->vidq.io_modes = VB2_READ | VB2_MMAP | VB2_DMABUF;
+ vc->vidq.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+ vc->vidq.drv_priv = vc;
+ vc->vidq.buf_struct_size = sizeof(struct tw686x_v4l2_buf);
+ vc->vidq.ops = &tw686x_video_qops;
+ vc->vidq.mem_ops = &vb2_vmalloc_memops;
+ vc->vidq.timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+ vc->vidq.min_buffers_needed = 2;
+ vc->vidq.lock = &vc->vb_mutex;
+ vc->vidq.gfp_flags = GFP_DMA32;
+
+ err = vb2_queue_init(&vc->vidq);
+ if (err) {
+ v4l2_err(&dev->v4l2_dev,
+ "dma%d: cannot init vb2 queue\n", ch);
+ goto error;
+ }
+
+ err = v4l2_ctrl_handler_init(&vc->ctrl_handler, 4);
+ if (err) {
+ v4l2_err(&dev->v4l2_dev,
+ "dma%d: cannot init ctrl handler\n", ch);
+ goto error;
+ }
+ v4l2_ctrl_new_std(&vc->ctrl_handler, &ctrl_ops,
+ V4L2_CID_BRIGHTNESS, -128, 127, 1, 0);
+ v4l2_ctrl_new_std(&vc->ctrl_handler, &ctrl_ops,
+ V4L2_CID_CONTRAST, 0, 255, 1, 100);
+ v4l2_ctrl_new_std(&vc->ctrl_handler, &ctrl_ops,
+ V4L2_CID_SATURATION, 0, 255, 1, 128);
+ v4l2_ctrl_new_std(&vc->ctrl_handler, &ctrl_ops,
+ V4L2_CID_HUE, -128, 127, 1, 0);
+ err = vc->ctrl_handler.error;
+ if (err)
+ goto error;
+
+ err = v4l2_ctrl_handler_setup(&vc->ctrl_handler);
+ if (err)
+ goto error;
+
+ vdev = video_device_alloc();
+ if (!vdev) {
+ v4l2_err(&dev->v4l2_dev,
+ "dma%d: unable to allocate device\n", ch);
+ err = -ENOMEM;
+ goto error;
+ }
+
+ snprintf(vdev->name, sizeof(vdev->name), "%s video", dev->name);
+ vdev->fops = &tw686x_video_fops;
+ vdev->ioctl_ops = &tw686x_video_ioctl_ops;
+ vdev->release = video_device_release;
+ vdev->v4l2_dev = &dev->v4l2_dev;
+ vdev->queue = &vc->vidq;
+ vdev->tvnorms = V4L2_STD_525_60 | V4L2_STD_625_50;
+ vdev->minor = -1;
+ vdev->lock = &vc->vb_mutex;
+ vdev->ctrl_handler = &vc->ctrl_handler;
+ vc->device = vdev;
+ video_set_drvdata(vdev, vc);
+
+ err = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+ if (err < 0)
+ goto error;
+ vc->num = vdev->num;
+ }
+
+ /* Set DMA frame mode on all channels. Only supported mode for now. */
+ val = TW686X_DEF_PHASE_REF;
+ for (ch = 0; ch < max_channels(dev); ch++)
+ val |= TW686X_FRAME_MODE << (16 + ch * 2);
+ reg_write(dev, PHASE_REF, val);
+
+ reg_write(dev, MISC2[0], 0xe7);
+ reg_write(dev, VCTRL1[0], 0xcc);
+ reg_write(dev, LOOP[0], 0xa5);
+ if (max_channels(dev) > 4) {
+ reg_write(dev, VCTRL1[1], 0xcc);
+ reg_write(dev, LOOP[1], 0xa5);
+ reg_write(dev, MISC2[1], 0xe7);
+ }
+ return 0;
+
+error:
+ tw686x_video_free(dev);
+ return err;
+}
diff --git a/drivers/media/pci/tw686x/tw686x.h b/drivers/media/pci/tw686x/tw686x.h
new file mode 100644
index 0000000..44b5755
--- /dev/null
+++ b/drivers/media/pci/tw686x/tw686x.h
@@ -0,0 +1,158 @@
+/*
+ * Copyright (C) 2015 VanguardiaSur - www.vanguardiasur.com.ar
+ *
+ * Copyright (C) 2015 Industrial Research Institute for Automation
+ * and Measurements PIAP
+ * Written by Krzysztof Ha?asa
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of version 2 of the GNU General Public License
+ * as published by the Free Software Foundation.
+ */
+
+#include <linux/mutex.h>
+#include <linux/pci.h>
+#include <linux/timer.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-common.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-v4l2.h>
+#include <sound/pcm.h>
+
+#include "tw686x-regs.h"
+
+#define TYPE_MAX_CHANNELS 0x0f
+#define TYPE_SECOND_GEN 0x10
+#define TW686X_DEF_PHASE_REF 0x1518
+
+#define TW686X_FIELD_MODE 0x3
+#define TW686X_FRAME_MODE 0x2
+/* 0x1 is reserved */
+#define TW686X_SG_MODE 0x0
+
+#define TW686X_AUDIO_PAGE_SZ 4096
+#define TW686X_AUDIO_PAGE_MAX 16
+#define TW686X_AUDIO_PERIODS_MIN 2
+#define TW686X_AUDIO_PERIODS_MAX TW686X_AUDIO_PAGE_MAX
+
+struct tw686x_format {
+ char *name;
+ unsigned int fourcc;
+ unsigned int depth;
+ unsigned int mode;
+};
+
+struct tw686x_dma_desc {
+ dma_addr_t phys;
+ void *virt;
+ unsigned int size;
+};
+
+struct tw686x_audio_buf {
+ dma_addr_t dma;
+ void *virt;
+ struct list_head list;
+};
+
+struct tw686x_v4l2_buf {
+ struct vb2_v4l2_buffer vb;
+ struct list_head list;
+};
+
+struct tw686x_audio_channel {
+ struct tw686x_dev *dev;
+ struct snd_pcm_substream *ss;
+ unsigned int ch;
+ struct tw686x_audio_buf *curr_bufs[2];
+ struct tw686x_dma_desc dma_descs[2];
+ dma_addr_t ptr;
+
+ struct tw686x_audio_buf buf[TW686X_AUDIO_PAGE_MAX];
+ struct list_head buf_list;
+ spinlock_t lock;
+};
+
+struct tw686x_video_channel {
+ struct tw686x_dev *dev;
+
+ struct vb2_queue vidq;
+ struct list_head vidq_queued;
+ struct video_device *device;
+ struct tw686x_v4l2_buf *curr_bufs[2];
+ struct tw686x_dma_desc dma_descs[2];
+
+ struct v4l2_ctrl_handler ctrl_handler;
+ const struct tw686x_format *format;
+ struct mutex vb_mutex;
+ spinlock_t qlock;
+ v4l2_std_id video_standard;
+ unsigned int width, height;
+ unsigned int h_halve, v_halve;
+ unsigned int ch;
+ unsigned int num;
+ unsigned int fps;
+ unsigned int input;
+ unsigned int sequence;
+ unsigned int pb;
+ bool no_signal;
+};
+
+/**
+ * struct tw686x_dev - global device status
+ * @lock: spinlock controlling access to the
+ * shared device registers (DMA enable/disable).
+ */
+struct tw686x_dev {
+ spinlock_t lock;
+
+ struct v4l2_device v4l2_dev;
+ struct snd_card *snd_card;
+
+ char name[32];
+ unsigned int type;
+ struct pci_dev *pci_dev;
+ __u32 __iomem *mmio;
+
+ void *alloc_ctx;
+
+ struct tw686x_video_channel *video_channels;
+ struct tw686x_audio_channel *audio_channels;
+
+ int audio_rate; /* per-device value */
+
+ struct timer_list dma_delay_timer;
+ u32 pending_dma_en; /* must be protected by lock */
+ u32 pending_dma_cmd; /* must be protected by lock */
+};
+
+static inline uint32_t reg_read(struct tw686x_dev *dev, unsigned int reg)
+{
+ return readl(dev->mmio + reg);
+}
+
+static inline void reg_write(struct tw686x_dev *dev, unsigned int reg,
+ uint32_t value)
+{
+ writel(value, dev->mmio + reg);
+}
+
+static inline unsigned int max_channels(struct tw686x_dev *dev)
+{
+ return dev->type & TYPE_MAX_CHANNELS; /* 4 or 8 channels */
+}
+
+void tw686x_enable_channel(struct tw686x_dev *dev, unsigned int channel);
+void tw686x_disable_channel(struct tw686x_dev *dev, unsigned int channel);
+
+int tw686x_video_init(struct tw686x_dev *dev);
+void tw686x_video_free(struct tw686x_dev *dev);
+void tw686x_video_irq(struct tw686x_dev *dev, unsigned long requests,
+ unsigned int pb_status, unsigned int fifo_status,
+ unsigned int *reset_ch);
+
+int tw686x_audio_init(struct tw686x_dev *dev);
+void tw686x_audio_free(struct tw686x_dev *dev);
+void tw686x_audio_irq(struct tw686x_dev *dev, unsigned long requests,
+ unsigned int pb_status);
diff --git a/drivers/media/pci/zoran/videocodec.c b/drivers/media/pci/zoran/videocodec.c
index c010716..13a3c07 100644
--- a/drivers/media/pci/zoran/videocodec.c
+++ b/drivers/media/pci/zoran/videocodec.c
@@ -116,8 +116,9 @@
goto out_module_put;
}
- snprintf(codec->name, sizeof(codec->name),
- "%s[%d]", codec->name, h->attached);
+ res = strlen(codec->name);
+ snprintf(codec->name + res, sizeof(codec->name) - res,
+ "[%d]", h->attached);
codec->master_data = master;
res = codec->setup(codec);
if (res == 0) {
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 201f5c2..84e041c 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -238,7 +238,7 @@
config VIDEO_RENESAS_JPU
tristate "Renesas JPEG Processing Unit"
depends on VIDEO_DEV && VIDEO_V4L2 && HAS_DMA
- depends on ARCH_SHMOBILE || COMPILE_TEST
+ depends on ARCH_RENESAS || COMPILE_TEST
select VIDEOBUF2_DMA_CONTIG
select V4L2_MEM2MEM_DEV
---help---
@@ -250,7 +250,7 @@
config VIDEO_RENESAS_VSP1
tristate "Renesas VSP1 Video Processing Engine"
depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API && HAS_DMA
- depends on (ARCH_SHMOBILE && OF) || COMPILE_TEST
+ depends on (ARCH_RENESAS && OF) || COMPILE_TEST
select VIDEOBUF2_DMA_CONTIG
---help---
This is a V4L2 driver for the Renesas VSP1 video processing engine.
diff --git a/drivers/media/platform/am437x/am437x-vpfe.c b/drivers/media/platform/am437x/am437x-vpfe.c
index de32e3a..e749eb7 100644
--- a/drivers/media/platform/am437x/am437x-vpfe.c
+++ b/drivers/media/platform/am437x/am437x-vpfe.c
@@ -1047,7 +1047,7 @@
static int vpfe_config_ccdc_image_format(struct vpfe_device *vpfe)
{
enum ccdc_frmfmt frm_fmt = CCDC_FRMFMT_INTERLACED;
- int ret;
+ int ret = 0;
vpfe_dbg(2, vpfe, "vpfe_config_ccdc_image_format\n");
@@ -1706,7 +1706,7 @@
sdinfo = &cfg->sub_devs[i];
client = v4l2_get_subdevdata(sdinfo->sd);
if (client->addr == curr_client->addr &&
- client->adapter->nr == client->adapter->nr) {
+ client->adapter->nr == curr_client->adapter->nr) {
if (vpfe->current_input >= 1)
return -1;
*app_input_index = j + vpfe->current_input;
diff --git a/drivers/media/platform/exynos-gsc/gsc-core.c b/drivers/media/platform/exynos-gsc/gsc-core.c
index 9b9e423..c049736 100644
--- a/drivers/media/platform/exynos-gsc/gsc-core.c
+++ b/drivers/media/platform/exynos-gsc/gsc-core.c
@@ -967,15 +967,6 @@
.lclk_frequency = 266000000UL,
};
-static const struct platform_device_id gsc_driver_ids[] = {
- {
- .name = "exynos-gsc",
- .driver_data = (unsigned long)&gsc_v_100_drvdata,
- },
- {},
-};
-MODULE_DEVICE_TABLE(platform, gsc_driver_ids);
-
static const struct of_device_id exynos_gsc_match[] = {
{
.compatible = "samsung,exynos5-gsc",
@@ -988,17 +979,11 @@
static void *gsc_get_drv_data(struct platform_device *pdev)
{
struct gsc_driverdata *driver_data = NULL;
+ const struct of_device_id *match;
- if (pdev->dev.of_node) {
- const struct of_device_id *match;
- match = of_match_node(exynos_gsc_match,
- pdev->dev.of_node);
- if (match)
- driver_data = (struct gsc_driverdata *)match->data;
- } else {
- driver_data = (struct gsc_driverdata *)
- platform_get_device_id(pdev)->driver_data;
- }
+ match = of_match_node(exynos_gsc_match, pdev->dev.of_node);
+ if (match)
+ driver_data = (struct gsc_driverdata *)match->data;
return driver_data;
}
@@ -1078,17 +1063,17 @@
struct resource *res;
struct gsc_driverdata *drv_data = gsc_get_drv_data(pdev);
struct device *dev = &pdev->dev;
- int ret = 0;
+ int ret;
gsc = devm_kzalloc(dev, sizeof(struct gsc_dev), GFP_KERNEL);
if (!gsc)
return -ENOMEM;
- if (dev->of_node)
- gsc->id = of_alias_get_id(pdev->dev.of_node, "gsc");
- else
- gsc->id = pdev->id;
+ ret = of_alias_get_id(pdev->dev.of_node, "gsc");
+ if (ret < 0)
+ return ret;
+ gsc->id = ret;
if (gsc->id >= drv_data->num_entities) {
dev_err(dev, "Invalid platform device id: %d\n", gsc->id);
return -EINVAL;
@@ -1096,7 +1081,6 @@
gsc->variant = drv_data->variant[gsc->id];
gsc->pdev = pdev;
- gsc->pdata = dev->platform_data;
init_waitqueue_head(&gsc->irq_queue);
spin_lock_init(&gsc->slock);
@@ -1253,7 +1237,6 @@
static struct platform_driver gsc_driver = {
.probe = gsc_probe,
.remove = gsc_remove,
- .id_table = gsc_driver_ids,
.driver = {
.name = GSC_MODULE_NAME,
.pm = &gsc_pm_ops,
diff --git a/drivers/media/platform/exynos-gsc/gsc-core.h b/drivers/media/platform/exynos-gsc/gsc-core.h
index e93a233..ec4000c 100644
--- a/drivers/media/platform/exynos-gsc/gsc-core.h
+++ b/drivers/media/platform/exynos-gsc/gsc-core.h
@@ -340,7 +340,6 @@
void __iomem *regs;
wait_queue_head_t irq_queue;
struct gsc_m2m_device m2m;
- struct exynos_platform_gscaler *pdata;
unsigned long state;
struct vb2_alloc_ctx *alloc_ctx;
struct video_device vdev;
diff --git a/drivers/media/platform/exynos4-is/fimc-core.c b/drivers/media/platform/exynos4-is/fimc-core.c
index cef2a7f..b1c1cea 100644
--- a/drivers/media/platform/exynos4-is/fimc-core.c
+++ b/drivers/media/platform/exynos4-is/fimc-core.c
@@ -1154,26 +1154,6 @@
},
};
-static const struct fimc_variant fimc0_variant_s5p = {
- .has_inp_rot = 1,
- .has_out_rot = 1,
- .has_cam_if = 1,
- .min_inp_pixsize = 16,
- .min_out_pixsize = 16,
- .hor_offs_align = 8,
- .min_vsize_align = 16,
- .pix_limit = &s5p_pix_limit[0],
-};
-
-static const struct fimc_variant fimc2_variant_s5p = {
- .has_cam_if = 1,
- .min_inp_pixsize = 16,
- .min_out_pixsize = 16,
- .hor_offs_align = 8,
- .min_vsize_align = 16,
- .pix_limit = &s5p_pix_limit[1],
-};
-
static const struct fimc_variant fimc0_variant_s5pv210 = {
.has_inp_rot = 1,
.has_out_rot = 1,
@@ -1206,18 +1186,6 @@
.pix_limit = &s5p_pix_limit[2],
};
-/* S5PC100 */
-static const struct fimc_drvdata fimc_drvdata_s5p = {
- .variant = {
- [0] = &fimc0_variant_s5p,
- [1] = &fimc0_variant_s5p,
- [2] = &fimc2_variant_s5p,
- },
- .num_entities = 3,
- .lclk_frequency = 133000000UL,
- .out_buf_count = 4,
-};
-
/* S5PV210, S5PC110 */
static const struct fimc_drvdata fimc_drvdata_s5pv210 = {
.variant = {
@@ -1251,23 +1219,6 @@
.out_buf_count = 32,
};
-static const struct platform_device_id fimc_driver_ids[] = {
- {
- .name = "s5p-fimc",
- .driver_data = (unsigned long)&fimc_drvdata_s5p,
- }, {
- .name = "s5pv210-fimc",
- .driver_data = (unsigned long)&fimc_drvdata_s5pv210,
- }, {
- .name = "exynos4-fimc",
- .driver_data = (unsigned long)&fimc_drvdata_exynos4210,
- }, {
- .name = "exynos4x12-fimc",
- .driver_data = (unsigned long)&fimc_drvdata_exynos4x12,
- },
- { },
-};
-
static const struct of_device_id fimc_of_match[] = {
{
.compatible = "samsung,s5pv210-fimc",
@@ -1290,7 +1241,6 @@
static struct platform_driver fimc_driver = {
.probe = fimc_probe,
.remove = fimc_remove,
- .id_table = fimc_driver_ids,
.driver = {
.of_match_table = fimc_of_match,
.name = FIMC_DRIVER_NAME,
diff --git a/drivers/media/platform/exynos4-is/media-dev.c b/drivers/media/platform/exynos4-is/media-dev.c
index 4f494ac..891625e 100644
--- a/drivers/media/platform/exynos4-is/media-dev.c
+++ b/drivers/media/platform/exynos4-is/media-dev.c
@@ -446,8 +446,10 @@
else
pd->fimc_bus_type = pd->sensor_bus_type;
- if (WARN_ON(index >= ARRAY_SIZE(fmd->sensor)))
+ if (WARN_ON(index >= ARRAY_SIZE(fmd->sensor))) {
+ of_node_put(rem);
return -EINVAL;
+ }
fmd->sensor[index].asd.match_type = V4L2_ASYNC_MATCH_OF;
fmd->sensor[index].asd.match.of.node = rem;
@@ -1130,7 +1132,7 @@
media_entity_graph_walk_start(graph, entity);
while ((entity = media_entity_graph_walk_next(graph))) {
- if (!is_media_entity_v4l2_io(entity))
+ if (!is_media_entity_v4l2_video_device(entity))
continue;
ret = __fimc_md_modify_pipeline(entity, enable);
@@ -1145,7 +1147,7 @@
media_entity_graph_walk_start(graph, entity_err);
while ((entity_err = media_entity_graph_walk_next(graph))) {
- if (!is_media_entity_v4l2_io(entity_err))
+ if (!is_media_entity_v4l2_video_device(entity_err))
continue;
__fimc_md_modify_pipeline(entity_err, !enable);
diff --git a/drivers/media/platform/exynos4-is/mipi-csis.c b/drivers/media/platform/exynos4-is/mipi-csis.c
index bd5c46c..bf95442 100644
--- a/drivers/media/platform/exynos4-is/mipi-csis.c
+++ b/drivers/media/platform/exynos4-is/mipi-csis.c
@@ -757,8 +757,10 @@
goto err;
state->index = endpoint.base.port - FIMC_INPUT_MIPI_CSI2_0;
- if (state->index >= CSIS_MAX_ENTITIES)
- return -ENXIO;
+ if (state->index >= CSIS_MAX_ENTITIES) {
+ ret = -ENXIO;
+ goto err;
+ }
/* Get MIPI CSI-2 bus configration from the endpoint node. */
of_property_read_u32(node, "samsung,csis-hs-settle",
diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
index ac76d29..1b1a95d 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -251,7 +251,7 @@
if (entity == &video->video.entity)
continue;
- if (!is_media_entity_v4l2_io(entity))
+ if (!is_media_entity_v4l2_video_device(entity))
continue;
__video = to_isp_video(media_entity_to_video_device(entity));
diff --git a/drivers/media/platform/s5p-g2d/g2d.c b/drivers/media/platform/s5p-g2d/g2d.c
index 74bd46c..612d1ea 100644
--- a/drivers/media/platform/s5p-g2d/g2d.c
+++ b/drivers/media/platform/s5p-g2d/g2d.c
@@ -719,16 +719,12 @@
def_frame.stride = (def_frame.width * def_frame.fmt->depth) >> 3;
- if (!pdev->dev.of_node) {
- dev->variant = g2d_get_drv_data(pdev);
- } else {
- of_id = of_match_node(exynos_g2d_match, pdev->dev.of_node);
- if (!of_id) {
- ret = -ENODEV;
- goto unreg_video_dev;
- }
- dev->variant = (struct g2d_variant *)of_id->data;
+ of_id = of_match_node(exynos_g2d_match, pdev->dev.of_node);
+ if (!of_id) {
+ ret = -ENODEV;
+ goto unreg_video_dev;
}
+ dev->variant = (struct g2d_variant *)of_id->data;
return 0;
@@ -788,22 +784,9 @@
};
MODULE_DEVICE_TABLE(of, exynos_g2d_match);
-static const struct platform_device_id g2d_driver_ids[] = {
- {
- .name = "s5p-g2d",
- .driver_data = (unsigned long)&g2d_drvdata_v3x,
- }, {
- .name = "s5p-g2d-v4x",
- .driver_data = (unsigned long)&g2d_drvdata_v4x,
- },
- {},
-};
-MODULE_DEVICE_TABLE(platform, g2d_driver_ids);
-
static struct platform_driver g2d_pdrv = {
.probe = g2d_probe,
.remove = g2d_remove,
- .id_table = g2d_driver_ids,
.driver = {
.name = G2D_NAME,
.of_match_table = exynos_g2d_match,
diff --git a/drivers/media/platform/s5p-g2d/g2d.h b/drivers/media/platform/s5p-g2d/g2d.h
index b0e52ab..e31df54 100644
--- a/drivers/media/platform/s5p-g2d/g2d.h
+++ b/drivers/media/platform/s5p-g2d/g2d.h
@@ -89,8 +89,3 @@
void g2d_set_v41_stretch(struct g2d_dev *d,
struct g2d_frame *src, struct g2d_frame *dst);
void g2d_set_cmd(struct g2d_dev *d, u32 c);
-
-static inline struct g2d_variant *g2d_get_drv_data(struct platform_device *pdev)
-{
- return (struct g2d_variant *)platform_get_device_id(pdev)->driver_data;
-}
diff --git a/drivers/media/platform/s5p-jpeg/jpeg-core.c b/drivers/media/platform/s5p-jpeg/jpeg-core.c
index c3b13a63..caa19b4 100644
--- a/drivers/media/platform/s5p-jpeg/jpeg-core.c
+++ b/drivers/media/platform/s5p-jpeg/jpeg-core.c
@@ -1548,8 +1548,10 @@
struct v4l2_pix_format *pix = &f->fmt.pix;
u32 pix_fmt = f->fmt.pix.pixelformat;
int w = pix->width, h = pix->height, wh_align;
+ int padding = 0;
if (pix_fmt == V4L2_PIX_FMT_RGB32 ||
+ pix_fmt == V4L2_PIX_FMT_RGB565 ||
pix_fmt == V4L2_PIX_FMT_NV24 ||
pix_fmt == V4L2_PIX_FMT_NV42 ||
pix_fmt == V4L2_PIX_FMT_NV12 ||
@@ -1564,7 +1566,10 @@
&h, S5P_JPEG_MIN_HEIGHT,
S5P_JPEG_MAX_HEIGHT, wh_align);
- return w * h * fmt_depth >> 3;
+ if (ctx->jpeg->variant->version == SJPEG_EXYNOS4)
+ padding = PAGE_SIZE;
+
+ return (w * h * fmt_depth >> 3) + padding;
}
static int exynos3250_jpeg_try_downscale(struct s5p_jpeg_ctx *ctx,
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc.c b/drivers/media/platform/s5p-mfc/s5p_mfc.c
index 927ab49..b16466f 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc.c
@@ -1489,27 +1489,6 @@
.fw_name[0] = "s5p-mfc-v8.fw",
};
-static const struct platform_device_id mfc_driver_ids[] = {
- {
- .name = "s5p-mfc",
- .driver_data = (unsigned long)&mfc_drvdata_v5,
- }, {
- .name = "s5p-mfc-v5",
- .driver_data = (unsigned long)&mfc_drvdata_v5,
- }, {
- .name = "s5p-mfc-v6",
- .driver_data = (unsigned long)&mfc_drvdata_v6,
- }, {
- .name = "s5p-mfc-v7",
- .driver_data = (unsigned long)&mfc_drvdata_v7,
- }, {
- .name = "s5p-mfc-v8",
- .driver_data = (unsigned long)&mfc_drvdata_v8,
- },
- {},
-};
-MODULE_DEVICE_TABLE(platform, mfc_driver_ids);
-
static const struct of_device_id exynos_mfc_match[] = {
{
.compatible = "samsung,mfc-v5",
@@ -1531,24 +1510,18 @@
static void *mfc_get_drv_data(struct platform_device *pdev)
{
struct s5p_mfc_variant *driver_data = NULL;
+ const struct of_device_id *match;
- if (pdev->dev.of_node) {
- const struct of_device_id *match;
- match = of_match_node(exynos_mfc_match,
- pdev->dev.of_node);
- if (match)
- driver_data = (struct s5p_mfc_variant *)match->data;
- } else {
- driver_data = (struct s5p_mfc_variant *)
- platform_get_device_id(pdev)->driver_data;
- }
+ match = of_match_node(exynos_mfc_match, pdev->dev.of_node);
+ if (match)
+ driver_data = (struct s5p_mfc_variant *)match->data;
+
return driver_data;
}
static struct platform_driver s5p_mfc_driver = {
.probe = s5p_mfc_probe,
.remove = s5p_mfc_remove,
- .id_table = mfc_driver_ids,
.driver = {
.name = S5P_MFC_NAME,
.pm = &s5p_mfc_pm_ops,
diff --git a/drivers/media/platform/s5p-tv/mixer.h b/drivers/media/platform/s5p-tv/mixer.h
index 42cd270..4dd62a9 100644
--- a/drivers/media/platform/s5p-tv/mixer.h
+++ b/drivers/media/platform/s5p-tv/mixer.h
@@ -300,7 +300,7 @@
struct mxr_layer *mxr_graph_layer_create(struct mxr_device *mdev, int idx);
struct mxr_layer *mxr_vp_layer_create(struct mxr_device *mdev, int idx);
struct mxr_layer *mxr_base_layer_create(struct mxr_device *mdev,
- int idx, char *name, struct mxr_layer_ops *ops);
+ int idx, char *name, const struct mxr_layer_ops *ops);
void mxr_base_layer_release(struct mxr_layer *layer);
void mxr_layer_release(struct mxr_layer *layer);
diff --git a/drivers/media/platform/s5p-tv/mixer_grp_layer.c b/drivers/media/platform/s5p-tv/mixer_grp_layer.c
index db3163b..d4d2564 100644
--- a/drivers/media/platform/s5p-tv/mixer_grp_layer.c
+++ b/drivers/media/platform/s5p-tv/mixer_grp_layer.c
@@ -235,7 +235,7 @@
{
struct mxr_layer *layer;
int ret;
- struct mxr_layer_ops ops = {
+ const struct mxr_layer_ops ops = {
.release = mxr_graph_layer_release,
.buffer_set = mxr_graph_buffer_set,
.stream_set = mxr_graph_stream_set,
diff --git a/drivers/media/platform/s5p-tv/mixer_video.c b/drivers/media/platform/s5p-tv/mixer_video.c
index d9e7f03..7ab5578 100644
--- a/drivers/media/platform/s5p-tv/mixer_video.c
+++ b/drivers/media/platform/s5p-tv/mixer_video.c
@@ -1070,7 +1070,7 @@
}
struct mxr_layer *mxr_base_layer_create(struct mxr_device *mdev,
- int idx, char *name, struct mxr_layer_ops *ops)
+ int idx, char *name, const struct mxr_layer_ops *ops)
{
struct mxr_layer *layer;
diff --git a/drivers/media/platform/s5p-tv/mixer_vp_layer.c b/drivers/media/platform/s5p-tv/mixer_vp_layer.c
index dd002a4..6fa6f67 100644
--- a/drivers/media/platform/s5p-tv/mixer_vp_layer.c
+++ b/drivers/media/platform/s5p-tv/mixer_vp_layer.c
@@ -207,7 +207,7 @@
{
struct mxr_layer *layer;
int ret;
- struct mxr_layer_ops ops = {
+ const struct mxr_layer_ops ops = {
.release = mxr_vp_layer_release,
.buffer_set = mxr_vp_buffer_set,
.stream_set = mxr_vp_stream_set,
diff --git a/drivers/media/platform/soc_camera/Kconfig b/drivers/media/platform/soc_camera/Kconfig
index 3552989..83029a4 100644
--- a/drivers/media/platform/soc_camera/Kconfig
+++ b/drivers/media/platform/soc_camera/Kconfig
@@ -28,7 +28,7 @@
config VIDEO_RCAR_VIN
tristate "R-Car Video Input (VIN) support"
depends on VIDEO_DEV && SOC_CAMERA
- depends on ARCH_SHMOBILE || COMPILE_TEST
+ depends on ARCH_RENESAS || COMPILE_TEST
depends on HAS_DMA
select VIDEOBUF2_DMA_CONTIG
select SOC_CAMERA_SCALE_CROP
@@ -45,7 +45,7 @@
config VIDEO_SH_MOBILE_CEU
tristate "SuperH Mobile CEU Interface driver"
depends on VIDEO_DEV && SOC_CAMERA && HAS_DMA && HAVE_CLK
- depends on ARCH_SHMOBILE || SUPERH || COMPILE_TEST
+ depends on ARCH_SHMOBILE || COMPILE_TEST
depends on HAS_DMA
select VIDEOBUF2_DMA_CONTIG
select SOC_CAMERA_SCALE_CROP
diff --git a/drivers/media/platform/soc_camera/rcar_vin.c b/drivers/media/platform/soc_camera/rcar_vin.c
index 3b8edf4..3f9c1b8 100644
--- a/drivers/media/platform/soc_camera/rcar_vin.c
+++ b/drivers/media/platform/soc_camera/rcar_vin.c
@@ -1845,6 +1845,8 @@
{ .compatible = "renesas,vin-r8a7790", .data = (void *)RCAR_GEN2 },
{ .compatible = "renesas,vin-r8a7779", .data = (void *)RCAR_H1 },
{ .compatible = "renesas,vin-r8a7778", .data = (void *)RCAR_M1 },
+ { .compatible = "renesas,rcar-gen3-vin", .data = (void *)RCAR_GEN3 },
+ { .compatible = "renesas,rcar-gen2-vin", .data = (void *)RCAR_GEN2 },
{ },
};
MODULE_DEVICE_TABLE(of, rcar_vin_of_table);
diff --git a/drivers/media/platform/sti/c8sectpfe/c8sectpfe-core.c b/drivers/media/platform/sti/c8sectpfe/c8sectpfe-core.c
index 78e3cb9..7dddf77 100644
--- a/drivers/media/platform/sti/c8sectpfe/c8sectpfe-core.c
+++ b/drivers/media/platform/sti/c8sectpfe/c8sectpfe-core.c
@@ -49,7 +49,7 @@
#define PID_TABLE_SIZE 1024
#define POLL_MSECS 50
-static int load_c8sectpfe_fw_step1(struct c8sectpfei *fei);
+static int load_c8sectpfe_fw(struct c8sectpfei *fei);
#define TS_PKT_SIZE 188
#define HEADER_SIZE (4)
@@ -130,7 +130,7 @@
writel(channel->back_buffer_busaddr, channel->irec +
DMA_PRDS_BUSRP_TP(0));
else
- writel(wp, channel->irec + DMA_PRDS_BUSWP_TP(0));
+ writel(wp, channel->irec + DMA_PRDS_BUSRP_TP(0));
}
static int c8sectpfe_start_feed(struct dvb_demux_feed *dvbdmxfeed)
@@ -141,6 +141,7 @@
struct channel_info *channel;
u32 tmp;
unsigned long *bitmap;
+ int ret;
switch (dvbdmxfeed->type) {
case DMX_TYPE_TS:
@@ -169,8 +170,9 @@
}
if (!atomic_read(&fei->fw_loaded)) {
- dev_err(fei->dev, "%s: c8sectpfe fw not loaded\n", __func__);
- return -EINVAL;
+ ret = load_c8sectpfe_fw(fei);
+ if (ret)
+ return ret;
}
mutex_lock(&fei->lock);
@@ -265,8 +267,9 @@
unsigned long *bitmap;
if (!atomic_read(&fei->fw_loaded)) {
- dev_err(fei->dev, "%s: c8sectpfe fw not loaded\n", __func__);
- return -EINVAL;
+ ret = load_c8sectpfe_fw(fei);
+ if (ret)
+ return ret;
}
mutex_lock(&fei->lock);
@@ -585,7 +588,7 @@
writel(tsin->pid_buffer_busaddr,
fei->io + PIDF_BASE(tsin->tsin_id));
- dev_info(fei->dev, "chan=%d PIDF_BASE=0x%x pid_bus_addr=%pad\n",
+ dev_dbg(fei->dev, "chan=%d PIDF_BASE=0x%x pid_bus_addr=%pad\n",
tsin->tsin_id, readl(fei->io + PIDF_BASE(tsin->tsin_id)),
&tsin->pid_buffer_busaddr);
@@ -880,13 +883,6 @@
goto err_clk_disable;
}
- /* ensure all other init has been done before requesting firmware */
- ret = load_c8sectpfe_fw_step1(fei);
- if (ret) {
- dev_err(dev, "Couldn't load slim core firmware\n");
- goto err_clk_disable;
- }
-
c8sectpfe_debugfs_init(fei);
return 0;
@@ -1091,15 +1087,14 @@
phdr->p_memsz - phdr->p_filesz);
}
-static int load_slim_core_fw(const struct firmware *fw, void *context)
+static int load_slim_core_fw(const struct firmware *fw, struct c8sectpfei *fei)
{
- struct c8sectpfei *fei = context;
Elf32_Ehdr *ehdr;
Elf32_Phdr *phdr;
u8 __iomem *dst;
int err = 0, i;
- if (!fw || !context)
+ if (!fw || !fei)
return -EINVAL;
ehdr = (Elf32_Ehdr *)fw->data;
@@ -1151,29 +1146,35 @@
return err;
}
-static void load_c8sectpfe_fw_cb(const struct firmware *fw, void *context)
+static int load_c8sectpfe_fw(struct c8sectpfei *fei)
{
- struct c8sectpfei *fei = context;
+ const struct firmware *fw;
int err;
+ dev_info(fei->dev, "Loading firmware: %s\n", FIRMWARE_MEMDMA);
+
+ err = request_firmware(&fw, FIRMWARE_MEMDMA, fei->dev);
+ if (err)
+ return err;
+
err = c8sectpfe_elf_sanity_check(fei, fw);
if (err) {
dev_err(fei->dev, "c8sectpfe_elf_sanity_check failed err=(%d)\n"
, err);
- goto err;
+ return err;
}
- err = load_slim_core_fw(fw, context);
+ err = load_slim_core_fw(fw, fei);
if (err) {
dev_err(fei->dev, "load_slim_core_fw failed err=(%d)\n", err);
- goto err;
+ return err;
}
/* now the firmware is loaded configure the input blocks */
err = configure_channels(fei);
if (err) {
dev_err(fei->dev, "configure_channels failed err=(%d)\n", err);
- goto err;
+ return err;
}
/*
@@ -1186,28 +1187,6 @@
writel(0x1, fei->io + DMA_CPU_RUN);
atomic_set(&fei->fw_loaded, 1);
-err:
- complete_all(&fei->fw_ack);
-}
-
-static int load_c8sectpfe_fw_step1(struct c8sectpfei *fei)
-{
- int err;
-
- dev_info(fei->dev, "Loading firmware: %s\n", FIRMWARE_MEMDMA);
-
- init_completion(&fei->fw_ack);
- atomic_set(&fei->fw_loaded, 0);
-
- err = request_firmware_nowait(THIS_MODULE, FW_ACTION_HOTPLUG,
- FIRMWARE_MEMDMA, fei->dev, GFP_KERNEL, fei,
- load_c8sectpfe_fw_cb);
-
- if (err) {
- dev_err(fei->dev, "request_firmware_nowait err: %d.\n", err);
- complete_all(&fei->fw_ack);
- return err;
- }
return 0;
}
diff --git a/drivers/media/platform/vivid/Kconfig b/drivers/media/platform/vivid/Kconfig
index 0885e93..f535f57 100644
--- a/drivers/media/platform/vivid/Kconfig
+++ b/drivers/media/platform/vivid/Kconfig
@@ -7,6 +7,7 @@
select FB_CFB_COPYAREA
select FB_CFB_IMAGEBLIT
select VIDEOBUF2_VMALLOC
+ select VIDEO_V4L2_TPG
default n
---help---
Enables a virtual video driver. This driver emulates a webcam,
diff --git a/drivers/media/platform/vivid/Makefile b/drivers/media/platform/vivid/Makefile
index 756fc12..633c8a1b 100644
--- a/drivers/media/platform/vivid/Makefile
+++ b/drivers/media/platform/vivid/Makefile
@@ -2,5 +2,5 @@
vivid-vid-cap.o vivid-vid-out.o vivid-kthread-cap.o vivid-kthread-out.o \
vivid-radio-rx.o vivid-radio-tx.o vivid-radio-common.o \
vivid-rds-gen.o vivid-sdr-cap.o vivid-vbi-cap.o vivid-vbi-out.o \
- vivid-osd.o vivid-tpg.o vivid-tpg-colors.o
+ vivid-osd.o
obj-$(CONFIG_VIDEO_VIVID) += vivid.o
diff --git a/drivers/media/platform/vivid/vivid-core.c b/drivers/media/platform/vivid/vivid-core.c
index ec125bec..c14da84 100644
--- a/drivers/media/platform/vivid/vivid-core.c
+++ b/drivers/media/platform/vivid/vivid-core.c
@@ -200,27 +200,12 @@
struct v4l2_capability *cap)
{
struct vivid_dev *dev = video_drvdata(file);
- struct video_device *vdev = video_devdata(file);
strcpy(cap->driver, "vivid");
strcpy(cap->card, "vivid");
snprintf(cap->bus_info, sizeof(cap->bus_info),
"platform:%s", dev->v4l2_dev.name);
- if (vdev->vfl_type == VFL_TYPE_GRABBER && vdev->vfl_dir == VFL_DIR_RX)
- cap->device_caps = dev->vid_cap_caps;
- if (vdev->vfl_type == VFL_TYPE_GRABBER && vdev->vfl_dir == VFL_DIR_TX)
- cap->device_caps = dev->vid_out_caps;
- else if (vdev->vfl_type == VFL_TYPE_VBI && vdev->vfl_dir == VFL_DIR_RX)
- cap->device_caps = dev->vbi_cap_caps;
- else if (vdev->vfl_type == VFL_TYPE_VBI && vdev->vfl_dir == VFL_DIR_TX)
- cap->device_caps = dev->vbi_out_caps;
- else if (vdev->vfl_type == VFL_TYPE_SDR)
- cap->device_caps = dev->sdr_cap_caps;
- else if (vdev->vfl_type == VFL_TYPE_RADIO && vdev->vfl_dir == VFL_DIR_RX)
- cap->device_caps = dev->radio_rx_caps;
- else if (vdev->vfl_type == VFL_TYPE_RADIO && vdev->vfl_dir == VFL_DIR_TX)
- cap->device_caps = dev->radio_tx_caps;
cap->capabilities = dev->vid_cap_caps | dev->vid_out_caps |
dev->vbi_cap_caps | dev->vbi_out_caps |
dev->radio_rx_caps | dev->radio_tx_caps |
@@ -1135,6 +1120,7 @@
strlcpy(vfd->name, "vivid-vid-cap", sizeof(vfd->name));
vfd->fops = &vivid_fops;
vfd->ioctl_ops = &vivid_ioctl_ops;
+ vfd->device_caps = dev->vid_cap_caps;
vfd->release = video_device_release_empty;
vfd->v4l2_dev = &dev->v4l2_dev;
vfd->queue = &dev->vb_vid_cap_q;
@@ -1160,6 +1146,7 @@
vfd->vfl_dir = VFL_DIR_TX;
vfd->fops = &vivid_fops;
vfd->ioctl_ops = &vivid_ioctl_ops;
+ vfd->device_caps = dev->vid_out_caps;
vfd->release = video_device_release_empty;
vfd->v4l2_dev = &dev->v4l2_dev;
vfd->queue = &dev->vb_vid_out_q;
@@ -1184,6 +1171,7 @@
strlcpy(vfd->name, "vivid-vbi-cap", sizeof(vfd->name));
vfd->fops = &vivid_fops;
vfd->ioctl_ops = &vivid_ioctl_ops;
+ vfd->device_caps = dev->vbi_cap_caps;
vfd->release = video_device_release_empty;
vfd->v4l2_dev = &dev->v4l2_dev;
vfd->queue = &dev->vb_vbi_cap_q;
@@ -1207,6 +1195,7 @@
vfd->vfl_dir = VFL_DIR_TX;
vfd->fops = &vivid_fops;
vfd->ioctl_ops = &vivid_ioctl_ops;
+ vfd->device_caps = dev->vbi_out_caps;
vfd->release = video_device_release_empty;
vfd->v4l2_dev = &dev->v4l2_dev;
vfd->queue = &dev->vb_vbi_out_q;
@@ -1229,6 +1218,7 @@
strlcpy(vfd->name, "vivid-sdr-cap", sizeof(vfd->name));
vfd->fops = &vivid_fops;
vfd->ioctl_ops = &vivid_ioctl_ops;
+ vfd->device_caps = dev->sdr_cap_caps;
vfd->release = video_device_release_empty;
vfd->v4l2_dev = &dev->v4l2_dev;
vfd->queue = &dev->vb_sdr_cap_q;
@@ -1247,6 +1237,7 @@
strlcpy(vfd->name, "vivid-rad-rx", sizeof(vfd->name));
vfd->fops = &vivid_radio_fops;
vfd->ioctl_ops = &vivid_ioctl_ops;
+ vfd->device_caps = dev->radio_rx_caps;
vfd->release = video_device_release_empty;
vfd->v4l2_dev = &dev->v4l2_dev;
vfd->lock = &dev->mutex;
@@ -1265,6 +1256,7 @@
vfd->vfl_dir = VFL_DIR_TX;
vfd->fops = &vivid_radio_fops;
vfd->ioctl_ops = &vivid_ioctl_ops;
+ vfd->device_caps = dev->radio_tx_caps;
vfd->release = video_device_release_empty;
vfd->v4l2_dev = &dev->v4l2_dev;
vfd->lock = &dev->mutex;
diff --git a/drivers/media/platform/vivid/vivid-core.h b/drivers/media/platform/vivid/vivid-core.h
index 751c1ba..776783b 100644
--- a/drivers/media/platform/vivid/vivid-core.h
+++ b/drivers/media/platform/vivid/vivid-core.h
@@ -25,7 +25,7 @@
#include <media/v4l2-device.h>
#include <media/v4l2-dev.h>
#include <media/v4l2-ctrls.h>
-#include "vivid-tpg.h"
+#include <media/v4l2-tpg.h>
#include "vivid-rds-gen.h"
#include "vivid-vbi-gen.h"
diff --git a/drivers/media/platform/vivid/vivid-kthread-cap.c b/drivers/media/platform/vivid/vivid-kthread-cap.c
index 9034281..3b8c101 100644
--- a/drivers/media/platform/vivid/vivid-kthread-cap.c
+++ b/drivers/media/platform/vivid/vivid-kthread-cap.c
@@ -36,6 +36,7 @@
#include <media/v4l2-ioctl.h>
#include <media/v4l2-fh.h>
#include <media/v4l2-event.h>
+#include <media/v4l2-rect.h>
#include "vivid-core.h"
#include "vivid-vid-common.h"
@@ -184,15 +185,15 @@
dev->compose_out.width, dev->compose_out.height
};
- dev->loop_vid_copy = rect_intersect(&dev->crop_cap, &dev->compose_out);
+ v4l2_rect_intersect(&dev->loop_vid_copy, &dev->crop_cap, &dev->compose_out);
dev->loop_vid_out = dev->loop_vid_copy;
- rect_scale(&dev->loop_vid_out, &dev->compose_out, &dev->crop_out);
+ v4l2_rect_scale(&dev->loop_vid_out, &dev->compose_out, &dev->crop_out);
dev->loop_vid_out.left += dev->crop_out.left;
dev->loop_vid_out.top += dev->crop_out.top;
dev->loop_vid_cap = dev->loop_vid_copy;
- rect_scale(&dev->loop_vid_cap, &dev->crop_cap, &dev->compose_cap);
+ v4l2_rect_scale(&dev->loop_vid_cap, &dev->crop_cap, &dev->compose_cap);
dprintk(dev, 1,
"loop_vid_copy: %dx%d@%dx%d loop_vid_out: %dx%d@%dx%d loop_vid_cap: %dx%d@%dx%d\n",
@@ -203,13 +204,13 @@
dev->loop_vid_cap.width, dev->loop_vid_cap.height,
dev->loop_vid_cap.left, dev->loop_vid_cap.top);
- r_overlay = rect_intersect(&r_fb, &r_overlay);
+ v4l2_rect_intersect(&r_overlay, &r_fb, &r_overlay);
/* shift r_overlay to the same origin as compose_out */
r_overlay.left += dev->compose_out.left - dev->overlay_out_left;
r_overlay.top += dev->compose_out.top - dev->overlay_out_top;
- dev->loop_vid_overlay = rect_intersect(&r_overlay, &dev->loop_vid_copy);
+ v4l2_rect_intersect(&dev->loop_vid_overlay, &r_overlay, &dev->loop_vid_copy);
dev->loop_fb_copy = dev->loop_vid_overlay;
/* shift dev->loop_fb_copy back again to the fb origin */
@@ -217,7 +218,7 @@
dev->loop_fb_copy.top -= dev->compose_out.top - dev->overlay_out_top;
dev->loop_vid_overlay_cap = dev->loop_vid_overlay;
- rect_scale(&dev->loop_vid_overlay_cap, &dev->crop_cap, &dev->compose_cap);
+ v4l2_rect_scale(&dev->loop_vid_overlay_cap, &dev->crop_cap, &dev->compose_cap);
dprintk(dev, 1,
"loop_fb_copy: %dx%d@%dx%d loop_vid_overlay: %dx%d@%dx%d loop_vid_overlay_cap: %dx%d@%dx%d\n",
diff --git a/drivers/media/platform/vivid/vivid-rds-gen.c b/drivers/media/platform/vivid/vivid-rds-gen.c
index c382343..53c7777 100644
--- a/drivers/media/platform/vivid/vivid-rds-gen.c
+++ b/drivers/media/platform/vivid/vivid-rds-gen.c
@@ -55,6 +55,7 @@
{
struct v4l2_rds_data *data = rds->data;
unsigned grp;
+ unsigned idx;
struct tm tm;
unsigned date;
unsigned time;
@@ -73,24 +74,26 @@
case 0 ... 3:
case 22 ... 25:
case 44 ... 47: /* Group 0B */
+ idx = (grp % 22) % 4;
data[1].lsb |= (rds->ta << 4) | (rds->ms << 3);
- data[1].lsb |= vivid_get_di(rds, grp % 22);
+ data[1].lsb |= vivid_get_di(rds, idx);
data[1].msb |= 1 << 3;
data[2].lsb = rds->picode & 0xff;
data[2].msb = rds->picode >> 8;
data[2].block = V4L2_RDS_BLOCK_C_ALT | (V4L2_RDS_BLOCK_C_ALT << 3);
- data[3].lsb = rds->psname[2 * (grp % 22) + 1];
- data[3].msb = rds->psname[2 * (grp % 22)];
+ data[3].lsb = rds->psname[2 * idx + 1];
+ data[3].msb = rds->psname[2 * idx];
break;
case 4 ... 19:
case 26 ... 41: /* Group 2A */
- data[1].lsb |= (grp - 4) % 22;
+ idx = ((grp - 4) % 22) % 16;
+ data[1].lsb |= idx;
data[1].msb |= 4 << 3;
- data[2].msb = rds->radiotext[4 * ((grp - 4) % 22)];
- data[2].lsb = rds->radiotext[4 * ((grp - 4) % 22) + 1];
+ data[2].msb = rds->radiotext[4 * idx];
+ data[2].lsb = rds->radiotext[4 * idx + 1];
data[2].block = V4L2_RDS_BLOCK_C | (V4L2_RDS_BLOCK_C << 3);
- data[3].msb = rds->radiotext[4 * ((grp - 4) % 22) + 2];
- data[3].lsb = rds->radiotext[4 * ((grp - 4) % 22) + 3];
+ data[3].msb = rds->radiotext[4 * idx + 2];
+ data[3].lsb = rds->radiotext[4 * idx + 3];
break;
case 56:
/*
diff --git a/drivers/media/platform/vivid/vivid-tpg-colors.c b/drivers/media/platform/vivid/vivid-tpg-colors.c
deleted file mode 100644
index 2299f0c..0000000
--- a/drivers/media/platform/vivid/vivid-tpg-colors.c
+++ /dev/null
@@ -1,1416 +0,0 @@
-/*
- * vivid-color.c - A table that converts colors to various colorspaces
- *
- * The test pattern generator uses the tpg_colors for its test patterns.
- * For testing colorspaces the first 8 colors of that table need to be
- * converted to their equivalent in the target colorspace.
- *
- * The tpg_csc_colors[] table is the result of that conversion and since
- * it is precalculated the colorspace conversion is just a simple table
- * lookup.
- *
- * This source also contains the code used to generate the tpg_csc_colors
- * table. Run the following command to compile it:
- *
- * gcc vivid-tpg-colors.c -DCOMPILE_APP -o gen-colors -lm
- *
- * and run the utility.
- *
- * Note that the converted colors are in the range 0x000-0xff0 (so times 16)
- * in order to preserve precision.
- *
- * Copyright 2014 Cisco Systems, Inc. and/or its affiliates. All rights reserved.
- *
- * This program is free software; you may redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; version 2 of the License.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
- * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
- * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
- * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
- * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
- * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
- * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
- * SOFTWARE.
- */
-
-#include <linux/videodev2.h>
-
-#include "vivid-tpg-colors.h"
-
-/* sRGB colors with range [0-255] */
-const struct color tpg_colors[TPG_COLOR_MAX] = {
- /*
- * Colors to test colorspace conversion: converting these colors
- * to other colorspaces will never lead to out-of-gamut colors.
- */
- { 191, 191, 191 }, /* TPG_COLOR_CSC_WHITE */
- { 191, 191, 50 }, /* TPG_COLOR_CSC_YELLOW */
- { 50, 191, 191 }, /* TPG_COLOR_CSC_CYAN */
- { 50, 191, 50 }, /* TPG_COLOR_CSC_GREEN */
- { 191, 50, 191 }, /* TPG_COLOR_CSC_MAGENTA */
- { 191, 50, 50 }, /* TPG_COLOR_CSC_RED */
- { 50, 50, 191 }, /* TPG_COLOR_CSC_BLUE */
- { 50, 50, 50 }, /* TPG_COLOR_CSC_BLACK */
-
- /* 75% colors */
- { 191, 191, 0 }, /* TPG_COLOR_75_YELLOW */
- { 0, 191, 191 }, /* TPG_COLOR_75_CYAN */
- { 0, 191, 0 }, /* TPG_COLOR_75_GREEN */
- { 191, 0, 191 }, /* TPG_COLOR_75_MAGENTA */
- { 191, 0, 0 }, /* TPG_COLOR_75_RED */
- { 0, 0, 191 }, /* TPG_COLOR_75_BLUE */
-
- /* 100% colors */
- { 255, 255, 255 }, /* TPG_COLOR_100_WHITE */
- { 255, 255, 0 }, /* TPG_COLOR_100_YELLOW */
- { 0, 255, 255 }, /* TPG_COLOR_100_CYAN */
- { 0, 255, 0 }, /* TPG_COLOR_100_GREEN */
- { 255, 0, 255 }, /* TPG_COLOR_100_MAGENTA */
- { 255, 0, 0 }, /* TPG_COLOR_100_RED */
- { 0, 0, 255 }, /* TPG_COLOR_100_BLUE */
- { 0, 0, 0 }, /* TPG_COLOR_100_BLACK */
-
- { 0, 0, 0 }, /* TPG_COLOR_RANDOM placeholder */
-};
-
-#ifndef COMPILE_APP
-
-/* Generated table */
-const unsigned short tpg_rec709_to_linear[255 * 16 + 1] = {
- 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 2, 3, 3, 3, 3,
- 4, 4, 4, 4, 4, 5, 5, 5, 5, 6, 6, 6, 6, 6, 7, 7,
- 7, 7, 8, 8, 8, 8, 8, 9, 9, 9, 9, 10, 10, 10, 10, 10,
- 11, 11, 11, 11, 12, 12, 12, 12, 12, 13, 13, 13, 13, 14, 14, 14,
- 14, 14, 15, 15, 15, 15, 16, 16, 16, 16, 16, 17, 17, 17, 17, 18,
- 18, 18, 18, 18, 19, 19, 19, 19, 20, 20, 20, 20, 20, 21, 21, 21,
- 21, 22, 22, 22, 22, 22, 23, 23, 23, 23, 24, 24, 24, 24, 24, 25,
- 25, 25, 25, 26, 26, 26, 26, 26, 27, 27, 27, 27, 28, 28, 28, 28,
- 28, 29, 29, 29, 29, 30, 30, 30, 30, 30, 31, 31, 31, 31, 32, 32,
- 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 34, 34, 35, 35, 35, 35,
- 36, 36, 36, 36, 36, 37, 37, 37, 37, 38, 38, 38, 38, 38, 39, 39,
- 39, 39, 40, 40, 40, 40, 40, 41, 41, 41, 41, 42, 42, 42, 42, 42,
- 43, 43, 43, 43, 44, 44, 44, 44, 44, 45, 45, 45, 45, 46, 46, 46,
- 46, 46, 47, 47, 47, 47, 48, 48, 48, 48, 48, 49, 49, 49, 49, 50,
- 50, 50, 50, 50, 51, 51, 51, 51, 52, 52, 52, 52, 52, 53, 53, 53,
- 53, 54, 54, 54, 54, 54, 55, 55, 55, 55, 56, 56, 56, 56, 56, 57,
- 57, 57, 57, 58, 58, 58, 58, 58, 59, 59, 59, 59, 60, 60, 60, 60,
- 60, 61, 61, 61, 61, 62, 62, 62, 62, 62, 63, 63, 63, 63, 64, 64,
- 64, 64, 64, 65, 65, 65, 65, 66, 66, 66, 66, 66, 67, 67, 67, 67,
- 68, 68, 68, 68, 68, 69, 69, 69, 69, 70, 70, 70, 70, 70, 71, 71,
- 71, 71, 72, 72, 72, 72, 72, 73, 73, 73, 73, 73, 74, 74, 74, 74,
- 74, 75, 75, 75, 75, 76, 76, 76, 76, 76, 77, 77, 77, 77, 78, 78,
- 78, 78, 79, 79, 79, 79, 79, 80, 80, 80, 80, 81, 81, 81, 81, 82,
- 82, 82, 82, 82, 83, 83, 83, 83, 84, 84, 84, 84, 85, 85, 85, 85,
- 86, 86, 86, 86, 87, 87, 87, 87, 88, 88, 88, 88, 89, 89, 89, 89,
- 90, 90, 90, 90, 91, 91, 91, 91, 92, 92, 92, 92, 93, 93, 93, 93,
- 94, 94, 94, 94, 95, 95, 95, 95, 96, 96, 96, 96, 97, 97, 97, 97,
- 98, 98, 98, 98, 99, 99, 99, 99, 100, 100, 100, 101, 101, 101, 101, 102,
- 102, 102, 102, 103, 103, 103, 103, 104, 104, 104, 105, 105, 105, 105, 106, 106,
- 106, 106, 107, 107, 107, 107, 108, 108, 108, 109, 109, 109, 109, 110, 110, 110,
- 111, 111, 111, 111, 112, 112, 112, 112, 113, 113, 113, 114, 114, 114, 114, 115,
- 115, 115, 116, 116, 116, 116, 117, 117, 117, 118, 118, 118, 118, 119, 119, 119,
- 120, 120, 120, 120, 121, 121, 121, 122, 122, 122, 123, 123, 123, 123, 124, 124,
- 124, 125, 125, 125, 125, 126, 126, 126, 127, 127, 127, 128, 128, 128, 128, 129,
- 129, 129, 130, 130, 130, 131, 131, 131, 132, 132, 132, 132, 133, 133, 133, 134,
- 134, 134, 135, 135, 135, 136, 136, 136, 136, 137, 137, 137, 138, 138, 138, 139,
- 139, 139, 140, 140, 140, 141, 141, 141, 142, 142, 142, 142, 143, 143, 143, 144,
- 144, 144, 145, 145, 145, 146, 146, 146, 147, 147, 147, 148, 148, 148, 149, 149,
- 149, 150, 150, 150, 151, 151, 151, 152, 152, 152, 153, 153, 153, 154, 154, 154,
- 155, 155, 155, 156, 156, 156, 157, 157, 157, 158, 158, 158, 159, 159, 159, 160,
- 160, 160, 161, 161, 161, 162, 162, 162, 163, 163, 163, 164, 164, 164, 165, 165,
- 165, 166, 166, 167, 167, 167, 168, 168, 168, 169, 169, 169, 170, 170, 170, 171,
- 171, 171, 172, 172, 172, 173, 173, 174, 174, 174, 175, 175, 175, 176, 176, 176,
- 177, 177, 177, 178, 178, 179, 179, 179, 180, 180, 180, 181, 181, 181, 182, 182,
- 183, 183, 183, 184, 184, 184, 185, 185, 186, 186, 186, 187, 187, 187, 188, 188,
- 188, 189, 189, 190, 190, 190, 191, 191, 191, 192, 192, 193, 193, 193, 194, 194,
- 194, 195, 195, 196, 196, 196, 197, 197, 198, 198, 198, 199, 199, 199, 200, 200,
- 201, 201, 201, 202, 202, 203, 203, 203, 204, 204, 204, 205, 205, 206, 206, 206,
- 207, 207, 208, 208, 208, 209, 209, 210, 210, 210, 211, 211, 212, 212, 212, 213,
- 213, 214, 214, 214, 215, 215, 216, 216, 216, 217, 217, 218, 218, 218, 219, 219,
- 220, 220, 220, 221, 221, 222, 222, 222, 223, 223, 224, 224, 224, 225, 225, 226,
- 226, 227, 227, 227, 228, 228, 229, 229, 229, 230, 230, 231, 231, 232, 232, 232,
- 233, 233, 234, 234, 234, 235, 235, 236, 236, 237, 237, 237, 238, 238, 239, 239,
- 240, 240, 240, 241, 241, 242, 242, 243, 243, 243, 244, 244, 245, 245, 246, 246,
- 246, 247, 247, 248, 248, 249, 249, 249, 250, 250, 251, 251, 252, 252, 252, 253,
- 253, 254, 254, 255, 255, 256, 256, 256, 257, 257, 258, 258, 259, 259, 260, 260,
- 260, 261, 261, 262, 262, 263, 263, 264, 264, 264, 265, 265, 266, 266, 267, 267,
- 268, 268, 269, 269, 269, 270, 270, 271, 271, 272, 272, 273, 273, 274, 274, 274,
- 275, 275, 276, 276, 277, 277, 278, 278, 279, 279, 279, 280, 280, 281, 281, 282,
- 282, 283, 283, 284, 284, 285, 285, 286, 286, 286, 287, 287, 288, 288, 289, 289,
- 290, 290, 291, 291, 292, 292, 293, 293, 294, 294, 295, 295, 295, 296, 296, 297,
- 297, 298, 298, 299, 299, 300, 300, 301, 301, 302, 302, 303, 303, 304, 304, 305,
- 305, 306, 306, 307, 307, 308, 308, 309, 309, 309, 310, 310, 311, 311, 312, 312,
- 313, 313, 314, 314, 315, 315, 316, 316, 317, 317, 318, 318, 319, 319, 320, 320,
- 321, 321, 322, 322, 323, 323, 324, 324, 325, 325, 326, 326, 327, 327, 328, 328,
- 329, 329, 330, 330, 331, 331, 332, 332, 333, 333, 334, 335, 335, 336, 336, 337,
- 337, 338, 338, 339, 339, 340, 340, 341, 341, 342, 342, 343, 343, 344, 344, 345,
- 345, 346, 346, 347, 347, 348, 348, 349, 349, 350, 351, 351, 352, 352, 353, 353,
- 354, 354, 355, 355, 356, 356, 357, 357, 358, 358, 359, 360, 360, 361, 361, 362,
- 362, 363, 363, 364, 364, 365, 365, 366, 366, 367, 368, 368, 369, 369, 370, 370,
- 371, 371, 372, 372, 373, 373, 374, 375, 375, 376, 376, 377, 377, 378, 378, 379,
- 379, 380, 381, 381, 382, 382, 383, 383, 384, 384, 385, 386, 386, 387, 387, 388,
- 388, 389, 389, 390, 391, 391, 392, 392, 393, 393, 394, 394, 395, 396, 396, 397,
- 397, 398, 398, 399, 399, 400, 401, 401, 402, 402, 403, 403, 404, 405, 405, 406,
- 406, 407, 407, 408, 409, 409, 410, 410, 411, 411, 412, 413, 413, 414, 414, 415,
- 415, 416, 417, 417, 418, 418, 419, 419, 420, 421, 421, 422, 422, 423, 424, 424,
- 425, 425, 426, 426, 427, 428, 428, 429, 429, 430, 431, 431, 432, 432, 433, 433,
- 434, 435, 435, 436, 436, 437, 438, 438, 439, 439, 440, 441, 441, 442, 442, 443,
- 444, 444, 445, 445, 446, 447, 447, 448, 448, 449, 450, 450, 451, 451, 452, 453,
- 453, 454, 454, 455, 456, 456, 457, 457, 458, 459, 459, 460, 460, 461, 462, 462,
- 463, 463, 464, 465, 465, 466, 467, 467, 468, 468, 469, 470, 470, 471, 471, 472,
- 473, 473, 474, 475, 475, 476, 476, 477, 478, 478, 479, 480, 480, 481, 481, 482,
- 483, 483, 484, 485, 485, 486, 486, 487, 488, 488, 489, 490, 490, 491, 491, 492,
- 493, 493, 494, 495, 495, 496, 497, 497, 498, 498, 499, 500, 500, 501, 502, 502,
- 503, 504, 504, 505, 505, 506, 507, 507, 508, 509, 509, 510, 511, 511, 512, 513,
- 513, 514, 514, 515, 516, 516, 517, 518, 518, 519, 520, 520, 521, 522, 522, 523,
- 524, 524, 525, 526, 526, 527, 528, 528, 529, 529, 530, 531, 531, 532, 533, 533,
- 534, 535, 535, 536, 537, 537, 538, 539, 539, 540, 541, 541, 542, 543, 543, 544,
- 545, 545, 546, 547, 547, 548, 549, 549, 550, 551, 551, 552, 553, 553, 554, 555,
- 555, 556, 557, 557, 558, 559, 560, 560, 561, 562, 562, 563, 564, 564, 565, 566,
- 566, 567, 568, 568, 569, 570, 570, 571, 572, 572, 573, 574, 575, 575, 576, 577,
- 577, 578, 579, 579, 580, 581, 581, 582, 583, 584, 584, 585, 586, 586, 587, 588,
- 588, 589, 590, 590, 591, 592, 593, 593, 594, 595, 595, 596, 597, 598, 598, 599,
- 600, 600, 601, 602, 602, 603, 604, 605, 605, 606, 607, 607, 608, 609, 610, 610,
- 611, 612, 612, 613, 614, 615, 615, 616, 617, 617, 618, 619, 620, 620, 621, 622,
- 622, 623, 624, 625, 625, 626, 627, 627, 628, 629, 630, 630, 631, 632, 632, 633,
- 634, 635, 635, 636, 637, 638, 638, 639, 640, 640, 641, 642, 643, 643, 644, 645,
- 646, 646, 647, 648, 649, 649, 650, 651, 652, 652, 653, 654, 654, 655, 656, 657,
- 657, 658, 659, 660, 660, 661, 662, 663, 663, 664, 665, 666, 666, 667, 668, 669,
- 669, 670, 671, 672, 672, 673, 674, 675, 675, 676, 677, 678, 678, 679, 680, 681,
- 681, 682, 683, 684, 684, 685, 686, 687, 687, 688, 689, 690, 690, 691, 692, 693,
- 694, 694, 695, 696, 697, 697, 698, 699, 700, 700, 701, 702, 703, 703, 704, 705,
- 706, 707, 707, 708, 709, 710, 710, 711, 712, 713, 714, 714, 715, 716, 717, 717,
- 718, 719, 720, 720, 721, 722, 723, 724, 724, 725, 726, 727, 728, 728, 729, 730,
- 731, 731, 732, 733, 734, 735, 735, 736, 737, 738, 739, 739, 740, 741, 742, 742,
- 743, 744, 745, 746, 746, 747, 748, 749, 750, 750, 751, 752, 753, 754, 754, 755,
- 756, 757, 758, 758, 759, 760, 761, 762, 762, 763, 764, 765, 766, 766, 767, 768,
- 769, 770, 771, 771, 772, 773, 774, 775, 775, 776, 777, 778, 779, 779, 780, 781,
- 782, 783, 783, 784, 785, 786, 787, 788, 788, 789, 790, 791, 792, 793, 793, 794,
- 795, 796, 797, 797, 798, 799, 800, 801, 802, 802, 803, 804, 805, 806, 807, 807,
- 808, 809, 810, 811, 812, 812, 813, 814, 815, 816, 817, 817, 818, 819, 820, 821,
- 822, 822, 823, 824, 825, 826, 827, 827, 828, 829, 830, 831, 832, 832, 833, 834,
- 835, 836, 837, 838, 838, 839, 840, 841, 842, 843, 843, 844, 845, 846, 847, 848,
- 849, 849, 850, 851, 852, 853, 854, 855, 855, 856, 857, 858, 859, 860, 861, 861,
- 862, 863, 864, 865, 866, 867, 867, 868, 869, 870, 871, 872, 873, 873, 874, 875,
- 876, 877, 878, 879, 880, 880, 881, 882, 883, 884, 885, 886, 887, 887, 888, 889,
- 890, 891, 892, 893, 894, 894, 895, 896, 897, 898, 899, 900, 901, 901, 902, 903,
- 904, 905, 906, 907, 908, 909, 909, 910, 911, 912, 913, 914, 915, 916, 916, 917,
- 918, 919, 920, 921, 922, 923, 924, 925, 925, 926, 927, 928, 929, 930, 931, 932,
- 933, 933, 934, 935, 936, 937, 938, 939, 940, 941, 942, 942, 943, 944, 945, 946,
- 947, 948, 949, 950, 951, 952, 952, 953, 954, 955, 956, 957, 958, 959, 960, 961,
- 962, 962, 963, 964, 965, 966, 967, 968, 969, 970, 971, 972, 973, 973, 974, 975,
- 976, 977, 978, 979, 980, 981, 982, 983, 984, 985, 985, 986, 987, 988, 989, 990,
- 991, 992, 993, 994, 995, 996, 997, 998, 998, 999, 1000, 1001, 1002, 1003, 1004, 1005,
- 1006, 1007, 1008, 1009, 1010, 1011, 1012, 1013, 1013, 1014, 1015, 1016, 1017, 1018, 1019, 1020,
- 1021, 1022, 1023, 1024, 1025, 1026, 1027, 1028, 1029, 1030, 1030, 1031, 1032, 1033, 1034, 1035,
- 1036, 1037, 1038, 1039, 1040, 1041, 1042, 1043, 1044, 1045, 1046, 1047, 1048, 1049, 1050, 1050,
- 1051, 1052, 1053, 1054, 1055, 1056, 1057, 1058, 1059, 1060, 1061, 1062, 1063, 1064, 1065, 1066,
- 1067, 1068, 1069, 1070, 1071, 1072, 1073, 1074, 1075, 1076, 1077, 1078, 1078, 1079, 1080, 1081,
- 1082, 1083, 1084, 1085, 1086, 1087, 1088, 1089, 1090, 1091, 1092, 1093, 1094, 1095, 1096, 1097,
- 1098, 1099, 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112, 1113,
- 1114, 1115, 1116, 1117, 1118, 1119, 1120, 1121, 1122, 1123, 1124, 1125, 1126, 1127, 1128, 1129,
- 1130, 1131, 1132, 1133, 1134, 1135, 1136, 1137, 1138, 1139, 1140, 1141, 1142, 1143, 1144, 1145,
- 1146, 1147, 1148, 1149, 1150, 1151, 1152, 1153, 1154, 1155, 1156, 1157, 1158, 1159, 1160, 1161,
- 1162, 1163, 1164, 1165, 1166, 1167, 1168, 1169, 1170, 1171, 1172, 1173, 1174, 1175, 1176, 1177,
- 1178, 1179, 1180, 1181, 1182, 1183, 1184, 1185, 1186, 1187, 1188, 1189, 1190, 1191, 1193, 1194,
- 1195, 1196, 1197, 1198, 1199, 1200, 1201, 1202, 1203, 1204, 1205, 1206, 1207, 1208, 1209, 1210,
- 1211, 1212, 1213, 1214, 1215, 1216, 1217, 1218, 1219, 1220, 1221, 1223, 1224, 1225, 1226, 1227,
- 1228, 1229, 1230, 1231, 1232, 1233, 1234, 1235, 1236, 1237, 1238, 1239, 1240, 1241, 1242, 1243,
- 1245, 1246, 1247, 1248, 1249, 1250, 1251, 1252, 1253, 1254, 1255, 1256, 1257, 1258, 1259, 1260,
- 1261, 1262, 1264, 1265, 1266, 1267, 1268, 1269, 1270, 1271, 1272, 1273, 1274, 1275, 1276, 1277,
- 1278, 1280, 1281, 1282, 1283, 1284, 1285, 1286, 1287, 1288, 1289, 1290, 1291, 1292, 1293, 1295,
- 1296, 1297, 1298, 1299, 1300, 1301, 1302, 1303, 1304, 1305, 1306, 1307, 1309, 1310, 1311, 1312,
- 1313, 1314, 1315, 1316, 1317, 1318, 1319, 1320, 1322, 1323, 1324, 1325, 1326, 1327, 1328, 1329,
- 1330, 1331, 1332, 1334, 1335, 1336, 1337, 1338, 1339, 1340, 1341, 1342, 1343, 1345, 1346, 1347,
- 1348, 1349, 1350, 1351, 1352, 1353, 1354, 1356, 1357, 1358, 1359, 1360, 1361, 1362, 1363, 1364,
- 1365, 1367, 1368, 1369, 1370, 1371, 1372, 1373, 1374, 1375, 1377, 1378, 1379, 1380, 1381, 1382,
- 1383, 1384, 1385, 1387, 1388, 1389, 1390, 1391, 1392, 1393, 1394, 1396, 1397, 1398, 1399, 1400,
- 1401, 1402, 1403, 1405, 1406, 1407, 1408, 1409, 1410, 1411, 1412, 1414, 1415, 1416, 1417, 1418,
- 1419, 1420, 1421, 1423, 1424, 1425, 1426, 1427, 1428, 1429, 1431, 1432, 1433, 1434, 1435, 1436,
- 1437, 1439, 1440, 1441, 1442, 1443, 1444, 1445, 1446, 1448, 1449, 1450, 1451, 1452, 1453, 1455,
- 1456, 1457, 1458, 1459, 1460, 1461, 1463, 1464, 1465, 1466, 1467, 1468, 1469, 1471, 1472, 1473,
- 1474, 1475, 1476, 1478, 1479, 1480, 1481, 1482, 1483, 1484, 1486, 1487, 1488, 1489, 1490, 1491,
- 1493, 1494, 1495, 1496, 1497, 1498, 1500, 1501, 1502, 1503, 1504, 1505, 1507, 1508, 1509, 1510,
- 1511, 1512, 1514, 1515, 1516, 1517, 1518, 1519, 1521, 1522, 1523, 1524, 1525, 1527, 1528, 1529,
- 1530, 1531, 1532, 1534, 1535, 1536, 1537, 1538, 1540, 1541, 1542, 1543, 1544, 1545, 1547, 1548,
- 1549, 1550, 1551, 1553, 1554, 1555, 1556, 1557, 1559, 1560, 1561, 1562, 1563, 1564, 1566, 1567,
- 1568, 1569, 1570, 1572, 1573, 1574, 1575, 1576, 1578, 1579, 1580, 1581, 1582, 1584, 1585, 1586,
- 1587, 1588, 1590, 1591, 1592, 1593, 1594, 1596, 1597, 1598, 1599, 1601, 1602, 1603, 1604, 1605,
- 1607, 1608, 1609, 1610, 1611, 1613, 1614, 1615, 1616, 1617, 1619, 1620, 1621, 1622, 1624, 1625,
- 1626, 1627, 1628, 1630, 1631, 1632, 1633, 1635, 1636, 1637, 1638, 1639, 1641, 1642, 1643, 1644,
- 1646, 1647, 1648, 1649, 1650, 1652, 1653, 1654, 1655, 1657, 1658, 1659, 1660, 1662, 1663, 1664,
- 1665, 1667, 1668, 1669, 1670, 1671, 1673, 1674, 1675, 1676, 1678, 1679, 1680, 1681, 1683, 1684,
- 1685, 1686, 1688, 1689, 1690, 1691, 1693, 1694, 1695, 1696, 1698, 1699, 1700, 1701, 1703, 1704,
- 1705, 1706, 1708, 1709, 1710, 1711, 1713, 1714, 1715, 1716, 1718, 1719, 1720, 1721, 1723, 1724,
- 1725, 1726, 1728, 1729, 1730, 1731, 1733, 1734, 1735, 1737, 1738, 1739, 1740, 1742, 1743, 1744,
- 1745, 1747, 1748, 1749, 1750, 1752, 1753, 1754, 1756, 1757, 1758, 1759, 1761, 1762, 1763, 1764,
- 1766, 1767, 1768, 1770, 1771, 1772, 1773, 1775, 1776, 1777, 1778, 1780, 1781, 1782, 1784, 1785,
- 1786, 1787, 1789, 1790, 1791, 1793, 1794, 1795, 1796, 1798, 1799, 1800, 1802, 1803, 1804, 1806,
- 1807, 1808, 1809, 1811, 1812, 1813, 1815, 1816, 1817, 1818, 1820, 1821, 1822, 1824, 1825, 1826,
- 1828, 1829, 1830, 1831, 1833, 1834, 1835, 1837, 1838, 1839, 1841, 1842, 1843, 1844, 1846, 1847,
- 1848, 1850, 1851, 1852, 1854, 1855, 1856, 1858, 1859, 1860, 1862, 1863, 1864, 1865, 1867, 1868,
- 1869, 1871, 1872, 1873, 1875, 1876, 1877, 1879, 1880, 1881, 1883, 1884, 1885, 1887, 1888, 1889,
- 1891, 1892, 1893, 1894, 1896, 1897, 1898, 1900, 1901, 1902, 1904, 1905, 1906, 1908, 1909, 1910,
- 1912, 1913, 1914, 1916, 1917, 1918, 1920, 1921, 1922, 1924, 1925, 1926, 1928, 1929, 1930, 1932,
- 1933, 1935, 1936, 1937, 1939, 1940, 1941, 1943, 1944, 1945, 1947, 1948, 1949, 1951, 1952, 1953,
- 1955, 1956, 1957, 1959, 1960, 1961, 1963, 1964, 1965, 1967, 1968, 1970, 1971, 1972, 1974, 1975,
- 1976, 1978, 1979, 1980, 1982, 1983, 1984, 1986, 1987, 1989, 1990, 1991, 1993, 1994, 1995, 1997,
- 1998, 1999, 2001, 2002, 2004, 2005, 2006, 2008, 2009, 2010, 2012, 2013, 2015, 2016, 2017, 2019,
- 2020, 2021, 2023, 2024, 2026, 2027, 2028, 2030, 2031, 2032, 2034, 2035, 2037, 2038, 2039, 2041,
- 2042, 2043, 2045, 2046, 2048, 2049, 2050, 2052, 2053, 2055, 2056, 2057, 2059, 2060, 2061, 2063,
- 2064, 2066, 2067, 2068, 2070, 2071, 2073, 2074, 2075, 2077, 2078, 2080, 2081, 2082, 2084, 2085,
- 2087, 2088, 2089, 2091, 2092, 2094, 2095, 2096, 2098, 2099, 2101, 2102, 2103, 2105, 2106, 2108,
- 2109, 2110, 2112, 2113, 2115, 2116, 2117, 2119, 2120, 2122, 2123, 2124, 2126, 2127, 2129, 2130,
- 2132, 2133, 2134, 2136, 2137, 2139, 2140, 2141, 2143, 2144, 2146, 2147, 2149, 2150, 2151, 2153,
- 2154, 2156, 2157, 2159, 2160, 2161, 2163, 2164, 2166, 2167, 2169, 2170, 2171, 2173, 2174, 2176,
- 2177, 2179, 2180, 2181, 2183, 2184, 2186, 2187, 2189, 2190, 2191, 2193, 2194, 2196, 2197, 2199,
- 2200, 2202, 2203, 2204, 2206, 2207, 2209, 2210, 2212, 2213, 2214, 2216, 2217, 2219, 2220, 2222,
- 2223, 2225, 2226, 2228, 2229, 2230, 2232, 2233, 2235, 2236, 2238, 2239, 2241, 2242, 2243, 2245,
- 2246, 2248, 2249, 2251, 2252, 2254, 2255, 2257, 2258, 2260, 2261, 2262, 2264, 2265, 2267, 2268,
- 2270, 2271, 2273, 2274, 2276, 2277, 2279, 2280, 2282, 2283, 2284, 2286, 2287, 2289, 2290, 2292,
- 2293, 2295, 2296, 2298, 2299, 2301, 2302, 2304, 2305, 2307, 2308, 2310, 2311, 2312, 2314, 2315,
- 2317, 2318, 2320, 2321, 2323, 2324, 2326, 2327, 2329, 2330, 2332, 2333, 2335, 2336, 2338, 2339,
- 2341, 2342, 2344, 2345, 2347, 2348, 2350, 2351, 2353, 2354, 2356, 2357, 2359, 2360, 2362, 2363,
- 2365, 2366, 2368, 2369, 2371, 2372, 2374, 2375, 2377, 2378, 2380, 2381, 2383, 2384, 2386, 2387,
- 2389, 2390, 2392, 2393, 2395, 2396, 2398, 2399, 2401, 2402, 2404, 2405, 2407, 2408, 2410, 2411,
- 2413, 2414, 2416, 2417, 2419, 2420, 2422, 2423, 2425, 2426, 2428, 2429, 2431, 2433, 2434, 2436,
- 2437, 2439, 2440, 2442, 2443, 2445, 2446, 2448, 2449, 2451, 2452, 2454, 2455, 2457, 2458, 2460,
- 2462, 2463, 2465, 2466, 2468, 2469, 2471, 2472, 2474, 2475, 2477, 2478, 2480, 2481, 2483, 2485,
- 2486, 2488, 2489, 2491, 2492, 2494, 2495, 2497, 2498, 2500, 2502, 2503, 2505, 2506, 2508, 2509,
- 2511, 2512, 2514, 2515, 2517, 2519, 2520, 2522, 2523, 2525, 2526, 2528, 2529, 2531, 2533, 2534,
- 2536, 2537, 2539, 2540, 2542, 2543, 2545, 2547, 2548, 2550, 2551, 2553, 2554, 2556, 2557, 2559,
- 2561, 2562, 2564, 2565, 2567, 2568, 2570, 2572, 2573, 2575, 2576, 2578, 2579, 2581, 2583, 2584,
- 2586, 2587, 2589, 2590, 2592, 2594, 2595, 2597, 2598, 2600, 2601, 2603, 2605, 2606, 2608, 2609,
- 2611, 2613, 2614, 2616, 2617, 2619, 2620, 2622, 2624, 2625, 2627, 2628, 2630, 2632, 2633, 2635,
- 2636, 2638, 2640, 2641, 2643, 2644, 2646, 2647, 2649, 2651, 2652, 2654, 2655, 2657, 2659, 2660,
- 2662, 2663, 2665, 2667, 2668, 2670, 2671, 2673, 2675, 2676, 2678, 2679, 2681, 2683, 2684, 2686,
- 2687, 2689, 2691, 2692, 2694, 2696, 2697, 2699, 2700, 2702, 2704, 2705, 2707, 2708, 2710, 2712,
- 2713, 2715, 2716, 2718, 2720, 2721, 2723, 2725, 2726, 2728, 2729, 2731, 2733, 2734, 2736, 2738,
- 2739, 2741, 2742, 2744, 2746, 2747, 2749, 2751, 2752, 2754, 2755, 2757, 2759, 2760, 2762, 2764,
- 2765, 2767, 2769, 2770, 2772, 2773, 2775, 2777, 2778, 2780, 2782, 2783, 2785, 2787, 2788, 2790,
- 2791, 2793, 2795, 2796, 2798, 2800, 2801, 2803, 2805, 2806, 2808, 2810, 2811, 2813, 2814, 2816,
- 2818, 2819, 2821, 2823, 2824, 2826, 2828, 2829, 2831, 2833, 2834, 2836, 2838, 2839, 2841, 2843,
- 2844, 2846, 2848, 2849, 2851, 2853, 2854, 2856, 2857, 2859, 2861, 2862, 2864, 2866, 2867, 2869,
- 2871, 2872, 2874, 2876, 2877, 2879, 2881, 2882, 2884, 2886, 2888, 2889, 2891, 2893, 2894, 2896,
- 2898, 2899, 2901, 2903, 2904, 2906, 2908, 2909, 2911, 2913, 2914, 2916, 2918, 2919, 2921, 2923,
- 2924, 2926, 2928, 2929, 2931, 2933, 2935, 2936, 2938, 2940, 2941, 2943, 2945, 2946, 2948, 2950,
- 2951, 2953, 2955, 2956, 2958, 2960, 2962, 2963, 2965, 2967, 2968, 2970, 2972, 2973, 2975, 2977,
- 2979, 2980, 2982, 2984, 2985, 2987, 2989, 2990, 2992, 2994, 2996, 2997, 2999, 3001, 3002, 3004,
- 3006, 3008, 3009, 3011, 3013, 3014, 3016, 3018, 3020, 3021, 3023, 3025, 3026, 3028, 3030, 3032,
- 3033, 3035, 3037, 3038, 3040, 3042, 3044, 3045, 3047, 3049, 3050, 3052, 3054, 3056, 3057, 3059,
- 3061, 3063, 3064, 3066, 3068, 3069, 3071, 3073, 3075, 3076, 3078, 3080, 3082, 3083, 3085, 3087,
- 3089, 3090, 3092, 3094, 3095, 3097, 3099, 3101, 3102, 3104, 3106, 3108, 3109, 3111, 3113, 3115,
- 3116, 3118, 3120, 3122, 3123, 3125, 3127, 3129, 3130, 3132, 3134, 3136, 3137, 3139, 3141, 3143,
- 3144, 3146, 3148, 3150, 3151, 3153, 3155, 3157, 3158, 3160, 3162, 3164, 3165, 3167, 3169, 3171,
- 3172, 3174, 3176, 3178, 3179, 3181, 3183, 3185, 3187, 3188, 3190, 3192, 3194, 3195, 3197, 3199,
- 3201, 3202, 3204, 3206, 3208, 3209, 3211, 3213, 3215, 3217, 3218, 3220, 3222, 3224, 3225, 3227,
- 3229, 3231, 3233, 3234, 3236, 3238, 3240, 3241, 3243, 3245, 3247, 3249, 3250, 3252, 3254, 3256,
- 3258, 3259, 3261, 3263, 3265, 3266, 3268, 3270, 3272, 3274, 3275, 3277, 3279, 3281, 3283, 3284,
- 3286, 3288, 3290, 3292, 3293, 3295, 3297, 3299, 3301, 3302, 3304, 3306, 3308, 3310, 3311, 3313,
- 3315, 3317, 3319, 3320, 3322, 3324, 3326, 3328, 3329, 3331, 3333, 3335, 3337, 3338, 3340, 3342,
- 3344, 3346, 3348, 3349, 3351, 3353, 3355, 3357, 3358, 3360, 3362, 3364, 3366, 3368, 3369, 3371,
- 3373, 3375, 3377, 3378, 3380, 3382, 3384, 3386, 3388, 3389, 3391, 3393, 3395, 3397, 3399, 3400,
- 3402, 3404, 3406, 3408, 3410, 3411, 3413, 3415, 3417, 3419, 3421, 3422, 3424, 3426, 3428, 3430,
- 3432, 3433, 3435, 3437, 3439, 3441, 3443, 3444, 3446, 3448, 3450, 3452, 3454, 3455, 3457, 3459,
- 3461, 3463, 3465, 3467, 3468, 3470, 3472, 3474, 3476, 3478, 3480, 3481, 3483, 3485, 3487, 3489,
- 3491, 3492, 3494, 3496, 3498, 3500, 3502, 3504, 3506, 3507, 3509, 3511, 3513, 3515, 3517, 3519,
- 3520, 3522, 3524, 3526, 3528, 3530, 3532, 3533, 3535, 3537, 3539, 3541, 3543, 3545, 3547, 3548,
- 3550, 3552, 3554, 3556, 3558, 3560, 3562, 3563, 3565, 3567, 3569, 3571, 3573, 3575, 3577, 3578,
- 3580, 3582, 3584, 3586, 3588, 3590, 3592, 3594, 3595, 3597, 3599, 3601, 3603, 3605, 3607, 3609,
- 3611, 3612, 3614, 3616, 3618, 3620, 3622, 3624, 3626, 3628, 3629, 3631, 3633, 3635, 3637, 3639,
- 3641, 3643, 3645, 3647, 3648, 3650, 3652, 3654, 3656, 3658, 3660, 3662, 3664, 3666, 3667, 3669,
- 3671, 3673, 3675, 3677, 3679, 3681, 3683, 3685, 3687, 3688, 3690, 3692, 3694, 3696, 3698, 3700,
- 3702, 3704, 3706, 3708, 3710, 3711, 3713, 3715, 3717, 3719, 3721, 3723, 3725, 3727, 3729, 3731,
- 3733, 3735, 3736, 3738, 3740, 3742, 3744, 3746, 3748, 3750, 3752, 3754, 3756, 3758, 3760, 3762,
- 3764, 3765, 3767, 3769, 3771, 3773, 3775, 3777, 3779, 3781, 3783, 3785, 3787, 3789, 3791, 3793,
- 3795, 3796, 3798, 3800, 3802, 3804, 3806, 3808, 3810, 3812, 3814, 3816, 3818, 3820, 3822, 3824,
- 3826, 3828, 3830, 3832, 3833, 3835, 3837, 3839, 3841, 3843, 3845, 3847, 3849, 3851, 3853, 3855,
- 3857, 3859, 3861, 3863, 3865, 3867, 3869, 3871, 3873, 3875, 3877, 3879, 3881, 3883, 3884, 3886,
- 3888, 3890, 3892, 3894, 3896, 3898, 3900, 3902, 3904, 3906, 3908, 3910, 3912, 3914, 3916, 3918,
- 3920, 3922, 3924, 3926, 3928, 3930, 3932, 3934, 3936, 3938, 3940, 3942, 3944, 3946, 3948, 3950,
- 3952, 3954, 3956, 3958, 3960, 3962, 3964, 3966, 3968, 3970, 3972, 3974, 3976, 3978, 3980, 3982,
- 3984, 3986, 3988, 3990, 3992, 3994, 3996, 3998, 4000, 4002, 4004, 4006, 4008, 4010, 4012, 4014,
- 4016, 4018, 4020, 4022, 4024, 4026, 4028, 4030, 4032, 4034, 4036, 4038, 4040, 4042, 4044, 4046,
- 4048, 4050, 4052, 4054, 4056, 4058, 4060, 4062, 4064, 4066, 4068, 4070, 4072, 4074, 4076, 4078,
- 4080,
-};
-
-/* Generated table */
-const unsigned short tpg_linear_to_rec709[255 * 16 + 1] = {
- 0, 5, 9, 14, 18, 22, 27, 32, 36, 41, 45, 50, 54, 59, 63, 68,
- 72, 77, 81, 86, 90, 95, 99, 104, 108, 113, 117, 122, 126, 131, 135, 139,
- 144, 149, 153, 158, 162, 167, 171, 176, 180, 185, 189, 194, 198, 203, 207, 212,
- 216, 221, 225, 230, 234, 239, 243, 248, 252, 257, 261, 266, 270, 275, 279, 284,
- 288, 293, 297, 302, 306, 311, 315, 320, 324, 328, 334, 338, 343, 347, 352, 356,
- 360, 365, 369, 373, 377, 381, 386, 390, 394, 398, 402, 406, 410, 414, 418, 422,
- 426, 430, 433, 437, 441, 445, 449, 452, 456, 460, 464, 467, 471, 475, 478, 482,
- 485, 489, 492, 496, 499, 503, 506, 510, 513, 517, 520, 524, 527, 530, 534, 537,
- 540, 544, 547, 550, 554, 557, 560, 563, 566, 570, 573, 576, 579, 582, 586, 589,
- 592, 595, 598, 601, 604, 607, 610, 613, 616, 619, 622, 625, 628, 631, 634, 637,
- 640, 643, 646, 649, 652, 655, 658, 660, 663, 666, 669, 672, 675, 677, 680, 683,
- 686, 689, 691, 694, 697, 700, 702, 705, 708, 711, 713, 716, 719, 721, 724, 727,
- 729, 732, 735, 737, 740, 743, 745, 748, 750, 753, 756, 758, 761, 763, 766, 768,
- 771, 773, 776, 779, 781, 784, 786, 789, 791, 794, 796, 799, 801, 803, 806, 808,
- 811, 813, 816, 818, 821, 823, 825, 828, 830, 833, 835, 837, 840, 842, 844, 847,
- 849, 851, 854, 856, 858, 861, 863, 865, 868, 870, 872, 875, 877, 879, 881, 884,
- 886, 888, 891, 893, 895, 897, 900, 902, 904, 906, 908, 911, 913, 915, 917, 919,
- 922, 924, 926, 928, 930, 933, 935, 937, 939, 941, 943, 946, 948, 950, 952, 954,
- 956, 958, 960, 963, 965, 967, 969, 971, 973, 975, 977, 979, 981, 984, 986, 988,
- 990, 992, 994, 996, 998, 1000, 1002, 1004, 1006, 1008, 1010, 1012, 1014, 1016, 1018, 1020,
- 1022, 1024, 1026, 1028, 1030, 1032, 1034, 1036, 1038, 1040, 1042, 1044, 1046, 1048, 1050, 1052,
- 1054, 1056, 1058, 1060, 1062, 1064, 1066, 1068, 1069, 1071, 1073, 1075, 1077, 1079, 1081, 1083,
- 1085, 1087, 1089, 1090, 1092, 1094, 1096, 1098, 1100, 1102, 1104, 1106, 1107, 1109, 1111, 1113,
- 1115, 1117, 1119, 1120, 1122, 1124, 1126, 1128, 1130, 1131, 1133, 1135, 1137, 1139, 1141, 1142,
- 1144, 1146, 1148, 1150, 1151, 1153, 1155, 1157, 1159, 1160, 1162, 1164, 1166, 1168, 1169, 1171,
- 1173, 1175, 1176, 1178, 1180, 1182, 1184, 1185, 1187, 1189, 1191, 1192, 1194, 1196, 1198, 1199,
- 1201, 1203, 1204, 1206, 1208, 1210, 1211, 1213, 1215, 1217, 1218, 1220, 1222, 1223, 1225, 1227,
- 1228, 1230, 1232, 1234, 1235, 1237, 1239, 1240, 1242, 1244, 1245, 1247, 1249, 1250, 1252, 1254,
- 1255, 1257, 1259, 1260, 1262, 1264, 1265, 1267, 1269, 1270, 1272, 1274, 1275, 1277, 1279, 1280,
- 1282, 1283, 1285, 1287, 1288, 1290, 1292, 1293, 1295, 1296, 1298, 1300, 1301, 1303, 1305, 1306,
- 1308, 1309, 1311, 1313, 1314, 1316, 1317, 1319, 1321, 1322, 1324, 1325, 1327, 1328, 1330, 1332,
- 1333, 1335, 1336, 1338, 1339, 1341, 1343, 1344, 1346, 1347, 1349, 1350, 1352, 1354, 1355, 1357,
- 1358, 1360, 1361, 1363, 1364, 1366, 1367, 1369, 1371, 1372, 1374, 1375, 1377, 1378, 1380, 1381,
- 1383, 1384, 1386, 1387, 1389, 1390, 1392, 1393, 1395, 1396, 1398, 1399, 1401, 1402, 1404, 1405,
- 1407, 1408, 1410, 1411, 1413, 1414, 1416, 1417, 1419, 1420, 1422, 1423, 1425, 1426, 1428, 1429,
- 1431, 1432, 1434, 1435, 1437, 1438, 1440, 1441, 1442, 1444, 1445, 1447, 1448, 1450, 1451, 1453,
- 1454, 1456, 1457, 1458, 1460, 1461, 1463, 1464, 1466, 1467, 1469, 1470, 1471, 1473, 1474, 1476,
- 1477, 1479, 1480, 1481, 1483, 1484, 1486, 1487, 1489, 1490, 1491, 1493, 1494, 1496, 1497, 1498,
- 1500, 1501, 1503, 1504, 1505, 1507, 1508, 1510, 1511, 1512, 1514, 1515, 1517, 1518, 1519, 1521,
- 1522, 1524, 1525, 1526, 1528, 1529, 1531, 1532, 1533, 1535, 1536, 1537, 1539, 1540, 1542, 1543,
- 1544, 1546, 1547, 1548, 1550, 1551, 1553, 1554, 1555, 1557, 1558, 1559, 1561, 1562, 1563, 1565,
- 1566, 1567, 1569, 1570, 1571, 1573, 1574, 1576, 1577, 1578, 1580, 1581, 1582, 1584, 1585, 1586,
- 1588, 1589, 1590, 1592, 1593, 1594, 1596, 1597, 1598, 1600, 1601, 1602, 1603, 1605, 1606, 1607,
- 1609, 1610, 1611, 1613, 1614, 1615, 1617, 1618, 1619, 1621, 1622, 1623, 1624, 1626, 1627, 1628,
- 1630, 1631, 1632, 1634, 1635, 1636, 1637, 1639, 1640, 1641, 1643, 1644, 1645, 1647, 1648, 1649,
- 1650, 1652, 1653, 1654, 1655, 1657, 1658, 1659, 1661, 1662, 1663, 1664, 1666, 1667, 1668, 1670,
- 1671, 1672, 1673, 1675, 1676, 1677, 1678, 1680, 1681, 1682, 1683, 1685, 1686, 1687, 1688, 1690,
- 1691, 1692, 1693, 1695, 1696, 1697, 1698, 1700, 1701, 1702, 1703, 1705, 1706, 1707, 1708, 1710,
- 1711, 1712, 1713, 1715, 1716, 1717, 1718, 1720, 1721, 1722, 1723, 1724, 1726, 1727, 1728, 1729,
- 1731, 1732, 1733, 1734, 1736, 1737, 1738, 1739, 1740, 1742, 1743, 1744, 1745, 1746, 1748, 1749,
- 1750, 1751, 1753, 1754, 1755, 1756, 1757, 1759, 1760, 1761, 1762, 1763, 1765, 1766, 1767, 1768,
- 1769, 1771, 1772, 1773, 1774, 1775, 1777, 1778, 1779, 1780, 1781, 1783, 1784, 1785, 1786, 1787,
- 1788, 1790, 1791, 1792, 1793, 1794, 1796, 1797, 1798, 1799, 1800, 1801, 1803, 1804, 1805, 1806,
- 1807, 1809, 1810, 1811, 1812, 1813, 1814, 1816, 1817, 1818, 1819, 1820, 1821, 1823, 1824, 1825,
- 1826, 1827, 1828, 1829, 1831, 1832, 1833, 1834, 1835, 1836, 1838, 1839, 1840, 1841, 1842, 1843,
- 1845, 1846, 1847, 1848, 1849, 1850, 1851, 1853, 1854, 1855, 1856, 1857, 1858, 1859, 1861, 1862,
- 1863, 1864, 1865, 1866, 1867, 1868, 1870, 1871, 1872, 1873, 1874, 1875, 1876, 1878, 1879, 1880,
- 1881, 1882, 1883, 1884, 1885, 1887, 1888, 1889, 1890, 1891, 1892, 1893, 1894, 1896, 1897, 1898,
- 1899, 1900, 1901, 1902, 1903, 1904, 1906, 1907, 1908, 1909, 1910, 1911, 1912, 1913, 1914, 1916,
- 1917, 1918, 1919, 1920, 1921, 1922, 1923, 1924, 1925, 1927, 1928, 1929, 1930, 1931, 1932, 1933,
- 1934, 1935, 1936, 1938, 1939, 1940, 1941, 1942, 1943, 1944, 1945, 1946, 1947, 1948, 1950, 1951,
- 1952, 1953, 1954, 1955, 1956, 1957, 1958, 1959, 1960, 1961, 1963, 1964, 1965, 1966, 1967, 1968,
- 1969, 1970, 1971, 1972, 1973, 1974, 1975, 1977, 1978, 1979, 1980, 1981, 1982, 1983, 1984, 1985,
- 1986, 1987, 1988, 1989, 1990, 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002,
- 2003, 2004, 2005, 2006, 2007, 2008, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019,
- 2020, 2021, 2022, 2023, 2024, 2025, 2026, 2027, 2028, 2029, 2031, 2032, 2033, 2034, 2035, 2036,
- 2037, 2038, 2039, 2040, 2041, 2042, 2043, 2044, 2045, 2046, 2047, 2048, 2049, 2050, 2051, 2052,
- 2053, 2054, 2055, 2056, 2057, 2058, 2060, 2061, 2062, 2063, 2064, 2065, 2066, 2067, 2068, 2069,
- 2070, 2071, 2072, 2073, 2074, 2075, 2076, 2077, 2078, 2079, 2080, 2081, 2082, 2083, 2084, 2085,
- 2086, 2087, 2088, 2089, 2090, 2091, 2092, 2093, 2094, 2095, 2096, 2097, 2098, 2099, 2100, 2101,
- 2102, 2103, 2104, 2105, 2106, 2107, 2108, 2109, 2110, 2111, 2112, 2113, 2114, 2115, 2116, 2117,
- 2118, 2119, 2120, 2121, 2122, 2123, 2124, 2125, 2126, 2127, 2128, 2129, 2130, 2131, 2132, 2133,
- 2134, 2135, 2136, 2137, 2138, 2139, 2140, 2141, 2142, 2143, 2144, 2145, 2146, 2147, 2148, 2149,
- 2150, 2151, 2152, 2153, 2154, 2155, 2156, 2157, 2158, 2159, 2160, 2161, 2162, 2163, 2164, 2165,
- 2166, 2167, 2168, 2169, 2170, 2171, 2172, 2173, 2173, 2174, 2175, 2176, 2177, 2178, 2179, 2180,
- 2181, 2182, 2183, 2184, 2185, 2186, 2187, 2188, 2189, 2190, 2191, 2192, 2193, 2194, 2195, 2196,
- 2197, 2198, 2199, 2200, 2201, 2202, 2202, 2203, 2204, 2205, 2206, 2207, 2208, 2209, 2210, 2211,
- 2212, 2213, 2214, 2215, 2216, 2217, 2218, 2219, 2220, 2221, 2222, 2223, 2224, 2224, 2225, 2226,
- 2227, 2228, 2229, 2230, 2231, 2232, 2233, 2234, 2235, 2236, 2237, 2238, 2239, 2240, 2241, 2241,
- 2242, 2243, 2244, 2245, 2246, 2247, 2248, 2249, 2250, 2251, 2252, 2253, 2254, 2255, 2256, 2257,
- 2257, 2258, 2259, 2260, 2261, 2262, 2263, 2264, 2265, 2266, 2267, 2268, 2269, 2270, 2271, 2271,
- 2272, 2273, 2274, 2275, 2276, 2277, 2278, 2279, 2280, 2281, 2282, 2283, 2283, 2284, 2285, 2286,
- 2287, 2288, 2289, 2290, 2291, 2292, 2293, 2294, 2295, 2295, 2296, 2297, 2298, 2299, 2300, 2301,
- 2302, 2303, 2304, 2305, 2306, 2306, 2307, 2308, 2309, 2310, 2311, 2312, 2313, 2314, 2315, 2316,
- 2317, 2317, 2318, 2319, 2320, 2321, 2322, 2323, 2324, 2325, 2326, 2327, 2327, 2328, 2329, 2330,
- 2331, 2332, 2333, 2334, 2335, 2336, 2336, 2337, 2338, 2339, 2340, 2341, 2342, 2343, 2344, 2345,
- 2345, 2346, 2347, 2348, 2349, 2350, 2351, 2352, 2353, 2354, 2354, 2355, 2356, 2357, 2358, 2359,
- 2360, 2361, 2362, 2363, 2363, 2364, 2365, 2366, 2367, 2368, 2369, 2370, 2371, 2371, 2372, 2373,
- 2374, 2375, 2376, 2377, 2378, 2379, 2379, 2380, 2381, 2382, 2383, 2384, 2385, 2386, 2386, 2387,
- 2388, 2389, 2390, 2391, 2392, 2393, 2394, 2394, 2395, 2396, 2397, 2398, 2399, 2400, 2401, 2401,
- 2402, 2403, 2404, 2405, 2406, 2407, 2408, 2408, 2409, 2410, 2411, 2412, 2413, 2414, 2415, 2415,
- 2416, 2417, 2418, 2419, 2420, 2421, 2422, 2422, 2423, 2424, 2425, 2426, 2427, 2428, 2428, 2429,
- 2430, 2431, 2432, 2433, 2434, 2435, 2435, 2436, 2437, 2438, 2439, 2440, 2441, 2441, 2442, 2443,
- 2444, 2445, 2446, 2447, 2447, 2448, 2449, 2450, 2451, 2452, 2453, 2453, 2454, 2455, 2456, 2457,
- 2458, 2459, 2459, 2460, 2461, 2462, 2463, 2464, 2465, 2465, 2466, 2467, 2468, 2469, 2470, 2471,
- 2471, 2472, 2473, 2474, 2475, 2476, 2477, 2477, 2478, 2479, 2480, 2481, 2482, 2482, 2483, 2484,
- 2485, 2486, 2487, 2488, 2488, 2489, 2490, 2491, 2492, 2493, 2493, 2494, 2495, 2496, 2497, 2498,
- 2499, 2499, 2500, 2501, 2502, 2503, 2504, 2504, 2505, 2506, 2507, 2508, 2509, 2509, 2510, 2511,
- 2512, 2513, 2514, 2514, 2515, 2516, 2517, 2518, 2519, 2519, 2520, 2521, 2522, 2523, 2524, 2524,
- 2525, 2526, 2527, 2528, 2529, 2529, 2530, 2531, 2532, 2533, 2534, 2534, 2535, 2536, 2537, 2538,
- 2539, 2539, 2540, 2541, 2542, 2543, 2544, 2544, 2545, 2546, 2547, 2548, 2548, 2549, 2550, 2551,
- 2552, 2553, 2553, 2554, 2555, 2556, 2557, 2558, 2558, 2559, 2560, 2561, 2562, 2562, 2563, 2564,
- 2565, 2566, 2567, 2567, 2568, 2569, 2570, 2571, 2571, 2572, 2573, 2574, 2575, 2576, 2576, 2577,
- 2578, 2579, 2580, 2580, 2581, 2582, 2583, 2584, 2584, 2585, 2586, 2587, 2588, 2589, 2589, 2590,
- 2591, 2592, 2593, 2593, 2594, 2595, 2596, 2597, 2597, 2598, 2599, 2600, 2601, 2601, 2602, 2603,
- 2604, 2605, 2605, 2606, 2607, 2608, 2609, 2610, 2610, 2611, 2612, 2613, 2614, 2614, 2615, 2616,
- 2617, 2618, 2618, 2619, 2620, 2621, 2622, 2622, 2623, 2624, 2625, 2626, 2626, 2627, 2628, 2629,
- 2630, 2630, 2631, 2632, 2633, 2634, 2634, 2635, 2636, 2637, 2637, 2638, 2639, 2640, 2641, 2641,
- 2642, 2643, 2644, 2645, 2645, 2646, 2647, 2648, 2649, 2649, 2650, 2651, 2652, 2653, 2653, 2654,
- 2655, 2656, 2656, 2657, 2658, 2659, 2660, 2660, 2661, 2662, 2663, 2664, 2664, 2665, 2666, 2667,
- 2668, 2668, 2669, 2670, 2671, 2671, 2672, 2673, 2674, 2675, 2675, 2676, 2677, 2678, 2678, 2679,
- 2680, 2681, 2682, 2682, 2683, 2684, 2685, 2686, 2686, 2687, 2688, 2689, 2689, 2690, 2691, 2692,
- 2693, 2693, 2694, 2695, 2696, 2696, 2697, 2698, 2699, 2700, 2700, 2701, 2702, 2703, 2703, 2704,
- 2705, 2706, 2706, 2707, 2708, 2709, 2710, 2710, 2711, 2712, 2713, 2713, 2714, 2715, 2716, 2717,
- 2717, 2718, 2719, 2720, 2720, 2721, 2722, 2723, 2723, 2724, 2725, 2726, 2727, 2727, 2728, 2729,
- 2730, 2730, 2731, 2732, 2733, 2733, 2734, 2735, 2736, 2736, 2737, 2738, 2739, 2740, 2740, 2741,
- 2742, 2743, 2743, 2744, 2745, 2746, 2746, 2747, 2748, 2749, 2749, 2750, 2751, 2752, 2752, 2753,
- 2754, 2755, 2755, 2756, 2757, 2758, 2759, 2759, 2760, 2761, 2762, 2762, 2763, 2764, 2765, 2765,
- 2766, 2767, 2768, 2768, 2769, 2770, 2771, 2771, 2772, 2773, 2774, 2774, 2775, 2776, 2777, 2777,
- 2778, 2779, 2780, 2780, 2781, 2782, 2783, 2783, 2784, 2785, 2786, 2786, 2787, 2788, 2789, 2789,
- 2790, 2791, 2792, 2792, 2793, 2794, 2795, 2795, 2796, 2797, 2798, 2798, 2799, 2800, 2801, 2801,
- 2802, 2803, 2804, 2804, 2805, 2806, 2807, 2807, 2808, 2809, 2810, 2810, 2811, 2812, 2813, 2813,
- 2814, 2815, 2815, 2816, 2817, 2818, 2818, 2819, 2820, 2821, 2821, 2822, 2823, 2824, 2824, 2825,
- 2826, 2827, 2827, 2828, 2829, 2830, 2830, 2831, 2832, 2832, 2833, 2834, 2835, 2835, 2836, 2837,
- 2838, 2838, 2839, 2840, 2841, 2841, 2842, 2843, 2844, 2844, 2845, 2846, 2846, 2847, 2848, 2849,
- 2849, 2850, 2851, 2852, 2852, 2853, 2854, 2855, 2855, 2856, 2857, 2857, 2858, 2859, 2860, 2860,
- 2861, 2862, 2863, 2863, 2864, 2865, 2865, 2866, 2867, 2868, 2868, 2869, 2870, 2871, 2871, 2872,
- 2873, 2873, 2874, 2875, 2876, 2876, 2877, 2878, 2879, 2879, 2880, 2881, 2881, 2882, 2883, 2884,
- 2884, 2885, 2886, 2886, 2887, 2888, 2889, 2889, 2890, 2891, 2892, 2892, 2893, 2894, 2894, 2895,
- 2896, 2897, 2897, 2898, 2899, 2899, 2900, 2901, 2902, 2902, 2903, 2904, 2904, 2905, 2906, 2907,
- 2907, 2908, 2909, 2909, 2910, 2911, 2912, 2912, 2913, 2914, 2914, 2915, 2916, 2917, 2917, 2918,
- 2919, 2919, 2920, 2921, 2922, 2922, 2923, 2924, 2924, 2925, 2926, 2927, 2927, 2928, 2929, 2929,
- 2930, 2931, 2932, 2932, 2933, 2934, 2934, 2935, 2936, 2937, 2937, 2938, 2939, 2939, 2940, 2941,
- 2941, 2942, 2943, 2944, 2944, 2945, 2946, 2946, 2947, 2948, 2949, 2949, 2950, 2951, 2951, 2952,
- 2953, 2953, 2954, 2955, 2956, 2956, 2957, 2958, 2958, 2959, 2960, 2961, 2961, 2962, 2963, 2963,
- 2964, 2965, 2965, 2966, 2967, 2968, 2968, 2969, 2970, 2970, 2971, 2972, 2972, 2973, 2974, 2975,
- 2975, 2976, 2977, 2977, 2978, 2979, 2979, 2980, 2981, 2982, 2982, 2983, 2984, 2984, 2985, 2986,
- 2986, 2987, 2988, 2988, 2989, 2990, 2991, 2991, 2992, 2993, 2993, 2994, 2995, 2995, 2996, 2997,
- 2998, 2998, 2999, 3000, 3000, 3001, 3002, 3002, 3003, 3004, 3004, 3005, 3006, 3006, 3007, 3008,
- 3009, 3009, 3010, 3011, 3011, 3012, 3013, 3013, 3014, 3015, 3015, 3016, 3017, 3018, 3018, 3019,
- 3020, 3020, 3021, 3022, 3022, 3023, 3024, 3024, 3025, 3026, 3026, 3027, 3028, 3029, 3029, 3030,
- 3031, 3031, 3032, 3033, 3033, 3034, 3035, 3035, 3036, 3037, 3037, 3038, 3039, 3039, 3040, 3041,
- 3042, 3042, 3043, 3044, 3044, 3045, 3046, 3046, 3047, 3048, 3048, 3049, 3050, 3050, 3051, 3052,
- 3052, 3053, 3054, 3054, 3055, 3056, 3056, 3057, 3058, 3059, 3059, 3060, 3061, 3061, 3062, 3063,
- 3063, 3064, 3065, 3065, 3066, 3067, 3067, 3068, 3069, 3069, 3070, 3071, 3071, 3072, 3073, 3073,
- 3074, 3075, 3075, 3076, 3077, 3077, 3078, 3079, 3079, 3080, 3081, 3081, 3082, 3083, 3084, 3084,
- 3085, 3086, 3086, 3087, 3088, 3088, 3089, 3090, 3090, 3091, 3092, 3092, 3093, 3094, 3094, 3095,
- 3096, 3096, 3097, 3098, 3098, 3099, 3100, 3100, 3101, 3102, 3102, 3103, 3104, 3104, 3105, 3106,
- 3106, 3107, 3108, 3108, 3109, 3110, 3110, 3111, 3112, 3112, 3113, 3114, 3114, 3115, 3116, 3116,
- 3117, 3118, 3118, 3119, 3120, 3120, 3121, 3122, 3122, 3123, 3124, 3124, 3125, 3126, 3126, 3127,
- 3128, 3128, 3129, 3130, 3130, 3131, 3132, 3132, 3133, 3134, 3134, 3135, 3135, 3136, 3137, 3137,
- 3138, 3139, 3139, 3140, 3141, 3141, 3142, 3143, 3143, 3144, 3145, 3145, 3146, 3147, 3147, 3148,
- 3149, 3149, 3150, 3151, 3151, 3152, 3153, 3153, 3154, 3155, 3155, 3156, 3157, 3157, 3158, 3159,
- 3159, 3160, 3160, 3161, 3162, 3162, 3163, 3164, 3164, 3165, 3166, 3166, 3167, 3168, 3168, 3169,
- 3170, 3170, 3171, 3172, 3172, 3173, 3174, 3174, 3175, 3175, 3176, 3177, 3177, 3178, 3179, 3179,
- 3180, 3181, 3181, 3182, 3183, 3183, 3184, 3185, 3185, 3186, 3187, 3187, 3188, 3188, 3189, 3190,
- 3190, 3191, 3192, 3192, 3193, 3194, 3194, 3195, 3196, 3196, 3197, 3198, 3198, 3199, 3199, 3200,
- 3201, 3201, 3202, 3203, 3203, 3204, 3205, 3205, 3206, 3207, 3207, 3208, 3209, 3209, 3210, 3210,
- 3211, 3212, 3212, 3213, 3214, 3214, 3215, 3216, 3216, 3217, 3218, 3218, 3219, 3219, 3220, 3221,
- 3221, 3222, 3223, 3223, 3224, 3225, 3225, 3226, 3227, 3227, 3228, 3228, 3229, 3230, 3230, 3231,
- 3232, 3232, 3233, 3234, 3234, 3235, 3235, 3236, 3237, 3237, 3238, 3239, 3239, 3240, 3241, 3241,
- 3242, 3242, 3243, 3244, 3244, 3245, 3246, 3246, 3247, 3248, 3248, 3249, 3249, 3250, 3251, 3251,
- 3252, 3253, 3253, 3254, 3255, 3255, 3256, 3256, 3257, 3258, 3258, 3259, 3260, 3260, 3261, 3262,
- 3262, 3263, 3263, 3264, 3265, 3265, 3266, 3267, 3267, 3268, 3268, 3269, 3270, 3270, 3271, 3272,
- 3272, 3273, 3274, 3274, 3275, 3275, 3276, 3277, 3277, 3278, 3279, 3279, 3280, 3280, 3281, 3282,
- 3282, 3283, 3284, 3284, 3285, 3285, 3286, 3287, 3287, 3288, 3289, 3289, 3290, 3290, 3291, 3292,
- 3292, 3293, 3294, 3294, 3295, 3295, 3296, 3297, 3297, 3298, 3299, 3299, 3300, 3300, 3301, 3302,
- 3302, 3303, 3304, 3304, 3305, 3305, 3306, 3307, 3307, 3308, 3309, 3309, 3310, 3310, 3311, 3312,
- 3312, 3313, 3314, 3314, 3315, 3315, 3316, 3317, 3317, 3318, 3319, 3319, 3320, 3320, 3321, 3322,
- 3322, 3323, 3323, 3324, 3325, 3325, 3326, 3327, 3327, 3328, 3328, 3329, 3330, 3330, 3331, 3332,
- 3332, 3333, 3333, 3334, 3335, 3335, 3336, 3336, 3337, 3338, 3338, 3339, 3340, 3340, 3341, 3341,
- 3342, 3343, 3343, 3344, 3345, 3345, 3346, 3346, 3347, 3348, 3348, 3349, 3349, 3350, 3351, 3351,
- 3352, 3352, 3353, 3354, 3354, 3355, 3356, 3356, 3357, 3357, 3358, 3359, 3359, 3360, 3360, 3361,
- 3362, 3362, 3363, 3364, 3364, 3365, 3365, 3366, 3367, 3367, 3368, 3368, 3369, 3370, 3370, 3371,
- 3371, 3372, 3373, 3373, 3374, 3375, 3375, 3376, 3376, 3377, 3378, 3378, 3379, 3379, 3380, 3381,
- 3381, 3382, 3382, 3383, 3384, 3384, 3385, 3385, 3386, 3387, 3387, 3388, 3389, 3389, 3390, 3390,
- 3391, 3392, 3392, 3393, 3393, 3394, 3395, 3395, 3396, 3396, 3397, 3398, 3398, 3399, 3399, 3400,
- 3401, 3401, 3402, 3402, 3403, 3404, 3404, 3405, 3405, 3406, 3407, 3407, 3408, 3408, 3409, 3410,
- 3410, 3411, 3411, 3412, 3413, 3413, 3414, 3414, 3415, 3416, 3416, 3417, 3418, 3418, 3419, 3419,
- 3420, 3421, 3421, 3422, 3422, 3423, 3424, 3424, 3425, 3425, 3426, 3427, 3427, 3428, 3428, 3429,
- 3430, 3430, 3431, 3431, 3432, 3433, 3433, 3434, 3434, 3435, 3435, 3436, 3437, 3437, 3438, 3438,
- 3439, 3440, 3440, 3441, 3441, 3442, 3443, 3443, 3444, 3444, 3445, 3446, 3446, 3447, 3447, 3448,
- 3449, 3449, 3450, 3450, 3451, 3452, 3452, 3453, 3453, 3454, 3455, 3455, 3456, 3456, 3457, 3458,
- 3458, 3459, 3459, 3460, 3461, 3461, 3462, 3462, 3463, 3463, 3464, 3465, 3465, 3466, 3466, 3467,
- 3468, 3468, 3469, 3469, 3470, 3471, 3471, 3472, 3472, 3473, 3474, 3474, 3475, 3475, 3476, 3476,
- 3477, 3478, 3478, 3479, 3479, 3480, 3481, 3481, 3482, 3482, 3483, 3484, 3484, 3485, 3485, 3486,
- 3486, 3487, 3488, 3488, 3489, 3489, 3490, 3491, 3491, 3492, 3492, 3493, 3494, 3494, 3495, 3495,
- 3496, 3496, 3497, 3498, 3498, 3499, 3499, 3500, 3501, 3501, 3502, 3502, 3503, 3504, 3504, 3505,
- 3505, 3506, 3506, 3507, 3508, 3508, 3509, 3509, 3510, 3511, 3511, 3512, 3512, 3513, 3513, 3514,
- 3515, 3515, 3516, 3516, 3517, 3518, 3518, 3519, 3519, 3520, 3520, 3521, 3522, 3522, 3523, 3523,
- 3524, 3525, 3525, 3526, 3526, 3527, 3527, 3528, 3529, 3529, 3530, 3530, 3531, 3531, 3532, 3533,
- 3533, 3534, 3534, 3535, 3536, 3536, 3537, 3537, 3538, 3538, 3539, 3540, 3540, 3541, 3541, 3542,
- 3542, 3543, 3544, 3544, 3545, 3545, 3546, 3547, 3547, 3548, 3548, 3549, 3549, 3550, 3551, 3551,
- 3552, 3552, 3553, 3553, 3554, 3555, 3555, 3556, 3556, 3557, 3557, 3558, 3559, 3559, 3560, 3560,
- 3561, 3561, 3562, 3563, 3563, 3564, 3564, 3565, 3566, 3566, 3567, 3567, 3568, 3568, 3569, 3570,
- 3570, 3571, 3571, 3572, 3572, 3573, 3574, 3574, 3575, 3575, 3576, 3576, 3577, 3578, 3578, 3579,
- 3579, 3580, 3580, 3581, 3582, 3582, 3583, 3583, 3584, 3584, 3585, 3586, 3586, 3587, 3587, 3588,
- 3588, 3589, 3590, 3590, 3591, 3591, 3592, 3592, 3593, 3594, 3594, 3595, 3595, 3596, 3596, 3597,
- 3597, 3598, 3599, 3599, 3600, 3600, 3601, 3601, 3602, 3603, 3603, 3604, 3604, 3605, 3605, 3606,
- 3607, 3607, 3608, 3608, 3609, 3609, 3610, 3611, 3611, 3612, 3612, 3613, 3613, 3614, 3615, 3615,
- 3616, 3616, 3617, 3617, 3618, 3618, 3619, 3620, 3620, 3621, 3621, 3622, 3622, 3623, 3624, 3624,
- 3625, 3625, 3626, 3626, 3627, 3627, 3628, 3629, 3629, 3630, 3630, 3631, 3631, 3632, 3633, 3633,
- 3634, 3634, 3635, 3635, 3636, 3636, 3637, 3638, 3638, 3639, 3639, 3640, 3640, 3641, 3642, 3642,
- 3643, 3643, 3644, 3644, 3645, 3645, 3646, 3647, 3647, 3648, 3648, 3649, 3649, 3650, 3650, 3651,
- 3652, 3652, 3653, 3653, 3654, 3654, 3655, 3656, 3656, 3657, 3657, 3658, 3658, 3659, 3659, 3660,
- 3661, 3661, 3662, 3662, 3663, 3663, 3664, 3664, 3665, 3666, 3666, 3667, 3667, 3668, 3668, 3669,
- 3669, 3670, 3671, 3671, 3672, 3672, 3673, 3673, 3674, 3674, 3675, 3676, 3676, 3677, 3677, 3678,
- 3678, 3679, 3679, 3680, 3681, 3681, 3682, 3682, 3683, 3683, 3684, 3684, 3685, 3686, 3686, 3687,
- 3687, 3688, 3688, 3689, 3689, 3690, 3691, 3691, 3692, 3692, 3693, 3693, 3694, 3694, 3695, 3695,
- 3696, 3697, 3697, 3698, 3698, 3699, 3699, 3700, 3700, 3701, 3702, 3702, 3703, 3703, 3704, 3704,
- 3705, 3705, 3706, 3707, 3707, 3708, 3708, 3709, 3709, 3710, 3710, 3711, 3711, 3712, 3713, 3713,
- 3714, 3714, 3715, 3715, 3716, 3716, 3717, 3717, 3718, 3719, 3719, 3720, 3720, 3721, 3721, 3722,
- 3722, 3723, 3724, 3724, 3725, 3725, 3726, 3726, 3727, 3727, 3728, 3728, 3729, 3730, 3730, 3731,
- 3731, 3732, 3732, 3733, 3733, 3734, 3734, 3735, 3736, 3736, 3737, 3737, 3738, 3738, 3739, 3739,
- 3740, 3740, 3741, 3742, 3742, 3743, 3743, 3744, 3744, 3745, 3745, 3746, 3746, 3747, 3748, 3748,
- 3749, 3749, 3750, 3750, 3751, 3751, 3752, 3752, 3753, 3753, 3754, 3755, 3755, 3756, 3756, 3757,
- 3757, 3758, 3758, 3759, 3759, 3760, 3761, 3761, 3762, 3762, 3763, 3763, 3764, 3764, 3765, 3765,
- 3766, 3766, 3767, 3768, 3768, 3769, 3769, 3770, 3770, 3771, 3771, 3772, 3772, 3773, 3773, 3774,
- 3775, 3775, 3776, 3776, 3777, 3777, 3778, 3778, 3779, 3779, 3780, 3781, 3781, 3782, 3782, 3783,
- 3783, 3784, 3784, 3785, 3785, 3786, 3786, 3787, 3787, 3788, 3789, 3789, 3790, 3790, 3791, 3791,
- 3792, 3792, 3793, 3793, 3794, 3794, 3795, 3796, 3796, 3797, 3797, 3798, 3798, 3799, 3799, 3800,
- 3800, 3801, 3801, 3802, 3802, 3803, 3804, 3804, 3805, 3805, 3806, 3806, 3807, 3807, 3808, 3808,
- 3809, 3809, 3810, 3811, 3811, 3812, 3812, 3813, 3813, 3814, 3814, 3815, 3815, 3816, 3816, 3817,
- 3817, 3818, 3819, 3819, 3820, 3820, 3821, 3821, 3822, 3822, 3823, 3823, 3824, 3824, 3825, 3825,
- 3826, 3826, 3827, 3828, 3828, 3829, 3829, 3830, 3830, 3831, 3831, 3832, 3832, 3833, 3833, 3834,
- 3834, 3835, 3835, 3836, 3837, 3837, 3838, 3838, 3839, 3839, 3840, 3840, 3841, 3841, 3842, 3842,
- 3843, 3843, 3844, 3844, 3845, 3846, 3846, 3847, 3847, 3848, 3848, 3849, 3849, 3850, 3850, 3851,
- 3851, 3852, 3852, 3853, 3853, 3854, 3855, 3855, 3856, 3856, 3857, 3857, 3858, 3858, 3859, 3859,
- 3860, 3860, 3861, 3861, 3862, 3862, 3863, 3863, 3864, 3864, 3865, 3866, 3866, 3867, 3867, 3868,
- 3868, 3869, 3869, 3870, 3870, 3871, 3871, 3872, 3872, 3873, 3873, 3874, 3874, 3875, 3876, 3876,
- 3877, 3877, 3878, 3878, 3879, 3879, 3880, 3880, 3881, 3881, 3882, 3882, 3883, 3883, 3884, 3884,
- 3885, 3885, 3886, 3886, 3887, 3888, 3888, 3889, 3889, 3890, 3890, 3891, 3891, 3892, 3892, 3893,
- 3893, 3894, 3894, 3895, 3895, 3896, 3896, 3897, 3897, 3898, 3898, 3899, 3900, 3900, 3901, 3901,
- 3902, 3902, 3903, 3903, 3904, 3904, 3905, 3905, 3906, 3906, 3907, 3907, 3908, 3908, 3909, 3909,
- 3910, 3910, 3911, 3911, 3912, 3912, 3913, 3914, 3914, 3915, 3915, 3916, 3916, 3917, 3917, 3918,
- 3918, 3919, 3919, 3920, 3920, 3921, 3921, 3922, 3922, 3923, 3923, 3924, 3924, 3925, 3925, 3926,
- 3926, 3927, 3927, 3928, 3929, 3929, 3930, 3930, 3931, 3931, 3932, 3932, 3933, 3933, 3934, 3934,
- 3935, 3935, 3936, 3936, 3937, 3937, 3938, 3938, 3939, 3939, 3940, 3940, 3941, 3941, 3942, 3942,
- 3943, 3943, 3944, 3944, 3945, 3945, 3946, 3947, 3947, 3948, 3948, 3949, 3949, 3950, 3950, 3951,
- 3951, 3952, 3952, 3953, 3953, 3954, 3954, 3955, 3955, 3956, 3956, 3957, 3957, 3958, 3958, 3959,
- 3959, 3960, 3960, 3961, 3961, 3962, 3962, 3963, 3963, 3964, 3964, 3965, 3965, 3966, 3966, 3967,
- 3967, 3968, 3969, 3969, 3970, 3970, 3971, 3971, 3972, 3972, 3973, 3973, 3974, 3974, 3975, 3975,
- 3976, 3976, 3977, 3977, 3978, 3978, 3979, 3979, 3980, 3980, 3981, 3981, 3982, 3982, 3983, 3983,
- 3984, 3984, 3985, 3985, 3986, 3986, 3987, 3987, 3988, 3988, 3989, 3989, 3990, 3990, 3991, 3991,
- 3992, 3992, 3993, 3993, 3994, 3994, 3995, 3995, 3996, 3996, 3997, 3997, 3998, 3998, 3999, 3999,
- 4000, 4001, 4001, 4002, 4002, 4003, 4003, 4004, 4004, 4005, 4005, 4006, 4006, 4007, 4007, 4008,
- 4008, 4009, 4009, 4010, 4010, 4011, 4011, 4012, 4012, 4013, 4013, 4014, 4014, 4015, 4015, 4016,
- 4016, 4017, 4017, 4018, 4018, 4019, 4019, 4020, 4020, 4021, 4021, 4022, 4022, 4023, 4023, 4024,
- 4024, 4025, 4025, 4026, 4026, 4027, 4027, 4028, 4028, 4029, 4029, 4030, 4030, 4031, 4031, 4032,
- 4032, 4033, 4033, 4034, 4034, 4035, 4035, 4036, 4036, 4037, 4037, 4038, 4038, 4039, 4039, 4040,
- 4040, 4041, 4041, 4042, 4042, 4043, 4043, 4044, 4044, 4045, 4045, 4046, 4046, 4047, 4047, 4048,
- 4048, 4049, 4049, 4050, 4050, 4051, 4051, 4052, 4052, 4053, 4053, 4054, 4054, 4055, 4055, 4056,
- 4056, 4057, 4057, 4058, 4058, 4059, 4059, 4060, 4060, 4061, 4061, 4062, 4062, 4063, 4063, 4064,
- 4064, 4065, 4065, 4066, 4066, 4067, 4067, 4068, 4068, 4069, 4069, 4070, 4070, 4071, 4071, 4072,
- 4072, 4073, 4073, 4074, 4074, 4075, 4075, 4076, 4076, 4077, 4077, 4078, 4078, 4079, 4079, 4080,
- 4080,
-};
-
-/* Generated table */
-const struct color16 tpg_csc_colors[V4L2_COLORSPACE_DCI_P3 + 1][V4L2_XFER_FUNC_SMPTE2084 + 1][TPG_COLOR_CSC_BLACK + 1] = {
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_709][0] = { 2939, 2939, 2939 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_709][1] = { 2953, 2963, 586 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_709][2] = { 0, 2967, 2937 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_709][3] = { 88, 2990, 575 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_709][4] = { 3016, 259, 2933 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_709][5] = { 3030, 405, 558 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_709][6] = { 478, 428, 2931 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_709][7] = { 547, 547, 547 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SRGB][0] = { 3056, 3056, 3056 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SRGB][1] = { 3068, 3077, 838 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SRGB][2] = { 0, 3081, 3053 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SRGB][3] = { 241, 3102, 828 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SRGB][4] = { 3126, 504, 3050 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SRGB][5] = { 3138, 657, 810 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SRGB][6] = { 731, 680, 3048 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SRGB][7] = { 800, 799, 800 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_ADOBERGB][0] = { 3033, 3033, 3033 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_ADOBERGB][1] = { 3046, 3054, 886 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_ADOBERGB][2] = { 0, 3058, 3031 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_ADOBERGB][3] = { 360, 3079, 877 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_ADOBERGB][4] = { 3103, 587, 3027 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_ADOBERGB][5] = { 3116, 723, 861 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_ADOBERGB][6] = { 789, 744, 3025 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_ADOBERGB][7] = { 851, 851, 851 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE240M][0] = { 2926, 2926, 2926 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE240M][1] = { 2941, 2950, 546 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE240M][2] = { 0, 2954, 2924 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE240M][3] = { 78, 2978, 536 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE240M][4] = { 3004, 230, 2920 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE240M][5] = { 3018, 363, 518 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE240M][6] = { 437, 387, 2918 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE240M][7] = { 507, 507, 507 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_NONE][0] = { 2125, 2125, 2125 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_NONE][1] = { 2145, 2159, 142 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_NONE][2] = { 0, 2164, 2122 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_NONE][3] = { 19, 2198, 138 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_NONE][4] = { 2236, 57, 2116 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_NONE][5] = { 2256, 90, 133 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_NONE][6] = { 110, 96, 2113 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_NONE][7] = { 130, 130, 130 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_DCI_P3][0] = { 3175, 3175, 3175 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_DCI_P3][1] = { 3186, 3194, 1121 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_DCI_P3][2] = { 0, 3197, 3173 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_DCI_P3][3] = { 523, 3216, 1112 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_DCI_P3][4] = { 3237, 792, 3169 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_DCI_P3][5] = { 3248, 944, 1094 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_DCI_P3][6] = { 1017, 967, 3168 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_DCI_P3][7] = { 1084, 1084, 1084 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE2084][0] = { 3798, 3798, 3798 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE2084][1] = { 3802, 3805, 2602 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE2084][2] = { 0, 3806, 3797 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE2084][3] = { 1780, 3812, 2592 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE2084][4] = { 3820, 2215, 3796 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE2084][5] = { 3824, 2409, 2574 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE2084][6] = { 2491, 2435, 3795 },
- [V4L2_COLORSPACE_SMPTE170M][V4L2_XFER_FUNC_SMPTE2084][7] = { 2563, 2563, 2563 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_709][0] = { 2939, 2939, 2939 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_709][1] = { 2953, 2963, 586 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_709][2] = { 0, 2967, 2937 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_709][3] = { 88, 2990, 575 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_709][4] = { 3016, 259, 2933 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_709][5] = { 3030, 405, 558 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_709][6] = { 478, 428, 2931 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_709][7] = { 547, 547, 547 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SRGB][0] = { 3056, 3056, 3056 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SRGB][1] = { 3068, 3077, 838 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SRGB][2] = { 0, 3081, 3053 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SRGB][3] = { 241, 3102, 828 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SRGB][4] = { 3126, 504, 3050 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SRGB][5] = { 3138, 657, 810 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SRGB][6] = { 731, 680, 3048 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SRGB][7] = { 800, 799, 800 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_ADOBERGB][0] = { 3033, 3033, 3033 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_ADOBERGB][1] = { 3046, 3054, 886 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_ADOBERGB][2] = { 0, 3058, 3031 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_ADOBERGB][3] = { 360, 3079, 877 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_ADOBERGB][4] = { 3103, 587, 3027 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_ADOBERGB][5] = { 3116, 723, 861 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_ADOBERGB][6] = { 789, 744, 3025 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_ADOBERGB][7] = { 851, 851, 851 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE240M][0] = { 2926, 2926, 2926 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE240M][1] = { 2941, 2950, 546 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE240M][2] = { 0, 2954, 2924 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE240M][3] = { 78, 2978, 536 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE240M][4] = { 3004, 230, 2920 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE240M][5] = { 3018, 363, 518 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE240M][6] = { 437, 387, 2918 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE240M][7] = { 507, 507, 507 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_NONE][0] = { 2125, 2125, 2125 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_NONE][1] = { 2145, 2159, 142 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_NONE][2] = { 0, 2164, 2122 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_NONE][3] = { 19, 2198, 138 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_NONE][4] = { 2236, 57, 2116 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_NONE][5] = { 2256, 90, 133 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_NONE][6] = { 110, 96, 2113 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_NONE][7] = { 130, 130, 130 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_DCI_P3][0] = { 3175, 3175, 3175 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_DCI_P3][1] = { 3186, 3194, 1121 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_DCI_P3][2] = { 0, 3197, 3173 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_DCI_P3][3] = { 523, 3216, 1112 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_DCI_P3][4] = { 3237, 792, 3169 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_DCI_P3][5] = { 3248, 944, 1094 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_DCI_P3][6] = { 1017, 967, 3168 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_DCI_P3][7] = { 1084, 1084, 1084 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE2084][0] = { 3798, 3798, 3798 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE2084][1] = { 3802, 3805, 2602 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE2084][2] = { 0, 3806, 3797 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE2084][3] = { 1780, 3812, 2592 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE2084][4] = { 3820, 2215, 3796 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE2084][5] = { 3824, 2409, 2574 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE2084][6] = { 2491, 2435, 3795 },
- [V4L2_COLORSPACE_SMPTE240M][V4L2_XFER_FUNC_SMPTE2084][7] = { 2563, 2563, 2563 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_709][0] = { 2939, 2939, 2939 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_709][1] = { 2939, 2939, 547 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_709][2] = { 547, 2939, 2939 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_709][3] = { 547, 2939, 547 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_709][4] = { 2939, 547, 2939 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_709][5] = { 2939, 547, 547 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_709][6] = { 547, 547, 2939 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_709][7] = { 547, 547, 547 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SRGB][0] = { 3056, 3056, 3056 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SRGB][1] = { 3056, 3056, 800 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SRGB][2] = { 800, 3056, 3056 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SRGB][3] = { 800, 3056, 800 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SRGB][4] = { 3056, 800, 3056 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SRGB][5] = { 3056, 800, 800 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SRGB][6] = { 800, 800, 3056 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SRGB][7] = { 800, 800, 800 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_ADOBERGB][0] = { 3033, 3033, 3033 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_ADOBERGB][1] = { 3033, 3033, 851 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_ADOBERGB][2] = { 851, 3033, 3033 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_ADOBERGB][3] = { 851, 3033, 851 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_ADOBERGB][4] = { 3033, 851, 3033 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_ADOBERGB][5] = { 3033, 851, 851 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_ADOBERGB][6] = { 851, 851, 3033 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_ADOBERGB][7] = { 851, 851, 851 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE240M][0] = { 2926, 2926, 2926 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE240M][1] = { 2926, 2926, 507 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE240M][2] = { 507, 2926, 2926 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE240M][3] = { 507, 2926, 507 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE240M][4] = { 2926, 507, 2926 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE240M][5] = { 2926, 507, 507 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE240M][6] = { 507, 507, 2926 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE240M][7] = { 507, 507, 507 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_NONE][0] = { 2125, 2125, 2125 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_NONE][1] = { 2125, 2125, 130 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_NONE][2] = { 130, 2125, 2125 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_NONE][3] = { 130, 2125, 130 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_NONE][4] = { 2125, 130, 2125 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_NONE][5] = { 2125, 130, 130 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_NONE][6] = { 130, 130, 2125 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_NONE][7] = { 130, 130, 130 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_DCI_P3][0] = { 3175, 3175, 3175 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_DCI_P3][1] = { 3175, 3175, 1084 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_DCI_P3][2] = { 1084, 3175, 3175 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_DCI_P3][3] = { 1084, 3175, 1084 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_DCI_P3][4] = { 3175, 1084, 3175 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_DCI_P3][5] = { 3175, 1084, 1084 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_DCI_P3][6] = { 1084, 1084, 3175 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_DCI_P3][7] = { 1084, 1084, 1084 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE2084][0] = { 3798, 3798, 3798 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE2084][1] = { 3798, 3798, 2563 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE2084][2] = { 2563, 3798, 3798 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE2084][3] = { 2563, 3798, 2563 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE2084][4] = { 3798, 2563, 3798 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE2084][5] = { 3798, 2563, 2563 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE2084][6] = { 2563, 2563, 3798 },
- [V4L2_COLORSPACE_REC709][V4L2_XFER_FUNC_SMPTE2084][7] = { 2563, 2563, 2563 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_709][0] = { 2939, 2939, 2939 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_709][1] = { 2892, 3034, 910 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_709][2] = { 1715, 2916, 2914 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_709][3] = { 1631, 3012, 828 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_709][4] = { 2497, 119, 2867 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_709][5] = { 2440, 649, 657 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_709][6] = { 740, 0, 2841 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_709][7] = { 547, 547, 547 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SRGB][0] = { 3056, 3055, 3056 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SRGB][1] = { 3013, 3142, 1157 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SRGB][2] = { 1926, 3034, 3032 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SRGB][3] = { 1847, 3121, 1076 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SRGB][4] = { 2651, 304, 2990 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SRGB][5] = { 2599, 901, 909 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SRGB][6] = { 991, 0, 2966 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SRGB][7] = { 800, 799, 800 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_ADOBERGB][0] = { 3033, 3033, 3033 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_ADOBERGB][1] = { 2989, 3120, 1180 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_ADOBERGB][2] = { 1913, 3011, 3009 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_ADOBERGB][3] = { 1836, 3099, 1105 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_ADOBERGB][4] = { 2627, 413, 2966 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_ADOBERGB][5] = { 2576, 943, 951 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_ADOBERGB][6] = { 1026, 0, 2942 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_ADOBERGB][7] = { 851, 851, 851 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE240M][0] = { 2926, 2926, 2926 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE240M][1] = { 2879, 3022, 874 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE240M][2] = { 1688, 2903, 2901 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE240M][3] = { 1603, 2999, 791 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE240M][4] = { 2479, 106, 2853 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE240M][5] = { 2422, 610, 618 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE240M][6] = { 702, 0, 2827 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE240M][7] = { 507, 507, 507 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_NONE][0] = { 2125, 2125, 2125 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_NONE][1] = { 2059, 2262, 266 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_NONE][2] = { 771, 2092, 2089 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_NONE][3] = { 705, 2229, 231 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_NONE][4] = { 1550, 26, 2024 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_NONE][5] = { 1484, 163, 165 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_NONE][6] = { 196, 0, 1988 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_NONE][7] = { 130, 130, 130 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_DCI_P3][0] = { 3175, 3175, 3175 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_DCI_P3][1] = { 3136, 3251, 1429 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_DCI_P3][2] = { 2150, 3156, 3154 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_DCI_P3][3] = { 2077, 3233, 1352 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_DCI_P3][4] = { 2812, 589, 3116 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_DCI_P3][5] = { 2765, 1182, 1190 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_DCI_P3][6] = { 1270, 0, 3094 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_DCI_P3][7] = { 1084, 1084, 1084 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE2084][0] = { 3798, 3798, 3798 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE2084][1] = { 3784, 3825, 2879 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE2084][2] = { 3351, 3791, 3790 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE2084][3] = { 3311, 3819, 2815 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE2084][4] = { 3659, 1900, 3777 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE2084][5] = { 3640, 2662, 2669 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE2084][6] = { 2743, 0, 3769 },
- [V4L2_COLORSPACE_470_SYSTEM_M][V4L2_XFER_FUNC_SMPTE2084][7] = { 2563, 2563, 2563 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_709][0] = { 2939, 2939, 2939 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_709][1] = { 2939, 2939, 464 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_709][2] = { 786, 2939, 2939 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_709][3] = { 786, 2939, 464 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_709][4] = { 2879, 547, 2956 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_709][5] = { 2879, 547, 547 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_709][6] = { 547, 547, 2956 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_709][7] = { 547, 547, 547 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SRGB][0] = { 3056, 3056, 3056 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SRGB][1] = { 3056, 3056, 717 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SRGB][2] = { 1036, 3056, 3056 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SRGB][3] = { 1036, 3056, 717 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SRGB][4] = { 3001, 800, 3071 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SRGB][5] = { 3001, 800, 799 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SRGB][6] = { 800, 800, 3071 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SRGB][7] = { 800, 800, 799 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_ADOBERGB][0] = { 3033, 3033, 3033 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_ADOBERGB][1] = { 3033, 3033, 776 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_ADOBERGB][2] = { 1068, 3033, 3033 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_ADOBERGB][3] = { 1068, 3033, 776 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_ADOBERGB][4] = { 2977, 851, 3048 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_ADOBERGB][5] = { 2977, 851, 851 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_ADOBERGB][6] = { 851, 851, 3048 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_ADOBERGB][7] = { 851, 851, 851 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE240M][0] = { 2926, 2926, 2926 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE240M][1] = { 2926, 2926, 423 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE240M][2] = { 749, 2926, 2926 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE240M][3] = { 749, 2926, 423 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE240M][4] = { 2865, 507, 2943 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE240M][5] = { 2865, 507, 507 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE240M][6] = { 507, 507, 2943 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE240M][7] = { 507, 507, 507 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_NONE][0] = { 2125, 2125, 2125 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_NONE][1] = { 2125, 2125, 106 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_NONE][2] = { 214, 2125, 2125 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_NONE][3] = { 214, 2125, 106 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_NONE][4] = { 2041, 130, 2149 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_NONE][5] = { 2041, 130, 130 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_NONE][6] = { 130, 130, 2149 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_NONE][7] = { 130, 130, 130 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_DCI_P3][0] = { 3175, 3175, 3175 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_DCI_P3][1] = { 3175, 3175, 1003 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_DCI_P3][2] = { 1313, 3175, 3175 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_DCI_P3][3] = { 1313, 3175, 1003 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_DCI_P3][4] = { 3126, 1084, 3188 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_DCI_P3][5] = { 3126, 1084, 1084 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_DCI_P3][6] = { 1084, 1084, 3188 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_DCI_P3][7] = { 1084, 1084, 1084 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE2084][0] = { 3798, 3798, 3798 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE2084][1] = { 3798, 3798, 2476 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE2084][2] = { 2782, 3798, 3798 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE2084][3] = { 2782, 3798, 2476 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE2084][4] = { 3780, 2563, 3803 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE2084][5] = { 3780, 2563, 2563 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE2084][6] = { 2563, 2563, 3803 },
- [V4L2_COLORSPACE_470_SYSTEM_BG][V4L2_XFER_FUNC_SMPTE2084][7] = { 2563, 2563, 2563 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_709][0] = { 2939, 2939, 2939 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_709][1] = { 2939, 2939, 547 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_709][2] = { 547, 2939, 2939 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_709][3] = { 547, 2939, 547 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_709][4] = { 2939, 547, 2939 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_709][5] = { 2939, 547, 547 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_709][6] = { 547, 547, 2939 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_709][7] = { 547, 547, 547 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SRGB][0] = { 3056, 3056, 3056 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SRGB][1] = { 3056, 3056, 800 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SRGB][2] = { 800, 3056, 3056 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SRGB][3] = { 800, 3056, 800 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SRGB][4] = { 3056, 800, 3056 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SRGB][5] = { 3056, 800, 800 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SRGB][6] = { 800, 800, 3056 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SRGB][7] = { 800, 800, 800 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_ADOBERGB][0] = { 3033, 3033, 3033 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_ADOBERGB][1] = { 3033, 3033, 851 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_ADOBERGB][2] = { 851, 3033, 3033 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_ADOBERGB][3] = { 851, 3033, 851 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_ADOBERGB][4] = { 3033, 851, 3033 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_ADOBERGB][5] = { 3033, 851, 851 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_ADOBERGB][6] = { 851, 851, 3033 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_ADOBERGB][7] = { 851, 851, 851 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE240M][0] = { 2926, 2926, 2926 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE240M][1] = { 2926, 2926, 507 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE240M][2] = { 507, 2926, 2926 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE240M][3] = { 507, 2926, 507 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE240M][4] = { 2926, 507, 2926 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE240M][5] = { 2926, 507, 507 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE240M][6] = { 507, 507, 2926 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE240M][7] = { 507, 507, 507 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_NONE][0] = { 2125, 2125, 2125 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_NONE][1] = { 2125, 2125, 130 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_NONE][2] = { 130, 2125, 2125 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_NONE][3] = { 130, 2125, 130 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_NONE][4] = { 2125, 130, 2125 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_NONE][5] = { 2125, 130, 130 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_NONE][6] = { 130, 130, 2125 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_NONE][7] = { 130, 130, 130 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_DCI_P3][0] = { 3175, 3175, 3175 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_DCI_P3][1] = { 3175, 3175, 1084 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_DCI_P3][2] = { 1084, 3175, 3175 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_DCI_P3][3] = { 1084, 3175, 1084 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_DCI_P3][4] = { 3175, 1084, 3175 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_DCI_P3][5] = { 3175, 1084, 1084 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_DCI_P3][6] = { 1084, 1084, 3175 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_DCI_P3][7] = { 1084, 1084, 1084 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE2084][0] = { 3798, 3798, 3798 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE2084][1] = { 3798, 3798, 2563 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE2084][2] = { 2563, 3798, 3798 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE2084][3] = { 2563, 3798, 2563 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE2084][4] = { 3798, 2563, 3798 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE2084][5] = { 3798, 2563, 2563 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE2084][6] = { 2563, 2563, 3798 },
- [V4L2_COLORSPACE_SRGB][V4L2_XFER_FUNC_SMPTE2084][7] = { 2563, 2563, 2563 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_709][0] = { 2939, 2939, 2939 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_709][1] = { 2939, 2939, 781 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_709][2] = { 1622, 2939, 2939 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_709][3] = { 1622, 2939, 781 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_709][4] = { 2502, 547, 2881 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_709][5] = { 2502, 547, 547 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_709][6] = { 547, 547, 2881 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_709][7] = { 547, 547, 547 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SRGB][0] = { 3056, 3056, 3056 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SRGB][1] = { 3056, 3056, 1031 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SRGB][2] = { 1838, 3056, 3056 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SRGB][3] = { 1838, 3056, 1031 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SRGB][4] = { 2657, 800, 3002 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SRGB][5] = { 2657, 800, 800 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SRGB][6] = { 800, 800, 3002 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SRGB][7] = { 800, 800, 800 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_ADOBERGB][0] = { 3033, 3033, 3033 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_ADOBERGB][1] = { 3033, 3033, 1063 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_ADOBERGB][2] = { 1828, 3033, 3033 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_ADOBERGB][3] = { 1828, 3033, 1063 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_ADOBERGB][4] = { 2633, 851, 2979 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_ADOBERGB][5] = { 2633, 851, 851 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_ADOBERGB][6] = { 851, 851, 2979 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_ADOBERGB][7] = { 851, 851, 851 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE240M][0] = { 2926, 2926, 2926 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE240M][1] = { 2926, 2926, 744 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE240M][2] = { 1594, 2926, 2926 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE240M][3] = { 1594, 2926, 744 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE240M][4] = { 2484, 507, 2867 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE240M][5] = { 2484, 507, 507 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE240M][6] = { 507, 507, 2867 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE240M][7] = { 507, 507, 507 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_NONE][0] = { 2125, 2125, 2125 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_NONE][1] = { 2125, 2125, 212 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_NONE][2] = { 698, 2125, 2125 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_NONE][3] = { 698, 2125, 212 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_NONE][4] = { 1557, 130, 2043 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_NONE][5] = { 1557, 130, 130 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_NONE][6] = { 130, 130, 2043 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_NONE][7] = { 130, 130, 130 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_DCI_P3][0] = { 3175, 3175, 3175 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_DCI_P3][1] = { 3175, 3175, 1308 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_DCI_P3][2] = { 2069, 3175, 3175 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_DCI_P3][3] = { 2069, 3175, 1308 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_DCI_P3][4] = { 2816, 1084, 3127 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_DCI_P3][5] = { 2816, 1084, 1084 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_DCI_P3][6] = { 1084, 1084, 3127 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_DCI_P3][7] = { 1084, 1084, 1084 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE2084][0] = { 3798, 3798, 3798 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE2084][1] = { 3798, 3798, 2778 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE2084][2] = { 3306, 3798, 3798 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE2084][3] = { 3306, 3798, 2778 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE2084][4] = { 3661, 2563, 3781 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE2084][5] = { 3661, 2563, 2563 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE2084][6] = { 2563, 2563, 3781 },
- [V4L2_COLORSPACE_ADOBERGB][V4L2_XFER_FUNC_SMPTE2084][7] = { 2563, 2563, 2563 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_709][0] = { 2939, 2939, 2939 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_709][1] = { 2877, 2923, 1058 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_709][2] = { 1837, 2840, 2916 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_709][3] = { 1734, 2823, 993 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_709][4] = { 2427, 961, 2812 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_709][5] = { 2351, 912, 648 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_709][6] = { 792, 618, 2788 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_709][7] = { 547, 547, 547 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SRGB][0] = { 3056, 3056, 3056 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SRGB][1] = { 2999, 3041, 1301 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SRGB][2] = { 2040, 2965, 3034 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SRGB][3] = { 1944, 2950, 1238 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SRGB][4] = { 2587, 1207, 2940 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SRGB][5] = { 2517, 1159, 900 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SRGB][6] = { 1042, 870, 2917 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SRGB][7] = { 800, 800, 800 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_ADOBERGB][0] = { 3033, 3033, 3033 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_ADOBERGB][1] = { 2976, 3018, 1315 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_ADOBERGB][2] = { 2024, 2942, 3011 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_ADOBERGB][3] = { 1930, 2926, 1256 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_ADOBERGB][4] = { 2563, 1227, 2916 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_ADOBERGB][5] = { 2494, 1183, 943 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_ADOBERGB][6] = { 1073, 916, 2894 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_ADOBERGB][7] = { 851, 851, 851 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE240M][0] = { 2926, 2926, 2926 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE240M][1] = { 2864, 2910, 1024 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE240M][2] = { 1811, 2826, 2903 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE240M][3] = { 1707, 2809, 958 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE240M][4] = { 2408, 926, 2798 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE240M][5] = { 2331, 876, 609 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE240M][6] = { 755, 579, 2773 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE240M][7] = { 507, 507, 507 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_NONE][0] = { 2125, 2125, 2125 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_NONE][1] = { 2039, 2102, 338 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_NONE][2] = { 873, 1987, 2092 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_NONE][3] = { 787, 1965, 305 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_NONE][4] = { 1468, 290, 1949 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_NONE][5] = { 1382, 268, 162 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_NONE][6] = { 216, 152, 1917 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_NONE][7] = { 130, 130, 130 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_DCI_P3][0] = { 3175, 3175, 3175 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_DCI_P3][1] = { 3124, 3161, 1566 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_DCI_P3][2] = { 2255, 3094, 3156 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_DCI_P3][3] = { 2166, 3080, 1506 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_DCI_P3][4] = { 2754, 1477, 3071 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_DCI_P3][5] = { 2690, 1431, 1182 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_DCI_P3][6] = { 1318, 1153, 3051 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_DCI_P3][7] = { 1084, 1084, 1084 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE2084][0] = { 3798, 3798, 3798 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE2084][1] = { 3780, 3793, 2984 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE2084][2] = { 3406, 3768, 3791 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE2084][3] = { 3359, 3763, 2939 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE2084][4] = { 3636, 2916, 3760 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE2084][5] = { 3609, 2880, 2661 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE2084][6] = { 2786, 2633, 3753 },
- [V4L2_COLORSPACE_BT2020][V4L2_XFER_FUNC_SMPTE2084][7] = { 2563, 2563, 2563 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_709][0] = { 2939, 2939, 2939 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_709][1] = { 2936, 2934, 992 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_709][2] = { 1159, 2890, 2916 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_709][3] = { 1150, 2885, 921 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_709][4] = { 2751, 766, 2837 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_709][5] = { 2747, 747, 650 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_709][6] = { 563, 570, 2812 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_709][7] = { 547, 547, 547 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SRGB][0] = { 3056, 3056, 3055 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SRGB][1] = { 3052, 3051, 1237 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SRGB][2] = { 1397, 3011, 3034 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SRGB][3] = { 1389, 3006, 1168 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SRGB][4] = { 2884, 1016, 2962 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SRGB][5] = { 2880, 998, 902 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SRGB][6] = { 816, 823, 2940 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SRGB][7] = { 800, 800, 799 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_ADOBERGB][0] = { 3033, 3033, 3033 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_ADOBERGB][1] = { 3029, 3028, 1255 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_ADOBERGB][2] = { 1406, 2988, 3011 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_ADOBERGB][3] = { 1398, 2983, 1190 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_ADOBERGB][4] = { 2860, 1050, 2939 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_ADOBERGB][5] = { 2857, 1033, 945 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_ADOBERGB][6] = { 866, 873, 2916 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_ADOBERGB][7] = { 851, 851, 851 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE240M][0] = { 2926, 2926, 2926 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE240M][1] = { 2923, 2921, 957 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE240M][2] = { 1125, 2877, 2902 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE240M][3] = { 1116, 2871, 885 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE240M][4] = { 2736, 729, 2823 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE240M][5] = { 2732, 710, 611 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE240M][6] = { 523, 531, 2798 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE240M][7] = { 507, 507, 507 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_NONE][0] = { 2125, 2125, 2125 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_NONE][1] = { 2120, 2118, 305 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_NONE][2] = { 392, 2056, 2092 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_NONE][3] = { 387, 2049, 271 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_NONE][4] = { 1868, 206, 1983 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_NONE][5] = { 1863, 199, 163 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_NONE][6] = { 135, 137, 1950 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_NONE][7] = { 130, 130, 130 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_DCI_P3][0] = { 3175, 3175, 3175 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_DCI_P3][1] = { 3172, 3170, 1505 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_DCI_P3][2] = { 1657, 3135, 3155 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_DCI_P3][3] = { 1649, 3130, 1439 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_DCI_P3][4] = { 3021, 1294, 3091 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_DCI_P3][5] = { 3018, 1276, 1184 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_DCI_P3][6] = { 1100, 1107, 3071 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_DCI_P3][7] = { 1084, 1084, 1084 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE2084][0] = { 3798, 3798, 3798 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE2084][1] = { 3797, 3796, 2938 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE2084][2] = { 3049, 3783, 3791 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE2084][3] = { 3044, 3782, 2887 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE2084][4] = { 3741, 2765, 3768 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE2084][5] = { 3740, 2749, 2663 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE2084][6] = { 2580, 2587, 3760 },
- [V4L2_COLORSPACE_DCI_P3][V4L2_XFER_FUNC_SMPTE2084][7] = { 2563, 2563, 2563 },
-};
-
-#else
-
-/* This code generates the table above */
-
-#include <math.h>
-#include <stdio.h>
-#include <stdlib.h>
-
-static const double rec709_to_ntsc1953[3][3] = {
- /*
- * This transform uses the Bradford method to compensate for
- * the different whitepoints.
- */
- { 0.6785011, 0.2883441, 0.0331548 },
- { 0.0165284, 1.0518725, -0.0684009 },
- { 0.0179230, 0.0506096, 0.9314674 }
-};
-
-static const double rec709_to_ebu[3][3] = {
- { 0.9578221, 0.0421779, -0.0000000 },
- { -0.0000000, 1.0000000, 0.0000000 },
- { -0.0000000, -0.0119367, 1.0119367 }
-};
-
-static const double rec709_to_170m[3][3] = {
- { 1.0653640, -0.0553900, -0.0099740 },
- { -0.0196361, 1.0363630, -0.0167269 },
- { 0.0016327, 0.0044133, 0.9939540 },
-};
-
-static const double rec709_to_240m[3][3] = {
- { 1.0653640, -0.0553900, -0.0099740 },
- { -0.0196361, 1.0363630, -0.0167269 },
- { 0.0016327, 0.0044133, 0.9939540 },
-};
-
-static const double rec709_to_adobergb[3][3] = {
- { 0.7151627, 0.2848373, -0.0000000 },
- { 0.0000000, 1.0000000, 0.0000000 },
- { -0.0000000, 0.0411705, 0.9588295 },
-};
-
-static const double rec709_to_bt2020[3][3] = {
- { 0.6274524, 0.3292485, 0.0432991 },
- { 0.0691092, 0.9195311, 0.0113597 },
- { 0.0163976, 0.0880301, 0.8955723 },
-};
-
-static const double rec709_to_dcip3[3][3] = {
- /*
- * This transform uses the Bradford method to compensate for
- * the different whitepoints.
- */
- { 0.8686648, 0.1288456, 0.0024896 },
- { 0.0345479, 0.9618084, 0.0036437 },
- { 0.0167785, 0.0710559, 0.9121655 }
-};
-
-static void mult_matrix(double *r, double *g, double *b, const double m[3][3])
-{
- double ir, ig, ib;
-
- ir = m[0][0] * (*r) + m[0][1] * (*g) + m[0][2] * (*b);
- ig = m[1][0] * (*r) + m[1][1] * (*g) + m[1][2] * (*b);
- ib = m[2][0] * (*r) + m[2][1] * (*g) + m[2][2] * (*b);
- *r = ir;
- *g = ig;
- *b = ib;
-}
-
-static double transfer_srgb_to_rgb(double v)
-{
- if (v < -0.04045)
- return pow((-v + 0.055) / 1.055, 2.4);
- return (v <= 0.04045) ? v / 12.92 : pow((v + 0.055) / 1.055, 2.4);
-}
-
-static double transfer_rgb_to_srgb(double v)
-{
- if (v <= -0.0031308)
- return -1.055 * pow(-v, 1.0 / 2.4) + 0.055;
- if (v <= 0.0031308)
- return v * 12.92;
- return 1.055 * pow(v, 1.0 / 2.4) - 0.055;
-}
-
-static double transfer_rgb_to_smpte240m(double v)
-{
- return (v <= 0.0228) ? v * 4.0 : 1.1115 * pow(v, 0.45) - 0.1115;
-}
-
-static double transfer_rgb_to_rec709(double v)
-{
- if (v <= -0.018)
- return -1.099 * pow(-v, 0.45) + 0.099;
- return (v < 0.018) ? v * 4.5 : 1.099 * pow(v, 0.45) - 0.099;
-}
-
-static double transfer_rec709_to_rgb(double v)
-{
- return (v < 0.081) ? v / 4.5 : pow((v + 0.099) / 1.099, 1.0 / 0.45);
-}
-
-static double transfer_rgb_to_adobergb(double v)
-{
- return pow(v, 1.0 / 2.19921875);
-}
-
-static double transfer_rgb_to_dcip3(double v)
-{
- return pow(v, 1.0 / 2.6);
-}
-
-static double transfer_rgb_to_smpte2084(double v)
-{
- const double m1 = (2610.0 / 4096.0) / 4.0;
- const double m2 = 128.0 * 2523.0 / 4096.0;
- const double c1 = 3424.0 / 4096.0;
- const double c2 = 32.0 * 2413.0 / 4096.0;
- const double c3 = 32.0 * 2392.0 / 4096.0;
-
- v = pow(v, m1);
- return pow((c1 + c2 * v) / (1 + c3 * v), m2);
-}
-
-static double transfer_srgb_to_rec709(double v)
-{
- return transfer_rgb_to_rec709(transfer_srgb_to_rgb(v));
-}
-
-static void csc(enum v4l2_colorspace colorspace, enum v4l2_xfer_func xfer_func,
- double *r, double *g, double *b)
-{
- int clamp = 1;
-
- *r = transfer_srgb_to_rgb(*r);
- *g = transfer_srgb_to_rgb(*g);
- *b = transfer_srgb_to_rgb(*b);
-
- /* Convert the primaries of Rec. 709 Linear RGB */
- switch (colorspace) {
- case V4L2_COLORSPACE_SMPTE240M:
- mult_matrix(r, g, b, rec709_to_240m);
- break;
- case V4L2_COLORSPACE_SMPTE170M:
- mult_matrix(r, g, b, rec709_to_170m);
- break;
- case V4L2_COLORSPACE_470_SYSTEM_BG:
- mult_matrix(r, g, b, rec709_to_ebu);
- break;
- case V4L2_COLORSPACE_470_SYSTEM_M:
- mult_matrix(r, g, b, rec709_to_ntsc1953);
- break;
- case V4L2_COLORSPACE_ADOBERGB:
- mult_matrix(r, g, b, rec709_to_adobergb);
- break;
- case V4L2_COLORSPACE_BT2020:
- mult_matrix(r, g, b, rec709_to_bt2020);
- break;
- case V4L2_COLORSPACE_DCI_P3:
- mult_matrix(r, g, b, rec709_to_dcip3);
- break;
- case V4L2_COLORSPACE_SRGB:
- case V4L2_COLORSPACE_REC709:
- break;
- default:
- break;
- }
-
- if (clamp) {
- *r = ((*r) < 0) ? 0 : (((*r) > 1) ? 1 : (*r));
- *g = ((*g) < 0) ? 0 : (((*g) > 1) ? 1 : (*g));
- *b = ((*b) < 0) ? 0 : (((*b) > 1) ? 1 : (*b));
- }
-
- switch (xfer_func) {
- case V4L2_XFER_FUNC_709:
- *r = transfer_rgb_to_rec709(*r);
- *g = transfer_rgb_to_rec709(*g);
- *b = transfer_rgb_to_rec709(*b);
- break;
- case V4L2_XFER_FUNC_SRGB:
- *r = transfer_rgb_to_srgb(*r);
- *g = transfer_rgb_to_srgb(*g);
- *b = transfer_rgb_to_srgb(*b);
- break;
- case V4L2_XFER_FUNC_ADOBERGB:
- *r = transfer_rgb_to_adobergb(*r);
- *g = transfer_rgb_to_adobergb(*g);
- *b = transfer_rgb_to_adobergb(*b);
- break;
- case V4L2_XFER_FUNC_DCI_P3:
- *r = transfer_rgb_to_dcip3(*r);
- *g = transfer_rgb_to_dcip3(*g);
- *b = transfer_rgb_to_dcip3(*b);
- break;
- case V4L2_XFER_FUNC_SMPTE2084:
- *r = transfer_rgb_to_smpte2084(*r);
- *g = transfer_rgb_to_smpte2084(*g);
- *b = transfer_rgb_to_smpte2084(*b);
- break;
- case V4L2_XFER_FUNC_SMPTE240M:
- *r = transfer_rgb_to_smpte240m(*r);
- *g = transfer_rgb_to_smpte240m(*g);
- *b = transfer_rgb_to_smpte240m(*b);
- break;
- case V4L2_XFER_FUNC_NONE:
- break;
- }
-}
-
-int main(int argc, char **argv)
-{
- static const unsigned colorspaces[] = {
- 0,
- V4L2_COLORSPACE_SMPTE170M,
- V4L2_COLORSPACE_SMPTE240M,
- V4L2_COLORSPACE_REC709,
- 0,
- V4L2_COLORSPACE_470_SYSTEM_M,
- V4L2_COLORSPACE_470_SYSTEM_BG,
- 0,
- V4L2_COLORSPACE_SRGB,
- V4L2_COLORSPACE_ADOBERGB,
- V4L2_COLORSPACE_BT2020,
- 0,
- V4L2_COLORSPACE_DCI_P3,
- };
- static const char * const colorspace_names[] = {
- "",
- "V4L2_COLORSPACE_SMPTE170M",
- "V4L2_COLORSPACE_SMPTE240M",
- "V4L2_COLORSPACE_REC709",
- "",
- "V4L2_COLORSPACE_470_SYSTEM_M",
- "V4L2_COLORSPACE_470_SYSTEM_BG",
- "",
- "V4L2_COLORSPACE_SRGB",
- "V4L2_COLORSPACE_ADOBERGB",
- "V4L2_COLORSPACE_BT2020",
- "",
- "V4L2_COLORSPACE_DCI_P3",
- };
- static const char * const xfer_func_names[] = {
- "",
- "V4L2_XFER_FUNC_709",
- "V4L2_XFER_FUNC_SRGB",
- "V4L2_XFER_FUNC_ADOBERGB",
- "V4L2_XFER_FUNC_SMPTE240M",
- "V4L2_XFER_FUNC_NONE",
- "V4L2_XFER_FUNC_DCI_P3",
- "V4L2_XFER_FUNC_SMPTE2084",
- };
- int i;
- int x;
- int c;
-
- printf("/* Generated table */\n");
- printf("const unsigned short tpg_rec709_to_linear[255 * 16 + 1] = {");
- for (i = 0; i <= 255 * 16; i++) {
- if (i % 16 == 0)
- printf("\n\t");
- printf("%4d,%s",
- (int)(0.5 + 16.0 * 255.0 *
- transfer_rec709_to_rgb(i / (16.0 * 255.0))),
- i % 16 == 15 || i == 255 * 16 ? "" : " ");
- }
- printf("\n};\n\n");
-
- printf("/* Generated table */\n");
- printf("const unsigned short tpg_linear_to_rec709[255 * 16 + 1] = {");
- for (i = 0; i <= 255 * 16; i++) {
- if (i % 16 == 0)
- printf("\n\t");
- printf("%4d,%s",
- (int)(0.5 + 16.0 * 255.0 *
- transfer_rgb_to_rec709(i / (16.0 * 255.0))),
- i % 16 == 15 || i == 255 * 16 ? "" : " ");
- }
- printf("\n};\n\n");
-
- printf("/* Generated table */\n");
- printf("const struct color16 tpg_csc_colors[V4L2_COLORSPACE_DCI_P3 + 1][V4L2_XFER_FUNC_SMPTE2084 + 1][TPG_COLOR_CSC_BLACK + 1] = {\n");
- for (c = 0; c <= V4L2_COLORSPACE_DCI_P3; c++) {
- for (x = 1; x <= V4L2_XFER_FUNC_SMPTE2084; x++) {
- for (i = 0; i <= TPG_COLOR_CSC_BLACK; i++) {
- double r, g, b;
-
- if (colorspaces[c] == 0)
- continue;
-
- r = tpg_colors[i].r / 255.0;
- g = tpg_colors[i].g / 255.0;
- b = tpg_colors[i].b / 255.0;
-
- csc(c, x, &r, &g, &b);
-
- printf("\t[%s][%s][%d] = { %d, %d, %d },\n",
- colorspace_names[c],
- xfer_func_names[x], i,
- (int)(r * 4080), (int)(g * 4080), (int)(b * 4080));
- }
- }
- }
- printf("};\n\n");
- return 0;
-}
-
-#endif
diff --git a/drivers/media/platform/vivid/vivid-tpg-colors.h b/drivers/media/platform/vivid/vivid-tpg-colors.h
deleted file mode 100644
index 4e5a76a..0000000
--- a/drivers/media/platform/vivid/vivid-tpg-colors.h
+++ /dev/null
@@ -1,68 +0,0 @@
-/*
- * vivid-color.h - Color definitions for the test pattern generator
- *
- * Copyright 2014 Cisco Systems, Inc. and/or its affiliates. All rights reserved.
- *
- * This program is free software; you may redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; version 2 of the License.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
- * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
- * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
- * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
- * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
- * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
- * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
- * SOFTWARE.
- */
-
-#ifndef _VIVID_COLORS_H_
-#define _VIVID_COLORS_H_
-
-struct color {
- unsigned char r, g, b;
-};
-
-struct color16 {
- int r, g, b;
-};
-
-enum tpg_color {
- TPG_COLOR_CSC_WHITE,
- TPG_COLOR_CSC_YELLOW,
- TPG_COLOR_CSC_CYAN,
- TPG_COLOR_CSC_GREEN,
- TPG_COLOR_CSC_MAGENTA,
- TPG_COLOR_CSC_RED,
- TPG_COLOR_CSC_BLUE,
- TPG_COLOR_CSC_BLACK,
- TPG_COLOR_75_YELLOW,
- TPG_COLOR_75_CYAN,
- TPG_COLOR_75_GREEN,
- TPG_COLOR_75_MAGENTA,
- TPG_COLOR_75_RED,
- TPG_COLOR_75_BLUE,
- TPG_COLOR_100_WHITE,
- TPG_COLOR_100_YELLOW,
- TPG_COLOR_100_CYAN,
- TPG_COLOR_100_GREEN,
- TPG_COLOR_100_MAGENTA,
- TPG_COLOR_100_RED,
- TPG_COLOR_100_BLUE,
- TPG_COLOR_100_BLACK,
- TPG_COLOR_TEXTFG,
- TPG_COLOR_TEXTBG,
- TPG_COLOR_RANDOM,
- TPG_COLOR_RAMP,
- TPG_COLOR_MAX = TPG_COLOR_RAMP + 256
-};
-
-extern const struct color tpg_colors[TPG_COLOR_MAX];
-extern const unsigned short tpg_rec709_to_linear[255 * 16 + 1];
-extern const unsigned short tpg_linear_to_rec709[255 * 16 + 1];
-extern const struct color16 tpg_csc_colors[V4L2_COLORSPACE_DCI_P3 + 1]
- [V4L2_XFER_FUNC_SMPTE2084 + 1]
- [TPG_COLOR_CSC_BLACK + 1];
-
-#endif
diff --git a/drivers/media/platform/vivid/vivid-tpg.c b/drivers/media/platform/vivid/vivid-tpg.c
deleted file mode 100644
index da862bb..0000000
--- a/drivers/media/platform/vivid/vivid-tpg.c
+++ /dev/null
@@ -1,2314 +0,0 @@
-/*
- * vivid-tpg.c - Test Pattern Generator
- *
- * Note: gen_twopix and tpg_gen_text are based on code from vivi.c. See the
- * vivi.c source for the copyright information of those functions.
- *
- * Copyright 2014 Cisco Systems, Inc. and/or its affiliates. All rights reserved.
- *
- * This program is free software; you may redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; version 2 of the License.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
- * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
- * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
- * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
- * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
- * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
- * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
- * SOFTWARE.
- */
-
-#include "vivid-tpg.h"
-
-/* Must remain in sync with enum tpg_pattern */
-const char * const tpg_pattern_strings[] = {
- "75% Colorbar",
- "100% Colorbar",
- "CSC Colorbar",
- "Horizontal 100% Colorbar",
- "100% Color Squares",
- "100% Black",
- "100% White",
- "100% Red",
- "100% Green",
- "100% Blue",
- "16x16 Checkers",
- "2x2 Checkers",
- "1x1 Checkers",
- "2x2 Red/Green Checkers",
- "1x1 Red/Green Checkers",
- "Alternating Hor Lines",
- "Alternating Vert Lines",
- "One Pixel Wide Cross",
- "Two Pixels Wide Cross",
- "Ten Pixels Wide Cross",
- "Gray Ramp",
- "Noise",
- NULL
-};
-
-/* Must remain in sync with enum tpg_aspect */
-const char * const tpg_aspect_strings[] = {
- "Source Width x Height",
- "4x3",
- "14x9",
- "16x9",
- "16x9 Anamorphic",
- NULL
-};
-
-/*
- * Sine table: sin[0] = 127 * sin(-180 degrees)
- * sin[128] = 127 * sin(0 degrees)
- * sin[256] = 127 * sin(180 degrees)
- */
-static const s8 sin[257] = {
- 0, -4, -7, -11, -13, -18, -20, -22, -26, -29, -33, -35, -37, -41, -43, -48,
- -50, -52, -56, -58, -62, -63, -65, -69, -71, -75, -76, -78, -82, -83, -87, -88,
- -90, -93, -94, -97, -99, -101, -103, -104, -107, -108, -110, -111, -112, -114, -115, -117,
- -118, -119, -120, -121, -122, -123, -123, -124, -125, -125, -126, -126, -127, -127, -127, -127,
- -127, -127, -127, -127, -126, -126, -125, -125, -124, -124, -123, -122, -121, -120, -119, -118,
- -117, -116, -114, -113, -111, -110, -109, -107, -105, -103, -101, -100, -97, -96, -93, -91,
- -90, -87, -85, -82, -80, -76, -75, -73, -69, -67, -63, -62, -60, -56, -54, -50,
- -48, -46, -41, -39, -35, -33, -31, -26, -24, -20, -18, -15, -11, -9, -4, -2,
- 0, 2, 4, 9, 11, 15, 18, 20, 24, 26, 31, 33, 35, 39, 41, 46,
- 48, 50, 54, 56, 60, 62, 64, 67, 69, 73, 75, 76, 80, 82, 85, 87,
- 90, 91, 93, 96, 97, 100, 101, 103, 105, 107, 109, 110, 111, 113, 114, 116,
- 117, 118, 119, 120, 121, 122, 123, 124, 124, 125, 125, 126, 126, 127, 127, 127,
- 127, 127, 127, 127, 127, 126, 126, 125, 125, 124, 123, 123, 122, 121, 120, 119,
- 118, 117, 115, 114, 112, 111, 110, 108, 107, 104, 103, 101, 99, 97, 94, 93,
- 90, 88, 87, 83, 82, 78, 76, 75, 71, 69, 65, 64, 62, 58, 56, 52,
- 50, 48, 43, 41, 37, 35, 33, 29, 26, 22, 20, 18, 13, 11, 7, 4,
- 0,
-};
-
-#define cos(idx) sin[((idx) + 64) % sizeof(sin)]
-
-/* Global font descriptor */
-static const u8 *font8x16;
-
-void tpg_set_font(const u8 *f)
-{
- font8x16 = f;
-}
-
-void tpg_init(struct tpg_data *tpg, unsigned w, unsigned h)
-{
- memset(tpg, 0, sizeof(*tpg));
- tpg->scaled_width = tpg->src_width = w;
- tpg->src_height = tpg->buf_height = h;
- tpg->crop.width = tpg->compose.width = w;
- tpg->crop.height = tpg->compose.height = h;
- tpg->recalc_colors = true;
- tpg->recalc_square_border = true;
- tpg->brightness = 128;
- tpg->contrast = 128;
- tpg->saturation = 128;
- tpg->hue = 0;
- tpg->mv_hor_mode = TPG_MOVE_NONE;
- tpg->mv_vert_mode = TPG_MOVE_NONE;
- tpg->field = V4L2_FIELD_NONE;
- tpg_s_fourcc(tpg, V4L2_PIX_FMT_RGB24);
- tpg->colorspace = V4L2_COLORSPACE_SRGB;
- tpg->perc_fill = 100;
-}
-
-int tpg_alloc(struct tpg_data *tpg, unsigned max_w)
-{
- unsigned pat;
- unsigned plane;
-
- tpg->max_line_width = max_w;
- for (pat = 0; pat < TPG_MAX_PAT_LINES; pat++) {
- for (plane = 0; plane < TPG_MAX_PLANES; plane++) {
- unsigned pixelsz = plane ? 2 : 4;
-
- tpg->lines[pat][plane] = vzalloc(max_w * 2 * pixelsz);
- if (!tpg->lines[pat][plane])
- return -ENOMEM;
- if (plane == 0)
- continue;
- tpg->downsampled_lines[pat][plane] = vzalloc(max_w * 2 * pixelsz);
- if (!tpg->downsampled_lines[pat][plane])
- return -ENOMEM;
- }
- }
- for (plane = 0; plane < TPG_MAX_PLANES; plane++) {
- unsigned pixelsz = plane ? 2 : 4;
-
- tpg->contrast_line[plane] = vzalloc(max_w * pixelsz);
- if (!tpg->contrast_line[plane])
- return -ENOMEM;
- tpg->black_line[plane] = vzalloc(max_w * pixelsz);
- if (!tpg->black_line[plane])
- return -ENOMEM;
- tpg->random_line[plane] = vzalloc(max_w * 2 * pixelsz);
- if (!tpg->random_line[plane])
- return -ENOMEM;
- }
- return 0;
-}
-
-void tpg_free(struct tpg_data *tpg)
-{
- unsigned pat;
- unsigned plane;
-
- for (pat = 0; pat < TPG_MAX_PAT_LINES; pat++)
- for (plane = 0; plane < TPG_MAX_PLANES; plane++) {
- vfree(tpg->lines[pat][plane]);
- tpg->lines[pat][plane] = NULL;
- if (plane == 0)
- continue;
- vfree(tpg->downsampled_lines[pat][plane]);
- tpg->downsampled_lines[pat][plane] = NULL;
- }
- for (plane = 0; plane < TPG_MAX_PLANES; plane++) {
- vfree(tpg->contrast_line[plane]);
- vfree(tpg->black_line[plane]);
- vfree(tpg->random_line[plane]);
- tpg->contrast_line[plane] = NULL;
- tpg->black_line[plane] = NULL;
- tpg->random_line[plane] = NULL;
- }
-}
-
-bool tpg_s_fourcc(struct tpg_data *tpg, u32 fourcc)
-{
- tpg->fourcc = fourcc;
- tpg->planes = 1;
- tpg->buffers = 1;
- tpg->recalc_colors = true;
- tpg->interleaved = false;
- tpg->vdownsampling[0] = 1;
- tpg->hdownsampling[0] = 1;
- tpg->hmask[0] = ~0;
- tpg->hmask[1] = ~0;
- tpg->hmask[2] = ~0;
-
- switch (fourcc) {
- case V4L2_PIX_FMT_SBGGR8:
- case V4L2_PIX_FMT_SGBRG8:
- case V4L2_PIX_FMT_SGRBG8:
- case V4L2_PIX_FMT_SRGGB8:
- case V4L2_PIX_FMT_SBGGR10:
- case V4L2_PIX_FMT_SGBRG10:
- case V4L2_PIX_FMT_SGRBG10:
- case V4L2_PIX_FMT_SRGGB10:
- case V4L2_PIX_FMT_SBGGR12:
- case V4L2_PIX_FMT_SGBRG12:
- case V4L2_PIX_FMT_SGRBG12:
- case V4L2_PIX_FMT_SRGGB12:
- tpg->interleaved = true;
- tpg->vdownsampling[1] = 1;
- tpg->hdownsampling[1] = 1;
- tpg->planes = 2;
- /* fall through */
- case V4L2_PIX_FMT_RGB332:
- case V4L2_PIX_FMT_RGB565:
- case V4L2_PIX_FMT_RGB565X:
- case V4L2_PIX_FMT_RGB444:
- case V4L2_PIX_FMT_XRGB444:
- case V4L2_PIX_FMT_ARGB444:
- case V4L2_PIX_FMT_RGB555:
- case V4L2_PIX_FMT_XRGB555:
- case V4L2_PIX_FMT_ARGB555:
- case V4L2_PIX_FMT_RGB555X:
- case V4L2_PIX_FMT_XRGB555X:
- case V4L2_PIX_FMT_ARGB555X:
- case V4L2_PIX_FMT_BGR666:
- case V4L2_PIX_FMT_RGB24:
- case V4L2_PIX_FMT_BGR24:
- case V4L2_PIX_FMT_RGB32:
- case V4L2_PIX_FMT_BGR32:
- case V4L2_PIX_FMT_XRGB32:
- case V4L2_PIX_FMT_XBGR32:
- case V4L2_PIX_FMT_ARGB32:
- case V4L2_PIX_FMT_ABGR32:
- case V4L2_PIX_FMT_GREY:
- case V4L2_PIX_FMT_Y16:
- case V4L2_PIX_FMT_Y16_BE:
- tpg->is_yuv = false;
- break;
- case V4L2_PIX_FMT_YUV444:
- case V4L2_PIX_FMT_YUV555:
- case V4L2_PIX_FMT_YUV565:
- case V4L2_PIX_FMT_YUV32:
- tpg->is_yuv = true;
- break;
- case V4L2_PIX_FMT_YUV420M:
- case V4L2_PIX_FMT_YVU420M:
- tpg->buffers = 3;
- /* fall through */
- case V4L2_PIX_FMT_YUV420:
- case V4L2_PIX_FMT_YVU420:
- tpg->vdownsampling[1] = 2;
- tpg->vdownsampling[2] = 2;
- tpg->hdownsampling[1] = 2;
- tpg->hdownsampling[2] = 2;
- tpg->planes = 3;
- tpg->is_yuv = true;
- break;
- case V4L2_PIX_FMT_YUV422M:
- case V4L2_PIX_FMT_YVU422M:
- tpg->buffers = 3;
- /* fall through */
- case V4L2_PIX_FMT_YUV422P:
- tpg->vdownsampling[1] = 1;
- tpg->vdownsampling[2] = 1;
- tpg->hdownsampling[1] = 2;
- tpg->hdownsampling[2] = 2;
- tpg->planes = 3;
- tpg->is_yuv = true;
- break;
- case V4L2_PIX_FMT_NV16M:
- case V4L2_PIX_FMT_NV61M:
- tpg->buffers = 2;
- /* fall through */
- case V4L2_PIX_FMT_NV16:
- case V4L2_PIX_FMT_NV61:
- tpg->vdownsampling[1] = 1;
- tpg->hdownsampling[1] = 1;
- tpg->hmask[1] = ~1;
- tpg->planes = 2;
- tpg->is_yuv = true;
- break;
- case V4L2_PIX_FMT_NV12M:
- case V4L2_PIX_FMT_NV21M:
- tpg->buffers = 2;
- /* fall through */
- case V4L2_PIX_FMT_NV12:
- case V4L2_PIX_FMT_NV21:
- tpg->vdownsampling[1] = 2;
- tpg->hdownsampling[1] = 1;
- tpg->hmask[1] = ~1;
- tpg->planes = 2;
- tpg->is_yuv = true;
- break;
- case V4L2_PIX_FMT_YUV444M:
- case V4L2_PIX_FMT_YVU444M:
- tpg->buffers = 3;
- tpg->planes = 3;
- tpg->vdownsampling[1] = 1;
- tpg->vdownsampling[2] = 1;
- tpg->hdownsampling[1] = 1;
- tpg->hdownsampling[2] = 1;
- tpg->is_yuv = true;
- break;
- case V4L2_PIX_FMT_NV24:
- case V4L2_PIX_FMT_NV42:
- tpg->vdownsampling[1] = 1;
- tpg->hdownsampling[1] = 1;
- tpg->planes = 2;
- tpg->is_yuv = true;
- break;
- case V4L2_PIX_FMT_YUYV:
- case V4L2_PIX_FMT_UYVY:
- case V4L2_PIX_FMT_YVYU:
- case V4L2_PIX_FMT_VYUY:
- tpg->hmask[0] = ~1;
- tpg->is_yuv = true;
- break;
- default:
- return false;
- }
-
- switch (fourcc) {
- case V4L2_PIX_FMT_GREY:
- case V4L2_PIX_FMT_RGB332:
- tpg->twopixelsize[0] = 2;
- break;
- case V4L2_PIX_FMT_RGB565:
- case V4L2_PIX_FMT_RGB565X:
- case V4L2_PIX_FMT_RGB444:
- case V4L2_PIX_FMT_XRGB444:
- case V4L2_PIX_FMT_ARGB444:
- case V4L2_PIX_FMT_RGB555:
- case V4L2_PIX_FMT_XRGB555:
- case V4L2_PIX_FMT_ARGB555:
- case V4L2_PIX_FMT_RGB555X:
- case V4L2_PIX_FMT_XRGB555X:
- case V4L2_PIX_FMT_ARGB555X:
- case V4L2_PIX_FMT_YUYV:
- case V4L2_PIX_FMT_UYVY:
- case V4L2_PIX_FMT_YVYU:
- case V4L2_PIX_FMT_VYUY:
- case V4L2_PIX_FMT_YUV444:
- case V4L2_PIX_FMT_YUV555:
- case V4L2_PIX_FMT_YUV565:
- case V4L2_PIX_FMT_Y16:
- case V4L2_PIX_FMT_Y16_BE:
- tpg->twopixelsize[0] = 2 * 2;
- break;
- case V4L2_PIX_FMT_RGB24:
- case V4L2_PIX_FMT_BGR24:
- tpg->twopixelsize[0] = 2 * 3;
- break;
- case V4L2_PIX_FMT_BGR666:
- case V4L2_PIX_FMT_RGB32:
- case V4L2_PIX_FMT_BGR32:
- case V4L2_PIX_FMT_XRGB32:
- case V4L2_PIX_FMT_XBGR32:
- case V4L2_PIX_FMT_ARGB32:
- case V4L2_PIX_FMT_ABGR32:
- case V4L2_PIX_FMT_YUV32:
- tpg->twopixelsize[0] = 2 * 4;
- break;
- case V4L2_PIX_FMT_NV12:
- case V4L2_PIX_FMT_NV21:
- case V4L2_PIX_FMT_NV12M:
- case V4L2_PIX_FMT_NV21M:
- case V4L2_PIX_FMT_NV16:
- case V4L2_PIX_FMT_NV61:
- case V4L2_PIX_FMT_NV16M:
- case V4L2_PIX_FMT_NV61M:
- case V4L2_PIX_FMT_SBGGR8:
- case V4L2_PIX_FMT_SGBRG8:
- case V4L2_PIX_FMT_SGRBG8:
- case V4L2_PIX_FMT_SRGGB8:
- tpg->twopixelsize[0] = 2;
- tpg->twopixelsize[1] = 2;
- break;
- case V4L2_PIX_FMT_SRGGB10:
- case V4L2_PIX_FMT_SGRBG10:
- case V4L2_PIX_FMT_SGBRG10:
- case V4L2_PIX_FMT_SBGGR10:
- case V4L2_PIX_FMT_SRGGB12:
- case V4L2_PIX_FMT_SGRBG12:
- case V4L2_PIX_FMT_SGBRG12:
- case V4L2_PIX_FMT_SBGGR12:
- tpg->twopixelsize[0] = 4;
- tpg->twopixelsize[1] = 4;
- break;
- case V4L2_PIX_FMT_YUV444M:
- case V4L2_PIX_FMT_YVU444M:
- case V4L2_PIX_FMT_YUV422M:
- case V4L2_PIX_FMT_YVU422M:
- case V4L2_PIX_FMT_YUV422P:
- case V4L2_PIX_FMT_YUV420:
- case V4L2_PIX_FMT_YVU420:
- case V4L2_PIX_FMT_YUV420M:
- case V4L2_PIX_FMT_YVU420M:
- tpg->twopixelsize[0] = 2;
- tpg->twopixelsize[1] = 2;
- tpg->twopixelsize[2] = 2;
- break;
- case V4L2_PIX_FMT_NV24:
- case V4L2_PIX_FMT_NV42:
- tpg->twopixelsize[0] = 2;
- tpg->twopixelsize[1] = 4;
- break;
- }
- return true;
-}
-
-void tpg_s_crop_compose(struct tpg_data *tpg, const struct v4l2_rect *crop,
- const struct v4l2_rect *compose)
-{
- tpg->crop = *crop;
- tpg->compose = *compose;
- tpg->scaled_width = (tpg->src_width * tpg->compose.width +
- tpg->crop.width - 1) / tpg->crop.width;
- tpg->scaled_width &= ~1;
- if (tpg->scaled_width > tpg->max_line_width)
- tpg->scaled_width = tpg->max_line_width;
- if (tpg->scaled_width < 2)
- tpg->scaled_width = 2;
- tpg->recalc_lines = true;
-}
-
-void tpg_reset_source(struct tpg_data *tpg, unsigned width, unsigned height,
- u32 field)
-{
- unsigned p;
-
- tpg->src_width = width;
- tpg->src_height = height;
- tpg->field = field;
- tpg->buf_height = height;
- if (V4L2_FIELD_HAS_T_OR_B(field))
- tpg->buf_height /= 2;
- tpg->scaled_width = width;
- tpg->crop.top = tpg->crop.left = 0;
- tpg->crop.width = width;
- tpg->crop.height = height;
- tpg->compose.top = tpg->compose.left = 0;
- tpg->compose.width = width;
- tpg->compose.height = tpg->buf_height;
- for (p = 0; p < tpg->planes; p++)
- tpg->bytesperline[p] = (width * tpg->twopixelsize[p]) /
- (2 * tpg->hdownsampling[p]);
- tpg->recalc_square_border = true;
-}
-
-static enum tpg_color tpg_get_textbg_color(struct tpg_data *tpg)
-{
- switch (tpg->pattern) {
- case TPG_PAT_BLACK:
- return TPG_COLOR_100_WHITE;
- case TPG_PAT_CSC_COLORBAR:
- return TPG_COLOR_CSC_BLACK;
- default:
- return TPG_COLOR_100_BLACK;
- }
-}
-
-static enum tpg_color tpg_get_textfg_color(struct tpg_data *tpg)
-{
- switch (tpg->pattern) {
- case TPG_PAT_75_COLORBAR:
- case TPG_PAT_CSC_COLORBAR:
- return TPG_COLOR_CSC_WHITE;
- case TPG_PAT_BLACK:
- return TPG_COLOR_100_BLACK;
- default:
- return TPG_COLOR_100_WHITE;
- }
-}
-
-static inline int rec709_to_linear(int v)
-{
- v = clamp(v, 0, 0xff0);
- return tpg_rec709_to_linear[v];
-}
-
-static inline int linear_to_rec709(int v)
-{
- v = clamp(v, 0, 0xff0);
- return tpg_linear_to_rec709[v];
-}
-
-static void rgb2ycbcr(const int m[3][3], int r, int g, int b,
- int y_offset, int *y, int *cb, int *cr)
-{
- *y = ((m[0][0] * r + m[0][1] * g + m[0][2] * b) >> 16) + (y_offset << 4);
- *cb = ((m[1][0] * r + m[1][1] * g + m[1][2] * b) >> 16) + (128 << 4);
- *cr = ((m[2][0] * r + m[2][1] * g + m[2][2] * b) >> 16) + (128 << 4);
-}
-
-static void color_to_ycbcr(struct tpg_data *tpg, int r, int g, int b,
- int *y, int *cb, int *cr)
-{
-#define COEFF(v, r) ((int)(0.5 + (v) * (r) * 256.0))
-
- static const int bt601[3][3] = {
- { COEFF(0.299, 219), COEFF(0.587, 219), COEFF(0.114, 219) },
- { COEFF(-0.169, 224), COEFF(-0.331, 224), COEFF(0.5, 224) },
- { COEFF(0.5, 224), COEFF(-0.419, 224), COEFF(-0.081, 224) },
- };
- static const int bt601_full[3][3] = {
- { COEFF(0.299, 255), COEFF(0.587, 255), COEFF(0.114, 255) },
- { COEFF(-0.169, 255), COEFF(-0.331, 255), COEFF(0.5, 255) },
- { COEFF(0.5, 255), COEFF(-0.419, 255), COEFF(-0.081, 255) },
- };
- static const int rec709[3][3] = {
- { COEFF(0.2126, 219), COEFF(0.7152, 219), COEFF(0.0722, 219) },
- { COEFF(-0.1146, 224), COEFF(-0.3854, 224), COEFF(0.5, 224) },
- { COEFF(0.5, 224), COEFF(-0.4542, 224), COEFF(-0.0458, 224) },
- };
- static const int rec709_full[3][3] = {
- { COEFF(0.2126, 255), COEFF(0.7152, 255), COEFF(0.0722, 255) },
- { COEFF(-0.1146, 255), COEFF(-0.3854, 255), COEFF(0.5, 255) },
- { COEFF(0.5, 255), COEFF(-0.4542, 255), COEFF(-0.0458, 255) },
- };
- static const int smpte240m[3][3] = {
- { COEFF(0.212, 219), COEFF(0.701, 219), COEFF(0.087, 219) },
- { COEFF(-0.116, 224), COEFF(-0.384, 224), COEFF(0.5, 224) },
- { COEFF(0.5, 224), COEFF(-0.445, 224), COEFF(-0.055, 224) },
- };
- static const int smpte240m_full[3][3] = {
- { COEFF(0.212, 255), COEFF(0.701, 255), COEFF(0.087, 255) },
- { COEFF(-0.116, 255), COEFF(-0.384, 255), COEFF(0.5, 255) },
- { COEFF(0.5, 255), COEFF(-0.445, 255), COEFF(-0.055, 255) },
- };
- static const int bt2020[3][3] = {
- { COEFF(0.2627, 219), COEFF(0.6780, 219), COEFF(0.0593, 219) },
- { COEFF(-0.1396, 224), COEFF(-0.3604, 224), COEFF(0.5, 224) },
- { COEFF(0.5, 224), COEFF(-0.4598, 224), COEFF(-0.0402, 224) },
- };
- static const int bt2020_full[3][3] = {
- { COEFF(0.2627, 255), COEFF(0.6780, 255), COEFF(0.0593, 255) },
- { COEFF(-0.1396, 255), COEFF(-0.3604, 255), COEFF(0.5, 255) },
- { COEFF(0.5, 255), COEFF(-0.4698, 255), COEFF(-0.0402, 255) },
- };
- static const int bt2020c[4] = {
- COEFF(1.0 / 1.9404, 224), COEFF(1.0 / 1.5816, 224),
- COEFF(1.0 / 1.7184, 224), COEFF(1.0 / 0.9936, 224),
- };
- static const int bt2020c_full[4] = {
- COEFF(1.0 / 1.9404, 255), COEFF(1.0 / 1.5816, 255),
- COEFF(1.0 / 1.7184, 255), COEFF(1.0 / 0.9936, 255),
- };
-
- bool full = tpg->real_quantization == V4L2_QUANTIZATION_FULL_RANGE;
- unsigned y_offset = full ? 0 : 16;
- int lin_y, yc;
-
- switch (tpg->real_ycbcr_enc) {
- case V4L2_YCBCR_ENC_601:
- case V4L2_YCBCR_ENC_SYCC:
- rgb2ycbcr(full ? bt601_full : bt601, r, g, b, y_offset, y, cb, cr);
- break;
- case V4L2_YCBCR_ENC_XV601:
- /* Ignore quantization range, there is only one possible
- * Y'CbCr encoding. */
- rgb2ycbcr(bt601, r, g, b, 16, y, cb, cr);
- break;
- case V4L2_YCBCR_ENC_XV709:
- /* Ignore quantization range, there is only one possible
- * Y'CbCr encoding. */
- rgb2ycbcr(rec709, r, g, b, 16, y, cb, cr);
- break;
- case V4L2_YCBCR_ENC_BT2020:
- rgb2ycbcr(full ? bt2020_full : bt2020, r, g, b, y_offset, y, cb, cr);
- break;
- case V4L2_YCBCR_ENC_BT2020_CONST_LUM:
- lin_y = (COEFF(0.2627, 255) * rec709_to_linear(r) +
- COEFF(0.6780, 255) * rec709_to_linear(g) +
- COEFF(0.0593, 255) * rec709_to_linear(b)) >> 16;
- yc = linear_to_rec709(lin_y);
- *y = full ? yc : (yc * 219) / 255 + (16 << 4);
- if (b <= yc)
- *cb = (((b - yc) * (full ? bt2020c_full[0] : bt2020c[0])) >> 16) + (128 << 4);
- else
- *cb = (((b - yc) * (full ? bt2020c_full[1] : bt2020c[1])) >> 16) + (128 << 4);
- if (r <= yc)
- *cr = (((r - yc) * (full ? bt2020c_full[2] : bt2020c[2])) >> 16) + (128 << 4);
- else
- *cr = (((r - yc) * (full ? bt2020c_full[3] : bt2020c[3])) >> 16) + (128 << 4);
- break;
- case V4L2_YCBCR_ENC_SMPTE240M:
- rgb2ycbcr(full ? smpte240m_full : smpte240m, r, g, b, y_offset, y, cb, cr);
- break;
- case V4L2_YCBCR_ENC_709:
- default:
- rgb2ycbcr(full ? rec709_full : rec709, r, g, b, y_offset, y, cb, cr);
- break;
- }
-}
-
-static void ycbcr2rgb(const int m[3][3], int y, int cb, int cr,
- int y_offset, int *r, int *g, int *b)
-{
- y -= y_offset << 4;
- cb -= 128 << 4;
- cr -= 128 << 4;
- *r = m[0][0] * y + m[0][1] * cb + m[0][2] * cr;
- *g = m[1][0] * y + m[1][1] * cb + m[1][2] * cr;
- *b = m[2][0] * y + m[2][1] * cb + m[2][2] * cr;
- *r = clamp(*r >> 12, 0, 0xff0);
- *g = clamp(*g >> 12, 0, 0xff0);
- *b = clamp(*b >> 12, 0, 0xff0);
-}
-
-static void ycbcr_to_color(struct tpg_data *tpg, int y, int cb, int cr,
- int *r, int *g, int *b)
-{
-#undef COEFF
-#define COEFF(v, r) ((int)(0.5 + (v) * ((255.0 * 255.0 * 16.0) / (r))))
- static const int bt601[3][3] = {
- { COEFF(1, 219), COEFF(0, 224), COEFF(1.4020, 224) },
- { COEFF(1, 219), COEFF(-0.3441, 224), COEFF(-0.7141, 224) },
- { COEFF(1, 219), COEFF(1.7720, 224), COEFF(0, 224) },
- };
- static const int bt601_full[3][3] = {
- { COEFF(1, 255), COEFF(0, 255), COEFF(1.4020, 255) },
- { COEFF(1, 255), COEFF(-0.3441, 255), COEFF(-0.7141, 255) },
- { COEFF(1, 255), COEFF(1.7720, 255), COEFF(0, 255) },
- };
- static const int rec709[3][3] = {
- { COEFF(1, 219), COEFF(0, 224), COEFF(1.5748, 224) },
- { COEFF(1, 219), COEFF(-0.1873, 224), COEFF(-0.4681, 224) },
- { COEFF(1, 219), COEFF(1.8556, 224), COEFF(0, 224) },
- };
- static const int rec709_full[3][3] = {
- { COEFF(1, 255), COEFF(0, 255), COEFF(1.5748, 255) },
- { COEFF(1, 255), COEFF(-0.1873, 255), COEFF(-0.4681, 255) },
- { COEFF(1, 255), COEFF(1.8556, 255), COEFF(0, 255) },
- };
- static const int smpte240m[3][3] = {
- { COEFF(1, 219), COEFF(0, 224), COEFF(1.5756, 224) },
- { COEFF(1, 219), COEFF(-0.2253, 224), COEFF(-0.4767, 224) },
- { COEFF(1, 219), COEFF(1.8270, 224), COEFF(0, 224) },
- };
- static const int smpte240m_full[3][3] = {
- { COEFF(1, 255), COEFF(0, 255), COEFF(1.5756, 255) },
- { COEFF(1, 255), COEFF(-0.2253, 255), COEFF(-0.4767, 255) },
- { COEFF(1, 255), COEFF(1.8270, 255), COEFF(0, 255) },
- };
- static const int bt2020[3][3] = {
- { COEFF(1, 219), COEFF(0, 224), COEFF(1.4746, 224) },
- { COEFF(1, 219), COEFF(-0.1646, 224), COEFF(-0.5714, 224) },
- { COEFF(1, 219), COEFF(1.8814, 224), COEFF(0, 224) },
- };
- static const int bt2020_full[3][3] = {
- { COEFF(1, 255), COEFF(0, 255), COEFF(1.4746, 255) },
- { COEFF(1, 255), COEFF(-0.1646, 255), COEFF(-0.5714, 255) },
- { COEFF(1, 255), COEFF(1.8814, 255), COEFF(0, 255) },
- };
- static const int bt2020c[4] = {
- COEFF(1.9404, 224), COEFF(1.5816, 224),
- COEFF(1.7184, 224), COEFF(0.9936, 224),
- };
- static const int bt2020c_full[4] = {
- COEFF(1.9404, 255), COEFF(1.5816, 255),
- COEFF(1.7184, 255), COEFF(0.9936, 255),
- };
-
- bool full = tpg->real_quantization == V4L2_QUANTIZATION_FULL_RANGE;
- unsigned y_offset = full ? 0 : 16;
- int y_fac = full ? COEFF(1.0, 255) : COEFF(1.0, 219);
- int lin_r, lin_g, lin_b, lin_y;
-
- switch (tpg->real_ycbcr_enc) {
- case V4L2_YCBCR_ENC_601:
- case V4L2_YCBCR_ENC_SYCC:
- ycbcr2rgb(full ? bt601_full : bt601, y, cb, cr, y_offset, r, g, b);
- break;
- case V4L2_YCBCR_ENC_XV601:
- /* Ignore quantization range, there is only one possible
- * Y'CbCr encoding. */
- ycbcr2rgb(bt601, y, cb, cr, 16, r, g, b);
- break;
- case V4L2_YCBCR_ENC_XV709:
- /* Ignore quantization range, there is only one possible
- * Y'CbCr encoding. */
- ycbcr2rgb(rec709, y, cb, cr, 16, r, g, b);
- break;
- case V4L2_YCBCR_ENC_BT2020:
- ycbcr2rgb(full ? bt2020_full : bt2020, y, cb, cr, y_offset, r, g, b);
- break;
- case V4L2_YCBCR_ENC_BT2020_CONST_LUM:
- y -= full ? 0 : 16 << 4;
- cb -= 128 << 4;
- cr -= 128 << 4;
-
- if (cb <= 0)
- *b = y_fac * y + (full ? bt2020c_full[0] : bt2020c[0]) * cb;
- else
- *b = y_fac * y + (full ? bt2020c_full[1] : bt2020c[1]) * cb;
- *b = *b >> 12;
- if (cr <= 0)
- *r = y_fac * y + (full ? bt2020c_full[2] : bt2020c[2]) * cr;
- else
- *r = y_fac * y + (full ? bt2020c_full[3] : bt2020c[3]) * cr;
- *r = *r >> 12;
- lin_r = rec709_to_linear(*r);
- lin_b = rec709_to_linear(*b);
- lin_y = rec709_to_linear((y * 255) / (full ? 255 : 219));
-
- lin_g = COEFF(1.0 / 0.6780, 255) * lin_y -
- COEFF(0.2627 / 0.6780, 255) * lin_r -
- COEFF(0.0593 / 0.6780, 255) * lin_b;
- *g = linear_to_rec709(lin_g >> 12);
- break;
- case V4L2_YCBCR_ENC_SMPTE240M:
- ycbcr2rgb(full ? smpte240m_full : smpte240m, y, cb, cr, y_offset, r, g, b);
- break;
- case V4L2_YCBCR_ENC_709:
- default:
- ycbcr2rgb(full ? rec709_full : rec709, y, cb, cr, y_offset, r, g, b);
- break;
- }
-}
-
-/* precalculate color bar values to speed up rendering */
-static void precalculate_color(struct tpg_data *tpg, int k)
-{
- int col = k;
- int r = tpg_colors[col].r;
- int g = tpg_colors[col].g;
- int b = tpg_colors[col].b;
-
- if (k == TPG_COLOR_TEXTBG) {
- col = tpg_get_textbg_color(tpg);
-
- r = tpg_colors[col].r;
- g = tpg_colors[col].g;
- b = tpg_colors[col].b;
- } else if (k == TPG_COLOR_TEXTFG) {
- col = tpg_get_textfg_color(tpg);
-
- r = tpg_colors[col].r;
- g = tpg_colors[col].g;
- b = tpg_colors[col].b;
- } else if (tpg->pattern == TPG_PAT_NOISE) {
- r = g = b = prandom_u32_max(256);
- } else if (k == TPG_COLOR_RANDOM) {
- r = g = b = tpg->qual_offset + prandom_u32_max(196);
- } else if (k >= TPG_COLOR_RAMP) {
- r = g = b = k - TPG_COLOR_RAMP;
- }
-
- if (tpg->pattern == TPG_PAT_CSC_COLORBAR && col <= TPG_COLOR_CSC_BLACK) {
- r = tpg_csc_colors[tpg->colorspace][tpg->real_xfer_func][col].r;
- g = tpg_csc_colors[tpg->colorspace][tpg->real_xfer_func][col].g;
- b = tpg_csc_colors[tpg->colorspace][tpg->real_xfer_func][col].b;
- } else {
- r <<= 4;
- g <<= 4;
- b <<= 4;
- }
- if (tpg->qual == TPG_QUAL_GRAY || tpg->fourcc == V4L2_PIX_FMT_GREY ||
- tpg->fourcc == V4L2_PIX_FMT_Y16 ||
- tpg->fourcc == V4L2_PIX_FMT_Y16_BE) {
- /* Rec. 709 Luma function */
- /* (0.2126, 0.7152, 0.0722) * (255 * 256) */
- r = g = b = (13879 * r + 46688 * g + 4713 * b) >> 16;
- }
-
- /*
- * The assumption is that the RGB output is always full range,
- * so only if the rgb_range overrides the 'real' rgb range do
- * we need to convert the RGB values.
- *
- * Remember that r, g and b are still in the 0 - 0xff0 range.
- */
- if (tpg->real_rgb_range == V4L2_DV_RGB_RANGE_LIMITED &&
- tpg->rgb_range == V4L2_DV_RGB_RANGE_FULL) {
- /*
- * Convert from full range (which is what r, g and b are)
- * to limited range (which is the 'real' RGB range), which
- * is then interpreted as full range.
- */
- r = (r * 219) / 255 + (16 << 4);
- g = (g * 219) / 255 + (16 << 4);
- b = (b * 219) / 255 + (16 << 4);
- } else if (tpg->real_rgb_range != V4L2_DV_RGB_RANGE_LIMITED &&
- tpg->rgb_range == V4L2_DV_RGB_RANGE_LIMITED) {
- /*
- * Clamp r, g and b to the limited range and convert to full
- * range since that's what we deliver.
- */
- r = clamp(r, 16 << 4, 235 << 4);
- g = clamp(g, 16 << 4, 235 << 4);
- b = clamp(b, 16 << 4, 235 << 4);
- r = (r - (16 << 4)) * 255 / 219;
- g = (g - (16 << 4)) * 255 / 219;
- b = (b - (16 << 4)) * 255 / 219;
- }
-
- if (tpg->brightness != 128 || tpg->contrast != 128 ||
- tpg->saturation != 128 || tpg->hue) {
- /* Implement these operations */
- int y, cb, cr;
- int tmp_cb, tmp_cr;
-
- /* First convert to YCbCr */
-
- color_to_ycbcr(tpg, r, g, b, &y, &cb, &cr);
-
- y = (16 << 4) + ((y - (16 << 4)) * tpg->contrast) / 128;
- y += (tpg->brightness << 4) - (128 << 4);
-
- cb -= 128 << 4;
- cr -= 128 << 4;
- tmp_cb = (cb * cos(128 + tpg->hue)) / 127 + (cr * sin[128 + tpg->hue]) / 127;
- tmp_cr = (cr * cos(128 + tpg->hue)) / 127 - (cb * sin[128 + tpg->hue]) / 127;
-
- cb = (128 << 4) + (tmp_cb * tpg->contrast * tpg->saturation) / (128 * 128);
- cr = (128 << 4) + (tmp_cr * tpg->contrast * tpg->saturation) / (128 * 128);
- if (tpg->is_yuv) {
- tpg->colors[k][0] = clamp(y >> 4, 1, 254);
- tpg->colors[k][1] = clamp(cb >> 4, 1, 254);
- tpg->colors[k][2] = clamp(cr >> 4, 1, 254);
- return;
- }
- ycbcr_to_color(tpg, y, cb, cr, &r, &g, &b);
- }
-
- if (tpg->is_yuv) {
- /* Convert to YCbCr */
- int y, cb, cr;
-
- color_to_ycbcr(tpg, r, g, b, &y, &cb, &cr);
-
- if (tpg->real_quantization == V4L2_QUANTIZATION_LIM_RANGE) {
- y = clamp(y, 16 << 4, 235 << 4);
- cb = clamp(cb, 16 << 4, 240 << 4);
- cr = clamp(cr, 16 << 4, 240 << 4);
- }
- y = clamp(y >> 4, 1, 254);
- cb = clamp(cb >> 4, 1, 254);
- cr = clamp(cr >> 4, 1, 254);
- switch (tpg->fourcc) {
- case V4L2_PIX_FMT_YUV444:
- y >>= 4;
- cb >>= 4;
- cr >>= 4;
- break;
- case V4L2_PIX_FMT_YUV555:
- y >>= 3;
- cb >>= 3;
- cr >>= 3;
- break;
- case V4L2_PIX_FMT_YUV565:
- y >>= 3;
- cb >>= 2;
- cr >>= 3;
- break;
- }
- tpg->colors[k][0] = y;
- tpg->colors[k][1] = cb;
- tpg->colors[k][2] = cr;
- } else {
- if (tpg->real_quantization == V4L2_QUANTIZATION_LIM_RANGE) {
- r = (r * 219) / 255 + (16 << 4);
- g = (g * 219) / 255 + (16 << 4);
- b = (b * 219) / 255 + (16 << 4);
- }
- switch (tpg->fourcc) {
- case V4L2_PIX_FMT_RGB332:
- r >>= 9;
- g >>= 9;
- b >>= 10;
- break;
- case V4L2_PIX_FMT_RGB565:
- case V4L2_PIX_FMT_RGB565X:
- r >>= 7;
- g >>= 6;
- b >>= 7;
- break;
- case V4L2_PIX_FMT_RGB444:
- case V4L2_PIX_FMT_XRGB444:
- case V4L2_PIX_FMT_ARGB444:
- r >>= 8;
- g >>= 8;
- b >>= 8;
- break;
- case V4L2_PIX_FMT_RGB555:
- case V4L2_PIX_FMT_XRGB555:
- case V4L2_PIX_FMT_ARGB555:
- case V4L2_PIX_FMT_RGB555X:
- case V4L2_PIX_FMT_XRGB555X:
- case V4L2_PIX_FMT_ARGB555X:
- r >>= 7;
- g >>= 7;
- b >>= 7;
- break;
- case V4L2_PIX_FMT_BGR666:
- r >>= 6;
- g >>= 6;
- b >>= 6;
- break;
- default:
- r >>= 4;
- g >>= 4;
- b >>= 4;
- break;
- }
-
- tpg->colors[k][0] = r;
- tpg->colors[k][1] = g;
- tpg->colors[k][2] = b;
- }
-}
-
-static void tpg_precalculate_colors(struct tpg_data *tpg)
-{
- int k;
-
- for (k = 0; k < TPG_COLOR_MAX; k++)
- precalculate_color(tpg, k);
-}
-
-/* 'odd' is true for pixels 1, 3, 5, etc. and false for pixels 0, 2, 4, etc. */
-static void gen_twopix(struct tpg_data *tpg,
- u8 buf[TPG_MAX_PLANES][8], int color, bool odd)
-{
- unsigned offset = odd * tpg->twopixelsize[0] / 2;
- u8 alpha = tpg->alpha_component;
- u8 r_y, g_u, b_v;
-
- if (tpg->alpha_red_only && color != TPG_COLOR_CSC_RED &&
- color != TPG_COLOR_100_RED &&
- color != TPG_COLOR_75_RED)
- alpha = 0;
- if (color == TPG_COLOR_RANDOM)
- precalculate_color(tpg, color);
- r_y = tpg->colors[color][0]; /* R or precalculated Y */
- g_u = tpg->colors[color][1]; /* G or precalculated U */
- b_v = tpg->colors[color][2]; /* B or precalculated V */
-
- switch (tpg->fourcc) {
- case V4L2_PIX_FMT_GREY:
- buf[0][offset] = r_y;
- break;
- case V4L2_PIX_FMT_Y16:
- /*
- * Ideally both bytes should be set to r_y, but then you won't
- * be able to detect endian problems. So keep it 0 except for
- * the corner case where r_y is 0xff so white really will be
- * white (0xffff).
- */
- buf[0][offset] = r_y == 0xff ? r_y : 0;
- buf[0][offset+1] = r_y;
- break;
- case V4L2_PIX_FMT_Y16_BE:
- /* See comment for V4L2_PIX_FMT_Y16 above */
- buf[0][offset] = r_y;
- buf[0][offset+1] = r_y == 0xff ? r_y : 0;
- break;
- case V4L2_PIX_FMT_YUV422M:
- case V4L2_PIX_FMT_YUV422P:
- case V4L2_PIX_FMT_YUV420:
- case V4L2_PIX_FMT_YUV420M:
- buf[0][offset] = r_y;
- if (odd) {
- buf[1][0] = (buf[1][0] + g_u) / 2;
- buf[2][0] = (buf[2][0] + b_v) / 2;
- buf[1][1] = buf[1][0];
- buf[2][1] = buf[2][0];
- break;
- }
- buf[1][0] = g_u;
- buf[2][0] = b_v;
- break;
- case V4L2_PIX_FMT_YVU422M:
- case V4L2_PIX_FMT_YVU420:
- case V4L2_PIX_FMT_YVU420M:
- buf[0][offset] = r_y;
- if (odd) {
- buf[1][0] = (buf[1][0] + b_v) / 2;
- buf[2][0] = (buf[2][0] + g_u) / 2;
- buf[1][1] = buf[1][0];
- buf[2][1] = buf[2][0];
- break;
- }
- buf[1][0] = b_v;
- buf[2][0] = g_u;
- break;
-
- case V4L2_PIX_FMT_NV12:
- case V4L2_PIX_FMT_NV12M:
- case V4L2_PIX_FMT_NV16:
- case V4L2_PIX_FMT_NV16M:
- buf[0][offset] = r_y;
- if (odd) {
- buf[1][0] = (buf[1][0] + g_u) / 2;
- buf[1][1] = (buf[1][1] + b_v) / 2;
- break;
- }
- buf[1][0] = g_u;
- buf[1][1] = b_v;
- break;
- case V4L2_PIX_FMT_NV21:
- case V4L2_PIX_FMT_NV21M:
- case V4L2_PIX_FMT_NV61:
- case V4L2_PIX_FMT_NV61M:
- buf[0][offset] = r_y;
- if (odd) {
- buf[1][0] = (buf[1][0] + b_v) / 2;
- buf[1][1] = (buf[1][1] + g_u) / 2;
- break;
- }
- buf[1][0] = b_v;
- buf[1][1] = g_u;
- break;
-
- case V4L2_PIX_FMT_YUV444M:
- buf[0][offset] = r_y;
- buf[1][offset] = g_u;
- buf[2][offset] = b_v;
- break;
-
- case V4L2_PIX_FMT_YVU444M:
- buf[0][offset] = r_y;
- buf[1][offset] = b_v;
- buf[2][offset] = g_u;
- break;
-
- case V4L2_PIX_FMT_NV24:
- buf[0][offset] = r_y;
- buf[1][2 * offset] = g_u;
- buf[1][2 * offset + 1] = b_v;
- break;
-
- case V4L2_PIX_FMT_NV42:
- buf[0][offset] = r_y;
- buf[1][2 * offset] = b_v;
- buf[1][2 * offset + 1] = g_u;
- break;
-
- case V4L2_PIX_FMT_YUYV:
- buf[0][offset] = r_y;
- if (odd) {
- buf[0][1] = (buf[0][1] + g_u) / 2;
- buf[0][3] = (buf[0][3] + b_v) / 2;
- break;
- }
- buf[0][1] = g_u;
- buf[0][3] = b_v;
- break;
- case V4L2_PIX_FMT_UYVY:
- buf[0][offset + 1] = r_y;
- if (odd) {
- buf[0][0] = (buf[0][0] + g_u) / 2;
- buf[0][2] = (buf[0][2] + b_v) / 2;
- break;
- }
- buf[0][0] = g_u;
- buf[0][2] = b_v;
- break;
- case V4L2_PIX_FMT_YVYU:
- buf[0][offset] = r_y;
- if (odd) {
- buf[0][1] = (buf[0][1] + b_v) / 2;
- buf[0][3] = (buf[0][3] + g_u) / 2;
- break;
- }
- buf[0][1] = b_v;
- buf[0][3] = g_u;
- break;
- case V4L2_PIX_FMT_VYUY:
- buf[0][offset + 1] = r_y;
- if (odd) {
- buf[0][0] = (buf[0][0] + b_v) / 2;
- buf[0][2] = (buf[0][2] + g_u) / 2;
- break;
- }
- buf[0][0] = b_v;
- buf[0][2] = g_u;
- break;
- case V4L2_PIX_FMT_RGB332:
- buf[0][offset] = (r_y << 5) | (g_u << 2) | b_v;
- break;
- case V4L2_PIX_FMT_YUV565:
- case V4L2_PIX_FMT_RGB565:
- buf[0][offset] = (g_u << 5) | b_v;
- buf[0][offset + 1] = (r_y << 3) | (g_u >> 3);
- break;
- case V4L2_PIX_FMT_RGB565X:
- buf[0][offset] = (r_y << 3) | (g_u >> 3);
- buf[0][offset + 1] = (g_u << 5) | b_v;
- break;
- case V4L2_PIX_FMT_RGB444:
- case V4L2_PIX_FMT_XRGB444:
- alpha = 0;
- /* fall through */
- case V4L2_PIX_FMT_YUV444:
- case V4L2_PIX_FMT_ARGB444:
- buf[0][offset] = (g_u << 4) | b_v;
- buf[0][offset + 1] = (alpha & 0xf0) | r_y;
- break;
- case V4L2_PIX_FMT_RGB555:
- case V4L2_PIX_FMT_XRGB555:
- alpha = 0;
- /* fall through */
- case V4L2_PIX_FMT_YUV555:
- case V4L2_PIX_FMT_ARGB555:
- buf[0][offset] = (g_u << 5) | b_v;
- buf[0][offset + 1] = (alpha & 0x80) | (r_y << 2) | (g_u >> 3);
- break;
- case V4L2_PIX_FMT_RGB555X:
- case V4L2_PIX_FMT_XRGB555X:
- alpha = 0;
- /* fall through */
- case V4L2_PIX_FMT_ARGB555X:
- buf[0][offset] = (alpha & 0x80) | (r_y << 2) | (g_u >> 3);
- buf[0][offset + 1] = (g_u << 5) | b_v;
- break;
- case V4L2_PIX_FMT_RGB24:
- buf[0][offset] = r_y;
- buf[0][offset + 1] = g_u;
- buf[0][offset + 2] = b_v;
- break;
- case V4L2_PIX_FMT_BGR24:
- buf[0][offset] = b_v;
- buf[0][offset + 1] = g_u;
- buf[0][offset + 2] = r_y;
- break;
- case V4L2_PIX_FMT_BGR666:
- buf[0][offset] = (b_v << 2) | (g_u >> 4);
- buf[0][offset + 1] = (g_u << 4) | (r_y >> 2);
- buf[0][offset + 2] = r_y << 6;
- buf[0][offset + 3] = 0;
- break;
- case V4L2_PIX_FMT_RGB32:
- case V4L2_PIX_FMT_XRGB32:
- alpha = 0;
- /* fall through */
- case V4L2_PIX_FMT_YUV32:
- case V4L2_PIX_FMT_ARGB32:
- buf[0][offset] = alpha;
- buf[0][offset + 1] = r_y;
- buf[0][offset + 2] = g_u;
- buf[0][offset + 3] = b_v;
- break;
- case V4L2_PIX_FMT_BGR32:
- case V4L2_PIX_FMT_XBGR32:
- alpha = 0;
- /* fall through */
- case V4L2_PIX_FMT_ABGR32:
- buf[0][offset] = b_v;
- buf[0][offset + 1] = g_u;
- buf[0][offset + 2] = r_y;
- buf[0][offset + 3] = alpha;
- break;
- case V4L2_PIX_FMT_SBGGR8:
- buf[0][offset] = odd ? g_u : b_v;
- buf[1][offset] = odd ? r_y : g_u;
- break;
- case V4L2_PIX_FMT_SGBRG8:
- buf[0][offset] = odd ? b_v : g_u;
- buf[1][offset] = odd ? g_u : r_y;
- break;
- case V4L2_PIX_FMT_SGRBG8:
- buf[0][offset] = odd ? r_y : g_u;
- buf[1][offset] = odd ? g_u : b_v;
- break;
- case V4L2_PIX_FMT_SRGGB8:
- buf[0][offset] = odd ? g_u : r_y;
- buf[1][offset] = odd ? b_v : g_u;
- break;
- case V4L2_PIX_FMT_SBGGR10:
- buf[0][offset] = odd ? g_u << 2 : b_v << 2;
- buf[0][offset + 1] = odd ? g_u >> 6 : b_v >> 6;
- buf[1][offset] = odd ? r_y << 2 : g_u << 2;
- buf[1][offset + 1] = odd ? r_y >> 6 : g_u >> 6;
- buf[0][offset] |= (buf[0][offset] >> 2) & 3;
- buf[1][offset] |= (buf[1][offset] >> 2) & 3;
- break;
- case V4L2_PIX_FMT_SGBRG10:
- buf[0][offset] = odd ? b_v << 2 : g_u << 2;
- buf[0][offset + 1] = odd ? b_v >> 6 : g_u >> 6;
- buf[1][offset] = odd ? g_u << 2 : r_y << 2;
- buf[1][offset + 1] = odd ? g_u >> 6 : r_y >> 6;
- buf[0][offset] |= (buf[0][offset] >> 2) & 3;
- buf[1][offset] |= (buf[1][offset] >> 2) & 3;
- break;
- case V4L2_PIX_FMT_SGRBG10:
- buf[0][offset] = odd ? r_y << 2 : g_u << 2;
- buf[0][offset + 1] = odd ? r_y >> 6 : g_u >> 6;
- buf[1][offset] = odd ? g_u << 2 : b_v << 2;
- buf[1][offset + 1] = odd ? g_u >> 6 : b_v >> 6;
- buf[0][offset] |= (buf[0][offset] >> 2) & 3;
- buf[1][offset] |= (buf[1][offset] >> 2) & 3;
- break;
- case V4L2_PIX_FMT_SRGGB10:
- buf[0][offset] = odd ? g_u << 2 : r_y << 2;
- buf[0][offset + 1] = odd ? g_u >> 6 : r_y >> 6;
- buf[1][offset] = odd ? b_v << 2 : g_u << 2;
- buf[1][offset + 1] = odd ? b_v >> 6 : g_u >> 6;
- buf[0][offset] |= (buf[0][offset] >> 2) & 3;
- buf[1][offset] |= (buf[1][offset] >> 2) & 3;
- break;
- case V4L2_PIX_FMT_SBGGR12:
- buf[0][offset] = odd ? g_u << 4 : b_v << 4;
- buf[0][offset + 1] = odd ? g_u >> 4 : b_v >> 4;
- buf[1][offset] = odd ? r_y << 4 : g_u << 4;
- buf[1][offset + 1] = odd ? r_y >> 4 : g_u >> 4;
- buf[0][offset] |= (buf[0][offset] >> 4) & 0xf;
- buf[1][offset] |= (buf[1][offset] >> 4) & 0xf;
- break;
- case V4L2_PIX_FMT_SGBRG12:
- buf[0][offset] = odd ? b_v << 4 : g_u << 4;
- buf[0][offset + 1] = odd ? b_v >> 4 : g_u >> 4;
- buf[1][offset] = odd ? g_u << 4 : r_y << 4;
- buf[1][offset + 1] = odd ? g_u >> 4 : r_y >> 4;
- buf[0][offset] |= (buf[0][offset] >> 4) & 0xf;
- buf[1][offset] |= (buf[1][offset] >> 4) & 0xf;
- break;
- case V4L2_PIX_FMT_SGRBG12:
- buf[0][offset] = odd ? r_y << 4 : g_u << 4;
- buf[0][offset + 1] = odd ? r_y >> 4 : g_u >> 4;
- buf[1][offset] = odd ? g_u << 4 : b_v << 4;
- buf[1][offset + 1] = odd ? g_u >> 4 : b_v >> 4;
- buf[0][offset] |= (buf[0][offset] >> 4) & 0xf;
- buf[1][offset] |= (buf[1][offset] >> 4) & 0xf;
- break;
- case V4L2_PIX_FMT_SRGGB12:
- buf[0][offset] = odd ? g_u << 4 : r_y << 4;
- buf[0][offset + 1] = odd ? g_u >> 4 : r_y >> 4;
- buf[1][offset] = odd ? b_v << 4 : g_u << 4;
- buf[1][offset + 1] = odd ? b_v >> 4 : g_u >> 4;
- buf[0][offset] |= (buf[0][offset] >> 4) & 0xf;
- buf[1][offset] |= (buf[1][offset] >> 4) & 0xf;
- break;
- }
-}
-
-unsigned tpg_g_interleaved_plane(const struct tpg_data *tpg, unsigned buf_line)
-{
- switch (tpg->fourcc) {
- case V4L2_PIX_FMT_SBGGR8:
- case V4L2_PIX_FMT_SGBRG8:
- case V4L2_PIX_FMT_SGRBG8:
- case V4L2_PIX_FMT_SRGGB8:
- case V4L2_PIX_FMT_SBGGR10:
- case V4L2_PIX_FMT_SGBRG10:
- case V4L2_PIX_FMT_SGRBG10:
- case V4L2_PIX_FMT_SRGGB10:
- case V4L2_PIX_FMT_SBGGR12:
- case V4L2_PIX_FMT_SGBRG12:
- case V4L2_PIX_FMT_SGRBG12:
- case V4L2_PIX_FMT_SRGGB12:
- return buf_line & 1;
- default:
- return 0;
- }
-}
-
-/* Return how many pattern lines are used by the current pattern. */
-static unsigned tpg_get_pat_lines(const struct tpg_data *tpg)
-{
- switch (tpg->pattern) {
- case TPG_PAT_CHECKERS_16X16:
- case TPG_PAT_CHECKERS_2X2:
- case TPG_PAT_CHECKERS_1X1:
- case TPG_PAT_COLOR_CHECKERS_2X2:
- case TPG_PAT_COLOR_CHECKERS_1X1:
- case TPG_PAT_ALTERNATING_HLINES:
- case TPG_PAT_CROSS_1_PIXEL:
- case TPG_PAT_CROSS_2_PIXELS:
- case TPG_PAT_CROSS_10_PIXELS:
- return 2;
- case TPG_PAT_100_COLORSQUARES:
- case TPG_PAT_100_HCOLORBAR:
- return 8;
- default:
- return 1;
- }
-}
-
-/* Which pattern line should be used for the given frame line. */
-static unsigned tpg_get_pat_line(const struct tpg_data *tpg, unsigned line)
-{
- switch (tpg->pattern) {
- case TPG_PAT_CHECKERS_16X16:
- return (line >> 4) & 1;
- case TPG_PAT_CHECKERS_1X1:
- case TPG_PAT_COLOR_CHECKERS_1X1:
- case TPG_PAT_ALTERNATING_HLINES:
- return line & 1;
- case TPG_PAT_CHECKERS_2X2:
- case TPG_PAT_COLOR_CHECKERS_2X2:
- return (line & 2) >> 1;
- case TPG_PAT_100_COLORSQUARES:
- case TPG_PAT_100_HCOLORBAR:
- return (line * 8) / tpg->src_height;
- case TPG_PAT_CROSS_1_PIXEL:
- return line == tpg->src_height / 2;
- case TPG_PAT_CROSS_2_PIXELS:
- return (line + 1) / 2 == tpg->src_height / 4;
- case TPG_PAT_CROSS_10_PIXELS:
- return (line + 10) / 20 == tpg->src_height / 40;
- default:
- return 0;
- }
-}
-
-/*
- * Which color should be used for the given pattern line and X coordinate.
- * Note: x is in the range 0 to 2 * tpg->src_width.
- */
-static enum tpg_color tpg_get_color(const struct tpg_data *tpg,
- unsigned pat_line, unsigned x)
-{
- /* Maximum number of bars are TPG_COLOR_MAX - otherwise, the input print code
- should be modified */
- static const enum tpg_color bars[3][8] = {
- /* Standard ITU-R 75% color bar sequence */
- { TPG_COLOR_CSC_WHITE, TPG_COLOR_75_YELLOW,
- TPG_COLOR_75_CYAN, TPG_COLOR_75_GREEN,
- TPG_COLOR_75_MAGENTA, TPG_COLOR_75_RED,
- TPG_COLOR_75_BLUE, TPG_COLOR_100_BLACK, },
- /* Standard ITU-R 100% color bar sequence */
- { TPG_COLOR_100_WHITE, TPG_COLOR_100_YELLOW,
- TPG_COLOR_100_CYAN, TPG_COLOR_100_GREEN,
- TPG_COLOR_100_MAGENTA, TPG_COLOR_100_RED,
- TPG_COLOR_100_BLUE, TPG_COLOR_100_BLACK, },
- /* Color bar sequence suitable to test CSC */
- { TPG_COLOR_CSC_WHITE, TPG_COLOR_CSC_YELLOW,
- TPG_COLOR_CSC_CYAN, TPG_COLOR_CSC_GREEN,
- TPG_COLOR_CSC_MAGENTA, TPG_COLOR_CSC_RED,
- TPG_COLOR_CSC_BLUE, TPG_COLOR_CSC_BLACK, },
- };
-
- switch (tpg->pattern) {
- case TPG_PAT_75_COLORBAR:
- case TPG_PAT_100_COLORBAR:
- case TPG_PAT_CSC_COLORBAR:
- return bars[tpg->pattern][((x * 8) / tpg->src_width) % 8];
- case TPG_PAT_100_COLORSQUARES:
- return bars[1][(pat_line + (x * 8) / tpg->src_width) % 8];
- case TPG_PAT_100_HCOLORBAR:
- return bars[1][pat_line];
- case TPG_PAT_BLACK:
- return TPG_COLOR_100_BLACK;
- case TPG_PAT_WHITE:
- return TPG_COLOR_100_WHITE;
- case TPG_PAT_RED:
- return TPG_COLOR_100_RED;
- case TPG_PAT_GREEN:
- return TPG_COLOR_100_GREEN;
- case TPG_PAT_BLUE:
- return TPG_COLOR_100_BLUE;
- case TPG_PAT_CHECKERS_16X16:
- return (((x >> 4) & 1) ^ (pat_line & 1)) ?
- TPG_COLOR_100_BLACK : TPG_COLOR_100_WHITE;
- case TPG_PAT_CHECKERS_1X1:
- return ((x & 1) ^ (pat_line & 1)) ?
- TPG_COLOR_100_WHITE : TPG_COLOR_100_BLACK;
- case TPG_PAT_COLOR_CHECKERS_1X1:
- return ((x & 1) ^ (pat_line & 1)) ?
- TPG_COLOR_100_RED : TPG_COLOR_100_BLUE;
- case TPG_PAT_CHECKERS_2X2:
- return (((x >> 1) & 1) ^ (pat_line & 1)) ?
- TPG_COLOR_100_WHITE : TPG_COLOR_100_BLACK;
- case TPG_PAT_COLOR_CHECKERS_2X2:
- return (((x >> 1) & 1) ^ (pat_line & 1)) ?
- TPG_COLOR_100_RED : TPG_COLOR_100_BLUE;
- case TPG_PAT_ALTERNATING_HLINES:
- return pat_line ? TPG_COLOR_100_WHITE : TPG_COLOR_100_BLACK;
- case TPG_PAT_ALTERNATING_VLINES:
- return (x & 1) ? TPG_COLOR_100_WHITE : TPG_COLOR_100_BLACK;
- case TPG_PAT_CROSS_1_PIXEL:
- if (pat_line || (x % tpg->src_width) == tpg->src_width / 2)
- return TPG_COLOR_100_BLACK;
- return TPG_COLOR_100_WHITE;
- case TPG_PAT_CROSS_2_PIXELS:
- if (pat_line || ((x % tpg->src_width) + 1) / 2 == tpg->src_width / 4)
- return TPG_COLOR_100_BLACK;
- return TPG_COLOR_100_WHITE;
- case TPG_PAT_CROSS_10_PIXELS:
- if (pat_line || ((x % tpg->src_width) + 10) / 20 == tpg->src_width / 40)
- return TPG_COLOR_100_BLACK;
- return TPG_COLOR_100_WHITE;
- case TPG_PAT_GRAY_RAMP:
- return TPG_COLOR_RAMP + ((x % tpg->src_width) * 256) / tpg->src_width;
- default:
- return TPG_COLOR_100_RED;
- }
-}
-
-/*
- * Given the pixel aspect ratio and video aspect ratio calculate the
- * coordinates of a centered square and the coordinates of the border of
- * the active video area. The coordinates are relative to the source
- * frame rectangle.
- */
-static void tpg_calculate_square_border(struct tpg_data *tpg)
-{
- unsigned w = tpg->src_width;
- unsigned h = tpg->src_height;
- unsigned sq_w, sq_h;
-
- sq_w = (w * 2 / 5) & ~1;
- if (((w - sq_w) / 2) & 1)
- sq_w += 2;
- sq_h = sq_w;
- tpg->square.width = sq_w;
- if (tpg->vid_aspect == TPG_VIDEO_ASPECT_16X9_ANAMORPHIC) {
- unsigned ana_sq_w = (sq_w / 4) * 3;
-
- if (((w - ana_sq_w) / 2) & 1)
- ana_sq_w += 2;
- tpg->square.width = ana_sq_w;
- }
- tpg->square.left = (w - tpg->square.width) / 2;
- if (tpg->pix_aspect == TPG_PIXEL_ASPECT_NTSC)
- sq_h = sq_w * 10 / 11;
- else if (tpg->pix_aspect == TPG_PIXEL_ASPECT_PAL)
- sq_h = sq_w * 59 / 54;
- tpg->square.height = sq_h;
- tpg->square.top = (h - sq_h) / 2;
- tpg->border.left = 0;
- tpg->border.width = w;
- tpg->border.top = 0;
- tpg->border.height = h;
- switch (tpg->vid_aspect) {
- case TPG_VIDEO_ASPECT_4X3:
- if (tpg->pix_aspect)
- return;
- if (3 * w >= 4 * h) {
- tpg->border.width = ((4 * h) / 3) & ~1;
- if (((w - tpg->border.width) / 2) & ~1)
- tpg->border.width -= 2;
- tpg->border.left = (w - tpg->border.width) / 2;
- break;
- }
- tpg->border.height = ((3 * w) / 4) & ~1;
- tpg->border.top = (h - tpg->border.height) / 2;
- break;
- case TPG_VIDEO_ASPECT_14X9_CENTRE:
- if (tpg->pix_aspect) {
- tpg->border.height = tpg->pix_aspect == TPG_PIXEL_ASPECT_NTSC ? 420 : 506;
- tpg->border.top = (h - tpg->border.height) / 2;
- break;
- }
- if (9 * w >= 14 * h) {
- tpg->border.width = ((14 * h) / 9) & ~1;
- if (((w - tpg->border.width) / 2) & ~1)
- tpg->border.width -= 2;
- tpg->border.left = (w - tpg->border.width) / 2;
- break;
- }
- tpg->border.height = ((9 * w) / 14) & ~1;
- tpg->border.top = (h - tpg->border.height) / 2;
- break;
- case TPG_VIDEO_ASPECT_16X9_CENTRE:
- if (tpg->pix_aspect) {
- tpg->border.height = tpg->pix_aspect == TPG_PIXEL_ASPECT_NTSC ? 368 : 442;
- tpg->border.top = (h - tpg->border.height) / 2;
- break;
- }
- if (9 * w >= 16 * h) {
- tpg->border.width = ((16 * h) / 9) & ~1;
- if (((w - tpg->border.width) / 2) & ~1)
- tpg->border.width -= 2;
- tpg->border.left = (w - tpg->border.width) / 2;
- break;
- }
- tpg->border.height = ((9 * w) / 16) & ~1;
- tpg->border.top = (h - tpg->border.height) / 2;
- break;
- default:
- break;
- }
-}
-
-static void tpg_precalculate_line(struct tpg_data *tpg)
-{
- enum tpg_color contrast;
- u8 pix[TPG_MAX_PLANES][8];
- unsigned pat;
- unsigned p;
- unsigned x;
-
- switch (tpg->pattern) {
- case TPG_PAT_GREEN:
- contrast = TPG_COLOR_100_RED;
- break;
- case TPG_PAT_CSC_COLORBAR:
- contrast = TPG_COLOR_CSC_GREEN;
- break;
- default:
- contrast = TPG_COLOR_100_GREEN;
- break;
- }
-
- for (pat = 0; pat < tpg_get_pat_lines(tpg); pat++) {
- /* Coarse scaling with Bresenham */
- unsigned int_part = tpg->src_width / tpg->scaled_width;
- unsigned fract_part = tpg->src_width % tpg->scaled_width;
- unsigned src_x = 0;
- unsigned error = 0;
-
- for (x = 0; x < tpg->scaled_width * 2; x += 2) {
- unsigned real_x = src_x;
- enum tpg_color color1, color2;
-
- real_x = tpg->hflip ? tpg->src_width * 2 - real_x - 2 : real_x;
- color1 = tpg_get_color(tpg, pat, real_x);
-
- src_x += int_part;
- error += fract_part;
- if (error >= tpg->scaled_width) {
- error -= tpg->scaled_width;
- src_x++;
- }
-
- real_x = src_x;
- real_x = tpg->hflip ? tpg->src_width * 2 - real_x - 2 : real_x;
- color2 = tpg_get_color(tpg, pat, real_x);
-
- src_x += int_part;
- error += fract_part;
- if (error >= tpg->scaled_width) {
- error -= tpg->scaled_width;
- src_x++;
- }
-
- gen_twopix(tpg, pix, tpg->hflip ? color2 : color1, 0);
- gen_twopix(tpg, pix, tpg->hflip ? color1 : color2, 1);
- for (p = 0; p < tpg->planes; p++) {
- unsigned twopixsize = tpg->twopixelsize[p];
- unsigned hdiv = tpg->hdownsampling[p];
- u8 *pos = tpg->lines[pat][p] + tpg_hdiv(tpg, p, x);
-
- memcpy(pos, pix[p], twopixsize / hdiv);
- }
- }
- }
-
- if (tpg->vdownsampling[tpg->planes - 1] > 1) {
- unsigned pat_lines = tpg_get_pat_lines(tpg);
-
- for (pat = 0; pat < pat_lines; pat++) {
- unsigned next_pat = (pat + 1) % pat_lines;
-
- for (p = 1; p < tpg->planes; p++) {
- unsigned w = tpg_hdiv(tpg, p, tpg->scaled_width * 2);
- u8 *pos1 = tpg->lines[pat][p];
- u8 *pos2 = tpg->lines[next_pat][p];
- u8 *dest = tpg->downsampled_lines[pat][p];
-
- for (x = 0; x < w; x++, pos1++, pos2++, dest++)
- *dest = ((u16)*pos1 + (u16)*pos2) / 2;
- }
- }
- }
-
- gen_twopix(tpg, pix, contrast, 0);
- gen_twopix(tpg, pix, contrast, 1);
- for (p = 0; p < tpg->planes; p++) {
- unsigned twopixsize = tpg->twopixelsize[p];
- u8 *pos = tpg->contrast_line[p];
-
- for (x = 0; x < tpg->scaled_width; x += 2, pos += twopixsize)
- memcpy(pos, pix[p], twopixsize);
- }
-
- gen_twopix(tpg, pix, TPG_COLOR_100_BLACK, 0);
- gen_twopix(tpg, pix, TPG_COLOR_100_BLACK, 1);
- for (p = 0; p < tpg->planes; p++) {
- unsigned twopixsize = tpg->twopixelsize[p];
- u8 *pos = tpg->black_line[p];
-
- for (x = 0; x < tpg->scaled_width; x += 2, pos += twopixsize)
- memcpy(pos, pix[p], twopixsize);
- }
-
- for (x = 0; x < tpg->scaled_width * 2; x += 2) {
- gen_twopix(tpg, pix, TPG_COLOR_RANDOM, 0);
- gen_twopix(tpg, pix, TPG_COLOR_RANDOM, 1);
- for (p = 0; p < tpg->planes; p++) {
- unsigned twopixsize = tpg->twopixelsize[p];
- u8 *pos = tpg->random_line[p] + x * twopixsize / 2;
-
- memcpy(pos, pix[p], twopixsize);
- }
- }
-
- gen_twopix(tpg, tpg->textbg, TPG_COLOR_TEXTBG, 0);
- gen_twopix(tpg, tpg->textbg, TPG_COLOR_TEXTBG, 1);
- gen_twopix(tpg, tpg->textfg, TPG_COLOR_TEXTFG, 0);
- gen_twopix(tpg, tpg->textfg, TPG_COLOR_TEXTFG, 1);
-}
-
-/* need this to do rgb24 rendering */
-typedef struct { u16 __; u8 _; } __packed x24;
-
-#define PRINTSTR(PIXTYPE) do { \
- unsigned vdiv = tpg->vdownsampling[p]; \
- unsigned hdiv = tpg->hdownsampling[p]; \
- int line; \
- PIXTYPE fg; \
- PIXTYPE bg; \
- memcpy(&fg, tpg->textfg[p], sizeof(PIXTYPE)); \
- memcpy(&bg, tpg->textbg[p], sizeof(PIXTYPE)); \
- \
- for (line = first; line < 16; line += vdiv * step) { \
- int l = tpg->vflip ? 15 - line : line; \
- PIXTYPE *pos = (PIXTYPE *)(basep[p][(line / vdiv) & 1] + \
- ((y * step + l) / (vdiv * div)) * tpg->bytesperline[p] + \
- (x / hdiv) * sizeof(PIXTYPE)); \
- unsigned s; \
- \
- for (s = 0; s < len; s++) { \
- u8 chr = font8x16[text[s] * 16 + line]; \
- \
- if (hdiv == 2 && tpg->hflip) { \
- pos[3] = (chr & (0x01 << 6) ? fg : bg); \
- pos[2] = (chr & (0x01 << 4) ? fg : bg); \
- pos[1] = (chr & (0x01 << 2) ? fg : bg); \
- pos[0] = (chr & (0x01 << 0) ? fg : bg); \
- } else if (hdiv == 2) { \
- pos[0] = (chr & (0x01 << 7) ? fg : bg); \
- pos[1] = (chr & (0x01 << 5) ? fg : bg); \
- pos[2] = (chr & (0x01 << 3) ? fg : bg); \
- pos[3] = (chr & (0x01 << 1) ? fg : bg); \
- } else if (tpg->hflip) { \
- pos[7] = (chr & (0x01 << 7) ? fg : bg); \
- pos[6] = (chr & (0x01 << 6) ? fg : bg); \
- pos[5] = (chr & (0x01 << 5) ? fg : bg); \
- pos[4] = (chr & (0x01 << 4) ? fg : bg); \
- pos[3] = (chr & (0x01 << 3) ? fg : bg); \
- pos[2] = (chr & (0x01 << 2) ? fg : bg); \
- pos[1] = (chr & (0x01 << 1) ? fg : bg); \
- pos[0] = (chr & (0x01 << 0) ? fg : bg); \
- } else { \
- pos[0] = (chr & (0x01 << 7) ? fg : bg); \
- pos[1] = (chr & (0x01 << 6) ? fg : bg); \
- pos[2] = (chr & (0x01 << 5) ? fg : bg); \
- pos[3] = (chr & (0x01 << 4) ? fg : bg); \
- pos[4] = (chr & (0x01 << 3) ? fg : bg); \
- pos[5] = (chr & (0x01 << 2) ? fg : bg); \
- pos[6] = (chr & (0x01 << 1) ? fg : bg); \
- pos[7] = (chr & (0x01 << 0) ? fg : bg); \
- } \
- \
- pos += (tpg->hflip ? -8 : 8) / hdiv; \
- } \
- } \
-} while (0)
-
-static noinline void tpg_print_str_2(const struct tpg_data *tpg, u8 *basep[TPG_MAX_PLANES][2],
- unsigned p, unsigned first, unsigned div, unsigned step,
- int y, int x, char *text, unsigned len)
-{
- PRINTSTR(u8);
-}
-
-static noinline void tpg_print_str_4(const struct tpg_data *tpg, u8 *basep[TPG_MAX_PLANES][2],
- unsigned p, unsigned first, unsigned div, unsigned step,
- int y, int x, char *text, unsigned len)
-{
- PRINTSTR(u16);
-}
-
-static noinline void tpg_print_str_6(const struct tpg_data *tpg, u8 *basep[TPG_MAX_PLANES][2],
- unsigned p, unsigned first, unsigned div, unsigned step,
- int y, int x, char *text, unsigned len)
-{
- PRINTSTR(x24);
-}
-
-static noinline void tpg_print_str_8(const struct tpg_data *tpg, u8 *basep[TPG_MAX_PLANES][2],
- unsigned p, unsigned first, unsigned div, unsigned step,
- int y, int x, char *text, unsigned len)
-{
- PRINTSTR(u32);
-}
-
-void tpg_gen_text(const struct tpg_data *tpg, u8 *basep[TPG_MAX_PLANES][2],
- int y, int x, char *text)
-{
- unsigned step = V4L2_FIELD_HAS_T_OR_B(tpg->field) ? 2 : 1;
- unsigned div = step;
- unsigned first = 0;
- unsigned len = strlen(text);
- unsigned p;
-
- if (font8x16 == NULL || basep == NULL)
- return;
-
- /* Checks if it is possible to show string */
- if (y + 16 >= tpg->compose.height || x + 8 >= tpg->compose.width)
- return;
-
- if (len > (tpg->compose.width - x) / 8)
- len = (tpg->compose.width - x) / 8;
- if (tpg->vflip)
- y = tpg->compose.height - y - 16;
- if (tpg->hflip)
- x = tpg->compose.width - x - 8;
- y += tpg->compose.top;
- x += tpg->compose.left;
- if (tpg->field == V4L2_FIELD_BOTTOM)
- first = 1;
- else if (tpg->field == V4L2_FIELD_SEQ_TB || tpg->field == V4L2_FIELD_SEQ_BT)
- div = 2;
-
- for (p = 0; p < tpg->planes; p++) {
- /* Print text */
- switch (tpg->twopixelsize[p]) {
- case 2:
- tpg_print_str_2(tpg, basep, p, first, div, step, y, x,
- text, len);
- break;
- case 4:
- tpg_print_str_4(tpg, basep, p, first, div, step, y, x,
- text, len);
- break;
- case 6:
- tpg_print_str_6(tpg, basep, p, first, div, step, y, x,
- text, len);
- break;
- case 8:
- tpg_print_str_8(tpg, basep, p, first, div, step, y, x,
- text, len);
- break;
- }
- }
-}
-
-void tpg_update_mv_step(struct tpg_data *tpg)
-{
- int factor = tpg->mv_hor_mode > TPG_MOVE_NONE ? -1 : 1;
-
- if (tpg->hflip)
- factor = -factor;
- switch (tpg->mv_hor_mode) {
- case TPG_MOVE_NEG_FAST:
- case TPG_MOVE_POS_FAST:
- tpg->mv_hor_step = ((tpg->src_width + 319) / 320) * 4;
- break;
- case TPG_MOVE_NEG:
- case TPG_MOVE_POS:
- tpg->mv_hor_step = ((tpg->src_width + 639) / 640) * 4;
- break;
- case TPG_MOVE_NEG_SLOW:
- case TPG_MOVE_POS_SLOW:
- tpg->mv_hor_step = 2;
- break;
- case TPG_MOVE_NONE:
- tpg->mv_hor_step = 0;
- break;
- }
- if (factor < 0)
- tpg->mv_hor_step = tpg->src_width - tpg->mv_hor_step;
-
- factor = tpg->mv_vert_mode > TPG_MOVE_NONE ? -1 : 1;
- switch (tpg->mv_vert_mode) {
- case TPG_MOVE_NEG_FAST:
- case TPG_MOVE_POS_FAST:
- tpg->mv_vert_step = ((tpg->src_width + 319) / 320) * 4;
- break;
- case TPG_MOVE_NEG:
- case TPG_MOVE_POS:
- tpg->mv_vert_step = ((tpg->src_width + 639) / 640) * 4;
- break;
- case TPG_MOVE_NEG_SLOW:
- case TPG_MOVE_POS_SLOW:
- tpg->mv_vert_step = 1;
- break;
- case TPG_MOVE_NONE:
- tpg->mv_vert_step = 0;
- break;
- }
- if (factor < 0)
- tpg->mv_vert_step = tpg->src_height - tpg->mv_vert_step;
-}
-
-/* Map the line number relative to the crop rectangle to a frame line number */
-static unsigned tpg_calc_frameline(const struct tpg_data *tpg, unsigned src_y,
- unsigned field)
-{
- switch (field) {
- case V4L2_FIELD_TOP:
- return tpg->crop.top + src_y * 2;
- case V4L2_FIELD_BOTTOM:
- return tpg->crop.top + src_y * 2 + 1;
- default:
- return src_y + tpg->crop.top;
- }
-}
-
-/*
- * Map the line number relative to the compose rectangle to a destination
- * buffer line number.
- */
-static unsigned tpg_calc_buffer_line(const struct tpg_data *tpg, unsigned y,
- unsigned field)
-{
- y += tpg->compose.top;
- switch (field) {
- case V4L2_FIELD_SEQ_TB:
- if (y & 1)
- return tpg->buf_height / 2 + y / 2;
- return y / 2;
- case V4L2_FIELD_SEQ_BT:
- if (y & 1)
- return y / 2;
- return tpg->buf_height / 2 + y / 2;
- default:
- return y;
- }
-}
-
-static void tpg_recalc(struct tpg_data *tpg)
-{
- if (tpg->recalc_colors) {
- tpg->recalc_colors = false;
- tpg->recalc_lines = true;
- tpg->real_xfer_func = tpg->xfer_func;
- tpg->real_ycbcr_enc = tpg->ycbcr_enc;
- tpg->real_quantization = tpg->quantization;
-
- if (tpg->xfer_func == V4L2_XFER_FUNC_DEFAULT)
- tpg->real_xfer_func =
- V4L2_MAP_XFER_FUNC_DEFAULT(tpg->colorspace);
-
- if (tpg->ycbcr_enc == V4L2_YCBCR_ENC_DEFAULT)
- tpg->real_ycbcr_enc =
- V4L2_MAP_YCBCR_ENC_DEFAULT(tpg->colorspace);
-
- if (tpg->quantization == V4L2_QUANTIZATION_DEFAULT)
- tpg->real_quantization =
- V4L2_MAP_QUANTIZATION_DEFAULT(!tpg->is_yuv,
- tpg->colorspace, tpg->real_ycbcr_enc);
-
- tpg_precalculate_colors(tpg);
- }
- if (tpg->recalc_square_border) {
- tpg->recalc_square_border = false;
- tpg_calculate_square_border(tpg);
- }
- if (tpg->recalc_lines) {
- tpg->recalc_lines = false;
- tpg_precalculate_line(tpg);
- }
-}
-
-void tpg_calc_text_basep(struct tpg_data *tpg,
- u8 *basep[TPG_MAX_PLANES][2], unsigned p, u8 *vbuf)
-{
- unsigned stride = tpg->bytesperline[p];
- unsigned h = tpg->buf_height;
-
- tpg_recalc(tpg);
-
- basep[p][0] = vbuf;
- basep[p][1] = vbuf;
- h /= tpg->vdownsampling[p];
- if (tpg->field == V4L2_FIELD_SEQ_TB)
- basep[p][1] += h * stride / 2;
- else if (tpg->field == V4L2_FIELD_SEQ_BT)
- basep[p][0] += h * stride / 2;
- if (p == 0 && tpg->interleaved)
- tpg_calc_text_basep(tpg, basep, 1, vbuf);
-}
-
-static int tpg_pattern_avg(const struct tpg_data *tpg,
- unsigned pat1, unsigned pat2)
-{
- unsigned pat_lines = tpg_get_pat_lines(tpg);
-
- if (pat1 == (pat2 + 1) % pat_lines)
- return pat2;
- if (pat2 == (pat1 + 1) % pat_lines)
- return pat1;
- return -1;
-}
-
-void tpg_log_status(struct tpg_data *tpg)
-{
- pr_info("tpg source WxH: %ux%u (%s)\n",
- tpg->src_width, tpg->src_height,
- tpg->is_yuv ? "YCbCr" : "RGB");
- pr_info("tpg field: %u\n", tpg->field);
- pr_info("tpg crop: %ux%u@%dx%d\n", tpg->crop.width, tpg->crop.height,
- tpg->crop.left, tpg->crop.top);
- pr_info("tpg compose: %ux%u@%dx%d\n", tpg->compose.width, tpg->compose.height,
- tpg->compose.left, tpg->compose.top);
- pr_info("tpg colorspace: %d\n", tpg->colorspace);
- pr_info("tpg transfer function: %d/%d\n", tpg->xfer_func, tpg->real_xfer_func);
- pr_info("tpg Y'CbCr encoding: %d/%d\n", tpg->ycbcr_enc, tpg->real_ycbcr_enc);
- pr_info("tpg quantization: %d/%d\n", tpg->quantization, tpg->real_quantization);
- pr_info("tpg RGB range: %d/%d\n", tpg->rgb_range, tpg->real_rgb_range);
-}
-
-/*
- * This struct contains common parameters used by both the drawing of the
- * test pattern and the drawing of the extras (borders, square, etc.)
- */
-struct tpg_draw_params {
- /* common data */
- bool is_tv;
- bool is_60hz;
- unsigned twopixsize;
- unsigned img_width;
- unsigned stride;
- unsigned hmax;
- unsigned frame_line;
- unsigned frame_line_next;
-
- /* test pattern */
- unsigned mv_hor_old;
- unsigned mv_hor_new;
- unsigned mv_vert_old;
- unsigned mv_vert_new;
-
- /* extras */
- unsigned wss_width;
- unsigned wss_random_offset;
- unsigned sav_eav_f;
- unsigned left_pillar_width;
- unsigned right_pillar_start;
-};
-
-static void tpg_fill_params_pattern(const struct tpg_data *tpg, unsigned p,
- struct tpg_draw_params *params)
-{
- params->mv_hor_old =
- tpg_hscale_div(tpg, p, tpg->mv_hor_count % tpg->src_width);
- params->mv_hor_new =
- tpg_hscale_div(tpg, p, (tpg->mv_hor_count + tpg->mv_hor_step) %
- tpg->src_width);
- params->mv_vert_old = tpg->mv_vert_count % tpg->src_height;
- params->mv_vert_new =
- (tpg->mv_vert_count + tpg->mv_vert_step) % tpg->src_height;
-}
-
-static void tpg_fill_params_extras(const struct tpg_data *tpg,
- unsigned p,
- struct tpg_draw_params *params)
-{
- unsigned left_pillar_width = 0;
- unsigned right_pillar_start = params->img_width;
-
- params->wss_width = tpg->crop.left < tpg->src_width / 2 ?
- tpg->src_width / 2 - tpg->crop.left : 0;
- if (params->wss_width > tpg->crop.width)
- params->wss_width = tpg->crop.width;
- params->wss_width = tpg_hscale_div(tpg, p, params->wss_width);
- params->wss_random_offset =
- params->twopixsize * prandom_u32_max(tpg->src_width / 2);
-
- if (tpg->crop.left < tpg->border.left) {
- left_pillar_width = tpg->border.left - tpg->crop.left;
- if (left_pillar_width > tpg->crop.width)
- left_pillar_width = tpg->crop.width;
- left_pillar_width = tpg_hscale_div(tpg, p, left_pillar_width);
- }
- params->left_pillar_width = left_pillar_width;
-
- if (tpg->crop.left + tpg->crop.width >
- tpg->border.left + tpg->border.width) {
- right_pillar_start =
- tpg->border.left + tpg->border.width - tpg->crop.left;
- right_pillar_start =
- tpg_hscale_div(tpg, p, right_pillar_start);
- if (right_pillar_start > params->img_width)
- right_pillar_start = params->img_width;
- }
- params->right_pillar_start = right_pillar_start;
-
- params->sav_eav_f = tpg->field ==
- (params->is_60hz ? V4L2_FIELD_TOP : V4L2_FIELD_BOTTOM);
-}
-
-static void tpg_fill_plane_extras(const struct tpg_data *tpg,
- const struct tpg_draw_params *params,
- unsigned p, unsigned h, u8 *vbuf)
-{
- unsigned twopixsize = params->twopixsize;
- unsigned img_width = params->img_width;
- unsigned frame_line = params->frame_line;
- const struct v4l2_rect *sq = &tpg->square;
- const struct v4l2_rect *b = &tpg->border;
- const struct v4l2_rect *c = &tpg->crop;
-
- if (params->is_tv && !params->is_60hz &&
- frame_line == 0 && params->wss_width) {
- /*
- * Replace the first half of the top line of a 50 Hz frame
- * with random data to simulate a WSS signal.
- */
- u8 *wss = tpg->random_line[p] + params->wss_random_offset;
-
- memcpy(vbuf, wss, params->wss_width);
- }
-
- if (tpg->show_border && frame_line >= b->top &&
- frame_line < b->top + b->height) {
- unsigned bottom = b->top + b->height - 1;
- unsigned left = params->left_pillar_width;
- unsigned right = params->right_pillar_start;
-
- if (frame_line == b->top || frame_line == b->top + 1 ||
- frame_line == bottom || frame_line == bottom - 1) {
- memcpy(vbuf + left, tpg->contrast_line[p],
- right - left);
- } else {
- if (b->left >= c->left &&
- b->left < c->left + c->width)
- memcpy(vbuf + left,
- tpg->contrast_line[p], twopixsize);
- if (b->left + b->width > c->left &&
- b->left + b->width <= c->left + c->width)
- memcpy(vbuf + right - twopixsize,
- tpg->contrast_line[p], twopixsize);
- }
- }
- if (tpg->qual != TPG_QUAL_NOISE && frame_line >= b->top &&
- frame_line < b->top + b->height) {
- memcpy(vbuf, tpg->black_line[p], params->left_pillar_width);
- memcpy(vbuf + params->right_pillar_start, tpg->black_line[p],
- img_width - params->right_pillar_start);
- }
- if (tpg->show_square && frame_line >= sq->top &&
- frame_line < sq->top + sq->height &&
- sq->left < c->left + c->width &&
- sq->left + sq->width >= c->left) {
- unsigned left = sq->left;
- unsigned width = sq->width;
-
- if (c->left > left) {
- width -= c->left - left;
- left = c->left;
- }
- if (c->left + c->width < left + width)
- width -= left + width - c->left - c->width;
- left -= c->left;
- left = tpg_hscale_div(tpg, p, left);
- width = tpg_hscale_div(tpg, p, width);
- memcpy(vbuf + left, tpg->contrast_line[p], width);
- }
- if (tpg->insert_sav) {
- unsigned offset = tpg_hdiv(tpg, p, tpg->compose.width / 3);
- u8 *p = vbuf + offset;
- unsigned vact = 0, hact = 0;
-
- p[0] = 0xff;
- p[1] = 0;
- p[2] = 0;
- p[3] = 0x80 | (params->sav_eav_f << 6) |
- (vact << 5) | (hact << 4) |
- ((hact ^ vact) << 3) |
- ((hact ^ params->sav_eav_f) << 2) |
- ((params->sav_eav_f ^ vact) << 1) |
- (hact ^ vact ^ params->sav_eav_f);
- }
- if (tpg->insert_eav) {
- unsigned offset = tpg_hdiv(tpg, p, tpg->compose.width * 2 / 3);
- u8 *p = vbuf + offset;
- unsigned vact = 0, hact = 1;
-
- p[0] = 0xff;
- p[1] = 0;
- p[2] = 0;
- p[3] = 0x80 | (params->sav_eav_f << 6) |
- (vact << 5) | (hact << 4) |
- ((hact ^ vact) << 3) |
- ((hact ^ params->sav_eav_f) << 2) |
- ((params->sav_eav_f ^ vact) << 1) |
- (hact ^ vact ^ params->sav_eav_f);
- }
-}
-
-static void tpg_fill_plane_pattern(const struct tpg_data *tpg,
- const struct tpg_draw_params *params,
- unsigned p, unsigned h, u8 *vbuf)
-{
- unsigned twopixsize = params->twopixsize;
- unsigned img_width = params->img_width;
- unsigned mv_hor_old = params->mv_hor_old;
- unsigned mv_hor_new = params->mv_hor_new;
- unsigned mv_vert_old = params->mv_vert_old;
- unsigned mv_vert_new = params->mv_vert_new;
- unsigned frame_line = params->frame_line;
- unsigned frame_line_next = params->frame_line_next;
- unsigned line_offset = tpg_hscale_div(tpg, p, tpg->crop.left);
- bool even;
- bool fill_blank = false;
- unsigned pat_line_old;
- unsigned pat_line_new;
- u8 *linestart_older;
- u8 *linestart_newer;
- u8 *linestart_top;
- u8 *linestart_bottom;
-
- even = !(frame_line & 1);
-
- if (h >= params->hmax) {
- if (params->hmax == tpg->compose.height)
- return;
- if (!tpg->perc_fill_blank)
- return;
- fill_blank = true;
- }
-
- if (tpg->vflip) {
- frame_line = tpg->src_height - frame_line - 1;
- frame_line_next = tpg->src_height - frame_line_next - 1;
- }
-
- if (fill_blank) {
- linestart_older = tpg->contrast_line[p];
- linestart_newer = tpg->contrast_line[p];
- } else if (tpg->qual != TPG_QUAL_NOISE &&
- (frame_line < tpg->border.top ||
- frame_line >= tpg->border.top + tpg->border.height)) {
- linestart_older = tpg->black_line[p];
- linestart_newer = tpg->black_line[p];
- } else if (tpg->pattern == TPG_PAT_NOISE || tpg->qual == TPG_QUAL_NOISE) {
- linestart_older = tpg->random_line[p] +
- twopixsize * prandom_u32_max(tpg->src_width / 2);
- linestart_newer = tpg->random_line[p] +
- twopixsize * prandom_u32_max(tpg->src_width / 2);
- } else {
- unsigned frame_line_old =
- (frame_line + mv_vert_old) % tpg->src_height;
- unsigned frame_line_new =
- (frame_line + mv_vert_new) % tpg->src_height;
- unsigned pat_line_next_old;
- unsigned pat_line_next_new;
-
- pat_line_old = tpg_get_pat_line(tpg, frame_line_old);
- pat_line_new = tpg_get_pat_line(tpg, frame_line_new);
- linestart_older = tpg->lines[pat_line_old][p] + mv_hor_old;
- linestart_newer = tpg->lines[pat_line_new][p] + mv_hor_new;
-
- if (tpg->vdownsampling[p] > 1 && frame_line != frame_line_next) {
- int avg_pat;
-
- /*
- * Now decide whether we need to use downsampled_lines[].
- * That's necessary if the two lines use different patterns.
- */
- pat_line_next_old = tpg_get_pat_line(tpg,
- (frame_line_next + mv_vert_old) % tpg->src_height);
- pat_line_next_new = tpg_get_pat_line(tpg,
- (frame_line_next + mv_vert_new) % tpg->src_height);
-
- switch (tpg->field) {
- case V4L2_FIELD_INTERLACED:
- case V4L2_FIELD_INTERLACED_BT:
- case V4L2_FIELD_INTERLACED_TB:
- avg_pat = tpg_pattern_avg(tpg, pat_line_old, pat_line_new);
- if (avg_pat < 0)
- break;
- linestart_older = tpg->downsampled_lines[avg_pat][p] + mv_hor_old;
- linestart_newer = linestart_older;
- break;
- case V4L2_FIELD_NONE:
- case V4L2_FIELD_TOP:
- case V4L2_FIELD_BOTTOM:
- case V4L2_FIELD_SEQ_BT:
- case V4L2_FIELD_SEQ_TB:
- avg_pat = tpg_pattern_avg(tpg, pat_line_old, pat_line_next_old);
- if (avg_pat >= 0)
- linestart_older = tpg->downsampled_lines[avg_pat][p] +
- mv_hor_old;
- avg_pat = tpg_pattern_avg(tpg, pat_line_new, pat_line_next_new);
- if (avg_pat >= 0)
- linestart_newer = tpg->downsampled_lines[avg_pat][p] +
- mv_hor_new;
- break;
- }
- }
- linestart_older += line_offset;
- linestart_newer += line_offset;
- }
- if (tpg->field_alternate) {
- linestart_top = linestart_bottom = linestart_older;
- } else if (params->is_60hz) {
- linestart_top = linestart_newer;
- linestart_bottom = linestart_older;
- } else {
- linestart_top = linestart_older;
- linestart_bottom = linestart_newer;
- }
-
- switch (tpg->field) {
- case V4L2_FIELD_INTERLACED:
- case V4L2_FIELD_INTERLACED_TB:
- case V4L2_FIELD_SEQ_TB:
- case V4L2_FIELD_SEQ_BT:
- if (even)
- memcpy(vbuf, linestart_top, img_width);
- else
- memcpy(vbuf, linestart_bottom, img_width);
- break;
- case V4L2_FIELD_INTERLACED_BT:
- if (even)
- memcpy(vbuf, linestart_bottom, img_width);
- else
- memcpy(vbuf, linestart_top, img_width);
- break;
- case V4L2_FIELD_TOP:
- memcpy(vbuf, linestart_top, img_width);
- break;
- case V4L2_FIELD_BOTTOM:
- memcpy(vbuf, linestart_bottom, img_width);
- break;
- case V4L2_FIELD_NONE:
- default:
- memcpy(vbuf, linestart_older, img_width);
- break;
- }
-}
-
-void tpg_fill_plane_buffer(struct tpg_data *tpg, v4l2_std_id std,
- unsigned p, u8 *vbuf)
-{
- struct tpg_draw_params params;
- unsigned factor = V4L2_FIELD_HAS_T_OR_B(tpg->field) ? 2 : 1;
-
- /* Coarse scaling with Bresenham */
- unsigned int_part = (tpg->crop.height / factor) / tpg->compose.height;
- unsigned fract_part = (tpg->crop.height / factor) % tpg->compose.height;
- unsigned src_y = 0;
- unsigned error = 0;
- unsigned h;
-
- tpg_recalc(tpg);
-
- params.is_tv = std;
- params.is_60hz = std & V4L2_STD_525_60;
- params.twopixsize = tpg->twopixelsize[p];
- params.img_width = tpg_hdiv(tpg, p, tpg->compose.width);
- params.stride = tpg->bytesperline[p];
- params.hmax = (tpg->compose.height * tpg->perc_fill) / 100;
-
- tpg_fill_params_pattern(tpg, p, ¶ms);
- tpg_fill_params_extras(tpg, p, ¶ms);
-
- vbuf += tpg_hdiv(tpg, p, tpg->compose.left);
-
- for (h = 0; h < tpg->compose.height; h++) {
- unsigned buf_line;
-
- params.frame_line = tpg_calc_frameline(tpg, src_y, tpg->field);
- params.frame_line_next = params.frame_line;
- buf_line = tpg_calc_buffer_line(tpg, h, tpg->field);
- src_y += int_part;
- error += fract_part;
- if (error >= tpg->compose.height) {
- error -= tpg->compose.height;
- src_y++;
- }
-
- /*
- * For line-interleaved formats determine the 'plane'
- * based on the buffer line.
- */
- if (tpg_g_interleaved(tpg))
- p = tpg_g_interleaved_plane(tpg, buf_line);
-
- if (tpg->vdownsampling[p] > 1) {
- /*
- * When doing vertical downsampling the field setting
- * matters: for SEQ_BT/TB we downsample each field
- * separately (i.e. lines 0+2 are combined, as are
- * lines 1+3), for the other field settings we combine
- * odd and even lines. Doing that for SEQ_BT/TB would
- * be really weird.
- */
- if (tpg->field == V4L2_FIELD_SEQ_BT ||
- tpg->field == V4L2_FIELD_SEQ_TB) {
- unsigned next_src_y = src_y;
-
- if ((h & 3) >= 2)
- continue;
- next_src_y += int_part;
- if (error + fract_part >= tpg->compose.height)
- next_src_y++;
- params.frame_line_next =
- tpg_calc_frameline(tpg, next_src_y, tpg->field);
- } else {
- if (h & 1)
- continue;
- params.frame_line_next =
- tpg_calc_frameline(tpg, src_y, tpg->field);
- }
-
- buf_line /= tpg->vdownsampling[p];
- }
- tpg_fill_plane_pattern(tpg, ¶ms, p, h,
- vbuf + buf_line * params.stride);
- tpg_fill_plane_extras(tpg, ¶ms, p, h,
- vbuf + buf_line * params.stride);
- }
-}
-
-void tpg_fillbuffer(struct tpg_data *tpg, v4l2_std_id std, unsigned p, u8 *vbuf)
-{
- unsigned offset = 0;
- unsigned i;
-
- if (tpg->buffers > 1) {
- tpg_fill_plane_buffer(tpg, std, p, vbuf);
- return;
- }
-
- for (i = 0; i < tpg_g_planes(tpg); i++) {
- tpg_fill_plane_buffer(tpg, std, i, vbuf + offset);
- offset += tpg_calc_plane_size(tpg, i);
- }
-}
diff --git a/drivers/media/platform/vivid/vivid-tpg.h b/drivers/media/platform/vivid/vivid-tpg.h
deleted file mode 100644
index 93fbaee..0000000
--- a/drivers/media/platform/vivid/vivid-tpg.h
+++ /dev/null
@@ -1,598 +0,0 @@
-/*
- * vivid-tpg.h - Test Pattern Generator
- *
- * Copyright 2014 Cisco Systems, Inc. and/or its affiliates. All rights reserved.
- *
- * This program is free software; you may redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; version 2 of the License.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
- * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
- * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
- * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
- * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
- * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
- * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
- * SOFTWARE.
- */
-
-#ifndef _VIVID_TPG_H_
-#define _VIVID_TPG_H_
-
-#include <linux/types.h>
-#include <linux/errno.h>
-#include <linux/random.h>
-#include <linux/slab.h>
-#include <linux/vmalloc.h>
-#include <linux/videodev2.h>
-
-#include "vivid-tpg-colors.h"
-
-enum tpg_pattern {
- TPG_PAT_75_COLORBAR,
- TPG_PAT_100_COLORBAR,
- TPG_PAT_CSC_COLORBAR,
- TPG_PAT_100_HCOLORBAR,
- TPG_PAT_100_COLORSQUARES,
- TPG_PAT_BLACK,
- TPG_PAT_WHITE,
- TPG_PAT_RED,
- TPG_PAT_GREEN,
- TPG_PAT_BLUE,
- TPG_PAT_CHECKERS_16X16,
- TPG_PAT_CHECKERS_2X2,
- TPG_PAT_CHECKERS_1X1,
- TPG_PAT_COLOR_CHECKERS_2X2,
- TPG_PAT_COLOR_CHECKERS_1X1,
- TPG_PAT_ALTERNATING_HLINES,
- TPG_PAT_ALTERNATING_VLINES,
- TPG_PAT_CROSS_1_PIXEL,
- TPG_PAT_CROSS_2_PIXELS,
- TPG_PAT_CROSS_10_PIXELS,
- TPG_PAT_GRAY_RAMP,
-
- /* Must be the last pattern */
- TPG_PAT_NOISE,
-};
-
-extern const char * const tpg_pattern_strings[];
-
-enum tpg_quality {
- TPG_QUAL_COLOR,
- TPG_QUAL_GRAY,
- TPG_QUAL_NOISE
-};
-
-enum tpg_video_aspect {
- TPG_VIDEO_ASPECT_IMAGE,
- TPG_VIDEO_ASPECT_4X3,
- TPG_VIDEO_ASPECT_14X9_CENTRE,
- TPG_VIDEO_ASPECT_16X9_CENTRE,
- TPG_VIDEO_ASPECT_16X9_ANAMORPHIC,
-};
-
-enum tpg_pixel_aspect {
- TPG_PIXEL_ASPECT_SQUARE,
- TPG_PIXEL_ASPECT_NTSC,
- TPG_PIXEL_ASPECT_PAL,
-};
-
-enum tpg_move_mode {
- TPG_MOVE_NEG_FAST,
- TPG_MOVE_NEG,
- TPG_MOVE_NEG_SLOW,
- TPG_MOVE_NONE,
- TPG_MOVE_POS_SLOW,
- TPG_MOVE_POS,
- TPG_MOVE_POS_FAST,
-};
-
-extern const char * const tpg_aspect_strings[];
-
-#define TPG_MAX_PLANES 3
-#define TPG_MAX_PAT_LINES 8
-
-struct tpg_data {
- /* Source frame size */
- unsigned src_width, src_height;
- /* Buffer height */
- unsigned buf_height;
- /* Scaled output frame size */
- unsigned scaled_width;
- u32 field;
- bool field_alternate;
- /* crop coordinates are frame-based */
- struct v4l2_rect crop;
- /* compose coordinates are format-based */
- struct v4l2_rect compose;
- /* border and square coordinates are frame-based */
- struct v4l2_rect border;
- struct v4l2_rect square;
-
- /* Color-related fields */
- enum tpg_quality qual;
- unsigned qual_offset;
- u8 alpha_component;
- bool alpha_red_only;
- u8 brightness;
- u8 contrast;
- u8 saturation;
- s16 hue;
- u32 fourcc;
- bool is_yuv;
- u32 colorspace;
- u32 xfer_func;
- u32 ycbcr_enc;
- /*
- * Stores the actual transfer function, i.e. will never be
- * V4L2_XFER_FUNC_DEFAULT.
- */
- u32 real_xfer_func;
- /*
- * Stores the actual Y'CbCr encoding, i.e. will never be
- * V4L2_YCBCR_ENC_DEFAULT.
- */
- u32 real_ycbcr_enc;
- u32 quantization;
- /*
- * Stores the actual quantization, i.e. will never be
- * V4L2_QUANTIZATION_DEFAULT.
- */
- u32 real_quantization;
- enum tpg_video_aspect vid_aspect;
- enum tpg_pixel_aspect pix_aspect;
- unsigned rgb_range;
- unsigned real_rgb_range;
- unsigned buffers;
- unsigned planes;
- bool interleaved;
- u8 vdownsampling[TPG_MAX_PLANES];
- u8 hdownsampling[TPG_MAX_PLANES];
- /*
- * horizontal positions must be ANDed with this value to enforce
- * correct boundaries for packed YUYV values.
- */
- unsigned hmask[TPG_MAX_PLANES];
- /* Used to store the colors in native format, either RGB or YUV */
- u8 colors[TPG_COLOR_MAX][3];
- u8 textfg[TPG_MAX_PLANES][8], textbg[TPG_MAX_PLANES][8];
- /* size in bytes for two pixels in each plane */
- unsigned twopixelsize[TPG_MAX_PLANES];
- unsigned bytesperline[TPG_MAX_PLANES];
-
- /* Configuration */
- enum tpg_pattern pattern;
- bool hflip;
- bool vflip;
- unsigned perc_fill;
- bool perc_fill_blank;
- bool show_border;
- bool show_square;
- bool insert_sav;
- bool insert_eav;
-
- /* Test pattern movement */
- enum tpg_move_mode mv_hor_mode;
- int mv_hor_count;
- int mv_hor_step;
- enum tpg_move_mode mv_vert_mode;
- int mv_vert_count;
- int mv_vert_step;
-
- bool recalc_colors;
- bool recalc_lines;
- bool recalc_square_border;
-
- /* Used to store TPG_MAX_PAT_LINES lines, each with up to two planes */
- unsigned max_line_width;
- u8 *lines[TPG_MAX_PAT_LINES][TPG_MAX_PLANES];
- u8 *downsampled_lines[TPG_MAX_PAT_LINES][TPG_MAX_PLANES];
- u8 *random_line[TPG_MAX_PLANES];
- u8 *contrast_line[TPG_MAX_PLANES];
- u8 *black_line[TPG_MAX_PLANES];
-};
-
-void tpg_init(struct tpg_data *tpg, unsigned w, unsigned h);
-int tpg_alloc(struct tpg_data *tpg, unsigned max_w);
-void tpg_free(struct tpg_data *tpg);
-void tpg_reset_source(struct tpg_data *tpg, unsigned width, unsigned height,
- u32 field);
-void tpg_log_status(struct tpg_data *tpg);
-
-void tpg_set_font(const u8 *f);
-void tpg_gen_text(const struct tpg_data *tpg,
- u8 *basep[TPG_MAX_PLANES][2], int y, int x, char *text);
-void tpg_calc_text_basep(struct tpg_data *tpg,
- u8 *basep[TPG_MAX_PLANES][2], unsigned p, u8 *vbuf);
-unsigned tpg_g_interleaved_plane(const struct tpg_data *tpg, unsigned buf_line);
-void tpg_fill_plane_buffer(struct tpg_data *tpg, v4l2_std_id std,
- unsigned p, u8 *vbuf);
-void tpg_fillbuffer(struct tpg_data *tpg, v4l2_std_id std,
- unsigned p, u8 *vbuf);
-bool tpg_s_fourcc(struct tpg_data *tpg, u32 fourcc);
-void tpg_s_crop_compose(struct tpg_data *tpg, const struct v4l2_rect *crop,
- const struct v4l2_rect *compose);
-
-static inline void tpg_s_pattern(struct tpg_data *tpg, enum tpg_pattern pattern)
-{
- if (tpg->pattern == pattern)
- return;
- tpg->pattern = pattern;
- tpg->recalc_colors = true;
-}
-
-static inline void tpg_s_quality(struct tpg_data *tpg,
- enum tpg_quality qual, unsigned qual_offset)
-{
- if (tpg->qual == qual && tpg->qual_offset == qual_offset)
- return;
- tpg->qual = qual;
- tpg->qual_offset = qual_offset;
- tpg->recalc_colors = true;
-}
-
-static inline enum tpg_quality tpg_g_quality(const struct tpg_data *tpg)
-{
- return tpg->qual;
-}
-
-static inline void tpg_s_alpha_component(struct tpg_data *tpg,
- u8 alpha_component)
-{
- if (tpg->alpha_component == alpha_component)
- return;
- tpg->alpha_component = alpha_component;
- tpg->recalc_colors = true;
-}
-
-static inline void tpg_s_alpha_mode(struct tpg_data *tpg,
- bool red_only)
-{
- if (tpg->alpha_red_only == red_only)
- return;
- tpg->alpha_red_only = red_only;
- tpg->recalc_colors = true;
-}
-
-static inline void tpg_s_brightness(struct tpg_data *tpg,
- u8 brightness)
-{
- if (tpg->brightness == brightness)
- return;
- tpg->brightness = brightness;
- tpg->recalc_colors = true;
-}
-
-static inline void tpg_s_contrast(struct tpg_data *tpg,
- u8 contrast)
-{
- if (tpg->contrast == contrast)
- return;
- tpg->contrast = contrast;
- tpg->recalc_colors = true;
-}
-
-static inline void tpg_s_saturation(struct tpg_data *tpg,
- u8 saturation)
-{
- if (tpg->saturation == saturation)
- return;
- tpg->saturation = saturation;
- tpg->recalc_colors = true;
-}
-
-static inline void tpg_s_hue(struct tpg_data *tpg,
- s16 hue)
-{
- if (tpg->hue == hue)
- return;
- tpg->hue = hue;
- tpg->recalc_colors = true;
-}
-
-static inline void tpg_s_rgb_range(struct tpg_data *tpg,
- unsigned rgb_range)
-{
- if (tpg->rgb_range == rgb_range)
- return;
- tpg->rgb_range = rgb_range;
- tpg->recalc_colors = true;
-}
-
-static inline void tpg_s_real_rgb_range(struct tpg_data *tpg,
- unsigned rgb_range)
-{
- if (tpg->real_rgb_range == rgb_range)
- return;
- tpg->real_rgb_range = rgb_range;
- tpg->recalc_colors = true;
-}
-
-static inline void tpg_s_colorspace(struct tpg_data *tpg, u32 colorspace)
-{
- if (tpg->colorspace == colorspace)
- return;
- tpg->colorspace = colorspace;
- tpg->recalc_colors = true;
-}
-
-static inline u32 tpg_g_colorspace(const struct tpg_data *tpg)
-{
- return tpg->colorspace;
-}
-
-static inline void tpg_s_ycbcr_enc(struct tpg_data *tpg, u32 ycbcr_enc)
-{
- if (tpg->ycbcr_enc == ycbcr_enc)
- return;
- tpg->ycbcr_enc = ycbcr_enc;
- tpg->recalc_colors = true;
-}
-
-static inline u32 tpg_g_ycbcr_enc(const struct tpg_data *tpg)
-{
- return tpg->ycbcr_enc;
-}
-
-static inline void tpg_s_xfer_func(struct tpg_data *tpg, u32 xfer_func)
-{
- if (tpg->xfer_func == xfer_func)
- return;
- tpg->xfer_func = xfer_func;
- tpg->recalc_colors = true;
-}
-
-static inline u32 tpg_g_xfer_func(const struct tpg_data *tpg)
-{
- return tpg->xfer_func;
-}
-
-static inline void tpg_s_quantization(struct tpg_data *tpg, u32 quantization)
-{
- if (tpg->quantization == quantization)
- return;
- tpg->quantization = quantization;
- tpg->recalc_colors = true;
-}
-
-static inline u32 tpg_g_quantization(const struct tpg_data *tpg)
-{
- return tpg->quantization;
-}
-
-static inline unsigned tpg_g_buffers(const struct tpg_data *tpg)
-{
- return tpg->buffers;
-}
-
-static inline unsigned tpg_g_planes(const struct tpg_data *tpg)
-{
- return tpg->interleaved ? 1 : tpg->planes;
-}
-
-static inline bool tpg_g_interleaved(const struct tpg_data *tpg)
-{
- return tpg->interleaved;
-}
-
-static inline unsigned tpg_g_twopixelsize(const struct tpg_data *tpg, unsigned plane)
-{
- return tpg->twopixelsize[plane];
-}
-
-static inline unsigned tpg_hdiv(const struct tpg_data *tpg,
- unsigned plane, unsigned x)
-{
- return ((x / tpg->hdownsampling[plane]) & tpg->hmask[plane]) *
- tpg->twopixelsize[plane] / 2;
-}
-
-static inline unsigned tpg_hscale(const struct tpg_data *tpg, unsigned x)
-{
- return (x * tpg->scaled_width) / tpg->src_width;
-}
-
-static inline unsigned tpg_hscale_div(const struct tpg_data *tpg,
- unsigned plane, unsigned x)
-{
- return tpg_hdiv(tpg, plane, tpg_hscale(tpg, x));
-}
-
-static inline unsigned tpg_g_bytesperline(const struct tpg_data *tpg, unsigned plane)
-{
- return tpg->bytesperline[plane];
-}
-
-static inline void tpg_s_bytesperline(struct tpg_data *tpg, unsigned plane, unsigned bpl)
-{
- unsigned p;
-
- if (tpg->buffers > 1) {
- tpg->bytesperline[plane] = bpl;
- return;
- }
-
- for (p = 0; p < tpg_g_planes(tpg); p++) {
- unsigned plane_w = bpl * tpg->twopixelsize[p] / tpg->twopixelsize[0];
-
- tpg->bytesperline[p] = plane_w / tpg->hdownsampling[p];
- }
- if (tpg_g_interleaved(tpg))
- tpg->bytesperline[1] = tpg->bytesperline[0];
-}
-
-
-static inline unsigned tpg_g_line_width(const struct tpg_data *tpg, unsigned plane)
-{
- unsigned w = 0;
- unsigned p;
-
- if (tpg->buffers > 1)
- return tpg_g_bytesperline(tpg, plane);
- for (p = 0; p < tpg_g_planes(tpg); p++) {
- unsigned plane_w = tpg_g_bytesperline(tpg, p);
-
- w += plane_w / tpg->vdownsampling[p];
- }
- return w;
-}
-
-static inline unsigned tpg_calc_line_width(const struct tpg_data *tpg,
- unsigned plane, unsigned bpl)
-{
- unsigned w = 0;
- unsigned p;
-
- if (tpg->buffers > 1)
- return bpl;
- for (p = 0; p < tpg_g_planes(tpg); p++) {
- unsigned plane_w = bpl * tpg->twopixelsize[p] / tpg->twopixelsize[0];
-
- plane_w /= tpg->hdownsampling[p];
- w += plane_w / tpg->vdownsampling[p];
- }
- return w;
-}
-
-static inline unsigned tpg_calc_plane_size(const struct tpg_data *tpg, unsigned plane)
-{
- if (plane >= tpg_g_planes(tpg))
- return 0;
-
- return tpg_g_bytesperline(tpg, plane) * tpg->buf_height /
- tpg->vdownsampling[plane];
-}
-
-static inline void tpg_s_buf_height(struct tpg_data *tpg, unsigned h)
-{
- tpg->buf_height = h;
-}
-
-static inline void tpg_s_field(struct tpg_data *tpg, unsigned field, bool alternate)
-{
- tpg->field = field;
- tpg->field_alternate = alternate;
-}
-
-static inline void tpg_s_perc_fill(struct tpg_data *tpg,
- unsigned perc_fill)
-{
- tpg->perc_fill = perc_fill;
-}
-
-static inline unsigned tpg_g_perc_fill(const struct tpg_data *tpg)
-{
- return tpg->perc_fill;
-}
-
-static inline void tpg_s_perc_fill_blank(struct tpg_data *tpg,
- bool perc_fill_blank)
-{
- tpg->perc_fill_blank = perc_fill_blank;
-}
-
-static inline void tpg_s_video_aspect(struct tpg_data *tpg,
- enum tpg_video_aspect vid_aspect)
-{
- if (tpg->vid_aspect == vid_aspect)
- return;
- tpg->vid_aspect = vid_aspect;
- tpg->recalc_square_border = true;
-}
-
-static inline enum tpg_video_aspect tpg_g_video_aspect(const struct tpg_data *tpg)
-{
- return tpg->vid_aspect;
-}
-
-static inline void tpg_s_pixel_aspect(struct tpg_data *tpg,
- enum tpg_pixel_aspect pix_aspect)
-{
- if (tpg->pix_aspect == pix_aspect)
- return;
- tpg->pix_aspect = pix_aspect;
- tpg->recalc_square_border = true;
-}
-
-static inline void tpg_s_show_border(struct tpg_data *tpg,
- bool show_border)
-{
- tpg->show_border = show_border;
-}
-
-static inline void tpg_s_show_square(struct tpg_data *tpg,
- bool show_square)
-{
- tpg->show_square = show_square;
-}
-
-static inline void tpg_s_insert_sav(struct tpg_data *tpg, bool insert_sav)
-{
- tpg->insert_sav = insert_sav;
-}
-
-static inline void tpg_s_insert_eav(struct tpg_data *tpg, bool insert_eav)
-{
- tpg->insert_eav = insert_eav;
-}
-
-void tpg_update_mv_step(struct tpg_data *tpg);
-
-static inline void tpg_s_mv_hor_mode(struct tpg_data *tpg,
- enum tpg_move_mode mv_hor_mode)
-{
- tpg->mv_hor_mode = mv_hor_mode;
- tpg_update_mv_step(tpg);
-}
-
-static inline void tpg_s_mv_vert_mode(struct tpg_data *tpg,
- enum tpg_move_mode mv_vert_mode)
-{
- tpg->mv_vert_mode = mv_vert_mode;
- tpg_update_mv_step(tpg);
-}
-
-static inline void tpg_init_mv_count(struct tpg_data *tpg)
-{
- tpg->mv_hor_count = tpg->mv_vert_count = 0;
-}
-
-static inline void tpg_update_mv_count(struct tpg_data *tpg, bool frame_is_field)
-{
- tpg->mv_hor_count += tpg->mv_hor_step * (frame_is_field ? 1 : 2);
- tpg->mv_vert_count += tpg->mv_vert_step * (frame_is_field ? 1 : 2);
-}
-
-static inline void tpg_s_hflip(struct tpg_data *tpg, bool hflip)
-{
- if (tpg->hflip == hflip)
- return;
- tpg->hflip = hflip;
- tpg_update_mv_step(tpg);
- tpg->recalc_lines = true;
-}
-
-static inline bool tpg_g_hflip(const struct tpg_data *tpg)
-{
- return tpg->hflip;
-}
-
-static inline void tpg_s_vflip(struct tpg_data *tpg, bool vflip)
-{
- tpg->vflip = vflip;
-}
-
-static inline bool tpg_g_vflip(const struct tpg_data *tpg)
-{
- return tpg->vflip;
-}
-
-static inline bool tpg_pattern_is_static(const struct tpg_data *tpg)
-{
- return tpg->pattern != TPG_PAT_NOISE &&
- tpg->mv_hor_mode == TPG_MOVE_NONE &&
- tpg->mv_vert_mode == TPG_MOVE_NONE;
-}
-
-#endif
diff --git a/drivers/media/platform/vivid/vivid-vid-cap.c b/drivers/media/platform/vivid/vivid-vid-cap.c
index b84f081..4f730f3 100644
--- a/drivers/media/platform/vivid/vivid-vid-cap.c
+++ b/drivers/media/platform/vivid/vivid-vid-cap.c
@@ -26,6 +26,7 @@
#include <media/v4l2-common.h>
#include <media/v4l2-event.h>
#include <media/v4l2-dv-timings.h>
+#include <media/v4l2-rect.h>
#include "vivid-core.h"
#include "vivid-vid-common.h"
@@ -590,16 +591,16 @@
} else {
struct v4l2_rect r = { 0, 0, mp->width, mp->height * factor };
- rect_set_min_size(&r, &vivid_min_rect);
- rect_set_max_size(&r, &vivid_max_rect);
+ v4l2_rect_set_min_size(&r, &vivid_min_rect);
+ v4l2_rect_set_max_size(&r, &vivid_max_rect);
if (dev->has_scaler_cap && !dev->has_compose_cap) {
struct v4l2_rect max_r = { 0, 0, MAX_ZOOM * w, MAX_ZOOM * h };
- rect_set_max_size(&r, &max_r);
+ v4l2_rect_set_max_size(&r, &max_r);
} else if (!dev->has_scaler_cap && dev->has_crop_cap && !dev->has_compose_cap) {
- rect_set_max_size(&r, &dev->src_rect);
+ v4l2_rect_set_max_size(&r, &dev->src_rect);
} else if (!dev->has_scaler_cap && !dev->has_crop_cap) {
- rect_set_min_size(&r, &dev->src_rect);
+ v4l2_rect_set_min_size(&r, &dev->src_rect);
}
mp->width = r.width;
mp->height = r.height / factor;
@@ -668,7 +669,7 @@
if (dev->has_scaler_cap) {
if (dev->has_compose_cap)
- rect_map_inside(compose, &r);
+ v4l2_rect_map_inside(compose, &r);
else
*compose = r;
if (dev->has_crop_cap && !dev->has_compose_cap) {
@@ -683,9 +684,9 @@
factor * r.height * MAX_ZOOM
};
- rect_set_min_size(crop, &min_r);
- rect_set_max_size(crop, &max_r);
- rect_map_inside(crop, &dev->crop_bounds_cap);
+ v4l2_rect_set_min_size(crop, &min_r);
+ v4l2_rect_set_max_size(crop, &max_r);
+ v4l2_rect_map_inside(crop, &dev->crop_bounds_cap);
} else if (dev->has_crop_cap) {
struct v4l2_rect min_r = {
0, 0,
@@ -698,27 +699,27 @@
factor * compose->height * MAX_ZOOM
};
- rect_set_min_size(crop, &min_r);
- rect_set_max_size(crop, &max_r);
- rect_map_inside(crop, &dev->crop_bounds_cap);
+ v4l2_rect_set_min_size(crop, &min_r);
+ v4l2_rect_set_max_size(crop, &max_r);
+ v4l2_rect_map_inside(crop, &dev->crop_bounds_cap);
}
} else if (dev->has_crop_cap && !dev->has_compose_cap) {
r.height *= factor;
- rect_set_size_to(crop, &r);
- rect_map_inside(crop, &dev->crop_bounds_cap);
+ v4l2_rect_set_size_to(crop, &r);
+ v4l2_rect_map_inside(crop, &dev->crop_bounds_cap);
r = *crop;
r.height /= factor;
- rect_set_size_to(compose, &r);
+ v4l2_rect_set_size_to(compose, &r);
} else if (!dev->has_crop_cap) {
- rect_map_inside(compose, &r);
+ v4l2_rect_map_inside(compose, &r);
} else {
r.height *= factor;
- rect_set_max_size(crop, &r);
- rect_map_inside(crop, &dev->crop_bounds_cap);
+ v4l2_rect_set_max_size(crop, &r);
+ v4l2_rect_map_inside(crop, &dev->crop_bounds_cap);
compose->top *= factor;
compose->height *= factor;
- rect_set_size_to(compose, crop);
- rect_map_inside(compose, &r);
+ v4l2_rect_set_size_to(compose, crop);
+ v4l2_rect_map_inside(compose, &r);
compose->top /= factor;
compose->height /= factor;
}
@@ -735,9 +736,9 @@
} else {
struct v4l2_rect r = { 0, 0, mp->width, mp->height };
- rect_set_size_to(compose, &r);
+ v4l2_rect_set_size_to(compose, &r);
r.height *= factor;
- rect_set_size_to(crop, &r);
+ v4l2_rect_set_size_to(crop, &r);
}
dev->fmt_cap_rect.width = mp->width;
@@ -886,9 +887,9 @@
ret = vivid_vid_adjust_sel(s->flags, &s->r);
if (ret)
return ret;
- rect_set_min_size(&s->r, &vivid_min_rect);
- rect_set_max_size(&s->r, &dev->src_rect);
- rect_map_inside(&s->r, &dev->crop_bounds_cap);
+ v4l2_rect_set_min_size(&s->r, &vivid_min_rect);
+ v4l2_rect_set_max_size(&s->r, &dev->src_rect);
+ v4l2_rect_map_inside(&s->r, &dev->crop_bounds_cap);
s->r.top /= factor;
s->r.height /= factor;
if (dev->has_scaler_cap) {
@@ -904,36 +905,36 @@
s->r.height / MAX_ZOOM
};
- rect_set_min_size(&fmt, &min_rect);
+ v4l2_rect_set_min_size(&fmt, &min_rect);
if (!dev->has_compose_cap)
- rect_set_max_size(&fmt, &max_rect);
- if (!rect_same_size(&dev->fmt_cap_rect, &fmt) &&
+ v4l2_rect_set_max_size(&fmt, &max_rect);
+ if (!v4l2_rect_same_size(&dev->fmt_cap_rect, &fmt) &&
vb2_is_busy(&dev->vb_vid_cap_q))
return -EBUSY;
if (dev->has_compose_cap) {
- rect_set_min_size(compose, &min_rect);
- rect_set_max_size(compose, &max_rect);
+ v4l2_rect_set_min_size(compose, &min_rect);
+ v4l2_rect_set_max_size(compose, &max_rect);
}
dev->fmt_cap_rect = fmt;
tpg_s_buf_height(&dev->tpg, fmt.height);
} else if (dev->has_compose_cap) {
struct v4l2_rect fmt = dev->fmt_cap_rect;
- rect_set_min_size(&fmt, &s->r);
- if (!rect_same_size(&dev->fmt_cap_rect, &fmt) &&
+ v4l2_rect_set_min_size(&fmt, &s->r);
+ if (!v4l2_rect_same_size(&dev->fmt_cap_rect, &fmt) &&
vb2_is_busy(&dev->vb_vid_cap_q))
return -EBUSY;
dev->fmt_cap_rect = fmt;
tpg_s_buf_height(&dev->tpg, fmt.height);
- rect_set_size_to(compose, &s->r);
- rect_map_inside(compose, &dev->fmt_cap_rect);
+ v4l2_rect_set_size_to(compose, &s->r);
+ v4l2_rect_map_inside(compose, &dev->fmt_cap_rect);
} else {
- if (!rect_same_size(&s->r, &dev->fmt_cap_rect) &&
+ if (!v4l2_rect_same_size(&s->r, &dev->fmt_cap_rect) &&
vb2_is_busy(&dev->vb_vid_cap_q))
return -EBUSY;
- rect_set_size_to(&dev->fmt_cap_rect, &s->r);
- rect_set_size_to(compose, &s->r);
- rect_map_inside(compose, &dev->fmt_cap_rect);
+ v4l2_rect_set_size_to(&dev->fmt_cap_rect, &s->r);
+ v4l2_rect_set_size_to(compose, &s->r);
+ v4l2_rect_map_inside(compose, &dev->fmt_cap_rect);
tpg_s_buf_height(&dev->tpg, dev->fmt_cap_rect.height);
}
s->r.top *= factor;
@@ -946,8 +947,8 @@
ret = vivid_vid_adjust_sel(s->flags, &s->r);
if (ret)
return ret;
- rect_set_min_size(&s->r, &vivid_min_rect);
- rect_set_max_size(&s->r, &dev->fmt_cap_rect);
+ v4l2_rect_set_min_size(&s->r, &vivid_min_rect);
+ v4l2_rect_set_max_size(&s->r, &dev->fmt_cap_rect);
if (dev->has_scaler_cap) {
struct v4l2_rect max_rect = {
0, 0,
@@ -955,7 +956,7 @@
(dev->src_rect.height / factor) * MAX_ZOOM
};
- rect_set_max_size(&s->r, &max_rect);
+ v4l2_rect_set_max_size(&s->r, &max_rect);
if (dev->has_crop_cap) {
struct v4l2_rect min_rect = {
0, 0,
@@ -968,23 +969,23 @@
(s->r.height * factor) * MAX_ZOOM
};
- rect_set_min_size(crop, &min_rect);
- rect_set_max_size(crop, &max_rect);
- rect_map_inside(crop, &dev->crop_bounds_cap);
+ v4l2_rect_set_min_size(crop, &min_rect);
+ v4l2_rect_set_max_size(crop, &max_rect);
+ v4l2_rect_map_inside(crop, &dev->crop_bounds_cap);
}
} else if (dev->has_crop_cap) {
s->r.top *= factor;
s->r.height *= factor;
- rect_set_max_size(&s->r, &dev->src_rect);
- rect_set_size_to(crop, &s->r);
- rect_map_inside(crop, &dev->crop_bounds_cap);
+ v4l2_rect_set_max_size(&s->r, &dev->src_rect);
+ v4l2_rect_set_size_to(crop, &s->r);
+ v4l2_rect_map_inside(crop, &dev->crop_bounds_cap);
s->r.top /= factor;
s->r.height /= factor;
} else {
- rect_set_size_to(&s->r, &dev->src_rect);
+ v4l2_rect_set_size_to(&s->r, &dev->src_rect);
s->r.height /= factor;
}
- rect_map_inside(&s->r, &dev->fmt_cap_rect);
+ v4l2_rect_map_inside(&s->r, &dev->fmt_cap_rect);
if (dev->bitmap_cap && (compose->width != s->r.width ||
compose->height != s->r.height)) {
kfree(dev->bitmap_cap);
@@ -1124,7 +1125,7 @@
for (j = i + 1; j < win->clipcount; j++) {
struct v4l2_rect *r2 = &dev->try_clips_cap[j].c;
- if (rect_overlap(r1, r2))
+ if (v4l2_rect_overlap(r1, r2))
return -EINVAL;
}
}
diff --git a/drivers/media/platform/vivid/vivid-vid-common.c b/drivers/media/platform/vivid/vivid-vid-common.c
index b0d4e3a..39ea228 100644
--- a/drivers/media/platform/vivid/vivid-vid-common.c
+++ b/drivers/media/platform/vivid/vivid-vid-common.c
@@ -653,103 +653,6 @@
return ret;
}
-/* v4l2_rect helper function: copy the width/height values */
-void rect_set_size_to(struct v4l2_rect *r, const struct v4l2_rect *size)
-{
- r->width = size->width;
- r->height = size->height;
-}
-
-/* v4l2_rect helper function: width and height of r should be >= min_size */
-void rect_set_min_size(struct v4l2_rect *r, const struct v4l2_rect *min_size)
-{
- if (r->width < min_size->width)
- r->width = min_size->width;
- if (r->height < min_size->height)
- r->height = min_size->height;
-}
-
-/* v4l2_rect helper function: width and height of r should be <= max_size */
-void rect_set_max_size(struct v4l2_rect *r, const struct v4l2_rect *max_size)
-{
- if (r->width > max_size->width)
- r->width = max_size->width;
- if (r->height > max_size->height)
- r->height = max_size->height;
-}
-
-/* v4l2_rect helper function: r should be inside boundary */
-void rect_map_inside(struct v4l2_rect *r, const struct v4l2_rect *boundary)
-{
- rect_set_max_size(r, boundary);
- if (r->left < boundary->left)
- r->left = boundary->left;
- if (r->top < boundary->top)
- r->top = boundary->top;
- if (r->left + r->width > boundary->width)
- r->left = boundary->width - r->width;
- if (r->top + r->height > boundary->height)
- r->top = boundary->height - r->height;
-}
-
-/* v4l2_rect helper function: return true if r1 has the same size as r2 */
-bool rect_same_size(const struct v4l2_rect *r1, const struct v4l2_rect *r2)
-{
- return r1->width == r2->width && r1->height == r2->height;
-}
-
-/* v4l2_rect helper function: calculate the intersection of two rects */
-struct v4l2_rect rect_intersect(const struct v4l2_rect *a, const struct v4l2_rect *b)
-{
- struct v4l2_rect r;
- int right, bottom;
-
- r.top = max(a->top, b->top);
- r.left = max(a->left, b->left);
- bottom = min(a->top + a->height, b->top + b->height);
- right = min(a->left + a->width, b->left + b->width);
- r.height = max(0, bottom - r.top);
- r.width = max(0, right - r.left);
- return r;
-}
-
-/*
- * v4l2_rect helper function: scale rect r by to->width / from->width and
- * to->height / from->height.
- */
-void rect_scale(struct v4l2_rect *r, const struct v4l2_rect *from,
- const struct v4l2_rect *to)
-{
- if (from->width == 0 || from->height == 0) {
- r->left = r->top = r->width = r->height = 0;
- return;
- }
- r->left = (((r->left - from->left) * to->width) / from->width) & ~1;
- r->width = ((r->width * to->width) / from->width) & ~1;
- r->top = ((r->top - from->top) * to->height) / from->height;
- r->height = (r->height * to->height) / from->height;
-}
-
-bool rect_overlap(const struct v4l2_rect *r1, const struct v4l2_rect *r2)
-{
- /*
- * IF the left side of r1 is to the right of the right side of r2 OR
- * the left side of r2 is to the right of the right side of r1 THEN
- * they do not overlap.
- */
- if (r1->left >= r2->left + r2->width ||
- r2->left >= r1->left + r1->width)
- return false;
- /*
- * IF the top side of r1 is below the bottom of r2 OR
- * the top side of r2 is below the bottom of r1 THEN
- * they do not overlap.
- */
- if (r1->top >= r2->top + r2->height ||
- r2->top >= r1->top + r1->height)
- return false;
- return true;
-}
int vivid_vid_adjust_sel(unsigned flags, struct v4l2_rect *r)
{
unsigned w = r->width;
diff --git a/drivers/media/platform/vivid/vivid-vid-common.h b/drivers/media/platform/vivid/vivid-vid-common.h
index 3ec4fa8..4b6175e 100644
--- a/drivers/media/platform/vivid/vivid-vid-common.h
+++ b/drivers/media/platform/vivid/vivid-vid-common.h
@@ -37,15 +37,6 @@
bool vivid_vid_can_loop(struct vivid_dev *dev);
void vivid_send_source_change(struct vivid_dev *dev, unsigned type);
-bool rect_overlap(const struct v4l2_rect *r1, const struct v4l2_rect *r2);
-void rect_set_size_to(struct v4l2_rect *r, const struct v4l2_rect *size);
-void rect_set_min_size(struct v4l2_rect *r, const struct v4l2_rect *min_size);
-void rect_set_max_size(struct v4l2_rect *r, const struct v4l2_rect *max_size);
-void rect_map_inside(struct v4l2_rect *r, const struct v4l2_rect *boundary);
-bool rect_same_size(const struct v4l2_rect *r1, const struct v4l2_rect *r2);
-struct v4l2_rect rect_intersect(const struct v4l2_rect *a, const struct v4l2_rect *b);
-void rect_scale(struct v4l2_rect *r, const struct v4l2_rect *from,
- const struct v4l2_rect *to);
int vivid_vid_adjust_sel(unsigned flags, struct v4l2_rect *r);
int vivid_enum_fmt_vid(struct file *file, void *priv, struct v4l2_fmtdesc *f);
diff --git a/drivers/media/platform/vivid/vivid-vid-out.c b/drivers/media/platform/vivid/vivid-vid-out.c
index 64e4d66..f92f449 100644
--- a/drivers/media/platform/vivid/vivid-vid-out.c
+++ b/drivers/media/platform/vivid/vivid-vid-out.c
@@ -25,6 +25,7 @@
#include <media/v4l2-common.h>
#include <media/v4l2-event.h>
#include <media/v4l2-dv-timings.h>
+#include <media/v4l2-rect.h>
#include "vivid-core.h"
#include "vivid-vid-common.h"
@@ -376,16 +377,16 @@
} else {
struct v4l2_rect r = { 0, 0, mp->width, mp->height * factor };
- rect_set_min_size(&r, &vivid_min_rect);
- rect_set_max_size(&r, &vivid_max_rect);
+ v4l2_rect_set_min_size(&r, &vivid_min_rect);
+ v4l2_rect_set_max_size(&r, &vivid_max_rect);
if (dev->has_scaler_out && !dev->has_crop_out) {
struct v4l2_rect max_r = { 0, 0, MAX_ZOOM * w, MAX_ZOOM * h };
- rect_set_max_size(&r, &max_r);
+ v4l2_rect_set_max_size(&r, &max_r);
} else if (!dev->has_scaler_out && dev->has_compose_out && !dev->has_crop_out) {
- rect_set_max_size(&r, &dev->sink_rect);
+ v4l2_rect_set_max_size(&r, &dev->sink_rect);
} else if (!dev->has_scaler_out && !dev->has_compose_out) {
- rect_set_min_size(&r, &dev->sink_rect);
+ v4l2_rect_set_min_size(&r, &dev->sink_rect);
}
mp->width = r.width;
mp->height = r.height / factor;
@@ -473,7 +474,7 @@
if (dev->has_scaler_out) {
if (dev->has_crop_out)
- rect_map_inside(crop, &r);
+ v4l2_rect_map_inside(crop, &r);
else
*crop = r;
if (dev->has_compose_out && !dev->has_crop_out) {
@@ -488,9 +489,9 @@
factor * r.height * MAX_ZOOM
};
- rect_set_min_size(compose, &min_r);
- rect_set_max_size(compose, &max_r);
- rect_map_inside(compose, &dev->compose_bounds_out);
+ v4l2_rect_set_min_size(compose, &min_r);
+ v4l2_rect_set_max_size(compose, &max_r);
+ v4l2_rect_map_inside(compose, &dev->compose_bounds_out);
} else if (dev->has_compose_out) {
struct v4l2_rect min_r = {
0, 0,
@@ -503,36 +504,36 @@
factor * crop->height * MAX_ZOOM
};
- rect_set_min_size(compose, &min_r);
- rect_set_max_size(compose, &max_r);
- rect_map_inside(compose, &dev->compose_bounds_out);
+ v4l2_rect_set_min_size(compose, &min_r);
+ v4l2_rect_set_max_size(compose, &max_r);
+ v4l2_rect_map_inside(compose, &dev->compose_bounds_out);
}
} else if (dev->has_compose_out && !dev->has_crop_out) {
- rect_set_size_to(crop, &r);
+ v4l2_rect_set_size_to(crop, &r);
r.height *= factor;
- rect_set_size_to(compose, &r);
- rect_map_inside(compose, &dev->compose_bounds_out);
+ v4l2_rect_set_size_to(compose, &r);
+ v4l2_rect_map_inside(compose, &dev->compose_bounds_out);
} else if (!dev->has_compose_out) {
- rect_map_inside(crop, &r);
+ v4l2_rect_map_inside(crop, &r);
r.height /= factor;
- rect_set_size_to(compose, &r);
+ v4l2_rect_set_size_to(compose, &r);
} else {
r.height *= factor;
- rect_set_max_size(compose, &r);
- rect_map_inside(compose, &dev->compose_bounds_out);
+ v4l2_rect_set_max_size(compose, &r);
+ v4l2_rect_map_inside(compose, &dev->compose_bounds_out);
crop->top *= factor;
crop->height *= factor;
- rect_set_size_to(crop, compose);
- rect_map_inside(crop, &r);
+ v4l2_rect_set_size_to(crop, compose);
+ v4l2_rect_map_inside(crop, &r);
crop->top /= factor;
crop->height /= factor;
}
} else {
struct v4l2_rect r = { 0, 0, mp->width, mp->height };
- rect_set_size_to(crop, &r);
+ v4l2_rect_set_size_to(crop, &r);
r.height /= factor;
- rect_set_size_to(compose, &r);
+ v4l2_rect_set_size_to(compose, &r);
}
dev->fmt_out_rect.width = mp->width;
@@ -683,8 +684,8 @@
ret = vivid_vid_adjust_sel(s->flags, &s->r);
if (ret)
return ret;
- rect_set_min_size(&s->r, &vivid_min_rect);
- rect_set_max_size(&s->r, &dev->fmt_out_rect);
+ v4l2_rect_set_min_size(&s->r, &vivid_min_rect);
+ v4l2_rect_set_max_size(&s->r, &dev->fmt_out_rect);
if (dev->has_scaler_out) {
struct v4l2_rect max_rect = {
0, 0,
@@ -692,7 +693,7 @@
(dev->sink_rect.height / factor) * MAX_ZOOM
};
- rect_set_max_size(&s->r, &max_rect);
+ v4l2_rect_set_max_size(&s->r, &max_rect);
if (dev->has_compose_out) {
struct v4l2_rect min_rect = {
0, 0,
@@ -705,23 +706,23 @@
(s->r.height * factor) * MAX_ZOOM
};
- rect_set_min_size(compose, &min_rect);
- rect_set_max_size(compose, &max_rect);
- rect_map_inside(compose, &dev->compose_bounds_out);
+ v4l2_rect_set_min_size(compose, &min_rect);
+ v4l2_rect_set_max_size(compose, &max_rect);
+ v4l2_rect_map_inside(compose, &dev->compose_bounds_out);
}
} else if (dev->has_compose_out) {
s->r.top *= factor;
s->r.height *= factor;
- rect_set_max_size(&s->r, &dev->sink_rect);
- rect_set_size_to(compose, &s->r);
- rect_map_inside(compose, &dev->compose_bounds_out);
+ v4l2_rect_set_max_size(&s->r, &dev->sink_rect);
+ v4l2_rect_set_size_to(compose, &s->r);
+ v4l2_rect_map_inside(compose, &dev->compose_bounds_out);
s->r.top /= factor;
s->r.height /= factor;
} else {
- rect_set_size_to(&s->r, &dev->sink_rect);
+ v4l2_rect_set_size_to(&s->r, &dev->sink_rect);
s->r.height /= factor;
}
- rect_map_inside(&s->r, &dev->fmt_out_rect);
+ v4l2_rect_map_inside(&s->r, &dev->fmt_out_rect);
*crop = s->r;
break;
case V4L2_SEL_TGT_COMPOSE:
@@ -730,9 +731,9 @@
ret = vivid_vid_adjust_sel(s->flags, &s->r);
if (ret)
return ret;
- rect_set_min_size(&s->r, &vivid_min_rect);
- rect_set_max_size(&s->r, &dev->sink_rect);
- rect_map_inside(&s->r, &dev->compose_bounds_out);
+ v4l2_rect_set_min_size(&s->r, &vivid_min_rect);
+ v4l2_rect_set_max_size(&s->r, &dev->sink_rect);
+ v4l2_rect_map_inside(&s->r, &dev->compose_bounds_out);
s->r.top /= factor;
s->r.height /= factor;
if (dev->has_scaler_out) {
@@ -748,35 +749,35 @@
s->r.height / MAX_ZOOM
};
- rect_set_min_size(&fmt, &min_rect);
+ v4l2_rect_set_min_size(&fmt, &min_rect);
if (!dev->has_crop_out)
- rect_set_max_size(&fmt, &max_rect);
- if (!rect_same_size(&dev->fmt_out_rect, &fmt) &&
+ v4l2_rect_set_max_size(&fmt, &max_rect);
+ if (!v4l2_rect_same_size(&dev->fmt_out_rect, &fmt) &&
vb2_is_busy(&dev->vb_vid_out_q))
return -EBUSY;
if (dev->has_crop_out) {
- rect_set_min_size(crop, &min_rect);
- rect_set_max_size(crop, &max_rect);
+ v4l2_rect_set_min_size(crop, &min_rect);
+ v4l2_rect_set_max_size(crop, &max_rect);
}
dev->fmt_out_rect = fmt;
} else if (dev->has_crop_out) {
struct v4l2_rect fmt = dev->fmt_out_rect;
- rect_set_min_size(&fmt, &s->r);
- if (!rect_same_size(&dev->fmt_out_rect, &fmt) &&
+ v4l2_rect_set_min_size(&fmt, &s->r);
+ if (!v4l2_rect_same_size(&dev->fmt_out_rect, &fmt) &&
vb2_is_busy(&dev->vb_vid_out_q))
return -EBUSY;
dev->fmt_out_rect = fmt;
- rect_set_size_to(crop, &s->r);
- rect_map_inside(crop, &dev->fmt_out_rect);
+ v4l2_rect_set_size_to(crop, &s->r);
+ v4l2_rect_map_inside(crop, &dev->fmt_out_rect);
} else {
- if (!rect_same_size(&s->r, &dev->fmt_out_rect) &&
+ if (!v4l2_rect_same_size(&s->r, &dev->fmt_out_rect) &&
vb2_is_busy(&dev->vb_vid_out_q))
return -EBUSY;
- rect_set_size_to(&dev->fmt_out_rect, &s->r);
- rect_set_size_to(crop, &s->r);
+ v4l2_rect_set_size_to(&dev->fmt_out_rect, &s->r);
+ v4l2_rect_set_size_to(crop, &s->r);
crop->height /= factor;
- rect_map_inside(crop, &dev->fmt_out_rect);
+ v4l2_rect_map_inside(crop, &dev->fmt_out_rect);
}
s->r.top *= factor;
s->r.height *= factor;
@@ -901,7 +902,7 @@
for (j = i + 1; j < win->clipcount; j++) {
struct v4l2_rect *r2 = &dev->try_clips_out[j].c;
- if (rect_overlap(r1, r2))
+ if (v4l2_rect_overlap(r1, r2))
return -EINVAL;
}
}
diff --git a/drivers/media/platform/vsp1/vsp1.h b/drivers/media/platform/vsp1/vsp1.h
index 910d6b8..46738b6 100644
--- a/drivers/media/platform/vsp1/vsp1.h
+++ b/drivers/media/platform/vsp1/vsp1.h
@@ -26,7 +26,6 @@
struct clk;
struct device;
-struct vsp1_dl;
struct vsp1_drm;
struct vsp1_entity;
struct vsp1_platform_data;
@@ -49,6 +48,7 @@
struct vsp1_device_info {
u32 version;
+ unsigned int gen;
unsigned int features;
unsigned int rpf_count;
unsigned int uds_count;
@@ -85,8 +85,6 @@
struct media_entity_operations media_ops;
struct vsp1_drm *drm;
-
- bool use_dl;
};
int vsp1_device_get(struct vsp1_device *vsp1);
@@ -104,14 +102,4 @@
iowrite32(data, vsp1->mmio + reg);
}
-#include "vsp1_dl.h"
-
-static inline void vsp1_mod_write(struct vsp1_entity *e, u32 reg, u32 data)
-{
- if (e->vsp1->use_dl)
- vsp1_dl_add(e, reg, data);
- else
- vsp1_write(e->vsp1, reg, data);
-}
-
#endif /* __VSP1_H__ */
diff --git a/drivers/media/platform/vsp1/vsp1_bru.c b/drivers/media/platform/vsp1/vsp1_bru.c
index cb0dbc1..b1068c0 100644
--- a/drivers/media/platform/vsp1/vsp1_bru.c
+++ b/drivers/media/platform/vsp1/vsp1_bru.c
@@ -18,6 +18,8 @@
#include "vsp1.h"
#include "vsp1_bru.h"
+#include "vsp1_dl.h"
+#include "vsp1_pipe.h"
#include "vsp1_rwpf.h"
#include "vsp1_video.h"
@@ -28,9 +30,10 @@
* Device Access
*/
-static inline void vsp1_bru_write(struct vsp1_bru *bru, u32 reg, u32 data)
+static inline void vsp1_bru_write(struct vsp1_bru *bru, struct vsp1_dl_list *dl,
+ u32 reg, u32 data)
{
- vsp1_mod_write(&bru->entity, reg, data);
+ vsp1_dl_list_write(dl, reg, data);
}
/* -----------------------------------------------------------------------------
@@ -42,13 +45,9 @@
struct vsp1_bru *bru =
container_of(ctrl->handler, struct vsp1_bru, ctrls);
- if (!vsp1_entity_is_streaming(&bru->entity))
- return 0;
-
switch (ctrl->id) {
case V4L2_CID_BG_COLOR:
- vsp1_bru_write(bru, VI6_BRU_VIRRPF_COL, ctrl->val |
- (0xff << VI6_BRU_VIRRPF_COL_A_SHIFT));
+ bru->bgcolor = ctrl->val;
break;
}
@@ -60,26 +59,225 @@
};
/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Core Operations
+ * V4L2 Subdevice Operations
*/
-static int bru_s_stream(struct v4l2_subdev *subdev, int enable)
+/*
+ * The BRU can't perform format conversion, all sink and source formats must be
+ * identical. We pick the format on the first sink pad (pad 0) and propagate it
+ * to all other pads.
+ */
+
+static int bru_enum_mbus_code(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_mbus_code_enum *code)
{
- struct vsp1_pipeline *pipe = to_vsp1_pipeline(&subdev->entity);
+ static const unsigned int codes[] = {
+ MEDIA_BUS_FMT_ARGB8888_1X32,
+ MEDIA_BUS_FMT_AYUV8_1X32,
+ };
+
+ return vsp1_subdev_enum_mbus_code(subdev, cfg, code, codes,
+ ARRAY_SIZE(codes));
+}
+
+static int bru_enum_frame_size(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_frame_size_enum *fse)
+{
+ if (fse->index)
+ return -EINVAL;
+
+ if (fse->code != MEDIA_BUS_FMT_ARGB8888_1X32 &&
+ fse->code != MEDIA_BUS_FMT_AYUV8_1X32)
+ return -EINVAL;
+
+ fse->min_width = BRU_MIN_SIZE;
+ fse->max_width = BRU_MAX_SIZE;
+ fse->min_height = BRU_MIN_SIZE;
+ fse->max_height = BRU_MAX_SIZE;
+
+ return 0;
+}
+
+static struct v4l2_rect *bru_get_compose(struct vsp1_bru *bru,
+ struct v4l2_subdev_pad_config *cfg,
+ unsigned int pad)
+{
+ return v4l2_subdev_get_try_compose(&bru->entity.subdev, cfg, pad);
+}
+
+static void bru_try_format(struct vsp1_bru *bru,
+ struct v4l2_subdev_pad_config *config,
+ unsigned int pad, struct v4l2_mbus_framefmt *fmt)
+{
+ struct v4l2_mbus_framefmt *format;
+
+ switch (pad) {
+ case BRU_PAD_SINK(0):
+ /* Default to YUV if the requested format is not supported. */
+ if (fmt->code != MEDIA_BUS_FMT_ARGB8888_1X32 &&
+ fmt->code != MEDIA_BUS_FMT_AYUV8_1X32)
+ fmt->code = MEDIA_BUS_FMT_AYUV8_1X32;
+ break;
+
+ default:
+ /* The BRU can't perform format conversion. */
+ format = vsp1_entity_get_pad_format(&bru->entity, config,
+ BRU_PAD_SINK(0));
+ fmt->code = format->code;
+ break;
+ }
+
+ fmt->width = clamp(fmt->width, BRU_MIN_SIZE, BRU_MAX_SIZE);
+ fmt->height = clamp(fmt->height, BRU_MIN_SIZE, BRU_MAX_SIZE);
+ fmt->field = V4L2_FIELD_NONE;
+ fmt->colorspace = V4L2_COLORSPACE_SRGB;
+}
+
+static int bru_set_format(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *fmt)
+{
struct vsp1_bru *bru = to_bru(subdev);
+ struct v4l2_subdev_pad_config *config;
+ struct v4l2_mbus_framefmt *format;
+
+ config = vsp1_entity_get_pad_config(&bru->entity, cfg, fmt->which);
+ if (!config)
+ return -EINVAL;
+
+ bru_try_format(bru, config, fmt->pad, &fmt->format);
+
+ format = vsp1_entity_get_pad_format(&bru->entity, config, fmt->pad);
+ *format = fmt->format;
+
+ /* Reset the compose rectangle */
+ if (fmt->pad != bru->entity.source_pad) {
+ struct v4l2_rect *compose;
+
+ compose = bru_get_compose(bru, config, fmt->pad);
+ compose->left = 0;
+ compose->top = 0;
+ compose->width = format->width;
+ compose->height = format->height;
+ }
+
+ /* Propagate the format code to all pads */
+ if (fmt->pad == BRU_PAD_SINK(0)) {
+ unsigned int i;
+
+ for (i = 0; i <= bru->entity.source_pad; ++i) {
+ format = vsp1_entity_get_pad_format(&bru->entity,
+ config, i);
+ format->code = fmt->format.code;
+ }
+ }
+
+ return 0;
+}
+
+static int bru_get_selection(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_selection *sel)
+{
+ struct vsp1_bru *bru = to_bru(subdev);
+ struct v4l2_subdev_pad_config *config;
+
+ if (sel->pad == bru->entity.source_pad)
+ return -EINVAL;
+
+ switch (sel->target) {
+ case V4L2_SEL_TGT_COMPOSE_BOUNDS:
+ sel->r.left = 0;
+ sel->r.top = 0;
+ sel->r.width = BRU_MAX_SIZE;
+ sel->r.height = BRU_MAX_SIZE;
+ return 0;
+
+ case V4L2_SEL_TGT_COMPOSE:
+ config = vsp1_entity_get_pad_config(&bru->entity, cfg,
+ sel->which);
+ if (!config)
+ return -EINVAL;
+
+ sel->r = *bru_get_compose(bru, config, sel->pad);
+ return 0;
+
+ default:
+ return -EINVAL;
+ }
+}
+
+static int bru_set_selection(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_selection *sel)
+{
+ struct vsp1_bru *bru = to_bru(subdev);
+ struct v4l2_subdev_pad_config *config;
+ struct v4l2_mbus_framefmt *format;
+ struct v4l2_rect *compose;
+
+ if (sel->pad == bru->entity.source_pad)
+ return -EINVAL;
+
+ if (sel->target != V4L2_SEL_TGT_COMPOSE)
+ return -EINVAL;
+
+ config = vsp1_entity_get_pad_config(&bru->entity, cfg, sel->which);
+ if (!config)
+ return -EINVAL;
+
+ /* The compose rectangle top left corner must be inside the output
+ * frame.
+ */
+ format = vsp1_entity_get_pad_format(&bru->entity, config,
+ bru->entity.source_pad);
+ sel->r.left = clamp_t(unsigned int, sel->r.left, 0, format->width - 1);
+ sel->r.top = clamp_t(unsigned int, sel->r.top, 0, format->height - 1);
+
+ /* Scaling isn't supported, the compose rectangle size must be identical
+ * to the sink format size.
+ */
+ format = vsp1_entity_get_pad_format(&bru->entity, config, sel->pad);
+ sel->r.width = format->width;
+ sel->r.height = format->height;
+
+ compose = bru_get_compose(bru, config, sel->pad);
+ *compose = sel->r;
+
+ return 0;
+}
+
+static struct v4l2_subdev_pad_ops bru_pad_ops = {
+ .init_cfg = vsp1_entity_init_cfg,
+ .enum_mbus_code = bru_enum_mbus_code,
+ .enum_frame_size = bru_enum_frame_size,
+ .get_fmt = vsp1_subdev_get_pad_format,
+ .set_fmt = bru_set_format,
+ .get_selection = bru_get_selection,
+ .set_selection = bru_set_selection,
+};
+
+static struct v4l2_subdev_ops bru_ops = {
+ .pad = &bru_pad_ops,
+};
+
+/* -----------------------------------------------------------------------------
+ * VSP1 Entity Operations
+ */
+
+static void bru_configure(struct vsp1_entity *entity,
+ struct vsp1_pipeline *pipe,
+ struct vsp1_dl_list *dl)
+{
+ struct vsp1_bru *bru = to_bru(&entity->subdev);
struct v4l2_mbus_framefmt *format;
unsigned int flags;
unsigned int i;
- int ret;
- ret = vsp1_entity_set_streaming(&bru->entity, enable);
- if (ret < 0)
- return ret;
-
- if (!enable)
- return 0;
-
- format = &bru->entity.formats[bru->entity.source_pad];
+ format = vsp1_entity_get_pad_format(&bru->entity, bru->entity.config,
+ bru->entity.source_pad);
/* The hardware is extremely flexible but we have no userspace API to
* expose all the parameters, nor is it clear whether we would have use
@@ -91,21 +289,26 @@
* format at the pipeline output is premultiplied.
*/
flags = pipe->output ? pipe->output->format.flags : 0;
- vsp1_bru_write(bru, VI6_BRU_INCTRL,
+ vsp1_bru_write(bru, dl, VI6_BRU_INCTRL,
flags & V4L2_PIX_FMT_FLAG_PREMUL_ALPHA ?
0 : VI6_BRU_INCTRL_NRM);
- /* Set the background position to cover the whole output image. */
- vsp1_bru_write(bru, VI6_BRU_VIRRPF_SIZE,
+ /* Set the background position to cover the whole output image and
+ * configure its color.
+ */
+ vsp1_bru_write(bru, dl, VI6_BRU_VIRRPF_SIZE,
(format->width << VI6_BRU_VIRRPF_SIZE_HSIZE_SHIFT) |
(format->height << VI6_BRU_VIRRPF_SIZE_VSIZE_SHIFT));
- vsp1_bru_write(bru, VI6_BRU_VIRRPF_LOC, 0);
+ vsp1_bru_write(bru, dl, VI6_BRU_VIRRPF_LOC, 0);
+
+ vsp1_bru_write(bru, dl, VI6_BRU_VIRRPF_COL, bru->bgcolor |
+ (0xff << VI6_BRU_VIRRPF_COL_A_SHIFT));
/* Route BRU input 1 as SRC input to the ROP unit and configure the ROP
* unit with a NOP operation to make BRU input 1 available as the
* Blend/ROP unit B SRC input.
*/
- vsp1_bru_write(bru, VI6_BRU_ROP, VI6_BRU_ROP_DSTSEL_BRUIN(1) |
+ vsp1_bru_write(bru, dl, VI6_BRU_ROP, VI6_BRU_ROP_DSTSEL_BRUIN(1) |
VI6_BRU_ROP_CROP(VI6_ROP_NOP) |
VI6_BRU_ROP_AROP(VI6_ROP_NOP));
@@ -142,7 +345,7 @@
if (i != 1)
ctrl |= VI6_BRU_CTRL_SRCSEL_BRUIN(i);
- vsp1_bru_write(bru, VI6_BRU_CTRL(i), ctrl);
+ vsp1_bru_write(bru, dl, VI6_BRU_CTRL(i), ctrl);
/* Harcode the blending formula to
*
@@ -156,7 +359,7 @@
*
* otherwise.
*/
- vsp1_bru_write(bru, VI6_BRU_BLD(i),
+ vsp1_bru_write(bru, dl, VI6_BRU_BLD(i),
VI6_BRU_BLD_CCMDX_255_SRC_A |
(premultiplied ? VI6_BRU_BLD_CCMDY_COEFY :
VI6_BRU_BLD_CCMDY_SRC_A) |
@@ -164,239 +367,10 @@
VI6_BRU_BLD_ACMDY_COEFY |
(0xff << VI6_BRU_BLD_COEFY_SHIFT));
}
-
- return 0;
}
-/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Pad Operations
- */
-
-/*
- * The BRU can't perform format conversion, all sink and source formats must be
- * identical. We pick the format on the first sink pad (pad 0) and propagate it
- * to all other pads.
- */
-
-static int bru_enum_mbus_code(struct v4l2_subdev *subdev,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_mbus_code_enum *code)
-{
- static const unsigned int codes[] = {
- MEDIA_BUS_FMT_ARGB8888_1X32,
- MEDIA_BUS_FMT_AYUV8_1X32,
- };
- struct vsp1_bru *bru = to_bru(subdev);
- struct v4l2_mbus_framefmt *format;
-
- if (code->pad == BRU_PAD_SINK(0)) {
- if (code->index >= ARRAY_SIZE(codes))
- return -EINVAL;
-
- code->code = codes[code->index];
- } else {
- if (code->index)
- return -EINVAL;
-
- format = vsp1_entity_get_pad_format(&bru->entity, cfg,
- BRU_PAD_SINK(0), code->which);
- code->code = format->code;
- }
-
- return 0;
-}
-
-static int bru_enum_frame_size(struct v4l2_subdev *subdev,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_frame_size_enum *fse)
-{
- if (fse->index)
- return -EINVAL;
-
- if (fse->code != MEDIA_BUS_FMT_ARGB8888_1X32 &&
- fse->code != MEDIA_BUS_FMT_AYUV8_1X32)
- return -EINVAL;
-
- fse->min_width = BRU_MIN_SIZE;
- fse->max_width = BRU_MAX_SIZE;
- fse->min_height = BRU_MIN_SIZE;
- fse->max_height = BRU_MAX_SIZE;
-
- return 0;
-}
-
-static struct v4l2_rect *bru_get_compose(struct vsp1_bru *bru,
- struct v4l2_subdev_pad_config *cfg,
- unsigned int pad, u32 which)
-{
- switch (which) {
- case V4L2_SUBDEV_FORMAT_TRY:
- return v4l2_subdev_get_try_crop(&bru->entity.subdev, cfg, pad);
- case V4L2_SUBDEV_FORMAT_ACTIVE:
- return &bru->inputs[pad].compose;
- default:
- return NULL;
- }
-}
-
-static int bru_get_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_format *fmt)
-{
- struct vsp1_bru *bru = to_bru(subdev);
-
- fmt->format = *vsp1_entity_get_pad_format(&bru->entity, cfg, fmt->pad,
- fmt->which);
-
- return 0;
-}
-
-static void bru_try_format(struct vsp1_bru *bru, struct v4l2_subdev_pad_config *cfg,
- unsigned int pad, struct v4l2_mbus_framefmt *fmt,
- enum v4l2_subdev_format_whence which)
-{
- struct v4l2_mbus_framefmt *format;
-
- switch (pad) {
- case BRU_PAD_SINK(0):
- /* Default to YUV if the requested format is not supported. */
- if (fmt->code != MEDIA_BUS_FMT_ARGB8888_1X32 &&
- fmt->code != MEDIA_BUS_FMT_AYUV8_1X32)
- fmt->code = MEDIA_BUS_FMT_AYUV8_1X32;
- break;
-
- default:
- /* The BRU can't perform format conversion. */
- format = vsp1_entity_get_pad_format(&bru->entity, cfg,
- BRU_PAD_SINK(0), which);
- fmt->code = format->code;
- break;
- }
-
- fmt->width = clamp(fmt->width, BRU_MIN_SIZE, BRU_MAX_SIZE);
- fmt->height = clamp(fmt->height, BRU_MIN_SIZE, BRU_MAX_SIZE);
- fmt->field = V4L2_FIELD_NONE;
- fmt->colorspace = V4L2_COLORSPACE_SRGB;
-}
-
-static int bru_set_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_format *fmt)
-{
- struct vsp1_bru *bru = to_bru(subdev);
- struct v4l2_mbus_framefmt *format;
-
- bru_try_format(bru, cfg, fmt->pad, &fmt->format, fmt->which);
-
- format = vsp1_entity_get_pad_format(&bru->entity, cfg, fmt->pad,
- fmt->which);
- *format = fmt->format;
-
- /* Reset the compose rectangle */
- if (fmt->pad != bru->entity.source_pad) {
- struct v4l2_rect *compose;
-
- compose = bru_get_compose(bru, cfg, fmt->pad, fmt->which);
- compose->left = 0;
- compose->top = 0;
- compose->width = format->width;
- compose->height = format->height;
- }
-
- /* Propagate the format code to all pads */
- if (fmt->pad == BRU_PAD_SINK(0)) {
- unsigned int i;
-
- for (i = 0; i <= bru->entity.source_pad; ++i) {
- format = vsp1_entity_get_pad_format(&bru->entity, cfg,
- i, fmt->which);
- format->code = fmt->format.code;
- }
- }
-
- return 0;
-}
-
-static int bru_get_selection(struct v4l2_subdev *subdev,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_selection *sel)
-{
- struct vsp1_bru *bru = to_bru(subdev);
-
- if (sel->pad == bru->entity.source_pad)
- return -EINVAL;
-
- switch (sel->target) {
- case V4L2_SEL_TGT_COMPOSE_BOUNDS:
- sel->r.left = 0;
- sel->r.top = 0;
- sel->r.width = BRU_MAX_SIZE;
- sel->r.height = BRU_MAX_SIZE;
- return 0;
-
- case V4L2_SEL_TGT_COMPOSE:
- sel->r = *bru_get_compose(bru, cfg, sel->pad, sel->which);
- return 0;
-
- default:
- return -EINVAL;
- }
-}
-
-static int bru_set_selection(struct v4l2_subdev *subdev,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_selection *sel)
-{
- struct vsp1_bru *bru = to_bru(subdev);
- struct v4l2_mbus_framefmt *format;
- struct v4l2_rect *compose;
-
- if (sel->pad == bru->entity.source_pad)
- return -EINVAL;
-
- if (sel->target != V4L2_SEL_TGT_COMPOSE)
- return -EINVAL;
-
- /* The compose rectangle top left corner must be inside the output
- * frame.
- */
- format = vsp1_entity_get_pad_format(&bru->entity, cfg,
- bru->entity.source_pad, sel->which);
- sel->r.left = clamp_t(unsigned int, sel->r.left, 0, format->width - 1);
- sel->r.top = clamp_t(unsigned int, sel->r.top, 0, format->height - 1);
-
- /* Scaling isn't supported, the compose rectangle size must be identical
- * to the sink format size.
- */
- format = vsp1_entity_get_pad_format(&bru->entity, cfg, sel->pad,
- sel->which);
- sel->r.width = format->width;
- sel->r.height = format->height;
-
- compose = bru_get_compose(bru, cfg, sel->pad, sel->which);
- *compose = sel->r;
-
- return 0;
-}
-
-/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Operations
- */
-
-static struct v4l2_subdev_video_ops bru_video_ops = {
- .s_stream = bru_s_stream,
-};
-
-static struct v4l2_subdev_pad_ops bru_pad_ops = {
- .enum_mbus_code = bru_enum_mbus_code,
- .enum_frame_size = bru_enum_frame_size,
- .get_fmt = bru_get_format,
- .set_fmt = bru_set_format,
- .get_selection = bru_get_selection,
- .set_selection = bru_set_selection,
-};
-
-static struct v4l2_subdev_ops bru_ops = {
- .video = &bru_video_ops,
- .pad = &bru_pad_ops,
+static const struct vsp1_entity_operations bru_entity_ops = {
+ .configure = bru_configure,
};
/* -----------------------------------------------------------------------------
@@ -405,7 +379,6 @@
struct vsp1_bru *vsp1_bru_create(struct vsp1_device *vsp1)
{
- struct v4l2_subdev *subdev;
struct vsp1_bru *bru;
int ret;
@@ -413,31 +386,21 @@
if (bru == NULL)
return ERR_PTR(-ENOMEM);
+ bru->entity.ops = &bru_entity_ops;
bru->entity.type = VSP1_ENTITY_BRU;
- ret = vsp1_entity_init(vsp1, &bru->entity,
- vsp1->info->num_bru_inputs + 1);
+ ret = vsp1_entity_init(vsp1, &bru->entity, "bru",
+ vsp1->info->num_bru_inputs + 1, &bru_ops);
if (ret < 0)
return ERR_PTR(ret);
- /* Initialize the V4L2 subdev. */
- subdev = &bru->entity.subdev;
- v4l2_subdev_init(subdev, &bru_ops);
-
- subdev->entity.ops = &vsp1->media_ops;
- subdev->internal_ops = &vsp1_subdev_internal_ops;
- snprintf(subdev->name, sizeof(subdev->name), "%s bru",
- dev_name(vsp1->dev));
- v4l2_set_subdevdata(subdev, bru);
- subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
-
- vsp1_entity_init_formats(subdev, NULL);
-
/* Initialize the control handler. */
v4l2_ctrl_handler_init(&bru->ctrls, 1);
v4l2_ctrl_new_std(&bru->ctrls, &bru_ctrl_ops, V4L2_CID_BG_COLOR,
0, 0xffffff, 1, 0);
+ bru->bgcolor = 0;
+
bru->entity.subdev.ctrl_handler = &bru->ctrls;
if (bru->ctrls.error) {
diff --git a/drivers/media/platform/vsp1/vsp1_bru.h b/drivers/media/platform/vsp1/vsp1_bru.h
index dbac968..828a3fc 100644
--- a/drivers/media/platform/vsp1/vsp1_bru.h
+++ b/drivers/media/platform/vsp1/vsp1_bru.h
@@ -31,8 +31,9 @@
struct {
struct vsp1_rwpf *rpf;
- struct v4l2_rect compose;
} inputs[VSP1_MAX_RPF];
+
+ u32 bgcolor;
};
static inline struct vsp1_bru *to_bru(struct v4l2_subdev *subdev)
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c b/drivers/media/platform/vsp1/vsp1_dl.c
index 1a9a585..e238d9b 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -18,139 +18,406 @@
#include "vsp1.h"
#include "vsp1_dl.h"
-#include "vsp1_pipe.h"
-/*
- * Global resources
- *
- * - Display-related interrupts (can be used for vblank evasion ?)
- * - Display-list enable
- * - Header-less for WPF0
- * - DL swap
- */
-
-#define VSP1_DL_BODY_SIZE (2 * 4 * 256)
+#define VSP1_DL_NUM_ENTRIES 256
#define VSP1_DL_NUM_LISTS 3
+#define VSP1_DLH_INT_ENABLE (1 << 1)
+#define VSP1_DLH_AUTO_START (1 << 0)
+
+struct vsp1_dl_header_list {
+ u32 num_bytes;
+ u32 addr;
+} __attribute__((__packed__));
+
+struct vsp1_dl_header {
+ u32 num_lists;
+ struct vsp1_dl_header_list lists[8];
+ u32 next_header;
+ u32 flags;
+} __attribute__((__packed__));
+
struct vsp1_dl_entry {
u32 addr;
u32 data;
} __attribute__((__packed__));
-struct vsp1_dl_list {
- size_t size;
- int reg_count;
+/**
+ * struct vsp1_dl_body - Display list body
+ * @list: entry in the display list list of bodies
+ * @vsp1: the VSP1 device
+ * @entries: array of entries
+ * @dma: DMA address of the entries
+ * @size: size of the DMA memory in bytes
+ * @num_entries: number of stored entries
+ */
+struct vsp1_dl_body {
+ struct list_head list;
+ struct vsp1_device *vsp1;
- bool in_use;
-
- struct vsp1_dl_entry *body;
+ struct vsp1_dl_entry *entries;
dma_addr_t dma;
+ size_t size;
+
+ unsigned int num_entries;
};
/**
- * struct vsp1_dl - Display List manager
+ * struct vsp1_dl_list - Display list
+ * @list: entry in the display list manager lists
+ * @dlm: the display list manager
+ * @header: display list header, NULL for headerless lists
+ * @dma: DMA address for the header
+ * @body0: first display list body
+ * @fragments: list of extra display list bodies
+ */
+struct vsp1_dl_list {
+ struct list_head list;
+ struct vsp1_dl_manager *dlm;
+
+ struct vsp1_dl_header *header;
+ dma_addr_t dma;
+
+ struct vsp1_dl_body body0;
+ struct list_head fragments;
+};
+
+enum vsp1_dl_mode {
+ VSP1_DL_MODE_HEADER,
+ VSP1_DL_MODE_HEADERLESS,
+};
+
+/**
+ * struct vsp1_dl_manager - Display List manager
+ * @index: index of the related WPF
+ * @mode: display list operation mode (header or headerless)
* @vsp1: the VSP1 device
* @lock: protects the active, queued and pending lists
- * @lists.all: array of all allocate display lists
- * @lists.active: list currently being processed (loaded) by hardware
- * @lists.queued: list queued to the hardware (written to the DL registers)
- * @lists.pending: list waiting to be queued to the hardware
- * @lists.write: list being written to by software
+ * @free: array of all free display lists
+ * @active: list currently being processed (loaded) by hardware
+ * @queued: list queued to the hardware (written to the DL registers)
+ * @pending: list waiting to be queued to the hardware
*/
-struct vsp1_dl {
+struct vsp1_dl_manager {
+ unsigned int index;
+ enum vsp1_dl_mode mode;
struct vsp1_device *vsp1;
spinlock_t lock;
-
- size_t size;
- dma_addr_t dma;
- void *mem;
-
- struct {
- struct vsp1_dl_list all[VSP1_DL_NUM_LISTS];
-
- struct vsp1_dl_list *active;
- struct vsp1_dl_list *queued;
- struct vsp1_dl_list *pending;
- struct vsp1_dl_list *write;
- } lists;
+ struct list_head free;
+ struct vsp1_dl_list *active;
+ struct vsp1_dl_list *queued;
+ struct vsp1_dl_list *pending;
};
/* -----------------------------------------------------------------------------
+ * Display List Body Management
+ */
+
+/*
+ * Initialize a display list body object and allocate DMA memory for the body
+ * data. The display list body object is expected to have been initialized to
+ * 0 when allocated.
+ */
+static int vsp1_dl_body_init(struct vsp1_device *vsp1,
+ struct vsp1_dl_body *dlb, unsigned int num_entries,
+ size_t extra_size)
+{
+ size_t size = num_entries * sizeof(*dlb->entries) + extra_size;
+
+ dlb->vsp1 = vsp1;
+ dlb->size = size;
+
+ dlb->entries = dma_alloc_wc(vsp1->dev, dlb->size, &dlb->dma,
+ GFP_KERNEL);
+ if (!dlb->entries)
+ return -ENOMEM;
+
+ return 0;
+}
+
+/*
+ * Cleanup a display list body and free allocated DMA memory allocated.
+ */
+static void vsp1_dl_body_cleanup(struct vsp1_dl_body *dlb)
+{
+ dma_free_wc(dlb->vsp1->dev, dlb->size, dlb->entries, dlb->dma);
+}
+
+/**
+ * vsp1_dl_fragment_alloc - Allocate a display list fragment
+ * @vsp1: The VSP1 device
+ * @num_entries: The maximum number of entries that the fragment can contain
+ *
+ * Allocate a display list fragment with enough memory to contain the requested
+ * number of entries.
+ *
+ * Return a pointer to a fragment on success or NULL if memory can't be
+ * allocated.
+ */
+struct vsp1_dl_body *vsp1_dl_fragment_alloc(struct vsp1_device *vsp1,
+ unsigned int num_entries)
+{
+ struct vsp1_dl_body *dlb;
+ int ret;
+
+ dlb = kzalloc(sizeof(*dlb), GFP_KERNEL);
+ if (!dlb)
+ return NULL;
+
+ ret = vsp1_dl_body_init(vsp1, dlb, num_entries, 0);
+ if (ret < 0) {
+ kfree(dlb);
+ return NULL;
+ }
+
+ return dlb;
+}
+
+/**
+ * vsp1_dl_fragment_free - Free a display list fragment
+ * @dlb: The fragment
+ *
+ * Free the given display list fragment and the associated DMA memory.
+ *
+ * Fragments must only be freed explicitly if they are not added to a display
+ * list, as the display list will take ownership of them and free them
+ * otherwise. Manual free typically happens at cleanup time for fragments that
+ * have been allocated but not used.
+ *
+ * Passing a NULL pointer to this function is safe, in that case no operation
+ * will be performed.
+ */
+void vsp1_dl_fragment_free(struct vsp1_dl_body *dlb)
+{
+ if (!dlb)
+ return;
+
+ vsp1_dl_body_cleanup(dlb);
+ kfree(dlb);
+}
+
+/**
+ * vsp1_dl_fragment_write - Write a register to a display list fragment
+ * @dlb: The fragment
+ * @reg: The register address
+ * @data: The register value
+ *
+ * Write the given register and value to the display list fragment. The maximum
+ * number of entries that can be written in a fragment is specified when the
+ * fragment is allocated by vsp1_dl_fragment_alloc().
+ */
+void vsp1_dl_fragment_write(struct vsp1_dl_body *dlb, u32 reg, u32 data)
+{
+ dlb->entries[dlb->num_entries].addr = reg;
+ dlb->entries[dlb->num_entries].data = data;
+ dlb->num_entries++;
+}
+
+/* -----------------------------------------------------------------------------
* Display List Transaction Management
*/
-static void vsp1_dl_free_list(struct vsp1_dl_list *list)
+static struct vsp1_dl_list *vsp1_dl_list_alloc(struct vsp1_dl_manager *dlm)
{
- if (!list)
+ struct vsp1_dl_list *dl;
+ size_t header_size;
+ int ret;
+
+ dl = kzalloc(sizeof(*dl), GFP_KERNEL);
+ if (!dl)
+ return NULL;
+
+ INIT_LIST_HEAD(&dl->fragments);
+ dl->dlm = dlm;
+
+ /* Initialize the display list body and allocate DMA memory for the body
+ * and the optional header. Both are allocated together to avoid memory
+ * fragmentation, with the header located right after the body in
+ * memory.
+ */
+ header_size = dlm->mode == VSP1_DL_MODE_HEADER
+ ? ALIGN(sizeof(struct vsp1_dl_header), 8)
+ : 0;
+
+ ret = vsp1_dl_body_init(dlm->vsp1, &dl->body0, VSP1_DL_NUM_ENTRIES,
+ header_size);
+ if (ret < 0) {
+ kfree(dl);
+ return NULL;
+ }
+
+ if (dlm->mode == VSP1_DL_MODE_HEADER) {
+ size_t header_offset = VSP1_DL_NUM_ENTRIES
+ * sizeof(*dl->body0.entries);
+
+ dl->header = ((void *)dl->body0.entries) + header_offset;
+ dl->dma = dl->body0.dma + header_offset;
+
+ memset(dl->header, 0, sizeof(*dl->header));
+ dl->header->lists[0].addr = dl->body0.dma;
+ dl->header->flags = VSP1_DLH_INT_ENABLE;
+ }
+
+ return dl;
+}
+
+static void vsp1_dl_list_free_fragments(struct vsp1_dl_list *dl)
+{
+ struct vsp1_dl_body *dlb, *next;
+
+ list_for_each_entry_safe(dlb, next, &dl->fragments, list) {
+ list_del(&dlb->list);
+ vsp1_dl_body_cleanup(dlb);
+ kfree(dlb);
+ }
+}
+
+static void vsp1_dl_list_free(struct vsp1_dl_list *dl)
+{
+ vsp1_dl_body_cleanup(&dl->body0);
+ vsp1_dl_list_free_fragments(dl);
+ kfree(dl);
+}
+
+/**
+ * vsp1_dl_list_get - Get a free display list
+ * @dlm: The display list manager
+ *
+ * Get a display list from the pool of free lists and return it.
+ *
+ * This function must be called without the display list manager lock held.
+ */
+struct vsp1_dl_list *vsp1_dl_list_get(struct vsp1_dl_manager *dlm)
+{
+ struct vsp1_dl_list *dl = NULL;
+ unsigned long flags;
+
+ spin_lock_irqsave(&dlm->lock, flags);
+
+ if (!list_empty(&dlm->free)) {
+ dl = list_first_entry(&dlm->free, struct vsp1_dl_list, list);
+ list_del(&dl->list);
+ }
+
+ spin_unlock_irqrestore(&dlm->lock, flags);
+
+ return dl;
+}
+
+/* This function must be called with the display list manager lock held.*/
+static void __vsp1_dl_list_put(struct vsp1_dl_list *dl)
+{
+ if (!dl)
return;
- list->in_use = false;
+ vsp1_dl_list_free_fragments(dl);
+ dl->body0.num_entries = 0;
+
+ list_add_tail(&dl->list, &dl->dlm->free);
}
-void vsp1_dl_reset(struct vsp1_dl *dl)
+/**
+ * vsp1_dl_list_put - Release a display list
+ * @dl: The display list
+ *
+ * Release the display list and return it to the pool of free lists.
+ *
+ * Passing a NULL pointer to this function is safe, in that case no operation
+ * will be performed.
+ */
+void vsp1_dl_list_put(struct vsp1_dl_list *dl)
{
- unsigned int i;
-
- dl->lists.active = NULL;
- dl->lists.queued = NULL;
- dl->lists.pending = NULL;
- dl->lists.write = NULL;
-
- for (i = 0; i < ARRAY_SIZE(dl->lists.all); ++i)
- dl->lists.all[i].in_use = false;
-}
-
-void vsp1_dl_begin(struct vsp1_dl *dl)
-{
- struct vsp1_dl_list *list = NULL;
unsigned long flags;
- unsigned int i;
- spin_lock_irqsave(&dl->lock, flags);
+ if (!dl)
+ return;
- for (i = 0; i < ARRAY_SIZE(dl->lists.all); ++i) {
- if (!dl->lists.all[i].in_use) {
- list = &dl->lists.all[i];
- break;
- }
- }
-
- if (!list) {
- list = dl->lists.pending;
- dl->lists.pending = NULL;
- }
-
- spin_unlock_irqrestore(&dl->lock, flags);
-
- dl->lists.write = list;
-
- list->in_use = true;
- list->reg_count = 0;
+ spin_lock_irqsave(&dl->dlm->lock, flags);
+ __vsp1_dl_list_put(dl);
+ spin_unlock_irqrestore(&dl->dlm->lock, flags);
}
-void vsp1_dl_add(struct vsp1_entity *e, u32 reg, u32 data)
+/**
+ * vsp1_dl_list_write - Write a register to the display list
+ * @dl: The display list
+ * @reg: The register address
+ * @data: The register value
+ *
+ * Write the given register and value to the display list. Up to 256 registers
+ * can be written per display list.
+ */
+void vsp1_dl_list_write(struct vsp1_dl_list *dl, u32 reg, u32 data)
{
- struct vsp1_pipeline *pipe = to_vsp1_pipeline(&e->subdev.entity);
- struct vsp1_dl *dl = pipe->dl;
- struct vsp1_dl_list *list = dl->lists.write;
-
- list->body[list->reg_count].addr = reg;
- list->body[list->reg_count].data = data;
- list->reg_count++;
+ vsp1_dl_fragment_write(&dl->body0, reg, data);
}
-void vsp1_dl_commit(struct vsp1_dl *dl)
+/**
+ * vsp1_dl_list_add_fragment - Add a fragment to the display list
+ * @dl: The display list
+ * @dlb: The fragment
+ *
+ * Add a display list body as a fragment to a display list. Registers contained
+ * in fragments are processed after registers contained in the main display
+ * list, in the order in which fragments are added.
+ *
+ * Adding a fragment to a display list passes ownership of the fragment to the
+ * list. The caller must not touch the fragment after this call, and must not
+ * free it explicitly with vsp1_dl_fragment_free().
+ *
+ * Fragments are only usable for display lists in header mode. Attempt to
+ * add a fragment to a header-less display list will return an error.
+ */
+int vsp1_dl_list_add_fragment(struct vsp1_dl_list *dl,
+ struct vsp1_dl_body *dlb)
{
- struct vsp1_device *vsp1 = dl->vsp1;
- struct vsp1_dl_list *list;
+ /* Multi-body lists are only available in header mode. */
+ if (dl->dlm->mode != VSP1_DL_MODE_HEADER)
+ return -EINVAL;
+
+ list_add_tail(&dlb->list, &dl->fragments);
+ return 0;
+}
+
+void vsp1_dl_list_commit(struct vsp1_dl_list *dl)
+{
+ struct vsp1_dl_manager *dlm = dl->dlm;
+ struct vsp1_device *vsp1 = dlm->vsp1;
unsigned long flags;
bool update;
- list = dl->lists.write;
- dl->lists.write = NULL;
+ spin_lock_irqsave(&dlm->lock, flags);
- spin_lock_irqsave(&dl->lock, flags);
+ if (dl->dlm->mode == VSP1_DL_MODE_HEADER) {
+ struct vsp1_dl_header_list *hdr = dl->header->lists;
+ struct vsp1_dl_body *dlb;
+ unsigned int num_lists = 0;
+
+ /* Fill the header with the display list bodies addresses and
+ * sizes. The address of the first body has already been filled
+ * when the display list was allocated.
+ *
+ * In header mode the caller guarantees that the hardware is
+ * idle at this point.
+ */
+ hdr->num_bytes = dl->body0.num_entries
+ * sizeof(*dl->header->lists);
+
+ list_for_each_entry(dlb, &dl->fragments, list) {
+ num_lists++;
+ hdr++;
+
+ hdr->addr = dlb->dma;
+ hdr->num_bytes = dlb->num_entries
+ * sizeof(*dl->header->lists);
+ }
+
+ dl->header->num_lists = num_lists;
+ vsp1_write(vsp1, VI6_DL_HDR_ADDR(dlm->index), dl->dma);
+
+ dlm->active = dl;
+ goto done;
+ }
/* Once the UPD bit has been set the hardware can start processing the
* display list at any time and we can't touch the address and size
@@ -159,8 +426,8 @@
*/
update = !!(vsp1_read(vsp1, VI6_DL_BODY_SIZE) & VI6_DL_BODY_SIZE_UPD);
if (update) {
- vsp1_dl_free_list(dl->lists.pending);
- dl->lists.pending = list;
+ __vsp1_dl_list_put(dlm->pending);
+ dlm->pending = dl;
goto done;
}
@@ -168,42 +435,51 @@
* The UPD bit will be cleared by the device when the display list is
* processed.
*/
- vsp1_write(vsp1, VI6_DL_HDR_ADDR(0), list->dma);
+ vsp1_write(vsp1, VI6_DL_HDR_ADDR(0), dl->body0.dma);
vsp1_write(vsp1, VI6_DL_BODY_SIZE, VI6_DL_BODY_SIZE_UPD |
- (list->reg_count * 8));
+ (dl->body0.num_entries * sizeof(*dl->header->lists)));
- vsp1_dl_free_list(dl->lists.queued);
- dl->lists.queued = list;
+ __vsp1_dl_list_put(dlm->queued);
+ dlm->queued = dl;
done:
- spin_unlock_irqrestore(&dl->lock, flags);
+ spin_unlock_irqrestore(&dlm->lock, flags);
}
/* -----------------------------------------------------------------------------
- * Interrupt Handling
+ * Display List Manager
*/
-void vsp1_dl_irq_display_start(struct vsp1_dl *dl)
+/* Interrupt Handling */
+void vsp1_dlm_irq_display_start(struct vsp1_dl_manager *dlm)
{
- spin_lock(&dl->lock);
+ spin_lock(&dlm->lock);
/* The display start interrupt signals the end of the display list
* processing by the device. The active display list, if any, won't be
* accessed anymore and can be reused.
*/
- if (dl->lists.active) {
- vsp1_dl_free_list(dl->lists.active);
- dl->lists.active = NULL;
- }
+ __vsp1_dl_list_put(dlm->active);
+ dlm->active = NULL;
- spin_unlock(&dl->lock);
+ spin_unlock(&dlm->lock);
}
-void vsp1_dl_irq_frame_end(struct vsp1_dl *dl)
+void vsp1_dlm_irq_frame_end(struct vsp1_dl_manager *dlm)
{
- struct vsp1_device *vsp1 = dl->vsp1;
+ struct vsp1_device *vsp1 = dlm->vsp1;
- spin_lock(&dl->lock);
+ spin_lock(&dlm->lock);
+
+ __vsp1_dl_list_put(dlm->active);
+ dlm->active = NULL;
+
+ /* Header mode is used for mem-to-mem pipelines only. We don't need to
+ * perform any operation as there can't be any new display list queued
+ * in that case.
+ */
+ if (dlm->mode == VSP1_DL_MODE_HEADER)
+ goto done;
/* The UPD bit set indicates that the commit operation raced with the
* interrupt and occurred after the frame end event and UPD clear but
@@ -216,42 +492,39 @@
/* The device starts processing the queued display list right after the
* frame end interrupt. The display list thus becomes active.
*/
- if (dl->lists.queued) {
- WARN_ON(dl->lists.active);
- dl->lists.active = dl->lists.queued;
- dl->lists.queued = NULL;
+ if (dlm->queued) {
+ dlm->active = dlm->queued;
+ dlm->queued = NULL;
}
/* Now that the UPD bit has been cleared we can queue the next display
* list to the hardware if one has been prepared.
*/
- if (dl->lists.pending) {
- struct vsp1_dl_list *list = dl->lists.pending;
+ if (dlm->pending) {
+ struct vsp1_dl_list *dl = dlm->pending;
- vsp1_write(vsp1, VI6_DL_HDR_ADDR(0), list->dma);
+ vsp1_write(vsp1, VI6_DL_HDR_ADDR(0), dl->body0.dma);
vsp1_write(vsp1, VI6_DL_BODY_SIZE, VI6_DL_BODY_SIZE_UPD |
- (list->reg_count * 8));
+ (dl->body0.num_entries *
+ sizeof(*dl->header->lists)));
- dl->lists.queued = list;
- dl->lists.pending = NULL;
+ dlm->queued = dl;
+ dlm->pending = NULL;
}
done:
- spin_unlock(&dl->lock);
+ spin_unlock(&dlm->lock);
}
-/* -----------------------------------------------------------------------------
- * Hardware Setup
- */
-
-void vsp1_dl_setup(struct vsp1_device *vsp1)
+/* Hardware Setup */
+void vsp1_dlm_setup(struct vsp1_device *vsp1)
{
u32 ctrl = (256 << VI6_DL_CTRL_AR_WAIT_SHIFT)
| VI6_DL_CTRL_DC2 | VI6_DL_CTRL_DC1 | VI6_DL_CTRL_DC0
| VI6_DL_CTRL_DLE;
- /* The DRM pipeline operates with header-less display lists in
- * Continuous Frame Mode.
+ /* The DRM pipeline operates with display lists in Continuous Frame
+ * Mode, all other pipelines use manual start.
*/
if (vsp1->drm)
ctrl |= VI6_DL_CTRL_CFM0 | VI6_DL_CTRL_NH0;
@@ -260,46 +533,64 @@
vsp1_write(vsp1, VI6_DL_SWAP, VI6_DL_SWAP_LWS);
}
-/* -----------------------------------------------------------------------------
- * Initialization and Cleanup
- */
-
-struct vsp1_dl *vsp1_dl_create(struct vsp1_device *vsp1)
+void vsp1_dlm_reset(struct vsp1_dl_manager *dlm)
{
- struct vsp1_dl *dl;
- unsigned int i;
+ unsigned long flags;
- dl = kzalloc(sizeof(*dl), GFP_KERNEL);
- if (!dl)
- return NULL;
+ spin_lock_irqsave(&dlm->lock, flags);
- spin_lock_init(&dl->lock);
+ __vsp1_dl_list_put(dlm->active);
+ __vsp1_dl_list_put(dlm->queued);
+ __vsp1_dl_list_put(dlm->pending);
- dl->vsp1 = vsp1;
- dl->size = VSP1_DL_BODY_SIZE * ARRAY_SIZE(dl->lists.all);
+ spin_unlock_irqrestore(&dlm->lock, flags);
- dl->mem = dma_alloc_wc(vsp1->dev, dl->size, &dl->dma,
- GFP_KERNEL);
- if (!dl->mem) {
- kfree(dl);
- return NULL;
- }
-
- for (i = 0; i < ARRAY_SIZE(dl->lists.all); ++i) {
- struct vsp1_dl_list *list = &dl->lists.all[i];
-
- list->size = VSP1_DL_BODY_SIZE;
- list->reg_count = 0;
- list->in_use = false;
- list->dma = dl->dma + VSP1_DL_BODY_SIZE * i;
- list->body = dl->mem + VSP1_DL_BODY_SIZE * i;
- }
-
- return dl;
+ dlm->active = NULL;
+ dlm->queued = NULL;
+ dlm->pending = NULL;
}
-void vsp1_dl_destroy(struct vsp1_dl *dl)
+struct vsp1_dl_manager *vsp1_dlm_create(struct vsp1_device *vsp1,
+ unsigned int index,
+ unsigned int prealloc)
{
- dma_free_wc(dl->vsp1->dev, dl->size, dl->mem, dl->dma);
- kfree(dl);
+ struct vsp1_dl_manager *dlm;
+ unsigned int i;
+
+ dlm = devm_kzalloc(vsp1->dev, sizeof(*dlm), GFP_KERNEL);
+ if (!dlm)
+ return NULL;
+
+ dlm->index = index;
+ dlm->mode = index == 0 && !vsp1->info->uapi
+ ? VSP1_DL_MODE_HEADERLESS : VSP1_DL_MODE_HEADER;
+ dlm->vsp1 = vsp1;
+
+ spin_lock_init(&dlm->lock);
+ INIT_LIST_HEAD(&dlm->free);
+
+ for (i = 0; i < prealloc; ++i) {
+ struct vsp1_dl_list *dl;
+
+ dl = vsp1_dl_list_alloc(dlm);
+ if (!dl)
+ return NULL;
+
+ list_add_tail(&dl->list, &dlm->free);
+ }
+
+ return dlm;
+}
+
+void vsp1_dlm_destroy(struct vsp1_dl_manager *dlm)
+{
+ struct vsp1_dl_list *dl, *next;
+
+ if (!dlm)
+ return;
+
+ list_for_each_entry_safe(dl, next, &dlm->free, list) {
+ list_del(&dl->list);
+ vsp1_dl_list_free(dl);
+ }
}
diff --git a/drivers/media/platform/vsp1/vsp1_dl.h b/drivers/media/platform/vsp1/vsp1_dl.h
index 448c425..de387cd 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.h
+++ b/drivers/media/platform/vsp1/vsp1_dl.h
@@ -13,30 +13,33 @@
#ifndef __VSP1_DL_H__
#define __VSP1_DL_H__
-#include "vsp1_entity.h"
+#include <linux/types.h>
struct vsp1_device;
-struct vsp1_dl;
+struct vsp1_dl_fragment;
+struct vsp1_dl_list;
+struct vsp1_dl_manager;
-struct vsp1_dl *vsp1_dl_create(struct vsp1_device *vsp1);
-void vsp1_dl_destroy(struct vsp1_dl *dl);
+void vsp1_dlm_setup(struct vsp1_device *vsp1);
-void vsp1_dl_setup(struct vsp1_device *vsp1);
+struct vsp1_dl_manager *vsp1_dlm_create(struct vsp1_device *vsp1,
+ unsigned int index,
+ unsigned int prealloc);
+void vsp1_dlm_destroy(struct vsp1_dl_manager *dlm);
+void vsp1_dlm_reset(struct vsp1_dl_manager *dlm);
+void vsp1_dlm_irq_display_start(struct vsp1_dl_manager *dlm);
+void vsp1_dlm_irq_frame_end(struct vsp1_dl_manager *dlm);
-void vsp1_dl_reset(struct vsp1_dl *dl);
-void vsp1_dl_begin(struct vsp1_dl *dl);
-void vsp1_dl_add(struct vsp1_entity *e, u32 reg, u32 data);
-void vsp1_dl_commit(struct vsp1_dl *dl);
+struct vsp1_dl_list *vsp1_dl_list_get(struct vsp1_dl_manager *dlm);
+void vsp1_dl_list_put(struct vsp1_dl_list *dl);
+void vsp1_dl_list_write(struct vsp1_dl_list *dl, u32 reg, u32 data);
+void vsp1_dl_list_commit(struct vsp1_dl_list *dl);
-void vsp1_dl_irq_display_start(struct vsp1_dl *dl);
-void vsp1_dl_irq_frame_end(struct vsp1_dl *dl);
-
-static inline void vsp1_dl_mod_write(struct vsp1_entity *e, u32 reg, u32 data)
-{
- if (e->vsp1->use_dl)
- vsp1_dl_add(e, reg, data);
- else
- vsp1_write(e->vsp1, reg, data);
-}
+struct vsp1_dl_body *vsp1_dl_fragment_alloc(struct vsp1_device *vsp1,
+ unsigned int num_entries);
+void vsp1_dl_fragment_free(struct vsp1_dl_body *dlb);
+void vsp1_dl_fragment_write(struct vsp1_dl_body *dlb, u32 reg, u32 data);
+int vsp1_dl_list_add_fragment(struct vsp1_dl_list *dl,
+ struct vsp1_dl_body *dlb);
#endif /* __VSP1_DL_H__ */
diff --git a/drivers/media/platform/vsp1/vsp1_drm.c b/drivers/media/platform/vsp1/vsp1_drm.c
index 021fe57..fc4bbc4 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.c
+++ b/drivers/media/platform/vsp1/vsp1_drm.c
@@ -13,10 +13,10 @@
#include <linux/device.h>
#include <linux/slab.h>
-#include <linux/vsp1.h>
#include <media/media-entity.h>
#include <media/v4l2-subdev.h>
+#include <media/vsp1.h>
#include "vsp1.h"
#include "vsp1_bru.h"
@@ -26,18 +26,14 @@
#include "vsp1_pipe.h"
#include "vsp1_rwpf.h"
+
/* -----------------------------------------------------------------------------
- * Runtime Handling
+ * Interrupt Handling
*/
-static void vsp1_drm_pipeline_frame_end(struct vsp1_pipeline *pipe)
+void vsp1_drm_display_start(struct vsp1_device *vsp1)
{
- unsigned long flags;
-
- spin_lock_irqsave(&pipe->irqlock, flags);
- if (pipe->num_inputs)
- vsp1_pipeline_run(pipe);
- spin_unlock_irqrestore(&pipe->irqlock, flags);
+ vsp1_dlm_irq_display_start(vsp1->drm->pipe.output->dlm);
}
/* -----------------------------------------------------------------------------
@@ -97,12 +93,14 @@
media_entity_pipeline_stop(&pipe->output->entity.subdev.entity);
for (i = 0; i < bru->entity.source_pad; ++i) {
+ vsp1->drm->inputs[i].enabled = false;
bru->inputs[i].rpf = NULL;
pipe->inputs[i] = NULL;
}
pipe->num_inputs = 0;
+ vsp1_dlm_reset(pipe->output->dlm);
vsp1_device_put(vsp1);
dev_dbg(vsp1->dev, "%s: pipeline disabled\n", __func__);
@@ -110,8 +108,6 @@
return 0;
}
- vsp1_dl_reset(vsp1->drm->dl);
-
/* Configure the format at the BRU sinks and propagate it through the
* pipeline.
*/
@@ -222,16 +218,11 @@
{
struct vsp1_device *vsp1 = dev_get_drvdata(dev);
struct vsp1_pipeline *pipe = &vsp1->drm->pipe;
- unsigned long flags;
-
- spin_lock_irqsave(&pipe->irqlock, flags);
vsp1->drm->num_inputs = pipe->num_inputs;
- spin_unlock_irqrestore(&pipe->irqlock, flags);
-
/* Prepare the display list. */
- vsp1_dl_begin(vsp1->drm->dl);
+ pipe->dl = vsp1_dl_list_get(pipe->output->dlm);
}
EXPORT_SYMBOL_GPL(vsp1_du_atomic_begin);
@@ -244,10 +235,13 @@
* @mem: DMA addresses of the memory buffers (one per plane)
* @src: the source crop rectangle for the RPF
* @dst: the destination compose rectangle for the BRU input
+ * @alpha: global alpha value for the input
+ * @zpos: the Z-order position of the input
*
* Configure the VSP to perform composition of the image referenced by @mem
* through RPF @rpf_index, using the @src crop rectangle and the @dst
- * composition rectangle. The Z-order is fixed with RPF 0 at the bottom.
+ * composition rectangle. The Z-order is configurable with higher @zpos values
+ * displayed on top.
*
* Image format as stored in memory is expressed as a V4L2 @pixelformat value.
* As a special case, setting the pixel format to 0 will disable the RPF. The
@@ -265,25 +259,17 @@
*
* This function isn't reentrant, the caller needs to serialize calls.
*
- * TODO: Implement Z-order control by decoupling the RPF index from the BRU
- * input index.
- *
* Return 0 on success or a negative error code on failure.
*/
-int vsp1_du_atomic_update(struct device *dev, unsigned int rpf_index,
- u32 pixelformat, unsigned int pitch,
- dma_addr_t mem[2], const struct v4l2_rect *src,
- const struct v4l2_rect *dst)
+int vsp1_du_atomic_update_ext(struct device *dev, unsigned int rpf_index,
+ u32 pixelformat, unsigned int pitch,
+ dma_addr_t mem[2], const struct v4l2_rect *src,
+ const struct v4l2_rect *dst, unsigned int alpha,
+ unsigned int zpos)
{
struct vsp1_device *vsp1 = dev_get_drvdata(dev);
- struct vsp1_pipeline *pipe = &vsp1->drm->pipe;
const struct vsp1_format_info *fmtinfo;
- struct v4l2_subdev_selection sel;
- struct v4l2_subdev_format format;
- struct vsp1_rwpf_memory memory;
struct vsp1_rwpf *rpf;
- unsigned long flags;
- int ret;
if (rpf_index >= vsp1->info->rpf_count)
return -EINVAL;
@@ -294,31 +280,20 @@
dev_dbg(vsp1->dev, "%s: RPF%u: disable requested\n", __func__,
rpf_index);
- spin_lock_irqsave(&pipe->irqlock, flags);
-
- if (pipe->inputs[rpf_index]) {
- /* Remove the RPF from the pipeline if it was previously
- * enabled.
- */
- vsp1->bru->inputs[rpf_index].rpf = NULL;
- pipe->inputs[rpf_index] = NULL;
-
- pipe->num_inputs--;
- }
-
- spin_unlock_irqrestore(&pipe->irqlock, flags);
-
+ vsp1->drm->inputs[rpf_index].enabled = false;
return 0;
}
dev_dbg(vsp1->dev,
- "%s: RPF%u: (%u,%u)/%ux%u -> (%u,%u)/%ux%u (%08x), pitch %u dma { %pad, %pad }\n",
+ "%s: RPF%u: (%u,%u)/%ux%u -> (%u,%u)/%ux%u (%08x), pitch %u dma { %pad, %pad } zpos %u\n",
__func__, rpf_index,
src->left, src->top, src->width, src->height,
dst->left, dst->top, dst->width, dst->height,
- pixelformat, pitch, &mem[0], &mem[1]);
+ pixelformat, pitch, &mem[0], &mem[1], zpos);
- /* Set the stride at the RPF input. */
+ /* Store the format, stride, memory buffer address, crop and compose
+ * rectangles and Z-order position and for the input.
+ */
fmtinfo = vsp1_get_format_info(pixelformat);
if (!fmtinfo) {
dev_dbg(vsp1->dev, "Unsupport pixel format %08x for RPF\n",
@@ -330,16 +305,40 @@
rpf->format.num_planes = fmtinfo->planes;
rpf->format.plane_fmt[0].bytesperline = pitch;
rpf->format.plane_fmt[1].bytesperline = pitch;
+ rpf->alpha = alpha;
+
+ rpf->mem.addr[0] = mem[0];
+ rpf->mem.addr[1] = mem[1];
+ rpf->mem.addr[2] = 0;
+
+ vsp1->drm->inputs[rpf_index].crop = *src;
+ vsp1->drm->inputs[rpf_index].compose = *dst;
+ vsp1->drm->inputs[rpf_index].zpos = zpos;
+ vsp1->drm->inputs[rpf_index].enabled = true;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(vsp1_du_atomic_update_ext);
+
+static int vsp1_du_setup_rpf_pipe(struct vsp1_device *vsp1,
+ struct vsp1_rwpf *rpf, unsigned int bru_input)
+{
+ struct v4l2_subdev_selection sel;
+ struct v4l2_subdev_format format;
+ const struct v4l2_rect *crop;
+ int ret;
/* Configure the format on the RPF sink pad and propagate it up to the
* BRU sink pad.
*/
+ crop = &vsp1->drm->inputs[rpf->entity.index].crop;
+
memset(&format, 0, sizeof(format));
format.which = V4L2_SUBDEV_FORMAT_ACTIVE;
format.pad = RWPF_PAD_SINK;
- format.format.width = src->width + src->left;
- format.format.height = src->height + src->top;
- format.format.code = fmtinfo->mbus;
+ format.format.width = crop->width + crop->left;
+ format.format.height = crop->height + crop->top;
+ format.format.code = rpf->fmtinfo->mbus;
format.format.field = V4L2_FIELD_NONE;
ret = v4l2_subdev_call(&rpf->entity.subdev, pad, set_fmt, NULL,
@@ -356,7 +355,7 @@
sel.which = V4L2_SUBDEV_FORMAT_ACTIVE;
sel.pad = RWPF_PAD_SINK;
sel.target = V4L2_SEL_TGT_CROP;
- sel.r = *src;
+ sel.r = *crop;
ret = v4l2_subdev_call(&rpf->entity.subdev, pad, set_selection, NULL,
&sel);
@@ -391,7 +390,7 @@
return ret;
/* BRU sink, propagate the format from the RPF source. */
- format.pad = rpf->entity.index;
+ format.pad = bru_input;
ret = v4l2_subdev_call(&vsp1->bru->entity.subdev, pad, set_fmt, NULL,
&format);
@@ -402,9 +401,9 @@
__func__, format.format.width, format.format.height,
format.format.code, format.pad);
- sel.pad = rpf->entity.index;
+ sel.pad = bru_input;
sel.target = V4L2_SEL_TGT_COMPOSE;
- sel.r = *dst;
+ sel.r = vsp1->drm->inputs[rpf->entity.index].compose;
ret = v4l2_subdev_call(&vsp1->bru->entity.subdev, pad, set_selection,
NULL, &sel);
@@ -416,33 +415,13 @@
__func__, sel.r.left, sel.r.top, sel.r.width, sel.r.height,
sel.pad);
- /* Store the compose rectangle coordinates in the RPF. */
- rpf->location.left = dst->left;
- rpf->location.top = dst->top;
-
- /* Set the memory buffer address. */
- memory.num_planes = fmtinfo->planes;
- memory.addr[0] = mem[0];
- memory.addr[1] = mem[1];
-
- rpf->ops->set_memory(rpf, &memory);
-
- spin_lock_irqsave(&pipe->irqlock, flags);
-
- /* If the RPF was previously stopped set the BRU input to the RPF and
- * store the RPF in the pipeline inputs array.
- */
- if (!pipe->inputs[rpf->entity.index]) {
- vsp1->bru->inputs[rpf_index].rpf = rpf;
- pipe->inputs[rpf->entity.index] = rpf;
- pipe->num_inputs++;
- }
-
- spin_unlock_irqrestore(&pipe->irqlock, flags);
-
return 0;
}
-EXPORT_SYMBOL_GPL(vsp1_du_atomic_update);
+
+static unsigned int rpf_zpos(struct vsp1_device *vsp1, struct vsp1_rwpf *rpf)
+{
+ return vsp1->drm->inputs[rpf->entity.index].zpos;
+}
/**
* vsp1_du_atomic_flush - Commit an atomic update
@@ -452,51 +431,96 @@
{
struct vsp1_device *vsp1 = dev_get_drvdata(dev);
struct vsp1_pipeline *pipe = &vsp1->drm->pipe;
+ struct vsp1_rwpf *inputs[VSP1_MAX_RPF] = { NULL, };
struct vsp1_entity *entity;
unsigned long flags;
- bool stop = false;
+ unsigned int i;
int ret;
+ /* Count the number of enabled inputs and sort them by Z-order. */
+ pipe->num_inputs = 0;
+
+ for (i = 0; i < vsp1->info->rpf_count; ++i) {
+ struct vsp1_rwpf *rpf = vsp1->rpf[i];
+ unsigned int j;
+
+ if (!vsp1->drm->inputs[i].enabled) {
+ pipe->inputs[i] = NULL;
+ continue;
+ }
+
+ pipe->inputs[i] = rpf;
+
+ /* Insert the RPF in the sorted RPFs array. */
+ for (j = pipe->num_inputs++; j > 0; --j) {
+ if (rpf_zpos(vsp1, inputs[j-1]) <= rpf_zpos(vsp1, rpf))
+ break;
+ inputs[j] = inputs[j-1];
+ }
+
+ inputs[j] = rpf;
+ }
+
+ /* Setup the RPF input pipeline for every enabled input. */
+ for (i = 0; i < vsp1->info->num_bru_inputs; ++i) {
+ struct vsp1_rwpf *rpf = inputs[i];
+
+ if (!rpf) {
+ vsp1->bru->inputs[i].rpf = NULL;
+ continue;
+ }
+
+ vsp1->bru->inputs[i].rpf = rpf;
+ rpf->bru_input = i;
+ rpf->entity.sink_pad = i;
+
+ dev_dbg(vsp1->dev, "%s: connecting RPF.%u to BRU:%u\n",
+ __func__, rpf->entity.index, i);
+
+ ret = vsp1_du_setup_rpf_pipe(vsp1, rpf, i);
+ if (ret < 0)
+ dev_err(vsp1->dev,
+ "%s: failed to setup RPF.%u\n",
+ __func__, rpf->entity.index);
+ }
+
+ /* Configure all entities in the pipeline. */
list_for_each_entry(entity, &pipe->entities, list_pipe) {
/* Disconnect unused RPFs from the pipeline. */
if (entity->type == VSP1_ENTITY_RPF) {
struct vsp1_rwpf *rpf = to_rwpf(&entity->subdev);
if (!pipe->inputs[rpf->entity.index]) {
- vsp1_mod_write(entity, entity->route->reg,
- VI6_DPR_NODE_UNUSED);
+ vsp1_dl_list_write(pipe->dl, entity->route->reg,
+ VI6_DPR_NODE_UNUSED);
continue;
}
}
- vsp1_entity_route_setup(entity);
+ vsp1_entity_route_setup(entity, pipe->dl);
- ret = v4l2_subdev_call(&entity->subdev, video,
- s_stream, 1);
- if (ret < 0) {
- dev_err(vsp1->dev,
- "DRM pipeline start failure on entity %s\n",
- entity->subdev.name);
- return;
- }
+ if (entity->ops->configure)
+ entity->ops->configure(entity, pipe, pipe->dl);
+
+ /* The memory buffer address must be applied after configuring
+ * the RPF to make sure the crop offset are computed.
+ */
+ if (entity->type == VSP1_ENTITY_RPF)
+ vsp1_rwpf_set_memory(to_rwpf(&entity->subdev),
+ pipe->dl);
}
- vsp1_dl_commit(vsp1->drm->dl);
-
- spin_lock_irqsave(&pipe->irqlock, flags);
+ vsp1_dl_list_commit(pipe->dl);
+ pipe->dl = NULL;
/* Start or stop the pipeline if needed. */
if (!vsp1->drm->num_inputs && pipe->num_inputs) {
vsp1_write(vsp1, VI6_DISP_IRQ_STA, 0);
vsp1_write(vsp1, VI6_DISP_IRQ_ENB, VI6_DISP_IRQ_ENB_DSTE);
+ spin_lock_irqsave(&pipe->irqlock, flags);
vsp1_pipeline_run(pipe);
+ spin_unlock_irqrestore(&pipe->irqlock, flags);
} else if (vsp1->drm->num_inputs && !pipe->num_inputs) {
- stop = true;
- }
-
- spin_unlock_irqrestore(&pipe->irqlock, flags);
-
- if (stop) {
vsp1_write(vsp1, VI6_DISP_IRQ_ENB, 0);
vsp1_pipeline_stop(pipe);
}
@@ -562,14 +586,9 @@
if (!vsp1->drm)
return -ENOMEM;
- vsp1->drm->dl = vsp1_dl_create(vsp1);
- if (!vsp1->drm->dl)
- return -ENOMEM;
-
pipe = &vsp1->drm->pipe;
vsp1_pipeline_init(pipe);
- pipe->frame_end = vsp1_drm_pipeline_frame_end;
/* The DRM pipeline is static, add entities manually. */
for (i = 0; i < vsp1->info->rpf_count; ++i) {
@@ -586,12 +605,9 @@
pipe->lif = &vsp1->lif->entity;
pipe->output = vsp1->wpf[0];
- pipe->dl = vsp1->drm->dl;
-
return 0;
}
void vsp1_drm_cleanup(struct vsp1_device *vsp1)
{
- vsp1_dl_destroy(vsp1->drm->dl);
}
diff --git a/drivers/media/platform/vsp1/vsp1_drm.h b/drivers/media/platform/vsp1/vsp1_drm.h
index f680568..9e28ab9 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.h
+++ b/drivers/media/platform/vsp1/vsp1_drm.h
@@ -13,37 +13,32 @@
#ifndef __VSP1_DRM_H__
#define __VSP1_DRM_H__
-#include "vsp1_pipe.h"
+#include <linux/videodev2.h>
-struct vsp1_dl;
+#include "vsp1_pipe.h"
/**
* vsp1_drm - State for the API exposed to the DRM driver
- * @dl: display list for DRM pipeline operation
* @pipe: the VSP1 pipeline used for display
* @num_inputs: number of active pipeline inputs at the beginning of an update
- * @update: the pipeline configuration has been updated
+ * @planes: source crop rectangle, destination compose rectangle and z-order
+ * position for every input
*/
struct vsp1_drm {
- struct vsp1_dl *dl;
struct vsp1_pipeline pipe;
unsigned int num_inputs;
- bool update;
+ struct {
+ bool enabled;
+ struct v4l2_rect crop;
+ struct v4l2_rect compose;
+ unsigned int zpos;
+ } inputs[VSP1_MAX_RPF];
};
int vsp1_drm_init(struct vsp1_device *vsp1);
void vsp1_drm_cleanup(struct vsp1_device *vsp1);
int vsp1_drm_create_links(struct vsp1_device *vsp1);
-int vsp1_du_init(struct device *dev);
-int vsp1_du_setup_lif(struct device *dev, unsigned int width,
- unsigned int height);
-void vsp1_du_atomic_begin(struct device *dev);
-int vsp1_du_atomic_update(struct device *dev, unsigned int rpf_index,
- u32 pixelformat, unsigned int pitch,
- dma_addr_t mem[2], const struct v4l2_rect *src,
- const struct v4l2_rect *dst);
-void vsp1_du_atomic_flush(struct device *dev);
-
+void vsp1_drm_display_start(struct vsp1_device *vsp1);
#endif /* __VSP1_DRM_H__ */
diff --git a/drivers/media/platform/vsp1/vsp1_drv.c b/drivers/media/platform/vsp1/vsp1_drv.c
index 25750a0..e2d779f 100644
--- a/drivers/media/platform/vsp1/vsp1_drv.c
+++ b/drivers/media/platform/vsp1/vsp1_drv.c
@@ -30,6 +30,7 @@
#include "vsp1_hsit.h"
#include "vsp1_lif.h"
#include "vsp1_lut.h"
+#include "vsp1_pipe.h"
#include "vsp1_rwpf.h"
#include "vsp1_sru.h"
#include "vsp1_uds.h"
@@ -49,17 +50,15 @@
for (i = 0; i < vsp1->info->wpf_count; ++i) {
struct vsp1_rwpf *wpf = vsp1->wpf[i];
- struct vsp1_pipeline *pipe;
if (wpf == NULL)
continue;
- pipe = to_vsp1_pipeline(&wpf->entity.subdev.entity);
status = vsp1_read(vsp1, VI6_WPF_IRQ_STA(i));
vsp1_write(vsp1, VI6_WPF_IRQ_STA(i), ~status & mask);
if (status & VI6_WFP_IRQ_STA_FRE) {
- vsp1_pipeline_frame_end(pipe);
+ vsp1_pipeline_frame_end(wpf->pipe);
ret = IRQ_HANDLED;
}
}
@@ -68,14 +67,7 @@
vsp1_write(vsp1, VI6_DISP_IRQ_STA, ~status & VI6_DISP_IRQ_STA_DST);
if (status & VI6_DISP_IRQ_STA_DST) {
- struct vsp1_rwpf *wpf = vsp1->wpf[0];
- struct vsp1_pipeline *pipe;
-
- if (wpf) {
- pipe = to_vsp1_pipeline(&wpf->entity.subdev.entity);
- vsp1_pipeline_display_start(pipe);
- }
-
+ vsp1_drm_display_start(vsp1);
ret = IRQ_HANDLED;
}
@@ -387,13 +379,10 @@
/* Register subdev nodes if the userspace API is enabled or initialize
* the DRM pipeline otherwise.
*/
- if (vsp1->info->uapi) {
- vsp1->use_dl = false;
+ if (vsp1->info->uapi)
ret = v4l2_device_register_subdev_nodes(&vsp1->v4l2_dev);
- } else {
- vsp1->use_dl = true;
+ else
ret = vsp1_drm_init(vsp1);
- }
if (ret < 0)
goto done;
@@ -465,8 +454,7 @@
vsp1_write(vsp1, VI6_DPR_HGT_SMPPT, (7 << VI6_DPR_SMPPT_TGW_SHIFT) |
(VI6_DPR_NODE_UNUSED << VI6_DPR_SMPPT_PT_SHIFT));
- if (vsp1->use_dl)
- vsp1_dl_setup(vsp1);
+ vsp1_dlm_setup(vsp1);
return 0;
}
@@ -570,6 +558,7 @@
static const struct vsp1_device_info vsp1_device_infos[] = {
{
.version = VI6_IP_VERSION_MODEL_VSPS_H2,
+ .gen = 2,
.features = VSP1_HAS_BRU | VSP1_HAS_LUT | VSP1_HAS_SRU,
.rpf_count = 5,
.uds_count = 3,
@@ -578,6 +567,7 @@
.uapi = true,
}, {
.version = VI6_IP_VERSION_MODEL_VSPR_H2,
+ .gen = 2,
.features = VSP1_HAS_BRU | VSP1_HAS_SRU,
.rpf_count = 5,
.uds_count = 1,
@@ -586,6 +576,7 @@
.uapi = true,
}, {
.version = VI6_IP_VERSION_MODEL_VSPD_GEN2,
+ .gen = 2,
.features = VSP1_HAS_BRU | VSP1_HAS_LIF | VSP1_HAS_LUT,
.rpf_count = 4,
.uds_count = 1,
@@ -594,6 +585,7 @@
.uapi = true,
}, {
.version = VI6_IP_VERSION_MODEL_VSPS_M2,
+ .gen = 2,
.features = VSP1_HAS_BRU | VSP1_HAS_LUT | VSP1_HAS_SRU,
.rpf_count = 5,
.uds_count = 3,
@@ -602,6 +594,7 @@
.uapi = true,
}, {
.version = VI6_IP_VERSION_MODEL_VSPI_GEN3,
+ .gen = 3,
.features = VSP1_HAS_LUT | VSP1_HAS_SRU,
.rpf_count = 1,
.uds_count = 1,
@@ -609,6 +602,7 @@
.uapi = true,
}, {
.version = VI6_IP_VERSION_MODEL_VSPBD_GEN3,
+ .gen = 3,
.features = VSP1_HAS_BRU,
.rpf_count = 5,
.wpf_count = 1,
@@ -616,6 +610,7 @@
.uapi = true,
}, {
.version = VI6_IP_VERSION_MODEL_VSPBC_GEN3,
+ .gen = 3,
.features = VSP1_HAS_BRU | VSP1_HAS_LUT,
.rpf_count = 5,
.wpf_count = 1,
@@ -623,7 +618,8 @@
.uapi = true,
}, {
.version = VI6_IP_VERSION_MODEL_VSPD_GEN3,
- .features = VSP1_HAS_BRU | VSP1_HAS_LIF | VSP1_HAS_LUT,
+ .gen = 3,
+ .features = VSP1_HAS_BRU | VSP1_HAS_LIF,
.rpf_count = 5,
.wpf_count = 2,
.num_bru_inputs = 5,
diff --git a/drivers/media/platform/vsp1/vsp1_entity.c b/drivers/media/platform/vsp1/vsp1_entity.c
index 20a78fb..3d070bc 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/vsp1/vsp1_entity.c
@@ -19,46 +19,11 @@
#include <media/v4l2-subdev.h>
#include "vsp1.h"
+#include "vsp1_dl.h"
#include "vsp1_entity.h"
-bool vsp1_entity_is_streaming(struct vsp1_entity *entity)
-{
- unsigned long flags;
- bool streaming;
-
- spin_lock_irqsave(&entity->lock, flags);
- streaming = entity->streaming;
- spin_unlock_irqrestore(&entity->lock, flags);
-
- return streaming;
-}
-
-int vsp1_entity_set_streaming(struct vsp1_entity *entity, bool streaming)
-{
- unsigned long flags;
- int ret;
-
- spin_lock_irqsave(&entity->lock, flags);
- entity->streaming = streaming;
- spin_unlock_irqrestore(&entity->lock, flags);
-
- if (!streaming)
- return 0;
-
- if (!entity->vsp1->info->uapi || !entity->subdev.ctrl_handler)
- return 0;
-
- ret = v4l2_ctrl_handler_setup(entity->subdev.ctrl_handler);
- if (ret < 0) {
- spin_lock_irqsave(&entity->lock, flags);
- entity->streaming = false;
- spin_unlock_irqrestore(&entity->lock, flags);
- }
-
- return ret;
-}
-
-void vsp1_entity_route_setup(struct vsp1_entity *source)
+void vsp1_entity_route_setup(struct vsp1_entity *source,
+ struct vsp1_dl_list *dl)
{
struct vsp1_entity *sink;
@@ -66,40 +31,74 @@
return;
sink = container_of(source->sink, struct vsp1_entity, subdev.entity);
- vsp1_mod_write(source, source->route->reg,
- sink->route->inputs[source->sink_pad]);
+ vsp1_dl_list_write(dl, source->route->reg,
+ sink->route->inputs[source->sink_pad]);
}
/* -----------------------------------------------------------------------------
* V4L2 Subdevice Operations
*/
-struct v4l2_mbus_framefmt *
-vsp1_entity_get_pad_format(struct vsp1_entity *entity,
+/**
+ * vsp1_entity_get_pad_config - Get the pad configuration for an entity
+ * @entity: the entity
+ * @cfg: the TRY pad configuration
+ * @which: configuration selector (ACTIVE or TRY)
+ *
+ * Return the pad configuration requested by the which argument. The TRY
+ * configuration is passed explicitly to the function through the cfg argument
+ * and simply returned when requested. The ACTIVE configuration comes from the
+ * entity structure.
+ */
+struct v4l2_subdev_pad_config *
+vsp1_entity_get_pad_config(struct vsp1_entity *entity,
struct v4l2_subdev_pad_config *cfg,
- unsigned int pad, u32 which)
+ enum v4l2_subdev_format_whence which)
{
switch (which) {
- case V4L2_SUBDEV_FORMAT_TRY:
- return v4l2_subdev_get_try_format(&entity->subdev, cfg, pad);
case V4L2_SUBDEV_FORMAT_ACTIVE:
- return &entity->formats[pad];
+ return entity->config;
+ case V4L2_SUBDEV_FORMAT_TRY:
default:
- return NULL;
+ return cfg;
}
}
+/**
+ * vsp1_entity_get_pad_format - Get a pad format from storage for an entity
+ * @entity: the entity
+ * @cfg: the configuration storage
+ * @pad: the pad number
+ *
+ * Return the format stored in the given configuration for an entity's pad. The
+ * configuration can be an ACTIVE or TRY configuration.
+ */
+struct v4l2_mbus_framefmt *
+vsp1_entity_get_pad_format(struct vsp1_entity *entity,
+ struct v4l2_subdev_pad_config *cfg,
+ unsigned int pad)
+{
+ return v4l2_subdev_get_try_format(&entity->subdev, cfg, pad);
+}
+
+struct v4l2_rect *
+vsp1_entity_get_pad_compose(struct vsp1_entity *entity,
+ struct v4l2_subdev_pad_config *cfg,
+ unsigned int pad)
+{
+ return v4l2_subdev_get_try_compose(&entity->subdev, cfg, pad);
+}
+
/*
- * vsp1_entity_init_formats - Initialize formats on all pads
+ * vsp1_entity_init_cfg - Initialize formats on all pads
* @subdev: V4L2 subdevice
* @cfg: V4L2 subdev pad configuration
*
- * Initialize all pad formats with default values. If cfg is not NULL, try
- * formats are initialized on the file handle. Otherwise active formats are
- * initialized on the device.
+ * Initialize all pad formats with default values in the given pad config. This
+ * function can be used as a handler for the subdev pad::init_cfg operation.
*/
-void vsp1_entity_init_formats(struct v4l2_subdev *subdev,
- struct v4l2_subdev_pad_config *cfg)
+int vsp1_entity_init_cfg(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_pad_config *cfg)
{
struct v4l2_subdev_format format;
unsigned int pad;
@@ -113,19 +112,132 @@
v4l2_subdev_call(subdev, pad, set_fmt, cfg, &format);
}
-}
-
-static int vsp1_entity_open(struct v4l2_subdev *subdev,
- struct v4l2_subdev_fh *fh)
-{
- vsp1_entity_init_formats(subdev, fh->pad);
return 0;
}
-const struct v4l2_subdev_internal_ops vsp1_subdev_internal_ops = {
- .open = vsp1_entity_open,
-};
+/*
+ * vsp1_subdev_get_pad_format - Subdev pad get_fmt handler
+ * @subdev: V4L2 subdevice
+ * @cfg: V4L2 subdev pad configuration
+ * @fmt: V4L2 subdev format
+ *
+ * This function implements the subdev get_fmt pad operation. It can be used as
+ * a direct drop-in for the operation handler.
+ */
+int vsp1_subdev_get_pad_format(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *fmt)
+{
+ struct vsp1_entity *entity = to_vsp1_entity(subdev);
+ struct v4l2_subdev_pad_config *config;
+
+ config = vsp1_entity_get_pad_config(entity, cfg, fmt->which);
+ if (!config)
+ return -EINVAL;
+
+ fmt->format = *vsp1_entity_get_pad_format(entity, config, fmt->pad);
+
+ return 0;
+}
+
+/*
+ * vsp1_subdev_enum_mbus_code - Subdev pad enum_mbus_code handler
+ * @subdev: V4L2 subdevice
+ * @cfg: V4L2 subdev pad configuration
+ * @code: Media bus code enumeration
+ * @codes: Array of supported media bus codes
+ * @ncodes: Number of supported media bus codes
+ *
+ * This function implements the subdev enum_mbus_code pad operation for entities
+ * that do not support format conversion. It enumerates the given supported
+ * media bus codes on the sink pad and reports a source pad format identical to
+ * the sink pad.
+ */
+int vsp1_subdev_enum_mbus_code(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_mbus_code_enum *code,
+ const unsigned int *codes, unsigned int ncodes)
+{
+ struct vsp1_entity *entity = to_vsp1_entity(subdev);
+
+ if (code->pad == 0) {
+ if (code->index >= ncodes)
+ return -EINVAL;
+
+ code->code = codes[code->index];
+ } else {
+ struct v4l2_subdev_pad_config *config;
+ struct v4l2_mbus_framefmt *format;
+
+ /* The entity can't perform format conversion, the sink format
+ * is always identical to the source format.
+ */
+ if (code->index)
+ return -EINVAL;
+
+ config = vsp1_entity_get_pad_config(entity, cfg, code->which);
+ if (!config)
+ return -EINVAL;
+
+ format = vsp1_entity_get_pad_format(entity, config, 0);
+ code->code = format->code;
+ }
+
+ return 0;
+}
+
+/*
+ * vsp1_subdev_enum_frame_size - Subdev pad enum_frame_size handler
+ * @subdev: V4L2 subdevice
+ * @cfg: V4L2 subdev pad configuration
+ * @fse: Frame size enumeration
+ * @min_width: Minimum image width
+ * @min_height: Minimum image height
+ * @max_width: Maximum image width
+ * @max_height: Maximum image height
+ *
+ * This function implements the subdev enum_frame_size pad operation for
+ * entities that do not support scaling or cropping. It reports the given
+ * minimum and maximum frame width and height on the sink pad, and a fixed
+ * source pad size identical to the sink pad.
+ */
+int vsp1_subdev_enum_frame_size(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_frame_size_enum *fse,
+ unsigned int min_width, unsigned int min_height,
+ unsigned int max_width, unsigned int max_height)
+{
+ struct vsp1_entity *entity = to_vsp1_entity(subdev);
+ struct v4l2_subdev_pad_config *config;
+ struct v4l2_mbus_framefmt *format;
+
+ config = vsp1_entity_get_pad_config(entity, cfg, fse->which);
+ if (!config)
+ return -EINVAL;
+
+ format = vsp1_entity_get_pad_format(entity, config, fse->pad);
+
+ if (fse->index || fse->code != format->code)
+ return -EINVAL;
+
+ if (fse->pad == 0) {
+ fse->min_width = min_width;
+ fse->max_width = max_width;
+ fse->min_height = min_height;
+ fse->max_height = max_height;
+ } else {
+ /* The size on the source pad are fixed and always identical to
+ * the size on the sink pad.
+ */
+ fse->min_width = format->width;
+ fse->max_width = format->width;
+ fse->min_height = format->height;
+ fse->max_height = format->height;
+ }
+
+ return 0;
+}
/* -----------------------------------------------------------------------------
* Media Operations
@@ -171,11 +283,11 @@
{ VSP1_ENTITY_HST, 0, VI6_DPR_HST_ROUTE, { VI6_DPR_NODE_HST, } },
{ VSP1_ENTITY_LIF, 0, 0, { VI6_DPR_NODE_LIF, } },
{ VSP1_ENTITY_LUT, 0, VI6_DPR_LUT_ROUTE, { VI6_DPR_NODE_LUT, } },
- { VSP1_ENTITY_RPF, 0, VI6_DPR_RPF_ROUTE(0), { VI6_DPR_NODE_RPF(0), } },
- { VSP1_ENTITY_RPF, 1, VI6_DPR_RPF_ROUTE(1), { VI6_DPR_NODE_RPF(1), } },
- { VSP1_ENTITY_RPF, 2, VI6_DPR_RPF_ROUTE(2), { VI6_DPR_NODE_RPF(2), } },
- { VSP1_ENTITY_RPF, 3, VI6_DPR_RPF_ROUTE(3), { VI6_DPR_NODE_RPF(3), } },
- { VSP1_ENTITY_RPF, 4, VI6_DPR_RPF_ROUTE(4), { VI6_DPR_NODE_RPF(4), } },
+ { VSP1_ENTITY_RPF, 0, VI6_DPR_RPF_ROUTE(0), { 0, } },
+ { VSP1_ENTITY_RPF, 1, VI6_DPR_RPF_ROUTE(1), { 0, } },
+ { VSP1_ENTITY_RPF, 2, VI6_DPR_RPF_ROUTE(2), { 0, } },
+ { VSP1_ENTITY_RPF, 3, VI6_DPR_RPF_ROUTE(3), { 0, } },
+ { VSP1_ENTITY_RPF, 4, VI6_DPR_RPF_ROUTE(4), { 0, } },
{ VSP1_ENTITY_SRU, 0, VI6_DPR_SRU_ROUTE, { VI6_DPR_NODE_SRU, } },
{ VSP1_ENTITY_UDS, 0, VI6_DPR_UDS_ROUTE(0), { VI6_DPR_NODE_UDS(0), } },
{ VSP1_ENTITY_UDS, 1, VI6_DPR_UDS_ROUTE(1), { VI6_DPR_NODE_UDS(1), } },
@@ -187,9 +299,12 @@
};
int vsp1_entity_init(struct vsp1_device *vsp1, struct vsp1_entity *entity,
- unsigned int num_pads)
+ const char *name, unsigned int num_pads,
+ const struct v4l2_subdev_ops *ops)
{
+ struct v4l2_subdev *subdev;
unsigned int i;
+ int ret;
for (i = 0; i < ARRAY_SIZE(vsp1_routes); ++i) {
if (vsp1_routes[i].type == entity->type &&
@@ -202,37 +317,56 @@
if (i == ARRAY_SIZE(vsp1_routes))
return -EINVAL;
- spin_lock_init(&entity->lock);
-
entity->vsp1 = vsp1;
entity->source_pad = num_pads - 1;
- /* Allocate formats and pads. */
- entity->formats = devm_kzalloc(vsp1->dev,
- num_pads * sizeof(*entity->formats),
- GFP_KERNEL);
- if (entity->formats == NULL)
- return -ENOMEM;
-
+ /* Allocate and initialize pads. */
entity->pads = devm_kzalloc(vsp1->dev, num_pads * sizeof(*entity->pads),
GFP_KERNEL);
if (entity->pads == NULL)
return -ENOMEM;
- /* Initialize pads. */
for (i = 0; i < num_pads - 1; ++i)
entity->pads[i].flags = MEDIA_PAD_FL_SINK;
entity->pads[num_pads - 1].flags = MEDIA_PAD_FL_SOURCE;
/* Initialize the media entity. */
- return media_entity_pads_init(&entity->subdev.entity, num_pads,
- entity->pads);
+ ret = media_entity_pads_init(&entity->subdev.entity, num_pads,
+ entity->pads);
+ if (ret < 0)
+ return ret;
+
+ /* Initialize the V4L2 subdev. */
+ subdev = &entity->subdev;
+ v4l2_subdev_init(subdev, ops);
+
+ subdev->entity.ops = &vsp1->media_ops;
+ subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+
+ snprintf(subdev->name, sizeof(subdev->name), "%s %s",
+ dev_name(vsp1->dev), name);
+
+ vsp1_entity_init_cfg(subdev, NULL);
+
+ /* Allocate the pad configuration to store formats and selection
+ * rectangles.
+ */
+ entity->config = v4l2_subdev_alloc_pad_config(&entity->subdev);
+ if (entity->config == NULL) {
+ media_entity_cleanup(&entity->subdev.entity);
+ return -ENOMEM;
+ }
+
+ return 0;
}
void vsp1_entity_destroy(struct vsp1_entity *entity)
{
+ if (entity->ops && entity->ops->destroy)
+ entity->ops->destroy(entity);
if (entity->subdev.ctrl_handler)
v4l2_ctrl_handler_free(entity->subdev.ctrl_handler);
+ v4l2_subdev_free_pad_config(entity->config);
media_entity_cleanup(&entity->subdev.entity);
}
diff --git a/drivers/media/platform/vsp1/vsp1_entity.h b/drivers/media/platform/vsp1/vsp1_entity.h
index 83570df..69eff4e 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.h
+++ b/drivers/media/platform/vsp1/vsp1_entity.h
@@ -19,6 +19,8 @@
#include <media/v4l2-subdev.h>
struct vsp1_device;
+struct vsp1_dl_list;
+struct vsp1_pipeline;
enum vsp1_entity_type {
VSP1_ENTITY_BRU,
@@ -53,9 +55,27 @@
unsigned int inputs[VSP1_ENTITY_MAX_INPUTS];
};
+/**
+ * struct vsp1_entity_operations - Entity operations
+ * @destroy: Destroy the entity.
+ * @set_memory: Setup memory buffer access. This operation applies the settings
+ * stored in the rwpf mem field to the display list. Valid for RPF
+ * and WPF only.
+ * @configure: Setup the hardware based on the entity state (pipeline, formats,
+ * selection rectangles, ...)
+ */
+struct vsp1_entity_operations {
+ void (*destroy)(struct vsp1_entity *);
+ void (*set_memory)(struct vsp1_entity *, struct vsp1_dl_list *dl);
+ void (*configure)(struct vsp1_entity *, struct vsp1_pipeline *,
+ struct vsp1_dl_list *);
+};
+
struct vsp1_entity {
struct vsp1_device *vsp1;
+ const struct vsp1_entity_operations *ops;
+
enum vsp1_entity_type type;
unsigned int index;
const struct vsp1_route *route;
@@ -70,10 +90,7 @@
unsigned int sink_pad;
struct v4l2_subdev subdev;
- struct v4l2_mbus_framefmt *formats;
-
- spinlock_t lock; /* Protects the streaming field */
- bool streaming;
+ struct v4l2_subdev_pad_config *config;
};
static inline struct vsp1_entity *to_vsp1_entity(struct v4l2_subdev *subdev)
@@ -82,7 +99,8 @@
}
int vsp1_entity_init(struct vsp1_device *vsp1, struct vsp1_entity *entity,
- unsigned int num_pads);
+ const char *name, unsigned int num_pads,
+ const struct v4l2_subdev_ops *ops);
void vsp1_entity_destroy(struct vsp1_entity *entity);
extern const struct v4l2_subdev_internal_ops vsp1_subdev_internal_ops;
@@ -91,16 +109,35 @@
const struct media_pad *local,
const struct media_pad *remote, u32 flags);
+struct v4l2_subdev_pad_config *
+vsp1_entity_get_pad_config(struct vsp1_entity *entity,
+ struct v4l2_subdev_pad_config *cfg,
+ enum v4l2_subdev_format_whence which);
struct v4l2_mbus_framefmt *
vsp1_entity_get_pad_format(struct vsp1_entity *entity,
struct v4l2_subdev_pad_config *cfg,
- unsigned int pad, u32 which);
-void vsp1_entity_init_formats(struct v4l2_subdev *subdev,
- struct v4l2_subdev_pad_config *cfg);
+ unsigned int pad);
+struct v4l2_rect *
+vsp1_entity_get_pad_compose(struct vsp1_entity *entity,
+ struct v4l2_subdev_pad_config *cfg,
+ unsigned int pad);
+int vsp1_entity_init_cfg(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_pad_config *cfg);
-bool vsp1_entity_is_streaming(struct vsp1_entity *entity);
-int vsp1_entity_set_streaming(struct vsp1_entity *entity, bool streaming);
+void vsp1_entity_route_setup(struct vsp1_entity *source,
+ struct vsp1_dl_list *dl);
-void vsp1_entity_route_setup(struct vsp1_entity *source);
+int vsp1_subdev_get_pad_format(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *fmt);
+int vsp1_subdev_enum_mbus_code(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_mbus_code_enum *code,
+ const unsigned int *codes, unsigned int ncodes);
+int vsp1_subdev_enum_frame_size(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_frame_size_enum *fse,
+ unsigned int min_w, unsigned int min_h,
+ unsigned int max_w, unsigned int max_h);
#endif /* __VSP1_ENTITY_H__ */
diff --git a/drivers/media/platform/vsp1/vsp1_hsit.c b/drivers/media/platform/vsp1/vsp1_hsit.c
index c1087cf..68b8567 100644
--- a/drivers/media/platform/vsp1/vsp1_hsit.c
+++ b/drivers/media/platform/vsp1/vsp1_hsit.c
@@ -17,6 +17,7 @@
#include <media/v4l2-subdev.h>
#include "vsp1.h"
+#include "vsp1_dl.h"
#include "vsp1_hsit.h"
#define HSIT_MIN_SIZE 4U
@@ -26,32 +27,14 @@
* Device Access
*/
-static inline void vsp1_hsit_write(struct vsp1_hsit *hsit, u32 reg, u32 data)
+static inline void vsp1_hsit_write(struct vsp1_hsit *hsit,
+ struct vsp1_dl_list *dl, u32 reg, u32 data)
{
- vsp1_write(hsit->entity.vsp1, reg, data);
+ vsp1_dl_list_write(dl, reg, data);
}
/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Core Operations
- */
-
-static int hsit_s_stream(struct v4l2_subdev *subdev, int enable)
-{
- struct vsp1_hsit *hsit = to_hsit(subdev);
-
- if (!enable)
- return 0;
-
- if (hsit->inverse)
- vsp1_hsit_write(hsit, VI6_HSI_CTRL, VI6_HSI_CTRL_EN);
- else
- vsp1_hsit_write(hsit, VI6_HST_CTRL, VI6_HST_CTRL_EN);
-
- return 0;
-}
-
-/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Pad Operations
+ * V4L2 Subdevice Operations
*/
static int hsit_enum_mbus_code(struct v4l2_subdev *subdev,
@@ -76,43 +59,9 @@
struct v4l2_subdev_pad_config *cfg,
struct v4l2_subdev_frame_size_enum *fse)
{
- struct vsp1_hsit *hsit = to_hsit(subdev);
- struct v4l2_mbus_framefmt *format;
-
- format = vsp1_entity_get_pad_format(&hsit->entity, cfg, fse->pad,
- fse->which);
-
- if (fse->index || fse->code != format->code)
- return -EINVAL;
-
- if (fse->pad == HSIT_PAD_SINK) {
- fse->min_width = HSIT_MIN_SIZE;
- fse->max_width = HSIT_MAX_SIZE;
- fse->min_height = HSIT_MIN_SIZE;
- fse->max_height = HSIT_MAX_SIZE;
- } else {
- /* The size on the source pad are fixed and always identical to
- * the size on the sink pad.
- */
- fse->min_width = format->width;
- fse->max_width = format->width;
- fse->min_height = format->height;
- fse->max_height = format->height;
- }
-
- return 0;
-}
-
-static int hsit_get_format(struct v4l2_subdev *subdev,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_format *fmt)
-{
- struct vsp1_hsit *hsit = to_hsit(subdev);
-
- fmt->format = *vsp1_entity_get_pad_format(&hsit->entity, cfg, fmt->pad,
- fmt->which);
-
- return 0;
+ return vsp1_subdev_enum_frame_size(subdev, cfg, fse, HSIT_MIN_SIZE,
+ HSIT_MIN_SIZE, HSIT_MAX_SIZE,
+ HSIT_MAX_SIZE);
}
static int hsit_set_format(struct v4l2_subdev *subdev,
@@ -120,10 +69,14 @@
struct v4l2_subdev_format *fmt)
{
struct vsp1_hsit *hsit = to_hsit(subdev);
+ struct v4l2_subdev_pad_config *config;
struct v4l2_mbus_framefmt *format;
- format = vsp1_entity_get_pad_format(&hsit->entity, cfg, fmt->pad,
- fmt->which);
+ config = vsp1_entity_get_pad_config(&hsit->entity, cfg, fmt->which);
+ if (!config)
+ return -EINVAL;
+
+ format = vsp1_entity_get_pad_format(&hsit->entity, config, fmt->pad);
if (fmt->pad == HSIT_PAD_SOURCE) {
/* The HST and HSI output format code and resolution can't be
@@ -145,8 +98,8 @@
fmt->format = *format;
/* Propagate the format to the source pad. */
- format = vsp1_entity_get_pad_format(&hsit->entity, cfg, HSIT_PAD_SOURCE,
- fmt->which);
+ format = vsp1_entity_get_pad_format(&hsit->entity, config,
+ HSIT_PAD_SOURCE);
*format = fmt->format;
format->code = hsit->inverse ? MEDIA_BUS_FMT_ARGB8888_1X32
: MEDIA_BUS_FMT_AHSV8888_1X32;
@@ -154,33 +107,44 @@
return 0;
}
-/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Operations
- */
-
-static struct v4l2_subdev_video_ops hsit_video_ops = {
- .s_stream = hsit_s_stream,
-};
-
static struct v4l2_subdev_pad_ops hsit_pad_ops = {
+ .init_cfg = vsp1_entity_init_cfg,
.enum_mbus_code = hsit_enum_mbus_code,
.enum_frame_size = hsit_enum_frame_size,
- .get_fmt = hsit_get_format,
+ .get_fmt = vsp1_subdev_get_pad_format,
.set_fmt = hsit_set_format,
};
static struct v4l2_subdev_ops hsit_ops = {
- .video = &hsit_video_ops,
.pad = &hsit_pad_ops,
};
/* -----------------------------------------------------------------------------
+ * VSP1 Entity Operations
+ */
+
+static void hsit_configure(struct vsp1_entity *entity,
+ struct vsp1_pipeline *pipe,
+ struct vsp1_dl_list *dl)
+{
+ struct vsp1_hsit *hsit = to_hsit(&entity->subdev);
+
+ if (hsit->inverse)
+ vsp1_hsit_write(hsit, dl, VI6_HSI_CTRL, VI6_HSI_CTRL_EN);
+ else
+ vsp1_hsit_write(hsit, dl, VI6_HST_CTRL, VI6_HST_CTRL_EN);
+}
+
+static const struct vsp1_entity_operations hsit_entity_ops = {
+ .configure = hsit_configure,
+};
+
+/* -----------------------------------------------------------------------------
* Initialization and Cleanup
*/
struct vsp1_hsit *vsp1_hsit_create(struct vsp1_device *vsp1, bool inverse)
{
- struct v4l2_subdev *subdev;
struct vsp1_hsit *hsit;
int ret;
@@ -190,27 +154,17 @@
hsit->inverse = inverse;
+ hsit->entity.ops = &hsit_entity_ops;
+
if (inverse)
hsit->entity.type = VSP1_ENTITY_HSI;
else
hsit->entity.type = VSP1_ENTITY_HST;
- ret = vsp1_entity_init(vsp1, &hsit->entity, 2);
+ ret = vsp1_entity_init(vsp1, &hsit->entity, inverse ? "hsi" : "hst", 2,
+ &hsit_ops);
if (ret < 0)
return ERR_PTR(ret);
- /* Initialize the V4L2 subdev. */
- subdev = &hsit->entity.subdev;
- v4l2_subdev_init(subdev, &hsit_ops);
-
- subdev->entity.ops = &vsp1->media_ops;
- subdev->internal_ops = &vsp1_subdev_internal_ops;
- snprintf(subdev->name, sizeof(subdev->name), "%s %s",
- dev_name(vsp1->dev), inverse ? "hsi" : "hst");
- v4l2_set_subdevdata(subdev, hsit);
- subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
-
- vsp1_entity_init_formats(subdev, NULL);
-
return hsit;
}
diff --git a/drivers/media/platform/vsp1/vsp1_lif.c b/drivers/media/platform/vsp1/vsp1_lif.c
index 433853c..0217393 100644
--- a/drivers/media/platform/vsp1/vsp1_lif.c
+++ b/drivers/media/platform/vsp1/vsp1_lif.c
@@ -17,55 +17,24 @@
#include <media/v4l2-subdev.h>
#include "vsp1.h"
+#include "vsp1_dl.h"
#include "vsp1_lif.h"
#define LIF_MIN_SIZE 2U
-#define LIF_MAX_SIZE 2048U
+#define LIF_MAX_SIZE 8190U
/* -----------------------------------------------------------------------------
* Device Access
*/
-static inline void vsp1_lif_write(struct vsp1_lif *lif, u32 reg, u32 data)
+static inline void vsp1_lif_write(struct vsp1_lif *lif, struct vsp1_dl_list *dl,
+ u32 reg, u32 data)
{
- vsp1_mod_write(&lif->entity, reg, data);
+ vsp1_dl_list_write(dl, reg, data);
}
/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Core Operations
- */
-
-static int lif_s_stream(struct v4l2_subdev *subdev, int enable)
-{
- const struct v4l2_mbus_framefmt *format;
- struct vsp1_lif *lif = to_lif(subdev);
- unsigned int hbth = 1300;
- unsigned int obth = 400;
- unsigned int lbth = 200;
-
- if (!enable) {
- vsp1_write(lif->entity.vsp1, VI6_LIF_CTRL, 0);
- return 0;
- }
-
- format = &lif->entity.formats[LIF_PAD_SOURCE];
-
- obth = min(obth, (format->width + 1) / 2 * format->height - 4);
-
- vsp1_lif_write(lif, VI6_LIF_CSBTH,
- (hbth << VI6_LIF_CSBTH_HBTH_SHIFT) |
- (lbth << VI6_LIF_CSBTH_LBTH_SHIFT));
-
- vsp1_lif_write(lif, VI6_LIF_CTRL,
- (obth << VI6_LIF_CTRL_OBTH_SHIFT) |
- (format->code == 0 ? VI6_LIF_CTRL_CFMT : 0) |
- VI6_LIF_CTRL_REQSEL | VI6_LIF_CTRL_LIF_EN);
-
- return 0;
-}
-
-/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Pad Operations
+ * V4L2 Subdevice Operations
*/
static int lif_enum_mbus_code(struct v4l2_subdev *subdev,
@@ -76,82 +45,38 @@
MEDIA_BUS_FMT_ARGB8888_1X32,
MEDIA_BUS_FMT_AYUV8_1X32,
};
- struct vsp1_lif *lif = to_lif(subdev);
- if (code->pad == LIF_PAD_SINK) {
- if (code->index >= ARRAY_SIZE(codes))
- return -EINVAL;
-
- code->code = codes[code->index];
- } else {
- struct v4l2_mbus_framefmt *format;
-
- /* The LIF can't perform format conversion, the sink format is
- * always identical to the source format.
- */
- if (code->index)
- return -EINVAL;
-
- format = vsp1_entity_get_pad_format(&lif->entity, cfg,
- LIF_PAD_SINK, code->which);
- code->code = format->code;
- }
-
- return 0;
+ return vsp1_subdev_enum_mbus_code(subdev, cfg, code, codes,
+ ARRAY_SIZE(codes));
}
static int lif_enum_frame_size(struct v4l2_subdev *subdev,
struct v4l2_subdev_pad_config *cfg,
struct v4l2_subdev_frame_size_enum *fse)
{
+ return vsp1_subdev_enum_frame_size(subdev, cfg, fse, LIF_MIN_SIZE,
+ LIF_MIN_SIZE, LIF_MAX_SIZE,
+ LIF_MAX_SIZE);
+}
+
+static int lif_set_format(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *fmt)
+{
struct vsp1_lif *lif = to_lif(subdev);
+ struct v4l2_subdev_pad_config *config;
struct v4l2_mbus_framefmt *format;
- format = vsp1_entity_get_pad_format(&lif->entity, cfg, LIF_PAD_SINK,
- fse->which);
-
- if (fse->index || fse->code != format->code)
+ config = vsp1_entity_get_pad_config(&lif->entity, cfg, fmt->which);
+ if (!config)
return -EINVAL;
- if (fse->pad == LIF_PAD_SINK) {
- fse->min_width = LIF_MIN_SIZE;
- fse->max_width = LIF_MAX_SIZE;
- fse->min_height = LIF_MIN_SIZE;
- fse->max_height = LIF_MAX_SIZE;
- } else {
- fse->min_width = format->width;
- fse->max_width = format->width;
- fse->min_height = format->height;
- fse->max_height = format->height;
- }
-
- return 0;
-}
-
-static int lif_get_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_format *fmt)
-{
- struct vsp1_lif *lif = to_lif(subdev);
-
- fmt->format = *vsp1_entity_get_pad_format(&lif->entity, cfg, fmt->pad,
- fmt->which);
-
- return 0;
-}
-
-static int lif_set_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_format *fmt)
-{
- struct vsp1_lif *lif = to_lif(subdev);
- struct v4l2_mbus_framefmt *format;
-
/* Default to YUV if the requested format is not supported. */
if (fmt->format.code != MEDIA_BUS_FMT_ARGB8888_1X32 &&
fmt->format.code != MEDIA_BUS_FMT_AYUV8_1X32)
fmt->format.code = MEDIA_BUS_FMT_AYUV8_1X32;
- format = vsp1_entity_get_pad_format(&lif->entity, cfg, fmt->pad,
- fmt->which);
+ format = vsp1_entity_get_pad_format(&lif->entity, config, fmt->pad);
if (fmt->pad == LIF_PAD_SOURCE) {
/* The LIF source format is always identical to its sink
@@ -172,40 +97,64 @@
fmt->format = *format;
/* Propagate the format to the source pad. */
- format = vsp1_entity_get_pad_format(&lif->entity, cfg, LIF_PAD_SOURCE,
- fmt->which);
+ format = vsp1_entity_get_pad_format(&lif->entity, config,
+ LIF_PAD_SOURCE);
*format = fmt->format;
return 0;
}
-/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Operations
- */
-
-static struct v4l2_subdev_video_ops lif_video_ops = {
- .s_stream = lif_s_stream,
-};
-
static struct v4l2_subdev_pad_ops lif_pad_ops = {
+ .init_cfg = vsp1_entity_init_cfg,
.enum_mbus_code = lif_enum_mbus_code,
.enum_frame_size = lif_enum_frame_size,
- .get_fmt = lif_get_format,
+ .get_fmt = vsp1_subdev_get_pad_format,
.set_fmt = lif_set_format,
};
static struct v4l2_subdev_ops lif_ops = {
- .video = &lif_video_ops,
.pad = &lif_pad_ops,
};
/* -----------------------------------------------------------------------------
+ * VSP1 Entity Operations
+ */
+
+static void lif_configure(struct vsp1_entity *entity,
+ struct vsp1_pipeline *pipe,
+ struct vsp1_dl_list *dl)
+{
+ const struct v4l2_mbus_framefmt *format;
+ struct vsp1_lif *lif = to_lif(&entity->subdev);
+ unsigned int hbth = 1300;
+ unsigned int obth = 400;
+ unsigned int lbth = 200;
+
+ format = vsp1_entity_get_pad_format(&lif->entity, lif->entity.config,
+ LIF_PAD_SOURCE);
+
+ obth = min(obth, (format->width + 1) / 2 * format->height - 4);
+
+ vsp1_lif_write(lif, dl, VI6_LIF_CSBTH,
+ (hbth << VI6_LIF_CSBTH_HBTH_SHIFT) |
+ (lbth << VI6_LIF_CSBTH_LBTH_SHIFT));
+
+ vsp1_lif_write(lif, dl, VI6_LIF_CTRL,
+ (obth << VI6_LIF_CTRL_OBTH_SHIFT) |
+ (format->code == 0 ? VI6_LIF_CTRL_CFMT : 0) |
+ VI6_LIF_CTRL_REQSEL | VI6_LIF_CTRL_LIF_EN);
+}
+
+static const struct vsp1_entity_operations lif_entity_ops = {
+ .configure = lif_configure,
+};
+
+/* -----------------------------------------------------------------------------
* Initialization and Cleanup
*/
struct vsp1_lif *vsp1_lif_create(struct vsp1_device *vsp1)
{
- struct v4l2_subdev *subdev;
struct vsp1_lif *lif;
int ret;
@@ -213,24 +162,12 @@
if (lif == NULL)
return ERR_PTR(-ENOMEM);
+ lif->entity.ops = &lif_entity_ops;
lif->entity.type = VSP1_ENTITY_LIF;
- ret = vsp1_entity_init(vsp1, &lif->entity, 2);
+ ret = vsp1_entity_init(vsp1, &lif->entity, "lif", 2, &lif_ops);
if (ret < 0)
return ERR_PTR(ret);
- /* Initialize the V4L2 subdev. */
- subdev = &lif->entity.subdev;
- v4l2_subdev_init(subdev, &lif_ops);
-
- subdev->entity.ops = &vsp1->media_ops;
- subdev->internal_ops = &vsp1_subdev_internal_ops;
- snprintf(subdev->name, sizeof(subdev->name), "%s lif",
- dev_name(vsp1->dev));
- v4l2_set_subdevdata(subdev, lif);
- subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
-
- vsp1_entity_init_formats(subdev, NULL);
-
return lif;
}
diff --git a/drivers/media/platform/vsp1/vsp1_lut.c b/drivers/media/platform/vsp1/vsp1_lut.c
index 4b89095..aa09e59 100644
--- a/drivers/media/platform/vsp1/vsp1_lut.c
+++ b/drivers/media/platform/vsp1/vsp1_lut.c
@@ -18,6 +18,7 @@
#include <media/v4l2-subdev.h>
#include "vsp1.h"
+#include "vsp1_dl.h"
#include "vsp1_lut.h"
#define LUT_MIN_SIZE 4U
@@ -27,19 +28,35 @@
* Device Access
*/
-static inline void vsp1_lut_write(struct vsp1_lut *lut, u32 reg, u32 data)
+static inline void vsp1_lut_write(struct vsp1_lut *lut, struct vsp1_dl_list *dl,
+ u32 reg, u32 data)
{
- vsp1_write(lut->entity.vsp1, reg, data);
+ vsp1_dl_list_write(dl, reg, data);
}
/* -----------------------------------------------------------------------------
* V4L2 Subdevice Core Operations
*/
-static void lut_configure(struct vsp1_lut *lut, struct vsp1_lut_config *config)
+static int lut_set_table(struct vsp1_lut *lut, struct vsp1_lut_config *config)
{
- memcpy_toio(lut->entity.vsp1->mmio + VI6_LUT_TABLE, config->lut,
- sizeof(config->lut));
+ struct vsp1_dl_body *dlb;
+ unsigned int i;
+
+ dlb = vsp1_dl_fragment_alloc(lut->entity.vsp1, ARRAY_SIZE(config->lut));
+ if (!dlb)
+ return -ENOMEM;
+
+ for (i = 0; i < ARRAY_SIZE(config->lut); ++i)
+ vsp1_dl_fragment_write(dlb, VI6_LUT_TABLE + 4 * i,
+ config->lut[i]);
+
+ mutex_lock(&lut->lock);
+ swap(lut->lut, dlb);
+ mutex_unlock(&lut->lock);
+
+ vsp1_dl_fragment_free(dlb);
+ return 0;
}
static long lut_ioctl(struct v4l2_subdev *subdev, unsigned int cmd, void *arg)
@@ -48,8 +65,7 @@
switch (cmd) {
case VIDIOC_VSP1_LUT_CONFIG:
- lut_configure(lut, arg);
- return 0;
+ return lut_set_table(lut, arg);
default:
return -ENOIOCTLCMD;
@@ -57,22 +73,6 @@
}
/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Video Operations
- */
-
-static int lut_s_stream(struct v4l2_subdev *subdev, int enable)
-{
- struct vsp1_lut *lut = to_lut(subdev);
-
- if (!enable)
- return 0;
-
- vsp1_lut_write(lut, VI6_LUT_CTRL, VI6_LUT_CTRL_EN);
-
- return 0;
-}
-
-/* -----------------------------------------------------------------------------
* V4L2 Subdevice Pad Operations
*/
@@ -85,85 +85,39 @@
MEDIA_BUS_FMT_AHSV8888_1X32,
MEDIA_BUS_FMT_AYUV8_1X32,
};
- struct vsp1_lut *lut = to_lut(subdev);
- struct v4l2_mbus_framefmt *format;
- if (code->pad == LUT_PAD_SINK) {
- if (code->index >= ARRAY_SIZE(codes))
- return -EINVAL;
-
- code->code = codes[code->index];
- } else {
- /* The LUT can't perform format conversion, the sink format is
- * always identical to the source format.
- */
- if (code->index)
- return -EINVAL;
-
- format = vsp1_entity_get_pad_format(&lut->entity, cfg,
- LUT_PAD_SINK, code->which);
- code->code = format->code;
- }
-
- return 0;
+ return vsp1_subdev_enum_mbus_code(subdev, cfg, code, codes,
+ ARRAY_SIZE(codes));
}
static int lut_enum_frame_size(struct v4l2_subdev *subdev,
struct v4l2_subdev_pad_config *cfg,
struct v4l2_subdev_frame_size_enum *fse)
{
+ return vsp1_subdev_enum_frame_size(subdev, cfg, fse, LUT_MIN_SIZE,
+ LUT_MIN_SIZE, LUT_MAX_SIZE,
+ LUT_MAX_SIZE);
+}
+
+static int lut_set_format(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *fmt)
+{
struct vsp1_lut *lut = to_lut(subdev);
+ struct v4l2_subdev_pad_config *config;
struct v4l2_mbus_framefmt *format;
- format = vsp1_entity_get_pad_format(&lut->entity, cfg,
- fse->pad, fse->which);
-
- if (fse->index || fse->code != format->code)
+ config = vsp1_entity_get_pad_config(&lut->entity, cfg, fmt->which);
+ if (!config)
return -EINVAL;
- if (fse->pad == LUT_PAD_SINK) {
- fse->min_width = LUT_MIN_SIZE;
- fse->max_width = LUT_MAX_SIZE;
- fse->min_height = LUT_MIN_SIZE;
- fse->max_height = LUT_MAX_SIZE;
- } else {
- /* The size on the source pad are fixed and always identical to
- * the size on the sink pad.
- */
- fse->min_width = format->width;
- fse->max_width = format->width;
- fse->min_height = format->height;
- fse->max_height = format->height;
- }
-
- return 0;
-}
-
-static int lut_get_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_format *fmt)
-{
- struct vsp1_lut *lut = to_lut(subdev);
-
- fmt->format = *vsp1_entity_get_pad_format(&lut->entity, cfg, fmt->pad,
- fmt->which);
-
- return 0;
-}
-
-static int lut_set_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_format *fmt)
-{
- struct vsp1_lut *lut = to_lut(subdev);
- struct v4l2_mbus_framefmt *format;
-
/* Default to YUV if the requested format is not supported. */
if (fmt->format.code != MEDIA_BUS_FMT_ARGB8888_1X32 &&
fmt->format.code != MEDIA_BUS_FMT_AHSV8888_1X32 &&
fmt->format.code != MEDIA_BUS_FMT_AYUV8_1X32)
fmt->format.code = MEDIA_BUS_FMT_AYUV8_1X32;
- format = vsp1_entity_get_pad_format(&lut->entity, cfg, fmt->pad,
- fmt->which);
+ format = vsp1_entity_get_pad_format(&lut->entity, config, fmt->pad);
if (fmt->pad == LUT_PAD_SOURCE) {
/* The LUT output format can't be modified. */
@@ -171,6 +125,7 @@
return 0;
}
+ format->code = fmt->format.code;
format->width = clamp_t(unsigned int, fmt->format.width,
LUT_MIN_SIZE, LUT_MAX_SIZE);
format->height = clamp_t(unsigned int, fmt->format.height,
@@ -181,8 +136,8 @@
fmt->format = *format;
/* Propagate the format to the source pad. */
- format = vsp1_entity_get_pad_format(&lut->entity, cfg, LUT_PAD_SOURCE,
- fmt->which);
+ format = vsp1_entity_get_pad_format(&lut->entity, config,
+ LUT_PAD_SOURCE);
*format = fmt->format;
return 0;
@@ -196,30 +151,49 @@
.ioctl = lut_ioctl,
};
-static struct v4l2_subdev_video_ops lut_video_ops = {
- .s_stream = lut_s_stream,
-};
-
static struct v4l2_subdev_pad_ops lut_pad_ops = {
+ .init_cfg = vsp1_entity_init_cfg,
.enum_mbus_code = lut_enum_mbus_code,
.enum_frame_size = lut_enum_frame_size,
- .get_fmt = lut_get_format,
+ .get_fmt = vsp1_subdev_get_pad_format,
.set_fmt = lut_set_format,
};
static struct v4l2_subdev_ops lut_ops = {
.core = &lut_core_ops,
- .video = &lut_video_ops,
.pad = &lut_pad_ops,
};
/* -----------------------------------------------------------------------------
+ * VSP1 Entity Operations
+ */
+
+static void lut_configure(struct vsp1_entity *entity,
+ struct vsp1_pipeline *pipe,
+ struct vsp1_dl_list *dl)
+{
+ struct vsp1_lut *lut = to_lut(&entity->subdev);
+
+ vsp1_lut_write(lut, dl, VI6_LUT_CTRL, VI6_LUT_CTRL_EN);
+
+ mutex_lock(&lut->lock);
+ if (lut->lut) {
+ vsp1_dl_list_add_fragment(dl, lut->lut);
+ lut->lut = NULL;
+ }
+ mutex_unlock(&lut->lock);
+}
+
+static const struct vsp1_entity_operations lut_entity_ops = {
+ .configure = lut_configure,
+};
+
+/* -----------------------------------------------------------------------------
* Initialization and Cleanup
*/
struct vsp1_lut *vsp1_lut_create(struct vsp1_device *vsp1)
{
- struct v4l2_subdev *subdev;
struct vsp1_lut *lut;
int ret;
@@ -227,24 +201,12 @@
if (lut == NULL)
return ERR_PTR(-ENOMEM);
+ lut->entity.ops = &lut_entity_ops;
lut->entity.type = VSP1_ENTITY_LUT;
- ret = vsp1_entity_init(vsp1, &lut->entity, 2);
+ ret = vsp1_entity_init(vsp1, &lut->entity, "lut", 2, &lut_ops);
if (ret < 0)
return ERR_PTR(ret);
- /* Initialize the V4L2 subdev. */
- subdev = &lut->entity.subdev;
- v4l2_subdev_init(subdev, &lut_ops);
-
- subdev->entity.ops = &vsp1->media_ops;
- subdev->internal_ops = &vsp1_subdev_internal_ops;
- snprintf(subdev->name, sizeof(subdev->name), "%s lut",
- dev_name(vsp1->dev));
- v4l2_set_subdevdata(subdev, lut);
- subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
-
- vsp1_entity_init_formats(subdev, NULL);
-
return lut;
}
diff --git a/drivers/media/platform/vsp1/vsp1_lut.h b/drivers/media/platform/vsp1/vsp1_lut.h
index f92ffb8..cef874f 100644
--- a/drivers/media/platform/vsp1/vsp1_lut.h
+++ b/drivers/media/platform/vsp1/vsp1_lut.h
@@ -13,6 +13,8 @@
#ifndef __VSP1_LUT_H__
#define __VSP1_LUT_H__
+#include <linux/mutex.h>
+
#include <media/media-entity.h>
#include <media/v4l2-subdev.h>
@@ -25,7 +27,9 @@
struct vsp1_lut {
struct vsp1_entity entity;
- u32 lut[256];
+
+ struct mutex lock;
+ struct vsp1_dl_body *lut;
};
static inline struct vsp1_lut *to_lut(struct v4l2_subdev *subdev)
diff --git a/drivers/media/platform/vsp1/vsp1_pipe.c b/drivers/media/platform/vsp1/vsp1_pipe.c
index 6659f06..4f3b4a1 100644
--- a/drivers/media/platform/vsp1/vsp1_pipe.c
+++ b/drivers/media/platform/vsp1/vsp1_pipe.c
@@ -43,7 +43,7 @@
{ V4L2_PIX_FMT_XRGB444, MEDIA_BUS_FMT_ARGB8888_1X32,
VI6_FMT_XRGB_4444, VI6_RPF_DSWAP_P_LLS | VI6_RPF_DSWAP_P_LWS |
VI6_RPF_DSWAP_P_WDS,
- 1, { 16, 0, 0 }, false, false, 1, 1, true },
+ 1, { 16, 0, 0 }, false, false, 1, 1, false },
{ V4L2_PIX_FMT_ARGB555, MEDIA_BUS_FMT_ARGB8888_1X32,
VI6_FMT_ARGB_1555, VI6_RPF_DSWAP_P_LLS | VI6_RPF_DSWAP_P_LWS |
VI6_RPF_DSWAP_P_WDS,
@@ -172,14 +172,18 @@
bru->inputs[i].rpf = NULL;
}
- for (i = 0; i < ARRAY_SIZE(pipe->inputs); ++i)
+ for (i = 0; i < pipe->num_inputs; ++i) {
+ pipe->inputs[i]->pipe = NULL;
pipe->inputs[i] = NULL;
+ }
+
+ pipe->output->pipe = NULL;
+ pipe->output = NULL;
INIT_LIST_HEAD(&pipe->entities);
pipe->state = VSP1_PIPELINE_STOPPED;
pipe->buffers_ready = 0;
pipe->num_inputs = 0;
- pipe->output = NULL;
pipe->bru = NULL;
pipe->lif = NULL;
pipe->uds = NULL;
@@ -190,11 +194,13 @@
mutex_init(&pipe->lock);
spin_lock_init(&pipe->irqlock);
init_waitqueue_head(&pipe->wq);
+ kref_init(&pipe->kref);
INIT_LIST_HEAD(&pipe->entities);
pipe->state = VSP1_PIPELINE_STOPPED;
}
+/* Must be called with the pipe irqlock held. */
void vsp1_pipeline_run(struct vsp1_pipeline *pipe)
{
struct vsp1_device *vsp1 = pipe->output->entity.vsp1;
@@ -226,7 +232,7 @@
unsigned long flags;
int ret;
- if (pipe->dl) {
+ if (pipe->lif) {
/* When using display lists in continuous frame mode the only
* way to stop the pipeline is to reset the hardware.
*/
@@ -253,10 +259,10 @@
if (entity->route && entity->route->reg)
vsp1_write(entity->vsp1, entity->route->reg,
VI6_DPR_NODE_UNUSED);
-
- v4l2_subdev_call(&entity->subdev, video, s_stream, 0);
}
+ v4l2_subdev_call(&pipe->output->entity.subdev, video, s_stream, 0);
+
return ret;
}
@@ -271,50 +277,15 @@
return pipe->buffers_ready == mask;
}
-void vsp1_pipeline_display_start(struct vsp1_pipeline *pipe)
-{
- if (pipe->dl)
- vsp1_dl_irq_display_start(pipe->dl);
-}
-
void vsp1_pipeline_frame_end(struct vsp1_pipeline *pipe)
{
- enum vsp1_pipeline_state state;
- unsigned long flags;
-
if (pipe == NULL)
return;
- if (pipe->dl)
- vsp1_dl_irq_frame_end(pipe->dl);
+ vsp1_dlm_irq_frame_end(pipe->output->dlm);
- /* Signal frame end to the pipeline handler. */
- pipe->frame_end(pipe);
-
- spin_lock_irqsave(&pipe->irqlock, flags);
-
- state = pipe->state;
-
- /* When using display lists in continuous frame mode the pipeline is
- * automatically restarted by the hardware.
- */
- if (!pipe->dl)
- pipe->state = VSP1_PIPELINE_STOPPED;
-
- /* If a stop has been requested, mark the pipeline as stopped and
- * return.
- */
- if (state == VSP1_PIPELINE_STOPPING) {
- wake_up(&pipe->wq);
- goto done;
- }
-
- /* Restart the pipeline if ready. */
- if (vsp1_pipeline_ready(pipe))
- vsp1_pipeline_run(pipe);
-
-done:
- spin_unlock_irqrestore(&pipe->irqlock, flags);
+ if (pipe->frame_end)
+ pipe->frame_end(pipe);
}
/*
@@ -324,9 +295,13 @@
* to be scaled, we disable alpha scaling when the UDS input has a fixed alpha
* value. The UDS then outputs a fixed alpha value which needs to be programmed
* from the input RPF alpha.
+ *
+ * This function can only be called from a subdev s_stream handler as it
+ * requires a valid display list context.
*/
void vsp1_pipeline_propagate_alpha(struct vsp1_pipeline *pipe,
struct vsp1_entity *input,
+ struct vsp1_dl_list *dl,
unsigned int alpha)
{
struct vsp1_entity *entity;
@@ -349,7 +324,7 @@
if (entity->type == VSP1_ENTITY_UDS) {
struct vsp1_uds *uds = to_uds(&entity->subdev);
- vsp1_uds_set_alpha(uds, alpha);
+ vsp1_uds_set_alpha(uds, dl, alpha);
break;
}
@@ -375,7 +350,7 @@
if (wpf == NULL)
continue;
- pipe = to_vsp1_pipeline(&wpf->entity.subdev.entity);
+ pipe = wpf->pipe;
if (pipe == NULL)
continue;
@@ -392,7 +367,7 @@
if (wpf == NULL)
continue;
- pipe = to_vsp1_pipeline(&wpf->entity.subdev.entity);
+ pipe = wpf->pipe;
if (pipe == NULL)
continue;
@@ -416,7 +391,7 @@
if (wpf == NULL)
continue;
- pipe = to_vsp1_pipeline(&wpf->entity.subdev.entity);
+ pipe = wpf->pipe;
if (pipe == NULL)
continue;
diff --git a/drivers/media/platform/vsp1/vsp1_pipe.h b/drivers/media/platform/vsp1/vsp1_pipe.h
index b2f3a8a..7b561135 100644
--- a/drivers/media/platform/vsp1/vsp1_pipe.h
+++ b/drivers/media/platform/vsp1/vsp1_pipe.h
@@ -13,13 +13,14 @@
#ifndef __VSP1_PIPE_H__
#define __VSP1_PIPE_H__
+#include <linux/kref.h>
#include <linux/list.h>
#include <linux/spinlock.h>
#include <linux/wait.h>
#include <media/media-entity.h>
-struct vsp1_dl;
+struct vsp1_dl_list;
struct vsp1_rwpf;
/*
@@ -63,7 +64,7 @@
* @wq: work queue to wait for state change completion
* @frame_end: frame end interrupt handler
* @lock: protects the pipeline use count and stream count
- * @use_count: number of video nodes using the pipeline
+ * @kref: pipeline reference count
* @stream_count: number of streaming video nodes
* @buffers_ready: bitmask of RPFs and WPFs with at least one buffer available
* @num_inputs: number of RPFs
@@ -86,7 +87,7 @@
void (*frame_end)(struct vsp1_pipeline *pipe);
struct mutex lock;
- unsigned int use_count;
+ struct kref kref;
unsigned int stream_count;
unsigned int buffers_ready;
@@ -100,17 +101,9 @@
struct list_head entities;
- struct vsp1_dl *dl;
+ struct vsp1_dl_list *dl;
};
-static inline struct vsp1_pipeline *to_vsp1_pipeline(struct media_entity *e)
-{
- if (likely(e->pipe))
- return container_of(e->pipe, struct vsp1_pipeline, pipe);
- else
- return NULL;
-}
-
void vsp1_pipeline_reset(struct vsp1_pipeline *pipe);
void vsp1_pipeline_init(struct vsp1_pipeline *pipe);
@@ -119,11 +112,11 @@
int vsp1_pipeline_stop(struct vsp1_pipeline *pipe);
bool vsp1_pipeline_ready(struct vsp1_pipeline *pipe);
-void vsp1_pipeline_display_start(struct vsp1_pipeline *pipe);
void vsp1_pipeline_frame_end(struct vsp1_pipeline *pipe);
void vsp1_pipeline_propagate_alpha(struct vsp1_pipeline *pipe,
struct vsp1_entity *input,
+ struct vsp1_dl_list *dl,
unsigned int alpha);
void vsp1_pipelines_suspend(struct vsp1_device *vsp1);
diff --git a/drivers/media/platform/vsp1/vsp1_regs.h b/drivers/media/platform/vsp1/vsp1_regs.h
index 069216f..927b5fb 100644
--- a/drivers/media/platform/vsp1/vsp1_regs.h
+++ b/drivers/media/platform/vsp1/vsp1_regs.h
@@ -217,6 +217,16 @@
#define VI6_RPF_SRCM_ADDR_C1 0x0344
#define VI6_RPF_SRCM_ADDR_AI 0x0348
+#define VI6_RPF_MULT_ALPHA 0x036c
+#define VI6_RPF_MULT_ALPHA_A_MMD_NONE (0 << 12)
+#define VI6_RPF_MULT_ALPHA_A_MMD_RATIO (1 << 12)
+#define VI6_RPF_MULT_ALPHA_P_MMD_NONE (0 << 8)
+#define VI6_RPF_MULT_ALPHA_P_MMD_RATIO (1 << 8)
+#define VI6_RPF_MULT_ALPHA_P_MMD_IMAGE (2 << 8)
+#define VI6_RPF_MULT_ALPHA_P_MMD_BOTH (3 << 8)
+#define VI6_RPF_MULT_ALPHA_RATIO_MASK (0xff < 0)
+#define VI6_RPF_MULT_ALPHA_RATIO_SHIFT 0
+
/* -----------------------------------------------------------------------------
* WPF Control Registers
*/
diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
index 5bc1d15..49168db 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -16,6 +16,8 @@
#include <media/v4l2-subdev.h>
#include "vsp1.h"
+#include "vsp1_dl.h"
+#include "vsp1_pipe.h"
#include "vsp1_rwpf.h"
#include "vsp1_video.h"
@@ -26,64 +28,50 @@
* Device Access
*/
-static inline void vsp1_rpf_write(struct vsp1_rwpf *rpf, u32 reg, u32 data)
+static inline void vsp1_rpf_write(struct vsp1_rwpf *rpf,
+ struct vsp1_dl_list *dl, u32 reg, u32 data)
{
- vsp1_mod_write(&rpf->entity, reg + rpf->entity.index * VI6_RPF_OFFSET,
- data);
+ vsp1_dl_list_write(dl, reg + rpf->entity.index * VI6_RPF_OFFSET, data);
}
/* -----------------------------------------------------------------------------
- * Controls
+ * V4L2 Subdevice Operations
*/
-static int rpf_s_ctrl(struct v4l2_ctrl *ctrl)
-{
- struct vsp1_rwpf *rpf =
- container_of(ctrl->handler, struct vsp1_rwpf, ctrls);
- struct vsp1_pipeline *pipe;
-
- if (!vsp1_entity_is_streaming(&rpf->entity))
- return 0;
-
- switch (ctrl->id) {
- case V4L2_CID_ALPHA_COMPONENT:
- vsp1_rpf_write(rpf, VI6_RPF_VRTCOL_SET,
- ctrl->val << VI6_RPF_VRTCOL_SET_LAYA_SHIFT);
-
- pipe = to_vsp1_pipeline(&rpf->entity.subdev.entity);
- vsp1_pipeline_propagate_alpha(pipe, &rpf->entity, ctrl->val);
- break;
- }
-
- return 0;
-}
-
-static const struct v4l2_ctrl_ops rpf_ctrl_ops = {
- .s_ctrl = rpf_s_ctrl,
+static struct v4l2_subdev_ops rpf_ops = {
+ .pad = &vsp1_rwpf_pad_ops,
};
/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Core Operations
+ * VSP1 Entity Operations
*/
-static int rpf_s_stream(struct v4l2_subdev *subdev, int enable)
+static void rpf_set_memory(struct vsp1_entity *entity, struct vsp1_dl_list *dl)
{
- struct vsp1_pipeline *pipe = to_vsp1_pipeline(&subdev->entity);
- struct vsp1_rwpf *rpf = to_rwpf(subdev);
- struct vsp1_device *vsp1 = rpf->entity.vsp1;
+ struct vsp1_rwpf *rpf = entity_to_rwpf(entity);
+
+ vsp1_rpf_write(rpf, dl, VI6_RPF_SRCM_ADDR_Y,
+ rpf->mem.addr[0] + rpf->offsets[0]);
+ vsp1_rpf_write(rpf, dl, VI6_RPF_SRCM_ADDR_C0,
+ rpf->mem.addr[1] + rpf->offsets[1]);
+ vsp1_rpf_write(rpf, dl, VI6_RPF_SRCM_ADDR_C1,
+ rpf->mem.addr[2] + rpf->offsets[1]);
+}
+
+static void rpf_configure(struct vsp1_entity *entity,
+ struct vsp1_pipeline *pipe,
+ struct vsp1_dl_list *dl)
+{
+ struct vsp1_rwpf *rpf = to_rwpf(&entity->subdev);
const struct vsp1_format_info *fmtinfo = rpf->fmtinfo;
const struct v4l2_pix_format_mplane *format = &rpf->format;
- const struct v4l2_rect *crop = &rpf->crop;
+ const struct v4l2_mbus_framefmt *source_format;
+ const struct v4l2_mbus_framefmt *sink_format;
+ const struct v4l2_rect *crop;
+ unsigned int left = 0;
+ unsigned int top = 0;
u32 pstride;
u32 infmt;
- int ret;
-
- ret = vsp1_entity_set_streaming(&rpf->entity, enable);
- if (ret < 0)
- return ret;
-
- if (!enable)
- return 0;
/* Source size, stride and crop offsets.
*
@@ -91,10 +79,12 @@
* left corner in the plane buffer. Only two offsets are needed, as
* planes 2 and 3 always have identical strides.
*/
- vsp1_rpf_write(rpf, VI6_RPF_SRC_BSIZE,
+ crop = vsp1_rwpf_get_crop(rpf, rpf->entity.config);
+
+ vsp1_rpf_write(rpf, dl, VI6_RPF_SRC_BSIZE,
(crop->width << VI6_RPF_SRC_BSIZE_BHSIZE_SHIFT) |
(crop->height << VI6_RPF_SRC_BSIZE_BVSIZE_SHIFT));
- vsp1_rpf_write(rpf, VI6_RPF_SRC_ESIZE,
+ vsp1_rpf_write(rpf, dl, VI6_RPF_SRC_ESIZE,
(crop->width << VI6_RPF_SRC_ESIZE_EHSIZE_SHIFT) |
(crop->height << VI6_RPF_SRC_ESIZE_EVSIZE_SHIFT));
@@ -103,26 +93,25 @@
pstride = format->plane_fmt[0].bytesperline
<< VI6_RPF_SRCM_PSTRIDE_Y_SHIFT;
- vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_Y,
- rpf->buf_addr[0] + rpf->offsets[0]);
-
if (format->num_planes > 1) {
rpf->offsets[1] = crop->top * format->plane_fmt[1].bytesperline
+ crop->left * fmtinfo->bpp[1] / 8;
pstride |= format->plane_fmt[1].bytesperline
<< VI6_RPF_SRCM_PSTRIDE_C_SHIFT;
-
- vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_C0,
- rpf->buf_addr[1] + rpf->offsets[1]);
-
- if (format->num_planes > 2)
- vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_C1,
- rpf->buf_addr[2] + rpf->offsets[1]);
+ } else {
+ rpf->offsets[1] = 0;
}
- vsp1_rpf_write(rpf, VI6_RPF_SRCM_PSTRIDE, pstride);
+ vsp1_rpf_write(rpf, dl, VI6_RPF_SRCM_PSTRIDE, pstride);
/* Format */
+ sink_format = vsp1_entity_get_pad_format(&rpf->entity,
+ rpf->entity.config,
+ RWPF_PAD_SINK);
+ source_format = vsp1_entity_get_pad_format(&rpf->entity,
+ rpf->entity.config,
+ RWPF_PAD_SOURCE);
+
infmt = VI6_RPF_INFMT_CIPM
| (fmtinfo->hwfmt << VI6_RPF_INFMT_RDFMT_SHIFT);
@@ -131,88 +120,98 @@
if (fmtinfo->swap_uv)
infmt |= VI6_RPF_INFMT_SPUVS;
- if (rpf->entity.formats[RWPF_PAD_SINK].code !=
- rpf->entity.formats[RWPF_PAD_SOURCE].code)
+ if (sink_format->code != source_format->code)
infmt |= VI6_RPF_INFMT_CSC;
- vsp1_rpf_write(rpf, VI6_RPF_INFMT, infmt);
- vsp1_rpf_write(rpf, VI6_RPF_DSWAP, fmtinfo->swap);
+ vsp1_rpf_write(rpf, dl, VI6_RPF_INFMT, infmt);
+ vsp1_rpf_write(rpf, dl, VI6_RPF_DSWAP, fmtinfo->swap);
/* Output location */
- vsp1_rpf_write(rpf, VI6_RPF_LOC,
- (rpf->location.left << VI6_RPF_LOC_HCOORD_SHIFT) |
- (rpf->location.top << VI6_RPF_LOC_VCOORD_SHIFT));
+ if (pipe->bru) {
+ const struct v4l2_rect *compose;
- /* Use the alpha channel (extended to 8 bits) when available or an
- * alpha value set through the V4L2_CID_ALPHA_COMPONENT control
- * otherwise. Disable color keying.
+ compose = vsp1_entity_get_pad_compose(pipe->bru,
+ pipe->bru->config,
+ rpf->bru_input);
+ left = compose->left;
+ top = compose->top;
+ }
+
+ vsp1_rpf_write(rpf, dl, VI6_RPF_LOC,
+ (left << VI6_RPF_LOC_HCOORD_SHIFT) |
+ (top << VI6_RPF_LOC_VCOORD_SHIFT));
+
+ /* On Gen2 use the alpha channel (extended to 8 bits) when available or
+ * a fixed alpha value set through the V4L2_CID_ALPHA_COMPONENT control
+ * otherwise.
+ *
+ * The Gen3 RPF has extended alpha capability and can both multiply the
+ * alpha channel by a fixed global alpha value, and multiply the pixel
+ * components to convert the input to premultiplied alpha.
+ *
+ * As alpha premultiplication is available in the BRU for both Gen2 and
+ * Gen3 we handle it there and use the Gen3 alpha multiplier for global
+ * alpha multiplication only. This however prevents conversion to
+ * premultiplied alpha if no BRU is present in the pipeline. If that use
+ * case turns out to be useful we will revisit the implementation (for
+ * Gen3 only).
+ *
+ * We enable alpha multiplication on Gen3 using the fixed alpha value
+ * set through the V4L2_CID_ALPHA_COMPONENT control when the input
+ * contains an alpha channel. On Gen2 the global alpha is ignored in
+ * that case.
+ *
+ * In all cases, disable color keying.
*/
- vsp1_rpf_write(rpf, VI6_RPF_ALPH_SEL, VI6_RPF_ALPH_SEL_AEXT_EXT |
+ vsp1_rpf_write(rpf, dl, VI6_RPF_ALPH_SEL, VI6_RPF_ALPH_SEL_AEXT_EXT |
(fmtinfo->alpha ? VI6_RPF_ALPH_SEL_ASEL_PACKED
: VI6_RPF_ALPH_SEL_ASEL_FIXED));
- if (vsp1->info->uapi)
- mutex_lock(rpf->ctrls.lock);
- vsp1_rpf_write(rpf, VI6_RPF_VRTCOL_SET,
- rpf->alpha->cur.val << VI6_RPF_VRTCOL_SET_LAYA_SHIFT);
- vsp1_pipeline_propagate_alpha(pipe, &rpf->entity, rpf->alpha->cur.val);
- if (vsp1->info->uapi)
- mutex_unlock(rpf->ctrls.lock);
+ vsp1_rpf_write(rpf, dl, VI6_RPF_VRTCOL_SET,
+ rpf->alpha << VI6_RPF_VRTCOL_SET_LAYA_SHIFT);
- vsp1_rpf_write(rpf, VI6_RPF_MSK_CTRL, 0);
- vsp1_rpf_write(rpf, VI6_RPF_CKEY_CTRL, 0);
+ if (entity->vsp1->info->gen == 3) {
+ u32 mult;
- return 0;
+ if (fmtinfo->alpha) {
+ /* When the input contains an alpha channel enable the
+ * alpha multiplier. If the input is premultiplied we
+ * need to multiply both the alpha channel and the pixel
+ * components by the global alpha value to keep them
+ * premultiplied. Otherwise multiply the alpha channel
+ * only.
+ */
+ bool premultiplied = format->flags
+ & V4L2_PIX_FMT_FLAG_PREMUL_ALPHA;
+
+ mult = VI6_RPF_MULT_ALPHA_A_MMD_RATIO
+ | (premultiplied ?
+ VI6_RPF_MULT_ALPHA_P_MMD_RATIO :
+ VI6_RPF_MULT_ALPHA_P_MMD_NONE)
+ | (rpf->alpha << VI6_RPF_MULT_ALPHA_RATIO_SHIFT);
+ } else {
+ /* When the input doesn't contain an alpha channel the
+ * global alpha value is applied in the unpacking unit,
+ * the alpha multiplier isn't needed and must be
+ * disabled.
+ */
+ mult = VI6_RPF_MULT_ALPHA_A_MMD_NONE
+ | VI6_RPF_MULT_ALPHA_P_MMD_NONE;
+ }
+
+ vsp1_rpf_write(rpf, dl, VI6_RPF_MULT_ALPHA, mult);
+ }
+
+ vsp1_pipeline_propagate_alpha(pipe, &rpf->entity, dl, rpf->alpha);
+
+ vsp1_rpf_write(rpf, dl, VI6_RPF_MSK_CTRL, 0);
+ vsp1_rpf_write(rpf, dl, VI6_RPF_CKEY_CTRL, 0);
+
}
-/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Operations
- */
-
-static struct v4l2_subdev_video_ops rpf_video_ops = {
- .s_stream = rpf_s_stream,
-};
-
-static struct v4l2_subdev_pad_ops rpf_pad_ops = {
- .enum_mbus_code = vsp1_rwpf_enum_mbus_code,
- .enum_frame_size = vsp1_rwpf_enum_frame_size,
- .get_fmt = vsp1_rwpf_get_format,
- .set_fmt = vsp1_rwpf_set_format,
- .get_selection = vsp1_rwpf_get_selection,
- .set_selection = vsp1_rwpf_set_selection,
-};
-
-static struct v4l2_subdev_ops rpf_ops = {
- .video = &rpf_video_ops,
- .pad = &rpf_pad_ops,
-};
-
-/* -----------------------------------------------------------------------------
- * Video Device Operations
- */
-
-static void rpf_set_memory(struct vsp1_rwpf *rpf, struct vsp1_rwpf_memory *mem)
-{
- unsigned int i;
-
- for (i = 0; i < 3; ++i)
- rpf->buf_addr[i] = mem->addr[i];
-
- if (!vsp1_entity_is_streaming(&rpf->entity))
- return;
-
- vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_Y,
- mem->addr[0] + rpf->offsets[0]);
- if (mem->num_planes > 1)
- vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_C0,
- mem->addr[1] + rpf->offsets[1]);
- if (mem->num_planes > 2)
- vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_C1,
- mem->addr[2] + rpf->offsets[1]);
-}
-
-static const struct vsp1_rwpf_operations rpf_vdev_ops = {
+static const struct vsp1_entity_operations rpf_entity_ops = {
.set_memory = rpf_set_memory,
+ .configure = rpf_configure,
};
/* -----------------------------------------------------------------------------
@@ -221,51 +220,31 @@
struct vsp1_rwpf *vsp1_rpf_create(struct vsp1_device *vsp1, unsigned int index)
{
- struct v4l2_subdev *subdev;
struct vsp1_rwpf *rpf;
+ char name[6];
int ret;
rpf = devm_kzalloc(vsp1->dev, sizeof(*rpf), GFP_KERNEL);
if (rpf == NULL)
return ERR_PTR(-ENOMEM);
- rpf->ops = &rpf_vdev_ops;
-
rpf->max_width = RPF_MAX_WIDTH;
rpf->max_height = RPF_MAX_HEIGHT;
+ rpf->entity.ops = &rpf_entity_ops;
rpf->entity.type = VSP1_ENTITY_RPF;
rpf->entity.index = index;
- ret = vsp1_entity_init(vsp1, &rpf->entity, 2);
+ sprintf(name, "rpf.%u", index);
+ ret = vsp1_entity_init(vsp1, &rpf->entity, name, 2, &rpf_ops);
if (ret < 0)
return ERR_PTR(ret);
- /* Initialize the V4L2 subdev. */
- subdev = &rpf->entity.subdev;
- v4l2_subdev_init(subdev, &rpf_ops);
-
- subdev->entity.ops = &vsp1->media_ops;
- subdev->internal_ops = &vsp1_subdev_internal_ops;
- snprintf(subdev->name, sizeof(subdev->name), "%s rpf.%u",
- dev_name(vsp1->dev), index);
- v4l2_set_subdevdata(subdev, rpf);
- subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
-
- vsp1_entity_init_formats(subdev, NULL);
-
/* Initialize the control handler. */
- v4l2_ctrl_handler_init(&rpf->ctrls, 1);
- rpf->alpha = v4l2_ctrl_new_std(&rpf->ctrls, &rpf_ctrl_ops,
- V4L2_CID_ALPHA_COMPONENT,
- 0, 255, 1, 255);
-
- rpf->entity.subdev.ctrl_handler = &rpf->ctrls;
-
- if (rpf->ctrls.error) {
+ ret = vsp1_rwpf_init_ctrls(rpf);
+ if (ret < 0) {
dev_err(vsp1->dev, "rpf%u: failed to initialize controls\n",
index);
- ret = rpf->ctrls.error;
goto error;
}
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.c b/drivers/media/platform/vsp1/vsp1_rwpf.c
index 9688c21..3b6e032 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.c
@@ -20,13 +20,20 @@
#define RWPF_MIN_WIDTH 1
#define RWPF_MIN_HEIGHT 1
+struct v4l2_rect *vsp1_rwpf_get_crop(struct vsp1_rwpf *rwpf,
+ struct v4l2_subdev_pad_config *config)
+{
+ return v4l2_subdev_get_try_crop(&rwpf->entity.subdev, config,
+ RWPF_PAD_SINK);
+}
+
/* -----------------------------------------------------------------------------
* V4L2 Subdevice Pad Operations
*/
-int vsp1_rwpf_enum_mbus_code(struct v4l2_subdev *subdev,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_mbus_code_enum *code)
+static int vsp1_rwpf_enum_mbus_code(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_mbus_code_enum *code)
{
static const unsigned int codes[] = {
MEDIA_BUS_FMT_ARGB8888_1X32,
@@ -41,75 +48,36 @@
return 0;
}
-int vsp1_rwpf_enum_frame_size(struct v4l2_subdev *subdev,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_frame_size_enum *fse)
-{
- struct vsp1_rwpf *rwpf = to_rwpf(subdev);
- struct v4l2_mbus_framefmt *format;
-
- format = vsp1_entity_get_pad_format(&rwpf->entity, cfg, fse->pad,
- fse->which);
-
- if (fse->index || fse->code != format->code)
- return -EINVAL;
-
- if (fse->pad == RWPF_PAD_SINK) {
- fse->min_width = RWPF_MIN_WIDTH;
- fse->max_width = rwpf->max_width;
- fse->min_height = RWPF_MIN_HEIGHT;
- fse->max_height = rwpf->max_height;
- } else {
- /* The size on the source pad are fixed and always identical to
- * the size on the sink pad.
- */
- fse->min_width = format->width;
- fse->max_width = format->width;
- fse->min_height = format->height;
- fse->max_height = format->height;
- }
-
- return 0;
-}
-
-static struct v4l2_rect *
-vsp1_rwpf_get_crop(struct vsp1_rwpf *rwpf, struct v4l2_subdev_pad_config *cfg, u32 which)
-{
- switch (which) {
- case V4L2_SUBDEV_FORMAT_TRY:
- return v4l2_subdev_get_try_crop(&rwpf->entity.subdev, cfg, RWPF_PAD_SINK);
- case V4L2_SUBDEV_FORMAT_ACTIVE:
- return &rwpf->crop;
- default:
- return NULL;
- }
-}
-
-int vsp1_rwpf_get_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_format *fmt)
+static int vsp1_rwpf_enum_frame_size(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_frame_size_enum *fse)
{
struct vsp1_rwpf *rwpf = to_rwpf(subdev);
- fmt->format = *vsp1_entity_get_pad_format(&rwpf->entity, cfg, fmt->pad,
- fmt->which);
-
- return 0;
+ return vsp1_subdev_enum_frame_size(subdev, cfg, fse, RWPF_MIN_WIDTH,
+ RWPF_MIN_HEIGHT, rwpf->max_width,
+ rwpf->max_height);
}
-int vsp1_rwpf_set_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_format *fmt)
+static int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *fmt)
{
struct vsp1_rwpf *rwpf = to_rwpf(subdev);
+ struct v4l2_subdev_pad_config *config;
struct v4l2_mbus_framefmt *format;
struct v4l2_rect *crop;
+ config = vsp1_entity_get_pad_config(&rwpf->entity, cfg, fmt->which);
+ if (!config)
+ return -EINVAL;
+
/* Default to YUV if the requested format is not supported. */
if (fmt->format.code != MEDIA_BUS_FMT_ARGB8888_1X32 &&
fmt->format.code != MEDIA_BUS_FMT_AYUV8_1X32)
fmt->format.code = MEDIA_BUS_FMT_AYUV8_1X32;
- format = vsp1_entity_get_pad_format(&rwpf->entity, cfg, fmt->pad,
- fmt->which);
+ format = vsp1_entity_get_pad_format(&rwpf->entity, config, fmt->pad);
if (fmt->pad == RWPF_PAD_SOURCE) {
/* The RWPF performs format conversion but can't scale, only the
@@ -131,39 +99,44 @@
fmt->format = *format;
/* Update the sink crop rectangle. */
- crop = vsp1_rwpf_get_crop(rwpf, cfg, fmt->which);
+ crop = vsp1_rwpf_get_crop(rwpf, config);
crop->left = 0;
crop->top = 0;
crop->width = fmt->format.width;
crop->height = fmt->format.height;
/* Propagate the format to the source pad. */
- format = vsp1_entity_get_pad_format(&rwpf->entity, cfg, RWPF_PAD_SOURCE,
- fmt->which);
+ format = vsp1_entity_get_pad_format(&rwpf->entity, config,
+ RWPF_PAD_SOURCE);
*format = fmt->format;
return 0;
}
-int vsp1_rwpf_get_selection(struct v4l2_subdev *subdev,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_selection *sel)
+static int vsp1_rwpf_get_selection(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_selection *sel)
{
struct vsp1_rwpf *rwpf = to_rwpf(subdev);
+ struct v4l2_subdev_pad_config *config;
struct v4l2_mbus_framefmt *format;
/* Cropping is implemented on the sink pad. */
if (sel->pad != RWPF_PAD_SINK)
return -EINVAL;
+ config = vsp1_entity_get_pad_config(&rwpf->entity, cfg, sel->which);
+ if (!config)
+ return -EINVAL;
+
switch (sel->target) {
case V4L2_SEL_TGT_CROP:
- sel->r = *vsp1_rwpf_get_crop(rwpf, cfg, sel->which);
+ sel->r = *vsp1_rwpf_get_crop(rwpf, config);
break;
case V4L2_SEL_TGT_CROP_BOUNDS:
- format = vsp1_entity_get_pad_format(&rwpf->entity, cfg,
- RWPF_PAD_SINK, sel->which);
+ format = vsp1_entity_get_pad_format(&rwpf->entity, config,
+ RWPF_PAD_SINK);
sel->r.left = 0;
sel->r.top = 0;
sel->r.width = format->width;
@@ -177,11 +150,12 @@
return 0;
}
-int vsp1_rwpf_set_selection(struct v4l2_subdev *subdev,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_selection *sel)
+static int vsp1_rwpf_set_selection(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_selection *sel)
{
struct vsp1_rwpf *rwpf = to_rwpf(subdev);
+ struct v4l2_subdev_pad_config *config;
struct v4l2_mbus_framefmt *format;
struct v4l2_rect *crop;
@@ -192,11 +166,15 @@
if (sel->target != V4L2_SEL_TGT_CROP)
return -EINVAL;
+ config = vsp1_entity_get_pad_config(&rwpf->entity, cfg, sel->which);
+ if (!config)
+ return -EINVAL;
+
/* Make sure the crop rectangle is entirely contained in the image. The
* WPF top and left offsets are limited to 255.
*/
- format = vsp1_entity_get_pad_format(&rwpf->entity, cfg, RWPF_PAD_SINK,
- sel->which);
+ format = vsp1_entity_get_pad_format(&rwpf->entity, config,
+ RWPF_PAD_SINK);
/* Restrict the crop rectangle coordinates to multiples of 2 to avoid
* shifting the color plane.
@@ -219,14 +197,59 @@
sel->r.height = min_t(unsigned int, sel->r.height,
format->height - sel->r.top);
- crop = vsp1_rwpf_get_crop(rwpf, cfg, sel->which);
+ crop = vsp1_rwpf_get_crop(rwpf, config);
*crop = sel->r;
/* Propagate the format to the source pad. */
- format = vsp1_entity_get_pad_format(&rwpf->entity, cfg, RWPF_PAD_SOURCE,
- sel->which);
+ format = vsp1_entity_get_pad_format(&rwpf->entity, config,
+ RWPF_PAD_SOURCE);
format->width = crop->width;
format->height = crop->height;
return 0;
}
+
+const struct v4l2_subdev_pad_ops vsp1_rwpf_pad_ops = {
+ .init_cfg = vsp1_entity_init_cfg,
+ .enum_mbus_code = vsp1_rwpf_enum_mbus_code,
+ .enum_frame_size = vsp1_rwpf_enum_frame_size,
+ .get_fmt = vsp1_subdev_get_pad_format,
+ .set_fmt = vsp1_rwpf_set_format,
+ .get_selection = vsp1_rwpf_get_selection,
+ .set_selection = vsp1_rwpf_set_selection,
+};
+
+/* -----------------------------------------------------------------------------
+ * Controls
+ */
+
+static int vsp1_rwpf_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct vsp1_rwpf *rwpf =
+ container_of(ctrl->handler, struct vsp1_rwpf, ctrls);
+
+ switch (ctrl->id) {
+ case V4L2_CID_ALPHA_COMPONENT:
+ rwpf->alpha = ctrl->val;
+ break;
+ }
+
+ return 0;
+}
+
+static const struct v4l2_ctrl_ops vsp1_rwpf_ctrl_ops = {
+ .s_ctrl = vsp1_rwpf_s_ctrl,
+};
+
+int vsp1_rwpf_init_ctrls(struct vsp1_rwpf *rwpf)
+{
+ rwpf->alpha = 255;
+
+ v4l2_ctrl_handler_init(&rwpf->ctrls, 1);
+ v4l2_ctrl_new_std(&rwpf->ctrls, &vsp1_rwpf_ctrl_ops,
+ V4L2_CID_ALPHA_COMPONENT, 0, 255, 1, 255);
+
+ rwpf->entity.subdev.ctrl_handler = &rwpf->ctrls;
+
+ return rwpf->ctrls.error;
+}
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.h b/drivers/media/platform/vsp1/vsp1_rwpf.h
index 8e82356..9ff7c78 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.h
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.h
@@ -24,42 +24,35 @@
#define RWPF_PAD_SOURCE 1
struct v4l2_ctrl;
+struct vsp1_dl_manager;
+struct vsp1_pipeline;
struct vsp1_rwpf;
struct vsp1_video;
struct vsp1_rwpf_memory {
- unsigned int num_planes;
dma_addr_t addr[3];
- unsigned int length[3];
-};
-
-struct vsp1_rwpf_operations {
- void (*set_memory)(struct vsp1_rwpf *rwpf,
- struct vsp1_rwpf_memory *mem);
};
struct vsp1_rwpf {
struct vsp1_entity entity;
struct v4l2_ctrl_handler ctrls;
- struct v4l2_ctrl *alpha;
+ struct vsp1_pipeline *pipe;
struct vsp1_video *video;
- const struct vsp1_rwpf_operations *ops;
-
unsigned int max_width;
unsigned int max_height;
struct v4l2_pix_format_mplane format;
const struct vsp1_format_info *fmtinfo;
- struct {
- unsigned int left;
- unsigned int top;
- } location;
- struct v4l2_rect crop;
+ unsigned int bru_input;
+
+ unsigned int alpha;
unsigned int offsets[2];
- dma_addr_t buf_addr[3];
+ struct vsp1_rwpf_memory mem;
+
+ struct vsp1_dl_manager *dlm;
};
static inline struct vsp1_rwpf *to_rwpf(struct v4l2_subdev *subdev)
@@ -67,24 +60,31 @@
return container_of(subdev, struct vsp1_rwpf, entity.subdev);
}
+static inline struct vsp1_rwpf *entity_to_rwpf(struct vsp1_entity *entity)
+{
+ return container_of(entity, struct vsp1_rwpf, entity);
+}
+
struct vsp1_rwpf *vsp1_rpf_create(struct vsp1_device *vsp1, unsigned int index);
struct vsp1_rwpf *vsp1_wpf_create(struct vsp1_device *vsp1, unsigned int index);
-int vsp1_rwpf_enum_mbus_code(struct v4l2_subdev *subdev,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_mbus_code_enum *code);
-int vsp1_rwpf_enum_frame_size(struct v4l2_subdev *subdev,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_frame_size_enum *fse);
-int vsp1_rwpf_get_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_format *fmt);
-int vsp1_rwpf_set_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_format *fmt);
-int vsp1_rwpf_get_selection(struct v4l2_subdev *subdev,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_selection *sel);
-int vsp1_rwpf_set_selection(struct v4l2_subdev *subdev,
- struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_selection *sel);
+int vsp1_rwpf_init_ctrls(struct vsp1_rwpf *rwpf);
+
+extern const struct v4l2_subdev_pad_ops vsp1_rwpf_pad_ops;
+
+struct v4l2_rect *vsp1_rwpf_get_crop(struct vsp1_rwpf *rwpf,
+ struct v4l2_subdev_pad_config *config);
+/**
+ * vsp1_rwpf_set_memory - Configure DMA addresses for a [RW]PF
+ * @rwpf: the [RW]PF instance
+ * @dl: the display list
+ *
+ * This function applies the cached memory buffer address to the display list.
+ */
+static inline void vsp1_rwpf_set_memory(struct vsp1_rwpf *rwpf,
+ struct vsp1_dl_list *dl)
+{
+ rwpf->entity.ops->set_memory(&rwpf->entity, dl);
+}
#endif /* __VSP1_RWPF_H__ */
diff --git a/drivers/media/platform/vsp1/vsp1_sru.c b/drivers/media/platform/vsp1/vsp1_sru.c
index cc09efb..97ef997 100644
--- a/drivers/media/platform/vsp1/vsp1_sru.c
+++ b/drivers/media/platform/vsp1/vsp1_sru.c
@@ -17,6 +17,7 @@
#include <media/v4l2-subdev.h>
#include "vsp1.h"
+#include "vsp1_dl.h"
#include "vsp1_sru.h"
#define SRU_MIN_SIZE 4U
@@ -26,14 +27,10 @@
* Device Access
*/
-static inline u32 vsp1_sru_read(struct vsp1_sru *sru, u32 reg)
+static inline void vsp1_sru_write(struct vsp1_sru *sru, struct vsp1_dl_list *dl,
+ u32 reg, u32 data)
{
- return vsp1_read(sru->entity.vsp1, reg);
-}
-
-static inline void vsp1_sru_write(struct vsp1_sru *sru, u32 reg, u32 data)
-{
- vsp1_write(sru->entity.vsp1, reg, data);
+ vsp1_dl_list_write(dl, reg, data);
}
/* -----------------------------------------------------------------------------
@@ -82,20 +79,10 @@
{
struct vsp1_sru *sru =
container_of(ctrl->handler, struct vsp1_sru, ctrls);
- const struct vsp1_sru_param *param;
- u32 value;
switch (ctrl->id) {
case V4L2_CID_VSP1_SRU_INTENSITY:
- param = &vsp1_sru_params[ctrl->val - 1];
-
- value = vsp1_sru_read(sru, VI6_SRU_CTRL0);
- value &= ~(VI6_SRU_CTRL0_PARAM0_MASK |
- VI6_SRU_CTRL0_PARAM1_MASK);
- value |= param->ctrl0;
- vsp1_sru_write(sru, VI6_SRU_CTRL0, value);
-
- vsp1_sru_write(sru, VI6_SRU_CTRL2, param->ctrl2);
+ sru->intensity = ctrl->val;
break;
}
@@ -118,54 +105,7 @@
};
/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Core Operations
- */
-
-static int sru_s_stream(struct v4l2_subdev *subdev, int enable)
-{
- struct vsp1_sru *sru = to_sru(subdev);
- struct v4l2_mbus_framefmt *input;
- struct v4l2_mbus_framefmt *output;
- u32 ctrl0;
- int ret;
-
- ret = vsp1_entity_set_streaming(&sru->entity, enable);
- if (ret < 0)
- return ret;
-
- if (!enable)
- return 0;
-
- input = &sru->entity.formats[SRU_PAD_SINK];
- output = &sru->entity.formats[SRU_PAD_SOURCE];
-
- if (input->code == MEDIA_BUS_FMT_ARGB8888_1X32)
- ctrl0 = VI6_SRU_CTRL0_PARAM2 | VI6_SRU_CTRL0_PARAM3
- | VI6_SRU_CTRL0_PARAM4;
- else
- ctrl0 = VI6_SRU_CTRL0_PARAM3;
-
- if (input->width != output->width)
- ctrl0 |= VI6_SRU_CTRL0_MODE_UPSCALE;
-
- /* Take the control handler lock to ensure that the CTRL0 value won't be
- * changed behind our back by a set control operation.
- */
- if (sru->entity.vsp1->info->uapi)
- mutex_lock(sru->ctrls.lock);
- ctrl0 |= vsp1_sru_read(sru, VI6_SRU_CTRL0)
- & (VI6_SRU_CTRL0_PARAM0_MASK | VI6_SRU_CTRL0_PARAM1_MASK);
- vsp1_sru_write(sru, VI6_SRU_CTRL0, ctrl0);
- if (sru->entity.vsp1->info->uapi)
- mutex_unlock(sru->ctrls.lock);
-
- vsp1_sru_write(sru, VI6_SRU_CTRL1, VI6_SRU_CTRL1_PARAM5);
-
- return 0;
-}
-
-/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Pad Operations
+ * V4L2 Subdevice Operations
*/
static int sru_enum_mbus_code(struct v4l2_subdev *subdev,
@@ -176,27 +116,9 @@
MEDIA_BUS_FMT_ARGB8888_1X32,
MEDIA_BUS_FMT_AYUV8_1X32,
};
- struct vsp1_sru *sru = to_sru(subdev);
- struct v4l2_mbus_framefmt *format;
- if (code->pad == SRU_PAD_SINK) {
- if (code->index >= ARRAY_SIZE(codes))
- return -EINVAL;
-
- code->code = codes[code->index];
- } else {
- /* The SRU can't perform format conversion, the sink format is
- * always identical to the source format.
- */
- if (code->index)
- return -EINVAL;
-
- format = vsp1_entity_get_pad_format(&sru->entity, cfg,
- SRU_PAD_SINK, code->which);
- code->code = format->code;
- }
-
- return 0;
+ return vsp1_subdev_enum_mbus_code(subdev, cfg, code, codes,
+ ARRAY_SIZE(codes));
}
static int sru_enum_frame_size(struct v4l2_subdev *subdev,
@@ -204,10 +126,14 @@
struct v4l2_subdev_frame_size_enum *fse)
{
struct vsp1_sru *sru = to_sru(subdev);
+ struct v4l2_subdev_pad_config *config;
struct v4l2_mbus_framefmt *format;
- format = vsp1_entity_get_pad_format(&sru->entity, cfg,
- SRU_PAD_SINK, fse->which);
+ config = vsp1_entity_get_pad_config(&sru->entity, cfg, fse->which);
+ if (!config)
+ return -EINVAL;
+
+ format = vsp1_entity_get_pad_format(&sru->entity, config, SRU_PAD_SINK);
if (fse->index || fse->code != format->code)
return -EINVAL;
@@ -233,20 +159,9 @@
return 0;
}
-static int sru_get_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_format *fmt)
-{
- struct vsp1_sru *sru = to_sru(subdev);
-
- fmt->format = *vsp1_entity_get_pad_format(&sru->entity, cfg, fmt->pad,
- fmt->which);
-
- return 0;
-}
-
-static void sru_try_format(struct vsp1_sru *sru, struct v4l2_subdev_pad_config *cfg,
- unsigned int pad, struct v4l2_mbus_framefmt *fmt,
- enum v4l2_subdev_format_whence which)
+static void sru_try_format(struct vsp1_sru *sru,
+ struct v4l2_subdev_pad_config *config,
+ unsigned int pad, struct v4l2_mbus_framefmt *fmt)
{
struct v4l2_mbus_framefmt *format;
unsigned int input_area;
@@ -265,8 +180,8 @@
case SRU_PAD_SOURCE:
/* The SRU can't perform format conversion. */
- format = vsp1_entity_get_pad_format(&sru->entity, cfg,
- SRU_PAD_SINK, which);
+ format = vsp1_entity_get_pad_format(&sru->entity, config,
+ SRU_PAD_SINK);
fmt->code = format->code;
/* We can upscale by 2 in both direction, but not independently.
@@ -295,57 +210,94 @@
fmt->colorspace = V4L2_COLORSPACE_SRGB;
}
-static int sru_set_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
+static int sru_set_format(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_pad_config *cfg,
struct v4l2_subdev_format *fmt)
{
struct vsp1_sru *sru = to_sru(subdev);
+ struct v4l2_subdev_pad_config *config;
struct v4l2_mbus_framefmt *format;
- sru_try_format(sru, cfg, fmt->pad, &fmt->format, fmt->which);
+ config = vsp1_entity_get_pad_config(&sru->entity, cfg, fmt->which);
+ if (!config)
+ return -EINVAL;
- format = vsp1_entity_get_pad_format(&sru->entity, cfg, fmt->pad,
- fmt->which);
+ sru_try_format(sru, config, fmt->pad, &fmt->format);
+
+ format = vsp1_entity_get_pad_format(&sru->entity, config, fmt->pad);
*format = fmt->format;
if (fmt->pad == SRU_PAD_SINK) {
/* Propagate the format to the source pad. */
- format = vsp1_entity_get_pad_format(&sru->entity, cfg,
- SRU_PAD_SOURCE, fmt->which);
+ format = vsp1_entity_get_pad_format(&sru->entity, config,
+ SRU_PAD_SOURCE);
*format = fmt->format;
- sru_try_format(sru, cfg, SRU_PAD_SOURCE, format, fmt->which);
+ sru_try_format(sru, config, SRU_PAD_SOURCE, format);
}
return 0;
}
-/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Operations
- */
-
-static struct v4l2_subdev_video_ops sru_video_ops = {
- .s_stream = sru_s_stream,
-};
-
static struct v4l2_subdev_pad_ops sru_pad_ops = {
+ .init_cfg = vsp1_entity_init_cfg,
.enum_mbus_code = sru_enum_mbus_code,
.enum_frame_size = sru_enum_frame_size,
- .get_fmt = sru_get_format,
+ .get_fmt = vsp1_subdev_get_pad_format,
.set_fmt = sru_set_format,
};
static struct v4l2_subdev_ops sru_ops = {
- .video = &sru_video_ops,
.pad = &sru_pad_ops,
};
/* -----------------------------------------------------------------------------
+ * VSP1 Entity Operations
+ */
+
+static void sru_configure(struct vsp1_entity *entity,
+ struct vsp1_pipeline *pipe,
+ struct vsp1_dl_list *dl)
+{
+ const struct vsp1_sru_param *param;
+ struct vsp1_sru *sru = to_sru(&entity->subdev);
+ struct v4l2_mbus_framefmt *input;
+ struct v4l2_mbus_framefmt *output;
+ u32 ctrl0;
+
+ input = vsp1_entity_get_pad_format(&sru->entity, sru->entity.config,
+ SRU_PAD_SINK);
+ output = vsp1_entity_get_pad_format(&sru->entity, sru->entity.config,
+ SRU_PAD_SOURCE);
+
+ if (input->code == MEDIA_BUS_FMT_ARGB8888_1X32)
+ ctrl0 = VI6_SRU_CTRL0_PARAM2 | VI6_SRU_CTRL0_PARAM3
+ | VI6_SRU_CTRL0_PARAM4;
+ else
+ ctrl0 = VI6_SRU_CTRL0_PARAM3;
+
+ if (input->width != output->width)
+ ctrl0 |= VI6_SRU_CTRL0_MODE_UPSCALE;
+
+ param = &vsp1_sru_params[sru->intensity - 1];
+
+ ctrl0 |= param->ctrl0;
+
+ vsp1_sru_write(sru, dl, VI6_SRU_CTRL0, ctrl0);
+ vsp1_sru_write(sru, dl, VI6_SRU_CTRL1, VI6_SRU_CTRL1_PARAM5);
+ vsp1_sru_write(sru, dl, VI6_SRU_CTRL2, param->ctrl2);
+}
+
+static const struct vsp1_entity_operations sru_entity_ops = {
+ .configure = sru_configure,
+};
+
+/* -----------------------------------------------------------------------------
* Initialization and Cleanup
*/
struct vsp1_sru *vsp1_sru_create(struct vsp1_device *vsp1)
{
- struct v4l2_subdev *subdev;
struct vsp1_sru *sru;
int ret;
@@ -353,29 +305,19 @@
if (sru == NULL)
return ERR_PTR(-ENOMEM);
+ sru->entity.ops = &sru_entity_ops;
sru->entity.type = VSP1_ENTITY_SRU;
- ret = vsp1_entity_init(vsp1, &sru->entity, 2);
+ ret = vsp1_entity_init(vsp1, &sru->entity, "sru", 2, &sru_ops);
if (ret < 0)
return ERR_PTR(ret);
- /* Initialize the V4L2 subdev. */
- subdev = &sru->entity.subdev;
- v4l2_subdev_init(subdev, &sru_ops);
-
- subdev->entity.ops = &vsp1->media_ops;
- subdev->internal_ops = &vsp1_subdev_internal_ops;
- snprintf(subdev->name, sizeof(subdev->name), "%s sru",
- dev_name(vsp1->dev));
- v4l2_set_subdevdata(subdev, sru);
- subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
-
- vsp1_entity_init_formats(subdev, NULL);
-
/* Initialize the control handler. */
v4l2_ctrl_handler_init(&sru->ctrls, 1);
v4l2_ctrl_new_custom(&sru->ctrls, &sru_intensity_control, NULL);
+ sru->intensity = 1;
+
sru->entity.subdev.ctrl_handler = &sru->ctrls;
if (sru->ctrls.error) {
diff --git a/drivers/media/platform/vsp1/vsp1_sru.h b/drivers/media/platform/vsp1/vsp1_sru.h
index b6768bf..85e24145 100644
--- a/drivers/media/platform/vsp1/vsp1_sru.h
+++ b/drivers/media/platform/vsp1/vsp1_sru.h
@@ -28,6 +28,8 @@
struct vsp1_entity entity;
struct v4l2_ctrl_handler ctrls;
+
+ unsigned int intensity;
};
static inline struct vsp1_sru *to_sru(struct v4l2_subdev *subdev)
diff --git a/drivers/media/platform/vsp1/vsp1_uds.c b/drivers/media/platform/vsp1/vsp1_uds.c
index bba6777..1875e29 100644
--- a/drivers/media/platform/vsp1/vsp1_uds.c
+++ b/drivers/media/platform/vsp1/vsp1_uds.c
@@ -17,6 +17,7 @@
#include <media/v4l2-subdev.h>
#include "vsp1.h"
+#include "vsp1_dl.h"
#include "vsp1_uds.h"
#define UDS_MIN_SIZE 4U
@@ -29,19 +30,21 @@
* Device Access
*/
-static inline void vsp1_uds_write(struct vsp1_uds *uds, u32 reg, u32 data)
+static inline void vsp1_uds_write(struct vsp1_uds *uds, struct vsp1_dl_list *dl,
+ u32 reg, u32 data)
{
- vsp1_write(uds->entity.vsp1,
- reg + uds->entity.index * VI6_UDS_OFFSET, data);
+ vsp1_dl_list_write(dl, reg + uds->entity.index * VI6_UDS_OFFSET, data);
}
/* -----------------------------------------------------------------------------
* Scaling Computation
*/
-void vsp1_uds_set_alpha(struct vsp1_uds *uds, unsigned int alpha)
+void vsp1_uds_set_alpha(struct vsp1_uds *uds, struct vsp1_dl_list *dl,
+ unsigned int alpha)
{
- vsp1_uds_write(uds, VI6_UDS_ALPVAL, alpha << VI6_UDS_ALPVAL_VAL0_SHIFT);
+ vsp1_uds_write(uds, dl, VI6_UDS_ALPVAL,
+ alpha << VI6_UDS_ALPVAL_VAL0_SHIFT);
}
/*
@@ -105,60 +108,6 @@
}
/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Core Operations
- */
-
-static int uds_s_stream(struct v4l2_subdev *subdev, int enable)
-{
- struct vsp1_uds *uds = to_uds(subdev);
- const struct v4l2_mbus_framefmt *output;
- const struct v4l2_mbus_framefmt *input;
- unsigned int hscale;
- unsigned int vscale;
- bool multitap;
-
- if (!enable)
- return 0;
-
- input = &uds->entity.formats[UDS_PAD_SINK];
- output = &uds->entity.formats[UDS_PAD_SOURCE];
-
- hscale = uds_compute_ratio(input->width, output->width);
- vscale = uds_compute_ratio(input->height, output->height);
-
- dev_dbg(uds->entity.vsp1->dev, "hscale %u vscale %u\n", hscale, vscale);
-
- /* Multi-tap scaling can't be enabled along with alpha scaling when
- * scaling down with a factor lower than or equal to 1/2 in either
- * direction.
- */
- if (uds->scale_alpha && (hscale >= 8192 || vscale >= 8192))
- multitap = false;
- else
- multitap = true;
-
- vsp1_uds_write(uds, VI6_UDS_CTRL,
- (uds->scale_alpha ? VI6_UDS_CTRL_AON : 0) |
- (multitap ? VI6_UDS_CTRL_BC : 0));
-
- vsp1_uds_write(uds, VI6_UDS_PASS_BWIDTH,
- (uds_passband_width(hscale)
- << VI6_UDS_PASS_BWIDTH_H_SHIFT) |
- (uds_passband_width(vscale)
- << VI6_UDS_PASS_BWIDTH_V_SHIFT));
-
- /* Set the scaling ratios and the output size. */
- vsp1_uds_write(uds, VI6_UDS_SCALE,
- (hscale << VI6_UDS_SCALE_HFRAC_SHIFT) |
- (vscale << VI6_UDS_SCALE_VFRAC_SHIFT));
- vsp1_uds_write(uds, VI6_UDS_CLIP_SIZE,
- (output->width << VI6_UDS_CLIP_SIZE_HSIZE_SHIFT) |
- (output->height << VI6_UDS_CLIP_SIZE_VSIZE_SHIFT));
-
- return 0;
-}
-
-/* -----------------------------------------------------------------------------
* V4L2 Subdevice Pad Operations
*/
@@ -170,28 +119,9 @@
MEDIA_BUS_FMT_ARGB8888_1X32,
MEDIA_BUS_FMT_AYUV8_1X32,
};
- struct vsp1_uds *uds = to_uds(subdev);
- if (code->pad == UDS_PAD_SINK) {
- if (code->index >= ARRAY_SIZE(codes))
- return -EINVAL;
-
- code->code = codes[code->index];
- } else {
- struct v4l2_mbus_framefmt *format;
-
- /* The UDS can't perform format conversion, the sink format is
- * always identical to the source format.
- */
- if (code->index)
- return -EINVAL;
-
- format = vsp1_entity_get_pad_format(&uds->entity, cfg,
- UDS_PAD_SINK, code->which);
- code->code = format->code;
- }
-
- return 0;
+ return vsp1_subdev_enum_mbus_code(subdev, cfg, code, codes,
+ ARRAY_SIZE(codes));
}
static int uds_enum_frame_size(struct v4l2_subdev *subdev,
@@ -199,10 +129,15 @@
struct v4l2_subdev_frame_size_enum *fse)
{
struct vsp1_uds *uds = to_uds(subdev);
+ struct v4l2_subdev_pad_config *config;
struct v4l2_mbus_framefmt *format;
- format = vsp1_entity_get_pad_format(&uds->entity, cfg,
- UDS_PAD_SINK, fse->which);
+ config = vsp1_entity_get_pad_config(&uds->entity, cfg, fse->which);
+ if (!config)
+ return -EINVAL;
+
+ format = vsp1_entity_get_pad_format(&uds->entity, config,
+ UDS_PAD_SINK);
if (fse->index || fse->code != format->code)
return -EINVAL;
@@ -222,20 +157,9 @@
return 0;
}
-static int uds_get_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
- struct v4l2_subdev_format *fmt)
-{
- struct vsp1_uds *uds = to_uds(subdev);
-
- fmt->format = *vsp1_entity_get_pad_format(&uds->entity, cfg, fmt->pad,
- fmt->which);
-
- return 0;
-}
-
-static void uds_try_format(struct vsp1_uds *uds, struct v4l2_subdev_pad_config *cfg,
- unsigned int pad, struct v4l2_mbus_framefmt *fmt,
- enum v4l2_subdev_format_whence which)
+static void uds_try_format(struct vsp1_uds *uds,
+ struct v4l2_subdev_pad_config *config,
+ unsigned int pad, struct v4l2_mbus_framefmt *fmt)
{
struct v4l2_mbus_framefmt *format;
unsigned int minimum;
@@ -254,8 +178,8 @@
case UDS_PAD_SOURCE:
/* The UDS scales but can't perform format conversion. */
- format = vsp1_entity_get_pad_format(&uds->entity, cfg,
- UDS_PAD_SINK, which);
+ format = vsp1_entity_get_pad_format(&uds->entity, config,
+ UDS_PAD_SINK);
fmt->code = format->code;
uds_output_limits(format->width, &minimum, &maximum);
@@ -269,25 +193,30 @@
fmt->colorspace = V4L2_COLORSPACE_SRGB;
}
-static int uds_set_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
+static int uds_set_format(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_pad_config *cfg,
struct v4l2_subdev_format *fmt)
{
struct vsp1_uds *uds = to_uds(subdev);
+ struct v4l2_subdev_pad_config *config;
struct v4l2_mbus_framefmt *format;
- uds_try_format(uds, cfg, fmt->pad, &fmt->format, fmt->which);
+ config = vsp1_entity_get_pad_config(&uds->entity, cfg, fmt->which);
+ if (!config)
+ return -EINVAL;
- format = vsp1_entity_get_pad_format(&uds->entity, cfg, fmt->pad,
- fmt->which);
+ uds_try_format(uds, config, fmt->pad, &fmt->format);
+
+ format = vsp1_entity_get_pad_format(&uds->entity, config, fmt->pad);
*format = fmt->format;
if (fmt->pad == UDS_PAD_SINK) {
/* Propagate the format to the source pad. */
- format = vsp1_entity_get_pad_format(&uds->entity, cfg,
- UDS_PAD_SOURCE, fmt->which);
+ format = vsp1_entity_get_pad_format(&uds->entity, config,
+ UDS_PAD_SOURCE);
*format = fmt->format;
- uds_try_format(uds, cfg, UDS_PAD_SOURCE, format, fmt->which);
+ uds_try_format(uds, config, UDS_PAD_SOURCE, format);
}
return 0;
@@ -297,55 +226,97 @@
* V4L2 Subdevice Operations
*/
-static struct v4l2_subdev_video_ops uds_video_ops = {
- .s_stream = uds_s_stream,
-};
-
static struct v4l2_subdev_pad_ops uds_pad_ops = {
+ .init_cfg = vsp1_entity_init_cfg,
.enum_mbus_code = uds_enum_mbus_code,
.enum_frame_size = uds_enum_frame_size,
- .get_fmt = uds_get_format,
+ .get_fmt = vsp1_subdev_get_pad_format,
.set_fmt = uds_set_format,
};
static struct v4l2_subdev_ops uds_ops = {
- .video = &uds_video_ops,
.pad = &uds_pad_ops,
};
/* -----------------------------------------------------------------------------
+ * VSP1 Entity Operations
+ */
+
+static void uds_configure(struct vsp1_entity *entity,
+ struct vsp1_pipeline *pipe,
+ struct vsp1_dl_list *dl)
+{
+ struct vsp1_uds *uds = to_uds(&entity->subdev);
+ const struct v4l2_mbus_framefmt *output;
+ const struct v4l2_mbus_framefmt *input;
+ unsigned int hscale;
+ unsigned int vscale;
+ bool multitap;
+
+ input = vsp1_entity_get_pad_format(&uds->entity, uds->entity.config,
+ UDS_PAD_SINK);
+ output = vsp1_entity_get_pad_format(&uds->entity, uds->entity.config,
+ UDS_PAD_SOURCE);
+
+ hscale = uds_compute_ratio(input->width, output->width);
+ vscale = uds_compute_ratio(input->height, output->height);
+
+ dev_dbg(uds->entity.vsp1->dev, "hscale %u vscale %u\n", hscale, vscale);
+
+ /* Multi-tap scaling can't be enabled along with alpha scaling when
+ * scaling down with a factor lower than or equal to 1/2 in either
+ * direction.
+ */
+ if (uds->scale_alpha && (hscale >= 8192 || vscale >= 8192))
+ multitap = false;
+ else
+ multitap = true;
+
+ vsp1_uds_write(uds, dl, VI6_UDS_CTRL,
+ (uds->scale_alpha ? VI6_UDS_CTRL_AON : 0) |
+ (multitap ? VI6_UDS_CTRL_BC : 0));
+
+ vsp1_uds_write(uds, dl, VI6_UDS_PASS_BWIDTH,
+ (uds_passband_width(hscale)
+ << VI6_UDS_PASS_BWIDTH_H_SHIFT) |
+ (uds_passband_width(vscale)
+ << VI6_UDS_PASS_BWIDTH_V_SHIFT));
+
+ /* Set the scaling ratios and the output size. */
+ vsp1_uds_write(uds, dl, VI6_UDS_SCALE,
+ (hscale << VI6_UDS_SCALE_HFRAC_SHIFT) |
+ (vscale << VI6_UDS_SCALE_VFRAC_SHIFT));
+ vsp1_uds_write(uds, dl, VI6_UDS_CLIP_SIZE,
+ (output->width << VI6_UDS_CLIP_SIZE_HSIZE_SHIFT) |
+ (output->height << VI6_UDS_CLIP_SIZE_VSIZE_SHIFT));
+}
+
+static const struct vsp1_entity_operations uds_entity_ops = {
+ .configure = uds_configure,
+};
+
+/* -----------------------------------------------------------------------------
* Initialization and Cleanup
*/
struct vsp1_uds *vsp1_uds_create(struct vsp1_device *vsp1, unsigned int index)
{
- struct v4l2_subdev *subdev;
struct vsp1_uds *uds;
+ char name[6];
int ret;
uds = devm_kzalloc(vsp1->dev, sizeof(*uds), GFP_KERNEL);
if (uds == NULL)
return ERR_PTR(-ENOMEM);
+ uds->entity.ops = &uds_entity_ops;
uds->entity.type = VSP1_ENTITY_UDS;
uds->entity.index = index;
- ret = vsp1_entity_init(vsp1, &uds->entity, 2);
+ sprintf(name, "uds.%u", index);
+ ret = vsp1_entity_init(vsp1, &uds->entity, name, 2, &uds_ops);
if (ret < 0)
return ERR_PTR(ret);
- /* Initialize the V4L2 subdev. */
- subdev = &uds->entity.subdev;
- v4l2_subdev_init(subdev, &uds_ops);
-
- subdev->entity.ops = &vsp1->media_ops;
- subdev->internal_ops = &vsp1_subdev_internal_ops;
- snprintf(subdev->name, sizeof(subdev->name), "%s uds.%u",
- dev_name(vsp1->dev), index);
- v4l2_set_subdevdata(subdev, uds);
- subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
-
- vsp1_entity_init_formats(subdev, NULL);
-
return uds;
}
diff --git a/drivers/media/platform/vsp1/vsp1_uds.h b/drivers/media/platform/vsp1/vsp1_uds.h
index 031ac0d..5c8cbfca 100644
--- a/drivers/media/platform/vsp1/vsp1_uds.h
+++ b/drivers/media/platform/vsp1/vsp1_uds.h
@@ -35,6 +35,7 @@
struct vsp1_uds *vsp1_uds_create(struct vsp1_device *vsp1, unsigned int index);
-void vsp1_uds_set_alpha(struct vsp1_uds *uds, unsigned int alpha);
+void vsp1_uds_set_alpha(struct vsp1_uds *uds, struct vsp1_dl_list *dl,
+ unsigned int alpha);
#endif /* __VSP1_UDS_H__ */
diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index 72cc7d3..a9aec5c 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -29,6 +29,7 @@
#include "vsp1.h"
#include "vsp1_bru.h"
+#include "vsp1_dl.h"
#include "vsp1_entity.h"
#include "vsp1_pipe.h"
#include "vsp1_rwpf.h"
@@ -171,209 +172,6 @@
* Pipeline Management
*/
-static int vsp1_video_pipeline_validate_branch(struct vsp1_pipeline *pipe,
- struct vsp1_rwpf *input,
- struct vsp1_rwpf *output)
-{
- struct vsp1_entity *entity;
- struct media_entity_enum ent_enum;
- struct media_pad *pad;
- int rval;
- bool bru_found = false;
-
- input->location.left = 0;
- input->location.top = 0;
-
- rval = media_entity_enum_init(
- &ent_enum, input->entity.pads[RWPF_PAD_SOURCE].graph_obj.mdev);
- if (rval)
- return rval;
-
- pad = media_entity_remote_pad(&input->entity.pads[RWPF_PAD_SOURCE]);
-
- while (1) {
- if (pad == NULL) {
- rval = -EPIPE;
- goto out;
- }
-
- /* We've reached a video node, that shouldn't have happened. */
- if (!is_media_entity_v4l2_subdev(pad->entity)) {
- rval = -EPIPE;
- goto out;
- }
-
- entity = to_vsp1_entity(
- media_entity_to_v4l2_subdev(pad->entity));
-
- /* A BRU is present in the pipeline, store the compose rectangle
- * location in the input RPF for use when configuring the RPF.
- */
- if (entity->type == VSP1_ENTITY_BRU) {
- struct vsp1_bru *bru = to_bru(&entity->subdev);
- struct v4l2_rect *rect =
- &bru->inputs[pad->index].compose;
-
- bru->inputs[pad->index].rpf = input;
-
- input->location.left = rect->left;
- input->location.top = rect->top;
-
- bru_found = true;
- }
-
- /* We've reached the WPF, we're done. */
- if (entity->type == VSP1_ENTITY_WPF)
- break;
-
- /* Ensure the branch has no loop. */
- if (media_entity_enum_test_and_set(&ent_enum,
- &entity->subdev.entity)) {
- rval = -EPIPE;
- goto out;
- }
-
- /* UDS can't be chained. */
- if (entity->type == VSP1_ENTITY_UDS) {
- if (pipe->uds) {
- rval = -EPIPE;
- goto out;
- }
-
- pipe->uds = entity;
- pipe->uds_input = bru_found ? pipe->bru
- : &input->entity;
- }
-
- /* Follow the source link. The link setup operations ensure
- * that the output fan-out can't be more than one, there is thus
- * no need to verify here that only a single source link is
- * activated.
- */
- pad = &entity->pads[entity->source_pad];
- pad = media_entity_remote_pad(pad);
- }
-
- /* The last entity must be the output WPF. */
- if (entity != &output->entity)
- rval = -EPIPE;
-
-out:
- media_entity_enum_cleanup(&ent_enum);
-
- return rval;
-}
-
-static int vsp1_video_pipeline_validate(struct vsp1_pipeline *pipe,
- struct vsp1_video *video)
-{
- struct media_entity_graph graph;
- struct media_entity *entity = &video->video.entity;
- struct media_device *mdev = entity->graph_obj.mdev;
- unsigned int i;
- int ret;
-
- mutex_lock(&mdev->graph_mutex);
-
- /* Walk the graph to locate the entities and video nodes. */
- ret = media_entity_graph_walk_init(&graph, mdev);
- if (ret) {
- mutex_unlock(&mdev->graph_mutex);
- return ret;
- }
-
- media_entity_graph_walk_start(&graph, entity);
-
- while ((entity = media_entity_graph_walk_next(&graph))) {
- struct v4l2_subdev *subdev;
- struct vsp1_rwpf *rwpf;
- struct vsp1_entity *e;
-
- if (!is_media_entity_v4l2_subdev(entity))
- continue;
-
- subdev = media_entity_to_v4l2_subdev(entity);
- e = to_vsp1_entity(subdev);
- list_add_tail(&e->list_pipe, &pipe->entities);
-
- if (e->type == VSP1_ENTITY_RPF) {
- rwpf = to_rwpf(subdev);
- pipe->inputs[rwpf->entity.index] = rwpf;
- rwpf->video->pipe_index = ++pipe->num_inputs;
- } else if (e->type == VSP1_ENTITY_WPF) {
- rwpf = to_rwpf(subdev);
- pipe->output = rwpf;
- rwpf->video->pipe_index = 0;
- } else if (e->type == VSP1_ENTITY_LIF) {
- pipe->lif = e;
- } else if (e->type == VSP1_ENTITY_BRU) {
- pipe->bru = e;
- }
- }
-
- mutex_unlock(&mdev->graph_mutex);
-
- media_entity_graph_walk_cleanup(&graph);
-
- /* We need one output and at least one input. */
- if (pipe->num_inputs == 0 || !pipe->output) {
- ret = -EPIPE;
- goto error;
- }
-
- /* Follow links downstream for each input and make sure the graph
- * contains no loop and that all branches end at the output WPF.
- */
- for (i = 0; i < video->vsp1->info->rpf_count; ++i) {
- if (!pipe->inputs[i])
- continue;
-
- ret = vsp1_video_pipeline_validate_branch(pipe, pipe->inputs[i],
- pipe->output);
- if (ret < 0)
- goto error;
- }
-
- return 0;
-
-error:
- vsp1_pipeline_reset(pipe);
- return ret;
-}
-
-static int vsp1_video_pipeline_init(struct vsp1_pipeline *pipe,
- struct vsp1_video *video)
-{
- int ret;
-
- mutex_lock(&pipe->lock);
-
- /* If we're the first user validate and initialize the pipeline. */
- if (pipe->use_count == 0) {
- ret = vsp1_video_pipeline_validate(pipe, video);
- if (ret < 0)
- goto done;
- }
-
- pipe->use_count++;
- ret = 0;
-
-done:
- mutex_unlock(&pipe->lock);
- return ret;
-}
-
-static void vsp1_video_pipeline_cleanup(struct vsp1_pipeline *pipe)
-{
- mutex_lock(&pipe->lock);
-
- /* If we're the last user clean up the pipeline. */
- if (--pipe->use_count == 0)
- vsp1_pipeline_reset(pipe);
-
- mutex_unlock(&pipe->lock);
-}
-
/*
* vsp1_video_complete_buffer - Complete the current buffer
* @video: the video node
@@ -391,7 +189,7 @@
static struct vsp1_vb2_buffer *
vsp1_video_complete_buffer(struct vsp1_video *video)
{
- struct vsp1_pipeline *pipe = to_vsp1_pipeline(&video->video.entity);
+ struct vsp1_pipeline *pipe = video->rwpf->pipe;
struct vsp1_vb2_buffer *next = NULL;
struct vsp1_vb2_buffer *done;
unsigned long flags;
@@ -425,7 +223,7 @@
done->buf.vb2_buf.timestamp = ktime_get_ns();
for (i = 0; i < done->buf.vb2_buf.num_planes; ++i)
vb2_set_plane_payload(&done->buf.vb2_buf, i,
- done->mem.length[i]);
+ vb2_plane_size(&done->buf.vb2_buf, i));
vb2_buffer_done(&done->buf.vb2_buf, VB2_BUF_STATE_DONE);
return next;
@@ -444,15 +242,41 @@
spin_lock_irqsave(&pipe->irqlock, flags);
- video->rwpf->ops->set_memory(video->rwpf, &buf->mem);
+ video->rwpf->mem = buf->mem;
pipe->buffers_ready |= 1 << video->pipe_index;
spin_unlock_irqrestore(&pipe->irqlock, flags);
}
+static void vsp1_video_pipeline_run(struct vsp1_pipeline *pipe)
+{
+ struct vsp1_device *vsp1 = pipe->output->entity.vsp1;
+ unsigned int i;
+
+ if (!pipe->dl)
+ pipe->dl = vsp1_dl_list_get(pipe->output->dlm);
+
+ for (i = 0; i < vsp1->info->rpf_count; ++i) {
+ struct vsp1_rwpf *rwpf = pipe->inputs[i];
+
+ if (rwpf)
+ vsp1_rwpf_set_memory(rwpf, pipe->dl);
+ }
+
+ if (!pipe->lif)
+ vsp1_rwpf_set_memory(pipe->output, pipe->dl);
+
+ vsp1_dl_list_commit(pipe->dl);
+ pipe->dl = NULL;
+
+ vsp1_pipeline_run(pipe);
+}
+
static void vsp1_video_pipeline_frame_end(struct vsp1_pipeline *pipe)
{
struct vsp1_device *vsp1 = pipe->output->entity.vsp1;
+ enum vsp1_pipeline_state state;
+ unsigned long flags;
unsigned int i;
/* Complete buffers on all video nodes. */
@@ -463,8 +287,230 @@
vsp1_video_frame_end(pipe, pipe->inputs[i]);
}
- if (!pipe->lif)
- vsp1_video_frame_end(pipe, pipe->output);
+ vsp1_video_frame_end(pipe, pipe->output);
+
+ spin_lock_irqsave(&pipe->irqlock, flags);
+
+ state = pipe->state;
+ pipe->state = VSP1_PIPELINE_STOPPED;
+
+ /* If a stop has been requested, mark the pipeline as stopped and
+ * return. Otherwise restart the pipeline if ready.
+ */
+ if (state == VSP1_PIPELINE_STOPPING)
+ wake_up(&pipe->wq);
+ else if (vsp1_pipeline_ready(pipe))
+ vsp1_video_pipeline_run(pipe);
+
+ spin_unlock_irqrestore(&pipe->irqlock, flags);
+}
+
+static int vsp1_video_pipeline_build_branch(struct vsp1_pipeline *pipe,
+ struct vsp1_rwpf *input,
+ struct vsp1_rwpf *output)
+{
+ struct media_entity_enum ent_enum;
+ struct vsp1_entity *entity;
+ struct media_pad *pad;
+ bool bru_found = false;
+ int ret;
+
+ ret = media_entity_enum_init(&ent_enum, &input->entity.vsp1->media_dev);
+ if (ret < 0)
+ return ret;
+
+ pad = media_entity_remote_pad(&input->entity.pads[RWPF_PAD_SOURCE]);
+
+ while (1) {
+ if (pad == NULL) {
+ ret = -EPIPE;
+ goto out;
+ }
+
+ /* We've reached a video node, that shouldn't have happened. */
+ if (!is_media_entity_v4l2_subdev(pad->entity)) {
+ ret = -EPIPE;
+ goto out;
+ }
+
+ entity = to_vsp1_entity(
+ media_entity_to_v4l2_subdev(pad->entity));
+
+ /* A BRU is present in the pipeline, store the BRU input pad
+ * number in the input RPF for use when configuring the RPF.
+ */
+ if (entity->type == VSP1_ENTITY_BRU) {
+ struct vsp1_bru *bru = to_bru(&entity->subdev);
+
+ bru->inputs[pad->index].rpf = input;
+ input->bru_input = pad->index;
+
+ bru_found = true;
+ }
+
+ /* We've reached the WPF, we're done. */
+ if (entity->type == VSP1_ENTITY_WPF)
+ break;
+
+ /* Ensure the branch has no loop. */
+ if (media_entity_enum_test_and_set(&ent_enum,
+ &entity->subdev.entity)) {
+ ret = -EPIPE;
+ goto out;
+ }
+
+ /* UDS can't be chained. */
+ if (entity->type == VSP1_ENTITY_UDS) {
+ if (pipe->uds) {
+ ret = -EPIPE;
+ goto out;
+ }
+
+ pipe->uds = entity;
+ pipe->uds_input = bru_found ? pipe->bru
+ : &input->entity;
+ }
+
+ /* Follow the source link. The link setup operations ensure
+ * that the output fan-out can't be more than one, there is thus
+ * no need to verify here that only a single source link is
+ * activated.
+ */
+ pad = &entity->pads[entity->source_pad];
+ pad = media_entity_remote_pad(pad);
+ }
+
+ /* The last entity must be the output WPF. */
+ if (entity != &output->entity)
+ ret = -EPIPE;
+
+out:
+ media_entity_enum_cleanup(&ent_enum);
+
+ return ret;
+}
+
+static int vsp1_video_pipeline_build(struct vsp1_pipeline *pipe,
+ struct vsp1_video *video)
+{
+ struct media_entity_graph graph;
+ struct media_entity *entity = &video->video.entity;
+ struct media_device *mdev = entity->graph_obj.mdev;
+ unsigned int i;
+ int ret;
+
+ /* Walk the graph to locate the entities and video nodes. */
+ ret = media_entity_graph_walk_init(&graph, mdev);
+ if (ret)
+ return ret;
+
+ media_entity_graph_walk_start(&graph, entity);
+
+ while ((entity = media_entity_graph_walk_next(&graph))) {
+ struct v4l2_subdev *subdev;
+ struct vsp1_rwpf *rwpf;
+ struct vsp1_entity *e;
+
+ if (!is_media_entity_v4l2_subdev(entity))
+ continue;
+
+ subdev = media_entity_to_v4l2_subdev(entity);
+ e = to_vsp1_entity(subdev);
+ list_add_tail(&e->list_pipe, &pipe->entities);
+
+ if (e->type == VSP1_ENTITY_RPF) {
+ rwpf = to_rwpf(subdev);
+ pipe->inputs[rwpf->entity.index] = rwpf;
+ rwpf->video->pipe_index = ++pipe->num_inputs;
+ rwpf->pipe = pipe;
+ } else if (e->type == VSP1_ENTITY_WPF) {
+ rwpf = to_rwpf(subdev);
+ pipe->output = rwpf;
+ rwpf->video->pipe_index = 0;
+ rwpf->pipe = pipe;
+ } else if (e->type == VSP1_ENTITY_LIF) {
+ pipe->lif = e;
+ } else if (e->type == VSP1_ENTITY_BRU) {
+ pipe->bru = e;
+ }
+ }
+
+ media_entity_graph_walk_cleanup(&graph);
+
+ /* We need one output and at least one input. */
+ if (pipe->num_inputs == 0 || !pipe->output)
+ return -EPIPE;
+
+ /* Follow links downstream for each input and make sure the graph
+ * contains no loop and that all branches end at the output WPF.
+ */
+ for (i = 0; i < video->vsp1->info->rpf_count; ++i) {
+ if (!pipe->inputs[i])
+ continue;
+
+ ret = vsp1_video_pipeline_build_branch(pipe, pipe->inputs[i],
+ pipe->output);
+ if (ret < 0)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int vsp1_video_pipeline_init(struct vsp1_pipeline *pipe,
+ struct vsp1_video *video)
+{
+ vsp1_pipeline_init(pipe);
+
+ pipe->frame_end = vsp1_video_pipeline_frame_end;
+
+ return vsp1_video_pipeline_build(pipe, video);
+}
+
+static struct vsp1_pipeline *vsp1_video_pipeline_get(struct vsp1_video *video)
+{
+ struct vsp1_pipeline *pipe;
+ int ret;
+
+ /* Get a pipeline object for the video node. If a pipeline has already
+ * been allocated just increment its reference count and return it.
+ * Otherwise allocate a new pipeline and initialize it, it will be freed
+ * when the last reference is released.
+ */
+ if (!video->rwpf->pipe) {
+ pipe = kzalloc(sizeof(*pipe), GFP_KERNEL);
+ if (!pipe)
+ return ERR_PTR(-ENOMEM);
+
+ ret = vsp1_video_pipeline_init(pipe, video);
+ if (ret < 0) {
+ vsp1_pipeline_reset(pipe);
+ kfree(pipe);
+ return ERR_PTR(ret);
+ }
+ } else {
+ pipe = video->rwpf->pipe;
+ kref_get(&pipe->kref);
+ }
+
+ return pipe;
+}
+
+static void vsp1_video_pipeline_release(struct kref *kref)
+{
+ struct vsp1_pipeline *pipe = container_of(kref, typeof(*pipe), kref);
+
+ vsp1_pipeline_reset(pipe);
+ kfree(pipe);
+}
+
+static void vsp1_video_pipeline_put(struct vsp1_pipeline *pipe)
+{
+ struct media_device *mdev = &pipe->output->entity.vsp1->media_dev;
+
+ mutex_lock(&mdev->graph_mutex);
+ kref_put(&pipe->kref, vsp1_video_pipeline_release);
+ mutex_unlock(&mdev->graph_mutex);
}
/* -----------------------------------------------------------------------------
@@ -513,16 +559,16 @@
if (vb->num_planes < format->num_planes)
return -EINVAL;
- buf->mem.num_planes = vb->num_planes;
-
for (i = 0; i < vb->num_planes; ++i) {
buf->mem.addr[i] = vb2_dma_contig_plane_dma_addr(vb, i);
- buf->mem.length[i] = vb2_plane_size(vb, i);
- if (buf->mem.length[i] < format->plane_fmt[i].sizeimage)
+ if (vb2_plane_size(vb, i) < format->plane_fmt[i].sizeimage)
return -EINVAL;
}
+ for ( ; i < 3; ++i)
+ buf->mem.addr[i] = 0;
+
return 0;
}
@@ -530,7 +576,7 @@
{
struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
struct vsp1_video *video = vb2_get_drv_priv(vb->vb2_queue);
- struct vsp1_pipeline *pipe = to_vsp1_pipeline(&video->video.entity);
+ struct vsp1_pipeline *pipe = video->rwpf->pipe;
struct vsp1_vb2_buffer *buf = to_vsp1_vb2_buffer(vbuf);
unsigned long flags;
bool empty;
@@ -545,54 +591,66 @@
spin_lock_irqsave(&pipe->irqlock, flags);
- video->rwpf->ops->set_memory(video->rwpf, &buf->mem);
+ video->rwpf->mem = buf->mem;
pipe->buffers_ready |= 1 << video->pipe_index;
if (vb2_is_streaming(&video->queue) &&
vsp1_pipeline_ready(pipe))
- vsp1_pipeline_run(pipe);
+ vsp1_video_pipeline_run(pipe);
spin_unlock_irqrestore(&pipe->irqlock, flags);
}
+static int vsp1_video_setup_pipeline(struct vsp1_pipeline *pipe)
+{
+ struct vsp1_entity *entity;
+
+ /* Prepare the display list. */
+ pipe->dl = vsp1_dl_list_get(pipe->output->dlm);
+ if (!pipe->dl)
+ return -ENOMEM;
+
+ if (pipe->uds) {
+ struct vsp1_uds *uds = to_uds(&pipe->uds->subdev);
+
+ /* If a BRU is present in the pipeline before the UDS, the alpha
+ * component doesn't need to be scaled as the BRU output alpha
+ * value is fixed to 255. Otherwise we need to scale the alpha
+ * component only when available at the input RPF.
+ */
+ if (pipe->uds_input->type == VSP1_ENTITY_BRU) {
+ uds->scale_alpha = false;
+ } else {
+ struct vsp1_rwpf *rpf =
+ to_rwpf(&pipe->uds_input->subdev);
+
+ uds->scale_alpha = rpf->fmtinfo->alpha;
+ }
+ }
+
+ list_for_each_entry(entity, &pipe->entities, list_pipe) {
+ vsp1_entity_route_setup(entity, pipe->dl);
+
+ if (entity->ops->configure)
+ entity->ops->configure(entity, pipe, pipe->dl);
+ }
+
+ return 0;
+}
+
static int vsp1_video_start_streaming(struct vb2_queue *vq, unsigned int count)
{
struct vsp1_video *video = vb2_get_drv_priv(vq);
- struct vsp1_pipeline *pipe = to_vsp1_pipeline(&video->video.entity);
- struct vsp1_entity *entity;
+ struct vsp1_pipeline *pipe = video->rwpf->pipe;
unsigned long flags;
int ret;
mutex_lock(&pipe->lock);
if (pipe->stream_count == pipe->num_inputs) {
- if (pipe->uds) {
- struct vsp1_uds *uds = to_uds(&pipe->uds->subdev);
-
- /* If a BRU is present in the pipeline before the UDS,
- * the alpha component doesn't need to be scaled as the
- * BRU output alpha value is fixed to 255. Otherwise we
- * need to scale the alpha component only when available
- * at the input RPF.
- */
- if (pipe->uds_input->type == VSP1_ENTITY_BRU) {
- uds->scale_alpha = false;
- } else {
- struct vsp1_rwpf *rpf =
- to_rwpf(&pipe->uds_input->subdev);
-
- uds->scale_alpha = rpf->fmtinfo->alpha;
- }
- }
-
- list_for_each_entry(entity, &pipe->entities, list_pipe) {
- vsp1_entity_route_setup(entity);
-
- ret = v4l2_subdev_call(&entity->subdev, video,
- s_stream, 1);
- if (ret < 0) {
- mutex_unlock(&pipe->lock);
- return ret;
- }
+ ret = vsp1_video_setup_pipeline(pipe);
+ if (ret < 0) {
+ mutex_unlock(&pipe->lock);
+ return ret;
}
}
@@ -601,7 +659,7 @@
spin_lock_irqsave(&pipe->irqlock, flags);
if (vsp1_pipeline_ready(pipe))
- vsp1_pipeline_run(pipe);
+ vsp1_video_pipeline_run(pipe);
spin_unlock_irqrestore(&pipe->irqlock, flags);
return 0;
@@ -610,7 +668,7 @@
static void vsp1_video_stop_streaming(struct vb2_queue *vq)
{
struct vsp1_video *video = vb2_get_drv_priv(vq);
- struct vsp1_pipeline *pipe = to_vsp1_pipeline(&video->video.entity);
+ struct vsp1_pipeline *pipe = video->rwpf->pipe;
struct vsp1_vb2_buffer *buffer;
unsigned long flags;
int ret;
@@ -621,11 +679,14 @@
ret = vsp1_pipeline_stop(pipe);
if (ret == -ETIMEDOUT)
dev_err(video->vsp1->dev, "pipeline stop timeout\n");
+
+ vsp1_dl_list_put(pipe->dl);
+ pipe->dl = NULL;
}
mutex_unlock(&pipe->lock);
- vsp1_video_pipeline_cleanup(pipe);
media_entity_pipeline_stop(&video->video.entity);
+ vsp1_video_pipeline_put(pipe);
/* Remove all buffers from the IRQ queue. */
spin_lock_irqsave(&video->irqlock, flags);
@@ -737,6 +798,7 @@
{
struct v4l2_fh *vfh = file->private_data;
struct vsp1_video *video = to_vsp1_video(vfh->vdev);
+ struct media_device *mdev = &video->vsp1->media_dev;
struct vsp1_pipeline *pipe;
int ret;
@@ -745,18 +807,25 @@
video->sequence = 0;
- /* Start streaming on the pipeline. No link touching an entity in the
- * pipeline can be activated or deactivated once streaming is started.
- *
- * Use the VSP1 pipeline object embedded in the first video object that
- * starts streaming.
+ /* Get a pipeline for the video node and start streaming on it. No link
+ * touching an entity in the pipeline can be activated or deactivated
+ * once streaming is started.
*/
- pipe = video->video.entity.pipe
- ? to_vsp1_pipeline(&video->video.entity) : &video->pipe;
+ mutex_lock(&mdev->graph_mutex);
- ret = media_entity_pipeline_start(&video->video.entity, &pipe->pipe);
- if (ret < 0)
- return ret;
+ pipe = vsp1_video_pipeline_get(video);
+ if (IS_ERR(pipe)) {
+ mutex_unlock(&mdev->graph_mutex);
+ return PTR_ERR(pipe);
+ }
+
+ ret = __media_entity_pipeline_start(&video->video.entity, &pipe->pipe);
+ if (ret < 0) {
+ mutex_unlock(&mdev->graph_mutex);
+ goto err_pipe;
+ }
+
+ mutex_unlock(&mdev->graph_mutex);
/* Verify that the configured format matches the output of the connected
* subdev.
@@ -765,21 +834,17 @@
if (ret < 0)
goto err_stop;
- ret = vsp1_video_pipeline_init(pipe, video);
- if (ret < 0)
- goto err_stop;
-
/* Start the queue. */
ret = vb2_streamon(&video->queue, type);
if (ret < 0)
- goto err_cleanup;
+ goto err_stop;
return 0;
-err_cleanup:
- vsp1_video_pipeline_cleanup(pipe);
err_stop:
media_entity_pipeline_stop(&video->video.entity);
+err_pipe:
+ vsp1_video_pipeline_put(pipe);
return ret;
}
@@ -895,26 +960,16 @@
spin_lock_init(&video->irqlock);
INIT_LIST_HEAD(&video->irqqueue);
- vsp1_pipeline_init(&video->pipe);
- video->pipe.frame_end = vsp1_video_pipeline_frame_end;
-
/* Initialize the media entity... */
ret = media_entity_pads_init(&video->video.entity, 1, &video->pad);
if (ret < 0)
return ERR_PTR(ret);
/* ... and the format ... */
- rwpf->fmtinfo = vsp1_get_format_info(VSP1_VIDEO_DEF_FORMAT);
- rwpf->format.pixelformat = rwpf->fmtinfo->fourcc;
- rwpf->format.colorspace = V4L2_COLORSPACE_SRGB;
- rwpf->format.field = V4L2_FIELD_NONE;
+ rwpf->format.pixelformat = VSP1_VIDEO_DEF_FORMAT;
rwpf->format.width = VSP1_VIDEO_DEF_WIDTH;
rwpf->format.height = VSP1_VIDEO_DEF_HEIGHT;
- rwpf->format.num_planes = 1;
- rwpf->format.plane_fmt[0].bytesperline =
- rwpf->format.width * rwpf->fmtinfo->bpp[0] / 8;
- rwpf->format.plane_fmt[0].sizeimage =
- rwpf->format.plane_fmt[0].bytesperline * rwpf->format.height;
+ __vsp1_video_try_format(video, &rwpf->format, &rwpf->fmtinfo);
/* ... and the video node... */
video->video.v4l2_dev = &video->vsp1->v4l2_dev;
diff --git a/drivers/media/platform/vsp1/vsp1_video.h b/drivers/media/platform/vsp1/vsp1_video.h
index 64abd39..867b008 100644
--- a/drivers/media/platform/vsp1/vsp1_video.h
+++ b/drivers/media/platform/vsp1/vsp1_video.h
@@ -18,7 +18,6 @@
#include <media/videobuf2-v4l2.h>
-#include "vsp1_pipe.h"
#include "vsp1_rwpf.h"
struct vsp1_vb2_buffer {
@@ -44,7 +43,6 @@
struct mutex lock;
- struct vsp1_pipeline pipe;
unsigned int pipe_index;
struct vb2_queue queue;
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c b/drivers/media/platform/vsp1/vsp1_wpf.c
index c78d4af..6c91eaa 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -16,83 +16,146 @@
#include <media/v4l2-subdev.h>
#include "vsp1.h"
+#include "vsp1_dl.h"
+#include "vsp1_pipe.h"
#include "vsp1_rwpf.h"
#include "vsp1_video.h"
-#define WPF_MAX_WIDTH 2048
-#define WPF_MAX_HEIGHT 2048
+#define WPF_GEN2_MAX_WIDTH 2048U
+#define WPF_GEN2_MAX_HEIGHT 2048U
+#define WPF_GEN3_MAX_WIDTH 8190U
+#define WPF_GEN3_MAX_HEIGHT 8190U
/* -----------------------------------------------------------------------------
* Device Access
*/
-static inline u32 vsp1_wpf_read(struct vsp1_rwpf *wpf, u32 reg)
+static inline void vsp1_wpf_write(struct vsp1_rwpf *wpf,
+ struct vsp1_dl_list *dl, u32 reg, u32 data)
{
- return vsp1_read(wpf->entity.vsp1,
- reg + wpf->entity.index * VI6_WPF_OFFSET);
+ vsp1_dl_list_write(dl, reg + wpf->entity.index * VI6_WPF_OFFSET, data);
}
-static inline void vsp1_wpf_write(struct vsp1_rwpf *wpf, u32 reg, u32 data)
-{
- vsp1_mod_write(&wpf->entity,
- reg + wpf->entity.index * VI6_WPF_OFFSET, data);
-}
-
-/* -----------------------------------------------------------------------------
- * Controls
- */
-
-static int wpf_s_ctrl(struct v4l2_ctrl *ctrl)
-{
- struct vsp1_rwpf *wpf =
- container_of(ctrl->handler, struct vsp1_rwpf, ctrls);
- u32 value;
-
- if (!vsp1_entity_is_streaming(&wpf->entity))
- return 0;
-
- switch (ctrl->id) {
- case V4L2_CID_ALPHA_COMPONENT:
- value = vsp1_wpf_read(wpf, VI6_WPF_OUTFMT);
- value &= ~VI6_WPF_OUTFMT_PDV_MASK;
- value |= ctrl->val << VI6_WPF_OUTFMT_PDV_SHIFT;
- vsp1_wpf_write(wpf, VI6_WPF_OUTFMT, value);
- break;
- }
-
- return 0;
-}
-
-static const struct v4l2_ctrl_ops wpf_ctrl_ops = {
- .s_ctrl = wpf_s_ctrl,
-};
-
/* -----------------------------------------------------------------------------
* V4L2 Subdevice Core Operations
*/
static int wpf_s_stream(struct v4l2_subdev *subdev, int enable)
{
- struct vsp1_pipeline *pipe = to_vsp1_pipeline(&subdev->entity);
struct vsp1_rwpf *wpf = to_rwpf(subdev);
struct vsp1_device *vsp1 = wpf->entity.vsp1;
- const struct v4l2_rect *crop = &wpf->crop;
- unsigned int i;
- u32 srcrpf = 0;
- u32 outfmt = 0;
- int ret;
- ret = vsp1_entity_set_streaming(&wpf->entity, enable);
- if (ret < 0)
- return ret;
-
- if (!enable) {
- vsp1_write(vsp1, VI6_WPF_IRQ_ENB(wpf->entity.index), 0);
- vsp1_write(vsp1, wpf->entity.index * VI6_WPF_OFFSET +
- VI6_WPF_SRCRPF, 0);
+ if (enable)
return 0;
+
+ /* Write to registers directly when stopping the stream as there will be
+ * no pipeline run to apply the display list.
+ */
+ vsp1_write(vsp1, VI6_WPF_IRQ_ENB(wpf->entity.index), 0);
+ vsp1_write(vsp1, wpf->entity.index * VI6_WPF_OFFSET +
+ VI6_WPF_SRCRPF, 0);
+
+ return 0;
+}
+
+/* -----------------------------------------------------------------------------
+ * V4L2 Subdevice Operations
+ */
+
+static struct v4l2_subdev_video_ops wpf_video_ops = {
+ .s_stream = wpf_s_stream,
+};
+
+static struct v4l2_subdev_ops wpf_ops = {
+ .video = &wpf_video_ops,
+ .pad = &vsp1_rwpf_pad_ops,
+};
+
+/* -----------------------------------------------------------------------------
+ * VSP1 Entity Operations
+ */
+
+static void vsp1_wpf_destroy(struct vsp1_entity *entity)
+{
+ struct vsp1_rwpf *wpf = entity_to_rwpf(entity);
+
+ vsp1_dlm_destroy(wpf->dlm);
+}
+
+static void wpf_set_memory(struct vsp1_entity *entity, struct vsp1_dl_list *dl)
+{
+ struct vsp1_rwpf *wpf = entity_to_rwpf(entity);
+
+ vsp1_wpf_write(wpf, dl, VI6_WPF_DSTM_ADDR_Y, wpf->mem.addr[0]);
+ vsp1_wpf_write(wpf, dl, VI6_WPF_DSTM_ADDR_C0, wpf->mem.addr[1]);
+ vsp1_wpf_write(wpf, dl, VI6_WPF_DSTM_ADDR_C1, wpf->mem.addr[2]);
+}
+
+static void wpf_configure(struct vsp1_entity *entity,
+ struct vsp1_pipeline *pipe,
+ struct vsp1_dl_list *dl)
+{
+ struct vsp1_rwpf *wpf = to_rwpf(&entity->subdev);
+ struct vsp1_device *vsp1 = wpf->entity.vsp1;
+ const struct v4l2_mbus_framefmt *source_format;
+ const struct v4l2_mbus_framefmt *sink_format;
+ const struct v4l2_rect *crop;
+ unsigned int i;
+ u32 outfmt = 0;
+ u32 srcrpf = 0;
+
+ /* Cropping */
+ crop = vsp1_rwpf_get_crop(wpf, wpf->entity.config);
+
+ vsp1_wpf_write(wpf, dl, VI6_WPF_HSZCLIP, VI6_WPF_SZCLIP_EN |
+ (crop->left << VI6_WPF_SZCLIP_OFST_SHIFT) |
+ (crop->width << VI6_WPF_SZCLIP_SIZE_SHIFT));
+ vsp1_wpf_write(wpf, dl, VI6_WPF_VSZCLIP, VI6_WPF_SZCLIP_EN |
+ (crop->top << VI6_WPF_SZCLIP_OFST_SHIFT) |
+ (crop->height << VI6_WPF_SZCLIP_SIZE_SHIFT));
+
+ /* Format */
+ sink_format = vsp1_entity_get_pad_format(&wpf->entity,
+ wpf->entity.config,
+ RWPF_PAD_SINK);
+ source_format = vsp1_entity_get_pad_format(&wpf->entity,
+ wpf->entity.config,
+ RWPF_PAD_SOURCE);
+
+ if (!pipe->lif) {
+ const struct v4l2_pix_format_mplane *format = &wpf->format;
+ const struct vsp1_format_info *fmtinfo = wpf->fmtinfo;
+
+ outfmt = fmtinfo->hwfmt << VI6_WPF_OUTFMT_WRFMT_SHIFT;
+
+ if (fmtinfo->alpha)
+ outfmt |= VI6_WPF_OUTFMT_PXA;
+ if (fmtinfo->swap_yc)
+ outfmt |= VI6_WPF_OUTFMT_SPYCS;
+ if (fmtinfo->swap_uv)
+ outfmt |= VI6_WPF_OUTFMT_SPUVS;
+
+ /* Destination stride and byte swapping. */
+ vsp1_wpf_write(wpf, dl, VI6_WPF_DSTM_STRIDE_Y,
+ format->plane_fmt[0].bytesperline);
+ if (format->num_planes > 1)
+ vsp1_wpf_write(wpf, dl, VI6_WPF_DSTM_STRIDE_C,
+ format->plane_fmt[1].bytesperline);
+
+ vsp1_wpf_write(wpf, dl, VI6_WPF_DSWAP, fmtinfo->swap);
}
+ if (sink_format->code != source_format->code)
+ outfmt |= VI6_WPF_OUTFMT_CSC;
+
+ outfmt |= wpf->alpha << VI6_WPF_OUTFMT_PDV_SHIFT;
+ vsp1_wpf_write(wpf, dl, VI6_WPF_OUTFMT, outfmt);
+
+ vsp1_dl_list_write(dl, VI6_DPR_WPF_FPORCH(wpf->entity.index),
+ VI6_DPR_WPF_FPORCH_FP_WPFN);
+
+ vsp1_dl_list_write(dl, VI6_WPF_WRBCK_CTRL, 0);
+
/* Sources. If the pipeline has a single input and BRU is not used,
* configure it as the master layer. Otherwise configure all
* inputs as sub-layers and select the virtual RPF as the master
@@ -112,106 +175,18 @@
if (pipe->bru || pipe->num_inputs > 1)
srcrpf |= VI6_WPF_SRCRPF_VIRACT_MST;
- vsp1_wpf_write(wpf, VI6_WPF_SRCRPF, srcrpf);
-
- /* Destination stride. */
- if (!pipe->lif) {
- struct v4l2_pix_format_mplane *format = &wpf->format;
-
- vsp1_wpf_write(wpf, VI6_WPF_DSTM_STRIDE_Y,
- format->plane_fmt[0].bytesperline);
- if (format->num_planes > 1)
- vsp1_wpf_write(wpf, VI6_WPF_DSTM_STRIDE_C,
- format->plane_fmt[1].bytesperline);
- }
-
- vsp1_wpf_write(wpf, VI6_WPF_HSZCLIP, VI6_WPF_SZCLIP_EN |
- (crop->left << VI6_WPF_SZCLIP_OFST_SHIFT) |
- (crop->width << VI6_WPF_SZCLIP_SIZE_SHIFT));
- vsp1_wpf_write(wpf, VI6_WPF_VSZCLIP, VI6_WPF_SZCLIP_EN |
- (crop->top << VI6_WPF_SZCLIP_OFST_SHIFT) |
- (crop->height << VI6_WPF_SZCLIP_SIZE_SHIFT));
-
- /* Format */
- if (!pipe->lif) {
- const struct vsp1_format_info *fmtinfo = wpf->fmtinfo;
-
- outfmt = fmtinfo->hwfmt << VI6_WPF_OUTFMT_WRFMT_SHIFT;
-
- if (fmtinfo->alpha)
- outfmt |= VI6_WPF_OUTFMT_PXA;
- if (fmtinfo->swap_yc)
- outfmt |= VI6_WPF_OUTFMT_SPYCS;
- if (fmtinfo->swap_uv)
- outfmt |= VI6_WPF_OUTFMT_SPUVS;
-
- vsp1_wpf_write(wpf, VI6_WPF_DSWAP, fmtinfo->swap);
- }
-
- if (wpf->entity.formats[RWPF_PAD_SINK].code !=
- wpf->entity.formats[RWPF_PAD_SOURCE].code)
- outfmt |= VI6_WPF_OUTFMT_CSC;
-
- /* Take the control handler lock to ensure that the PDV value won't be
- * changed behind our back by a set control operation.
- */
- if (vsp1->info->uapi)
- mutex_lock(wpf->ctrls.lock);
- outfmt |= wpf->alpha->cur.val << VI6_WPF_OUTFMT_PDV_SHIFT;
- vsp1_wpf_write(wpf, VI6_WPF_OUTFMT, outfmt);
- if (vsp1->info->uapi)
- mutex_unlock(wpf->ctrls.lock);
-
- vsp1_mod_write(&wpf->entity, VI6_DPR_WPF_FPORCH(wpf->entity.index),
- VI6_DPR_WPF_FPORCH_FP_WPFN);
-
- vsp1_mod_write(&wpf->entity, VI6_WPF_WRBCK_CTRL, 0);
+ vsp1_wpf_write(wpf, dl, VI6_WPF_SRCRPF, srcrpf);
/* Enable interrupts */
- vsp1_write(vsp1, VI6_WPF_IRQ_STA(wpf->entity.index), 0);
- vsp1_write(vsp1, VI6_WPF_IRQ_ENB(wpf->entity.index),
- VI6_WFP_IRQ_ENB_FREE);
-
- return 0;
+ vsp1_dl_list_write(dl, VI6_WPF_IRQ_STA(wpf->entity.index), 0);
+ vsp1_dl_list_write(dl, VI6_WPF_IRQ_ENB(wpf->entity.index),
+ VI6_WFP_IRQ_ENB_FREE);
}
-/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Operations
- */
-
-static struct v4l2_subdev_video_ops wpf_video_ops = {
- .s_stream = wpf_s_stream,
-};
-
-static struct v4l2_subdev_pad_ops wpf_pad_ops = {
- .enum_mbus_code = vsp1_rwpf_enum_mbus_code,
- .enum_frame_size = vsp1_rwpf_enum_frame_size,
- .get_fmt = vsp1_rwpf_get_format,
- .set_fmt = vsp1_rwpf_set_format,
- .get_selection = vsp1_rwpf_get_selection,
- .set_selection = vsp1_rwpf_set_selection,
-};
-
-static struct v4l2_subdev_ops wpf_ops = {
- .video = &wpf_video_ops,
- .pad = &wpf_pad_ops,
-};
-
-/* -----------------------------------------------------------------------------
- * Video Device Operations
- */
-
-static void wpf_set_memory(struct vsp1_rwpf *wpf, struct vsp1_rwpf_memory *mem)
-{
- vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_Y, mem->addr[0]);
- if (mem->num_planes > 1)
- vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_C0, mem->addr[1]);
- if (mem->num_planes > 2)
- vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_C1, mem->addr[2]);
-}
-
-static const struct vsp1_rwpf_operations wpf_vdev_ops = {
+static const struct vsp1_entity_operations wpf_entity_ops = {
+ .destroy = vsp1_wpf_destroy,
.set_memory = wpf_set_memory,
+ .configure = wpf_configure,
};
/* -----------------------------------------------------------------------------
@@ -220,51 +195,43 @@
struct vsp1_rwpf *vsp1_wpf_create(struct vsp1_device *vsp1, unsigned int index)
{
- struct v4l2_subdev *subdev;
struct vsp1_rwpf *wpf;
+ char name[6];
int ret;
wpf = devm_kzalloc(vsp1->dev, sizeof(*wpf), GFP_KERNEL);
if (wpf == NULL)
return ERR_PTR(-ENOMEM);
- wpf->ops = &wpf_vdev_ops;
+ if (vsp1->info->gen == 2) {
+ wpf->max_width = WPF_GEN2_MAX_WIDTH;
+ wpf->max_height = WPF_GEN2_MAX_HEIGHT;
+ } else {
+ wpf->max_width = WPF_GEN3_MAX_WIDTH;
+ wpf->max_height = WPF_GEN3_MAX_HEIGHT;
+ }
- wpf->max_width = WPF_MAX_WIDTH;
- wpf->max_height = WPF_MAX_HEIGHT;
-
+ wpf->entity.ops = &wpf_entity_ops;
wpf->entity.type = VSP1_ENTITY_WPF;
wpf->entity.index = index;
- ret = vsp1_entity_init(vsp1, &wpf->entity, 2);
+ sprintf(name, "wpf.%u", index);
+ ret = vsp1_entity_init(vsp1, &wpf->entity, name, 2, &wpf_ops);
if (ret < 0)
return ERR_PTR(ret);
- /* Initialize the V4L2 subdev. */
- subdev = &wpf->entity.subdev;
- v4l2_subdev_init(subdev, &wpf_ops);
-
- subdev->entity.ops = &vsp1->media_ops;
- subdev->internal_ops = &vsp1_subdev_internal_ops;
- snprintf(subdev->name, sizeof(subdev->name), "%s wpf.%u",
- dev_name(vsp1->dev), index);
- v4l2_set_subdevdata(subdev, wpf);
- subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
-
- vsp1_entity_init_formats(subdev, NULL);
+ /* Initialize the display list manager. */
+ wpf->dlm = vsp1_dlm_create(vsp1, index, 4);
+ if (!wpf->dlm) {
+ ret = -ENOMEM;
+ goto error;
+ }
/* Initialize the control handler. */
- v4l2_ctrl_handler_init(&wpf->ctrls, 1);
- wpf->alpha = v4l2_ctrl_new_std(&wpf->ctrls, &wpf_ctrl_ops,
- V4L2_CID_ALPHA_COMPONENT,
- 0, 255, 1, 255);
-
- wpf->entity.subdev.ctrl_handler = &wpf->ctrls;
-
- if (wpf->ctrls.error) {
+ ret = vsp1_rwpf_init_ctrls(wpf);
+ if (ret < 0) {
dev_err(vsp1->dev, "wpf%u: failed to initialize controls\n",
index);
- ret = wpf->ctrls.error;
goto error;
}
diff --git a/drivers/media/platform/xilinx/xilinx-vipp.c b/drivers/media/platform/xilinx/xilinx-vipp.c
index e795a45..feb3b2f 100644
--- a/drivers/media/platform/xilinx/xilinx-vipp.c
+++ b/drivers/media/platform/xilinx/xilinx-vipp.c
@@ -351,19 +351,15 @@
struct xvip_graph_entity *entity;
struct device_node *remote;
struct device_node *ep = NULL;
- struct device_node *next;
int ret = 0;
dev_dbg(xdev->dev, "parsing node %s\n", node->full_name);
while (1) {
- next = of_graph_get_next_endpoint(node, ep);
- if (next == NULL)
+ ep = of_graph_get_next_endpoint(node, ep);
+ if (ep == NULL)
break;
- of_node_put(ep);
- ep = next;
-
dev_dbg(xdev->dev, "handling endpoint %s\n", ep->full_name);
remote = of_graph_get_remote_port_parent(ep);
diff --git a/drivers/media/rc/ati_remote.c b/drivers/media/rc/ati_remote.c
index 3f61d77..9f5b597 100644
--- a/drivers/media/rc/ati_remote.c
+++ b/drivers/media/rc/ati_remote.c
@@ -873,13 +873,10 @@
strlcat(ati_remote->rc_phys, "/input0", sizeof(ati_remote->rc_phys));
strlcat(ati_remote->mouse_phys, "/input1", sizeof(ati_remote->mouse_phys));
- if (udev->manufacturer)
- strlcpy(ati_remote->rc_name, udev->manufacturer,
- sizeof(ati_remote->rc_name));
-
- if (udev->product)
- snprintf(ati_remote->rc_name, sizeof(ati_remote->rc_name),
- "%s %s", ati_remote->rc_name, udev->product);
+ snprintf(ati_remote->rc_name, sizeof(ati_remote->rc_name), "%s%s%s",
+ udev->manufacturer ?: "",
+ udev->manufacturer && udev->product ? " " : "",
+ udev->product ?: "");
if (!strlen(ati_remote->rc_name))
snprintf(ati_remote->rc_name, sizeof(ati_remote->rc_name),
diff --git a/drivers/media/rc/mceusb.c b/drivers/media/rc/mceusb.c
index 35155ae..5cf2e74 100644
--- a/drivers/media/rc/mceusb.c
+++ b/drivers/media/rc/mceusb.c
@@ -188,6 +188,7 @@
#define VENDOR_TWISTEDMELON 0x2596
#define VENDOR_HAUPPAUGE 0x2040
#define VENDOR_PCTV 0x2013
+#define VENDOR_ADAPTEC 0x03f3
enum mceusb_model_type {
MCE_GEN2 = 0, /* Most boards */
@@ -302,6 +303,9 @@
/* SMK/I-O Data GV-MC7/RCKIT Receiver */
{ USB_DEVICE(VENDOR_SMK, 0x0353),
.driver_info = MCE_GEN2_NO_TX },
+ /* SMK RXX6000 Infrared Receiver */
+ { USB_DEVICE(VENDOR_SMK, 0x0357),
+ .driver_info = MCE_GEN2_NO_TX },
/* Tatung eHome Infrared Transceiver */
{ USB_DEVICE(VENDOR_TATUNG, 0x9150) },
/* Shuttle eHome Infrared Transceiver */
@@ -405,6 +409,8 @@
.driver_info = HAUPPAUGE_CX_HYBRID_TV },
{ USB_DEVICE(VENDOR_PCTV, 0x025e),
.driver_info = HAUPPAUGE_CX_HYBRID_TV },
+ /* Adaptec / HP eHome Receiver */
+ { USB_DEVICE(VENDOR_ADAPTEC, 0x0094) },
/* Terminating entry */
{ }
diff --git a/drivers/media/rc/rc-main.c b/drivers/media/rc/rc-main.c
index 4e9bbe7..7dfc7c2 100644
--- a/drivers/media/rc/rc-main.c
+++ b/drivers/media/rc/rc-main.c
@@ -1263,6 +1263,9 @@
static void rc_dev_release(struct device *device)
{
+ struct rc_dev *dev = to_rc_dev(device);
+
+ kfree(dev);
}
#define ADD_HOTPLUG_VAR(fmt, val...) \
@@ -1384,7 +1387,9 @@
put_device(&dev->dev);
- kfree(dev);
+ /* kfree(dev) will be called by the callback function
+ rc_dev_release() */
+
module_put(THIS_MODULE);
}
EXPORT_SYMBOL_GPL(rc_free_device);
@@ -1492,9 +1497,7 @@
}
/* Allow the RC sysfs nodes to be accessible */
- mutex_lock(&dev->lock);
atomic_set(&dev->initialized, 1);
- mutex_unlock(&dev->lock);
IR_dprintk(1, "Registered rc%u (driver: %s, remote: %s, mode %s)\n",
dev->minor,
diff --git a/drivers/media/tuners/qm1d1c0042.c b/drivers/media/tuners/qm1d1c0042.c
index 18bc745..9af2a15 100644
--- a/drivers/media/tuners/qm1d1c0042.c
+++ b/drivers/media/tuners/qm1d1c0042.c
@@ -32,14 +32,24 @@
#include "qm1d1c0042.h"
#define QM1D1C0042_NUM_REGS 0x20
+#define QM1D1C0042_NUM_REG_ROWS 2
-static const u8 reg_initval[QM1D1C0042_NUM_REGS] = {
- 0x48, 0x1c, 0xa0, 0x10, 0xbc, 0xc5, 0x20, 0x33,
- 0x06, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00,
- 0x00, 0xff, 0xf3, 0x00, 0x2a, 0x64, 0xa6, 0x86,
- 0x8c, 0xcf, 0xb8, 0xf1, 0xa8, 0xf2, 0x89, 0x00
+static const u8
+reg_initval[QM1D1C0042_NUM_REG_ROWS][QM1D1C0042_NUM_REGS] = { {
+ 0x48, 0x1c, 0xa0, 0x10, 0xbc, 0xc5, 0x20, 0x33,
+ 0x06, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00,
+ 0x00, 0xff, 0xf3, 0x00, 0x2a, 0x64, 0xa6, 0x86,
+ 0x8c, 0xcf, 0xb8, 0xf1, 0xa8, 0xf2, 0x89, 0x00
+ }, {
+ 0x68, 0x1c, 0xc0, 0x10, 0xbc, 0xc1, 0x11, 0x33,
+ 0x03, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00,
+ 0x00, 0xff, 0xf3, 0x00, 0x3f, 0x25, 0x5c, 0xd6,
+ 0x55, 0xcf, 0x95, 0xf6, 0x36, 0xf2, 0x09, 0x00
+ }
};
+static int reg_index;
+
static const struct qm1d1c0042_config default_cfg = {
.xtal_freq = 16000,
.lpf = 1,
@@ -320,7 +330,6 @@
int i, ret;
state = fe->tuner_priv;
- memcpy(state->regs, reg_initval, sizeof(reg_initval));
reg_write(state, 0x01, 0x0c);
reg_write(state, 0x01, 0x0c);
@@ -330,15 +339,22 @@
goto failed;
usleep_range(2000, 3000);
- val = state->regs[0x01] | 0x10;
- ret = reg_write(state, 0x01, val); /* soft reset off */
+ ret = reg_write(state, 0x01, 0x1c); /* soft reset off */
if (ret < 0)
goto failed;
- /* check ID */
+ /* check ID and choose initial registers corresponding ID */
ret = reg_read(state, 0x00, &val);
- if (ret < 0 || val != 0x48)
+ if (ret < 0)
goto failed;
+ for (reg_index = 0; reg_index < QM1D1C0042_NUM_REG_ROWS;
+ reg_index++) {
+ if (val == reg_initval[reg_index][0x00])
+ break;
+ }
+ if (reg_index >= QM1D1C0042_NUM_REG_ROWS)
+ goto failed;
+ memcpy(state->regs, reg_initval[reg_index], QM1D1C0042_NUM_REGS);
usleep_range(2000, 3000);
state->regs[0x0c] |= 0x40;
diff --git a/drivers/media/tuners/si2157.c b/drivers/media/tuners/si2157.c
index 243ac38..b07a681 100644
--- a/drivers/media/tuners/si2157.c
+++ b/drivers/media/tuners/si2157.c
@@ -84,11 +84,22 @@
struct si2157_cmd cmd;
const struct firmware *fw;
const char *fw_name;
- unsigned int chip_id;
+ unsigned int uitmp, chip_id;
dev_dbg(&client->dev, "\n");
- if (dev->fw_loaded)
+ /* Returned IF frequency is garbage when firmware is not running */
+ memcpy(cmd.args, "\x15\x00\x06\x07", 4);
+ cmd.wlen = 4;
+ cmd.rlen = 4;
+ ret = si2157_cmd_execute(client, &cmd);
+ if (ret)
+ goto err;
+
+ uitmp = cmd.args[2] << 0 | cmd.args[3] << 8;
+ dev_dbg(&client->dev, "if_frequency kHz=%u\n", uitmp);
+
+ if (uitmp == dev->if_frequency / 1000)
goto warm;
/* power up */
@@ -203,9 +214,6 @@
dev_info(&client->dev, "firmware version: %c.%c.%d\n",
cmd.args[6], cmd.args[7], cmd.args[8]);
-
- dev->fw_loaded = true;
-
warm:
/* init statistics in order signal app which are supported */
c->strength.len = 1;
@@ -422,7 +430,6 @@
dev->fe = cfg->fe;
dev->inversion = cfg->inversion;
dev->if_port = cfg->if_port;
- dev->fw_loaded = false;
dev->chiptype = (u8)id->driver_data;
dev->if_frequency = 5000000; /* default value of property 0x0706 */
mutex_init(&dev->i2c_mutex);
diff --git a/drivers/media/tuners/si2157_priv.h b/drivers/media/tuners/si2157_priv.h
index 589d558..d6b2c7b 100644
--- a/drivers/media/tuners/si2157_priv.h
+++ b/drivers/media/tuners/si2157_priv.h
@@ -26,7 +26,6 @@
struct mutex i2c_mutex;
struct dvb_frontend *fe;
bool active;
- bool fw_loaded;
bool inversion;
u8 chiptype;
u8 if_port;
diff --git a/drivers/media/usb/au0828/au0828-core.c b/drivers/media/usb/au0828/au0828-core.c
index cc22b32..321ea5c 100644
--- a/drivers/media/usb/au0828/au0828-core.c
+++ b/drivers/media/usb/au0828/au0828-core.c
@@ -131,22 +131,36 @@
return status;
}
+#ifdef CONFIG_MEDIA_CONTROLLER
+static void au0828_media_graph_notify(struct media_entity *new,
+ void *notify_data);
+#endif
+
static void au0828_unregister_media_device(struct au0828_dev *dev)
{
-
#ifdef CONFIG_MEDIA_CONTROLLER
- if (dev->media_dev &&
- media_devnode_is_registered(&dev->media_dev->devnode)) {
- /* clear enable_source, disable_source */
- dev->media_dev->source_priv = NULL;
- dev->media_dev->enable_source = NULL;
- dev->media_dev->disable_source = NULL;
+ struct media_device *mdev = dev->media_dev;
+ struct media_entity_notify *notify, *nextp;
- media_device_unregister(dev->media_dev);
- media_device_cleanup(dev->media_dev);
- kfree(dev->media_dev);
- dev->media_dev = NULL;
+ if (!mdev || !media_devnode_is_registered(&mdev->devnode))
+ return;
+
+ /* Remove au0828 entity_notify callbacks */
+ list_for_each_entry_safe(notify, nextp, &mdev->entity_notify, list) {
+ if (notify->notify != au0828_media_graph_notify)
+ continue;
+ media_device_unregister_entity_notify(mdev, notify);
}
+
+ /* clear enable_source, disable_source */
+ dev->media_dev->source_priv = NULL;
+ dev->media_dev->enable_source = NULL;
+ dev->media_dev->disable_source = NULL;
+
+ media_device_unregister(dev->media_dev);
+ media_device_cleanup(dev->media_dev);
+ kfree(dev->media_dev);
+ dev->media_dev = NULL;
#endif
}
diff --git a/drivers/media/usb/au0828/au0828-video.c b/drivers/media/usb/au0828/au0828-video.c
index 32d7db9..7d0ec4c 100644
--- a/drivers/media/usb/au0828/au0828-video.c
+++ b/drivers/media/usb/au0828/au0828-video.c
@@ -679,8 +679,6 @@
if (retval) {
pr_err("%s() v4l2_device_register failed\n",
__func__);
- mutex_unlock(&dev->lock);
- kfree(dev);
return retval;
}
@@ -691,8 +689,6 @@
if (retval) {
pr_err("%s() v4l2_ctrl_handler_init failed\n",
__func__);
- mutex_unlock(&dev->lock);
- kfree(dev);
return retval;
}
dev->v4l2_dev.ctrl_handler = &dev->v4l2_ctrl_hdl;
diff --git a/drivers/media/usb/au0828/au0828.h b/drivers/media/usb/au0828/au0828.h
index 87f3284..dd7b378 100644
--- a/drivers/media/usb/au0828/au0828.h
+++ b/drivers/media/usb/au0828/au0828.h
@@ -55,7 +55,6 @@
#define NTSC_STD_H 480
#define AU0828_INTERLACED_DEFAULT 1
-#define V4L2_CID_PRIVATE_SHARPNESS (V4L2_CID_PRIVATE_BASE + 0)
/* Defination for AU0828 USB transfer */
#define AU0828_MAX_ISO_BUFS 12 /* maybe resize this value in the future */
diff --git a/drivers/media/usb/cx231xx/cx231xx-417.c b/drivers/media/usb/cx231xx/cx231xx-417.c
index c9320d6..00da024 100644
--- a/drivers/media/usb/cx231xx/cx231xx-417.c
+++ b/drivers/media/usb/cx231xx/cx231xx-417.c
@@ -360,7 +360,7 @@
if (count++ > 100) {
dprintk(3, "ERROR: Timeout - gpio=%x\n", gpio);
- return -1;
+ return -EIO;
}
}
return 0;
@@ -856,7 +856,7 @@
}
}
dprintk(3, "Mailbox signature values not found!\n");
- return -1;
+ return -EIO;
}
static void mci_write_memory_to_gpio(struct cx231xx *dev, u32 address, u32 value,
@@ -960,13 +960,14 @@
p_fw = p_current_fw;
if (p_current_fw == NULL) {
dprintk(2, "FAIL!!!\n");
- return -1;
+ return -ENOMEM;
}
p_buffer = vmalloc(4096);
if (p_buffer == NULL) {
dprintk(2, "FAIL!!!\n");
- return -1;
+ vfree(p_current_fw);
+ return -ENOMEM;
}
dprintk(2, "%s()\n", __func__);
@@ -989,7 +990,9 @@
if (retval != 0) {
dev_err(dev->dev,
"%s: Error with mc417_register_write\n", __func__);
- return -1;
+ vfree(p_current_fw);
+ vfree(p_buffer);
+ return retval;
}
retval = request_firmware(&firmware, CX231xx_FIRM_IMAGE_NAME,
@@ -1001,7 +1004,9 @@
CX231xx_FIRM_IMAGE_NAME);
dev_err(dev->dev,
"Please fix your hotplug setup, the board will not work without firmware loaded!\n");
- return -1;
+ vfree(p_current_fw);
+ vfree(p_buffer);
+ return retval;
}
if (firmware->size != CX231xx_FIRM_IMAGE_SIZE) {
@@ -1009,14 +1014,18 @@
"ERROR: Firmware size mismatch (have %zd, expected %d)\n",
firmware->size, CX231xx_FIRM_IMAGE_SIZE);
release_firmware(firmware);
- return -1;
+ vfree(p_current_fw);
+ vfree(p_buffer);
+ return -EINVAL;
}
if (0 != memcmp(firmware->data, magic, 8)) {
dev_err(dev->dev,
"ERROR: Firmware magic mismatch, wrong file?\n");
release_firmware(firmware);
- return -1;
+ vfree(p_current_fw);
+ vfree(p_buffer);
+ return -EINVAL;
}
initGPIO(dev);
@@ -1131,21 +1140,21 @@
if (retval < 0) {
dev_err(dev->dev, "%s: mailbox < 0, error\n",
__func__);
- return -1;
+ return retval;
}
dev->cx23417_mailbox = retval;
retval = cx231xx_api_cmd(dev, CX2341X_ENC_PING_FW, 0, 0);
if (retval < 0) {
dev_err(dev->dev,
"ERROR: cx23417 firmware ping failed!\n");
- return -1;
+ return retval;
}
retval = cx231xx_api_cmd(dev, CX2341X_ENC_GET_VERSION, 0, 1,
&version);
if (retval < 0) {
dev_err(dev->dev,
"ERROR: cx23417 firmware get encoder: version failed!\n");
- return -1;
+ return retval;
}
dprintk(1, "cx23417 firmware version is 0x%08x\n", version);
msleep(200);
diff --git a/drivers/media/usb/cx231xx/cx231xx-core.c b/drivers/media/usb/cx231xx/cx231xx-core.c
index f497888..630f4fc 100644
--- a/drivers/media/usb/cx231xx/cx231xx-core.c
+++ b/drivers/media/usb/cx231xx/cx231xx-core.c
@@ -752,7 +752,8 @@
int cx231xx_ep5_bulkout(struct cx231xx *dev, u8 *firmware, u16 size)
{
int errCode = 0;
- int actlen, ret = -ENOMEM;
+ int actlen = -1;
+ int ret = -ENOMEM;
u32 *buffer;
buffer = kzalloc(4096, GFP_KERNEL);
@@ -1304,6 +1305,9 @@
cx231xx_i2c_register(&dev->i2c_bus[1]);
cx231xx_i2c_register(&dev->i2c_bus[2]);
+ errCode = cx231xx_i2c_mux_create(dev);
+ if (errCode < 0)
+ return errCode;
cx231xx_i2c_mux_register(dev, 0);
cx231xx_i2c_mux_register(dev, 1);
@@ -1426,8 +1430,7 @@
void cx231xx_dev_uninit(struct cx231xx *dev)
{
/* Un Initialize I2C bus */
- cx231xx_i2c_mux_unregister(dev, 1);
- cx231xx_i2c_mux_unregister(dev, 0);
+ cx231xx_i2c_mux_unregister(dev);
cx231xx_i2c_unregister(&dev->i2c_bus[2]);
cx231xx_i2c_unregister(&dev->i2c_bus[1]);
cx231xx_i2c_unregister(&dev->i2c_bus[0]);
diff --git a/drivers/media/usb/cx231xx/cx231xx-i2c.c b/drivers/media/usb/cx231xx/cx231xx-i2c.c
index a29c345..473cd34 100644
--- a/drivers/media/usb/cx231xx/cx231xx-i2c.c
+++ b/drivers/media/usb/cx231xx/cx231xx-i2c.c
@@ -557,40 +557,41 @@
* cx231xx_i2c_mux_select()
* switch i2c master number 1 between port1 and port3
*/
-static int cx231xx_i2c_mux_select(struct i2c_adapter *adap,
- void *mux_priv, u32 chan_id)
+static int cx231xx_i2c_mux_select(struct i2c_mux_core *muxc, u32 chan_id)
{
- struct cx231xx *dev = mux_priv;
+ struct cx231xx *dev = i2c_mux_priv(muxc);
return cx231xx_enable_i2c_port_3(dev, chan_id);
}
-int cx231xx_i2c_mux_register(struct cx231xx *dev, int mux_no)
+int cx231xx_i2c_mux_create(struct cx231xx *dev)
{
- struct i2c_adapter *i2c_parent = &dev->i2c_bus[1].i2c_adap;
- /* what is the correct mux_dev? */
- struct device *mux_dev = dev->dev;
-
- dev->i2c_mux_adap[mux_no] = i2c_add_mux_adapter(i2c_parent,
- mux_dev,
- dev /* mux_priv */,
- 0,
- mux_no /* chan_id */,
- 0 /* class */,
- &cx231xx_i2c_mux_select,
- NULL);
-
- if (!dev->i2c_mux_adap[mux_no])
- dev_warn(dev->dev,
- "i2c mux %d register FAILED\n", mux_no);
-
+ dev->muxc = i2c_mux_alloc(&dev->i2c_bus[1].i2c_adap, dev->dev, 2, 0, 0,
+ cx231xx_i2c_mux_select, NULL);
+ if (!dev->muxc)
+ return -ENOMEM;
+ dev->muxc->priv = dev;
return 0;
}
-void cx231xx_i2c_mux_unregister(struct cx231xx *dev, int mux_no)
+int cx231xx_i2c_mux_register(struct cx231xx *dev, int mux_no)
{
- i2c_del_mux_adapter(dev->i2c_mux_adap[mux_no]);
- dev->i2c_mux_adap[mux_no] = NULL;
+ int rc;
+
+ rc = i2c_mux_add_adapter(dev->muxc,
+ 0,
+ mux_no /* chan_id */,
+ 0 /* class */);
+ if (rc)
+ dev_warn(dev->dev,
+ "i2c mux %d register FAILED\n", mux_no);
+
+ return rc;
+}
+
+void cx231xx_i2c_mux_unregister(struct cx231xx *dev)
+{
+ i2c_mux_del_adapters(dev->muxc);
}
struct i2c_adapter *cx231xx_get_i2c_adap(struct cx231xx *dev, int i2c_port)
@@ -603,9 +604,9 @@
case I2C_2:
return &dev->i2c_bus[2].i2c_adap;
case I2C_1_MUX_1:
- return dev->i2c_mux_adap[0];
+ return dev->muxc->adapter[0];
case I2C_1_MUX_3:
- return dev->i2c_mux_adap[1];
+ return dev->muxc->adapter[1];
default:
return NULL;
}
diff --git a/drivers/media/usb/cx231xx/cx231xx.h b/drivers/media/usb/cx231xx/cx231xx.h
index 69f6d20..90c8676 100644
--- a/drivers/media/usb/cx231xx/cx231xx.h
+++ b/drivers/media/usb/cx231xx/cx231xx.h
@@ -624,6 +624,7 @@
/* I2C adapters: Master 1 & 2 (External) & Master 3 (Internal only) */
struct cx231xx_i2c i2c_bus[3];
+ struct i2c_mux_core *muxc;
struct i2c_adapter *i2c_mux_adap[2];
unsigned int xc_fw_load_done:1;
@@ -760,8 +761,9 @@
void cx231xx_do_i2c_scan(struct cx231xx *dev, int i2c_port);
int cx231xx_i2c_register(struct cx231xx_i2c *bus);
int cx231xx_i2c_unregister(struct cx231xx_i2c *bus);
+int cx231xx_i2c_mux_create(struct cx231xx *dev);
int cx231xx_i2c_mux_register(struct cx231xx *dev, int mux_no);
-void cx231xx_i2c_mux_unregister(struct cx231xx *dev, int mux_no);
+void cx231xx_i2c_mux_unregister(struct cx231xx *dev);
struct i2c_adapter *cx231xx_get_i2c_adap(struct cx231xx *dev, int i2c_port);
/* Internal block control functions */
diff --git a/drivers/media/usb/dvb-usb-v2/af9035.h b/drivers/media/usb/dvb-usb-v2/af9035.h
index df22001..89e629a 100644
--- a/drivers/media/usb/dvb-usb-v2/af9035.h
+++ b/drivers/media/usb/dvb-usb-v2/af9035.h
@@ -118,20 +118,20 @@
* Values 0, 3 and 5 are seen to this day. 0 for single TS and 3/5 for dual TS.
*/
-#define EEPROM_BASE_AF9035 0x42fd
-#define EEPROM_BASE_IT9135 0x499c
+#define EEPROM_BASE_AF9035 0x42f5
+#define EEPROM_BASE_IT9135 0x4994
#define EEPROM_SHIFT 0x10
-#define EEPROM_IR_MODE 0x10
-#define EEPROM_TS_MODE 0x29
-#define EEPROM_2ND_DEMOD_ADDR 0x2a
-#define EEPROM_IR_TYPE 0x2c
-#define EEPROM_1_IF_L 0x30
-#define EEPROM_1_IF_H 0x31
-#define EEPROM_1_TUNER_ID 0x34
-#define EEPROM_2_IF_L 0x40
-#define EEPROM_2_IF_H 0x41
-#define EEPROM_2_TUNER_ID 0x44
+#define EEPROM_IR_MODE 0x18
+#define EEPROM_TS_MODE 0x31
+#define EEPROM_2ND_DEMOD_ADDR 0x32
+#define EEPROM_IR_TYPE 0x34
+#define EEPROM_1_IF_L 0x38
+#define EEPROM_1_IF_H 0x39
+#define EEPROM_1_TUNER_ID 0x3c
+#define EEPROM_2_IF_L 0x48
+#define EEPROM_2_IF_H 0x49
+#define EEPROM_2_TUNER_ID 0x4c
/* USB commands */
#define CMD_MEM_RD 0x00
diff --git a/drivers/media/usb/dvb-usb-v2/rtl28xxu.c b/drivers/media/usb/dvb-usb-v2/rtl28xxu.c
index fa72642..eb7af8c 100644
--- a/drivers/media/usb/dvb-usb-v2/rtl28xxu.c
+++ b/drivers/media/usb/dvb-usb-v2/rtl28xxu.c
@@ -1333,10 +1333,7 @@
case TUNER_RTL2832_R828D:
pdata.clk = dev->rtl2832_platform_data.clk;
pdata.tuner = dev->tuner;
- pdata.i2c_client = dev->i2c_client_demod;
- pdata.bulk_read = dev->rtl2832_platform_data.bulk_read;
- pdata.bulk_write = dev->rtl2832_platform_data.bulk_write;
- pdata.update_bits = dev->rtl2832_platform_data.update_bits;
+ pdata.regmap = dev->rtl2832_platform_data.regmap;
pdata.dvb_frontend = adap->fe[0];
pdata.dvb_usb_device = d;
pdata.v4l2_subdev = subdev;
diff --git a/drivers/media/usb/dvb-usb/az6027.c b/drivers/media/usb/dvb-usb/az6027.c
index 92e47d6..2e71136 100644
--- a/drivers/media/usb/dvb-usb/az6027.c
+++ b/drivers/media/usb/dvb-usb/az6027.c
@@ -1090,6 +1090,7 @@
{ USB_DEVICE(USB_VID_TECHNISAT, USB_PID_TECHNISAT_USB2_HDCI_V2) },
{ USB_DEVICE(USB_VID_ELGATO, USB_PID_ELGATO_EYETV_SAT) },
{ USB_DEVICE(USB_VID_ELGATO, USB_PID_ELGATO_EYETV_SAT_V2) },
+ { USB_DEVICE(USB_VID_ELGATO, USB_PID_ELGATO_EYETV_SAT_V3) },
{ },
};
@@ -1138,7 +1139,7 @@
.i2c_algo = &az6027_i2c_algo,
- .num_device_descs = 7,
+ .num_device_descs = 8,
.devices = {
{
.name = "AZUREWAVE DVB-S/S2 USB2.0 (AZ6027)",
@@ -1168,6 +1169,10 @@
.name = "Elgato EyeTV Sat",
.cold_ids = { &az6027_usb_table[6], NULL },
.warm_ids = { NULL },
+ }, {
+ .name = "Elgato EyeTV Sat",
+ .cold_ids = { &az6027_usb_table[7], NULL },
+ .warm_ids = { NULL },
},
{ NULL },
}
diff --git a/drivers/media/usb/dvb-usb/dib0700_core.c b/drivers/media/usb/dvb-usb/dib0700_core.c
index c16f999..bf890c3 100644
--- a/drivers/media/usb/dvb-usb/dib0700_core.c
+++ b/drivers/media/usb/dvb-usb/dib0700_core.c
@@ -517,7 +517,7 @@
if (nb_packet_buffer_size < 1)
nb_packet_buffer_size = 1;
- /* get the fimware version */
+ /* get the firmware version */
usb_control_msg(udev, usb_rcvctrlpipe(udev, 0),
REQUEST_GET_VERSION,
USB_TYPE_VENDOR | USB_DIR_IN, 0, 0,
diff --git a/drivers/media/usb/dvb-usb/dib0700_devices.c b/drivers/media/usb/dvb-usb/dib0700_devices.c
index ea0391e..0857b56 100644
--- a/drivers/media/usb/dvb-usb/dib0700_devices.c
+++ b/drivers/media/usb/dvb-usb/dib0700_devices.c
@@ -3814,6 +3814,7 @@
{ USB_DEVICE(USB_VID_PCTV, USB_PID_PCTV_2002E) },
{ USB_DEVICE(USB_VID_PCTV, USB_PID_PCTV_2002E_SE) },
{ USB_DEVICE(USB_VID_PCTV, USB_PID_DIBCOM_STK8096PVR) },
+ { USB_DEVICE(USB_VID_DIBCOM, USB_PID_DIBCOM_STK8096PVR) },
{ 0 } /* Terminating entry */
};
MODULE_DEVICE_TABLE(usb, dib0700_usb_id_table);
@@ -5017,7 +5018,8 @@
.num_device_descs = 1,
.devices = {
{ "DiBcom STK8096-PVR reference design",
- { &dib0700_usb_id_table[83], NULL },
+ { &dib0700_usb_id_table[83],
+ &dib0700_usb_id_table[84], NULL},
{ NULL },
},
},
diff --git a/drivers/media/usb/dvb-usb/dibusb-common.c b/drivers/media/usb/dvb-usb/dibusb-common.c
index 35de609..6eea4e6 100644
--- a/drivers/media/usb/dvb-usb/dibusb-common.c
+++ b/drivers/media/usb/dvb-usb/dibusb-common.c
@@ -184,6 +184,8 @@
}
EXPORT_SYMBOL(dibusb_read_eeprom_byte);
+#if IS_ENABLED(CONFIG_DVB_DIB3000MC)
+
/* 3000MC/P stuff */
// Config Adjacent channels Perf -cal22
static struct dibx000_agc_config dib3000p_mt2060_agc_config = {
@@ -242,8 +244,6 @@
.agc2_slope2 = 0x1e,
};
-#if IS_ENABLED(CONFIG_DVB_DIB3000MC)
-
static struct dib3000mc_config mod3000p_dib3000p_config = {
&dib3000p_panasonic_agc_config,
diff --git a/drivers/media/usb/dvb-usb/dw2102.c b/drivers/media/usb/dvb-usb/dw2102.c
index 6d0dd85..49b55d7 100644
--- a/drivers/media/usb/dvb-usb/dw2102.c
+++ b/drivers/media/usb/dvb-usb/dw2102.c
@@ -13,6 +13,7 @@
*
* see Documentation/dvb/README.dvb-usb for more information
*/
+#include "dvb-usb-ids.h"
#include "dw2102.h"
#include "si21xx.h"
#include "stv0299.h"
@@ -38,61 +39,6 @@
/* Max transfer size done by I2C transfer functions */
#define MAX_XFER_SIZE 64
-#ifndef USB_PID_DW2102
-#define USB_PID_DW2102 0x2102
-#endif
-
-#ifndef USB_PID_DW2104
-#define USB_PID_DW2104 0x2104
-#endif
-
-#ifndef USB_PID_DW3101
-#define USB_PID_DW3101 0x3101
-#endif
-
-#ifndef USB_PID_CINERGY_S
-#define USB_PID_CINERGY_S 0x0064
-#endif
-
-#ifndef USB_PID_TEVII_S630
-#define USB_PID_TEVII_S630 0xd630
-#endif
-
-#ifndef USB_PID_TEVII_S650
-#define USB_PID_TEVII_S650 0xd650
-#endif
-
-#ifndef USB_PID_TEVII_S660
-#define USB_PID_TEVII_S660 0xd660
-#endif
-
-#ifndef USB_PID_TEVII_S662
-#define USB_PID_TEVII_S662 0xd662
-#endif
-
-#ifndef USB_PID_TEVII_S480_1
-#define USB_PID_TEVII_S480_1 0xd481
-#endif
-
-#ifndef USB_PID_TEVII_S480_2
-#define USB_PID_TEVII_S480_2 0xd482
-#endif
-
-#ifndef USB_PID_PROF_1100
-#define USB_PID_PROF_1100 0xb012
-#endif
-
-#ifndef USB_PID_TEVII_S421
-#define USB_PID_TEVII_S421 0xd421
-#endif
-
-#ifndef USB_PID_TEVII_S632
-#define USB_PID_TEVII_S632 0xd632
-#endif
-
-#ifndef USB_PID_GOTVIEW_SAT_HD
-#define USB_PID_GOTVIEW_SAT_HD 0x5456
-#endif
#define DW210X_READ_MSG 0
#define DW210X_WRITE_MSG 1
@@ -1709,7 +1655,7 @@
[CYPRESS_DW2101] = {USB_DEVICE(USB_VID_CYPRESS, 0x2101)},
[CYPRESS_DW2104] = {USB_DEVICE(USB_VID_CYPRESS, USB_PID_DW2104)},
[TEVII_S650] = {USB_DEVICE(0x9022, USB_PID_TEVII_S650)},
- [TERRATEC_CINERGY_S] = {USB_DEVICE(USB_VID_TERRATEC, USB_PID_CINERGY_S)},
+ [TERRATEC_CINERGY_S] = {USB_DEVICE(USB_VID_TERRATEC, USB_PID_TERRATEC_CINERGY_S)},
[CYPRESS_DW3101] = {USB_DEVICE(USB_VID_CYPRESS, USB_PID_DW3101)},
[TEVII_S630] = {USB_DEVICE(0x9022, USB_PID_TEVII_S630)},
[PROF_1100] = {USB_DEVICE(0x3011, USB_PID_PROF_1100)},
@@ -1801,7 +1747,7 @@
dw210x_op_rw(dev, 0xbf, 0x0040, 0, &reset, 0,
DW210X_WRITE_MSG);
break;
- case USB_PID_CINERGY_S:
+ case USB_PID_TERRATEC_CINERGY_S:
case USB_PID_DW2102:
dw210x_op_rw(dev, 0xbf, 0x0040, 0, &reset, 0,
DW210X_WRITE_MSG);
@@ -1843,6 +1789,9 @@
msleep(100);
kfree(p);
}
+
+ if (le16_to_cpu(dev->descriptor.idProduct) == 0x2101)
+ release_firmware(fw);
return ret;
}
diff --git a/drivers/media/usb/dvb-usb/pctv452e.c b/drivers/media/usb/dvb-usb/pctv452e.c
index ec397c4..c05de1b 100644
--- a/drivers/media/usb/dvb-usb/pctv452e.c
+++ b/drivers/media/usb/dvb-usb/pctv452e.c
@@ -995,11 +995,11 @@
/* parameter for the MPEG2-data transfer */
.stream = {
.type = USB_ISOC,
- .count = 7,
+ .count = 4,
.endpoint = 0x02,
.u = {
.isoc = {
- .framesperurb = 4,
+ .framesperurb = 64,
.framesize = 940,
.interval = 1
}
diff --git a/drivers/media/usb/em28xx/Kconfig b/drivers/media/usb/em28xx/Kconfig
index e382210..d917b0a 100644
--- a/drivers/media/usb/em28xx/Kconfig
+++ b/drivers/media/usb/em28xx/Kconfig
@@ -59,6 +59,8 @@
select DVB_DRX39XYJ if MEDIA_SUBDRV_AUTOSELECT
select DVB_SI2168 if MEDIA_SUBDRV_AUTOSELECT
select MEDIA_TUNER_SI2157 if MEDIA_SUBDRV_AUTOSELECT
+ select DVB_TC90522 if MEDIA_SUBDRV_AUTOSELECT
+ select MEDIA_TUNER_QM1D1C0042 if MEDIA_SUBDRV_AUTOSELECT
---help---
This adds support for DVB cards based on the
Empiatech em28xx chips.
diff --git a/drivers/media/usb/em28xx/em28xx-cards.c b/drivers/media/usb/em28xx/em28xx-cards.c
index 930e3e3..e397f54 100644
--- a/drivers/media/usb/em28xx/em28xx-cards.c
+++ b/drivers/media/usb/em28xx/em28xx-cards.c
@@ -492,6 +492,44 @@
{-1, -1, -1, -1},
};
+static struct em28xx_reg_seq plex_px_bcud[] = {
+ {EM2874_R80_GPIO_P0_CTRL, 0xff, 0xff, 0},
+ {0x0d, 0xff, 0xff, 0},
+ {EM2874_R50_IR_CONFIG, 0x01, 0xff, 0},
+ {EM28XX_R06_I2C_CLK, 0x40, 0xff, 0},
+ {EM2874_R80_GPIO_P0_CTRL, 0xfd, 0xff, 100},
+ {EM28XX_R12_VINENABLE, 0x20, 0x20, 0},
+ {0x0d, 0x42, 0xff, 1000},
+ {EM2874_R80_GPIO_P0_CTRL, 0xfc, 0xff, 10},
+ {EM2874_R80_GPIO_P0_CTRL, 0xfd, 0xff, 10},
+ {0x73, 0xfd, 0xff, 100},
+ {-1, -1, -1, -1},
+};
+
+/*
+ * 2040:0265 Hauppauge WinTV-dualHD DVB
+ * reg 0x80/0x84:
+ * GPIO_0: Yellow LED tuner 1, 0=on, 1=off
+ * GPIO_1: Green LED tuner 1, 0=on, 1=off
+ * GPIO_2: Yellow LED tuner 2, 0=on, 1=off
+ * GPIO_3: Green LED tuner 2, 0=on, 1=off
+ * GPIO_5: Reset #2, 0=active
+ * GPIO_6: Reset #1, 0=active
+ */
+static struct em28xx_reg_seq hauppauge_dualhd_dvb[] = {
+ {EM2874_R80_GPIO_P0_CTRL, 0xff, 0xff, 0},
+ {0x0d, 0xff, 0xff, 200},
+ {0x50, 0x04, 0xff, 300},
+ {EM2874_R80_GPIO_P0_CTRL, 0xbf, 0xff, 100}, /* demod 1 reset */
+ {EM2874_R80_GPIO_P0_CTRL, 0xff, 0xff, 100},
+ {EM2874_R80_GPIO_P0_CTRL, 0xdf, 0xff, 100}, /* demod 2 reset */
+ {EM2874_R80_GPIO_P0_CTRL, 0xff, 0xff, 100},
+ {EM2874_R5F_TS_ENABLE, 0x44, 0xff, 50},
+ {EM2874_R5D_TS1_PKT_SIZE, 0x05, 0xff, 50},
+ {EM2874_R5E_TS2_PKT_SIZE, 0x05, 0xff, 50},
+ {-1, -1, -1, -1},
+};
+
/*
* Button definitions
*/
@@ -571,6 +609,22 @@
{-1, 0, 0, 0},
};
+static struct em28xx_led hauppauge_dualhd_leds[] = {
+ {
+ .role = EM28XX_LED_DIGITAL_CAPTURING,
+ .gpio_reg = EM2874_R80_GPIO_P0_CTRL,
+ .gpio_mask = EM_GPIO_1,
+ .inverted = 1,
+ },
+ {
+ .role = EM28XX_LED_DIGITAL_CAPTURING_TS2,
+ .gpio_reg = EM2874_R80_GPIO_P0_CTRL,
+ .gpio_mask = EM_GPIO_3,
+ .inverted = 1,
+ },
+ {-1, 0, 0, 0},
+};
+
/*
* Board definitions
*/
@@ -2306,6 +2360,35 @@
.has_dvb = 1,
.ir_codes = RC_MAP_TERRATEC_SLIM_2,
},
+
+ /*
+ * 3275:0085 PLEX PX-BCUD.
+ * Empia EM28178, TOSHIBA TC90532XBG, Sharp QM1D1C0042
+ */
+ [EM28178_BOARD_PLEX_PX_BCUD] = {
+ .name = "PLEX PX-BCUD",
+ .xclk = EM28XX_XCLK_FREQUENCY_4_3MHZ,
+ .def_i2c_bus = 1,
+ .i2c_speed = EM28XX_I2C_CLK_WAIT_ENABLE,
+ .tuner_type = TUNER_ABSENT,
+ .tuner_gpio = plex_px_bcud,
+ .has_dvb = 1,
+ },
+ /*
+ * 2040:0265 Hauppauge WinTV-dualHD (DVB version).
+ * Empia EM28274, 2x Silicon Labs Si2168, 2x Silicon Labs Si2157
+ */
+ [EM28174_BOARD_HAUPPAUGE_WINTV_DUALHD_DVB] = {
+ .name = "Hauppauge WinTV-dualHD DVB",
+ .def_i2c_bus = 1,
+ .i2c_speed = EM28XX_I2C_CLK_WAIT_ENABLE |
+ EM28XX_I2C_FREQ_400_KHZ,
+ .tuner_type = TUNER_ABSENT,
+ .tuner_gpio = hauppauge_dualhd_dvb,
+ .has_dvb = 1,
+ .ir_codes = RC_MAP_HAUPPAUGE,
+ .leds = hauppauge_dualhd_leds,
+ },
};
EXPORT_SYMBOL_GPL(em28xx_boards);
@@ -2429,6 +2512,8 @@
.driver_info = EM2883_BOARD_HAUPPAUGE_WINTV_HVR_950 },
{ USB_DEVICE(0x2040, 0x651f),
.driver_info = EM2883_BOARD_HAUPPAUGE_WINTV_HVR_850 },
+ { USB_DEVICE(0x2040, 0x0265),
+ .driver_info = EM28174_BOARD_HAUPPAUGE_WINTV_DUALHD_DVB },
{ USB_DEVICE(0x0438, 0xb002),
.driver_info = EM2880_BOARD_AMD_ATI_TV_WONDER_HD_600 },
{ USB_DEVICE(0x2001, 0xf112),
@@ -2495,6 +2580,8 @@
.driver_info = EM2861_BOARD_LEADTEK_VC100 },
{ USB_DEVICE(0xeb1a, 0x8179),
.driver_info = EM28178_BOARD_TERRATEC_T2_STICK_HD },
+ { USB_DEVICE(0x3275, 0x0085),
+ .driver_info = EM28178_BOARD_PLEX_PX_BCUD },
{ },
};
MODULE_DEVICE_TABLE(usb, em28xx_id_table);
@@ -2861,6 +2948,7 @@
case EM2883_BOARD_HAUPPAUGE_WINTV_HVR_850:
case EM2883_BOARD_HAUPPAUGE_WINTV_HVR_950:
case EM2884_BOARD_HAUPPAUGE_WINTV_HVR_930C:
+ case EM28174_BOARD_HAUPPAUGE_WINTV_DUALHD_DVB:
{
struct tveeprom tv;
diff --git a/drivers/media/usb/em28xx/em28xx-dvb.c b/drivers/media/usb/em28xx/em28xx-dvb.c
index 5d209c7..1a5c012 100644
--- a/drivers/media/usb/em28xx/em28xx-dvb.c
+++ b/drivers/media/usb/em28xx/em28xx-dvb.c
@@ -58,6 +58,8 @@
#include "ts2020.h"
#include "si2168.h"
#include "si2157.h"
+#include "tc90522.h"
+#include "qm1d1c0042.h"
MODULE_AUTHOR("Mauro Carvalho Chehab <mchehab@infradead.org>");
MODULE_LICENSE("GPL");
@@ -787,6 +789,68 @@
return 0;
}
+static void px_bcud_init(struct em28xx *dev)
+{
+ int i;
+ struct {
+ unsigned char r[4];
+ int len;
+ } regs1[] = {
+ {{ 0x0e, 0x77 }, 2},
+ {{ 0x0f, 0x77 }, 2},
+ {{ 0x03, 0x90 }, 2},
+ }, regs2[] = {
+ {{ 0x07, 0x01 }, 2},
+ {{ 0x08, 0x10 }, 2},
+ {{ 0x13, 0x00 }, 2},
+ {{ 0x17, 0x00 }, 2},
+ {{ 0x03, 0x01 }, 2},
+ {{ 0x10, 0xb1 }, 2},
+ {{ 0x11, 0x40 }, 2},
+ {{ 0x85, 0x7a }, 2},
+ {{ 0x87, 0x04 }, 2},
+ };
+ static struct em28xx_reg_seq gpio[] = {
+ {EM28XX_R06_I2C_CLK, 0x40, 0xff, 300},
+ {EM2874_R80_GPIO_P0_CTRL, 0xfd, 0xff, 60},
+ {EM28XX_R15_RGAIN, 0x20, 0xff, 0},
+ {EM28XX_R16_GGAIN, 0x20, 0xff, 0},
+ {EM28XX_R17_BGAIN, 0x20, 0xff, 0},
+ {EM28XX_R18_ROFFSET, 0x00, 0xff, 0},
+ {EM28XX_R19_GOFFSET, 0x00, 0xff, 0},
+ {EM28XX_R1A_BOFFSET, 0x00, 0xff, 0},
+ {EM28XX_R23_UOFFSET, 0x00, 0xff, 0},
+ {EM28XX_R24_VOFFSET, 0x00, 0xff, 0},
+ {EM28XX_R26_COMPR, 0x00, 0xff, 0},
+ {0x13, 0x08, 0xff, 0},
+ {EM28XX_R12_VINENABLE, 0x27, 0xff, 0},
+ {EM28XX_R0C_USBSUSP, 0x10, 0xff, 0},
+ {EM28XX_R27_OUTFMT, 0x00, 0xff, 0},
+ {EM28XX_R10_VINMODE, 0x00, 0xff, 0},
+ {EM28XX_R11_VINCTRL, 0x11, 0xff, 0},
+ {EM2874_R50_IR_CONFIG, 0x01, 0xff, 0},
+ {EM2874_R5F_TS_ENABLE, 0x80, 0xff, 0},
+ {EM28XX_R06_I2C_CLK, 0x46, 0xff, 0},
+ };
+ em28xx_write_reg(dev, EM28XX_R06_I2C_CLK, 0x46);
+ /* sleeping ISDB-T */
+ dev->dvb->i2c_client_demod->addr = 0x14;
+ for (i = 0; i < ARRAY_SIZE(regs1); i++)
+ i2c_master_send(dev->dvb->i2c_client_demod, regs1[i].r,
+ regs1[i].len);
+ /* sleeping ISDB-S */
+ dev->dvb->i2c_client_demod->addr = 0x15;
+ for (i = 0; i < ARRAY_SIZE(regs2); i++)
+ i2c_master_send(dev->dvb->i2c_client_demod, regs2[i].r,
+ regs2[i].len);
+ for (i = 0; i < ARRAY_SIZE(gpio); i++) {
+ em28xx_write_reg_bits(dev, gpio[i].reg, gpio[i].val,
+ gpio[i].mask);
+ if (gpio[i].sleep > 0)
+ msleep(gpio[i].sleep);
+ }
+};
+
static struct mt352_config terratec_xs_mt352_cfg = {
.demod_address = (0x1e >> 1),
.no_tuner = 1,
@@ -1762,6 +1826,127 @@
dvb->i2c_client_tuner = client;
}
break;
+
+ case EM28178_BOARD_PLEX_PX_BCUD:
+ {
+ struct i2c_client *client;
+ struct i2c_board_info info;
+ struct tc90522_config tc90522_config;
+ struct qm1d1c0042_config qm1d1c0042_config;
+
+ /* attach demod */
+ memset(&tc90522_config, 0, sizeof(tc90522_config));
+ memset(&info, 0, sizeof(struct i2c_board_info));
+ strlcpy(info.type, "tc90522sat", I2C_NAME_SIZE);
+ info.addr = 0x15;
+ info.platform_data = &tc90522_config;
+ request_module("tc90522");
+ client = i2c_new_device(&dev->i2c_adap[dev->def_i2c_bus], &info);
+ if (client == NULL || client->dev.driver == NULL) {
+ result = -ENODEV;
+ goto out_free;
+ }
+ dvb->i2c_client_demod = client;
+ if (!try_module_get(client->dev.driver->owner)) {
+ i2c_unregister_device(client);
+ result = -ENODEV;
+ goto out_free;
+ }
+
+ /* attach tuner */
+ memset(&qm1d1c0042_config, 0,
+ sizeof(qm1d1c0042_config));
+ qm1d1c0042_config.fe = tc90522_config.fe;
+ qm1d1c0042_config.lpf = 1;
+ memset(&info, 0, sizeof(struct i2c_board_info));
+ strlcpy(info.type, "qm1d1c0042", I2C_NAME_SIZE);
+ info.addr = 0x61;
+ info.platform_data = &qm1d1c0042_config;
+ request_module(info.type);
+ client = i2c_new_device(tc90522_config.tuner_i2c,
+ &info);
+ if (client == NULL || client->dev.driver == NULL) {
+ module_put(dvb->i2c_client_demod->dev.driver->owner);
+ i2c_unregister_device(dvb->i2c_client_demod);
+ result = -ENODEV;
+ goto out_free;
+ }
+ dvb->i2c_client_tuner = client;
+ if (!try_module_get(client->dev.driver->owner)) {
+ i2c_unregister_device(client);
+ module_put(dvb->i2c_client_demod->dev.driver->owner);
+ i2c_unregister_device(dvb->i2c_client_demod);
+ result = -ENODEV;
+ goto out_free;
+ }
+ dvb->fe[0] = tc90522_config.fe;
+ px_bcud_init(dev);
+ }
+ break;
+ case EM28174_BOARD_HAUPPAUGE_WINTV_DUALHD_DVB:
+ {
+ struct i2c_adapter *adapter;
+ struct i2c_client *client;
+ struct i2c_board_info info;
+ struct si2168_config si2168_config;
+ struct si2157_config si2157_config;
+
+ /* attach demod */
+ memset(&si2168_config, 0, sizeof(si2168_config));
+ si2168_config.i2c_adapter = &adapter;
+ si2168_config.fe = &dvb->fe[0];
+ si2168_config.ts_mode = SI2168_TS_SERIAL;
+ memset(&info, 0, sizeof(struct i2c_board_info));
+ strlcpy(info.type, "si2168", I2C_NAME_SIZE);
+ info.addr = 0x64;
+ info.platform_data = &si2168_config;
+ request_module(info.type);
+ client = i2c_new_device(&dev->i2c_adap[dev->def_i2c_bus], &info);
+ if (client == NULL || client->dev.driver == NULL) {
+ result = -ENODEV;
+ goto out_free;
+ }
+
+ if (!try_module_get(client->dev.driver->owner)) {
+ i2c_unregister_device(client);
+ result = -ENODEV;
+ goto out_free;
+ }
+
+ dvb->i2c_client_demod = client;
+
+ /* attach tuner */
+ memset(&si2157_config, 0, sizeof(si2157_config));
+ si2157_config.fe = dvb->fe[0];
+ si2157_config.if_port = 1;
+#ifdef CONFIG_MEDIA_CONTROLLER_DVB
+ si2157_config.mdev = dev->media_dev;
+#endif
+ memset(&info, 0, sizeof(struct i2c_board_info));
+ strlcpy(info.type, "si2157", I2C_NAME_SIZE);
+ info.addr = 0x60;
+ info.platform_data = &si2157_config;
+ request_module(info.type);
+ client = i2c_new_device(adapter, &info);
+ if (client == NULL || client->dev.driver == NULL) {
+ module_put(dvb->i2c_client_demod->dev.driver->owner);
+ i2c_unregister_device(dvb->i2c_client_demod);
+ result = -ENODEV;
+ goto out_free;
+ }
+
+ if (!try_module_get(client->dev.driver->owner)) {
+ i2c_unregister_device(client);
+ module_put(dvb->i2c_client_demod->dev.driver->owner);
+ i2c_unregister_device(dvb->i2c_client_demod);
+ result = -ENODEV;
+ goto out_free;
+ }
+
+ dvb->i2c_client_tuner = client;
+
+ }
+ break;
default:
em28xx_errdev("/2: The frontend of your DVB/ATSC card"
" isn't supported yet\n");
diff --git a/drivers/media/usb/em28xx/em28xx-reg.h b/drivers/media/usb/em28xx/em28xx-reg.h
index 13cbb7f..afe7a66 100644
--- a/drivers/media/usb/em28xx/em28xx-reg.h
+++ b/drivers/media/usb/em28xx/em28xx-reg.h
@@ -193,6 +193,19 @@
/* em2874 registers */
#define EM2874_R50_IR_CONFIG 0x50
#define EM2874_R51_IR 0x51
+#define EM2874_R5D_TS1_PKT_SIZE 0x5d
+#define EM2874_R5E_TS2_PKT_SIZE 0x5e
+ /*
+ * For both TS1 and TS2, In isochronous mode:
+ * 0x01 188 bytes
+ * 0x02 376 bytes
+ * 0x03 564 bytes
+ * 0x04 752 bytes
+ * 0x05 940 bytes
+ * In bulk mode:
+ * 0x01..0xff total packet count in 188-byte
+ */
+
#define EM2874_R5F_TS_ENABLE 0x5f
/* em2874/174/84, em25xx, em276x/7x/8x GPIO registers */
diff --git a/drivers/media/usb/em28xx/em28xx.h b/drivers/media/usb/em28xx/em28xx.h
index 2674449..d148463 100644
--- a/drivers/media/usb/em28xx/em28xx.h
+++ b/drivers/media/usb/em28xx/em28xx.h
@@ -145,6 +145,8 @@
#define EM2861_BOARD_LEADTEK_VC100 95
#define EM28178_BOARD_TERRATEC_T2_STICK_HD 96
#define EM2884_BOARD_ELGATO_EYETV_HYBRID_2008 97
+#define EM28178_BOARD_PLEX_PX_BCUD 98
+#define EM28174_BOARD_HAUPPAUGE_WINTV_DUALHD_DVB 99
/* Limits minimum and default number of buffers */
#define EM28XX_MIN_BUF 4
@@ -406,6 +408,7 @@
enum em28xx_led_role {
EM28XX_LED_ANALOG_CAPTURING = 0,
EM28XX_LED_DIGITAL_CAPTURING,
+ EM28XX_LED_DIGITAL_CAPTURING_TS2,
EM28XX_LED_ILLUMINATION,
EM28XX_NUM_LED_ROLES, /* must be the last */
};
diff --git a/drivers/media/usb/go7007/go7007-v4l2.c b/drivers/media/usb/go7007/go7007-v4l2.c
index 358c1c1..ea01ee5 100644
--- a/drivers/media/usb/go7007/go7007-v4l2.c
+++ b/drivers/media/usb/go7007/go7007-v4l2.c
@@ -1125,7 +1125,7 @@
vdev->queue = &go->vidq;
video_set_drvdata(vdev, go);
vdev->v4l2_dev = &go->v4l2_dev;
- if (!v4l2_device_has_op(&go->v4l2_dev, video, querystd))
+ if (!v4l2_device_has_op(&go->v4l2_dev, 0, video, querystd))
v4l2_disable_ioctl(vdev, VIDIOC_QUERYSTD);
if (!(go->board_info->flags & GO7007_BOARD_HAS_TUNER)) {
v4l2_disable_ioctl(vdev, VIDIOC_S_FREQUENCY);
diff --git a/drivers/media/usb/pvrusb2/pvrusb2-hdw.c b/drivers/media/usb/pvrusb2/pvrusb2-hdw.c
index 1a093e5..83e9a3e 100644
--- a/drivers/media/usb/pvrusb2/pvrusb2-hdw.c
+++ b/drivers/media/usb/pvrusb2/pvrusb2-hdw.c
@@ -3672,11 +3672,10 @@
hdw->cmd_debug_state = 1;
- if (write_len) {
+ if (write_len && write_data)
hdw->cmd_debug_code = ((unsigned char *)write_data)[0];
- } else {
+ else
hdw->cmd_debug_code = 0;
- }
hdw->cmd_debug_write_len = write_len;
hdw->cmd_debug_read_len = read_len;
@@ -3688,7 +3687,7 @@
setup_timer(&timer, pvr2_ctl_timeout, (unsigned long)hdw);
timer.expires = jiffies + timeout;
- if (write_len) {
+ if (write_len && write_data) {
hdw->cmd_debug_state = 2;
/* Transfer write data to internal buffer */
for (idx = 0; idx < write_len; idx++) {
@@ -3795,7 +3794,7 @@
goto done;
}
}
- if (read_len) {
+ if (read_len && read_data) {
/* Validate results of read request */
if ((hdw->ctl_read_urb->status != 0) &&
(hdw->ctl_read_urb->status != -ENOENT) &&
diff --git a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
index 019644f..bacecbd 100644
--- a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
+++ b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
@@ -280,7 +280,8 @@
static int put_v4l2_create32(struct v4l2_create_buffers *kp, struct v4l2_create_buffers32 __user *up)
{
if (!access_ok(VERIFY_WRITE, up, sizeof(struct v4l2_create_buffers32)) ||
- copy_to_user(up, kp, offsetof(struct v4l2_create_buffers32, format)))
+ copy_to_user(up, kp, offsetof(struct v4l2_create_buffers32, format)) ||
+ copy_to_user(up->reserved, kp->reserved, sizeof(kp->reserved)))
return -EFAULT;
return __put_v4l2_format32(&kp->format, &up->format);
}
diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c
index d8e5994..70b559d 100644
--- a/drivers/media/v4l2-core/v4l2-dev.c
+++ b/drivers/media/v4l2-core/v4l2-dev.c
@@ -735,6 +735,7 @@
if (!vdev->v4l2_dev->mdev)
return 0;
+ vdev->entity.obj_type = MEDIA_ENTITY_TYPE_VIDEO_DEVICE;
vdev->entity.function = MEDIA_ENT_F_UNKNOWN;
switch (type) {
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 170dd68..28e5be2 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1020,9 +1020,12 @@
struct file *file, void *fh, void *arg)
{
struct v4l2_capability *cap = (struct v4l2_capability *)arg;
+ struct video_device *vfd = video_devdata(file);
int ret;
cap->version = LINUX_VERSION_CODE;
+ cap->device_caps = vfd->device_caps;
+ cap->capabilities = vfd->device_caps | V4L2_CAP_DEVICE_CAPS;
ret = ops->vidioc_querycap(file, fh, cap);
@@ -2157,40 +2160,56 @@
struct file *file, void *fh, void *arg)
{
struct v4l2_cropcap *p = arg;
-
- if (ops->vidioc_g_selection) {
- struct v4l2_selection s = { .type = p->type };
- int ret;
-
- /* obtaining bounds */
- if (V4L2_TYPE_IS_OUTPUT(p->type))
- s.target = V4L2_SEL_TGT_COMPOSE_BOUNDS;
- else
- s.target = V4L2_SEL_TGT_CROP_BOUNDS;
-
- ret = ops->vidioc_g_selection(file, fh, &s);
- if (ret)
- return ret;
- p->bounds = s.r;
-
- /* obtaining defrect */
- if (V4L2_TYPE_IS_OUTPUT(p->type))
- s.target = V4L2_SEL_TGT_COMPOSE_DEFAULT;
- else
- s.target = V4L2_SEL_TGT_CROP_DEFAULT;
-
- ret = ops->vidioc_g_selection(file, fh, &s);
- if (ret)
- return ret;
- p->defrect = s.r;
- }
+ struct v4l2_selection s = { .type = p->type };
+ int ret = 0;
/* setting trivial pixelaspect */
p->pixelaspect.numerator = 1;
p->pixelaspect.denominator = 1;
+ /*
+ * The determine_valid_ioctls() call already should ensure
+ * that this can never happen, but just in case...
+ */
+ if (WARN_ON(!ops->vidioc_cropcap && !ops->vidioc_cropcap))
+ return -ENOTTY;
+
if (ops->vidioc_cropcap)
- return ops->vidioc_cropcap(file, fh, p);
+ ret = ops->vidioc_cropcap(file, fh, p);
+
+ if (!ops->vidioc_g_selection)
+ return ret;
+
+ /*
+ * Ignore ENOTTY or ENOIOCTLCMD error returns, just use the
+ * square pixel aspect ratio in that case.
+ */
+ if (ret && ret != -ENOTTY && ret != -ENOIOCTLCMD)
+ return ret;
+
+ /* Use g_selection() to fill in the bounds and defrect rectangles */
+
+ /* obtaining bounds */
+ if (V4L2_TYPE_IS_OUTPUT(p->type))
+ s.target = V4L2_SEL_TGT_COMPOSE_BOUNDS;
+ else
+ s.target = V4L2_SEL_TGT_CROP_BOUNDS;
+
+ ret = ops->vidioc_g_selection(file, fh, &s);
+ if (ret)
+ return ret;
+ p->bounds = s.r;
+
+ /* obtaining defrect */
+ if (V4L2_TYPE_IS_OUTPUT(p->type))
+ s.target = V4L2_SEL_TGT_COMPOSE_DEFAULT;
+ else
+ s.target = V4L2_SEL_TGT_CROP_DEFAULT;
+
+ ret = ops->vidioc_g_selection(file, fh, &s);
+ if (ret)
+ return ret;
+ p->defrect = s.r;
return 0;
}
diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
index 2228cd3..ca94bde 100644
--- a/drivers/media/v4l2-core/v4l2-mc.c
+++ b/drivers/media/v4l2-core/v4l2-mc.c
@@ -263,7 +263,7 @@
media_entity_graph_walk_start(graph, entity);
while ((entity = media_entity_graph_walk_next(graph))) {
- if (is_media_entity_v4l2_io(entity))
+ if (is_media_entity_v4l2_video_device(entity))
use += entity->use_count;
}
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index d630838..953eab0 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -35,9 +35,11 @@
static int subdev_fh_init(struct v4l2_subdev_fh *fh, struct v4l2_subdev *sd)
{
#if defined(CONFIG_VIDEO_V4L2_SUBDEV_API)
- fh->pad = kzalloc(sizeof(*fh->pad) * sd->entity.num_pads, GFP_KERNEL);
- if (fh->pad == NULL)
- return -ENOMEM;
+ if (sd->entity.num_pads) {
+ fh->pad = v4l2_subdev_alloc_pad_config(sd);
+ if (fh->pad == NULL)
+ return -ENOMEM;
+ }
#endif
return 0;
}
@@ -45,7 +47,7 @@
static void subdev_fh_free(struct v4l2_subdev_fh *fh)
{
#if defined(CONFIG_VIDEO_V4L2_SUBDEV_API)
- kfree(fh->pad);
+ v4l2_subdev_free_pad_config(fh->pad);
fh->pad = NULL;
#endif
}
@@ -508,7 +510,7 @@
if (source_fmt->format.width != sink_fmt->format.width
|| source_fmt->format.height != sink_fmt->format.height
|| source_fmt->format.code != sink_fmt->format.code)
- return -EINVAL;
+ return -EPIPE;
/* The field order must match, or the sink field order must be NONE
* to support interlaced hardware connected to bridges that support
@@ -516,7 +518,7 @@
*/
if (source_fmt->format.field != sink_fmt->format.field &&
sink_fmt->format.field != V4L2_FIELD_NONE)
- return -EINVAL;
+ return -EPIPE;
return 0;
}
@@ -569,6 +571,35 @@
sink, link, &source_fmt, &sink_fmt);
}
EXPORT_SYMBOL_GPL(v4l2_subdev_link_validate);
+
+struct v4l2_subdev_pad_config *
+v4l2_subdev_alloc_pad_config(struct v4l2_subdev *sd)
+{
+ struct v4l2_subdev_pad_config *cfg;
+ int ret;
+
+ if (!sd->entity.num_pads)
+ return NULL;
+
+ cfg = kcalloc(sd->entity.num_pads, sizeof(*cfg), GFP_KERNEL);
+ if (!cfg)
+ return NULL;
+
+ ret = v4l2_subdev_call(sd, pad, init_cfg, cfg);
+ if (ret < 0 && ret != -ENOIOCTLCMD) {
+ kfree(cfg);
+ return NULL;
+ }
+
+ return cfg;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_alloc_pad_config);
+
+void v4l2_subdev_free_pad_config(struct v4l2_subdev_pad_config *cfg)
+{
+ kfree(cfg);
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_free_pad_config);
#endif /* CONFIG_MEDIA_CONTROLLER */
void v4l2_subdev_init(struct v4l2_subdev *sd, const struct v4l2_subdev_ops *ops)
@@ -584,6 +615,7 @@
sd->host_priv = NULL;
#if defined(CONFIG_MEDIA_CONTROLLER)
sd->entity.name = sd->name;
+ sd->entity.obj_type = MEDIA_ENTITY_TYPE_V4L2_SUBDEV;
sd->entity.function = MEDIA_ENT_F_V4L2_SUBDEV_UNKNOWN;
#endif
}
diff --git a/drivers/memory/Kconfig b/drivers/memory/Kconfig
index c61a284..81ddb17 100644
--- a/drivers/memory/Kconfig
+++ b/drivers/memory/Kconfig
@@ -51,6 +51,7 @@
config OMAP_GPMC
bool
+ select GPIOLIB
help
This driver is for the General Purpose Memory Controller (GPMC)
present on Texas Instruments SoCs (e.g. OMAP2+). GPMC allows
diff --git a/drivers/memory/fsl_ifc.c b/drivers/memory/fsl_ifc.c
index 2a691da..904b4af 100644
--- a/drivers/memory/fsl_ifc.c
+++ b/drivers/memory/fsl_ifc.c
@@ -59,11 +59,11 @@
{
int i = 0;
- if (!fsl_ifc_ctrl_dev || !fsl_ifc_ctrl_dev->regs)
+ if (!fsl_ifc_ctrl_dev || !fsl_ifc_ctrl_dev->gregs)
return -ENODEV;
for (i = 0; i < fsl_ifc_ctrl_dev->banks; i++) {
- u32 cspr = ifc_in32(&fsl_ifc_ctrl_dev->regs->cspr_cs[i].cspr);
+ u32 cspr = ifc_in32(&fsl_ifc_ctrl_dev->gregs->cspr_cs[i].cspr);
if (cspr & CSPR_V && (cspr & CSPR_BA) ==
convert_ifc_address(addr_base))
return i;
@@ -75,7 +75,7 @@
static int fsl_ifc_ctrl_init(struct fsl_ifc_ctrl *ctrl)
{
- struct fsl_ifc_regs __iomem *ifc = ctrl->regs;
+ struct fsl_ifc_global __iomem *ifc = ctrl->gregs;
/*
* Clear all the common status and event registers
@@ -104,7 +104,7 @@
irq_dispose_mapping(ctrl->nand_irq);
irq_dispose_mapping(ctrl->irq);
- iounmap(ctrl->regs);
+ iounmap(ctrl->gregs);
dev_set_drvdata(&dev->dev, NULL);
kfree(ctrl);
@@ -122,7 +122,7 @@
static u32 check_nand_stat(struct fsl_ifc_ctrl *ctrl)
{
- struct fsl_ifc_regs __iomem *ifc = ctrl->regs;
+ struct fsl_ifc_runtime __iomem *ifc = ctrl->rregs;
unsigned long flags;
u32 stat;
@@ -157,7 +157,7 @@
static irqreturn_t fsl_ifc_ctrl_irq(int irqno, void *data)
{
struct fsl_ifc_ctrl *ctrl = data;
- struct fsl_ifc_regs __iomem *ifc = ctrl->regs;
+ struct fsl_ifc_global __iomem *ifc = ctrl->gregs;
u32 err_axiid, err_srcid, status, cs_err, err_addr;
irqreturn_t ret = IRQ_NONE;
@@ -215,6 +215,7 @@
{
int ret = 0;
int version, banks;
+ void __iomem *addr;
dev_info(&dev->dev, "Freescale Integrated Flash Controller\n");
@@ -225,22 +226,13 @@
dev_set_drvdata(&dev->dev, fsl_ifc_ctrl_dev);
/* IOMAP the entire IFC region */
- fsl_ifc_ctrl_dev->regs = of_iomap(dev->dev.of_node, 0);
- if (!fsl_ifc_ctrl_dev->regs) {
+ fsl_ifc_ctrl_dev->gregs = of_iomap(dev->dev.of_node, 0);
+ if (!fsl_ifc_ctrl_dev->gregs) {
dev_err(&dev->dev, "failed to get memory region\n");
ret = -ENODEV;
goto err;
}
- version = ifc_in32(&fsl_ifc_ctrl_dev->regs->ifc_rev) &
- FSL_IFC_VERSION_MASK;
- banks = (version == FSL_IFC_VERSION_1_0_0) ? 4 : 8;
- dev_info(&dev->dev, "IFC version %d.%d, %d banks\n",
- version >> 24, (version >> 16) & 0xf, banks);
-
- fsl_ifc_ctrl_dev->version = version;
- fsl_ifc_ctrl_dev->banks = banks;
-
if (of_property_read_bool(dev->dev.of_node, "little-endian")) {
fsl_ifc_ctrl_dev->little_endian = true;
dev_dbg(&dev->dev, "IFC REGISTERS are LITTLE endian\n");
@@ -249,8 +241,9 @@
dev_dbg(&dev->dev, "IFC REGISTERS are BIG endian\n");
}
- version = ioread32be(&fsl_ifc_ctrl_dev->regs->ifc_rev) &
+ version = ifc_in32(&fsl_ifc_ctrl_dev->gregs->ifc_rev) &
FSL_IFC_VERSION_MASK;
+
banks = (version == FSL_IFC_VERSION_1_0_0) ? 4 : 8;
dev_info(&dev->dev, "IFC version %d.%d, %d banks\n",
version >> 24, (version >> 16) & 0xf, banks);
@@ -258,6 +251,13 @@
fsl_ifc_ctrl_dev->version = version;
fsl_ifc_ctrl_dev->banks = banks;
+ addr = fsl_ifc_ctrl_dev->gregs;
+ if (version >= FSL_IFC_VERSION_2_0_0)
+ addr += PGOFFSET_64K;
+ else
+ addr += PGOFFSET_4K;
+ fsl_ifc_ctrl_dev->rregs = addr;
+
/* get the Controller level irq */
fsl_ifc_ctrl_dev->irq = irq_of_parse_and_map(dev->dev.of_node, 0);
if (fsl_ifc_ctrl_dev->irq == 0) {
diff --git a/drivers/memory/mtk-smi.c b/drivers/memory/mtk-smi.c
index 089091f..f6b5757 100644
--- a/drivers/memory/mtk-smi.c
+++ b/drivers/memory/mtk-smi.c
@@ -91,6 +91,7 @@
return 0;
}
+EXPORT_SYMBOL_GPL(mtk_smi_larb_get);
void mtk_smi_larb_put(struct device *larbdev)
{
@@ -106,6 +107,7 @@
mtk_smi_disable(&larb->smi);
mtk_smi_disable(common);
}
+EXPORT_SYMBOL_GPL(mtk_smi_larb_put);
static int
mtk_smi_larb_bind(struct device *dev, struct device *master, void *data)
diff --git a/drivers/memory/of_memory.c b/drivers/memory/of_memory.c
index 6007435..9daf94b 100644
--- a/drivers/memory/of_memory.c
+++ b/drivers/memory/of_memory.c
@@ -109,7 +109,7 @@
struct lpddr2_timings *timings = NULL;
u32 arr_sz = 0, i = 0;
struct device_node *np_tim;
- char *tim_compat;
+ char *tim_compat = NULL;
switch (device_type) {
case DDR_TYPE_LPDDR2_S2:
diff --git a/drivers/memory/omap-gpmc.c b/drivers/memory/omap-gpmc.c
index 21825dd..af4884b 100644
--- a/drivers/memory/omap-gpmc.c
+++ b/drivers/memory/omap-gpmc.c
@@ -21,15 +21,15 @@
#include <linux/spinlock.h>
#include <linux/io.h>
#include <linux/module.h>
+#include <linux/gpio/driver.h>
#include <linux/interrupt.h>
+#include <linux/irqdomain.h>
#include <linux/platform_device.h>
#include <linux/of.h>
#include <linux/of_address.h>
-#include <linux/of_mtd.h>
#include <linux/of_device.h>
#include <linux/of_platform.h>
#include <linux/omap-gpmc.h>
-#include <linux/mtd/nand.h>
#include <linux/pm_runtime.h>
#include <linux/platform_data/mtd-nand-omap2.h>
@@ -81,6 +81,8 @@
#define GPMC_CONFIG_LIMITEDADDRESS BIT(1)
+#define GPMC_STATUS_EMPTYWRITEBUFFERSTATUS BIT(0)
+
#define GPMC_CONFIG2_CSEXTRADELAY BIT(7)
#define GPMC_CONFIG3_ADVEXTRADELAY BIT(7)
#define GPMC_CONFIG4_OEEXTRADELAY BIT(7)
@@ -92,6 +94,14 @@
#define GPMC_CS_SIZE 0x30
#define GPMC_BCH_SIZE 0x10
+/*
+ * The first 1MB of GPMC address space is typically mapped to
+ * the internal ROM. Never allocate the first page, to
+ * facilitate bug detection; even if we didn't boot from ROM.
+ * As GPMC minimum partition size is 16MB we can only start from
+ * there.
+ */
+#define GPMC_MEM_START 0x1000000
#define GPMC_MEM_END 0x3FFFFFFF
#define GPMC_CHUNK_SHIFT 24 /* 16 MB */
@@ -125,7 +135,6 @@
#define GPMC_CONFIG_RDY_BSY 0x00000001
#define GPMC_CONFIG_DEV_SIZE 0x00000002
#define GPMC_CONFIG_DEV_TYPE 0x00000003
-#define GPMC_SET_IRQ_STATUS 0x00000004
#define GPMC_CONFIG1_WRAPBURST_SUPP (1 << 31)
#define GPMC_CONFIG1_READMULTIPLE_SUPP (1 << 30)
@@ -174,16 +183,12 @@
#define GPMC_CONFIG_WRITEPROTECT 0x00000010
#define WR_RD_PIN_MONITORING 0x00600000
-#define GPMC_ENABLE_IRQ 0x0000000d
-
/* ECC commands */
#define GPMC_ECC_READ 0 /* Reset Hardware ECC for read */
#define GPMC_ECC_WRITE 1 /* Reset Hardware ECC for write */
#define GPMC_ECC_READSYN 2 /* Reset before syndrom is read back */
-/* XXX: Only NAND irq has been considered,currently these are the only ones used
- */
-#define GPMC_NR_IRQ 2
+#define GPMC_NR_NAND_IRQS 2 /* number of NAND specific IRQs */
enum gpmc_clk_domain {
GPMC_CD_FCLK,
@@ -199,11 +204,6 @@
struct resource mem;
};
-struct gpmc_client_irq {
- unsigned irq;
- u32 bitmask;
-};
-
/* Structure to save gpmc cs context */
struct gpmc_cs_config {
u32 config1;
@@ -231,9 +231,15 @@
struct gpmc_cs_config cs_context[GPMC_CS_NUM];
};
-static struct gpmc_client_irq gpmc_client_irq[GPMC_NR_IRQ];
-static struct irq_chip gpmc_irq_chip;
-static int gpmc_irq_start;
+struct gpmc_device {
+ struct device *dev;
+ int irq;
+ struct irq_chip irq_chip;
+ struct gpio_chip gpio_chip;
+ int nirqs;
+};
+
+static struct irq_domain *gpmc_irq_domain;
static struct resource gpmc_mem_root;
static struct gpmc_cs_data gpmc_cs[GPMC_CS_NUM];
@@ -241,8 +247,6 @@
/* Define chip-selects as reserved by default until probe completes */
static unsigned int gpmc_cs_num = GPMC_CS_NUM;
static unsigned int gpmc_nr_waitpins;
-static struct device *gpmc_dev;
-static int gpmc_irq;
static resource_size_t phys_base, mem_size;
static unsigned gpmc_capability;
static void __iomem *gpmc_base;
@@ -1054,14 +1058,6 @@
u32 regval;
switch (cmd) {
- case GPMC_ENABLE_IRQ:
- gpmc_write_reg(GPMC_IRQENABLE, wval);
- break;
-
- case GPMC_SET_IRQ_STATUS:
- gpmc_write_reg(GPMC_IRQSTATUS, wval);
- break;
-
case GPMC_CONFIG_WP:
regval = gpmc_read_reg(GPMC_CONFIG);
if (wval)
@@ -1084,7 +1080,7 @@
{
int i;
- reg->gpmc_status = gpmc_base + GPMC_STATUS;
+ reg->gpmc_status = NULL; /* deprecated */
reg->gpmc_nand_command = gpmc_base + GPMC_CS0_OFFSET +
GPMC_CS_NAND_COMMAND + GPMC_CS_SIZE * cs;
reg->gpmc_nand_address = gpmc_base + GPMC_CS0_OFFSET +
@@ -1118,87 +1114,201 @@
}
}
-int gpmc_get_client_irq(unsigned irq_config)
+static bool gpmc_nand_writebuffer_empty(void)
{
- int i;
+ if (gpmc_read_reg(GPMC_STATUS) & GPMC_STATUS_EMPTYWRITEBUFFERSTATUS)
+ return true;
- if (hweight32(irq_config) > 1)
- return 0;
-
- for (i = 0; i < GPMC_NR_IRQ; i++)
- if (gpmc_client_irq[i].bitmask & irq_config)
- return gpmc_client_irq[i].irq;
-
- return 0;
+ return false;
}
-static int gpmc_irq_endis(unsigned irq, bool endis)
+static struct gpmc_nand_ops nand_ops = {
+ .nand_writebuffer_empty = gpmc_nand_writebuffer_empty,
+};
+
+/**
+ * gpmc_omap_get_nand_ops - Get the GPMC NAND interface
+ * @regs: the GPMC NAND register map exclusive for NAND use.
+ * @cs: GPMC chip select number on which the NAND sits. The
+ * register map returned will be specific to this chip select.
+ *
+ * Returns NULL on error e.g. invalid cs.
+ */
+struct gpmc_nand_ops *gpmc_omap_get_nand_ops(struct gpmc_nand_regs *reg, int cs)
{
- int i;
+ if (cs >= gpmc_cs_num)
+ return NULL;
+
+ gpmc_update_nand_reg(reg, cs);
+
+ return &nand_ops;
+}
+EXPORT_SYMBOL_GPL(gpmc_omap_get_nand_ops);
+
+int gpmc_get_client_irq(unsigned irq_config)
+{
+ if (!gpmc_irq_domain) {
+ pr_warn("%s called before GPMC IRQ domain available\n",
+ __func__);
+ return 0;
+ }
+
+ /* we restrict this to NAND IRQs only */
+ if (irq_config >= GPMC_NR_NAND_IRQS)
+ return 0;
+
+ return irq_create_mapping(gpmc_irq_domain, irq_config);
+}
+
+static int gpmc_irq_endis(unsigned long hwirq, bool endis)
+{
u32 regval;
- for (i = 0; i < GPMC_NR_IRQ; i++)
- if (irq == gpmc_client_irq[i].irq) {
- regval = gpmc_read_reg(GPMC_IRQENABLE);
- if (endis)
- regval |= gpmc_client_irq[i].bitmask;
- else
- regval &= ~gpmc_client_irq[i].bitmask;
- gpmc_write_reg(GPMC_IRQENABLE, regval);
- break;
- }
+ /* bits GPMC_NR_NAND_IRQS to 8 are reserved */
+ if (hwirq >= GPMC_NR_NAND_IRQS)
+ hwirq += 8 - GPMC_NR_NAND_IRQS;
+
+ regval = gpmc_read_reg(GPMC_IRQENABLE);
+ if (endis)
+ regval |= BIT(hwirq);
+ else
+ regval &= ~BIT(hwirq);
+ gpmc_write_reg(GPMC_IRQENABLE, regval);
return 0;
}
static void gpmc_irq_disable(struct irq_data *p)
{
- gpmc_irq_endis(p->irq, false);
+ gpmc_irq_endis(p->hwirq, false);
}
static void gpmc_irq_enable(struct irq_data *p)
{
- gpmc_irq_endis(p->irq, true);
+ gpmc_irq_endis(p->hwirq, true);
}
-static void gpmc_irq_noop(struct irq_data *data) { }
-
-static unsigned int gpmc_irq_noop_ret(struct irq_data *data) { return 0; }
-
-static int gpmc_setup_irq(void)
+static void gpmc_irq_mask(struct irq_data *d)
{
- int i;
+ gpmc_irq_endis(d->hwirq, false);
+}
+
+static void gpmc_irq_unmask(struct irq_data *d)
+{
+ gpmc_irq_endis(d->hwirq, true);
+}
+
+static void gpmc_irq_edge_config(unsigned long hwirq, bool rising_edge)
+{
u32 regval;
- if (!gpmc_irq)
+ /* NAND IRQs polarity is not configurable */
+ if (hwirq < GPMC_NR_NAND_IRQS)
+ return;
+
+ /* WAITPIN starts at BIT 8 */
+ hwirq += 8 - GPMC_NR_NAND_IRQS;
+
+ regval = gpmc_read_reg(GPMC_CONFIG);
+ if (rising_edge)
+ regval &= ~BIT(hwirq);
+ else
+ regval |= BIT(hwirq);
+
+ gpmc_write_reg(GPMC_CONFIG, regval);
+}
+
+static void gpmc_irq_ack(struct irq_data *d)
+{
+ unsigned int hwirq = d->hwirq;
+
+ /* skip reserved bits */
+ if (hwirq >= GPMC_NR_NAND_IRQS)
+ hwirq += 8 - GPMC_NR_NAND_IRQS;
+
+ /* Setting bit to 1 clears (or Acks) the interrupt */
+ gpmc_write_reg(GPMC_IRQSTATUS, BIT(hwirq));
+}
+
+static int gpmc_irq_set_type(struct irq_data *d, unsigned int trigger)
+{
+ /* can't set type for NAND IRQs */
+ if (d->hwirq < GPMC_NR_NAND_IRQS)
return -EINVAL;
- gpmc_irq_start = irq_alloc_descs(-1, 0, GPMC_NR_IRQ, 0);
- if (gpmc_irq_start < 0) {
- pr_err("irq_alloc_descs failed\n");
- return gpmc_irq_start;
+ /* We can support either rising or falling edge at a time */
+ if (trigger == IRQ_TYPE_EDGE_FALLING)
+ gpmc_irq_edge_config(d->hwirq, false);
+ else if (trigger == IRQ_TYPE_EDGE_RISING)
+ gpmc_irq_edge_config(d->hwirq, true);
+ else
+ return -EINVAL;
+
+ return 0;
+}
+
+static int gpmc_irq_map(struct irq_domain *d, unsigned int virq,
+ irq_hw_number_t hw)
+{
+ struct gpmc_device *gpmc = d->host_data;
+
+ irq_set_chip_data(virq, gpmc);
+ if (hw < GPMC_NR_NAND_IRQS) {
+ irq_modify_status(virq, IRQ_NOREQUEST, IRQ_NOAUTOEN);
+ irq_set_chip_and_handler(virq, &gpmc->irq_chip,
+ handle_simple_irq);
+ } else {
+ irq_set_chip_and_handler(virq, &gpmc->irq_chip,
+ handle_edge_irq);
}
- gpmc_irq_chip.name = "gpmc";
- gpmc_irq_chip.irq_startup = gpmc_irq_noop_ret;
- gpmc_irq_chip.irq_enable = gpmc_irq_enable;
- gpmc_irq_chip.irq_disable = gpmc_irq_disable;
- gpmc_irq_chip.irq_shutdown = gpmc_irq_noop;
- gpmc_irq_chip.irq_ack = gpmc_irq_noop;
- gpmc_irq_chip.irq_mask = gpmc_irq_noop;
- gpmc_irq_chip.irq_unmask = gpmc_irq_noop;
+ return 0;
+}
- gpmc_client_irq[0].bitmask = GPMC_IRQ_FIFOEVENTENABLE;
- gpmc_client_irq[1].bitmask = GPMC_IRQ_COUNT_EVENT;
+static const struct irq_domain_ops gpmc_irq_domain_ops = {
+ .map = gpmc_irq_map,
+ .xlate = irq_domain_xlate_twocell,
+};
- for (i = 0; i < GPMC_NR_IRQ; i++) {
- gpmc_client_irq[i].irq = gpmc_irq_start + i;
- irq_set_chip_and_handler(gpmc_client_irq[i].irq,
- &gpmc_irq_chip, handle_simple_irq);
- irq_modify_status(gpmc_client_irq[i].irq, IRQ_NOREQUEST,
- IRQ_NOAUTOEN);
+static irqreturn_t gpmc_handle_irq(int irq, void *data)
+{
+ int hwirq, virq;
+ u32 regval, regvalx;
+ struct gpmc_device *gpmc = data;
+
+ regval = gpmc_read_reg(GPMC_IRQSTATUS);
+ regvalx = regval;
+
+ if (!regval)
+ return IRQ_NONE;
+
+ for (hwirq = 0; hwirq < gpmc->nirqs; hwirq++) {
+ /* skip reserved status bits */
+ if (hwirq == GPMC_NR_NAND_IRQS)
+ regvalx >>= 8 - GPMC_NR_NAND_IRQS;
+
+ if (regvalx & BIT(hwirq)) {
+ virq = irq_find_mapping(gpmc_irq_domain, hwirq);
+ if (!virq) {
+ dev_warn(gpmc->dev,
+ "spurious irq detected hwirq %d, virq %d\n",
+ hwirq, virq);
+ }
+
+ generic_handle_irq(virq);
+ }
}
+ gpmc_write_reg(GPMC_IRQSTATUS, regval);
+
+ return IRQ_HANDLED;
+}
+
+static int gpmc_setup_irq(struct gpmc_device *gpmc)
+{
+ u32 regval;
+ int rc;
+
/* Disable interrupts */
gpmc_write_reg(GPMC_IRQENABLE, 0);
@@ -1206,22 +1316,45 @@
regval = gpmc_read_reg(GPMC_IRQSTATUS);
gpmc_write_reg(GPMC_IRQSTATUS, regval);
- return request_irq(gpmc_irq, gpmc_handle_irq, 0, "gpmc", NULL);
-}
+ gpmc->irq_chip.name = "gpmc";
+ gpmc->irq_chip.irq_enable = gpmc_irq_enable;
+ gpmc->irq_chip.irq_disable = gpmc_irq_disable;
+ gpmc->irq_chip.irq_ack = gpmc_irq_ack;
+ gpmc->irq_chip.irq_mask = gpmc_irq_mask;
+ gpmc->irq_chip.irq_unmask = gpmc_irq_unmask;
+ gpmc->irq_chip.irq_set_type = gpmc_irq_set_type;
-static int gpmc_free_irq(void)
-{
- int i;
-
- if (gpmc_irq)
- free_irq(gpmc_irq, NULL);
-
- for (i = 0; i < GPMC_NR_IRQ; i++) {
- irq_set_handler(gpmc_client_irq[i].irq, NULL);
- irq_set_chip(gpmc_client_irq[i].irq, &no_irq_chip);
+ gpmc_irq_domain = irq_domain_add_linear(gpmc->dev->of_node,
+ gpmc->nirqs,
+ &gpmc_irq_domain_ops,
+ gpmc);
+ if (!gpmc_irq_domain) {
+ dev_err(gpmc->dev, "IRQ domain add failed\n");
+ return -ENODEV;
}
- irq_free_descs(gpmc_irq_start, GPMC_NR_IRQ);
+ rc = request_irq(gpmc->irq, gpmc_handle_irq, 0, "gpmc", gpmc);
+ if (rc) {
+ dev_err(gpmc->dev, "failed to request irq %d: %d\n",
+ gpmc->irq, rc);
+ irq_domain_remove(gpmc_irq_domain);
+ gpmc_irq_domain = NULL;
+ }
+
+ return rc;
+}
+
+static int gpmc_free_irq(struct gpmc_device *gpmc)
+{
+ int hwirq;
+
+ free_irq(gpmc->irq, gpmc);
+
+ for (hwirq = 0; hwirq < gpmc->nirqs; hwirq++)
+ irq_dispose_mapping(irq_find_mapping(gpmc_irq_domain, hwirq));
+
+ irq_domain_remove(gpmc_irq_domain);
+ gpmc_irq_domain = NULL;
return 0;
}
@@ -1242,12 +1375,7 @@
{
int cs;
- /*
- * The first 1MB of GPMC address space is typically mapped to
- * the internal ROM. Never allocate the first page, to
- * facilitate bug detection; even if we didn't boot from ROM.
- */
- gpmc_mem_root.start = SZ_1M;
+ gpmc_mem_root.start = GPMC_MEM_START;
gpmc_mem_root.end = GPMC_MEM_END;
/* Reserve all regions that has been set up by bootloader */
@@ -1796,105 +1924,6 @@
of_property_read_bool(np, "gpmc,time-para-granularity");
}
-#if IS_ENABLED(CONFIG_MTD_NAND)
-
-static const char * const nand_xfer_types[] = {
- [NAND_OMAP_PREFETCH_POLLED] = "prefetch-polled",
- [NAND_OMAP_POLLED] = "polled",
- [NAND_OMAP_PREFETCH_DMA] = "prefetch-dma",
- [NAND_OMAP_PREFETCH_IRQ] = "prefetch-irq",
-};
-
-static int gpmc_probe_nand_child(struct platform_device *pdev,
- struct device_node *child)
-{
- u32 val;
- const char *s;
- struct gpmc_timings gpmc_t;
- struct omap_nand_platform_data *gpmc_nand_data;
-
- if (of_property_read_u32(child, "reg", &val) < 0) {
- dev_err(&pdev->dev, "%s has no 'reg' property\n",
- child->full_name);
- return -ENODEV;
- }
-
- gpmc_nand_data = devm_kzalloc(&pdev->dev, sizeof(*gpmc_nand_data),
- GFP_KERNEL);
- if (!gpmc_nand_data)
- return -ENOMEM;
-
- gpmc_nand_data->cs = val;
- gpmc_nand_data->of_node = child;
-
- /* Detect availability of ELM module */
- gpmc_nand_data->elm_of_node = of_parse_phandle(child, "ti,elm-id", 0);
- if (gpmc_nand_data->elm_of_node == NULL)
- gpmc_nand_data->elm_of_node =
- of_parse_phandle(child, "elm_id", 0);
-
- /* select ecc-scheme for NAND */
- if (of_property_read_string(child, "ti,nand-ecc-opt", &s)) {
- pr_err("%s: ti,nand-ecc-opt not found\n", __func__);
- return -ENODEV;
- }
-
- if (!strcmp(s, "sw"))
- gpmc_nand_data->ecc_opt = OMAP_ECC_HAM1_CODE_SW;
- else if (!strcmp(s, "ham1") ||
- !strcmp(s, "hw") || !strcmp(s, "hw-romcode"))
- gpmc_nand_data->ecc_opt =
- OMAP_ECC_HAM1_CODE_HW;
- else if (!strcmp(s, "bch4"))
- if (gpmc_nand_data->elm_of_node)
- gpmc_nand_data->ecc_opt =
- OMAP_ECC_BCH4_CODE_HW;
- else
- gpmc_nand_data->ecc_opt =
- OMAP_ECC_BCH4_CODE_HW_DETECTION_SW;
- else if (!strcmp(s, "bch8"))
- if (gpmc_nand_data->elm_of_node)
- gpmc_nand_data->ecc_opt =
- OMAP_ECC_BCH8_CODE_HW;
- else
- gpmc_nand_data->ecc_opt =
- OMAP_ECC_BCH8_CODE_HW_DETECTION_SW;
- else if (!strcmp(s, "bch16"))
- if (gpmc_nand_data->elm_of_node)
- gpmc_nand_data->ecc_opt =
- OMAP_ECC_BCH16_CODE_HW;
- else
- pr_err("%s: BCH16 requires ELM support\n", __func__);
- else
- pr_err("%s: ti,nand-ecc-opt invalid value\n", __func__);
-
- /* select data transfer mode for NAND controller */
- if (!of_property_read_string(child, "ti,nand-xfer-type", &s))
- for (val = 0; val < ARRAY_SIZE(nand_xfer_types); val++)
- if (!strcasecmp(s, nand_xfer_types[val])) {
- gpmc_nand_data->xfer_type = val;
- break;
- }
-
- gpmc_nand_data->flash_bbt = of_get_nand_on_flash_bbt(child);
-
- val = of_get_nand_bus_width(child);
- if (val == 16)
- gpmc_nand_data->devsize = NAND_BUSWIDTH_16;
-
- gpmc_read_timings_dt(child, &gpmc_t);
- gpmc_nand_init(gpmc_nand_data, &gpmc_t);
-
- return 0;
-}
-#else
-static int gpmc_probe_nand_child(struct platform_device *pdev,
- struct device_node *child)
-{
- return 0;
-}
-#endif
-
#if IS_ENABLED(CONFIG_MTD_ONENAND)
static int gpmc_probe_onenand_child(struct platform_device *pdev,
struct device_node *child)
@@ -1950,6 +1979,8 @@
const char *name;
int ret, cs;
u32 val;
+ struct gpio_desc *waitpin_desc = NULL;
+ struct gpmc_device *gpmc = platform_get_drvdata(pdev);
if (of_property_read_u32(child, "reg", &cs) < 0) {
dev_err(&pdev->dev, "%s has no 'reg' property\n",
@@ -2010,23 +2041,80 @@
if (ret < 0) {
dev_err(&pdev->dev, "cannot remap GPMC CS %d to %pa\n",
cs, &res.start);
+ if (res.start < GPMC_MEM_START) {
+ dev_info(&pdev->dev,
+ "GPMC CS %d start cannot be lesser than 0x%x\n",
+ cs, GPMC_MEM_START);
+ } else if (res.end > GPMC_MEM_END) {
+ dev_info(&pdev->dev,
+ "GPMC CS %d end cannot be greater than 0x%x\n",
+ cs, GPMC_MEM_END);
+ }
goto err;
}
- ret = of_property_read_u32(child, "bank-width", &gpmc_s.device_width);
- if (ret < 0)
- goto err;
+ if (of_node_cmp(child->name, "nand") == 0) {
+ /* Warn about older DT blobs with no compatible property */
+ if (!of_property_read_bool(child, "compatible")) {
+ dev_warn(&pdev->dev,
+ "Incompatible NAND node: missing compatible");
+ ret = -EINVAL;
+ goto err;
+ }
+ }
+
+ if (of_device_is_compatible(child, "ti,omap2-nand")) {
+ /* NAND specific setup */
+ val = 8;
+ of_property_read_u32(child, "nand-bus-width", &val);
+ switch (val) {
+ case 8:
+ gpmc_s.device_width = GPMC_DEVWIDTH_8BIT;
+ break;
+ case 16:
+ gpmc_s.device_width = GPMC_DEVWIDTH_16BIT;
+ break;
+ default:
+ dev_err(&pdev->dev, "%s: invalid 'nand-bus-width'\n",
+ child->name);
+ ret = -EINVAL;
+ goto err;
+ }
+
+ /* disable write protect */
+ gpmc_configure(GPMC_CONFIG_WP, 0);
+ gpmc_s.device_nand = true;
+ } else {
+ ret = of_property_read_u32(child, "bank-width",
+ &gpmc_s.device_width);
+ if (ret < 0)
+ goto err;
+ }
+
+ /* Reserve wait pin if it is required and valid */
+ if (gpmc_s.wait_on_read || gpmc_s.wait_on_write) {
+ unsigned int wait_pin = gpmc_s.wait_pin;
+
+ waitpin_desc = gpiochip_request_own_desc(&gpmc->gpio_chip,
+ wait_pin, "WAITPIN");
+ if (IS_ERR(waitpin_desc)) {
+ dev_err(&pdev->dev, "invalid wait-pin: %d\n", wait_pin);
+ ret = PTR_ERR(waitpin_desc);
+ goto err;
+ }
+ }
gpmc_cs_show_timings(cs, "before gpmc_cs_program_settings");
+
ret = gpmc_cs_program_settings(cs, &gpmc_s);
if (ret < 0)
- goto err;
+ goto err_cs;
ret = gpmc_cs_set_timings(cs, &gpmc_t, &gpmc_s);
if (ret) {
dev_err(&pdev->dev, "failed to set gpmc timings for: %s\n",
child->name);
- goto err;
+ goto err_cs;
}
/* Clear limited address i.e. enable A26-A11 */
@@ -2057,16 +2145,81 @@
dev_err(&pdev->dev, "failed to create gpmc child %s\n", child->name);
ret = -ENODEV;
+err_cs:
+ if (waitpin_desc)
+ gpiochip_free_own_desc(waitpin_desc);
+
err:
gpmc_cs_free(cs);
return ret;
}
+static int gpmc_gpio_get_direction(struct gpio_chip *chip, unsigned int offset)
+{
+ return 1; /* we're input only */
+}
+
+static int gpmc_gpio_direction_input(struct gpio_chip *chip,
+ unsigned int offset)
+{
+ return 0; /* we're input only */
+}
+
+static int gpmc_gpio_direction_output(struct gpio_chip *chip,
+ unsigned int offset, int value)
+{
+ return -EINVAL; /* we're input only */
+}
+
+static void gpmc_gpio_set(struct gpio_chip *chip, unsigned int offset,
+ int value)
+{
+}
+
+static int gpmc_gpio_get(struct gpio_chip *chip, unsigned int offset)
+{
+ u32 reg;
+
+ offset += 8;
+
+ reg = gpmc_read_reg(GPMC_STATUS) & BIT(offset);
+
+ return !!reg;
+}
+
+static int gpmc_gpio_init(struct gpmc_device *gpmc)
+{
+ int ret;
+
+ gpmc->gpio_chip.parent = gpmc->dev;
+ gpmc->gpio_chip.owner = THIS_MODULE;
+ gpmc->gpio_chip.label = DEVICE_NAME;
+ gpmc->gpio_chip.ngpio = gpmc_nr_waitpins;
+ gpmc->gpio_chip.get_direction = gpmc_gpio_get_direction;
+ gpmc->gpio_chip.direction_input = gpmc_gpio_direction_input;
+ gpmc->gpio_chip.direction_output = gpmc_gpio_direction_output;
+ gpmc->gpio_chip.set = gpmc_gpio_set;
+ gpmc->gpio_chip.get = gpmc_gpio_get;
+ gpmc->gpio_chip.base = -1;
+
+ ret = gpiochip_add(&gpmc->gpio_chip);
+ if (ret < 0) {
+ dev_err(gpmc->dev, "could not register gpio chip: %d\n", ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static void gpmc_gpio_exit(struct gpmc_device *gpmc)
+{
+ gpiochip_remove(&gpmc->gpio_chip);
+}
+
static int gpmc_probe_dt(struct platform_device *pdev)
{
int ret;
- struct device_node *child;
const struct of_device_id *of_id =
of_match_device(gpmc_dt_ids, &pdev->dev);
@@ -2094,17 +2247,26 @@
return ret;
}
+ return 0;
+}
+
+static int gpmc_probe_dt_children(struct platform_device *pdev)
+{
+ int ret;
+ struct device_node *child;
+
for_each_available_child_of_node(pdev->dev.of_node, child) {
if (!child->name)
continue;
- if (of_node_cmp(child->name, "nand") == 0)
- ret = gpmc_probe_nand_child(pdev, child);
- else if (of_node_cmp(child->name, "onenand") == 0)
+ if (of_node_cmp(child->name, "onenand") == 0)
ret = gpmc_probe_onenand_child(pdev, child);
else
ret = gpmc_probe_generic_child(pdev, child);
+
+ if (ret)
+ return ret;
}
return 0;
@@ -2114,6 +2276,11 @@
{
return 0;
}
+
+static int gpmc_probe_dt_children(struct platform_device *pdev)
+{
+ return 0;
+}
#endif
static int gpmc_probe(struct platform_device *pdev)
@@ -2121,6 +2288,14 @@
int rc;
u32 l;
struct resource *res;
+ struct gpmc_device *gpmc;
+
+ gpmc = devm_kzalloc(&pdev->dev, sizeof(*gpmc), GFP_KERNEL);
+ if (!gpmc)
+ return -ENOMEM;
+
+ gpmc->dev = &pdev->dev;
+ platform_set_drvdata(pdev, gpmc);
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
if (res == NULL)
@@ -2134,15 +2309,16 @@
return PTR_ERR(gpmc_base);
res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
- if (res == NULL)
- dev_warn(&pdev->dev, "Failed to get resource: irq\n");
- else
- gpmc_irq = res->start;
+ if (!res) {
+ dev_err(&pdev->dev, "Failed to get resource: irq\n");
+ return -ENOENT;
+ }
+
+ gpmc->irq = res->start;
gpmc_l3_clk = devm_clk_get(&pdev->dev, "fck");
if (IS_ERR(gpmc_l3_clk)) {
dev_err(&pdev->dev, "Failed to get GPMC fck\n");
- gpmc_irq = 0;
return PTR_ERR(gpmc_l3_clk);
}
@@ -2151,11 +2327,18 @@
return -EINVAL;
}
+ if (pdev->dev.of_node) {
+ rc = gpmc_probe_dt(pdev);
+ if (rc)
+ return rc;
+ } else {
+ gpmc_cs_num = GPMC_CS_NUM;
+ gpmc_nr_waitpins = GPMC_NR_WAITPINS;
+ }
+
pm_runtime_enable(&pdev->dev);
pm_runtime_get_sync(&pdev->dev);
- gpmc_dev = &pdev->dev;
-
l = gpmc_read_reg(GPMC_REVISION);
/*
@@ -2174,36 +2357,51 @@
gpmc_capability = GPMC_HAS_WR_ACCESS | GPMC_HAS_WR_DATA_MUX_BUS;
if (GPMC_REVISION_MAJOR(l) > 0x5)
gpmc_capability |= GPMC_HAS_MUX_AAD;
- dev_info(gpmc_dev, "GPMC revision %d.%d\n", GPMC_REVISION_MAJOR(l),
+ dev_info(gpmc->dev, "GPMC revision %d.%d\n", GPMC_REVISION_MAJOR(l),
GPMC_REVISION_MINOR(l));
gpmc_mem_init();
+ rc = gpmc_gpio_init(gpmc);
+ if (rc)
+ goto gpio_init_failed;
- if (gpmc_setup_irq() < 0)
- dev_warn(gpmc_dev, "gpmc_setup_irq failed\n");
-
- if (!pdev->dev.of_node) {
- gpmc_cs_num = GPMC_CS_NUM;
- gpmc_nr_waitpins = GPMC_NR_WAITPINS;
+ gpmc->nirqs = GPMC_NR_NAND_IRQS + gpmc_nr_waitpins;
+ rc = gpmc_setup_irq(gpmc);
+ if (rc) {
+ dev_err(gpmc->dev, "gpmc_setup_irq failed\n");
+ goto setup_irq_failed;
}
- rc = gpmc_probe_dt(pdev);
+ rc = gpmc_probe_dt_children(pdev);
if (rc < 0) {
- pm_runtime_put_sync(&pdev->dev);
- dev_err(gpmc_dev, "failed to probe DT parameters\n");
- return rc;
+ dev_err(gpmc->dev, "failed to probe DT children\n");
+ goto dt_children_failed;
}
return 0;
+
+dt_children_failed:
+ gpmc_free_irq(gpmc);
+setup_irq_failed:
+ gpmc_gpio_exit(gpmc);
+gpio_init_failed:
+ gpmc_mem_exit();
+ pm_runtime_put_sync(&pdev->dev);
+ pm_runtime_disable(&pdev->dev);
+
+ return rc;
}
static int gpmc_remove(struct platform_device *pdev)
{
- gpmc_free_irq();
+ struct gpmc_device *gpmc = platform_get_drvdata(pdev);
+
+ gpmc_free_irq(gpmc);
+ gpmc_gpio_exit(gpmc);
gpmc_mem_exit();
pm_runtime_put_sync(&pdev->dev);
pm_runtime_disable(&pdev->dev);
- gpmc_dev = NULL;
+
return 0;
}
@@ -2249,25 +2447,6 @@
postcore_initcall(gpmc_init);
module_exit(gpmc_exit);
-static irqreturn_t gpmc_handle_irq(int irq, void *dev)
-{
- int i;
- u32 regval;
-
- regval = gpmc_read_reg(GPMC_IRQSTATUS);
-
- if (!regval)
- return IRQ_NONE;
-
- for (i = 0; i < GPMC_NR_IRQ; i++)
- if (regval & gpmc_client_irq[i].bitmask)
- generic_handle_irq(gpmc_client_irq[i].irq);
-
- gpmc_write_reg(GPMC_IRQSTATUS, regval);
-
- return IRQ_HANDLED;
-}
-
static struct omap3_gpmc_regs gpmc_context;
void omap3_gpmc_save_context(void)
diff --git a/drivers/memstick/core/mspro_block.c b/drivers/memstick/core/mspro_block.c
index 922a750..0fb27d3 100644
--- a/drivers/memstick/core/mspro_block.c
+++ b/drivers/memstick/core/mspro_block.c
@@ -1033,12 +1033,11 @@
}
msb->attr_group.name = "media_attributes";
- buffer = kmalloc(attr_len, GFP_KERNEL);
+ buffer = kmemdup(attr, attr_len, GFP_KERNEL);
if (!buffer) {
rc = -ENOMEM;
goto out_free_attr;
}
- memcpy(buffer, (char *)attr, attr_len);
for (cnt = 0; cnt < attr_count; ++cnt) {
s_attr = kzalloc(sizeof(struct mspro_sys_attr), GFP_KERNEL);
diff --git a/drivers/memstick/host/rtsx_usb_ms.c b/drivers/memstick/host/rtsx_usb_ms.c
index 1105db2..d34bc35 100644
--- a/drivers/memstick/host/rtsx_usb_ms.c
+++ b/drivers/memstick/host/rtsx_usb_ms.c
@@ -706,7 +706,7 @@
if (host->eject)
break;
- msleep(1000);
+ schedule_timeout_idle(HZ);
}
complete(&host->detect_ms_exit);
diff --git a/drivers/message/fusion/mptsas.c b/drivers/message/fusion/mptsas.c
index 7ebccfa..7ee1667 100644
--- a/drivers/message/fusion/mptsas.c
+++ b/drivers/message/fusion/mptsas.c
@@ -2281,7 +2281,7 @@
dma_addr_out = pci_map_single(ioc->pcidev, bio_data(req->bio),
blk_rq_bytes(req), PCI_DMA_BIDIRECTIONAL);
- if (!dma_addr_out)
+ if (pci_dma_mapping_error(ioc->pcidev, dma_addr_out))
goto put_mf;
ioc->add_sge(psge, flagsLength, dma_addr_out);
psge += ioc->SGE_size;
@@ -2296,7 +2296,7 @@
flagsLength |= blk_rq_bytes(rsp) + 4;
dma_addr_in = pci_map_single(ioc->pcidev, bio_data(rsp->bio),
blk_rq_bytes(rsp), PCI_DMA_BIDIRECTIONAL);
- if (!dma_addr_in)
+ if (pci_dma_mapping_error(ioc->pcidev, dma_addr_in))
goto unmap;
ioc->add_sge(psge, flagsLength, dma_addr_in);
diff --git a/drivers/message/fusion/mptspi.c b/drivers/message/fusion/mptspi.c
index 613231c..031e088 100644
--- a/drivers/message/fusion/mptspi.c
+++ b/drivers/message/fusion/mptspi.c
@@ -1150,7 +1150,7 @@
}
shost_printk(KERN_INFO, shost, MYIOC_s_FMT
"Integrated RAID detects new device %d\n", ioc->name, disk);
- scsi_scan_target(&ioc->sh->shost_gendev, 1, disk, 0, 1);
+ scsi_scan_target(&ioc->sh->shost_gendev, 1, disk, 0, SCSI_SCAN_RESCAN);
}
diff --git a/drivers/mfd/Kconfig b/drivers/mfd/Kconfig
index eea61e3..1bcf601 100644
--- a/drivers/mfd/Kconfig
+++ b/drivers/mfd/Kconfig
@@ -134,7 +134,7 @@
select MFD_CORE
select CHROME_PLATFORMS
select CROS_EC_PROTO
- depends on X86 || ARM || COMPILE_TEST
+ depends on X86 || ARM || ARM64 || COMPILE_TEST
help
If you say Y here you get support for the ChromeOS Embedded
Controller (EC) providing keyboard, battery and power services.
@@ -319,6 +319,16 @@
menus in order to enable them.
We communicate with the Hi6421 via memory-mapped I/O.
+config MFD_HI655X_PMIC
+ tristate "HiSilicon Hi655X series PMU/Codec IC"
+ depends on ARCH_HISI || COMPILE_TEST
+ depends on OF
+ select MFD_CORE
+ select REGMAP_MMIO
+ select REGMAP_IRQ
+ help
+ Select this option to enable Hisilicon hi655x series pmic driver.
+
config HTC_EGPIO
bool "HTC EGPIO support"
depends on GPIOLIB && ARM
@@ -527,6 +537,21 @@
additional drivers must be enabled in order to use the functionality
of the device.
+config MFD_MAX77620
+ bool "Maxim Semiconductor MAX77620 and MAX20024 PMIC Support"
+ depends on I2C=y
+ depends on OF
+ select MFD_CORE
+ select REGMAP_I2C
+ select REGMAP_IRQ
+ select IRQ_DOMAIN
+ help
+ Say yes here to add support for Maxim Semiconductor MAX77620 and
+ MAX20024 which are Power Management IC with General purpose pins,
+ RTC, regulators, clock generator, watchdog etc. This driver
+ provides common support for accessing the device; additional drivers
+ must be enabled in order to use the functionality of the device.
+
config MFD_MAX77686
tristate "Maxim Semiconductor MAX77686/802 PMIC Support"
depends on I2C
@@ -543,8 +568,8 @@
of the device.
config MFD_MAX77693
- bool "Maxim Semiconductor MAX77693 PMIC Support"
- depends on I2C=y
+ tristate "Maxim Semiconductor MAX77693 PMIC Support"
+ depends on I2C
select MFD_CORE
select REGMAP_I2C
select REGMAP_IRQ
@@ -1568,7 +1593,7 @@
config MFD_VEXPRESS_SYSREG
bool "Versatile Express System Registers"
- depends on VEXPRESS_CONFIG && GPIOLIB
+ depends on VEXPRESS_CONFIG && GPIOLIB && !ARCH_USES_GETTIMEOFFSET
default y
select CLKSRC_MMIO
select GPIO_GENERIC_PLATFORM
diff --git a/drivers/mfd/Makefile b/drivers/mfd/Makefile
index 5eaa6465d..42a66e1 100644
--- a/drivers/mfd/Makefile
+++ b/drivers/mfd/Makefile
@@ -128,6 +128,7 @@
obj-$(CONFIG_MFD_DA9150) += da9150-core.o
obj-$(CONFIG_MFD_MAX14577) += max14577.o
+obj-$(CONFIG_MFD_MAX77620) += max77620.o
obj-$(CONFIG_MFD_MAX77686) += max77686.o
obj-$(CONFIG_MFD_MAX77693) += max77693.o
obj-$(CONFIG_MFD_MAX77843) += max77843.o
@@ -195,6 +196,7 @@
obj-$(CONFIG_MFD_IPAQ_MICRO) += ipaq-micro.o
obj-$(CONFIG_MFD_MENF21BMC) += menf21bmc.o
obj-$(CONFIG_MFD_HI6421_PMIC) += hi6421-pmic-core.o
+obj-$(CONFIG_MFD_HI655X_PMIC) += hi655x-pmic.o
obj-$(CONFIG_MFD_DLN2) += dln2.o
obj-$(CONFIG_MFD_RT5033) += rt5033.o
obj-$(CONFIG_MFD_SKY81452) += sky81452.o
diff --git a/drivers/mfd/act8945a.c b/drivers/mfd/act8945a.c
index 525b546..10c6d2d 100644
--- a/drivers/mfd/act8945a.c
+++ b/drivers/mfd/act8945a.c
@@ -46,8 +46,9 @@
i2c_set_clientdata(i2c, regmap);
- ret = mfd_add_devices(&i2c->dev, PLATFORM_DEVID_NONE, act8945a_devs,
- ARRAY_SIZE(act8945a_devs), NULL, 0, NULL);
+ ret = devm_mfd_add_devices(&i2c->dev, PLATFORM_DEVID_NONE,
+ act8945a_devs, ARRAY_SIZE(act8945a_devs),
+ NULL, 0, NULL);
if (ret) {
dev_err(&i2c->dev, "Failed to add sub devices\n");
return ret;
@@ -56,13 +57,6 @@
return 0;
}
-static int act8945a_i2c_remove(struct i2c_client *i2c)
-{
- mfd_remove_devices(&i2c->dev);
-
- return 0;
-}
-
static const struct i2c_device_id act8945a_i2c_id[] = {
{ "act8945a", 0 },
{}
@@ -81,7 +75,6 @@
.of_match_table = of_match_ptr(act8945a_of_match),
},
.probe = act8945a_i2c_probe,
- .remove = act8945a_i2c_remove,
.id_table = act8945a_i2c_id,
};
diff --git a/drivers/mfd/arizona-core.c b/drivers/mfd/arizona-core.c
index 5319f25..bf27179 100644
--- a/drivers/mfd/arizona-core.c
+++ b/drivers/mfd/arizona-core.c
@@ -908,12 +908,12 @@
static const struct mfd_cell wm5102_devs[] = {
{ .name = "arizona-micsupp" },
+ { .name = "arizona-gpio" },
{
.name = "arizona-extcon",
.parent_supplies = wm5102_supplies,
.num_parent_supplies = 1, /* We only need MICVDD */
},
- { .name = "arizona-gpio" },
{ .name = "arizona-haptics" },
{ .name = "arizona-pwm" },
{
@@ -925,12 +925,12 @@
static const struct mfd_cell wm5110_devs[] = {
{ .name = "arizona-micsupp" },
+ { .name = "arizona-gpio" },
{
.name = "arizona-extcon",
.parent_supplies = wm5102_supplies,
.num_parent_supplies = 1, /* We only need MICVDD */
},
- { .name = "arizona-gpio" },
{ .name = "arizona-haptics" },
{ .name = "arizona-pwm" },
{
@@ -966,12 +966,12 @@
static const struct mfd_cell wm8997_devs[] = {
{ .name = "arizona-micsupp" },
+ { .name = "arizona-gpio" },
{
.name = "arizona-extcon",
.parent_supplies = wm8997_supplies,
.num_parent_supplies = 1, /* We only need MICVDD */
},
- { .name = "arizona-gpio" },
{ .name = "arizona-haptics" },
{ .name = "arizona-pwm" },
{
@@ -982,12 +982,13 @@
};
static const struct mfd_cell wm8998_devs[] = {
+ { .name = "arizona-micsupp" },
+ { .name = "arizona-gpio" },
{
.name = "arizona-extcon",
.parent_supplies = wm5102_supplies,
.num_parent_supplies = 1, /* We only need MICVDD */
},
- { .name = "arizona-gpio" },
{ .name = "arizona-haptics" },
{ .name = "arizona-pwm" },
{
@@ -995,7 +996,6 @@
.parent_supplies = wm5102_supplies,
.num_parent_supplies = ARRAY_SIZE(wm5102_supplies),
},
- { .name = "arizona-micsupp" },
};
int arizona_dev_init(struct arizona *arizona)
diff --git a/drivers/mfd/arizona-irq.c b/drivers/mfd/arizona-irq.c
index 5fef014..edeb495 100644
--- a/drivers/mfd/arizona-irq.c
+++ b/drivers/mfd/arizona-irq.c
@@ -168,12 +168,15 @@
.irq_set_wake = arizona_irq_set_wake,
};
+static struct lock_class_key arizona_irq_lock_class;
+
static int arizona_irq_map(struct irq_domain *h, unsigned int virq,
irq_hw_number_t hw)
{
struct arizona *data = h->host_data;
irq_set_chip_data(virq, data);
+ irq_set_lockdep_class(virq, &arizona_irq_lock_class);
irq_set_chip_and_handler(virq, &arizona_irq_chip, handle_simple_irq);
irq_set_nested_thread(virq, 1);
irq_set_noprobe(virq);
diff --git a/drivers/mfd/as3711.c b/drivers/mfd/as3711.c
index 09e1483..67b1241 100644
--- a/drivers/mfd/as3711.c
+++ b/drivers/mfd/as3711.c
@@ -189,22 +189,14 @@
as3711_subdevs[AS3711_BACKLIGHT].pdata_size = 0;
}
- ret = mfd_add_devices(as3711->dev, -1, as3711_subdevs,
- ARRAY_SIZE(as3711_subdevs), NULL, 0, NULL);
+ ret = devm_mfd_add_devices(as3711->dev, -1, as3711_subdevs,
+ ARRAY_SIZE(as3711_subdevs), NULL, 0, NULL);
if (ret < 0)
dev_err(&client->dev, "add mfd devices failed: %d\n", ret);
return ret;
}
-static int as3711_i2c_remove(struct i2c_client *client)
-{
- struct as3711 *as3711 = i2c_get_clientdata(client);
-
- mfd_remove_devices(as3711->dev);
- return 0;
-}
-
static const struct i2c_device_id as3711_i2c_id[] = {
{.name = "as3711", .driver_data = 0},
{}
@@ -218,7 +210,6 @@
.of_match_table = of_match_ptr(as3711_of_match),
},
.probe = as3711_i2c_probe,
- .remove = as3711_i2c_remove,
.id_table = as3711_i2c_id,
};
diff --git a/drivers/mfd/as3722.c b/drivers/mfd/as3722.c
index e1f597f..f87342c 100644
--- a/drivers/mfd/as3722.c
+++ b/drivers/mfd/as3722.c
@@ -385,9 +385,10 @@
return ret;
irq_flags = as3722->irq_flags | IRQF_ONESHOT;
- ret = regmap_add_irq_chip(as3722->regmap, as3722->chip_irq,
- irq_flags, -1, &as3722_irq_chip,
- &as3722->irq_data);
+ ret = devm_regmap_add_irq_chip(as3722->dev, as3722->regmap,
+ as3722->chip_irq,
+ irq_flags, -1, &as3722_irq_chip,
+ &as3722->irq_data);
if (ret < 0) {
dev_err(as3722->dev, "Failed to add regmap irq: %d\n", ret);
return ret;
@@ -395,33 +396,20 @@
ret = as3722_configure_pullups(as3722);
if (ret < 0)
- goto scrub;
+ return ret;
- ret = mfd_add_devices(&i2c->dev, -1, as3722_devs,
- ARRAY_SIZE(as3722_devs), NULL, 0,
- regmap_irq_get_domain(as3722->irq_data));
+ ret = devm_mfd_add_devices(&i2c->dev, -1, as3722_devs,
+ ARRAY_SIZE(as3722_devs), NULL, 0,
+ regmap_irq_get_domain(as3722->irq_data));
if (ret) {
dev_err(as3722->dev, "Failed to add MFD devices: %d\n", ret);
- goto scrub;
+ return ret;
}
device_init_wakeup(as3722->dev, true);
dev_dbg(as3722->dev, "AS3722 core driver initialized successfully\n");
return 0;
-
-scrub:
- regmap_del_irq_chip(as3722->chip_irq, as3722->irq_data);
- return ret;
-}
-
-static int as3722_i2c_remove(struct i2c_client *i2c)
-{
- struct as3722 *as3722 = i2c_get_clientdata(i2c);
-
- mfd_remove_devices(as3722->dev);
- regmap_del_irq_chip(as3722->chip_irq, as3722->irq_data);
- return 0;
}
static int __maybe_unused as3722_i2c_suspend(struct device *dev)
@@ -470,7 +458,6 @@
.pm = &as3722_pm_ops,
},
.probe = as3722_i2c_probe,
- .remove = as3722_i2c_remove,
.id_table = as3722_i2c_id,
};
diff --git a/drivers/mfd/asic3.c b/drivers/mfd/asic3.c
index 4dca6bc..0413c81 100644
--- a/drivers/mfd/asic3.c
+++ b/drivers/mfd/asic3.c
@@ -446,7 +446,7 @@
unsigned long flags;
struct asic3 *asic;
- asic = container_of(chip, struct asic3, gpio);
+ asic = gpiochip_get_data(chip);
gpio_base = ASIC3_GPIO_TO_BASE(offset);
if (gpio_base > ASIC3_GPIO_D_BASE) {
@@ -492,7 +492,7 @@
u32 mask = ASIC3_GPIO_TO_MASK(offset);
struct asic3 *asic;
- asic = container_of(chip, struct asic3, gpio);
+ asic = gpiochip_get_data(chip);
gpio_base = ASIC3_GPIO_TO_BASE(offset);
if (gpio_base > ASIC3_GPIO_D_BASE) {
@@ -513,7 +513,7 @@
unsigned long flags;
struct asic3 *asic;
- asic = container_of(chip, struct asic3, gpio);
+ asic = gpiochip_get_data(chip);
gpio_base = ASIC3_GPIO_TO_BASE(offset);
if (gpio_base > ASIC3_GPIO_D_BASE) {
@@ -540,7 +540,7 @@
static int asic3_gpio_to_irq(struct gpio_chip *chip, unsigned offset)
{
- struct asic3 *asic = container_of(chip, struct asic3, gpio);
+ struct asic3 *asic = gpiochip_get_data(chip);
return asic->irq_base + offset;
}
@@ -595,7 +595,7 @@
alt_reg[i]);
}
- return gpiochip_add(&asic->gpio);
+ return gpiochip_add_data(&asic->gpio, asic);
}
static int asic3_gpio_remove(struct platform_device *pdev)
diff --git a/drivers/mfd/atmel-hlcdc.c b/drivers/mfd/atmel-hlcdc.c
index 06c2058..eca7ea6 100644
--- a/drivers/mfd/atmel-hlcdc.c
+++ b/drivers/mfd/atmel-hlcdc.c
@@ -128,16 +128,9 @@
dev_set_drvdata(dev, hlcdc);
- return mfd_add_devices(dev, -1, atmel_hlcdc_cells,
- ARRAY_SIZE(atmel_hlcdc_cells),
- NULL, 0, NULL);
-}
-
-static int atmel_hlcdc_remove(struct platform_device *pdev)
-{
- mfd_remove_devices(&pdev->dev);
-
- return 0;
+ return devm_mfd_add_devices(dev, -1, atmel_hlcdc_cells,
+ ARRAY_SIZE(atmel_hlcdc_cells),
+ NULL, 0, NULL);
}
static const struct of_device_id atmel_hlcdc_match[] = {
@@ -152,7 +145,6 @@
static struct platform_driver atmel_hlcdc_driver = {
.probe = atmel_hlcdc_probe,
- .remove = atmel_hlcdc_remove,
.driver = {
.name = "atmel-hlcdc",
.of_match_table = atmel_hlcdc_match,
diff --git a/drivers/mfd/axp20x-rsb.c b/drivers/mfd/axp20x-rsb.c
index 28c2024..a407527 100644
--- a/drivers/mfd/axp20x-rsb.c
+++ b/drivers/mfd/axp20x-rsb.c
@@ -61,6 +61,7 @@
static const struct of_device_id axp20x_rsb_of_match[] = {
{ .compatible = "x-powers,axp223", .data = (void *)AXP223_ID },
+ { .compatible = "x-powers,axp809", .data = (void *)AXP809_ID },
{ },
};
MODULE_DEVICE_TABLE(of, axp20x_rsb_of_match);
diff --git a/drivers/mfd/axp20x.c b/drivers/mfd/axp20x.c
index a57d6e9..e4e3297 100644
--- a/drivers/mfd/axp20x.c
+++ b/drivers/mfd/axp20x.c
@@ -37,6 +37,7 @@
"AXP221",
"AXP223",
"AXP288",
+ "AXP809",
};
static const struct regmap_range axp152_writeable_ranges[] = {
@@ -85,6 +86,7 @@
.n_yes_ranges = ARRAY_SIZE(axp20x_volatile_ranges),
};
+/* AXP22x ranges are shared with the AXP809, as they cover the same range */
static const struct regmap_range axp22x_writeable_ranges[] = {
regmap_reg_range(AXP20X_DATACACHE(0), AXP20X_IRQ5_STATE),
regmap_reg_range(AXP20X_DCDC_MODE, AXP22X_BATLOW_THRES1),
@@ -128,6 +130,12 @@
DEFINE_RES_IRQ_NAMED(AXP152_IRQ_PEK_FAL_EDGE, "PEK_DBF"),
};
+static struct resource axp20x_ac_power_supply_resources[] = {
+ DEFINE_RES_IRQ_NAMED(AXP20X_IRQ_ACIN_PLUGIN, "ACIN_PLUGIN"),
+ DEFINE_RES_IRQ_NAMED(AXP20X_IRQ_ACIN_REMOVAL, "ACIN_REMOVAL"),
+ DEFINE_RES_IRQ_NAMED(AXP20X_IRQ_ACIN_OVER_V, "ACIN_OVER_V"),
+};
+
static struct resource axp20x_pek_resources[] = {
{
.name = "PEK_DBR",
@@ -211,6 +219,20 @@
},
};
+static struct resource axp809_pek_resources[] = {
+ {
+ .name = "PEK_DBR",
+ .start = AXP809_IRQ_PEK_RIS_EDGE,
+ .end = AXP809_IRQ_PEK_RIS_EDGE,
+ .flags = IORESOURCE_IRQ,
+ }, {
+ .name = "PEK_DBF",
+ .start = AXP809_IRQ_PEK_FAL_EDGE,
+ .end = AXP809_IRQ_PEK_FAL_EDGE,
+ .flags = IORESOURCE_IRQ,
+ },
+};
+
static const struct regmap_config axp152_regmap_config = {
.reg_bits = 8,
.val_bits = 8,
@@ -378,6 +400,41 @@
INIT_REGMAP_IRQ(AXP288, BC_USB_CHNG, 5, 1),
};
+static const struct regmap_irq axp809_regmap_irqs[] = {
+ INIT_REGMAP_IRQ(AXP809, ACIN_OVER_V, 0, 7),
+ INIT_REGMAP_IRQ(AXP809, ACIN_PLUGIN, 0, 6),
+ INIT_REGMAP_IRQ(AXP809, ACIN_REMOVAL, 0, 5),
+ INIT_REGMAP_IRQ(AXP809, VBUS_OVER_V, 0, 4),
+ INIT_REGMAP_IRQ(AXP809, VBUS_PLUGIN, 0, 3),
+ INIT_REGMAP_IRQ(AXP809, VBUS_REMOVAL, 0, 2),
+ INIT_REGMAP_IRQ(AXP809, VBUS_V_LOW, 0, 1),
+ INIT_REGMAP_IRQ(AXP809, BATT_PLUGIN, 1, 7),
+ INIT_REGMAP_IRQ(AXP809, BATT_REMOVAL, 1, 6),
+ INIT_REGMAP_IRQ(AXP809, BATT_ENT_ACT_MODE, 1, 5),
+ INIT_REGMAP_IRQ(AXP809, BATT_EXIT_ACT_MODE, 1, 4),
+ INIT_REGMAP_IRQ(AXP809, CHARG, 1, 3),
+ INIT_REGMAP_IRQ(AXP809, CHARG_DONE, 1, 2),
+ INIT_REGMAP_IRQ(AXP809, BATT_CHG_TEMP_HIGH, 2, 7),
+ INIT_REGMAP_IRQ(AXP809, BATT_CHG_TEMP_HIGH_END, 2, 6),
+ INIT_REGMAP_IRQ(AXP809, BATT_CHG_TEMP_LOW, 2, 5),
+ INIT_REGMAP_IRQ(AXP809, BATT_CHG_TEMP_LOW_END, 2, 4),
+ INIT_REGMAP_IRQ(AXP809, BATT_ACT_TEMP_HIGH, 2, 3),
+ INIT_REGMAP_IRQ(AXP809, BATT_ACT_TEMP_HIGH_END, 2, 2),
+ INIT_REGMAP_IRQ(AXP809, BATT_ACT_TEMP_LOW, 2, 1),
+ INIT_REGMAP_IRQ(AXP809, BATT_ACT_TEMP_LOW_END, 2, 0),
+ INIT_REGMAP_IRQ(AXP809, DIE_TEMP_HIGH, 3, 7),
+ INIT_REGMAP_IRQ(AXP809, LOW_PWR_LVL1, 3, 1),
+ INIT_REGMAP_IRQ(AXP809, LOW_PWR_LVL2, 3, 0),
+ INIT_REGMAP_IRQ(AXP809, TIMER, 4, 7),
+ INIT_REGMAP_IRQ(AXP809, PEK_RIS_EDGE, 4, 6),
+ INIT_REGMAP_IRQ(AXP809, PEK_FAL_EDGE, 4, 5),
+ INIT_REGMAP_IRQ(AXP809, PEK_SHORT, 4, 4),
+ INIT_REGMAP_IRQ(AXP809, PEK_LONG, 4, 3),
+ INIT_REGMAP_IRQ(AXP809, PEK_OVER_OFF, 4, 2),
+ INIT_REGMAP_IRQ(AXP809, GPIO1_INPUT, 4, 1),
+ INIT_REGMAP_IRQ(AXP809, GPIO0_INPUT, 4, 0),
+};
+
static const struct regmap_irq_chip axp152_regmap_irq_chip = {
.name = "axp152_irq_chip",
.status_base = AXP152_IRQ1_STATE,
@@ -428,6 +485,18 @@
};
+static const struct regmap_irq_chip axp809_regmap_irq_chip = {
+ .name = "axp809",
+ .status_base = AXP20X_IRQ1_STATE,
+ .ack_base = AXP20X_IRQ1_STATE,
+ .mask_base = AXP20X_IRQ1_EN,
+ .mask_invert = true,
+ .init_ack_masked = true,
+ .irqs = axp809_regmap_irqs,
+ .num_irqs = ARRAY_SIZE(axp809_regmap_irqs),
+ .num_regs = 5,
+};
+
static struct mfd_cell axp20x_cells[] = {
{
.name = "axp20x-pek",
@@ -436,6 +505,11 @@
}, {
.name = "axp20x-regulator",
}, {
+ .name = "axp20x-ac-power-supply",
+ .of_compatible = "x-powers,axp202-ac-power-supply",
+ .num_resources = ARRAY_SIZE(axp20x_ac_power_supply_resources),
+ .resources = axp20x_ac_power_supply_resources,
+ }, {
.name = "axp20x-usb-power-supply",
.of_compatible = "x-powers,axp202-usb-power-supply",
.num_resources = ARRAY_SIZE(axp20x_usb_power_supply_resources),
@@ -572,6 +646,16 @@
},
};
+static struct mfd_cell axp809_cells[] = {
+ {
+ .name = "axp20x-pek",
+ .num_resources = ARRAY_SIZE(axp809_pek_resources),
+ .resources = axp809_pek_resources,
+ }, {
+ .name = "axp20x-regulator",
+ },
+};
+
static struct axp20x_dev *axp20x_pm_power_off;
static void axp20x_power_off(void)
{
@@ -631,6 +715,12 @@
axp20x->regmap_cfg = &axp288_regmap_config;
axp20x->regmap_irq_chip = &axp288_regmap_irq_chip;
break;
+ case AXP809_ID:
+ axp20x->nr_cells = ARRAY_SIZE(axp809_cells);
+ axp20x->cells = axp809_cells;
+ axp20x->regmap_cfg = &axp22x_regmap_config;
+ axp20x->regmap_irq_chip = &axp809_regmap_irq_chip;
+ break;
default:
dev_err(dev, "unsupported AXP20X ID %lu\n", axp20x->variant);
return -EINVAL;
diff --git a/drivers/mfd/bcm590xx.c b/drivers/mfd/bcm590xx.c
index 320aaef..0d76d69 100644
--- a/drivers/mfd/bcm590xx.c
+++ b/drivers/mfd/bcm590xx.c
@@ -82,8 +82,8 @@
goto err;
}
- ret = mfd_add_devices(&i2c_pri->dev, -1, bcm590xx_devs,
- ARRAY_SIZE(bcm590xx_devs), NULL, 0, NULL);
+ ret = devm_mfd_add_devices(&i2c_pri->dev, -1, bcm590xx_devs,
+ ARRAY_SIZE(bcm590xx_devs), NULL, 0, NULL);
if (ret < 0) {
dev_err(&i2c_pri->dev, "failed to add sub-devices: %d\n", ret);
goto err;
@@ -96,12 +96,6 @@
return ret;
}
-static int bcm590xx_i2c_remove(struct i2c_client *i2c)
-{
- mfd_remove_devices(&i2c->dev);
- return 0;
-}
-
static const struct of_device_id bcm590xx_of_match[] = {
{ .compatible = "brcm,bcm59056" },
{ }
@@ -120,7 +114,6 @@
.of_match_table = of_match_ptr(bcm590xx_of_match),
},
.probe = bcm590xx_i2c_probe,
- .remove = bcm590xx_i2c_remove,
.id_table = bcm590xx_i2c_id,
};
module_i2c_driver(bcm590xx_i2c_driver);
diff --git a/drivers/mfd/da9063-irq.c b/drivers/mfd/da9063-irq.c
index 2630263..7e903fc 100644
--- a/drivers/mfd/da9063-irq.c
+++ b/drivers/mfd/da9063-irq.c
@@ -25,14 +25,6 @@
#define DA9063_REG_EVENT_B_OFFSET 1
#define DA9063_REG_EVENT_C_OFFSET 2
#define DA9063_REG_EVENT_D_OFFSET 3
-#define EVENTS_BUF_LEN 4
-
-static const u8 mask_events_buf[] = { [0 ... (EVENTS_BUF_LEN - 1)] = ~0 };
-
-struct da9063_irq_data {
- u16 reg;
- u8 mask;
-};
static const struct regmap_irq da9063_irqs[] = {
/* DA9063 event A register */
diff --git a/drivers/mfd/dm355evm_msp.c b/drivers/mfd/dm355evm_msp.c
index ec4438e..14661ec 100644
--- a/drivers/mfd/dm355evm_msp.c
+++ b/drivers/mfd/dm355evm_msp.c
@@ -33,25 +33,25 @@
* This driver was tested with firmware revision A4.
*/
-#if defined(CONFIG_INPUT_DM355EVM) || defined(CONFIG_INPUT_DM355EVM_MODULE)
+#if IS_ENABLED(CONFIG_INPUT_DM355EVM)
#define msp_has_keyboard() true
#else
#define msp_has_keyboard() false
#endif
-#if defined(CONFIG_LEDS_GPIO) || defined(CONFIG_LEDS_GPIO_MODULE)
+#if IS_ENABLED(CONFIG_LEDS_GPIO)
#define msp_has_leds() true
#else
#define msp_has_leds() false
#endif
-#if defined(CONFIG_RTC_DRV_DM355EVM) || defined(CONFIG_RTC_DRV_DM355EVM_MODULE)
+#if IS_ENABLED(CONFIG_RTC_DRV_DM355EVM)
#define msp_has_rtc() true
#else
#define msp_has_rtc() false
#endif
-#if defined(CONFIG_VIDEO_TVP514X) || defined(CONFIG_VIDEO_TVP514X_MODULE)
+#if IS_ENABLED(CONFIG_VIDEO_TVP514X)
#define msp_has_tvp() true
#else
#define msp_has_tvp() false
@@ -260,7 +260,7 @@
/* GPIO-ish stuff */
dm355evm_msp_gpio.parent = &client->dev;
- status = gpiochip_add(&dm355evm_msp_gpio);
+ status = gpiochip_add_data(&dm355evm_msp_gpio, NULL);
if (status < 0)
return status;
diff --git a/drivers/mfd/hi6421-pmic-core.c b/drivers/mfd/hi6421-pmic-core.c
index f9ded45..3fd703f 100644
--- a/drivers/mfd/hi6421-pmic-core.c
+++ b/drivers/mfd/hi6421-pmic-core.c
@@ -76,8 +76,8 @@
platform_set_drvdata(pdev, pmic);
- ret = mfd_add_devices(&pdev->dev, 0, hi6421_devs,
- ARRAY_SIZE(hi6421_devs), NULL, 0, NULL);
+ ret = devm_mfd_add_devices(&pdev->dev, 0, hi6421_devs,
+ ARRAY_SIZE(hi6421_devs), NULL, 0, NULL);
if (ret) {
dev_err(&pdev->dev, "add mfd devices failed: %d\n", ret);
return ret;
@@ -86,13 +86,6 @@
return 0;
}
-static int hi6421_pmic_remove(struct platform_device *pdev)
-{
- mfd_remove_devices(&pdev->dev);
-
- return 0;
-}
-
static const struct of_device_id of_hi6421_pmic_match_tbl[] = {
{ .compatible = "hisilicon,hi6421-pmic", },
{ },
@@ -105,7 +98,6 @@
.of_match_table = of_hi6421_pmic_match_tbl,
},
.probe = hi6421_pmic_probe,
- .remove = hi6421_pmic_remove,
};
module_platform_driver(hi6421_pmic_driver);
diff --git a/drivers/mfd/hi655x-pmic.c b/drivers/mfd/hi655x-pmic.c
new file mode 100644
index 0000000..05ddc78
--- /dev/null
+++ b/drivers/mfd/hi655x-pmic.c
@@ -0,0 +1,162 @@
+/*
+ * Device driver for MFD hi655x PMIC
+ *
+ * Copyright (c) 2016 Hisilicon.
+ *
+ * Authors:
+ * Chen Feng <puck.chen@hisilicon.com>
+ * Fei Wang <w.f@huawei.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/gpio.h>
+#include <linux/io.h>
+#include <linux/interrupt.h>
+#include <linux/init.h>
+#include <linux/mfd/core.h>
+#include <linux/mfd/hi655x-pmic.h>
+#include <linux/module.h>
+#include <linux/of_gpio.h>
+#include <linux/of_platform.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+
+static const struct mfd_cell hi655x_pmic_devs[] = {
+ { .name = "hi655x-regulator", },
+};
+
+static const struct regmap_irq hi655x_irqs[] = {
+ { .reg_offset = 0, .mask = OTMP_D1R_INT },
+ { .reg_offset = 0, .mask = VSYS_2P5_R_INT },
+ { .reg_offset = 0, .mask = VSYS_UV_D3R_INT },
+ { .reg_offset = 0, .mask = VSYS_6P0_D200UR_INT },
+ { .reg_offset = 0, .mask = PWRON_D4SR_INT },
+ { .reg_offset = 0, .mask = PWRON_D20F_INT },
+ { .reg_offset = 0, .mask = PWRON_D20R_INT },
+ { .reg_offset = 0, .mask = RESERVE_INT },
+};
+
+static const struct regmap_irq_chip hi655x_irq_chip = {
+ .name = "hi655x-pmic",
+ .irqs = hi655x_irqs,
+ .num_regs = 1,
+ .num_irqs = ARRAY_SIZE(hi655x_irqs),
+ .status_base = HI655X_IRQ_STAT_BASE,
+ .mask_base = HI655X_IRQ_MASK_BASE,
+};
+
+static struct regmap_config hi655x_regmap_config = {
+ .reg_bits = 32,
+ .reg_stride = HI655X_STRIDE,
+ .val_bits = 8,
+ .max_register = HI655X_BUS_ADDR(0xFFF),
+};
+
+static void hi655x_local_irq_clear(struct regmap *map)
+{
+ int i;
+
+ regmap_write(map, HI655X_ANA_IRQM_BASE, HI655X_IRQ_CLR);
+ for (i = 0; i < HI655X_IRQ_ARRAY; i++) {
+ regmap_write(map, HI655X_IRQ_STAT_BASE + i * HI655X_STRIDE,
+ HI655X_IRQ_CLR);
+ }
+}
+
+static int hi655x_pmic_probe(struct platform_device *pdev)
+{
+ int ret;
+ struct hi655x_pmic *pmic;
+ struct device *dev = &pdev->dev;
+ struct device_node *np = dev->of_node;
+ void __iomem *base;
+
+ pmic = devm_kzalloc(dev, sizeof(*pmic), GFP_KERNEL);
+ if (!pmic)
+ return -ENOMEM;
+ pmic->dev = dev;
+
+ pmic->res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (!pmic->res)
+ return -ENOENT;
+
+ base = devm_ioremap_resource(dev, pmic->res);
+ if (!base)
+ return -ENOMEM;
+
+ pmic->regmap = devm_regmap_init_mmio_clk(dev, NULL, base,
+ &hi655x_regmap_config);
+
+ regmap_read(pmic->regmap, HI655X_BUS_ADDR(HI655X_VER_REG), &pmic->ver);
+ if ((pmic->ver < PMU_VER_START) || (pmic->ver > PMU_VER_END)) {
+ dev_warn(dev, "PMU version %d unsupported\n", pmic->ver);
+ return -EINVAL;
+ }
+
+ hi655x_local_irq_clear(pmic->regmap);
+
+ pmic->gpio = of_get_named_gpio(np, "pmic-gpios", 0);
+ if (!gpio_is_valid(pmic->gpio)) {
+ dev_err(dev, "Failed to get the pmic-gpios\n");
+ return -ENODEV;
+ }
+
+ ret = devm_gpio_request_one(dev, pmic->gpio, GPIOF_IN,
+ "hi655x_pmic_irq");
+ if (ret < 0) {
+ dev_err(dev, "Failed to request gpio %d ret = %d\n",
+ pmic->gpio, ret);
+ return ret;
+ }
+
+ ret = regmap_add_irq_chip(pmic->regmap, gpio_to_irq(pmic->gpio),
+ IRQF_TRIGGER_LOW | IRQF_NO_SUSPEND, 0,
+ &hi655x_irq_chip, &pmic->irq_data);
+ if (ret) {
+ dev_err(dev, "Failed to obtain 'hi655x_pmic_irq' %d\n", ret);
+ return ret;
+ }
+
+ platform_set_drvdata(pdev, pmic);
+
+ ret = mfd_add_devices(dev, PLATFORM_DEVID_AUTO, hi655x_pmic_devs,
+ ARRAY_SIZE(hi655x_pmic_devs), NULL, 0, NULL);
+ if (ret) {
+ dev_err(dev, "Failed to register device %d\n", ret);
+ regmap_del_irq_chip(gpio_to_irq(pmic->gpio), pmic->irq_data);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int hi655x_pmic_remove(struct platform_device *pdev)
+{
+ struct hi655x_pmic *pmic = platform_get_drvdata(pdev);
+
+ regmap_del_irq_chip(gpio_to_irq(pmic->gpio), pmic->irq_data);
+ mfd_remove_devices(&pdev->dev);
+ return 0;
+}
+
+static const struct of_device_id hi655x_pmic_match[] = {
+ { .compatible = "hisilicon,hi655x-pmic", },
+ {},
+};
+
+static struct platform_driver hi655x_pmic_driver = {
+ .driver = {
+ .name = "hi655x-pmic",
+ .of_match_table = of_match_ptr(hi655x_pmic_match),
+ },
+ .probe = hi655x_pmic_probe,
+ .remove = hi655x_pmic_remove,
+};
+module_platform_driver(hi655x_pmic_driver);
+
+MODULE_AUTHOR("Chen Feng <puck.chen@hisilicon.com>");
+MODULE_DESCRIPTION("Hisilicon hi655x PMIC driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/mfd/htc-egpio.c b/drivers/mfd/htc-egpio.c
index c636b5f..513cfc5 100644
--- a/drivers/mfd/htc-egpio.c
+++ b/drivers/mfd/htc-egpio.c
@@ -155,7 +155,7 @@
pr_debug("egpio_get_value(%d)\n", chip->base + offset);
- egpio = container_of(chip, struct egpio_chip, chip);
+ egpio = gpiochip_get_data(chip);
ei = dev_get_drvdata(egpio->dev);
bit = egpio_bit(ei, offset);
reg = egpio->reg_start + egpio_pos(ei, offset);
@@ -170,7 +170,7 @@
{
struct egpio_chip *egpio;
- egpio = container_of(chip, struct egpio_chip, chip);
+ egpio = gpiochip_get_data(chip);
return test_bit(offset, &egpio->is_out) ? -EINVAL : 0;
}
@@ -192,7 +192,7 @@
pr_debug("egpio_set(%s, %d(%d), %d)\n",
chip->label, offset, offset+chip->base, value);
- egpio = container_of(chip, struct egpio_chip, chip);
+ egpio = gpiochip_get_data(chip);
ei = dev_get_drvdata(egpio->dev);
bit = egpio_bit(ei, offset);
pos = egpio_pos(ei, offset);
@@ -216,7 +216,7 @@
{
struct egpio_chip *egpio;
- egpio = container_of(chip, struct egpio_chip, chip);
+ egpio = gpiochip_get_data(chip);
if (test_bit(offset, &egpio->is_out)) {
egpio_set(chip, offset, value);
return 0;
@@ -330,7 +330,7 @@
chip->base = pdata->chip[i].gpio_base;
chip->ngpio = pdata->chip[i].num_gpios;
- gpiochip_add(chip);
+ gpiochip_add_data(chip, &ei->chip[i]);
}
/* Set initial pin values */
diff --git a/drivers/mfd/htc-i2cpld.c b/drivers/mfd/htc-i2cpld.c
index bd6b96d..3f9eee5 100644
--- a/drivers/mfd/htc-i2cpld.c
+++ b/drivers/mfd/htc-i2cpld.c
@@ -227,8 +227,7 @@
static void htcpld_chip_set(struct gpio_chip *chip, unsigned offset, int val)
{
struct i2c_client *client;
- struct htcpld_chip *chip_data =
- container_of(chip, struct htcpld_chip, chip_out);
+ struct htcpld_chip *chip_data = gpiochip_get_data(chip);
unsigned long flags;
client = chip_data->client;
@@ -257,14 +256,12 @@
static int htcpld_chip_get(struct gpio_chip *chip, unsigned offset)
{
- struct htcpld_chip *chip_data;
+ struct htcpld_chip *chip_data = gpiochip_get_data(chip);
u8 cache;
if (!strncmp(chip->label, "htcpld-out", 10)) {
- chip_data = container_of(chip, struct htcpld_chip, chip_out);
cache = chip_data->cache_out;
} else if (!strncmp(chip->label, "htcpld-in", 9)) {
- chip_data = container_of(chip, struct htcpld_chip, chip_in);
cache = chip_data->cache_in;
} else
return -EINVAL;
@@ -291,9 +288,7 @@
static int htcpld_chip_to_irq(struct gpio_chip *chip, unsigned offset)
{
- struct htcpld_chip *chip_data;
-
- chip_data = container_of(chip, struct htcpld_chip, chip_in);
+ struct htcpld_chip *chip_data = gpiochip_get_data(chip);
if (offset < chip_data->nirqs)
return chip_data->irq_start + offset;
@@ -451,14 +446,14 @@
gpio_chip->ngpio = plat_chip_data->num_gpios;
/* Add the GPIO chips */
- ret = gpiochip_add(&(chip->chip_out));
+ ret = gpiochip_add_data(&(chip->chip_out), chip);
if (ret) {
dev_warn(dev, "Unable to register output GPIOs for 0x%x: %d\n",
plat_chip_data->addr, ret);
return ret;
}
- ret = gpiochip_add(&(chip->chip_in));
+ ret = gpiochip_add_data(&(chip->chip_in), chip);
if (ret) {
dev_warn(dev, "Unable to register input GPIOs for 0x%x: %d\n",
plat_chip_data->addr, ret);
diff --git a/drivers/mfd/intel-lpss.c b/drivers/mfd/intel-lpss.c
index 6352aab..41b1138 100644
--- a/drivers/mfd/intel-lpss.c
+++ b/drivers/mfd/intel-lpss.c
@@ -34,6 +34,7 @@
#define LPSS_DEV_SIZE 0x200
#define LPSS_PRIV_OFFSET 0x200
#define LPSS_PRIV_SIZE 0x100
+#define LPSS_PRIV_REG_COUNT (LPSS_PRIV_SIZE / 4)
#define LPSS_IDMA64_OFFSET 0x800
#define LPSS_IDMA64_SIZE 0x800
@@ -76,6 +77,7 @@
struct mfd_cell *cell;
struct device *dev;
void __iomem *priv;
+ u32 priv_ctx[LPSS_PRIV_REG_COUNT];
int devid;
u32 caps;
u32 active_ltr;
@@ -336,8 +338,8 @@
return 0;
/* Root clock */
- clk = clk_register_fixed_rate(NULL, dev_name(lpss->dev), NULL,
- CLK_IS_ROOT, lpss->info->clk_rate);
+ clk = clk_register_fixed_rate(NULL, dev_name(lpss->dev), NULL, 0,
+ lpss->info->clk_rate);
if (IS_ERR(clk))
return PTR_ERR(clk);
@@ -493,6 +495,16 @@
int intel_lpss_suspend(struct device *dev)
{
+ struct intel_lpss *lpss = dev_get_drvdata(dev);
+ unsigned int i;
+
+ /* Save device context */
+ for (i = 0; i < LPSS_PRIV_REG_COUNT; i++)
+ lpss->priv_ctx[i] = readl(lpss->priv + i * 4);
+
+ /* Put the device into reset state */
+ writel(0, lpss->priv + LPSS_PRIV_RESETS);
+
return 0;
}
EXPORT_SYMBOL_GPL(intel_lpss_suspend);
@@ -500,8 +512,13 @@
int intel_lpss_resume(struct device *dev)
{
struct intel_lpss *lpss = dev_get_drvdata(dev);
+ unsigned int i;
- intel_lpss_init_dev(lpss);
+ intel_lpss_deassert_reset(lpss);
+
+ /* Restore device context */
+ for (i = 0; i < LPSS_PRIV_REG_COUNT; i++)
+ writel(lpss->priv_ctx[i], lpss->priv + i * 4);
return 0;
}
diff --git a/drivers/mfd/intel_quark_i2c_gpio.c b/drivers/mfd/intel_quark_i2c_gpio.c
index a24b35f..7946d6e 100644
--- a/drivers/mfd/intel_quark_i2c_gpio.c
+++ b/drivers/mfd/intel_quark_i2c_gpio.c
@@ -53,7 +53,7 @@
#define INTEL_QUARK_I2C_CLK_HZ 33000000
struct intel_quark_mfd {
- struct pci_dev *pdev;
+ struct device *dev;
struct clk *i2c_clk;
struct clk_lookup *i2c_clk_lookup;
};
@@ -123,14 +123,14 @@
};
MODULE_DEVICE_TABLE(pci, intel_quark_mfd_ids);
-static int intel_quark_register_i2c_clk(struct intel_quark_mfd *quark_mfd)
+static int intel_quark_register_i2c_clk(struct device *dev)
{
- struct pci_dev *pdev = quark_mfd->pdev;
+ struct intel_quark_mfd *quark_mfd = dev_get_drvdata(dev);
struct clk *i2c_clk;
- i2c_clk = clk_register_fixed_rate(&pdev->dev,
+ i2c_clk = clk_register_fixed_rate(dev,
INTEL_QUARK_I2C_CONTROLLER_CLK, NULL,
- CLK_IS_ROOT, INTEL_QUARK_I2C_CLK_HZ);
+ 0, INTEL_QUARK_I2C_CLK_HZ);
if (IS_ERR(i2c_clk))
return PTR_ERR(i2c_clk);
@@ -139,18 +139,19 @@
INTEL_QUARK_I2C_CONTROLLER_CLK);
if (!quark_mfd->i2c_clk_lookup) {
- dev_err(&pdev->dev, "Fixed clk register failed\n");
+ clk_unregister(quark_mfd->i2c_clk);
+ dev_err(dev, "Fixed clk register failed\n");
return -ENOMEM;
}
return 0;
}
-static void intel_quark_unregister_i2c_clk(struct pci_dev *pdev)
+static void intel_quark_unregister_i2c_clk(struct device *dev)
{
- struct intel_quark_mfd *quark_mfd = dev_get_drvdata(&pdev->dev);
+ struct intel_quark_mfd *quark_mfd = dev_get_drvdata(dev);
- if (!quark_mfd->i2c_clk || !quark_mfd->i2c_clk_lookup)
+ if (!quark_mfd->i2c_clk_lookup)
return;
clkdev_drop(quark_mfd->i2c_clk_lookup);
@@ -245,30 +246,38 @@
quark_mfd = devm_kzalloc(&pdev->dev, sizeof(*quark_mfd), GFP_KERNEL);
if (!quark_mfd)
return -ENOMEM;
- quark_mfd->pdev = pdev;
- ret = intel_quark_register_i2c_clk(quark_mfd);
+ quark_mfd->dev = &pdev->dev;
+ dev_set_drvdata(&pdev->dev, quark_mfd);
+
+ ret = intel_quark_register_i2c_clk(&pdev->dev);
if (ret)
return ret;
- dev_set_drvdata(&pdev->dev, quark_mfd);
-
ret = intel_quark_i2c_setup(pdev, &intel_quark_mfd_cells[1]);
if (ret)
- return ret;
+ goto err_unregister_i2c_clk;
ret = intel_quark_gpio_setup(pdev, &intel_quark_mfd_cells[0]);
if (ret)
- return ret;
+ goto err_unregister_i2c_clk;
- return mfd_add_devices(&pdev->dev, 0, intel_quark_mfd_cells,
- ARRAY_SIZE(intel_quark_mfd_cells), NULL, 0,
- NULL);
+ ret = mfd_add_devices(&pdev->dev, 0, intel_quark_mfd_cells,
+ ARRAY_SIZE(intel_quark_mfd_cells), NULL, 0,
+ NULL);
+ if (ret)
+ goto err_unregister_i2c_clk;
+
+ return 0;
+
+err_unregister_i2c_clk:
+ intel_quark_unregister_i2c_clk(&pdev->dev);
+ return ret;
}
static void intel_quark_mfd_remove(struct pci_dev *pdev)
{
- intel_quark_unregister_i2c_clk(pdev);
+ intel_quark_unregister_i2c_clk(&pdev->dev);
mfd_remove_devices(&pdev->dev);
}
diff --git a/drivers/mfd/intel_soc_pmic_core.c b/drivers/mfd/intel_soc_pmic_core.c
index d9e15cf..12d6ebb4 100644
--- a/drivers/mfd/intel_soc_pmic_core.c
+++ b/drivers/mfd/intel_soc_pmic_core.c
@@ -35,6 +35,7 @@
.table = {
/* Panel EN/DISABLE */
GPIO_LOOKUP("gpio_crystalcove", 94, "panel", GPIO_ACTIVE_HIGH),
+ { },
},
};
diff --git a/drivers/mfd/lp3943.c b/drivers/mfd/lp3943.c
index eecbb13..65a2a8f1 100644
--- a/drivers/mfd/lp3943.c
+++ b/drivers/mfd/lp3943.c
@@ -123,16 +123,9 @@
lp3943->mux_cfg = lp3943_mux_cfg;
i2c_set_clientdata(cl, lp3943);
- return mfd_add_devices(dev, -1, lp3943_devs, ARRAY_SIZE(lp3943_devs),
- NULL, 0, NULL);
-}
-
-static int lp3943_remove(struct i2c_client *cl)
-{
- struct lp3943 *lp3943 = i2c_get_clientdata(cl);
-
- mfd_remove_devices(lp3943->dev);
- return 0;
+ return devm_mfd_add_devices(dev, -1, lp3943_devs,
+ ARRAY_SIZE(lp3943_devs),
+ NULL, 0, NULL);
}
static const struct i2c_device_id lp3943_ids[] = {
@@ -151,7 +144,6 @@
static struct i2c_driver lp3943_driver = {
.probe = lp3943_probe,
- .remove = lp3943_remove,
.driver = {
.name = "lp3943",
.of_match_table = of_match_ptr(lp3943_of_match),
diff --git a/drivers/mfd/lp8788-irq.c b/drivers/mfd/lp8788-irq.c
index c7a9825..792d51b 100644
--- a/drivers/mfd/lp8788-irq.c
+++ b/drivers/mfd/lp8788-irq.c
@@ -112,7 +112,7 @@
struct lp8788_irq_data *irqd = ptr;
struct lp8788 *lp = irqd->lp;
u8 status[NUM_REGS], addr, mask;
- bool handled;
+ bool handled = false;
int i;
if (lp8788_read_multi_bytes(lp, LP8788_INT_1, status, NUM_REGS))
diff --git a/drivers/mfd/max77620.c b/drivers/mfd/max77620.c
new file mode 100644
index 0000000..199d261
--- /dev/null
+++ b/drivers/mfd/max77620.c
@@ -0,0 +1,590 @@
+/*
+ * Maxim MAX77620 MFD Driver
+ *
+ * Copyright (C) 2016 NVIDIA CORPORATION. All rights reserved.
+ *
+ * Author:
+ * Laxman Dewangan <ldewangan@nvidia.com>
+ * Chaitanya Bandi <bandik@nvidia.com>
+ * Mallikarjun Kasoju <mkasoju@nvidia.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+/****************** Teminology used in driver ********************
+ * Here are some terminology used from datasheet for quick reference:
+ * Flexible Power Sequence (FPS):
+ * The Flexible Power Sequencer (FPS) allows each regulator to power up under
+ * hardware or software control. Additionally, each regulator can power on
+ * independently or among a group of other regulators with an adjustable
+ * power-up and power-down delays (sequencing). GPIO1, GPIO2, and GPIO3 can
+ * be programmed to be part of a sequence allowing external regulators to be
+ * sequenced along with internal regulators. 32KHz clock can be programmed to
+ * be part of a sequence.
+ * There is 3 FPS confguration registers and all resources are configured to
+ * any of these FPS or no FPS.
+ */
+
+#include <linux/i2c.h>
+#include <linux/interrupt.h>
+#include <linux/mfd/core.h>
+#include <linux/mfd/max77620.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/regmap.h>
+#include <linux/slab.h>
+
+static struct resource gpio_resources[] = {
+ DEFINE_RES_IRQ(MAX77620_IRQ_TOP_GPIO),
+};
+
+static struct resource power_resources[] = {
+ DEFINE_RES_IRQ(MAX77620_IRQ_LBT_MBATLOW),
+};
+
+static struct resource rtc_resources[] = {
+ DEFINE_RES_IRQ(MAX77620_IRQ_TOP_RTC),
+};
+
+static struct resource thermal_resources[] = {
+ DEFINE_RES_IRQ(MAX77620_IRQ_LBT_TJALRM1),
+ DEFINE_RES_IRQ(MAX77620_IRQ_LBT_TJALRM2),
+};
+
+static const struct regmap_irq max77620_top_irqs[] = {
+ REGMAP_IRQ_REG(MAX77620_IRQ_TOP_GLBL, 0, MAX77620_IRQ_TOP_GLBL_MASK),
+ REGMAP_IRQ_REG(MAX77620_IRQ_TOP_SD, 0, MAX77620_IRQ_TOP_SD_MASK),
+ REGMAP_IRQ_REG(MAX77620_IRQ_TOP_LDO, 0, MAX77620_IRQ_TOP_LDO_MASK),
+ REGMAP_IRQ_REG(MAX77620_IRQ_TOP_GPIO, 0, MAX77620_IRQ_TOP_GPIO_MASK),
+ REGMAP_IRQ_REG(MAX77620_IRQ_TOP_RTC, 0, MAX77620_IRQ_TOP_RTC_MASK),
+ REGMAP_IRQ_REG(MAX77620_IRQ_TOP_32K, 0, MAX77620_IRQ_TOP_32K_MASK),
+ REGMAP_IRQ_REG(MAX77620_IRQ_TOP_ONOFF, 0, MAX77620_IRQ_TOP_ONOFF_MASK),
+ REGMAP_IRQ_REG(MAX77620_IRQ_LBT_MBATLOW, 1, MAX77620_IRQ_LBM_MASK),
+ REGMAP_IRQ_REG(MAX77620_IRQ_LBT_TJALRM1, 1, MAX77620_IRQ_TJALRM1_MASK),
+ REGMAP_IRQ_REG(MAX77620_IRQ_LBT_TJALRM2, 1, MAX77620_IRQ_TJALRM2_MASK),
+};
+
+static const struct mfd_cell max77620_children[] = {
+ { .name = "max77620-pinctrl", },
+ { .name = "max77620-clock", },
+ { .name = "max77620-pmic", },
+ { .name = "max77620-watchdog", },
+ {
+ .name = "max77620-gpio",
+ .resources = gpio_resources,
+ .num_resources = ARRAY_SIZE(gpio_resources),
+ }, {
+ .name = "max77620-rtc",
+ .resources = rtc_resources,
+ .num_resources = ARRAY_SIZE(rtc_resources),
+ }, {
+ .name = "max77620-power",
+ .resources = power_resources,
+ .num_resources = ARRAY_SIZE(power_resources),
+ }, {
+ .name = "max77620-thermal",
+ .resources = thermal_resources,
+ .num_resources = ARRAY_SIZE(thermal_resources),
+ },
+};
+
+static const struct mfd_cell max20024_children[] = {
+ { .name = "max20024-pinctrl", },
+ { .name = "max77620-clock", },
+ { .name = "max20024-pmic", },
+ { .name = "max77620-watchdog", },
+ {
+ .name = "max77620-gpio",
+ .resources = gpio_resources,
+ .num_resources = ARRAY_SIZE(gpio_resources),
+ }, {
+ .name = "max77620-rtc",
+ .resources = rtc_resources,
+ .num_resources = ARRAY_SIZE(rtc_resources),
+ }, {
+ .name = "max20024-power",
+ .resources = power_resources,
+ .num_resources = ARRAY_SIZE(power_resources),
+ },
+};
+
+static struct regmap_irq_chip max77620_top_irq_chip = {
+ .name = "max77620-top",
+ .irqs = max77620_top_irqs,
+ .num_irqs = ARRAY_SIZE(max77620_top_irqs),
+ .num_regs = 2,
+ .status_base = MAX77620_REG_IRQTOP,
+ .mask_base = MAX77620_REG_IRQTOPM,
+};
+
+static const struct regmap_range max77620_readable_ranges[] = {
+ regmap_reg_range(MAX77620_REG_CNFGGLBL1, MAX77620_REG_DVSSD4),
+};
+
+static const struct regmap_access_table max77620_readable_table = {
+ .yes_ranges = max77620_readable_ranges,
+ .n_yes_ranges = ARRAY_SIZE(max77620_readable_ranges),
+};
+
+static const struct regmap_range max20024_readable_ranges[] = {
+ regmap_reg_range(MAX77620_REG_CNFGGLBL1, MAX77620_REG_DVSSD4),
+ regmap_reg_range(MAX20024_REG_MAX_ADD, MAX20024_REG_MAX_ADD),
+};
+
+static const struct regmap_access_table max20024_readable_table = {
+ .yes_ranges = max20024_readable_ranges,
+ .n_yes_ranges = ARRAY_SIZE(max20024_readable_ranges),
+};
+
+static const struct regmap_range max77620_writable_ranges[] = {
+ regmap_reg_range(MAX77620_REG_CNFGGLBL1, MAX77620_REG_DVSSD4),
+};
+
+static const struct regmap_access_table max77620_writable_table = {
+ .yes_ranges = max77620_writable_ranges,
+ .n_yes_ranges = ARRAY_SIZE(max77620_writable_ranges),
+};
+
+static const struct regmap_range max77620_cacheable_ranges[] = {
+ regmap_reg_range(MAX77620_REG_SD0_CFG, MAX77620_REG_LDO_CFG3),
+ regmap_reg_range(MAX77620_REG_FPS_CFG0, MAX77620_REG_FPS_SD3),
+};
+
+static const struct regmap_access_table max77620_volatile_table = {
+ .no_ranges = max77620_cacheable_ranges,
+ .n_no_ranges = ARRAY_SIZE(max77620_cacheable_ranges),
+};
+
+static const struct regmap_config max77620_regmap_config = {
+ .name = "power-slave",
+ .reg_bits = 8,
+ .val_bits = 8,
+ .max_register = MAX77620_REG_DVSSD4 + 1,
+ .cache_type = REGCACHE_RBTREE,
+ .rd_table = &max77620_readable_table,
+ .wr_table = &max77620_writable_table,
+ .volatile_table = &max77620_volatile_table,
+};
+
+static const struct regmap_config max20024_regmap_config = {
+ .name = "power-slave",
+ .reg_bits = 8,
+ .val_bits = 8,
+ .max_register = MAX20024_REG_MAX_ADD + 1,
+ .cache_type = REGCACHE_RBTREE,
+ .rd_table = &max20024_readable_table,
+ .wr_table = &max77620_writable_table,
+ .volatile_table = &max77620_volatile_table,
+};
+
+/* max77620_get_fps_period_reg_value: Get FPS bit field value from
+ * requested periods.
+ * MAX77620 supports the FPS period of 40, 80, 160, 320, 540, 1280, 2560
+ * and 5120 microseconds. MAX20024 supports the FPS period of 20, 40, 80,
+ * 160, 320, 540, 1280 and 2560 microseconds.
+ * The FPS register has 3 bits field to set the FPS period as
+ * bits max77620 max20024
+ * 000 40 20
+ * 001 80 40
+ * :::
+*/
+static int max77620_get_fps_period_reg_value(struct max77620_chip *chip,
+ int tperiod)
+{
+ int fps_min_period;
+ int i;
+
+ switch (chip->chip_id) {
+ case MAX20024:
+ fps_min_period = MAX20024_FPS_PERIOD_MIN_US;
+ break;
+ case MAX77620:
+ fps_min_period = MAX77620_FPS_PERIOD_MIN_US;
+ default:
+ return -EINVAL;
+ }
+
+ for (i = 0; i < 7; i++) {
+ if (fps_min_period >= tperiod)
+ return i;
+ fps_min_period *= 2;
+ }
+
+ return i;
+}
+
+/* max77620_config_fps: Configure FPS configuration registers
+ * based on platform specific information.
+ */
+static int max77620_config_fps(struct max77620_chip *chip,
+ struct device_node *fps_np)
+{
+ struct device *dev = chip->dev;
+ unsigned int mask = 0, config = 0;
+ u32 fps_max_period;
+ u32 param_val;
+ int tperiod, fps_id;
+ int ret;
+ char fps_name[10];
+
+ switch (chip->chip_id) {
+ case MAX20024:
+ fps_max_period = MAX20024_FPS_PERIOD_MAX_US;
+ break;
+ case MAX77620:
+ fps_max_period = MAX77620_FPS_PERIOD_MAX_US;
+ default:
+ return -EINVAL;
+ }
+
+ for (fps_id = 0; fps_id < MAX77620_FPS_COUNT; fps_id++) {
+ sprintf(fps_name, "fps%d", fps_id);
+ if (!strcmp(fps_np->name, fps_name))
+ break;
+ }
+
+ if (fps_id == MAX77620_FPS_COUNT) {
+ dev_err(dev, "FPS node name %s is not valid\n", fps_np->name);
+ return -EINVAL;
+ }
+
+ ret = of_property_read_u32(fps_np, "maxim,shutdown-fps-time-period-us",
+ ¶m_val);
+ if (!ret) {
+ mask |= MAX77620_FPS_TIME_PERIOD_MASK;
+ chip->shutdown_fps_period[fps_id] = min(param_val,
+ fps_max_period);
+ tperiod = max77620_get_fps_period_reg_value(chip,
+ chip->shutdown_fps_period[fps_id]);
+ config |= tperiod << MAX77620_FPS_TIME_PERIOD_SHIFT;
+ }
+
+ ret = of_property_read_u32(fps_np, "maxim,suspend-fps-time-period-us",
+ ¶m_val);
+ if (!ret)
+ chip->suspend_fps_period[fps_id] = min(param_val,
+ fps_max_period);
+
+ ret = of_property_read_u32(fps_np, "maxim,fps-event-source",
+ ¶m_val);
+ if (!ret) {
+ if (param_val > 2) {
+ dev_err(dev, "FPS%d event-source invalid\n", fps_id);
+ return -EINVAL;
+ }
+ mask |= MAX77620_FPS_EN_SRC_MASK;
+ config |= param_val << MAX77620_FPS_EN_SRC_SHIFT;
+ if (param_val == 2) {
+ mask |= MAX77620_FPS_ENFPS_SW_MASK;
+ config |= MAX77620_FPS_ENFPS_SW;
+ }
+ }
+
+ if (!chip->sleep_enable && !chip->enable_global_lpm) {
+ ret = of_property_read_u32(fps_np,
+ "maxim,device-state-on-disabled-event",
+ ¶m_val);
+ if (!ret) {
+ if (param_val == 0)
+ chip->sleep_enable = true;
+ else if (param_val == 1)
+ chip->enable_global_lpm = true;
+ }
+ }
+
+ ret = regmap_update_bits(chip->rmap, MAX77620_REG_FPS_CFG0 + fps_id,
+ mask, config);
+ if (ret < 0) {
+ dev_err(dev, "Failed to update FPS CFG: %d\n", ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int max77620_initialise_fps(struct max77620_chip *chip)
+{
+ struct device *dev = chip->dev;
+ struct device_node *fps_np, *fps_child;
+ u8 config;
+ int fps_id;
+ int ret;
+
+ for (fps_id = 0; fps_id < MAX77620_FPS_COUNT; fps_id++) {
+ chip->shutdown_fps_period[fps_id] = -1;
+ chip->suspend_fps_period[fps_id] = -1;
+ }
+
+ fps_np = of_get_child_by_name(dev->of_node, "fps");
+ if (!fps_np)
+ goto skip_fps;
+
+ for_each_child_of_node(fps_np, fps_child) {
+ ret = max77620_config_fps(chip, fps_child);
+ if (ret < 0)
+ return ret;
+ }
+
+ config = chip->enable_global_lpm ? MAX77620_ONOFFCNFG2_SLP_LPM_MSK : 0;
+ ret = regmap_update_bits(chip->rmap, MAX77620_REG_ONOFFCNFG2,
+ MAX77620_ONOFFCNFG2_SLP_LPM_MSK, config);
+ if (ret < 0) {
+ dev_err(dev, "Failed to update SLP_LPM: %d\n", ret);
+ return ret;
+ }
+
+skip_fps:
+ /* Enable wake on EN0 pin */
+ ret = regmap_update_bits(chip->rmap, MAX77620_REG_ONOFFCNFG2,
+ MAX77620_ONOFFCNFG2_WK_EN0,
+ MAX77620_ONOFFCNFG2_WK_EN0);
+ if (ret < 0) {
+ dev_err(dev, "Failed to update WK_EN0: %d\n", ret);
+ return ret;
+ }
+
+ /* For MAX20024, SLPEN will be POR reset if CLRSE is b11 */
+ if ((chip->chip_id == MAX20024) && chip->sleep_enable) {
+ config = MAX77620_ONOFFCNFG1_SLPEN | MAX20024_ONOFFCNFG1_CLRSE;
+ ret = regmap_update_bits(chip->rmap, MAX77620_REG_ONOFFCNFG1,
+ config, config);
+ if (ret < 0) {
+ dev_err(dev, "Failed to update SLPEN: %d\n", ret);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+static int max77620_read_es_version(struct max77620_chip *chip)
+{
+ unsigned int val;
+ u8 cid_val[6];
+ int i;
+ int ret;
+
+ for (i = MAX77620_REG_CID0; i <= MAX77620_REG_CID5; i++) {
+ ret = regmap_read(chip->rmap, i, &val);
+ if (ret < 0) {
+ dev_err(chip->dev, "Failed to read CID: %d\n", ret);
+ return ret;
+ }
+ dev_dbg(chip->dev, "CID%d: 0x%02x\n",
+ i - MAX77620_REG_CID0, val);
+ cid_val[i - MAX77620_REG_CID0] = val;
+ }
+
+ /* CID4 is OTP Version and CID5 is ES version */
+ dev_info(chip->dev, "PMIC Version OTP:0x%02X and ES:0x%X\n",
+ cid_val[4], MAX77620_CID5_DIDM(cid_val[5]));
+
+ return ret;
+}
+
+static int max77620_probe(struct i2c_client *client,
+ const struct i2c_device_id *id)
+{
+ const struct regmap_config *rmap_config;
+ struct max77620_chip *chip;
+ const struct mfd_cell *mfd_cells;
+ int n_mfd_cells;
+ int ret;
+
+ chip = devm_kzalloc(&client->dev, sizeof(*chip), GFP_KERNEL);
+ if (!chip)
+ return -ENOMEM;
+
+ i2c_set_clientdata(client, chip);
+ chip->dev = &client->dev;
+ chip->irq_base = -1;
+ chip->chip_irq = client->irq;
+ chip->chip_id = (enum max77620_chip_id)id->driver_data;
+
+ switch (chip->chip_id) {
+ case MAX77620:
+ mfd_cells = max77620_children;
+ n_mfd_cells = ARRAY_SIZE(max77620_children);
+ rmap_config = &max77620_regmap_config;
+ break;
+ case MAX20024:
+ mfd_cells = max20024_children;
+ n_mfd_cells = ARRAY_SIZE(max20024_children);
+ rmap_config = &max20024_regmap_config;
+ break;
+ default:
+ dev_err(chip->dev, "ChipID is invalid %d\n", chip->chip_id);
+ return -EINVAL;
+ }
+
+ chip->rmap = devm_regmap_init_i2c(client, rmap_config);
+ if (IS_ERR(chip->rmap)) {
+ ret = PTR_ERR(chip->rmap);
+ dev_err(chip->dev, "Failed to intialise regmap: %d\n", ret);
+ return ret;
+ }
+
+ ret = max77620_read_es_version(chip);
+ if (ret < 0)
+ return ret;
+
+ ret = devm_regmap_add_irq_chip(chip->dev, chip->rmap, client->irq,
+ IRQF_ONESHOT | IRQF_SHARED,
+ chip->irq_base, &max77620_top_irq_chip,
+ &chip->top_irq_data);
+ if (ret < 0) {
+ dev_err(chip->dev, "Failed to add regmap irq: %d\n", ret);
+ return ret;
+ }
+
+ ret = max77620_initialise_fps(chip);
+ if (ret < 0)
+ return ret;
+
+ ret = devm_mfd_add_devices(chip->dev, PLATFORM_DEVID_NONE,
+ mfd_cells, n_mfd_cells, NULL, 0,
+ regmap_irq_get_domain(chip->top_irq_data));
+ if (ret < 0) {
+ dev_err(chip->dev, "Failed to add MFD children: %d\n", ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+#ifdef CONFIG_PM_SLEEP
+static int max77620_set_fps_period(struct max77620_chip *chip,
+ int fps_id, int time_period)
+{
+ int period = max77620_get_fps_period_reg_value(chip, time_period);
+ int ret;
+
+ ret = regmap_update_bits(chip->rmap, MAX77620_REG_FPS_CFG0 + fps_id,
+ MAX77620_FPS_TIME_PERIOD_MASK,
+ period << MAX77620_FPS_TIME_PERIOD_SHIFT);
+ if (ret < 0) {
+ dev_err(chip->dev, "Failed to update FPS period: %d\n", ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int max77620_i2c_suspend(struct device *dev)
+{
+ struct max77620_chip *chip = dev_get_drvdata(dev);
+ struct i2c_client *client = to_i2c_client(dev);
+ unsigned int config;
+ int fps;
+ int ret;
+
+ for (fps = 0; fps < MAX77620_FPS_COUNT; fps++) {
+ if (chip->suspend_fps_period[fps] < 0)
+ continue;
+
+ ret = max77620_set_fps_period(chip, fps,
+ chip->suspend_fps_period[fps]);
+ if (ret < 0)
+ return ret;
+ }
+
+ /*
+ * For MAX20024: No need to configure SLPEN on suspend as
+ * it will be configured on Init.
+ */
+ if (chip->chip_id == MAX20024)
+ goto out;
+
+ config = (chip->sleep_enable) ? MAX77620_ONOFFCNFG1_SLPEN : 0;
+ ret = regmap_update_bits(chip->rmap, MAX77620_REG_ONOFFCNFG1,
+ MAX77620_ONOFFCNFG1_SLPEN,
+ config);
+ if (ret < 0) {
+ dev_err(dev, "Failed to configure sleep in suspend: %d\n", ret);
+ return ret;
+ }
+
+ /* Disable WK_EN0 */
+ ret = regmap_update_bits(chip->rmap, MAX77620_REG_ONOFFCNFG2,
+ MAX77620_ONOFFCNFG2_WK_EN0, 0);
+ if (ret < 0) {
+ dev_err(dev, "Failed to configure WK_EN in suspend: %d\n", ret);
+ return ret;
+ }
+
+out:
+ disable_irq(client->irq);
+
+ return 0;
+}
+
+static int max77620_i2c_resume(struct device *dev)
+{
+ struct max77620_chip *chip = dev_get_drvdata(dev);
+ struct i2c_client *client = to_i2c_client(dev);
+ int ret;
+ int fps;
+
+ for (fps = 0; fps < MAX77620_FPS_COUNT; fps++) {
+ if (chip->shutdown_fps_period[fps] < 0)
+ continue;
+
+ ret = max77620_set_fps_period(chip, fps,
+ chip->shutdown_fps_period[fps]);
+ if (ret < 0)
+ return ret;
+ }
+
+ /*
+ * For MAX20024: No need to configure WKEN0 on resume as
+ * it is configured on Init.
+ */
+ if (chip->chip_id == MAX20024)
+ goto out;
+
+ /* Enable WK_EN0 */
+ ret = regmap_update_bits(chip->rmap, MAX77620_REG_ONOFFCNFG2,
+ MAX77620_ONOFFCNFG2_WK_EN0,
+ MAX77620_ONOFFCNFG2_WK_EN0);
+ if (ret < 0) {
+ dev_err(dev, "Failed to configure WK_EN0 n resume: %d\n", ret);
+ return ret;
+ }
+
+out:
+ enable_irq(client->irq);
+
+ return 0;
+}
+#endif
+
+static const struct i2c_device_id max77620_id[] = {
+ {"max77620", MAX77620},
+ {"max20024", MAX20024},
+ {},
+};
+MODULE_DEVICE_TABLE(i2c, max77620_id);
+
+static const struct dev_pm_ops max77620_pm_ops = {
+ SET_SYSTEM_SLEEP_PM_OPS(max77620_i2c_suspend, max77620_i2c_resume)
+};
+
+static struct i2c_driver max77620_driver = {
+ .driver = {
+ .name = "max77620",
+ .pm = &max77620_pm_ops,
+ },
+ .probe = max77620_probe,
+ .id_table = max77620_id,
+};
+
+module_i2c_driver(max77620_driver);
+
+MODULE_DESCRIPTION("MAX77620/MAX20024 Multi Function Device Core Driver");
+MODULE_AUTHOR("Laxman Dewangan <ldewangan@nvidia.com>");
+MODULE_AUTHOR("Chaitanya Bandi <bandik@nvidia.com>");
+MODULE_AUTHOR("Mallikarjun Kasoju <mkasoju@nvidia.com>");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/mfd/max77686.c b/drivers/mfd/max77686.c
index c1aff46..7b68ed7 100644
--- a/drivers/mfd/max77686.c
+++ b/drivers/mfd/max77686.c
@@ -2,7 +2,7 @@
* max77686.c - mfd core driver for the Maxim 77686/802
*
* Copyright (C) 2012 Samsung Electronics
- * Chiwoong Byun <woong.byun@smasung.com>
+ * Chiwoong Byun <woong.byun@samsung.com>
* Jonghwa Lee <jonghwa3.lee@samsung.com>
*
* This program is free software; you can redistribute it and/or modify
@@ -230,38 +230,24 @@
return -ENODEV;
}
- ret = regmap_add_irq_chip(max77686->regmap, max77686->irq,
- IRQF_TRIGGER_FALLING | IRQF_ONESHOT |
- IRQF_SHARED, 0, irq_chip,
- &max77686->irq_data);
+ ret = devm_regmap_add_irq_chip(&i2c->dev, max77686->regmap,
+ max77686->irq,
+ IRQF_TRIGGER_FALLING | IRQF_ONESHOT |
+ IRQF_SHARED, 0, irq_chip,
+ &max77686->irq_data);
if (ret < 0) {
dev_err(&i2c->dev, "failed to add PMIC irq chip: %d\n", ret);
return ret;
}
- ret = mfd_add_devices(max77686->dev, -1, cells, n_devs, NULL, 0, NULL);
+ ret = devm_mfd_add_devices(max77686->dev, -1, cells, n_devs, NULL,
+ 0, NULL);
if (ret < 0) {
dev_err(&i2c->dev, "failed to add MFD devices: %d\n", ret);
- goto err_del_irqc;
+ return ret;
}
return 0;
-
-err_del_irqc:
- regmap_del_irq_chip(max77686->irq, max77686->irq_data);
-
- return ret;
-}
-
-static int max77686_i2c_remove(struct i2c_client *i2c)
-{
- struct max77686_dev *max77686 = i2c_get_clientdata(i2c);
-
- mfd_remove_devices(max77686->dev);
-
- regmap_del_irq_chip(max77686->irq, max77686->irq_data);
-
- return 0;
}
static const struct i2c_device_id max77686_i2c_id[] = {
@@ -317,22 +303,10 @@
.of_match_table = of_match_ptr(max77686_pmic_dt_match),
},
.probe = max77686_i2c_probe,
- .remove = max77686_i2c_remove,
.id_table = max77686_i2c_id,
};
-static int __init max77686_i2c_init(void)
-{
- return i2c_add_driver(&max77686_i2c_driver);
-}
-/* init early so consumer devices can complete system boot */
-subsys_initcall(max77686_i2c_init);
-
-static void __exit max77686_i2c_exit(void)
-{
- i2c_del_driver(&max77686_i2c_driver);
-}
-module_exit(max77686_i2c_exit);
+module_i2c_driver(max77686_i2c_driver);
MODULE_DESCRIPTION("MAXIM 77686/802 multi-function core driver");
MODULE_AUTHOR("Chiwoong Byun <woong.byun@samsung.com>");
diff --git a/drivers/mfd/max77693.c b/drivers/mfd/max77693.c
index b83b7a7..662ae0d 100644
--- a/drivers/mfd/max77693.c
+++ b/drivers/mfd/max77693.c
@@ -2,7 +2,7 @@
* max77693.c - mfd core driver for the MAX 77693
*
* Copyright (C) 2012 Samsung Electronics
- * SangYoung Son <hello.son@smasung.com>
+ * SangYoung Son <hello.son@samsung.com>
*
* This program is not provided / owned by Maxim Integrated Products.
*
@@ -368,6 +368,7 @@
{ .compatible = "maxim,max77693" },
{},
};
+MODULE_DEVICE_TABLE(of, max77693_dt_match);
#endif
static struct i2c_driver max77693_i2c_driver = {
@@ -381,18 +382,7 @@
.id_table = max77693_i2c_id,
};
-static int __init max77693_i2c_init(void)
-{
- return i2c_add_driver(&max77693_i2c_driver);
-}
-/* init early so consumer devices can complete system boot */
-subsys_initcall(max77693_i2c_init);
-
-static void __exit max77693_i2c_exit(void)
-{
- i2c_del_driver(&max77693_i2c_driver);
-}
-module_exit(max77693_i2c_exit);
+module_i2c_driver(max77693_i2c_driver);
MODULE_DESCRIPTION("MAXIM 77693 multi-function core driver");
MODULE_AUTHOR("SangYoung, Son <hello.son@samsung.com>");
diff --git a/drivers/mfd/menf21bmc.c b/drivers/mfd/menf21bmc.c
index 1c27434..3ad2def 100644
--- a/drivers/mfd/menf21bmc.c
+++ b/drivers/mfd/menf21bmc.c
@@ -96,8 +96,8 @@
return ret;
}
- ret = mfd_add_devices(&client->dev, 0, menf21bmc_cell,
- ARRAY_SIZE(menf21bmc_cell), NULL, 0, NULL);
+ ret = devm_mfd_add_devices(&client->dev, 0, menf21bmc_cell,
+ ARRAY_SIZE(menf21bmc_cell), NULL, 0, NULL);
if (ret < 0) {
dev_err(&client->dev, "failed to add BMC sub-devices\n");
return ret;
@@ -106,12 +106,6 @@
return 0;
}
-static int menf21bmc_remove(struct i2c_client *client)
-{
- mfd_remove_devices(&client->dev);
- return 0;
-}
-
static const struct i2c_device_id menf21bmc_id_table[] = {
{ "menf21bmc" },
{ }
@@ -122,7 +116,6 @@
.driver.name = "menf21bmc",
.id_table = menf21bmc_id_table,
.probe = menf21bmc_probe,
- .remove = menf21bmc_remove,
};
module_i2c_driver(menf21bmc_driver);
diff --git a/drivers/mfd/mfd-core.c b/drivers/mfd/mfd-core.c
index fc1c1fc..3ac486a 100644
--- a/drivers/mfd/mfd-core.c
+++ b/drivers/mfd/mfd-core.c
@@ -107,7 +107,7 @@
strlcpy(ids[0].id, match->pnpid, sizeof(ids[0].id));
list_for_each_entry(child, &parent->children, node) {
- if (acpi_match_device_ids(child, ids)) {
+ if (!acpi_match_device_ids(child, ids)) {
adev = child;
break;
}
@@ -334,6 +334,44 @@
}
EXPORT_SYMBOL(mfd_remove_devices);
+static void devm_mfd_dev_release(struct device *dev, void *res)
+{
+ mfd_remove_devices(dev);
+}
+
+/**
+ * devm_mfd_add_devices - Resource managed version of mfd_add_devices()
+ *
+ * Returns 0 on success or an appropriate negative error number on failure.
+ * All child-devices of the MFD will automatically be removed when it gets
+ * unbinded.
+ */
+int devm_mfd_add_devices(struct device *dev, int id,
+ const struct mfd_cell *cells, int n_devs,
+ struct resource *mem_base,
+ int irq_base, struct irq_domain *domain)
+{
+ struct device **ptr;
+ int ret;
+
+ ptr = devres_alloc(devm_mfd_dev_release, sizeof(*ptr), GFP_KERNEL);
+ if (!ptr)
+ return -ENOMEM;
+
+ ret = mfd_add_devices(dev, id, cells, n_devs, mem_base,
+ irq_base, domain);
+ if (ret < 0) {
+ devres_free(ptr);
+ return ret;
+ }
+
+ *ptr = dev;
+ devres_add(dev, ptr);
+
+ return ret;
+}
+EXPORT_SYMBOL(devm_mfd_add_devices);
+
int mfd_clone_cell(const char *cell, const char **clones, size_t n_clones)
{
struct mfd_cell cell_entry;
diff --git a/drivers/mfd/mt6397-core.c b/drivers/mfd/mt6397-core.c
index 8e8d932..e14d8b0 100644
--- a/drivers/mfd/mt6397-core.c
+++ b/drivers/mfd/mt6397-core.c
@@ -267,17 +267,26 @@
ret = regmap_read(pmic->regmap, MT6397_CID, &id);
if (ret) {
dev_err(pmic->dev, "Failed to read chip id: %d\n", ret);
- goto fail_irq;
+ return ret;
}
+ pmic->irq = platform_get_irq(pdev, 0);
+ if (pmic->irq <= 0)
+ return pmic->irq;
+
switch (id & 0xff) {
case MT6323_CID_CODE:
pmic->int_con[0] = MT6323_INT_CON0;
pmic->int_con[1] = MT6323_INT_CON1;
pmic->int_status[0] = MT6323_INT_STATUS0;
pmic->int_status[1] = MT6323_INT_STATUS1;
- ret = mfd_add_devices(&pdev->dev, -1, mt6323_devs,
- ARRAY_SIZE(mt6323_devs), NULL, 0, NULL);
+ ret = mt6397_irq_init(pmic);
+ if (ret)
+ return ret;
+
+ ret = devm_mfd_add_devices(&pdev->dev, -1, mt6323_devs,
+ ARRAY_SIZE(mt6323_devs), NULL,
+ 0, NULL);
break;
case MT6397_CID_CODE:
@@ -286,8 +295,13 @@
pmic->int_con[1] = MT6397_INT_CON1;
pmic->int_status[0] = MT6397_INT_STATUS0;
pmic->int_status[1] = MT6397_INT_STATUS1;
- ret = mfd_add_devices(&pdev->dev, -1, mt6397_devs,
- ARRAY_SIZE(mt6397_devs), NULL, 0, NULL);
+ ret = mt6397_irq_init(pmic);
+ if (ret)
+ return ret;
+
+ ret = devm_mfd_add_devices(&pdev->dev, -1, mt6397_devs,
+ ARRAY_SIZE(mt6397_devs), NULL,
+ 0, NULL);
break;
default:
@@ -296,14 +310,6 @@
break;
}
- pmic->irq = platform_get_irq(pdev, 0);
- if (pmic->irq > 0) {
- ret = mt6397_irq_init(pmic);
- if (ret)
- return ret;
- }
-
-fail_irq:
if (ret) {
irq_domain_remove(pmic->irq_domain);
dev_err(&pdev->dev, "failed to add child devices: %d\n", ret);
@@ -312,13 +318,6 @@
return ret;
}
-static int mt6397_remove(struct platform_device *pdev)
-{
- mfd_remove_devices(&pdev->dev);
-
- return 0;
-}
-
static const struct of_device_id mt6397_of_match[] = {
{ .compatible = "mediatek,mt6397" },
{ .compatible = "mediatek,mt6323" },
@@ -334,7 +333,6 @@
static struct platform_driver mt6397_driver = {
.probe = mt6397_probe,
- .remove = mt6397_remove,
.driver = {
.name = "mt6397",
.of_match_table = of_match_ptr(mt6397_of_match),
diff --git a/drivers/mfd/omap-usb-tll.c b/drivers/mfd/omap-usb-tll.c
index b7b3e8e..c30290f 100644
--- a/drivers/mfd/omap-usb-tll.c
+++ b/drivers/mfd/omap-usb-tll.c
@@ -269,6 +269,8 @@
if (IS_ERR(tll->ch_clk[i]))
dev_dbg(dev, "can't get clock : %s\n", clkname);
+ else
+ clk_prepare(tll->ch_clk[i]);
}
pm_runtime_put_sync(dev);
@@ -301,9 +303,12 @@
tll_dev = NULL;
spin_unlock(&tll_lock);
- for (i = 0; i < tll->nch; i++)
- if (!IS_ERR(tll->ch_clk[i]))
+ for (i = 0; i < tll->nch; i++) {
+ if (!IS_ERR(tll->ch_clk[i])) {
+ clk_unprepare(tll->ch_clk[i]);
clk_put(tll->ch_clk[i]);
+ }
+ }
pm_runtime_disable(&pdev->dev);
return 0;
@@ -420,7 +425,7 @@
if (IS_ERR(tll->ch_clk[i]))
continue;
- r = clk_prepare_enable(tll->ch_clk[i]);
+ r = clk_enable(tll->ch_clk[i]);
if (r) {
dev_err(tll_dev,
"Error enabling ch %d clock: %d\n", i, r);
@@ -448,7 +453,7 @@
for (i = 0; i < tll->nch; i++) {
if (omap_usb_mode_needs_tll(pdata->port_mode[i])) {
if (!IS_ERR(tll->ch_clk[i]))
- clk_disable_unprepare(tll->ch_clk[i]);
+ clk_disable(tll->ch_clk[i]);
}
}
diff --git a/drivers/mfd/rc5t583-irq.c b/drivers/mfd/rc5t583-irq.c
index 3f8812d..f8dde59 100644
--- a/drivers/mfd/rc5t583-irq.c
+++ b/drivers/mfd/rc5t583-irq.c
@@ -389,17 +389,10 @@
irq_clear_status_flags(__irq, IRQ_NOREQUEST);
}
- ret = request_threaded_irq(irq, NULL, rc5t583_irq, IRQF_ONESHOT,
- "rc5t583", rc5t583);
+ ret = devm_request_threaded_irq(rc5t583->dev, irq, NULL, rc5t583_irq,
+ IRQF_ONESHOT, "rc5t583", rc5t583);
if (ret < 0)
dev_err(rc5t583->dev,
"Error in registering interrupt error: %d\n", ret);
return ret;
}
-
-int rc5t583_irq_exit(struct rc5t583 *rc5t583)
-{
- if (rc5t583->chip_irq)
- free_irq(rc5t583->chip_irq, rc5t583);
- return 0;
-}
diff --git a/drivers/mfd/rc5t583.c b/drivers/mfd/rc5t583.c
index fc2b2d9..d12243d 100644
--- a/drivers/mfd/rc5t583.c
+++ b/drivers/mfd/rc5t583.c
@@ -252,7 +252,6 @@
struct rc5t583 *rc5t583;
struct rc5t583_platform_data *pdata = dev_get_platdata(&i2c->dev);
int ret;
- bool irq_init_success = false;
if (!pdata) {
dev_err(&i2c->dev, "Err: Platform data not found\n");
@@ -284,32 +283,16 @@
/* Still continue with warning, if irq init fails */
if (ret)
dev_warn(&i2c->dev, "IRQ init failed: %d\n", ret);
- else
- irq_init_success = true;
}
- ret = mfd_add_devices(rc5t583->dev, -1, rc5t583_subdevs,
- ARRAY_SIZE(rc5t583_subdevs), NULL, 0, NULL);
+ ret = devm_mfd_add_devices(rc5t583->dev, -1, rc5t583_subdevs,
+ ARRAY_SIZE(rc5t583_subdevs), NULL, 0, NULL);
if (ret) {
dev_err(&i2c->dev, "add mfd devices failed: %d\n", ret);
- goto err_add_devs;
+ return ret;
}
return 0;
-
-err_add_devs:
- if (irq_init_success)
- rc5t583_irq_exit(rc5t583);
- return ret;
-}
-
-static int rc5t583_i2c_remove(struct i2c_client *i2c)
-{
- struct rc5t583 *rc5t583 = i2c_get_clientdata(i2c);
-
- mfd_remove_devices(rc5t583->dev);
- rc5t583_irq_exit(rc5t583);
- return 0;
}
static const struct i2c_device_id rc5t583_i2c_id[] = {
@@ -324,7 +307,6 @@
.name = "rc5t583",
},
.probe = rc5t583_i2c_probe,
- .remove = rc5t583_i2c_remove,
.id_table = rc5t583_i2c_id,
};
diff --git a/drivers/mfd/rdc321x-southbridge.c b/drivers/mfd/rdc321x-southbridge.c
index 6575585..2bd8c5b 100644
--- a/drivers/mfd/rdc321x-southbridge.c
+++ b/drivers/mfd/rdc321x-southbridge.c
@@ -85,14 +85,10 @@
rdc321x_gpio_pdata.sb_pdev = pdev;
rdc321x_wdt_pdata.sb_pdev = pdev;
- return mfd_add_devices(&pdev->dev, -1,
- rdc321x_sb_cells, ARRAY_SIZE(rdc321x_sb_cells),
- NULL, 0, NULL);
-}
-
-static void rdc321x_sb_remove(struct pci_dev *pdev)
-{
- mfd_remove_devices(&pdev->dev);
+ return devm_mfd_add_devices(&pdev->dev, -1,
+ rdc321x_sb_cells,
+ ARRAY_SIZE(rdc321x_sb_cells),
+ NULL, 0, NULL);
}
static const struct pci_device_id rdc321x_sb_table[] = {
@@ -105,7 +101,6 @@
.name = "RDC321x Southbridge",
.id_table = rdc321x_sb_table,
.probe = rdc321x_sb_probe,
- .remove = rdc321x_sb_remove,
};
module_pci_driver(rdc321x_sb_driver);
diff --git a/drivers/mfd/rk808.c b/drivers/mfd/rk808.c
index 4b1e439..49d7f62 100644
--- a/drivers/mfd/rk808.c
+++ b/drivers/mfd/rk808.c
@@ -213,9 +213,9 @@
rk808->i2c = client;
i2c_set_clientdata(client, rk808);
- ret = mfd_add_devices(&client->dev, -1,
- rk808s, ARRAY_SIZE(rk808s),
- NULL, 0, regmap_irq_get_domain(rk808->irq_data));
+ ret = devm_mfd_add_devices(&client->dev, -1,
+ rk808s, ARRAY_SIZE(rk808s), NULL, 0,
+ regmap_irq_get_domain(rk808->irq_data));
if (ret) {
dev_err(&client->dev, "failed to add MFD devices %d\n", ret);
goto err_irq;
@@ -240,7 +240,6 @@
struct rk808 *rk808 = i2c_get_clientdata(client);
regmap_del_irq_chip(client->irq, rk808->irq_data);
- mfd_remove_devices(&client->dev);
pm_power_off = NULL;
return 0;
diff --git a/drivers/mfd/rn5t618.c b/drivers/mfd/rn5t618.c
index 6668571..0ad51d7 100644
--- a/drivers/mfd/rn5t618.c
+++ b/drivers/mfd/rn5t618.c
@@ -78,8 +78,8 @@
return ret;
}
- ret = mfd_add_devices(&i2c->dev, -1, rn5t618_cells,
- ARRAY_SIZE(rn5t618_cells), NULL, 0, NULL);
+ ret = devm_mfd_add_devices(&i2c->dev, -1, rn5t618_cells,
+ ARRAY_SIZE(rn5t618_cells), NULL, 0, NULL);
if (ret) {
dev_err(&i2c->dev, "failed to add sub-devices: %d\n", ret);
return ret;
@@ -102,7 +102,6 @@
pm_power_off = NULL;
}
- mfd_remove_devices(&i2c->dev);
return 0;
}
diff --git a/drivers/mfd/rt5033.c b/drivers/mfd/rt5033.c
index 2b95485..9bd089c 100644
--- a/drivers/mfd/rt5033.c
+++ b/drivers/mfd/rt5033.c
@@ -97,9 +97,9 @@
return ret;
}
- ret = mfd_add_devices(rt5033->dev, -1, rt5033_devs,
- ARRAY_SIZE(rt5033_devs), NULL, 0,
- regmap_irq_get_domain(rt5033->irq_data));
+ ret = devm_mfd_add_devices(rt5033->dev, -1, rt5033_devs,
+ ARRAY_SIZE(rt5033_devs), NULL, 0,
+ regmap_irq_get_domain(rt5033->irq_data));
if (ret < 0) {
dev_err(&i2c->dev, "Failed to add RT5033 child devices.\n");
return ret;
@@ -110,13 +110,6 @@
return 0;
}
-static int rt5033_i2c_remove(struct i2c_client *i2c)
-{
- mfd_remove_devices(&i2c->dev);
-
- return 0;
-}
-
static const struct i2c_device_id rt5033_i2c_id[] = {
{ "rt5033", },
{ }
@@ -135,7 +128,6 @@
.of_match_table = of_match_ptr(rt5033_dt_match),
},
.probe = rt5033_i2c_probe,
- .remove = rt5033_i2c_remove,
.id_table = rt5033_i2c_id,
};
module_i2c_driver(rt5033_driver);
diff --git a/drivers/mfd/sec-core.c b/drivers/mfd/sec-core.c
index 400e1d7..ca6b80d 100644
--- a/drivers/mfd/sec-core.c
+++ b/drivers/mfd/sec-core.c
@@ -481,29 +481,16 @@
/* If this happens the probe function is problem */
BUG();
}
- ret = mfd_add_devices(sec_pmic->dev, -1, sec_devs, num_sec_devs, NULL,
- 0, NULL);
+ ret = devm_mfd_add_devices(sec_pmic->dev, -1, sec_devs, num_sec_devs,
+ NULL, 0, NULL);
if (ret)
- goto err_mfd;
+ return ret;
device_init_wakeup(sec_pmic->dev, sec_pmic->wakeup);
sec_pmic_configure(sec_pmic);
sec_pmic_dump_rev(sec_pmic);
return ret;
-
-err_mfd:
- sec_irq_exit(sec_pmic);
- return ret;
-}
-
-static int sec_pmic_remove(struct i2c_client *i2c)
-{
- struct sec_pmic_dev *sec_pmic = i2c_get_clientdata(i2c);
-
- mfd_remove_devices(sec_pmic->dev);
- sec_irq_exit(sec_pmic);
- return 0;
}
static void sec_pmic_shutdown(struct i2c_client *i2c)
@@ -583,7 +570,6 @@
.of_match_table = of_match_ptr(sec_dt_match),
},
.probe = sec_pmic_probe,
- .remove = sec_pmic_remove,
.shutdown = sec_pmic_shutdown,
.id_table = sec_pmic_id,
};
diff --git a/drivers/mfd/sec-irq.c b/drivers/mfd/sec-irq.c
index d77de43..5eb59c233d5 100644
--- a/drivers/mfd/sec-irq.c
+++ b/drivers/mfd/sec-irq.c
@@ -483,10 +483,11 @@
return -EINVAL;
}
- ret = regmap_add_irq_chip(sec_pmic->regmap_pmic, sec_pmic->irq,
- IRQF_TRIGGER_FALLING | IRQF_ONESHOT,
- sec_pmic->irq_base, sec_irq_chip,
- &sec_pmic->irq_data);
+ ret = devm_regmap_add_irq_chip(sec_pmic->dev, sec_pmic->regmap_pmic,
+ sec_pmic->irq,
+ IRQF_TRIGGER_FALLING | IRQF_ONESHOT,
+ sec_pmic->irq_base, sec_irq_chip,
+ &sec_pmic->irq_data);
if (ret != 0) {
dev_err(sec_pmic->dev, "Failed to register IRQ chip: %d\n", ret);
return ret;
@@ -500,8 +501,3 @@
return 0;
}
-
-void sec_irq_exit(struct sec_pmic_dev *sec_pmic)
-{
- regmap_del_irq_chip(sec_pmic->irq, sec_pmic->irq_data);
-}
diff --git a/drivers/mfd/sky81452.c b/drivers/mfd/sky81452.c
index b0c9b04..30a2a67 100644
--- a/drivers/mfd/sky81452.c
+++ b/drivers/mfd/sky81452.c
@@ -64,19 +64,14 @@
cells[1].platform_data = pdata->regulator_init_data;
cells[1].pdata_size = sizeof(*pdata->regulator_init_data);
- ret = mfd_add_devices(dev, -1, cells, ARRAY_SIZE(cells), NULL, 0, NULL);
+ ret = devm_mfd_add_devices(dev, -1, cells, ARRAY_SIZE(cells),
+ NULL, 0, NULL);
if (ret)
dev_err(dev, "failed to add child devices. err=%d\n", ret);
return ret;
}
-static int sky81452_remove(struct i2c_client *client)
-{
- mfd_remove_devices(&client->dev);
- return 0;
-}
-
static const struct i2c_device_id sky81452_ids[] = {
{ "sky81452" },
{ }
@@ -97,7 +92,6 @@
.of_match_table = of_match_ptr(sky81452_of_match),
},
.probe = sky81452_probe,
- .remove = sky81452_remove,
.id_table = sky81452_ids,
};
diff --git a/drivers/mfd/sm501.c b/drivers/mfd/sm501.c
index c646784..65cd0d2 100644
--- a/drivers/mfd/sm501.c
+++ b/drivers/mfd/sm501.c
@@ -879,11 +879,6 @@
#ifdef CONFIG_MFD_SM501_GPIO
-static inline struct sm501_gpio_chip *to_sm501_gpio(struct gpio_chip *gc)
-{
- return container_of(gc, struct sm501_gpio_chip, gpio);
-}
-
static inline struct sm501_devdata *sm501_gpio_to_dev(struct sm501_gpio *gpio)
{
return container_of(gpio, struct sm501_devdata, gpio);
@@ -892,7 +887,7 @@
static int sm501_gpio_get(struct gpio_chip *chip, unsigned offset)
{
- struct sm501_gpio_chip *smgpio = to_sm501_gpio(chip);
+ struct sm501_gpio_chip *smgpio = gpiochip_get_data(chip);
unsigned long result;
result = smc501_readl(smgpio->regbase + SM501_GPIO_DATA_LOW);
@@ -923,7 +918,7 @@
static void sm501_gpio_set(struct gpio_chip *chip, unsigned offset, int value)
{
- struct sm501_gpio_chip *smchip = to_sm501_gpio(chip);
+ struct sm501_gpio_chip *smchip = gpiochip_get_data(chip);
struct sm501_gpio *smgpio = smchip->ourgpio;
unsigned long bit = 1 << offset;
void __iomem *regs = smchip->regbase;
@@ -948,7 +943,7 @@
static int sm501_gpio_input(struct gpio_chip *chip, unsigned offset)
{
- struct sm501_gpio_chip *smchip = to_sm501_gpio(chip);
+ struct sm501_gpio_chip *smchip = gpiochip_get_data(chip);
struct sm501_gpio *smgpio = smchip->ourgpio;
void __iomem *regs = smchip->regbase;
unsigned long bit = 1 << offset;
@@ -974,7 +969,7 @@
static int sm501_gpio_output(struct gpio_chip *chip,
unsigned offset, int value)
{
- struct sm501_gpio_chip *smchip = to_sm501_gpio(chip);
+ struct sm501_gpio_chip *smchip = gpiochip_get_data(chip);
struct sm501_gpio *smgpio = smchip->ourgpio;
unsigned long bit = 1 << offset;
void __iomem *regs = smchip->regbase;
@@ -1039,7 +1034,7 @@
gchip->base = base;
chip->ourgpio = gpio;
- return gpiochip_add(gchip);
+ return gpiochip_add_data(gchip, chip);
}
static int sm501_register_gpio(struct sm501_devdata *sm)
diff --git a/drivers/mfd/smsc-ece1099.c b/drivers/mfd/smsc-ece1099.c
index a4c0df7..7f89e89 100644
--- a/drivers/mfd/smsc-ece1099.c
+++ b/drivers/mfd/smsc-ece1099.c
@@ -80,15 +80,6 @@
return ret;
}
-static int smsc_i2c_remove(struct i2c_client *i2c)
-{
- struct smsc *smsc = i2c_get_clientdata(i2c);
-
- mfd_remove_devices(smsc->dev);
-
- return 0;
-}
-
static const struct i2c_device_id smsc_i2c_id[] = {
{ "smscece1099", 0},
{},
@@ -100,7 +91,6 @@
.name = "smsc",
},
.probe = smsc_i2c_probe,
- .remove = smsc_i2c_remove,
.id_table = smsc_i2c_id,
};
diff --git a/drivers/mfd/stw481x.c b/drivers/mfd/stw481x.c
index ca613df..ab949ea 100644
--- a/drivers/mfd/stw481x.c
+++ b/drivers/mfd/stw481x.c
@@ -206,8 +206,8 @@
stw481x_cells[i].pdata_size = sizeof(*stw481x);
}
- ret = mfd_add_devices(&client->dev, 0, stw481x_cells,
- ARRAY_SIZE(stw481x_cells), NULL, 0, NULL);
+ ret = devm_mfd_add_devices(&client->dev, 0, stw481x_cells,
+ ARRAY_SIZE(stw481x_cells), NULL, 0, NULL);
if (ret)
return ret;
@@ -216,12 +216,6 @@
return ret;
}
-static int stw481x_remove(struct i2c_client *client)
-{
- mfd_remove_devices(&client->dev);
- return 0;
-}
-
/*
* This ID table is completely unused, as this is a pure
* device-tree probed driver, but it has to be here due to
@@ -246,7 +240,6 @@
.of_match_table = stw481x_match,
},
.probe = stw481x_probe,
- .remove = stw481x_remove,
.id_table = stw481x_id,
};
diff --git a/drivers/mfd/tc6393xb.c b/drivers/mfd/tc6393xb.c
index 1ecbfa4..d42d322 100644
--- a/drivers/mfd/tc6393xb.c
+++ b/drivers/mfd/tc6393xb.c
@@ -24,7 +24,7 @@
#include <linux/mfd/core.h>
#include <linux/mfd/tmio.h>
#include <linux/mfd/tc6393xb.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
#include <linux/slab.h>
#define SCR_REVID 0x08 /* b Revision ID */
@@ -434,7 +434,7 @@
static int tc6393xb_gpio_get(struct gpio_chip *chip,
unsigned offset)
{
- struct tc6393xb *tc6393xb = container_of(chip, struct tc6393xb, gpio);
+ struct tc6393xb *tc6393xb = gpiochip_get_data(chip);
/* XXX: does dsr also represent inputs? */
return !!(tmio_ioread8(tc6393xb->scr + SCR_GPO_DSR(offset / 8))
@@ -444,7 +444,7 @@
static void __tc6393xb_gpio_set(struct gpio_chip *chip,
unsigned offset, int value)
{
- struct tc6393xb *tc6393xb = container_of(chip, struct tc6393xb, gpio);
+ struct tc6393xb *tc6393xb = gpiochip_get_data(chip);
u8 dsr;
dsr = tmio_ioread8(tc6393xb->scr + SCR_GPO_DSR(offset / 8));
@@ -459,7 +459,7 @@
static void tc6393xb_gpio_set(struct gpio_chip *chip,
unsigned offset, int value)
{
- struct tc6393xb *tc6393xb = container_of(chip, struct tc6393xb, gpio);
+ struct tc6393xb *tc6393xb = gpiochip_get_data(chip);
unsigned long flags;
spin_lock_irqsave(&tc6393xb->lock, flags);
@@ -472,7 +472,7 @@
static int tc6393xb_gpio_direction_input(struct gpio_chip *chip,
unsigned offset)
{
- struct tc6393xb *tc6393xb = container_of(chip, struct tc6393xb, gpio);
+ struct tc6393xb *tc6393xb = gpiochip_get_data(chip);
unsigned long flags;
u8 doecr;
@@ -490,7 +490,7 @@
static int tc6393xb_gpio_direction_output(struct gpio_chip *chip,
unsigned offset, int value)
{
- struct tc6393xb *tc6393xb = container_of(chip, struct tc6393xb, gpio);
+ struct tc6393xb *tc6393xb = gpiochip_get_data(chip);
unsigned long flags;
u8 doecr;
@@ -517,7 +517,7 @@
tc6393xb->gpio.direction_input = tc6393xb_gpio_direction_input;
tc6393xb->gpio.direction_output = tc6393xb_gpio_direction_output;
- return gpiochip_add(&tc6393xb->gpio);
+ return gpiochip_add_data(&tc6393xb->gpio, tc6393xb);
}
/*--------------------------------------------------------------------------*/
diff --git a/drivers/mfd/tps6105x.c b/drivers/mfd/tps6105x.c
index 51c5495..baa12ea 100644
--- a/drivers/mfd/tps6105x.c
+++ b/drivers/mfd/tps6105x.c
@@ -21,7 +21,6 @@
#include <linux/spinlock.h>
#include <linux/slab.h>
#include <linux/err.h>
-#include <linux/regulator/driver.h>
#include <linux/mfd/core.h>
#include <linux/mfd/tps6105x.h>
diff --git a/drivers/mfd/tps65010.c b/drivers/mfd/tps65010.c
index 495e451..d829a61 100644
--- a/drivers/mfd/tps65010.c
+++ b/drivers/mfd/tps65010.c
@@ -34,7 +34,7 @@
#include <linux/i2c/tps65010.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
/*-------------------------------------------------------------------------*/
@@ -477,7 +477,7 @@
if (offset < 4) {
struct tps65010 *tps;
- tps = container_of(chip, struct tps65010, chip);
+ tps = gpiochip_get_data(chip);
if (!(tps->outmask & (1 << offset)))
return -EINVAL;
tps65010_set_gpio_out_value(offset + 1, value);
@@ -494,7 +494,7 @@
int value;
struct tps65010 *tps;
- tps = container_of(chip, struct tps65010, chip);
+ tps = gpiochip_get_data(chip);
if (offset < 4) {
value = i2c_smbus_read_byte_data(tps->client, TPS_DEFGPIO);
@@ -651,7 +651,7 @@
tps->chip.ngpio = 7;
tps->chip.can_sleep = 1;
- status = gpiochip_add(&tps->chip);
+ status = gpiochip_add_data(&tps->chip, tps);
if (status < 0)
dev_err(&client->dev, "can't add gpiochip, err %d\n",
status);
diff --git a/drivers/mfd/tps6507x.c b/drivers/mfd/tps6507x.c
index 1ab3dd6..40beb2f 100644
--- a/drivers/mfd/tps6507x.c
+++ b/drivers/mfd/tps6507x.c
@@ -100,16 +100,8 @@
tps6507x->read_dev = tps6507x_i2c_read_device;
tps6507x->write_dev = tps6507x_i2c_write_device;
- return mfd_add_devices(tps6507x->dev, -1, tps6507x_devs,
- ARRAY_SIZE(tps6507x_devs), NULL, 0, NULL);
-}
-
-static int tps6507x_i2c_remove(struct i2c_client *i2c)
-{
- struct tps6507x_dev *tps6507x = i2c_get_clientdata(i2c);
-
- mfd_remove_devices(tps6507x->dev);
- return 0;
+ return devm_mfd_add_devices(tps6507x->dev, -1, tps6507x_devs,
+ ARRAY_SIZE(tps6507x_devs), NULL, 0, NULL);
}
static const struct i2c_device_id tps6507x_i2c_id[] = {
@@ -132,7 +124,6 @@
.of_match_table = of_match_ptr(tps6507x_of_match),
},
.probe = tps6507x_i2c_probe,
- .remove = tps6507x_i2c_remove,
.id_table = tps6507x_i2c_id,
};
diff --git a/drivers/mfd/tps65217.c b/drivers/mfd/tps65217.c
index d32b5442..049a6fc 100644
--- a/drivers/mfd/tps65217.c
+++ b/drivers/mfd/tps65217.c
@@ -205,8 +205,8 @@
return ret;
}
- ret = mfd_add_devices(tps->dev, -1, tps65217s,
- ARRAY_SIZE(tps65217s), NULL, 0, NULL);
+ ret = devm_mfd_add_devices(tps->dev, -1, tps65217s,
+ ARRAY_SIZE(tps65217s), NULL, 0, NULL);
if (ret < 0) {
dev_err(tps->dev, "mfd_add_devices failed: %d\n", ret);
return ret;
@@ -235,15 +235,6 @@
return 0;
}
-static int tps65217_remove(struct i2c_client *client)
-{
- struct tps65217 *tps = i2c_get_clientdata(client);
-
- mfd_remove_devices(tps->dev);
-
- return 0;
-}
-
static const struct i2c_device_id tps65217_id_table[] = {
{"tps65217", TPS65217},
{ /* sentinel */ }
@@ -257,7 +248,6 @@
},
.id_table = tps65217_id_table,
.probe = tps65217_probe,
- .remove = tps65217_remove,
};
static int __init tps65217_init(void)
diff --git a/drivers/mfd/tps65910.c b/drivers/mfd/tps65910.c
index f7ab115..11cab15 100644
--- a/drivers/mfd/tps65910.c
+++ b/drivers/mfd/tps65910.c
@@ -252,9 +252,10 @@
}
tps65910->chip_irq = irq;
- ret = regmap_add_irq_chip(tps65910->regmap, tps65910->chip_irq,
- IRQF_ONESHOT, pdata->irq_base,
- tps6591x_irqs_chip, &tps65910->irq_data);
+ ret = devm_regmap_add_irq_chip(tps65910->dev, tps65910->regmap,
+ tps65910->chip_irq,
+ IRQF_ONESHOT, pdata->irq_base,
+ tps6591x_irqs_chip, &tps65910->irq_data);
if (ret < 0) {
dev_warn(tps65910->dev, "Failed to add irq_chip %d\n", ret);
tps65910->chip_irq = 0;
@@ -262,13 +263,6 @@
return ret;
}
-static int tps65910_irq_exit(struct tps65910 *tps65910)
-{
- if (tps65910->chip_irq > 0)
- regmap_del_irq_chip(tps65910->chip_irq, tps65910->irq_data);
- return 0;
-}
-
static bool is_volatile_reg(struct device *dev, unsigned int reg)
{
struct tps65910 *tps65910 = dev_get_drvdata(dev);
@@ -510,29 +504,18 @@
pm_power_off = tps65910_power_off;
}
- ret = mfd_add_devices(tps65910->dev, -1,
- tps65910s, ARRAY_SIZE(tps65910s),
- NULL, 0,
- regmap_irq_get_domain(tps65910->irq_data));
+ ret = devm_mfd_add_devices(tps65910->dev, -1,
+ tps65910s, ARRAY_SIZE(tps65910s),
+ NULL, 0,
+ regmap_irq_get_domain(tps65910->irq_data));
if (ret < 0) {
dev_err(&i2c->dev, "mfd_add_devices failed: %d\n", ret);
- tps65910_irq_exit(tps65910);
return ret;
}
return ret;
}
-static int tps65910_i2c_remove(struct i2c_client *i2c)
-{
- struct tps65910 *tps65910 = i2c_get_clientdata(i2c);
-
- tps65910_irq_exit(tps65910);
- mfd_remove_devices(tps65910->dev);
-
- return 0;
-}
-
static const struct i2c_device_id tps65910_i2c_id[] = {
{ "tps65910", TPS65910 },
{ "tps65911", TPS65911 },
@@ -547,7 +530,6 @@
.of_match_table = of_match_ptr(tps65910_of_match),
},
.probe = tps65910_i2c_probe,
- .remove = tps65910_i2c_remove,
.id_table = tps65910_i2c_id,
};
diff --git a/drivers/mfd/twl4030-power.c b/drivers/mfd/twl4030-power.c
index 04b5398..1beb722 100644
--- a/drivers/mfd/twl4030-power.c
+++ b/drivers/mfd/twl4030-power.c
@@ -1,5 +1,4 @@
/*
- * linux/drivers/i2c/chips/twl4030-power.c
*
* Handle TWL4030 Power initialization
*
diff --git a/drivers/mfd/twl6040.c b/drivers/mfd/twl6040.c
index 08a693c..852d587 100644
--- a/drivers/mfd/twl6040.c
+++ b/drivers/mfd/twl6040.c
@@ -291,7 +291,11 @@
if (twl6040->power_count++)
goto out;
- clk_prepare_enable(twl6040->clk32k);
+ ret = clk_prepare_enable(twl6040->clk32k);
+ if (ret) {
+ twl6040->power_count = 0;
+ goto out;
+ }
/* Allow writes to the chip */
regcache_cache_only(twl6040->regmap, false);
@@ -300,6 +304,7 @@
/* use automatic power-up sequence */
ret = twl6040_power_up_automatic(twl6040);
if (ret) {
+ clk_disable_unprepare(twl6040->clk32k);
twl6040->power_count = 0;
goto out;
}
@@ -307,6 +312,7 @@
/* use manual power-up sequence */
ret = twl6040_power_up_manual(twl6040);
if (ret) {
+ clk_disable_unprepare(twl6040->clk32k);
twl6040->power_count = 0;
goto out;
}
diff --git a/drivers/mfd/ucb1x00-core.c b/drivers/mfd/ucb1x00-core.c
index bcafe1e..9ab9ec4 100644
--- a/drivers/mfd/ucb1x00-core.c
+++ b/drivers/mfd/ucb1x00-core.c
@@ -28,7 +28,7 @@
#include <linux/mutex.h>
#include <linux/mfd/ucb1x00.h>
#include <linux/pm.h>
-#include <linux/gpio.h>
+#include <linux/gpio/driver.h>
static DEFINE_MUTEX(ucb1x00_mutex);
static LIST_HEAD(ucb1x00_drivers);
@@ -109,7 +109,7 @@
static void ucb1x00_gpio_set(struct gpio_chip *chip, unsigned offset, int value)
{
- struct ucb1x00 *ucb = container_of(chip, struct ucb1x00, gpio);
+ struct ucb1x00 *ucb = gpiochip_get_data(chip);
unsigned long flags;
spin_lock_irqsave(&ucb->io_lock, flags);
@@ -126,7 +126,7 @@
static int ucb1x00_gpio_get(struct gpio_chip *chip, unsigned offset)
{
- struct ucb1x00 *ucb = container_of(chip, struct ucb1x00, gpio);
+ struct ucb1x00 *ucb = gpiochip_get_data(chip);
unsigned val;
ucb1x00_enable(ucb);
@@ -138,7 +138,7 @@
static int ucb1x00_gpio_direction_input(struct gpio_chip *chip, unsigned offset)
{
- struct ucb1x00 *ucb = container_of(chip, struct ucb1x00, gpio);
+ struct ucb1x00 *ucb = gpiochip_get_data(chip);
unsigned long flags;
spin_lock_irqsave(&ucb->io_lock, flags);
@@ -154,7 +154,7 @@
static int ucb1x00_gpio_direction_output(struct gpio_chip *chip, unsigned offset
, int value)
{
- struct ucb1x00 *ucb = container_of(chip, struct ucb1x00, gpio);
+ struct ucb1x00 *ucb = gpiochip_get_data(chip);
unsigned long flags;
unsigned old, mask = 1 << offset;
@@ -181,7 +181,7 @@
static int ucb1x00_to_irq(struct gpio_chip *chip, unsigned offset)
{
- struct ucb1x00 *ucb = container_of(chip, struct ucb1x00, gpio);
+ struct ucb1x00 *ucb = gpiochip_get_data(chip);
return ucb->irq_base > 0 ? ucb->irq_base + offset : -ENXIO;
}
@@ -579,7 +579,7 @@
ucb->gpio.direction_input = ucb1x00_gpio_direction_input;
ucb->gpio.direction_output = ucb1x00_gpio_direction_output;
ucb->gpio.to_irq = ucb1x00_to_irq;
- ret = gpiochip_add(&ucb->gpio);
+ ret = gpiochip_add_data(&ucb->gpio, ucb);
if (ret)
goto err_gpio_add;
} else
diff --git a/drivers/mfd/vexpress-sysreg.c b/drivers/mfd/vexpress-sysreg.c
index 855c020..201a3ea2 100644
--- a/drivers/mfd/vexpress-sysreg.c
+++ b/drivers/mfd/vexpress-sysreg.c
@@ -202,7 +202,7 @@
bgpio_init(mmc_gpio_chip, &pdev->dev, 0x4, base + SYS_MCI,
NULL, NULL, NULL, NULL, 0);
mmc_gpio_chip->ngpio = 2;
- gpiochip_add(mmc_gpio_chip);
+ gpiochip_add_data(mmc_gpio_chip, NULL);
return mfd_add_devices(&pdev->dev, PLATFORM_DEVID_AUTO,
vexpress_sysreg_cells,
diff --git a/drivers/mfd/wl1273-core.c b/drivers/mfd/wl1273-core.c
index f7c52d9..7080465 100644
--- a/drivers/mfd/wl1273-core.c
+++ b/drivers/mfd/wl1273-core.c
@@ -170,15 +170,6 @@
return 0;
}
-static int wl1273_core_remove(struct i2c_client *client)
-{
- dev_dbg(&client->dev, "%s\n", __func__);
-
- mfd_remove_devices(&client->dev);
-
- return 0;
-}
-
static int wl1273_core_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
@@ -237,8 +228,8 @@
dev_dbg(&client->dev, "%s: number of children: %d.\n",
__func__, children);
- r = mfd_add_devices(&client->dev, -1, core->cells,
- children, NULL, 0, NULL);
+ r = devm_mfd_add_devices(&client->dev, -1, core->cells,
+ children, NULL, 0, NULL);
if (r)
goto err;
@@ -258,7 +249,6 @@
},
.probe = wl1273_core_probe,
.id_table = wl1273_driver_id_table,
- .remove = wl1273_core_remove,
};
static int __init wl1273_core_init(void)
diff --git a/drivers/mfd/wm5110-tables.c b/drivers/mfd/wm5110-tables.c
index 8e74e71..1ee68bd 100644
--- a/drivers/mfd/wm5110-tables.c
+++ b/drivers/mfd/wm5110-tables.c
@@ -3066,6 +3066,7 @@
case ARIZONA_AOD_IRQ_RAW_STATUS:
case ARIZONA_FX_CTRL2:
case ARIZONA_ASRC_STATUS:
+ case ARIZONA_CLOCK_CONTROL:
case ARIZONA_DSP_STATUS:
case ARIZONA_DSP1_STATUS_1:
case ARIZONA_DSP1_STATUS_2:
diff --git a/drivers/mfd/wm8400-core.c b/drivers/mfd/wm8400-core.c
index 3bd44a4..8a98a2f 100644
--- a/drivers/mfd/wm8400-core.c
+++ b/drivers/mfd/wm8400-core.c
@@ -35,27 +35,6 @@
}
}
-/**
- * wm8400_reg_read - Single register read
- *
- * @wm8400: Pointer to wm8400 control structure
- * @reg: Register to read
- *
- * @return Read value
- */
-u16 wm8400_reg_read(struct wm8400 *wm8400, u8 reg)
-{
- unsigned int val;
- int ret;
-
- ret = regmap_read(wm8400->regmap, reg, &val);
- if (ret < 0)
- return ret;
-
- return val;
-}
-EXPORT_SYMBOL_GPL(wm8400_reg_read);
-
int wm8400_block_read(struct wm8400 *wm8400, u8 reg, int count, u16 *data)
{
return regmap_bulk_read(wm8400->regmap, reg, data, count);
@@ -70,7 +49,7 @@
.pdata_size = sizeof(*wm8400),
};
- return mfd_add_devices(wm8400->dev, -1, &cell, 1, NULL, 0, NULL);
+ return devm_mfd_add_devices(wm8400->dev, -1, &cell, 1, NULL, 0, NULL);
}
/*
@@ -111,7 +90,7 @@
ret = wm8400_register_codec(wm8400);
if (ret != 0) {
dev_err(wm8400->dev, "Failed to register codec\n");
- goto err_children;
+ return ret;
}
if (pdata && pdata->platform_init) {
@@ -119,21 +98,12 @@
if (ret != 0) {
dev_err(wm8400->dev, "Platform init failed: %d\n",
ret);
- goto err_children;
+ return ret;
}
} else
dev_warn(wm8400->dev, "No platform initialisation supplied\n");
return 0;
-
-err_children:
- mfd_remove_devices(wm8400->dev);
- return ret;
-}
-
-static void wm8400_release(struct wm8400 *wm8400)
-{
- mfd_remove_devices(wm8400->dev);
}
static const struct regmap_config wm8400_regmap_config = {
@@ -156,7 +126,7 @@
}
EXPORT_SYMBOL_GPL(wm8400_reset_codec_reg_cache);
-#if defined(CONFIG_I2C) || defined(CONFIG_I2C_MODULE)
+#if IS_ENABLED(CONFIG_I2C)
static int wm8400_i2c_probe(struct i2c_client *i2c,
const struct i2c_device_id *id)
{
@@ -176,15 +146,6 @@
return wm8400_init(wm8400, dev_get_platdata(&i2c->dev));
}
-static int wm8400_i2c_remove(struct i2c_client *i2c)
-{
- struct wm8400 *wm8400 = i2c_get_clientdata(i2c);
-
- wm8400_release(wm8400);
-
- return 0;
-}
-
static const struct i2c_device_id wm8400_i2c_id[] = {
{ "wm8400", 0 },
{ }
@@ -196,7 +157,6 @@
.name = "WM8400",
},
.probe = wm8400_i2c_probe,
- .remove = wm8400_i2c_remove,
.id_table = wm8400_i2c_id,
};
#endif
@@ -205,7 +165,7 @@
{
int ret = -ENODEV;
-#if defined(CONFIG_I2C) || defined(CONFIG_I2C_MODULE)
+#if IS_ENABLED(CONFIG_I2C)
ret = i2c_add_driver(&wm8400_i2c_driver);
if (ret != 0)
pr_err("Failed to register I2C driver: %d\n", ret);
@@ -217,7 +177,7 @@
static void __exit wm8400_module_exit(void)
{
-#if defined(CONFIG_I2C) || defined(CONFIG_I2C_MODULE)
+#if IS_ENABLED(CONFIG_I2C)
i2c_del_driver(&wm8400_i2c_driver);
#endif
}
diff --git a/drivers/misc/cxl/api.c b/drivers/misc/cxl/api.c
index 2107c94..6d228cc 100644
--- a/drivers/misc/cxl/api.c
+++ b/drivers/misc/cxl/api.c
@@ -68,15 +68,6 @@
}
EXPORT_SYMBOL_GPL(cxl_get_context);
-struct device *cxl_get_phys_dev(struct pci_dev *dev)
-{
- struct cxl_afu *afu;
-
- afu = cxl_pci_to_afu(dev);
-
- return afu->adapter->dev.parent;
-}
-
int cxl_release_context(struct cxl_context *ctx)
{
if (ctx->status >= STARTED)
@@ -192,6 +183,7 @@
ctx->pid = get_task_pid(task, PIDTYPE_PID);
ctx->glpid = get_task_pid(task->group_leader, PIDTYPE_PID);
kernel = false;
+ ctx->real_mode = false;
}
cxl_ctx_get();
@@ -228,6 +220,24 @@
}
EXPORT_SYMBOL_GPL(cxl_set_master);
+int cxl_set_translation_mode(struct cxl_context *ctx, bool real_mode)
+{
+ if (ctx->status == STARTED) {
+ /*
+ * We could potentially update the PE and issue an update LLCMD
+ * to support this, but it doesn't seem to have a good use case
+ * since it's trivial to just create a second kernel context
+ * with different translation modes, so until someone convinces
+ * me otherwise:
+ */
+ return -EBUSY;
+ }
+
+ ctx->real_mode = real_mode;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(cxl_set_translation_mode);
+
/* wrappers around afu_* file ops which are EXPORTED */
int cxl_fd_open(struct inode *inode, struct file *file)
{
diff --git a/drivers/misc/cxl/context.c b/drivers/misc/cxl/context.c
index 7edea9c..26d206b 100644
--- a/drivers/misc/cxl/context.c
+++ b/drivers/misc/cxl/context.c
@@ -297,8 +297,7 @@
if (ctx->kernelapi)
kfree(ctx->mapping);
- if (ctx->irq_bitmap)
- kfree(ctx->irq_bitmap);
+ kfree(ctx->irq_bitmap);
/* Drop ref to the afu device taken during cxl_context_init */
cxl_afu_put(ctx->afu);
diff --git a/drivers/misc/cxl/cxl.h b/drivers/misc/cxl/cxl.h
index 73dc2a3..4fe5078 100644
--- a/drivers/misc/cxl/cxl.h
+++ b/drivers/misc/cxl/cxl.h
@@ -178,15 +178,6 @@
#define CXL_PSL_SR_An_MP (1ull << (63-62)) /* Master Process */
#define CXL_PSL_SR_An_LE (1ull << (63-63)) /* Little Endian */
-/****** CXL_PSL_LLCMD_An ****************************************************/
-#define CXL_LLCMD_TERMINATE 0x0001000000000000ULL
-#define CXL_LLCMD_REMOVE 0x0002000000000000ULL
-#define CXL_LLCMD_SUSPEND 0x0003000000000000ULL
-#define CXL_LLCMD_RESUME 0x0004000000000000ULL
-#define CXL_LLCMD_ADD 0x0005000000000000ULL
-#define CXL_LLCMD_UPDATE 0x0006000000000000ULL
-#define CXL_LLCMD_HANDLE_MASK 0x000000000000ffffULL
-
/****** CXL_PSL_ID_An ****************************************************/
#define CXL_PSL_ID_An_F (1ull << (63-31))
#define CXL_PSL_ID_An_L (1ull << (63-30))
@@ -376,11 +367,13 @@
};
struct cxl_afu_guest {
+ struct cxl_afu *parent;
u64 handle;
phys_addr_t p2n_phys;
u64 p2n_size;
int max_ints;
- struct mutex recovery_lock;
+ bool handle_err;
+ struct delayed_work work_err;
int previous_state;
};
@@ -524,6 +517,7 @@
bool pe_inserted;
bool master;
bool kernel;
+ bool real_mode;
bool pending_irq;
bool pending_fault;
bool pending_afu_err;
@@ -580,6 +574,7 @@
bool perst_loads_image;
bool perst_select_user;
bool perst_same_image;
+ bool psl_timebase_synced;
};
int cxl_pci_alloc_one_irq(struct cxl *adapter);
diff --git a/drivers/misc/cxl/fault.c b/drivers/misc/cxl/fault.c
index 9a8650b..377e650 100644
--- a/drivers/misc/cxl/fault.c
+++ b/drivers/misc/cxl/fault.c
@@ -149,11 +149,13 @@
* update_mmu_cache() will not have loaded the hash since current->trap
* is not a 0x400 or 0x300, so just call hash_page_mm() here.
*/
- access = _PAGE_PRESENT;
+ access = _PAGE_PRESENT | _PAGE_READ;
if (dsisr & CXL_PSL_DSISR_An_S)
- access |= _PAGE_RW;
- if ((!ctx->kernel) || ~(dar & (1ULL << 63)))
- access |= _PAGE_USER;
+ access |= _PAGE_WRITE;
+
+ access |= _PAGE_PRIVILEGED;
+ if ((!ctx->kernel) || (REGION_ID(dar) == USER_REGION_ID))
+ access &= ~_PAGE_PRIVILEGED;
if (dsisr & DSISR_NOHPTE)
inv_flags |= HPTE_NOHPTE_UPDATE;
diff --git a/drivers/misc/cxl/guest.c b/drivers/misc/cxl/guest.c
index 8213372..bc8d0b9 100644
--- a/drivers/misc/cxl/guest.c
+++ b/drivers/misc/cxl/guest.c
@@ -178,6 +178,9 @@
u64 state;
int rc = 0;
+ if (!afu)
+ return -EIO;
+
rc = cxl_h_read_error_state(afu->guest->handle, &state);
if (!rc) {
WARN_ON(state != H_STATE_NORMAL &&
@@ -552,6 +555,17 @@
elem->common.sstp0 = cpu_to_be64(ctx->sstp0);
elem->common.sstp1 = cpu_to_be64(ctx->sstp1);
+
+ /*
+ * Ensure we have at least one interrupt allocated to take faults for
+ * kernel contexts that may not have allocated any AFU IRQs at all:
+ */
+ if (ctx->irqs.range[0] == 0) {
+ rc = afu_register_irqs(ctx, 0);
+ if (rc)
+ goto out_free;
+ }
+
for (r = 0; r < CXL_IRQ_RANGES; r++) {
for (i = 0; i < ctx->irqs.range[r]; i++) {
if (r == 0 && i == 0) {
@@ -597,6 +611,7 @@
enable_afu_irqs(ctx);
}
+out_free:
free_page((u64)elem);
return rc;
}
@@ -605,6 +620,9 @@
{
pr_devel("in %s\n", __func__);
+ if (ctx->real_mode)
+ return -EPERM;
+
ctx->kernel = kernel;
if (ctx->afu->current_mode == CXL_MODE_DIRECTED)
return attach_afu_directed(ctx, wed, amr);
@@ -818,7 +836,6 @@
switch (cur_state) {
case H_STATE_NORMAL:
afu->guest->previous_state = cur_state;
- rc = 1;
break;
case H_STATE_DISABLE:
@@ -834,7 +851,6 @@
pci_error_handlers(afu, CXL_SLOT_RESET_EVENT,
pci_channel_io_normal);
pci_error_handlers(afu, CXL_RESUME_EVENT, 0);
- rc = 1;
}
afu->guest->previous_state = 0;
break;
@@ -859,39 +875,30 @@
return rc;
}
-static int afu_do_recovery(struct cxl_afu *afu)
+static void afu_handle_errstate(struct work_struct *work)
{
- int rc;
+ struct cxl_afu_guest *afu_guest =
+ container_of(to_delayed_work(work), struct cxl_afu_guest, work_err);
- /* many threads can arrive here, in case of detach_all for example.
- * Only one needs to drive the recovery
- */
- if (mutex_trylock(&afu->guest->recovery_lock)) {
- rc = afu_update_state(afu);
- mutex_unlock(&afu->guest->recovery_lock);
- return rc;
- }
- return 0;
+ if (!afu_update_state(afu_guest->parent) &&
+ afu_guest->previous_state == H_STATE_PERM_UNAVAILABLE)
+ return;
+
+ if (afu_guest->handle_err == true)
+ schedule_delayed_work(&afu_guest->work_err,
+ msecs_to_jiffies(3000));
}
static bool guest_link_ok(struct cxl *cxl, struct cxl_afu *afu)
{
int state;
- if (afu) {
- if (afu_read_error_state(afu, &state) ||
- state != H_STATE_NORMAL) {
- if (afu_do_recovery(afu) > 0) {
- /* check again in case we've just fixed it */
- if (!afu_read_error_state(afu, &state) &&
- state == H_STATE_NORMAL)
- return true;
- }
- return false;
- }
+ if (afu && (!afu_read_error_state(afu, &state))) {
+ if (state == H_STATE_NORMAL)
+ return true;
}
- return true;
+ return false;
}
static int afu_properties_look_ok(struct cxl_afu *afu)
@@ -929,8 +936,6 @@
return -ENOMEM;
}
- mutex_init(&afu->guest->recovery_lock);
-
if ((rc = dev_set_name(&afu->dev, "afu%i.%i",
adapter->adapter_num,
slice)))
@@ -986,6 +991,15 @@
afu->enabled = true;
+ /*
+ * wake up the cpu periodically to check the state
+ * of the AFU using "afu" stored in the guest structure.
+ */
+ afu->guest->parent = afu;
+ afu->guest->handle_err = true;
+ INIT_DELAYED_WORK(&afu->guest->work_err, afu_handle_errstate);
+ schedule_delayed_work(&afu->guest->work_err, msecs_to_jiffies(1000));
+
if ((rc = cxl_pci_vphb_add(afu)))
dev_info(&afu->dev, "Can't register vPHB\n");
@@ -1014,6 +1028,10 @@
if (!afu)
return;
+ /* flush and stop pending job */
+ afu->guest->handle_err = false;
+ flush_delayed_work(&afu->guest->work_err);
+
cxl_pci_vphb_remove(afu);
cxl_sysfs_afu_remove(afu);
@@ -1101,6 +1119,12 @@
adapter->dev.release = release_adapter;
dev_set_drvdata(&pdev->dev, adapter);
+ /*
+ * Hypervisor controls PSL timebase initialization (p1 register).
+ * On FW840, PSL is initialized.
+ */
+ adapter->psl_timebase_synced = true;
+
if ((rc = cxl_of_read_adapter_handle(adapter, np)))
goto err1;
diff --git a/drivers/misc/cxl/native.c b/drivers/misc/cxl/native.c
index ecf7557..55d8a14 100644
--- a/drivers/misc/cxl/native.c
+++ b/drivers/misc/cxl/native.c
@@ -186,16 +186,25 @@
int cxl_alloc_spa(struct cxl_afu *afu)
{
+ unsigned spa_size;
+
/* Work out how many pages to allocate */
afu->native->spa_order = 0;
do {
afu->native->spa_order++;
- afu->native->spa_size = (1 << afu->native->spa_order) * PAGE_SIZE;
+ spa_size = (1 << afu->native->spa_order) * PAGE_SIZE;
+
+ if (spa_size > 0x100000) {
+ dev_warn(&afu->dev, "num_of_processes too large for the SPA, limiting to %i (0x%x)\n",
+ afu->native->spa_max_procs, afu->native->spa_size);
+ afu->num_procs = afu->native->spa_max_procs;
+ break;
+ }
+
+ afu->native->spa_size = spa_size;
afu->native->spa_max_procs = spa_max_procs(afu->native->spa_size);
} while (afu->native->spa_max_procs < afu->num_procs);
- WARN_ON(afu->native->spa_size > 0x100000); /* Max size supported by the hardware */
-
if (!(afu->native->spa = (struct cxl_process_element *)
__get_free_pages(GFP_KERNEL | __GFP_ZERO, afu->native->spa_order))) {
pr_err("cxl_alloc_spa: Unable to allocate scheduled process area\n");
@@ -486,8 +495,9 @@
if (mfspr(SPRN_LPCR) & LPCR_TC)
sr |= CXL_PSL_SR_An_TC;
if (ctx->kernel) {
- sr |= CXL_PSL_SR_An_R | (mfmsr() & MSR_SF);
- sr |= CXL_PSL_SR_An_HV;
+ if (!ctx->real_mode)
+ sr |= CXL_PSL_SR_An_R;
+ sr |= (mfmsr() & MSR_SF) | CXL_PSL_SR_An_HV;
} else {
sr |= CXL_PSL_SR_An_PR | CXL_PSL_SR_An_R;
sr &= ~(CXL_PSL_SR_An_HV);
@@ -526,6 +536,15 @@
ctx->elem->common.sstp0 = cpu_to_be64(ctx->sstp0);
ctx->elem->common.sstp1 = cpu_to_be64(ctx->sstp1);
+ /*
+ * Ensure we have the multiplexed PSL interrupt set up to take faults
+ * for kernel contexts that may not have allocated any AFU IRQs at all:
+ */
+ if (ctx->irqs.range[0] == 0) {
+ ctx->irqs.offset[0] = ctx->afu->native->psl_hwirq;
+ ctx->irqs.range[0] = 1;
+ }
+
for (r = 0; r < CXL_IRQ_RANGES; r++) {
ctx->elem->ivte_offsets[r] = cpu_to_be16(ctx->irqs.offset[r]);
ctx->elem->ivte_ranges[r] = cpu_to_be16(ctx->irqs.range[r]);
diff --git a/drivers/misc/cxl/pci.c b/drivers/misc/cxl/pci.c
index 2844e97..a08fcc8 100644
--- a/drivers/misc/cxl/pci.c
+++ b/drivers/misc/cxl/pci.c
@@ -21,6 +21,7 @@
#include <asm/msi_bitmap.h>
#include <asm/pnv-pci.h>
#include <asm/io.h>
+#include <asm/reg.h>
#include "cxl.h"
#include <misc/cxl.h>
@@ -321,12 +322,43 @@
#undef show_reg
}
+#define CAPP_UNIT0_ID 0xBA
+#define CAPP_UNIT1_ID 0XBE
+
+static u64 get_capp_unit_id(struct device_node *np)
+{
+ u32 phb_index;
+
+ /*
+ * For chips other than POWER8NVL, we only have CAPP 0,
+ * irrespective of which PHB is used.
+ */
+ if (!pvr_version_is(PVR_POWER8NVL))
+ return CAPP_UNIT0_ID;
+
+ /*
+ * For POWER8NVL, assume CAPP 0 is attached to PHB0 and
+ * CAPP 1 is attached to PHB1.
+ */
+ if (of_property_read_u32(np, "ibm,phb-index", &phb_index))
+ return 0;
+
+ if (phb_index == 0)
+ return CAPP_UNIT0_ID;
+
+ if (phb_index == 1)
+ return CAPP_UNIT1_ID;
+
+ return 0;
+}
+
static int init_implementation_adapter_regs(struct cxl *adapter, struct pci_dev *dev)
{
struct device_node *np;
const __be32 *prop;
u64 psl_dsnctl;
u64 chipid;
+ u64 capp_unit_id;
if (!(np = pnv_pci_get_phb_node(dev)))
return -ENODEV;
@@ -336,10 +368,19 @@
if (!np)
return -ENODEV;
chipid = be32_to_cpup(prop);
+ capp_unit_id = get_capp_unit_id(np);
of_node_put(np);
+ if (!capp_unit_id) {
+ pr_err("cxl: invalid capp unit id\n");
+ return -ENODEV;
+ }
+ psl_dsnctl = 0x0000900000000000ULL; /* pteupd ttype, scdone */
+ psl_dsnctl |= (0x2ULL << (63-38)); /* MMIO hang pulse: 256 us */
/* Tell PSL where to route data to */
- psl_dsnctl = 0x02E8900002000000ULL | (chipid << (63-5));
+ psl_dsnctl |= (chipid << (63-5));
+ psl_dsnctl |= (capp_unit_id << (63-13));
+
cxl_p1_write(adapter, CXL_PSL_DSNDCTL, psl_dsnctl);
cxl_p1_write(adapter, CXL_PSL_RESLCKTO, 0x20000000200ULL);
/* snoop write mask */
@@ -355,22 +396,24 @@
#define TBSYNC_CNT(n) (((u64)n & 0x7) << (63-6))
#define _2048_250MHZ_CYCLES 1
-static int cxl_setup_psl_timebase(struct cxl *adapter, struct pci_dev *dev)
+static void cxl_setup_psl_timebase(struct cxl *adapter, struct pci_dev *dev)
{
u64 psl_tb;
int delta;
unsigned int retry = 0;
struct device_node *np;
+ adapter->psl_timebase_synced = false;
+
if (!(np = pnv_pci_get_phb_node(dev)))
- return -ENODEV;
+ return;
/* Do not fail when CAPP timebase sync is not supported by OPAL */
of_node_get(np);
if (! of_get_property(np, "ibm,capp-timebase-sync", NULL)) {
of_node_put(np);
- pr_err("PSL: Timebase sync: OPAL support missing\n");
- return 0;
+ dev_info(&dev->dev, "PSL timebase inactive: OPAL support missing\n");
+ return;
}
of_node_put(np);
@@ -389,8 +432,8 @@
do {
msleep(1);
if (retry++ > 5) {
- pr_err("PSL: Timebase sync: giving up!\n");
- return -EIO;
+ dev_info(&dev->dev, "PSL timebase can't synchronize\n");
+ return;
}
psl_tb = cxl_p1_read(adapter, CXL_PSL_Timebase);
delta = mftb() - psl_tb;
@@ -398,7 +441,8 @@
delta = -delta;
} while (tb_to_ns(delta) > 16000);
- return 0;
+ adapter->psl_timebase_synced = true;
+ return;
}
static int init_implementation_afu_regs(struct cxl_afu *afu)
@@ -1144,8 +1188,8 @@
if ((rc = pnv_phb_to_cxl_mode(dev, OPAL_PHB_CAPI_MODE_SNOOP_ON)))
goto err;
- if ((rc = cxl_setup_psl_timebase(adapter, dev)))
- goto err;
+ /* Ignore error, adapter init is not dependant on timebase sync */
+ cxl_setup_psl_timebase(adapter, dev);
if ((rc = cxl_native_register_psl_err_irq(adapter)))
goto err;
diff --git a/drivers/misc/cxl/sysfs.c b/drivers/misc/cxl/sysfs.c
index 25913c0..b043c20 100644
--- a/drivers/misc/cxl/sysfs.c
+++ b/drivers/misc/cxl/sysfs.c
@@ -57,6 +57,15 @@
return scnprintf(buf, PAGE_SIZE, "factory\n");
}
+static ssize_t psl_timebase_synced_show(struct device *device,
+ struct device_attribute *attr,
+ char *buf)
+{
+ struct cxl *adapter = to_cxl_adapter(device);
+
+ return scnprintf(buf, PAGE_SIZE, "%i\n", adapter->psl_timebase_synced);
+}
+
static ssize_t reset_adapter_store(struct device *device,
struct device_attribute *attr,
const char *buf, size_t count)
@@ -142,6 +151,7 @@
__ATTR_RO(psl_revision),
__ATTR_RO(base_image),
__ATTR_RO(image_loaded),
+ __ATTR_RO(psl_timebase_synced),
__ATTR_RW(load_image_on_perst),
__ATTR_RW(perst_reloads_same_image),
__ATTR(reset, S_IWUSR, NULL, reset_adapter_store),
diff --git a/drivers/misc/eeprom/Kconfig b/drivers/misc/eeprom/Kconfig
index cfc493c..c4e41c2 100644
--- a/drivers/misc/eeprom/Kconfig
+++ b/drivers/misc/eeprom/Kconfig
@@ -3,7 +3,6 @@
config EEPROM_AT24
tristate "I2C EEPROMs / RAMs / ROMs from most vendors"
depends on I2C && SYSFS
- select REGMAP
select NVMEM
help
Enable this driver to get read/write support to most I2C EEPROMs
@@ -32,7 +31,6 @@
config EEPROM_AT25
tristate "SPI EEPROMs from most vendors"
depends on SPI && SYSFS
- select REGMAP
select NVMEM
help
Enable this driver to get read/write support to most SPI EEPROMs,
diff --git a/drivers/misc/eeprom/at24.c b/drivers/misc/eeprom/at24.c
index 089d694..9ceb63b 100644
--- a/drivers/misc/eeprom/at24.c
+++ b/drivers/misc/eeprom/at24.c
@@ -23,7 +23,6 @@
#include <linux/acpi.h>
#include <linux/i2c.h>
#include <linux/nvmem-provider.h>
-#include <linux/regmap.h>
#include <linux/platform_data/at24.h>
/*
@@ -69,7 +68,6 @@
unsigned write_max;
unsigned num_addresses;
- struct regmap_config regmap_config;
struct nvmem_config nvmem_config;
struct nvmem_device *nvmem;
@@ -245,17 +243,16 @@
if (status == count)
return count;
- /* REVISIT: at HZ=100, this is sloooow */
- msleep(1);
+ usleep_range(1000, 1500);
} while (time_before(read_time, timeout));
return -ETIMEDOUT;
}
-static ssize_t at24_read(struct at24_data *at24,
- char *buf, loff_t off, size_t count)
+static int at24_read(void *priv, unsigned int off, void *val, size_t count)
{
- ssize_t retval = 0;
+ struct at24_data *at24 = priv;
+ char *buf = val;
if (unlikely(!count))
return count;
@@ -267,23 +264,21 @@
mutex_lock(&at24->lock);
while (count) {
- ssize_t status;
+ int status;
status = at24_eeprom_read(at24, buf, off, count);
- if (status <= 0) {
- if (retval == 0)
- retval = status;
- break;
+ if (status < 0) {
+ mutex_unlock(&at24->lock);
+ return status;
}
buf += status;
off += status;
count -= status;
- retval += status;
}
mutex_unlock(&at24->lock);
- return retval;
+ return 0;
}
/*
@@ -365,20 +360,19 @@
if (status == count)
return count;
- /* REVISIT: at HZ=100, this is sloooow */
- msleep(1);
+ usleep_range(1000, 1500);
} while (time_before(write_time, timeout));
return -ETIMEDOUT;
}
-static ssize_t at24_write(struct at24_data *at24, const char *buf, loff_t off,
- size_t count)
+static int at24_write(void *priv, unsigned int off, void *val, size_t count)
{
- ssize_t retval = 0;
+ struct at24_data *at24 = priv;
+ char *buf = val;
if (unlikely(!count))
- return count;
+ return -EINVAL;
/*
* Write data to chip, protecting against concurrent updates
@@ -387,70 +381,23 @@
mutex_lock(&at24->lock);
while (count) {
- ssize_t status;
+ int status;
status = at24_eeprom_write(at24, buf, off, count);
- if (status <= 0) {
- if (retval == 0)
- retval = status;
- break;
+ if (status < 0) {
+ mutex_unlock(&at24->lock);
+ return status;
}
buf += status;
off += status;
count -= status;
- retval += status;
}
mutex_unlock(&at24->lock);
- return retval;
-}
-
-/*-------------------------------------------------------------------------*/
-
-/*
- * Provide a regmap interface, which is registered with the NVMEM
- * framework
-*/
-static int at24_regmap_read(void *context, const void *reg, size_t reg_size,
- void *val, size_t val_size)
-{
- struct at24_data *at24 = context;
- off_t offset = *(u32 *)reg;
- int err;
-
- err = at24_read(at24, val, offset, val_size);
- if (err)
- return err;
return 0;
}
-static int at24_regmap_write(void *context, const void *data, size_t count)
-{
- struct at24_data *at24 = context;
- const char *buf;
- u32 offset;
- size_t len;
- int err;
-
- memcpy(&offset, data, sizeof(offset));
- buf = (const char *)data + sizeof(offset);
- len = count - sizeof(offset);
-
- err = at24_write(at24, buf, offset, len);
- if (err)
- return err;
- return 0;
-}
-
-static const struct regmap_bus at24_regmap_bus = {
- .read = at24_regmap_read,
- .write = at24_regmap_write,
- .reg_format_endian_default = REGMAP_ENDIAN_NATIVE,
-};
-
-/*-------------------------------------------------------------------------*/
-
#ifdef CONFIG_OF
static void at24_get_ofdata(struct i2c_client *client,
struct at24_platform_data *chip)
@@ -482,7 +429,6 @@
struct at24_data *at24;
int err;
unsigned i, num_addresses;
- struct regmap *regmap;
if (client->dev.platform_data) {
chip = *(struct at24_platform_data *)client->dev.platform_data;
@@ -544,10 +490,7 @@
} else {
return -EPFNOSUPPORT;
}
- }
- /* Use I2C operations unless we're stuck with SMBus extensions. */
- if (!i2c_check_functionality(client->adapter, I2C_FUNC_I2C)) {
if (i2c_check_functionality(client->adapter,
I2C_FUNC_SMBUS_WRITE_I2C_BLOCK)) {
use_smbus_write = I2C_SMBUS_I2C_BLOCK_DATA;
@@ -612,19 +555,6 @@
}
}
- at24->regmap_config.reg_bits = 32;
- at24->regmap_config.val_bits = 8;
- at24->regmap_config.reg_stride = 1;
- at24->regmap_config.max_register = chip.byte_len - 1;
-
- regmap = devm_regmap_init(&client->dev, &at24_regmap_bus, at24,
- &at24->regmap_config);
- if (IS_ERR(regmap)) {
- dev_err(&client->dev, "regmap init failed\n");
- err = PTR_ERR(regmap);
- goto err_clients;
- }
-
at24->nvmem_config.name = dev_name(&client->dev);
at24->nvmem_config.dev = &client->dev;
at24->nvmem_config.read_only = !writable;
@@ -632,6 +562,12 @@
at24->nvmem_config.owner = THIS_MODULE;
at24->nvmem_config.compat = true;
at24->nvmem_config.base_dev = &client->dev;
+ at24->nvmem_config.reg_read = at24_read;
+ at24->nvmem_config.reg_write = at24_write;
+ at24->nvmem_config.priv = at24;
+ at24->nvmem_config.stride = 4;
+ at24->nvmem_config.word_size = 1;
+ at24->nvmem_config.size = chip.byte_len;
at24->nvmem = nvmem_register(&at24->nvmem_config);
diff --git a/drivers/misc/eeprom/at25.c b/drivers/misc/eeprom/at25.c
index fa36a6e..2c6c7c8 100644
--- a/drivers/misc/eeprom/at25.c
+++ b/drivers/misc/eeprom/at25.c
@@ -17,7 +17,6 @@
#include <linux/sched.h>
#include <linux/nvmem-provider.h>
-#include <linux/regmap.h>
#include <linux/spi/spi.h>
#include <linux/spi/eeprom.h>
#include <linux/property.h>
@@ -34,7 +33,6 @@
struct mutex lock;
struct spi_eeprom chip;
unsigned addrlen;
- struct regmap_config regmap_config;
struct nvmem_config nvmem_config;
struct nvmem_device *nvmem;
};
@@ -65,14 +63,11 @@
#define io_limit PAGE_SIZE /* bytes */
-static ssize_t
-at25_ee_read(
- struct at25_data *at25,
- char *buf,
- unsigned offset,
- size_t count
-)
+static int at25_ee_read(void *priv, unsigned int offset,
+ void *val, size_t count)
{
+ struct at25_data *at25 = priv;
+ char *buf = val;
u8 command[EE_MAXADDRLEN + 1];
u8 *cp;
ssize_t status;
@@ -81,11 +76,11 @@
u8 instr;
if (unlikely(offset >= at25->chip.byte_len))
- return 0;
+ return -EINVAL;
if ((offset + count) > at25->chip.byte_len)
count = at25->chip.byte_len - offset;
if (unlikely(!count))
- return count;
+ return -EINVAL;
cp = command;
@@ -131,28 +126,14 @@
count, offset, (int) status);
mutex_unlock(&at25->lock);
- return status ? status : count;
+ return status;
}
-static int at25_regmap_read(void *context, const void *reg, size_t reg_size,
- void *val, size_t val_size)
+static int at25_ee_write(void *priv, unsigned int off, void *val, size_t count)
{
- struct at25_data *at25 = context;
- off_t offset = *(u32 *)reg;
- int err;
-
- err = at25_ee_read(at25, val, offset, val_size);
- if (err)
- return err;
- return 0;
-}
-
-static ssize_t
-at25_ee_write(struct at25_data *at25, const char *buf, loff_t off,
- size_t count)
-{
- ssize_t status = 0;
- unsigned written = 0;
+ struct at25_data *at25 = priv;
+ const char *buf = val;
+ int status = 0;
unsigned buf_size;
u8 *bounce;
@@ -161,7 +142,7 @@
if ((off + count) > at25->chip.byte_len)
count = at25->chip.byte_len - off;
if (unlikely(!count))
- return count;
+ return -EINVAL;
/* Temp buffer starts with command and address */
buf_size = at25->chip.page_size;
@@ -256,40 +237,15 @@
off += segment;
buf += segment;
count -= segment;
- written += segment;
} while (count > 0);
mutex_unlock(&at25->lock);
kfree(bounce);
- return written ? written : status;
+ return status;
}
-static int at25_regmap_write(void *context, const void *data, size_t count)
-{
- struct at25_data *at25 = context;
- const char *buf;
- u32 offset;
- size_t len;
- int err;
-
- memcpy(&offset, data, sizeof(offset));
- buf = (const char *)data + sizeof(offset);
- len = count - sizeof(offset);
-
- err = at25_ee_write(at25, buf, offset, len);
- if (err)
- return err;
- return 0;
-}
-
-static const struct regmap_bus at25_regmap_bus = {
- .read = at25_regmap_read,
- .write = at25_regmap_write,
- .reg_format_endian_default = REGMAP_ENDIAN_NATIVE,
-};
-
/*-------------------------------------------------------------------------*/
static int at25_fw_to_chip(struct device *dev, struct spi_eeprom *chip)
@@ -349,7 +305,6 @@
{
struct at25_data *at25 = NULL;
struct spi_eeprom chip;
- struct regmap *regmap;
int err;
int sr;
int addrlen;
@@ -390,22 +345,10 @@
mutex_init(&at25->lock);
at25->chip = chip;
- at25->spi = spi_dev_get(spi);
+ at25->spi = spi;
spi_set_drvdata(spi, at25);
at25->addrlen = addrlen;
- at25->regmap_config.reg_bits = 32;
- at25->regmap_config.val_bits = 8;
- at25->regmap_config.reg_stride = 1;
- at25->regmap_config.max_register = chip.byte_len - 1;
-
- regmap = devm_regmap_init(&spi->dev, &at25_regmap_bus, at25,
- &at25->regmap_config);
- if (IS_ERR(regmap)) {
- dev_err(&spi->dev, "regmap init failed\n");
- return PTR_ERR(regmap);
- }
-
at25->nvmem_config.name = dev_name(&spi->dev);
at25->nvmem_config.dev = &spi->dev;
at25->nvmem_config.read_only = chip.flags & EE_READONLY;
@@ -413,6 +356,12 @@
at25->nvmem_config.owner = THIS_MODULE;
at25->nvmem_config.compat = true;
at25->nvmem_config.base_dev = &spi->dev;
+ at25->nvmem_config.reg_read = at25_ee_read;
+ at25->nvmem_config.reg_write = at25_ee_write;
+ at25->nvmem_config.priv = at25;
+ at25->nvmem_config.stride = 4;
+ at25->nvmem_config.word_size = 1;
+ at25->nvmem_config.size = chip.byte_len;
at25->nvmem = nvmem_register(&at25->nvmem_config);
if (IS_ERR(at25->nvmem))
diff --git a/drivers/misc/eeprom/eeprom_93xx46.c b/drivers/misc/eeprom/eeprom_93xx46.c
index 426fe2f..94cc035 100644
--- a/drivers/misc/eeprom/eeprom_93xx46.c
+++ b/drivers/misc/eeprom/eeprom_93xx46.c
@@ -20,7 +20,6 @@
#include <linux/slab.h>
#include <linux/spi/spi.h>
#include <linux/nvmem-provider.h>
-#include <linux/regmap.h>
#include <linux/eeprom_93xx46.h>
#define OP_START 0x4
@@ -43,7 +42,6 @@
struct spi_device *spi;
struct eeprom_93xx46_platform_data *pdata;
struct mutex lock;
- struct regmap_config regmap_config;
struct nvmem_config nvmem_config;
struct nvmem_device *nvmem;
int addrlen;
@@ -60,11 +58,12 @@
return edev->pdata->quirks & EEPROM_93XX46_QUIRK_INSTRUCTION_LENGTH;
}
-static ssize_t
-eeprom_93xx46_read(struct eeprom_93xx46_dev *edev, char *buf,
- unsigned off, size_t count)
+static int eeprom_93xx46_read(void *priv, unsigned int off,
+ void *val, size_t count)
{
- ssize_t ret = 0;
+ struct eeprom_93xx46_dev *edev = priv;
+ char *buf = val;
+ int err = 0;
if (unlikely(off >= edev->size))
return 0;
@@ -84,7 +83,6 @@
u16 cmd_addr = OP_READ << edev->addrlen;
size_t nbytes = count;
int bits;
- int err;
if (edev->addrlen == 7) {
cmd_addr |= off & 0x7f;
@@ -120,21 +118,20 @@
if (err) {
dev_err(&edev->spi->dev, "read %zu bytes at %d: err. %d\n",
nbytes, (int)off, err);
- ret = err;
break;
}
buf += nbytes;
off += nbytes;
count -= nbytes;
- ret += nbytes;
}
if (edev->pdata->finish)
edev->pdata->finish(edev);
mutex_unlock(&edev->lock);
- return ret;
+
+ return err;
}
static int eeprom_93xx46_ew(struct eeprom_93xx46_dev *edev, int is_on)
@@ -230,10 +227,11 @@
return ret;
}
-static ssize_t
-eeprom_93xx46_write(struct eeprom_93xx46_dev *edev, const char *buf,
- loff_t off, size_t count)
+static int eeprom_93xx46_write(void *priv, unsigned int off,
+ void *val, size_t count)
{
+ struct eeprom_93xx46_dev *edev = priv;
+ char *buf = val;
int i, ret, step = 1;
if (unlikely(off >= edev->size))
@@ -275,52 +273,9 @@
/* erase/write disable */
eeprom_93xx46_ew(edev, 0);
- return ret ? : count;
+ return ret;
}
-/*
- * Provide a regmap interface, which is registered with the NVMEM
- * framework
-*/
-static int eeprom_93xx46_regmap_read(void *context, const void *reg,
- size_t reg_size, void *val,
- size_t val_size)
-{
- struct eeprom_93xx46_dev *eeprom_93xx46 = context;
- off_t offset = *(u32 *)reg;
- int err;
-
- err = eeprom_93xx46_read(eeprom_93xx46, val, offset, val_size);
- if (err)
- return err;
- return 0;
-}
-
-static int eeprom_93xx46_regmap_write(void *context, const void *data,
- size_t count)
-{
- struct eeprom_93xx46_dev *eeprom_93xx46 = context;
- const char *buf;
- u32 offset;
- size_t len;
- int err;
-
- memcpy(&offset, data, sizeof(offset));
- buf = (const char *)data + sizeof(offset);
- len = count - sizeof(offset);
-
- err = eeprom_93xx46_write(eeprom_93xx46, buf, offset, len);
- if (err)
- return err;
- return 0;
-}
-
-static const struct regmap_bus eeprom_93xx46_regmap_bus = {
- .read = eeprom_93xx46_regmap_read,
- .write = eeprom_93xx46_regmap_write,
- .reg_format_endian_default = REGMAP_ENDIAN_NATIVE,
-};
-
static int eeprom_93xx46_eral(struct eeprom_93xx46_dev *edev)
{
struct eeprom_93xx46_platform_data *pd = edev->pdata;
@@ -480,7 +435,6 @@
{
struct eeprom_93xx46_platform_data *pd;
struct eeprom_93xx46_dev *edev;
- struct regmap *regmap;
int err;
if (spi->dev.of_node) {
@@ -511,24 +465,10 @@
mutex_init(&edev->lock);
- edev->spi = spi_dev_get(spi);
+ edev->spi = spi;
edev->pdata = pd;
edev->size = 128;
-
- edev->regmap_config.reg_bits = 32;
- edev->regmap_config.val_bits = 8;
- edev->regmap_config.reg_stride = 1;
- edev->regmap_config.max_register = edev->size - 1;
-
- regmap = devm_regmap_init(&spi->dev, &eeprom_93xx46_regmap_bus, edev,
- &edev->regmap_config);
- if (IS_ERR(regmap)) {
- dev_err(&spi->dev, "regmap init failed\n");
- err = PTR_ERR(regmap);
- goto fail;
- }
-
edev->nvmem_config.name = dev_name(&spi->dev);
edev->nvmem_config.dev = &spi->dev;
edev->nvmem_config.read_only = pd->flags & EE_READONLY;
@@ -536,6 +476,12 @@
edev->nvmem_config.owner = THIS_MODULE;
edev->nvmem_config.compat = true;
edev->nvmem_config.base_dev = &spi->dev;
+ edev->nvmem_config.reg_read = eeprom_93xx46_read;
+ edev->nvmem_config.reg_write = eeprom_93xx46_write;
+ edev->nvmem_config.priv = edev;
+ edev->nvmem_config.stride = 4;
+ edev->nvmem_config.word_size = 1;
+ edev->nvmem_config.size = edev->size;
edev->nvmem = nvmem_register(&edev->nvmem_config);
if (IS_ERR(edev->nvmem)) {
diff --git a/drivers/misc/mei/amthif.c b/drivers/misc/mei/amthif.c
index 194360a..a039a5d 100644
--- a/drivers/misc/mei/amthif.c
+++ b/drivers/misc/mei/amthif.c
@@ -380,8 +380,10 @@
dev = cl->dev;
- if (dev->iamthif_state != MEI_IAMTHIF_READING)
+ if (dev->iamthif_state != MEI_IAMTHIF_READING) {
+ mei_irq_discard_msg(dev, mei_hdr);
return 0;
+ }
ret = mei_cl_irq_read_msg(cl, mei_hdr, cmpl_list);
if (ret)
diff --git a/drivers/misc/mei/bus.c b/drivers/misc/mei/bus.c
index 5d5996e..1f33fea 100644
--- a/drivers/misc/mei/bus.c
+++ b/drivers/misc/mei/bus.c
@@ -220,17 +220,23 @@
static void mei_cl_bus_event_work(struct work_struct *work)
{
struct mei_cl_device *cldev;
+ struct mei_device *bus;
cldev = container_of(work, struct mei_cl_device, event_work);
+ bus = cldev->bus;
+
if (cldev->event_cb)
cldev->event_cb(cldev, cldev->events, cldev->event_context);
cldev->events = 0;
/* Prepare for the next read */
- if (cldev->events_mask & BIT(MEI_CL_EVENT_RX))
+ if (cldev->events_mask & BIT(MEI_CL_EVENT_RX)) {
+ mutex_lock(&bus->device_lock);
mei_cl_read_start(cldev->cl, 0, NULL);
+ mutex_unlock(&bus->device_lock);
+ }
}
/**
@@ -304,6 +310,7 @@
unsigned long events_mask,
mei_cldev_event_cb_t event_cb, void *context)
{
+ struct mei_device *bus = cldev->bus;
int ret;
if (cldev->event_cb)
@@ -316,15 +323,17 @@
INIT_WORK(&cldev->event_work, mei_cl_bus_event_work);
if (cldev->events_mask & BIT(MEI_CL_EVENT_RX)) {
+ mutex_lock(&bus->device_lock);
ret = mei_cl_read_start(cldev->cl, 0, NULL);
+ mutex_unlock(&bus->device_lock);
if (ret && ret != -EBUSY)
return ret;
}
if (cldev->events_mask & BIT(MEI_CL_EVENT_NOTIF)) {
- mutex_lock(&cldev->cl->dev->device_lock);
+ mutex_lock(&bus->device_lock);
ret = mei_cl_notify_request(cldev->cl, NULL, event_cb ? 1 : 0);
- mutex_unlock(&cldev->cl->dev->device_lock);
+ mutex_unlock(&bus->device_lock);
if (ret)
return ret;
}
@@ -580,6 +589,7 @@
struct mei_cl_device *cldev;
struct mei_cl_driver *cldrv;
const struct mei_cl_device_id *id;
+ int ret;
cldev = to_mei_cl_device(dev);
cldrv = to_mei_cl_driver(dev->driver);
@@ -594,9 +604,12 @@
if (!id)
return -ENODEV;
- __module_get(THIS_MODULE);
+ ret = cldrv->probe(cldev, id);
+ if (ret)
+ return ret;
- return cldrv->probe(cldev, id);
+ __module_get(THIS_MODULE);
+ return 0;
}
/**
@@ -634,11 +647,8 @@
char *buf)
{
struct mei_cl_device *cldev = to_mei_cl_device(dev);
- size_t len;
- len = snprintf(buf, PAGE_SIZE, "%s", cldev->name);
-
- return (len >= PAGE_SIZE) ? (PAGE_SIZE - 1) : len;
+ return scnprintf(buf, PAGE_SIZE, "%s", cldev->name);
}
static DEVICE_ATTR_RO(name);
@@ -647,11 +657,8 @@
{
struct mei_cl_device *cldev = to_mei_cl_device(dev);
const uuid_le *uuid = mei_me_cl_uuid(cldev->me_cl);
- size_t len;
- len = snprintf(buf, PAGE_SIZE, "%pUl", uuid);
-
- return (len >= PAGE_SIZE) ? (PAGE_SIZE - 1) : len;
+ return scnprintf(buf, PAGE_SIZE, "%pUl", uuid);
}
static DEVICE_ATTR_RO(uuid);
@@ -660,11 +667,8 @@
{
struct mei_cl_device *cldev = to_mei_cl_device(dev);
u8 version = mei_me_cl_ver(cldev->me_cl);
- size_t len;
- len = snprintf(buf, PAGE_SIZE, "%02X", version);
-
- return (len >= PAGE_SIZE) ? (PAGE_SIZE - 1) : len;
+ return scnprintf(buf, PAGE_SIZE, "%02X", version);
}
static DEVICE_ATTR_RO(version);
@@ -673,10 +677,8 @@
{
struct mei_cl_device *cldev = to_mei_cl_device(dev);
const uuid_le *uuid = mei_me_cl_uuid(cldev->me_cl);
- size_t len;
- len = snprintf(buf, PAGE_SIZE, "mei:%s:%pUl:", cldev->name, uuid);
- return (len >= PAGE_SIZE) ? (PAGE_SIZE - 1) : len;
+ return scnprintf(buf, PAGE_SIZE, "mei:%s:%pUl:", cldev->name, uuid);
}
static DEVICE_ATTR_RO(modalias);
diff --git a/drivers/misc/mei/client.c b/drivers/misc/mei/client.c
index bab17e4..eed254d 100644
--- a/drivers/misc/mei/client.c
+++ b/drivers/misc/mei/client.c
@@ -727,6 +727,11 @@
cl_dbg(dev, cl, "Waking up waiting for event clients!\n");
wake_up_interruptible(&cl->ev_wait);
}
+ /* synchronized under device mutex */
+ if (waitqueue_active(&cl->wait)) {
+ cl_dbg(dev, cl, "Waking up ctrl write clients!\n");
+ wake_up_interruptible(&cl->wait);
+ }
}
/**
@@ -879,12 +884,15 @@
}
mutex_unlock(&dev->device_lock);
- wait_event_timeout(cl->wait, cl->state == MEI_FILE_DISCONNECT_REPLY,
+ wait_event_timeout(cl->wait,
+ cl->state == MEI_FILE_DISCONNECT_REPLY ||
+ cl->state == MEI_FILE_DISCONNECTED,
mei_secs_to_jiffies(MEI_CL_CONNECT_TIMEOUT));
mutex_lock(&dev->device_lock);
rets = cl->status;
- if (cl->state != MEI_FILE_DISCONNECT_REPLY) {
+ if (cl->state != MEI_FILE_DISCONNECT_REPLY &&
+ cl->state != MEI_FILE_DISCONNECTED) {
cl_dbg(dev, cl, "timeout on disconnect from FW client.\n");
rets = -ETIME;
}
@@ -1085,6 +1093,7 @@
mutex_unlock(&dev->device_lock);
wait_event_timeout(cl->wait,
(cl->state == MEI_FILE_CONNECTED ||
+ cl->state == MEI_FILE_DISCONNECTED ||
cl->state == MEI_FILE_DISCONNECT_REQUIRED ||
cl->state == MEI_FILE_DISCONNECT_REPLY),
mei_secs_to_jiffies(MEI_CL_CONNECT_TIMEOUT));
@@ -1333,16 +1342,13 @@
}
mutex_unlock(&dev->device_lock);
- wait_event_timeout(cl->wait, cl->notify_en == request,
- mei_secs_to_jiffies(MEI_CL_CONNECT_TIMEOUT));
+ wait_event_timeout(cl->wait,
+ cl->notify_en == request || !mei_cl_is_connected(cl),
+ mei_secs_to_jiffies(MEI_CL_CONNECT_TIMEOUT));
mutex_lock(&dev->device_lock);
- if (cl->notify_en != request) {
- mei_io_list_flush(&dev->ctrl_rd_list, cl);
- mei_io_list_flush(&dev->ctrl_wr_list, cl);
- if (!cl->status)
- cl->status = -EFAULT;
- }
+ if (cl->notify_en != request && !cl->status)
+ cl->status = -EFAULT;
rets = cl->status;
@@ -1767,6 +1773,10 @@
wake_up(&cl->wait);
break;
+ case MEI_FOP_DISCONNECT_RSP:
+ mei_io_cb_free(cb);
+ mei_cl_set_disconnected(cl);
+ break;
default:
BUG_ON(0);
}
diff --git a/drivers/misc/mei/hbm.c b/drivers/misc/mei/hbm.c
index 5e305d2..5aa606c 100644
--- a/drivers/misc/mei/hbm.c
+++ b/drivers/misc/mei/hbm.c
@@ -113,8 +113,6 @@
*/
void mei_hbm_reset(struct mei_device *dev)
{
- dev->me_client_index = 0;
-
mei_me_cl_rm_all(dev);
mei_hbm_idle(dev);
@@ -530,24 +528,22 @@
* mei_hbm_prop_req - request property for a single client
*
* @dev: the device structure
+ * @start_idx: client index to start search
*
* Return: 0 on success and < 0 on failure
*/
-
-static int mei_hbm_prop_req(struct mei_device *dev)
+static int mei_hbm_prop_req(struct mei_device *dev, unsigned long start_idx)
{
-
struct mei_msg_hdr *mei_hdr = &dev->wr_msg.hdr;
struct hbm_props_request *prop_req;
const size_t len = sizeof(struct hbm_props_request);
- unsigned long next_client_index;
+ unsigned long addr;
int ret;
- next_client_index = find_next_bit(dev->me_clients_map, MEI_CLIENTS_MAX,
- dev->me_client_index);
+ addr = find_next_bit(dev->me_clients_map, MEI_CLIENTS_MAX, start_idx);
/* We got all client properties */
- if (next_client_index == MEI_CLIENTS_MAX) {
+ if (addr == MEI_CLIENTS_MAX) {
dev->hbm_state = MEI_HBM_STARTED;
mei_host_client_init(dev);
@@ -560,7 +556,7 @@
memset(prop_req, 0, sizeof(struct hbm_props_request));
prop_req->hbm_cmd = HOST_CLIENT_PROPERTIES_REQ_CMD;
- prop_req->me_addr = next_client_index;
+ prop_req->me_addr = addr;
ret = mei_write_message(dev, mei_hdr, dev->wr_msg.data);
if (ret) {
@@ -570,7 +566,6 @@
}
dev->init_clients_timer = MEI_CLIENTS_INIT_TIMEOUT;
- dev->me_client_index = next_client_index;
return 0;
}
@@ -882,8 +877,7 @@
cb = mei_io_cb_init(cl, MEI_FOP_DISCONNECT_RSP, NULL);
if (!cb)
return -ENOMEM;
- cl_dbg(dev, cl, "add disconnect response as first\n");
- list_add(&cb->list, &dev->ctrl_wr_list.list);
+ list_add_tail(&cb->list, &dev->ctrl_wr_list.list);
}
return 0;
}
@@ -1152,10 +1146,8 @@
mei_hbm_me_cl_add(dev, props_res);
- dev->me_client_index++;
-
/* request property for the next client */
- if (mei_hbm_prop_req(dev))
+ if (mei_hbm_prop_req(dev, props_res->me_addr + 1))
return -EIO;
break;
@@ -1181,7 +1173,7 @@
dev->hbm_state = MEI_HBM_CLIENT_PROPERTIES;
/* first property request */
- if (mei_hbm_prop_req(dev))
+ if (mei_hbm_prop_req(dev, 0))
return -EIO;
break;
diff --git a/drivers/misc/mei/interrupt.c b/drivers/misc/mei/interrupt.c
index 1e5cb1f..3831a7b 100644
--- a/drivers/misc/mei/interrupt.c
+++ b/drivers/misc/mei/interrupt.c
@@ -76,7 +76,6 @@
* @dev: mei device
* @hdr: message header
*/
-static inline
void mei_irq_discard_msg(struct mei_device *dev, struct mei_msg_hdr *hdr)
{
/*
@@ -194,10 +193,7 @@
return -EMSGSIZE;
ret = mei_hbm_cl_disconnect_rsp(dev, cl);
- mei_cl_set_disconnected(cl);
- mei_io_cb_free(cb);
- mei_me_cl_put(cl->me_cl);
- cl->me_cl = NULL;
+ list_move_tail(&cb->list, &cmpl_list->list);
return ret;
}
diff --git a/drivers/misc/mei/mei_dev.h b/drivers/misc/mei/mei_dev.h
index db78e6d..c9e0102 100644
--- a/drivers/misc/mei/mei_dev.h
+++ b/drivers/misc/mei/mei_dev.h
@@ -396,7 +396,6 @@
* @me_clients : list of FW clients
* @me_clients_map : FW clients bit map
* @host_clients_map : host clients id pool
- * @me_client_index : last FW client index in enumeration
*
* @allow_fixed_address: allow user space to connect a fixed client
* @override_fixed_address: force allow fixed address behavior
@@ -486,7 +485,6 @@
struct list_head me_clients;
DECLARE_BITMAP(me_clients_map, MEI_CLIENTS_MAX);
DECLARE_BITMAP(host_clients_map, MEI_CLIENTS_MAX);
- unsigned long me_client_index;
bool allow_fixed_address;
bool override_fixed_address;
@@ -704,6 +702,8 @@
bool mei_write_is_idle(struct mei_device *dev);
+void mei_irq_discard_msg(struct mei_device *dev, struct mei_msg_hdr *hdr);
+
#if IS_ENABLED(CONFIG_DEBUG_FS)
int mei_dbgfs_register(struct mei_device *dev, const char *name);
void mei_dbgfs_deregister(struct mei_device *dev);
diff --git a/drivers/misc/mic/Kconfig b/drivers/misc/mic/Kconfig
index 2e4f3ba..89e5917 100644
--- a/drivers/misc/mic/Kconfig
+++ b/drivers/misc/mic/Kconfig
@@ -132,6 +132,7 @@
tristate "VOP Driver"
depends on 64BIT && PCI && X86 && VOP_BUS
select VHOST_RING
+ select VIRTIO
help
This enables VOP (Virtio over PCIe) Driver support for the Intel
Many Integrated Core (MIC) family of PCIe form factor coprocessor
diff --git a/drivers/misc/mic/host/mic_boot.c b/drivers/misc/mic/host/mic_boot.c
index 8c91c99..e047efd 100644
--- a/drivers/misc/mic/host/mic_boot.c
+++ b/drivers/misc/mic/host/mic_boot.c
@@ -76,7 +76,7 @@
{
struct mic_device *mdev = vpdev_to_mdev(&vpdev->dev);
- return mic_free_irq(mdev, cookie, data);
+ mic_free_irq(mdev, cookie, data);
}
static void __mic_ack_interrupt(struct vop_device *vpdev, int num)
@@ -272,7 +272,7 @@
{
struct mic_device *mdev = scdev_to_mdev(scdev);
- return mic_free_irq(mdev, cookie, data);
+ mic_free_irq(mdev, cookie, data);
}
static void ___mic_ack_interrupt(struct scif_hw_dev *scdev, int num)
@@ -362,7 +362,7 @@
static void _mic_free_irq(struct mbus_device *mbdev,
struct mic_irq *cookie, void *data)
{
- return mic_free_irq(mbdev_to_mdev(mbdev), cookie, data);
+ mic_free_irq(mbdev_to_mdev(mbdev), cookie, data);
}
static void _mic_ack_interrupt(struct mbus_device *mbdev, int num)
diff --git a/drivers/misc/mic/scif/scif_fence.c b/drivers/misc/mic/scif/scif_fence.c
index 7f2c96f..cac3bcc 100644
--- a/drivers/misc/mic/scif/scif_fence.c
+++ b/drivers/misc/mic/scif/scif_fence.c
@@ -27,7 +27,8 @@
void scif_recv_mark(struct scif_dev *scifdev, struct scifmsg *msg)
{
struct scif_endpt *ep = (struct scif_endpt *)msg->payload[0];
- int mark, err;
+ int mark = 0;
+ int err;
err = _scif_fence_mark(ep, &mark);
if (err)
diff --git a/drivers/misc/qcom-coincell.c b/drivers/misc/qcom-coincell.c
index 7b4a2da..829a61d 100644
--- a/drivers/misc/qcom-coincell.c
+++ b/drivers/misc/qcom-coincell.c
@@ -94,7 +94,8 @@
{
struct device_node *node = pdev->dev.of_node;
struct qcom_coincell chgr;
- u32 rset, vset;
+ u32 rset = 0;
+ u32 vset = 0;
bool enable;
int rc;
diff --git a/drivers/misc/sram.c b/drivers/misc/sram.c
index 69cdabe..f84b53d 100644
--- a/drivers/misc/sram.c
+++ b/drivers/misc/sram.c
@@ -364,8 +364,8 @@
sram->virt_base = devm_ioremap(sram->dev, res->start, size);
else
sram->virt_base = devm_ioremap_wc(sram->dev, res->start, size);
- if (IS_ERR(sram->virt_base))
- return PTR_ERR(sram->virt_base);
+ if (!sram->virt_base)
+ return -ENOMEM;
sram->pool = devm_gen_pool_create(sram->dev, ilog2(SRAM_GRANULARITY),
NUMA_NO_NODE, NULL);
diff --git a/drivers/misc/ti-st/st_kim.c b/drivers/misc/ti-st/st_kim.c
index 71b6455..bf0d770 100644
--- a/drivers/misc/ti-st/st_kim.c
+++ b/drivers/misc/ti-st/st_kim.c
@@ -78,7 +78,6 @@
memcpy(kim_gdata->resp_buffer,
kim_gdata->rx_skb->data,
kim_gdata->rx_skb->len);
- complete_all(&kim_gdata->kim_rcvd);
kim_gdata->rx_state = ST_W4_PACKET_TYPE;
kim_gdata->rx_skb = NULL;
kim_gdata->rx_count = 0;
diff --git a/drivers/mmc/card/sdio_uart.c b/drivers/mmc/card/sdio_uart.c
index 5415056..5af6fb9 100644
--- a/drivers/mmc/card/sdio_uart.c
+++ b/drivers/mmc/card/sdio_uart.c
@@ -895,7 +895,7 @@
/* Handle transition away from B0 status */
if (!(old_termios->c_cflag & CBAUD) && (cflag & CBAUD)) {
unsigned int mask = TIOCM_DTR;
- if (!(cflag & CRTSCTS) || !test_bit(TTY_THROTTLED, &tty->flags))
+ if (!(cflag & CRTSCTS) || !tty_throttled(tty))
mask |= TIOCM_RTS;
sdio_uart_set_mctrl(port, mask);
}
diff --git a/drivers/mtd/chips/Kconfig b/drivers/mtd/chips/Kconfig
index 3b3dabc..bbfa1f1 100644
--- a/drivers/mtd/chips/Kconfig
+++ b/drivers/mtd/chips/Kconfig
@@ -115,6 +115,7 @@
config MTD_MAP_BANK_WIDTH_32
bool "Support 256-bit buswidth" if MTD_CFI_GEOMETRY
+ select MTD_COMPLEX_MAPPINGS if HAS_IOMEM
default n
help
If you wish to support CFI devices on a physical bus which is
diff --git a/drivers/mtd/devices/bcm47xxsflash.c b/drivers/mtd/devices/bcm47xxsflash.c
index 347bb83..1c65c15 100644
--- a/drivers/mtd/devices/bcm47xxsflash.c
+++ b/drivers/mtd/devices/bcm47xxsflash.c
@@ -2,6 +2,7 @@
#include <linux/module.h>
#include <linux/slab.h>
#include <linux/delay.h>
+#include <linux/ioport.h>
#include <linux/mtd/mtd.h>
#include <linux/platform_device.h>
#include <linux/bcma/bcma.h>
@@ -109,8 +110,7 @@
if ((from + len) > mtd->size)
return -EINVAL;
- memcpy_fromio(buf, (void __iomem *)KSEG0ADDR(b47s->window + from),
- len);
+ memcpy_fromio(buf, b47s->window + from, len);
*retlen = len;
return len;
@@ -275,15 +275,33 @@
static int bcm47xxsflash_bcma_probe(struct platform_device *pdev)
{
- struct bcma_sflash *sflash = dev_get_platdata(&pdev->dev);
+ struct device *dev = &pdev->dev;
+ struct bcma_sflash *sflash = dev_get_platdata(dev);
struct bcm47xxsflash *b47s;
+ struct resource *res;
int err;
- b47s = devm_kzalloc(&pdev->dev, sizeof(*b47s), GFP_KERNEL);
+ b47s = devm_kzalloc(dev, sizeof(*b47s), GFP_KERNEL);
if (!b47s)
return -ENOMEM;
sflash->priv = b47s;
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (!res) {
+ dev_err(dev, "invalid resource\n");
+ return -EINVAL;
+ }
+ if (!devm_request_mem_region(dev, res->start, resource_size(res),
+ res->name)) {
+ dev_err(dev, "can't request region for resource %pR\n", res);
+ return -EBUSY;
+ }
+ b47s->window = ioremap_cache(res->start, resource_size(res));
+ if (!b47s->window) {
+ dev_err(dev, "ioremap failed for resource %pR\n", res);
+ return -ENOMEM;
+ }
+
b47s->bcma_cc = container_of(sflash, struct bcma_drv_cc, sflash);
b47s->cc_read = bcm47xxsflash_bcma_cc_read;
b47s->cc_write = bcm47xxsflash_bcma_cc_write;
@@ -297,7 +315,6 @@
break;
}
- b47s->window = sflash->window;
b47s->blocksize = sflash->blocksize;
b47s->numblocks = sflash->numblocks;
b47s->size = sflash->size;
@@ -306,6 +323,7 @@
err = mtd_device_parse_register(&b47s->mtd, probes, NULL, NULL, 0);
if (err) {
pr_err("Failed to register MTD device: %d\n", err);
+ iounmap(b47s->window);
return err;
}
@@ -321,6 +339,7 @@
struct bcm47xxsflash *b47s = sflash->priv;
mtd_device_unregister(&b47s->mtd);
+ iounmap(b47s->window);
return 0;
}
diff --git a/drivers/mtd/devices/bcm47xxsflash.h b/drivers/mtd/devices/bcm47xxsflash.h
index fe93daf..1564b62 100644
--- a/drivers/mtd/devices/bcm47xxsflash.h
+++ b/drivers/mtd/devices/bcm47xxsflash.h
@@ -65,7 +65,8 @@
enum bcm47xxsflash_type type;
- u32 window;
+ void __iomem *window;
+
u32 blocksize;
u16 numblocks;
u32 size;
diff --git a/drivers/mtd/devices/docg3.c b/drivers/mtd/devices/docg3.c
index e7b2e43..b833e6c 100644
--- a/drivers/mtd/devices/docg3.c
+++ b/drivers/mtd/devices/docg3.c
@@ -67,16 +67,40 @@
MODULE_PARM_DESC(reliable_mode, "Set the docg3 mode (0=normal MLC, 1=fast, "
"2=reliable) : MLC normal operations are in normal mode");
-/**
- * struct docg3_oobinfo - DiskOnChip G3 OOB layout
- * @eccbytes: 8 bytes are used (1 for Hamming ECC, 7 for BCH ECC)
- * @eccpos: ecc positions (byte 7 is Hamming ECC, byte 8-14 are BCH ECC)
- * @oobfree: free pageinfo bytes (byte 0 until byte 6, byte 15
- */
-static struct nand_ecclayout docg3_oobinfo = {
- .eccbytes = 8,
- .eccpos = {7, 8, 9, 10, 11, 12, 13, 14},
- .oobfree = {{0, 7}, {15, 1} },
+static int docg3_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section)
+ return -ERANGE;
+
+ /* byte 7 is Hamming ECC, byte 8-14 are BCH ECC */
+ oobregion->offset = 7;
+ oobregion->length = 8;
+
+ return 0;
+}
+
+static int docg3_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section > 1)
+ return -ERANGE;
+
+ /* free bytes: byte 0 until byte 6, byte 15 */
+ if (!section) {
+ oobregion->offset = 0;
+ oobregion->length = 7;
+ } else {
+ oobregion->offset = 15;
+ oobregion->length = 1;
+ }
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops nand_ooblayout_docg3_ops = {
+ .ecc = docg3_ooblayout_ecc,
+ .free = docg3_ooblayout_free,
};
static inline u8 doc_readb(struct docg3 *docg3, u16 reg)
@@ -1857,7 +1881,7 @@
mtd->_read_oob = doc_read_oob;
mtd->_write_oob = doc_write_oob;
mtd->_block_isbad = doc_block_isbad;
- mtd->ecclayout = &docg3_oobinfo;
+ mtd_set_ooblayout(mtd, &nand_ooblayout_docg3_ops);
mtd->oobavail = 8;
mtd->ecc_strength = DOC_ECC_BCH_T;
diff --git a/drivers/mtd/devices/m25p80.c b/drivers/mtd/devices/m25p80.c
index c9c3b7f..9d68544 100644
--- a/drivers/mtd/devices/m25p80.c
+++ b/drivers/mtd/devices/m25p80.c
@@ -131,6 +131,28 @@
/* convert the dummy cycles to the number of bytes */
dummy /= 8;
+ if (spi_flash_read_supported(spi)) {
+ struct spi_flash_read_message msg;
+ int ret;
+
+ memset(&msg, 0, sizeof(msg));
+
+ msg.buf = buf;
+ msg.from = from;
+ msg.len = len;
+ msg.read_opcode = nor->read_opcode;
+ msg.addr_width = nor->addr_width;
+ msg.dummy_bytes = dummy;
+ /* TODO: Support other combinations */
+ msg.opcode_nbits = SPI_NBITS_SINGLE;
+ msg.addr_nbits = SPI_NBITS_SINGLE;
+ msg.data_nbits = m25p80_rx_nbits(nor);
+
+ ret = spi_flash_read(spi, &msg);
+ *retlen = msg.retlen;
+ return ret;
+ }
+
spi_message_init(&m);
memset(t, 0, (sizeof t));
diff --git a/drivers/mtd/devices/pmc551.c b/drivers/mtd/devices/pmc551.c
index 708b7e8..220f920 100644
--- a/drivers/mtd/devices/pmc551.c
+++ b/drivers/mtd/devices/pmc551.c
@@ -353,7 +353,7 @@
* mechanism
* returns the size of the memory region found.
*/
-static int fixup_pmc551(struct pci_dev *dev)
+static int __init fixup_pmc551(struct pci_dev *dev)
{
#ifdef CONFIG_MTD_PMC551_BUGFIX
u32 dram_data;
diff --git a/drivers/mtd/maps/ck804xrom.c b/drivers/mtd/maps/ck804xrom.c
index 0455166..4f206a9 100644
--- a/drivers/mtd/maps/ck804xrom.c
+++ b/drivers/mtd/maps/ck804xrom.c
@@ -112,8 +112,8 @@
}
-static int ck804xrom_init_one(struct pci_dev *pdev,
- const struct pci_device_id *ent)
+static int __init ck804xrom_init_one(struct pci_dev *pdev,
+ const struct pci_device_id *ent)
{
static char *rom_probe_types[] = { "cfi_probe", "jedec_probe", NULL };
u8 byte;
diff --git a/drivers/mtd/maps/esb2rom.c b/drivers/mtd/maps/esb2rom.c
index 76ed651..9646b07 100644
--- a/drivers/mtd/maps/esb2rom.c
+++ b/drivers/mtd/maps/esb2rom.c
@@ -144,8 +144,8 @@
pci_dev_put(window->pdev);
}
-static int esb2rom_init_one(struct pci_dev *pdev,
- const struct pci_device_id *ent)
+static int __init esb2rom_init_one(struct pci_dev *pdev,
+ const struct pci_device_id *ent)
{
static char *rom_probe_types[] = { "cfi_probe", "jedec_probe", NULL };
struct esb2rom_window *window = &esb2rom_window;
diff --git a/drivers/mtd/maps/ichxrom.c b/drivers/mtd/maps/ichxrom.c
index 8636bba..e17d02a 100644
--- a/drivers/mtd/maps/ichxrom.c
+++ b/drivers/mtd/maps/ichxrom.c
@@ -84,8 +84,8 @@
}
-static int ichxrom_init_one(struct pci_dev *pdev,
- const struct pci_device_id *ent)
+static int __init ichxrom_init_one(struct pci_dev *pdev,
+ const struct pci_device_id *ent)
{
static char *rom_probe_types[] = { "cfi_probe", "jedec_probe", NULL };
struct ichxrom_window *window = &ichxrom_window;
diff --git a/drivers/mtd/maps/pxa2xx-flash.c b/drivers/mtd/maps/pxa2xx-flash.c
index 7497090..2cde28e 100644
--- a/drivers/mtd/maps/pxa2xx-flash.c
+++ b/drivers/mtd/maps/pxa2xx-flash.c
@@ -71,8 +71,8 @@
info->map.name);
return -ENOMEM;
}
- info->map.cached = memremap(info->map.phys, info->map.size,
- MEMREMAP_WB);
+ info->map.cached =
+ ioremap_cached(info->map.phys, info->map.size);
if (!info->map.cached)
printk(KERN_WARNING "Failed to ioremap cached %s\n",
info->map.name);
@@ -111,7 +111,7 @@
map_destroy(info->mtd);
iounmap(info->map.virt);
if (info->map.cached)
- memunmap(info->map.cached);
+ iounmap(info->map.cached);
kfree(info);
return 0;
}
diff --git a/drivers/mtd/maps/uclinux.c b/drivers/mtd/maps/uclinux.c
index c1af83d..00a8190 100644
--- a/drivers/mtd/maps/uclinux.c
+++ b/drivers/mtd/maps/uclinux.c
@@ -4,11 +4,13 @@
* uclinux.c -- generic memory mapped MTD driver for uclinux
*
* (C) Copyright 2002, Greg Ungerer (gerg@snapgear.com)
+ *
+ * License: GPL
*/
/****************************************************************************/
-#include <linux/module.h>
+#include <linux/moduleparam.h>
#include <linux/types.h>
#include <linux/init.h>
#include <linux/kernel.h>
@@ -117,27 +119,6 @@
return(0);
}
-
-/****************************************************************************/
-
-static void __exit uclinux_mtd_cleanup(void)
-{
- if (uclinux_ram_mtdinfo) {
- mtd_device_unregister(uclinux_ram_mtdinfo);
- map_destroy(uclinux_ram_mtdinfo);
- uclinux_ram_mtdinfo = NULL;
- }
- if (uclinux_ram_map.virt)
- uclinux_ram_map.virt = 0;
-}
-
-/****************************************************************************/
-
-module_init(uclinux_mtd_init);
-module_exit(uclinux_mtd_cleanup);
-
-MODULE_LICENSE("GPL");
-MODULE_AUTHOR("Greg Ungerer <gerg@snapgear.com>");
-MODULE_DESCRIPTION("Generic MTD for uClinux");
+device_initcall(uclinux_mtd_init);
/****************************************************************************/
diff --git a/drivers/mtd/mtdchar.c b/drivers/mtd/mtdchar.c
index 6d19835..2a47a3f 100644
--- a/drivers/mtd/mtdchar.c
+++ b/drivers/mtd/mtdchar.c
@@ -465,38 +465,111 @@
}
/*
- * Copies (and truncates, if necessary) data from the larger struct,
- * nand_ecclayout, to the smaller, deprecated layout struct,
- * nand_ecclayout_user. This is necessary only to support the deprecated
- * API ioctl ECCGETLAYOUT while allowing all new functionality to use
- * nand_ecclayout flexibly (i.e. the struct may change size in new
- * releases without requiring major rewrites).
+ * Copies (and truncates, if necessary) OOB layout information to the
+ * deprecated layout struct, nand_ecclayout_user. This is necessary only to
+ * support the deprecated API ioctl ECCGETLAYOUT while allowing all new
+ * functionality to use mtd_ooblayout_ops flexibly (i.e. mtd_ooblayout_ops
+ * can describe any kind of OOB layout with almost zero overhead from a
+ * memory usage point of view).
*/
-static int shrink_ecclayout(const struct nand_ecclayout *from,
- struct nand_ecclayout_user *to)
+static int shrink_ecclayout(struct mtd_info *mtd,
+ struct nand_ecclayout_user *to)
{
- int i;
+ struct mtd_oob_region oobregion;
+ int i, section = 0, ret;
- if (!from || !to)
+ if (!mtd || !to)
return -EINVAL;
memset(to, 0, sizeof(*to));
- to->eccbytes = min((int)from->eccbytes, MTD_MAX_ECCPOS_ENTRIES);
- for (i = 0; i < to->eccbytes; i++)
- to->eccpos[i] = from->eccpos[i];
+ to->eccbytes = 0;
+ for (i = 0; i < MTD_MAX_ECCPOS_ENTRIES;) {
+ u32 eccpos;
+
+ ret = mtd_ooblayout_ecc(mtd, section, &oobregion);
+ if (ret < 0) {
+ if (ret != -ERANGE)
+ return ret;
+
+ break;
+ }
+
+ eccpos = oobregion.offset;
+ for (; i < MTD_MAX_ECCPOS_ENTRIES &&
+ eccpos < oobregion.offset + oobregion.length; i++) {
+ to->eccpos[i] = eccpos++;
+ to->eccbytes++;
+ }
+ }
for (i = 0; i < MTD_MAX_OOBFREE_ENTRIES; i++) {
- if (from->oobfree[i].length == 0 &&
- from->oobfree[i].offset == 0)
+ ret = mtd_ooblayout_free(mtd, i, &oobregion);
+ if (ret < 0) {
+ if (ret != -ERANGE)
+ return ret;
+
break;
- to->oobavail += from->oobfree[i].length;
- to->oobfree[i] = from->oobfree[i];
+ }
+
+ to->oobfree[i].offset = oobregion.offset;
+ to->oobfree[i].length = oobregion.length;
+ to->oobavail += to->oobfree[i].length;
}
return 0;
}
+static int get_oobinfo(struct mtd_info *mtd, struct nand_oobinfo *to)
+{
+ struct mtd_oob_region oobregion;
+ int i, section = 0, ret;
+
+ if (!mtd || !to)
+ return -EINVAL;
+
+ memset(to, 0, sizeof(*to));
+
+ to->eccbytes = 0;
+ for (i = 0; i < ARRAY_SIZE(to->eccpos);) {
+ u32 eccpos;
+
+ ret = mtd_ooblayout_ecc(mtd, section, &oobregion);
+ if (ret < 0) {
+ if (ret != -ERANGE)
+ return ret;
+
+ break;
+ }
+
+ if (oobregion.length + i > ARRAY_SIZE(to->eccpos))
+ return -EINVAL;
+
+ eccpos = oobregion.offset;
+ for (; eccpos < oobregion.offset + oobregion.length; i++) {
+ to->eccpos[i] = eccpos++;
+ to->eccbytes++;
+ }
+ }
+
+ for (i = 0; i < 8; i++) {
+ ret = mtd_ooblayout_free(mtd, i, &oobregion);
+ if (ret < 0) {
+ if (ret != -ERANGE)
+ return ret;
+
+ break;
+ }
+
+ to->oobfree[i][0] = oobregion.offset;
+ to->oobfree[i][1] = oobregion.length;
+ }
+
+ to->useecc = MTD_NANDECC_AUTOPLACE;
+
+ return 0;
+}
+
static int mtdchar_blkpg_ioctl(struct mtd_info *mtd,
struct blkpg_ioctl_arg *arg)
{
@@ -815,16 +888,12 @@
{
struct nand_oobinfo oi;
- if (!mtd->ecclayout)
+ if (!mtd->ooblayout)
return -EOPNOTSUPP;
- if (mtd->ecclayout->eccbytes > ARRAY_SIZE(oi.eccpos))
- return -EINVAL;
- oi.useecc = MTD_NANDECC_AUTOPLACE;
- memcpy(&oi.eccpos, mtd->ecclayout->eccpos, sizeof(oi.eccpos));
- memcpy(&oi.oobfree, mtd->ecclayout->oobfree,
- sizeof(oi.oobfree));
- oi.eccbytes = mtd->ecclayout->eccbytes;
+ ret = get_oobinfo(mtd, &oi);
+ if (ret)
+ return ret;
if (copy_to_user(argp, &oi, sizeof(struct nand_oobinfo)))
return -EFAULT;
@@ -913,14 +982,14 @@
{
struct nand_ecclayout_user *usrlay;
- if (!mtd->ecclayout)
+ if (!mtd->ooblayout)
return -EOPNOTSUPP;
usrlay = kmalloc(sizeof(*usrlay), GFP_KERNEL);
if (!usrlay)
return -ENOMEM;
- shrink_ecclayout(mtd->ecclayout, usrlay);
+ shrink_ecclayout(mtd, usrlay);
if (copy_to_user(argp, usrlay, sizeof(*usrlay)))
ret = -EFAULT;
diff --git a/drivers/mtd/mtdconcat.c b/drivers/mtd/mtdconcat.c
index 239a8c8..d573606 100644
--- a/drivers/mtd/mtdconcat.c
+++ b/drivers/mtd/mtdconcat.c
@@ -777,7 +777,7 @@
}
- concat->mtd.ecclayout = subdev[0]->ecclayout;
+ mtd_set_ooblayout(&concat->mtd, subdev[0]->ooblayout);
concat->num_subdev = num_devs;
concat->mtd.name = name;
diff --git a/drivers/mtd/mtdcore.c b/drivers/mtd/mtdcore.c
index bee180bd..e3936b8 100644
--- a/drivers/mtd/mtdcore.c
+++ b/drivers/mtd/mtdcore.c
@@ -1016,6 +1016,366 @@
}
EXPORT_SYMBOL_GPL(mtd_write_oob);
+/**
+ * mtd_ooblayout_ecc - Get the OOB region definition of a specific ECC section
+ * @mtd: MTD device structure
+ * @section: ECC section. Depending on the layout you may have all the ECC
+ * bytes stored in a single contiguous section, or one section
+ * per ECC chunk (and sometime several sections for a single ECC
+ * ECC chunk)
+ * @oobecc: OOB region struct filled with the appropriate ECC position
+ * information
+ *
+ * This functions return ECC section information in the OOB area. I you want
+ * to get all the ECC bytes information, then you should call
+ * mtd_ooblayout_ecc(mtd, section++, oobecc) until it returns -ERANGE.
+ *
+ * Returns zero on success, a negative error code otherwise.
+ */
+int mtd_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobecc)
+{
+ memset(oobecc, 0, sizeof(*oobecc));
+
+ if (!mtd || section < 0)
+ return -EINVAL;
+
+ if (!mtd->ooblayout || !mtd->ooblayout->ecc)
+ return -ENOTSUPP;
+
+ return mtd->ooblayout->ecc(mtd, section, oobecc);
+}
+EXPORT_SYMBOL_GPL(mtd_ooblayout_ecc);
+
+/**
+ * mtd_ooblayout_free - Get the OOB region definition of a specific free
+ * section
+ * @mtd: MTD device structure
+ * @section: Free section you are interested in. Depending on the layout
+ * you may have all the free bytes stored in a single contiguous
+ * section, or one section per ECC chunk plus an extra section
+ * for the remaining bytes (or other funky layout).
+ * @oobfree: OOB region struct filled with the appropriate free position
+ * information
+ *
+ * This functions return free bytes position in the OOB area. I you want
+ * to get all the free bytes information, then you should call
+ * mtd_ooblayout_free(mtd, section++, oobfree) until it returns -ERANGE.
+ *
+ * Returns zero on success, a negative error code otherwise.
+ */
+int mtd_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobfree)
+{
+ memset(oobfree, 0, sizeof(*oobfree));
+
+ if (!mtd || section < 0)
+ return -EINVAL;
+
+ if (!mtd->ooblayout || !mtd->ooblayout->free)
+ return -ENOTSUPP;
+
+ return mtd->ooblayout->free(mtd, section, oobfree);
+}
+EXPORT_SYMBOL_GPL(mtd_ooblayout_free);
+
+/**
+ * mtd_ooblayout_find_region - Find the region attached to a specific byte
+ * @mtd: mtd info structure
+ * @byte: the byte we are searching for
+ * @sectionp: pointer where the section id will be stored
+ * @oobregion: used to retrieve the ECC position
+ * @iter: iterator function. Should be either mtd_ooblayout_free or
+ * mtd_ooblayout_ecc depending on the region type you're searching for
+ *
+ * This functions returns the section id and oobregion information of a
+ * specific byte. For example, say you want to know where the 4th ECC byte is
+ * stored, you'll use:
+ *
+ * mtd_ooblayout_find_region(mtd, 3, §ion, &oobregion, mtd_ooblayout_ecc);
+ *
+ * Returns zero on success, a negative error code otherwise.
+ */
+static int mtd_ooblayout_find_region(struct mtd_info *mtd, int byte,
+ int *sectionp, struct mtd_oob_region *oobregion,
+ int (*iter)(struct mtd_info *,
+ int section,
+ struct mtd_oob_region *oobregion))
+{
+ int pos = 0, ret, section = 0;
+
+ memset(oobregion, 0, sizeof(*oobregion));
+
+ while (1) {
+ ret = iter(mtd, section, oobregion);
+ if (ret)
+ return ret;
+
+ if (pos + oobregion->length > byte)
+ break;
+
+ pos += oobregion->length;
+ section++;
+ }
+
+ /*
+ * Adjust region info to make it start at the beginning at the
+ * 'start' ECC byte.
+ */
+ oobregion->offset += byte - pos;
+ oobregion->length -= byte - pos;
+ *sectionp = section;
+
+ return 0;
+}
+
+/**
+ * mtd_ooblayout_find_eccregion - Find the ECC region attached to a specific
+ * ECC byte
+ * @mtd: mtd info structure
+ * @eccbyte: the byte we are searching for
+ * @sectionp: pointer where the section id will be stored
+ * @oobregion: OOB region information
+ *
+ * Works like mtd_ooblayout_find_region() except it searches for a specific ECC
+ * byte.
+ *
+ * Returns zero on success, a negative error code otherwise.
+ */
+int mtd_ooblayout_find_eccregion(struct mtd_info *mtd, int eccbyte,
+ int *section,
+ struct mtd_oob_region *oobregion)
+{
+ return mtd_ooblayout_find_region(mtd, eccbyte, section, oobregion,
+ mtd_ooblayout_ecc);
+}
+EXPORT_SYMBOL_GPL(mtd_ooblayout_find_eccregion);
+
+/**
+ * mtd_ooblayout_get_bytes - Extract OOB bytes from the oob buffer
+ * @mtd: mtd info structure
+ * @buf: destination buffer to store OOB bytes
+ * @oobbuf: OOB buffer
+ * @start: first byte to retrieve
+ * @nbytes: number of bytes to retrieve
+ * @iter: section iterator
+ *
+ * Extract bytes attached to a specific category (ECC or free)
+ * from the OOB buffer and copy them into buf.
+ *
+ * Returns zero on success, a negative error code otherwise.
+ */
+static int mtd_ooblayout_get_bytes(struct mtd_info *mtd, u8 *buf,
+ const u8 *oobbuf, int start, int nbytes,
+ int (*iter)(struct mtd_info *,
+ int section,
+ struct mtd_oob_region *oobregion))
+{
+ struct mtd_oob_region oobregion = { };
+ int section = 0, ret;
+
+ ret = mtd_ooblayout_find_region(mtd, start, §ion,
+ &oobregion, iter);
+
+ while (!ret) {
+ int cnt;
+
+ cnt = oobregion.length > nbytes ? nbytes : oobregion.length;
+ memcpy(buf, oobbuf + oobregion.offset, cnt);
+ buf += cnt;
+ nbytes -= cnt;
+
+ if (!nbytes)
+ break;
+
+ ret = iter(mtd, ++section, &oobregion);
+ }
+
+ return ret;
+}
+
+/**
+ * mtd_ooblayout_set_bytes - put OOB bytes into the oob buffer
+ * @mtd: mtd info structure
+ * @buf: source buffer to get OOB bytes from
+ * @oobbuf: OOB buffer
+ * @start: first OOB byte to set
+ * @nbytes: number of OOB bytes to set
+ * @iter: section iterator
+ *
+ * Fill the OOB buffer with data provided in buf. The category (ECC or free)
+ * is selected by passing the appropriate iterator.
+ *
+ * Returns zero on success, a negative error code otherwise.
+ */
+static int mtd_ooblayout_set_bytes(struct mtd_info *mtd, const u8 *buf,
+ u8 *oobbuf, int start, int nbytes,
+ int (*iter)(struct mtd_info *,
+ int section,
+ struct mtd_oob_region *oobregion))
+{
+ struct mtd_oob_region oobregion = { };
+ int section = 0, ret;
+
+ ret = mtd_ooblayout_find_region(mtd, start, §ion,
+ &oobregion, iter);
+
+ while (!ret) {
+ int cnt;
+
+ cnt = oobregion.length > nbytes ? nbytes : oobregion.length;
+ memcpy(oobbuf + oobregion.offset, buf, cnt);
+ buf += cnt;
+ nbytes -= cnt;
+
+ if (!nbytes)
+ break;
+
+ ret = iter(mtd, ++section, &oobregion);
+ }
+
+ return ret;
+}
+
+/**
+ * mtd_ooblayout_count_bytes - count the number of bytes in a OOB category
+ * @mtd: mtd info structure
+ * @iter: category iterator
+ *
+ * Count the number of bytes in a given category.
+ *
+ * Returns a positive value on success, a negative error code otherwise.
+ */
+static int mtd_ooblayout_count_bytes(struct mtd_info *mtd,
+ int (*iter)(struct mtd_info *,
+ int section,
+ struct mtd_oob_region *oobregion))
+{
+ struct mtd_oob_region oobregion = { };
+ int section = 0, ret, nbytes = 0;
+
+ while (1) {
+ ret = iter(mtd, section++, &oobregion);
+ if (ret) {
+ if (ret == -ERANGE)
+ ret = nbytes;
+ break;
+ }
+
+ nbytes += oobregion.length;
+ }
+
+ return ret;
+}
+
+/**
+ * mtd_ooblayout_get_eccbytes - extract ECC bytes from the oob buffer
+ * @mtd: mtd info structure
+ * @eccbuf: destination buffer to store ECC bytes
+ * @oobbuf: OOB buffer
+ * @start: first ECC byte to retrieve
+ * @nbytes: number of ECC bytes to retrieve
+ *
+ * Works like mtd_ooblayout_get_bytes(), except it acts on ECC bytes.
+ *
+ * Returns zero on success, a negative error code otherwise.
+ */
+int mtd_ooblayout_get_eccbytes(struct mtd_info *mtd, u8 *eccbuf,
+ const u8 *oobbuf, int start, int nbytes)
+{
+ return mtd_ooblayout_get_bytes(mtd, eccbuf, oobbuf, start, nbytes,
+ mtd_ooblayout_ecc);
+}
+EXPORT_SYMBOL_GPL(mtd_ooblayout_get_eccbytes);
+
+/**
+ * mtd_ooblayout_set_eccbytes - set ECC bytes into the oob buffer
+ * @mtd: mtd info structure
+ * @eccbuf: source buffer to get ECC bytes from
+ * @oobbuf: OOB buffer
+ * @start: first ECC byte to set
+ * @nbytes: number of ECC bytes to set
+ *
+ * Works like mtd_ooblayout_set_bytes(), except it acts on ECC bytes.
+ *
+ * Returns zero on success, a negative error code otherwise.
+ */
+int mtd_ooblayout_set_eccbytes(struct mtd_info *mtd, const u8 *eccbuf,
+ u8 *oobbuf, int start, int nbytes)
+{
+ return mtd_ooblayout_set_bytes(mtd, eccbuf, oobbuf, start, nbytes,
+ mtd_ooblayout_ecc);
+}
+EXPORT_SYMBOL_GPL(mtd_ooblayout_set_eccbytes);
+
+/**
+ * mtd_ooblayout_get_databytes - extract data bytes from the oob buffer
+ * @mtd: mtd info structure
+ * @databuf: destination buffer to store ECC bytes
+ * @oobbuf: OOB buffer
+ * @start: first ECC byte to retrieve
+ * @nbytes: number of ECC bytes to retrieve
+ *
+ * Works like mtd_ooblayout_get_bytes(), except it acts on free bytes.
+ *
+ * Returns zero on success, a negative error code otherwise.
+ */
+int mtd_ooblayout_get_databytes(struct mtd_info *mtd, u8 *databuf,
+ const u8 *oobbuf, int start, int nbytes)
+{
+ return mtd_ooblayout_get_bytes(mtd, databuf, oobbuf, start, nbytes,
+ mtd_ooblayout_free);
+}
+EXPORT_SYMBOL_GPL(mtd_ooblayout_get_databytes);
+
+/**
+ * mtd_ooblayout_get_eccbytes - set data bytes into the oob buffer
+ * @mtd: mtd info structure
+ * @eccbuf: source buffer to get data bytes from
+ * @oobbuf: OOB buffer
+ * @start: first ECC byte to set
+ * @nbytes: number of ECC bytes to set
+ *
+ * Works like mtd_ooblayout_get_bytes(), except it acts on free bytes.
+ *
+ * Returns zero on success, a negative error code otherwise.
+ */
+int mtd_ooblayout_set_databytes(struct mtd_info *mtd, const u8 *databuf,
+ u8 *oobbuf, int start, int nbytes)
+{
+ return mtd_ooblayout_set_bytes(mtd, databuf, oobbuf, start, nbytes,
+ mtd_ooblayout_free);
+}
+EXPORT_SYMBOL_GPL(mtd_ooblayout_set_databytes);
+
+/**
+ * mtd_ooblayout_count_freebytes - count the number of free bytes in OOB
+ * @mtd: mtd info structure
+ *
+ * Works like mtd_ooblayout_count_bytes(), except it count free bytes.
+ *
+ * Returns zero on success, a negative error code otherwise.
+ */
+int mtd_ooblayout_count_freebytes(struct mtd_info *mtd)
+{
+ return mtd_ooblayout_count_bytes(mtd, mtd_ooblayout_free);
+}
+EXPORT_SYMBOL_GPL(mtd_ooblayout_count_freebytes);
+
+/**
+ * mtd_ooblayout_count_freebytes - count the number of ECC bytes in OOB
+ * @mtd: mtd info structure
+ *
+ * Works like mtd_ooblayout_count_bytes(), except it count ECC bytes.
+ *
+ * Returns zero on success, a negative error code otherwise.
+ */
+int mtd_ooblayout_count_eccbytes(struct mtd_info *mtd)
+{
+ return mtd_ooblayout_count_bytes(mtd, mtd_ooblayout_ecc);
+}
+EXPORT_SYMBOL_GPL(mtd_ooblayout_count_eccbytes);
+
/*
* Method to access the protection register area, present in some flash
* devices. The user data is one time programmable but the factory data is read
diff --git a/drivers/mtd/mtdpart.c b/drivers/mtd/mtdpart.c
index 08de4b2..1f13e32 100644
--- a/drivers/mtd/mtdpart.c
+++ b/drivers/mtd/mtdpart.c
@@ -317,6 +317,27 @@
return res;
}
+static int part_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct mtd_part *part = mtd_to_part(mtd);
+
+ return mtd_ooblayout_ecc(part->master, section, oobregion);
+}
+
+static int part_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct mtd_part *part = mtd_to_part(mtd);
+
+ return mtd_ooblayout_free(part->master, section, oobregion);
+}
+
+static const struct mtd_ooblayout_ops part_ooblayout_ops = {
+ .ecc = part_ooblayout_ecc,
+ .free = part_ooblayout_free,
+};
+
static inline void free_partition(struct mtd_part *p)
{
kfree(p->mtd.name);
@@ -533,7 +554,7 @@
part->name);
}
- slave->mtd.ecclayout = master->ecclayout;
+ mtd_set_ooblayout(&slave->mtd, &part_ooblayout_ops);
slave->mtd.ecc_step_size = master->ecc_step_size;
slave->mtd.ecc_strength = master->ecc_strength;
slave->mtd.bitflip_threshold = master->bitflip_threshold;
diff --git a/drivers/mtd/nand/ams-delta.c b/drivers/mtd/nand/ams-delta.c
index 68b58c8..78e12cc 100644
--- a/drivers/mtd/nand/ams-delta.c
+++ b/drivers/mtd/nand/ams-delta.c
@@ -224,6 +224,7 @@
/* 25 us command delay time */
this->chip_delay = 30;
this->ecc.mode = NAND_ECC_SOFT;
+ this->ecc.algo = NAND_ECC_HAMMING;
platform_set_drvdata(pdev, io_base);
diff --git a/drivers/mtd/nand/atmel_nand.c b/drivers/mtd/nand/atmel_nand.c
index 20cbaab..efc8ea2 100644
--- a/drivers/mtd/nand/atmel_nand.c
+++ b/drivers/mtd/nand/atmel_nand.c
@@ -36,7 +36,6 @@
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/of_gpio.h>
-#include <linux/of_mtd.h>
#include <linux/mtd/mtd.h>
#include <linux/mtd/nand.h>
#include <linux/mtd/partitions.h>
@@ -72,30 +71,44 @@
uint32_t rb_mask;
};
-/* oob layout for large page size
+/*
+ * oob layout for large page size
* bad block info is on bytes 0 and 1
* the bytes have to be consecutives to avoid
* several NAND_CMD_RNDOUT during read
- */
-static struct nand_ecclayout atmel_oobinfo_large = {
- .eccbytes = 4,
- .eccpos = {60, 61, 62, 63},
- .oobfree = {
- {2, 58}
- },
-};
-
-/* oob layout for small page size
+ *
+ * oob layout for small page size
* bad block info is on bytes 4 and 5
* the bytes have to be consecutives to avoid
* several NAND_CMD_RNDOUT during read
*/
-static struct nand_ecclayout atmel_oobinfo_small = {
- .eccbytes = 4,
- .eccpos = {0, 1, 2, 3},
- .oobfree = {
- {6, 10}
- },
+static int atmel_ooblayout_ecc_sp(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section)
+ return -ERANGE;
+
+ oobregion->length = 4;
+ oobregion->offset = 0;
+
+ return 0;
+}
+
+static int atmel_ooblayout_free_sp(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section)
+ return -ERANGE;
+
+ oobregion->offset = 6;
+ oobregion->length = mtd->oobsize - oobregion->offset;
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops atmel_ooblayout_sp_ops = {
+ .ecc = atmel_ooblayout_ecc_sp,
+ .free = atmel_ooblayout_free_sp,
};
struct atmel_nfc {
@@ -163,8 +176,6 @@
int *pmecc_delta;
};
-static struct nand_ecclayout atmel_pmecc_oobinfo;
-
/*
* Enable NAND.
*/
@@ -434,14 +445,13 @@
static void atmel_read_buf(struct mtd_info *mtd, u8 *buf, int len)
{
struct nand_chip *chip = mtd_to_nand(mtd);
- struct atmel_nand_host *host = nand_get_controller_data(chip);
if (use_dma && len > mtd->oobsize)
/* only use DMA for bigger than oob size: better performances */
if (atmel_nand_dma_op(mtd, buf, len, 1) == 0)
return;
- if (host->board.bus_width_16)
+ if (chip->options & NAND_BUSWIDTH_16)
atmel_read_buf16(mtd, buf, len);
else
atmel_read_buf8(mtd, buf, len);
@@ -450,14 +460,13 @@
static void atmel_write_buf(struct mtd_info *mtd, const u8 *buf, int len)
{
struct nand_chip *chip = mtd_to_nand(mtd);
- struct atmel_nand_host *host = nand_get_controller_data(chip);
if (use_dma && len > mtd->oobsize)
/* only use DMA for bigger than oob size: better performances */
if (atmel_nand_dma_op(mtd, (void *)buf, len, 0) == 0)
return;
- if (host->board.bus_width_16)
+ if (chip->options & NAND_BUSWIDTH_16)
atmel_write_buf16(mtd, buf, len);
else
atmel_write_buf8(mtd, buf, len);
@@ -483,22 +492,6 @@
return (m * cap + 7) / 8;
}
-static void pmecc_config_ecc_layout(struct nand_ecclayout *layout,
- int oobsize, int ecc_len)
-{
- int i;
-
- layout->eccbytes = ecc_len;
-
- /* ECC will occupy the last ecc_len bytes continuously */
- for (i = 0; i < ecc_len; i++)
- layout->eccpos[i] = oobsize - ecc_len + i;
-
- layout->oobfree[0].offset = PMECC_OOB_RESERVED_BYTES;
- layout->oobfree[0].length =
- oobsize - ecc_len - layout->oobfree[0].offset;
-}
-
static void __iomem *pmecc_get_alpha_to(struct atmel_nand_host *host)
{
int table_size;
@@ -836,13 +829,16 @@
dev_dbg(host->dev, "Bit flip in data area, byte_pos: %d, bit_pos: %d, 0x%02x -> 0x%02x\n",
pos, bit_pos, err_byte, *(buf + byte_pos));
} else {
+ struct mtd_oob_region oobregion;
+
/* Bit flip in OOB area */
tmp = sector_num * nand_chip->ecc.bytes
+ (byte_pos - sector_size);
err_byte = ecc[tmp];
ecc[tmp] ^= (1 << bit_pos);
- pos = tmp + nand_chip->ecc.layout->eccpos[0];
+ mtd_ooblayout_ecc(mtd, 0, &oobregion);
+ pos = tmp + oobregion.offset;
dev_dbg(host->dev, "Bit flip in OOB, oob_byte_pos: %d, bit_pos: %d, 0x%02x -> 0x%02x\n",
pos, bit_pos, err_byte, ecc[tmp]);
}
@@ -863,17 +859,6 @@
uint8_t *buf_pos;
int max_bitflips = 0;
- /* If can correct bitfilps from erased page, do the normal check */
- if (host->caps->pmecc_correct_erase_page)
- goto normal_check;
-
- for (i = 0; i < nand_chip->ecc.total; i++)
- if (ecc[i] != 0xff)
- goto normal_check;
- /* Erased page, return OK */
- return 0;
-
-normal_check:
for (i = 0; i < nand_chip->ecc.steps; i++) {
err_nbr = 0;
if (pmecc_stat & 0x1) {
@@ -884,16 +869,30 @@
pmecc_get_sigma(mtd);
err_nbr = pmecc_err_location(mtd);
- if (err_nbr == -1) {
+ if (err_nbr >= 0) {
+ pmecc_correct_data(mtd, buf_pos, ecc, i,
+ nand_chip->ecc.bytes,
+ err_nbr);
+ } else if (!host->caps->pmecc_correct_erase_page) {
+ u8 *ecc_pos = ecc + (i * nand_chip->ecc.bytes);
+
+ /* Try to detect erased pages */
+ err_nbr = nand_check_erased_ecc_chunk(buf_pos,
+ host->pmecc_sector_size,
+ ecc_pos,
+ nand_chip->ecc.bytes,
+ NULL, 0,
+ nand_chip->ecc.strength);
+ }
+
+ if (err_nbr < 0) {
dev_err(host->dev, "PMECC: Too many errors\n");
mtd->ecc_stats.failed++;
return -EIO;
- } else {
- pmecc_correct_data(mtd, buf_pos, ecc, i,
- nand_chip->ecc.bytes, err_nbr);
- mtd->ecc_stats.corrected += err_nbr;
- max_bitflips = max_t(int, max_bitflips, err_nbr);
}
+
+ mtd->ecc_stats.corrected += err_nbr;
+ max_bitflips = max_t(int, max_bitflips, err_nbr);
}
pmecc_stat >>= 1;
}
@@ -931,7 +930,6 @@
struct atmel_nand_host *host = nand_get_controller_data(chip);
int eccsize = chip->ecc.size * chip->ecc.steps;
uint8_t *oob = chip->oob_poi;
- uint32_t *eccpos = chip->ecc.layout->eccpos;
uint32_t stat;
unsigned long end_time;
int bitflips = 0;
@@ -953,7 +951,11 @@
stat = pmecc_readl_relaxed(host->ecc, ISR);
if (stat != 0) {
- bitflips = pmecc_correction(mtd, stat, buf, &oob[eccpos[0]]);
+ struct mtd_oob_region oobregion;
+
+ mtd_ooblayout_ecc(mtd, 0, &oobregion);
+ bitflips = pmecc_correction(mtd, stat, buf,
+ &oob[oobregion.offset]);
if (bitflips < 0)
/* uncorrectable errors */
return 0;
@@ -967,8 +969,8 @@
int page)
{
struct atmel_nand_host *host = nand_get_controller_data(chip);
- uint32_t *eccpos = chip->ecc.layout->eccpos;
- int i, j;
+ struct mtd_oob_region oobregion = { };
+ int i, j, section = 0;
unsigned long end_time;
if (!host->nfc || !host->nfc->write_by_sram) {
@@ -987,11 +989,14 @@
for (i = 0; i < chip->ecc.steps; i++) {
for (j = 0; j < chip->ecc.bytes; j++) {
- int pos;
+ if (!oobregion.length)
+ mtd_ooblayout_ecc(mtd, section, &oobregion);
- pos = i * chip->ecc.bytes + j;
- chip->oob_poi[eccpos[pos]] =
+ chip->oob_poi[oobregion.offset] =
pmecc_readb_ecc_relaxed(host->ecc, i, j);
+ oobregion.length--;
+ oobregion.offset++;
+ section++;
}
}
chip->write_buf(mtd, chip->oob_poi, mtd->oobsize);
@@ -1003,8 +1008,9 @@
{
struct nand_chip *nand_chip = mtd_to_nand(mtd);
struct atmel_nand_host *host = nand_get_controller_data(nand_chip);
+ int eccbytes = mtd_ooblayout_count_eccbytes(mtd);
uint32_t val = 0;
- struct nand_ecclayout *ecc_layout;
+ struct mtd_oob_region oobregion;
pmecc_writel(host->ecc, CTRL, PMECC_CTRL_RST);
pmecc_writel(host->ecc, CTRL, PMECC_CTRL_DISABLE);
@@ -1054,11 +1060,11 @@
| PMECC_CFG_AUTO_DISABLE);
pmecc_writel(host->ecc, CFG, val);
- ecc_layout = nand_chip->ecc.layout;
pmecc_writel(host->ecc, SAREA, mtd->oobsize - 1);
- pmecc_writel(host->ecc, SADDR, ecc_layout->eccpos[0]);
+ mtd_ooblayout_ecc(mtd, 0, &oobregion);
+ pmecc_writel(host->ecc, SADDR, oobregion.offset);
pmecc_writel(host->ecc, EADDR,
- ecc_layout->eccpos[ecc_layout->eccbytes - 1]);
+ oobregion.offset + eccbytes - 1);
/* See datasheet about PMECC Clock Control Register */
pmecc_writel(host->ecc, CLK, 2);
pmecc_writel(host->ecc, IDR, 0xff);
@@ -1206,6 +1212,7 @@
dev_warn(host->dev,
"Can't get I/O resource regs for PMECC controller, rolling back on software ECC\n");
nand_chip->ecc.mode = NAND_ECC_SOFT;
+ nand_chip->ecc.algo = NAND_ECC_HAMMING;
return 0;
}
@@ -1280,11 +1287,8 @@
err_no = -EINVAL;
goto err;
}
- pmecc_config_ecc_layout(&atmel_pmecc_oobinfo,
- mtd->oobsize,
- nand_chip->ecc.total);
- nand_chip->ecc.layout = &atmel_pmecc_oobinfo;
+ mtd_set_ooblayout(mtd, &nand_ooblayout_lp_ops);
break;
default:
dev_warn(host->dev,
@@ -1292,6 +1296,7 @@
/* page size not handled by HW ECC */
/* switching back to soft ECC */
nand_chip->ecc.mode = NAND_ECC_SOFT;
+ nand_chip->ecc.algo = NAND_ECC_HAMMING;
return 0;
}
@@ -1359,12 +1364,12 @@
{
int eccsize = chip->ecc.size;
int eccbytes = chip->ecc.bytes;
- uint32_t *eccpos = chip->ecc.layout->eccpos;
uint8_t *p = buf;
uint8_t *oob = chip->oob_poi;
uint8_t *ecc_pos;
int stat;
unsigned int max_bitflips = 0;
+ struct mtd_oob_region oobregion = {};
/*
* Errata: ALE is incorrectly wired up to the ECC controller
@@ -1382,19 +1387,20 @@
chip->read_buf(mtd, p, eccsize);
/* move to ECC position if needed */
- if (eccpos[0] != 0) {
- /* This only works on large pages
- * because the ECC controller waits for
- * NAND_CMD_RNDOUTSTART after the
- * NAND_CMD_RNDOUT.
- * anyway, for small pages, the eccpos[0] == 0
+ mtd_ooblayout_ecc(mtd, 0, &oobregion);
+ if (oobregion.offset != 0) {
+ /*
+ * This only works on large pages because the ECC controller
+ * waits for NAND_CMD_RNDOUTSTART after the NAND_CMD_RNDOUT.
+ * Anyway, for small pages, the first ECC byte is at offset
+ * 0 in the OOB area.
*/
chip->cmdfunc(mtd, NAND_CMD_RNDOUT,
- mtd->writesize + eccpos[0], -1);
+ mtd->writesize + oobregion.offset, -1);
}
/* the ECC controller needs to read the ECC just after the data */
- ecc_pos = oob + eccpos[0];
+ ecc_pos = oob + oobregion.offset;
chip->read_buf(mtd, ecc_pos, eccbytes);
/* check if there's an error */
@@ -1504,58 +1510,17 @@
ecc_writel(host->ecc, CR, ATMEL_ECC_RST);
}
-static int atmel_of_init_port(struct atmel_nand_host *host,
- struct device_node *np)
+static int atmel_of_init_ecc(struct atmel_nand_host *host,
+ struct device_node *np)
{
- u32 val;
u32 offset[2];
- int ecc_mode;
- struct atmel_nand_data *board = &host->board;
- enum of_gpio_flags flags = 0;
-
- host->caps = (struct atmel_nand_caps *)
- of_device_get_match_data(host->dev);
-
- if (of_property_read_u32(np, "atmel,nand-addr-offset", &val) == 0) {
- if (val >= 32) {
- dev_err(host->dev, "invalid addr-offset %u\n", val);
- return -EINVAL;
- }
- board->ale = val;
- }
-
- if (of_property_read_u32(np, "atmel,nand-cmd-offset", &val) == 0) {
- if (val >= 32) {
- dev_err(host->dev, "invalid cmd-offset %u\n", val);
- return -EINVAL;
- }
- board->cle = val;
- }
-
- ecc_mode = of_get_nand_ecc_mode(np);
-
- board->ecc_mode = ecc_mode < 0 ? NAND_ECC_SOFT : ecc_mode;
-
- board->on_flash_bbt = of_get_nand_on_flash_bbt(np);
-
- board->has_dma = of_property_read_bool(np, "atmel,nand-has-dma");
-
- if (of_get_nand_bus_width(np) == 16)
- board->bus_width_16 = 1;
-
- board->rdy_pin = of_get_gpio_flags(np, 0, &flags);
- board->rdy_pin_active_low = (flags == OF_GPIO_ACTIVE_LOW);
-
- board->enable_pin = of_get_gpio(np, 1);
- board->det_pin = of_get_gpio(np, 2);
+ u32 val;
host->has_pmecc = of_property_read_bool(np, "atmel,has-pmecc");
- /* load the nfc driver if there is */
- of_platform_populate(np, NULL, NULL, host->dev);
-
- if (!(board->ecc_mode == NAND_ECC_HW) || !host->has_pmecc)
- return 0; /* Not using PMECC */
+ /* Not using PMECC */
+ if (!(host->nand_chip.ecc.mode == NAND_ECC_HW) || !host->has_pmecc)
+ return 0;
/* use PMECC, get correction capability, sector size and lookup
* table offset.
@@ -1596,16 +1561,65 @@
/* Will build a lookup table and initialize the offset later */
return 0;
}
+
if (!offset[0] && !offset[1]) {
dev_err(host->dev, "Invalid PMECC lookup table offset\n");
return -EINVAL;
}
+
host->pmecc_lookup_table_offset_512 = offset[0];
host->pmecc_lookup_table_offset_1024 = offset[1];
return 0;
}
+static int atmel_of_init_port(struct atmel_nand_host *host,
+ struct device_node *np)
+{
+ u32 val;
+ struct atmel_nand_data *board = &host->board;
+ enum of_gpio_flags flags = 0;
+
+ host->caps = (struct atmel_nand_caps *)
+ of_device_get_match_data(host->dev);
+
+ if (of_property_read_u32(np, "atmel,nand-addr-offset", &val) == 0) {
+ if (val >= 32) {
+ dev_err(host->dev, "invalid addr-offset %u\n", val);
+ return -EINVAL;
+ }
+ board->ale = val;
+ }
+
+ if (of_property_read_u32(np, "atmel,nand-cmd-offset", &val) == 0) {
+ if (val >= 32) {
+ dev_err(host->dev, "invalid cmd-offset %u\n", val);
+ return -EINVAL;
+ }
+ board->cle = val;
+ }
+
+ board->has_dma = of_property_read_bool(np, "atmel,nand-has-dma");
+
+ board->rdy_pin = of_get_gpio_flags(np, 0, &flags);
+ board->rdy_pin_active_low = (flags == OF_GPIO_ACTIVE_LOW);
+
+ board->enable_pin = of_get_gpio(np, 1);
+ board->det_pin = of_get_gpio(np, 2);
+
+ /* load the nfc driver if there is */
+ of_platform_populate(np, NULL, NULL, host->dev);
+
+ /*
+ * Initialize ECC mode to NAND_ECC_SOFT so that we have a correct value
+ * even if the nand-ecc-mode property is not defined.
+ */
+ host->nand_chip.ecc.mode = NAND_ECC_SOFT;
+ host->nand_chip.ecc.algo = NAND_ECC_HAMMING;
+
+ return 0;
+}
+
static int atmel_hw_nand_init_params(struct platform_device *pdev,
struct atmel_nand_host *host)
{
@@ -1618,6 +1632,7 @@
dev_err(host->dev,
"Can't get I/O resource regs, use software ECC\n");
nand_chip->ecc.mode = NAND_ECC_SOFT;
+ nand_chip->ecc.algo = NAND_ECC_HAMMING;
return 0;
}
@@ -1631,25 +1646,26 @@
/* set ECC page size and oob layout */
switch (mtd->writesize) {
case 512:
- nand_chip->ecc.layout = &atmel_oobinfo_small;
+ mtd_set_ooblayout(mtd, &atmel_ooblayout_sp_ops);
ecc_writel(host->ecc, MR, ATMEL_ECC_PAGESIZE_528);
break;
case 1024:
- nand_chip->ecc.layout = &atmel_oobinfo_large;
+ mtd_set_ooblayout(mtd, &nand_ooblayout_lp_ops);
ecc_writel(host->ecc, MR, ATMEL_ECC_PAGESIZE_1056);
break;
case 2048:
- nand_chip->ecc.layout = &atmel_oobinfo_large;
+ mtd_set_ooblayout(mtd, &nand_ooblayout_lp_ops);
ecc_writel(host->ecc, MR, ATMEL_ECC_PAGESIZE_2112);
break;
case 4096:
- nand_chip->ecc.layout = &atmel_oobinfo_large;
+ mtd_set_ooblayout(mtd, &nand_ooblayout_lp_ops);
ecc_writel(host->ecc, MR, ATMEL_ECC_PAGESIZE_4224);
break;
default:
/* page size not handled by HW ECC */
/* switching back to soft ECC */
nand_chip->ecc.mode = NAND_ECC_SOFT;
+ nand_chip->ecc.algo = NAND_ECC_HAMMING;
return 0;
}
@@ -2147,6 +2163,19 @@
} else {
memcpy(&host->board, dev_get_platdata(&pdev->dev),
sizeof(struct atmel_nand_data));
+ nand_chip->ecc.mode = host->board.ecc_mode;
+
+ /*
+ * When using software ECC every supported avr32 board means
+ * Hamming algorithm. If that ever changes we'll need to add
+ * ecc_algo field to the struct atmel_nand_data.
+ */
+ if (nand_chip->ecc.mode == NAND_ECC_SOFT)
+ nand_chip->ecc.algo = NAND_ECC_HAMMING;
+
+ /* 16-bit bus width */
+ if (host->board.bus_width_16)
+ nand_chip->options |= NAND_BUSWIDTH_16;
}
/* link the private data structures */
@@ -2188,11 +2217,8 @@
nand_chip->cmd_ctrl = atmel_nand_cmd_ctrl;
}
- nand_chip->ecc.mode = host->board.ecc_mode;
nand_chip->chip_delay = 40; /* 40us command delay time */
- if (host->board.bus_width_16) /* 16-bit bus width */
- nand_chip->options |= NAND_BUSWIDTH_16;
nand_chip->read_buf = atmel_read_buf;
nand_chip->write_buf = atmel_write_buf;
@@ -2225,11 +2251,6 @@
}
}
- if (host->board.on_flash_bbt || on_flash_bbt) {
- dev_info(&pdev->dev, "Use On Flash BBT\n");
- nand_chip->bbt_options |= NAND_BBT_USE_FLASH;
- }
-
if (!host->board.has_dma)
use_dma = 0;
@@ -2256,6 +2277,18 @@
goto err_scan_ident;
}
+ if (host->board.on_flash_bbt || on_flash_bbt)
+ nand_chip->bbt_options |= NAND_BBT_USE_FLASH;
+
+ if (nand_chip->bbt_options & NAND_BBT_USE_FLASH)
+ dev_info(&pdev->dev, "Use On Flash BBT\n");
+
+ if (IS_ENABLED(CONFIG_OF) && pdev->dev.of_node) {
+ res = atmel_of_init_ecc(host, pdev->dev.of_node);
+ if (res)
+ goto err_hw_ecc;
+ }
+
if (nand_chip->ecc.mode == NAND_ECC_HW) {
if (host->has_pmecc)
res = atmel_pmecc_nand_init_params(pdev, host);
diff --git a/drivers/mtd/nand/au1550nd.c b/drivers/mtd/nand/au1550nd.c
index 341ea49..9bf6d99 100644
--- a/drivers/mtd/nand/au1550nd.c
+++ b/drivers/mtd/nand/au1550nd.c
@@ -459,6 +459,7 @@
/* 30 us command delay time */
this->chip_delay = 30;
this->ecc.mode = NAND_ECC_SOFT;
+ this->ecc.algo = NAND_ECC_HAMMING;
if (pd->devwidth)
this->options |= NAND_BUSWIDTH_16;
diff --git a/drivers/mtd/nand/bf5xx_nand.c b/drivers/mtd/nand/bf5xx_nand.c
index 7f6b30e..37da423 100644
--- a/drivers/mtd/nand/bf5xx_nand.c
+++ b/drivers/mtd/nand/bf5xx_nand.c
@@ -109,28 +109,33 @@
0};
#ifdef CONFIG_MTD_NAND_BF5XX_BOOTROM_ECC
-static struct nand_ecclayout bootrom_ecclayout = {
- .eccbytes = 24,
- .eccpos = {
- 0x8 * 0, 0x8 * 0 + 1, 0x8 * 0 + 2,
- 0x8 * 1, 0x8 * 1 + 1, 0x8 * 1 + 2,
- 0x8 * 2, 0x8 * 2 + 1, 0x8 * 2 + 2,
- 0x8 * 3, 0x8 * 3 + 1, 0x8 * 3 + 2,
- 0x8 * 4, 0x8 * 4 + 1, 0x8 * 4 + 2,
- 0x8 * 5, 0x8 * 5 + 1, 0x8 * 5 + 2,
- 0x8 * 6, 0x8 * 6 + 1, 0x8 * 6 + 2,
- 0x8 * 7, 0x8 * 7 + 1, 0x8 * 7 + 2
- },
- .oobfree = {
- { 0x8 * 0 + 3, 5 },
- { 0x8 * 1 + 3, 5 },
- { 0x8 * 2 + 3, 5 },
- { 0x8 * 3 + 3, 5 },
- { 0x8 * 4 + 3, 5 },
- { 0x8 * 5 + 3, 5 },
- { 0x8 * 6 + 3, 5 },
- { 0x8 * 7 + 3, 5 },
- }
+static int bootrom_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section > 7)
+ return -ERANGE;
+
+ oobregion->offset = section * 8;
+ oobregion->length = 3;
+
+ return 0;
+}
+
+static int bootrom_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section > 7)
+ return -ERANGE;
+
+ oobregion->offset = (section * 8) + 3;
+ oobregion->length = 5;
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops bootrom_ooblayout_ops = {
+ .ecc = bootrom_ooblayout_ecc,
+ .free = bootrom_ooblayout_free,
};
#endif
@@ -800,7 +805,7 @@
/* setup hardware ECC data struct */
if (hardware_ecc) {
#ifdef CONFIG_MTD_NAND_BF5XX_BOOTROM_ECC
- chip->ecc.layout = &bootrom_ecclayout;
+ mtd_set_ooblayout(mtd, &bootrom_ooblayout_ops);
#endif
chip->read_buf = bf5xx_nand_dma_read_buf;
chip->write_buf = bf5xx_nand_dma_write_buf;
@@ -812,6 +817,7 @@
chip->ecc.write_page_raw = bf5xx_nand_write_page_raw;
} else {
chip->ecc.mode = NAND_ECC_SOFT;
+ chip->ecc.algo = NAND_ECC_HAMMING;
}
/* scan hardware nand chip and setup mtd info data struct */
diff --git a/drivers/mtd/nand/brcmnand/brcmnand.c b/drivers/mtd/nand/brcmnand/brcmnand.c
index e052839..b76ad7c 100644
--- a/drivers/mtd/nand/brcmnand/brcmnand.c
+++ b/drivers/mtd/nand/brcmnand/brcmnand.c
@@ -32,7 +32,6 @@
#include <linux/mtd/nand.h>
#include <linux/mtd/partitions.h>
#include <linux/of.h>
-#include <linux/of_mtd.h>
#include <linux/of_platform.h>
#include <linux/slab.h>
#include <linux/list.h>
@@ -601,7 +600,7 @@
static inline int brcmnand_cmd_shift(struct brcmnand_controller *ctrl)
{
- if (ctrl->nand_version < 0x0700)
+ if (ctrl->nand_version < 0x0602)
return 24;
return 0;
}
@@ -781,55 +780,161 @@
}
/*
- * Returns a nand_ecclayout strucutre for the given layout/configuration.
- * Returns NULL on failure.
+ * Set mtd->ooblayout to the appropriate mtd_ooblayout_ops given
+ * the layout/configuration.
+ * Returns -ERRCODE on failure.
*/
-static struct nand_ecclayout *brcmnand_create_layout(int ecc_level,
- struct brcmnand_host *host)
+static int brcmnand_hamming_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+ struct brcmnand_host *host = nand_get_controller_data(chip);
struct brcmnand_cfg *cfg = &host->hwcfg;
- int i, j;
- struct nand_ecclayout *layout;
- int req;
- int sectors;
- int sas;
- int idx1, idx2;
+ int sas = cfg->spare_area_size << cfg->sector_size_1k;
+ int sectors = cfg->page_size / (512 << cfg->sector_size_1k);
- layout = devm_kzalloc(&host->pdev->dev, sizeof(*layout), GFP_KERNEL);
- if (!layout)
- return NULL;
+ if (section >= sectors)
+ return -ERANGE;
- sectors = cfg->page_size / (512 << cfg->sector_size_1k);
- sas = cfg->spare_area_size << cfg->sector_size_1k;
+ oobregion->offset = (section * sas) + 6;
+ oobregion->length = 3;
- /* Hamming */
- if (is_hamming_ecc(cfg)) {
- for (i = 0, idx1 = 0, idx2 = 0; i < sectors; i++) {
- /* First sector of each page may have BBI */
- if (i == 0) {
- layout->oobfree[idx2].offset = i * sas + 1;
- /* Small-page NAND use byte 6 for BBI */
- if (cfg->page_size == 512)
- layout->oobfree[idx2].offset--;
- layout->oobfree[idx2].length = 5;
- } else {
- layout->oobfree[idx2].offset = i * sas;
- layout->oobfree[idx2].length = 6;
- }
- idx2++;
- layout->eccpos[idx1++] = i * sas + 6;
- layout->eccpos[idx1++] = i * sas + 7;
- layout->eccpos[idx1++] = i * sas + 8;
- layout->oobfree[idx2].offset = i * sas + 9;
- layout->oobfree[idx2].length = 7;
- idx2++;
- /* Leave zero-terminated entry for OOBFREE */
- if (idx1 >= MTD_MAX_ECCPOS_ENTRIES_LARGE ||
- idx2 >= MTD_MAX_OOBFREE_ENTRIES_LARGE - 1)
- break;
+ return 0;
+}
+
+static int brcmnand_hamming_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+ struct brcmnand_host *host = nand_get_controller_data(chip);
+ struct brcmnand_cfg *cfg = &host->hwcfg;
+ int sas = cfg->spare_area_size << cfg->sector_size_1k;
+ int sectors = cfg->page_size / (512 << cfg->sector_size_1k);
+
+ if (section >= sectors * 2)
+ return -ERANGE;
+
+ oobregion->offset = (section / 2) * sas;
+
+ if (section & 1) {
+ oobregion->offset += 9;
+ oobregion->length = 7;
+ } else {
+ oobregion->length = 6;
+
+ /* First sector of each page may have BBI */
+ if (!section) {
+ /*
+ * Small-page NAND use byte 6 for BBI while large-page
+ * NAND use byte 0.
+ */
+ if (cfg->page_size > 512)
+ oobregion->offset++;
+ oobregion->length--;
}
+ }
- return layout;
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops brcmnand_hamming_ooblayout_ops = {
+ .ecc = brcmnand_hamming_ooblayout_ecc,
+ .free = brcmnand_hamming_ooblayout_free,
+};
+
+static int brcmnand_bch_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+ struct brcmnand_host *host = nand_get_controller_data(chip);
+ struct brcmnand_cfg *cfg = &host->hwcfg;
+ int sas = cfg->spare_area_size << cfg->sector_size_1k;
+ int sectors = cfg->page_size / (512 << cfg->sector_size_1k);
+
+ if (section >= sectors)
+ return -ERANGE;
+
+ oobregion->offset = (section * (sas + 1)) - chip->ecc.bytes;
+ oobregion->length = chip->ecc.bytes;
+
+ return 0;
+}
+
+static int brcmnand_bch_ooblayout_free_lp(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+ struct brcmnand_host *host = nand_get_controller_data(chip);
+ struct brcmnand_cfg *cfg = &host->hwcfg;
+ int sas = cfg->spare_area_size << cfg->sector_size_1k;
+ int sectors = cfg->page_size / (512 << cfg->sector_size_1k);
+
+ if (section >= sectors)
+ return -ERANGE;
+
+ if (sas <= chip->ecc.bytes)
+ return 0;
+
+ oobregion->offset = section * sas;
+ oobregion->length = sas - chip->ecc.bytes;
+
+ if (!section) {
+ oobregion->offset++;
+ oobregion->length--;
+ }
+
+ return 0;
+}
+
+static int brcmnand_bch_ooblayout_free_sp(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+ struct brcmnand_host *host = nand_get_controller_data(chip);
+ struct brcmnand_cfg *cfg = &host->hwcfg;
+ int sas = cfg->spare_area_size << cfg->sector_size_1k;
+
+ if (section > 1 || sas - chip->ecc.bytes < 6 ||
+ (section && sas - chip->ecc.bytes == 6))
+ return -ERANGE;
+
+ if (!section) {
+ oobregion->offset = 0;
+ oobregion->length = 5;
+ } else {
+ oobregion->offset = 6;
+ oobregion->length = sas - chip->ecc.bytes - 6;
+ }
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops brcmnand_bch_lp_ooblayout_ops = {
+ .ecc = brcmnand_bch_ooblayout_ecc,
+ .free = brcmnand_bch_ooblayout_free_lp,
+};
+
+static const struct mtd_ooblayout_ops brcmnand_bch_sp_ooblayout_ops = {
+ .ecc = brcmnand_bch_ooblayout_ecc,
+ .free = brcmnand_bch_ooblayout_free_sp,
+};
+
+static int brcmstb_choose_ecc_layout(struct brcmnand_host *host)
+{
+ struct brcmnand_cfg *p = &host->hwcfg;
+ struct mtd_info *mtd = nand_to_mtd(&host->chip);
+ struct nand_ecc_ctrl *ecc = &host->chip.ecc;
+ unsigned int ecc_level = p->ecc_level;
+ int sas = p->spare_area_size << p->sector_size_1k;
+ int sectors = p->page_size / (512 << p->sector_size_1k);
+
+ if (p->sector_size_1k)
+ ecc_level <<= 1;
+
+ if (is_hamming_ecc(p)) {
+ ecc->bytes = 3 * sectors;
+ mtd_set_ooblayout(mtd, &brcmnand_hamming_ooblayout_ops);
+ return 0;
}
/*
@@ -838,70 +943,20 @@
* >= v5.0: ECC_REQ = ceil(BCH_T * 14/8)
* But we will just be conservative.
*/
- req = DIV_ROUND_UP(ecc_level * 14, 8);
- if (req >= sas) {
+ ecc->bytes = DIV_ROUND_UP(ecc_level * 14, 8);
+ if (p->page_size == 512)
+ mtd_set_ooblayout(mtd, &brcmnand_bch_sp_ooblayout_ops);
+ else
+ mtd_set_ooblayout(mtd, &brcmnand_bch_lp_ooblayout_ops);
+
+ if (ecc->bytes >= sas) {
dev_err(&host->pdev->dev,
"error: ECC too large for OOB (ECC bytes %d, spare sector %d)\n",
- req, sas);
- return NULL;
+ ecc->bytes, sas);
+ return -EINVAL;
}
- layout->eccbytes = req * sectors;
- for (i = 0, idx1 = 0, idx2 = 0; i < sectors; i++) {
- for (j = sas - req; j < sas && idx1 <
- MTD_MAX_ECCPOS_ENTRIES_LARGE; j++, idx1++)
- layout->eccpos[idx1] = i * sas + j;
-
- /* First sector of each page may have BBI */
- if (i == 0) {
- if (cfg->page_size == 512 && (sas - req >= 6)) {
- /* Small-page NAND use byte 6 for BBI */
- layout->oobfree[idx2].offset = 0;
- layout->oobfree[idx2].length = 5;
- idx2++;
- if (sas - req > 6) {
- layout->oobfree[idx2].offset = 6;
- layout->oobfree[idx2].length =
- sas - req - 6;
- idx2++;
- }
- } else if (sas > req + 1) {
- layout->oobfree[idx2].offset = i * sas + 1;
- layout->oobfree[idx2].length = sas - req - 1;
- idx2++;
- }
- } else if (sas > req) {
- layout->oobfree[idx2].offset = i * sas;
- layout->oobfree[idx2].length = sas - req;
- idx2++;
- }
- /* Leave zero-terminated entry for OOBFREE */
- if (idx1 >= MTD_MAX_ECCPOS_ENTRIES_LARGE ||
- idx2 >= MTD_MAX_OOBFREE_ENTRIES_LARGE - 1)
- break;
- }
-
- return layout;
-}
-
-static struct nand_ecclayout *brcmstb_choose_ecc_layout(
- struct brcmnand_host *host)
-{
- struct nand_ecclayout *layout;
- struct brcmnand_cfg *p = &host->hwcfg;
- unsigned int ecc_level = p->ecc_level;
-
- if (p->sector_size_1k)
- ecc_level <<= 1;
-
- layout = brcmnand_create_layout(ecc_level, host);
- if (!layout) {
- dev_err(&host->pdev->dev,
- "no proper ecc_layout for this NAND cfg\n");
- return NULL;
- }
-
- return layout;
+ return 0;
}
static void brcmnand_wp(struct mtd_info *mtd, int wp)
@@ -1870,9 +1925,31 @@
cfg->col_adr_bytes = 2;
cfg->blk_adr_bytes = get_blk_adr_bytes(mtd->size, mtd->writesize);
+ if (chip->ecc.mode != NAND_ECC_HW) {
+ dev_err(ctrl->dev, "only HW ECC supported; selected: %d\n",
+ chip->ecc.mode);
+ return -EINVAL;
+ }
+
+ if (chip->ecc.algo == NAND_ECC_UNKNOWN) {
+ if (chip->ecc.strength == 1 && chip->ecc.size == 512)
+ /* Default to Hamming for 1-bit ECC, if unspecified */
+ chip->ecc.algo = NAND_ECC_HAMMING;
+ else
+ /* Otherwise, BCH */
+ chip->ecc.algo = NAND_ECC_BCH;
+ }
+
+ if (chip->ecc.algo == NAND_ECC_HAMMING && (chip->ecc.strength != 1 ||
+ chip->ecc.size != 512)) {
+ dev_err(ctrl->dev, "invalid Hamming params: %d bits per %d bytes\n",
+ chip->ecc.strength, chip->ecc.size);
+ return -EINVAL;
+ }
+
switch (chip->ecc.size) {
case 512:
- if (chip->ecc.strength == 1) /* Hamming */
+ if (chip->ecc.algo == NAND_ECC_HAMMING)
cfg->ecc_level = 15;
else
cfg->ecc_level = chip->ecc.strength;
@@ -2001,8 +2078,8 @@
*/
chip->options |= NAND_USE_BOUNCE_BUFFER;
- if (of_get_nand_on_flash_bbt(dn))
- chip->bbt_options |= NAND_BBT_USE_FLASH | NAND_BBT_NO_OOB;
+ if (chip->bbt_options & NAND_BBT_USE_FLASH)
+ chip->bbt_options |= NAND_BBT_NO_OOB;
if (brcmnand_setup_dev(host))
return -ENXIO;
@@ -2011,9 +2088,9 @@
/* only use our internal HW threshold */
mtd->bitflip_threshold = 1;
- chip->ecc.layout = brcmstb_choose_ecc_layout(host);
- if (!chip->ecc.layout)
- return -ENXIO;
+ ret = brcmstb_choose_ecc_layout(host);
+ if (ret)
+ return ret;
if (nand_scan_tail(mtd))
return -ENXIO;
@@ -2115,6 +2192,7 @@
{ .compatible = "brcm,brcmnand-v5.0" },
{ .compatible = "brcm,brcmnand-v6.0" },
{ .compatible = "brcm,brcmnand-v6.1" },
+ { .compatible = "brcm,brcmnand-v6.2" },
{ .compatible = "brcm,brcmnand-v7.0" },
{ .compatible = "brcm,brcmnand-v7.1" },
{},
diff --git a/drivers/mtd/nand/cafe_nand.c b/drivers/mtd/nand/cafe_nand.c
index e553aff..0b0c937 100644
--- a/drivers/mtd/nand/cafe_nand.c
+++ b/drivers/mtd/nand/cafe_nand.c
@@ -459,10 +459,37 @@
return max_bitflips;
}
-static struct nand_ecclayout cafe_oobinfo_2048 = {
- .eccbytes = 14,
- .eccpos = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13},
- .oobfree = {{14, 50}}
+static int cafe_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+
+ if (section)
+ return -ERANGE;
+
+ oobregion->offset = 0;
+ oobregion->length = chip->ecc.total;
+
+ return 0;
+}
+
+static int cafe_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+
+ if (section)
+ return -ERANGE;
+
+ oobregion->offset = chip->ecc.total;
+ oobregion->length = mtd->oobsize - chip->ecc.total;
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops cafe_ooblayout_ops = {
+ .ecc = cafe_ooblayout_ecc,
+ .free = cafe_ooblayout_free,
};
/* Ick. The BBT code really ought to be able to work this bit out
@@ -494,12 +521,6 @@
.pattern = cafe_mirror_pattern_2048
};
-static struct nand_ecclayout cafe_oobinfo_512 = {
- .eccbytes = 14,
- .eccpos = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13},
- .oobfree = {{14, 2}}
-};
-
static struct nand_bbt_descr cafe_bbt_main_descr_512 = {
.options = NAND_BBT_LASTBLOCK | NAND_BBT_CREATE | NAND_BBT_WRITE
| NAND_BBT_2BIT | NAND_BBT_VERSION,
@@ -743,12 +764,11 @@
cafe->ctl2 |= 1<<29; /* 2KiB page size */
/* Set up ECC according to the type of chip we found */
+ mtd_set_ooblayout(mtd, &cafe_ooblayout_ops);
if (mtd->writesize == 2048) {
- cafe->nand.ecc.layout = &cafe_oobinfo_2048;
cafe->nand.bbt_td = &cafe_bbt_main_descr_2048;
cafe->nand.bbt_md = &cafe_bbt_mirror_descr_2048;
} else if (mtd->writesize == 512) {
- cafe->nand.ecc.layout = &cafe_oobinfo_512;
cafe->nand.bbt_td = &cafe_bbt_main_descr_512;
cafe->nand.bbt_md = &cafe_bbt_mirror_descr_512;
} else {
diff --git a/drivers/mtd/nand/cmx270_nand.c b/drivers/mtd/nand/cmx270_nand.c
index 6f97ebb..4913378 100644
--- a/drivers/mtd/nand/cmx270_nand.c
+++ b/drivers/mtd/nand/cmx270_nand.c
@@ -187,6 +187,7 @@
/* 15 us command delay time */
this->chip_delay = 20;
this->ecc.mode = NAND_ECC_SOFT;
+ this->ecc.algo = NAND_ECC_HAMMING;
/* read/write functions */
this->read_byte = cmx270_read_byte;
diff --git a/drivers/mtd/nand/davinci_nand.c b/drivers/mtd/nand/davinci_nand.c
index 8cb821b..cc07ba0 100644
--- a/drivers/mtd/nand/davinci_nand.c
+++ b/drivers/mtd/nand/davinci_nand.c
@@ -34,7 +34,6 @@
#include <linux/slab.h>
#include <linux/of_device.h>
#include <linux/of.h>
-#include <linux/of_mtd.h>
#include <linux/platform_data/mtd-davinci.h>
#include <linux/platform_data/mtd-davinci-aemif.h>
@@ -54,7 +53,6 @@
*/
struct davinci_nand_info {
struct nand_chip chip;
- struct nand_ecclayout ecclayout;
struct device *dev;
struct clk *clk;
@@ -480,63 +478,46 @@
* ten ECC bytes plus the manufacturer's bad block marker byte, and
* and not overlapping the default BBT markers.
*/
-static struct nand_ecclayout hwecc4_small = {
- .eccbytes = 10,
- .eccpos = { 0, 1, 2, 3, 4,
- /* offset 5 holds the badblock marker */
- 6, 7,
- 13, 14, 15, },
- .oobfree = {
- {.offset = 8, .length = 5, },
- {.offset = 16, },
- },
-};
+static int hwecc4_ooblayout_small_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section > 2)
+ return -ERANGE;
-/* An ECC layout for using 4-bit ECC with large-page (2048bytes) flash,
- * storing ten ECC bytes plus the manufacturer's bad block marker byte,
- * and not overlapping the default BBT markers.
- */
-static struct nand_ecclayout hwecc4_2048 = {
- .eccbytes = 40,
- .eccpos = {
- /* at the end of spare sector */
- 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,
- 34, 35, 36, 37, 38, 39, 40, 41, 42, 43,
- 44, 45, 46, 47, 48, 49, 50, 51, 52, 53,
- 54, 55, 56, 57, 58, 59, 60, 61, 62, 63,
- },
- .oobfree = {
- /* 2 bytes at offset 0 hold manufacturer badblock markers */
- {.offset = 2, .length = 22, },
- /* 5 bytes at offset 8 hold BBT markers */
- /* 8 bytes at offset 16 hold JFFS2 clean markers */
- },
-};
+ if (!section) {
+ oobregion->offset = 0;
+ oobregion->length = 5;
+ } else if (section == 1) {
+ oobregion->offset = 6;
+ oobregion->length = 2;
+ } else {
+ oobregion->offset = 13;
+ oobregion->length = 3;
+ }
-/*
- * An ECC layout for using 4-bit ECC with large-page (4096bytes) flash,
- * storing ten ECC bytes plus the manufacturer's bad block marker byte,
- * and not overlapping the default BBT markers.
- */
-static struct nand_ecclayout hwecc4_4096 = {
- .eccbytes = 80,
- .eccpos = {
- /* at the end of spare sector */
- 48, 49, 50, 51, 52, 53, 54, 55, 56, 57,
- 58, 59, 60, 61, 62, 63, 64, 65, 66, 67,
- 68, 69, 70, 71, 72, 73, 74, 75, 76, 77,
- 78, 79, 80, 81, 82, 83, 84, 85, 86, 87,
- 88, 89, 90, 91, 92, 93, 94, 95, 96, 97,
- 98, 99, 100, 101, 102, 103, 104, 105, 106, 107,
- 108, 109, 110, 111, 112, 113, 114, 115, 116, 117,
- 118, 119, 120, 121, 122, 123, 124, 125, 126, 127,
- },
- .oobfree = {
- /* 2 bytes at offset 0 hold manufacturer badblock markers */
- {.offset = 2, .length = 46, },
- /* 5 bytes at offset 8 hold BBT markers */
- /* 8 bytes at offset 16 hold JFFS2 clean markers */
- },
+ return 0;
+}
+
+static int hwecc4_ooblayout_small_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section > 1)
+ return -ERANGE;
+
+ if (!section) {
+ oobregion->offset = 8;
+ oobregion->length = 5;
+ } else {
+ oobregion->offset = 16;
+ oobregion->length = mtd->oobsize - 16;
+ }
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops hwecc4_small_ooblayout_ops = {
+ .ecc = hwecc4_ooblayout_small_ecc,
+ .free = hwecc4_ooblayout_small_free,
};
#if defined(CONFIG_OF)
@@ -577,8 +558,6 @@
"ti,davinci-mask-chipsel", &prop))
pdata->mask_chipsel = prop;
if (!of_property_read_string(pdev->dev.of_node,
- "nand-ecc-mode", &mode) ||
- !of_property_read_string(pdev->dev.of_node,
"ti,davinci-ecc-mode", &mode)) {
if (!strncmp("none", mode, 4))
pdata->ecc_mode = NAND_ECC_NONE;
@@ -591,14 +570,11 @@
"ti,davinci-ecc-bits", &prop))
pdata->ecc_bits = prop;
- prop = of_get_nand_bus_width(pdev->dev.of_node);
- if (0 < prop || !of_property_read_u32(pdev->dev.of_node,
- "ti,davinci-nand-buswidth", &prop))
- if (prop == 16)
- pdata->options |= NAND_BUSWIDTH_16;
+ if (!of_property_read_u32(pdev->dev.of_node,
+ "ti,davinci-nand-buswidth", &prop) && prop == 16)
+ pdata->options |= NAND_BUSWIDTH_16;
+
if (of_property_read_bool(pdev->dev.of_node,
- "nand-on-flash-bbt") ||
- of_property_read_bool(pdev->dev.of_node,
"ti,davinci-nand-use-bbt"))
pdata->bbt_options = NAND_BBT_USE_FLASH;
@@ -628,7 +604,6 @@
void __iomem *base;
int ret;
uint32_t val;
- nand_ecc_modes_t ecc_mode;
struct mtd_info *mtd;
pdata = nand_davinci_get_pdata(pdev);
@@ -712,13 +687,53 @@
info->chip.write_buf = nand_davinci_write_buf;
/* Use board-specific ECC config */
- ecc_mode = pdata->ecc_mode;
+ info->chip.ecc.mode = pdata->ecc_mode;
ret = -EINVAL;
- switch (ecc_mode) {
+
+ info->clk = devm_clk_get(&pdev->dev, "aemif");
+ if (IS_ERR(info->clk)) {
+ ret = PTR_ERR(info->clk);
+ dev_dbg(&pdev->dev, "unable to get AEMIF clock, err %d\n", ret);
+ return ret;
+ }
+
+ ret = clk_prepare_enable(info->clk);
+ if (ret < 0) {
+ dev_dbg(&pdev->dev, "unable to enable AEMIF clock, err %d\n",
+ ret);
+ goto err_clk_enable;
+ }
+
+ spin_lock_irq(&davinci_nand_lock);
+
+ /* put CSxNAND into NAND mode */
+ val = davinci_nand_readl(info, NANDFCR_OFFSET);
+ val |= BIT(info->core_chipsel);
+ davinci_nand_writel(info, NANDFCR_OFFSET, val);
+
+ spin_unlock_irq(&davinci_nand_lock);
+
+ /* Scan to find existence of the device(s) */
+ ret = nand_scan_ident(mtd, pdata->mask_chipsel ? 2 : 1, NULL);
+ if (ret < 0) {
+ dev_dbg(&pdev->dev, "no NAND chip(s) found\n");
+ goto err;
+ }
+
+ switch (info->chip.ecc.mode) {
case NAND_ECC_NONE:
+ pdata->ecc_bits = 0;
+ break;
case NAND_ECC_SOFT:
pdata->ecc_bits = 0;
+ /*
+ * This driver expects Hamming based ECC when ecc_mode is set
+ * to NAND_ECC_SOFT. Force ecc.algo to NAND_ECC_HAMMING to
+ * avoid adding an extra ->ecc_algo field to
+ * davinci_nand_pdata.
+ */
+ info->chip.ecc.algo = NAND_ECC_HAMMING;
break;
case NAND_ECC_HW:
if (pdata->ecc_bits == 4) {
@@ -754,37 +769,6 @@
default:
return -EINVAL;
}
- info->chip.ecc.mode = ecc_mode;
-
- info->clk = devm_clk_get(&pdev->dev, "aemif");
- if (IS_ERR(info->clk)) {
- ret = PTR_ERR(info->clk);
- dev_dbg(&pdev->dev, "unable to get AEMIF clock, err %d\n", ret);
- return ret;
- }
-
- ret = clk_prepare_enable(info->clk);
- if (ret < 0) {
- dev_dbg(&pdev->dev, "unable to enable AEMIF clock, err %d\n",
- ret);
- goto err_clk_enable;
- }
-
- spin_lock_irq(&davinci_nand_lock);
-
- /* put CSxNAND into NAND mode */
- val = davinci_nand_readl(info, NANDFCR_OFFSET);
- val |= BIT(info->core_chipsel);
- davinci_nand_writel(info, NANDFCR_OFFSET, val);
-
- spin_unlock_irq(&davinci_nand_lock);
-
- /* Scan to find existence of the device(s) */
- ret = nand_scan_ident(mtd, pdata->mask_chipsel ? 2 : 1, NULL);
- if (ret < 0) {
- dev_dbg(&pdev->dev, "no NAND chip(s) found\n");
- goto err;
- }
/* Update ECC layout if needed ... for 1-bit HW ECC, the default
* is OK, but it allocates 6 bytes when only 3 are needed (for
@@ -805,26 +789,14 @@
* table marker fits in the free bytes.
*/
if (chunks == 1) {
- info->ecclayout = hwecc4_small;
- info->ecclayout.oobfree[1].length = mtd->oobsize - 16;
- goto syndrome_done;
- }
- if (chunks == 4) {
- info->ecclayout = hwecc4_2048;
+ mtd_set_ooblayout(mtd, &hwecc4_small_ooblayout_ops);
+ } else if (chunks == 4 || chunks == 8) {
+ mtd_set_ooblayout(mtd, &nand_ooblayout_lp_ops);
info->chip.ecc.mode = NAND_ECC_HW_OOB_FIRST;
- goto syndrome_done;
+ } else {
+ ret = -EIO;
+ goto err;
}
- if (chunks == 8) {
- info->ecclayout = hwecc4_4096;
- info->chip.ecc.mode = NAND_ECC_HW_OOB_FIRST;
- goto syndrome_done;
- }
-
- ret = -EIO;
- goto err;
-
-syndrome_done:
- info->chip.ecc.layout = &info->ecclayout;
}
ret = nand_scan_tail(mtd);
@@ -850,7 +822,7 @@
err_clk_enable:
spin_lock_irq(&davinci_nand_lock);
- if (ecc_mode == NAND_ECC_HW_SYNDROME)
+ if (info->chip.ecc.mode == NAND_ECC_HW_SYNDROME)
ecc4_busy = false;
spin_unlock_irq(&davinci_nand_lock);
return ret;
diff --git a/drivers/mtd/nand/denali.c b/drivers/mtd/nand/denali.c
index 30bf5f6..0476ae8 100644
--- a/drivers/mtd/nand/denali.c
+++ b/drivers/mtd/nand/denali.c
@@ -1374,13 +1374,41 @@
* correction
*/
#define ECC_8BITS 14
-static struct nand_ecclayout nand_8bit_oob = {
- .eccbytes = 14,
-};
-
#define ECC_15BITS 26
-static struct nand_ecclayout nand_15bit_oob = {
- .eccbytes = 26,
+
+static int denali_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct denali_nand_info *denali = mtd_to_denali(mtd);
+ struct nand_chip *chip = mtd_to_nand(mtd);
+
+ if (section)
+ return -ERANGE;
+
+ oobregion->offset = denali->bbtskipbytes;
+ oobregion->length = chip->ecc.total;
+
+ return 0;
+}
+
+static int denali_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct denali_nand_info *denali = mtd_to_denali(mtd);
+ struct nand_chip *chip = mtd_to_nand(mtd);
+
+ if (section)
+ return -ERANGE;
+
+ oobregion->offset = chip->ecc.total + denali->bbtskipbytes;
+ oobregion->length = mtd->oobsize - oobregion->offset;
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops denali_ooblayout_ops = {
+ .ecc = denali_ooblayout_ecc,
+ .free = denali_ooblayout_free,
};
static uint8_t bbt_pattern[] = {'B', 'b', 't', '0' };
@@ -1561,7 +1589,6 @@
ECC_SECTOR_SIZE)))) {
/* if MLC OOB size is large enough, use 15bit ECC*/
denali->nand.ecc.strength = 15;
- denali->nand.ecc.layout = &nand_15bit_oob;
denali->nand.ecc.bytes = ECC_15BITS;
iowrite32(15, denali->flash_reg + ECC_CORRECTION);
} else if (mtd->oobsize < (denali->bbtskipbytes +
@@ -1571,20 +1598,13 @@
goto failed_req_irq;
} else {
denali->nand.ecc.strength = 8;
- denali->nand.ecc.layout = &nand_8bit_oob;
denali->nand.ecc.bytes = ECC_8BITS;
iowrite32(8, denali->flash_reg + ECC_CORRECTION);
}
+ mtd_set_ooblayout(mtd, &denali_ooblayout_ops);
denali->nand.ecc.bytes *= denali->devnum;
denali->nand.ecc.strength *= denali->devnum;
- denali->nand.ecc.layout->eccbytes *=
- mtd->writesize / ECC_SECTOR_SIZE;
- denali->nand.ecc.layout->oobfree[0].offset =
- denali->bbtskipbytes + denali->nand.ecc.layout->eccbytes;
- denali->nand.ecc.layout->oobfree[0].length =
- mtd->oobsize - denali->nand.ecc.layout->eccbytes -
- denali->bbtskipbytes;
/*
* Let driver know the total blocks number and how many blocks
diff --git a/drivers/mtd/nand/diskonchip.c b/drivers/mtd/nand/diskonchip.c
index 547c100..a023ab9 100644
--- a/drivers/mtd/nand/diskonchip.c
+++ b/drivers/mtd/nand/diskonchip.c
@@ -950,20 +950,50 @@
//u_char mydatabuf[528];
-/* The strange out-of-order .oobfree list below is a (possibly unneeded)
- * attempt to retain compatibility. It used to read:
- * .oobfree = { {8, 8} }
- * Since that leaves two bytes unusable, it was changed. But the following
- * scheme might affect existing jffs2 installs by moving the cleanmarker:
- * .oobfree = { {6, 10} }
- * jffs2 seems to handle the above gracefully, but the current scheme seems
- * safer. The only problem with it is that any code that parses oobfree must
- * be able to handle out-of-order segments.
- */
-static struct nand_ecclayout doc200x_oobinfo = {
- .eccbytes = 6,
- .eccpos = {0, 1, 2, 3, 4, 5},
- .oobfree = {{8, 8}, {6, 2}}
+static int doc200x_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section)
+ return -ERANGE;
+
+ oobregion->offset = 0;
+ oobregion->length = 6;
+
+ return 0;
+}
+
+static int doc200x_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section > 1)
+ return -ERANGE;
+
+ /*
+ * The strange out-of-order free bytes definition is a (possibly
+ * unneeded) attempt to retain compatibility. It used to read:
+ * .oobfree = { {8, 8} }
+ * Since that leaves two bytes unusable, it was changed. But the
+ * following scheme might affect existing jffs2 installs by moving the
+ * cleanmarker:
+ * .oobfree = { {6, 10} }
+ * jffs2 seems to handle the above gracefully, but the current scheme
+ * seems safer. The only problem with it is that any code retrieving
+ * free bytes position must be able to handle out-of-order segments.
+ */
+ if (!section) {
+ oobregion->offset = 8;
+ oobregion->length = 8;
+ } else {
+ oobregion->offset = 6;
+ oobregion->length = 2;
+ }
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops doc200x_ooblayout_ops = {
+ .ecc = doc200x_ooblayout_ecc,
+ .free = doc200x_ooblayout_free,
};
/* Find the (I)NFTL Media Header, and optionally also the mirror media header.
@@ -1537,6 +1567,7 @@
nand->bbt_md = nand->bbt_td + 1;
mtd->owner = THIS_MODULE;
+ mtd_set_ooblayout(mtd, &doc200x_ooblayout_ops);
nand_set_controller_data(nand, doc);
nand->select_chip = doc200x_select_chip;
@@ -1548,7 +1579,6 @@
nand->ecc.calculate = doc200x_calculate_ecc;
nand->ecc.correct = doc200x_correct_data;
- nand->ecc.layout = &doc200x_oobinfo;
nand->ecc.mode = NAND_ECC_HW_SYNDROME;
nand->ecc.size = 512;
nand->ecc.bytes = 6;
diff --git a/drivers/mtd/nand/docg4.c b/drivers/mtd/nand/docg4.c
index d86a60e..4731699 100644
--- a/drivers/mtd/nand/docg4.c
+++ b/drivers/mtd/nand/docg4.c
@@ -222,10 +222,33 @@
* Bytes 8 - 14 are hw-generated ecc covering entire page + oob bytes 0 - 14.
* Byte 15 (the last) is used by the driver as a "page written" flag.
*/
-static struct nand_ecclayout docg4_oobinfo = {
- .eccbytes = 9,
- .eccpos = {7, 8, 9, 10, 11, 12, 13, 14, 15},
- .oobfree = { {.offset = 2, .length = 5} }
+static int docg4_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section)
+ return -ERANGE;
+
+ oobregion->offset = 7;
+ oobregion->length = 9;
+
+ return 0;
+}
+
+static int docg4_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section)
+ return -ERANGE;
+
+ oobregion->offset = 2;
+ oobregion->length = 5;
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops docg4_ooblayout_ops = {
+ .ecc = docg4_ooblayout_ecc,
+ .free = docg4_ooblayout_free,
};
/*
@@ -1209,6 +1232,7 @@
mtd->writesize = DOCG4_PAGE_SIZE;
mtd->erasesize = DOCG4_BLOCK_SIZE;
mtd->oobsize = DOCG4_OOB_SIZE;
+ mtd_set_ooblayout(mtd, &docg4_ooblayout_ops);
nand->chipsize = DOCG4_CHIP_SIZE;
nand->chip_shift = DOCG4_CHIP_SHIFT;
nand->bbt_erase_shift = nand->phys_erase_shift = DOCG4_ERASE_SHIFT;
@@ -1217,7 +1241,6 @@
nand->pagemask = 0x3ffff;
nand->badblockpos = NAND_LARGE_BADBLOCK_POS;
nand->badblockbits = 8;
- nand->ecc.layout = &docg4_oobinfo;
nand->ecc.mode = NAND_ECC_HW_SYNDROME;
nand->ecc.size = DOCG4_PAGE_SIZE;
nand->ecc.prepad = 8;
diff --git a/drivers/mtd/nand/fsl_elbc_nand.c b/drivers/mtd/nand/fsl_elbc_nand.c
index 059d5f7..60a88f2 100644
--- a/drivers/mtd/nand/fsl_elbc_nand.c
+++ b/drivers/mtd/nand/fsl_elbc_nand.c
@@ -79,32 +79,53 @@
/* These map to the positions used by the FCM hardware ECC generator */
-/* Small Page FLASH with FMR[ECCM] = 0 */
-static struct nand_ecclayout fsl_elbc_oob_sp_eccm0 = {
- .eccbytes = 3,
- .eccpos = {6, 7, 8},
- .oobfree = { {0, 5}, {9, 7} },
-};
+static int fsl_elbc_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+ struct fsl_elbc_mtd *priv = nand_get_controller_data(chip);
-/* Small Page FLASH with FMR[ECCM] = 1 */
-static struct nand_ecclayout fsl_elbc_oob_sp_eccm1 = {
- .eccbytes = 3,
- .eccpos = {8, 9, 10},
- .oobfree = { {0, 5}, {6, 2}, {11, 5} },
-};
+ if (section >= chip->ecc.steps)
+ return -ERANGE;
-/* Large Page FLASH with FMR[ECCM] = 0 */
-static struct nand_ecclayout fsl_elbc_oob_lp_eccm0 = {
- .eccbytes = 12,
- .eccpos = {6, 7, 8, 22, 23, 24, 38, 39, 40, 54, 55, 56},
- .oobfree = { {1, 5}, {9, 13}, {25, 13}, {41, 13}, {57, 7} },
-};
+ oobregion->offset = (16 * section) + 6;
+ if (priv->fmr & FMR_ECCM)
+ oobregion->offset += 2;
-/* Large Page FLASH with FMR[ECCM] = 1 */
-static struct nand_ecclayout fsl_elbc_oob_lp_eccm1 = {
- .eccbytes = 12,
- .eccpos = {8, 9, 10, 24, 25, 26, 40, 41, 42, 56, 57, 58},
- .oobfree = { {1, 7}, {11, 13}, {27, 13}, {43, 13}, {59, 5} },
+ oobregion->length = chip->ecc.bytes;
+
+ return 0;
+}
+
+static int fsl_elbc_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+ struct fsl_elbc_mtd *priv = nand_get_controller_data(chip);
+
+ if (section > chip->ecc.steps)
+ return -ERANGE;
+
+ if (!section) {
+ oobregion->offset = 0;
+ if (mtd->writesize > 512)
+ oobregion->offset++;
+ oobregion->length = (priv->fmr & FMR_ECCM) ? 7 : 5;
+ } else {
+ oobregion->offset = (16 * section) -
+ ((priv->fmr & FMR_ECCM) ? 5 : 7);
+ if (section < chip->ecc.steps)
+ oobregion->length = 13;
+ else
+ oobregion->length = mtd->oobsize - oobregion->offset;
+ }
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops fsl_elbc_ooblayout_ops = {
+ .ecc = fsl_elbc_ooblayout_ecc,
+ .free = fsl_elbc_ooblayout_free,
};
/*
@@ -657,8 +678,8 @@
chip->ecc.bytes);
dev_dbg(priv->dev, "fsl_elbc_init: nand->ecc.total = %d\n",
chip->ecc.total);
- dev_dbg(priv->dev, "fsl_elbc_init: nand->ecc.layout = %p\n",
- chip->ecc.layout);
+ dev_dbg(priv->dev, "fsl_elbc_init: mtd->ooblayout = %p\n",
+ mtd->ooblayout);
dev_dbg(priv->dev, "fsl_elbc_init: mtd->flags = %08x\n", mtd->flags);
dev_dbg(priv->dev, "fsl_elbc_init: mtd->size = %lld\n", mtd->size);
dev_dbg(priv->dev, "fsl_elbc_init: mtd->erasesize = %d\n",
@@ -675,14 +696,6 @@
} else if (mtd->writesize == 2048) {
priv->page_size = 1;
setbits32(&lbc->bank[priv->bank].or, OR_FCM_PGS);
- /* adjust ecc setup if needed */
- if ((in_be32(&lbc->bank[priv->bank].br) & BR_DECC) ==
- BR_DECC_CHK_GEN) {
- chip->ecc.size = 512;
- chip->ecc.layout = (priv->fmr & FMR_ECCM) ?
- &fsl_elbc_oob_lp_eccm1 :
- &fsl_elbc_oob_lp_eccm0;
- }
} else {
dev_err(priv->dev,
"fsl_elbc_init: page size %d is not supported\n",
@@ -780,15 +793,14 @@
if ((in_be32(&lbc->bank[priv->bank].br) & BR_DECC) ==
BR_DECC_CHK_GEN) {
chip->ecc.mode = NAND_ECC_HW;
- /* put in small page settings and adjust later if needed */
- chip->ecc.layout = (priv->fmr & FMR_ECCM) ?
- &fsl_elbc_oob_sp_eccm1 : &fsl_elbc_oob_sp_eccm0;
+ mtd_set_ooblayout(mtd, &fsl_elbc_ooblayout_ops);
chip->ecc.size = 512;
chip->ecc.bytes = 3;
chip->ecc.strength = 1;
} else {
/* otherwise fall back to default software ECC */
chip->ecc.mode = NAND_ECC_SOFT;
+ chip->ecc.algo = NAND_ECC_HAMMING;
}
return 0;
diff --git a/drivers/mtd/nand/fsl_ifc_nand.c b/drivers/mtd/nand/fsl_ifc_nand.c
index 43f5a3a..4e9e5fd 100644
--- a/drivers/mtd/nand/fsl_ifc_nand.c
+++ b/drivers/mtd/nand/fsl_ifc_nand.c
@@ -67,136 +67,6 @@
static struct fsl_ifc_nand_ctrl *ifc_nand_ctrl;
-/* 512-byte page with 4-bit ECC, 8-bit */
-static struct nand_ecclayout oob_512_8bit_ecc4 = {
- .eccbytes = 8,
- .eccpos = {8, 9, 10, 11, 12, 13, 14, 15},
- .oobfree = { {0, 5}, {6, 2} },
-};
-
-/* 512-byte page with 4-bit ECC, 16-bit */
-static struct nand_ecclayout oob_512_16bit_ecc4 = {
- .eccbytes = 8,
- .eccpos = {8, 9, 10, 11, 12, 13, 14, 15},
- .oobfree = { {2, 6}, },
-};
-
-/* 2048-byte page size with 4-bit ECC */
-static struct nand_ecclayout oob_2048_ecc4 = {
- .eccbytes = 32,
- .eccpos = {
- 8, 9, 10, 11, 12, 13, 14, 15,
- 16, 17, 18, 19, 20, 21, 22, 23,
- 24, 25, 26, 27, 28, 29, 30, 31,
- 32, 33, 34, 35, 36, 37, 38, 39,
- },
- .oobfree = { {2, 6}, {40, 24} },
-};
-
-/* 4096-byte page size with 4-bit ECC */
-static struct nand_ecclayout oob_4096_ecc4 = {
- .eccbytes = 64,
- .eccpos = {
- 8, 9, 10, 11, 12, 13, 14, 15,
- 16, 17, 18, 19, 20, 21, 22, 23,
- 24, 25, 26, 27, 28, 29, 30, 31,
- 32, 33, 34, 35, 36, 37, 38, 39,
- 40, 41, 42, 43, 44, 45, 46, 47,
- 48, 49, 50, 51, 52, 53, 54, 55,
- 56, 57, 58, 59, 60, 61, 62, 63,
- 64, 65, 66, 67, 68, 69, 70, 71,
- },
- .oobfree = { {2, 6}, {72, 56} },
-};
-
-/* 4096-byte page size with 8-bit ECC -- requires 218-byte OOB */
-static struct nand_ecclayout oob_4096_ecc8 = {
- .eccbytes = 128,
- .eccpos = {
- 8, 9, 10, 11, 12, 13, 14, 15,
- 16, 17, 18, 19, 20, 21, 22, 23,
- 24, 25, 26, 27, 28, 29, 30, 31,
- 32, 33, 34, 35, 36, 37, 38, 39,
- 40, 41, 42, 43, 44, 45, 46, 47,
- 48, 49, 50, 51, 52, 53, 54, 55,
- 56, 57, 58, 59, 60, 61, 62, 63,
- 64, 65, 66, 67, 68, 69, 70, 71,
- 72, 73, 74, 75, 76, 77, 78, 79,
- 80, 81, 82, 83, 84, 85, 86, 87,
- 88, 89, 90, 91, 92, 93, 94, 95,
- 96, 97, 98, 99, 100, 101, 102, 103,
- 104, 105, 106, 107, 108, 109, 110, 111,
- 112, 113, 114, 115, 116, 117, 118, 119,
- 120, 121, 122, 123, 124, 125, 126, 127,
- 128, 129, 130, 131, 132, 133, 134, 135,
- },
- .oobfree = { {2, 6}, {136, 82} },
-};
-
-/* 8192-byte page size with 4-bit ECC */
-static struct nand_ecclayout oob_8192_ecc4 = {
- .eccbytes = 128,
- .eccpos = {
- 8, 9, 10, 11, 12, 13, 14, 15,
- 16, 17, 18, 19, 20, 21, 22, 23,
- 24, 25, 26, 27, 28, 29, 30, 31,
- 32, 33, 34, 35, 36, 37, 38, 39,
- 40, 41, 42, 43, 44, 45, 46, 47,
- 48, 49, 50, 51, 52, 53, 54, 55,
- 56, 57, 58, 59, 60, 61, 62, 63,
- 64, 65, 66, 67, 68, 69, 70, 71,
- 72, 73, 74, 75, 76, 77, 78, 79,
- 80, 81, 82, 83, 84, 85, 86, 87,
- 88, 89, 90, 91, 92, 93, 94, 95,
- 96, 97, 98, 99, 100, 101, 102, 103,
- 104, 105, 106, 107, 108, 109, 110, 111,
- 112, 113, 114, 115, 116, 117, 118, 119,
- 120, 121, 122, 123, 124, 125, 126, 127,
- 128, 129, 130, 131, 132, 133, 134, 135,
- },
- .oobfree = { {2, 6}, {136, 208} },
-};
-
-/* 8192-byte page size with 8-bit ECC -- requires 218-byte OOB */
-static struct nand_ecclayout oob_8192_ecc8 = {
- .eccbytes = 256,
- .eccpos = {
- 8, 9, 10, 11, 12, 13, 14, 15,
- 16, 17, 18, 19, 20, 21, 22, 23,
- 24, 25, 26, 27, 28, 29, 30, 31,
- 32, 33, 34, 35, 36, 37, 38, 39,
- 40, 41, 42, 43, 44, 45, 46, 47,
- 48, 49, 50, 51, 52, 53, 54, 55,
- 56, 57, 58, 59, 60, 61, 62, 63,
- 64, 65, 66, 67, 68, 69, 70, 71,
- 72, 73, 74, 75, 76, 77, 78, 79,
- 80, 81, 82, 83, 84, 85, 86, 87,
- 88, 89, 90, 91, 92, 93, 94, 95,
- 96, 97, 98, 99, 100, 101, 102, 103,
- 104, 105, 106, 107, 108, 109, 110, 111,
- 112, 113, 114, 115, 116, 117, 118, 119,
- 120, 121, 122, 123, 124, 125, 126, 127,
- 128, 129, 130, 131, 132, 133, 134, 135,
- 136, 137, 138, 139, 140, 141, 142, 143,
- 144, 145, 146, 147, 148, 149, 150, 151,
- 152, 153, 154, 155, 156, 157, 158, 159,
- 160, 161, 162, 163, 164, 165, 166, 167,
- 168, 169, 170, 171, 172, 173, 174, 175,
- 176, 177, 178, 179, 180, 181, 182, 183,
- 184, 185, 186, 187, 188, 189, 190, 191,
- 192, 193, 194, 195, 196, 197, 198, 199,
- 200, 201, 202, 203, 204, 205, 206, 207,
- 208, 209, 210, 211, 212, 213, 214, 215,
- 216, 217, 218, 219, 220, 221, 222, 223,
- 224, 225, 226, 227, 228, 229, 230, 231,
- 232, 233, 234, 235, 236, 237, 238, 239,
- 240, 241, 242, 243, 244, 245, 246, 247,
- 248, 249, 250, 251, 252, 253, 254, 255,
- 256, 257, 258, 259, 260, 261, 262, 263,
- },
- .oobfree = { {2, 6}, {264, 80} },
-};
-
/*
* Generic flash bbt descriptors
*/
@@ -223,6 +93,57 @@
.pattern = mirror_pattern,
};
+static int fsl_ifc_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+
+ if (section)
+ return -ERANGE;
+
+ oobregion->offset = 8;
+ oobregion->length = chip->ecc.total;
+
+ return 0;
+}
+
+static int fsl_ifc_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+
+ if (section > 1)
+ return -ERANGE;
+
+ if (mtd->writesize == 512 &&
+ !(chip->options & NAND_BUSWIDTH_16)) {
+ if (!section) {
+ oobregion->offset = 0;
+ oobregion->length = 5;
+ } else {
+ oobregion->offset = 6;
+ oobregion->length = 2;
+ }
+
+ return 0;
+ }
+
+ if (!section) {
+ oobregion->offset = 2;
+ oobregion->length = 6;
+ } else {
+ oobregion->offset = chip->ecc.total + 8;
+ oobregion->length = mtd->oobsize - oobregion->offset;
+ }
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops fsl_ifc_ooblayout_ops = {
+ .ecc = fsl_ifc_ooblayout_ecc,
+ .free = fsl_ifc_ooblayout_free,
+};
+
/*
* Set up the IFC hardware block and page address fields, and the ifc nand
* structure addr field to point to the correct IFC buffer in memory
@@ -232,7 +153,7 @@
struct nand_chip *chip = mtd_to_nand(mtd);
struct fsl_ifc_mtd *priv = nand_get_controller_data(chip);
struct fsl_ifc_ctrl *ctrl = priv->ctrl;
- struct fsl_ifc_regs __iomem *ifc = ctrl->regs;
+ struct fsl_ifc_runtime __iomem *ifc = ctrl->rregs;
int buf_num;
ifc_nand_ctrl->page = page_addr;
@@ -257,18 +178,22 @@
u8 __iomem *addr = priv->vbase + bufnum * (mtd->writesize * 2);
u32 __iomem *mainarea = (u32 __iomem *)addr;
u8 __iomem *oob = addr + mtd->writesize;
- int i;
+ struct mtd_oob_region oobregion = { };
+ int i, section = 0;
for (i = 0; i < mtd->writesize / 4; i++) {
if (__raw_readl(&mainarea[i]) != 0xffffffff)
return 0;
}
- for (i = 0; i < chip->ecc.layout->eccbytes; i++) {
- int pos = chip->ecc.layout->eccpos[i];
+ mtd_ooblayout_ecc(mtd, section++, &oobregion);
+ while (oobregion.length) {
+ for (i = 0; i < oobregion.length; i++) {
+ if (__raw_readb(&oob[oobregion.offset + i]) != 0xff)
+ return 0;
+ }
- if (__raw_readb(&oob[pos]) != 0xff)
- return 0;
+ mtd_ooblayout_ecc(mtd, section++, &oobregion);
}
return 1;
@@ -295,7 +220,7 @@
struct fsl_ifc_mtd *priv = nand_get_controller_data(chip);
struct fsl_ifc_ctrl *ctrl = priv->ctrl;
struct fsl_ifc_nand_ctrl *nctrl = ifc_nand_ctrl;
- struct fsl_ifc_regs __iomem *ifc = ctrl->regs;
+ struct fsl_ifc_runtime __iomem *ifc = ctrl->rregs;
u32 eccstat[4];
int i;
@@ -371,7 +296,7 @@
{
struct fsl_ifc_mtd *priv = nand_get_controller_data(chip);
struct fsl_ifc_ctrl *ctrl = priv->ctrl;
- struct fsl_ifc_regs __iomem *ifc = ctrl->regs;
+ struct fsl_ifc_runtime __iomem *ifc = ctrl->rregs;
/* Program FIR/IFC_NAND_FCR0 for Small/Large page */
if (mtd->writesize > 512) {
@@ -411,7 +336,7 @@
struct nand_chip *chip = mtd_to_nand(mtd);
struct fsl_ifc_mtd *priv = nand_get_controller_data(chip);
struct fsl_ifc_ctrl *ctrl = priv->ctrl;
- struct fsl_ifc_regs __iomem *ifc = ctrl->regs;
+ struct fsl_ifc_runtime __iomem *ifc = ctrl->rregs;
/* clear the read buffer */
ifc_nand_ctrl->read_bytes = 0;
@@ -723,7 +648,7 @@
{
struct fsl_ifc_mtd *priv = nand_get_controller_data(chip);
struct fsl_ifc_ctrl *ctrl = priv->ctrl;
- struct fsl_ifc_regs __iomem *ifc = ctrl->regs;
+ struct fsl_ifc_runtime __iomem *ifc = ctrl->rregs;
u32 nand_fsr;
/* Use READ_STATUS command, but wait for the device to be ready */
@@ -808,8 +733,8 @@
chip->ecc.bytes);
dev_dbg(priv->dev, "%s: nand->ecc.total = %d\n", __func__,
chip->ecc.total);
- dev_dbg(priv->dev, "%s: nand->ecc.layout = %p\n", __func__,
- chip->ecc.layout);
+ dev_dbg(priv->dev, "%s: mtd->ooblayout = %p\n", __func__,
+ mtd->ooblayout);
dev_dbg(priv->dev, "%s: mtd->flags = %08x\n", __func__, mtd->flags);
dev_dbg(priv->dev, "%s: mtd->size = %lld\n", __func__, mtd->size);
dev_dbg(priv->dev, "%s: mtd->erasesize = %d\n", __func__,
@@ -825,39 +750,42 @@
static void fsl_ifc_sram_init(struct fsl_ifc_mtd *priv)
{
struct fsl_ifc_ctrl *ctrl = priv->ctrl;
- struct fsl_ifc_regs __iomem *ifc = ctrl->regs;
+ struct fsl_ifc_runtime __iomem *ifc_runtime = ctrl->rregs;
+ struct fsl_ifc_global __iomem *ifc_global = ctrl->gregs;
uint32_t csor = 0, csor_8k = 0, csor_ext = 0;
uint32_t cs = priv->bank;
/* Save CSOR and CSOR_ext */
- csor = ifc_in32(&ifc->csor_cs[cs].csor);
- csor_ext = ifc_in32(&ifc->csor_cs[cs].csor_ext);
+ csor = ifc_in32(&ifc_global->csor_cs[cs].csor);
+ csor_ext = ifc_in32(&ifc_global->csor_cs[cs].csor_ext);
/* chage PageSize 8K and SpareSize 1K*/
csor_8k = (csor & ~(CSOR_NAND_PGS_MASK)) | 0x0018C000;
- ifc_out32(csor_8k, &ifc->csor_cs[cs].csor);
- ifc_out32(0x0000400, &ifc->csor_cs[cs].csor_ext);
+ ifc_out32(csor_8k, &ifc_global->csor_cs[cs].csor);
+ ifc_out32(0x0000400, &ifc_global->csor_cs[cs].csor_ext);
/* READID */
ifc_out32((IFC_FIR_OP_CW0 << IFC_NAND_FIR0_OP0_SHIFT) |
- (IFC_FIR_OP_UA << IFC_NAND_FIR0_OP1_SHIFT) |
- (IFC_FIR_OP_RB << IFC_NAND_FIR0_OP2_SHIFT),
- &ifc->ifc_nand.nand_fir0);
+ (IFC_FIR_OP_UA << IFC_NAND_FIR0_OP1_SHIFT) |
+ (IFC_FIR_OP_RB << IFC_NAND_FIR0_OP2_SHIFT),
+ &ifc_runtime->ifc_nand.nand_fir0);
ifc_out32(NAND_CMD_READID << IFC_NAND_FCR0_CMD0_SHIFT,
- &ifc->ifc_nand.nand_fcr0);
- ifc_out32(0x0, &ifc->ifc_nand.row3);
+ &ifc_runtime->ifc_nand.nand_fcr0);
+ ifc_out32(0x0, &ifc_runtime->ifc_nand.row3);
- ifc_out32(0x0, &ifc->ifc_nand.nand_fbcr);
+ ifc_out32(0x0, &ifc_runtime->ifc_nand.nand_fbcr);
/* Program ROW0/COL0 */
- ifc_out32(0x0, &ifc->ifc_nand.row0);
- ifc_out32(0x0, &ifc->ifc_nand.col0);
+ ifc_out32(0x0, &ifc_runtime->ifc_nand.row0);
+ ifc_out32(0x0, &ifc_runtime->ifc_nand.col0);
/* set the chip select for NAND Transaction */
- ifc_out32(cs << IFC_NAND_CSEL_SHIFT, &ifc->ifc_nand.nand_csel);
+ ifc_out32(cs << IFC_NAND_CSEL_SHIFT,
+ &ifc_runtime->ifc_nand.nand_csel);
/* start read seq */
- ifc_out32(IFC_NAND_SEQ_STRT_FIR_STRT, &ifc->ifc_nand.nandseq_strt);
+ ifc_out32(IFC_NAND_SEQ_STRT_FIR_STRT,
+ &ifc_runtime->ifc_nand.nandseq_strt);
/* wait for command complete flag or timeout */
wait_event_timeout(ctrl->nand_wait, ctrl->nand_stat,
@@ -867,17 +795,17 @@
printk(KERN_ERR "fsl-ifc: Failed to Initialise SRAM\n");
/* Restore CSOR and CSOR_ext */
- ifc_out32(csor, &ifc->csor_cs[cs].csor);
- ifc_out32(csor_ext, &ifc->csor_cs[cs].csor_ext);
+ ifc_out32(csor, &ifc_global->csor_cs[cs].csor);
+ ifc_out32(csor_ext, &ifc_global->csor_cs[cs].csor_ext);
}
static int fsl_ifc_chip_init(struct fsl_ifc_mtd *priv)
{
struct fsl_ifc_ctrl *ctrl = priv->ctrl;
- struct fsl_ifc_regs __iomem *ifc = ctrl->regs;
+ struct fsl_ifc_global __iomem *ifc_global = ctrl->gregs;
+ struct fsl_ifc_runtime __iomem *ifc_runtime = ctrl->rregs;
struct nand_chip *chip = &priv->chip;
struct mtd_info *mtd = nand_to_mtd(&priv->chip);
- struct nand_ecclayout *layout;
u32 csor;
/* Fill in fsl_ifc_mtd structure */
@@ -886,7 +814,8 @@
/* fill in nand_chip structure */
/* set up function call table */
- if ((ifc_in32(&ifc->cspr_cs[priv->bank].cspr)) & CSPR_PORT_SIZE_16)
+ if ((ifc_in32(&ifc_global->cspr_cs[priv->bank].cspr))
+ & CSPR_PORT_SIZE_16)
chip->read_byte = fsl_ifc_read_byte16;
else
chip->read_byte = fsl_ifc_read_byte;
@@ -900,13 +829,14 @@
chip->bbt_td = &bbt_main_descr;
chip->bbt_md = &bbt_mirror_descr;
- ifc_out32(0x0, &ifc->ifc_nand.ncfgr);
+ ifc_out32(0x0, &ifc_runtime->ifc_nand.ncfgr);
/* set up nand options */
chip->bbt_options = NAND_BBT_USE_FLASH;
chip->options = NAND_NO_SUBPAGE_WRITE;
- if (ifc_in32(&ifc->cspr_cs[priv->bank].cspr) & CSPR_PORT_SIZE_16) {
+ if (ifc_in32(&ifc_global->cspr_cs[priv->bank].cspr)
+ & CSPR_PORT_SIZE_16) {
chip->read_byte = fsl_ifc_read_byte16;
chip->options |= NAND_BUSWIDTH_16;
} else {
@@ -919,20 +849,11 @@
chip->ecc.read_page = fsl_ifc_read_page;
chip->ecc.write_page = fsl_ifc_write_page;
- csor = ifc_in32(&ifc->csor_cs[priv->bank].csor);
-
- /* Hardware generates ECC per 512 Bytes */
- chip->ecc.size = 512;
- chip->ecc.bytes = 8;
- chip->ecc.strength = 4;
+ csor = ifc_in32(&ifc_global->csor_cs[priv->bank].csor);
switch (csor & CSOR_NAND_PGS_MASK) {
case CSOR_NAND_PGS_512:
- if (chip->options & NAND_BUSWIDTH_16) {
- layout = &oob_512_16bit_ecc4;
- } else {
- layout = &oob_512_8bit_ecc4;
-
+ if (!(chip->options & NAND_BUSWIDTH_16)) {
/* Avoid conflict with bad block marker */
bbt_main_descr.offs = 0;
bbt_mirror_descr.offs = 0;
@@ -942,35 +863,16 @@
break;
case CSOR_NAND_PGS_2K:
- layout = &oob_2048_ecc4;
priv->bufnum_mask = 3;
break;
case CSOR_NAND_PGS_4K:
- if ((csor & CSOR_NAND_ECC_MODE_MASK) ==
- CSOR_NAND_ECC_MODE_4) {
- layout = &oob_4096_ecc4;
- } else {
- layout = &oob_4096_ecc8;
- chip->ecc.bytes = 16;
- chip->ecc.strength = 8;
- }
-
priv->bufnum_mask = 1;
break;
case CSOR_NAND_PGS_8K:
- if ((csor & CSOR_NAND_ECC_MODE_MASK) ==
- CSOR_NAND_ECC_MODE_4) {
- layout = &oob_8192_ecc4;
- } else {
- layout = &oob_8192_ecc8;
- chip->ecc.bytes = 16;
- chip->ecc.strength = 8;
- }
-
priv->bufnum_mask = 0;
- break;
+ break;
default:
dev_err(priv->dev, "bad csor %#x: bad page size\n", csor);
@@ -980,9 +882,20 @@
/* Must also set CSOR_NAND_ECC_ENC_EN if DEC_EN set */
if (csor & CSOR_NAND_ECC_DEC_EN) {
chip->ecc.mode = NAND_ECC_HW;
- chip->ecc.layout = layout;
+ mtd_set_ooblayout(mtd, &fsl_ifc_ooblayout_ops);
+
+ /* Hardware generates ECC per 512 Bytes */
+ chip->ecc.size = 512;
+ if ((csor & CSOR_NAND_ECC_MODE_MASK) == CSOR_NAND_ECC_MODE_4) {
+ chip->ecc.bytes = 8;
+ chip->ecc.strength = 4;
+ } else {
+ chip->ecc.bytes = 16;
+ chip->ecc.strength = 8;
+ }
} else {
chip->ecc.mode = NAND_ECC_SOFT;
+ chip->ecc.algo = NAND_ECC_HAMMING;
}
if (ctrl->version == FSL_IFC_VERSION_1_1_0)
@@ -1007,10 +920,10 @@
return 0;
}
-static int match_bank(struct fsl_ifc_regs __iomem *ifc, int bank,
+static int match_bank(struct fsl_ifc_global __iomem *ifc_global, int bank,
phys_addr_t addr)
{
- u32 cspr = ifc_in32(&ifc->cspr_cs[bank].cspr);
+ u32 cspr = ifc_in32(&ifc_global->cspr_cs[bank].cspr);
if (!(cspr & CSPR_V))
return 0;
@@ -1024,7 +937,7 @@
static int fsl_ifc_nand_probe(struct platform_device *dev)
{
- struct fsl_ifc_regs __iomem *ifc;
+ struct fsl_ifc_runtime __iomem *ifc;
struct fsl_ifc_mtd *priv;
struct resource res;
static const char *part_probe_types[]
@@ -1034,9 +947,9 @@
struct device_node *node = dev->dev.of_node;
struct mtd_info *mtd;
- if (!fsl_ifc_ctrl_dev || !fsl_ifc_ctrl_dev->regs)
+ if (!fsl_ifc_ctrl_dev || !fsl_ifc_ctrl_dev->rregs)
return -ENODEV;
- ifc = fsl_ifc_ctrl_dev->regs;
+ ifc = fsl_ifc_ctrl_dev->rregs;
/* get, allocate and map the memory resource */
ret = of_address_to_resource(node, 0, &res);
@@ -1047,7 +960,7 @@
/* find which chip select it is connected to */
for (bank = 0; bank < fsl_ifc_ctrl_dev->banks; bank++) {
- if (match_bank(ifc, bank, res.start))
+ if (match_bank(fsl_ifc_ctrl_dev->gregs, bank, res.start))
break;
}
diff --git a/drivers/mtd/nand/fsl_upm.c b/drivers/mtd/nand/fsl_upm.c
index cafd12d..d85fa25 100644
--- a/drivers/mtd/nand/fsl_upm.c
+++ b/drivers/mtd/nand/fsl_upm.c
@@ -170,6 +170,7 @@
fun->chip.read_buf = fun_read_buf;
fun->chip.write_buf = fun_write_buf;
fun->chip.ecc.mode = NAND_ECC_SOFT;
+ fun->chip.ecc.algo = NAND_ECC_HAMMING;
if (fun->mchip_count > 1)
fun->chip.select_chip = fun_select_chip;
diff --git a/drivers/mtd/nand/fsmc_nand.c b/drivers/mtd/nand/fsmc_nand.c
index 1bdcd4f..d4f454a 100644
--- a/drivers/mtd/nand/fsmc_nand.c
+++ b/drivers/mtd/nand/fsmc_nand.c
@@ -39,210 +39,41 @@
#include <linux/amba/bus.h>
#include <mtd/mtd-abi.h>
-static struct nand_ecclayout fsmc_ecc1_128_layout = {
- .eccbytes = 24,
- .eccpos = {2, 3, 4, 18, 19, 20, 34, 35, 36, 50, 51, 52,
- 66, 67, 68, 82, 83, 84, 98, 99, 100, 114, 115, 116},
- .oobfree = {
- {.offset = 8, .length = 8},
- {.offset = 24, .length = 8},
- {.offset = 40, .length = 8},
- {.offset = 56, .length = 8},
- {.offset = 72, .length = 8},
- {.offset = 88, .length = 8},
- {.offset = 104, .length = 8},
- {.offset = 120, .length = 8}
- }
-};
+static int fsmc_ecc1_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
-static struct nand_ecclayout fsmc_ecc1_64_layout = {
- .eccbytes = 12,
- .eccpos = {2, 3, 4, 18, 19, 20, 34, 35, 36, 50, 51, 52},
- .oobfree = {
- {.offset = 8, .length = 8},
- {.offset = 24, .length = 8},
- {.offset = 40, .length = 8},
- {.offset = 56, .length = 8},
- }
-};
+ if (section >= chip->ecc.steps)
+ return -ERANGE;
-static struct nand_ecclayout fsmc_ecc1_16_layout = {
- .eccbytes = 3,
- .eccpos = {2, 3, 4},
- .oobfree = {
- {.offset = 8, .length = 8},
- }
-};
+ oobregion->offset = (section * 16) + 2;
+ oobregion->length = 3;
-/*
- * ECC4 layout for NAND of pagesize 8192 bytes & OOBsize 256 bytes. 13*16 bytes
- * of OB size is reserved for ECC, Byte no. 0 & 1 reserved for bad block and 46
- * bytes are free for use.
- */
-static struct nand_ecclayout fsmc_ecc4_256_layout = {
- .eccbytes = 208,
- .eccpos = { 2, 3, 4, 5, 6, 7, 8,
- 9, 10, 11, 12, 13, 14,
- 18, 19, 20, 21, 22, 23, 24,
- 25, 26, 27, 28, 29, 30,
- 34, 35, 36, 37, 38, 39, 40,
- 41, 42, 43, 44, 45, 46,
- 50, 51, 52, 53, 54, 55, 56,
- 57, 58, 59, 60, 61, 62,
- 66, 67, 68, 69, 70, 71, 72,
- 73, 74, 75, 76, 77, 78,
- 82, 83, 84, 85, 86, 87, 88,
- 89, 90, 91, 92, 93, 94,
- 98, 99, 100, 101, 102, 103, 104,
- 105, 106, 107, 108, 109, 110,
- 114, 115, 116, 117, 118, 119, 120,
- 121, 122, 123, 124, 125, 126,
- 130, 131, 132, 133, 134, 135, 136,
- 137, 138, 139, 140, 141, 142,
- 146, 147, 148, 149, 150, 151, 152,
- 153, 154, 155, 156, 157, 158,
- 162, 163, 164, 165, 166, 167, 168,
- 169, 170, 171, 172, 173, 174,
- 178, 179, 180, 181, 182, 183, 184,
- 185, 186, 187, 188, 189, 190,
- 194, 195, 196, 197, 198, 199, 200,
- 201, 202, 203, 204, 205, 206,
- 210, 211, 212, 213, 214, 215, 216,
- 217, 218, 219, 220, 221, 222,
- 226, 227, 228, 229, 230, 231, 232,
- 233, 234, 235, 236, 237, 238,
- 242, 243, 244, 245, 246, 247, 248,
- 249, 250, 251, 252, 253, 254
- },
- .oobfree = {
- {.offset = 15, .length = 3},
- {.offset = 31, .length = 3},
- {.offset = 47, .length = 3},
- {.offset = 63, .length = 3},
- {.offset = 79, .length = 3},
- {.offset = 95, .length = 3},
- {.offset = 111, .length = 3},
- {.offset = 127, .length = 3},
- {.offset = 143, .length = 3},
- {.offset = 159, .length = 3},
- {.offset = 175, .length = 3},
- {.offset = 191, .length = 3},
- {.offset = 207, .length = 3},
- {.offset = 223, .length = 3},
- {.offset = 239, .length = 3},
- {.offset = 255, .length = 1}
- }
-};
+ return 0;
+}
-/*
- * ECC4 layout for NAND of pagesize 4096 bytes & OOBsize 224 bytes. 13*8 bytes
- * of OOB size is reserved for ECC, Byte no. 0 & 1 reserved for bad block & 118
- * bytes are free for use.
- */
-static struct nand_ecclayout fsmc_ecc4_224_layout = {
- .eccbytes = 104,
- .eccpos = { 2, 3, 4, 5, 6, 7, 8,
- 9, 10, 11, 12, 13, 14,
- 18, 19, 20, 21, 22, 23, 24,
- 25, 26, 27, 28, 29, 30,
- 34, 35, 36, 37, 38, 39, 40,
- 41, 42, 43, 44, 45, 46,
- 50, 51, 52, 53, 54, 55, 56,
- 57, 58, 59, 60, 61, 62,
- 66, 67, 68, 69, 70, 71, 72,
- 73, 74, 75, 76, 77, 78,
- 82, 83, 84, 85, 86, 87, 88,
- 89, 90, 91, 92, 93, 94,
- 98, 99, 100, 101, 102, 103, 104,
- 105, 106, 107, 108, 109, 110,
- 114, 115, 116, 117, 118, 119, 120,
- 121, 122, 123, 124, 125, 126
- },
- .oobfree = {
- {.offset = 15, .length = 3},
- {.offset = 31, .length = 3},
- {.offset = 47, .length = 3},
- {.offset = 63, .length = 3},
- {.offset = 79, .length = 3},
- {.offset = 95, .length = 3},
- {.offset = 111, .length = 3},
- {.offset = 127, .length = 97}
- }
-};
+static int fsmc_ecc1_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
-/*
- * ECC4 layout for NAND of pagesize 4096 bytes & OOBsize 128 bytes. 13*8 bytes
- * of OOB size is reserved for ECC, Byte no. 0 & 1 reserved for bad block & 22
- * bytes are free for use.
- */
-static struct nand_ecclayout fsmc_ecc4_128_layout = {
- .eccbytes = 104,
- .eccpos = { 2, 3, 4, 5, 6, 7, 8,
- 9, 10, 11, 12, 13, 14,
- 18, 19, 20, 21, 22, 23, 24,
- 25, 26, 27, 28, 29, 30,
- 34, 35, 36, 37, 38, 39, 40,
- 41, 42, 43, 44, 45, 46,
- 50, 51, 52, 53, 54, 55, 56,
- 57, 58, 59, 60, 61, 62,
- 66, 67, 68, 69, 70, 71, 72,
- 73, 74, 75, 76, 77, 78,
- 82, 83, 84, 85, 86, 87, 88,
- 89, 90, 91, 92, 93, 94,
- 98, 99, 100, 101, 102, 103, 104,
- 105, 106, 107, 108, 109, 110,
- 114, 115, 116, 117, 118, 119, 120,
- 121, 122, 123, 124, 125, 126
- },
- .oobfree = {
- {.offset = 15, .length = 3},
- {.offset = 31, .length = 3},
- {.offset = 47, .length = 3},
- {.offset = 63, .length = 3},
- {.offset = 79, .length = 3},
- {.offset = 95, .length = 3},
- {.offset = 111, .length = 3},
- {.offset = 127, .length = 1}
- }
-};
+ if (section >= chip->ecc.steps)
+ return -ERANGE;
-/*
- * ECC4 layout for NAND of pagesize 2048 bytes & OOBsize 64 bytes. 13*4 bytes of
- * OOB size is reserved for ECC, Byte no. 0 & 1 reserved for bad block and 10
- * bytes are free for use.
- */
-static struct nand_ecclayout fsmc_ecc4_64_layout = {
- .eccbytes = 52,
- .eccpos = { 2, 3, 4, 5, 6, 7, 8,
- 9, 10, 11, 12, 13, 14,
- 18, 19, 20, 21, 22, 23, 24,
- 25, 26, 27, 28, 29, 30,
- 34, 35, 36, 37, 38, 39, 40,
- 41, 42, 43, 44, 45, 46,
- 50, 51, 52, 53, 54, 55, 56,
- 57, 58, 59, 60, 61, 62,
- },
- .oobfree = {
- {.offset = 15, .length = 3},
- {.offset = 31, .length = 3},
- {.offset = 47, .length = 3},
- {.offset = 63, .length = 1},
- }
-};
+ oobregion->offset = (section * 16) + 8;
-/*
- * ECC4 layout for NAND of pagesize 512 bytes & OOBsize 16 bytes. 13 bytes of
- * OOB size is reserved for ECC, Byte no. 4 & 5 reserved for bad block and One
- * byte is free for use.
- */
-static struct nand_ecclayout fsmc_ecc4_16_layout = {
- .eccbytes = 13,
- .eccpos = { 0, 1, 2, 3, 6, 7, 8,
- 9, 10, 11, 12, 13, 14
- },
- .oobfree = {
- {.offset = 15, .length = 1},
- }
+ if (section < chip->ecc.steps - 1)
+ oobregion->length = 8;
+ else
+ oobregion->length = mtd->oobsize - oobregion->offset;
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops fsmc_ecc1_ooblayout_ops = {
+ .ecc = fsmc_ecc1_ooblayout_ecc,
+ .free = fsmc_ecc1_ooblayout_free,
};
/*
@@ -250,28 +81,46 @@
* There are 13 bytes of ecc for every 512 byte block and it has to be read
* consecutively and immediately after the 512 byte data block for hardware to
* generate the error bit offsets in 512 byte data.
- * Managing the ecc bytes in the following way makes it easier for software to
- * read ecc bytes consecutive to data bytes. This way is similar to
- * oobfree structure maintained already in generic nand driver
*/
-static struct fsmc_eccplace fsmc_ecc4_lp_place = {
- .eccplace = {
- {.offset = 2, .length = 13},
- {.offset = 18, .length = 13},
- {.offset = 34, .length = 13},
- {.offset = 50, .length = 13},
- {.offset = 66, .length = 13},
- {.offset = 82, .length = 13},
- {.offset = 98, .length = 13},
- {.offset = 114, .length = 13}
- }
-};
+static int fsmc_ecc4_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
-static struct fsmc_eccplace fsmc_ecc4_sp_place = {
- .eccplace = {
- {.offset = 0, .length = 4},
- {.offset = 6, .length = 9}
- }
+ if (section >= chip->ecc.steps)
+ return -ERANGE;
+
+ oobregion->length = chip->ecc.bytes;
+
+ if (!section && mtd->writesize <= 512)
+ oobregion->offset = 0;
+ else
+ oobregion->offset = (section * 16) + 2;
+
+ return 0;
+}
+
+static int fsmc_ecc4_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+
+ if (section >= chip->ecc.steps)
+ return -ERANGE;
+
+ oobregion->offset = (section * 16) + 15;
+
+ if (section < chip->ecc.steps - 1)
+ oobregion->length = 3;
+ else
+ oobregion->length = mtd->oobsize - oobregion->offset;
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops fsmc_ecc4_ooblayout_ops = {
+ .ecc = fsmc_ecc4_ooblayout_ecc,
+ .free = fsmc_ecc4_ooblayout_free,
};
/**
@@ -283,7 +132,6 @@
* @partitions: Partition info for a NAND Flash.
* @nr_partitions: Total number of partition of a NAND flash.
*
- * @ecc_place: ECC placing locations in oobfree type format.
* @bank: Bank number for probed device.
* @clk: Clock structure for FSMC.
*
@@ -303,7 +151,6 @@
struct mtd_partition *partitions;
unsigned int nr_partitions;
- struct fsmc_eccplace *ecc_place;
unsigned int bank;
struct device *dev;
enum access_mode mode;
@@ -710,8 +557,6 @@
static int fsmc_read_page_hwecc(struct mtd_info *mtd, struct nand_chip *chip,
uint8_t *buf, int oob_required, int page)
{
- struct fsmc_nand_data *host = mtd_to_fsmc(mtd);
- struct fsmc_eccplace *ecc_place = host->ecc_place;
int i, j, s, stat, eccsize = chip->ecc.size;
int eccbytes = chip->ecc.bytes;
int eccsteps = chip->ecc.steps;
@@ -734,9 +579,15 @@
chip->read_buf(mtd, p, eccsize);
for (j = 0; j < eccbytes;) {
- off = ecc_place->eccplace[group].offset;
- len = ecc_place->eccplace[group].length;
- group++;
+ struct mtd_oob_region oobregion;
+ int ret;
+
+ ret = mtd_ooblayout_ecc(mtd, group++, &oobregion);
+ if (ret)
+ return ret;
+
+ off = oobregion.offset;
+ len = oobregion.length;
/*
* length is intentionally kept a higher multiple of 2
@@ -1084,24 +935,10 @@
if (AMBA_REV_BITS(host->pid) >= 8) {
switch (mtd->oobsize) {
case 16:
- nand->ecc.layout = &fsmc_ecc4_16_layout;
- host->ecc_place = &fsmc_ecc4_sp_place;
- break;
case 64:
- nand->ecc.layout = &fsmc_ecc4_64_layout;
- host->ecc_place = &fsmc_ecc4_lp_place;
- break;
case 128:
- nand->ecc.layout = &fsmc_ecc4_128_layout;
- host->ecc_place = &fsmc_ecc4_lp_place;
- break;
case 224:
- nand->ecc.layout = &fsmc_ecc4_224_layout;
- host->ecc_place = &fsmc_ecc4_lp_place;
- break;
case 256:
- nand->ecc.layout = &fsmc_ecc4_256_layout;
- host->ecc_place = &fsmc_ecc4_lp_place;
break;
default:
dev_warn(&pdev->dev, "No oob scheme defined for oobsize %d\n",
@@ -1109,6 +946,8 @@
ret = -EINVAL;
goto err_probe;
}
+
+ mtd_set_ooblayout(mtd, &fsmc_ecc4_ooblayout_ops);
} else {
switch (nand->ecc.mode) {
case NAND_ECC_HW:
@@ -1119,9 +958,11 @@
nand->ecc.strength = 1;
break;
- case NAND_ECC_SOFT_BCH:
- dev_info(&pdev->dev, "Using 4-bit SW BCH ECC scheme\n");
- break;
+ case NAND_ECC_SOFT:
+ if (nand->ecc.algo == NAND_ECC_BCH) {
+ dev_info(&pdev->dev, "Using 4-bit SW BCH ECC scheme\n");
+ break;
+ }
default:
dev_err(&pdev->dev, "Unsupported ECC mode!\n");
@@ -1132,16 +973,13 @@
* Don't set layout for BCH4 SW ECC. This will be
* generated later in nand_bch_init() later.
*/
- if (nand->ecc.mode != NAND_ECC_SOFT_BCH) {
+ if (nand->ecc.mode == NAND_ECC_HW) {
switch (mtd->oobsize) {
case 16:
- nand->ecc.layout = &fsmc_ecc1_16_layout;
- break;
case 64:
- nand->ecc.layout = &fsmc_ecc1_64_layout;
- break;
case 128:
- nand->ecc.layout = &fsmc_ecc1_128_layout;
+ mtd_set_ooblayout(mtd,
+ &fsmc_ecc1_ooblayout_ops);
break;
default:
dev_warn(&pdev->dev,
diff --git a/drivers/mtd/nand/gpio.c b/drivers/mtd/nand/gpio.c
index ded658f..6317f68 100644
--- a/drivers/mtd/nand/gpio.c
+++ b/drivers/mtd/nand/gpio.c
@@ -273,6 +273,7 @@
nand_set_flash_node(chip, pdev->dev.of_node);
chip->IO_ADDR_W = chip->IO_ADDR_R;
chip->ecc.mode = NAND_ECC_SOFT;
+ chip->ecc.algo = NAND_ECC_HAMMING;
chip->options = gpiomtd->plat.options;
chip->chip_delay = gpiomtd->plat.chip_delay;
chip->cmd_ctrl = gpio_nand_cmd_ctrl;
diff --git a/drivers/mtd/nand/gpmi-nand/gpmi-nand.c b/drivers/mtd/nand/gpmi-nand/gpmi-nand.c
index 8122c69..6e46156 100644
--- a/drivers/mtd/nand/gpmi-nand/gpmi-nand.c
+++ b/drivers/mtd/nand/gpmi-nand/gpmi-nand.c
@@ -25,7 +25,6 @@
#include <linux/mtd/partitions.h>
#include <linux/of.h>
#include <linux/of_device.h>
-#include <linux/of_mtd.h>
#include "gpmi-nand.h"
#include "bch-regs.h"
@@ -47,10 +46,44 @@
* We may change the layout if we can get the ECC info from the datasheet,
* else we will use all the (page + OOB).
*/
-static struct nand_ecclayout gpmi_hw_ecclayout = {
- .eccbytes = 0,
- .eccpos = { 0, },
- .oobfree = { {.offset = 0, .length = 0} }
+static int gpmi_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+ struct gpmi_nand_data *this = nand_get_controller_data(chip);
+ struct bch_geometry *geo = &this->bch_geometry;
+
+ if (section)
+ return -ERANGE;
+
+ oobregion->offset = 0;
+ oobregion->length = geo->page_size - mtd->writesize;
+
+ return 0;
+}
+
+static int gpmi_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+ struct gpmi_nand_data *this = nand_get_controller_data(chip);
+ struct bch_geometry *geo = &this->bch_geometry;
+
+ if (section)
+ return -ERANGE;
+
+ /* The available oob size we have. */
+ if (geo->page_size < mtd->writesize + mtd->oobsize) {
+ oobregion->offset = geo->page_size - mtd->writesize;
+ oobregion->length = mtd->oobsize - oobregion->offset;
+ }
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops gpmi_ooblayout_ops = {
+ .ecc = gpmi_ooblayout_ecc,
+ .free = gpmi_ooblayout_free,
};
static const struct gpmi_devdata gpmi_devdata_imx23 = {
@@ -141,7 +174,6 @@
struct bch_geometry *geo = &this->bch_geometry;
struct nand_chip *chip = &this->nand;
struct mtd_info *mtd = nand_to_mtd(chip);
- struct nand_oobfree *of = gpmi_hw_ecclayout.oobfree;
unsigned int block_mark_bit_offset;
if (!(chip->ecc_strength_ds > 0 && chip->ecc_step_ds > 0))
@@ -229,12 +261,6 @@
geo->page_size = mtd->writesize + geo->metadata_size +
(geo->gf_len * geo->ecc_strength * geo->ecc_chunk_count) / 8;
- /* The available oob size we have. */
- if (geo->page_size < mtd->writesize + mtd->oobsize) {
- of->offset = geo->page_size - mtd->writesize;
- of->length = mtd->oobsize - of->offset;
- }
-
geo->payload_size = mtd->writesize;
geo->auxiliary_status_offset = ALIGN(geo->metadata_size, 4);
@@ -797,6 +823,7 @@
this->cmd_buffer = NULL;
this->data_buffer_dma = NULL;
+ this->raw_buffer = NULL;
this->page_buffer_virt = NULL;
this->page_buffer_size = 0;
}
@@ -1037,14 +1064,87 @@
/* Loop over status bytes, accumulating ECC status. */
status = auxiliary_virt + nfc_geo->auxiliary_status_offset;
+ read_page_swap_end(this, buf, nfc_geo->payload_size,
+ this->payload_virt, this->payload_phys,
+ nfc_geo->payload_size,
+ payload_virt, payload_phys);
+
for (i = 0; i < nfc_geo->ecc_chunk_count; i++, status++) {
if ((*status == STATUS_GOOD) || (*status == STATUS_ERASED))
continue;
if (*status == STATUS_UNCORRECTABLE) {
+ int eccbits = nfc_geo->ecc_strength * nfc_geo->gf_len;
+ u8 *eccbuf = this->raw_buffer;
+ int offset, bitoffset;
+ int eccbytes;
+ int flips;
+
+ /* Read ECC bytes into our internal raw_buffer */
+ offset = nfc_geo->metadata_size * 8;
+ offset += ((8 * nfc_geo->ecc_chunk_size) + eccbits) * (i + 1);
+ offset -= eccbits;
+ bitoffset = offset % 8;
+ eccbytes = DIV_ROUND_UP(offset + eccbits, 8);
+ offset /= 8;
+ eccbytes -= offset;
+ chip->cmdfunc(mtd, NAND_CMD_RNDOUT, offset, -1);
+ chip->read_buf(mtd, eccbuf, eccbytes);
+
+ /*
+ * ECC data are not byte aligned and we may have
+ * in-band data in the first and last byte of
+ * eccbuf. Set non-eccbits to one so that
+ * nand_check_erased_ecc_chunk() does not count them
+ * as bitflips.
+ */
+ if (bitoffset)
+ eccbuf[0] |= GENMASK(bitoffset - 1, 0);
+
+ bitoffset = (bitoffset + eccbits) % 8;
+ if (bitoffset)
+ eccbuf[eccbytes - 1] |= GENMASK(7, bitoffset);
+
+ /*
+ * The ECC hardware has an uncorrectable ECC status
+ * code in case we have bitflips in an erased page. As
+ * nothing was written into this subpage the ECC is
+ * obviously wrong and we can not trust it. We assume
+ * at this point that we are reading an erased page and
+ * try to correct the bitflips in buffer up to
+ * ecc_strength bitflips. If this is a page with random
+ * data, we exceed this number of bitflips and have a
+ * ECC failure. Otherwise we use the corrected buffer.
+ */
+ if (i == 0) {
+ /* The first block includes metadata */
+ flips = nand_check_erased_ecc_chunk(
+ buf + i * nfc_geo->ecc_chunk_size,
+ nfc_geo->ecc_chunk_size,
+ eccbuf, eccbytes,
+ auxiliary_virt,
+ nfc_geo->metadata_size,
+ nfc_geo->ecc_strength);
+ } else {
+ flips = nand_check_erased_ecc_chunk(
+ buf + i * nfc_geo->ecc_chunk_size,
+ nfc_geo->ecc_chunk_size,
+ eccbuf, eccbytes,
+ NULL, 0,
+ nfc_geo->ecc_strength);
+ }
+
+ if (flips > 0) {
+ max_bitflips = max_t(unsigned int, max_bitflips,
+ flips);
+ mtd->ecc_stats.corrected += flips;
+ continue;
+ }
+
mtd->ecc_stats.failed++;
continue;
}
+
mtd->ecc_stats.corrected += *status;
max_bitflips = max_t(unsigned int, max_bitflips, *status);
}
@@ -1064,11 +1164,6 @@
chip->oob_poi[0] = ((uint8_t *) auxiliary_virt)[0];
}
- read_page_swap_end(this, buf, nfc_geo->payload_size,
- this->payload_virt, this->payload_phys,
- nfc_geo->payload_size,
- payload_virt, payload_phys);
-
return max_bitflips;
}
@@ -1327,18 +1422,19 @@
static int
gpmi_ecc_write_oob(struct mtd_info *mtd, struct nand_chip *chip, int page)
{
- struct nand_oobfree *of = mtd->ecclayout->oobfree;
+ struct mtd_oob_region of = { };
int status = 0;
/* Do we have available oob area? */
- if (!of->length)
+ mtd_ooblayout_free(mtd, 0, &of);
+ if (!of.length)
return -EPERM;
if (!nand_is_slc(chip))
return -EPERM;
- chip->cmdfunc(mtd, NAND_CMD_SEQIN, mtd->writesize + of->offset, page);
- chip->write_buf(mtd, chip->oob_poi + of->offset, of->length);
+ chip->cmdfunc(mtd, NAND_CMD_SEQIN, mtd->writesize + of.offset, page);
+ chip->write_buf(mtd, chip->oob_poi + of.offset, of.length);
chip->cmdfunc(mtd, NAND_CMD_PAGEPROG, -1, -1);
status = chip->waitfunc(mtd, chip);
@@ -1840,6 +1936,7 @@
static int gpmi_init_last(struct gpmi_nand_data *this)
{
struct nand_chip *chip = &this->nand;
+ struct mtd_info *mtd = nand_to_mtd(chip);
struct nand_ecc_ctrl *ecc = &chip->ecc;
struct bch_geometry *bch_geo = &this->bch_geometry;
int ret;
@@ -1861,7 +1958,7 @@
ecc->mode = NAND_ECC_HW;
ecc->size = bch_geo->ecc_chunk_size;
ecc->strength = bch_geo->ecc_strength;
- ecc->layout = &gpmi_hw_ecclayout;
+ mtd_set_ooblayout(mtd, &gpmi_ooblayout_ops);
/*
* We only enable the subpage read when:
@@ -1914,16 +2011,6 @@
/* Set up swap_block_mark, must be set before the gpmi_set_geometry() */
this->swap_block_mark = !GPMI_IS_MX23(this);
- if (of_get_nand_on_flash_bbt(this->dev->of_node)) {
- chip->bbt_options |= NAND_BBT_USE_FLASH | NAND_BBT_NO_OOB;
-
- if (of_property_read_bool(this->dev->of_node,
- "fsl,no-blockmark-swap"))
- this->swap_block_mark = false;
- }
- dev_dbg(this->dev, "Blockmark swapping %sabled\n",
- this->swap_block_mark ? "en" : "dis");
-
/*
* Allocate a temporary DMA buffer for reading ID in the
* nand_scan_ident().
@@ -1938,6 +2025,16 @@
if (ret)
goto err_out;
+ if (chip->bbt_options & NAND_BBT_USE_FLASH) {
+ chip->bbt_options |= NAND_BBT_NO_OOB;
+
+ if (of_property_read_bool(this->dev->of_node,
+ "fsl,no-blockmark-swap"))
+ this->swap_block_mark = false;
+ }
+ dev_dbg(this->dev, "Blockmark swapping %sabled\n",
+ this->swap_block_mark ? "en" : "dis");
+
ret = gpmi_init_last(this);
if (ret)
goto err_out;
diff --git a/drivers/mtd/nand/hisi504_nand.c b/drivers/mtd/nand/hisi504_nand.c
index 96502b6..9432546 100644
--- a/drivers/mtd/nand/hisi504_nand.c
+++ b/drivers/mtd/nand/hisi504_nand.c
@@ -19,7 +19,6 @@
* GNU General Public License for more details.
*/
#include <linux/of.h>
-#include <linux/of_mtd.h>
#include <linux/mtd/mtd.h>
#include <linux/sizes.h>
#include <linux/clk.h>
@@ -631,8 +630,28 @@
hinfc_write(host, HINFC504_INTEN_DMA, HINFC504_INTEN);
}
-static struct nand_ecclayout nand_ecc_2K_16bits = {
- .oobfree = { {2, 6} },
+static int hisi_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ /* FIXME: add ECC bytes position */
+ return -ENOTSUPP;
+}
+
+static int hisi_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section)
+ return -ERANGE;
+
+ oobregion->offset = 2;
+ oobregion->length = 6;
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops hisi_ooblayout_ops = {
+ .ecc = hisi_ooblayout_ecc,
+ .free = hisi_ooblayout_free,
};
static int hisi_nfc_ecc_probe(struct hinfc_host *host)
@@ -642,10 +661,9 @@
struct device *dev = host->dev;
struct nand_chip *chip = &host->chip;
struct mtd_info *mtd = nand_to_mtd(chip);
- struct device_node *np = host->dev->of_node;
- size = of_get_nand_ecc_step_size(np);
- strength = of_get_nand_ecc_strength(np);
+ size = chip->ecc.size;
+ strength = chip->ecc.strength;
if (size != 1024) {
dev_err(dev, "error ecc size: %d\n", size);
return -EINVAL;
@@ -668,7 +686,7 @@
case 16:
ecc_bits = 6;
if (mtd->writesize == 2048)
- chip->ecc.layout = &nand_ecc_2K_16bits;
+ mtd_set_ooblayout(mtd, &hisi_ooblayout_ops);
/* TODO: add more page size support */
break;
@@ -695,7 +713,7 @@
static int hisi_nfc_probe(struct platform_device *pdev)
{
- int ret = 0, irq, buswidth, flag, max_chips = HINFC504_MAX_CHIP;
+ int ret = 0, irq, flag, max_chips = HINFC504_MAX_CHIP;
struct device *dev = &pdev->dev;
struct hinfc_host *host;
struct nand_chip *chip;
@@ -747,12 +765,6 @@
chip->read_buf = hisi_nfc_read_buf;
chip->chip_delay = HINFC504_CHIP_DELAY;
- chip->ecc.mode = of_get_nand_ecc_mode(np);
-
- buswidth = of_get_nand_bus_width(np);
- if (buswidth == 16)
- chip->options |= NAND_BUSWIDTH_16;
-
hisi_nfc_host_init(host);
ret = devm_request_irq(dev, irq, hinfc_irq_handle, 0x0, "nandc", host);
diff --git a/drivers/mtd/nand/jz4740_nand.c b/drivers/mtd/nand/jz4740_nand.c
index 673ceb2..5551c36 100644
--- a/drivers/mtd/nand/jz4740_nand.c
+++ b/drivers/mtd/nand/jz4740_nand.c
@@ -221,7 +221,6 @@
struct jz_nand *nand = mtd_to_jz_nand(mtd);
int i, error_count, index;
uint32_t reg, status, error;
- uint32_t t;
unsigned int timeout = 1000;
for (i = 0; i < 9; ++i)
@@ -476,7 +475,7 @@
}
if (pdata && pdata->ident_callback) {
- pdata->ident_callback(pdev, chip, &pdata->partitions,
+ pdata->ident_callback(pdev, mtd, &pdata->partitions,
&pdata->num_partitions);
}
diff --git a/drivers/mtd/nand/jz4780_bch.c b/drivers/mtd/nand/jz4780_bch.c
index 755499c..d74f4ba 100644
--- a/drivers/mtd/nand/jz4780_bch.c
+++ b/drivers/mtd/nand/jz4780_bch.c
@@ -287,7 +287,6 @@
bch = platform_get_drvdata(pdev);
clk_prepare_enable(bch->clk);
- bch->dev = &pdev->dev;
return bch;
}
diff --git a/drivers/mtd/nand/jz4780_nand.c b/drivers/mtd/nand/jz4780_nand.c
index e1c016c..daf3c42 100644
--- a/drivers/mtd/nand/jz4780_nand.c
+++ b/drivers/mtd/nand/jz4780_nand.c
@@ -17,7 +17,6 @@
#include <linux/of.h>
#include <linux/of_address.h>
#include <linux/gpio/consumer.h>
-#include <linux/of_mtd.h>
#include <linux/platform_device.h>
#include <linux/slab.h>
#include <linux/mtd/mtd.h>
@@ -56,8 +55,6 @@
struct nand_chip chip;
struct list_head chip_list;
- struct nand_ecclayout ecclayout;
-
struct gpio_desc *busy_gpio;
struct gpio_desc *wp_gpio;
unsigned int reading: 1;
@@ -165,8 +162,7 @@
struct nand_chip *chip = &nand->chip;
struct mtd_info *mtd = nand_to_mtd(chip);
struct jz4780_nand_controller *nfc = to_jz4780_nand_controller(chip->controller);
- struct nand_ecclayout *layout = &nand->ecclayout;
- u32 start, i;
+ int eccbytes;
chip->ecc.bytes = fls((1 + 8) * chip->ecc.size) *
(chip->ecc.strength / 8);
@@ -183,7 +179,6 @@
chip->ecc.correct = jz4780_nand_ecc_correct;
/* fall through */
case NAND_ECC_SOFT:
- case NAND_ECC_SOFT_BCH:
dev_info(dev, "using %s (strength %d, size %d, bytes %d)\n",
(nfc->bch) ? "hardware BCH" : "software ECC",
chip->ecc.strength, chip->ecc.size, chip->ecc.bytes);
@@ -201,23 +196,17 @@
return 0;
/* Generate ECC layout. ECC codes are right aligned in the OOB area. */
- layout->eccbytes = mtd->writesize / chip->ecc.size * chip->ecc.bytes;
+ eccbytes = mtd->writesize / chip->ecc.size * chip->ecc.bytes;
- if (layout->eccbytes > mtd->oobsize - 2) {
+ if (eccbytes > mtd->oobsize - 2) {
dev_err(dev,
"invalid ECC config: required %d ECC bytes, but only %d are available",
- layout->eccbytes, mtd->oobsize - 2);
+ eccbytes, mtd->oobsize - 2);
return -EINVAL;
}
- start = mtd->oobsize - layout->eccbytes;
- for (i = 0; i < layout->eccbytes; i++)
- layout->eccpos[i] = start + i;
+ mtd->ooblayout = &nand_ooblayout_lp_ops;
- layout->oobfree[0].offset = 2;
- layout->oobfree[0].length = mtd->oobsize - layout->eccbytes - 2;
-
- chip->ecc.layout = layout;
return 0;
}
diff --git a/drivers/mtd/nand/lpc32xx_mlc.c b/drivers/mtd/nand/lpc32xx_mlc.c
index d8c3e7a..8523881 100644
--- a/drivers/mtd/nand/lpc32xx_mlc.c
+++ b/drivers/mtd/nand/lpc32xx_mlc.c
@@ -35,7 +35,6 @@
#include <linux/completion.h>
#include <linux/interrupt.h>
#include <linux/of.h>
-#include <linux/of_mtd.h>
#include <linux/of_gpio.h>
#include <linux/mtd/lpc32xx_mlc.h>
#include <linux/io.h>
@@ -139,22 +138,37 @@
unsigned num_parts;
};
-static struct nand_ecclayout lpc32xx_nand_oob = {
- .eccbytes = 40,
- .eccpos = { 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,
- 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,
- 38, 39, 40, 41, 42, 43, 44, 45, 46, 47,
- 54, 55, 56, 57, 58, 59, 60, 61, 62, 63 },
- .oobfree = {
- { .offset = 0,
- .length = 6, },
- { .offset = 16,
- .length = 6, },
- { .offset = 32,
- .length = 6, },
- { .offset = 48,
- .length = 6, },
- },
+static int lpc32xx_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *nand_chip = mtd_to_nand(mtd);
+
+ if (section >= nand_chip->ecc.steps)
+ return -ERANGE;
+
+ oobregion->offset = ((section + 1) * 16) - nand_chip->ecc.bytes;
+ oobregion->length = nand_chip->ecc.bytes;
+
+ return 0;
+}
+
+static int lpc32xx_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *nand_chip = mtd_to_nand(mtd);
+
+ if (section >= nand_chip->ecc.steps)
+ return -ERANGE;
+
+ oobregion->offset = 16 * section;
+ oobregion->length = 16 - nand_chip->ecc.bytes;
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops lpc32xx_ooblayout_ops = {
+ .ecc = lpc32xx_ooblayout_ecc,
+ .free = lpc32xx_ooblayout_free,
};
static struct nand_bbt_descr lpc32xx_nand_bbt = {
@@ -713,6 +727,7 @@
nand_chip->ecc.write_oob = lpc32xx_write_oob;
nand_chip->ecc.read_oob = lpc32xx_read_oob;
nand_chip->ecc.strength = 4;
+ nand_chip->ecc.bytes = 10;
nand_chip->waitfunc = lpc32xx_waitfunc;
nand_chip->options = NAND_NO_SUBPAGE_WRITE;
@@ -751,7 +766,7 @@
nand_chip->ecc.mode = NAND_ECC_HW;
nand_chip->ecc.size = 512;
- nand_chip->ecc.layout = &lpc32xx_nand_oob;
+ mtd_set_ooblayout(mtd, &lpc32xx_ooblayout_ops);
host->mlcsubpages = mtd->writesize / 512;
/* initially clear interrupt status */
diff --git a/drivers/mtd/nand/lpc32xx_slc.c b/drivers/mtd/nand/lpc32xx_slc.c
index 3b8f373..8d3edc3 100644
--- a/drivers/mtd/nand/lpc32xx_slc.c
+++ b/drivers/mtd/nand/lpc32xx_slc.c
@@ -35,7 +35,6 @@
#include <linux/mtd/nand_ecc.h>
#include <linux/gpio.h>
#include <linux/of.h>
-#include <linux/of_mtd.h>
#include <linux/of_gpio.h>
#include <linux/mtd/lpc32xx_slc.h>
@@ -146,13 +145,38 @@
* NAND ECC Layout for small page NAND devices
* Note: For large and huge page devices, the default layouts are used
*/
-static struct nand_ecclayout lpc32xx_nand_oob_16 = {
- .eccbytes = 6,
- .eccpos = {10, 11, 12, 13, 14, 15},
- .oobfree = {
- { .offset = 0, .length = 4 },
- { .offset = 6, .length = 4 },
- },
+static int lpc32xx_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section)
+ return -ERANGE;
+
+ oobregion->length = 6;
+ oobregion->offset = 10;
+
+ return 0;
+}
+
+static int lpc32xx_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section > 1)
+ return -ERANGE;
+
+ if (!section) {
+ oobregion->offset = 0;
+ oobregion->length = 4;
+ } else {
+ oobregion->offset = 6;
+ oobregion->length = 4;
+ }
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops lpc32xx_ooblayout_ops = {
+ .ecc = lpc32xx_ooblayout_ecc,
+ .free = lpc32xx_ooblayout_free,
};
static u8 bbt_pattern[] = {'B', 'b', 't', '0' };
@@ -194,7 +218,6 @@
uint32_t rwidth;
uint32_t rhold;
uint32_t rsetup;
- bool use_bbt;
int wp_gpio;
struct mtd_partition *parts;
unsigned num_parts;
@@ -604,7 +627,8 @@
int oob_required, int page)
{
struct lpc32xx_nand_host *host = nand_get_controller_data(chip);
- int stat, i, status;
+ struct mtd_oob_region oobregion = { };
+ int stat, i, status, error;
uint8_t *oobecc, tmpecc[LPC32XX_ECC_SAVE_SIZE];
/* Issue read command */
@@ -620,7 +644,11 @@
lpc32xx_slc_ecc_copy(tmpecc, (uint32_t *) host->ecc_buf, chip->ecc.steps);
/* Pointer to ECC data retrieved from NAND spare area */
- oobecc = chip->oob_poi + chip->ecc.layout->eccpos[0];
+ error = mtd_ooblayout_ecc(mtd, 0, &oobregion);
+ if (error)
+ return error;
+
+ oobecc = chip->oob_poi + oobregion.offset;
for (i = 0; i < chip->ecc.steps; i++) {
stat = chip->ecc.correct(mtd, buf, oobecc,
@@ -666,7 +694,8 @@
int oob_required, int page)
{
struct lpc32xx_nand_host *host = nand_get_controller_data(chip);
- uint8_t *pb = chip->oob_poi + chip->ecc.layout->eccpos[0];
+ struct mtd_oob_region oobregion = { };
+ uint8_t *pb;
int error;
/* Write data, calculate ECC on outbound data */
@@ -678,6 +707,11 @@
* The calculated ECC needs some manual work done to it before
* committing it to NAND. Process the calculated ECC and place
* the resultant values directly into the OOB buffer. */
+ error = mtd_ooblayout_ecc(mtd, 0, &oobregion);
+ if (error)
+ return error;
+
+ pb = chip->oob_poi + oobregion.offset;
lpc32xx_slc_ecc_copy(pb, (uint32_t *)host->ecc_buf, chip->ecc.steps);
/* Write ECC data to device */
@@ -747,7 +781,6 @@
return NULL;
}
- ncfg->use_bbt = of_get_nand_on_flash_bbt(np);
ncfg->wp_gpio = of_get_named_gpio(np, "gpios", 0);
return ncfg;
@@ -875,26 +908,22 @@
* custom BBT marker layout.
*/
if (mtd->writesize <= 512)
- chip->ecc.layout = &lpc32xx_nand_oob_16;
+ mtd_set_ooblayout(mtd, &lpc32xx_ooblayout_ops);
/* These sizes remain the same regardless of page size */
chip->ecc.size = 256;
chip->ecc.bytes = LPC32XX_SLC_DEV_ECC_BYTES;
chip->ecc.prepad = chip->ecc.postpad = 0;
- /* Avoid extra scan if using BBT, setup BBT support */
- if (host->ncfg->use_bbt) {
- chip->bbt_options |= NAND_BBT_USE_FLASH;
-
- /*
- * Use a custom BBT marker setup for small page FLASH that
- * won't interfere with the ECC layout. Large and huge page
- * FLASH use the standard layout.
- */
- if (mtd->writesize <= 512) {
- chip->bbt_td = &bbt_smallpage_main_descr;
- chip->bbt_md = &bbt_smallpage_mirror_descr;
- }
+ /*
+ * Use a custom BBT marker setup for small page FLASH that
+ * won't interfere with the ECC layout. Large and huge page
+ * FLASH use the standard layout.
+ */
+ if ((chip->bbt_options & NAND_BBT_USE_FLASH) &&
+ mtd->writesize <= 512) {
+ chip->bbt_td = &bbt_smallpage_main_descr;
+ chip->bbt_md = &bbt_smallpage_mirror_descr;
}
/*
diff --git a/drivers/mtd/nand/mpc5121_nfc.c b/drivers/mtd/nand/mpc5121_nfc.c
index 5d7843f..7eacb2f 100644
--- a/drivers/mtd/nand/mpc5121_nfc.c
+++ b/drivers/mtd/nand/mpc5121_nfc.c
@@ -710,6 +710,7 @@
chip->select_chip = mpc5121_nfc_select_chip;
chip->bbt_options = NAND_BBT_USE_FLASH;
chip->ecc.mode = NAND_ECC_SOFT;
+ chip->ecc.algo = NAND_ECC_HAMMING;
/* Support external chip-select logic on ADS5121 board */
if (of_machine_is_compatible("fsl,mpc5121ads")) {
diff --git a/drivers/mtd/nand/mxc_nand.c b/drivers/mtd/nand/mxc_nand.c
index 854c832..5173fad 100644
--- a/drivers/mtd/nand/mxc_nand.c
+++ b/drivers/mtd/nand/mxc_nand.c
@@ -34,7 +34,6 @@
#include <linux/completion.h>
#include <linux/of.h>
#include <linux/of_device.h>
-#include <linux/of_mtd.h>
#include <asm/mach/flash.h>
#include <linux/platform_data/mtd-mxc_nand.h>
@@ -149,7 +148,7 @@
int (*check_int)(struct mxc_nand_host *);
void (*irq_control)(struct mxc_nand_host *, int);
u32 (*get_ecc_status)(struct mxc_nand_host *);
- struct nand_ecclayout *ecclayout_512, *ecclayout_2k, *ecclayout_4k;
+ const struct mtd_ooblayout_ops *ooblayout;
void (*select_chip)(struct mtd_info *mtd, int chip);
int (*correct_data)(struct mtd_info *mtd, u_char *dat,
u_char *read_ecc, u_char *calc_ecc);
@@ -200,73 +199,6 @@
struct mxc_nand_platform_data pdata;
};
-/* OOB placement block for use with hardware ecc generation */
-static struct nand_ecclayout nandv1_hw_eccoob_smallpage = {
- .eccbytes = 5,
- .eccpos = {6, 7, 8, 9, 10},
- .oobfree = {{0, 5}, {12, 4}, }
-};
-
-static struct nand_ecclayout nandv1_hw_eccoob_largepage = {
- .eccbytes = 20,
- .eccpos = {6, 7, 8, 9, 10, 22, 23, 24, 25, 26,
- 38, 39, 40, 41, 42, 54, 55, 56, 57, 58},
- .oobfree = {{2, 4}, {11, 10}, {27, 10}, {43, 10}, {59, 5}, }
-};
-
-/* OOB description for 512 byte pages with 16 byte OOB */
-static struct nand_ecclayout nandv2_hw_eccoob_smallpage = {
- .eccbytes = 1 * 9,
- .eccpos = {
- 7, 8, 9, 10, 11, 12, 13, 14, 15
- },
- .oobfree = {
- {.offset = 0, .length = 5}
- }
-};
-
-/* OOB description for 2048 byte pages with 64 byte OOB */
-static struct nand_ecclayout nandv2_hw_eccoob_largepage = {
- .eccbytes = 4 * 9,
- .eccpos = {
- 7, 8, 9, 10, 11, 12, 13, 14, 15,
- 23, 24, 25, 26, 27, 28, 29, 30, 31,
- 39, 40, 41, 42, 43, 44, 45, 46, 47,
- 55, 56, 57, 58, 59, 60, 61, 62, 63
- },
- .oobfree = {
- {.offset = 2, .length = 4},
- {.offset = 16, .length = 7},
- {.offset = 32, .length = 7},
- {.offset = 48, .length = 7}
- }
-};
-
-/* OOB description for 4096 byte pages with 128 byte OOB */
-static struct nand_ecclayout nandv2_hw_eccoob_4k = {
- .eccbytes = 8 * 9,
- .eccpos = {
- 7, 8, 9, 10, 11, 12, 13, 14, 15,
- 23, 24, 25, 26, 27, 28, 29, 30, 31,
- 39, 40, 41, 42, 43, 44, 45, 46, 47,
- 55, 56, 57, 58, 59, 60, 61, 62, 63,
- 71, 72, 73, 74, 75, 76, 77, 78, 79,
- 87, 88, 89, 90, 91, 92, 93, 94, 95,
- 103, 104, 105, 106, 107, 108, 109, 110, 111,
- 119, 120, 121, 122, 123, 124, 125, 126, 127,
- },
- .oobfree = {
- {.offset = 2, .length = 4},
- {.offset = 16, .length = 7},
- {.offset = 32, .length = 7},
- {.offset = 48, .length = 7},
- {.offset = 64, .length = 7},
- {.offset = 80, .length = 7},
- {.offset = 96, .length = 7},
- {.offset = 112, .length = 7},
- }
-};
-
static const char * const part_probes[] = {
"cmdlinepart", "RedBoot", "ofpart", NULL };
@@ -942,6 +874,99 @@
}
}
+static int mxc_v1_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *nand_chip = mtd_to_nand(mtd);
+
+ if (section >= nand_chip->ecc.steps)
+ return -ERANGE;
+
+ oobregion->offset = (section * 16) + 6;
+ oobregion->length = nand_chip->ecc.bytes;
+
+ return 0;
+}
+
+static int mxc_v1_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *nand_chip = mtd_to_nand(mtd);
+
+ if (section > nand_chip->ecc.steps)
+ return -ERANGE;
+
+ if (!section) {
+ if (mtd->writesize <= 512) {
+ oobregion->offset = 0;
+ oobregion->length = 5;
+ } else {
+ oobregion->offset = 2;
+ oobregion->length = 4;
+ }
+ } else {
+ oobregion->offset = ((section - 1) * 16) +
+ nand_chip->ecc.bytes + 6;
+ if (section < nand_chip->ecc.steps)
+ oobregion->length = (section * 16) + 6 -
+ oobregion->offset;
+ else
+ oobregion->length = mtd->oobsize - oobregion->offset;
+ }
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops mxc_v1_ooblayout_ops = {
+ .ecc = mxc_v1_ooblayout_ecc,
+ .free = mxc_v1_ooblayout_free,
+};
+
+static int mxc_v2_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *nand_chip = mtd_to_nand(mtd);
+ int stepsize = nand_chip->ecc.bytes == 9 ? 16 : 26;
+
+ if (section >= nand_chip->ecc.steps)
+ return -ERANGE;
+
+ oobregion->offset = (section * stepsize) + 7;
+ oobregion->length = nand_chip->ecc.bytes;
+
+ return 0;
+}
+
+static int mxc_v2_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *nand_chip = mtd_to_nand(mtd);
+ int stepsize = nand_chip->ecc.bytes == 9 ? 16 : 26;
+
+ if (section > nand_chip->ecc.steps)
+ return -ERANGE;
+
+ if (!section) {
+ if (mtd->writesize <= 512) {
+ oobregion->offset = 0;
+ oobregion->length = 5;
+ } else {
+ oobregion->offset = 2;
+ oobregion->length = 4;
+ }
+ } else {
+ oobregion->offset = section * stepsize;
+ oobregion->length = 7;
+ }
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops mxc_v2_ooblayout_ops = {
+ .ecc = mxc_v2_ooblayout_ecc,
+ .free = mxc_v2_ooblayout_free,
+};
+
/*
* v2 and v3 type controllers can do 4bit or 8bit ecc depending
* on how much oob the nand chip has. For 8bit ecc we need at least
@@ -959,23 +984,6 @@
return 8;
}
-static void ecc_8bit_layout_4k(struct nand_ecclayout *layout)
-{
- int i, j;
-
- layout->eccbytes = 8*18;
- for (i = 0; i < 8; i++)
- for (j = 0; j < 18; j++)
- layout->eccpos[i*18 + j] = i*26 + j + 7;
-
- layout->oobfree[0].offset = 2;
- layout->oobfree[0].length = 4;
- for (i = 1; i < 8; i++) {
- layout->oobfree[i].offset = i*26;
- layout->oobfree[i].length = 7;
- }
-}
-
static void preset_v1(struct mtd_info *mtd)
{
struct nand_chip *nand_chip = mtd_to_nand(mtd);
@@ -1269,9 +1277,7 @@
.check_int = check_int_v1_v2,
.irq_control = irq_control_v1_v2,
.get_ecc_status = get_ecc_status_v1,
- .ecclayout_512 = &nandv1_hw_eccoob_smallpage,
- .ecclayout_2k = &nandv1_hw_eccoob_largepage,
- .ecclayout_4k = &nandv1_hw_eccoob_smallpage, /* XXX: needs fix */
+ .ooblayout = &mxc_v1_ooblayout_ops,
.select_chip = mxc_nand_select_chip_v1_v3,
.correct_data = mxc_nand_correct_data_v1,
.irqpending_quirk = 1,
@@ -1294,9 +1300,7 @@
.check_int = check_int_v1_v2,
.irq_control = irq_control_v1_v2,
.get_ecc_status = get_ecc_status_v1,
- .ecclayout_512 = &nandv1_hw_eccoob_smallpage,
- .ecclayout_2k = &nandv1_hw_eccoob_largepage,
- .ecclayout_4k = &nandv1_hw_eccoob_smallpage, /* XXX: needs fix */
+ .ooblayout = &mxc_v1_ooblayout_ops,
.select_chip = mxc_nand_select_chip_v1_v3,
.correct_data = mxc_nand_correct_data_v1,
.irqpending_quirk = 0,
@@ -1320,9 +1324,7 @@
.check_int = check_int_v1_v2,
.irq_control = irq_control_v1_v2,
.get_ecc_status = get_ecc_status_v2,
- .ecclayout_512 = &nandv2_hw_eccoob_smallpage,
- .ecclayout_2k = &nandv2_hw_eccoob_largepage,
- .ecclayout_4k = &nandv2_hw_eccoob_4k,
+ .ooblayout = &mxc_v2_ooblayout_ops,
.select_chip = mxc_nand_select_chip_v2,
.correct_data = mxc_nand_correct_data_v2_v3,
.irqpending_quirk = 0,
@@ -1346,9 +1348,7 @@
.check_int = check_int_v3,
.irq_control = irq_control_v3,
.get_ecc_status = get_ecc_status_v3,
- .ecclayout_512 = &nandv2_hw_eccoob_smallpage,
- .ecclayout_2k = &nandv2_hw_eccoob_largepage,
- .ecclayout_4k = &nandv2_hw_eccoob_smallpage, /* XXX: needs fix */
+ .ooblayout = &mxc_v2_ooblayout_ops,
.select_chip = mxc_nand_select_chip_v1_v3,
.correct_data = mxc_nand_correct_data_v2_v3,
.irqpending_quirk = 0,
@@ -1373,9 +1373,7 @@
.check_int = check_int_v3,
.irq_control = irq_control_v3,
.get_ecc_status = get_ecc_status_v3,
- .ecclayout_512 = &nandv2_hw_eccoob_smallpage,
- .ecclayout_2k = &nandv2_hw_eccoob_largepage,
- .ecclayout_4k = &nandv2_hw_eccoob_smallpage, /* XXX: needs fix */
+ .ooblayout = &mxc_v2_ooblayout_ops,
.select_chip = mxc_nand_select_chip_v1_v3,
.correct_data = mxc_nand_correct_data_v2_v3,
.irqpending_quirk = 0,
@@ -1461,25 +1459,12 @@
static int __init mxcnd_probe_dt(struct mxc_nand_host *host)
{
struct device_node *np = host->dev->of_node;
- struct mxc_nand_platform_data *pdata = &host->pdata;
const struct of_device_id *of_id =
of_match_device(mxcnd_dt_ids, host->dev);
- int buswidth;
if (!np)
return 1;
- if (of_get_nand_ecc_mode(np) >= 0)
- pdata->hw_ecc = 1;
-
- pdata->flash_bbt = of_get_nand_on_flash_bbt(np);
-
- buswidth = of_get_nand_bus_width(np);
- if (buswidth < 0)
- return buswidth;
-
- pdata->width = buswidth / 8;
-
host->devtype_data = of_id->data;
return 0;
@@ -1576,27 +1561,22 @@
this->select_chip = host->devtype_data->select_chip;
this->ecc.size = 512;
- this->ecc.layout = host->devtype_data->ecclayout_512;
+ mtd_set_ooblayout(mtd, host->devtype_data->ooblayout);
if (host->pdata.hw_ecc) {
- this->ecc.calculate = mxc_nand_calculate_ecc;
- this->ecc.hwctl = mxc_nand_enable_hwecc;
- this->ecc.correct = host->devtype_data->correct_data;
this->ecc.mode = NAND_ECC_HW;
} else {
this->ecc.mode = NAND_ECC_SOFT;
+ this->ecc.algo = NAND_ECC_HAMMING;
}
/* NAND bus width determines access functions used by upper layer */
if (host->pdata.width == 2)
this->options |= NAND_BUSWIDTH_16;
- if (host->pdata.flash_bbt) {
- this->bbt_td = &bbt_main_descr;
- this->bbt_md = &bbt_mirror_descr;
- /* update flash based bbt */
+ /* update flash based bbt */
+ if (host->pdata.flash_bbt)
this->bbt_options |= NAND_BBT_USE_FLASH;
- }
init_completion(&host->op_completion);
@@ -1637,6 +1617,26 @@
goto escan;
}
+ switch (this->ecc.mode) {
+ case NAND_ECC_HW:
+ this->ecc.calculate = mxc_nand_calculate_ecc;
+ this->ecc.hwctl = mxc_nand_enable_hwecc;
+ this->ecc.correct = host->devtype_data->correct_data;
+ break;
+
+ case NAND_ECC_SOFT:
+ break;
+
+ default:
+ err = -EINVAL;
+ goto escan;
+ }
+
+ if (this->bbt_options & NAND_BBT_USE_FLASH) {
+ this->bbt_td = &bbt_main_descr;
+ this->bbt_md = &bbt_mirror_descr;
+ }
+
/* allocate the right size buffer now */
devm_kfree(&pdev->dev, (void *)host->data_buf);
host->data_buf = devm_kzalloc(&pdev->dev, mtd->writesize + mtd->oobsize,
@@ -1649,12 +1649,11 @@
/* Call preset again, with correct writesize this time */
host->devtype_data->preset(mtd);
- if (mtd->writesize == 2048)
- this->ecc.layout = host->devtype_data->ecclayout_2k;
- else if (mtd->writesize == 4096) {
- this->ecc.layout = host->devtype_data->ecclayout_4k;
- if (get_eccsize(mtd) == 8)
- ecc_8bit_layout_4k(this->ecc.layout);
+ if (!this->ecc.bytes) {
+ if (host->eccsize == 8)
+ this->ecc.bytes = 18;
+ else if (host->eccsize == 4)
+ this->ecc.bytes = 9;
}
/*
diff --git a/drivers/mtd/nand/nand_base.c b/drivers/mtd/nand/nand_base.c
index ba4f603..0b0dc29 100644
--- a/drivers/mtd/nand/nand_base.c
+++ b/drivers/mtd/nand/nand_base.c
@@ -45,57 +45,99 @@
#include <linux/bitops.h>
#include <linux/io.h>
#include <linux/mtd/partitions.h>
-#include <linux/of_mtd.h>
-
-/* Define default oob placement schemes for large and small page devices */
-static struct nand_ecclayout nand_oob_8 = {
- .eccbytes = 3,
- .eccpos = {0, 1, 2},
- .oobfree = {
- {.offset = 3,
- .length = 2},
- {.offset = 6,
- .length = 2} }
-};
-
-static struct nand_ecclayout nand_oob_16 = {
- .eccbytes = 6,
- .eccpos = {0, 1, 2, 3, 6, 7},
- .oobfree = {
- {.offset = 8,
- . length = 8} }
-};
-
-static struct nand_ecclayout nand_oob_64 = {
- .eccbytes = 24,
- .eccpos = {
- 40, 41, 42, 43, 44, 45, 46, 47,
- 48, 49, 50, 51, 52, 53, 54, 55,
- 56, 57, 58, 59, 60, 61, 62, 63},
- .oobfree = {
- {.offset = 2,
- .length = 38} }
-};
-
-static struct nand_ecclayout nand_oob_128 = {
- .eccbytes = 48,
- .eccpos = {
- 80, 81, 82, 83, 84, 85, 86, 87,
- 88, 89, 90, 91, 92, 93, 94, 95,
- 96, 97, 98, 99, 100, 101, 102, 103,
- 104, 105, 106, 107, 108, 109, 110, 111,
- 112, 113, 114, 115, 116, 117, 118, 119,
- 120, 121, 122, 123, 124, 125, 126, 127},
- .oobfree = {
- {.offset = 2,
- .length = 78} }
-};
+#include <linux/of.h>
static int nand_get_device(struct mtd_info *mtd, int new_state);
static int nand_do_write_oob(struct mtd_info *mtd, loff_t to,
struct mtd_oob_ops *ops);
+/* Define default oob placement schemes for large and small page devices */
+static int nand_ooblayout_ecc_sp(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+ struct nand_ecc_ctrl *ecc = &chip->ecc;
+
+ if (section > 1)
+ return -ERANGE;
+
+ if (!section) {
+ oobregion->offset = 0;
+ oobregion->length = 4;
+ } else {
+ oobregion->offset = 6;
+ oobregion->length = ecc->total - 4;
+ }
+
+ return 0;
+}
+
+static int nand_ooblayout_free_sp(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section > 1)
+ return -ERANGE;
+
+ if (mtd->oobsize == 16) {
+ if (section)
+ return -ERANGE;
+
+ oobregion->length = 8;
+ oobregion->offset = 8;
+ } else {
+ oobregion->length = 2;
+ if (!section)
+ oobregion->offset = 3;
+ else
+ oobregion->offset = 6;
+ }
+
+ return 0;
+}
+
+const struct mtd_ooblayout_ops nand_ooblayout_sp_ops = {
+ .ecc = nand_ooblayout_ecc_sp,
+ .free = nand_ooblayout_free_sp,
+};
+EXPORT_SYMBOL_GPL(nand_ooblayout_sp_ops);
+
+static int nand_ooblayout_ecc_lp(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+ struct nand_ecc_ctrl *ecc = &chip->ecc;
+
+ if (section)
+ return -ERANGE;
+
+ oobregion->length = ecc->total;
+ oobregion->offset = mtd->oobsize - oobregion->length;
+
+ return 0;
+}
+
+static int nand_ooblayout_free_lp(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+ struct nand_ecc_ctrl *ecc = &chip->ecc;
+
+ if (section)
+ return -ERANGE;
+
+ oobregion->length = mtd->oobsize - ecc->total - 2;
+ oobregion->offset = 2;
+
+ return 0;
+}
+
+const struct mtd_ooblayout_ops nand_ooblayout_lp_ops = {
+ .ecc = nand_ooblayout_ecc_lp,
+ .free = nand_ooblayout_free_lp,
+};
+EXPORT_SYMBOL_GPL(nand_ooblayout_lp_ops);
+
static int check_offs_len(struct mtd_info *mtd,
loff_t ofs, uint64_t len)
{
@@ -1279,13 +1321,12 @@
static int nand_read_page_swecc(struct mtd_info *mtd, struct nand_chip *chip,
uint8_t *buf, int oob_required, int page)
{
- int i, eccsize = chip->ecc.size;
+ int i, eccsize = chip->ecc.size, ret;
int eccbytes = chip->ecc.bytes;
int eccsteps = chip->ecc.steps;
uint8_t *p = buf;
uint8_t *ecc_calc = chip->buffers->ecccalc;
uint8_t *ecc_code = chip->buffers->ecccode;
- uint32_t *eccpos = chip->ecc.layout->eccpos;
unsigned int max_bitflips = 0;
chip->ecc.read_page_raw(mtd, chip, buf, 1, page);
@@ -1293,8 +1334,10 @@
for (i = 0; eccsteps; eccsteps--, i += eccbytes, p += eccsize)
chip->ecc.calculate(mtd, p, &ecc_calc[i]);
- for (i = 0; i < chip->ecc.total; i++)
- ecc_code[i] = chip->oob_poi[eccpos[i]];
+ ret = mtd_ooblayout_get_eccbytes(mtd, ecc_code, chip->oob_poi, 0,
+ chip->ecc.total);
+ if (ret)
+ return ret;
eccsteps = chip->ecc.steps;
p = buf;
@@ -1326,14 +1369,14 @@
uint32_t data_offs, uint32_t readlen, uint8_t *bufpoi,
int page)
{
- int start_step, end_step, num_steps;
- uint32_t *eccpos = chip->ecc.layout->eccpos;
+ int start_step, end_step, num_steps, ret;
uint8_t *p;
int data_col_addr, i, gaps = 0;
int datafrag_len, eccfrag_len, aligned_len, aligned_pos;
int busw = (chip->options & NAND_BUSWIDTH_16) ? 2 : 1;
- int index;
+ int index, section = 0;
unsigned int max_bitflips = 0;
+ struct mtd_oob_region oobregion = { };
/* Column address within the page aligned to ECC size (256bytes) */
start_step = data_offs / chip->ecc.size;
@@ -1361,12 +1404,13 @@
* The performance is faster if we position offsets according to
* ecc.pos. Let's make sure that there are no gaps in ECC positions.
*/
- for (i = 0; i < eccfrag_len - 1; i++) {
- if (eccpos[i + index] + 1 != eccpos[i + index + 1]) {
- gaps = 1;
- break;
- }
- }
+ ret = mtd_ooblayout_find_eccregion(mtd, index, §ion, &oobregion);
+ if (ret)
+ return ret;
+
+ if (oobregion.length < eccfrag_len)
+ gaps = 1;
+
if (gaps) {
chip->cmdfunc(mtd, NAND_CMD_RNDOUT, mtd->writesize, -1);
chip->read_buf(mtd, chip->oob_poi, mtd->oobsize);
@@ -1375,20 +1419,23 @@
* Send the command to read the particular ECC bytes take care
* about buswidth alignment in read_buf.
*/
- aligned_pos = eccpos[index] & ~(busw - 1);
+ aligned_pos = oobregion.offset & ~(busw - 1);
aligned_len = eccfrag_len;
- if (eccpos[index] & (busw - 1))
+ if (oobregion.offset & (busw - 1))
aligned_len++;
- if (eccpos[index + (num_steps * chip->ecc.bytes)] & (busw - 1))
+ if ((oobregion.offset + (num_steps * chip->ecc.bytes)) &
+ (busw - 1))
aligned_len++;
chip->cmdfunc(mtd, NAND_CMD_RNDOUT,
- mtd->writesize + aligned_pos, -1);
+ mtd->writesize + aligned_pos, -1);
chip->read_buf(mtd, &chip->oob_poi[aligned_pos], aligned_len);
}
- for (i = 0; i < eccfrag_len; i++)
- chip->buffers->ecccode[i] = chip->oob_poi[eccpos[i + index]];
+ ret = mtd_ooblayout_get_eccbytes(mtd, chip->buffers->ecccode,
+ chip->oob_poi, index, eccfrag_len);
+ if (ret)
+ return ret;
p = bufpoi + data_col_addr;
for (i = 0; i < eccfrag_len ; i += chip->ecc.bytes, p += chip->ecc.size) {
@@ -1429,13 +1476,12 @@
static int nand_read_page_hwecc(struct mtd_info *mtd, struct nand_chip *chip,
uint8_t *buf, int oob_required, int page)
{
- int i, eccsize = chip->ecc.size;
+ int i, eccsize = chip->ecc.size, ret;
int eccbytes = chip->ecc.bytes;
int eccsteps = chip->ecc.steps;
uint8_t *p = buf;
uint8_t *ecc_calc = chip->buffers->ecccalc;
uint8_t *ecc_code = chip->buffers->ecccode;
- uint32_t *eccpos = chip->ecc.layout->eccpos;
unsigned int max_bitflips = 0;
for (i = 0; eccsteps; eccsteps--, i += eccbytes, p += eccsize) {
@@ -1445,8 +1491,10 @@
}
chip->read_buf(mtd, chip->oob_poi, mtd->oobsize);
- for (i = 0; i < chip->ecc.total; i++)
- ecc_code[i] = chip->oob_poi[eccpos[i]];
+ ret = mtd_ooblayout_get_eccbytes(mtd, ecc_code, chip->oob_poi, 0,
+ chip->ecc.total);
+ if (ret)
+ return ret;
eccsteps = chip->ecc.steps;
p = buf;
@@ -1491,12 +1539,11 @@
static int nand_read_page_hwecc_oob_first(struct mtd_info *mtd,
struct nand_chip *chip, uint8_t *buf, int oob_required, int page)
{
- int i, eccsize = chip->ecc.size;
+ int i, eccsize = chip->ecc.size, ret;
int eccbytes = chip->ecc.bytes;
int eccsteps = chip->ecc.steps;
uint8_t *p = buf;
uint8_t *ecc_code = chip->buffers->ecccode;
- uint32_t *eccpos = chip->ecc.layout->eccpos;
uint8_t *ecc_calc = chip->buffers->ecccalc;
unsigned int max_bitflips = 0;
@@ -1505,8 +1552,10 @@
chip->read_buf(mtd, chip->oob_poi, mtd->oobsize);
chip->cmdfunc(mtd, NAND_CMD_READ0, 0, page);
- for (i = 0; i < chip->ecc.total; i++)
- ecc_code[i] = chip->oob_poi[eccpos[i]];
+ ret = mtd_ooblayout_get_eccbytes(mtd, ecc_code, chip->oob_poi, 0,
+ chip->ecc.total);
+ if (ret)
+ return ret;
for (i = 0; eccsteps; eccsteps--, i += eccbytes, p += eccsize) {
int stat;
@@ -1607,14 +1656,17 @@
/**
* nand_transfer_oob - [INTERN] Transfer oob to client buffer
- * @chip: nand chip structure
+ * @mtd: mtd info structure
* @oob: oob destination address
* @ops: oob ops structure
* @len: size of oob to transfer
*/
-static uint8_t *nand_transfer_oob(struct nand_chip *chip, uint8_t *oob,
+static uint8_t *nand_transfer_oob(struct mtd_info *mtd, uint8_t *oob,
struct mtd_oob_ops *ops, size_t len)
{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+ int ret;
+
switch (ops->mode) {
case MTD_OPS_PLACE_OOB:
@@ -1622,31 +1674,12 @@
memcpy(oob, chip->oob_poi + ops->ooboffs, len);
return oob + len;
- case MTD_OPS_AUTO_OOB: {
- struct nand_oobfree *free = chip->ecc.layout->oobfree;
- uint32_t boffs = 0, roffs = ops->ooboffs;
- size_t bytes = 0;
+ case MTD_OPS_AUTO_OOB:
+ ret = mtd_ooblayout_get_databytes(mtd, oob, chip->oob_poi,
+ ops->ooboffs, len);
+ BUG_ON(ret);
+ return oob + len;
- for (; free->length && len; free++, len -= bytes) {
- /* Read request not from offset 0? */
- if (unlikely(roffs)) {
- if (roffs >= free->length) {
- roffs -= free->length;
- continue;
- }
- boffs = free->offset + roffs;
- bytes = min_t(size_t, len,
- (free->length - roffs));
- roffs = 0;
- } else {
- bytes = min_t(size_t, len, free->length);
- boffs = free->offset;
- }
- memcpy(oob, chip->oob_poi + boffs, bytes);
- oob += bytes;
- }
- return oob;
- }
default:
BUG();
}
@@ -1780,7 +1813,7 @@
int toread = min(oobreadlen, max_oobsize);
if (toread) {
- oob = nand_transfer_oob(chip,
+ oob = nand_transfer_oob(mtd,
oob, ops, toread);
oobreadlen -= toread;
}
@@ -1893,13 +1926,13 @@
* @chip: nand chip info structure
* @page: page number to read
*/
-static int nand_read_oob_std(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+int nand_read_oob_std(struct mtd_info *mtd, struct nand_chip *chip, int page)
{
chip->cmdfunc(mtd, NAND_CMD_READOOB, 0, page);
chip->read_buf(mtd, chip->oob_poi, mtd->oobsize);
return 0;
}
+EXPORT_SYMBOL(nand_read_oob_std);
/**
* nand_read_oob_syndrome - [REPLACEABLE] OOB data read function for HW ECC
@@ -1908,8 +1941,8 @@
* @chip: nand chip info structure
* @page: page number to read
*/
-static int nand_read_oob_syndrome(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+int nand_read_oob_syndrome(struct mtd_info *mtd, struct nand_chip *chip,
+ int page)
{
int length = mtd->oobsize;
int chunk = chip->ecc.bytes + chip->ecc.prepad + chip->ecc.postpad;
@@ -1937,6 +1970,7 @@
return 0;
}
+EXPORT_SYMBOL(nand_read_oob_syndrome);
/**
* nand_write_oob_std - [REPLACEABLE] the most common OOB data write function
@@ -1944,8 +1978,7 @@
* @chip: nand chip info structure
* @page: page number to write
*/
-static int nand_write_oob_std(struct mtd_info *mtd, struct nand_chip *chip,
- int page)
+int nand_write_oob_std(struct mtd_info *mtd, struct nand_chip *chip, int page)
{
int status = 0;
const uint8_t *buf = chip->oob_poi;
@@ -1960,6 +1993,7 @@
return status & NAND_STATUS_FAIL ? -EIO : 0;
}
+EXPORT_SYMBOL(nand_write_oob_std);
/**
* nand_write_oob_syndrome - [REPLACEABLE] OOB data write function for HW ECC
@@ -1968,8 +2002,8 @@
* @chip: nand chip info structure
* @page: page number to write
*/
-static int nand_write_oob_syndrome(struct mtd_info *mtd,
- struct nand_chip *chip, int page)
+int nand_write_oob_syndrome(struct mtd_info *mtd, struct nand_chip *chip,
+ int page)
{
int chunk = chip->ecc.bytes + chip->ecc.prepad + chip->ecc.postpad;
int eccsize = chip->ecc.size, length = mtd->oobsize;
@@ -2019,6 +2053,7 @@
return status & NAND_STATUS_FAIL ? -EIO : 0;
}
+EXPORT_SYMBOL(nand_write_oob_syndrome);
/**
* nand_do_read_oob - [INTERN] NAND read out-of-band
@@ -2078,7 +2113,7 @@
break;
len = min(len, readlen);
- buf = nand_transfer_oob(chip, buf, ops, len);
+ buf = nand_transfer_oob(mtd, buf, ops, len);
if (chip->options & NAND_NEED_READRDY) {
/* Apply delay or wait for ready/busy pin */
@@ -2237,19 +2272,20 @@
const uint8_t *buf, int oob_required,
int page)
{
- int i, eccsize = chip->ecc.size;
+ int i, eccsize = chip->ecc.size, ret;
int eccbytes = chip->ecc.bytes;
int eccsteps = chip->ecc.steps;
uint8_t *ecc_calc = chip->buffers->ecccalc;
const uint8_t *p = buf;
- uint32_t *eccpos = chip->ecc.layout->eccpos;
/* Software ECC calculation */
for (i = 0; eccsteps; eccsteps--, i += eccbytes, p += eccsize)
chip->ecc.calculate(mtd, p, &ecc_calc[i]);
- for (i = 0; i < chip->ecc.total; i++)
- chip->oob_poi[eccpos[i]] = ecc_calc[i];
+ ret = mtd_ooblayout_set_eccbytes(mtd, ecc_calc, chip->oob_poi, 0,
+ chip->ecc.total);
+ if (ret)
+ return ret;
return chip->ecc.write_page_raw(mtd, chip, buf, 1, page);
}
@@ -2266,12 +2302,11 @@
const uint8_t *buf, int oob_required,
int page)
{
- int i, eccsize = chip->ecc.size;
+ int i, eccsize = chip->ecc.size, ret;
int eccbytes = chip->ecc.bytes;
int eccsteps = chip->ecc.steps;
uint8_t *ecc_calc = chip->buffers->ecccalc;
const uint8_t *p = buf;
- uint32_t *eccpos = chip->ecc.layout->eccpos;
for (i = 0; eccsteps; eccsteps--, i += eccbytes, p += eccsize) {
chip->ecc.hwctl(mtd, NAND_ECC_WRITE);
@@ -2279,8 +2314,10 @@
chip->ecc.calculate(mtd, p, &ecc_calc[i]);
}
- for (i = 0; i < chip->ecc.total; i++)
- chip->oob_poi[eccpos[i]] = ecc_calc[i];
+ ret = mtd_ooblayout_set_eccbytes(mtd, ecc_calc, chip->oob_poi, 0,
+ chip->ecc.total);
+ if (ret)
+ return ret;
chip->write_buf(mtd, chip->oob_poi, mtd->oobsize);
@@ -2308,11 +2345,10 @@
int ecc_size = chip->ecc.size;
int ecc_bytes = chip->ecc.bytes;
int ecc_steps = chip->ecc.steps;
- uint32_t *eccpos = chip->ecc.layout->eccpos;
uint32_t start_step = offset / ecc_size;
uint32_t end_step = (offset + data_len - 1) / ecc_size;
int oob_bytes = mtd->oobsize / ecc_steps;
- int step, i;
+ int step, ret;
for (step = 0; step < ecc_steps; step++) {
/* configure controller for WRITE access */
@@ -2340,8 +2376,10 @@
/* copy calculated ECC for whole page to chip->buffer->oob */
/* this include masked-value(0xFF) for unwritten subpages */
ecc_calc = chip->buffers->ecccalc;
- for (i = 0; i < chip->ecc.total; i++)
- chip->oob_poi[eccpos[i]] = ecc_calc[i];
+ ret = mtd_ooblayout_set_eccbytes(mtd, ecc_calc, chip->oob_poi, 0,
+ chip->ecc.total);
+ if (ret)
+ return ret;
/* write OOB buffer to NAND device */
chip->write_buf(mtd, chip->oob_poi, mtd->oobsize);
@@ -2478,6 +2516,7 @@
struct mtd_oob_ops *ops)
{
struct nand_chip *chip = mtd_to_nand(mtd);
+ int ret;
/*
* Initialise to all 0xFF, to avoid the possibility of left over OOB
@@ -2492,31 +2531,12 @@
memcpy(chip->oob_poi + ops->ooboffs, oob, len);
return oob + len;
- case MTD_OPS_AUTO_OOB: {
- struct nand_oobfree *free = chip->ecc.layout->oobfree;
- uint32_t boffs = 0, woffs = ops->ooboffs;
- size_t bytes = 0;
+ case MTD_OPS_AUTO_OOB:
+ ret = mtd_ooblayout_set_databytes(mtd, oob, chip->oob_poi,
+ ops->ooboffs, len);
+ BUG_ON(ret);
+ return oob + len;
- for (; free->length && len; free++, len -= bytes) {
- /* Write request not from offset 0? */
- if (unlikely(woffs)) {
- if (woffs >= free->length) {
- woffs -= free->length;
- continue;
- }
- boffs = free->offset + woffs;
- bytes = min_t(size_t, len,
- (free->length - woffs));
- woffs = 0;
- } else {
- bytes = min_t(size_t, len, free->length);
- boffs = free->offset;
- }
- memcpy(chip->oob_poi + boffs, oob, bytes);
- oob += bytes;
- }
- return oob;
- }
default:
BUG();
}
@@ -3951,10 +3971,115 @@
return type;
}
+static const char * const nand_ecc_modes[] = {
+ [NAND_ECC_NONE] = "none",
+ [NAND_ECC_SOFT] = "soft",
+ [NAND_ECC_HW] = "hw",
+ [NAND_ECC_HW_SYNDROME] = "hw_syndrome",
+ [NAND_ECC_HW_OOB_FIRST] = "hw_oob_first",
+};
+
+static int of_get_nand_ecc_mode(struct device_node *np)
+{
+ const char *pm;
+ int err, i;
+
+ err = of_property_read_string(np, "nand-ecc-mode", &pm);
+ if (err < 0)
+ return err;
+
+ for (i = 0; i < ARRAY_SIZE(nand_ecc_modes); i++)
+ if (!strcasecmp(pm, nand_ecc_modes[i]))
+ return i;
+
+ /*
+ * For backward compatibility we support few obsoleted values that don't
+ * have their mappings into nand_ecc_modes_t anymore (they were merged
+ * with other enums).
+ */
+ if (!strcasecmp(pm, "soft_bch"))
+ return NAND_ECC_SOFT;
+
+ return -ENODEV;
+}
+
+static const char * const nand_ecc_algos[] = {
+ [NAND_ECC_HAMMING] = "hamming",
+ [NAND_ECC_BCH] = "bch",
+};
+
+static int of_get_nand_ecc_algo(struct device_node *np)
+{
+ const char *pm;
+ int err, i;
+
+ err = of_property_read_string(np, "nand-ecc-algo", &pm);
+ if (!err) {
+ for (i = NAND_ECC_HAMMING; i < ARRAY_SIZE(nand_ecc_algos); i++)
+ if (!strcasecmp(pm, nand_ecc_algos[i]))
+ return i;
+ return -ENODEV;
+ }
+
+ /*
+ * For backward compatibility we also read "nand-ecc-mode" checking
+ * for some obsoleted values that were specifying ECC algorithm.
+ */
+ err = of_property_read_string(np, "nand-ecc-mode", &pm);
+ if (err < 0)
+ return err;
+
+ if (!strcasecmp(pm, "soft"))
+ return NAND_ECC_HAMMING;
+ else if (!strcasecmp(pm, "soft_bch"))
+ return NAND_ECC_BCH;
+
+ return -ENODEV;
+}
+
+static int of_get_nand_ecc_step_size(struct device_node *np)
+{
+ int ret;
+ u32 val;
+
+ ret = of_property_read_u32(np, "nand-ecc-step-size", &val);
+ return ret ? ret : val;
+}
+
+static int of_get_nand_ecc_strength(struct device_node *np)
+{
+ int ret;
+ u32 val;
+
+ ret = of_property_read_u32(np, "nand-ecc-strength", &val);
+ return ret ? ret : val;
+}
+
+static int of_get_nand_bus_width(struct device_node *np)
+{
+ u32 val;
+
+ if (of_property_read_u32(np, "nand-bus-width", &val))
+ return 8;
+
+ switch (val) {
+ case 8:
+ case 16:
+ return val;
+ default:
+ return -EIO;
+ }
+}
+
+static bool of_get_nand_on_flash_bbt(struct device_node *np)
+{
+ return of_property_read_bool(np, "nand-on-flash-bbt");
+}
+
static int nand_dt_init(struct nand_chip *chip)
{
struct device_node *dn = nand_get_flash_node(chip);
- int ecc_mode, ecc_strength, ecc_step;
+ int ecc_mode, ecc_algo, ecc_strength, ecc_step;
if (!dn)
return 0;
@@ -3966,6 +4091,7 @@
chip->bbt_options |= NAND_BBT_USE_FLASH;
ecc_mode = of_get_nand_ecc_mode(dn);
+ ecc_algo = of_get_nand_ecc_algo(dn);
ecc_strength = of_get_nand_ecc_strength(dn);
ecc_step = of_get_nand_ecc_step_size(dn);
@@ -3978,6 +4104,9 @@
if (ecc_mode >= 0)
chip->ecc.mode = ecc_mode;
+ if (ecc_algo >= 0)
+ chip->ecc.algo = ecc_algo;
+
if (ecc_strength >= 0)
chip->ecc.strength = ecc_strength;
@@ -4054,6 +4183,82 @@
}
EXPORT_SYMBOL(nand_scan_ident);
+static int nand_set_ecc_soft_ops(struct mtd_info *mtd)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+ struct nand_ecc_ctrl *ecc = &chip->ecc;
+
+ if (WARN_ON(ecc->mode != NAND_ECC_SOFT))
+ return -EINVAL;
+
+ switch (ecc->algo) {
+ case NAND_ECC_HAMMING:
+ ecc->calculate = nand_calculate_ecc;
+ ecc->correct = nand_correct_data;
+ ecc->read_page = nand_read_page_swecc;
+ ecc->read_subpage = nand_read_subpage;
+ ecc->write_page = nand_write_page_swecc;
+ ecc->read_page_raw = nand_read_page_raw;
+ ecc->write_page_raw = nand_write_page_raw;
+ ecc->read_oob = nand_read_oob_std;
+ ecc->write_oob = nand_write_oob_std;
+ if (!ecc->size)
+ ecc->size = 256;
+ ecc->bytes = 3;
+ ecc->strength = 1;
+ return 0;
+ case NAND_ECC_BCH:
+ if (!mtd_nand_has_bch()) {
+ WARN(1, "CONFIG_MTD_NAND_ECC_BCH not enabled\n");
+ return -EINVAL;
+ }
+ ecc->calculate = nand_bch_calculate_ecc;
+ ecc->correct = nand_bch_correct_data;
+ ecc->read_page = nand_read_page_swecc;
+ ecc->read_subpage = nand_read_subpage;
+ ecc->write_page = nand_write_page_swecc;
+ ecc->read_page_raw = nand_read_page_raw;
+ ecc->write_page_raw = nand_write_page_raw;
+ ecc->read_oob = nand_read_oob_std;
+ ecc->write_oob = nand_write_oob_std;
+ /*
+ * Board driver should supply ecc.size and ecc.strength
+ * values to select how many bits are correctable.
+ * Otherwise, default to 4 bits for large page devices.
+ */
+ if (!ecc->size && (mtd->oobsize >= 64)) {
+ ecc->size = 512;
+ ecc->strength = 4;
+ }
+
+ /*
+ * if no ecc placement scheme was provided pickup the default
+ * large page one.
+ */
+ if (!mtd->ooblayout) {
+ /* handle large page devices only */
+ if (mtd->oobsize < 64) {
+ WARN(1, "OOB layout is required when using software BCH on small pages\n");
+ return -EINVAL;
+ }
+
+ mtd_set_ooblayout(mtd, &nand_ooblayout_lp_ops);
+ }
+
+ /* See nand_bch_init() for details. */
+ ecc->bytes = 0;
+ ecc->priv = nand_bch_init(mtd);
+ if (!ecc->priv) {
+ WARN(1, "BCH ECC initialization failed!\n");
+ return -EINVAL;
+ }
+ return 0;
+ default:
+ WARN(1, "Unsupported ECC algorithm!\n");
+ return -EINVAL;
+ }
+}
+
/*
* Check if the chip configuration meet the datasheet requirements.
@@ -4098,14 +4303,15 @@
*/
int nand_scan_tail(struct mtd_info *mtd)
{
- int i;
struct nand_chip *chip = mtd_to_nand(mtd);
struct nand_ecc_ctrl *ecc = &chip->ecc;
struct nand_buffers *nbuf;
+ int ret;
/* New bad blocks should be marked in OOB, flash-based BBT, or both */
- BUG_ON((chip->bbt_options & NAND_BBT_NO_OOB_BBM) &&
- !(chip->bbt_options & NAND_BBT_USE_FLASH));
+ if (WARN_ON((chip->bbt_options & NAND_BBT_NO_OOB_BBM) &&
+ !(chip->bbt_options & NAND_BBT_USE_FLASH)))
+ return -EINVAL;
if (!(chip->options & NAND_OWN_BUFFERS)) {
nbuf = kzalloc(sizeof(*nbuf) + mtd->writesize
@@ -4128,24 +4334,22 @@
/*
* If no default placement scheme is given, select an appropriate one.
*/
- if (!ecc->layout && (ecc->mode != NAND_ECC_SOFT_BCH)) {
+ if (!mtd->ooblayout &&
+ !(ecc->mode == NAND_ECC_SOFT && ecc->algo == NAND_ECC_BCH)) {
switch (mtd->oobsize) {
case 8:
- ecc->layout = &nand_oob_8;
- break;
case 16:
- ecc->layout = &nand_oob_16;
+ mtd_set_ooblayout(mtd, &nand_ooblayout_sp_ops);
break;
case 64:
- ecc->layout = &nand_oob_64;
- break;
case 128:
- ecc->layout = &nand_oob_128;
+ mtd_set_ooblayout(mtd, &nand_ooblayout_lp_ops);
break;
default:
- pr_warn("No oob scheme defined for oobsize %d\n",
- mtd->oobsize);
- BUG();
+ WARN(1, "No oob scheme defined for oobsize %d\n",
+ mtd->oobsize);
+ ret = -EINVAL;
+ goto err_free;
}
}
@@ -4161,8 +4365,9 @@
case NAND_ECC_HW_OOB_FIRST:
/* Similar to NAND_ECC_HW, but a separate read_page handle */
if (!ecc->calculate || !ecc->correct || !ecc->hwctl) {
- pr_warn("No ECC functions supplied; hardware ECC not possible\n");
- BUG();
+ WARN(1, "No ECC functions supplied; hardware ECC not possible\n");
+ ret = -EINVAL;
+ goto err_free;
}
if (!ecc->read_page)
ecc->read_page = nand_read_page_hwecc_oob_first;
@@ -4192,8 +4397,9 @@
ecc->read_page == nand_read_page_hwecc ||
!ecc->write_page ||
ecc->write_page == nand_write_page_hwecc)) {
- pr_warn("No ECC functions supplied; hardware ECC not possible\n");
- BUG();
+ WARN(1, "No ECC functions supplied; hardware ECC not possible\n");
+ ret = -EINVAL;
+ goto err_free;
}
/* Use standard syndrome read/write page function? */
if (!ecc->read_page)
@@ -4211,61 +4417,22 @@
if (mtd->writesize >= ecc->size) {
if (!ecc->strength) {
- pr_warn("Driver must set ecc.strength when using hardware ECC\n");
- BUG();
+ WARN(1, "Driver must set ecc.strength when using hardware ECC\n");
+ ret = -EINVAL;
+ goto err_free;
}
break;
}
pr_warn("%d byte HW ECC not possible on %d byte page size, fallback to SW ECC\n",
ecc->size, mtd->writesize);
ecc->mode = NAND_ECC_SOFT;
+ ecc->algo = NAND_ECC_HAMMING;
case NAND_ECC_SOFT:
- ecc->calculate = nand_calculate_ecc;
- ecc->correct = nand_correct_data;
- ecc->read_page = nand_read_page_swecc;
- ecc->read_subpage = nand_read_subpage;
- ecc->write_page = nand_write_page_swecc;
- ecc->read_page_raw = nand_read_page_raw;
- ecc->write_page_raw = nand_write_page_raw;
- ecc->read_oob = nand_read_oob_std;
- ecc->write_oob = nand_write_oob_std;
- if (!ecc->size)
- ecc->size = 256;
- ecc->bytes = 3;
- ecc->strength = 1;
- break;
-
- case NAND_ECC_SOFT_BCH:
- if (!mtd_nand_has_bch()) {
- pr_warn("CONFIG_MTD_NAND_ECC_BCH not enabled\n");
- BUG();
- }
- ecc->calculate = nand_bch_calculate_ecc;
- ecc->correct = nand_bch_correct_data;
- ecc->read_page = nand_read_page_swecc;
- ecc->read_subpage = nand_read_subpage;
- ecc->write_page = nand_write_page_swecc;
- ecc->read_page_raw = nand_read_page_raw;
- ecc->write_page_raw = nand_write_page_raw;
- ecc->read_oob = nand_read_oob_std;
- ecc->write_oob = nand_write_oob_std;
- /*
- * Board driver should supply ecc.size and ecc.strength values
- * to select how many bits are correctable. Otherwise, default
- * to 4 bits for large page devices.
- */
- if (!ecc->size && (mtd->oobsize >= 64)) {
- ecc->size = 512;
- ecc->strength = 4;
- }
-
- /* See nand_bch_init() for details. */
- ecc->bytes = 0;
- ecc->priv = nand_bch_init(mtd);
- if (!ecc->priv) {
- pr_warn("BCH ECC initialization failed!\n");
- BUG();
+ ret = nand_set_ecc_soft_ops(mtd);
+ if (ret) {
+ ret = -EINVAL;
+ goto err_free;
}
break;
@@ -4283,8 +4450,9 @@
break;
default:
- pr_warn("Invalid NAND_ECC_MODE %d\n", ecc->mode);
- BUG();
+ WARN(1, "Invalid NAND_ECC_MODE %d\n", ecc->mode);
+ ret = -EINVAL;
+ goto err_free;
}
/* For many systems, the standard OOB write also works for raw */
@@ -4293,20 +4461,9 @@
if (!ecc->write_oob_raw)
ecc->write_oob_raw = ecc->write_oob;
- /*
- * The number of bytes available for a client to place data into
- * the out of band area.
- */
- mtd->oobavail = 0;
- if (ecc->layout) {
- for (i = 0; ecc->layout->oobfree[i].length; i++)
- mtd->oobavail += ecc->layout->oobfree[i].length;
- }
-
- /* ECC sanity check: warn if it's too weak */
- if (!nand_ecc_strength_good(mtd))
- pr_warn("WARNING: %s: the ECC used on your system is too weak compared to the one required by the NAND chip\n",
- mtd->name);
+ /* propagate ecc info to mtd_info */
+ mtd->ecc_strength = ecc->strength;
+ mtd->ecc_step_size = ecc->size;
/*
* Set the number of read / write steps for one page depending on ECC
@@ -4314,11 +4471,27 @@
*/
ecc->steps = mtd->writesize / ecc->size;
if (ecc->steps * ecc->size != mtd->writesize) {
- pr_warn("Invalid ECC parameters\n");
- BUG();
+ WARN(1, "Invalid ECC parameters\n");
+ ret = -EINVAL;
+ goto err_free;
}
ecc->total = ecc->steps * ecc->bytes;
+ /*
+ * The number of bytes available for a client to place data into
+ * the out of band area.
+ */
+ ret = mtd_ooblayout_count_freebytes(mtd);
+ if (ret < 0)
+ ret = 0;
+
+ mtd->oobavail = ret;
+
+ /* ECC sanity check: warn if it's too weak */
+ if (!nand_ecc_strength_good(mtd))
+ pr_warn("WARNING: %s: the ECC used on your system is too weak compared to the one required by the NAND chip\n",
+ mtd->name);
+
/* Allow subpage writes up to ecc.steps. Not possible for MLC flash */
if (!(chip->options & NAND_NO_SUBPAGE_WRITE) && nand_is_slc(chip)) {
switch (ecc->steps) {
@@ -4343,7 +4516,6 @@
/* Large page NAND with SOFT_ECC should support subpage reads */
switch (ecc->mode) {
case NAND_ECC_SOFT:
- case NAND_ECC_SOFT_BCH:
if (chip->page_shift > 9)
chip->options |= NAND_SUBPAGE_READ;
break;
@@ -4375,10 +4547,6 @@
mtd->_block_markbad = nand_block_markbad;
mtd->writebufsize = mtd->writesize;
- /* propagate ecc info to mtd_info */
- mtd->ecclayout = ecc->layout;
- mtd->ecc_strength = ecc->strength;
- mtd->ecc_step_size = ecc->size;
/*
* Initialize bitflip_threshold to its default prior scan_bbt() call.
* scan_bbt() might invoke mtd_read(), thus bitflip_threshold must be
@@ -4393,6 +4561,10 @@
/* Build bad block table */
return chip->scan_bbt(mtd);
+err_free:
+ if (!(chip->options & NAND_OWN_BUFFERS))
+ kfree(chip->buffers);
+ return ret;
}
EXPORT_SYMBOL(nand_scan_tail);
@@ -4436,7 +4608,8 @@
{
struct nand_chip *chip = mtd_to_nand(mtd);
- if (chip->ecc.mode == NAND_ECC_SOFT_BCH)
+ if (chip->ecc.mode == NAND_ECC_SOFT &&
+ chip->ecc.algo == NAND_ECC_BCH)
nand_bch_free((struct nand_bch_control *)chip->ecc.priv);
mtd_device_unregister(mtd);
diff --git a/drivers/mtd/nand/nand_bch.c b/drivers/mtd/nand/nand_bch.c
index b585bae..44763f8 100644
--- a/drivers/mtd/nand/nand_bch.c
+++ b/drivers/mtd/nand/nand_bch.c
@@ -32,13 +32,11 @@
/**
* struct nand_bch_control - private NAND BCH control structure
* @bch: BCH control structure
- * @ecclayout: private ecc layout for this BCH configuration
* @errloc: error location array
* @eccmask: XOR ecc mask, allows erased pages to be decoded as valid
*/
struct nand_bch_control {
struct bch_control *bch;
- struct nand_ecclayout ecclayout;
unsigned int *errloc;
unsigned char *eccmask;
};
@@ -124,7 +122,6 @@
{
struct nand_chip *nand = mtd_to_nand(mtd);
unsigned int m, t, eccsteps, i;
- struct nand_ecclayout *layout = nand->ecc.layout;
struct nand_bch_control *nbc = NULL;
unsigned char *erased_page;
unsigned int eccsize = nand->ecc.size;
@@ -161,34 +158,10 @@
eccsteps = mtd->writesize/eccsize;
- /* if no ecc placement scheme was provided, build one */
- if (!layout) {
-
- /* handle large page devices only */
- if (mtd->oobsize < 64) {
- printk(KERN_WARNING "must provide an oob scheme for "
- "oobsize %d\n", mtd->oobsize);
- goto fail;
- }
-
- layout = &nbc->ecclayout;
- layout->eccbytes = eccsteps*eccbytes;
-
- /* reserve 2 bytes for bad block marker */
- if (layout->eccbytes+2 > mtd->oobsize) {
- printk(KERN_WARNING "no suitable oob scheme available "
- "for oobsize %d eccbytes %u\n", mtd->oobsize,
- eccbytes);
- goto fail;
- }
- /* put ecc bytes at oob tail */
- for (i = 0; i < layout->eccbytes; i++)
- layout->eccpos[i] = mtd->oobsize-layout->eccbytes+i;
-
- layout->oobfree[0].offset = 2;
- layout->oobfree[0].length = mtd->oobsize-2-layout->eccbytes;
-
- nand->ecc.layout = layout;
+ /* Check that we have an oob layout description. */
+ if (!mtd->ooblayout) {
+ pr_warn("missing oob scheme");
+ goto fail;
}
/* sanity checks */
@@ -196,7 +169,18 @@
printk(KERN_WARNING "eccsize %u is too large\n", eccsize);
goto fail;
}
- if (layout->eccbytes != (eccsteps*eccbytes)) {
+
+ /*
+ * ecc->steps and ecc->total might be used by mtd->ooblayout->ecc(),
+ * which is called by mtd_ooblayout_count_eccbytes().
+ * Make sure they are properly initialized before calling
+ * mtd_ooblayout_count_eccbytes().
+ * FIXME: we should probably rework the sequencing in nand_scan_tail()
+ * to avoid setting those fields twice.
+ */
+ nand->ecc.steps = eccsteps;
+ nand->ecc.total = eccsteps * eccbytes;
+ if (mtd_ooblayout_count_eccbytes(mtd) != (eccsteps*eccbytes)) {
printk(KERN_WARNING "invalid ecc layout\n");
goto fail;
}
diff --git a/drivers/mtd/nand/nandsim.c b/drivers/mtd/nand/nandsim.c
index a58169a2..1eb9344 100644
--- a/drivers/mtd/nand/nandsim.c
+++ b/drivers/mtd/nand/nandsim.c
@@ -569,7 +569,7 @@
*
* RETURNS: 0 if success, -ENOMEM if memory alloc fails.
*/
-static int alloc_device(struct nandsim *ns)
+static int __init alloc_device(struct nandsim *ns)
{
struct file *cfile;
int i, err;
@@ -654,7 +654,7 @@
}
}
-static char *get_partition_name(int i)
+static char __init *get_partition_name(int i)
{
return kasprintf(GFP_KERNEL, "NAND simulator partition %d", i);
}
@@ -664,7 +664,7 @@
*
* RETURNS: 0 if success, -ERRNO if failure.
*/
-static int init_nandsim(struct mtd_info *mtd)
+static int __init init_nandsim(struct mtd_info *mtd)
{
struct nand_chip *chip = mtd_to_nand(mtd);
struct nandsim *ns = nand_get_controller_data(chip);
@@ -2261,6 +2261,7 @@
chip->read_buf = ns_nand_read_buf;
chip->read_word = ns_nand_read_word;
chip->ecc.mode = NAND_ECC_SOFT;
+ chip->ecc.algo = NAND_ECC_HAMMING;
/* The NAND_SKIP_BBTSCAN option is necessary for 'overridesize' */
/* and 'badblocks' parameters to work */
chip->options |= NAND_SKIP_BBTSCAN;
@@ -2338,7 +2339,8 @@
retval = -EINVAL;
goto error;
}
- chip->ecc.mode = NAND_ECC_SOFT_BCH;
+ chip->ecc.mode = NAND_ECC_SOFT;
+ chip->ecc.algo = NAND_ECC_BCH;
chip->ecc.size = 512;
chip->ecc.strength = bch;
chip->ecc.bytes = eccbytes;
diff --git a/drivers/mtd/nand/nuc900_nand.c b/drivers/mtd/nand/nuc900_nand.c
index dbc5b57..8f64011 100644
--- a/drivers/mtd/nand/nuc900_nand.c
+++ b/drivers/mtd/nand/nuc900_nand.c
@@ -261,6 +261,7 @@
chip->chip_delay = 50;
chip->options = 0;
chip->ecc.mode = NAND_ECC_SOFT;
+ chip->ecc.algo = NAND_ECC_HAMMING;
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
nuc900_nand->reg = devm_ioremap_resource(&pdev->dev, res);
diff --git a/drivers/mtd/nand/omap2.c b/drivers/mtd/nand/omap2.c
index 0749ca1..08e1588 100644
--- a/drivers/mtd/nand/omap2.c
+++ b/drivers/mtd/nand/omap2.c
@@ -12,6 +12,7 @@
#include <linux/dmaengine.h>
#include <linux/dma-mapping.h>
#include <linux/delay.h>
+#include <linux/gpio/consumer.h>
#include <linux/module.h>
#include <linux/interrupt.h>
#include <linux/jiffies.h>
@@ -28,6 +29,7 @@
#include <linux/mtd/nand_bch.h>
#include <linux/platform_data/elm.h>
+#include <linux/omap-gpmc.h>
#include <linux/platform_data/mtd-nand-omap2.h>
#define DRIVER_NAME "omap2-nand"
@@ -151,13 +153,17 @@
};
struct omap_nand_info {
- struct omap_nand_platform_data *pdata;
struct nand_chip nand;
struct platform_device *pdev;
int gpmc_cs;
- unsigned long phys_base;
+ bool dev_ready;
+ enum nand_io xfer_type;
+ int devsize;
enum omap_ecc ecc_opt;
+ struct device_node *elm_of_node;
+
+ unsigned long phys_base;
struct completion comp;
struct dma_chan *dma;
int gpmc_irq_fifo;
@@ -168,12 +174,14 @@
} iomode;
u_char *buf;
int buf_len;
+ /* Interface to GPMC */
struct gpmc_nand_regs reg;
- /* generated at runtime depending on ECC algorithm and layout selected */
- struct nand_ecclayout oobinfo;
+ struct gpmc_nand_ops *ops;
+ bool flash_bbt;
/* fields specific for BCHx_HW ECC scheme */
struct device *elm_dev;
- struct device_node *of_node;
+ /* NAND ready gpio */
+ struct gpio_desc *ready_gpiod;
};
static inline struct omap_nand_info *mtd_to_omap(struct mtd_info *mtd)
@@ -208,7 +216,7 @@
*/
val = ((cs << PREFETCH_CONFIG1_CS_SHIFT) |
PREFETCH_FIFOTHRESHOLD(fifo_th) | ENABLE_PREFETCH |
- (dma_mode << DMA_MPU_MODE_SHIFT) | (0x1 & is_write));
+ (dma_mode << DMA_MPU_MODE_SHIFT) | (is_write & 0x1));
writel(val, info->reg.gpmc_prefetch_config1);
/* Start the prefetch engine */
@@ -288,14 +296,13 @@
{
struct omap_nand_info *info = mtd_to_omap(mtd);
u_char *p = (u_char *)buf;
- u32 status = 0;
+ bool status;
while (len--) {
iowrite8(*p++, info->nand.IO_ADDR_W);
/* wait until buffer is available for write */
do {
- status = readl(info->reg.gpmc_status) &
- STATUS_BUFF_EMPTY;
+ status = info->ops->nand_writebuffer_empty();
} while (!status);
}
}
@@ -323,7 +330,7 @@
{
struct omap_nand_info *info = mtd_to_omap(mtd);
u16 *p = (u16 *) buf;
- u32 status = 0;
+ bool status;
/* FIXME try bursts of writesw() or DMA ... */
len >>= 1;
@@ -331,8 +338,7 @@
iowrite16(*p++, info->nand.IO_ADDR_W);
/* wait until buffer is available for write */
do {
- status = readl(info->reg.gpmc_status) &
- STATUS_BUFF_EMPTY;
+ status = info->ops->nand_writebuffer_empty();
} while (!status);
}
}
@@ -467,17 +473,8 @@
int ret;
u32 val;
- if (addr >= high_memory) {
- struct page *p1;
-
- if (((size_t)addr & PAGE_MASK) !=
- ((size_t)(addr + len - 1) & PAGE_MASK))
- goto out_copy;
- p1 = vmalloc_to_page(addr);
- if (!p1)
- goto out_copy;
- addr = page_address(p1) + ((size_t)addr & ~PAGE_MASK);
- }
+ if (!virt_addr_valid(addr))
+ goto out_copy;
sg_init_one(&sg, addr, len);
n = dma_map_sg(info->dma->device->dev, &sg, 1, dir);
@@ -497,6 +494,11 @@
tx->callback_param = &info->comp;
dmaengine_submit(tx);
+ init_completion(&info->comp);
+
+ /* setup and start DMA using dma_addr */
+ dma_async_issue_pending(info->dma);
+
/* configure and start prefetch transfer */
ret = omap_prefetch_enable(info->gpmc_cs,
PREFETCH_FIFOTHRESHOLD_MAX, 0x1, len, is_write, info);
@@ -504,10 +506,6 @@
/* PFPW engine is busy, use cpu copy method */
goto out_copy_unmap;
- init_completion(&info->comp);
- dma_async_issue_pending(info->dma);
-
- /* setup and start DMA using dma_addr */
wait_for_completion(&info->comp);
tim = 0;
limit = (loops_per_jiffy * msecs_to_jiffies(OMAP_NAND_TIMEOUT_MS));
@@ -1017,21 +1015,16 @@
}
/**
- * omap_dev_ready - calls the platform specific dev_ready function
+ * omap_dev_ready - checks the NAND Ready GPIO line
* @mtd: MTD device structure
+ *
+ * Returns true if ready and false if busy.
*/
static int omap_dev_ready(struct mtd_info *mtd)
{
- unsigned int val = 0;
struct omap_nand_info *info = mtd_to_omap(mtd);
- val = readl(info->reg.gpmc_status);
-
- if ((val & 0x100) == 0x100) {
- return 1;
- } else {
- return 0;
- }
+ return gpiod_get_value(info->ready_gpiod);
}
/**
@@ -1495,9 +1488,8 @@
static int omap_write_page_bch(struct mtd_info *mtd, struct nand_chip *chip,
const uint8_t *buf, int oob_required, int page)
{
- int i;
+ int ret;
uint8_t *ecc_calc = chip->buffers->ecccalc;
- uint32_t *eccpos = chip->ecc.layout->eccpos;
/* Enable GPMC ecc engine */
chip->ecc.hwctl(mtd, NAND_ECC_WRITE);
@@ -1508,8 +1500,10 @@
/* Update ecc vector from GPMC result registers */
chip->ecc.calculate(mtd, buf, &ecc_calc[0]);
- for (i = 0; i < chip->ecc.total; i++)
- chip->oob_poi[eccpos[i]] = ecc_calc[i];
+ ret = mtd_ooblayout_set_eccbytes(mtd, ecc_calc, chip->oob_poi, 0,
+ chip->ecc.total);
+ if (ret)
+ return ret;
/* Write ecc vector to OOB area */
chip->write_buf(mtd, chip->oob_poi, mtd->oobsize);
@@ -1536,10 +1530,7 @@
{
uint8_t *ecc_calc = chip->buffers->ecccalc;
uint8_t *ecc_code = chip->buffers->ecccode;
- uint32_t *eccpos = chip->ecc.layout->eccpos;
- uint8_t *oob = &chip->oob_poi[eccpos[0]];
- uint32_t oob_pos = mtd->writesize + chip->ecc.layout->eccpos[0];
- int stat;
+ int stat, ret;
unsigned int max_bitflips = 0;
/* Enable GPMC ecc engine */
@@ -1549,13 +1540,18 @@
chip->read_buf(mtd, buf, mtd->writesize);
/* Read oob bytes */
- chip->cmdfunc(mtd, NAND_CMD_RNDOUT, oob_pos, -1);
- chip->read_buf(mtd, oob, chip->ecc.total);
+ chip->cmdfunc(mtd, NAND_CMD_RNDOUT,
+ mtd->writesize + BADBLOCK_MARKER_LENGTH, -1);
+ chip->read_buf(mtd, chip->oob_poi + BADBLOCK_MARKER_LENGTH,
+ chip->ecc.total);
/* Calculate ecc bytes */
chip->ecc.calculate(mtd, buf, ecc_calc);
- memcpy(ecc_code, &chip->oob_poi[eccpos[0]], chip->ecc.total);
+ ret = mtd_ooblayout_get_eccbytes(mtd, ecc_code, chip->oob_poi, 0,
+ chip->ecc.total);
+ if (ret)
+ return ret;
stat = chip->ecc.correct(mtd, buf, ecc_code, ecc_calc);
@@ -1630,7 +1626,7 @@
"CONFIG_MTD_NAND_OMAP_BCH not enabled\n");
return false;
}
- if (ecc_needs_elm && !is_elm_present(info, pdata->elm_of_node)) {
+ if (ecc_needs_elm && !is_elm_present(info, info->elm_of_node)) {
dev_err(&info->pdev->dev, "ELM not available\n");
return false;
}
@@ -1638,43 +1634,227 @@
return true;
}
+static const char * const nand_xfer_types[] = {
+ [NAND_OMAP_PREFETCH_POLLED] = "prefetch-polled",
+ [NAND_OMAP_POLLED] = "polled",
+ [NAND_OMAP_PREFETCH_DMA] = "prefetch-dma",
+ [NAND_OMAP_PREFETCH_IRQ] = "prefetch-irq",
+};
+
+static int omap_get_dt_info(struct device *dev, struct omap_nand_info *info)
+{
+ struct device_node *child = dev->of_node;
+ int i;
+ const char *s;
+ u32 cs;
+
+ if (of_property_read_u32(child, "reg", &cs) < 0) {
+ dev_err(dev, "reg not found in DT\n");
+ return -EINVAL;
+ }
+
+ info->gpmc_cs = cs;
+
+ /* detect availability of ELM module. Won't be present pre-OMAP4 */
+ info->elm_of_node = of_parse_phandle(child, "ti,elm-id", 0);
+ if (!info->elm_of_node)
+ dev_dbg(dev, "ti,elm-id not in DT\n");
+
+ /* select ecc-scheme for NAND */
+ if (of_property_read_string(child, "ti,nand-ecc-opt", &s)) {
+ dev_err(dev, "ti,nand-ecc-opt not found\n");
+ return -EINVAL;
+ }
+
+ if (!strcmp(s, "sw")) {
+ info->ecc_opt = OMAP_ECC_HAM1_CODE_SW;
+ } else if (!strcmp(s, "ham1") ||
+ !strcmp(s, "hw") || !strcmp(s, "hw-romcode")) {
+ info->ecc_opt = OMAP_ECC_HAM1_CODE_HW;
+ } else if (!strcmp(s, "bch4")) {
+ if (info->elm_of_node)
+ info->ecc_opt = OMAP_ECC_BCH4_CODE_HW;
+ else
+ info->ecc_opt = OMAP_ECC_BCH4_CODE_HW_DETECTION_SW;
+ } else if (!strcmp(s, "bch8")) {
+ if (info->elm_of_node)
+ info->ecc_opt = OMAP_ECC_BCH8_CODE_HW;
+ else
+ info->ecc_opt = OMAP_ECC_BCH8_CODE_HW_DETECTION_SW;
+ } else if (!strcmp(s, "bch16")) {
+ info->ecc_opt = OMAP_ECC_BCH16_CODE_HW;
+ } else {
+ dev_err(dev, "unrecognized value for ti,nand-ecc-opt\n");
+ return -EINVAL;
+ }
+
+ /* select data transfer mode */
+ if (!of_property_read_string(child, "ti,nand-xfer-type", &s)) {
+ for (i = 0; i < ARRAY_SIZE(nand_xfer_types); i++) {
+ if (!strcasecmp(s, nand_xfer_types[i])) {
+ info->xfer_type = i;
+ return 0;
+ }
+ }
+
+ dev_err(dev, "unrecognized value for ti,nand-xfer-type\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int omap_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct omap_nand_info *info = mtd_to_omap(mtd);
+ struct nand_chip *chip = &info->nand;
+ int off = BADBLOCK_MARKER_LENGTH;
+
+ if (info->ecc_opt == OMAP_ECC_HAM1_CODE_HW &&
+ !(chip->options & NAND_BUSWIDTH_16))
+ off = 1;
+
+ if (section)
+ return -ERANGE;
+
+ oobregion->offset = off;
+ oobregion->length = chip->ecc.total;
+
+ return 0;
+}
+
+static int omap_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct omap_nand_info *info = mtd_to_omap(mtd);
+ struct nand_chip *chip = &info->nand;
+ int off = BADBLOCK_MARKER_LENGTH;
+
+ if (info->ecc_opt == OMAP_ECC_HAM1_CODE_HW &&
+ !(chip->options & NAND_BUSWIDTH_16))
+ off = 1;
+
+ if (section)
+ return -ERANGE;
+
+ off += chip->ecc.total;
+ if (off >= mtd->oobsize)
+ return -ERANGE;
+
+ oobregion->offset = off;
+ oobregion->length = mtd->oobsize - off;
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops omap_ooblayout_ops = {
+ .ecc = omap_ooblayout_ecc,
+ .free = omap_ooblayout_free,
+};
+
+static int omap_sw_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+ int off = BADBLOCK_MARKER_LENGTH;
+
+ if (section >= chip->ecc.steps)
+ return -ERANGE;
+
+ /*
+ * When SW correction is employed, one OMAP specific marker byte is
+ * reserved after each ECC step.
+ */
+ oobregion->offset = off + (section * (chip->ecc.bytes + 1));
+ oobregion->length = chip->ecc.bytes;
+
+ return 0;
+}
+
+static int omap_sw_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+ int off = BADBLOCK_MARKER_LENGTH;
+
+ if (section)
+ return -ERANGE;
+
+ /*
+ * When SW correction is employed, one OMAP specific marker byte is
+ * reserved after each ECC step.
+ */
+ off += ((chip->ecc.bytes + 1) * chip->ecc.steps);
+ if (off >= mtd->oobsize)
+ return -ERANGE;
+
+ oobregion->offset = off;
+ oobregion->length = mtd->oobsize - off;
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops omap_sw_ooblayout_ops = {
+ .ecc = omap_sw_ooblayout_ecc,
+ .free = omap_sw_ooblayout_free,
+};
+
static int omap_nand_probe(struct platform_device *pdev)
{
struct omap_nand_info *info;
- struct omap_nand_platform_data *pdata;
+ struct omap_nand_platform_data *pdata = NULL;
struct mtd_info *mtd;
struct nand_chip *nand_chip;
- struct nand_ecclayout *ecclayout;
int err;
- int i;
dma_cap_mask_t mask;
unsigned sig;
- unsigned oob_index;
struct resource *res;
-
- pdata = dev_get_platdata(&pdev->dev);
- if (pdata == NULL) {
- dev_err(&pdev->dev, "platform data missing\n");
- return -ENODEV;
- }
+ struct device *dev = &pdev->dev;
+ int min_oobbytes = BADBLOCK_MARKER_LENGTH;
+ int oobbytes_per_step;
info = devm_kzalloc(&pdev->dev, sizeof(struct omap_nand_info),
GFP_KERNEL);
if (!info)
return -ENOMEM;
- platform_set_drvdata(pdev, info);
+ info->pdev = pdev;
- info->pdev = pdev;
- info->gpmc_cs = pdata->cs;
- info->reg = pdata->reg;
- info->of_node = pdata->of_node;
- info->ecc_opt = pdata->ecc_opt;
+ if (dev->of_node) {
+ if (omap_get_dt_info(dev, info))
+ return -EINVAL;
+ } else {
+ pdata = dev_get_platdata(&pdev->dev);
+ if (!pdata) {
+ dev_err(&pdev->dev, "platform data missing\n");
+ return -EINVAL;
+ }
+
+ info->gpmc_cs = pdata->cs;
+ info->reg = pdata->reg;
+ info->ecc_opt = pdata->ecc_opt;
+ if (pdata->dev_ready)
+ dev_info(&pdev->dev, "pdata->dev_ready is deprecated\n");
+
+ info->xfer_type = pdata->xfer_type;
+ info->devsize = pdata->devsize;
+ info->elm_of_node = pdata->elm_of_node;
+ info->flash_bbt = pdata->flash_bbt;
+ }
+
+ platform_set_drvdata(pdev, info);
+ info->ops = gpmc_omap_get_nand_ops(&info->reg, info->gpmc_cs);
+ if (!info->ops) {
+ dev_err(&pdev->dev, "Failed to get GPMC->NAND interface\n");
+ return -ENODEV;
+ }
+
nand_chip = &info->nand;
mtd = nand_to_mtd(nand_chip);
mtd->dev.parent = &pdev->dev;
nand_chip->ecc.priv = NULL;
- nand_set_flash_node(nand_chip, pdata->of_node);
+ nand_set_flash_node(nand_chip, dev->of_node);
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
nand_chip->IO_ADDR_R = devm_ioremap_resource(&pdev->dev, res);
@@ -1688,6 +1868,13 @@
nand_chip->IO_ADDR_W = nand_chip->IO_ADDR_R;
nand_chip->cmd_ctrl = omap_hwcontrol;
+ info->ready_gpiod = devm_gpiod_get_optional(&pdev->dev, "rb",
+ GPIOD_IN);
+ if (IS_ERR(info->ready_gpiod)) {
+ dev_err(dev, "failed to get ready gpio\n");
+ return PTR_ERR(info->ready_gpiod);
+ }
+
/*
* If RDY/BSY line is connected to OMAP then use the omap ready
* function and the generic nand_wait function which reads the status
@@ -1695,7 +1882,7 @@
* chip delay which is slightly more than tR (AC Timing) of the NAND
* device and read status register until you get a failure or success
*/
- if (pdata->dev_ready) {
+ if (info->ready_gpiod) {
nand_chip->dev_ready = omap_dev_ready;
nand_chip->chip_delay = 0;
} else {
@@ -1703,21 +1890,25 @@
nand_chip->chip_delay = 50;
}
- if (pdata->flash_bbt)
- nand_chip->bbt_options |= NAND_BBT_USE_FLASH | NAND_BBT_NO_OOB;
- else
- nand_chip->options |= NAND_SKIP_BBTSCAN;
+ if (info->flash_bbt)
+ nand_chip->bbt_options |= NAND_BBT_USE_FLASH;
/* scan NAND device connected to chip controller */
- nand_chip->options |= pdata->devsize & NAND_BUSWIDTH_16;
+ nand_chip->options |= info->devsize & NAND_BUSWIDTH_16;
if (nand_scan_ident(mtd, 1, NULL)) {
- dev_err(&info->pdev->dev, "scan failed, may be bus-width mismatch\n");
+ dev_err(&info->pdev->dev,
+ "scan failed, may be bus-width mismatch\n");
err = -ENXIO;
goto return_error;
}
+ if (nand_chip->bbt_options & NAND_BBT_USE_FLASH)
+ nand_chip->bbt_options |= NAND_BBT_NO_OOB;
+ else
+ nand_chip->options |= NAND_SKIP_BBTSCAN;
+
/* re-populate low-level callbacks based on xfer modes */
- switch (pdata->xfer_type) {
+ switch (info->xfer_type) {
case NAND_OMAP_PREFETCH_POLLED:
nand_chip->read_buf = omap_read_buf_pref;
nand_chip->write_buf = omap_write_buf_pref;
@@ -1797,7 +1988,7 @@
default:
dev_err(&pdev->dev,
- "xfer_type(%d) not supported!\n", pdata->xfer_type);
+ "xfer_type(%d) not supported!\n", info->xfer_type);
err = -EINVAL;
goto return_error;
}
@@ -1809,16 +2000,15 @@
/*
* Bail out earlier to let NAND_ECC_SOFT code create its own
- * ecclayout instead of using ours.
+ * ooblayout instead of using ours.
*/
if (info->ecc_opt == OMAP_ECC_HAM1_CODE_SW) {
nand_chip->ecc.mode = NAND_ECC_SOFT;
+ nand_chip->ecc.algo = NAND_ECC_HAMMING;
goto scan_tail;
}
/* populate MTD interface based on ECC scheme */
- ecclayout = &info->oobinfo;
- nand_chip->ecc.layout = ecclayout;
switch (info->ecc_opt) {
case OMAP_ECC_HAM1_CODE_HW:
pr_info("nand: using OMAP_ECC_HAM1_CODE_HW\n");
@@ -1829,19 +2019,12 @@
nand_chip->ecc.calculate = omap_calculate_ecc;
nand_chip->ecc.hwctl = omap_enable_hwecc;
nand_chip->ecc.correct = omap_correct_data;
- /* define ECC layout */
- ecclayout->eccbytes = nand_chip->ecc.bytes *
- (mtd->writesize /
- nand_chip->ecc.size);
- if (nand_chip->options & NAND_BUSWIDTH_16)
- oob_index = BADBLOCK_MARKER_LENGTH;
- else
- oob_index = 1;
- for (i = 0; i < ecclayout->eccbytes; i++, oob_index++)
- ecclayout->eccpos[i] = oob_index;
- /* no reserved-marker in ecclayout for this ecc-scheme */
- ecclayout->oobfree->offset =
- ecclayout->eccpos[ecclayout->eccbytes - 1] + 1;
+ mtd_set_ooblayout(mtd, &omap_ooblayout_ops);
+ oobbytes_per_step = nand_chip->ecc.bytes;
+
+ if (!(nand_chip->options & NAND_BUSWIDTH_16))
+ min_oobbytes = 1;
+
break;
case OMAP_ECC_BCH4_CODE_HW_DETECTION_SW:
@@ -1853,19 +2036,9 @@
nand_chip->ecc.hwctl = omap_enable_hwecc_bch;
nand_chip->ecc.correct = nand_bch_correct_data;
nand_chip->ecc.calculate = omap_calculate_ecc_bch;
- /* define ECC layout */
- ecclayout->eccbytes = nand_chip->ecc.bytes *
- (mtd->writesize /
- nand_chip->ecc.size);
- oob_index = BADBLOCK_MARKER_LENGTH;
- for (i = 0; i < ecclayout->eccbytes; i++, oob_index++) {
- ecclayout->eccpos[i] = oob_index;
- if (((i + 1) % nand_chip->ecc.bytes) == 0)
- oob_index++;
- }
- /* include reserved-marker in ecclayout->oobfree calculation */
- ecclayout->oobfree->offset = 1 +
- ecclayout->eccpos[ecclayout->eccbytes - 1] + 1;
+ mtd_set_ooblayout(mtd, &omap_sw_ooblayout_ops);
+ /* Reserve one byte for the OMAP marker */
+ oobbytes_per_step = nand_chip->ecc.bytes + 1;
/* software bch library is used for locating errors */
nand_chip->ecc.priv = nand_bch_init(mtd);
if (!nand_chip->ecc.priv) {
@@ -1887,16 +2060,8 @@
nand_chip->ecc.calculate = omap_calculate_ecc_bch;
nand_chip->ecc.read_page = omap_read_page_bch;
nand_chip->ecc.write_page = omap_write_page_bch;
- /* define ECC layout */
- ecclayout->eccbytes = nand_chip->ecc.bytes *
- (mtd->writesize /
- nand_chip->ecc.size);
- oob_index = BADBLOCK_MARKER_LENGTH;
- for (i = 0; i < ecclayout->eccbytes; i++, oob_index++)
- ecclayout->eccpos[i] = oob_index;
- /* reserved marker already included in ecclayout->eccbytes */
- ecclayout->oobfree->offset =
- ecclayout->eccpos[ecclayout->eccbytes - 1] + 1;
+ mtd_set_ooblayout(mtd, &omap_ooblayout_ops);
+ oobbytes_per_step = nand_chip->ecc.bytes;
err = elm_config(info->elm_dev, BCH4_ECC,
mtd->writesize / nand_chip->ecc.size,
@@ -1914,19 +2079,9 @@
nand_chip->ecc.hwctl = omap_enable_hwecc_bch;
nand_chip->ecc.correct = nand_bch_correct_data;
nand_chip->ecc.calculate = omap_calculate_ecc_bch;
- /* define ECC layout */
- ecclayout->eccbytes = nand_chip->ecc.bytes *
- (mtd->writesize /
- nand_chip->ecc.size);
- oob_index = BADBLOCK_MARKER_LENGTH;
- for (i = 0; i < ecclayout->eccbytes; i++, oob_index++) {
- ecclayout->eccpos[i] = oob_index;
- if (((i + 1) % nand_chip->ecc.bytes) == 0)
- oob_index++;
- }
- /* include reserved-marker in ecclayout->oobfree calculation */
- ecclayout->oobfree->offset = 1 +
- ecclayout->eccpos[ecclayout->eccbytes - 1] + 1;
+ mtd_set_ooblayout(mtd, &omap_sw_ooblayout_ops);
+ /* Reserve one byte for the OMAP marker */
+ oobbytes_per_step = nand_chip->ecc.bytes + 1;
/* software bch library is used for locating errors */
nand_chip->ecc.priv = nand_bch_init(mtd);
if (!nand_chip->ecc.priv) {
@@ -1948,6 +2103,8 @@
nand_chip->ecc.calculate = omap_calculate_ecc_bch;
nand_chip->ecc.read_page = omap_read_page_bch;
nand_chip->ecc.write_page = omap_write_page_bch;
+ mtd_set_ooblayout(mtd, &omap_ooblayout_ops);
+ oobbytes_per_step = nand_chip->ecc.bytes;
err = elm_config(info->elm_dev, BCH8_ECC,
mtd->writesize / nand_chip->ecc.size,
@@ -1955,16 +2112,6 @@
if (err < 0)
goto return_error;
- /* define ECC layout */
- ecclayout->eccbytes = nand_chip->ecc.bytes *
- (mtd->writesize /
- nand_chip->ecc.size);
- oob_index = BADBLOCK_MARKER_LENGTH;
- for (i = 0; i < ecclayout->eccbytes; i++, oob_index++)
- ecclayout->eccpos[i] = oob_index;
- /* reserved marker already included in ecclayout->eccbytes */
- ecclayout->oobfree->offset =
- ecclayout->eccpos[ecclayout->eccbytes - 1] + 1;
break;
case OMAP_ECC_BCH16_CODE_HW:
@@ -1978,6 +2125,8 @@
nand_chip->ecc.calculate = omap_calculate_ecc_bch;
nand_chip->ecc.read_page = omap_read_page_bch;
nand_chip->ecc.write_page = omap_write_page_bch;
+ mtd_set_ooblayout(mtd, &omap_ooblayout_ops);
+ oobbytes_per_step = nand_chip->ecc.bytes;
err = elm_config(info->elm_dev, BCH16_ECC,
mtd->writesize / nand_chip->ecc.size,
@@ -1985,16 +2134,6 @@
if (err < 0)
goto return_error;
- /* define ECC layout */
- ecclayout->eccbytes = nand_chip->ecc.bytes *
- (mtd->writesize /
- nand_chip->ecc.size);
- oob_index = BADBLOCK_MARKER_LENGTH;
- for (i = 0; i < ecclayout->eccbytes; i++, oob_index++)
- ecclayout->eccpos[i] = oob_index;
- /* reserved marker already included in ecclayout->eccbytes */
- ecclayout->oobfree->offset =
- ecclayout->eccpos[ecclayout->eccbytes - 1] + 1;
break;
default:
dev_err(&info->pdev->dev, "invalid or unsupported ECC scheme\n");
@@ -2002,13 +2141,13 @@
goto return_error;
}
- /* all OOB bytes from oobfree->offset till end off OOB are free */
- ecclayout->oobfree->length = mtd->oobsize - ecclayout->oobfree->offset;
/* check if NAND device's OOB is enough to store ECC signatures */
- if (mtd->oobsize < (ecclayout->eccbytes + BADBLOCK_MARKER_LENGTH)) {
+ min_oobbytes += (oobbytes_per_step *
+ (mtd->writesize / nand_chip->ecc.size));
+ if (mtd->oobsize < min_oobbytes) {
dev_err(&info->pdev->dev,
"not enough OOB bytes required = %d, available=%d\n",
- ecclayout->eccbytes, mtd->oobsize);
+ min_oobbytes, mtd->oobsize);
err = -EINVAL;
goto return_error;
}
@@ -2020,7 +2159,10 @@
goto return_error;
}
- mtd_device_register(mtd, pdata->parts, pdata->nr_parts);
+ if (dev->of_node)
+ mtd_device_register(mtd, NULL, 0);
+ else
+ mtd_device_register(mtd, pdata->parts, pdata->nr_parts);
platform_set_drvdata(pdev, mtd);
@@ -2051,11 +2193,17 @@
return 0;
}
+static const struct of_device_id omap_nand_ids[] = {
+ { .compatible = "ti,omap2-nand", },
+ {},
+};
+
static struct platform_driver omap_nand_driver = {
.probe = omap_nand_probe,
.remove = omap_nand_remove,
.driver = {
.name = DRIVER_NAME,
+ .of_match_table = of_match_ptr(omap_nand_ids),
},
};
diff --git a/drivers/mtd/nand/orion_nand.c b/drivers/mtd/nand/orion_nand.c
index d4614bf..40a7c4a 100644
--- a/drivers/mtd/nand/orion_nand.c
+++ b/drivers/mtd/nand/orion_nand.c
@@ -130,6 +130,7 @@
nc->cmd_ctrl = orion_nand_cmd_ctrl;
nc->read_buf = orion_nand_read_buf;
nc->ecc.mode = NAND_ECC_SOFT;
+ nc->ecc.algo = NAND_ECC_HAMMING;
if (board->chip_delay)
nc->chip_delay = board->chip_delay;
diff --git a/drivers/mtd/nand/pasemi_nand.c b/drivers/mtd/nand/pasemi_nand.c
index 3ab53ca..5de7591 100644
--- a/drivers/mtd/nand/pasemi_nand.c
+++ b/drivers/mtd/nand/pasemi_nand.c
@@ -92,8 +92,9 @@
static int pasemi_nand_probe(struct platform_device *ofdev)
{
+ struct device *dev = &ofdev->dev;
struct pci_dev *pdev;
- struct device_node *np = ofdev->dev.of_node;
+ struct device_node *np = dev->of_node;
struct resource res;
struct nand_chip *chip;
int err = 0;
@@ -107,13 +108,11 @@
if (pasemi_nand_mtd)
return -ENODEV;
- pr_debug("pasemi_nand at %pR\n", &res);
+ dev_dbg(dev, "pasemi_nand at %pR\n", &res);
/* Allocate memory for MTD device structure and private data */
chip = kzalloc(sizeof(struct nand_chip), GFP_KERNEL);
if (!chip) {
- printk(KERN_WARNING
- "Unable to allocate PASEMI NAND MTD device structure\n");
err = -ENOMEM;
goto out;
}
@@ -121,7 +120,7 @@
pasemi_nand_mtd = nand_to_mtd(chip);
/* Link the private data with the MTD structure */
- pasemi_nand_mtd->dev.parent = &ofdev->dev;
+ pasemi_nand_mtd->dev.parent = dev;
chip->IO_ADDR_R = of_iomap(np, 0);
chip->IO_ADDR_W = chip->IO_ADDR_R;
@@ -151,6 +150,7 @@
chip->write_buf = pasemi_write_buf;
chip->chip_delay = 0;
chip->ecc.mode = NAND_ECC_SOFT;
+ chip->ecc.algo = NAND_ECC_HAMMING;
/* Enable the following for a flash based bad block table */
chip->bbt_options = NAND_BBT_USE_FLASH;
@@ -162,13 +162,13 @@
}
if (mtd_device_register(pasemi_nand_mtd, NULL, 0)) {
- printk(KERN_ERR "pasemi_nand: Unable to register MTD device\n");
+ dev_err(dev, "Unable to register MTD device\n");
err = -ENODEV;
goto out_lpc;
}
- printk(KERN_INFO "PA Semi NAND flash at %08llx, control at I/O %x\n",
- res.start, lpcctl);
+ dev_info(dev, "PA Semi NAND flash at %pR, control at I/O %x\n", &res,
+ lpcctl);
return 0;
diff --git a/drivers/mtd/nand/plat_nand.c b/drivers/mtd/nand/plat_nand.c
index e4e50da..415a53a 100644
--- a/drivers/mtd/nand/plat_nand.c
+++ b/drivers/mtd/nand/plat_nand.c
@@ -74,6 +74,7 @@
data->chip.ecc.hwctl = pdata->ctrl.hwcontrol;
data->chip.ecc.mode = NAND_ECC_SOFT;
+ data->chip.ecc.algo = NAND_ECC_HAMMING;
platform_set_drvdata(pdev, data);
diff --git a/drivers/mtd/nand/pxa3xx_nand.c b/drivers/mtd/nand/pxa3xx_nand.c
index d650885..436dd6d 100644
--- a/drivers/mtd/nand/pxa3xx_nand.c
+++ b/drivers/mtd/nand/pxa3xx_nand.c
@@ -29,7 +29,6 @@
#include <linux/slab.h>
#include <linux/of.h>
#include <linux/of_device.h>
-#include <linux/of_mtd.h>
#include <linux/platform_data/mtd-nand-pxa3xx.h>
#define CHIP_DELAY_TIMEOUT msecs_to_jiffies(200)
@@ -324,6 +323,62 @@
{ 0xba20, 16, 16, &timing[3] },
};
+static int pxa3xx_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+ struct pxa3xx_nand_host *host = nand_get_controller_data(chip);
+ struct pxa3xx_nand_info *info = host->info_data;
+ int nchunks = mtd->writesize / info->chunk_size;
+
+ if (section >= nchunks)
+ return -ERANGE;
+
+ oobregion->offset = ((info->ecc_size + info->spare_size) * section) +
+ info->spare_size;
+ oobregion->length = info->ecc_size;
+
+ return 0;
+}
+
+static int pxa3xx_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+ struct pxa3xx_nand_host *host = nand_get_controller_data(chip);
+ struct pxa3xx_nand_info *info = host->info_data;
+ int nchunks = mtd->writesize / info->chunk_size;
+
+ if (section >= nchunks)
+ return -ERANGE;
+
+ if (!info->spare_size)
+ return 0;
+
+ oobregion->offset = section * (info->ecc_size + info->spare_size);
+ oobregion->length = info->spare_size;
+ if (!section) {
+ /*
+ * Bootrom looks in bytes 0 & 5 for bad blocks for the
+ * 4KB page / 4bit BCH combination.
+ */
+ if (mtd->writesize == 4096 && info->chunk_size == 2048) {
+ oobregion->offset += 6;
+ oobregion->length -= 6;
+ } else {
+ oobregion->offset += 2;
+ oobregion->length -= 2;
+ }
+ }
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops pxa3xx_ooblayout_ops = {
+ .ecc = pxa3xx_ooblayout_ecc,
+ .free = pxa3xx_ooblayout_free,
+};
+
static u8 bbt_pattern[] = {'M', 'V', 'B', 'b', 't', '0' };
static u8 bbt_mirror_pattern[] = {'1', 't', 'b', 'B', 'V', 'M' };
@@ -347,41 +402,6 @@
.pattern = bbt_mirror_pattern
};
-static struct nand_ecclayout ecc_layout_2KB_bch4bit = {
- .eccbytes = 32,
- .eccpos = {
- 32, 33, 34, 35, 36, 37, 38, 39,
- 40, 41, 42, 43, 44, 45, 46, 47,
- 48, 49, 50, 51, 52, 53, 54, 55,
- 56, 57, 58, 59, 60, 61, 62, 63},
- .oobfree = { {2, 30} }
-};
-
-static struct nand_ecclayout ecc_layout_4KB_bch4bit = {
- .eccbytes = 64,
- .eccpos = {
- 32, 33, 34, 35, 36, 37, 38, 39,
- 40, 41, 42, 43, 44, 45, 46, 47,
- 48, 49, 50, 51, 52, 53, 54, 55,
- 56, 57, 58, 59, 60, 61, 62, 63,
- 96, 97, 98, 99, 100, 101, 102, 103,
- 104, 105, 106, 107, 108, 109, 110, 111,
- 112, 113, 114, 115, 116, 117, 118, 119,
- 120, 121, 122, 123, 124, 125, 126, 127},
- /* Bootrom looks in bytes 0 & 5 for bad blocks */
- .oobfree = { {6, 26}, { 64, 32} }
-};
-
-static struct nand_ecclayout ecc_layout_4KB_bch8bit = {
- .eccbytes = 128,
- .eccpos = {
- 32, 33, 34, 35, 36, 37, 38, 39,
- 40, 41, 42, 43, 44, 45, 46, 47,
- 48, 49, 50, 51, 52, 53, 54, 55,
- 56, 57, 58, 59, 60, 61, 62, 63},
- .oobfree = { }
-};
-
#define NDTR0_tCH(c) (min((c), 7) << 19)
#define NDTR0_tCS(c) (min((c), 7) << 16)
#define NDTR0_tWH(c) (min((c), 7) << 11)
@@ -1546,9 +1566,12 @@
}
static int pxa_ecc_init(struct pxa3xx_nand_info *info,
- struct nand_ecc_ctrl *ecc,
+ struct mtd_info *mtd,
int strength, int ecc_stepsize, int page_size)
{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+ struct nand_ecc_ctrl *ecc = &chip->ecc;
+
if (strength == 1 && ecc_stepsize == 512 && page_size == 2048) {
info->nfullchunks = 1;
info->ntotalchunks = 1;
@@ -1582,7 +1605,7 @@
info->ecc_size = 32;
ecc->mode = NAND_ECC_HW;
ecc->size = info->chunk_size;
- ecc->layout = &ecc_layout_2KB_bch4bit;
+ mtd_set_ooblayout(mtd, &pxa3xx_ooblayout_ops);
ecc->strength = 16;
} else if (strength == 4 && ecc_stepsize == 512 && page_size == 4096) {
@@ -1594,7 +1617,7 @@
info->ecc_size = 32;
ecc->mode = NAND_ECC_HW;
ecc->size = info->chunk_size;
- ecc->layout = &ecc_layout_4KB_bch4bit;
+ mtd_set_ooblayout(mtd, &pxa3xx_ooblayout_ops);
ecc->strength = 16;
/*
@@ -1612,7 +1635,7 @@
info->ecc_size = 32;
ecc->mode = NAND_ECC_HW;
ecc->size = info->chunk_size;
- ecc->layout = &ecc_layout_4KB_bch8bit;
+ mtd_set_ooblayout(mtd, &pxa3xx_ooblayout_ops);
ecc->strength = 16;
} else {
dev_err(&info->pdev->dev,
@@ -1651,6 +1674,12 @@
if (info->variant == PXA3XX_NAND_VARIANT_ARMADA370)
nand_writel(info, NDECCCTRL, 0x0);
+ if (pdata->flash_bbt)
+ chip->bbt_options |= NAND_BBT_USE_FLASH;
+
+ chip->ecc.strength = pdata->ecc_strength;
+ chip->ecc.size = pdata->ecc_step_size;
+
if (nand_scan_ident(mtd, 1, NULL))
return -ENODEV;
@@ -1663,13 +1692,12 @@
}
}
- if (pdata->flash_bbt) {
+ if (chip->bbt_options & NAND_BBT_USE_FLASH) {
/*
* We'll use a bad block table stored in-flash and don't
* allow writing the bad block marker to the flash.
*/
- chip->bbt_options |= NAND_BBT_USE_FLASH |
- NAND_BBT_NO_OOB_BBM;
+ chip->bbt_options |= NAND_BBT_NO_OOB_BBM;
chip->bbt_td = &bbt_main_descr;
chip->bbt_md = &bbt_mirror_descr;
}
@@ -1689,10 +1717,9 @@
}
}
- if (pdata->ecc_strength && pdata->ecc_step_size) {
- ecc_strength = pdata->ecc_strength;
- ecc_step = pdata->ecc_step_size;
- } else {
+ ecc_strength = chip->ecc.strength;
+ ecc_step = chip->ecc.size;
+ if (!ecc_strength || !ecc_step) {
ecc_strength = chip->ecc_strength_ds;
ecc_step = chip->ecc_step_ds;
}
@@ -1703,7 +1730,7 @@
ecc_step = 512;
}
- ret = pxa_ecc_init(info, &chip->ecc, ecc_strength,
+ ret = pxa_ecc_init(info, mtd, ecc_strength,
ecc_step, mtd->writesize);
if (ret)
return ret;
@@ -1903,15 +1930,6 @@
if (of_get_property(np, "marvell,nand-keep-config", NULL))
pdata->keep_config = 1;
of_property_read_u32(np, "num-cs", &pdata->num_cs);
- pdata->flash_bbt = of_get_nand_on_flash_bbt(np);
-
- pdata->ecc_strength = of_get_nand_ecc_strength(np);
- if (pdata->ecc_strength < 0)
- pdata->ecc_strength = 0;
-
- pdata->ecc_step_size = of_get_nand_ecc_step_size(np);
- if (pdata->ecc_step_size < 0)
- pdata->ecc_step_size = 0;
pdev->dev.platform_data = pdata;
diff --git a/drivers/mtd/nand/qcom_nandc.c b/drivers/mtd/nand/qcom_nandc.c
index f550a57..de7d28e 100644
--- a/drivers/mtd/nand/qcom_nandc.c
+++ b/drivers/mtd/nand/qcom_nandc.c
@@ -21,7 +21,6 @@
#include <linux/mtd/partitions.h>
#include <linux/of.h>
#include <linux/of_device.h>
-#include <linux/of_mtd.h>
#include <linux/delay.h>
/* NANDc reg offsets */
@@ -1437,7 +1436,6 @@
struct qcom_nand_controller *nandc = get_qcom_nand_controller(chip);
struct nand_ecc_ctrl *ecc = &chip->ecc;
u8 *oob = chip->oob_poi;
- int free_boff;
int data_size, oob_size;
int ret, status = 0;
@@ -1451,12 +1449,11 @@
/* calculate the data and oob size for the last codeword/step */
data_size = ecc->size - ((ecc->steps - 1) << 2);
- oob_size = ecc->steps << 2;
-
- free_boff = ecc->layout->oobfree[0].offset;
+ oob_size = mtd->oobavail;
/* override new oob content to last codeword */
- memcpy(nandc->data_buffer + data_size, oob + free_boff, oob_size);
+ mtd_ooblayout_get_databytes(mtd, nandc->data_buffer + data_size, oob,
+ 0, mtd->oobavail);
set_address(host, host->cw_size * (ecc->steps - 1), page);
update_rw_regs(host, 1, false);
@@ -1710,61 +1707,52 @@
* This layout is read as is when ECC is disabled. When ECC is enabled, the
* inaccessible Bad Block byte(s) are ignored when we write to a page/oob,
* and assumed as 0xffs when we read a page/oob. The ECC, unused and
- * dummy/real bad block bytes are grouped as ecc bytes in nand_ecclayout (i.e,
- * ecc->bytes is the sum of the three).
+ * dummy/real bad block bytes are grouped as ecc bytes (i.e, ecc->bytes is
+ * the sum of the three).
*/
-
-static struct nand_ecclayout *
-qcom_nand_create_layout(struct qcom_nand_host *host)
+static int qcom_nand_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
{
- struct nand_chip *chip = &host->chip;
- struct mtd_info *mtd = nand_to_mtd(chip);
- struct qcom_nand_controller *nandc = get_qcom_nand_controller(chip);
+ struct nand_chip *chip = mtd_to_nand(mtd);
+ struct qcom_nand_host *host = to_qcom_nand_host(chip);
struct nand_ecc_ctrl *ecc = &chip->ecc;
- struct nand_ecclayout *layout;
- int i, j, steps, pos = 0, shift = 0;
- layout = devm_kzalloc(nandc->dev, sizeof(*layout), GFP_KERNEL);
- if (!layout)
- return NULL;
+ if (section > 1)
+ return -ERANGE;
- steps = mtd->writesize / ecc->size;
- layout->eccbytes = steps * ecc->bytes;
-
- layout->oobfree[0].offset = (steps - 1) * ecc->bytes + host->bbm_size;
- layout->oobfree[0].length = steps << 2;
-
- /*
- * the oob bytes in the first n - 1 codewords are all grouped together
- * in the format:
- * DUMMY_BBM + UNUSED + ECC
- */
- for (i = 0; i < steps - 1; i++) {
- for (j = 0; j < ecc->bytes; j++)
- layout->eccpos[pos++] = i * ecc->bytes + j;
+ if (!section) {
+ oobregion->length = (ecc->bytes * (ecc->steps - 1)) +
+ host->bbm_size;
+ oobregion->offset = 0;
+ } else {
+ oobregion->length = host->ecc_bytes_hw + host->spare_bytes;
+ oobregion->offset = mtd->oobsize - oobregion->length;
}
- /*
- * the oob bytes in the last codeword are grouped in the format:
- * BBM + FREE OOB + UNUSED + ECC
- */
-
- /* fill up the bbm positions */
- for (j = 0; j < host->bbm_size; j++)
- layout->eccpos[pos++] = i * ecc->bytes + j;
-
- /*
- * fill up the ecc and reserved positions, their indices are offseted
- * by the free oob region
- */
- shift = layout->oobfree[0].length + host->bbm_size;
-
- for (j = 0; j < (host->ecc_bytes_hw + host->spare_bytes); j++)
- layout->eccpos[pos++] = i * ecc->bytes + shift + j;
-
- return layout;
+ return 0;
}
+static int qcom_nand_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+ struct qcom_nand_host *host = to_qcom_nand_host(chip);
+ struct nand_ecc_ctrl *ecc = &chip->ecc;
+
+ if (section)
+ return -ERANGE;
+
+ oobregion->length = ecc->steps * 4;
+ oobregion->offset = ((ecc->steps - 1) * ecc->bytes) + host->bbm_size;
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops qcom_nand_ooblayout_ops = {
+ .ecc = qcom_nand_ooblayout_ecc,
+ .free = qcom_nand_ooblayout_free,
+};
+
static int qcom_nand_host_setup(struct qcom_nand_host *host)
{
struct nand_chip *chip = &host->chip;
@@ -1851,9 +1839,7 @@
ecc->mode = NAND_ECC_HW;
- ecc->layout = qcom_nand_create_layout(host);
- if (!ecc->layout)
- return -ENOMEM;
+ mtd_set_ooblayout(mtd, &qcom_nand_ooblayout_ops);
cwperpage = mtd->writesize / ecc->size;
diff --git a/drivers/mtd/nand/s3c2410.c b/drivers/mtd/nand/s3c2410.c
index 9c9397b..d9309cf 100644
--- a/drivers/mtd/nand/s3c2410.c
+++ b/drivers/mtd/nand/s3c2410.c
@@ -84,11 +84,33 @@
/* new oob placement block for use with hardware ecc generation
*/
+static int s3c2410_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section)
+ return -ERANGE;
-static struct nand_ecclayout nand_hw_eccoob = {
- .eccbytes = 3,
- .eccpos = {0, 1, 2},
- .oobfree = {{8, 8}}
+ oobregion->offset = 0;
+ oobregion->length = 3;
+
+ return 0;
+}
+
+static int s3c2410_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section)
+ return -ERANGE;
+
+ oobregion->offset = 8;
+ oobregion->length = 8;
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops s3c2410_ooblayout_ops = {
+ .ecc = s3c2410_ooblayout_ecc,
+ .free = s3c2410_ooblayout_free,
};
/* controller and mtd information */
@@ -542,7 +564,8 @@
diff0 |= (diff1 << 8);
diff0 |= (diff2 << 16);
- if ((diff0 & ~(1<<fls(diff0))) == 0)
+ /* equal to "(diff0 & ~(1 << __ffs(diff0)))" */
+ if ((diff0 & (diff0 - 1)) == 0)
return 1;
return -1;
@@ -859,6 +882,7 @@
}
#else
chip->ecc.mode = NAND_ECC_SOFT;
+ chip->ecc.algo = NAND_ECC_HAMMING;
#endif
if (set->disable_ecc)
@@ -919,7 +943,7 @@
} else {
chip->ecc.size = 512;
chip->ecc.bytes = 3;
- chip->ecc.layout = &nand_hw_eccoob;
+ mtd_set_ooblayout(nand_to_mtd(chip), &s3c2410_ooblayout_ops);
}
}
diff --git a/drivers/mtd/nand/sh_flctl.c b/drivers/mtd/nand/sh_flctl.c
index 4814402..6fa3bcd 100644
--- a/drivers/mtd/nand/sh_flctl.c
+++ b/drivers/mtd/nand/sh_flctl.c
@@ -31,7 +31,6 @@
#include <linux/io.h>
#include <linux/of.h>
#include <linux/of_device.h>
-#include <linux/of_mtd.h>
#include <linux/platform_device.h>
#include <linux/pm_runtime.h>
#include <linux/sh_dma.h>
@@ -43,26 +42,73 @@
#include <linux/mtd/partitions.h>
#include <linux/mtd/sh_flctl.h>
-static struct nand_ecclayout flctl_4secc_oob_16 = {
- .eccbytes = 10,
- .eccpos = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9},
- .oobfree = {
- {.offset = 12,
- . length = 4} },
+static int flctl_4secc_ooblayout_sp_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+
+ if (section)
+ return -ERANGE;
+
+ oobregion->offset = 0;
+ oobregion->length = chip->ecc.bytes;
+
+ return 0;
+}
+
+static int flctl_4secc_ooblayout_sp_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section)
+ return -ERANGE;
+
+ oobregion->offset = 12;
+ oobregion->length = 4;
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops flctl_4secc_oob_smallpage_ops = {
+ .ecc = flctl_4secc_ooblayout_sp_ecc,
+ .free = flctl_4secc_ooblayout_sp_free,
};
-static struct nand_ecclayout flctl_4secc_oob_64 = {
- .eccbytes = 4 * 10,
- .eccpos = {
- 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,
- 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,
- 38, 39, 40, 41, 42, 43, 44, 45, 46, 47,
- 54, 55, 56, 57, 58, 59, 60, 61, 62, 63 },
- .oobfree = {
- {.offset = 2, .length = 4},
- {.offset = 16, .length = 6},
- {.offset = 32, .length = 6},
- {.offset = 48, .length = 6} },
+static int flctl_4secc_ooblayout_lp_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+
+ if (section >= chip->ecc.steps)
+ return -ERANGE;
+
+ oobregion->offset = (section * 16) + 6;
+ oobregion->length = chip->ecc.bytes;
+
+ return 0;
+}
+
+static int flctl_4secc_ooblayout_lp_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *chip = mtd_to_nand(mtd);
+
+ if (section >= chip->ecc.steps)
+ return -ERANGE;
+
+ oobregion->offset = section * 16;
+ oobregion->length = 6;
+
+ if (!section) {
+ oobregion->offset += 2;
+ oobregion->length -= 2;
+ }
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops flctl_4secc_oob_largepage_ops = {
+ .ecc = flctl_4secc_ooblayout_lp_ecc,
+ .free = flctl_4secc_ooblayout_lp_free,
};
static uint8_t scan_ff_pattern[] = { 0xff, 0xff };
@@ -987,10 +1033,10 @@
if (flctl->hwecc) {
if (mtd->writesize == 512) {
- chip->ecc.layout = &flctl_4secc_oob_16;
+ mtd_set_ooblayout(mtd, &flctl_4secc_oob_smallpage_ops);
chip->badblock_pattern = &flctl_4secc_smallpage;
} else {
- chip->ecc.layout = &flctl_4secc_oob_64;
+ mtd_set_ooblayout(mtd, &flctl_4secc_oob_largepage_ops);
chip->badblock_pattern = &flctl_4secc_largepage;
}
@@ -1005,6 +1051,7 @@
flctl->flcmncr_base |= _4ECCEN;
} else {
chip->ecc.mode = NAND_ECC_SOFT;
+ chip->ecc.algo = NAND_ECC_HAMMING;
}
return 0;
@@ -1044,8 +1091,6 @@
const struct of_device_id *match;
struct flctl_soc_config *config;
struct sh_flctl_platform_data *pdata;
- struct device_node *dn = dev->of_node;
- int ret;
match = of_match_device(of_flctl_match, dev);
if (match)
@@ -1065,15 +1110,6 @@
pdata->has_hwecc = config->has_hwecc;
pdata->use_holden = config->use_holden;
- /* parse user defined options */
- ret = of_get_nand_bus_width(dn);
- if (ret == 16)
- pdata->flcmncr_val |= SEL_16BIT;
- else if (ret != 8) {
- dev_err(dev, "%s: invalid bus width\n", __func__);
- return NULL;
- }
-
return pdata;
}
@@ -1136,15 +1172,14 @@
nand->chip_delay = 20;
nand->read_byte = flctl_read_byte;
+ nand->read_word = flctl_read_word;
nand->write_buf = flctl_write_buf;
nand->read_buf = flctl_read_buf;
nand->select_chip = flctl_select_chip;
nand->cmdfunc = flctl_cmdfunc;
- if (pdata->flcmncr_val & SEL_16BIT) {
+ if (pdata->flcmncr_val & SEL_16BIT)
nand->options |= NAND_BUSWIDTH_16;
- nand->read_word = flctl_read_word;
- }
pm_runtime_enable(&pdev->dev);
pm_runtime_resume(&pdev->dev);
@@ -1155,6 +1190,16 @@
if (ret)
goto err_chip;
+ if (nand->options & NAND_BUSWIDTH_16) {
+ /*
+ * NAND_BUSWIDTH_16 may have been set by nand_scan_ident().
+ * Add the SEL_16BIT flag in pdata->flcmncr_val and re-assign
+ * flctl->flcmncr_base to pdata->flcmncr_val.
+ */
+ pdata->flcmncr_val |= SEL_16BIT;
+ flctl->flcmncr_base = pdata->flcmncr_val;
+ }
+
ret = flctl_chip_init_tail(flctl_mtd);
if (ret)
goto err_chip;
diff --git a/drivers/mtd/nand/sharpsl.c b/drivers/mtd/nand/sharpsl.c
index b7d1b55..064ca17 100644
--- a/drivers/mtd/nand/sharpsl.c
+++ b/drivers/mtd/nand/sharpsl.c
@@ -148,6 +148,7 @@
/* Link the private data with the MTD structure */
mtd = nand_to_mtd(this);
mtd->dev.parent = &pdev->dev;
+ mtd_set_ooblayout(mtd, data->ecc_layout);
platform_set_drvdata(pdev, sharpsl);
@@ -170,7 +171,6 @@
this->ecc.bytes = 3;
this->ecc.strength = 1;
this->badblock_pattern = data->badblock_pattern;
- this->ecc.layout = data->ecc_layout;
this->ecc.hwctl = sharpsl_nand_enable_hwecc;
this->ecc.calculate = sharpsl_nand_calculate_ecc;
this->ecc.correct = nand_correct_data;
diff --git a/drivers/mtd/nand/sm_common.c b/drivers/mtd/nand/sm_common.c
index c514740..5939dff 100644
--- a/drivers/mtd/nand/sm_common.c
+++ b/drivers/mtd/nand/sm_common.c
@@ -12,14 +12,47 @@
#include <linux/sizes.h>
#include "sm_common.h"
-static struct nand_ecclayout nand_oob_sm = {
- .eccbytes = 6,
- .eccpos = {8, 9, 10, 13, 14, 15},
- .oobfree = {
- {.offset = 0 , .length = 4}, /* reserved */
- {.offset = 6 , .length = 2}, /* LBA1 */
- {.offset = 11, .length = 2} /* LBA2 */
+static int oob_sm_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section > 1)
+ return -ERANGE;
+
+ oobregion->length = 3;
+ oobregion->offset = ((section + 1) * 8) - 3;
+
+ return 0;
+}
+
+static int oob_sm_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ switch (section) {
+ case 0:
+ /* reserved */
+ oobregion->offset = 0;
+ oobregion->length = 4;
+ break;
+ case 1:
+ /* LBA1 */
+ oobregion->offset = 6;
+ oobregion->length = 2;
+ break;
+ case 2:
+ /* LBA2 */
+ oobregion->offset = 11;
+ oobregion->length = 2;
+ break;
+ default:
+ return -ERANGE;
}
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops oob_sm_ops = {
+ .ecc = oob_sm_ooblayout_ecc,
+ .free = oob_sm_ooblayout_free,
};
/* NOTE: This layout is is not compatabable with SmartMedia, */
@@ -28,15 +61,43 @@
/* If you use smftl, it will bypass this and work correctly */
/* If you not, then you break SmartMedia compliance anyway */
-static struct nand_ecclayout nand_oob_sm_small = {
- .eccbytes = 3,
- .eccpos = {0, 1, 2},
- .oobfree = {
- {.offset = 3 , .length = 2}, /* reserved */
- {.offset = 6 , .length = 2}, /* LBA1 */
- }
-};
+static int oob_sm_small_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section)
+ return -ERANGE;
+ oobregion->length = 3;
+ oobregion->offset = 0;
+
+ return 0;
+}
+
+static int oob_sm_small_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ switch (section) {
+ case 0:
+ /* reserved */
+ oobregion->offset = 3;
+ oobregion->length = 2;
+ break;
+ case 1:
+ /* LBA1 */
+ oobregion->offset = 6;
+ oobregion->length = 2;
+ break;
+ default:
+ return -ERANGE;
+ }
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops oob_sm_small_ops = {
+ .ecc = oob_sm_small_ooblayout_ecc,
+ .free = oob_sm_small_ooblayout_free,
+};
static int sm_block_markbad(struct mtd_info *mtd, loff_t ofs)
{
@@ -121,9 +182,9 @@
/* ECC layout */
if (mtd->writesize == SM_SECTOR_SIZE)
- chip->ecc.layout = &nand_oob_sm;
+ mtd_set_ooblayout(mtd, &oob_sm_ops);
else if (mtd->writesize == SM_SMALL_PAGE)
- chip->ecc.layout = &nand_oob_sm_small;
+ mtd_set_ooblayout(mtd, &oob_sm_small_ops);
else
return -ENODEV;
diff --git a/drivers/mtd/nand/socrates_nand.c b/drivers/mtd/nand/socrates_nand.c
index e3305f9..888fd31 100644
--- a/drivers/mtd/nand/socrates_nand.c
+++ b/drivers/mtd/nand/socrates_nand.c
@@ -180,6 +180,7 @@
nand_chip->dev_ready = socrates_nand_device_ready;
nand_chip->ecc.mode = NAND_ECC_SOFT; /* enable ECC */
+ nand_chip->ecc.algo = NAND_ECC_HAMMING;
/* TODO: I have no idea what real delay is. */
nand_chip->chip_delay = 20; /* 20us command delay time */
diff --git a/drivers/mtd/nand/sunxi_nand.c b/drivers/mtd/nand/sunxi_nand.c
index 1c03eee..a83a690 100644
--- a/drivers/mtd/nand/sunxi_nand.c
+++ b/drivers/mtd/nand/sunxi_nand.c
@@ -30,7 +30,6 @@
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/of_gpio.h>
-#include <linux/of_mtd.h>
#include <linux/mtd/mtd.h>
#include <linux/mtd/nand.h>
#include <linux/mtd/partitions.h>
@@ -39,7 +38,7 @@
#include <linux/dmaengine.h>
#include <linux/gpio.h>
#include <linux/interrupt.h>
-#include <linux/io.h>
+#include <linux/iopoll.h>
#define NFC_REG_CTL 0x0000
#define NFC_REG_ST 0x0004
@@ -155,7 +154,7 @@
/* define bit use in NFC_ECC_ST */
#define NFC_ECC_ERR(x) BIT(x)
#define NFC_ECC_PAT_FOUND(x) BIT(x + 16)
-#define NFC_ECC_ERR_CNT(b, x) (((x) >> ((b) * 8)) & 0xff)
+#define NFC_ECC_ERR_CNT(b, x) (((x) >> (((b) % 4) * 8)) & 0xff)
#define NFC_DEFAULT_TIMEOUT_MS 1000
@@ -212,12 +211,9 @@
* sunxi HW ECC infos: stores information related to HW ECC support
*
* @mode: the sunxi ECC mode field deduced from ECC requirements
- * @layout: the OOB layout depending on the ECC requirements and the
- * selected ECC mode
*/
struct sunxi_nand_hw_ecc {
int mode;
- struct nand_ecclayout layout;
};
/*
@@ -239,6 +235,10 @@
u32 timing_cfg;
u32 timing_ctl;
int selected;
+ int addr_cycles;
+ u32 addr[2];
+ int cmd_cycles;
+ u8 cmd[2];
int nsels;
struct sunxi_nand_chip_sel sels[0];
};
@@ -298,54 +298,71 @@
return IRQ_HANDLED;
}
-static int sunxi_nfc_wait_int(struct sunxi_nfc *nfc, u32 flags,
- unsigned int timeout_ms)
+static int sunxi_nfc_wait_events(struct sunxi_nfc *nfc, u32 events,
+ bool use_polling, unsigned int timeout_ms)
{
- init_completion(&nfc->complete);
+ int ret;
- writel(flags, nfc->regs + NFC_REG_INT);
+ if (events & ~NFC_INT_MASK)
+ return -EINVAL;
if (!timeout_ms)
timeout_ms = NFC_DEFAULT_TIMEOUT_MS;
- if (!wait_for_completion_timeout(&nfc->complete,
- msecs_to_jiffies(timeout_ms))) {
- dev_err(nfc->dev, "wait interrupt timedout\n");
- return -ETIMEDOUT;
+ if (!use_polling) {
+ init_completion(&nfc->complete);
+
+ writel(events, nfc->regs + NFC_REG_INT);
+
+ ret = wait_for_completion_timeout(&nfc->complete,
+ msecs_to_jiffies(timeout_ms));
+
+ writel(0, nfc->regs + NFC_REG_INT);
+ } else {
+ u32 status;
+
+ ret = readl_poll_timeout(nfc->regs + NFC_REG_ST, status,
+ (status & events) == events, 1,
+ timeout_ms * 1000);
}
- return 0;
+ writel(events & NFC_INT_MASK, nfc->regs + NFC_REG_ST);
+
+ if (ret)
+ dev_err(nfc->dev, "wait interrupt timedout\n");
+
+ return ret;
}
static int sunxi_nfc_wait_cmd_fifo_empty(struct sunxi_nfc *nfc)
{
- unsigned long timeout = jiffies +
- msecs_to_jiffies(NFC_DEFAULT_TIMEOUT_MS);
+ u32 status;
+ int ret;
- do {
- if (!(readl(nfc->regs + NFC_REG_ST) & NFC_CMD_FIFO_STATUS))
- return 0;
- } while (time_before(jiffies, timeout));
+ ret = readl_poll_timeout(nfc->regs + NFC_REG_ST, status,
+ !(status & NFC_CMD_FIFO_STATUS), 1,
+ NFC_DEFAULT_TIMEOUT_MS * 1000);
+ if (ret)
+ dev_err(nfc->dev, "wait for empty cmd FIFO timedout\n");
- dev_err(nfc->dev, "wait for empty cmd FIFO timedout\n");
- return -ETIMEDOUT;
+ return ret;
}
static int sunxi_nfc_rst(struct sunxi_nfc *nfc)
{
- unsigned long timeout = jiffies +
- msecs_to_jiffies(NFC_DEFAULT_TIMEOUT_MS);
+ u32 ctl;
+ int ret;
writel(0, nfc->regs + NFC_REG_ECC_CTL);
writel(NFC_RESET, nfc->regs + NFC_REG_CTL);
- do {
- if (!(readl(nfc->regs + NFC_REG_CTL) & NFC_RESET))
- return 0;
- } while (time_before(jiffies, timeout));
+ ret = readl_poll_timeout(nfc->regs + NFC_REG_CTL, ctl,
+ !(ctl & NFC_RESET), 1,
+ NFC_DEFAULT_TIMEOUT_MS * 1000);
+ if (ret)
+ dev_err(nfc->dev, "wait for NAND controller reset timedout\n");
- dev_err(nfc->dev, "wait for NAND controller reset timedout\n");
- return -ETIMEDOUT;
+ return ret;
}
static int sunxi_nfc_dev_ready(struct mtd_info *mtd)
@@ -354,7 +371,6 @@
struct sunxi_nand_chip *sunxi_nand = to_sunxi_nand(nand);
struct sunxi_nfc *nfc = to_sunxi_nfc(sunxi_nand->nand.controller);
struct sunxi_nand_rb *rb;
- unsigned long timeo = (sunxi_nand->nand.state == FL_ERASING ? 400 : 20);
int ret;
if (sunxi_nand->selected < 0)
@@ -366,12 +382,6 @@
case RB_NATIVE:
ret = !!(readl(nfc->regs + NFC_REG_ST) &
NFC_RB_STATE(rb->info.nativeid));
- if (ret)
- break;
-
- sunxi_nfc_wait_int(nfc, NFC_RB_B2R, timeo);
- ret = !!(readl(nfc->regs + NFC_REG_ST) &
- NFC_RB_STATE(rb->info.nativeid));
break;
case RB_GPIO:
ret = gpio_get_value(rb->info.gpio);
@@ -407,7 +417,7 @@
sel = &sunxi_nand->sels[chip];
ctl |= NFC_CE_SEL(sel->cs) | NFC_EN |
- NFC_PAGE_SHIFT(nand->page_shift - 10);
+ NFC_PAGE_SHIFT(nand->page_shift);
if (sel->rb.type == RB_NONE) {
nand->dev_ready = NULL;
} else {
@@ -452,7 +462,7 @@
tmp = NFC_DATA_TRANS | NFC_DATA_SWAP_METHOD;
writel(tmp, nfc->regs + NFC_REG_CMD);
- ret = sunxi_nfc_wait_int(nfc, NFC_CMD_INT_FLAG, 0);
+ ret = sunxi_nfc_wait_events(nfc, NFC_CMD_INT_FLAG, true, 0);
if (ret)
break;
@@ -487,7 +497,7 @@
NFC_ACCESS_DIR;
writel(tmp, nfc->regs + NFC_REG_CMD);
- ret = sunxi_nfc_wait_int(nfc, NFC_CMD_INT_FLAG, 0);
+ ret = sunxi_nfc_wait_events(nfc, NFC_CMD_INT_FLAG, true, 0);
if (ret)
break;
@@ -511,32 +521,54 @@
struct sunxi_nand_chip *sunxi_nand = to_sunxi_nand(nand);
struct sunxi_nfc *nfc = to_sunxi_nfc(sunxi_nand->nand.controller);
int ret;
- u32 tmp;
ret = sunxi_nfc_wait_cmd_fifo_empty(nfc);
if (ret)
return;
- if (ctrl & NAND_CTRL_CHANGE) {
- tmp = readl(nfc->regs + NFC_REG_CTL);
- if (ctrl & NAND_NCE)
- tmp |= NFC_CE_CTL;
- else
- tmp &= ~NFC_CE_CTL;
- writel(tmp, nfc->regs + NFC_REG_CTL);
- }
+ if (dat == NAND_CMD_NONE && (ctrl & NAND_NCE) &&
+ !(ctrl & (NAND_CLE | NAND_ALE))) {
+ u32 cmd = 0;
- if (dat == NAND_CMD_NONE)
- return;
+ if (!sunxi_nand->addr_cycles && !sunxi_nand->cmd_cycles)
+ return;
+
+ if (sunxi_nand->cmd_cycles--)
+ cmd |= NFC_SEND_CMD1 | sunxi_nand->cmd[0];
+
+ if (sunxi_nand->cmd_cycles--) {
+ cmd |= NFC_SEND_CMD2;
+ writel(sunxi_nand->cmd[1],
+ nfc->regs + NFC_REG_RCMD_SET);
+ }
+
+ sunxi_nand->cmd_cycles = 0;
+
+ if (sunxi_nand->addr_cycles) {
+ cmd |= NFC_SEND_ADR |
+ NFC_ADR_NUM(sunxi_nand->addr_cycles);
+ writel(sunxi_nand->addr[0],
+ nfc->regs + NFC_REG_ADDR_LOW);
+ }
+
+ if (sunxi_nand->addr_cycles > 4)
+ writel(sunxi_nand->addr[1],
+ nfc->regs + NFC_REG_ADDR_HIGH);
+
+ writel(cmd, nfc->regs + NFC_REG_CMD);
+ sunxi_nand->addr[0] = 0;
+ sunxi_nand->addr[1] = 0;
+ sunxi_nand->addr_cycles = 0;
+ sunxi_nfc_wait_events(nfc, NFC_CMD_INT_FLAG, true, 0);
+ }
if (ctrl & NAND_CLE) {
- writel(NFC_SEND_CMD1 | dat, nfc->regs + NFC_REG_CMD);
- } else {
- writel(dat, nfc->regs + NFC_REG_ADDR_LOW);
- writel(NFC_SEND_ADR, nfc->regs + NFC_REG_CMD);
+ sunxi_nand->cmd[sunxi_nand->cmd_cycles++] = dat;
+ } else if (ctrl & NAND_ALE) {
+ sunxi_nand->addr[sunxi_nand->addr_cycles / 4] |=
+ dat << ((sunxi_nand->addr_cycles % 4) * 8);
+ sunxi_nand->addr_cycles++;
}
-
- sunxi_nfc_wait_int(nfc, NFC_CMD_INT_FLAG, 0);
}
/* These seed values have been extracted from Allwinner's BSP */
@@ -717,7 +749,8 @@
ecc_ctl = readl(nfc->regs + NFC_REG_ECC_CTL);
ecc_ctl &= ~(NFC_ECC_MODE_MSK | NFC_ECC_PIPELINE |
NFC_ECC_BLOCK_SIZE_MSK);
- ecc_ctl |= NFC_ECC_EN | NFC_ECC_MODE(data->mode) | NFC_ECC_EXCEPTION;
+ ecc_ctl |= NFC_ECC_EN | NFC_ECC_MODE(data->mode) | NFC_ECC_EXCEPTION |
+ NFC_ECC_PIPELINE;
writel(ecc_ctl, nfc->regs + NFC_REG_ECC_CTL);
}
@@ -739,18 +772,106 @@
buf[3] = user_data >> 24;
}
+static inline u32 sunxi_nfc_buf_to_user_data(const u8 *buf)
+{
+ return buf[0] | (buf[1] << 8) | (buf[2] << 16) | (buf[3] << 24);
+}
+
+static void sunxi_nfc_hw_ecc_get_prot_oob_bytes(struct mtd_info *mtd, u8 *oob,
+ int step, bool bbm, int page)
+{
+ struct nand_chip *nand = mtd_to_nand(mtd);
+ struct sunxi_nfc *nfc = to_sunxi_nfc(nand->controller);
+
+ sunxi_nfc_user_data_to_buf(readl(nfc->regs + NFC_REG_USER_DATA(step)),
+ oob);
+
+ /* De-randomize the Bad Block Marker. */
+ if (bbm && (nand->options & NAND_NEED_SCRAMBLING))
+ sunxi_nfc_randomize_bbm(mtd, page, oob);
+}
+
+static void sunxi_nfc_hw_ecc_set_prot_oob_bytes(struct mtd_info *mtd,
+ const u8 *oob, int step,
+ bool bbm, int page)
+{
+ struct nand_chip *nand = mtd_to_nand(mtd);
+ struct sunxi_nfc *nfc = to_sunxi_nfc(nand->controller);
+ u8 user_data[4];
+
+ /* Randomize the Bad Block Marker. */
+ if (bbm && (nand->options & NAND_NEED_SCRAMBLING)) {
+ memcpy(user_data, oob, sizeof(user_data));
+ sunxi_nfc_randomize_bbm(mtd, page, user_data);
+ oob = user_data;
+ }
+
+ writel(sunxi_nfc_buf_to_user_data(oob),
+ nfc->regs + NFC_REG_USER_DATA(step));
+}
+
+static void sunxi_nfc_hw_ecc_update_stats(struct mtd_info *mtd,
+ unsigned int *max_bitflips, int ret)
+{
+ if (ret < 0) {
+ mtd->ecc_stats.failed++;
+ } else {
+ mtd->ecc_stats.corrected += ret;
+ *max_bitflips = max_t(unsigned int, *max_bitflips, ret);
+ }
+}
+
+static int sunxi_nfc_hw_ecc_correct(struct mtd_info *mtd, u8 *data, u8 *oob,
+ int step, bool *erased)
+{
+ struct nand_chip *nand = mtd_to_nand(mtd);
+ struct sunxi_nfc *nfc = to_sunxi_nfc(nand->controller);
+ struct nand_ecc_ctrl *ecc = &nand->ecc;
+ u32 status, tmp;
+
+ *erased = false;
+
+ status = readl(nfc->regs + NFC_REG_ECC_ST);
+
+ if (status & NFC_ECC_ERR(step))
+ return -EBADMSG;
+
+ if (status & NFC_ECC_PAT_FOUND(step)) {
+ u8 pattern;
+
+ if (unlikely(!(readl(nfc->regs + NFC_REG_PAT_ID) & 0x1))) {
+ pattern = 0x0;
+ } else {
+ pattern = 0xff;
+ *erased = true;
+ }
+
+ if (data)
+ memset(data, pattern, ecc->size);
+
+ if (oob)
+ memset(oob, pattern, ecc->bytes + 4);
+
+ return 0;
+ }
+
+ tmp = readl(nfc->regs + NFC_REG_ECC_ERR_CNT(step));
+
+ return NFC_ECC_ERR_CNT(step, tmp);
+}
+
static int sunxi_nfc_hw_ecc_read_chunk(struct mtd_info *mtd,
u8 *data, int data_off,
u8 *oob, int oob_off,
int *cur_off,
unsigned int *max_bitflips,
- bool bbm, int page)
+ bool bbm, bool oob_required, int page)
{
struct nand_chip *nand = mtd_to_nand(mtd);
struct sunxi_nfc *nfc = to_sunxi_nfc(nand->controller);
struct nand_ecc_ctrl *ecc = &nand->ecc;
int raw_mode = 0;
- u32 status;
+ bool erased;
int ret;
if (*cur_off != data_off)
@@ -769,34 +890,19 @@
writel(NFC_DATA_TRANS | NFC_DATA_SWAP_METHOD | NFC_ECC_OP,
nfc->regs + NFC_REG_CMD);
- ret = sunxi_nfc_wait_int(nfc, NFC_CMD_INT_FLAG, 0);
+ ret = sunxi_nfc_wait_events(nfc, NFC_CMD_INT_FLAG, true, 0);
sunxi_nfc_randomizer_disable(mtd);
if (ret)
return ret;
*cur_off = oob_off + ecc->bytes + 4;
- status = readl(nfc->regs + NFC_REG_ECC_ST);
- if (status & NFC_ECC_PAT_FOUND(0)) {
- u8 pattern = 0xff;
-
- if (unlikely(!(readl(nfc->regs + NFC_REG_PAT_ID) & 0x1)))
- pattern = 0x0;
-
- memset(data, pattern, ecc->size);
- memset(oob, pattern, ecc->bytes + 4);
-
+ ret = sunxi_nfc_hw_ecc_correct(mtd, data, oob_required ? oob : NULL, 0,
+ &erased);
+ if (erased)
return 1;
- }
- ret = NFC_ECC_ERR_CNT(0, readl(nfc->regs + NFC_REG_ECC_ERR_CNT(0)));
-
- memcpy_fromio(data, nfc->regs + NFC_RAM0_BASE, ecc->size);
-
- nand->cmdfunc(mtd, NAND_CMD_RNDOUT, oob_off, -1);
- sunxi_nfc_randomizer_read_buf(mtd, oob, ecc->bytes + 4, true, page);
-
- if (status & NFC_ECC_ERR(0)) {
+ if (ret < 0) {
/*
* Re-read the data with the randomizer disabled to identify
* bitflips in erased pages.
@@ -804,34 +910,33 @@
if (nand->options & NAND_NEED_SCRAMBLING) {
nand->cmdfunc(mtd, NAND_CMD_RNDOUT, data_off, -1);
nand->read_buf(mtd, data, ecc->size);
- nand->cmdfunc(mtd, NAND_CMD_RNDOUT, oob_off, -1);
- nand->read_buf(mtd, oob, ecc->bytes + 4);
+ } else {
+ memcpy_fromio(data, nfc->regs + NFC_RAM0_BASE,
+ ecc->size);
}
+ nand->cmdfunc(mtd, NAND_CMD_RNDOUT, oob_off, -1);
+ nand->read_buf(mtd, oob, ecc->bytes + 4);
+
ret = nand_check_erased_ecc_chunk(data, ecc->size,
oob, ecc->bytes + 4,
NULL, 0, ecc->strength);
if (ret >= 0)
raw_mode = 1;
} else {
- /*
- * The engine protects 4 bytes of OOB data per chunk.
- * Retrieve the corrected OOB bytes.
- */
- sunxi_nfc_user_data_to_buf(readl(nfc->regs + NFC_REG_USER_DATA(0)),
- oob);
+ memcpy_fromio(data, nfc->regs + NFC_RAM0_BASE, ecc->size);
- /* De-randomize the Bad Block Marker. */
- if (bbm && nand->options & NAND_NEED_SCRAMBLING)
- sunxi_nfc_randomize_bbm(mtd, page, oob);
+ if (oob_required) {
+ nand->cmdfunc(mtd, NAND_CMD_RNDOUT, oob_off, -1);
+ sunxi_nfc_randomizer_read_buf(mtd, oob, ecc->bytes + 4,
+ true, page);
+
+ sunxi_nfc_hw_ecc_get_prot_oob_bytes(mtd, oob, 0,
+ bbm, page);
+ }
}
- if (ret < 0) {
- mtd->ecc_stats.failed++;
- } else {
- mtd->ecc_stats.corrected += ret;
- *max_bitflips = max_t(unsigned int, *max_bitflips, ret);
- }
+ sunxi_nfc_hw_ecc_update_stats(mtd, max_bitflips, ret);
return raw_mode;
}
@@ -848,7 +953,7 @@
if (len <= 0)
return;
- if (*cur_off != offset)
+ if (!cur_off || *cur_off != offset)
nand->cmdfunc(mtd, NAND_CMD_RNDOUT,
offset + mtd->writesize, -1);
@@ -858,12 +963,8 @@
sunxi_nfc_randomizer_read_buf(mtd, oob + offset, len,
false, page);
- *cur_off = mtd->oobsize + mtd->writesize;
-}
-
-static inline u32 sunxi_nfc_buf_to_user_data(const u8 *buf)
-{
- return buf[0] | (buf[1] << 8) | (buf[2] << 16) | (buf[3] << 24);
+ if (cur_off)
+ *cur_off = mtd->oobsize + mtd->writesize;
}
static int sunxi_nfc_hw_ecc_write_chunk(struct mtd_info *mtd,
@@ -882,19 +983,6 @@
sunxi_nfc_randomizer_write_buf(mtd, data, ecc->size, false, page);
- /* Fill OOB data in */
- if ((nand->options & NAND_NEED_SCRAMBLING) && bbm) {
- u8 user_data[4];
-
- memcpy(user_data, oob, 4);
- sunxi_nfc_randomize_bbm(mtd, page, user_data);
- writel(sunxi_nfc_buf_to_user_data(user_data),
- nfc->regs + NFC_REG_USER_DATA(0));
- } else {
- writel(sunxi_nfc_buf_to_user_data(oob),
- nfc->regs + NFC_REG_USER_DATA(0));
- }
-
if (data_off + ecc->size != oob_off)
nand->cmdfunc(mtd, NAND_CMD_RNDIN, oob_off, -1);
@@ -903,11 +991,13 @@
return ret;
sunxi_nfc_randomizer_enable(mtd);
+ sunxi_nfc_hw_ecc_set_prot_oob_bytes(mtd, oob, 0, bbm, page);
+
writel(NFC_DATA_TRANS | NFC_DATA_SWAP_METHOD |
NFC_ACCESS_DIR | NFC_ECC_OP,
nfc->regs + NFC_REG_CMD);
- ret = sunxi_nfc_wait_int(nfc, NFC_CMD_INT_FLAG, 0);
+ ret = sunxi_nfc_wait_events(nfc, NFC_CMD_INT_FLAG, true, 0);
sunxi_nfc_randomizer_disable(mtd);
if (ret)
return ret;
@@ -929,13 +1019,14 @@
if (len <= 0)
return;
- if (*cur_off != offset)
+ if (!cur_off || *cur_off != offset)
nand->cmdfunc(mtd, NAND_CMD_RNDIN,
offset + mtd->writesize, -1);
sunxi_nfc_randomizer_write_buf(mtd, oob + offset, len, false, page);
- *cur_off = mtd->oobsize + mtd->writesize;
+ if (cur_off)
+ *cur_off = mtd->oobsize + mtd->writesize;
}
static int sunxi_nfc_hw_ecc_read_page(struct mtd_info *mtd,
@@ -958,7 +1049,7 @@
ret = sunxi_nfc_hw_ecc_read_chunk(mtd, data, data_off, oob,
oob_off + mtd->writesize,
&cur_off, &max_bitflips,
- !i, page);
+ !i, oob_required, page);
if (ret < 0)
return ret;
else if (ret)
@@ -974,6 +1065,39 @@
return max_bitflips;
}
+static int sunxi_nfc_hw_ecc_read_subpage(struct mtd_info *mtd,
+ struct nand_chip *chip,
+ u32 data_offs, u32 readlen,
+ u8 *bufpoi, int page)
+{
+ struct nand_ecc_ctrl *ecc = &chip->ecc;
+ int ret, i, cur_off = 0;
+ unsigned int max_bitflips = 0;
+
+ sunxi_nfc_hw_ecc_enable(mtd);
+
+ chip->cmdfunc(mtd, NAND_CMD_READ0, 0, page);
+ for (i = data_offs / ecc->size;
+ i < DIV_ROUND_UP(data_offs + readlen, ecc->size); i++) {
+ int data_off = i * ecc->size;
+ int oob_off = i * (ecc->bytes + 4);
+ u8 *data = bufpoi + data_off;
+ u8 *oob = chip->oob_poi + oob_off;
+
+ ret = sunxi_nfc_hw_ecc_read_chunk(mtd, data, data_off,
+ oob,
+ oob_off + mtd->writesize,
+ &cur_off, &max_bitflips, !i,
+ false, page);
+ if (ret < 0)
+ return ret;
+ }
+
+ sunxi_nfc_hw_ecc_disable(mtd);
+
+ return max_bitflips;
+}
+
static int sunxi_nfc_hw_ecc_write_page(struct mtd_info *mtd,
struct nand_chip *chip,
const uint8_t *buf, int oob_required,
@@ -1026,7 +1150,9 @@
ret = sunxi_nfc_hw_ecc_read_chunk(mtd, data, data_off, oob,
oob_off, &cur_off,
- &max_bitflips, !i, page);
+ &max_bitflips, !i,
+ oob_required,
+ page);
if (ret < 0)
return ret;
else if (ret)
@@ -1074,6 +1200,40 @@
return 0;
}
+static int sunxi_nfc_hw_common_ecc_read_oob(struct mtd_info *mtd,
+ struct nand_chip *chip,
+ int page)
+{
+ chip->cmdfunc(mtd, NAND_CMD_READ0, 0, page);
+
+ chip->pagebuf = -1;
+
+ return chip->ecc.read_page(mtd, chip, chip->buffers->databuf, 1, page);
+}
+
+static int sunxi_nfc_hw_common_ecc_write_oob(struct mtd_info *mtd,
+ struct nand_chip *chip,
+ int page)
+{
+ int ret, status;
+
+ chip->cmdfunc(mtd, NAND_CMD_SEQIN, 0, page);
+
+ chip->pagebuf = -1;
+
+ memset(chip->buffers->databuf, 0xff, mtd->writesize);
+ ret = chip->ecc.write_page(mtd, chip, chip->buffers->databuf, 1, page);
+ if (ret)
+ return ret;
+
+ /* Send command to program the OOB data */
+ chip->cmdfunc(mtd, NAND_CMD_PAGEPROG, -1, -1);
+
+ status = chip->waitfunc(mtd, chip);
+
+ return status & NAND_STATUS_FAIL ? -EIO : 0;
+}
+
static const s32 tWB_lut[] = {6, 12, 16, 20};
static const s32 tRHW_lut[] = {4, 8, 12, 20};
@@ -1101,6 +1261,7 @@
struct sunxi_nfc *nfc = to_sunxi_nfc(chip->nand.controller);
u32 min_clk_period = 0;
s32 tWB, tADL, tWHR, tRHW, tCAD;
+ long real_clk_rate;
/* T1 <=> tCLS */
if (timings->tCLS_min > min_clk_period)
@@ -1163,6 +1324,18 @@
min_clk_period = DIV_ROUND_UP(timings->tWC_min, 2);
/* T16 - T19 + tCAD */
+ if (timings->tWB_max > (min_clk_period * 20))
+ min_clk_period = DIV_ROUND_UP(timings->tWB_max, 20);
+
+ if (timings->tADL_min > (min_clk_period * 32))
+ min_clk_period = DIV_ROUND_UP(timings->tADL_min, 32);
+
+ if (timings->tWHR_min > (min_clk_period * 32))
+ min_clk_period = DIV_ROUND_UP(timings->tWHR_min, 32);
+
+ if (timings->tRHW_min > (min_clk_period * 20))
+ min_clk_period = DIV_ROUND_UP(timings->tRHW_min, 20);
+
tWB = sunxi_nand_lookup_timing(tWB_lut, timings->tWB_max,
min_clk_period);
if (tWB < 0) {
@@ -1198,23 +1371,26 @@
/* TODO: A83 has some more bits for CDQSS, CS, CLHZ, CCS, WC */
chip->timing_cfg = NFC_TIMING_CFG(tWB, tADL, tWHR, tRHW, tCAD);
+ /* Convert min_clk_period from picoseconds to nanoseconds */
+ min_clk_period = DIV_ROUND_UP(min_clk_period, 1000);
+
+ /*
+ * Unlike what is stated in Allwinner datasheet, the clk_rate should
+ * be set to (1 / min_clk_period), and not (2 / min_clk_period).
+ * This new formula was verified with a scope and validated by
+ * Allwinner engineers.
+ */
+ chip->clk_rate = NSEC_PER_SEC / min_clk_period;
+ real_clk_rate = clk_round_rate(nfc->mod_clk, chip->clk_rate);
+
/*
* ONFI specification 3.1, paragraph 4.15.2 dictates that EDO data
* output cycle timings shall be used if the host drives tRC less than
* 30 ns.
*/
- chip->timing_ctl = (timings->tRC_min < 30000) ? NFC_TIMING_CTL_EDO : 0;
-
- /* Convert min_clk_period from picoseconds to nanoseconds */
- min_clk_period = DIV_ROUND_UP(min_clk_period, 1000);
-
- /*
- * Convert min_clk_period into a clk frequency, then get the
- * appropriate rate for the NAND controller IP given this formula
- * (specified in the datasheet):
- * nand clk_rate = 2 * min_clk_rate
- */
- chip->clk_rate = (2 * NSEC_PER_SEC) / min_clk_period;
+ min_clk_period = NSEC_PER_SEC / real_clk_rate;
+ chip->timing_ctl = ((min_clk_period * 2) < 30) ?
+ NFC_TIMING_CTL_EDO : 0;
return 0;
}
@@ -1257,6 +1433,57 @@
return sunxi_nand_chip_set_timings(chip, timings);
}
+static int sunxi_nand_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *nand = mtd_to_nand(mtd);
+ struct nand_ecc_ctrl *ecc = &nand->ecc;
+
+ if (section >= ecc->steps)
+ return -ERANGE;
+
+ oobregion->offset = section * (ecc->bytes + 4) + 4;
+ oobregion->length = ecc->bytes;
+
+ return 0;
+}
+
+static int sunxi_nand_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ struct nand_chip *nand = mtd_to_nand(mtd);
+ struct nand_ecc_ctrl *ecc = &nand->ecc;
+
+ if (section > ecc->steps)
+ return -ERANGE;
+
+ /*
+ * The first 2 bytes are used for BB markers, hence we
+ * only have 2 bytes available in the first user data
+ * section.
+ */
+ if (!section && ecc->mode == NAND_ECC_HW) {
+ oobregion->offset = 2;
+ oobregion->length = 2;
+
+ return 0;
+ }
+
+ oobregion->offset = section * (ecc->bytes + 4);
+
+ if (section < ecc->steps)
+ oobregion->length = 4;
+ else
+ oobregion->offset = mtd->oobsize - oobregion->offset;
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops sunxi_nand_ooblayout_ops = {
+ .ecc = sunxi_nand_ooblayout_ecc,
+ .free = sunxi_nand_ooblayout_free,
+};
+
static int sunxi_nand_hw_common_ecc_ctrl_init(struct mtd_info *mtd,
struct nand_ecc_ctrl *ecc,
struct device_node *np)
@@ -1266,7 +1493,6 @@
struct sunxi_nand_chip *sunxi_nand = to_sunxi_nand(nand);
struct sunxi_nfc *nfc = to_sunxi_nfc(sunxi_nand->nand.controller);
struct sunxi_nand_hw_ecc *data;
- struct nand_ecclayout *layout;
int nsectors;
int ret;
int i;
@@ -1295,7 +1521,6 @@
/* HW ECC always work with even numbers of ECC bytes */
ecc->bytes = ALIGN(ecc->bytes, 2);
- layout = &data->layout;
nsectors = mtd->writesize / ecc->size;
if (mtd->oobsize < ((ecc->bytes + 4) * nsectors)) {
@@ -1303,9 +1528,9 @@
goto err;
}
- layout->eccbytes = (ecc->bytes * nsectors);
-
- ecc->layout = layout;
+ ecc->read_oob = sunxi_nfc_hw_common_ecc_read_oob;
+ ecc->write_oob = sunxi_nfc_hw_common_ecc_write_oob;
+ mtd_set_ooblayout(mtd, &sunxi_nand_ooblayout_ops);
ecc->priv = data;
return 0;
@@ -1325,9 +1550,6 @@
struct nand_ecc_ctrl *ecc,
struct device_node *np)
{
- struct nand_ecclayout *layout;
- int nsectors;
- int i, j;
int ret;
ret = sunxi_nand_hw_common_ecc_ctrl_init(mtd, ecc, np);
@@ -1336,40 +1558,9 @@
ecc->read_page = sunxi_nfc_hw_ecc_read_page;
ecc->write_page = sunxi_nfc_hw_ecc_write_page;
- layout = ecc->layout;
- nsectors = mtd->writesize / ecc->size;
-
- for (i = 0; i < nsectors; i++) {
- if (i) {
- layout->oobfree[i].offset =
- layout->oobfree[i - 1].offset +
- layout->oobfree[i - 1].length +
- ecc->bytes;
- layout->oobfree[i].length = 4;
- } else {
- /*
- * The first 2 bytes are used for BB markers, hence we
- * only have 2 bytes available in the first user data
- * section.
- */
- layout->oobfree[i].length = 2;
- layout->oobfree[i].offset = 2;
- }
-
- for (j = 0; j < ecc->bytes; j++)
- layout->eccpos[(ecc->bytes * i) + j] =
- layout->oobfree[i].offset +
- layout->oobfree[i].length + j;
- }
-
- if (mtd->oobsize > (ecc->bytes + 4) * nsectors) {
- layout->oobfree[nsectors].offset =
- layout->oobfree[nsectors - 1].offset +
- layout->oobfree[nsectors - 1].length +
- ecc->bytes;
- layout->oobfree[nsectors].length = mtd->oobsize -
- ((ecc->bytes + 4) * nsectors);
- }
+ ecc->read_oob_raw = nand_read_oob_std;
+ ecc->write_oob_raw = nand_write_oob_std;
+ ecc->read_subpage = sunxi_nfc_hw_ecc_read_subpage;
return 0;
}
@@ -1378,9 +1569,6 @@
struct nand_ecc_ctrl *ecc,
struct device_node *np)
{
- struct nand_ecclayout *layout;
- int nsectors;
- int i;
int ret;
ret = sunxi_nand_hw_common_ecc_ctrl_init(mtd, ecc, np);
@@ -1390,15 +1578,8 @@
ecc->prepad = 4;
ecc->read_page = sunxi_nfc_hw_syndrome_ecc_read_page;
ecc->write_page = sunxi_nfc_hw_syndrome_ecc_write_page;
-
- layout = ecc->layout;
- nsectors = mtd->writesize / ecc->size;
-
- for (i = 0; i < (ecc->bytes * nsectors); i++)
- layout->eccpos[i] = i;
-
- layout->oobfree[0].length = mtd->oobsize - i;
- layout->oobfree[0].offset = i;
+ ecc->read_oob_raw = nand_read_oob_syndrome;
+ ecc->write_oob_raw = nand_write_oob_syndrome;
return 0;
}
@@ -1411,7 +1592,6 @@
sunxi_nand_hw_common_ecc_ctrl_cleanup(ecc);
break;
case NAND_ECC_NONE:
- kfree(ecc->layout);
default:
break;
}
@@ -1432,8 +1612,6 @@
return -EINVAL;
switch (ecc->mode) {
- case NAND_ECC_SOFT_BCH:
- break;
case NAND_ECC_HW:
ret = sunxi_nand_hw_ecc_ctrl_init(mtd, ecc, np);
if (ret)
@@ -1445,10 +1623,6 @@
return ret;
break;
case NAND_ECC_NONE:
- ecc->layout = kzalloc(sizeof(*ecc->layout), GFP_KERNEL);
- if (!ecc->layout)
- return -ENOMEM;
- ecc->layout->oobfree[0].length = mtd->oobsize;
case NAND_ECC_SOFT:
break;
default:
@@ -1536,21 +1710,6 @@
}
}
- timings = onfi_async_timing_mode_to_sdr_timings(0);
- if (IS_ERR(timings)) {
- ret = PTR_ERR(timings);
- dev_err(dev,
- "could not retrieve timings for ONFI mode 0: %d\n",
- ret);
- return ret;
- }
-
- ret = sunxi_nand_chip_set_timings(chip, timings);
- if (ret) {
- dev_err(dev, "could not configure chip timings: %d\n", ret);
- return ret;
- }
-
nand = &chip->nand;
/* Default tR value specified in the ONFI spec (chapter 4.15.1) */
nand->chip_delay = 200;
@@ -1570,6 +1729,21 @@
mtd = nand_to_mtd(nand);
mtd->dev.parent = dev;
+ timings = onfi_async_timing_mode_to_sdr_timings(0);
+ if (IS_ERR(timings)) {
+ ret = PTR_ERR(timings);
+ dev_err(dev,
+ "could not retrieve timings for ONFI mode 0: %d\n",
+ ret);
+ return ret;
+ }
+
+ ret = sunxi_nand_chip_set_timings(chip, timings);
+ if (ret) {
+ dev_err(dev, "could not configure chip timings: %d\n", ret);
+ return ret;
+ }
+
ret = nand_scan_ident(mtd, nsels, NULL);
if (ret)
return ret;
@@ -1580,6 +1754,8 @@
if (nand->options & NAND_NEED_SCRAMBLING)
nand->options |= NAND_NO_SUBPAGE_WRITE;
+ nand->options |= NAND_SUBPAGE_READ;
+
ret = sunxi_nand_chip_init_timings(chip, np);
if (ret) {
dev_err(dev, "could not configure chip timings: %d\n", ret);
@@ -1728,6 +1904,8 @@
struct sunxi_nfc *nfc = platform_get_drvdata(pdev);
sunxi_nand_chips_cleanup(nfc);
+ clk_disable_unprepare(nfc->mod_clk);
+ clk_disable_unprepare(nfc->ahb_clk);
return 0;
}
diff --git a/drivers/mtd/nand/vf610_nfc.c b/drivers/mtd/nand/vf610_nfc.c
index 293feb1..3ad514c 100644
--- a/drivers/mtd/nand/vf610_nfc.c
+++ b/drivers/mtd/nand/vf610_nfc.c
@@ -33,7 +33,6 @@
#include <linux/mtd/mtd.h>
#include <linux/mtd/nand.h>
#include <linux/mtd/partitions.h>
-#include <linux/of_mtd.h>
#include <linux/of_device.h>
#include <linux/pinctrl/consumer.h>
#include <linux/platform_device.h>
@@ -175,34 +174,6 @@
return container_of(mtd_to_nand(mtd), struct vf610_nfc, chip);
}
-static struct nand_ecclayout vf610_nfc_ecc45 = {
- .eccbytes = 45,
- .eccpos = {19, 20, 21, 22, 23,
- 24, 25, 26, 27, 28, 29, 30, 31,
- 32, 33, 34, 35, 36, 37, 38, 39,
- 40, 41, 42, 43, 44, 45, 46, 47,
- 48, 49, 50, 51, 52, 53, 54, 55,
- 56, 57, 58, 59, 60, 61, 62, 63},
- .oobfree = {
- {.offset = 2,
- .length = 17} }
-};
-
-static struct nand_ecclayout vf610_nfc_ecc60 = {
- .eccbytes = 60,
- .eccpos = { 4, 5, 6, 7, 8, 9, 10, 11,
- 12, 13, 14, 15, 16, 17, 18, 19,
- 20, 21, 22, 23, 24, 25, 26, 27,
- 28, 29, 30, 31, 32, 33, 34, 35,
- 36, 37, 38, 39, 40, 41, 42, 43,
- 44, 45, 46, 47, 48, 49, 50, 51,
- 52, 53, 54, 55, 56, 57, 58, 59,
- 60, 61, 62, 63 },
- .oobfree = {
- {.offset = 2,
- .length = 2} }
-};
-
static inline u32 vf610_nfc_read(struct vf610_nfc *nfc, uint reg)
{
return readl(nfc->regs + reg);
@@ -781,14 +752,16 @@
if (mtd->oobsize > 64)
mtd->oobsize = 64;
+ /*
+ * mtd->ecclayout is not specified here because we're using the
+ * default large page ECC layout defined in NAND core.
+ */
if (chip->ecc.strength == 32) {
nfc->ecc_mode = ECC_60_BYTE;
chip->ecc.bytes = 60;
- chip->ecc.layout = &vf610_nfc_ecc60;
} else if (chip->ecc.strength == 24) {
nfc->ecc_mode = ECC_45_BYTE;
chip->ecc.bytes = 45;
- chip->ecc.layout = &vf610_nfc_ecc45;
} else {
dev_err(nfc->dev, "Unsupported ECC strength\n");
err = -ENXIO;
diff --git a/drivers/mtd/onenand/onenand_base.c b/drivers/mtd/onenand/onenand_base.c
index af28bb3..a4b029a 100644
--- a/drivers/mtd/onenand/onenand_base.c
+++ b/drivers/mtd/onenand/onenand_base.c
@@ -68,21 +68,33 @@
* flexonenand_oob_128 - oob info for Flex-Onenand with 4KB page
* For now, we expose only 64 out of 80 ecc bytes
*/
-static struct nand_ecclayout flexonenand_oob_128 = {
- .eccbytes = 64,
- .eccpos = {
- 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,
- 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,
- 38, 39, 40, 41, 42, 43, 44, 45, 46, 47,
- 54, 55, 56, 57, 58, 59, 60, 61, 62, 63,
- 70, 71, 72, 73, 74, 75, 76, 77, 78, 79,
- 86, 87, 88, 89, 90, 91, 92, 93, 94, 95,
- 102, 103, 104, 105
- },
- .oobfree = {
- {2, 4}, {18, 4}, {34, 4}, {50, 4},
- {66, 4}, {82, 4}, {98, 4}, {114, 4}
- }
+static int flexonenand_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section > 7)
+ return -ERANGE;
+
+ oobregion->offset = (section * 16) + 6;
+ oobregion->length = 10;
+
+ return 0;
+}
+
+static int flexonenand_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section > 7)
+ return -ERANGE;
+
+ oobregion->offset = (section * 16) + 2;
+ oobregion->length = 4;
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops flexonenand_ooblayout_ops = {
+ .ecc = flexonenand_ooblayout_ecc,
+ .free = flexonenand_ooblayout_free,
};
/*
@@ -91,56 +103,77 @@
* Based on specification:
* 4Gb M-die OneNAND Flash (KFM4G16Q4M, KFN8G16Q4M). Rev. 1.3, Apr. 2010
*
- * For eccpos we expose only 64 bytes out of 72 (see struct nand_ecclayout)
- *
- * oobfree uses the spare area fields marked as
- * "Managed by internal ECC logic for Logical Sector Number area"
*/
-static struct nand_ecclayout onenand_oob_128 = {
- .eccbytes = 64,
- .eccpos = {
- 7, 8, 9, 10, 11, 12, 13, 14, 15,
- 23, 24, 25, 26, 27, 28, 29, 30, 31,
- 39, 40, 41, 42, 43, 44, 45, 46, 47,
- 55, 56, 57, 58, 59, 60, 61, 62, 63,
- 71, 72, 73, 74, 75, 76, 77, 78, 79,
- 87, 88, 89, 90, 91, 92, 93, 94, 95,
- 103, 104, 105, 106, 107, 108, 109, 110, 111,
- 119
- },
- .oobfree = {
- {2, 3}, {18, 3}, {34, 3}, {50, 3},
- {66, 3}, {82, 3}, {98, 3}, {114, 3}
- }
+static int onenand_ooblayout_128_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section > 7)
+ return -ERANGE;
+
+ oobregion->offset = (section * 16) + 7;
+ oobregion->length = 9;
+
+ return 0;
+}
+
+static int onenand_ooblayout_128_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section >= 8)
+ return -ERANGE;
+
+ /*
+ * free bytes are using the spare area fields marked as
+ * "Managed by internal ECC logic for Logical Sector Number area"
+ */
+ oobregion->offset = (section * 16) + 2;
+ oobregion->length = 3;
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops onenand_oob_128_ooblayout_ops = {
+ .ecc = onenand_ooblayout_128_ecc,
+ .free = onenand_ooblayout_128_free,
};
/**
- * onenand_oob_64 - oob info for large (2KB) page
+ * onenand_oob_32_64 - oob info for large (2KB) page
*/
-static struct nand_ecclayout onenand_oob_64 = {
- .eccbytes = 20,
- .eccpos = {
- 8, 9, 10, 11, 12,
- 24, 25, 26, 27, 28,
- 40, 41, 42, 43, 44,
- 56, 57, 58, 59, 60,
- },
- .oobfree = {
- {2, 3}, {14, 2}, {18, 3}, {30, 2},
- {34, 3}, {46, 2}, {50, 3}, {62, 2}
- }
-};
+static int onenand_ooblayout_32_64_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section > 3)
+ return -ERANGE;
-/**
- * onenand_oob_32 - oob info for middle (1KB) page
- */
-static struct nand_ecclayout onenand_oob_32 = {
- .eccbytes = 10,
- .eccpos = {
- 8, 9, 10, 11, 12,
- 24, 25, 26, 27, 28,
- },
- .oobfree = { {2, 3}, {14, 2}, {18, 3}, {30, 2} }
+ oobregion->offset = (section * 16) + 8;
+ oobregion->length = 5;
+
+ return 0;
+}
+
+static int onenand_ooblayout_32_64_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ int sections = (mtd->oobsize / 32) * 2;
+
+ if (section >= sections)
+ return -ERANGE;
+
+ if (section & 1) {
+ oobregion->offset = ((section - 1) * 16) + 14;
+ oobregion->length = 2;
+ } else {
+ oobregion->offset = (section * 16) + 2;
+ oobregion->length = 3;
+ }
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops onenand_oob_32_64_ooblayout_ops = {
+ .ecc = onenand_ooblayout_32_64_ecc,
+ .free = onenand_ooblayout_32_64_free,
};
static const unsigned char ffchars[] = {
@@ -1024,34 +1057,15 @@
int thislen)
{
struct onenand_chip *this = mtd->priv;
- struct nand_oobfree *free;
- int readcol = column;
- int readend = column + thislen;
- int lastgap = 0;
- unsigned int i;
- uint8_t *oob_buf = this->oob_buf;
+ int ret;
- free = this->ecclayout->oobfree;
- for (i = 0; i < MTD_MAX_OOBFREE_ENTRIES && free->length; i++, free++) {
- if (readcol >= lastgap)
- readcol += free->offset - lastgap;
- if (readend >= lastgap)
- readend += free->offset - lastgap;
- lastgap = free->offset + free->length;
- }
- this->read_bufferram(mtd, ONENAND_SPARERAM, oob_buf, 0, mtd->oobsize);
- free = this->ecclayout->oobfree;
- for (i = 0; i < MTD_MAX_OOBFREE_ENTRIES && free->length; i++, free++) {
- int free_end = free->offset + free->length;
- if (free->offset < readend && free_end > readcol) {
- int st = max_t(int,free->offset,readcol);
- int ed = min_t(int,free_end,readend);
- int n = ed - st;
- memcpy(buf, oob_buf + st, n);
- buf += n;
- } else if (column == 0)
- break;
- }
+ this->read_bufferram(mtd, ONENAND_SPARERAM, this->oob_buf, 0,
+ mtd->oobsize);
+ ret = mtd_ooblayout_get_databytes(mtd, buf, this->oob_buf,
+ column, thislen);
+ if (ret)
+ return ret;
+
return 0;
}
@@ -1808,34 +1822,7 @@
static int onenand_fill_auto_oob(struct mtd_info *mtd, u_char *oob_buf,
const u_char *buf, int column, int thislen)
{
- struct onenand_chip *this = mtd->priv;
- struct nand_oobfree *free;
- int writecol = column;
- int writeend = column + thislen;
- int lastgap = 0;
- unsigned int i;
-
- free = this->ecclayout->oobfree;
- for (i = 0; i < MTD_MAX_OOBFREE_ENTRIES && free->length; i++, free++) {
- if (writecol >= lastgap)
- writecol += free->offset - lastgap;
- if (writeend >= lastgap)
- writeend += free->offset - lastgap;
- lastgap = free->offset + free->length;
- }
- free = this->ecclayout->oobfree;
- for (i = 0; i < MTD_MAX_OOBFREE_ENTRIES && free->length; i++, free++) {
- int free_end = free->offset + free->length;
- if (free->offset < writeend && free_end > writecol) {
- int st = max_t(int,free->offset,writecol);
- int ed = min_t(int,free_end,writeend);
- int n = ed - st;
- memcpy(oob_buf + st, buf, n);
- buf += n;
- } else if (column == 0)
- break;
- }
- return 0;
+ return mtd_ooblayout_set_databytes(mtd, buf, oob_buf, column, thislen);
}
/**
@@ -4003,22 +3990,22 @@
switch (mtd->oobsize) {
case 128:
if (FLEXONENAND(this)) {
- this->ecclayout = &flexonenand_oob_128;
+ mtd_set_ooblayout(mtd, &flexonenand_ooblayout_ops);
mtd->subpage_sft = 0;
} else {
- this->ecclayout = &onenand_oob_128;
+ mtd_set_ooblayout(mtd, &onenand_oob_128_ooblayout_ops);
mtd->subpage_sft = 2;
}
if (ONENAND_IS_NOP_1(this))
mtd->subpage_sft = 0;
break;
case 64:
- this->ecclayout = &onenand_oob_64;
+ mtd_set_ooblayout(mtd, &onenand_oob_32_64_ooblayout_ops);
mtd->subpage_sft = 2;
break;
case 32:
- this->ecclayout = &onenand_oob_32;
+ mtd_set_ooblayout(mtd, &onenand_oob_32_64_ooblayout_ops);
mtd->subpage_sft = 1;
break;
@@ -4027,7 +4014,7 @@
__func__, mtd->oobsize);
mtd->subpage_sft = 0;
/* To prevent kernel oops */
- this->ecclayout = &onenand_oob_32;
+ mtd_set_ooblayout(mtd, &onenand_oob_32_64_ooblayout_ops);
break;
}
@@ -4037,12 +4024,12 @@
* The number of bytes available for a client to place data into
* the out of band area
*/
- mtd->oobavail = 0;
- for (i = 0; i < MTD_MAX_OOBFREE_ENTRIES &&
- this->ecclayout->oobfree[i].length; i++)
- mtd->oobavail += this->ecclayout->oobfree[i].length;
+ ret = mtd_ooblayout_count_freebytes(mtd);
+ if (ret < 0)
+ ret = 0;
- mtd->ecclayout = this->ecclayout;
+ mtd->oobavail = ret;
+
mtd->ecc_strength = 1;
/* Fill in remaining MTD driver data */
diff --git a/drivers/mtd/spi-nor/spi-nor.c b/drivers/mtd/spi-nor/spi-nor.c
index 157841d..c52e455 100644
--- a/drivers/mtd/spi-nor/spi-nor.c
+++ b/drivers/mtd/spi-nor/spi-nor.c
@@ -832,6 +832,7 @@
/* GigaDevice */
{ "gd25q32", INFO(0xc84016, 0, 64 * 1024, 64, SECT_4K) },
{ "gd25q64", INFO(0xc84017, 0, 64 * 1024, 128, SECT_4K) },
+ { "gd25lq64c", INFO(0xc86017, 0, 64 * 1024, 128, SECT_4K | SPI_NOR_DUAL_READ | SPI_NOR_QUAD_READ) },
{ "gd25q128", INFO(0xc84018, 0, 64 * 1024, 256, SECT_4K) },
/* Intel/Numonyx -- xxxs33b */
diff --git a/drivers/net/Space.c b/drivers/net/Space.c
index 67977f1..11fe712 100644
--- a/drivers/net/Space.c
+++ b/drivers/net/Space.c
@@ -35,8 +35,8 @@
#include <net/Space.h>
/* A unified ethernet device probe. This is the easiest way to have every
- ethernet adaptor have the name "eth[0123...]".
- */
+ * ethernet adaptor have the name "eth[0123...]".
+ */
struct devprobe2 {
struct net_device *(*probe)(int unit);
@@ -46,6 +46,7 @@
static int __init probe_list2(int unit, struct devprobe2 *p, int autoprobe)
{
struct net_device *dev;
+
for (; p->probe; p++) {
if (autoprobe && p->status)
continue;
@@ -58,8 +59,7 @@
return -ENODEV;
}
-/*
- * ISA probes that touch addresses < 0x400 (including those that also
+/* ISA probes that touch addresses < 0x400 (including those that also
* look for EISA/PCI cards in addition to ISA cards).
*/
static struct devprobe2 isa_probes[] __initdata = {
@@ -86,11 +86,11 @@
#endif
#ifdef CONFIG_CS89x0
#ifndef CONFIG_CS89x0_PLATFORM
- {cs89x0_probe, 0},
+ {cs89x0_probe, 0},
#endif
#endif
-#if defined(CONFIG_MVME16x_NET) || defined(CONFIG_BVME6000_NET) /* Intel I82596 */
- {i82596_probe, 0},
+#if defined(CONFIG_MVME16x_NET) || defined(CONFIG_BVME6000_NET) /* Intel */
+ {i82596_probe, 0}, /* I82596 */
#endif
#ifdef CONFIG_NI65
{ni65_probe, 0},
@@ -118,13 +118,12 @@
{mac8390_probe, 0},
#endif
#ifdef CONFIG_MAC89x0
- {mac89x0_probe, 0},
+ {mac89x0_probe, 0},
#endif
{NULL, 0},
};
-/*
- * Unified ethernet device probe, segmented per architecture and
+/* Unified ethernet device probe, segmented per architecture and
* per bus interface. This drives the legacy devices only for now.
*/
@@ -135,7 +134,7 @@
if (base_addr == 1)
return;
- (void)( probe_list2(unit, m68k_probes, base_addr == 0) &&
+ (void)(probe_list2(unit, m68k_probes, base_addr == 0) &&
probe_list2(unit, isa_probes, base_addr == 0));
}
diff --git a/drivers/net/ethernet/aeroflex/greth.c b/drivers/net/ethernet/aeroflex/greth.c
index b873531..bca07c5 100644
--- a/drivers/net/ethernet/aeroflex/greth.c
+++ b/drivers/net/ethernet/aeroflex/greth.c
@@ -1323,7 +1323,7 @@
static int greth_mdio_init(struct greth_private *greth)
{
- int ret, phy;
+ int ret;
unsigned long timeout;
greth->mdio = mdiobus_alloc();
diff --git a/drivers/net/ethernet/amd/au1000_eth.c b/drivers/net/ethernet/amd/au1000_eth.c
index 9af309e..e0fb0f1 100644
--- a/drivers/net/ethernet/amd/au1000_eth.c
+++ b/drivers/net/ethernet/amd/au1000_eth.c
@@ -1269,7 +1269,7 @@
aup->phy_irq = pd->phy_irq;
}
- if (aup->phy_busid && aup->phy_busid > 0) {
+ if (aup->phy_busid > 0) {
dev_err(&pdev->dev, "MAC0-associated PHY attached 2nd MACs MII bus not supported yet\n");
err = -ENODEV;
goto err_mdiobus_alloc;
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c
index d465bd7..0a5b770 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c
@@ -13259,12 +13259,11 @@
NETIF_F_RXHASH | NETIF_F_HW_VLAN_CTAG_TX;
if (!chip_is_e1x) {
dev->hw_features |= NETIF_F_GSO_GRE | NETIF_F_GSO_UDP_TUNNEL |
- NETIF_F_GSO_IPIP | NETIF_F_GSO_SIT;
+ NETIF_F_GSO_IPXIP4;
dev->hw_enc_features =
NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM | NETIF_F_SG |
NETIF_F_TSO | NETIF_F_TSO_ECN | NETIF_F_TSO6 |
- NETIF_F_GSO_IPIP |
- NETIF_F_GSO_SIT |
+ NETIF_F_GSO_IPXIP4 |
NETIF_F_GSO_GRE | NETIF_F_GSO_UDP_TUNNEL;
}
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 5a0dca3..72a2eff 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -6311,7 +6311,7 @@
dev->hw_features = NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM | NETIF_F_SG |
NETIF_F_TSO | NETIF_F_TSO6 |
NETIF_F_GSO_UDP_TUNNEL | NETIF_F_GSO_GRE |
- NETIF_F_GSO_IPIP | NETIF_F_GSO_SIT |
+ NETIF_F_GSO_IPXIP4 |
NETIF_F_GSO_UDP_TUNNEL_CSUM | NETIF_F_GSO_GRE_CSUM |
NETIF_F_GSO_PARTIAL | NETIF_F_RXHASH |
NETIF_F_RXCSUM | NETIF_F_LRO | NETIF_F_GRO;
@@ -6321,8 +6321,7 @@
NETIF_F_TSO | NETIF_F_TSO6 |
NETIF_F_GSO_UDP_TUNNEL | NETIF_F_GSO_GRE |
NETIF_F_GSO_UDP_TUNNEL_CSUM | NETIF_F_GSO_GRE_CSUM |
- NETIF_F_GSO_IPIP | NETIF_F_GSO_SIT |
- NETIF_F_GSO_PARTIAL;
+ NETIF_F_GSO_IPXIP4 | NETIF_F_GSO_PARTIAL;
dev->gso_partial_features = NETIF_F_GSO_UDP_TUNNEL_CSUM |
NETIF_F_GSO_GRE_CSUM;
dev->vlan_features = dev->hw_features | NETIF_F_HIGHDMA;
diff --git a/drivers/net/ethernet/cavium/liquidio/octeon_device.c b/drivers/net/ethernet/cavium/liquidio/octeon_device.c
index f67641a..8e23e3f 100644
--- a/drivers/net/ethernet/cavium/liquidio/octeon_device.c
+++ b/drivers/net/ethernet/cavium/liquidio/octeon_device.c
@@ -602,12 +602,10 @@
snprintf(oct->fw_info.liquidio_firmware_version, 32, "LIQUIDIO: %s",
h->version);
- buffer = kmalloc(size, GFP_KERNEL);
+ buffer = kmemdup(data, size, GFP_KERNEL);
if (!buffer)
return -ENOMEM;
- memcpy(buffer, data, size);
-
p = buffer + sizeof(struct octeon_firmware_file_header);
/* load all images */
diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
index 06b819d..0ff8e60 100644
--- a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
+++ b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
@@ -23,7 +23,7 @@
if (!nic->rb_pageref || !nic->rb_page)
return;
- atomic_add(nic->rb_pageref, &nic->rb_page->_count);
+ page_ref_add(nic->rb_page, nic->rb_pageref);
nic->rb_pageref = 0;
}
diff --git a/drivers/net/ethernet/cavium/thunder/thunder_bgx.c b/drivers/net/ethernet/cavium/thunder/thunder_bgx.c
index d20539a..3ed2198 100644
--- a/drivers/net/ethernet/cavium/thunder/thunder_bgx.c
+++ b/drivers/net/ethernet/cavium/thunder/thunder_bgx.c
@@ -274,12 +274,14 @@
static void bgx_lmac_handler(struct net_device *netdev)
{
struct lmac *lmac = container_of(netdev, struct lmac, netdev);
- struct phy_device *phydev = lmac->phydev;
+ struct phy_device *phydev;
int link_changed = 0;
if (!lmac)
return;
+ phydev = lmac->phydev;
+
if (!phydev->link && lmac->last_link)
link_changed = -1;
diff --git a/drivers/net/ethernet/chelsio/cxgb4/t4_msg.h b/drivers/net/ethernet/chelsio/cxgb4/t4_msg.h
index 80417fc..4705e2d 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/t4_msg.h
+++ b/drivers/net/ethernet/chelsio/cxgb4/t4_msg.h
@@ -1392,6 +1392,10 @@
#define T5_ULP_MEMIO_ORDER_V(x) ((x) << T5_ULP_MEMIO_ORDER_S)
#define T5_ULP_MEMIO_ORDER_F T5_ULP_MEMIO_ORDER_V(1U)
+#define T5_ULP_MEMIO_FID_S 4
+#define T5_ULP_MEMIO_FID_M 0x7ff
+#define T5_ULP_MEMIO_FID_V(x) ((x) << T5_ULP_MEMIO_FID_S)
+
/* ulp_mem_io.lock_addr fields */
#define ULP_MEMIO_ADDR_S 0
#define ULP_MEMIO_ADDR_V(x) ((x) << ULP_MEMIO_ADDR_S)
diff --git a/drivers/net/ethernet/ibm/ehea/ehea_main.c b/drivers/net/ethernet/ibm/ehea/ehea_main.c
index 2a0dc12..54efa9a 100644
--- a/drivers/net/ethernet/ibm/ehea/ehea_main.c
+++ b/drivers/net/ethernet/ibm/ehea/ehea_main.c
@@ -1169,16 +1169,15 @@
ec = EHEA_BMASK_GET(NEQE_EVENT_CODE, eqe);
portnum = EHEA_BMASK_GET(NEQE_PORTNUM, eqe);
port = ehea_get_port(adapter, portnum);
+ if (!port) {
+ netdev_err(NULL, "unknown portnum %x\n", portnum);
+ return;
+ }
dev = port->netdev;
switch (ec) {
case EHEA_EC_PORTSTATE_CHG: /* port state change */
- if (!port) {
- netdev_err(dev, "unknown portnum %x\n", portnum);
- break;
- }
-
if (EHEA_BMASK_GET(NEQE_PORT_UP, eqe)) {
if (!netif_carrier_ok(dev)) {
ret = ehea_sense_port_attr(port);
diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
index 1cd0ebf..5ea2200 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_main.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
@@ -9083,8 +9083,8 @@
NETIF_F_TSO6 |
NETIF_F_GSO_GRE |
NETIF_F_GSO_GRE_CSUM |
- NETIF_F_GSO_IPIP |
- NETIF_F_GSO_SIT |
+ NETIF_F_GSO_IPXIP4 |
+ NETIF_F_GSO_IPXIP6 |
NETIF_F_GSO_UDP_TUNNEL |
NETIF_F_GSO_UDP_TUNNEL_CSUM |
NETIF_F_GSO_PARTIAL |
diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
index 99a524d..55f151f 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
@@ -2284,8 +2284,8 @@
if (skb_shinfo(skb)->gso_type & (SKB_GSO_GRE |
SKB_GSO_GRE_CSUM |
- SKB_GSO_IPIP |
- SKB_GSO_SIT |
+ SKB_GSO_IPXIP4 |
+ SKB_GSO_IPXIP6 |
SKB_GSO_UDP_TUNNEL |
SKB_GSO_UDP_TUNNEL_CSUM)) {
if (!(skb_shinfo(skb)->gso_type & SKB_GSO_PARTIAL) &&
diff --git a/drivers/net/ethernet/intel/i40evf/i40e_txrx.c b/drivers/net/ethernet/intel/i40evf/i40e_txrx.c
index fd7dae46..be99189 100644
--- a/drivers/net/ethernet/intel/i40evf/i40e_txrx.c
+++ b/drivers/net/ethernet/intel/i40evf/i40e_txrx.c
@@ -1559,8 +1559,8 @@
if (skb_shinfo(skb)->gso_type & (SKB_GSO_GRE |
SKB_GSO_GRE_CSUM |
- SKB_GSO_IPIP |
- SKB_GSO_SIT |
+ SKB_GSO_IPXIP4 |
+ SKB_GSO_IPXIP6 |
SKB_GSO_UDP_TUNNEL |
SKB_GSO_UDP_TUNNEL_CSUM)) {
if (!(skb_shinfo(skb)->gso_type & SKB_GSO_PARTIAL) &&
diff --git a/drivers/net/ethernet/intel/i40evf/i40evf_main.c b/drivers/net/ethernet/intel/i40evf/i40evf_main.c
index 642bb45..16c5529 100644
--- a/drivers/net/ethernet/intel/i40evf/i40evf_main.c
+++ b/drivers/net/ethernet/intel/i40evf/i40evf_main.c
@@ -2230,8 +2230,8 @@
NETIF_F_TSO6 |
NETIF_F_GSO_GRE |
NETIF_F_GSO_GRE_CSUM |
- NETIF_F_GSO_IPIP |
- NETIF_F_GSO_SIT |
+ NETIF_F_GSO_IPXIP4 |
+ NETIF_F_GSO_IPXIP6 |
NETIF_F_GSO_UDP_TUNNEL |
NETIF_F_GSO_UDP_TUNNEL_CSUM |
NETIF_F_GSO_PARTIAL |
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 2172769..ef3d642 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -2418,8 +2418,8 @@
#define IGB_GSO_PARTIAL_FEATURES (NETIF_F_GSO_GRE | \
NETIF_F_GSO_GRE_CSUM | \
- NETIF_F_GSO_IPIP | \
- NETIF_F_GSO_SIT | \
+ NETIF_F_GSO_IPXIP4 | \
+ NETIF_F_GSO_IPXIP6 | \
NETIF_F_GSO_UDP_TUNNEL | \
NETIF_F_GSO_UDP_TUNNEL_CSUM)
diff --git a/drivers/net/ethernet/intel/igbvf/netdev.c b/drivers/net/ethernet/intel/igbvf/netdev.c
index 322a2d7..b0778ba 100644
--- a/drivers/net/ethernet/intel/igbvf/netdev.c
+++ b/drivers/net/ethernet/intel/igbvf/netdev.c
@@ -2763,8 +2763,8 @@
#define IGBVF_GSO_PARTIAL_FEATURES (NETIF_F_GSO_GRE | \
NETIF_F_GSO_GRE_CSUM | \
- NETIF_F_GSO_IPIP | \
- NETIF_F_GSO_SIT | \
+ NETIF_F_GSO_IPXIP4 | \
+ NETIF_F_GSO_IPXIP6 | \
NETIF_F_GSO_UDP_TUNNEL | \
NETIF_F_GSO_UDP_TUNNEL_CSUM)
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index 9f3677c..088c47c 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -9482,8 +9482,8 @@
#define IXGBE_GSO_PARTIAL_FEATURES (NETIF_F_GSO_GRE | \
NETIF_F_GSO_GRE_CSUM | \
- NETIF_F_GSO_IPIP | \
- NETIF_F_GSO_SIT | \
+ NETIF_F_GSO_IPXIP4 | \
+ NETIF_F_GSO_IPXIP6 | \
NETIF_F_GSO_UDP_TUNNEL | \
NETIF_F_GSO_UDP_TUNNEL_CSUM)
diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
index 5e348b1..acc2401 100644
--- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
+++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
@@ -4062,8 +4062,8 @@
#define IXGBEVF_GSO_PARTIAL_FEATURES (NETIF_F_GSO_GRE | \
NETIF_F_GSO_GRE_CSUM | \
- NETIF_F_GSO_IPIP | \
- NETIF_F_GSO_SIT | \
+ NETIF_F_GSO_IPXIP4 | \
+ NETIF_F_GSO_IPXIP6 | \
NETIF_F_GSO_UDP_TUNNEL | \
NETIF_F_GSO_UDP_TUNNEL_CSUM)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/cq.c b/drivers/net/ethernet/mellanox/mlx5/core/cq.c
index b51e42d..873a631 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/cq.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/cq.c
@@ -39,6 +39,53 @@
#include <linux/mlx5/cq.h>
#include "mlx5_core.h"
+#define TASKLET_MAX_TIME 2
+#define TASKLET_MAX_TIME_JIFFIES msecs_to_jiffies(TASKLET_MAX_TIME)
+
+void mlx5_cq_tasklet_cb(unsigned long data)
+{
+ unsigned long flags;
+ unsigned long end = jiffies + TASKLET_MAX_TIME_JIFFIES;
+ struct mlx5_eq_tasklet *ctx = (struct mlx5_eq_tasklet *)data;
+ struct mlx5_core_cq *mcq;
+ struct mlx5_core_cq *temp;
+
+ spin_lock_irqsave(&ctx->lock, flags);
+ list_splice_tail_init(&ctx->list, &ctx->process_list);
+ spin_unlock_irqrestore(&ctx->lock, flags);
+
+ list_for_each_entry_safe(mcq, temp, &ctx->process_list,
+ tasklet_ctx.list) {
+ list_del_init(&mcq->tasklet_ctx.list);
+ mcq->tasklet_ctx.comp(mcq);
+ if (atomic_dec_and_test(&mcq->refcount))
+ complete(&mcq->free);
+ if (time_after(jiffies, end))
+ break;
+ }
+
+ if (!list_empty(&ctx->process_list))
+ tasklet_schedule(&ctx->task);
+}
+
+static void mlx5_add_cq_to_tasklet(struct mlx5_core_cq *cq)
+{
+ unsigned long flags;
+ struct mlx5_eq_tasklet *tasklet_ctx = cq->tasklet_ctx.priv;
+
+ spin_lock_irqsave(&tasklet_ctx->lock, flags);
+ /* When migrating CQs between EQs will be implemented, please note
+ * that you need to sync this point. It is possible that
+ * while migrating a CQ, completions on the old EQs could
+ * still arrive.
+ */
+ if (list_empty_careful(&cq->tasklet_ctx.list)) {
+ atomic_inc(&cq->refcount);
+ list_add_tail(&cq->tasklet_ctx.list, &tasklet_ctx->list);
+ }
+ spin_unlock_irqrestore(&tasklet_ctx->lock, flags);
+}
+
void mlx5_cq_completion(struct mlx5_core_dev *dev, u32 cqn)
{
struct mlx5_core_cq *cq;
@@ -96,6 +143,13 @@
struct mlx5_create_cq_mbox_out out;
struct mlx5_destroy_cq_mbox_in din;
struct mlx5_destroy_cq_mbox_out dout;
+ int eqn = MLX5_GET(cqc, MLX5_ADDR_OF(create_cq_in, in, cq_context),
+ c_eqn);
+ struct mlx5_eq *eq;
+
+ eq = mlx5_eqn2eq(dev, eqn);
+ if (IS_ERR(eq))
+ return PTR_ERR(eq);
in->hdr.opcode = cpu_to_be16(MLX5_CMD_OP_CREATE_CQ);
memset(&out, 0, sizeof(out));
@@ -111,6 +165,11 @@
cq->arm_sn = 0;
atomic_set(&cq->refcount, 1);
init_completion(&cq->free);
+ if (!cq->comp)
+ cq->comp = mlx5_add_cq_to_tasklet;
+ /* assuming CQ will be deleted before the EQ */
+ cq->tasklet_ctx.priv = &eq->tasklet_ctx;
+ INIT_LIST_HEAD(&cq->tasklet_ctx.list);
spin_lock_irq(&table->lock);
err = radix_tree_insert(&table->tree, cq->cqn, cq);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
index f345679..bd94770 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
@@ -433,8 +433,8 @@
for (i = 0; i < MLX5_MPWRQ_PAGES_PER_WQE; i++) {
if (unlikely(mlx5e_alloc_and_map_page(rq, wi, i)))
goto err_unmap;
- atomic_add(mlx5e_mpwqe_strides_per_page(rq),
- &wi->umr.dma_info[i].page->_count);
+ page_ref_add(wi->umr.dma_info[i].page,
+ mlx5e_mpwqe_strides_per_page(rq));
wi->skbs_frags[i] = 0;
}
@@ -452,8 +452,8 @@
while (--i >= 0) {
dma_unmap_page(rq->pdev, wi->umr.dma_info[i].addr, PAGE_SIZE,
PCI_DMA_FROMDEVICE);
- atomic_sub(mlx5e_mpwqe_strides_per_page(rq),
- &wi->umr.dma_info[i].page->_count);
+ page_ref_sub(wi->umr.dma_info[i].page,
+ mlx5e_mpwqe_strides_per_page(rq));
put_page(wi->umr.dma_info[i].page);
}
dma_unmap_single(rq->pdev, wi->umr.mtt_addr, mtt_sz, PCI_DMA_TODEVICE);
@@ -477,8 +477,8 @@
for (i = 0; i < MLX5_MPWRQ_PAGES_PER_WQE; i++) {
dma_unmap_page(rq->pdev, wi->umr.dma_info[i].addr, PAGE_SIZE,
PCI_DMA_FROMDEVICE);
- atomic_sub(mlx5e_mpwqe_strides_per_page(rq) - wi->skbs_frags[i],
- &wi->umr.dma_info[i].page->_count);
+ page_ref_sub(wi->umr.dma_info[i].page,
+ mlx5e_mpwqe_strides_per_page(rq) - wi->skbs_frags[i]);
put_page(wi->umr.dma_info[i].page);
}
dma_unmap_single(rq->pdev, wi->umr.mtt_addr, mtt_sz, PCI_DMA_TODEVICE);
@@ -527,8 +527,8 @@
*/
split_page(wi->dma_info.page, MLX5_MPWRQ_WQE_PAGE_ORDER);
for (i = 0; i < MLX5_MPWRQ_PAGES_PER_WQE; i++) {
- atomic_add(mlx5e_mpwqe_strides_per_page(rq),
- &wi->dma_info.page[i]._count);
+ page_ref_add(&wi->dma_info.page[i],
+ mlx5e_mpwqe_strides_per_page(rq));
wi->skbs_frags[i] = 0;
}
@@ -551,8 +551,8 @@
dma_unmap_page(rq->pdev, wi->dma_info.addr, rq->wqe_sz,
PCI_DMA_FROMDEVICE);
for (i = 0; i < MLX5_MPWRQ_PAGES_PER_WQE; i++) {
- atomic_sub(mlx5e_mpwqe_strides_per_page(rq) - wi->skbs_frags[i],
- &wi->dma_info.page[i]._count);
+ page_ref_sub(&wi->dma_info.page[i],
+ mlx5e_mpwqe_strides_per_page(rq) - wi->skbs_frags[i]);
put_page(&wi->dma_info.page[i]);
}
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eq.c b/drivers/net/ethernet/mellanox/mlx5/core/eq.c
index 18fccec..0e30602 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eq.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eq.c
@@ -202,7 +202,7 @@
struct mlx5_eqe *eqe;
int eqes_found = 0;
int set_ci = 0;
- u32 cqn;
+ u32 cqn = -1;
u32 rsn;
u8 port;
@@ -320,6 +320,9 @@
eq_update_ci(eq, 1);
+ if (cqn != -1)
+ tasklet_schedule(&eq->tasklet_ctx.task);
+
return eqes_found;
}
@@ -403,6 +406,12 @@
if (err)
goto err_irq;
+ INIT_LIST_HEAD(&eq->tasklet_ctx.list);
+ INIT_LIST_HEAD(&eq->tasklet_ctx.process_list);
+ spin_lock_init(&eq->tasklet_ctx.lock);
+ tasklet_init(&eq->tasklet_ctx.task, mlx5_cq_tasklet_cb,
+ (unsigned long)&eq->tasklet_ctx);
+
/* EQs are created in ARMED state
*/
eq_update_ci(eq, 1);
@@ -436,6 +445,7 @@
mlx5_core_warn(dev, "failed to destroy a previously created eq: eqn %d\n",
eq->eqn);
synchronize_irq(eq->irqn);
+ tasklet_disable(&eq->tasklet_ctx.task);
mlx5_buf_free(dev, &eq->buf);
return err;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index 6feef7f..a19b593 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
@@ -663,6 +663,23 @@
}
EXPORT_SYMBOL(mlx5_vector2eqn);
+struct mlx5_eq *mlx5_eqn2eq(struct mlx5_core_dev *dev, int eqn)
+{
+ struct mlx5_eq_table *table = &dev->priv.eq_table;
+ struct mlx5_eq *eq;
+
+ spin_lock(&table->lock);
+ list_for_each_entry(eq, &table->comp_eqs_list, list)
+ if (eq->eqn == eqn) {
+ spin_unlock(&table->lock);
+ return eq;
+ }
+
+ spin_unlock(&table->lock);
+
+ return ERR_PTR(-ENOENT);
+}
+
static void free_comp_eqs(struct mlx5_core_dev *dev)
{
struct mlx5_eq_table *table = &dev->priv.eq_table;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h
index 482604b..2f86ec6 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h
@@ -102,6 +102,8 @@
int mlx5_wait_for_vf_pages(struct mlx5_core_dev *dev);
cycle_t mlx5_read_internal_timer(struct mlx5_core_dev *dev);
u32 mlx5_get_msix_vec(struct mlx5_core_dev *dev, int vecidx);
+struct mlx5_eq *mlx5_eqn2eq(struct mlx5_core_dev *dev, int eqn);
+void mlx5_cq_tasklet_cb(unsigned long data);
void mlx5e_init(void);
void mlx5e_cleanup(void);
diff --git a/drivers/net/ethernet/qlogic/qede/qede_main.c b/drivers/net/ethernet/qlogic/qede/qede_main.c
index 8114541..337e839 100644
--- a/drivers/net/ethernet/qlogic/qede/qede_main.c
+++ b/drivers/net/ethernet/qlogic/qede/qede_main.c
@@ -920,7 +920,7 @@
* network stack to take the ownership of the page
* which can be recycled multiple times by the driver.
*/
- atomic_inc(&curr_cons->data->_count);
+ page_ref_inc(curr_cons->data);
qede_reuse_page(edev, rxq, curr_cons);
}
@@ -1036,7 +1036,7 @@
/* Incr page ref count to reuse on allocation failure
* so that it doesn't get freed while freeing SKB.
*/
- atomic_inc(¤t_bd->data->_count);
+ page_ref_inc(current_bd->data);
goto out;
}
@@ -1076,8 +1076,7 @@
* start until its over and we don't want to risk allocation failing
* here, so re-allocate when aggregation will be over.
*/
- dma_unmap_addr_set(sw_rx_data_prod, mapping,
- dma_unmap_addr(replace_buf, mapping));
+ sw_rx_data_prod->mapping = replace_buf->mapping;
sw_rx_data_prod->data = replace_buf->data;
rx_bd_prod->addr.hi = cpu_to_le32(upper_32_bits(mapping));
@@ -1487,7 +1486,7 @@
* freeing SKB.
*/
- atomic_inc(&sw_rx_data->data->_count);
+ page_ref_inc(sw_rx_data->data);
rxq->rx_alloc_errors++;
qede_recycle_rx_bd_ring(rxq, edev,
fp_cqe->bd_num);
@@ -2655,7 +2654,7 @@
if (replace_buf->data) {
dma_unmap_page(&edev->pdev->dev,
- dma_unmap_addr(replace_buf, mapping),
+ replace_buf->mapping,
PAGE_SIZE, DMA_FROM_DEVICE);
__free_page(replace_buf->data);
}
@@ -2755,7 +2754,7 @@
goto err;
}
- dma_unmap_addr_set(replace_buf, mapping, mapping);
+ replace_buf->mapping = mapping;
tpa_info->replace_buf.page_offset = 0;
tpa_info->replace_buf_mapping = mapping;
diff --git a/drivers/net/ethernet/toshiba/ps3_gelic_wireless.c b/drivers/net/ethernet/toshiba/ps3_gelic_wireless.c
index 743b182..446ea58 100644
--- a/drivers/net/ethernet/toshiba/ps3_gelic_wireless.c
+++ b/drivers/net/ethernet/toshiba/ps3_gelic_wireless.c
@@ -1616,13 +1616,13 @@
target->valid = 1;
target->eurus_index = i;
kfree(target->hwinfo);
- target->hwinfo = kzalloc(be16_to_cpu(scan_info->size),
+ target->hwinfo = kmemdup(scan_info,
+ be16_to_cpu(scan_info->size),
GFP_KERNEL);
if (!target->hwinfo)
continue;
/* copy hw scan info */
- memcpy(target->hwinfo, scan_info, be16_to_cpu(scan_info->size));
target->essid_len = strnlen(scan_info->essid,
sizeof(scan_info->essid));
target->rate_len = 0;
diff --git a/drivers/net/geneve.c b/drivers/net/geneve.c
index a6dc11c..cadefe4 100644
--- a/drivers/net/geneve.c
+++ b/drivers/net/geneve.c
@@ -335,15 +335,15 @@
/* Need Geneve and inner Ethernet header to be present */
if (unlikely(!pskb_may_pull(skb, GENEVE_BASE_HLEN)))
- goto error;
+ goto drop;
/* Return packets with reserved bits set */
geneveh = geneve_hdr(skb);
if (unlikely(geneveh->ver != GENEVE_VER))
- goto error;
+ goto drop;
if (unlikely(geneveh->proto_type != htons(ETH_P_TEB)))
- goto error;
+ goto drop;
gs = rcu_dereference_sk_user_data(sk);
if (!gs)
@@ -366,10 +366,6 @@
/* Consume bad packet */
kfree_skb(skb);
return 0;
-
-error:
- /* Let the UDP layer deal with the skb */
- return 1;
}
static struct socket *geneve_create_sock(struct net *net, bool ipv6,
diff --git a/drivers/net/ieee802154/adf7242.c b/drivers/net/ieee802154/adf7242.c
index b82e39d..9fa7ac9 100644
--- a/drivers/net/ieee802154/adf7242.c
+++ b/drivers/net/ieee802154/adf7242.c
@@ -915,7 +915,6 @@
(stat & 0xf) == RC_STATUS_PHY_RDY ? "RC_STATUS_PHY_RDY" : "",
(stat & 0xf) == RC_STATUS_RX ? "RC_STATUS_RX" : "",
(stat & 0xf) == RC_STATUS_TX ? "RC_STATUS_TX" : "");
- }
#endif
}
diff --git a/drivers/net/irda/nsc-ircc.c b/drivers/net/irda/nsc-ircc.c
index 9ef13d8..aaecc3b 100644
--- a/drivers/net/irda/nsc-ircc.c
+++ b/drivers/net/irda/nsc-ircc.c
@@ -1253,7 +1253,7 @@
*/
static __u8 nsc_ircc_change_speed(struct nsc_ircc_cb *self, __u32 speed)
{
- struct net_device *dev = self->netdev;
+ struct net_device *dev;
__u8 mcr = MCR_SIR;
int iobase;
__u8 bank;
@@ -1263,6 +1263,7 @@
IRDA_ASSERT(self != NULL, return 0;);
+ dev = self->netdev;
iobase = self->io.fir_base;
/* Update accounting for new speed */
diff --git a/drivers/net/macsec.c b/drivers/net/macsec.c
index 460740c..47ee2c8 100644
--- a/drivers/net/macsec.c
+++ b/drivers/net/macsec.c
@@ -1646,7 +1646,7 @@
if (tb_sa[MACSEC_SA_ATTR_ACTIVE])
rx_sa->active = !!nla_get_u8(tb_sa[MACSEC_SA_ATTR_ACTIVE]);
- nla_memcpy(rx_sa->key.id, tb_sa[MACSEC_SA_ATTR_KEY], MACSEC_KEYID_LEN);
+ nla_memcpy(rx_sa->key.id, tb_sa[MACSEC_SA_ATTR_KEYID], MACSEC_KEYID_LEN);
rx_sa->sc = rx_sc;
rcu_assign_pointer(rx_sc->sa[assoc_num], rx_sa);
@@ -1785,7 +1785,7 @@
return -ENOMEM;
}
- nla_memcpy(tx_sa->key.id, tb_sa[MACSEC_SA_ATTR_KEY], MACSEC_KEYID_LEN);
+ nla_memcpy(tx_sa->key.id, tb_sa[MACSEC_SA_ATTR_KEYID], MACSEC_KEYID_LEN);
spin_lock_bh(&tx_sa->lock);
tx_sa->next_pn = nla_get_u32(tb_sa[MACSEC_SA_ATTR_PN]);
diff --git a/drivers/net/phy/fixed_phy.c b/drivers/net/phy/fixed_phy.c
index 9050f21..2d2e433 100644
--- a/drivers/net/phy/fixed_phy.c
+++ b/drivers/net/phy/fixed_phy.c
@@ -255,7 +255,8 @@
memset(fp->regs, 0xFF, sizeof(fp->regs[0]) * MII_REGS_NUM);
- fmb->mii_bus->irq[phy_addr] = irq;
+ if (irq != PHY_POLL)
+ fmb->mii_bus->irq[phy_addr] = irq;
fp->addr = phy_addr;
fp->status = *status;
@@ -314,6 +315,9 @@
int phy_addr;
int ret;
+ if (!fmb->mii_bus || fmb->mii_bus->state != MDIOBUS_REGISTERED)
+ return ERR_PTR(-EPROBE_DEFER);
+
/* Get the next available PHY address, up to PHY_MAX_ADDR */
spin_lock(&phy_fixed_addr_lock);
if (phy_fixed_addr == PHY_MAX_ADDR) {
diff --git a/drivers/net/phy/phy_device.c b/drivers/net/phy/phy_device.c
index 307f72a..e977ba9 100644
--- a/drivers/net/phy/phy_device.c
+++ b/drivers/net/phy/phy_device.c
@@ -34,7 +34,6 @@
#include <linux/io.h>
#include <linux/uaccess.h>
#include <linux/of.h>
-#include <linux/gpio/consumer.h>
#include <asm/irq.h>
@@ -1571,16 +1570,9 @@
struct device_driver *drv = phydev->mdio.dev.driver;
struct phy_driver *phydrv = to_phy_driver(drv);
int err = 0;
- struct gpio_descs *reset_gpios;
phydev->drv = phydrv;
- /* take phy out of reset */
- reset_gpios = devm_gpiod_get_array_optional(dev, "reset",
- GPIOD_OUT_LOW);
- if (IS_ERR(reset_gpios))
- return PTR_ERR(reset_gpios);
-
/* Disable the interrupt if the PHY doesn't support it
* but the interrupt is still a valid one
*/
diff --git a/drivers/net/tun.c b/drivers/net/tun.c
index 425e983..e16487c 100644
--- a/drivers/net/tun.c
+++ b/drivers/net/tun.c
@@ -580,11 +580,13 @@
for (i = 0; i < n; i++) {
tfile = rtnl_dereference(tun->tfiles[i]);
BUG_ON(!tfile);
+ tfile->socket.sk->sk_shutdown = RCV_SHUTDOWN;
tfile->socket.sk->sk_data_ready(tfile->socket.sk);
RCU_INIT_POINTER(tfile->tun, NULL);
--tun->numqueues;
}
list_for_each_entry(tfile, &tun->disabled, next) {
+ tfile->socket.sk->sk_shutdown = RCV_SHUTDOWN;
tfile->socket.sk->sk_data_ready(tfile->socket.sk);
RCU_INIT_POINTER(tfile->tun, NULL);
}
@@ -641,6 +643,7 @@
goto out;
}
tfile->queue_index = tun->numqueues;
+ tfile->socket.sk->sk_shutdown &= ~RCV_SHUTDOWN;
rcu_assign_pointer(tfile->tun, tun);
rcu_assign_pointer(tun->tfiles[tun->numqueues], tfile);
tun->numqueues++;
@@ -1491,9 +1494,6 @@
if (!iov_iter_count(to))
return 0;
- if (tun->dev->reg_state != NETREG_REGISTERED)
- return -EIO;
-
/* Read frames from queue */
skb = __skb_recv_datagram(tfile->socket.sk, noblock ? MSG_DONTWAIT : 0,
&peeked, &off, &err);
diff --git a/drivers/net/usb/cdc_ncm.c b/drivers/net/usb/cdc_ncm.c
index 2fb31ed..53759c3 100644
--- a/drivers/net/usb/cdc_ncm.c
+++ b/drivers/net/usb/cdc_ncm.c
@@ -740,12 +740,14 @@
int cdc_ncm_change_mtu(struct net_device *net, int new_mtu)
{
struct usbnet *dev = netdev_priv(net);
- struct cdc_ncm_ctx *ctx = (struct cdc_ncm_ctx *)dev->data[0];
- int maxmtu = ctx->max_datagram_size - cdc_ncm_eth_hlen(dev);
+ int maxmtu = cdc_ncm_max_dgram_size(dev) - cdc_ncm_eth_hlen(dev);
if (new_mtu <= 0 || new_mtu > maxmtu)
return -EINVAL;
+
net->mtu = new_mtu;
+ cdc_ncm_set_dgram_size(dev, new_mtu + cdc_ncm_eth_hlen(dev));
+
return 0;
}
EXPORT_SYMBOL_GPL(cdc_ncm_change_mtu);
diff --git a/drivers/net/usb/ch9200.c b/drivers/net/usb/ch9200.c
index 5e151e6..8a40202 100644
--- a/drivers/net/usb/ch9200.c
+++ b/drivers/net/usb/ch9200.c
@@ -155,12 +155,11 @@
index, size);
if (data) {
- buf = kmalloc(size, GFP_KERNEL);
+ buf = kmemdup(data, size, GFP_KERNEL);
if (!buf) {
err = -ENOMEM;
goto err_out;
}
- memcpy(buf, data, size);
}
err = usb_control_msg(dev->udev,
diff --git a/drivers/net/usb/hso.c b/drivers/net/usb/hso.c
index 111d907..4b44586 100644
--- a/drivers/net/usb/hso.c
+++ b/drivers/net/usb/hso.c
@@ -2029,7 +2029,7 @@
tty = tty_port_tty_get(&serial->port);
- if (tty && test_bit(TTY_THROTTLED, &tty->flags)) {
+ if (tty && tty_throttled(tty)) {
tty_kref_put(tty);
return -1;
}
diff --git a/drivers/net/vxlan.c b/drivers/net/vxlan.c
index 25ab6bf..8ff30c3 100644
--- a/drivers/net/vxlan.c
+++ b/drivers/net/vxlan.c
@@ -1304,7 +1304,7 @@
/* Need UDP and VXLAN header to be present */
if (!pskb_may_pull(skb, VXLAN_HLEN))
- return 1;
+ goto drop;
unparsed = *vxlan_hdr(skb);
/* VNI flag always required to be set */
@@ -1313,7 +1313,7 @@
ntohl(vxlan_hdr(skb)->vx_flags),
ntohl(vxlan_hdr(skb)->vx_vni));
/* Return non vxlan pkt */
- return 1;
+ goto drop;
}
unparsed.vx_flags &= ~VXLAN_HF_VNI;
unparsed.vx_vni &= ~VXLAN_VNI_MASK;
diff --git a/drivers/net/wireless/ath/ath10k/core.c b/drivers/net/wireless/ath/ath10k/core.c
index e94cb87..49af624 100644
--- a/drivers/net/wireless/ath/ath10k/core.c
+++ b/drivers/net/wireless/ath/ath10k/core.c
@@ -202,6 +202,7 @@
.name = "qca4019 hw1.0",
.patch_load_addr = QCA4019_HW_1_0_PATCH_LOAD_ADDR,
.uart_pin = 7,
+ .has_shifted_cc_wraparound = true,
.otp_exe_param = 0x0010000,
.continuous_frag_desc = true,
.channel_counters_freq_hz = 125000,
@@ -686,6 +687,9 @@
if (!IS_ERR(ar->cal_file))
release_firmware(ar->cal_file);
+ if (!IS_ERR(ar->pre_cal_file))
+ release_firmware(ar->pre_cal_file);
+
ath10k_swap_code_seg_release(ar);
ar->normal_mode_fw.fw_file.otp_data = NULL;
@@ -696,6 +700,7 @@
ar->normal_mode_fw.fw_file.firmware_len = 0;
ar->cal_file = NULL;
+ ar->pre_cal_file = NULL;
}
static int ath10k_fetch_cal_file(struct ath10k *ar)
@@ -1392,6 +1397,7 @@
complete_all(&ar->install_key_done);
complete_all(&ar->vdev_setup_done);
complete_all(&ar->thermal.wmi_sync);
+ complete_all(&ar->bss_survey_done);
wake_up(&ar->htt.empty_tx_wq);
wake_up(&ar->wmi.tx_credits_wq);
wake_up(&ar->peer_mapping_wq);
@@ -1724,6 +1730,9 @@
if (ath10k_peer_stats_enabled(ar))
val = WMI_10_4_PEER_STATS;
+ if (test_bit(WMI_SERVICE_BSS_CHANNEL_INFO_64, ar->wmi.svc_map))
+ val |= WMI_10_4_BSS_CHANNEL_INFO_64;
+
status = ath10k_mac_ext_resource_config(ar, val);
if (status) {
ath10k_err(ar,
@@ -1758,6 +1767,10 @@
goto err_hif_stop;
}
+ ar->free_vdev_map = (1LL << ar->max_num_vdevs) - 1;
+
+ INIT_LIST_HEAD(&ar->arvifs);
+
/* we don't care about HTT in UTF mode */
if (mode == ATH10K_FIRMWARE_MODE_NORMAL) {
status = ath10k_htt_setup(&ar->htt);
@@ -1771,10 +1784,6 @@
if (status)
goto err_hif_stop;
- ar->free_vdev_map = (1LL << ar->max_num_vdevs) - 1;
-
- INIT_LIST_HEAD(&ar->arvifs);
-
return 0;
err_hif_stop:
@@ -2085,6 +2094,7 @@
init_completion(&ar->install_key_done);
init_completion(&ar->vdev_setup_done);
init_completion(&ar->thermal.wmi_sync);
+ init_completion(&ar->bss_survey_done);
INIT_DELAYED_WORK(&ar->scan.timeout, ath10k_scan_timeout_work);
diff --git a/drivers/net/wireless/ath/ath10k/core.h b/drivers/net/wireless/ath/ath10k/core.h
index 1379054..1852e0e 100644
--- a/drivers/net/wireless/ath/ath10k/core.h
+++ b/drivers/net/wireless/ath/ath10k/core.h
@@ -876,6 +876,7 @@
* avoid reporting garbage data.
*/
bool ch_info_can_report_survey;
+ struct completion bss_survey_done;
struct dfs_pattern_detector *dfs_detector;
@@ -883,8 +884,6 @@
#ifdef CONFIG_ATH10K_DEBUGFS
struct ath10k_debug debug;
-#endif
-
struct {
/* relay(fs) channel for spectral scan */
struct rchan *rfs_chan_spec_scan;
@@ -893,6 +892,7 @@
enum ath10k_spectral_mode mode;
struct ath10k_spec_scan config;
} spectral;
+#endif
struct {
/* protected by conf_mutex */
diff --git a/drivers/net/wireless/ath/ath10k/mac.c b/drivers/net/wireless/ath/ath10k/mac.c
index 0e24f9e..6dd1d26 100644
--- a/drivers/net/wireless/ath/ath10k/mac.c
+++ b/drivers/net/wireless/ath/ath10k/mac.c
@@ -4278,9 +4278,6 @@
if (ar->phy_capability & WHAL_WLAN_11G_CAPABILITY) {
band = &ar->mac.sbands[NL80211_BAND_2GHZ];
band->ht_cap = ht_cap;
-
- /* Enable the VHT support at 2.4 GHz */
- band->vht_cap = vht_cap;
}
if (ar->phy_capability & WHAL_WLAN_11A_CAPABILITY) {
band = &ar->mac.sbands[NL80211_BAND_5GHZ];
@@ -4346,7 +4343,7 @@
/*
* This makes sense only when restarting hw. It is harmless to call
- * uncoditionally. This is necessary to make sure no HTT/WMI tx
+ * unconditionally. This is necessary to make sure no HTT/WMI tx
* commands will be submitted while restarting.
*/
ath10k_drain_tx(ar);
@@ -6407,6 +6404,39 @@
mutex_unlock(&ar->conf_mutex);
}
+static void
+ath10k_mac_update_bss_chan_survey(struct ath10k *ar,
+ struct ieee80211_channel *channel)
+{
+ int ret;
+ enum wmi_bss_survey_req_type type = WMI_BSS_SURVEY_REQ_TYPE_READ_CLEAR;
+
+ lockdep_assert_held(&ar->conf_mutex);
+
+ if (!test_bit(WMI_SERVICE_BSS_CHANNEL_INFO_64, ar->wmi.svc_map) ||
+ (ar->rx_channel != channel))
+ return;
+
+ if (ar->scan.state != ATH10K_SCAN_IDLE) {
+ ath10k_dbg(ar, ATH10K_DBG_MAC, "ignoring bss chan info request while scanning..\n");
+ return;
+ }
+
+ reinit_completion(&ar->bss_survey_done);
+
+ ret = ath10k_wmi_pdev_bss_chan_info_request(ar, type);
+ if (ret) {
+ ath10k_warn(ar, "failed to send pdev bss chan info request\n");
+ return;
+ }
+
+ ret = wait_for_completion_timeout(&ar->bss_survey_done, 3 * HZ);
+ if (!ret) {
+ ath10k_warn(ar, "bss channel survey timed out\n");
+ return;
+ }
+}
+
static int ath10k_get_survey(struct ieee80211_hw *hw, int idx,
struct survey_info *survey)
{
@@ -6431,6 +6461,8 @@
goto exit;
}
+ ath10k_mac_update_bss_chan_survey(ar, survey->channel);
+
spin_lock_bh(&ar->data_lock);
memcpy(survey, ar_survey, sizeof(*survey));
spin_unlock_bh(&ar->data_lock);
diff --git a/drivers/net/wireless/ath/ath10k/wmi-ops.h b/drivers/net/wireless/ath/ath10k/wmi-ops.h
index 7fb00dc..64ebd30 100644
--- a/drivers/net/wireless/ath/ath10k/wmi-ops.h
+++ b/drivers/net/wireless/ath/ath10k/wmi-ops.h
@@ -191,6 +191,9 @@
u32 fw_feature_bitmap);
int (*get_vdev_subtype)(struct ath10k *ar,
enum wmi_vdev_subtype subtype);
+ struct sk_buff *(*gen_pdev_bss_chan_info_req)
+ (struct ath10k *ar,
+ enum wmi_bss_survey_req_type type);
};
int ath10k_wmi_cmd_send(struct ath10k *ar, struct sk_buff *skb, u32 cmd_id);
@@ -1361,4 +1364,22 @@
return ar->wmi.ops->get_vdev_subtype(ar, subtype);
}
+static inline int
+ath10k_wmi_pdev_bss_chan_info_request(struct ath10k *ar,
+ enum wmi_bss_survey_req_type type)
+{
+ struct ath10k_wmi *wmi = &ar->wmi;
+ struct sk_buff *skb;
+
+ if (!wmi->ops->gen_pdev_bss_chan_info_req)
+ return -EOPNOTSUPP;
+
+ skb = wmi->ops->gen_pdev_bss_chan_info_req(ar, type);
+ if (IS_ERR(skb))
+ return PTR_ERR(skb);
+
+ return ath10k_wmi_cmd_send(ar, skb,
+ wmi->cmd->pdev_bss_chan_info_request_cmdid);
+}
+
#endif
diff --git a/drivers/net/wireless/ath/ath10k/wmi.c b/drivers/net/wireless/ath/ath10k/wmi.c
index 621019f..2c30032 100644
--- a/drivers/net/wireless/ath/ath10k/wmi.c
+++ b/drivers/net/wireless/ath/ath10k/wmi.c
@@ -521,7 +521,8 @@
.vdev_filter_neighbor_rx_packets_cmdid = WMI_CMD_UNSUPPORTED,
.mu_cal_start_cmdid = WMI_CMD_UNSUPPORTED,
.set_cca_params_cmdid = WMI_CMD_UNSUPPORTED,
- .pdev_bss_chan_info_request_cmdid = WMI_CMD_UNSUPPORTED,
+ .pdev_bss_chan_info_request_cmdid =
+ WMI_10_2_PDEV_BSS_CHAN_INFO_REQUEST_CMDID,
};
/* 10.4 WMI cmd track */
@@ -1633,6 +1634,7 @@
ch->max_power = arg->max_power;
ch->reg_power = arg->max_reg_power;
ch->antenna_max = arg->max_antenna_gain;
+ ch->max_tx_power = arg->max_power;
/* mode & flags share storage */
ch->mode = arg->mode;
@@ -4792,6 +4794,58 @@
return 0;
}
+static int ath10k_wmi_event_pdev_bss_chan_info(struct ath10k *ar,
+ struct sk_buff *skb)
+{
+ struct wmi_pdev_bss_chan_info_event *ev;
+ struct survey_info *survey;
+ u64 busy, total, tx, rx, rx_bss;
+ u32 freq, noise_floor;
+ u32 cc_freq_hz = ar->hw_params.channel_counters_freq_hz;
+ int idx;
+
+ ev = (struct wmi_pdev_bss_chan_info_event *)skb->data;
+ if (WARN_ON(skb->len < sizeof(*ev)))
+ return -EPROTO;
+
+ freq = __le32_to_cpu(ev->freq);
+ noise_floor = __le32_to_cpu(ev->noise_floor);
+ busy = __le64_to_cpu(ev->cycle_busy);
+ total = __le64_to_cpu(ev->cycle_total);
+ tx = __le64_to_cpu(ev->cycle_tx);
+ rx = __le64_to_cpu(ev->cycle_rx);
+ rx_bss = __le64_to_cpu(ev->cycle_rx_bss);
+
+ ath10k_dbg(ar, ATH10K_DBG_WMI,
+ "wmi event pdev bss chan info:\n freq: %d noise: %d cycle: busy %llu total %llu tx %llu rx %llu rx_bss %llu\n",
+ freq, noise_floor, busy, total, tx, rx, rx_bss);
+
+ spin_lock_bh(&ar->data_lock);
+ idx = freq_to_idx(ar, freq);
+ if (idx >= ARRAY_SIZE(ar->survey)) {
+ ath10k_warn(ar, "bss chan info: invalid frequency %d (idx %d out of bounds)\n",
+ freq, idx);
+ goto exit;
+ }
+
+ survey = &ar->survey[idx];
+
+ survey->noise = noise_floor;
+ survey->time = div_u64(total, cc_freq_hz);
+ survey->time_busy = div_u64(busy, cc_freq_hz);
+ survey->time_rx = div_u64(rx_bss, cc_freq_hz);
+ survey->time_tx = div_u64(tx, cc_freq_hz);
+ survey->filled |= (SURVEY_INFO_NOISE_DBM |
+ SURVEY_INFO_TIME |
+ SURVEY_INFO_TIME_BUSY |
+ SURVEY_INFO_TIME_RX |
+ SURVEY_INFO_TIME_TX);
+exit:
+ spin_unlock_bh(&ar->data_lock);
+ complete(&ar->bss_survey_done);
+ return 0;
+}
+
static void ath10k_wmi_op_rx(struct ath10k *ar, struct sk_buff *skb)
{
struct wmi_cmd_hdr *cmd_hdr;
@@ -5135,6 +5189,9 @@
case WMI_10_2_PDEV_TEMPERATURE_EVENTID:
ath10k_wmi_event_temperature(ar, skb);
break;
+ case WMI_10_2_PDEV_BSS_CHAN_INFO_EVENTID:
+ ath10k_wmi_event_pdev_bss_chan_info(ar, skb);
+ break;
case WMI_10_2_RTT_KEEPALIVE_EVENTID:
case WMI_10_2_GPIO_INPUT_EVENTID:
case WMI_10_2_PEER_RATECODE_LIST_EVENTID:
@@ -5212,6 +5269,7 @@
ath10k_wmi_event_vdev_stopped(ar, skb);
break;
case WMI_10_4_WOW_WAKEUP_HOST_EVENTID:
+ case WMI_10_4_PEER_RATECODE_LIST_EVENTID:
ath10k_dbg(ar, ATH10K_DBG_WMI,
"received event id %d not implemented\n", id);
break;
@@ -5221,6 +5279,9 @@
case WMI_10_4_PDEV_TEMPERATURE_EVENTID:
ath10k_wmi_event_temperature(ar, skb);
break;
+ case WMI_10_4_PDEV_BSS_CHAN_INFO_EVENTID:
+ ath10k_wmi_event_pdev_bss_chan_info(ar, skb);
+ break;
default:
ath10k_warn(ar, "Unknown eventid: %d\n", id);
break;
@@ -5606,6 +5667,9 @@
if (ath10k_peer_stats_enabled(ar))
features |= WMI_10_2_PEER_STATS;
+ if (test_bit(WMI_SERVICE_BSS_CHANNEL_INFO_64, ar->wmi.svc_map))
+ features |= WMI_10_2_BSS_CHAN_INFO;
+
cmd->resource_config.feature_mask = __cpu_to_le32(features);
memcpy(&cmd->resource_config.common, &config, sizeof(config));
@@ -6636,6 +6700,26 @@
return skb;
}
+static struct sk_buff *
+ath10k_wmi_10_2_op_gen_pdev_bss_chan_info(struct ath10k *ar,
+ enum wmi_bss_survey_req_type type)
+{
+ struct wmi_pdev_chan_info_req_cmd *cmd;
+ struct sk_buff *skb;
+
+ skb = ath10k_wmi_alloc_skb(ar, sizeof(*cmd));
+ if (!skb)
+ return ERR_PTR(-ENOMEM);
+
+ cmd = (struct wmi_pdev_chan_info_req_cmd *)skb->data;
+ cmd->type = __cpu_to_le32(type);
+
+ ath10k_dbg(ar, ATH10K_DBG_WMI,
+ "wmi pdev bss info request type %d\n", type);
+
+ return skb;
+}
+
/* This function assumes the beacon is already DMA mapped */
static struct sk_buff *
ath10k_wmi_op_gen_beacon_dma(struct ath10k *ar, u32 vdev_id, const void *bcn,
@@ -7735,6 +7819,7 @@
.gen_init = ath10k_wmi_10_2_op_gen_init,
.gen_peer_assoc = ath10k_wmi_10_2_op_gen_peer_assoc,
.gen_pdev_get_temperature = ath10k_wmi_10_2_op_gen_pdev_get_temperature,
+ .gen_pdev_bss_chan_info_req = ath10k_wmi_10_2_op_gen_pdev_bss_chan_info,
/* shared with 10.1 */
.map_svc = wmi_10x_svc_map,
@@ -7861,6 +7946,7 @@
.gen_request_stats = ath10k_wmi_op_gen_request_stats,
.gen_pdev_get_temperature = ath10k_wmi_10_2_op_gen_pdev_get_temperature,
.get_vdev_subtype = ath10k_wmi_10_4_op_get_vdev_subtype,
+ .gen_pdev_bss_chan_info_req = ath10k_wmi_10_2_op_gen_pdev_bss_chan_info,
};
int ath10k_wmi_attach(struct ath10k *ar)
diff --git a/drivers/net/wireless/ath/ath10k/wmi.h b/drivers/net/wireless/ath/ath10k/wmi.h
index db25535..9fdf47e 100644
--- a/drivers/net/wireless/ath/ath10k/wmi.h
+++ b/drivers/net/wireless/ath/ath10k/wmi.h
@@ -1444,6 +1444,7 @@
WMI_10_2_MU_CAL_START_CMDID,
WMI_10_2_SET_LTEU_CONFIG_CMDID,
WMI_10_2_SET_CCA_PARAMS,
+ WMI_10_2_PDEV_BSS_CHAN_INFO_REQUEST_CMDID,
WMI_10_2_PDEV_UTF_CMDID = WMI_10_2_END_CMDID - 1,
};
@@ -1487,6 +1488,8 @@
WMI_10_2_WDS_PEER_EVENTID,
WMI_10_2_PEER_STA_PS_STATECHG_EVENTID,
WMI_10_2_PDEV_TEMPERATURE_EVENTID,
+ WMI_10_2_MU_REPORT_EVENTID,
+ WMI_10_2_PDEV_BSS_CHAN_INFO_EVENTID,
WMI_10_2_PDEV_UTF_EVENTID = WMI_10_2_END_EVENTID - 1,
};
@@ -1795,6 +1798,7 @@
__le32 reginfo1;
struct {
u8 antenna_max;
+ u8 max_tx_power;
} __packed;
} __packed;
} __packed;
@@ -2450,6 +2454,7 @@
WMI_10_2_RX_BATCH_MODE = BIT(0),
WMI_10_2_ATF_CONFIG = BIT(1),
WMI_10_2_COEX_GPIO = BIT(3),
+ WMI_10_2_BSS_CHAN_INFO = BIT(6),
WMI_10_2_PEER_STATS = BIT(7),
};
@@ -6280,6 +6285,17 @@
__le32 temperature;
} __packed;
+struct wmi_pdev_bss_chan_info_event {
+ __le32 freq;
+ __le32 noise_floor;
+ __le64 cycle_busy;
+ __le64 cycle_total;
+ __le64 cycle_tx;
+ __le64 cycle_rx;
+ __le64 cycle_rx_bss;
+ __le32 reserved;
+} __packed;
+
/* WOW structures */
enum wmi_wow_wakeup_event {
WOW_BMISS_EVENT = 0,
@@ -6483,6 +6499,16 @@
WMI_HOST_PLATFORM_LOW_PERF,
};
+enum wmi_bss_survey_req_type {
+ WMI_BSS_SURVEY_REQ_TYPE_READ = 1,
+ WMI_BSS_SURVEY_REQ_TYPE_READ_CLEAR,
+};
+
+struct wmi_pdev_chan_info_req_cmd {
+ __le32 type;
+ __le32 reserved;
+} __packed;
+
struct ath10k;
struct ath10k_vif;
struct ath10k_fw_stats_pdev;
diff --git a/drivers/net/wireless/ath/ath6kl/core.c b/drivers/net/wireless/ath/ath6kl/core.c
index 4ec02ce..ebb9f16 100644
--- a/drivers/net/wireless/ath/ath6kl/core.c
+++ b/drivers/net/wireless/ath/ath6kl/core.c
@@ -31,6 +31,7 @@
static unsigned int suspend_mode;
static unsigned int wow_mode;
static unsigned int uart_debug;
+static unsigned int uart_rate = 115200;
static unsigned int ath6kl_p2p;
static unsigned int testmode;
static unsigned int recovery_enable;
@@ -40,6 +41,7 @@
module_param(suspend_mode, uint, 0644);
module_param(wow_mode, uint, 0644);
module_param(uart_debug, uint, 0644);
+module_param(uart_rate, uint, 0644);
module_param(ath6kl_p2p, uint, 0644);
module_param(testmode, uint, 0644);
module_param(recovery_enable, uint, 0644);
@@ -180,6 +182,7 @@
if (uart_debug)
ar->conf_flags |= ATH6KL_CONF_UART_DEBUG;
+ ar->hw.uarttx_rate = uart_rate;
set_bit(FIRST_BOOT, &ar->flag);
diff --git a/drivers/net/wireless/ath/ath6kl/core.h b/drivers/net/wireless/ath/ath6kl/core.h
index 713a571..7a1970e 100644
--- a/drivers/net/wireless/ath/ath6kl/core.h
+++ b/drivers/net/wireless/ath/ath6kl/core.h
@@ -781,6 +781,7 @@
u32 board_addr;
u32 refclk_hz;
u32 uarttx_pin;
+ u32 uarttx_rate;
u32 testscript_addr;
u8 tx_ant;
u8 rx_ant;
diff --git a/drivers/net/wireless/ath/ath6kl/init.c b/drivers/net/wireless/ath/ath6kl/init.c
index da557dc..58fb227 100644
--- a/drivers/net/wireless/ath/ath6kl/init.c
+++ b/drivers/net/wireless/ath/ath6kl/init.c
@@ -173,6 +173,7 @@
.reserved_ram_size = 7168,
.board_addr = 0x436400,
.testscript_addr = 0,
+ .uarttx_pin = 11,
.flags = 0,
.fw = {
@@ -650,6 +651,14 @@
if (status)
return status;
+ /* Only set the baud rate if we're actually doing debug */
+ if (ar->conf_flags & ATH6KL_CONF_UART_DEBUG) {
+ status = ath6kl_bmi_write_hi32(ar, hi_desired_baud_rate,
+ ar->hw.uarttx_rate);
+ if (status)
+ return status;
+ }
+
/* Configure target refclk_hz */
if (ar->hw.refclk_hz != 0) {
status = ath6kl_bmi_write_hi32(ar, hi_refclk_hz,
diff --git a/drivers/net/wireless/ath/ath9k/Kconfig b/drivers/net/wireless/ath/ath9k/Kconfig
index 40fa915..f68cb00 100644
--- a/drivers/net/wireless/ath/ath9k/Kconfig
+++ b/drivers/net/wireless/ath/ath9k/Kconfig
@@ -75,6 +75,26 @@
---help---
This option enables detailed statistics for association stations.
+config ATH9K_TX99
+ bool "Atheros ath9k TX99 testing support"
+ depends on ATH9K_DEBUGFS && CFG80211_CERTIFICATION_ONUS
+ default n
+ ---help---
+ Say N. This should only be enabled on systems undergoing
+ certification testing and evaluation in a controlled environment.
+ Enabling this will only enable TX99 support, all other modes of
+ operation will be disabled.
+
+ TX99 support enables Specific Absorption Rate (SAR) testing.
+ SAR is the unit of measurement for the amount of radio frequency(RF)
+ absorbed by the body when using a wireless device. The RF exposure
+ limits used are expressed in the terms of SAR, which is a measure
+ of the electric and magnetic field strength and power density for
+ transmitters operating at frequencies from 300 kHz to 100 GHz.
+ Regulatory bodies around the world require that wireless device
+ be evaluated to meet the RF exposure limits set forth in the
+ governmental SAR regulations.
+
config ATH9K_DFS_CERTIFIED
bool "Atheros DFS support for certified platforms"
depends on ATH9K && CFG80211_CERTIFICATION_ONUS
@@ -103,26 +123,6 @@
based on ACK frame RX timestamp, TX frame timestamp and frame
duration
-config ATH9K_TX99
- bool "Atheros ath9k TX99 testing support"
- depends on ATH9K_DEBUGFS && CFG80211_CERTIFICATION_ONUS
- default n
- ---help---
- Say N. This should only be enabled on systems undergoing
- certification testing and evaluation in a controlled environment.
- Enabling this will only enable TX99 support, all other modes of
- operation will be disabled.
-
- TX99 support enables Specific Absorption Rate (SAR) testing.
- SAR is the unit of measurement for the amount of radio frequency(RF)
- absorbed by the body when using a wireless device. The RF exposure
- limits used are expressed in the terms of SAR, which is a measure
- of the electric and magnetic field strength and power density for
- transmitters operating at frequencies from 300 kHz to 100 GHz.
- Regulatory bodies around the world require that wireless device
- be evaluated to meet the RF exposure limits set forth in the
- governmental SAR regulations.
-
config ATH9K_WOW
bool "Wake on Wireless LAN support (EXPERIMENTAL)"
depends on ATH9K && PM
diff --git a/drivers/net/wireless/ath/ath9k/ar9003_eeprom.c b/drivers/net/wireless/ath/ath9k/ar9003_eeprom.c
index f680982..dec1a31 100644
--- a/drivers/net/wireless/ath/ath9k/ar9003_eeprom.c
+++ b/drivers/net/wireless/ath/ath9k/ar9003_eeprom.c
@@ -4402,7 +4402,7 @@
}
/* Set tx power registers to array of values passed in */
-static int ar9003_hw_tx_power_regwrite(struct ath_hw *ah, u8 * pPwrArray)
+int ar9003_hw_tx_power_regwrite(struct ath_hw *ah, u8 * pPwrArray)
{
#define POW_SM(_r, _s) (((_r) & 0x3f) << (_s))
/* make sure forced gain is not set */
diff --git a/drivers/net/wireless/ath/ath9k/ar9003_eeprom.h b/drivers/net/wireless/ath/ath9k/ar9003_eeprom.h
index 694ca2e..107bcfb 100644
--- a/drivers/net/wireless/ath/ath9k/ar9003_eeprom.h
+++ b/drivers/net/wireless/ath/ath9k/ar9003_eeprom.h
@@ -355,5 +355,6 @@
struct ath9k_channel *chan);
void ar9003_hw_internal_regulator_apply(struct ath_hw *ah);
+int ar9003_hw_tx_power_regwrite(struct ath_hw *ah, u8 * pPwrArray);
#endif
diff --git a/drivers/net/wireless/ath/ath9k/ar9003_phy.c b/drivers/net/wireless/ath/ath9k/ar9003_phy.c
index be14a8e..ae304355 100644
--- a/drivers/net/wireless/ath/ath9k/ar9003_phy.c
+++ b/drivers/net/wireless/ath/ath9k/ar9003_phy.c
@@ -17,6 +17,7 @@
#include <linux/export.h>
#include "hw.h"
#include "ar9003_phy.h"
+#include "ar9003_eeprom.h"
#define AR9300_OFDM_RATES 8
#define AR9300_HT_SS_RATES 8
@@ -1009,7 +1010,7 @@
if (IS_CHAN_A_FAST_CLOCK(ah, chan))
rfMode |= (AR_PHY_MODE_DYNAMIC | AR_PHY_MODE_DYN_CCK_DISABLE);
- if (rfMode & (AR_PHY_MODE_QUARTER | AR_PHY_MODE_HALF))
+ if (IS_CHAN_HALF_RATE(chan) || IS_CHAN_QUARTER_RATE(chan))
REG_RMW_FIELD(ah, AR_PHY_FRAME_CTL,
AR_PHY_FRAME_CTL_CF_OVERLAP_WINDOW, 3);
@@ -1840,73 +1841,14 @@
static void ar9003_hw_tx99_set_txpower(struct ath_hw *ah, u8 txpower)
{
- static s16 p_pwr_array[ar9300RateSize] = { 0 };
+ static u8 p_pwr_array[ar9300RateSize] = { 0 };
unsigned int i;
- if (txpower <= MAX_RATE_POWER) {
- for (i = 0; i < ar9300RateSize; i++)
- p_pwr_array[i] = txpower;
- } else {
- for (i = 0; i < ar9300RateSize; i++)
- p_pwr_array[i] = MAX_RATE_POWER;
- }
+ txpower = txpower <= MAX_RATE_POWER ? txpower : MAX_RATE_POWER;
+ for (i = 0; i < ar9300RateSize; i++)
+ p_pwr_array[i] = txpower;
- REG_WRITE(ah, 0xa458, 0);
-
- REG_WRITE(ah, 0xa3c0,
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_LEGACY_6_24], 24) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_LEGACY_6_24], 16) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_LEGACY_6_24], 8) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_LEGACY_6_24], 0));
- REG_WRITE(ah, 0xa3c4,
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_LEGACY_54], 24) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_LEGACY_48], 16) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_LEGACY_36], 8) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_LEGACY_6_24], 0));
- REG_WRITE(ah, 0xa3c8,
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_LEGACY_1L_5L], 24) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_LEGACY_1L_5L], 16) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_LEGACY_1L_5L], 0));
- REG_WRITE(ah, 0xa3cc,
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_LEGACY_11S], 24) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_LEGACY_11L], 16) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_LEGACY_5S], 8) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_LEGACY_1L_5L], 0));
- REG_WRITE(ah, 0xa3d0,
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT20_5], 24) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT20_4], 16) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT20_1_3_9_11_17_19], 8)|
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT20_0_8_16], 0));
- REG_WRITE(ah, 0xa3d4,
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT20_13], 24) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT20_12], 16) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT20_7], 8) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT20_6], 0));
- REG_WRITE(ah, 0xa3e4,
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT20_21], 24) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT20_20], 16) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT20_15], 8) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT20_14], 0));
- REG_WRITE(ah, 0xa3e8,
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT40_23], 24) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT40_22], 16) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT20_23], 8) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT20_22], 0));
- REG_WRITE(ah, 0xa3d8,
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT40_5], 24) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT40_4], 16) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT40_1_3_9_11_17_19], 8) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT40_0_8_16], 0));
- REG_WRITE(ah, 0xa3dc,
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT40_13], 24) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT40_12], 16) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT40_7], 8) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT40_6], 0));
- REG_WRITE(ah, 0xa3ec,
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT40_21], 24) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT40_20], 16) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT40_15], 8) |
- ATH9K_POW_SM(p_pwr_array[ALL_TARGET_HT40_14], 0));
+ ar9003_hw_tx_power_regwrite(ah, p_pwr_array);
}
static void ar9003_hw_init_txpower_cck(struct ath_hw *ah, u8 *rate_array)
diff --git a/drivers/net/wireless/ath/wcn36xx/smd.h b/drivers/net/wireless/ath/wcn36xx/smd.h
index d74d781..d93e3fd 100644
--- a/drivers/net/wireless/ath/wcn36xx/smd.h
+++ b/drivers/net/wireless/ath/wcn36xx/smd.h
@@ -24,7 +24,7 @@
#define WCN36XX_HAL_BUF_SIZE 4096
-#define HAL_MSG_TIMEOUT 500
+#define HAL_MSG_TIMEOUT 10000
#define WCN36XX_SMSM_WLAN_TX_ENABLE 0x00000400
#define WCN36XX_SMSM_WLAN_TX_RINGS_EMPTY 0x00000200
/* The PNO version info be contained in the rsp msg */
diff --git a/drivers/net/wireless/ath/wil6210/cfg80211.c b/drivers/net/wireless/ath/wil6210/cfg80211.c
index 0fb3a79..5769811 100644
--- a/drivers/net/wireless/ath/wil6210/cfg80211.c
+++ b/drivers/net/wireless/ath/wil6210/cfg80211.c
@@ -375,8 +375,9 @@
return -EBUSY;
}
- /* scan on P2P_DEVICE is handled as p2p search */
- if (wdev->iftype == NL80211_IFTYPE_P2P_DEVICE) {
+ /* social scan on P2P_DEVICE is handled as p2p search */
+ if (wdev->iftype == NL80211_IFTYPE_P2P_DEVICE &&
+ wil_p2p_is_social_scan(request)) {
wil->scan_request = request;
wil->radio_wdev = wdev;
rc = wil_p2p_search(wil, request);
diff --git a/drivers/net/wireless/ath/wil6210/debug.c b/drivers/net/wireless/ath/wil6210/debug.c
index 3249562..c312a66 100644
--- a/drivers/net/wireless/ath/wil6210/debug.c
+++ b/drivers/net/wireless/ath/wil6210/debug.c
@@ -17,7 +17,7 @@
#include "wil6210.h"
#include "trace.h"
-void wil_err(struct wil6210_priv *wil, const char *fmt, ...)
+void __wil_err(struct wil6210_priv *wil, const char *fmt, ...)
{
struct net_device *ndev = wil_to_ndev(wil);
struct va_format vaf = {
@@ -32,7 +32,7 @@
va_end(args);
}
-void wil_err_ratelimited(struct wil6210_priv *wil, const char *fmt, ...)
+void __wil_err_ratelimited(struct wil6210_priv *wil, const char *fmt, ...)
{
if (net_ratelimit()) {
struct net_device *ndev = wil_to_ndev(wil);
@@ -49,7 +49,23 @@
}
}
-void wil_info(struct wil6210_priv *wil, const char *fmt, ...)
+void wil_dbg_ratelimited(const struct wil6210_priv *wil, const char *fmt, ...)
+{
+ struct va_format vaf;
+ va_list args;
+
+ if (!net_ratelimit())
+ return;
+
+ va_start(args, fmt);
+ vaf.fmt = fmt;
+ vaf.va = &args;
+ netdev_dbg(wil_to_ndev(wil), "%pV", &vaf);
+ trace_wil6210_log_dbg(&vaf);
+ va_end(args);
+}
+
+void __wil_info(struct wil6210_priv *wil, const char *fmt, ...)
{
struct net_device *ndev = wil_to_ndev(wil);
struct va_format vaf = {
diff --git a/drivers/net/wireless/ath/wil6210/debugfs.c b/drivers/net/wireless/ath/wil6210/debugfs.c
index b338a09..a8098b4 100644
--- a/drivers/net/wireless/ath/wil6210/debugfs.c
+++ b/drivers/net/wireless/ath/wil6210/debugfs.c
@@ -171,6 +171,8 @@
int rsize;
uint i;
+ wil_halp_vote(wil);
+
wil_memcpy_fromio_32(&r, off, sizeof(r));
wil_mbox_ring_le2cpus(&r);
/*
@@ -236,6 +238,7 @@
}
out:
seq_puts(s, "}\n");
+ wil_halp_unvote(wil);
}
static int wil_mbox_debugfs_show(struct seq_file *s, void *data)
@@ -500,9 +503,9 @@
size_t count, loff_t *ppos)
{
enum { max_count = 4096 };
- struct debugfs_blob_wrapper *blob = file->private_data;
+ struct wil_blob_wrapper *wil_blob = file->private_data;
loff_t pos = *ppos;
- size_t available = blob->size;
+ size_t available = wil_blob->blob.size;
void *buf;
size_t ret;
@@ -521,8 +524,9 @@
if (!buf)
return -ENOMEM;
- wil_memcpy_fromio_32(buf, (const volatile void __iomem *)blob->data +
- pos, count);
+ wil_memcpy_fromio_halp_vote(wil_blob->wil, buf,
+ (const volatile void __iomem *)
+ wil_blob->blob.data + pos, count);
ret = copy_to_user(user_buf, buf, count);
kfree(buf);
@@ -545,9 +549,9 @@
struct dentry *wil_debugfs_create_ioblob(const char *name,
umode_t mode,
struct dentry *parent,
- struct debugfs_blob_wrapper *blob)
+ struct wil_blob_wrapper *wil_blob)
{
- return debugfs_create_file(name, mode, parent, blob, &fops_ioblob);
+ return debugfs_create_file(name, mode, parent, wil_blob, &fops_ioblob);
}
/*---reset---*/
@@ -1437,6 +1441,118 @@
.llseek = seq_lseek,
};
+static ssize_t wil_read_file_led_cfg(struct file *file, char __user *user_buf,
+ size_t count, loff_t *ppos)
+{
+ char buf[80];
+ int n;
+
+ n = snprintf(buf, sizeof(buf),
+ "led_id is set to %d, echo 1 to enable, 0 to disable\n",
+ led_id);
+
+ n = min_t(int, n, sizeof(buf));
+
+ return simple_read_from_buffer(user_buf, count, ppos,
+ buf, n);
+}
+
+static ssize_t wil_write_file_led_cfg(struct file *file,
+ const char __user *buf_,
+ size_t count, loff_t *ppos)
+{
+ struct wil6210_priv *wil = file->private_data;
+ int val;
+ int rc;
+
+ rc = kstrtoint_from_user(buf_, count, 0, &val);
+ if (rc) {
+ wil_err(wil, "Invalid argument\n");
+ return rc;
+ }
+
+ wil_info(wil, "%s led %d\n", val ? "Enabling" : "Disabling", led_id);
+ rc = wmi_led_cfg(wil, val);
+ if (rc) {
+ wil_info(wil, "%s led %d failed\n",
+ val ? "Enabling" : "Disabling", led_id);
+ return rc;
+ }
+
+ return count;
+}
+
+static const struct file_operations fops_led_cfg = {
+ .read = wil_read_file_led_cfg,
+ .write = wil_write_file_led_cfg,
+ .open = simple_open,
+};
+
+/* led_blink_time, write:
+ * "<blink_on_slow> <blink_off_slow> <blink_on_med> <blink_off_med> <blink_on_fast> <blink_off_fast>
+ */
+static ssize_t wil_write_led_blink_time(struct file *file,
+ const char __user *buf,
+ size_t len, loff_t *ppos)
+{
+ int rc;
+ char *kbuf = kmalloc(len + 1, GFP_KERNEL);
+
+ if (!kbuf)
+ return -ENOMEM;
+
+ rc = simple_write_to_buffer(kbuf, len, ppos, buf, len);
+ if (rc != len) {
+ kfree(kbuf);
+ return rc >= 0 ? -EIO : rc;
+ }
+
+ kbuf[len] = '\0';
+ rc = sscanf(kbuf, "%d %d %d %d %d %d",
+ &led_blink_time[WIL_LED_TIME_SLOW].on_ms,
+ &led_blink_time[WIL_LED_TIME_SLOW].off_ms,
+ &led_blink_time[WIL_LED_TIME_MED].on_ms,
+ &led_blink_time[WIL_LED_TIME_MED].off_ms,
+ &led_blink_time[WIL_LED_TIME_FAST].on_ms,
+ &led_blink_time[WIL_LED_TIME_FAST].off_ms);
+ kfree(kbuf);
+
+ if (rc < 0)
+ return rc;
+ if (rc < 6)
+ return -EINVAL;
+
+ return len;
+}
+
+static ssize_t wil_read_led_blink_time(struct file *file, char __user *user_buf,
+ size_t count, loff_t *ppos)
+{
+ static char text[400];
+
+ snprintf(text, sizeof(text),
+ "To set led blink on/off time variables write:\n"
+ "<blink_on_slow> <blink_off_slow> <blink_on_med> "
+ "<blink_off_med> <blink_on_fast> <blink_off_fast>\n"
+ "The current values are:\n"
+ "%d %d %d %d %d %d\n",
+ led_blink_time[WIL_LED_TIME_SLOW].on_ms,
+ led_blink_time[WIL_LED_TIME_SLOW].off_ms,
+ led_blink_time[WIL_LED_TIME_MED].on_ms,
+ led_blink_time[WIL_LED_TIME_MED].off_ms,
+ led_blink_time[WIL_LED_TIME_FAST].on_ms,
+ led_blink_time[WIL_LED_TIME_FAST].off_ms);
+
+ return simple_read_from_buffer(user_buf, count, ppos, text,
+ sizeof(text));
+}
+
+static const struct file_operations fops_led_blink_time = {
+ .read = wil_read_led_blink_time,
+ .write = wil_write_led_blink_time,
+ .open = simple_open,
+};
+
/*----------------*/
static void wil6210_debugfs_init_blobs(struct wil6210_priv *wil,
struct dentry *dbg)
@@ -1445,16 +1561,18 @@
char name[32];
for (i = 0; i < ARRAY_SIZE(fw_mapping); i++) {
- struct debugfs_blob_wrapper *blob = &wil->blobs[i];
+ struct wil_blob_wrapper *wil_blob = &wil->blobs[i];
+ struct debugfs_blob_wrapper *blob = &wil_blob->blob;
const struct fw_map *map = &fw_mapping[i];
if (!map->name)
continue;
+ wil_blob->wil = wil;
blob->data = (void * __force)wil->csr + HOSTADDR(map->host);
blob->size = map->to - map->from;
snprintf(name, sizeof(name), "blob_%s", map->name);
- wil_debugfs_create_ioblob(name, S_IRUGO, dbg, blob);
+ wil_debugfs_create_ioblob(name, S_IRUGO, dbg, wil_blob);
}
}
@@ -1483,6 +1601,8 @@
{"link", S_IRUGO, &fops_link},
{"info", S_IRUGO, &fops_info},
{"recovery", S_IRUGO | S_IWUSR, &fops_recovery},
+ {"led_cfg", S_IRUGO | S_IWUSR, &fops_led_cfg},
+ {"led_blink_time", S_IRUGO | S_IWUSR, &fops_led_blink_time},
};
static void wil6210_debugfs_init_files(struct wil6210_priv *wil,
@@ -1545,6 +1665,7 @@
{"mem_addr", S_IRUGO | S_IWUSR, (ulong)&mem_addr, doff_u32},
{"vring_idle_trsh", S_IRUGO | S_IWUSR, (ulong)&vring_idle_trsh,
doff_u32},
+ {"led_polarity", S_IRUGO | S_IWUSR, (ulong)&led_polarity, doff_u8},
{},
};
diff --git a/drivers/net/wireless/ath/wil6210/interrupt.c b/drivers/net/wireless/ath/wil6210/interrupt.c
index fe66b2b..011e741 100644
--- a/drivers/net/wireless/ath/wil6210/interrupt.c
+++ b/drivers/net/wireless/ath/wil6210/interrupt.c
@@ -35,15 +35,19 @@
*
*/
-#define WIL6210_IRQ_DISABLE (0xFFFFFFFFUL)
+#define WIL6210_IRQ_DISABLE (0xFFFFFFFFUL)
+#define WIL6210_IRQ_DISABLE_NO_HALP (0xF7FFFFFFUL)
#define WIL6210_IMC_RX (BIT_DMA_EP_RX_ICR_RX_DONE | \
BIT_DMA_EP_RX_ICR_RX_HTRSH)
+#define WIL6210_IMC_RX_NO_RX_HTRSH (WIL6210_IMC_RX & \
+ (~(BIT_DMA_EP_RX_ICR_RX_HTRSH)))
#define WIL6210_IMC_TX (BIT_DMA_EP_TX_ICR_TX_DONE | \
BIT_DMA_EP_TX_ICR_TX_DONE_N(0))
-#define WIL6210_IMC_MISC (ISR_MISC_FW_READY | \
- ISR_MISC_MBOX_EVT | \
- ISR_MISC_FW_ERROR)
-
+#define WIL6210_IMC_MISC_NO_HALP (ISR_MISC_FW_READY | \
+ ISR_MISC_MBOX_EVT | \
+ ISR_MISC_FW_ERROR)
+#define WIL6210_IMC_MISC (WIL6210_IMC_MISC_NO_HALP | \
+ BIT_DMA_EP_MISC_ICR_HALP)
#define WIL6210_IRQ_PSEUDO_MASK (u32)(~(BIT_DMA_PSEUDO_CAUSE_RX | \
BIT_DMA_PSEUDO_CAUSE_TX | \
BIT_DMA_PSEUDO_CAUSE_MISC))
@@ -51,6 +55,7 @@
#if defined(CONFIG_WIL6210_ISR_COR)
/* configure to Clear-On-Read mode */
#define WIL_ICR_ICC_VALUE (0xFFFFFFFFUL)
+#define WIL_ICR_ICC_MISC_VALUE (0xF7FFFFFFUL)
static inline void wil_icr_clear(u32 x, void __iomem *addr)
{
@@ -58,6 +63,7 @@
#else /* defined(CONFIG_WIL6210_ISR_COR) */
/* configure to Write-1-to-Clear mode */
#define WIL_ICR_ICC_VALUE (0UL)
+#define WIL_ICR_ICC_MISC_VALUE (0UL)
static inline void wil_icr_clear(u32 x, void __iomem *addr)
{
@@ -86,10 +92,21 @@
WIL6210_IRQ_DISABLE);
}
-static void wil6210_mask_irq_misc(struct wil6210_priv *wil)
+static void wil6210_mask_irq_misc(struct wil6210_priv *wil, bool mask_halp)
{
+ wil_dbg_irq(wil, "%s: mask_halp(%s)\n", __func__,
+ mask_halp ? "true" : "false");
+
wil_w(wil, RGF_DMA_EP_MISC_ICR + offsetof(struct RGF_ICR, IMS),
- WIL6210_IRQ_DISABLE);
+ mask_halp ? WIL6210_IRQ_DISABLE : WIL6210_IRQ_DISABLE_NO_HALP);
+}
+
+static void wil6210_mask_halp(struct wil6210_priv *wil)
+{
+ wil_dbg_irq(wil, "%s()\n", __func__);
+
+ wil_w(wil, RGF_DMA_EP_MISC_ICR + offsetof(struct RGF_ICR, IMS),
+ BIT_DMA_EP_MISC_ICR_HALP);
}
static void wil6210_mask_irq_pseudo(struct wil6210_priv *wil)
@@ -109,14 +126,27 @@
void wil6210_unmask_irq_rx(struct wil6210_priv *wil)
{
+ bool unmask_rx_htrsh = test_bit(wil_status_fwconnected, wil->status);
+
wil_w(wil, RGF_DMA_EP_RX_ICR + offsetof(struct RGF_ICR, IMC),
- WIL6210_IMC_RX);
+ unmask_rx_htrsh ? WIL6210_IMC_RX : WIL6210_IMC_RX_NO_RX_HTRSH);
}
-static void wil6210_unmask_irq_misc(struct wil6210_priv *wil)
+static void wil6210_unmask_irq_misc(struct wil6210_priv *wil, bool unmask_halp)
{
+ wil_dbg_irq(wil, "%s: unmask_halp(%s)\n", __func__,
+ unmask_halp ? "true" : "false");
+
wil_w(wil, RGF_DMA_EP_MISC_ICR + offsetof(struct RGF_ICR, IMC),
- WIL6210_IMC_MISC);
+ unmask_halp ? WIL6210_IMC_MISC : WIL6210_IMC_MISC_NO_HALP);
+}
+
+static void wil6210_unmask_halp(struct wil6210_priv *wil)
+{
+ wil_dbg_irq(wil, "%s()\n", __func__);
+
+ wil_w(wil, RGF_DMA_EP_MISC_ICR + offsetof(struct RGF_ICR, IMC),
+ BIT_DMA_EP_MISC_ICR_HALP);
}
static void wil6210_unmask_irq_pseudo(struct wil6210_priv *wil)
@@ -134,7 +164,7 @@
wil6210_mask_irq_tx(wil);
wil6210_mask_irq_rx(wil);
- wil6210_mask_irq_misc(wil);
+ wil6210_mask_irq_misc(wil, true);
wil6210_mask_irq_pseudo(wil);
}
@@ -147,12 +177,12 @@
wil_w(wil, RGF_DMA_EP_TX_ICR + offsetof(struct RGF_ICR, ICC),
WIL_ICR_ICC_VALUE);
wil_w(wil, RGF_DMA_EP_MISC_ICR + offsetof(struct RGF_ICR, ICC),
- WIL_ICR_ICC_VALUE);
+ WIL_ICR_ICC_MISC_VALUE);
wil6210_unmask_irq_pseudo(wil);
wil6210_unmask_irq_tx(wil);
wil6210_unmask_irq_rx(wil);
- wil6210_unmask_irq_misc(wil);
+ wil6210_unmask_irq_misc(wil, true);
}
void wil_configure_interrupt_moderation(struct wil6210_priv *wil)
@@ -228,11 +258,8 @@
*/
if (likely(isr & (BIT_DMA_EP_RX_ICR_RX_DONE |
BIT_DMA_EP_RX_ICR_RX_HTRSH))) {
- wil_dbg_irq(wil, "RX done\n");
-
- if (unlikely(isr & BIT_DMA_EP_RX_ICR_RX_HTRSH))
- wil_err_ratelimited(wil,
- "Received \"Rx buffer is in risk of overflow\" interrupt\n");
+ wil_dbg_irq(wil, "RX done / RX_HTRSH received, ISR (0x%x)\n",
+ isr);
isr &= ~(BIT_DMA_EP_RX_ICR_RX_DONE |
BIT_DMA_EP_RX_ICR_RX_HTRSH);
@@ -344,7 +371,7 @@
return IRQ_NONE;
}
- wil6210_mask_irq_misc(wil);
+ wil6210_mask_irq_misc(wil, false);
if (isr & ISR_MISC_FW_ERROR) {
u32 fw_assert_code = wil_r(wil, RGF_FW_ASSERT_CODE);
@@ -372,12 +399,19 @@
isr &= ~ISR_MISC_FW_READY;
}
+ if (isr & BIT_DMA_EP_MISC_ICR_HALP) {
+ wil_dbg_irq(wil, "%s: HALP IRQ invoked\n", __func__);
+ wil6210_mask_halp(wil);
+ isr &= ~BIT_DMA_EP_MISC_ICR_HALP;
+ complete(&wil->halp.comp);
+ }
+
wil->isr_misc = isr;
if (isr) {
return IRQ_WAKE_THREAD;
} else {
- wil6210_unmask_irq_misc(wil);
+ wil6210_unmask_irq_misc(wil, false);
return IRQ_HANDLED;
}
}
@@ -414,7 +448,7 @@
wil->isr_misc = 0;
- wil6210_unmask_irq_misc(wil);
+ wil6210_unmask_irq_misc(wil, false);
return IRQ_HANDLED;
}
@@ -556,6 +590,23 @@
wmb(); /* make sure write completed */
}
+void wil6210_set_halp(struct wil6210_priv *wil)
+{
+ wil_dbg_misc(wil, "%s()\n", __func__);
+
+ wil_w(wil, RGF_DMA_EP_MISC_ICR + offsetof(struct RGF_ICR, ICS),
+ BIT_DMA_EP_MISC_ICR_HALP);
+}
+
+void wil6210_clear_halp(struct wil6210_priv *wil)
+{
+ wil_dbg_misc(wil, "%s()\n", __func__);
+
+ wil_w(wil, RGF_DMA_EP_MISC_ICR + offsetof(struct RGF_ICR, ICR),
+ BIT_DMA_EP_MISC_ICR_HALP);
+ wil6210_unmask_halp(wil);
+}
+
int wil6210_init_irq(struct wil6210_priv *wil, int irq, bool use_msi)
{
int rc;
diff --git a/drivers/net/wireless/ath/wil6210/main.c b/drivers/net/wireless/ath/wil6210/main.c
index 8d4e884..8e31d75 100644
--- a/drivers/net/wireless/ath/wil6210/main.c
+++ b/drivers/net/wireless/ath/wil6210/main.c
@@ -23,6 +23,8 @@
#include "wmi.h"
#include "boot_loader.h"
+#define WAIT_FOR_HALP_VOTE_MS 100
+
bool debug_fw; /* = false; */
module_param(debug_fw, bool, S_IRUGO);
MODULE_PARM_DESC(debug_fw, " do not perform card reset. For FW debug");
@@ -132,6 +134,14 @@
*d++ = __raw_readl(s++);
}
+void wil_memcpy_fromio_halp_vote(struct wil6210_priv *wil, void *dst,
+ const volatile void __iomem *src, size_t count)
+{
+ wil_halp_vote(wil);
+ wil_memcpy_fromio_32(dst, src, count);
+ wil_halp_unvote(wil);
+}
+
void wil_memcpy_toio_32(volatile void __iomem *dst, const void *src,
size_t count)
{
@@ -142,6 +152,15 @@
__raw_writel(*s++, d++);
}
+void wil_memcpy_toio_halp_vote(struct wil6210_priv *wil,
+ volatile void __iomem *dst,
+ const void *src, size_t count)
+{
+ wil_halp_vote(wil);
+ wil_memcpy_toio_32(dst, src, count);
+ wil_halp_unvote(wil);
+}
+
static void wil_disconnect_cid(struct wil6210_priv *wil, int cid,
u16 reason_code, bool from_event)
__acquires(&sta->tid_rx_lock) __releases(&sta->tid_rx_lock)
@@ -194,6 +213,18 @@
memset(&sta->stats, 0, sizeof(sta->stats));
}
+static bool wil_ap_is_connected(struct wil6210_priv *wil)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(wil->sta); i++) {
+ if (wil->sta[i].status == wil_sta_connected)
+ return true;
+ }
+
+ return false;
+}
+
static void _wil6210_disconnect(struct wil6210_priv *wil, const u8 *bssid,
u16 reason_code, bool from_event)
{
@@ -247,6 +278,11 @@
}
clear_bit(wil_status_fwconnecting, wil->status);
break;
+ case NL80211_IFTYPE_AP:
+ case NL80211_IFTYPE_P2P_GO:
+ if (!wil_ap_is_connected(wil))
+ clear_bit(wil_status_fwconnected, wil->status);
+ break;
default:
break;
}
@@ -457,9 +493,11 @@
mutex_init(&wil->wmi_mutex);
mutex_init(&wil->probe_client_mutex);
mutex_init(&wil->p2p_wdev_mutex);
+ mutex_init(&wil->halp.lock);
init_completion(&wil->wmi_ready);
init_completion(&wil->wmi_call);
+ init_completion(&wil->halp.comp);
wil->bcast_vring = -1;
setup_timer(&wil->connect_timer, wil_connect_timer_fn, (ulong)wil);
@@ -555,11 +593,10 @@
static void wil_set_oob_mode(struct wil6210_priv *wil, bool enable)
{
wil_info(wil, "%s: enable=%d\n", __func__, enable);
- if (enable) {
+ if (enable)
wil_s(wil, RGF_USER_USAGE_6, BIT_USER_OOB_MODE);
- } else {
+ else
wil_c(wil, RGF_USER_USAGE_6, BIT_USER_OOB_MODE);
- }
}
static int wil_target_reset(struct wil6210_priv *wil)
@@ -804,6 +841,9 @@
wil6210_disconnect(wil, NULL, WLAN_REASON_DEAUTH_LEAVING, false);
wil_bcast_fini(wil);
+ /* Disable device led before reset*/
+ wmi_led_cfg(wil, false);
+
/* prevent NAPI from being scheduled and prevent wmi commands */
mutex_lock(&wil->wmi_mutex);
bitmap_zero(wil->status, wil_status_last);
@@ -871,6 +911,7 @@
wil->ap_isolate = 0;
reinit_completion(&wil->wmi_ready);
reinit_completion(&wil->wmi_call);
+ reinit_completion(&wil->halp.comp);
if (load_fw) {
wil_configure_interrupt_moderation(wil);
@@ -1061,3 +1102,51 @@
return rc;
}
+
+void wil_halp_vote(struct wil6210_priv *wil)
+{
+ unsigned long rc;
+ unsigned long to_jiffies = msecs_to_jiffies(WAIT_FOR_HALP_VOTE_MS);
+
+ mutex_lock(&wil->halp.lock);
+
+ wil_dbg_misc(wil, "%s: start, HALP ref_cnt (%d)\n", __func__,
+ wil->halp.ref_cnt);
+
+ if (++wil->halp.ref_cnt == 1) {
+ wil6210_set_halp(wil);
+ rc = wait_for_completion_timeout(&wil->halp.comp, to_jiffies);
+ if (!rc)
+ wil_err(wil, "%s: HALP vote timed out\n", __func__);
+ else
+ wil_dbg_misc(wil,
+ "%s: HALP vote completed after %d ms\n",
+ __func__,
+ jiffies_to_msecs(to_jiffies - rc));
+ }
+
+ wil_dbg_misc(wil, "%s: end, HALP ref_cnt (%d)\n", __func__,
+ wil->halp.ref_cnt);
+
+ mutex_unlock(&wil->halp.lock);
+}
+
+void wil_halp_unvote(struct wil6210_priv *wil)
+{
+ WARN_ON(wil->halp.ref_cnt == 0);
+
+ mutex_lock(&wil->halp.lock);
+
+ wil_dbg_misc(wil, "%s: start, HALP ref_cnt (%d)\n", __func__,
+ wil->halp.ref_cnt);
+
+ if (--wil->halp.ref_cnt == 0) {
+ wil6210_clear_halp(wil);
+ wil_dbg_misc(wil, "%s: HALP unvote\n", __func__);
+ }
+
+ wil_dbg_misc(wil, "%s: end, HALP ref_cnt (%d)\n", __func__,
+ wil->halp.ref_cnt);
+
+ mutex_unlock(&wil->halp.lock);
+}
diff --git a/drivers/net/wireless/ath/wil6210/p2p.c b/drivers/net/wireless/ath/wil6210/p2p.c
index 2c1b895..1c91538 100644
--- a/drivers/net/wireless/ath/wil6210/p2p.c
+++ b/drivers/net/wireless/ath/wil6210/p2p.c
@@ -22,6 +22,12 @@
#define P2P_SEARCH_DURATION_MS 500
#define P2P_DEFAULT_BI 100
+bool wil_p2p_is_social_scan(struct cfg80211_scan_request *request)
+{
+ return (request->n_channels == 1) &&
+ (request->channels[0]->hw_value == P2P_DMG_SOCIAL_CHANNEL);
+}
+
void wil_p2p_discovery_timer_fn(ulong x)
{
struct wil6210_priv *wil = (void *)x;
diff --git a/drivers/net/wireless/ath/wil6210/txrx.c b/drivers/net/wireless/ath/wil6210/txrx.c
index f260b232..a4e4379 100644
--- a/drivers/net/wireless/ath/wil6210/txrx.c
+++ b/drivers/net/wireless/ath/wil6210/txrx.c
@@ -1759,7 +1759,7 @@
goto drop;
}
if (unlikely(!test_bit(wil_status_fwconnected, wil->status))) {
- wil_err_ratelimited(wil, "FW not connected\n");
+ wil_dbg_ratelimited(wil, "FW not connected, packet dropped\n");
goto drop;
}
if (unlikely(wil->wdev->iftype == NL80211_IFTYPE_MONITOR)) {
diff --git a/drivers/net/wireless/ath/wil6210/wil6210.h b/drivers/net/wireless/ath/wil6210/wil6210.h
index 4d699ea4..aa09cbc 100644
--- a/drivers/net/wireless/ath/wil6210/wil6210.h
+++ b/drivers/net/wireless/ath/wil6210/wil6210.h
@@ -168,6 +168,7 @@
#define RGF_DMA_EP_MISC_ICR (0x881bec) /* struct RGF_ICR */
#define BIT_DMA_EP_MISC_ICR_RX_HTRSH BIT(0)
#define BIT_DMA_EP_MISC_ICR_TX_NO_ACT BIT(1)
+ #define BIT_DMA_EP_MISC_ICR_HALP BIT(27)
#define BIT_DMA_EP_MISC_ICR_FW_INT(n) BIT(28+n) /* n = [0..3] */
/* Legacy interrupt moderation control (before Sparrow v2)*/
@@ -534,6 +535,41 @@
int descriptor_size;
};
+struct wil_halp {
+ struct mutex lock; /* protect halp ref_cnt */
+ unsigned int ref_cnt;
+ struct completion comp;
+};
+
+struct wil_blob_wrapper {
+ struct wil6210_priv *wil;
+ struct debugfs_blob_wrapper blob;
+};
+
+#define WIL_LED_MAX_ID (2)
+#define WIL_LED_INVALID_ID (0xF)
+#define WIL_LED_BLINK_ON_SLOW_MS (300)
+#define WIL_LED_BLINK_OFF_SLOW_MS (300)
+#define WIL_LED_BLINK_ON_MED_MS (200)
+#define WIL_LED_BLINK_OFF_MED_MS (200)
+#define WIL_LED_BLINK_ON_FAST_MS (100)
+#define WIL_LED_BLINK_OFF_FAST_MS (100)
+enum {
+ WIL_LED_TIME_SLOW = 0,
+ WIL_LED_TIME_MED,
+ WIL_LED_TIME_FAST,
+ WIL_LED_TIME_LAST,
+};
+
+struct blink_on_off_time {
+ u32 on_ms;
+ u32 off_ms;
+};
+
+extern struct blink_on_off_time led_blink_time[WIL_LED_TIME_LAST];
+extern u8 led_id;
+extern u8 led_polarity;
+
struct wil6210_priv {
struct pci_dev *pdev;
struct wireless_dev *wdev;
@@ -606,7 +642,7 @@
atomic_t isr_count_rx, isr_count_tx;
/* debugfs */
struct dentry *debug;
- struct debugfs_blob_wrapper blobs[ARRAY_SIZE(fw_mapping)];
+ struct wil_blob_wrapper blobs[ARRAY_SIZE(fw_mapping)];
u8 discovery_mode;
void *platform_handle;
@@ -622,6 +658,10 @@
struct wireless_dev *p2p_wdev;
struct mutex p2p_wdev_mutex; /* protect @p2p_wdev */
struct wireless_dev *radio_wdev;
+
+ /* High Access Latency Policy voting */
+ struct wil_halp halp;
+
};
#define wil_to_wiphy(i) (i->wdev->wiphy)
@@ -635,11 +675,13 @@
__printf(2, 3)
void wil_dbg_trace(struct wil6210_priv *wil, const char *fmt, ...);
__printf(2, 3)
-void wil_err(struct wil6210_priv *wil, const char *fmt, ...);
+void __wil_err(struct wil6210_priv *wil, const char *fmt, ...);
__printf(2, 3)
-void wil_err_ratelimited(struct wil6210_priv *wil, const char *fmt, ...);
+void __wil_err_ratelimited(struct wil6210_priv *wil, const char *fmt, ...);
__printf(2, 3)
-void wil_info(struct wil6210_priv *wil, const char *fmt, ...);
+void __wil_info(struct wil6210_priv *wil, const char *fmt, ...);
+__printf(2, 3)
+void wil_dbg_ratelimited(const struct wil6210_priv *wil, const char *fmt, ...);
#define wil_dbg(wil, fmt, arg...) do { \
netdev_dbg(wil_to_ndev(wil), fmt, ##arg); \
wil_dbg_trace(wil, fmt, ##arg); \
@@ -650,6 +692,10 @@
#define wil_dbg_wmi(wil, fmt, arg...) wil_dbg(wil, "DBG[ WMI]" fmt, ##arg)
#define wil_dbg_misc(wil, fmt, arg...) wil_dbg(wil, "DBG[MISC]" fmt, ##arg)
#define wil_dbg_pm(wil, fmt, arg...) wil_dbg(wil, "DBG[ PM ]" fmt, ##arg)
+#define wil_err(wil, fmt, arg...) __wil_err(wil, "%s: " fmt, __func__, ##arg)
+#define wil_info(wil, fmt, arg...) __wil_info(wil, "%s: " fmt, __func__, ##arg)
+#define wil_err_ratelimited(wil, fmt, arg...) \
+ __wil_err_ratelimited(wil, "%s: " fmt, __func__, ##arg)
/* target operations */
/* register read */
@@ -707,6 +753,12 @@
size_t count);
void wil_memcpy_toio_32(volatile void __iomem *dst, const void *src,
size_t count);
+void wil_memcpy_fromio_halp_vote(struct wil6210_priv *wil, void *dst,
+ const volatile void __iomem *src,
+ size_t count);
+void wil_memcpy_toio_halp_vote(struct wil6210_priv *wil,
+ volatile void __iomem *dst,
+ const void *src, size_t count);
void *wil_if_alloc(struct device *dev);
void wil_if_free(struct wil6210_priv *wil);
@@ -772,6 +824,7 @@
void wil_enable_irq(struct wil6210_priv *wil);
/* P2P */
+bool wil_p2p_is_social_scan(struct cfg80211_scan_request *request);
void wil_p2p_discovery_timer_fn(ulong x);
int wil_p2p_search(struct wil6210_priv *wil,
struct cfg80211_scan_request *request);
@@ -805,6 +858,7 @@
int wmi_pcp_start(struct wil6210_priv *wil, int bi, u8 wmi_nettype,
u8 chan, u8 hidden_ssid, u8 is_go);
int wmi_pcp_stop(struct wil6210_priv *wil);
+int wmi_led_cfg(struct wil6210_priv *wil, bool enable);
void wil6210_disconnect(struct wil6210_priv *wil, const u8 *bssid,
u16 reason_code, bool from_event);
void wil_probe_client_flush(struct wil6210_priv *wil);
@@ -842,4 +896,9 @@
int wil_fw_copy_crash_dump(struct wil6210_priv *wil, void *dest, u32 size);
void wil_fw_core_dump(struct wil6210_priv *wil);
+void wil_halp_vote(struct wil6210_priv *wil);
+void wil_halp_unvote(struct wil6210_priv *wil);
+void wil6210_set_halp(struct wil6210_priv *wil);
+void wil6210_clear_halp(struct wil6210_priv *wil);
+
#endif /* __WIL6210_H__ */
diff --git a/drivers/net/wireless/ath/wil6210/wmi.c b/drivers/net/wireless/ath/wil6210/wmi.c
index 6ca28c3..b80c5d8 100644
--- a/drivers/net/wireless/ath/wil6210/wmi.c
+++ b/drivers/net/wireless/ath/wil6210/wmi.c
@@ -32,6 +32,11 @@
MODULE_PARM_DESC(agg_wsize, " Window size for Tx Block Ack after connect;"
" 0 - use default; < 0 - don't auto-establish");
+u8 led_id = WIL_LED_INVALID_ID;
+module_param(led_id, byte, S_IRUGO);
+MODULE_PARM_DESC(led_id,
+ " 60G device led enablement. Set the led ID (0-2) to enable");
+
/**
* WMI event receiving - theory of operations
*
@@ -94,6 +99,14 @@
*/
};
+struct blink_on_off_time led_blink_time[] = {
+ {WIL_LED_BLINK_ON_SLOW_MS, WIL_LED_BLINK_OFF_SLOW_MS},
+ {WIL_LED_BLINK_ON_MED_MS, WIL_LED_BLINK_OFF_MED_MS},
+ {WIL_LED_BLINK_ON_FAST_MS, WIL_LED_BLINK_OFF_FAST_MS},
+};
+
+u8 led_polarity = LED_POLARITY_LOW_ACTIVE;
+
/**
* return AHB address for given firmware/ucode internal (linker) address
* @x - internal address
@@ -194,6 +207,7 @@
void __iomem *dst;
void __iomem *head = wmi_addr(wil, r->head);
uint retry;
+ int rc = 0;
if (sizeof(cmd) + len > r->entry_size) {
wil_err(wil, "WMI size too large: %d bytes, max is %d\n",
@@ -212,6 +226,9 @@
wil_err(wil, "WMI head is garbage: 0x%08x\n", r->head);
return -EINVAL;
}
+
+ wil_halp_vote(wil);
+
/* read Tx head till it is not busy */
for (retry = 5; retry > 0; retry--) {
wil_memcpy_fromio_32(&d_head, head, sizeof(d_head));
@@ -221,7 +238,8 @@
}
if (d_head.sync != 0) {
wil_err(wil, "WMI head busy\n");
- return -EBUSY;
+ rc = -EBUSY;
+ goto out;
}
/* next head */
next_head = r->base + ((r->head - r->base + sizeof(d_head)) % r->size);
@@ -230,7 +248,8 @@
for (retry = 5; retry > 0; retry--) {
if (!test_bit(wil_status_fwready, wil->status)) {
wil_err(wil, "WMI: cannot send command while FW not ready\n");
- return -EAGAIN;
+ rc = -EAGAIN;
+ goto out;
}
r->tail = wil_r(wil, RGF_MBOX +
offsetof(struct wil6210_mbox_ctl, tx.tail));
@@ -240,13 +259,15 @@
}
if (next_head == r->tail) {
wil_err(wil, "WMI ring full\n");
- return -EBUSY;
+ rc = -EBUSY;
+ goto out;
}
dst = wmi_buffer(wil, d_head.addr);
if (!dst) {
wil_err(wil, "invalid WMI buffer: 0x%08x\n",
le32_to_cpu(d_head.addr));
- return -EINVAL;
+ rc = -EAGAIN;
+ goto out;
}
cmd.hdr.seq = cpu_to_le16(++wil->wmi_seq);
/* set command */
@@ -269,7 +290,9 @@
wil_w(wil, RGF_USER_USER_ICR + offsetof(struct RGF_ICR, ICS),
SW_INT_MBOX);
- return 0;
+out:
+ wil_halp_unvote(wil);
+ return rc;
}
int wmi_send(struct wil6210_priv *wil, u16 cmdid, void *buf, u16 len)
@@ -961,6 +984,60 @@
return wmi_send(wil, WMI_SET_MAC_ADDRESS_CMDID, &cmd, sizeof(cmd));
}
+int wmi_led_cfg(struct wil6210_priv *wil, bool enable)
+{
+ int rc = 0;
+ struct wmi_led_cfg_cmd cmd = {
+ .led_mode = enable,
+ .id = led_id,
+ .slow_blink_cfg.blink_on =
+ cpu_to_le32(led_blink_time[WIL_LED_TIME_SLOW].on_ms),
+ .slow_blink_cfg.blink_off =
+ cpu_to_le32(led_blink_time[WIL_LED_TIME_SLOW].off_ms),
+ .medium_blink_cfg.blink_on =
+ cpu_to_le32(led_blink_time[WIL_LED_TIME_MED].on_ms),
+ .medium_blink_cfg.blink_off =
+ cpu_to_le32(led_blink_time[WIL_LED_TIME_MED].off_ms),
+ .fast_blink_cfg.blink_on =
+ cpu_to_le32(led_blink_time[WIL_LED_TIME_FAST].on_ms),
+ .fast_blink_cfg.blink_off =
+ cpu_to_le32(led_blink_time[WIL_LED_TIME_FAST].off_ms),
+ .led_polarity = led_polarity,
+ };
+ struct {
+ struct wmi_cmd_hdr wmi;
+ struct wmi_led_cfg_done_event evt;
+ } __packed reply;
+
+ if (led_id == WIL_LED_INVALID_ID)
+ goto out;
+
+ if (led_id > WIL_LED_MAX_ID) {
+ wil_err(wil, "Invalid led id %d\n", led_id);
+ rc = -EINVAL;
+ goto out;
+ }
+
+ wil_dbg_wmi(wil,
+ "%s led %d\n",
+ enable ? "enabling" : "disabling", led_id);
+
+ rc = wmi_call(wil, WMI_LED_CFG_CMDID, &cmd, sizeof(cmd),
+ WMI_LED_CFG_DONE_EVENTID, &reply, sizeof(reply),
+ 100);
+ if (rc)
+ goto out;
+
+ if (reply.evt.status) {
+ wil_err(wil, "led %d cfg failed with status %d\n",
+ led_id, le32_to_cpu(reply.evt.status));
+ rc = -EINVAL;
+ }
+
+out:
+ return rc;
+}
+
int wmi_pcp_start(struct wil6210_priv *wil, int bi, u8 wmi_nettype,
u8 chan, u8 hidden_ssid, u8 is_go)
{
@@ -1003,11 +1080,21 @@
if (reply.evt.status != WMI_FW_STATUS_SUCCESS)
rc = -EINVAL;
+ if (wmi_nettype != WMI_NETTYPE_P2P)
+ /* Don't fail due to error in the led configuration */
+ wmi_led_cfg(wil, true);
+
return rc;
}
int wmi_pcp_stop(struct wil6210_priv *wil)
{
+ int rc;
+
+ rc = wmi_led_cfg(wil, false);
+ if (rc)
+ return rc;
+
return wmi_call(wil, WMI_PCP_STOP_CMDID, NULL, 0,
WMI_PCP_STOPPED_EVENTID, NULL, 0, 20);
}
diff --git a/drivers/net/wireless/ath/wil6210/wmi.h b/drivers/net/wireless/ath/wil6210/wmi.h
index 29865e0..685fe0d 100644
--- a/drivers/net/wireless/ath/wil6210/wmi.h
+++ b/drivers/net/wireless/ath/wil6210/wmi.h
@@ -129,6 +129,7 @@
WMI_THERMAL_THROTTLING_GET_STATUS_CMDID = 0x855,
WMI_OTP_READ_CMDID = 0x856,
WMI_OTP_WRITE_CMDID = 0x857,
+ WMI_LED_CFG_CMDID = 0x858,
/* Performance monitoring commands */
WMI_BF_CTRL_CMDID = 0x862,
WMI_NOTIFY_REQ_CMDID = 0x863,
@@ -868,6 +869,7 @@
WMI_RX_MGMT_PACKET_EVENTID = 0x1840,
WMI_TX_MGMT_PACKET_EVENTID = 0x1841,
WMI_OTP_READ_RESULT_EVENTID = 0x1856,
+ WMI_LED_CFG_DONE_EVENTID = 0x1858,
/* Performance monitoring events */
WMI_DATA_PORT_OPEN_EVENTID = 0x1860,
WMI_WBE_LINK_DOWN_EVENTID = 0x1861,
@@ -1349,4 +1351,63 @@
WMI_HIDDEN_SSID_CLEAR = 0xFE,
};
+/* WMI_LED_CFG_CMDID
+ *
+ * Configure LED On\Off\Blinking operation
+ *
+ * Returned events:
+ * - WMI_LED_CFG_DONE_EVENTID
+ */
+enum led_mode {
+ LED_DISABLE = 0x00,
+ LED_ENABLE = 0x01,
+};
+
+/* The names of the led as
+ * described on HW schemes.
+ */
+enum wmi_led_id {
+ WMI_LED_WLAN = 0x00,
+ WMI_LED_WPAN = 0x01,
+ WMI_LED_WWAN = 0x02,
+};
+
+/* Led polarity mode. */
+enum wmi_led_polarity {
+ LED_POLARITY_HIGH_ACTIVE = 0x00,
+ LED_POLARITY_LOW_ACTIVE = 0x01,
+};
+
+/* Combination of on and off
+ * creates the blinking period
+ */
+struct wmi_led_blink_mode {
+ __le32 blink_on;
+ __le32 blink_off;
+} __packed;
+
+/* WMI_LED_CFG_CMDID */
+struct wmi_led_cfg_cmd {
+ /* enum led_mode_e */
+ u8 led_mode;
+ /* enum wmi_led_id_e */
+ u8 id;
+ /* slow speed blinking combination */
+ struct wmi_led_blink_mode slow_blink_cfg;
+ /* medium speed blinking combination */
+ struct wmi_led_blink_mode medium_blink_cfg;
+ /* high speed blinking combination */
+ struct wmi_led_blink_mode fast_blink_cfg;
+ /* polarity of the led */
+ u8 led_polarity;
+ /* reserved */
+ u8 reserved;
+} __packed;
+
+/* WMI_LED_CFG_DONE_EVENTID */
+struct wmi_led_cfg_done_event {
+ /* led config status */
+ __le32 status;
+} __packed;
+
#endif /* __WILOCITY_WMI_H__ */
diff --git a/drivers/net/wireless/atmel/atmel.c b/drivers/net/wireless/atmel/atmel.c
index 8f8f37f..bf2e9a0 100644
--- a/drivers/net/wireless/atmel/atmel.c
+++ b/drivers/net/wireless/atmel/atmel.c
@@ -2275,7 +2275,7 @@
fwrq->m = ieee80211_frequency_to_channel(f);
}
/* Setting by channel number */
- if ((fwrq->m > 1000) || (fwrq->e > 0))
+ if (fwrq->m < 0 || fwrq->m > 1000 || fwrq->e > 0)
rc = -EOPNOTSUPP;
else {
int channel = fwrq->m;
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c
index 2fc0597..c7550da 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/bcmsdh.c
@@ -1098,6 +1098,7 @@
BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_43430),
BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_4345),
BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_4354),
+ BRCMF_SDIO_DEVICE(SDIO_DEVICE_ID_BROADCOM_4356),
{ /* end: all zeroes */ }
};
MODULE_DEVICE_TABLE(sdio, brcmf_sdmmc_ids);
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.c
index 0e8f2a0..d3fd6b1 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/chip.c
@@ -1333,6 +1333,7 @@
switch (pub->chip) {
case BRCM_CC_4354_CHIP_ID:
+ case BRCM_CC_4356_CHIP_ID:
/* explicitly check SR engine enable bit */
pmu_cc3_mask = BIT(2);
/* fall-through */
diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c
index 4252fa8..67e69bf 100644
--- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c
+++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/sdio.c
@@ -609,6 +609,7 @@
BRCMF_FW_NVRAM_DEF(43430, "brcmfmac43430-sdio.bin", "brcmfmac43430-sdio.txt");
BRCMF_FW_NVRAM_DEF(43455, "brcmfmac43455-sdio.bin", "brcmfmac43455-sdio.txt");
BRCMF_FW_NVRAM_DEF(4354, "brcmfmac4354-sdio.bin", "brcmfmac4354-sdio.txt");
+BRCMF_FW_NVRAM_DEF(4356, "brcmfmac4356-sdio.bin", "brcmfmac4356-sdio.txt");
static struct brcmf_firmware_mapping brcmf_sdio_fwnames[] = {
BRCMF_FW_NVRAM_ENTRY(BRCM_CC_43143_CHIP_ID, 0xFFFFFFFF, 43143),
@@ -624,7 +625,8 @@
BRCMF_FW_NVRAM_ENTRY(BRCM_CC_4339_CHIP_ID, 0xFFFFFFFF, 4339),
BRCMF_FW_NVRAM_ENTRY(BRCM_CC_43430_CHIP_ID, 0xFFFFFFFF, 43430),
BRCMF_FW_NVRAM_ENTRY(BRCM_CC_4345_CHIP_ID, 0xFFFFFFC0, 43455),
- BRCMF_FW_NVRAM_ENTRY(BRCM_CC_4354_CHIP_ID, 0xFFFFFFFF, 4354)
+ BRCMF_FW_NVRAM_ENTRY(BRCM_CC_4354_CHIP_ID, 0xFFFFFFFF, 4354),
+ BRCMF_FW_NVRAM_ENTRY(BRCM_CC_4356_CHIP_ID, 0xFFFFFFFF, 4356)
};
static void pkt_align(struct sk_buff *p, int len, int align)
diff --git a/drivers/net/wireless/cisco/airo.c b/drivers/net/wireless/cisco/airo.c
index 55456f7..ca3cd21 100644
--- a/drivers/net/wireless/cisco/airo.c
+++ b/drivers/net/wireless/cisco/airo.c
@@ -5794,7 +5794,7 @@
fwrq->m = ieee80211_frequency_to_channel(f);
}
/* Setting by channel number */
- if((fwrq->m > 1000) || (fwrq->e > 0))
+ if (fwrq->m < 0 || fwrq->m > 1000 || fwrq->e > 0)
rc = -EOPNOTSUPP;
else {
int channel = fwrq->m;
diff --git a/drivers/net/wireless/intel/iwlwifi/Kconfig b/drivers/net/wireless/intel/iwlwifi/Kconfig
index 492035f..b64db47 100644
--- a/drivers/net/wireless/intel/iwlwifi/Kconfig
+++ b/drivers/net/wireless/intel/iwlwifi/Kconfig
@@ -134,12 +134,6 @@
is a low-impact option that allows getting insight into the
driver's state at runtime.
-config IWLWIFI_DEBUG_EXPERIMENTAL_UCODE
- bool "Experimental uCode support"
- depends on IWLWIFI_DEBUG
- ---help---
- Enable use of experimental ucode for testing and debugging.
-
config IWLWIFI_DEVICE_TRACING
bool "iwlwifi device access tracing"
depends on EVENT_TRACING
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-1000.c b/drivers/net/wireless/intel/iwlwifi/iwl-1000.c
index 5c2aae6..b2573b1 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-1000.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-1000.c
@@ -52,7 +52,7 @@
static const struct iwl_base_params iwl1000_base_params = {
.num_of_queues = IWLAGN_NUM_QUEUES,
.eeprom_size = OTP_LOW_IMAGE_SIZE,
- .pll_cfg_val = CSR50_ANA_PLL_CFG_VAL,
+ .pll_cfg = true,
.max_ll_items = OTP_MAX_LL_ITEMS_1000,
.shadow_ram_support = false,
.led_compensation = 51,
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-2000.c b/drivers/net/wireless/intel/iwlwifi/iwl-2000.c
index 2e823bd..1b32ad4 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-2000.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-2000.c
@@ -62,7 +62,6 @@
static const struct iwl_base_params iwl2000_base_params = {
.eeprom_size = OTP_LOW_IMAGE_SIZE,
.num_of_queues = IWLAGN_NUM_QUEUES,
- .pll_cfg_val = 0,
.max_ll_items = OTP_MAX_LL_ITEMS_2x00,
.shadow_ram_support = true,
.led_compensation = 51,
@@ -76,7 +75,6 @@
static const struct iwl_base_params iwl2030_base_params = {
.eeprom_size = OTP_LOW_IMAGE_SIZE,
.num_of_queues = IWLAGN_NUM_QUEUES,
- .pll_cfg_val = 0,
.max_ll_items = OTP_MAX_LL_ITEMS_2x00,
.shadow_ram_support = true,
.led_compensation = 57,
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-5000.c b/drivers/net/wireless/intel/iwlwifi/iwl-5000.c
index 4c3e3cf..4aa8f0a 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-5000.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-5000.c
@@ -53,7 +53,7 @@
static const struct iwl_base_params iwl5000_base_params = {
.eeprom_size = IWLAGN_EEPROM_IMG_SIZE,
.num_of_queues = IWLAGN_NUM_QUEUES,
- .pll_cfg_val = CSR50_ANA_PLL_CFG_VAL,
+ .pll_cfg = true,
.led_compensation = 51,
.wd_timeout = IWL_WATCHDOG_DISABLED,
.max_event_log_size = 512,
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-6000.c b/drivers/net/wireless/intel/iwlwifi/iwl-6000.c
index 5a7b7e1..0b9f6a7 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-6000.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-6000.c
@@ -71,7 +71,6 @@
static const struct iwl_base_params iwl6000_base_params = {
.eeprom_size = OTP_LOW_IMAGE_SIZE,
.num_of_queues = IWLAGN_NUM_QUEUES,
- .pll_cfg_val = 0,
.max_ll_items = OTP_MAX_LL_ITEMS_6x00,
.shadow_ram_support = true,
.led_compensation = 51,
@@ -84,7 +83,6 @@
static const struct iwl_base_params iwl6050_base_params = {
.eeprom_size = OTP_LOW_IMAGE_SIZE,
.num_of_queues = IWLAGN_NUM_QUEUES,
- .pll_cfg_val = 0,
.max_ll_items = OTP_MAX_LL_ITEMS_6x50,
.shadow_ram_support = true,
.led_compensation = 51,
@@ -97,7 +95,6 @@
static const struct iwl_base_params iwl6000_g2_base_params = {
.eeprom_size = OTP_LOW_IMAGE_SIZE,
.num_of_queues = IWLAGN_NUM_QUEUES,
- .pll_cfg_val = 0,
.max_ll_items = OTP_MAX_LL_ITEMS_6x00,
.shadow_ram_support = true,
.led_compensation = 57,
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-7000.c b/drivers/net/wireless/intel/iwlwifi/iwl-7000.c
index abd2904..f4d9215 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-7000.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-7000.c
@@ -122,7 +122,6 @@
static const struct iwl_base_params iwl7000_base_params = {
.eeprom_size = OTP_LOW_IMAGE_SIZE_FAMILY_7000,
.num_of_queues = 31,
- .pll_cfg_val = 0,
.shadow_ram_support = true,
.led_compensation = 57,
.wd_timeout = IWL_LONG_WD_TIMEOUT,
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-8000.c b/drivers/net/wireless/intel/iwlwifi/iwl-8000.c
index 2d20556..8bf11c9 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-8000.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-8000.c
@@ -112,7 +112,6 @@
static const struct iwl_base_params iwl8000_base_params = {
.eeprom_size = OTP_LOW_IMAGE_SIZE_FAMILY_8000,
.num_of_queues = 31,
- .pll_cfg_val = 0,
.shadow_ram_support = true,
.led_compensation = 57,
.wd_timeout = IWL_LONG_WD_TIMEOUT,
@@ -237,6 +236,20 @@
.max_vht_ampdu_exponent = MAX_VHT_AMPDU_EXPONENT_8260_SDIO,
};
+const struct iwl_cfg iwl8265_2ac_sdio_cfg = {
+ .name = "Intel(R) Dual Band Wireless-AC 8265",
+ .fw_name_pre = IWL8265_FW_PRE,
+ IWL_DEVICE_8265,
+ .ht_params = &iwl8000_ht_params,
+ .nvm_ver = IWL8000_NVM_VERSION,
+ .nvm_calib_ver = IWL8000_TX_POWER_VERSION,
+ .max_rx_agg_size = MAX_RX_AGG_SIZE_8260_SDIO,
+ .max_tx_agg_size = MAX_TX_AGG_SIZE_8260_SDIO,
+ .disable_dummy_notification = true,
+ .max_ht_ampdu_exponent = MAX_HT_AMPDU_EXPONENT_8260_SDIO,
+ .max_vht_ampdu_exponent = MAX_VHT_AMPDU_EXPONENT_8260_SDIO,
+};
+
const struct iwl_cfg iwl4165_2ac_sdio_cfg = {
.name = "Intel(R) Dual Band Wireless-AC 4165",
.fw_name_pre = IWL8000_FW_PRE,
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-9000.c b/drivers/net/wireless/intel/iwlwifi/iwl-9000.c
index b9aca37..3ac298f 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-9000.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-9000.c
@@ -72,16 +72,21 @@
#define IWL9000_SMEM_OFFSET 0x400000
#define IWL9000_SMEM_LEN 0x68000
-#define IWL9000_FW_PRE "iwlwifi-9000-"
+#define IWL9000_FW_PRE "iwlwifi-9000-pu-a0-lc-a0-"
+#define IWL9260_FW_PRE "iwlwifi-9260-th-a0-jf-a0-"
+#define IWL9260LC_FW_PRE "iwlwifi-9260-th-a0-lc-a0-"
#define IWL9000_MODULE_FIRMWARE(api) \
IWL9000_FW_PRE "-" __stringify(api) ".ucode"
+#define IWL9260_MODULE_FIRMWARE(api) \
+ IWL9260_FW_PRE "-" __stringify(api) ".ucode"
+#define IWL9260LC_MODULE_FIRMWARE(api) \
+ IWL9260LC_FW_PRE "-" __stringify(api) ".ucode"
#define NVM_HW_SECTION_NUM_FAMILY_9000 10
static const struct iwl_base_params iwl9000_base_params = {
.eeprom_size = OTP_LOW_IMAGE_SIZE_FAMILY_9000,
.num_of_queues = 31,
- .pll_cfg_val = 0,
.shadow_ram_support = true,
.led_compensation = 57,
.wd_timeout = IWL_LONG_WD_TIMEOUT,
@@ -138,11 +143,26 @@
.apmg_not_supported = true, \
.mq_rx_supported = true, \
.vht_mu_mimo_supported = true, \
- .mac_addr_from_csr = true
+ .mac_addr_from_csr = true, \
+ .rf_id = true
-const struct iwl_cfg iwl9560_2ac_cfg = {
- .name = "Intel(R) Dual Band Wireless AC 9560",
- .fw_name_pre = IWL9000_FW_PRE,
+const struct iwl_cfg iwl9260_2ac_cfg = {
+ .name = "Intel(R) Dual Band Wireless AC 9260",
+ .fw_name_pre = IWL9260_FW_PRE,
+ IWL_DEVICE_9000,
+ .ht_params = &iwl9000_ht_params,
+ .nvm_ver = IWL9000_NVM_VERSION,
+ .nvm_calib_ver = IWL9000_TX_POWER_VERSION,
+ .max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K,
+};
+
+/*
+ * TODO the struct below is for internal testing only this should be
+ * removed by EO 2016~
+ */
+const struct iwl_cfg iwl9260lc_2ac_cfg = {
+ .name = "Intel(R) Dual Band Wireless AC 9260",
+ .fw_name_pre = IWL9260LC_FW_PRE,
IWL_DEVICE_9000,
.ht_params = &iwl9000_ht_params,
.nvm_ver = IWL9000_NVM_VERSION,
@@ -161,3 +181,5 @@
};
MODULE_FIRMWARE(IWL9000_MODULE_FIRMWARE(IWL9000_UCODE_API_MAX));
+MODULE_FIRMWARE(IWL9260_MODULE_FIRMWARE(IWL9000_UCODE_API_MAX));
+MODULE_FIRMWARE(IWL9260LC_MODULE_FIRMWARE(IWL9000_UCODE_API_MAX));
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-config.h b/drivers/net/wireless/intel/iwlwifi/iwl-config.h
index 7206798..4a0af7d 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-config.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-config.h
@@ -6,6 +6,7 @@
* GPL LICENSE SUMMARY
*
* Copyright(c) 2007 - 2014 Intel Corporation. All rights reserved.
+ * Copyright (C) 2016 Intel Deutschland GmbH
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
@@ -31,6 +32,7 @@
* BSD LICENSE
*
* Copyright(c) 2005 - 2014 Intel Corporation. All rights reserved.
+ * Copyright (C) 2016 Intel Deutschland GmbH
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -165,20 +167,22 @@
* @scd_chain_ext_wa: should the chain extension feature in SCD be disabled.
*/
struct iwl_base_params {
- int eeprom_size;
- int num_of_queues; /* def: HW dependent */
- /* for iwl_pcie_apm_init() */
- u32 pll_cfg_val;
-
- const u16 max_ll_items;
- const bool shadow_ram_support;
- u16 led_compensation;
unsigned int wd_timeout;
- u32 max_event_log_size;
- const bool shadow_reg_enable;
- const bool pcie_l1_allowed;
- const bool apmg_wake_up_wa;
- const bool scd_chain_ext_wa;
+
+ u16 eeprom_size;
+ u16 max_event_log_size;
+
+ u8 pll_cfg:1, /* for iwl_pcie_apm_init() */
+ shadow_ram_support:1,
+ shadow_reg_enable:1,
+ pcie_l1_allowed:1,
+ apmg_wake_up_wa:1,
+ scd_chain_ext_wa:1;
+
+ u8 num_of_queues; /* def: HW dependent */
+
+ u8 max_ll_items;
+ u8 led_compensation;
};
/*
@@ -189,10 +193,10 @@
*/
struct iwl_ht_params {
enum ieee80211_smps_mode smps_mode;
- const bool ht_greenfield_support; /* if used set to true */
- const bool stbc;
- const bool ldpc;
- bool use_rts_for_aggregation;
+ u8 ht_greenfield_support:1,
+ stbc:1,
+ ldpc:1,
+ use_rts_for_aggregation:1;
u8 ht40_bands;
};
@@ -233,10 +237,10 @@
u32 tx_protection_entry;
u32 tx_protection_exit;
struct iwl_tt_tx_backoff tx_backoff[TT_TX_BACKOFF_SIZE];
- bool support_ct_kill;
- bool support_dynamic_smps;
- bool support_tx_protection;
- bool support_tx_backoff;
+ u8 support_ct_kill:1,
+ support_dynamic_smps:1,
+ support_tx_protection:1,
+ support_tx_backoff:1;
};
/*
@@ -314,6 +318,7 @@
* @smem_len: the length of SMEM
* @mq_rx_supported: multi-queue rx support
* @vht_mu_mimo_supported: VHT MU-MIMO support
+ * @rf_id: need to read rf_id to determine the firmware image
*
* We enable the driver to be backward compatible wrt. hardware features.
* API differences in uCode shouldn't be handled here but through TLVs
@@ -323,50 +328,51 @@
/* params specific to an individual device within a device family */
const char *name;
const char *fw_name_pre;
- const unsigned int ucode_api_max;
- const unsigned int ucode_api_min;
- const enum iwl_device_family device_family;
- const u32 max_data_size;
- const u32 max_inst_size;
- u8 valid_tx_ant;
- u8 valid_rx_ant;
- u8 non_shared_ant;
- bool bt_shared_single_ant;
- u16 nvm_ver;
- u16 nvm_calib_ver;
/* params not likely to change within a device family */
const struct iwl_base_params *base_params;
/* params likely to change within a device family */
const struct iwl_ht_params *ht_params;
const struct iwl_eeprom_params *eeprom_params;
- enum iwl_led_mode led_mode;
- const bool rx_with_siso_diversity;
- const bool internal_wimax_coex;
- const bool host_interrupt_operation_mode;
- bool high_temp;
- u8 nvm_hw_section_num;
- bool mac_addr_from_csr;
- bool lp_xtal_workaround;
const struct iwl_pwr_tx_backoff *pwr_tx_backoffs;
- bool no_power_up_nic_in_init;
const char *default_nvm_file_B_step;
const char *default_nvm_file_C_step;
- netdev_features_t features;
- unsigned int max_rx_agg_size;
- bool disable_dummy_notification;
- unsigned int max_tx_agg_size;
- unsigned int max_ht_ampdu_exponent;
- unsigned int max_vht_ampdu_exponent;
- const u32 dccm_offset;
- const u32 dccm_len;
- const u32 dccm2_offset;
- const u32 dccm2_len;
- const u32 smem_offset;
- const u32 smem_len;
const struct iwl_tt_params *thermal_params;
- bool apmg_not_supported;
- bool mq_rx_supported;
- bool vht_mu_mimo_supported;
+ enum iwl_device_family device_family;
+ enum iwl_led_mode led_mode;
+ u32 max_data_size;
+ u32 max_inst_size;
+ netdev_features_t features;
+ u32 dccm_offset;
+ u32 dccm_len;
+ u32 dccm2_offset;
+ u32 dccm2_len;
+ u32 smem_offset;
+ u32 smem_len;
+ u16 nvm_ver;
+ u16 nvm_calib_ver;
+ u16 rx_with_siso_diversity:1,
+ bt_shared_single_ant:1,
+ internal_wimax_coex:1,
+ host_interrupt_operation_mode:1,
+ high_temp:1,
+ mac_addr_from_csr:1,
+ lp_xtal_workaround:1,
+ no_power_up_nic_in_init:1,
+ disable_dummy_notification:1,
+ apmg_not_supported:1,
+ mq_rx_supported:1,
+ vht_mu_mimo_supported:1,
+ rf_id:1;
+ u8 valid_tx_ant;
+ u8 valid_rx_ant;
+ u8 non_shared_ant;
+ u8 nvm_hw_section_num;
+ u8 max_rx_agg_size;
+ u8 max_tx_agg_size;
+ u8 max_ht_ampdu_exponent;
+ u8 max_vht_ampdu_exponent;
+ u8 ucode_api_max;
+ u8 ucode_api_min;
};
/*
@@ -437,8 +443,10 @@
extern const struct iwl_cfg iwl8265_2ac_cfg;
extern const struct iwl_cfg iwl4165_2ac_cfg;
extern const struct iwl_cfg iwl8260_2ac_sdio_cfg;
+extern const struct iwl_cfg iwl8265_2ac_sdio_cfg;
extern const struct iwl_cfg iwl4165_2ac_sdio_cfg;
-extern const struct iwl_cfg iwl9560_2ac_cfg;
+extern const struct iwl_cfg iwl9260_2ac_cfg;
+extern const struct iwl_cfg iwl9260lc_2ac_cfg;
extern const struct iwl_cfg iwl5165_2ac_cfg;
#endif /* CONFIG_IWLMVM */
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-csr.h b/drivers/net/wireless/intel/iwlwifi/iwl-csr.h
index b978f6c..b5291344 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-csr.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-csr.h
@@ -108,6 +108,17 @@
#define CSR_HW_REV (CSR_BASE+0x028)
/*
+ * RF ID revision info
+ * Bit fields:
+ * 31:24: Reserved (set to 0x0)
+ * 23:12: Type
+ * 11:8: Step (A - 0x0, B - 0x1, etc)
+ * 7:4: Dash
+ * 3:0: Flavor
+ */
+#define CSR_HW_RF_ID (CSR_BASE+0x09c)
+
+/*
* EEPROM and OTP (one-time-programmable) memory reads
*
* NOTE: Device must be awake, initialized via apm_ops.init(),
@@ -333,6 +344,10 @@
#define CSR_HW_REV_TYPE_7265D (0x0000210)
#define CSR_HW_REV_TYPE_NONE (0x00001F0)
+/* RF_ID value */
+#define CSR_HW_RF_ID_TYPE_JF (0x00105000)
+#define CSR_HW_RF_ID_TYPE_LC (0x00101000)
+
/* EEPROM REG */
#define CSR_EEPROM_REG_READ_VALID_MSK (0x00000001)
#define CSR_EEPROM_REG_BIT_CMD (0x00000002)
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-drv.c b/drivers/net/wireless/intel/iwlwifi/iwl-drv.c
index ff18b06..f52ff75 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-drv.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-drv.c
@@ -117,7 +117,7 @@
const struct iwl_cfg *cfg;
int fw_index; /* firmware we're trying to load */
- char firmware_name[32]; /* name of firmware file to load */
+ char firmware_name[64]; /* name of firmware file to load */
struct completion request_firmware_complete;
@@ -211,20 +211,12 @@
static void iwl_req_fw_callback(const struct firmware *ucode_raw,
void *context);
-#define UCODE_EXPERIMENTAL_INDEX 100
-#define UCODE_EXPERIMENTAL_TAG "exp"
-
static int iwl_request_firmware(struct iwl_drv *drv, bool first)
{
const char *name_pre = drv->cfg->fw_name_pre;
char tag[8];
if (first) {
-#ifdef CONFIG_IWLWIFI_DEBUG_EXPERIMENTAL_UCODE
- drv->fw_index = UCODE_EXPERIMENTAL_INDEX;
- strcpy(tag, UCODE_EXPERIMENTAL_TAG);
- } else if (drv->fw_index == UCODE_EXPERIMENTAL_INDEX) {
-#endif
drv->fw_index = drv->cfg->ucode_api_max;
sprintf(tag, "%d", drv->fw_index);
} else {
@@ -240,9 +232,7 @@
snprintf(drv->firmware_name, sizeof(drv->firmware_name), "%s%s.ucode",
name_pre, tag);
- IWL_DEBUG_INFO(drv, "attempting to load firmware %s'%s'\n",
- (drv->fw_index == UCODE_EXPERIMENTAL_INDEX)
- ? "EXPERIMENTAL " : "",
+ IWL_DEBUG_INFO(drv, "attempting to load firmware '%s'\n",
drv->firmware_name);
return request_firmware_nowait(THIS_MODULE, 1, drv->firmware_name,
@@ -541,9 +531,7 @@
}
if (build)
- sprintf(buildstr, " build %u%s", build,
- (drv->fw_index == UCODE_EXPERIMENTAL_INDEX)
- ? " (EXP)" : "");
+ sprintf(buildstr, " build %u", build);
else
buildstr[0] = '\0';
@@ -627,9 +615,7 @@
build = le32_to_cpu(ucode->build);
if (build)
- sprintf(buildstr, " build %u%s", build,
- (drv->fw_index == UCODE_EXPERIMENTAL_INDEX)
- ? " (EXP)" : "");
+ sprintf(buildstr, " build %u", build);
else
buildstr[0] = '\0';
@@ -1277,15 +1263,12 @@
* firmware filename ... but we don't check for that and only rely
* on the API version read from firmware header from here on forward
*/
- /* no api version check required for experimental uCode */
- if (drv->fw_index != UCODE_EXPERIMENTAL_INDEX) {
- if (api_ver < api_min || api_ver > api_max) {
- IWL_ERR(drv,
- "Driver unable to support your firmware API. "
- "Driver supports v%u, firmware is v%u.\n",
- api_max, api_ver);
- goto try_again;
- }
+ if (api_ver < api_min || api_ver > api_max) {
+ IWL_ERR(drv,
+ "Driver unable to support your firmware API. "
+ "Driver supports v%u, firmware is v%u.\n",
+ api_max, api_ver);
+ goto try_again;
}
/*
@@ -1744,4 +1727,4 @@
module_param_named(disable_11ac, iwlwifi_mod_params.disable_11ac, bool,
S_IRUGO);
-MODULE_PARM_DESC(disable_11ac, "Disable VHT capabilities");
+MODULE_PARM_DESC(disable_11ac, "Disable VHT capabilities (default: false)");
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.h b/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.h
index 53f39a3..1f4e502 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-eeprom-parse.h
@@ -98,6 +98,7 @@
s8 max_tx_pwr_half_dbm;
bool lar_enabled;
+ bool vht160_supported;
struct ieee80211_supported_band bands[NUM_NL80211_BANDS];
struct ieee80211_channel channels[];
};
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-fh.h b/drivers/net/wireless/intel/iwlwifi/iwl-fh.h
index 582008a..270f39e 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-fh.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-fh.h
@@ -321,6 +321,9 @@
/* Write index table */
#define RFH_Q0_FRBDCB_WIDX 0xA08080
#define RFH_Q_FRBDCB_WIDX(q) (RFH_Q0_FRBDCB_WIDX + (q) * 4)
+/* Write index table - shadow registers */
+#define RFH_Q0_FRBDCB_WIDX_TRG 0x1C80
+#define RFH_Q_FRBDCB_WIDX_TRG(q) (RFH_Q0_FRBDCB_WIDX_TRG + (q) * 4)
/* Read index table */
#define RFH_Q0_FRBDCB_RIDX 0xA080C0
#define RFH_Q_FRBDCB_RIDX(q) (RFH_Q0_FRBDCB_RIDX + (q) * 4)
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c b/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c
index 14743c3..21653fe 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c
@@ -288,6 +288,9 @@
!data->sku_cap_band_52GHz_enable)
continue;
+ if (ch_flags & NVM_CHANNEL_160MHZ)
+ data->vht160_supported = true;
+
if (!lar_supported && !(ch_flags & NVM_CHANNEL_VALID)) {
/*
* Channels might become valid later if lar is
@@ -331,17 +334,20 @@
channel->flags = 0;
IWL_DEBUG_EEPROM(dev,
- "Ch. %d [%sGHz] %s%s%s%s%s%s%s(0x%02x %ddBm): Ad-Hoc %ssupported\n",
+ "Ch. %d [%sGHz] flags 0x%x %s%s%s%s%s%s%s%s%s%s(%ddBm): Ad-Hoc %ssupported\n",
channel->hw_value,
is_5ghz ? "5.2" : "2.4",
+ ch_flags,
CHECK_AND_PRINT_I(VALID),
CHECK_AND_PRINT_I(IBSS),
CHECK_AND_PRINT_I(ACTIVE),
CHECK_AND_PRINT_I(RADAR),
- CHECK_AND_PRINT_I(WIDE),
CHECK_AND_PRINT_I(INDOOR_ONLY),
CHECK_AND_PRINT_I(GO_CONCURRENT),
- ch_flags,
+ CHECK_AND_PRINT_I(WIDE),
+ CHECK_AND_PRINT_I(40MHZ),
+ CHECK_AND_PRINT_I(80MHZ),
+ CHECK_AND_PRINT_I(160MHZ),
channel->max_power,
((ch_flags & NVM_CHANNEL_IBSS) &&
!(ch_flags & NVM_CHANNEL_RADAR))
@@ -370,6 +376,10 @@
max_ampdu_exponent <<
IEEE80211_VHT_CAP_MAX_A_MPDU_LENGTH_EXPONENT_SHIFT;
+ if (data->vht160_supported)
+ vht_cap->cap |= IEEE80211_VHT_CAP_SUPP_CHAN_WIDTH_160MHZ |
+ IEEE80211_VHT_CAP_SHORT_GI_160;
+
if (cfg->vht_mu_mimo_supported)
vht_cap->cap |= IEEE80211_VHT_CAP_MU_BEAMFORMEE_CAPABLE;
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-phy-db.c b/drivers/net/wireless/intel/iwlwifi/iwl-phy-db.c
index 4a4dea0..7beba9a 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-phy-db.c
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-phy-db.c
@@ -6,6 +6,7 @@
* GPL LICENSE SUMMARY
*
* Copyright(c) 2007 - 2014 Intel Corporation. All rights reserved.
+ * Copyright(c) 2016 Intel Deutschland GmbH
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
@@ -72,8 +73,6 @@
#include "iwl-trans.h"
#define CHANNEL_NUM_SIZE 4 /* num of channels in calib_ch size */
-#define IWL_NUM_PAPD_CH_GROUPS 9
-#define IWL_NUM_TXP_CH_GROUPS 9
struct iwl_phy_db_entry {
u16 size;
@@ -86,14 +85,18 @@
* @cfg: phy configuration.
* @calib_nch: non channel specific calibration data.
* @calib_ch: channel specific calibration data.
+ * @n_group_papd: number of entries in papd channel group.
* @calib_ch_group_papd: calibration data related to papd channel group.
+ * @n_group_txp: number of entries in tx power channel group.
* @calib_ch_group_txp: calibration data related to tx power chanel group.
*/
struct iwl_phy_db {
struct iwl_phy_db_entry cfg;
struct iwl_phy_db_entry calib_nch;
- struct iwl_phy_db_entry calib_ch_group_papd[IWL_NUM_PAPD_CH_GROUPS];
- struct iwl_phy_db_entry calib_ch_group_txp[IWL_NUM_TXP_CH_GROUPS];
+ int n_group_papd;
+ struct iwl_phy_db_entry *calib_ch_group_papd;
+ int n_group_txp;
+ struct iwl_phy_db_entry *calib_ch_group_txp;
struct iwl_trans *trans;
};
@@ -143,6 +146,9 @@
phy_db->trans = trans;
+ phy_db->n_group_txp = -1;
+ phy_db->n_group_papd = -1;
+
/* TODO: add default values of the phy db. */
return phy_db;
}
@@ -166,11 +172,11 @@
case IWL_PHY_DB_CALIB_NCH:
return &phy_db->calib_nch;
case IWL_PHY_DB_CALIB_CHG_PAPD:
- if (chg_id >= IWL_NUM_PAPD_CH_GROUPS)
+ if (chg_id >= phy_db->n_group_papd)
return NULL;
return &phy_db->calib_ch_group_papd[chg_id];
case IWL_PHY_DB_CALIB_CHG_TXP:
- if (chg_id >= IWL_NUM_TXP_CH_GROUPS)
+ if (chg_id >= phy_db->n_group_txp)
return NULL;
return &phy_db->calib_ch_group_txp[chg_id];
default:
@@ -202,17 +208,21 @@
iwl_phy_db_free_section(phy_db, IWL_PHY_DB_CFG, 0);
iwl_phy_db_free_section(phy_db, IWL_PHY_DB_CALIB_NCH, 0);
- for (i = 0; i < IWL_NUM_PAPD_CH_GROUPS; i++)
+
+ for (i = 0; i < phy_db->n_group_papd; i++)
iwl_phy_db_free_section(phy_db, IWL_PHY_DB_CALIB_CHG_PAPD, i);
- for (i = 0; i < IWL_NUM_TXP_CH_GROUPS; i++)
+ kfree(phy_db->calib_ch_group_papd);
+
+ for (i = 0; i < phy_db->n_group_txp; i++)
iwl_phy_db_free_section(phy_db, IWL_PHY_DB_CALIB_CHG_TXP, i);
+ kfree(phy_db->calib_ch_group_txp);
kfree(phy_db);
}
IWL_EXPORT_SYMBOL(iwl_phy_db_free);
-int iwl_phy_db_set_section(struct iwl_phy_db *phy_db, struct iwl_rx_packet *pkt,
- gfp_t alloc_ctx)
+int iwl_phy_db_set_section(struct iwl_phy_db *phy_db,
+ struct iwl_rx_packet *pkt)
{
struct iwl_calib_res_notif_phy_db *phy_db_notif =
(struct iwl_calib_res_notif_phy_db *)pkt->data;
@@ -224,16 +234,42 @@
if (!phy_db)
return -EINVAL;
- if (type == IWL_PHY_DB_CALIB_CHG_PAPD ||
- type == IWL_PHY_DB_CALIB_CHG_TXP)
+ if (type == IWL_PHY_DB_CALIB_CHG_PAPD) {
chg_id = le16_to_cpup((__le16 *)phy_db_notif->data);
+ if (phy_db && !phy_db->calib_ch_group_papd) {
+ /*
+ * Firmware sends the largest index first, so we can use
+ * it to know how much we should allocate.
+ */
+ phy_db->calib_ch_group_papd = kcalloc(chg_id + 1,
+ sizeof(struct iwl_phy_db_entry),
+ GFP_ATOMIC);
+ if (!phy_db->calib_ch_group_papd)
+ return -ENOMEM;
+ phy_db->n_group_papd = chg_id + 1;
+ }
+ } else if (type == IWL_PHY_DB_CALIB_CHG_TXP) {
+ chg_id = le16_to_cpup((__le16 *)phy_db_notif->data);
+ if (phy_db && !phy_db->calib_ch_group_txp) {
+ /*
+ * Firmware sends the largest index first, so we can use
+ * it to know how much we should allocate.
+ */
+ phy_db->calib_ch_group_txp = kcalloc(chg_id + 1,
+ sizeof(struct iwl_phy_db_entry),
+ GFP_ATOMIC);
+ if (!phy_db->calib_ch_group_txp)
+ return -ENOMEM;
+ phy_db->n_group_txp = chg_id + 1;
+ }
+ }
entry = iwl_phy_db_get_section(phy_db, type, chg_id);
if (!entry)
return -EINVAL;
kfree(entry->data);
- entry->data = kmemdup(phy_db_notif->data, size, alloc_ctx);
+ entry->data = kmemdup(phy_db_notif->data, size, GFP_ATOMIC);
if (!entry->data) {
entry->size = 0;
return -ENOMEM;
@@ -296,7 +332,7 @@
if (ch_index == 0xff)
return 0xff;
- for (i = 0; i < IWL_NUM_TXP_CH_GROUPS; i++) {
+ for (i = 0; i < phy_db->n_group_txp; i++) {
txp_chg = (void *)phy_db->calib_ch_group_txp[i].data;
if (!txp_chg)
return 0xff;
@@ -447,7 +483,7 @@
/* Send all the TXP channel specific data */
err = iwl_phy_db_send_all_channel_groups(phy_db,
IWL_PHY_DB_CALIB_CHG_PAPD,
- IWL_NUM_PAPD_CH_GROUPS);
+ phy_db->n_group_papd);
if (err) {
IWL_ERR(phy_db->trans,
"Cannot send channel specific PAPD groups\n");
@@ -457,7 +493,7 @@
/* Send all the TXP channel specific data */
err = iwl_phy_db_send_all_channel_groups(phy_db,
IWL_PHY_DB_CALIB_CHG_TXP,
- IWL_NUM_TXP_CH_GROUPS);
+ phy_db->n_group_txp);
if (err) {
IWL_ERR(phy_db->trans,
"Cannot send channel specific TX power groups\n");
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-phy-db.h b/drivers/net/wireless/intel/iwlwifi/iwl-phy-db.h
index 2410387..d34de3f 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-phy-db.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-phy-db.h
@@ -73,8 +73,8 @@
void iwl_phy_db_free(struct iwl_phy_db *phy_db);
-int iwl_phy_db_set_section(struct iwl_phy_db *phy_db, struct iwl_rx_packet *pkt,
- gfp_t alloc_ctx);
+int iwl_phy_db_set_section(struct iwl_phy_db *phy_db,
+ struct iwl_rx_packet *pkt);
int iwl_send_phy_db_data(struct iwl_phy_db *phy_db);
diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-trans.h b/drivers/net/wireless/intel/iwlwifi/iwl-trans.h
index fa4ab4b..8193d36 100644
--- a/drivers/net/wireless/intel/iwlwifi/iwl-trans.h
+++ b/drivers/net/wireless/intel/iwlwifi/iwl-trans.h
@@ -753,6 +753,7 @@
* @dev - pointer to struct device * that represents the device
* @max_skb_frags: maximum number of fragments an SKB can have when transmitted.
* 0 indicates that frag SKBs (NETIF_F_SG) aren't supported.
+ * @hw_rf_id a u32 with the device RF ID
* @hw_id: a u32 with the ID of the device / sub-device.
* Set during transport allocation.
* @hw_id_str: a string with info about HW ID. Set during transport allocation.
@@ -797,6 +798,7 @@
struct device *dev;
u32 max_skb_frags;
u32 hw_rev;
+ u32 hw_rf_id;
u32 hw_id;
char hw_id_str[52];
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/constants.h b/drivers/net/wireless/intel/iwlwifi/mvm/constants.h
index b96b1c6..4eeb6b7 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/constants.h
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/constants.h
@@ -109,6 +109,7 @@
#define IWL_MVM_RS_80_20_FAR_RANGE_TWEAK 1
#define IWL_MVM_TOF_IS_RESPONDER 0
#define IWL_MVM_SW_TX_CSUM_OFFLOAD 0
+#define IWL_MVM_HW_CSUM_DISABLE 0
#define IWL_MVM_COLLECT_FW_ERR_DUMP 1
#define IWL_MVM_RS_NUM_TRY_BEFORE_ANT_TOGGLE 1
#define IWL_MVM_RS_HT_VHT_RETRIES_PER_RATE 2
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/d3.c b/drivers/net/wireless/intel/iwlwifi/mvm/d3.c
index e3561bb..4fdc3da 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/d3.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/d3.c
@@ -1804,7 +1804,6 @@
struct iwl_wowlan_status *fw_status;
int i;
bool keep;
- struct ieee80211_sta *ap_sta;
struct iwl_mvm_sta *mvm_ap_sta;
fw_status = iwl_mvm_get_wakeup_status(mvm, vif);
@@ -1823,13 +1822,10 @@
status.wake_packet = fw_status->wake_packet;
/* still at hard-coded place 0 for D3 image */
- ap_sta = rcu_dereference_protected(
- mvm->fw_id_to_mac_id[0],
- lockdep_is_held(&mvm->mutex));
- if (IS_ERR_OR_NULL(ap_sta))
+ mvm_ap_sta = iwl_mvm_sta_from_staid_protected(mvm, 0);
+ if (!mvm_ap_sta)
goto out_free;
- mvm_ap_sta = iwl_mvm_sta_from_mac80211(ap_sta);
for (i = 0; i < IWL_MAX_TID_COUNT; i++) {
u16 seq = status.qos_seq_ctr[i];
/* firmware stores last-used value, we store next value */
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/debugfs-vif.c b/drivers/net/wireless/intel/iwlwifi/mvm/debugfs-vif.c
index fb96bc0..b232717 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/debugfs-vif.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/debugfs-vif.c
@@ -281,13 +281,10 @@
if (vif->type == NL80211_IFTYPE_STATION &&
ap_sta_id != IWL_MVM_STATION_COUNT) {
- struct ieee80211_sta *sta;
+ struct iwl_mvm_sta *mvm_sta;
- sta = rcu_dereference_protected(mvm->fw_id_to_mac_id[ap_sta_id],
- lockdep_is_held(&mvm->mutex));
- if (!IS_ERR_OR_NULL(sta)) {
- struct iwl_mvm_sta *mvm_sta = iwl_mvm_sta_from_mac80211(sta);
-
+ mvm_sta = iwl_mvm_sta_from_staid_protected(mvm, ap_sta_id);
+ if (mvm_sta) {
pos += scnprintf(buf+pos, bufsz-pos,
"ap_sta_id %d - reduced Tx power %d\n",
ap_sta_id,
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c b/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c
index 362a546..406cf1c 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c
@@ -1309,6 +1309,8 @@
PRINT_MVM_REF(IWL_MVM_REF_PROTECT_CSA);
PRINT_MVM_REF(IWL_MVM_REF_FW_DBG_COLLECT);
PRINT_MVM_REF(IWL_MVM_REF_INIT_UCODE);
+ PRINT_MVM_REF(IWL_MVM_REF_SENDING_CMD);
+ PRINT_MVM_REF(IWL_MVM_REF_RX);
return simple_read_from_buffer(user_buf, count, ppos, buf, pos);
}
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/fw-api-d3.h b/drivers/net/wireless/intel/iwlwifi/mvm/fw-api-d3.h
index eec52c5..5f22cc7 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/fw-api-d3.h
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/fw-api-d3.h
@@ -368,7 +368,7 @@
u8 decrypt_key[16];
u8 tkip_mic_key[8];
struct iwl_wowlan_rsc_tsc_params_cmd rsc;
-} __packed;
+} __packed; /* WOWLAN_GTK_MATERIAL_VER_1 */
struct iwl_wowlan_status {
struct iwl_wowlan_gtk_status gtk;
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/fw-api-rx.h b/drivers/net/wireless/intel/iwlwifi/mvm/fw-api-rx.h
index 4c086d0..1ca8e49 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/fw-api-rx.h
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/fw-api-rx.h
@@ -437,21 +437,28 @@
/**
* Internal message identifier
*
+* @IWL_MVM_RXQ_EMPTY: empty sync notification
* @IWL_MVM_RXQ_NOTIF_DEL_BA: notify RSS queues of delBA
*/
enum iwl_mvm_rxq_notif_type {
+ IWL_MVM_RXQ_EMPTY,
IWL_MVM_RXQ_NOTIF_DEL_BA,
};
/**
* struct iwl_mvm_internal_rxq_notif - Internal representation of the data sent
* in &iwl_rxq_sync_cmd. Should be DWORD aligned.
+* FW is agnostic to the payload, so there are no endianity requirements.
*
* @type: value from &iwl_mvm_rxq_notif_type
+* @sync: ctrl path is waiting for all notifications to be received
+* @cookie: internal cookie to identify old notifications
* @data: payload
*/
struct iwl_mvm_internal_rxq_notif {
- u32 type;
+ u16 type;
+ u16 sync;
+ u32 cookie;
u8 data[];
} __packed;
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/fw-api-sta.h b/drivers/net/wireless/intel/iwlwifi/mvm/fw-api-sta.h
index 90d9113..38b1d04 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/fw-api-sta.h
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/fw-api-sta.h
@@ -173,7 +173,7 @@
/**
* enum iwl_sta_modify_flag - indicate to the fw what flag are being changed
- * @STA_MODIFY_KEY: this command modifies %key
+ * @STA_MODIFY_QUEUE_REMOVAL: this command removes a queue
* @STA_MODIFY_TID_DISABLE_TX: this command modifies %tid_disable_tx
* @STA_MODIFY_TX_RATE: unused
* @STA_MODIFY_ADD_BA_TID: this command modifies %add_immediate_ba_tid
@@ -183,7 +183,7 @@
* @STA_MODIFY_QUEUES: modify the queues used by this station
*/
enum iwl_sta_modify_flag {
- STA_MODIFY_KEY = BIT(0),
+ STA_MODIFY_QUEUE_REMOVAL = BIT(0),
STA_MODIFY_TID_DISABLE_TX = BIT(1),
STA_MODIFY_TX_RATE = BIT(2),
STA_MODIFY_ADD_BA_TID = BIT(3),
@@ -255,8 +255,10 @@
__le64 hw_tkip_mic_tx_key;
} __packed;
-#define IWL_ADD_STA_STATUS_MASK 0xFF
-#define IWL_ADD_STA_BAID_MASK 0xFF00
+#define IWL_ADD_STA_STATUS_MASK 0xFF
+#define IWL_ADD_STA_BAID_VALID_MASK 0x8000
+#define IWL_ADD_STA_BAID_MASK 0x7F00
+#define IWL_ADD_STA_BAID_SHIFT 8
/**
* struct iwl_mvm_add_sta_cmd_v7 - Add/modify a station in the fw's sta table.
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/fw-api.h b/drivers/net/wireless/intel/iwlwifi/mvm/fw-api.h
index 60eed84..41b80ae 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/fw-api.h
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/fw-api.h
@@ -90,6 +90,7 @@
* DQA queue numbers
*
* @IWL_MVM_DQA_CMD_QUEUE: a queue reserved for sending HCMDs to the FW
+ * @IWL_MVM_DQA_P2P_DEVICE_QUEUE: a queue reserved for P2P device frames
* @IWL_MVM_DQA_GCAST_QUEUE: a queue reserved for P2P GO/SoftAP GCAST frames
* @IWL_MVM_DQA_BSS_CLIENT_QUEUE: a queue reserved for BSS activity, to ensure
* that we are never left without the possibility to connect to an AP.
@@ -97,6 +98,8 @@
* Each MGMT queue is mapped to a single STA
* MGMT frames are frames that return true on ieee80211_is_mgmt()
* @IWL_MVM_DQA_MAX_MGMT_QUEUE: last TXQ in pool for MGMT frames
+ * @IWL_MVM_DQA_AP_PROBE_RESP_QUEUE: a queue reserved for P2P GO/SoftAP probe
+ * responses
* @IWL_MVM_DQA_MIN_DATA_QUEUE: first TXQ in pool for DATA frames.
* DATA frames are intended for !ieee80211_is_mgmt() frames, but if
* the MGMT TXQ pool is exhausted, mgmt frames can be sent on DATA queues
@@ -105,10 +108,12 @@
*/
enum iwl_mvm_dqa_txq {
IWL_MVM_DQA_CMD_QUEUE = 0,
+ IWL_MVM_DQA_P2P_DEVICE_QUEUE = 2,
IWL_MVM_DQA_GCAST_QUEUE = 3,
IWL_MVM_DQA_BSS_CLIENT_QUEUE = 4,
IWL_MVM_DQA_MIN_MGMT_QUEUE = 5,
IWL_MVM_DQA_MAX_MGMT_QUEUE = 8,
+ IWL_MVM_DQA_AP_PROBE_RESP_QUEUE = 9,
IWL_MVM_DQA_MIN_DATA_QUEUE = 10,
IWL_MVM_DQA_MAX_DATA_QUEUE = 31,
};
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/fw-dbg.c b/drivers/net/wireless/intel/iwlwifi/mvm/fw-dbg.c
index e25171f..e1b6b2c 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/fw-dbg.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/fw-dbg.c
@@ -71,7 +71,7 @@
#include "iwl-csr.h"
static ssize_t iwl_mvm_read_coredump(char *buffer, loff_t offset, size_t count,
- const void *data, size_t datalen)
+ void *data, size_t datalen)
{
const struct iwl_mvm_dump_ptrs *dump_ptrs = data;
ssize_t bytes_read;
@@ -104,7 +104,7 @@
return bytes_read + bytes_read_trans;
}
-static void iwl_mvm_free_coredump(const void *data)
+static void iwl_mvm_free_coredump(void *data)
{
const struct iwl_mvm_dump_ptrs *fw_error_dump = data;
@@ -271,9 +271,6 @@
for (i = 0;
i < ARRAY_SIZE(mvm->shared_mem_cfg.internal_txfifo_size);
i++) {
- /* Mark the number of TXF we're pulling now */
- iwl_trans_write_prph(mvm->trans, TXF_CPU2_NUM, i);
-
fifo_hdr = (void *)(*dump_data)->data;
fifo_data = (void *)fifo_hdr->data;
fifo_len = mvm->shared_mem_cfg.internal_txfifo_size[i];
@@ -289,6 +286,10 @@
cpu_to_le32(fifo_len + sizeof(*fifo_hdr));
fifo_hdr->fifo_num = cpu_to_le32(i);
+
+ /* Mark the number of TXF we're pulling now */
+ iwl_trans_write_prph(mvm->trans, TXF_CPU2_NUM, i);
+
fifo_hdr->available_bytes =
cpu_to_le32(iwl_trans_read_prph(mvm->trans,
TXF_CPU2_FIFO_ITEM_CNT));
@@ -339,9 +340,11 @@
#define IWL8260_ICCM_OFFSET 0x44000 /* Only for B-step */
#define IWL8260_ICCM_LEN 0xC000 /* Only for B-step */
-static const struct {
+struct iwl_prph_range {
u32 start, end;
-} iwl_prph_dump_addr[] = {
+};
+
+static const struct iwl_prph_range iwl_prph_dump_addr_comm[] = {
{ .start = 0x00a00000, .end = 0x00a00000 },
{ .start = 0x00a0000c, .end = 0x00a00024 },
{ .start = 0x00a0002c, .end = 0x00a0003c },
@@ -439,8 +442,18 @@
{ .start = 0x00a44000, .end = 0x00a7bf80 },
};
+static const struct iwl_prph_range iwl_prph_dump_addr_9000[] = {
+ { .start = 0x00a05c00, .end = 0x00a05c18 },
+ { .start = 0x00a05400, .end = 0x00a056e8 },
+ { .start = 0x00a08000, .end = 0x00a098bc },
+ { .start = 0x00adfc00, .end = 0x00adfd1c },
+ { .start = 0x00a02400, .end = 0x00a02758 },
+};
+
static u32 iwl_dump_prph(struct iwl_trans *trans,
- struct iwl_fw_error_dump_data **data)
+ struct iwl_fw_error_dump_data **data,
+ const struct iwl_prph_range *iwl_prph_dump_addr,
+ u32 range_len)
{
struct iwl_fw_error_dump_prph *prph;
unsigned long flags;
@@ -449,7 +462,7 @@
if (!iwl_trans_grab_nic_access(trans, &flags))
return 0;
- for (i = 0; i < ARRAY_SIZE(iwl_prph_dump_addr); i++) {
+ for (i = 0; i < range_len; i++) {
/* The range includes both boundaries */
int num_bytes_in_chunk = iwl_prph_dump_addr[i].end -
iwl_prph_dump_addr[i].start + 4;
@@ -572,16 +585,31 @@
}
/* Make room for PRPH registers */
- for (i = 0; i < ARRAY_SIZE(iwl_prph_dump_addr); i++) {
+ for (i = 0; i < ARRAY_SIZE(iwl_prph_dump_addr_comm); i++) {
/* The range includes both boundaries */
- int num_bytes_in_chunk = iwl_prph_dump_addr[i].end -
- iwl_prph_dump_addr[i].start + 4;
+ int num_bytes_in_chunk =
+ iwl_prph_dump_addr_comm[i].end -
+ iwl_prph_dump_addr_comm[i].start + 4;
prph_len += sizeof(*dump_data) +
sizeof(struct iwl_fw_error_dump_prph) +
num_bytes_in_chunk;
}
+ if (mvm->cfg->mq_rx_supported) {
+ for (i = 0; i <
+ ARRAY_SIZE(iwl_prph_dump_addr_9000); i++) {
+ /* The range includes both boundaries */
+ int num_bytes_in_chunk =
+ iwl_prph_dump_addr_9000[i].end -
+ iwl_prph_dump_addr_9000[i].start + 4;
+
+ prph_len += sizeof(*dump_data) +
+ sizeof(struct iwl_fw_error_dump_prph) +
+ num_bytes_in_chunk;
+ }
+ }
+
if (mvm->cfg->device_family == IWL_DEVICE_FAMILY_7000)
radio_len = sizeof(*dump_data) + RADIO_REG_MAX_READ;
}
@@ -769,8 +797,16 @@
}
}
- if (prph_len)
- iwl_dump_prph(mvm->trans, &dump_data);
+ if (prph_len) {
+ iwl_dump_prph(mvm->trans, &dump_data,
+ iwl_prph_dump_addr_comm,
+ ARRAY_SIZE(iwl_prph_dump_addr_comm));
+
+ if (mvm->cfg->mq_rx_supported)
+ iwl_dump_prph(mvm->trans, &dump_data,
+ iwl_prph_dump_addr_9000,
+ ARRAY_SIZE(iwl_prph_dump_addr_9000));
+ }
dump_trans_data:
fw_error_dump->trans_ptr = iwl_trans_dump_data(mvm->trans,
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/fw.c b/drivers/net/wireless/intel/iwlwifi/mvm/fw.c
index b70f453..7057f35 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/fw.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/fw.c
@@ -535,7 +535,7 @@
return true;
}
- WARN_ON(iwl_phy_db_set_section(phy_db, pkt, GFP_ATOMIC));
+ WARN_ON(iwl_phy_db_set_section(phy_db, pkt));
return false;
}
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mac-ctxt.c b/drivers/net/wireless/intel/iwlwifi/mvm/mac-ctxt.c
index 456067b..7aae068 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/mac-ctxt.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/mac-ctxt.c
@@ -501,9 +501,11 @@
switch (vif->type) {
case NL80211_IFTYPE_P2P_DEVICE:
- iwl_mvm_enable_ac_txq(mvm, IWL_MVM_OFFCHANNEL_QUEUE,
- IWL_MVM_OFFCHANNEL_QUEUE,
- IWL_MVM_TX_FIFO_VO, 0, wdg_timeout);
+ if (!iwl_mvm_is_dqa_supported(mvm))
+ iwl_mvm_enable_ac_txq(mvm, IWL_MVM_OFFCHANNEL_QUEUE,
+ IWL_MVM_OFFCHANNEL_QUEUE,
+ IWL_MVM_TX_FIFO_VO, 0,
+ wdg_timeout);
break;
case NL80211_IFTYPE_AP:
iwl_mvm_enable_ac_txq(mvm, vif->cab_queue, vif->cab_queue,
@@ -533,13 +535,21 @@
switch (vif->type) {
case NL80211_IFTYPE_P2P_DEVICE:
- iwl_mvm_disable_txq(mvm, IWL_MVM_OFFCHANNEL_QUEUE,
- IWL_MVM_OFFCHANNEL_QUEUE, IWL_MAX_TID_COUNT,
- 0);
+ if (!iwl_mvm_is_dqa_supported(mvm))
+ iwl_mvm_disable_txq(mvm, IWL_MVM_OFFCHANNEL_QUEUE,
+ IWL_MVM_OFFCHANNEL_QUEUE,
+ IWL_MAX_TID_COUNT, 0);
+
break;
case NL80211_IFTYPE_AP:
iwl_mvm_disable_txq(mvm, vif->cab_queue, vif->cab_queue,
IWL_MAX_TID_COUNT, 0);
+
+ if (iwl_mvm_is_dqa_supported(mvm))
+ iwl_mvm_disable_txq(mvm,
+ IWL_MVM_DQA_AP_PROBE_RESP_QUEUE,
+ vif->hw_queue[0], IWL_MAX_TID_COUNT,
+ 0);
/* fall through */
default:
/*
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c b/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c
index 5ace468..e5f267b 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c
@@ -229,7 +229,11 @@
IWL_DEBUG_RPM(mvm, "Leave mvm reference - type %d\n", ref_type);
spin_lock_bh(&mvm->refs_lock);
- WARN_ON(!mvm->refs[ref_type]--);
+ if (WARN_ON(!mvm->refs[ref_type])) {
+ spin_unlock_bh(&mvm->refs_lock);
+ return;
+ }
+ mvm->refs[ref_type]--;
spin_unlock_bh(&mvm->refs_lock);
iwl_trans_unref(mvm->trans);
}
@@ -439,11 +443,19 @@
ieee80211_hw_set(hw, SUPPORTS_CLONED_SKBS);
ieee80211_hw_set(hw, SUPPORTS_AMSDU_IN_AMPDU);
ieee80211_hw_set(hw, NEEDS_UNIQUE_STA_ADDR);
+ if (iwl_mvm_has_new_rx_api(mvm))
+ ieee80211_hw_set(hw, SUPPORTS_REORDERING_BUFFER);
+
+ if (mvm->trans->num_rx_queues > 1)
+ ieee80211_hw_set(hw, USES_RSS);
if (mvm->trans->max_skb_frags)
hw->netdev_features = NETIF_F_HIGHDMA | NETIF_F_SG;
- hw->queues = mvm->first_agg_queue;
+ if (!iwl_mvm_is_dqa_supported(mvm))
+ hw->queues = mvm->first_agg_queue;
+ else
+ hw->queues = IEEE80211_MAX_QUEUES;
hw->offchannel_tx_hw_queue = IWL_MVM_OFFCHANNEL_QUEUE;
hw->radiotap_mcs_details |= IEEE80211_RADIOTAP_MCS_HAVE_FEC |
IEEE80211_RADIOTAP_MCS_HAVE_STBC;
@@ -848,6 +860,7 @@
u16 *ssn = ¶ms->ssn;
u8 buf_size = params->buf_size;
bool amsdu = params->amsdu;
+ u16 timeout = params->timeout;
IWL_DEBUG_HT(mvm, "A-MPDU action on addr %pM tid %d: action %d\n",
sta->addr, tid, action);
@@ -888,10 +901,12 @@
ret = -EINVAL;
break;
}
- ret = iwl_mvm_sta_rx_agg(mvm, sta, tid, *ssn, true, buf_size);
+ ret = iwl_mvm_sta_rx_agg(mvm, sta, tid, *ssn, true, buf_size,
+ timeout);
break;
case IEEE80211_AMPDU_RX_STOP:
- ret = iwl_mvm_sta_rx_agg(mvm, sta, tid, 0, false, buf_size);
+ ret = iwl_mvm_sta_rx_agg(mvm, sta, tid, 0, false, buf_size,
+ timeout);
break;
case IEEE80211_AMPDU_TX_START:
if (!iwl_enable_tx_ampdu(mvm->cfg)) {
@@ -4037,6 +4052,55 @@
}
}
+void iwl_mvm_sync_rx_queues_internal(struct iwl_mvm *mvm,
+ struct iwl_mvm_internal_rxq_notif *notif,
+ u32 size)
+{
+ DECLARE_WAIT_QUEUE_HEAD_ONSTACK(notif_waitq);
+ u32 qmask = BIT(mvm->trans->num_rx_queues) - 1;
+ int ret;
+
+ lockdep_assert_held(&mvm->mutex);
+
+ if (!iwl_mvm_has_new_rx_api(mvm))
+ return;
+
+ notif->cookie = mvm->queue_sync_cookie;
+
+ if (notif->sync)
+ atomic_set(&mvm->queue_sync_counter,
+ mvm->trans->num_rx_queues);
+
+ ret = iwl_mvm_notify_rx_queue(mvm, qmask, (u8 *)notif, size);
+ if (ret) {
+ IWL_ERR(mvm, "Failed to trigger RX queues sync (%d)\n", ret);
+ goto out;
+ }
+
+ if (notif->sync)
+ ret = wait_event_timeout(notif_waitq,
+ atomic_read(&mvm->queue_sync_counter) == 0,
+ HZ);
+ WARN_ON_ONCE(!ret);
+
+out:
+ atomic_set(&mvm->queue_sync_counter, 0);
+ mvm->queue_sync_cookie++;
+}
+
+static void iwl_mvm_sync_rx_queues(struct ieee80211_hw *hw)
+{
+ struct iwl_mvm *mvm = IWL_MAC80211_GET_MVM(hw);
+ struct iwl_mvm_internal_rxq_notif data = {
+ .type = IWL_MVM_RXQ_EMPTY,
+ .sync = 1,
+ };
+
+ mutex_lock(&mvm->mutex);
+ iwl_mvm_sync_rx_queues_internal(mvm, &data, sizeof(data));
+ mutex_unlock(&mvm->mutex);
+}
+
const struct ieee80211_ops iwl_mvm_hw_ops = {
.tx = iwl_mvm_mac_tx,
.ampdu_action = iwl_mvm_mac_ampdu_action,
@@ -4093,6 +4157,8 @@
.event_callback = iwl_mvm_mac_event_callback,
+ .sync_rx_queues = iwl_mvm_sync_rx_queues,
+
CFG80211_TESTMODE_CMD(iwl_mvm_mac_testmode_cmd)
#ifdef CONFIG_PM_SLEEP
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h b/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h
index 85800ba..ffbd41d 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/mvm.h
@@ -301,6 +301,8 @@
IWL_MVM_REF_PROTECT_CSA,
IWL_MVM_REF_FW_DBG_COLLECT,
IWL_MVM_REF_INIT_UCODE,
+ IWL_MVM_REF_SENDING_CMD,
+ IWL_MVM_REF_RX,
/* update debugfs.c when changing this */
@@ -613,6 +615,84 @@
u32 internal_txfifo_size[TX_FIFO_INTERNAL_MAX_NUM];
};
+/**
+ * struct iwl_mvm_reorder_buffer - per ra/tid/queue reorder buffer
+ * @head_sn: reorder window head sn
+ * @num_stored: number of mpdus stored in the buffer
+ * @buf_size: the reorder buffer size as set by the last addba request
+ * @sta_id: sta id of this reorder buffer
+ * @queue: queue of this reorder buffer
+ * @last_amsdu: track last ASMDU SN for duplication detection
+ * @last_sub_index: track ASMDU sub frame index for duplication detection
+ * @entries: list of skbs stored
+ * @reorder_time: time the packet was stored in the reorder buffer
+ * @reorder_timer: timer for frames are in the reorder buffer. For AMSDU
+ * it is the time of last received sub-frame
+ * @removed: prevent timer re-arming
+ * @lock: protect reorder buffer internal state
+ * @mvm: mvm pointer, needed for frame timer context
+ */
+struct iwl_mvm_reorder_buffer {
+ u16 head_sn;
+ u16 num_stored;
+ u8 buf_size;
+ u8 sta_id;
+ int queue;
+ u16 last_amsdu;
+ u8 last_sub_index;
+ struct sk_buff_head entries[IEEE80211_MAX_AMPDU_BUF];
+ unsigned long reorder_time[IEEE80211_MAX_AMPDU_BUF];
+ struct timer_list reorder_timer;
+ bool removed;
+ spinlock_t lock;
+ struct iwl_mvm *mvm;
+} ____cacheline_aligned_in_smp;
+
+/**
+ * struct iwl_mvm_baid_data - BA session data
+ * @sta_id: station id
+ * @tid: tid of the session
+ * @baid baid of the session
+ * @timeout: the timeout set in the addba request
+ * @last_rx: last rx jiffies, updated only if timeout passed from last update
+ * @session_timer: timer to check if BA session expired, runs at 2 * timeout
+ * @mvm: mvm pointer, needed for timer context
+ * @reorder_buf: reorder buffer, allocated per queue
+ */
+struct iwl_mvm_baid_data {
+ struct rcu_head rcu_head;
+ u8 sta_id;
+ u8 tid;
+ u8 baid;
+ u16 timeout;
+ unsigned long last_rx;
+ struct timer_list session_timer;
+ struct iwl_mvm *mvm;
+ struct iwl_mvm_reorder_buffer reorder_buf[];
+};
+
+/*
+ * enum iwl_mvm_queue_status - queue status
+ * @IWL_MVM_QUEUE_FREE: the queue is not allocated nor reserved
+ * Basically, this means that this queue can be used for any purpose
+ * @IWL_MVM_QUEUE_RESERVED: queue is reserved but not yet in use
+ * This is the state of a queue that has been dedicated for some RATID
+ * (agg'd or not), but that hasn't yet gone through the actual enablement
+ * of iwl_mvm_enable_txq(), and therefore no traffic can go through it yet.
+ * Note that in this state there is no requirement to already know what TID
+ * should be used with this queue, it is just marked as a queue that will
+ * be used, and shouldn't be allocated to anyone else.
+ * @IWL_MVM_QUEUE_READY: queue is ready to be used
+ * This is the state of a queue that has been fully configured (including
+ * SCD pointers, etc), has a specific RA/TID assigned to it, and can be
+ * used to send traffic.
+ */
+enum iwl_mvm_queue_status {
+ IWL_MVM_QUEUE_FREE,
+ IWL_MVM_QUEUE_RESERVED,
+ IWL_MVM_QUEUE_READY,
+};
+
struct iwl_mvm {
/* for logger access */
struct device *dev;
@@ -633,6 +713,8 @@
unsigned long status;
+ u32 queue_sync_cookie;
+ atomic_t queue_sync_counter;
/*
* for beacon filtering -
* currently only one interface can be supported
@@ -666,13 +748,8 @@
u32 hw_queue_to_mac80211;
u8 hw_queue_refcount;
u8 ra_sta_id; /* The RA this queue is mapped to, if exists */
- /*
- * This is to mark that queue is reserved for a STA but not yet
- * allocated. This is needed to make sure we have at least one
- * available queue to use when adding a new STA
- */
- bool setup_reserved;
u16 tid_bitmap; /* Bitmap of the TIDs mapped to this queue */
+ enum iwl_mvm_queue_status status;
} queue_info[IWL_MAX_HW_QUEUES];
spinlock_t queue_info_lock; /* For syncing queue mgmt operations */
struct work_struct add_stream_wk; /* To add streams to queues */
@@ -920,6 +997,10 @@
u32 ciphers[6];
struct iwl_mvm_tof_data tof_data;
+ struct ieee80211_vif *nan_vif;
+#define IWL_MAX_BAID 32
+ struct iwl_mvm_baid_data __rcu *baid_map[IWL_MAX_BAID];
+
/*
* Drop beacons from other APs in AP mode when there are no connected
* clients.
@@ -1065,7 +1146,8 @@
static inline bool iwl_mvm_is_csum_supported(struct iwl_mvm *mvm)
{
return fw_has_capa(&mvm->fw->ucode_capa,
- IWL_UCODE_TLV_CAPA_CSUM_SUPPORT);
+ IWL_UCODE_TLV_CAPA_CSUM_SUPPORT) &&
+ !IWL_MVM_HW_CSUM_DISABLE;
}
static inline bool iwl_mvm_is_mplut_supported(struct iwl_mvm *mvm)
@@ -1242,7 +1324,7 @@
void iwl_mvm_rx_phy_cmd_mq(struct iwl_mvm *mvm, struct iwl_rx_cmd_buffer *rxb);
void iwl_mvm_rx_mpdu_mq(struct iwl_mvm *mvm, struct napi_struct *napi,
struct iwl_rx_cmd_buffer *rxb, int queue);
-void iwl_mvm_rx_frame_release(struct iwl_mvm *mvm,
+void iwl_mvm_rx_frame_release(struct iwl_mvm *mvm, struct napi_struct *napi,
struct iwl_rx_cmd_buffer *rxb, int queue);
int iwl_mvm_notify_rx_queue(struct iwl_mvm *mvm, u32 rxq_mask,
const u8 *data, u32 count);
@@ -1566,6 +1648,10 @@
void iwl_mvm_start_mac_queues(struct iwl_mvm *mvm, unsigned long mq);
void iwl_mvm_stop_mac_queues(struct iwl_mvm *mvm, unsigned long mq);
+/* Re-configure the SCD for a queue that has already been configured */
+int iwl_mvm_reconfig_scd(struct iwl_mvm *mvm, int queue, int fifo, int sta_id,
+ int tid, int frame_limit, u16 ssn);
+
/* Thermal management and CT-kill */
void iwl_mvm_tt_tx_backoff(struct iwl_mvm *mvm, u32 backoff);
void iwl_mvm_tt_temp_changed(struct iwl_mvm *mvm, u32 temp);
@@ -1628,6 +1714,10 @@
void iwl_mvm_rx_tdls_notif(struct iwl_mvm *mvm, struct iwl_rx_cmd_buffer *rxb);
void iwl_mvm_tdls_ch_switch_work(struct work_struct *work);
+void iwl_mvm_sync_rx_queues_internal(struct iwl_mvm *mvm,
+ struct iwl_mvm_internal_rxq_notif *notif,
+ u32 size);
+void iwl_mvm_reorder_timer_expired(unsigned long data);
struct ieee80211_vif *iwl_mvm_get_bss_vif(struct iwl_mvm *mvm);
void iwl_mvm_nic_restart(struct iwl_mvm *mvm, bool fw_error);
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/ops.c b/drivers/net/wireless/intel/iwlwifi/mvm/ops.c
index 8bfb8e0..a68054f 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/ops.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/ops.c
@@ -554,8 +554,13 @@
mvm->restart_fw = iwlwifi_mod_params.restart_fw ? -1 : 0;
mvm->aux_queue = 15;
- mvm->first_agg_queue = 16;
- mvm->last_agg_queue = mvm->cfg->base_params->num_of_queues - 1;
+ if (!iwl_mvm_is_dqa_supported(mvm)) {
+ mvm->first_agg_queue = 16;
+ mvm->last_agg_queue = mvm->cfg->base_params->num_of_queues - 1;
+ } else {
+ mvm->first_agg_queue = IWL_MVM_DQA_MIN_DATA_QUEUE;
+ mvm->last_agg_queue = IWL_MVM_DQA_MAX_DATA_QUEUE;
+ }
if (mvm->cfg->base_params->num_of_queues == 16) {
mvm->aux_queue = 11;
mvm->first_agg_queue = 12;
@@ -586,6 +591,8 @@
skb_queue_head_init(&mvm->d0i3_tx);
init_waitqueue_head(&mvm->d0i3_exit_waitq);
+ atomic_set(&mvm->queue_sync_counter, 0);
+
SET_IEEE80211_DEV(mvm->hw, mvm->trans->dev);
/*
@@ -930,7 +937,7 @@
if (likely(pkt->hdr.cmd == REPLY_RX_MPDU_CMD))
iwl_mvm_rx_rx_mpdu(mvm, napi, rxb);
else if (pkt->hdr.cmd == FRAME_RELEASE)
- iwl_mvm_rx_frame_release(mvm, rxb, 0);
+ iwl_mvm_rx_frame_release(mvm, napi, rxb, 0);
else if (pkt->hdr.cmd == REPLY_RX_PHY_CMD)
iwl_mvm_rx_rx_phy_cmd(mvm, rxb);
else
@@ -1208,7 +1215,6 @@
struct iwl_d0i3_iter_data *iter_data)
{
struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
- struct ieee80211_sta *ap_sta;
struct iwl_mvm_sta *mvmsta;
u32 available_tids = 0;
u8 tid;
@@ -1217,11 +1223,10 @@
mvmvif->ap_sta_id == IWL_MVM_STATION_COUNT))
return false;
- ap_sta = rcu_dereference(mvm->fw_id_to_mac_id[mvmvif->ap_sta_id]);
- if (IS_ERR_OR_NULL(ap_sta))
+ mvmsta = iwl_mvm_sta_from_staid_rcu(mvm, mvmvif->ap_sta_id);
+ if (!mvmsta)
return false;
- mvmsta = iwl_mvm_sta_from_mac80211(ap_sta);
spin_lock_bh(&mvmsta->lock);
for (tid = 0; tid < IWL_MAX_TID_COUNT; tid++) {
struct iwl_mvm_tid_data *tid_data = &mvmsta->tid_data[tid];
@@ -1632,7 +1637,7 @@
struct iwl_rx_packet *pkt = rxb_addr(rxb);
if (unlikely(pkt->hdr.cmd == FRAME_RELEASE))
- iwl_mvm_rx_frame_release(mvm, rxb, queue);
+ iwl_mvm_rx_frame_release(mvm, napi, rxb, queue);
else if (unlikely(pkt->hdr.cmd == RX_QUEUES_NOTIFICATION &&
pkt->hdr.group_id == DATA_PATH_GROUP))
iwl_mvm_rx_queue_notif(mvm, rxb, queue);
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/rx.c b/drivers/net/wireless/intel/iwlwifi/mvm/rx.c
index 263e8a8..ab7f7ed 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/rx.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/rx.c
@@ -97,6 +97,7 @@
* Adds the rxb to a new skb and give it to mac80211
*/
static void iwl_mvm_pass_packet_to_mac80211(struct iwl_mvm *mvm,
+ struct ieee80211_sta *sta,
struct napi_struct *napi,
struct sk_buff *skb,
struct ieee80211_hdr *hdr, u16 len,
@@ -131,7 +132,7 @@
fraglen, rxb->truesize);
}
- ieee80211_rx_napi(mvm->hw, NULL, skb, napi);
+ ieee80211_rx_napi(mvm->hw, sta, skb, napi);
}
/*
@@ -271,6 +272,7 @@
u32 rate_n_flags;
u32 rx_pkt_status;
u8 crypt_len = 0;
+ bool take_ref;
phy_info = &mvm->last_phy_info;
rx_res = (struct iwl_rx_mpdu_res_start *)pkt->data;
@@ -453,8 +455,26 @@
mvm->sched_scan_pass_all == SCHED_SCAN_PASS_ALL_ENABLED))
mvm->sched_scan_pass_all = SCHED_SCAN_PASS_ALL_FOUND;
- iwl_mvm_pass_packet_to_mac80211(mvm, napi, skb, hdr, len, ampdu_status,
- crypt_len, rxb);
+ if (unlikely(ieee80211_is_beacon(hdr->frame_control) ||
+ ieee80211_is_probe_resp(hdr->frame_control)))
+ rx_status->boottime_ns = ktime_get_boot_ns();
+
+ /* Take a reference briefly to kick off a d0i3 entry delay so
+ * we can handle bursts of RX packets without toggling the
+ * state too often. But don't do this for beacons if we are
+ * going to idle because the beacon filtering changes we make
+ * cause the firmware to send us collateral beacons. */
+ take_ref = !(test_bit(STATUS_TRANS_GOING_IDLE, &mvm->trans->status) &&
+ ieee80211_is_beacon(hdr->frame_control));
+
+ if (take_ref)
+ iwl_mvm_ref(mvm, IWL_MVM_REF_RX);
+
+ iwl_mvm_pass_packet_to_mac80211(mvm, sta, napi, skb, hdr, len,
+ ampdu_status, crypt_len, rxb);
+
+ if (take_ref)
+ iwl_mvm_unref(mvm, IWL_MVM_REF_RX);
}
static void iwl_mvm_update_rx_statistics(struct iwl_mvm *mvm,
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c b/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c
index 651604d..ac2c571 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c
@@ -210,7 +210,7 @@
if (iwl_mvm_check_pn(mvm, skb, queue, sta))
kfree_skb(skb);
else
- ieee80211_rx_napi(mvm->hw, NULL, skb, napi);
+ ieee80211_rx_napi(mvm->hw, sta, skb, napi);
}
static void iwl_mvm_get_signal_strength(struct iwl_mvm *mvm,
@@ -395,6 +395,150 @@
return ret;
}
+/*
+ * Returns true if sn2 - buffer_size < sn1 < sn2.
+ * To be used only in order to compare reorder buffer head with NSSN.
+ * We fully trust NSSN unless it is behind us due to reorder timeout.
+ * Reorder timeout can only bring us up to buffer_size SNs ahead of NSSN.
+ */
+static bool iwl_mvm_is_sn_less(u16 sn1, u16 sn2, u16 buffer_size)
+{
+ return ieee80211_sn_less(sn1, sn2) &&
+ !ieee80211_sn_less(sn1, sn2 - buffer_size);
+}
+
+#define RX_REORDER_BUF_TIMEOUT_MQ (HZ / 10)
+
+static void iwl_mvm_release_frames(struct iwl_mvm *mvm,
+ struct ieee80211_sta *sta,
+ struct napi_struct *napi,
+ struct iwl_mvm_reorder_buffer *reorder_buf,
+ u16 nssn)
+{
+ u16 ssn = reorder_buf->head_sn;
+
+ lockdep_assert_held(&reorder_buf->lock);
+
+ /* ignore nssn smaller than head sn - this can happen due to timeout */
+ if (iwl_mvm_is_sn_less(nssn, ssn, reorder_buf->buf_size))
+ return;
+
+ while (iwl_mvm_is_sn_less(ssn, nssn, reorder_buf->buf_size)) {
+ int index = ssn % reorder_buf->buf_size;
+ struct sk_buff_head *skb_list = &reorder_buf->entries[index];
+ struct sk_buff *skb;
+
+ ssn = ieee80211_sn_inc(ssn);
+
+ /* holes are valid since nssn indicates frames were received. */
+ if (skb_queue_empty(skb_list) || !skb_peek_tail(skb_list))
+ continue;
+ /* Empty the list. Will have more than one frame for A-MSDU */
+ while ((skb = __skb_dequeue(skb_list))) {
+ iwl_mvm_pass_packet_to_mac80211(mvm, napi, skb,
+ reorder_buf->queue,
+ sta);
+ reorder_buf->num_stored--;
+ }
+ }
+ reorder_buf->head_sn = nssn;
+
+ if (reorder_buf->num_stored && !reorder_buf->removed) {
+ u16 index = reorder_buf->head_sn % reorder_buf->buf_size;
+
+ while (!skb_peek_tail(&reorder_buf->entries[index]))
+ index = (index + 1) % reorder_buf->buf_size;
+ /* modify timer to match next frame's expiration time */
+ mod_timer(&reorder_buf->reorder_timer,
+ reorder_buf->reorder_time[index] + 1 +
+ RX_REORDER_BUF_TIMEOUT_MQ);
+ } else {
+ del_timer(&reorder_buf->reorder_timer);
+ }
+}
+
+void iwl_mvm_reorder_timer_expired(unsigned long data)
+{
+ struct iwl_mvm_reorder_buffer *buf = (void *)data;
+ int i;
+ u16 sn = 0, index = 0;
+ bool expired = false;
+
+ spin_lock_bh(&buf->lock);
+
+ if (!buf->num_stored || buf->removed) {
+ spin_unlock_bh(&buf->lock);
+ return;
+ }
+
+ for (i = 0; i < buf->buf_size ; i++) {
+ index = (buf->head_sn + i) % buf->buf_size;
+
+ if (!skb_peek_tail(&buf->entries[index]))
+ continue;
+ if (!time_after(jiffies, buf->reorder_time[index] +
+ RX_REORDER_BUF_TIMEOUT_MQ))
+ break;
+ expired = true;
+ sn = ieee80211_sn_add(buf->head_sn, i + 1);
+ }
+
+ if (expired) {
+ struct ieee80211_sta *sta;
+
+ rcu_read_lock();
+ sta = rcu_dereference(buf->mvm->fw_id_to_mac_id[buf->sta_id]);
+ /* SN is set to the last expired frame + 1 */
+ iwl_mvm_release_frames(buf->mvm, sta, NULL, buf, sn);
+ rcu_read_unlock();
+ } else if (buf->num_stored) {
+ /*
+ * If no frame expired and there are stored frames, index is now
+ * pointing to the first unexpired frame - modify timer
+ * accordingly to this frame.
+ */
+ mod_timer(&buf->reorder_timer,
+ buf->reorder_time[index] +
+ 1 + RX_REORDER_BUF_TIMEOUT_MQ);
+ }
+ spin_unlock_bh(&buf->lock);
+}
+
+static void iwl_mvm_del_ba(struct iwl_mvm *mvm, int queue,
+ struct iwl_mvm_delba_data *data)
+{
+ struct iwl_mvm_baid_data *ba_data;
+ struct ieee80211_sta *sta;
+ struct iwl_mvm_reorder_buffer *reorder_buf;
+ u8 baid = data->baid;
+
+ if (WARN_ON_ONCE(baid >= IWL_RX_REORDER_DATA_INVALID_BAID))
+ return;
+
+ rcu_read_lock();
+
+ ba_data = rcu_dereference(mvm->baid_map[baid]);
+ if (WARN_ON_ONCE(!ba_data))
+ goto out;
+
+ sta = rcu_dereference(mvm->fw_id_to_mac_id[ba_data->sta_id]);
+ if (WARN_ON_ONCE(IS_ERR_OR_NULL(sta)))
+ goto out;
+
+ reorder_buf = &ba_data->reorder_buf[queue];
+
+ /* release all frames that are in the reorder buffer to the stack */
+ spin_lock_bh(&reorder_buf->lock);
+ iwl_mvm_release_frames(mvm, sta, NULL, reorder_buf,
+ ieee80211_sn_add(reorder_buf->head_sn,
+ reorder_buf->buf_size));
+ spin_unlock_bh(&reorder_buf->lock);
+ del_timer_sync(&reorder_buf->reorder_timer);
+
+out:
+ rcu_read_unlock();
+}
+
void iwl_mvm_rx_queue_notif(struct iwl_mvm *mvm, struct iwl_rx_cmd_buffer *rxb,
int queue)
{
@@ -405,15 +549,182 @@
notif = (void *)pkt->data;
internal_notif = (void *)notif->payload;
+ if (internal_notif->sync) {
+ if (mvm->queue_sync_cookie != internal_notif->cookie) {
+ WARN_ONCE(1,
+ "Received expired RX queue sync message\n");
+ return;
+ }
+ atomic_dec(&mvm->queue_sync_counter);
+ }
+
switch (internal_notif->type) {
+ case IWL_MVM_RXQ_EMPTY:
+ break;
case IWL_MVM_RXQ_NOTIF_DEL_BA:
- /* TODO */
+ iwl_mvm_del_ba(mvm, queue, (void *)internal_notif->data);
break;
default:
WARN_ONCE(1, "Invalid identifier %d", internal_notif->type);
}
}
+/*
+ * Returns true if the MPDU was buffered\dropped, false if it should be passed
+ * to upper layer.
+ */
+static bool iwl_mvm_reorder(struct iwl_mvm *mvm,
+ struct napi_struct *napi,
+ int queue,
+ struct ieee80211_sta *sta,
+ struct sk_buff *skb,
+ struct iwl_rx_mpdu_desc *desc)
+{
+ struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
+ struct iwl_mvm_sta *mvm_sta = iwl_mvm_sta_from_mac80211(sta);
+ struct iwl_mvm_baid_data *baid_data;
+ struct iwl_mvm_reorder_buffer *buffer;
+ struct sk_buff *tail;
+ u32 reorder = le32_to_cpu(desc->reorder_data);
+ bool amsdu = desc->mac_flags2 & IWL_RX_MPDU_MFLG2_AMSDU;
+ u8 tid = *ieee80211_get_qos_ctl(hdr) & IEEE80211_QOS_CTL_TID_MASK;
+ u8 sub_frame_idx = desc->amsdu_info &
+ IWL_RX_MPDU_AMSDU_SUBFRAME_IDX_MASK;
+ int index;
+ u16 nssn, sn;
+ u8 baid;
+
+ baid = (reorder & IWL_RX_MPDU_REORDER_BAID_MASK) >>
+ IWL_RX_MPDU_REORDER_BAID_SHIFT;
+
+ if (baid == IWL_RX_REORDER_DATA_INVALID_BAID)
+ return false;
+
+ /* no sta yet */
+ if (WARN_ON(IS_ERR_OR_NULL(sta)))
+ return false;
+
+ /* not a data packet */
+ if (!ieee80211_is_data_qos(hdr->frame_control) ||
+ is_multicast_ether_addr(hdr->addr1))
+ return false;
+
+ if (unlikely(!ieee80211_is_data_present(hdr->frame_control)))
+ return false;
+
+ baid_data = rcu_dereference(mvm->baid_map[baid]);
+ if (WARN(!baid_data,
+ "Received baid %d, but no data exists for this BAID\n", baid))
+ return false;
+ if (WARN(tid != baid_data->tid || mvm_sta->sta_id != baid_data->sta_id,
+ "baid 0x%x is mapped to sta:%d tid:%d, but was received for sta:%d tid:%d\n",
+ baid, baid_data->sta_id, baid_data->tid, mvm_sta->sta_id,
+ tid))
+ return false;
+
+ nssn = reorder & IWL_RX_MPDU_REORDER_NSSN_MASK;
+ sn = (reorder & IWL_RX_MPDU_REORDER_SN_MASK) >>
+ IWL_RX_MPDU_REORDER_SN_SHIFT;
+
+ buffer = &baid_data->reorder_buf[queue];
+
+ spin_lock_bh(&buffer->lock);
+
+ /*
+ * If there was a significant jump in the nssn - adjust.
+ * If the SN is smaller than the NSSN it might need to first go into
+ * the reorder buffer, in which case we just release up to it and the
+ * rest of the function will take of storing it and releasing up to the
+ * nssn
+ */
+ if (!iwl_mvm_is_sn_less(nssn, buffer->head_sn + buffer->buf_size,
+ buffer->buf_size)) {
+ u16 min_sn = ieee80211_sn_less(sn, nssn) ? sn : nssn;
+
+ iwl_mvm_release_frames(mvm, sta, napi, buffer, min_sn);
+ }
+
+ /* drop any oudated packets */
+ if (ieee80211_sn_less(sn, buffer->head_sn))
+ goto drop;
+
+ /* release immediately if allowed by nssn and no stored frames */
+ if (!buffer->num_stored && ieee80211_sn_less(sn, nssn)) {
+ if (iwl_mvm_is_sn_less(buffer->head_sn, nssn,
+ buffer->buf_size))
+ buffer->head_sn = nssn;
+ /* No need to update AMSDU last SN - we are moving the head */
+ spin_unlock_bh(&buffer->lock);
+ return false;
+ }
+
+ index = sn % buffer->buf_size;
+
+ /*
+ * Check if we already stored this frame
+ * As AMSDU is either received or not as whole, logic is simple:
+ * If we have frames in that position in the buffer and the last frame
+ * originated from AMSDU had a different SN then it is a retransmission.
+ * If it is the same SN then if the subframe index is incrementing it
+ * is the same AMSDU - otherwise it is a retransmission.
+ */
+ tail = skb_peek_tail(&buffer->entries[index]);
+ if (tail && !amsdu)
+ goto drop;
+ else if (tail && (sn != buffer->last_amsdu ||
+ buffer->last_sub_index >= sub_frame_idx))
+ goto drop;
+
+ /* put in reorder buffer */
+ __skb_queue_tail(&buffer->entries[index], skb);
+ buffer->num_stored++;
+ buffer->reorder_time[index] = jiffies;
+
+ if (amsdu) {
+ buffer->last_amsdu = sn;
+ buffer->last_sub_index = sub_frame_idx;
+ }
+
+ iwl_mvm_release_frames(mvm, sta, napi, buffer, nssn);
+ spin_unlock_bh(&buffer->lock);
+ return true;
+
+drop:
+ kfree_skb(skb);
+ spin_unlock_bh(&buffer->lock);
+ return true;
+}
+
+static void iwl_mvm_agg_rx_received(struct iwl_mvm *mvm, u8 baid)
+{
+ unsigned long now = jiffies;
+ unsigned long timeout;
+ struct iwl_mvm_baid_data *data;
+
+ rcu_read_lock();
+
+ data = rcu_dereference(mvm->baid_map[baid]);
+ if (WARN_ON(!data))
+ goto out;
+
+ if (!data->timeout)
+ goto out;
+
+ timeout = data->timeout;
+ /*
+ * Do not update last rx all the time to avoid cache bouncing
+ * between the rx queues.
+ * Update it every timeout. Worst case is the session will
+ * expire after ~ 2 * timeout, which doesn't matter that much.
+ */
+ if (time_before(data->last_rx + TU_TO_JIFFIES(timeout), now))
+ /* Update is atomic */
+ data->last_rx = now;
+
+out:
+ rcu_read_unlock();
+}
+
void iwl_mvm_rx_mpdu_mq(struct iwl_mvm *mvm, struct napi_struct *napi,
struct iwl_rx_cmd_buffer *rxb, int queue)
{
@@ -484,6 +795,9 @@
if (sta) {
struct iwl_mvm_sta *mvmsta = iwl_mvm_sta_from_mac80211(sta);
+ u8 baid = (u8)((le32_to_cpu(desc->reorder_data) &
+ IWL_RX_MPDU_REORDER_BAID_MASK) >>
+ IWL_RX_MPDU_REORDER_BAID_SHIFT);
/*
* We have tx blocked stations (with CS bit). If we heard
@@ -536,6 +850,8 @@
*qc &= ~IEEE80211_QOS_CTL_A_MSDU_PRESENT;
}
+ if (baid != IWL_RX_REORDER_DATA_INVALID_BAID)
+ iwl_mvm_agg_rx_received(mvm, baid);
}
/*
@@ -593,12 +909,42 @@
/* TODO: PHY info - gscan */
iwl_mvm_create_skb(skb, hdr, len, crypt_len, rxb);
- iwl_mvm_pass_packet_to_mac80211(mvm, napi, skb, queue, sta);
+ if (!iwl_mvm_reorder(mvm, napi, queue, sta, skb, desc))
+ iwl_mvm_pass_packet_to_mac80211(mvm, napi, skb, queue, sta);
rcu_read_unlock();
}
-void iwl_mvm_rx_frame_release(struct iwl_mvm *mvm,
+void iwl_mvm_rx_frame_release(struct iwl_mvm *mvm, struct napi_struct *napi,
struct iwl_rx_cmd_buffer *rxb, int queue)
{
- /* TODO */
+ struct iwl_rx_packet *pkt = rxb_addr(rxb);
+ struct iwl_frame_release *release = (void *)pkt->data;
+ struct ieee80211_sta *sta;
+ struct iwl_mvm_reorder_buffer *reorder_buf;
+ struct iwl_mvm_baid_data *ba_data;
+
+ int baid = release->baid;
+
+ if (WARN_ON_ONCE(baid == IWL_RX_REORDER_DATA_INVALID_BAID))
+ return;
+
+ rcu_read_lock();
+
+ ba_data = rcu_dereference(mvm->baid_map[baid]);
+ if (WARN_ON_ONCE(!ba_data))
+ goto out;
+
+ sta = rcu_dereference(mvm->fw_id_to_mac_id[ba_data->sta_id]);
+ if (WARN_ON_ONCE(IS_ERR_OR_NULL(sta)))
+ goto out;
+
+ reorder_buf = &ba_data->reorder_buf[queue];
+
+ spin_lock_bh(&reorder_buf->lock);
+ iwl_mvm_release_frames(mvm, sta, napi, reorder_buf,
+ le16_to_cpu(release->nssn));
+ spin_unlock_bh(&reorder_buf->lock);
+
+out:
+ rcu_read_unlock();
}
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/sta.c b/drivers/net/wireless/intel/iwlwifi/mvm/sta.c
index 12614b7..fea4d3437 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/sta.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/sta.c
@@ -223,6 +223,39 @@
return ret;
}
+static void iwl_mvm_rx_agg_session_expired(unsigned long data)
+{
+ struct iwl_mvm_baid_data __rcu **rcu_ptr = (void *)data;
+ struct iwl_mvm_baid_data *ba_data;
+ struct ieee80211_sta *sta;
+ struct iwl_mvm_sta *mvm_sta;
+ unsigned long timeout;
+
+ rcu_read_lock();
+
+ ba_data = rcu_dereference(*rcu_ptr);
+
+ if (WARN_ON(!ba_data))
+ goto unlock;
+
+ if (!ba_data->timeout)
+ goto unlock;
+
+ timeout = ba_data->last_rx + TU_TO_JIFFIES(ba_data->timeout * 2);
+ if (time_is_after_jiffies(timeout)) {
+ mod_timer(&ba_data->session_timer, timeout);
+ goto unlock;
+ }
+
+ /* Timer expired */
+ sta = rcu_dereference(ba_data->mvm->fw_id_to_mac_id[ba_data->sta_id]);
+ mvm_sta = iwl_mvm_sta_from_mac80211(sta);
+ ieee80211_stop_rx_ba_session_offl(mvm_sta->vif,
+ sta->addr, ba_data->tid);
+unlock:
+ rcu_read_unlock();
+}
+
static int iwl_mvm_tdls_sta_init(struct iwl_mvm *mvm,
struct ieee80211_sta *sta)
{
@@ -293,6 +326,7 @@
u8 mac_queue = mvmsta->vif->hw_queue[ac];
int queue = -1;
int ssn;
+ int ret;
lockdep_assert_held(&mvm->mutex);
@@ -321,8 +355,15 @@
if (queue < 0)
queue = iwl_mvm_find_free_queue(mvm, IWL_MVM_DQA_MIN_DATA_QUEUE,
IWL_MVM_DQA_MAX_DATA_QUEUE);
+
+ /*
+ * Mark TXQ as ready, even though it hasn't been fully configured yet,
+ * to make sure no one else takes it.
+ * This will allow avoiding re-acquiring the lock at the end of the
+ * configuration. On error we'll mark it back as free.
+ */
if (queue >= 0)
- mvm->queue_info[queue].setup_reserved = false;
+ mvm->queue_info[queue].status = IWL_MVM_QUEUE_READY;
spin_unlock_bh(&mvm->queue_info_lock);
@@ -354,7 +395,16 @@
mvmsta->reserved_queue = IEEE80211_INVAL_HW_QUEUE;
spin_unlock_bh(&mvmsta->lock);
- return iwl_mvm_sta_send_to_fw(mvm, sta, true, STA_MODIFY_QUEUES);
+ ret = iwl_mvm_sta_send_to_fw(mvm, sta, true, STA_MODIFY_QUEUES);
+ if (ret)
+ goto out_err;
+
+ return 0;
+
+out_err:
+ iwl_mvm_disable_txq(mvm, queue, mac_queue, tid, 0);
+
+ return ret;
}
static inline u8 iwl_mvm_tid_to_ac_queue(int tid)
@@ -460,7 +510,8 @@
/* Make sure we have free resources for this STA */
if (vif_type == NL80211_IFTYPE_STATION && !sta->tdls &&
!mvm->queue_info[IWL_MVM_DQA_BSS_CLIENT_QUEUE].hw_queue_refcount &&
- !mvm->queue_info[IWL_MVM_DQA_BSS_CLIENT_QUEUE].setup_reserved)
+ (mvm->queue_info[IWL_MVM_DQA_BSS_CLIENT_QUEUE].status ==
+ IWL_MVM_QUEUE_FREE))
queue = IWL_MVM_DQA_BSS_CLIENT_QUEUE;
else
queue = iwl_mvm_find_free_queue(mvm, IWL_MVM_DQA_MIN_DATA_QUEUE,
@@ -470,7 +521,7 @@
IWL_ERR(mvm, "No available queues for new station\n");
return -ENOSPC;
}
- mvm->queue_info[queue].setup_reserved = true;
+ mvm->queue_info[queue].status = IWL_MVM_QUEUE_RESERVED;
spin_unlock_bh(&mvm->queue_info_lock);
@@ -1000,6 +1051,33 @@
lockdep_assert_held(&mvm->mutex);
+ if (iwl_mvm_is_dqa_supported(mvm)) {
+ struct iwl_trans_txq_scd_cfg cfg = {
+ .fifo = IWL_MVM_TX_FIFO_VO,
+ .sta_id = mvmvif->bcast_sta.sta_id,
+ .tid = IWL_MAX_TID_COUNT,
+ .aggregate = false,
+ .frame_limit = IWL_FRAME_LIMIT,
+ };
+ unsigned int wdg_timeout =
+ iwl_mvm_get_wd_timeout(mvm, vif, false, false);
+ int queue;
+
+ if ((vif->type == NL80211_IFTYPE_AP) &&
+ (mvmvif->bcast_sta.tfd_queue_msk &
+ BIT(IWL_MVM_DQA_AP_PROBE_RESP_QUEUE)))
+ queue = IWL_MVM_DQA_AP_PROBE_RESP_QUEUE;
+ else if ((vif->type == NL80211_IFTYPE_P2P_DEVICE) &&
+ (mvmvif->bcast_sta.tfd_queue_msk &
+ BIT(IWL_MVM_DQA_P2P_DEVICE_QUEUE)))
+ queue = IWL_MVM_DQA_P2P_DEVICE_QUEUE;
+ else if (WARN(1, "Missed required TXQ for adding bcast STA\n"))
+ return -EINVAL;
+
+ iwl_mvm_enable_txq(mvm, queue, vif->hw_queue[0], 0, &cfg,
+ wdg_timeout);
+ }
+
if (vif->type == NL80211_IFTYPE_ADHOC)
baddr = vif->bss_conf.bssid;
@@ -1028,20 +1106,28 @@
int iwl_mvm_alloc_bcast_sta(struct iwl_mvm *mvm, struct ieee80211_vif *vif)
{
struct iwl_mvm_vif *mvmvif = iwl_mvm_vif_from_mac80211(vif);
- u32 qmask;
+ u32 qmask = 0;
lockdep_assert_held(&mvm->mutex);
- qmask = iwl_mvm_mac_get_queues_mask(vif);
+ if (!iwl_mvm_is_dqa_supported(mvm))
+ qmask = iwl_mvm_mac_get_queues_mask(vif);
- /*
- * The firmware defines the TFD queue mask to only be relevant
- * for *unicast* queues, so the multicast (CAB) queue shouldn't
- * be included.
- */
- if (vif->type == NL80211_IFTYPE_AP)
+ if (vif->type == NL80211_IFTYPE_AP) {
+ /*
+ * The firmware defines the TFD queue mask to only be relevant
+ * for *unicast* queues, so the multicast (CAB) queue shouldn't
+ * be included.
+ */
qmask &= ~BIT(vif->cab_queue);
+ if (iwl_mvm_is_dqa_supported(mvm))
+ qmask |= BIT(IWL_MVM_DQA_AP_PROBE_RESP_QUEUE);
+ } else if (iwl_mvm_is_dqa_supported(mvm) &&
+ vif->type == NL80211_IFTYPE_P2P_DEVICE) {
+ qmask |= BIT(IWL_MVM_DQA_P2P_DEVICE_QUEUE);
+ }
+
return iwl_mvm_allocate_int_sta(mvm, &mvmvif->bcast_sta, qmask,
ieee80211_vif_type_p2p(vif));
}
@@ -1099,11 +1185,92 @@
#define IWL_MAX_RX_BA_SESSIONS 16
+static void iwl_mvm_sync_rxq_del_ba(struct iwl_mvm *mvm, u8 baid)
+{
+ struct iwl_mvm_delba_notif notif = {
+ .metadata.type = IWL_MVM_RXQ_NOTIF_DEL_BA,
+ .metadata.sync = 1,
+ .delba.baid = baid,
+ };
+ iwl_mvm_sync_rx_queues_internal(mvm, (void *)¬if, sizeof(notif));
+};
+
+static void iwl_mvm_free_reorder(struct iwl_mvm *mvm,
+ struct iwl_mvm_baid_data *data)
+{
+ int i;
+
+ iwl_mvm_sync_rxq_del_ba(mvm, data->baid);
+
+ for (i = 0; i < mvm->trans->num_rx_queues; i++) {
+ int j;
+ struct iwl_mvm_reorder_buffer *reorder_buf =
+ &data->reorder_buf[i];
+
+ spin_lock_bh(&reorder_buf->lock);
+ if (likely(!reorder_buf->num_stored)) {
+ spin_unlock_bh(&reorder_buf->lock);
+ continue;
+ }
+
+ /*
+ * This shouldn't happen in regular DELBA since the internal
+ * delBA notification should trigger a release of all frames in
+ * the reorder buffer.
+ */
+ WARN_ON(1);
+
+ for (j = 0; j < reorder_buf->buf_size; j++)
+ __skb_queue_purge(&reorder_buf->entries[j]);
+ /*
+ * Prevent timer re-arm. This prevents a very far fetched case
+ * where we timed out on the notification. There may be prior
+ * RX frames pending in the RX queue before the notification
+ * that might get processed between now and the actual deletion
+ * and we would re-arm the timer although we are deleting the
+ * reorder buffer.
+ */
+ reorder_buf->removed = true;
+ spin_unlock_bh(&reorder_buf->lock);
+ del_timer_sync(&reorder_buf->reorder_timer);
+ }
+}
+
+static void iwl_mvm_init_reorder_buffer(struct iwl_mvm *mvm,
+ u32 sta_id,
+ struct iwl_mvm_baid_data *data,
+ u16 ssn, u8 buf_size)
+{
+ int i;
+
+ for (i = 0; i < mvm->trans->num_rx_queues; i++) {
+ struct iwl_mvm_reorder_buffer *reorder_buf =
+ &data->reorder_buf[i];
+ int j;
+
+ reorder_buf->num_stored = 0;
+ reorder_buf->head_sn = ssn;
+ reorder_buf->buf_size = buf_size;
+ /* rx reorder timer */
+ reorder_buf->reorder_timer.function =
+ iwl_mvm_reorder_timer_expired;
+ reorder_buf->reorder_timer.data = (unsigned long)reorder_buf;
+ init_timer(&reorder_buf->reorder_timer);
+ spin_lock_init(&reorder_buf->lock);
+ reorder_buf->mvm = mvm;
+ reorder_buf->queue = i;
+ reorder_buf->sta_id = sta_id;
+ for (j = 0; j < reorder_buf->buf_size; j++)
+ __skb_queue_head_init(&reorder_buf->entries[j]);
+ }
+}
+
int iwl_mvm_sta_rx_agg(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
- int tid, u16 ssn, bool start, u8 buf_size)
+ int tid, u16 ssn, bool start, u8 buf_size, u16 timeout)
{
struct iwl_mvm_sta *mvm_sta = iwl_mvm_sta_from_mac80211(sta);
struct iwl_mvm_add_sta_cmd cmd = {};
+ struct iwl_mvm_baid_data *baid_data = NULL;
int ret;
u32 status;
@@ -1114,6 +1281,19 @@
return -ENOSPC;
}
+ if (iwl_mvm_has_new_rx_api(mvm) && start) {
+ /*
+ * Allocate here so if allocation fails we can bail out early
+ * before starting the BA session in the firmware
+ */
+ baid_data = kzalloc(sizeof(*baid_data) +
+ mvm->trans->num_rx_queues *
+ sizeof(baid_data->reorder_buf[0]),
+ GFP_KERNEL);
+ if (!baid_data)
+ return -ENOMEM;
+ }
+
cmd.mac_id_n_color = cpu_to_le32(mvm_sta->mac_id_n_color);
cmd.sta_id = mvm_sta->sta_id;
cmd.add_modify = STA_MODE_MODIFY;
@@ -1132,7 +1312,7 @@
iwl_mvm_add_sta_cmd_size(mvm),
&cmd, &status);
if (ret)
- return ret;
+ goto out_free;
switch (status & IWL_ADD_STA_STATUS_MASK) {
case ADD_STA_SUCCESS:
@@ -1150,14 +1330,75 @@
break;
}
- if (!ret) {
- if (start)
- mvm->rx_ba_sessions++;
- else if (mvm->rx_ba_sessions > 0)
- /* check that restart flow didn't zero the counter */
- mvm->rx_ba_sessions--;
- }
+ if (ret)
+ goto out_free;
+ if (start) {
+ u8 baid;
+
+ mvm->rx_ba_sessions++;
+
+ if (!iwl_mvm_has_new_rx_api(mvm))
+ return 0;
+
+ if (WARN_ON(!(status & IWL_ADD_STA_BAID_VALID_MASK))) {
+ ret = -EINVAL;
+ goto out_free;
+ }
+ baid = (u8)((status & IWL_ADD_STA_BAID_MASK) >>
+ IWL_ADD_STA_BAID_SHIFT);
+ baid_data->baid = baid;
+ baid_data->timeout = timeout;
+ baid_data->last_rx = jiffies;
+ init_timer(&baid_data->session_timer);
+ baid_data->session_timer.function =
+ iwl_mvm_rx_agg_session_expired;
+ baid_data->session_timer.data =
+ (unsigned long)&mvm->baid_map[baid];
+ baid_data->mvm = mvm;
+ baid_data->tid = tid;
+ baid_data->sta_id = mvm_sta->sta_id;
+
+ mvm_sta->tid_to_baid[tid] = baid;
+ if (timeout)
+ mod_timer(&baid_data->session_timer,
+ TU_TO_EXP_TIME(timeout * 2));
+
+ iwl_mvm_init_reorder_buffer(mvm, mvm_sta->sta_id,
+ baid_data, ssn, buf_size);
+ /*
+ * protect the BA data with RCU to cover a case where our
+ * internal RX sync mechanism will timeout (not that it's
+ * supposed to happen) and we will free the session data while
+ * RX is being processed in parallel
+ */
+ WARN_ON(rcu_access_pointer(mvm->baid_map[baid]));
+ rcu_assign_pointer(mvm->baid_map[baid], baid_data);
+ } else if (mvm->rx_ba_sessions > 0) {
+ u8 baid = mvm_sta->tid_to_baid[tid];
+
+ /* check that restart flow didn't zero the counter */
+ mvm->rx_ba_sessions--;
+ if (!iwl_mvm_has_new_rx_api(mvm))
+ return 0;
+
+ if (WARN_ON(baid == IWL_RX_REORDER_DATA_INVALID_BAID))
+ return -EINVAL;
+
+ baid_data = rcu_access_pointer(mvm->baid_map[baid]);
+ if (WARN_ON(!baid_data))
+ return -EINVAL;
+
+ /* synchronize all rx queues so we can safely delete */
+ iwl_mvm_free_reorder(mvm, baid_data);
+ del_timer_sync(&baid_data->session_timer);
+ RCU_INIT_POINTER(mvm->baid_map[baid], NULL);
+ kfree_rcu(baid_data, rcu_head);
+ }
+ return 0;
+
+out_free:
+ kfree(baid_data);
return ret;
}
@@ -1175,7 +1416,9 @@
mvm_sta->tfd_queue_msk |= BIT(queue);
mvm_sta->tid_disable_agg &= ~BIT(tid);
} else {
- mvm_sta->tfd_queue_msk &= ~BIT(queue);
+ /* In DQA-mode the queue isn't removed on agg termination */
+ if (!iwl_mvm_is_dqa_supported(mvm))
+ mvm_sta->tfd_queue_msk &= ~BIT(queue);
mvm_sta->tid_disable_agg |= BIT(tid);
}
@@ -1258,17 +1501,35 @@
spin_lock_bh(&mvm->queue_info_lock);
- txq_id = iwl_mvm_find_free_queue(mvm, mvm->first_agg_queue,
- mvm->last_agg_queue);
- if (txq_id < 0) {
- ret = txq_id;
- spin_unlock_bh(&mvm->queue_info_lock);
- IWL_ERR(mvm, "Failed to allocate agg queue\n");
- goto release_locks;
+ /*
+ * Note the possible cases:
+ * 1. In DQA mode with an enabled TXQ - TXQ needs to become agg'ed
+ * 2. Non-DQA mode: the TXQ hasn't yet been enabled, so find a free
+ * one and mark it as reserved
+ * 3. In DQA mode, but no traffic yet on this TID: same treatment as in
+ * non-DQA mode, since the TXQ hasn't yet been allocated
+ */
+ txq_id = mvmsta->tid_data[tid].txq_id;
+ if (!iwl_mvm_is_dqa_supported(mvm) ||
+ mvm->queue_info[txq_id].status != IWL_MVM_QUEUE_READY) {
+ txq_id = iwl_mvm_find_free_queue(mvm, mvm->first_agg_queue,
+ mvm->last_agg_queue);
+ if (txq_id < 0) {
+ ret = txq_id;
+ spin_unlock_bh(&mvm->queue_info_lock);
+ IWL_ERR(mvm, "Failed to allocate agg queue\n");
+ goto release_locks;
+ }
+
+ /* TXQ hasn't yet been enabled, so mark it only as reserved */
+ mvm->queue_info[txq_id].status = IWL_MVM_QUEUE_RESERVED;
}
- mvm->queue_info[txq_id].setup_reserved = true;
spin_unlock_bh(&mvm->queue_info_lock);
+ IWL_DEBUG_TX_QUEUES(mvm,
+ "AGG for tid %d will be on queue #%d\n",
+ tid, txq_id);
+
tid_data = &mvmsta->tid_data[tid];
tid_data->ssn = IEEE80211_SEQ_TO_SN(tid_data->seq_number);
tid_data->txq_id = txq_id;
@@ -1303,6 +1564,7 @@
unsigned int wdg_timeout =
iwl_mvm_get_wd_timeout(mvm, vif, sta->tdls, false);
int queue, ret;
+ bool alloc_queue = true;
u16 ssn;
struct iwl_trans_txq_scd_cfg cfg = {
@@ -1328,8 +1590,46 @@
cfg.fifo = iwl_mvm_ac_to_tx_fifo[tid_to_mac80211_ac[tid]];
- iwl_mvm_enable_txq(mvm, queue, vif->hw_queue[tid_to_mac80211_ac[tid]],
- ssn, &cfg, wdg_timeout);
+ /* In DQA mode, the existing queue might need to be reconfigured */
+ if (iwl_mvm_is_dqa_supported(mvm)) {
+ spin_lock_bh(&mvm->queue_info_lock);
+ /* Maybe there is no need to even alloc a queue... */
+ if (mvm->queue_info[queue].status == IWL_MVM_QUEUE_READY)
+ alloc_queue = false;
+ spin_unlock_bh(&mvm->queue_info_lock);
+
+ /*
+ * Only reconfig the SCD for the queue if the window size has
+ * changed from current (become smaller)
+ */
+ if (!alloc_queue && buf_size < mvmsta->max_agg_bufsize) {
+ /*
+ * If reconfiguring an existing queue, it first must be
+ * drained
+ */
+ ret = iwl_trans_wait_tx_queue_empty(mvm->trans,
+ BIT(queue));
+ if (ret) {
+ IWL_ERR(mvm,
+ "Error draining queue before reconfig\n");
+ return ret;
+ }
+
+ ret = iwl_mvm_reconfig_scd(mvm, queue, cfg.fifo,
+ mvmsta->sta_id, tid,
+ buf_size, ssn);
+ if (ret) {
+ IWL_ERR(mvm,
+ "Error reconfiguring TXQ #%d\n", queue);
+ return ret;
+ }
+ }
+ }
+
+ if (alloc_queue)
+ iwl_mvm_enable_txq(mvm, queue,
+ vif->hw_queue[tid_to_mac80211_ac[tid]], ssn,
+ &cfg, wdg_timeout);
ret = iwl_mvm_sta_tx_agg(mvm, sta, tid, queue, true);
if (ret)
@@ -1337,7 +1637,7 @@
/* No need to mark as reserved */
spin_lock_bh(&mvm->queue_info_lock);
- mvm->queue_info[queue].setup_reserved = false;
+ mvm->queue_info[queue].status = IWL_MVM_QUEUE_READY;
spin_unlock_bh(&mvm->queue_info_lock);
/*
@@ -1384,9 +1684,16 @@
mvmsta->agg_tids &= ~BIT(tid);
- /* No need to mark as reserved anymore */
spin_lock_bh(&mvm->queue_info_lock);
- mvm->queue_info[txq_id].setup_reserved = false;
+ /*
+ * The TXQ is marked as reserved only if no traffic came through yet
+ * This means no traffic has been sent on this TID (agg'd or not), so
+ * we no longer have use for the queue. Since it hasn't even been
+ * allocated through iwl_mvm_enable_txq, so we can just mark it back as
+ * free.
+ */
+ if (mvm->queue_info[txq_id].status == IWL_MVM_QUEUE_RESERVED)
+ mvm->queue_info[txq_id].status = IWL_MVM_QUEUE_FREE;
spin_unlock_bh(&mvm->queue_info_lock);
switch (tid_data->state) {
@@ -1412,9 +1719,11 @@
iwl_mvm_sta_tx_agg(mvm, sta, tid, txq_id, false);
- iwl_mvm_disable_txq(mvm, txq_id,
- vif->hw_queue[tid_to_mac80211_ac[tid]], tid,
- 0);
+ if (!iwl_mvm_is_dqa_supported(mvm)) {
+ int mac_queue = vif->hw_queue[tid_to_mac80211_ac[tid]];
+
+ iwl_mvm_disable_txq(mvm, txq_id, mac_queue, tid, 0);
+ }
return 0;
case IWL_AGG_STARTING:
case IWL_EMPTYING_HW_QUEUE_ADDBA:
@@ -1465,9 +1774,16 @@
mvmsta->agg_tids &= ~BIT(tid);
spin_unlock_bh(&mvmsta->lock);
- /* No need to mark as reserved */
spin_lock_bh(&mvm->queue_info_lock);
- mvm->queue_info[txq_id].setup_reserved = false;
+ /*
+ * The TXQ is marked as reserved only if no traffic came through yet
+ * This means no traffic has been sent on this TID (agg'd or not), so
+ * we no longer have use for the queue. Since it hasn't even been
+ * allocated through iwl_mvm_enable_txq, so we can just mark it back as
+ * free.
+ */
+ if (mvm->queue_info[txq_id].status == IWL_MVM_QUEUE_RESERVED)
+ mvm->queue_info[txq_id].status = IWL_MVM_QUEUE_FREE;
spin_unlock_bh(&mvm->queue_info_lock);
if (old_state >= IWL_AGG_ON) {
@@ -1480,9 +1796,12 @@
iwl_mvm_sta_tx_agg(mvm, sta, tid, txq_id, false);
- iwl_mvm_disable_txq(mvm, tid_data->txq_id,
- vif->hw_queue[tid_to_mac80211_ac[tid]], tid,
- 0);
+ if (!iwl_mvm_is_dqa_supported(mvm)) {
+ int mac_queue = vif->hw_queue[tid_to_mac80211_ac[tid]];
+
+ iwl_mvm_disable_txq(mvm, tid_data->txq_id, mac_queue,
+ tid, 0);
+ }
}
return 0;
@@ -1533,17 +1852,12 @@
mvmvif->ap_sta_id != IWL_MVM_STATION_COUNT) {
u8 sta_id = mvmvif->ap_sta_id;
- sta = rcu_dereference_check(mvm->fw_id_to_mac_id[sta_id],
- lockdep_is_held(&mvm->mutex));
/*
* It is possible that the 'sta' parameter is NULL,
* for example when a GTK is removed - the sta_id will then
* be the AP ID, and no station was passed by mac80211.
*/
- if (IS_ERR_OR_NULL(sta))
- return NULL;
-
- return iwl_mvm_sta_from_mac80211(sta);
+ return iwl_mvm_sta_from_staid_protected(mvm, sta_id);
}
return NULL;
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/sta.h b/drivers/net/wireless/intel/iwlwifi/mvm/sta.h
index e3efdcd..d2c58f1 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/sta.h
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/sta.h
@@ -348,6 +348,15 @@
} ____cacheline_aligned_in_smp q[];
};
+struct iwl_mvm_delba_data {
+ u32 baid;
+} __packed;
+
+struct iwl_mvm_delba_notif {
+ struct iwl_mvm_internal_rxq_notif metadata;
+ struct iwl_mvm_delba_data delba;
+} __packed;
+
/**
* struct iwl_mvm_rxq_dup_data - per station per rx queue data
* @last_seq: last sequence per tid for duplicate packet detection
@@ -373,6 +382,7 @@
* @lock: lock to protect the whole struct. Since %tid_data is access from Tx
* and from Tx response flow, it needs a spinlock.
* @tid_data: per tid data + mgmt. Look at %iwl_mvm_tid_data.
+ * @tid_to_baid: a simple map of TID to baid
* @reserved_queue: the queue reserved for this STA for DQA purposes
* Every STA has is given one reserved queue to allow it to operate. If no
* such queue can be guaranteed, the STA addition will fail.
@@ -406,6 +416,7 @@
bool next_status_eosp;
spinlock_t lock;
struct iwl_mvm_tid_data tid_data[IWL_MAX_TID_COUNT + 1];
+ u8 tid_to_baid[IWL_MAX_TID_COUNT];
struct iwl_lq_sta lq_sta;
struct ieee80211_vif *vif;
struct iwl_mvm_key_pn __rcu *ptk_pn[4];
@@ -487,7 +498,7 @@
/* AMPDU */
int iwl_mvm_sta_rx_agg(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
- int tid, u16 ssn, bool start, u8 buf_size);
+ int tid, u16 ssn, bool start, u8 buf_size, u16 timeout);
int iwl_mvm_sta_tx_agg_start(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
struct ieee80211_sta *sta, u16 tid, u16 *ssn);
int iwl_mvm_sta_tx_agg_oper(struct iwl_mvm *mvm, struct ieee80211_vif *vif,
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/tt.c b/drivers/net/wireless/intel/iwlwifi/mvm/tt.c
index eb3f460..58fc7b3 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/tt.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/tt.c
@@ -359,16 +359,14 @@
static void iwl_mvm_tt_tx_protection(struct iwl_mvm *mvm, bool enable)
{
- struct ieee80211_sta *sta;
struct iwl_mvm_sta *mvmsta;
int i, err;
for (i = 0; i < IWL_MVM_STATION_COUNT; i++) {
- sta = rcu_dereference_protected(mvm->fw_id_to_mac_id[i],
- lockdep_is_held(&mvm->mutex));
- if (IS_ERR_OR_NULL(sta))
+ mvmsta = iwl_mvm_sta_from_staid_protected(mvm, i);
+ if (!mvmsta)
continue;
- mvmsta = iwl_mvm_sta_from_mac80211(sta);
+
if (enable == mvmsta->tt_tx_protection)
continue;
err = iwl_mvm_tx_protection(mvm, mvmsta, enable);
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/tx.c b/drivers/net/wireless/intel/iwlwifi/mvm/tx.c
index c53aa0f..779bafc 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/tx.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/tx.c
@@ -475,6 +475,21 @@
return dev_cmd;
}
+static int iwl_mvm_get_ctrl_vif_queue(struct iwl_mvm *mvm,
+ struct ieee80211_tx_info *info, __le16 fc)
+{
+ if (iwl_mvm_is_dqa_supported(mvm)) {
+ if (info->control.vif->type == NL80211_IFTYPE_AP &&
+ ieee80211_is_probe_resp(fc))
+ return IWL_MVM_DQA_AP_PROBE_RESP_QUEUE;
+ else if (ieee80211_is_mgmt(fc) &&
+ info->control.vif->type == NL80211_IFTYPE_P2P_DEVICE)
+ return IWL_MVM_DQA_P2P_DEVICE_QUEUE;
+ }
+
+ return info->hw_queue;
+}
+
int iwl_mvm_tx_skb_non_sta(struct iwl_mvm *mvm, struct sk_buff *skb)
{
struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
@@ -484,6 +499,7 @@
struct iwl_tx_cmd *tx_cmd;
u8 sta_id;
int hdrlen = ieee80211_hdrlen(hdr->frame_control);
+ int queue;
memcpy(&info, skb->cb, sizeof(info));
@@ -508,6 +524,8 @@
info.control.vif->type == NL80211_IFTYPE_STATION)
IEEE80211_SKB_CB(skb)->hw_queue = mvm->aux_queue;
+ queue = info.hw_queue;
+
/*
* If the interface on which the frame is sent is the P2P_DEVICE
* or an AP/GO interface use the broadcast station associated
@@ -523,10 +541,12 @@
iwl_mvm_vif_from_mac80211(info.control.vif);
if (info.control.vif->type == NL80211_IFTYPE_P2P_DEVICE ||
- info.control.vif->type == NL80211_IFTYPE_AP)
+ info.control.vif->type == NL80211_IFTYPE_AP) {
sta_id = mvmvif->bcast_sta.sta_id;
- else if (info.control.vif->type == NL80211_IFTYPE_STATION &&
- is_multicast_ether_addr(hdr->addr1)) {
+ queue = iwl_mvm_get_ctrl_vif_queue(mvm, &info,
+ hdr->frame_control);
+ } else if (info.control.vif->type == NL80211_IFTYPE_STATION &&
+ is_multicast_ether_addr(hdr->addr1)) {
u8 ap_sta_id = ACCESS_ONCE(mvmvif->ap_sta_id);
if (ap_sta_id != IWL_MVM_STATION_COUNT)
@@ -534,7 +554,7 @@
}
}
- IWL_DEBUG_TX(mvm, "station Id %d, queue=%d\n", sta_id, info.hw_queue);
+ IWL_DEBUG_TX(mvm, "station Id %d, queue=%d\n", sta_id, queue);
dev_cmd = iwl_mvm_set_tx_params(mvm, skb, &info, hdrlen, NULL, sta_id);
if (!dev_cmd)
@@ -545,7 +565,7 @@
/* Copy MAC header from skb into command buffer */
memcpy(tx_cmd->hdr, hdr, hdrlen);
- if (iwl_trans_tx(mvm->trans, skb, dev_cmd, info.hw_queue)) {
+ if (iwl_trans_tx(mvm->trans, skb, dev_cmd, queue)) {
iwl_trans_free_tx_cmd(mvm->trans, dev_cmd);
return -1;
}
@@ -589,9 +609,11 @@
if (WARN_ON_ONCE(tid >= IWL_MAX_TID_COUNT))
return -EINVAL;
+ dbg_max_amsdu_len = ACCESS_ONCE(mvm->max_amsdu_len);
+
if (!sta->max_amsdu_len ||
!ieee80211_is_data_qos(hdr->frame_control) ||
- !mvmsta->tlc_amsdu) {
+ (!mvmsta->tlc_amsdu && !dbg_max_amsdu_len)) {
num_subframes = 1;
pad = 0;
goto segment;
@@ -622,7 +644,6 @@
}
max_amsdu_len = sta->max_amsdu_len;
- dbg_max_amsdu_len = ACCESS_ONCE(mvm->max_amsdu_len);
/* the Tx FIFO to which this A-MSDU will be routed */
txf = iwl_mvm_ac_to_tx_fifo[tid_to_mac80211_ac[tid]];
@@ -636,7 +657,7 @@
max_amsdu_len = min_t(unsigned int, max_amsdu_len,
mvm->shared_mem_cfg.txfifo_size[txf] - 256);
- if (dbg_max_amsdu_len)
+ if (unlikely(dbg_max_amsdu_len))
max_amsdu_len = min_t(unsigned int, max_amsdu_len,
dbg_max_amsdu_len);
@@ -912,7 +933,8 @@
spin_unlock(&mvmsta->lock);
- if (txq_id < mvm->first_agg_queue)
+ /* Increase pending frames count if this isn't AMPDU */
+ if (!is_ampdu)
atomic_inc(&mvm->pending_frames[mvmsta->sta_id]);
return 0;
@@ -1160,6 +1182,7 @@
u8 skb_freed = 0;
u16 next_reclaimed, seq_ctl;
bool is_ndp = false;
+ bool txq_agg = false; /* Is this TXQ aggregated */
__skb_queue_head_init(&skbs);
@@ -1290,6 +1313,8 @@
bool send_eosp_ndp = false;
spin_lock_bh(&mvmsta->lock);
+ txq_agg = (mvmsta->tid_data[tid].state == IWL_AGG_ON);
+
if (!is_ndp) {
tid_data->next_reclaimed = next_reclaimed;
IWL_DEBUG_TX_REPLY(mvm,
@@ -1345,11 +1370,11 @@
* If the txq is not an AMPDU queue, there is no chance we freed
* several skbs. Check that out...
*/
- if (txq_id >= mvm->first_agg_queue)
+ if (txq_agg)
goto out;
/* We can't free more than one frame at once on a shared queue */
- WARN_ON(skb_freed > 1);
+ WARN_ON(!iwl_mvm_is_dqa_supported(mvm) && (skb_freed > 1));
/* If we have still frames for this STA nothing to do here */
if (!atomic_sub_and_test(skb_freed, &mvm->pending_frames[sta_id]))
@@ -1443,9 +1468,12 @@
int sta_id = IWL_MVM_TX_RES_GET_RA(tx_resp->ra_tid);
int tid = IWL_MVM_TX_RES_GET_TID(tx_resp->ra_tid);
u16 sequence = le16_to_cpu(pkt->hdr.sequence);
- struct ieee80211_sta *sta;
+ struct iwl_mvm_sta *mvmsta;
+ int queue = SEQ_TO_QUEUE(sequence);
- if (WARN_ON_ONCE(SEQ_TO_QUEUE(sequence) < mvm->first_agg_queue))
+ if (WARN_ON_ONCE(queue < mvm->first_agg_queue &&
+ (!iwl_mvm_is_dqa_supported(mvm) ||
+ (queue != IWL_MVM_DQA_BSS_CLIENT_QUEUE))))
return;
if (WARN_ON_ONCE(tid == IWL_TID_NON_QOS))
@@ -1455,10 +1483,9 @@
rcu_read_lock();
- sta = rcu_dereference(mvm->fw_id_to_mac_id[sta_id]);
+ mvmsta = iwl_mvm_sta_from_staid_rcu(mvm, sta_id);
- if (!WARN_ON_ONCE(IS_ERR_OR_NULL(sta))) {
- struct iwl_mvm_sta *mvmsta = iwl_mvm_sta_from_mac80211(sta);
+ if (!WARN_ON_ONCE(!mvmsta)) {
mvmsta->tid_data[tid].rate_n_flags =
le32_to_cpu(tx_resp->initial_rate);
mvmsta->tid_data[tid].tx_time =
diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/utils.c b/drivers/net/wireless/intel/iwlwifi/mvm/utils.c
index f0ffd62..161b99e 100644
--- a/drivers/net/wireless/intel/iwlwifi/mvm/utils.c
+++ b/drivers/net/wireless/intel/iwlwifi/mvm/utils.c
@@ -90,11 +90,17 @@
* the mutex, this ensures we don't try to send two
* (or more) synchronous commands at a time.
*/
- if (!(cmd->flags & CMD_ASYNC))
+ if (!(cmd->flags & CMD_ASYNC)) {
lockdep_assert_held(&mvm->mutex);
+ if (!(cmd->flags & CMD_SEND_IN_IDLE))
+ iwl_mvm_ref(mvm, IWL_MVM_REF_SENDING_CMD);
+ }
ret = iwl_trans_send_cmd(mvm->trans, cmd);
+ if (!(cmd->flags & (CMD_ASYNC | CMD_SEND_IN_IDLE)))
+ iwl_mvm_unref(mvm, IWL_MVM_REF_SENDING_CMD);
+
/*
* If the caller wants the SKB, then don't hide any problems, the
* caller might access the response buffer which will be NULL if
@@ -581,12 +587,45 @@
for (i = minq; i <= maxq; i++)
if (mvm->queue_info[i].hw_queue_refcount == 0 &&
- !mvm->queue_info[i].setup_reserved)
+ mvm->queue_info[i].status == IWL_MVM_QUEUE_FREE)
return i;
return -ENOSPC;
}
+int iwl_mvm_reconfig_scd(struct iwl_mvm *mvm, int queue, int fifo, int sta_id,
+ int tid, int frame_limit, u16 ssn)
+{
+ struct iwl_scd_txq_cfg_cmd cmd = {
+ .scd_queue = queue,
+ .enable = 1,
+ .window = frame_limit,
+ .sta_id = sta_id,
+ .ssn = cpu_to_le16(ssn),
+ .tx_fifo = fifo,
+ .aggregate = (queue >= IWL_MVM_DQA_MIN_DATA_QUEUE ||
+ queue == IWL_MVM_DQA_BSS_CLIENT_QUEUE),
+ .tid = tid,
+ };
+ int ret;
+
+ spin_lock_bh(&mvm->queue_info_lock);
+ if (WARN(mvm->queue_info[queue].hw_queue_refcount == 0,
+ "Trying to reconfig unallocated queue %d\n", queue)) {
+ spin_unlock_bh(&mvm->queue_info_lock);
+ return -ENXIO;
+ }
+ spin_unlock_bh(&mvm->queue_info_lock);
+
+ IWL_DEBUG_TX_QUEUES(mvm, "Reconfig SCD for TXQ #%d\n", queue);
+
+ ret = iwl_mvm_send_cmd_pdu(mvm, SCD_QUEUE_CFG, 0, sizeof(cmd), &cmd);
+ WARN_ONCE(ret, "Failed to re-configure queue %d on FIFO %d, ret=%d\n",
+ queue, fifo, ret);
+
+ return ret;
+}
+
void iwl_mvm_enable_txq(struct iwl_mvm *mvm, int queue, int mac80211_queue,
u16 ssn, const struct iwl_trans_txq_scd_cfg *cfg,
unsigned int wdg_timeout)
@@ -682,6 +721,8 @@
mvm->queue_info[queue].hw_queue_refcount--;
cmd.enable = mvm->queue_info[queue].hw_queue_refcount ? 1 : 0;
+ if (!cmd.enable)
+ mvm->queue_info[queue].status = IWL_MVM_QUEUE_FREE;
IWL_DEBUG_TX_QUEUES(mvm,
"Disabling TXQ #%d refcount=%d (mac80211 map:0x%x)\n",
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/drv.c b/drivers/net/wireless/intel/iwlwifi/pcie/drv.c
index de42066..a588b05 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/drv.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/drv.c
@@ -493,19 +493,20 @@
{IWL_PCI_DEVICE(0x24FD, 0x8130, iwl8265_2ac_cfg)},
/* 9000 Series */
- {IWL_PCI_DEVICE(0x9DF0, 0x0A10, iwl9560_2ac_cfg)},
- {IWL_PCI_DEVICE(0x9DF0, 0x0010, iwl9560_2ac_cfg)},
- {IWL_PCI_DEVICE(0x9DF0, 0x2A10, iwl5165_2ac_cfg)},
+ {IWL_PCI_DEVICE(0x2526, 0x0000, iwl9260_2ac_cfg)},
+ {IWL_PCI_DEVICE(0x2526, 0x0010, iwl9260_2ac_cfg)},
+ {IWL_PCI_DEVICE(0x9DF0, 0x0A10, iwl9260_2ac_cfg)},
+ {IWL_PCI_DEVICE(0x9DF0, 0x0010, iwl9260_2ac_cfg)},
+ {IWL_PCI_DEVICE(0x9DF0, 0x0210, iwl9260_2ac_cfg)},
+ {IWL_PCI_DEVICE(0x9DF0, 0x0410, iwl9260_2ac_cfg)},
+ {IWL_PCI_DEVICE(0x9DF0, 0x0610, iwl9260_2ac_cfg)},
+ {IWL_PCI_DEVICE(0x9DF0, 0x0310, iwl5165_2ac_cfg)},
+ {IWL_PCI_DEVICE(0x9DF0, 0x0000, iwl5165_2ac_cfg)},
+ {IWL_PCI_DEVICE(0x9DF0, 0x0510, iwl5165_2ac_cfg)},
{IWL_PCI_DEVICE(0x9DF0, 0x2010, iwl5165_2ac_cfg)},
{IWL_PCI_DEVICE(0x2526, 0x1420, iwl5165_2ac_cfg)},
- {IWL_PCI_DEVICE(0x2526, 0x0010, iwl5165_2ac_cfg)},
- {IWL_PCI_DEVICE(0x9DF0, 0x0000, iwl5165_2ac_cfg)},
- {IWL_PCI_DEVICE(0x9DF0, 0x0310, iwl5165_2ac_cfg)},
- {IWL_PCI_DEVICE(0x9DF0, 0x0510, iwl5165_2ac_cfg)},
{IWL_PCI_DEVICE(0x9DF0, 0x0710, iwl5165_2ac_cfg)},
- {IWL_PCI_DEVICE(0x9DF0, 0x0210, iwl9560_2ac_cfg)},
- {IWL_PCI_DEVICE(0x9DF0, 0x0410, iwl9560_2ac_cfg)},
- {IWL_PCI_DEVICE(0x9DF0, 0x0610, iwl9560_2ac_cfg)},
+ {IWL_PCI_DEVICE(0x9DF0, 0x2A10, iwl5165_2ac_cfg)},
#endif /* CONFIG_IWLMVM */
{0}
@@ -595,6 +596,7 @@
{
const struct iwl_cfg *cfg = (struct iwl_cfg *)(ent->driver_data);
const struct iwl_cfg *cfg_7265d __maybe_unused = NULL;
+ const struct iwl_cfg *cfg_9260lc __maybe_unused = NULL;
struct iwl_trans *iwl_trans;
struct iwl_trans_pcie *trans_pcie;
int ret;
@@ -622,6 +624,15 @@
cfg = cfg_7265d;
iwl_trans->cfg = cfg_7265d;
}
+
+ if (iwl_trans->cfg->rf_id) {
+ if (cfg == &iwl9260_2ac_cfg)
+ cfg_9260lc = &iwl9260lc_2ac_cfg;
+ if (cfg_9260lc && iwl_trans->hw_rf_id == CSR_HW_RF_ID_TYPE_LC) {
+ cfg = cfg_9260lc;
+ iwl_trans->cfg = cfg_9260lc;
+ }
+ }
#endif
pci_set_drvdata(pdev, iwl_trans);
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/internal.h b/drivers/net/wireless/intel/iwlwifi/pcie/internal.h
index 9ce4ec6..de6974f 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/internal.h
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/internal.h
@@ -481,9 +481,6 @@
struct sk_buff_head *skbs);
void iwl_trans_pcie_tx_reset(struct iwl_trans *trans);
-void iwl_trans_pcie_ref(struct iwl_trans *trans);
-void iwl_trans_pcie_unref(struct iwl_trans *trans);
-
static inline u16 iwl_pcie_tfd_tb_get_len(struct iwl_tfd *tfd, u8 idx)
{
struct iwl_tfd_tb *tb = &tfd->tbs[idx];
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/rx.c b/drivers/net/wireless/intel/iwlwifi/pcie/rx.c
index 7f8a232..0a4a3c5 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/rx.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/rx.c
@@ -161,10 +161,11 @@
return cpu_to_le32((u32)(dma_addr >> 8));
}
-static void iwl_pcie_write_prph_64(struct iwl_trans *trans, u64 ofs, u64 val)
+static void iwl_pcie_write_prph_64_no_grab(struct iwl_trans *trans, u64 ofs,
+ u64 val)
{
- iwl_write_prph(trans, ofs, val & 0xffffffff);
- iwl_write_prph(trans, ofs + 4, val >> 32);
+ iwl_write_prph_no_grab(trans, ofs, val & 0xffffffff);
+ iwl_write_prph_no_grab(trans, ofs + 4, val >> 32);
}
/*
@@ -208,8 +209,8 @@
rxq->write_actual = round_down(rxq->write, 8);
if (trans->cfg->mq_rx_supported)
- iwl_write_prph(trans, RFH_Q_FRBDCB_WIDX(rxq->id),
- rxq->write_actual);
+ iwl_write32(trans, RFH_Q_FRBDCB_WIDX_TRG(rxq->id),
+ rxq->write_actual);
/*
* write to FH_RSCSR_CHNL0_WPTR register even in MQ as a W/A to
* hardware shadow registers bug - writing to RFH_Q_FRBDCB_WIDX will
@@ -698,6 +699,7 @@
{
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
u32 rb_size;
+ unsigned long flags;
const u32 rfdnlog = RX_QUEUE_SIZE_LOG; /* 256 RBDs */
switch (trans_pcie->rx_buf_size) {
@@ -715,23 +717,26 @@
rb_size = FH_RCSR_RX_CONFIG_REG_VAL_RB_SIZE_4K;
}
+ if (!iwl_trans_grab_nic_access(trans, &flags))
+ return;
+
/* Stop Rx DMA */
- iwl_write_direct32(trans, FH_MEM_RCSR_CHNL0_CONFIG_REG, 0);
+ iwl_write32(trans, FH_MEM_RCSR_CHNL0_CONFIG_REG, 0);
/* reset and flush pointers */
- iwl_write_direct32(trans, FH_MEM_RCSR_CHNL0_RBDCB_WPTR, 0);
- iwl_write_direct32(trans, FH_MEM_RCSR_CHNL0_FLUSH_RB_REQ, 0);
- iwl_write_direct32(trans, FH_RSCSR_CHNL0_RDPTR, 0);
+ iwl_write32(trans, FH_MEM_RCSR_CHNL0_RBDCB_WPTR, 0);
+ iwl_write32(trans, FH_MEM_RCSR_CHNL0_FLUSH_RB_REQ, 0);
+ iwl_write32(trans, FH_RSCSR_CHNL0_RDPTR, 0);
/* Reset driver's Rx queue write index */
- iwl_write_direct32(trans, FH_RSCSR_CHNL0_RBDCB_WPTR_REG, 0);
+ iwl_write32(trans, FH_RSCSR_CHNL0_RBDCB_WPTR_REG, 0);
/* Tell device where to find RBD circular buffer in DRAM */
- iwl_write_direct32(trans, FH_RSCSR_CHNL0_RBDCB_BASE_REG,
- (u32)(rxq->bd_dma >> 8));
+ iwl_write32(trans, FH_RSCSR_CHNL0_RBDCB_BASE_REG,
+ (u32)(rxq->bd_dma >> 8));
/* Tell device where in DRAM to update its Rx status */
- iwl_write_direct32(trans, FH_RSCSR_CHNL0_STTS_WPTR_REG,
- rxq->rb_stts_dma >> 4);
+ iwl_write32(trans, FH_RSCSR_CHNL0_STTS_WPTR_REG,
+ rxq->rb_stts_dma >> 4);
/* Enable Rx DMA
* FH_RCSR_CHNL0_RX_IGNORE_RXF_EMPTY is set because of HW bug in
@@ -741,13 +746,15 @@
* RB timeout 0x10
* 256 RBDs
*/
- iwl_write_direct32(trans, FH_MEM_RCSR_CHNL0_CONFIG_REG,
- FH_RCSR_RX_CONFIG_CHNL_EN_ENABLE_VAL |
- FH_RCSR_CHNL0_RX_IGNORE_RXF_EMPTY |
- FH_RCSR_CHNL0_RX_CONFIG_IRQ_DEST_INT_HOST_VAL |
- rb_size|
- (RX_RB_TIMEOUT << FH_RCSR_RX_CONFIG_REG_IRQ_RBTH_POS)|
- (rfdnlog << FH_RCSR_RX_CONFIG_RBDCB_SIZE_POS));
+ iwl_write32(trans, FH_MEM_RCSR_CHNL0_CONFIG_REG,
+ FH_RCSR_RX_CONFIG_CHNL_EN_ENABLE_VAL |
+ FH_RCSR_CHNL0_RX_IGNORE_RXF_EMPTY |
+ FH_RCSR_CHNL0_RX_CONFIG_IRQ_DEST_INT_HOST_VAL |
+ rb_size |
+ (RX_RB_TIMEOUT << FH_RCSR_RX_CONFIG_REG_IRQ_RBTH_POS) |
+ (rfdnlog << FH_RCSR_RX_CONFIG_RBDCB_SIZE_POS));
+
+ iwl_trans_release_nic_access(trans, &flags);
/* Set interrupt coalescing timer to default (2048 usecs) */
iwl_write8(trans, CSR_INT_COALESCING, IWL_HOST_INT_TIMEOUT_DEF);
@@ -761,6 +768,7 @@
{
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
u32 rb_size, enabled = 0;
+ unsigned long flags;
int i;
switch (trans_pcie->rx_buf_size) {
@@ -778,25 +786,31 @@
rb_size = RFH_RXF_DMA_RB_SIZE_4K;
}
+ if (!iwl_trans_grab_nic_access(trans, &flags))
+ return;
+
/* Stop Rx DMA */
- iwl_write_prph(trans, RFH_RXF_DMA_CFG, 0);
+ iwl_write_prph_no_grab(trans, RFH_RXF_DMA_CFG, 0);
/* disable free amd used rx queue operation */
- iwl_write_prph(trans, RFH_RXF_RXQ_ACTIVE, 0);
+ iwl_write_prph_no_grab(trans, RFH_RXF_RXQ_ACTIVE, 0);
for (i = 0; i < trans->num_rx_queues; i++) {
/* Tell device where to find RBD free table in DRAM */
- iwl_pcie_write_prph_64(trans, RFH_Q_FRBDCB_BA_LSB(i),
- (u64)(trans_pcie->rxq[i].bd_dma));
+ iwl_pcie_write_prph_64_no_grab(trans,
+ RFH_Q_FRBDCB_BA_LSB(i),
+ trans_pcie->rxq[i].bd_dma);
/* Tell device where to find RBD used table in DRAM */
- iwl_pcie_write_prph_64(trans, RFH_Q_URBDCB_BA_LSB(i),
- (u64)(trans_pcie->rxq[i].used_bd_dma));
+ iwl_pcie_write_prph_64_no_grab(trans,
+ RFH_Q_URBDCB_BA_LSB(i),
+ trans_pcie->rxq[i].used_bd_dma);
/* Tell device where in DRAM to update its Rx status */
- iwl_pcie_write_prph_64(trans, RFH_Q_URBD_STTS_WPTR_LSB(i),
- trans_pcie->rxq[i].rb_stts_dma);
+ iwl_pcie_write_prph_64_no_grab(trans,
+ RFH_Q_URBD_STTS_WPTR_LSB(i),
+ trans_pcie->rxq[i].rb_stts_dma);
/* Reset device indice tables */
- iwl_write_prph(trans, RFH_Q_FRBDCB_WIDX(i), 0);
- iwl_write_prph(trans, RFH_Q_FRBDCB_RIDX(i), 0);
- iwl_write_prph(trans, RFH_Q_URBDCB_WIDX(i), 0);
+ iwl_write_prph_no_grab(trans, RFH_Q_FRBDCB_WIDX(i), 0);
+ iwl_write_prph_no_grab(trans, RFH_Q_FRBDCB_RIDX(i), 0);
+ iwl_write_prph_no_grab(trans, RFH_Q_URBDCB_WIDX(i), 0);
enabled |= BIT(i) | BIT(i + 16);
}
@@ -812,23 +826,26 @@
* Drop frames that exceed RB size
* 512 RBDs
*/
- iwl_write_prph(trans, RFH_RXF_DMA_CFG,
- RFH_DMA_EN_ENABLE_VAL |
- rb_size | RFH_RXF_DMA_SINGLE_FRAME_MASK |
- RFH_RXF_DMA_MIN_RB_4_8 |
- RFH_RXF_DMA_DROP_TOO_LARGE_MASK |
- RFH_RXF_DMA_RBDCB_SIZE_512);
+ iwl_write_prph_no_grab(trans, RFH_RXF_DMA_CFG,
+ RFH_DMA_EN_ENABLE_VAL |
+ rb_size | RFH_RXF_DMA_SINGLE_FRAME_MASK |
+ RFH_RXF_DMA_MIN_RB_4_8 |
+ RFH_RXF_DMA_DROP_TOO_LARGE_MASK |
+ RFH_RXF_DMA_RBDCB_SIZE_512);
/*
* Activate DMA snooping.
* Set RX DMA chunk size to 64B
* Default queue is 0
*/
- iwl_write_prph(trans, RFH_GEN_CFG, RFH_GEN_CFG_RFH_DMA_SNOOP |
- (DEFAULT_RXQ_NUM << RFH_GEN_CFG_DEFAULT_RXQ_NUM_POS) |
- RFH_GEN_CFG_SERVICE_DMA_SNOOP);
+ iwl_write_prph_no_grab(trans, RFH_GEN_CFG, RFH_GEN_CFG_RFH_DMA_SNOOP |
+ (DEFAULT_RXQ_NUM <<
+ RFH_GEN_CFG_DEFAULT_RXQ_NUM_POS) |
+ RFH_GEN_CFG_SERVICE_DMA_SNOOP);
/* Enable the relevant rx queues */
- iwl_write_prph(trans, RFH_RXF_RXQ_ACTIVE, enabled);
+ iwl_write_prph_no_grab(trans, RFH_RXF_RXQ_ACTIVE, enabled);
+
+ iwl_trans_release_nic_access(trans, &flags);
/* Set interrupt coalescing timer to default (2048 usecs) */
iwl_write8(trans, CSR_INT_COALESCING, IWL_HOST_INT_TIMEOUT_DEF);
@@ -1298,7 +1315,7 @@
* write 1 clear (W1C) register, meaning that it's being clear
* by writing 1 to the bit.
*/
- iwl_write_direct32(trans, CSR_MSIX_AUTOMASK_ST_AD, BIT(entry->entry));
+ iwl_write32(trans, CSR_MSIX_AUTOMASK_ST_AD, BIT(entry->entry));
}
/*
@@ -1817,13 +1834,13 @@
lock_map_acquire(&trans->sync_cmd_lockdep_map);
spin_lock(&trans_pcie->irq_lock);
- inta_fh = iwl_read_direct32(trans, CSR_MSIX_FH_INT_CAUSES_AD);
- inta_hw = iwl_read_direct32(trans, CSR_MSIX_HW_INT_CAUSES_AD);
+ inta_fh = iwl_read32(trans, CSR_MSIX_FH_INT_CAUSES_AD);
+ inta_hw = iwl_read32(trans, CSR_MSIX_HW_INT_CAUSES_AD);
/*
* Clear causes registers to avoid being handling the same cause.
*/
- iwl_write_direct32(trans, CSR_MSIX_FH_INT_CAUSES_AD, inta_fh);
- iwl_write_direct32(trans, CSR_MSIX_HW_INT_CAUSES_AD, inta_hw);
+ iwl_write32(trans, CSR_MSIX_FH_INT_CAUSES_AD, inta_fh);
+ iwl_write32(trans, CSR_MSIX_HW_INT_CAUSES_AD, inta_hw);
spin_unlock(&trans_pcie->irq_lock);
if (unlikely(!(inta_fh | inta_hw))) {
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/trans.c b/drivers/net/wireless/intel/iwlwifi/pcie/trans.c
index ee081c2..f603d78 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/trans.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/trans.c
@@ -269,9 +269,8 @@
iwl_pcie_apm_config(trans);
/* Configure analog phase-lock-loop before activating to D0A */
- if (trans->cfg->base_params->pll_cfg_val)
- iwl_set_bit(trans, CSR_ANA_PLL_CFG,
- trans->cfg->base_params->pll_cfg_val);
+ if (trans->cfg->base_params->pll_cfg)
+ iwl_set_bit(trans, CSR_ANA_PLL_CFG, CSR50_ANA_PLL_CFG_VAL);
/*
* Set "initialization complete" bit to move adapter from
@@ -361,8 +360,7 @@
/* Reset entire device - do controller reset (results in SHRD_HW_RST) */
iwl_set_bit(trans, CSR_RESET, CSR_RESET_REG_FLAG_SW_RESET);
-
- udelay(10);
+ usleep_range(1000, 2000);
/*
* Set "initialization complete" bit to move adapter from
@@ -408,8 +406,7 @@
* SHRD_HW_RST). Turn MAC off before proceeding.
*/
iwl_set_bit(trans, CSR_RESET, CSR_RESET_REG_FLAG_SW_RESET);
-
- udelay(10);
+ usleep_range(1000, 2000);
/* Enable LP XTAL by indirect access through CSR */
apmg_gp1_reg = iwl_trans_pcie_read_shr(trans, SHR_APMG_GP1_REG);
@@ -506,8 +503,7 @@
/* Reset the entire device */
iwl_set_bit(trans, CSR_RESET, CSR_RESET_REG_FLAG_SW_RESET);
-
- udelay(10);
+ usleep_range(1000, 2000);
/*
* Clear "initialization complete" bit to move adapter from
@@ -586,7 +582,7 @@
iwl_set_bit(trans, CSR_DBG_LINK_PWR_MGMT_REG,
CSR_RESET_LINK_PWR_MGMT_DISABLED);
- msleep(1);
+ usleep_range(1000, 2000);
for (iter = 0; iter < 10; iter++) {
/* If HW is not ready, prepare the conditions to check again */
@@ -1074,7 +1070,7 @@
/* stop and reset the on-board processor */
iwl_write32(trans, CSR_RESET, CSR_RESET_REG_FLAG_SW_RESET);
- udelay(20);
+ usleep_range(1000, 2000);
/*
* Upon stop, the APM issues an interrupt if HW RF kill is set.
@@ -1526,8 +1522,7 @@
/* Reset the entire device */
iwl_write32(trans, CSR_RESET, CSR_RESET_REG_FLAG_SW_RESET);
-
- usleep_range(10, 15);
+ usleep_range(1000, 2000);
iwl_pcie_apm_init(trans);
@@ -1950,7 +1945,7 @@
"WR pointer moved while flushing %d -> %d\n",
wr_ptr, write_ptr))
return -ETIMEDOUT;
- msleep(1);
+ usleep_range(1000, 2000);
}
if (q->read_ptr != q->write_ptr) {
@@ -2013,7 +2008,7 @@
spin_unlock_irqrestore(&trans_pcie->reg_lock, flags);
}
-void iwl_trans_pcie_ref(struct iwl_trans *trans)
+static void iwl_trans_pcie_ref(struct iwl_trans *trans)
{
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
@@ -2028,7 +2023,7 @@
#endif /* CONFIG_PM */
}
-void iwl_trans_pcie_unref(struct iwl_trans *trans)
+static void iwl_trans_pcie_unref(struct iwl_trans *trans)
{
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
@@ -2907,6 +2902,8 @@
}
}
+ trans->hw_rf_id = iwl_read32(trans, CSR_HW_RF_ID);
+
iwl_pcie_set_interrupt_capa(pdev, trans);
trans->hw_id = (pdev->device << 16) + pdev->subsystem_device;
snprintf(trans->hw_id_str, sizeof(trans->hw_id_str),
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/tx.c b/drivers/net/wireless/intel/iwlwifi/pcie/tx.c
index e1f7a3f..d6beac9 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/tx.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/tx.c
@@ -32,6 +32,7 @@
#include <linux/ieee80211.h>
#include <linux/slab.h>
#include <linux/sched.h>
+#include <linux/pm_runtime.h>
#include <net/ip6_checksum.h>
#include <net/tso.h>
@@ -605,7 +606,7 @@
if (trans_pcie->ref_cmd_in_flight) {
trans_pcie->ref_cmd_in_flight = false;
IWL_DEBUG_RPM(trans, "clear ref_cmd_in_flight - unref\n");
- iwl_trans_pcie_unref(trans);
+ iwl_trans_unref(trans);
}
if (!trans->cfg->base_params->apmg_wake_up_wa)
@@ -650,7 +651,7 @@
if (txq_id != trans_pcie->cmd_queue) {
IWL_DEBUG_RPM(trans, "Q %d - last tx freed\n",
q->id);
- iwl_trans_pcie_unref(trans);
+ iwl_trans_unref(trans);
} else {
iwl_pcie_clear_cmd_in_flight(trans);
}
@@ -1134,7 +1135,7 @@
if (q->read_ptr == q->write_ptr) {
IWL_DEBUG_RPM(trans, "Q %d - last tx reclaimed\n", q->id);
- iwl_trans_pcie_unref(trans);
+ iwl_trans_unref(trans);
}
out:
@@ -1153,7 +1154,7 @@
!trans_pcie->ref_cmd_in_flight) {
trans_pcie->ref_cmd_in_flight = true;
IWL_DEBUG_RPM(trans, "set ref_cmd_in_flight - ref\n");
- iwl_trans_pcie_ref(trans);
+ iwl_trans_ref(trans);
}
/*
@@ -1799,6 +1800,16 @@
IWL_DEBUG_INFO(trans, "Setting HCMD_ACTIVE for command %s\n",
iwl_get_cmd_string(trans, cmd->id));
+ if (pm_runtime_suspended(&trans_pcie->pci_dev->dev)) {
+ ret = wait_event_timeout(trans_pcie->d0i3_waitq,
+ pm_runtime_active(&trans_pcie->pci_dev->dev),
+ msecs_to_jiffies(IWL_TRANS_IDLE_TIMEOUT));
+ if (!ret) {
+ IWL_ERR(trans, "Timeout exiting D0i3 before hcmd\n");
+ return -ETIMEDOUT;
+ }
+ }
+
cmd_idx = iwl_pcie_enqueue_hcmd(trans, cmd);
if (cmd_idx < 0) {
ret = cmd_idx;
@@ -2362,7 +2373,7 @@
txq->frozen_expiry_remainder = txq->wd_timeout;
}
IWL_DEBUG_RPM(trans, "Q: %d first tx - take ref\n", q->id);
- iwl_trans_pcie_ref(trans);
+ iwl_trans_ref(trans);
}
/* Tell device the write index *just past* this latest filled TFD */
diff --git a/drivers/net/wireless/marvell/mwifiex/pcie.h b/drivers/net/wireless/marvell/mwifiex/pcie.h
index 5770b43..2592e63 100644
--- a/drivers/net/wireless/marvell/mwifiex/pcie.h
+++ b/drivers/net/wireless/marvell/mwifiex/pcie.h
@@ -23,7 +23,6 @@
#define _MWIFIEX_PCIE_H
#include <linux/pci.h>
-#include <linux/pcieport_if.h>
#include <linux/interrupt.h>
#include "decl.h"
@@ -117,7 +116,7 @@
/* FW awake cookie after FW ready */
#define FW_AWAKE_COOKIE (0xAA55AA55)
#define MWIFIEX_DEF_SLEEP_COOKIE 0xBEEFBEEF
-#define MWIFIEX_MAX_DELAY_COUNT 5
+#define MWIFIEX_MAX_DELAY_COUNT 100
struct mwifiex_pcie_card_reg {
u16 cmd_addr_lo;
diff --git a/drivers/net/wireless/marvell/mwifiex/sdio.c b/drivers/net/wireless/marvell/mwifiex/sdio.c
index 099722e..bdc51ff 100644
--- a/drivers/net/wireless/marvell/mwifiex/sdio.c
+++ b/drivers/net/wireless/marvell/mwifiex/sdio.c
@@ -104,7 +104,7 @@
if (!dev->of_node ||
!of_match_node(mwifiex_sdio_of_match_table, dev->of_node)) {
- pr_err("sdio platform data not available");
+ dev_err(dev, "sdio platform data not available\n");
return -1;
}
@@ -115,7 +115,8 @@
if (cfg && card->plt_of_node) {
cfg->irq_wifi = irq_of_parse_and_map(card->plt_of_node, 0);
if (!cfg->irq_wifi) {
- dev_err(dev, "fail to parse irq_wifi from device tree");
+ dev_err(dev,
+ "fail to parse irq_wifi from device tree\n");
} else {
ret = devm_request_irq(dev, cfg->irq_wifi,
mwifiex_wake_irq_wifi,
diff --git a/drivers/net/wireless/realtek/rtl818x/rtl8187/rtl8187.h b/drivers/net/wireless/realtek/rtl818x/rtl8187/rtl8187.h
index a6ad79f..324451d 100644
--- a/drivers/net/wireless/realtek/rtl818x/rtl8187/rtl8187.h
+++ b/drivers/net/wireless/realtek/rtl818x/rtl8187/rtl8187.h
@@ -160,104 +160,40 @@
void rtl8187_write_phy(struct ieee80211_hw *dev, u8 addr, u32 data);
-static inline u8 rtl818x_ioread8_idx(struct rtl8187_priv *priv,
- u8 *addr, u8 idx)
-{
- u8 val;
-
- mutex_lock(&priv->io_mutex);
- usb_control_msg(priv->udev, usb_rcvctrlpipe(priv->udev, 0),
- RTL8187_REQ_GET_REG, RTL8187_REQT_READ,
- (unsigned long)addr, idx & 0x03,
- &priv->io_dmabuf->bits8, sizeof(val), HZ / 2);
-
- val = priv->io_dmabuf->bits8;
- mutex_unlock(&priv->io_mutex);
-
- return val;
-}
+u8 rtl818x_ioread8_idx(struct rtl8187_priv *priv,
+ u8 *addr, u8 idx);
static inline u8 rtl818x_ioread8(struct rtl8187_priv *priv, u8 *addr)
{
return rtl818x_ioread8_idx(priv, addr, 0);
}
-static inline u16 rtl818x_ioread16_idx(struct rtl8187_priv *priv,
- __le16 *addr, u8 idx)
-{
- __le16 val;
-
- mutex_lock(&priv->io_mutex);
- usb_control_msg(priv->udev, usb_rcvctrlpipe(priv->udev, 0),
- RTL8187_REQ_GET_REG, RTL8187_REQT_READ,
- (unsigned long)addr, idx & 0x03,
- &priv->io_dmabuf->bits16, sizeof(val), HZ / 2);
-
- val = priv->io_dmabuf->bits16;
- mutex_unlock(&priv->io_mutex);
-
- return le16_to_cpu(val);
-}
+u16 rtl818x_ioread16_idx(struct rtl8187_priv *priv,
+ __le16 *addr, u8 idx);
static inline u16 rtl818x_ioread16(struct rtl8187_priv *priv, __le16 *addr)
{
return rtl818x_ioread16_idx(priv, addr, 0);
}
-static inline u32 rtl818x_ioread32_idx(struct rtl8187_priv *priv,
- __le32 *addr, u8 idx)
-{
- __le32 val;
-
- mutex_lock(&priv->io_mutex);
- usb_control_msg(priv->udev, usb_rcvctrlpipe(priv->udev, 0),
- RTL8187_REQ_GET_REG, RTL8187_REQT_READ,
- (unsigned long)addr, idx & 0x03,
- &priv->io_dmabuf->bits32, sizeof(val), HZ / 2);
-
- val = priv->io_dmabuf->bits32;
- mutex_unlock(&priv->io_mutex);
-
- return le32_to_cpu(val);
-}
+u32 rtl818x_ioread32_idx(struct rtl8187_priv *priv,
+ __le32 *addr, u8 idx);
static inline u32 rtl818x_ioread32(struct rtl8187_priv *priv, __le32 *addr)
{
return rtl818x_ioread32_idx(priv, addr, 0);
}
-static inline void rtl818x_iowrite8_idx(struct rtl8187_priv *priv,
- u8 *addr, u8 val, u8 idx)
-{
- mutex_lock(&priv->io_mutex);
-
- priv->io_dmabuf->bits8 = val;
- usb_control_msg(priv->udev, usb_sndctrlpipe(priv->udev, 0),
- RTL8187_REQ_SET_REG, RTL8187_REQT_WRITE,
- (unsigned long)addr, idx & 0x03,
- &priv->io_dmabuf->bits8, sizeof(val), HZ / 2);
-
- mutex_unlock(&priv->io_mutex);
-}
+void rtl818x_iowrite8_idx(struct rtl8187_priv *priv,
+ u8 *addr, u8 val, u8 idx);
static inline void rtl818x_iowrite8(struct rtl8187_priv *priv, u8 *addr, u8 val)
{
rtl818x_iowrite8_idx(priv, addr, val, 0);
}
-static inline void rtl818x_iowrite16_idx(struct rtl8187_priv *priv,
- __le16 *addr, u16 val, u8 idx)
-{
- mutex_lock(&priv->io_mutex);
-
- priv->io_dmabuf->bits16 = cpu_to_le16(val);
- usb_control_msg(priv->udev, usb_sndctrlpipe(priv->udev, 0),
- RTL8187_REQ_SET_REG, RTL8187_REQT_WRITE,
- (unsigned long)addr, idx & 0x03,
- &priv->io_dmabuf->bits16, sizeof(val), HZ / 2);
-
- mutex_unlock(&priv->io_mutex);
-}
+void rtl818x_iowrite16_idx(struct rtl8187_priv *priv,
+ __le16 *addr, u16 val, u8 idx);
static inline void rtl818x_iowrite16(struct rtl8187_priv *priv, __le16 *addr,
u16 val)
@@ -265,19 +201,8 @@
rtl818x_iowrite16_idx(priv, addr, val, 0);
}
-static inline void rtl818x_iowrite32_idx(struct rtl8187_priv *priv,
- __le32 *addr, u32 val, u8 idx)
-{
- mutex_lock(&priv->io_mutex);
-
- priv->io_dmabuf->bits32 = cpu_to_le32(val);
- usb_control_msg(priv->udev, usb_sndctrlpipe(priv->udev, 0),
- RTL8187_REQ_SET_REG, RTL8187_REQT_WRITE,
- (unsigned long)addr, idx & 0x03,
- &priv->io_dmabuf->bits32, sizeof(val), HZ / 2);
-
- mutex_unlock(&priv->io_mutex);
-}
+void rtl818x_iowrite32_idx(struct rtl8187_priv *priv,
+ __le32 *addr, u32 val, u8 idx);
static inline void rtl818x_iowrite32(struct rtl8187_priv *priv, __le32 *addr,
u32 val)
diff --git a/drivers/net/wireless/realtek/rtl818x/rtl8187/rtl8225.c b/drivers/net/wireless/realtek/rtl818x/rtl8187/rtl8225.c
index 5ecf18e..e6668ff 100644
--- a/drivers/net/wireless/realtek/rtl818x/rtl8187/rtl8225.c
+++ b/drivers/net/wireless/realtek/rtl818x/rtl8187/rtl8225.c
@@ -22,6 +22,99 @@
#include "rtl8187.h"
#include "rtl8225.h"
+u8 rtl818x_ioread8_idx(struct rtl8187_priv *priv,
+ u8 *addr, u8 idx)
+{
+ u8 val;
+
+ mutex_lock(&priv->io_mutex);
+ usb_control_msg(priv->udev, usb_rcvctrlpipe(priv->udev, 0),
+ RTL8187_REQ_GET_REG, RTL8187_REQT_READ,
+ (unsigned long)addr, idx & 0x03,
+ &priv->io_dmabuf->bits8, sizeof(val), HZ / 2);
+
+ val = priv->io_dmabuf->bits8;
+ mutex_unlock(&priv->io_mutex);
+
+ return val;
+}
+
+u16 rtl818x_ioread16_idx(struct rtl8187_priv *priv,
+ __le16 *addr, u8 idx)
+{
+ __le16 val;
+
+ mutex_lock(&priv->io_mutex);
+ usb_control_msg(priv->udev, usb_rcvctrlpipe(priv->udev, 0),
+ RTL8187_REQ_GET_REG, RTL8187_REQT_READ,
+ (unsigned long)addr, idx & 0x03,
+ &priv->io_dmabuf->bits16, sizeof(val), HZ / 2);
+
+ val = priv->io_dmabuf->bits16;
+ mutex_unlock(&priv->io_mutex);
+
+ return le16_to_cpu(val);
+}
+
+u32 rtl818x_ioread32_idx(struct rtl8187_priv *priv,
+ __le32 *addr, u8 idx)
+{
+ __le32 val;
+
+ mutex_lock(&priv->io_mutex);
+ usb_control_msg(priv->udev, usb_rcvctrlpipe(priv->udev, 0),
+ RTL8187_REQ_GET_REG, RTL8187_REQT_READ,
+ (unsigned long)addr, idx & 0x03,
+ &priv->io_dmabuf->bits32, sizeof(val), HZ / 2);
+
+ val = priv->io_dmabuf->bits32;
+ mutex_unlock(&priv->io_mutex);
+
+ return le32_to_cpu(val);
+}
+
+void rtl818x_iowrite8_idx(struct rtl8187_priv *priv,
+ u8 *addr, u8 val, u8 idx)
+{
+ mutex_lock(&priv->io_mutex);
+
+ priv->io_dmabuf->bits8 = val;
+ usb_control_msg(priv->udev, usb_sndctrlpipe(priv->udev, 0),
+ RTL8187_REQ_SET_REG, RTL8187_REQT_WRITE,
+ (unsigned long)addr, idx & 0x03,
+ &priv->io_dmabuf->bits8, sizeof(val), HZ / 2);
+
+ mutex_unlock(&priv->io_mutex);
+}
+
+void rtl818x_iowrite16_idx(struct rtl8187_priv *priv,
+ __le16 *addr, u16 val, u8 idx)
+{
+ mutex_lock(&priv->io_mutex);
+
+ priv->io_dmabuf->bits16 = cpu_to_le16(val);
+ usb_control_msg(priv->udev, usb_sndctrlpipe(priv->udev, 0),
+ RTL8187_REQ_SET_REG, RTL8187_REQT_WRITE,
+ (unsigned long)addr, idx & 0x03,
+ &priv->io_dmabuf->bits16, sizeof(val), HZ / 2);
+
+ mutex_unlock(&priv->io_mutex);
+}
+
+void rtl818x_iowrite32_idx(struct rtl8187_priv *priv,
+ __le32 *addr, u32 val, u8 idx)
+{
+ mutex_lock(&priv->io_mutex);
+
+ priv->io_dmabuf->bits32 = cpu_to_le32(val);
+ usb_control_msg(priv->udev, usb_sndctrlpipe(priv->udev, 0),
+ RTL8187_REQ_SET_REG, RTL8187_REQT_WRITE,
+ (unsigned long)addr, idx & 0x03,
+ &priv->io_dmabuf->bits32, sizeof(val), HZ / 2);
+
+ mutex_unlock(&priv->io_mutex);
+}
+
static void rtl8225_write_bitbang(struct ieee80211_hw *dev, u8 addr, u16 data)
{
struct rtl8187_priv *priv = dev->priv;
diff --git a/drivers/net/wireless/realtek/rtl8xxxu/Makefile b/drivers/net/wireless/realtek/rtl8xxxu/Makefile
index 5dea3bb..1cf951e 100644
--- a/drivers/net/wireless/realtek/rtl8xxxu/Makefile
+++ b/drivers/net/wireless/realtek/rtl8xxxu/Makefile
@@ -1 +1,4 @@
obj-$(CONFIG_RTL8XXXU) += rtl8xxxu.o
+
+rtl8xxxu-y := rtl8xxxu_core.o rtl8xxxu_8192e.o rtl8xxxu_8723b.o \
+ rtl8xxxu_8723a.o rtl8xxxu_8192c.o
diff --git a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu.c b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu.c
deleted file mode 100644
index f2ce8c9..0000000
--- a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu.c
+++ /dev/null
@@ -1,10213 +0,0 @@
-/*
- * RTL8XXXU mac80211 USB driver
- *
- * Copyright (c) 2014 - 2016 Jes Sorensen <Jes.Sorensen@redhat.com>
- *
- * Portions, notably calibration code:
- * Copyright(c) 2007 - 2011 Realtek Corporation. All rights reserved.
- *
- * This driver was written as a replacement for the vendor provided
- * rtl8723au driver. As the Realtek 8xxx chips are very similar in
- * their programming interface, I have started adding support for
- * additional 8xxx chips like the 8192cu, 8188cus, etc.
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of version 2 of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
- * more details.
- */
-
-#include <linux/init.h>
-#include <linux/kernel.h>
-#include <linux/sched.h>
-#include <linux/errno.h>
-#include <linux/slab.h>
-#include <linux/module.h>
-#include <linux/spinlock.h>
-#include <linux/list.h>
-#include <linux/usb.h>
-#include <linux/netdevice.h>
-#include <linux/etherdevice.h>
-#include <linux/ethtool.h>
-#include <linux/wireless.h>
-#include <linux/firmware.h>
-#include <linux/moduleparam.h>
-#include <net/mac80211.h>
-#include "rtl8xxxu.h"
-#include "rtl8xxxu_regs.h"
-
-#define DRIVER_NAME "rtl8xxxu"
-
-static int rtl8xxxu_debug = RTL8XXXU_DEBUG_EFUSE;
-static bool rtl8xxxu_ht40_2g;
-
-MODULE_AUTHOR("Jes Sorensen <Jes.Sorensen@redhat.com>");
-MODULE_DESCRIPTION("RTL8XXXu USB mac80211 Wireless LAN Driver");
-MODULE_LICENSE("GPL");
-MODULE_FIRMWARE("rtlwifi/rtl8723aufw_A.bin");
-MODULE_FIRMWARE("rtlwifi/rtl8723aufw_B.bin");
-MODULE_FIRMWARE("rtlwifi/rtl8723aufw_B_NoBT.bin");
-MODULE_FIRMWARE("rtlwifi/rtl8192cufw_A.bin");
-MODULE_FIRMWARE("rtlwifi/rtl8192cufw_B.bin");
-MODULE_FIRMWARE("rtlwifi/rtl8192cufw_TMSC.bin");
-MODULE_FIRMWARE("rtlwifi/rtl8192eu_nic.bin");
-MODULE_FIRMWARE("rtlwifi/rtl8723bu_nic.bin");
-MODULE_FIRMWARE("rtlwifi/rtl8723bu_bt.bin");
-
-module_param_named(debug, rtl8xxxu_debug, int, 0600);
-MODULE_PARM_DESC(debug, "Set debug mask");
-module_param_named(ht40_2g, rtl8xxxu_ht40_2g, bool, 0600);
-MODULE_PARM_DESC(ht40_2g, "Enable HT40 support on the 2.4GHz band");
-
-#define USB_VENDOR_ID_REALTEK 0x0bda
-/* Minimum IEEE80211_MAX_FRAME_LEN */
-#define RTL_RX_BUFFER_SIZE IEEE80211_MAX_FRAME_LEN
-#define RTL8XXXU_RX_URBS 32
-#define RTL8XXXU_RX_URB_PENDING_WATER 8
-#define RTL8XXXU_TX_URBS 64
-#define RTL8XXXU_TX_URB_LOW_WATER 25
-#define RTL8XXXU_TX_URB_HIGH_WATER 32
-
-static int rtl8xxxu_submit_rx_urb(struct rtl8xxxu_priv *priv,
- struct rtl8xxxu_rx_urb *rx_urb);
-
-static struct ieee80211_rate rtl8xxxu_rates[] = {
- { .bitrate = 10, .hw_value = DESC_RATE_1M, .flags = 0 },
- { .bitrate = 20, .hw_value = DESC_RATE_2M, .flags = 0 },
- { .bitrate = 55, .hw_value = DESC_RATE_5_5M, .flags = 0 },
- { .bitrate = 110, .hw_value = DESC_RATE_11M, .flags = 0 },
- { .bitrate = 60, .hw_value = DESC_RATE_6M, .flags = 0 },
- { .bitrate = 90, .hw_value = DESC_RATE_9M, .flags = 0 },
- { .bitrate = 120, .hw_value = DESC_RATE_12M, .flags = 0 },
- { .bitrate = 180, .hw_value = DESC_RATE_18M, .flags = 0 },
- { .bitrate = 240, .hw_value = DESC_RATE_24M, .flags = 0 },
- { .bitrate = 360, .hw_value = DESC_RATE_36M, .flags = 0 },
- { .bitrate = 480, .hw_value = DESC_RATE_48M, .flags = 0 },
- { .bitrate = 540, .hw_value = DESC_RATE_54M, .flags = 0 },
-};
-
-static struct ieee80211_channel rtl8xxxu_channels_2g[] = {
- { .band = NL80211_BAND_2GHZ, .center_freq = 2412,
- .hw_value = 1, .max_power = 30 },
- { .band = NL80211_BAND_2GHZ, .center_freq = 2417,
- .hw_value = 2, .max_power = 30 },
- { .band = NL80211_BAND_2GHZ, .center_freq = 2422,
- .hw_value = 3, .max_power = 30 },
- { .band = NL80211_BAND_2GHZ, .center_freq = 2427,
- .hw_value = 4, .max_power = 30 },
- { .band = NL80211_BAND_2GHZ, .center_freq = 2432,
- .hw_value = 5, .max_power = 30 },
- { .band = NL80211_BAND_2GHZ, .center_freq = 2437,
- .hw_value = 6, .max_power = 30 },
- { .band = NL80211_BAND_2GHZ, .center_freq = 2442,
- .hw_value = 7, .max_power = 30 },
- { .band = NL80211_BAND_2GHZ, .center_freq = 2447,
- .hw_value = 8, .max_power = 30 },
- { .band = NL80211_BAND_2GHZ, .center_freq = 2452,
- .hw_value = 9, .max_power = 30 },
- { .band = NL80211_BAND_2GHZ, .center_freq = 2457,
- .hw_value = 10, .max_power = 30 },
- { .band = NL80211_BAND_2GHZ, .center_freq = 2462,
- .hw_value = 11, .max_power = 30 },
- { .band = NL80211_BAND_2GHZ, .center_freq = 2467,
- .hw_value = 12, .max_power = 30 },
- { .band = NL80211_BAND_2GHZ, .center_freq = 2472,
- .hw_value = 13, .max_power = 30 },
- { .band = NL80211_BAND_2GHZ, .center_freq = 2484,
- .hw_value = 14, .max_power = 30 }
-};
-
-static struct ieee80211_supported_band rtl8xxxu_supported_band = {
- .channels = rtl8xxxu_channels_2g,
- .n_channels = ARRAY_SIZE(rtl8xxxu_channels_2g),
- .bitrates = rtl8xxxu_rates,
- .n_bitrates = ARRAY_SIZE(rtl8xxxu_rates),
-};
-
-static struct rtl8xxxu_reg8val rtl8xxxu_gen1_mac_init_table[] = {
- {0x420, 0x80}, {0x423, 0x00}, {0x430, 0x00}, {0x431, 0x00},
- {0x432, 0x00}, {0x433, 0x01}, {0x434, 0x04}, {0x435, 0x05},
- {0x436, 0x06}, {0x437, 0x07}, {0x438, 0x00}, {0x439, 0x00},
- {0x43a, 0x00}, {0x43b, 0x01}, {0x43c, 0x04}, {0x43d, 0x05},
- {0x43e, 0x06}, {0x43f, 0x07}, {0x440, 0x5d}, {0x441, 0x01},
- {0x442, 0x00}, {0x444, 0x15}, {0x445, 0xf0}, {0x446, 0x0f},
- {0x447, 0x00}, {0x458, 0x41}, {0x459, 0xa8}, {0x45a, 0x72},
- {0x45b, 0xb9}, {0x460, 0x66}, {0x461, 0x66}, {0x462, 0x08},
- {0x463, 0x03}, {0x4c8, 0xff}, {0x4c9, 0x08}, {0x4cc, 0xff},
- {0x4cd, 0xff}, {0x4ce, 0x01}, {0x500, 0x26}, {0x501, 0xa2},
- {0x502, 0x2f}, {0x503, 0x00}, {0x504, 0x28}, {0x505, 0xa3},
- {0x506, 0x5e}, {0x507, 0x00}, {0x508, 0x2b}, {0x509, 0xa4},
- {0x50a, 0x5e}, {0x50b, 0x00}, {0x50c, 0x4f}, {0x50d, 0xa4},
- {0x50e, 0x00}, {0x50f, 0x00}, {0x512, 0x1c}, {0x514, 0x0a},
- {0x515, 0x10}, {0x516, 0x0a}, {0x517, 0x10}, {0x51a, 0x16},
- {0x524, 0x0f}, {0x525, 0x4f}, {0x546, 0x40}, {0x547, 0x00},
- {0x550, 0x10}, {0x551, 0x10}, {0x559, 0x02}, {0x55a, 0x02},
- {0x55d, 0xff}, {0x605, 0x30}, {0x608, 0x0e}, {0x609, 0x2a},
- {0x652, 0x20}, {0x63c, 0x0a}, {0x63d, 0x0a}, {0x63e, 0x0e},
- {0x63f, 0x0e}, {0x66e, 0x05}, {0x700, 0x21}, {0x701, 0x43},
- {0x702, 0x65}, {0x703, 0x87}, {0x708, 0x21}, {0x709, 0x43},
- {0x70a, 0x65}, {0x70b, 0x87}, {0xffff, 0xff},
-};
-
-static struct rtl8xxxu_reg8val rtl8723b_mac_init_table[] = {
- {0x02f, 0x30}, {0x035, 0x00}, {0x039, 0x08}, {0x04e, 0xe0},
- {0x064, 0x00}, {0x067, 0x20}, {0x428, 0x0a}, {0x429, 0x10},
- {0x430, 0x00}, {0x431, 0x00},
- {0x432, 0x00}, {0x433, 0x01}, {0x434, 0x04}, {0x435, 0x05},
- {0x436, 0x07}, {0x437, 0x08}, {0x43c, 0x04}, {0x43d, 0x05},
- {0x43e, 0x07}, {0x43f, 0x08}, {0x440, 0x5d}, {0x441, 0x01},
- {0x442, 0x00}, {0x444, 0x10}, {0x445, 0x00}, {0x446, 0x00},
- {0x447, 0x00}, {0x448, 0x00}, {0x449, 0xf0}, {0x44a, 0x0f},
- {0x44b, 0x3e}, {0x44c, 0x10}, {0x44d, 0x00}, {0x44e, 0x00},
- {0x44f, 0x00}, {0x450, 0x00}, {0x451, 0xf0}, {0x452, 0x0f},
- {0x453, 0x00}, {0x456, 0x5e}, {0x460, 0x66}, {0x461, 0x66},
- {0x4c8, 0xff}, {0x4c9, 0x08}, {0x4cc, 0xff},
- {0x4cd, 0xff}, {0x4ce, 0x01}, {0x500, 0x26}, {0x501, 0xa2},
- {0x502, 0x2f}, {0x503, 0x00}, {0x504, 0x28}, {0x505, 0xa3},
- {0x506, 0x5e}, {0x507, 0x00}, {0x508, 0x2b}, {0x509, 0xa4},
- {0x50a, 0x5e}, {0x50b, 0x00}, {0x50c, 0x4f}, {0x50d, 0xa4},
- {0x50e, 0x00}, {0x50f, 0x00}, {0x512, 0x1c}, {0x514, 0x0a},
- {0x516, 0x0a}, {0x525, 0x4f},
- {0x550, 0x10}, {0x551, 0x10}, {0x559, 0x02}, {0x55c, 0x50},
- {0x55d, 0xff}, {0x605, 0x30}, {0x608, 0x0e}, {0x609, 0x2a},
- {0x620, 0xff}, {0x621, 0xff}, {0x622, 0xff}, {0x623, 0xff},
- {0x624, 0xff}, {0x625, 0xff}, {0x626, 0xff}, {0x627, 0xff},
- {0x638, 0x50}, {0x63c, 0x0a}, {0x63d, 0x0a}, {0x63e, 0x0e},
- {0x63f, 0x0e}, {0x640, 0x40}, {0x642, 0x40}, {0x643, 0x00},
- {0x652, 0xc8}, {0x66e, 0x05}, {0x700, 0x21}, {0x701, 0x43},
- {0x702, 0x65}, {0x703, 0x87}, {0x708, 0x21}, {0x709, 0x43},
- {0x70a, 0x65}, {0x70b, 0x87}, {0x765, 0x18}, {0x76e, 0x04},
- {0xffff, 0xff},
-};
-
-static struct rtl8xxxu_reg8val rtl8192e_mac_init_table[] = {
- {0x011, 0xeb}, {0x012, 0x07}, {0x014, 0x75}, {0x303, 0xa7},
- {0x428, 0x0a}, {0x429, 0x10}, {0x430, 0x00}, {0x431, 0x00},
- {0x432, 0x00}, {0x433, 0x01}, {0x434, 0x04}, {0x435, 0x05},
- {0x436, 0x07}, {0x437, 0x08}, {0x43c, 0x04}, {0x43d, 0x05},
- {0x43e, 0x07}, {0x43f, 0x08}, {0x440, 0x5d}, {0x441, 0x01},
- {0x442, 0x00}, {0x444, 0x10}, {0x445, 0x00}, {0x446, 0x00},
- {0x447, 0x00}, {0x448, 0x00}, {0x449, 0xf0}, {0x44a, 0x0f},
- {0x44b, 0x3e}, {0x44c, 0x10}, {0x44d, 0x00}, {0x44e, 0x00},
- {0x44f, 0x00}, {0x450, 0x00}, {0x451, 0xf0}, {0x452, 0x0f},
- {0x453, 0x00}, {0x456, 0x5e}, {0x460, 0x66}, {0x461, 0x66},
- {0x4c8, 0xff}, {0x4c9, 0x08}, {0x4cc, 0xff}, {0x4cd, 0xff},
- {0x4ce, 0x01}, {0x500, 0x26}, {0x501, 0xa2}, {0x502, 0x2f},
- {0x503, 0x00}, {0x504, 0x28}, {0x505, 0xa3}, {0x506, 0x5e},
- {0x507, 0x00}, {0x508, 0x2b}, {0x509, 0xa4}, {0x50a, 0x5e},
- {0x50b, 0x00}, {0x50c, 0x4f}, {0x50d, 0xa4}, {0x50e, 0x00},
- {0x50f, 0x00}, {0x512, 0x1c}, {0x514, 0x0a}, {0x516, 0x0a},
- {0x525, 0x4f}, {0x540, 0x12}, {0x541, 0x64}, {0x550, 0x10},
- {0x551, 0x10}, {0x559, 0x02}, {0x55c, 0x50}, {0x55d, 0xff},
- {0x605, 0x30}, {0x608, 0x0e}, {0x609, 0x2a}, {0x620, 0xff},
- {0x621, 0xff}, {0x622, 0xff}, {0x623, 0xff}, {0x624, 0xff},
- {0x625, 0xff}, {0x626, 0xff}, {0x627, 0xff}, {0x638, 0x50},
- {0x63c, 0x0a}, {0x63d, 0x0a}, {0x63e, 0x0e}, {0x63f, 0x0e},
- {0x640, 0x40}, {0x642, 0x40}, {0x643, 0x00}, {0x652, 0xc8},
- {0x66e, 0x05}, {0x700, 0x21}, {0x701, 0x43}, {0x702, 0x65},
- {0x703, 0x87}, {0x708, 0x21}, {0x709, 0x43}, {0x70a, 0x65},
- {0x70b, 0x87},
- {0xffff, 0xff},
-};
-
-#ifdef CONFIG_RTL8XXXU_UNTESTED
-static struct rtl8xxxu_power_base rtl8188r_power_base = {
- .reg_0e00 = 0x06080808,
- .reg_0e04 = 0x00040406,
- .reg_0e08 = 0x00000000,
- .reg_086c = 0x00000000,
-
- .reg_0e10 = 0x04060608,
- .reg_0e14 = 0x00020204,
- .reg_0e18 = 0x04060608,
- .reg_0e1c = 0x00020204,
-
- .reg_0830 = 0x06080808,
- .reg_0834 = 0x00040406,
- .reg_0838 = 0x00000000,
- .reg_086c_2 = 0x00000000,
-
- .reg_083c = 0x04060608,
- .reg_0848 = 0x00020204,
- .reg_084c = 0x04060608,
- .reg_0868 = 0x00020204,
-};
-
-static struct rtl8xxxu_power_base rtl8192c_power_base = {
- .reg_0e00 = 0x07090c0c,
- .reg_0e04 = 0x01020405,
- .reg_0e08 = 0x00000000,
- .reg_086c = 0x00000000,
-
- .reg_0e10 = 0x0b0c0c0e,
- .reg_0e14 = 0x01030506,
- .reg_0e18 = 0x0b0c0d0e,
- .reg_0e1c = 0x01030509,
-
- .reg_0830 = 0x07090c0c,
- .reg_0834 = 0x01020405,
- .reg_0838 = 0x00000000,
- .reg_086c_2 = 0x00000000,
-
- .reg_083c = 0x0b0c0d0e,
- .reg_0848 = 0x01030509,
- .reg_084c = 0x0b0c0d0e,
- .reg_0868 = 0x01030509,
-};
-#endif
-
-static struct rtl8xxxu_power_base rtl8723a_power_base = {
- .reg_0e00 = 0x0a0c0c0c,
- .reg_0e04 = 0x02040608,
- .reg_0e08 = 0x00000000,
- .reg_086c = 0x00000000,
-
- .reg_0e10 = 0x0a0c0d0e,
- .reg_0e14 = 0x02040608,
- .reg_0e18 = 0x0a0c0d0e,
- .reg_0e1c = 0x02040608,
-
- .reg_0830 = 0x0a0c0c0c,
- .reg_0834 = 0x02040608,
- .reg_0838 = 0x00000000,
- .reg_086c_2 = 0x00000000,
-
- .reg_083c = 0x0a0c0d0e,
- .reg_0848 = 0x02040608,
- .reg_084c = 0x0a0c0d0e,
- .reg_0868 = 0x02040608,
-};
-
-static struct rtl8xxxu_reg32val rtl8723a_phy_1t_init_table[] = {
- {0x800, 0x80040000}, {0x804, 0x00000003},
- {0x808, 0x0000fc00}, {0x80c, 0x0000000a},
- {0x810, 0x10001331}, {0x814, 0x020c3d10},
- {0x818, 0x02200385}, {0x81c, 0x00000000},
- {0x820, 0x01000100}, {0x824, 0x00390004},
- {0x828, 0x00000000}, {0x82c, 0x00000000},
- {0x830, 0x00000000}, {0x834, 0x00000000},
- {0x838, 0x00000000}, {0x83c, 0x00000000},
- {0x840, 0x00010000}, {0x844, 0x00000000},
- {0x848, 0x00000000}, {0x84c, 0x00000000},
- {0x850, 0x00000000}, {0x854, 0x00000000},
- {0x858, 0x569a569a}, {0x85c, 0x001b25a4},
- {0x860, 0x66f60110}, {0x864, 0x061f0130},
- {0x868, 0x00000000}, {0x86c, 0x32323200},
- {0x870, 0x07000760}, {0x874, 0x22004000},
- {0x878, 0x00000808}, {0x87c, 0x00000000},
- {0x880, 0xc0083070}, {0x884, 0x000004d5},
- {0x888, 0x00000000}, {0x88c, 0xccc000c0},
- {0x890, 0x00000800}, {0x894, 0xfffffffe},
- {0x898, 0x40302010}, {0x89c, 0x00706050},
- {0x900, 0x00000000}, {0x904, 0x00000023},
- {0x908, 0x00000000}, {0x90c, 0x81121111},
- {0xa00, 0x00d047c8}, {0xa04, 0x80ff000c},
- {0xa08, 0x8c838300}, {0xa0c, 0x2e68120f},
- {0xa10, 0x9500bb78}, {0xa14, 0x11144028},
- {0xa18, 0x00881117}, {0xa1c, 0x89140f00},
- {0xa20, 0x1a1b0000}, {0xa24, 0x090e1317},
- {0xa28, 0x00000204}, {0xa2c, 0x00d30000},
- {0xa70, 0x101fbf00}, {0xa74, 0x00000007},
- {0xa78, 0x00000900},
- {0xc00, 0x48071d40}, {0xc04, 0x03a05611},
- {0xc08, 0x000000e4}, {0xc0c, 0x6c6c6c6c},
- {0xc10, 0x08800000}, {0xc14, 0x40000100},
- {0xc18, 0x08800000}, {0xc1c, 0x40000100},
- {0xc20, 0x00000000}, {0xc24, 0x00000000},
- {0xc28, 0x00000000}, {0xc2c, 0x00000000},
- {0xc30, 0x69e9ac44}, {0xc34, 0x469652af},
- {0xc38, 0x49795994}, {0xc3c, 0x0a97971c},
- {0xc40, 0x1f7c403f}, {0xc44, 0x000100b7},
- {0xc48, 0xec020107}, {0xc4c, 0x007f037f},
- {0xc50, 0x69543420}, {0xc54, 0x43bc0094},
- {0xc58, 0x69543420}, {0xc5c, 0x433c0094},
- {0xc60, 0x00000000}, {0xc64, 0x7112848b},
- {0xc68, 0x47c00bff}, {0xc6c, 0x00000036},
- {0xc70, 0x2c7f000d}, {0xc74, 0x018610db},
- {0xc78, 0x0000001f}, {0xc7c, 0x00b91612},
- {0xc80, 0x40000100}, {0xc84, 0x20f60000},
- {0xc88, 0x40000100}, {0xc8c, 0x20200000},
- {0xc90, 0x00121820}, {0xc94, 0x00000000},
- {0xc98, 0x00121820}, {0xc9c, 0x00007f7f},
- {0xca0, 0x00000000}, {0xca4, 0x00000080},
- {0xca8, 0x00000000}, {0xcac, 0x00000000},
- {0xcb0, 0x00000000}, {0xcb4, 0x00000000},
- {0xcb8, 0x00000000}, {0xcbc, 0x28000000},
- {0xcc0, 0x00000000}, {0xcc4, 0x00000000},
- {0xcc8, 0x00000000}, {0xccc, 0x00000000},
- {0xcd0, 0x00000000}, {0xcd4, 0x00000000},
- {0xcd8, 0x64b22427}, {0xcdc, 0x00766932},
- {0xce0, 0x00222222}, {0xce4, 0x00000000},
- {0xce8, 0x37644302}, {0xcec, 0x2f97d40c},
- {0xd00, 0x00080740}, {0xd04, 0x00020401},
- {0xd08, 0x0000907f}, {0xd0c, 0x20010201},
- {0xd10, 0xa0633333}, {0xd14, 0x3333bc43},
- {0xd18, 0x7a8f5b6b}, {0xd2c, 0xcc979975},
- {0xd30, 0x00000000}, {0xd34, 0x80608000},
- {0xd38, 0x00000000}, {0xd3c, 0x00027293},
- {0xd40, 0x00000000}, {0xd44, 0x00000000},
- {0xd48, 0x00000000}, {0xd4c, 0x00000000},
- {0xd50, 0x6437140a}, {0xd54, 0x00000000},
- {0xd58, 0x00000000}, {0xd5c, 0x30032064},
- {0xd60, 0x4653de68}, {0xd64, 0x04518a3c},
- {0xd68, 0x00002101}, {0xd6c, 0x2a201c16},
- {0xd70, 0x1812362e}, {0xd74, 0x322c2220},
- {0xd78, 0x000e3c24}, {0xe00, 0x2a2a2a2a},
- {0xe04, 0x2a2a2a2a}, {0xe08, 0x03902a2a},
- {0xe10, 0x2a2a2a2a}, {0xe14, 0x2a2a2a2a},
- {0xe18, 0x2a2a2a2a}, {0xe1c, 0x2a2a2a2a},
- {0xe28, 0x00000000}, {0xe30, 0x1000dc1f},
- {0xe34, 0x10008c1f}, {0xe38, 0x02140102},
- {0xe3c, 0x681604c2}, {0xe40, 0x01007c00},
- {0xe44, 0x01004800}, {0xe48, 0xfb000000},
- {0xe4c, 0x000028d1}, {0xe50, 0x1000dc1f},
- {0xe54, 0x10008c1f}, {0xe58, 0x02140102},
- {0xe5c, 0x28160d05}, {0xe60, 0x00000008},
- {0xe68, 0x001b25a4}, {0xe6c, 0x631b25a0},
- {0xe70, 0x631b25a0}, {0xe74, 0x081b25a0},
- {0xe78, 0x081b25a0}, {0xe7c, 0x081b25a0},
- {0xe80, 0x081b25a0}, {0xe84, 0x631b25a0},
- {0xe88, 0x081b25a0}, {0xe8c, 0x631b25a0},
- {0xed0, 0x631b25a0}, {0xed4, 0x631b25a0},
- {0xed8, 0x631b25a0}, {0xedc, 0x001b25a0},
- {0xee0, 0x001b25a0}, {0xeec, 0x6b1b25a0},
- {0xf14, 0x00000003}, {0xf4c, 0x00000000},
- {0xf00, 0x00000300},
- {0xffff, 0xffffffff},
-};
-
-static struct rtl8xxxu_reg32val rtl8723b_phy_1t_init_table[] = {
- {0x800, 0x80040000}, {0x804, 0x00000003},
- {0x808, 0x0000fc00}, {0x80c, 0x0000000a},
- {0x810, 0x10001331}, {0x814, 0x020c3d10},
- {0x818, 0x02200385}, {0x81c, 0x00000000},
- {0x820, 0x01000100}, {0x824, 0x00190204},
- {0x828, 0x00000000}, {0x82c, 0x00000000},
- {0x830, 0x00000000}, {0x834, 0x00000000},
- {0x838, 0x00000000}, {0x83c, 0x00000000},
- {0x840, 0x00010000}, {0x844, 0x00000000},
- {0x848, 0x00000000}, {0x84c, 0x00000000},
- {0x850, 0x00000000}, {0x854, 0x00000000},
- {0x858, 0x569a11a9}, {0x85c, 0x01000014},
- {0x860, 0x66f60110}, {0x864, 0x061f0649},
- {0x868, 0x00000000}, {0x86c, 0x27272700},
- {0x870, 0x07000760}, {0x874, 0x25004000},
- {0x878, 0x00000808}, {0x87c, 0x00000000},
- {0x880, 0xb0000c1c}, {0x884, 0x00000001},
- {0x888, 0x00000000}, {0x88c, 0xccc000c0},
- {0x890, 0x00000800}, {0x894, 0xfffffffe},
- {0x898, 0x40302010}, {0x89c, 0x00706050},
- {0x900, 0x00000000}, {0x904, 0x00000023},
- {0x908, 0x00000000}, {0x90c, 0x81121111},
- {0x910, 0x00000002}, {0x914, 0x00000201},
- {0xa00, 0x00d047c8}, {0xa04, 0x80ff800c},
- {0xa08, 0x8c838300}, {0xa0c, 0x2e7f120f},
- {0xa10, 0x9500bb78}, {0xa14, 0x1114d028},
- {0xa18, 0x00881117}, {0xa1c, 0x89140f00},
- {0xa20, 0x1a1b0000}, {0xa24, 0x090e1317},
- {0xa28, 0x00000204}, {0xa2c, 0x00d30000},
- {0xa70, 0x101fbf00}, {0xa74, 0x00000007},
- {0xa78, 0x00000900}, {0xa7c, 0x225b0606},
- {0xa80, 0x21806490}, {0xb2c, 0x00000000},
- {0xc00, 0x48071d40}, {0xc04, 0x03a05611},
- {0xc08, 0x000000e4}, {0xc0c, 0x6c6c6c6c},
- {0xc10, 0x08800000}, {0xc14, 0x40000100},
- {0xc18, 0x08800000}, {0xc1c, 0x40000100},
- {0xc20, 0x00000000}, {0xc24, 0x00000000},
- {0xc28, 0x00000000}, {0xc2c, 0x00000000},
- {0xc30, 0x69e9ac44}, {0xc34, 0x469652af},
- {0xc38, 0x49795994}, {0xc3c, 0x0a97971c},
- {0xc40, 0x1f7c403f}, {0xc44, 0x000100b7},
- {0xc48, 0xec020107}, {0xc4c, 0x007f037f},
- {0xc50, 0x69553420}, {0xc54, 0x43bc0094},
- {0xc58, 0x00013149}, {0xc5c, 0x00250492},
- {0xc60, 0x00000000}, {0xc64, 0x7112848b},
- {0xc68, 0x47c00bff}, {0xc6c, 0x00000036},
- {0xc70, 0x2c7f000d}, {0xc74, 0x020610db},
- {0xc78, 0x0000001f}, {0xc7c, 0x00b91612},
- {0xc80, 0x390000e4}, {0xc84, 0x20f60000},
- {0xc88, 0x40000100}, {0xc8c, 0x20200000},
- {0xc90, 0x00020e1a}, {0xc94, 0x00000000},
- {0xc98, 0x00020e1a}, {0xc9c, 0x00007f7f},
- {0xca0, 0x00000000}, {0xca4, 0x000300a0},
- {0xca8, 0x00000000}, {0xcac, 0x00000000},
- {0xcb0, 0x00000000}, {0xcb4, 0x00000000},
- {0xcb8, 0x00000000}, {0xcbc, 0x28000000},
- {0xcc0, 0x00000000}, {0xcc4, 0x00000000},
- {0xcc8, 0x00000000}, {0xccc, 0x00000000},
- {0xcd0, 0x00000000}, {0xcd4, 0x00000000},
- {0xcd8, 0x64b22427}, {0xcdc, 0x00766932},
- {0xce0, 0x00222222}, {0xce4, 0x00000000},
- {0xce8, 0x37644302}, {0xcec, 0x2f97d40c},
- {0xd00, 0x00000740}, {0xd04, 0x40020401},
- {0xd08, 0x0000907f}, {0xd0c, 0x20010201},
- {0xd10, 0xa0633333}, {0xd14, 0x3333bc53},
- {0xd18, 0x7a8f5b6f}, {0xd2c, 0xcc979975},
- {0xd30, 0x00000000}, {0xd34, 0x80608000},
- {0xd38, 0x00000000}, {0xd3c, 0x00127353},
- {0xd40, 0x00000000}, {0xd44, 0x00000000},
- {0xd48, 0x00000000}, {0xd4c, 0x00000000},
- {0xd50, 0x6437140a}, {0xd54, 0x00000000},
- {0xd58, 0x00000282}, {0xd5c, 0x30032064},
- {0xd60, 0x4653de68}, {0xd64, 0x04518a3c},
- {0xd68, 0x00002101}, {0xd6c, 0x2a201c16},
- {0xd70, 0x1812362e}, {0xd74, 0x322c2220},
- {0xd78, 0x000e3c24}, {0xe00, 0x2d2d2d2d},
- {0xe04, 0x2d2d2d2d}, {0xe08, 0x0390272d},
- {0xe10, 0x2d2d2d2d}, {0xe14, 0x2d2d2d2d},
- {0xe18, 0x2d2d2d2d}, {0xe1c, 0x2d2d2d2d},
- {0xe28, 0x00000000}, {0xe30, 0x1000dc1f},
- {0xe34, 0x10008c1f}, {0xe38, 0x02140102},
- {0xe3c, 0x681604c2}, {0xe40, 0x01007c00},
- {0xe44, 0x01004800}, {0xe48, 0xfb000000},
- {0xe4c, 0x000028d1}, {0xe50, 0x1000dc1f},
- {0xe54, 0x10008c1f}, {0xe58, 0x02140102},
- {0xe5c, 0x28160d05}, {0xe60, 0x00000008},
- {0xe68, 0x001b2556}, {0xe6c, 0x00c00096},
- {0xe70, 0x00c00096}, {0xe74, 0x01000056},
- {0xe78, 0x01000014}, {0xe7c, 0x01000056},
- {0xe80, 0x01000014}, {0xe84, 0x00c00096},
- {0xe88, 0x01000056}, {0xe8c, 0x00c00096},
- {0xed0, 0x00c00096}, {0xed4, 0x00c00096},
- {0xed8, 0x00c00096}, {0xedc, 0x000000d6},
- {0xee0, 0x000000d6}, {0xeec, 0x01c00016},
- {0xf14, 0x00000003}, {0xf4c, 0x00000000},
- {0xf00, 0x00000300},
- {0x820, 0x01000100}, {0x800, 0x83040000},
- {0xffff, 0xffffffff},
-};
-
-static struct rtl8xxxu_reg32val rtl8192cu_phy_2t_init_table[] = {
- {0x024, 0x0011800f}, {0x028, 0x00ffdb83},
- {0x800, 0x80040002}, {0x804, 0x00000003},
- {0x808, 0x0000fc00}, {0x80c, 0x0000000a},
- {0x810, 0x10000330}, {0x814, 0x020c3d10},
- {0x818, 0x02200385}, {0x81c, 0x00000000},
- {0x820, 0x01000100}, {0x824, 0x00390004},
- {0x828, 0x01000100}, {0x82c, 0x00390004},
- {0x830, 0x27272727}, {0x834, 0x27272727},
- {0x838, 0x27272727}, {0x83c, 0x27272727},
- {0x840, 0x00010000}, {0x844, 0x00010000},
- {0x848, 0x27272727}, {0x84c, 0x27272727},
- {0x850, 0x00000000}, {0x854, 0x00000000},
- {0x858, 0x569a569a}, {0x85c, 0x0c1b25a4},
- {0x860, 0x66e60230}, {0x864, 0x061f0130},
- {0x868, 0x27272727}, {0x86c, 0x2b2b2b27},
- {0x870, 0x07000700}, {0x874, 0x22184000},
- {0x878, 0x08080808}, {0x87c, 0x00000000},
- {0x880, 0xc0083070}, {0x884, 0x000004d5},
- {0x888, 0x00000000}, {0x88c, 0xcc0000c0},
- {0x890, 0x00000800}, {0x894, 0xfffffffe},
- {0x898, 0x40302010}, {0x89c, 0x00706050},
- {0x900, 0x00000000}, {0x904, 0x00000023},
- {0x908, 0x00000000}, {0x90c, 0x81121313},
- {0xa00, 0x00d047c8}, {0xa04, 0x80ff000c},
- {0xa08, 0x8c838300}, {0xa0c, 0x2e68120f},
- {0xa10, 0x9500bb78}, {0xa14, 0x11144028},
- {0xa18, 0x00881117}, {0xa1c, 0x89140f00},
- {0xa20, 0x1a1b0000}, {0xa24, 0x090e1317},
- {0xa28, 0x00000204}, {0xa2c, 0x00d30000},
- {0xa70, 0x101fbf00}, {0xa74, 0x00000007},
- {0xc00, 0x48071d40}, {0xc04, 0x03a05633},
- {0xc08, 0x000000e4}, {0xc0c, 0x6c6c6c6c},
- {0xc10, 0x08800000}, {0xc14, 0x40000100},
- {0xc18, 0x08800000}, {0xc1c, 0x40000100},
- {0xc20, 0x00000000}, {0xc24, 0x00000000},
- {0xc28, 0x00000000}, {0xc2c, 0x00000000},
- {0xc30, 0x69e9ac44}, {0xc34, 0x469652cf},
- {0xc38, 0x49795994}, {0xc3c, 0x0a97971c},
- {0xc40, 0x1f7c403f}, {0xc44, 0x000100b7},
- {0xc48, 0xec020107}, {0xc4c, 0x007f037f},
- {0xc50, 0x69543420}, {0xc54, 0x43bc0094},
- {0xc58, 0x69543420}, {0xc5c, 0x433c0094},
- {0xc60, 0x00000000}, {0xc64, 0x5116848b},
- {0xc68, 0x47c00bff}, {0xc6c, 0x00000036},
- {0xc70, 0x2c7f000d}, {0xc74, 0x2186115b},
- {0xc78, 0x0000001f}, {0xc7c, 0x00b99612},
- {0xc80, 0x40000100}, {0xc84, 0x20f60000},
- {0xc88, 0x40000100}, {0xc8c, 0xa0e40000},
- {0xc90, 0x00121820}, {0xc94, 0x00000000},
- {0xc98, 0x00121820}, {0xc9c, 0x00007f7f},
- {0xca0, 0x00000000}, {0xca4, 0x00000080},
- {0xca8, 0x00000000}, {0xcac, 0x00000000},
- {0xcb0, 0x00000000}, {0xcb4, 0x00000000},
- {0xcb8, 0x00000000}, {0xcbc, 0x28000000},
- {0xcc0, 0x00000000}, {0xcc4, 0x00000000},
- {0xcc8, 0x00000000}, {0xccc, 0x00000000},
- {0xcd0, 0x00000000}, {0xcd4, 0x00000000},
- {0xcd8, 0x64b22427}, {0xcdc, 0x00766932},
- {0xce0, 0x00222222}, {0xce4, 0x00000000},
- {0xce8, 0x37644302}, {0xcec, 0x2f97d40c},
- {0xd00, 0x00080740}, {0xd04, 0x00020403},
- {0xd08, 0x0000907f}, {0xd0c, 0x20010201},
- {0xd10, 0xa0633333}, {0xd14, 0x3333bc43},
- {0xd18, 0x7a8f5b6b}, {0xd2c, 0xcc979975},
- {0xd30, 0x00000000}, {0xd34, 0x80608000},
- {0xd38, 0x00000000}, {0xd3c, 0x00027293},
- {0xd40, 0x00000000}, {0xd44, 0x00000000},
- {0xd48, 0x00000000}, {0xd4c, 0x00000000},
- {0xd50, 0x6437140a}, {0xd54, 0x00000000},
- {0xd58, 0x00000000}, {0xd5c, 0x30032064},
- {0xd60, 0x4653de68}, {0xd64, 0x04518a3c},
- {0xd68, 0x00002101}, {0xd6c, 0x2a201c16},
- {0xd70, 0x1812362e}, {0xd74, 0x322c2220},
- {0xd78, 0x000e3c24}, {0xe00, 0x2a2a2a2a},
- {0xe04, 0x2a2a2a2a}, {0xe08, 0x03902a2a},
- {0xe10, 0x2a2a2a2a}, {0xe14, 0x2a2a2a2a},
- {0xe18, 0x2a2a2a2a}, {0xe1c, 0x2a2a2a2a},
- {0xe28, 0x00000000}, {0xe30, 0x1000dc1f},
- {0xe34, 0x10008c1f}, {0xe38, 0x02140102},
- {0xe3c, 0x681604c2}, {0xe40, 0x01007c00},
- {0xe44, 0x01004800}, {0xe48, 0xfb000000},
- {0xe4c, 0x000028d1}, {0xe50, 0x1000dc1f},
- {0xe54, 0x10008c1f}, {0xe58, 0x02140102},
- {0xe5c, 0x28160d05}, {0xe60, 0x00000010},
- {0xe68, 0x001b25a4}, {0xe6c, 0x63db25a4},
- {0xe70, 0x63db25a4}, {0xe74, 0x0c1b25a4},
- {0xe78, 0x0c1b25a4}, {0xe7c, 0x0c1b25a4},
- {0xe80, 0x0c1b25a4}, {0xe84, 0x63db25a4},
- {0xe88, 0x0c1b25a4}, {0xe8c, 0x63db25a4},
- {0xed0, 0x63db25a4}, {0xed4, 0x63db25a4},
- {0xed8, 0x63db25a4}, {0xedc, 0x001b25a4},
- {0xee0, 0x001b25a4}, {0xeec, 0x6fdb25a4},
- {0xf14, 0x00000003}, {0xf4c, 0x00000000},
- {0xf00, 0x00000300},
- {0xffff, 0xffffffff},
-};
-
-static struct rtl8xxxu_reg32val rtl8188ru_phy_1t_highpa_table[] = {
- {0x024, 0x0011800f}, {0x028, 0x00ffdb83},
- {0x040, 0x000c0004}, {0x800, 0x80040000},
- {0x804, 0x00000001}, {0x808, 0x0000fc00},
- {0x80c, 0x0000000a}, {0x810, 0x10005388},
- {0x814, 0x020c3d10}, {0x818, 0x02200385},
- {0x81c, 0x00000000}, {0x820, 0x01000100},
- {0x824, 0x00390204}, {0x828, 0x00000000},
- {0x82c, 0x00000000}, {0x830, 0x00000000},
- {0x834, 0x00000000}, {0x838, 0x00000000},
- {0x83c, 0x00000000}, {0x840, 0x00010000},
- {0x844, 0x00000000}, {0x848, 0x00000000},
- {0x84c, 0x00000000}, {0x850, 0x00000000},
- {0x854, 0x00000000}, {0x858, 0x569a569a},
- {0x85c, 0x001b25a4}, {0x860, 0x66e60230},
- {0x864, 0x061f0130}, {0x868, 0x00000000},
- {0x86c, 0x20202000}, {0x870, 0x03000300},
- {0x874, 0x22004000}, {0x878, 0x00000808},
- {0x87c, 0x00ffc3f1}, {0x880, 0xc0083070},
- {0x884, 0x000004d5}, {0x888, 0x00000000},
- {0x88c, 0xccc000c0}, {0x890, 0x00000800},
- {0x894, 0xfffffffe}, {0x898, 0x40302010},
- {0x89c, 0x00706050}, {0x900, 0x00000000},
- {0x904, 0x00000023}, {0x908, 0x00000000},
- {0x90c, 0x81121111}, {0xa00, 0x00d047c8},
- {0xa04, 0x80ff000c}, {0xa08, 0x8c838300},
- {0xa0c, 0x2e68120f}, {0xa10, 0x9500bb78},
- {0xa14, 0x11144028}, {0xa18, 0x00881117},
- {0xa1c, 0x89140f00}, {0xa20, 0x15160000},
- {0xa24, 0x070b0f12}, {0xa28, 0x00000104},
- {0xa2c, 0x00d30000}, {0xa70, 0x101fbf00},
- {0xa74, 0x00000007}, {0xc00, 0x48071d40},
- {0xc04, 0x03a05611}, {0xc08, 0x000000e4},
- {0xc0c, 0x6c6c6c6c}, {0xc10, 0x08800000},
- {0xc14, 0x40000100}, {0xc18, 0x08800000},
- {0xc1c, 0x40000100}, {0xc20, 0x00000000},
- {0xc24, 0x00000000}, {0xc28, 0x00000000},
- {0xc2c, 0x00000000}, {0xc30, 0x69e9ac44},
- {0xc34, 0x469652cf}, {0xc38, 0x49795994},
- {0xc3c, 0x0a97971c}, {0xc40, 0x1f7c403f},
- {0xc44, 0x000100b7}, {0xc48, 0xec020107},
- {0xc4c, 0x007f037f}, {0xc50, 0x6954342e},
- {0xc54, 0x43bc0094}, {0xc58, 0x6954342f},
- {0xc5c, 0x433c0094}, {0xc60, 0x00000000},
- {0xc64, 0x5116848b}, {0xc68, 0x47c00bff},
- {0xc6c, 0x00000036}, {0xc70, 0x2c46000d},
- {0xc74, 0x018610db}, {0xc78, 0x0000001f},
- {0xc7c, 0x00b91612}, {0xc80, 0x24000090},
- {0xc84, 0x20f60000}, {0xc88, 0x24000090},
- {0xc8c, 0x20200000}, {0xc90, 0x00121820},
- {0xc94, 0x00000000}, {0xc98, 0x00121820},
- {0xc9c, 0x00007f7f}, {0xca0, 0x00000000},
- {0xca4, 0x00000080}, {0xca8, 0x00000000},
- {0xcac, 0x00000000}, {0xcb0, 0x00000000},
- {0xcb4, 0x00000000}, {0xcb8, 0x00000000},
- {0xcbc, 0x28000000}, {0xcc0, 0x00000000},
- {0xcc4, 0x00000000}, {0xcc8, 0x00000000},
- {0xccc, 0x00000000}, {0xcd0, 0x00000000},
- {0xcd4, 0x00000000}, {0xcd8, 0x64b22427},
- {0xcdc, 0x00766932}, {0xce0, 0x00222222},
- {0xce4, 0x00000000}, {0xce8, 0x37644302},
- {0xcec, 0x2f97d40c}, {0xd00, 0x00080740},
- {0xd04, 0x00020401}, {0xd08, 0x0000907f},
- {0xd0c, 0x20010201}, {0xd10, 0xa0633333},
- {0xd14, 0x3333bc43}, {0xd18, 0x7a8f5b6b},
- {0xd2c, 0xcc979975}, {0xd30, 0x00000000},
- {0xd34, 0x80608000}, {0xd38, 0x00000000},
- {0xd3c, 0x00027293}, {0xd40, 0x00000000},
- {0xd44, 0x00000000}, {0xd48, 0x00000000},
- {0xd4c, 0x00000000}, {0xd50, 0x6437140a},
- {0xd54, 0x00000000}, {0xd58, 0x00000000},
- {0xd5c, 0x30032064}, {0xd60, 0x4653de68},
- {0xd64, 0x04518a3c}, {0xd68, 0x00002101},
- {0xd6c, 0x2a201c16}, {0xd70, 0x1812362e},
- {0xd74, 0x322c2220}, {0xd78, 0x000e3c24},
- {0xe00, 0x24242424}, {0xe04, 0x24242424},
- {0xe08, 0x03902024}, {0xe10, 0x24242424},
- {0xe14, 0x24242424}, {0xe18, 0x24242424},
- {0xe1c, 0x24242424}, {0xe28, 0x00000000},
- {0xe30, 0x1000dc1f}, {0xe34, 0x10008c1f},
- {0xe38, 0x02140102}, {0xe3c, 0x681604c2},
- {0xe40, 0x01007c00}, {0xe44, 0x01004800},
- {0xe48, 0xfb000000}, {0xe4c, 0x000028d1},
- {0xe50, 0x1000dc1f}, {0xe54, 0x10008c1f},
- {0xe58, 0x02140102}, {0xe5c, 0x28160d05},
- {0xe60, 0x00000008}, {0xe68, 0x001b25a4},
- {0xe6c, 0x631b25a0}, {0xe70, 0x631b25a0},
- {0xe74, 0x081b25a0}, {0xe78, 0x081b25a0},
- {0xe7c, 0x081b25a0}, {0xe80, 0x081b25a0},
- {0xe84, 0x631b25a0}, {0xe88, 0x081b25a0},
- {0xe8c, 0x631b25a0}, {0xed0, 0x631b25a0},
- {0xed4, 0x631b25a0}, {0xed8, 0x631b25a0},
- {0xedc, 0x001b25a0}, {0xee0, 0x001b25a0},
- {0xeec, 0x6b1b25a0}, {0xee8, 0x31555448},
- {0xf14, 0x00000003}, {0xf4c, 0x00000000},
- {0xf00, 0x00000300},
- {0xffff, 0xffffffff},
-};
-
-static struct rtl8xxxu_reg32val rtl8192eu_phy_init_table[] = {
- {0x800, 0x80040000}, {0x804, 0x00000003},
- {0x808, 0x0000fc00}, {0x80c, 0x0000000a},
- {0x810, 0x10001331}, {0x814, 0x020c3d10},
- {0x818, 0x02220385}, {0x81c, 0x00000000},
- {0x820, 0x01000100}, {0x824, 0x00390204},
- {0x828, 0x01000100}, {0x82c, 0x00390204},
- {0x830, 0x32323232}, {0x834, 0x30303030},
- {0x838, 0x30303030}, {0x83c, 0x30303030},
- {0x840, 0x00010000}, {0x844, 0x00010000},
- {0x848, 0x28282828}, {0x84c, 0x28282828},
- {0x850, 0x00000000}, {0x854, 0x00000000},
- {0x858, 0x009a009a}, {0x85c, 0x01000014},
- {0x860, 0x66f60000}, {0x864, 0x061f0000},
- {0x868, 0x30303030}, {0x86c, 0x30303030},
- {0x870, 0x00000000}, {0x874, 0x55004200},
- {0x878, 0x08080808}, {0x87c, 0x00000000},
- {0x880, 0xb0000c1c}, {0x884, 0x00000001},
- {0x888, 0x00000000}, {0x88c, 0xcc0000c0},
- {0x890, 0x00000800}, {0x894, 0xfffffffe},
- {0x898, 0x40302010}, {0x900, 0x00000000},
- {0x904, 0x00000023}, {0x908, 0x00000000},
- {0x90c, 0x81121313}, {0x910, 0x806c0001},
- {0x914, 0x00000001}, {0x918, 0x00000000},
- {0x91c, 0x00010000}, {0x924, 0x00000001},
- {0x928, 0x00000000}, {0x92c, 0x00000000},
- {0x930, 0x00000000}, {0x934, 0x00000000},
- {0x938, 0x00000000}, {0x93c, 0x00000000},
- {0x940, 0x00000000}, {0x944, 0x00000000},
- {0x94c, 0x00000008}, {0xa00, 0x00d0c7c8},
- {0xa04, 0x81ff000c}, {0xa08, 0x8c838300},
- {0xa0c, 0x2e68120f}, {0xa10, 0x95009b78},
- {0xa14, 0x1114d028}, {0xa18, 0x00881117},
- {0xa1c, 0x89140f00}, {0xa20, 0x1a1b0000},
- {0xa24, 0x090e1317}, {0xa28, 0x00000204},
- {0xa2c, 0x00d30000}, {0xa70, 0x101fff00},
- {0xa74, 0x00000007}, {0xa78, 0x00000900},
- {0xa7c, 0x225b0606}, {0xa80, 0x218075b1},
- {0xb38, 0x00000000}, {0xc00, 0x48071d40},
- {0xc04, 0x03a05633}, {0xc08, 0x000000e4},
- {0xc0c, 0x6c6c6c6c}, {0xc10, 0x08800000},
- {0xc14, 0x40000100}, {0xc18, 0x08800000},
- {0xc1c, 0x40000100}, {0xc20, 0x00000000},
- {0xc24, 0x00000000}, {0xc28, 0x00000000},
- {0xc2c, 0x00000000}, {0xc30, 0x69e9ac47},
- {0xc34, 0x469652af}, {0xc38, 0x49795994},
- {0xc3c, 0x0a97971c}, {0xc40, 0x1f7c403f},
- {0xc44, 0x000100b7}, {0xc48, 0xec020107},
- {0xc4c, 0x007f037f},
-#ifdef EXT_PA_8192EU
- /* External PA or external LNA */
- {0xc50, 0x00340220},
-#else
- {0xc50, 0x00340020},
-#endif
- {0xc54, 0x0080801f},
-#ifdef EXT_PA_8192EU
- /* External PA or external LNA */
- {0xc58, 0x00000220},
-#else
- {0xc58, 0x00000020},
-#endif
- {0xc5c, 0x00248492}, {0xc60, 0x00000000},
- {0xc64, 0x7112848b}, {0xc68, 0x47c00bff},
- {0xc6c, 0x00000036}, {0xc70, 0x00000600},
- {0xc74, 0x02013169}, {0xc78, 0x0000001f},
- {0xc7c, 0x00b91612},
-#ifdef EXT_PA_8192EU
- /* External PA or external LNA */
- {0xc80, 0x2d4000b5},
-#else
- {0xc80, 0x40000100},
-#endif
- {0xc84, 0x21f60000},
-#ifdef EXT_PA_8192EU
- /* External PA or external LNA */
- {0xc88, 0x2d4000b5},
-#else
- {0xc88, 0x40000100},
-#endif
- {0xc8c, 0xa0e40000}, {0xc90, 0x00121820},
- {0xc94, 0x00000000}, {0xc98, 0x00121820},
- {0xc9c, 0x00007f7f}, {0xca0, 0x00000000},
- {0xca4, 0x000300a0}, {0xca8, 0x00000000},
- {0xcac, 0x00000000}, {0xcb0, 0x00000000},
- {0xcb4, 0x00000000}, {0xcb8, 0x00000000},
- {0xcbc, 0x28000000}, {0xcc0, 0x00000000},
- {0xcc4, 0x00000000}, {0xcc8, 0x00000000},
- {0xccc, 0x00000000}, {0xcd0, 0x00000000},
- {0xcd4, 0x00000000}, {0xcd8, 0x64b22427},
- {0xcdc, 0x00766932}, {0xce0, 0x00222222},
- {0xce4, 0x00040000}, {0xce8, 0x77644302},
- {0xcec, 0x2f97d40c}, {0xd00, 0x00080740},
- {0xd04, 0x00020403}, {0xd08, 0x0000907f},
- {0xd0c, 0x20010201}, {0xd10, 0xa0633333},
- {0xd14, 0x3333bc43}, {0xd18, 0x7a8f5b6b},
- {0xd1c, 0x0000007f}, {0xd2c, 0xcc979975},
- {0xd30, 0x00000000}, {0xd34, 0x80608000},
- {0xd38, 0x00000000}, {0xd3c, 0x00127353},
- {0xd40, 0x00000000}, {0xd44, 0x00000000},
- {0xd48, 0x00000000}, {0xd4c, 0x00000000},
- {0xd50, 0x6437140a}, {0xd54, 0x00000000},
- {0xd58, 0x00000282}, {0xd5c, 0x30032064},
- {0xd60, 0x4653de68}, {0xd64, 0x04518a3c},
- {0xd68, 0x00002101}, {0xd6c, 0x2a201c16},
- {0xd70, 0x1812362e}, {0xd74, 0x322c2220},
- {0xd78, 0x000e3c24}, {0xd80, 0x01081008},
- {0xd84, 0x00000800}, {0xd88, 0xf0b50000},
- {0xe00, 0x30303030}, {0xe04, 0x30303030},
- {0xe08, 0x03903030}, {0xe10, 0x30303030},
- {0xe14, 0x30303030}, {0xe18, 0x30303030},
- {0xe1c, 0x30303030}, {0xe28, 0x00000000},
- {0xe30, 0x1000dc1f}, {0xe34, 0x10008c1f},
- {0xe38, 0x02140102}, {0xe3c, 0x681604c2},
- {0xe40, 0x01007c00}, {0xe44, 0x01004800},
- {0xe48, 0xfb000000}, {0xe4c, 0x000028d1},
- {0xe50, 0x1000dc1f}, {0xe54, 0x10008c1f},
- {0xe58, 0x02140102}, {0xe5c, 0x28160d05},
- {0xe60, 0x00000008}, {0xe68, 0x0fc05656},
- {0xe6c, 0x03c09696}, {0xe70, 0x03c09696},
- {0xe74, 0x0c005656}, {0xe78, 0x0c005656},
- {0xe7c, 0x0c005656}, {0xe80, 0x0c005656},
- {0xe84, 0x03c09696}, {0xe88, 0x0c005656},
- {0xe8c, 0x03c09696}, {0xed0, 0x03c09696},
- {0xed4, 0x03c09696}, {0xed8, 0x03c09696},
- {0xedc, 0x0000d6d6}, {0xee0, 0x0000d6d6},
- {0xeec, 0x0fc01616}, {0xee4, 0xb0000c1c},
- {0xee8, 0x00000001}, {0xf14, 0x00000003},
- {0xf4c, 0x00000000}, {0xf00, 0x00000300},
- {0xffff, 0xffffffff},
-};
-
-static struct rtl8xxxu_reg32val rtl8xxx_agc_standard_table[] = {
- {0xc78, 0x7b000001}, {0xc78, 0x7b010001},
- {0xc78, 0x7b020001}, {0xc78, 0x7b030001},
- {0xc78, 0x7b040001}, {0xc78, 0x7b050001},
- {0xc78, 0x7a060001}, {0xc78, 0x79070001},
- {0xc78, 0x78080001}, {0xc78, 0x77090001},
- {0xc78, 0x760a0001}, {0xc78, 0x750b0001},
- {0xc78, 0x740c0001}, {0xc78, 0x730d0001},
- {0xc78, 0x720e0001}, {0xc78, 0x710f0001},
- {0xc78, 0x70100001}, {0xc78, 0x6f110001},
- {0xc78, 0x6e120001}, {0xc78, 0x6d130001},
- {0xc78, 0x6c140001}, {0xc78, 0x6b150001},
- {0xc78, 0x6a160001}, {0xc78, 0x69170001},
- {0xc78, 0x68180001}, {0xc78, 0x67190001},
- {0xc78, 0x661a0001}, {0xc78, 0x651b0001},
- {0xc78, 0x641c0001}, {0xc78, 0x631d0001},
- {0xc78, 0x621e0001}, {0xc78, 0x611f0001},
- {0xc78, 0x60200001}, {0xc78, 0x49210001},
- {0xc78, 0x48220001}, {0xc78, 0x47230001},
- {0xc78, 0x46240001}, {0xc78, 0x45250001},
- {0xc78, 0x44260001}, {0xc78, 0x43270001},
- {0xc78, 0x42280001}, {0xc78, 0x41290001},
- {0xc78, 0x402a0001}, {0xc78, 0x262b0001},
- {0xc78, 0x252c0001}, {0xc78, 0x242d0001},
- {0xc78, 0x232e0001}, {0xc78, 0x222f0001},
- {0xc78, 0x21300001}, {0xc78, 0x20310001},
- {0xc78, 0x06320001}, {0xc78, 0x05330001},
- {0xc78, 0x04340001}, {0xc78, 0x03350001},
- {0xc78, 0x02360001}, {0xc78, 0x01370001},
- {0xc78, 0x00380001}, {0xc78, 0x00390001},
- {0xc78, 0x003a0001}, {0xc78, 0x003b0001},
- {0xc78, 0x003c0001}, {0xc78, 0x003d0001},
- {0xc78, 0x003e0001}, {0xc78, 0x003f0001},
- {0xc78, 0x7b400001}, {0xc78, 0x7b410001},
- {0xc78, 0x7b420001}, {0xc78, 0x7b430001},
- {0xc78, 0x7b440001}, {0xc78, 0x7b450001},
- {0xc78, 0x7a460001}, {0xc78, 0x79470001},
- {0xc78, 0x78480001}, {0xc78, 0x77490001},
- {0xc78, 0x764a0001}, {0xc78, 0x754b0001},
- {0xc78, 0x744c0001}, {0xc78, 0x734d0001},
- {0xc78, 0x724e0001}, {0xc78, 0x714f0001},
- {0xc78, 0x70500001}, {0xc78, 0x6f510001},
- {0xc78, 0x6e520001}, {0xc78, 0x6d530001},
- {0xc78, 0x6c540001}, {0xc78, 0x6b550001},
- {0xc78, 0x6a560001}, {0xc78, 0x69570001},
- {0xc78, 0x68580001}, {0xc78, 0x67590001},
- {0xc78, 0x665a0001}, {0xc78, 0x655b0001},
- {0xc78, 0x645c0001}, {0xc78, 0x635d0001},
- {0xc78, 0x625e0001}, {0xc78, 0x615f0001},
- {0xc78, 0x60600001}, {0xc78, 0x49610001},
- {0xc78, 0x48620001}, {0xc78, 0x47630001},
- {0xc78, 0x46640001}, {0xc78, 0x45650001},
- {0xc78, 0x44660001}, {0xc78, 0x43670001},
- {0xc78, 0x42680001}, {0xc78, 0x41690001},
- {0xc78, 0x406a0001}, {0xc78, 0x266b0001},
- {0xc78, 0x256c0001}, {0xc78, 0x246d0001},
- {0xc78, 0x236e0001}, {0xc78, 0x226f0001},
- {0xc78, 0x21700001}, {0xc78, 0x20710001},
- {0xc78, 0x06720001}, {0xc78, 0x05730001},
- {0xc78, 0x04740001}, {0xc78, 0x03750001},
- {0xc78, 0x02760001}, {0xc78, 0x01770001},
- {0xc78, 0x00780001}, {0xc78, 0x00790001},
- {0xc78, 0x007a0001}, {0xc78, 0x007b0001},
- {0xc78, 0x007c0001}, {0xc78, 0x007d0001},
- {0xc78, 0x007e0001}, {0xc78, 0x007f0001},
- {0xc78, 0x3800001e}, {0xc78, 0x3801001e},
- {0xc78, 0x3802001e}, {0xc78, 0x3803001e},
- {0xc78, 0x3804001e}, {0xc78, 0x3805001e},
- {0xc78, 0x3806001e}, {0xc78, 0x3807001e},
- {0xc78, 0x3808001e}, {0xc78, 0x3c09001e},
- {0xc78, 0x3e0a001e}, {0xc78, 0x400b001e},
- {0xc78, 0x440c001e}, {0xc78, 0x480d001e},
- {0xc78, 0x4c0e001e}, {0xc78, 0x500f001e},
- {0xc78, 0x5210001e}, {0xc78, 0x5611001e},
- {0xc78, 0x5a12001e}, {0xc78, 0x5e13001e},
- {0xc78, 0x6014001e}, {0xc78, 0x6015001e},
- {0xc78, 0x6016001e}, {0xc78, 0x6217001e},
- {0xc78, 0x6218001e}, {0xc78, 0x6219001e},
- {0xc78, 0x621a001e}, {0xc78, 0x621b001e},
- {0xc78, 0x621c001e}, {0xc78, 0x621d001e},
- {0xc78, 0x621e001e}, {0xc78, 0x621f001e},
- {0xffff, 0xffffffff}
-};
-
-static struct rtl8xxxu_reg32val rtl8xxx_agc_highpa_table[] = {
- {0xc78, 0x7b000001}, {0xc78, 0x7b010001},
- {0xc78, 0x7b020001}, {0xc78, 0x7b030001},
- {0xc78, 0x7b040001}, {0xc78, 0x7b050001},
- {0xc78, 0x7b060001}, {0xc78, 0x7b070001},
- {0xc78, 0x7b080001}, {0xc78, 0x7a090001},
- {0xc78, 0x790a0001}, {0xc78, 0x780b0001},
- {0xc78, 0x770c0001}, {0xc78, 0x760d0001},
- {0xc78, 0x750e0001}, {0xc78, 0x740f0001},
- {0xc78, 0x73100001}, {0xc78, 0x72110001},
- {0xc78, 0x71120001}, {0xc78, 0x70130001},
- {0xc78, 0x6f140001}, {0xc78, 0x6e150001},
- {0xc78, 0x6d160001}, {0xc78, 0x6c170001},
- {0xc78, 0x6b180001}, {0xc78, 0x6a190001},
- {0xc78, 0x691a0001}, {0xc78, 0x681b0001},
- {0xc78, 0x671c0001}, {0xc78, 0x661d0001},
- {0xc78, 0x651e0001}, {0xc78, 0x641f0001},
- {0xc78, 0x63200001}, {0xc78, 0x62210001},
- {0xc78, 0x61220001}, {0xc78, 0x60230001},
- {0xc78, 0x46240001}, {0xc78, 0x45250001},
- {0xc78, 0x44260001}, {0xc78, 0x43270001},
- {0xc78, 0x42280001}, {0xc78, 0x41290001},
- {0xc78, 0x402a0001}, {0xc78, 0x262b0001},
- {0xc78, 0x252c0001}, {0xc78, 0x242d0001},
- {0xc78, 0x232e0001}, {0xc78, 0x222f0001},
- {0xc78, 0x21300001}, {0xc78, 0x20310001},
- {0xc78, 0x06320001}, {0xc78, 0x05330001},
- {0xc78, 0x04340001}, {0xc78, 0x03350001},
- {0xc78, 0x02360001}, {0xc78, 0x01370001},
- {0xc78, 0x00380001}, {0xc78, 0x00390001},
- {0xc78, 0x003a0001}, {0xc78, 0x003b0001},
- {0xc78, 0x003c0001}, {0xc78, 0x003d0001},
- {0xc78, 0x003e0001}, {0xc78, 0x003f0001},
- {0xc78, 0x7b400001}, {0xc78, 0x7b410001},
- {0xc78, 0x7b420001}, {0xc78, 0x7b430001},
- {0xc78, 0x7b440001}, {0xc78, 0x7b450001},
- {0xc78, 0x7b460001}, {0xc78, 0x7b470001},
- {0xc78, 0x7b480001}, {0xc78, 0x7a490001},
- {0xc78, 0x794a0001}, {0xc78, 0x784b0001},
- {0xc78, 0x774c0001}, {0xc78, 0x764d0001},
- {0xc78, 0x754e0001}, {0xc78, 0x744f0001},
- {0xc78, 0x73500001}, {0xc78, 0x72510001},
- {0xc78, 0x71520001}, {0xc78, 0x70530001},
- {0xc78, 0x6f540001}, {0xc78, 0x6e550001},
- {0xc78, 0x6d560001}, {0xc78, 0x6c570001},
- {0xc78, 0x6b580001}, {0xc78, 0x6a590001},
- {0xc78, 0x695a0001}, {0xc78, 0x685b0001},
- {0xc78, 0x675c0001}, {0xc78, 0x665d0001},
- {0xc78, 0x655e0001}, {0xc78, 0x645f0001},
- {0xc78, 0x63600001}, {0xc78, 0x62610001},
- {0xc78, 0x61620001}, {0xc78, 0x60630001},
- {0xc78, 0x46640001}, {0xc78, 0x45650001},
- {0xc78, 0x44660001}, {0xc78, 0x43670001},
- {0xc78, 0x42680001}, {0xc78, 0x41690001},
- {0xc78, 0x406a0001}, {0xc78, 0x266b0001},
- {0xc78, 0x256c0001}, {0xc78, 0x246d0001},
- {0xc78, 0x236e0001}, {0xc78, 0x226f0001},
- {0xc78, 0x21700001}, {0xc78, 0x20710001},
- {0xc78, 0x06720001}, {0xc78, 0x05730001},
- {0xc78, 0x04740001}, {0xc78, 0x03750001},
- {0xc78, 0x02760001}, {0xc78, 0x01770001},
- {0xc78, 0x00780001}, {0xc78, 0x00790001},
- {0xc78, 0x007a0001}, {0xc78, 0x007b0001},
- {0xc78, 0x007c0001}, {0xc78, 0x007d0001},
- {0xc78, 0x007e0001}, {0xc78, 0x007f0001},
- {0xc78, 0x3800001e}, {0xc78, 0x3801001e},
- {0xc78, 0x3802001e}, {0xc78, 0x3803001e},
- {0xc78, 0x3804001e}, {0xc78, 0x3805001e},
- {0xc78, 0x3806001e}, {0xc78, 0x3807001e},
- {0xc78, 0x3808001e}, {0xc78, 0x3c09001e},
- {0xc78, 0x3e0a001e}, {0xc78, 0x400b001e},
- {0xc78, 0x440c001e}, {0xc78, 0x480d001e},
- {0xc78, 0x4c0e001e}, {0xc78, 0x500f001e},
- {0xc78, 0x5210001e}, {0xc78, 0x5611001e},
- {0xc78, 0x5a12001e}, {0xc78, 0x5e13001e},
- {0xc78, 0x6014001e}, {0xc78, 0x6015001e},
- {0xc78, 0x6016001e}, {0xc78, 0x6217001e},
- {0xc78, 0x6218001e}, {0xc78, 0x6219001e},
- {0xc78, 0x621a001e}, {0xc78, 0x621b001e},
- {0xc78, 0x621c001e}, {0xc78, 0x621d001e},
- {0xc78, 0x621e001e}, {0xc78, 0x621f001e},
- {0xffff, 0xffffffff}
-};
-
-static struct rtl8xxxu_reg32val rtl8xxx_agc_8723bu_table[] = {
- {0xc78, 0xfd000001}, {0xc78, 0xfc010001},
- {0xc78, 0xfb020001}, {0xc78, 0xfa030001},
- {0xc78, 0xf9040001}, {0xc78, 0xf8050001},
- {0xc78, 0xf7060001}, {0xc78, 0xf6070001},
- {0xc78, 0xf5080001}, {0xc78, 0xf4090001},
- {0xc78, 0xf30a0001}, {0xc78, 0xf20b0001},
- {0xc78, 0xf10c0001}, {0xc78, 0xf00d0001},
- {0xc78, 0xef0e0001}, {0xc78, 0xee0f0001},
- {0xc78, 0xed100001}, {0xc78, 0xec110001},
- {0xc78, 0xeb120001}, {0xc78, 0xea130001},
- {0xc78, 0xe9140001}, {0xc78, 0xe8150001},
- {0xc78, 0xe7160001}, {0xc78, 0xe6170001},
- {0xc78, 0xe5180001}, {0xc78, 0xe4190001},
- {0xc78, 0xe31a0001}, {0xc78, 0xa51b0001},
- {0xc78, 0xa41c0001}, {0xc78, 0xa31d0001},
- {0xc78, 0x671e0001}, {0xc78, 0x661f0001},
- {0xc78, 0x65200001}, {0xc78, 0x64210001},
- {0xc78, 0x63220001}, {0xc78, 0x4a230001},
- {0xc78, 0x49240001}, {0xc78, 0x48250001},
- {0xc78, 0x47260001}, {0xc78, 0x46270001},
- {0xc78, 0x45280001}, {0xc78, 0x44290001},
- {0xc78, 0x432a0001}, {0xc78, 0x422b0001},
- {0xc78, 0x292c0001}, {0xc78, 0x282d0001},
- {0xc78, 0x272e0001}, {0xc78, 0x262f0001},
- {0xc78, 0x0a300001}, {0xc78, 0x09310001},
- {0xc78, 0x08320001}, {0xc78, 0x07330001},
- {0xc78, 0x06340001}, {0xc78, 0x05350001},
- {0xc78, 0x04360001}, {0xc78, 0x03370001},
- {0xc78, 0x02380001}, {0xc78, 0x01390001},
- {0xc78, 0x013a0001}, {0xc78, 0x013b0001},
- {0xc78, 0x013c0001}, {0xc78, 0x013d0001},
- {0xc78, 0x013e0001}, {0xc78, 0x013f0001},
- {0xc78, 0xfc400001}, {0xc78, 0xfb410001},
- {0xc78, 0xfa420001}, {0xc78, 0xf9430001},
- {0xc78, 0xf8440001}, {0xc78, 0xf7450001},
- {0xc78, 0xf6460001}, {0xc78, 0xf5470001},
- {0xc78, 0xf4480001}, {0xc78, 0xf3490001},
- {0xc78, 0xf24a0001}, {0xc78, 0xf14b0001},
- {0xc78, 0xf04c0001}, {0xc78, 0xef4d0001},
- {0xc78, 0xee4e0001}, {0xc78, 0xed4f0001},
- {0xc78, 0xec500001}, {0xc78, 0xeb510001},
- {0xc78, 0xea520001}, {0xc78, 0xe9530001},
- {0xc78, 0xe8540001}, {0xc78, 0xe7550001},
- {0xc78, 0xe6560001}, {0xc78, 0xe5570001},
- {0xc78, 0xe4580001}, {0xc78, 0xe3590001},
- {0xc78, 0xa65a0001}, {0xc78, 0xa55b0001},
- {0xc78, 0xa45c0001}, {0xc78, 0xa35d0001},
- {0xc78, 0x675e0001}, {0xc78, 0x665f0001},
- {0xc78, 0x65600001}, {0xc78, 0x64610001},
- {0xc78, 0x63620001}, {0xc78, 0x62630001},
- {0xc78, 0x61640001}, {0xc78, 0x48650001},
- {0xc78, 0x47660001}, {0xc78, 0x46670001},
- {0xc78, 0x45680001}, {0xc78, 0x44690001},
- {0xc78, 0x436a0001}, {0xc78, 0x426b0001},
- {0xc78, 0x286c0001}, {0xc78, 0x276d0001},
- {0xc78, 0x266e0001}, {0xc78, 0x256f0001},
- {0xc78, 0x24700001}, {0xc78, 0x09710001},
- {0xc78, 0x08720001}, {0xc78, 0x07730001},
- {0xc78, 0x06740001}, {0xc78, 0x05750001},
- {0xc78, 0x04760001}, {0xc78, 0x03770001},
- {0xc78, 0x02780001}, {0xc78, 0x01790001},
- {0xc78, 0x017a0001}, {0xc78, 0x017b0001},
- {0xc78, 0x017c0001}, {0xc78, 0x017d0001},
- {0xc78, 0x017e0001}, {0xc78, 0x017f0001},
- {0xc50, 0x69553422},
- {0xc50, 0x69553420},
- {0x824, 0x00390204},
- {0xffff, 0xffffffff}
-};
-
-static struct rtl8xxxu_reg32val rtl8xxx_agc_8192eu_std_table[] = {
- {0xc78, 0xfb000001}, {0xc78, 0xfb010001},
- {0xc78, 0xfb020001}, {0xc78, 0xfb030001},
- {0xc78, 0xfb040001}, {0xc78, 0xfb050001},
- {0xc78, 0xfa060001}, {0xc78, 0xf9070001},
- {0xc78, 0xf8080001}, {0xc78, 0xf7090001},
- {0xc78, 0xf60a0001}, {0xc78, 0xf50b0001},
- {0xc78, 0xf40c0001}, {0xc78, 0xf30d0001},
- {0xc78, 0xf20e0001}, {0xc78, 0xf10f0001},
- {0xc78, 0xf0100001}, {0xc78, 0xef110001},
- {0xc78, 0xee120001}, {0xc78, 0xed130001},
- {0xc78, 0xec140001}, {0xc78, 0xeb150001},
- {0xc78, 0xea160001}, {0xc78, 0xe9170001},
- {0xc78, 0xe8180001}, {0xc78, 0xe7190001},
- {0xc78, 0xc81a0001}, {0xc78, 0xc71b0001},
- {0xc78, 0xc61c0001}, {0xc78, 0x071d0001},
- {0xc78, 0x061e0001}, {0xc78, 0x051f0001},
- {0xc78, 0x04200001}, {0xc78, 0x03210001},
- {0xc78, 0xaa220001}, {0xc78, 0xa9230001},
- {0xc78, 0xa8240001}, {0xc78, 0xa7250001},
- {0xc78, 0xa6260001}, {0xc78, 0x85270001},
- {0xc78, 0x84280001}, {0xc78, 0x83290001},
- {0xc78, 0x252a0001}, {0xc78, 0x242b0001},
- {0xc78, 0x232c0001}, {0xc78, 0x222d0001},
- {0xc78, 0x672e0001}, {0xc78, 0x662f0001},
- {0xc78, 0x65300001}, {0xc78, 0x64310001},
- {0xc78, 0x63320001}, {0xc78, 0x62330001},
- {0xc78, 0x61340001}, {0xc78, 0x45350001},
- {0xc78, 0x44360001}, {0xc78, 0x43370001},
- {0xc78, 0x42380001}, {0xc78, 0x41390001},
- {0xc78, 0x403a0001}, {0xc78, 0x403b0001},
- {0xc78, 0x403c0001}, {0xc78, 0x403d0001},
- {0xc78, 0x403e0001}, {0xc78, 0x403f0001},
- {0xc78, 0xfb400001}, {0xc78, 0xfb410001},
- {0xc78, 0xfb420001}, {0xc78, 0xfb430001},
- {0xc78, 0xfb440001}, {0xc78, 0xfb450001},
- {0xc78, 0xfa460001}, {0xc78, 0xf9470001},
- {0xc78, 0xf8480001}, {0xc78, 0xf7490001},
- {0xc78, 0xf64a0001}, {0xc78, 0xf54b0001},
- {0xc78, 0xf44c0001}, {0xc78, 0xf34d0001},
- {0xc78, 0xf24e0001}, {0xc78, 0xf14f0001},
- {0xc78, 0xf0500001}, {0xc78, 0xef510001},
- {0xc78, 0xee520001}, {0xc78, 0xed530001},
- {0xc78, 0xec540001}, {0xc78, 0xeb550001},
- {0xc78, 0xea560001}, {0xc78, 0xe9570001},
- {0xc78, 0xe8580001}, {0xc78, 0xe7590001},
- {0xc78, 0xe65a0001}, {0xc78, 0xe55b0001},
- {0xc78, 0xe45c0001}, {0xc78, 0xe35d0001},
- {0xc78, 0xe25e0001}, {0xc78, 0xe15f0001},
- {0xc78, 0x8a600001}, {0xc78, 0x89610001},
- {0xc78, 0x88620001}, {0xc78, 0x87630001},
- {0xc78, 0x86640001}, {0xc78, 0x85650001},
- {0xc78, 0x84660001}, {0xc78, 0x83670001},
- {0xc78, 0x82680001}, {0xc78, 0x6b690001},
- {0xc78, 0x6a6a0001}, {0xc78, 0x696b0001},
- {0xc78, 0x686c0001}, {0xc78, 0x676d0001},
- {0xc78, 0x666e0001}, {0xc78, 0x656f0001},
- {0xc78, 0x64700001}, {0xc78, 0x63710001},
- {0xc78, 0x62720001}, {0xc78, 0x61730001},
- {0xc78, 0x49740001}, {0xc78, 0x48750001},
- {0xc78, 0x47760001}, {0xc78, 0x46770001},
- {0xc78, 0x45780001}, {0xc78, 0x44790001},
- {0xc78, 0x437a0001}, {0xc78, 0x427b0001},
- {0xc78, 0x417c0001}, {0xc78, 0x407d0001},
- {0xc78, 0x407e0001}, {0xc78, 0x407f0001},
- {0xc50, 0x00040022}, {0xc50, 0x00040020},
- {0xffff, 0xffffffff}
-};
-
-static struct rtl8xxxu_reg32val rtl8xxx_agc_8192eu_highpa_table[] = {
- {0xc78, 0xfa000001}, {0xc78, 0xf9010001},
- {0xc78, 0xf8020001}, {0xc78, 0xf7030001},
- {0xc78, 0xf6040001}, {0xc78, 0xf5050001},
- {0xc78, 0xf4060001}, {0xc78, 0xf3070001},
- {0xc78, 0xf2080001}, {0xc78, 0xf1090001},
- {0xc78, 0xf00a0001}, {0xc78, 0xef0b0001},
- {0xc78, 0xee0c0001}, {0xc78, 0xed0d0001},
- {0xc78, 0xec0e0001}, {0xc78, 0xeb0f0001},
- {0xc78, 0xea100001}, {0xc78, 0xe9110001},
- {0xc78, 0xe8120001}, {0xc78, 0xe7130001},
- {0xc78, 0xe6140001}, {0xc78, 0xe5150001},
- {0xc78, 0xe4160001}, {0xc78, 0xe3170001},
- {0xc78, 0xe2180001}, {0xc78, 0xe1190001},
- {0xc78, 0x8a1a0001}, {0xc78, 0x891b0001},
- {0xc78, 0x881c0001}, {0xc78, 0x871d0001},
- {0xc78, 0x861e0001}, {0xc78, 0x851f0001},
- {0xc78, 0x84200001}, {0xc78, 0x83210001},
- {0xc78, 0x82220001}, {0xc78, 0x6a230001},
- {0xc78, 0x69240001}, {0xc78, 0x68250001},
- {0xc78, 0x67260001}, {0xc78, 0x66270001},
- {0xc78, 0x65280001}, {0xc78, 0x64290001},
- {0xc78, 0x632a0001}, {0xc78, 0x622b0001},
- {0xc78, 0x612c0001}, {0xc78, 0x602d0001},
- {0xc78, 0x472e0001}, {0xc78, 0x462f0001},
- {0xc78, 0x45300001}, {0xc78, 0x44310001},
- {0xc78, 0x43320001}, {0xc78, 0x42330001},
- {0xc78, 0x41340001}, {0xc78, 0x40350001},
- {0xc78, 0x40360001}, {0xc78, 0x40370001},
- {0xc78, 0x40380001}, {0xc78, 0x40390001},
- {0xc78, 0x403a0001}, {0xc78, 0x403b0001},
- {0xc78, 0x403c0001}, {0xc78, 0x403d0001},
- {0xc78, 0x403e0001}, {0xc78, 0x403f0001},
- {0xc78, 0xfa400001}, {0xc78, 0xf9410001},
- {0xc78, 0xf8420001}, {0xc78, 0xf7430001},
- {0xc78, 0xf6440001}, {0xc78, 0xf5450001},
- {0xc78, 0xf4460001}, {0xc78, 0xf3470001},
- {0xc78, 0xf2480001}, {0xc78, 0xf1490001},
- {0xc78, 0xf04a0001}, {0xc78, 0xef4b0001},
- {0xc78, 0xee4c0001}, {0xc78, 0xed4d0001},
- {0xc78, 0xec4e0001}, {0xc78, 0xeb4f0001},
- {0xc78, 0xea500001}, {0xc78, 0xe9510001},
- {0xc78, 0xe8520001}, {0xc78, 0xe7530001},
- {0xc78, 0xe6540001}, {0xc78, 0xe5550001},
- {0xc78, 0xe4560001}, {0xc78, 0xe3570001},
- {0xc78, 0xe2580001}, {0xc78, 0xe1590001},
- {0xc78, 0x8a5a0001}, {0xc78, 0x895b0001},
- {0xc78, 0x885c0001}, {0xc78, 0x875d0001},
- {0xc78, 0x865e0001}, {0xc78, 0x855f0001},
- {0xc78, 0x84600001}, {0xc78, 0x83610001},
- {0xc78, 0x82620001}, {0xc78, 0x6a630001},
- {0xc78, 0x69640001}, {0xc78, 0x68650001},
- {0xc78, 0x67660001}, {0xc78, 0x66670001},
- {0xc78, 0x65680001}, {0xc78, 0x64690001},
- {0xc78, 0x636a0001}, {0xc78, 0x626b0001},
- {0xc78, 0x616c0001}, {0xc78, 0x606d0001},
- {0xc78, 0x476e0001}, {0xc78, 0x466f0001},
- {0xc78, 0x45700001}, {0xc78, 0x44710001},
- {0xc78, 0x43720001}, {0xc78, 0x42730001},
- {0xc78, 0x41740001}, {0xc78, 0x40750001},
- {0xc78, 0x40760001}, {0xc78, 0x40770001},
- {0xc78, 0x40780001}, {0xc78, 0x40790001},
- {0xc78, 0x407a0001}, {0xc78, 0x407b0001},
- {0xc78, 0x407c0001}, {0xc78, 0x407d0001},
- {0xc78, 0x407e0001}, {0xc78, 0x407f0001},
- {0xc50, 0x00040222}, {0xc50, 0x00040220},
- {0xffff, 0xffffffff}
-};
-
-static struct rtl8xxxu_rfregval rtl8723au_radioa_1t_init_table[] = {
- {0x00, 0x00030159}, {0x01, 0x00031284},
- {0x02, 0x00098000}, {0x03, 0x00039c63},
- {0x04, 0x000210e7}, {0x09, 0x0002044f},
- {0x0a, 0x0001a3f1}, {0x0b, 0x00014787},
- {0x0c, 0x000896fe}, {0x0d, 0x0000e02c},
- {0x0e, 0x00039ce7}, {0x0f, 0x00000451},
- {0x19, 0x00000000}, {0x1a, 0x00030355},
- {0x1b, 0x00060a00}, {0x1c, 0x000fc378},
- {0x1d, 0x000a1250}, {0x1e, 0x0000024f},
- {0x1f, 0x00000000}, {0x20, 0x0000b614},
- {0x21, 0x0006c000}, {0x22, 0x00000000},
- {0x23, 0x00001558}, {0x24, 0x00000060},
- {0x25, 0x00000483}, {0x26, 0x0004f000},
- {0x27, 0x000ec7d9}, {0x28, 0x00057730},
- {0x29, 0x00004783}, {0x2a, 0x00000001},
- {0x2b, 0x00021334}, {0x2a, 0x00000000},
- {0x2b, 0x00000054}, {0x2a, 0x00000001},
- {0x2b, 0x00000808}, {0x2b, 0x00053333},
- {0x2c, 0x0000000c}, {0x2a, 0x00000002},
- {0x2b, 0x00000808}, {0x2b, 0x0005b333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000003},
- {0x2b, 0x00000808}, {0x2b, 0x00063333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000004},
- {0x2b, 0x00000808}, {0x2b, 0x0006b333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000005},
- {0x2b, 0x00000808}, {0x2b, 0x00073333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000006},
- {0x2b, 0x00000709}, {0x2b, 0x0005b333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000007},
- {0x2b, 0x00000709}, {0x2b, 0x00063333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000008},
- {0x2b, 0x0000060a}, {0x2b, 0x0004b333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000009},
- {0x2b, 0x0000060a}, {0x2b, 0x00053333},
- {0x2c, 0x0000000d}, {0x2a, 0x0000000a},
- {0x2b, 0x0000060a}, {0x2b, 0x0005b333},
- {0x2c, 0x0000000d}, {0x2a, 0x0000000b},
- {0x2b, 0x0000060a}, {0x2b, 0x00063333},
- {0x2c, 0x0000000d}, {0x2a, 0x0000000c},
- {0x2b, 0x0000060a}, {0x2b, 0x0006b333},
- {0x2c, 0x0000000d}, {0x2a, 0x0000000d},
- {0x2b, 0x0000060a}, {0x2b, 0x00073333},
- {0x2c, 0x0000000d}, {0x2a, 0x0000000e},
- {0x2b, 0x0000050b}, {0x2b, 0x00066666},
- {0x2c, 0x0000001a}, {0x2a, 0x000e0000},
- {0x10, 0x0004000f}, {0x11, 0x000e31fc},
- {0x10, 0x0006000f}, {0x11, 0x000ff9f8},
- {0x10, 0x0002000f}, {0x11, 0x000203f9},
- {0x10, 0x0003000f}, {0x11, 0x000ff500},
- {0x10, 0x00000000}, {0x11, 0x00000000},
- {0x10, 0x0008000f}, {0x11, 0x0003f100},
- {0x10, 0x0009000f}, {0x11, 0x00023100},
- {0x12, 0x00032000}, {0x12, 0x00071000},
- {0x12, 0x000b0000}, {0x12, 0x000fc000},
- {0x13, 0x000287b3}, {0x13, 0x000244b7},
- {0x13, 0x000204ab}, {0x13, 0x0001c49f},
- {0x13, 0x00018493}, {0x13, 0x0001429b},
- {0x13, 0x00010299}, {0x13, 0x0000c29c},
- {0x13, 0x000081a0}, {0x13, 0x000040ac},
- {0x13, 0x00000020}, {0x14, 0x0001944c},
- {0x14, 0x00059444}, {0x14, 0x0009944c},
- {0x14, 0x000d9444}, {0x15, 0x0000f474},
- {0x15, 0x0004f477}, {0x15, 0x0008f455},
- {0x15, 0x000cf455}, {0x16, 0x00000339},
- {0x16, 0x00040339}, {0x16, 0x00080339},
- {0x16, 0x000c0366}, {0x00, 0x00010159},
- {0x18, 0x0000f401}, {0xfe, 0x00000000},
- {0xfe, 0x00000000}, {0x1f, 0x00000003},
- {0xfe, 0x00000000}, {0xfe, 0x00000000},
- {0x1e, 0x00000247}, {0x1f, 0x00000000},
- {0x00, 0x00030159},
- {0xff, 0xffffffff}
-};
-
-static struct rtl8xxxu_rfregval rtl8723bu_radioa_1t_init_table[] = {
- {0x00, 0x00010000}, {0xb0, 0x000dffe0},
- {0xfe, 0x00000000}, {0xfe, 0x00000000},
- {0xfe, 0x00000000}, {0xb1, 0x00000018},
- {0xfe, 0x00000000}, {0xfe, 0x00000000},
- {0xfe, 0x00000000}, {0xb2, 0x00084c00},
- {0xb5, 0x0000d2cc}, {0xb6, 0x000925aa},
- {0xb7, 0x00000010}, {0xb8, 0x0000907f},
- {0x5c, 0x00000002}, {0x7c, 0x00000002},
- {0x7e, 0x00000005}, {0x8b, 0x0006fc00},
- {0xb0, 0x000ff9f0}, {0x1c, 0x000739d2},
- {0x1e, 0x00000000}, {0xdf, 0x00000780},
- {0x50, 0x00067435},
- /*
- * The 8723bu vendor driver indicates that bit 8 should be set in
- * 0x51 for package types TFBGA90, TFBGA80, and TFBGA79. However
- * they never actually check the package type - and just default
- * to not setting it.
- */
- {0x51, 0x0006b04e},
- {0x52, 0x000007d2}, {0x53, 0x00000000},
- {0x54, 0x00050400}, {0x55, 0x0004026e},
- {0xdd, 0x0000004c}, {0x70, 0x00067435},
- /*
- * 0x71 has same package type condition as for register 0x51
- */
- {0x71, 0x0006b04e},
- {0x72, 0x000007d2}, {0x73, 0x00000000},
- {0x74, 0x00050400}, {0x75, 0x0004026e},
- {0xef, 0x00000100}, {0x34, 0x0000add7},
- {0x35, 0x00005c00}, {0x34, 0x00009dd4},
- {0x35, 0x00005000}, {0x34, 0x00008dd1},
- {0x35, 0x00004400}, {0x34, 0x00007dce},
- {0x35, 0x00003800}, {0x34, 0x00006cd1},
- {0x35, 0x00004400}, {0x34, 0x00005cce},
- {0x35, 0x00003800}, {0x34, 0x000048ce},
- {0x35, 0x00004400}, {0x34, 0x000034ce},
- {0x35, 0x00003800}, {0x34, 0x00002451},
- {0x35, 0x00004400}, {0x34, 0x0000144e},
- {0x35, 0x00003800}, {0x34, 0x00000051},
- {0x35, 0x00004400}, {0xef, 0x00000000},
- {0xef, 0x00000100}, {0xed, 0x00000010},
- {0x44, 0x0000add7}, {0x44, 0x00009dd4},
- {0x44, 0x00008dd1}, {0x44, 0x00007dce},
- {0x44, 0x00006cc1}, {0x44, 0x00005cce},
- {0x44, 0x000044d1}, {0x44, 0x000034ce},
- {0x44, 0x00002451}, {0x44, 0x0000144e},
- {0x44, 0x00000051}, {0xef, 0x00000000},
- {0xed, 0x00000000}, {0x7f, 0x00020080},
- {0xef, 0x00002000}, {0x3b, 0x000380ef},
- {0x3b, 0x000302fe}, {0x3b, 0x00028ce6},
- {0x3b, 0x000200bc}, {0x3b, 0x000188a5},
- {0x3b, 0x00010fbc}, {0x3b, 0x00008f71},
- {0x3b, 0x00000900}, {0xef, 0x00000000},
- {0xed, 0x00000001}, {0x40, 0x000380ef},
- {0x40, 0x000302fe}, {0x40, 0x00028ce6},
- {0x40, 0x000200bc}, {0x40, 0x000188a5},
- {0x40, 0x00010fbc}, {0x40, 0x00008f71},
- {0x40, 0x00000900}, {0xed, 0x00000000},
- {0x82, 0x00080000}, {0x83, 0x00008000},
- {0x84, 0x00048d80}, {0x85, 0x00068000},
- {0xa2, 0x00080000}, {0xa3, 0x00008000},
- {0xa4, 0x00048d80}, {0xa5, 0x00068000},
- {0xed, 0x00000002}, {0xef, 0x00000002},
- {0x56, 0x00000032}, {0x76, 0x00000032},
- {0x01, 0x00000780},
- {0xff, 0xffffffff}
-};
-
-#ifdef CONFIG_RTL8XXXU_UNTESTED
-static struct rtl8xxxu_rfregval rtl8192cu_radioa_2t_init_table[] = {
- {0x00, 0x00030159}, {0x01, 0x00031284},
- {0x02, 0x00098000}, {0x03, 0x00018c63},
- {0x04, 0x000210e7}, {0x09, 0x0002044f},
- {0x0a, 0x0001adb1}, {0x0b, 0x00054867},
- {0x0c, 0x0008992e}, {0x0d, 0x0000e52c},
- {0x0e, 0x00039ce7}, {0x0f, 0x00000451},
- {0x19, 0x00000000}, {0x1a, 0x00010255},
- {0x1b, 0x00060a00}, {0x1c, 0x000fc378},
- {0x1d, 0x000a1250}, {0x1e, 0x0004445f},
- {0x1f, 0x00080001}, {0x20, 0x0000b614},
- {0x21, 0x0006c000}, {0x22, 0x00000000},
- {0x23, 0x00001558}, {0x24, 0x00000060},
- {0x25, 0x00000483}, {0x26, 0x0004f000},
- {0x27, 0x000ec7d9}, {0x28, 0x000577c0},
- {0x29, 0x00004783}, {0x2a, 0x00000001},
- {0x2b, 0x00021334}, {0x2a, 0x00000000},
- {0x2b, 0x00000054}, {0x2a, 0x00000001},
- {0x2b, 0x00000808}, {0x2b, 0x00053333},
- {0x2c, 0x0000000c}, {0x2a, 0x00000002},
- {0x2b, 0x00000808}, {0x2b, 0x0005b333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000003},
- {0x2b, 0x00000808}, {0x2b, 0x00063333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000004},
- {0x2b, 0x00000808}, {0x2b, 0x0006b333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000005},
- {0x2b, 0x00000808}, {0x2b, 0x00073333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000006},
- {0x2b, 0x00000709}, {0x2b, 0x0005b333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000007},
- {0x2b, 0x00000709}, {0x2b, 0x00063333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000008},
- {0x2b, 0x0000060a}, {0x2b, 0x0004b333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000009},
- {0x2b, 0x0000060a}, {0x2b, 0x00053333},
- {0x2c, 0x0000000d}, {0x2a, 0x0000000a},
- {0x2b, 0x0000060a}, {0x2b, 0x0005b333},
- {0x2c, 0x0000000d}, {0x2a, 0x0000000b},
- {0x2b, 0x0000060a}, {0x2b, 0x00063333},
- {0x2c, 0x0000000d}, {0x2a, 0x0000000c},
- {0x2b, 0x0000060a}, {0x2b, 0x0006b333},
- {0x2c, 0x0000000d}, {0x2a, 0x0000000d},
- {0x2b, 0x0000060a}, {0x2b, 0x00073333},
- {0x2c, 0x0000000d}, {0x2a, 0x0000000e},
- {0x2b, 0x0000050b}, {0x2b, 0x00066666},
- {0x2c, 0x0000001a}, {0x2a, 0x000e0000},
- {0x10, 0x0004000f}, {0x11, 0x000e31fc},
- {0x10, 0x0006000f}, {0x11, 0x000ff9f8},
- {0x10, 0x0002000f}, {0x11, 0x000203f9},
- {0x10, 0x0003000f}, {0x11, 0x000ff500},
- {0x10, 0x00000000}, {0x11, 0x00000000},
- {0x10, 0x0008000f}, {0x11, 0x0003f100},
- {0x10, 0x0009000f}, {0x11, 0x00023100},
- {0x12, 0x00032000}, {0x12, 0x00071000},
- {0x12, 0x000b0000}, {0x12, 0x000fc000},
- {0x13, 0x000287b3}, {0x13, 0x000244b7},
- {0x13, 0x000204ab}, {0x13, 0x0001c49f},
- {0x13, 0x00018493}, {0x13, 0x0001429b},
- {0x13, 0x00010299}, {0x13, 0x0000c29c},
- {0x13, 0x000081a0}, {0x13, 0x000040ac},
- {0x13, 0x00000020}, {0x14, 0x0001944c},
- {0x14, 0x00059444}, {0x14, 0x0009944c},
- {0x14, 0x000d9444}, {0x15, 0x0000f424},
- {0x15, 0x0004f424}, {0x15, 0x0008f424},
- {0x15, 0x000cf424}, {0x16, 0x000e0330},
- {0x16, 0x000a0330}, {0x16, 0x00060330},
- {0x16, 0x00020330}, {0x00, 0x00010159},
- {0x18, 0x0000f401}, {0xfe, 0x00000000},
- {0xfe, 0x00000000}, {0x1f, 0x00080003},
- {0xfe, 0x00000000}, {0xfe, 0x00000000},
- {0x1e, 0x00044457}, {0x1f, 0x00080000},
- {0x00, 0x00030159},
- {0xff, 0xffffffff}
-};
-
-static struct rtl8xxxu_rfregval rtl8192cu_radiob_2t_init_table[] = {
- {0x00, 0x00030159}, {0x01, 0x00031284},
- {0x02, 0x00098000}, {0x03, 0x00018c63},
- {0x04, 0x000210e7}, {0x09, 0x0002044f},
- {0x0a, 0x0001adb1}, {0x0b, 0x00054867},
- {0x0c, 0x0008992e}, {0x0d, 0x0000e52c},
- {0x0e, 0x00039ce7}, {0x0f, 0x00000451},
- {0x12, 0x00032000}, {0x12, 0x00071000},
- {0x12, 0x000b0000}, {0x12, 0x000fc000},
- {0x13, 0x000287af}, {0x13, 0x000244b7},
- {0x13, 0x000204ab}, {0x13, 0x0001c49f},
- {0x13, 0x00018493}, {0x13, 0x00014297},
- {0x13, 0x00010295}, {0x13, 0x0000c298},
- {0x13, 0x0000819c}, {0x13, 0x000040a8},
- {0x13, 0x0000001c}, {0x14, 0x0001944c},
- {0x14, 0x00059444}, {0x14, 0x0009944c},
- {0x14, 0x000d9444}, {0x15, 0x0000f424},
- {0x15, 0x0004f424}, {0x15, 0x0008f424},
- {0x15, 0x000cf424}, {0x16, 0x000e0330},
- {0x16, 0x000a0330}, {0x16, 0x00060330},
- {0x16, 0x00020330},
- {0xff, 0xffffffff}
-};
-
-static struct rtl8xxxu_rfregval rtl8192cu_radioa_1t_init_table[] = {
- {0x00, 0x00030159}, {0x01, 0x00031284},
- {0x02, 0x00098000}, {0x03, 0x00018c63},
- {0x04, 0x000210e7}, {0x09, 0x0002044f},
- {0x0a, 0x0001adb1}, {0x0b, 0x00054867},
- {0x0c, 0x0008992e}, {0x0d, 0x0000e52c},
- {0x0e, 0x00039ce7}, {0x0f, 0x00000451},
- {0x19, 0x00000000}, {0x1a, 0x00010255},
- {0x1b, 0x00060a00}, {0x1c, 0x000fc378},
- {0x1d, 0x000a1250}, {0x1e, 0x0004445f},
- {0x1f, 0x00080001}, {0x20, 0x0000b614},
- {0x21, 0x0006c000}, {0x22, 0x00000000},
- {0x23, 0x00001558}, {0x24, 0x00000060},
- {0x25, 0x00000483}, {0x26, 0x0004f000},
- {0x27, 0x000ec7d9}, {0x28, 0x000577c0},
- {0x29, 0x00004783}, {0x2a, 0x00000001},
- {0x2b, 0x00021334}, {0x2a, 0x00000000},
- {0x2b, 0x00000054}, {0x2a, 0x00000001},
- {0x2b, 0x00000808}, {0x2b, 0x00053333},
- {0x2c, 0x0000000c}, {0x2a, 0x00000002},
- {0x2b, 0x00000808}, {0x2b, 0x0005b333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000003},
- {0x2b, 0x00000808}, {0x2b, 0x00063333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000004},
- {0x2b, 0x00000808}, {0x2b, 0x0006b333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000005},
- {0x2b, 0x00000808}, {0x2b, 0x00073333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000006},
- {0x2b, 0x00000709}, {0x2b, 0x0005b333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000007},
- {0x2b, 0x00000709}, {0x2b, 0x00063333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000008},
- {0x2b, 0x0000060a}, {0x2b, 0x0004b333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000009},
- {0x2b, 0x0000060a}, {0x2b, 0x00053333},
- {0x2c, 0x0000000d}, {0x2a, 0x0000000a},
- {0x2b, 0x0000060a}, {0x2b, 0x0005b333},
- {0x2c, 0x0000000d}, {0x2a, 0x0000000b},
- {0x2b, 0x0000060a}, {0x2b, 0x00063333},
- {0x2c, 0x0000000d}, {0x2a, 0x0000000c},
- {0x2b, 0x0000060a}, {0x2b, 0x0006b333},
- {0x2c, 0x0000000d}, {0x2a, 0x0000000d},
- {0x2b, 0x0000060a}, {0x2b, 0x00073333},
- {0x2c, 0x0000000d}, {0x2a, 0x0000000e},
- {0x2b, 0x0000050b}, {0x2b, 0x00066666},
- {0x2c, 0x0000001a}, {0x2a, 0x000e0000},
- {0x10, 0x0004000f}, {0x11, 0x000e31fc},
- {0x10, 0x0006000f}, {0x11, 0x000ff9f8},
- {0x10, 0x0002000f}, {0x11, 0x000203f9},
- {0x10, 0x0003000f}, {0x11, 0x000ff500},
- {0x10, 0x00000000}, {0x11, 0x00000000},
- {0x10, 0x0008000f}, {0x11, 0x0003f100},
- {0x10, 0x0009000f}, {0x11, 0x00023100},
- {0x12, 0x00032000}, {0x12, 0x00071000},
- {0x12, 0x000b0000}, {0x12, 0x000fc000},
- {0x13, 0x000287b3}, {0x13, 0x000244b7},
- {0x13, 0x000204ab}, {0x13, 0x0001c49f},
- {0x13, 0x00018493}, {0x13, 0x0001429b},
- {0x13, 0x00010299}, {0x13, 0x0000c29c},
- {0x13, 0x000081a0}, {0x13, 0x000040ac},
- {0x13, 0x00000020}, {0x14, 0x0001944c},
- {0x14, 0x00059444}, {0x14, 0x0009944c},
- {0x14, 0x000d9444}, {0x15, 0x0000f405},
- {0x15, 0x0004f405}, {0x15, 0x0008f405},
- {0x15, 0x000cf405}, {0x16, 0x000e0330},
- {0x16, 0x000a0330}, {0x16, 0x00060330},
- {0x16, 0x00020330}, {0x00, 0x00010159},
- {0x18, 0x0000f401}, {0xfe, 0x00000000},
- {0xfe, 0x00000000}, {0x1f, 0x00080003},
- {0xfe, 0x00000000}, {0xfe, 0x00000000},
- {0x1e, 0x00044457}, {0x1f, 0x00080000},
- {0x00, 0x00030159},
- {0xff, 0xffffffff}
-};
-
-static struct rtl8xxxu_rfregval rtl8188ru_radioa_1t_highpa_table[] = {
- {0x00, 0x00030159}, {0x01, 0x00031284},
- {0x02, 0x00098000}, {0x03, 0x00018c63},
- {0x04, 0x000210e7}, {0x09, 0x0002044f},
- {0x0a, 0x0001adb0}, {0x0b, 0x00054867},
- {0x0c, 0x0008992e}, {0x0d, 0x0000e529},
- {0x0e, 0x00039ce7}, {0x0f, 0x00000451},
- {0x19, 0x00000000}, {0x1a, 0x00000255},
- {0x1b, 0x00060a00}, {0x1c, 0x000fc378},
- {0x1d, 0x000a1250}, {0x1e, 0x0004445f},
- {0x1f, 0x00080001}, {0x20, 0x0000b614},
- {0x21, 0x0006c000}, {0x22, 0x0000083c},
- {0x23, 0x00001558}, {0x24, 0x00000060},
- {0x25, 0x00000483}, {0x26, 0x0004f000},
- {0x27, 0x000ec7d9}, {0x28, 0x000977c0},
- {0x29, 0x00004783}, {0x2a, 0x00000001},
- {0x2b, 0x00021334}, {0x2a, 0x00000000},
- {0x2b, 0x00000054}, {0x2a, 0x00000001},
- {0x2b, 0x00000808}, {0x2b, 0x00053333},
- {0x2c, 0x0000000c}, {0x2a, 0x00000002},
- {0x2b, 0x00000808}, {0x2b, 0x0005b333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000003},
- {0x2b, 0x00000808}, {0x2b, 0x00063333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000004},
- {0x2b, 0x00000808}, {0x2b, 0x0006b333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000005},
- {0x2b, 0x00000808}, {0x2b, 0x00073333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000006},
- {0x2b, 0x00000709}, {0x2b, 0x0005b333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000007},
- {0x2b, 0x00000709}, {0x2b, 0x00063333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000008},
- {0x2b, 0x0000060a}, {0x2b, 0x0004b333},
- {0x2c, 0x0000000d}, {0x2a, 0x00000009},
- {0x2b, 0x0000060a}, {0x2b, 0x00053333},
- {0x2c, 0x0000000d}, {0x2a, 0x0000000a},
- {0x2b, 0x0000060a}, {0x2b, 0x0005b333},
- {0x2c, 0x0000000d}, {0x2a, 0x0000000b},
- {0x2b, 0x0000060a}, {0x2b, 0x00063333},
- {0x2c, 0x0000000d}, {0x2a, 0x0000000c},
- {0x2b, 0x0000060a}, {0x2b, 0x0006b333},
- {0x2c, 0x0000000d}, {0x2a, 0x0000000d},
- {0x2b, 0x0000060a}, {0x2b, 0x00073333},
- {0x2c, 0x0000000d}, {0x2a, 0x0000000e},
- {0x2b, 0x0000050b}, {0x2b, 0x00066666},
- {0x2c, 0x0000001a}, {0x2a, 0x000e0000},
- {0x10, 0x0004000f}, {0x11, 0x000e31fc},
- {0x10, 0x0006000f}, {0x11, 0x000ff9f8},
- {0x10, 0x0002000f}, {0x11, 0x000203f9},
- {0x10, 0x0003000f}, {0x11, 0x000ff500},
- {0x10, 0x00000000}, {0x11, 0x00000000},
- {0x10, 0x0008000f}, {0x11, 0x0003f100},
- {0x10, 0x0009000f}, {0x11, 0x00023100},
- {0x12, 0x000d8000}, {0x12, 0x00090000},
- {0x12, 0x00051000}, {0x12, 0x00012000},
- {0x13, 0x00028fb4}, {0x13, 0x00024fa8},
- {0x13, 0x000207a4}, {0x13, 0x0001c3b0},
- {0x13, 0x000183a4}, {0x13, 0x00014398},
- {0x13, 0x000101a4}, {0x13, 0x0000c198},
- {0x13, 0x000080a4}, {0x13, 0x00004098},
- {0x13, 0x00000000}, {0x14, 0x0001944c},
- {0x14, 0x00059444}, {0x14, 0x0009944c},
- {0x14, 0x000d9444}, {0x15, 0x0000f405},
- {0x15, 0x0004f405}, {0x15, 0x0008f405},
- {0x15, 0x000cf405}, {0x16, 0x000e0330},
- {0x16, 0x000a0330}, {0x16, 0x00060330},
- {0x16, 0x00020330}, {0x00, 0x00010159},
- {0x18, 0x0000f401}, {0xfe, 0x00000000},
- {0xfe, 0x00000000}, {0x1f, 0x00080003},
- {0xfe, 0x00000000}, {0xfe, 0x00000000},
- {0x1e, 0x00044457}, {0x1f, 0x00080000},
- {0x00, 0x00030159},
- {0xff, 0xffffffff}
-};
-#endif
-
-static struct rtl8xxxu_rfregval rtl8192eu_radioa_init_table[] = {
- {0x7f, 0x00000082}, {0x81, 0x0003fc00},
- {0x00, 0x00030000}, {0x08, 0x00008400},
- {0x18, 0x00000407}, {0x19, 0x00000012},
- {0x1b, 0x00000064}, {0x1e, 0x00080009},
- {0x1f, 0x00000880}, {0x2f, 0x0001a060},
- {0x3f, 0x00000000}, {0x42, 0x000060c0},
- {0x57, 0x000d0000}, {0x58, 0x000be180},
- {0x67, 0x00001552}, {0x83, 0x00000000},
- {0xb0, 0x000ff9f1}, {0xb1, 0x00055418},
- {0xb2, 0x0008cc00}, {0xb4, 0x00043083},
- {0xb5, 0x00008166}, {0xb6, 0x0000803e},
- {0xb7, 0x0001c69f}, {0xb8, 0x0000407f},
- {0xb9, 0x00080001}, {0xba, 0x00040001},
- {0xbb, 0x00000400}, {0xbf, 0x000c0000},
- {0xc2, 0x00002400}, {0xc3, 0x00000009},
- {0xc4, 0x00040c91}, {0xc5, 0x00099999},
- {0xc6, 0x000000a3}, {0xc7, 0x00088820},
- {0xc8, 0x00076c06}, {0xc9, 0x00000000},
- {0xca, 0x00080000}, {0xdf, 0x00000180},
- {0xef, 0x000001a0}, {0x51, 0x00069545},
- {0x52, 0x0007e45e}, {0x53, 0x00000071},
- {0x56, 0x00051ff3}, {0x35, 0x000000a8},
- {0x35, 0x000001e2}, {0x35, 0x000002a8},
- {0x36, 0x00001c24}, {0x36, 0x00009c24},
- {0x36, 0x00011c24}, {0x36, 0x00019c24},
- {0x18, 0x00000c07}, {0x5a, 0x00048000},
- {0x19, 0x000739d0},
-#ifdef EXT_PA_8192EU
- /* External PA or external LNA */
- {0x34, 0x0000a093}, {0x34, 0x0000908f},
- {0x34, 0x0000808c}, {0x34, 0x0000704d},
- {0x34, 0x0000604a}, {0x34, 0x00005047},
- {0x34, 0x0000400a}, {0x34, 0x00003007},
- {0x34, 0x00002004}, {0x34, 0x00001001},
- {0x34, 0x00000000},
-#else
- /* Regular */
- {0x34, 0x0000add7}, {0x34, 0x00009dd4},
- {0x34, 0x00008dd1}, {0x34, 0x00007dce},
- {0x34, 0x00006dcb}, {0x34, 0x00005dc8},
- {0x34, 0x00004dc5}, {0x34, 0x000034cc},
- {0x34, 0x0000244f}, {0x34, 0x0000144c},
- {0x34, 0x00000014},
-#endif
- {0x00, 0x00030159},
- {0x84, 0x00068180},
- {0x86, 0x0000014e},
- {0x87, 0x00048e00},
- {0x8e, 0x00065540},
- {0x8f, 0x00088000},
- {0xef, 0x000020a0},
-#ifdef EXT_PA_8192EU
- /* External PA or external LNA */
- {0x3b, 0x000f07b0},
-#else
- {0x3b, 0x000f02b0},
-#endif
- {0x3b, 0x000ef7b0}, {0x3b, 0x000d4fb0},
- {0x3b, 0x000cf060}, {0x3b, 0x000b0090},
- {0x3b, 0x000a0080}, {0x3b, 0x00090080},
- {0x3b, 0x0008f780},
-#ifdef EXT_PA_8192EU
- /* External PA or external LNA */
- {0x3b, 0x000787b0},
-#else
- {0x3b, 0x00078730},
-#endif
- {0x3b, 0x00060fb0}, {0x3b, 0x0005ffa0},
- {0x3b, 0x00040620}, {0x3b, 0x00037090},
- {0x3b, 0x00020080}, {0x3b, 0x0001f060},
- {0x3b, 0x0000ffb0}, {0xef, 0x000000a0},
- {0xfe, 0x00000000}, {0x18, 0x0000fc07},
- {0xfe, 0x00000000}, {0xfe, 0x00000000},
- {0xfe, 0x00000000}, {0xfe, 0x00000000},
- {0x1e, 0x00000001}, {0x1f, 0x00080000},
- {0x00, 0x00033e70},
- {0xff, 0xffffffff}
-};
-
-static struct rtl8xxxu_rfregval rtl8192eu_radiob_init_table[] = {
- {0x7f, 0x00000082}, {0x81, 0x0003fc00},
- {0x00, 0x00030000}, {0x08, 0x00008400},
- {0x18, 0x00000407}, {0x19, 0x00000012},
- {0x1b, 0x00000064}, {0x1e, 0x00080009},
- {0x1f, 0x00000880}, {0x2f, 0x0001a060},
- {0x3f, 0x00000000}, {0x42, 0x000060c0},
- {0x57, 0x000d0000}, {0x58, 0x000be180},
- {0x67, 0x00001552}, {0x7f, 0x00000082},
- {0x81, 0x0003f000}, {0x83, 0x00000000},
- {0xdf, 0x00000180}, {0xef, 0x000001a0},
- {0x51, 0x00069545}, {0x52, 0x0007e42e},
- {0x53, 0x00000071}, {0x56, 0x00051ff3},
- {0x35, 0x000000a8}, {0x35, 0x000001e0},
- {0x35, 0x000002a8}, {0x36, 0x00001ca8},
- {0x36, 0x00009c24}, {0x36, 0x00011c24},
- {0x36, 0x00019c24}, {0x18, 0x00000c07},
- {0x5a, 0x00048000}, {0x19, 0x000739d0},
-#ifdef EXT_PA_8192EU
- /* External PA or external LNA */
- {0x34, 0x0000a093}, {0x34, 0x0000908f},
- {0x34, 0x0000808c}, {0x34, 0x0000704d},
- {0x34, 0x0000604a}, {0x34, 0x00005047},
- {0x34, 0x0000400a}, {0x34, 0x00003007},
- {0x34, 0x00002004}, {0x34, 0x00001001},
- {0x34, 0x00000000},
-#else
- {0x34, 0x0000add7}, {0x34, 0x00009dd4},
- {0x34, 0x00008dd1}, {0x34, 0x00007dce},
- {0x34, 0x00006dcb}, {0x34, 0x00005dc8},
- {0x34, 0x00004dc5}, {0x34, 0x000034cc},
- {0x34, 0x0000244f}, {0x34, 0x0000144c},
- {0x34, 0x00000014},
-#endif
- {0x00, 0x00030159}, {0x84, 0x00068180},
- {0x86, 0x000000ce}, {0x87, 0x00048a00},
- {0x8e, 0x00065540}, {0x8f, 0x00088000},
- {0xef, 0x000020a0},
-#ifdef EXT_PA_8192EU
- /* External PA or external LNA */
- {0x3b, 0x000f07b0},
-#else
- {0x3b, 0x000f02b0},
-#endif
-
- {0x3b, 0x000ef7b0}, {0x3b, 0x000d4fb0},
- {0x3b, 0x000cf060}, {0x3b, 0x000b0090},
- {0x3b, 0x000a0080}, {0x3b, 0x00090080},
- {0x3b, 0x0008f780},
-#ifdef EXT_PA_8192EU
- /* External PA or external LNA */
- {0x3b, 0x000787b0},
-#else
- {0x3b, 0x00078730},
-#endif
- {0x3b, 0x00060fb0}, {0x3b, 0x0005ffa0},
- {0x3b, 0x00040620}, {0x3b, 0x00037090},
- {0x3b, 0x00020080}, {0x3b, 0x0001f060},
- {0x3b, 0x0000ffb0}, {0xef, 0x000000a0},
- {0x00, 0x00010159}, {0xfe, 0x00000000},
- {0xfe, 0x00000000}, {0xfe, 0x00000000},
- {0xfe, 0x00000000}, {0x1e, 0x00000001},
- {0x1f, 0x00080000}, {0x00, 0x00033e70},
- {0xff, 0xffffffff}
-};
-
-static struct rtl8xxxu_rfregs rtl8xxxu_rfregs[] = {
- { /* RF_A */
- .hssiparm1 = REG_FPGA0_XA_HSSI_PARM1,
- .hssiparm2 = REG_FPGA0_XA_HSSI_PARM2,
- .lssiparm = REG_FPGA0_XA_LSSI_PARM,
- .hspiread = REG_HSPI_XA_READBACK,
- .lssiread = REG_FPGA0_XA_LSSI_READBACK,
- .rf_sw_ctrl = REG_FPGA0_XA_RF_SW_CTRL,
- },
- { /* RF_B */
- .hssiparm1 = REG_FPGA0_XB_HSSI_PARM1,
- .hssiparm2 = REG_FPGA0_XB_HSSI_PARM2,
- .lssiparm = REG_FPGA0_XB_LSSI_PARM,
- .hspiread = REG_HSPI_XB_READBACK,
- .lssiread = REG_FPGA0_XB_LSSI_READBACK,
- .rf_sw_ctrl = REG_FPGA0_XB_RF_SW_CTRL,
- },
-};
-
-static const u32 rtl8xxxu_iqk_phy_iq_bb_reg[RTL8XXXU_BB_REGS] = {
- REG_OFDM0_XA_RX_IQ_IMBALANCE,
- REG_OFDM0_XB_RX_IQ_IMBALANCE,
- REG_OFDM0_ENERGY_CCA_THRES,
- REG_OFDM0_AGCR_SSI_TABLE,
- REG_OFDM0_XA_TX_IQ_IMBALANCE,
- REG_OFDM0_XB_TX_IQ_IMBALANCE,
- REG_OFDM0_XC_TX_AFE,
- REG_OFDM0_XD_TX_AFE,
- REG_OFDM0_RX_IQ_EXT_ANTA
-};
-
-static u8 rtl8xxxu_read8(struct rtl8xxxu_priv *priv, u16 addr)
-{
- struct usb_device *udev = priv->udev;
- int len;
- u8 data;
-
- mutex_lock(&priv->usb_buf_mutex);
- len = usb_control_msg(udev, usb_rcvctrlpipe(udev, 0),
- REALTEK_USB_CMD_REQ, REALTEK_USB_READ,
- addr, 0, &priv->usb_buf.val8, sizeof(u8),
- RTW_USB_CONTROL_MSG_TIMEOUT);
- data = priv->usb_buf.val8;
- mutex_unlock(&priv->usb_buf_mutex);
-
- if (rtl8xxxu_debug & RTL8XXXU_DEBUG_REG_READ)
- dev_info(&udev->dev, "%s(%04x) = 0x%02x, len %i\n",
- __func__, addr, data, len);
- return data;
-}
-
-static u16 rtl8xxxu_read16(struct rtl8xxxu_priv *priv, u16 addr)
-{
- struct usb_device *udev = priv->udev;
- int len;
- u16 data;
-
- mutex_lock(&priv->usb_buf_mutex);
- len = usb_control_msg(udev, usb_rcvctrlpipe(udev, 0),
- REALTEK_USB_CMD_REQ, REALTEK_USB_READ,
- addr, 0, &priv->usb_buf.val16, sizeof(u16),
- RTW_USB_CONTROL_MSG_TIMEOUT);
- data = le16_to_cpu(priv->usb_buf.val16);
- mutex_unlock(&priv->usb_buf_mutex);
-
- if (rtl8xxxu_debug & RTL8XXXU_DEBUG_REG_READ)
- dev_info(&udev->dev, "%s(%04x) = 0x%04x, len %i\n",
- __func__, addr, data, len);
- return data;
-}
-
-static u32 rtl8xxxu_read32(struct rtl8xxxu_priv *priv, u16 addr)
-{
- struct usb_device *udev = priv->udev;
- int len;
- u32 data;
-
- mutex_lock(&priv->usb_buf_mutex);
- len = usb_control_msg(udev, usb_rcvctrlpipe(udev, 0),
- REALTEK_USB_CMD_REQ, REALTEK_USB_READ,
- addr, 0, &priv->usb_buf.val32, sizeof(u32),
- RTW_USB_CONTROL_MSG_TIMEOUT);
- data = le32_to_cpu(priv->usb_buf.val32);
- mutex_unlock(&priv->usb_buf_mutex);
-
- if (rtl8xxxu_debug & RTL8XXXU_DEBUG_REG_READ)
- dev_info(&udev->dev, "%s(%04x) = 0x%08x, len %i\n",
- __func__, addr, data, len);
- return data;
-}
-
-static int rtl8xxxu_write8(struct rtl8xxxu_priv *priv, u16 addr, u8 val)
-{
- struct usb_device *udev = priv->udev;
- int ret;
-
- mutex_lock(&priv->usb_buf_mutex);
- priv->usb_buf.val8 = val;
- ret = usb_control_msg(udev, usb_sndctrlpipe(udev, 0),
- REALTEK_USB_CMD_REQ, REALTEK_USB_WRITE,
- addr, 0, &priv->usb_buf.val8, sizeof(u8),
- RTW_USB_CONTROL_MSG_TIMEOUT);
-
- mutex_unlock(&priv->usb_buf_mutex);
-
- if (rtl8xxxu_debug & RTL8XXXU_DEBUG_REG_WRITE)
- dev_info(&udev->dev, "%s(%04x) = 0x%02x\n",
- __func__, addr, val);
- return ret;
-}
-
-static int rtl8xxxu_write16(struct rtl8xxxu_priv *priv, u16 addr, u16 val)
-{
- struct usb_device *udev = priv->udev;
- int ret;
-
- mutex_lock(&priv->usb_buf_mutex);
- priv->usb_buf.val16 = cpu_to_le16(val);
- ret = usb_control_msg(udev, usb_sndctrlpipe(udev, 0),
- REALTEK_USB_CMD_REQ, REALTEK_USB_WRITE,
- addr, 0, &priv->usb_buf.val16, sizeof(u16),
- RTW_USB_CONTROL_MSG_TIMEOUT);
- mutex_unlock(&priv->usb_buf_mutex);
-
- if (rtl8xxxu_debug & RTL8XXXU_DEBUG_REG_WRITE)
- dev_info(&udev->dev, "%s(%04x) = 0x%04x\n",
- __func__, addr, val);
- return ret;
-}
-
-static int rtl8xxxu_write32(struct rtl8xxxu_priv *priv, u16 addr, u32 val)
-{
- struct usb_device *udev = priv->udev;
- int ret;
-
- mutex_lock(&priv->usb_buf_mutex);
- priv->usb_buf.val32 = cpu_to_le32(val);
- ret = usb_control_msg(udev, usb_sndctrlpipe(udev, 0),
- REALTEK_USB_CMD_REQ, REALTEK_USB_WRITE,
- addr, 0, &priv->usb_buf.val32, sizeof(u32),
- RTW_USB_CONTROL_MSG_TIMEOUT);
- mutex_unlock(&priv->usb_buf_mutex);
-
- if (rtl8xxxu_debug & RTL8XXXU_DEBUG_REG_WRITE)
- dev_info(&udev->dev, "%s(%04x) = 0x%08x\n",
- __func__, addr, val);
- return ret;
-}
-
-static int
-rtl8xxxu_writeN(struct rtl8xxxu_priv *priv, u16 addr, u8 *buf, u16 len)
-{
- struct usb_device *udev = priv->udev;
- int blocksize = priv->fops->writeN_block_size;
- int ret, i, count, remainder;
-
- count = len / blocksize;
- remainder = len % blocksize;
-
- for (i = 0; i < count; i++) {
- ret = usb_control_msg(udev, usb_sndctrlpipe(udev, 0),
- REALTEK_USB_CMD_REQ, REALTEK_USB_WRITE,
- addr, 0, buf, blocksize,
- RTW_USB_CONTROL_MSG_TIMEOUT);
- if (ret != blocksize)
- goto write_error;
-
- addr += blocksize;
- buf += blocksize;
- }
-
- if (remainder) {
- ret = usb_control_msg(udev, usb_sndctrlpipe(udev, 0),
- REALTEK_USB_CMD_REQ, REALTEK_USB_WRITE,
- addr, 0, buf, remainder,
- RTW_USB_CONTROL_MSG_TIMEOUT);
- if (ret != remainder)
- goto write_error;
- }
-
- return len;
-
-write_error:
- dev_info(&udev->dev,
- "%s: Failed to write block at addr: %04x size: %04x\n",
- __func__, addr, blocksize);
- return -EAGAIN;
-}
-
-static u32 rtl8xxxu_read_rfreg(struct rtl8xxxu_priv *priv,
- enum rtl8xxxu_rfpath path, u8 reg)
-{
- u32 hssia, val32, retval;
-
- hssia = rtl8xxxu_read32(priv, REG_FPGA0_XA_HSSI_PARM2);
- if (path != RF_A)
- val32 = rtl8xxxu_read32(priv, rtl8xxxu_rfregs[path].hssiparm2);
- else
- val32 = hssia;
-
- val32 &= ~FPGA0_HSSI_PARM2_ADDR_MASK;
- val32 |= (reg << FPGA0_HSSI_PARM2_ADDR_SHIFT);
- val32 |= FPGA0_HSSI_PARM2_EDGE_READ;
- hssia &= ~FPGA0_HSSI_PARM2_EDGE_READ;
- rtl8xxxu_write32(priv, REG_FPGA0_XA_HSSI_PARM2, hssia);
-
- udelay(10);
-
- rtl8xxxu_write32(priv, rtl8xxxu_rfregs[path].hssiparm2, val32);
- udelay(100);
-
- hssia |= FPGA0_HSSI_PARM2_EDGE_READ;
- rtl8xxxu_write32(priv, REG_FPGA0_XA_HSSI_PARM2, hssia);
- udelay(10);
-
- val32 = rtl8xxxu_read32(priv, rtl8xxxu_rfregs[path].hssiparm1);
- if (val32 & FPGA0_HSSI_PARM1_PI)
- retval = rtl8xxxu_read32(priv, rtl8xxxu_rfregs[path].hspiread);
- else
- retval = rtl8xxxu_read32(priv, rtl8xxxu_rfregs[path].lssiread);
-
- retval &= 0xfffff;
-
- if (rtl8xxxu_debug & RTL8XXXU_DEBUG_RFREG_READ)
- dev_info(&priv->udev->dev, "%s(%02x) = 0x%06x\n",
- __func__, reg, retval);
- return retval;
-}
-
-/*
- * The RTL8723BU driver indicates that registers 0xb2 and 0xb6 can
- * have write issues in high temperature conditions. We may have to
- * retry writing them.
- */
-static int rtl8xxxu_write_rfreg(struct rtl8xxxu_priv *priv,
- enum rtl8xxxu_rfpath path, u8 reg, u32 data)
-{
- int ret, retval;
- u32 dataaddr, val32;
-
- if (rtl8xxxu_debug & RTL8XXXU_DEBUG_RFREG_WRITE)
- dev_info(&priv->udev->dev, "%s(%02x) = 0x%06x\n",
- __func__, reg, data);
-
- data &= FPGA0_LSSI_PARM_DATA_MASK;
- dataaddr = (reg << FPGA0_LSSI_PARM_ADDR_SHIFT) | data;
-
- if (priv->rtl_chip == RTL8192E) {
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_POWER_SAVE);
- val32 &= ~0x20000;
- rtl8xxxu_write32(priv, REG_FPGA0_POWER_SAVE, val32);
- }
-
- /* Use XB for path B */
- ret = rtl8xxxu_write32(priv, rtl8xxxu_rfregs[path].lssiparm, dataaddr);
- if (ret != sizeof(dataaddr))
- retval = -EIO;
- else
- retval = 0;
-
- udelay(1);
-
- if (priv->rtl_chip == RTL8192E) {
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_POWER_SAVE);
- val32 |= 0x20000;
- rtl8xxxu_write32(priv, REG_FPGA0_POWER_SAVE, val32);
- }
-
- return retval;
-}
-
-static int rtl8723a_h2c_cmd(struct rtl8xxxu_priv *priv,
- struct h2c_cmd *h2c, int len)
-{
- struct device *dev = &priv->udev->dev;
- int mbox_nr, retry, retval = 0;
- int mbox_reg, mbox_ext_reg;
- u8 val8;
-
- mutex_lock(&priv->h2c_mutex);
-
- mbox_nr = priv->next_mbox;
- mbox_reg = REG_HMBOX_0 + (mbox_nr * 4);
- mbox_ext_reg = priv->fops->mbox_ext_reg +
- (mbox_nr * priv->fops->mbox_ext_width);
-
- /*
- * MBOX ready?
- */
- retry = 100;
- do {
- val8 = rtl8xxxu_read8(priv, REG_HMTFR);
- if (!(val8 & BIT(mbox_nr)))
- break;
- } while (retry--);
-
- if (!retry) {
- dev_info(dev, "%s: Mailbox busy\n", __func__);
- retval = -EBUSY;
- goto error;
- }
-
- /*
- * Need to swap as it's being swapped again by rtl8xxxu_write16/32()
- */
- if (len > sizeof(u32)) {
- if (priv->fops->mbox_ext_width == 4) {
- rtl8xxxu_write32(priv, mbox_ext_reg,
- le32_to_cpu(h2c->raw_wide.ext));
- if (rtl8xxxu_debug & RTL8XXXU_DEBUG_H2C)
- dev_info(dev, "H2C_EXT %08x\n",
- le32_to_cpu(h2c->raw_wide.ext));
- } else {
- rtl8xxxu_write16(priv, mbox_ext_reg,
- le16_to_cpu(h2c->raw.ext));
- if (rtl8xxxu_debug & RTL8XXXU_DEBUG_H2C)
- dev_info(dev, "H2C_EXT %04x\n",
- le16_to_cpu(h2c->raw.ext));
- }
- }
- rtl8xxxu_write32(priv, mbox_reg, le32_to_cpu(h2c->raw.data));
- if (rtl8xxxu_debug & RTL8XXXU_DEBUG_H2C)
- dev_info(dev, "H2C %08x\n", le32_to_cpu(h2c->raw.data));
-
- priv->next_mbox = (mbox_nr + 1) % H2C_MAX_MBOX;
-
-error:
- mutex_unlock(&priv->h2c_mutex);
- return retval;
-}
-
-static void rtl8723bu_write_btreg(struct rtl8xxxu_priv *priv, u8 reg, u8 data)
-{
- struct h2c_cmd h2c;
- int reqnum = 0;
-
- memset(&h2c, 0, sizeof(struct h2c_cmd));
- h2c.bt_mp_oper.cmd = H2C_8723B_BT_MP_OPER;
- h2c.bt_mp_oper.operreq = 0 | (reqnum << 4);
- h2c.bt_mp_oper.opcode = BT_MP_OP_WRITE_REG_VALUE;
- h2c.bt_mp_oper.data = data;
- rtl8723a_h2c_cmd(priv, &h2c, sizeof(h2c.bt_mp_oper));
-
- reqnum++;
- memset(&h2c, 0, sizeof(struct h2c_cmd));
- h2c.bt_mp_oper.cmd = H2C_8723B_BT_MP_OPER;
- h2c.bt_mp_oper.operreq = 0 | (reqnum << 4);
- h2c.bt_mp_oper.opcode = BT_MP_OP_WRITE_REG_VALUE;
- h2c.bt_mp_oper.addr = reg;
- rtl8723a_h2c_cmd(priv, &h2c, sizeof(h2c.bt_mp_oper));
-}
-
-static void rtl8xxxu_gen1_enable_rf(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
- u32 val32;
-
- val8 = rtl8xxxu_read8(priv, REG_SPS0_CTRL);
- val8 |= BIT(0) | BIT(3);
- rtl8xxxu_write8(priv, REG_SPS0_CTRL, val8);
-
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_XAB_RF_PARM);
- val32 &= ~(BIT(4) | BIT(5));
- val32 |= BIT(3);
- if (priv->rf_paths == 2) {
- val32 &= ~(BIT(20) | BIT(21));
- val32 |= BIT(19);
- }
- rtl8xxxu_write32(priv, REG_FPGA0_XAB_RF_PARM, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_TRX_PATH_ENABLE);
- val32 &= ~OFDM_RF_PATH_TX_MASK;
- if (priv->tx_paths == 2)
- val32 |= OFDM_RF_PATH_TX_A | OFDM_RF_PATH_TX_B;
- else if (priv->rtl_chip == RTL8192C || priv->rtl_chip == RTL8191C)
- val32 |= OFDM_RF_PATH_TX_B;
- else
- val32 |= OFDM_RF_PATH_TX_A;
- rtl8xxxu_write32(priv, REG_OFDM0_TRX_PATH_ENABLE, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_RF_MODE);
- val32 &= ~FPGA_RF_MODE_JAPAN;
- rtl8xxxu_write32(priv, REG_FPGA0_RF_MODE, val32);
-
- if (priv->rf_paths == 2)
- rtl8xxxu_write32(priv, REG_RX_WAIT_CCA, 0x63db25a0);
- else
- rtl8xxxu_write32(priv, REG_RX_WAIT_CCA, 0x631b25a0);
-
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_AC, 0x32d95);
- if (priv->rf_paths == 2)
- rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_AC, 0x32d95);
-
- rtl8xxxu_write8(priv, REG_TXPAUSE, 0x00);
-}
-
-static void rtl8xxxu_gen1_disable_rf(struct rtl8xxxu_priv *priv)
-{
- u8 sps0;
- u32 val32;
-
- sps0 = rtl8xxxu_read8(priv, REG_SPS0_CTRL);
-
- /* RF RX code for preamble power saving */
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_XAB_RF_PARM);
- val32 &= ~(BIT(3) | BIT(4) | BIT(5));
- if (priv->rf_paths == 2)
- val32 &= ~(BIT(19) | BIT(20) | BIT(21));
- rtl8xxxu_write32(priv, REG_FPGA0_XAB_RF_PARM, val32);
-
- /* Disable TX for four paths */
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_TRX_PATH_ENABLE);
- val32 &= ~OFDM_RF_PATH_TX_MASK;
- rtl8xxxu_write32(priv, REG_OFDM0_TRX_PATH_ENABLE, val32);
-
- /* Enable power saving */
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_RF_MODE);
- val32 |= FPGA_RF_MODE_JAPAN;
- rtl8xxxu_write32(priv, REG_FPGA0_RF_MODE, val32);
-
- /* AFE control register to power down bits [30:22] */
- if (priv->rf_paths == 2)
- rtl8xxxu_write32(priv, REG_RX_WAIT_CCA, 0x00db25a0);
- else
- rtl8xxxu_write32(priv, REG_RX_WAIT_CCA, 0x001b25a0);
-
- /* Power down RF module */
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_AC, 0);
- if (priv->rf_paths == 2)
- rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_AC, 0);
-
- sps0 &= ~(BIT(0) | BIT(3));
- rtl8xxxu_write8(priv, REG_SPS0_CTRL, sps0);
-}
-
-
-static void rtl8723a_stop_tx_beacon(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
-
- val8 = rtl8xxxu_read8(priv, REG_FWHW_TXQ_CTRL + 2);
- val8 &= ~BIT(6);
- rtl8xxxu_write8(priv, REG_FWHW_TXQ_CTRL + 2, val8);
-
- rtl8xxxu_write8(priv, REG_TBTT_PROHIBIT + 1, 0x64);
- val8 = rtl8xxxu_read8(priv, REG_TBTT_PROHIBIT + 2);
- val8 &= ~BIT(0);
- rtl8xxxu_write8(priv, REG_TBTT_PROHIBIT + 2, val8);
-}
-
-
-/*
- * The rtl8723a has 3 channel groups for it's efuse settings. It only
- * supports the 2.4GHz band, so channels 1 - 14:
- * group 0: channels 1 - 3
- * group 1: channels 4 - 9
- * group 2: channels 10 - 14
- *
- * Note: We index from 0 in the code
- */
-static int rtl8723a_channel_to_group(int channel)
-{
- int group;
-
- if (channel < 4)
- group = 0;
- else if (channel < 10)
- group = 1;
- else
- group = 2;
-
- return group;
-}
-
-/*
- * Valid for rtl8723bu and rtl8192eu
- */
-static int rtl8xxxu_gen2_channel_to_group(int channel)
-{
- int group;
-
- if (channel < 3)
- group = 0;
- else if (channel < 6)
- group = 1;
- else if (channel < 9)
- group = 2;
- else if (channel < 12)
- group = 3;
- else
- group = 4;
-
- return group;
-}
-
-static void rtl8xxxu_gen1_config_channel(struct ieee80211_hw *hw)
-{
- struct rtl8xxxu_priv *priv = hw->priv;
- u32 val32, rsr;
- u8 val8, opmode;
- bool ht = true;
- int sec_ch_above, channel;
- int i;
-
- opmode = rtl8xxxu_read8(priv, REG_BW_OPMODE);
- rsr = rtl8xxxu_read32(priv, REG_RESPONSE_RATE_SET);
- channel = hw->conf.chandef.chan->hw_value;
-
- switch (hw->conf.chandef.width) {
- case NL80211_CHAN_WIDTH_20_NOHT:
- ht = false;
- case NL80211_CHAN_WIDTH_20:
- opmode |= BW_OPMODE_20MHZ;
- rtl8xxxu_write8(priv, REG_BW_OPMODE, opmode);
-
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_RF_MODE);
- val32 &= ~FPGA_RF_MODE;
- rtl8xxxu_write32(priv, REG_FPGA0_RF_MODE, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_FPGA1_RF_MODE);
- val32 &= ~FPGA_RF_MODE;
- rtl8xxxu_write32(priv, REG_FPGA1_RF_MODE, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_ANALOG2);
- val32 |= FPGA0_ANALOG2_20MHZ;
- rtl8xxxu_write32(priv, REG_FPGA0_ANALOG2, val32);
- break;
- case NL80211_CHAN_WIDTH_40:
- if (hw->conf.chandef.center_freq1 >
- hw->conf.chandef.chan->center_freq) {
- sec_ch_above = 1;
- channel += 2;
- } else {
- sec_ch_above = 0;
- channel -= 2;
- }
-
- opmode &= ~BW_OPMODE_20MHZ;
- rtl8xxxu_write8(priv, REG_BW_OPMODE, opmode);
- rsr &= ~RSR_RSC_BANDWIDTH_40M;
- if (sec_ch_above)
- rsr |= RSR_RSC_UPPER_SUB_CHANNEL;
- else
- rsr |= RSR_RSC_LOWER_SUB_CHANNEL;
- rtl8xxxu_write32(priv, REG_RESPONSE_RATE_SET, rsr);
-
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_RF_MODE);
- val32 |= FPGA_RF_MODE;
- rtl8xxxu_write32(priv, REG_FPGA0_RF_MODE, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_FPGA1_RF_MODE);
- val32 |= FPGA_RF_MODE;
- rtl8xxxu_write32(priv, REG_FPGA1_RF_MODE, val32);
-
- /*
- * Set Control channel to upper or lower. These settings
- * are required only for 40MHz
- */
- val32 = rtl8xxxu_read32(priv, REG_CCK0_SYSTEM);
- val32 &= ~CCK0_SIDEBAND;
- if (!sec_ch_above)
- val32 |= CCK0_SIDEBAND;
- rtl8xxxu_write32(priv, REG_CCK0_SYSTEM, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM1_LSTF);
- val32 &= ~OFDM_LSTF_PRIME_CH_MASK; /* 0xc00 */
- if (sec_ch_above)
- val32 |= OFDM_LSTF_PRIME_CH_LOW;
- else
- val32 |= OFDM_LSTF_PRIME_CH_HIGH;
- rtl8xxxu_write32(priv, REG_OFDM1_LSTF, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_ANALOG2);
- val32 &= ~FPGA0_ANALOG2_20MHZ;
- rtl8xxxu_write32(priv, REG_FPGA0_ANALOG2, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_POWER_SAVE);
- val32 &= ~(FPGA0_PS_LOWER_CHANNEL | FPGA0_PS_UPPER_CHANNEL);
- if (sec_ch_above)
- val32 |= FPGA0_PS_UPPER_CHANNEL;
- else
- val32 |= FPGA0_PS_LOWER_CHANNEL;
- rtl8xxxu_write32(priv, REG_FPGA0_POWER_SAVE, val32);
- break;
-
- default:
- break;
- }
-
- for (i = RF_A; i < priv->rf_paths; i++) {
- val32 = rtl8xxxu_read_rfreg(priv, i, RF6052_REG_MODE_AG);
- val32 &= ~MODE_AG_CHANNEL_MASK;
- val32 |= channel;
- rtl8xxxu_write_rfreg(priv, i, RF6052_REG_MODE_AG, val32);
- }
-
- if (ht)
- val8 = 0x0e;
- else
- val8 = 0x0a;
-
- rtl8xxxu_write8(priv, REG_SIFS_CCK + 1, val8);
- rtl8xxxu_write8(priv, REG_SIFS_OFDM + 1, val8);
-
- rtl8xxxu_write16(priv, REG_R2T_SIFS, 0x0808);
- rtl8xxxu_write16(priv, REG_T2T_SIFS, 0x0a0a);
-
- for (i = RF_A; i < priv->rf_paths; i++) {
- val32 = rtl8xxxu_read_rfreg(priv, i, RF6052_REG_MODE_AG);
- if (hw->conf.chandef.width == NL80211_CHAN_WIDTH_40)
- val32 &= ~MODE_AG_CHANNEL_20MHZ;
- else
- val32 |= MODE_AG_CHANNEL_20MHZ;
- rtl8xxxu_write_rfreg(priv, i, RF6052_REG_MODE_AG, val32);
- }
-}
-
-static void rtl8xxxu_gen2_config_channel(struct ieee80211_hw *hw)
-{
- struct rtl8xxxu_priv *priv = hw->priv;
- u32 val32, rsr;
- u8 val8, subchannel;
- u16 rf_mode_bw;
- bool ht = true;
- int sec_ch_above, channel;
- int i;
-
- rf_mode_bw = rtl8xxxu_read16(priv, REG_WMAC_TRXPTCL_CTL);
- rf_mode_bw &= ~WMAC_TRXPTCL_CTL_BW_MASK;
- rsr = rtl8xxxu_read32(priv, REG_RESPONSE_RATE_SET);
- channel = hw->conf.chandef.chan->hw_value;
-
-/* Hack */
- subchannel = 0;
-
- switch (hw->conf.chandef.width) {
- case NL80211_CHAN_WIDTH_20_NOHT:
- ht = false;
- case NL80211_CHAN_WIDTH_20:
- rf_mode_bw |= WMAC_TRXPTCL_CTL_BW_20;
- subchannel = 0;
-
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_RF_MODE);
- val32 &= ~FPGA_RF_MODE;
- rtl8xxxu_write32(priv, REG_FPGA0_RF_MODE, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_FPGA1_RF_MODE);
- val32 &= ~FPGA_RF_MODE;
- rtl8xxxu_write32(priv, REG_FPGA1_RF_MODE, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_TX_PSDO_NOISE_WEIGHT);
- val32 &= ~(BIT(30) | BIT(31));
- rtl8xxxu_write32(priv, REG_OFDM0_TX_PSDO_NOISE_WEIGHT, val32);
-
- break;
- case NL80211_CHAN_WIDTH_40:
- rf_mode_bw |= WMAC_TRXPTCL_CTL_BW_40;
-
- if (hw->conf.chandef.center_freq1 >
- hw->conf.chandef.chan->center_freq) {
- sec_ch_above = 1;
- channel += 2;
- } else {
- sec_ch_above = 0;
- channel -= 2;
- }
-
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_RF_MODE);
- val32 |= FPGA_RF_MODE;
- rtl8xxxu_write32(priv, REG_FPGA0_RF_MODE, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_FPGA1_RF_MODE);
- val32 |= FPGA_RF_MODE;
- rtl8xxxu_write32(priv, REG_FPGA1_RF_MODE, val32);
-
- /*
- * Set Control channel to upper or lower. These settings
- * are required only for 40MHz
- */
- val32 = rtl8xxxu_read32(priv, REG_CCK0_SYSTEM);
- val32 &= ~CCK0_SIDEBAND;
- if (!sec_ch_above)
- val32 |= CCK0_SIDEBAND;
- rtl8xxxu_write32(priv, REG_CCK0_SYSTEM, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM1_LSTF);
- val32 &= ~OFDM_LSTF_PRIME_CH_MASK; /* 0xc00 */
- if (sec_ch_above)
- val32 |= OFDM_LSTF_PRIME_CH_LOW;
- else
- val32 |= OFDM_LSTF_PRIME_CH_HIGH;
- rtl8xxxu_write32(priv, REG_OFDM1_LSTF, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_POWER_SAVE);
- val32 &= ~(FPGA0_PS_LOWER_CHANNEL | FPGA0_PS_UPPER_CHANNEL);
- if (sec_ch_above)
- val32 |= FPGA0_PS_UPPER_CHANNEL;
- else
- val32 |= FPGA0_PS_LOWER_CHANNEL;
- rtl8xxxu_write32(priv, REG_FPGA0_POWER_SAVE, val32);
- break;
- case NL80211_CHAN_WIDTH_80:
- rf_mode_bw |= WMAC_TRXPTCL_CTL_BW_80;
- break;
- default:
- break;
- }
-
- for (i = RF_A; i < priv->rf_paths; i++) {
- val32 = rtl8xxxu_read_rfreg(priv, i, RF6052_REG_MODE_AG);
- val32 &= ~MODE_AG_CHANNEL_MASK;
- val32 |= channel;
- rtl8xxxu_write_rfreg(priv, i, RF6052_REG_MODE_AG, val32);
- }
-
- rtl8xxxu_write16(priv, REG_WMAC_TRXPTCL_CTL, rf_mode_bw);
- rtl8xxxu_write8(priv, REG_DATA_SUBCHANNEL, subchannel);
-
- if (ht)
- val8 = 0x0e;
- else
- val8 = 0x0a;
-
- rtl8xxxu_write8(priv, REG_SIFS_CCK + 1, val8);
- rtl8xxxu_write8(priv, REG_SIFS_OFDM + 1, val8);
-
- rtl8xxxu_write16(priv, REG_R2T_SIFS, 0x0808);
- rtl8xxxu_write16(priv, REG_T2T_SIFS, 0x0a0a);
-
- for (i = RF_A; i < priv->rf_paths; i++) {
- val32 = rtl8xxxu_read_rfreg(priv, i, RF6052_REG_MODE_AG);
- val32 &= ~MODE_AG_BW_MASK;
- switch(hw->conf.chandef.width) {
- case NL80211_CHAN_WIDTH_80:
- val32 |= MODE_AG_BW_80MHZ_8723B;
- break;
- case NL80211_CHAN_WIDTH_40:
- val32 |= MODE_AG_BW_40MHZ_8723B;
- break;
- default:
- val32 |= MODE_AG_BW_20MHZ_8723B;
- break;
- }
- rtl8xxxu_write_rfreg(priv, i, RF6052_REG_MODE_AG, val32);
- }
-}
-
-static void
-rtl8xxxu_gen1_set_tx_power(struct rtl8xxxu_priv *priv, int channel, bool ht40)
-{
- struct rtl8xxxu_power_base *power_base = priv->power_base;
- u8 cck[RTL8723A_MAX_RF_PATHS], ofdm[RTL8723A_MAX_RF_PATHS];
- u8 ofdmbase[RTL8723A_MAX_RF_PATHS], mcsbase[RTL8723A_MAX_RF_PATHS];
- u32 val32, ofdm_a, ofdm_b, mcs_a, mcs_b;
- u8 val8;
- int group, i;
-
- group = rtl8723a_channel_to_group(channel);
-
- cck[0] = priv->cck_tx_power_index_A[group] - 1;
- cck[1] = priv->cck_tx_power_index_B[group] - 1;
-
- if (priv->hi_pa) {
- if (cck[0] > 0x20)
- cck[0] = 0x20;
- if (cck[1] > 0x20)
- cck[1] = 0x20;
- }
-
- ofdm[0] = priv->ht40_1s_tx_power_index_A[group];
- ofdm[1] = priv->ht40_1s_tx_power_index_B[group];
- if (ofdm[0])
- ofdm[0] -= 1;
- if (ofdm[1])
- ofdm[1] -= 1;
-
- ofdmbase[0] = ofdm[0] + priv->ofdm_tx_power_index_diff[group].a;
- ofdmbase[1] = ofdm[1] + priv->ofdm_tx_power_index_diff[group].b;
-
- mcsbase[0] = ofdm[0];
- mcsbase[1] = ofdm[1];
- if (!ht40) {
- mcsbase[0] += priv->ht20_tx_power_index_diff[group].a;
- mcsbase[1] += priv->ht20_tx_power_index_diff[group].b;
- }
-
- if (priv->tx_paths > 1) {
- if (ofdm[0] > priv->ht40_2s_tx_power_index_diff[group].a)
- ofdm[0] -= priv->ht40_2s_tx_power_index_diff[group].a;
- if (ofdm[1] > priv->ht40_2s_tx_power_index_diff[group].b)
- ofdm[1] -= priv->ht40_2s_tx_power_index_diff[group].b;
- }
-
- if (rtl8xxxu_debug & RTL8XXXU_DEBUG_CHANNEL)
- dev_info(&priv->udev->dev,
- "%s: Setting TX power CCK A: %02x, "
- "CCK B: %02x, OFDM A: %02x, OFDM B: %02x\n",
- __func__, cck[0], cck[1], ofdm[0], ofdm[1]);
-
- for (i = 0; i < RTL8723A_MAX_RF_PATHS; i++) {
- if (cck[i] > RF6052_MAX_TX_PWR)
- cck[i] = RF6052_MAX_TX_PWR;
- if (ofdm[i] > RF6052_MAX_TX_PWR)
- ofdm[i] = RF6052_MAX_TX_PWR;
- }
-
- val32 = rtl8xxxu_read32(priv, REG_TX_AGC_A_CCK1_MCS32);
- val32 &= 0xffff00ff;
- val32 |= (cck[0] << 8);
- rtl8xxxu_write32(priv, REG_TX_AGC_A_CCK1_MCS32, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_TX_AGC_B_CCK11_A_CCK2_11);
- val32 &= 0xff;
- val32 |= ((cck[0] << 8) | (cck[0] << 16) | (cck[0] << 24));
- rtl8xxxu_write32(priv, REG_TX_AGC_B_CCK11_A_CCK2_11, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_TX_AGC_B_CCK11_A_CCK2_11);
- val32 &= 0xffffff00;
- val32 |= cck[1];
- rtl8xxxu_write32(priv, REG_TX_AGC_B_CCK11_A_CCK2_11, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_TX_AGC_B_CCK1_55_MCS32);
- val32 &= 0xff;
- val32 |= ((cck[1] << 8) | (cck[1] << 16) | (cck[1] << 24));
- rtl8xxxu_write32(priv, REG_TX_AGC_B_CCK1_55_MCS32, val32);
-
- ofdm_a = ofdmbase[0] | ofdmbase[0] << 8 |
- ofdmbase[0] << 16 | ofdmbase[0] << 24;
- ofdm_b = ofdmbase[1] | ofdmbase[1] << 8 |
- ofdmbase[1] << 16 | ofdmbase[1] << 24;
-
- rtl8xxxu_write32(priv, REG_TX_AGC_A_RATE18_06,
- ofdm_a + power_base->reg_0e00);
- rtl8xxxu_write32(priv, REG_TX_AGC_B_RATE18_06,
- ofdm_b + power_base->reg_0830);
-
- rtl8xxxu_write32(priv, REG_TX_AGC_A_RATE54_24,
- ofdm_a + power_base->reg_0e04);
- rtl8xxxu_write32(priv, REG_TX_AGC_B_RATE54_24,
- ofdm_b + power_base->reg_0834);
-
- mcs_a = mcsbase[0] | mcsbase[0] << 8 |
- mcsbase[0] << 16 | mcsbase[0] << 24;
- mcs_b = mcsbase[1] | mcsbase[1] << 8 |
- mcsbase[1] << 16 | mcsbase[1] << 24;
-
- rtl8xxxu_write32(priv, REG_TX_AGC_A_MCS03_MCS00,
- mcs_a + power_base->reg_0e10);
- rtl8xxxu_write32(priv, REG_TX_AGC_B_MCS03_MCS00,
- mcs_b + power_base->reg_083c);
-
- rtl8xxxu_write32(priv, REG_TX_AGC_A_MCS07_MCS04,
- mcs_a + power_base->reg_0e14);
- rtl8xxxu_write32(priv, REG_TX_AGC_B_MCS07_MCS04,
- mcs_b + power_base->reg_0848);
-
- rtl8xxxu_write32(priv, REG_TX_AGC_A_MCS11_MCS08,
- mcs_a + power_base->reg_0e18);
- rtl8xxxu_write32(priv, REG_TX_AGC_B_MCS11_MCS08,
- mcs_b + power_base->reg_084c);
-
- rtl8xxxu_write32(priv, REG_TX_AGC_A_MCS15_MCS12,
- mcs_a + power_base->reg_0e1c);
- for (i = 0; i < 3; i++) {
- if (i != 2)
- val8 = (mcsbase[0] > 8) ? (mcsbase[0] - 8) : 0;
- else
- val8 = (mcsbase[0] > 6) ? (mcsbase[0] - 6) : 0;
- rtl8xxxu_write8(priv, REG_OFDM0_XC_TX_IQ_IMBALANCE + i, val8);
- }
- rtl8xxxu_write32(priv, REG_TX_AGC_B_MCS15_MCS12,
- mcs_b + power_base->reg_0868);
- for (i = 0; i < 3; i++) {
- if (i != 2)
- val8 = (mcsbase[1] > 8) ? (mcsbase[1] - 8) : 0;
- else
- val8 = (mcsbase[1] > 6) ? (mcsbase[1] - 6) : 0;
- rtl8xxxu_write8(priv, REG_OFDM0_XD_TX_IQ_IMBALANCE + i, val8);
- }
-}
-
-static void
-rtl8723b_set_tx_power(struct rtl8xxxu_priv *priv, int channel, bool ht40)
-{
- u32 val32, ofdm, mcs;
- u8 cck, ofdmbase, mcsbase;
- int group, tx_idx;
-
- tx_idx = 0;
- group = rtl8xxxu_gen2_channel_to_group(channel);
-
- cck = priv->cck_tx_power_index_B[group];
- val32 = rtl8xxxu_read32(priv, REG_TX_AGC_A_CCK1_MCS32);
- val32 &= 0xffff00ff;
- val32 |= (cck << 8);
- rtl8xxxu_write32(priv, REG_TX_AGC_A_CCK1_MCS32, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_TX_AGC_B_CCK11_A_CCK2_11);
- val32 &= 0xff;
- val32 |= ((cck << 8) | (cck << 16) | (cck << 24));
- rtl8xxxu_write32(priv, REG_TX_AGC_B_CCK11_A_CCK2_11, val32);
-
- ofdmbase = priv->ht40_1s_tx_power_index_B[group];
- ofdmbase += priv->ofdm_tx_power_diff[tx_idx].b;
- ofdm = ofdmbase | ofdmbase << 8 | ofdmbase << 16 | ofdmbase << 24;
-
- rtl8xxxu_write32(priv, REG_TX_AGC_A_RATE18_06, ofdm);
- rtl8xxxu_write32(priv, REG_TX_AGC_A_RATE54_24, ofdm);
-
- mcsbase = priv->ht40_1s_tx_power_index_B[group];
- if (ht40)
- mcsbase += priv->ht40_tx_power_diff[tx_idx++].b;
- else
- mcsbase += priv->ht20_tx_power_diff[tx_idx++].b;
- mcs = mcsbase | mcsbase << 8 | mcsbase << 16 | mcsbase << 24;
-
- rtl8xxxu_write32(priv, REG_TX_AGC_A_MCS03_MCS00, mcs);
- rtl8xxxu_write32(priv, REG_TX_AGC_A_MCS07_MCS04, mcs);
-}
-
-static void
-rtl8192e_set_tx_power(struct rtl8xxxu_priv *priv, int channel, bool ht40)
-{
- u32 val32, ofdm, mcs;
- u8 cck, ofdmbase, mcsbase;
- int group, tx_idx;
-
- tx_idx = 0;
- group = rtl8xxxu_gen2_channel_to_group(channel);
-
- cck = priv->cck_tx_power_index_A[group];
-
- val32 = rtl8xxxu_read32(priv, REG_TX_AGC_A_CCK1_MCS32);
- val32 &= 0xffff00ff;
- val32 |= (cck << 8);
- rtl8xxxu_write32(priv, REG_TX_AGC_A_CCK1_MCS32, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_TX_AGC_B_CCK11_A_CCK2_11);
- val32 &= 0xff;
- val32 |= ((cck << 8) | (cck << 16) | (cck << 24));
- rtl8xxxu_write32(priv, REG_TX_AGC_B_CCK11_A_CCK2_11, val32);
-
- ofdmbase = priv->ht40_1s_tx_power_index_A[group];
- ofdmbase += priv->ofdm_tx_power_diff[tx_idx].a;
- ofdm = ofdmbase | ofdmbase << 8 | ofdmbase << 16 | ofdmbase << 24;
-
- rtl8xxxu_write32(priv, REG_TX_AGC_A_RATE18_06, ofdm);
- rtl8xxxu_write32(priv, REG_TX_AGC_A_RATE54_24, ofdm);
-
- mcsbase = priv->ht40_1s_tx_power_index_A[group];
- if (ht40)
- mcsbase += priv->ht40_tx_power_diff[tx_idx++].a;
- else
- mcsbase += priv->ht20_tx_power_diff[tx_idx++].a;
- mcs = mcsbase | mcsbase << 8 | mcsbase << 16 | mcsbase << 24;
-
- rtl8xxxu_write32(priv, REG_TX_AGC_A_MCS03_MCS00, mcs);
- rtl8xxxu_write32(priv, REG_TX_AGC_A_MCS07_MCS04, mcs);
- rtl8xxxu_write32(priv, REG_TX_AGC_A_MCS11_MCS08, mcs);
- rtl8xxxu_write32(priv, REG_TX_AGC_A_MCS15_MCS12, mcs);
-
- if (priv->tx_paths > 1) {
- cck = priv->cck_tx_power_index_B[group];
-
- val32 = rtl8xxxu_read32(priv, REG_TX_AGC_B_CCK1_55_MCS32);
- val32 &= 0xff;
- val32 |= ((cck << 8) | (cck << 16) | (cck << 24));
- rtl8xxxu_write32(priv, REG_TX_AGC_B_CCK1_55_MCS32, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_TX_AGC_B_CCK11_A_CCK2_11);
- val32 &= 0xffffff00;
- val32 |= cck;
- rtl8xxxu_write32(priv, REG_TX_AGC_B_CCK11_A_CCK2_11, val32);
-
- ofdmbase = priv->ht40_1s_tx_power_index_B[group];
- ofdmbase += priv->ofdm_tx_power_diff[tx_idx].b;
- ofdm = ofdmbase | ofdmbase << 8 |
- ofdmbase << 16 | ofdmbase << 24;
-
- rtl8xxxu_write32(priv, REG_TX_AGC_B_RATE18_06, ofdm);
- rtl8xxxu_write32(priv, REG_TX_AGC_B_RATE54_24, ofdm);
-
- mcsbase = priv->ht40_1s_tx_power_index_B[group];
- if (ht40)
- mcsbase += priv->ht40_tx_power_diff[tx_idx++].b;
- else
- mcsbase += priv->ht20_tx_power_diff[tx_idx++].b;
- mcs = mcsbase | mcsbase << 8 | mcsbase << 16 | mcsbase << 24;
-
- rtl8xxxu_write32(priv, REG_TX_AGC_B_MCS03_MCS00, mcs);
- rtl8xxxu_write32(priv, REG_TX_AGC_B_MCS07_MCS04, mcs);
- rtl8xxxu_write32(priv, REG_TX_AGC_B_MCS11_MCS08, mcs);
- rtl8xxxu_write32(priv, REG_TX_AGC_B_MCS15_MCS12, mcs);
- }
-}
-
-static void rtl8xxxu_set_linktype(struct rtl8xxxu_priv *priv,
- enum nl80211_iftype linktype)
-{
- u8 val8;
-
- val8 = rtl8xxxu_read8(priv, REG_MSR);
- val8 &= ~MSR_LINKTYPE_MASK;
-
- switch (linktype) {
- case NL80211_IFTYPE_UNSPECIFIED:
- val8 |= MSR_LINKTYPE_NONE;
- break;
- case NL80211_IFTYPE_ADHOC:
- val8 |= MSR_LINKTYPE_ADHOC;
- break;
- case NL80211_IFTYPE_STATION:
- val8 |= MSR_LINKTYPE_STATION;
- break;
- case NL80211_IFTYPE_AP:
- val8 |= MSR_LINKTYPE_AP;
- break;
- default:
- goto out;
- }
-
- rtl8xxxu_write8(priv, REG_MSR, val8);
-out:
- return;
-}
-
-static void
-rtl8xxxu_set_retry(struct rtl8xxxu_priv *priv, u16 short_retry, u16 long_retry)
-{
- u16 val16;
-
- val16 = ((short_retry << RETRY_LIMIT_SHORT_SHIFT) &
- RETRY_LIMIT_SHORT_MASK) |
- ((long_retry << RETRY_LIMIT_LONG_SHIFT) &
- RETRY_LIMIT_LONG_MASK);
-
- rtl8xxxu_write16(priv, REG_RETRY_LIMIT, val16);
-}
-
-static void
-rtl8xxxu_set_spec_sifs(struct rtl8xxxu_priv *priv, u16 cck, u16 ofdm)
-{
- u16 val16;
-
- val16 = ((cck << SPEC_SIFS_CCK_SHIFT) & SPEC_SIFS_CCK_MASK) |
- ((ofdm << SPEC_SIFS_OFDM_SHIFT) & SPEC_SIFS_OFDM_MASK);
-
- rtl8xxxu_write16(priv, REG_SPEC_SIFS, val16);
-}
-
-static void rtl8xxxu_print_chipinfo(struct rtl8xxxu_priv *priv)
-{
- struct device *dev = &priv->udev->dev;
- char *cut;
-
- switch (priv->chip_cut) {
- case 0:
- cut = "A";
- break;
- case 1:
- cut = "B";
- break;
- case 2:
- cut = "C";
- break;
- case 3:
- cut = "D";
- break;
- case 4:
- cut = "E";
- break;
- default:
- cut = "unknown";
- }
-
- dev_info(dev,
- "RTL%s rev %s (%s) %iT%iR, TX queues %i, WiFi=%i, BT=%i, GPS=%i, HI PA=%i\n",
- priv->chip_name, cut, priv->chip_vendor, priv->tx_paths,
- priv->rx_paths, priv->ep_tx_count, priv->has_wifi,
- priv->has_bluetooth, priv->has_gps, priv->hi_pa);
-
- dev_info(dev, "RTL%s MAC: %pM\n", priv->chip_name, priv->mac_addr);
-}
-
-static int rtl8xxxu_identify_chip(struct rtl8xxxu_priv *priv)
-{
- struct device *dev = &priv->udev->dev;
- u32 val32, bonding;
- u16 val16;
-
- val32 = rtl8xxxu_read32(priv, REG_SYS_CFG);
- priv->chip_cut = (val32 & SYS_CFG_CHIP_VERSION_MASK) >>
- SYS_CFG_CHIP_VERSION_SHIFT;
- if (val32 & SYS_CFG_TRP_VAUX_EN) {
- dev_info(dev, "Unsupported test chip\n");
- return -ENOTSUPP;
- }
-
- if (val32 & SYS_CFG_BT_FUNC) {
- if (priv->chip_cut >= 3) {
- sprintf(priv->chip_name, "8723BU");
- priv->rtl_chip = RTL8723B;
- } else {
- sprintf(priv->chip_name, "8723AU");
- priv->usb_interrupts = 1;
- priv->rtl_chip = RTL8723A;
- }
-
- priv->rf_paths = 1;
- priv->rx_paths = 1;
- priv->tx_paths = 1;
-
- val32 = rtl8xxxu_read32(priv, REG_MULTI_FUNC_CTRL);
- if (val32 & MULTI_WIFI_FUNC_EN)
- priv->has_wifi = 1;
- if (val32 & MULTI_BT_FUNC_EN)
- priv->has_bluetooth = 1;
- if (val32 & MULTI_GPS_FUNC_EN)
- priv->has_gps = 1;
- priv->is_multi_func = 1;
- } else if (val32 & SYS_CFG_TYPE_ID) {
- bonding = rtl8xxxu_read32(priv, REG_HPON_FSM);
- bonding &= HPON_FSM_BONDING_MASK;
- if (priv->fops->tx_desc_size ==
- sizeof(struct rtl8xxxu_txdesc40)) {
- if (bonding == HPON_FSM_BONDING_1T2R) {
- sprintf(priv->chip_name, "8191EU");
- priv->rf_paths = 2;
- priv->rx_paths = 2;
- priv->tx_paths = 1;
- priv->rtl_chip = RTL8191E;
- } else {
- sprintf(priv->chip_name, "8192EU");
- priv->rf_paths = 2;
- priv->rx_paths = 2;
- priv->tx_paths = 2;
- priv->rtl_chip = RTL8192E;
- }
- } else if (bonding == HPON_FSM_BONDING_1T2R) {
- sprintf(priv->chip_name, "8191CU");
- priv->rf_paths = 2;
- priv->rx_paths = 2;
- priv->tx_paths = 1;
- priv->usb_interrupts = 1;
- priv->rtl_chip = RTL8191C;
- } else {
- sprintf(priv->chip_name, "8192CU");
- priv->rf_paths = 2;
- priv->rx_paths = 2;
- priv->tx_paths = 2;
- priv->usb_interrupts = 1;
- priv->rtl_chip = RTL8192C;
- }
- priv->has_wifi = 1;
- } else {
- sprintf(priv->chip_name, "8188CU");
- priv->rf_paths = 1;
- priv->rx_paths = 1;
- priv->tx_paths = 1;
- priv->rtl_chip = RTL8188C;
- priv->usb_interrupts = 1;
- priv->has_wifi = 1;
- }
-
- switch (priv->rtl_chip) {
- case RTL8188E:
- case RTL8192E:
- case RTL8723B:
- switch (val32 & SYS_CFG_VENDOR_EXT_MASK) {
- case SYS_CFG_VENDOR_ID_TSMC:
- sprintf(priv->chip_vendor, "TSMC");
- break;
- case SYS_CFG_VENDOR_ID_SMIC:
- sprintf(priv->chip_vendor, "SMIC");
- priv->vendor_smic = 1;
- break;
- case SYS_CFG_VENDOR_ID_UMC:
- sprintf(priv->chip_vendor, "UMC");
- priv->vendor_umc = 1;
- break;
- default:
- sprintf(priv->chip_vendor, "unknown");
- }
- break;
- default:
- if (val32 & SYS_CFG_VENDOR_ID) {
- sprintf(priv->chip_vendor, "UMC");
- priv->vendor_umc = 1;
- } else {
- sprintf(priv->chip_vendor, "TSMC");
- }
- }
-
- val32 = rtl8xxxu_read32(priv, REG_GPIO_OUTSTS);
- priv->rom_rev = (val32 & GPIO_RF_RL_ID) >> 28;
-
- val16 = rtl8xxxu_read16(priv, REG_NORMAL_SIE_EP_TX);
- if (val16 & NORMAL_SIE_EP_TX_HIGH_MASK) {
- priv->ep_tx_high_queue = 1;
- priv->ep_tx_count++;
- }
-
- if (val16 & NORMAL_SIE_EP_TX_NORMAL_MASK) {
- priv->ep_tx_normal_queue = 1;
- priv->ep_tx_count++;
- }
-
- if (val16 & NORMAL_SIE_EP_TX_LOW_MASK) {
- priv->ep_tx_low_queue = 1;
- priv->ep_tx_count++;
- }
-
- /*
- * Fallback for devices that do not provide REG_NORMAL_SIE_EP_TX
- */
- if (!priv->ep_tx_count) {
- switch (priv->nr_out_eps) {
- case 4:
- case 3:
- priv->ep_tx_low_queue = 1;
- priv->ep_tx_count++;
- case 2:
- priv->ep_tx_normal_queue = 1;
- priv->ep_tx_count++;
- case 1:
- priv->ep_tx_high_queue = 1;
- priv->ep_tx_count++;
- break;
- default:
- dev_info(dev, "Unsupported USB TX end-points\n");
- return -ENOTSUPP;
- }
- }
-
- return 0;
-}
-
-static int rtl8723au_parse_efuse(struct rtl8xxxu_priv *priv)
-{
- struct rtl8723au_efuse *efuse = &priv->efuse_wifi.efuse8723;
-
- if (efuse->rtl_id != cpu_to_le16(0x8129))
- return -EINVAL;
-
- ether_addr_copy(priv->mac_addr, efuse->mac_addr);
-
- memcpy(priv->cck_tx_power_index_A,
- efuse->cck_tx_power_index_A,
- sizeof(efuse->cck_tx_power_index_A));
- memcpy(priv->cck_tx_power_index_B,
- efuse->cck_tx_power_index_B,
- sizeof(efuse->cck_tx_power_index_B));
-
- memcpy(priv->ht40_1s_tx_power_index_A,
- efuse->ht40_1s_tx_power_index_A,
- sizeof(efuse->ht40_1s_tx_power_index_A));
- memcpy(priv->ht40_1s_tx_power_index_B,
- efuse->ht40_1s_tx_power_index_B,
- sizeof(efuse->ht40_1s_tx_power_index_B));
-
- memcpy(priv->ht20_tx_power_index_diff,
- efuse->ht20_tx_power_index_diff,
- sizeof(efuse->ht20_tx_power_index_diff));
- memcpy(priv->ofdm_tx_power_index_diff,
- efuse->ofdm_tx_power_index_diff,
- sizeof(efuse->ofdm_tx_power_index_diff));
-
- memcpy(priv->ht40_max_power_offset,
- efuse->ht40_max_power_offset,
- sizeof(efuse->ht40_max_power_offset));
- memcpy(priv->ht20_max_power_offset,
- efuse->ht20_max_power_offset,
- sizeof(efuse->ht20_max_power_offset));
-
- if (priv->efuse_wifi.efuse8723.version >= 0x01) {
- priv->has_xtalk = 1;
- priv->xtalk = priv->efuse_wifi.efuse8723.xtal_k & 0x3f;
- }
-
- priv->power_base = &rtl8723a_power_base;
-
- dev_info(&priv->udev->dev, "Vendor: %.7s\n",
- efuse->vendor_name);
- dev_info(&priv->udev->dev, "Product: %.41s\n",
- efuse->device_name);
- return 0;
-}
-
-static int rtl8723bu_parse_efuse(struct rtl8xxxu_priv *priv)
-{
- struct rtl8723bu_efuse *efuse = &priv->efuse_wifi.efuse8723bu;
- int i;
-
- if (efuse->rtl_id != cpu_to_le16(0x8129))
- return -EINVAL;
-
- ether_addr_copy(priv->mac_addr, efuse->mac_addr);
-
- memcpy(priv->cck_tx_power_index_A, efuse->tx_power_index_A.cck_base,
- sizeof(efuse->tx_power_index_A.cck_base));
- memcpy(priv->cck_tx_power_index_B, efuse->tx_power_index_B.cck_base,
- sizeof(efuse->tx_power_index_B.cck_base));
-
- memcpy(priv->ht40_1s_tx_power_index_A,
- efuse->tx_power_index_A.ht40_base,
- sizeof(efuse->tx_power_index_A.ht40_base));
- memcpy(priv->ht40_1s_tx_power_index_B,
- efuse->tx_power_index_B.ht40_base,
- sizeof(efuse->tx_power_index_B.ht40_base));
-
- priv->ofdm_tx_power_diff[0].a =
- efuse->tx_power_index_A.ht20_ofdm_1s_diff.a;
- priv->ofdm_tx_power_diff[0].b =
- efuse->tx_power_index_B.ht20_ofdm_1s_diff.a;
-
- priv->ht20_tx_power_diff[0].a =
- efuse->tx_power_index_A.ht20_ofdm_1s_diff.b;
- priv->ht20_tx_power_diff[0].b =
- efuse->tx_power_index_B.ht20_ofdm_1s_diff.b;
-
- priv->ht40_tx_power_diff[0].a = 0;
- priv->ht40_tx_power_diff[0].b = 0;
-
- for (i = 1; i < RTL8723B_TX_COUNT; i++) {
- priv->ofdm_tx_power_diff[i].a =
- efuse->tx_power_index_A.pwr_diff[i - 1].ofdm;
- priv->ofdm_tx_power_diff[i].b =
- efuse->tx_power_index_B.pwr_diff[i - 1].ofdm;
-
- priv->ht20_tx_power_diff[i].a =
- efuse->tx_power_index_A.pwr_diff[i - 1].ht20;
- priv->ht20_tx_power_diff[i].b =
- efuse->tx_power_index_B.pwr_diff[i - 1].ht20;
-
- priv->ht40_tx_power_diff[i].a =
- efuse->tx_power_index_A.pwr_diff[i - 1].ht40;
- priv->ht40_tx_power_diff[i].b =
- efuse->tx_power_index_B.pwr_diff[i - 1].ht40;
- }
-
- priv->has_xtalk = 1;
- priv->xtalk = priv->efuse_wifi.efuse8723bu.xtal_k & 0x3f;
-
- dev_info(&priv->udev->dev, "Vendor: %.7s\n", efuse->vendor_name);
- dev_info(&priv->udev->dev, "Product: %.41s\n", efuse->device_name);
-
- if (rtl8xxxu_debug & RTL8XXXU_DEBUG_EFUSE) {
- int i;
- unsigned char *raw = priv->efuse_wifi.raw;
-
- dev_info(&priv->udev->dev,
- "%s: dumping efuse (0x%02zx bytes):\n",
- __func__, sizeof(struct rtl8723bu_efuse));
- for (i = 0; i < sizeof(struct rtl8723bu_efuse); i += 8) {
- dev_info(&priv->udev->dev, "%02x: "
- "%02x %02x %02x %02x %02x %02x %02x %02x\n", i,
- raw[i], raw[i + 1], raw[i + 2],
- raw[i + 3], raw[i + 4], raw[i + 5],
- raw[i + 6], raw[i + 7]);
- }
- }
-
- return 0;
-}
-
-#ifdef CONFIG_RTL8XXXU_UNTESTED
-
-static int rtl8192cu_parse_efuse(struct rtl8xxxu_priv *priv)
-{
- struct rtl8192cu_efuse *efuse = &priv->efuse_wifi.efuse8192;
- int i;
-
- if (efuse->rtl_id != cpu_to_le16(0x8129))
- return -EINVAL;
-
- ether_addr_copy(priv->mac_addr, efuse->mac_addr);
-
- memcpy(priv->cck_tx_power_index_A,
- efuse->cck_tx_power_index_A,
- sizeof(efuse->cck_tx_power_index_A));
- memcpy(priv->cck_tx_power_index_B,
- efuse->cck_tx_power_index_B,
- sizeof(efuse->cck_tx_power_index_B));
-
- memcpy(priv->ht40_1s_tx_power_index_A,
- efuse->ht40_1s_tx_power_index_A,
- sizeof(efuse->ht40_1s_tx_power_index_A));
- memcpy(priv->ht40_1s_tx_power_index_B,
- efuse->ht40_1s_tx_power_index_B,
- sizeof(efuse->ht40_1s_tx_power_index_B));
- memcpy(priv->ht40_2s_tx_power_index_diff,
- efuse->ht40_2s_tx_power_index_diff,
- sizeof(efuse->ht40_2s_tx_power_index_diff));
-
- memcpy(priv->ht20_tx_power_index_diff,
- efuse->ht20_tx_power_index_diff,
- sizeof(efuse->ht20_tx_power_index_diff));
- memcpy(priv->ofdm_tx_power_index_diff,
- efuse->ofdm_tx_power_index_diff,
- sizeof(efuse->ofdm_tx_power_index_diff));
-
- memcpy(priv->ht40_max_power_offset,
- efuse->ht40_max_power_offset,
- sizeof(efuse->ht40_max_power_offset));
- memcpy(priv->ht20_max_power_offset,
- efuse->ht20_max_power_offset,
- sizeof(efuse->ht20_max_power_offset));
-
- dev_info(&priv->udev->dev, "Vendor: %.7s\n",
- efuse->vendor_name);
- dev_info(&priv->udev->dev, "Product: %.20s\n",
- efuse->device_name);
-
- priv->power_base = &rtl8192c_power_base;
-
- if (efuse->rf_regulatory & 0x20) {
- sprintf(priv->chip_name, "8188RU");
- priv->rtl_chip = RTL8188R;
- priv->hi_pa = 1;
- priv->no_pape = 1;
- priv->power_base = &rtl8188r_power_base;
- }
-
- if (rtl8xxxu_debug & RTL8XXXU_DEBUG_EFUSE) {
- unsigned char *raw = priv->efuse_wifi.raw;
-
- dev_info(&priv->udev->dev,
- "%s: dumping efuse (0x%02zx bytes):\n",
- __func__, sizeof(struct rtl8192cu_efuse));
- for (i = 0; i < sizeof(struct rtl8192cu_efuse); i += 8) {
- dev_info(&priv->udev->dev, "%02x: "
- "%02x %02x %02x %02x %02x %02x %02x %02x\n", i,
- raw[i], raw[i + 1], raw[i + 2],
- raw[i + 3], raw[i + 4], raw[i + 5],
- raw[i + 6], raw[i + 7]);
- }
- }
- return 0;
-}
-
-#endif
-
-static int rtl8192eu_parse_efuse(struct rtl8xxxu_priv *priv)
-{
- struct rtl8192eu_efuse *efuse = &priv->efuse_wifi.efuse8192eu;
- int i;
-
- if (efuse->rtl_id != cpu_to_le16(0x8129))
- return -EINVAL;
-
- ether_addr_copy(priv->mac_addr, efuse->mac_addr);
-
- memcpy(priv->cck_tx_power_index_A, efuse->tx_power_index_A.cck_base,
- sizeof(efuse->tx_power_index_A.cck_base));
- memcpy(priv->cck_tx_power_index_B, efuse->tx_power_index_B.cck_base,
- sizeof(efuse->tx_power_index_B.cck_base));
-
- memcpy(priv->ht40_1s_tx_power_index_A,
- efuse->tx_power_index_A.ht40_base,
- sizeof(efuse->tx_power_index_A.ht40_base));
- memcpy(priv->ht40_1s_tx_power_index_B,
- efuse->tx_power_index_B.ht40_base,
- sizeof(efuse->tx_power_index_B.ht40_base));
-
- priv->ht20_tx_power_diff[0].a =
- efuse->tx_power_index_A.ht20_ofdm_1s_diff.b;
- priv->ht20_tx_power_diff[0].b =
- efuse->tx_power_index_B.ht20_ofdm_1s_diff.b;
-
- priv->ht40_tx_power_diff[0].a = 0;
- priv->ht40_tx_power_diff[0].b = 0;
-
- for (i = 1; i < RTL8723B_TX_COUNT; i++) {
- priv->ofdm_tx_power_diff[i].a =
- efuse->tx_power_index_A.pwr_diff[i - 1].ofdm;
- priv->ofdm_tx_power_diff[i].b =
- efuse->tx_power_index_B.pwr_diff[i - 1].ofdm;
-
- priv->ht20_tx_power_diff[i].a =
- efuse->tx_power_index_A.pwr_diff[i - 1].ht20;
- priv->ht20_tx_power_diff[i].b =
- efuse->tx_power_index_B.pwr_diff[i - 1].ht20;
-
- priv->ht40_tx_power_diff[i].a =
- efuse->tx_power_index_A.pwr_diff[i - 1].ht40;
- priv->ht40_tx_power_diff[i].b =
- efuse->tx_power_index_B.pwr_diff[i - 1].ht40;
- }
-
- priv->has_xtalk = 1;
- priv->xtalk = priv->efuse_wifi.efuse8192eu.xtal_k & 0x3f;
-
- dev_info(&priv->udev->dev, "Vendor: %.7s\n", efuse->vendor_name);
- dev_info(&priv->udev->dev, "Product: %.11s\n", efuse->device_name);
- dev_info(&priv->udev->dev, "Serial: %.11s\n", efuse->serial);
-
- if (rtl8xxxu_debug & RTL8XXXU_DEBUG_EFUSE) {
- unsigned char *raw = priv->efuse_wifi.raw;
-
- dev_info(&priv->udev->dev,
- "%s: dumping efuse (0x%02zx bytes):\n",
- __func__, sizeof(struct rtl8192eu_efuse));
- for (i = 0; i < sizeof(struct rtl8192eu_efuse); i += 8) {
- dev_info(&priv->udev->dev, "%02x: "
- "%02x %02x %02x %02x %02x %02x %02x %02x\n", i,
- raw[i], raw[i + 1], raw[i + 2],
- raw[i + 3], raw[i + 4], raw[i + 5],
- raw[i + 6], raw[i + 7]);
- }
- }
- return 0;
-}
-
-static int
-rtl8xxxu_read_efuse8(struct rtl8xxxu_priv *priv, u16 offset, u8 *data)
-{
- int i;
- u8 val8;
- u32 val32;
-
- /* Write Address */
- rtl8xxxu_write8(priv, REG_EFUSE_CTRL + 1, offset & 0xff);
- val8 = rtl8xxxu_read8(priv, REG_EFUSE_CTRL + 2);
- val8 &= 0xfc;
- val8 |= (offset >> 8) & 0x03;
- rtl8xxxu_write8(priv, REG_EFUSE_CTRL + 2, val8);
-
- val8 = rtl8xxxu_read8(priv, REG_EFUSE_CTRL + 3);
- rtl8xxxu_write8(priv, REG_EFUSE_CTRL + 3, val8 & 0x7f);
-
- /* Poll for data read */
- val32 = rtl8xxxu_read32(priv, REG_EFUSE_CTRL);
- for (i = 0; i < RTL8XXXU_MAX_REG_POLL; i++) {
- val32 = rtl8xxxu_read32(priv, REG_EFUSE_CTRL);
- if (val32 & BIT(31))
- break;
- }
-
- if (i == RTL8XXXU_MAX_REG_POLL)
- return -EIO;
-
- udelay(50);
- val32 = rtl8xxxu_read32(priv, REG_EFUSE_CTRL);
-
- *data = val32 & 0xff;
- return 0;
-}
-
-static int rtl8xxxu_read_efuse(struct rtl8xxxu_priv *priv)
-{
- struct device *dev = &priv->udev->dev;
- int i, ret = 0;
- u8 val8, word_mask, header, extheader;
- u16 val16, efuse_addr, offset;
- u32 val32;
-
- val16 = rtl8xxxu_read16(priv, REG_9346CR);
- if (val16 & EEPROM_ENABLE)
- priv->has_eeprom = 1;
- if (val16 & EEPROM_BOOT)
- priv->boot_eeprom = 1;
-
- if (priv->is_multi_func) {
- val32 = rtl8xxxu_read32(priv, REG_EFUSE_TEST);
- val32 = (val32 & ~EFUSE_SELECT_MASK) | EFUSE_WIFI_SELECT;
- rtl8xxxu_write32(priv, REG_EFUSE_TEST, val32);
- }
-
- dev_dbg(dev, "Booting from %s\n",
- priv->boot_eeprom ? "EEPROM" : "EFUSE");
-
- rtl8xxxu_write8(priv, REG_EFUSE_ACCESS, EFUSE_ACCESS_ENABLE);
-
- /* 1.2V Power: From VDDON with Power Cut(0x0000[15]), default valid */
- val16 = rtl8xxxu_read16(priv, REG_SYS_ISO_CTRL);
- if (!(val16 & SYS_ISO_PWC_EV12V)) {
- val16 |= SYS_ISO_PWC_EV12V;
- rtl8xxxu_write16(priv, REG_SYS_ISO_CTRL, val16);
- }
- /* Reset: 0x0000[28], default valid */
- val16 = rtl8xxxu_read16(priv, REG_SYS_FUNC);
- if (!(val16 & SYS_FUNC_ELDR)) {
- val16 |= SYS_FUNC_ELDR;
- rtl8xxxu_write16(priv, REG_SYS_FUNC, val16);
- }
-
- /*
- * Clock: Gated(0x0008[5]) 8M(0x0008[1]) clock from ANA, default valid
- */
- val16 = rtl8xxxu_read16(priv, REG_SYS_CLKR);
- if (!(val16 & SYS_CLK_LOADER_ENABLE) || !(val16 & SYS_CLK_ANA8M)) {
- val16 |= (SYS_CLK_LOADER_ENABLE | SYS_CLK_ANA8M);
- rtl8xxxu_write16(priv, REG_SYS_CLKR, val16);
- }
-
- /* Default value is 0xff */
- memset(priv->efuse_wifi.raw, 0xff, EFUSE_MAP_LEN);
-
- efuse_addr = 0;
- while (efuse_addr < EFUSE_REAL_CONTENT_LEN_8723A) {
- u16 map_addr;
-
- ret = rtl8xxxu_read_efuse8(priv, efuse_addr++, &header);
- if (ret || header == 0xff)
- goto exit;
-
- if ((header & 0x1f) == 0x0f) { /* extended header */
- offset = (header & 0xe0) >> 5;
-
- ret = rtl8xxxu_read_efuse8(priv, efuse_addr++,
- &extheader);
- if (ret)
- goto exit;
- /* All words disabled */
- if ((extheader & 0x0f) == 0x0f)
- continue;
-
- offset |= ((extheader & 0xf0) >> 1);
- word_mask = extheader & 0x0f;
- } else {
- offset = (header >> 4) & 0x0f;
- word_mask = header & 0x0f;
- }
-
- /* Get word enable value from PG header */
-
- /* We have 8 bits to indicate validity */
- map_addr = offset * 8;
- if (map_addr >= EFUSE_MAP_LEN) {
- dev_warn(dev, "%s: Illegal map_addr (%04x), "
- "efuse corrupt!\n",
- __func__, map_addr);
- ret = -EINVAL;
- goto exit;
- }
- for (i = 0; i < EFUSE_MAX_WORD_UNIT; i++) {
- /* Check word enable condition in the section */
- if (word_mask & BIT(i)) {
- map_addr += 2;
- continue;
- }
-
- ret = rtl8xxxu_read_efuse8(priv, efuse_addr++, &val8);
- if (ret)
- goto exit;
- priv->efuse_wifi.raw[map_addr++] = val8;
-
- ret = rtl8xxxu_read_efuse8(priv, efuse_addr++, &val8);
- if (ret)
- goto exit;
- priv->efuse_wifi.raw[map_addr++] = val8;
- }
- }
-
-exit:
- rtl8xxxu_write8(priv, REG_EFUSE_ACCESS, EFUSE_ACCESS_DISABLE);
-
- return ret;
-}
-
-static void rtl8xxxu_reset_8051(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
- u16 sys_func;
-
- val8 = rtl8xxxu_read8(priv, REG_RSV_CTRL + 1);
- val8 &= ~BIT(0);
- rtl8xxxu_write8(priv, REG_RSV_CTRL + 1, val8);
-
- sys_func = rtl8xxxu_read16(priv, REG_SYS_FUNC);
- sys_func &= ~SYS_FUNC_CPU_ENABLE;
- rtl8xxxu_write16(priv, REG_SYS_FUNC, sys_func);
-
- val8 = rtl8xxxu_read8(priv, REG_RSV_CTRL + 1);
- val8 |= BIT(0);
- rtl8xxxu_write8(priv, REG_RSV_CTRL + 1, val8);
-
- sys_func |= SYS_FUNC_CPU_ENABLE;
- rtl8xxxu_write16(priv, REG_SYS_FUNC, sys_func);
-}
-
-static void rtl8723bu_reset_8051(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
- u16 sys_func;
-
- val8 = rtl8xxxu_read8(priv, REG_RSV_CTRL);
- val8 &= ~BIT(1);
- rtl8xxxu_write8(priv, REG_RSV_CTRL, val8);
-
- val8 = rtl8xxxu_read8(priv, REG_RSV_CTRL + 1);
- val8 &= ~BIT(0);
- rtl8xxxu_write8(priv, REG_RSV_CTRL + 1, val8);
-
- sys_func = rtl8xxxu_read16(priv, REG_SYS_FUNC);
- sys_func &= ~SYS_FUNC_CPU_ENABLE;
- rtl8xxxu_write16(priv, REG_SYS_FUNC, sys_func);
-
- val8 = rtl8xxxu_read8(priv, REG_RSV_CTRL);
- val8 &= ~BIT(1);
- rtl8xxxu_write8(priv, REG_RSV_CTRL, val8);
-
- val8 = rtl8xxxu_read8(priv, REG_RSV_CTRL + 1);
- val8 |= BIT(0);
- rtl8xxxu_write8(priv, REG_RSV_CTRL + 1, val8);
-
- sys_func |= SYS_FUNC_CPU_ENABLE;
- rtl8xxxu_write16(priv, REG_SYS_FUNC, sys_func);
-}
-
-static int rtl8xxxu_start_firmware(struct rtl8xxxu_priv *priv)
-{
- struct device *dev = &priv->udev->dev;
- int ret = 0, i;
- u32 val32;
-
- /* Poll checksum report */
- for (i = 0; i < RTL8XXXU_FIRMWARE_POLL_MAX; i++) {
- val32 = rtl8xxxu_read32(priv, REG_MCU_FW_DL);
- if (val32 & MCU_FW_DL_CSUM_REPORT)
- break;
- }
-
- if (i == RTL8XXXU_FIRMWARE_POLL_MAX) {
- dev_warn(dev, "Firmware checksum poll timed out\n");
- ret = -EAGAIN;
- goto exit;
- }
-
- val32 = rtl8xxxu_read32(priv, REG_MCU_FW_DL);
- val32 |= MCU_FW_DL_READY;
- val32 &= ~MCU_WINT_INIT_READY;
- rtl8xxxu_write32(priv, REG_MCU_FW_DL, val32);
-
- /*
- * Reset the 8051 in order for the firmware to start running,
- * otherwise it won't come up on the 8192eu
- */
- priv->fops->reset_8051(priv);
-
- /* Wait for firmware to become ready */
- for (i = 0; i < RTL8XXXU_FIRMWARE_POLL_MAX; i++) {
- val32 = rtl8xxxu_read32(priv, REG_MCU_FW_DL);
- if (val32 & MCU_WINT_INIT_READY)
- break;
-
- udelay(100);
- }
-
- if (i == RTL8XXXU_FIRMWARE_POLL_MAX) {
- dev_warn(dev, "Firmware failed to start\n");
- ret = -EAGAIN;
- goto exit;
- }
-
- /*
- * Init H2C command
- */
- if (priv->rtl_chip == RTL8723B)
- rtl8xxxu_write8(priv, REG_HMTFR, 0x0f);
-exit:
- return ret;
-}
-
-static int rtl8xxxu_download_firmware(struct rtl8xxxu_priv *priv)
-{
- int pages, remainder, i, ret;
- u8 val8;
- u16 val16;
- u32 val32;
- u8 *fwptr;
-
- val8 = rtl8xxxu_read8(priv, REG_SYS_FUNC + 1);
- val8 |= 4;
- rtl8xxxu_write8(priv, REG_SYS_FUNC + 1, val8);
-
- /* 8051 enable */
- val16 = rtl8xxxu_read16(priv, REG_SYS_FUNC);
- val16 |= SYS_FUNC_CPU_ENABLE;
- rtl8xxxu_write16(priv, REG_SYS_FUNC, val16);
-
- val8 = rtl8xxxu_read8(priv, REG_MCU_FW_DL);
- if (val8 & MCU_FW_RAM_SEL) {
- pr_info("do the RAM reset\n");
- rtl8xxxu_write8(priv, REG_MCU_FW_DL, 0x00);
- priv->fops->reset_8051(priv);
- }
-
- /* MCU firmware download enable */
- val8 = rtl8xxxu_read8(priv, REG_MCU_FW_DL);
- val8 |= MCU_FW_DL_ENABLE;
- rtl8xxxu_write8(priv, REG_MCU_FW_DL, val8);
-
- /* 8051 reset */
- val32 = rtl8xxxu_read32(priv, REG_MCU_FW_DL);
- val32 &= ~BIT(19);
- rtl8xxxu_write32(priv, REG_MCU_FW_DL, val32);
-
- /* Reset firmware download checksum */
- val8 = rtl8xxxu_read8(priv, REG_MCU_FW_DL);
- val8 |= MCU_FW_DL_CSUM_REPORT;
- rtl8xxxu_write8(priv, REG_MCU_FW_DL, val8);
-
- pages = priv->fw_size / RTL_FW_PAGE_SIZE;
- remainder = priv->fw_size % RTL_FW_PAGE_SIZE;
-
- fwptr = priv->fw_data->data;
-
- for (i = 0; i < pages; i++) {
- val8 = rtl8xxxu_read8(priv, REG_MCU_FW_DL + 2) & 0xF8;
- val8 |= i;
- rtl8xxxu_write8(priv, REG_MCU_FW_DL + 2, val8);
-
- ret = rtl8xxxu_writeN(priv, REG_FW_START_ADDRESS,
- fwptr, RTL_FW_PAGE_SIZE);
- if (ret != RTL_FW_PAGE_SIZE) {
- ret = -EAGAIN;
- goto fw_abort;
- }
-
- fwptr += RTL_FW_PAGE_SIZE;
- }
-
- if (remainder) {
- val8 = rtl8xxxu_read8(priv, REG_MCU_FW_DL + 2) & 0xF8;
- val8 |= i;
- rtl8xxxu_write8(priv, REG_MCU_FW_DL + 2, val8);
- ret = rtl8xxxu_writeN(priv, REG_FW_START_ADDRESS,
- fwptr, remainder);
- if (ret != remainder) {
- ret = -EAGAIN;
- goto fw_abort;
- }
- }
-
- ret = 0;
-fw_abort:
- /* MCU firmware download disable */
- val16 = rtl8xxxu_read16(priv, REG_MCU_FW_DL);
- val16 &= ~MCU_FW_DL_ENABLE;
- rtl8xxxu_write16(priv, REG_MCU_FW_DL, val16);
-
- return ret;
-}
-
-static int rtl8xxxu_load_firmware(struct rtl8xxxu_priv *priv, char *fw_name)
-{
- struct device *dev = &priv->udev->dev;
- const struct firmware *fw;
- int ret = 0;
- u16 signature;
-
- dev_info(dev, "%s: Loading firmware %s\n", DRIVER_NAME, fw_name);
- if (request_firmware(&fw, fw_name, &priv->udev->dev)) {
- dev_warn(dev, "request_firmware(%s) failed\n", fw_name);
- ret = -EAGAIN;
- goto exit;
- }
- if (!fw) {
- dev_warn(dev, "Firmware data not available\n");
- ret = -EINVAL;
- goto exit;
- }
-
- priv->fw_data = kmemdup(fw->data, fw->size, GFP_KERNEL);
- if (!priv->fw_data) {
- ret = -ENOMEM;
- goto exit;
- }
- priv->fw_size = fw->size - sizeof(struct rtl8xxxu_firmware_header);
-
- signature = le16_to_cpu(priv->fw_data->signature);
- switch (signature & 0xfff0) {
- case 0x92e0:
- case 0x92c0:
- case 0x88c0:
- case 0x5300:
- case 0x2300:
- break;
- default:
- ret = -EINVAL;
- dev_warn(dev, "%s: Invalid firmware signature: 0x%04x\n",
- __func__, signature);
- }
-
- dev_info(dev, "Firmware revision %i.%i (signature 0x%04x)\n",
- le16_to_cpu(priv->fw_data->major_version),
- priv->fw_data->minor_version, signature);
-
-exit:
- release_firmware(fw);
- return ret;
-}
-
-static int rtl8723au_load_firmware(struct rtl8xxxu_priv *priv)
-{
- char *fw_name;
- int ret;
-
- switch (priv->chip_cut) {
- case 0:
- fw_name = "rtlwifi/rtl8723aufw_A.bin";
- break;
- case 1:
- if (priv->enable_bluetooth)
- fw_name = "rtlwifi/rtl8723aufw_B.bin";
- else
- fw_name = "rtlwifi/rtl8723aufw_B_NoBT.bin";
-
- break;
- default:
- return -EINVAL;
- }
-
- ret = rtl8xxxu_load_firmware(priv, fw_name);
- return ret;
-}
-
-static int rtl8723bu_load_firmware(struct rtl8xxxu_priv *priv)
-{
- char *fw_name;
- int ret;
-
- if (priv->enable_bluetooth)
- fw_name = "rtlwifi/rtl8723bu_bt.bin";
- else
- fw_name = "rtlwifi/rtl8723bu_nic.bin";
-
- ret = rtl8xxxu_load_firmware(priv, fw_name);
- return ret;
-}
-
-#ifdef CONFIG_RTL8XXXU_UNTESTED
-
-static int rtl8192cu_load_firmware(struct rtl8xxxu_priv *priv)
-{
- char *fw_name;
- int ret;
-
- if (!priv->vendor_umc)
- fw_name = "rtlwifi/rtl8192cufw_TMSC.bin";
- else if (priv->chip_cut || priv->rtl_chip == RTL8192C)
- fw_name = "rtlwifi/rtl8192cufw_B.bin";
- else
- fw_name = "rtlwifi/rtl8192cufw_A.bin";
-
- ret = rtl8xxxu_load_firmware(priv, fw_name);
-
- return ret;
-}
-
-#endif
-
-static int rtl8192eu_load_firmware(struct rtl8xxxu_priv *priv)
-{
- char *fw_name;
- int ret;
-
- fw_name = "rtlwifi/rtl8192eu_nic.bin";
-
- ret = rtl8xxxu_load_firmware(priv, fw_name);
-
- return ret;
-}
-
-static void rtl8xxxu_firmware_self_reset(struct rtl8xxxu_priv *priv)
-{
- u16 val16;
- int i = 100;
-
- /* Inform 8051 to perform reset */
- rtl8xxxu_write8(priv, REG_HMTFR + 3, 0x20);
-
- for (i = 100; i > 0; i--) {
- val16 = rtl8xxxu_read16(priv, REG_SYS_FUNC);
-
- if (!(val16 & SYS_FUNC_CPU_ENABLE)) {
- dev_dbg(&priv->udev->dev,
- "%s: Firmware self reset success!\n", __func__);
- break;
- }
- udelay(50);
- }
-
- if (!i) {
- /* Force firmware reset */
- val16 = rtl8xxxu_read16(priv, REG_SYS_FUNC);
- val16 &= ~SYS_FUNC_CPU_ENABLE;
- rtl8xxxu_write16(priv, REG_SYS_FUNC, val16);
- }
-}
-
-static void rtl8723bu_phy_init_antenna_selection(struct rtl8xxxu_priv *priv)
-{
- u32 val32;
-
- val32 = rtl8xxxu_read32(priv, REG_PAD_CTRL1);
- val32 &= ~(BIT(20) | BIT(24));
- rtl8xxxu_write32(priv, REG_PAD_CTRL1, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_GPIO_MUXCFG);
- val32 &= ~BIT(4);
- rtl8xxxu_write32(priv, REG_GPIO_MUXCFG, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_GPIO_MUXCFG);
- val32 |= BIT(3);
- rtl8xxxu_write32(priv, REG_GPIO_MUXCFG, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_LEDCFG0);
- val32 |= BIT(24);
- rtl8xxxu_write32(priv, REG_LEDCFG0, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_LEDCFG0);
- val32 &= ~BIT(23);
- rtl8xxxu_write32(priv, REG_LEDCFG0, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_RFE_BUFFER);
- val32 |= (BIT(0) | BIT(1));
- rtl8xxxu_write32(priv, REG_RFE_BUFFER, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_RFE_CTRL_ANTA_SRC);
- val32 &= 0xffffff00;
- val32 |= 0x77;
- rtl8xxxu_write32(priv, REG_RFE_CTRL_ANTA_SRC, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_PWR_DATA);
- val32 |= PWR_DATA_EEPRPAD_RFE_CTRL_EN;
- rtl8xxxu_write32(priv, REG_PWR_DATA, val32);
-}
-
-static int
-rtl8xxxu_init_mac(struct rtl8xxxu_priv *priv)
-{
- struct rtl8xxxu_reg8val *array = priv->fops->mactable;
- int i, ret;
- u16 reg;
- u8 val;
-
- for (i = 0; ; i++) {
- reg = array[i].reg;
- val = array[i].val;
-
- if (reg == 0xffff && val == 0xff)
- break;
-
- ret = rtl8xxxu_write8(priv, reg, val);
- if (ret != 1) {
- dev_warn(&priv->udev->dev,
- "Failed to initialize MAC "
- "(reg: %04x, val %02x)\n", reg, val);
- return -EAGAIN;
- }
- }
-
- if (priv->rtl_chip != RTL8723B && priv->rtl_chip != RTL8192E)
- rtl8xxxu_write8(priv, REG_MAX_AGGR_NUM, 0x0a);
-
- return 0;
-}
-
-static int rtl8xxxu_init_phy_regs(struct rtl8xxxu_priv *priv,
- struct rtl8xxxu_reg32val *array)
-{
- int i, ret;
- u16 reg;
- u32 val;
-
- for (i = 0; ; i++) {
- reg = array[i].reg;
- val = array[i].val;
-
- if (reg == 0xffff && val == 0xffffffff)
- break;
-
- ret = rtl8xxxu_write32(priv, reg, val);
- if (ret != sizeof(val)) {
- dev_warn(&priv->udev->dev,
- "Failed to initialize PHY\n");
- return -EAGAIN;
- }
- udelay(1);
- }
-
- return 0;
-}
-
-static void rtl8xxxu_gen1_init_phy_bb(struct rtl8xxxu_priv *priv)
-{
- u8 val8, ldoa15, ldov12d, lpldo, ldohci12;
- u16 val16;
- u32 val32;
-
- val8 = rtl8xxxu_read8(priv, REG_AFE_PLL_CTRL);
- udelay(2);
- val8 |= AFE_PLL_320_ENABLE;
- rtl8xxxu_write8(priv, REG_AFE_PLL_CTRL, val8);
- udelay(2);
-
- rtl8xxxu_write8(priv, REG_AFE_PLL_CTRL + 1, 0xff);
- udelay(2);
-
- val16 = rtl8xxxu_read16(priv, REG_SYS_FUNC);
- val16 |= SYS_FUNC_BB_GLB_RSTN | SYS_FUNC_BBRSTB;
- rtl8xxxu_write16(priv, REG_SYS_FUNC, val16);
-
- val32 = rtl8xxxu_read32(priv, REG_AFE_XTAL_CTRL);
- val32 &= ~AFE_XTAL_RF_GATE;
- if (priv->has_bluetooth)
- val32 &= ~AFE_XTAL_BT_GATE;
- rtl8xxxu_write32(priv, REG_AFE_XTAL_CTRL, val32);
-
- /* 6. 0x1f[7:0] = 0x07 */
- val8 = RF_ENABLE | RF_RSTB | RF_SDMRSTB;
- rtl8xxxu_write8(priv, REG_RF_CTRL, val8);
-
- if (priv->hi_pa)
- rtl8xxxu_init_phy_regs(priv, rtl8188ru_phy_1t_highpa_table);
- else if (priv->tx_paths == 2)
- rtl8xxxu_init_phy_regs(priv, rtl8192cu_phy_2t_init_table);
- else
- rtl8xxxu_init_phy_regs(priv, rtl8723a_phy_1t_init_table);
-
- if (priv->rtl_chip == RTL8188R && priv->hi_pa &&
- priv->vendor_umc && priv->chip_cut == 1)
- rtl8xxxu_write8(priv, REG_OFDM0_AGC_PARM1 + 2, 0x50);
-
- if (priv->hi_pa)
- rtl8xxxu_init_phy_regs(priv, rtl8xxx_agc_highpa_table);
- else
- rtl8xxxu_init_phy_regs(priv, rtl8xxx_agc_standard_table);
-
- ldoa15 = LDOA15_ENABLE | LDOA15_OBUF;
- ldov12d = LDOV12D_ENABLE | BIT(2) | (2 << LDOV12D_VADJ_SHIFT);
- ldohci12 = 0x57;
- lpldo = 1;
- val32 = (lpldo << 24) | (ldohci12 << 16) | (ldov12d << 8) | ldoa15;
- rtl8xxxu_write32(priv, REG_LDOA15_CTRL, val32);
-}
-
-static void rtl8723bu_init_phy_bb(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
- u16 val16;
-
- val16 = rtl8xxxu_read16(priv, REG_SYS_FUNC);
- val16 |= SYS_FUNC_BB_GLB_RSTN | SYS_FUNC_BBRSTB | SYS_FUNC_DIO_RF;
- rtl8xxxu_write16(priv, REG_SYS_FUNC, val16);
-
- rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, 0x00);
-
- /* 6. 0x1f[7:0] = 0x07 */
- val8 = RF_ENABLE | RF_RSTB | RF_SDMRSTB;
- rtl8xxxu_write8(priv, REG_RF_CTRL, val8);
-
- /* Why? */
- rtl8xxxu_write8(priv, REG_SYS_FUNC, 0xe3);
- rtl8xxxu_write8(priv, REG_AFE_XTAL_CTRL + 1, 0x80);
- rtl8xxxu_init_phy_regs(priv, rtl8723b_phy_1t_init_table);
-
- rtl8xxxu_init_phy_regs(priv, rtl8xxx_agc_8723bu_table);
-}
-
-static void rtl8192eu_init_phy_bb(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
- u16 val16;
-
- val16 = rtl8xxxu_read16(priv, REG_SYS_FUNC);
- val16 |= SYS_FUNC_BB_GLB_RSTN | SYS_FUNC_BBRSTB | SYS_FUNC_DIO_RF;
- rtl8xxxu_write16(priv, REG_SYS_FUNC, val16);
-
- /* 6. 0x1f[7:0] = 0x07 */
- val8 = RF_ENABLE | RF_RSTB | RF_SDMRSTB;
- rtl8xxxu_write8(priv, REG_RF_CTRL, val8);
-
- val16 = rtl8xxxu_read16(priv, REG_SYS_FUNC);
- val16 |= (SYS_FUNC_USBA | SYS_FUNC_USBD | SYS_FUNC_DIO_RF |
- SYS_FUNC_BB_GLB_RSTN | SYS_FUNC_BBRSTB);
- rtl8xxxu_write16(priv, REG_SYS_FUNC, val16);
- val8 = RF_ENABLE | RF_RSTB | RF_SDMRSTB;
- rtl8xxxu_write8(priv, REG_RF_CTRL, val8);
- rtl8xxxu_init_phy_regs(priv, rtl8192eu_phy_init_table);
-
- if (priv->hi_pa)
- rtl8xxxu_init_phy_regs(priv, rtl8xxx_agc_8192eu_highpa_table);
- else
- rtl8xxxu_init_phy_regs(priv, rtl8xxx_agc_8192eu_std_table);
-}
-
-/*
- * Most of this is black magic retrieved from the old rtl8723au driver
- */
-static int rtl8xxxu_init_phy_bb(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
- u32 val32;
-
- priv->fops->init_phy_bb(priv);
-
- if (priv->tx_paths == 1 && priv->rx_paths == 2) {
- /*
- * For 1T2R boards, patch the registers.
- *
- * It looks like 8191/2 1T2R boards use path B for TX
- */
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_TX_INFO);
- val32 &= ~(BIT(0) | BIT(1));
- val32 |= BIT(1);
- rtl8xxxu_write32(priv, REG_FPGA0_TX_INFO, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_FPGA1_TX_INFO);
- val32 &= ~0x300033;
- val32 |= 0x200022;
- rtl8xxxu_write32(priv, REG_FPGA1_TX_INFO, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_CCK0_AFE_SETTING);
- val32 &= ~CCK0_AFE_RX_MASK;
- val32 &= 0x00ffffff;
- val32 |= 0x40000000;
- val32 |= CCK0_AFE_RX_ANT_B;
- rtl8xxxu_write32(priv, REG_CCK0_AFE_SETTING, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_TRX_PATH_ENABLE);
- val32 &= ~(OFDM_RF_PATH_RX_MASK | OFDM_RF_PATH_TX_MASK);
- val32 |= (OFDM_RF_PATH_RX_A | OFDM_RF_PATH_RX_B |
- OFDM_RF_PATH_TX_B);
- rtl8xxxu_write32(priv, REG_OFDM0_TRX_PATH_ENABLE, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_AGC_PARM1);
- val32 &= ~(BIT(4) | BIT(5));
- val32 |= BIT(4);
- rtl8xxxu_write32(priv, REG_OFDM0_AGC_PARM1, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_TX_CCK_RFON);
- val32 &= ~(BIT(27) | BIT(26));
- val32 |= BIT(27);
- rtl8xxxu_write32(priv, REG_TX_CCK_RFON, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_TX_CCK_BBON);
- val32 &= ~(BIT(27) | BIT(26));
- val32 |= BIT(27);
- rtl8xxxu_write32(priv, REG_TX_CCK_BBON, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_TX_OFDM_RFON);
- val32 &= ~(BIT(27) | BIT(26));
- val32 |= BIT(27);
- rtl8xxxu_write32(priv, REG_TX_OFDM_RFON, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_TX_OFDM_BBON);
- val32 &= ~(BIT(27) | BIT(26));
- val32 |= BIT(27);
- rtl8xxxu_write32(priv, REG_TX_OFDM_BBON, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_TX_TO_TX);
- val32 &= ~(BIT(27) | BIT(26));
- val32 |= BIT(27);
- rtl8xxxu_write32(priv, REG_TX_TO_TX, val32);
- }
-
- if (priv->has_xtalk) {
- val32 = rtl8xxxu_read32(priv, REG_MAC_PHY_CTRL);
-
- val8 = priv->xtalk;
- val32 &= 0xff000fff;
- val32 |= ((val8 | (val8 << 6)) << 12);
-
- rtl8xxxu_write32(priv, REG_MAC_PHY_CTRL, val32);
- }
-
- if (priv->rtl_chip == RTL8192E)
- rtl8xxxu_write32(priv, REG_AFE_XTAL_CTRL, 0x000f81fb);
-
- return 0;
-}
-
-static int rtl8xxxu_init_rf_regs(struct rtl8xxxu_priv *priv,
- struct rtl8xxxu_rfregval *array,
- enum rtl8xxxu_rfpath path)
-{
- int i, ret;
- u8 reg;
- u32 val;
-
- for (i = 0; ; i++) {
- reg = array[i].reg;
- val = array[i].val;
-
- if (reg == 0xff && val == 0xffffffff)
- break;
-
- switch (reg) {
- case 0xfe:
- msleep(50);
- continue;
- case 0xfd:
- mdelay(5);
- continue;
- case 0xfc:
- mdelay(1);
- continue;
- case 0xfb:
- udelay(50);
- continue;
- case 0xfa:
- udelay(5);
- continue;
- case 0xf9:
- udelay(1);
- continue;
- }
-
- ret = rtl8xxxu_write_rfreg(priv, path, reg, val);
- if (ret) {
- dev_warn(&priv->udev->dev,
- "Failed to initialize RF\n");
- return -EAGAIN;
- }
- udelay(1);
- }
-
- return 0;
-}
-
-static int rtl8xxxu_init_phy_rf(struct rtl8xxxu_priv *priv,
- struct rtl8xxxu_rfregval *table,
- enum rtl8xxxu_rfpath path)
-{
- u32 val32;
- u16 val16, rfsi_rfenv;
- u16 reg_sw_ctrl, reg_int_oe, reg_hssi_parm2;
-
- switch (path) {
- case RF_A:
- reg_sw_ctrl = REG_FPGA0_XA_RF_SW_CTRL;
- reg_int_oe = REG_FPGA0_XA_RF_INT_OE;
- reg_hssi_parm2 = REG_FPGA0_XA_HSSI_PARM2;
- break;
- case RF_B:
- reg_sw_ctrl = REG_FPGA0_XB_RF_SW_CTRL;
- reg_int_oe = REG_FPGA0_XB_RF_INT_OE;
- reg_hssi_parm2 = REG_FPGA0_XB_HSSI_PARM2;
- break;
- default:
- dev_err(&priv->udev->dev, "%s:Unsupported RF path %c\n",
- __func__, path + 'A');
- return -EINVAL;
- }
- /* For path B, use XB */
- rfsi_rfenv = rtl8xxxu_read16(priv, reg_sw_ctrl);
- rfsi_rfenv &= FPGA0_RF_RFENV;
-
- /*
- * These two we might be able to optimize into one
- */
- val32 = rtl8xxxu_read32(priv, reg_int_oe);
- val32 |= BIT(20); /* 0x10 << 16 */
- rtl8xxxu_write32(priv, reg_int_oe, val32);
- udelay(1);
-
- val32 = rtl8xxxu_read32(priv, reg_int_oe);
- val32 |= BIT(4);
- rtl8xxxu_write32(priv, reg_int_oe, val32);
- udelay(1);
-
- /*
- * These two we might be able to optimize into one
- */
- val32 = rtl8xxxu_read32(priv, reg_hssi_parm2);
- val32 &= ~FPGA0_HSSI_3WIRE_ADDR_LEN;
- rtl8xxxu_write32(priv, reg_hssi_parm2, val32);
- udelay(1);
-
- val32 = rtl8xxxu_read32(priv, reg_hssi_parm2);
- val32 &= ~FPGA0_HSSI_3WIRE_DATA_LEN;
- rtl8xxxu_write32(priv, reg_hssi_parm2, val32);
- udelay(1);
-
- rtl8xxxu_init_rf_regs(priv, table, path);
-
- /* For path B, use XB */
- val16 = rtl8xxxu_read16(priv, reg_sw_ctrl);
- val16 &= ~FPGA0_RF_RFENV;
- val16 |= rfsi_rfenv;
- rtl8xxxu_write16(priv, reg_sw_ctrl, val16);
-
- return 0;
-}
-
-static int rtl8723au_init_phy_rf(struct rtl8xxxu_priv *priv)
-{
- int ret;
-
- ret = rtl8xxxu_init_phy_rf(priv, rtl8723au_radioa_1t_init_table, RF_A);
-
- /* Reduce 80M spur */
- rtl8xxxu_write32(priv, REG_AFE_XTAL_CTRL, 0x0381808d);
- rtl8xxxu_write32(priv, REG_AFE_PLL_CTRL, 0xf0ffff83);
- rtl8xxxu_write32(priv, REG_AFE_PLL_CTRL, 0xf0ffff82);
- rtl8xxxu_write32(priv, REG_AFE_PLL_CTRL, 0xf0ffff83);
-
- return ret;
-}
-
-static int rtl8723bu_init_phy_rf(struct rtl8xxxu_priv *priv)
-{
- int ret;
-
- ret = rtl8xxxu_init_phy_rf(priv, rtl8723bu_radioa_1t_init_table, RF_A);
- /*
- * PHY LCK
- */
- rtl8xxxu_write_rfreg(priv, RF_A, 0xb0, 0xdfbe0);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_MODE_AG, 0x8c01);
- msleep(200);
- rtl8xxxu_write_rfreg(priv, RF_A, 0xb0, 0xdffe0);
-
- return ret;
-}
-
-#ifdef CONFIG_RTL8XXXU_UNTESTED
-static int rtl8192cu_init_phy_rf(struct rtl8xxxu_priv *priv)
-{
- struct rtl8xxxu_rfregval *rftable;
- int ret;
-
- if (priv->rtl_chip == RTL8188R) {
- rftable = rtl8188ru_radioa_1t_highpa_table;
- ret = rtl8xxxu_init_phy_rf(priv, rftable, RF_A);
- } else if (priv->rf_paths == 1) {
- rftable = rtl8192cu_radioa_1t_init_table;
- ret = rtl8xxxu_init_phy_rf(priv, rftable, RF_A);
- } else {
- rftable = rtl8192cu_radioa_2t_init_table;
- ret = rtl8xxxu_init_phy_rf(priv, rftable, RF_A);
- if (ret)
- goto exit;
- rftable = rtl8192cu_radiob_2t_init_table;
- ret = rtl8xxxu_init_phy_rf(priv, rftable, RF_B);
- }
-
-exit:
- return ret;
-}
-#endif
-
-static int rtl8192eu_init_phy_rf(struct rtl8xxxu_priv *priv)
-{
- int ret;
-
- ret = rtl8xxxu_init_phy_rf(priv, rtl8192eu_radioa_init_table, RF_A);
- if (ret)
- goto exit;
-
- ret = rtl8xxxu_init_phy_rf(priv, rtl8192eu_radiob_init_table, RF_B);
-
-exit:
- return ret;
-}
-
-static int rtl8xxxu_llt_write(struct rtl8xxxu_priv *priv, u8 address, u8 data)
-{
- int ret = -EBUSY;
- int count = 0;
- u32 value;
-
- value = LLT_OP_WRITE | address << 8 | data;
-
- rtl8xxxu_write32(priv, REG_LLT_INIT, value);
-
- do {
- value = rtl8xxxu_read32(priv, REG_LLT_INIT);
- if ((value & LLT_OP_MASK) == LLT_OP_INACTIVE) {
- ret = 0;
- break;
- }
- } while (count++ < 20);
-
- return ret;
-}
-
-static int rtl8xxxu_init_llt_table(struct rtl8xxxu_priv *priv, u8 last_tx_page)
-{
- int ret;
- int i;
-
- for (i = 0; i < last_tx_page; i++) {
- ret = rtl8xxxu_llt_write(priv, i, i + 1);
- if (ret)
- goto exit;
- }
-
- ret = rtl8xxxu_llt_write(priv, last_tx_page, 0xff);
- if (ret)
- goto exit;
-
- /* Mark remaining pages as a ring buffer */
- for (i = last_tx_page + 1; i < 0xff; i++) {
- ret = rtl8xxxu_llt_write(priv, i, (i + 1));
- if (ret)
- goto exit;
- }
-
- /* Let last entry point to the start entry of ring buffer */
- ret = rtl8xxxu_llt_write(priv, 0xff, last_tx_page + 1);
- if (ret)
- goto exit;
-
-exit:
- return ret;
-}
-
-static int rtl8xxxu_auto_llt_table(struct rtl8xxxu_priv *priv, u8 last_tx_page)
-{
- u32 val32;
- int ret = 0;
- int i;
-
- val32 = rtl8xxxu_read32(priv, REG_AUTO_LLT);
- val32 |= AUTO_LLT_INIT_LLT;
- rtl8xxxu_write32(priv, REG_AUTO_LLT, val32);
-
- for (i = 500; i; i--) {
- val32 = rtl8xxxu_read32(priv, REG_AUTO_LLT);
- if (!(val32 & AUTO_LLT_INIT_LLT))
- break;
- usleep_range(2, 4);
- }
-
- if (!i) {
- ret = -EBUSY;
- dev_warn(&priv->udev->dev, "LLT table init failed\n");
- }
-
- return ret;
-}
-
-static int rtl8xxxu_init_queue_priority(struct rtl8xxxu_priv *priv)
-{
- u16 val16, hi, lo;
- u16 hiq, mgq, bkq, beq, viq, voq;
- int hip, mgp, bkp, bep, vip, vop;
- int ret = 0;
-
- switch (priv->ep_tx_count) {
- case 1:
- if (priv->ep_tx_high_queue) {
- hi = TRXDMA_QUEUE_HIGH;
- } else if (priv->ep_tx_low_queue) {
- hi = TRXDMA_QUEUE_LOW;
- } else if (priv->ep_tx_normal_queue) {
- hi = TRXDMA_QUEUE_NORMAL;
- } else {
- hi = 0;
- ret = -EINVAL;
- }
-
- hiq = hi;
- mgq = hi;
- bkq = hi;
- beq = hi;
- viq = hi;
- voq = hi;
-
- hip = 0;
- mgp = 0;
- bkp = 0;
- bep = 0;
- vip = 0;
- vop = 0;
- break;
- case 2:
- if (priv->ep_tx_high_queue && priv->ep_tx_low_queue) {
- hi = TRXDMA_QUEUE_HIGH;
- lo = TRXDMA_QUEUE_LOW;
- } else if (priv->ep_tx_normal_queue && priv->ep_tx_low_queue) {
- hi = TRXDMA_QUEUE_NORMAL;
- lo = TRXDMA_QUEUE_LOW;
- } else if (priv->ep_tx_high_queue && priv->ep_tx_normal_queue) {
- hi = TRXDMA_QUEUE_HIGH;
- lo = TRXDMA_QUEUE_NORMAL;
- } else {
- ret = -EINVAL;
- hi = 0;
- lo = 0;
- }
-
- hiq = hi;
- mgq = hi;
- bkq = lo;
- beq = lo;
- viq = hi;
- voq = hi;
-
- hip = 0;
- mgp = 0;
- bkp = 1;
- bep = 1;
- vip = 0;
- vop = 0;
- break;
- case 3:
- beq = TRXDMA_QUEUE_LOW;
- bkq = TRXDMA_QUEUE_LOW;
- viq = TRXDMA_QUEUE_NORMAL;
- voq = TRXDMA_QUEUE_HIGH;
- mgq = TRXDMA_QUEUE_HIGH;
- hiq = TRXDMA_QUEUE_HIGH;
-
- hip = hiq ^ 3;
- mgp = mgq ^ 3;
- bkp = bkq ^ 3;
- bep = beq ^ 3;
- vip = viq ^ 3;
- vop = viq ^ 3;
- break;
- default:
- ret = -EINVAL;
- }
-
- /*
- * None of the vendor drivers are configuring the beacon
- * queue here .... why?
- */
- if (!ret) {
- val16 = rtl8xxxu_read16(priv, REG_TRXDMA_CTRL);
- val16 &= 0x7;
- val16 |= (voq << TRXDMA_CTRL_VOQ_SHIFT) |
- (viq << TRXDMA_CTRL_VIQ_SHIFT) |
- (beq << TRXDMA_CTRL_BEQ_SHIFT) |
- (bkq << TRXDMA_CTRL_BKQ_SHIFT) |
- (mgq << TRXDMA_CTRL_MGQ_SHIFT) |
- (hiq << TRXDMA_CTRL_HIQ_SHIFT);
- rtl8xxxu_write16(priv, REG_TRXDMA_CTRL, val16);
-
- priv->pipe_out[TXDESC_QUEUE_VO] =
- usb_sndbulkpipe(priv->udev, priv->out_ep[vop]);
- priv->pipe_out[TXDESC_QUEUE_VI] =
- usb_sndbulkpipe(priv->udev, priv->out_ep[vip]);
- priv->pipe_out[TXDESC_QUEUE_BE] =
- usb_sndbulkpipe(priv->udev, priv->out_ep[bep]);
- priv->pipe_out[TXDESC_QUEUE_BK] =
- usb_sndbulkpipe(priv->udev, priv->out_ep[bkp]);
- priv->pipe_out[TXDESC_QUEUE_BEACON] =
- usb_sndbulkpipe(priv->udev, priv->out_ep[0]);
- priv->pipe_out[TXDESC_QUEUE_MGNT] =
- usb_sndbulkpipe(priv->udev, priv->out_ep[mgp]);
- priv->pipe_out[TXDESC_QUEUE_HIGH] =
- usb_sndbulkpipe(priv->udev, priv->out_ep[hip]);
- priv->pipe_out[TXDESC_QUEUE_CMD] =
- usb_sndbulkpipe(priv->udev, priv->out_ep[0]);
- }
-
- return ret;
-}
-
-static void rtl8xxxu_fill_iqk_matrix_a(struct rtl8xxxu_priv *priv,
- bool iqk_ok, int result[][8],
- int candidate, bool tx_only)
-{
- u32 oldval, x, tx0_a, reg;
- int y, tx0_c;
- u32 val32;
-
- if (!iqk_ok)
- return;
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_XA_TX_IQ_IMBALANCE);
- oldval = val32 >> 22;
-
- x = result[candidate][0];
- if ((x & 0x00000200) != 0)
- x = x | 0xfffffc00;
- tx0_a = (x * oldval) >> 8;
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_XA_TX_IQ_IMBALANCE);
- val32 &= ~0x3ff;
- val32 |= tx0_a;
- rtl8xxxu_write32(priv, REG_OFDM0_XA_TX_IQ_IMBALANCE, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_ENERGY_CCA_THRES);
- val32 &= ~BIT(31);
- if ((x * oldval >> 7) & 0x1)
- val32 |= BIT(31);
- rtl8xxxu_write32(priv, REG_OFDM0_ENERGY_CCA_THRES, val32);
-
- y = result[candidate][1];
- if ((y & 0x00000200) != 0)
- y = y | 0xfffffc00;
- tx0_c = (y * oldval) >> 8;
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_XC_TX_AFE);
- val32 &= ~0xf0000000;
- val32 |= (((tx0_c & 0x3c0) >> 6) << 28);
- rtl8xxxu_write32(priv, REG_OFDM0_XC_TX_AFE, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_XA_TX_IQ_IMBALANCE);
- val32 &= ~0x003f0000;
- val32 |= ((tx0_c & 0x3f) << 16);
- rtl8xxxu_write32(priv, REG_OFDM0_XA_TX_IQ_IMBALANCE, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_ENERGY_CCA_THRES);
- val32 &= ~BIT(29);
- if ((y * oldval >> 7) & 0x1)
- val32 |= BIT(29);
- rtl8xxxu_write32(priv, REG_OFDM0_ENERGY_CCA_THRES, val32);
-
- if (tx_only) {
- dev_dbg(&priv->udev->dev, "%s: only TX\n", __func__);
- return;
- }
-
- reg = result[candidate][2];
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_XA_RX_IQ_IMBALANCE);
- val32 &= ~0x3ff;
- val32 |= (reg & 0x3ff);
- rtl8xxxu_write32(priv, REG_OFDM0_XA_RX_IQ_IMBALANCE, val32);
-
- reg = result[candidate][3] & 0x3F;
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_XA_RX_IQ_IMBALANCE);
- val32 &= ~0xfc00;
- val32 |= ((reg << 10) & 0xfc00);
- rtl8xxxu_write32(priv, REG_OFDM0_XA_RX_IQ_IMBALANCE, val32);
-
- reg = (result[candidate][3] >> 6) & 0xF;
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_RX_IQ_EXT_ANTA);
- val32 &= ~0xf0000000;
- val32 |= (reg << 28);
- rtl8xxxu_write32(priv, REG_OFDM0_RX_IQ_EXT_ANTA, val32);
-}
-
-static void rtl8xxxu_fill_iqk_matrix_b(struct rtl8xxxu_priv *priv,
- bool iqk_ok, int result[][8],
- int candidate, bool tx_only)
-{
- u32 oldval, x, tx1_a, reg;
- int y, tx1_c;
- u32 val32;
-
- if (!iqk_ok)
- return;
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_XB_TX_IQ_IMBALANCE);
- oldval = val32 >> 22;
-
- x = result[candidate][4];
- if ((x & 0x00000200) != 0)
- x = x | 0xfffffc00;
- tx1_a = (x * oldval) >> 8;
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_XB_TX_IQ_IMBALANCE);
- val32 &= ~0x3ff;
- val32 |= tx1_a;
- rtl8xxxu_write32(priv, REG_OFDM0_XB_TX_IQ_IMBALANCE, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_ENERGY_CCA_THRES);
- val32 &= ~BIT(27);
- if ((x * oldval >> 7) & 0x1)
- val32 |= BIT(27);
- rtl8xxxu_write32(priv, REG_OFDM0_ENERGY_CCA_THRES, val32);
-
- y = result[candidate][5];
- if ((y & 0x00000200) != 0)
- y = y | 0xfffffc00;
- tx1_c = (y * oldval) >> 8;
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_XD_TX_AFE);
- val32 &= ~0xf0000000;
- val32 |= (((tx1_c & 0x3c0) >> 6) << 28);
- rtl8xxxu_write32(priv, REG_OFDM0_XD_TX_AFE, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_XB_TX_IQ_IMBALANCE);
- val32 &= ~0x003f0000;
- val32 |= ((tx1_c & 0x3f) << 16);
- rtl8xxxu_write32(priv, REG_OFDM0_XB_TX_IQ_IMBALANCE, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_ENERGY_CCA_THRES);
- val32 &= ~BIT(25);
- if ((y * oldval >> 7) & 0x1)
- val32 |= BIT(25);
- rtl8xxxu_write32(priv, REG_OFDM0_ENERGY_CCA_THRES, val32);
-
- if (tx_only) {
- dev_dbg(&priv->udev->dev, "%s: only TX\n", __func__);
- return;
- }
-
- reg = result[candidate][6];
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_XB_RX_IQ_IMBALANCE);
- val32 &= ~0x3ff;
- val32 |= (reg & 0x3ff);
- rtl8xxxu_write32(priv, REG_OFDM0_XB_RX_IQ_IMBALANCE, val32);
-
- reg = result[candidate][7] & 0x3f;
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_XB_RX_IQ_IMBALANCE);
- val32 &= ~0xfc00;
- val32 |= ((reg << 10) & 0xfc00);
- rtl8xxxu_write32(priv, REG_OFDM0_XB_RX_IQ_IMBALANCE, val32);
-
- reg = (result[candidate][7] >> 6) & 0xf;
-
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_AGCR_SSI_TABLE);
- val32 &= ~0x0000f000;
- val32 |= (reg << 12);
- rtl8xxxu_write32(priv, REG_OFDM0_AGCR_SSI_TABLE, val32);
-}
-
-#define MAX_TOLERANCE 5
-
-static bool rtl8xxxu_simularity_compare(struct rtl8xxxu_priv *priv,
- int result[][8], int c1, int c2)
-{
- u32 i, j, diff, simubitmap, bound = 0;
- int candidate[2] = {-1, -1}; /* for path A and path B */
- bool retval = true;
-
- if (priv->tx_paths > 1)
- bound = 8;
- else
- bound = 4;
-
- simubitmap = 0;
-
- for (i = 0; i < bound; i++) {
- diff = (result[c1][i] > result[c2][i]) ?
- (result[c1][i] - result[c2][i]) :
- (result[c2][i] - result[c1][i]);
- if (diff > MAX_TOLERANCE) {
- if ((i == 2 || i == 6) && !simubitmap) {
- if (result[c1][i] + result[c1][i + 1] == 0)
- candidate[(i / 4)] = c2;
- else if (result[c2][i] + result[c2][i + 1] == 0)
- candidate[(i / 4)] = c1;
- else
- simubitmap = simubitmap | (1 << i);
- } else {
- simubitmap = simubitmap | (1 << i);
- }
- }
- }
-
- if (simubitmap == 0) {
- for (i = 0; i < (bound / 4); i++) {
- if (candidate[i] >= 0) {
- for (j = i * 4; j < (i + 1) * 4 - 2; j++)
- result[3][j] = result[candidate[i]][j];
- retval = false;
- }
- }
- return retval;
- } else if (!(simubitmap & 0x0f)) {
- /* path A OK */
- for (i = 0; i < 4; i++)
- result[3][i] = result[c1][i];
- } else if (!(simubitmap & 0xf0) && priv->tx_paths > 1) {
- /* path B OK */
- for (i = 4; i < 8; i++)
- result[3][i] = result[c1][i];
- }
-
- return false;
-}
-
-static bool rtl8xxxu_gen2_simularity_compare(struct rtl8xxxu_priv *priv,
- int result[][8], int c1, int c2)
-{
- u32 i, j, diff, simubitmap, bound = 0;
- int candidate[2] = {-1, -1}; /* for path A and path B */
- int tmp1, tmp2;
- bool retval = true;
-
- if (priv->tx_paths > 1)
- bound = 8;
- else
- bound = 4;
-
- simubitmap = 0;
-
- for (i = 0; i < bound; i++) {
- if (i & 1) {
- if ((result[c1][i] & 0x00000200))
- tmp1 = result[c1][i] | 0xfffffc00;
- else
- tmp1 = result[c1][i];
-
- if ((result[c2][i]& 0x00000200))
- tmp2 = result[c2][i] | 0xfffffc00;
- else
- tmp2 = result[c2][i];
- } else {
- tmp1 = result[c1][i];
- tmp2 = result[c2][i];
- }
-
- diff = (tmp1 > tmp2) ? (tmp1 - tmp2) : (tmp2 - tmp1);
-
- if (diff > MAX_TOLERANCE) {
- if ((i == 2 || i == 6) && !simubitmap) {
- if (result[c1][i] + result[c1][i + 1] == 0)
- candidate[(i / 4)] = c2;
- else if (result[c2][i] + result[c2][i + 1] == 0)
- candidate[(i / 4)] = c1;
- else
- simubitmap = simubitmap | (1 << i);
- } else {
- simubitmap = simubitmap | (1 << i);
- }
- }
- }
-
- if (simubitmap == 0) {
- for (i = 0; i < (bound / 4); i++) {
- if (candidate[i] >= 0) {
- for (j = i * 4; j < (i + 1) * 4 - 2; j++)
- result[3][j] = result[candidate[i]][j];
- retval = false;
- }
- }
- return retval;
- } else {
- if (!(simubitmap & 0x03)) {
- /* path A TX OK */
- for (i = 0; i < 2; i++)
- result[3][i] = result[c1][i];
- }
-
- if (!(simubitmap & 0x0c)) {
- /* path A RX OK */
- for (i = 2; i < 4; i++)
- result[3][i] = result[c1][i];
- }
-
- if (!(simubitmap & 0x30) && priv->tx_paths > 1) {
- /* path B RX OK */
- for (i = 4; i < 6; i++)
- result[3][i] = result[c1][i];
- }
-
- if (!(simubitmap & 0x30) && priv->tx_paths > 1) {
- /* path B RX OK */
- for (i = 6; i < 8; i++)
- result[3][i] = result[c1][i];
- }
- }
-
- return false;
-}
-
-static void
-rtl8xxxu_save_mac_regs(struct rtl8xxxu_priv *priv, const u32 *reg, u32 *backup)
-{
- int i;
-
- for (i = 0; i < (RTL8XXXU_MAC_REGS - 1); i++)
- backup[i] = rtl8xxxu_read8(priv, reg[i]);
-
- backup[i] = rtl8xxxu_read32(priv, reg[i]);
-}
-
-static void rtl8xxxu_restore_mac_regs(struct rtl8xxxu_priv *priv,
- const u32 *reg, u32 *backup)
-{
- int i;
-
- for (i = 0; i < (RTL8XXXU_MAC_REGS - 1); i++)
- rtl8xxxu_write8(priv, reg[i], backup[i]);
-
- rtl8xxxu_write32(priv, reg[i], backup[i]);
-}
-
-static void rtl8xxxu_save_regs(struct rtl8xxxu_priv *priv, const u32 *regs,
- u32 *backup, int count)
-{
- int i;
-
- for (i = 0; i < count; i++)
- backup[i] = rtl8xxxu_read32(priv, regs[i]);
-}
-
-static void rtl8xxxu_restore_regs(struct rtl8xxxu_priv *priv, const u32 *regs,
- u32 *backup, int count)
-{
- int i;
-
- for (i = 0; i < count; i++)
- rtl8xxxu_write32(priv, regs[i], backup[i]);
-}
-
-
-static void rtl8xxxu_path_adda_on(struct rtl8xxxu_priv *priv, const u32 *regs,
- bool path_a_on)
-{
- u32 path_on;
- int i;
-
- if (priv->tx_paths == 1) {
- path_on = priv->fops->adda_1t_path_on;
- rtl8xxxu_write32(priv, regs[0], priv->fops->adda_1t_init);
- } else {
- path_on = path_a_on ? priv->fops->adda_2t_path_on_a :
- priv->fops->adda_2t_path_on_b;
-
- rtl8xxxu_write32(priv, regs[0], path_on);
- }
-
- for (i = 1 ; i < RTL8XXXU_ADDA_REGS ; i++)
- rtl8xxxu_write32(priv, regs[i], path_on);
-}
-
-static void rtl8xxxu_mac_calibration(struct rtl8xxxu_priv *priv,
- const u32 *regs, u32 *backup)
-{
- int i = 0;
-
- rtl8xxxu_write8(priv, regs[i], 0x3f);
-
- for (i = 1 ; i < (RTL8XXXU_MAC_REGS - 1); i++)
- rtl8xxxu_write8(priv, regs[i], (u8)(backup[i] & ~BIT(3)));
-
- rtl8xxxu_write8(priv, regs[i], (u8)(backup[i] & ~BIT(5)));
-}
-
-static int rtl8xxxu_iqk_path_a(struct rtl8xxxu_priv *priv)
-{
- u32 reg_eac, reg_e94, reg_e9c, reg_ea4, val32;
- int result = 0;
-
- /* path-A IQK setting */
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x10008c1f);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x10008c1f);
- rtl8xxxu_write32(priv, REG_TX_IQK_PI_A, 0x82140102);
-
- val32 = (priv->rf_paths > 1) ? 0x28160202 :
- /*IS_81xxC_VENDOR_UMC_B_CUT(pHalData->VersionID)?0x28160202: */
- 0x28160502;
- rtl8xxxu_write32(priv, REG_RX_IQK_PI_A, val32);
-
- /* path-B IQK setting */
- if (priv->rf_paths > 1) {
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_B, 0x10008c22);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_B, 0x10008c22);
- rtl8xxxu_write32(priv, REG_TX_IQK_PI_B, 0x82140102);
- rtl8xxxu_write32(priv, REG_RX_IQK_PI_B, 0x28160202);
- }
-
- /* LO calibration setting */
- rtl8xxxu_write32(priv, REG_IQK_AGC_RSP, 0x001028d1);
-
- /* One shot, path A LOK & IQK */
- rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf9000000);
- rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf8000000);
-
- mdelay(1);
-
- /* Check failed */
- reg_eac = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_A_2);
- reg_e94 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_A);
- reg_e9c = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_A);
- reg_ea4 = rtl8xxxu_read32(priv, REG_RX_POWER_BEFORE_IQK_A_2);
-
- if (!(reg_eac & BIT(28)) &&
- ((reg_e94 & 0x03ff0000) != 0x01420000) &&
- ((reg_e9c & 0x03ff0000) != 0x00420000))
- result |= 0x01;
- else /* If TX not OK, ignore RX */
- goto out;
-
- /* If TX is OK, check whether RX is OK */
- if (!(reg_eac & BIT(27)) &&
- ((reg_ea4 & 0x03ff0000) != 0x01320000) &&
- ((reg_eac & 0x03ff0000) != 0x00360000))
- result |= 0x02;
- else
- dev_warn(&priv->udev->dev, "%s: Path A RX IQK failed!\n",
- __func__);
-out:
- return result;
-}
-
-static int rtl8xxxu_iqk_path_b(struct rtl8xxxu_priv *priv)
-{
- u32 reg_eac, reg_eb4, reg_ebc, reg_ec4, reg_ecc;
- int result = 0;
-
- /* One shot, path B LOK & IQK */
- rtl8xxxu_write32(priv, REG_IQK_AGC_CONT, 0x00000002);
- rtl8xxxu_write32(priv, REG_IQK_AGC_CONT, 0x00000000);
-
- mdelay(1);
-
- /* Check failed */
- reg_eac = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_A_2);
- reg_eb4 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_B);
- reg_ebc = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_B);
- reg_ec4 = rtl8xxxu_read32(priv, REG_RX_POWER_BEFORE_IQK_B_2);
- reg_ecc = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_B_2);
-
- if (!(reg_eac & BIT(31)) &&
- ((reg_eb4 & 0x03ff0000) != 0x01420000) &&
- ((reg_ebc & 0x03ff0000) != 0x00420000))
- result |= 0x01;
- else
- goto out;
-
- if (!(reg_eac & BIT(30)) &&
- (((reg_ec4 & 0x03ff0000) >> 16) != 0x132) &&
- (((reg_ecc & 0x03ff0000) >> 16) != 0x36))
- result |= 0x02;
- else
- dev_warn(&priv->udev->dev, "%s: Path B RX IQK failed!\n",
- __func__);
-out:
- return result;
-}
-
-static int rtl8723bu_iqk_path_a(struct rtl8xxxu_priv *priv)
-{
- u32 reg_eac, reg_e94, reg_e9c, path_sel, val32;
- int result = 0;
-
- path_sel = rtl8xxxu_read32(priv, REG_S0S1_PATH_SWITCH);
-
- /*
- * Leave IQK mode
- */
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
- val32 &= 0x000000ff;
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
-
- /*
- * Enable path A PA in TX IQK mode
- */
- val32 = rtl8xxxu_read_rfreg(priv, RF_A, RF6052_REG_WE_LUT);
- val32 |= 0x80000;
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_WE_LUT, val32);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_RCK_OS, 0x20000);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G1, 0x0003f);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G2, 0xc7f87);
-
- /*
- * Tx IQK setting
- */
- rtl8xxxu_write32(priv, REG_TX_IQK, 0x01007c00);
- rtl8xxxu_write32(priv, REG_RX_IQK, 0x01004800);
-
- /* path-A IQK setting */
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x18008c1c);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x38008c1c);
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_B, 0x38008c1c);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_B, 0x38008c1c);
-
- rtl8xxxu_write32(priv, REG_TX_IQK_PI_A, 0x821403ea);
- rtl8xxxu_write32(priv, REG_RX_IQK_PI_A, 0x28110000);
- rtl8xxxu_write32(priv, REG_TX_IQK_PI_B, 0x82110000);
- rtl8xxxu_write32(priv, REG_RX_IQK_PI_B, 0x28110000);
-
- /* LO calibration setting */
- rtl8xxxu_write32(priv, REG_IQK_AGC_RSP, 0x00462911);
-
- /*
- * Enter IQK mode
- */
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
- val32 &= 0x000000ff;
- val32 |= 0x80800000;
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
-
- /*
- * The vendor driver indicates the USB module is always using
- * S0S1 path 1 for the 8723bu. This may be different for 8192eu
- */
- if (priv->rf_paths > 1)
- rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, 0x00000000);
- else
- rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, 0x00000280);
-
- /*
- * Bit 12 seems to be BT_GRANT, and is only found in the 8723bu.
- * No trace of this in the 8192eu or 8188eu vendor drivers.
- */
- rtl8xxxu_write32(priv, REG_BT_CONTROL_8723BU, 0x00000800);
-
- /* One shot, path A LOK & IQK */
- rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf9000000);
- rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf8000000);
-
- mdelay(1);
-
- /* Restore Ant Path */
- rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, path_sel);
-#ifdef RTL8723BU_BT
- /* GNT_BT = 1 */
- rtl8xxxu_write32(priv, REG_BT_CONTROL_8723BU, 0x00001800);
-#endif
-
- /*
- * Leave IQK mode
- */
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
- val32 &= 0x000000ff;
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
-
- /* Check failed */
- reg_eac = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_A_2);
- reg_e94 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_A);
- reg_e9c = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_A);
-
- val32 = (reg_e9c >> 16) & 0x3ff;
- if (val32 & 0x200)
- val32 = 0x400 - val32;
-
- if (!(reg_eac & BIT(28)) &&
- ((reg_e94 & 0x03ff0000) != 0x01420000) &&
- ((reg_e9c & 0x03ff0000) != 0x00420000) &&
- ((reg_e94 & 0x03ff0000) < 0x01100000) &&
- ((reg_e94 & 0x03ff0000) > 0x00f00000) &&
- val32 < 0xf)
- result |= 0x01;
- else /* If TX not OK, ignore RX */
- goto out;
-
-out:
- return result;
-}
-
-static int rtl8723bu_rx_iqk_path_a(struct rtl8xxxu_priv *priv)
-{
- u32 reg_ea4, reg_eac, reg_e94, reg_e9c, path_sel, val32;
- int result = 0;
-
- path_sel = rtl8xxxu_read32(priv, REG_S0S1_PATH_SWITCH);
-
- /*
- * Leave IQK mode
- */
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
- val32 &= 0x000000ff;
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
-
- /*
- * Enable path A PA in TX IQK mode
- */
- val32 = rtl8xxxu_read_rfreg(priv, RF_A, RF6052_REG_WE_LUT);
- val32 |= 0x80000;
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_WE_LUT, val32);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_RCK_OS, 0x30000);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G1, 0x0001f);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G2, 0xf7fb7);
-
- /*
- * Tx IQK setting
- */
- rtl8xxxu_write32(priv, REG_TX_IQK, 0x01007c00);
- rtl8xxxu_write32(priv, REG_RX_IQK, 0x01004800);
-
- /* path-A IQK setting */
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x18008c1c);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x38008c1c);
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_B, 0x38008c1c);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_B, 0x38008c1c);
-
- rtl8xxxu_write32(priv, REG_TX_IQK_PI_A, 0x82160ff0);
- rtl8xxxu_write32(priv, REG_RX_IQK_PI_A, 0x28110000);
- rtl8xxxu_write32(priv, REG_TX_IQK_PI_B, 0x82110000);
- rtl8xxxu_write32(priv, REG_RX_IQK_PI_B, 0x28110000);
-
- /* LO calibration setting */
- rtl8xxxu_write32(priv, REG_IQK_AGC_RSP, 0x0046a911);
-
- /*
- * Enter IQK mode
- */
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
- val32 &= 0x000000ff;
- val32 |= 0x80800000;
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
-
- /*
- * The vendor driver indicates the USB module is always using
- * S0S1 path 1 for the 8723bu. This may be different for 8192eu
- */
- if (priv->rf_paths > 1)
- rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, 0x00000000);
- else
- rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, 0x00000280);
-
- /*
- * Bit 12 seems to be BT_GRANT, and is only found in the 8723bu.
- * No trace of this in the 8192eu or 8188eu vendor drivers.
- */
- rtl8xxxu_write32(priv, REG_BT_CONTROL_8723BU, 0x00000800);
-
- /* One shot, path A LOK & IQK */
- rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf9000000);
- rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf8000000);
-
- mdelay(1);
-
- /* Restore Ant Path */
- rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, path_sel);
-#ifdef RTL8723BU_BT
- /* GNT_BT = 1 */
- rtl8xxxu_write32(priv, REG_BT_CONTROL_8723BU, 0x00001800);
-#endif
-
- /*
- * Leave IQK mode
- */
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
- val32 &= 0x000000ff;
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
-
- /* Check failed */
- reg_eac = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_A_2);
- reg_e94 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_A);
- reg_e9c = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_A);
-
- val32 = (reg_e9c >> 16) & 0x3ff;
- if (val32 & 0x200)
- val32 = 0x400 - val32;
-
- if (!(reg_eac & BIT(28)) &&
- ((reg_e94 & 0x03ff0000) != 0x01420000) &&
- ((reg_e9c & 0x03ff0000) != 0x00420000) &&
- ((reg_e94 & 0x03ff0000) < 0x01100000) &&
- ((reg_e94 & 0x03ff0000) > 0x00f00000) &&
- val32 < 0xf)
- result |= 0x01;
- else /* If TX not OK, ignore RX */
- goto out;
-
- val32 = 0x80007c00 | (reg_e94 &0x3ff0000) |
- ((reg_e9c & 0x3ff0000) >> 16);
- rtl8xxxu_write32(priv, REG_TX_IQK, val32);
-
- /*
- * Modify RX IQK mode
- */
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
- val32 &= 0x000000ff;
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
- val32 = rtl8xxxu_read_rfreg(priv, RF_A, RF6052_REG_WE_LUT);
- val32 |= 0x80000;
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_WE_LUT, val32);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_RCK_OS, 0x30000);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G1, 0x0001f);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G2, 0xf7d77);
-
- /*
- * PA, PAD setting
- */
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_DF, 0xf80);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_55, 0x4021f);
-
- /*
- * RX IQK setting
- */
- rtl8xxxu_write32(priv, REG_RX_IQK, 0x01004800);
-
- /* path-A IQK setting */
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x38008c1c);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x18008c1c);
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_B, 0x38008c1c);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_B, 0x38008c1c);
-
- rtl8xxxu_write32(priv, REG_TX_IQK_PI_A, 0x82110000);
- rtl8xxxu_write32(priv, REG_RX_IQK_PI_A, 0x2816001f);
- rtl8xxxu_write32(priv, REG_TX_IQK_PI_B, 0x82110000);
- rtl8xxxu_write32(priv, REG_RX_IQK_PI_B, 0x28110000);
-
- /* LO calibration setting */
- rtl8xxxu_write32(priv, REG_IQK_AGC_RSP, 0x0046a8d1);
-
- /*
- * Enter IQK mode
- */
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
- val32 &= 0x000000ff;
- val32 |= 0x80800000;
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
-
- if (priv->rf_paths > 1)
- rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, 0x00000000);
- else
- rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, 0x00000280);
-
- /*
- * Disable BT
- */
- rtl8xxxu_write32(priv, REG_BT_CONTROL_8723BU, 0x00000800);
-
- /* One shot, path A LOK & IQK */
- rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf9000000);
- rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf8000000);
-
- mdelay(1);
-
- /* Restore Ant Path */
- rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, path_sel);
-#ifdef RTL8723BU_BT
- /* GNT_BT = 1 */
- rtl8xxxu_write32(priv, REG_BT_CONTROL_8723BU, 0x00001800);
-#endif
-
- /*
- * Leave IQK mode
- */
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
- val32 &= 0x000000ff;
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
-
- /* Check failed */
- reg_eac = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_A_2);
- reg_ea4 = rtl8xxxu_read32(priv, REG_RX_POWER_BEFORE_IQK_A_2);
-
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_DF, 0x780);
-
- val32 = (reg_eac >> 16) & 0x3ff;
- if (val32 & 0x200)
- val32 = 0x400 - val32;
-
- if (!(reg_eac & BIT(27)) &&
- ((reg_ea4 & 0x03ff0000) != 0x01320000) &&
- ((reg_eac & 0x03ff0000) != 0x00360000) &&
- ((reg_ea4 & 0x03ff0000) < 0x01100000) &&
- ((reg_ea4 & 0x03ff0000) > 0x00f00000) &&
- val32 < 0xf)
- result |= 0x02;
- else /* If TX not OK, ignore RX */
- goto out;
-out:
- return result;
-}
-
-static int rtl8192eu_iqk_path_a(struct rtl8xxxu_priv *priv)
-{
- u32 reg_eac, reg_e94, reg_e9c;
- int result = 0;
-
- /*
- * TX IQK
- * PA/PAD controlled by 0x0
- */
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_DF, 0x00180);
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
-
- /* Path A IQK setting */
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x18008c1c);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x38008c1c);
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_B, 0x38008c1c);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_B, 0x38008c1c);
-
- rtl8xxxu_write32(priv, REG_TX_IQK_PI_A, 0x82140303);
- rtl8xxxu_write32(priv, REG_RX_IQK_PI_A, 0x68160000);
-
- /* LO calibration setting */
- rtl8xxxu_write32(priv, REG_IQK_AGC_RSP, 0x00462911);
-
- /* One shot, path A LOK & IQK */
- rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf9000000);
- rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf8000000);
-
- mdelay(10);
-
- /* Check failed */
- reg_eac = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_A_2);
- reg_e94 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_A);
- reg_e9c = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_A);
-
- if (!(reg_eac & BIT(28)) &&
- ((reg_e94 & 0x03ff0000) != 0x01420000) &&
- ((reg_e9c & 0x03ff0000) != 0x00420000))
- result |= 0x01;
-
- return result;
-}
-
-static int rtl8192eu_rx_iqk_path_a(struct rtl8xxxu_priv *priv)
-{
- u32 reg_ea4, reg_eac, reg_e94, reg_e9c, val32;
- int result = 0;
-
- /* Leave IQK mode */
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00);
-
- /* Enable path A PA in TX IQK mode */
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_WE_LUT, 0x800a0);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_RCK_OS, 0x30000);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G1, 0x0000f);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G2, 0xf117b);
-
- /* PA/PAD control by 0x56, and set = 0x0 */
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_DF, 0x00980);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_56, 0x51000);
-
- /* Enter IQK mode */
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
-
- /* TX IQK setting */
- rtl8xxxu_write32(priv, REG_TX_IQK, 0x01007c00);
- rtl8xxxu_write32(priv, REG_RX_IQK, 0x01004800);
-
- /* path-A IQK setting */
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x18008c1c);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x38008c1c);
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_B, 0x38008c1c);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_B, 0x38008c1c);
-
- rtl8xxxu_write32(priv, REG_TX_IQK_PI_A, 0x82160c1f);
- rtl8xxxu_write32(priv, REG_RX_IQK_PI_A, 0x68160c1f);
-
- /* LO calibration setting */
- rtl8xxxu_write32(priv, REG_IQK_AGC_RSP, 0x0046a911);
-
- /* One shot, path A LOK & IQK */
- rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xfa000000);
- rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf8000000);
-
- mdelay(10);
-
- /* Check failed */
- reg_eac = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_A_2);
- reg_e94 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_A);
- reg_e9c = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_A);
-
- if (!(reg_eac & BIT(28)) &&
- ((reg_e94 & 0x03ff0000) != 0x01420000) &&
- ((reg_e9c & 0x03ff0000) != 0x00420000)) {
- result |= 0x01;
- } else {
- /* PA/PAD controlled by 0x0 */
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_DF, 0x180);
- goto out;
- }
-
- val32 = 0x80007c00 |
- (reg_e94 & 0x03ff0000) | ((reg_e9c >> 16) & 0x03ff);
- rtl8xxxu_write32(priv, REG_TX_IQK, val32);
-
- /* Modify RX IQK mode table */
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
-
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_WE_LUT, 0x800a0);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_RCK_OS, 0x30000);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G1, 0x0000f);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G2, 0xf7ffa);
-
- /* PA/PAD control by 0x56, and set = 0x0 */
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_DF, 0x00980);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_56, 0x51000);
-
- /* Enter IQK mode */
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
-
- /* IQK setting */
- rtl8xxxu_write32(priv, REG_RX_IQK, 0x01004800);
-
- /* Path A IQK setting */
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x38008c1c);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x18008c1c);
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_B, 0x38008c1c);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_B, 0x38008c1c);
-
- rtl8xxxu_write32(priv, REG_TX_IQK_PI_A, 0x82160c1f);
- rtl8xxxu_write32(priv, REG_RX_IQK_PI_A, 0x28160c1f);
-
- /* LO calibration setting */
- rtl8xxxu_write32(priv, REG_IQK_AGC_RSP, 0x0046a891);
-
- /* One shot, path A LOK & IQK */
- rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xfa000000);
- rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf8000000);
-
- mdelay(10);
-
- reg_eac = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_A_2);
- reg_ea4 = rtl8xxxu_read32(priv, REG_RX_POWER_BEFORE_IQK_A_2);
-
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_DF, 0x180);
-
- if (!(reg_eac & BIT(27)) &&
- ((reg_ea4 & 0x03ff0000) != 0x01320000) &&
- ((reg_eac & 0x03ff0000) != 0x00360000))
- result |= 0x02;
- else
- dev_warn(&priv->udev->dev, "%s: Path A RX IQK failed!\n",
- __func__);
-
-out:
- return result;
-}
-
-static int rtl8192eu_iqk_path_b(struct rtl8xxxu_priv *priv)
-{
- u32 reg_eac, reg_eb4, reg_ebc;
- int result = 0;
-
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
- rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_UNKNOWN_DF, 0x00180);
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
-
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
-
- /* Path B IQK setting */
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x38008c1c);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x38008c1c);
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_B, 0x18008c1c);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_B, 0x38008c1c);
-
- rtl8xxxu_write32(priv, REG_TX_IQK_PI_B, 0x821403e2);
- rtl8xxxu_write32(priv, REG_RX_IQK_PI_B, 0x68160000);
-
- /* LO calibration setting */
- rtl8xxxu_write32(priv, REG_IQK_AGC_RSP, 0x00492911);
-
- /* One shot, path A LOK & IQK */
- rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xfa000000);
- rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf8000000);
-
- mdelay(1);
-
- /* Check failed */
- reg_eac = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_A_2);
- reg_eb4 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_B);
- reg_ebc = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_B);
-
- if (!(reg_eac & BIT(31)) &&
- ((reg_eb4 & 0x03ff0000) != 0x01420000) &&
- ((reg_ebc & 0x03ff0000) != 0x00420000))
- result |= 0x01;
- else
- dev_warn(&priv->udev->dev, "%s: Path B IQK failed!\n",
- __func__);
-
- return result;
-}
-
-static int rtl8192eu_rx_iqk_path_b(struct rtl8xxxu_priv *priv)
-{
- u32 reg_eac, reg_eb4, reg_ebc, reg_ec4, reg_ecc, val32;
- int result = 0;
-
- /* Leave IQK mode */
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
-
- /* Enable path A PA in TX IQK mode */
- rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_WE_LUT, 0x800a0);
- rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_RCK_OS, 0x30000);
- rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_TXPA_G1, 0x0000f);
- rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_TXPA_G2, 0xf117b);
-
- /* PA/PAD control by 0x56, and set = 0x0 */
- rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_UNKNOWN_DF, 0x00980);
- rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_UNKNOWN_56, 0x51000);
-
- /* Enter IQK mode */
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
-
- /* TX IQK setting */
- rtl8xxxu_write32(priv, REG_TX_IQK, 0x01007c00);
- rtl8xxxu_write32(priv, REG_RX_IQK, 0x01004800);
-
- /* path-A IQK setting */
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x38008c1c);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x38008c1c);
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_B, 0x18008c1c);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_B, 0x38008c1c);
-
- rtl8xxxu_write32(priv, REG_TX_IQK_PI_B, 0x82160c1f);
- rtl8xxxu_write32(priv, REG_RX_IQK_PI_B, 0x68160c1f);
-
- /* LO calibration setting */
- rtl8xxxu_write32(priv, REG_IQK_AGC_RSP, 0x0046a911);
-
- /* One shot, path A LOK & IQK */
- rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xfa000000);
- rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf8000000);
-
- mdelay(10);
-
- /* Check failed */
- reg_eac = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_A_2);
- reg_eb4 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_B);
- reg_ebc = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_B);
-
- if (!(reg_eac & BIT(31)) &&
- ((reg_eb4 & 0x03ff0000) != 0x01420000) &&
- ((reg_ebc & 0x03ff0000) != 0x00420000)) {
- result |= 0x01;
- } else {
- /*
- * PA/PAD controlled by 0x0
- * Vendor driver restores RF_A here which I believe is a bug
- */
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
- rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_UNKNOWN_DF, 0x180);
- goto out;
- }
-
- val32 = 0x80007c00 |
- (reg_eb4 & 0x03ff0000) | ((reg_ebc >> 16) & 0x03ff);
- rtl8xxxu_write32(priv, REG_TX_IQK, val32);
-
- /* Modify RX IQK mode table */
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
-
- rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_WE_LUT, 0x800a0);
- rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_RCK_OS, 0x30000);
- rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_TXPA_G1, 0x0000f);
- rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_TXPA_G2, 0xf7ffa);
-
- /* PA/PAD control by 0x56, and set = 0x0 */
- rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_UNKNOWN_DF, 0x00980);
- rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_UNKNOWN_56, 0x51000);
-
- /* Enter IQK mode */
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
-
- /* IQK setting */
- rtl8xxxu_write32(priv, REG_RX_IQK, 0x01004800);
-
- /* Path A IQK setting */
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x38008c1c);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x38008c1c);
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_B, 0x38008c1c);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_B, 0x18008c1c);
-
- rtl8xxxu_write32(priv, REG_TX_IQK_PI_A, 0x82160c1f);
- rtl8xxxu_write32(priv, REG_RX_IQK_PI_A, 0x28160c1f);
-
- /* LO calibration setting */
- rtl8xxxu_write32(priv, REG_IQK_AGC_RSP, 0x0046a891);
-
- /* One shot, path A LOK & IQK */
- rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xfa000000);
- rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf8000000);
-
- mdelay(10);
-
- reg_eac = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_A_2);
- reg_ec4 = rtl8xxxu_read32(priv, REG_RX_POWER_BEFORE_IQK_B_2);
- reg_ecc = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_B_2);
-
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
- rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_UNKNOWN_DF, 0x180);
-
- if (!(reg_eac & BIT(30)) &&
- ((reg_ec4 & 0x03ff0000) != 0x01320000) &&
- ((reg_ecc & 0x03ff0000) != 0x00360000))
- result |= 0x02;
- else
- dev_warn(&priv->udev->dev, "%s: Path B RX IQK failed!\n",
- __func__);
-
-out:
- return result;
-}
-
-static void rtl8xxxu_phy_iqcalibrate(struct rtl8xxxu_priv *priv,
- int result[][8], int t)
-{
- struct device *dev = &priv->udev->dev;
- u32 i, val32;
- int path_a_ok, path_b_ok;
- int retry = 2;
- const u32 adda_regs[RTL8XXXU_ADDA_REGS] = {
- REG_FPGA0_XCD_SWITCH_CTRL, REG_BLUETOOTH,
- REG_RX_WAIT_CCA, REG_TX_CCK_RFON,
- REG_TX_CCK_BBON, REG_TX_OFDM_RFON,
- REG_TX_OFDM_BBON, REG_TX_TO_RX,
- REG_TX_TO_TX, REG_RX_CCK,
- REG_RX_OFDM, REG_RX_WAIT_RIFS,
- REG_RX_TO_RX, REG_STANDBY,
- REG_SLEEP, REG_PMPD_ANAEN
- };
- const u32 iqk_mac_regs[RTL8XXXU_MAC_REGS] = {
- REG_TXPAUSE, REG_BEACON_CTRL,
- REG_BEACON_CTRL_1, REG_GPIO_MUXCFG
- };
- const u32 iqk_bb_regs[RTL8XXXU_BB_REGS] = {
- REG_OFDM0_TRX_PATH_ENABLE, REG_OFDM0_TR_MUX_PAR,
- REG_FPGA0_XCD_RF_SW_CTRL, REG_CONFIG_ANT_A, REG_CONFIG_ANT_B,
- REG_FPGA0_XAB_RF_SW_CTRL, REG_FPGA0_XA_RF_INT_OE,
- REG_FPGA0_XB_RF_INT_OE, REG_FPGA0_RF_MODE
- };
-
- /*
- * Note: IQ calibration must be performed after loading
- * PHY_REG.txt , and radio_a, radio_b.txt
- */
-
- if (t == 0) {
- /* Save ADDA parameters, turn Path A ADDA on */
- rtl8xxxu_save_regs(priv, adda_regs, priv->adda_backup,
- RTL8XXXU_ADDA_REGS);
- rtl8xxxu_save_mac_regs(priv, iqk_mac_regs, priv->mac_backup);
- rtl8xxxu_save_regs(priv, iqk_bb_regs,
- priv->bb_backup, RTL8XXXU_BB_REGS);
- }
-
- rtl8xxxu_path_adda_on(priv, adda_regs, true);
-
- if (t == 0) {
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_XA_HSSI_PARM1);
- if (val32 & FPGA0_HSSI_PARM1_PI)
- priv->pi_enabled = 1;
- }
-
- if (!priv->pi_enabled) {
- /* Switch BB to PI mode to do IQ Calibration. */
- rtl8xxxu_write32(priv, REG_FPGA0_XA_HSSI_PARM1, 0x01000100);
- rtl8xxxu_write32(priv, REG_FPGA0_XB_HSSI_PARM1, 0x01000100);
- }
-
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_RF_MODE);
- val32 &= ~FPGA_RF_MODE_CCK;
- rtl8xxxu_write32(priv, REG_FPGA0_RF_MODE, val32);
-
- rtl8xxxu_write32(priv, REG_OFDM0_TRX_PATH_ENABLE, 0x03a05600);
- rtl8xxxu_write32(priv, REG_OFDM0_TR_MUX_PAR, 0x000800e4);
- rtl8xxxu_write32(priv, REG_FPGA0_XCD_RF_SW_CTRL, 0x22204000);
-
- if (!priv->no_pape) {
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_XAB_RF_SW_CTRL);
- val32 |= (FPGA0_RF_PAPE |
- (FPGA0_RF_PAPE << FPGA0_RF_BD_CTRL_SHIFT));
- rtl8xxxu_write32(priv, REG_FPGA0_XAB_RF_SW_CTRL, val32);
- }
-
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_XA_RF_INT_OE);
- val32 &= ~BIT(10);
- rtl8xxxu_write32(priv, REG_FPGA0_XA_RF_INT_OE, val32);
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_XB_RF_INT_OE);
- val32 &= ~BIT(10);
- rtl8xxxu_write32(priv, REG_FPGA0_XB_RF_INT_OE, val32);
-
- if (priv->tx_paths > 1) {
- rtl8xxxu_write32(priv, REG_FPGA0_XA_LSSI_PARM, 0x00010000);
- rtl8xxxu_write32(priv, REG_FPGA0_XB_LSSI_PARM, 0x00010000);
- }
-
- /* MAC settings */
- rtl8xxxu_mac_calibration(priv, iqk_mac_regs, priv->mac_backup);
-
- /* Page B init */
- rtl8xxxu_write32(priv, REG_CONFIG_ANT_A, 0x00080000);
-
- if (priv->tx_paths > 1)
- rtl8xxxu_write32(priv, REG_CONFIG_ANT_B, 0x00080000);
-
- /* IQ calibration setting */
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
- rtl8xxxu_write32(priv, REG_TX_IQK, 0x01007c00);
- rtl8xxxu_write32(priv, REG_RX_IQK, 0x01004800);
-
- for (i = 0; i < retry; i++) {
- path_a_ok = rtl8xxxu_iqk_path_a(priv);
- if (path_a_ok == 0x03) {
- val32 = rtl8xxxu_read32(priv,
- REG_TX_POWER_BEFORE_IQK_A);
- result[t][0] = (val32 >> 16) & 0x3ff;
- val32 = rtl8xxxu_read32(priv,
- REG_TX_POWER_AFTER_IQK_A);
- result[t][1] = (val32 >> 16) & 0x3ff;
- val32 = rtl8xxxu_read32(priv,
- REG_RX_POWER_BEFORE_IQK_A_2);
- result[t][2] = (val32 >> 16) & 0x3ff;
- val32 = rtl8xxxu_read32(priv,
- REG_RX_POWER_AFTER_IQK_A_2);
- result[t][3] = (val32 >> 16) & 0x3ff;
- break;
- } else if (i == (retry - 1) && path_a_ok == 0x01) {
- /* TX IQK OK */
- dev_dbg(dev, "%s: Path A IQK Only Tx Success!!\n",
- __func__);
-
- val32 = rtl8xxxu_read32(priv,
- REG_TX_POWER_BEFORE_IQK_A);
- result[t][0] = (val32 >> 16) & 0x3ff;
- val32 = rtl8xxxu_read32(priv,
- REG_TX_POWER_AFTER_IQK_A);
- result[t][1] = (val32 >> 16) & 0x3ff;
- }
- }
-
- if (!path_a_ok)
- dev_dbg(dev, "%s: Path A IQK failed!\n", __func__);
-
- if (priv->tx_paths > 1) {
- /*
- * Path A into standby
- */
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x0);
- rtl8xxxu_write32(priv, REG_FPGA0_XA_LSSI_PARM, 0x00010000);
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
-
- /* Turn Path B ADDA on */
- rtl8xxxu_path_adda_on(priv, adda_regs, false);
-
- for (i = 0; i < retry; i++) {
- path_b_ok = rtl8xxxu_iqk_path_b(priv);
- if (path_b_ok == 0x03) {
- val32 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_B);
- result[t][4] = (val32 >> 16) & 0x3ff;
- val32 = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_B);
- result[t][5] = (val32 >> 16) & 0x3ff;
- val32 = rtl8xxxu_read32(priv, REG_RX_POWER_BEFORE_IQK_B_2);
- result[t][6] = (val32 >> 16) & 0x3ff;
- val32 = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_B_2);
- result[t][7] = (val32 >> 16) & 0x3ff;
- break;
- } else if (i == (retry - 1) && path_b_ok == 0x01) {
- /* TX IQK OK */
- val32 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_B);
- result[t][4] = (val32 >> 16) & 0x3ff;
- val32 = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_B);
- result[t][5] = (val32 >> 16) & 0x3ff;
- }
- }
-
- if (!path_b_ok)
- dev_dbg(dev, "%s: Path B IQK failed!\n", __func__);
- }
-
- /* Back to BB mode, load original value */
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0);
-
- if (t) {
- if (!priv->pi_enabled) {
- /*
- * Switch back BB to SI mode after finishing
- * IQ Calibration
- */
- val32 = 0x01000000;
- rtl8xxxu_write32(priv, REG_FPGA0_XA_HSSI_PARM1, val32);
- rtl8xxxu_write32(priv, REG_FPGA0_XB_HSSI_PARM1, val32);
- }
-
- /* Reload ADDA power saving parameters */
- rtl8xxxu_restore_regs(priv, adda_regs, priv->adda_backup,
- RTL8XXXU_ADDA_REGS);
-
- /* Reload MAC parameters */
- rtl8xxxu_restore_mac_regs(priv, iqk_mac_regs, priv->mac_backup);
-
- /* Reload BB parameters */
- rtl8xxxu_restore_regs(priv, iqk_bb_regs,
- priv->bb_backup, RTL8XXXU_BB_REGS);
-
- /* Restore RX initial gain */
- rtl8xxxu_write32(priv, REG_FPGA0_XA_LSSI_PARM, 0x00032ed3);
-
- if (priv->tx_paths > 1) {
- rtl8xxxu_write32(priv, REG_FPGA0_XB_LSSI_PARM,
- 0x00032ed3);
- }
-
- /* Load 0xe30 IQC default value */
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x01008c00);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x01008c00);
- }
-}
-
-static void rtl8723bu_phy_iqcalibrate(struct rtl8xxxu_priv *priv,
- int result[][8], int t)
-{
- struct device *dev = &priv->udev->dev;
- u32 i, val32;
- int path_a_ok /*, path_b_ok */;
- int retry = 2;
- const u32 adda_regs[RTL8XXXU_ADDA_REGS] = {
- REG_FPGA0_XCD_SWITCH_CTRL, REG_BLUETOOTH,
- REG_RX_WAIT_CCA, REG_TX_CCK_RFON,
- REG_TX_CCK_BBON, REG_TX_OFDM_RFON,
- REG_TX_OFDM_BBON, REG_TX_TO_RX,
- REG_TX_TO_TX, REG_RX_CCK,
- REG_RX_OFDM, REG_RX_WAIT_RIFS,
- REG_RX_TO_RX, REG_STANDBY,
- REG_SLEEP, REG_PMPD_ANAEN
- };
- const u32 iqk_mac_regs[RTL8XXXU_MAC_REGS] = {
- REG_TXPAUSE, REG_BEACON_CTRL,
- REG_BEACON_CTRL_1, REG_GPIO_MUXCFG
- };
- const u32 iqk_bb_regs[RTL8XXXU_BB_REGS] = {
- REG_OFDM0_TRX_PATH_ENABLE, REG_OFDM0_TR_MUX_PAR,
- REG_FPGA0_XCD_RF_SW_CTRL, REG_CONFIG_ANT_A, REG_CONFIG_ANT_B,
- REG_FPGA0_XAB_RF_SW_CTRL, REG_FPGA0_XA_RF_INT_OE,
- REG_FPGA0_XB_RF_INT_OE, REG_FPGA0_RF_MODE
- };
- u8 xa_agc = rtl8xxxu_read32(priv, REG_OFDM0_XA_AGC_CORE1) & 0xff;
- u8 xb_agc = rtl8xxxu_read32(priv, REG_OFDM0_XB_AGC_CORE1) & 0xff;
-
- /*
- * Note: IQ calibration must be performed after loading
- * PHY_REG.txt , and radio_a, radio_b.txt
- */
-
- if (t == 0) {
- /* Save ADDA parameters, turn Path A ADDA on */
- rtl8xxxu_save_regs(priv, adda_regs, priv->adda_backup,
- RTL8XXXU_ADDA_REGS);
- rtl8xxxu_save_mac_regs(priv, iqk_mac_regs, priv->mac_backup);
- rtl8xxxu_save_regs(priv, iqk_bb_regs,
- priv->bb_backup, RTL8XXXU_BB_REGS);
- }
-
- rtl8xxxu_path_adda_on(priv, adda_regs, true);
-
- /* MAC settings */
- rtl8xxxu_mac_calibration(priv, iqk_mac_regs, priv->mac_backup);
-
- val32 = rtl8xxxu_read32(priv, REG_CCK0_AFE_SETTING);
- val32 |= 0x0f000000;
- rtl8xxxu_write32(priv, REG_CCK0_AFE_SETTING, val32);
-
- rtl8xxxu_write32(priv, REG_OFDM0_TRX_PATH_ENABLE, 0x03a05600);
- rtl8xxxu_write32(priv, REG_OFDM0_TR_MUX_PAR, 0x000800e4);
- rtl8xxxu_write32(priv, REG_FPGA0_XCD_RF_SW_CTRL, 0x22204000);
-
- /*
- * RX IQ calibration setting for 8723B D cut large current issue
- * when leaving IPS
- */
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
- val32 &= 0x000000ff;
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
-
- val32 = rtl8xxxu_read_rfreg(priv, RF_A, RF6052_REG_WE_LUT);
- val32 |= 0x80000;
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_WE_LUT, val32);
-
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_RCK_OS, 0x30000);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G1, 0x0001f);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G2, 0xf7fb7);
-
- val32 = rtl8xxxu_read_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_ED);
- val32 |= 0x20;
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_ED, val32);
-
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_43, 0x60fbd);
-
- for (i = 0; i < retry; i++) {
- path_a_ok = rtl8723bu_iqk_path_a(priv);
- if (path_a_ok == 0x01) {
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
- val32 &= 0x000000ff;
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
-
- val32 = rtl8xxxu_read32(priv,
- REG_TX_POWER_BEFORE_IQK_A);
- result[t][0] = (val32 >> 16) & 0x3ff;
- val32 = rtl8xxxu_read32(priv,
- REG_TX_POWER_AFTER_IQK_A);
- result[t][1] = (val32 >> 16) & 0x3ff;
-
- break;
- }
- }
-
- if (!path_a_ok)
- dev_dbg(dev, "%s: Path A TX IQK failed!\n", __func__);
-
- for (i = 0; i < retry; i++) {
- path_a_ok = rtl8723bu_rx_iqk_path_a(priv);
- if (path_a_ok == 0x03) {
- val32 = rtl8xxxu_read32(priv,
- REG_RX_POWER_BEFORE_IQK_A_2);
- result[t][2] = (val32 >> 16) & 0x3ff;
- val32 = rtl8xxxu_read32(priv,
- REG_RX_POWER_AFTER_IQK_A_2);
- result[t][3] = (val32 >> 16) & 0x3ff;
-
- break;
- }
- }
-
- if (!path_a_ok)
- dev_dbg(dev, "%s: Path A RX IQK failed!\n", __func__);
-
- if (priv->tx_paths > 1) {
-#if 1
- dev_warn(dev, "%s: Path B not supported\n", __func__);
-#else
-
- /*
- * Path A into standby
- */
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
- val32 &= 0x000000ff;
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_AC, 0x10000);
-
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
- val32 &= 0x000000ff;
- val32 |= 0x80800000;
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
-
- /* Turn Path B ADDA on */
- rtl8xxxu_path_adda_on(priv, adda_regs, false);
-
- for (i = 0; i < retry; i++) {
- path_b_ok = rtl8xxxu_iqk_path_b(priv);
- if (path_b_ok == 0x03) {
- val32 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_B);
- result[t][4] = (val32 >> 16) & 0x3ff;
- val32 = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_B);
- result[t][5] = (val32 >> 16) & 0x3ff;
- break;
- }
- }
-
- if (!path_b_ok)
- dev_dbg(dev, "%s: Path B IQK failed!\n", __func__);
-
- for (i = 0; i < retry; i++) {
- path_b_ok = rtl8723bu_rx_iqk_path_b(priv);
- if (path_a_ok == 0x03) {
- val32 = rtl8xxxu_read32(priv,
- REG_RX_POWER_BEFORE_IQK_B_2);
- result[t][6] = (val32 >> 16) & 0x3ff;
- val32 = rtl8xxxu_read32(priv,
- REG_RX_POWER_AFTER_IQK_B_2);
- result[t][7] = (val32 >> 16) & 0x3ff;
- break;
- }
- }
-
- if (!path_b_ok)
- dev_dbg(dev, "%s: Path B RX IQK failed!\n", __func__);
-#endif
- }
-
- /* Back to BB mode, load original value */
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
- val32 &= 0x000000ff;
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
-
- if (t) {
- /* Reload ADDA power saving parameters */
- rtl8xxxu_restore_regs(priv, adda_regs, priv->adda_backup,
- RTL8XXXU_ADDA_REGS);
-
- /* Reload MAC parameters */
- rtl8xxxu_restore_mac_regs(priv, iqk_mac_regs, priv->mac_backup);
-
- /* Reload BB parameters */
- rtl8xxxu_restore_regs(priv, iqk_bb_regs,
- priv->bb_backup, RTL8XXXU_BB_REGS);
-
- /* Restore RX initial gain */
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_XA_AGC_CORE1);
- val32 &= 0xffffff00;
- rtl8xxxu_write32(priv, REG_OFDM0_XA_AGC_CORE1, val32 | 0x50);
- rtl8xxxu_write32(priv, REG_OFDM0_XA_AGC_CORE1, val32 | xa_agc);
-
- if (priv->tx_paths > 1) {
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_XB_AGC_CORE1);
- val32 &= 0xffffff00;
- rtl8xxxu_write32(priv, REG_OFDM0_XB_AGC_CORE1,
- val32 | 0x50);
- rtl8xxxu_write32(priv, REG_OFDM0_XB_AGC_CORE1,
- val32 | xb_agc);
- }
-
- /* Load 0xe30 IQC default value */
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x01008c00);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x01008c00);
- }
-}
-
-static void rtl8192eu_phy_iqcalibrate(struct rtl8xxxu_priv *priv,
- int result[][8], int t)
-{
- struct device *dev = &priv->udev->dev;
- u32 i, val32;
- int path_a_ok, path_b_ok;
- int retry = 2;
- const u32 adda_regs[RTL8XXXU_ADDA_REGS] = {
- REG_FPGA0_XCD_SWITCH_CTRL, REG_BLUETOOTH,
- REG_RX_WAIT_CCA, REG_TX_CCK_RFON,
- REG_TX_CCK_BBON, REG_TX_OFDM_RFON,
- REG_TX_OFDM_BBON, REG_TX_TO_RX,
- REG_TX_TO_TX, REG_RX_CCK,
- REG_RX_OFDM, REG_RX_WAIT_RIFS,
- REG_RX_TO_RX, REG_STANDBY,
- REG_SLEEP, REG_PMPD_ANAEN
- };
- const u32 iqk_mac_regs[RTL8XXXU_MAC_REGS] = {
- REG_TXPAUSE, REG_BEACON_CTRL,
- REG_BEACON_CTRL_1, REG_GPIO_MUXCFG
- };
- const u32 iqk_bb_regs[RTL8XXXU_BB_REGS] = {
- REG_OFDM0_TRX_PATH_ENABLE, REG_OFDM0_TR_MUX_PAR,
- REG_FPGA0_XCD_RF_SW_CTRL, REG_CONFIG_ANT_A, REG_CONFIG_ANT_B,
- REG_FPGA0_XAB_RF_SW_CTRL, REG_FPGA0_XA_RF_INT_OE,
- REG_FPGA0_XB_RF_INT_OE, REG_CCK0_AFE_SETTING
- };
- u8 xa_agc = rtl8xxxu_read32(priv, REG_OFDM0_XA_AGC_CORE1) & 0xff;
- u8 xb_agc = rtl8xxxu_read32(priv, REG_OFDM0_XB_AGC_CORE1) & 0xff;
-
- /*
- * Note: IQ calibration must be performed after loading
- * PHY_REG.txt , and radio_a, radio_b.txt
- */
-
- if (t == 0) {
- /* Save ADDA parameters, turn Path A ADDA on */
- rtl8xxxu_save_regs(priv, adda_regs, priv->adda_backup,
- RTL8XXXU_ADDA_REGS);
- rtl8xxxu_save_mac_regs(priv, iqk_mac_regs, priv->mac_backup);
- rtl8xxxu_save_regs(priv, iqk_bb_regs,
- priv->bb_backup, RTL8XXXU_BB_REGS);
- }
-
- rtl8xxxu_path_adda_on(priv, adda_regs, true);
-
- /* MAC settings */
- rtl8xxxu_mac_calibration(priv, iqk_mac_regs, priv->mac_backup);
-
- val32 = rtl8xxxu_read32(priv, REG_CCK0_AFE_SETTING);
- val32 |= 0x0f000000;
- rtl8xxxu_write32(priv, REG_CCK0_AFE_SETTING, val32);
-
- rtl8xxxu_write32(priv, REG_OFDM0_TRX_PATH_ENABLE, 0x03a05600);
- rtl8xxxu_write32(priv, REG_OFDM0_TR_MUX_PAR, 0x000800e4);
- rtl8xxxu_write32(priv, REG_FPGA0_XCD_RF_SW_CTRL, 0x22208200);
-
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_XAB_RF_SW_CTRL);
- val32 |= (FPGA0_RF_PAPE | (FPGA0_RF_PAPE << FPGA0_RF_BD_CTRL_SHIFT));
- rtl8xxxu_write32(priv, REG_FPGA0_XAB_RF_SW_CTRL, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_XA_RF_INT_OE);
- val32 |= BIT(10);
- rtl8xxxu_write32(priv, REG_FPGA0_XA_RF_INT_OE, val32);
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_XB_RF_INT_OE);
- val32 |= BIT(10);
- rtl8xxxu_write32(priv, REG_FPGA0_XB_RF_INT_OE, val32);
-
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
- rtl8xxxu_write32(priv, REG_TX_IQK, 0x01007c00);
- rtl8xxxu_write32(priv, REG_RX_IQK, 0x01004800);
-
- for (i = 0; i < retry; i++) {
- path_a_ok = rtl8192eu_iqk_path_a(priv);
- if (path_a_ok == 0x01) {
- val32 = rtl8xxxu_read32(priv,
- REG_TX_POWER_BEFORE_IQK_A);
- result[t][0] = (val32 >> 16) & 0x3ff;
- val32 = rtl8xxxu_read32(priv,
- REG_TX_POWER_AFTER_IQK_A);
- result[t][1] = (val32 >> 16) & 0x3ff;
-
- break;
- }
- }
-
- if (!path_a_ok)
- dev_dbg(dev, "%s: Path A TX IQK failed!\n", __func__);
-
- for (i = 0; i < retry; i++) {
- path_a_ok = rtl8192eu_rx_iqk_path_a(priv);
- if (path_a_ok == 0x03) {
- val32 = rtl8xxxu_read32(priv,
- REG_RX_POWER_BEFORE_IQK_A_2);
- result[t][2] = (val32 >> 16) & 0x3ff;
- val32 = rtl8xxxu_read32(priv,
- REG_RX_POWER_AFTER_IQK_A_2);
- result[t][3] = (val32 >> 16) & 0x3ff;
-
- break;
- }
- }
-
- if (!path_a_ok)
- dev_dbg(dev, "%s: Path A RX IQK failed!\n", __func__);
-
- if (priv->rf_paths > 1) {
- /* Path A into standby */
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_AC, 0x10000);
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
-
- /* Turn Path B ADDA on */
- rtl8xxxu_path_adda_on(priv, adda_regs, false);
-
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
- rtl8xxxu_write32(priv, REG_TX_IQK, 0x01007c00);
- rtl8xxxu_write32(priv, REG_RX_IQK, 0x01004800);
-
- for (i = 0; i < retry; i++) {
- path_b_ok = rtl8192eu_iqk_path_b(priv);
- if (path_b_ok == 0x01) {
- val32 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_B);
- result[t][4] = (val32 >> 16) & 0x3ff;
- val32 = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_B);
- result[t][5] = (val32 >> 16) & 0x3ff;
- break;
- }
- }
-
- if (!path_b_ok)
- dev_dbg(dev, "%s: Path B IQK failed!\n", __func__);
-
- for (i = 0; i < retry; i++) {
- path_b_ok = rtl8192eu_rx_iqk_path_b(priv);
- if (path_a_ok == 0x03) {
- val32 = rtl8xxxu_read32(priv,
- REG_RX_POWER_BEFORE_IQK_B_2);
- result[t][6] = (val32 >> 16) & 0x3ff;
- val32 = rtl8xxxu_read32(priv,
- REG_RX_POWER_AFTER_IQK_B_2);
- result[t][7] = (val32 >> 16) & 0x3ff;
- break;
- }
- }
-
- if (!path_b_ok)
- dev_dbg(dev, "%s: Path B RX IQK failed!\n", __func__);
- }
-
- /* Back to BB mode, load original value */
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
-
- if (t) {
- /* Reload ADDA power saving parameters */
- rtl8xxxu_restore_regs(priv, adda_regs, priv->adda_backup,
- RTL8XXXU_ADDA_REGS);
-
- /* Reload MAC parameters */
- rtl8xxxu_restore_mac_regs(priv, iqk_mac_regs, priv->mac_backup);
-
- /* Reload BB parameters */
- rtl8xxxu_restore_regs(priv, iqk_bb_regs,
- priv->bb_backup, RTL8XXXU_BB_REGS);
-
- /* Restore RX initial gain */
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_XA_AGC_CORE1);
- val32 &= 0xffffff00;
- rtl8xxxu_write32(priv, REG_OFDM0_XA_AGC_CORE1, val32 | 0x50);
- rtl8xxxu_write32(priv, REG_OFDM0_XA_AGC_CORE1, val32 | xa_agc);
-
- if (priv->rf_paths > 1) {
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_XB_AGC_CORE1);
- val32 &= 0xffffff00;
- rtl8xxxu_write32(priv, REG_OFDM0_XB_AGC_CORE1,
- val32 | 0x50);
- rtl8xxxu_write32(priv, REG_OFDM0_XB_AGC_CORE1,
- val32 | xb_agc);
- }
-
- /* Load 0xe30 IQC default value */
- rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x01008c00);
- rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x01008c00);
- }
-}
-
-static void rtl8xxxu_prepare_calibrate(struct rtl8xxxu_priv *priv, u8 start)
-{
- struct h2c_cmd h2c;
-
- if (priv->fops->mbox_ext_width < 4)
- return;
-
- memset(&h2c, 0, sizeof(struct h2c_cmd));
- h2c.bt_wlan_calibration.cmd = H2C_8723B_BT_WLAN_CALIBRATION;
- h2c.bt_wlan_calibration.data = start;
-
- rtl8723a_h2c_cmd(priv, &h2c, sizeof(h2c.bt_wlan_calibration));
-}
-
-static void rtl8xxxu_gen1_phy_iq_calibrate(struct rtl8xxxu_priv *priv)
-{
- struct device *dev = &priv->udev->dev;
- int result[4][8]; /* last is final result */
- int i, candidate;
- bool path_a_ok, path_b_ok;
- u32 reg_e94, reg_e9c, reg_ea4, reg_eac;
- u32 reg_eb4, reg_ebc, reg_ec4, reg_ecc;
- s32 reg_tmp = 0;
- bool simu;
-
- rtl8xxxu_prepare_calibrate(priv, 1);
-
- memset(result, 0, sizeof(result));
- candidate = -1;
-
- path_a_ok = false;
- path_b_ok = false;
-
- rtl8xxxu_read32(priv, REG_FPGA0_RF_MODE);
-
- for (i = 0; i < 3; i++) {
- rtl8xxxu_phy_iqcalibrate(priv, result, i);
-
- if (i == 1) {
- simu = rtl8xxxu_simularity_compare(priv, result, 0, 1);
- if (simu) {
- candidate = 0;
- break;
- }
- }
-
- if (i == 2) {
- simu = rtl8xxxu_simularity_compare(priv, result, 0, 2);
- if (simu) {
- candidate = 0;
- break;
- }
-
- simu = rtl8xxxu_simularity_compare(priv, result, 1, 2);
- if (simu) {
- candidate = 1;
- } else {
- for (i = 0; i < 8; i++)
- reg_tmp += result[3][i];
-
- if (reg_tmp)
- candidate = 3;
- else
- candidate = -1;
- }
- }
- }
-
- for (i = 0; i < 4; i++) {
- reg_e94 = result[i][0];
- reg_e9c = result[i][1];
- reg_ea4 = result[i][2];
- reg_eac = result[i][3];
- reg_eb4 = result[i][4];
- reg_ebc = result[i][5];
- reg_ec4 = result[i][6];
- reg_ecc = result[i][7];
- }
-
- if (candidate >= 0) {
- reg_e94 = result[candidate][0];
- priv->rege94 = reg_e94;
- reg_e9c = result[candidate][1];
- priv->rege9c = reg_e9c;
- reg_ea4 = result[candidate][2];
- reg_eac = result[candidate][3];
- reg_eb4 = result[candidate][4];
- priv->regeb4 = reg_eb4;
- reg_ebc = result[candidate][5];
- priv->regebc = reg_ebc;
- reg_ec4 = result[candidate][6];
- reg_ecc = result[candidate][7];
- dev_dbg(dev, "%s: candidate is %x\n", __func__, candidate);
- dev_dbg(dev,
- "%s: e94 =%x e9c=%x ea4=%x eac=%x eb4=%x ebc=%x ec4=%x "
- "ecc=%x\n ", __func__, reg_e94, reg_e9c,
- reg_ea4, reg_eac, reg_eb4, reg_ebc, reg_ec4, reg_ecc);
- path_a_ok = true;
- path_b_ok = true;
- } else {
- reg_e94 = reg_eb4 = priv->rege94 = priv->regeb4 = 0x100;
- reg_e9c = reg_ebc = priv->rege9c = priv->regebc = 0x0;
- }
-
- if (reg_e94 && candidate >= 0)
- rtl8xxxu_fill_iqk_matrix_a(priv, path_a_ok, result,
- candidate, (reg_ea4 == 0));
-
- if (priv->tx_paths > 1 && reg_eb4)
- rtl8xxxu_fill_iqk_matrix_b(priv, path_b_ok, result,
- candidate, (reg_ec4 == 0));
-
- rtl8xxxu_save_regs(priv, rtl8xxxu_iqk_phy_iq_bb_reg,
- priv->bb_recovery_backup, RTL8XXXU_BB_REGS);
-
- rtl8xxxu_prepare_calibrate(priv, 0);
-}
-
-static void rtl8723bu_phy_iq_calibrate(struct rtl8xxxu_priv *priv)
-{
- struct device *dev = &priv->udev->dev;
- int result[4][8]; /* last is final result */
- int i, candidate;
- bool path_a_ok, path_b_ok;
- u32 reg_e94, reg_e9c, reg_ea4, reg_eac;
- u32 reg_eb4, reg_ebc, reg_ec4, reg_ecc;
- u32 val32, bt_control;
- s32 reg_tmp = 0;
- bool simu;
-
- rtl8xxxu_prepare_calibrate(priv, 1);
-
- memset(result, 0, sizeof(result));
- candidate = -1;
-
- path_a_ok = false;
- path_b_ok = false;
-
- bt_control = rtl8xxxu_read32(priv, REG_BT_CONTROL_8723BU);
-
- for (i = 0; i < 3; i++) {
- rtl8723bu_phy_iqcalibrate(priv, result, i);
-
- if (i == 1) {
- simu = rtl8xxxu_gen2_simularity_compare(priv,
- result, 0, 1);
- if (simu) {
- candidate = 0;
- break;
- }
- }
-
- if (i == 2) {
- simu = rtl8xxxu_gen2_simularity_compare(priv,
- result, 0, 2);
- if (simu) {
- candidate = 0;
- break;
- }
-
- simu = rtl8xxxu_gen2_simularity_compare(priv,
- result, 1, 2);
- if (simu) {
- candidate = 1;
- } else {
- for (i = 0; i < 8; i++)
- reg_tmp += result[3][i];
-
- if (reg_tmp)
- candidate = 3;
- else
- candidate = -1;
- }
- }
- }
-
- for (i = 0; i < 4; i++) {
- reg_e94 = result[i][0];
- reg_e9c = result[i][1];
- reg_ea4 = result[i][2];
- reg_eac = result[i][3];
- reg_eb4 = result[i][4];
- reg_ebc = result[i][5];
- reg_ec4 = result[i][6];
- reg_ecc = result[i][7];
- }
-
- if (candidate >= 0) {
- reg_e94 = result[candidate][0];
- priv->rege94 = reg_e94;
- reg_e9c = result[candidate][1];
- priv->rege9c = reg_e9c;
- reg_ea4 = result[candidate][2];
- reg_eac = result[candidate][3];
- reg_eb4 = result[candidate][4];
- priv->regeb4 = reg_eb4;
- reg_ebc = result[candidate][5];
- priv->regebc = reg_ebc;
- reg_ec4 = result[candidate][6];
- reg_ecc = result[candidate][7];
- dev_dbg(dev, "%s: candidate is %x\n", __func__, candidate);
- dev_dbg(dev,
- "%s: e94 =%x e9c=%x ea4=%x eac=%x eb4=%x ebc=%x ec4=%x "
- "ecc=%x\n ", __func__, reg_e94, reg_e9c,
- reg_ea4, reg_eac, reg_eb4, reg_ebc, reg_ec4, reg_ecc);
- path_a_ok = true;
- path_b_ok = true;
- } else {
- reg_e94 = reg_eb4 = priv->rege94 = priv->regeb4 = 0x100;
- reg_e9c = reg_ebc = priv->rege9c = priv->regebc = 0x0;
- }
-
- if (reg_e94 && candidate >= 0)
- rtl8xxxu_fill_iqk_matrix_a(priv, path_a_ok, result,
- candidate, (reg_ea4 == 0));
-
- if (priv->tx_paths > 1 && reg_eb4)
- rtl8xxxu_fill_iqk_matrix_b(priv, path_b_ok, result,
- candidate, (reg_ec4 == 0));
-
- rtl8xxxu_save_regs(priv, rtl8xxxu_iqk_phy_iq_bb_reg,
- priv->bb_recovery_backup, RTL8XXXU_BB_REGS);
-
- rtl8xxxu_write32(priv, REG_BT_CONTROL_8723BU, bt_control);
-
- val32 = rtl8xxxu_read_rfreg(priv, RF_A, RF6052_REG_WE_LUT);
- val32 |= 0x80000;
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_WE_LUT, val32);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_RCK_OS, 0x18000);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G1, 0x0001f);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G2, 0xe6177);
- val32 = rtl8xxxu_read_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_ED);
- val32 |= 0x20;
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_ED, val32);
- rtl8xxxu_write_rfreg(priv, RF_A, 0x43, 0x300bd);
-
- if (priv->rf_paths > 1)
- dev_dbg(dev, "%s: 8723BU 2T not supported\n", __func__);
-
- rtl8xxxu_prepare_calibrate(priv, 0);
-}
-
-static void rtl8192eu_phy_iq_calibrate(struct rtl8xxxu_priv *priv)
-{
- struct device *dev = &priv->udev->dev;
- int result[4][8]; /* last is final result */
- int i, candidate;
- bool path_a_ok, path_b_ok;
- u32 reg_e94, reg_e9c, reg_ea4, reg_eac;
- u32 reg_eb4, reg_ebc, reg_ec4, reg_ecc;
- bool simu;
-
- memset(result, 0, sizeof(result));
- candidate = -1;
-
- path_a_ok = false;
- path_b_ok = false;
-
- for (i = 0; i < 3; i++) {
- rtl8192eu_phy_iqcalibrate(priv, result, i);
-
- if (i == 1) {
- simu = rtl8xxxu_gen2_simularity_compare(priv,
- result, 0, 1);
- if (simu) {
- candidate = 0;
- break;
- }
- }
-
- if (i == 2) {
- simu = rtl8xxxu_gen2_simularity_compare(priv,
- result, 0, 2);
- if (simu) {
- candidate = 0;
- break;
- }
-
- simu = rtl8xxxu_gen2_simularity_compare(priv,
- result, 1, 2);
- if (simu)
- candidate = 1;
- else
- candidate = 3;
- }
- }
-
- for (i = 0; i < 4; i++) {
- reg_e94 = result[i][0];
- reg_e9c = result[i][1];
- reg_ea4 = result[i][2];
- reg_eac = result[i][3];
- reg_eb4 = result[i][4];
- reg_ebc = result[i][5];
- reg_ec4 = result[i][6];
- reg_ecc = result[i][7];
- }
-
- if (candidate >= 0) {
- reg_e94 = result[candidate][0];
- priv->rege94 = reg_e94;
- reg_e9c = result[candidate][1];
- priv->rege9c = reg_e9c;
- reg_ea4 = result[candidate][2];
- reg_eac = result[candidate][3];
- reg_eb4 = result[candidate][4];
- priv->regeb4 = reg_eb4;
- reg_ebc = result[candidate][5];
- priv->regebc = reg_ebc;
- reg_ec4 = result[candidate][6];
- reg_ecc = result[candidate][7];
- dev_dbg(dev, "%s: candidate is %x\n", __func__, candidate);
- dev_dbg(dev,
- "%s: e94 =%x e9c=%x ea4=%x eac=%x eb4=%x ebc=%x ec4=%x "
- "ecc=%x\n ", __func__, reg_e94, reg_e9c,
- reg_ea4, reg_eac, reg_eb4, reg_ebc, reg_ec4, reg_ecc);
- path_a_ok = true;
- path_b_ok = true;
- } else {
- reg_e94 = reg_eb4 = priv->rege94 = priv->regeb4 = 0x100;
- reg_e9c = reg_ebc = priv->rege9c = priv->regebc = 0x0;
- }
-
- if (reg_e94 && candidate >= 0)
- rtl8xxxu_fill_iqk_matrix_a(priv, path_a_ok, result,
- candidate, (reg_ea4 == 0));
-
- if (priv->rf_paths > 1)
- rtl8xxxu_fill_iqk_matrix_b(priv, path_b_ok, result,
- candidate, (reg_ec4 == 0));
-
- rtl8xxxu_save_regs(priv, rtl8xxxu_iqk_phy_iq_bb_reg,
- priv->bb_recovery_backup, RTL8XXXU_BB_REGS);
-}
-
-static void rtl8723a_phy_lc_calibrate(struct rtl8xxxu_priv *priv)
-{
- u32 val32;
- u32 rf_amode, rf_bmode = 0, lstf;
-
- /* Check continuous TX and Packet TX */
- lstf = rtl8xxxu_read32(priv, REG_OFDM1_LSTF);
-
- if (lstf & OFDM_LSTF_MASK) {
- /* Disable all continuous TX */
- val32 = lstf & ~OFDM_LSTF_MASK;
- rtl8xxxu_write32(priv, REG_OFDM1_LSTF, val32);
-
- /* Read original RF mode Path A */
- rf_amode = rtl8xxxu_read_rfreg(priv, RF_A, RF6052_REG_AC);
-
- /* Set RF mode to standby Path A */
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_AC,
- (rf_amode & 0x8ffff) | 0x10000);
-
- /* Path-B */
- if (priv->tx_paths > 1) {
- rf_bmode = rtl8xxxu_read_rfreg(priv, RF_B,
- RF6052_REG_AC);
-
- rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_AC,
- (rf_bmode & 0x8ffff) | 0x10000);
- }
- } else {
- /* Deal with Packet TX case */
- /* block all queues */
- rtl8xxxu_write8(priv, REG_TXPAUSE, 0xff);
- }
-
- /* Start LC calibration */
- if (priv->fops->has_s0s1)
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_S0S1, 0xdfbe0);
- val32 = rtl8xxxu_read_rfreg(priv, RF_A, RF6052_REG_MODE_AG);
- val32 |= 0x08000;
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_MODE_AG, val32);
-
- msleep(100);
-
- if (priv->fops->has_s0s1)
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_S0S1, 0xdffe0);
-
- /* Restore original parameters */
- if (lstf & OFDM_LSTF_MASK) {
- /* Path-A */
- rtl8xxxu_write32(priv, REG_OFDM1_LSTF, lstf);
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_AC, rf_amode);
-
- /* Path-B */
- if (priv->tx_paths > 1)
- rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_AC,
- rf_bmode);
- } else /* Deal with Packet TX case */
- rtl8xxxu_write8(priv, REG_TXPAUSE, 0x00);
-}
-
-static int rtl8xxxu_set_mac(struct rtl8xxxu_priv *priv)
-{
- int i;
- u16 reg;
-
- reg = REG_MACID;
-
- for (i = 0; i < ETH_ALEN; i++)
- rtl8xxxu_write8(priv, reg + i, priv->mac_addr[i]);
-
- return 0;
-}
-
-static int rtl8xxxu_set_bssid(struct rtl8xxxu_priv *priv, const u8 *bssid)
-{
- int i;
- u16 reg;
-
- dev_dbg(&priv->udev->dev, "%s: (%pM)\n", __func__, bssid);
-
- reg = REG_BSSID;
-
- for (i = 0; i < ETH_ALEN; i++)
- rtl8xxxu_write8(priv, reg + i, bssid[i]);
-
- return 0;
-}
-
-static void
-rtl8xxxu_set_ampdu_factor(struct rtl8xxxu_priv *priv, u8 ampdu_factor)
-{
- u8 vals[4] = { 0x41, 0xa8, 0x72, 0xb9 };
- u8 max_agg = 0xf;
- int i;
-
- ampdu_factor = 1 << (ampdu_factor + 2);
- if (ampdu_factor > max_agg)
- ampdu_factor = max_agg;
-
- for (i = 0; i < 4; i++) {
- if ((vals[i] & 0xf0) > (ampdu_factor << 4))
- vals[i] = (vals[i] & 0x0f) | (ampdu_factor << 4);
-
- if ((vals[i] & 0x0f) > ampdu_factor)
- vals[i] = (vals[i] & 0xf0) | ampdu_factor;
-
- rtl8xxxu_write8(priv, REG_AGGLEN_LMT + i, vals[i]);
- }
-}
-
-static void rtl8xxxu_set_ampdu_min_space(struct rtl8xxxu_priv *priv, u8 density)
-{
- u8 val8;
-
- val8 = rtl8xxxu_read8(priv, REG_AMPDU_MIN_SPACE);
- val8 &= 0xf8;
- val8 |= density;
- rtl8xxxu_write8(priv, REG_AMPDU_MIN_SPACE, val8);
-}
-
-static int rtl8xxxu_active_to_emu(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
- int count, ret = 0;
-
- /* Start of rtl8723AU_card_enable_flow */
- /* Act to Cardemu sequence*/
- /* Turn off RF */
- rtl8xxxu_write8(priv, REG_RF_CTRL, 0);
-
- /* 0x004E[7] = 0, switch DPDT_SEL_P output from register 0x0065[2] */
- val8 = rtl8xxxu_read8(priv, REG_LEDCFG2);
- val8 &= ~LEDCFG2_DPDT_SELECT;
- rtl8xxxu_write8(priv, REG_LEDCFG2, val8);
-
- /* 0x0005[1] = 1 turn off MAC by HW state machine*/
- val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
- val8 |= BIT(1);
- rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
-
- for (count = RTL8XXXU_MAX_REG_POLL; count; count--) {
- val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
- if ((val8 & BIT(1)) == 0)
- break;
- udelay(10);
- }
-
- if (!count) {
- dev_warn(&priv->udev->dev, "%s: Disabling MAC timed out\n",
- __func__);
- ret = -EBUSY;
- goto exit;
- }
-
- /* 0x0000[5] = 1 analog Ips to digital, 1:isolation */
- val8 = rtl8xxxu_read8(priv, REG_SYS_ISO_CTRL);
- val8 |= SYS_ISO_ANALOG_IPS;
- rtl8xxxu_write8(priv, REG_SYS_ISO_CTRL, val8);
-
- /* 0x0020[0] = 0 disable LDOA12 MACRO block*/
- val8 = rtl8xxxu_read8(priv, REG_LDOA15_CTRL);
- val8 &= ~LDOA15_ENABLE;
- rtl8xxxu_write8(priv, REG_LDOA15_CTRL, val8);
-
-exit:
- return ret;
-}
-
-static int rtl8723bu_active_to_emu(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
- u16 val16;
- u32 val32;
- int count, ret = 0;
-
- /* Turn off RF */
- rtl8xxxu_write8(priv, REG_RF_CTRL, 0);
-
- /* Enable rising edge triggering interrupt */
- val16 = rtl8xxxu_read16(priv, REG_GPIO_INTM);
- val16 &= ~GPIO_INTM_EDGE_TRIG_IRQ;
- rtl8xxxu_write16(priv, REG_GPIO_INTM, val16);
-
- /* Release WLON reset 0x04[16]= 1*/
- val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
- val32 |= APS_FSMCO_WLON_RESET;
- rtl8xxxu_write32(priv, REG_APS_FSMCO, val32);
-
- /* 0x0005[1] = 1 turn off MAC by HW state machine*/
- val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
- val8 |= BIT(1);
- rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
-
- for (count = RTL8XXXU_MAX_REG_POLL; count; count--) {
- val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
- if ((val8 & BIT(1)) == 0)
- break;
- udelay(10);
- }
-
- if (!count) {
- dev_warn(&priv->udev->dev, "%s: Disabling MAC timed out\n",
- __func__);
- ret = -EBUSY;
- goto exit;
- }
-
- /* Enable BT control XTAL setting */
- val8 = rtl8xxxu_read8(priv, REG_AFE_MISC);
- val8 &= ~AFE_MISC_WL_XTAL_CTRL;
- rtl8xxxu_write8(priv, REG_AFE_MISC, val8);
-
- /* 0x0000[5] = 1 analog Ips to digital, 1:isolation */
- val8 = rtl8xxxu_read8(priv, REG_SYS_ISO_CTRL);
- val8 |= SYS_ISO_ANALOG_IPS;
- rtl8xxxu_write8(priv, REG_SYS_ISO_CTRL, val8);
-
- /* 0x0020[0] = 0 disable LDOA12 MACRO block*/
- val8 = rtl8xxxu_read8(priv, REG_LDOA15_CTRL);
- val8 &= ~LDOA15_ENABLE;
- rtl8xxxu_write8(priv, REG_LDOA15_CTRL, val8);
-
-exit:
- return ret;
-}
-
-static int rtl8xxxu_active_to_lps(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
- u8 val32;
- int count, ret = 0;
-
- rtl8xxxu_write8(priv, REG_TXPAUSE, 0xff);
-
- /*
- * Poll - wait for RX packet to complete
- */
- for (count = RTL8XXXU_MAX_REG_POLL; count; count--) {
- val32 = rtl8xxxu_read32(priv, 0x5f8);
- if (!val32)
- break;
- udelay(10);
- }
-
- if (!count) {
- dev_warn(&priv->udev->dev,
- "%s: RX poll timed out (0x05f8)\n", __func__);
- ret = -EBUSY;
- goto exit;
- }
-
- /* Disable CCK and OFDM, clock gated */
- val8 = rtl8xxxu_read8(priv, REG_SYS_FUNC);
- val8 &= ~SYS_FUNC_BBRSTB;
- rtl8xxxu_write8(priv, REG_SYS_FUNC, val8);
-
- udelay(2);
-
- /* Reset baseband */
- val8 = rtl8xxxu_read8(priv, REG_SYS_FUNC);
- val8 &= ~SYS_FUNC_BB_GLB_RSTN;
- rtl8xxxu_write8(priv, REG_SYS_FUNC, val8);
-
- /* Reset MAC TRX */
- val8 = rtl8xxxu_read8(priv, REG_CR);
- val8 = CR_HCI_TXDMA_ENABLE | CR_HCI_RXDMA_ENABLE;
- rtl8xxxu_write8(priv, REG_CR, val8);
-
- /* Reset MAC TRX */
- val8 = rtl8xxxu_read8(priv, REG_CR + 1);
- val8 &= ~BIT(1); /* CR_SECURITY_ENABLE */
- rtl8xxxu_write8(priv, REG_CR + 1, val8);
-
- /* Respond TX OK to scheduler */
- val8 = rtl8xxxu_read8(priv, REG_DUAL_TSF_RST);
- val8 |= DUAL_TSF_TX_OK;
- rtl8xxxu_write8(priv, REG_DUAL_TSF_RST, val8);
-
-exit:
- return ret;
-}
-
-static void rtl8723a_disabled_to_emu(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
-
- /* Clear suspend enable and power down enable*/
- val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
- val8 &= ~(BIT(3) | BIT(7));
- rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
-
- /* 0x48[16] = 0 to disable GPIO9 as EXT WAKEUP*/
- val8 = rtl8xxxu_read8(priv, REG_GPIO_INTM + 2);
- val8 &= ~BIT(0);
- rtl8xxxu_write8(priv, REG_GPIO_INTM + 2, val8);
-
- /* 0x04[12:11] = 11 enable WL suspend*/
- val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
- val8 &= ~(BIT(3) | BIT(4));
- rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
-}
-
-static void rtl8192e_disabled_to_emu(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
-
- /* Clear suspend enable and power down enable*/
- val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
- val8 &= ~(BIT(3) | BIT(4));
- rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
-}
-
-static int rtl8192e_emu_to_active(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
- u32 val32;
- int count, ret = 0;
-
- /* disable HWPDN 0x04[15]=0*/
- val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
- val8 &= ~BIT(7);
- rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
-
- /* disable SW LPS 0x04[10]= 0 */
- val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
- val8 &= ~BIT(2);
- rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
-
- /* disable WL suspend*/
- val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
- val8 &= ~(BIT(3) | BIT(4));
- rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
-
- /* wait till 0x04[17] = 1 power ready*/
- for (count = RTL8XXXU_MAX_REG_POLL; count; count--) {
- val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
- if (val32 & BIT(17))
- break;
-
- udelay(10);
- }
-
- if (!count) {
- ret = -EBUSY;
- goto exit;
- }
-
- /* We should be able to optimize the following three entries into one */
-
- /* release WLON reset 0x04[16]= 1*/
- val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 2);
- val8 |= BIT(0);
- rtl8xxxu_write8(priv, REG_APS_FSMCO + 2, val8);
-
- /* set, then poll until 0 */
- val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
- val32 |= APS_FSMCO_MAC_ENABLE;
- rtl8xxxu_write32(priv, REG_APS_FSMCO, val32);
-
- for (count = RTL8XXXU_MAX_REG_POLL; count; count--) {
- val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
- if ((val32 & APS_FSMCO_MAC_ENABLE) == 0) {
- ret = 0;
- break;
- }
- udelay(10);
- }
-
- if (!count) {
- ret = -EBUSY;
- goto exit;
- }
-
-exit:
- return ret;
-}
-
-static int rtl8723a_emu_to_active(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
- u32 val32;
- int count, ret = 0;
-
- /* 0x20[0] = 1 enable LDOA12 MACRO block for all interface*/
- val8 = rtl8xxxu_read8(priv, REG_LDOA15_CTRL);
- val8 |= LDOA15_ENABLE;
- rtl8xxxu_write8(priv, REG_LDOA15_CTRL, val8);
-
- /* 0x67[0] = 0 to disable BT_GPS_SEL pins*/
- val8 = rtl8xxxu_read8(priv, 0x0067);
- val8 &= ~BIT(4);
- rtl8xxxu_write8(priv, 0x0067, val8);
-
- mdelay(1);
-
- /* 0x00[5] = 0 release analog Ips to digital, 1:isolation */
- val8 = rtl8xxxu_read8(priv, REG_SYS_ISO_CTRL);
- val8 &= ~SYS_ISO_ANALOG_IPS;
- rtl8xxxu_write8(priv, REG_SYS_ISO_CTRL, val8);
-
- /* disable SW LPS 0x04[10]= 0 */
- val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
- val8 &= ~BIT(2);
- rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
-
- /* wait till 0x04[17] = 1 power ready*/
- for (count = RTL8XXXU_MAX_REG_POLL; count; count--) {
- val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
- if (val32 & BIT(17))
- break;
-
- udelay(10);
- }
-
- if (!count) {
- ret = -EBUSY;
- goto exit;
- }
-
- /* We should be able to optimize the following three entries into one */
-
- /* release WLON reset 0x04[16]= 1*/
- val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 2);
- val8 |= BIT(0);
- rtl8xxxu_write8(priv, REG_APS_FSMCO + 2, val8);
-
- /* disable HWPDN 0x04[15]= 0*/
- val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
- val8 &= ~BIT(7);
- rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
-
- /* disable WL suspend*/
- val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
- val8 &= ~(BIT(3) | BIT(4));
- rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
-
- /* set, then poll until 0 */
- val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
- val32 |= APS_FSMCO_MAC_ENABLE;
- rtl8xxxu_write32(priv, REG_APS_FSMCO, val32);
-
- for (count = RTL8XXXU_MAX_REG_POLL; count; count--) {
- val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
- if ((val32 & APS_FSMCO_MAC_ENABLE) == 0) {
- ret = 0;
- break;
- }
- udelay(10);
- }
-
- if (!count) {
- ret = -EBUSY;
- goto exit;
- }
-
- /* 0x4C[23] = 0x4E[7] = 1, switch DPDT_SEL_P output from WL BB */
- /*
- * Note: Vendor driver actually clears this bit, despite the
- * documentation claims it's being set!
- */
- val8 = rtl8xxxu_read8(priv, REG_LEDCFG2);
- val8 |= LEDCFG2_DPDT_SELECT;
- val8 &= ~LEDCFG2_DPDT_SELECT;
- rtl8xxxu_write8(priv, REG_LEDCFG2, val8);
-
-exit:
- return ret;
-}
-
-static int rtl8723b_emu_to_active(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
- u32 val32;
- int count, ret = 0;
-
- /* 0x20[0] = 1 enable LDOA12 MACRO block for all interface */
- val8 = rtl8xxxu_read8(priv, REG_LDOA15_CTRL);
- val8 |= LDOA15_ENABLE;
- rtl8xxxu_write8(priv, REG_LDOA15_CTRL, val8);
-
- /* 0x67[0] = 0 to disable BT_GPS_SEL pins*/
- val8 = rtl8xxxu_read8(priv, 0x0067);
- val8 &= ~BIT(4);
- rtl8xxxu_write8(priv, 0x0067, val8);
-
- mdelay(1);
-
- /* 0x00[5] = 0 release analog Ips to digital, 1:isolation */
- val8 = rtl8xxxu_read8(priv, REG_SYS_ISO_CTRL);
- val8 &= ~SYS_ISO_ANALOG_IPS;
- rtl8xxxu_write8(priv, REG_SYS_ISO_CTRL, val8);
-
- /* Disable SW LPS 0x04[10]= 0 */
- val32 = rtl8xxxu_read8(priv, REG_APS_FSMCO);
- val32 &= ~APS_FSMCO_SW_LPS;
- rtl8xxxu_write32(priv, REG_APS_FSMCO, val32);
-
- /* Wait until 0x04[17] = 1 power ready */
- for (count = RTL8XXXU_MAX_REG_POLL; count; count--) {
- val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
- if (val32 & BIT(17))
- break;
-
- udelay(10);
- }
-
- if (!count) {
- ret = -EBUSY;
- goto exit;
- }
-
- /* We should be able to optimize the following three entries into one */
-
- /* Release WLON reset 0x04[16]= 1*/
- val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
- val32 |= APS_FSMCO_WLON_RESET;
- rtl8xxxu_write32(priv, REG_APS_FSMCO, val32);
-
- /* Disable HWPDN 0x04[15]= 0*/
- val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
- val32 &= ~APS_FSMCO_HW_POWERDOWN;
- rtl8xxxu_write32(priv, REG_APS_FSMCO, val32);
-
- /* Disable WL suspend*/
- val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
- val32 &= ~(APS_FSMCO_HW_SUSPEND | APS_FSMCO_PCIE);
- rtl8xxxu_write32(priv, REG_APS_FSMCO, val32);
-
- /* Set, then poll until 0 */
- val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
- val32 |= APS_FSMCO_MAC_ENABLE;
- rtl8xxxu_write32(priv, REG_APS_FSMCO, val32);
-
- for (count = RTL8XXXU_MAX_REG_POLL; count; count--) {
- val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
- if ((val32 & APS_FSMCO_MAC_ENABLE) == 0) {
- ret = 0;
- break;
- }
- udelay(10);
- }
-
- if (!count) {
- ret = -EBUSY;
- goto exit;
- }
-
- /* Enable WL control XTAL setting */
- val8 = rtl8xxxu_read8(priv, REG_AFE_MISC);
- val8 |= AFE_MISC_WL_XTAL_CTRL;
- rtl8xxxu_write8(priv, REG_AFE_MISC, val8);
-
- /* Enable falling edge triggering interrupt */
- val8 = rtl8xxxu_read8(priv, REG_GPIO_INTM + 1);
- val8 |= BIT(1);
- rtl8xxxu_write8(priv, REG_GPIO_INTM + 1, val8);
-
- /* Enable GPIO9 interrupt mode */
- val8 = rtl8xxxu_read8(priv, REG_GPIO_IO_SEL_2 + 1);
- val8 |= BIT(1);
- rtl8xxxu_write8(priv, REG_GPIO_IO_SEL_2 + 1, val8);
-
- /* Enable GPIO9 input mode */
- val8 = rtl8xxxu_read8(priv, REG_GPIO_IO_SEL_2);
- val8 &= ~BIT(1);
- rtl8xxxu_write8(priv, REG_GPIO_IO_SEL_2, val8);
-
- /* Enable HSISR GPIO[C:0] interrupt */
- val8 = rtl8xxxu_read8(priv, REG_HSIMR);
- val8 |= BIT(0);
- rtl8xxxu_write8(priv, REG_HSIMR, val8);
-
- /* Enable HSISR GPIO9 interrupt */
- val8 = rtl8xxxu_read8(priv, REG_HSIMR + 2);
- val8 |= BIT(1);
- rtl8xxxu_write8(priv, REG_HSIMR + 2, val8);
-
- val8 = rtl8xxxu_read8(priv, REG_MULTI_FUNC_CTRL);
- val8 |= MULTI_WIFI_HW_ROF_EN;
- rtl8xxxu_write8(priv, REG_MULTI_FUNC_CTRL, val8);
-
- /* For GPIO9 internal pull high setting BIT(14) */
- val8 = rtl8xxxu_read8(priv, REG_MULTI_FUNC_CTRL + 1);
- val8 |= BIT(6);
- rtl8xxxu_write8(priv, REG_MULTI_FUNC_CTRL + 1, val8);
-
-exit:
- return ret;
-}
-
-static int rtl8xxxu_emu_to_disabled(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
-
- /* 0x0007[7:0] = 0x20 SOP option to disable BG/MB */
- rtl8xxxu_write8(priv, REG_APS_FSMCO + 3, 0x20);
-
- /* 0x04[12:11] = 01 enable WL suspend */
- val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
- val8 &= ~BIT(4);
- val8 |= BIT(3);
- rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
-
- val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
- val8 |= BIT(7);
- rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
-
- /* 0x48[16] = 1 to enable GPIO9 as EXT wakeup */
- val8 = rtl8xxxu_read8(priv, REG_GPIO_INTM + 2);
- val8 |= BIT(0);
- rtl8xxxu_write8(priv, REG_GPIO_INTM + 2, val8);
-
- return 0;
-}
-
-static int rtl8xxxu_flush_fifo(struct rtl8xxxu_priv *priv)
-{
- struct device *dev = &priv->udev->dev;
- u32 val32;
- int retry, retval;
-
- rtl8xxxu_write8(priv, REG_TXPAUSE, 0xff);
-
- val32 = rtl8xxxu_read32(priv, REG_RXPKT_NUM);
- val32 |= RXPKT_NUM_RW_RELEASE_EN;
- rtl8xxxu_write32(priv, REG_RXPKT_NUM, val32);
-
- retry = 100;
- retval = -EBUSY;
-
- do {
- val32 = rtl8xxxu_read32(priv, REG_RXPKT_NUM);
- if (val32 & RXPKT_NUM_RXDMA_IDLE) {
- retval = 0;
- break;
- }
- } while (retry--);
-
- rtl8xxxu_write16(priv, REG_RQPN_NPQ, 0);
- rtl8xxxu_write32(priv, REG_RQPN, 0x80000000);
- mdelay(2);
-
- if (!retry)
- dev_warn(dev, "Failed to flush FIFO\n");
-
- return retval;
-}
-
-static void rtl8xxxu_gen1_usb_quirks(struct rtl8xxxu_priv *priv)
-{
- /* Fix USB interface interference issue */
- rtl8xxxu_write8(priv, 0xfe40, 0xe0);
- rtl8xxxu_write8(priv, 0xfe41, 0x8d);
- rtl8xxxu_write8(priv, 0xfe42, 0x80);
- /*
- * This sets TXDMA_OFFSET_DROP_DATA_EN (bit 9) as well as bits
- * 8 and 5, for which I have found no documentation.
- */
- rtl8xxxu_write32(priv, REG_TXDMA_OFFSET_CHK, 0xfd0320);
-
- /*
- * Solve too many protocol error on USB bus.
- * Can't do this for 8188/8192 UMC A cut parts
- */
- if (!(!priv->chip_cut && priv->vendor_umc)) {
- rtl8xxxu_write8(priv, 0xfe40, 0xe6);
- rtl8xxxu_write8(priv, 0xfe41, 0x94);
- rtl8xxxu_write8(priv, 0xfe42, 0x80);
-
- rtl8xxxu_write8(priv, 0xfe40, 0xe0);
- rtl8xxxu_write8(priv, 0xfe41, 0x19);
- rtl8xxxu_write8(priv, 0xfe42, 0x80);
-
- rtl8xxxu_write8(priv, 0xfe40, 0xe5);
- rtl8xxxu_write8(priv, 0xfe41, 0x91);
- rtl8xxxu_write8(priv, 0xfe42, 0x80);
-
- rtl8xxxu_write8(priv, 0xfe40, 0xe2);
- rtl8xxxu_write8(priv, 0xfe41, 0x81);
- rtl8xxxu_write8(priv, 0xfe42, 0x80);
- }
-}
-
-static void rtl8xxxu_gen2_usb_quirks(struct rtl8xxxu_priv *priv)
-{
- u32 val32;
-
- val32 = rtl8xxxu_read32(priv, REG_TXDMA_OFFSET_CHK);
- val32 |= TXDMA_OFFSET_DROP_DATA_EN;
- rtl8xxxu_write32(priv, REG_TXDMA_OFFSET_CHK, val32);
-}
-
-static int rtl8723au_power_on(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
- u16 val16;
- u32 val32;
- int ret;
-
- /*
- * RSV_CTRL 0x001C[7:0] = 0x00, unlock ISO/CLK/Power control register
- */
- rtl8xxxu_write8(priv, REG_RSV_CTRL, 0x0);
-
- rtl8723a_disabled_to_emu(priv);
-
- ret = rtl8723a_emu_to_active(priv);
- if (ret)
- goto exit;
-
- /*
- * 0x0004[19] = 1, reset 8051
- */
- val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 2);
- val8 |= BIT(3);
- rtl8xxxu_write8(priv, REG_APS_FSMCO + 2, val8);
-
- /*
- * Enable MAC DMA/WMAC/SCHEDULE/SEC block
- * Set CR bit10 to enable 32k calibration.
- */
- val16 = rtl8xxxu_read16(priv, REG_CR);
- val16 |= (CR_HCI_TXDMA_ENABLE | CR_HCI_RXDMA_ENABLE |
- CR_TXDMA_ENABLE | CR_RXDMA_ENABLE |
- CR_PROTOCOL_ENABLE | CR_SCHEDULE_ENABLE |
- CR_MAC_TX_ENABLE | CR_MAC_RX_ENABLE |
- CR_SECURITY_ENABLE | CR_CALTIMER_ENABLE);
- rtl8xxxu_write16(priv, REG_CR, val16);
-
- /* For EFuse PG */
- val32 = rtl8xxxu_read32(priv, REG_EFUSE_CTRL);
- val32 &= ~(BIT(28) | BIT(29) | BIT(30));
- val32 |= (0x06 << 28);
- rtl8xxxu_write32(priv, REG_EFUSE_CTRL, val32);
-exit:
- return ret;
-}
-
-static int rtl8723bu_power_on(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
- u16 val16;
- u32 val32;
- int ret;
-
- rtl8723a_disabled_to_emu(priv);
-
- ret = rtl8723b_emu_to_active(priv);
- if (ret)
- goto exit;
-
- /*
- * Enable MAC DMA/WMAC/SCHEDULE/SEC block
- * Set CR bit10 to enable 32k calibration.
- */
- val16 = rtl8xxxu_read16(priv, REG_CR);
- val16 |= (CR_HCI_TXDMA_ENABLE | CR_HCI_RXDMA_ENABLE |
- CR_TXDMA_ENABLE | CR_RXDMA_ENABLE |
- CR_PROTOCOL_ENABLE | CR_SCHEDULE_ENABLE |
- CR_MAC_TX_ENABLE | CR_MAC_RX_ENABLE |
- CR_SECURITY_ENABLE | CR_CALTIMER_ENABLE);
- rtl8xxxu_write16(priv, REG_CR, val16);
-
- /*
- * BT coexist power on settings. This is identical for 1 and 2
- * antenna parts.
- */
- rtl8xxxu_write8(priv, REG_PAD_CTRL1 + 3, 0x20);
-
- val16 = rtl8xxxu_read16(priv, REG_SYS_FUNC);
- val16 |= SYS_FUNC_BBRSTB | SYS_FUNC_BB_GLB_RSTN;
- rtl8xxxu_write16(priv, REG_SYS_FUNC, val16);
-
- rtl8xxxu_write8(priv, REG_BT_CONTROL_8723BU + 1, 0x18);
- rtl8xxxu_write8(priv, REG_WLAN_ACT_CONTROL_8723B, 0x04);
- rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, 0x00);
- /* Antenna inverse */
- rtl8xxxu_write8(priv, 0xfe08, 0x01);
-
- val16 = rtl8xxxu_read16(priv, REG_PWR_DATA);
- val16 |= PWR_DATA_EEPRPAD_RFE_CTRL_EN;
- rtl8xxxu_write16(priv, REG_PWR_DATA, val16);
-
- val32 = rtl8xxxu_read32(priv, REG_LEDCFG0);
- val32 |= LEDCFG0_DPDT_SELECT;
- rtl8xxxu_write32(priv, REG_LEDCFG0, val32);
-
- val8 = rtl8xxxu_read8(priv, REG_PAD_CTRL1);
- val8 &= ~PAD_CTRL1_SW_DPDT_SEL_DATA;
- rtl8xxxu_write8(priv, REG_PAD_CTRL1, val8);
-exit:
- return ret;
-}
-
-#ifdef CONFIG_RTL8XXXU_UNTESTED
-
-static int rtl8192cu_power_on(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
- u16 val16;
- u32 val32;
- int i;
-
- for (i = 100; i; i--) {
- val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO);
- if (val8 & APS_FSMCO_PFM_ALDN)
- break;
- }
-
- if (!i) {
- pr_info("%s: Poll failed\n", __func__);
- return -ENODEV;
- }
-
- /*
- * RSV_CTRL 0x001C[7:0] = 0x00, unlock ISO/CLK/Power control register
- */
- rtl8xxxu_write8(priv, REG_RSV_CTRL, 0x0);
- rtl8xxxu_write8(priv, REG_SPS0_CTRL, 0x2b);
- udelay(100);
-
- val8 = rtl8xxxu_read8(priv, REG_LDOV12D_CTRL);
- if (!(val8 & LDOV12D_ENABLE)) {
- pr_info("%s: Enabling LDOV12D (%02x)\n", __func__, val8);
- val8 |= LDOV12D_ENABLE;
- rtl8xxxu_write8(priv, REG_LDOV12D_CTRL, val8);
-
- udelay(100);
-
- val8 = rtl8xxxu_read8(priv, REG_SYS_ISO_CTRL);
- val8 &= ~SYS_ISO_MD2PP;
- rtl8xxxu_write8(priv, REG_SYS_ISO_CTRL, val8);
- }
-
- /*
- * Auto enable WLAN
- */
- val16 = rtl8xxxu_read16(priv, REG_APS_FSMCO);
- val16 |= APS_FSMCO_MAC_ENABLE;
- rtl8xxxu_write16(priv, REG_APS_FSMCO, val16);
-
- for (i = 1000; i; i--) {
- val16 = rtl8xxxu_read16(priv, REG_APS_FSMCO);
- if (!(val16 & APS_FSMCO_MAC_ENABLE))
- break;
- }
- if (!i) {
- pr_info("%s: FSMCO_MAC_ENABLE poll failed\n", __func__);
- return -EBUSY;
- }
-
- /*
- * Enable radio, GPIO, LED
- */
- val16 = APS_FSMCO_HW_SUSPEND | APS_FSMCO_ENABLE_POWERDOWN |
- APS_FSMCO_PFM_ALDN;
- rtl8xxxu_write16(priv, REG_APS_FSMCO, val16);
-
- /*
- * Release RF digital isolation
- */
- val16 = rtl8xxxu_read16(priv, REG_SYS_ISO_CTRL);
- val16 &= ~SYS_ISO_DIOR;
- rtl8xxxu_write16(priv, REG_SYS_ISO_CTRL, val16);
-
- val8 = rtl8xxxu_read8(priv, REG_APSD_CTRL);
- val8 &= ~APSD_CTRL_OFF;
- rtl8xxxu_write8(priv, REG_APSD_CTRL, val8);
- for (i = 200; i; i--) {
- val8 = rtl8xxxu_read8(priv, REG_APSD_CTRL);
- if (!(val8 & APSD_CTRL_OFF_STATUS))
- break;
- }
-
- if (!i) {
- pr_info("%s: APSD_CTRL poll failed\n", __func__);
- return -EBUSY;
- }
-
- /*
- * Enable MAC DMA/WMAC/SCHEDULE/SEC block
- */
- val16 = rtl8xxxu_read16(priv, REG_CR);
- val16 |= CR_HCI_TXDMA_ENABLE | CR_HCI_RXDMA_ENABLE |
- CR_TXDMA_ENABLE | CR_RXDMA_ENABLE | CR_PROTOCOL_ENABLE |
- CR_SCHEDULE_ENABLE | CR_MAC_TX_ENABLE | CR_MAC_RX_ENABLE;
- rtl8xxxu_write16(priv, REG_CR, val16);
-
- rtl8xxxu_write8(priv, 0xfe10, 0x19);
-
- /*
- * Workaround for 8188RU LNA power leakage problem.
- */
- if (priv->rtl_chip == RTL8188R) {
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_XCD_RF_PARM);
- val32 &= ~BIT(1);
- rtl8xxxu_write32(priv, REG_FPGA0_XCD_RF_PARM, val32);
- }
- return 0;
-}
-
-#endif
-
-/*
- * This is needed for 8723bu as well, presumable
- */
-static void rtl8192e_crystal_afe_adjust(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
- u32 val32;
-
- /*
- * 40Mhz crystal source, MAC 0x28[2]=0
- */
- val8 = rtl8xxxu_read8(priv, REG_AFE_PLL_CTRL);
- val8 &= 0xfb;
- rtl8xxxu_write8(priv, REG_AFE_PLL_CTRL, val8);
-
- val32 = rtl8xxxu_read32(priv, REG_AFE_CTRL4);
- val32 &= 0xfffffc7f;
- rtl8xxxu_write32(priv, REG_AFE_CTRL4, val32);
-
- /*
- * 92e AFE parameter
- * AFE PLL KVCO selection, MAC 0x28[6]=1
- */
- val8 = rtl8xxxu_read8(priv, REG_AFE_PLL_CTRL);
- val8 &= 0xbf;
- rtl8xxxu_write8(priv, REG_AFE_PLL_CTRL, val8);
-
- /*
- * AFE PLL KVCO selection, MAC 0x78[21]=0
- */
- val32 = rtl8xxxu_read32(priv, REG_AFE_CTRL4);
- val32 &= 0xffdfffff;
- rtl8xxxu_write32(priv, REG_AFE_CTRL4, val32);
-}
-
-static int rtl8192eu_power_on(struct rtl8xxxu_priv *priv)
-{
- u16 val16;
- u32 val32;
- int ret;
-
- ret = 0;
-
- val32 = rtl8xxxu_read32(priv, REG_SYS_CFG);
- if (val32 & SYS_CFG_SPS_LDO_SEL) {
- rtl8xxxu_write8(priv, REG_LDO_SW_CTRL, 0xc3);
- } else {
- /*
- * Raise 1.2V voltage
- */
- val32 = rtl8xxxu_read32(priv, REG_8192E_LDOV12_CTRL);
- val32 &= 0xff0fffff;
- val32 |= 0x00500000;
- rtl8xxxu_write32(priv, REG_8192E_LDOV12_CTRL, val32);
- rtl8xxxu_write8(priv, REG_LDO_SW_CTRL, 0x83);
- }
-
- /*
- * Adjust AFE before enabling PLL
- */
- rtl8192e_crystal_afe_adjust(priv);
- rtl8192e_disabled_to_emu(priv);
-
- ret = rtl8192e_emu_to_active(priv);
- if (ret)
- goto exit;
-
- rtl8xxxu_write16(priv, REG_CR, 0x0000);
-
- /*
- * Enable MAC DMA/WMAC/SCHEDULE/SEC block
- * Set CR bit10 to enable 32k calibration.
- */
- val16 = rtl8xxxu_read16(priv, REG_CR);
- val16 |= (CR_HCI_TXDMA_ENABLE | CR_HCI_RXDMA_ENABLE |
- CR_TXDMA_ENABLE | CR_RXDMA_ENABLE |
- CR_PROTOCOL_ENABLE | CR_SCHEDULE_ENABLE |
- CR_MAC_TX_ENABLE | CR_MAC_RX_ENABLE |
- CR_SECURITY_ENABLE | CR_CALTIMER_ENABLE);
- rtl8xxxu_write16(priv, REG_CR, val16);
-
-exit:
- return ret;
-}
-
-static void rtl8xxxu_power_off(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
- u16 val16;
- u32 val32;
-
- /*
- * Workaround for 8188RU LNA power leakage problem.
- */
- if (priv->rtl_chip == RTL8188R) {
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_XCD_RF_PARM);
- val32 |= BIT(1);
- rtl8xxxu_write32(priv, REG_FPGA0_XCD_RF_PARM, val32);
- }
-
- rtl8xxxu_flush_fifo(priv);
-
- rtl8xxxu_active_to_lps(priv);
-
- /* Turn off RF */
- rtl8xxxu_write8(priv, REG_RF_CTRL, 0x00);
-
- /* Reset Firmware if running in RAM */
- if (rtl8xxxu_read8(priv, REG_MCU_FW_DL) & MCU_FW_RAM_SEL)
- rtl8xxxu_firmware_self_reset(priv);
-
- /* Reset MCU */
- val16 = rtl8xxxu_read16(priv, REG_SYS_FUNC);
- val16 &= ~SYS_FUNC_CPU_ENABLE;
- rtl8xxxu_write16(priv, REG_SYS_FUNC, val16);
-
- /* Reset MCU ready status */
- rtl8xxxu_write8(priv, REG_MCU_FW_DL, 0x00);
-
- rtl8xxxu_active_to_emu(priv);
- rtl8xxxu_emu_to_disabled(priv);
-
- /* Reset MCU IO Wrapper */
- val8 = rtl8xxxu_read8(priv, REG_RSV_CTRL + 1);
- val8 &= ~BIT(0);
- rtl8xxxu_write8(priv, REG_RSV_CTRL + 1, val8);
-
- val8 = rtl8xxxu_read8(priv, REG_RSV_CTRL + 1);
- val8 |= BIT(0);
- rtl8xxxu_write8(priv, REG_RSV_CTRL + 1, val8);
-
- /* RSV_CTRL 0x1C[7:0] = 0x0e lock ISO/CLK/Power control register */
- rtl8xxxu_write8(priv, REG_RSV_CTRL, 0x0e);
-}
-
-static void rtl8723bu_power_off(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
- u16 val16;
-
- rtl8xxxu_flush_fifo(priv);
-
- /*
- * Disable TX report timer
- */
- val8 = rtl8xxxu_read8(priv, REG_TX_REPORT_CTRL);
- val8 &= ~TX_REPORT_CTRL_TIMER_ENABLE;
- rtl8xxxu_write8(priv, REG_TX_REPORT_CTRL, val8);
-
- rtl8xxxu_write8(priv, REG_CR, 0x0000);
-
- rtl8xxxu_active_to_lps(priv);
-
- /* Reset Firmware if running in RAM */
- if (rtl8xxxu_read8(priv, REG_MCU_FW_DL) & MCU_FW_RAM_SEL)
- rtl8xxxu_firmware_self_reset(priv);
-
- /* Reset MCU */
- val16 = rtl8xxxu_read16(priv, REG_SYS_FUNC);
- val16 &= ~SYS_FUNC_CPU_ENABLE;
- rtl8xxxu_write16(priv, REG_SYS_FUNC, val16);
-
- /* Reset MCU ready status */
- rtl8xxxu_write8(priv, REG_MCU_FW_DL, 0x00);
-
- rtl8723bu_active_to_emu(priv);
-
- val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
- val8 |= BIT(3); /* APS_FSMCO_HW_SUSPEND */
- rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
-
- /* 0x48[16] = 1 to enable GPIO9 as EXT wakeup */
- val8 = rtl8xxxu_read8(priv, REG_GPIO_INTM + 2);
- val8 |= BIT(0);
- rtl8xxxu_write8(priv, REG_GPIO_INTM + 2, val8);
-}
-
-#ifdef NEED_PS_TDMA
-static void rtl8723bu_set_ps_tdma(struct rtl8xxxu_priv *priv,
- u8 arg1, u8 arg2, u8 arg3, u8 arg4, u8 arg5)
-{
- struct h2c_cmd h2c;
-
- memset(&h2c, 0, sizeof(struct h2c_cmd));
- h2c.b_type_dma.cmd = H2C_8723B_B_TYPE_TDMA;
- h2c.b_type_dma.data1 = arg1;
- h2c.b_type_dma.data2 = arg2;
- h2c.b_type_dma.data3 = arg3;
- h2c.b_type_dma.data4 = arg4;
- h2c.b_type_dma.data5 = arg5;
- rtl8723a_h2c_cmd(priv, &h2c, sizeof(h2c.b_type_dma));
-}
-#endif
-
-static void rtl8192e_enable_rf(struct rtl8xxxu_priv *priv)
-{
- u32 val32;
- u8 val8;
-
- val8 = rtl8xxxu_read8(priv, REG_GPIO_MUXCFG);
- val8 |= BIT(5);
- rtl8xxxu_write8(priv, REG_GPIO_MUXCFG, val8);
-
- /*
- * WLAN action by PTA
- */
- rtl8xxxu_write8(priv, REG_WLAN_ACT_CONTROL_8723B, 0x04);
-
- val32 = rtl8xxxu_read32(priv, REG_PWR_DATA);
- val32 |= PWR_DATA_EEPRPAD_RFE_CTRL_EN;
- rtl8xxxu_write32(priv, REG_PWR_DATA, val32);
-
- val32 = rtl8xxxu_read32(priv, REG_RFE_BUFFER);
- val32 |= (BIT(0) | BIT(1));
- rtl8xxxu_write32(priv, REG_RFE_BUFFER, val32);
-
- rtl8xxxu_write8(priv, REG_RFE_CTRL_ANTA_SRC, 0x77);
-
- val32 = rtl8xxxu_read32(priv, REG_LEDCFG0);
- val32 &= ~BIT(24);
- val32 |= BIT(23);
- rtl8xxxu_write32(priv, REG_LEDCFG0, val32);
-
- /*
- * Fix external switch Main->S1, Aux->S0
- */
- val8 = rtl8xxxu_read8(priv, REG_PAD_CTRL1);
- val8 &= ~BIT(0);
- rtl8xxxu_write8(priv, REG_PAD_CTRL1, val8);
-}
-
-static void rtl8723b_enable_rf(struct rtl8xxxu_priv *priv)
-{
- struct h2c_cmd h2c;
- u32 val32;
- u8 val8;
-
- /*
- * No indication anywhere as to what 0x0790 does. The 2 antenna
- * vendor code preserves bits 6-7 here.
- */
- rtl8xxxu_write8(priv, 0x0790, 0x05);
- /*
- * 0x0778 seems to be related to enabling the number of antennas
- * In the vendor driver halbtc8723b2ant_InitHwConfig() sets it
- * to 0x03, while halbtc8723b1ant_InitHwConfig() sets it to 0x01
- */
- rtl8xxxu_write8(priv, 0x0778, 0x01);
-
- val8 = rtl8xxxu_read8(priv, REG_GPIO_MUXCFG);
- val8 |= BIT(5);
- rtl8xxxu_write8(priv, REG_GPIO_MUXCFG, val8);
-
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_IQADJ_G1, 0x780);
-
- rtl8723bu_write_btreg(priv, 0x3c, 0x15); /* BT TRx Mask on */
-
- /*
- * Set BT grant to low
- */
- memset(&h2c, 0, sizeof(struct h2c_cmd));
- h2c.bt_grant.cmd = H2C_8723B_BT_GRANT;
- h2c.bt_grant.data = 0;
- rtl8723a_h2c_cmd(priv, &h2c, sizeof(h2c.bt_grant));
-
- /*
- * WLAN action by PTA
- */
- rtl8xxxu_write8(priv, REG_WLAN_ACT_CONTROL_8723B, 0x04);
-
- /*
- * BT select S0/S1 controlled by WiFi
- */
- val8 = rtl8xxxu_read8(priv, 0x0067);
- val8 |= BIT(5);
- rtl8xxxu_write8(priv, 0x0067, val8);
-
- val32 = rtl8xxxu_read32(priv, REG_PWR_DATA);
- val32 |= PWR_DATA_EEPRPAD_RFE_CTRL_EN;
- rtl8xxxu_write32(priv, REG_PWR_DATA, val32);
-
- /*
- * Bits 6/7 are marked in/out ... but for what?
- */
- rtl8xxxu_write8(priv, 0x0974, 0xff);
-
- val32 = rtl8xxxu_read32(priv, REG_RFE_BUFFER);
- val32 |= (BIT(0) | BIT(1));
- rtl8xxxu_write32(priv, REG_RFE_BUFFER, val32);
-
- rtl8xxxu_write8(priv, REG_RFE_CTRL_ANTA_SRC, 0x77);
-
- val32 = rtl8xxxu_read32(priv, REG_LEDCFG0);
- val32 &= ~BIT(24);
- val32 |= BIT(23);
- rtl8xxxu_write32(priv, REG_LEDCFG0, val32);
-
- /*
- * Fix external switch Main->S1, Aux->S0
- */
- val8 = rtl8xxxu_read8(priv, REG_PAD_CTRL1);
- val8 &= ~BIT(0);
- rtl8xxxu_write8(priv, REG_PAD_CTRL1, val8);
-
- memset(&h2c, 0, sizeof(struct h2c_cmd));
- h2c.ant_sel_rsv.cmd = H2C_8723B_ANT_SEL_RSV;
- h2c.ant_sel_rsv.ant_inverse = 1;
- h2c.ant_sel_rsv.int_switch_type = 0;
- rtl8723a_h2c_cmd(priv, &h2c, sizeof(h2c.ant_sel_rsv));
-
- /*
- * 0x280, 0x00, 0x200, 0x80 - not clear
- */
- rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, 0x00);
-
- /*
- * Software control, antenna at WiFi side
- */
-#ifdef NEED_PS_TDMA
- rtl8723bu_set_ps_tdma(priv, 0x08, 0x00, 0x00, 0x00, 0x00);
-#endif
-
- rtl8xxxu_write32(priv, REG_BT_COEX_TABLE1, 0x55555555);
- rtl8xxxu_write32(priv, REG_BT_COEX_TABLE2, 0x55555555);
- rtl8xxxu_write32(priv, REG_BT_COEX_TABLE3, 0x00ffffff);
- rtl8xxxu_write8(priv, REG_BT_COEX_TABLE4, 0x03);
-
- memset(&h2c, 0, sizeof(struct h2c_cmd));
- h2c.bt_info.cmd = H2C_8723B_BT_INFO;
- h2c.bt_info.data = BIT(0);
- rtl8723a_h2c_cmd(priv, &h2c, sizeof(h2c.bt_info));
-
- memset(&h2c, 0, sizeof(struct h2c_cmd));
- h2c.ignore_wlan.cmd = H2C_8723B_BT_IGNORE_WLANACT;
- h2c.ignore_wlan.data = 0;
- rtl8723a_h2c_cmd(priv, &h2c, sizeof(h2c.ignore_wlan));
-}
-
-static void rtl8xxxu_gen2_disable_rf(struct rtl8xxxu_priv *priv)
-{
- u32 val32;
-
- val32 = rtl8xxxu_read32(priv, REG_RX_WAIT_CCA);
- val32 &= ~(BIT(22) | BIT(23));
- rtl8xxxu_write32(priv, REG_RX_WAIT_CCA, val32);
-}
-
-static void rtl8723bu_init_aggregation(struct rtl8xxxu_priv *priv)
-{
- u32 agg_rx;
- u8 agg_ctrl;
-
- /*
- * For now simply disable RX aggregation
- */
- agg_ctrl = rtl8xxxu_read8(priv, REG_TRXDMA_CTRL);
- agg_ctrl &= ~TRXDMA_CTRL_RXDMA_AGG_EN;
-
- agg_rx = rtl8xxxu_read32(priv, REG_RXDMA_AGG_PG_TH);
- agg_rx &= ~RXDMA_USB_AGG_ENABLE;
- agg_rx &= ~0xff0f;
-
- rtl8xxxu_write8(priv, REG_TRXDMA_CTRL, agg_ctrl);
- rtl8xxxu_write32(priv, REG_RXDMA_AGG_PG_TH, agg_rx);
-}
-
-static void rtl8723bu_init_statistics(struct rtl8xxxu_priv *priv)
-{
- u32 val32;
-
- /* Time duration for NHM unit: 4us, 0x2710=40ms */
- rtl8xxxu_write16(priv, REG_NHM_TIMER_8723B + 2, 0x2710);
- rtl8xxxu_write16(priv, REG_NHM_TH9_TH10_8723B + 2, 0xffff);
- rtl8xxxu_write32(priv, REG_NHM_TH3_TO_TH0_8723B, 0xffffff52);
- rtl8xxxu_write32(priv, REG_NHM_TH7_TO_TH4_8723B, 0xffffffff);
- /* TH8 */
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
- val32 |= 0xff;
- rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
- /* Enable CCK */
- val32 = rtl8xxxu_read32(priv, REG_NHM_TH9_TH10_8723B);
- val32 |= BIT(8) | BIT(9) | BIT(10);
- rtl8xxxu_write32(priv, REG_NHM_TH9_TH10_8723B, val32);
- /* Max power amongst all RX antennas */
- val32 = rtl8xxxu_read32(priv, REG_OFDM0_FA_RSTC);
- val32 |= BIT(7);
- rtl8xxxu_write32(priv, REG_OFDM0_FA_RSTC, val32);
-}
-
-static void rtl8xxxu_old_init_queue_reserved_page(struct rtl8xxxu_priv *priv)
-{
- u8 val8;
- u32 val32;
-
- if (priv->ep_tx_normal_queue)
- val8 = TX_PAGE_NUM_NORM_PQ;
- else
- val8 = 0;
-
- rtl8xxxu_write8(priv, REG_RQPN_NPQ, val8);
-
- val32 = (TX_PAGE_NUM_PUBQ << RQPN_PUB_PQ_SHIFT) | RQPN_LOAD;
-
- if (priv->ep_tx_high_queue)
- val32 |= (TX_PAGE_NUM_HI_PQ << RQPN_HI_PQ_SHIFT);
- if (priv->ep_tx_low_queue)
- val32 |= (TX_PAGE_NUM_LO_PQ << RQPN_LO_PQ_SHIFT);
-
- rtl8xxxu_write32(priv, REG_RQPN, val32);
-}
-
-static void rtl8xxxu_init_queue_reserved_page(struct rtl8xxxu_priv *priv)
-{
- struct rtl8xxxu_fileops *fops = priv->fops;
- u32 hq, lq, nq, eq, pubq;
- u32 val32;
-
- hq = 0;
- lq = 0;
- nq = 0;
- eq = 0;
- pubq = 0;
-
- if (priv->ep_tx_high_queue)
- hq = fops->page_num_hi;
- if (priv->ep_tx_low_queue)
- lq = fops->page_num_lo;
- if (priv->ep_tx_normal_queue)
- nq = fops->page_num_norm;
-
- val32 = (nq << RQPN_NPQ_SHIFT) | (eq << RQPN_EPQ_SHIFT);
- rtl8xxxu_write32(priv, REG_RQPN_NPQ, val32);
-
- pubq = fops->total_page_num - hq - lq - nq;
-
- val32 = RQPN_LOAD;
- val32 |= (hq << RQPN_HI_PQ_SHIFT);
- val32 |= (lq << RQPN_LO_PQ_SHIFT);
- val32 |= (pubq << RQPN_PUB_PQ_SHIFT);
-
- rtl8xxxu_write32(priv, REG_RQPN, val32);
-}
-
-static int rtl8xxxu_init_device(struct ieee80211_hw *hw)
-{
- struct rtl8xxxu_priv *priv = hw->priv;
- struct device *dev = &priv->udev->dev;
- bool macpower;
- int ret;
- u8 val8;
- u16 val16;
- u32 val32;
-
- /* Check if MAC is already powered on */
- val8 = rtl8xxxu_read8(priv, REG_CR);
-
- /*
- * Fix 92DU-VC S3 hang with the reason is that secondary mac is not
- * initialized. First MAC returns 0xea, second MAC returns 0x00
- */
- if (val8 == 0xea)
- macpower = false;
- else
- macpower = true;
-
- ret = priv->fops->power_on(priv);
- if (ret < 0) {
- dev_warn(dev, "%s: Failed power on\n", __func__);
- goto exit;
- }
-
- if (!macpower) {
- if (priv->fops->total_page_num)
- rtl8xxxu_init_queue_reserved_page(priv);
- else
- rtl8xxxu_old_init_queue_reserved_page(priv);
- }
-
- ret = rtl8xxxu_init_queue_priority(priv);
- dev_dbg(dev, "%s: init_queue_priority %i\n", __func__, ret);
- if (ret)
- goto exit;
-
- /*
- * Set RX page boundary
- */
- rtl8xxxu_write16(priv, REG_TRXFF_BNDY + 2, priv->fops->trxff_boundary);
-
- ret = rtl8xxxu_download_firmware(priv);
- dev_dbg(dev, "%s: download_fiwmare %i\n", __func__, ret);
- if (ret)
- goto exit;
- ret = rtl8xxxu_start_firmware(priv);
- dev_dbg(dev, "%s: start_fiwmare %i\n", __func__, ret);
- if (ret)
- goto exit;
-
- if (priv->fops->phy_init_antenna_selection)
- priv->fops->phy_init_antenna_selection(priv);
-
- ret = rtl8xxxu_init_mac(priv);
-
- dev_dbg(dev, "%s: init_mac %i\n", __func__, ret);
- if (ret)
- goto exit;
-
- ret = rtl8xxxu_init_phy_bb(priv);
- dev_dbg(dev, "%s: init_phy_bb %i\n", __func__, ret);
- if (ret)
- goto exit;
-
- ret = priv->fops->init_phy_rf(priv);
- if (ret)
- goto exit;
-
- /* RFSW Control - clear bit 14 ?? */
- if (priv->rtl_chip != RTL8723B && priv->rtl_chip != RTL8192E)
- rtl8xxxu_write32(priv, REG_FPGA0_TX_INFO, 0x00000003);
-
- val32 = FPGA0_RF_TRSW | FPGA0_RF_TRSWB | FPGA0_RF_ANTSW |
- FPGA0_RF_ANTSWB |
- ((FPGA0_RF_ANTSW | FPGA0_RF_ANTSWB) << FPGA0_RF_BD_CTRL_SHIFT);
- if (!priv->no_pape) {
- val32 |= (FPGA0_RF_PAPE |
- (FPGA0_RF_PAPE << FPGA0_RF_BD_CTRL_SHIFT));
- }
- rtl8xxxu_write32(priv, REG_FPGA0_XAB_RF_SW_CTRL, val32);
-
- /* 0x860[6:5]= 00 - why? - this sets antenna B */
- if (priv->rtl_chip != RTL8192E)
- rtl8xxxu_write32(priv, REG_FPGA0_XA_RF_INT_OE, 0x66f60210);
-
- if (!macpower) {
- /*
- * Set TX buffer boundary
- */
- if (priv->rtl_chip == RTL8192E)
- val8 = TX_TOTAL_PAGE_NUM_8192E + 1;
- else
- val8 = TX_TOTAL_PAGE_NUM + 1;
-
- if (priv->rtl_chip == RTL8723B)
- val8 -= 1;
-
- rtl8xxxu_write8(priv, REG_TXPKTBUF_BCNQ_BDNY, val8);
- rtl8xxxu_write8(priv, REG_TXPKTBUF_MGQ_BDNY, val8);
- rtl8xxxu_write8(priv, REG_TXPKTBUF_WMAC_LBK_BF_HD, val8);
- rtl8xxxu_write8(priv, REG_TRXFF_BNDY, val8);
- rtl8xxxu_write8(priv, REG_TDECTRL + 1, val8);
- }
-
- /*
- * The vendor drivers set PBP for all devices, except 8192e.
- * There is no explanation for this in any of the sources.
- */
- val8 = (priv->fops->pbp_rx << PBP_PAGE_SIZE_RX_SHIFT) |
- (priv->fops->pbp_tx << PBP_PAGE_SIZE_TX_SHIFT);
- if (priv->rtl_chip != RTL8192E)
- rtl8xxxu_write8(priv, REG_PBP, val8);
-
- dev_dbg(dev, "%s: macpower %i\n", __func__, macpower);
- if (!macpower) {
- ret = priv->fops->llt_init(priv, TX_TOTAL_PAGE_NUM);
- if (ret) {
- dev_warn(dev, "%s: LLT table init failed\n", __func__);
- goto exit;
- }
-
- /*
- * Chip specific quirks
- */
- priv->fops->usb_quirks(priv);
-
- /*
- * Presumably this is for 8188EU as well
- * Enable TX report and TX report timer
- */
- if (priv->rtl_chip == RTL8723B) {
- val8 = rtl8xxxu_read8(priv, REG_TX_REPORT_CTRL);
- val8 |= TX_REPORT_CTRL_TIMER_ENABLE;
- rtl8xxxu_write8(priv, REG_TX_REPORT_CTRL, val8);
- /* Set MAX RPT MACID */
- rtl8xxxu_write8(priv, REG_TX_REPORT_CTRL + 1, 0x02);
- /* TX report Timer. Unit: 32us */
- rtl8xxxu_write16(priv, REG_TX_REPORT_TIME, 0xcdf0);
-
- /* tmp ps ? */
- val8 = rtl8xxxu_read8(priv, 0xa3);
- val8 &= 0xf8;
- rtl8xxxu_write8(priv, 0xa3, val8);
- }
- }
-
- /*
- * Unit in 8 bytes, not obvious what it is used for
- */
- rtl8xxxu_write8(priv, REG_RX_DRVINFO_SZ, 4);
-
- if (priv->rtl_chip == RTL8192E) {
- rtl8xxxu_write32(priv, REG_HIMR0, 0x00);
- rtl8xxxu_write32(priv, REG_HIMR1, 0x00);
- } else {
- /*
- * Enable all interrupts - not obvious USB needs to do this
- */
- rtl8xxxu_write32(priv, REG_HISR, 0xffffffff);
- rtl8xxxu_write32(priv, REG_HIMR, 0xffffffff);
- }
-
- rtl8xxxu_set_mac(priv);
- rtl8xxxu_set_linktype(priv, NL80211_IFTYPE_STATION);
-
- /*
- * Configure initial WMAC settings
- */
- val32 = RCR_ACCEPT_PHYS_MATCH | RCR_ACCEPT_MCAST | RCR_ACCEPT_BCAST |
- RCR_ACCEPT_MGMT_FRAME | RCR_HTC_LOC_CTRL |
- RCR_APPEND_PHYSTAT | RCR_APPEND_ICV | RCR_APPEND_MIC;
- rtl8xxxu_write32(priv, REG_RCR, val32);
-
- /*
- * Accept all multicast
- */
- rtl8xxxu_write32(priv, REG_MAR, 0xffffffff);
- rtl8xxxu_write32(priv, REG_MAR + 4, 0xffffffff);
-
- /*
- * Init adaptive controls
- */
- val32 = rtl8xxxu_read32(priv, REG_RESPONSE_RATE_SET);
- val32 &= ~RESPONSE_RATE_BITMAP_ALL;
- val32 |= RESPONSE_RATE_RRSR_CCK_ONLY_1M;
- rtl8xxxu_write32(priv, REG_RESPONSE_RATE_SET, val32);
-
- /* CCK = 0x0a, OFDM = 0x10 */
- rtl8xxxu_set_spec_sifs(priv, 0x10, 0x10);
- rtl8xxxu_set_retry(priv, 0x30, 0x30);
- rtl8xxxu_set_spec_sifs(priv, 0x0a, 0x10);
-
- /*
- * Init EDCA
- */
- rtl8xxxu_write16(priv, REG_MAC_SPEC_SIFS, 0x100a);
-
- /* Set CCK SIFS */
- rtl8xxxu_write16(priv, REG_SIFS_CCK, 0x100a);
-
- /* Set OFDM SIFS */
- rtl8xxxu_write16(priv, REG_SIFS_OFDM, 0x100a);
-
- /* TXOP */
- rtl8xxxu_write32(priv, REG_EDCA_BE_PARAM, 0x005ea42b);
- rtl8xxxu_write32(priv, REG_EDCA_BK_PARAM, 0x0000a44f);
- rtl8xxxu_write32(priv, REG_EDCA_VI_PARAM, 0x005ea324);
- rtl8xxxu_write32(priv, REG_EDCA_VO_PARAM, 0x002fa226);
-
- /* Set data auto rate fallback retry count */
- rtl8xxxu_write32(priv, REG_DARFRC, 0x00000000);
- rtl8xxxu_write32(priv, REG_DARFRC + 4, 0x10080404);
- rtl8xxxu_write32(priv, REG_RARFRC, 0x04030201);
- rtl8xxxu_write32(priv, REG_RARFRC + 4, 0x08070605);
-
- val8 = rtl8xxxu_read8(priv, REG_FWHW_TXQ_CTRL);
- val8 |= FWHW_TXQ_CTRL_AMPDU_RETRY;
- rtl8xxxu_write8(priv, REG_FWHW_TXQ_CTRL, val8);
-
- /* Set ACK timeout */
- rtl8xxxu_write8(priv, REG_ACKTO, 0x40);
-
- /*
- * Initialize beacon parameters
- */
- val16 = BEACON_DISABLE_TSF_UPDATE | (BEACON_DISABLE_TSF_UPDATE << 8);
- rtl8xxxu_write16(priv, REG_BEACON_CTRL, val16);
- rtl8xxxu_write16(priv, REG_TBTT_PROHIBIT, 0x6404);
- rtl8xxxu_write8(priv, REG_DRIVER_EARLY_INT, DRIVER_EARLY_INT_TIME);
- rtl8xxxu_write8(priv, REG_BEACON_DMA_TIME, BEACON_DMA_ATIME_INT_TIME);
- rtl8xxxu_write16(priv, REG_BEACON_TCFG, 0x660F);
-
- /*
- * Initialize burst parameters
- */
- if (priv->rtl_chip == RTL8723B) {
- /*
- * For USB high speed set 512B packets
- */
- val8 = rtl8xxxu_read8(priv, REG_RXDMA_PRO_8723B);
- val8 &= ~(BIT(4) | BIT(5));
- val8 |= BIT(4);
- val8 |= BIT(1) | BIT(2) | BIT(3);
- rtl8xxxu_write8(priv, REG_RXDMA_PRO_8723B, val8);
-
- /*
- * For USB high speed set 512B packets
- */
- val8 = rtl8xxxu_read8(priv, REG_HT_SINGLE_AMPDU_8723B);
- val8 |= BIT(7);
- rtl8xxxu_write8(priv, REG_HT_SINGLE_AMPDU_8723B, val8);
-
- rtl8xxxu_write16(priv, REG_MAX_AGGR_NUM, 0x0c14);
- rtl8xxxu_write8(priv, REG_AMPDU_MAX_TIME_8723B, 0x5e);
- rtl8xxxu_write32(priv, REG_AGGLEN_LMT, 0xffffffff);
- rtl8xxxu_write8(priv, REG_RX_PKT_LIMIT, 0x18);
- rtl8xxxu_write8(priv, REG_PIFS, 0x00);
- rtl8xxxu_write8(priv, REG_USTIME_TSF_8723B, 0x50);
- rtl8xxxu_write8(priv, REG_USTIME_EDCA, 0x50);
-
- val8 = rtl8xxxu_read8(priv, REG_RSV_CTRL);
- val8 |= BIT(5) | BIT(6);
- rtl8xxxu_write8(priv, REG_RSV_CTRL, val8);
- }
-
- if (priv->fops->init_aggregation)
- priv->fops->init_aggregation(priv);
-
- /*
- * Enable CCK and OFDM block
- */
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_RF_MODE);
- val32 |= (FPGA_RF_MODE_CCK | FPGA_RF_MODE_OFDM);
- rtl8xxxu_write32(priv, REG_FPGA0_RF_MODE, val32);
-
- /*
- * Invalidate all CAM entries - bit 30 is undocumented
- */
- rtl8xxxu_write32(priv, REG_CAM_CMD, CAM_CMD_POLLING | BIT(30));
-
- /*
- * Start out with default power levels for channel 6, 20MHz
- */
- priv->fops->set_tx_power(priv, 1, false);
-
- /* Let the 8051 take control of antenna setting */
- if (priv->rtl_chip != RTL8192E) {
- val8 = rtl8xxxu_read8(priv, REG_LEDCFG2);
- val8 |= LEDCFG2_DPDT_SELECT;
- rtl8xxxu_write8(priv, REG_LEDCFG2, val8);
- }
-
- rtl8xxxu_write8(priv, REG_HWSEQ_CTRL, 0xff);
-
- /* Disable BAR - not sure if this has any effect on USB */
- rtl8xxxu_write32(priv, REG_BAR_MODE_CTRL, 0x0201ffff);
-
- rtl8xxxu_write16(priv, REG_FAST_EDCA_CTRL, 0);
-
- if (priv->fops->init_statistics)
- priv->fops->init_statistics(priv);
-
- if (priv->rtl_chip == RTL8192E) {
- /*
- * 0x4c6[3] 1: RTS BW = Data BW
- * 0: RTS BW depends on CCA / secondary CCA result.
- */
- val8 = rtl8xxxu_read8(priv, REG_QUEUE_CTRL);
- val8 &= ~BIT(3);
- rtl8xxxu_write8(priv, REG_QUEUE_CTRL, val8);
- /*
- * Reset USB mode switch setting
- */
- rtl8xxxu_write8(priv, REG_ACLK_MON, 0x00);
- }
-
- rtl8723a_phy_lc_calibrate(priv);
-
- priv->fops->phy_iq_calibrate(priv);
-
- /*
- * This should enable thermal meter
- */
- if (priv->fops->tx_desc_size == sizeof(struct rtl8xxxu_txdesc40))
- rtl8xxxu_write_rfreg(priv,
- RF_A, RF6052_REG_T_METER_8723B, 0x37cf8);
- else
- rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_T_METER, 0x60);
-
- /* Set NAV_UPPER to 30000us */
- val8 = ((30000 + NAV_UPPER_UNIT - 1) / NAV_UPPER_UNIT);
- rtl8xxxu_write8(priv, REG_NAV_UPPER, val8);
-
- if (priv->rtl_chip == RTL8723A) {
- /*
- * 2011/03/09 MH debug only, UMC-B cut pass 2500 S5 test,
- * but we need to find root cause.
- * This is 8723au only.
- */
- val32 = rtl8xxxu_read32(priv, REG_FPGA0_RF_MODE);
- if ((val32 & 0xff000000) != 0x83000000) {
- val32 |= FPGA_RF_MODE_CCK;
- rtl8xxxu_write32(priv, REG_FPGA0_RF_MODE, val32);
- }
- } else if (priv->rtl_chip == RTL8192E) {
- rtl8xxxu_write8(priv, REG_USB_HRPWM, 0x00);
- }
-
- val32 = rtl8xxxu_read32(priv, REG_FWHW_TXQ_CTRL);
- val32 |= FWHW_TXQ_CTRL_XMIT_MGMT_ACK;
- /* ack for xmit mgmt frames. */
- rtl8xxxu_write32(priv, REG_FWHW_TXQ_CTRL, val32);
-
- if (priv->rtl_chip == RTL8192E) {
- /*
- * Fix LDPC rx hang issue.
- */
- val32 = rtl8xxxu_read32(priv, REG_AFE_MISC);
- rtl8xxxu_write8(priv, REG_8192E_LDOV12_CTRL, 0x75);
- val32 &= 0xfff00fff;
- val32 |= 0x0007e000;
- rtl8xxxu_write32(priv, REG_AFE_MISC, val32);
- }
-exit:
- return ret;
-}
-
-static void rtl8xxxu_cam_write(struct rtl8xxxu_priv *priv,
- struct ieee80211_key_conf *key, const u8 *mac)
-{
- u32 cmd, val32, addr, ctrl;
- int j, i, tmp_debug;
-
- tmp_debug = rtl8xxxu_debug;
- if (rtl8xxxu_debug & RTL8XXXU_DEBUG_KEY)
- rtl8xxxu_debug |= RTL8XXXU_DEBUG_REG_WRITE;
-
- /*
- * This is a bit of a hack - the lower bits of the cipher
- * suite selector happens to match the cipher index in the CAM
- */
- addr = key->keyidx << CAM_CMD_KEY_SHIFT;
- ctrl = (key->cipher & 0x0f) << 2 | key->keyidx | CAM_WRITE_VALID;
-
- for (j = 5; j >= 0; j--) {
- switch (j) {
- case 0:
- val32 = ctrl | (mac[0] << 16) | (mac[1] << 24);
- break;
- case 1:
- val32 = mac[2] | (mac[3] << 8) |
- (mac[4] << 16) | (mac[5] << 24);
- break;
- default:
- i = (j - 2) << 2;
- val32 = key->key[i] | (key->key[i + 1] << 8) |
- key->key[i + 2] << 16 | key->key[i + 3] << 24;
- break;
- }
-
- rtl8xxxu_write32(priv, REG_CAM_WRITE, val32);
- cmd = CAM_CMD_POLLING | CAM_CMD_WRITE | (addr + j);
- rtl8xxxu_write32(priv, REG_CAM_CMD, cmd);
- udelay(100);
- }
-
- rtl8xxxu_debug = tmp_debug;
-}
-
-static void rtl8xxxu_sw_scan_start(struct ieee80211_hw *hw,
- struct ieee80211_vif *vif, const u8 *mac)
-{
- struct rtl8xxxu_priv *priv = hw->priv;
- u8 val8;
-
- val8 = rtl8xxxu_read8(priv, REG_BEACON_CTRL);
- val8 |= BEACON_DISABLE_TSF_UPDATE;
- rtl8xxxu_write8(priv, REG_BEACON_CTRL, val8);
-}
-
-static void rtl8xxxu_sw_scan_complete(struct ieee80211_hw *hw,
- struct ieee80211_vif *vif)
-{
- struct rtl8xxxu_priv *priv = hw->priv;
- u8 val8;
-
- val8 = rtl8xxxu_read8(priv, REG_BEACON_CTRL);
- val8 &= ~BEACON_DISABLE_TSF_UPDATE;
- rtl8xxxu_write8(priv, REG_BEACON_CTRL, val8);
-}
-
-static void rtl8xxxu_update_rate_mask(struct rtl8xxxu_priv *priv,
- u32 ramask, int sgi)
-{
- struct h2c_cmd h2c;
-
- memset(&h2c, 0, sizeof(struct h2c_cmd));
-
- h2c.ramask.cmd = H2C_SET_RATE_MASK;
- h2c.ramask.mask_lo = cpu_to_le16(ramask & 0xffff);
- h2c.ramask.mask_hi = cpu_to_le16(ramask >> 16);
-
- h2c.ramask.arg = 0x80;
- if (sgi)
- h2c.ramask.arg |= 0x20;
-
- dev_dbg(&priv->udev->dev, "%s: rate mask %08x, arg %02x, size %zi\n",
- __func__, ramask, h2c.ramask.arg, sizeof(h2c.ramask));
- rtl8723a_h2c_cmd(priv, &h2c, sizeof(h2c.ramask));
-}
-
-static void rtl8xxxu_gen2_update_rate_mask(struct rtl8xxxu_priv *priv,
- u32 ramask, int sgi)
-{
- struct h2c_cmd h2c;
- u8 bw = 0;
-
- memset(&h2c, 0, sizeof(struct h2c_cmd));
-
- h2c.b_macid_cfg.cmd = H2C_8723B_MACID_CFG_RAID;
- h2c.b_macid_cfg.ramask0 = ramask & 0xff;
- h2c.b_macid_cfg.ramask1 = (ramask >> 8) & 0xff;
- h2c.b_macid_cfg.ramask2 = (ramask >> 16) & 0xff;
- h2c.b_macid_cfg.ramask3 = (ramask >> 24) & 0xff;
-
- h2c.ramask.arg = 0x80;
- h2c.b_macid_cfg.data1 = 0;
- if (sgi)
- h2c.b_macid_cfg.data1 |= BIT(7);
-
- h2c.b_macid_cfg.data2 = bw;
-
- dev_dbg(&priv->udev->dev, "%s: rate mask %08x, arg %02x, size %zi\n",
- __func__, ramask, h2c.ramask.arg, sizeof(h2c.b_macid_cfg));
- rtl8723a_h2c_cmd(priv, &h2c, sizeof(h2c.b_macid_cfg));
-}
-
-static void rtl8xxxu_gen1_report_connect(struct rtl8xxxu_priv *priv,
- u8 macid, bool connect)
-{
- struct h2c_cmd h2c;
-
- memset(&h2c, 0, sizeof(struct h2c_cmd));
-
- h2c.joinbss.cmd = H2C_JOIN_BSS_REPORT;
-
- if (connect)
- h2c.joinbss.data = H2C_JOIN_BSS_CONNECT;
- else
- h2c.joinbss.data = H2C_JOIN_BSS_DISCONNECT;
-
- rtl8723a_h2c_cmd(priv, &h2c, sizeof(h2c.joinbss));
-}
-
-static void rtl8xxxu_gen2_report_connect(struct rtl8xxxu_priv *priv,
- u8 macid, bool connect)
-{
- struct h2c_cmd h2c;
-
- memset(&h2c, 0, sizeof(struct h2c_cmd));
-
- h2c.media_status_rpt.cmd = H2C_8723B_MEDIA_STATUS_RPT;
- if (connect)
- h2c.media_status_rpt.parm |= BIT(0);
- else
- h2c.media_status_rpt.parm &= ~BIT(0);
-
- rtl8723a_h2c_cmd(priv, &h2c, sizeof(h2c.media_status_rpt));
-}
-
-static void rtl8xxxu_set_basic_rates(struct rtl8xxxu_priv *priv, u32 rate_cfg)
-{
- u32 val32;
- u8 rate_idx = 0;
-
- rate_cfg &= RESPONSE_RATE_BITMAP_ALL;
-
- val32 = rtl8xxxu_read32(priv, REG_RESPONSE_RATE_SET);
- val32 &= ~RESPONSE_RATE_BITMAP_ALL;
- val32 |= rate_cfg;
- rtl8xxxu_write32(priv, REG_RESPONSE_RATE_SET, val32);
-
- dev_dbg(&priv->udev->dev, "%s: rates %08x\n", __func__, rate_cfg);
-
- while (rate_cfg) {
- rate_cfg = (rate_cfg >> 1);
- rate_idx++;
- }
- rtl8xxxu_write8(priv, REG_INIRTS_RATE_SEL, rate_idx);
-}
-
-static void
-rtl8xxxu_bss_info_changed(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
- struct ieee80211_bss_conf *bss_conf, u32 changed)
-{
- struct rtl8xxxu_priv *priv = hw->priv;
- struct device *dev = &priv->udev->dev;
- struct ieee80211_sta *sta;
- u32 val32;
- u8 val8;
-
- if (changed & BSS_CHANGED_ASSOC) {
- dev_dbg(dev, "Changed ASSOC: %i!\n", bss_conf->assoc);
-
- rtl8xxxu_set_linktype(priv, vif->type);
-
- if (bss_conf->assoc) {
- u32 ramask;
- int sgi = 0;
-
- rcu_read_lock();
- sta = ieee80211_find_sta(vif, bss_conf->bssid);
- if (!sta) {
- dev_info(dev, "%s: ASSOC no sta found\n",
- __func__);
- rcu_read_unlock();
- goto error;
- }
-
- if (sta->ht_cap.ht_supported)
- dev_info(dev, "%s: HT supported\n", __func__);
- if (sta->vht_cap.vht_supported)
- dev_info(dev, "%s: VHT supported\n", __func__);
-
- /* TODO: Set bits 28-31 for rate adaptive id */
- ramask = (sta->supp_rates[0] & 0xfff) |
- sta->ht_cap.mcs.rx_mask[0] << 12 |
- sta->ht_cap.mcs.rx_mask[1] << 20;
- if (sta->ht_cap.cap &
- (IEEE80211_HT_CAP_SGI_40 | IEEE80211_HT_CAP_SGI_20))
- sgi = 1;
- rcu_read_unlock();
-
- priv->fops->update_rate_mask(priv, ramask, sgi);
-
- rtl8xxxu_write8(priv, REG_BCN_MAX_ERR, 0xff);
-
- rtl8723a_stop_tx_beacon(priv);
-
- /* joinbss sequence */
- rtl8xxxu_write16(priv, REG_BCN_PSR_RPT,
- 0xc000 | bss_conf->aid);
-
- priv->fops->report_connect(priv, 0, true);
- } else {
- val8 = rtl8xxxu_read8(priv, REG_BEACON_CTRL);
- val8 |= BEACON_DISABLE_TSF_UPDATE;
- rtl8xxxu_write8(priv, REG_BEACON_CTRL, val8);
-
- priv->fops->report_connect(priv, 0, false);
- }
- }
-
- if (changed & BSS_CHANGED_ERP_PREAMBLE) {
- dev_dbg(dev, "Changed ERP_PREAMBLE: Use short preamble %i\n",
- bss_conf->use_short_preamble);
- val32 = rtl8xxxu_read32(priv, REG_RESPONSE_RATE_SET);
- if (bss_conf->use_short_preamble)
- val32 |= RSR_ACK_SHORT_PREAMBLE;
- else
- val32 &= ~RSR_ACK_SHORT_PREAMBLE;
- rtl8xxxu_write32(priv, REG_RESPONSE_RATE_SET, val32);
- }
-
- if (changed & BSS_CHANGED_ERP_SLOT) {
- dev_dbg(dev, "Changed ERP_SLOT: short_slot_time %i\n",
- bss_conf->use_short_slot);
-
- if (bss_conf->use_short_slot)
- val8 = 9;
- else
- val8 = 20;
- rtl8xxxu_write8(priv, REG_SLOT, val8);
- }
-
- if (changed & BSS_CHANGED_BSSID) {
- dev_dbg(dev, "Changed BSSID!\n");
- rtl8xxxu_set_bssid(priv, bss_conf->bssid);
- }
-
- if (changed & BSS_CHANGED_BASIC_RATES) {
- dev_dbg(dev, "Changed BASIC_RATES!\n");
- rtl8xxxu_set_basic_rates(priv, bss_conf->basic_rates);
- }
-error:
- return;
-}
-
-static u32 rtl8xxxu_80211_to_rtl_queue(u32 queue)
-{
- u32 rtlqueue;
-
- switch (queue) {
- case IEEE80211_AC_VO:
- rtlqueue = TXDESC_QUEUE_VO;
- break;
- case IEEE80211_AC_VI:
- rtlqueue = TXDESC_QUEUE_VI;
- break;
- case IEEE80211_AC_BE:
- rtlqueue = TXDESC_QUEUE_BE;
- break;
- case IEEE80211_AC_BK:
- rtlqueue = TXDESC_QUEUE_BK;
- break;
- default:
- rtlqueue = TXDESC_QUEUE_BE;
- }
-
- return rtlqueue;
-}
-
-static u32 rtl8xxxu_queue_select(struct ieee80211_hw *hw, struct sk_buff *skb)
-{
- struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
- u32 queue;
-
- if (ieee80211_is_mgmt(hdr->frame_control))
- queue = TXDESC_QUEUE_MGNT;
- else
- queue = rtl8xxxu_80211_to_rtl_queue(skb_get_queue_mapping(skb));
-
- return queue;
-}
-
-/*
- * Despite newer chips 8723b/8812/8821 having a larger TX descriptor
- * format. The descriptor checksum is still only calculated over the
- * initial 32 bytes of the descriptor!
- */
-static void rtl8xxxu_calc_tx_desc_csum(struct rtl8xxxu_txdesc32 *tx_desc)
-{
- __le16 *ptr = (__le16 *)tx_desc;
- u16 csum = 0;
- int i;
-
- /*
- * Clear csum field before calculation, as the csum field is
- * in the middle of the struct.
- */
- tx_desc->csum = cpu_to_le16(0);
-
- for (i = 0; i < (sizeof(struct rtl8xxxu_txdesc32) / sizeof(u16)); i++)
- csum = csum ^ le16_to_cpu(ptr[i]);
-
- tx_desc->csum |= cpu_to_le16(csum);
-}
-
-static void rtl8xxxu_free_tx_resources(struct rtl8xxxu_priv *priv)
-{
- struct rtl8xxxu_tx_urb *tx_urb, *tmp;
- unsigned long flags;
-
- spin_lock_irqsave(&priv->tx_urb_lock, flags);
- list_for_each_entry_safe(tx_urb, tmp, &priv->tx_urb_free_list, list) {
- list_del(&tx_urb->list);
- priv->tx_urb_free_count--;
- usb_free_urb(&tx_urb->urb);
- }
- spin_unlock_irqrestore(&priv->tx_urb_lock, flags);
-}
-
-static struct rtl8xxxu_tx_urb *
-rtl8xxxu_alloc_tx_urb(struct rtl8xxxu_priv *priv)
-{
- struct rtl8xxxu_tx_urb *tx_urb;
- unsigned long flags;
-
- spin_lock_irqsave(&priv->tx_urb_lock, flags);
- tx_urb = list_first_entry_or_null(&priv->tx_urb_free_list,
- struct rtl8xxxu_tx_urb, list);
- if (tx_urb) {
- list_del(&tx_urb->list);
- priv->tx_urb_free_count--;
- if (priv->tx_urb_free_count < RTL8XXXU_TX_URB_LOW_WATER &&
- !priv->tx_stopped) {
- priv->tx_stopped = true;
- ieee80211_stop_queues(priv->hw);
- }
- }
-
- spin_unlock_irqrestore(&priv->tx_urb_lock, flags);
-
- return tx_urb;
-}
-
-static void rtl8xxxu_free_tx_urb(struct rtl8xxxu_priv *priv,
- struct rtl8xxxu_tx_urb *tx_urb)
-{
- unsigned long flags;
-
- INIT_LIST_HEAD(&tx_urb->list);
-
- spin_lock_irqsave(&priv->tx_urb_lock, flags);
-
- list_add(&tx_urb->list, &priv->tx_urb_free_list);
- priv->tx_urb_free_count++;
- if (priv->tx_urb_free_count > RTL8XXXU_TX_URB_HIGH_WATER &&
- priv->tx_stopped) {
- priv->tx_stopped = false;
- ieee80211_wake_queues(priv->hw);
- }
-
- spin_unlock_irqrestore(&priv->tx_urb_lock, flags);
-}
-
-static void rtl8xxxu_tx_complete(struct urb *urb)
-{
- struct sk_buff *skb = (struct sk_buff *)urb->context;
- struct ieee80211_tx_info *tx_info;
- struct ieee80211_hw *hw;
- struct rtl8xxxu_priv *priv;
- struct rtl8xxxu_tx_urb *tx_urb =
- container_of(urb, struct rtl8xxxu_tx_urb, urb);
-
- tx_info = IEEE80211_SKB_CB(skb);
- hw = tx_info->rate_driver_data[0];
- priv = hw->priv;
-
- skb_pull(skb, priv->fops->tx_desc_size);
-
- ieee80211_tx_info_clear_status(tx_info);
- tx_info->status.rates[0].idx = -1;
- tx_info->status.rates[0].count = 0;
-
- if (!urb->status)
- tx_info->flags |= IEEE80211_TX_STAT_ACK;
-
- ieee80211_tx_status_irqsafe(hw, skb);
-
- rtl8xxxu_free_tx_urb(priv, tx_urb);
-}
-
-static void rtl8xxxu_dump_action(struct device *dev,
- struct ieee80211_hdr *hdr)
-{
- struct ieee80211_mgmt *mgmt = (struct ieee80211_mgmt *)hdr;
- u16 cap, timeout;
-
- if (!(rtl8xxxu_debug & RTL8XXXU_DEBUG_ACTION))
- return;
-
- switch (mgmt->u.action.u.addba_resp.action_code) {
- case WLAN_ACTION_ADDBA_RESP:
- cap = le16_to_cpu(mgmt->u.action.u.addba_resp.capab);
- timeout = le16_to_cpu(mgmt->u.action.u.addba_resp.timeout);
- dev_info(dev, "WLAN_ACTION_ADDBA_RESP: "
- "timeout %i, tid %02x, buf_size %02x, policy %02x, "
- "status %02x\n",
- timeout,
- (cap & IEEE80211_ADDBA_PARAM_TID_MASK) >> 2,
- (cap & IEEE80211_ADDBA_PARAM_BUF_SIZE_MASK) >> 6,
- (cap >> 1) & 0x1,
- le16_to_cpu(mgmt->u.action.u.addba_resp.status));
- break;
- case WLAN_ACTION_ADDBA_REQ:
- cap = le16_to_cpu(mgmt->u.action.u.addba_req.capab);
- timeout = le16_to_cpu(mgmt->u.action.u.addba_req.timeout);
- dev_info(dev, "WLAN_ACTION_ADDBA_REQ: "
- "timeout %i, tid %02x, buf_size %02x, policy %02x\n",
- timeout,
- (cap & IEEE80211_ADDBA_PARAM_TID_MASK) >> 2,
- (cap & IEEE80211_ADDBA_PARAM_BUF_SIZE_MASK) >> 6,
- (cap >> 1) & 0x1);
- break;
- default:
- dev_info(dev, "action frame %02x\n",
- mgmt->u.action.u.addba_resp.action_code);
- break;
- }
-}
-
-static void rtl8xxxu_tx(struct ieee80211_hw *hw,
- struct ieee80211_tx_control *control,
- struct sk_buff *skb)
-{
- struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
- struct ieee80211_tx_info *tx_info = IEEE80211_SKB_CB(skb);
- struct ieee80211_rate *tx_rate = ieee80211_get_tx_rate(hw, tx_info);
- struct rtl8xxxu_priv *priv = hw->priv;
- struct rtl8xxxu_txdesc32 *tx_desc;
- struct rtl8xxxu_txdesc40 *tx_desc40;
- struct rtl8xxxu_tx_urb *tx_urb;
- struct ieee80211_sta *sta = NULL;
- struct ieee80211_vif *vif = tx_info->control.vif;
- struct device *dev = &priv->udev->dev;
- u32 queue, rate;
- u16 pktlen = skb->len;
- u16 seq_number;
- u16 rate_flag = tx_info->control.rates[0].flags;
- int tx_desc_size = priv->fops->tx_desc_size;
- int ret;
- bool usedesc40, ampdu_enable;
-
- if (skb_headroom(skb) < tx_desc_size) {
- dev_warn(dev,
- "%s: Not enough headroom (%i) for tx descriptor\n",
- __func__, skb_headroom(skb));
- goto error;
- }
-
- if (unlikely(skb->len > (65535 - tx_desc_size))) {
- dev_warn(dev, "%s: Trying to send over-sized skb (%i)\n",
- __func__, skb->len);
- goto error;
- }
-
- tx_urb = rtl8xxxu_alloc_tx_urb(priv);
- if (!tx_urb) {
- dev_warn(dev, "%s: Unable to allocate tx urb\n", __func__);
- goto error;
- }
-
- if (rtl8xxxu_debug & RTL8XXXU_DEBUG_TX)
- dev_info(dev, "%s: TX rate: %d (%d), pkt size %d\n",
- __func__, tx_rate->bitrate, tx_rate->hw_value, pktlen);
-
- if (ieee80211_is_action(hdr->frame_control))
- rtl8xxxu_dump_action(dev, hdr);
-
- usedesc40 = (tx_desc_size == 40);
- tx_info->rate_driver_data[0] = hw;
-
- if (control && control->sta)
- sta = control->sta;
-
- tx_desc = (struct rtl8xxxu_txdesc32 *)skb_push(skb, tx_desc_size);
-
- memset(tx_desc, 0, tx_desc_size);
- tx_desc->pkt_size = cpu_to_le16(pktlen);
- tx_desc->pkt_offset = tx_desc_size;
-
- tx_desc->txdw0 =
- TXDESC_OWN | TXDESC_FIRST_SEGMENT | TXDESC_LAST_SEGMENT;
- if (is_multicast_ether_addr(ieee80211_get_DA(hdr)) ||
- is_broadcast_ether_addr(ieee80211_get_DA(hdr)))
- tx_desc->txdw0 |= TXDESC_BROADMULTICAST;
-
- queue = rtl8xxxu_queue_select(hw, skb);
- tx_desc->txdw1 = cpu_to_le32(queue << TXDESC_QUEUE_SHIFT);
-
- if (tx_info->control.hw_key) {
- switch (tx_info->control.hw_key->cipher) {
- case WLAN_CIPHER_SUITE_WEP40:
- case WLAN_CIPHER_SUITE_WEP104:
- case WLAN_CIPHER_SUITE_TKIP:
- tx_desc->txdw1 |= cpu_to_le32(TXDESC_SEC_RC4);
- break;
- case WLAN_CIPHER_SUITE_CCMP:
- tx_desc->txdw1 |= cpu_to_le32(TXDESC_SEC_AES);
- break;
- default:
- break;
- }
- }
-
- /* (tx_info->flags & IEEE80211_TX_CTL_AMPDU) && */
- ampdu_enable = false;
- if (ieee80211_is_data_qos(hdr->frame_control) && sta) {
- if (sta->ht_cap.ht_supported) {
- u32 ampdu, val32;
-
- ampdu = (u32)sta->ht_cap.ampdu_density;
- val32 = ampdu << TXDESC_AMPDU_DENSITY_SHIFT;
- tx_desc->txdw2 |= cpu_to_le32(val32);
-
- ampdu_enable = true;
- }
- }
-
- if (rate_flag & IEEE80211_TX_RC_MCS)
- rate = tx_info->control.rates[0].idx + DESC_RATE_MCS0;
- else
- rate = tx_rate->hw_value;
-
- seq_number = IEEE80211_SEQ_TO_SN(le16_to_cpu(hdr->seq_ctrl));
- if (!usedesc40) {
- tx_desc->txdw5 = cpu_to_le32(rate);
-
- if (ieee80211_is_data(hdr->frame_control))
- tx_desc->txdw5 |= cpu_to_le32(0x0001ff00);
-
- tx_desc->txdw3 =
- cpu_to_le32((u32)seq_number << TXDESC32_SEQ_SHIFT);
-
- if (ampdu_enable)
- tx_desc->txdw1 |= cpu_to_le32(TXDESC32_AGG_ENABLE);
- else
- tx_desc->txdw1 |= cpu_to_le32(TXDESC32_AGG_BREAK);
-
- if (ieee80211_is_mgmt(hdr->frame_control)) {
- tx_desc->txdw5 = cpu_to_le32(tx_rate->hw_value);
- tx_desc->txdw4 |=
- cpu_to_le32(TXDESC32_USE_DRIVER_RATE);
- tx_desc->txdw5 |=
- cpu_to_le32(6 << TXDESC32_RETRY_LIMIT_SHIFT);
- tx_desc->txdw5 |=
- cpu_to_le32(TXDESC32_RETRY_LIMIT_ENABLE);
- }
-
- if (ieee80211_is_data_qos(hdr->frame_control))
- tx_desc->txdw4 |= cpu_to_le32(TXDESC32_QOS);
-
- if (rate_flag & IEEE80211_TX_RC_USE_SHORT_PREAMBLE ||
- (sta && vif && vif->bss_conf.use_short_preamble))
- tx_desc->txdw4 |= cpu_to_le32(TXDESC32_SHORT_PREAMBLE);
-
- if (rate_flag & IEEE80211_TX_RC_SHORT_GI ||
- (ieee80211_is_data_qos(hdr->frame_control) &&
- sta && sta->ht_cap.cap &
- (IEEE80211_HT_CAP_SGI_40 | IEEE80211_HT_CAP_SGI_20))) {
- tx_desc->txdw5 |= cpu_to_le32(TXDESC32_SHORT_GI);
- }
-
- if (rate_flag & IEEE80211_TX_RC_USE_RTS_CTS) {
- /*
- * Use RTS rate 24M - does the mac80211 tell
- * us which to use?
- */
- tx_desc->txdw4 |=
- cpu_to_le32(DESC_RATE_24M <<
- TXDESC32_RTS_RATE_SHIFT);
- tx_desc->txdw4 |=
- cpu_to_le32(TXDESC32_RTS_CTS_ENABLE);
- tx_desc->txdw4 |= cpu_to_le32(TXDESC32_HW_RTS_ENABLE);
- }
- } else {
- tx_desc40 = (struct rtl8xxxu_txdesc40 *)tx_desc;
-
- tx_desc40->txdw4 = cpu_to_le32(rate);
- if (ieee80211_is_data(hdr->frame_control)) {
- tx_desc->txdw4 |=
- cpu_to_le32(0x1f <<
- TXDESC40_DATA_RATE_FB_SHIFT);
- }
-
- tx_desc40->txdw9 =
- cpu_to_le32((u32)seq_number << TXDESC40_SEQ_SHIFT);
-
- if (ampdu_enable)
- tx_desc40->txdw2 |= cpu_to_le32(TXDESC40_AGG_ENABLE);
- else
- tx_desc40->txdw2 |= cpu_to_le32(TXDESC40_AGG_BREAK);
-
- if (ieee80211_is_mgmt(hdr->frame_control)) {
- tx_desc40->txdw4 = cpu_to_le32(tx_rate->hw_value);
- tx_desc40->txdw3 |=
- cpu_to_le32(TXDESC40_USE_DRIVER_RATE);
- tx_desc40->txdw4 |=
- cpu_to_le32(6 << TXDESC40_RETRY_LIMIT_SHIFT);
- tx_desc40->txdw4 |=
- cpu_to_le32(TXDESC40_RETRY_LIMIT_ENABLE);
- }
-
- if (rate_flag & IEEE80211_TX_RC_USE_SHORT_PREAMBLE ||
- (sta && vif && vif->bss_conf.use_short_preamble))
- tx_desc40->txdw5 |=
- cpu_to_le32(TXDESC40_SHORT_PREAMBLE);
-
- if (rate_flag & IEEE80211_TX_RC_USE_RTS_CTS) {
- /*
- * Use RTS rate 24M - does the mac80211 tell
- * us which to use?
- */
- tx_desc->txdw4 |=
- cpu_to_le32(DESC_RATE_24M <<
- TXDESC40_RTS_RATE_SHIFT);
- tx_desc->txdw3 |= cpu_to_le32(TXDESC40_RTS_CTS_ENABLE);
- tx_desc->txdw3 |= cpu_to_le32(TXDESC40_HW_RTS_ENABLE);
- }
- }
-
- rtl8xxxu_calc_tx_desc_csum(tx_desc);
-
- usb_fill_bulk_urb(&tx_urb->urb, priv->udev, priv->pipe_out[queue],
- skb->data, skb->len, rtl8xxxu_tx_complete, skb);
-
- usb_anchor_urb(&tx_urb->urb, &priv->tx_anchor);
- ret = usb_submit_urb(&tx_urb->urb, GFP_ATOMIC);
- if (ret) {
- usb_unanchor_urb(&tx_urb->urb);
- rtl8xxxu_free_tx_urb(priv, tx_urb);
- goto error;
- }
- return;
-error:
- dev_kfree_skb(skb);
-}
-
-static void rtl8xxxu_rx_parse_phystats(struct rtl8xxxu_priv *priv,
- struct ieee80211_rx_status *rx_status,
- struct rtl8723au_phy_stats *phy_stats,
- u32 rxmcs)
-{
- if (phy_stats->sgi_en)
- rx_status->flag |= RX_FLAG_SHORT_GI;
-
- if (rxmcs < DESC_RATE_6M) {
- /*
- * Handle PHY stats for CCK rates
- */
- u8 cck_agc_rpt = phy_stats->cck_agc_rpt_ofdm_cfosho_a;
-
- switch (cck_agc_rpt & 0xc0) {
- case 0xc0:
- rx_status->signal = -46 - (cck_agc_rpt & 0x3e);
- break;
- case 0x80:
- rx_status->signal = -26 - (cck_agc_rpt & 0x3e);
- break;
- case 0x40:
- rx_status->signal = -12 - (cck_agc_rpt & 0x3e);
- break;
- case 0x00:
- rx_status->signal = 16 - (cck_agc_rpt & 0x3e);
- break;
- }
- } else {
- rx_status->signal =
- (phy_stats->cck_sig_qual_ofdm_pwdb_all >> 1) - 110;
- }
-}
-
-static void rtl8xxxu_free_rx_resources(struct rtl8xxxu_priv *priv)
-{
- struct rtl8xxxu_rx_urb *rx_urb, *tmp;
- unsigned long flags;
-
- spin_lock_irqsave(&priv->rx_urb_lock, flags);
-
- list_for_each_entry_safe(rx_urb, tmp,
- &priv->rx_urb_pending_list, list) {
- list_del(&rx_urb->list);
- priv->rx_urb_pending_count--;
- usb_free_urb(&rx_urb->urb);
- }
-
- spin_unlock_irqrestore(&priv->rx_urb_lock, flags);
-}
-
-static void rtl8xxxu_queue_rx_urb(struct rtl8xxxu_priv *priv,
- struct rtl8xxxu_rx_urb *rx_urb)
-{
- struct sk_buff *skb;
- unsigned long flags;
- int pending = 0;
-
- spin_lock_irqsave(&priv->rx_urb_lock, flags);
-
- if (!priv->shutdown) {
- list_add_tail(&rx_urb->list, &priv->rx_urb_pending_list);
- priv->rx_urb_pending_count++;
- pending = priv->rx_urb_pending_count;
- } else {
- skb = (struct sk_buff *)rx_urb->urb.context;
- dev_kfree_skb(skb);
- usb_free_urb(&rx_urb->urb);
- }
-
- spin_unlock_irqrestore(&priv->rx_urb_lock, flags);
-
- if (pending > RTL8XXXU_RX_URB_PENDING_WATER)
- schedule_work(&priv->rx_urb_wq);
-}
-
-static void rtl8xxxu_rx_urb_work(struct work_struct *work)
-{
- struct rtl8xxxu_priv *priv;
- struct rtl8xxxu_rx_urb *rx_urb, *tmp;
- struct list_head local;
- struct sk_buff *skb;
- unsigned long flags;
- int ret;
-
- priv = container_of(work, struct rtl8xxxu_priv, rx_urb_wq);
- INIT_LIST_HEAD(&local);
-
- spin_lock_irqsave(&priv->rx_urb_lock, flags);
-
- list_splice_init(&priv->rx_urb_pending_list, &local);
- priv->rx_urb_pending_count = 0;
-
- spin_unlock_irqrestore(&priv->rx_urb_lock, flags);
-
- list_for_each_entry_safe(rx_urb, tmp, &local, list) {
- list_del_init(&rx_urb->list);
- ret = rtl8xxxu_submit_rx_urb(priv, rx_urb);
- /*
- * If out of memory or temporary error, put it back on the
- * queue and try again. Otherwise the device is dead/gone
- * and we should drop it.
- */
- switch (ret) {
- case 0:
- break;
- case -ENOMEM:
- case -EAGAIN:
- rtl8xxxu_queue_rx_urb(priv, rx_urb);
- break;
- default:
- pr_info("failed to requeue urb %i\n", ret);
- skb = (struct sk_buff *)rx_urb->urb.context;
- dev_kfree_skb(skb);
- usb_free_urb(&rx_urb->urb);
- }
- }
-}
-
-static int rtl8xxxu_parse_rxdesc16(struct rtl8xxxu_priv *priv,
- struct sk_buff *skb,
- struct ieee80211_rx_status *rx_status)
-{
- struct rtl8xxxu_rxdesc16 *rx_desc =
- (struct rtl8xxxu_rxdesc16 *)skb->data;
- struct rtl8723au_phy_stats *phy_stats;
- __le32 *_rx_desc_le = (__le32 *)skb->data;
- u32 *_rx_desc = (u32 *)skb->data;
- int drvinfo_sz, desc_shift;
- int i;
-
- for (i = 0; i < (sizeof(struct rtl8xxxu_rxdesc16) / sizeof(u32)); i++)
- _rx_desc[i] = le32_to_cpu(_rx_desc_le[i]);
-
- skb_pull(skb, sizeof(struct rtl8xxxu_rxdesc16));
-
- phy_stats = (struct rtl8723au_phy_stats *)skb->data;
-
- drvinfo_sz = rx_desc->drvinfo_sz * 8;
- desc_shift = rx_desc->shift;
- skb_pull(skb, drvinfo_sz + desc_shift);
-
- if (rx_desc->phy_stats)
- rtl8xxxu_rx_parse_phystats(priv, rx_status, phy_stats,
- rx_desc->rxmcs);
-
- rx_status->mactime = le32_to_cpu(rx_desc->tsfl);
- rx_status->flag |= RX_FLAG_MACTIME_START;
-
- if (!rx_desc->swdec)
- rx_status->flag |= RX_FLAG_DECRYPTED;
- if (rx_desc->crc32)
- rx_status->flag |= RX_FLAG_FAILED_FCS_CRC;
- if (rx_desc->bw)
- rx_status->flag |= RX_FLAG_40MHZ;
-
- if (rx_desc->rxht) {
- rx_status->flag |= RX_FLAG_HT;
- rx_status->rate_idx = rx_desc->rxmcs - DESC_RATE_MCS0;
- } else {
- rx_status->rate_idx = rx_desc->rxmcs;
- }
-
- return RX_TYPE_DATA_PKT;
-}
-
-static int rtl8xxxu_parse_rxdesc24(struct rtl8xxxu_priv *priv,
- struct sk_buff *skb,
- struct ieee80211_rx_status *rx_status)
-{
- struct rtl8xxxu_rxdesc24 *rx_desc =
- (struct rtl8xxxu_rxdesc24 *)skb->data;
- struct rtl8723au_phy_stats *phy_stats;
- __le32 *_rx_desc_le = (__le32 *)skb->data;
- u32 *_rx_desc = (u32 *)skb->data;
- int drvinfo_sz, desc_shift;
- int i;
-
- for (i = 0; i < (sizeof(struct rtl8xxxu_rxdesc24) / sizeof(u32)); i++)
- _rx_desc[i] = le32_to_cpu(_rx_desc_le[i]);
-
- skb_pull(skb, sizeof(struct rtl8xxxu_rxdesc24));
-
- phy_stats = (struct rtl8723au_phy_stats *)skb->data;
-
- drvinfo_sz = rx_desc->drvinfo_sz * 8;
- desc_shift = rx_desc->shift;
- skb_pull(skb, drvinfo_sz + desc_shift);
-
- if (rx_desc->rpt_sel) {
- struct device *dev = &priv->udev->dev;
- dev_dbg(dev, "%s: C2H packet\n", __func__);
- return RX_TYPE_C2H;
- }
-
- if (rx_desc->phy_stats)
- rtl8xxxu_rx_parse_phystats(priv, rx_status, phy_stats,
- rx_desc->rxmcs);
-
- rx_status->mactime = le32_to_cpu(rx_desc->tsfl);
- rx_status->flag |= RX_FLAG_MACTIME_START;
-
- if (!rx_desc->swdec)
- rx_status->flag |= RX_FLAG_DECRYPTED;
- if (rx_desc->crc32)
- rx_status->flag |= RX_FLAG_FAILED_FCS_CRC;
- if (rx_desc->bw)
- rx_status->flag |= RX_FLAG_40MHZ;
-
- if (rx_desc->rxmcs >= DESC_RATE_MCS0) {
- rx_status->flag |= RX_FLAG_HT;
- rx_status->rate_idx = rx_desc->rxmcs - DESC_RATE_MCS0;
- } else {
- rx_status->rate_idx = rx_desc->rxmcs;
- }
-
- return RX_TYPE_DATA_PKT;
-}
-
-static void rtl8723bu_handle_c2h(struct rtl8xxxu_priv *priv,
- struct sk_buff *skb)
-{
- struct rtl8723bu_c2h *c2h = (struct rtl8723bu_c2h *)skb->data;
- struct device *dev = &priv->udev->dev;
- int len;
-
- len = skb->len - 2;
-
- dev_dbg(dev, "C2H ID %02x seq %02x, len %02x source %02x\n",
- c2h->id, c2h->seq, len, c2h->bt_info.response_source);
-
- switch(c2h->id) {
- case C2H_8723B_BT_INFO:
- if (c2h->bt_info.response_source >
- BT_INFO_SRC_8723B_BT_ACTIVE_SEND)
- dev_dbg(dev, "C2H_BT_INFO WiFi only firmware\n");
- else
- dev_dbg(dev, "C2H_BT_INFO BT/WiFi coexist firmware\n");
-
- if (c2h->bt_info.bt_has_reset)
- dev_dbg(dev, "BT has been reset\n");
- if (c2h->bt_info.tx_rx_mask)
- dev_dbg(dev, "BT TRx mask\n");
-
- break;
- case C2H_8723B_BT_MP_INFO:
- dev_dbg(dev, "C2H_MP_INFO ext ID %02x, status %02x\n",
- c2h->bt_mp_info.ext_id, c2h->bt_mp_info.status);
- break;
- case C2H_8723B_RA_REPORT:
- dev_dbg(dev,
- "C2H RA RPT: rate %02x, unk %i, macid %02x, noise %i\n",
- c2h->ra_report.rate, c2h->ra_report.dummy0_0,
- c2h->ra_report.macid, c2h->ra_report.noisy_state);
- break;
- default:
- dev_info(dev, "Unhandled C2H event %02x seq %02x\n",
- c2h->id, c2h->seq);
- print_hex_dump(KERN_INFO, "C2H content: ", DUMP_PREFIX_NONE,
- 16, 1, c2h->raw.payload, len, false);
- break;
- }
-}
-
-static void rtl8xxxu_rx_complete(struct urb *urb)
-{
- struct rtl8xxxu_rx_urb *rx_urb =
- container_of(urb, struct rtl8xxxu_rx_urb, urb);
- struct ieee80211_hw *hw = rx_urb->hw;
- struct rtl8xxxu_priv *priv = hw->priv;
- struct sk_buff *skb = (struct sk_buff *)urb->context;
- struct ieee80211_rx_status *rx_status = IEEE80211_SKB_RXCB(skb);
- struct device *dev = &priv->udev->dev;
- int rx_type;
-
- skb_put(skb, urb->actual_length);
-
- if (urb->status == 0) {
- memset(rx_status, 0, sizeof(struct ieee80211_rx_status));
-
- rx_type = priv->fops->parse_rx_desc(priv, skb, rx_status);
-
- rx_status->freq = hw->conf.chandef.chan->center_freq;
- rx_status->band = hw->conf.chandef.chan->band;
-
- if (rx_type == RX_TYPE_DATA_PKT)
- ieee80211_rx_irqsafe(hw, skb);
- else {
- rtl8723bu_handle_c2h(priv, skb);
- dev_kfree_skb(skb);
- }
-
- skb = NULL;
- rx_urb->urb.context = NULL;
- rtl8xxxu_queue_rx_urb(priv, rx_urb);
- } else {
- dev_dbg(dev, "%s: status %i\n", __func__, urb->status);
- goto cleanup;
- }
- return;
-
-cleanup:
- usb_free_urb(urb);
- dev_kfree_skb(skb);
- return;
-}
-
-static int rtl8xxxu_submit_rx_urb(struct rtl8xxxu_priv *priv,
- struct rtl8xxxu_rx_urb *rx_urb)
-{
- struct sk_buff *skb;
- int skb_size;
- int ret, rx_desc_sz;
-
- rx_desc_sz = priv->fops->rx_desc_size;
- skb_size = rx_desc_sz + RTL_RX_BUFFER_SIZE;
- skb = __netdev_alloc_skb(NULL, skb_size, GFP_KERNEL);
- if (!skb)
- return -ENOMEM;
-
- memset(skb->data, 0, rx_desc_sz);
- usb_fill_bulk_urb(&rx_urb->urb, priv->udev, priv->pipe_in, skb->data,
- skb_size, rtl8xxxu_rx_complete, skb);
- usb_anchor_urb(&rx_urb->urb, &priv->rx_anchor);
- ret = usb_submit_urb(&rx_urb->urb, GFP_ATOMIC);
- if (ret)
- usb_unanchor_urb(&rx_urb->urb);
- return ret;
-}
-
-static void rtl8xxxu_int_complete(struct urb *urb)
-{
- struct rtl8xxxu_priv *priv = (struct rtl8xxxu_priv *)urb->context;
- struct device *dev = &priv->udev->dev;
- int ret;
-
- dev_dbg(dev, "%s: status %i\n", __func__, urb->status);
- if (urb->status == 0) {
- usb_anchor_urb(urb, &priv->int_anchor);
- ret = usb_submit_urb(urb, GFP_ATOMIC);
- if (ret)
- usb_unanchor_urb(urb);
- } else {
- dev_info(dev, "%s: Error %i\n", __func__, urb->status);
- }
-}
-
-
-static int rtl8xxxu_submit_int_urb(struct ieee80211_hw *hw)
-{
- struct rtl8xxxu_priv *priv = hw->priv;
- struct urb *urb;
- u32 val32;
- int ret;
-
- urb = usb_alloc_urb(0, GFP_KERNEL);
- if (!urb)
- return -ENOMEM;
-
- usb_fill_int_urb(urb, priv->udev, priv->pipe_interrupt,
- priv->int_buf, USB_INTR_CONTENT_LENGTH,
- rtl8xxxu_int_complete, priv, 1);
- usb_anchor_urb(urb, &priv->int_anchor);
- ret = usb_submit_urb(urb, GFP_KERNEL);
- if (ret) {
- usb_unanchor_urb(urb);
- goto error;
- }
-
- val32 = rtl8xxxu_read32(priv, REG_USB_HIMR);
- val32 |= USB_HIMR_CPWM;
- rtl8xxxu_write32(priv, REG_USB_HIMR, val32);
-
-error:
- return ret;
-}
-
-static int rtl8xxxu_add_interface(struct ieee80211_hw *hw,
- struct ieee80211_vif *vif)
-{
- struct rtl8xxxu_priv *priv = hw->priv;
- int ret;
- u8 val8;
-
- switch (vif->type) {
- case NL80211_IFTYPE_STATION:
- rtl8723a_stop_tx_beacon(priv);
-
- val8 = rtl8xxxu_read8(priv, REG_BEACON_CTRL);
- val8 |= BEACON_ATIM | BEACON_FUNCTION_ENABLE |
- BEACON_DISABLE_TSF_UPDATE;
- rtl8xxxu_write8(priv, REG_BEACON_CTRL, val8);
- ret = 0;
- break;
- default:
- ret = -EOPNOTSUPP;
- }
-
- rtl8xxxu_set_linktype(priv, vif->type);
-
- return ret;
-}
-
-static void rtl8xxxu_remove_interface(struct ieee80211_hw *hw,
- struct ieee80211_vif *vif)
-{
- struct rtl8xxxu_priv *priv = hw->priv;
-
- dev_dbg(&priv->udev->dev, "%s\n", __func__);
-}
-
-static int rtl8xxxu_config(struct ieee80211_hw *hw, u32 changed)
-{
- struct rtl8xxxu_priv *priv = hw->priv;
- struct device *dev = &priv->udev->dev;
- u16 val16;
- int ret = 0, channel;
- bool ht40;
-
- if (rtl8xxxu_debug & RTL8XXXU_DEBUG_CHANNEL)
- dev_info(dev,
- "%s: channel: %i (changed %08x chandef.width %02x)\n",
- __func__, hw->conf.chandef.chan->hw_value,
- changed, hw->conf.chandef.width);
-
- if (changed & IEEE80211_CONF_CHANGE_RETRY_LIMITS) {
- val16 = ((hw->conf.long_frame_max_tx_count <<
- RETRY_LIMIT_LONG_SHIFT) & RETRY_LIMIT_LONG_MASK) |
- ((hw->conf.short_frame_max_tx_count <<
- RETRY_LIMIT_SHORT_SHIFT) & RETRY_LIMIT_SHORT_MASK);
- rtl8xxxu_write16(priv, REG_RETRY_LIMIT, val16);
- }
-
- if (changed & IEEE80211_CONF_CHANGE_CHANNEL) {
- switch (hw->conf.chandef.width) {
- case NL80211_CHAN_WIDTH_20_NOHT:
- case NL80211_CHAN_WIDTH_20:
- ht40 = false;
- break;
- case NL80211_CHAN_WIDTH_40:
- ht40 = true;
- break;
- default:
- ret = -ENOTSUPP;
- goto exit;
- }
-
- channel = hw->conf.chandef.chan->hw_value;
-
- priv->fops->set_tx_power(priv, channel, ht40);
-
- priv->fops->config_channel(hw);
- }
-
-exit:
- return ret;
-}
-
-static int rtl8xxxu_conf_tx(struct ieee80211_hw *hw,
- struct ieee80211_vif *vif, u16 queue,
- const struct ieee80211_tx_queue_params *param)
-{
- struct rtl8xxxu_priv *priv = hw->priv;
- struct device *dev = &priv->udev->dev;
- u32 val32;
- u8 aifs, acm_ctrl, acm_bit;
-
- aifs = param->aifs;
-
- val32 = aifs |
- fls(param->cw_min) << EDCA_PARAM_ECW_MIN_SHIFT |
- fls(param->cw_max) << EDCA_PARAM_ECW_MAX_SHIFT |
- (u32)param->txop << EDCA_PARAM_TXOP_SHIFT;
-
- acm_ctrl = rtl8xxxu_read8(priv, REG_ACM_HW_CTRL);
- dev_dbg(dev,
- "%s: IEEE80211 queue %02x val %08x, acm %i, acm_ctrl %02x\n",
- __func__, queue, val32, param->acm, acm_ctrl);
-
- switch (queue) {
- case IEEE80211_AC_VO:
- acm_bit = ACM_HW_CTRL_VO;
- rtl8xxxu_write32(priv, REG_EDCA_VO_PARAM, val32);
- break;
- case IEEE80211_AC_VI:
- acm_bit = ACM_HW_CTRL_VI;
- rtl8xxxu_write32(priv, REG_EDCA_VI_PARAM, val32);
- break;
- case IEEE80211_AC_BE:
- acm_bit = ACM_HW_CTRL_BE;
- rtl8xxxu_write32(priv, REG_EDCA_BE_PARAM, val32);
- break;
- case IEEE80211_AC_BK:
- acm_bit = ACM_HW_CTRL_BK;
- rtl8xxxu_write32(priv, REG_EDCA_BK_PARAM, val32);
- break;
- default:
- acm_bit = 0;
- break;
- }
-
- if (param->acm)
- acm_ctrl |= acm_bit;
- else
- acm_ctrl &= ~acm_bit;
- rtl8xxxu_write8(priv, REG_ACM_HW_CTRL, acm_ctrl);
-
- return 0;
-}
-
-static void rtl8xxxu_configure_filter(struct ieee80211_hw *hw,
- unsigned int changed_flags,
- unsigned int *total_flags, u64 multicast)
-{
- struct rtl8xxxu_priv *priv = hw->priv;
- u32 rcr = rtl8xxxu_read32(priv, REG_RCR);
-
- dev_dbg(&priv->udev->dev, "%s: changed_flags %08x, total_flags %08x\n",
- __func__, changed_flags, *total_flags);
-
- /*
- * FIF_ALLMULTI ignored as all multicast frames are accepted (REG_MAR)
- */
-
- if (*total_flags & FIF_FCSFAIL)
- rcr |= RCR_ACCEPT_CRC32;
- else
- rcr &= ~RCR_ACCEPT_CRC32;
-
- /*
- * FIF_PLCPFAIL not supported?
- */
-
- if (*total_flags & FIF_BCN_PRBRESP_PROMISC)
- rcr &= ~RCR_CHECK_BSSID_BEACON;
- else
- rcr |= RCR_CHECK_BSSID_BEACON;
-
- if (*total_flags & FIF_CONTROL)
- rcr |= RCR_ACCEPT_CTRL_FRAME;
- else
- rcr &= ~RCR_ACCEPT_CTRL_FRAME;
-
- if (*total_flags & FIF_OTHER_BSS) {
- rcr |= RCR_ACCEPT_AP;
- rcr &= ~RCR_CHECK_BSSID_MATCH;
- } else {
- rcr &= ~RCR_ACCEPT_AP;
- rcr |= RCR_CHECK_BSSID_MATCH;
- }
-
- if (*total_flags & FIF_PSPOLL)
- rcr |= RCR_ACCEPT_PM;
- else
- rcr &= ~RCR_ACCEPT_PM;
-
- /*
- * FIF_PROBE_REQ ignored as probe requests always seem to be accepted
- */
-
- rtl8xxxu_write32(priv, REG_RCR, rcr);
-
- *total_flags &= (FIF_ALLMULTI | FIF_FCSFAIL | FIF_BCN_PRBRESP_PROMISC |
- FIF_CONTROL | FIF_OTHER_BSS | FIF_PSPOLL |
- FIF_PROBE_REQ);
-}
-
-static int rtl8xxxu_set_rts_threshold(struct ieee80211_hw *hw, u32 rts)
-{
- if (rts > 2347)
- return -EINVAL;
-
- return 0;
-}
-
-static int rtl8xxxu_set_key(struct ieee80211_hw *hw, enum set_key_cmd cmd,
- struct ieee80211_vif *vif,
- struct ieee80211_sta *sta,
- struct ieee80211_key_conf *key)
-{
- struct rtl8xxxu_priv *priv = hw->priv;
- struct device *dev = &priv->udev->dev;
- u8 mac_addr[ETH_ALEN];
- u8 val8;
- u16 val16;
- u32 val32;
- int retval = -EOPNOTSUPP;
-
- dev_dbg(dev, "%s: cmd %02x, cipher %08x, index %i\n",
- __func__, cmd, key->cipher, key->keyidx);
-
- if (vif->type != NL80211_IFTYPE_STATION)
- return -EOPNOTSUPP;
-
- if (key->keyidx > 3)
- return -EOPNOTSUPP;
-
- switch (key->cipher) {
- case WLAN_CIPHER_SUITE_WEP40:
- case WLAN_CIPHER_SUITE_WEP104:
-
- break;
- case WLAN_CIPHER_SUITE_CCMP:
- key->flags |= IEEE80211_KEY_FLAG_SW_MGMT_TX;
- break;
- case WLAN_CIPHER_SUITE_TKIP:
- key->flags |= IEEE80211_KEY_FLAG_GENERATE_MMIC;
- default:
- return -EOPNOTSUPP;
- }
-
- if (key->flags & IEEE80211_KEY_FLAG_PAIRWISE) {
- dev_dbg(dev, "%s: pairwise key\n", __func__);
- ether_addr_copy(mac_addr, sta->addr);
- } else {
- dev_dbg(dev, "%s: group key\n", __func__);
- eth_broadcast_addr(mac_addr);
- }
-
- val16 = rtl8xxxu_read16(priv, REG_CR);
- val16 |= CR_SECURITY_ENABLE;
- rtl8xxxu_write16(priv, REG_CR, val16);
-
- val8 = SEC_CFG_TX_SEC_ENABLE | SEC_CFG_TXBC_USE_DEFKEY |
- SEC_CFG_RX_SEC_ENABLE | SEC_CFG_RXBC_USE_DEFKEY;
- val8 |= SEC_CFG_TX_USE_DEFKEY | SEC_CFG_RX_USE_DEFKEY;
- rtl8xxxu_write8(priv, REG_SECURITY_CFG, val8);
-
- switch (cmd) {
- case SET_KEY:
- key->hw_key_idx = key->keyidx;
- key->flags |= IEEE80211_KEY_FLAG_GENERATE_IV;
- rtl8xxxu_cam_write(priv, key, mac_addr);
- retval = 0;
- break;
- case DISABLE_KEY:
- rtl8xxxu_write32(priv, REG_CAM_WRITE, 0x00000000);
- val32 = CAM_CMD_POLLING | CAM_CMD_WRITE |
- key->keyidx << CAM_CMD_KEY_SHIFT;
- rtl8xxxu_write32(priv, REG_CAM_CMD, val32);
- retval = 0;
- break;
- default:
- dev_warn(dev, "%s: Unsupported command %02x\n", __func__, cmd);
- }
-
- return retval;
-}
-
-static int
-rtl8xxxu_ampdu_action(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
- struct ieee80211_ampdu_params *params)
-{
- struct rtl8xxxu_priv *priv = hw->priv;
- struct device *dev = &priv->udev->dev;
- u8 ampdu_factor, ampdu_density;
- struct ieee80211_sta *sta = params->sta;
- enum ieee80211_ampdu_mlme_action action = params->action;
-
- switch (action) {
- case IEEE80211_AMPDU_TX_START:
- dev_info(dev, "%s: IEEE80211_AMPDU_TX_START\n", __func__);
- ampdu_factor = sta->ht_cap.ampdu_factor;
- ampdu_density = sta->ht_cap.ampdu_density;
- rtl8xxxu_set_ampdu_factor(priv, ampdu_factor);
- rtl8xxxu_set_ampdu_min_space(priv, ampdu_density);
- dev_dbg(dev,
- "Changed HT: ampdu_factor %02x, ampdu_density %02x\n",
- ampdu_factor, ampdu_density);
- break;
- case IEEE80211_AMPDU_TX_STOP_FLUSH:
- dev_info(dev, "%s: IEEE80211_AMPDU_TX_STOP_FLUSH\n", __func__);
- rtl8xxxu_set_ampdu_factor(priv, 0);
- rtl8xxxu_set_ampdu_min_space(priv, 0);
- break;
- case IEEE80211_AMPDU_TX_STOP_FLUSH_CONT:
- dev_info(dev, "%s: IEEE80211_AMPDU_TX_STOP_FLUSH_CONT\n",
- __func__);
- rtl8xxxu_set_ampdu_factor(priv, 0);
- rtl8xxxu_set_ampdu_min_space(priv, 0);
- break;
- case IEEE80211_AMPDU_RX_START:
- dev_info(dev, "%s: IEEE80211_AMPDU_RX_START\n", __func__);
- break;
- case IEEE80211_AMPDU_RX_STOP:
- dev_info(dev, "%s: IEEE80211_AMPDU_RX_STOP\n", __func__);
- break;
- default:
- break;
- }
- return 0;
-}
-
-static int rtl8xxxu_start(struct ieee80211_hw *hw)
-{
- struct rtl8xxxu_priv *priv = hw->priv;
- struct rtl8xxxu_rx_urb *rx_urb;
- struct rtl8xxxu_tx_urb *tx_urb;
- unsigned long flags;
- int ret, i;
-
- ret = 0;
-
- init_usb_anchor(&priv->rx_anchor);
- init_usb_anchor(&priv->tx_anchor);
- init_usb_anchor(&priv->int_anchor);
-
- priv->fops->enable_rf(priv);
- if (priv->usb_interrupts) {
- ret = rtl8xxxu_submit_int_urb(hw);
- if (ret)
- goto exit;
- }
-
- for (i = 0; i < RTL8XXXU_TX_URBS; i++) {
- tx_urb = kmalloc(sizeof(struct rtl8xxxu_tx_urb), GFP_KERNEL);
- if (!tx_urb) {
- if (!i)
- ret = -ENOMEM;
-
- goto error_out;
- }
- usb_init_urb(&tx_urb->urb);
- INIT_LIST_HEAD(&tx_urb->list);
- tx_urb->hw = hw;
- list_add(&tx_urb->list, &priv->tx_urb_free_list);
- priv->tx_urb_free_count++;
- }
-
- priv->tx_stopped = false;
-
- spin_lock_irqsave(&priv->rx_urb_lock, flags);
- priv->shutdown = false;
- spin_unlock_irqrestore(&priv->rx_urb_lock, flags);
-
- for (i = 0; i < RTL8XXXU_RX_URBS; i++) {
- rx_urb = kmalloc(sizeof(struct rtl8xxxu_rx_urb), GFP_KERNEL);
- if (!rx_urb) {
- if (!i)
- ret = -ENOMEM;
-
- goto error_out;
- }
- usb_init_urb(&rx_urb->urb);
- INIT_LIST_HEAD(&rx_urb->list);
- rx_urb->hw = hw;
-
- ret = rtl8xxxu_submit_rx_urb(priv, rx_urb);
- }
-exit:
- /*
- * Accept all data and mgmt frames
- */
- rtl8xxxu_write16(priv, REG_RXFLTMAP2, 0xffff);
- rtl8xxxu_write16(priv, REG_RXFLTMAP0, 0xffff);
-
- rtl8xxxu_write32(priv, REG_OFDM0_XA_AGC_CORE1, 0x6954341e);
-
- return ret;
-
-error_out:
- rtl8xxxu_free_tx_resources(priv);
- /*
- * Disable all data and mgmt frames
- */
- rtl8xxxu_write16(priv, REG_RXFLTMAP2, 0x0000);
- rtl8xxxu_write16(priv, REG_RXFLTMAP0, 0x0000);
-
- return ret;
-}
-
-static void rtl8xxxu_stop(struct ieee80211_hw *hw)
-{
- struct rtl8xxxu_priv *priv = hw->priv;
- unsigned long flags;
-
- rtl8xxxu_write8(priv, REG_TXPAUSE, 0xff);
-
- rtl8xxxu_write16(priv, REG_RXFLTMAP0, 0x0000);
- rtl8xxxu_write16(priv, REG_RXFLTMAP2, 0x0000);
-
- spin_lock_irqsave(&priv->rx_urb_lock, flags);
- priv->shutdown = true;
- spin_unlock_irqrestore(&priv->rx_urb_lock, flags);
-
- usb_kill_anchored_urbs(&priv->rx_anchor);
- usb_kill_anchored_urbs(&priv->tx_anchor);
- if (priv->usb_interrupts)
- usb_kill_anchored_urbs(&priv->int_anchor);
-
- rtl8xxxu_write8(priv, REG_TXPAUSE, 0xff);
-
- priv->fops->disable_rf(priv);
-
- /*
- * Disable interrupts
- */
- if (priv->usb_interrupts)
- rtl8xxxu_write32(priv, REG_USB_HIMR, 0);
-
- rtl8xxxu_free_rx_resources(priv);
- rtl8xxxu_free_tx_resources(priv);
-}
-
-static const struct ieee80211_ops rtl8xxxu_ops = {
- .tx = rtl8xxxu_tx,
- .add_interface = rtl8xxxu_add_interface,
- .remove_interface = rtl8xxxu_remove_interface,
- .config = rtl8xxxu_config,
- .conf_tx = rtl8xxxu_conf_tx,
- .bss_info_changed = rtl8xxxu_bss_info_changed,
- .configure_filter = rtl8xxxu_configure_filter,
- .set_rts_threshold = rtl8xxxu_set_rts_threshold,
- .start = rtl8xxxu_start,
- .stop = rtl8xxxu_stop,
- .sw_scan_start = rtl8xxxu_sw_scan_start,
- .sw_scan_complete = rtl8xxxu_sw_scan_complete,
- .set_key = rtl8xxxu_set_key,
- .ampdu_action = rtl8xxxu_ampdu_action,
-};
-
-static int rtl8xxxu_parse_usb(struct rtl8xxxu_priv *priv,
- struct usb_interface *interface)
-{
- struct usb_interface_descriptor *interface_desc;
- struct usb_host_interface *host_interface;
- struct usb_endpoint_descriptor *endpoint;
- struct device *dev = &priv->udev->dev;
- int i, j = 0, endpoints;
- u8 dir, xtype, num;
- int ret = 0;
-
- host_interface = &interface->altsetting[0];
- interface_desc = &host_interface->desc;
- endpoints = interface_desc->bNumEndpoints;
-
- for (i = 0; i < endpoints; i++) {
- endpoint = &host_interface->endpoint[i].desc;
-
- dir = endpoint->bEndpointAddress & USB_ENDPOINT_DIR_MASK;
- num = usb_endpoint_num(endpoint);
- xtype = usb_endpoint_type(endpoint);
- if (rtl8xxxu_debug & RTL8XXXU_DEBUG_USB)
- dev_dbg(dev,
- "%s: endpoint: dir %02x, # %02x, type %02x\n",
- __func__, dir, num, xtype);
- if (usb_endpoint_dir_in(endpoint) &&
- usb_endpoint_xfer_bulk(endpoint)) {
- if (rtl8xxxu_debug & RTL8XXXU_DEBUG_USB)
- dev_dbg(dev, "%s: in endpoint num %i\n",
- __func__, num);
-
- if (priv->pipe_in) {
- dev_warn(dev,
- "%s: Too many IN pipes\n", __func__);
- ret = -EINVAL;
- goto exit;
- }
-
- priv->pipe_in = usb_rcvbulkpipe(priv->udev, num);
- }
-
- if (usb_endpoint_dir_in(endpoint) &&
- usb_endpoint_xfer_int(endpoint)) {
- if (rtl8xxxu_debug & RTL8XXXU_DEBUG_USB)
- dev_dbg(dev, "%s: interrupt endpoint num %i\n",
- __func__, num);
-
- if (priv->pipe_interrupt) {
- dev_warn(dev, "%s: Too many INTERRUPT pipes\n",
- __func__);
- ret = -EINVAL;
- goto exit;
- }
-
- priv->pipe_interrupt = usb_rcvintpipe(priv->udev, num);
- }
-
- if (usb_endpoint_dir_out(endpoint) &&
- usb_endpoint_xfer_bulk(endpoint)) {
- if (rtl8xxxu_debug & RTL8XXXU_DEBUG_USB)
- dev_dbg(dev, "%s: out endpoint num %i\n",
- __func__, num);
- if (j >= RTL8XXXU_OUT_ENDPOINTS) {
- dev_warn(dev,
- "%s: Too many OUT pipes\n", __func__);
- ret = -EINVAL;
- goto exit;
- }
- priv->out_ep[j++] = num;
- }
- }
-exit:
- priv->nr_out_eps = j;
- return ret;
-}
-
-static int rtl8xxxu_probe(struct usb_interface *interface,
- const struct usb_device_id *id)
-{
- struct rtl8xxxu_priv *priv;
- struct ieee80211_hw *hw;
- struct usb_device *udev;
- struct ieee80211_supported_band *sband;
- int ret = 0;
- int untested = 1;
-
- udev = usb_get_dev(interface_to_usbdev(interface));
-
- switch (id->idVendor) {
- case USB_VENDOR_ID_REALTEK:
- switch(id->idProduct) {
- case 0x1724:
- case 0x8176:
- case 0x8178:
- case 0x817f:
- untested = 0;
- break;
- }
- break;
- case 0x7392:
- if (id->idProduct == 0x7811)
- untested = 0;
- break;
- case 0x050d:
- if (id->idProduct == 0x1004)
- untested = 0;
- break;
- default:
- break;
- }
-
- if (untested) {
- rtl8xxxu_debug |= RTL8XXXU_DEBUG_EFUSE;
- dev_info(&udev->dev,
- "This Realtek USB WiFi dongle (0x%04x:0x%04x) is untested!\n",
- id->idVendor, id->idProduct);
- dev_info(&udev->dev,
- "Please report results to Jes.Sorensen@gmail.com\n");
- }
-
- hw = ieee80211_alloc_hw(sizeof(struct rtl8xxxu_priv), &rtl8xxxu_ops);
- if (!hw) {
- ret = -ENOMEM;
- goto exit;
- }
-
- priv = hw->priv;
- priv->hw = hw;
- priv->udev = udev;
- priv->fops = (struct rtl8xxxu_fileops *)id->driver_info;
- mutex_init(&priv->usb_buf_mutex);
- mutex_init(&priv->h2c_mutex);
- INIT_LIST_HEAD(&priv->tx_urb_free_list);
- spin_lock_init(&priv->tx_urb_lock);
- INIT_LIST_HEAD(&priv->rx_urb_pending_list);
- spin_lock_init(&priv->rx_urb_lock);
- INIT_WORK(&priv->rx_urb_wq, rtl8xxxu_rx_urb_work);
-
- usb_set_intfdata(interface, hw);
-
- ret = rtl8xxxu_parse_usb(priv, interface);
- if (ret)
- goto exit;
-
- ret = rtl8xxxu_identify_chip(priv);
- if (ret) {
- dev_err(&udev->dev, "Fatal - failed to identify chip\n");
- goto exit;
- }
-
- ret = rtl8xxxu_read_efuse(priv);
- if (ret) {
- dev_err(&udev->dev, "Fatal - failed to read EFuse\n");
- goto exit;
- }
-
- ret = priv->fops->parse_efuse(priv);
- if (ret) {
- dev_err(&udev->dev, "Fatal - failed to parse EFuse\n");
- goto exit;
- }
-
- rtl8xxxu_print_chipinfo(priv);
-
- ret = priv->fops->load_firmware(priv);
- if (ret) {
- dev_err(&udev->dev, "Fatal - failed to load firmware\n");
- goto exit;
- }
-
- ret = rtl8xxxu_init_device(hw);
-
- hw->wiphy->max_scan_ssids = 1;
- hw->wiphy->max_scan_ie_len = IEEE80211_MAX_DATA_LEN;
- hw->wiphy->interface_modes = BIT(NL80211_IFTYPE_STATION);
- hw->queues = 4;
-
- sband = &rtl8xxxu_supported_band;
- sband->ht_cap.ht_supported = true;
- sband->ht_cap.ampdu_factor = IEEE80211_HT_MAX_AMPDU_64K;
- sband->ht_cap.ampdu_density = IEEE80211_HT_MPDU_DENSITY_16;
- sband->ht_cap.cap = IEEE80211_HT_CAP_SGI_20 | IEEE80211_HT_CAP_SGI_40;
- memset(&sband->ht_cap.mcs, 0, sizeof(sband->ht_cap.mcs));
- sband->ht_cap.mcs.rx_mask[0] = 0xff;
- sband->ht_cap.mcs.rx_mask[4] = 0x01;
- if (priv->rf_paths > 1) {
- sband->ht_cap.mcs.rx_mask[1] = 0xff;
- sband->ht_cap.cap |= IEEE80211_HT_CAP_SGI_40;
- }
- sband->ht_cap.mcs.tx_params = IEEE80211_HT_MCS_TX_DEFINED;
- /*
- * Some APs will negotiate HT20_40 in a noisy environment leading
- * to miserable performance. Rather than defaulting to this, only
- * enable it if explicitly requested at module load time.
- */
- if (rtl8xxxu_ht40_2g) {
- dev_info(&udev->dev, "Enabling HT_20_40 on the 2.4GHz band\n");
- sband->ht_cap.cap |= IEEE80211_HT_CAP_SUP_WIDTH_20_40;
- }
- hw->wiphy->bands[NL80211_BAND_2GHZ] = sband;
-
- hw->wiphy->rts_threshold = 2347;
-
- SET_IEEE80211_DEV(priv->hw, &interface->dev);
- SET_IEEE80211_PERM_ADDR(hw, priv->mac_addr);
-
- hw->extra_tx_headroom = priv->fops->tx_desc_size;
- ieee80211_hw_set(hw, SIGNAL_DBM);
- /*
- * The firmware handles rate control
- */
- ieee80211_hw_set(hw, HAS_RATE_CONTROL);
- ieee80211_hw_set(hw, AMPDU_AGGREGATION);
-
- ret = ieee80211_register_hw(priv->hw);
- if (ret) {
- dev_err(&udev->dev, "%s: Failed to register: %i\n",
- __func__, ret);
- goto exit;
- }
-
-exit:
- if (ret < 0)
- usb_put_dev(udev);
- return ret;
-}
-
-static void rtl8xxxu_disconnect(struct usb_interface *interface)
-{
- struct rtl8xxxu_priv *priv;
- struct ieee80211_hw *hw;
-
- hw = usb_get_intfdata(interface);
- priv = hw->priv;
-
- ieee80211_unregister_hw(hw);
-
- priv->fops->power_off(priv);
-
- usb_set_intfdata(interface, NULL);
-
- dev_info(&priv->udev->dev, "disconnecting\n");
-
- kfree(priv->fw_data);
- mutex_destroy(&priv->usb_buf_mutex);
- mutex_destroy(&priv->h2c_mutex);
-
- usb_put_dev(priv->udev);
- ieee80211_free_hw(hw);
-}
-
-static struct rtl8xxxu_fileops rtl8723au_fops = {
- .parse_efuse = rtl8723au_parse_efuse,
- .load_firmware = rtl8723au_load_firmware,
- .power_on = rtl8723au_power_on,
- .power_off = rtl8xxxu_power_off,
- .reset_8051 = rtl8xxxu_reset_8051,
- .llt_init = rtl8xxxu_init_llt_table,
- .init_phy_bb = rtl8xxxu_gen1_init_phy_bb,
- .init_phy_rf = rtl8723au_init_phy_rf,
- .phy_iq_calibrate = rtl8xxxu_gen1_phy_iq_calibrate,
- .config_channel = rtl8xxxu_gen1_config_channel,
- .parse_rx_desc = rtl8xxxu_parse_rxdesc16,
- .enable_rf = rtl8xxxu_gen1_enable_rf,
- .disable_rf = rtl8xxxu_gen1_disable_rf,
- .usb_quirks = rtl8xxxu_gen1_usb_quirks,
- .set_tx_power = rtl8xxxu_gen1_set_tx_power,
- .update_rate_mask = rtl8xxxu_update_rate_mask,
- .report_connect = rtl8xxxu_gen1_report_connect,
- .writeN_block_size = 1024,
- .mbox_ext_reg = REG_HMBOX_EXT_0,
- .mbox_ext_width = 2,
- .tx_desc_size = sizeof(struct rtl8xxxu_txdesc32),
- .rx_desc_size = sizeof(struct rtl8xxxu_rxdesc16),
- .adda_1t_init = 0x0b1b25a0,
- .adda_1t_path_on = 0x0bdb25a0,
- .adda_2t_path_on_a = 0x04db25a4,
- .adda_2t_path_on_b = 0x0b1b25a4,
- .trxff_boundary = 0x27ff,
- .pbp_rx = PBP_PAGE_SIZE_128,
- .pbp_tx = PBP_PAGE_SIZE_128,
- .mactable = rtl8xxxu_gen1_mac_init_table,
-};
-
-static struct rtl8xxxu_fileops rtl8723bu_fops = {
- .parse_efuse = rtl8723bu_parse_efuse,
- .load_firmware = rtl8723bu_load_firmware,
- .power_on = rtl8723bu_power_on,
- .power_off = rtl8723bu_power_off,
- .reset_8051 = rtl8723bu_reset_8051,
- .llt_init = rtl8xxxu_auto_llt_table,
- .init_phy_bb = rtl8723bu_init_phy_bb,
- .init_phy_rf = rtl8723bu_init_phy_rf,
- .phy_init_antenna_selection = rtl8723bu_phy_init_antenna_selection,
- .phy_iq_calibrate = rtl8723bu_phy_iq_calibrate,
- .config_channel = rtl8xxxu_gen2_config_channel,
- .parse_rx_desc = rtl8xxxu_parse_rxdesc24,
- .init_aggregation = rtl8723bu_init_aggregation,
- .init_statistics = rtl8723bu_init_statistics,
- .enable_rf = rtl8723b_enable_rf,
- .disable_rf = rtl8xxxu_gen2_disable_rf,
- .usb_quirks = rtl8xxxu_gen2_usb_quirks,
- .set_tx_power = rtl8723b_set_tx_power,
- .update_rate_mask = rtl8xxxu_gen2_update_rate_mask,
- .report_connect = rtl8xxxu_gen2_report_connect,
- .writeN_block_size = 1024,
- .mbox_ext_reg = REG_HMBOX_EXT0_8723B,
- .mbox_ext_width = 4,
- .tx_desc_size = sizeof(struct rtl8xxxu_txdesc40),
- .rx_desc_size = sizeof(struct rtl8xxxu_rxdesc24),
- .has_s0s1 = 1,
- .adda_1t_init = 0x01c00014,
- .adda_1t_path_on = 0x01c00014,
- .adda_2t_path_on_a = 0x01c00014,
- .adda_2t_path_on_b = 0x01c00014,
- .trxff_boundary = 0x3f7f,
- .pbp_rx = PBP_PAGE_SIZE_256,
- .pbp_tx = PBP_PAGE_SIZE_256,
- .mactable = rtl8723b_mac_init_table,
-};
-
-#ifdef CONFIG_RTL8XXXU_UNTESTED
-
-static struct rtl8xxxu_fileops rtl8192cu_fops = {
- .parse_efuse = rtl8192cu_parse_efuse,
- .load_firmware = rtl8192cu_load_firmware,
- .power_on = rtl8192cu_power_on,
- .power_off = rtl8xxxu_power_off,
- .reset_8051 = rtl8xxxu_reset_8051,
- .llt_init = rtl8xxxu_init_llt_table,
- .init_phy_bb = rtl8xxxu_gen1_init_phy_bb,
- .init_phy_rf = rtl8192cu_init_phy_rf,
- .phy_iq_calibrate = rtl8xxxu_gen1_phy_iq_calibrate,
- .config_channel = rtl8xxxu_gen1_config_channel,
- .parse_rx_desc = rtl8xxxu_parse_rxdesc16,
- .enable_rf = rtl8xxxu_gen1_enable_rf,
- .disable_rf = rtl8xxxu_gen1_disable_rf,
- .usb_quirks = rtl8xxxu_gen1_usb_quirks,
- .set_tx_power = rtl8xxxu_gen1_set_tx_power,
- .update_rate_mask = rtl8xxxu_update_rate_mask,
- .report_connect = rtl8xxxu_gen1_report_connect,
- .writeN_block_size = 128,
- .mbox_ext_reg = REG_HMBOX_EXT_0,
- .mbox_ext_width = 2,
- .tx_desc_size = sizeof(struct rtl8xxxu_txdesc32),
- .rx_desc_size = sizeof(struct rtl8xxxu_rxdesc16),
- .adda_1t_init = 0x0b1b25a0,
- .adda_1t_path_on = 0x0bdb25a0,
- .adda_2t_path_on_a = 0x04db25a4,
- .adda_2t_path_on_b = 0x0b1b25a4,
- .trxff_boundary = 0x27ff,
- .pbp_rx = PBP_PAGE_SIZE_128,
- .pbp_tx = PBP_PAGE_SIZE_128,
- .mactable = rtl8xxxu_gen1_mac_init_table,
-};
-
-#endif
-
-static struct rtl8xxxu_fileops rtl8192eu_fops = {
- .parse_efuse = rtl8192eu_parse_efuse,
- .load_firmware = rtl8192eu_load_firmware,
- .power_on = rtl8192eu_power_on,
- .power_off = rtl8xxxu_power_off,
- .reset_8051 = rtl8xxxu_reset_8051,
- .llt_init = rtl8xxxu_auto_llt_table,
- .init_phy_bb = rtl8192eu_init_phy_bb,
- .init_phy_rf = rtl8192eu_init_phy_rf,
- .phy_iq_calibrate = rtl8192eu_phy_iq_calibrate,
- .config_channel = rtl8xxxu_gen2_config_channel,
- .parse_rx_desc = rtl8xxxu_parse_rxdesc24,
- .enable_rf = rtl8192e_enable_rf,
- .disable_rf = rtl8xxxu_gen2_disable_rf,
- .usb_quirks = rtl8xxxu_gen2_usb_quirks,
- .set_tx_power = rtl8192e_set_tx_power,
- .update_rate_mask = rtl8xxxu_gen2_update_rate_mask,
- .report_connect = rtl8xxxu_gen2_report_connect,
- .writeN_block_size = 128,
- .mbox_ext_reg = REG_HMBOX_EXT0_8723B,
- .mbox_ext_width = 4,
- .tx_desc_size = sizeof(struct rtl8xxxu_txdesc40),
- .rx_desc_size = sizeof(struct rtl8xxxu_rxdesc24),
- .has_s0s1 = 0,
- .adda_1t_init = 0x0fc01616,
- .adda_1t_path_on = 0x0fc01616,
- .adda_2t_path_on_a = 0x0fc01616,
- .adda_2t_path_on_b = 0x0fc01616,
- .trxff_boundary = 0x3cff,
- .mactable = rtl8192e_mac_init_table,
- .total_page_num = TX_TOTAL_PAGE_NUM_8192E,
- .page_num_hi = TX_PAGE_NUM_HI_PQ_8192E,
- .page_num_lo = TX_PAGE_NUM_LO_PQ_8192E,
- .page_num_norm = TX_PAGE_NUM_NORM_PQ_8192E,
-};
-
-static struct usb_device_id dev_table[] = {
-{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x8724, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8723au_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x1724, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8723au_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x0724, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8723au_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x818b, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192eu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0xb720, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8723bu_fops},
-#ifdef CONFIG_RTL8XXXU_UNTESTED
-/* Still supported by rtlwifi */
-{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x8176, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x8178, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x817f, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-/* Tested by Larry Finger */
-{USB_DEVICE_AND_INTERFACE_INFO(0x7392, 0x7811, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-/* Tested by Andrea Merello */
-{USB_DEVICE_AND_INTERFACE_INFO(0x050d, 0x1004, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-/* Currently untested 8188 series devices */
-{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x8191, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x8170, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x8177, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x817a, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x817b, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x817d, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x817e, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x818a, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x317f, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x1058, 0x0631, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x04bb, 0x094c, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x050d, 0x1102, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x06f8, 0xe033, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x07b8, 0x8189, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x0846, 0x9041, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x0b05, 0x17ba, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x1e1e, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x5088, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x0df6, 0x0052, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x0df6, 0x005c, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x0eb0, 0x9071, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x103c, 0x1629, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x13d3, 0x3357, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x2001, 0x3308, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x2001, 0x330b, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x2019, 0x4902, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x2019, 0xab2a, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x2019, 0xab2e, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x2019, 0xed17, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x20f4, 0x648b, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x4855, 0x0090, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x4856, 0x0091, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0xcdab, 0x8010, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x04f2, 0xaff7, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x04f2, 0xaff9, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x04f2, 0xaffa, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x04f2, 0xaff8, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x04f2, 0xaffb, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x04f2, 0xaffc, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x2019, 0x1201, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-/* Currently untested 8192 series devices */
-{USB_DEVICE_AND_INTERFACE_INFO(0x04bb, 0x0950, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x050d, 0x2102, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x050d, 0x2103, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x0586, 0x341f, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x06f8, 0xe035, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x0b05, 0x17ab, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x0df6, 0x0061, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x0df6, 0x0070, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x0789, 0x016d, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x07aa, 0x0056, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x07b8, 0x8178, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x0846, 0x9021, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x0846, 0xf001, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x2e2e, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x0e66, 0x0019, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x0e66, 0x0020, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x2001, 0x3307, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x2001, 0x3309, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x2001, 0x330a, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x2019, 0xab2b, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x20f4, 0x624d, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x2357, 0x0100, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x4855, 0x0091, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-{USB_DEVICE_AND_INTERFACE_INFO(0x7392, 0x7822, 0xff, 0xff, 0xff),
- .driver_info = (unsigned long)&rtl8192cu_fops},
-#endif
-{ }
-};
-
-static struct usb_driver rtl8xxxu_driver = {
- .name = DRIVER_NAME,
- .probe = rtl8xxxu_probe,
- .disconnect = rtl8xxxu_disconnect,
- .id_table = dev_table,
- .no_dynamic_id = 1,
- .disable_hub_initiated_lpm = 1,
-};
-
-static int __init rtl8xxxu_module_init(void)
-{
- int res;
-
- res = usb_register(&rtl8xxxu_driver);
- if (res < 0)
- pr_err(DRIVER_NAME ": usb_register() failed (%i)\n", res);
-
- return res;
-}
-
-static void __exit rtl8xxxu_module_exit(void)
-{
- usb_deregister(&rtl8xxxu_driver);
-}
-
-
-MODULE_DEVICE_TABLE(usb, dev_table);
-
-module_init(rtl8xxxu_module_init);
-module_exit(rtl8xxxu_module_exit);
diff --git a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu.h b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu.h
index 3e2643c..870c9cd 100644
--- a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu.h
+++ b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu.h
@@ -1329,8 +1329,6 @@
void (*report_connect) (struct rtl8xxxu_priv *priv,
u8 macid, bool connect);
int writeN_block_size;
- u16 mbox_ext_reg;
- char mbox_ext_width;
char tx_desc_size;
char rx_desc_size;
char has_s0s1;
@@ -1347,3 +1345,82 @@
u8 page_num_lo;
u8 page_num_norm;
};
+
+extern int rtl8xxxu_debug;
+
+extern struct rtl8xxxu_reg8val rtl8xxxu_gen1_mac_init_table[];
+extern const u32 rtl8xxxu_iqk_phy_iq_bb_reg[];
+u8 rtl8xxxu_read8(struct rtl8xxxu_priv *priv, u16 addr);
+u16 rtl8xxxu_read16(struct rtl8xxxu_priv *priv, u16 addr);
+u32 rtl8xxxu_read32(struct rtl8xxxu_priv *priv, u16 addr);
+int rtl8xxxu_write8(struct rtl8xxxu_priv *priv, u16 addr, u8 val);
+int rtl8xxxu_write16(struct rtl8xxxu_priv *priv, u16 addr, u16 val);
+int rtl8xxxu_write32(struct rtl8xxxu_priv *priv, u16 addr, u32 val);
+u32 rtl8xxxu_read_rfreg(struct rtl8xxxu_priv *priv,
+ enum rtl8xxxu_rfpath path, u8 reg);
+int rtl8xxxu_write_rfreg(struct rtl8xxxu_priv *priv,
+ enum rtl8xxxu_rfpath path, u8 reg, u32 data);
+void rtl8xxxu_save_regs(struct rtl8xxxu_priv *priv, const u32 *regs,
+ u32 *backup, int count);
+void rtl8xxxu_restore_regs(struct rtl8xxxu_priv *priv, const u32 *regs,
+ u32 *backup, int count);
+void rtl8xxxu_save_mac_regs(struct rtl8xxxu_priv *priv,
+ const u32 *reg, u32 *backup);
+void rtl8xxxu_restore_mac_regs(struct rtl8xxxu_priv *priv,
+ const u32 *reg, u32 *backup);
+void rtl8xxxu_path_adda_on(struct rtl8xxxu_priv *priv, const u32 *regs,
+ bool path_a_on);
+void rtl8xxxu_mac_calibration(struct rtl8xxxu_priv *priv,
+ const u32 *regs, u32 *backup);
+void rtl8xxxu_fill_iqk_matrix_a(struct rtl8xxxu_priv *priv, bool iqk_ok,
+ int result[][8], int candidate, bool tx_only);
+void rtl8xxxu_fill_iqk_matrix_b(struct rtl8xxxu_priv *priv, bool iqk_ok,
+ int result[][8], int candidate, bool tx_only);
+int rtl8xxxu_init_phy_rf(struct rtl8xxxu_priv *priv,
+ struct rtl8xxxu_rfregval *table,
+ enum rtl8xxxu_rfpath path);
+int rtl8xxxu_init_phy_regs(struct rtl8xxxu_priv *priv,
+ struct rtl8xxxu_reg32val *array);
+int rtl8xxxu_load_firmware(struct rtl8xxxu_priv *priv, char *fw_name);
+void rtl8xxxu_firmware_self_reset(struct rtl8xxxu_priv *priv);
+void rtl8xxxu_power_off(struct rtl8xxxu_priv *priv);
+void rtl8xxxu_reset_8051(struct rtl8xxxu_priv *priv);
+int rtl8xxxu_auto_llt_table(struct rtl8xxxu_priv *priv, u8 last_tx_page);
+void rtl8xxxu_gen2_prepare_calibrate(struct rtl8xxxu_priv *priv, u8 start);
+int rtl8xxxu_flush_fifo(struct rtl8xxxu_priv *priv);
+int rtl8xxxu_gen2_h2c_cmd(struct rtl8xxxu_priv *priv,
+ struct h2c_cmd *h2c, int len);
+int rtl8xxxu_active_to_lps(struct rtl8xxxu_priv *priv);
+void rtl8xxxu_disabled_to_emu(struct rtl8xxxu_priv *priv);
+int rtl8xxxu_init_llt_table(struct rtl8xxxu_priv *priv, u8 last_tx_page);
+void rtl8xxxu_gen1_phy_iq_calibrate(struct rtl8xxxu_priv *priv);
+void rtl8xxxu_gen1_init_phy_bb(struct rtl8xxxu_priv *priv);
+void rtl8xxxu_gen1_set_tx_power(struct rtl8xxxu_priv *priv,
+ int channel, bool ht40);
+void rtl8xxxu_gen1_config_channel(struct ieee80211_hw *hw);
+void rtl8xxxu_gen2_config_channel(struct ieee80211_hw *hw);
+void rtl8xxxu_gen1_usb_quirks(struct rtl8xxxu_priv *priv);
+void rtl8xxxu_gen2_usb_quirks(struct rtl8xxxu_priv *priv);
+void rtl8xxxu_update_rate_mask(struct rtl8xxxu_priv *priv,
+ u32 ramask, int sgi);
+void rtl8xxxu_gen2_update_rate_mask(struct rtl8xxxu_priv *priv,
+ u32 ramask, int sgi);
+void rtl8xxxu_gen1_report_connect(struct rtl8xxxu_priv *priv,
+ u8 macid, bool connect);
+void rtl8xxxu_gen2_report_connect(struct rtl8xxxu_priv *priv,
+ u8 macid, bool connect);
+void rtl8xxxu_gen1_enable_rf(struct rtl8xxxu_priv *priv);
+void rtl8xxxu_gen1_disable_rf(struct rtl8xxxu_priv *priv);
+void rtl8xxxu_gen2_disable_rf(struct rtl8xxxu_priv *priv);
+int rtl8xxxu_parse_rxdesc16(struct rtl8xxxu_priv *priv, struct sk_buff *skb,
+ struct ieee80211_rx_status *rx_status);
+int rtl8xxxu_parse_rxdesc24(struct rtl8xxxu_priv *priv, struct sk_buff *skb,
+ struct ieee80211_rx_status *rx_status);
+int rtl8xxxu_gen2_channel_to_group(int channel);
+bool rtl8xxxu_gen2_simularity_compare(struct rtl8xxxu_priv *priv,
+ int result[][8], int c1, int c2);
+
+extern struct rtl8xxxu_fileops rtl8192cu_fops;
+extern struct rtl8xxxu_fileops rtl8192eu_fops;
+extern struct rtl8xxxu_fileops rtl8723au_fops;
+extern struct rtl8xxxu_fileops rtl8723bu_fops;
diff --git a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8192c.c b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8192c.c
new file mode 100644
index 0000000..2c86b55
--- /dev/null
+++ b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8192c.c
@@ -0,0 +1,586 @@
+/*
+ * RTL8XXXU mac80211 USB driver - 8188c/8188r/8192c specific subdriver
+ *
+ * Copyright (c) 2014 - 2016 Jes Sorensen <Jes.Sorensen@redhat.com>
+ *
+ * Portions, notably calibration code:
+ * Copyright(c) 2007 - 2011 Realtek Corporation. All rights reserved.
+ *
+ * This driver was written as a replacement for the vendor provided
+ * rtl8723au driver. As the Realtek 8xxx chips are very similar in
+ * their programming interface, I have started adding support for
+ * additional 8xxx chips like the 8192cu, 8188cus, etc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ */
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <linux/errno.h>
+#include <linux/slab.h>
+#include <linux/module.h>
+#include <linux/spinlock.h>
+#include <linux/list.h>
+#include <linux/usb.h>
+#include <linux/netdevice.h>
+#include <linux/etherdevice.h>
+#include <linux/ethtool.h>
+#include <linux/wireless.h>
+#include <linux/firmware.h>
+#include <linux/moduleparam.h>
+#include <net/mac80211.h>
+#include "rtl8xxxu.h"
+#include "rtl8xxxu_regs.h"
+
+#ifdef CONFIG_RTL8XXXU_UNTESTED
+static struct rtl8xxxu_power_base rtl8192c_power_base = {
+ .reg_0e00 = 0x07090c0c,
+ .reg_0e04 = 0x01020405,
+ .reg_0e08 = 0x00000000,
+ .reg_086c = 0x00000000,
+
+ .reg_0e10 = 0x0b0c0c0e,
+ .reg_0e14 = 0x01030506,
+ .reg_0e18 = 0x0b0c0d0e,
+ .reg_0e1c = 0x01030509,
+
+ .reg_0830 = 0x07090c0c,
+ .reg_0834 = 0x01020405,
+ .reg_0838 = 0x00000000,
+ .reg_086c_2 = 0x00000000,
+
+ .reg_083c = 0x0b0c0d0e,
+ .reg_0848 = 0x01030509,
+ .reg_084c = 0x0b0c0d0e,
+ .reg_0868 = 0x01030509,
+};
+
+static struct rtl8xxxu_power_base rtl8188r_power_base = {
+ .reg_0e00 = 0x06080808,
+ .reg_0e04 = 0x00040406,
+ .reg_0e08 = 0x00000000,
+ .reg_086c = 0x00000000,
+
+ .reg_0e10 = 0x04060608,
+ .reg_0e14 = 0x00020204,
+ .reg_0e18 = 0x04060608,
+ .reg_0e1c = 0x00020204,
+
+ .reg_0830 = 0x06080808,
+ .reg_0834 = 0x00040406,
+ .reg_0838 = 0x00000000,
+ .reg_086c_2 = 0x00000000,
+
+ .reg_083c = 0x04060608,
+ .reg_0848 = 0x00020204,
+ .reg_084c = 0x04060608,
+ .reg_0868 = 0x00020204,
+};
+
+static struct rtl8xxxu_rfregval rtl8192cu_radioa_2t_init_table[] = {
+ {0x00, 0x00030159}, {0x01, 0x00031284},
+ {0x02, 0x00098000}, {0x03, 0x00018c63},
+ {0x04, 0x000210e7}, {0x09, 0x0002044f},
+ {0x0a, 0x0001adb1}, {0x0b, 0x00054867},
+ {0x0c, 0x0008992e}, {0x0d, 0x0000e52c},
+ {0x0e, 0x00039ce7}, {0x0f, 0x00000451},
+ {0x19, 0x00000000}, {0x1a, 0x00010255},
+ {0x1b, 0x00060a00}, {0x1c, 0x000fc378},
+ {0x1d, 0x000a1250}, {0x1e, 0x0004445f},
+ {0x1f, 0x00080001}, {0x20, 0x0000b614},
+ {0x21, 0x0006c000}, {0x22, 0x00000000},
+ {0x23, 0x00001558}, {0x24, 0x00000060},
+ {0x25, 0x00000483}, {0x26, 0x0004f000},
+ {0x27, 0x000ec7d9}, {0x28, 0x000577c0},
+ {0x29, 0x00004783}, {0x2a, 0x00000001},
+ {0x2b, 0x00021334}, {0x2a, 0x00000000},
+ {0x2b, 0x00000054}, {0x2a, 0x00000001},
+ {0x2b, 0x00000808}, {0x2b, 0x00053333},
+ {0x2c, 0x0000000c}, {0x2a, 0x00000002},
+ {0x2b, 0x00000808}, {0x2b, 0x0005b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000003},
+ {0x2b, 0x00000808}, {0x2b, 0x00063333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000004},
+ {0x2b, 0x00000808}, {0x2b, 0x0006b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000005},
+ {0x2b, 0x00000808}, {0x2b, 0x00073333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000006},
+ {0x2b, 0x00000709}, {0x2b, 0x0005b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000007},
+ {0x2b, 0x00000709}, {0x2b, 0x00063333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000008},
+ {0x2b, 0x0000060a}, {0x2b, 0x0004b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000009},
+ {0x2b, 0x0000060a}, {0x2b, 0x00053333},
+ {0x2c, 0x0000000d}, {0x2a, 0x0000000a},
+ {0x2b, 0x0000060a}, {0x2b, 0x0005b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x0000000b},
+ {0x2b, 0x0000060a}, {0x2b, 0x00063333},
+ {0x2c, 0x0000000d}, {0x2a, 0x0000000c},
+ {0x2b, 0x0000060a}, {0x2b, 0x0006b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x0000000d},
+ {0x2b, 0x0000060a}, {0x2b, 0x00073333},
+ {0x2c, 0x0000000d}, {0x2a, 0x0000000e},
+ {0x2b, 0x0000050b}, {0x2b, 0x00066666},
+ {0x2c, 0x0000001a}, {0x2a, 0x000e0000},
+ {0x10, 0x0004000f}, {0x11, 0x000e31fc},
+ {0x10, 0x0006000f}, {0x11, 0x000ff9f8},
+ {0x10, 0x0002000f}, {0x11, 0x000203f9},
+ {0x10, 0x0003000f}, {0x11, 0x000ff500},
+ {0x10, 0x00000000}, {0x11, 0x00000000},
+ {0x10, 0x0008000f}, {0x11, 0x0003f100},
+ {0x10, 0x0009000f}, {0x11, 0x00023100},
+ {0x12, 0x00032000}, {0x12, 0x00071000},
+ {0x12, 0x000b0000}, {0x12, 0x000fc000},
+ {0x13, 0x000287b3}, {0x13, 0x000244b7},
+ {0x13, 0x000204ab}, {0x13, 0x0001c49f},
+ {0x13, 0x00018493}, {0x13, 0x0001429b},
+ {0x13, 0x00010299}, {0x13, 0x0000c29c},
+ {0x13, 0x000081a0}, {0x13, 0x000040ac},
+ {0x13, 0x00000020}, {0x14, 0x0001944c},
+ {0x14, 0x00059444}, {0x14, 0x0009944c},
+ {0x14, 0x000d9444}, {0x15, 0x0000f424},
+ {0x15, 0x0004f424}, {0x15, 0x0008f424},
+ {0x15, 0x000cf424}, {0x16, 0x000e0330},
+ {0x16, 0x000a0330}, {0x16, 0x00060330},
+ {0x16, 0x00020330}, {0x00, 0x00010159},
+ {0x18, 0x0000f401}, {0xfe, 0x00000000},
+ {0xfe, 0x00000000}, {0x1f, 0x00080003},
+ {0xfe, 0x00000000}, {0xfe, 0x00000000},
+ {0x1e, 0x00044457}, {0x1f, 0x00080000},
+ {0x00, 0x00030159},
+ {0xff, 0xffffffff}
+};
+
+static struct rtl8xxxu_rfregval rtl8192cu_radiob_2t_init_table[] = {
+ {0x00, 0x00030159}, {0x01, 0x00031284},
+ {0x02, 0x00098000}, {0x03, 0x00018c63},
+ {0x04, 0x000210e7}, {0x09, 0x0002044f},
+ {0x0a, 0x0001adb1}, {0x0b, 0x00054867},
+ {0x0c, 0x0008992e}, {0x0d, 0x0000e52c},
+ {0x0e, 0x00039ce7}, {0x0f, 0x00000451},
+ {0x12, 0x00032000}, {0x12, 0x00071000},
+ {0x12, 0x000b0000}, {0x12, 0x000fc000},
+ {0x13, 0x000287af}, {0x13, 0x000244b7},
+ {0x13, 0x000204ab}, {0x13, 0x0001c49f},
+ {0x13, 0x00018493}, {0x13, 0x00014297},
+ {0x13, 0x00010295}, {0x13, 0x0000c298},
+ {0x13, 0x0000819c}, {0x13, 0x000040a8},
+ {0x13, 0x0000001c}, {0x14, 0x0001944c},
+ {0x14, 0x00059444}, {0x14, 0x0009944c},
+ {0x14, 0x000d9444}, {0x15, 0x0000f424},
+ {0x15, 0x0004f424}, {0x15, 0x0008f424},
+ {0x15, 0x000cf424}, {0x16, 0x000e0330},
+ {0x16, 0x000a0330}, {0x16, 0x00060330},
+ {0x16, 0x00020330},
+ {0xff, 0xffffffff}
+};
+
+static struct rtl8xxxu_rfregval rtl8192cu_radioa_1t_init_table[] = {
+ {0x00, 0x00030159}, {0x01, 0x00031284},
+ {0x02, 0x00098000}, {0x03, 0x00018c63},
+ {0x04, 0x000210e7}, {0x09, 0x0002044f},
+ {0x0a, 0x0001adb1}, {0x0b, 0x00054867},
+ {0x0c, 0x0008992e}, {0x0d, 0x0000e52c},
+ {0x0e, 0x00039ce7}, {0x0f, 0x00000451},
+ {0x19, 0x00000000}, {0x1a, 0x00010255},
+ {0x1b, 0x00060a00}, {0x1c, 0x000fc378},
+ {0x1d, 0x000a1250}, {0x1e, 0x0004445f},
+ {0x1f, 0x00080001}, {0x20, 0x0000b614},
+ {0x21, 0x0006c000}, {0x22, 0x00000000},
+ {0x23, 0x00001558}, {0x24, 0x00000060},
+ {0x25, 0x00000483}, {0x26, 0x0004f000},
+ {0x27, 0x000ec7d9}, {0x28, 0x000577c0},
+ {0x29, 0x00004783}, {0x2a, 0x00000001},
+ {0x2b, 0x00021334}, {0x2a, 0x00000000},
+ {0x2b, 0x00000054}, {0x2a, 0x00000001},
+ {0x2b, 0x00000808}, {0x2b, 0x00053333},
+ {0x2c, 0x0000000c}, {0x2a, 0x00000002},
+ {0x2b, 0x00000808}, {0x2b, 0x0005b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000003},
+ {0x2b, 0x00000808}, {0x2b, 0x00063333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000004},
+ {0x2b, 0x00000808}, {0x2b, 0x0006b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000005},
+ {0x2b, 0x00000808}, {0x2b, 0x00073333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000006},
+ {0x2b, 0x00000709}, {0x2b, 0x0005b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000007},
+ {0x2b, 0x00000709}, {0x2b, 0x00063333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000008},
+ {0x2b, 0x0000060a}, {0x2b, 0x0004b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000009},
+ {0x2b, 0x0000060a}, {0x2b, 0x00053333},
+ {0x2c, 0x0000000d}, {0x2a, 0x0000000a},
+ {0x2b, 0x0000060a}, {0x2b, 0x0005b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x0000000b},
+ {0x2b, 0x0000060a}, {0x2b, 0x00063333},
+ {0x2c, 0x0000000d}, {0x2a, 0x0000000c},
+ {0x2b, 0x0000060a}, {0x2b, 0x0006b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x0000000d},
+ {0x2b, 0x0000060a}, {0x2b, 0x00073333},
+ {0x2c, 0x0000000d}, {0x2a, 0x0000000e},
+ {0x2b, 0x0000050b}, {0x2b, 0x00066666},
+ {0x2c, 0x0000001a}, {0x2a, 0x000e0000},
+ {0x10, 0x0004000f}, {0x11, 0x000e31fc},
+ {0x10, 0x0006000f}, {0x11, 0x000ff9f8},
+ {0x10, 0x0002000f}, {0x11, 0x000203f9},
+ {0x10, 0x0003000f}, {0x11, 0x000ff500},
+ {0x10, 0x00000000}, {0x11, 0x00000000},
+ {0x10, 0x0008000f}, {0x11, 0x0003f100},
+ {0x10, 0x0009000f}, {0x11, 0x00023100},
+ {0x12, 0x00032000}, {0x12, 0x00071000},
+ {0x12, 0x000b0000}, {0x12, 0x000fc000},
+ {0x13, 0x000287b3}, {0x13, 0x000244b7},
+ {0x13, 0x000204ab}, {0x13, 0x0001c49f},
+ {0x13, 0x00018493}, {0x13, 0x0001429b},
+ {0x13, 0x00010299}, {0x13, 0x0000c29c},
+ {0x13, 0x000081a0}, {0x13, 0x000040ac},
+ {0x13, 0x00000020}, {0x14, 0x0001944c},
+ {0x14, 0x00059444}, {0x14, 0x0009944c},
+ {0x14, 0x000d9444}, {0x15, 0x0000f405},
+ {0x15, 0x0004f405}, {0x15, 0x0008f405},
+ {0x15, 0x000cf405}, {0x16, 0x000e0330},
+ {0x16, 0x000a0330}, {0x16, 0x00060330},
+ {0x16, 0x00020330}, {0x00, 0x00010159},
+ {0x18, 0x0000f401}, {0xfe, 0x00000000},
+ {0xfe, 0x00000000}, {0x1f, 0x00080003},
+ {0xfe, 0x00000000}, {0xfe, 0x00000000},
+ {0x1e, 0x00044457}, {0x1f, 0x00080000},
+ {0x00, 0x00030159},
+ {0xff, 0xffffffff}
+};
+
+static struct rtl8xxxu_rfregval rtl8188ru_radioa_1t_highpa_table[] = {
+ {0x00, 0x00030159}, {0x01, 0x00031284},
+ {0x02, 0x00098000}, {0x03, 0x00018c63},
+ {0x04, 0x000210e7}, {0x09, 0x0002044f},
+ {0x0a, 0x0001adb0}, {0x0b, 0x00054867},
+ {0x0c, 0x0008992e}, {0x0d, 0x0000e529},
+ {0x0e, 0x00039ce7}, {0x0f, 0x00000451},
+ {0x19, 0x00000000}, {0x1a, 0x00000255},
+ {0x1b, 0x00060a00}, {0x1c, 0x000fc378},
+ {0x1d, 0x000a1250}, {0x1e, 0x0004445f},
+ {0x1f, 0x00080001}, {0x20, 0x0000b614},
+ {0x21, 0x0006c000}, {0x22, 0x0000083c},
+ {0x23, 0x00001558}, {0x24, 0x00000060},
+ {0x25, 0x00000483}, {0x26, 0x0004f000},
+ {0x27, 0x000ec7d9}, {0x28, 0x000977c0},
+ {0x29, 0x00004783}, {0x2a, 0x00000001},
+ {0x2b, 0x00021334}, {0x2a, 0x00000000},
+ {0x2b, 0x00000054}, {0x2a, 0x00000001},
+ {0x2b, 0x00000808}, {0x2b, 0x00053333},
+ {0x2c, 0x0000000c}, {0x2a, 0x00000002},
+ {0x2b, 0x00000808}, {0x2b, 0x0005b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000003},
+ {0x2b, 0x00000808}, {0x2b, 0x00063333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000004},
+ {0x2b, 0x00000808}, {0x2b, 0x0006b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000005},
+ {0x2b, 0x00000808}, {0x2b, 0x00073333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000006},
+ {0x2b, 0x00000709}, {0x2b, 0x0005b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000007},
+ {0x2b, 0x00000709}, {0x2b, 0x00063333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000008},
+ {0x2b, 0x0000060a}, {0x2b, 0x0004b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000009},
+ {0x2b, 0x0000060a}, {0x2b, 0x00053333},
+ {0x2c, 0x0000000d}, {0x2a, 0x0000000a},
+ {0x2b, 0x0000060a}, {0x2b, 0x0005b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x0000000b},
+ {0x2b, 0x0000060a}, {0x2b, 0x00063333},
+ {0x2c, 0x0000000d}, {0x2a, 0x0000000c},
+ {0x2b, 0x0000060a}, {0x2b, 0x0006b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x0000000d},
+ {0x2b, 0x0000060a}, {0x2b, 0x00073333},
+ {0x2c, 0x0000000d}, {0x2a, 0x0000000e},
+ {0x2b, 0x0000050b}, {0x2b, 0x00066666},
+ {0x2c, 0x0000001a}, {0x2a, 0x000e0000},
+ {0x10, 0x0004000f}, {0x11, 0x000e31fc},
+ {0x10, 0x0006000f}, {0x11, 0x000ff9f8},
+ {0x10, 0x0002000f}, {0x11, 0x000203f9},
+ {0x10, 0x0003000f}, {0x11, 0x000ff500},
+ {0x10, 0x00000000}, {0x11, 0x00000000},
+ {0x10, 0x0008000f}, {0x11, 0x0003f100},
+ {0x10, 0x0009000f}, {0x11, 0x00023100},
+ {0x12, 0x000d8000}, {0x12, 0x00090000},
+ {0x12, 0x00051000}, {0x12, 0x00012000},
+ {0x13, 0x00028fb4}, {0x13, 0x00024fa8},
+ {0x13, 0x000207a4}, {0x13, 0x0001c3b0},
+ {0x13, 0x000183a4}, {0x13, 0x00014398},
+ {0x13, 0x000101a4}, {0x13, 0x0000c198},
+ {0x13, 0x000080a4}, {0x13, 0x00004098},
+ {0x13, 0x00000000}, {0x14, 0x0001944c},
+ {0x14, 0x00059444}, {0x14, 0x0009944c},
+ {0x14, 0x000d9444}, {0x15, 0x0000f405},
+ {0x15, 0x0004f405}, {0x15, 0x0008f405},
+ {0x15, 0x000cf405}, {0x16, 0x000e0330},
+ {0x16, 0x000a0330}, {0x16, 0x00060330},
+ {0x16, 0x00020330}, {0x00, 0x00010159},
+ {0x18, 0x0000f401}, {0xfe, 0x00000000},
+ {0xfe, 0x00000000}, {0x1f, 0x00080003},
+ {0xfe, 0x00000000}, {0xfe, 0x00000000},
+ {0x1e, 0x00044457}, {0x1f, 0x00080000},
+ {0x00, 0x00030159},
+ {0xff, 0xffffffff}
+};
+
+static int rtl8192cu_load_firmware(struct rtl8xxxu_priv *priv)
+{
+ char *fw_name;
+ int ret;
+
+ if (!priv->vendor_umc)
+ fw_name = "rtlwifi/rtl8192cufw_TMSC.bin";
+ else if (priv->chip_cut || priv->rtl_chip == RTL8192C)
+ fw_name = "rtlwifi/rtl8192cufw_B.bin";
+ else
+ fw_name = "rtlwifi/rtl8192cufw_A.bin";
+
+ ret = rtl8xxxu_load_firmware(priv, fw_name);
+
+ return ret;
+}
+
+static int rtl8192cu_parse_efuse(struct rtl8xxxu_priv *priv)
+{
+ struct rtl8192cu_efuse *efuse = &priv->efuse_wifi.efuse8192;
+ int i;
+
+ if (efuse->rtl_id != cpu_to_le16(0x8129))
+ return -EINVAL;
+
+ ether_addr_copy(priv->mac_addr, efuse->mac_addr);
+
+ memcpy(priv->cck_tx_power_index_A,
+ efuse->cck_tx_power_index_A,
+ sizeof(efuse->cck_tx_power_index_A));
+ memcpy(priv->cck_tx_power_index_B,
+ efuse->cck_tx_power_index_B,
+ sizeof(efuse->cck_tx_power_index_B));
+
+ memcpy(priv->ht40_1s_tx_power_index_A,
+ efuse->ht40_1s_tx_power_index_A,
+ sizeof(efuse->ht40_1s_tx_power_index_A));
+ memcpy(priv->ht40_1s_tx_power_index_B,
+ efuse->ht40_1s_tx_power_index_B,
+ sizeof(efuse->ht40_1s_tx_power_index_B));
+ memcpy(priv->ht40_2s_tx_power_index_diff,
+ efuse->ht40_2s_tx_power_index_diff,
+ sizeof(efuse->ht40_2s_tx_power_index_diff));
+
+ memcpy(priv->ht20_tx_power_index_diff,
+ efuse->ht20_tx_power_index_diff,
+ sizeof(efuse->ht20_tx_power_index_diff));
+ memcpy(priv->ofdm_tx_power_index_diff,
+ efuse->ofdm_tx_power_index_diff,
+ sizeof(efuse->ofdm_tx_power_index_diff));
+
+ memcpy(priv->ht40_max_power_offset,
+ efuse->ht40_max_power_offset,
+ sizeof(efuse->ht40_max_power_offset));
+ memcpy(priv->ht20_max_power_offset,
+ efuse->ht20_max_power_offset,
+ sizeof(efuse->ht20_max_power_offset));
+
+ dev_info(&priv->udev->dev, "Vendor: %.7s\n",
+ efuse->vendor_name);
+ dev_info(&priv->udev->dev, "Product: %.20s\n",
+ efuse->device_name);
+
+ priv->power_base = &rtl8192c_power_base;
+
+ if (efuse->rf_regulatory & 0x20) {
+ sprintf(priv->chip_name, "8188RU");
+ priv->rtl_chip = RTL8188R;
+ priv->hi_pa = 1;
+ priv->no_pape = 1;
+ priv->power_base = &rtl8188r_power_base;
+ }
+
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_EFUSE) {
+ unsigned char *raw = priv->efuse_wifi.raw;
+
+ dev_info(&priv->udev->dev,
+ "%s: dumping efuse (0x%02zx bytes):\n",
+ __func__, sizeof(struct rtl8192cu_efuse));
+ for (i = 0; i < sizeof(struct rtl8192cu_efuse); i += 8) {
+ dev_info(&priv->udev->dev, "%02x: "
+ "%02x %02x %02x %02x %02x %02x %02x %02x\n", i,
+ raw[i], raw[i + 1], raw[i + 2],
+ raw[i + 3], raw[i + 4], raw[i + 5],
+ raw[i + 6], raw[i + 7]);
+ }
+ }
+ return 0;
+}
+
+static int rtl8192cu_init_phy_rf(struct rtl8xxxu_priv *priv)
+{
+ struct rtl8xxxu_rfregval *rftable;
+ int ret;
+
+ if (priv->rtl_chip == RTL8188R) {
+ rftable = rtl8188ru_radioa_1t_highpa_table;
+ ret = rtl8xxxu_init_phy_rf(priv, rftable, RF_A);
+ } else if (priv->rf_paths == 1) {
+ rftable = rtl8192cu_radioa_1t_init_table;
+ ret = rtl8xxxu_init_phy_rf(priv, rftable, RF_A);
+ } else {
+ rftable = rtl8192cu_radioa_2t_init_table;
+ ret = rtl8xxxu_init_phy_rf(priv, rftable, RF_A);
+ if (ret)
+ goto exit;
+ rftable = rtl8192cu_radiob_2t_init_table;
+ ret = rtl8xxxu_init_phy_rf(priv, rftable, RF_B);
+ }
+
+exit:
+ return ret;
+}
+
+static int rtl8192cu_power_on(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+ u16 val16;
+ u32 val32;
+ int i;
+
+ for (i = 100; i; i--) {
+ val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO);
+ if (val8 & APS_FSMCO_PFM_ALDN)
+ break;
+ }
+
+ if (!i) {
+ pr_info("%s: Poll failed\n", __func__);
+ return -ENODEV;
+ }
+
+ /*
+ * RSV_CTRL 0x001C[7:0] = 0x00, unlock ISO/CLK/Power control register
+ */
+ rtl8xxxu_write8(priv, REG_RSV_CTRL, 0x0);
+ rtl8xxxu_write8(priv, REG_SPS0_CTRL, 0x2b);
+ udelay(100);
+
+ val8 = rtl8xxxu_read8(priv, REG_LDOV12D_CTRL);
+ if (!(val8 & LDOV12D_ENABLE)) {
+ pr_info("%s: Enabling LDOV12D (%02x)\n", __func__, val8);
+ val8 |= LDOV12D_ENABLE;
+ rtl8xxxu_write8(priv, REG_LDOV12D_CTRL, val8);
+
+ udelay(100);
+
+ val8 = rtl8xxxu_read8(priv, REG_SYS_ISO_CTRL);
+ val8 &= ~SYS_ISO_MD2PP;
+ rtl8xxxu_write8(priv, REG_SYS_ISO_CTRL, val8);
+ }
+
+ /*
+ * Auto enable WLAN
+ */
+ val16 = rtl8xxxu_read16(priv, REG_APS_FSMCO);
+ val16 |= APS_FSMCO_MAC_ENABLE;
+ rtl8xxxu_write16(priv, REG_APS_FSMCO, val16);
+
+ for (i = 1000; i; i--) {
+ val16 = rtl8xxxu_read16(priv, REG_APS_FSMCO);
+ if (!(val16 & APS_FSMCO_MAC_ENABLE))
+ break;
+ }
+ if (!i) {
+ pr_info("%s: FSMCO_MAC_ENABLE poll failed\n", __func__);
+ return -EBUSY;
+ }
+
+ /*
+ * Enable radio, GPIO, LED
+ */
+ val16 = APS_FSMCO_HW_SUSPEND | APS_FSMCO_ENABLE_POWERDOWN |
+ APS_FSMCO_PFM_ALDN;
+ rtl8xxxu_write16(priv, REG_APS_FSMCO, val16);
+
+ /*
+ * Release RF digital isolation
+ */
+ val16 = rtl8xxxu_read16(priv, REG_SYS_ISO_CTRL);
+ val16 &= ~SYS_ISO_DIOR;
+ rtl8xxxu_write16(priv, REG_SYS_ISO_CTRL, val16);
+
+ val8 = rtl8xxxu_read8(priv, REG_APSD_CTRL);
+ val8 &= ~APSD_CTRL_OFF;
+ rtl8xxxu_write8(priv, REG_APSD_CTRL, val8);
+ for (i = 200; i; i--) {
+ val8 = rtl8xxxu_read8(priv, REG_APSD_CTRL);
+ if (!(val8 & APSD_CTRL_OFF_STATUS))
+ break;
+ }
+
+ if (!i) {
+ pr_info("%s: APSD_CTRL poll failed\n", __func__);
+ return -EBUSY;
+ }
+
+ /*
+ * Enable MAC DMA/WMAC/SCHEDULE/SEC block
+ */
+ val16 = rtl8xxxu_read16(priv, REG_CR);
+ val16 |= CR_HCI_TXDMA_ENABLE | CR_HCI_RXDMA_ENABLE |
+ CR_TXDMA_ENABLE | CR_RXDMA_ENABLE | CR_PROTOCOL_ENABLE |
+ CR_SCHEDULE_ENABLE | CR_MAC_TX_ENABLE | CR_MAC_RX_ENABLE;
+ rtl8xxxu_write16(priv, REG_CR, val16);
+
+ rtl8xxxu_write8(priv, 0xfe10, 0x19);
+
+ /*
+ * Workaround for 8188RU LNA power leakage problem.
+ */
+ if (priv->rtl_chip == RTL8188R) {
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_XCD_RF_PARM);
+ val32 &= ~BIT(1);
+ rtl8xxxu_write32(priv, REG_FPGA0_XCD_RF_PARM, val32);
+ }
+ return 0;
+}
+
+struct rtl8xxxu_fileops rtl8192cu_fops = {
+ .parse_efuse = rtl8192cu_parse_efuse,
+ .load_firmware = rtl8192cu_load_firmware,
+ .power_on = rtl8192cu_power_on,
+ .power_off = rtl8xxxu_power_off,
+ .reset_8051 = rtl8xxxu_reset_8051,
+ .llt_init = rtl8xxxu_init_llt_table,
+ .init_phy_bb = rtl8xxxu_gen1_init_phy_bb,
+ .init_phy_rf = rtl8192cu_init_phy_rf,
+ .phy_iq_calibrate = rtl8xxxu_gen1_phy_iq_calibrate,
+ .config_channel = rtl8xxxu_gen1_config_channel,
+ .parse_rx_desc = rtl8xxxu_parse_rxdesc16,
+ .enable_rf = rtl8xxxu_gen1_enable_rf,
+ .disable_rf = rtl8xxxu_gen1_disable_rf,
+ .usb_quirks = rtl8xxxu_gen1_usb_quirks,
+ .set_tx_power = rtl8xxxu_gen1_set_tx_power,
+ .update_rate_mask = rtl8xxxu_update_rate_mask,
+ .report_connect = rtl8xxxu_gen1_report_connect,
+ .writeN_block_size = 128,
+ .tx_desc_size = sizeof(struct rtl8xxxu_txdesc32),
+ .rx_desc_size = sizeof(struct rtl8xxxu_rxdesc16),
+ .adda_1t_init = 0x0b1b25a0,
+ .adda_1t_path_on = 0x0bdb25a0,
+ .adda_2t_path_on_a = 0x04db25a4,
+ .adda_2t_path_on_b = 0x0b1b25a4,
+ .trxff_boundary = 0x27ff,
+ .pbp_rx = PBP_PAGE_SIZE_128,
+ .pbp_tx = PBP_PAGE_SIZE_128,
+ .mactable = rtl8xxxu_gen1_mac_init_table,
+};
+#endif
diff --git a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8192e.c b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8192e.c
new file mode 100644
index 0000000..fe19ace
--- /dev/null
+++ b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8192e.c
@@ -0,0 +1,1525 @@
+/*
+ * RTL8XXXU mac80211 USB driver - 8192e specific subdriver
+ *
+ * Copyright (c) 2014 - 2016 Jes Sorensen <Jes.Sorensen@redhat.com>
+ *
+ * Portions, notably calibration code:
+ * Copyright(c) 2007 - 2011 Realtek Corporation. All rights reserved.
+ *
+ * This driver was written as a replacement for the vendor provided
+ * rtl8723au driver. As the Realtek 8xxx chips are very similar in
+ * their programming interface, I have started adding support for
+ * additional 8xxx chips like the 8192cu, 8188cus, etc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ */
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <linux/errno.h>
+#include <linux/slab.h>
+#include <linux/module.h>
+#include <linux/spinlock.h>
+#include <linux/list.h>
+#include <linux/usb.h>
+#include <linux/netdevice.h>
+#include <linux/etherdevice.h>
+#include <linux/ethtool.h>
+#include <linux/wireless.h>
+#include <linux/firmware.h>
+#include <linux/moduleparam.h>
+#include <net/mac80211.h>
+#include "rtl8xxxu.h"
+#include "rtl8xxxu_regs.h"
+
+static struct rtl8xxxu_reg8val rtl8192e_mac_init_table[] = {
+ {0x011, 0xeb}, {0x012, 0x07}, {0x014, 0x75}, {0x303, 0xa7},
+ {0x428, 0x0a}, {0x429, 0x10}, {0x430, 0x00}, {0x431, 0x00},
+ {0x432, 0x00}, {0x433, 0x01}, {0x434, 0x04}, {0x435, 0x05},
+ {0x436, 0x07}, {0x437, 0x08}, {0x43c, 0x04}, {0x43d, 0x05},
+ {0x43e, 0x07}, {0x43f, 0x08}, {0x440, 0x5d}, {0x441, 0x01},
+ {0x442, 0x00}, {0x444, 0x10}, {0x445, 0x00}, {0x446, 0x00},
+ {0x447, 0x00}, {0x448, 0x00}, {0x449, 0xf0}, {0x44a, 0x0f},
+ {0x44b, 0x3e}, {0x44c, 0x10}, {0x44d, 0x00}, {0x44e, 0x00},
+ {0x44f, 0x00}, {0x450, 0x00}, {0x451, 0xf0}, {0x452, 0x0f},
+ {0x453, 0x00}, {0x456, 0x5e}, {0x460, 0x66}, {0x461, 0x66},
+ {0x4c8, 0xff}, {0x4c9, 0x08}, {0x4cc, 0xff}, {0x4cd, 0xff},
+ {0x4ce, 0x01}, {0x500, 0x26}, {0x501, 0xa2}, {0x502, 0x2f},
+ {0x503, 0x00}, {0x504, 0x28}, {0x505, 0xa3}, {0x506, 0x5e},
+ {0x507, 0x00}, {0x508, 0x2b}, {0x509, 0xa4}, {0x50a, 0x5e},
+ {0x50b, 0x00}, {0x50c, 0x4f}, {0x50d, 0xa4}, {0x50e, 0x00},
+ {0x50f, 0x00}, {0x512, 0x1c}, {0x514, 0x0a}, {0x516, 0x0a},
+ {0x525, 0x4f}, {0x540, 0x12}, {0x541, 0x64}, {0x550, 0x10},
+ {0x551, 0x10}, {0x559, 0x02}, {0x55c, 0x50}, {0x55d, 0xff},
+ {0x605, 0x30}, {0x608, 0x0e}, {0x609, 0x2a}, {0x620, 0xff},
+ {0x621, 0xff}, {0x622, 0xff}, {0x623, 0xff}, {0x624, 0xff},
+ {0x625, 0xff}, {0x626, 0xff}, {0x627, 0xff}, {0x638, 0x50},
+ {0x63c, 0x0a}, {0x63d, 0x0a}, {0x63e, 0x0e}, {0x63f, 0x0e},
+ {0x640, 0x40}, {0x642, 0x40}, {0x643, 0x00}, {0x652, 0xc8},
+ {0x66e, 0x05}, {0x700, 0x21}, {0x701, 0x43}, {0x702, 0x65},
+ {0x703, 0x87}, {0x708, 0x21}, {0x709, 0x43}, {0x70a, 0x65},
+ {0x70b, 0x87},
+ {0xffff, 0xff},
+};
+
+static struct rtl8xxxu_reg32val rtl8192eu_phy_init_table[] = {
+ {0x800, 0x80040000}, {0x804, 0x00000003},
+ {0x808, 0x0000fc00}, {0x80c, 0x0000000a},
+ {0x810, 0x10001331}, {0x814, 0x020c3d10},
+ {0x818, 0x02220385}, {0x81c, 0x00000000},
+ {0x820, 0x01000100}, {0x824, 0x00390204},
+ {0x828, 0x01000100}, {0x82c, 0x00390204},
+ {0x830, 0x32323232}, {0x834, 0x30303030},
+ {0x838, 0x30303030}, {0x83c, 0x30303030},
+ {0x840, 0x00010000}, {0x844, 0x00010000},
+ {0x848, 0x28282828}, {0x84c, 0x28282828},
+ {0x850, 0x00000000}, {0x854, 0x00000000},
+ {0x858, 0x009a009a}, {0x85c, 0x01000014},
+ {0x860, 0x66f60000}, {0x864, 0x061f0000},
+ {0x868, 0x30303030}, {0x86c, 0x30303030},
+ {0x870, 0x00000000}, {0x874, 0x55004200},
+ {0x878, 0x08080808}, {0x87c, 0x00000000},
+ {0x880, 0xb0000c1c}, {0x884, 0x00000001},
+ {0x888, 0x00000000}, {0x88c, 0xcc0000c0},
+ {0x890, 0x00000800}, {0x894, 0xfffffffe},
+ {0x898, 0x40302010}, {0x900, 0x00000000},
+ {0x904, 0x00000023}, {0x908, 0x00000000},
+ {0x90c, 0x81121313}, {0x910, 0x806c0001},
+ {0x914, 0x00000001}, {0x918, 0x00000000},
+ {0x91c, 0x00010000}, {0x924, 0x00000001},
+ {0x928, 0x00000000}, {0x92c, 0x00000000},
+ {0x930, 0x00000000}, {0x934, 0x00000000},
+ {0x938, 0x00000000}, {0x93c, 0x00000000},
+ {0x940, 0x00000000}, {0x944, 0x00000000},
+ {0x94c, 0x00000008}, {0xa00, 0x00d0c7c8},
+ {0xa04, 0x81ff000c}, {0xa08, 0x8c838300},
+ {0xa0c, 0x2e68120f}, {0xa10, 0x95009b78},
+ {0xa14, 0x1114d028}, {0xa18, 0x00881117},
+ {0xa1c, 0x89140f00}, {0xa20, 0x1a1b0000},
+ {0xa24, 0x090e1317}, {0xa28, 0x00000204},
+ {0xa2c, 0x00d30000}, {0xa70, 0x101fff00},
+ {0xa74, 0x00000007}, {0xa78, 0x00000900},
+ {0xa7c, 0x225b0606}, {0xa80, 0x218075b1},
+ {0xb38, 0x00000000}, {0xc00, 0x48071d40},
+ {0xc04, 0x03a05633}, {0xc08, 0x000000e4},
+ {0xc0c, 0x6c6c6c6c}, {0xc10, 0x08800000},
+ {0xc14, 0x40000100}, {0xc18, 0x08800000},
+ {0xc1c, 0x40000100}, {0xc20, 0x00000000},
+ {0xc24, 0x00000000}, {0xc28, 0x00000000},
+ {0xc2c, 0x00000000}, {0xc30, 0x69e9ac47},
+ {0xc34, 0x469652af}, {0xc38, 0x49795994},
+ {0xc3c, 0x0a97971c}, {0xc40, 0x1f7c403f},
+ {0xc44, 0x000100b7}, {0xc48, 0xec020107},
+ {0xc4c, 0x007f037f},
+#ifdef EXT_PA_8192EU
+ /* External PA or external LNA */
+ {0xc50, 0x00340220},
+#else
+ {0xc50, 0x00340020},
+#endif
+ {0xc54, 0x0080801f},
+#ifdef EXT_PA_8192EU
+ /* External PA or external LNA */
+ {0xc58, 0x00000220},
+#else
+ {0xc58, 0x00000020},
+#endif
+ {0xc5c, 0x00248492}, {0xc60, 0x00000000},
+ {0xc64, 0x7112848b}, {0xc68, 0x47c00bff},
+ {0xc6c, 0x00000036}, {0xc70, 0x00000600},
+ {0xc74, 0x02013169}, {0xc78, 0x0000001f},
+ {0xc7c, 0x00b91612},
+#ifdef EXT_PA_8192EU
+ /* External PA or external LNA */
+ {0xc80, 0x2d4000b5},
+#else
+ {0xc80, 0x40000100},
+#endif
+ {0xc84, 0x21f60000},
+#ifdef EXT_PA_8192EU
+ /* External PA or external LNA */
+ {0xc88, 0x2d4000b5},
+#else
+ {0xc88, 0x40000100},
+#endif
+ {0xc8c, 0xa0e40000}, {0xc90, 0x00121820},
+ {0xc94, 0x00000000}, {0xc98, 0x00121820},
+ {0xc9c, 0x00007f7f}, {0xca0, 0x00000000},
+ {0xca4, 0x000300a0}, {0xca8, 0x00000000},
+ {0xcac, 0x00000000}, {0xcb0, 0x00000000},
+ {0xcb4, 0x00000000}, {0xcb8, 0x00000000},
+ {0xcbc, 0x28000000}, {0xcc0, 0x00000000},
+ {0xcc4, 0x00000000}, {0xcc8, 0x00000000},
+ {0xccc, 0x00000000}, {0xcd0, 0x00000000},
+ {0xcd4, 0x00000000}, {0xcd8, 0x64b22427},
+ {0xcdc, 0x00766932}, {0xce0, 0x00222222},
+ {0xce4, 0x00040000}, {0xce8, 0x77644302},
+ {0xcec, 0x2f97d40c}, {0xd00, 0x00080740},
+ {0xd04, 0x00020403}, {0xd08, 0x0000907f},
+ {0xd0c, 0x20010201}, {0xd10, 0xa0633333},
+ {0xd14, 0x3333bc43}, {0xd18, 0x7a8f5b6b},
+ {0xd1c, 0x0000007f}, {0xd2c, 0xcc979975},
+ {0xd30, 0x00000000}, {0xd34, 0x80608000},
+ {0xd38, 0x00000000}, {0xd3c, 0x00127353},
+ {0xd40, 0x00000000}, {0xd44, 0x00000000},
+ {0xd48, 0x00000000}, {0xd4c, 0x00000000},
+ {0xd50, 0x6437140a}, {0xd54, 0x00000000},
+ {0xd58, 0x00000282}, {0xd5c, 0x30032064},
+ {0xd60, 0x4653de68}, {0xd64, 0x04518a3c},
+ {0xd68, 0x00002101}, {0xd6c, 0x2a201c16},
+ {0xd70, 0x1812362e}, {0xd74, 0x322c2220},
+ {0xd78, 0x000e3c24}, {0xd80, 0x01081008},
+ {0xd84, 0x00000800}, {0xd88, 0xf0b50000},
+ {0xe00, 0x30303030}, {0xe04, 0x30303030},
+ {0xe08, 0x03903030}, {0xe10, 0x30303030},
+ {0xe14, 0x30303030}, {0xe18, 0x30303030},
+ {0xe1c, 0x30303030}, {0xe28, 0x00000000},
+ {0xe30, 0x1000dc1f}, {0xe34, 0x10008c1f},
+ {0xe38, 0x02140102}, {0xe3c, 0x681604c2},
+ {0xe40, 0x01007c00}, {0xe44, 0x01004800},
+ {0xe48, 0xfb000000}, {0xe4c, 0x000028d1},
+ {0xe50, 0x1000dc1f}, {0xe54, 0x10008c1f},
+ {0xe58, 0x02140102}, {0xe5c, 0x28160d05},
+ {0xe60, 0x00000008}, {0xe68, 0x0fc05656},
+ {0xe6c, 0x03c09696}, {0xe70, 0x03c09696},
+ {0xe74, 0x0c005656}, {0xe78, 0x0c005656},
+ {0xe7c, 0x0c005656}, {0xe80, 0x0c005656},
+ {0xe84, 0x03c09696}, {0xe88, 0x0c005656},
+ {0xe8c, 0x03c09696}, {0xed0, 0x03c09696},
+ {0xed4, 0x03c09696}, {0xed8, 0x03c09696},
+ {0xedc, 0x0000d6d6}, {0xee0, 0x0000d6d6},
+ {0xeec, 0x0fc01616}, {0xee4, 0xb0000c1c},
+ {0xee8, 0x00000001}, {0xf14, 0x00000003},
+ {0xf4c, 0x00000000}, {0xf00, 0x00000300},
+ {0xffff, 0xffffffff},
+};
+
+static struct rtl8xxxu_reg32val rtl8xxx_agc_8192eu_std_table[] = {
+ {0xc78, 0xfb000001}, {0xc78, 0xfb010001},
+ {0xc78, 0xfb020001}, {0xc78, 0xfb030001},
+ {0xc78, 0xfb040001}, {0xc78, 0xfb050001},
+ {0xc78, 0xfa060001}, {0xc78, 0xf9070001},
+ {0xc78, 0xf8080001}, {0xc78, 0xf7090001},
+ {0xc78, 0xf60a0001}, {0xc78, 0xf50b0001},
+ {0xc78, 0xf40c0001}, {0xc78, 0xf30d0001},
+ {0xc78, 0xf20e0001}, {0xc78, 0xf10f0001},
+ {0xc78, 0xf0100001}, {0xc78, 0xef110001},
+ {0xc78, 0xee120001}, {0xc78, 0xed130001},
+ {0xc78, 0xec140001}, {0xc78, 0xeb150001},
+ {0xc78, 0xea160001}, {0xc78, 0xe9170001},
+ {0xc78, 0xe8180001}, {0xc78, 0xe7190001},
+ {0xc78, 0xc81a0001}, {0xc78, 0xc71b0001},
+ {0xc78, 0xc61c0001}, {0xc78, 0x071d0001},
+ {0xc78, 0x061e0001}, {0xc78, 0x051f0001},
+ {0xc78, 0x04200001}, {0xc78, 0x03210001},
+ {0xc78, 0xaa220001}, {0xc78, 0xa9230001},
+ {0xc78, 0xa8240001}, {0xc78, 0xa7250001},
+ {0xc78, 0xa6260001}, {0xc78, 0x85270001},
+ {0xc78, 0x84280001}, {0xc78, 0x83290001},
+ {0xc78, 0x252a0001}, {0xc78, 0x242b0001},
+ {0xc78, 0x232c0001}, {0xc78, 0x222d0001},
+ {0xc78, 0x672e0001}, {0xc78, 0x662f0001},
+ {0xc78, 0x65300001}, {0xc78, 0x64310001},
+ {0xc78, 0x63320001}, {0xc78, 0x62330001},
+ {0xc78, 0x61340001}, {0xc78, 0x45350001},
+ {0xc78, 0x44360001}, {0xc78, 0x43370001},
+ {0xc78, 0x42380001}, {0xc78, 0x41390001},
+ {0xc78, 0x403a0001}, {0xc78, 0x403b0001},
+ {0xc78, 0x403c0001}, {0xc78, 0x403d0001},
+ {0xc78, 0x403e0001}, {0xc78, 0x403f0001},
+ {0xc78, 0xfb400001}, {0xc78, 0xfb410001},
+ {0xc78, 0xfb420001}, {0xc78, 0xfb430001},
+ {0xc78, 0xfb440001}, {0xc78, 0xfb450001},
+ {0xc78, 0xfa460001}, {0xc78, 0xf9470001},
+ {0xc78, 0xf8480001}, {0xc78, 0xf7490001},
+ {0xc78, 0xf64a0001}, {0xc78, 0xf54b0001},
+ {0xc78, 0xf44c0001}, {0xc78, 0xf34d0001},
+ {0xc78, 0xf24e0001}, {0xc78, 0xf14f0001},
+ {0xc78, 0xf0500001}, {0xc78, 0xef510001},
+ {0xc78, 0xee520001}, {0xc78, 0xed530001},
+ {0xc78, 0xec540001}, {0xc78, 0xeb550001},
+ {0xc78, 0xea560001}, {0xc78, 0xe9570001},
+ {0xc78, 0xe8580001}, {0xc78, 0xe7590001},
+ {0xc78, 0xe65a0001}, {0xc78, 0xe55b0001},
+ {0xc78, 0xe45c0001}, {0xc78, 0xe35d0001},
+ {0xc78, 0xe25e0001}, {0xc78, 0xe15f0001},
+ {0xc78, 0x8a600001}, {0xc78, 0x89610001},
+ {0xc78, 0x88620001}, {0xc78, 0x87630001},
+ {0xc78, 0x86640001}, {0xc78, 0x85650001},
+ {0xc78, 0x84660001}, {0xc78, 0x83670001},
+ {0xc78, 0x82680001}, {0xc78, 0x6b690001},
+ {0xc78, 0x6a6a0001}, {0xc78, 0x696b0001},
+ {0xc78, 0x686c0001}, {0xc78, 0x676d0001},
+ {0xc78, 0x666e0001}, {0xc78, 0x656f0001},
+ {0xc78, 0x64700001}, {0xc78, 0x63710001},
+ {0xc78, 0x62720001}, {0xc78, 0x61730001},
+ {0xc78, 0x49740001}, {0xc78, 0x48750001},
+ {0xc78, 0x47760001}, {0xc78, 0x46770001},
+ {0xc78, 0x45780001}, {0xc78, 0x44790001},
+ {0xc78, 0x437a0001}, {0xc78, 0x427b0001},
+ {0xc78, 0x417c0001}, {0xc78, 0x407d0001},
+ {0xc78, 0x407e0001}, {0xc78, 0x407f0001},
+ {0xc50, 0x00040022}, {0xc50, 0x00040020},
+ {0xffff, 0xffffffff}
+};
+
+static struct rtl8xxxu_reg32val rtl8xxx_agc_8192eu_highpa_table[] = {
+ {0xc78, 0xfa000001}, {0xc78, 0xf9010001},
+ {0xc78, 0xf8020001}, {0xc78, 0xf7030001},
+ {0xc78, 0xf6040001}, {0xc78, 0xf5050001},
+ {0xc78, 0xf4060001}, {0xc78, 0xf3070001},
+ {0xc78, 0xf2080001}, {0xc78, 0xf1090001},
+ {0xc78, 0xf00a0001}, {0xc78, 0xef0b0001},
+ {0xc78, 0xee0c0001}, {0xc78, 0xed0d0001},
+ {0xc78, 0xec0e0001}, {0xc78, 0xeb0f0001},
+ {0xc78, 0xea100001}, {0xc78, 0xe9110001},
+ {0xc78, 0xe8120001}, {0xc78, 0xe7130001},
+ {0xc78, 0xe6140001}, {0xc78, 0xe5150001},
+ {0xc78, 0xe4160001}, {0xc78, 0xe3170001},
+ {0xc78, 0xe2180001}, {0xc78, 0xe1190001},
+ {0xc78, 0x8a1a0001}, {0xc78, 0x891b0001},
+ {0xc78, 0x881c0001}, {0xc78, 0x871d0001},
+ {0xc78, 0x861e0001}, {0xc78, 0x851f0001},
+ {0xc78, 0x84200001}, {0xc78, 0x83210001},
+ {0xc78, 0x82220001}, {0xc78, 0x6a230001},
+ {0xc78, 0x69240001}, {0xc78, 0x68250001},
+ {0xc78, 0x67260001}, {0xc78, 0x66270001},
+ {0xc78, 0x65280001}, {0xc78, 0x64290001},
+ {0xc78, 0x632a0001}, {0xc78, 0x622b0001},
+ {0xc78, 0x612c0001}, {0xc78, 0x602d0001},
+ {0xc78, 0x472e0001}, {0xc78, 0x462f0001},
+ {0xc78, 0x45300001}, {0xc78, 0x44310001},
+ {0xc78, 0x43320001}, {0xc78, 0x42330001},
+ {0xc78, 0x41340001}, {0xc78, 0x40350001},
+ {0xc78, 0x40360001}, {0xc78, 0x40370001},
+ {0xc78, 0x40380001}, {0xc78, 0x40390001},
+ {0xc78, 0x403a0001}, {0xc78, 0x403b0001},
+ {0xc78, 0x403c0001}, {0xc78, 0x403d0001},
+ {0xc78, 0x403e0001}, {0xc78, 0x403f0001},
+ {0xc78, 0xfa400001}, {0xc78, 0xf9410001},
+ {0xc78, 0xf8420001}, {0xc78, 0xf7430001},
+ {0xc78, 0xf6440001}, {0xc78, 0xf5450001},
+ {0xc78, 0xf4460001}, {0xc78, 0xf3470001},
+ {0xc78, 0xf2480001}, {0xc78, 0xf1490001},
+ {0xc78, 0xf04a0001}, {0xc78, 0xef4b0001},
+ {0xc78, 0xee4c0001}, {0xc78, 0xed4d0001},
+ {0xc78, 0xec4e0001}, {0xc78, 0xeb4f0001},
+ {0xc78, 0xea500001}, {0xc78, 0xe9510001},
+ {0xc78, 0xe8520001}, {0xc78, 0xe7530001},
+ {0xc78, 0xe6540001}, {0xc78, 0xe5550001},
+ {0xc78, 0xe4560001}, {0xc78, 0xe3570001},
+ {0xc78, 0xe2580001}, {0xc78, 0xe1590001},
+ {0xc78, 0x8a5a0001}, {0xc78, 0x895b0001},
+ {0xc78, 0x885c0001}, {0xc78, 0x875d0001},
+ {0xc78, 0x865e0001}, {0xc78, 0x855f0001},
+ {0xc78, 0x84600001}, {0xc78, 0x83610001},
+ {0xc78, 0x82620001}, {0xc78, 0x6a630001},
+ {0xc78, 0x69640001}, {0xc78, 0x68650001},
+ {0xc78, 0x67660001}, {0xc78, 0x66670001},
+ {0xc78, 0x65680001}, {0xc78, 0x64690001},
+ {0xc78, 0x636a0001}, {0xc78, 0x626b0001},
+ {0xc78, 0x616c0001}, {0xc78, 0x606d0001},
+ {0xc78, 0x476e0001}, {0xc78, 0x466f0001},
+ {0xc78, 0x45700001}, {0xc78, 0x44710001},
+ {0xc78, 0x43720001}, {0xc78, 0x42730001},
+ {0xc78, 0x41740001}, {0xc78, 0x40750001},
+ {0xc78, 0x40760001}, {0xc78, 0x40770001},
+ {0xc78, 0x40780001}, {0xc78, 0x40790001},
+ {0xc78, 0x407a0001}, {0xc78, 0x407b0001},
+ {0xc78, 0x407c0001}, {0xc78, 0x407d0001},
+ {0xc78, 0x407e0001}, {0xc78, 0x407f0001},
+ {0xc50, 0x00040222}, {0xc50, 0x00040220},
+ {0xffff, 0xffffffff}
+};
+
+static struct rtl8xxxu_rfregval rtl8192eu_radioa_init_table[] = {
+ {0x7f, 0x00000082}, {0x81, 0x0003fc00},
+ {0x00, 0x00030000}, {0x08, 0x00008400},
+ {0x18, 0x00000407}, {0x19, 0x00000012},
+ {0x1b, 0x00000064}, {0x1e, 0x00080009},
+ {0x1f, 0x00000880}, {0x2f, 0x0001a060},
+ {0x3f, 0x00000000}, {0x42, 0x000060c0},
+ {0x57, 0x000d0000}, {0x58, 0x000be180},
+ {0x67, 0x00001552}, {0x83, 0x00000000},
+ {0xb0, 0x000ff9f1}, {0xb1, 0x00055418},
+ {0xb2, 0x0008cc00}, {0xb4, 0x00043083},
+ {0xb5, 0x00008166}, {0xb6, 0x0000803e},
+ {0xb7, 0x0001c69f}, {0xb8, 0x0000407f},
+ {0xb9, 0x00080001}, {0xba, 0x00040001},
+ {0xbb, 0x00000400}, {0xbf, 0x000c0000},
+ {0xc2, 0x00002400}, {0xc3, 0x00000009},
+ {0xc4, 0x00040c91}, {0xc5, 0x00099999},
+ {0xc6, 0x000000a3}, {0xc7, 0x00088820},
+ {0xc8, 0x00076c06}, {0xc9, 0x00000000},
+ {0xca, 0x00080000}, {0xdf, 0x00000180},
+ {0xef, 0x000001a0}, {0x51, 0x00069545},
+ {0x52, 0x0007e45e}, {0x53, 0x00000071},
+ {0x56, 0x00051ff3}, {0x35, 0x000000a8},
+ {0x35, 0x000001e2}, {0x35, 0x000002a8},
+ {0x36, 0x00001c24}, {0x36, 0x00009c24},
+ {0x36, 0x00011c24}, {0x36, 0x00019c24},
+ {0x18, 0x00000c07}, {0x5a, 0x00048000},
+ {0x19, 0x000739d0},
+#ifdef EXT_PA_8192EU
+ /* External PA or external LNA */
+ {0x34, 0x0000a093}, {0x34, 0x0000908f},
+ {0x34, 0x0000808c}, {0x34, 0x0000704d},
+ {0x34, 0x0000604a}, {0x34, 0x00005047},
+ {0x34, 0x0000400a}, {0x34, 0x00003007},
+ {0x34, 0x00002004}, {0x34, 0x00001001},
+ {0x34, 0x00000000},
+#else
+ /* Regular */
+ {0x34, 0x0000add7}, {0x34, 0x00009dd4},
+ {0x34, 0x00008dd1}, {0x34, 0x00007dce},
+ {0x34, 0x00006dcb}, {0x34, 0x00005dc8},
+ {0x34, 0x00004dc5}, {0x34, 0x000034cc},
+ {0x34, 0x0000244f}, {0x34, 0x0000144c},
+ {0x34, 0x00000014},
+#endif
+ {0x00, 0x00030159},
+ {0x84, 0x00068180},
+ {0x86, 0x0000014e},
+ {0x87, 0x00048e00},
+ {0x8e, 0x00065540},
+ {0x8f, 0x00088000},
+ {0xef, 0x000020a0},
+#ifdef EXT_PA_8192EU
+ /* External PA or external LNA */
+ {0x3b, 0x000f07b0},
+#else
+ {0x3b, 0x000f02b0},
+#endif
+ {0x3b, 0x000ef7b0}, {0x3b, 0x000d4fb0},
+ {0x3b, 0x000cf060}, {0x3b, 0x000b0090},
+ {0x3b, 0x000a0080}, {0x3b, 0x00090080},
+ {0x3b, 0x0008f780},
+#ifdef EXT_PA_8192EU
+ /* External PA or external LNA */
+ {0x3b, 0x000787b0},
+#else
+ {0x3b, 0x00078730},
+#endif
+ {0x3b, 0x00060fb0}, {0x3b, 0x0005ffa0},
+ {0x3b, 0x00040620}, {0x3b, 0x00037090},
+ {0x3b, 0x00020080}, {0x3b, 0x0001f060},
+ {0x3b, 0x0000ffb0}, {0xef, 0x000000a0},
+ {0xfe, 0x00000000}, {0x18, 0x0000fc07},
+ {0xfe, 0x00000000}, {0xfe, 0x00000000},
+ {0xfe, 0x00000000}, {0xfe, 0x00000000},
+ {0x1e, 0x00000001}, {0x1f, 0x00080000},
+ {0x00, 0x00033e70},
+ {0xff, 0xffffffff}
+};
+
+static struct rtl8xxxu_rfregval rtl8192eu_radiob_init_table[] = {
+ {0x7f, 0x00000082}, {0x81, 0x0003fc00},
+ {0x00, 0x00030000}, {0x08, 0x00008400},
+ {0x18, 0x00000407}, {0x19, 0x00000012},
+ {0x1b, 0x00000064}, {0x1e, 0x00080009},
+ {0x1f, 0x00000880}, {0x2f, 0x0001a060},
+ {0x3f, 0x00000000}, {0x42, 0x000060c0},
+ {0x57, 0x000d0000}, {0x58, 0x000be180},
+ {0x67, 0x00001552}, {0x7f, 0x00000082},
+ {0x81, 0x0003f000}, {0x83, 0x00000000},
+ {0xdf, 0x00000180}, {0xef, 0x000001a0},
+ {0x51, 0x00069545}, {0x52, 0x0007e42e},
+ {0x53, 0x00000071}, {0x56, 0x00051ff3},
+ {0x35, 0x000000a8}, {0x35, 0x000001e0},
+ {0x35, 0x000002a8}, {0x36, 0x00001ca8},
+ {0x36, 0x00009c24}, {0x36, 0x00011c24},
+ {0x36, 0x00019c24}, {0x18, 0x00000c07},
+ {0x5a, 0x00048000}, {0x19, 0x000739d0},
+#ifdef EXT_PA_8192EU
+ /* External PA or external LNA */
+ {0x34, 0x0000a093}, {0x34, 0x0000908f},
+ {0x34, 0x0000808c}, {0x34, 0x0000704d},
+ {0x34, 0x0000604a}, {0x34, 0x00005047},
+ {0x34, 0x0000400a}, {0x34, 0x00003007},
+ {0x34, 0x00002004}, {0x34, 0x00001001},
+ {0x34, 0x00000000},
+#else
+ {0x34, 0x0000add7}, {0x34, 0x00009dd4},
+ {0x34, 0x00008dd1}, {0x34, 0x00007dce},
+ {0x34, 0x00006dcb}, {0x34, 0x00005dc8},
+ {0x34, 0x00004dc5}, {0x34, 0x000034cc},
+ {0x34, 0x0000244f}, {0x34, 0x0000144c},
+ {0x34, 0x00000014},
+#endif
+ {0x00, 0x00030159}, {0x84, 0x00068180},
+ {0x86, 0x000000ce}, {0x87, 0x00048a00},
+ {0x8e, 0x00065540}, {0x8f, 0x00088000},
+ {0xef, 0x000020a0},
+#ifdef EXT_PA_8192EU
+ /* External PA or external LNA */
+ {0x3b, 0x000f07b0},
+#else
+ {0x3b, 0x000f02b0},
+#endif
+
+ {0x3b, 0x000ef7b0}, {0x3b, 0x000d4fb0},
+ {0x3b, 0x000cf060}, {0x3b, 0x000b0090},
+ {0x3b, 0x000a0080}, {0x3b, 0x00090080},
+ {0x3b, 0x0008f780},
+#ifdef EXT_PA_8192EU
+ /* External PA or external LNA */
+ {0x3b, 0x000787b0},
+#else
+ {0x3b, 0x00078730},
+#endif
+ {0x3b, 0x00060fb0}, {0x3b, 0x0005ffa0},
+ {0x3b, 0x00040620}, {0x3b, 0x00037090},
+ {0x3b, 0x00020080}, {0x3b, 0x0001f060},
+ {0x3b, 0x0000ffb0}, {0xef, 0x000000a0},
+ {0x00, 0x00010159}, {0xfe, 0x00000000},
+ {0xfe, 0x00000000}, {0xfe, 0x00000000},
+ {0xfe, 0x00000000}, {0x1e, 0x00000001},
+ {0x1f, 0x00080000}, {0x00, 0x00033e70},
+ {0xff, 0xffffffff}
+};
+
+static void
+rtl8192e_set_tx_power(struct rtl8xxxu_priv *priv, int channel, bool ht40)
+{
+ u32 val32, ofdm, mcs;
+ u8 cck, ofdmbase, mcsbase;
+ int group, tx_idx;
+
+ tx_idx = 0;
+ group = rtl8xxxu_gen2_channel_to_group(channel);
+
+ cck = priv->cck_tx_power_index_A[group];
+
+ val32 = rtl8xxxu_read32(priv, REG_TX_AGC_A_CCK1_MCS32);
+ val32 &= 0xffff00ff;
+ val32 |= (cck << 8);
+ rtl8xxxu_write32(priv, REG_TX_AGC_A_CCK1_MCS32, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_TX_AGC_B_CCK11_A_CCK2_11);
+ val32 &= 0xff;
+ val32 |= ((cck << 8) | (cck << 16) | (cck << 24));
+ rtl8xxxu_write32(priv, REG_TX_AGC_B_CCK11_A_CCK2_11, val32);
+
+ ofdmbase = priv->ht40_1s_tx_power_index_A[group];
+ ofdmbase += priv->ofdm_tx_power_diff[tx_idx].a;
+ ofdm = ofdmbase | ofdmbase << 8 | ofdmbase << 16 | ofdmbase << 24;
+
+ rtl8xxxu_write32(priv, REG_TX_AGC_A_RATE18_06, ofdm);
+ rtl8xxxu_write32(priv, REG_TX_AGC_A_RATE54_24, ofdm);
+
+ mcsbase = priv->ht40_1s_tx_power_index_A[group];
+ if (ht40)
+ mcsbase += priv->ht40_tx_power_diff[tx_idx++].a;
+ else
+ mcsbase += priv->ht20_tx_power_diff[tx_idx++].a;
+ mcs = mcsbase | mcsbase << 8 | mcsbase << 16 | mcsbase << 24;
+
+ rtl8xxxu_write32(priv, REG_TX_AGC_A_MCS03_MCS00, mcs);
+ rtl8xxxu_write32(priv, REG_TX_AGC_A_MCS07_MCS04, mcs);
+ rtl8xxxu_write32(priv, REG_TX_AGC_A_MCS11_MCS08, mcs);
+ rtl8xxxu_write32(priv, REG_TX_AGC_A_MCS15_MCS12, mcs);
+
+ if (priv->tx_paths > 1) {
+ cck = priv->cck_tx_power_index_B[group];
+
+ val32 = rtl8xxxu_read32(priv, REG_TX_AGC_B_CCK1_55_MCS32);
+ val32 &= 0xff;
+ val32 |= ((cck << 8) | (cck << 16) | (cck << 24));
+ rtl8xxxu_write32(priv, REG_TX_AGC_B_CCK1_55_MCS32, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_TX_AGC_B_CCK11_A_CCK2_11);
+ val32 &= 0xffffff00;
+ val32 |= cck;
+ rtl8xxxu_write32(priv, REG_TX_AGC_B_CCK11_A_CCK2_11, val32);
+
+ ofdmbase = priv->ht40_1s_tx_power_index_B[group];
+ ofdmbase += priv->ofdm_tx_power_diff[tx_idx].b;
+ ofdm = ofdmbase | ofdmbase << 8 |
+ ofdmbase << 16 | ofdmbase << 24;
+
+ rtl8xxxu_write32(priv, REG_TX_AGC_B_RATE18_06, ofdm);
+ rtl8xxxu_write32(priv, REG_TX_AGC_B_RATE54_24, ofdm);
+
+ mcsbase = priv->ht40_1s_tx_power_index_B[group];
+ if (ht40)
+ mcsbase += priv->ht40_tx_power_diff[tx_idx++].b;
+ else
+ mcsbase += priv->ht20_tx_power_diff[tx_idx++].b;
+ mcs = mcsbase | mcsbase << 8 | mcsbase << 16 | mcsbase << 24;
+
+ rtl8xxxu_write32(priv, REG_TX_AGC_B_MCS03_MCS00, mcs);
+ rtl8xxxu_write32(priv, REG_TX_AGC_B_MCS07_MCS04, mcs);
+ rtl8xxxu_write32(priv, REG_TX_AGC_B_MCS11_MCS08, mcs);
+ rtl8xxxu_write32(priv, REG_TX_AGC_B_MCS15_MCS12, mcs);
+ }
+}
+
+static int rtl8192eu_parse_efuse(struct rtl8xxxu_priv *priv)
+{
+ struct rtl8192eu_efuse *efuse = &priv->efuse_wifi.efuse8192eu;
+ int i;
+
+ if (efuse->rtl_id != cpu_to_le16(0x8129))
+ return -EINVAL;
+
+ ether_addr_copy(priv->mac_addr, efuse->mac_addr);
+
+ memcpy(priv->cck_tx_power_index_A, efuse->tx_power_index_A.cck_base,
+ sizeof(efuse->tx_power_index_A.cck_base));
+ memcpy(priv->cck_tx_power_index_B, efuse->tx_power_index_B.cck_base,
+ sizeof(efuse->tx_power_index_B.cck_base));
+
+ memcpy(priv->ht40_1s_tx_power_index_A,
+ efuse->tx_power_index_A.ht40_base,
+ sizeof(efuse->tx_power_index_A.ht40_base));
+ memcpy(priv->ht40_1s_tx_power_index_B,
+ efuse->tx_power_index_B.ht40_base,
+ sizeof(efuse->tx_power_index_B.ht40_base));
+
+ priv->ht20_tx_power_diff[0].a =
+ efuse->tx_power_index_A.ht20_ofdm_1s_diff.b;
+ priv->ht20_tx_power_diff[0].b =
+ efuse->tx_power_index_B.ht20_ofdm_1s_diff.b;
+
+ priv->ht40_tx_power_diff[0].a = 0;
+ priv->ht40_tx_power_diff[0].b = 0;
+
+ for (i = 1; i < RTL8723B_TX_COUNT; i++) {
+ priv->ofdm_tx_power_diff[i].a =
+ efuse->tx_power_index_A.pwr_diff[i - 1].ofdm;
+ priv->ofdm_tx_power_diff[i].b =
+ efuse->tx_power_index_B.pwr_diff[i - 1].ofdm;
+
+ priv->ht20_tx_power_diff[i].a =
+ efuse->tx_power_index_A.pwr_diff[i - 1].ht20;
+ priv->ht20_tx_power_diff[i].b =
+ efuse->tx_power_index_B.pwr_diff[i - 1].ht20;
+
+ priv->ht40_tx_power_diff[i].a =
+ efuse->tx_power_index_A.pwr_diff[i - 1].ht40;
+ priv->ht40_tx_power_diff[i].b =
+ efuse->tx_power_index_B.pwr_diff[i - 1].ht40;
+ }
+
+ priv->has_xtalk = 1;
+ priv->xtalk = priv->efuse_wifi.efuse8192eu.xtal_k & 0x3f;
+
+ dev_info(&priv->udev->dev, "Vendor: %.7s\n", efuse->vendor_name);
+ dev_info(&priv->udev->dev, "Product: %.11s\n", efuse->device_name);
+ dev_info(&priv->udev->dev, "Serial: %.11s\n", efuse->serial);
+
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_EFUSE) {
+ unsigned char *raw = priv->efuse_wifi.raw;
+
+ dev_info(&priv->udev->dev,
+ "%s: dumping efuse (0x%02zx bytes):\n",
+ __func__, sizeof(struct rtl8192eu_efuse));
+ for (i = 0; i < sizeof(struct rtl8192eu_efuse); i += 8) {
+ dev_info(&priv->udev->dev, "%02x: "
+ "%02x %02x %02x %02x %02x %02x %02x %02x\n", i,
+ raw[i], raw[i + 1], raw[i + 2],
+ raw[i + 3], raw[i + 4], raw[i + 5],
+ raw[i + 6], raw[i + 7]);
+ }
+ }
+ return 0;
+}
+
+static int rtl8192eu_load_firmware(struct rtl8xxxu_priv *priv)
+{
+ char *fw_name;
+ int ret;
+
+ fw_name = "rtlwifi/rtl8192eu_nic.bin";
+
+ ret = rtl8xxxu_load_firmware(priv, fw_name);
+
+ return ret;
+}
+
+static void rtl8192eu_init_phy_bb(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+ u16 val16;
+
+ val16 = rtl8xxxu_read16(priv, REG_SYS_FUNC);
+ val16 |= SYS_FUNC_BB_GLB_RSTN | SYS_FUNC_BBRSTB | SYS_FUNC_DIO_RF;
+ rtl8xxxu_write16(priv, REG_SYS_FUNC, val16);
+
+ /* 6. 0x1f[7:0] = 0x07 */
+ val8 = RF_ENABLE | RF_RSTB | RF_SDMRSTB;
+ rtl8xxxu_write8(priv, REG_RF_CTRL, val8);
+
+ val16 = rtl8xxxu_read16(priv, REG_SYS_FUNC);
+ val16 |= (SYS_FUNC_USBA | SYS_FUNC_USBD | SYS_FUNC_DIO_RF |
+ SYS_FUNC_BB_GLB_RSTN | SYS_FUNC_BBRSTB);
+ rtl8xxxu_write16(priv, REG_SYS_FUNC, val16);
+ val8 = RF_ENABLE | RF_RSTB | RF_SDMRSTB;
+ rtl8xxxu_write8(priv, REG_RF_CTRL, val8);
+ rtl8xxxu_init_phy_regs(priv, rtl8192eu_phy_init_table);
+
+ if (priv->hi_pa)
+ rtl8xxxu_init_phy_regs(priv, rtl8xxx_agc_8192eu_highpa_table);
+ else
+ rtl8xxxu_init_phy_regs(priv, rtl8xxx_agc_8192eu_std_table);
+}
+
+static int rtl8192eu_init_phy_rf(struct rtl8xxxu_priv *priv)
+{
+ int ret;
+
+ ret = rtl8xxxu_init_phy_rf(priv, rtl8192eu_radioa_init_table, RF_A);
+ if (ret)
+ goto exit;
+
+ ret = rtl8xxxu_init_phy_rf(priv, rtl8192eu_radiob_init_table, RF_B);
+
+exit:
+ return ret;
+}
+
+static int rtl8192eu_iqk_path_a(struct rtl8xxxu_priv *priv)
+{
+ u32 reg_eac, reg_e94, reg_e9c;
+ int result = 0;
+
+ /*
+ * TX IQK
+ * PA/PAD controlled by 0x0
+ */
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_DF, 0x00180);
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
+
+ /* Path A IQK setting */
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x18008c1c);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x38008c1c);
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_B, 0x38008c1c);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_B, 0x38008c1c);
+
+ rtl8xxxu_write32(priv, REG_TX_IQK_PI_A, 0x82140303);
+ rtl8xxxu_write32(priv, REG_RX_IQK_PI_A, 0x68160000);
+
+ /* LO calibration setting */
+ rtl8xxxu_write32(priv, REG_IQK_AGC_RSP, 0x00462911);
+
+ /* One shot, path A LOK & IQK */
+ rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf9000000);
+ rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf8000000);
+
+ mdelay(10);
+
+ /* Check failed */
+ reg_eac = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_A_2);
+ reg_e94 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_A);
+ reg_e9c = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_A);
+
+ if (!(reg_eac & BIT(28)) &&
+ ((reg_e94 & 0x03ff0000) != 0x01420000) &&
+ ((reg_e9c & 0x03ff0000) != 0x00420000))
+ result |= 0x01;
+
+ return result;
+}
+
+static int rtl8192eu_rx_iqk_path_a(struct rtl8xxxu_priv *priv)
+{
+ u32 reg_ea4, reg_eac, reg_e94, reg_e9c, val32;
+ int result = 0;
+
+ /* Leave IQK mode */
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00);
+
+ /* Enable path A PA in TX IQK mode */
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_WE_LUT, 0x800a0);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_RCK_OS, 0x30000);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G1, 0x0000f);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G2, 0xf117b);
+
+ /* PA/PAD control by 0x56, and set = 0x0 */
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_DF, 0x00980);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_56, 0x51000);
+
+ /* Enter IQK mode */
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
+
+ /* TX IQK setting */
+ rtl8xxxu_write32(priv, REG_TX_IQK, 0x01007c00);
+ rtl8xxxu_write32(priv, REG_RX_IQK, 0x01004800);
+
+ /* path-A IQK setting */
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x18008c1c);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x38008c1c);
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_B, 0x38008c1c);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_B, 0x38008c1c);
+
+ rtl8xxxu_write32(priv, REG_TX_IQK_PI_A, 0x82160c1f);
+ rtl8xxxu_write32(priv, REG_RX_IQK_PI_A, 0x68160c1f);
+
+ /* LO calibration setting */
+ rtl8xxxu_write32(priv, REG_IQK_AGC_RSP, 0x0046a911);
+
+ /* One shot, path A LOK & IQK */
+ rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xfa000000);
+ rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf8000000);
+
+ mdelay(10);
+
+ /* Check failed */
+ reg_eac = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_A_2);
+ reg_e94 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_A);
+ reg_e9c = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_A);
+
+ if (!(reg_eac & BIT(28)) &&
+ ((reg_e94 & 0x03ff0000) != 0x01420000) &&
+ ((reg_e9c & 0x03ff0000) != 0x00420000)) {
+ result |= 0x01;
+ } else {
+ /* PA/PAD controlled by 0x0 */
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_DF, 0x180);
+ goto out;
+ }
+
+ val32 = 0x80007c00 |
+ (reg_e94 & 0x03ff0000) | ((reg_e9c >> 16) & 0x03ff);
+ rtl8xxxu_write32(priv, REG_TX_IQK, val32);
+
+ /* Modify RX IQK mode table */
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
+
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_WE_LUT, 0x800a0);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_RCK_OS, 0x30000);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G1, 0x0000f);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G2, 0xf7ffa);
+
+ /* PA/PAD control by 0x56, and set = 0x0 */
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_DF, 0x00980);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_56, 0x51000);
+
+ /* Enter IQK mode */
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
+
+ /* IQK setting */
+ rtl8xxxu_write32(priv, REG_RX_IQK, 0x01004800);
+
+ /* Path A IQK setting */
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x38008c1c);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x18008c1c);
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_B, 0x38008c1c);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_B, 0x38008c1c);
+
+ rtl8xxxu_write32(priv, REG_TX_IQK_PI_A, 0x82160c1f);
+ rtl8xxxu_write32(priv, REG_RX_IQK_PI_A, 0x28160c1f);
+
+ /* LO calibration setting */
+ rtl8xxxu_write32(priv, REG_IQK_AGC_RSP, 0x0046a891);
+
+ /* One shot, path A LOK & IQK */
+ rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xfa000000);
+ rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf8000000);
+
+ mdelay(10);
+
+ reg_eac = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_A_2);
+ reg_ea4 = rtl8xxxu_read32(priv, REG_RX_POWER_BEFORE_IQK_A_2);
+
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_DF, 0x180);
+
+ if (!(reg_eac & BIT(27)) &&
+ ((reg_ea4 & 0x03ff0000) != 0x01320000) &&
+ ((reg_eac & 0x03ff0000) != 0x00360000))
+ result |= 0x02;
+ else
+ dev_warn(&priv->udev->dev, "%s: Path A RX IQK failed!\n",
+ __func__);
+
+out:
+ return result;
+}
+
+static int rtl8192eu_iqk_path_b(struct rtl8xxxu_priv *priv)
+{
+ u32 reg_eac, reg_eb4, reg_ebc;
+ int result = 0;
+
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
+ rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_UNKNOWN_DF, 0x00180);
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
+
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
+
+ /* Path B IQK setting */
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x38008c1c);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x38008c1c);
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_B, 0x18008c1c);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_B, 0x38008c1c);
+
+ rtl8xxxu_write32(priv, REG_TX_IQK_PI_B, 0x821403e2);
+ rtl8xxxu_write32(priv, REG_RX_IQK_PI_B, 0x68160000);
+
+ /* LO calibration setting */
+ rtl8xxxu_write32(priv, REG_IQK_AGC_RSP, 0x00492911);
+
+ /* One shot, path A LOK & IQK */
+ rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xfa000000);
+ rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf8000000);
+
+ mdelay(1);
+
+ /* Check failed */
+ reg_eac = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_A_2);
+ reg_eb4 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_B);
+ reg_ebc = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_B);
+
+ if (!(reg_eac & BIT(31)) &&
+ ((reg_eb4 & 0x03ff0000) != 0x01420000) &&
+ ((reg_ebc & 0x03ff0000) != 0x00420000))
+ result |= 0x01;
+ else
+ dev_warn(&priv->udev->dev, "%s: Path B IQK failed!\n",
+ __func__);
+
+ return result;
+}
+
+static int rtl8192eu_rx_iqk_path_b(struct rtl8xxxu_priv *priv)
+{
+ u32 reg_eac, reg_eb4, reg_ebc, reg_ec4, reg_ecc, val32;
+ int result = 0;
+
+ /* Leave IQK mode */
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
+
+ /* Enable path A PA in TX IQK mode */
+ rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_WE_LUT, 0x800a0);
+ rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_RCK_OS, 0x30000);
+ rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_TXPA_G1, 0x0000f);
+ rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_TXPA_G2, 0xf117b);
+
+ /* PA/PAD control by 0x56, and set = 0x0 */
+ rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_UNKNOWN_DF, 0x00980);
+ rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_UNKNOWN_56, 0x51000);
+
+ /* Enter IQK mode */
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
+
+ /* TX IQK setting */
+ rtl8xxxu_write32(priv, REG_TX_IQK, 0x01007c00);
+ rtl8xxxu_write32(priv, REG_RX_IQK, 0x01004800);
+
+ /* path-A IQK setting */
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x38008c1c);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x38008c1c);
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_B, 0x18008c1c);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_B, 0x38008c1c);
+
+ rtl8xxxu_write32(priv, REG_TX_IQK_PI_B, 0x82160c1f);
+ rtl8xxxu_write32(priv, REG_RX_IQK_PI_B, 0x68160c1f);
+
+ /* LO calibration setting */
+ rtl8xxxu_write32(priv, REG_IQK_AGC_RSP, 0x0046a911);
+
+ /* One shot, path A LOK & IQK */
+ rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xfa000000);
+ rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf8000000);
+
+ mdelay(10);
+
+ /* Check failed */
+ reg_eac = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_A_2);
+ reg_eb4 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_B);
+ reg_ebc = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_B);
+
+ if (!(reg_eac & BIT(31)) &&
+ ((reg_eb4 & 0x03ff0000) != 0x01420000) &&
+ ((reg_ebc & 0x03ff0000) != 0x00420000)) {
+ result |= 0x01;
+ } else {
+ /*
+ * PA/PAD controlled by 0x0
+ * Vendor driver restores RF_A here which I believe is a bug
+ */
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
+ rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_UNKNOWN_DF, 0x180);
+ goto out;
+ }
+
+ val32 = 0x80007c00 |
+ (reg_eb4 & 0x03ff0000) | ((reg_ebc >> 16) & 0x03ff);
+ rtl8xxxu_write32(priv, REG_TX_IQK, val32);
+
+ /* Modify RX IQK mode table */
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
+
+ rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_WE_LUT, 0x800a0);
+ rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_RCK_OS, 0x30000);
+ rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_TXPA_G1, 0x0000f);
+ rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_TXPA_G2, 0xf7ffa);
+
+ /* PA/PAD control by 0x56, and set = 0x0 */
+ rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_UNKNOWN_DF, 0x00980);
+ rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_UNKNOWN_56, 0x51000);
+
+ /* Enter IQK mode */
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
+
+ /* IQK setting */
+ rtl8xxxu_write32(priv, REG_RX_IQK, 0x01004800);
+
+ /* Path A IQK setting */
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x38008c1c);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x38008c1c);
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_B, 0x38008c1c);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_B, 0x18008c1c);
+
+ rtl8xxxu_write32(priv, REG_TX_IQK_PI_A, 0x82160c1f);
+ rtl8xxxu_write32(priv, REG_RX_IQK_PI_A, 0x28160c1f);
+
+ /* LO calibration setting */
+ rtl8xxxu_write32(priv, REG_IQK_AGC_RSP, 0x0046a891);
+
+ /* One shot, path A LOK & IQK */
+ rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xfa000000);
+ rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf8000000);
+
+ mdelay(10);
+
+ reg_eac = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_A_2);
+ reg_ec4 = rtl8xxxu_read32(priv, REG_RX_POWER_BEFORE_IQK_B_2);
+ reg_ecc = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_B_2);
+
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
+ rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_UNKNOWN_DF, 0x180);
+
+ if (!(reg_eac & BIT(30)) &&
+ ((reg_ec4 & 0x03ff0000) != 0x01320000) &&
+ ((reg_ecc & 0x03ff0000) != 0x00360000))
+ result |= 0x02;
+ else
+ dev_warn(&priv->udev->dev, "%s: Path B RX IQK failed!\n",
+ __func__);
+
+out:
+ return result;
+}
+
+static void rtl8192eu_phy_iqcalibrate(struct rtl8xxxu_priv *priv,
+ int result[][8], int t)
+{
+ struct device *dev = &priv->udev->dev;
+ u32 i, val32;
+ int path_a_ok, path_b_ok;
+ int retry = 2;
+ const u32 adda_regs[RTL8XXXU_ADDA_REGS] = {
+ REG_FPGA0_XCD_SWITCH_CTRL, REG_BLUETOOTH,
+ REG_RX_WAIT_CCA, REG_TX_CCK_RFON,
+ REG_TX_CCK_BBON, REG_TX_OFDM_RFON,
+ REG_TX_OFDM_BBON, REG_TX_TO_RX,
+ REG_TX_TO_TX, REG_RX_CCK,
+ REG_RX_OFDM, REG_RX_WAIT_RIFS,
+ REG_RX_TO_RX, REG_STANDBY,
+ REG_SLEEP, REG_PMPD_ANAEN
+ };
+ const u32 iqk_mac_regs[RTL8XXXU_MAC_REGS] = {
+ REG_TXPAUSE, REG_BEACON_CTRL,
+ REG_BEACON_CTRL_1, REG_GPIO_MUXCFG
+ };
+ const u32 iqk_bb_regs[RTL8XXXU_BB_REGS] = {
+ REG_OFDM0_TRX_PATH_ENABLE, REG_OFDM0_TR_MUX_PAR,
+ REG_FPGA0_XCD_RF_SW_CTRL, REG_CONFIG_ANT_A, REG_CONFIG_ANT_B,
+ REG_FPGA0_XAB_RF_SW_CTRL, REG_FPGA0_XA_RF_INT_OE,
+ REG_FPGA0_XB_RF_INT_OE, REG_CCK0_AFE_SETTING
+ };
+ u8 xa_agc = rtl8xxxu_read32(priv, REG_OFDM0_XA_AGC_CORE1) & 0xff;
+ u8 xb_agc = rtl8xxxu_read32(priv, REG_OFDM0_XB_AGC_CORE1) & 0xff;
+
+ /*
+ * Note: IQ calibration must be performed after loading
+ * PHY_REG.txt , and radio_a, radio_b.txt
+ */
+
+ if (t == 0) {
+ /* Save ADDA parameters, turn Path A ADDA on */
+ rtl8xxxu_save_regs(priv, adda_regs, priv->adda_backup,
+ RTL8XXXU_ADDA_REGS);
+ rtl8xxxu_save_mac_regs(priv, iqk_mac_regs, priv->mac_backup);
+ rtl8xxxu_save_regs(priv, iqk_bb_regs,
+ priv->bb_backup, RTL8XXXU_BB_REGS);
+ }
+
+ rtl8xxxu_path_adda_on(priv, adda_regs, true);
+
+ /* MAC settings */
+ rtl8xxxu_mac_calibration(priv, iqk_mac_regs, priv->mac_backup);
+
+ val32 = rtl8xxxu_read32(priv, REG_CCK0_AFE_SETTING);
+ val32 |= 0x0f000000;
+ rtl8xxxu_write32(priv, REG_CCK0_AFE_SETTING, val32);
+
+ rtl8xxxu_write32(priv, REG_OFDM0_TRX_PATH_ENABLE, 0x03a05600);
+ rtl8xxxu_write32(priv, REG_OFDM0_TR_MUX_PAR, 0x000800e4);
+ rtl8xxxu_write32(priv, REG_FPGA0_XCD_RF_SW_CTRL, 0x22208200);
+
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_XAB_RF_SW_CTRL);
+ val32 |= (FPGA0_RF_PAPE | (FPGA0_RF_PAPE << FPGA0_RF_BD_CTRL_SHIFT));
+ rtl8xxxu_write32(priv, REG_FPGA0_XAB_RF_SW_CTRL, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_XA_RF_INT_OE);
+ val32 |= BIT(10);
+ rtl8xxxu_write32(priv, REG_FPGA0_XA_RF_INT_OE, val32);
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_XB_RF_INT_OE);
+ val32 |= BIT(10);
+ rtl8xxxu_write32(priv, REG_FPGA0_XB_RF_INT_OE, val32);
+
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
+ rtl8xxxu_write32(priv, REG_TX_IQK, 0x01007c00);
+ rtl8xxxu_write32(priv, REG_RX_IQK, 0x01004800);
+
+ for (i = 0; i < retry; i++) {
+ path_a_ok = rtl8192eu_iqk_path_a(priv);
+ if (path_a_ok == 0x01) {
+ val32 = rtl8xxxu_read32(priv,
+ REG_TX_POWER_BEFORE_IQK_A);
+ result[t][0] = (val32 >> 16) & 0x3ff;
+ val32 = rtl8xxxu_read32(priv,
+ REG_TX_POWER_AFTER_IQK_A);
+ result[t][1] = (val32 >> 16) & 0x3ff;
+
+ break;
+ }
+ }
+
+ if (!path_a_ok)
+ dev_dbg(dev, "%s: Path A TX IQK failed!\n", __func__);
+
+ for (i = 0; i < retry; i++) {
+ path_a_ok = rtl8192eu_rx_iqk_path_a(priv);
+ if (path_a_ok == 0x03) {
+ val32 = rtl8xxxu_read32(priv,
+ REG_RX_POWER_BEFORE_IQK_A_2);
+ result[t][2] = (val32 >> 16) & 0x3ff;
+ val32 = rtl8xxxu_read32(priv,
+ REG_RX_POWER_AFTER_IQK_A_2);
+ result[t][3] = (val32 >> 16) & 0x3ff;
+
+ break;
+ }
+ }
+
+ if (!path_a_ok)
+ dev_dbg(dev, "%s: Path A RX IQK failed!\n", __func__);
+
+ if (priv->rf_paths > 1) {
+ /* Path A into standby */
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_AC, 0x10000);
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
+
+ /* Turn Path B ADDA on */
+ rtl8xxxu_path_adda_on(priv, adda_regs, false);
+
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
+ rtl8xxxu_write32(priv, REG_TX_IQK, 0x01007c00);
+ rtl8xxxu_write32(priv, REG_RX_IQK, 0x01004800);
+
+ for (i = 0; i < retry; i++) {
+ path_b_ok = rtl8192eu_iqk_path_b(priv);
+ if (path_b_ok == 0x01) {
+ val32 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_B);
+ result[t][4] = (val32 >> 16) & 0x3ff;
+ val32 = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_B);
+ result[t][5] = (val32 >> 16) & 0x3ff;
+ break;
+ }
+ }
+
+ if (!path_b_ok)
+ dev_dbg(dev, "%s: Path B IQK failed!\n", __func__);
+
+ for (i = 0; i < retry; i++) {
+ path_b_ok = rtl8192eu_rx_iqk_path_b(priv);
+ if (path_a_ok == 0x03) {
+ val32 = rtl8xxxu_read32(priv,
+ REG_RX_POWER_BEFORE_IQK_B_2);
+ result[t][6] = (val32 >> 16) & 0x3ff;
+ val32 = rtl8xxxu_read32(priv,
+ REG_RX_POWER_AFTER_IQK_B_2);
+ result[t][7] = (val32 >> 16) & 0x3ff;
+ break;
+ }
+ }
+
+ if (!path_b_ok)
+ dev_dbg(dev, "%s: Path B RX IQK failed!\n", __func__);
+ }
+
+ /* Back to BB mode, load original value */
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x00000000);
+
+ if (t) {
+ /* Reload ADDA power saving parameters */
+ rtl8xxxu_restore_regs(priv, adda_regs, priv->adda_backup,
+ RTL8XXXU_ADDA_REGS);
+
+ /* Reload MAC parameters */
+ rtl8xxxu_restore_mac_regs(priv, iqk_mac_regs, priv->mac_backup);
+
+ /* Reload BB parameters */
+ rtl8xxxu_restore_regs(priv, iqk_bb_regs,
+ priv->bb_backup, RTL8XXXU_BB_REGS);
+
+ /* Restore RX initial gain */
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_XA_AGC_CORE1);
+ val32 &= 0xffffff00;
+ rtl8xxxu_write32(priv, REG_OFDM0_XA_AGC_CORE1, val32 | 0x50);
+ rtl8xxxu_write32(priv, REG_OFDM0_XA_AGC_CORE1, val32 | xa_agc);
+
+ if (priv->rf_paths > 1) {
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_XB_AGC_CORE1);
+ val32 &= 0xffffff00;
+ rtl8xxxu_write32(priv, REG_OFDM0_XB_AGC_CORE1,
+ val32 | 0x50);
+ rtl8xxxu_write32(priv, REG_OFDM0_XB_AGC_CORE1,
+ val32 | xb_agc);
+ }
+
+ /* Load 0xe30 IQC default value */
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x01008c00);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x01008c00);
+ }
+}
+
+static void rtl8192eu_phy_iq_calibrate(struct rtl8xxxu_priv *priv)
+{
+ struct device *dev = &priv->udev->dev;
+ int result[4][8]; /* last is final result */
+ int i, candidate;
+ bool path_a_ok, path_b_ok;
+ u32 reg_e94, reg_e9c, reg_ea4, reg_eac;
+ u32 reg_eb4, reg_ebc, reg_ec4, reg_ecc;
+ bool simu;
+
+ memset(result, 0, sizeof(result));
+ candidate = -1;
+
+ path_a_ok = false;
+ path_b_ok = false;
+
+ for (i = 0; i < 3; i++) {
+ rtl8192eu_phy_iqcalibrate(priv, result, i);
+
+ if (i == 1) {
+ simu = rtl8xxxu_gen2_simularity_compare(priv,
+ result, 0, 1);
+ if (simu) {
+ candidate = 0;
+ break;
+ }
+ }
+
+ if (i == 2) {
+ simu = rtl8xxxu_gen2_simularity_compare(priv,
+ result, 0, 2);
+ if (simu) {
+ candidate = 0;
+ break;
+ }
+
+ simu = rtl8xxxu_gen2_simularity_compare(priv,
+ result, 1, 2);
+ if (simu)
+ candidate = 1;
+ else
+ candidate = 3;
+ }
+ }
+
+ for (i = 0; i < 4; i++) {
+ reg_e94 = result[i][0];
+ reg_e9c = result[i][1];
+ reg_ea4 = result[i][2];
+ reg_eac = result[i][3];
+ reg_eb4 = result[i][4];
+ reg_ebc = result[i][5];
+ reg_ec4 = result[i][6];
+ reg_ecc = result[i][7];
+ }
+
+ if (candidate >= 0) {
+ reg_e94 = result[candidate][0];
+ priv->rege94 = reg_e94;
+ reg_e9c = result[candidate][1];
+ priv->rege9c = reg_e9c;
+ reg_ea4 = result[candidate][2];
+ reg_eac = result[candidate][3];
+ reg_eb4 = result[candidate][4];
+ priv->regeb4 = reg_eb4;
+ reg_ebc = result[candidate][5];
+ priv->regebc = reg_ebc;
+ reg_ec4 = result[candidate][6];
+ reg_ecc = result[candidate][7];
+ dev_dbg(dev, "%s: candidate is %x\n", __func__, candidate);
+ dev_dbg(dev,
+ "%s: e94 =%x e9c=%x ea4=%x eac=%x eb4=%x ebc=%x ec4=%x "
+ "ecc=%x\n ", __func__, reg_e94, reg_e9c,
+ reg_ea4, reg_eac, reg_eb4, reg_ebc, reg_ec4, reg_ecc);
+ path_a_ok = true;
+ path_b_ok = true;
+ } else {
+ reg_e94 = reg_eb4 = priv->rege94 = priv->regeb4 = 0x100;
+ reg_e9c = reg_ebc = priv->rege9c = priv->regebc = 0x0;
+ }
+
+ if (reg_e94 && candidate >= 0)
+ rtl8xxxu_fill_iqk_matrix_a(priv, path_a_ok, result,
+ candidate, (reg_ea4 == 0));
+
+ if (priv->rf_paths > 1)
+ rtl8xxxu_fill_iqk_matrix_b(priv, path_b_ok, result,
+ candidate, (reg_ec4 == 0));
+
+ rtl8xxxu_save_regs(priv, rtl8xxxu_iqk_phy_iq_bb_reg,
+ priv->bb_recovery_backup, RTL8XXXU_BB_REGS);
+}
+
+/*
+ * This is needed for 8723bu as well, presumable
+ */
+static void rtl8192e_crystal_afe_adjust(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+ u32 val32;
+
+ /*
+ * 40Mhz crystal source, MAC 0x28[2]=0
+ */
+ val8 = rtl8xxxu_read8(priv, REG_AFE_PLL_CTRL);
+ val8 &= 0xfb;
+ rtl8xxxu_write8(priv, REG_AFE_PLL_CTRL, val8);
+
+ val32 = rtl8xxxu_read32(priv, REG_AFE_CTRL4);
+ val32 &= 0xfffffc7f;
+ rtl8xxxu_write32(priv, REG_AFE_CTRL4, val32);
+
+ /*
+ * 92e AFE parameter
+ * AFE PLL KVCO selection, MAC 0x28[6]=1
+ */
+ val8 = rtl8xxxu_read8(priv, REG_AFE_PLL_CTRL);
+ val8 &= 0xbf;
+ rtl8xxxu_write8(priv, REG_AFE_PLL_CTRL, val8);
+
+ /*
+ * AFE PLL KVCO selection, MAC 0x78[21]=0
+ */
+ val32 = rtl8xxxu_read32(priv, REG_AFE_CTRL4);
+ val32 &= 0xffdfffff;
+ rtl8xxxu_write32(priv, REG_AFE_CTRL4, val32);
+}
+
+static void rtl8192e_disabled_to_emu(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+
+ /* Clear suspend enable and power down enable*/
+ val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
+ val8 &= ~(BIT(3) | BIT(4));
+ rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
+}
+
+static int rtl8192e_emu_to_active(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+ u32 val32;
+ int count, ret = 0;
+
+ /* disable HWPDN 0x04[15]=0*/
+ val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
+ val8 &= ~BIT(7);
+ rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
+
+ /* disable SW LPS 0x04[10]= 0 */
+ val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
+ val8 &= ~BIT(2);
+ rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
+
+ /* disable WL suspend*/
+ val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
+ val8 &= ~(BIT(3) | BIT(4));
+ rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
+
+ /* wait till 0x04[17] = 1 power ready*/
+ for (count = RTL8XXXU_MAX_REG_POLL; count; count--) {
+ val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
+ if (val32 & BIT(17))
+ break;
+
+ udelay(10);
+ }
+
+ if (!count) {
+ ret = -EBUSY;
+ goto exit;
+ }
+
+ /* We should be able to optimize the following three entries into one */
+
+ /* release WLON reset 0x04[16]= 1*/
+ val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 2);
+ val8 |= BIT(0);
+ rtl8xxxu_write8(priv, REG_APS_FSMCO + 2, val8);
+
+ /* set, then poll until 0 */
+ val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
+ val32 |= APS_FSMCO_MAC_ENABLE;
+ rtl8xxxu_write32(priv, REG_APS_FSMCO, val32);
+
+ for (count = RTL8XXXU_MAX_REG_POLL; count; count--) {
+ val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
+ if ((val32 & APS_FSMCO_MAC_ENABLE) == 0) {
+ ret = 0;
+ break;
+ }
+ udelay(10);
+ }
+
+ if (!count) {
+ ret = -EBUSY;
+ goto exit;
+ }
+
+exit:
+ return ret;
+}
+
+static int rtl8192eu_power_on(struct rtl8xxxu_priv *priv)
+{
+ u16 val16;
+ u32 val32;
+ int ret;
+
+ ret = 0;
+
+ val32 = rtl8xxxu_read32(priv, REG_SYS_CFG);
+ if (val32 & SYS_CFG_SPS_LDO_SEL) {
+ rtl8xxxu_write8(priv, REG_LDO_SW_CTRL, 0xc3);
+ } else {
+ /*
+ * Raise 1.2V voltage
+ */
+ val32 = rtl8xxxu_read32(priv, REG_8192E_LDOV12_CTRL);
+ val32 &= 0xff0fffff;
+ val32 |= 0x00500000;
+ rtl8xxxu_write32(priv, REG_8192E_LDOV12_CTRL, val32);
+ rtl8xxxu_write8(priv, REG_LDO_SW_CTRL, 0x83);
+ }
+
+ /*
+ * Adjust AFE before enabling PLL
+ */
+ rtl8192e_crystal_afe_adjust(priv);
+ rtl8192e_disabled_to_emu(priv);
+
+ ret = rtl8192e_emu_to_active(priv);
+ if (ret)
+ goto exit;
+
+ rtl8xxxu_write16(priv, REG_CR, 0x0000);
+
+ /*
+ * Enable MAC DMA/WMAC/SCHEDULE/SEC block
+ * Set CR bit10 to enable 32k calibration.
+ */
+ val16 = rtl8xxxu_read16(priv, REG_CR);
+ val16 |= (CR_HCI_TXDMA_ENABLE | CR_HCI_RXDMA_ENABLE |
+ CR_TXDMA_ENABLE | CR_RXDMA_ENABLE |
+ CR_PROTOCOL_ENABLE | CR_SCHEDULE_ENABLE |
+ CR_MAC_TX_ENABLE | CR_MAC_RX_ENABLE |
+ CR_SECURITY_ENABLE | CR_CALTIMER_ENABLE);
+ rtl8xxxu_write16(priv, REG_CR, val16);
+
+exit:
+ return ret;
+}
+
+static void rtl8192e_enable_rf(struct rtl8xxxu_priv *priv)
+{
+ u32 val32;
+ u8 val8;
+
+ val8 = rtl8xxxu_read8(priv, REG_GPIO_MUXCFG);
+ val8 |= BIT(5);
+ rtl8xxxu_write8(priv, REG_GPIO_MUXCFG, val8);
+
+ /*
+ * WLAN action by PTA
+ */
+ rtl8xxxu_write8(priv, REG_WLAN_ACT_CONTROL_8723B, 0x04);
+
+ val32 = rtl8xxxu_read32(priv, REG_PWR_DATA);
+ val32 |= PWR_DATA_EEPRPAD_RFE_CTRL_EN;
+ rtl8xxxu_write32(priv, REG_PWR_DATA, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_RFE_BUFFER);
+ val32 |= (BIT(0) | BIT(1));
+ rtl8xxxu_write32(priv, REG_RFE_BUFFER, val32);
+
+ rtl8xxxu_write8(priv, REG_RFE_CTRL_ANTA_SRC, 0x77);
+
+ val32 = rtl8xxxu_read32(priv, REG_LEDCFG0);
+ val32 &= ~BIT(24);
+ val32 |= BIT(23);
+ rtl8xxxu_write32(priv, REG_LEDCFG0, val32);
+
+ /*
+ * Fix external switch Main->S1, Aux->S0
+ */
+ val8 = rtl8xxxu_read8(priv, REG_PAD_CTRL1);
+ val8 &= ~BIT(0);
+ rtl8xxxu_write8(priv, REG_PAD_CTRL1, val8);
+}
+
+struct rtl8xxxu_fileops rtl8192eu_fops = {
+ .parse_efuse = rtl8192eu_parse_efuse,
+ .load_firmware = rtl8192eu_load_firmware,
+ .power_on = rtl8192eu_power_on,
+ .power_off = rtl8xxxu_power_off,
+ .reset_8051 = rtl8xxxu_reset_8051,
+ .llt_init = rtl8xxxu_auto_llt_table,
+ .init_phy_bb = rtl8192eu_init_phy_bb,
+ .init_phy_rf = rtl8192eu_init_phy_rf,
+ .phy_iq_calibrate = rtl8192eu_phy_iq_calibrate,
+ .config_channel = rtl8xxxu_gen2_config_channel,
+ .parse_rx_desc = rtl8xxxu_parse_rxdesc24,
+ .enable_rf = rtl8192e_enable_rf,
+ .disable_rf = rtl8xxxu_gen2_disable_rf,
+ .usb_quirks = rtl8xxxu_gen2_usb_quirks,
+ .set_tx_power = rtl8192e_set_tx_power,
+ .update_rate_mask = rtl8xxxu_gen2_update_rate_mask,
+ .report_connect = rtl8xxxu_gen2_report_connect,
+ .writeN_block_size = 128,
+ .tx_desc_size = sizeof(struct rtl8xxxu_txdesc40),
+ .rx_desc_size = sizeof(struct rtl8xxxu_rxdesc24),
+ .has_s0s1 = 0,
+ .adda_1t_init = 0x0fc01616,
+ .adda_1t_path_on = 0x0fc01616,
+ .adda_2t_path_on_a = 0x0fc01616,
+ .adda_2t_path_on_b = 0x0fc01616,
+ .trxff_boundary = 0x3cff,
+ .mactable = rtl8192e_mac_init_table,
+ .total_page_num = TX_TOTAL_PAGE_NUM_8192E,
+ .page_num_hi = TX_PAGE_NUM_HI_PQ_8192E,
+ .page_num_lo = TX_PAGE_NUM_LO_PQ_8192E,
+ .page_num_norm = TX_PAGE_NUM_NORM_PQ_8192E,
+};
diff --git a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8723a.c b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8723a.c
new file mode 100644
index 0000000..a8e172c
--- /dev/null
+++ b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8723a.c
@@ -0,0 +1,397 @@
+/*
+ * RTL8XXXU mac80211 USB driver - 8723a specific subdriver
+ *
+ * Copyright (c) 2014 - 2016 Jes Sorensen <Jes.Sorensen@redhat.com>
+ *
+ * Portions, notably calibration code:
+ * Copyright(c) 2007 - 2011 Realtek Corporation. All rights reserved.
+ *
+ * This driver was written as a replacement for the vendor provided
+ * rtl8723au driver. As the Realtek 8xxx chips are very similar in
+ * their programming interface, I have started adding support for
+ * additional 8xxx chips like the 8192cu, 8188cus, etc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ */
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <linux/errno.h>
+#include <linux/slab.h>
+#include <linux/module.h>
+#include <linux/spinlock.h>
+#include <linux/list.h>
+#include <linux/usb.h>
+#include <linux/netdevice.h>
+#include <linux/etherdevice.h>
+#include <linux/ethtool.h>
+#include <linux/wireless.h>
+#include <linux/firmware.h>
+#include <linux/moduleparam.h>
+#include <net/mac80211.h>
+#include "rtl8xxxu.h"
+#include "rtl8xxxu_regs.h"
+
+static struct rtl8xxxu_power_base rtl8723a_power_base = {
+ .reg_0e00 = 0x0a0c0c0c,
+ .reg_0e04 = 0x02040608,
+ .reg_0e08 = 0x00000000,
+ .reg_086c = 0x00000000,
+
+ .reg_0e10 = 0x0a0c0d0e,
+ .reg_0e14 = 0x02040608,
+ .reg_0e18 = 0x0a0c0d0e,
+ .reg_0e1c = 0x02040608,
+
+ .reg_0830 = 0x0a0c0c0c,
+ .reg_0834 = 0x02040608,
+ .reg_0838 = 0x00000000,
+ .reg_086c_2 = 0x00000000,
+
+ .reg_083c = 0x0a0c0d0e,
+ .reg_0848 = 0x02040608,
+ .reg_084c = 0x0a0c0d0e,
+ .reg_0868 = 0x02040608,
+};
+
+static struct rtl8xxxu_rfregval rtl8723au_radioa_1t_init_table[] = {
+ {0x00, 0x00030159}, {0x01, 0x00031284},
+ {0x02, 0x00098000}, {0x03, 0x00039c63},
+ {0x04, 0x000210e7}, {0x09, 0x0002044f},
+ {0x0a, 0x0001a3f1}, {0x0b, 0x00014787},
+ {0x0c, 0x000896fe}, {0x0d, 0x0000e02c},
+ {0x0e, 0x00039ce7}, {0x0f, 0x00000451},
+ {0x19, 0x00000000}, {0x1a, 0x00030355},
+ {0x1b, 0x00060a00}, {0x1c, 0x000fc378},
+ {0x1d, 0x000a1250}, {0x1e, 0x0000024f},
+ {0x1f, 0x00000000}, {0x20, 0x0000b614},
+ {0x21, 0x0006c000}, {0x22, 0x00000000},
+ {0x23, 0x00001558}, {0x24, 0x00000060},
+ {0x25, 0x00000483}, {0x26, 0x0004f000},
+ {0x27, 0x000ec7d9}, {0x28, 0x00057730},
+ {0x29, 0x00004783}, {0x2a, 0x00000001},
+ {0x2b, 0x00021334}, {0x2a, 0x00000000},
+ {0x2b, 0x00000054}, {0x2a, 0x00000001},
+ {0x2b, 0x00000808}, {0x2b, 0x00053333},
+ {0x2c, 0x0000000c}, {0x2a, 0x00000002},
+ {0x2b, 0x00000808}, {0x2b, 0x0005b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000003},
+ {0x2b, 0x00000808}, {0x2b, 0x00063333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000004},
+ {0x2b, 0x00000808}, {0x2b, 0x0006b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000005},
+ {0x2b, 0x00000808}, {0x2b, 0x00073333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000006},
+ {0x2b, 0x00000709}, {0x2b, 0x0005b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000007},
+ {0x2b, 0x00000709}, {0x2b, 0x00063333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000008},
+ {0x2b, 0x0000060a}, {0x2b, 0x0004b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x00000009},
+ {0x2b, 0x0000060a}, {0x2b, 0x00053333},
+ {0x2c, 0x0000000d}, {0x2a, 0x0000000a},
+ {0x2b, 0x0000060a}, {0x2b, 0x0005b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x0000000b},
+ {0x2b, 0x0000060a}, {0x2b, 0x00063333},
+ {0x2c, 0x0000000d}, {0x2a, 0x0000000c},
+ {0x2b, 0x0000060a}, {0x2b, 0x0006b333},
+ {0x2c, 0x0000000d}, {0x2a, 0x0000000d},
+ {0x2b, 0x0000060a}, {0x2b, 0x00073333},
+ {0x2c, 0x0000000d}, {0x2a, 0x0000000e},
+ {0x2b, 0x0000050b}, {0x2b, 0x00066666},
+ {0x2c, 0x0000001a}, {0x2a, 0x000e0000},
+ {0x10, 0x0004000f}, {0x11, 0x000e31fc},
+ {0x10, 0x0006000f}, {0x11, 0x000ff9f8},
+ {0x10, 0x0002000f}, {0x11, 0x000203f9},
+ {0x10, 0x0003000f}, {0x11, 0x000ff500},
+ {0x10, 0x00000000}, {0x11, 0x00000000},
+ {0x10, 0x0008000f}, {0x11, 0x0003f100},
+ {0x10, 0x0009000f}, {0x11, 0x00023100},
+ {0x12, 0x00032000}, {0x12, 0x00071000},
+ {0x12, 0x000b0000}, {0x12, 0x000fc000},
+ {0x13, 0x000287b3}, {0x13, 0x000244b7},
+ {0x13, 0x000204ab}, {0x13, 0x0001c49f},
+ {0x13, 0x00018493}, {0x13, 0x0001429b},
+ {0x13, 0x00010299}, {0x13, 0x0000c29c},
+ {0x13, 0x000081a0}, {0x13, 0x000040ac},
+ {0x13, 0x00000020}, {0x14, 0x0001944c},
+ {0x14, 0x00059444}, {0x14, 0x0009944c},
+ {0x14, 0x000d9444}, {0x15, 0x0000f474},
+ {0x15, 0x0004f477}, {0x15, 0x0008f455},
+ {0x15, 0x000cf455}, {0x16, 0x00000339},
+ {0x16, 0x00040339}, {0x16, 0x00080339},
+ {0x16, 0x000c0366}, {0x00, 0x00010159},
+ {0x18, 0x0000f401}, {0xfe, 0x00000000},
+ {0xfe, 0x00000000}, {0x1f, 0x00000003},
+ {0xfe, 0x00000000}, {0xfe, 0x00000000},
+ {0x1e, 0x00000247}, {0x1f, 0x00000000},
+ {0x00, 0x00030159},
+ {0xff, 0xffffffff}
+};
+
+static int rtl8723au_parse_efuse(struct rtl8xxxu_priv *priv)
+{
+ struct rtl8723au_efuse *efuse = &priv->efuse_wifi.efuse8723;
+
+ if (efuse->rtl_id != cpu_to_le16(0x8129))
+ return -EINVAL;
+
+ ether_addr_copy(priv->mac_addr, efuse->mac_addr);
+
+ memcpy(priv->cck_tx_power_index_A,
+ efuse->cck_tx_power_index_A,
+ sizeof(efuse->cck_tx_power_index_A));
+ memcpy(priv->cck_tx_power_index_B,
+ efuse->cck_tx_power_index_B,
+ sizeof(efuse->cck_tx_power_index_B));
+
+ memcpy(priv->ht40_1s_tx_power_index_A,
+ efuse->ht40_1s_tx_power_index_A,
+ sizeof(efuse->ht40_1s_tx_power_index_A));
+ memcpy(priv->ht40_1s_tx_power_index_B,
+ efuse->ht40_1s_tx_power_index_B,
+ sizeof(efuse->ht40_1s_tx_power_index_B));
+
+ memcpy(priv->ht20_tx_power_index_diff,
+ efuse->ht20_tx_power_index_diff,
+ sizeof(efuse->ht20_tx_power_index_diff));
+ memcpy(priv->ofdm_tx_power_index_diff,
+ efuse->ofdm_tx_power_index_diff,
+ sizeof(efuse->ofdm_tx_power_index_diff));
+
+ memcpy(priv->ht40_max_power_offset,
+ efuse->ht40_max_power_offset,
+ sizeof(efuse->ht40_max_power_offset));
+ memcpy(priv->ht20_max_power_offset,
+ efuse->ht20_max_power_offset,
+ sizeof(efuse->ht20_max_power_offset));
+
+ if (priv->efuse_wifi.efuse8723.version >= 0x01) {
+ priv->has_xtalk = 1;
+ priv->xtalk = priv->efuse_wifi.efuse8723.xtal_k & 0x3f;
+ }
+
+ priv->power_base = &rtl8723a_power_base;
+
+ dev_info(&priv->udev->dev, "Vendor: %.7s\n",
+ efuse->vendor_name);
+ dev_info(&priv->udev->dev, "Product: %.41s\n",
+ efuse->device_name);
+ return 0;
+}
+
+static int rtl8723au_load_firmware(struct rtl8xxxu_priv *priv)
+{
+ char *fw_name;
+ int ret;
+
+ switch (priv->chip_cut) {
+ case 0:
+ fw_name = "rtlwifi/rtl8723aufw_A.bin";
+ break;
+ case 1:
+ if (priv->enable_bluetooth)
+ fw_name = "rtlwifi/rtl8723aufw_B.bin";
+ else
+ fw_name = "rtlwifi/rtl8723aufw_B_NoBT.bin";
+
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ ret = rtl8xxxu_load_firmware(priv, fw_name);
+ return ret;
+}
+
+static int rtl8723au_init_phy_rf(struct rtl8xxxu_priv *priv)
+{
+ int ret;
+
+ ret = rtl8xxxu_init_phy_rf(priv, rtl8723au_radioa_1t_init_table, RF_A);
+
+ /* Reduce 80M spur */
+ rtl8xxxu_write32(priv, REG_AFE_XTAL_CTRL, 0x0381808d);
+ rtl8xxxu_write32(priv, REG_AFE_PLL_CTRL, 0xf0ffff83);
+ rtl8xxxu_write32(priv, REG_AFE_PLL_CTRL, 0xf0ffff82);
+ rtl8xxxu_write32(priv, REG_AFE_PLL_CTRL, 0xf0ffff83);
+
+ return ret;
+}
+
+static int rtl8723a_emu_to_active(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+ u32 val32;
+ int count, ret = 0;
+
+ /* 0x20[0] = 1 enable LDOA12 MACRO block for all interface*/
+ val8 = rtl8xxxu_read8(priv, REG_LDOA15_CTRL);
+ val8 |= LDOA15_ENABLE;
+ rtl8xxxu_write8(priv, REG_LDOA15_CTRL, val8);
+
+ /* 0x67[0] = 0 to disable BT_GPS_SEL pins*/
+ val8 = rtl8xxxu_read8(priv, 0x0067);
+ val8 &= ~BIT(4);
+ rtl8xxxu_write8(priv, 0x0067, val8);
+
+ mdelay(1);
+
+ /* 0x00[5] = 0 release analog Ips to digital, 1:isolation */
+ val8 = rtl8xxxu_read8(priv, REG_SYS_ISO_CTRL);
+ val8 &= ~SYS_ISO_ANALOG_IPS;
+ rtl8xxxu_write8(priv, REG_SYS_ISO_CTRL, val8);
+
+ /* disable SW LPS 0x04[10]= 0 */
+ val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
+ val8 &= ~BIT(2);
+ rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
+
+ /* wait till 0x04[17] = 1 power ready*/
+ for (count = RTL8XXXU_MAX_REG_POLL; count; count--) {
+ val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
+ if (val32 & BIT(17))
+ break;
+
+ udelay(10);
+ }
+
+ if (!count) {
+ ret = -EBUSY;
+ goto exit;
+ }
+
+ /* We should be able to optimize the following three entries into one */
+
+ /* release WLON reset 0x04[16]= 1*/
+ val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 2);
+ val8 |= BIT(0);
+ rtl8xxxu_write8(priv, REG_APS_FSMCO + 2, val8);
+
+ /* disable HWPDN 0x04[15]= 0*/
+ val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
+ val8 &= ~BIT(7);
+ rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
+
+ /* disable WL suspend*/
+ val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
+ val8 &= ~(BIT(3) | BIT(4));
+ rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
+
+ /* set, then poll until 0 */
+ val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
+ val32 |= APS_FSMCO_MAC_ENABLE;
+ rtl8xxxu_write32(priv, REG_APS_FSMCO, val32);
+
+ for (count = RTL8XXXU_MAX_REG_POLL; count; count--) {
+ val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
+ if ((val32 & APS_FSMCO_MAC_ENABLE) == 0) {
+ ret = 0;
+ break;
+ }
+ udelay(10);
+ }
+
+ if (!count) {
+ ret = -EBUSY;
+ goto exit;
+ }
+
+ /* 0x4C[23] = 0x4E[7] = 1, switch DPDT_SEL_P output from WL BB */
+ /*
+ * Note: Vendor driver actually clears this bit, despite the
+ * documentation claims it's being set!
+ */
+ val8 = rtl8xxxu_read8(priv, REG_LEDCFG2);
+ val8 |= LEDCFG2_DPDT_SELECT;
+ val8 &= ~LEDCFG2_DPDT_SELECT;
+ rtl8xxxu_write8(priv, REG_LEDCFG2, val8);
+
+exit:
+ return ret;
+}
+
+static int rtl8723au_power_on(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+ u16 val16;
+ u32 val32;
+ int ret;
+
+ /*
+ * RSV_CTRL 0x001C[7:0] = 0x00, unlock ISO/CLK/Power control register
+ */
+ rtl8xxxu_write8(priv, REG_RSV_CTRL, 0x0);
+
+ rtl8xxxu_disabled_to_emu(priv);
+
+ ret = rtl8723a_emu_to_active(priv);
+ if (ret)
+ goto exit;
+
+ /*
+ * 0x0004[19] = 1, reset 8051
+ */
+ val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 2);
+ val8 |= BIT(3);
+ rtl8xxxu_write8(priv, REG_APS_FSMCO + 2, val8);
+
+ /*
+ * Enable MAC DMA/WMAC/SCHEDULE/SEC block
+ * Set CR bit10 to enable 32k calibration.
+ */
+ val16 = rtl8xxxu_read16(priv, REG_CR);
+ val16 |= (CR_HCI_TXDMA_ENABLE | CR_HCI_RXDMA_ENABLE |
+ CR_TXDMA_ENABLE | CR_RXDMA_ENABLE |
+ CR_PROTOCOL_ENABLE | CR_SCHEDULE_ENABLE |
+ CR_MAC_TX_ENABLE | CR_MAC_RX_ENABLE |
+ CR_SECURITY_ENABLE | CR_CALTIMER_ENABLE);
+ rtl8xxxu_write16(priv, REG_CR, val16);
+
+ /* For EFuse PG */
+ val32 = rtl8xxxu_read32(priv, REG_EFUSE_CTRL);
+ val32 &= ~(BIT(28) | BIT(29) | BIT(30));
+ val32 |= (0x06 << 28);
+ rtl8xxxu_write32(priv, REG_EFUSE_CTRL, val32);
+exit:
+ return ret;
+}
+
+struct rtl8xxxu_fileops rtl8723au_fops = {
+ .parse_efuse = rtl8723au_parse_efuse,
+ .load_firmware = rtl8723au_load_firmware,
+ .power_on = rtl8723au_power_on,
+ .power_off = rtl8xxxu_power_off,
+ .reset_8051 = rtl8xxxu_reset_8051,
+ .llt_init = rtl8xxxu_init_llt_table,
+ .init_phy_bb = rtl8xxxu_gen1_init_phy_bb,
+ .init_phy_rf = rtl8723au_init_phy_rf,
+ .phy_iq_calibrate = rtl8xxxu_gen1_phy_iq_calibrate,
+ .config_channel = rtl8xxxu_gen1_config_channel,
+ .parse_rx_desc = rtl8xxxu_parse_rxdesc16,
+ .enable_rf = rtl8xxxu_gen1_enable_rf,
+ .disable_rf = rtl8xxxu_gen1_disable_rf,
+ .usb_quirks = rtl8xxxu_gen1_usb_quirks,
+ .set_tx_power = rtl8xxxu_gen1_set_tx_power,
+ .update_rate_mask = rtl8xxxu_update_rate_mask,
+ .report_connect = rtl8xxxu_gen1_report_connect,
+ .writeN_block_size = 1024,
+ .tx_desc_size = sizeof(struct rtl8xxxu_txdesc32),
+ .rx_desc_size = sizeof(struct rtl8xxxu_rxdesc16),
+ .adda_1t_init = 0x0b1b25a0,
+ .adda_1t_path_on = 0x0bdb25a0,
+ .adda_2t_path_on_a = 0x04db25a4,
+ .adda_2t_path_on_b = 0x0b1b25a4,
+ .trxff_boundary = 0x27ff,
+ .pbp_rx = PBP_PAGE_SIZE_128,
+ .pbp_tx = PBP_PAGE_SIZE_128,
+ .mactable = rtl8xxxu_gen1_mac_init_table,
+};
diff --git a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8723b.c b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8723b.c
new file mode 100644
index 0000000..4186e7c
--- /dev/null
+++ b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_8723b.c
@@ -0,0 +1,1682 @@
+/*
+ * RTL8XXXU mac80211 USB driver - 8723b specific subdriver
+ *
+ * Copyright (c) 2014 - 2016 Jes Sorensen <Jes.Sorensen@redhat.com>
+ *
+ * Portions, notably calibration code:
+ * Copyright(c) 2007 - 2011 Realtek Corporation. All rights reserved.
+ *
+ * This driver was written as a replacement for the vendor provided
+ * rtl8723au driver. As the Realtek 8xxx chips are very similar in
+ * their programming interface, I have started adding support for
+ * additional 8xxx chips like the 8192cu, 8188cus, etc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ */
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <linux/errno.h>
+#include <linux/slab.h>
+#include <linux/module.h>
+#include <linux/spinlock.h>
+#include <linux/list.h>
+#include <linux/usb.h>
+#include <linux/netdevice.h>
+#include <linux/etherdevice.h>
+#include <linux/ethtool.h>
+#include <linux/wireless.h>
+#include <linux/firmware.h>
+#include <linux/moduleparam.h>
+#include <net/mac80211.h>
+#include "rtl8xxxu.h"
+#include "rtl8xxxu_regs.h"
+
+static struct rtl8xxxu_reg8val rtl8723b_mac_init_table[] = {
+ {0x02f, 0x30}, {0x035, 0x00}, {0x039, 0x08}, {0x04e, 0xe0},
+ {0x064, 0x00}, {0x067, 0x20}, {0x428, 0x0a}, {0x429, 0x10},
+ {0x430, 0x00}, {0x431, 0x00},
+ {0x432, 0x00}, {0x433, 0x01}, {0x434, 0x04}, {0x435, 0x05},
+ {0x436, 0x07}, {0x437, 0x08}, {0x43c, 0x04}, {0x43d, 0x05},
+ {0x43e, 0x07}, {0x43f, 0x08}, {0x440, 0x5d}, {0x441, 0x01},
+ {0x442, 0x00}, {0x444, 0x10}, {0x445, 0x00}, {0x446, 0x00},
+ {0x447, 0x00}, {0x448, 0x00}, {0x449, 0xf0}, {0x44a, 0x0f},
+ {0x44b, 0x3e}, {0x44c, 0x10}, {0x44d, 0x00}, {0x44e, 0x00},
+ {0x44f, 0x00}, {0x450, 0x00}, {0x451, 0xf0}, {0x452, 0x0f},
+ {0x453, 0x00}, {0x456, 0x5e}, {0x460, 0x66}, {0x461, 0x66},
+ {0x4c8, 0xff}, {0x4c9, 0x08}, {0x4cc, 0xff},
+ {0x4cd, 0xff}, {0x4ce, 0x01}, {0x500, 0x26}, {0x501, 0xa2},
+ {0x502, 0x2f}, {0x503, 0x00}, {0x504, 0x28}, {0x505, 0xa3},
+ {0x506, 0x5e}, {0x507, 0x00}, {0x508, 0x2b}, {0x509, 0xa4},
+ {0x50a, 0x5e}, {0x50b, 0x00}, {0x50c, 0x4f}, {0x50d, 0xa4},
+ {0x50e, 0x00}, {0x50f, 0x00}, {0x512, 0x1c}, {0x514, 0x0a},
+ {0x516, 0x0a}, {0x525, 0x4f},
+ {0x550, 0x10}, {0x551, 0x10}, {0x559, 0x02}, {0x55c, 0x50},
+ {0x55d, 0xff}, {0x605, 0x30}, {0x608, 0x0e}, {0x609, 0x2a},
+ {0x620, 0xff}, {0x621, 0xff}, {0x622, 0xff}, {0x623, 0xff},
+ {0x624, 0xff}, {0x625, 0xff}, {0x626, 0xff}, {0x627, 0xff},
+ {0x638, 0x50}, {0x63c, 0x0a}, {0x63d, 0x0a}, {0x63e, 0x0e},
+ {0x63f, 0x0e}, {0x640, 0x40}, {0x642, 0x40}, {0x643, 0x00},
+ {0x652, 0xc8}, {0x66e, 0x05}, {0x700, 0x21}, {0x701, 0x43},
+ {0x702, 0x65}, {0x703, 0x87}, {0x708, 0x21}, {0x709, 0x43},
+ {0x70a, 0x65}, {0x70b, 0x87}, {0x765, 0x18}, {0x76e, 0x04},
+ {0xffff, 0xff},
+};
+
+static struct rtl8xxxu_reg32val rtl8723b_phy_1t_init_table[] = {
+ {0x800, 0x80040000}, {0x804, 0x00000003},
+ {0x808, 0x0000fc00}, {0x80c, 0x0000000a},
+ {0x810, 0x10001331}, {0x814, 0x020c3d10},
+ {0x818, 0x02200385}, {0x81c, 0x00000000},
+ {0x820, 0x01000100}, {0x824, 0x00190204},
+ {0x828, 0x00000000}, {0x82c, 0x00000000},
+ {0x830, 0x00000000}, {0x834, 0x00000000},
+ {0x838, 0x00000000}, {0x83c, 0x00000000},
+ {0x840, 0x00010000}, {0x844, 0x00000000},
+ {0x848, 0x00000000}, {0x84c, 0x00000000},
+ {0x850, 0x00000000}, {0x854, 0x00000000},
+ {0x858, 0x569a11a9}, {0x85c, 0x01000014},
+ {0x860, 0x66f60110}, {0x864, 0x061f0649},
+ {0x868, 0x00000000}, {0x86c, 0x27272700},
+ {0x870, 0x07000760}, {0x874, 0x25004000},
+ {0x878, 0x00000808}, {0x87c, 0x00000000},
+ {0x880, 0xb0000c1c}, {0x884, 0x00000001},
+ {0x888, 0x00000000}, {0x88c, 0xccc000c0},
+ {0x890, 0x00000800}, {0x894, 0xfffffffe},
+ {0x898, 0x40302010}, {0x89c, 0x00706050},
+ {0x900, 0x00000000}, {0x904, 0x00000023},
+ {0x908, 0x00000000}, {0x90c, 0x81121111},
+ {0x910, 0x00000002}, {0x914, 0x00000201},
+ {0xa00, 0x00d047c8}, {0xa04, 0x80ff800c},
+ {0xa08, 0x8c838300}, {0xa0c, 0x2e7f120f},
+ {0xa10, 0x9500bb78}, {0xa14, 0x1114d028},
+ {0xa18, 0x00881117}, {0xa1c, 0x89140f00},
+ {0xa20, 0x1a1b0000}, {0xa24, 0x090e1317},
+ {0xa28, 0x00000204}, {0xa2c, 0x00d30000},
+ {0xa70, 0x101fbf00}, {0xa74, 0x00000007},
+ {0xa78, 0x00000900}, {0xa7c, 0x225b0606},
+ {0xa80, 0x21806490}, {0xb2c, 0x00000000},
+ {0xc00, 0x48071d40}, {0xc04, 0x03a05611},
+ {0xc08, 0x000000e4}, {0xc0c, 0x6c6c6c6c},
+ {0xc10, 0x08800000}, {0xc14, 0x40000100},
+ {0xc18, 0x08800000}, {0xc1c, 0x40000100},
+ {0xc20, 0x00000000}, {0xc24, 0x00000000},
+ {0xc28, 0x00000000}, {0xc2c, 0x00000000},
+ {0xc30, 0x69e9ac44}, {0xc34, 0x469652af},
+ {0xc38, 0x49795994}, {0xc3c, 0x0a97971c},
+ {0xc40, 0x1f7c403f}, {0xc44, 0x000100b7},
+ {0xc48, 0xec020107}, {0xc4c, 0x007f037f},
+ {0xc50, 0x69553420}, {0xc54, 0x43bc0094},
+ {0xc58, 0x00013149}, {0xc5c, 0x00250492},
+ {0xc60, 0x00000000}, {0xc64, 0x7112848b},
+ {0xc68, 0x47c00bff}, {0xc6c, 0x00000036},
+ {0xc70, 0x2c7f000d}, {0xc74, 0x020610db},
+ {0xc78, 0x0000001f}, {0xc7c, 0x00b91612},
+ {0xc80, 0x390000e4}, {0xc84, 0x20f60000},
+ {0xc88, 0x40000100}, {0xc8c, 0x20200000},
+ {0xc90, 0x00020e1a}, {0xc94, 0x00000000},
+ {0xc98, 0x00020e1a}, {0xc9c, 0x00007f7f},
+ {0xca0, 0x00000000}, {0xca4, 0x000300a0},
+ {0xca8, 0x00000000}, {0xcac, 0x00000000},
+ {0xcb0, 0x00000000}, {0xcb4, 0x00000000},
+ {0xcb8, 0x00000000}, {0xcbc, 0x28000000},
+ {0xcc0, 0x00000000}, {0xcc4, 0x00000000},
+ {0xcc8, 0x00000000}, {0xccc, 0x00000000},
+ {0xcd0, 0x00000000}, {0xcd4, 0x00000000},
+ {0xcd8, 0x64b22427}, {0xcdc, 0x00766932},
+ {0xce0, 0x00222222}, {0xce4, 0x00000000},
+ {0xce8, 0x37644302}, {0xcec, 0x2f97d40c},
+ {0xd00, 0x00000740}, {0xd04, 0x40020401},
+ {0xd08, 0x0000907f}, {0xd0c, 0x20010201},
+ {0xd10, 0xa0633333}, {0xd14, 0x3333bc53},
+ {0xd18, 0x7a8f5b6f}, {0xd2c, 0xcc979975},
+ {0xd30, 0x00000000}, {0xd34, 0x80608000},
+ {0xd38, 0x00000000}, {0xd3c, 0x00127353},
+ {0xd40, 0x00000000}, {0xd44, 0x00000000},
+ {0xd48, 0x00000000}, {0xd4c, 0x00000000},
+ {0xd50, 0x6437140a}, {0xd54, 0x00000000},
+ {0xd58, 0x00000282}, {0xd5c, 0x30032064},
+ {0xd60, 0x4653de68}, {0xd64, 0x04518a3c},
+ {0xd68, 0x00002101}, {0xd6c, 0x2a201c16},
+ {0xd70, 0x1812362e}, {0xd74, 0x322c2220},
+ {0xd78, 0x000e3c24}, {0xe00, 0x2d2d2d2d},
+ {0xe04, 0x2d2d2d2d}, {0xe08, 0x0390272d},
+ {0xe10, 0x2d2d2d2d}, {0xe14, 0x2d2d2d2d},
+ {0xe18, 0x2d2d2d2d}, {0xe1c, 0x2d2d2d2d},
+ {0xe28, 0x00000000}, {0xe30, 0x1000dc1f},
+ {0xe34, 0x10008c1f}, {0xe38, 0x02140102},
+ {0xe3c, 0x681604c2}, {0xe40, 0x01007c00},
+ {0xe44, 0x01004800}, {0xe48, 0xfb000000},
+ {0xe4c, 0x000028d1}, {0xe50, 0x1000dc1f},
+ {0xe54, 0x10008c1f}, {0xe58, 0x02140102},
+ {0xe5c, 0x28160d05}, {0xe60, 0x00000008},
+ {0xe68, 0x001b2556}, {0xe6c, 0x00c00096},
+ {0xe70, 0x00c00096}, {0xe74, 0x01000056},
+ {0xe78, 0x01000014}, {0xe7c, 0x01000056},
+ {0xe80, 0x01000014}, {0xe84, 0x00c00096},
+ {0xe88, 0x01000056}, {0xe8c, 0x00c00096},
+ {0xed0, 0x00c00096}, {0xed4, 0x00c00096},
+ {0xed8, 0x00c00096}, {0xedc, 0x000000d6},
+ {0xee0, 0x000000d6}, {0xeec, 0x01c00016},
+ {0xf14, 0x00000003}, {0xf4c, 0x00000000},
+ {0xf00, 0x00000300},
+ {0x820, 0x01000100}, {0x800, 0x83040000},
+ {0xffff, 0xffffffff},
+};
+
+static struct rtl8xxxu_reg32val rtl8xxx_agc_8723bu_table[] = {
+ {0xc78, 0xfd000001}, {0xc78, 0xfc010001},
+ {0xc78, 0xfb020001}, {0xc78, 0xfa030001},
+ {0xc78, 0xf9040001}, {0xc78, 0xf8050001},
+ {0xc78, 0xf7060001}, {0xc78, 0xf6070001},
+ {0xc78, 0xf5080001}, {0xc78, 0xf4090001},
+ {0xc78, 0xf30a0001}, {0xc78, 0xf20b0001},
+ {0xc78, 0xf10c0001}, {0xc78, 0xf00d0001},
+ {0xc78, 0xef0e0001}, {0xc78, 0xee0f0001},
+ {0xc78, 0xed100001}, {0xc78, 0xec110001},
+ {0xc78, 0xeb120001}, {0xc78, 0xea130001},
+ {0xc78, 0xe9140001}, {0xc78, 0xe8150001},
+ {0xc78, 0xe7160001}, {0xc78, 0xe6170001},
+ {0xc78, 0xe5180001}, {0xc78, 0xe4190001},
+ {0xc78, 0xe31a0001}, {0xc78, 0xa51b0001},
+ {0xc78, 0xa41c0001}, {0xc78, 0xa31d0001},
+ {0xc78, 0x671e0001}, {0xc78, 0x661f0001},
+ {0xc78, 0x65200001}, {0xc78, 0x64210001},
+ {0xc78, 0x63220001}, {0xc78, 0x4a230001},
+ {0xc78, 0x49240001}, {0xc78, 0x48250001},
+ {0xc78, 0x47260001}, {0xc78, 0x46270001},
+ {0xc78, 0x45280001}, {0xc78, 0x44290001},
+ {0xc78, 0x432a0001}, {0xc78, 0x422b0001},
+ {0xc78, 0x292c0001}, {0xc78, 0x282d0001},
+ {0xc78, 0x272e0001}, {0xc78, 0x262f0001},
+ {0xc78, 0x0a300001}, {0xc78, 0x09310001},
+ {0xc78, 0x08320001}, {0xc78, 0x07330001},
+ {0xc78, 0x06340001}, {0xc78, 0x05350001},
+ {0xc78, 0x04360001}, {0xc78, 0x03370001},
+ {0xc78, 0x02380001}, {0xc78, 0x01390001},
+ {0xc78, 0x013a0001}, {0xc78, 0x013b0001},
+ {0xc78, 0x013c0001}, {0xc78, 0x013d0001},
+ {0xc78, 0x013e0001}, {0xc78, 0x013f0001},
+ {0xc78, 0xfc400001}, {0xc78, 0xfb410001},
+ {0xc78, 0xfa420001}, {0xc78, 0xf9430001},
+ {0xc78, 0xf8440001}, {0xc78, 0xf7450001},
+ {0xc78, 0xf6460001}, {0xc78, 0xf5470001},
+ {0xc78, 0xf4480001}, {0xc78, 0xf3490001},
+ {0xc78, 0xf24a0001}, {0xc78, 0xf14b0001},
+ {0xc78, 0xf04c0001}, {0xc78, 0xef4d0001},
+ {0xc78, 0xee4e0001}, {0xc78, 0xed4f0001},
+ {0xc78, 0xec500001}, {0xc78, 0xeb510001},
+ {0xc78, 0xea520001}, {0xc78, 0xe9530001},
+ {0xc78, 0xe8540001}, {0xc78, 0xe7550001},
+ {0xc78, 0xe6560001}, {0xc78, 0xe5570001},
+ {0xc78, 0xe4580001}, {0xc78, 0xe3590001},
+ {0xc78, 0xa65a0001}, {0xc78, 0xa55b0001},
+ {0xc78, 0xa45c0001}, {0xc78, 0xa35d0001},
+ {0xc78, 0x675e0001}, {0xc78, 0x665f0001},
+ {0xc78, 0x65600001}, {0xc78, 0x64610001},
+ {0xc78, 0x63620001}, {0xc78, 0x62630001},
+ {0xc78, 0x61640001}, {0xc78, 0x48650001},
+ {0xc78, 0x47660001}, {0xc78, 0x46670001},
+ {0xc78, 0x45680001}, {0xc78, 0x44690001},
+ {0xc78, 0x436a0001}, {0xc78, 0x426b0001},
+ {0xc78, 0x286c0001}, {0xc78, 0x276d0001},
+ {0xc78, 0x266e0001}, {0xc78, 0x256f0001},
+ {0xc78, 0x24700001}, {0xc78, 0x09710001},
+ {0xc78, 0x08720001}, {0xc78, 0x07730001},
+ {0xc78, 0x06740001}, {0xc78, 0x05750001},
+ {0xc78, 0x04760001}, {0xc78, 0x03770001},
+ {0xc78, 0x02780001}, {0xc78, 0x01790001},
+ {0xc78, 0x017a0001}, {0xc78, 0x017b0001},
+ {0xc78, 0x017c0001}, {0xc78, 0x017d0001},
+ {0xc78, 0x017e0001}, {0xc78, 0x017f0001},
+ {0xc50, 0x69553422},
+ {0xc50, 0x69553420},
+ {0x824, 0x00390204},
+ {0xffff, 0xffffffff}
+};
+
+static struct rtl8xxxu_rfregval rtl8723bu_radioa_1t_init_table[] = {
+ {0x00, 0x00010000}, {0xb0, 0x000dffe0},
+ {0xfe, 0x00000000}, {0xfe, 0x00000000},
+ {0xfe, 0x00000000}, {0xb1, 0x00000018},
+ {0xfe, 0x00000000}, {0xfe, 0x00000000},
+ {0xfe, 0x00000000}, {0xb2, 0x00084c00},
+ {0xb5, 0x0000d2cc}, {0xb6, 0x000925aa},
+ {0xb7, 0x00000010}, {0xb8, 0x0000907f},
+ {0x5c, 0x00000002}, {0x7c, 0x00000002},
+ {0x7e, 0x00000005}, {0x8b, 0x0006fc00},
+ {0xb0, 0x000ff9f0}, {0x1c, 0x000739d2},
+ {0x1e, 0x00000000}, {0xdf, 0x00000780},
+ {0x50, 0x00067435},
+ /*
+ * The 8723bu vendor driver indicates that bit 8 should be set in
+ * 0x51 for package types TFBGA90, TFBGA80, and TFBGA79. However
+ * they never actually check the package type - and just default
+ * to not setting it.
+ */
+ {0x51, 0x0006b04e},
+ {0x52, 0x000007d2}, {0x53, 0x00000000},
+ {0x54, 0x00050400}, {0x55, 0x0004026e},
+ {0xdd, 0x0000004c}, {0x70, 0x00067435},
+ /*
+ * 0x71 has same package type condition as for register 0x51
+ */
+ {0x71, 0x0006b04e},
+ {0x72, 0x000007d2}, {0x73, 0x00000000},
+ {0x74, 0x00050400}, {0x75, 0x0004026e},
+ {0xef, 0x00000100}, {0x34, 0x0000add7},
+ {0x35, 0x00005c00}, {0x34, 0x00009dd4},
+ {0x35, 0x00005000}, {0x34, 0x00008dd1},
+ {0x35, 0x00004400}, {0x34, 0x00007dce},
+ {0x35, 0x00003800}, {0x34, 0x00006cd1},
+ {0x35, 0x00004400}, {0x34, 0x00005cce},
+ {0x35, 0x00003800}, {0x34, 0x000048ce},
+ {0x35, 0x00004400}, {0x34, 0x000034ce},
+ {0x35, 0x00003800}, {0x34, 0x00002451},
+ {0x35, 0x00004400}, {0x34, 0x0000144e},
+ {0x35, 0x00003800}, {0x34, 0x00000051},
+ {0x35, 0x00004400}, {0xef, 0x00000000},
+ {0xef, 0x00000100}, {0xed, 0x00000010},
+ {0x44, 0x0000add7}, {0x44, 0x00009dd4},
+ {0x44, 0x00008dd1}, {0x44, 0x00007dce},
+ {0x44, 0x00006cc1}, {0x44, 0x00005cce},
+ {0x44, 0x000044d1}, {0x44, 0x000034ce},
+ {0x44, 0x00002451}, {0x44, 0x0000144e},
+ {0x44, 0x00000051}, {0xef, 0x00000000},
+ {0xed, 0x00000000}, {0x7f, 0x00020080},
+ {0xef, 0x00002000}, {0x3b, 0x000380ef},
+ {0x3b, 0x000302fe}, {0x3b, 0x00028ce6},
+ {0x3b, 0x000200bc}, {0x3b, 0x000188a5},
+ {0x3b, 0x00010fbc}, {0x3b, 0x00008f71},
+ {0x3b, 0x00000900}, {0xef, 0x00000000},
+ {0xed, 0x00000001}, {0x40, 0x000380ef},
+ {0x40, 0x000302fe}, {0x40, 0x00028ce6},
+ {0x40, 0x000200bc}, {0x40, 0x000188a5},
+ {0x40, 0x00010fbc}, {0x40, 0x00008f71},
+ {0x40, 0x00000900}, {0xed, 0x00000000},
+ {0x82, 0x00080000}, {0x83, 0x00008000},
+ {0x84, 0x00048d80}, {0x85, 0x00068000},
+ {0xa2, 0x00080000}, {0xa3, 0x00008000},
+ {0xa4, 0x00048d80}, {0xa5, 0x00068000},
+ {0xed, 0x00000002}, {0xef, 0x00000002},
+ {0x56, 0x00000032}, {0x76, 0x00000032},
+ {0x01, 0x00000780},
+ {0xff, 0xffffffff}
+};
+
+static void rtl8723bu_write_btreg(struct rtl8xxxu_priv *priv, u8 reg, u8 data)
+{
+ struct h2c_cmd h2c;
+ int reqnum = 0;
+
+ memset(&h2c, 0, sizeof(struct h2c_cmd));
+ h2c.bt_mp_oper.cmd = H2C_8723B_BT_MP_OPER;
+ h2c.bt_mp_oper.operreq = 0 | (reqnum << 4);
+ h2c.bt_mp_oper.opcode = BT_MP_OP_WRITE_REG_VALUE;
+ h2c.bt_mp_oper.data = data;
+ rtl8xxxu_gen2_h2c_cmd(priv, &h2c, sizeof(h2c.bt_mp_oper));
+
+ reqnum++;
+ memset(&h2c, 0, sizeof(struct h2c_cmd));
+ h2c.bt_mp_oper.cmd = H2C_8723B_BT_MP_OPER;
+ h2c.bt_mp_oper.operreq = 0 | (reqnum << 4);
+ h2c.bt_mp_oper.opcode = BT_MP_OP_WRITE_REG_VALUE;
+ h2c.bt_mp_oper.addr = reg;
+ rtl8xxxu_gen2_h2c_cmd(priv, &h2c, sizeof(h2c.bt_mp_oper));
+}
+
+static void rtl8723bu_reset_8051(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+ u16 sys_func;
+
+ val8 = rtl8xxxu_read8(priv, REG_RSV_CTRL);
+ val8 &= ~BIT(1);
+ rtl8xxxu_write8(priv, REG_RSV_CTRL, val8);
+
+ val8 = rtl8xxxu_read8(priv, REG_RSV_CTRL + 1);
+ val8 &= ~BIT(0);
+ rtl8xxxu_write8(priv, REG_RSV_CTRL + 1, val8);
+
+ sys_func = rtl8xxxu_read16(priv, REG_SYS_FUNC);
+ sys_func &= ~SYS_FUNC_CPU_ENABLE;
+ rtl8xxxu_write16(priv, REG_SYS_FUNC, sys_func);
+
+ val8 = rtl8xxxu_read8(priv, REG_RSV_CTRL);
+ val8 &= ~BIT(1);
+ rtl8xxxu_write8(priv, REG_RSV_CTRL, val8);
+
+ val8 = rtl8xxxu_read8(priv, REG_RSV_CTRL + 1);
+ val8 |= BIT(0);
+ rtl8xxxu_write8(priv, REG_RSV_CTRL + 1, val8);
+
+ sys_func |= SYS_FUNC_CPU_ENABLE;
+ rtl8xxxu_write16(priv, REG_SYS_FUNC, sys_func);
+}
+
+static void
+rtl8723b_set_tx_power(struct rtl8xxxu_priv *priv, int channel, bool ht40)
+{
+ u32 val32, ofdm, mcs;
+ u8 cck, ofdmbase, mcsbase;
+ int group, tx_idx;
+
+ tx_idx = 0;
+ group = rtl8xxxu_gen2_channel_to_group(channel);
+
+ cck = priv->cck_tx_power_index_B[group];
+ val32 = rtl8xxxu_read32(priv, REG_TX_AGC_A_CCK1_MCS32);
+ val32 &= 0xffff00ff;
+ val32 |= (cck << 8);
+ rtl8xxxu_write32(priv, REG_TX_AGC_A_CCK1_MCS32, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_TX_AGC_B_CCK11_A_CCK2_11);
+ val32 &= 0xff;
+ val32 |= ((cck << 8) | (cck << 16) | (cck << 24));
+ rtl8xxxu_write32(priv, REG_TX_AGC_B_CCK11_A_CCK2_11, val32);
+
+ ofdmbase = priv->ht40_1s_tx_power_index_B[group];
+ ofdmbase += priv->ofdm_tx_power_diff[tx_idx].b;
+ ofdm = ofdmbase | ofdmbase << 8 | ofdmbase << 16 | ofdmbase << 24;
+
+ rtl8xxxu_write32(priv, REG_TX_AGC_A_RATE18_06, ofdm);
+ rtl8xxxu_write32(priv, REG_TX_AGC_A_RATE54_24, ofdm);
+
+ mcsbase = priv->ht40_1s_tx_power_index_B[group];
+ if (ht40)
+ mcsbase += priv->ht40_tx_power_diff[tx_idx++].b;
+ else
+ mcsbase += priv->ht20_tx_power_diff[tx_idx++].b;
+ mcs = mcsbase | mcsbase << 8 | mcsbase << 16 | mcsbase << 24;
+
+ rtl8xxxu_write32(priv, REG_TX_AGC_A_MCS03_MCS00, mcs);
+ rtl8xxxu_write32(priv, REG_TX_AGC_A_MCS07_MCS04, mcs);
+}
+
+static int rtl8723bu_parse_efuse(struct rtl8xxxu_priv *priv)
+{
+ struct rtl8723bu_efuse *efuse = &priv->efuse_wifi.efuse8723bu;
+ int i;
+
+ if (efuse->rtl_id != cpu_to_le16(0x8129))
+ return -EINVAL;
+
+ ether_addr_copy(priv->mac_addr, efuse->mac_addr);
+
+ memcpy(priv->cck_tx_power_index_A, efuse->tx_power_index_A.cck_base,
+ sizeof(efuse->tx_power_index_A.cck_base));
+ memcpy(priv->cck_tx_power_index_B, efuse->tx_power_index_B.cck_base,
+ sizeof(efuse->tx_power_index_B.cck_base));
+
+ memcpy(priv->ht40_1s_tx_power_index_A,
+ efuse->tx_power_index_A.ht40_base,
+ sizeof(efuse->tx_power_index_A.ht40_base));
+ memcpy(priv->ht40_1s_tx_power_index_B,
+ efuse->tx_power_index_B.ht40_base,
+ sizeof(efuse->tx_power_index_B.ht40_base));
+
+ priv->ofdm_tx_power_diff[0].a =
+ efuse->tx_power_index_A.ht20_ofdm_1s_diff.a;
+ priv->ofdm_tx_power_diff[0].b =
+ efuse->tx_power_index_B.ht20_ofdm_1s_diff.a;
+
+ priv->ht20_tx_power_diff[0].a =
+ efuse->tx_power_index_A.ht20_ofdm_1s_diff.b;
+ priv->ht20_tx_power_diff[0].b =
+ efuse->tx_power_index_B.ht20_ofdm_1s_diff.b;
+
+ priv->ht40_tx_power_diff[0].a = 0;
+ priv->ht40_tx_power_diff[0].b = 0;
+
+ for (i = 1; i < RTL8723B_TX_COUNT; i++) {
+ priv->ofdm_tx_power_diff[i].a =
+ efuse->tx_power_index_A.pwr_diff[i - 1].ofdm;
+ priv->ofdm_tx_power_diff[i].b =
+ efuse->tx_power_index_B.pwr_diff[i - 1].ofdm;
+
+ priv->ht20_tx_power_diff[i].a =
+ efuse->tx_power_index_A.pwr_diff[i - 1].ht20;
+ priv->ht20_tx_power_diff[i].b =
+ efuse->tx_power_index_B.pwr_diff[i - 1].ht20;
+
+ priv->ht40_tx_power_diff[i].a =
+ efuse->tx_power_index_A.pwr_diff[i - 1].ht40;
+ priv->ht40_tx_power_diff[i].b =
+ efuse->tx_power_index_B.pwr_diff[i - 1].ht40;
+ }
+
+ priv->has_xtalk = 1;
+ priv->xtalk = priv->efuse_wifi.efuse8723bu.xtal_k & 0x3f;
+
+ dev_info(&priv->udev->dev, "Vendor: %.7s\n", efuse->vendor_name);
+ dev_info(&priv->udev->dev, "Product: %.41s\n", efuse->device_name);
+
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_EFUSE) {
+ int i;
+ unsigned char *raw = priv->efuse_wifi.raw;
+
+ dev_info(&priv->udev->dev,
+ "%s: dumping efuse (0x%02zx bytes):\n",
+ __func__, sizeof(struct rtl8723bu_efuse));
+ for (i = 0; i < sizeof(struct rtl8723bu_efuse); i += 8) {
+ dev_info(&priv->udev->dev, "%02x: "
+ "%02x %02x %02x %02x %02x %02x %02x %02x\n", i,
+ raw[i], raw[i + 1], raw[i + 2],
+ raw[i + 3], raw[i + 4], raw[i + 5],
+ raw[i + 6], raw[i + 7]);
+ }
+ }
+
+ return 0;
+}
+
+static int rtl8723bu_load_firmware(struct rtl8xxxu_priv *priv)
+{
+ char *fw_name;
+ int ret;
+
+ if (priv->enable_bluetooth)
+ fw_name = "rtlwifi/rtl8723bu_bt.bin";
+ else
+ fw_name = "rtlwifi/rtl8723bu_nic.bin";
+
+ ret = rtl8xxxu_load_firmware(priv, fw_name);
+ return ret;
+}
+
+static void rtl8723bu_init_phy_bb(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+ u16 val16;
+
+ val16 = rtl8xxxu_read16(priv, REG_SYS_FUNC);
+ val16 |= SYS_FUNC_BB_GLB_RSTN | SYS_FUNC_BBRSTB | SYS_FUNC_DIO_RF;
+ rtl8xxxu_write16(priv, REG_SYS_FUNC, val16);
+
+ rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, 0x00);
+
+ /* 6. 0x1f[7:0] = 0x07 */
+ val8 = RF_ENABLE | RF_RSTB | RF_SDMRSTB;
+ rtl8xxxu_write8(priv, REG_RF_CTRL, val8);
+
+ /* Why? */
+ rtl8xxxu_write8(priv, REG_SYS_FUNC, 0xe3);
+ rtl8xxxu_write8(priv, REG_AFE_XTAL_CTRL + 1, 0x80);
+ rtl8xxxu_init_phy_regs(priv, rtl8723b_phy_1t_init_table);
+
+ rtl8xxxu_init_phy_regs(priv, rtl8xxx_agc_8723bu_table);
+}
+
+static int rtl8723bu_init_phy_rf(struct rtl8xxxu_priv *priv)
+{
+ int ret;
+
+ ret = rtl8xxxu_init_phy_rf(priv, rtl8723bu_radioa_1t_init_table, RF_A);
+ /*
+ * PHY LCK
+ */
+ rtl8xxxu_write_rfreg(priv, RF_A, 0xb0, 0xdfbe0);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_MODE_AG, 0x8c01);
+ msleep(200);
+ rtl8xxxu_write_rfreg(priv, RF_A, 0xb0, 0xdffe0);
+
+ return ret;
+}
+
+static void rtl8723bu_phy_init_antenna_selection(struct rtl8xxxu_priv *priv)
+{
+ u32 val32;
+
+ val32 = rtl8xxxu_read32(priv, REG_PAD_CTRL1);
+ val32 &= ~(BIT(20) | BIT(24));
+ rtl8xxxu_write32(priv, REG_PAD_CTRL1, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_GPIO_MUXCFG);
+ val32 &= ~BIT(4);
+ rtl8xxxu_write32(priv, REG_GPIO_MUXCFG, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_GPIO_MUXCFG);
+ val32 |= BIT(3);
+ rtl8xxxu_write32(priv, REG_GPIO_MUXCFG, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_LEDCFG0);
+ val32 |= BIT(24);
+ rtl8xxxu_write32(priv, REG_LEDCFG0, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_LEDCFG0);
+ val32 &= ~BIT(23);
+ rtl8xxxu_write32(priv, REG_LEDCFG0, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_RFE_BUFFER);
+ val32 |= (BIT(0) | BIT(1));
+ rtl8xxxu_write32(priv, REG_RFE_BUFFER, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_RFE_CTRL_ANTA_SRC);
+ val32 &= 0xffffff00;
+ val32 |= 0x77;
+ rtl8xxxu_write32(priv, REG_RFE_CTRL_ANTA_SRC, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_PWR_DATA);
+ val32 |= PWR_DATA_EEPRPAD_RFE_CTRL_EN;
+ rtl8xxxu_write32(priv, REG_PWR_DATA, val32);
+}
+
+static int rtl8723bu_iqk_path_a(struct rtl8xxxu_priv *priv)
+{
+ u32 reg_eac, reg_e94, reg_e9c, path_sel, val32;
+ int result = 0;
+
+ path_sel = rtl8xxxu_read32(priv, REG_S0S1_PATH_SWITCH);
+
+ /*
+ * Leave IQK mode
+ */
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
+ val32 &= 0x000000ff;
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
+
+ /*
+ * Enable path A PA in TX IQK mode
+ */
+ val32 = rtl8xxxu_read_rfreg(priv, RF_A, RF6052_REG_WE_LUT);
+ val32 |= 0x80000;
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_WE_LUT, val32);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_RCK_OS, 0x20000);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G1, 0x0003f);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G2, 0xc7f87);
+
+ /*
+ * Tx IQK setting
+ */
+ rtl8xxxu_write32(priv, REG_TX_IQK, 0x01007c00);
+ rtl8xxxu_write32(priv, REG_RX_IQK, 0x01004800);
+
+ /* path-A IQK setting */
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x18008c1c);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x38008c1c);
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_B, 0x38008c1c);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_B, 0x38008c1c);
+
+ rtl8xxxu_write32(priv, REG_TX_IQK_PI_A, 0x821403ea);
+ rtl8xxxu_write32(priv, REG_RX_IQK_PI_A, 0x28110000);
+ rtl8xxxu_write32(priv, REG_TX_IQK_PI_B, 0x82110000);
+ rtl8xxxu_write32(priv, REG_RX_IQK_PI_B, 0x28110000);
+
+ /* LO calibration setting */
+ rtl8xxxu_write32(priv, REG_IQK_AGC_RSP, 0x00462911);
+
+ /*
+ * Enter IQK mode
+ */
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
+ val32 &= 0x000000ff;
+ val32 |= 0x80800000;
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
+
+ /*
+ * The vendor driver indicates the USB module is always using
+ * S0S1 path 1 for the 8723bu. This may be different for 8192eu
+ */
+ if (priv->rf_paths > 1)
+ rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, 0x00000000);
+ else
+ rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, 0x00000280);
+
+ /*
+ * Bit 12 seems to be BT_GRANT, and is only found in the 8723bu.
+ * No trace of this in the 8192eu or 8188eu vendor drivers.
+ */
+ rtl8xxxu_write32(priv, REG_BT_CONTROL_8723BU, 0x00000800);
+
+ /* One shot, path A LOK & IQK */
+ rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf9000000);
+ rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf8000000);
+
+ mdelay(1);
+
+ /* Restore Ant Path */
+ rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, path_sel);
+#ifdef RTL8723BU_BT
+ /* GNT_BT = 1 */
+ rtl8xxxu_write32(priv, REG_BT_CONTROL_8723BU, 0x00001800);
+#endif
+
+ /*
+ * Leave IQK mode
+ */
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
+ val32 &= 0x000000ff;
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
+
+ /* Check failed */
+ reg_eac = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_A_2);
+ reg_e94 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_A);
+ reg_e9c = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_A);
+
+ val32 = (reg_e9c >> 16) & 0x3ff;
+ if (val32 & 0x200)
+ val32 = 0x400 - val32;
+
+ if (!(reg_eac & BIT(28)) &&
+ ((reg_e94 & 0x03ff0000) != 0x01420000) &&
+ ((reg_e9c & 0x03ff0000) != 0x00420000) &&
+ ((reg_e94 & 0x03ff0000) < 0x01100000) &&
+ ((reg_e94 & 0x03ff0000) > 0x00f00000) &&
+ val32 < 0xf)
+ result |= 0x01;
+ else /* If TX not OK, ignore RX */
+ goto out;
+
+out:
+ return result;
+}
+
+static int rtl8723bu_rx_iqk_path_a(struct rtl8xxxu_priv *priv)
+{
+ u32 reg_ea4, reg_eac, reg_e94, reg_e9c, path_sel, val32;
+ int result = 0;
+
+ path_sel = rtl8xxxu_read32(priv, REG_S0S1_PATH_SWITCH);
+
+ /*
+ * Leave IQK mode
+ */
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
+ val32 &= 0x000000ff;
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
+
+ /*
+ * Enable path A PA in TX IQK mode
+ */
+ val32 = rtl8xxxu_read_rfreg(priv, RF_A, RF6052_REG_WE_LUT);
+ val32 |= 0x80000;
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_WE_LUT, val32);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_RCK_OS, 0x30000);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G1, 0x0001f);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G2, 0xf7fb7);
+
+ /*
+ * Tx IQK setting
+ */
+ rtl8xxxu_write32(priv, REG_TX_IQK, 0x01007c00);
+ rtl8xxxu_write32(priv, REG_RX_IQK, 0x01004800);
+
+ /* path-A IQK setting */
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x18008c1c);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x38008c1c);
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_B, 0x38008c1c);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_B, 0x38008c1c);
+
+ rtl8xxxu_write32(priv, REG_TX_IQK_PI_A, 0x82160ff0);
+ rtl8xxxu_write32(priv, REG_RX_IQK_PI_A, 0x28110000);
+ rtl8xxxu_write32(priv, REG_TX_IQK_PI_B, 0x82110000);
+ rtl8xxxu_write32(priv, REG_RX_IQK_PI_B, 0x28110000);
+
+ /* LO calibration setting */
+ rtl8xxxu_write32(priv, REG_IQK_AGC_RSP, 0x0046a911);
+
+ /*
+ * Enter IQK mode
+ */
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
+ val32 &= 0x000000ff;
+ val32 |= 0x80800000;
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
+
+ /*
+ * The vendor driver indicates the USB module is always using
+ * S0S1 path 1 for the 8723bu. This may be different for 8192eu
+ */
+ if (priv->rf_paths > 1)
+ rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, 0x00000000);
+ else
+ rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, 0x00000280);
+
+ /*
+ * Bit 12 seems to be BT_GRANT, and is only found in the 8723bu.
+ * No trace of this in the 8192eu or 8188eu vendor drivers.
+ */
+ rtl8xxxu_write32(priv, REG_BT_CONTROL_8723BU, 0x00000800);
+
+ /* One shot, path A LOK & IQK */
+ rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf9000000);
+ rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf8000000);
+
+ mdelay(1);
+
+ /* Restore Ant Path */
+ rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, path_sel);
+#ifdef RTL8723BU_BT
+ /* GNT_BT = 1 */
+ rtl8xxxu_write32(priv, REG_BT_CONTROL_8723BU, 0x00001800);
+#endif
+
+ /*
+ * Leave IQK mode
+ */
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
+ val32 &= 0x000000ff;
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
+
+ /* Check failed */
+ reg_eac = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_A_2);
+ reg_e94 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_A);
+ reg_e9c = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_A);
+
+ val32 = (reg_e9c >> 16) & 0x3ff;
+ if (val32 & 0x200)
+ val32 = 0x400 - val32;
+
+ if (!(reg_eac & BIT(28)) &&
+ ((reg_e94 & 0x03ff0000) != 0x01420000) &&
+ ((reg_e9c & 0x03ff0000) != 0x00420000) &&
+ ((reg_e94 & 0x03ff0000) < 0x01100000) &&
+ ((reg_e94 & 0x03ff0000) > 0x00f00000) &&
+ val32 < 0xf)
+ result |= 0x01;
+ else /* If TX not OK, ignore RX */
+ goto out;
+
+ val32 = 0x80007c00 | (reg_e94 &0x3ff0000) |
+ ((reg_e9c & 0x3ff0000) >> 16);
+ rtl8xxxu_write32(priv, REG_TX_IQK, val32);
+
+ /*
+ * Modify RX IQK mode
+ */
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
+ val32 &= 0x000000ff;
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
+ val32 = rtl8xxxu_read_rfreg(priv, RF_A, RF6052_REG_WE_LUT);
+ val32 |= 0x80000;
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_WE_LUT, val32);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_RCK_OS, 0x30000);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G1, 0x0001f);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G2, 0xf7d77);
+
+ /*
+ * PA, PAD setting
+ */
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_DF, 0xf80);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_55, 0x4021f);
+
+ /*
+ * RX IQK setting
+ */
+ rtl8xxxu_write32(priv, REG_RX_IQK, 0x01004800);
+
+ /* path-A IQK setting */
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x38008c1c);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x18008c1c);
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_B, 0x38008c1c);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_B, 0x38008c1c);
+
+ rtl8xxxu_write32(priv, REG_TX_IQK_PI_A, 0x82110000);
+ rtl8xxxu_write32(priv, REG_RX_IQK_PI_A, 0x2816001f);
+ rtl8xxxu_write32(priv, REG_TX_IQK_PI_B, 0x82110000);
+ rtl8xxxu_write32(priv, REG_RX_IQK_PI_B, 0x28110000);
+
+ /* LO calibration setting */
+ rtl8xxxu_write32(priv, REG_IQK_AGC_RSP, 0x0046a8d1);
+
+ /*
+ * Enter IQK mode
+ */
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
+ val32 &= 0x000000ff;
+ val32 |= 0x80800000;
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
+
+ if (priv->rf_paths > 1)
+ rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, 0x00000000);
+ else
+ rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, 0x00000280);
+
+ /*
+ * Disable BT
+ */
+ rtl8xxxu_write32(priv, REG_BT_CONTROL_8723BU, 0x00000800);
+
+ /* One shot, path A LOK & IQK */
+ rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf9000000);
+ rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf8000000);
+
+ mdelay(1);
+
+ /* Restore Ant Path */
+ rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, path_sel);
+#ifdef RTL8723BU_BT
+ /* GNT_BT = 1 */
+ rtl8xxxu_write32(priv, REG_BT_CONTROL_8723BU, 0x00001800);
+#endif
+
+ /*
+ * Leave IQK mode
+ */
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
+ val32 &= 0x000000ff;
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
+
+ /* Check failed */
+ reg_eac = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_A_2);
+ reg_ea4 = rtl8xxxu_read32(priv, REG_RX_POWER_BEFORE_IQK_A_2);
+
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_DF, 0x780);
+
+ val32 = (reg_eac >> 16) & 0x3ff;
+ if (val32 & 0x200)
+ val32 = 0x400 - val32;
+
+ if (!(reg_eac & BIT(27)) &&
+ ((reg_ea4 & 0x03ff0000) != 0x01320000) &&
+ ((reg_eac & 0x03ff0000) != 0x00360000) &&
+ ((reg_ea4 & 0x03ff0000) < 0x01100000) &&
+ ((reg_ea4 & 0x03ff0000) > 0x00f00000) &&
+ val32 < 0xf)
+ result |= 0x02;
+ else /* If TX not OK, ignore RX */
+ goto out;
+out:
+ return result;
+}
+
+static void rtl8723bu_phy_iqcalibrate(struct rtl8xxxu_priv *priv,
+ int result[][8], int t)
+{
+ struct device *dev = &priv->udev->dev;
+ u32 i, val32;
+ int path_a_ok /*, path_b_ok */;
+ int retry = 2;
+ const u32 adda_regs[RTL8XXXU_ADDA_REGS] = {
+ REG_FPGA0_XCD_SWITCH_CTRL, REG_BLUETOOTH,
+ REG_RX_WAIT_CCA, REG_TX_CCK_RFON,
+ REG_TX_CCK_BBON, REG_TX_OFDM_RFON,
+ REG_TX_OFDM_BBON, REG_TX_TO_RX,
+ REG_TX_TO_TX, REG_RX_CCK,
+ REG_RX_OFDM, REG_RX_WAIT_RIFS,
+ REG_RX_TO_RX, REG_STANDBY,
+ REG_SLEEP, REG_PMPD_ANAEN
+ };
+ const u32 iqk_mac_regs[RTL8XXXU_MAC_REGS] = {
+ REG_TXPAUSE, REG_BEACON_CTRL,
+ REG_BEACON_CTRL_1, REG_GPIO_MUXCFG
+ };
+ const u32 iqk_bb_regs[RTL8XXXU_BB_REGS] = {
+ REG_OFDM0_TRX_PATH_ENABLE, REG_OFDM0_TR_MUX_PAR,
+ REG_FPGA0_XCD_RF_SW_CTRL, REG_CONFIG_ANT_A, REG_CONFIG_ANT_B,
+ REG_FPGA0_XAB_RF_SW_CTRL, REG_FPGA0_XA_RF_INT_OE,
+ REG_FPGA0_XB_RF_INT_OE, REG_FPGA0_RF_MODE
+ };
+ u8 xa_agc = rtl8xxxu_read32(priv, REG_OFDM0_XA_AGC_CORE1) & 0xff;
+ u8 xb_agc = rtl8xxxu_read32(priv, REG_OFDM0_XB_AGC_CORE1) & 0xff;
+
+ /*
+ * Note: IQ calibration must be performed after loading
+ * PHY_REG.txt , and radio_a, radio_b.txt
+ */
+
+ if (t == 0) {
+ /* Save ADDA parameters, turn Path A ADDA on */
+ rtl8xxxu_save_regs(priv, adda_regs, priv->adda_backup,
+ RTL8XXXU_ADDA_REGS);
+ rtl8xxxu_save_mac_regs(priv, iqk_mac_regs, priv->mac_backup);
+ rtl8xxxu_save_regs(priv, iqk_bb_regs,
+ priv->bb_backup, RTL8XXXU_BB_REGS);
+ }
+
+ rtl8xxxu_path_adda_on(priv, adda_regs, true);
+
+ /* MAC settings */
+ rtl8xxxu_mac_calibration(priv, iqk_mac_regs, priv->mac_backup);
+
+ val32 = rtl8xxxu_read32(priv, REG_CCK0_AFE_SETTING);
+ val32 |= 0x0f000000;
+ rtl8xxxu_write32(priv, REG_CCK0_AFE_SETTING, val32);
+
+ rtl8xxxu_write32(priv, REG_OFDM0_TRX_PATH_ENABLE, 0x03a05600);
+ rtl8xxxu_write32(priv, REG_OFDM0_TR_MUX_PAR, 0x000800e4);
+ rtl8xxxu_write32(priv, REG_FPGA0_XCD_RF_SW_CTRL, 0x22204000);
+
+ /*
+ * RX IQ calibration setting for 8723B D cut large current issue
+ * when leaving IPS
+ */
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
+ val32 &= 0x000000ff;
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
+
+ val32 = rtl8xxxu_read_rfreg(priv, RF_A, RF6052_REG_WE_LUT);
+ val32 |= 0x80000;
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_WE_LUT, val32);
+
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_RCK_OS, 0x30000);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G1, 0x0001f);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G2, 0xf7fb7);
+
+ val32 = rtl8xxxu_read_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_ED);
+ val32 |= 0x20;
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_ED, val32);
+
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_43, 0x60fbd);
+
+ for (i = 0; i < retry; i++) {
+ path_a_ok = rtl8723bu_iqk_path_a(priv);
+ if (path_a_ok == 0x01) {
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
+ val32 &= 0x000000ff;
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
+
+ val32 = rtl8xxxu_read32(priv,
+ REG_TX_POWER_BEFORE_IQK_A);
+ result[t][0] = (val32 >> 16) & 0x3ff;
+ val32 = rtl8xxxu_read32(priv,
+ REG_TX_POWER_AFTER_IQK_A);
+ result[t][1] = (val32 >> 16) & 0x3ff;
+
+ break;
+ }
+ }
+
+ if (!path_a_ok)
+ dev_dbg(dev, "%s: Path A TX IQK failed!\n", __func__);
+
+ for (i = 0; i < retry; i++) {
+ path_a_ok = rtl8723bu_rx_iqk_path_a(priv);
+ if (path_a_ok == 0x03) {
+ val32 = rtl8xxxu_read32(priv,
+ REG_RX_POWER_BEFORE_IQK_A_2);
+ result[t][2] = (val32 >> 16) & 0x3ff;
+ val32 = rtl8xxxu_read32(priv,
+ REG_RX_POWER_AFTER_IQK_A_2);
+ result[t][3] = (val32 >> 16) & 0x3ff;
+
+ break;
+ }
+ }
+
+ if (!path_a_ok)
+ dev_dbg(dev, "%s: Path A RX IQK failed!\n", __func__);
+
+ if (priv->tx_paths > 1) {
+#if 1
+ dev_warn(dev, "%s: Path B not supported\n", __func__);
+#else
+
+ /*
+ * Path A into standby
+ */
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
+ val32 &= 0x000000ff;
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_AC, 0x10000);
+
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
+ val32 &= 0x000000ff;
+ val32 |= 0x80800000;
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
+
+ /* Turn Path B ADDA on */
+ rtl8xxxu_path_adda_on(priv, adda_regs, false);
+
+ for (i = 0; i < retry; i++) {
+ path_b_ok = rtl8xxxu_iqk_path_b(priv);
+ if (path_b_ok == 0x03) {
+ val32 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_B);
+ result[t][4] = (val32 >> 16) & 0x3ff;
+ val32 = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_B);
+ result[t][5] = (val32 >> 16) & 0x3ff;
+ break;
+ }
+ }
+
+ if (!path_b_ok)
+ dev_dbg(dev, "%s: Path B IQK failed!\n", __func__);
+
+ for (i = 0; i < retry; i++) {
+ path_b_ok = rtl8723bu_rx_iqk_path_b(priv);
+ if (path_a_ok == 0x03) {
+ val32 = rtl8xxxu_read32(priv,
+ REG_RX_POWER_BEFORE_IQK_B_2);
+ result[t][6] = (val32 >> 16) & 0x3ff;
+ val32 = rtl8xxxu_read32(priv,
+ REG_RX_POWER_AFTER_IQK_B_2);
+ result[t][7] = (val32 >> 16) & 0x3ff;
+ break;
+ }
+ }
+
+ if (!path_b_ok)
+ dev_dbg(dev, "%s: Path B RX IQK failed!\n", __func__);
+#endif
+ }
+
+ /* Back to BB mode, load original value */
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
+ val32 &= 0x000000ff;
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
+
+ if (t) {
+ /* Reload ADDA power saving parameters */
+ rtl8xxxu_restore_regs(priv, adda_regs, priv->adda_backup,
+ RTL8XXXU_ADDA_REGS);
+
+ /* Reload MAC parameters */
+ rtl8xxxu_restore_mac_regs(priv, iqk_mac_regs, priv->mac_backup);
+
+ /* Reload BB parameters */
+ rtl8xxxu_restore_regs(priv, iqk_bb_regs,
+ priv->bb_backup, RTL8XXXU_BB_REGS);
+
+ /* Restore RX initial gain */
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_XA_AGC_CORE1);
+ val32 &= 0xffffff00;
+ rtl8xxxu_write32(priv, REG_OFDM0_XA_AGC_CORE1, val32 | 0x50);
+ rtl8xxxu_write32(priv, REG_OFDM0_XA_AGC_CORE1, val32 | xa_agc);
+
+ if (priv->tx_paths > 1) {
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_XB_AGC_CORE1);
+ val32 &= 0xffffff00;
+ rtl8xxxu_write32(priv, REG_OFDM0_XB_AGC_CORE1,
+ val32 | 0x50);
+ rtl8xxxu_write32(priv, REG_OFDM0_XB_AGC_CORE1,
+ val32 | xb_agc);
+ }
+
+ /* Load 0xe30 IQC default value */
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x01008c00);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x01008c00);
+ }
+}
+
+static void rtl8723bu_phy_iq_calibrate(struct rtl8xxxu_priv *priv)
+{
+ struct device *dev = &priv->udev->dev;
+ int result[4][8]; /* last is final result */
+ int i, candidate;
+ bool path_a_ok, path_b_ok;
+ u32 reg_e94, reg_e9c, reg_ea4, reg_eac;
+ u32 reg_eb4, reg_ebc, reg_ec4, reg_ecc;
+ u32 val32, bt_control;
+ s32 reg_tmp = 0;
+ bool simu;
+
+ rtl8xxxu_gen2_prepare_calibrate(priv, 1);
+
+ memset(result, 0, sizeof(result));
+ candidate = -1;
+
+ path_a_ok = false;
+ path_b_ok = false;
+
+ bt_control = rtl8xxxu_read32(priv, REG_BT_CONTROL_8723BU);
+
+ for (i = 0; i < 3; i++) {
+ rtl8723bu_phy_iqcalibrate(priv, result, i);
+
+ if (i == 1) {
+ simu = rtl8xxxu_gen2_simularity_compare(priv,
+ result, 0, 1);
+ if (simu) {
+ candidate = 0;
+ break;
+ }
+ }
+
+ if (i == 2) {
+ simu = rtl8xxxu_gen2_simularity_compare(priv,
+ result, 0, 2);
+ if (simu) {
+ candidate = 0;
+ break;
+ }
+
+ simu = rtl8xxxu_gen2_simularity_compare(priv,
+ result, 1, 2);
+ if (simu) {
+ candidate = 1;
+ } else {
+ for (i = 0; i < 8; i++)
+ reg_tmp += result[3][i];
+
+ if (reg_tmp)
+ candidate = 3;
+ else
+ candidate = -1;
+ }
+ }
+ }
+
+ for (i = 0; i < 4; i++) {
+ reg_e94 = result[i][0];
+ reg_e9c = result[i][1];
+ reg_ea4 = result[i][2];
+ reg_eac = result[i][3];
+ reg_eb4 = result[i][4];
+ reg_ebc = result[i][5];
+ reg_ec4 = result[i][6];
+ reg_ecc = result[i][7];
+ }
+
+ if (candidate >= 0) {
+ reg_e94 = result[candidate][0];
+ priv->rege94 = reg_e94;
+ reg_e9c = result[candidate][1];
+ priv->rege9c = reg_e9c;
+ reg_ea4 = result[candidate][2];
+ reg_eac = result[candidate][3];
+ reg_eb4 = result[candidate][4];
+ priv->regeb4 = reg_eb4;
+ reg_ebc = result[candidate][5];
+ priv->regebc = reg_ebc;
+ reg_ec4 = result[candidate][6];
+ reg_ecc = result[candidate][7];
+ dev_dbg(dev, "%s: candidate is %x\n", __func__, candidate);
+ dev_dbg(dev,
+ "%s: e94 =%x e9c=%x ea4=%x eac=%x eb4=%x ebc=%x ec4=%x "
+ "ecc=%x\n ", __func__, reg_e94, reg_e9c,
+ reg_ea4, reg_eac, reg_eb4, reg_ebc, reg_ec4, reg_ecc);
+ path_a_ok = true;
+ path_b_ok = true;
+ } else {
+ reg_e94 = reg_eb4 = priv->rege94 = priv->regeb4 = 0x100;
+ reg_e9c = reg_ebc = priv->rege9c = priv->regebc = 0x0;
+ }
+
+ if (reg_e94 && candidate >= 0)
+ rtl8xxxu_fill_iqk_matrix_a(priv, path_a_ok, result,
+ candidate, (reg_ea4 == 0));
+
+ if (priv->tx_paths > 1 && reg_eb4)
+ rtl8xxxu_fill_iqk_matrix_b(priv, path_b_ok, result,
+ candidate, (reg_ec4 == 0));
+
+ rtl8xxxu_save_regs(priv, rtl8xxxu_iqk_phy_iq_bb_reg,
+ priv->bb_recovery_backup, RTL8XXXU_BB_REGS);
+
+ rtl8xxxu_write32(priv, REG_BT_CONTROL_8723BU, bt_control);
+
+ val32 = rtl8xxxu_read_rfreg(priv, RF_A, RF6052_REG_WE_LUT);
+ val32 |= 0x80000;
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_WE_LUT, val32);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_RCK_OS, 0x18000);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G1, 0x0001f);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_TXPA_G2, 0xe6177);
+ val32 = rtl8xxxu_read_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_ED);
+ val32 |= 0x20;
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_UNKNOWN_ED, val32);
+ rtl8xxxu_write_rfreg(priv, RF_A, 0x43, 0x300bd);
+
+ if (priv->rf_paths > 1)
+ dev_dbg(dev, "%s: 8723BU 2T not supported\n", __func__);
+
+ rtl8xxxu_gen2_prepare_calibrate(priv, 0);
+}
+
+static int rtl8723bu_active_to_emu(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+ u16 val16;
+ u32 val32;
+ int count, ret = 0;
+
+ /* Turn off RF */
+ rtl8xxxu_write8(priv, REG_RF_CTRL, 0);
+
+ /* Enable rising edge triggering interrupt */
+ val16 = rtl8xxxu_read16(priv, REG_GPIO_INTM);
+ val16 &= ~GPIO_INTM_EDGE_TRIG_IRQ;
+ rtl8xxxu_write16(priv, REG_GPIO_INTM, val16);
+
+ /* Release WLON reset 0x04[16]= 1*/
+ val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
+ val32 |= APS_FSMCO_WLON_RESET;
+ rtl8xxxu_write32(priv, REG_APS_FSMCO, val32);
+
+ /* 0x0005[1] = 1 turn off MAC by HW state machine*/
+ val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
+ val8 |= BIT(1);
+ rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
+
+ for (count = RTL8XXXU_MAX_REG_POLL; count; count--) {
+ val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
+ if ((val8 & BIT(1)) == 0)
+ break;
+ udelay(10);
+ }
+
+ if (!count) {
+ dev_warn(&priv->udev->dev, "%s: Disabling MAC timed out\n",
+ __func__);
+ ret = -EBUSY;
+ goto exit;
+ }
+
+ /* Enable BT control XTAL setting */
+ val8 = rtl8xxxu_read8(priv, REG_AFE_MISC);
+ val8 &= ~AFE_MISC_WL_XTAL_CTRL;
+ rtl8xxxu_write8(priv, REG_AFE_MISC, val8);
+
+ /* 0x0000[5] = 1 analog Ips to digital, 1:isolation */
+ val8 = rtl8xxxu_read8(priv, REG_SYS_ISO_CTRL);
+ val8 |= SYS_ISO_ANALOG_IPS;
+ rtl8xxxu_write8(priv, REG_SYS_ISO_CTRL, val8);
+
+ /* 0x0020[0] = 0 disable LDOA12 MACRO block*/
+ val8 = rtl8xxxu_read8(priv, REG_LDOA15_CTRL);
+ val8 &= ~LDOA15_ENABLE;
+ rtl8xxxu_write8(priv, REG_LDOA15_CTRL, val8);
+
+exit:
+ return ret;
+}
+
+static int rtl8723b_emu_to_active(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+ u32 val32;
+ int count, ret = 0;
+
+ /* 0x20[0] = 1 enable LDOA12 MACRO block for all interface */
+ val8 = rtl8xxxu_read8(priv, REG_LDOA15_CTRL);
+ val8 |= LDOA15_ENABLE;
+ rtl8xxxu_write8(priv, REG_LDOA15_CTRL, val8);
+
+ /* 0x67[0] = 0 to disable BT_GPS_SEL pins*/
+ val8 = rtl8xxxu_read8(priv, 0x0067);
+ val8 &= ~BIT(4);
+ rtl8xxxu_write8(priv, 0x0067, val8);
+
+ mdelay(1);
+
+ /* 0x00[5] = 0 release analog Ips to digital, 1:isolation */
+ val8 = rtl8xxxu_read8(priv, REG_SYS_ISO_CTRL);
+ val8 &= ~SYS_ISO_ANALOG_IPS;
+ rtl8xxxu_write8(priv, REG_SYS_ISO_CTRL, val8);
+
+ /* Disable SW LPS 0x04[10]= 0 */
+ val32 = rtl8xxxu_read8(priv, REG_APS_FSMCO);
+ val32 &= ~APS_FSMCO_SW_LPS;
+ rtl8xxxu_write32(priv, REG_APS_FSMCO, val32);
+
+ /* Wait until 0x04[17] = 1 power ready */
+ for (count = RTL8XXXU_MAX_REG_POLL; count; count--) {
+ val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
+ if (val32 & BIT(17))
+ break;
+
+ udelay(10);
+ }
+
+ if (!count) {
+ ret = -EBUSY;
+ goto exit;
+ }
+
+ /* We should be able to optimize the following three entries into one */
+
+ /* Release WLON reset 0x04[16]= 1*/
+ val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
+ val32 |= APS_FSMCO_WLON_RESET;
+ rtl8xxxu_write32(priv, REG_APS_FSMCO, val32);
+
+ /* Disable HWPDN 0x04[15]= 0*/
+ val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
+ val32 &= ~APS_FSMCO_HW_POWERDOWN;
+ rtl8xxxu_write32(priv, REG_APS_FSMCO, val32);
+
+ /* Disable WL suspend*/
+ val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
+ val32 &= ~(APS_FSMCO_HW_SUSPEND | APS_FSMCO_PCIE);
+ rtl8xxxu_write32(priv, REG_APS_FSMCO, val32);
+
+ /* Set, then poll until 0 */
+ val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
+ val32 |= APS_FSMCO_MAC_ENABLE;
+ rtl8xxxu_write32(priv, REG_APS_FSMCO, val32);
+
+ for (count = RTL8XXXU_MAX_REG_POLL; count; count--) {
+ val32 = rtl8xxxu_read32(priv, REG_APS_FSMCO);
+ if ((val32 & APS_FSMCO_MAC_ENABLE) == 0) {
+ ret = 0;
+ break;
+ }
+ udelay(10);
+ }
+
+ if (!count) {
+ ret = -EBUSY;
+ goto exit;
+ }
+
+ /* Enable WL control XTAL setting */
+ val8 = rtl8xxxu_read8(priv, REG_AFE_MISC);
+ val8 |= AFE_MISC_WL_XTAL_CTRL;
+ rtl8xxxu_write8(priv, REG_AFE_MISC, val8);
+
+ /* Enable falling edge triggering interrupt */
+ val8 = rtl8xxxu_read8(priv, REG_GPIO_INTM + 1);
+ val8 |= BIT(1);
+ rtl8xxxu_write8(priv, REG_GPIO_INTM + 1, val8);
+
+ /* Enable GPIO9 interrupt mode */
+ val8 = rtl8xxxu_read8(priv, REG_GPIO_IO_SEL_2 + 1);
+ val8 |= BIT(1);
+ rtl8xxxu_write8(priv, REG_GPIO_IO_SEL_2 + 1, val8);
+
+ /* Enable GPIO9 input mode */
+ val8 = rtl8xxxu_read8(priv, REG_GPIO_IO_SEL_2);
+ val8 &= ~BIT(1);
+ rtl8xxxu_write8(priv, REG_GPIO_IO_SEL_2, val8);
+
+ /* Enable HSISR GPIO[C:0] interrupt */
+ val8 = rtl8xxxu_read8(priv, REG_HSIMR);
+ val8 |= BIT(0);
+ rtl8xxxu_write8(priv, REG_HSIMR, val8);
+
+ /* Enable HSISR GPIO9 interrupt */
+ val8 = rtl8xxxu_read8(priv, REG_HSIMR + 2);
+ val8 |= BIT(1);
+ rtl8xxxu_write8(priv, REG_HSIMR + 2, val8);
+
+ val8 = rtl8xxxu_read8(priv, REG_MULTI_FUNC_CTRL);
+ val8 |= MULTI_WIFI_HW_ROF_EN;
+ rtl8xxxu_write8(priv, REG_MULTI_FUNC_CTRL, val8);
+
+ /* For GPIO9 internal pull high setting BIT(14) */
+ val8 = rtl8xxxu_read8(priv, REG_MULTI_FUNC_CTRL + 1);
+ val8 |= BIT(6);
+ rtl8xxxu_write8(priv, REG_MULTI_FUNC_CTRL + 1, val8);
+
+exit:
+ return ret;
+}
+
+static int rtl8723bu_power_on(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+ u16 val16;
+ u32 val32;
+ int ret;
+
+ rtl8xxxu_disabled_to_emu(priv);
+
+ ret = rtl8723b_emu_to_active(priv);
+ if (ret)
+ goto exit;
+
+ /*
+ * Enable MAC DMA/WMAC/SCHEDULE/SEC block
+ * Set CR bit10 to enable 32k calibration.
+ */
+ val16 = rtl8xxxu_read16(priv, REG_CR);
+ val16 |= (CR_HCI_TXDMA_ENABLE | CR_HCI_RXDMA_ENABLE |
+ CR_TXDMA_ENABLE | CR_RXDMA_ENABLE |
+ CR_PROTOCOL_ENABLE | CR_SCHEDULE_ENABLE |
+ CR_MAC_TX_ENABLE | CR_MAC_RX_ENABLE |
+ CR_SECURITY_ENABLE | CR_CALTIMER_ENABLE);
+ rtl8xxxu_write16(priv, REG_CR, val16);
+
+ /*
+ * BT coexist power on settings. This is identical for 1 and 2
+ * antenna parts.
+ */
+ rtl8xxxu_write8(priv, REG_PAD_CTRL1 + 3, 0x20);
+
+ val16 = rtl8xxxu_read16(priv, REG_SYS_FUNC);
+ val16 |= SYS_FUNC_BBRSTB | SYS_FUNC_BB_GLB_RSTN;
+ rtl8xxxu_write16(priv, REG_SYS_FUNC, val16);
+
+ rtl8xxxu_write8(priv, REG_BT_CONTROL_8723BU + 1, 0x18);
+ rtl8xxxu_write8(priv, REG_WLAN_ACT_CONTROL_8723B, 0x04);
+ rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, 0x00);
+ /* Antenna inverse */
+ rtl8xxxu_write8(priv, 0xfe08, 0x01);
+
+ val16 = rtl8xxxu_read16(priv, REG_PWR_DATA);
+ val16 |= PWR_DATA_EEPRPAD_RFE_CTRL_EN;
+ rtl8xxxu_write16(priv, REG_PWR_DATA, val16);
+
+ val32 = rtl8xxxu_read32(priv, REG_LEDCFG0);
+ val32 |= LEDCFG0_DPDT_SELECT;
+ rtl8xxxu_write32(priv, REG_LEDCFG0, val32);
+
+ val8 = rtl8xxxu_read8(priv, REG_PAD_CTRL1);
+ val8 &= ~PAD_CTRL1_SW_DPDT_SEL_DATA;
+ rtl8xxxu_write8(priv, REG_PAD_CTRL1, val8);
+exit:
+ return ret;
+}
+
+static void rtl8723bu_power_off(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+ u16 val16;
+
+ rtl8xxxu_flush_fifo(priv);
+
+ /*
+ * Disable TX report timer
+ */
+ val8 = rtl8xxxu_read8(priv, REG_TX_REPORT_CTRL);
+ val8 &= ~TX_REPORT_CTRL_TIMER_ENABLE;
+ rtl8xxxu_write8(priv, REG_TX_REPORT_CTRL, val8);
+
+ rtl8xxxu_write8(priv, REG_CR, 0x0000);
+
+ rtl8xxxu_active_to_lps(priv);
+
+ /* Reset Firmware if running in RAM */
+ if (rtl8xxxu_read8(priv, REG_MCU_FW_DL) & MCU_FW_RAM_SEL)
+ rtl8xxxu_firmware_self_reset(priv);
+
+ /* Reset MCU */
+ val16 = rtl8xxxu_read16(priv, REG_SYS_FUNC);
+ val16 &= ~SYS_FUNC_CPU_ENABLE;
+ rtl8xxxu_write16(priv, REG_SYS_FUNC, val16);
+
+ /* Reset MCU ready status */
+ rtl8xxxu_write8(priv, REG_MCU_FW_DL, 0x00);
+
+ rtl8723bu_active_to_emu(priv);
+
+ val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
+ val8 |= BIT(3); /* APS_FSMCO_HW_SUSPEND */
+ rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
+
+ /* 0x48[16] = 1 to enable GPIO9 as EXT wakeup */
+ val8 = rtl8xxxu_read8(priv, REG_GPIO_INTM + 2);
+ val8 |= BIT(0);
+ rtl8xxxu_write8(priv, REG_GPIO_INTM + 2, val8);
+}
+
+static void rtl8723b_enable_rf(struct rtl8xxxu_priv *priv)
+{
+ struct h2c_cmd h2c;
+ u32 val32;
+ u8 val8;
+
+ /*
+ * No indication anywhere as to what 0x0790 does. The 2 antenna
+ * vendor code preserves bits 6-7 here.
+ */
+ rtl8xxxu_write8(priv, 0x0790, 0x05);
+ /*
+ * 0x0778 seems to be related to enabling the number of antennas
+ * In the vendor driver halbtc8723b2ant_InitHwConfig() sets it
+ * to 0x03, while halbtc8723b1ant_InitHwConfig() sets it to 0x01
+ */
+ rtl8xxxu_write8(priv, 0x0778, 0x01);
+
+ val8 = rtl8xxxu_read8(priv, REG_GPIO_MUXCFG);
+ val8 |= BIT(5);
+ rtl8xxxu_write8(priv, REG_GPIO_MUXCFG, val8);
+
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_IQADJ_G1, 0x780);
+
+ rtl8723bu_write_btreg(priv, 0x3c, 0x15); /* BT TRx Mask on */
+
+ /*
+ * Set BT grant to low
+ */
+ memset(&h2c, 0, sizeof(struct h2c_cmd));
+ h2c.bt_grant.cmd = H2C_8723B_BT_GRANT;
+ h2c.bt_grant.data = 0;
+ rtl8xxxu_gen2_h2c_cmd(priv, &h2c, sizeof(h2c.bt_grant));
+
+ /*
+ * WLAN action by PTA
+ */
+ rtl8xxxu_write8(priv, REG_WLAN_ACT_CONTROL_8723B, 0x04);
+
+ /*
+ * BT select S0/S1 controlled by WiFi
+ */
+ val8 = rtl8xxxu_read8(priv, 0x0067);
+ val8 |= BIT(5);
+ rtl8xxxu_write8(priv, 0x0067, val8);
+
+ val32 = rtl8xxxu_read32(priv, REG_PWR_DATA);
+ val32 |= PWR_DATA_EEPRPAD_RFE_CTRL_EN;
+ rtl8xxxu_write32(priv, REG_PWR_DATA, val32);
+
+ /*
+ * Bits 6/7 are marked in/out ... but for what?
+ */
+ rtl8xxxu_write8(priv, 0x0974, 0xff);
+
+ val32 = rtl8xxxu_read32(priv, REG_RFE_BUFFER);
+ val32 |= (BIT(0) | BIT(1));
+ rtl8xxxu_write32(priv, REG_RFE_BUFFER, val32);
+
+ rtl8xxxu_write8(priv, REG_RFE_CTRL_ANTA_SRC, 0x77);
+
+ val32 = rtl8xxxu_read32(priv, REG_LEDCFG0);
+ val32 &= ~BIT(24);
+ val32 |= BIT(23);
+ rtl8xxxu_write32(priv, REG_LEDCFG0, val32);
+
+ /*
+ * Fix external switch Main->S1, Aux->S0
+ */
+ val8 = rtl8xxxu_read8(priv, REG_PAD_CTRL1);
+ val8 &= ~BIT(0);
+ rtl8xxxu_write8(priv, REG_PAD_CTRL1, val8);
+
+ memset(&h2c, 0, sizeof(struct h2c_cmd));
+ h2c.ant_sel_rsv.cmd = H2C_8723B_ANT_SEL_RSV;
+ h2c.ant_sel_rsv.ant_inverse = 1;
+ h2c.ant_sel_rsv.int_switch_type = 0;
+ rtl8xxxu_gen2_h2c_cmd(priv, &h2c, sizeof(h2c.ant_sel_rsv));
+
+ /*
+ * 0x280, 0x00, 0x200, 0x80 - not clear
+ */
+ rtl8xxxu_write32(priv, REG_S0S1_PATH_SWITCH, 0x00);
+
+ /*
+ * Software control, antenna at WiFi side
+ */
+#ifdef NEED_PS_TDMA
+ rtl8723bu_set_ps_tdma(priv, 0x08, 0x00, 0x00, 0x00, 0x00);
+#endif
+
+ rtl8xxxu_write32(priv, REG_BT_COEX_TABLE1, 0x55555555);
+ rtl8xxxu_write32(priv, REG_BT_COEX_TABLE2, 0x55555555);
+ rtl8xxxu_write32(priv, REG_BT_COEX_TABLE3, 0x00ffffff);
+ rtl8xxxu_write8(priv, REG_BT_COEX_TABLE4, 0x03);
+
+ memset(&h2c, 0, sizeof(struct h2c_cmd));
+ h2c.bt_info.cmd = H2C_8723B_BT_INFO;
+ h2c.bt_info.data = BIT(0);
+ rtl8xxxu_gen2_h2c_cmd(priv, &h2c, sizeof(h2c.bt_info));
+
+ memset(&h2c, 0, sizeof(struct h2c_cmd));
+ h2c.ignore_wlan.cmd = H2C_8723B_BT_IGNORE_WLANACT;
+ h2c.ignore_wlan.data = 0;
+ rtl8xxxu_gen2_h2c_cmd(priv, &h2c, sizeof(h2c.ignore_wlan));
+}
+
+static void rtl8723bu_init_aggregation(struct rtl8xxxu_priv *priv)
+{
+ u32 agg_rx;
+ u8 agg_ctrl;
+
+ /*
+ * For now simply disable RX aggregation
+ */
+ agg_ctrl = rtl8xxxu_read8(priv, REG_TRXDMA_CTRL);
+ agg_ctrl &= ~TRXDMA_CTRL_RXDMA_AGG_EN;
+
+ agg_rx = rtl8xxxu_read32(priv, REG_RXDMA_AGG_PG_TH);
+ agg_rx &= ~RXDMA_USB_AGG_ENABLE;
+ agg_rx &= ~0xff0f;
+
+ rtl8xxxu_write8(priv, REG_TRXDMA_CTRL, agg_ctrl);
+ rtl8xxxu_write32(priv, REG_RXDMA_AGG_PG_TH, agg_rx);
+}
+
+static void rtl8723bu_init_statistics(struct rtl8xxxu_priv *priv)
+{
+ u32 val32;
+
+ /* Time duration for NHM unit: 4us, 0x2710=40ms */
+ rtl8xxxu_write16(priv, REG_NHM_TIMER_8723B + 2, 0x2710);
+ rtl8xxxu_write16(priv, REG_NHM_TH9_TH10_8723B + 2, 0xffff);
+ rtl8xxxu_write32(priv, REG_NHM_TH3_TO_TH0_8723B, 0xffffff52);
+ rtl8xxxu_write32(priv, REG_NHM_TH7_TO_TH4_8723B, 0xffffffff);
+ /* TH8 */
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_IQK);
+ val32 |= 0xff;
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, val32);
+ /* Enable CCK */
+ val32 = rtl8xxxu_read32(priv, REG_NHM_TH9_TH10_8723B);
+ val32 |= BIT(8) | BIT(9) | BIT(10);
+ rtl8xxxu_write32(priv, REG_NHM_TH9_TH10_8723B, val32);
+ /* Max power amongst all RX antennas */
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_FA_RSTC);
+ val32 |= BIT(7);
+ rtl8xxxu_write32(priv, REG_OFDM0_FA_RSTC, val32);
+}
+
+struct rtl8xxxu_fileops rtl8723bu_fops = {
+ .parse_efuse = rtl8723bu_parse_efuse,
+ .load_firmware = rtl8723bu_load_firmware,
+ .power_on = rtl8723bu_power_on,
+ .power_off = rtl8723bu_power_off,
+ .reset_8051 = rtl8723bu_reset_8051,
+ .llt_init = rtl8xxxu_auto_llt_table,
+ .init_phy_bb = rtl8723bu_init_phy_bb,
+ .init_phy_rf = rtl8723bu_init_phy_rf,
+ .phy_init_antenna_selection = rtl8723bu_phy_init_antenna_selection,
+ .phy_iq_calibrate = rtl8723bu_phy_iq_calibrate,
+ .config_channel = rtl8xxxu_gen2_config_channel,
+ .parse_rx_desc = rtl8xxxu_parse_rxdesc24,
+ .init_aggregation = rtl8723bu_init_aggregation,
+ .init_statistics = rtl8723bu_init_statistics,
+ .enable_rf = rtl8723b_enable_rf,
+ .disable_rf = rtl8xxxu_gen2_disable_rf,
+ .usb_quirks = rtl8xxxu_gen2_usb_quirks,
+ .set_tx_power = rtl8723b_set_tx_power,
+ .update_rate_mask = rtl8xxxu_gen2_update_rate_mask,
+ .report_connect = rtl8xxxu_gen2_report_connect,
+ .writeN_block_size = 1024,
+ .tx_desc_size = sizeof(struct rtl8xxxu_txdesc40),
+ .rx_desc_size = sizeof(struct rtl8xxxu_rxdesc24),
+ .has_s0s1 = 1,
+ .adda_1t_init = 0x01c00014,
+ .adda_1t_path_on = 0x01c00014,
+ .adda_2t_path_on_a = 0x01c00014,
+ .adda_2t_path_on_b = 0x01c00014,
+ .trxff_boundary = 0x3f7f,
+ .pbp_rx = PBP_PAGE_SIZE_256,
+ .pbp_tx = PBP_PAGE_SIZE_256,
+ .mactable = rtl8723b_mac_init_table,
+};
diff --git a/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_core.c b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_core.c
new file mode 100644
index 0000000..9f6dbb4
--- /dev/null
+++ b/drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_core.c
@@ -0,0 +1,6187 @@
+/*
+ * RTL8XXXU mac80211 USB driver
+ *
+ * Copyright (c) 2014 - 2016 Jes Sorensen <Jes.Sorensen@redhat.com>
+ *
+ * Portions, notably calibration code:
+ * Copyright(c) 2007 - 2011 Realtek Corporation. All rights reserved.
+ *
+ * This driver was written as a replacement for the vendor provided
+ * rtl8723au driver. As the Realtek 8xxx chips are very similar in
+ * their programming interface, I have started adding support for
+ * additional 8xxx chips like the 8192cu, 8188cus, etc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ */
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <linux/errno.h>
+#include <linux/slab.h>
+#include <linux/module.h>
+#include <linux/spinlock.h>
+#include <linux/list.h>
+#include <linux/usb.h>
+#include <linux/netdevice.h>
+#include <linux/etherdevice.h>
+#include <linux/ethtool.h>
+#include <linux/wireless.h>
+#include <linux/firmware.h>
+#include <linux/moduleparam.h>
+#include <net/mac80211.h>
+#include "rtl8xxxu.h"
+#include "rtl8xxxu_regs.h"
+
+#define DRIVER_NAME "rtl8xxxu"
+
+int rtl8xxxu_debug = RTL8XXXU_DEBUG_EFUSE;
+static bool rtl8xxxu_ht40_2g;
+
+MODULE_AUTHOR("Jes Sorensen <Jes.Sorensen@redhat.com>");
+MODULE_DESCRIPTION("RTL8XXXu USB mac80211 Wireless LAN Driver");
+MODULE_LICENSE("GPL");
+MODULE_FIRMWARE("rtlwifi/rtl8723aufw_A.bin");
+MODULE_FIRMWARE("rtlwifi/rtl8723aufw_B.bin");
+MODULE_FIRMWARE("rtlwifi/rtl8723aufw_B_NoBT.bin");
+MODULE_FIRMWARE("rtlwifi/rtl8192cufw_A.bin");
+MODULE_FIRMWARE("rtlwifi/rtl8192cufw_B.bin");
+MODULE_FIRMWARE("rtlwifi/rtl8192cufw_TMSC.bin");
+MODULE_FIRMWARE("rtlwifi/rtl8192eu_nic.bin");
+MODULE_FIRMWARE("rtlwifi/rtl8723bu_nic.bin");
+MODULE_FIRMWARE("rtlwifi/rtl8723bu_bt.bin");
+
+module_param_named(debug, rtl8xxxu_debug, int, 0600);
+MODULE_PARM_DESC(debug, "Set debug mask");
+module_param_named(ht40_2g, rtl8xxxu_ht40_2g, bool, 0600);
+MODULE_PARM_DESC(ht40_2g, "Enable HT40 support on the 2.4GHz band");
+
+#define USB_VENDOR_ID_REALTEK 0x0bda
+/* Minimum IEEE80211_MAX_FRAME_LEN */
+#define RTL_RX_BUFFER_SIZE IEEE80211_MAX_FRAME_LEN
+#define RTL8XXXU_RX_URBS 32
+#define RTL8XXXU_RX_URB_PENDING_WATER 8
+#define RTL8XXXU_TX_URBS 64
+#define RTL8XXXU_TX_URB_LOW_WATER 25
+#define RTL8XXXU_TX_URB_HIGH_WATER 32
+
+static int rtl8xxxu_submit_rx_urb(struct rtl8xxxu_priv *priv,
+ struct rtl8xxxu_rx_urb *rx_urb);
+
+static struct ieee80211_rate rtl8xxxu_rates[] = {
+ { .bitrate = 10, .hw_value = DESC_RATE_1M, .flags = 0 },
+ { .bitrate = 20, .hw_value = DESC_RATE_2M, .flags = 0 },
+ { .bitrate = 55, .hw_value = DESC_RATE_5_5M, .flags = 0 },
+ { .bitrate = 110, .hw_value = DESC_RATE_11M, .flags = 0 },
+ { .bitrate = 60, .hw_value = DESC_RATE_6M, .flags = 0 },
+ { .bitrate = 90, .hw_value = DESC_RATE_9M, .flags = 0 },
+ { .bitrate = 120, .hw_value = DESC_RATE_12M, .flags = 0 },
+ { .bitrate = 180, .hw_value = DESC_RATE_18M, .flags = 0 },
+ { .bitrate = 240, .hw_value = DESC_RATE_24M, .flags = 0 },
+ { .bitrate = 360, .hw_value = DESC_RATE_36M, .flags = 0 },
+ { .bitrate = 480, .hw_value = DESC_RATE_48M, .flags = 0 },
+ { .bitrate = 540, .hw_value = DESC_RATE_54M, .flags = 0 },
+};
+
+static struct ieee80211_channel rtl8xxxu_channels_2g[] = {
+ { .band = NL80211_BAND_2GHZ, .center_freq = 2412,
+ .hw_value = 1, .max_power = 30 },
+ { .band = NL80211_BAND_2GHZ, .center_freq = 2417,
+ .hw_value = 2, .max_power = 30 },
+ { .band = NL80211_BAND_2GHZ, .center_freq = 2422,
+ .hw_value = 3, .max_power = 30 },
+ { .band = NL80211_BAND_2GHZ, .center_freq = 2427,
+ .hw_value = 4, .max_power = 30 },
+ { .band = NL80211_BAND_2GHZ, .center_freq = 2432,
+ .hw_value = 5, .max_power = 30 },
+ { .band = NL80211_BAND_2GHZ, .center_freq = 2437,
+ .hw_value = 6, .max_power = 30 },
+ { .band = NL80211_BAND_2GHZ, .center_freq = 2442,
+ .hw_value = 7, .max_power = 30 },
+ { .band = NL80211_BAND_2GHZ, .center_freq = 2447,
+ .hw_value = 8, .max_power = 30 },
+ { .band = NL80211_BAND_2GHZ, .center_freq = 2452,
+ .hw_value = 9, .max_power = 30 },
+ { .band = NL80211_BAND_2GHZ, .center_freq = 2457,
+ .hw_value = 10, .max_power = 30 },
+ { .band = NL80211_BAND_2GHZ, .center_freq = 2462,
+ .hw_value = 11, .max_power = 30 },
+ { .band = NL80211_BAND_2GHZ, .center_freq = 2467,
+ .hw_value = 12, .max_power = 30 },
+ { .band = NL80211_BAND_2GHZ, .center_freq = 2472,
+ .hw_value = 13, .max_power = 30 },
+ { .band = NL80211_BAND_2GHZ, .center_freq = 2484,
+ .hw_value = 14, .max_power = 30 }
+};
+
+static struct ieee80211_supported_band rtl8xxxu_supported_band = {
+ .channels = rtl8xxxu_channels_2g,
+ .n_channels = ARRAY_SIZE(rtl8xxxu_channels_2g),
+ .bitrates = rtl8xxxu_rates,
+ .n_bitrates = ARRAY_SIZE(rtl8xxxu_rates),
+};
+
+struct rtl8xxxu_reg8val rtl8xxxu_gen1_mac_init_table[] = {
+ {0x420, 0x80}, {0x423, 0x00}, {0x430, 0x00}, {0x431, 0x00},
+ {0x432, 0x00}, {0x433, 0x01}, {0x434, 0x04}, {0x435, 0x05},
+ {0x436, 0x06}, {0x437, 0x07}, {0x438, 0x00}, {0x439, 0x00},
+ {0x43a, 0x00}, {0x43b, 0x01}, {0x43c, 0x04}, {0x43d, 0x05},
+ {0x43e, 0x06}, {0x43f, 0x07}, {0x440, 0x5d}, {0x441, 0x01},
+ {0x442, 0x00}, {0x444, 0x15}, {0x445, 0xf0}, {0x446, 0x0f},
+ {0x447, 0x00}, {0x458, 0x41}, {0x459, 0xa8}, {0x45a, 0x72},
+ {0x45b, 0xb9}, {0x460, 0x66}, {0x461, 0x66}, {0x462, 0x08},
+ {0x463, 0x03}, {0x4c8, 0xff}, {0x4c9, 0x08}, {0x4cc, 0xff},
+ {0x4cd, 0xff}, {0x4ce, 0x01}, {0x500, 0x26}, {0x501, 0xa2},
+ {0x502, 0x2f}, {0x503, 0x00}, {0x504, 0x28}, {0x505, 0xa3},
+ {0x506, 0x5e}, {0x507, 0x00}, {0x508, 0x2b}, {0x509, 0xa4},
+ {0x50a, 0x5e}, {0x50b, 0x00}, {0x50c, 0x4f}, {0x50d, 0xa4},
+ {0x50e, 0x00}, {0x50f, 0x00}, {0x512, 0x1c}, {0x514, 0x0a},
+ {0x515, 0x10}, {0x516, 0x0a}, {0x517, 0x10}, {0x51a, 0x16},
+ {0x524, 0x0f}, {0x525, 0x4f}, {0x546, 0x40}, {0x547, 0x00},
+ {0x550, 0x10}, {0x551, 0x10}, {0x559, 0x02}, {0x55a, 0x02},
+ {0x55d, 0xff}, {0x605, 0x30}, {0x608, 0x0e}, {0x609, 0x2a},
+ {0x652, 0x20}, {0x63c, 0x0a}, {0x63d, 0x0a}, {0x63e, 0x0e},
+ {0x63f, 0x0e}, {0x66e, 0x05}, {0x700, 0x21}, {0x701, 0x43},
+ {0x702, 0x65}, {0x703, 0x87}, {0x708, 0x21}, {0x709, 0x43},
+ {0x70a, 0x65}, {0x70b, 0x87}, {0xffff, 0xff},
+};
+
+static struct rtl8xxxu_reg32val rtl8723a_phy_1t_init_table[] = {
+ {0x800, 0x80040000}, {0x804, 0x00000003},
+ {0x808, 0x0000fc00}, {0x80c, 0x0000000a},
+ {0x810, 0x10001331}, {0x814, 0x020c3d10},
+ {0x818, 0x02200385}, {0x81c, 0x00000000},
+ {0x820, 0x01000100}, {0x824, 0x00390004},
+ {0x828, 0x00000000}, {0x82c, 0x00000000},
+ {0x830, 0x00000000}, {0x834, 0x00000000},
+ {0x838, 0x00000000}, {0x83c, 0x00000000},
+ {0x840, 0x00010000}, {0x844, 0x00000000},
+ {0x848, 0x00000000}, {0x84c, 0x00000000},
+ {0x850, 0x00000000}, {0x854, 0x00000000},
+ {0x858, 0x569a569a}, {0x85c, 0x001b25a4},
+ {0x860, 0x66f60110}, {0x864, 0x061f0130},
+ {0x868, 0x00000000}, {0x86c, 0x32323200},
+ {0x870, 0x07000760}, {0x874, 0x22004000},
+ {0x878, 0x00000808}, {0x87c, 0x00000000},
+ {0x880, 0xc0083070}, {0x884, 0x000004d5},
+ {0x888, 0x00000000}, {0x88c, 0xccc000c0},
+ {0x890, 0x00000800}, {0x894, 0xfffffffe},
+ {0x898, 0x40302010}, {0x89c, 0x00706050},
+ {0x900, 0x00000000}, {0x904, 0x00000023},
+ {0x908, 0x00000000}, {0x90c, 0x81121111},
+ {0xa00, 0x00d047c8}, {0xa04, 0x80ff000c},
+ {0xa08, 0x8c838300}, {0xa0c, 0x2e68120f},
+ {0xa10, 0x9500bb78}, {0xa14, 0x11144028},
+ {0xa18, 0x00881117}, {0xa1c, 0x89140f00},
+ {0xa20, 0x1a1b0000}, {0xa24, 0x090e1317},
+ {0xa28, 0x00000204}, {0xa2c, 0x00d30000},
+ {0xa70, 0x101fbf00}, {0xa74, 0x00000007},
+ {0xa78, 0x00000900},
+ {0xc00, 0x48071d40}, {0xc04, 0x03a05611},
+ {0xc08, 0x000000e4}, {0xc0c, 0x6c6c6c6c},
+ {0xc10, 0x08800000}, {0xc14, 0x40000100},
+ {0xc18, 0x08800000}, {0xc1c, 0x40000100},
+ {0xc20, 0x00000000}, {0xc24, 0x00000000},
+ {0xc28, 0x00000000}, {0xc2c, 0x00000000},
+ {0xc30, 0x69e9ac44}, {0xc34, 0x469652af},
+ {0xc38, 0x49795994}, {0xc3c, 0x0a97971c},
+ {0xc40, 0x1f7c403f}, {0xc44, 0x000100b7},
+ {0xc48, 0xec020107}, {0xc4c, 0x007f037f},
+ {0xc50, 0x69543420}, {0xc54, 0x43bc0094},
+ {0xc58, 0x69543420}, {0xc5c, 0x433c0094},
+ {0xc60, 0x00000000}, {0xc64, 0x7112848b},
+ {0xc68, 0x47c00bff}, {0xc6c, 0x00000036},
+ {0xc70, 0x2c7f000d}, {0xc74, 0x018610db},
+ {0xc78, 0x0000001f}, {0xc7c, 0x00b91612},
+ {0xc80, 0x40000100}, {0xc84, 0x20f60000},
+ {0xc88, 0x40000100}, {0xc8c, 0x20200000},
+ {0xc90, 0x00121820}, {0xc94, 0x00000000},
+ {0xc98, 0x00121820}, {0xc9c, 0x00007f7f},
+ {0xca0, 0x00000000}, {0xca4, 0x00000080},
+ {0xca8, 0x00000000}, {0xcac, 0x00000000},
+ {0xcb0, 0x00000000}, {0xcb4, 0x00000000},
+ {0xcb8, 0x00000000}, {0xcbc, 0x28000000},
+ {0xcc0, 0x00000000}, {0xcc4, 0x00000000},
+ {0xcc8, 0x00000000}, {0xccc, 0x00000000},
+ {0xcd0, 0x00000000}, {0xcd4, 0x00000000},
+ {0xcd8, 0x64b22427}, {0xcdc, 0x00766932},
+ {0xce0, 0x00222222}, {0xce4, 0x00000000},
+ {0xce8, 0x37644302}, {0xcec, 0x2f97d40c},
+ {0xd00, 0x00080740}, {0xd04, 0x00020401},
+ {0xd08, 0x0000907f}, {0xd0c, 0x20010201},
+ {0xd10, 0xa0633333}, {0xd14, 0x3333bc43},
+ {0xd18, 0x7a8f5b6b}, {0xd2c, 0xcc979975},
+ {0xd30, 0x00000000}, {0xd34, 0x80608000},
+ {0xd38, 0x00000000}, {0xd3c, 0x00027293},
+ {0xd40, 0x00000000}, {0xd44, 0x00000000},
+ {0xd48, 0x00000000}, {0xd4c, 0x00000000},
+ {0xd50, 0x6437140a}, {0xd54, 0x00000000},
+ {0xd58, 0x00000000}, {0xd5c, 0x30032064},
+ {0xd60, 0x4653de68}, {0xd64, 0x04518a3c},
+ {0xd68, 0x00002101}, {0xd6c, 0x2a201c16},
+ {0xd70, 0x1812362e}, {0xd74, 0x322c2220},
+ {0xd78, 0x000e3c24}, {0xe00, 0x2a2a2a2a},
+ {0xe04, 0x2a2a2a2a}, {0xe08, 0x03902a2a},
+ {0xe10, 0x2a2a2a2a}, {0xe14, 0x2a2a2a2a},
+ {0xe18, 0x2a2a2a2a}, {0xe1c, 0x2a2a2a2a},
+ {0xe28, 0x00000000}, {0xe30, 0x1000dc1f},
+ {0xe34, 0x10008c1f}, {0xe38, 0x02140102},
+ {0xe3c, 0x681604c2}, {0xe40, 0x01007c00},
+ {0xe44, 0x01004800}, {0xe48, 0xfb000000},
+ {0xe4c, 0x000028d1}, {0xe50, 0x1000dc1f},
+ {0xe54, 0x10008c1f}, {0xe58, 0x02140102},
+ {0xe5c, 0x28160d05}, {0xe60, 0x00000008},
+ {0xe68, 0x001b25a4}, {0xe6c, 0x631b25a0},
+ {0xe70, 0x631b25a0}, {0xe74, 0x081b25a0},
+ {0xe78, 0x081b25a0}, {0xe7c, 0x081b25a0},
+ {0xe80, 0x081b25a0}, {0xe84, 0x631b25a0},
+ {0xe88, 0x081b25a0}, {0xe8c, 0x631b25a0},
+ {0xed0, 0x631b25a0}, {0xed4, 0x631b25a0},
+ {0xed8, 0x631b25a0}, {0xedc, 0x001b25a0},
+ {0xee0, 0x001b25a0}, {0xeec, 0x6b1b25a0},
+ {0xf14, 0x00000003}, {0xf4c, 0x00000000},
+ {0xf00, 0x00000300},
+ {0xffff, 0xffffffff},
+};
+
+static struct rtl8xxxu_reg32val rtl8192cu_phy_2t_init_table[] = {
+ {0x024, 0x0011800f}, {0x028, 0x00ffdb83},
+ {0x800, 0x80040002}, {0x804, 0x00000003},
+ {0x808, 0x0000fc00}, {0x80c, 0x0000000a},
+ {0x810, 0x10000330}, {0x814, 0x020c3d10},
+ {0x818, 0x02200385}, {0x81c, 0x00000000},
+ {0x820, 0x01000100}, {0x824, 0x00390004},
+ {0x828, 0x01000100}, {0x82c, 0x00390004},
+ {0x830, 0x27272727}, {0x834, 0x27272727},
+ {0x838, 0x27272727}, {0x83c, 0x27272727},
+ {0x840, 0x00010000}, {0x844, 0x00010000},
+ {0x848, 0x27272727}, {0x84c, 0x27272727},
+ {0x850, 0x00000000}, {0x854, 0x00000000},
+ {0x858, 0x569a569a}, {0x85c, 0x0c1b25a4},
+ {0x860, 0x66e60230}, {0x864, 0x061f0130},
+ {0x868, 0x27272727}, {0x86c, 0x2b2b2b27},
+ {0x870, 0x07000700}, {0x874, 0x22184000},
+ {0x878, 0x08080808}, {0x87c, 0x00000000},
+ {0x880, 0xc0083070}, {0x884, 0x000004d5},
+ {0x888, 0x00000000}, {0x88c, 0xcc0000c0},
+ {0x890, 0x00000800}, {0x894, 0xfffffffe},
+ {0x898, 0x40302010}, {0x89c, 0x00706050},
+ {0x900, 0x00000000}, {0x904, 0x00000023},
+ {0x908, 0x00000000}, {0x90c, 0x81121313},
+ {0xa00, 0x00d047c8}, {0xa04, 0x80ff000c},
+ {0xa08, 0x8c838300}, {0xa0c, 0x2e68120f},
+ {0xa10, 0x9500bb78}, {0xa14, 0x11144028},
+ {0xa18, 0x00881117}, {0xa1c, 0x89140f00},
+ {0xa20, 0x1a1b0000}, {0xa24, 0x090e1317},
+ {0xa28, 0x00000204}, {0xa2c, 0x00d30000},
+ {0xa70, 0x101fbf00}, {0xa74, 0x00000007},
+ {0xc00, 0x48071d40}, {0xc04, 0x03a05633},
+ {0xc08, 0x000000e4}, {0xc0c, 0x6c6c6c6c},
+ {0xc10, 0x08800000}, {0xc14, 0x40000100},
+ {0xc18, 0x08800000}, {0xc1c, 0x40000100},
+ {0xc20, 0x00000000}, {0xc24, 0x00000000},
+ {0xc28, 0x00000000}, {0xc2c, 0x00000000},
+ {0xc30, 0x69e9ac44}, {0xc34, 0x469652cf},
+ {0xc38, 0x49795994}, {0xc3c, 0x0a97971c},
+ {0xc40, 0x1f7c403f}, {0xc44, 0x000100b7},
+ {0xc48, 0xec020107}, {0xc4c, 0x007f037f},
+ {0xc50, 0x69543420}, {0xc54, 0x43bc0094},
+ {0xc58, 0x69543420}, {0xc5c, 0x433c0094},
+ {0xc60, 0x00000000}, {0xc64, 0x5116848b},
+ {0xc68, 0x47c00bff}, {0xc6c, 0x00000036},
+ {0xc70, 0x2c7f000d}, {0xc74, 0x2186115b},
+ {0xc78, 0x0000001f}, {0xc7c, 0x00b99612},
+ {0xc80, 0x40000100}, {0xc84, 0x20f60000},
+ {0xc88, 0x40000100}, {0xc8c, 0xa0e40000},
+ {0xc90, 0x00121820}, {0xc94, 0x00000000},
+ {0xc98, 0x00121820}, {0xc9c, 0x00007f7f},
+ {0xca0, 0x00000000}, {0xca4, 0x00000080},
+ {0xca8, 0x00000000}, {0xcac, 0x00000000},
+ {0xcb0, 0x00000000}, {0xcb4, 0x00000000},
+ {0xcb8, 0x00000000}, {0xcbc, 0x28000000},
+ {0xcc0, 0x00000000}, {0xcc4, 0x00000000},
+ {0xcc8, 0x00000000}, {0xccc, 0x00000000},
+ {0xcd0, 0x00000000}, {0xcd4, 0x00000000},
+ {0xcd8, 0x64b22427}, {0xcdc, 0x00766932},
+ {0xce0, 0x00222222}, {0xce4, 0x00000000},
+ {0xce8, 0x37644302}, {0xcec, 0x2f97d40c},
+ {0xd00, 0x00080740}, {0xd04, 0x00020403},
+ {0xd08, 0x0000907f}, {0xd0c, 0x20010201},
+ {0xd10, 0xa0633333}, {0xd14, 0x3333bc43},
+ {0xd18, 0x7a8f5b6b}, {0xd2c, 0xcc979975},
+ {0xd30, 0x00000000}, {0xd34, 0x80608000},
+ {0xd38, 0x00000000}, {0xd3c, 0x00027293},
+ {0xd40, 0x00000000}, {0xd44, 0x00000000},
+ {0xd48, 0x00000000}, {0xd4c, 0x00000000},
+ {0xd50, 0x6437140a}, {0xd54, 0x00000000},
+ {0xd58, 0x00000000}, {0xd5c, 0x30032064},
+ {0xd60, 0x4653de68}, {0xd64, 0x04518a3c},
+ {0xd68, 0x00002101}, {0xd6c, 0x2a201c16},
+ {0xd70, 0x1812362e}, {0xd74, 0x322c2220},
+ {0xd78, 0x000e3c24}, {0xe00, 0x2a2a2a2a},
+ {0xe04, 0x2a2a2a2a}, {0xe08, 0x03902a2a},
+ {0xe10, 0x2a2a2a2a}, {0xe14, 0x2a2a2a2a},
+ {0xe18, 0x2a2a2a2a}, {0xe1c, 0x2a2a2a2a},
+ {0xe28, 0x00000000}, {0xe30, 0x1000dc1f},
+ {0xe34, 0x10008c1f}, {0xe38, 0x02140102},
+ {0xe3c, 0x681604c2}, {0xe40, 0x01007c00},
+ {0xe44, 0x01004800}, {0xe48, 0xfb000000},
+ {0xe4c, 0x000028d1}, {0xe50, 0x1000dc1f},
+ {0xe54, 0x10008c1f}, {0xe58, 0x02140102},
+ {0xe5c, 0x28160d05}, {0xe60, 0x00000010},
+ {0xe68, 0x001b25a4}, {0xe6c, 0x63db25a4},
+ {0xe70, 0x63db25a4}, {0xe74, 0x0c1b25a4},
+ {0xe78, 0x0c1b25a4}, {0xe7c, 0x0c1b25a4},
+ {0xe80, 0x0c1b25a4}, {0xe84, 0x63db25a4},
+ {0xe88, 0x0c1b25a4}, {0xe8c, 0x63db25a4},
+ {0xed0, 0x63db25a4}, {0xed4, 0x63db25a4},
+ {0xed8, 0x63db25a4}, {0xedc, 0x001b25a4},
+ {0xee0, 0x001b25a4}, {0xeec, 0x6fdb25a4},
+ {0xf14, 0x00000003}, {0xf4c, 0x00000000},
+ {0xf00, 0x00000300},
+ {0xffff, 0xffffffff},
+};
+
+static struct rtl8xxxu_reg32val rtl8188ru_phy_1t_highpa_table[] = {
+ {0x024, 0x0011800f}, {0x028, 0x00ffdb83},
+ {0x040, 0x000c0004}, {0x800, 0x80040000},
+ {0x804, 0x00000001}, {0x808, 0x0000fc00},
+ {0x80c, 0x0000000a}, {0x810, 0x10005388},
+ {0x814, 0x020c3d10}, {0x818, 0x02200385},
+ {0x81c, 0x00000000}, {0x820, 0x01000100},
+ {0x824, 0x00390204}, {0x828, 0x00000000},
+ {0x82c, 0x00000000}, {0x830, 0x00000000},
+ {0x834, 0x00000000}, {0x838, 0x00000000},
+ {0x83c, 0x00000000}, {0x840, 0x00010000},
+ {0x844, 0x00000000}, {0x848, 0x00000000},
+ {0x84c, 0x00000000}, {0x850, 0x00000000},
+ {0x854, 0x00000000}, {0x858, 0x569a569a},
+ {0x85c, 0x001b25a4}, {0x860, 0x66e60230},
+ {0x864, 0x061f0130}, {0x868, 0x00000000},
+ {0x86c, 0x20202000}, {0x870, 0x03000300},
+ {0x874, 0x22004000}, {0x878, 0x00000808},
+ {0x87c, 0x00ffc3f1}, {0x880, 0xc0083070},
+ {0x884, 0x000004d5}, {0x888, 0x00000000},
+ {0x88c, 0xccc000c0}, {0x890, 0x00000800},
+ {0x894, 0xfffffffe}, {0x898, 0x40302010},
+ {0x89c, 0x00706050}, {0x900, 0x00000000},
+ {0x904, 0x00000023}, {0x908, 0x00000000},
+ {0x90c, 0x81121111}, {0xa00, 0x00d047c8},
+ {0xa04, 0x80ff000c}, {0xa08, 0x8c838300},
+ {0xa0c, 0x2e68120f}, {0xa10, 0x9500bb78},
+ {0xa14, 0x11144028}, {0xa18, 0x00881117},
+ {0xa1c, 0x89140f00}, {0xa20, 0x15160000},
+ {0xa24, 0x070b0f12}, {0xa28, 0x00000104},
+ {0xa2c, 0x00d30000}, {0xa70, 0x101fbf00},
+ {0xa74, 0x00000007}, {0xc00, 0x48071d40},
+ {0xc04, 0x03a05611}, {0xc08, 0x000000e4},
+ {0xc0c, 0x6c6c6c6c}, {0xc10, 0x08800000},
+ {0xc14, 0x40000100}, {0xc18, 0x08800000},
+ {0xc1c, 0x40000100}, {0xc20, 0x00000000},
+ {0xc24, 0x00000000}, {0xc28, 0x00000000},
+ {0xc2c, 0x00000000}, {0xc30, 0x69e9ac44},
+ {0xc34, 0x469652cf}, {0xc38, 0x49795994},
+ {0xc3c, 0x0a97971c}, {0xc40, 0x1f7c403f},
+ {0xc44, 0x000100b7}, {0xc48, 0xec020107},
+ {0xc4c, 0x007f037f}, {0xc50, 0x6954342e},
+ {0xc54, 0x43bc0094}, {0xc58, 0x6954342f},
+ {0xc5c, 0x433c0094}, {0xc60, 0x00000000},
+ {0xc64, 0x5116848b}, {0xc68, 0x47c00bff},
+ {0xc6c, 0x00000036}, {0xc70, 0x2c46000d},
+ {0xc74, 0x018610db}, {0xc78, 0x0000001f},
+ {0xc7c, 0x00b91612}, {0xc80, 0x24000090},
+ {0xc84, 0x20f60000}, {0xc88, 0x24000090},
+ {0xc8c, 0x20200000}, {0xc90, 0x00121820},
+ {0xc94, 0x00000000}, {0xc98, 0x00121820},
+ {0xc9c, 0x00007f7f}, {0xca0, 0x00000000},
+ {0xca4, 0x00000080}, {0xca8, 0x00000000},
+ {0xcac, 0x00000000}, {0xcb0, 0x00000000},
+ {0xcb4, 0x00000000}, {0xcb8, 0x00000000},
+ {0xcbc, 0x28000000}, {0xcc0, 0x00000000},
+ {0xcc4, 0x00000000}, {0xcc8, 0x00000000},
+ {0xccc, 0x00000000}, {0xcd0, 0x00000000},
+ {0xcd4, 0x00000000}, {0xcd8, 0x64b22427},
+ {0xcdc, 0x00766932}, {0xce0, 0x00222222},
+ {0xce4, 0x00000000}, {0xce8, 0x37644302},
+ {0xcec, 0x2f97d40c}, {0xd00, 0x00080740},
+ {0xd04, 0x00020401}, {0xd08, 0x0000907f},
+ {0xd0c, 0x20010201}, {0xd10, 0xa0633333},
+ {0xd14, 0x3333bc43}, {0xd18, 0x7a8f5b6b},
+ {0xd2c, 0xcc979975}, {0xd30, 0x00000000},
+ {0xd34, 0x80608000}, {0xd38, 0x00000000},
+ {0xd3c, 0x00027293}, {0xd40, 0x00000000},
+ {0xd44, 0x00000000}, {0xd48, 0x00000000},
+ {0xd4c, 0x00000000}, {0xd50, 0x6437140a},
+ {0xd54, 0x00000000}, {0xd58, 0x00000000},
+ {0xd5c, 0x30032064}, {0xd60, 0x4653de68},
+ {0xd64, 0x04518a3c}, {0xd68, 0x00002101},
+ {0xd6c, 0x2a201c16}, {0xd70, 0x1812362e},
+ {0xd74, 0x322c2220}, {0xd78, 0x000e3c24},
+ {0xe00, 0x24242424}, {0xe04, 0x24242424},
+ {0xe08, 0x03902024}, {0xe10, 0x24242424},
+ {0xe14, 0x24242424}, {0xe18, 0x24242424},
+ {0xe1c, 0x24242424}, {0xe28, 0x00000000},
+ {0xe30, 0x1000dc1f}, {0xe34, 0x10008c1f},
+ {0xe38, 0x02140102}, {0xe3c, 0x681604c2},
+ {0xe40, 0x01007c00}, {0xe44, 0x01004800},
+ {0xe48, 0xfb000000}, {0xe4c, 0x000028d1},
+ {0xe50, 0x1000dc1f}, {0xe54, 0x10008c1f},
+ {0xe58, 0x02140102}, {0xe5c, 0x28160d05},
+ {0xe60, 0x00000008}, {0xe68, 0x001b25a4},
+ {0xe6c, 0x631b25a0}, {0xe70, 0x631b25a0},
+ {0xe74, 0x081b25a0}, {0xe78, 0x081b25a0},
+ {0xe7c, 0x081b25a0}, {0xe80, 0x081b25a0},
+ {0xe84, 0x631b25a0}, {0xe88, 0x081b25a0},
+ {0xe8c, 0x631b25a0}, {0xed0, 0x631b25a0},
+ {0xed4, 0x631b25a0}, {0xed8, 0x631b25a0},
+ {0xedc, 0x001b25a0}, {0xee0, 0x001b25a0},
+ {0xeec, 0x6b1b25a0}, {0xee8, 0x31555448},
+ {0xf14, 0x00000003}, {0xf4c, 0x00000000},
+ {0xf00, 0x00000300},
+ {0xffff, 0xffffffff},
+};
+
+static struct rtl8xxxu_reg32val rtl8xxx_agc_standard_table[] = {
+ {0xc78, 0x7b000001}, {0xc78, 0x7b010001},
+ {0xc78, 0x7b020001}, {0xc78, 0x7b030001},
+ {0xc78, 0x7b040001}, {0xc78, 0x7b050001},
+ {0xc78, 0x7a060001}, {0xc78, 0x79070001},
+ {0xc78, 0x78080001}, {0xc78, 0x77090001},
+ {0xc78, 0x760a0001}, {0xc78, 0x750b0001},
+ {0xc78, 0x740c0001}, {0xc78, 0x730d0001},
+ {0xc78, 0x720e0001}, {0xc78, 0x710f0001},
+ {0xc78, 0x70100001}, {0xc78, 0x6f110001},
+ {0xc78, 0x6e120001}, {0xc78, 0x6d130001},
+ {0xc78, 0x6c140001}, {0xc78, 0x6b150001},
+ {0xc78, 0x6a160001}, {0xc78, 0x69170001},
+ {0xc78, 0x68180001}, {0xc78, 0x67190001},
+ {0xc78, 0x661a0001}, {0xc78, 0x651b0001},
+ {0xc78, 0x641c0001}, {0xc78, 0x631d0001},
+ {0xc78, 0x621e0001}, {0xc78, 0x611f0001},
+ {0xc78, 0x60200001}, {0xc78, 0x49210001},
+ {0xc78, 0x48220001}, {0xc78, 0x47230001},
+ {0xc78, 0x46240001}, {0xc78, 0x45250001},
+ {0xc78, 0x44260001}, {0xc78, 0x43270001},
+ {0xc78, 0x42280001}, {0xc78, 0x41290001},
+ {0xc78, 0x402a0001}, {0xc78, 0x262b0001},
+ {0xc78, 0x252c0001}, {0xc78, 0x242d0001},
+ {0xc78, 0x232e0001}, {0xc78, 0x222f0001},
+ {0xc78, 0x21300001}, {0xc78, 0x20310001},
+ {0xc78, 0x06320001}, {0xc78, 0x05330001},
+ {0xc78, 0x04340001}, {0xc78, 0x03350001},
+ {0xc78, 0x02360001}, {0xc78, 0x01370001},
+ {0xc78, 0x00380001}, {0xc78, 0x00390001},
+ {0xc78, 0x003a0001}, {0xc78, 0x003b0001},
+ {0xc78, 0x003c0001}, {0xc78, 0x003d0001},
+ {0xc78, 0x003e0001}, {0xc78, 0x003f0001},
+ {0xc78, 0x7b400001}, {0xc78, 0x7b410001},
+ {0xc78, 0x7b420001}, {0xc78, 0x7b430001},
+ {0xc78, 0x7b440001}, {0xc78, 0x7b450001},
+ {0xc78, 0x7a460001}, {0xc78, 0x79470001},
+ {0xc78, 0x78480001}, {0xc78, 0x77490001},
+ {0xc78, 0x764a0001}, {0xc78, 0x754b0001},
+ {0xc78, 0x744c0001}, {0xc78, 0x734d0001},
+ {0xc78, 0x724e0001}, {0xc78, 0x714f0001},
+ {0xc78, 0x70500001}, {0xc78, 0x6f510001},
+ {0xc78, 0x6e520001}, {0xc78, 0x6d530001},
+ {0xc78, 0x6c540001}, {0xc78, 0x6b550001},
+ {0xc78, 0x6a560001}, {0xc78, 0x69570001},
+ {0xc78, 0x68580001}, {0xc78, 0x67590001},
+ {0xc78, 0x665a0001}, {0xc78, 0x655b0001},
+ {0xc78, 0x645c0001}, {0xc78, 0x635d0001},
+ {0xc78, 0x625e0001}, {0xc78, 0x615f0001},
+ {0xc78, 0x60600001}, {0xc78, 0x49610001},
+ {0xc78, 0x48620001}, {0xc78, 0x47630001},
+ {0xc78, 0x46640001}, {0xc78, 0x45650001},
+ {0xc78, 0x44660001}, {0xc78, 0x43670001},
+ {0xc78, 0x42680001}, {0xc78, 0x41690001},
+ {0xc78, 0x406a0001}, {0xc78, 0x266b0001},
+ {0xc78, 0x256c0001}, {0xc78, 0x246d0001},
+ {0xc78, 0x236e0001}, {0xc78, 0x226f0001},
+ {0xc78, 0x21700001}, {0xc78, 0x20710001},
+ {0xc78, 0x06720001}, {0xc78, 0x05730001},
+ {0xc78, 0x04740001}, {0xc78, 0x03750001},
+ {0xc78, 0x02760001}, {0xc78, 0x01770001},
+ {0xc78, 0x00780001}, {0xc78, 0x00790001},
+ {0xc78, 0x007a0001}, {0xc78, 0x007b0001},
+ {0xc78, 0x007c0001}, {0xc78, 0x007d0001},
+ {0xc78, 0x007e0001}, {0xc78, 0x007f0001},
+ {0xc78, 0x3800001e}, {0xc78, 0x3801001e},
+ {0xc78, 0x3802001e}, {0xc78, 0x3803001e},
+ {0xc78, 0x3804001e}, {0xc78, 0x3805001e},
+ {0xc78, 0x3806001e}, {0xc78, 0x3807001e},
+ {0xc78, 0x3808001e}, {0xc78, 0x3c09001e},
+ {0xc78, 0x3e0a001e}, {0xc78, 0x400b001e},
+ {0xc78, 0x440c001e}, {0xc78, 0x480d001e},
+ {0xc78, 0x4c0e001e}, {0xc78, 0x500f001e},
+ {0xc78, 0x5210001e}, {0xc78, 0x5611001e},
+ {0xc78, 0x5a12001e}, {0xc78, 0x5e13001e},
+ {0xc78, 0x6014001e}, {0xc78, 0x6015001e},
+ {0xc78, 0x6016001e}, {0xc78, 0x6217001e},
+ {0xc78, 0x6218001e}, {0xc78, 0x6219001e},
+ {0xc78, 0x621a001e}, {0xc78, 0x621b001e},
+ {0xc78, 0x621c001e}, {0xc78, 0x621d001e},
+ {0xc78, 0x621e001e}, {0xc78, 0x621f001e},
+ {0xffff, 0xffffffff}
+};
+
+static struct rtl8xxxu_reg32val rtl8xxx_agc_highpa_table[] = {
+ {0xc78, 0x7b000001}, {0xc78, 0x7b010001},
+ {0xc78, 0x7b020001}, {0xc78, 0x7b030001},
+ {0xc78, 0x7b040001}, {0xc78, 0x7b050001},
+ {0xc78, 0x7b060001}, {0xc78, 0x7b070001},
+ {0xc78, 0x7b080001}, {0xc78, 0x7a090001},
+ {0xc78, 0x790a0001}, {0xc78, 0x780b0001},
+ {0xc78, 0x770c0001}, {0xc78, 0x760d0001},
+ {0xc78, 0x750e0001}, {0xc78, 0x740f0001},
+ {0xc78, 0x73100001}, {0xc78, 0x72110001},
+ {0xc78, 0x71120001}, {0xc78, 0x70130001},
+ {0xc78, 0x6f140001}, {0xc78, 0x6e150001},
+ {0xc78, 0x6d160001}, {0xc78, 0x6c170001},
+ {0xc78, 0x6b180001}, {0xc78, 0x6a190001},
+ {0xc78, 0x691a0001}, {0xc78, 0x681b0001},
+ {0xc78, 0x671c0001}, {0xc78, 0x661d0001},
+ {0xc78, 0x651e0001}, {0xc78, 0x641f0001},
+ {0xc78, 0x63200001}, {0xc78, 0x62210001},
+ {0xc78, 0x61220001}, {0xc78, 0x60230001},
+ {0xc78, 0x46240001}, {0xc78, 0x45250001},
+ {0xc78, 0x44260001}, {0xc78, 0x43270001},
+ {0xc78, 0x42280001}, {0xc78, 0x41290001},
+ {0xc78, 0x402a0001}, {0xc78, 0x262b0001},
+ {0xc78, 0x252c0001}, {0xc78, 0x242d0001},
+ {0xc78, 0x232e0001}, {0xc78, 0x222f0001},
+ {0xc78, 0x21300001}, {0xc78, 0x20310001},
+ {0xc78, 0x06320001}, {0xc78, 0x05330001},
+ {0xc78, 0x04340001}, {0xc78, 0x03350001},
+ {0xc78, 0x02360001}, {0xc78, 0x01370001},
+ {0xc78, 0x00380001}, {0xc78, 0x00390001},
+ {0xc78, 0x003a0001}, {0xc78, 0x003b0001},
+ {0xc78, 0x003c0001}, {0xc78, 0x003d0001},
+ {0xc78, 0x003e0001}, {0xc78, 0x003f0001},
+ {0xc78, 0x7b400001}, {0xc78, 0x7b410001},
+ {0xc78, 0x7b420001}, {0xc78, 0x7b430001},
+ {0xc78, 0x7b440001}, {0xc78, 0x7b450001},
+ {0xc78, 0x7b460001}, {0xc78, 0x7b470001},
+ {0xc78, 0x7b480001}, {0xc78, 0x7a490001},
+ {0xc78, 0x794a0001}, {0xc78, 0x784b0001},
+ {0xc78, 0x774c0001}, {0xc78, 0x764d0001},
+ {0xc78, 0x754e0001}, {0xc78, 0x744f0001},
+ {0xc78, 0x73500001}, {0xc78, 0x72510001},
+ {0xc78, 0x71520001}, {0xc78, 0x70530001},
+ {0xc78, 0x6f540001}, {0xc78, 0x6e550001},
+ {0xc78, 0x6d560001}, {0xc78, 0x6c570001},
+ {0xc78, 0x6b580001}, {0xc78, 0x6a590001},
+ {0xc78, 0x695a0001}, {0xc78, 0x685b0001},
+ {0xc78, 0x675c0001}, {0xc78, 0x665d0001},
+ {0xc78, 0x655e0001}, {0xc78, 0x645f0001},
+ {0xc78, 0x63600001}, {0xc78, 0x62610001},
+ {0xc78, 0x61620001}, {0xc78, 0x60630001},
+ {0xc78, 0x46640001}, {0xc78, 0x45650001},
+ {0xc78, 0x44660001}, {0xc78, 0x43670001},
+ {0xc78, 0x42680001}, {0xc78, 0x41690001},
+ {0xc78, 0x406a0001}, {0xc78, 0x266b0001},
+ {0xc78, 0x256c0001}, {0xc78, 0x246d0001},
+ {0xc78, 0x236e0001}, {0xc78, 0x226f0001},
+ {0xc78, 0x21700001}, {0xc78, 0x20710001},
+ {0xc78, 0x06720001}, {0xc78, 0x05730001},
+ {0xc78, 0x04740001}, {0xc78, 0x03750001},
+ {0xc78, 0x02760001}, {0xc78, 0x01770001},
+ {0xc78, 0x00780001}, {0xc78, 0x00790001},
+ {0xc78, 0x007a0001}, {0xc78, 0x007b0001},
+ {0xc78, 0x007c0001}, {0xc78, 0x007d0001},
+ {0xc78, 0x007e0001}, {0xc78, 0x007f0001},
+ {0xc78, 0x3800001e}, {0xc78, 0x3801001e},
+ {0xc78, 0x3802001e}, {0xc78, 0x3803001e},
+ {0xc78, 0x3804001e}, {0xc78, 0x3805001e},
+ {0xc78, 0x3806001e}, {0xc78, 0x3807001e},
+ {0xc78, 0x3808001e}, {0xc78, 0x3c09001e},
+ {0xc78, 0x3e0a001e}, {0xc78, 0x400b001e},
+ {0xc78, 0x440c001e}, {0xc78, 0x480d001e},
+ {0xc78, 0x4c0e001e}, {0xc78, 0x500f001e},
+ {0xc78, 0x5210001e}, {0xc78, 0x5611001e},
+ {0xc78, 0x5a12001e}, {0xc78, 0x5e13001e},
+ {0xc78, 0x6014001e}, {0xc78, 0x6015001e},
+ {0xc78, 0x6016001e}, {0xc78, 0x6217001e},
+ {0xc78, 0x6218001e}, {0xc78, 0x6219001e},
+ {0xc78, 0x621a001e}, {0xc78, 0x621b001e},
+ {0xc78, 0x621c001e}, {0xc78, 0x621d001e},
+ {0xc78, 0x621e001e}, {0xc78, 0x621f001e},
+ {0xffff, 0xffffffff}
+};
+
+static struct rtl8xxxu_rfregs rtl8xxxu_rfregs[] = {
+ { /* RF_A */
+ .hssiparm1 = REG_FPGA0_XA_HSSI_PARM1,
+ .hssiparm2 = REG_FPGA0_XA_HSSI_PARM2,
+ .lssiparm = REG_FPGA0_XA_LSSI_PARM,
+ .hspiread = REG_HSPI_XA_READBACK,
+ .lssiread = REG_FPGA0_XA_LSSI_READBACK,
+ .rf_sw_ctrl = REG_FPGA0_XA_RF_SW_CTRL,
+ },
+ { /* RF_B */
+ .hssiparm1 = REG_FPGA0_XB_HSSI_PARM1,
+ .hssiparm2 = REG_FPGA0_XB_HSSI_PARM2,
+ .lssiparm = REG_FPGA0_XB_LSSI_PARM,
+ .hspiread = REG_HSPI_XB_READBACK,
+ .lssiread = REG_FPGA0_XB_LSSI_READBACK,
+ .rf_sw_ctrl = REG_FPGA0_XB_RF_SW_CTRL,
+ },
+};
+
+const u32 rtl8xxxu_iqk_phy_iq_bb_reg[RTL8XXXU_BB_REGS] = {
+ REG_OFDM0_XA_RX_IQ_IMBALANCE,
+ REG_OFDM0_XB_RX_IQ_IMBALANCE,
+ REG_OFDM0_ENERGY_CCA_THRES,
+ REG_OFDM0_AGCR_SSI_TABLE,
+ REG_OFDM0_XA_TX_IQ_IMBALANCE,
+ REG_OFDM0_XB_TX_IQ_IMBALANCE,
+ REG_OFDM0_XC_TX_AFE,
+ REG_OFDM0_XD_TX_AFE,
+ REG_OFDM0_RX_IQ_EXT_ANTA
+};
+
+u8 rtl8xxxu_read8(struct rtl8xxxu_priv *priv, u16 addr)
+{
+ struct usb_device *udev = priv->udev;
+ int len;
+ u8 data;
+
+ mutex_lock(&priv->usb_buf_mutex);
+ len = usb_control_msg(udev, usb_rcvctrlpipe(udev, 0),
+ REALTEK_USB_CMD_REQ, REALTEK_USB_READ,
+ addr, 0, &priv->usb_buf.val8, sizeof(u8),
+ RTW_USB_CONTROL_MSG_TIMEOUT);
+ data = priv->usb_buf.val8;
+ mutex_unlock(&priv->usb_buf_mutex);
+
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_REG_READ)
+ dev_info(&udev->dev, "%s(%04x) = 0x%02x, len %i\n",
+ __func__, addr, data, len);
+ return data;
+}
+
+u16 rtl8xxxu_read16(struct rtl8xxxu_priv *priv, u16 addr)
+{
+ struct usb_device *udev = priv->udev;
+ int len;
+ u16 data;
+
+ mutex_lock(&priv->usb_buf_mutex);
+ len = usb_control_msg(udev, usb_rcvctrlpipe(udev, 0),
+ REALTEK_USB_CMD_REQ, REALTEK_USB_READ,
+ addr, 0, &priv->usb_buf.val16, sizeof(u16),
+ RTW_USB_CONTROL_MSG_TIMEOUT);
+ data = le16_to_cpu(priv->usb_buf.val16);
+ mutex_unlock(&priv->usb_buf_mutex);
+
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_REG_READ)
+ dev_info(&udev->dev, "%s(%04x) = 0x%04x, len %i\n",
+ __func__, addr, data, len);
+ return data;
+}
+
+u32 rtl8xxxu_read32(struct rtl8xxxu_priv *priv, u16 addr)
+{
+ struct usb_device *udev = priv->udev;
+ int len;
+ u32 data;
+
+ mutex_lock(&priv->usb_buf_mutex);
+ len = usb_control_msg(udev, usb_rcvctrlpipe(udev, 0),
+ REALTEK_USB_CMD_REQ, REALTEK_USB_READ,
+ addr, 0, &priv->usb_buf.val32, sizeof(u32),
+ RTW_USB_CONTROL_MSG_TIMEOUT);
+ data = le32_to_cpu(priv->usb_buf.val32);
+ mutex_unlock(&priv->usb_buf_mutex);
+
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_REG_READ)
+ dev_info(&udev->dev, "%s(%04x) = 0x%08x, len %i\n",
+ __func__, addr, data, len);
+ return data;
+}
+
+int rtl8xxxu_write8(struct rtl8xxxu_priv *priv, u16 addr, u8 val)
+{
+ struct usb_device *udev = priv->udev;
+ int ret;
+
+ mutex_lock(&priv->usb_buf_mutex);
+ priv->usb_buf.val8 = val;
+ ret = usb_control_msg(udev, usb_sndctrlpipe(udev, 0),
+ REALTEK_USB_CMD_REQ, REALTEK_USB_WRITE,
+ addr, 0, &priv->usb_buf.val8, sizeof(u8),
+ RTW_USB_CONTROL_MSG_TIMEOUT);
+
+ mutex_unlock(&priv->usb_buf_mutex);
+
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_REG_WRITE)
+ dev_info(&udev->dev, "%s(%04x) = 0x%02x\n",
+ __func__, addr, val);
+ return ret;
+}
+
+int rtl8xxxu_write16(struct rtl8xxxu_priv *priv, u16 addr, u16 val)
+{
+ struct usb_device *udev = priv->udev;
+ int ret;
+
+ mutex_lock(&priv->usb_buf_mutex);
+ priv->usb_buf.val16 = cpu_to_le16(val);
+ ret = usb_control_msg(udev, usb_sndctrlpipe(udev, 0),
+ REALTEK_USB_CMD_REQ, REALTEK_USB_WRITE,
+ addr, 0, &priv->usb_buf.val16, sizeof(u16),
+ RTW_USB_CONTROL_MSG_TIMEOUT);
+ mutex_unlock(&priv->usb_buf_mutex);
+
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_REG_WRITE)
+ dev_info(&udev->dev, "%s(%04x) = 0x%04x\n",
+ __func__, addr, val);
+ return ret;
+}
+
+int rtl8xxxu_write32(struct rtl8xxxu_priv *priv, u16 addr, u32 val)
+{
+ struct usb_device *udev = priv->udev;
+ int ret;
+
+ mutex_lock(&priv->usb_buf_mutex);
+ priv->usb_buf.val32 = cpu_to_le32(val);
+ ret = usb_control_msg(udev, usb_sndctrlpipe(udev, 0),
+ REALTEK_USB_CMD_REQ, REALTEK_USB_WRITE,
+ addr, 0, &priv->usb_buf.val32, sizeof(u32),
+ RTW_USB_CONTROL_MSG_TIMEOUT);
+ mutex_unlock(&priv->usb_buf_mutex);
+
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_REG_WRITE)
+ dev_info(&udev->dev, "%s(%04x) = 0x%08x\n",
+ __func__, addr, val);
+ return ret;
+}
+
+static int
+rtl8xxxu_writeN(struct rtl8xxxu_priv *priv, u16 addr, u8 *buf, u16 len)
+{
+ struct usb_device *udev = priv->udev;
+ int blocksize = priv->fops->writeN_block_size;
+ int ret, i, count, remainder;
+
+ count = len / blocksize;
+ remainder = len % blocksize;
+
+ for (i = 0; i < count; i++) {
+ ret = usb_control_msg(udev, usb_sndctrlpipe(udev, 0),
+ REALTEK_USB_CMD_REQ, REALTEK_USB_WRITE,
+ addr, 0, buf, blocksize,
+ RTW_USB_CONTROL_MSG_TIMEOUT);
+ if (ret != blocksize)
+ goto write_error;
+
+ addr += blocksize;
+ buf += blocksize;
+ }
+
+ if (remainder) {
+ ret = usb_control_msg(udev, usb_sndctrlpipe(udev, 0),
+ REALTEK_USB_CMD_REQ, REALTEK_USB_WRITE,
+ addr, 0, buf, remainder,
+ RTW_USB_CONTROL_MSG_TIMEOUT);
+ if (ret != remainder)
+ goto write_error;
+ }
+
+ return len;
+
+write_error:
+ dev_info(&udev->dev,
+ "%s: Failed to write block at addr: %04x size: %04x\n",
+ __func__, addr, blocksize);
+ return -EAGAIN;
+}
+
+u32 rtl8xxxu_read_rfreg(struct rtl8xxxu_priv *priv,
+ enum rtl8xxxu_rfpath path, u8 reg)
+{
+ u32 hssia, val32, retval;
+
+ hssia = rtl8xxxu_read32(priv, REG_FPGA0_XA_HSSI_PARM2);
+ if (path != RF_A)
+ val32 = rtl8xxxu_read32(priv, rtl8xxxu_rfregs[path].hssiparm2);
+ else
+ val32 = hssia;
+
+ val32 &= ~FPGA0_HSSI_PARM2_ADDR_MASK;
+ val32 |= (reg << FPGA0_HSSI_PARM2_ADDR_SHIFT);
+ val32 |= FPGA0_HSSI_PARM2_EDGE_READ;
+ hssia &= ~FPGA0_HSSI_PARM2_EDGE_READ;
+ rtl8xxxu_write32(priv, REG_FPGA0_XA_HSSI_PARM2, hssia);
+
+ udelay(10);
+
+ rtl8xxxu_write32(priv, rtl8xxxu_rfregs[path].hssiparm2, val32);
+ udelay(100);
+
+ hssia |= FPGA0_HSSI_PARM2_EDGE_READ;
+ rtl8xxxu_write32(priv, REG_FPGA0_XA_HSSI_PARM2, hssia);
+ udelay(10);
+
+ val32 = rtl8xxxu_read32(priv, rtl8xxxu_rfregs[path].hssiparm1);
+ if (val32 & FPGA0_HSSI_PARM1_PI)
+ retval = rtl8xxxu_read32(priv, rtl8xxxu_rfregs[path].hspiread);
+ else
+ retval = rtl8xxxu_read32(priv, rtl8xxxu_rfregs[path].lssiread);
+
+ retval &= 0xfffff;
+
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_RFREG_READ)
+ dev_info(&priv->udev->dev, "%s(%02x) = 0x%06x\n",
+ __func__, reg, retval);
+ return retval;
+}
+
+/*
+ * The RTL8723BU driver indicates that registers 0xb2 and 0xb6 can
+ * have write issues in high temperature conditions. We may have to
+ * retry writing them.
+ */
+int rtl8xxxu_write_rfreg(struct rtl8xxxu_priv *priv,
+ enum rtl8xxxu_rfpath path, u8 reg, u32 data)
+{
+ int ret, retval;
+ u32 dataaddr, val32;
+
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_RFREG_WRITE)
+ dev_info(&priv->udev->dev, "%s(%02x) = 0x%06x\n",
+ __func__, reg, data);
+
+ data &= FPGA0_LSSI_PARM_DATA_MASK;
+ dataaddr = (reg << FPGA0_LSSI_PARM_ADDR_SHIFT) | data;
+
+ if (priv->rtl_chip == RTL8192E) {
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_POWER_SAVE);
+ val32 &= ~0x20000;
+ rtl8xxxu_write32(priv, REG_FPGA0_POWER_SAVE, val32);
+ }
+
+ /* Use XB for path B */
+ ret = rtl8xxxu_write32(priv, rtl8xxxu_rfregs[path].lssiparm, dataaddr);
+ if (ret != sizeof(dataaddr))
+ retval = -EIO;
+ else
+ retval = 0;
+
+ udelay(1);
+
+ if (priv->rtl_chip == RTL8192E) {
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_POWER_SAVE);
+ val32 |= 0x20000;
+ rtl8xxxu_write32(priv, REG_FPGA0_POWER_SAVE, val32);
+ }
+
+ return retval;
+}
+
+int
+rtl8xxxu_gen1_h2c_cmd(struct rtl8xxxu_priv *priv, struct h2c_cmd *h2c, int len)
+{
+ struct device *dev = &priv->udev->dev;
+ int mbox_nr, retry, retval = 0;
+ int mbox_reg, mbox_ext_reg;
+ u8 val8;
+
+ mutex_lock(&priv->h2c_mutex);
+
+ mbox_nr = priv->next_mbox;
+ mbox_reg = REG_HMBOX_0 + (mbox_nr * 4);
+ mbox_ext_reg = REG_HMBOX_EXT_0 + (mbox_nr * 2);
+
+ /*
+ * MBOX ready?
+ */
+ retry = 100;
+ do {
+ val8 = rtl8xxxu_read8(priv, REG_HMTFR);
+ if (!(val8 & BIT(mbox_nr)))
+ break;
+ } while (retry--);
+
+ if (!retry) {
+ dev_info(dev, "%s: Mailbox busy\n", __func__);
+ retval = -EBUSY;
+ goto error;
+ }
+
+ /*
+ * Need to swap as it's being swapped again by rtl8xxxu_write16/32()
+ */
+ if (len > sizeof(u32)) {
+ rtl8xxxu_write16(priv, mbox_ext_reg, le16_to_cpu(h2c->raw.ext));
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_H2C)
+ dev_info(dev, "H2C_EXT %04x\n",
+ le16_to_cpu(h2c->raw.ext));
+ }
+ rtl8xxxu_write32(priv, mbox_reg, le32_to_cpu(h2c->raw.data));
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_H2C)
+ dev_info(dev, "H2C %08x\n", le32_to_cpu(h2c->raw.data));
+
+ priv->next_mbox = (mbox_nr + 1) % H2C_MAX_MBOX;
+
+error:
+ mutex_unlock(&priv->h2c_mutex);
+ return retval;
+}
+
+int
+rtl8xxxu_gen2_h2c_cmd(struct rtl8xxxu_priv *priv, struct h2c_cmd *h2c, int len)
+{
+ struct device *dev = &priv->udev->dev;
+ int mbox_nr, retry, retval = 0;
+ int mbox_reg, mbox_ext_reg;
+ u8 val8;
+
+ mutex_lock(&priv->h2c_mutex);
+
+ mbox_nr = priv->next_mbox;
+ mbox_reg = REG_HMBOX_0 + (mbox_nr * 4);
+ mbox_ext_reg = REG_HMBOX_EXT0_8723B + (mbox_nr * 4);
+
+ /*
+ * MBOX ready?
+ */
+ retry = 100;
+ do {
+ val8 = rtl8xxxu_read8(priv, REG_HMTFR);
+ if (!(val8 & BIT(mbox_nr)))
+ break;
+ } while (retry--);
+
+ if (!retry) {
+ dev_info(dev, "%s: Mailbox busy\n", __func__);
+ retval = -EBUSY;
+ goto error;
+ }
+
+ /*
+ * Need to swap as it's being swapped again by rtl8xxxu_write16/32()
+ */
+ if (len > sizeof(u32)) {
+ rtl8xxxu_write32(priv, mbox_ext_reg,
+ le32_to_cpu(h2c->raw_wide.ext));
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_H2C)
+ dev_info(dev, "H2C_EXT %08x\n",
+ le32_to_cpu(h2c->raw_wide.ext));
+ }
+ rtl8xxxu_write32(priv, mbox_reg, le32_to_cpu(h2c->raw.data));
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_H2C)
+ dev_info(dev, "H2C %08x\n", le32_to_cpu(h2c->raw.data));
+
+ priv->next_mbox = (mbox_nr + 1) % H2C_MAX_MBOX;
+
+error:
+ mutex_unlock(&priv->h2c_mutex);
+ return retval;
+}
+
+void rtl8xxxu_gen1_enable_rf(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+ u32 val32;
+
+ val8 = rtl8xxxu_read8(priv, REG_SPS0_CTRL);
+ val8 |= BIT(0) | BIT(3);
+ rtl8xxxu_write8(priv, REG_SPS0_CTRL, val8);
+
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_XAB_RF_PARM);
+ val32 &= ~(BIT(4) | BIT(5));
+ val32 |= BIT(3);
+ if (priv->rf_paths == 2) {
+ val32 &= ~(BIT(20) | BIT(21));
+ val32 |= BIT(19);
+ }
+ rtl8xxxu_write32(priv, REG_FPGA0_XAB_RF_PARM, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_TRX_PATH_ENABLE);
+ val32 &= ~OFDM_RF_PATH_TX_MASK;
+ if (priv->tx_paths == 2)
+ val32 |= OFDM_RF_PATH_TX_A | OFDM_RF_PATH_TX_B;
+ else if (priv->rtl_chip == RTL8192C || priv->rtl_chip == RTL8191C)
+ val32 |= OFDM_RF_PATH_TX_B;
+ else
+ val32 |= OFDM_RF_PATH_TX_A;
+ rtl8xxxu_write32(priv, REG_OFDM0_TRX_PATH_ENABLE, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_RF_MODE);
+ val32 &= ~FPGA_RF_MODE_JAPAN;
+ rtl8xxxu_write32(priv, REG_FPGA0_RF_MODE, val32);
+
+ if (priv->rf_paths == 2)
+ rtl8xxxu_write32(priv, REG_RX_WAIT_CCA, 0x63db25a0);
+ else
+ rtl8xxxu_write32(priv, REG_RX_WAIT_CCA, 0x631b25a0);
+
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_AC, 0x32d95);
+ if (priv->rf_paths == 2)
+ rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_AC, 0x32d95);
+
+ rtl8xxxu_write8(priv, REG_TXPAUSE, 0x00);
+}
+
+void rtl8xxxu_gen1_disable_rf(struct rtl8xxxu_priv *priv)
+{
+ u8 sps0;
+ u32 val32;
+
+ sps0 = rtl8xxxu_read8(priv, REG_SPS0_CTRL);
+
+ /* RF RX code for preamble power saving */
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_XAB_RF_PARM);
+ val32 &= ~(BIT(3) | BIT(4) | BIT(5));
+ if (priv->rf_paths == 2)
+ val32 &= ~(BIT(19) | BIT(20) | BIT(21));
+ rtl8xxxu_write32(priv, REG_FPGA0_XAB_RF_PARM, val32);
+
+ /* Disable TX for four paths */
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_TRX_PATH_ENABLE);
+ val32 &= ~OFDM_RF_PATH_TX_MASK;
+ rtl8xxxu_write32(priv, REG_OFDM0_TRX_PATH_ENABLE, val32);
+
+ /* Enable power saving */
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_RF_MODE);
+ val32 |= FPGA_RF_MODE_JAPAN;
+ rtl8xxxu_write32(priv, REG_FPGA0_RF_MODE, val32);
+
+ /* AFE control register to power down bits [30:22] */
+ if (priv->rf_paths == 2)
+ rtl8xxxu_write32(priv, REG_RX_WAIT_CCA, 0x00db25a0);
+ else
+ rtl8xxxu_write32(priv, REG_RX_WAIT_CCA, 0x001b25a0);
+
+ /* Power down RF module */
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_AC, 0);
+ if (priv->rf_paths == 2)
+ rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_AC, 0);
+
+ sps0 &= ~(BIT(0) | BIT(3));
+ rtl8xxxu_write8(priv, REG_SPS0_CTRL, sps0);
+}
+
+static void rtl8xxxu_stop_tx_beacon(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+
+ val8 = rtl8xxxu_read8(priv, REG_FWHW_TXQ_CTRL + 2);
+ val8 &= ~BIT(6);
+ rtl8xxxu_write8(priv, REG_FWHW_TXQ_CTRL + 2, val8);
+
+ rtl8xxxu_write8(priv, REG_TBTT_PROHIBIT + 1, 0x64);
+ val8 = rtl8xxxu_read8(priv, REG_TBTT_PROHIBIT + 2);
+ val8 &= ~BIT(0);
+ rtl8xxxu_write8(priv, REG_TBTT_PROHIBIT + 2, val8);
+}
+
+
+/*
+ * The rtl8723a has 3 channel groups for it's efuse settings. It only
+ * supports the 2.4GHz band, so channels 1 - 14:
+ * group 0: channels 1 - 3
+ * group 1: channels 4 - 9
+ * group 2: channels 10 - 14
+ *
+ * Note: We index from 0 in the code
+ */
+static int rtl8xxxu_gen1_channel_to_group(int channel)
+{
+ int group;
+
+ if (channel < 4)
+ group = 0;
+ else if (channel < 10)
+ group = 1;
+ else
+ group = 2;
+
+ return group;
+}
+
+/*
+ * Valid for rtl8723bu and rtl8192eu
+ */
+int rtl8xxxu_gen2_channel_to_group(int channel)
+{
+ int group;
+
+ if (channel < 3)
+ group = 0;
+ else if (channel < 6)
+ group = 1;
+ else if (channel < 9)
+ group = 2;
+ else if (channel < 12)
+ group = 3;
+ else
+ group = 4;
+
+ return group;
+}
+
+void rtl8xxxu_gen1_config_channel(struct ieee80211_hw *hw)
+{
+ struct rtl8xxxu_priv *priv = hw->priv;
+ u32 val32, rsr;
+ u8 val8, opmode;
+ bool ht = true;
+ int sec_ch_above, channel;
+ int i;
+
+ opmode = rtl8xxxu_read8(priv, REG_BW_OPMODE);
+ rsr = rtl8xxxu_read32(priv, REG_RESPONSE_RATE_SET);
+ channel = hw->conf.chandef.chan->hw_value;
+
+ switch (hw->conf.chandef.width) {
+ case NL80211_CHAN_WIDTH_20_NOHT:
+ ht = false;
+ case NL80211_CHAN_WIDTH_20:
+ opmode |= BW_OPMODE_20MHZ;
+ rtl8xxxu_write8(priv, REG_BW_OPMODE, opmode);
+
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_RF_MODE);
+ val32 &= ~FPGA_RF_MODE;
+ rtl8xxxu_write32(priv, REG_FPGA0_RF_MODE, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_FPGA1_RF_MODE);
+ val32 &= ~FPGA_RF_MODE;
+ rtl8xxxu_write32(priv, REG_FPGA1_RF_MODE, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_ANALOG2);
+ val32 |= FPGA0_ANALOG2_20MHZ;
+ rtl8xxxu_write32(priv, REG_FPGA0_ANALOG2, val32);
+ break;
+ case NL80211_CHAN_WIDTH_40:
+ if (hw->conf.chandef.center_freq1 >
+ hw->conf.chandef.chan->center_freq) {
+ sec_ch_above = 1;
+ channel += 2;
+ } else {
+ sec_ch_above = 0;
+ channel -= 2;
+ }
+
+ opmode &= ~BW_OPMODE_20MHZ;
+ rtl8xxxu_write8(priv, REG_BW_OPMODE, opmode);
+ rsr &= ~RSR_RSC_BANDWIDTH_40M;
+ if (sec_ch_above)
+ rsr |= RSR_RSC_UPPER_SUB_CHANNEL;
+ else
+ rsr |= RSR_RSC_LOWER_SUB_CHANNEL;
+ rtl8xxxu_write32(priv, REG_RESPONSE_RATE_SET, rsr);
+
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_RF_MODE);
+ val32 |= FPGA_RF_MODE;
+ rtl8xxxu_write32(priv, REG_FPGA0_RF_MODE, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_FPGA1_RF_MODE);
+ val32 |= FPGA_RF_MODE;
+ rtl8xxxu_write32(priv, REG_FPGA1_RF_MODE, val32);
+
+ /*
+ * Set Control channel to upper or lower. These settings
+ * are required only for 40MHz
+ */
+ val32 = rtl8xxxu_read32(priv, REG_CCK0_SYSTEM);
+ val32 &= ~CCK0_SIDEBAND;
+ if (!sec_ch_above)
+ val32 |= CCK0_SIDEBAND;
+ rtl8xxxu_write32(priv, REG_CCK0_SYSTEM, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM1_LSTF);
+ val32 &= ~OFDM_LSTF_PRIME_CH_MASK; /* 0xc00 */
+ if (sec_ch_above)
+ val32 |= OFDM_LSTF_PRIME_CH_LOW;
+ else
+ val32 |= OFDM_LSTF_PRIME_CH_HIGH;
+ rtl8xxxu_write32(priv, REG_OFDM1_LSTF, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_ANALOG2);
+ val32 &= ~FPGA0_ANALOG2_20MHZ;
+ rtl8xxxu_write32(priv, REG_FPGA0_ANALOG2, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_POWER_SAVE);
+ val32 &= ~(FPGA0_PS_LOWER_CHANNEL | FPGA0_PS_UPPER_CHANNEL);
+ if (sec_ch_above)
+ val32 |= FPGA0_PS_UPPER_CHANNEL;
+ else
+ val32 |= FPGA0_PS_LOWER_CHANNEL;
+ rtl8xxxu_write32(priv, REG_FPGA0_POWER_SAVE, val32);
+ break;
+
+ default:
+ break;
+ }
+
+ for (i = RF_A; i < priv->rf_paths; i++) {
+ val32 = rtl8xxxu_read_rfreg(priv, i, RF6052_REG_MODE_AG);
+ val32 &= ~MODE_AG_CHANNEL_MASK;
+ val32 |= channel;
+ rtl8xxxu_write_rfreg(priv, i, RF6052_REG_MODE_AG, val32);
+ }
+
+ if (ht)
+ val8 = 0x0e;
+ else
+ val8 = 0x0a;
+
+ rtl8xxxu_write8(priv, REG_SIFS_CCK + 1, val8);
+ rtl8xxxu_write8(priv, REG_SIFS_OFDM + 1, val8);
+
+ rtl8xxxu_write16(priv, REG_R2T_SIFS, 0x0808);
+ rtl8xxxu_write16(priv, REG_T2T_SIFS, 0x0a0a);
+
+ for (i = RF_A; i < priv->rf_paths; i++) {
+ val32 = rtl8xxxu_read_rfreg(priv, i, RF6052_REG_MODE_AG);
+ if (hw->conf.chandef.width == NL80211_CHAN_WIDTH_40)
+ val32 &= ~MODE_AG_CHANNEL_20MHZ;
+ else
+ val32 |= MODE_AG_CHANNEL_20MHZ;
+ rtl8xxxu_write_rfreg(priv, i, RF6052_REG_MODE_AG, val32);
+ }
+}
+
+void rtl8xxxu_gen2_config_channel(struct ieee80211_hw *hw)
+{
+ struct rtl8xxxu_priv *priv = hw->priv;
+ u32 val32, rsr;
+ u8 val8, subchannel;
+ u16 rf_mode_bw;
+ bool ht = true;
+ int sec_ch_above, channel;
+ int i;
+
+ rf_mode_bw = rtl8xxxu_read16(priv, REG_WMAC_TRXPTCL_CTL);
+ rf_mode_bw &= ~WMAC_TRXPTCL_CTL_BW_MASK;
+ rsr = rtl8xxxu_read32(priv, REG_RESPONSE_RATE_SET);
+ channel = hw->conf.chandef.chan->hw_value;
+
+/* Hack */
+ subchannel = 0;
+
+ switch (hw->conf.chandef.width) {
+ case NL80211_CHAN_WIDTH_20_NOHT:
+ ht = false;
+ case NL80211_CHAN_WIDTH_20:
+ rf_mode_bw |= WMAC_TRXPTCL_CTL_BW_20;
+ subchannel = 0;
+
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_RF_MODE);
+ val32 &= ~FPGA_RF_MODE;
+ rtl8xxxu_write32(priv, REG_FPGA0_RF_MODE, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_FPGA1_RF_MODE);
+ val32 &= ~FPGA_RF_MODE;
+ rtl8xxxu_write32(priv, REG_FPGA1_RF_MODE, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_TX_PSDO_NOISE_WEIGHT);
+ val32 &= ~(BIT(30) | BIT(31));
+ rtl8xxxu_write32(priv, REG_OFDM0_TX_PSDO_NOISE_WEIGHT, val32);
+
+ break;
+ case NL80211_CHAN_WIDTH_40:
+ rf_mode_bw |= WMAC_TRXPTCL_CTL_BW_40;
+
+ if (hw->conf.chandef.center_freq1 >
+ hw->conf.chandef.chan->center_freq) {
+ sec_ch_above = 1;
+ channel += 2;
+ } else {
+ sec_ch_above = 0;
+ channel -= 2;
+ }
+
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_RF_MODE);
+ val32 |= FPGA_RF_MODE;
+ rtl8xxxu_write32(priv, REG_FPGA0_RF_MODE, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_FPGA1_RF_MODE);
+ val32 |= FPGA_RF_MODE;
+ rtl8xxxu_write32(priv, REG_FPGA1_RF_MODE, val32);
+
+ /*
+ * Set Control channel to upper or lower. These settings
+ * are required only for 40MHz
+ */
+ val32 = rtl8xxxu_read32(priv, REG_CCK0_SYSTEM);
+ val32 &= ~CCK0_SIDEBAND;
+ if (!sec_ch_above)
+ val32 |= CCK0_SIDEBAND;
+ rtl8xxxu_write32(priv, REG_CCK0_SYSTEM, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM1_LSTF);
+ val32 &= ~OFDM_LSTF_PRIME_CH_MASK; /* 0xc00 */
+ if (sec_ch_above)
+ val32 |= OFDM_LSTF_PRIME_CH_LOW;
+ else
+ val32 |= OFDM_LSTF_PRIME_CH_HIGH;
+ rtl8xxxu_write32(priv, REG_OFDM1_LSTF, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_POWER_SAVE);
+ val32 &= ~(FPGA0_PS_LOWER_CHANNEL | FPGA0_PS_UPPER_CHANNEL);
+ if (sec_ch_above)
+ val32 |= FPGA0_PS_UPPER_CHANNEL;
+ else
+ val32 |= FPGA0_PS_LOWER_CHANNEL;
+ rtl8xxxu_write32(priv, REG_FPGA0_POWER_SAVE, val32);
+ break;
+ case NL80211_CHAN_WIDTH_80:
+ rf_mode_bw |= WMAC_TRXPTCL_CTL_BW_80;
+ break;
+ default:
+ break;
+ }
+
+ for (i = RF_A; i < priv->rf_paths; i++) {
+ val32 = rtl8xxxu_read_rfreg(priv, i, RF6052_REG_MODE_AG);
+ val32 &= ~MODE_AG_CHANNEL_MASK;
+ val32 |= channel;
+ rtl8xxxu_write_rfreg(priv, i, RF6052_REG_MODE_AG, val32);
+ }
+
+ rtl8xxxu_write16(priv, REG_WMAC_TRXPTCL_CTL, rf_mode_bw);
+ rtl8xxxu_write8(priv, REG_DATA_SUBCHANNEL, subchannel);
+
+ if (ht)
+ val8 = 0x0e;
+ else
+ val8 = 0x0a;
+
+ rtl8xxxu_write8(priv, REG_SIFS_CCK + 1, val8);
+ rtl8xxxu_write8(priv, REG_SIFS_OFDM + 1, val8);
+
+ rtl8xxxu_write16(priv, REG_R2T_SIFS, 0x0808);
+ rtl8xxxu_write16(priv, REG_T2T_SIFS, 0x0a0a);
+
+ for (i = RF_A; i < priv->rf_paths; i++) {
+ val32 = rtl8xxxu_read_rfreg(priv, i, RF6052_REG_MODE_AG);
+ val32 &= ~MODE_AG_BW_MASK;
+ switch(hw->conf.chandef.width) {
+ case NL80211_CHAN_WIDTH_80:
+ val32 |= MODE_AG_BW_80MHZ_8723B;
+ break;
+ case NL80211_CHAN_WIDTH_40:
+ val32 |= MODE_AG_BW_40MHZ_8723B;
+ break;
+ default:
+ val32 |= MODE_AG_BW_20MHZ_8723B;
+ break;
+ }
+ rtl8xxxu_write_rfreg(priv, i, RF6052_REG_MODE_AG, val32);
+ }
+}
+
+void
+rtl8xxxu_gen1_set_tx_power(struct rtl8xxxu_priv *priv, int channel, bool ht40)
+{
+ struct rtl8xxxu_power_base *power_base = priv->power_base;
+ u8 cck[RTL8723A_MAX_RF_PATHS], ofdm[RTL8723A_MAX_RF_PATHS];
+ u8 ofdmbase[RTL8723A_MAX_RF_PATHS], mcsbase[RTL8723A_MAX_RF_PATHS];
+ u32 val32, ofdm_a, ofdm_b, mcs_a, mcs_b;
+ u8 val8;
+ int group, i;
+
+ group = rtl8xxxu_gen1_channel_to_group(channel);
+
+ cck[0] = priv->cck_tx_power_index_A[group] - 1;
+ cck[1] = priv->cck_tx_power_index_B[group] - 1;
+
+ if (priv->hi_pa) {
+ if (cck[0] > 0x20)
+ cck[0] = 0x20;
+ if (cck[1] > 0x20)
+ cck[1] = 0x20;
+ }
+
+ ofdm[0] = priv->ht40_1s_tx_power_index_A[group];
+ ofdm[1] = priv->ht40_1s_tx_power_index_B[group];
+ if (ofdm[0])
+ ofdm[0] -= 1;
+ if (ofdm[1])
+ ofdm[1] -= 1;
+
+ ofdmbase[0] = ofdm[0] + priv->ofdm_tx_power_index_diff[group].a;
+ ofdmbase[1] = ofdm[1] + priv->ofdm_tx_power_index_diff[group].b;
+
+ mcsbase[0] = ofdm[0];
+ mcsbase[1] = ofdm[1];
+ if (!ht40) {
+ mcsbase[0] += priv->ht20_tx_power_index_diff[group].a;
+ mcsbase[1] += priv->ht20_tx_power_index_diff[group].b;
+ }
+
+ if (priv->tx_paths > 1) {
+ if (ofdm[0] > priv->ht40_2s_tx_power_index_diff[group].a)
+ ofdm[0] -= priv->ht40_2s_tx_power_index_diff[group].a;
+ if (ofdm[1] > priv->ht40_2s_tx_power_index_diff[group].b)
+ ofdm[1] -= priv->ht40_2s_tx_power_index_diff[group].b;
+ }
+
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_CHANNEL)
+ dev_info(&priv->udev->dev,
+ "%s: Setting TX power CCK A: %02x, "
+ "CCK B: %02x, OFDM A: %02x, OFDM B: %02x\n",
+ __func__, cck[0], cck[1], ofdm[0], ofdm[1]);
+
+ for (i = 0; i < RTL8723A_MAX_RF_PATHS; i++) {
+ if (cck[i] > RF6052_MAX_TX_PWR)
+ cck[i] = RF6052_MAX_TX_PWR;
+ if (ofdm[i] > RF6052_MAX_TX_PWR)
+ ofdm[i] = RF6052_MAX_TX_PWR;
+ }
+
+ val32 = rtl8xxxu_read32(priv, REG_TX_AGC_A_CCK1_MCS32);
+ val32 &= 0xffff00ff;
+ val32 |= (cck[0] << 8);
+ rtl8xxxu_write32(priv, REG_TX_AGC_A_CCK1_MCS32, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_TX_AGC_B_CCK11_A_CCK2_11);
+ val32 &= 0xff;
+ val32 |= ((cck[0] << 8) | (cck[0] << 16) | (cck[0] << 24));
+ rtl8xxxu_write32(priv, REG_TX_AGC_B_CCK11_A_CCK2_11, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_TX_AGC_B_CCK11_A_CCK2_11);
+ val32 &= 0xffffff00;
+ val32 |= cck[1];
+ rtl8xxxu_write32(priv, REG_TX_AGC_B_CCK11_A_CCK2_11, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_TX_AGC_B_CCK1_55_MCS32);
+ val32 &= 0xff;
+ val32 |= ((cck[1] << 8) | (cck[1] << 16) | (cck[1] << 24));
+ rtl8xxxu_write32(priv, REG_TX_AGC_B_CCK1_55_MCS32, val32);
+
+ ofdm_a = ofdmbase[0] | ofdmbase[0] << 8 |
+ ofdmbase[0] << 16 | ofdmbase[0] << 24;
+ ofdm_b = ofdmbase[1] | ofdmbase[1] << 8 |
+ ofdmbase[1] << 16 | ofdmbase[1] << 24;
+
+ rtl8xxxu_write32(priv, REG_TX_AGC_A_RATE18_06,
+ ofdm_a + power_base->reg_0e00);
+ rtl8xxxu_write32(priv, REG_TX_AGC_B_RATE18_06,
+ ofdm_b + power_base->reg_0830);
+
+ rtl8xxxu_write32(priv, REG_TX_AGC_A_RATE54_24,
+ ofdm_a + power_base->reg_0e04);
+ rtl8xxxu_write32(priv, REG_TX_AGC_B_RATE54_24,
+ ofdm_b + power_base->reg_0834);
+
+ mcs_a = mcsbase[0] | mcsbase[0] << 8 |
+ mcsbase[0] << 16 | mcsbase[0] << 24;
+ mcs_b = mcsbase[1] | mcsbase[1] << 8 |
+ mcsbase[1] << 16 | mcsbase[1] << 24;
+
+ rtl8xxxu_write32(priv, REG_TX_AGC_A_MCS03_MCS00,
+ mcs_a + power_base->reg_0e10);
+ rtl8xxxu_write32(priv, REG_TX_AGC_B_MCS03_MCS00,
+ mcs_b + power_base->reg_083c);
+
+ rtl8xxxu_write32(priv, REG_TX_AGC_A_MCS07_MCS04,
+ mcs_a + power_base->reg_0e14);
+ rtl8xxxu_write32(priv, REG_TX_AGC_B_MCS07_MCS04,
+ mcs_b + power_base->reg_0848);
+
+ rtl8xxxu_write32(priv, REG_TX_AGC_A_MCS11_MCS08,
+ mcs_a + power_base->reg_0e18);
+ rtl8xxxu_write32(priv, REG_TX_AGC_B_MCS11_MCS08,
+ mcs_b + power_base->reg_084c);
+
+ rtl8xxxu_write32(priv, REG_TX_AGC_A_MCS15_MCS12,
+ mcs_a + power_base->reg_0e1c);
+ for (i = 0; i < 3; i++) {
+ if (i != 2)
+ val8 = (mcsbase[0] > 8) ? (mcsbase[0] - 8) : 0;
+ else
+ val8 = (mcsbase[0] > 6) ? (mcsbase[0] - 6) : 0;
+ rtl8xxxu_write8(priv, REG_OFDM0_XC_TX_IQ_IMBALANCE + i, val8);
+ }
+ rtl8xxxu_write32(priv, REG_TX_AGC_B_MCS15_MCS12,
+ mcs_b + power_base->reg_0868);
+ for (i = 0; i < 3; i++) {
+ if (i != 2)
+ val8 = (mcsbase[1] > 8) ? (mcsbase[1] - 8) : 0;
+ else
+ val8 = (mcsbase[1] > 6) ? (mcsbase[1] - 6) : 0;
+ rtl8xxxu_write8(priv, REG_OFDM0_XD_TX_IQ_IMBALANCE + i, val8);
+ }
+}
+
+static void rtl8xxxu_set_linktype(struct rtl8xxxu_priv *priv,
+ enum nl80211_iftype linktype)
+{
+ u8 val8;
+
+ val8 = rtl8xxxu_read8(priv, REG_MSR);
+ val8 &= ~MSR_LINKTYPE_MASK;
+
+ switch (linktype) {
+ case NL80211_IFTYPE_UNSPECIFIED:
+ val8 |= MSR_LINKTYPE_NONE;
+ break;
+ case NL80211_IFTYPE_ADHOC:
+ val8 |= MSR_LINKTYPE_ADHOC;
+ break;
+ case NL80211_IFTYPE_STATION:
+ val8 |= MSR_LINKTYPE_STATION;
+ break;
+ case NL80211_IFTYPE_AP:
+ val8 |= MSR_LINKTYPE_AP;
+ break;
+ default:
+ goto out;
+ }
+
+ rtl8xxxu_write8(priv, REG_MSR, val8);
+out:
+ return;
+}
+
+static void
+rtl8xxxu_set_retry(struct rtl8xxxu_priv *priv, u16 short_retry, u16 long_retry)
+{
+ u16 val16;
+
+ val16 = ((short_retry << RETRY_LIMIT_SHORT_SHIFT) &
+ RETRY_LIMIT_SHORT_MASK) |
+ ((long_retry << RETRY_LIMIT_LONG_SHIFT) &
+ RETRY_LIMIT_LONG_MASK);
+
+ rtl8xxxu_write16(priv, REG_RETRY_LIMIT, val16);
+}
+
+static void
+rtl8xxxu_set_spec_sifs(struct rtl8xxxu_priv *priv, u16 cck, u16 ofdm)
+{
+ u16 val16;
+
+ val16 = ((cck << SPEC_SIFS_CCK_SHIFT) & SPEC_SIFS_CCK_MASK) |
+ ((ofdm << SPEC_SIFS_OFDM_SHIFT) & SPEC_SIFS_OFDM_MASK);
+
+ rtl8xxxu_write16(priv, REG_SPEC_SIFS, val16);
+}
+
+static void rtl8xxxu_print_chipinfo(struct rtl8xxxu_priv *priv)
+{
+ struct device *dev = &priv->udev->dev;
+ char *cut;
+
+ switch (priv->chip_cut) {
+ case 0:
+ cut = "A";
+ break;
+ case 1:
+ cut = "B";
+ break;
+ case 2:
+ cut = "C";
+ break;
+ case 3:
+ cut = "D";
+ break;
+ case 4:
+ cut = "E";
+ break;
+ default:
+ cut = "unknown";
+ }
+
+ dev_info(dev,
+ "RTL%s rev %s (%s) %iT%iR, TX queues %i, WiFi=%i, BT=%i, GPS=%i, HI PA=%i\n",
+ priv->chip_name, cut, priv->chip_vendor, priv->tx_paths,
+ priv->rx_paths, priv->ep_tx_count, priv->has_wifi,
+ priv->has_bluetooth, priv->has_gps, priv->hi_pa);
+
+ dev_info(dev, "RTL%s MAC: %pM\n", priv->chip_name, priv->mac_addr);
+}
+
+static int rtl8xxxu_identify_chip(struct rtl8xxxu_priv *priv)
+{
+ struct device *dev = &priv->udev->dev;
+ u32 val32, bonding;
+ u16 val16;
+
+ val32 = rtl8xxxu_read32(priv, REG_SYS_CFG);
+ priv->chip_cut = (val32 & SYS_CFG_CHIP_VERSION_MASK) >>
+ SYS_CFG_CHIP_VERSION_SHIFT;
+ if (val32 & SYS_CFG_TRP_VAUX_EN) {
+ dev_info(dev, "Unsupported test chip\n");
+ return -ENOTSUPP;
+ }
+
+ if (val32 & SYS_CFG_BT_FUNC) {
+ if (priv->chip_cut >= 3) {
+ sprintf(priv->chip_name, "8723BU");
+ priv->rtl_chip = RTL8723B;
+ } else {
+ sprintf(priv->chip_name, "8723AU");
+ priv->usb_interrupts = 1;
+ priv->rtl_chip = RTL8723A;
+ }
+
+ priv->rf_paths = 1;
+ priv->rx_paths = 1;
+ priv->tx_paths = 1;
+
+ val32 = rtl8xxxu_read32(priv, REG_MULTI_FUNC_CTRL);
+ if (val32 & MULTI_WIFI_FUNC_EN)
+ priv->has_wifi = 1;
+ if (val32 & MULTI_BT_FUNC_EN)
+ priv->has_bluetooth = 1;
+ if (val32 & MULTI_GPS_FUNC_EN)
+ priv->has_gps = 1;
+ priv->is_multi_func = 1;
+ } else if (val32 & SYS_CFG_TYPE_ID) {
+ bonding = rtl8xxxu_read32(priv, REG_HPON_FSM);
+ bonding &= HPON_FSM_BONDING_MASK;
+ if (priv->fops->tx_desc_size ==
+ sizeof(struct rtl8xxxu_txdesc40)) {
+ if (bonding == HPON_FSM_BONDING_1T2R) {
+ sprintf(priv->chip_name, "8191EU");
+ priv->rf_paths = 2;
+ priv->rx_paths = 2;
+ priv->tx_paths = 1;
+ priv->rtl_chip = RTL8191E;
+ } else {
+ sprintf(priv->chip_name, "8192EU");
+ priv->rf_paths = 2;
+ priv->rx_paths = 2;
+ priv->tx_paths = 2;
+ priv->rtl_chip = RTL8192E;
+ }
+ } else if (bonding == HPON_FSM_BONDING_1T2R) {
+ sprintf(priv->chip_name, "8191CU");
+ priv->rf_paths = 2;
+ priv->rx_paths = 2;
+ priv->tx_paths = 1;
+ priv->usb_interrupts = 1;
+ priv->rtl_chip = RTL8191C;
+ } else {
+ sprintf(priv->chip_name, "8192CU");
+ priv->rf_paths = 2;
+ priv->rx_paths = 2;
+ priv->tx_paths = 2;
+ priv->usb_interrupts = 1;
+ priv->rtl_chip = RTL8192C;
+ }
+ priv->has_wifi = 1;
+ } else {
+ sprintf(priv->chip_name, "8188CU");
+ priv->rf_paths = 1;
+ priv->rx_paths = 1;
+ priv->tx_paths = 1;
+ priv->rtl_chip = RTL8188C;
+ priv->usb_interrupts = 1;
+ priv->has_wifi = 1;
+ }
+
+ switch (priv->rtl_chip) {
+ case RTL8188E:
+ case RTL8192E:
+ case RTL8723B:
+ switch (val32 & SYS_CFG_VENDOR_EXT_MASK) {
+ case SYS_CFG_VENDOR_ID_TSMC:
+ sprintf(priv->chip_vendor, "TSMC");
+ break;
+ case SYS_CFG_VENDOR_ID_SMIC:
+ sprintf(priv->chip_vendor, "SMIC");
+ priv->vendor_smic = 1;
+ break;
+ case SYS_CFG_VENDOR_ID_UMC:
+ sprintf(priv->chip_vendor, "UMC");
+ priv->vendor_umc = 1;
+ break;
+ default:
+ sprintf(priv->chip_vendor, "unknown");
+ }
+ break;
+ default:
+ if (val32 & SYS_CFG_VENDOR_ID) {
+ sprintf(priv->chip_vendor, "UMC");
+ priv->vendor_umc = 1;
+ } else {
+ sprintf(priv->chip_vendor, "TSMC");
+ }
+ }
+
+ val32 = rtl8xxxu_read32(priv, REG_GPIO_OUTSTS);
+ priv->rom_rev = (val32 & GPIO_RF_RL_ID) >> 28;
+
+ val16 = rtl8xxxu_read16(priv, REG_NORMAL_SIE_EP_TX);
+ if (val16 & NORMAL_SIE_EP_TX_HIGH_MASK) {
+ priv->ep_tx_high_queue = 1;
+ priv->ep_tx_count++;
+ }
+
+ if (val16 & NORMAL_SIE_EP_TX_NORMAL_MASK) {
+ priv->ep_tx_normal_queue = 1;
+ priv->ep_tx_count++;
+ }
+
+ if (val16 & NORMAL_SIE_EP_TX_LOW_MASK) {
+ priv->ep_tx_low_queue = 1;
+ priv->ep_tx_count++;
+ }
+
+ /*
+ * Fallback for devices that do not provide REG_NORMAL_SIE_EP_TX
+ */
+ if (!priv->ep_tx_count) {
+ switch (priv->nr_out_eps) {
+ case 4:
+ case 3:
+ priv->ep_tx_low_queue = 1;
+ priv->ep_tx_count++;
+ case 2:
+ priv->ep_tx_normal_queue = 1;
+ priv->ep_tx_count++;
+ case 1:
+ priv->ep_tx_high_queue = 1;
+ priv->ep_tx_count++;
+ break;
+ default:
+ dev_info(dev, "Unsupported USB TX end-points\n");
+ return -ENOTSUPP;
+ }
+ }
+
+ return 0;
+}
+
+static int
+rtl8xxxu_read_efuse8(struct rtl8xxxu_priv *priv, u16 offset, u8 *data)
+{
+ int i;
+ u8 val8;
+ u32 val32;
+
+ /* Write Address */
+ rtl8xxxu_write8(priv, REG_EFUSE_CTRL + 1, offset & 0xff);
+ val8 = rtl8xxxu_read8(priv, REG_EFUSE_CTRL + 2);
+ val8 &= 0xfc;
+ val8 |= (offset >> 8) & 0x03;
+ rtl8xxxu_write8(priv, REG_EFUSE_CTRL + 2, val8);
+
+ val8 = rtl8xxxu_read8(priv, REG_EFUSE_CTRL + 3);
+ rtl8xxxu_write8(priv, REG_EFUSE_CTRL + 3, val8 & 0x7f);
+
+ /* Poll for data read */
+ val32 = rtl8xxxu_read32(priv, REG_EFUSE_CTRL);
+ for (i = 0; i < RTL8XXXU_MAX_REG_POLL; i++) {
+ val32 = rtl8xxxu_read32(priv, REG_EFUSE_CTRL);
+ if (val32 & BIT(31))
+ break;
+ }
+
+ if (i == RTL8XXXU_MAX_REG_POLL)
+ return -EIO;
+
+ udelay(50);
+ val32 = rtl8xxxu_read32(priv, REG_EFUSE_CTRL);
+
+ *data = val32 & 0xff;
+ return 0;
+}
+
+static int rtl8xxxu_read_efuse(struct rtl8xxxu_priv *priv)
+{
+ struct device *dev = &priv->udev->dev;
+ int i, ret = 0;
+ u8 val8, word_mask, header, extheader;
+ u16 val16, efuse_addr, offset;
+ u32 val32;
+
+ val16 = rtl8xxxu_read16(priv, REG_9346CR);
+ if (val16 & EEPROM_ENABLE)
+ priv->has_eeprom = 1;
+ if (val16 & EEPROM_BOOT)
+ priv->boot_eeprom = 1;
+
+ if (priv->is_multi_func) {
+ val32 = rtl8xxxu_read32(priv, REG_EFUSE_TEST);
+ val32 = (val32 & ~EFUSE_SELECT_MASK) | EFUSE_WIFI_SELECT;
+ rtl8xxxu_write32(priv, REG_EFUSE_TEST, val32);
+ }
+
+ dev_dbg(dev, "Booting from %s\n",
+ priv->boot_eeprom ? "EEPROM" : "EFUSE");
+
+ rtl8xxxu_write8(priv, REG_EFUSE_ACCESS, EFUSE_ACCESS_ENABLE);
+
+ /* 1.2V Power: From VDDON with Power Cut(0x0000[15]), default valid */
+ val16 = rtl8xxxu_read16(priv, REG_SYS_ISO_CTRL);
+ if (!(val16 & SYS_ISO_PWC_EV12V)) {
+ val16 |= SYS_ISO_PWC_EV12V;
+ rtl8xxxu_write16(priv, REG_SYS_ISO_CTRL, val16);
+ }
+ /* Reset: 0x0000[28], default valid */
+ val16 = rtl8xxxu_read16(priv, REG_SYS_FUNC);
+ if (!(val16 & SYS_FUNC_ELDR)) {
+ val16 |= SYS_FUNC_ELDR;
+ rtl8xxxu_write16(priv, REG_SYS_FUNC, val16);
+ }
+
+ /*
+ * Clock: Gated(0x0008[5]) 8M(0x0008[1]) clock from ANA, default valid
+ */
+ val16 = rtl8xxxu_read16(priv, REG_SYS_CLKR);
+ if (!(val16 & SYS_CLK_LOADER_ENABLE) || !(val16 & SYS_CLK_ANA8M)) {
+ val16 |= (SYS_CLK_LOADER_ENABLE | SYS_CLK_ANA8M);
+ rtl8xxxu_write16(priv, REG_SYS_CLKR, val16);
+ }
+
+ /* Default value is 0xff */
+ memset(priv->efuse_wifi.raw, 0xff, EFUSE_MAP_LEN);
+
+ efuse_addr = 0;
+ while (efuse_addr < EFUSE_REAL_CONTENT_LEN_8723A) {
+ u16 map_addr;
+
+ ret = rtl8xxxu_read_efuse8(priv, efuse_addr++, &header);
+ if (ret || header == 0xff)
+ goto exit;
+
+ if ((header & 0x1f) == 0x0f) { /* extended header */
+ offset = (header & 0xe0) >> 5;
+
+ ret = rtl8xxxu_read_efuse8(priv, efuse_addr++,
+ &extheader);
+ if (ret)
+ goto exit;
+ /* All words disabled */
+ if ((extheader & 0x0f) == 0x0f)
+ continue;
+
+ offset |= ((extheader & 0xf0) >> 1);
+ word_mask = extheader & 0x0f;
+ } else {
+ offset = (header >> 4) & 0x0f;
+ word_mask = header & 0x0f;
+ }
+
+ /* Get word enable value from PG header */
+
+ /* We have 8 bits to indicate validity */
+ map_addr = offset * 8;
+ if (map_addr >= EFUSE_MAP_LEN) {
+ dev_warn(dev, "%s: Illegal map_addr (%04x), "
+ "efuse corrupt!\n",
+ __func__, map_addr);
+ ret = -EINVAL;
+ goto exit;
+ }
+ for (i = 0; i < EFUSE_MAX_WORD_UNIT; i++) {
+ /* Check word enable condition in the section */
+ if (word_mask & BIT(i)) {
+ map_addr += 2;
+ continue;
+ }
+
+ ret = rtl8xxxu_read_efuse8(priv, efuse_addr++, &val8);
+ if (ret)
+ goto exit;
+ priv->efuse_wifi.raw[map_addr++] = val8;
+
+ ret = rtl8xxxu_read_efuse8(priv, efuse_addr++, &val8);
+ if (ret)
+ goto exit;
+ priv->efuse_wifi.raw[map_addr++] = val8;
+ }
+ }
+
+exit:
+ rtl8xxxu_write8(priv, REG_EFUSE_ACCESS, EFUSE_ACCESS_DISABLE);
+
+ return ret;
+}
+
+void rtl8xxxu_reset_8051(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+ u16 sys_func;
+
+ val8 = rtl8xxxu_read8(priv, REG_RSV_CTRL + 1);
+ val8 &= ~BIT(0);
+ rtl8xxxu_write8(priv, REG_RSV_CTRL + 1, val8);
+
+ sys_func = rtl8xxxu_read16(priv, REG_SYS_FUNC);
+ sys_func &= ~SYS_FUNC_CPU_ENABLE;
+ rtl8xxxu_write16(priv, REG_SYS_FUNC, sys_func);
+
+ val8 = rtl8xxxu_read8(priv, REG_RSV_CTRL + 1);
+ val8 |= BIT(0);
+ rtl8xxxu_write8(priv, REG_RSV_CTRL + 1, val8);
+
+ sys_func |= SYS_FUNC_CPU_ENABLE;
+ rtl8xxxu_write16(priv, REG_SYS_FUNC, sys_func);
+}
+
+static int rtl8xxxu_start_firmware(struct rtl8xxxu_priv *priv)
+{
+ struct device *dev = &priv->udev->dev;
+ int ret = 0, i;
+ u32 val32;
+
+ /* Poll checksum report */
+ for (i = 0; i < RTL8XXXU_FIRMWARE_POLL_MAX; i++) {
+ val32 = rtl8xxxu_read32(priv, REG_MCU_FW_DL);
+ if (val32 & MCU_FW_DL_CSUM_REPORT)
+ break;
+ }
+
+ if (i == RTL8XXXU_FIRMWARE_POLL_MAX) {
+ dev_warn(dev, "Firmware checksum poll timed out\n");
+ ret = -EAGAIN;
+ goto exit;
+ }
+
+ val32 = rtl8xxxu_read32(priv, REG_MCU_FW_DL);
+ val32 |= MCU_FW_DL_READY;
+ val32 &= ~MCU_WINT_INIT_READY;
+ rtl8xxxu_write32(priv, REG_MCU_FW_DL, val32);
+
+ /*
+ * Reset the 8051 in order for the firmware to start running,
+ * otherwise it won't come up on the 8192eu
+ */
+ priv->fops->reset_8051(priv);
+
+ /* Wait for firmware to become ready */
+ for (i = 0; i < RTL8XXXU_FIRMWARE_POLL_MAX; i++) {
+ val32 = rtl8xxxu_read32(priv, REG_MCU_FW_DL);
+ if (val32 & MCU_WINT_INIT_READY)
+ break;
+
+ udelay(100);
+ }
+
+ if (i == RTL8XXXU_FIRMWARE_POLL_MAX) {
+ dev_warn(dev, "Firmware failed to start\n");
+ ret = -EAGAIN;
+ goto exit;
+ }
+
+ /*
+ * Init H2C command
+ */
+ if (priv->rtl_chip == RTL8723B)
+ rtl8xxxu_write8(priv, REG_HMTFR, 0x0f);
+exit:
+ return ret;
+}
+
+static int rtl8xxxu_download_firmware(struct rtl8xxxu_priv *priv)
+{
+ int pages, remainder, i, ret;
+ u8 val8;
+ u16 val16;
+ u32 val32;
+ u8 *fwptr;
+
+ val8 = rtl8xxxu_read8(priv, REG_SYS_FUNC + 1);
+ val8 |= 4;
+ rtl8xxxu_write8(priv, REG_SYS_FUNC + 1, val8);
+
+ /* 8051 enable */
+ val16 = rtl8xxxu_read16(priv, REG_SYS_FUNC);
+ val16 |= SYS_FUNC_CPU_ENABLE;
+ rtl8xxxu_write16(priv, REG_SYS_FUNC, val16);
+
+ val8 = rtl8xxxu_read8(priv, REG_MCU_FW_DL);
+ if (val8 & MCU_FW_RAM_SEL) {
+ pr_info("do the RAM reset\n");
+ rtl8xxxu_write8(priv, REG_MCU_FW_DL, 0x00);
+ priv->fops->reset_8051(priv);
+ }
+
+ /* MCU firmware download enable */
+ val8 = rtl8xxxu_read8(priv, REG_MCU_FW_DL);
+ val8 |= MCU_FW_DL_ENABLE;
+ rtl8xxxu_write8(priv, REG_MCU_FW_DL, val8);
+
+ /* 8051 reset */
+ val32 = rtl8xxxu_read32(priv, REG_MCU_FW_DL);
+ val32 &= ~BIT(19);
+ rtl8xxxu_write32(priv, REG_MCU_FW_DL, val32);
+
+ /* Reset firmware download checksum */
+ val8 = rtl8xxxu_read8(priv, REG_MCU_FW_DL);
+ val8 |= MCU_FW_DL_CSUM_REPORT;
+ rtl8xxxu_write8(priv, REG_MCU_FW_DL, val8);
+
+ pages = priv->fw_size / RTL_FW_PAGE_SIZE;
+ remainder = priv->fw_size % RTL_FW_PAGE_SIZE;
+
+ fwptr = priv->fw_data->data;
+
+ for (i = 0; i < pages; i++) {
+ val8 = rtl8xxxu_read8(priv, REG_MCU_FW_DL + 2) & 0xF8;
+ val8 |= i;
+ rtl8xxxu_write8(priv, REG_MCU_FW_DL + 2, val8);
+
+ ret = rtl8xxxu_writeN(priv, REG_FW_START_ADDRESS,
+ fwptr, RTL_FW_PAGE_SIZE);
+ if (ret != RTL_FW_PAGE_SIZE) {
+ ret = -EAGAIN;
+ goto fw_abort;
+ }
+
+ fwptr += RTL_FW_PAGE_SIZE;
+ }
+
+ if (remainder) {
+ val8 = rtl8xxxu_read8(priv, REG_MCU_FW_DL + 2) & 0xF8;
+ val8 |= i;
+ rtl8xxxu_write8(priv, REG_MCU_FW_DL + 2, val8);
+ ret = rtl8xxxu_writeN(priv, REG_FW_START_ADDRESS,
+ fwptr, remainder);
+ if (ret != remainder) {
+ ret = -EAGAIN;
+ goto fw_abort;
+ }
+ }
+
+ ret = 0;
+fw_abort:
+ /* MCU firmware download disable */
+ val16 = rtl8xxxu_read16(priv, REG_MCU_FW_DL);
+ val16 &= ~MCU_FW_DL_ENABLE;
+ rtl8xxxu_write16(priv, REG_MCU_FW_DL, val16);
+
+ return ret;
+}
+
+int rtl8xxxu_load_firmware(struct rtl8xxxu_priv *priv, char *fw_name)
+{
+ struct device *dev = &priv->udev->dev;
+ const struct firmware *fw;
+ int ret = 0;
+ u16 signature;
+
+ dev_info(dev, "%s: Loading firmware %s\n", DRIVER_NAME, fw_name);
+ if (request_firmware(&fw, fw_name, &priv->udev->dev)) {
+ dev_warn(dev, "request_firmware(%s) failed\n", fw_name);
+ ret = -EAGAIN;
+ goto exit;
+ }
+ if (!fw) {
+ dev_warn(dev, "Firmware data not available\n");
+ ret = -EINVAL;
+ goto exit;
+ }
+
+ priv->fw_data = kmemdup(fw->data, fw->size, GFP_KERNEL);
+ if (!priv->fw_data) {
+ ret = -ENOMEM;
+ goto exit;
+ }
+ priv->fw_size = fw->size - sizeof(struct rtl8xxxu_firmware_header);
+
+ signature = le16_to_cpu(priv->fw_data->signature);
+ switch (signature & 0xfff0) {
+ case 0x92e0:
+ case 0x92c0:
+ case 0x88c0:
+ case 0x5300:
+ case 0x2300:
+ break;
+ default:
+ ret = -EINVAL;
+ dev_warn(dev, "%s: Invalid firmware signature: 0x%04x\n",
+ __func__, signature);
+ }
+
+ dev_info(dev, "Firmware revision %i.%i (signature 0x%04x)\n",
+ le16_to_cpu(priv->fw_data->major_version),
+ priv->fw_data->minor_version, signature);
+
+exit:
+ release_firmware(fw);
+ return ret;
+}
+
+void rtl8xxxu_firmware_self_reset(struct rtl8xxxu_priv *priv)
+{
+ u16 val16;
+ int i = 100;
+
+ /* Inform 8051 to perform reset */
+ rtl8xxxu_write8(priv, REG_HMTFR + 3, 0x20);
+
+ for (i = 100; i > 0; i--) {
+ val16 = rtl8xxxu_read16(priv, REG_SYS_FUNC);
+
+ if (!(val16 & SYS_FUNC_CPU_ENABLE)) {
+ dev_dbg(&priv->udev->dev,
+ "%s: Firmware self reset success!\n", __func__);
+ break;
+ }
+ udelay(50);
+ }
+
+ if (!i) {
+ /* Force firmware reset */
+ val16 = rtl8xxxu_read16(priv, REG_SYS_FUNC);
+ val16 &= ~SYS_FUNC_CPU_ENABLE;
+ rtl8xxxu_write16(priv, REG_SYS_FUNC, val16);
+ }
+}
+
+static int
+rtl8xxxu_init_mac(struct rtl8xxxu_priv *priv)
+{
+ struct rtl8xxxu_reg8val *array = priv->fops->mactable;
+ int i, ret;
+ u16 reg;
+ u8 val;
+
+ for (i = 0; ; i++) {
+ reg = array[i].reg;
+ val = array[i].val;
+
+ if (reg == 0xffff && val == 0xff)
+ break;
+
+ ret = rtl8xxxu_write8(priv, reg, val);
+ if (ret != 1) {
+ dev_warn(&priv->udev->dev,
+ "Failed to initialize MAC "
+ "(reg: %04x, val %02x)\n", reg, val);
+ return -EAGAIN;
+ }
+ }
+
+ if (priv->rtl_chip != RTL8723B && priv->rtl_chip != RTL8192E)
+ rtl8xxxu_write8(priv, REG_MAX_AGGR_NUM, 0x0a);
+
+ return 0;
+}
+
+int rtl8xxxu_init_phy_regs(struct rtl8xxxu_priv *priv,
+ struct rtl8xxxu_reg32val *array)
+{
+ int i, ret;
+ u16 reg;
+ u32 val;
+
+ for (i = 0; ; i++) {
+ reg = array[i].reg;
+ val = array[i].val;
+
+ if (reg == 0xffff && val == 0xffffffff)
+ break;
+
+ ret = rtl8xxxu_write32(priv, reg, val);
+ if (ret != sizeof(val)) {
+ dev_warn(&priv->udev->dev,
+ "Failed to initialize PHY\n");
+ return -EAGAIN;
+ }
+ udelay(1);
+ }
+
+ return 0;
+}
+
+void rtl8xxxu_gen1_init_phy_bb(struct rtl8xxxu_priv *priv)
+{
+ u8 val8, ldoa15, ldov12d, lpldo, ldohci12;
+ u16 val16;
+ u32 val32;
+
+ val8 = rtl8xxxu_read8(priv, REG_AFE_PLL_CTRL);
+ udelay(2);
+ val8 |= AFE_PLL_320_ENABLE;
+ rtl8xxxu_write8(priv, REG_AFE_PLL_CTRL, val8);
+ udelay(2);
+
+ rtl8xxxu_write8(priv, REG_AFE_PLL_CTRL + 1, 0xff);
+ udelay(2);
+
+ val16 = rtl8xxxu_read16(priv, REG_SYS_FUNC);
+ val16 |= SYS_FUNC_BB_GLB_RSTN | SYS_FUNC_BBRSTB;
+ rtl8xxxu_write16(priv, REG_SYS_FUNC, val16);
+
+ val32 = rtl8xxxu_read32(priv, REG_AFE_XTAL_CTRL);
+ val32 &= ~AFE_XTAL_RF_GATE;
+ if (priv->has_bluetooth)
+ val32 &= ~AFE_XTAL_BT_GATE;
+ rtl8xxxu_write32(priv, REG_AFE_XTAL_CTRL, val32);
+
+ /* 6. 0x1f[7:0] = 0x07 */
+ val8 = RF_ENABLE | RF_RSTB | RF_SDMRSTB;
+ rtl8xxxu_write8(priv, REG_RF_CTRL, val8);
+
+ if (priv->hi_pa)
+ rtl8xxxu_init_phy_regs(priv, rtl8188ru_phy_1t_highpa_table);
+ else if (priv->tx_paths == 2)
+ rtl8xxxu_init_phy_regs(priv, rtl8192cu_phy_2t_init_table);
+ else
+ rtl8xxxu_init_phy_regs(priv, rtl8723a_phy_1t_init_table);
+
+ if (priv->rtl_chip == RTL8188R && priv->hi_pa &&
+ priv->vendor_umc && priv->chip_cut == 1)
+ rtl8xxxu_write8(priv, REG_OFDM0_AGC_PARM1 + 2, 0x50);
+
+ if (priv->hi_pa)
+ rtl8xxxu_init_phy_regs(priv, rtl8xxx_agc_highpa_table);
+ else
+ rtl8xxxu_init_phy_regs(priv, rtl8xxx_agc_standard_table);
+
+ ldoa15 = LDOA15_ENABLE | LDOA15_OBUF;
+ ldov12d = LDOV12D_ENABLE | BIT(2) | (2 << LDOV12D_VADJ_SHIFT);
+ ldohci12 = 0x57;
+ lpldo = 1;
+ val32 = (lpldo << 24) | (ldohci12 << 16) | (ldov12d << 8) | ldoa15;
+ rtl8xxxu_write32(priv, REG_LDOA15_CTRL, val32);
+}
+
+/*
+ * Most of this is black magic retrieved from the old rtl8723au driver
+ */
+static int rtl8xxxu_init_phy_bb(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+ u32 val32;
+
+ priv->fops->init_phy_bb(priv);
+
+ if (priv->tx_paths == 1 && priv->rx_paths == 2) {
+ /*
+ * For 1T2R boards, patch the registers.
+ *
+ * It looks like 8191/2 1T2R boards use path B for TX
+ */
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_TX_INFO);
+ val32 &= ~(BIT(0) | BIT(1));
+ val32 |= BIT(1);
+ rtl8xxxu_write32(priv, REG_FPGA0_TX_INFO, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_FPGA1_TX_INFO);
+ val32 &= ~0x300033;
+ val32 |= 0x200022;
+ rtl8xxxu_write32(priv, REG_FPGA1_TX_INFO, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_CCK0_AFE_SETTING);
+ val32 &= ~CCK0_AFE_RX_MASK;
+ val32 &= 0x00ffffff;
+ val32 |= 0x40000000;
+ val32 |= CCK0_AFE_RX_ANT_B;
+ rtl8xxxu_write32(priv, REG_CCK0_AFE_SETTING, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_TRX_PATH_ENABLE);
+ val32 &= ~(OFDM_RF_PATH_RX_MASK | OFDM_RF_PATH_TX_MASK);
+ val32 |= (OFDM_RF_PATH_RX_A | OFDM_RF_PATH_RX_B |
+ OFDM_RF_PATH_TX_B);
+ rtl8xxxu_write32(priv, REG_OFDM0_TRX_PATH_ENABLE, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_AGC_PARM1);
+ val32 &= ~(BIT(4) | BIT(5));
+ val32 |= BIT(4);
+ rtl8xxxu_write32(priv, REG_OFDM0_AGC_PARM1, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_TX_CCK_RFON);
+ val32 &= ~(BIT(27) | BIT(26));
+ val32 |= BIT(27);
+ rtl8xxxu_write32(priv, REG_TX_CCK_RFON, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_TX_CCK_BBON);
+ val32 &= ~(BIT(27) | BIT(26));
+ val32 |= BIT(27);
+ rtl8xxxu_write32(priv, REG_TX_CCK_BBON, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_TX_OFDM_RFON);
+ val32 &= ~(BIT(27) | BIT(26));
+ val32 |= BIT(27);
+ rtl8xxxu_write32(priv, REG_TX_OFDM_RFON, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_TX_OFDM_BBON);
+ val32 &= ~(BIT(27) | BIT(26));
+ val32 |= BIT(27);
+ rtl8xxxu_write32(priv, REG_TX_OFDM_BBON, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_TX_TO_TX);
+ val32 &= ~(BIT(27) | BIT(26));
+ val32 |= BIT(27);
+ rtl8xxxu_write32(priv, REG_TX_TO_TX, val32);
+ }
+
+ if (priv->has_xtalk) {
+ val32 = rtl8xxxu_read32(priv, REG_MAC_PHY_CTRL);
+
+ val8 = priv->xtalk;
+ val32 &= 0xff000fff;
+ val32 |= ((val8 | (val8 << 6)) << 12);
+
+ rtl8xxxu_write32(priv, REG_MAC_PHY_CTRL, val32);
+ }
+
+ if (priv->rtl_chip == RTL8192E)
+ rtl8xxxu_write32(priv, REG_AFE_XTAL_CTRL, 0x000f81fb);
+
+ return 0;
+}
+
+static int rtl8xxxu_init_rf_regs(struct rtl8xxxu_priv *priv,
+ struct rtl8xxxu_rfregval *array,
+ enum rtl8xxxu_rfpath path)
+{
+ int i, ret;
+ u8 reg;
+ u32 val;
+
+ for (i = 0; ; i++) {
+ reg = array[i].reg;
+ val = array[i].val;
+
+ if (reg == 0xff && val == 0xffffffff)
+ break;
+
+ switch (reg) {
+ case 0xfe:
+ msleep(50);
+ continue;
+ case 0xfd:
+ mdelay(5);
+ continue;
+ case 0xfc:
+ mdelay(1);
+ continue;
+ case 0xfb:
+ udelay(50);
+ continue;
+ case 0xfa:
+ udelay(5);
+ continue;
+ case 0xf9:
+ udelay(1);
+ continue;
+ }
+
+ ret = rtl8xxxu_write_rfreg(priv, path, reg, val);
+ if (ret) {
+ dev_warn(&priv->udev->dev,
+ "Failed to initialize RF\n");
+ return -EAGAIN;
+ }
+ udelay(1);
+ }
+
+ return 0;
+}
+
+int rtl8xxxu_init_phy_rf(struct rtl8xxxu_priv *priv,
+ struct rtl8xxxu_rfregval *table,
+ enum rtl8xxxu_rfpath path)
+{
+ u32 val32;
+ u16 val16, rfsi_rfenv;
+ u16 reg_sw_ctrl, reg_int_oe, reg_hssi_parm2;
+
+ switch (path) {
+ case RF_A:
+ reg_sw_ctrl = REG_FPGA0_XA_RF_SW_CTRL;
+ reg_int_oe = REG_FPGA0_XA_RF_INT_OE;
+ reg_hssi_parm2 = REG_FPGA0_XA_HSSI_PARM2;
+ break;
+ case RF_B:
+ reg_sw_ctrl = REG_FPGA0_XB_RF_SW_CTRL;
+ reg_int_oe = REG_FPGA0_XB_RF_INT_OE;
+ reg_hssi_parm2 = REG_FPGA0_XB_HSSI_PARM2;
+ break;
+ default:
+ dev_err(&priv->udev->dev, "%s:Unsupported RF path %c\n",
+ __func__, path + 'A');
+ return -EINVAL;
+ }
+ /* For path B, use XB */
+ rfsi_rfenv = rtl8xxxu_read16(priv, reg_sw_ctrl);
+ rfsi_rfenv &= FPGA0_RF_RFENV;
+
+ /*
+ * These two we might be able to optimize into one
+ */
+ val32 = rtl8xxxu_read32(priv, reg_int_oe);
+ val32 |= BIT(20); /* 0x10 << 16 */
+ rtl8xxxu_write32(priv, reg_int_oe, val32);
+ udelay(1);
+
+ val32 = rtl8xxxu_read32(priv, reg_int_oe);
+ val32 |= BIT(4);
+ rtl8xxxu_write32(priv, reg_int_oe, val32);
+ udelay(1);
+
+ /*
+ * These two we might be able to optimize into one
+ */
+ val32 = rtl8xxxu_read32(priv, reg_hssi_parm2);
+ val32 &= ~FPGA0_HSSI_3WIRE_ADDR_LEN;
+ rtl8xxxu_write32(priv, reg_hssi_parm2, val32);
+ udelay(1);
+
+ val32 = rtl8xxxu_read32(priv, reg_hssi_parm2);
+ val32 &= ~FPGA0_HSSI_3WIRE_DATA_LEN;
+ rtl8xxxu_write32(priv, reg_hssi_parm2, val32);
+ udelay(1);
+
+ rtl8xxxu_init_rf_regs(priv, table, path);
+
+ /* For path B, use XB */
+ val16 = rtl8xxxu_read16(priv, reg_sw_ctrl);
+ val16 &= ~FPGA0_RF_RFENV;
+ val16 |= rfsi_rfenv;
+ rtl8xxxu_write16(priv, reg_sw_ctrl, val16);
+
+ return 0;
+}
+
+static int rtl8xxxu_llt_write(struct rtl8xxxu_priv *priv, u8 address, u8 data)
+{
+ int ret = -EBUSY;
+ int count = 0;
+ u32 value;
+
+ value = LLT_OP_WRITE | address << 8 | data;
+
+ rtl8xxxu_write32(priv, REG_LLT_INIT, value);
+
+ do {
+ value = rtl8xxxu_read32(priv, REG_LLT_INIT);
+ if ((value & LLT_OP_MASK) == LLT_OP_INACTIVE) {
+ ret = 0;
+ break;
+ }
+ } while (count++ < 20);
+
+ return ret;
+}
+
+int rtl8xxxu_init_llt_table(struct rtl8xxxu_priv *priv, u8 last_tx_page)
+{
+ int ret;
+ int i;
+
+ for (i = 0; i < last_tx_page; i++) {
+ ret = rtl8xxxu_llt_write(priv, i, i + 1);
+ if (ret)
+ goto exit;
+ }
+
+ ret = rtl8xxxu_llt_write(priv, last_tx_page, 0xff);
+ if (ret)
+ goto exit;
+
+ /* Mark remaining pages as a ring buffer */
+ for (i = last_tx_page + 1; i < 0xff; i++) {
+ ret = rtl8xxxu_llt_write(priv, i, (i + 1));
+ if (ret)
+ goto exit;
+ }
+
+ /* Let last entry point to the start entry of ring buffer */
+ ret = rtl8xxxu_llt_write(priv, 0xff, last_tx_page + 1);
+ if (ret)
+ goto exit;
+
+exit:
+ return ret;
+}
+
+int rtl8xxxu_auto_llt_table(struct rtl8xxxu_priv *priv, u8 last_tx_page)
+{
+ u32 val32;
+ int ret = 0;
+ int i;
+
+ val32 = rtl8xxxu_read32(priv, REG_AUTO_LLT);
+ val32 |= AUTO_LLT_INIT_LLT;
+ rtl8xxxu_write32(priv, REG_AUTO_LLT, val32);
+
+ for (i = 500; i; i--) {
+ val32 = rtl8xxxu_read32(priv, REG_AUTO_LLT);
+ if (!(val32 & AUTO_LLT_INIT_LLT))
+ break;
+ usleep_range(2, 4);
+ }
+
+ if (!i) {
+ ret = -EBUSY;
+ dev_warn(&priv->udev->dev, "LLT table init failed\n");
+ }
+
+ return ret;
+}
+
+static int rtl8xxxu_init_queue_priority(struct rtl8xxxu_priv *priv)
+{
+ u16 val16, hi, lo;
+ u16 hiq, mgq, bkq, beq, viq, voq;
+ int hip, mgp, bkp, bep, vip, vop;
+ int ret = 0;
+
+ switch (priv->ep_tx_count) {
+ case 1:
+ if (priv->ep_tx_high_queue) {
+ hi = TRXDMA_QUEUE_HIGH;
+ } else if (priv->ep_tx_low_queue) {
+ hi = TRXDMA_QUEUE_LOW;
+ } else if (priv->ep_tx_normal_queue) {
+ hi = TRXDMA_QUEUE_NORMAL;
+ } else {
+ hi = 0;
+ ret = -EINVAL;
+ }
+
+ hiq = hi;
+ mgq = hi;
+ bkq = hi;
+ beq = hi;
+ viq = hi;
+ voq = hi;
+
+ hip = 0;
+ mgp = 0;
+ bkp = 0;
+ bep = 0;
+ vip = 0;
+ vop = 0;
+ break;
+ case 2:
+ if (priv->ep_tx_high_queue && priv->ep_tx_low_queue) {
+ hi = TRXDMA_QUEUE_HIGH;
+ lo = TRXDMA_QUEUE_LOW;
+ } else if (priv->ep_tx_normal_queue && priv->ep_tx_low_queue) {
+ hi = TRXDMA_QUEUE_NORMAL;
+ lo = TRXDMA_QUEUE_LOW;
+ } else if (priv->ep_tx_high_queue && priv->ep_tx_normal_queue) {
+ hi = TRXDMA_QUEUE_HIGH;
+ lo = TRXDMA_QUEUE_NORMAL;
+ } else {
+ ret = -EINVAL;
+ hi = 0;
+ lo = 0;
+ }
+
+ hiq = hi;
+ mgq = hi;
+ bkq = lo;
+ beq = lo;
+ viq = hi;
+ voq = hi;
+
+ hip = 0;
+ mgp = 0;
+ bkp = 1;
+ bep = 1;
+ vip = 0;
+ vop = 0;
+ break;
+ case 3:
+ beq = TRXDMA_QUEUE_LOW;
+ bkq = TRXDMA_QUEUE_LOW;
+ viq = TRXDMA_QUEUE_NORMAL;
+ voq = TRXDMA_QUEUE_HIGH;
+ mgq = TRXDMA_QUEUE_HIGH;
+ hiq = TRXDMA_QUEUE_HIGH;
+
+ hip = hiq ^ 3;
+ mgp = mgq ^ 3;
+ bkp = bkq ^ 3;
+ bep = beq ^ 3;
+ vip = viq ^ 3;
+ vop = viq ^ 3;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+
+ /*
+ * None of the vendor drivers are configuring the beacon
+ * queue here .... why?
+ */
+ if (!ret) {
+ val16 = rtl8xxxu_read16(priv, REG_TRXDMA_CTRL);
+ val16 &= 0x7;
+ val16 |= (voq << TRXDMA_CTRL_VOQ_SHIFT) |
+ (viq << TRXDMA_CTRL_VIQ_SHIFT) |
+ (beq << TRXDMA_CTRL_BEQ_SHIFT) |
+ (bkq << TRXDMA_CTRL_BKQ_SHIFT) |
+ (mgq << TRXDMA_CTRL_MGQ_SHIFT) |
+ (hiq << TRXDMA_CTRL_HIQ_SHIFT);
+ rtl8xxxu_write16(priv, REG_TRXDMA_CTRL, val16);
+
+ priv->pipe_out[TXDESC_QUEUE_VO] =
+ usb_sndbulkpipe(priv->udev, priv->out_ep[vop]);
+ priv->pipe_out[TXDESC_QUEUE_VI] =
+ usb_sndbulkpipe(priv->udev, priv->out_ep[vip]);
+ priv->pipe_out[TXDESC_QUEUE_BE] =
+ usb_sndbulkpipe(priv->udev, priv->out_ep[bep]);
+ priv->pipe_out[TXDESC_QUEUE_BK] =
+ usb_sndbulkpipe(priv->udev, priv->out_ep[bkp]);
+ priv->pipe_out[TXDESC_QUEUE_BEACON] =
+ usb_sndbulkpipe(priv->udev, priv->out_ep[0]);
+ priv->pipe_out[TXDESC_QUEUE_MGNT] =
+ usb_sndbulkpipe(priv->udev, priv->out_ep[mgp]);
+ priv->pipe_out[TXDESC_QUEUE_HIGH] =
+ usb_sndbulkpipe(priv->udev, priv->out_ep[hip]);
+ priv->pipe_out[TXDESC_QUEUE_CMD] =
+ usb_sndbulkpipe(priv->udev, priv->out_ep[0]);
+ }
+
+ return ret;
+}
+
+void rtl8xxxu_fill_iqk_matrix_a(struct rtl8xxxu_priv *priv, bool iqk_ok,
+ int result[][8], int candidate, bool tx_only)
+{
+ u32 oldval, x, tx0_a, reg;
+ int y, tx0_c;
+ u32 val32;
+
+ if (!iqk_ok)
+ return;
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_XA_TX_IQ_IMBALANCE);
+ oldval = val32 >> 22;
+
+ x = result[candidate][0];
+ if ((x & 0x00000200) != 0)
+ x = x | 0xfffffc00;
+ tx0_a = (x * oldval) >> 8;
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_XA_TX_IQ_IMBALANCE);
+ val32 &= ~0x3ff;
+ val32 |= tx0_a;
+ rtl8xxxu_write32(priv, REG_OFDM0_XA_TX_IQ_IMBALANCE, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_ENERGY_CCA_THRES);
+ val32 &= ~BIT(31);
+ if ((x * oldval >> 7) & 0x1)
+ val32 |= BIT(31);
+ rtl8xxxu_write32(priv, REG_OFDM0_ENERGY_CCA_THRES, val32);
+
+ y = result[candidate][1];
+ if ((y & 0x00000200) != 0)
+ y = y | 0xfffffc00;
+ tx0_c = (y * oldval) >> 8;
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_XC_TX_AFE);
+ val32 &= ~0xf0000000;
+ val32 |= (((tx0_c & 0x3c0) >> 6) << 28);
+ rtl8xxxu_write32(priv, REG_OFDM0_XC_TX_AFE, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_XA_TX_IQ_IMBALANCE);
+ val32 &= ~0x003f0000;
+ val32 |= ((tx0_c & 0x3f) << 16);
+ rtl8xxxu_write32(priv, REG_OFDM0_XA_TX_IQ_IMBALANCE, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_ENERGY_CCA_THRES);
+ val32 &= ~BIT(29);
+ if ((y * oldval >> 7) & 0x1)
+ val32 |= BIT(29);
+ rtl8xxxu_write32(priv, REG_OFDM0_ENERGY_CCA_THRES, val32);
+
+ if (tx_only) {
+ dev_dbg(&priv->udev->dev, "%s: only TX\n", __func__);
+ return;
+ }
+
+ reg = result[candidate][2];
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_XA_RX_IQ_IMBALANCE);
+ val32 &= ~0x3ff;
+ val32 |= (reg & 0x3ff);
+ rtl8xxxu_write32(priv, REG_OFDM0_XA_RX_IQ_IMBALANCE, val32);
+
+ reg = result[candidate][3] & 0x3F;
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_XA_RX_IQ_IMBALANCE);
+ val32 &= ~0xfc00;
+ val32 |= ((reg << 10) & 0xfc00);
+ rtl8xxxu_write32(priv, REG_OFDM0_XA_RX_IQ_IMBALANCE, val32);
+
+ reg = (result[candidate][3] >> 6) & 0xF;
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_RX_IQ_EXT_ANTA);
+ val32 &= ~0xf0000000;
+ val32 |= (reg << 28);
+ rtl8xxxu_write32(priv, REG_OFDM0_RX_IQ_EXT_ANTA, val32);
+}
+
+void rtl8xxxu_fill_iqk_matrix_b(struct rtl8xxxu_priv *priv, bool iqk_ok,
+ int result[][8], int candidate, bool tx_only)
+{
+ u32 oldval, x, tx1_a, reg;
+ int y, tx1_c;
+ u32 val32;
+
+ if (!iqk_ok)
+ return;
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_XB_TX_IQ_IMBALANCE);
+ oldval = val32 >> 22;
+
+ x = result[candidate][4];
+ if ((x & 0x00000200) != 0)
+ x = x | 0xfffffc00;
+ tx1_a = (x * oldval) >> 8;
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_XB_TX_IQ_IMBALANCE);
+ val32 &= ~0x3ff;
+ val32 |= tx1_a;
+ rtl8xxxu_write32(priv, REG_OFDM0_XB_TX_IQ_IMBALANCE, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_ENERGY_CCA_THRES);
+ val32 &= ~BIT(27);
+ if ((x * oldval >> 7) & 0x1)
+ val32 |= BIT(27);
+ rtl8xxxu_write32(priv, REG_OFDM0_ENERGY_CCA_THRES, val32);
+
+ y = result[candidate][5];
+ if ((y & 0x00000200) != 0)
+ y = y | 0xfffffc00;
+ tx1_c = (y * oldval) >> 8;
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_XD_TX_AFE);
+ val32 &= ~0xf0000000;
+ val32 |= (((tx1_c & 0x3c0) >> 6) << 28);
+ rtl8xxxu_write32(priv, REG_OFDM0_XD_TX_AFE, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_XB_TX_IQ_IMBALANCE);
+ val32 &= ~0x003f0000;
+ val32 |= ((tx1_c & 0x3f) << 16);
+ rtl8xxxu_write32(priv, REG_OFDM0_XB_TX_IQ_IMBALANCE, val32);
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_ENERGY_CCA_THRES);
+ val32 &= ~BIT(25);
+ if ((y * oldval >> 7) & 0x1)
+ val32 |= BIT(25);
+ rtl8xxxu_write32(priv, REG_OFDM0_ENERGY_CCA_THRES, val32);
+
+ if (tx_only) {
+ dev_dbg(&priv->udev->dev, "%s: only TX\n", __func__);
+ return;
+ }
+
+ reg = result[candidate][6];
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_XB_RX_IQ_IMBALANCE);
+ val32 &= ~0x3ff;
+ val32 |= (reg & 0x3ff);
+ rtl8xxxu_write32(priv, REG_OFDM0_XB_RX_IQ_IMBALANCE, val32);
+
+ reg = result[candidate][7] & 0x3f;
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_XB_RX_IQ_IMBALANCE);
+ val32 &= ~0xfc00;
+ val32 |= ((reg << 10) & 0xfc00);
+ rtl8xxxu_write32(priv, REG_OFDM0_XB_RX_IQ_IMBALANCE, val32);
+
+ reg = (result[candidate][7] >> 6) & 0xf;
+
+ val32 = rtl8xxxu_read32(priv, REG_OFDM0_AGCR_SSI_TABLE);
+ val32 &= ~0x0000f000;
+ val32 |= (reg << 12);
+ rtl8xxxu_write32(priv, REG_OFDM0_AGCR_SSI_TABLE, val32);
+}
+
+#define MAX_TOLERANCE 5
+
+static bool rtl8xxxu_simularity_compare(struct rtl8xxxu_priv *priv,
+ int result[][8], int c1, int c2)
+{
+ u32 i, j, diff, simubitmap, bound = 0;
+ int candidate[2] = {-1, -1}; /* for path A and path B */
+ bool retval = true;
+
+ if (priv->tx_paths > 1)
+ bound = 8;
+ else
+ bound = 4;
+
+ simubitmap = 0;
+
+ for (i = 0; i < bound; i++) {
+ diff = (result[c1][i] > result[c2][i]) ?
+ (result[c1][i] - result[c2][i]) :
+ (result[c2][i] - result[c1][i]);
+ if (diff > MAX_TOLERANCE) {
+ if ((i == 2 || i == 6) && !simubitmap) {
+ if (result[c1][i] + result[c1][i + 1] == 0)
+ candidate[(i / 4)] = c2;
+ else if (result[c2][i] + result[c2][i + 1] == 0)
+ candidate[(i / 4)] = c1;
+ else
+ simubitmap = simubitmap | (1 << i);
+ } else {
+ simubitmap = simubitmap | (1 << i);
+ }
+ }
+ }
+
+ if (simubitmap == 0) {
+ for (i = 0; i < (bound / 4); i++) {
+ if (candidate[i] >= 0) {
+ for (j = i * 4; j < (i + 1) * 4 - 2; j++)
+ result[3][j] = result[candidate[i]][j];
+ retval = false;
+ }
+ }
+ return retval;
+ } else if (!(simubitmap & 0x0f)) {
+ /* path A OK */
+ for (i = 0; i < 4; i++)
+ result[3][i] = result[c1][i];
+ } else if (!(simubitmap & 0xf0) && priv->tx_paths > 1) {
+ /* path B OK */
+ for (i = 4; i < 8; i++)
+ result[3][i] = result[c1][i];
+ }
+
+ return false;
+}
+
+bool rtl8xxxu_gen2_simularity_compare(struct rtl8xxxu_priv *priv,
+ int result[][8], int c1, int c2)
+{
+ u32 i, j, diff, simubitmap, bound = 0;
+ int candidate[2] = {-1, -1}; /* for path A and path B */
+ int tmp1, tmp2;
+ bool retval = true;
+
+ if (priv->tx_paths > 1)
+ bound = 8;
+ else
+ bound = 4;
+
+ simubitmap = 0;
+
+ for (i = 0; i < bound; i++) {
+ if (i & 1) {
+ if ((result[c1][i] & 0x00000200))
+ tmp1 = result[c1][i] | 0xfffffc00;
+ else
+ tmp1 = result[c1][i];
+
+ if ((result[c2][i]& 0x00000200))
+ tmp2 = result[c2][i] | 0xfffffc00;
+ else
+ tmp2 = result[c2][i];
+ } else {
+ tmp1 = result[c1][i];
+ tmp2 = result[c2][i];
+ }
+
+ diff = (tmp1 > tmp2) ? (tmp1 - tmp2) : (tmp2 - tmp1);
+
+ if (diff > MAX_TOLERANCE) {
+ if ((i == 2 || i == 6) && !simubitmap) {
+ if (result[c1][i] + result[c1][i + 1] == 0)
+ candidate[(i / 4)] = c2;
+ else if (result[c2][i] + result[c2][i + 1] == 0)
+ candidate[(i / 4)] = c1;
+ else
+ simubitmap = simubitmap | (1 << i);
+ } else {
+ simubitmap = simubitmap | (1 << i);
+ }
+ }
+ }
+
+ if (simubitmap == 0) {
+ for (i = 0; i < (bound / 4); i++) {
+ if (candidate[i] >= 0) {
+ for (j = i * 4; j < (i + 1) * 4 - 2; j++)
+ result[3][j] = result[candidate[i]][j];
+ retval = false;
+ }
+ }
+ return retval;
+ } else {
+ if (!(simubitmap & 0x03)) {
+ /* path A TX OK */
+ for (i = 0; i < 2; i++)
+ result[3][i] = result[c1][i];
+ }
+
+ if (!(simubitmap & 0x0c)) {
+ /* path A RX OK */
+ for (i = 2; i < 4; i++)
+ result[3][i] = result[c1][i];
+ }
+
+ if (!(simubitmap & 0x30) && priv->tx_paths > 1) {
+ /* path B RX OK */
+ for (i = 4; i < 6; i++)
+ result[3][i] = result[c1][i];
+ }
+
+ if (!(simubitmap & 0x30) && priv->tx_paths > 1) {
+ /* path B RX OK */
+ for (i = 6; i < 8; i++)
+ result[3][i] = result[c1][i];
+ }
+ }
+
+ return false;
+}
+
+void
+rtl8xxxu_save_mac_regs(struct rtl8xxxu_priv *priv, const u32 *reg, u32 *backup)
+{
+ int i;
+
+ for (i = 0; i < (RTL8XXXU_MAC_REGS - 1); i++)
+ backup[i] = rtl8xxxu_read8(priv, reg[i]);
+
+ backup[i] = rtl8xxxu_read32(priv, reg[i]);
+}
+
+void rtl8xxxu_restore_mac_regs(struct rtl8xxxu_priv *priv,
+ const u32 *reg, u32 *backup)
+{
+ int i;
+
+ for (i = 0; i < (RTL8XXXU_MAC_REGS - 1); i++)
+ rtl8xxxu_write8(priv, reg[i], backup[i]);
+
+ rtl8xxxu_write32(priv, reg[i], backup[i]);
+}
+
+void rtl8xxxu_save_regs(struct rtl8xxxu_priv *priv, const u32 *regs,
+ u32 *backup, int count)
+{
+ int i;
+
+ for (i = 0; i < count; i++)
+ backup[i] = rtl8xxxu_read32(priv, regs[i]);
+}
+
+void rtl8xxxu_restore_regs(struct rtl8xxxu_priv *priv, const u32 *regs,
+ u32 *backup, int count)
+{
+ int i;
+
+ for (i = 0; i < count; i++)
+ rtl8xxxu_write32(priv, regs[i], backup[i]);
+}
+
+
+void rtl8xxxu_path_adda_on(struct rtl8xxxu_priv *priv, const u32 *regs,
+ bool path_a_on)
+{
+ u32 path_on;
+ int i;
+
+ if (priv->tx_paths == 1) {
+ path_on = priv->fops->adda_1t_path_on;
+ rtl8xxxu_write32(priv, regs[0], priv->fops->adda_1t_init);
+ } else {
+ path_on = path_a_on ? priv->fops->adda_2t_path_on_a :
+ priv->fops->adda_2t_path_on_b;
+
+ rtl8xxxu_write32(priv, regs[0], path_on);
+ }
+
+ for (i = 1 ; i < RTL8XXXU_ADDA_REGS ; i++)
+ rtl8xxxu_write32(priv, regs[i], path_on);
+}
+
+void rtl8xxxu_mac_calibration(struct rtl8xxxu_priv *priv,
+ const u32 *regs, u32 *backup)
+{
+ int i = 0;
+
+ rtl8xxxu_write8(priv, regs[i], 0x3f);
+
+ for (i = 1 ; i < (RTL8XXXU_MAC_REGS - 1); i++)
+ rtl8xxxu_write8(priv, regs[i], (u8)(backup[i] & ~BIT(3)));
+
+ rtl8xxxu_write8(priv, regs[i], (u8)(backup[i] & ~BIT(5)));
+}
+
+static int rtl8xxxu_iqk_path_a(struct rtl8xxxu_priv *priv)
+{
+ u32 reg_eac, reg_e94, reg_e9c, reg_ea4, val32;
+ int result = 0;
+
+ /* path-A IQK setting */
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x10008c1f);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x10008c1f);
+ rtl8xxxu_write32(priv, REG_TX_IQK_PI_A, 0x82140102);
+
+ val32 = (priv->rf_paths > 1) ? 0x28160202 :
+ /*IS_81xxC_VENDOR_UMC_B_CUT(pHalData->VersionID)?0x28160202: */
+ 0x28160502;
+ rtl8xxxu_write32(priv, REG_RX_IQK_PI_A, val32);
+
+ /* path-B IQK setting */
+ if (priv->rf_paths > 1) {
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_B, 0x10008c22);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_B, 0x10008c22);
+ rtl8xxxu_write32(priv, REG_TX_IQK_PI_B, 0x82140102);
+ rtl8xxxu_write32(priv, REG_RX_IQK_PI_B, 0x28160202);
+ }
+
+ /* LO calibration setting */
+ rtl8xxxu_write32(priv, REG_IQK_AGC_RSP, 0x001028d1);
+
+ /* One shot, path A LOK & IQK */
+ rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf9000000);
+ rtl8xxxu_write32(priv, REG_IQK_AGC_PTS, 0xf8000000);
+
+ mdelay(1);
+
+ /* Check failed */
+ reg_eac = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_A_2);
+ reg_e94 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_A);
+ reg_e9c = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_A);
+ reg_ea4 = rtl8xxxu_read32(priv, REG_RX_POWER_BEFORE_IQK_A_2);
+
+ if (!(reg_eac & BIT(28)) &&
+ ((reg_e94 & 0x03ff0000) != 0x01420000) &&
+ ((reg_e9c & 0x03ff0000) != 0x00420000))
+ result |= 0x01;
+ else /* If TX not OK, ignore RX */
+ goto out;
+
+ /* If TX is OK, check whether RX is OK */
+ if (!(reg_eac & BIT(27)) &&
+ ((reg_ea4 & 0x03ff0000) != 0x01320000) &&
+ ((reg_eac & 0x03ff0000) != 0x00360000))
+ result |= 0x02;
+ else
+ dev_warn(&priv->udev->dev, "%s: Path A RX IQK failed!\n",
+ __func__);
+out:
+ return result;
+}
+
+static int rtl8xxxu_iqk_path_b(struct rtl8xxxu_priv *priv)
+{
+ u32 reg_eac, reg_eb4, reg_ebc, reg_ec4, reg_ecc;
+ int result = 0;
+
+ /* One shot, path B LOK & IQK */
+ rtl8xxxu_write32(priv, REG_IQK_AGC_CONT, 0x00000002);
+ rtl8xxxu_write32(priv, REG_IQK_AGC_CONT, 0x00000000);
+
+ mdelay(1);
+
+ /* Check failed */
+ reg_eac = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_A_2);
+ reg_eb4 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_B);
+ reg_ebc = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_B);
+ reg_ec4 = rtl8xxxu_read32(priv, REG_RX_POWER_BEFORE_IQK_B_2);
+ reg_ecc = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_B_2);
+
+ if (!(reg_eac & BIT(31)) &&
+ ((reg_eb4 & 0x03ff0000) != 0x01420000) &&
+ ((reg_ebc & 0x03ff0000) != 0x00420000))
+ result |= 0x01;
+ else
+ goto out;
+
+ if (!(reg_eac & BIT(30)) &&
+ (((reg_ec4 & 0x03ff0000) >> 16) != 0x132) &&
+ (((reg_ecc & 0x03ff0000) >> 16) != 0x36))
+ result |= 0x02;
+ else
+ dev_warn(&priv->udev->dev, "%s: Path B RX IQK failed!\n",
+ __func__);
+out:
+ return result;
+}
+
+static void rtl8xxxu_phy_iqcalibrate(struct rtl8xxxu_priv *priv,
+ int result[][8], int t)
+{
+ struct device *dev = &priv->udev->dev;
+ u32 i, val32;
+ int path_a_ok, path_b_ok;
+ int retry = 2;
+ const u32 adda_regs[RTL8XXXU_ADDA_REGS] = {
+ REG_FPGA0_XCD_SWITCH_CTRL, REG_BLUETOOTH,
+ REG_RX_WAIT_CCA, REG_TX_CCK_RFON,
+ REG_TX_CCK_BBON, REG_TX_OFDM_RFON,
+ REG_TX_OFDM_BBON, REG_TX_TO_RX,
+ REG_TX_TO_TX, REG_RX_CCK,
+ REG_RX_OFDM, REG_RX_WAIT_RIFS,
+ REG_RX_TO_RX, REG_STANDBY,
+ REG_SLEEP, REG_PMPD_ANAEN
+ };
+ const u32 iqk_mac_regs[RTL8XXXU_MAC_REGS] = {
+ REG_TXPAUSE, REG_BEACON_CTRL,
+ REG_BEACON_CTRL_1, REG_GPIO_MUXCFG
+ };
+ const u32 iqk_bb_regs[RTL8XXXU_BB_REGS] = {
+ REG_OFDM0_TRX_PATH_ENABLE, REG_OFDM0_TR_MUX_PAR,
+ REG_FPGA0_XCD_RF_SW_CTRL, REG_CONFIG_ANT_A, REG_CONFIG_ANT_B,
+ REG_FPGA0_XAB_RF_SW_CTRL, REG_FPGA0_XA_RF_INT_OE,
+ REG_FPGA0_XB_RF_INT_OE, REG_FPGA0_RF_MODE
+ };
+
+ /*
+ * Note: IQ calibration must be performed after loading
+ * PHY_REG.txt , and radio_a, radio_b.txt
+ */
+
+ if (t == 0) {
+ /* Save ADDA parameters, turn Path A ADDA on */
+ rtl8xxxu_save_regs(priv, adda_regs, priv->adda_backup,
+ RTL8XXXU_ADDA_REGS);
+ rtl8xxxu_save_mac_regs(priv, iqk_mac_regs, priv->mac_backup);
+ rtl8xxxu_save_regs(priv, iqk_bb_regs,
+ priv->bb_backup, RTL8XXXU_BB_REGS);
+ }
+
+ rtl8xxxu_path_adda_on(priv, adda_regs, true);
+
+ if (t == 0) {
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_XA_HSSI_PARM1);
+ if (val32 & FPGA0_HSSI_PARM1_PI)
+ priv->pi_enabled = 1;
+ }
+
+ if (!priv->pi_enabled) {
+ /* Switch BB to PI mode to do IQ Calibration. */
+ rtl8xxxu_write32(priv, REG_FPGA0_XA_HSSI_PARM1, 0x01000100);
+ rtl8xxxu_write32(priv, REG_FPGA0_XB_HSSI_PARM1, 0x01000100);
+ }
+
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_RF_MODE);
+ val32 &= ~FPGA_RF_MODE_CCK;
+ rtl8xxxu_write32(priv, REG_FPGA0_RF_MODE, val32);
+
+ rtl8xxxu_write32(priv, REG_OFDM0_TRX_PATH_ENABLE, 0x03a05600);
+ rtl8xxxu_write32(priv, REG_OFDM0_TR_MUX_PAR, 0x000800e4);
+ rtl8xxxu_write32(priv, REG_FPGA0_XCD_RF_SW_CTRL, 0x22204000);
+
+ if (!priv->no_pape) {
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_XAB_RF_SW_CTRL);
+ val32 |= (FPGA0_RF_PAPE |
+ (FPGA0_RF_PAPE << FPGA0_RF_BD_CTRL_SHIFT));
+ rtl8xxxu_write32(priv, REG_FPGA0_XAB_RF_SW_CTRL, val32);
+ }
+
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_XA_RF_INT_OE);
+ val32 &= ~BIT(10);
+ rtl8xxxu_write32(priv, REG_FPGA0_XA_RF_INT_OE, val32);
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_XB_RF_INT_OE);
+ val32 &= ~BIT(10);
+ rtl8xxxu_write32(priv, REG_FPGA0_XB_RF_INT_OE, val32);
+
+ if (priv->tx_paths > 1) {
+ rtl8xxxu_write32(priv, REG_FPGA0_XA_LSSI_PARM, 0x00010000);
+ rtl8xxxu_write32(priv, REG_FPGA0_XB_LSSI_PARM, 0x00010000);
+ }
+
+ /* MAC settings */
+ rtl8xxxu_mac_calibration(priv, iqk_mac_regs, priv->mac_backup);
+
+ /* Page B init */
+ rtl8xxxu_write32(priv, REG_CONFIG_ANT_A, 0x00080000);
+
+ if (priv->tx_paths > 1)
+ rtl8xxxu_write32(priv, REG_CONFIG_ANT_B, 0x00080000);
+
+ /* IQ calibration setting */
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
+ rtl8xxxu_write32(priv, REG_TX_IQK, 0x01007c00);
+ rtl8xxxu_write32(priv, REG_RX_IQK, 0x01004800);
+
+ for (i = 0; i < retry; i++) {
+ path_a_ok = rtl8xxxu_iqk_path_a(priv);
+ if (path_a_ok == 0x03) {
+ val32 = rtl8xxxu_read32(priv,
+ REG_TX_POWER_BEFORE_IQK_A);
+ result[t][0] = (val32 >> 16) & 0x3ff;
+ val32 = rtl8xxxu_read32(priv,
+ REG_TX_POWER_AFTER_IQK_A);
+ result[t][1] = (val32 >> 16) & 0x3ff;
+ val32 = rtl8xxxu_read32(priv,
+ REG_RX_POWER_BEFORE_IQK_A_2);
+ result[t][2] = (val32 >> 16) & 0x3ff;
+ val32 = rtl8xxxu_read32(priv,
+ REG_RX_POWER_AFTER_IQK_A_2);
+ result[t][3] = (val32 >> 16) & 0x3ff;
+ break;
+ } else if (i == (retry - 1) && path_a_ok == 0x01) {
+ /* TX IQK OK */
+ dev_dbg(dev, "%s: Path A IQK Only Tx Success!!\n",
+ __func__);
+
+ val32 = rtl8xxxu_read32(priv,
+ REG_TX_POWER_BEFORE_IQK_A);
+ result[t][0] = (val32 >> 16) & 0x3ff;
+ val32 = rtl8xxxu_read32(priv,
+ REG_TX_POWER_AFTER_IQK_A);
+ result[t][1] = (val32 >> 16) & 0x3ff;
+ }
+ }
+
+ if (!path_a_ok)
+ dev_dbg(dev, "%s: Path A IQK failed!\n", __func__);
+
+ if (priv->tx_paths > 1) {
+ /*
+ * Path A into standby
+ */
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x0);
+ rtl8xxxu_write32(priv, REG_FPGA0_XA_LSSI_PARM, 0x00010000);
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0x80800000);
+
+ /* Turn Path B ADDA on */
+ rtl8xxxu_path_adda_on(priv, adda_regs, false);
+
+ for (i = 0; i < retry; i++) {
+ path_b_ok = rtl8xxxu_iqk_path_b(priv);
+ if (path_b_ok == 0x03) {
+ val32 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_B);
+ result[t][4] = (val32 >> 16) & 0x3ff;
+ val32 = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_B);
+ result[t][5] = (val32 >> 16) & 0x3ff;
+ val32 = rtl8xxxu_read32(priv, REG_RX_POWER_BEFORE_IQK_B_2);
+ result[t][6] = (val32 >> 16) & 0x3ff;
+ val32 = rtl8xxxu_read32(priv, REG_RX_POWER_AFTER_IQK_B_2);
+ result[t][7] = (val32 >> 16) & 0x3ff;
+ break;
+ } else if (i == (retry - 1) && path_b_ok == 0x01) {
+ /* TX IQK OK */
+ val32 = rtl8xxxu_read32(priv, REG_TX_POWER_BEFORE_IQK_B);
+ result[t][4] = (val32 >> 16) & 0x3ff;
+ val32 = rtl8xxxu_read32(priv, REG_TX_POWER_AFTER_IQK_B);
+ result[t][5] = (val32 >> 16) & 0x3ff;
+ }
+ }
+
+ if (!path_b_ok)
+ dev_dbg(dev, "%s: Path B IQK failed!\n", __func__);
+ }
+
+ /* Back to BB mode, load original value */
+ rtl8xxxu_write32(priv, REG_FPGA0_IQK, 0);
+
+ if (t) {
+ if (!priv->pi_enabled) {
+ /*
+ * Switch back BB to SI mode after finishing
+ * IQ Calibration
+ */
+ val32 = 0x01000000;
+ rtl8xxxu_write32(priv, REG_FPGA0_XA_HSSI_PARM1, val32);
+ rtl8xxxu_write32(priv, REG_FPGA0_XB_HSSI_PARM1, val32);
+ }
+
+ /* Reload ADDA power saving parameters */
+ rtl8xxxu_restore_regs(priv, adda_regs, priv->adda_backup,
+ RTL8XXXU_ADDA_REGS);
+
+ /* Reload MAC parameters */
+ rtl8xxxu_restore_mac_regs(priv, iqk_mac_regs, priv->mac_backup);
+
+ /* Reload BB parameters */
+ rtl8xxxu_restore_regs(priv, iqk_bb_regs,
+ priv->bb_backup, RTL8XXXU_BB_REGS);
+
+ /* Restore RX initial gain */
+ rtl8xxxu_write32(priv, REG_FPGA0_XA_LSSI_PARM, 0x00032ed3);
+
+ if (priv->tx_paths > 1) {
+ rtl8xxxu_write32(priv, REG_FPGA0_XB_LSSI_PARM,
+ 0x00032ed3);
+ }
+
+ /* Load 0xe30 IQC default value */
+ rtl8xxxu_write32(priv, REG_TX_IQK_TONE_A, 0x01008c00);
+ rtl8xxxu_write32(priv, REG_RX_IQK_TONE_A, 0x01008c00);
+ }
+}
+
+void rtl8xxxu_gen2_prepare_calibrate(struct rtl8xxxu_priv *priv, u8 start)
+{
+ struct h2c_cmd h2c;
+
+ memset(&h2c, 0, sizeof(struct h2c_cmd));
+ h2c.bt_wlan_calibration.cmd = H2C_8723B_BT_WLAN_CALIBRATION;
+ h2c.bt_wlan_calibration.data = start;
+
+ rtl8xxxu_gen2_h2c_cmd(priv, &h2c, sizeof(h2c.bt_wlan_calibration));
+}
+
+void rtl8xxxu_gen1_phy_iq_calibrate(struct rtl8xxxu_priv *priv)
+{
+ struct device *dev = &priv->udev->dev;
+ int result[4][8]; /* last is final result */
+ int i, candidate;
+ bool path_a_ok, path_b_ok;
+ u32 reg_e94, reg_e9c, reg_ea4, reg_eac;
+ u32 reg_eb4, reg_ebc, reg_ec4, reg_ecc;
+ s32 reg_tmp = 0;
+ bool simu;
+
+ memset(result, 0, sizeof(result));
+ candidate = -1;
+
+ path_a_ok = false;
+ path_b_ok = false;
+
+ rtl8xxxu_read32(priv, REG_FPGA0_RF_MODE);
+
+ for (i = 0; i < 3; i++) {
+ rtl8xxxu_phy_iqcalibrate(priv, result, i);
+
+ if (i == 1) {
+ simu = rtl8xxxu_simularity_compare(priv, result, 0, 1);
+ if (simu) {
+ candidate = 0;
+ break;
+ }
+ }
+
+ if (i == 2) {
+ simu = rtl8xxxu_simularity_compare(priv, result, 0, 2);
+ if (simu) {
+ candidate = 0;
+ break;
+ }
+
+ simu = rtl8xxxu_simularity_compare(priv, result, 1, 2);
+ if (simu) {
+ candidate = 1;
+ } else {
+ for (i = 0; i < 8; i++)
+ reg_tmp += result[3][i];
+
+ if (reg_tmp)
+ candidate = 3;
+ else
+ candidate = -1;
+ }
+ }
+ }
+
+ for (i = 0; i < 4; i++) {
+ reg_e94 = result[i][0];
+ reg_e9c = result[i][1];
+ reg_ea4 = result[i][2];
+ reg_eac = result[i][3];
+ reg_eb4 = result[i][4];
+ reg_ebc = result[i][5];
+ reg_ec4 = result[i][6];
+ reg_ecc = result[i][7];
+ }
+
+ if (candidate >= 0) {
+ reg_e94 = result[candidate][0];
+ priv->rege94 = reg_e94;
+ reg_e9c = result[candidate][1];
+ priv->rege9c = reg_e9c;
+ reg_ea4 = result[candidate][2];
+ reg_eac = result[candidate][3];
+ reg_eb4 = result[candidate][4];
+ priv->regeb4 = reg_eb4;
+ reg_ebc = result[candidate][5];
+ priv->regebc = reg_ebc;
+ reg_ec4 = result[candidate][6];
+ reg_ecc = result[candidate][7];
+ dev_dbg(dev, "%s: candidate is %x\n", __func__, candidate);
+ dev_dbg(dev,
+ "%s: e94 =%x e9c=%x ea4=%x eac=%x eb4=%x ebc=%x ec4=%x "
+ "ecc=%x\n ", __func__, reg_e94, reg_e9c,
+ reg_ea4, reg_eac, reg_eb4, reg_ebc, reg_ec4, reg_ecc);
+ path_a_ok = true;
+ path_b_ok = true;
+ } else {
+ reg_e94 = reg_eb4 = priv->rege94 = priv->regeb4 = 0x100;
+ reg_e9c = reg_ebc = priv->rege9c = priv->regebc = 0x0;
+ }
+
+ if (reg_e94 && candidate >= 0)
+ rtl8xxxu_fill_iqk_matrix_a(priv, path_a_ok, result,
+ candidate, (reg_ea4 == 0));
+
+ if (priv->tx_paths > 1 && reg_eb4)
+ rtl8xxxu_fill_iqk_matrix_b(priv, path_b_ok, result,
+ candidate, (reg_ec4 == 0));
+
+ rtl8xxxu_save_regs(priv, rtl8xxxu_iqk_phy_iq_bb_reg,
+ priv->bb_recovery_backup, RTL8XXXU_BB_REGS);
+}
+
+static void rtl8723a_phy_lc_calibrate(struct rtl8xxxu_priv *priv)
+{
+ u32 val32;
+ u32 rf_amode, rf_bmode = 0, lstf;
+
+ /* Check continuous TX and Packet TX */
+ lstf = rtl8xxxu_read32(priv, REG_OFDM1_LSTF);
+
+ if (lstf & OFDM_LSTF_MASK) {
+ /* Disable all continuous TX */
+ val32 = lstf & ~OFDM_LSTF_MASK;
+ rtl8xxxu_write32(priv, REG_OFDM1_LSTF, val32);
+
+ /* Read original RF mode Path A */
+ rf_amode = rtl8xxxu_read_rfreg(priv, RF_A, RF6052_REG_AC);
+
+ /* Set RF mode to standby Path A */
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_AC,
+ (rf_amode & 0x8ffff) | 0x10000);
+
+ /* Path-B */
+ if (priv->tx_paths > 1) {
+ rf_bmode = rtl8xxxu_read_rfreg(priv, RF_B,
+ RF6052_REG_AC);
+
+ rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_AC,
+ (rf_bmode & 0x8ffff) | 0x10000);
+ }
+ } else {
+ /* Deal with Packet TX case */
+ /* block all queues */
+ rtl8xxxu_write8(priv, REG_TXPAUSE, 0xff);
+ }
+
+ /* Start LC calibration */
+ if (priv->fops->has_s0s1)
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_S0S1, 0xdfbe0);
+ val32 = rtl8xxxu_read_rfreg(priv, RF_A, RF6052_REG_MODE_AG);
+ val32 |= 0x08000;
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_MODE_AG, val32);
+
+ msleep(100);
+
+ if (priv->fops->has_s0s1)
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_S0S1, 0xdffe0);
+
+ /* Restore original parameters */
+ if (lstf & OFDM_LSTF_MASK) {
+ /* Path-A */
+ rtl8xxxu_write32(priv, REG_OFDM1_LSTF, lstf);
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_AC, rf_amode);
+
+ /* Path-B */
+ if (priv->tx_paths > 1)
+ rtl8xxxu_write_rfreg(priv, RF_B, RF6052_REG_AC,
+ rf_bmode);
+ } else /* Deal with Packet TX case */
+ rtl8xxxu_write8(priv, REG_TXPAUSE, 0x00);
+}
+
+static int rtl8xxxu_set_mac(struct rtl8xxxu_priv *priv)
+{
+ int i;
+ u16 reg;
+
+ reg = REG_MACID;
+
+ for (i = 0; i < ETH_ALEN; i++)
+ rtl8xxxu_write8(priv, reg + i, priv->mac_addr[i]);
+
+ return 0;
+}
+
+static int rtl8xxxu_set_bssid(struct rtl8xxxu_priv *priv, const u8 *bssid)
+{
+ int i;
+ u16 reg;
+
+ dev_dbg(&priv->udev->dev, "%s: (%pM)\n", __func__, bssid);
+
+ reg = REG_BSSID;
+
+ for (i = 0; i < ETH_ALEN; i++)
+ rtl8xxxu_write8(priv, reg + i, bssid[i]);
+
+ return 0;
+}
+
+static void
+rtl8xxxu_set_ampdu_factor(struct rtl8xxxu_priv *priv, u8 ampdu_factor)
+{
+ u8 vals[4] = { 0x41, 0xa8, 0x72, 0xb9 };
+ u8 max_agg = 0xf;
+ int i;
+
+ ampdu_factor = 1 << (ampdu_factor + 2);
+ if (ampdu_factor > max_agg)
+ ampdu_factor = max_agg;
+
+ for (i = 0; i < 4; i++) {
+ if ((vals[i] & 0xf0) > (ampdu_factor << 4))
+ vals[i] = (vals[i] & 0x0f) | (ampdu_factor << 4);
+
+ if ((vals[i] & 0x0f) > ampdu_factor)
+ vals[i] = (vals[i] & 0xf0) | ampdu_factor;
+
+ rtl8xxxu_write8(priv, REG_AGGLEN_LMT + i, vals[i]);
+ }
+}
+
+static void rtl8xxxu_set_ampdu_min_space(struct rtl8xxxu_priv *priv, u8 density)
+{
+ u8 val8;
+
+ val8 = rtl8xxxu_read8(priv, REG_AMPDU_MIN_SPACE);
+ val8 &= 0xf8;
+ val8 |= density;
+ rtl8xxxu_write8(priv, REG_AMPDU_MIN_SPACE, val8);
+}
+
+static int rtl8xxxu_active_to_emu(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+ int count, ret = 0;
+
+ /* Start of rtl8723AU_card_enable_flow */
+ /* Act to Cardemu sequence*/
+ /* Turn off RF */
+ rtl8xxxu_write8(priv, REG_RF_CTRL, 0);
+
+ /* 0x004E[7] = 0, switch DPDT_SEL_P output from register 0x0065[2] */
+ val8 = rtl8xxxu_read8(priv, REG_LEDCFG2);
+ val8 &= ~LEDCFG2_DPDT_SELECT;
+ rtl8xxxu_write8(priv, REG_LEDCFG2, val8);
+
+ /* 0x0005[1] = 1 turn off MAC by HW state machine*/
+ val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
+ val8 |= BIT(1);
+ rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
+
+ for (count = RTL8XXXU_MAX_REG_POLL; count; count--) {
+ val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
+ if ((val8 & BIT(1)) == 0)
+ break;
+ udelay(10);
+ }
+
+ if (!count) {
+ dev_warn(&priv->udev->dev, "%s: Disabling MAC timed out\n",
+ __func__);
+ ret = -EBUSY;
+ goto exit;
+ }
+
+ /* 0x0000[5] = 1 analog Ips to digital, 1:isolation */
+ val8 = rtl8xxxu_read8(priv, REG_SYS_ISO_CTRL);
+ val8 |= SYS_ISO_ANALOG_IPS;
+ rtl8xxxu_write8(priv, REG_SYS_ISO_CTRL, val8);
+
+ /* 0x0020[0] = 0 disable LDOA12 MACRO block*/
+ val8 = rtl8xxxu_read8(priv, REG_LDOA15_CTRL);
+ val8 &= ~LDOA15_ENABLE;
+ rtl8xxxu_write8(priv, REG_LDOA15_CTRL, val8);
+
+exit:
+ return ret;
+}
+
+int rtl8xxxu_active_to_lps(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+ u8 val32;
+ int count, ret = 0;
+
+ rtl8xxxu_write8(priv, REG_TXPAUSE, 0xff);
+
+ /*
+ * Poll - wait for RX packet to complete
+ */
+ for (count = RTL8XXXU_MAX_REG_POLL; count; count--) {
+ val32 = rtl8xxxu_read32(priv, 0x5f8);
+ if (!val32)
+ break;
+ udelay(10);
+ }
+
+ if (!count) {
+ dev_warn(&priv->udev->dev,
+ "%s: RX poll timed out (0x05f8)\n", __func__);
+ ret = -EBUSY;
+ goto exit;
+ }
+
+ /* Disable CCK and OFDM, clock gated */
+ val8 = rtl8xxxu_read8(priv, REG_SYS_FUNC);
+ val8 &= ~SYS_FUNC_BBRSTB;
+ rtl8xxxu_write8(priv, REG_SYS_FUNC, val8);
+
+ udelay(2);
+
+ /* Reset baseband */
+ val8 = rtl8xxxu_read8(priv, REG_SYS_FUNC);
+ val8 &= ~SYS_FUNC_BB_GLB_RSTN;
+ rtl8xxxu_write8(priv, REG_SYS_FUNC, val8);
+
+ /* Reset MAC TRX */
+ val8 = rtl8xxxu_read8(priv, REG_CR);
+ val8 = CR_HCI_TXDMA_ENABLE | CR_HCI_RXDMA_ENABLE;
+ rtl8xxxu_write8(priv, REG_CR, val8);
+
+ /* Reset MAC TRX */
+ val8 = rtl8xxxu_read8(priv, REG_CR + 1);
+ val8 &= ~BIT(1); /* CR_SECURITY_ENABLE */
+ rtl8xxxu_write8(priv, REG_CR + 1, val8);
+
+ /* Respond TX OK to scheduler */
+ val8 = rtl8xxxu_read8(priv, REG_DUAL_TSF_RST);
+ val8 |= DUAL_TSF_TX_OK;
+ rtl8xxxu_write8(priv, REG_DUAL_TSF_RST, val8);
+
+exit:
+ return ret;
+}
+
+void rtl8xxxu_disabled_to_emu(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+
+ /* Clear suspend enable and power down enable*/
+ val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
+ val8 &= ~(BIT(3) | BIT(7));
+ rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
+
+ /* 0x48[16] = 0 to disable GPIO9 as EXT WAKEUP*/
+ val8 = rtl8xxxu_read8(priv, REG_GPIO_INTM + 2);
+ val8 &= ~BIT(0);
+ rtl8xxxu_write8(priv, REG_GPIO_INTM + 2, val8);
+
+ /* 0x04[12:11] = 11 enable WL suspend*/
+ val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
+ val8 &= ~(BIT(3) | BIT(4));
+ rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
+}
+
+static int rtl8xxxu_emu_to_disabled(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+
+ /* 0x0007[7:0] = 0x20 SOP option to disable BG/MB */
+ rtl8xxxu_write8(priv, REG_APS_FSMCO + 3, 0x20);
+
+ /* 0x04[12:11] = 01 enable WL suspend */
+ val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
+ val8 &= ~BIT(4);
+ val8 |= BIT(3);
+ rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
+
+ val8 = rtl8xxxu_read8(priv, REG_APS_FSMCO + 1);
+ val8 |= BIT(7);
+ rtl8xxxu_write8(priv, REG_APS_FSMCO + 1, val8);
+
+ /* 0x48[16] = 1 to enable GPIO9 as EXT wakeup */
+ val8 = rtl8xxxu_read8(priv, REG_GPIO_INTM + 2);
+ val8 |= BIT(0);
+ rtl8xxxu_write8(priv, REG_GPIO_INTM + 2, val8);
+
+ return 0;
+}
+
+int rtl8xxxu_flush_fifo(struct rtl8xxxu_priv *priv)
+{
+ struct device *dev = &priv->udev->dev;
+ u32 val32;
+ int retry, retval;
+
+ rtl8xxxu_write8(priv, REG_TXPAUSE, 0xff);
+
+ val32 = rtl8xxxu_read32(priv, REG_RXPKT_NUM);
+ val32 |= RXPKT_NUM_RW_RELEASE_EN;
+ rtl8xxxu_write32(priv, REG_RXPKT_NUM, val32);
+
+ retry = 100;
+ retval = -EBUSY;
+
+ do {
+ val32 = rtl8xxxu_read32(priv, REG_RXPKT_NUM);
+ if (val32 & RXPKT_NUM_RXDMA_IDLE) {
+ retval = 0;
+ break;
+ }
+ } while (retry--);
+
+ rtl8xxxu_write16(priv, REG_RQPN_NPQ, 0);
+ rtl8xxxu_write32(priv, REG_RQPN, 0x80000000);
+ mdelay(2);
+
+ if (!retry)
+ dev_warn(dev, "Failed to flush FIFO\n");
+
+ return retval;
+}
+
+void rtl8xxxu_gen1_usb_quirks(struct rtl8xxxu_priv *priv)
+{
+ /* Fix USB interface interference issue */
+ rtl8xxxu_write8(priv, 0xfe40, 0xe0);
+ rtl8xxxu_write8(priv, 0xfe41, 0x8d);
+ rtl8xxxu_write8(priv, 0xfe42, 0x80);
+ /*
+ * This sets TXDMA_OFFSET_DROP_DATA_EN (bit 9) as well as bits
+ * 8 and 5, for which I have found no documentation.
+ */
+ rtl8xxxu_write32(priv, REG_TXDMA_OFFSET_CHK, 0xfd0320);
+
+ /*
+ * Solve too many protocol error on USB bus.
+ * Can't do this for 8188/8192 UMC A cut parts
+ */
+ if (!(!priv->chip_cut && priv->vendor_umc)) {
+ rtl8xxxu_write8(priv, 0xfe40, 0xe6);
+ rtl8xxxu_write8(priv, 0xfe41, 0x94);
+ rtl8xxxu_write8(priv, 0xfe42, 0x80);
+
+ rtl8xxxu_write8(priv, 0xfe40, 0xe0);
+ rtl8xxxu_write8(priv, 0xfe41, 0x19);
+ rtl8xxxu_write8(priv, 0xfe42, 0x80);
+
+ rtl8xxxu_write8(priv, 0xfe40, 0xe5);
+ rtl8xxxu_write8(priv, 0xfe41, 0x91);
+ rtl8xxxu_write8(priv, 0xfe42, 0x80);
+
+ rtl8xxxu_write8(priv, 0xfe40, 0xe2);
+ rtl8xxxu_write8(priv, 0xfe41, 0x81);
+ rtl8xxxu_write8(priv, 0xfe42, 0x80);
+ }
+}
+
+void rtl8xxxu_gen2_usb_quirks(struct rtl8xxxu_priv *priv)
+{
+ u32 val32;
+
+ val32 = rtl8xxxu_read32(priv, REG_TXDMA_OFFSET_CHK);
+ val32 |= TXDMA_OFFSET_DROP_DATA_EN;
+ rtl8xxxu_write32(priv, REG_TXDMA_OFFSET_CHK, val32);
+}
+
+void rtl8xxxu_power_off(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+ u16 val16;
+ u32 val32;
+
+ /*
+ * Workaround for 8188RU LNA power leakage problem.
+ */
+ if (priv->rtl_chip == RTL8188R) {
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_XCD_RF_PARM);
+ val32 |= BIT(1);
+ rtl8xxxu_write32(priv, REG_FPGA0_XCD_RF_PARM, val32);
+ }
+
+ rtl8xxxu_flush_fifo(priv);
+
+ rtl8xxxu_active_to_lps(priv);
+
+ /* Turn off RF */
+ rtl8xxxu_write8(priv, REG_RF_CTRL, 0x00);
+
+ /* Reset Firmware if running in RAM */
+ if (rtl8xxxu_read8(priv, REG_MCU_FW_DL) & MCU_FW_RAM_SEL)
+ rtl8xxxu_firmware_self_reset(priv);
+
+ /* Reset MCU */
+ val16 = rtl8xxxu_read16(priv, REG_SYS_FUNC);
+ val16 &= ~SYS_FUNC_CPU_ENABLE;
+ rtl8xxxu_write16(priv, REG_SYS_FUNC, val16);
+
+ /* Reset MCU ready status */
+ rtl8xxxu_write8(priv, REG_MCU_FW_DL, 0x00);
+
+ rtl8xxxu_active_to_emu(priv);
+ rtl8xxxu_emu_to_disabled(priv);
+
+ /* Reset MCU IO Wrapper */
+ val8 = rtl8xxxu_read8(priv, REG_RSV_CTRL + 1);
+ val8 &= ~BIT(0);
+ rtl8xxxu_write8(priv, REG_RSV_CTRL + 1, val8);
+
+ val8 = rtl8xxxu_read8(priv, REG_RSV_CTRL + 1);
+ val8 |= BIT(0);
+ rtl8xxxu_write8(priv, REG_RSV_CTRL + 1, val8);
+
+ /* RSV_CTRL 0x1C[7:0] = 0x0e lock ISO/CLK/Power control register */
+ rtl8xxxu_write8(priv, REG_RSV_CTRL, 0x0e);
+}
+
+#ifdef NEED_PS_TDMA
+static void rtl8723bu_set_ps_tdma(struct rtl8xxxu_priv *priv,
+ u8 arg1, u8 arg2, u8 arg3, u8 arg4, u8 arg5)
+{
+ struct h2c_cmd h2c;
+
+ memset(&h2c, 0, sizeof(struct h2c_cmd));
+ h2c.b_type_dma.cmd = H2C_8723B_B_TYPE_TDMA;
+ h2c.b_type_dma.data1 = arg1;
+ h2c.b_type_dma.data2 = arg2;
+ h2c.b_type_dma.data3 = arg3;
+ h2c.b_type_dma.data4 = arg4;
+ h2c.b_type_dma.data5 = arg5;
+ rtl8xxxu_gen2_h2c_cmd(priv, &h2c, sizeof(h2c.b_type_dma));
+}
+#endif
+
+void rtl8xxxu_gen2_disable_rf(struct rtl8xxxu_priv *priv)
+{
+ u32 val32;
+
+ val32 = rtl8xxxu_read32(priv, REG_RX_WAIT_CCA);
+ val32 &= ~(BIT(22) | BIT(23));
+ rtl8xxxu_write32(priv, REG_RX_WAIT_CCA, val32);
+}
+
+static void rtl8xxxu_old_init_queue_reserved_page(struct rtl8xxxu_priv *priv)
+{
+ u8 val8;
+ u32 val32;
+
+ if (priv->ep_tx_normal_queue)
+ val8 = TX_PAGE_NUM_NORM_PQ;
+ else
+ val8 = 0;
+
+ rtl8xxxu_write8(priv, REG_RQPN_NPQ, val8);
+
+ val32 = (TX_PAGE_NUM_PUBQ << RQPN_PUB_PQ_SHIFT) | RQPN_LOAD;
+
+ if (priv->ep_tx_high_queue)
+ val32 |= (TX_PAGE_NUM_HI_PQ << RQPN_HI_PQ_SHIFT);
+ if (priv->ep_tx_low_queue)
+ val32 |= (TX_PAGE_NUM_LO_PQ << RQPN_LO_PQ_SHIFT);
+
+ rtl8xxxu_write32(priv, REG_RQPN, val32);
+}
+
+static void rtl8xxxu_init_queue_reserved_page(struct rtl8xxxu_priv *priv)
+{
+ struct rtl8xxxu_fileops *fops = priv->fops;
+ u32 hq, lq, nq, eq, pubq;
+ u32 val32;
+
+ hq = 0;
+ lq = 0;
+ nq = 0;
+ eq = 0;
+ pubq = 0;
+
+ if (priv->ep_tx_high_queue)
+ hq = fops->page_num_hi;
+ if (priv->ep_tx_low_queue)
+ lq = fops->page_num_lo;
+ if (priv->ep_tx_normal_queue)
+ nq = fops->page_num_norm;
+
+ val32 = (nq << RQPN_NPQ_SHIFT) | (eq << RQPN_EPQ_SHIFT);
+ rtl8xxxu_write32(priv, REG_RQPN_NPQ, val32);
+
+ pubq = fops->total_page_num - hq - lq - nq;
+
+ val32 = RQPN_LOAD;
+ val32 |= (hq << RQPN_HI_PQ_SHIFT);
+ val32 |= (lq << RQPN_LO_PQ_SHIFT);
+ val32 |= (pubq << RQPN_PUB_PQ_SHIFT);
+
+ rtl8xxxu_write32(priv, REG_RQPN, val32);
+}
+
+static int rtl8xxxu_init_device(struct ieee80211_hw *hw)
+{
+ struct rtl8xxxu_priv *priv = hw->priv;
+ struct device *dev = &priv->udev->dev;
+ bool macpower;
+ int ret;
+ u8 val8;
+ u16 val16;
+ u32 val32;
+
+ /* Check if MAC is already powered on */
+ val8 = rtl8xxxu_read8(priv, REG_CR);
+
+ /*
+ * Fix 92DU-VC S3 hang with the reason is that secondary mac is not
+ * initialized. First MAC returns 0xea, second MAC returns 0x00
+ */
+ if (val8 == 0xea)
+ macpower = false;
+ else
+ macpower = true;
+
+ ret = priv->fops->power_on(priv);
+ if (ret < 0) {
+ dev_warn(dev, "%s: Failed power on\n", __func__);
+ goto exit;
+ }
+
+ if (!macpower) {
+ if (priv->fops->total_page_num)
+ rtl8xxxu_init_queue_reserved_page(priv);
+ else
+ rtl8xxxu_old_init_queue_reserved_page(priv);
+ }
+
+ ret = rtl8xxxu_init_queue_priority(priv);
+ dev_dbg(dev, "%s: init_queue_priority %i\n", __func__, ret);
+ if (ret)
+ goto exit;
+
+ /*
+ * Set RX page boundary
+ */
+ rtl8xxxu_write16(priv, REG_TRXFF_BNDY + 2, priv->fops->trxff_boundary);
+
+ ret = rtl8xxxu_download_firmware(priv);
+ dev_dbg(dev, "%s: download_fiwmare %i\n", __func__, ret);
+ if (ret)
+ goto exit;
+ ret = rtl8xxxu_start_firmware(priv);
+ dev_dbg(dev, "%s: start_fiwmare %i\n", __func__, ret);
+ if (ret)
+ goto exit;
+
+ if (priv->fops->phy_init_antenna_selection)
+ priv->fops->phy_init_antenna_selection(priv);
+
+ ret = rtl8xxxu_init_mac(priv);
+
+ dev_dbg(dev, "%s: init_mac %i\n", __func__, ret);
+ if (ret)
+ goto exit;
+
+ ret = rtl8xxxu_init_phy_bb(priv);
+ dev_dbg(dev, "%s: init_phy_bb %i\n", __func__, ret);
+ if (ret)
+ goto exit;
+
+ ret = priv->fops->init_phy_rf(priv);
+ if (ret)
+ goto exit;
+
+ /* RFSW Control - clear bit 14 ?? */
+ if (priv->rtl_chip != RTL8723B && priv->rtl_chip != RTL8192E)
+ rtl8xxxu_write32(priv, REG_FPGA0_TX_INFO, 0x00000003);
+
+ val32 = FPGA0_RF_TRSW | FPGA0_RF_TRSWB | FPGA0_RF_ANTSW |
+ FPGA0_RF_ANTSWB |
+ ((FPGA0_RF_ANTSW | FPGA0_RF_ANTSWB) << FPGA0_RF_BD_CTRL_SHIFT);
+ if (!priv->no_pape) {
+ val32 |= (FPGA0_RF_PAPE |
+ (FPGA0_RF_PAPE << FPGA0_RF_BD_CTRL_SHIFT));
+ }
+ rtl8xxxu_write32(priv, REG_FPGA0_XAB_RF_SW_CTRL, val32);
+
+ /* 0x860[6:5]= 00 - why? - this sets antenna B */
+ if (priv->rtl_chip != RTL8192E)
+ rtl8xxxu_write32(priv, REG_FPGA0_XA_RF_INT_OE, 0x66f60210);
+
+ if (!macpower) {
+ /*
+ * Set TX buffer boundary
+ */
+ if (priv->rtl_chip == RTL8192E)
+ val8 = TX_TOTAL_PAGE_NUM_8192E + 1;
+ else
+ val8 = TX_TOTAL_PAGE_NUM + 1;
+
+ if (priv->rtl_chip == RTL8723B)
+ val8 -= 1;
+
+ rtl8xxxu_write8(priv, REG_TXPKTBUF_BCNQ_BDNY, val8);
+ rtl8xxxu_write8(priv, REG_TXPKTBUF_MGQ_BDNY, val8);
+ rtl8xxxu_write8(priv, REG_TXPKTBUF_WMAC_LBK_BF_HD, val8);
+ rtl8xxxu_write8(priv, REG_TRXFF_BNDY, val8);
+ rtl8xxxu_write8(priv, REG_TDECTRL + 1, val8);
+ }
+
+ /*
+ * The vendor drivers set PBP for all devices, except 8192e.
+ * There is no explanation for this in any of the sources.
+ */
+ val8 = (priv->fops->pbp_rx << PBP_PAGE_SIZE_RX_SHIFT) |
+ (priv->fops->pbp_tx << PBP_PAGE_SIZE_TX_SHIFT);
+ if (priv->rtl_chip != RTL8192E)
+ rtl8xxxu_write8(priv, REG_PBP, val8);
+
+ dev_dbg(dev, "%s: macpower %i\n", __func__, macpower);
+ if (!macpower) {
+ ret = priv->fops->llt_init(priv, TX_TOTAL_PAGE_NUM);
+ if (ret) {
+ dev_warn(dev, "%s: LLT table init failed\n", __func__);
+ goto exit;
+ }
+
+ /*
+ * Chip specific quirks
+ */
+ priv->fops->usb_quirks(priv);
+
+ /*
+ * Presumably this is for 8188EU as well
+ * Enable TX report and TX report timer
+ */
+ if (priv->rtl_chip == RTL8723B) {
+ val8 = rtl8xxxu_read8(priv, REG_TX_REPORT_CTRL);
+ val8 |= TX_REPORT_CTRL_TIMER_ENABLE;
+ rtl8xxxu_write8(priv, REG_TX_REPORT_CTRL, val8);
+ /* Set MAX RPT MACID */
+ rtl8xxxu_write8(priv, REG_TX_REPORT_CTRL + 1, 0x02);
+ /* TX report Timer. Unit: 32us */
+ rtl8xxxu_write16(priv, REG_TX_REPORT_TIME, 0xcdf0);
+
+ /* tmp ps ? */
+ val8 = rtl8xxxu_read8(priv, 0xa3);
+ val8 &= 0xf8;
+ rtl8xxxu_write8(priv, 0xa3, val8);
+ }
+ }
+
+ /*
+ * Unit in 8 bytes, not obvious what it is used for
+ */
+ rtl8xxxu_write8(priv, REG_RX_DRVINFO_SZ, 4);
+
+ if (priv->rtl_chip == RTL8192E) {
+ rtl8xxxu_write32(priv, REG_HIMR0, 0x00);
+ rtl8xxxu_write32(priv, REG_HIMR1, 0x00);
+ } else {
+ /*
+ * Enable all interrupts - not obvious USB needs to do this
+ */
+ rtl8xxxu_write32(priv, REG_HISR, 0xffffffff);
+ rtl8xxxu_write32(priv, REG_HIMR, 0xffffffff);
+ }
+
+ rtl8xxxu_set_mac(priv);
+ rtl8xxxu_set_linktype(priv, NL80211_IFTYPE_STATION);
+
+ /*
+ * Configure initial WMAC settings
+ */
+ val32 = RCR_ACCEPT_PHYS_MATCH | RCR_ACCEPT_MCAST | RCR_ACCEPT_BCAST |
+ RCR_ACCEPT_MGMT_FRAME | RCR_HTC_LOC_CTRL |
+ RCR_APPEND_PHYSTAT | RCR_APPEND_ICV | RCR_APPEND_MIC;
+ rtl8xxxu_write32(priv, REG_RCR, val32);
+
+ /*
+ * Accept all multicast
+ */
+ rtl8xxxu_write32(priv, REG_MAR, 0xffffffff);
+ rtl8xxxu_write32(priv, REG_MAR + 4, 0xffffffff);
+
+ /*
+ * Init adaptive controls
+ */
+ val32 = rtl8xxxu_read32(priv, REG_RESPONSE_RATE_SET);
+ val32 &= ~RESPONSE_RATE_BITMAP_ALL;
+ val32 |= RESPONSE_RATE_RRSR_CCK_ONLY_1M;
+ rtl8xxxu_write32(priv, REG_RESPONSE_RATE_SET, val32);
+
+ /* CCK = 0x0a, OFDM = 0x10 */
+ rtl8xxxu_set_spec_sifs(priv, 0x10, 0x10);
+ rtl8xxxu_set_retry(priv, 0x30, 0x30);
+ rtl8xxxu_set_spec_sifs(priv, 0x0a, 0x10);
+
+ /*
+ * Init EDCA
+ */
+ rtl8xxxu_write16(priv, REG_MAC_SPEC_SIFS, 0x100a);
+
+ /* Set CCK SIFS */
+ rtl8xxxu_write16(priv, REG_SIFS_CCK, 0x100a);
+
+ /* Set OFDM SIFS */
+ rtl8xxxu_write16(priv, REG_SIFS_OFDM, 0x100a);
+
+ /* TXOP */
+ rtl8xxxu_write32(priv, REG_EDCA_BE_PARAM, 0x005ea42b);
+ rtl8xxxu_write32(priv, REG_EDCA_BK_PARAM, 0x0000a44f);
+ rtl8xxxu_write32(priv, REG_EDCA_VI_PARAM, 0x005ea324);
+ rtl8xxxu_write32(priv, REG_EDCA_VO_PARAM, 0x002fa226);
+
+ /* Set data auto rate fallback retry count */
+ rtl8xxxu_write32(priv, REG_DARFRC, 0x00000000);
+ rtl8xxxu_write32(priv, REG_DARFRC + 4, 0x10080404);
+ rtl8xxxu_write32(priv, REG_RARFRC, 0x04030201);
+ rtl8xxxu_write32(priv, REG_RARFRC + 4, 0x08070605);
+
+ val8 = rtl8xxxu_read8(priv, REG_FWHW_TXQ_CTRL);
+ val8 |= FWHW_TXQ_CTRL_AMPDU_RETRY;
+ rtl8xxxu_write8(priv, REG_FWHW_TXQ_CTRL, val8);
+
+ /* Set ACK timeout */
+ rtl8xxxu_write8(priv, REG_ACKTO, 0x40);
+
+ /*
+ * Initialize beacon parameters
+ */
+ val16 = BEACON_DISABLE_TSF_UPDATE | (BEACON_DISABLE_TSF_UPDATE << 8);
+ rtl8xxxu_write16(priv, REG_BEACON_CTRL, val16);
+ rtl8xxxu_write16(priv, REG_TBTT_PROHIBIT, 0x6404);
+ rtl8xxxu_write8(priv, REG_DRIVER_EARLY_INT, DRIVER_EARLY_INT_TIME);
+ rtl8xxxu_write8(priv, REG_BEACON_DMA_TIME, BEACON_DMA_ATIME_INT_TIME);
+ rtl8xxxu_write16(priv, REG_BEACON_TCFG, 0x660F);
+
+ /*
+ * Initialize burst parameters
+ */
+ if (priv->rtl_chip == RTL8723B) {
+ /*
+ * For USB high speed set 512B packets
+ */
+ val8 = rtl8xxxu_read8(priv, REG_RXDMA_PRO_8723B);
+ val8 &= ~(BIT(4) | BIT(5));
+ val8 |= BIT(4);
+ val8 |= BIT(1) | BIT(2) | BIT(3);
+ rtl8xxxu_write8(priv, REG_RXDMA_PRO_8723B, val8);
+
+ /*
+ * For USB high speed set 512B packets
+ */
+ val8 = rtl8xxxu_read8(priv, REG_HT_SINGLE_AMPDU_8723B);
+ val8 |= BIT(7);
+ rtl8xxxu_write8(priv, REG_HT_SINGLE_AMPDU_8723B, val8);
+
+ rtl8xxxu_write16(priv, REG_MAX_AGGR_NUM, 0x0c14);
+ rtl8xxxu_write8(priv, REG_AMPDU_MAX_TIME_8723B, 0x5e);
+ rtl8xxxu_write32(priv, REG_AGGLEN_LMT, 0xffffffff);
+ rtl8xxxu_write8(priv, REG_RX_PKT_LIMIT, 0x18);
+ rtl8xxxu_write8(priv, REG_PIFS, 0x00);
+ rtl8xxxu_write8(priv, REG_USTIME_TSF_8723B, 0x50);
+ rtl8xxxu_write8(priv, REG_USTIME_EDCA, 0x50);
+
+ val8 = rtl8xxxu_read8(priv, REG_RSV_CTRL);
+ val8 |= BIT(5) | BIT(6);
+ rtl8xxxu_write8(priv, REG_RSV_CTRL, val8);
+ }
+
+ if (priv->fops->init_aggregation)
+ priv->fops->init_aggregation(priv);
+
+ /*
+ * Enable CCK and OFDM block
+ */
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_RF_MODE);
+ val32 |= (FPGA_RF_MODE_CCK | FPGA_RF_MODE_OFDM);
+ rtl8xxxu_write32(priv, REG_FPGA0_RF_MODE, val32);
+
+ /*
+ * Invalidate all CAM entries - bit 30 is undocumented
+ */
+ rtl8xxxu_write32(priv, REG_CAM_CMD, CAM_CMD_POLLING | BIT(30));
+
+ /*
+ * Start out with default power levels for channel 6, 20MHz
+ */
+ priv->fops->set_tx_power(priv, 1, false);
+
+ /* Let the 8051 take control of antenna setting */
+ if (priv->rtl_chip != RTL8192E) {
+ val8 = rtl8xxxu_read8(priv, REG_LEDCFG2);
+ val8 |= LEDCFG2_DPDT_SELECT;
+ rtl8xxxu_write8(priv, REG_LEDCFG2, val8);
+ }
+
+ rtl8xxxu_write8(priv, REG_HWSEQ_CTRL, 0xff);
+
+ /* Disable BAR - not sure if this has any effect on USB */
+ rtl8xxxu_write32(priv, REG_BAR_MODE_CTRL, 0x0201ffff);
+
+ rtl8xxxu_write16(priv, REG_FAST_EDCA_CTRL, 0);
+
+ if (priv->fops->init_statistics)
+ priv->fops->init_statistics(priv);
+
+ if (priv->rtl_chip == RTL8192E) {
+ /*
+ * 0x4c6[3] 1: RTS BW = Data BW
+ * 0: RTS BW depends on CCA / secondary CCA result.
+ */
+ val8 = rtl8xxxu_read8(priv, REG_QUEUE_CTRL);
+ val8 &= ~BIT(3);
+ rtl8xxxu_write8(priv, REG_QUEUE_CTRL, val8);
+ /*
+ * Reset USB mode switch setting
+ */
+ rtl8xxxu_write8(priv, REG_ACLK_MON, 0x00);
+ }
+
+ rtl8723a_phy_lc_calibrate(priv);
+
+ priv->fops->phy_iq_calibrate(priv);
+
+ /*
+ * This should enable thermal meter
+ */
+ if (priv->fops->tx_desc_size == sizeof(struct rtl8xxxu_txdesc40))
+ rtl8xxxu_write_rfreg(priv,
+ RF_A, RF6052_REG_T_METER_8723B, 0x37cf8);
+ else
+ rtl8xxxu_write_rfreg(priv, RF_A, RF6052_REG_T_METER, 0x60);
+
+ /* Set NAV_UPPER to 30000us */
+ val8 = ((30000 + NAV_UPPER_UNIT - 1) / NAV_UPPER_UNIT);
+ rtl8xxxu_write8(priv, REG_NAV_UPPER, val8);
+
+ if (priv->rtl_chip == RTL8723A) {
+ /*
+ * 2011/03/09 MH debug only, UMC-B cut pass 2500 S5 test,
+ * but we need to find root cause.
+ * This is 8723au only.
+ */
+ val32 = rtl8xxxu_read32(priv, REG_FPGA0_RF_MODE);
+ if ((val32 & 0xff000000) != 0x83000000) {
+ val32 |= FPGA_RF_MODE_CCK;
+ rtl8xxxu_write32(priv, REG_FPGA0_RF_MODE, val32);
+ }
+ } else if (priv->rtl_chip == RTL8192E) {
+ rtl8xxxu_write8(priv, REG_USB_HRPWM, 0x00);
+ }
+
+ val32 = rtl8xxxu_read32(priv, REG_FWHW_TXQ_CTRL);
+ val32 |= FWHW_TXQ_CTRL_XMIT_MGMT_ACK;
+ /* ack for xmit mgmt frames. */
+ rtl8xxxu_write32(priv, REG_FWHW_TXQ_CTRL, val32);
+
+ if (priv->rtl_chip == RTL8192E) {
+ /*
+ * Fix LDPC rx hang issue.
+ */
+ val32 = rtl8xxxu_read32(priv, REG_AFE_MISC);
+ rtl8xxxu_write8(priv, REG_8192E_LDOV12_CTRL, 0x75);
+ val32 &= 0xfff00fff;
+ val32 |= 0x0007e000;
+ rtl8xxxu_write32(priv, REG_AFE_MISC, val32);
+ }
+exit:
+ return ret;
+}
+
+static void rtl8xxxu_cam_write(struct rtl8xxxu_priv *priv,
+ struct ieee80211_key_conf *key, const u8 *mac)
+{
+ u32 cmd, val32, addr, ctrl;
+ int j, i, tmp_debug;
+
+ tmp_debug = rtl8xxxu_debug;
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_KEY)
+ rtl8xxxu_debug |= RTL8XXXU_DEBUG_REG_WRITE;
+
+ /*
+ * This is a bit of a hack - the lower bits of the cipher
+ * suite selector happens to match the cipher index in the CAM
+ */
+ addr = key->keyidx << CAM_CMD_KEY_SHIFT;
+ ctrl = (key->cipher & 0x0f) << 2 | key->keyidx | CAM_WRITE_VALID;
+
+ for (j = 5; j >= 0; j--) {
+ switch (j) {
+ case 0:
+ val32 = ctrl | (mac[0] << 16) | (mac[1] << 24);
+ break;
+ case 1:
+ val32 = mac[2] | (mac[3] << 8) |
+ (mac[4] << 16) | (mac[5] << 24);
+ break;
+ default:
+ i = (j - 2) << 2;
+ val32 = key->key[i] | (key->key[i + 1] << 8) |
+ key->key[i + 2] << 16 | key->key[i + 3] << 24;
+ break;
+ }
+
+ rtl8xxxu_write32(priv, REG_CAM_WRITE, val32);
+ cmd = CAM_CMD_POLLING | CAM_CMD_WRITE | (addr + j);
+ rtl8xxxu_write32(priv, REG_CAM_CMD, cmd);
+ udelay(100);
+ }
+
+ rtl8xxxu_debug = tmp_debug;
+}
+
+static void rtl8xxxu_sw_scan_start(struct ieee80211_hw *hw,
+ struct ieee80211_vif *vif, const u8 *mac)
+{
+ struct rtl8xxxu_priv *priv = hw->priv;
+ u8 val8;
+
+ val8 = rtl8xxxu_read8(priv, REG_BEACON_CTRL);
+ val8 |= BEACON_DISABLE_TSF_UPDATE;
+ rtl8xxxu_write8(priv, REG_BEACON_CTRL, val8);
+}
+
+static void rtl8xxxu_sw_scan_complete(struct ieee80211_hw *hw,
+ struct ieee80211_vif *vif)
+{
+ struct rtl8xxxu_priv *priv = hw->priv;
+ u8 val8;
+
+ val8 = rtl8xxxu_read8(priv, REG_BEACON_CTRL);
+ val8 &= ~BEACON_DISABLE_TSF_UPDATE;
+ rtl8xxxu_write8(priv, REG_BEACON_CTRL, val8);
+}
+
+void rtl8xxxu_update_rate_mask(struct rtl8xxxu_priv *priv, u32 ramask, int sgi)
+{
+ struct h2c_cmd h2c;
+
+ memset(&h2c, 0, sizeof(struct h2c_cmd));
+
+ h2c.ramask.cmd = H2C_SET_RATE_MASK;
+ h2c.ramask.mask_lo = cpu_to_le16(ramask & 0xffff);
+ h2c.ramask.mask_hi = cpu_to_le16(ramask >> 16);
+
+ h2c.ramask.arg = 0x80;
+ if (sgi)
+ h2c.ramask.arg |= 0x20;
+
+ dev_dbg(&priv->udev->dev, "%s: rate mask %08x, arg %02x, size %zi\n",
+ __func__, ramask, h2c.ramask.arg, sizeof(h2c.ramask));
+ rtl8xxxu_gen1_h2c_cmd(priv, &h2c, sizeof(h2c.ramask));
+}
+
+void rtl8xxxu_gen2_update_rate_mask(struct rtl8xxxu_priv *priv,
+ u32 ramask, int sgi)
+{
+ struct h2c_cmd h2c;
+ u8 bw = 0;
+
+ memset(&h2c, 0, sizeof(struct h2c_cmd));
+
+ h2c.b_macid_cfg.cmd = H2C_8723B_MACID_CFG_RAID;
+ h2c.b_macid_cfg.ramask0 = ramask & 0xff;
+ h2c.b_macid_cfg.ramask1 = (ramask >> 8) & 0xff;
+ h2c.b_macid_cfg.ramask2 = (ramask >> 16) & 0xff;
+ h2c.b_macid_cfg.ramask3 = (ramask >> 24) & 0xff;
+
+ h2c.ramask.arg = 0x80;
+ h2c.b_macid_cfg.data1 = 0;
+ if (sgi)
+ h2c.b_macid_cfg.data1 |= BIT(7);
+
+ h2c.b_macid_cfg.data2 = bw;
+
+ dev_dbg(&priv->udev->dev, "%s: rate mask %08x, arg %02x, size %zi\n",
+ __func__, ramask, h2c.ramask.arg, sizeof(h2c.b_macid_cfg));
+ rtl8xxxu_gen2_h2c_cmd(priv, &h2c, sizeof(h2c.b_macid_cfg));
+}
+
+void rtl8xxxu_gen1_report_connect(struct rtl8xxxu_priv *priv,
+ u8 macid, bool connect)
+{
+ struct h2c_cmd h2c;
+
+ memset(&h2c, 0, sizeof(struct h2c_cmd));
+
+ h2c.joinbss.cmd = H2C_JOIN_BSS_REPORT;
+
+ if (connect)
+ h2c.joinbss.data = H2C_JOIN_BSS_CONNECT;
+ else
+ h2c.joinbss.data = H2C_JOIN_BSS_DISCONNECT;
+
+ rtl8xxxu_gen1_h2c_cmd(priv, &h2c, sizeof(h2c.joinbss));
+}
+
+void rtl8xxxu_gen2_report_connect(struct rtl8xxxu_priv *priv,
+ u8 macid, bool connect)
+{
+ struct h2c_cmd h2c;
+
+ memset(&h2c, 0, sizeof(struct h2c_cmd));
+
+ h2c.media_status_rpt.cmd = H2C_8723B_MEDIA_STATUS_RPT;
+ if (connect)
+ h2c.media_status_rpt.parm |= BIT(0);
+ else
+ h2c.media_status_rpt.parm &= ~BIT(0);
+
+ rtl8xxxu_gen2_h2c_cmd(priv, &h2c, sizeof(h2c.media_status_rpt));
+}
+
+static void rtl8xxxu_set_basic_rates(struct rtl8xxxu_priv *priv, u32 rate_cfg)
+{
+ u32 val32;
+ u8 rate_idx = 0;
+
+ rate_cfg &= RESPONSE_RATE_BITMAP_ALL;
+
+ val32 = rtl8xxxu_read32(priv, REG_RESPONSE_RATE_SET);
+ val32 &= ~RESPONSE_RATE_BITMAP_ALL;
+ val32 |= rate_cfg;
+ rtl8xxxu_write32(priv, REG_RESPONSE_RATE_SET, val32);
+
+ dev_dbg(&priv->udev->dev, "%s: rates %08x\n", __func__, rate_cfg);
+
+ while (rate_cfg) {
+ rate_cfg = (rate_cfg >> 1);
+ rate_idx++;
+ }
+ rtl8xxxu_write8(priv, REG_INIRTS_RATE_SEL, rate_idx);
+}
+
+static void
+rtl8xxxu_bss_info_changed(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
+ struct ieee80211_bss_conf *bss_conf, u32 changed)
+{
+ struct rtl8xxxu_priv *priv = hw->priv;
+ struct device *dev = &priv->udev->dev;
+ struct ieee80211_sta *sta;
+ u32 val32;
+ u8 val8;
+
+ if (changed & BSS_CHANGED_ASSOC) {
+ dev_dbg(dev, "Changed ASSOC: %i!\n", bss_conf->assoc);
+
+ rtl8xxxu_set_linktype(priv, vif->type);
+
+ if (bss_conf->assoc) {
+ u32 ramask;
+ int sgi = 0;
+
+ rcu_read_lock();
+ sta = ieee80211_find_sta(vif, bss_conf->bssid);
+ if (!sta) {
+ dev_info(dev, "%s: ASSOC no sta found\n",
+ __func__);
+ rcu_read_unlock();
+ goto error;
+ }
+
+ if (sta->ht_cap.ht_supported)
+ dev_info(dev, "%s: HT supported\n", __func__);
+ if (sta->vht_cap.vht_supported)
+ dev_info(dev, "%s: VHT supported\n", __func__);
+
+ /* TODO: Set bits 28-31 for rate adaptive id */
+ ramask = (sta->supp_rates[0] & 0xfff) |
+ sta->ht_cap.mcs.rx_mask[0] << 12 |
+ sta->ht_cap.mcs.rx_mask[1] << 20;
+ if (sta->ht_cap.cap &
+ (IEEE80211_HT_CAP_SGI_40 | IEEE80211_HT_CAP_SGI_20))
+ sgi = 1;
+ rcu_read_unlock();
+
+ priv->fops->update_rate_mask(priv, ramask, sgi);
+
+ rtl8xxxu_write8(priv, REG_BCN_MAX_ERR, 0xff);
+
+ rtl8xxxu_stop_tx_beacon(priv);
+
+ /* joinbss sequence */
+ rtl8xxxu_write16(priv, REG_BCN_PSR_RPT,
+ 0xc000 | bss_conf->aid);
+
+ priv->fops->report_connect(priv, 0, true);
+ } else {
+ val8 = rtl8xxxu_read8(priv, REG_BEACON_CTRL);
+ val8 |= BEACON_DISABLE_TSF_UPDATE;
+ rtl8xxxu_write8(priv, REG_BEACON_CTRL, val8);
+
+ priv->fops->report_connect(priv, 0, false);
+ }
+ }
+
+ if (changed & BSS_CHANGED_ERP_PREAMBLE) {
+ dev_dbg(dev, "Changed ERP_PREAMBLE: Use short preamble %i\n",
+ bss_conf->use_short_preamble);
+ val32 = rtl8xxxu_read32(priv, REG_RESPONSE_RATE_SET);
+ if (bss_conf->use_short_preamble)
+ val32 |= RSR_ACK_SHORT_PREAMBLE;
+ else
+ val32 &= ~RSR_ACK_SHORT_PREAMBLE;
+ rtl8xxxu_write32(priv, REG_RESPONSE_RATE_SET, val32);
+ }
+
+ if (changed & BSS_CHANGED_ERP_SLOT) {
+ dev_dbg(dev, "Changed ERP_SLOT: short_slot_time %i\n",
+ bss_conf->use_short_slot);
+
+ if (bss_conf->use_short_slot)
+ val8 = 9;
+ else
+ val8 = 20;
+ rtl8xxxu_write8(priv, REG_SLOT, val8);
+ }
+
+ if (changed & BSS_CHANGED_BSSID) {
+ dev_dbg(dev, "Changed BSSID!\n");
+ rtl8xxxu_set_bssid(priv, bss_conf->bssid);
+ }
+
+ if (changed & BSS_CHANGED_BASIC_RATES) {
+ dev_dbg(dev, "Changed BASIC_RATES!\n");
+ rtl8xxxu_set_basic_rates(priv, bss_conf->basic_rates);
+ }
+error:
+ return;
+}
+
+static u32 rtl8xxxu_80211_to_rtl_queue(u32 queue)
+{
+ u32 rtlqueue;
+
+ switch (queue) {
+ case IEEE80211_AC_VO:
+ rtlqueue = TXDESC_QUEUE_VO;
+ break;
+ case IEEE80211_AC_VI:
+ rtlqueue = TXDESC_QUEUE_VI;
+ break;
+ case IEEE80211_AC_BE:
+ rtlqueue = TXDESC_QUEUE_BE;
+ break;
+ case IEEE80211_AC_BK:
+ rtlqueue = TXDESC_QUEUE_BK;
+ break;
+ default:
+ rtlqueue = TXDESC_QUEUE_BE;
+ }
+
+ return rtlqueue;
+}
+
+static u32 rtl8xxxu_queue_select(struct ieee80211_hw *hw, struct sk_buff *skb)
+{
+ struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
+ u32 queue;
+
+ if (ieee80211_is_mgmt(hdr->frame_control))
+ queue = TXDESC_QUEUE_MGNT;
+ else
+ queue = rtl8xxxu_80211_to_rtl_queue(skb_get_queue_mapping(skb));
+
+ return queue;
+}
+
+/*
+ * Despite newer chips 8723b/8812/8821 having a larger TX descriptor
+ * format. The descriptor checksum is still only calculated over the
+ * initial 32 bytes of the descriptor!
+ */
+static void rtl8xxxu_calc_tx_desc_csum(struct rtl8xxxu_txdesc32 *tx_desc)
+{
+ __le16 *ptr = (__le16 *)tx_desc;
+ u16 csum = 0;
+ int i;
+
+ /*
+ * Clear csum field before calculation, as the csum field is
+ * in the middle of the struct.
+ */
+ tx_desc->csum = cpu_to_le16(0);
+
+ for (i = 0; i < (sizeof(struct rtl8xxxu_txdesc32) / sizeof(u16)); i++)
+ csum = csum ^ le16_to_cpu(ptr[i]);
+
+ tx_desc->csum |= cpu_to_le16(csum);
+}
+
+static void rtl8xxxu_free_tx_resources(struct rtl8xxxu_priv *priv)
+{
+ struct rtl8xxxu_tx_urb *tx_urb, *tmp;
+ unsigned long flags;
+
+ spin_lock_irqsave(&priv->tx_urb_lock, flags);
+ list_for_each_entry_safe(tx_urb, tmp, &priv->tx_urb_free_list, list) {
+ list_del(&tx_urb->list);
+ priv->tx_urb_free_count--;
+ usb_free_urb(&tx_urb->urb);
+ }
+ spin_unlock_irqrestore(&priv->tx_urb_lock, flags);
+}
+
+static struct rtl8xxxu_tx_urb *
+rtl8xxxu_alloc_tx_urb(struct rtl8xxxu_priv *priv)
+{
+ struct rtl8xxxu_tx_urb *tx_urb;
+ unsigned long flags;
+
+ spin_lock_irqsave(&priv->tx_urb_lock, flags);
+ tx_urb = list_first_entry_or_null(&priv->tx_urb_free_list,
+ struct rtl8xxxu_tx_urb, list);
+ if (tx_urb) {
+ list_del(&tx_urb->list);
+ priv->tx_urb_free_count--;
+ if (priv->tx_urb_free_count < RTL8XXXU_TX_URB_LOW_WATER &&
+ !priv->tx_stopped) {
+ priv->tx_stopped = true;
+ ieee80211_stop_queues(priv->hw);
+ }
+ }
+
+ spin_unlock_irqrestore(&priv->tx_urb_lock, flags);
+
+ return tx_urb;
+}
+
+static void rtl8xxxu_free_tx_urb(struct rtl8xxxu_priv *priv,
+ struct rtl8xxxu_tx_urb *tx_urb)
+{
+ unsigned long flags;
+
+ INIT_LIST_HEAD(&tx_urb->list);
+
+ spin_lock_irqsave(&priv->tx_urb_lock, flags);
+
+ list_add(&tx_urb->list, &priv->tx_urb_free_list);
+ priv->tx_urb_free_count++;
+ if (priv->tx_urb_free_count > RTL8XXXU_TX_URB_HIGH_WATER &&
+ priv->tx_stopped) {
+ priv->tx_stopped = false;
+ ieee80211_wake_queues(priv->hw);
+ }
+
+ spin_unlock_irqrestore(&priv->tx_urb_lock, flags);
+}
+
+static void rtl8xxxu_tx_complete(struct urb *urb)
+{
+ struct sk_buff *skb = (struct sk_buff *)urb->context;
+ struct ieee80211_tx_info *tx_info;
+ struct ieee80211_hw *hw;
+ struct rtl8xxxu_priv *priv;
+ struct rtl8xxxu_tx_urb *tx_urb =
+ container_of(urb, struct rtl8xxxu_tx_urb, urb);
+
+ tx_info = IEEE80211_SKB_CB(skb);
+ hw = tx_info->rate_driver_data[0];
+ priv = hw->priv;
+
+ skb_pull(skb, priv->fops->tx_desc_size);
+
+ ieee80211_tx_info_clear_status(tx_info);
+ tx_info->status.rates[0].idx = -1;
+ tx_info->status.rates[0].count = 0;
+
+ if (!urb->status)
+ tx_info->flags |= IEEE80211_TX_STAT_ACK;
+
+ ieee80211_tx_status_irqsafe(hw, skb);
+
+ rtl8xxxu_free_tx_urb(priv, tx_urb);
+}
+
+static void rtl8xxxu_dump_action(struct device *dev,
+ struct ieee80211_hdr *hdr)
+{
+ struct ieee80211_mgmt *mgmt = (struct ieee80211_mgmt *)hdr;
+ u16 cap, timeout;
+
+ if (!(rtl8xxxu_debug & RTL8XXXU_DEBUG_ACTION))
+ return;
+
+ switch (mgmt->u.action.u.addba_resp.action_code) {
+ case WLAN_ACTION_ADDBA_RESP:
+ cap = le16_to_cpu(mgmt->u.action.u.addba_resp.capab);
+ timeout = le16_to_cpu(mgmt->u.action.u.addba_resp.timeout);
+ dev_info(dev, "WLAN_ACTION_ADDBA_RESP: "
+ "timeout %i, tid %02x, buf_size %02x, policy %02x, "
+ "status %02x\n",
+ timeout,
+ (cap & IEEE80211_ADDBA_PARAM_TID_MASK) >> 2,
+ (cap & IEEE80211_ADDBA_PARAM_BUF_SIZE_MASK) >> 6,
+ (cap >> 1) & 0x1,
+ le16_to_cpu(mgmt->u.action.u.addba_resp.status));
+ break;
+ case WLAN_ACTION_ADDBA_REQ:
+ cap = le16_to_cpu(mgmt->u.action.u.addba_req.capab);
+ timeout = le16_to_cpu(mgmt->u.action.u.addba_req.timeout);
+ dev_info(dev, "WLAN_ACTION_ADDBA_REQ: "
+ "timeout %i, tid %02x, buf_size %02x, policy %02x\n",
+ timeout,
+ (cap & IEEE80211_ADDBA_PARAM_TID_MASK) >> 2,
+ (cap & IEEE80211_ADDBA_PARAM_BUF_SIZE_MASK) >> 6,
+ (cap >> 1) & 0x1);
+ break;
+ default:
+ dev_info(dev, "action frame %02x\n",
+ mgmt->u.action.u.addba_resp.action_code);
+ break;
+ }
+}
+
+static void rtl8xxxu_tx(struct ieee80211_hw *hw,
+ struct ieee80211_tx_control *control,
+ struct sk_buff *skb)
+{
+ struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
+ struct ieee80211_tx_info *tx_info = IEEE80211_SKB_CB(skb);
+ struct ieee80211_rate *tx_rate = ieee80211_get_tx_rate(hw, tx_info);
+ struct rtl8xxxu_priv *priv = hw->priv;
+ struct rtl8xxxu_txdesc32 *tx_desc;
+ struct rtl8xxxu_txdesc40 *tx_desc40;
+ struct rtl8xxxu_tx_urb *tx_urb;
+ struct ieee80211_sta *sta = NULL;
+ struct ieee80211_vif *vif = tx_info->control.vif;
+ struct device *dev = &priv->udev->dev;
+ u32 queue, rate;
+ u16 pktlen = skb->len;
+ u16 seq_number;
+ u16 rate_flag = tx_info->control.rates[0].flags;
+ int tx_desc_size = priv->fops->tx_desc_size;
+ int ret;
+ bool usedesc40, ampdu_enable;
+
+ if (skb_headroom(skb) < tx_desc_size) {
+ dev_warn(dev,
+ "%s: Not enough headroom (%i) for tx descriptor\n",
+ __func__, skb_headroom(skb));
+ goto error;
+ }
+
+ if (unlikely(skb->len > (65535 - tx_desc_size))) {
+ dev_warn(dev, "%s: Trying to send over-sized skb (%i)\n",
+ __func__, skb->len);
+ goto error;
+ }
+
+ tx_urb = rtl8xxxu_alloc_tx_urb(priv);
+ if (!tx_urb) {
+ dev_warn(dev, "%s: Unable to allocate tx urb\n", __func__);
+ goto error;
+ }
+
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_TX)
+ dev_info(dev, "%s: TX rate: %d (%d), pkt size %d\n",
+ __func__, tx_rate->bitrate, tx_rate->hw_value, pktlen);
+
+ if (ieee80211_is_action(hdr->frame_control))
+ rtl8xxxu_dump_action(dev, hdr);
+
+ usedesc40 = (tx_desc_size == 40);
+ tx_info->rate_driver_data[0] = hw;
+
+ if (control && control->sta)
+ sta = control->sta;
+
+ tx_desc = (struct rtl8xxxu_txdesc32 *)skb_push(skb, tx_desc_size);
+
+ memset(tx_desc, 0, tx_desc_size);
+ tx_desc->pkt_size = cpu_to_le16(pktlen);
+ tx_desc->pkt_offset = tx_desc_size;
+
+ tx_desc->txdw0 =
+ TXDESC_OWN | TXDESC_FIRST_SEGMENT | TXDESC_LAST_SEGMENT;
+ if (is_multicast_ether_addr(ieee80211_get_DA(hdr)) ||
+ is_broadcast_ether_addr(ieee80211_get_DA(hdr)))
+ tx_desc->txdw0 |= TXDESC_BROADMULTICAST;
+
+ queue = rtl8xxxu_queue_select(hw, skb);
+ tx_desc->txdw1 = cpu_to_le32(queue << TXDESC_QUEUE_SHIFT);
+
+ if (tx_info->control.hw_key) {
+ switch (tx_info->control.hw_key->cipher) {
+ case WLAN_CIPHER_SUITE_WEP40:
+ case WLAN_CIPHER_SUITE_WEP104:
+ case WLAN_CIPHER_SUITE_TKIP:
+ tx_desc->txdw1 |= cpu_to_le32(TXDESC_SEC_RC4);
+ break;
+ case WLAN_CIPHER_SUITE_CCMP:
+ tx_desc->txdw1 |= cpu_to_le32(TXDESC_SEC_AES);
+ break;
+ default:
+ break;
+ }
+ }
+
+ /* (tx_info->flags & IEEE80211_TX_CTL_AMPDU) && */
+ ampdu_enable = false;
+ if (ieee80211_is_data_qos(hdr->frame_control) && sta) {
+ if (sta->ht_cap.ht_supported) {
+ u32 ampdu, val32;
+
+ ampdu = (u32)sta->ht_cap.ampdu_density;
+ val32 = ampdu << TXDESC_AMPDU_DENSITY_SHIFT;
+ tx_desc->txdw2 |= cpu_to_le32(val32);
+
+ ampdu_enable = true;
+ }
+ }
+
+ if (rate_flag & IEEE80211_TX_RC_MCS)
+ rate = tx_info->control.rates[0].idx + DESC_RATE_MCS0;
+ else
+ rate = tx_rate->hw_value;
+
+ seq_number = IEEE80211_SEQ_TO_SN(le16_to_cpu(hdr->seq_ctrl));
+ if (!usedesc40) {
+ tx_desc->txdw5 = cpu_to_le32(rate);
+
+ if (ieee80211_is_data(hdr->frame_control))
+ tx_desc->txdw5 |= cpu_to_le32(0x0001ff00);
+
+ tx_desc->txdw3 =
+ cpu_to_le32((u32)seq_number << TXDESC32_SEQ_SHIFT);
+
+ if (ampdu_enable)
+ tx_desc->txdw1 |= cpu_to_le32(TXDESC32_AGG_ENABLE);
+ else
+ tx_desc->txdw1 |= cpu_to_le32(TXDESC32_AGG_BREAK);
+
+ if (ieee80211_is_mgmt(hdr->frame_control)) {
+ tx_desc->txdw5 = cpu_to_le32(tx_rate->hw_value);
+ tx_desc->txdw4 |=
+ cpu_to_le32(TXDESC32_USE_DRIVER_RATE);
+ tx_desc->txdw5 |=
+ cpu_to_le32(6 << TXDESC32_RETRY_LIMIT_SHIFT);
+ tx_desc->txdw5 |=
+ cpu_to_le32(TXDESC32_RETRY_LIMIT_ENABLE);
+ }
+
+ if (ieee80211_is_data_qos(hdr->frame_control))
+ tx_desc->txdw4 |= cpu_to_le32(TXDESC32_QOS);
+
+ if (rate_flag & IEEE80211_TX_RC_USE_SHORT_PREAMBLE ||
+ (sta && vif && vif->bss_conf.use_short_preamble))
+ tx_desc->txdw4 |= cpu_to_le32(TXDESC32_SHORT_PREAMBLE);
+
+ if (rate_flag & IEEE80211_TX_RC_SHORT_GI ||
+ (ieee80211_is_data_qos(hdr->frame_control) &&
+ sta && sta->ht_cap.cap &
+ (IEEE80211_HT_CAP_SGI_40 | IEEE80211_HT_CAP_SGI_20))) {
+ tx_desc->txdw5 |= cpu_to_le32(TXDESC32_SHORT_GI);
+ }
+
+ if (rate_flag & IEEE80211_TX_RC_USE_RTS_CTS) {
+ /*
+ * Use RTS rate 24M - does the mac80211 tell
+ * us which to use?
+ */
+ tx_desc->txdw4 |=
+ cpu_to_le32(DESC_RATE_24M <<
+ TXDESC32_RTS_RATE_SHIFT);
+ tx_desc->txdw4 |=
+ cpu_to_le32(TXDESC32_RTS_CTS_ENABLE);
+ tx_desc->txdw4 |= cpu_to_le32(TXDESC32_HW_RTS_ENABLE);
+ }
+ } else {
+ tx_desc40 = (struct rtl8xxxu_txdesc40 *)tx_desc;
+
+ tx_desc40->txdw4 = cpu_to_le32(rate);
+ if (ieee80211_is_data(hdr->frame_control)) {
+ tx_desc->txdw4 |=
+ cpu_to_le32(0x1f <<
+ TXDESC40_DATA_RATE_FB_SHIFT);
+ }
+
+ tx_desc40->txdw9 =
+ cpu_to_le32((u32)seq_number << TXDESC40_SEQ_SHIFT);
+
+ if (ampdu_enable)
+ tx_desc40->txdw2 |= cpu_to_le32(TXDESC40_AGG_ENABLE);
+ else
+ tx_desc40->txdw2 |= cpu_to_le32(TXDESC40_AGG_BREAK);
+
+ if (ieee80211_is_mgmt(hdr->frame_control)) {
+ tx_desc40->txdw4 = cpu_to_le32(tx_rate->hw_value);
+ tx_desc40->txdw3 |=
+ cpu_to_le32(TXDESC40_USE_DRIVER_RATE);
+ tx_desc40->txdw4 |=
+ cpu_to_le32(6 << TXDESC40_RETRY_LIMIT_SHIFT);
+ tx_desc40->txdw4 |=
+ cpu_to_le32(TXDESC40_RETRY_LIMIT_ENABLE);
+ }
+
+ if (rate_flag & IEEE80211_TX_RC_USE_SHORT_PREAMBLE ||
+ (sta && vif && vif->bss_conf.use_short_preamble))
+ tx_desc40->txdw5 |=
+ cpu_to_le32(TXDESC40_SHORT_PREAMBLE);
+
+ if (rate_flag & IEEE80211_TX_RC_USE_RTS_CTS) {
+ /*
+ * Use RTS rate 24M - does the mac80211 tell
+ * us which to use?
+ */
+ tx_desc->txdw4 |=
+ cpu_to_le32(DESC_RATE_24M <<
+ TXDESC40_RTS_RATE_SHIFT);
+ tx_desc->txdw3 |= cpu_to_le32(TXDESC40_RTS_CTS_ENABLE);
+ tx_desc->txdw3 |= cpu_to_le32(TXDESC40_HW_RTS_ENABLE);
+ }
+ }
+
+ rtl8xxxu_calc_tx_desc_csum(tx_desc);
+
+ usb_fill_bulk_urb(&tx_urb->urb, priv->udev, priv->pipe_out[queue],
+ skb->data, skb->len, rtl8xxxu_tx_complete, skb);
+
+ usb_anchor_urb(&tx_urb->urb, &priv->tx_anchor);
+ ret = usb_submit_urb(&tx_urb->urb, GFP_ATOMIC);
+ if (ret) {
+ usb_unanchor_urb(&tx_urb->urb);
+ rtl8xxxu_free_tx_urb(priv, tx_urb);
+ goto error;
+ }
+ return;
+error:
+ dev_kfree_skb(skb);
+}
+
+static void rtl8xxxu_rx_parse_phystats(struct rtl8xxxu_priv *priv,
+ struct ieee80211_rx_status *rx_status,
+ struct rtl8723au_phy_stats *phy_stats,
+ u32 rxmcs)
+{
+ if (phy_stats->sgi_en)
+ rx_status->flag |= RX_FLAG_SHORT_GI;
+
+ if (rxmcs < DESC_RATE_6M) {
+ /*
+ * Handle PHY stats for CCK rates
+ */
+ u8 cck_agc_rpt = phy_stats->cck_agc_rpt_ofdm_cfosho_a;
+
+ switch (cck_agc_rpt & 0xc0) {
+ case 0xc0:
+ rx_status->signal = -46 - (cck_agc_rpt & 0x3e);
+ break;
+ case 0x80:
+ rx_status->signal = -26 - (cck_agc_rpt & 0x3e);
+ break;
+ case 0x40:
+ rx_status->signal = -12 - (cck_agc_rpt & 0x3e);
+ break;
+ case 0x00:
+ rx_status->signal = 16 - (cck_agc_rpt & 0x3e);
+ break;
+ }
+ } else {
+ rx_status->signal =
+ (phy_stats->cck_sig_qual_ofdm_pwdb_all >> 1) - 110;
+ }
+}
+
+static void rtl8xxxu_free_rx_resources(struct rtl8xxxu_priv *priv)
+{
+ struct rtl8xxxu_rx_urb *rx_urb, *tmp;
+ unsigned long flags;
+
+ spin_lock_irqsave(&priv->rx_urb_lock, flags);
+
+ list_for_each_entry_safe(rx_urb, tmp,
+ &priv->rx_urb_pending_list, list) {
+ list_del(&rx_urb->list);
+ priv->rx_urb_pending_count--;
+ usb_free_urb(&rx_urb->urb);
+ }
+
+ spin_unlock_irqrestore(&priv->rx_urb_lock, flags);
+}
+
+static void rtl8xxxu_queue_rx_urb(struct rtl8xxxu_priv *priv,
+ struct rtl8xxxu_rx_urb *rx_urb)
+{
+ struct sk_buff *skb;
+ unsigned long flags;
+ int pending = 0;
+
+ spin_lock_irqsave(&priv->rx_urb_lock, flags);
+
+ if (!priv->shutdown) {
+ list_add_tail(&rx_urb->list, &priv->rx_urb_pending_list);
+ priv->rx_urb_pending_count++;
+ pending = priv->rx_urb_pending_count;
+ } else {
+ skb = (struct sk_buff *)rx_urb->urb.context;
+ dev_kfree_skb(skb);
+ usb_free_urb(&rx_urb->urb);
+ }
+
+ spin_unlock_irqrestore(&priv->rx_urb_lock, flags);
+
+ if (pending > RTL8XXXU_RX_URB_PENDING_WATER)
+ schedule_work(&priv->rx_urb_wq);
+}
+
+static void rtl8xxxu_rx_urb_work(struct work_struct *work)
+{
+ struct rtl8xxxu_priv *priv;
+ struct rtl8xxxu_rx_urb *rx_urb, *tmp;
+ struct list_head local;
+ struct sk_buff *skb;
+ unsigned long flags;
+ int ret;
+
+ priv = container_of(work, struct rtl8xxxu_priv, rx_urb_wq);
+ INIT_LIST_HEAD(&local);
+
+ spin_lock_irqsave(&priv->rx_urb_lock, flags);
+
+ list_splice_init(&priv->rx_urb_pending_list, &local);
+ priv->rx_urb_pending_count = 0;
+
+ spin_unlock_irqrestore(&priv->rx_urb_lock, flags);
+
+ list_for_each_entry_safe(rx_urb, tmp, &local, list) {
+ list_del_init(&rx_urb->list);
+ ret = rtl8xxxu_submit_rx_urb(priv, rx_urb);
+ /*
+ * If out of memory or temporary error, put it back on the
+ * queue and try again. Otherwise the device is dead/gone
+ * and we should drop it.
+ */
+ switch (ret) {
+ case 0:
+ break;
+ case -ENOMEM:
+ case -EAGAIN:
+ rtl8xxxu_queue_rx_urb(priv, rx_urb);
+ break;
+ default:
+ pr_info("failed to requeue urb %i\n", ret);
+ skb = (struct sk_buff *)rx_urb->urb.context;
+ dev_kfree_skb(skb);
+ usb_free_urb(&rx_urb->urb);
+ }
+ }
+}
+
+int rtl8xxxu_parse_rxdesc16(struct rtl8xxxu_priv *priv, struct sk_buff *skb,
+ struct ieee80211_rx_status *rx_status)
+{
+ struct rtl8xxxu_rxdesc16 *rx_desc =
+ (struct rtl8xxxu_rxdesc16 *)skb->data;
+ struct rtl8723au_phy_stats *phy_stats;
+ __le32 *_rx_desc_le = (__le32 *)skb->data;
+ u32 *_rx_desc = (u32 *)skb->data;
+ int drvinfo_sz, desc_shift;
+ int i;
+
+ for (i = 0; i < (sizeof(struct rtl8xxxu_rxdesc16) / sizeof(u32)); i++)
+ _rx_desc[i] = le32_to_cpu(_rx_desc_le[i]);
+
+ skb_pull(skb, sizeof(struct rtl8xxxu_rxdesc16));
+
+ phy_stats = (struct rtl8723au_phy_stats *)skb->data;
+
+ drvinfo_sz = rx_desc->drvinfo_sz * 8;
+ desc_shift = rx_desc->shift;
+ skb_pull(skb, drvinfo_sz + desc_shift);
+
+ if (rx_desc->phy_stats)
+ rtl8xxxu_rx_parse_phystats(priv, rx_status, phy_stats,
+ rx_desc->rxmcs);
+
+ rx_status->mactime = le32_to_cpu(rx_desc->tsfl);
+ rx_status->flag |= RX_FLAG_MACTIME_START;
+
+ if (!rx_desc->swdec)
+ rx_status->flag |= RX_FLAG_DECRYPTED;
+ if (rx_desc->crc32)
+ rx_status->flag |= RX_FLAG_FAILED_FCS_CRC;
+ if (rx_desc->bw)
+ rx_status->flag |= RX_FLAG_40MHZ;
+
+ if (rx_desc->rxht) {
+ rx_status->flag |= RX_FLAG_HT;
+ rx_status->rate_idx = rx_desc->rxmcs - DESC_RATE_MCS0;
+ } else {
+ rx_status->rate_idx = rx_desc->rxmcs;
+ }
+
+ return RX_TYPE_DATA_PKT;
+}
+
+int rtl8xxxu_parse_rxdesc24(struct rtl8xxxu_priv *priv, struct sk_buff *skb,
+ struct ieee80211_rx_status *rx_status)
+{
+ struct rtl8xxxu_rxdesc24 *rx_desc =
+ (struct rtl8xxxu_rxdesc24 *)skb->data;
+ struct rtl8723au_phy_stats *phy_stats;
+ __le32 *_rx_desc_le = (__le32 *)skb->data;
+ u32 *_rx_desc = (u32 *)skb->data;
+ int drvinfo_sz, desc_shift;
+ int i;
+
+ for (i = 0; i < (sizeof(struct rtl8xxxu_rxdesc24) / sizeof(u32)); i++)
+ _rx_desc[i] = le32_to_cpu(_rx_desc_le[i]);
+
+ skb_pull(skb, sizeof(struct rtl8xxxu_rxdesc24));
+
+ phy_stats = (struct rtl8723au_phy_stats *)skb->data;
+
+ drvinfo_sz = rx_desc->drvinfo_sz * 8;
+ desc_shift = rx_desc->shift;
+ skb_pull(skb, drvinfo_sz + desc_shift);
+
+ if (rx_desc->rpt_sel) {
+ struct device *dev = &priv->udev->dev;
+ dev_dbg(dev, "%s: C2H packet\n", __func__);
+ return RX_TYPE_C2H;
+ }
+
+ if (rx_desc->phy_stats)
+ rtl8xxxu_rx_parse_phystats(priv, rx_status, phy_stats,
+ rx_desc->rxmcs);
+
+ rx_status->mactime = le32_to_cpu(rx_desc->tsfl);
+ rx_status->flag |= RX_FLAG_MACTIME_START;
+
+ if (!rx_desc->swdec)
+ rx_status->flag |= RX_FLAG_DECRYPTED;
+ if (rx_desc->crc32)
+ rx_status->flag |= RX_FLAG_FAILED_FCS_CRC;
+ if (rx_desc->bw)
+ rx_status->flag |= RX_FLAG_40MHZ;
+
+ if (rx_desc->rxmcs >= DESC_RATE_MCS0) {
+ rx_status->flag |= RX_FLAG_HT;
+ rx_status->rate_idx = rx_desc->rxmcs - DESC_RATE_MCS0;
+ } else {
+ rx_status->rate_idx = rx_desc->rxmcs;
+ }
+
+ return RX_TYPE_DATA_PKT;
+}
+
+static void rtl8723bu_handle_c2h(struct rtl8xxxu_priv *priv,
+ struct sk_buff *skb)
+{
+ struct rtl8723bu_c2h *c2h = (struct rtl8723bu_c2h *)skb->data;
+ struct device *dev = &priv->udev->dev;
+ int len;
+
+ len = skb->len - 2;
+
+ dev_dbg(dev, "C2H ID %02x seq %02x, len %02x source %02x\n",
+ c2h->id, c2h->seq, len, c2h->bt_info.response_source);
+
+ switch(c2h->id) {
+ case C2H_8723B_BT_INFO:
+ if (c2h->bt_info.response_source >
+ BT_INFO_SRC_8723B_BT_ACTIVE_SEND)
+ dev_dbg(dev, "C2H_BT_INFO WiFi only firmware\n");
+ else
+ dev_dbg(dev, "C2H_BT_INFO BT/WiFi coexist firmware\n");
+
+ if (c2h->bt_info.bt_has_reset)
+ dev_dbg(dev, "BT has been reset\n");
+ if (c2h->bt_info.tx_rx_mask)
+ dev_dbg(dev, "BT TRx mask\n");
+
+ break;
+ case C2H_8723B_BT_MP_INFO:
+ dev_dbg(dev, "C2H_MP_INFO ext ID %02x, status %02x\n",
+ c2h->bt_mp_info.ext_id, c2h->bt_mp_info.status);
+ break;
+ case C2H_8723B_RA_REPORT:
+ dev_dbg(dev,
+ "C2H RA RPT: rate %02x, unk %i, macid %02x, noise %i\n",
+ c2h->ra_report.rate, c2h->ra_report.dummy0_0,
+ c2h->ra_report.macid, c2h->ra_report.noisy_state);
+ break;
+ default:
+ dev_info(dev, "Unhandled C2H event %02x seq %02x\n",
+ c2h->id, c2h->seq);
+ print_hex_dump(KERN_INFO, "C2H content: ", DUMP_PREFIX_NONE,
+ 16, 1, c2h->raw.payload, len, false);
+ break;
+ }
+}
+
+static void rtl8xxxu_rx_complete(struct urb *urb)
+{
+ struct rtl8xxxu_rx_urb *rx_urb =
+ container_of(urb, struct rtl8xxxu_rx_urb, urb);
+ struct ieee80211_hw *hw = rx_urb->hw;
+ struct rtl8xxxu_priv *priv = hw->priv;
+ struct sk_buff *skb = (struct sk_buff *)urb->context;
+ struct ieee80211_rx_status *rx_status = IEEE80211_SKB_RXCB(skb);
+ struct device *dev = &priv->udev->dev;
+ int rx_type;
+
+ skb_put(skb, urb->actual_length);
+
+ if (urb->status == 0) {
+ memset(rx_status, 0, sizeof(struct ieee80211_rx_status));
+
+ rx_type = priv->fops->parse_rx_desc(priv, skb, rx_status);
+
+ rx_status->freq = hw->conf.chandef.chan->center_freq;
+ rx_status->band = hw->conf.chandef.chan->band;
+
+ if (rx_type == RX_TYPE_DATA_PKT)
+ ieee80211_rx_irqsafe(hw, skb);
+ else {
+ rtl8723bu_handle_c2h(priv, skb);
+ dev_kfree_skb(skb);
+ }
+
+ skb = NULL;
+ rx_urb->urb.context = NULL;
+ rtl8xxxu_queue_rx_urb(priv, rx_urb);
+ } else {
+ dev_dbg(dev, "%s: status %i\n", __func__, urb->status);
+ goto cleanup;
+ }
+ return;
+
+cleanup:
+ usb_free_urb(urb);
+ dev_kfree_skb(skb);
+ return;
+}
+
+static int rtl8xxxu_submit_rx_urb(struct rtl8xxxu_priv *priv,
+ struct rtl8xxxu_rx_urb *rx_urb)
+{
+ struct sk_buff *skb;
+ int skb_size;
+ int ret, rx_desc_sz;
+
+ rx_desc_sz = priv->fops->rx_desc_size;
+ skb_size = rx_desc_sz + RTL_RX_BUFFER_SIZE;
+ skb = __netdev_alloc_skb(NULL, skb_size, GFP_KERNEL);
+ if (!skb)
+ return -ENOMEM;
+
+ memset(skb->data, 0, rx_desc_sz);
+ usb_fill_bulk_urb(&rx_urb->urb, priv->udev, priv->pipe_in, skb->data,
+ skb_size, rtl8xxxu_rx_complete, skb);
+ usb_anchor_urb(&rx_urb->urb, &priv->rx_anchor);
+ ret = usb_submit_urb(&rx_urb->urb, GFP_ATOMIC);
+ if (ret)
+ usb_unanchor_urb(&rx_urb->urb);
+ return ret;
+}
+
+static void rtl8xxxu_int_complete(struct urb *urb)
+{
+ struct rtl8xxxu_priv *priv = (struct rtl8xxxu_priv *)urb->context;
+ struct device *dev = &priv->udev->dev;
+ int ret;
+
+ dev_dbg(dev, "%s: status %i\n", __func__, urb->status);
+ if (urb->status == 0) {
+ usb_anchor_urb(urb, &priv->int_anchor);
+ ret = usb_submit_urb(urb, GFP_ATOMIC);
+ if (ret)
+ usb_unanchor_urb(urb);
+ } else {
+ dev_info(dev, "%s: Error %i\n", __func__, urb->status);
+ }
+}
+
+
+static int rtl8xxxu_submit_int_urb(struct ieee80211_hw *hw)
+{
+ struct rtl8xxxu_priv *priv = hw->priv;
+ struct urb *urb;
+ u32 val32;
+ int ret;
+
+ urb = usb_alloc_urb(0, GFP_KERNEL);
+ if (!urb)
+ return -ENOMEM;
+
+ usb_fill_int_urb(urb, priv->udev, priv->pipe_interrupt,
+ priv->int_buf, USB_INTR_CONTENT_LENGTH,
+ rtl8xxxu_int_complete, priv, 1);
+ usb_anchor_urb(urb, &priv->int_anchor);
+ ret = usb_submit_urb(urb, GFP_KERNEL);
+ if (ret) {
+ usb_unanchor_urb(urb);
+ goto error;
+ }
+
+ val32 = rtl8xxxu_read32(priv, REG_USB_HIMR);
+ val32 |= USB_HIMR_CPWM;
+ rtl8xxxu_write32(priv, REG_USB_HIMR, val32);
+
+error:
+ return ret;
+}
+
+static int rtl8xxxu_add_interface(struct ieee80211_hw *hw,
+ struct ieee80211_vif *vif)
+{
+ struct rtl8xxxu_priv *priv = hw->priv;
+ int ret;
+ u8 val8;
+
+ switch (vif->type) {
+ case NL80211_IFTYPE_STATION:
+ rtl8xxxu_stop_tx_beacon(priv);
+
+ val8 = rtl8xxxu_read8(priv, REG_BEACON_CTRL);
+ val8 |= BEACON_ATIM | BEACON_FUNCTION_ENABLE |
+ BEACON_DISABLE_TSF_UPDATE;
+ rtl8xxxu_write8(priv, REG_BEACON_CTRL, val8);
+ ret = 0;
+ break;
+ default:
+ ret = -EOPNOTSUPP;
+ }
+
+ rtl8xxxu_set_linktype(priv, vif->type);
+
+ return ret;
+}
+
+static void rtl8xxxu_remove_interface(struct ieee80211_hw *hw,
+ struct ieee80211_vif *vif)
+{
+ struct rtl8xxxu_priv *priv = hw->priv;
+
+ dev_dbg(&priv->udev->dev, "%s\n", __func__);
+}
+
+static int rtl8xxxu_config(struct ieee80211_hw *hw, u32 changed)
+{
+ struct rtl8xxxu_priv *priv = hw->priv;
+ struct device *dev = &priv->udev->dev;
+ u16 val16;
+ int ret = 0, channel;
+ bool ht40;
+
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_CHANNEL)
+ dev_info(dev,
+ "%s: channel: %i (changed %08x chandef.width %02x)\n",
+ __func__, hw->conf.chandef.chan->hw_value,
+ changed, hw->conf.chandef.width);
+
+ if (changed & IEEE80211_CONF_CHANGE_RETRY_LIMITS) {
+ val16 = ((hw->conf.long_frame_max_tx_count <<
+ RETRY_LIMIT_LONG_SHIFT) & RETRY_LIMIT_LONG_MASK) |
+ ((hw->conf.short_frame_max_tx_count <<
+ RETRY_LIMIT_SHORT_SHIFT) & RETRY_LIMIT_SHORT_MASK);
+ rtl8xxxu_write16(priv, REG_RETRY_LIMIT, val16);
+ }
+
+ if (changed & IEEE80211_CONF_CHANGE_CHANNEL) {
+ switch (hw->conf.chandef.width) {
+ case NL80211_CHAN_WIDTH_20_NOHT:
+ case NL80211_CHAN_WIDTH_20:
+ ht40 = false;
+ break;
+ case NL80211_CHAN_WIDTH_40:
+ ht40 = true;
+ break;
+ default:
+ ret = -ENOTSUPP;
+ goto exit;
+ }
+
+ channel = hw->conf.chandef.chan->hw_value;
+
+ priv->fops->set_tx_power(priv, channel, ht40);
+
+ priv->fops->config_channel(hw);
+ }
+
+exit:
+ return ret;
+}
+
+static int rtl8xxxu_conf_tx(struct ieee80211_hw *hw,
+ struct ieee80211_vif *vif, u16 queue,
+ const struct ieee80211_tx_queue_params *param)
+{
+ struct rtl8xxxu_priv *priv = hw->priv;
+ struct device *dev = &priv->udev->dev;
+ u32 val32;
+ u8 aifs, acm_ctrl, acm_bit;
+
+ aifs = param->aifs;
+
+ val32 = aifs |
+ fls(param->cw_min) << EDCA_PARAM_ECW_MIN_SHIFT |
+ fls(param->cw_max) << EDCA_PARAM_ECW_MAX_SHIFT |
+ (u32)param->txop << EDCA_PARAM_TXOP_SHIFT;
+
+ acm_ctrl = rtl8xxxu_read8(priv, REG_ACM_HW_CTRL);
+ dev_dbg(dev,
+ "%s: IEEE80211 queue %02x val %08x, acm %i, acm_ctrl %02x\n",
+ __func__, queue, val32, param->acm, acm_ctrl);
+
+ switch (queue) {
+ case IEEE80211_AC_VO:
+ acm_bit = ACM_HW_CTRL_VO;
+ rtl8xxxu_write32(priv, REG_EDCA_VO_PARAM, val32);
+ break;
+ case IEEE80211_AC_VI:
+ acm_bit = ACM_HW_CTRL_VI;
+ rtl8xxxu_write32(priv, REG_EDCA_VI_PARAM, val32);
+ break;
+ case IEEE80211_AC_BE:
+ acm_bit = ACM_HW_CTRL_BE;
+ rtl8xxxu_write32(priv, REG_EDCA_BE_PARAM, val32);
+ break;
+ case IEEE80211_AC_BK:
+ acm_bit = ACM_HW_CTRL_BK;
+ rtl8xxxu_write32(priv, REG_EDCA_BK_PARAM, val32);
+ break;
+ default:
+ acm_bit = 0;
+ break;
+ }
+
+ if (param->acm)
+ acm_ctrl |= acm_bit;
+ else
+ acm_ctrl &= ~acm_bit;
+ rtl8xxxu_write8(priv, REG_ACM_HW_CTRL, acm_ctrl);
+
+ return 0;
+}
+
+static void rtl8xxxu_configure_filter(struct ieee80211_hw *hw,
+ unsigned int changed_flags,
+ unsigned int *total_flags, u64 multicast)
+{
+ struct rtl8xxxu_priv *priv = hw->priv;
+ u32 rcr = rtl8xxxu_read32(priv, REG_RCR);
+
+ dev_dbg(&priv->udev->dev, "%s: changed_flags %08x, total_flags %08x\n",
+ __func__, changed_flags, *total_flags);
+
+ /*
+ * FIF_ALLMULTI ignored as all multicast frames are accepted (REG_MAR)
+ */
+
+ if (*total_flags & FIF_FCSFAIL)
+ rcr |= RCR_ACCEPT_CRC32;
+ else
+ rcr &= ~RCR_ACCEPT_CRC32;
+
+ /*
+ * FIF_PLCPFAIL not supported?
+ */
+
+ if (*total_flags & FIF_BCN_PRBRESP_PROMISC)
+ rcr &= ~RCR_CHECK_BSSID_BEACON;
+ else
+ rcr |= RCR_CHECK_BSSID_BEACON;
+
+ if (*total_flags & FIF_CONTROL)
+ rcr |= RCR_ACCEPT_CTRL_FRAME;
+ else
+ rcr &= ~RCR_ACCEPT_CTRL_FRAME;
+
+ if (*total_flags & FIF_OTHER_BSS) {
+ rcr |= RCR_ACCEPT_AP;
+ rcr &= ~RCR_CHECK_BSSID_MATCH;
+ } else {
+ rcr &= ~RCR_ACCEPT_AP;
+ rcr |= RCR_CHECK_BSSID_MATCH;
+ }
+
+ if (*total_flags & FIF_PSPOLL)
+ rcr |= RCR_ACCEPT_PM;
+ else
+ rcr &= ~RCR_ACCEPT_PM;
+
+ /*
+ * FIF_PROBE_REQ ignored as probe requests always seem to be accepted
+ */
+
+ rtl8xxxu_write32(priv, REG_RCR, rcr);
+
+ *total_flags &= (FIF_ALLMULTI | FIF_FCSFAIL | FIF_BCN_PRBRESP_PROMISC |
+ FIF_CONTROL | FIF_OTHER_BSS | FIF_PSPOLL |
+ FIF_PROBE_REQ);
+}
+
+static int rtl8xxxu_set_rts_threshold(struct ieee80211_hw *hw, u32 rts)
+{
+ if (rts > 2347)
+ return -EINVAL;
+
+ return 0;
+}
+
+static int rtl8xxxu_set_key(struct ieee80211_hw *hw, enum set_key_cmd cmd,
+ struct ieee80211_vif *vif,
+ struct ieee80211_sta *sta,
+ struct ieee80211_key_conf *key)
+{
+ struct rtl8xxxu_priv *priv = hw->priv;
+ struct device *dev = &priv->udev->dev;
+ u8 mac_addr[ETH_ALEN];
+ u8 val8;
+ u16 val16;
+ u32 val32;
+ int retval = -EOPNOTSUPP;
+
+ dev_dbg(dev, "%s: cmd %02x, cipher %08x, index %i\n",
+ __func__, cmd, key->cipher, key->keyidx);
+
+ if (vif->type != NL80211_IFTYPE_STATION)
+ return -EOPNOTSUPP;
+
+ if (key->keyidx > 3)
+ return -EOPNOTSUPP;
+
+ switch (key->cipher) {
+ case WLAN_CIPHER_SUITE_WEP40:
+ case WLAN_CIPHER_SUITE_WEP104:
+
+ break;
+ case WLAN_CIPHER_SUITE_CCMP:
+ key->flags |= IEEE80211_KEY_FLAG_SW_MGMT_TX;
+ break;
+ case WLAN_CIPHER_SUITE_TKIP:
+ key->flags |= IEEE80211_KEY_FLAG_GENERATE_MMIC;
+ default:
+ return -EOPNOTSUPP;
+ }
+
+ if (key->flags & IEEE80211_KEY_FLAG_PAIRWISE) {
+ dev_dbg(dev, "%s: pairwise key\n", __func__);
+ ether_addr_copy(mac_addr, sta->addr);
+ } else {
+ dev_dbg(dev, "%s: group key\n", __func__);
+ eth_broadcast_addr(mac_addr);
+ }
+
+ val16 = rtl8xxxu_read16(priv, REG_CR);
+ val16 |= CR_SECURITY_ENABLE;
+ rtl8xxxu_write16(priv, REG_CR, val16);
+
+ val8 = SEC_CFG_TX_SEC_ENABLE | SEC_CFG_TXBC_USE_DEFKEY |
+ SEC_CFG_RX_SEC_ENABLE | SEC_CFG_RXBC_USE_DEFKEY;
+ val8 |= SEC_CFG_TX_USE_DEFKEY | SEC_CFG_RX_USE_DEFKEY;
+ rtl8xxxu_write8(priv, REG_SECURITY_CFG, val8);
+
+ switch (cmd) {
+ case SET_KEY:
+ key->hw_key_idx = key->keyidx;
+ key->flags |= IEEE80211_KEY_FLAG_GENERATE_IV;
+ rtl8xxxu_cam_write(priv, key, mac_addr);
+ retval = 0;
+ break;
+ case DISABLE_KEY:
+ rtl8xxxu_write32(priv, REG_CAM_WRITE, 0x00000000);
+ val32 = CAM_CMD_POLLING | CAM_CMD_WRITE |
+ key->keyidx << CAM_CMD_KEY_SHIFT;
+ rtl8xxxu_write32(priv, REG_CAM_CMD, val32);
+ retval = 0;
+ break;
+ default:
+ dev_warn(dev, "%s: Unsupported command %02x\n", __func__, cmd);
+ }
+
+ return retval;
+}
+
+static int
+rtl8xxxu_ampdu_action(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
+ struct ieee80211_ampdu_params *params)
+{
+ struct rtl8xxxu_priv *priv = hw->priv;
+ struct device *dev = &priv->udev->dev;
+ u8 ampdu_factor, ampdu_density;
+ struct ieee80211_sta *sta = params->sta;
+ enum ieee80211_ampdu_mlme_action action = params->action;
+
+ switch (action) {
+ case IEEE80211_AMPDU_TX_START:
+ dev_info(dev, "%s: IEEE80211_AMPDU_TX_START\n", __func__);
+ ampdu_factor = sta->ht_cap.ampdu_factor;
+ ampdu_density = sta->ht_cap.ampdu_density;
+ rtl8xxxu_set_ampdu_factor(priv, ampdu_factor);
+ rtl8xxxu_set_ampdu_min_space(priv, ampdu_density);
+ dev_dbg(dev,
+ "Changed HT: ampdu_factor %02x, ampdu_density %02x\n",
+ ampdu_factor, ampdu_density);
+ break;
+ case IEEE80211_AMPDU_TX_STOP_FLUSH:
+ dev_info(dev, "%s: IEEE80211_AMPDU_TX_STOP_FLUSH\n", __func__);
+ rtl8xxxu_set_ampdu_factor(priv, 0);
+ rtl8xxxu_set_ampdu_min_space(priv, 0);
+ break;
+ case IEEE80211_AMPDU_TX_STOP_FLUSH_CONT:
+ dev_info(dev, "%s: IEEE80211_AMPDU_TX_STOP_FLUSH_CONT\n",
+ __func__);
+ rtl8xxxu_set_ampdu_factor(priv, 0);
+ rtl8xxxu_set_ampdu_min_space(priv, 0);
+ break;
+ case IEEE80211_AMPDU_RX_START:
+ dev_info(dev, "%s: IEEE80211_AMPDU_RX_START\n", __func__);
+ break;
+ case IEEE80211_AMPDU_RX_STOP:
+ dev_info(dev, "%s: IEEE80211_AMPDU_RX_STOP\n", __func__);
+ break;
+ default:
+ break;
+ }
+ return 0;
+}
+
+static int rtl8xxxu_start(struct ieee80211_hw *hw)
+{
+ struct rtl8xxxu_priv *priv = hw->priv;
+ struct rtl8xxxu_rx_urb *rx_urb;
+ struct rtl8xxxu_tx_urb *tx_urb;
+ unsigned long flags;
+ int ret, i;
+
+ ret = 0;
+
+ init_usb_anchor(&priv->rx_anchor);
+ init_usb_anchor(&priv->tx_anchor);
+ init_usb_anchor(&priv->int_anchor);
+
+ priv->fops->enable_rf(priv);
+ if (priv->usb_interrupts) {
+ ret = rtl8xxxu_submit_int_urb(hw);
+ if (ret)
+ goto exit;
+ }
+
+ for (i = 0; i < RTL8XXXU_TX_URBS; i++) {
+ tx_urb = kmalloc(sizeof(struct rtl8xxxu_tx_urb), GFP_KERNEL);
+ if (!tx_urb) {
+ if (!i)
+ ret = -ENOMEM;
+
+ goto error_out;
+ }
+ usb_init_urb(&tx_urb->urb);
+ INIT_LIST_HEAD(&tx_urb->list);
+ tx_urb->hw = hw;
+ list_add(&tx_urb->list, &priv->tx_urb_free_list);
+ priv->tx_urb_free_count++;
+ }
+
+ priv->tx_stopped = false;
+
+ spin_lock_irqsave(&priv->rx_urb_lock, flags);
+ priv->shutdown = false;
+ spin_unlock_irqrestore(&priv->rx_urb_lock, flags);
+
+ for (i = 0; i < RTL8XXXU_RX_URBS; i++) {
+ rx_urb = kmalloc(sizeof(struct rtl8xxxu_rx_urb), GFP_KERNEL);
+ if (!rx_urb) {
+ if (!i)
+ ret = -ENOMEM;
+
+ goto error_out;
+ }
+ usb_init_urb(&rx_urb->urb);
+ INIT_LIST_HEAD(&rx_urb->list);
+ rx_urb->hw = hw;
+
+ ret = rtl8xxxu_submit_rx_urb(priv, rx_urb);
+ }
+exit:
+ /*
+ * Accept all data and mgmt frames
+ */
+ rtl8xxxu_write16(priv, REG_RXFLTMAP2, 0xffff);
+ rtl8xxxu_write16(priv, REG_RXFLTMAP0, 0xffff);
+
+ rtl8xxxu_write32(priv, REG_OFDM0_XA_AGC_CORE1, 0x6954341e);
+
+ return ret;
+
+error_out:
+ rtl8xxxu_free_tx_resources(priv);
+ /*
+ * Disable all data and mgmt frames
+ */
+ rtl8xxxu_write16(priv, REG_RXFLTMAP2, 0x0000);
+ rtl8xxxu_write16(priv, REG_RXFLTMAP0, 0x0000);
+
+ return ret;
+}
+
+static void rtl8xxxu_stop(struct ieee80211_hw *hw)
+{
+ struct rtl8xxxu_priv *priv = hw->priv;
+ unsigned long flags;
+
+ rtl8xxxu_write8(priv, REG_TXPAUSE, 0xff);
+
+ rtl8xxxu_write16(priv, REG_RXFLTMAP0, 0x0000);
+ rtl8xxxu_write16(priv, REG_RXFLTMAP2, 0x0000);
+
+ spin_lock_irqsave(&priv->rx_urb_lock, flags);
+ priv->shutdown = true;
+ spin_unlock_irqrestore(&priv->rx_urb_lock, flags);
+
+ usb_kill_anchored_urbs(&priv->rx_anchor);
+ usb_kill_anchored_urbs(&priv->tx_anchor);
+ if (priv->usb_interrupts)
+ usb_kill_anchored_urbs(&priv->int_anchor);
+
+ rtl8xxxu_write8(priv, REG_TXPAUSE, 0xff);
+
+ priv->fops->disable_rf(priv);
+
+ /*
+ * Disable interrupts
+ */
+ if (priv->usb_interrupts)
+ rtl8xxxu_write32(priv, REG_USB_HIMR, 0);
+
+ rtl8xxxu_free_rx_resources(priv);
+ rtl8xxxu_free_tx_resources(priv);
+}
+
+static const struct ieee80211_ops rtl8xxxu_ops = {
+ .tx = rtl8xxxu_tx,
+ .add_interface = rtl8xxxu_add_interface,
+ .remove_interface = rtl8xxxu_remove_interface,
+ .config = rtl8xxxu_config,
+ .conf_tx = rtl8xxxu_conf_tx,
+ .bss_info_changed = rtl8xxxu_bss_info_changed,
+ .configure_filter = rtl8xxxu_configure_filter,
+ .set_rts_threshold = rtl8xxxu_set_rts_threshold,
+ .start = rtl8xxxu_start,
+ .stop = rtl8xxxu_stop,
+ .sw_scan_start = rtl8xxxu_sw_scan_start,
+ .sw_scan_complete = rtl8xxxu_sw_scan_complete,
+ .set_key = rtl8xxxu_set_key,
+ .ampdu_action = rtl8xxxu_ampdu_action,
+};
+
+static int rtl8xxxu_parse_usb(struct rtl8xxxu_priv *priv,
+ struct usb_interface *interface)
+{
+ struct usb_interface_descriptor *interface_desc;
+ struct usb_host_interface *host_interface;
+ struct usb_endpoint_descriptor *endpoint;
+ struct device *dev = &priv->udev->dev;
+ int i, j = 0, endpoints;
+ u8 dir, xtype, num;
+ int ret = 0;
+
+ host_interface = &interface->altsetting[0];
+ interface_desc = &host_interface->desc;
+ endpoints = interface_desc->bNumEndpoints;
+
+ for (i = 0; i < endpoints; i++) {
+ endpoint = &host_interface->endpoint[i].desc;
+
+ dir = endpoint->bEndpointAddress & USB_ENDPOINT_DIR_MASK;
+ num = usb_endpoint_num(endpoint);
+ xtype = usb_endpoint_type(endpoint);
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_USB)
+ dev_dbg(dev,
+ "%s: endpoint: dir %02x, # %02x, type %02x\n",
+ __func__, dir, num, xtype);
+ if (usb_endpoint_dir_in(endpoint) &&
+ usb_endpoint_xfer_bulk(endpoint)) {
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_USB)
+ dev_dbg(dev, "%s: in endpoint num %i\n",
+ __func__, num);
+
+ if (priv->pipe_in) {
+ dev_warn(dev,
+ "%s: Too many IN pipes\n", __func__);
+ ret = -EINVAL;
+ goto exit;
+ }
+
+ priv->pipe_in = usb_rcvbulkpipe(priv->udev, num);
+ }
+
+ if (usb_endpoint_dir_in(endpoint) &&
+ usb_endpoint_xfer_int(endpoint)) {
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_USB)
+ dev_dbg(dev, "%s: interrupt endpoint num %i\n",
+ __func__, num);
+
+ if (priv->pipe_interrupt) {
+ dev_warn(dev, "%s: Too many INTERRUPT pipes\n",
+ __func__);
+ ret = -EINVAL;
+ goto exit;
+ }
+
+ priv->pipe_interrupt = usb_rcvintpipe(priv->udev, num);
+ }
+
+ if (usb_endpoint_dir_out(endpoint) &&
+ usb_endpoint_xfer_bulk(endpoint)) {
+ if (rtl8xxxu_debug & RTL8XXXU_DEBUG_USB)
+ dev_dbg(dev, "%s: out endpoint num %i\n",
+ __func__, num);
+ if (j >= RTL8XXXU_OUT_ENDPOINTS) {
+ dev_warn(dev,
+ "%s: Too many OUT pipes\n", __func__);
+ ret = -EINVAL;
+ goto exit;
+ }
+ priv->out_ep[j++] = num;
+ }
+ }
+exit:
+ priv->nr_out_eps = j;
+ return ret;
+}
+
+static int rtl8xxxu_probe(struct usb_interface *interface,
+ const struct usb_device_id *id)
+{
+ struct rtl8xxxu_priv *priv;
+ struct ieee80211_hw *hw;
+ struct usb_device *udev;
+ struct ieee80211_supported_band *sband;
+ int ret = 0;
+ int untested = 1;
+
+ udev = usb_get_dev(interface_to_usbdev(interface));
+
+ switch (id->idVendor) {
+ case USB_VENDOR_ID_REALTEK:
+ switch(id->idProduct) {
+ case 0x1724:
+ case 0x8176:
+ case 0x8178:
+ case 0x817f:
+ untested = 0;
+ break;
+ }
+ break;
+ case 0x7392:
+ if (id->idProduct == 0x7811)
+ untested = 0;
+ break;
+ case 0x050d:
+ if (id->idProduct == 0x1004)
+ untested = 0;
+ break;
+ default:
+ break;
+ }
+
+ if (untested) {
+ rtl8xxxu_debug |= RTL8XXXU_DEBUG_EFUSE;
+ dev_info(&udev->dev,
+ "This Realtek USB WiFi dongle (0x%04x:0x%04x) is untested!\n",
+ id->idVendor, id->idProduct);
+ dev_info(&udev->dev,
+ "Please report results to Jes.Sorensen@gmail.com\n");
+ }
+
+ hw = ieee80211_alloc_hw(sizeof(struct rtl8xxxu_priv), &rtl8xxxu_ops);
+ if (!hw) {
+ ret = -ENOMEM;
+ goto exit;
+ }
+
+ priv = hw->priv;
+ priv->hw = hw;
+ priv->udev = udev;
+ priv->fops = (struct rtl8xxxu_fileops *)id->driver_info;
+ mutex_init(&priv->usb_buf_mutex);
+ mutex_init(&priv->h2c_mutex);
+ INIT_LIST_HEAD(&priv->tx_urb_free_list);
+ spin_lock_init(&priv->tx_urb_lock);
+ INIT_LIST_HEAD(&priv->rx_urb_pending_list);
+ spin_lock_init(&priv->rx_urb_lock);
+ INIT_WORK(&priv->rx_urb_wq, rtl8xxxu_rx_urb_work);
+
+ usb_set_intfdata(interface, hw);
+
+ ret = rtl8xxxu_parse_usb(priv, interface);
+ if (ret)
+ goto exit;
+
+ ret = rtl8xxxu_identify_chip(priv);
+ if (ret) {
+ dev_err(&udev->dev, "Fatal - failed to identify chip\n");
+ goto exit;
+ }
+
+ ret = rtl8xxxu_read_efuse(priv);
+ if (ret) {
+ dev_err(&udev->dev, "Fatal - failed to read EFuse\n");
+ goto exit;
+ }
+
+ ret = priv->fops->parse_efuse(priv);
+ if (ret) {
+ dev_err(&udev->dev, "Fatal - failed to parse EFuse\n");
+ goto exit;
+ }
+
+ rtl8xxxu_print_chipinfo(priv);
+
+ ret = priv->fops->load_firmware(priv);
+ if (ret) {
+ dev_err(&udev->dev, "Fatal - failed to load firmware\n");
+ goto exit;
+ }
+
+ ret = rtl8xxxu_init_device(hw);
+
+ hw->wiphy->max_scan_ssids = 1;
+ hw->wiphy->max_scan_ie_len = IEEE80211_MAX_DATA_LEN;
+ hw->wiphy->interface_modes = BIT(NL80211_IFTYPE_STATION);
+ hw->queues = 4;
+
+ sband = &rtl8xxxu_supported_band;
+ sband->ht_cap.ht_supported = true;
+ sband->ht_cap.ampdu_factor = IEEE80211_HT_MAX_AMPDU_64K;
+ sband->ht_cap.ampdu_density = IEEE80211_HT_MPDU_DENSITY_16;
+ sband->ht_cap.cap = IEEE80211_HT_CAP_SGI_20 | IEEE80211_HT_CAP_SGI_40;
+ memset(&sband->ht_cap.mcs, 0, sizeof(sband->ht_cap.mcs));
+ sband->ht_cap.mcs.rx_mask[0] = 0xff;
+ sband->ht_cap.mcs.rx_mask[4] = 0x01;
+ if (priv->rf_paths > 1) {
+ sband->ht_cap.mcs.rx_mask[1] = 0xff;
+ sband->ht_cap.cap |= IEEE80211_HT_CAP_SGI_40;
+ }
+ sband->ht_cap.mcs.tx_params = IEEE80211_HT_MCS_TX_DEFINED;
+ /*
+ * Some APs will negotiate HT20_40 in a noisy environment leading
+ * to miserable performance. Rather than defaulting to this, only
+ * enable it if explicitly requested at module load time.
+ */
+ if (rtl8xxxu_ht40_2g) {
+ dev_info(&udev->dev, "Enabling HT_20_40 on the 2.4GHz band\n");
+ sband->ht_cap.cap |= IEEE80211_HT_CAP_SUP_WIDTH_20_40;
+ }
+ hw->wiphy->bands[NL80211_BAND_2GHZ] = sband;
+
+ hw->wiphy->rts_threshold = 2347;
+
+ SET_IEEE80211_DEV(priv->hw, &interface->dev);
+ SET_IEEE80211_PERM_ADDR(hw, priv->mac_addr);
+
+ hw->extra_tx_headroom = priv->fops->tx_desc_size;
+ ieee80211_hw_set(hw, SIGNAL_DBM);
+ /*
+ * The firmware handles rate control
+ */
+ ieee80211_hw_set(hw, HAS_RATE_CONTROL);
+ ieee80211_hw_set(hw, AMPDU_AGGREGATION);
+
+ ret = ieee80211_register_hw(priv->hw);
+ if (ret) {
+ dev_err(&udev->dev, "%s: Failed to register: %i\n",
+ __func__, ret);
+ goto exit;
+ }
+
+exit:
+ if (ret < 0)
+ usb_put_dev(udev);
+ return ret;
+}
+
+static void rtl8xxxu_disconnect(struct usb_interface *interface)
+{
+ struct rtl8xxxu_priv *priv;
+ struct ieee80211_hw *hw;
+
+ hw = usb_get_intfdata(interface);
+ priv = hw->priv;
+
+ ieee80211_unregister_hw(hw);
+
+ priv->fops->power_off(priv);
+
+ usb_set_intfdata(interface, NULL);
+
+ dev_info(&priv->udev->dev, "disconnecting\n");
+
+ kfree(priv->fw_data);
+ mutex_destroy(&priv->usb_buf_mutex);
+ mutex_destroy(&priv->h2c_mutex);
+
+ usb_put_dev(priv->udev);
+ ieee80211_free_hw(hw);
+}
+
+static struct usb_device_id dev_table[] = {
+{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x8724, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8723au_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x1724, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8723au_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x0724, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8723au_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x818b, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192eu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0xb720, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8723bu_fops},
+#ifdef CONFIG_RTL8XXXU_UNTESTED
+/* Still supported by rtlwifi */
+{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x8176, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x8178, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x817f, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+/* Tested by Larry Finger */
+{USB_DEVICE_AND_INTERFACE_INFO(0x7392, 0x7811, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+/* Tested by Andrea Merello */
+{USB_DEVICE_AND_INTERFACE_INFO(0x050d, 0x1004, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+/* Currently untested 8188 series devices */
+{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x8191, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x8170, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x8177, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x817a, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x817b, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x817d, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x817e, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x818a, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x317f, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x1058, 0x0631, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x04bb, 0x094c, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x050d, 0x1102, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x06f8, 0xe033, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x07b8, 0x8189, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x0846, 0x9041, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x0b05, 0x17ba, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x1e1e, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x5088, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x0df6, 0x0052, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x0df6, 0x005c, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x0eb0, 0x9071, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x103c, 0x1629, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x13d3, 0x3357, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x2001, 0x3308, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x2001, 0x330b, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x2019, 0x4902, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x2019, 0xab2a, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x2019, 0xab2e, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x2019, 0xed17, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x20f4, 0x648b, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x4855, 0x0090, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x4856, 0x0091, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0xcdab, 0x8010, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x04f2, 0xaff7, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x04f2, 0xaff9, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x04f2, 0xaffa, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x04f2, 0xaff8, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x04f2, 0xaffb, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x04f2, 0xaffc, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x2019, 0x1201, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+/* Currently untested 8192 series devices */
+{USB_DEVICE_AND_INTERFACE_INFO(0x04bb, 0x0950, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x050d, 0x2102, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x050d, 0x2103, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x0586, 0x341f, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x06f8, 0xe035, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x0b05, 0x17ab, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x0df6, 0x0061, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x0df6, 0x0070, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x0789, 0x016d, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x07aa, 0x0056, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x07b8, 0x8178, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x0846, 0x9021, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x0846, 0xf001, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(USB_VENDOR_ID_REALTEK, 0x2e2e, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x0e66, 0x0019, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x0e66, 0x0020, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x2001, 0x3307, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x2001, 0x3309, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x2001, 0x330a, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x2019, 0xab2b, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x20f4, 0x624d, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x2357, 0x0100, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x4855, 0x0091, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+{USB_DEVICE_AND_INTERFACE_INFO(0x7392, 0x7822, 0xff, 0xff, 0xff),
+ .driver_info = (unsigned long)&rtl8192cu_fops},
+#endif
+{ }
+};
+
+static struct usb_driver rtl8xxxu_driver = {
+ .name = DRIVER_NAME,
+ .probe = rtl8xxxu_probe,
+ .disconnect = rtl8xxxu_disconnect,
+ .id_table = dev_table,
+ .no_dynamic_id = 1,
+ .disable_hub_initiated_lpm = 1,
+};
+
+static int __init rtl8xxxu_module_init(void)
+{
+ int res;
+
+ res = usb_register(&rtl8xxxu_driver);
+ if (res < 0)
+ pr_err(DRIVER_NAME ": usb_register() failed (%i)\n", res);
+
+ return res;
+}
+
+static void __exit rtl8xxxu_module_exit(void)
+{
+ usb_deregister(&rtl8xxxu_driver);
+}
+
+
+MODULE_DEVICE_TABLE(usb, dev_table);
+
+module_init(rtl8xxxu_module_init);
+module_exit(rtl8xxxu_module_exit);
diff --git a/drivers/net/wireless/realtek/rtlwifi/base.c b/drivers/net/wireless/realtek/rtlwifi/base.c
index c74eb13..264466f 100644
--- a/drivers/net/wireless/realtek/rtlwifi/base.c
+++ b/drivers/net/wireless/realtek/rtlwifi/base.c
@@ -1660,9 +1660,9 @@
if (((rtlpriv->link_info.num_rx_inperiod +
rtlpriv->link_info.num_tx_inperiod) > 8) ||
(rtlpriv->link_info.num_rx_inperiod > 2))
- rtl_lps_enter(hw);
- else
rtl_lps_leave(hw);
+ else
+ rtl_lps_enter(hw);
}
rtlpriv->link_info.num_rx_inperiod = 0;
diff --git a/drivers/net/wireless/realtek/rtlwifi/pci.c b/drivers/net/wireless/realtek/rtlwifi/pci.c
index 1ac41b8..d12586d 100644
--- a/drivers/net/wireless/realtek/rtlwifi/pci.c
+++ b/drivers/net/wireless/realtek/rtlwifi/pci.c
@@ -1572,7 +1572,7 @@
true,
HW_DESC_TXBUFF_ADDR),
skb->len, PCI_DMA_TODEVICE);
- kfree_skb(skb);
+ dev_kfree_skb_irq(skb);
ring->idx = (ring->idx + 1) % ring->entries;
}
ring->idx = 0;
@@ -2456,7 +2456,7 @@
EXPORT_SYMBOL(rtl_pci_resume);
#endif /* CONFIG_PM_SLEEP */
-struct rtl_intf_ops rtl_pci_ops = {
+const struct rtl_intf_ops rtl_pci_ops = {
.read_efuse_byte = read_efuse_byte,
.adapter_start = rtl_pci_start,
.adapter_stop = rtl_pci_stop,
diff --git a/drivers/net/wireless/realtek/rtlwifi/pci.h b/drivers/net/wireless/realtek/rtlwifi/pci.h
index 5da6703..b951eba 100644
--- a/drivers/net/wireless/realtek/rtlwifi/pci.h
+++ b/drivers/net/wireless/realtek/rtlwifi/pci.h
@@ -286,7 +286,7 @@
int rtl_pci_reset_trx_ring(struct ieee80211_hw *hw);
-extern struct rtl_intf_ops rtl_pci_ops;
+extern const struct rtl_intf_ops rtl_pci_ops;
int rtl_pci_probe(struct pci_dev *pdev,
const struct pci_device_id *id);
diff --git a/drivers/net/wireless/realtek/rtlwifi/ps.c b/drivers/net/wireless/realtek/rtlwifi/ps.c
index b69321d..93579ca 100644
--- a/drivers/net/wireless/realtek/rtlwifi/ps.c
+++ b/drivers/net/wireless/realtek/rtlwifi/ps.c
@@ -443,14 +443,10 @@
spin_lock_irqsave(&rtlpriv->locks.lps_lock, flag);
- /* Idle for a while if we connect to AP a while ago. */
- if (mac->cnt_after_linked >= 2) {
- if (ppsc->dot11_psmode == EACTIVE) {
- RT_TRACE(rtlpriv, COMP_POWER, DBG_LOUD,
- "Enter 802.11 power save mode...\n");
-
- rtl_lps_set_psmode(hw, EAUTOPS);
- }
+ if (ppsc->dot11_psmode == EACTIVE) {
+ RT_TRACE(rtlpriv, COMP_POWER, DBG_LOUD,
+ "Enter 802.11 power save mode...\n");
+ rtl_lps_set_psmode(hw, EAUTOPS);
}
spin_unlock_irqrestore(&rtlpriv->locks.lps_lock, flag);
diff --git a/drivers/net/wireless/realtek/rtlwifi/rtl8192se/rf.c b/drivers/net/wireless/realtek/rtlwifi/rtl8192se/rf.c
index 78a81c1..9475aa2 100644
--- a/drivers/net/wireless/realtek/rtlwifi/rtl8192se/rf.c
+++ b/drivers/net/wireless/realtek/rtlwifi/rtl8192se/rf.c
@@ -208,8 +208,7 @@
"Realtek regulatory, 40MHz, writeval = 0x%x\n",
writeval);
} else {
- if (rtlphy->pwrgroup_cnt == 1)
- chnlgroup = 0;
+ chnlgroup = 0;
if (rtlphy->pwrgroup_cnt >= 3) {
if (chnl <= 3)
diff --git a/drivers/net/wireless/realtek/rtlwifi/usb.c b/drivers/net/wireless/realtek/rtlwifi/usb.c
index aac1ed3..41617b7 100644
--- a/drivers/net/wireless/realtek/rtlwifi/usb.c
+++ b/drivers/net/wireless/realtek/rtlwifi/usb.c
@@ -1049,7 +1049,7 @@
rtlpriv->cfg->ops->fill_h2c_cmd(hw, H2C_RA_MASK, 5, rtlpriv->rate_mask);
}
-static struct rtl_intf_ops rtl_usb_ops = {
+static const struct rtl_intf_ops rtl_usb_ops = {
.adapter_start = rtl_usb_start,
.adapter_stop = rtl_usb_stop,
.adapter_tx = rtl_usb_tx,
diff --git a/drivers/net/wireless/realtek/rtlwifi/wifi.h b/drivers/net/wireless/realtek/rtlwifi/wifi.h
index 11d9c23..4e0ab4d 100644
--- a/drivers/net/wireless/realtek/rtlwifi/wifi.h
+++ b/drivers/net/wireless/realtek/rtlwifi/wifi.h
@@ -2593,7 +2593,7 @@
*intf_ops : for diff interrface usb/pcie
*/
struct rtl_hal_cfg *cfg;
- struct rtl_intf_ops *intf_ops;
+ const struct rtl_intf_ops *intf_ops;
/*this var will be set by set_bit,
and was used to indicate status of
diff --git a/drivers/net/wireless/ti/wl12xx/main.c b/drivers/net/wireless/ti/wl12xx/main.c
index 58b9d3c..22009e1 100644
--- a/drivers/net/wireless/ti/wl12xx/main.c
+++ b/drivers/net/wireless/ti/wl12xx/main.c
@@ -553,8 +553,8 @@
.size = 0x00000004
},
.mem3 = {
- .start = 0x00040404,
- .size = 0x00000000
+ .start = 0x00000000,
+ .size = 0x00040404
},
},
diff --git a/drivers/net/wireless/ti/wlcore/io.c b/drivers/net/wireless/ti/wlcore/io.c
index 564ca75..1cc6d5a 100644
--- a/drivers/net/wireless/ti/wlcore/io.c
+++ b/drivers/net/wireless/ti/wlcore/io.c
@@ -175,14 +175,25 @@
if (ret < 0)
goto out;
- /* We don't need the size of the last partition, as it is
- * automatically calculated based on the total memory size and
- * the sizes of the previous partitions.
+ /* wl12xx only: We don't need the size of the last partition,
+ * as it is automatically calculated based on the total memory
+ * size and the sizes of the previous partitions.
+ *
+ * wl18xx re-defines the HW_PART3 addresses for logger over
+ * SDIO support. wl12xx is expecting the write to
+ * HW_PART3_START_ADDR at offset 24. This creates conflict
+ * between the addresses.
+ * In order to fix this the expected value is written to
+ * HW_PART3_SIZE_ADDR instead which is at offset 24 after changes.
*/
ret = wlcore_raw_write32(wl, HW_PART3_START_ADDR, p->mem3.start);
if (ret < 0)
goto out;
+ ret = wlcore_raw_write32(wl, HW_PART3_SIZE_ADDR, p->mem3.size);
+ if (ret < 0)
+ goto out;
+
out:
return ret;
}
diff --git a/drivers/net/wireless/ti/wlcore/io.h b/drivers/net/wireless/ti/wlcore/io.h
index 10cf374..704ce64 100644
--- a/drivers/net/wireless/ti/wlcore/io.h
+++ b/drivers/net/wireless/ti/wlcore/io.h
@@ -36,7 +36,8 @@
#define HW_PART1_START_ADDR (HW_PARTITION_REGISTERS_ADDR + 12)
#define HW_PART2_SIZE_ADDR (HW_PARTITION_REGISTERS_ADDR + 16)
#define HW_PART2_START_ADDR (HW_PARTITION_REGISTERS_ADDR + 20)
-#define HW_PART3_START_ADDR (HW_PARTITION_REGISTERS_ADDR + 24)
+#define HW_PART3_SIZE_ADDR (HW_PARTITION_REGISTERS_ADDR + 24)
+#define HW_PART3_START_ADDR (HW_PARTITION_REGISTERS_ADDR + 28)
#define HW_ACCESS_REGISTER_SIZE 4
diff --git a/drivers/net/xen-netback/hash.c b/drivers/net/xen-netback/hash.c
index 392e392..fb87cb3 100644
--- a/drivers/net/xen-netback/hash.c
+++ b/drivers/net/xen-netback/hash.c
@@ -311,7 +311,7 @@
if (len > XEN_NETBK_MAX_HASH_KEY_SIZE)
return XEN_NETIF_CTRL_STATUS_INVALID_PARAMETER;
- if (len != 0) {
+ if (copy_op.len != 0) {
gnttab_batch_copy(©_op, 1);
if (copy_op.status != GNTST_okay)
@@ -359,7 +359,7 @@
if (mapping[off++] >= vif->num_queues)
return XEN_NETIF_CTRL_STATUS_INVALID_PARAMETER;
- if (len != 0) {
+ if (copy_op.len != 0) {
gnttab_batch_copy(©_op, 1);
if (copy_op.status != GNTST_okay)
diff --git a/drivers/net/xen-netback/interface.c b/drivers/net/xen-netback/interface.c
index 1c7f49b..83deeeb 100644
--- a/drivers/net/xen-netback/interface.c
+++ b/drivers/net/xen-netback/interface.c
@@ -780,9 +780,8 @@
vif->ctrl_task = NULL;
}
- xenvif_deinit_hash(vif);
-
if (vif->ctrl_irq) {
+ xenvif_deinit_hash(vif);
unbind_from_irqhandler(vif->ctrl_irq, vif);
vif->ctrl_irq = 0;
}
diff --git a/drivers/nvdimm/Kconfig b/drivers/nvdimm/Kconfig
index 53c1162..7c8a3bf 100644
--- a/drivers/nvdimm/Kconfig
+++ b/drivers/nvdimm/Kconfig
@@ -88,4 +88,17 @@
Select Y if unsure
+config NVDIMM_DAX
+ bool "NVDIMM DAX: Raw access to persistent memory"
+ default LIBNVDIMM
+ depends on NVDIMM_PFN
+ help
+ Support raw device dax access to a persistent memory
+ namespace. For environments that want to hard partition
+ peristent memory, this capability provides a mechanism to
+ sub-divide a namespace into character devices that can only be
+ accessed via DAX (mmap(2)).
+
+ Select Y if unsure
+
endif
diff --git a/drivers/nvdimm/Makefile b/drivers/nvdimm/Makefile
index ea84d3c..909554c 100644
--- a/drivers/nvdimm/Makefile
+++ b/drivers/nvdimm/Makefile
@@ -23,3 +23,4 @@
libnvdimm-$(CONFIG_ND_CLAIM) += claim.o
libnvdimm-$(CONFIG_BTT) += btt_devs.o
libnvdimm-$(CONFIG_NVDIMM_PFN) += pfn_devs.o
+libnvdimm-$(CONFIG_NVDIMM_DAX) += dax_devs.o
diff --git a/drivers/nvdimm/blk.c b/drivers/nvdimm/blk.c
index e9ff922..495e06d9 100644
--- a/drivers/nvdimm/blk.c
+++ b/drivers/nvdimm/blk.c
@@ -21,19 +21,19 @@
#include <linux/sizes.h>
#include "nd.h"
-struct nd_blk_device {
- struct request_queue *queue;
- struct gendisk *disk;
- struct nd_namespace_blk *nsblk;
- struct nd_blk_region *ndbr;
- size_t disk_size;
- u32 sector_size;
- u32 internal_lbasize;
-};
-
-static u32 nd_blk_meta_size(struct nd_blk_device *blk_dev)
+static u32 nsblk_meta_size(struct nd_namespace_blk *nsblk)
{
- return blk_dev->nsblk->lbasize - blk_dev->sector_size;
+ return nsblk->lbasize - ((nsblk->lbasize >= 4096) ? 4096 : 512);
+}
+
+static u32 nsblk_internal_lbasize(struct nd_namespace_blk *nsblk)
+{
+ return roundup(nsblk->lbasize, INT_LBASIZE_ALIGNMENT);
+}
+
+static u32 nsblk_sector_size(struct nd_namespace_blk *nsblk)
+{
+ return nsblk->lbasize - nsblk_meta_size(nsblk);
}
static resource_size_t to_dev_offset(struct nd_namespace_blk *nsblk,
@@ -57,20 +57,29 @@
return SIZE_MAX;
}
-#ifdef CONFIG_BLK_DEV_INTEGRITY
-static int nd_blk_rw_integrity(struct nd_blk_device *blk_dev,
- struct bio_integrity_payload *bip, u64 lba,
- int rw)
+static struct nd_blk_region *to_ndbr(struct nd_namespace_blk *nsblk)
{
- unsigned int len = nd_blk_meta_size(blk_dev);
+ struct nd_region *nd_region;
+ struct device *parent;
+
+ parent = nsblk->common.dev.parent;
+ nd_region = container_of(parent, struct nd_region, dev);
+ return container_of(nd_region, struct nd_blk_region, nd_region);
+}
+
+#ifdef CONFIG_BLK_DEV_INTEGRITY
+static int nd_blk_rw_integrity(struct nd_namespace_blk *nsblk,
+ struct bio_integrity_payload *bip, u64 lba, int rw)
+{
+ struct nd_blk_region *ndbr = to_ndbr(nsblk);
+ unsigned int len = nsblk_meta_size(nsblk);
resource_size_t dev_offset, ns_offset;
- struct nd_namespace_blk *nsblk;
- struct nd_blk_region *ndbr;
+ u32 internal_lbasize, sector_size;
int err = 0;
- nsblk = blk_dev->nsblk;
- ndbr = blk_dev->ndbr;
- ns_offset = lba * blk_dev->internal_lbasize + blk_dev->sector_size;
+ internal_lbasize = nsblk_internal_lbasize(nsblk);
+ sector_size = nsblk_sector_size(nsblk);
+ ns_offset = lba * internal_lbasize + sector_size;
dev_offset = to_dev_offset(nsblk, ns_offset, len);
if (dev_offset == SIZE_MAX)
return -EIO;
@@ -104,25 +113,26 @@
}
#else /* CONFIG_BLK_DEV_INTEGRITY */
-static int nd_blk_rw_integrity(struct nd_blk_device *blk_dev,
- struct bio_integrity_payload *bip, u64 lba,
- int rw)
+static int nd_blk_rw_integrity(struct nd_namespace_blk *nsblk,
+ struct bio_integrity_payload *bip, u64 lba, int rw)
{
return 0;
}
#endif
-static int nd_blk_do_bvec(struct nd_blk_device *blk_dev,
- struct bio_integrity_payload *bip, struct page *page,
- unsigned int len, unsigned int off, int rw,
- sector_t sector)
+static int nsblk_do_bvec(struct nd_namespace_blk *nsblk,
+ struct bio_integrity_payload *bip, struct page *page,
+ unsigned int len, unsigned int off, int rw, sector_t sector)
{
- struct nd_blk_region *ndbr = blk_dev->ndbr;
+ struct nd_blk_region *ndbr = to_ndbr(nsblk);
resource_size_t dev_offset, ns_offset;
+ u32 internal_lbasize, sector_size;
int err = 0;
void *iobuf;
u64 lba;
+ internal_lbasize = nsblk_internal_lbasize(nsblk);
+ sector_size = nsblk_sector_size(nsblk);
while (len) {
unsigned int cur_len;
@@ -132,11 +142,11 @@
* Block Window setup/move steps. the do_io routine is capable
* of handling len <= PAGE_SIZE.
*/
- cur_len = bip ? min(len, blk_dev->sector_size) : len;
+ cur_len = bip ? min(len, sector_size) : len;
- lba = div_u64(sector << SECTOR_SHIFT, blk_dev->sector_size);
- ns_offset = lba * blk_dev->internal_lbasize;
- dev_offset = to_dev_offset(blk_dev->nsblk, ns_offset, cur_len);
+ lba = div_u64(sector << SECTOR_SHIFT, sector_size);
+ ns_offset = lba * internal_lbasize;
+ dev_offset = to_dev_offset(nsblk, ns_offset, cur_len);
if (dev_offset == SIZE_MAX)
return -EIO;
@@ -147,13 +157,13 @@
return err;
if (bip) {
- err = nd_blk_rw_integrity(blk_dev, bip, lba, rw);
+ err = nd_blk_rw_integrity(nsblk, bip, lba, rw);
if (err)
return err;
}
len -= cur_len;
off += cur_len;
- sector += blk_dev->sector_size >> SECTOR_SHIFT;
+ sector += sector_size >> SECTOR_SHIFT;
}
return err;
@@ -161,10 +171,8 @@
static blk_qc_t nd_blk_make_request(struct request_queue *q, struct bio *bio)
{
- struct block_device *bdev = bio->bi_bdev;
- struct gendisk *disk = bdev->bd_disk;
struct bio_integrity_payload *bip;
- struct nd_blk_device *blk_dev;
+ struct nd_namespace_blk *nsblk;
struct bvec_iter iter;
unsigned long start;
struct bio_vec bvec;
@@ -183,17 +191,17 @@
}
bip = bio_integrity(bio);
- blk_dev = disk->private_data;
+ nsblk = q->queuedata;
rw = bio_data_dir(bio);
do_acct = nd_iostat_start(bio, &start);
bio_for_each_segment(bvec, bio, iter) {
unsigned int len = bvec.bv_len;
BUG_ON(len > PAGE_SIZE);
- err = nd_blk_do_bvec(blk_dev, bip, bvec.bv_page, len,
- bvec.bv_offset, rw, iter.bi_sector);
+ err = nsblk_do_bvec(nsblk, bip, bvec.bv_page, len,
+ bvec.bv_offset, rw, iter.bi_sector);
if (err) {
- dev_info(&blk_dev->nsblk->common.dev,
+ dev_dbg(&nsblk->common.dev,
"io error in %s sector %lld, len %d,\n",
(rw == READ) ? "READ" : "WRITE",
(unsigned long long) iter.bi_sector, len);
@@ -209,17 +217,16 @@
return BLK_QC_T_NONE;
}
-static int nd_blk_rw_bytes(struct nd_namespace_common *ndns,
+static int nsblk_rw_bytes(struct nd_namespace_common *ndns,
resource_size_t offset, void *iobuf, size_t n, int rw)
{
- struct nd_blk_device *blk_dev = dev_get_drvdata(ndns->claim);
- struct nd_namespace_blk *nsblk = blk_dev->nsblk;
- struct nd_blk_region *ndbr = blk_dev->ndbr;
+ struct nd_namespace_blk *nsblk = to_nd_namespace_blk(&ndns->dev);
+ struct nd_blk_region *ndbr = to_ndbr(nsblk);
resource_size_t dev_offset;
dev_offset = to_dev_offset(nsblk, offset, n);
- if (unlikely(offset + n > blk_dev->disk_size)) {
+ if (unlikely(offset + n > nsblk->size)) {
dev_WARN_ONCE(&ndns->dev, 1, "request out of range\n");
return -EFAULT;
}
@@ -235,51 +242,65 @@
.revalidate_disk = nvdimm_revalidate_disk,
};
-static int nd_blk_attach_disk(struct nd_namespace_common *ndns,
- struct nd_blk_device *blk_dev)
+static void nd_blk_release_queue(void *q)
{
+ blk_cleanup_queue(q);
+}
+
+static void nd_blk_release_disk(void *disk)
+{
+ del_gendisk(disk);
+ put_disk(disk);
+}
+
+static int nsblk_attach_disk(struct nd_namespace_blk *nsblk)
+{
+ struct device *dev = &nsblk->common.dev;
resource_size_t available_disk_size;
+ struct request_queue *q;
struct gendisk *disk;
u64 internal_nlba;
- internal_nlba = div_u64(blk_dev->disk_size, blk_dev->internal_lbasize);
- available_disk_size = internal_nlba * blk_dev->sector_size;
+ internal_nlba = div_u64(nsblk->size, nsblk_internal_lbasize(nsblk));
+ available_disk_size = internal_nlba * nsblk_sector_size(nsblk);
- blk_dev->queue = blk_alloc_queue(GFP_KERNEL);
- if (!blk_dev->queue)
+ q = blk_alloc_queue(GFP_KERNEL);
+ if (!q)
return -ENOMEM;
-
- blk_queue_make_request(blk_dev->queue, nd_blk_make_request);
- blk_queue_max_hw_sectors(blk_dev->queue, UINT_MAX);
- blk_queue_bounce_limit(blk_dev->queue, BLK_BOUNCE_ANY);
- blk_queue_logical_block_size(blk_dev->queue, blk_dev->sector_size);
- queue_flag_set_unlocked(QUEUE_FLAG_NONROT, blk_dev->queue);
-
- disk = blk_dev->disk = alloc_disk(0);
- if (!disk) {
- blk_cleanup_queue(blk_dev->queue);
+ if (devm_add_action(dev, nd_blk_release_queue, q)) {
+ blk_cleanup_queue(q);
return -ENOMEM;
}
- disk->driverfs_dev = &ndns->dev;
+ blk_queue_make_request(q, nd_blk_make_request);
+ blk_queue_max_hw_sectors(q, UINT_MAX);
+ blk_queue_bounce_limit(q, BLK_BOUNCE_ANY);
+ blk_queue_logical_block_size(q, nsblk_sector_size(nsblk));
+ queue_flag_set_unlocked(QUEUE_FLAG_NONROT, q);
+ q->queuedata = nsblk;
+
+ disk = alloc_disk(0);
+ if (!disk)
+ return -ENOMEM;
+ if (devm_add_action(dev, nd_blk_release_disk, disk)) {
+ put_disk(disk);
+ return -ENOMEM;
+ }
+
+ disk->driverfs_dev = dev;
disk->first_minor = 0;
disk->fops = &nd_blk_fops;
- disk->private_data = blk_dev;
- disk->queue = blk_dev->queue;
+ disk->queue = q;
disk->flags = GENHD_FL_EXT_DEVT;
- nvdimm_namespace_disk_name(ndns, disk->disk_name);
+ nvdimm_namespace_disk_name(&nsblk->common, disk->disk_name);
set_capacity(disk, 0);
add_disk(disk);
- if (nd_blk_meta_size(blk_dev)) {
- int rc = nd_integrity_init(disk, nd_blk_meta_size(blk_dev));
+ if (nsblk_meta_size(nsblk)) {
+ int rc = nd_integrity_init(disk, nsblk_meta_size(nsblk));
- if (rc) {
- del_gendisk(disk);
- put_disk(disk);
- blk_cleanup_queue(blk_dev->queue);
+ if (rc)
return rc;
- }
}
set_capacity(disk, available_disk_size >> SECTOR_SHIFT);
@@ -291,56 +312,29 @@
{
struct nd_namespace_common *ndns;
struct nd_namespace_blk *nsblk;
- struct nd_blk_device *blk_dev;
- int rc;
ndns = nvdimm_namespace_common_probe(dev);
if (IS_ERR(ndns))
return PTR_ERR(ndns);
- blk_dev = kzalloc(sizeof(*blk_dev), GFP_KERNEL);
- if (!blk_dev)
- return -ENOMEM;
-
nsblk = to_nd_namespace_blk(&ndns->dev);
- blk_dev->disk_size = nvdimm_namespace_capacity(ndns);
- blk_dev->ndbr = to_nd_blk_region(dev->parent);
- blk_dev->nsblk = to_nd_namespace_blk(&ndns->dev);
- blk_dev->internal_lbasize = roundup(nsblk->lbasize,
- INT_LBASIZE_ALIGNMENT);
- blk_dev->sector_size = ((nsblk->lbasize >= 4096) ? 4096 : 512);
- dev_set_drvdata(dev, blk_dev);
+ nsblk->size = nvdimm_namespace_capacity(ndns);
+ dev_set_drvdata(dev, nsblk);
- ndns->rw_bytes = nd_blk_rw_bytes;
+ ndns->rw_bytes = nsblk_rw_bytes;
if (is_nd_btt(dev))
- rc = nvdimm_namespace_attach_btt(ndns);
- else if (nd_btt_probe(ndns, blk_dev) == 0) {
+ return nvdimm_namespace_attach_btt(ndns);
+ else if (nd_btt_probe(dev, ndns) == 0) {
/* we'll come back as btt-blk */
- rc = -ENXIO;
+ return -ENXIO;
} else
- rc = nd_blk_attach_disk(ndns, blk_dev);
- if (rc)
- kfree(blk_dev);
- return rc;
-}
-
-static void nd_blk_detach_disk(struct nd_blk_device *blk_dev)
-{
- del_gendisk(blk_dev->disk);
- put_disk(blk_dev->disk);
- blk_cleanup_queue(blk_dev->queue);
+ return nsblk_attach_disk(nsblk);
}
static int nd_blk_remove(struct device *dev)
{
- struct nd_blk_device *blk_dev = dev_get_drvdata(dev);
-
if (is_nd_btt(dev))
- nvdimm_namespace_detach_btt(to_nd_btt(dev)->ndns);
- else
- nd_blk_detach_disk(blk_dev);
- kfree(blk_dev);
-
+ nvdimm_namespace_detach_btt(to_nd_btt(dev));
return 0;
}
diff --git a/drivers/nvdimm/btt.c b/drivers/nvdimm/btt.c
index f068b65..68a7c3c 100644
--- a/drivers/nvdimm/btt.c
+++ b/drivers/nvdimm/btt.c
@@ -1306,7 +1306,7 @@
struct btt *btt;
struct device *dev = &nd_btt->dev;
- btt = kzalloc(sizeof(struct btt), GFP_KERNEL);
+ btt = devm_kzalloc(dev, sizeof(struct btt), GFP_KERNEL);
if (!btt)
return NULL;
@@ -1321,13 +1321,13 @@
ret = discover_arenas(btt);
if (ret) {
dev_err(dev, "init: error in arena_discover: %d\n", ret);
- goto out_free;
+ return NULL;
}
if (btt->init_state != INIT_READY && nd_region->ro) {
dev_info(dev, "%s is read-only, unable to init btt metadata\n",
dev_name(&nd_region->dev));
- goto out_free;
+ return NULL;
} else if (btt->init_state != INIT_READY) {
btt->num_arenas = (rawsize / ARENA_MAX_SIZE) +
((rawsize % ARENA_MAX_SIZE) ? 1 : 0);
@@ -1337,29 +1337,25 @@
ret = create_arenas(btt);
if (ret) {
dev_info(dev, "init: create_arenas: %d\n", ret);
- goto out_free;
+ return NULL;
}
ret = btt_meta_init(btt);
if (ret) {
dev_err(dev, "init: error in meta_init: %d\n", ret);
- goto out_free;
+ return NULL;
}
}
ret = btt_blk_init(btt);
if (ret) {
dev_err(dev, "init: error in blk_init: %d\n", ret);
- goto out_free;
+ return NULL;
}
btt_debugfs_init(btt);
return btt;
-
- out_free:
- kfree(btt);
- return NULL;
}
/**
@@ -1377,7 +1373,6 @@
btt_blk_cleanup(btt);
free_arenas(btt);
debugfs_remove_recursive(btt->debugfs_dir);
- kfree(btt);
}
}
@@ -1388,11 +1383,15 @@
struct btt *btt;
size_t rawsize;
- if (!nd_btt->uuid || !nd_btt->ndns || !nd_btt->lbasize)
+ if (!nd_btt->uuid || !nd_btt->ndns || !nd_btt->lbasize) {
+ dev_dbg(&nd_btt->dev, "incomplete btt configuration\n");
return -ENODEV;
+ }
rawsize = nvdimm_namespace_capacity(ndns) - SZ_4K;
if (rawsize < ARENA_MIN_SIZE) {
+ dev_dbg(&nd_btt->dev, "%s must be at least %ld bytes\n",
+ dev_name(&ndns->dev), ARENA_MIN_SIZE + SZ_4K);
return -ENXIO;
}
nd_region = to_nd_region(nd_btt->dev.parent);
@@ -1406,9 +1405,8 @@
}
EXPORT_SYMBOL(nvdimm_namespace_attach_btt);
-int nvdimm_namespace_detach_btt(struct nd_namespace_common *ndns)
+int nvdimm_namespace_detach_btt(struct nd_btt *nd_btt)
{
- struct nd_btt *nd_btt = to_nd_btt(ndns->claim);
struct btt *btt = nd_btt->btt;
btt_fini(btt);
diff --git a/drivers/nvdimm/btt_devs.c b/drivers/nvdimm/btt_devs.c
index cb47751..816d0da 100644
--- a/drivers/nvdimm/btt_devs.c
+++ b/drivers/nvdimm/btt_devs.c
@@ -273,10 +273,10 @@
return 0;
}
-int nd_btt_probe(struct nd_namespace_common *ndns, void *drvdata)
+int nd_btt_probe(struct device *dev, struct nd_namespace_common *ndns)
{
int rc;
- struct device *dev;
+ struct device *btt_dev;
struct btt_sb *btt_sb;
struct nd_region *nd_region = to_nd_region(ndns->dev.parent);
@@ -284,21 +284,19 @@
return -ENODEV;
nvdimm_bus_lock(&ndns->dev);
- dev = __nd_btt_create(nd_region, 0, NULL, ndns);
+ btt_dev = __nd_btt_create(nd_region, 0, NULL, ndns);
nvdimm_bus_unlock(&ndns->dev);
- if (!dev)
+ if (!btt_dev)
return -ENOMEM;
- dev_set_drvdata(dev, drvdata);
- btt_sb = kzalloc(sizeof(*btt_sb), GFP_KERNEL);
- rc = __nd_btt_probe(to_nd_btt(dev), ndns, btt_sb);
- kfree(btt_sb);
- dev_dbg(&ndns->dev, "%s: btt: %s\n", __func__,
- rc == 0 ? dev_name(dev) : "<none>");
+ btt_sb = devm_kzalloc(dev, sizeof(*btt_sb), GFP_KERNEL);
+ rc = __nd_btt_probe(to_nd_btt(btt_dev), ndns, btt_sb);
+ dev_dbg(dev, "%s: btt: %s\n", __func__,
+ rc == 0 ? dev_name(btt_dev) : "<none>");
if (rc < 0) {
- struct nd_btt *nd_btt = to_nd_btt(dev);
+ struct nd_btt *nd_btt = to_nd_btt(btt_dev);
- __nd_detach_ndns(dev, &nd_btt->ndns);
- put_device(dev);
+ __nd_detach_ndns(btt_dev, &nd_btt->ndns);
+ put_device(btt_dev);
}
return rc;
diff --git a/drivers/nvdimm/bus.c b/drivers/nvdimm/bus.c
index 19f822d..f085f8b 100644
--- a/drivers/nvdimm/bus.c
+++ b/drivers/nvdimm/bus.c
@@ -40,6 +40,8 @@
return ND_DEVICE_REGION_PMEM;
else if (is_nd_blk(dev))
return ND_DEVICE_REGION_BLK;
+ else if (is_nd_dax(dev))
+ return ND_DEVICE_DAX_PMEM;
else if (is_nd_pmem(dev->parent) || is_nd_blk(dev->parent))
return nd_region_to_nstype(to_nd_region(dev->parent));
@@ -122,9 +124,10 @@
struct nd_device_driver *nd_drv = to_nd_device_driver(dev->driver);
struct module *provider = to_bus_provider(dev);
struct nvdimm_bus *nvdimm_bus = walk_to_nvdimm_bus(dev);
- int rc;
+ int rc = 0;
- rc = nd_drv->remove(dev);
+ if (nd_drv->remove)
+ rc = nd_drv->remove(dev);
nd_region_disable(nvdimm_bus, dev);
dev_dbg(&nvdimm_bus->dev, "%s.remove(%s) = %d\n", dev->driver->name,
@@ -246,6 +249,8 @@
void __nd_device_register(struct device *dev)
{
+ if (!dev)
+ return;
dev->bus = &nvdimm_bus_type;
get_device(dev);
async_schedule_domain(nd_async_device_register, dev,
@@ -292,8 +297,8 @@
return -EINVAL;
}
- if (!nd_drv->probe || !nd_drv->remove) {
- pr_debug("->probe() and ->remove() must be specified\n");
+ if (!nd_drv->probe) {
+ pr_debug("%s ->probe() must be specified\n", mod_name);
return -EINVAL;
}
@@ -439,6 +444,12 @@
.out_num = 3,
.out_sizes = { 4, 4, UINT_MAX, },
},
+ [ND_CMD_CALL] = {
+ .in_num = 2,
+ .in_sizes = { sizeof(struct nd_cmd_pkg), UINT_MAX, },
+ .out_num = 1,
+ .out_sizes = { UINT_MAX, },
+ },
};
const struct nd_cmd_desc *nd_cmd_dimm_desc(int cmd)
@@ -473,6 +484,12 @@
.out_num = 3,
.out_sizes = { 4, 4, 8, },
},
+ [ND_CMD_CALL] = {
+ .in_num = 2,
+ .in_sizes = { sizeof(struct nd_cmd_pkg), UINT_MAX, },
+ .out_num = 1,
+ .out_sizes = { UINT_MAX, },
+ },
};
const struct nd_cmd_desc *nd_cmd_bus_desc(int cmd)
@@ -500,6 +517,10 @@
struct nd_cmd_vendor_hdr *hdr = buf;
return hdr->in_length;
+ } else if (cmd == ND_CMD_CALL) {
+ struct nd_cmd_pkg *pkg = buf;
+
+ return pkg->nd_size_in;
}
return UINT_MAX;
@@ -522,6 +543,12 @@
return out_field[1];
else if (!nvdimm && cmd == ND_CMD_ARS_STATUS && idx == 2)
return out_field[1] - 8;
+ else if (cmd == ND_CMD_CALL) {
+ struct nd_cmd_pkg *pkg = (struct nd_cmd_pkg *) in_field;
+
+ return pkg->nd_size_out;
+ }
+
return UINT_MAX;
}
@@ -588,25 +615,31 @@
unsigned int cmd = _IOC_NR(ioctl_cmd);
void __user *p = (void __user *) arg;
struct device *dev = &nvdimm_bus->dev;
+ struct nd_cmd_pkg pkg;
const char *cmd_name, *dimm_name;
- unsigned long dsm_mask;
+ unsigned long cmd_mask;
void *buf;
int rc, i;
if (nvdimm) {
desc = nd_cmd_dimm_desc(cmd);
cmd_name = nvdimm_cmd_name(cmd);
- dsm_mask = nvdimm->dsm_mask ? *(nvdimm->dsm_mask) : 0;
+ cmd_mask = nvdimm->cmd_mask;
dimm_name = dev_name(&nvdimm->dev);
} else {
desc = nd_cmd_bus_desc(cmd);
cmd_name = nvdimm_bus_cmd_name(cmd);
- dsm_mask = nd_desc->dsm_mask;
+ cmd_mask = nd_desc->cmd_mask;
dimm_name = "bus";
}
+ if (cmd == ND_CMD_CALL) {
+ if (copy_from_user(&pkg, p, sizeof(pkg)))
+ return -EFAULT;
+ }
+
if (!desc || (desc->out_num + desc->in_num == 0) ||
- !test_bit(cmd, &dsm_mask))
+ !test_bit(cmd, &cmd_mask))
return -ENOTTY;
/* fail write commands (when read-only) */
@@ -616,6 +649,7 @@
case ND_CMD_SET_CONFIG_DATA:
case ND_CMD_ARS_START:
case ND_CMD_CLEAR_ERROR:
+ case ND_CMD_CALL:
dev_dbg(&nvdimm_bus->dev, "'%s' command while read-only.\n",
nvdimm ? nvdimm_cmd_name(cmd)
: nvdimm_bus_cmd_name(cmd));
@@ -643,6 +677,16 @@
in_len += in_size;
}
+ if (cmd == ND_CMD_CALL) {
+ dev_dbg(dev, "%s:%s, idx: %llu, in: %zu, out: %zu, len %zu\n",
+ __func__, dimm_name, pkg.nd_command,
+ in_len, out_len, buf_len);
+
+ for (i = 0; i < ARRAY_SIZE(pkg.nd_reserved2); i++)
+ if (pkg.nd_reserved2[i])
+ return -EINVAL;
+ }
+
/* process an output envelope */
for (i = 0; i < desc->out_num; i++) {
u32 out_size = nd_cmd_out_size(nvdimm, cmd, desc, i,
@@ -783,6 +827,9 @@
{
int rc;
+ BUILD_BUG_ON(sizeof(struct nd_smart_payload) != 128);
+ BUILD_BUG_ON(sizeof(struct nd_smart_threshold_payload) != 8);
+
rc = bus_register(&nvdimm_bus_type);
if (rc)
return rc;
diff --git a/drivers/nvdimm/claim.c b/drivers/nvdimm/claim.c
index e8f03b0..8b2e3c4 100644
--- a/drivers/nvdimm/claim.c
+++ b/drivers/nvdimm/claim.c
@@ -12,6 +12,7 @@
*/
#include <linux/device.h>
#include <linux/sizes.h>
+#include <linux/pmem.h>
#include "nd-core.h"
#include "pfn.h"
#include "btt.h"
@@ -84,12 +85,33 @@
seed = nd_region->btt_seed;
else if (is_nd_pfn(dev))
seed = nd_region->pfn_seed;
+ else if (is_nd_dax(dev))
+ seed = nd_region->dax_seed;
if (seed == dev || ndns || dev->driver)
return false;
return true;
}
+struct nd_pfn *to_nd_pfn_safe(struct device *dev)
+{
+ /*
+ * pfn device attributes are re-used by dax device instances, so we
+ * need to be careful to correct device-to-nd_pfn conversion.
+ */
+ if (is_nd_pfn(dev))
+ return to_nd_pfn(dev);
+
+ if (is_nd_dax(dev)) {
+ struct nd_dax *nd_dax = to_nd_dax(dev);
+
+ return &nd_dax->nd_pfn;
+ }
+
+ WARN_ON(1);
+ return NULL;
+}
+
static void nd_detach_and_reset(struct device *dev,
struct nd_namespace_common **_ndns)
{
@@ -103,8 +125,8 @@
nd_btt->lbasize = 0;
kfree(nd_btt->uuid);
nd_btt->uuid = NULL;
- } else if (is_nd_pfn(dev)) {
- struct nd_pfn *nd_pfn = to_nd_pfn(dev);
+ } else if (is_nd_pfn(dev) || is_nd_dax(dev)) {
+ struct nd_pfn *nd_pfn = to_nd_pfn_safe(dev);
kfree(nd_pfn->uuid);
nd_pfn->uuid = NULL;
@@ -199,3 +221,63 @@
return sum;
}
EXPORT_SYMBOL(nd_sb_checksum);
+
+static int nsio_rw_bytes(struct nd_namespace_common *ndns,
+ resource_size_t offset, void *buf, size_t size, int rw)
+{
+ struct nd_namespace_io *nsio = to_nd_namespace_io(&ndns->dev);
+
+ if (unlikely(offset + size > nsio->size)) {
+ dev_WARN_ONCE(&ndns->dev, 1, "request out of range\n");
+ return -EFAULT;
+ }
+
+ if (rw == READ) {
+ unsigned int sz_align = ALIGN(size + (offset & (512 - 1)), 512);
+
+ if (unlikely(is_bad_pmem(&nsio->bb, offset / 512, sz_align)))
+ return -EIO;
+ return memcpy_from_pmem(buf, nsio->addr + offset, size);
+ } else {
+ memcpy_to_pmem(nsio->addr + offset, buf, size);
+ wmb_pmem();
+ }
+
+ return 0;
+}
+
+int devm_nsio_enable(struct device *dev, struct nd_namespace_io *nsio)
+{
+ struct resource *res = &nsio->res;
+ struct nd_namespace_common *ndns = &nsio->common;
+
+ nsio->size = resource_size(res);
+ if (!devm_request_mem_region(dev, res->start, resource_size(res),
+ dev_name(dev))) {
+ dev_warn(dev, "could not reserve region %pR\n", res);
+ return -EBUSY;
+ }
+
+ ndns->rw_bytes = nsio_rw_bytes;
+ if (devm_init_badblocks(dev, &nsio->bb))
+ return -ENOMEM;
+ nvdimm_badblocks_populate(to_nd_region(ndns->dev.parent), &nsio->bb,
+ &nsio->res);
+
+ nsio->addr = devm_memremap(dev, res->start, resource_size(res),
+ ARCH_MEMREMAP_PMEM);
+ if (IS_ERR(nsio->addr))
+ return PTR_ERR(nsio->addr);
+ return 0;
+}
+EXPORT_SYMBOL_GPL(devm_nsio_enable);
+
+void devm_nsio_disable(struct device *dev, struct nd_namespace_io *nsio)
+{
+ struct resource *res = &nsio->res;
+
+ devm_memunmap(dev, nsio->addr);
+ devm_exit_badblocks(dev, &nsio->bb);
+ devm_release_mem_region(dev, res->start, resource_size(res));
+}
+EXPORT_SYMBOL_GPL(devm_nsio_disable);
diff --git a/drivers/nvdimm/core.c b/drivers/nvdimm/core.c
index 182a93f..be89764 100644
--- a/drivers/nvdimm/core.c
+++ b/drivers/nvdimm/core.c
@@ -251,7 +251,7 @@
struct nvdimm_bus *nvdimm_bus = to_nvdimm_bus(dev);
struct nvdimm_bus_descriptor *nd_desc = nvdimm_bus->nd_desc;
- for_each_set_bit(cmd, &nd_desc->dsm_mask, BITS_PER_LONG)
+ for_each_set_bit(cmd, &nd_desc->cmd_mask, BITS_PER_LONG)
len += sprintf(buf + len, "%s ", nvdimm_bus_cmd_name(cmd));
len += sprintf(buf + len, "\n");
return len;
@@ -648,6 +648,9 @@
nd_region_exit();
nvdimm_exit();
nvdimm_bus_exit();
+ nd_region_devs_exit();
+ nvdimm_devs_exit();
+ ida_destroy(&nd_ida);
}
MODULE_LICENSE("GPL v2");
diff --git a/drivers/nvdimm/dax_devs.c b/drivers/nvdimm/dax_devs.c
new file mode 100644
index 0000000..45fa82c
--- /dev/null
+++ b/drivers/nvdimm/dax_devs.c
@@ -0,0 +1,134 @@
+/*
+ * Copyright(c) 2013-2016 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ */
+#include <linux/device.h>
+#include <linux/sizes.h>
+#include <linux/slab.h>
+#include <linux/mm.h>
+#include "nd-core.h"
+#include "pfn.h"
+#include "nd.h"
+
+static void nd_dax_release(struct device *dev)
+{
+ struct nd_region *nd_region = to_nd_region(dev->parent);
+ struct nd_dax *nd_dax = to_nd_dax(dev);
+ struct nd_pfn *nd_pfn = &nd_dax->nd_pfn;
+
+ dev_dbg(dev, "%s\n", __func__);
+ nd_detach_ndns(dev, &nd_pfn->ndns);
+ ida_simple_remove(&nd_region->dax_ida, nd_pfn->id);
+ kfree(nd_pfn->uuid);
+ kfree(nd_dax);
+}
+
+static struct device_type nd_dax_device_type = {
+ .name = "nd_dax",
+ .release = nd_dax_release,
+};
+
+bool is_nd_dax(struct device *dev)
+{
+ return dev ? dev->type == &nd_dax_device_type : false;
+}
+EXPORT_SYMBOL(is_nd_dax);
+
+struct nd_dax *to_nd_dax(struct device *dev)
+{
+ struct nd_dax *nd_dax = container_of(dev, struct nd_dax, nd_pfn.dev);
+
+ WARN_ON(!is_nd_dax(dev));
+ return nd_dax;
+}
+EXPORT_SYMBOL(to_nd_dax);
+
+static const struct attribute_group *nd_dax_attribute_groups[] = {
+ &nd_pfn_attribute_group,
+ &nd_device_attribute_group,
+ &nd_numa_attribute_group,
+ NULL,
+};
+
+static struct nd_dax *nd_dax_alloc(struct nd_region *nd_region)
+{
+ struct nd_pfn *nd_pfn;
+ struct nd_dax *nd_dax;
+ struct device *dev;
+
+ nd_dax = kzalloc(sizeof(*nd_dax), GFP_KERNEL);
+ if (!nd_dax)
+ return NULL;
+
+ nd_pfn = &nd_dax->nd_pfn;
+ nd_pfn->id = ida_simple_get(&nd_region->dax_ida, 0, 0, GFP_KERNEL);
+ if (nd_pfn->id < 0) {
+ kfree(nd_dax);
+ return NULL;
+ }
+
+ dev = &nd_pfn->dev;
+ dev_set_name(dev, "dax%d.%d", nd_region->id, nd_pfn->id);
+ dev->groups = nd_dax_attribute_groups;
+ dev->type = &nd_dax_device_type;
+ dev->parent = &nd_region->dev;
+
+ return nd_dax;
+}
+
+struct device *nd_dax_create(struct nd_region *nd_region)
+{
+ struct device *dev = NULL;
+ struct nd_dax *nd_dax;
+
+ if (!is_nd_pmem(&nd_region->dev))
+ return NULL;
+
+ nd_dax = nd_dax_alloc(nd_region);
+ if (nd_dax)
+ dev = nd_pfn_devinit(&nd_dax->nd_pfn, NULL);
+ __nd_device_register(dev);
+ return dev;
+}
+
+int nd_dax_probe(struct device *dev, struct nd_namespace_common *ndns)
+{
+ int rc;
+ struct nd_dax *nd_dax;
+ struct device *dax_dev;
+ struct nd_pfn *nd_pfn;
+ struct nd_pfn_sb *pfn_sb;
+ struct nd_region *nd_region = to_nd_region(ndns->dev.parent);
+
+ if (ndns->force_raw)
+ return -ENODEV;
+
+ nvdimm_bus_lock(&ndns->dev);
+ nd_dax = nd_dax_alloc(nd_region);
+ nd_pfn = &nd_dax->nd_pfn;
+ dax_dev = nd_pfn_devinit(nd_pfn, ndns);
+ nvdimm_bus_unlock(&ndns->dev);
+ if (!dax_dev)
+ return -ENOMEM;
+ pfn_sb = devm_kzalloc(dev, sizeof(*pfn_sb), GFP_KERNEL);
+ nd_pfn->pfn_sb = pfn_sb;
+ rc = nd_pfn_validate(nd_pfn, DAX_SIG);
+ dev_dbg(dev, "%s: dax: %s\n", __func__,
+ rc == 0 ? dev_name(dax_dev) : "<none>");
+ if (rc < 0) {
+ __nd_detach_ndns(dax_dev, &nd_pfn->ndns);
+ put_device(dax_dev);
+ } else
+ __nd_device_register(dax_dev);
+
+ return rc;
+}
+EXPORT_SYMBOL(nd_dax_probe);
diff --git a/drivers/nvdimm/dimm_devs.c b/drivers/nvdimm/dimm_devs.c
index c56f882..bbde28d 100644
--- a/drivers/nvdimm/dimm_devs.c
+++ b/drivers/nvdimm/dimm_devs.c
@@ -37,9 +37,9 @@
nvdimm = to_nvdimm(ndd->dev);
- if (!nvdimm->dsm_mask)
+ if (!nvdimm->cmd_mask)
return -ENXIO;
- if (!test_bit(ND_CMD_GET_CONFIG_DATA, nvdimm->dsm_mask))
+ if (!test_bit(ND_CMD_GET_CONFIG_DATA, &nvdimm->cmd_mask))
return -ENXIO;
return 0;
@@ -263,6 +263,12 @@
}
EXPORT_SYMBOL_GPL(nvdimm_name);
+unsigned long nvdimm_cmd_mask(struct nvdimm *nvdimm)
+{
+ return nvdimm->cmd_mask;
+}
+EXPORT_SYMBOL_GPL(nvdimm_cmd_mask);
+
void *nvdimm_provider_data(struct nvdimm *nvdimm)
{
if (nvdimm)
@@ -277,10 +283,10 @@
struct nvdimm *nvdimm = to_nvdimm(dev);
int cmd, len = 0;
- if (!nvdimm->dsm_mask)
+ if (!nvdimm->cmd_mask)
return sprintf(buf, "\n");
- for_each_set_bit(cmd, nvdimm->dsm_mask, BITS_PER_LONG)
+ for_each_set_bit(cmd, &nvdimm->cmd_mask, BITS_PER_LONG)
len += sprintf(buf + len, "%s ", nvdimm_cmd_name(cmd));
len += sprintf(buf + len, "\n");
return len;
@@ -340,7 +346,7 @@
struct nvdimm *nvdimm_create(struct nvdimm_bus *nvdimm_bus, void *provider_data,
const struct attribute_group **groups, unsigned long flags,
- unsigned long *dsm_mask)
+ unsigned long cmd_mask)
{
struct nvdimm *nvdimm = kzalloc(sizeof(*nvdimm), GFP_KERNEL);
struct device *dev;
@@ -355,7 +361,7 @@
}
nvdimm->provider_data = provider_data;
nvdimm->flags = flags;
- nvdimm->dsm_mask = dsm_mask;
+ nvdimm->cmd_mask = cmd_mask;
atomic_set(&nvdimm->busy, 0);
dev = &nvdimm->dev;
dev_set_name(dev, "nmem%d", nvdimm->id);
@@ -546,3 +552,8 @@
return 0;
}
EXPORT_SYMBOL_GPL(nvdimm_bus_check_dimm_count);
+
+void __exit nvdimm_devs_exit(void)
+{
+ ida_destroy(&dimm_ida);
+}
diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c
index f5cb886..c5e3196 100644
--- a/drivers/nvdimm/namespace_devs.c
+++ b/drivers/nvdimm/namespace_devs.c
@@ -1288,6 +1288,8 @@
mode = "safe";
else if (claim && is_nd_pfn(claim))
mode = "memory";
+ else if (claim && is_nd_dax(claim))
+ mode = "dax";
else if (!claim && pmem_should_map_pages(dev))
mode = "memory";
else
@@ -1379,21 +1381,19 @@
{
struct nd_btt *nd_btt = is_nd_btt(dev) ? to_nd_btt(dev) : NULL;
struct nd_pfn *nd_pfn = is_nd_pfn(dev) ? to_nd_pfn(dev) : NULL;
- struct nd_namespace_common *ndns;
+ struct nd_dax *nd_dax = is_nd_dax(dev) ? to_nd_dax(dev) : NULL;
+ struct nd_namespace_common *ndns = NULL;
resource_size_t size;
- if (nd_btt || nd_pfn) {
- struct device *host = NULL;
-
- if (nd_btt) {
- host = &nd_btt->dev;
+ if (nd_btt || nd_pfn || nd_dax) {
+ if (nd_btt)
ndns = nd_btt->ndns;
- } else if (nd_pfn) {
- host = &nd_pfn->dev;
+ else if (nd_pfn)
ndns = nd_pfn->ndns;
- }
+ else if (nd_dax)
+ ndns = nd_dax->nd_pfn.ndns;
- if (!ndns || !host)
+ if (!ndns)
return ERR_PTR(-ENODEV);
/*
@@ -1404,12 +1404,12 @@
device_unlock(&ndns->dev);
if (ndns->dev.driver) {
dev_dbg(&ndns->dev, "is active, can't bind %s\n",
- dev_name(host));
+ dev_name(dev));
return ERR_PTR(-EBUSY);
}
- if (dev_WARN_ONCE(&ndns->dev, ndns->claim != host,
+ if (dev_WARN_ONCE(&ndns->dev, ndns->claim != dev,
"host (%s) vs claim (%s) mismatch\n",
- dev_name(host),
+ dev_name(dev),
dev_name(ndns->claim)))
return ERR_PTR(-ENXIO);
} else {
@@ -1784,6 +1784,18 @@
nd_device_register(nd_region->ns_seed);
}
+void nd_region_create_dax_seed(struct nd_region *nd_region)
+{
+ WARN_ON(!is_nvdimm_bus_locked(&nd_region->dev));
+ nd_region->dax_seed = nd_dax_create(nd_region);
+ /*
+ * Seed creation failures are not fatal, provisioning is simply
+ * disabled until memory becomes available
+ */
+ if (!nd_region->dax_seed)
+ dev_err(&nd_region->dev, "failed to create dax namespace\n");
+}
+
void nd_region_create_pfn_seed(struct nd_region *nd_region)
{
WARN_ON(!is_nvdimm_bus_locked(&nd_region->dev));
diff --git a/drivers/nvdimm/nd-core.h b/drivers/nvdimm/nd-core.h
index 1d1500f..284cdaa 100644
--- a/drivers/nvdimm/nd-core.h
+++ b/drivers/nvdimm/nd-core.h
@@ -37,7 +37,7 @@
struct nvdimm {
unsigned long flags;
void *provider_data;
- unsigned long *dsm_mask;
+ unsigned long cmd_mask;
struct device dev;
atomic_t busy;
int id;
@@ -49,11 +49,14 @@
struct nvdimm_bus *walk_to_nvdimm_bus(struct device *nd_dev);
int __init nvdimm_bus_init(void);
void nvdimm_bus_exit(void);
+void nvdimm_devs_exit(void);
+void nd_region_devs_exit(void);
void nd_region_probe_success(struct nvdimm_bus *nvdimm_bus, struct device *dev);
struct nd_region;
void nd_region_create_blk_seed(struct nd_region *nd_region);
void nd_region_create_btt_seed(struct nd_region *nd_region);
void nd_region_create_pfn_seed(struct nd_region *nd_region);
+void nd_region_create_dax_seed(struct nd_region *nd_region);
void nd_region_disable(struct nvdimm_bus *nvdimm_bus, struct device *dev);
int nvdimm_bus_create_ndctl(struct nvdimm_bus *nvdimm_bus);
void nvdimm_bus_destroy_ndctl(struct nvdimm_bus *nvdimm_bus);
@@ -91,4 +94,5 @@
ssize_t nd_namespace_store(struct device *dev,
struct nd_namespace_common **_ndns, const char *buf,
size_t len);
+struct nd_pfn *to_nd_pfn_safe(struct device *dev);
#endif /* __ND_CORE_H__ */
diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
index 875c524..d0ac93c 100644
--- a/drivers/nvdimm/nd.h
+++ b/drivers/nvdimm/nd.h
@@ -13,6 +13,7 @@
#ifndef __ND_H__
#define __ND_H__
#include <linux/libnvdimm.h>
+#include <linux/badblocks.h>
#include <linux/blkdev.h>
#include <linux/device.h>
#include <linux/mutex.h>
@@ -100,10 +101,12 @@
struct ida ns_ida;
struct ida btt_ida;
struct ida pfn_ida;
+ struct ida dax_ida;
unsigned long flags;
struct device *ns_seed;
struct device *btt_seed;
struct device *pfn_seed;
+ struct device *dax_seed;
u16 ndr_mappings;
u64 ndr_size;
u64 ndr_start;
@@ -160,6 +163,10 @@
struct nd_namespace_common *ndns;
};
+struct nd_dax {
+ struct nd_pfn nd_pfn;
+};
+
enum nd_async_mode {
ND_SYNC,
ND_ASYNC,
@@ -197,11 +204,12 @@
u64 nd_sb_checksum(struct nd_gen_sb *sb);
#if IS_ENABLED(CONFIG_BTT)
-int nd_btt_probe(struct nd_namespace_common *ndns, void *drvdata);
+int nd_btt_probe(struct device *dev, struct nd_namespace_common *ndns);
bool is_nd_btt(struct device *dev);
struct device *nd_btt_create(struct nd_region *nd_region);
#else
-static inline int nd_btt_probe(struct nd_namespace_common *ndns, void *drvdata)
+static inline int nd_btt_probe(struct device *dev,
+ struct nd_namespace_common *ndns)
{
return -ENODEV;
}
@@ -219,12 +227,16 @@
struct nd_pfn *to_nd_pfn(struct device *dev);
#if IS_ENABLED(CONFIG_NVDIMM_PFN)
-int nd_pfn_probe(struct nd_namespace_common *ndns, void *drvdata);
+int nd_pfn_probe(struct device *dev, struct nd_namespace_common *ndns);
bool is_nd_pfn(struct device *dev);
struct device *nd_pfn_create(struct nd_region *nd_region);
-int nd_pfn_validate(struct nd_pfn *nd_pfn);
+struct device *nd_pfn_devinit(struct nd_pfn *nd_pfn,
+ struct nd_namespace_common *ndns);
+int nd_pfn_validate(struct nd_pfn *nd_pfn, const char *sig);
+extern struct attribute_group nd_pfn_attribute_group;
#else
-static inline int nd_pfn_probe(struct nd_namespace_common *ndns, void *drvdata)
+static inline int nd_pfn_probe(struct device *dev,
+ struct nd_namespace_common *ndns)
{
return -ENODEV;
}
@@ -239,12 +251,35 @@
return NULL;
}
-static inline int nd_pfn_validate(struct nd_pfn *nd_pfn)
+static inline int nd_pfn_validate(struct nd_pfn *nd_pfn, const char *sig)
{
return -ENODEV;
}
#endif
+struct nd_dax *to_nd_dax(struct device *dev);
+#if IS_ENABLED(CONFIG_NVDIMM_DAX)
+int nd_dax_probe(struct device *dev, struct nd_namespace_common *ndns);
+bool is_nd_dax(struct device *dev);
+struct device *nd_dax_create(struct nd_region *nd_region);
+#else
+static inline int nd_dax_probe(struct device *dev,
+ struct nd_namespace_common *ndns)
+{
+ return -ENODEV;
+}
+
+static inline bool is_nd_dax(struct device *dev)
+{
+ return false;
+}
+
+static inline struct device *nd_dax_create(struct nd_region *nd_region)
+{
+ return NULL;
+}
+#endif
+
struct nd_region *to_nd_region(struct device *dev);
int nd_region_to_nstype(struct nd_region *nd_region);
int nd_region_register_namespaces(struct nd_region *nd_region, int *err);
@@ -263,11 +298,32 @@
resource_size_t nvdimm_namespace_capacity(struct nd_namespace_common *ndns);
struct nd_namespace_common *nvdimm_namespace_common_probe(struct device *dev);
int nvdimm_namespace_attach_btt(struct nd_namespace_common *ndns);
-int nvdimm_namespace_detach_btt(struct nd_namespace_common *ndns);
+int nvdimm_namespace_detach_btt(struct nd_btt *nd_btt);
const char *nvdimm_namespace_disk_name(struct nd_namespace_common *ndns,
char *name);
void nvdimm_badblocks_populate(struct nd_region *nd_region,
struct badblocks *bb, const struct resource *res);
+#if IS_ENABLED(CONFIG_ND_CLAIM)
+struct vmem_altmap *nvdimm_setup_pfn(struct nd_pfn *nd_pfn,
+ struct resource *res, struct vmem_altmap *altmap);
+int devm_nsio_enable(struct device *dev, struct nd_namespace_io *nsio);
+void devm_nsio_disable(struct device *dev, struct nd_namespace_io *nsio);
+#else
+static inline struct vmem_altmap *nvdimm_setup_pfn(struct nd_pfn *nd_pfn,
+ struct resource *res, struct vmem_altmap *altmap)
+{
+ return ERR_PTR(-ENXIO);
+}
+static inline int devm_nsio_enable(struct device *dev,
+ struct nd_namespace_io *nsio)
+{
+ return -ENXIO;
+}
+static inline void devm_nsio_disable(struct device *dev,
+ struct nd_namespace_io *nsio)
+{
+}
+#endif
int nd_blk_region_init(struct nd_region *nd_region);
void __nd_iostat_start(struct bio *bio, unsigned long *start);
static inline bool nd_iostat_start(struct bio *bio, unsigned long *start)
@@ -281,6 +337,19 @@
return true;
}
void nd_iostat_end(struct bio *bio, unsigned long start);
+static inline bool is_bad_pmem(struct badblocks *bb, sector_t sector,
+ unsigned int len)
+{
+ if (bb->count) {
+ sector_t first_bad;
+ int num_bad;
+
+ return !!badblocks_check(bb, sector, len / 512, &first_bad,
+ &num_bad);
+ }
+
+ return false;
+}
resource_size_t nd_namespace_blk_validate(struct nd_namespace_blk *nsblk);
const u8 *nd_dev_to_uuid(struct device *dev);
bool pmem_should_map_pages(struct device *dev);
diff --git a/drivers/nvdimm/pfn.h b/drivers/nvdimm/pfn.h
index 8e343a3..dde9853 100644
--- a/drivers/nvdimm/pfn.h
+++ b/drivers/nvdimm/pfn.h
@@ -19,6 +19,7 @@
#define PFN_SIG_LEN 16
#define PFN_SIG "NVDIMM_PFN_INFO\0"
+#define DAX_SIG "NVDIMM_DAX_INFO\0"
struct nd_pfn_sb {
u8 signature[PFN_SIG_LEN];
@@ -33,7 +34,9 @@
/* minor-version-1 additions for section alignment */
__le32 start_pad;
__le32 end_trunc;
- u8 padding[4004];
+ /* minor-version-2 record the base alignment of the mapping */
+ __le32 align;
+ u8 padding[4000];
__le64 checksum;
};
diff --git a/drivers/nvdimm/pfn_devs.c b/drivers/nvdimm/pfn_devs.c
index e071e21..f7718ec 100644
--- a/drivers/nvdimm/pfn_devs.c
+++ b/drivers/nvdimm/pfn_devs.c
@@ -1,5 +1,5 @@
/*
- * Copyright(c) 2013-2015 Intel Corporation. All rights reserved.
+ * Copyright(c) 2013-2016 Intel Corporation. All rights reserved.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
@@ -10,6 +10,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* General Public License for more details.
*/
+#include <linux/memremap.h>
#include <linux/blkdev.h>
#include <linux/device.h>
#include <linux/genhd.h>
@@ -56,7 +57,7 @@
static ssize_t mode_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
- struct nd_pfn *nd_pfn = to_nd_pfn(dev);
+ struct nd_pfn *nd_pfn = to_nd_pfn_safe(dev);
switch (nd_pfn->mode) {
case PFN_MODE_RAM:
@@ -71,7 +72,7 @@
static ssize_t mode_store(struct device *dev,
struct device_attribute *attr, const char *buf, size_t len)
{
- struct nd_pfn *nd_pfn = to_nd_pfn(dev);
+ struct nd_pfn *nd_pfn = to_nd_pfn_safe(dev);
ssize_t rc = 0;
device_lock(dev);
@@ -105,7 +106,7 @@
static ssize_t align_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
- struct nd_pfn *nd_pfn = to_nd_pfn(dev);
+ struct nd_pfn *nd_pfn = to_nd_pfn_safe(dev);
return sprintf(buf, "%lx\n", nd_pfn->align);
}
@@ -133,7 +134,7 @@
static ssize_t align_store(struct device *dev,
struct device_attribute *attr, const char *buf, size_t len)
{
- struct nd_pfn *nd_pfn = to_nd_pfn(dev);
+ struct nd_pfn *nd_pfn = to_nd_pfn_safe(dev);
ssize_t rc;
device_lock(dev);
@@ -151,7 +152,7 @@
static ssize_t uuid_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
- struct nd_pfn *nd_pfn = to_nd_pfn(dev);
+ struct nd_pfn *nd_pfn = to_nd_pfn_safe(dev);
if (nd_pfn->uuid)
return sprintf(buf, "%pUb\n", nd_pfn->uuid);
@@ -161,7 +162,7 @@
static ssize_t uuid_store(struct device *dev,
struct device_attribute *attr, const char *buf, size_t len)
{
- struct nd_pfn *nd_pfn = to_nd_pfn(dev);
+ struct nd_pfn *nd_pfn = to_nd_pfn_safe(dev);
ssize_t rc;
device_lock(dev);
@@ -177,7 +178,7 @@
static ssize_t namespace_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
- struct nd_pfn *nd_pfn = to_nd_pfn(dev);
+ struct nd_pfn *nd_pfn = to_nd_pfn_safe(dev);
ssize_t rc;
nvdimm_bus_lock(dev);
@@ -190,7 +191,7 @@
static ssize_t namespace_store(struct device *dev,
struct device_attribute *attr, const char *buf, size_t len)
{
- struct nd_pfn *nd_pfn = to_nd_pfn(dev);
+ struct nd_pfn *nd_pfn = to_nd_pfn_safe(dev);
ssize_t rc;
device_lock(dev);
@@ -208,7 +209,7 @@
static ssize_t resource_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
- struct nd_pfn *nd_pfn = to_nd_pfn(dev);
+ struct nd_pfn *nd_pfn = to_nd_pfn_safe(dev);
ssize_t rc;
device_lock(dev);
@@ -234,7 +235,7 @@
static ssize_t size_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
- struct nd_pfn *nd_pfn = to_nd_pfn(dev);
+ struct nd_pfn *nd_pfn = to_nd_pfn_safe(dev);
ssize_t rc;
device_lock(dev);
@@ -269,7 +270,7 @@
NULL,
};
-static struct attribute_group nd_pfn_attribute_group = {
+struct attribute_group nd_pfn_attribute_group = {
.attrs = nd_pfn_attributes,
};
@@ -280,16 +281,32 @@
NULL,
};
-static struct device *__nd_pfn_create(struct nd_region *nd_region,
+struct device *nd_pfn_devinit(struct nd_pfn *nd_pfn,
struct nd_namespace_common *ndns)
{
+ struct device *dev = &nd_pfn->dev;
+
+ if (!nd_pfn)
+ return NULL;
+
+ nd_pfn->mode = PFN_MODE_NONE;
+ nd_pfn->align = HPAGE_SIZE;
+ dev = &nd_pfn->dev;
+ device_initialize(&nd_pfn->dev);
+ if (ndns && !__nd_attach_ndns(&nd_pfn->dev, ndns, &nd_pfn->ndns)) {
+ dev_dbg(&ndns->dev, "%s failed, already claimed by %s\n",
+ __func__, dev_name(ndns->claim));
+ put_device(dev);
+ return NULL;
+ }
+ return dev;
+}
+
+static struct nd_pfn *nd_pfn_alloc(struct nd_region *nd_region)
+{
struct nd_pfn *nd_pfn;
struct device *dev;
- /* we can only create pages for contiguous ranged of pmem */
- if (!is_nd_pmem(&nd_region->dev))
- return NULL;
-
nd_pfn = kzalloc(sizeof(*nd_pfn), GFP_KERNEL);
if (!nd_pfn)
return NULL;
@@ -300,33 +317,31 @@
return NULL;
}
- nd_pfn->mode = PFN_MODE_NONE;
- nd_pfn->align = HPAGE_SIZE;
dev = &nd_pfn->dev;
dev_set_name(dev, "pfn%d.%d", nd_region->id, nd_pfn->id);
- dev->parent = &nd_region->dev;
- dev->type = &nd_pfn_device_type;
dev->groups = nd_pfn_attribute_groups;
- device_initialize(&nd_pfn->dev);
- if (ndns && !__nd_attach_ndns(&nd_pfn->dev, ndns, &nd_pfn->ndns)) {
- dev_dbg(&ndns->dev, "%s failed, already claimed by %s\n",
- __func__, dev_name(ndns->claim));
- put_device(dev);
- return NULL;
- }
- return dev;
+ dev->type = &nd_pfn_device_type;
+ dev->parent = &nd_region->dev;
+
+ return nd_pfn;
}
struct device *nd_pfn_create(struct nd_region *nd_region)
{
- struct device *dev = __nd_pfn_create(nd_region, NULL);
+ struct nd_pfn *nd_pfn;
+ struct device *dev;
- if (dev)
- __nd_device_register(dev);
+ if (!is_nd_pmem(&nd_region->dev))
+ return NULL;
+
+ nd_pfn = nd_pfn_alloc(nd_region);
+ dev = nd_pfn_devinit(nd_pfn, NULL);
+
+ __nd_device_register(dev);
return dev;
}
-int nd_pfn_validate(struct nd_pfn *nd_pfn)
+int nd_pfn_validate(struct nd_pfn *nd_pfn, const char *sig)
{
u64 checksum, offset;
struct nd_namespace_io *nsio;
@@ -343,7 +358,7 @@
if (nvdimm_read_bytes(ndns, SZ_4K, pfn_sb, sizeof(*pfn_sb)))
return -ENXIO;
- if (memcmp(pfn_sb->signature, PFN_SIG, PFN_SIG_LEN) != 0)
+ if (memcmp(pfn_sb->signature, sig, PFN_SIG_LEN) != 0)
return -ENODEV;
checksum = le64_to_cpu(pfn_sb->checksum);
@@ -360,6 +375,9 @@
pfn_sb->end_trunc = 0;
}
+ if (__le16_to_cpu(pfn_sb->version_minor) < 2)
+ pfn_sb->align = 0;
+
switch (le32_to_cpu(pfn_sb->mode)) {
case PFN_MODE_RAM:
case PFN_MODE_PMEM:
@@ -379,6 +397,8 @@
return -ENODEV;
}
+ if (nd_pfn->align == 0)
+ nd_pfn->align = le32_to_cpu(pfn_sb->align);
if (nd_pfn->align > nvdimm_namespace_capacity(ndns)) {
dev_err(&nd_pfn->dev, "alignment: %lx exceeds capacity %llx\n",
nd_pfn->align, nvdimm_namespace_capacity(ndns));
@@ -399,8 +419,8 @@
return -EBUSY;
}
- nd_pfn->align = 1UL << ilog2(offset);
- if (!is_power_of_2(offset) || offset < PAGE_SIZE) {
+ if ((nd_pfn->align && !IS_ALIGNED(offset, nd_pfn->align))
+ || !IS_ALIGNED(offset, PAGE_SIZE)) {
dev_err(&nd_pfn->dev, "bad offset: %#llx dax disabled\n",
offset);
return -ENXIO;
@@ -410,11 +430,11 @@
}
EXPORT_SYMBOL(nd_pfn_validate);
-int nd_pfn_probe(struct nd_namespace_common *ndns, void *drvdata)
+int nd_pfn_probe(struct device *dev, struct nd_namespace_common *ndns)
{
int rc;
- struct device *dev;
struct nd_pfn *nd_pfn;
+ struct device *pfn_dev;
struct nd_pfn_sb *pfn_sb;
struct nd_region *nd_region = to_nd_region(ndns->dev.parent);
@@ -422,25 +442,218 @@
return -ENODEV;
nvdimm_bus_lock(&ndns->dev);
- dev = __nd_pfn_create(nd_region, ndns);
+ nd_pfn = nd_pfn_alloc(nd_region);
+ pfn_dev = nd_pfn_devinit(nd_pfn, ndns);
nvdimm_bus_unlock(&ndns->dev);
- if (!dev)
+ if (!pfn_dev)
return -ENOMEM;
- dev_set_drvdata(dev, drvdata);
- pfn_sb = kzalloc(sizeof(*pfn_sb), GFP_KERNEL);
- nd_pfn = to_nd_pfn(dev);
+ pfn_sb = devm_kzalloc(dev, sizeof(*pfn_sb), GFP_KERNEL);
+ nd_pfn = to_nd_pfn(pfn_dev);
nd_pfn->pfn_sb = pfn_sb;
- rc = nd_pfn_validate(nd_pfn);
- nd_pfn->pfn_sb = NULL;
- kfree(pfn_sb);
- dev_dbg(&ndns->dev, "%s: pfn: %s\n", __func__,
- rc == 0 ? dev_name(dev) : "<none>");
+ rc = nd_pfn_validate(nd_pfn, PFN_SIG);
+ dev_dbg(dev, "%s: pfn: %s\n", __func__,
+ rc == 0 ? dev_name(pfn_dev) : "<none>");
if (rc < 0) {
- __nd_detach_ndns(dev, &nd_pfn->ndns);
- put_device(dev);
+ __nd_detach_ndns(pfn_dev, &nd_pfn->ndns);
+ put_device(pfn_dev);
} else
- __nd_device_register(&nd_pfn->dev);
+ __nd_device_register(pfn_dev);
return rc;
}
EXPORT_SYMBOL(nd_pfn_probe);
+
+/*
+ * We hotplug memory at section granularity, pad the reserved area from
+ * the previous section base to the namespace base address.
+ */
+static unsigned long init_altmap_base(resource_size_t base)
+{
+ unsigned long base_pfn = PHYS_PFN(base);
+
+ return PFN_SECTION_ALIGN_DOWN(base_pfn);
+}
+
+static unsigned long init_altmap_reserve(resource_size_t base)
+{
+ unsigned long reserve = PHYS_PFN(SZ_8K);
+ unsigned long base_pfn = PHYS_PFN(base);
+
+ reserve += base_pfn - PFN_SECTION_ALIGN_DOWN(base_pfn);
+ return reserve;
+}
+
+static struct vmem_altmap *__nvdimm_setup_pfn(struct nd_pfn *nd_pfn,
+ struct resource *res, struct vmem_altmap *altmap)
+{
+ struct nd_pfn_sb *pfn_sb = nd_pfn->pfn_sb;
+ u64 offset = le64_to_cpu(pfn_sb->dataoff);
+ u32 start_pad = __le32_to_cpu(pfn_sb->start_pad);
+ u32 end_trunc = __le32_to_cpu(pfn_sb->end_trunc);
+ struct nd_namespace_common *ndns = nd_pfn->ndns;
+ struct nd_namespace_io *nsio = to_nd_namespace_io(&ndns->dev);
+ resource_size_t base = nsio->res.start + start_pad;
+ struct vmem_altmap __altmap = {
+ .base_pfn = init_altmap_base(base),
+ .reserve = init_altmap_reserve(base),
+ };
+
+ memcpy(res, &nsio->res, sizeof(*res));
+ res->start += start_pad;
+ res->end -= end_trunc;
+
+ nd_pfn->mode = le32_to_cpu(nd_pfn->pfn_sb->mode);
+ if (nd_pfn->mode == PFN_MODE_RAM) {
+ if (offset < SZ_8K)
+ return ERR_PTR(-EINVAL);
+ nd_pfn->npfns = le64_to_cpu(pfn_sb->npfns);
+ altmap = NULL;
+ } else if (nd_pfn->mode == PFN_MODE_PMEM) {
+ nd_pfn->npfns = (resource_size(res) - offset) / PAGE_SIZE;
+ if (le64_to_cpu(nd_pfn->pfn_sb->npfns) > nd_pfn->npfns)
+ dev_info(&nd_pfn->dev,
+ "number of pfns truncated from %lld to %ld\n",
+ le64_to_cpu(nd_pfn->pfn_sb->npfns),
+ nd_pfn->npfns);
+ memcpy(altmap, &__altmap, sizeof(*altmap));
+ altmap->free = PHYS_PFN(offset - SZ_8K);
+ altmap->alloc = 0;
+ } else
+ return ERR_PTR(-ENXIO);
+
+ return altmap;
+}
+
+static int nd_pfn_init(struct nd_pfn *nd_pfn)
+{
+ u32 dax_label_reserve = is_nd_dax(&nd_pfn->dev) ? SZ_128K : 0;
+ struct nd_namespace_common *ndns = nd_pfn->ndns;
+ u32 start_pad = 0, end_trunc = 0;
+ resource_size_t start, size;
+ struct nd_namespace_io *nsio;
+ struct nd_region *nd_region;
+ struct nd_pfn_sb *pfn_sb;
+ unsigned long npfns;
+ phys_addr_t offset;
+ const char *sig;
+ u64 checksum;
+ int rc;
+
+ pfn_sb = devm_kzalloc(&nd_pfn->dev, sizeof(*pfn_sb), GFP_KERNEL);
+ if (!pfn_sb)
+ return -ENOMEM;
+
+ nd_pfn->pfn_sb = pfn_sb;
+ if (is_nd_dax(&nd_pfn->dev))
+ sig = DAX_SIG;
+ else
+ sig = PFN_SIG;
+ rc = nd_pfn_validate(nd_pfn, sig);
+ if (rc != -ENODEV)
+ return rc;
+
+ /* no info block, do init */;
+ nd_region = to_nd_region(nd_pfn->dev.parent);
+ if (nd_region->ro) {
+ dev_info(&nd_pfn->dev,
+ "%s is read-only, unable to init metadata\n",
+ dev_name(&nd_region->dev));
+ return -ENXIO;
+ }
+
+ memset(pfn_sb, 0, sizeof(*pfn_sb));
+
+ /*
+ * Check if pmem collides with 'System RAM' when section aligned and
+ * trim it accordingly
+ */
+ nsio = to_nd_namespace_io(&ndns->dev);
+ start = PHYS_SECTION_ALIGN_DOWN(nsio->res.start);
+ size = resource_size(&nsio->res);
+ if (region_intersects(start, size, IORESOURCE_SYSTEM_RAM,
+ IORES_DESC_NONE) == REGION_MIXED) {
+ start = nsio->res.start;
+ start_pad = PHYS_SECTION_ALIGN_UP(start) - start;
+ }
+
+ start = nsio->res.start;
+ size = PHYS_SECTION_ALIGN_UP(start + size) - start;
+ if (region_intersects(start, size, IORESOURCE_SYSTEM_RAM,
+ IORES_DESC_NONE) == REGION_MIXED) {
+ size = resource_size(&nsio->res);
+ end_trunc = start + size - PHYS_SECTION_ALIGN_DOWN(start + size);
+ }
+
+ if (start_pad + end_trunc)
+ dev_info(&nd_pfn->dev, "%s section collision, truncate %d bytes\n",
+ dev_name(&ndns->dev), start_pad + end_trunc);
+
+ /*
+ * Note, we use 64 here for the standard size of struct page,
+ * debugging options may cause it to be larger in which case the
+ * implementation will limit the pfns advertised through
+ * ->direct_access() to those that are included in the memmap.
+ */
+ start += start_pad;
+ size = resource_size(&nsio->res);
+ npfns = (size - start_pad - end_trunc - SZ_8K) / SZ_4K;
+ if (nd_pfn->mode == PFN_MODE_PMEM) {
+ unsigned long memmap_size;
+
+ /*
+ * vmemmap_populate_hugepages() allocates the memmap array in
+ * HPAGE_SIZE chunks.
+ */
+ memmap_size = ALIGN(64 * npfns, HPAGE_SIZE);
+ offset = ALIGN(start + SZ_8K + memmap_size + dax_label_reserve,
+ nd_pfn->align) - start;
+ } else if (nd_pfn->mode == PFN_MODE_RAM)
+ offset = ALIGN(start + SZ_8K + dax_label_reserve,
+ nd_pfn->align) - start;
+ else
+ return -ENXIO;
+
+ if (offset + start_pad + end_trunc >= size) {
+ dev_err(&nd_pfn->dev, "%s unable to satisfy requested alignment\n",
+ dev_name(&ndns->dev));
+ return -ENXIO;
+ }
+
+ npfns = (size - offset - start_pad - end_trunc) / SZ_4K;
+ pfn_sb->mode = cpu_to_le32(nd_pfn->mode);
+ pfn_sb->dataoff = cpu_to_le64(offset);
+ pfn_sb->npfns = cpu_to_le64(npfns);
+ memcpy(pfn_sb->signature, sig, PFN_SIG_LEN);
+ memcpy(pfn_sb->uuid, nd_pfn->uuid, 16);
+ memcpy(pfn_sb->parent_uuid, nd_dev_to_uuid(&ndns->dev), 16);
+ pfn_sb->version_major = cpu_to_le16(1);
+ pfn_sb->version_minor = cpu_to_le16(2);
+ pfn_sb->start_pad = cpu_to_le32(start_pad);
+ pfn_sb->end_trunc = cpu_to_le32(end_trunc);
+ pfn_sb->align = cpu_to_le32(nd_pfn->align);
+ checksum = nd_sb_checksum((struct nd_gen_sb *) pfn_sb);
+ pfn_sb->checksum = cpu_to_le64(checksum);
+
+ return nvdimm_write_bytes(ndns, SZ_4K, pfn_sb, sizeof(*pfn_sb));
+}
+
+/*
+ * Determine the effective resource range and vmem_altmap from an nd_pfn
+ * instance.
+ */
+struct vmem_altmap *nvdimm_setup_pfn(struct nd_pfn *nd_pfn,
+ struct resource *res, struct vmem_altmap *altmap)
+{
+ int rc;
+
+ if (!nd_pfn->uuid || !nd_pfn->ndns)
+ return ERR_PTR(-ENODEV);
+
+ rc = nd_pfn_init(nd_pfn);
+ if (rc)
+ return ERR_PTR(rc);
+
+ /* we need a valid pfn_sb before we can init a vmem_altmap */
+ return __nvdimm_setup_pfn(nd_pfn, res, altmap);
+}
+EXPORT_SYMBOL_GPL(nvdimm_setup_pfn);
diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c
index 92f5365..042baec 100644
--- a/drivers/nvdimm/pmem.c
+++ b/drivers/nvdimm/pmem.c
@@ -33,10 +33,6 @@
#include "nd.h"
struct pmem_device {
- struct request_queue *pmem_queue;
- struct gendisk *pmem_disk;
- struct nd_namespace_common *ndns;
-
/* One contiguous memory region per device */
phys_addr_t phys_addr;
/* when non-zero this device is hosting a 'pfn' instance */
@@ -50,23 +46,10 @@
struct badblocks bb;
};
-static bool is_bad_pmem(struct badblocks *bb, sector_t sector, unsigned int len)
-{
- if (bb->count) {
- sector_t first_bad;
- int num_bad;
-
- return !!badblocks_check(bb, sector, len / 512, &first_bad,
- &num_bad);
- }
-
- return false;
-}
-
static void pmem_clear_poison(struct pmem_device *pmem, phys_addr_t offset,
unsigned int len)
{
- struct device *dev = disk_to_dev(pmem->pmem_disk);
+ struct device *dev = pmem->bb.dev;
sector_t sector;
long cleared;
@@ -136,8 +119,7 @@
unsigned long start;
struct bio_vec bvec;
struct bvec_iter iter;
- struct block_device *bdev = bio->bi_bdev;
- struct pmem_device *pmem = bdev->bd_disk->private_data;
+ struct pmem_device *pmem = q->queuedata;
do_acct = nd_iostat_start(bio, &start);
bio_for_each_segment(bvec, bio, iter) {
@@ -162,7 +144,7 @@
static int pmem_rw_page(struct block_device *bdev, sector_t sector,
struct page *page, int rw)
{
- struct pmem_device *pmem = bdev->bd_disk->private_data;
+ struct pmem_device *pmem = bdev->bd_queue->queuedata;
int rc;
rc = pmem_do_bvec(pmem, page, PAGE_SIZE, 0, rw, sector);
@@ -184,7 +166,7 @@
static long pmem_direct_access(struct block_device *bdev, sector_t sector,
void __pmem **kaddr, pfn_t *pfn)
{
- struct pmem_device *pmem = bdev->bd_disk->private_data;
+ struct pmem_device *pmem = bdev->bd_queue->queuedata;
resource_size_t offset = sector * 512 + pmem->data_offset;
*kaddr = pmem->virt_addr + offset;
@@ -200,104 +182,119 @@
.revalidate_disk = nvdimm_revalidate_disk,
};
-static struct pmem_device *pmem_alloc(struct device *dev,
- struct resource *res, int id)
+static void pmem_release_queue(void *q)
{
+ blk_cleanup_queue(q);
+}
+
+void pmem_release_disk(void *disk)
+{
+ del_gendisk(disk);
+ put_disk(disk);
+}
+
+static int pmem_attach_disk(struct device *dev,
+ struct nd_namespace_common *ndns)
+{
+ struct nd_namespace_io *nsio = to_nd_namespace_io(&ndns->dev);
+ struct vmem_altmap __altmap, *altmap = NULL;
+ struct resource *res = &nsio->res;
+ struct nd_pfn *nd_pfn = NULL;
+ int nid = dev_to_node(dev);
+ struct nd_pfn_sb *pfn_sb;
struct pmem_device *pmem;
+ struct resource pfn_res;
struct request_queue *q;
+ struct gendisk *disk;
+ void *addr;
+
+ /* while nsio_rw_bytes is active, parse a pfn info block if present */
+ if (is_nd_pfn(dev)) {
+ nd_pfn = to_nd_pfn(dev);
+ altmap = nvdimm_setup_pfn(nd_pfn, &pfn_res, &__altmap);
+ if (IS_ERR(altmap))
+ return PTR_ERR(altmap);
+ }
+
+ /* we're attaching a block device, disable raw namespace access */
+ devm_nsio_disable(dev, nsio);
pmem = devm_kzalloc(dev, sizeof(*pmem), GFP_KERNEL);
if (!pmem)
- return ERR_PTR(-ENOMEM);
+ return -ENOMEM;
+ dev_set_drvdata(dev, pmem);
pmem->phys_addr = res->start;
pmem->size = resource_size(res);
if (!arch_has_wmb_pmem())
dev_warn(dev, "unable to guarantee persistence of writes\n");
- if (!devm_request_mem_region(dev, pmem->phys_addr, pmem->size,
- dev_name(dev))) {
- dev_warn(dev, "could not reserve region [0x%pa:0x%zx]\n",
- &pmem->phys_addr, pmem->size);
- return ERR_PTR(-EBUSY);
+ if (!devm_request_mem_region(dev, res->start, resource_size(res),
+ dev_name(dev))) {
+ dev_warn(dev, "could not reserve region %pR\n", res);
+ return -EBUSY;
}
q = blk_alloc_queue_node(GFP_KERNEL, dev_to_node(dev));
if (!q)
- return ERR_PTR(-ENOMEM);
+ return -ENOMEM;
pmem->pfn_flags = PFN_DEV;
- if (pmem_should_map_pages(dev)) {
- pmem->virt_addr = (void __pmem *) devm_memremap_pages(dev, res,
+ if (is_nd_pfn(dev)) {
+ addr = devm_memremap_pages(dev, &pfn_res, &q->q_usage_counter,
+ altmap);
+ pfn_sb = nd_pfn->pfn_sb;
+ pmem->data_offset = le64_to_cpu(pfn_sb->dataoff);
+ pmem->pfn_pad = resource_size(res) - resource_size(&pfn_res);
+ pmem->pfn_flags |= PFN_MAP;
+ res = &pfn_res; /* for badblocks populate */
+ res->start += pmem->data_offset;
+ } else if (pmem_should_map_pages(dev)) {
+ addr = devm_memremap_pages(dev, &nsio->res,
&q->q_usage_counter, NULL);
pmem->pfn_flags |= PFN_MAP;
} else
- pmem->virt_addr = (void __pmem *) devm_memremap(dev,
- pmem->phys_addr, pmem->size,
- ARCH_MEMREMAP_PMEM);
+ addr = devm_memremap(dev, pmem->phys_addr,
+ pmem->size, ARCH_MEMREMAP_PMEM);
- if (IS_ERR(pmem->virt_addr)) {
+ /*
+ * At release time the queue must be dead before
+ * devm_memremap_pages is unwound
+ */
+ if (devm_add_action(dev, pmem_release_queue, q)) {
blk_cleanup_queue(q);
- return (void __force *) pmem->virt_addr;
+ return -ENOMEM;
}
- pmem->pmem_queue = q;
- return pmem;
-}
+ if (IS_ERR(addr))
+ return PTR_ERR(addr);
+ pmem->virt_addr = (void __pmem *) addr;
-static void pmem_detach_disk(struct pmem_device *pmem)
-{
- if (!pmem->pmem_disk)
- return;
-
- del_gendisk(pmem->pmem_disk);
- put_disk(pmem->pmem_disk);
- blk_cleanup_queue(pmem->pmem_queue);
-}
-
-static int pmem_attach_disk(struct device *dev,
- struct nd_namespace_common *ndns, struct pmem_device *pmem)
-{
- struct nd_namespace_io *nsio = to_nd_namespace_io(&ndns->dev);
- int nid = dev_to_node(dev);
- struct resource bb_res;
- struct gendisk *disk;
-
- blk_queue_make_request(pmem->pmem_queue, pmem_make_request);
- blk_queue_physical_block_size(pmem->pmem_queue, PAGE_SIZE);
- blk_queue_max_hw_sectors(pmem->pmem_queue, UINT_MAX);
- blk_queue_bounce_limit(pmem->pmem_queue, BLK_BOUNCE_ANY);
- queue_flag_set_unlocked(QUEUE_FLAG_NONROT, pmem->pmem_queue);
+ blk_queue_make_request(q, pmem_make_request);
+ blk_queue_physical_block_size(q, PAGE_SIZE);
+ blk_queue_max_hw_sectors(q, UINT_MAX);
+ blk_queue_bounce_limit(q, BLK_BOUNCE_ANY);
+ queue_flag_set_unlocked(QUEUE_FLAG_NONROT, q);
+ q->queuedata = pmem;
disk = alloc_disk_node(0, nid);
- if (!disk) {
- blk_cleanup_queue(pmem->pmem_queue);
+ if (!disk)
+ return -ENOMEM;
+ if (devm_add_action(dev, pmem_release_disk, disk)) {
+ put_disk(disk);
return -ENOMEM;
}
disk->fops = &pmem_fops;
- disk->private_data = pmem;
- disk->queue = pmem->pmem_queue;
+ disk->queue = q;
disk->flags = GENHD_FL_EXT_DEVT;
nvdimm_namespace_disk_name(ndns, disk->disk_name);
disk->driverfs_dev = dev;
set_capacity(disk, (pmem->size - pmem->pfn_pad - pmem->data_offset)
/ 512);
- pmem->pmem_disk = disk;
- devm_exit_badblocks(dev, &pmem->bb);
if (devm_init_badblocks(dev, &pmem->bb))
return -ENOMEM;
- bb_res.start = nsio->res.start + pmem->data_offset;
- bb_res.end = nsio->res.end;
- if (is_nd_pfn(dev)) {
- struct nd_pfn *nd_pfn = to_nd_pfn(dev);
- struct nd_pfn_sb *pfn_sb = nd_pfn->pfn_sb;
-
- bb_res.start += __le32_to_cpu(pfn_sb->start_pad);
- bb_res.end -= __le32_to_cpu(pfn_sb->end_trunc);
- }
- nvdimm_badblocks_populate(to_nd_region(dev->parent), &pmem->bb,
- &bb_res);
+ nvdimm_badblocks_populate(to_nd_region(dev->parent), &pmem->bb, res);
disk->bb = &pmem->bb;
add_disk(disk);
revalidate_disk(disk);
@@ -305,346 +302,68 @@
return 0;
}
-static int pmem_rw_bytes(struct nd_namespace_common *ndns,
- resource_size_t offset, void *buf, size_t size, int rw)
-{
- struct pmem_device *pmem = dev_get_drvdata(ndns->claim);
-
- if (unlikely(offset + size > pmem->size)) {
- dev_WARN_ONCE(&ndns->dev, 1, "request out of range\n");
- return -EFAULT;
- }
-
- if (rw == READ) {
- unsigned int sz_align = ALIGN(size + (offset & (512 - 1)), 512);
-
- if (unlikely(is_bad_pmem(&pmem->bb, offset / 512, sz_align)))
- return -EIO;
- return memcpy_from_pmem(buf, pmem->virt_addr + offset, size);
- } else {
- memcpy_to_pmem(pmem->virt_addr + offset, buf, size);
- wmb_pmem();
- }
-
- return 0;
-}
-
-static int nd_pfn_init(struct nd_pfn *nd_pfn)
-{
- struct nd_pfn_sb *pfn_sb = kzalloc(sizeof(*pfn_sb), GFP_KERNEL);
- struct pmem_device *pmem = dev_get_drvdata(&nd_pfn->dev);
- struct nd_namespace_common *ndns = nd_pfn->ndns;
- u32 start_pad = 0, end_trunc = 0;
- resource_size_t start, size;
- struct nd_namespace_io *nsio;
- struct nd_region *nd_region;
- unsigned long npfns;
- phys_addr_t offset;
- u64 checksum;
- int rc;
-
- if (!pfn_sb)
- return -ENOMEM;
-
- nd_pfn->pfn_sb = pfn_sb;
- rc = nd_pfn_validate(nd_pfn);
- if (rc == -ENODEV)
- /* no info block, do init */;
- else
- return rc;
-
- nd_region = to_nd_region(nd_pfn->dev.parent);
- if (nd_region->ro) {
- dev_info(&nd_pfn->dev,
- "%s is read-only, unable to init metadata\n",
- dev_name(&nd_region->dev));
- goto err;
- }
-
- memset(pfn_sb, 0, sizeof(*pfn_sb));
-
- /*
- * Check if pmem collides with 'System RAM' when section aligned and
- * trim it accordingly
- */
- nsio = to_nd_namespace_io(&ndns->dev);
- start = PHYS_SECTION_ALIGN_DOWN(nsio->res.start);
- size = resource_size(&nsio->res);
- if (region_intersects(start, size, IORESOURCE_SYSTEM_RAM,
- IORES_DESC_NONE) == REGION_MIXED) {
-
- start = nsio->res.start;
- start_pad = PHYS_SECTION_ALIGN_UP(start) - start;
- }
-
- start = nsio->res.start;
- size = PHYS_SECTION_ALIGN_UP(start + size) - start;
- if (region_intersects(start, size, IORESOURCE_SYSTEM_RAM,
- IORES_DESC_NONE) == REGION_MIXED) {
- size = resource_size(&nsio->res);
- end_trunc = start + size - PHYS_SECTION_ALIGN_DOWN(start + size);
- }
-
- if (start_pad + end_trunc)
- dev_info(&nd_pfn->dev, "%s section collision, truncate %d bytes\n",
- dev_name(&ndns->dev), start_pad + end_trunc);
-
- /*
- * Note, we use 64 here for the standard size of struct page,
- * debugging options may cause it to be larger in which case the
- * implementation will limit the pfns advertised through
- * ->direct_access() to those that are included in the memmap.
- */
- start += start_pad;
- npfns = (pmem->size - start_pad - end_trunc - SZ_8K) / SZ_4K;
- if (nd_pfn->mode == PFN_MODE_PMEM) {
- unsigned long memmap_size;
-
- /*
- * vmemmap_populate_hugepages() allocates the memmap array in
- * PMD_SIZE chunks.
- */
- memmap_size = ALIGN(64 * npfns, PMD_SIZE);
- offset = ALIGN(start + SZ_8K + memmap_size, nd_pfn->align)
- - start;
- } else if (nd_pfn->mode == PFN_MODE_RAM)
- offset = ALIGN(start + SZ_8K, nd_pfn->align) - start;
- else
- goto err;
-
- if (offset + start_pad + end_trunc >= pmem->size) {
- dev_err(&nd_pfn->dev, "%s unable to satisfy requested alignment\n",
- dev_name(&ndns->dev));
- goto err;
- }
-
- npfns = (pmem->size - offset - start_pad - end_trunc) / SZ_4K;
- pfn_sb->mode = cpu_to_le32(nd_pfn->mode);
- pfn_sb->dataoff = cpu_to_le64(offset);
- pfn_sb->npfns = cpu_to_le64(npfns);
- memcpy(pfn_sb->signature, PFN_SIG, PFN_SIG_LEN);
- memcpy(pfn_sb->uuid, nd_pfn->uuid, 16);
- memcpy(pfn_sb->parent_uuid, nd_dev_to_uuid(&ndns->dev), 16);
- pfn_sb->version_major = cpu_to_le16(1);
- pfn_sb->version_minor = cpu_to_le16(1);
- pfn_sb->start_pad = cpu_to_le32(start_pad);
- pfn_sb->end_trunc = cpu_to_le32(end_trunc);
- checksum = nd_sb_checksum((struct nd_gen_sb *) pfn_sb);
- pfn_sb->checksum = cpu_to_le64(checksum);
-
- rc = nvdimm_write_bytes(ndns, SZ_4K, pfn_sb, sizeof(*pfn_sb));
- if (rc)
- goto err;
-
- return 0;
- err:
- nd_pfn->pfn_sb = NULL;
- kfree(pfn_sb);
- return -ENXIO;
-}
-
-static int nvdimm_namespace_detach_pfn(struct nd_namespace_common *ndns)
-{
- struct nd_pfn *nd_pfn = to_nd_pfn(ndns->claim);
- struct pmem_device *pmem;
-
- /* free pmem disk */
- pmem = dev_get_drvdata(&nd_pfn->dev);
- pmem_detach_disk(pmem);
-
- /* release nd_pfn resources */
- kfree(nd_pfn->pfn_sb);
- nd_pfn->pfn_sb = NULL;
-
- return 0;
-}
-
-/*
- * We hotplug memory at section granularity, pad the reserved area from
- * the previous section base to the namespace base address.
- */
-static unsigned long init_altmap_base(resource_size_t base)
-{
- unsigned long base_pfn = PHYS_PFN(base);
-
- return PFN_SECTION_ALIGN_DOWN(base_pfn);
-}
-
-static unsigned long init_altmap_reserve(resource_size_t base)
-{
- unsigned long reserve = PHYS_PFN(SZ_8K);
- unsigned long base_pfn = PHYS_PFN(base);
-
- reserve += base_pfn - PFN_SECTION_ALIGN_DOWN(base_pfn);
- return reserve;
-}
-
-static int __nvdimm_namespace_attach_pfn(struct nd_pfn *nd_pfn)
-{
- int rc;
- struct resource res;
- struct request_queue *q;
- struct pmem_device *pmem;
- struct vmem_altmap *altmap;
- struct device *dev = &nd_pfn->dev;
- struct nd_pfn_sb *pfn_sb = nd_pfn->pfn_sb;
- struct nd_namespace_common *ndns = nd_pfn->ndns;
- u32 start_pad = __le32_to_cpu(pfn_sb->start_pad);
- u32 end_trunc = __le32_to_cpu(pfn_sb->end_trunc);
- struct nd_namespace_io *nsio = to_nd_namespace_io(&ndns->dev);
- resource_size_t base = nsio->res.start + start_pad;
- struct vmem_altmap __altmap = {
- .base_pfn = init_altmap_base(base),
- .reserve = init_altmap_reserve(base),
- };
-
- pmem = dev_get_drvdata(dev);
- pmem->data_offset = le64_to_cpu(pfn_sb->dataoff);
- pmem->pfn_pad = start_pad + end_trunc;
- nd_pfn->mode = le32_to_cpu(nd_pfn->pfn_sb->mode);
- if (nd_pfn->mode == PFN_MODE_RAM) {
- if (pmem->data_offset < SZ_8K)
- return -EINVAL;
- nd_pfn->npfns = le64_to_cpu(pfn_sb->npfns);
- altmap = NULL;
- } else if (nd_pfn->mode == PFN_MODE_PMEM) {
- nd_pfn->npfns = (pmem->size - pmem->pfn_pad - pmem->data_offset)
- / PAGE_SIZE;
- if (le64_to_cpu(nd_pfn->pfn_sb->npfns) > nd_pfn->npfns)
- dev_info(&nd_pfn->dev,
- "number of pfns truncated from %lld to %ld\n",
- le64_to_cpu(nd_pfn->pfn_sb->npfns),
- nd_pfn->npfns);
- altmap = & __altmap;
- altmap->free = PHYS_PFN(pmem->data_offset - SZ_8K);
- altmap->alloc = 0;
- } else {
- rc = -ENXIO;
- goto err;
- }
-
- /* establish pfn range for lookup, and switch to direct map */
- q = pmem->pmem_queue;
- memcpy(&res, &nsio->res, sizeof(res));
- res.start += start_pad;
- res.end -= end_trunc;
- devm_memunmap(dev, (void __force *) pmem->virt_addr);
- pmem->virt_addr = (void __pmem *) devm_memremap_pages(dev, &res,
- &q->q_usage_counter, altmap);
- pmem->pfn_flags |= PFN_MAP;
- if (IS_ERR(pmem->virt_addr)) {
- rc = PTR_ERR(pmem->virt_addr);
- goto err;
- }
-
- /* attach pmem disk in "pfn-mode" */
- rc = pmem_attach_disk(dev, ndns, pmem);
- if (rc)
- goto err;
-
- return rc;
- err:
- nvdimm_namespace_detach_pfn(ndns);
- return rc;
-
-}
-
-static int nvdimm_namespace_attach_pfn(struct nd_namespace_common *ndns)
-{
- struct nd_pfn *nd_pfn = to_nd_pfn(ndns->claim);
- int rc;
-
- if (!nd_pfn->uuid || !nd_pfn->ndns)
- return -ENODEV;
-
- rc = nd_pfn_init(nd_pfn);
- if (rc)
- return rc;
- /* we need a valid pfn_sb before we can init a vmem_altmap */
- return __nvdimm_namespace_attach_pfn(nd_pfn);
-}
-
static int nd_pmem_probe(struct device *dev)
{
- struct nd_region *nd_region = to_nd_region(dev->parent);
struct nd_namespace_common *ndns;
- struct nd_namespace_io *nsio;
- struct pmem_device *pmem;
ndns = nvdimm_namespace_common_probe(dev);
if (IS_ERR(ndns))
return PTR_ERR(ndns);
- nsio = to_nd_namespace_io(&ndns->dev);
- pmem = pmem_alloc(dev, &nsio->res, nd_region->id);
- if (IS_ERR(pmem))
- return PTR_ERR(pmem);
+ if (devm_nsio_enable(dev, to_nd_namespace_io(&ndns->dev)))
+ return -ENXIO;
- pmem->ndns = ndns;
- dev_set_drvdata(dev, pmem);
- ndns->rw_bytes = pmem_rw_bytes;
- if (devm_init_badblocks(dev, &pmem->bb))
- return -ENOMEM;
- nvdimm_badblocks_populate(nd_region, &pmem->bb, &nsio->res);
-
- if (is_nd_btt(dev)) {
- /* btt allocates its own request_queue */
- blk_cleanup_queue(pmem->pmem_queue);
- pmem->pmem_queue = NULL;
+ if (is_nd_btt(dev))
return nvdimm_namespace_attach_btt(ndns);
- }
if (is_nd_pfn(dev))
- return nvdimm_namespace_attach_pfn(ndns);
+ return pmem_attach_disk(dev, ndns);
- if (nd_btt_probe(ndns, pmem) == 0 || nd_pfn_probe(ndns, pmem) == 0) {
- /*
- * We'll come back as either btt-pmem, or pfn-pmem, so
- * drop the queue allocation for now.
- */
- blk_cleanup_queue(pmem->pmem_queue);
+ /* if we find a valid info-block we'll come back as that personality */
+ if (nd_btt_probe(dev, ndns) == 0 || nd_pfn_probe(dev, ndns) == 0
+ || nd_dax_probe(dev, ndns) == 0)
return -ENXIO;
- }
- return pmem_attach_disk(dev, ndns, pmem);
+ /* ...otherwise we're just a raw pmem device */
+ return pmem_attach_disk(dev, ndns);
}
static int nd_pmem_remove(struct device *dev)
{
- struct pmem_device *pmem = dev_get_drvdata(dev);
-
if (is_nd_btt(dev))
- nvdimm_namespace_detach_btt(pmem->ndns);
- else if (is_nd_pfn(dev))
- nvdimm_namespace_detach_pfn(pmem->ndns);
- else
- pmem_detach_disk(pmem);
-
+ nvdimm_namespace_detach_btt(to_nd_btt(dev));
return 0;
}
static void nd_pmem_notify(struct device *dev, enum nvdimm_event event)
{
- struct pmem_device *pmem = dev_get_drvdata(dev);
- struct nd_namespace_common *ndns = pmem->ndns;
struct nd_region *nd_region = to_nd_region(dev->parent);
- struct nd_namespace_io *nsio = to_nd_namespace_io(&ndns->dev);
- struct resource res = {
- .start = nsio->res.start + pmem->data_offset,
- .end = nsio->res.end,
- };
+ struct pmem_device *pmem = dev_get_drvdata(dev);
+ resource_size_t offset = 0, end_trunc = 0;
+ struct nd_namespace_common *ndns;
+ struct nd_namespace_io *nsio;
+ struct resource res;
if (event != NVDIMM_REVALIDATE_POISON)
return;
- if (is_nd_pfn(dev)) {
+ if (is_nd_btt(dev)) {
+ struct nd_btt *nd_btt = to_nd_btt(dev);
+
+ ndns = nd_btt->ndns;
+ } else if (is_nd_pfn(dev)) {
struct nd_pfn *nd_pfn = to_nd_pfn(dev);
struct nd_pfn_sb *pfn_sb = nd_pfn->pfn_sb;
- res.start += __le32_to_cpu(pfn_sb->start_pad);
- res.end -= __le32_to_cpu(pfn_sb->end_trunc);
- }
+ ndns = nd_pfn->ndns;
+ offset = pmem->data_offset + __le32_to_cpu(pfn_sb->start_pad);
+ end_trunc = __le32_to_cpu(pfn_sb->end_trunc);
+ } else
+ ndns = to_ndns(dev);
+ nsio = to_nd_namespace_io(&ndns->dev);
+ res.start = nsio->res.start + offset;
+ res.end = nsio->res.end - end_trunc;
nvdimm_badblocks_populate(nd_region, &pmem->bb, &res);
}
diff --git a/drivers/nvdimm/region.c b/drivers/nvdimm/region.c
index 4b7715e..05a9123 100644
--- a/drivers/nvdimm/region.c
+++ b/drivers/nvdimm/region.c
@@ -54,6 +54,7 @@
nd_region->btt_seed = nd_btt_create(nd_region);
nd_region->pfn_seed = nd_pfn_create(nd_region);
+ nd_region->dax_seed = nd_dax_create(nd_region);
if (err == 0)
return 0;
@@ -86,6 +87,7 @@
nd_region->ns_seed = NULL;
nd_region->btt_seed = NULL;
nd_region->pfn_seed = NULL;
+ nd_region->dax_seed = NULL;
dev_set_drvdata(dev, NULL);
nvdimm_bus_unlock(dev);
diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c
index 139bf71..40fcfea 100644
--- a/drivers/nvdimm/region_devs.c
+++ b/drivers/nvdimm/region_devs.c
@@ -306,6 +306,23 @@
}
static DEVICE_ATTR_RO(pfn_seed);
+static ssize_t dax_seed_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct nd_region *nd_region = to_nd_region(dev);
+ ssize_t rc;
+
+ nvdimm_bus_lock(dev);
+ if (nd_region->dax_seed)
+ rc = sprintf(buf, "%s\n", dev_name(nd_region->dax_seed));
+ else
+ rc = sprintf(buf, "\n");
+ nvdimm_bus_unlock(dev);
+
+ return rc;
+}
+static DEVICE_ATTR_RO(dax_seed);
+
static ssize_t read_only_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
@@ -335,6 +352,7 @@
&dev_attr_mappings.attr,
&dev_attr_btt_seed.attr,
&dev_attr_pfn_seed.attr,
+ &dev_attr_dax_seed.attr,
&dev_attr_read_only.attr,
&dev_attr_set_cookie.attr,
&dev_attr_available_size.attr,
@@ -353,6 +371,9 @@
if (!is_nd_pmem(dev) && a == &dev_attr_pfn_seed.attr)
return 0;
+ if (!is_nd_pmem(dev) && a == &dev_attr_dax_seed.attr)
+ return 0;
+
if (a != &dev_attr_set_cookie.attr
&& a != &dev_attr_available_size.attr)
return a->mode;
@@ -441,6 +462,13 @@
nd_region_create_pfn_seed(nd_region);
nvdimm_bus_unlock(dev);
}
+ if (is_nd_dax(dev) && probe) {
+ nd_region = to_nd_region(dev->parent);
+ nvdimm_bus_lock(dev);
+ if (nd_region->dax_seed == dev)
+ nd_region_create_dax_seed(nd_region);
+ nvdimm_bus_unlock(dev);
+ }
}
void nd_region_probe_success(struct nvdimm_bus *nvdimm_bus, struct device *dev)
@@ -718,6 +746,7 @@
ida_init(&nd_region->ns_ida);
ida_init(&nd_region->btt_ida);
ida_init(&nd_region->pfn_ida);
+ ida_init(&nd_region->dax_ida);
dev = &nd_region->dev;
dev_set_name(dev, "region%d", nd_region->id);
dev->parent = &nvdimm_bus->dev;
@@ -764,3 +793,8 @@
__func__);
}
EXPORT_SYMBOL_GPL(nvdimm_volatile_region_create);
+
+void __exit nd_region_devs_exit(void)
+{
+ ida_destroy(®ion_ida);
+}
diff --git a/drivers/nvmem/Kconfig b/drivers/nvmem/Kconfig
index ca52952..3041d48 100644
--- a/drivers/nvmem/Kconfig
+++ b/drivers/nvmem/Kconfig
@@ -1,6 +1,5 @@
menuconfig NVMEM
tristate "NVMEM Support"
- select REGMAP
help
Support for NVMEM(Non Volatile Memory) devices like EEPROM, EFUSES...
@@ -28,6 +27,7 @@
config NVMEM_LPC18XX_EEPROM
tristate "NXP LPC18XX EEPROM Memory Support"
depends on ARCH_LPC18XX || COMPILE_TEST
+ depends on HAS_IOMEM
help
Say Y here to include support for NXP LPC18xx EEPROM memory found in
NXP LPC185x/3x and LPC435x/3x/2x/1x devices.
@@ -49,6 +49,7 @@
config MTK_EFUSE
tristate "Mediatek SoCs EFUSE support"
depends on ARCH_MEDIATEK || COMPILE_TEST
+ depends on HAS_IOMEM
select REGMAP_MMIO
help
This is a driver to access hardware related data like sensor
@@ -61,7 +62,6 @@
tristate "QCOM QFPROM Support"
depends on ARCH_QCOM || COMPILE_TEST
depends on HAS_IOMEM
- select REGMAP_MMIO
help
Say y here to enable QFPROM support. The QFPROM provides access
functions for QFPROM data to rest of the drivers via nvmem interface.
@@ -83,7 +83,6 @@
config NVMEM_SUNXI_SID
tristate "Allwinner SoCs SID support"
depends on ARCH_SUNXI
- select REGMAP_MMIO
help
This is a driver for the 'security ID' available on various Allwinner
devices.
diff --git a/drivers/nvmem/core.c b/drivers/nvmem/core.c
index 0de3d87..bb4ea12 100644
--- a/drivers/nvmem/core.c
+++ b/drivers/nvmem/core.c
@@ -23,12 +23,10 @@
#include <linux/nvmem-consumer.h>
#include <linux/nvmem-provider.h>
#include <linux/of.h>
-#include <linux/regmap.h>
#include <linux/slab.h>
struct nvmem_device {
const char *name;
- struct regmap *regmap;
struct module *owner;
struct device dev;
int stride;
@@ -41,6 +39,9 @@
int flags;
struct bin_attribute eeprom;
struct device *base_dev;
+ nvmem_reg_read_t reg_read;
+ nvmem_reg_write_t reg_write;
+ void *priv;
};
#define FLAG_COMPAT BIT(0)
@@ -66,6 +67,23 @@
#endif
#define to_nvmem_device(d) container_of(d, struct nvmem_device, dev)
+static int nvmem_reg_read(struct nvmem_device *nvmem, unsigned int offset,
+ void *val, size_t bytes)
+{
+ if (nvmem->reg_read)
+ return nvmem->reg_read(nvmem->priv, offset, val, bytes);
+
+ return -EINVAL;
+}
+
+static int nvmem_reg_write(struct nvmem_device *nvmem, unsigned int offset,
+ void *val, size_t bytes)
+{
+ if (nvmem->reg_write)
+ return nvmem->reg_write(nvmem->priv, offset, val, bytes);
+
+ return -EINVAL;
+}
static ssize_t bin_attr_nvmem_read(struct file *filp, struct kobject *kobj,
struct bin_attribute *attr,
@@ -93,7 +111,7 @@
count = round_down(count, nvmem->word_size);
- rc = regmap_raw_read(nvmem->regmap, pos, buf, count);
+ rc = nvmem_reg_read(nvmem, pos, buf, count);
if (IS_ERR_VALUE(rc))
return rc;
@@ -127,7 +145,7 @@
count = round_down(count, nvmem->word_size);
- rc = regmap_raw_write(nvmem->regmap, pos, buf, count);
+ rc = nvmem_reg_write(nvmem, pos, buf, count);
if (IS_ERR_VALUE(rc))
return rc;
@@ -421,18 +439,11 @@
{
struct nvmem_device *nvmem;
struct device_node *np;
- struct regmap *rm;
int rval;
if (!config->dev)
return ERR_PTR(-EINVAL);
- rm = dev_get_regmap(config->dev, NULL);
- if (!rm) {
- dev_err(config->dev, "Regmap not found\n");
- return ERR_PTR(-EINVAL);
- }
-
nvmem = kzalloc(sizeof(*nvmem), GFP_KERNEL);
if (!nvmem)
return ERR_PTR(-ENOMEM);
@@ -444,14 +455,16 @@
}
nvmem->id = rval;
- nvmem->regmap = rm;
nvmem->owner = config->owner;
- nvmem->stride = regmap_get_reg_stride(rm);
- nvmem->word_size = regmap_get_val_bytes(rm);
- nvmem->size = regmap_get_max_register(rm) + nvmem->stride;
+ nvmem->stride = config->stride;
+ nvmem->word_size = config->word_size;
+ nvmem->size = config->size;
nvmem->dev.type = &nvmem_provider_type;
nvmem->dev.bus = &nvmem_bus_type;
nvmem->dev.parent = config->dev;
+ nvmem->priv = config->priv;
+ nvmem->reg_read = config->reg_read;
+ nvmem->reg_write = config->reg_write;
np = config->dev->of_node;
nvmem->dev.of_node = np;
dev_set_name(&nvmem->dev, "%s%d",
@@ -948,7 +961,7 @@
{
int rc;
- rc = regmap_raw_read(nvmem->regmap, cell->offset, buf, cell->bytes);
+ rc = nvmem_reg_read(nvmem, cell->offset, buf, cell->bytes);
if (IS_ERR_VALUE(rc))
return rc;
@@ -977,7 +990,7 @@
u8 *buf;
int rc;
- if (!nvmem || !nvmem->regmap)
+ if (!nvmem)
return ERR_PTR(-EINVAL);
buf = kzalloc(cell->bytes, GFP_KERNEL);
@@ -1014,7 +1027,7 @@
*b <<= bit_offset;
/* setup the first byte with lsb bits from nvmem */
- rc = regmap_raw_read(nvmem->regmap, cell->offset, &v, 1);
+ rc = nvmem_reg_read(nvmem, cell->offset, &v, 1);
*b++ |= GENMASK(bit_offset - 1, 0) & v;
/* setup rest of the byte if any */
@@ -1031,7 +1044,7 @@
/* if it's not end on byte boundary */
if ((nbits + bit_offset) % BITS_PER_BYTE) {
/* setup the last byte with msb bits from nvmem */
- rc = regmap_raw_read(nvmem->regmap,
+ rc = nvmem_reg_read(nvmem,
cell->offset + cell->bytes - 1, &v, 1);
*p |= GENMASK(7, (nbits + bit_offset) % BITS_PER_BYTE) & v;
@@ -1054,7 +1067,7 @@
struct nvmem_device *nvmem = cell->nvmem;
int rc;
- if (!nvmem || !nvmem->regmap || nvmem->read_only ||
+ if (!nvmem || nvmem->read_only ||
(cell->bit_offset == 0 && len != cell->bytes))
return -EINVAL;
@@ -1064,7 +1077,7 @@
return PTR_ERR(buf);
}
- rc = regmap_raw_write(nvmem->regmap, cell->offset, buf, cell->bytes);
+ rc = nvmem_reg_write(nvmem, cell->offset, buf, cell->bytes);
/* free the tmp buffer */
if (cell->bit_offset || cell->nbits)
@@ -1094,7 +1107,7 @@
int rc;
ssize_t len;
- if (!nvmem || !nvmem->regmap)
+ if (!nvmem)
return -EINVAL;
rc = nvmem_cell_info_to_nvmem_cell(nvmem, info, &cell);
@@ -1124,7 +1137,7 @@
struct nvmem_cell cell;
int rc;
- if (!nvmem || !nvmem->regmap)
+ if (!nvmem)
return -EINVAL;
rc = nvmem_cell_info_to_nvmem_cell(nvmem, info, &cell);
@@ -1152,10 +1165,10 @@
{
int rc;
- if (!nvmem || !nvmem->regmap)
+ if (!nvmem)
return -EINVAL;
- rc = regmap_raw_read(nvmem->regmap, offset, buf, bytes);
+ rc = nvmem_reg_read(nvmem, offset, buf, bytes);
if (IS_ERR_VALUE(rc))
return rc;
@@ -1180,10 +1193,10 @@
{
int rc;
- if (!nvmem || !nvmem->regmap)
+ if (!nvmem)
return -EINVAL;
- rc = regmap_raw_write(nvmem->regmap, offset, buf, bytes);
+ rc = nvmem_reg_write(nvmem, offset, buf, bytes);
if (IS_ERR_VALUE(rc))
return rc;
diff --git a/drivers/nvmem/imx-ocotp.c b/drivers/nvmem/imx-ocotp.c
index d7796eb..75e66ef 100644
--- a/drivers/nvmem/imx-ocotp.c
+++ b/drivers/nvmem/imx-ocotp.c
@@ -22,7 +22,6 @@
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/platform_device.h>
-#include <linux/regmap.h>
#include <linux/slab.h>
struct ocotp_priv {
@@ -31,59 +30,34 @@
unsigned int nregs;
};
-static int imx_ocotp_read(void *context, const void *reg, size_t reg_size,
- void *val, size_t val_size)
+static int imx_ocotp_read(void *context, unsigned int offset,
+ void *val, size_t bytes)
{
struct ocotp_priv *priv = context;
- unsigned int offset = *(u32 *)reg;
unsigned int count;
+ u32 *buf = val;
int i;
u32 index;
index = offset >> 2;
- count = val_size >> 2;
+ count = bytes >> 2;
if (count > (priv->nregs - index))
count = priv->nregs - index;
- for (i = index; i < (index + count); i++) {
- *(u32 *)val = readl(priv->base + 0x400 + i * 0x10);
- val += 4;
- }
+ for (i = index; i < (index + count); i++)
+ *buf++ = readl(priv->base + 0x400 + i * 0x10);
return 0;
}
-static int imx_ocotp_write(void *context, const void *data, size_t count)
-{
- /* Not implemented */
- return 0;
-}
-
-static struct regmap_bus imx_ocotp_bus = {
- .read = imx_ocotp_read,
- .write = imx_ocotp_write,
- .reg_format_endian_default = REGMAP_ENDIAN_NATIVE,
- .val_format_endian_default = REGMAP_ENDIAN_NATIVE,
-};
-
-static bool imx_ocotp_writeable_reg(struct device *dev, unsigned int reg)
-{
- return false;
-}
-
-static struct regmap_config imx_ocotp_regmap_config = {
- .reg_bits = 32,
- .val_bits = 32,
- .reg_stride = 4,
- .writeable_reg = imx_ocotp_writeable_reg,
- .name = "imx-ocotp",
-};
-
static struct nvmem_config imx_ocotp_nvmem_config = {
.name = "imx-ocotp",
.read_only = true,
+ .word_size = 4,
+ .stride = 4,
.owner = THIS_MODULE,
+ .reg_read = imx_ocotp_read,
};
static const struct of_device_id imx_ocotp_dt_ids[] = {
@@ -99,7 +73,6 @@
const struct of_device_id *of_id;
struct device *dev = &pdev->dev;
struct resource *res;
- struct regmap *regmap;
struct ocotp_priv *priv;
struct nvmem_device *nvmem;
@@ -114,15 +87,9 @@
of_id = of_match_device(imx_ocotp_dt_ids, dev);
priv->nregs = (unsigned int)of_id->data;
- imx_ocotp_regmap_config.max_register = 4 * priv->nregs - 4;
-
- regmap = devm_regmap_init(dev, &imx_ocotp_bus, priv,
- &imx_ocotp_regmap_config);
- if (IS_ERR(regmap)) {
- dev_err(dev, "regmap init failed\n");
- return PTR_ERR(regmap);
- }
+ imx_ocotp_nvmem_config.size = 4 * priv->nregs;
imx_ocotp_nvmem_config.dev = dev;
+ imx_ocotp_nvmem_config.priv = priv;
nvmem = nvmem_register(&imx_ocotp_nvmem_config);
if (IS_ERR(nvmem))
return PTR_ERR(nvmem);
diff --git a/drivers/nvmem/lpc18xx_eeprom.c b/drivers/nvmem/lpc18xx_eeprom.c
index 878fce7..c81ae4c 100644
--- a/drivers/nvmem/lpc18xx_eeprom.c
+++ b/drivers/nvmem/lpc18xx_eeprom.c
@@ -16,7 +16,6 @@
#include <linux/module.h>
#include <linux/nvmem-provider.h>
#include <linux/platform_device.h>
-#include <linux/regmap.h>
#include <linux/reset.h>
/* Registers */
@@ -51,12 +50,7 @@
struct nvmem_device *nvmem;
unsigned reg_bytes;
unsigned val_bytes;
-};
-
-static struct regmap_config lpc18xx_regmap_config = {
- .reg_bits = 32,
- .reg_stride = 4,
- .val_bits = 32,
+ int size;
};
static inline void lpc18xx_eeprom_writel(struct lpc18xx_eeprom_dev *eeprom,
@@ -95,30 +89,35 @@
return -ETIMEDOUT;
}
-static int lpc18xx_eeprom_gather_write(void *context, const void *reg,
- size_t reg_size, const void *val,
- size_t val_size)
+static int lpc18xx_eeprom_gather_write(void *context, unsigned int reg,
+ void *val, size_t bytes)
{
struct lpc18xx_eeprom_dev *eeprom = context;
- unsigned int offset = *(u32 *)reg;
+ unsigned int offset = reg;
int ret;
- if (offset % lpc18xx_regmap_config.reg_stride)
+ /*
+ * The last page contains the EEPROM initialization data and is not
+ * writable.
+ */
+ if ((reg > eeprom->size - LPC18XX_EEPROM_PAGE_SIZE) ||
+ (reg + bytes > eeprom->size - LPC18XX_EEPROM_PAGE_SIZE))
return -EINVAL;
+
lpc18xx_eeprom_writel(eeprom, LPC18XX_EEPROM_PWRDWN,
LPC18XX_EEPROM_PWRDWN_NO);
/* Wait 100 us while the EEPROM wakes up */
usleep_range(100, 200);
- while (val_size) {
+ while (bytes) {
writel(*(u32 *)val, eeprom->mem_base + offset);
ret = lpc18xx_eeprom_busywait_until_prog(eeprom);
if (ret < 0)
return ret;
- val_size -= eeprom->val_bytes;
+ bytes -= eeprom->val_bytes;
val += eeprom->val_bytes;
offset += eeprom->val_bytes;
}
@@ -129,23 +128,10 @@
return 0;
}
-static int lpc18xx_eeprom_write(void *context, const void *data, size_t count)
+static int lpc18xx_eeprom_read(void *context, unsigned int offset,
+ void *val, size_t bytes)
{
struct lpc18xx_eeprom_dev *eeprom = context;
- unsigned int offset = eeprom->reg_bytes;
-
- if (count <= offset)
- return -EINVAL;
-
- return lpc18xx_eeprom_gather_write(context, data, eeprom->reg_bytes,
- data + offset, count - offset);
-}
-
-static int lpc18xx_eeprom_read(void *context, const void *reg, size_t reg_size,
- void *val, size_t val_size)
-{
- struct lpc18xx_eeprom_dev *eeprom = context;
- unsigned int offset = *(u32 *)reg;
lpc18xx_eeprom_writel(eeprom, LPC18XX_EEPROM_PWRDWN,
LPC18XX_EEPROM_PWRDWN_NO);
@@ -153,9 +139,9 @@
/* Wait 100 us while the EEPROM wakes up */
usleep_range(100, 200);
- while (val_size) {
+ while (bytes) {
*(u32 *)val = readl(eeprom->mem_base + offset);
- val_size -= eeprom->val_bytes;
+ bytes -= eeprom->val_bytes;
val += eeprom->val_bytes;
offset += eeprom->val_bytes;
}
@@ -166,31 +152,13 @@
return 0;
}
-static struct regmap_bus lpc18xx_eeprom_bus = {
- .write = lpc18xx_eeprom_write,
- .gather_write = lpc18xx_eeprom_gather_write,
- .read = lpc18xx_eeprom_read,
- .reg_format_endian_default = REGMAP_ENDIAN_NATIVE,
- .val_format_endian_default = REGMAP_ENDIAN_NATIVE,
-};
-
-static bool lpc18xx_eeprom_writeable_reg(struct device *dev, unsigned int reg)
-{
- /*
- * The last page contains the EEPROM initialization data and is not
- * writable.
- */
- return reg <= lpc18xx_regmap_config.max_register -
- LPC18XX_EEPROM_PAGE_SIZE;
-}
-
-static bool lpc18xx_eeprom_readable_reg(struct device *dev, unsigned int reg)
-{
- return reg <= lpc18xx_regmap_config.max_register;
-}
static struct nvmem_config lpc18xx_nvmem_config = {
.name = "lpc18xx-eeprom",
+ .stride = 4,
+ .word_size = 4,
+ .reg_read = lpc18xx_eeprom_read,
+ .reg_write = lpc18xx_eeprom_gather_write,
.owner = THIS_MODULE,
};
@@ -200,7 +168,6 @@
struct device *dev = &pdev->dev;
struct reset_control *rst;
unsigned long clk_rate;
- struct regmap *regmap;
struct resource *res;
int ret;
@@ -243,8 +210,8 @@
goto err_clk;
}
- eeprom->val_bytes = lpc18xx_regmap_config.val_bits / BITS_PER_BYTE;
- eeprom->reg_bytes = lpc18xx_regmap_config.reg_bits / BITS_PER_BYTE;
+ eeprom->val_bytes = 4;
+ eeprom->reg_bytes = 4;
/*
* Clock rate is generated by dividing the system bus clock by the
@@ -264,19 +231,10 @@
lpc18xx_eeprom_writel(eeprom, LPC18XX_EEPROM_PWRDWN,
LPC18XX_EEPROM_PWRDWN_YES);
- lpc18xx_regmap_config.max_register = resource_size(res) - 1;
- lpc18xx_regmap_config.writeable_reg = lpc18xx_eeprom_writeable_reg;
- lpc18xx_regmap_config.readable_reg = lpc18xx_eeprom_readable_reg;
-
- regmap = devm_regmap_init(dev, &lpc18xx_eeprom_bus, eeprom,
- &lpc18xx_regmap_config);
- if (IS_ERR(regmap)) {
- dev_err(dev, "regmap init failed: %ld\n", PTR_ERR(regmap));
- ret = PTR_ERR(regmap);
- goto err_clk;
- }
-
+ eeprom->size = resource_size(res);
+ lpc18xx_nvmem_config.size = resource_size(res);
lpc18xx_nvmem_config.dev = dev;
+ lpc18xx_nvmem_config.priv = eeprom;
eeprom->nvmem = nvmem_register(&lpc18xx_nvmem_config);
if (IS_ERR(eeprom->nvmem)) {
diff --git a/drivers/nvmem/qfprom.c b/drivers/nvmem/qfprom.c
index 3829e5f..b5305f0 100644
--- a/drivers/nvmem/qfprom.c
+++ b/drivers/nvmem/qfprom.c
@@ -13,21 +13,35 @@
#include <linux/device.h>
#include <linux/module.h>
+#include <linux/io.h>
#include <linux/nvmem-provider.h>
#include <linux/platform_device.h>
-#include <linux/regmap.h>
-static struct regmap_config qfprom_regmap_config = {
- .reg_bits = 32,
- .val_bits = 8,
- .reg_stride = 1,
- .val_format_endian = REGMAP_ENDIAN_LITTLE,
-};
+static int qfprom_reg_read(void *context,
+ unsigned int reg, void *_val, size_t bytes)
+{
+ void __iomem *base = context;
+ u32 *val = _val;
+ int i = 0, words = bytes / 4;
-static struct nvmem_config econfig = {
- .name = "qfprom",
- .owner = THIS_MODULE,
-};
+ while (words--)
+ *val++ = readl(base + reg + (i++ * 4));
+
+ return 0;
+}
+
+static int qfprom_reg_write(void *context,
+ unsigned int reg, void *_val, size_t bytes)
+{
+ void __iomem *base = context;
+ u32 *val = _val;
+ int i = 0, words = bytes / 4;
+
+ while (words--)
+ writel(*val++, base + reg + (i++ * 4));
+
+ return 0;
+}
static int qfprom_remove(struct platform_device *pdev)
{
@@ -36,12 +50,20 @@
return nvmem_unregister(nvmem);
}
+static struct nvmem_config econfig = {
+ .name = "qfprom",
+ .owner = THIS_MODULE,
+ .stride = 4,
+ .word_size = 1,
+ .reg_read = qfprom_reg_read,
+ .reg_write = qfprom_reg_write,
+};
+
static int qfprom_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct resource *res;
struct nvmem_device *nvmem;
- struct regmap *regmap;
void __iomem *base;
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
@@ -49,14 +71,10 @@
if (IS_ERR(base))
return PTR_ERR(base);
- qfprom_regmap_config.max_register = resource_size(res) - 1;
-
- regmap = devm_regmap_init_mmio(dev, base, &qfprom_regmap_config);
- if (IS_ERR(regmap)) {
- dev_err(dev, "regmap init failed\n");
- return PTR_ERR(regmap);
- }
+ econfig.size = resource_size(res);
econfig.dev = dev;
+ econfig.priv = base;
+
nvmem = nvmem_register(&econfig);
if (IS_ERR(nvmem))
return PTR_ERR(nvmem);
diff --git a/drivers/nvmem/rockchip-efuse.c b/drivers/nvmem/rockchip-efuse.c
index a009795..4d3f391 100644
--- a/drivers/nvmem/rockchip-efuse.c
+++ b/drivers/nvmem/rockchip-efuse.c
@@ -23,7 +23,6 @@
#include <linux/slab.h>
#include <linux/of.h>
#include <linux/platform_device.h>
-#include <linux/regmap.h>
#define EFUSE_A_SHIFT 6
#define EFUSE_A_MASK 0x3ff
@@ -41,17 +40,9 @@
struct clk *clk;
};
-static int rockchip_efuse_write(void *context, const void *data, size_t count)
+static int rockchip_efuse_read(void *context, unsigned int offset,
+ void *val, size_t bytes)
{
- /* Nothing TBD, Read-Only */
- return 0;
-}
-
-static int rockchip_efuse_read(void *context,
- const void *reg, size_t reg_size,
- void *val, size_t val_size)
-{
- unsigned int offset = *(u32 *)reg;
struct rockchip_efuse_chip *efuse = context;
u8 *buf = val;
int ret;
@@ -64,12 +55,12 @@
writel(EFUSE_LOAD | EFUSE_PGENB, efuse->base + REG_EFUSE_CTRL);
udelay(1);
- while (val_size) {
+ while (bytes--) {
writel(readl(efuse->base + REG_EFUSE_CTRL) &
(~(EFUSE_A_MASK << EFUSE_A_SHIFT)),
efuse->base + REG_EFUSE_CTRL);
writel(readl(efuse->base + REG_EFUSE_CTRL) |
- ((offset & EFUSE_A_MASK) << EFUSE_A_SHIFT),
+ ((offset++ & EFUSE_A_MASK) << EFUSE_A_SHIFT),
efuse->base + REG_EFUSE_CTRL);
udelay(1);
writel(readl(efuse->base + REG_EFUSE_CTRL) |
@@ -79,9 +70,6 @@
writel(readl(efuse->base + REG_EFUSE_CTRL) &
(~EFUSE_STROBE), efuse->base + REG_EFUSE_CTRL);
udelay(1);
-
- val_size -= 1;
- offset += 1;
}
/* Switch to standby mode */
@@ -92,22 +80,11 @@
return 0;
}
-static struct regmap_bus rockchip_efuse_bus = {
- .read = rockchip_efuse_read,
- .write = rockchip_efuse_write,
- .reg_format_endian_default = REGMAP_ENDIAN_NATIVE,
- .val_format_endian_default = REGMAP_ENDIAN_NATIVE,
-};
-
-static struct regmap_config rockchip_efuse_regmap_config = {
- .reg_bits = 32,
- .reg_stride = 1,
- .val_bits = 8,
-};
-
static struct nvmem_config econfig = {
.name = "rockchip-efuse",
.owner = THIS_MODULE,
+ .stride = 1,
+ .word_size = 1,
.read_only = true,
};
@@ -121,7 +98,6 @@
{
struct resource *res;
struct nvmem_device *nvmem;
- struct regmap *regmap;
struct rockchip_efuse_chip *efuse;
efuse = devm_kzalloc(&pdev->dev, sizeof(struct rockchip_efuse_chip),
@@ -139,16 +115,9 @@
return PTR_ERR(efuse->clk);
efuse->dev = &pdev->dev;
-
- rockchip_efuse_regmap_config.max_register = resource_size(res) - 1;
-
- regmap = devm_regmap_init(efuse->dev, &rockchip_efuse_bus,
- efuse, &rockchip_efuse_regmap_config);
- if (IS_ERR(regmap)) {
- dev_err(efuse->dev, "regmap init failed\n");
- return PTR_ERR(regmap);
- }
-
+ econfig.size = resource_size(res);
+ econfig.reg_read = rockchip_efuse_read;
+ econfig.priv = efuse;
econfig.dev = efuse->dev;
nvmem = nvmem_register(&econfig);
if (IS_ERR(nvmem))
diff --git a/drivers/nvmem/sunxi_sid.c b/drivers/nvmem/sunxi_sid.c
index bc88b40..1567ccc 100644
--- a/drivers/nvmem/sunxi_sid.c
+++ b/drivers/nvmem/sunxi_sid.c
@@ -21,13 +21,14 @@
#include <linux/nvmem-provider.h>
#include <linux/of.h>
#include <linux/platform_device.h>
-#include <linux/regmap.h>
#include <linux/slab.h>
#include <linux/random.h>
static struct nvmem_config econfig = {
.name = "sunxi-sid",
.read_only = true,
+ .stride = 4,
+ .word_size = 1,
.owner = THIS_MODULE,
};
@@ -51,54 +52,23 @@
return sid_key; /* Only return the last byte */
}
-static int sunxi_sid_read(void *context,
- const void *reg, size_t reg_size,
- void *val, size_t val_size)
+static int sunxi_sid_read(void *context, unsigned int offset,
+ void *val, size_t bytes)
{
struct sunxi_sid *sid = context;
- unsigned int offset = *(u32 *)reg;
u8 *buf = val;
- while (val_size) {
- *buf++ = sunxi_sid_read_byte(sid, offset);
- val_size--;
- offset++;
- }
+ while (bytes--)
+ *buf++ = sunxi_sid_read_byte(sid, offset++);
return 0;
}
-static int sunxi_sid_write(void *context, const void *data, size_t count)
-{
- /* Unimplemented, dummy to keep regmap core happy */
- return 0;
-}
-
-static struct regmap_bus sunxi_sid_bus = {
- .read = sunxi_sid_read,
- .write = sunxi_sid_write,
- .reg_format_endian_default = REGMAP_ENDIAN_NATIVE,
- .val_format_endian_default = REGMAP_ENDIAN_NATIVE,
-};
-
-static bool sunxi_sid_writeable_reg(struct device *dev, unsigned int reg)
-{
- return false;
-}
-
-static struct regmap_config sunxi_sid_regmap_config = {
- .reg_bits = 32,
- .val_bits = 8,
- .reg_stride = 1,
- .writeable_reg = sunxi_sid_writeable_reg,
-};
-
static int sunxi_sid_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct resource *res;
struct nvmem_device *nvmem;
- struct regmap *regmap;
struct sunxi_sid *sid;
int ret, i, size;
char *randomness;
@@ -113,16 +83,10 @@
return PTR_ERR(sid->base);
size = resource_size(res) - 1;
- sunxi_sid_regmap_config.max_register = size;
-
- regmap = devm_regmap_init(dev, &sunxi_sid_bus, sid,
- &sunxi_sid_regmap_config);
- if (IS_ERR(regmap)) {
- dev_err(dev, "regmap init failed\n");
- return PTR_ERR(regmap);
- }
-
+ econfig.size = resource_size(res);
econfig.dev = dev;
+ econfig.reg_read = sunxi_sid_read;
+ econfig.priv = sid;
nvmem = nvmem_register(&econfig);
if (IS_ERR(nvmem))
return PTR_ERR(nvmem);
diff --git a/drivers/nvmem/vf610-ocotp.c b/drivers/nvmem/vf610-ocotp.c
index 8641319..72e4faa 100644
--- a/drivers/nvmem/vf610-ocotp.c
+++ b/drivers/nvmem/vf610-ocotp.c
@@ -25,7 +25,6 @@
#include <linux/nvmem-provider.h>
#include <linux/of.h>
#include <linux/platform_device.h>
-#include <linux/regmap.h>
#include <linux/slab.h>
/* OCOTP Register Offsets */
@@ -152,23 +151,16 @@
return -EINVAL;
}
-static int vf610_ocotp_write(void *context, const void *data, size_t count)
-{
- return 0;
-}
-
-static int vf610_ocotp_read(void *context,
- const void *off, size_t reg_size,
- void *val, size_t val_size)
+static int vf610_ocotp_read(void *context, unsigned int offset,
+ void *val, size_t bytes)
{
struct vf610_ocotp *ocotp = context;
void __iomem *base = ocotp->base;
- unsigned int offset = *(u32 *)off;
u32 reg, *buf = val;
int fuse_addr;
int ret;
- while (val_size > 0) {
+ while (bytes > 0) {
fuse_addr = vf610_get_fuse_address(offset);
if (fuse_addr > 0) {
writel(ocotp->timing, base + OCOTP_TIMING);
@@ -205,29 +197,19 @@
}
buf++;
- val_size--;
- offset += reg_size;
+ bytes -= 4;
+ offset += 4;
}
return 0;
}
-static struct regmap_bus vf610_ocotp_bus = {
- .read = vf610_ocotp_read,
- .write = vf610_ocotp_write,
- .reg_format_endian_default = REGMAP_ENDIAN_NATIVE,
- .val_format_endian_default = REGMAP_ENDIAN_NATIVE,
-};
-
-static struct regmap_config ocotp_regmap_config = {
- .reg_bits = 32,
- .val_bits = 32,
- .reg_stride = 4,
-};
-
static struct nvmem_config ocotp_config = {
.name = "ocotp",
.owner = THIS_MODULE,
+ .stride = 4,
+ .word_size = 4,
+ .reg_read = vf610_ocotp_read,
};
static const struct of_device_id ocotp_of_match[] = {
@@ -247,7 +229,6 @@
{
struct device *dev = &pdev->dev;
struct resource *res;
- struct regmap *regmap;
struct vf610_ocotp *ocotp_dev;
ocotp_dev = devm_kzalloc(&pdev->dev,
@@ -267,13 +248,8 @@
return PTR_ERR(ocotp_dev->clk);
}
- ocotp_regmap_config.max_register = resource_size(res);
- regmap = devm_regmap_init(dev,
- &vf610_ocotp_bus, ocotp_dev, &ocotp_regmap_config);
- if (IS_ERR(regmap)) {
- dev_err(dev, "regmap init failed\n");
- return PTR_ERR(regmap);
- }
+ ocotp_config.size = resource_size(res);
+ ocotp_config.priv = ocotp_dev;
ocotp_config.dev = dev;
ocotp_dev->nvmem = nvmem_register(&ocotp_config);
diff --git a/drivers/of/Makefile b/drivers/of/Makefile
index bee3fa9..d7efd9d 100644
--- a/drivers/of/Makefile
+++ b/drivers/of/Makefile
@@ -10,7 +10,6 @@
obj-$(CONFIG_OF_MDIO) += of_mdio.o
obj-$(CONFIG_OF_PCI) += of_pci.o
obj-$(CONFIG_OF_PCI_IRQ) += of_pci_irq.o
-obj-$(CONFIG_OF_MTD) += of_mtd.o
obj-$(CONFIG_OF_RESERVED_MEM) += of_reserved_mem.o
obj-$(CONFIG_OF_RESOLVE) += resolver.o
obj-$(CONFIG_OF_OVERLAY) += overlay.o
diff --git a/drivers/of/address.c b/drivers/of/address.c
index 91a469d..0a553c0 100644
--- a/drivers/of/address.c
+++ b/drivers/of/address.c
@@ -4,6 +4,7 @@
#include <linux/ioport.h>
#include <linux/module.h>
#include <linux/of_address.h>
+#include <linux/pci.h>
#include <linux/pci_regs.h>
#include <linux/sizes.h>
#include <linux/slab.h>
@@ -673,121 +674,6 @@
}
EXPORT_SYMBOL(of_get_address);
-#ifdef PCI_IOBASE
-struct io_range {
- struct list_head list;
- phys_addr_t start;
- resource_size_t size;
-};
-
-static LIST_HEAD(io_range_list);
-static DEFINE_SPINLOCK(io_range_lock);
-#endif
-
-/*
- * Record the PCI IO range (expressed as CPU physical address + size).
- * Return a negative value if an error has occured, zero otherwise
- */
-int __weak pci_register_io_range(phys_addr_t addr, resource_size_t size)
-{
- int err = 0;
-
-#ifdef PCI_IOBASE
- struct io_range *range;
- resource_size_t allocated_size = 0;
-
- /* check if the range hasn't been previously recorded */
- spin_lock(&io_range_lock);
- list_for_each_entry(range, &io_range_list, list) {
- if (addr >= range->start && addr + size <= range->start + size) {
- /* range already registered, bail out */
- goto end_register;
- }
- allocated_size += range->size;
- }
-
- /* range not registed yet, check for available space */
- if (allocated_size + size - 1 > IO_SPACE_LIMIT) {
- /* if it's too big check if 64K space can be reserved */
- if (allocated_size + SZ_64K - 1 > IO_SPACE_LIMIT) {
- err = -E2BIG;
- goto end_register;
- }
-
- size = SZ_64K;
- pr_warn("Requested IO range too big, new size set to 64K\n");
- }
-
- /* add the range to the list */
- range = kzalloc(sizeof(*range), GFP_ATOMIC);
- if (!range) {
- err = -ENOMEM;
- goto end_register;
- }
-
- range->start = addr;
- range->size = size;
-
- list_add_tail(&range->list, &io_range_list);
-
-end_register:
- spin_unlock(&io_range_lock);
-#endif
-
- return err;
-}
-
-phys_addr_t pci_pio_to_address(unsigned long pio)
-{
- phys_addr_t address = (phys_addr_t)OF_BAD_ADDR;
-
-#ifdef PCI_IOBASE
- struct io_range *range;
- resource_size_t allocated_size = 0;
-
- if (pio > IO_SPACE_LIMIT)
- return address;
-
- spin_lock(&io_range_lock);
- list_for_each_entry(range, &io_range_list, list) {
- if (pio >= allocated_size && pio < allocated_size + range->size) {
- address = range->start + pio - allocated_size;
- break;
- }
- allocated_size += range->size;
- }
- spin_unlock(&io_range_lock);
-#endif
-
- return address;
-}
-
-unsigned long __weak pci_address_to_pio(phys_addr_t address)
-{
-#ifdef PCI_IOBASE
- struct io_range *res;
- resource_size_t offset = 0;
- unsigned long addr = -1;
-
- spin_lock(&io_range_lock);
- list_for_each_entry(res, &io_range_list, list) {
- if (address >= res->start && address < res->start + res->size) {
- addr = address - res->start + offset;
- break;
- }
- offset += res->size;
- }
- spin_unlock(&io_range_lock);
-
- return addr;
-#else
- if (address > IO_SPACE_LIMIT)
- return (unsigned long)-1;
-
- return (unsigned long) address;
-#endif
-}
-
static int __of_address_to_resource(struct device_node *dev,
const __be32 *addrp, u64 size, unsigned int flags,
const char *name, struct resource *r)
diff --git a/drivers/of/base.c b/drivers/of/base.c
index b299de2..ebf84e3 100644
--- a/drivers/of/base.c
+++ b/drivers/of/base.c
@@ -394,7 +394,8 @@
* before booting secondary cores. This function uses arch_match_cpu_phys_id
* which can be overridden by architecture specific implementation.
*
- * Returns a node pointer for the logical cpu if found, else NULL.
+ * Returns a node pointer for the logical cpu with refcount incremented, use
+ * of_node_put() on it when done. Returns NULL if not found.
*/
struct device_node *of_get_cpu_node(int cpu, unsigned int *thread)
{
@@ -1440,106 +1441,155 @@
printk("\n");
}
+int of_phandle_iterator_init(struct of_phandle_iterator *it,
+ const struct device_node *np,
+ const char *list_name,
+ const char *cells_name,
+ int cell_count)
+{
+ const __be32 *list;
+ int size;
+
+ memset(it, 0, sizeof(*it));
+
+ list = of_get_property(np, list_name, &size);
+ if (!list)
+ return -ENOENT;
+
+ it->cells_name = cells_name;
+ it->cell_count = cell_count;
+ it->parent = np;
+ it->list_end = list + size / sizeof(*list);
+ it->phandle_end = list;
+ it->cur = list;
+
+ return 0;
+}
+
+int of_phandle_iterator_next(struct of_phandle_iterator *it)
+{
+ uint32_t count = 0;
+
+ if (it->node) {
+ of_node_put(it->node);
+ it->node = NULL;
+ }
+
+ if (!it->cur || it->phandle_end >= it->list_end)
+ return -ENOENT;
+
+ it->cur = it->phandle_end;
+
+ /* If phandle is 0, then it is an empty entry with no arguments. */
+ it->phandle = be32_to_cpup(it->cur++);
+
+ if (it->phandle) {
+
+ /*
+ * Find the provider node and parse the #*-cells property to
+ * determine the argument length.
+ */
+ it->node = of_find_node_by_phandle(it->phandle);
+
+ if (it->cells_name) {
+ if (!it->node) {
+ pr_err("%s: could not find phandle\n",
+ it->parent->full_name);
+ goto err;
+ }
+
+ if (of_property_read_u32(it->node, it->cells_name,
+ &count)) {
+ pr_err("%s: could not get %s for %s\n",
+ it->parent->full_name,
+ it->cells_name,
+ it->node->full_name);
+ goto err;
+ }
+ } else {
+ count = it->cell_count;
+ }
+
+ /*
+ * Make sure that the arguments actually fit in the remaining
+ * property data length
+ */
+ if (it->cur + count > it->list_end) {
+ pr_err("%s: arguments longer than property\n",
+ it->parent->full_name);
+ goto err;
+ }
+ }
+
+ it->phandle_end = it->cur + count;
+ it->cur_count = count;
+
+ return 0;
+
+err:
+ if (it->node) {
+ of_node_put(it->node);
+ it->node = NULL;
+ }
+
+ return -EINVAL;
+}
+
+int of_phandle_iterator_args(struct of_phandle_iterator *it,
+ uint32_t *args,
+ int size)
+{
+ int i, count;
+
+ count = it->cur_count;
+
+ if (WARN_ON(size < count))
+ count = size;
+
+ for (i = 0; i < count; i++)
+ args[i] = be32_to_cpup(it->cur++);
+
+ return count;
+}
+
static int __of_parse_phandle_with_args(const struct device_node *np,
const char *list_name,
const char *cells_name,
int cell_count, int index,
struct of_phandle_args *out_args)
{
- const __be32 *list, *list_end;
- int rc = 0, size, cur_index = 0;
- uint32_t count = 0;
- struct device_node *node = NULL;
- phandle phandle;
-
- /* Retrieve the phandle list property */
- list = of_get_property(np, list_name, &size);
- if (!list)
- return -ENOENT;
- list_end = list + size / sizeof(*list);
+ struct of_phandle_iterator it;
+ int rc, cur_index = 0;
/* Loop over the phandles until all the requested entry is found */
- while (list < list_end) {
- rc = -EINVAL;
- count = 0;
-
+ of_for_each_phandle(&it, rc, np, list_name, cells_name, cell_count) {
/*
- * If phandle is 0, then it is an empty entry with no
- * arguments. Skip forward to the next entry.
- */
- phandle = be32_to_cpup(list++);
- if (phandle) {
- /*
- * Find the provider node and parse the #*-cells
- * property to determine the argument length.
- *
- * This is not needed if the cell count is hard-coded
- * (i.e. cells_name not set, but cell_count is set),
- * except when we're going to return the found node
- * below.
- */
- if (cells_name || cur_index == index) {
- node = of_find_node_by_phandle(phandle);
- if (!node) {
- pr_err("%s: could not find phandle\n",
- np->full_name);
- goto err;
- }
- }
-
- if (cells_name) {
- if (of_property_read_u32(node, cells_name,
- &count)) {
- pr_err("%s: could not get %s for %s\n",
- np->full_name, cells_name,
- node->full_name);
- goto err;
- }
- } else {
- count = cell_count;
- }
-
- /*
- * Make sure that the arguments actually fit in the
- * remaining property data length
- */
- if (list + count > list_end) {
- pr_err("%s: arguments longer than property\n",
- np->full_name);
- goto err;
- }
- }
-
- /*
- * All of the error cases above bail out of the loop, so at
+ * All of the error cases bail out of the loop, so at
* this point, the parsing is successful. If the requested
* index matches, then fill the out_args structure and return,
* or return -ENOENT for an empty entry.
*/
rc = -ENOENT;
if (cur_index == index) {
- if (!phandle)
+ if (!it.phandle)
goto err;
if (out_args) {
- int i;
- if (WARN_ON(count > MAX_PHANDLE_ARGS))
- count = MAX_PHANDLE_ARGS;
- out_args->np = node;
- out_args->args_count = count;
- for (i = 0; i < count; i++)
- out_args->args[i] = be32_to_cpup(list++);
+ int c;
+
+ c = of_phandle_iterator_args(&it,
+ out_args->args,
+ MAX_PHANDLE_ARGS);
+ out_args->np = it.node;
+ out_args->args_count = c;
} else {
- of_node_put(node);
+ of_node_put(it.node);
}
/* Found it! return success */
return 0;
}
- of_node_put(node);
- node = NULL;
- list += count;
cur_index++;
}
@@ -1547,12 +1597,11 @@
* Unlock node before returning result; will be one of:
* -ENOENT : index is for empty phandle
* -EINVAL : parsing error on data
- * [1..n] : Number of phandle (count mode; when index = -1)
*/
- rc = index < 0 ? cur_index : -ENOENT;
+
err:
- if (node)
- of_node_put(node);
+ if (it.node)
+ of_node_put(it.node);
return rc;
}
@@ -1684,8 +1733,20 @@
int of_count_phandle_with_args(const struct device_node *np, const char *list_name,
const char *cells_name)
{
- return __of_parse_phandle_with_args(np, list_name, cells_name, 0, -1,
- NULL);
+ struct of_phandle_iterator it;
+ int rc, cur_index = 0;
+
+ rc = of_phandle_iterator_init(&it, np, list_name, cells_name, 0);
+ if (rc)
+ return rc;
+
+ while ((rc = of_phandle_iterator_next(&it)) == 0)
+ cur_index += 1;
+
+ if (rc != -ENOENT)
+ return rc;
+
+ return cur_index;
}
EXPORT_SYMBOL(of_count_phandle_with_args);
@@ -1777,6 +1838,9 @@
unsigned long flags;
int rc;
+ if (!prop)
+ return -ENODEV;
+
mutex_lock(&of_mutex);
raw_spin_lock_irqsave(&devtree_lock, flags);
diff --git a/drivers/of/device.c b/drivers/of/device.c
index e5f47ce..fd5cfad 100644
--- a/drivers/of/device.c
+++ b/drivers/of/device.c
@@ -88,7 +88,7 @@
int ret;
bool coherent;
unsigned long offset;
- struct iommu_ops *iommu;
+ const struct iommu_ops *iommu;
/*
* Set default coherent_dma_mask to 32 bit. Drivers are expected to
diff --git a/drivers/of/dynamic.c b/drivers/of/dynamic.c
index c647bd1..3033fa3 100644
--- a/drivers/of/dynamic.c
+++ b/drivers/of/dynamic.c
@@ -311,6 +311,7 @@
return rc;
}
+EXPORT_SYMBOL_GPL(of_detach_node);
/**
* of_node_release() - release a dynamically allocated node
@@ -497,6 +498,11 @@
case OF_RECONFIG_UPDATE_PROPERTY:
rce->old_prop = ce->prop;
rce->prop = ce->old_prop;
+ /* update was used but original property did not exist */
+ if (!rce->prop) {
+ rce->action = OF_RECONFIG_REMOVE_PROPERTY;
+ rce->prop = ce->prop;
+ }
break;
}
}
diff --git a/drivers/of/fdt.c b/drivers/of/fdt.c
index 3349d2a..14f2f8c 100644
--- a/drivers/of/fdt.c
+++ b/drivers/of/fdt.c
@@ -161,39 +161,127 @@
return res;
}
-/**
- * unflatten_dt_node - Alloc and populate a device_node from the flat tree
- * @blob: The parent device tree blob
- * @mem: Memory chunk to use for allocating device nodes and properties
- * @poffset: pointer to node in flat tree
- * @dad: Parent struct device_node
- * @nodepp: The device_node tree created by the call
- * @fpsize: Size of the node path up at the current depth.
- * @dryrun: If true, do not allocate device nodes but still calculate needed
- * memory size
- */
-static void * unflatten_dt_node(const void *blob,
- void *mem,
- int *poffset,
- struct device_node *dad,
- struct device_node **nodepp,
- unsigned long fpsize,
+static void populate_properties(const void *blob,
+ int offset,
+ void **mem,
+ struct device_node *np,
+ const char *nodename,
bool dryrun)
{
- const __be32 *p;
+ struct property *pp, **pprev = NULL;
+ int cur;
+ bool has_name = false;
+
+ pprev = &np->properties;
+ for (cur = fdt_first_property_offset(blob, offset);
+ cur >= 0;
+ cur = fdt_next_property_offset(blob, cur)) {
+ const __be32 *val;
+ const char *pname;
+ u32 sz;
+
+ val = fdt_getprop_by_offset(blob, cur, &pname, &sz);
+ if (!val) {
+ pr_warn("%s: Cannot locate property at 0x%x\n",
+ __func__, cur);
+ continue;
+ }
+
+ if (!pname) {
+ pr_warn("%s: Cannot find property name at 0x%x\n",
+ __func__, cur);
+ continue;
+ }
+
+ if (!strcmp(pname, "name"))
+ has_name = true;
+
+ pp = unflatten_dt_alloc(mem, sizeof(struct property),
+ __alignof__(struct property));
+ if (dryrun)
+ continue;
+
+ /* We accept flattened tree phandles either in
+ * ePAPR-style "phandle" properties, or the
+ * legacy "linux,phandle" properties. If both
+ * appear and have different values, things
+ * will get weird. Don't do that.
+ */
+ if (!strcmp(pname, "phandle") ||
+ !strcmp(pname, "linux,phandle")) {
+ if (!np->phandle)
+ np->phandle = be32_to_cpup(val);
+ }
+
+ /* And we process the "ibm,phandle" property
+ * used in pSeries dynamic device tree
+ * stuff
+ */
+ if (!strcmp(pname, "ibm,phandle"))
+ np->phandle = be32_to_cpup(val);
+
+ pp->name = (char *)pname;
+ pp->length = sz;
+ pp->value = (__be32 *)val;
+ *pprev = pp;
+ pprev = &pp->next;
+ }
+
+ /* With version 0x10 we may not have the name property,
+ * recreate it here from the unit name if absent
+ */
+ if (!has_name) {
+ const char *p = nodename, *ps = p, *pa = NULL;
+ int len;
+
+ while (*p) {
+ if ((*p) == '@')
+ pa = p;
+ else if ((*p) == '/')
+ ps = p + 1;
+ p++;
+ }
+
+ if (pa < ps)
+ pa = p;
+ len = (pa - ps) + 1;
+ pp = unflatten_dt_alloc(mem, sizeof(struct property) + len,
+ __alignof__(struct property));
+ if (!dryrun) {
+ pp->name = "name";
+ pp->length = len;
+ pp->value = pp + 1;
+ *pprev = pp;
+ pprev = &pp->next;
+ memcpy(pp->value, ps, len - 1);
+ ((char *)pp->value)[len - 1] = 0;
+ pr_debug("fixed up name for %s -> %s\n",
+ nodename, (char *)pp->value);
+ }
+ }
+
+ if (!dryrun)
+ *pprev = NULL;
+}
+
+static unsigned int populate_node(const void *blob,
+ int offset,
+ void **mem,
+ struct device_node *dad,
+ unsigned int fpsize,
+ struct device_node **pnp,
+ bool dryrun)
+{
struct device_node *np;
- struct property *pp, **prev_pp = NULL;
const char *pathp;
unsigned int l, allocl;
- static int depth;
- int old_depth;
- int offset;
- int has_name = 0;
int new_format = 0;
- pathp = fdt_get_name(blob, *poffset, &l);
- if (!pathp)
- return mem;
+ pathp = fdt_get_name(blob, offset, &l);
+ if (!pathp) {
+ *pnp = NULL;
+ return 0;
+ }
allocl = ++l;
@@ -223,7 +311,7 @@
}
}
- np = unflatten_dt_alloc(&mem, sizeof(struct device_node) + allocl,
+ np = unflatten_dt_alloc(mem, sizeof(struct device_node) + allocl,
__alignof__(struct device_node));
if (!dryrun) {
char *fn;
@@ -246,89 +334,15 @@
}
memcpy(fn, pathp, l);
- prev_pp = &np->properties;
if (dad != NULL) {
np->parent = dad;
np->sibling = dad->child;
dad->child = np;
}
}
- /* process properties */
- for (offset = fdt_first_property_offset(blob, *poffset);
- (offset >= 0);
- (offset = fdt_next_property_offset(blob, offset))) {
- const char *pname;
- u32 sz;
- if (!(p = fdt_getprop_by_offset(blob, offset, &pname, &sz))) {
- offset = -FDT_ERR_INTERNAL;
- break;
- }
-
- if (pname == NULL) {
- pr_info("Can't find property name in list !\n");
- break;
- }
- if (strcmp(pname, "name") == 0)
- has_name = 1;
- pp = unflatten_dt_alloc(&mem, sizeof(struct property),
- __alignof__(struct property));
- if (!dryrun) {
- /* We accept flattened tree phandles either in
- * ePAPR-style "phandle" properties, or the
- * legacy "linux,phandle" properties. If both
- * appear and have different values, things
- * will get weird. Don't do that. */
- if ((strcmp(pname, "phandle") == 0) ||
- (strcmp(pname, "linux,phandle") == 0)) {
- if (np->phandle == 0)
- np->phandle = be32_to_cpup(p);
- }
- /* And we process the "ibm,phandle" property
- * used in pSeries dynamic device tree
- * stuff */
- if (strcmp(pname, "ibm,phandle") == 0)
- np->phandle = be32_to_cpup(p);
- pp->name = (char *)pname;
- pp->length = sz;
- pp->value = (__be32 *)p;
- *prev_pp = pp;
- prev_pp = &pp->next;
- }
- }
- /* with version 0x10 we may not have the name property, recreate
- * it here from the unit name if absent
- */
- if (!has_name) {
- const char *p1 = pathp, *ps = pathp, *pa = NULL;
- int sz;
-
- while (*p1) {
- if ((*p1) == '@')
- pa = p1;
- if ((*p1) == '/')
- ps = p1 + 1;
- p1++;
- }
- if (pa < ps)
- pa = p1;
- sz = (pa - ps) + 1;
- pp = unflatten_dt_alloc(&mem, sizeof(struct property) + sz,
- __alignof__(struct property));
- if (!dryrun) {
- pp->name = "name";
- pp->length = sz;
- pp->value = pp + 1;
- *prev_pp = pp;
- prev_pp = &pp->next;
- memcpy(pp->value, ps, sz - 1);
- ((char *)pp->value)[sz - 1] = 0;
- pr_debug("fixed up name for %s -> %s\n", pathp,
- (char *)pp->value);
- }
- }
+ populate_properties(blob, offset, mem, np, pathp, dryrun);
if (!dryrun) {
- *prev_pp = NULL;
np->name = of_get_property(np, "name", NULL);
np->type = of_get_property(np, "device_type", NULL);
@@ -338,36 +352,94 @@
np->type = "<NULL>";
}
- old_depth = depth;
- *poffset = fdt_next_node(blob, *poffset, &depth);
- if (depth < 0)
- depth = 0;
- while (*poffset > 0 && depth > old_depth)
- mem = unflatten_dt_node(blob, mem, poffset, np, NULL,
- fpsize, dryrun);
+ *pnp = np;
+ return fpsize;
+}
- if (*poffset < 0 && *poffset != -FDT_ERR_NOTFOUND)
- pr_err("unflatten: error %d processing FDT\n", *poffset);
+static void reverse_nodes(struct device_node *parent)
+{
+ struct device_node *child, *next;
+
+ /* In-depth first */
+ child = parent->child;
+ while (child) {
+ reverse_nodes(child);
+
+ child = child->sibling;
+ }
+
+ /* Reverse the nodes in the child list */
+ child = parent->child;
+ parent->child = NULL;
+ while (child) {
+ next = child->sibling;
+
+ child->sibling = parent->child;
+ parent->child = child;
+ child = next;
+ }
+}
+
+/**
+ * unflatten_dt_nodes - Alloc and populate a device_node from the flat tree
+ * @blob: The parent device tree blob
+ * @mem: Memory chunk to use for allocating device nodes and properties
+ * @dad: Parent struct device_node
+ * @nodepp: The device_node tree created by the call
+ *
+ * It returns the size of unflattened device tree or error code
+ */
+static int unflatten_dt_nodes(const void *blob,
+ void *mem,
+ struct device_node *dad,
+ struct device_node **nodepp)
+{
+ struct device_node *root;
+ int offset = 0, depth = 0;
+#define FDT_MAX_DEPTH 64
+ unsigned int fpsizes[FDT_MAX_DEPTH];
+ struct device_node *nps[FDT_MAX_DEPTH];
+ void *base = mem;
+ bool dryrun = !base;
+
+ if (nodepp)
+ *nodepp = NULL;
+
+ root = dad;
+ fpsizes[depth] = dad ? strlen(of_node_full_name(dad)) : 0;
+ nps[depth] = dad;
+ for (offset = 0;
+ offset >= 0 && depth >= 0;
+ offset = fdt_next_node(blob, offset, &depth)) {
+ if (WARN_ON_ONCE(depth >= FDT_MAX_DEPTH))
+ continue;
+
+ fpsizes[depth+1] = populate_node(blob, offset, &mem,
+ nps[depth],
+ fpsizes[depth],
+ &nps[depth+1], dryrun);
+ if (!fpsizes[depth+1])
+ return mem - base;
+
+ if (!dryrun && nodepp && !*nodepp)
+ *nodepp = nps[depth+1];
+ if (!dryrun && !root)
+ root = nps[depth+1];
+ }
+
+ if (offset < 0 && offset != -FDT_ERR_NOTFOUND) {
+ pr_err("%s: Error %d processing FDT\n", __func__, offset);
+ return -EINVAL;
+ }
/*
* Reverse the child list. Some drivers assumes node order matches .dts
* node order
*/
- if (!dryrun && np->child) {
- struct device_node *child = np->child;
- np->child = NULL;
- while (child) {
- struct device_node *next = child->sibling;
- child->sibling = np->child;
- np->child = child;
- child = next;
- }
- }
+ if (!dryrun)
+ reverse_nodes(root);
- if (nodepp)
- *nodepp = np;
-
- return mem;
+ return mem - base;
}
/**
@@ -378,23 +450,27 @@
* pointers of the nodes so the normal device-tree walking functions
* can be used.
* @blob: The blob to expand
+ * @dad: Parent device node
* @mynodes: The device_node tree created by the call
* @dt_alloc: An allocator that provides a virtual address to memory
* for the resulting tree
+ *
+ * Returns NULL on failure or the memory chunk containing the unflattened
+ * device tree on success.
*/
-static void __unflatten_device_tree(const void *blob,
- struct device_node **mynodes,
- void * (*dt_alloc)(u64 size, u64 align))
+static void *__unflatten_device_tree(const void *blob,
+ struct device_node *dad,
+ struct device_node **mynodes,
+ void *(*dt_alloc)(u64 size, u64 align))
{
- unsigned long size;
- int start;
+ int size;
void *mem;
pr_debug(" -> unflatten_device_tree()\n");
if (!blob) {
pr_debug("No device tree pointer\n");
- return;
+ return NULL;
}
pr_debug("Unflattening device tree:\n");
@@ -404,15 +480,16 @@
if (fdt_check_header(blob)) {
pr_err("Invalid device tree blob header\n");
- return;
+ return NULL;
}
/* First pass, scan for size */
- start = 0;
- size = (unsigned long)unflatten_dt_node(blob, NULL, &start, NULL, NULL, 0, true);
- size = ALIGN(size, 4);
+ size = unflatten_dt_nodes(blob, NULL, dad, NULL);
+ if (size < 0)
+ return NULL;
- pr_debug(" size is %lx, allocating...\n", size);
+ size = ALIGN(size, 4);
+ pr_debug(" size is %d, allocating...\n", size);
/* Allocate memory for the expanded device tree */
mem = dt_alloc(size + 4, __alignof__(struct device_node));
@@ -423,13 +500,13 @@
pr_debug(" unflattening %p...\n", mem);
/* Second pass, do actual unflattening */
- start = 0;
- unflatten_dt_node(blob, mem, &start, NULL, mynodes, 0, false);
+ unflatten_dt_nodes(blob, mem, dad, mynodes);
if (be32_to_cpup(mem + size) != 0xdeadbeef)
pr_warning("End of tree marker overwritten: %08x\n",
be32_to_cpup(mem + size));
pr_debug(" <- unflatten_device_tree()\n");
+ return mem;
}
static void *kernel_tree_alloc(u64 size, u64 align)
@@ -441,18 +518,29 @@
/**
* of_fdt_unflatten_tree - create tree of device_nodes from flat blob
+ * @blob: Flat device tree blob
+ * @dad: Parent device node
+ * @mynodes: The device tree created by the call
*
* unflattens the device-tree passed by the firmware, creating the
* tree of struct device_node. It also fills the "name" and "type"
* pointers of the nodes so the normal device-tree walking functions
* can be used.
+ *
+ * Returns NULL on failure or the memory chunk containing the unflattened
+ * device tree on success.
*/
-void of_fdt_unflatten_tree(const unsigned long *blob,
- struct device_node **mynodes)
+void *of_fdt_unflatten_tree(const unsigned long *blob,
+ struct device_node *dad,
+ struct device_node **mynodes)
{
+ void *mem;
+
mutex_lock(&of_fdt_unflatten_mutex);
- __unflatten_device_tree(blob, mynodes, &kernel_tree_alloc);
+ mem = __unflatten_device_tree(blob, dad, mynodes, &kernel_tree_alloc);
mutex_unlock(&of_fdt_unflatten_mutex);
+
+ return mem;
}
EXPORT_SYMBOL_GPL(of_fdt_unflatten_tree);
@@ -969,10 +1057,16 @@
* is set in which case we override whatever was found earlier.
*/
#ifdef CONFIG_CMDLINE
-#ifndef CONFIG_CMDLINE_FORCE
+#if defined(CONFIG_CMDLINE_EXTEND)
+ strlcat(data, " ", COMMAND_LINE_SIZE);
+ strlcat(data, CONFIG_CMDLINE, COMMAND_LINE_SIZE);
+#elif defined(CONFIG_CMDLINE_FORCE)
+ strlcpy(data, CONFIG_CMDLINE, COMMAND_LINE_SIZE);
+#else
+ /* No arguments from boot loader, use kernel's cmdl*/
if (!((char *)data)[0])
-#endif
strlcpy(data, CONFIG_CMDLINE, COMMAND_LINE_SIZE);
+#endif
#endif /* CONFIG_CMDLINE */
pr_debug("Command line is: %s\n", (char*)data);
@@ -1118,7 +1212,7 @@
*/
void __init unflatten_device_tree(void)
{
- __unflatten_device_tree(initial_boot_params, &of_root,
+ __unflatten_device_tree(initial_boot_params, NULL, &of_root,
early_init_dt_alloc_memory_arch);
/* Get pointer to "/chosen" and "/aliases" nodes for use everywhere */
diff --git a/drivers/of/of_mtd.c b/drivers/of/of_mtd.c
deleted file mode 100644
index b7361ed..0000000
--- a/drivers/of/of_mtd.c
+++ /dev/null
@@ -1,119 +0,0 @@
-/*
- * Copyright 2012 Jean-Christophe PLAGNIOL-VILLARD <plagnioj@jcrosoft.com>
- *
- * OF helpers for mtd.
- *
- * This file is released under the GPLv2
- *
- */
-#include <linux/kernel.h>
-#include <linux/of_mtd.h>
-#include <linux/mtd/nand.h>
-#include <linux/export.h>
-
-/**
- * It maps 'enum nand_ecc_modes_t' found in include/linux/mtd/nand.h
- * into the device tree binding of 'nand-ecc', so that MTD
- * device driver can get nand ecc from device tree.
- */
-static const char *nand_ecc_modes[] = {
- [NAND_ECC_NONE] = "none",
- [NAND_ECC_SOFT] = "soft",
- [NAND_ECC_HW] = "hw",
- [NAND_ECC_HW_SYNDROME] = "hw_syndrome",
- [NAND_ECC_HW_OOB_FIRST] = "hw_oob_first",
- [NAND_ECC_SOFT_BCH] = "soft_bch",
-};
-
-/**
- * of_get_nand_ecc_mode - Get nand ecc mode for given device_node
- * @np: Pointer to the given device_node
- *
- * The function gets ecc mode string from property 'nand-ecc-mode',
- * and return its index in nand_ecc_modes table, or errno in error case.
- */
-int of_get_nand_ecc_mode(struct device_node *np)
-{
- const char *pm;
- int err, i;
-
- err = of_property_read_string(np, "nand-ecc-mode", &pm);
- if (err < 0)
- return err;
-
- for (i = 0; i < ARRAY_SIZE(nand_ecc_modes); i++)
- if (!strcasecmp(pm, nand_ecc_modes[i]))
- return i;
-
- return -ENODEV;
-}
-EXPORT_SYMBOL_GPL(of_get_nand_ecc_mode);
-
-/**
- * of_get_nand_ecc_step_size - Get ECC step size associated to
- * the required ECC strength (see below).
- * @np: Pointer to the given device_node
- *
- * return the ECC step size, or errno in error case.
- */
-int of_get_nand_ecc_step_size(struct device_node *np)
-{
- int ret;
- u32 val;
-
- ret = of_property_read_u32(np, "nand-ecc-step-size", &val);
- return ret ? ret : val;
-}
-EXPORT_SYMBOL_GPL(of_get_nand_ecc_step_size);
-
-/**
- * of_get_nand_ecc_strength - Get required ECC strength over the
- * correspnding step size as defined by 'nand-ecc-size'
- * @np: Pointer to the given device_node
- *
- * return the ECC strength, or errno in error case.
- */
-int of_get_nand_ecc_strength(struct device_node *np)
-{
- int ret;
- u32 val;
-
- ret = of_property_read_u32(np, "nand-ecc-strength", &val);
- return ret ? ret : val;
-}
-EXPORT_SYMBOL_GPL(of_get_nand_ecc_strength);
-
-/**
- * of_get_nand_bus_width - Get nand bus witdh for given device_node
- * @np: Pointer to the given device_node
- *
- * return bus width option, or errno in error case.
- */
-int of_get_nand_bus_width(struct device_node *np)
-{
- u32 val;
-
- if (of_property_read_u32(np, "nand-bus-width", &val))
- return 8;
-
- switch(val) {
- case 8:
- case 16:
- return val;
- default:
- return -EIO;
- }
-}
-EXPORT_SYMBOL_GPL(of_get_nand_bus_width);
-
-/**
- * of_get_nand_on_flash_bbt - Get nand on flash bbt for given device_node
- * @np: Pointer to the given device_node
- *
- * return true if present false other wise
- */
-bool of_get_nand_on_flash_bbt(struct device_node *np)
-{
- return of_property_read_bool(np, "nand-on-flash-bbt");
-}
-EXPORT_SYMBOL_GPL(of_get_nand_on_flash_bbt);
diff --git a/drivers/of/unittest.c b/drivers/of/unittest.c
index e986e6e..f34ed93 100644
--- a/drivers/of/unittest.c
+++ b/drivers/of/unittest.c
@@ -8,7 +8,6 @@
#include <linux/err.h>
#include <linux/errno.h>
#include <linux/hashtable.h>
-#include <linux/module.h>
#include <linux/of.h>
#include <linux/of_fdt.h>
#include <linux/of_irq.h>
@@ -921,7 +920,7 @@
"not running tests\n", __func__);
return -ENOMEM;
}
- of_fdt_unflatten_tree(unittest_data, &unittest_data_node);
+ of_fdt_unflatten_tree(unittest_data, NULL, &unittest_data_node);
if (!unittest_data_node) {
pr_warn("%s: No tree to attach; not running tests\n", __func__);
return -ENODATA;
@@ -1692,13 +1691,7 @@
#if IS_BUILTIN(CONFIG_I2C_MUX)
-struct unittest_i2c_mux_data {
- int nchans;
- struct i2c_adapter *adap[];
-};
-
-static int unittest_i2c_mux_select_chan(struct i2c_adapter *adap,
- void *client, u32 chan)
+static int unittest_i2c_mux_select_chan(struct i2c_mux_core *muxc, u32 chan)
{
return 0;
}
@@ -1706,11 +1699,11 @@
static int unittest_i2c_mux_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
- int ret, i, nchans, size;
+ int ret, i, nchans;
struct device *dev = &client->dev;
struct i2c_adapter *adap = to_i2c_adapter(dev->parent);
struct device_node *np = client->dev.of_node, *child;
- struct unittest_i2c_mux_data *stm;
+ struct i2c_mux_core *muxc;
u32 reg, max_reg;
dev_dbg(dev, "%s for node @%s\n", __func__, np->full_name);
@@ -1734,25 +1727,20 @@
return -EINVAL;
}
- size = offsetof(struct unittest_i2c_mux_data, adap[nchans]);
- stm = devm_kzalloc(dev, size, GFP_KERNEL);
- if (!stm) {
- dev_err(dev, "Out of memory\n");
+ muxc = i2c_mux_alloc(adap, dev, nchans, 0, 0,
+ unittest_i2c_mux_select_chan, NULL);
+ if (!muxc)
return -ENOMEM;
- }
- stm->nchans = nchans;
for (i = 0; i < nchans; i++) {
- stm->adap[i] = i2c_add_mux_adapter(adap, dev, client,
- 0, i, 0, unittest_i2c_mux_select_chan, NULL);
- if (!stm->adap[i]) {
+ ret = i2c_mux_add_adapter(muxc, 0, i, 0);
+ if (ret) {
dev_err(dev, "Failed to register mux #%d\n", i);
- for (i--; i >= 0; i--)
- i2c_del_mux_adapter(stm->adap[i]);
+ i2c_mux_del_adapters(muxc);
return -ENODEV;
}
}
- i2c_set_clientdata(client, stm);
+ i2c_set_clientdata(client, muxc);
return 0;
};
@@ -1761,12 +1749,10 @@
{
struct device *dev = &client->dev;
struct device_node *np = client->dev.of_node;
- struct unittest_i2c_mux_data *stm = i2c_get_clientdata(client);
- int i;
+ struct i2c_mux_core *muxc = i2c_get_clientdata(client);
dev_dbg(dev, "%s for node @%s\n", __func__, np->full_name);
- for (i = stm->nchans - 1; i >= 0; i--)
- i2c_del_mux_adapter(stm->adap[i]);
+ i2c_mux_del_adapters(muxc);
return 0;
}
diff --git a/drivers/parport/procfs.c b/drivers/parport/procfs.c
index c776333..74ed3e4 100644
--- a/drivers/parport/procfs.c
+++ b/drivers/parport/procfs.c
@@ -617,5 +617,5 @@
}
#endif
-module_init(parport_default_proc_register)
+subsys_initcall(parport_default_proc_register)
module_exit(parport_default_proc_unregister)
diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
index 209292e..56389be 100644
--- a/drivers/pci/Kconfig
+++ b/drivers/pci/Kconfig
@@ -83,6 +83,9 @@
config PCI_ATS
bool
+config PCI_ECAM
+ bool
+
config PCI_IOV
bool "PCI IOV support"
depends on PCI
diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile
index 2154092..1fa6925 100644
--- a/drivers/pci/Makefile
+++ b/drivers/pci/Makefile
@@ -55,6 +55,8 @@
obj-$(CONFIG_PCI_STUB) += pci-stub.o
+obj-$(CONFIG_PCI_ECAM) += ecam.o
+
obj-$(CONFIG_XEN_PCIDEV_FRONTEND) += xen-pcifront.o
obj-$(CONFIG_OF) += of.o
diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
new file mode 100644
index 0000000..f9832ad
--- /dev/null
+++ b/drivers/pci/ecam.c
@@ -0,0 +1,164 @@
+/*
+ * Copyright 2016 Broadcom
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation (the "GPL").
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License version 2 (GPLv2) for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 (GPLv2) along with this source code.
+ */
+
+#include <linux/device.h>
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/slab.h>
+
+#include "ecam.h"
+
+/*
+ * On 64-bit systems, we do a single ioremap for the whole config space
+ * since we have enough virtual address range available. On 32-bit, we
+ * ioremap the config space for each bus individually.
+ */
+static const bool per_bus_mapping = !config_enabled(CONFIG_64BIT);
+
+/*
+ * Create a PCI config space window
+ * - reserve mem region
+ * - alloc struct pci_config_window with space for all mappings
+ * - ioremap the config space
+ */
+struct pci_config_window *pci_ecam_create(struct device *dev,
+ struct resource *cfgres, struct resource *busr,
+ struct pci_ecam_ops *ops)
+{
+ struct pci_config_window *cfg;
+ unsigned int bus_range, bus_range_max, bsz;
+ struct resource *conflict;
+ int i, err;
+
+ if (busr->start > busr->end)
+ return ERR_PTR(-EINVAL);
+
+ cfg = kzalloc(sizeof(*cfg), GFP_KERNEL);
+ if (!cfg)
+ return ERR_PTR(-ENOMEM);
+
+ cfg->ops = ops;
+ cfg->busr.start = busr->start;
+ cfg->busr.end = busr->end;
+ cfg->busr.flags = IORESOURCE_BUS;
+ bus_range = resource_size(&cfg->busr);
+ bus_range_max = resource_size(cfgres) >> ops->bus_shift;
+ if (bus_range > bus_range_max) {
+ bus_range = bus_range_max;
+ cfg->busr.end = busr->start + bus_range - 1;
+ dev_warn(dev, "ECAM area %pR can only accommodate %pR (reduced from %pR desired)\n",
+ cfgres, &cfg->busr, busr);
+ }
+ bsz = 1 << ops->bus_shift;
+
+ cfg->res.start = cfgres->start;
+ cfg->res.end = cfgres->end;
+ cfg->res.flags = IORESOURCE_MEM | IORESOURCE_BUSY;
+ cfg->res.name = "PCI ECAM";
+
+ conflict = request_resource_conflict(&iomem_resource, &cfg->res);
+ if (conflict) {
+ err = -EBUSY;
+ dev_err(dev, "can't claim ECAM area %pR: address conflict with %s %pR\n",
+ &cfg->res, conflict->name, conflict);
+ goto err_exit;
+ }
+
+ if (per_bus_mapping) {
+ cfg->winp = kcalloc(bus_range, sizeof(*cfg->winp), GFP_KERNEL);
+ if (!cfg->winp)
+ goto err_exit_malloc;
+ for (i = 0; i < bus_range; i++) {
+ cfg->winp[i] = ioremap(cfgres->start + i * bsz, bsz);
+ if (!cfg->winp[i])
+ goto err_exit_iomap;
+ }
+ } else {
+ cfg->win = ioremap(cfgres->start, bus_range * bsz);
+ if (!cfg->win)
+ goto err_exit_iomap;
+ }
+
+ if (ops->init) {
+ err = ops->init(dev, cfg);
+ if (err)
+ goto err_exit;
+ }
+ dev_info(dev, "ECAM at %pR for %pR\n", &cfg->res, &cfg->busr);
+ return cfg;
+
+err_exit_iomap:
+ dev_err(dev, "ECAM ioremap failed\n");
+err_exit_malloc:
+ err = -ENOMEM;
+err_exit:
+ pci_ecam_free(cfg);
+ return ERR_PTR(err);
+}
+
+void pci_ecam_free(struct pci_config_window *cfg)
+{
+ int i;
+
+ if (per_bus_mapping) {
+ if (cfg->winp) {
+ for (i = 0; i < resource_size(&cfg->busr); i++)
+ if (cfg->winp[i])
+ iounmap(cfg->winp[i]);
+ kfree(cfg->winp);
+ }
+ } else {
+ if (cfg->win)
+ iounmap(cfg->win);
+ }
+ if (cfg->res.parent)
+ release_resource(&cfg->res);
+ kfree(cfg);
+}
+
+/*
+ * Function to implement the pci_ops ->map_bus method
+ */
+void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
+ int where)
+{
+ struct pci_config_window *cfg = bus->sysdata;
+ unsigned int devfn_shift = cfg->ops->bus_shift - 8;
+ unsigned int busn = bus->number;
+ void __iomem *base;
+
+ if (busn < cfg->busr.start || busn > cfg->busr.end)
+ return NULL;
+
+ busn -= cfg->busr.start;
+ if (per_bus_mapping)
+ base = cfg->winp[busn];
+ else
+ base = cfg->win + (busn << cfg->ops->bus_shift);
+ return base + (devfn << devfn_shift) + where;
+}
+
+/* ECAM ops */
+struct pci_ecam_ops pci_generic_ecam_ops = {
+ .bus_shift = 20,
+ .pci_ops = {
+ .map_bus = pci_ecam_map_bus,
+ .read = pci_generic_config_read,
+ .write = pci_generic_config_write,
+ }
+};
diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h
new file mode 100644
index 0000000..9878beb
--- /dev/null
+++ b/drivers/pci/ecam.h
@@ -0,0 +1,67 @@
+/*
+ * Copyright 2016 Broadcom
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation (the "GPL").
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License version 2 (GPLv2) for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 (GPLv2) along with this source code.
+ */
+#ifndef DRIVERS_PCI_ECAM_H
+#define DRIVERS_PCI_ECAM_H
+
+#include <linux/kernel.h>
+#include <linux/platform_device.h>
+
+/*
+ * struct to hold pci ops and bus shift of the config window
+ * for a PCI controller.
+ */
+struct pci_config_window;
+struct pci_ecam_ops {
+ unsigned int bus_shift;
+ struct pci_ops pci_ops;
+ int (*init)(struct device *,
+ struct pci_config_window *);
+};
+
+/*
+ * struct to hold the mappings of a config space window. This
+ * is expected to be used as sysdata for PCI controllers that
+ * use ECAM.
+ */
+struct pci_config_window {
+ struct resource res;
+ struct resource busr;
+ void *priv;
+ struct pci_ecam_ops *ops;
+ union {
+ void __iomem *win; /* 64-bit single mapping */
+ void __iomem **winp; /* 32-bit per-bus mapping */
+ };
+};
+
+/* create and free pci_config_window */
+struct pci_config_window *pci_ecam_create(struct device *dev,
+ struct resource *cfgres, struct resource *busr,
+ struct pci_ecam_ops *ops);
+void pci_ecam_free(struct pci_config_window *cfg);
+
+/* map_bus when ->sysdata is an instance of pci_config_window */
+void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
+ int where);
+/* default ECAM ops */
+extern struct pci_ecam_ops pci_generic_ecam_ops;
+
+#ifdef CONFIG_PCI_HOST_GENERIC
+/* for DT-based PCI controllers that support ECAM */
+int pci_host_common_probe(struct platform_device *pdev,
+ struct pci_ecam_ops *ops);
+#endif
+#endif
diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig
index 8fb1cf5..5d2374e 100644
--- a/drivers/pci/host/Kconfig
+++ b/drivers/pci/host/Kconfig
@@ -72,11 +72,14 @@
config PCIE_RCAR
bool "Renesas R-Car PCIe controller"
depends on ARCH_RENESAS || (ARM && COMPILE_TEST)
+ select PCI_MSI
+ select PCI_MSI_IRQ_DOMAIN
help
Say Y here if you want PCIe controller support on R-Car SoCs.
config PCI_HOST_COMMON
bool
+ select PCI_ECAM
config PCI_HOST_GENERIC
bool "Generic PCI host controller"
@@ -231,4 +234,15 @@
help
Say Y here if you want ECAM support for CN88XX-Pass-1.x Cavium Thunder SoCs.
+config PCIE_ARMADA_8K
+ bool "Marvell Armada-8K PCIe controller"
+ depends on ARCH_MVEBU
+ select PCIE_DW
+ select PCIEPORTBUS
+ help
+ Say Y here if you want to enable PCIe controller support on
+ Armada-8K SoCs. The PCIe controller on Armada-8K is based on
+ Designware hardware and therefore the driver re-uses the
+ Designware core functions to implement the driver.
+
endmenu
diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
index d3d8e1b..9c8698e 100644
--- a/drivers/pci/host/Makefile
+++ b/drivers/pci/host/Makefile
@@ -28,3 +28,4 @@
obj-$(CONFIG_PCIE_QCOM) += pcie-qcom.o
obj-$(CONFIG_PCI_HOST_THUNDER_ECAM) += pci-thunder-ecam.o
obj-$(CONFIG_PCI_HOST_THUNDER_PEM) += pci-thunder-pem.o
+obj-$(CONFIG_PCIE_ARMADA_8K) += pcie-armada8k.o
diff --git a/drivers/pci/host/pci-dra7xx.c b/drivers/pci/host/pci-dra7xx.c
index 2ca3a1f..f441130 100644
--- a/drivers/pci/host/pci-dra7xx.c
+++ b/drivers/pci/host/pci-dra7xx.c
@@ -142,13 +142,13 @@
static void dra7xx_pcie_host_init(struct pcie_port *pp)
{
- dw_pcie_setup_rc(pp);
-
pp->io_base &= DRA7XX_CPU_TO_BUS_ADDR;
pp->mem_base &= DRA7XX_CPU_TO_BUS_ADDR;
pp->cfg0_base &= DRA7XX_CPU_TO_BUS_ADDR;
pp->cfg1_base &= DRA7XX_CPU_TO_BUS_ADDR;
+ dw_pcie_setup_rc(pp);
+
dra7xx_pcie_establish_link(pp);
if (IS_ENABLED(CONFIG_PCI_MSI))
dw_pcie_msi_init(pp);
diff --git a/drivers/pci/host/pci-host-common.c b/drivers/pci/host/pci-host-common.c
index e9f850f..8cba7ab 100644
--- a/drivers/pci/host/pci-host-common.c
+++ b/drivers/pci/host/pci-host-common.c
@@ -22,27 +22,21 @@
#include <linux/of_pci.h>
#include <linux/platform_device.h>
-#include "pci-host-common.h"
+#include "../ecam.h"
-static void gen_pci_release_of_pci_ranges(struct gen_pci *pci)
-{
- pci_free_resource_list(&pci->resources);
-}
-
-static int gen_pci_parse_request_of_pci_ranges(struct gen_pci *pci)
+static int gen_pci_parse_request_of_pci_ranges(struct device *dev,
+ struct list_head *resources, struct resource **bus_range)
{
int err, res_valid = 0;
- struct device *dev = pci->host.dev.parent;
struct device_node *np = dev->of_node;
resource_size_t iobase;
struct resource_entry *win;
- err = of_pci_get_host_bridge_resources(np, 0, 0xff, &pci->resources,
- &iobase);
+ err = of_pci_get_host_bridge_resources(np, 0, 0xff, resources, &iobase);
if (err)
return err;
- resource_list_for_each_entry(win, &pci->resources) {
+ resource_list_for_each_entry(win, resources) {
struct resource *parent, *res = win->res;
switch (resource_type(res)) {
@@ -60,7 +54,7 @@
res_valid |= !(res->flags & IORESOURCE_PREFETCH);
break;
case IORESOURCE_BUS:
- pci->cfg.bus_range = res;
+ *bus_range = res;
default:
continue;
}
@@ -79,65 +73,60 @@
return 0;
out_release_res:
- gen_pci_release_of_pci_ranges(pci);
return err;
}
-static int gen_pci_parse_map_cfg_windows(struct gen_pci *pci)
+static void gen_pci_unmap_cfg(void *ptr)
+{
+ pci_ecam_free((struct pci_config_window *)ptr);
+}
+
+static struct pci_config_window *gen_pci_init(struct device *dev,
+ struct list_head *resources, struct pci_ecam_ops *ops)
{
int err;
- u8 bus_max;
- resource_size_t busn;
- struct resource *bus_range;
- struct device *dev = pci->host.dev.parent;
- struct device_node *np = dev->of_node;
- u32 sz = 1 << pci->cfg.ops->bus_shift;
+ struct resource cfgres;
+ struct resource *bus_range = NULL;
+ struct pci_config_window *cfg;
- err = of_address_to_resource(np, 0, &pci->cfg.res);
+ /* Parse our PCI ranges and request their resources */
+ err = gen_pci_parse_request_of_pci_ranges(dev, resources, &bus_range);
+ if (err)
+ goto err_out;
+
+ err = of_address_to_resource(dev->of_node, 0, &cfgres);
if (err) {
dev_err(dev, "missing \"reg\" property\n");
- return err;
+ goto err_out;
}
- /* Limit the bus-range to fit within reg */
- bus_max = pci->cfg.bus_range->start +
- (resource_size(&pci->cfg.res) >> pci->cfg.ops->bus_shift) - 1;
- pci->cfg.bus_range->end = min_t(resource_size_t,
- pci->cfg.bus_range->end, bus_max);
-
- pci->cfg.win = devm_kcalloc(dev, resource_size(pci->cfg.bus_range),
- sizeof(*pci->cfg.win), GFP_KERNEL);
- if (!pci->cfg.win)
- return -ENOMEM;
-
- /* Map our Configuration Space windows */
- if (!devm_request_mem_region(dev, pci->cfg.res.start,
- resource_size(&pci->cfg.res),
- "Configuration Space"))
- return -ENOMEM;
-
- bus_range = pci->cfg.bus_range;
- for (busn = bus_range->start; busn <= bus_range->end; ++busn) {
- u32 idx = busn - bus_range->start;
-
- pci->cfg.win[idx] = devm_ioremap(dev,
- pci->cfg.res.start + idx * sz,
- sz);
- if (!pci->cfg.win[idx])
- return -ENOMEM;
+ cfg = pci_ecam_create(dev, &cfgres, bus_range, ops);
+ if (IS_ERR(cfg)) {
+ err = PTR_ERR(cfg);
+ goto err_out;
}
- return 0;
+ err = devm_add_action(dev, gen_pci_unmap_cfg, cfg);
+ if (err) {
+ gen_pci_unmap_cfg(cfg);
+ goto err_out;
+ }
+ return cfg;
+
+err_out:
+ pci_free_resource_list(resources);
+ return ERR_PTR(err);
}
int pci_host_common_probe(struct platform_device *pdev,
- struct gen_pci *pci)
+ struct pci_ecam_ops *ops)
{
- int err;
const char *type;
struct device *dev = &pdev->dev;
struct device_node *np = dev->of_node;
struct pci_bus *bus, *child;
+ struct pci_config_window *cfg;
+ struct list_head resources;
type = of_get_property(np, "device_type", NULL);
if (!type || strcmp(type, "pci")) {
@@ -147,29 +136,18 @@
of_pci_check_probe_only();
- pci->host.dev.parent = dev;
- INIT_LIST_HEAD(&pci->host.windows);
- INIT_LIST_HEAD(&pci->resources);
-
- /* Parse our PCI ranges and request their resources */
- err = gen_pci_parse_request_of_pci_ranges(pci);
- if (err)
- return err;
-
/* Parse and map our Configuration Space windows */
- err = gen_pci_parse_map_cfg_windows(pci);
- if (err) {
- gen_pci_release_of_pci_ranges(pci);
- return err;
- }
+ INIT_LIST_HEAD(&resources);
+ cfg = gen_pci_init(dev, &resources, ops);
+ if (IS_ERR(cfg))
+ return PTR_ERR(cfg);
/* Do not reassign resources if probe only */
if (!pci_has_flag(PCI_PROBE_ONLY))
pci_add_flags(PCI_REASSIGN_ALL_RSRC | PCI_REASSIGN_ALL_BUS);
-
- bus = pci_scan_root_bus(dev, pci->cfg.bus_range->start,
- &pci->cfg.ops->ops, pci, &pci->resources);
+ bus = pci_scan_root_bus(dev, cfg->busr.start, &ops->pci_ops, cfg,
+ &resources);
if (!bus) {
dev_err(dev, "Scanning rootbus failed");
return -ENODEV;
diff --git a/drivers/pci/host/pci-host-common.h b/drivers/pci/host/pci-host-common.h
deleted file mode 100644
index 09f3fa0..0000000
--- a/drivers/pci/host/pci-host-common.h
+++ /dev/null
@@ -1,47 +0,0 @@
-/*
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- *
- * Copyright (C) 2014 ARM Limited
- *
- * Author: Will Deacon <will.deacon@arm.com>
- */
-
-#ifndef _PCI_HOST_COMMON_H
-#define _PCI_HOST_COMMON_H
-
-#include <linux/kernel.h>
-#include <linux/platform_device.h>
-
-struct gen_pci_cfg_bus_ops {
- u32 bus_shift;
- struct pci_ops ops;
-};
-
-struct gen_pci_cfg_windows {
- struct resource res;
- struct resource *bus_range;
- void __iomem **win;
-
- struct gen_pci_cfg_bus_ops *ops;
-};
-
-struct gen_pci {
- struct pci_host_bridge host;
- struct gen_pci_cfg_windows cfg;
- struct list_head resources;
-};
-
-int pci_host_common_probe(struct platform_device *pdev,
- struct gen_pci *pci);
-
-#endif /* _PCI_HOST_COMMON_H */
diff --git a/drivers/pci/host/pci-host-generic.c b/drivers/pci/host/pci-host-generic.c
index e8aa78f..6eaceab 100644
--- a/drivers/pci/host/pci-host-generic.c
+++ b/drivers/pci/host/pci-host-generic.c
@@ -25,41 +25,12 @@
#include <linux/of_pci.h>
#include <linux/platform_device.h>
-#include "pci-host-common.h"
+#include "../ecam.h"
-static void __iomem *gen_pci_map_cfg_bus_cam(struct pci_bus *bus,
- unsigned int devfn,
- int where)
-{
- struct gen_pci *pci = bus->sysdata;
- resource_size_t idx = bus->number - pci->cfg.bus_range->start;
-
- return pci->cfg.win[idx] + ((devfn << 8) | where);
-}
-
-static struct gen_pci_cfg_bus_ops gen_pci_cfg_cam_bus_ops = {
+static struct pci_ecam_ops gen_pci_cfg_cam_bus_ops = {
.bus_shift = 16,
- .ops = {
- .map_bus = gen_pci_map_cfg_bus_cam,
- .read = pci_generic_config_read,
- .write = pci_generic_config_write,
- }
-};
-
-static void __iomem *gen_pci_map_cfg_bus_ecam(struct pci_bus *bus,
- unsigned int devfn,
- int where)
-{
- struct gen_pci *pci = bus->sysdata;
- resource_size_t idx = bus->number - pci->cfg.bus_range->start;
-
- return pci->cfg.win[idx] + ((devfn << 12) | where);
-}
-
-static struct gen_pci_cfg_bus_ops gen_pci_cfg_ecam_bus_ops = {
- .bus_shift = 20,
- .ops = {
- .map_bus = gen_pci_map_cfg_bus_ecam,
+ .pci_ops = {
+ .map_bus = pci_ecam_map_bus,
.read = pci_generic_config_read,
.write = pci_generic_config_write,
}
@@ -70,25 +41,22 @@
.data = &gen_pci_cfg_cam_bus_ops },
{ .compatible = "pci-host-ecam-generic",
- .data = &gen_pci_cfg_ecam_bus_ops },
+ .data = &pci_generic_ecam_ops },
{ },
};
+
MODULE_DEVICE_TABLE(of, gen_pci_of_match);
static int gen_pci_probe(struct platform_device *pdev)
{
- struct device *dev = &pdev->dev;
const struct of_device_id *of_id;
- struct gen_pci *pci = devm_kzalloc(dev, sizeof(*pci), GFP_KERNEL);
+ struct pci_ecam_ops *ops;
- if (!pci)
- return -ENOMEM;
+ of_id = of_match_node(gen_pci_of_match, pdev->dev.of_node);
+ ops = (struct pci_ecam_ops *)of_id->data;
- of_id = of_match_node(gen_pci_of_match, dev->of_node);
- pci->cfg.ops = (struct gen_pci_cfg_bus_ops *)of_id->data;
-
- return pci_host_common_probe(pdev, pci);
+ return pci_host_common_probe(pdev, ops);
}
static struct platform_driver gen_pci_driver = {
diff --git a/drivers/pci/host/pci-hyperv.c b/drivers/pci/host/pci-hyperv.c
index ed651ba..7e9b2de 100644
--- a/drivers/pci/host/pci-hyperv.c
+++ b/drivers/pci/host/pci-hyperv.c
@@ -553,6 +553,8 @@
spin_lock_irqsave(&hpdev->hbus->config_lock, flags);
/* Choose the function to be read. (See comment above) */
writel(hpdev->desc.win_slot.slot, hpdev->hbus->cfg_addr);
+ /* Make sure the function was chosen before we start reading. */
+ mb();
/* Read from that function's config space. */
switch (size) {
case 1:
@@ -565,6 +567,11 @@
*val = readl(addr);
break;
}
+ /*
+ * Make sure the write was done before we release the spinlock
+ * allowing consecutive reads/writes.
+ */
+ mb();
spin_unlock_irqrestore(&hpdev->hbus->config_lock, flags);
} else {
dev_err(&hpdev->hbus->hdev->device,
@@ -592,6 +599,8 @@
spin_lock_irqsave(&hpdev->hbus->config_lock, flags);
/* Choose the function to be written. (See comment above) */
writel(hpdev->desc.win_slot.slot, hpdev->hbus->cfg_addr);
+ /* Make sure the function was chosen before we start writing. */
+ wmb();
/* Write to that function's config space. */
switch (size) {
case 1:
@@ -604,6 +613,11 @@
writel(val, addr);
break;
}
+ /*
+ * Make sure the write was done before we release the spinlock
+ * allowing consecutive reads/writes.
+ */
+ mb();
spin_unlock_irqrestore(&hpdev->hbus->config_lock, flags);
} else {
dev_err(&hpdev->hbus->hdev->device,
@@ -1795,14 +1809,14 @@
if (hbus->low_mmio_space && hbus->low_mmio_res) {
hbus->low_mmio_res->flags |= IORESOURCE_BUSY;
- release_mem_region(hbus->low_mmio_res->start,
- resource_size(hbus->low_mmio_res));
+ vmbus_free_mmio(hbus->low_mmio_res->start,
+ resource_size(hbus->low_mmio_res));
}
if (hbus->high_mmio_space && hbus->high_mmio_res) {
hbus->high_mmio_res->flags |= IORESOURCE_BUSY;
- release_mem_region(hbus->high_mmio_res->start,
- resource_size(hbus->high_mmio_res));
+ vmbus_free_mmio(hbus->high_mmio_res->start,
+ resource_size(hbus->high_mmio_res));
}
}
@@ -1880,8 +1894,8 @@
release_low_mmio:
if (hbus->low_mmio_res) {
- release_mem_region(hbus->low_mmio_res->start,
- resource_size(hbus->low_mmio_res));
+ vmbus_free_mmio(hbus->low_mmio_res->start,
+ resource_size(hbus->low_mmio_res));
}
return ret;
@@ -1924,7 +1938,7 @@
static void hv_free_config_window(struct hv_pcibus_device *hbus)
{
- release_mem_region(hbus->mem_config->start, PCI_CONFIG_MMIO_LENGTH);
+ vmbus_free_mmio(hbus->mem_config->start, PCI_CONFIG_MMIO_LENGTH);
}
/**
@@ -2268,11 +2282,6 @@
hbus = hv_get_drvdata(hdev);
- ret = hv_send_resources_released(hdev);
- if (ret)
- dev_err(&hdev->device,
- "Couldn't send resources released packet(s)\n");
-
memset(&pkt.teardown_packet, 0, sizeof(pkt.teardown_packet));
init_completion(&comp_pkt.host_event);
pkt.teardown_packet.completion_func = hv_pci_generic_compl;
@@ -2295,6 +2304,11 @@
pci_unlock_rescan_remove();
}
+ ret = hv_send_resources_released(hdev);
+ if (ret)
+ dev_err(&hdev->device,
+ "Couldn't send resources released packet(s)\n");
+
vmbus_close(hdev->channel);
/* Delete any children which might still exist. */
diff --git a/drivers/pci/host/pci-imx6.c b/drivers/pci/host/pci-imx6.c
index 2f817fa..b741a36 100644
--- a/drivers/pci/host/pci-imx6.c
+++ b/drivers/pci/host/pci-imx6.c
@@ -19,6 +19,7 @@
#include <linux/mfd/syscon/imx6q-iomuxc-gpr.h>
#include <linux/module.h>
#include <linux/of_gpio.h>
+#include <linux/of_device.h>
#include <linux/pci.h>
#include <linux/platform_device.h>
#include <linux/regmap.h>
@@ -31,19 +32,29 @@
#define to_imx6_pcie(x) container_of(x, struct imx6_pcie, pp)
+enum imx6_pcie_variants {
+ IMX6Q,
+ IMX6SX,
+ IMX6QP,
+};
+
struct imx6_pcie {
int reset_gpio;
+ bool gpio_active_high;
struct clk *pcie_bus;
struct clk *pcie_phy;
+ struct clk *pcie_inbound_axi;
struct clk *pcie;
struct pcie_port pp;
struct regmap *iomuxc_gpr;
+ enum imx6_pcie_variants variant;
void __iomem *mem_base;
u32 tx_deemph_gen1;
u32 tx_deemph_gen2_3p5db;
u32 tx_deemph_gen2_6db;
u32 tx_swing_full;
u32 tx_swing_low;
+ int link_gen;
};
/* PCIe Root Complex registers (memory-mapped) */
@@ -236,39 +247,95 @@
struct imx6_pcie *imx6_pcie = to_imx6_pcie(pp);
u32 val, gpr1, gpr12;
- /*
- * If the bootloader already enabled the link we need some special
- * handling to get the core back into a state where it is safe to
- * touch it for configuration. As there is no dedicated reset signal
- * wired up for MX6QDL, we need to manually force LTSSM into "detect"
- * state before completely disabling LTSSM, which is a prerequisite
- * for core configuration.
- *
- * If both LTSSM_ENABLE and REF_SSP_ENABLE are active we have a strong
- * indication that the bootloader activated the link.
- */
- regmap_read(imx6_pcie->iomuxc_gpr, IOMUXC_GPR1, &gpr1);
- regmap_read(imx6_pcie->iomuxc_gpr, IOMUXC_GPR12, &gpr12);
-
- if ((gpr1 & IMX6Q_GPR1_PCIE_REF_CLK_EN) &&
- (gpr12 & IMX6Q_GPR12_PCIE_CTL_2)) {
- val = readl(pp->dbi_base + PCIE_PL_PFLR);
- val &= ~PCIE_PL_PFLR_LINK_STATE_MASK;
- val |= PCIE_PL_PFLR_FORCE_LINK;
- writel(val, pp->dbi_base + PCIE_PL_PFLR);
-
+ switch (imx6_pcie->variant) {
+ case IMX6SX:
regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR12,
- IMX6Q_GPR12_PCIE_CTL_2, 0 << 10);
+ IMX6SX_GPR12_PCIE_TEST_POWERDOWN,
+ IMX6SX_GPR12_PCIE_TEST_POWERDOWN);
+ /* Force PCIe PHY reset */
+ regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR5,
+ IMX6SX_GPR5_PCIE_BTNRST_RESET,
+ IMX6SX_GPR5_PCIE_BTNRST_RESET);
+ break;
+ case IMX6QP:
+ regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR1,
+ IMX6Q_GPR1_PCIE_SW_RST,
+ IMX6Q_GPR1_PCIE_SW_RST);
+ break;
+ case IMX6Q:
+ /*
+ * If the bootloader already enabled the link we need some
+ * special handling to get the core back into a state where
+ * it is safe to touch it for configuration. As there is
+ * no dedicated reset signal wired up for MX6QDL, we need
+ * to manually force LTSSM into "detect" state before
+ * completely disabling LTSSM, which is a prerequisite for
+ * core configuration.
+ *
+ * If both LTSSM_ENABLE and REF_SSP_ENABLE are active we
+ * have a strong indication that the bootloader activated
+ * the link.
+ */
+ regmap_read(imx6_pcie->iomuxc_gpr, IOMUXC_GPR1, &gpr1);
+ regmap_read(imx6_pcie->iomuxc_gpr, IOMUXC_GPR12, &gpr12);
+
+ if ((gpr1 & IMX6Q_GPR1_PCIE_REF_CLK_EN) &&
+ (gpr12 & IMX6Q_GPR12_PCIE_CTL_2)) {
+ val = readl(pp->dbi_base + PCIE_PL_PFLR);
+ val &= ~PCIE_PL_PFLR_LINK_STATE_MASK;
+ val |= PCIE_PL_PFLR_FORCE_LINK;
+ writel(val, pp->dbi_base + PCIE_PL_PFLR);
+
+ regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR12,
+ IMX6Q_GPR12_PCIE_CTL_2, 0 << 10);
+ }
+
+ regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR1,
+ IMX6Q_GPR1_PCIE_TEST_PD, 1 << 18);
+ regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR1,
+ IMX6Q_GPR1_PCIE_REF_CLK_EN, 0 << 16);
+ break;
}
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR1,
- IMX6Q_GPR1_PCIE_TEST_PD, 1 << 18);
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR1,
- IMX6Q_GPR1_PCIE_REF_CLK_EN, 0 << 16);
-
return 0;
}
+static int imx6_pcie_enable_ref_clk(struct imx6_pcie *imx6_pcie)
+{
+ struct pcie_port *pp = &imx6_pcie->pp;
+ int ret = 0;
+
+ switch (imx6_pcie->variant) {
+ case IMX6SX:
+ ret = clk_prepare_enable(imx6_pcie->pcie_inbound_axi);
+ if (ret) {
+ dev_err(pp->dev, "unable to enable pcie_axi clock\n");
+ break;
+ }
+
+ regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR12,
+ IMX6SX_GPR12_PCIE_TEST_POWERDOWN, 0);
+ break;
+ case IMX6QP: /* FALLTHROUGH */
+ case IMX6Q:
+ /* power up core phy and enable ref clock */
+ regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR1,
+ IMX6Q_GPR1_PCIE_TEST_PD, 0 << 18);
+ /*
+ * the async reset input need ref clock to sync internally,
+ * when the ref clock comes after reset, internal synced
+ * reset time is too short, cannot meet the requirement.
+ * add one ~10us delay here.
+ */
+ udelay(10);
+ regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR1,
+ IMX6Q_GPR1_PCIE_REF_CLK_EN, 1 << 16);
+ break;
+ }
+
+ return ret;
+}
+
static int imx6_pcie_deassert_core_reset(struct pcie_port *pp)
{
struct imx6_pcie *imx6_pcie = to_imx6_pcie(pp);
@@ -292,43 +359,60 @@
goto err_pcie;
}
- /* power up core phy and enable ref clock */
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR1,
- IMX6Q_GPR1_PCIE_TEST_PD, 0 << 18);
- /*
- * the async reset input need ref clock to sync internally,
- * when the ref clock comes after reset, internal synced
- * reset time is too short, cannot meet the requirement.
- * add one ~10us delay here.
- */
- udelay(10);
- regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR1,
- IMX6Q_GPR1_PCIE_REF_CLK_EN, 1 << 16);
+ ret = imx6_pcie_enable_ref_clk(imx6_pcie);
+ if (ret) {
+ dev_err(pp->dev, "unable to enable pcie ref clock\n");
+ goto err_ref_clk;
+ }
/* allow the clocks to stabilize */
usleep_range(200, 500);
/* Some boards don't have PCIe reset GPIO. */
if (gpio_is_valid(imx6_pcie->reset_gpio)) {
- gpio_set_value_cansleep(imx6_pcie->reset_gpio, 0);
+ gpio_set_value_cansleep(imx6_pcie->reset_gpio,
+ imx6_pcie->gpio_active_high);
msleep(100);
- gpio_set_value_cansleep(imx6_pcie->reset_gpio, 1);
+ gpio_set_value_cansleep(imx6_pcie->reset_gpio,
+ !imx6_pcie->gpio_active_high);
}
+
+ switch (imx6_pcie->variant) {
+ case IMX6SX:
+ regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR5,
+ IMX6SX_GPR5_PCIE_BTNRST_RESET, 0);
+ break;
+ case IMX6QP:
+ regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR1,
+ IMX6Q_GPR1_PCIE_SW_RST, 0);
+
+ usleep_range(200, 500);
+ break;
+ case IMX6Q: /* Nothing to do */
+ break;
+ }
+
return 0;
+err_ref_clk:
+ clk_disable_unprepare(imx6_pcie->pcie);
err_pcie:
clk_disable_unprepare(imx6_pcie->pcie_bus);
err_pcie_bus:
clk_disable_unprepare(imx6_pcie->pcie_phy);
err_pcie_phy:
return ret;
-
}
static void imx6_pcie_init_phy(struct pcie_port *pp)
{
struct imx6_pcie *imx6_pcie = to_imx6_pcie(pp);
+ if (imx6_pcie->variant == IMX6SX)
+ regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR12,
+ IMX6SX_GPR12_PCIE_RX_EQ_MASK,
+ IMX6SX_GPR12_PCIE_RX_EQ_2);
+
regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR12,
IMX6Q_GPR12_PCIE_CTL_2, 0 << 10);
@@ -417,11 +501,15 @@
goto err_reset_phy;
}
- /* Allow Gen2 mode after the link is up. */
- tmp = readl(pp->dbi_base + PCIE_RC_LCR);
- tmp &= ~PCIE_RC_LCR_MAX_LINK_SPEEDS_MASK;
- tmp |= PCIE_RC_LCR_MAX_LINK_SPEEDS_GEN2;
- writel(tmp, pp->dbi_base + PCIE_RC_LCR);
+ if (imx6_pcie->link_gen == 2) {
+ /* Allow Gen2 mode after the link is up. */
+ tmp = readl(pp->dbi_base + PCIE_RC_LCR);
+ tmp &= ~PCIE_RC_LCR_MAX_LINK_SPEEDS_MASK;
+ tmp |= PCIE_RC_LCR_MAX_LINK_SPEEDS_GEN2;
+ writel(tmp, pp->dbi_base + PCIE_RC_LCR);
+ } else {
+ dev_info(pp->dev, "Link: Gen2 disabled\n");
+ }
/*
* Start Directed Speed Change so the best possible speed both link
@@ -445,8 +533,7 @@
}
tmp = readl(pp->dbi_base + PCIE_RC_LCSR);
- dev_dbg(pp->dev, "Link up, Gen=%i\n", (tmp >> 16) & 0xf);
-
+ dev_info(pp->dev, "Link up, Gen%i\n", (tmp >> 16) & 0xf);
return 0;
err_reset_phy:
@@ -535,6 +622,9 @@
pp = &imx6_pcie->pp;
pp->dev = &pdev->dev;
+ imx6_pcie->variant =
+ (enum imx6_pcie_variants)of_device_get_match_data(&pdev->dev);
+
/* Added for PCI abort handling */
hook_fault_code(16 + 6, imx6q_pcie_abort_handler, SIGBUS, 0,
"imprecise external abort");
@@ -546,9 +636,14 @@
/* Fetch GPIOs */
imx6_pcie->reset_gpio = of_get_named_gpio(np, "reset-gpio", 0);
+ imx6_pcie->gpio_active_high = of_property_read_bool(np,
+ "reset-gpio-active-high");
if (gpio_is_valid(imx6_pcie->reset_gpio)) {
ret = devm_gpio_request_one(&pdev->dev, imx6_pcie->reset_gpio,
- GPIOF_OUT_INIT_LOW, "PCIe reset");
+ imx6_pcie->gpio_active_high ?
+ GPIOF_OUT_INIT_HIGH :
+ GPIOF_OUT_INIT_LOW,
+ "PCIe reset");
if (ret) {
dev_err(&pdev->dev, "unable to get reset gpio\n");
return ret;
@@ -577,6 +672,16 @@
return PTR_ERR(imx6_pcie->pcie);
}
+ if (imx6_pcie->variant == IMX6SX) {
+ imx6_pcie->pcie_inbound_axi = devm_clk_get(&pdev->dev,
+ "pcie_inbound_axi");
+ if (IS_ERR(imx6_pcie->pcie_inbound_axi)) {
+ dev_err(&pdev->dev,
+ "pcie_incbound_axi clock missing or invalid\n");
+ return PTR_ERR(imx6_pcie->pcie_inbound_axi);
+ }
+ }
+
/* Grab GPR config register range */
imx6_pcie->iomuxc_gpr =
syscon_regmap_lookup_by_compatible("fsl,imx6q-iomuxc-gpr");
@@ -606,6 +711,12 @@
&imx6_pcie->tx_swing_low))
imx6_pcie->tx_swing_low = 127;
+ /* Limit link speed */
+ ret = of_property_read_u32(pp->dev->of_node, "fsl,max-link-speed",
+ &imx6_pcie->link_gen);
+ if (ret)
+ imx6_pcie->link_gen = 1;
+
ret = imx6_add_pcie_port(pp, pdev);
if (ret < 0)
return ret;
@@ -623,7 +734,9 @@
}
static const struct of_device_id imx6_pcie_of_match[] = {
- { .compatible = "fsl,imx6q-pcie", },
+ { .compatible = "fsl,imx6q-pcie", .data = (void *)IMX6Q, },
+ { .compatible = "fsl,imx6sx-pcie", .data = (void *)IMX6SX, },
+ { .compatible = "fsl,imx6qp-pcie", .data = (void *)IMX6QP, },
{},
};
MODULE_DEVICE_TABLE(of, imx6_pcie_of_match);
diff --git a/drivers/pci/host/pci-keystone-dw.c b/drivers/pci/host/pci-keystone-dw.c
index 6153853..4151509 100644
--- a/drivers/pci/host/pci-keystone-dw.c
+++ b/drivers/pci/host/pci-keystone-dw.c
@@ -14,6 +14,7 @@
#include <linux/irq.h>
#include <linux/irqdomain.h>
+#include <linux/irqreturn.h>
#include <linux/module.h>
#include <linux/of.h>
#include <linux/of_pci.h>
@@ -53,6 +54,21 @@
#define IRQ_STATUS 0x184
#define MSI_IRQ_OFFSET 4
+/* Error IRQ bits */
+#define ERR_AER BIT(5) /* ECRC error */
+#define ERR_AXI BIT(4) /* AXI tag lookup fatal error */
+#define ERR_CORR BIT(3) /* Correctable error */
+#define ERR_NONFATAL BIT(2) /* Non-fatal error */
+#define ERR_FATAL BIT(1) /* Fatal error */
+#define ERR_SYS BIT(0) /* System (fatal, non-fatal, or correctable) */
+#define ERR_IRQ_ALL (ERR_AER | ERR_AXI | ERR_CORR | \
+ ERR_NONFATAL | ERR_FATAL | ERR_SYS)
+#define ERR_FATAL_IRQ (ERR_FATAL | ERR_AXI)
+#define ERR_IRQ_STATUS_RAW 0x1c0
+#define ERR_IRQ_STATUS 0x1c4
+#define ERR_IRQ_ENABLE_SET 0x1c8
+#define ERR_IRQ_ENABLE_CLR 0x1cc
+
/* Config space registers */
#define DEBUG0 0x728
@@ -243,6 +259,28 @@
writel(offset, ks_pcie->va_app_base + IRQ_EOI);
}
+void ks_dw_pcie_enable_error_irq(void __iomem *reg_base)
+{
+ writel(ERR_IRQ_ALL, reg_base + ERR_IRQ_ENABLE_SET);
+}
+
+irqreturn_t ks_dw_pcie_handle_error_irq(struct device *dev,
+ void __iomem *reg_base)
+{
+ u32 status;
+
+ status = readl(reg_base + ERR_IRQ_STATUS_RAW) & ERR_IRQ_ALL;
+ if (!status)
+ return IRQ_NONE;
+
+ if (status & ERR_FATAL_IRQ)
+ dev_err(dev, "fatal error (status %#010x)\n", status);
+
+ /* Ack the IRQ; status bits are RW1C */
+ writel(status, reg_base + ERR_IRQ_STATUS);
+ return IRQ_HANDLED;
+}
+
static void ks_dw_pcie_ack_legacy_irq(struct irq_data *d)
{
}
diff --git a/drivers/pci/host/pci-keystone.c b/drivers/pci/host/pci-keystone.c
index b71f55b..6b8301e 100644
--- a/drivers/pci/host/pci-keystone.c
+++ b/drivers/pci/host/pci-keystone.c
@@ -15,6 +15,7 @@
#include <linux/irqchip/chained_irq.h>
#include <linux/clk.h>
#include <linux/delay.h>
+#include <linux/interrupt.h>
#include <linux/irqdomain.h>
#include <linux/module.h>
#include <linux/msi.h>
@@ -159,7 +160,7 @@
static int ks_pcie_get_irq_controller_info(struct keystone_pcie *ks_pcie,
char *controller, int *num_irqs)
{
- int temp, max_host_irqs, legacy = 1, *host_irqs, ret = -EINVAL;
+ int temp, max_host_irqs, legacy = 1, *host_irqs;
struct device *dev = ks_pcie->pp.dev;
struct device_node *np_pcie = dev->of_node, **np_temp;
@@ -180,11 +181,15 @@
*np_temp = of_find_node_by_name(np_pcie, controller);
if (!(*np_temp)) {
dev_err(dev, "Node for %s is absent\n", controller);
- goto out;
+ return -EINVAL;
}
+
temp = of_irq_count(*np_temp);
- if (!temp)
- goto out;
+ if (!temp) {
+ dev_err(dev, "No IRQ entries in %s\n", controller);
+ return -EINVAL;
+ }
+
if (temp > max_host_irqs)
dev_warn(dev, "Too many %s interrupts defined %u\n",
(legacy ? "legacy" : "MSI"), temp);
@@ -198,12 +203,13 @@
if (!host_irqs[temp])
break;
}
+
if (temp) {
*num_irqs = temp;
- ret = 0;
+ return 0;
}
-out:
- return ret;
+
+ return -EINVAL;
}
static void ks_pcie_setup_interrupts(struct keystone_pcie *ks_pcie)
@@ -226,6 +232,9 @@
ks_pcie);
}
}
+
+ if (ks_pcie->error_irq > 0)
+ ks_dw_pcie_enable_error_irq(ks_pcie->va_app_base);
}
/*
@@ -289,6 +298,14 @@
.scan_bus = ks_dw_pcie_v3_65_scan_bus,
};
+static irqreturn_t pcie_err_irq_handler(int irq, void *priv)
+{
+ struct keystone_pcie *ks_pcie = priv;
+
+ return ks_dw_pcie_handle_error_irq(ks_pcie->pp.dev,
+ ks_pcie->va_app_base);
+}
+
static int __init ks_add_pcie_port(struct keystone_pcie *ks_pcie,
struct platform_device *pdev)
{
@@ -309,6 +326,22 @@
return ret;
}
+ /*
+ * Index 0 is the platform interrupt for error interrupt
+ * from RC. This is optional.
+ */
+ ks_pcie->error_irq = irq_of_parse_and_map(ks_pcie->np, 0);
+ if (ks_pcie->error_irq <= 0)
+ dev_info(&pdev->dev, "no error IRQ defined\n");
+ else {
+ if (request_irq(ks_pcie->error_irq, pcie_err_irq_handler,
+ IRQF_SHARED, "pcie-error-irq", ks_pcie) < 0) {
+ dev_err(&pdev->dev, "failed to request error IRQ %d\n",
+ ks_pcie->error_irq);
+ return ret;
+ }
+ }
+
pp->root_bus_nr = -1;
pp->ops = &keystone_pcie_host_ops;
ret = ks_dw_pcie_host_init(ks_pcie, ks_pcie->msi_intc_np);
@@ -317,7 +350,7 @@
return ret;
}
- return ret;
+ return 0;
}
static const struct of_device_id ks_pcie_of_match[] = {
@@ -346,7 +379,7 @@
struct resource *res;
void __iomem *reg_p;
struct phy *phy;
- int ret = 0;
+ int ret;
ks_pcie = devm_kzalloc(&pdev->dev, sizeof(*ks_pcie),
GFP_KERNEL);
@@ -376,6 +409,7 @@
devm_release_mem_region(dev, res->start, resource_size(res));
pp->dev = dev;
+ ks_pcie->np = dev->of_node;
platform_set_drvdata(pdev, ks_pcie);
ks_pcie->clk = devm_clk_get(dev, "pcie");
if (IS_ERR(ks_pcie->clk)) {
diff --git a/drivers/pci/host/pci-keystone.h b/drivers/pci/host/pci-keystone.h
index f0944e8..a5b0cb2 100644
--- a/drivers/pci/host/pci-keystone.h
+++ b/drivers/pci/host/pci-keystone.h
@@ -29,6 +29,9 @@
int msi_host_irqs[MAX_MSI_HOST_IRQS];
struct device_node *msi_intc_np;
struct irq_domain *legacy_irq_domain;
+ struct device_node *np;
+
+ int error_irq;
/* Application register space */
void __iomem *va_app_base;
@@ -42,6 +45,9 @@
/* Keystone specific PCI controller APIs */
void ks_dw_pcie_enable_legacy_irqs(struct keystone_pcie *ks_pcie);
void ks_dw_pcie_handle_legacy_irq(struct keystone_pcie *ks_pcie, int offset);
+void ks_dw_pcie_enable_error_irq(void __iomem *reg_base);
+irqreturn_t ks_dw_pcie_handle_error_irq(struct device *dev,
+ void __iomem *reg_base);
int ks_dw_pcie_host_init(struct keystone_pcie *ks_pcie,
struct device_node *msi_intc_np);
int ks_dw_pcie_wr_other_conf(struct pcie_port *pp, struct pci_bus *bus,
diff --git a/drivers/pci/host/pci-mvebu.c b/drivers/pci/host/pci-mvebu.c
index 53b79c5..6b451df 100644
--- a/drivers/pci/host/pci-mvebu.c
+++ b/drivers/pci/host/pci-mvebu.c
@@ -1003,6 +1003,7 @@
pcie->msi->dev = &pcie->pdev->dev;
}
+#ifdef CONFIG_PM_SLEEP
static int mvebu_pcie_suspend(struct device *dev)
{
struct mvebu_pcie *pcie;
@@ -1031,6 +1032,7 @@
return 0;
}
+#endif
static void mvebu_pcie_port_clk_put(void *data)
{
@@ -1298,9 +1300,8 @@
};
MODULE_DEVICE_TABLE(of, mvebu_pcie_of_match_table);
-static struct dev_pm_ops mvebu_pcie_pm_ops = {
- .suspend_noirq = mvebu_pcie_suspend,
- .resume_noirq = mvebu_pcie_resume,
+static const struct dev_pm_ops mvebu_pcie_pm_ops = {
+ SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(mvebu_pcie_suspend, mvebu_pcie_resume)
};
static struct platform_driver mvebu_pcie_driver = {
diff --git a/drivers/pci/host/pci-thunder-ecam.c b/drivers/pci/host/pci-thunder-ecam.c
index d71935cb..540d030 100644
--- a/drivers/pci/host/pci-thunder-ecam.c
+++ b/drivers/pci/host/pci-thunder-ecam.c
@@ -13,18 +13,7 @@
#include <linux/of.h>
#include <linux/platform_device.h>
-#include "pci-host-common.h"
-
-/* Mapping is standard ECAM */
-static void __iomem *thunder_ecam_map_bus(struct pci_bus *bus,
- unsigned int devfn,
- int where)
-{
- struct gen_pci *pci = bus->sysdata;
- resource_size_t idx = bus->number - pci->cfg.bus_range->start;
-
- return pci->cfg.win[idx] + ((devfn << 12) | where);
-}
+#include "../ecam.h"
static void set_val(u32 v, int where, int size, u32 *val)
{
@@ -99,7 +88,7 @@
static int thunder_ecam_p2_config_read(struct pci_bus *bus, unsigned int devfn,
int where, int size, u32 *val)
{
- struct gen_pci *pci = bus->sysdata;
+ struct pci_config_window *cfg = bus->sysdata;
int where_a = where & ~3;
void __iomem *addr;
u32 node_bits;
@@ -129,7 +118,7 @@
* the config space access window. Since we are working with
* the high-order 32 bits, shift everything down by 32 bits.
*/
- node_bits = (pci->cfg.res.start >> 32) & (1 << 12);
+ node_bits = (cfg->res.start >> 32) & (1 << 12);
v |= node_bits;
set_val(v, where, size, val);
@@ -358,36 +347,24 @@
return pci_generic_config_write(bus, devfn, where, size, val);
}
-static struct gen_pci_cfg_bus_ops thunder_ecam_bus_ops = {
+static struct pci_ecam_ops pci_thunder_ecam_ops = {
.bus_shift = 20,
- .ops = {
- .map_bus = thunder_ecam_map_bus,
+ .pci_ops = {
+ .map_bus = pci_ecam_map_bus,
.read = thunder_ecam_config_read,
.write = thunder_ecam_config_write,
}
};
static const struct of_device_id thunder_ecam_of_match[] = {
- { .compatible = "cavium,pci-host-thunder-ecam",
- .data = &thunder_ecam_bus_ops },
-
+ { .compatible = "cavium,pci-host-thunder-ecam" },
{ },
};
MODULE_DEVICE_TABLE(of, thunder_ecam_of_match);
static int thunder_ecam_probe(struct platform_device *pdev)
{
- struct device *dev = &pdev->dev;
- const struct of_device_id *of_id;
- struct gen_pci *pci = devm_kzalloc(dev, sizeof(*pci), GFP_KERNEL);
-
- if (!pci)
- return -ENOMEM;
-
- of_id = of_match_node(thunder_ecam_of_match, dev->of_node);
- pci->cfg.ops = (struct gen_pci_cfg_bus_ops *)of_id->data;
-
- return pci_host_common_probe(pdev, pci);
+ return pci_host_common_probe(pdev, &pci_thunder_ecam_ops);
}
static struct platform_driver thunder_ecam_driver = {
diff --git a/drivers/pci/host/pci-thunder-pem.c b/drivers/pci/host/pci-thunder-pem.c
index cabb92a..9b8ab94 100644
--- a/drivers/pci/host/pci-thunder-pem.c
+++ b/drivers/pci/host/pci-thunder-pem.c
@@ -20,34 +20,22 @@
#include <linux/of_pci.h>
#include <linux/platform_device.h>
-#include "pci-host-common.h"
+#include "../ecam.h"
#define PEM_CFG_WR 0x28
#define PEM_CFG_RD 0x30
struct thunder_pem_pci {
- struct gen_pci gen_pci;
u32 ea_entry[3];
void __iomem *pem_reg_base;
};
-static void __iomem *thunder_pem_map_bus(struct pci_bus *bus,
- unsigned int devfn, int where)
-{
- struct gen_pci *pci = bus->sysdata;
- resource_size_t idx = bus->number - pci->cfg.bus_range->start;
-
- return pci->cfg.win[idx] + ((devfn << 16) | where);
-}
-
static int thunder_pem_bridge_read(struct pci_bus *bus, unsigned int devfn,
int where, int size, u32 *val)
{
u64 read_val;
- struct thunder_pem_pci *pem_pci;
- struct gen_pci *pci = bus->sysdata;
-
- pem_pci = container_of(pci, struct thunder_pem_pci, gen_pci);
+ struct pci_config_window *cfg = bus->sysdata;
+ struct thunder_pem_pci *pem_pci = (struct thunder_pem_pci *)cfg->priv;
if (devfn != 0 || where >= 2048) {
*val = ~0;
@@ -132,17 +120,17 @@
static int thunder_pem_config_read(struct pci_bus *bus, unsigned int devfn,
int where, int size, u32 *val)
{
- struct gen_pci *pci = bus->sysdata;
+ struct pci_config_window *cfg = bus->sysdata;
- if (bus->number < pci->cfg.bus_range->start ||
- bus->number > pci->cfg.bus_range->end)
+ if (bus->number < cfg->busr.start ||
+ bus->number > cfg->busr.end)
return PCIBIOS_DEVICE_NOT_FOUND;
/*
* The first device on the bus is the PEM PCIe bridge.
* Special case its config access.
*/
- if (bus->number == pci->cfg.bus_range->start)
+ if (bus->number == cfg->busr.start)
return thunder_pem_bridge_read(bus, devfn, where, size, val);
return pci_generic_config_read(bus, devfn, where, size, val);
@@ -153,11 +141,11 @@
* reserved bits, this makes the code simpler and is OK as the bits
* are not affected by writing zeros to them.
*/
-static u32 thunder_pem_bridge_w1c_bits(int where)
+static u32 thunder_pem_bridge_w1c_bits(u64 where_aligned)
{
u32 w1c_bits = 0;
- switch (where & ~3) {
+ switch (where_aligned) {
case 0x04: /* Command/Status */
case 0x1c: /* Base and I/O Limit/Secondary Status */
w1c_bits = 0xff000000;
@@ -184,15 +172,36 @@
return w1c_bits;
}
+/* Some bits must be written to one so they appear to be read-only. */
+static u32 thunder_pem_bridge_w1_bits(u64 where_aligned)
+{
+ u32 w1_bits;
+
+ switch (where_aligned) {
+ case 0x1c: /* I/O Base / I/O Limit, Secondary Status */
+ /* Force 32-bit I/O addressing. */
+ w1_bits = 0x0101;
+ break;
+ case 0x24: /* Prefetchable Memory Base / Prefetchable Memory Limit */
+ /* Force 64-bit addressing */
+ w1_bits = 0x00010001;
+ break;
+ default:
+ w1_bits = 0;
+ break;
+ }
+ return w1_bits;
+}
+
static int thunder_pem_bridge_write(struct pci_bus *bus, unsigned int devfn,
int where, int size, u32 val)
{
- struct gen_pci *pci = bus->sysdata;
- struct thunder_pem_pci *pem_pci;
+ struct pci_config_window *cfg = bus->sysdata;
+ struct thunder_pem_pci *pem_pci = (struct thunder_pem_pci *)cfg->priv;
u64 write_val, read_val;
+ u64 where_aligned = where & ~3ull;
u32 mask = 0;
- pem_pci = container_of(pci, struct thunder_pem_pci, gen_pci);
if (devfn != 0 || where >= 2048)
return PCIBIOS_DEVICE_NOT_FOUND;
@@ -205,8 +214,7 @@
*/
switch (size) {
case 1:
- read_val = where & ~3ull;
- writeq(read_val, pem_pci->pem_reg_base + PEM_CFG_RD);
+ writeq(where_aligned, pem_pci->pem_reg_base + PEM_CFG_RD);
read_val = readq(pem_pci->pem_reg_base + PEM_CFG_RD);
read_val >>= 32;
mask = ~(0xff << (8 * (where & 3)));
@@ -215,8 +223,7 @@
val |= (u32)read_val;
break;
case 2:
- read_val = where & ~3ull;
- writeq(read_val, pem_pci->pem_reg_base + PEM_CFG_RD);
+ writeq(where_aligned, pem_pci->pem_reg_base + PEM_CFG_RD);
read_val = readq(pem_pci->pem_reg_base + PEM_CFG_RD);
read_val >>= 32;
mask = ~(0xffff << (8 * (where & 3)));
@@ -244,11 +251,17 @@
}
/*
+ * Some bits must be read-only with value of one. Since the
+ * access method allows these to be cleared if a zero is
+ * written, force them to one before writing.
+ */
+ val |= thunder_pem_bridge_w1_bits(where_aligned);
+
+ /*
* Low order bits are the config address, the high order 32
* bits are the data to be written.
*/
- write_val = where & ~3ull;
- write_val |= (((u64)val) << 32);
+ write_val = (((u64)val) << 32) | where_aligned;
writeq(write_val, pem_pci->pem_reg_base + PEM_CFG_WR);
return PCIBIOS_SUCCESSFUL;
}
@@ -256,53 +269,38 @@
static int thunder_pem_config_write(struct pci_bus *bus, unsigned int devfn,
int where, int size, u32 val)
{
- struct gen_pci *pci = bus->sysdata;
+ struct pci_config_window *cfg = bus->sysdata;
- if (bus->number < pci->cfg.bus_range->start ||
- bus->number > pci->cfg.bus_range->end)
+ if (bus->number < cfg->busr.start ||
+ bus->number > cfg->busr.end)
return PCIBIOS_DEVICE_NOT_FOUND;
/*
* The first device on the bus is the PEM PCIe bridge.
* Special case its config access.
*/
- if (bus->number == pci->cfg.bus_range->start)
+ if (bus->number == cfg->busr.start)
return thunder_pem_bridge_write(bus, devfn, where, size, val);
return pci_generic_config_write(bus, devfn, where, size, val);
}
-static struct gen_pci_cfg_bus_ops thunder_pem_bus_ops = {
- .bus_shift = 24,
- .ops = {
- .map_bus = thunder_pem_map_bus,
- .read = thunder_pem_config_read,
- .write = thunder_pem_config_write,
- }
-};
-
-static const struct of_device_id thunder_pem_of_match[] = {
- { .compatible = "cavium,pci-host-thunder-pem",
- .data = &thunder_pem_bus_ops },
-
- { },
-};
-MODULE_DEVICE_TABLE(of, thunder_pem_of_match);
-
-static int thunder_pem_probe(struct platform_device *pdev)
+static int thunder_pem_init(struct device *dev, struct pci_config_window *cfg)
{
- struct device *dev = &pdev->dev;
- const struct of_device_id *of_id;
resource_size_t bar4_start;
struct resource *res_pem;
struct thunder_pem_pci *pem_pci;
+ struct platform_device *pdev;
+
+ /* Only OF support for now */
+ if (!dev->of_node)
+ return -EINVAL;
pem_pci = devm_kzalloc(dev, sizeof(*pem_pci), GFP_KERNEL);
if (!pem_pci)
return -ENOMEM;
- of_id = of_match_node(thunder_pem_of_match, dev->of_node);
- pem_pci->gen_pci.cfg.ops = (struct gen_pci_cfg_bus_ops *)of_id->data;
+ pdev = to_platform_device(dev);
/*
* The second register range is the PEM bridge to the PCIe
@@ -330,7 +328,29 @@
pem_pci->ea_entry[1] = (u32)(res_pem->end - bar4_start) & ~3u;
pem_pci->ea_entry[2] = (u32)(bar4_start >> 32);
- return pci_host_common_probe(pdev, &pem_pci->gen_pci);
+ cfg->priv = pem_pci;
+ return 0;
+}
+
+static struct pci_ecam_ops pci_thunder_pem_ops = {
+ .bus_shift = 24,
+ .init = thunder_pem_init,
+ .pci_ops = {
+ .map_bus = pci_ecam_map_bus,
+ .read = thunder_pem_config_read,
+ .write = thunder_pem_config_write,
+ }
+};
+
+static const struct of_device_id thunder_pem_of_match[] = {
+ { .compatible = "cavium,pci-host-thunder-pem" },
+ { },
+};
+MODULE_DEVICE_TABLE(of, thunder_pem_of_match);
+
+static int thunder_pem_probe(struct platform_device *pdev)
+{
+ return pci_host_common_probe(pdev, &pci_thunder_pem_ops);
}
static struct platform_driver thunder_pem_driver = {
diff --git a/drivers/pci/host/pcie-armada8k.c b/drivers/pci/host/pcie-armada8k.c
new file mode 100644
index 0000000..5572356
--- /dev/null
+++ b/drivers/pci/host/pcie-armada8k.c
@@ -0,0 +1,262 @@
+/*
+ * PCIe host controller driver for Marvell Armada-8K SoCs
+ *
+ * Armada-8K PCIe Glue Layer Source Code
+ *
+ * Copyright (C) 2016 Marvell Technology Group Ltd.
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2. This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/pci.h>
+#include <linux/phy/phy.h>
+#include <linux/platform_device.h>
+#include <linux/resource.h>
+#include <linux/of_pci.h>
+#include <linux/of_irq.h>
+
+#include "pcie-designware.h"
+
+struct armada8k_pcie {
+ void __iomem *base;
+ struct clk *clk;
+ struct pcie_port pp;
+};
+
+#define PCIE_VENDOR_REGS_OFFSET 0x8000
+
+#define PCIE_GLOBAL_CONTROL_REG 0x0
+#define PCIE_APP_LTSSM_EN BIT(2)
+#define PCIE_DEVICE_TYPE_SHIFT 4
+#define PCIE_DEVICE_TYPE_MASK 0xF
+#define PCIE_DEVICE_TYPE_RC 0x4 /* Root complex */
+
+#define PCIE_GLOBAL_STATUS_REG 0x8
+#define PCIE_GLB_STS_RDLH_LINK_UP BIT(1)
+#define PCIE_GLB_STS_PHY_LINK_UP BIT(9)
+
+#define PCIE_GLOBAL_INT_CAUSE1_REG 0x1C
+#define PCIE_GLOBAL_INT_MASK1_REG 0x20
+#define PCIE_INT_A_ASSERT_MASK BIT(9)
+#define PCIE_INT_B_ASSERT_MASK BIT(10)
+#define PCIE_INT_C_ASSERT_MASK BIT(11)
+#define PCIE_INT_D_ASSERT_MASK BIT(12)
+
+#define PCIE_ARCACHE_TRC_REG 0x50
+#define PCIE_AWCACHE_TRC_REG 0x54
+#define PCIE_ARUSER_REG 0x5C
+#define PCIE_AWUSER_REG 0x60
+/*
+ * AR/AW Cache defauls: Normal memory, Write-Back, Read / Write
+ * allocate
+ */
+#define ARCACHE_DEFAULT_VALUE 0x3511
+#define AWCACHE_DEFAULT_VALUE 0x5311
+
+#define DOMAIN_OUTER_SHAREABLE 0x2
+#define AX_USER_DOMAIN_MASK 0x3
+#define AX_USER_DOMAIN_SHIFT 4
+
+#define to_armada8k_pcie(x) container_of(x, struct armada8k_pcie, pp)
+
+static int armada8k_pcie_link_up(struct pcie_port *pp)
+{
+ struct armada8k_pcie *pcie = to_armada8k_pcie(pp);
+ u32 reg;
+ u32 mask = PCIE_GLB_STS_RDLH_LINK_UP | PCIE_GLB_STS_PHY_LINK_UP;
+
+ reg = readl(pcie->base + PCIE_GLOBAL_STATUS_REG);
+
+ if ((reg & mask) == mask)
+ return 1;
+
+ dev_dbg(pp->dev, "No link detected (Global-Status: 0x%08x).\n", reg);
+ return 0;
+}
+
+static void armada8k_pcie_establish_link(struct pcie_port *pp)
+{
+ struct armada8k_pcie *pcie = to_armada8k_pcie(pp);
+ void __iomem *base = pcie->base;
+ u32 reg;
+
+ if (!dw_pcie_link_up(pp)) {
+ /* Disable LTSSM state machine to enable configuration */
+ reg = readl(base + PCIE_GLOBAL_CONTROL_REG);
+ reg &= ~(PCIE_APP_LTSSM_EN);
+ writel(reg, base + PCIE_GLOBAL_CONTROL_REG);
+ }
+
+ /* Set the device to root complex mode */
+ reg = readl(base + PCIE_GLOBAL_CONTROL_REG);
+ reg &= ~(PCIE_DEVICE_TYPE_MASK << PCIE_DEVICE_TYPE_SHIFT);
+ reg |= PCIE_DEVICE_TYPE_RC << PCIE_DEVICE_TYPE_SHIFT;
+ writel(reg, base + PCIE_GLOBAL_CONTROL_REG);
+
+ /* Set the PCIe master AxCache attributes */
+ writel(ARCACHE_DEFAULT_VALUE, base + PCIE_ARCACHE_TRC_REG);
+ writel(AWCACHE_DEFAULT_VALUE, base + PCIE_AWCACHE_TRC_REG);
+
+ /* Set the PCIe master AxDomain attributes */
+ reg = readl(base + PCIE_ARUSER_REG);
+ reg &= ~(AX_USER_DOMAIN_MASK << AX_USER_DOMAIN_SHIFT);
+ reg |= DOMAIN_OUTER_SHAREABLE << AX_USER_DOMAIN_SHIFT;
+ writel(reg, base + PCIE_ARUSER_REG);
+
+ reg = readl(base + PCIE_AWUSER_REG);
+ reg &= ~(AX_USER_DOMAIN_MASK << AX_USER_DOMAIN_SHIFT);
+ reg |= DOMAIN_OUTER_SHAREABLE << AX_USER_DOMAIN_SHIFT;
+ writel(reg, base + PCIE_AWUSER_REG);
+
+ /* Enable INT A-D interrupts */
+ reg = readl(base + PCIE_GLOBAL_INT_MASK1_REG);
+ reg |= PCIE_INT_A_ASSERT_MASK | PCIE_INT_B_ASSERT_MASK |
+ PCIE_INT_C_ASSERT_MASK | PCIE_INT_D_ASSERT_MASK;
+ writel(reg, base + PCIE_GLOBAL_INT_MASK1_REG);
+
+ if (!dw_pcie_link_up(pp)) {
+ /* Configuration done. Start LTSSM */
+ reg = readl(base + PCIE_GLOBAL_CONTROL_REG);
+ reg |= PCIE_APP_LTSSM_EN;
+ writel(reg, base + PCIE_GLOBAL_CONTROL_REG);
+ }
+
+ /* Wait until the link becomes active again */
+ if (dw_pcie_wait_for_link(pp))
+ dev_err(pp->dev, "Link not up after reconfiguration\n");
+}
+
+static void armada8k_pcie_host_init(struct pcie_port *pp)
+{
+ dw_pcie_setup_rc(pp);
+ armada8k_pcie_establish_link(pp);
+}
+
+static irqreturn_t armada8k_pcie_irq_handler(int irq, void *arg)
+{
+ struct pcie_port *pp = arg;
+ struct armada8k_pcie *pcie = to_armada8k_pcie(pp);
+ void __iomem *base = pcie->base;
+ u32 val;
+
+ /*
+ * Interrupts are directly handled by the device driver of the
+ * PCI device. However, they are also latched into the PCIe
+ * controller, so we simply discard them.
+ */
+ val = readl(base + PCIE_GLOBAL_INT_CAUSE1_REG);
+ writel(val, base + PCIE_GLOBAL_INT_CAUSE1_REG);
+
+ return IRQ_HANDLED;
+}
+
+static struct pcie_host_ops armada8k_pcie_host_ops = {
+ .link_up = armada8k_pcie_link_up,
+ .host_init = armada8k_pcie_host_init,
+};
+
+static int armada8k_add_pcie_port(struct pcie_port *pp,
+ struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ int ret;
+
+ pp->root_bus_nr = -1;
+ pp->ops = &armada8k_pcie_host_ops;
+
+ pp->irq = platform_get_irq(pdev, 0);
+ if (!pp->irq) {
+ dev_err(dev, "failed to get irq for port\n");
+ return -ENODEV;
+ }
+
+ ret = devm_request_irq(dev, pp->irq, armada8k_pcie_irq_handler,
+ IRQF_SHARED, "armada8k-pcie", pp);
+ if (ret) {
+ dev_err(dev, "failed to request irq %d\n", pp->irq);
+ return ret;
+ }
+
+ ret = dw_pcie_host_init(pp);
+ if (ret) {
+ dev_err(dev, "failed to initialize host: %d\n", ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int armada8k_pcie_probe(struct platform_device *pdev)
+{
+ struct armada8k_pcie *pcie;
+ struct pcie_port *pp;
+ struct device *dev = &pdev->dev;
+ struct resource *base;
+ int ret;
+
+ pcie = devm_kzalloc(dev, sizeof(*pcie), GFP_KERNEL);
+ if (!pcie)
+ return -ENOMEM;
+
+ pcie->clk = devm_clk_get(dev, NULL);
+ if (IS_ERR(pcie->clk))
+ return PTR_ERR(pcie->clk);
+
+ clk_prepare_enable(pcie->clk);
+
+ pp = &pcie->pp;
+ pp->dev = dev;
+ platform_set_drvdata(pdev, pcie);
+
+ /* Get the dw-pcie unit configuration/control registers base. */
+ base = platform_get_resource_byname(pdev, IORESOURCE_MEM, "ctrl");
+ pp->dbi_base = devm_ioremap_resource(dev, base);
+ if (IS_ERR(pp->dbi_base)) {
+ dev_err(dev, "couldn't remap regs base %p\n", base);
+ ret = PTR_ERR(pp->dbi_base);
+ goto fail;
+ }
+
+ pcie->base = pp->dbi_base + PCIE_VENDOR_REGS_OFFSET;
+
+ ret = armada8k_add_pcie_port(pp, pdev);
+ if (ret)
+ goto fail;
+
+ return 0;
+
+fail:
+ if (!IS_ERR(pcie->clk))
+ clk_disable_unprepare(pcie->clk);
+
+ return ret;
+}
+
+static const struct of_device_id armada8k_pcie_of_match[] = {
+ { .compatible = "marvell,armada8k-pcie", },
+ {},
+};
+MODULE_DEVICE_TABLE(of, armada8k_pcie_of_match);
+
+static struct platform_driver armada8k_pcie_driver = {
+ .probe = armada8k_pcie_probe,
+ .driver = {
+ .name = "armada8k-pcie",
+ .of_match_table = of_match_ptr(armada8k_pcie_of_match),
+ },
+};
+
+module_platform_driver(armada8k_pcie_driver);
+
+MODULE_DESCRIPTION("Armada 8k PCIe host controller driver");
+MODULE_AUTHOR("Yehuda Yitshak <yehuday@marvell.com>");
+MODULE_AUTHOR("Shadi Ammouri <shadi@marvell.com>");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/pci/host/pcie-designware.c b/drivers/pci/host/pcie-designware.c
index a4cccd3..aafd766 100644
--- a/drivers/pci/host/pcie-designware.c
+++ b/drivers/pci/host/pcie-designware.c
@@ -434,7 +434,6 @@
struct platform_device *pdev = to_platform_device(pp->dev);
struct pci_bus *bus, *child;
struct resource *cfg_res;
- u32 val;
int i, ret;
LIST_HEAD(res);
struct resource_entry *win;
@@ -544,25 +543,6 @@
if (pp->ops->host_init)
pp->ops->host_init(pp);
- /*
- * If the platform provides ->rd_other_conf, it means the platform
- * uses its own address translation component rather than ATU, so
- * we should not program the ATU here.
- */
- if (!pp->ops->rd_other_conf)
- dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX1,
- PCIE_ATU_TYPE_MEM, pp->mem_base,
- pp->mem_bus_addr, pp->mem_size);
-
- dw_pcie_wr_own_conf(pp, PCI_BASE_ADDRESS_0, 4, 0);
-
- /* program correct class for RC */
- dw_pcie_wr_own_conf(pp, PCI_CLASS_DEVICE, 2, PCI_CLASS_BRIDGE_PCI);
-
- dw_pcie_rd_own_conf(pp, PCIE_LINK_WIDTH_SPEED_CONTROL, 4, &val);
- val |= PORT_LOGIC_SPEED_CHANGE;
- dw_pcie_wr_own_conf(pp, PCIE_LINK_WIDTH_SPEED_CONTROL, 4, val);
-
pp->root_bus_nr = pp->busn->start;
if (IS_ENABLED(CONFIG_PCI_MSI)) {
bus = pci_scan_root_bus_msi(pp->dev, pp->root_bus_nr,
@@ -728,8 +708,6 @@
void dw_pcie_setup_rc(struct pcie_port *pp)
{
u32 val;
- u32 membase;
- u32 memlimit;
/* set the number of lanes */
dw_pcie_readl_rc(pp, PCIE_PORT_LINK_CONTROL, &val);
@@ -788,18 +766,31 @@
val |= 0x00010100;
dw_pcie_writel_rc(pp, val, PCI_PRIMARY_BUS);
- /* setup memory base, memory limit */
- membase = ((u32)pp->mem_base & 0xfff00000) >> 16;
- memlimit = (pp->mem_size + (u32)pp->mem_base) & 0xfff00000;
- val = memlimit | membase;
- dw_pcie_writel_rc(pp, val, PCI_MEMORY_BASE);
-
/* setup command register */
dw_pcie_readl_rc(pp, PCI_COMMAND, &val);
val &= 0xffff0000;
val |= PCI_COMMAND_IO | PCI_COMMAND_MEMORY |
PCI_COMMAND_MASTER | PCI_COMMAND_SERR;
dw_pcie_writel_rc(pp, val, PCI_COMMAND);
+
+ /*
+ * If the platform provides ->rd_other_conf, it means the platform
+ * uses its own address translation component rather than ATU, so
+ * we should not program the ATU here.
+ */
+ if (!pp->ops->rd_other_conf)
+ dw_pcie_prog_outbound_atu(pp, PCIE_ATU_REGION_INDEX1,
+ PCIE_ATU_TYPE_MEM, pp->mem_base,
+ pp->mem_bus_addr, pp->mem_size);
+
+ dw_pcie_wr_own_conf(pp, PCI_BASE_ADDRESS_0, 4, 0);
+
+ /* program correct class for RC */
+ dw_pcie_wr_own_conf(pp, PCI_CLASS_DEVICE, 2, PCI_CLASS_BRIDGE_PCI);
+
+ dw_pcie_rd_own_conf(pp, PCIE_LINK_WIDTH_SPEED_CONTROL, 4, &val);
+ val |= PORT_LOGIC_SPEED_CHANGE;
+ dw_pcie_wr_own_conf(pp, PCIE_LINK_WIDTH_SPEED_CONTROL, 4, val);
}
MODULE_AUTHOR("Jingoo Han <jg1.han@samsung.com>");
diff --git a/drivers/pci/host/pcie-xilinx-nwl.c b/drivers/pci/host/pcie-xilinx-nwl.c
index 5139e64..3479d30 100644
--- a/drivers/pci/host/pcie-xilinx-nwl.c
+++ b/drivers/pci/host/pcie-xilinx-nwl.c
@@ -819,7 +819,7 @@
err = nwl_pcie_bridge_init(pcie);
if (err) {
- dev_err(pcie->dev, "HW Initalization failed\n");
+ dev_err(pcie->dev, "HW Initialization failed\n");
return err;
}
diff --git a/drivers/pci/hotplug/acpiphp_ibm.c b/drivers/pci/hotplug/acpiphp_ibm.c
index 2f6d3a1..f6221d7 100644
--- a/drivers/pci/hotplug/acpiphp_ibm.c
+++ b/drivers/pci/hotplug/acpiphp_ibm.c
@@ -138,6 +138,8 @@
char *table;
size = ibm_get_table_from_acpi(&table);
+ if (size < 0)
+ return NULL;
des = (union apci_descriptor *)table;
if (memcmp(des->header.sig, "aPCI", 4) != 0)
goto ibm_slot_done;
diff --git a/drivers/pci/hotplug/rpadlpar_core.c b/drivers/pci/hotplug/rpadlpar_core.c
index b46b57d..dc67f39 100644
--- a/drivers/pci/hotplug/rpadlpar_core.c
+++ b/drivers/pci/hotplug/rpadlpar_core.c
@@ -175,7 +175,7 @@
struct pci_dev *dev;
struct pci_controller *phb;
- if (pcibios_find_pci_bus(dn))
+ if (pci_find_bus_by_node(dn))
return -EINVAL;
/* Add pci bus */
@@ -212,7 +212,7 @@
struct pci_dn *pdn;
int rc = 0;
- if (!pcibios_find_pci_bus(dn))
+ if (!pci_find_bus_by_node(dn))
return -EINVAL;
/* If pci slot is hotpluggable, use hotplug to remove it */
@@ -356,7 +356,7 @@
pci_lock_rescan_remove();
- bus = pcibios_find_pci_bus(dn);
+ bus = pci_find_bus_by_node(dn);
if (!bus) {
ret = -EINVAL;
goto out;
@@ -380,7 +380,7 @@
}
/* Remove all devices below slot */
- pcibios_remove_pci_devices(bus);
+ pci_hp_remove_devices(bus);
/* Unmap PCI IO space */
if (pcibios_unmap_io_space(bus)) {
diff --git a/drivers/pci/hotplug/rpaphp_core.c b/drivers/pci/hotplug/rpaphp_core.c
index 611f605..8d13202 100644
--- a/drivers/pci/hotplug/rpaphp_core.c
+++ b/drivers/pci/hotplug/rpaphp_core.c
@@ -404,7 +404,7 @@
if (state == PRESENT) {
pci_lock_rescan_remove();
- pcibios_add_pci_devices(slot->bus);
+ pci_hp_add_devices(slot->bus);
pci_unlock_rescan_remove();
slot->state = CONFIGURED;
} else if (state == EMPTY) {
@@ -426,7 +426,7 @@
return -EINVAL;
pci_lock_rescan_remove();
- pcibios_remove_pci_devices(slot->bus);
+ pci_hp_remove_devices(slot->bus);
pci_unlock_rescan_remove();
vm_unmap_aliases();
diff --git a/drivers/pci/hotplug/rpaphp_pci.c b/drivers/pci/hotplug/rpaphp_pci.c
index 7836d69..ea41ea1 100644
--- a/drivers/pci/hotplug/rpaphp_pci.c
+++ b/drivers/pci/hotplug/rpaphp_pci.c
@@ -93,7 +93,7 @@
if (rc)
return rc;
- bus = pcibios_find_pci_bus(slot->dn);
+ bus = pci_find_bus_by_node(slot->dn);
if (!bus) {
err("%s: no pci_bus for dn %s\n", __func__, slot->dn->full_name);
return -EINVAL;
@@ -116,7 +116,7 @@
}
if (list_empty(&bus->devices))
- pcibios_add_pci_devices(bus);
+ pci_hp_add_devices(bus);
if (!list_empty(&bus->devices)) {
info->adapter_status = CONFIGURED;
diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c
index 342b691..d319a9c 100644
--- a/drivers/pci/pci-sysfs.c
+++ b/drivers/pci/pci-sysfs.c
@@ -1008,6 +1008,9 @@
if (i >= PCI_ROM_RESOURCE)
return -ENODEV;
+ if (res->flags & IORESOURCE_MEM && iomem_is_exclusive(res->start))
+ return -EINVAL;
+
if (!pci_mmap_fits(pdev, i, vma, PCI_MMAP_SYSFS)) {
WARN(1, "process \"%s\" tried to map 0x%08lx bytes at page 0x%08lx on %s BAR %d (start 0x%16Lx, size 0x%16Lx)\n",
current->comm, vma->vm_end-vma->vm_start, vma->vm_pgoff,
@@ -1024,10 +1027,6 @@
pci_resource_to_user(pdev, i, res, &start, &end);
vma->vm_pgoff += start >> PAGE_SHIFT;
mmap_type = res->flags & IORESOURCE_MEM ? pci_mmap_mem : pci_mmap_io;
-
- if (res->flags & IORESOURCE_MEM && iomem_is_exclusive(start))
- return -EINVAL;
-
return pci_mmap_page_range(pdev, vma, mmap_type, write_combine);
}
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 25e0327..c8b4dbd 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -2228,7 +2228,7 @@
static unsigned long pci_ea_flags(struct pci_dev *dev, u8 prop)
{
- unsigned long flags = IORESOURCE_PCI_FIXED;
+ unsigned long flags = IORESOURCE_PCI_FIXED | IORESOURCE_PCI_EA_BEI;
switch (prop) {
case PCI_EA_P_MEM:
@@ -2389,7 +2389,7 @@
return offset + ent_size;
}
-/* Enhanced Allocation Initalization */
+/* Enhanced Allocation Initialization */
void pci_ea_init(struct pci_dev *dev)
{
int ea;
@@ -2547,7 +2547,7 @@
* pci_std_enable_acs - enable ACS on devices using standard ACS capabilites
* @dev: the PCI device
*/
-static int pci_std_enable_acs(struct pci_dev *dev)
+static void pci_std_enable_acs(struct pci_dev *dev)
{
int pos;
u16 cap;
@@ -2555,7 +2555,7 @@
pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ACS);
if (!pos)
- return -ENODEV;
+ return;
pci_read_config_word(dev, pos + PCI_ACS_CAP, &cap);
pci_read_config_word(dev, pos + PCI_ACS_CTRL, &ctrl);
@@ -2573,8 +2573,6 @@
ctrl |= (cap & PCI_ACS_UF);
pci_write_config_word(dev, pos + PCI_ACS_CTRL, ctrl);
-
- return 0;
}
/**
@@ -2586,10 +2584,10 @@
if (!pci_acs_enable)
return;
- if (!pci_std_enable_acs(dev))
+ if (!pci_dev_specific_enable_acs(dev))
return;
- pci_dev_specific_enable_acs(dev);
+ pci_std_enable_acs(dev);
}
static bool pci_acs_flags_enabled(struct pci_dev *pdev, u16 acs_flags)
@@ -3021,6 +3019,121 @@
}
EXPORT_SYMBOL(pci_request_regions_exclusive);
+#ifdef PCI_IOBASE
+struct io_range {
+ struct list_head list;
+ phys_addr_t start;
+ resource_size_t size;
+};
+
+static LIST_HEAD(io_range_list);
+static DEFINE_SPINLOCK(io_range_lock);
+#endif
+
+/*
+ * Record the PCI IO range (expressed as CPU physical address + size).
+ * Return a negative value if an error has occured, zero otherwise
+ */
+int __weak pci_register_io_range(phys_addr_t addr, resource_size_t size)
+{
+ int err = 0;
+
+#ifdef PCI_IOBASE
+ struct io_range *range;
+ resource_size_t allocated_size = 0;
+
+ /* check if the range hasn't been previously recorded */
+ spin_lock(&io_range_lock);
+ list_for_each_entry(range, &io_range_list, list) {
+ if (addr >= range->start && addr + size <= range->start + size) {
+ /* range already registered, bail out */
+ goto end_register;
+ }
+ allocated_size += range->size;
+ }
+
+ /* range not registed yet, check for available space */
+ if (allocated_size + size - 1 > IO_SPACE_LIMIT) {
+ /* if it's too big check if 64K space can be reserved */
+ if (allocated_size + SZ_64K - 1 > IO_SPACE_LIMIT) {
+ err = -E2BIG;
+ goto end_register;
+ }
+
+ size = SZ_64K;
+ pr_warn("Requested IO range too big, new size set to 64K\n");
+ }
+
+ /* add the range to the list */
+ range = kzalloc(sizeof(*range), GFP_ATOMIC);
+ if (!range) {
+ err = -ENOMEM;
+ goto end_register;
+ }
+
+ range->start = addr;
+ range->size = size;
+
+ list_add_tail(&range->list, &io_range_list);
+
+end_register:
+ spin_unlock(&io_range_lock);
+#endif
+
+ return err;
+}
+
+phys_addr_t pci_pio_to_address(unsigned long pio)
+{
+ phys_addr_t address = (phys_addr_t)OF_BAD_ADDR;
+
+#ifdef PCI_IOBASE
+ struct io_range *range;
+ resource_size_t allocated_size = 0;
+
+ if (pio > IO_SPACE_LIMIT)
+ return address;
+
+ spin_lock(&io_range_lock);
+ list_for_each_entry(range, &io_range_list, list) {
+ if (pio >= allocated_size && pio < allocated_size + range->size) {
+ address = range->start + pio - allocated_size;
+ break;
+ }
+ allocated_size += range->size;
+ }
+ spin_unlock(&io_range_lock);
+#endif
+
+ return address;
+}
+
+unsigned long __weak pci_address_to_pio(phys_addr_t address)
+{
+#ifdef PCI_IOBASE
+ struct io_range *res;
+ resource_size_t offset = 0;
+ unsigned long addr = -1;
+
+ spin_lock(&io_range_lock);
+ list_for_each_entry(res, &io_range_list, list) {
+ if (address >= res->start && address < res->start + res->size) {
+ addr = address - res->start + offset;
+ break;
+ }
+ offset += res->size;
+ }
+ spin_unlock(&io_range_lock);
+
+ return addr;
+#else
+ if (address > IO_SPACE_LIMIT)
+ return (unsigned long)-1;
+
+ return (unsigned long) address;
+#endif
+}
+
/**
* pci_remap_iospace - Remap the memory mapped I/O space
* @res: Resource describing the I/O space
@@ -4578,6 +4691,37 @@
return 0;
}
+/**
+ * pci_add_dma_alias - Add a DMA devfn alias for a device
+ * @dev: the PCI device for which alias is added
+ * @devfn: alias slot and function
+ *
+ * This helper encodes 8-bit devfn as bit number in dma_alias_mask.
+ * It should be called early, preferably as PCI fixup header quirk.
+ */
+void pci_add_dma_alias(struct pci_dev *dev, u8 devfn)
+{
+ if (!dev->dma_alias_mask)
+ dev->dma_alias_mask = kcalloc(BITS_TO_LONGS(U8_MAX),
+ sizeof(long), GFP_KERNEL);
+ if (!dev->dma_alias_mask) {
+ dev_warn(&dev->dev, "Unable to allocate DMA alias mask\n");
+ return;
+ }
+
+ set_bit(devfn, dev->dma_alias_mask);
+ dev_info(&dev->dev, "Enabling fixed DMA alias to %02x.%d\n",
+ PCI_SLOT(devfn), PCI_FUNC(devfn));
+}
+
+bool pci_devs_are_dma_aliases(struct pci_dev *dev1, struct pci_dev *dev2)
+{
+ return (dev1->dma_alias_mask &&
+ test_bit(dev2->devfn, dev1->dma_alias_mask)) ||
+ (dev2->dma_alias_mask &&
+ test_bit(dev1->devfn, dev2->dma_alias_mask));
+}
+
bool pci_device_is_present(struct pci_dev *pdev)
{
u32 v;
diff --git a/drivers/pci/pcie/Kconfig b/drivers/pci/pcie/Kconfig
index 72db7f4..22ca641 100644
--- a/drivers/pci/pcie/Kconfig
+++ b/drivers/pci/pcie/Kconfig
@@ -81,3 +81,17 @@
config PCIE_PME
def_bool y
depends on PCIEPORTBUS && PM
+
+config PCIE_DPC
+ tristate "PCIe Downstream Port Containment support"
+ depends on PCIEPORTBUS
+ default n
+ help
+ This enables PCI Express Downstream Port Containment (DPC)
+ driver support. DPC events from Root and Downstream ports
+ will be handled by the DPC driver. If your system doesn't
+ have this capability or you do not want to use this feature,
+ it is safe to answer N.
+
+ To compile this driver as a module, choose M here: the module
+ will be called pcie-dpc.
diff --git a/drivers/pci/pcie/Makefile b/drivers/pci/pcie/Makefile
index 00c62df..b24525b 100644
--- a/drivers/pci/pcie/Makefile
+++ b/drivers/pci/pcie/Makefile
@@ -14,3 +14,5 @@
obj-$(CONFIG_PCIEAER) += aer/
obj-$(CONFIG_PCIE_PME) += pme.o
+
+obj-$(CONFIG_PCIE_DPC) += pcie-dpc.o
diff --git a/drivers/pci/pcie/pcie-dpc.c b/drivers/pci/pcie/pcie-dpc.c
new file mode 100644
index 0000000..ab552f1
--- /dev/null
+++ b/drivers/pci/pcie/pcie-dpc.c
@@ -0,0 +1,163 @@
+/*
+ * PCI Express Downstream Port Containment services driver
+ * Copyright (C) 2016 Intel Corp.
+ *
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License. See the file "COPYING" in the main directory of this archive
+ * for more details.
+ */
+
+#include <linux/delay.h>
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/pcieport_if.h>
+
+struct dpc_dev {
+ struct pcie_device *dev;
+ struct work_struct work;
+ int cap_pos;
+};
+
+static void dpc_wait_link_inactive(struct pci_dev *pdev)
+{
+ unsigned long timeout = jiffies + HZ;
+ u16 lnk_status;
+
+ pcie_capability_read_word(pdev, PCI_EXP_LNKSTA, &lnk_status);
+ while (lnk_status & PCI_EXP_LNKSTA_DLLLA &&
+ !time_after(jiffies, timeout)) {
+ msleep(10);
+ pcie_capability_read_word(pdev, PCI_EXP_LNKSTA, &lnk_status);
+ }
+ if (lnk_status & PCI_EXP_LNKSTA_DLLLA)
+ dev_warn(&pdev->dev, "Link state not disabled for DPC event");
+}
+
+static void interrupt_event_handler(struct work_struct *work)
+{
+ struct dpc_dev *dpc = container_of(work, struct dpc_dev, work);
+ struct pci_dev *dev, *temp, *pdev = dpc->dev->port;
+ struct pci_bus *parent = pdev->subordinate;
+
+ pci_lock_rescan_remove();
+ list_for_each_entry_safe_reverse(dev, temp, &parent->devices,
+ bus_list) {
+ pci_dev_get(dev);
+ pci_stop_and_remove_bus_device(dev);
+ pci_dev_put(dev);
+ }
+ pci_unlock_rescan_remove();
+
+ dpc_wait_link_inactive(pdev);
+ pci_write_config_word(pdev, dpc->cap_pos + PCI_EXP_DPC_STATUS,
+ PCI_EXP_DPC_STATUS_TRIGGER | PCI_EXP_DPC_STATUS_INTERRUPT);
+}
+
+static irqreturn_t dpc_irq(int irq, void *context)
+{
+ struct dpc_dev *dpc = (struct dpc_dev *)context;
+ struct pci_dev *pdev = dpc->dev->port;
+ u16 status, source;
+
+ pci_read_config_word(pdev, dpc->cap_pos + PCI_EXP_DPC_STATUS, &status);
+ pci_read_config_word(pdev, dpc->cap_pos + PCI_EXP_DPC_SOURCE_ID,
+ &source);
+ if (!status)
+ return IRQ_NONE;
+
+ dev_info(&dpc->dev->device, "DPC containment event, status:%#06x source:%#06x\n",
+ status, source);
+
+ if (status & PCI_EXP_DPC_STATUS_TRIGGER) {
+ u16 reason = (status >> 1) & 0x3;
+
+ dev_warn(&dpc->dev->device, "DPC %s triggered, remove downstream devices\n",
+ (reason == 0) ? "unmasked uncorrectable error" :
+ (reason == 1) ? "ERR_NONFATAL" :
+ (reason == 2) ? "ERR_FATAL" : "extended error");
+ schedule_work(&dpc->work);
+ }
+ return IRQ_HANDLED;
+}
+
+#define FLAG(x, y) (((x) & (y)) ? '+' : '-')
+static int dpc_probe(struct pcie_device *dev)
+{
+ struct dpc_dev *dpc;
+ struct pci_dev *pdev = dev->port;
+ int status;
+ u16 ctl, cap;
+
+ dpc = kzalloc(sizeof(*dpc), GFP_KERNEL);
+ if (!dpc)
+ return -ENOMEM;
+
+ dpc->cap_pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_DPC);
+ dpc->dev = dev;
+ INIT_WORK(&dpc->work, interrupt_event_handler);
+ set_service_data(dev, dpc);
+
+ status = request_irq(dev->irq, dpc_irq, IRQF_SHARED, "pcie-dpc", dpc);
+ if (status) {
+ dev_warn(&dev->device, "request IRQ%d failed: %d\n", dev->irq,
+ status);
+ goto out;
+ }
+
+ pci_read_config_word(pdev, dpc->cap_pos + PCI_EXP_DPC_CAP, &cap);
+ pci_read_config_word(pdev, dpc->cap_pos + PCI_EXP_DPC_CTL, &ctl);
+
+ ctl |= PCI_EXP_DPC_CTL_EN_NONFATAL | PCI_EXP_DPC_CTL_INT_EN;
+ pci_write_config_word(pdev, dpc->cap_pos + PCI_EXP_DPC_CTL, ctl);
+
+ dev_info(&dev->device, "DPC error containment capabilities: Int Msg #%d, RPExt%c PoisonedTLP%c SwTrigger%c RP PIO Log %d, DL_ActiveErr%c\n",
+ cap & 0xf, FLAG(cap, PCI_EXP_DPC_CAP_RP_EXT),
+ FLAG(cap, PCI_EXP_DPC_CAP_POISONED_TLP),
+ FLAG(cap, PCI_EXP_DPC_CAP_SW_TRIGGER), (cap >> 8) & 0xf,
+ FLAG(cap, PCI_EXP_DPC_CAP_DL_ACTIVE));
+ return status;
+ out:
+ kfree(dpc);
+ return status;
+}
+
+static void dpc_remove(struct pcie_device *dev)
+{
+ struct dpc_dev *dpc = get_service_data(dev);
+ struct pci_dev *pdev = dev->port;
+ u16 ctl;
+
+ pci_read_config_word(pdev, dpc->cap_pos + PCI_EXP_DPC_CTL, &ctl);
+ ctl &= ~(PCI_EXP_DPC_CTL_EN_NONFATAL | PCI_EXP_DPC_CTL_INT_EN);
+ pci_write_config_word(pdev, dpc->cap_pos + PCI_EXP_DPC_CTL, ctl);
+
+ free_irq(dev->irq, dpc);
+ kfree(dpc);
+}
+
+static struct pcie_port_service_driver dpcdriver = {
+ .name = "dpc",
+ .port_type = PCI_EXP_TYPE_ROOT_PORT | PCI_EXP_TYPE_DOWNSTREAM,
+ .service = PCIE_PORT_SERVICE_DPC,
+ .probe = dpc_probe,
+ .remove = dpc_remove,
+};
+
+static int __init dpc_service_init(void)
+{
+ return pcie_port_service_register(&dpcdriver);
+}
+
+static void __exit dpc_service_exit(void)
+{
+ pcie_port_service_unregister(&dpcdriver);
+}
+
+MODULE_DESCRIPTION("PCI Express Downstream Port Containment driver");
+MODULE_AUTHOR("Keith Busch <keith.busch@intel.com>");
+MODULE_LICENSE("GPL");
+MODULE_VERSION("0.1");
+
+module_init(dpc_service_init);
+module_exit(dpc_service_exit);
diff --git a/drivers/pci/pcie/portdrv.h b/drivers/pci/pcie/portdrv.h
index d525548..587aef3 100644
--- a/drivers/pci/pcie/portdrv.h
+++ b/drivers/pci/pcie/portdrv.h
@@ -11,14 +11,14 @@
#include <linux/compiler.h>
-#define PCIE_PORT_DEVICE_MAXSERVICES 4
+#define PCIE_PORT_DEVICE_MAXSERVICES 5
/*
* According to the PCI Express Base Specification 2.0, the indices of
* the MSI-X table entries used by port services must not exceed 31
*/
#define PCIE_PORT_MAX_MSIX_ENTRIES 32
-#define get_descriptor_id(type, service) (((type - 4) << 4) | service)
+#define get_descriptor_id(type, service) (((type - 4) << 8) | service)
extern struct bus_type pcie_port_bus_type;
int pcie_port_device_register(struct pci_dev *dev);
@@ -67,17 +67,14 @@
#endif /* !CONFIG_PCIE_PME */
#ifdef CONFIG_ACPI
-int pcie_port_acpi_setup(struct pci_dev *port, int *mask);
+void pcie_port_acpi_setup(struct pci_dev *port, int *mask);
-static inline int pcie_port_platform_notify(struct pci_dev *port, int *mask)
+static inline void pcie_port_platform_notify(struct pci_dev *port, int *mask)
{
- return pcie_port_acpi_setup(port, mask);
+ pcie_port_acpi_setup(port, mask);
}
#else /* !CONFIG_ACPI */
-static inline int pcie_port_platform_notify(struct pci_dev *port, int *mask)
-{
- return 0;
-}
+static inline void pcie_port_platform_notify(struct pci_dev *port, int *mask){}
#endif /* !CONFIG_ACPI */
#endif /* _PORTDRV_H_ */
diff --git a/drivers/pci/pcie/portdrv_acpi.c b/drivers/pci/pcie/portdrv_acpi.c
index b4d2894..6b8c2f1 100644
--- a/drivers/pci/pcie/portdrv_acpi.c
+++ b/drivers/pci/pcie/portdrv_acpi.c
@@ -32,32 +32,30 @@
* NOTE: It turns out that we cannot do that for individual port services
* separately, because that would make some systems work incorrectly.
*/
-int pcie_port_acpi_setup(struct pci_dev *port, int *srv_mask)
+void pcie_port_acpi_setup(struct pci_dev *port, int *srv_mask)
{
struct acpi_pci_root *root;
acpi_handle handle;
u32 flags;
if (acpi_pci_disabled)
- return 0;
+ return;
handle = acpi_find_root_bridge_handle(port);
if (!handle)
- return -EINVAL;
+ return;
root = acpi_pci_find_root(handle);
if (!root)
- return -ENODEV;
+ return;
flags = root->osc_control_set;
- *srv_mask = PCIE_PORT_SERVICE_VC;
+ *srv_mask = PCIE_PORT_SERVICE_VC | PCIE_PORT_SERVICE_DPC;
if (flags & OSC_PCI_EXPRESS_NATIVE_HP_CONTROL)
*srv_mask |= PCIE_PORT_SERVICE_HP;
if (flags & OSC_PCI_EXPRESS_PME_CONTROL)
*srv_mask |= PCIE_PORT_SERVICE_PME;
if (flags & OSC_PCI_EXPRESS_AER_CONTROL)
*srv_mask |= PCIE_PORT_SERVICE_AER;
-
- return 0;
}
diff --git a/drivers/pci/pcie/portdrv_core.c b/drivers/pci/pcie/portdrv_core.c
index 88122dc..32d4d0a 100644
--- a/drivers/pci/pcie/portdrv_core.c
+++ b/drivers/pci/pcie/portdrv_core.c
@@ -254,38 +254,28 @@
static int get_port_device_capability(struct pci_dev *dev)
{
int services = 0;
- u32 reg32;
int cap_mask = 0;
- int err;
if (pcie_ports_disabled)
return 0;
cap_mask = PCIE_PORT_SERVICE_PME | PCIE_PORT_SERVICE_HP
- | PCIE_PORT_SERVICE_VC;
+ | PCIE_PORT_SERVICE_VC | PCIE_PORT_SERVICE_DPC;
if (pci_aer_available())
cap_mask |= PCIE_PORT_SERVICE_AER;
- if (pcie_ports_auto) {
- err = pcie_port_platform_notify(dev, &cap_mask);
- if (err)
- return 0;
- }
+ if (pcie_ports_auto)
+ pcie_port_platform_notify(dev, &cap_mask);
/* Hot-Plug Capable */
- if ((cap_mask & PCIE_PORT_SERVICE_HP) &&
- pcie_caps_reg(dev) & PCI_EXP_FLAGS_SLOT) {
- pcie_capability_read_dword(dev, PCI_EXP_SLTCAP, ®32);
- if (reg32 & PCI_EXP_SLTCAP_HPC) {
- services |= PCIE_PORT_SERVICE_HP;
- /*
- * Disable hot-plug interrupts in case they have been
- * enabled by the BIOS and the hot-plug service driver
- * is not loaded.
- */
- pcie_capability_clear_word(dev, PCI_EXP_SLTCTL,
- PCI_EXP_SLTCTL_CCIE | PCI_EXP_SLTCTL_HPIE);
- }
+ if ((cap_mask & PCIE_PORT_SERVICE_HP) && dev->is_hotplug_bridge) {
+ services |= PCIE_PORT_SERVICE_HP;
+ /*
+ * Disable hot-plug interrupts in case they have been enabled
+ * by the BIOS and the hot-plug service driver is not loaded.
+ */
+ pcie_capability_clear_word(dev, PCI_EXP_SLTCTL,
+ PCI_EXP_SLTCTL_CCIE | PCI_EXP_SLTCTL_HPIE);
}
/* AER capable */
if ((cap_mask & PCIE_PORT_SERVICE_AER)
@@ -311,6 +301,8 @@
*/
pcie_pme_interrupt_enable(dev, false);
}
+ if (pci_find_ext_capability(dev, PCI_EXT_CAP_ID_DPC))
+ services |= PCIE_PORT_SERVICE_DPC;
return services;
}
@@ -338,7 +330,7 @@
device = &pcie->device;
device->bus = &pcie_port_bus_type;
device->release = release_pcie_device; /* callback to free pcie dev */
- dev_set_name(device, "%s:pcie%02x",
+ dev_set_name(device, "%s:pcie%03x",
pci_name(pdev),
get_descriptor_id(pci_pcie_type(pdev), service));
device->parent = &pdev->dev;
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 8004f67..8e3ef72 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -179,9 +179,6 @@
u16 orig_cmd;
struct pci_bus_region region, inverted_region;
- if (dev->non_compliant_bars)
- return 0;
-
mask = type ? PCI_ROM_ADDRESS_MASK : ~0;
/* No printks while decoding is disabled! */
@@ -322,6 +319,9 @@
{
unsigned int pos, reg;
+ if (dev->non_compliant_bars)
+ return;
+
for (pos = 0; pos < howmany; pos++) {
struct resource *res = &dev->resource[pos];
reg = PCI_BASE_ADDRESS_0 + (pos << 2);
@@ -1537,6 +1537,7 @@
pcibios_release_device(pci_dev);
pci_bus_put(pci_dev->bus);
kfree(pci_dev->driver_override);
+ kfree(pci_dev->dma_alias_mask);
kfree(pci_dev);
}
diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 8e67802..ee72ebe 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -3150,6 +3150,39 @@
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_MELLANOX, PCI_ANY_ID,
quirk_broken_intx_masking);
+/*
+ * Intel i40e (XL710/X710) 10/20/40GbE NICs all have broken INTx masking,
+ * DisINTx can be set but the interrupt status bit is non-functional.
+ */
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1572,
+ quirk_broken_intx_masking);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1574,
+ quirk_broken_intx_masking);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1580,
+ quirk_broken_intx_masking);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1581,
+ quirk_broken_intx_masking);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1583,
+ quirk_broken_intx_masking);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1584,
+ quirk_broken_intx_masking);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1585,
+ quirk_broken_intx_masking);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1586,
+ quirk_broken_intx_masking);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1587,
+ quirk_broken_intx_masking);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1588,
+ quirk_broken_intx_masking);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1589,
+ quirk_broken_intx_masking);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x37d0,
+ quirk_broken_intx_masking);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x37d1,
+ quirk_broken_intx_masking);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x37d2,
+ quirk_broken_intx_masking);
+
static void quirk_no_bus_reset(struct pci_dev *dev)
{
dev->dev_flags |= PCI_DEV_FLAGS_NO_BUS_RESET;
@@ -3185,6 +3218,29 @@
DECLARE_PCI_FIXUP_CLASS_HEADER(PCI_VENDOR_ID_ATI, PCI_ANY_ID,
PCI_CLASS_DISPLAY_VGA, 8, quirk_no_pm_reset);
+/*
+ * Thunderbolt controllers with broken MSI hotplug signaling:
+ * Entire 1st generation (Light Ridge, Eagle Ridge, Light Peak) and part
+ * of the 2nd generation (Cactus Ridge 4C up to revision 1, Port Ridge).
+ */
+static void quirk_thunderbolt_hotplug_msi(struct pci_dev *pdev)
+{
+ if (pdev->is_hotplug_bridge &&
+ (pdev->device != PCI_DEVICE_ID_INTEL_CACTUS_RIDGE_4C ||
+ pdev->revision <= 1))
+ pdev->no_msi = 1;
+}
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_LIGHT_RIDGE,
+ quirk_thunderbolt_hotplug_msi);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_EAGLE_RIDGE,
+ quirk_thunderbolt_hotplug_msi);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_LIGHT_PEAK,
+ quirk_thunderbolt_hotplug_msi);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_CACTUS_RIDGE_4C,
+ quirk_thunderbolt_hotplug_msi);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_PORT_RIDGE,
+ quirk_thunderbolt_hotplug_msi);
+
#ifdef CONFIG_ACPI
/*
* Apple: Shutdown Cactus Ridge Thunderbolt controller.
@@ -3232,7 +3288,8 @@
acpi_execute_simple_method(SXIO, NULL, 0);
acpi_execute_simple_method(SXLV, NULL, 0);
}
-DECLARE_PCI_FIXUP_SUSPEND_LATE(PCI_VENDOR_ID_INTEL, 0x1547,
+DECLARE_PCI_FIXUP_SUSPEND_LATE(PCI_VENDOR_ID_INTEL,
+ PCI_DEVICE_ID_INTEL_CACTUS_RIDGE_4C,
quirk_apple_poweroff_thunderbolt);
/*
@@ -3266,9 +3323,11 @@
if (!nhi)
goto out;
if (nhi->vendor != PCI_VENDOR_ID_INTEL
- || (nhi->device != 0x1547 && nhi->device != 0x156c)
- || nhi->subsystem_vendor != 0x2222
- || nhi->subsystem_device != 0x1111)
+ || (nhi->device != PCI_DEVICE_ID_INTEL_LIGHT_RIDGE &&
+ nhi->device != PCI_DEVICE_ID_INTEL_CACTUS_RIDGE_4C &&
+ nhi->device != PCI_DEVICE_ID_INTEL_FALCON_RIDGE_4C_NHI)
+ || nhi->subsystem_vendor != 0x2222
+ || nhi->subsystem_device != 0x1111)
goto out;
dev_info(&dev->dev, "quirk: waiting for thunderbolt to reestablish PCI tunnels...\n");
device_pm_wait_for_dev(&dev->dev, &nhi->dev);
@@ -3276,9 +3335,14 @@
pci_dev_put(nhi);
pci_dev_put(sibling);
}
-DECLARE_PCI_FIXUP_RESUME_EARLY(PCI_VENDOR_ID_INTEL, 0x1547,
+DECLARE_PCI_FIXUP_RESUME_EARLY(PCI_VENDOR_ID_INTEL,
+ PCI_DEVICE_ID_INTEL_LIGHT_RIDGE,
quirk_apple_wait_for_thunderbolt);
-DECLARE_PCI_FIXUP_RESUME_EARLY(PCI_VENDOR_ID_INTEL, 0x156d,
+DECLARE_PCI_FIXUP_RESUME_EARLY(PCI_VENDOR_ID_INTEL,
+ PCI_DEVICE_ID_INTEL_CACTUS_RIDGE_4C,
+ quirk_apple_wait_for_thunderbolt);
+DECLARE_PCI_FIXUP_RESUME_EARLY(PCI_VENDOR_ID_INTEL,
+ PCI_DEVICE_ID_INTEL_FALCON_RIDGE_4C_BRIDGE,
quirk_apple_wait_for_thunderbolt);
#endif
@@ -3610,10 +3674,8 @@
static void quirk_dma_func0_alias(struct pci_dev *dev)
{
- if (PCI_FUNC(dev->devfn) != 0) {
- dev->dma_alias_devfn = PCI_DEVFN(PCI_SLOT(dev->devfn), 0);
- dev->dev_flags |= PCI_DEV_FLAGS_DMA_ALIAS_DEVFN;
- }
+ if (PCI_FUNC(dev->devfn) != 0)
+ pci_add_dma_alias(dev, PCI_DEVFN(PCI_SLOT(dev->devfn), 0));
}
/*
@@ -3626,10 +3688,8 @@
static void quirk_dma_func1_alias(struct pci_dev *dev)
{
- if (PCI_FUNC(dev->devfn) != 1) {
- dev->dma_alias_devfn = PCI_DEVFN(PCI_SLOT(dev->devfn), 1);
- dev->dev_flags |= PCI_DEV_FLAGS_DMA_ALIAS_DEVFN;
- }
+ if (PCI_FUNC(dev->devfn) != 1)
+ pci_add_dma_alias(dev, PCI_DEVFN(PCI_SLOT(dev->devfn), 1));
}
/*
@@ -3695,13 +3755,8 @@
const struct pci_device_id *id;
id = pci_match_id(fixed_dma_alias_tbl, dev);
- if (id) {
- dev->dma_alias_devfn = id->driver_data;
- dev->dev_flags |= PCI_DEV_FLAGS_DMA_ALIAS_DEVFN;
- dev_info(&dev->dev, "Enabling fixed DMA alias to %02x.%d\n",
- PCI_SLOT(dev->dma_alias_devfn),
- PCI_FUNC(dev->dma_alias_devfn));
- }
+ if (id)
+ pci_add_dma_alias(dev, id->driver_data);
}
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_ADAPTEC2, 0x0285, quirk_fixed_dma_alias);
@@ -3734,6 +3789,21 @@
DECLARE_PCI_FIXUP_HEADER(0x8086, 0x244e, quirk_use_pcie_bridge_dma_alias);
/*
+ * MIC x200 NTB forwards PCIe traffic using multiple alien RIDs. They have to
+ * be added as aliases to the DMA device in order to allow buffer access
+ * when IOMMU is enabled. Following devfns have to match RIT-LUT table
+ * programmed in the EEPROM.
+ */
+static void quirk_mic_x200_dma_alias(struct pci_dev *pdev)
+{
+ pci_add_dma_alias(pdev, PCI_DEVFN(0x10, 0x0));
+ pci_add_dma_alias(pdev, PCI_DEVFN(0x11, 0x0));
+ pci_add_dma_alias(pdev, PCI_DEVFN(0x12, 0x3));
+}
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x2260, quirk_mic_x200_dma_alias);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x2264, quirk_mic_x200_dma_alias);
+
+/*
* Intersil/Techwell TW686[4589]-based video capture cards have an empty (zero)
* class code. Fix it.
*/
@@ -3936,6 +4006,55 @@
return acs_flags & ~flags ? 0 : 1;
}
+/*
+ * Sunrise Point PCH root ports implement ACS, but unfortunately as shown in
+ * the datasheet (Intel 100 Series Chipset Family PCH Datasheet, Vol. 2,
+ * 12.1.46, 12.1.47)[1] this chipset uses dwords for the ACS capability and
+ * control registers whereas the PCIe spec packs them into words (Rev 3.0,
+ * 7.16 ACS Extended Capability). The bit definitions are correct, but the
+ * control register is at offset 8 instead of 6 and we should probably use
+ * dword accesses to them. This applies to the following PCI Device IDs, as
+ * found in volume 1 of the datasheet[2]:
+ *
+ * 0xa110-0xa11f Sunrise Point-H PCI Express Root Port #{0-16}
+ * 0xa167-0xa16a Sunrise Point-H PCI Express Root Port #{17-20}
+ *
+ * N.B. This doesn't fix what lspci shows.
+ *
+ * [1] http://www.intel.com/content/www/us/en/chipsets/100-series-chipset-datasheet-vol-2.html
+ * [2] http://www.intel.com/content/www/us/en/chipsets/100-series-chipset-datasheet-vol-1.html
+ */
+static bool pci_quirk_intel_spt_pch_acs_match(struct pci_dev *dev)
+{
+ return pci_is_pcie(dev) &&
+ pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT &&
+ ((dev->device & ~0xf) == 0xa110 ||
+ (dev->device >= 0xa167 && dev->device <= 0xa16a));
+}
+
+#define INTEL_SPT_ACS_CTRL (PCI_ACS_CAP + 4)
+
+static int pci_quirk_intel_spt_pch_acs(struct pci_dev *dev, u16 acs_flags)
+{
+ int pos;
+ u32 cap, ctrl;
+
+ if (!pci_quirk_intel_spt_pch_acs_match(dev))
+ return -ENOTTY;
+
+ pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ACS);
+ if (!pos)
+ return -ENOTTY;
+
+ /* see pci_acs_flags_enabled() */
+ pci_read_config_dword(dev, pos + PCI_ACS_CAP, &cap);
+ acs_flags &= (cap | PCI_ACS_EC);
+
+ pci_read_config_dword(dev, pos + INTEL_SPT_ACS_CTRL, &ctrl);
+
+ return acs_flags & ~ctrl ? 0 : 1;
+}
+
static int pci_quirk_mf_endpoint_acs(struct pci_dev *dev, u16 acs_flags)
{
/*
@@ -4024,6 +4143,7 @@
{ PCI_VENDOR_ID_INTEL, 0x15b8, pci_quirk_mf_endpoint_acs },
/* Intel PCH root ports */
{ PCI_VENDOR_ID_INTEL, PCI_ANY_ID, pci_quirk_intel_pch_acs },
+ { PCI_VENDOR_ID_INTEL, PCI_ANY_ID, pci_quirk_intel_spt_pch_acs },
{ 0x19a2, 0x710, pci_quirk_mf_endpoint_acs }, /* Emulex BE3-R */
{ 0x10df, 0x720, pci_quirk_mf_endpoint_acs }, /* Emulex Skyhawk-R */
/* Cavium ThunderX */
@@ -4159,16 +4279,44 @@
return 0;
}
+static int pci_quirk_enable_intel_spt_pch_acs(struct pci_dev *dev)
+{
+ int pos;
+ u32 cap, ctrl;
+
+ if (!pci_quirk_intel_spt_pch_acs_match(dev))
+ return -ENOTTY;
+
+ pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ACS);
+ if (!pos)
+ return -ENOTTY;
+
+ pci_read_config_dword(dev, pos + PCI_ACS_CAP, &cap);
+ pci_read_config_dword(dev, pos + INTEL_SPT_ACS_CTRL, &ctrl);
+
+ ctrl |= (cap & PCI_ACS_SV);
+ ctrl |= (cap & PCI_ACS_RR);
+ ctrl |= (cap & PCI_ACS_CR);
+ ctrl |= (cap & PCI_ACS_UF);
+
+ pci_write_config_dword(dev, pos + INTEL_SPT_ACS_CTRL, ctrl);
+
+ dev_info(&dev->dev, "Intel SPT PCH root port ACS workaround enabled\n");
+
+ return 0;
+}
+
static const struct pci_dev_enable_acs {
u16 vendor;
u16 device;
int (*enable_acs)(struct pci_dev *dev);
} pci_dev_enable_acs[] = {
{ PCI_VENDOR_ID_INTEL, PCI_ANY_ID, pci_quirk_enable_intel_pch_acs },
+ { PCI_VENDOR_ID_INTEL, PCI_ANY_ID, pci_quirk_enable_intel_spt_pch_acs },
{ 0 }
};
-void pci_dev_specific_enable_acs(struct pci_dev *dev)
+int pci_dev_specific_enable_acs(struct pci_dev *dev)
{
const struct pci_dev_enable_acs *i;
int ret;
@@ -4180,9 +4328,11 @@
i->device == (u16)PCI_ANY_ID)) {
ret = i->enable_acs(dev);
if (ret >= 0)
- return;
+ return ret;
}
}
+
+ return -ENOTTY;
}
/*
diff --git a/drivers/pci/search.c b/drivers/pci/search.c
index a20ce7d..33e0f03 100644
--- a/drivers/pci/search.c
+++ b/drivers/pci/search.c
@@ -40,11 +40,15 @@
* If the device is broken and uses an alias requester ID for
* DMA, iterate over that too.
*/
- if (unlikely(pdev->dev_flags & PCI_DEV_FLAGS_DMA_ALIAS_DEVFN)) {
- ret = fn(pdev, PCI_DEVID(pdev->bus->number,
- pdev->dma_alias_devfn), data);
- if (ret)
- return ret;
+ if (unlikely(pdev->dma_alias_mask)) {
+ u8 devfn;
+
+ for_each_set_bit(devfn, pdev->dma_alias_mask, U8_MAX) {
+ ret = fn(pdev, PCI_DEVID(pdev->bus->number, devfn),
+ data);
+ if (ret)
+ return ret;
+ }
}
for (bus = pdev->bus; !pci_is_root_bus(bus); bus = bus->parent) {
diff --git a/drivers/pcmcia/electra_cf.c b/drivers/pcmcia/electra_cf.c
index 61cf61a..4d7bc3f 100644
--- a/drivers/pcmcia/electra_cf.c
+++ b/drivers/pcmcia/electra_cf.c
@@ -228,7 +228,7 @@
if (!cf->mem_base || !cf->io_virt || !cf->gpio_base ||
(__ioremap_at(io.start, cf->io_virt, cf->io_size,
- _PAGE_NO_CACHE | _PAGE_GUARDED) == NULL)) {
+ pgprot_val(pgprot_noncached(__pgprot(0)))) == NULL)) {
dev_err(device, "can't ioremap ranges\n");
status = -ENOMEM;
goto fail1;
diff --git a/drivers/phy/Kconfig b/drivers/phy/Kconfig
index 27e5f6e..b869b98 100644
--- a/drivers/phy/Kconfig
+++ b/drivers/phy/Kconfig
@@ -15,6 +15,15 @@
phy users can obtain reference to the PHY. All the users of this
framework should select this config.
+config PHY_BCM_NS_USB2
+ tristate "Broadcom Northstar USB 2.0 PHY Driver"
+ depends on ARCH_BCM_IPROC || COMPILE_TEST
+ depends on HAS_IOMEM && OF
+ select GENERIC_PHY
+ help
+ Enable this to support Broadcom USB 2.0 PHY connected to the USB
+ controller on Northstar family.
+
config PHY_BERLIN_USB
tristate "Marvell Berlin USB PHY Driver"
depends on ARCH_BERLIN && RESET_CONTROLLER && HAS_IOMEM && OF
@@ -113,14 +122,15 @@
config PHY_RCAR_GEN2
tristate "Renesas R-Car generation 2 USB PHY driver"
- depends on ARCH_SHMOBILE
+ depends on ARCH_RENESAS
depends on GENERIC_PHY
help
Support for USB PHY found on Renesas R-Car generation 2 SoCs.
config PHY_RCAR_GEN3_USB2
tristate "Renesas R-Car generation 3 USB 2.0 PHY driver"
- depends on OF && ARCH_SHMOBILE
+ depends on ARCH_RENESAS
+ depends on EXTCON
select GENERIC_PHY
help
Support for USB 2.0 PHY found on Renesas R-Car generation 3 SoCs.
@@ -218,9 +228,8 @@
depends on ARCH_MEDIATEK && OF
select GENERIC_PHY
help
- Say 'Y' here to add support for Mediatek USB3.0 PHY driver
- for mt65xx SoCs. it supports two usb2.0 ports and
- one usb3.0 port.
+ Say 'Y' here to add support for Mediatek USB3.0 PHY driver,
+ it supports multiple usb2.0 and usb3.0 ports.
config PHY_HI6220_USB
tristate "hi6220 USB PHY support"
@@ -250,7 +259,8 @@
tristate "Allwinner sun9i SoC USB PHY driver"
depends on ARCH_SUNXI && HAS_IOMEM && OF
depends on RESET_CONTROLLER
- depends on USB_COMMON
+ depends on USB_SUPPORT
+ select USB_COMMON
select GENERIC_PHY
help
Enable this to support the transceiver that is part of Allwinner
@@ -403,14 +413,15 @@
help
Support for TI TUSB1210 USB ULPI PHY.
-config PHY_BRCMSTB_SATA
- tristate "Broadcom STB SATA PHY driver"
- depends on ARCH_BRCMSTB || BMIPS_GENERIC
+config PHY_BRCM_SATA
+ tristate "Broadcom SATA PHY driver"
+ depends on ARCH_BRCMSTB || ARCH_BCM_IPROC || BMIPS_GENERIC || COMPILE_TEST
depends on OF
select GENERIC_PHY
+ default ARCH_BCM_IPROC
help
- Enable this to support the SATA3 PHY on 28nm or 40nm Broadcom STB SoCs.
- Likely useful only with CONFIG_SATA_BRCMSTB enabled.
+ Enable this to support the Broadcom SATA PHY.
+ If unsure, say N.
config PHY_CYGNUS_PCIE
tristate "Broadcom Cygnus PCIe PHY driver"
diff --git a/drivers/phy/Makefile b/drivers/phy/Makefile
index d4f06e6..9c3e73c 100644
--- a/drivers/phy/Makefile
+++ b/drivers/phy/Makefile
@@ -3,6 +3,7 @@
#
obj-$(CONFIG_GENERIC_PHY) += phy-core.o
+obj-$(CONFIG_PHY_BCM_NS_USB2) += phy-bcm-ns-usb2.o
obj-$(CONFIG_PHY_BERLIN_USB) += phy-berlin-usb.o
obj-$(CONFIG_PHY_BERLIN_SATA) += phy-berlin-sata.o
obj-$(CONFIG_PHY_DM816X_USB) += phy-dm816x-usb.o
@@ -49,7 +50,7 @@
obj-$(CONFIG_PHY_QCOM_UFS) += phy-qcom-ufs-qmp-20nm.o
obj-$(CONFIG_PHY_QCOM_UFS) += phy-qcom-ufs-qmp-14nm.o
obj-$(CONFIG_PHY_TUSB1210) += phy-tusb1210.o
-obj-$(CONFIG_PHY_BRCMSTB_SATA) += phy-brcmstb-sata.o
+obj-$(CONFIG_PHY_BRCM_SATA) += phy-brcm-sata.o
obj-$(CONFIG_PHY_PISTACHIO_USB) += phy-pistachio-usb.o
obj-$(CONFIG_PHY_CYGNUS_PCIE) += phy-bcm-cygnus-pcie.o
diff --git a/drivers/phy/phy-bcm-ns-usb2.c b/drivers/phy/phy-bcm-ns-usb2.c
new file mode 100644
index 0000000..95ab6b2
--- /dev/null
+++ b/drivers/phy/phy-bcm-ns-usb2.c
@@ -0,0 +1,137 @@
+/*
+ * Broadcom Northstar USB 2.0 PHY Driver
+ *
+ * Copyright (C) 2016 Rafał Miłecki <zajec5@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#include <linux/bcma/bcma.h>
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/err.h>
+#include <linux/module.h>
+#include <linux/of_address.h>
+#include <linux/of_platform.h>
+#include <linux/phy/phy.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+
+struct bcm_ns_usb2 {
+ struct device *dev;
+ struct clk *ref_clk;
+ struct phy *phy;
+ void __iomem *dmu;
+};
+
+static int bcm_ns_usb2_phy_init(struct phy *phy)
+{
+ struct bcm_ns_usb2 *usb2 = phy_get_drvdata(phy);
+ struct device *dev = usb2->dev;
+ void __iomem *dmu = usb2->dmu;
+ u32 ref_clk_rate, usb2ctl, usb_pll_ndiv, usb_pll_pdiv;
+ int err = 0;
+
+ err = clk_prepare_enable(usb2->ref_clk);
+ if (err < 0) {
+ dev_err(dev, "Failed to prepare ref clock: %d\n", err);
+ goto err_out;
+ }
+
+ ref_clk_rate = clk_get_rate(usb2->ref_clk);
+ if (!ref_clk_rate) {
+ dev_err(dev, "Failed to get ref clock rate\n");
+ err = -EINVAL;
+ goto err_clk_off;
+ }
+
+ usb2ctl = readl(dmu + BCMA_DMU_CRU_USB2_CONTROL);
+
+ if (usb2ctl & BCMA_DMU_CRU_USB2_CONTROL_USB_PLL_PDIV_MASK) {
+ usb_pll_pdiv = usb2ctl;
+ usb_pll_pdiv &= BCMA_DMU_CRU_USB2_CONTROL_USB_PLL_PDIV_MASK;
+ usb_pll_pdiv >>= BCMA_DMU_CRU_USB2_CONTROL_USB_PLL_PDIV_SHIFT;
+ } else {
+ usb_pll_pdiv = 1 << 3;
+ }
+
+ /* Calculate ndiv based on a solid 1920 MHz that is for USB2 PHY */
+ usb_pll_ndiv = (1920000000 * usb_pll_pdiv) / ref_clk_rate;
+
+ /* Unlock DMU PLL settings with some magic value */
+ writel(0x0000ea68, dmu + BCMA_DMU_CRU_CLKSET_KEY);
+
+ /* Write USB 2.0 PLL control setting */
+ usb2ctl &= ~BCMA_DMU_CRU_USB2_CONTROL_USB_PLL_NDIV_MASK;
+ usb2ctl |= usb_pll_ndiv << BCMA_DMU_CRU_USB2_CONTROL_USB_PLL_NDIV_SHIFT;
+ writel(usb2ctl, dmu + BCMA_DMU_CRU_USB2_CONTROL);
+
+ /* Lock DMU PLL settings */
+ writel(0x00000000, dmu + BCMA_DMU_CRU_CLKSET_KEY);
+
+err_clk_off:
+ clk_disable_unprepare(usb2->ref_clk);
+err_out:
+ return err;
+}
+
+static const struct phy_ops ops = {
+ .init = bcm_ns_usb2_phy_init,
+ .owner = THIS_MODULE,
+};
+
+static int bcm_ns_usb2_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct bcm_ns_usb2 *usb2;
+ struct resource *res;
+ struct phy_provider *phy_provider;
+
+ usb2 = devm_kzalloc(&pdev->dev, sizeof(*usb2), GFP_KERNEL);
+ if (!usb2)
+ return -ENOMEM;
+ usb2->dev = dev;
+
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "dmu");
+ usb2->dmu = devm_ioremap_resource(dev, res);
+ if (IS_ERR(usb2->dmu)) {
+ dev_err(dev, "Failed to map DMU regs\n");
+ return PTR_ERR(usb2->dmu);
+ }
+
+ usb2->ref_clk = devm_clk_get(dev, "phy-ref-clk");
+ if (IS_ERR(usb2->ref_clk)) {
+ dev_err(dev, "Clock not defined\n");
+ return PTR_ERR(usb2->ref_clk);
+ }
+
+ usb2->phy = devm_phy_create(dev, NULL, &ops);
+ if (IS_ERR(dev))
+ return PTR_ERR(dev);
+
+ phy_set_drvdata(usb2->phy, usb2);
+ platform_set_drvdata(pdev, usb2);
+
+ phy_provider = devm_of_phy_provider_register(dev, of_phy_simple_xlate);
+ return PTR_ERR_OR_ZERO(phy_provider);
+}
+
+static const struct of_device_id bcm_ns_usb2_id_table[] = {
+ { .compatible = "brcm,ns-usb2-phy", },
+ {},
+};
+MODULE_DEVICE_TABLE(of, bcm_ns_usb2_id_table);
+
+static struct platform_driver bcm_ns_usb2_driver = {
+ .probe = bcm_ns_usb2_probe,
+ .driver = {
+ .name = "bcm_ns_usb2",
+ .of_match_table = bcm_ns_usb2_id_table,
+ },
+};
+module_platform_driver(bcm_ns_usb2_driver);
+
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/phy/phy-brcm-sata.c b/drivers/phy/phy-brcm-sata.c
new file mode 100644
index 0000000..6c4c5cb
--- /dev/null
+++ b/drivers/phy/phy-brcm-sata.c
@@ -0,0 +1,412 @@
+/*
+ * Broadcom SATA3 AHCI Controller PHY Driver
+ *
+ * Copyright (C) 2016 Broadcom
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/phy/phy.h>
+#include <linux/platform_device.h>
+
+#define SATA_PCB_BANK_OFFSET 0x23c
+#define SATA_PCB_REG_OFFSET(ofs) ((ofs) * 4)
+
+#define MAX_PORTS 2
+
+/* Register offset between PHYs in PCB space */
+#define SATA_PCB_REG_28NM_SPACE_SIZE 0x1000
+
+/* The older SATA PHY registers duplicated per port registers within the map,
+ * rather than having a separate map per port.
+ */
+#define SATA_PCB_REG_40NM_SPACE_SIZE 0x10
+
+/* Register offset between PHYs in PHY control space */
+#define SATA_PHY_CTRL_REG_28NM_SPACE_SIZE 0x8
+
+enum brcm_sata_phy_version {
+ BRCM_SATA_PHY_STB_28NM,
+ BRCM_SATA_PHY_STB_40NM,
+ BRCM_SATA_PHY_IPROC_NS2,
+};
+
+struct brcm_sata_port {
+ int portnum;
+ struct phy *phy;
+ struct brcm_sata_phy *phy_priv;
+ bool ssc_en;
+};
+
+struct brcm_sata_phy {
+ struct device *dev;
+ void __iomem *phy_base;
+ void __iomem *ctrl_base;
+ enum brcm_sata_phy_version version;
+
+ struct brcm_sata_port phys[MAX_PORTS];
+};
+
+enum sata_phy_regs {
+ BLOCK0_REG_BANK = 0x000,
+ BLOCK0_XGXSSTATUS = 0x81,
+ BLOCK0_XGXSSTATUS_PLL_LOCK = BIT(12),
+ BLOCK0_SPARE = 0x8d,
+ BLOCK0_SPARE_OOB_CLK_SEL_MASK = 0x3,
+ BLOCK0_SPARE_OOB_CLK_SEL_REFBY2 = 0x1,
+
+ PLL_REG_BANK_0 = 0x050,
+ PLL_REG_BANK_0_PLLCONTROL_0 = 0x81,
+
+ PLL1_REG_BANK = 0x060,
+ PLL1_ACTRL2 = 0x82,
+ PLL1_ACTRL3 = 0x83,
+ PLL1_ACTRL4 = 0x84,
+
+ OOB_REG_BANK = 0x150,
+ OOB_CTRL1 = 0x80,
+ OOB_CTRL1_BURST_MAX_MASK = 0xf,
+ OOB_CTRL1_BURST_MAX_SHIFT = 12,
+ OOB_CTRL1_BURST_MIN_MASK = 0xf,
+ OOB_CTRL1_BURST_MIN_SHIFT = 8,
+ OOB_CTRL1_WAKE_IDLE_MAX_MASK = 0xf,
+ OOB_CTRL1_WAKE_IDLE_MAX_SHIFT = 4,
+ OOB_CTRL1_WAKE_IDLE_MIN_MASK = 0xf,
+ OOB_CTRL1_WAKE_IDLE_MIN_SHIFT = 0,
+ OOB_CTRL2 = 0x81,
+ OOB_CTRL2_SEL_ENA_SHIFT = 15,
+ OOB_CTRL2_SEL_ENA_RC_SHIFT = 14,
+ OOB_CTRL2_RESET_IDLE_MAX_MASK = 0x3f,
+ OOB_CTRL2_RESET_IDLE_MAX_SHIFT = 8,
+ OOB_CTRL2_BURST_CNT_MASK = 0x3,
+ OOB_CTRL2_BURST_CNT_SHIFT = 6,
+ OOB_CTRL2_RESET_IDLE_MIN_MASK = 0x3f,
+ OOB_CTRL2_RESET_IDLE_MIN_SHIFT = 0,
+
+ TXPMD_REG_BANK = 0x1a0,
+ TXPMD_CONTROL1 = 0x81,
+ TXPMD_CONTROL1_TX_SSC_EN_FRC = BIT(0),
+ TXPMD_CONTROL1_TX_SSC_EN_FRC_VAL = BIT(1),
+ TXPMD_TX_FREQ_CTRL_CONTROL1 = 0x82,
+ TXPMD_TX_FREQ_CTRL_CONTROL2 = 0x83,
+ TXPMD_TX_FREQ_CTRL_CONTROL2_FMIN_MASK = 0x3ff,
+ TXPMD_TX_FREQ_CTRL_CONTROL3 = 0x84,
+ TXPMD_TX_FREQ_CTRL_CONTROL3_FMAX_MASK = 0x3ff,
+};
+
+enum sata_phy_ctrl_regs {
+ PHY_CTRL_1 = 0x0,
+ PHY_CTRL_1_RESET = BIT(0),
+};
+
+static inline void __iomem *brcm_sata_pcb_base(struct brcm_sata_port *port)
+{
+ struct brcm_sata_phy *priv = port->phy_priv;
+ u32 size = 0;
+
+ switch (priv->version) {
+ case BRCM_SATA_PHY_STB_28NM:
+ case BRCM_SATA_PHY_IPROC_NS2:
+ size = SATA_PCB_REG_28NM_SPACE_SIZE;
+ break;
+ case BRCM_SATA_PHY_STB_40NM:
+ size = SATA_PCB_REG_40NM_SPACE_SIZE;
+ break;
+ default:
+ dev_err(priv->dev, "invalid phy version\n");
+ break;
+ };
+
+ return priv->phy_base + (port->portnum * size);
+}
+
+static inline void __iomem *brcm_sata_ctrl_base(struct brcm_sata_port *port)
+{
+ struct brcm_sata_phy *priv = port->phy_priv;
+ u32 size = 0;
+
+ switch (priv->version) {
+ case BRCM_SATA_PHY_IPROC_NS2:
+ size = SATA_PHY_CTRL_REG_28NM_SPACE_SIZE;
+ break;
+ default:
+ dev_err(priv->dev, "invalid phy version\n");
+ break;
+ };
+
+ return priv->ctrl_base + (port->portnum * size);
+}
+
+static void brcm_sata_phy_wr(void __iomem *pcb_base, u32 bank,
+ u32 ofs, u32 msk, u32 value)
+{
+ u32 tmp;
+
+ writel(bank, pcb_base + SATA_PCB_BANK_OFFSET);
+ tmp = readl(pcb_base + SATA_PCB_REG_OFFSET(ofs));
+ tmp = (tmp & msk) | value;
+ writel(tmp, pcb_base + SATA_PCB_REG_OFFSET(ofs));
+}
+
+static u32 brcm_sata_phy_rd(void __iomem *pcb_base, u32 bank, u32 ofs)
+{
+ writel(bank, pcb_base + SATA_PCB_BANK_OFFSET);
+ return readl(pcb_base + SATA_PCB_REG_OFFSET(ofs));
+}
+
+/* These defaults were characterized by H/W group */
+#define STB_FMIN_VAL_DEFAULT 0x3df
+#define STB_FMAX_VAL_DEFAULT 0x3df
+#define STB_FMAX_VAL_SSC 0x83
+
+static int brcm_stb_sata_init(struct brcm_sata_port *port)
+{
+ void __iomem *base = brcm_sata_pcb_base(port);
+ struct brcm_sata_phy *priv = port->phy_priv;
+ u32 tmp;
+
+ /* override the TX spread spectrum setting */
+ tmp = TXPMD_CONTROL1_TX_SSC_EN_FRC_VAL | TXPMD_CONTROL1_TX_SSC_EN_FRC;
+ brcm_sata_phy_wr(base, TXPMD_REG_BANK, TXPMD_CONTROL1, ~tmp, tmp);
+
+ /* set fixed min freq */
+ brcm_sata_phy_wr(base, TXPMD_REG_BANK, TXPMD_TX_FREQ_CTRL_CONTROL2,
+ ~TXPMD_TX_FREQ_CTRL_CONTROL2_FMIN_MASK,
+ STB_FMIN_VAL_DEFAULT);
+
+ /* set fixed max freq depending on SSC config */
+ if (port->ssc_en) {
+ dev_info(priv->dev, "enabling SSC on port%d\n", port->portnum);
+ tmp = STB_FMAX_VAL_SSC;
+ } else {
+ tmp = STB_FMAX_VAL_DEFAULT;
+ }
+
+ brcm_sata_phy_wr(base, TXPMD_REG_BANK, TXPMD_TX_FREQ_CTRL_CONTROL3,
+ ~TXPMD_TX_FREQ_CTRL_CONTROL3_FMAX_MASK, tmp);
+
+ return 0;
+}
+
+/* NS2 SATA PLL1 defaults were characterized by H/W group */
+#define NS2_PLL1_ACTRL2_MAGIC 0x1df8
+#define NS2_PLL1_ACTRL3_MAGIC 0x2b00
+#define NS2_PLL1_ACTRL4_MAGIC 0x8824
+
+static int brcm_ns2_sata_init(struct brcm_sata_port *port)
+{
+ int try;
+ unsigned int val;
+ void __iomem *base = brcm_sata_pcb_base(port);
+ void __iomem *ctrl_base = brcm_sata_ctrl_base(port);
+ struct device *dev = port->phy_priv->dev;
+
+ /* Configure OOB control */
+ val = 0x0;
+ val |= (0xc << OOB_CTRL1_BURST_MAX_SHIFT);
+ val |= (0x4 << OOB_CTRL1_BURST_MIN_SHIFT);
+ val |= (0x9 << OOB_CTRL1_WAKE_IDLE_MAX_SHIFT);
+ val |= (0x3 << OOB_CTRL1_WAKE_IDLE_MIN_SHIFT);
+ brcm_sata_phy_wr(base, OOB_REG_BANK, OOB_CTRL1, 0x0, val);
+ val = 0x0;
+ val |= (0x1b << OOB_CTRL2_RESET_IDLE_MAX_SHIFT);
+ val |= (0x2 << OOB_CTRL2_BURST_CNT_SHIFT);
+ val |= (0x9 << OOB_CTRL2_RESET_IDLE_MIN_SHIFT);
+ brcm_sata_phy_wr(base, OOB_REG_BANK, OOB_CTRL2, 0x0, val);
+
+ /* Configure PHY PLL register bank 1 */
+ val = NS2_PLL1_ACTRL2_MAGIC;
+ brcm_sata_phy_wr(base, PLL1_REG_BANK, PLL1_ACTRL2, 0x0, val);
+ val = NS2_PLL1_ACTRL3_MAGIC;
+ brcm_sata_phy_wr(base, PLL1_REG_BANK, PLL1_ACTRL3, 0x0, val);
+ val = NS2_PLL1_ACTRL4_MAGIC;
+ brcm_sata_phy_wr(base, PLL1_REG_BANK, PLL1_ACTRL4, 0x0, val);
+
+ /* Configure PHY BLOCK0 register bank */
+ /* Set oob_clk_sel to refclk/2 */
+ brcm_sata_phy_wr(base, BLOCK0_REG_BANK, BLOCK0_SPARE,
+ ~BLOCK0_SPARE_OOB_CLK_SEL_MASK,
+ BLOCK0_SPARE_OOB_CLK_SEL_REFBY2);
+
+ /* Strobe PHY reset using PHY control register */
+ writel(PHY_CTRL_1_RESET, ctrl_base + PHY_CTRL_1);
+ mdelay(1);
+ writel(0x0, ctrl_base + PHY_CTRL_1);
+ mdelay(1);
+
+ /* Wait for PHY PLL lock by polling pll_lock bit */
+ try = 50;
+ while (try) {
+ val = brcm_sata_phy_rd(base, BLOCK0_REG_BANK,
+ BLOCK0_XGXSSTATUS);
+ if (val & BLOCK0_XGXSSTATUS_PLL_LOCK)
+ break;
+ msleep(20);
+ try--;
+ }
+ if (!try) {
+ /* PLL did not lock; give up */
+ dev_err(dev, "port%d PLL did not lock\n", port->portnum);
+ return -ETIMEDOUT;
+ }
+
+ dev_dbg(dev, "port%d initialized\n", port->portnum);
+
+ return 0;
+}
+
+static int brcm_sata_phy_init(struct phy *phy)
+{
+ int rc;
+ struct brcm_sata_port *port = phy_get_drvdata(phy);
+
+ switch (port->phy_priv->version) {
+ case BRCM_SATA_PHY_STB_28NM:
+ case BRCM_SATA_PHY_STB_40NM:
+ rc = brcm_stb_sata_init(port);
+ break;
+ case BRCM_SATA_PHY_IPROC_NS2:
+ rc = brcm_ns2_sata_init(port);
+ break;
+ default:
+ rc = -ENODEV;
+ };
+
+ return 0;
+}
+
+static const struct phy_ops phy_ops = {
+ .init = brcm_sata_phy_init,
+ .owner = THIS_MODULE,
+};
+
+static const struct of_device_id brcm_sata_phy_of_match[] = {
+ { .compatible = "brcm,bcm7445-sata-phy",
+ .data = (void *)BRCM_SATA_PHY_STB_28NM },
+ { .compatible = "brcm,bcm7425-sata-phy",
+ .data = (void *)BRCM_SATA_PHY_STB_40NM },
+ { .compatible = "brcm,iproc-ns2-sata-phy",
+ .data = (void *)BRCM_SATA_PHY_IPROC_NS2 },
+ {},
+};
+MODULE_DEVICE_TABLE(of, brcm_sata_phy_of_match);
+
+static int brcm_sata_phy_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *dn = dev->of_node, *child;
+ const struct of_device_id *of_id;
+ struct brcm_sata_phy *priv;
+ struct resource *res;
+ struct phy_provider *provider;
+ int ret, count = 0;
+
+ if (of_get_child_count(dn) == 0)
+ return -ENODEV;
+
+ priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+ dev_set_drvdata(dev, priv);
+ priv->dev = dev;
+
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "phy");
+ priv->phy_base = devm_ioremap_resource(dev, res);
+ if (IS_ERR(priv->phy_base))
+ return PTR_ERR(priv->phy_base);
+
+ of_id = of_match_node(brcm_sata_phy_of_match, dn);
+ if (of_id)
+ priv->version = (enum brcm_sata_phy_version)of_id->data;
+ else
+ priv->version = BRCM_SATA_PHY_STB_28NM;
+
+ if (priv->version == BRCM_SATA_PHY_IPROC_NS2) {
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
+ "phy-ctrl");
+ priv->ctrl_base = devm_ioremap_resource(dev, res);
+ if (IS_ERR(priv->ctrl_base))
+ return PTR_ERR(priv->ctrl_base);
+ }
+
+ for_each_available_child_of_node(dn, child) {
+ unsigned int id;
+ struct brcm_sata_port *port;
+
+ if (of_property_read_u32(child, "reg", &id)) {
+ dev_err(dev, "missing reg property in node %s\n",
+ child->name);
+ ret = -EINVAL;
+ goto put_child;
+ }
+
+ if (id >= MAX_PORTS) {
+ dev_err(dev, "invalid reg: %u\n", id);
+ ret = -EINVAL;
+ goto put_child;
+ }
+ if (priv->phys[id].phy) {
+ dev_err(dev, "already registered port %u\n", id);
+ ret = -EINVAL;
+ goto put_child;
+ }
+
+ port = &priv->phys[id];
+ port->portnum = id;
+ port->phy_priv = priv;
+ port->phy = devm_phy_create(dev, child, &phy_ops);
+ port->ssc_en = of_property_read_bool(child, "brcm,enable-ssc");
+ if (IS_ERR(port->phy)) {
+ dev_err(dev, "failed to create PHY\n");
+ ret = PTR_ERR(port->phy);
+ goto put_child;
+ }
+
+ phy_set_drvdata(port->phy, port);
+ count++;
+ }
+
+ provider = devm_of_phy_provider_register(dev, of_phy_simple_xlate);
+ if (IS_ERR(provider)) {
+ dev_err(dev, "could not register PHY provider\n");
+ return PTR_ERR(provider);
+ }
+
+ dev_info(dev, "registered %d port(s)\n", count);
+
+ return 0;
+put_child:
+ of_node_put(child);
+ return ret;
+}
+
+static struct platform_driver brcm_sata_phy_driver = {
+ .probe = brcm_sata_phy_probe,
+ .driver = {
+ .of_match_table = brcm_sata_phy_of_match,
+ .name = "brcm-sata-phy",
+ }
+};
+module_platform_driver(brcm_sata_phy_driver);
+
+MODULE_DESCRIPTION("Broadcom SATA PHY driver");
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Marc Carino");
+MODULE_AUTHOR("Brian Norris");
+MODULE_ALIAS("platform:phy-brcm-sata");
diff --git a/drivers/phy/phy-brcmstb-sata.c b/drivers/phy/phy-brcmstb-sata.c
deleted file mode 100644
index a23172f..0000000
--- a/drivers/phy/phy-brcmstb-sata.c
+++ /dev/null
@@ -1,250 +0,0 @@
-/*
- * Broadcom SATA3 AHCI Controller PHY Driver
- *
- * Copyright © 2009-2015 Broadcom Corporation
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2, or (at your option)
- * any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- */
-
-#include <linux/device.h>
-#include <linux/init.h>
-#include <linux/interrupt.h>
-#include <linux/io.h>
-#include <linux/kernel.h>
-#include <linux/module.h>
-#include <linux/of.h>
-#include <linux/phy/phy.h>
-#include <linux/platform_device.h>
-
-#define SATA_MDIO_BANK_OFFSET 0x23c
-#define SATA_MDIO_REG_OFFSET(ofs) ((ofs) * 4)
-
-#define MAX_PORTS 2
-
-/* Register offset between PHYs in PCB space */
-#define SATA_MDIO_REG_28NM_SPACE_SIZE 0x1000
-
-/* The older SATA PHY registers duplicated per port registers within the map,
- * rather than having a separate map per port.
- */
-#define SATA_MDIO_REG_40NM_SPACE_SIZE 0x10
-
-enum brcm_sata_phy_version {
- BRCM_SATA_PHY_28NM,
- BRCM_SATA_PHY_40NM,
-};
-
-struct brcm_sata_port {
- int portnum;
- struct phy *phy;
- struct brcm_sata_phy *phy_priv;
- bool ssc_en;
-};
-
-struct brcm_sata_phy {
- struct device *dev;
- void __iomem *phy_base;
- enum brcm_sata_phy_version version;
-
- struct brcm_sata_port phys[MAX_PORTS];
-};
-
-enum sata_mdio_phy_regs {
- PLL_REG_BANK_0 = 0x50,
- PLL_REG_BANK_0_PLLCONTROL_0 = 0x81,
-
- TXPMD_REG_BANK = 0x1a0,
- TXPMD_CONTROL1 = 0x81,
- TXPMD_CONTROL1_TX_SSC_EN_FRC = BIT(0),
- TXPMD_CONTROL1_TX_SSC_EN_FRC_VAL = BIT(1),
- TXPMD_TX_FREQ_CTRL_CONTROL1 = 0x82,
- TXPMD_TX_FREQ_CTRL_CONTROL2 = 0x83,
- TXPMD_TX_FREQ_CTRL_CONTROL2_FMIN_MASK = 0x3ff,
- TXPMD_TX_FREQ_CTRL_CONTROL3 = 0x84,
- TXPMD_TX_FREQ_CTRL_CONTROL3_FMAX_MASK = 0x3ff,
-};
-
-static inline void __iomem *brcm_sata_phy_base(struct brcm_sata_port *port)
-{
- struct brcm_sata_phy *priv = port->phy_priv;
- u32 offset = 0;
-
- if (priv->version == BRCM_SATA_PHY_28NM)
- offset = SATA_MDIO_REG_28NM_SPACE_SIZE;
- else if (priv->version == BRCM_SATA_PHY_40NM)
- offset = SATA_MDIO_REG_40NM_SPACE_SIZE;
- else
- dev_err(priv->dev, "invalid phy version\n");
-
- return priv->phy_base + (port->portnum * offset);
-}
-
-static void brcm_sata_mdio_wr(void __iomem *addr, u32 bank, u32 ofs,
- u32 msk, u32 value)
-{
- u32 tmp;
-
- writel(bank, addr + SATA_MDIO_BANK_OFFSET);
- tmp = readl(addr + SATA_MDIO_REG_OFFSET(ofs));
- tmp = (tmp & msk) | value;
- writel(tmp, addr + SATA_MDIO_REG_OFFSET(ofs));
-}
-
-/* These defaults were characterized by H/W group */
-#define FMIN_VAL_DEFAULT 0x3df
-#define FMAX_VAL_DEFAULT 0x3df
-#define FMAX_VAL_SSC 0x83
-
-static void brcm_sata_cfg_ssc(struct brcm_sata_port *port)
-{
- void __iomem *base = brcm_sata_phy_base(port);
- struct brcm_sata_phy *priv = port->phy_priv;
- u32 tmp;
-
- /* override the TX spread spectrum setting */
- tmp = TXPMD_CONTROL1_TX_SSC_EN_FRC_VAL | TXPMD_CONTROL1_TX_SSC_EN_FRC;
- brcm_sata_mdio_wr(base, TXPMD_REG_BANK, TXPMD_CONTROL1, ~tmp, tmp);
-
- /* set fixed min freq */
- brcm_sata_mdio_wr(base, TXPMD_REG_BANK, TXPMD_TX_FREQ_CTRL_CONTROL2,
- ~TXPMD_TX_FREQ_CTRL_CONTROL2_FMIN_MASK,
- FMIN_VAL_DEFAULT);
-
- /* set fixed max freq depending on SSC config */
- if (port->ssc_en) {
- dev_info(priv->dev, "enabling SSC on port %d\n", port->portnum);
- tmp = FMAX_VAL_SSC;
- } else {
- tmp = FMAX_VAL_DEFAULT;
- }
-
- brcm_sata_mdio_wr(base, TXPMD_REG_BANK, TXPMD_TX_FREQ_CTRL_CONTROL3,
- ~TXPMD_TX_FREQ_CTRL_CONTROL3_FMAX_MASK, tmp);
-}
-
-static int brcm_sata_phy_init(struct phy *phy)
-{
- struct brcm_sata_port *port = phy_get_drvdata(phy);
-
- brcm_sata_cfg_ssc(port);
-
- return 0;
-}
-
-static const struct phy_ops phy_ops = {
- .init = brcm_sata_phy_init,
- .owner = THIS_MODULE,
-};
-
-static const struct of_device_id brcm_sata_phy_of_match[] = {
- { .compatible = "brcm,bcm7445-sata-phy",
- .data = (void *)BRCM_SATA_PHY_28NM },
- { .compatible = "brcm,bcm7425-sata-phy",
- .data = (void *)BRCM_SATA_PHY_40NM },
- {},
-};
-MODULE_DEVICE_TABLE(of, brcm_sata_phy_of_match);
-
-static int brcm_sata_phy_probe(struct platform_device *pdev)
-{
- struct device *dev = &pdev->dev;
- struct device_node *dn = dev->of_node, *child;
- const struct of_device_id *of_id;
- struct brcm_sata_phy *priv;
- struct resource *res;
- struct phy_provider *provider;
- int ret, count = 0;
-
- if (of_get_child_count(dn) == 0)
- return -ENODEV;
-
- priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
- if (!priv)
- return -ENOMEM;
- dev_set_drvdata(dev, priv);
- priv->dev = dev;
-
- res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "phy");
- priv->phy_base = devm_ioremap_resource(dev, res);
- if (IS_ERR(priv->phy_base))
- return PTR_ERR(priv->phy_base);
-
- of_id = of_match_node(brcm_sata_phy_of_match, dn);
- if (of_id)
- priv->version = (enum brcm_sata_phy_version)of_id->data;
- else
- priv->version = BRCM_SATA_PHY_28NM;
-
- for_each_available_child_of_node(dn, child) {
- unsigned int id;
- struct brcm_sata_port *port;
-
- if (of_property_read_u32(child, "reg", &id)) {
- dev_err(dev, "missing reg property in node %s\n",
- child->name);
- ret = -EINVAL;
- goto put_child;
- }
-
- if (id >= MAX_PORTS) {
- dev_err(dev, "invalid reg: %u\n", id);
- ret = -EINVAL;
- goto put_child;
- }
- if (priv->phys[id].phy) {
- dev_err(dev, "already registered port %u\n", id);
- ret = -EINVAL;
- goto put_child;
- }
-
- port = &priv->phys[id];
- port->portnum = id;
- port->phy_priv = priv;
- port->phy = devm_phy_create(dev, child, &phy_ops);
- port->ssc_en = of_property_read_bool(child, "brcm,enable-ssc");
- if (IS_ERR(port->phy)) {
- dev_err(dev, "failed to create PHY\n");
- ret = PTR_ERR(port->phy);
- goto put_child;
- }
-
- phy_set_drvdata(port->phy, port);
- count++;
- }
-
- provider = devm_of_phy_provider_register(dev, of_phy_simple_xlate);
- if (IS_ERR(provider)) {
- dev_err(dev, "could not register PHY provider\n");
- return PTR_ERR(provider);
- }
-
- dev_info(dev, "registered %d port(s)\n", count);
-
- return 0;
-put_child:
- of_node_put(child);
- return ret;
-}
-
-static struct platform_driver brcm_sata_phy_driver = {
- .probe = brcm_sata_phy_probe,
- .driver = {
- .of_match_table = brcm_sata_phy_of_match,
- .name = "brcmstb-sata-phy",
- }
-};
-module_platform_driver(brcm_sata_phy_driver);
-
-MODULE_DESCRIPTION("Broadcom STB SATA PHY driver");
-MODULE_LICENSE("GPL");
-MODULE_AUTHOR("Marc Carino");
-MODULE_AUTHOR("Brian Norris");
-MODULE_ALIAS("platform:phy-brcmstb-sata");
diff --git a/drivers/phy/phy-exynos-mipi-video.c b/drivers/phy/phy-exynos-mipi-video.c
index 2a54cab..cc093eb 100644
--- a/drivers/phy/phy-exynos-mipi-video.c
+++ b/drivers/phy/phy-exynos-mipi-video.c
@@ -1,7 +1,7 @@
/*
* Samsung S5P/EXYNOS SoC series MIPI CSIS/DSIM DPHY driver
*
- * Copyright (C) 2013 Samsung Electronics Co., Ltd.
+ * Copyright (C) 2013,2016 Samsung Electronics Co., Ltd.
* Author: Sylwester Nawrocki <s.nawrocki@samsung.com>
*
* This program is free software; you can redistribute it and/or modify
@@ -13,96 +13,276 @@
#include <linux/io.h>
#include <linux/kernel.h>
#include <linux/mfd/syscon/exynos4-pmu.h>
+#include <linux/mfd/syscon/exynos5-pmu.h>
#include <linux/module.h>
#include <linux/of.h>
#include <linux/of_address.h>
+#include <linux/of_device.h>
#include <linux/phy/phy.h>
-#include <linux/platform_device.h>
#include <linux/regmap.h>
#include <linux/spinlock.h>
#include <linux/mfd/syscon.h>
-/* MIPI_PHYn_CONTROL reg. offset (for base address from ioremap): n = 0..1 */
-#define EXYNOS_MIPI_PHY_CONTROL(n) ((n) * 4)
-
enum exynos_mipi_phy_id {
+ EXYNOS_MIPI_PHY_ID_NONE = -1,
EXYNOS_MIPI_PHY_ID_CSIS0,
EXYNOS_MIPI_PHY_ID_DSIM0,
EXYNOS_MIPI_PHY_ID_CSIS1,
EXYNOS_MIPI_PHY_ID_DSIM1,
+ EXYNOS_MIPI_PHY_ID_CSIS2,
EXYNOS_MIPI_PHYS_NUM
};
-#define is_mipi_dsim_phy_id(id) \
- ((id) == EXYNOS_MIPI_PHY_ID_DSIM0 || (id) == EXYNOS_MIPI_PHY_ID_DSIM1)
+enum exynos_mipi_phy_regmap_id {
+ EXYNOS_MIPI_REGMAP_PMU,
+ EXYNOS_MIPI_REGMAP_DISP,
+ EXYNOS_MIPI_REGMAP_CAM0,
+ EXYNOS_MIPI_REGMAP_CAM1,
+ EXYNOS_MIPI_REGMAPS_NUM
+};
+
+struct mipi_phy_device_desc {
+ int num_phys;
+ int num_regmaps;
+ const char *regmap_names[EXYNOS_MIPI_REGMAPS_NUM];
+ struct exynos_mipi_phy_desc {
+ enum exynos_mipi_phy_id coupled_phy_id;
+ u32 enable_val;
+ unsigned int enable_reg;
+ enum exynos_mipi_phy_regmap_id enable_map;
+ u32 resetn_val;
+ unsigned int resetn_reg;
+ enum exynos_mipi_phy_regmap_id resetn_map;
+ } phys[EXYNOS_MIPI_PHYS_NUM];
+};
+
+static const struct mipi_phy_device_desc s5pv210_mipi_phy = {
+ .num_regmaps = 1,
+ .regmap_names = {"syscon"},
+ .num_phys = 4,
+ .phys = {
+ {
+ /* EXYNOS_MIPI_PHY_ID_CSIS0 */
+ .coupled_phy_id = EXYNOS_MIPI_PHY_ID_DSIM0,
+ .enable_val = EXYNOS4_MIPI_PHY_ENABLE,
+ .enable_reg = EXYNOS4_MIPI_PHY_CONTROL(0),
+ .enable_map = EXYNOS_MIPI_REGMAP_PMU,
+ .resetn_val = EXYNOS4_MIPI_PHY_SRESETN,
+ .resetn_reg = EXYNOS4_MIPI_PHY_CONTROL(0),
+ .resetn_map = EXYNOS_MIPI_REGMAP_PMU,
+ }, {
+ /* EXYNOS_MIPI_PHY_ID_DSIM0 */
+ .coupled_phy_id = EXYNOS_MIPI_PHY_ID_CSIS0,
+ .enable_val = EXYNOS4_MIPI_PHY_ENABLE,
+ .enable_reg = EXYNOS4_MIPI_PHY_CONTROL(0),
+ .enable_map = EXYNOS_MIPI_REGMAP_PMU,
+ .resetn_val = EXYNOS4_MIPI_PHY_MRESETN,
+ .resetn_reg = EXYNOS4_MIPI_PHY_CONTROL(0),
+ .resetn_map = EXYNOS_MIPI_REGMAP_PMU,
+ }, {
+ /* EXYNOS_MIPI_PHY_ID_CSIS1 */
+ .coupled_phy_id = EXYNOS_MIPI_PHY_ID_DSIM1,
+ .enable_val = EXYNOS4_MIPI_PHY_ENABLE,
+ .enable_reg = EXYNOS4_MIPI_PHY_CONTROL(1),
+ .enable_map = EXYNOS_MIPI_REGMAP_PMU,
+ .resetn_val = EXYNOS4_MIPI_PHY_SRESETN,
+ .resetn_reg = EXYNOS4_MIPI_PHY_CONTROL(1),
+ .resetn_map = EXYNOS_MIPI_REGMAP_PMU,
+ }, {
+ /* EXYNOS_MIPI_PHY_ID_DSIM1 */
+ .coupled_phy_id = EXYNOS_MIPI_PHY_ID_CSIS1,
+ .enable_val = EXYNOS4_MIPI_PHY_ENABLE,
+ .enable_reg = EXYNOS4_MIPI_PHY_CONTROL(1),
+ .enable_map = EXYNOS_MIPI_REGMAP_PMU,
+ .resetn_val = EXYNOS4_MIPI_PHY_MRESETN,
+ .resetn_reg = EXYNOS4_MIPI_PHY_CONTROL(1),
+ .resetn_map = EXYNOS_MIPI_REGMAP_PMU,
+ },
+ },
+};
+
+static const struct mipi_phy_device_desc exynos5420_mipi_phy = {
+ .num_regmaps = 1,
+ .regmap_names = {"syscon"},
+ .num_phys = 5,
+ .phys = {
+ {
+ /* EXYNOS_MIPI_PHY_ID_CSIS0 */
+ .coupled_phy_id = EXYNOS_MIPI_PHY_ID_DSIM0,
+ .enable_val = EXYNOS5_PHY_ENABLE,
+ .enable_reg = EXYNOS5420_MIPI_PHY0_CONTROL,
+ .enable_map = EXYNOS_MIPI_REGMAP_PMU,
+ .resetn_val = EXYNOS5_MIPI_PHY_S_RESETN,
+ .resetn_reg = EXYNOS5420_MIPI_PHY0_CONTROL,
+ .resetn_map = EXYNOS_MIPI_REGMAP_PMU,
+ }, {
+ /* EXYNOS_MIPI_PHY_ID_DSIM0 */
+ .coupled_phy_id = EXYNOS_MIPI_PHY_ID_CSIS0,
+ .enable_val = EXYNOS5_PHY_ENABLE,
+ .enable_reg = EXYNOS5420_MIPI_PHY0_CONTROL,
+ .enable_map = EXYNOS_MIPI_REGMAP_PMU,
+ .resetn_val = EXYNOS5_MIPI_PHY_M_RESETN,
+ .resetn_reg = EXYNOS5420_MIPI_PHY0_CONTROL,
+ .resetn_map = EXYNOS_MIPI_REGMAP_PMU,
+ }, {
+ /* EXYNOS_MIPI_PHY_ID_CSIS1 */
+ .coupled_phy_id = EXYNOS_MIPI_PHY_ID_DSIM1,
+ .enable_val = EXYNOS5_PHY_ENABLE,
+ .enable_reg = EXYNOS5420_MIPI_PHY1_CONTROL,
+ .enable_map = EXYNOS_MIPI_REGMAP_PMU,
+ .resetn_val = EXYNOS5_MIPI_PHY_S_RESETN,
+ .resetn_reg = EXYNOS5420_MIPI_PHY1_CONTROL,
+ .resetn_map = EXYNOS_MIPI_REGMAP_PMU,
+ }, {
+ /* EXYNOS_MIPI_PHY_ID_DSIM1 */
+ .coupled_phy_id = EXYNOS_MIPI_PHY_ID_CSIS1,
+ .enable_val = EXYNOS5_PHY_ENABLE,
+ .enable_reg = EXYNOS5420_MIPI_PHY1_CONTROL,
+ .enable_map = EXYNOS_MIPI_REGMAP_PMU,
+ .resetn_val = EXYNOS5_MIPI_PHY_M_RESETN,
+ .resetn_reg = EXYNOS5420_MIPI_PHY1_CONTROL,
+ .resetn_map = EXYNOS_MIPI_REGMAP_PMU,
+ }, {
+ /* EXYNOS_MIPI_PHY_ID_CSIS2 */
+ .coupled_phy_id = EXYNOS_MIPI_PHY_ID_NONE,
+ .enable_val = EXYNOS5_PHY_ENABLE,
+ .enable_reg = EXYNOS5420_MIPI_PHY2_CONTROL,
+ .enable_map = EXYNOS_MIPI_REGMAP_PMU,
+ .resetn_val = EXYNOS5_MIPI_PHY_S_RESETN,
+ .resetn_reg = EXYNOS5420_MIPI_PHY2_CONTROL,
+ .resetn_map = EXYNOS_MIPI_REGMAP_PMU,
+ },
+ },
+};
+
+#define EXYNOS5433_SYSREG_DISP_MIPI_PHY 0x100C
+#define EXYNOS5433_SYSREG_CAM0_MIPI_DPHY_CON 0x1014
+#define EXYNOS5433_SYSREG_CAM1_MIPI_DPHY_CON 0x1020
+
+static const struct mipi_phy_device_desc exynos5433_mipi_phy = {
+ .num_regmaps = 4,
+ .regmap_names = {
+ "samsung,pmu-syscon",
+ "samsung,disp-sysreg",
+ "samsung,cam0-sysreg",
+ "samsung,cam1-sysreg"
+ },
+ .num_phys = 5,
+ .phys = {
+ {
+ /* EXYNOS_MIPI_PHY_ID_CSIS0 */
+ .coupled_phy_id = EXYNOS_MIPI_PHY_ID_DSIM0,
+ .enable_val = EXYNOS5_PHY_ENABLE,
+ .enable_reg = EXYNOS5433_MIPI_PHY0_CONTROL,
+ .enable_map = EXYNOS_MIPI_REGMAP_PMU,
+ .resetn_val = BIT(0),
+ .resetn_reg = EXYNOS5433_SYSREG_CAM0_MIPI_DPHY_CON,
+ .resetn_map = EXYNOS_MIPI_REGMAP_CAM0,
+ }, {
+ /* EXYNOS_MIPI_PHY_ID_DSIM0 */
+ .coupled_phy_id = EXYNOS_MIPI_PHY_ID_CSIS0,
+ .enable_val = EXYNOS5_PHY_ENABLE,
+ .enable_reg = EXYNOS5433_MIPI_PHY0_CONTROL,
+ .enable_map = EXYNOS_MIPI_REGMAP_PMU,
+ .resetn_val = BIT(0),
+ .resetn_reg = EXYNOS5433_SYSREG_DISP_MIPI_PHY,
+ .resetn_map = EXYNOS_MIPI_REGMAP_DISP,
+ }, {
+ /* EXYNOS_MIPI_PHY_ID_CSIS1 */
+ .coupled_phy_id = EXYNOS_MIPI_PHY_ID_NONE,
+ .enable_val = EXYNOS5_PHY_ENABLE,
+ .enable_reg = EXYNOS5433_MIPI_PHY1_CONTROL,
+ .enable_map = EXYNOS_MIPI_REGMAP_PMU,
+ .resetn_val = BIT(1),
+ .resetn_reg = EXYNOS5433_SYSREG_CAM0_MIPI_DPHY_CON,
+ .resetn_map = EXYNOS_MIPI_REGMAP_CAM0,
+ }, {
+ /* EXYNOS_MIPI_PHY_ID_DSIM1 */
+ .coupled_phy_id = EXYNOS_MIPI_PHY_ID_NONE,
+ .enable_val = EXYNOS5_PHY_ENABLE,
+ .enable_reg = EXYNOS5433_MIPI_PHY1_CONTROL,
+ .enable_map = EXYNOS_MIPI_REGMAP_PMU,
+ .resetn_val = BIT(1),
+ .resetn_reg = EXYNOS5433_SYSREG_DISP_MIPI_PHY,
+ .resetn_map = EXYNOS_MIPI_REGMAP_DISP,
+ }, {
+ /* EXYNOS_MIPI_PHY_ID_CSIS2 */
+ .coupled_phy_id = EXYNOS_MIPI_PHY_ID_NONE,
+ .enable_val = EXYNOS5_PHY_ENABLE,
+ .enable_reg = EXYNOS5433_MIPI_PHY2_CONTROL,
+ .enable_map = EXYNOS_MIPI_REGMAP_PMU,
+ .resetn_val = BIT(0),
+ .resetn_reg = EXYNOS5433_SYSREG_CAM1_MIPI_DPHY_CON,
+ .resetn_map = EXYNOS_MIPI_REGMAP_CAM1,
+ },
+ },
+};
struct exynos_mipi_video_phy {
+ struct regmap *regmaps[EXYNOS_MIPI_REGMAPS_NUM];
+ int num_phys;
struct video_phy_desc {
struct phy *phy;
unsigned int index;
+ const struct exynos_mipi_phy_desc *data;
} phys[EXYNOS_MIPI_PHYS_NUM];
spinlock_t slock;
- void __iomem *regs;
- struct regmap *regmap;
};
-static int __set_phy_state(struct exynos_mipi_video_phy *state,
- enum exynos_mipi_phy_id id, unsigned int on)
+static inline int __is_running(const struct exynos_mipi_phy_desc *data,
+ struct exynos_mipi_video_phy *state)
{
- const unsigned int offset = EXYNOS4_MIPI_PHY_CONTROL(id / 2);
- void __iomem *addr;
- u32 val, reset;
+ u32 val;
- if (is_mipi_dsim_phy_id(id))
- reset = EXYNOS4_MIPI_PHY_MRESETN;
- else
- reset = EXYNOS4_MIPI_PHY_SRESETN;
+ regmap_read(state->regmaps[data->resetn_map], data->resetn_reg, &val);
+ return val & data->resetn_val;
+}
+
+static int __set_phy_state(const struct exynos_mipi_phy_desc *data,
+ struct exynos_mipi_video_phy *state, unsigned int on)
+{
+ u32 val;
spin_lock(&state->slock);
- if (!IS_ERR(state->regmap)) {
- regmap_read(state->regmap, offset, &val);
- if (on)
- val |= reset;
- else
- val &= ~reset;
- regmap_write(state->regmap, offset, val);
- if (on)
- val |= EXYNOS4_MIPI_PHY_ENABLE;
- else if (!(val & EXYNOS4_MIPI_PHY_RESET_MASK))
- val &= ~EXYNOS4_MIPI_PHY_ENABLE;
- regmap_write(state->regmap, offset, val);
- } else {
- addr = state->regs + EXYNOS_MIPI_PHY_CONTROL(id / 2);
+ /* disable in PMU sysreg */
+ if (!on && data->coupled_phy_id >= 0 &&
+ !__is_running(state->phys[data->coupled_phy_id].data, state)) {
+ regmap_read(state->regmaps[data->enable_map], data->enable_reg,
+ &val);
+ val &= ~data->enable_val;
+ regmap_write(state->regmaps[data->enable_map], data->enable_reg,
+ val);
+ }
- val = readl(addr);
- if (on)
- val |= reset;
- else
- val &= ~reset;
- writel(val, addr);
- /* Clear ENABLE bit only if MRESETN, SRESETN bits are not set */
- if (on)
- val |= EXYNOS4_MIPI_PHY_ENABLE;
- else if (!(val & EXYNOS4_MIPI_PHY_RESET_MASK))
- val &= ~EXYNOS4_MIPI_PHY_ENABLE;
+ /* PHY reset */
+ regmap_read(state->regmaps[data->resetn_map], data->resetn_reg, &val);
+ val = on ? (val | data->resetn_val) : (val & ~data->resetn_val);
+ regmap_write(state->regmaps[data->resetn_map], data->resetn_reg, val);
- writel(val, addr);
+ /* enable in PMU sysreg */
+ if (on) {
+ regmap_read(state->regmaps[data->enable_map], data->enable_reg,
+ &val);
+ val |= data->enable_val;
+ regmap_write(state->regmaps[data->enable_map], data->enable_reg,
+ val);
}
spin_unlock(&state->slock);
+
return 0;
}
#define to_mipi_video_phy(desc) \
- container_of((desc), struct exynos_mipi_video_phy, phys[(desc)->index]);
+ container_of((desc), struct exynos_mipi_video_phy, phys[(desc)->index])
static int exynos_mipi_video_phy_power_on(struct phy *phy)
{
struct video_phy_desc *phy_desc = phy_get_drvdata(phy);
struct exynos_mipi_video_phy *state = to_mipi_video_phy(phy_desc);
- return __set_phy_state(state, phy_desc->index, 1);
+ return __set_phy_state(phy_desc->data, state, 1);
}
static int exynos_mipi_video_phy_power_off(struct phy *phy)
@@ -110,7 +290,7 @@
struct video_phy_desc *phy_desc = phy_get_drvdata(phy);
struct exynos_mipi_video_phy *state = to_mipi_video_phy(phy_desc);
- return __set_phy_state(state, phy_desc->index, 0);
+ return __set_phy_state(phy_desc->data, state, 0);
}
static struct phy *exynos_mipi_video_phy_xlate(struct device *dev,
@@ -118,7 +298,7 @@
{
struct exynos_mipi_video_phy *state = dev_get_drvdata(dev);
- if (WARN_ON(args->args[0] >= EXYNOS_MIPI_PHYS_NUM))
+ if (WARN_ON(args->args[0] >= state->num_phys))
return ERR_PTR(-ENODEV);
return state->phys[args->args[0]].phy;
@@ -132,32 +312,33 @@
static int exynos_mipi_video_phy_probe(struct platform_device *pdev)
{
+ const struct mipi_phy_device_desc *phy_dev;
struct exynos_mipi_video_phy *state;
struct device *dev = &pdev->dev;
+ struct device_node *np = dev->of_node;
struct phy_provider *phy_provider;
unsigned int i;
+ phy_dev = of_device_get_match_data(dev);
+ if (!phy_dev)
+ return -ENODEV;
+
state = devm_kzalloc(dev, sizeof(*state), GFP_KERNEL);
if (!state)
return -ENOMEM;
- state->regmap = syscon_regmap_lookup_by_phandle(dev->of_node, "syscon");
- if (IS_ERR(state->regmap)) {
- struct resource *res;
-
- dev_info(dev, "regmap lookup failed: %ld\n",
- PTR_ERR(state->regmap));
-
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- state->regs = devm_ioremap_resource(dev, res);
- if (IS_ERR(state->regs))
- return PTR_ERR(state->regs);
+ for (i = 0; i < phy_dev->num_regmaps; i++) {
+ state->regmaps[i] = syscon_regmap_lookup_by_phandle(np,
+ phy_dev->regmap_names[i]);
+ if (IS_ERR(state->regmaps[i]))
+ return PTR_ERR(state->regmaps[i]);
}
-
- dev_set_drvdata(dev, state);
+ state->num_phys = phy_dev->num_phys;
spin_lock_init(&state->slock);
- for (i = 0; i < EXYNOS_MIPI_PHYS_NUM; i++) {
+ dev_set_drvdata(dev, state);
+
+ for (i = 0; i < state->num_phys; i++) {
struct phy *phy = devm_phy_create(dev, NULL,
&exynos_mipi_video_phy_ops);
if (IS_ERR(phy)) {
@@ -167,6 +348,7 @@
state->phys[i].phy = phy;
state->phys[i].index = i;
+ state->phys[i].data = &phy_dev->phys[i];
phy_set_drvdata(phy, &state->phys[i]);
}
@@ -177,8 +359,17 @@
}
static const struct of_device_id exynos_mipi_video_phy_of_match[] = {
- { .compatible = "samsung,s5pv210-mipi-video-phy" },
- { },
+ {
+ .compatible = "samsung,s5pv210-mipi-video-phy",
+ .data = &s5pv210_mipi_phy,
+ }, {
+ .compatible = "samsung,exynos5420-mipi-video-phy",
+ .data = &exynos5420_mipi_phy,
+ }, {
+ .compatible = "samsung,exynos5433-mipi-video-phy",
+ .data = &exynos5433_mipi_phy,
+ },
+ { /* sentinel */ },
};
MODULE_DEVICE_TABLE(of, exynos_mipi_video_phy_of_match);
diff --git a/drivers/phy/phy-mt65xx-usb3.c b/drivers/phy/phy-mt65xx-usb3.c
index c0e7b4b..4d85e73 100644
--- a/drivers/phy/phy-mt65xx-usb3.c
+++ b/drivers/phy/phy-mt65xx-usb3.c
@@ -134,6 +134,11 @@
#define U3P_SR_COEF_DIVISOR 1000
#define U3P_FM_DET_CYCLE_CNT 1024
+struct mt65xx_phy_pdata {
+ /* avoid RX sensitivity level degradation only for mt8173 */
+ bool avoid_rx_sen_degradation;
+};
+
struct mt65xx_phy_instance {
struct phy *phy;
void __iomem *port_base;
@@ -145,6 +150,7 @@
struct device *dev;
void __iomem *sif_base; /* include sif2, but exclude port's */
struct clk *u3phya_ref; /* reference clock of usb3 anolog phy */
+ const struct mt65xx_phy_pdata *pdata;
struct mt65xx_phy_instance **phys;
int nphys;
};
@@ -241,22 +247,26 @@
tmp = readl(port_base + U3P_U2PHYACR4);
tmp &= ~P2C_U2_GPIO_CTR_MSK;
writel(tmp, port_base + U3P_U2PHYACR4);
+ }
- tmp = readl(port_base + U3P_USBPHYACR2);
- tmp |= PA2_RG_SIF_U2PLL_FORCE_EN;
- writel(tmp, port_base + U3P_USBPHYACR2);
+ if (u3phy->pdata->avoid_rx_sen_degradation) {
+ if (!index) {
+ tmp = readl(port_base + U3P_USBPHYACR2);
+ tmp |= PA2_RG_SIF_U2PLL_FORCE_EN;
+ writel(tmp, port_base + U3P_USBPHYACR2);
- tmp = readl(port_base + U3D_U2PHYDCR0);
- tmp &= ~P2C_RG_SIF_U2PLL_FORCE_ON;
- writel(tmp, port_base + U3D_U2PHYDCR0);
- } else {
- tmp = readl(port_base + U3D_U2PHYDCR0);
- tmp |= P2C_RG_SIF_U2PLL_FORCE_ON;
- writel(tmp, port_base + U3D_U2PHYDCR0);
+ tmp = readl(port_base + U3D_U2PHYDCR0);
+ tmp &= ~P2C_RG_SIF_U2PLL_FORCE_ON;
+ writel(tmp, port_base + U3D_U2PHYDCR0);
+ } else {
+ tmp = readl(port_base + U3D_U2PHYDCR0);
+ tmp |= P2C_RG_SIF_U2PLL_FORCE_ON;
+ writel(tmp, port_base + U3D_U2PHYDCR0);
- tmp = readl(port_base + U3P_U2PHYDTM0);
- tmp |= P2C_RG_SUSPENDM | P2C_FORCE_SUSPENDM;
- writel(tmp, port_base + U3P_U2PHYDTM0);
+ tmp = readl(port_base + U3P_U2PHYDTM0);
+ tmp |= P2C_RG_SUSPENDM | P2C_FORCE_SUSPENDM;
+ writel(tmp, port_base + U3P_U2PHYDTM0);
+ }
}
tmp = readl(port_base + U3P_USBPHYACR6);
@@ -318,7 +328,7 @@
tmp |= XC3_RG_U3_XTAL_RX_PWD | XC3_RG_U3_FRC_XTAL_RX_PWD;
writel(tmp, u3phy->sif_base + U3P_XTALCTL3);
- /* [mt8173]switch 100uA current to SSUSB */
+ /* switch 100uA current to SSUSB */
tmp = readl(port_base + U3P_USBPHYACR5);
tmp |= PA5_RG_U2_HS_100U_U3_EN;
writel(tmp, port_base + U3P_USBPHYACR5);
@@ -335,7 +345,7 @@
tmp |= PA5_RG_U2_HSTX_SRCTRL_VAL(4);
writel(tmp, port_base + U3P_USBPHYACR5);
- if (index) {
+ if (u3phy->pdata->avoid_rx_sen_degradation && index) {
tmp = readl(port_base + U3D_U2PHYDCR0);
tmp |= P2C_RG_SIF_U2PLL_FORCE_ON;
writel(tmp, port_base + U3D_U2PHYDCR0);
@@ -386,7 +396,9 @@
tmp = readl(port_base + U3P_U3_PHYA_REG0);
tmp &= ~P3A_RG_U3_VUSB10_ON;
writel(tmp, port_base + U3P_U3_PHYA_REG0);
- } else {
+ }
+
+ if (u3phy->pdata->avoid_rx_sen_degradation && index) {
tmp = readl(port_base + U3D_U2PHYDCR0);
tmp &= ~P2C_RG_SIF_U2PLL_FORCE_ON;
writel(tmp, port_base + U3D_U2PHYDCR0);
@@ -402,7 +414,7 @@
u32 index = instance->index;
u32 tmp;
- if (index) {
+ if (u3phy->pdata->avoid_rx_sen_degradation && index) {
tmp = readl(port_base + U3D_U2PHYDCR0);
tmp &= ~P2C_RG_SIF_U2PLL_FORCE_ON;
writel(tmp, port_base + U3D_U2PHYDCR0);
@@ -502,8 +514,24 @@
.owner = THIS_MODULE,
};
+static const struct mt65xx_phy_pdata mt2701_pdata = {
+ .avoid_rx_sen_degradation = false,
+};
+
+static const struct mt65xx_phy_pdata mt8173_pdata = {
+ .avoid_rx_sen_degradation = true,
+};
+
+static const struct of_device_id mt65xx_u3phy_id_table[] = {
+ { .compatible = "mediatek,mt2701-u3phy", .data = &mt2701_pdata },
+ { .compatible = "mediatek,mt8173-u3phy", .data = &mt8173_pdata },
+ { },
+};
+MODULE_DEVICE_TABLE(of, mt65xx_u3phy_id_table);
+
static int mt65xx_u3phy_probe(struct platform_device *pdev)
{
+ const struct of_device_id *match;
struct device *dev = &pdev->dev;
struct device_node *np = dev->of_node;
struct device_node *child_np;
@@ -513,10 +541,15 @@
struct resource res;
int port, retval;
+ match = of_match_node(mt65xx_u3phy_id_table, pdev->dev.of_node);
+ if (!match)
+ return -EINVAL;
+
u3phy = devm_kzalloc(dev, sizeof(*u3phy), GFP_KERNEL);
if (!u3phy)
return -ENOMEM;
+ u3phy->pdata = match->data;
u3phy->nphys = of_get_child_count(np);
u3phy->phys = devm_kcalloc(dev, u3phy->nphys,
sizeof(*u3phy->phys), GFP_KERNEL);
@@ -587,12 +620,6 @@
return retval;
}
-static const struct of_device_id mt65xx_u3phy_id_table[] = {
- { .compatible = "mediatek,mt8173-u3phy", },
- { },
-};
-MODULE_DEVICE_TABLE(of, mt65xx_u3phy_id_table);
-
static struct platform_driver mt65xx_u3phy_driver = {
.probe = mt65xx_u3phy_probe,
.driver = {
diff --git a/drivers/phy/phy-rcar-gen2.c b/drivers/phy/phy-rcar-gen2.c
index c7a0599..97d4dd6 100644
--- a/drivers/phy/phy-rcar-gen2.c
+++ b/drivers/phy/phy-rcar-gen2.c
@@ -195,6 +195,7 @@
{ .compatible = "renesas,usb-phy-r8a7790" },
{ .compatible = "renesas,usb-phy-r8a7791" },
{ .compatible = "renesas,usb-phy-r8a7794" },
+ { .compatible = "renesas,rcar-gen2-usb-phy" },
{ }
};
MODULE_DEVICE_TABLE(of, rcar_gen2_phy_match_table);
diff --git a/drivers/phy/phy-rcar-gen3-usb2.c b/drivers/phy/phy-rcar-gen3-usb2.c
index bc4f7dd..76bb88f 100644
--- a/drivers/phy/phy-rcar-gen3-usb2.c
+++ b/drivers/phy/phy-rcar-gen3-usb2.c
@@ -12,6 +12,7 @@
* published by the Free Software Foundation.
*/
+#include <linux/extcon.h>
#include <linux/interrupt.h>
#include <linux/io.h>
#include <linux/module.h>
@@ -19,6 +20,7 @@
#include <linux/of_address.h>
#include <linux/phy/phy.h>
#include <linux/platform_device.h>
+#include <linux/regulator/consumer.h>
/******* USB2.0 Host registers (original offset is +0x200) *******/
#define USB2_INT_ENABLE 0x000
@@ -74,20 +76,17 @@
#define USB2_ADPCTRL_IDPULLUP BIT(5) /* 1 = ID sampling is enabled */
#define USB2_ADPCTRL_DRVVBUS BIT(4)
-struct rcar_gen3_data {
- void __iomem *base;
- struct clk *clk;
-};
-
struct rcar_gen3_chan {
- struct rcar_gen3_data usb2;
+ void __iomem *base;
+ struct extcon_dev *extcon;
struct phy *phy;
+ struct regulator *vbus;
bool has_otg;
};
static void rcar_gen3_set_host_mode(struct rcar_gen3_chan *ch, int host)
{
- void __iomem *usb2_base = ch->usb2.base;
+ void __iomem *usb2_base = ch->base;
u32 val = readl(usb2_base + USB2_COMMCTRL);
dev_vdbg(&ch->phy->dev, "%s: %08x, %d\n", __func__, val, host);
@@ -100,7 +99,7 @@
static void rcar_gen3_set_linectrl(struct rcar_gen3_chan *ch, int dp, int dm)
{
- void __iomem *usb2_base = ch->usb2.base;
+ void __iomem *usb2_base = ch->base;
u32 val = readl(usb2_base + USB2_LINECTRL1);
dev_vdbg(&ch->phy->dev, "%s: %08x, %d, %d\n", __func__, val, dp, dm);
@@ -114,7 +113,7 @@
static void rcar_gen3_enable_vbus_ctrl(struct rcar_gen3_chan *ch, int vbus)
{
- void __iomem *usb2_base = ch->usb2.base;
+ void __iomem *usb2_base = ch->base;
u32 val = readl(usb2_base + USB2_ADPCTRL);
dev_vdbg(&ch->phy->dev, "%s: %08x, %d\n", __func__, val, vbus);
@@ -130,6 +129,9 @@
rcar_gen3_set_linectrl(ch, 1, 1);
rcar_gen3_set_host_mode(ch, 1);
rcar_gen3_enable_vbus_ctrl(ch, 1);
+
+ extcon_set_cable_state_(ch->extcon, EXTCON_USB_HOST, true);
+ extcon_set_cable_state_(ch->extcon, EXTCON_USB, false);
}
static void rcar_gen3_init_for_peri(struct rcar_gen3_chan *ch)
@@ -137,17 +139,20 @@
rcar_gen3_set_linectrl(ch, 0, 1);
rcar_gen3_set_host_mode(ch, 0);
rcar_gen3_enable_vbus_ctrl(ch, 0);
+
+ extcon_set_cable_state_(ch->extcon, EXTCON_USB_HOST, false);
+ extcon_set_cable_state_(ch->extcon, EXTCON_USB, true);
}
static bool rcar_gen3_check_vbus(struct rcar_gen3_chan *ch)
{
- return !!(readl(ch->usb2.base + USB2_ADPCTRL) &
+ return !!(readl(ch->base + USB2_ADPCTRL) &
USB2_ADPCTRL_OTGSESSVLD);
}
static bool rcar_gen3_check_id(struct rcar_gen3_chan *ch)
{
- return !!(readl(ch->usb2.base + USB2_ADPCTRL) & USB2_ADPCTRL_IDDIG);
+ return !!(readl(ch->base + USB2_ADPCTRL) & USB2_ADPCTRL_IDDIG);
}
static void rcar_gen3_device_recognition(struct rcar_gen3_chan *ch)
@@ -166,7 +171,7 @@
static void rcar_gen3_init_otg(struct rcar_gen3_chan *ch)
{
- void __iomem *usb2_base = ch->usb2.base;
+ void __iomem *usb2_base = ch->base;
u32 val;
val = readl(usb2_base + USB2_VBCTRL);
@@ -187,7 +192,7 @@
static int rcar_gen3_phy_usb2_init(struct phy *p)
{
struct rcar_gen3_chan *channel = phy_get_drvdata(p);
- void __iomem *usb2_base = channel->usb2.base;
+ void __iomem *usb2_base = channel->base;
/* Initialize USB2 part */
writel(USB2_INT_ENABLE_INIT, usb2_base + USB2_INT_ENABLE);
@@ -205,7 +210,7 @@
{
struct rcar_gen3_chan *channel = phy_get_drvdata(p);
- writel(0, channel->usb2.base + USB2_INT_ENABLE);
+ writel(0, channel->base + USB2_INT_ENABLE);
return 0;
}
@@ -213,8 +218,15 @@
static int rcar_gen3_phy_usb2_power_on(struct phy *p)
{
struct rcar_gen3_chan *channel = phy_get_drvdata(p);
- void __iomem *usb2_base = channel->usb2.base;
+ void __iomem *usb2_base = channel->base;
u32 val;
+ int ret;
+
+ if (channel->vbus) {
+ ret = regulator_enable(channel->vbus);
+ if (ret)
+ return ret;
+ }
val = readl(usb2_base + USB2_USBCTR);
val |= USB2_USBCTR_PLL_RST;
@@ -225,17 +237,29 @@
return 0;
}
+static int rcar_gen3_phy_usb2_power_off(struct phy *p)
+{
+ struct rcar_gen3_chan *channel = phy_get_drvdata(p);
+ int ret = 0;
+
+ if (channel->vbus)
+ ret = regulator_disable(channel->vbus);
+
+ return ret;
+}
+
static struct phy_ops rcar_gen3_phy_usb2_ops = {
.init = rcar_gen3_phy_usb2_init,
.exit = rcar_gen3_phy_usb2_exit,
.power_on = rcar_gen3_phy_usb2_power_on,
+ .power_off = rcar_gen3_phy_usb2_power_off,
.owner = THIS_MODULE,
};
static irqreturn_t rcar_gen3_phy_usb2_irq(int irq, void *_ch)
{
struct rcar_gen3_chan *ch = _ch;
- void __iomem *usb2_base = ch->usb2.base;
+ void __iomem *usb2_base = ch->base;
u32 status = readl(usb2_base + USB2_OBINTSTA);
irqreturn_t ret = IRQ_NONE;
@@ -251,10 +275,17 @@
static const struct of_device_id rcar_gen3_phy_usb2_match_table[] = {
{ .compatible = "renesas,usb2-phy-r8a7795" },
+ { .compatible = "renesas,rcar-gen3-usb2-phy" },
{ }
};
MODULE_DEVICE_TABLE(of, rcar_gen3_phy_usb2_match_table);
+static const unsigned int rcar_gen3_phy_cable[] = {
+ EXTCON_USB,
+ EXTCON_USB_HOST,
+ EXTCON_NONE,
+};
+
static int rcar_gen3_phy_usb2_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
@@ -273,18 +304,30 @@
return -ENOMEM;
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- channel->usb2.base = devm_ioremap_resource(dev, res);
- if (IS_ERR(channel->usb2.base))
- return PTR_ERR(channel->usb2.base);
+ channel->base = devm_ioremap_resource(dev, res);
+ if (IS_ERR(channel->base))
+ return PTR_ERR(channel->base);
/* call request_irq for OTG */
irq = platform_get_irq(pdev, 0);
if (irq >= 0) {
+ int ret;
+
irq = devm_request_irq(dev, irq, rcar_gen3_phy_usb2_irq,
IRQF_SHARED, dev_name(dev), channel);
if (irq < 0)
dev_err(dev, "No irq handler (%d)\n", irq);
channel->has_otg = true;
+ channel->extcon = devm_extcon_dev_allocate(dev,
+ rcar_gen3_phy_cable);
+ if (IS_ERR(channel->extcon))
+ return PTR_ERR(channel->extcon);
+
+ ret = devm_extcon_dev_register(dev, channel->extcon);
+ if (ret < 0) {
+ dev_err(dev, "Failed to register extcon\n");
+ return ret;
+ }
}
/* devm_phy_create() will call pm_runtime_enable(dev); */
@@ -294,6 +337,13 @@
return PTR_ERR(channel->phy);
}
+ channel->vbus = devm_regulator_get_optional(dev, "vbus");
+ if (IS_ERR(channel->vbus)) {
+ if (PTR_ERR(channel->vbus) == -EPROBE_DEFER)
+ return PTR_ERR(channel->vbus);
+ channel->vbus = NULL;
+ }
+
phy_set_drvdata(channel->phy, channel);
provider = devm_of_phy_provider_register(dev, of_phy_simple_xlate);
diff --git a/drivers/phy/phy-rockchip-usb.c b/drivers/phy/phy-rockchip-usb.c
index f62d899..d60b149 100644
--- a/drivers/phy/phy-rockchip-usb.c
+++ b/drivers/phy/phy-rockchip-usb.c
@@ -216,7 +216,7 @@
init.parent_names = &clk_name;
init.num_parents = 1;
} else {
- init.flags = CLK_IS_ROOT;
+ init.flags = 0;
init.parent_names = NULL;
init.num_parents = 0;
}
diff --git a/drivers/pinctrl/bcm/Kconfig b/drivers/pinctrl/bcm/Kconfig
index 2cc7438..c356223 100644
--- a/drivers/pinctrl/bcm/Kconfig
+++ b/drivers/pinctrl/bcm/Kconfig
@@ -86,3 +86,16 @@
The ChipcommonA GPIO controller support basic PINCONF functions such
as bias pull up, pull down, and drive strength configurations, when
these pins are muxed to GPIO.
+
+config PINCTRL_NS2_MUX
+ bool "Broadcom Northstar2 pinmux driver"
+ depends on OF
+ depends on ARCH_BCM_IPROC || COMPILE_TEST
+ select PINMUX
+ select GENERIC_PINCONF
+ default ARM64 && ARCH_BCM_IPROC
+ help
+ Say yes here to enable the Broadcom NS2 MUX driver.
+
+ The Broadcom Northstar2 IOMUX driver supports group based IOMUX
+ configuration.
diff --git a/drivers/pinctrl/bcm/Makefile b/drivers/pinctrl/bcm/Makefile
index 6148367d..3861a1c 100644
--- a/drivers/pinctrl/bcm/Makefile
+++ b/drivers/pinctrl/bcm/Makefile
@@ -5,3 +5,4 @@
obj-$(CONFIG_PINCTRL_IPROC_GPIO) += pinctrl-iproc-gpio.o
obj-$(CONFIG_PINCTRL_CYGNUS_MUX) += pinctrl-cygnus-mux.o
obj-$(CONFIG_PINCTRL_NSP_GPIO) += pinctrl-nsp-gpio.o
+obj-$(CONFIG_PINCTRL_NS2_MUX) += pinctrl-ns2-mux.o
diff --git a/drivers/pinctrl/bcm/pinctrl-bcm281xx.c b/drivers/pinctrl/bcm/pinctrl-bcm281xx.c
index c3c692e..582f6df 100644
--- a/drivers/pinctrl/bcm/pinctrl-bcm281xx.c
+++ b/drivers/pinctrl/bcm/pinctrl-bcm281xx.c
@@ -1024,7 +1024,7 @@
.get_group_pins = bcm281xx_pinctrl_get_group_pins,
.pin_dbg_show = bcm281xx_pinctrl_pin_dbg_show,
.dt_node_to_map = pinconf_generic_dt_node_to_map_pin,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
static int bcm281xx_pinctrl_get_fcns_count(struct pinctrl_dev *pctldev)
@@ -1422,9 +1422,7 @@
bcm281xx_pinctrl_desc.pins = bcm281xx_pinctrl.pins;
bcm281xx_pinctrl_desc.npins = bcm281xx_pinctrl.npins;
- pctl = pinctrl_register(&bcm281xx_pinctrl_desc,
- &pdev->dev,
- pdata);
+ pctl = devm_pinctrl_register(&pdev->dev, &bcm281xx_pinctrl_desc, pdata);
if (IS_ERR(pctl)) {
dev_err(&pdev->dev, "Failed to register pinctrl\n");
return PTR_ERR(pctl);
diff --git a/drivers/pinctrl/bcm/pinctrl-bcm2835.c b/drivers/pinctrl/bcm/pinctrl-bcm2835.c
index 08b1d93..fa77165 100644
--- a/drivers/pinctrl/bcm/pinctrl-bcm2835.c
+++ b/drivers/pinctrl/bcm/pinctrl-bcm2835.c
@@ -342,6 +342,18 @@
return bcm2835_gpio_get_bit(pc, GPLEV0, offset);
}
+static int bcm2835_gpio_get_direction(struct gpio_chip *chip, unsigned int offset)
+{
+ struct bcm2835_pinctrl *pc = gpiochip_get_data(chip);
+ enum bcm2835_fsel fsel = bcm2835_pinctrl_fsel_get(pc, offset);
+
+ /* Alternative function doesn't clearly provide a direction */
+ if (fsel > BCM2835_FSEL_GPIO_OUT)
+ return -EINVAL;
+
+ return (fsel == BCM2835_FSEL_GPIO_IN);
+}
+
static void bcm2835_gpio_set(struct gpio_chip *chip, unsigned offset, int value)
{
struct bcm2835_pinctrl *pc = gpiochip_get_data(chip);
@@ -370,6 +382,7 @@
.free = gpiochip_generic_free,
.direction_input = bcm2835_gpio_direction_input,
.direction_output = bcm2835_gpio_direction_output,
+ .get_direction = bcm2835_gpio_get_direction,
.get = bcm2835_gpio_get,
.set = bcm2835_gpio_set,
.to_irq = bcm2835_gpio_to_irq,
@@ -1027,7 +1040,7 @@
return err;
}
- pc->pctl_dev = pinctrl_register(&bcm2835_pinctrl_desc, dev, pc);
+ pc->pctl_dev = devm_pinctrl_register(dev, &bcm2835_pinctrl_desc, pc);
if (IS_ERR(pc->pctl_dev)) {
gpiochip_remove(&pc->gpio_chip);
return PTR_ERR(pc->pctl_dev);
@@ -1045,7 +1058,6 @@
{
struct bcm2835_pinctrl *pc = platform_get_drvdata(pdev);
- pinctrl_unregister(pc->pctl_dev);
gpiochip_remove(&pc->gpio_chip);
return 0;
diff --git a/drivers/pinctrl/bcm/pinctrl-cygnus-mux.c b/drivers/pinctrl/bcm/pinctrl-cygnus-mux.c
index 9728f3d..d31c957 100644
--- a/drivers/pinctrl/bcm/pinctrl-cygnus-mux.c
+++ b/drivers/pinctrl/bcm/pinctrl-cygnus-mux.c
@@ -737,7 +737,7 @@
.get_group_pins = cygnus_get_group_pins,
.pin_dbg_show = cygnus_pin_dbg_show,
.dt_node_to_map = pinconf_generic_dt_node_to_map_group,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
static int cygnus_get_functions_count(struct pinctrl_dev *pctrl_dev)
@@ -987,7 +987,7 @@
cygnus_pinctrl_desc.pins = pins;
cygnus_pinctrl_desc.npins = num_pins;
- pinctrl->pctl = pinctrl_register(&cygnus_pinctrl_desc, &pdev->dev,
+ pinctrl->pctl = devm_pinctrl_register(&pdev->dev, &cygnus_pinctrl_desc,
pinctrl);
if (IS_ERR(pinctrl->pctl)) {
dev_err(&pdev->dev, "unable to register Cygnus IOMUX pinctrl\n");
diff --git a/drivers/pinctrl/bcm/pinctrl-iproc-gpio.c b/drivers/pinctrl/bcm/pinctrl-iproc-gpio.c
index d530ab4..3670f5e 100644
--- a/drivers/pinctrl/bcm/pinctrl-iproc-gpio.c
+++ b/drivers/pinctrl/bcm/pinctrl-iproc-gpio.c
@@ -379,7 +379,7 @@
.get_groups_count = iproc_get_groups_count,
.get_group_name = iproc_get_group_name,
.dt_node_to_map = pinconf_generic_dt_node_to_map_pin,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
static int iproc_gpio_set_pull(struct iproc_gpio *chip, unsigned gpio,
@@ -623,7 +623,7 @@
pctldesc->npins = gc->ngpio;
pctldesc->confops = &iproc_pconf_ops;
- chip->pctl = pinctrl_register(pctldesc, chip->dev, chip);
+ chip->pctl = devm_pinctrl_register(chip->dev, pctldesc, chip);
if (IS_ERR(chip->pctl)) {
dev_err(chip->dev, "unable to register pinctrl device\n");
return PTR_ERR(chip->pctl);
@@ -632,11 +632,6 @@
return 0;
}
-static void iproc_gpio_unregister_pinconf(struct iproc_gpio *chip)
-{
- pinctrl_unregister(chip->pctl);
-}
-
static const struct of_device_id iproc_gpio_of_match[] = {
{ .compatible = "brcm,cygnus-ccm-gpio" },
{ .compatible = "brcm,cygnus-asiu-gpio" },
@@ -720,7 +715,7 @@
handle_simple_irq, IRQ_TYPE_NONE);
if (ret) {
dev_err(dev, "no GPIO irqchip\n");
- goto err_unregister_pinconf;
+ goto err_rm_gpiochip;
}
gpiochip_set_chained_irqchip(gc, &iproc_gpio_irq_chip, irq,
@@ -729,9 +724,6 @@
return 0;
-err_unregister_pinconf:
- iproc_gpio_unregister_pinconf(chip);
-
err_rm_gpiochip:
gpiochip_remove(gc);
diff --git a/drivers/pinctrl/bcm/pinctrl-ns2-mux.c b/drivers/pinctrl/bcm/pinctrl-ns2-mux.c
new file mode 100644
index 0000000..3fefd14
--- /dev/null
+++ b/drivers/pinctrl/bcm/pinctrl-ns2-mux.c
@@ -0,0 +1,1117 @@
+/* Copyright (C) 2016 Broadcom Corporation
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation version 2.
+ *
+ * This program is distributed "as is" WITHOUT ANY WARRANTY of any
+ * kind, whether express or implied; without even the implied warranty
+ * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * This file contains the Northstar2 IOMUX driver that supports group
+ * based PINMUX configuration. The PWM is functional only when the
+ * corresponding mfio pin group is selected as gpio.
+ */
+
+#include <linux/err.h>
+#include <linux/io.h>
+#include <linux/of.h>
+#include <linux/pinctrl/pinconf.h>
+#include <linux/pinctrl/pinconf-generic.h>
+#include <linux/pinctrl/pinctrl.h>
+#include <linux/pinctrl/pinmux.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+
+#include "../core.h"
+#include "../pinctrl-utils.h"
+
+#define NS2_NUM_IOMUX 19
+#define NS2_NUM_PWM_MUX 4
+
+#define NS2_PIN_MUX_BASE0 0x00
+#define NS2_PIN_MUX_BASE1 0x01
+#define NS2_PIN_CONF_BASE 0x02
+#define NS2_MUX_PAD_FUNC1_OFFSET 0x04
+
+#define NS2_PIN_SRC_MASK 0x01
+#define NS2_PIN_PULL_MASK 0x03
+#define NS2_PIN_DRIVE_STRENGTH_MASK 0x07
+
+#define NS2_PIN_PULL_UP 0x01
+#define NS2_PIN_PULL_DOWN 0x02
+
+#define NS2_PIN_INPUT_EN_MASK 0x01
+
+/*
+ * Northstar2 IOMUX register description
+ *
+ * @base: base address number
+ * @offset: register offset for mux configuration of a group
+ * @shift: bit shift for mux configuration of a group
+ * @mask: mask bits
+ * @alt: alternate function to set to
+ */
+struct ns2_mux {
+ unsigned int base;
+ unsigned int offset;
+ unsigned int shift;
+ unsigned int mask;
+ unsigned int alt;
+};
+
+/*
+ * Keep track of Northstar2 IOMUX configuration and prevent double
+ * configuration
+ *
+ * @ns2_mux: Northstar2 IOMUX register description
+ * @is_configured: flag to indicate whether a mux setting has already
+ * been configured
+ */
+struct ns2_mux_log {
+ struct ns2_mux mux;
+ bool is_configured;
+};
+
+/*
+ * Group based IOMUX configuration
+ *
+ * @name: name of the group
+ * @pins: array of pins used by this group
+ * @num_pins: total number of pins used by this group
+ * @mux: Northstar2 group based IOMUX configuration
+ */
+struct ns2_pin_group {
+ const char *name;
+ const unsigned int *pins;
+ const unsigned int num_pins;
+ const struct ns2_mux mux;
+};
+
+/*
+ * Northstar2 mux function and supported pin groups
+ *
+ * @name: name of the function
+ * @groups: array of groups that can be supported by this function
+ * @num_groups: total number of groups that can be supported by function
+ */
+struct ns2_pin_function {
+ const char *name;
+ const char * const *groups;
+ const unsigned int num_groups;
+};
+
+/*
+ * Northstar2 IOMUX pinctrl core
+ *
+ * @pctl: pointer to pinctrl_dev
+ * @dev: pointer to device
+ * @base0: first IOMUX register base
+ * @base1: second IOMUX register base
+ * @pinconf_base: configuration register base
+ * @groups: pointer to array of groups
+ * @num_groups: total number of groups
+ * @functions: pointer to array of functions
+ * @num_functions: total number of functions
+ * @mux_log: pointer to the array of mux logs
+ * @lock: lock to protect register access
+ */
+struct ns2_pinctrl {
+ struct pinctrl_dev *pctl;
+ struct device *dev;
+ void __iomem *base0;
+ void __iomem *base1;
+ void __iomem *pinconf_base;
+
+ const struct ns2_pin_group *groups;
+ unsigned int num_groups;
+
+ const struct ns2_pin_function *functions;
+ unsigned int num_functions;
+
+ struct ns2_mux_log *mux_log;
+
+ spinlock_t lock;
+};
+
+/*
+ * Pin configuration info
+ *
+ * @base: base address number
+ * @offset: register offset from base
+ * @src_shift: slew rate control bit shift in the register
+ * @input_en: input enable control bit shift
+ * @pull_shift: pull-up/pull-down control bit shift in the register
+ * @drive_shift: drive strength control bit shift in the register
+ */
+struct ns2_pinconf {
+ unsigned int base;
+ unsigned int offset;
+ unsigned int src_shift;
+ unsigned int input_en;
+ unsigned int pull_shift;
+ unsigned int drive_shift;
+};
+
+/*
+ * Description of a pin in Northstar2
+ *
+ * @pin: pin number
+ * @name: pin name
+ * @pin_conf: pin configuration structure
+ */
+struct ns2_pin {
+ unsigned int pin;
+ char *name;
+ struct ns2_pinconf pin_conf;
+};
+
+#define NS2_PIN_DESC(p, n, b, o, s, i, pu, d) \
+{ \
+ .pin = p, \
+ .name = n, \
+ .pin_conf = { \
+ .base = b, \
+ .offset = o, \
+ .src_shift = s, \
+ .input_en = i, \
+ .pull_shift = pu, \
+ .drive_shift = d, \
+ } \
+}
+
+/*
+ * List of pins in Northstar2
+ */
+static struct ns2_pin ns2_pins[] = {
+ NS2_PIN_DESC(0, "mfio_0", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(1, "mfio_1", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(2, "mfio_2", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(3, "mfio_3", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(4, "mfio_4", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(5, "mfio_5", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(6, "mfio_6", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(7, "mfio_7", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(8, "mfio_8", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(9, "mfio_9", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(10, "mfio_10", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(11, "mfio_11", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(12, "mfio_12", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(13, "mfio_13", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(14, "mfio_14", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(15, "mfio_15", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(16, "mfio_16", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(17, "mfio_17", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(18, "mfio_18", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(19, "mfio_19", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(20, "mfio_20", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(21, "mfio_21", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(22, "mfio_22", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(23, "mfio_23", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(24, "mfio_24", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(25, "mfio_25", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(26, "mfio_26", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(27, "mfio_27", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(28, "mfio_28", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(29, "mfio_29", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(30, "mfio_30", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(31, "mfio_31", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(32, "mfio_32", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(33, "mfio_33", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(34, "mfio_34", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(35, "mfio_35", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(36, "mfio_36", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(37, "mfio_37", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(38, "mfio_38", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(39, "mfio_39", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(40, "mfio_40", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(41, "mfio_41", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(42, "mfio_42", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(43, "mfio_43", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(44, "mfio_44", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(45, "mfio_45", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(46, "mfio_46", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(47, "mfio_47", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(48, "mfio_48", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(49, "mfio_49", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(50, "mfio_50", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(51, "mfio_51", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(52, "mfio_52", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(53, "mfio_53", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(54, "mfio_54", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(55, "mfio_55", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(56, "mfio_56", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(57, "mfio_57", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(58, "mfio_58", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(59, "mfio_59", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(60, "mfio_60", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(61, "mfio_61", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(62, "mfio_62", -1, 0, 0, 0, 0, 0),
+ NS2_PIN_DESC(63, "qspi_wp", 2, 0x0, 31, 30, 27, 24),
+ NS2_PIN_DESC(64, "qspi_hold", 2, 0x0, 23, 22, 19, 16),
+ NS2_PIN_DESC(65, "qspi_cs", 2, 0x0, 15, 14, 11, 8),
+ NS2_PIN_DESC(66, "qspi_sck", 2, 0x0, 7, 6, 3, 0),
+ NS2_PIN_DESC(67, "uart3_sin", 2, 0x04, 31, 30, 27, 24),
+ NS2_PIN_DESC(68, "uart3_sout", 2, 0x04, 23, 22, 19, 16),
+ NS2_PIN_DESC(69, "qspi_mosi", 2, 0x04, 15, 14, 11, 8),
+ NS2_PIN_DESC(70, "qspi_miso", 2, 0x04, 7, 6, 3, 0),
+ NS2_PIN_DESC(71, "spi0_fss", 2, 0x08, 31, 30, 27, 24),
+ NS2_PIN_DESC(72, "spi0_rxd", 2, 0x08, 23, 22, 19, 16),
+ NS2_PIN_DESC(73, "spi0_txd", 2, 0x08, 15, 14, 11, 8),
+ NS2_PIN_DESC(74, "spi0_sck", 2, 0x08, 7, 6, 3, 0),
+ NS2_PIN_DESC(75, "spi1_fss", 2, 0x0c, 31, 30, 27, 24),
+ NS2_PIN_DESC(76, "spi1_rxd", 2, 0x0c, 23, 22, 19, 16),
+ NS2_PIN_DESC(77, "spi1_txd", 2, 0x0c, 15, 14, 11, 8),
+ NS2_PIN_DESC(78, "spi1_sck", 2, 0x0c, 7, 6, 3, 0),
+ NS2_PIN_DESC(79, "sdio0_data7", 2, 0x10, 31, 30, 27, 24),
+ NS2_PIN_DESC(80, "sdio0_emmc_rst", 2, 0x10, 23, 22, 19, 16),
+ NS2_PIN_DESC(81, "sdio0_led_on", 2, 0x10, 15, 14, 11, 8),
+ NS2_PIN_DESC(82, "sdio0_wp", 2, 0x10, 7, 6, 3, 0),
+ NS2_PIN_DESC(83, "sdio0_data3", 2, 0x14, 31, 30, 27, 24),
+ NS2_PIN_DESC(84, "sdio0_data4", 2, 0x14, 23, 22, 19, 16),
+ NS2_PIN_DESC(85, "sdio0_data5", 2, 0x14, 15, 14, 11, 8),
+ NS2_PIN_DESC(86, "sdio0_data6", 2, 0x14, 7, 6, 3, 0),
+ NS2_PIN_DESC(87, "sdio0_cmd", 2, 0x18, 31, 30, 27, 24),
+ NS2_PIN_DESC(88, "sdio0_data0", 2, 0x18, 23, 22, 19, 16),
+ NS2_PIN_DESC(89, "sdio0_data1", 2, 0x18, 15, 14, 11, 8),
+ NS2_PIN_DESC(90, "sdio0_data2", 2, 0x18, 7, 6, 3, 0),
+ NS2_PIN_DESC(91, "sdio1_led_on", 2, 0x1c, 31, 30, 27, 24),
+ NS2_PIN_DESC(92, "sdio1_wp", 2, 0x1c, 23, 22, 19, 16),
+ NS2_PIN_DESC(93, "sdio0_cd_l", 2, 0x1c, 15, 14, 11, 8),
+ NS2_PIN_DESC(94, "sdio0_clk", 2, 0x1c, 7, 6, 3, 0),
+ NS2_PIN_DESC(95, "sdio1_data5", 2, 0x20, 31, 30, 27, 24),
+ NS2_PIN_DESC(96, "sdio1_data6", 2, 0x20, 23, 22, 19, 16),
+ NS2_PIN_DESC(97, "sdio1_data7", 2, 0x20, 15, 14, 11, 8),
+ NS2_PIN_DESC(98, "sdio1_emmc_rst", 2, 0x20, 7, 6, 3, 0),
+ NS2_PIN_DESC(99, "sdio1_data1", 2, 0x24, 31, 30, 27, 24),
+ NS2_PIN_DESC(100, "sdio1_data2", 2, 0x24, 23, 22, 19, 16),
+ NS2_PIN_DESC(101, "sdio1_data3", 2, 0x24, 15, 14, 11, 8),
+ NS2_PIN_DESC(102, "sdio1_data4", 2, 0x24, 7, 6, 3, 0),
+ NS2_PIN_DESC(103, "sdio1_cd_l", 2, 0x28, 31, 30, 27, 24),
+ NS2_PIN_DESC(104, "sdio1_clk", 2, 0x28, 23, 22, 19, 16),
+ NS2_PIN_DESC(105, "sdio1_cmd", 2, 0x28, 15, 14, 11, 8),
+ NS2_PIN_DESC(106, "sdio1_data0", 2, 0x28, 7, 6, 3, 0),
+ NS2_PIN_DESC(107, "ext_mdio_0", 2, 0x2c, 15, 14, 11, 8),
+ NS2_PIN_DESC(108, "ext_mdc_0", 2, 0x2c, 7, 6, 3, 0),
+ NS2_PIN_DESC(109, "usb3_p1_vbus_ppc", 2, 0x34, 31, 30, 27, 24),
+ NS2_PIN_DESC(110, "usb3_p1_overcurrent", 2, 0x34, 23, 22, 19, 16),
+ NS2_PIN_DESC(111, "usb3_p0_vbus_ppc", 2, 0x34, 15, 14, 11, 8),
+ NS2_PIN_DESC(112, "usb3_p0_overcurrent", 2, 0x34, 7, 6, 3, 0),
+ NS2_PIN_DESC(113, "usb2_presence_indication", 2, 0x38, 31, 30, 27, 24),
+ NS2_PIN_DESC(114, "usb2_vbus_present", 2, 0x38, 23, 22, 19, 16),
+ NS2_PIN_DESC(115, "usb2_vbus_ppc", 2, 0x38, 15, 14, 11, 8),
+ NS2_PIN_DESC(116, "usb2_overcurrent", 2, 0x38, 7, 6, 3, 0),
+ NS2_PIN_DESC(117, "sata_led1", 2, 0x3c, 15, 14, 11, 8),
+ NS2_PIN_DESC(118, "sata_led0", 2, 0x3c, 7, 6, 3, 0),
+};
+
+/*
+ * List of groups of pins
+ */
+
+static const unsigned int nand_pins[] = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
+ 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23};
+static const unsigned int nor_data_pins[] = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
+ 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25};
+
+static const unsigned int gpio_0_1_pins[] = {24, 25};
+static const unsigned int pwm_0_pins[] = {24};
+static const unsigned int pwm_1_pins[] = {25};
+
+static const unsigned int uart1_ext_clk_pins[] = {26};
+static const unsigned int nor_adv_pins[] = {26};
+
+static const unsigned int gpio_2_5_pins[] = {27, 28, 29, 30};
+static const unsigned int pcie_ab1_clk_wak_pins[] = {27, 28, 29, 30};
+static const unsigned int nor_addr_0_3_pins[] = {27, 28, 29, 30};
+static const unsigned int pwm_2_pins[] = {27};
+static const unsigned int pwm_3_pins[] = {28};
+
+static const unsigned int gpio_6_7_pins[] = {31, 32};
+static const unsigned int pcie_a3_clk_wak_pins[] = {31, 32};
+static const unsigned int nor_addr_4_5_pins[] = {31, 32};
+
+static const unsigned int gpio_8_9_pins[] = {33, 34};
+static const unsigned int pcie_b3_clk_wak_pins[] = {33, 34};
+static const unsigned int nor_addr_6_7_pins[] = {33, 34};
+
+static const unsigned int gpio_10_11_pins[] = {35, 36};
+static const unsigned int pcie_b2_clk_wak_pins[] = {35, 36};
+static const unsigned int nor_addr_8_9_pins[] = {35, 36};
+
+static const unsigned int gpio_12_13_pins[] = {37, 38};
+static const unsigned int pcie_a2_clk_wak_pins[] = {37, 38};
+static const unsigned int nor_addr_10_11_pins[] = {37, 38};
+
+static const unsigned int gpio_14_17_pins[] = {39, 40, 41, 42};
+static const unsigned int uart0_modem_pins[] = {39, 40, 41, 42};
+static const unsigned int nor_addr_12_15_pins[] = {39, 40, 41, 42};
+
+static const unsigned int gpio_18_19_pins[] = {43, 44};
+static const unsigned int uart0_rts_cts_pins[] = {43, 44};
+
+static const unsigned int gpio_20_21_pins[] = {45, 46};
+static const unsigned int uart0_in_out_pins[] = {45, 46};
+
+static const unsigned int gpio_22_23_pins[] = {47, 48};
+static const unsigned int uart1_dcd_dsr_pins[] = {47, 48};
+
+static const unsigned int gpio_24_25_pins[] = {49, 50};
+static const unsigned int uart1_ri_dtr_pins[] = {49, 50};
+
+static const unsigned int gpio_26_27_pins[] = {51, 52};
+static const unsigned int uart1_rts_cts_pins[] = {51, 52};
+
+static const unsigned int gpio_28_29_pins[] = {53, 54};
+static const unsigned int uart1_in_out_pins[] = {53, 54};
+
+static const unsigned int gpio_30_31_pins[] = {55, 56};
+static const unsigned int uart2_rts_cts_pins[] = {55, 56};
+
+#define NS2_PIN_GROUP(group_name, ba, off, sh, ma, al) \
+{ \
+ .name = __stringify(group_name) "_grp", \
+ .pins = group_name ## _pins, \
+ .num_pins = ARRAY_SIZE(group_name ## _pins), \
+ .mux = { \
+ .base = ba, \
+ .offset = off, \
+ .shift = sh, \
+ .mask = ma, \
+ .alt = al, \
+ } \
+}
+
+/*
+ * List of Northstar2 pin groups
+ */
+static const struct ns2_pin_group ns2_pin_groups[] = {
+ NS2_PIN_GROUP(nand, 0, 0, 31, 1, 0),
+ NS2_PIN_GROUP(nor_data, 0, 0, 31, 1, 1),
+ NS2_PIN_GROUP(gpio_0_1, 0, 0, 31, 1, 0),
+
+ NS2_PIN_GROUP(uart1_ext_clk, 0, 4, 30, 3, 1),
+ NS2_PIN_GROUP(nor_adv, 0, 4, 30, 3, 2),
+
+ NS2_PIN_GROUP(gpio_2_5, 0, 4, 28, 3, 0),
+ NS2_PIN_GROUP(pcie_ab1_clk_wak, 0, 4, 28, 3, 1),
+ NS2_PIN_GROUP(nor_addr_0_3, 0, 4, 28, 3, 2),
+
+ NS2_PIN_GROUP(gpio_6_7, 0, 4, 26, 3, 0),
+ NS2_PIN_GROUP(pcie_a3_clk_wak, 0, 4, 26, 3, 1),
+ NS2_PIN_GROUP(nor_addr_4_5, 0, 4, 26, 3, 2),
+
+ NS2_PIN_GROUP(gpio_8_9, 0, 4, 24, 3, 0),
+ NS2_PIN_GROUP(pcie_b3_clk_wak, 0, 4, 24, 3, 1),
+ NS2_PIN_GROUP(nor_addr_6_7, 0, 4, 24, 3, 2),
+
+ NS2_PIN_GROUP(gpio_10_11, 0, 4, 22, 3, 0),
+ NS2_PIN_GROUP(pcie_b2_clk_wak, 0, 4, 22, 3, 1),
+ NS2_PIN_GROUP(nor_addr_8_9, 0, 4, 22, 3, 2),
+
+ NS2_PIN_GROUP(gpio_12_13, 0, 4, 20, 3, 0),
+ NS2_PIN_GROUP(pcie_a2_clk_wak, 0, 4, 20, 3, 1),
+ NS2_PIN_GROUP(nor_addr_10_11, 0, 4, 20, 3, 2),
+
+ NS2_PIN_GROUP(gpio_14_17, 0, 4, 18, 3, 0),
+ NS2_PIN_GROUP(uart0_modem, 0, 4, 18, 3, 1),
+ NS2_PIN_GROUP(nor_addr_12_15, 0, 4, 18, 3, 2),
+
+ NS2_PIN_GROUP(gpio_18_19, 0, 4, 16, 3, 0),
+ NS2_PIN_GROUP(uart0_rts_cts, 0, 4, 16, 3, 1),
+
+ NS2_PIN_GROUP(gpio_20_21, 0, 4, 14, 3, 0),
+ NS2_PIN_GROUP(uart0_in_out, 0, 4, 14, 3, 1),
+
+ NS2_PIN_GROUP(gpio_22_23, 0, 4, 12, 3, 0),
+ NS2_PIN_GROUP(uart1_dcd_dsr, 0, 4, 12, 3, 1),
+
+ NS2_PIN_GROUP(gpio_24_25, 0, 4, 10, 3, 0),
+ NS2_PIN_GROUP(uart1_ri_dtr, 0, 4, 10, 3, 1),
+
+ NS2_PIN_GROUP(gpio_26_27, 0, 4, 8, 3, 0),
+ NS2_PIN_GROUP(uart1_rts_cts, 0, 4, 8, 3, 1),
+
+ NS2_PIN_GROUP(gpio_28_29, 0, 4, 6, 3, 0),
+ NS2_PIN_GROUP(uart1_in_out, 0, 4, 6, 3, 1),
+
+ NS2_PIN_GROUP(gpio_30_31, 0, 4, 4, 3, 0),
+ NS2_PIN_GROUP(uart2_rts_cts, 0, 4, 4, 3, 1),
+
+ NS2_PIN_GROUP(pwm_0, 1, 0, 0, 1, 1),
+ NS2_PIN_GROUP(pwm_1, 1, 0, 1, 1, 1),
+ NS2_PIN_GROUP(pwm_2, 1, 0, 2, 1, 1),
+ NS2_PIN_GROUP(pwm_3, 1, 0, 3, 1, 1),
+};
+
+/*
+ * List of groups supported by functions
+ */
+
+static const char * const nand_grps[] = {"nand_grp"};
+
+static const char * const nor_grps[] = {"nor_data_grp", "nor_adv_grp",
+ "nor_addr_0_3_grp", "nor_addr_4_5_grp", "nor_addr_6_7_grp",
+ "nor_addr_8_9_grp", "nor_addr_10_11_grp", "nor_addr_12_15_grp"};
+
+static const char * const gpio_grps[] = {"gpio_0_1_grp", "gpio_2_5_grp",
+ "gpio_6_7_grp", "gpio_8_9_grp", "gpio_10_11_grp", "gpio_12_13_grp",
+ "gpio_14_17_grp", "gpio_18_19_grp", "gpio_20_21_grp", "gpio_22_23_grp",
+ "gpio_24_25_grp", "gpio_26_27_grp", "gpio_28_29_grp",
+ "gpio_30_31_grp"};
+
+static const char * const pcie_grps[] = {"pcie_ab1_clk_wak_grp",
+ "pcie_a3_clk_wak_grp", "pcie_b3_clk_wak_grp", "pcie_b2_clk_wak_grp",
+ "pcie_a2_clk_wak_grp"};
+
+static const char * const uart0_grps[] = {"uart0_modem_grp",
+ "uart0_rts_cts_grp", "uart0_in_out_grp"};
+
+static const char * const uart1_grps[] = {"uart1_ext_clk_grp",
+ "uart1_dcd_dsr_grp", "uart1_ri_dtr_grp", "uart1_rts_cts_grp",
+ "uart1_in_out_grp"};
+
+static const char * const uart2_grps[] = {"uart2_rts_cts_grp"};
+
+static const char * const pwm_grps[] = {"pwm_0_grp", "pwm_1_grp",
+ "pwm_2_grp", "pwm_3_grp"};
+
+#define NS2_PIN_FUNCTION(func) \
+{ \
+ .name = #func, \
+ .groups = func ## _grps, \
+ .num_groups = ARRAY_SIZE(func ## _grps), \
+}
+
+/*
+ * List of supported functions
+ */
+static const struct ns2_pin_function ns2_pin_functions[] = {
+ NS2_PIN_FUNCTION(nand),
+ NS2_PIN_FUNCTION(nor),
+ NS2_PIN_FUNCTION(gpio),
+ NS2_PIN_FUNCTION(pcie),
+ NS2_PIN_FUNCTION(uart0),
+ NS2_PIN_FUNCTION(uart1),
+ NS2_PIN_FUNCTION(uart2),
+ NS2_PIN_FUNCTION(pwm),
+};
+
+static int ns2_get_groups_count(struct pinctrl_dev *pctrl_dev)
+{
+ struct ns2_pinctrl *pinctrl = pinctrl_dev_get_drvdata(pctrl_dev);
+
+ return pinctrl->num_groups;
+}
+
+static const char *ns2_get_group_name(struct pinctrl_dev *pctrl_dev,
+ unsigned int selector)
+{
+ struct ns2_pinctrl *pinctrl = pinctrl_dev_get_drvdata(pctrl_dev);
+
+ return pinctrl->groups[selector].name;
+}
+
+static int ns2_get_group_pins(struct pinctrl_dev *pctrl_dev,
+ unsigned int selector, const unsigned int **pins,
+ unsigned int *num_pins)
+{
+ struct ns2_pinctrl *pinctrl = pinctrl_dev_get_drvdata(pctrl_dev);
+
+ *pins = pinctrl->groups[selector].pins;
+ *num_pins = pinctrl->groups[selector].num_pins;
+
+ return 0;
+}
+
+static void ns2_pin_dbg_show(struct pinctrl_dev *pctrl_dev,
+ struct seq_file *s, unsigned int offset)
+{
+ seq_printf(s, " %s", dev_name(pctrl_dev->dev));
+}
+
+static struct pinctrl_ops ns2_pinctrl_ops = {
+ .get_groups_count = ns2_get_groups_count,
+ .get_group_name = ns2_get_group_name,
+ .get_group_pins = ns2_get_group_pins,
+ .pin_dbg_show = ns2_pin_dbg_show,
+ .dt_node_to_map = pinconf_generic_dt_node_to_map_pin,
+ .dt_free_map = pinctrl_utils_free_map,
+};
+
+static int ns2_get_functions_count(struct pinctrl_dev *pctrl_dev)
+{
+ struct ns2_pinctrl *pinctrl = pinctrl_dev_get_drvdata(pctrl_dev);
+
+ return pinctrl->num_functions;
+}
+
+static const char *ns2_get_function_name(struct pinctrl_dev *pctrl_dev,
+ unsigned int selector)
+{
+ struct ns2_pinctrl *pinctrl = pinctrl_dev_get_drvdata(pctrl_dev);
+
+ return pinctrl->functions[selector].name;
+}
+
+static int ns2_get_function_groups(struct pinctrl_dev *pctrl_dev,
+ unsigned int selector,
+ const char * const **groups,
+ unsigned int * const num_groups)
+{
+ struct ns2_pinctrl *pinctrl = pinctrl_dev_get_drvdata(pctrl_dev);
+
+ *groups = pinctrl->functions[selector].groups;
+ *num_groups = pinctrl->functions[selector].num_groups;
+
+ return 0;
+}
+
+static int ns2_pinmux_set(struct ns2_pinctrl *pinctrl,
+ const struct ns2_pin_function *func,
+ const struct ns2_pin_group *grp,
+ struct ns2_mux_log *mux_log)
+{
+ const struct ns2_mux *mux = &grp->mux;
+ int i;
+ u32 val, mask;
+ unsigned long flags;
+ void __iomem *base_address;
+
+ for (i = 0; i < NS2_NUM_IOMUX; i++) {
+ if ((mux->shift != mux_log[i].mux.shift) ||
+ (mux->base != mux_log[i].mux.base) ||
+ (mux->offset != mux_log[i].mux.offset))
+ continue;
+
+ /* if this is a new configuration, just do it! */
+ if (!mux_log[i].is_configured)
+ break;
+
+ /*
+ * IOMUX has been configured previously and one is trying to
+ * configure it to a different function
+ */
+ if (mux_log[i].mux.alt != mux->alt) {
+ dev_err(pinctrl->dev,
+ "double configuration error detected!\n");
+ dev_err(pinctrl->dev, "func:%s grp:%s\n",
+ func->name, grp->name);
+ return -EINVAL;
+ }
+
+ return 0;
+ }
+ if (i == NS2_NUM_IOMUX)
+ return -EINVAL;
+
+ mask = mux->mask;
+ mux_log[i].mux.alt = mux->alt;
+ mux_log[i].is_configured = true;
+
+ switch (mux->base) {
+ case NS2_PIN_MUX_BASE0:
+ base_address = pinctrl->base0;
+ break;
+
+ case NS2_PIN_MUX_BASE1:
+ base_address = pinctrl->base1;
+ break;
+
+ default:
+ return -EINVAL;
+ }
+
+ spin_lock_irqsave(&pinctrl->lock, flags);
+ val = readl(base_address + grp->mux.offset);
+ val &= ~(mask << grp->mux.shift);
+ val |= grp->mux.alt << grp->mux.shift;
+ writel(val, (base_address + grp->mux.offset));
+ spin_unlock_irqrestore(&pinctrl->lock, flags);
+
+ return 0;
+}
+
+static int ns2_pinmux_enable(struct pinctrl_dev *pctrl_dev,
+ unsigned int func_select, unsigned int grp_select)
+{
+ struct ns2_pinctrl *pinctrl = pinctrl_dev_get_drvdata(pctrl_dev);
+ const struct ns2_pin_function *func;
+ const struct ns2_pin_group *grp;
+
+ if (grp_select > pinctrl->num_groups ||
+ func_select > pinctrl->num_functions)
+ return -EINVAL;
+
+ func = &pinctrl->functions[func_select];
+ grp = &pinctrl->groups[grp_select];
+
+ dev_dbg(pctrl_dev->dev, "func:%u name:%s grp:%u name:%s\n",
+ func_select, func->name, grp_select, grp->name);
+
+ dev_dbg(pctrl_dev->dev, "offset:0x%08x shift:%u alt:%u\n",
+ grp->mux.offset, grp->mux.shift, grp->mux.alt);
+
+ return ns2_pinmux_set(pinctrl, func, grp, pinctrl->mux_log);
+}
+
+static int ns2_pin_set_enable(struct pinctrl_dev *pctrldev, unsigned int pin,
+ u16 enable)
+{
+ struct ns2_pinctrl *pinctrl = pinctrl_dev_get_drvdata(pctrldev);
+ struct ns2_pin *pin_data = pctrldev->desc->pins[pin].drv_data;
+ unsigned long flags;
+ u32 val;
+ void __iomem *base_address;
+
+ base_address = pinctrl->pinconf_base;
+ spin_lock_irqsave(&pinctrl->lock, flags);
+ val = readl(base_address + pin_data->pin_conf.offset);
+ val &= ~(NS2_PIN_SRC_MASK << pin_data->pin_conf.input_en);
+
+ if (!enable)
+ val |= NS2_PIN_INPUT_EN_MASK << pin_data->pin_conf.input_en;
+
+ writel(val, (base_address + pin_data->pin_conf.offset));
+ spin_unlock_irqrestore(&pinctrl->lock, flags);
+
+ dev_dbg(pctrldev->dev, "pin:%u set enable:%d\n", pin, enable);
+ return 0;
+}
+
+static int ns2_pin_get_enable(struct pinctrl_dev *pctrldev, unsigned int pin)
+{
+ struct ns2_pinctrl *pinctrl = pinctrl_dev_get_drvdata(pctrldev);
+ struct ns2_pin *pin_data = pctrldev->desc->pins[pin].drv_data;
+ unsigned long flags;
+ int enable;
+
+ spin_lock_irqsave(&pinctrl->lock, flags);
+ enable = readl(pinctrl->pinconf_base + pin_data->pin_conf.offset);
+ enable = (enable >> pin_data->pin_conf.input_en) &
+ NS2_PIN_INPUT_EN_MASK;
+ spin_unlock_irqrestore(&pinctrl->lock, flags);
+
+ if (!enable)
+ enable = NS2_PIN_INPUT_EN_MASK;
+ else
+ enable = 0;
+
+ dev_dbg(pctrldev->dev, "pin:%u get disable:%d\n", pin, enable);
+ return enable;
+}
+
+static int ns2_pin_set_slew(struct pinctrl_dev *pctrldev, unsigned int pin,
+ u16 slew)
+{
+ struct ns2_pinctrl *pinctrl = pinctrl_dev_get_drvdata(pctrldev);
+ struct ns2_pin *pin_data = pctrldev->desc->pins[pin].drv_data;
+ unsigned long flags;
+ u32 val;
+ void __iomem *base_address;
+
+ base_address = pinctrl->pinconf_base;
+ spin_lock_irqsave(&pinctrl->lock, flags);
+ val = readl(base_address + pin_data->pin_conf.offset);
+ val &= ~(NS2_PIN_SRC_MASK << pin_data->pin_conf.src_shift);
+
+ if (slew)
+ val |= NS2_PIN_SRC_MASK << pin_data->pin_conf.src_shift;
+
+ writel(val, (base_address + pin_data->pin_conf.offset));
+ spin_unlock_irqrestore(&pinctrl->lock, flags);
+
+ dev_dbg(pctrldev->dev, "pin:%u set slew:%d\n", pin, slew);
+ return 0;
+}
+
+static int ns2_pin_get_slew(struct pinctrl_dev *pctrldev, unsigned int pin,
+ u16 *slew)
+{
+ struct ns2_pinctrl *pinctrl = pinctrl_dev_get_drvdata(pctrldev);
+ struct ns2_pin *pin_data = pctrldev->desc->pins[pin].drv_data;
+ unsigned long flags;
+ u32 val;
+
+ spin_lock_irqsave(&pinctrl->lock, flags);
+ val = readl(pinctrl->pinconf_base + pin_data->pin_conf.offset);
+ *slew = (val >> pin_data->pin_conf.src_shift) & NS2_PIN_SRC_MASK;
+ spin_unlock_irqrestore(&pinctrl->lock, flags);
+
+ dev_dbg(pctrldev->dev, "pin:%u get slew:%d\n", pin, *slew);
+ return 0;
+}
+
+static int ns2_pin_set_pull(struct pinctrl_dev *pctrldev, unsigned int pin,
+ bool pull_up, bool pull_down)
+{
+ struct ns2_pinctrl *pinctrl = pinctrl_dev_get_drvdata(pctrldev);
+ struct ns2_pin *pin_data = pctrldev->desc->pins[pin].drv_data;
+ unsigned long flags;
+ u32 val;
+ void __iomem *base_address;
+
+ base_address = pinctrl->pinconf_base;
+ spin_lock_irqsave(&pinctrl->lock, flags);
+ val = readl(base_address + pin_data->pin_conf.offset);
+ val &= ~(NS2_PIN_PULL_MASK << pin_data->pin_conf.pull_shift);
+
+ if (pull_up == true)
+ val |= NS2_PIN_PULL_UP << pin_data->pin_conf.pull_shift;
+ if (pull_down == true)
+ val |= NS2_PIN_PULL_DOWN << pin_data->pin_conf.pull_shift;
+ writel(val, (base_address + pin_data->pin_conf.offset));
+ spin_unlock_irqrestore(&pinctrl->lock, flags);
+
+ dev_dbg(pctrldev->dev, "pin:%u set pullup:%d pulldown: %d\n",
+ pin, pull_up, pull_down);
+ return 0;
+}
+
+static void ns2_pin_get_pull(struct pinctrl_dev *pctrldev,
+ unsigned int pin, bool *pull_up,
+ bool *pull_down)
+{
+ struct ns2_pinctrl *pinctrl = pinctrl_dev_get_drvdata(pctrldev);
+ struct ns2_pin *pin_data = pctrldev->desc->pins[pin].drv_data;
+ unsigned long flags;
+ u32 val;
+
+ spin_lock_irqsave(&pinctrl->lock, flags);
+ val = readl(pinctrl->pinconf_base + pin_data->pin_conf.offset);
+ val = (val >> pin_data->pin_conf.pull_shift) & NS2_PIN_PULL_MASK;
+ *pull_up = false;
+ *pull_down = false;
+
+ if (val == NS2_PIN_PULL_UP)
+ *pull_up = true;
+
+ if (val == NS2_PIN_PULL_DOWN)
+ *pull_down = true;
+ spin_unlock_irqrestore(&pinctrl->lock, flags);
+}
+
+static int ns2_pin_set_strength(struct pinctrl_dev *pctrldev, unsigned int pin,
+ u16 strength)
+{
+ struct ns2_pinctrl *pinctrl = pinctrl_dev_get_drvdata(pctrldev);
+ struct ns2_pin *pin_data = pctrldev->desc->pins[pin].drv_data;
+ u32 val;
+ unsigned long flags;
+ void __iomem *base_address;
+
+ /* make sure drive strength is supported */
+ if (strength < 2 || strength > 16 || (strength % 2))
+ return -ENOTSUPP;
+
+ base_address = pinctrl->pinconf_base;
+ spin_lock_irqsave(&pinctrl->lock, flags);
+ val = readl(base_address + pin_data->pin_conf.offset);
+ val &= ~(NS2_PIN_DRIVE_STRENGTH_MASK << pin_data->pin_conf.drive_shift);
+ val |= ((strength / 2) - 1) << pin_data->pin_conf.drive_shift;
+ writel(val, (base_address + pin_data->pin_conf.offset));
+ spin_unlock_irqrestore(&pinctrl->lock, flags);
+
+ dev_dbg(pctrldev->dev, "pin:%u set drive strength:%d mA\n",
+ pin, strength);
+ return 0;
+}
+
+static int ns2_pin_get_strength(struct pinctrl_dev *pctrldev, unsigned int pin,
+ u16 *strength)
+{
+ struct ns2_pinctrl *pinctrl = pinctrl_dev_get_drvdata(pctrldev);
+ struct ns2_pin *pin_data = pctrldev->desc->pins[pin].drv_data;
+ u32 val;
+ unsigned long flags;
+
+ spin_lock_irqsave(&pinctrl->lock, flags);
+ val = readl(pinctrl->pinconf_base + pin_data->pin_conf.offset);
+ *strength = (val >> pin_data->pin_conf.drive_shift) &
+ NS2_PIN_DRIVE_STRENGTH_MASK;
+ *strength = (*strength + 1) * 2;
+ spin_unlock_irqrestore(&pinctrl->lock, flags);
+
+ dev_dbg(pctrldev->dev, "pin:%u get drive strength:%d mA\n",
+ pin, *strength);
+ return 0;
+}
+
+static int ns2_pin_config_get(struct pinctrl_dev *pctldev, unsigned int pin,
+ unsigned long *config)
+{
+ struct ns2_pin *pin_data = pctldev->desc->pins[pin].drv_data;
+ enum pin_config_param param = pinconf_to_config_param(*config);
+ bool pull_up, pull_down;
+ u16 arg = 0;
+ int ret;
+
+ if (pin_data->pin_conf.base == -1)
+ return -ENOTSUPP;
+
+ switch (param) {
+ case PIN_CONFIG_BIAS_DISABLE:
+ ns2_pin_get_pull(pctldev, pin, &pull_up, &pull_down);
+ if ((pull_up == false) && (pull_down == false))
+ return 0;
+ else
+ return -EINVAL;
+
+ case PIN_CONFIG_BIAS_PULL_UP:
+ ns2_pin_get_pull(pctldev, pin, &pull_up, &pull_down);
+ if (pull_up)
+ return 0;
+ else
+ return -EINVAL;
+
+ case PIN_CONFIG_BIAS_PULL_DOWN:
+ ns2_pin_get_pull(pctldev, pin, &pull_up, &pull_down);
+ if (pull_down)
+ return 0;
+ else
+ return -EINVAL;
+
+ case PIN_CONFIG_DRIVE_STRENGTH:
+ ret = ns2_pin_get_strength(pctldev, pin, &arg);
+ if (ret)
+ return ret;
+ *config = pinconf_to_config_packed(param, arg);
+ return 0;
+
+ case PIN_CONFIG_SLEW_RATE:
+ ret = ns2_pin_get_slew(pctldev, pin, &arg);
+ if (ret)
+ return ret;
+ *config = pinconf_to_config_packed(param, arg);
+ return 0;
+
+ case PIN_CONFIG_INPUT_ENABLE:
+ ret = ns2_pin_get_enable(pctldev, pin);
+ if (ret)
+ return 0;
+ else
+ return -EINVAL;
+
+ default:
+ return -ENOTSUPP;
+ }
+}
+
+static int ns2_pin_config_set(struct pinctrl_dev *pctrldev, unsigned int pin,
+ unsigned long *configs, unsigned int num_configs)
+{
+ struct ns2_pin *pin_data = pctrldev->desc->pins[pin].drv_data;
+ enum pin_config_param param;
+ unsigned int i;
+ u16 arg;
+ int ret = -ENOTSUPP;
+
+ if (pin_data->pin_conf.base == -1)
+ return -ENOTSUPP;
+
+ for (i = 0; i < num_configs; i++) {
+ param = pinconf_to_config_param(configs[i]);
+ arg = pinconf_to_config_argument(configs[i]);
+
+ switch (param) {
+ case PIN_CONFIG_BIAS_DISABLE:
+ ret = ns2_pin_set_pull(pctrldev, pin, false, false);
+ if (ret < 0)
+ goto out;
+ break;
+
+ case PIN_CONFIG_BIAS_PULL_UP:
+ ret = ns2_pin_set_pull(pctrldev, pin, true, false);
+ if (ret < 0)
+ goto out;
+ break;
+
+ case PIN_CONFIG_BIAS_PULL_DOWN:
+ ret = ns2_pin_set_pull(pctrldev, pin, false, true);
+ if (ret < 0)
+ goto out;
+ break;
+
+ case PIN_CONFIG_DRIVE_STRENGTH:
+ ret = ns2_pin_set_strength(pctrldev, pin, arg);
+ if (ret < 0)
+ goto out;
+ break;
+
+ case PIN_CONFIG_SLEW_RATE:
+ ret = ns2_pin_set_slew(pctrldev, pin, arg);
+ if (ret < 0)
+ goto out;
+ break;
+
+ case PIN_CONFIG_INPUT_ENABLE:
+ ret = ns2_pin_set_enable(pctrldev, pin, arg);
+ if (ret < 0)
+ goto out;
+ break;
+
+ default:
+ dev_err(pctrldev->dev, "invalid configuration\n");
+ return -ENOTSUPP;
+ }
+ }
+out:
+ return ret;
+}
+static struct pinmux_ops ns2_pinmux_ops = {
+ .get_functions_count = ns2_get_functions_count,
+ .get_function_name = ns2_get_function_name,
+ .get_function_groups = ns2_get_function_groups,
+ .set_mux = ns2_pinmux_enable,
+};
+
+static const struct pinconf_ops ns2_pinconf_ops = {
+ .is_generic = true,
+ .pin_config_get = ns2_pin_config_get,
+ .pin_config_set = ns2_pin_config_set,
+};
+
+static struct pinctrl_desc ns2_pinctrl_desc = {
+ .name = "ns2-pinmux",
+ .pctlops = &ns2_pinctrl_ops,
+ .pmxops = &ns2_pinmux_ops,
+ .confops = &ns2_pinconf_ops,
+};
+
+static int ns2_mux_log_init(struct ns2_pinctrl *pinctrl)
+{
+ struct ns2_mux_log *log;
+ unsigned int i;
+
+ pinctrl->mux_log = devm_kcalloc(pinctrl->dev, NS2_NUM_IOMUX,
+ sizeof(struct ns2_mux_log),
+ GFP_KERNEL);
+ if (!pinctrl->mux_log)
+ return -ENOMEM;
+
+ for (i = 0; i < NS2_NUM_IOMUX; i++)
+ pinctrl->mux_log[i].is_configured = false;
+ /* Group 0 uses bit 31 in the IOMUX_PAD_FUNCTION_0 register */
+ log = &pinctrl->mux_log[0];
+ log->mux.base = NS2_PIN_MUX_BASE0;
+ log->mux.offset = 0;
+ log->mux.shift = 31;
+ log->mux.alt = 0;
+
+ /*
+ * Groups 1 through 14 use two bits each in the
+ * IOMUX_PAD_FUNCTION_1 register starting with
+ * bit position 30.
+ */
+ for (i = 1; i < (NS2_NUM_IOMUX - NS2_NUM_PWM_MUX); i++) {
+ log = &pinctrl->mux_log[i];
+ log->mux.base = NS2_PIN_MUX_BASE0;
+ log->mux.offset = NS2_MUX_PAD_FUNC1_OFFSET;
+ log->mux.shift = 32 - (i * 2);
+ log->mux.alt = 0;
+ }
+
+ /*
+ * Groups 15 through 18 use one bit each in the
+ * AUX_SEL register.
+ */
+ for (i = 0; i < NS2_NUM_PWM_MUX; i++) {
+ log = &pinctrl->mux_log[(NS2_NUM_IOMUX - NS2_NUM_PWM_MUX) + i];
+ log->mux.base = NS2_PIN_MUX_BASE1;
+ log->mux.offset = 0;
+ log->mux.shift = i;
+ log->mux.alt = 0;
+ }
+ return 0;
+}
+
+static int ns2_pinmux_probe(struct platform_device *pdev)
+{
+ struct ns2_pinctrl *pinctrl;
+ struct resource *res;
+ int i, ret;
+ struct pinctrl_pin_desc *pins;
+ unsigned int num_pins = ARRAY_SIZE(ns2_pins);
+
+ pinctrl = devm_kzalloc(&pdev->dev, sizeof(*pinctrl), GFP_KERNEL);
+ if (!pinctrl)
+ return -ENOMEM;
+
+ pinctrl->dev = &pdev->dev;
+ platform_set_drvdata(pdev, pinctrl);
+ spin_lock_init(&pinctrl->lock);
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ pinctrl->base0 = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(pinctrl->base0)) {
+ dev_err(&pdev->dev, "unable to map I/O space\n");
+ return PTR_ERR(pinctrl->base0);
+ }
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 1);
+ pinctrl->base1 = devm_ioremap_nocache(&pdev->dev, res->start,
+ resource_size(res));
+ if (!pinctrl->base1) {
+ dev_err(&pdev->dev, "unable to map I/O space\n");
+ return -ENOMEM;
+ }
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 2);
+ pinctrl->pinconf_base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(pinctrl->pinconf_base)) {
+ dev_err(&pdev->dev, "unable to map I/O space\n");
+ return PTR_ERR(pinctrl->pinconf_base);
+ }
+
+ ret = ns2_mux_log_init(pinctrl);
+ if (ret) {
+ dev_err(&pdev->dev, "unable to initialize IOMUX log\n");
+ return ret;
+ }
+
+ pins = devm_kcalloc(&pdev->dev, num_pins, sizeof(*pins), GFP_KERNEL);
+ if (!pins)
+ return -ENOMEM;
+
+ for (i = 0; i < num_pins; i++) {
+ pins[i].number = ns2_pins[i].pin;
+ pins[i].name = ns2_pins[i].name;
+ pins[i].drv_data = &ns2_pins[i];
+ }
+
+ pinctrl->groups = ns2_pin_groups;
+ pinctrl->num_groups = ARRAY_SIZE(ns2_pin_groups);
+ pinctrl->functions = ns2_pin_functions;
+ pinctrl->num_functions = ARRAY_SIZE(ns2_pin_functions);
+ ns2_pinctrl_desc.pins = pins;
+ ns2_pinctrl_desc.npins = num_pins;
+
+ pinctrl->pctl = pinctrl_register(&ns2_pinctrl_desc, &pdev->dev,
+ pinctrl);
+ if (!pinctrl->pctl) {
+ dev_err(&pdev->dev, "unable to register IOMUX pinctrl\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static const struct of_device_id ns2_pinmux_of_match[] = {
+ {.compatible = "brcm,ns2-pinmux"},
+ { }
+};
+
+static struct platform_driver ns2_pinmux_driver = {
+ .driver = {
+ .name = "ns2-pinmux",
+ .of_match_table = ns2_pinmux_of_match,
+ },
+ .probe = ns2_pinmux_probe,
+};
+
+static int __init ns2_pinmux_init(void)
+{
+ return platform_driver_register(&ns2_pinmux_driver);
+}
+arch_initcall(ns2_pinmux_init);
diff --git a/drivers/pinctrl/bcm/pinctrl-nsp-gpio.c b/drivers/pinctrl/bcm/pinctrl-nsp-gpio.c
index ac90043..a8b37a9 100644
--- a/drivers/pinctrl/bcm/pinctrl-nsp-gpio.c
+++ b/drivers/pinctrl/bcm/pinctrl-nsp-gpio.c
@@ -363,7 +363,7 @@
.get_groups_count = nsp_get_groups_count,
.get_group_name = nsp_get_group_name,
.dt_node_to_map = pinconf_generic_dt_node_to_map_pin,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
static int nsp_gpio_set_slew(struct nsp_gpio *chip, unsigned gpio, u16 slew)
@@ -609,7 +609,7 @@
pctldesc->npins = gc->ngpio;
pctldesc->confops = &nsp_pconf_ops;
- chip->pctl = pinctrl_register(pctldesc, chip->dev, chip);
+ chip->pctl = devm_pinctrl_register(chip->dev, pctldesc, chip);
if (IS_ERR(chip->pctl)) {
dev_err(chip->dev, "unable to register pinctrl device\n");
return PTR_ERR(chip->pctl);
diff --git a/drivers/pinctrl/berlin/berlin.c b/drivers/pinctrl/berlin/berlin.c
index 46f2b48..8f0dc02 100644
--- a/drivers/pinctrl/berlin/berlin.c
+++ b/drivers/pinctrl/berlin/berlin.c
@@ -104,7 +104,7 @@
.get_groups_count = &berlin_pinctrl_get_group_count,
.get_group_name = &berlin_pinctrl_get_group_name,
.dt_node_to_map = &berlin_pinctrl_dt_node_to_map,
- .dt_free_map = &pinctrl_utils_dt_free_map,
+ .dt_free_map = &pinctrl_utils_free_map,
};
static int berlin_pinmux_get_functions_count(struct pinctrl_dev *pctrl_dev)
@@ -316,7 +316,8 @@
return ret;
}
- pctrl->pctrl_dev = pinctrl_register(&berlin_pctrl_desc, dev, pctrl);
+ pctrl->pctrl_dev = devm_pinctrl_register(dev, &berlin_pctrl_desc,
+ pctrl);
if (IS_ERR(pctrl->pctrl_dev)) {
dev_err(dev, "failed to register pinctrl driver\n");
return PTR_ERR(pctrl->pctrl_dev);
diff --git a/drivers/pinctrl/core.c b/drivers/pinctrl/core.c
index f67a8b7..98d2a1b 100644
--- a/drivers/pinctrl/core.c
+++ b/drivers/pinctrl/core.c
@@ -1872,6 +1872,69 @@
}
EXPORT_SYMBOL_GPL(pinctrl_unregister);
+static void devm_pinctrl_dev_release(struct device *dev, void *res)
+{
+ struct pinctrl_dev *pctldev = *(struct pinctrl_dev **)res;
+
+ pinctrl_unregister(pctldev);
+}
+
+static int devm_pinctrl_dev_match(struct device *dev, void *res, void *data)
+{
+ struct pctldev **r = res;
+
+ if (WARN_ON(!r || !*r))
+ return 0;
+
+ return *r == data;
+}
+
+/**
+ * devm_pinctrl_register() - Resource managed version of pinctrl_register().
+ * @dev: parent device for this pin controller
+ * @pctldesc: descriptor for this pin controller
+ * @driver_data: private pin controller data for this pin controller
+ *
+ * Returns an error pointer if pincontrol register failed. Otherwise
+ * it returns valid pinctrl handle.
+ *
+ * The pinctrl device will be automatically released when the device is unbound.
+ */
+struct pinctrl_dev *devm_pinctrl_register(struct device *dev,
+ struct pinctrl_desc *pctldesc,
+ void *driver_data)
+{
+ struct pinctrl_dev **ptr, *pctldev;
+
+ ptr = devres_alloc(devm_pinctrl_dev_release, sizeof(*ptr), GFP_KERNEL);
+ if (!ptr)
+ return ERR_PTR(-ENOMEM);
+
+ pctldev = pinctrl_register(pctldesc, dev, driver_data);
+ if (IS_ERR(pctldev)) {
+ devres_free(ptr);
+ return pctldev;
+ }
+
+ *ptr = pctldev;
+ devres_add(dev, ptr);
+
+ return pctldev;
+}
+EXPORT_SYMBOL_GPL(devm_pinctrl_register);
+
+/**
+ * devm_pinctrl_unregister() - Resource managed version of pinctrl_unregister().
+ * @dev: device for which which resource was allocated
+ * @pctldev: the pinctrl device to unregister.
+ */
+void devm_pinctrl_unregister(struct device *dev, struct pinctrl_dev *pctldev)
+{
+ WARN_ON(devres_release(dev, devm_pinctrl_dev_release,
+ devm_pinctrl_dev_match, pctldev));
+}
+EXPORT_SYMBOL_GPL(devm_pinctrl_unregister);
+
static int __init pinctrl_init(void)
{
pr_info("initialized pinctrl subsystem\n");
diff --git a/drivers/pinctrl/freescale/pinctrl-imx.c b/drivers/pinctrl/freescale/pinctrl-imx.c
index 9cfa544..47ccfcc 100644
--- a/drivers/pinctrl/freescale/pinctrl-imx.c
+++ b/drivers/pinctrl/freescale/pinctrl-imx.c
@@ -789,7 +789,7 @@
ipctl->info = info;
ipctl->dev = info->dev;
platform_set_drvdata(pdev, ipctl);
- ipctl->pctl = pinctrl_register(&imx_pinctrl_desc, &pdev->dev, ipctl);
+ ipctl->pctl = devm_pinctrl_register(&pdev->dev, &imx_pinctrl_desc, ipctl);
if (IS_ERR(ipctl->pctl)) {
dev_err(&pdev->dev, "could not register IMX pinctrl driver\n");
return PTR_ERR(ipctl->pctl);
@@ -799,12 +799,3 @@
return 0;
}
-
-int imx_pinctrl_remove(struct platform_device *pdev)
-{
- struct imx_pinctrl *ipctl = platform_get_drvdata(pdev);
-
- pinctrl_unregister(ipctl->pctl);
-
- return 0;
-}
diff --git a/drivers/pinctrl/freescale/pinctrl-imx.h b/drivers/pinctrl/freescale/pinctrl-imx.h
index 3b8bd81..8af8aa2 100644
--- a/drivers/pinctrl/freescale/pinctrl-imx.h
+++ b/drivers/pinctrl/freescale/pinctrl-imx.h
@@ -99,5 +99,4 @@
int imx_pinctrl_probe(struct platform_device *pdev,
struct imx_pinctrl_soc_info *info);
-int imx_pinctrl_remove(struct platform_device *pdev);
#endif /* __DRIVERS_PINCTRL_IMX_H */
diff --git a/drivers/pinctrl/freescale/pinctrl-imx1-core.c b/drivers/pinctrl/freescale/pinctrl-imx1-core.c
index acaf84c..b4400cb 100644
--- a/drivers/pinctrl/freescale/pinctrl-imx1-core.c
+++ b/drivers/pinctrl/freescale/pinctrl-imx1-core.c
@@ -635,7 +635,7 @@
ipctl->info = info;
ipctl->dev = info->dev;
platform_set_drvdata(pdev, ipctl);
- ipctl->pctl = pinctrl_register(pctl_desc, &pdev->dev, ipctl);
+ ipctl->pctl = devm_pinctrl_register(&pdev->dev, pctl_desc, ipctl);
if (IS_ERR(ipctl->pctl)) {
dev_err(&pdev->dev, "could not register IMX pinctrl driver\n");
return PTR_ERR(ipctl->pctl);
@@ -652,12 +652,3 @@
return 0;
}
-
-int imx1_pinctrl_core_remove(struct platform_device *pdev)
-{
- struct imx1_pinctrl *ipctl = platform_get_drvdata(pdev);
-
- pinctrl_unregister(ipctl->pctl);
-
- return 0;
-}
diff --git a/drivers/pinctrl/freescale/pinctrl-imx1.c b/drivers/pinctrl/freescale/pinctrl-imx1.c
index d3bacb7..0472345 100644
--- a/drivers/pinctrl/freescale/pinctrl-imx1.c
+++ b/drivers/pinctrl/freescale/pinctrl-imx1.c
@@ -269,7 +269,6 @@
.name = "imx1-pinctrl",
.of_match_table = imx1_pinctrl_of_match,
},
- .remove = imx1_pinctrl_core_remove,
};
module_platform_driver_probe(imx1_pinctrl_driver, imx1_pinctrl_probe);
diff --git a/drivers/pinctrl/freescale/pinctrl-imx1.h b/drivers/pinctrl/freescale/pinctrl-imx1.h
index 692a54c..1740743 100644
--- a/drivers/pinctrl/freescale/pinctrl-imx1.h
+++ b/drivers/pinctrl/freescale/pinctrl-imx1.h
@@ -69,5 +69,4 @@
int imx1_pinctrl_core_probe(struct platform_device *pdev,
struct imx1_pinctrl_soc_info *info);
-int imx1_pinctrl_core_remove(struct platform_device *pdev);
#endif /* __DRIVERS_PINCTRL_IMX1_H */
diff --git a/drivers/pinctrl/freescale/pinctrl-imx21.c b/drivers/pinctrl/freescale/pinctrl-imx21.c
index 9d9aca3..aa1221f 100644
--- a/drivers/pinctrl/freescale/pinctrl-imx21.c
+++ b/drivers/pinctrl/freescale/pinctrl-imx21.c
@@ -332,7 +332,6 @@
.name = "imx21-pinctrl",
.of_match_table = imx21_pinctrl_of_match,
},
- .remove = imx1_pinctrl_core_remove,
};
module_platform_driver_probe(imx21_pinctrl_driver, imx21_pinctrl_probe);
diff --git a/drivers/pinctrl/freescale/pinctrl-imx25.c b/drivers/pinctrl/freescale/pinctrl-imx25.c
index 293ed43..81ad546d 100644
--- a/drivers/pinctrl/freescale/pinctrl-imx25.c
+++ b/drivers/pinctrl/freescale/pinctrl-imx25.c
@@ -331,7 +331,6 @@
.of_match_table = of_match_ptr(imx25_pinctrl_of_match),
},
.probe = imx25_pinctrl_probe,
- .remove = imx_pinctrl_remove,
};
static int __init imx25_pinctrl_init(void)
diff --git a/drivers/pinctrl/freescale/pinctrl-imx27.c b/drivers/pinctrl/freescale/pinctrl-imx27.c
index a461d588..f828fbb 100644
--- a/drivers/pinctrl/freescale/pinctrl-imx27.c
+++ b/drivers/pinctrl/freescale/pinctrl-imx27.c
@@ -405,7 +405,6 @@
.of_match_table = of_match_ptr(imx27_pinctrl_of_match),
},
.probe = imx27_pinctrl_probe,
- .remove = imx1_pinctrl_core_remove,
};
static int __init imx27_pinctrl_init(void)
diff --git a/drivers/pinctrl/freescale/pinctrl-imx35.c b/drivers/pinctrl/freescale/pinctrl-imx35.c
index 9109c10..13eb224 100644
--- a/drivers/pinctrl/freescale/pinctrl-imx35.c
+++ b/drivers/pinctrl/freescale/pinctrl-imx35.c
@@ -1021,7 +1021,6 @@
.of_match_table = imx35_pinctrl_of_match,
},
.probe = imx35_pinctrl_probe,
- .remove = imx_pinctrl_remove,
};
static int __init imx35_pinctrl_init(void)
diff --git a/drivers/pinctrl/freescale/pinctrl-imx50.c b/drivers/pinctrl/freescale/pinctrl-imx50.c
index 8acc4d9..95a36c8 100644
--- a/drivers/pinctrl/freescale/pinctrl-imx50.c
+++ b/drivers/pinctrl/freescale/pinctrl-imx50.c
@@ -408,7 +408,6 @@
.of_match_table = of_match_ptr(imx50_pinctrl_of_match),
},
.probe = imx50_pinctrl_probe,
- .remove = imx_pinctrl_remove,
};
static int __init imx50_pinctrl_init(void)
diff --git a/drivers/pinctrl/freescale/pinctrl-imx51.c b/drivers/pinctrl/freescale/pinctrl-imx51.c
index 8dec494..0863e527 100644
--- a/drivers/pinctrl/freescale/pinctrl-imx51.c
+++ b/drivers/pinctrl/freescale/pinctrl-imx51.c
@@ -784,7 +784,6 @@
.of_match_table = imx51_pinctrl_of_match,
},
.probe = imx51_pinctrl_probe,
- .remove = imx_pinctrl_remove,
};
static int __init imx51_pinctrl_init(void)
diff --git a/drivers/pinctrl/freescale/pinctrl-imx53.c b/drivers/pinctrl/freescale/pinctrl-imx53.c
index d39dfd6..64c9cbe 100644
--- a/drivers/pinctrl/freescale/pinctrl-imx53.c
+++ b/drivers/pinctrl/freescale/pinctrl-imx53.c
@@ -471,7 +471,6 @@
.of_match_table = imx53_pinctrl_of_match,
},
.probe = imx53_pinctrl_probe,
- .remove = imx_pinctrl_remove,
};
static int __init imx53_pinctrl_init(void)
diff --git a/drivers/pinctrl/freescale/pinctrl-imx6dl.c b/drivers/pinctrl/freescale/pinctrl-imx6dl.c
index 5a2cdb0..de17bac 100644
--- a/drivers/pinctrl/freescale/pinctrl-imx6dl.c
+++ b/drivers/pinctrl/freescale/pinctrl-imx6dl.c
@@ -477,7 +477,6 @@
.of_match_table = imx6dl_pinctrl_of_match,
},
.probe = imx6dl_pinctrl_probe,
- .remove = imx_pinctrl_remove,
};
static int __init imx6dl_pinctrl_init(void)
diff --git a/drivers/pinctrl/freescale/pinctrl-imx6q.c b/drivers/pinctrl/freescale/pinctrl-imx6q.c
index 7d50a36..55cd8a0 100644
--- a/drivers/pinctrl/freescale/pinctrl-imx6q.c
+++ b/drivers/pinctrl/freescale/pinctrl-imx6q.c
@@ -483,7 +483,6 @@
.of_match_table = imx6q_pinctrl_of_match,
},
.probe = imx6q_pinctrl_probe,
- .remove = imx_pinctrl_remove,
};
static int __init imx6q_pinctrl_init(void)
diff --git a/drivers/pinctrl/freescale/pinctrl-imx6sl.c b/drivers/pinctrl/freescale/pinctrl-imx6sl.c
index e27d17fd..bf455b8 100644
--- a/drivers/pinctrl/freescale/pinctrl-imx6sl.c
+++ b/drivers/pinctrl/freescale/pinctrl-imx6sl.c
@@ -384,7 +384,6 @@
.of_match_table = imx6sl_pinctrl_of_match,
},
.probe = imx6sl_pinctrl_probe,
- .remove = imx_pinctrl_remove,
};
static int __init imx6sl_pinctrl_init(void)
diff --git a/drivers/pinctrl/freescale/pinctrl-imx6sx.c b/drivers/pinctrl/freescale/pinctrl-imx6sx.c
index 117180c..84118c3 100644
--- a/drivers/pinctrl/freescale/pinctrl-imx6sx.c
+++ b/drivers/pinctrl/freescale/pinctrl-imx6sx.c
@@ -387,7 +387,6 @@
.of_match_table = of_match_ptr(imx6sx_pinctrl_of_match),
},
.probe = imx6sx_pinctrl_probe,
- .remove = imx_pinctrl_remove,
};
static int __init imx6sx_pinctrl_init(void)
diff --git a/drivers/pinctrl/freescale/pinctrl-imx6ul.c b/drivers/pinctrl/freescale/pinctrl-imx6ul.c
index 78627c7..c707fdd 100644
--- a/drivers/pinctrl/freescale/pinctrl-imx6ul.c
+++ b/drivers/pinctrl/freescale/pinctrl-imx6ul.c
@@ -303,7 +303,6 @@
.of_match_table = of_match_ptr(imx6ul_pinctrl_of_match),
},
.probe = imx6ul_pinctrl_probe,
- .remove = imx_pinctrl_remove,
};
static int __init imx6ul_pinctrl_init(void)
diff --git a/drivers/pinctrl/freescale/pinctrl-imx7d.c b/drivers/pinctrl/freescale/pinctrl-imx7d.c
index 1c89613..d30d91f 100644
--- a/drivers/pinctrl/freescale/pinctrl-imx7d.c
+++ b/drivers/pinctrl/freescale/pinctrl-imx7d.c
@@ -395,7 +395,6 @@
.of_match_table = of_match_ptr(imx7d_pinctrl_of_match),
},
.probe = imx7d_pinctrl_probe,
- .remove = imx_pinctrl_remove,
};
static int __init imx7d_pinctrl_init(void)
diff --git a/drivers/pinctrl/freescale/pinctrl-vf610.c b/drivers/pinctrl/freescale/pinctrl-vf610.c
index 587d1ff..6d81be0 100644
--- a/drivers/pinctrl/freescale/pinctrl-vf610.c
+++ b/drivers/pinctrl/freescale/pinctrl-vf610.c
@@ -318,7 +318,6 @@
.of_match_table = vf610_pinctrl_of_match,
},
.probe = vf610_pinctrl_probe,
- .remove = imx_pinctrl_remove,
};
static int __init vf610_pinctrl_init(void)
diff --git a/drivers/pinctrl/intel/Kconfig b/drivers/pinctrl/intel/Kconfig
index 4d2efad..1c74e03 100644
--- a/drivers/pinctrl/intel/Kconfig
+++ b/drivers/pinctrl/intel/Kconfig
@@ -6,6 +6,9 @@
bool "Intel Baytrail GPIO pin control"
depends on GPIOLIB && ACPI
select GPIOLIB_IRQCHIP
+ select PINMUX
+ select PINCONF
+ select GENERIC_PINCONF
help
driver for memory mapped GPIO functionality on Intel Baytrail
platforms. Supports 3 banks with 102, 28 and 44 gpios.
diff --git a/drivers/pinctrl/intel/pinctrl-baytrail.c b/drivers/pinctrl/intel/pinctrl-baytrail.c
index 21b79a4..55182fc 100644
--- a/drivers/pinctrl/intel/pinctrl-baytrail.c
+++ b/drivers/pinctrl/intel/pinctrl-baytrail.c
@@ -20,6 +20,7 @@
#include <linux/types.h>
#include <linux/bitops.h>
#include <linux/interrupt.h>
+#include <linux/gpio.h>
#include <linux/gpio/driver.h>
#include <linux/acpi.h>
#include <linux/platform_device.h>
@@ -27,6 +28,9 @@
#include <linux/io.h>
#include <linux/pm_runtime.h>
#include <linux/pinctrl/pinctrl.h>
+#include <linux/pinctrl/pinmux.h>
+#include <linux/pinctrl/pinconf.h>
+#include <linux/pinctrl/pinconf-generic.h>
/* memory mapped register offsets */
#define BYT_CONF0_REG 0x000
@@ -34,6 +38,7 @@
#define BYT_VAL_REG 0x008
#define BYT_DFT_REG 0x00c
#define BYT_INT_STAT_REG 0x800
+#define BYT_DEBOUNCE_REG 0x9d0
/* BYT_CONF0_REG register bits */
#define BYT_IODEN BIT(31)
@@ -41,6 +46,7 @@
#define BYT_TRIG_NEG BIT(26)
#define BYT_TRIG_POS BIT(25)
#define BYT_TRIG_LVL BIT(24)
+#define BYT_DEBOUNCE_EN BIT(20)
#define BYT_PULL_STR_SHIFT 9
#define BYT_PULL_STR_MASK (3 << BYT_PULL_STR_SHIFT)
#define BYT_PULL_STR_2K (0 << BYT_PULL_STR_SHIFT)
@@ -65,6 +71,16 @@
BYT_PIN_MUX)
#define BYT_VAL_RESTORE_MASK (BYT_DIR_MASK | BYT_LEVEL)
+/* BYT_DEBOUNCE_REG bits */
+#define BYT_DEBOUNCE_PULSE_MASK 0x7
+#define BYT_DEBOUNCE_PULSE_375US 1
+#define BYT_DEBOUNCE_PULSE_750US 2
+#define BYT_DEBOUNCE_PULSE_1500US 3
+#define BYT_DEBOUNCE_PULSE_3MS 4
+#define BYT_DEBOUNCE_PULSE_6MS 5
+#define BYT_DEBOUNCE_PULSE_12MS 6
+#define BYT_DEBOUNCE_PULSE_24MS 7
+
#define BYT_NGPIO_SCORE 102
#define BYT_NGPIO_NCORE 28
#define BYT_NGPIO_SUS 44
@@ -74,18 +90,227 @@
#define BYT_SUS_ACPI_UID "3"
/*
- * Baytrail gpio controller consist of three separate sub-controllers called
- * SCORE, NCORE and SUS. The sub-controllers are identified by their acpi UID.
- *
- * GPIO numbering is _not_ ordered meaning that gpio # 0 in ACPI namespace does
- * _not_ correspond to the first gpio register at controller's gpio base.
- * There is no logic or pattern in mapping gpio numbers to registers (pads) so
- * each sub-controller needs to have its own mapping table
+ * This is the function value most pins have for GPIO muxing. If the value
+ * differs from the default one, it must be explicitly mentioned. Otherwise, the
+ * pin control implementation will set the muxing value to default GPIO if it
+ * does not find a match for the requested function.
*/
+#define BYT_DEFAULT_GPIO_MUX 0
-/* score_pins[gpio_nr] = pad_nr */
+struct byt_gpio_pin_context {
+ u32 conf0;
+ u32 val;
+};
-static unsigned const score_pins[BYT_NGPIO_SCORE] = {
+struct byt_simple_func_mux {
+ const char *name;
+ unsigned short func;
+};
+
+struct byt_mixed_func_mux {
+ const char *name;
+ const unsigned short *func_values;
+};
+
+struct byt_pingroup {
+ const char *name;
+ const unsigned int *pins;
+ size_t npins;
+ unsigned short has_simple_funcs;
+ union {
+ const struct byt_simple_func_mux *simple_funcs;
+ const struct byt_mixed_func_mux *mixed_funcs;
+ };
+ size_t nfuncs;
+};
+
+struct byt_function {
+ const char *name;
+ const char * const *groups;
+ size_t ngroups;
+};
+
+struct byt_community {
+ unsigned int pin_base;
+ size_t npins;
+ const unsigned int *pad_map;
+ void __iomem *reg_base;
+};
+
+#define SIMPLE_FUNC(n, f) \
+ { \
+ .name = (n), \
+ .func = (f), \
+ }
+#define MIXED_FUNC(n, f) \
+ { \
+ .name = (n), \
+ .func_values = (f), \
+ }
+
+#define PIN_GROUP_SIMPLE(n, p, f) \
+ { \
+ .name = (n), \
+ .pins = (p), \
+ .npins = ARRAY_SIZE((p)), \
+ .has_simple_funcs = 1, \
+ .simple_funcs = (f), \
+ .nfuncs = ARRAY_SIZE((f)), \
+ }
+#define PIN_GROUP_MIXED(n, p, f) \
+ { \
+ .name = (n), \
+ .pins = (p), \
+ .npins = ARRAY_SIZE((p)), \
+ .has_simple_funcs = 0, \
+ .mixed_funcs = (f), \
+ .nfuncs = ARRAY_SIZE((f)), \
+ }
+
+#define FUNCTION(n, g) \
+ { \
+ .name = (n), \
+ .groups = (g), \
+ .ngroups = ARRAY_SIZE((g)), \
+ }
+
+#define COMMUNITY(p, n, map) \
+ { \
+ .pin_base = (p), \
+ .npins = (n), \
+ .pad_map = (map),\
+ }
+
+struct byt_pinctrl_soc_data {
+ const char *uid;
+ const struct pinctrl_pin_desc *pins;
+ size_t npins;
+ const struct byt_pingroup *groups;
+ size_t ngroups;
+ const struct byt_function *functions;
+ size_t nfunctions;
+ const struct byt_community *communities;
+ size_t ncommunities;
+};
+
+struct byt_gpio {
+ struct gpio_chip chip;
+ struct platform_device *pdev;
+ struct pinctrl_dev *pctl_dev;
+ struct pinctrl_desc pctl_desc;
+ raw_spinlock_t lock;
+ const struct byt_pinctrl_soc_data *soc_data;
+ struct byt_community *communities_copy;
+ struct byt_gpio_pin_context *saved_context;
+};
+
+/* SCORE pins, aka GPIOC_<pin_no> or GPIO_S0_SC[<pin_no>] */
+static const struct pinctrl_pin_desc byt_score_pins[] = {
+ PINCTRL_PIN(0, "SATA_GP0"),
+ PINCTRL_PIN(1, "SATA_GP1"),
+ PINCTRL_PIN(2, "SATA_LED#"),
+ PINCTRL_PIN(3, "PCIE_CLKREQ0"),
+ PINCTRL_PIN(4, "PCIE_CLKREQ1"),
+ PINCTRL_PIN(5, "PCIE_CLKREQ2"),
+ PINCTRL_PIN(6, "PCIE_CLKREQ3"),
+ PINCTRL_PIN(7, "SD3_WP"),
+ PINCTRL_PIN(8, "HDA_RST"),
+ PINCTRL_PIN(9, "HDA_SYNC"),
+ PINCTRL_PIN(10, "HDA_CLK"),
+ PINCTRL_PIN(11, "HDA_SDO"),
+ PINCTRL_PIN(12, "HDA_SDI0"),
+ PINCTRL_PIN(13, "HDA_SDI1"),
+ PINCTRL_PIN(14, "GPIO_S0_SC14"),
+ PINCTRL_PIN(15, "GPIO_S0_SC15"),
+ PINCTRL_PIN(16, "MMC1_CLK"),
+ PINCTRL_PIN(17, "MMC1_D0"),
+ PINCTRL_PIN(18, "MMC1_D1"),
+ PINCTRL_PIN(19, "MMC1_D2"),
+ PINCTRL_PIN(20, "MMC1_D3"),
+ PINCTRL_PIN(21, "MMC1_D4"),
+ PINCTRL_PIN(22, "MMC1_D5"),
+ PINCTRL_PIN(23, "MMC1_D6"),
+ PINCTRL_PIN(24, "MMC1_D7"),
+ PINCTRL_PIN(25, "MMC1_CMD"),
+ PINCTRL_PIN(26, "MMC1_RST"),
+ PINCTRL_PIN(27, "SD2_CLK"),
+ PINCTRL_PIN(28, "SD2_D0"),
+ PINCTRL_PIN(29, "SD2_D1"),
+ PINCTRL_PIN(30, "SD2_D2"),
+ PINCTRL_PIN(31, "SD2_D3_CD"),
+ PINCTRL_PIN(32, "SD2_CMD"),
+ PINCTRL_PIN(33, "SD3_CLK"),
+ PINCTRL_PIN(34, "SD3_D0"),
+ PINCTRL_PIN(35, "SD3_D1"),
+ PINCTRL_PIN(36, "SD3_D2"),
+ PINCTRL_PIN(37, "SD3_D3"),
+ PINCTRL_PIN(38, "SD3_CD"),
+ PINCTRL_PIN(39, "SD3_CMD"),
+ PINCTRL_PIN(40, "SD3_1P8EN"),
+ PINCTRL_PIN(41, "SD3_PWREN#"),
+ PINCTRL_PIN(42, "ILB_LPC_AD0"),
+ PINCTRL_PIN(43, "ILB_LPC_AD1"),
+ PINCTRL_PIN(44, "ILB_LPC_AD2"),
+ PINCTRL_PIN(45, "ILB_LPC_AD3"),
+ PINCTRL_PIN(46, "ILB_LPC_FRAME"),
+ PINCTRL_PIN(47, "ILB_LPC_CLK0"),
+ PINCTRL_PIN(48, "ILB_LPC_CLK1"),
+ PINCTRL_PIN(49, "ILB_LPC_CLKRUN"),
+ PINCTRL_PIN(50, "ILB_LPC_SERIRQ"),
+ PINCTRL_PIN(51, "PCU_SMB_DATA"),
+ PINCTRL_PIN(52, "PCU_SMB_CLK"),
+ PINCTRL_PIN(53, "PCU_SMB_ALERT"),
+ PINCTRL_PIN(54, "ILB_8254_SPKR"),
+ PINCTRL_PIN(55, "GPIO_S0_SC55"),
+ PINCTRL_PIN(56, "GPIO_S0_SC56"),
+ PINCTRL_PIN(57, "GPIO_S0_SC57"),
+ PINCTRL_PIN(58, "GPIO_S0_SC58"),
+ PINCTRL_PIN(59, "GPIO_S0_SC59"),
+ PINCTRL_PIN(60, "GPIO_S0_SC60"),
+ PINCTRL_PIN(61, "GPIO_S0_SC61"),
+ PINCTRL_PIN(62, "LPE_I2S2_CLK"),
+ PINCTRL_PIN(63, "LPE_I2S2_FRM"),
+ PINCTRL_PIN(64, "LPE_I2S2_DATAIN"),
+ PINCTRL_PIN(65, "LPE_I2S2_DATAOUT"),
+ PINCTRL_PIN(66, "SIO_SPI_CS"),
+ PINCTRL_PIN(67, "SIO_SPI_MISO"),
+ PINCTRL_PIN(68, "SIO_SPI_MOSI"),
+ PINCTRL_PIN(69, "SIO_SPI_CLK"),
+ PINCTRL_PIN(70, "SIO_UART1_RXD"),
+ PINCTRL_PIN(71, "SIO_UART1_TXD"),
+ PINCTRL_PIN(72, "SIO_UART1_RTS"),
+ PINCTRL_PIN(73, "SIO_UART1_CTS"),
+ PINCTRL_PIN(74, "SIO_UART2_RXD"),
+ PINCTRL_PIN(75, "SIO_UART2_TXD"),
+ PINCTRL_PIN(76, "SIO_UART2_RTS"),
+ PINCTRL_PIN(77, "SIO_UART2_CTS"),
+ PINCTRL_PIN(78, "SIO_I2C0_DATA"),
+ PINCTRL_PIN(79, "SIO_I2C0_CLK"),
+ PINCTRL_PIN(80, "SIO_I2C1_DATA"),
+ PINCTRL_PIN(81, "SIO_I2C1_CLK"),
+ PINCTRL_PIN(82, "SIO_I2C2_DATA"),
+ PINCTRL_PIN(83, "SIO_I2C2_CLK"),
+ PINCTRL_PIN(84, "SIO_I2C3_DATA"),
+ PINCTRL_PIN(85, "SIO_I2C3_CLK"),
+ PINCTRL_PIN(86, "SIO_I2C4_DATA"),
+ PINCTRL_PIN(87, "SIO_I2C4_CLK"),
+ PINCTRL_PIN(88, "SIO_I2C5_DATA"),
+ PINCTRL_PIN(89, "SIO_I2C5_CLK"),
+ PINCTRL_PIN(90, "SIO_I2C6_DATA"),
+ PINCTRL_PIN(91, "SIO_I2C6_CLK"),
+ PINCTRL_PIN(92, "GPIO_S0_SC92"),
+ PINCTRL_PIN(93, "GPIO_S0_SC93"),
+ PINCTRL_PIN(94, "SIO_PWM0"),
+ PINCTRL_PIN(95, "SIO_PWM1"),
+ PINCTRL_PIN(96, "PMC_PLT_CLK0"),
+ PINCTRL_PIN(97, "PMC_PLT_CLK1"),
+ PINCTRL_PIN(98, "PMC_PLT_CLK2"),
+ PINCTRL_PIN(99, "PMC_PLT_CLK3"),
+ PINCTRL_PIN(100, "PMC_PLT_CLK4"),
+ PINCTRL_PIN(101, "PMC_PLT_CLK5"),
+};
+
+static const unsigned int byt_score_pins_map[BYT_NGPIO_SCORE] = {
85, 89, 93, 96, 99, 102, 98, 101, 34, 37,
36, 38, 39, 35, 40, 84, 62, 61, 64, 59,
54, 56, 60, 55, 63, 57, 51, 50, 53, 47,
@@ -99,13 +324,263 @@
97, 100,
};
-static unsigned const ncore_pins[BYT_NGPIO_NCORE] = {
- 19, 18, 17, 20, 21, 22, 24, 25, 23, 16,
- 14, 15, 12, 26, 27, 1, 4, 8, 11, 0,
- 3, 6, 10, 13, 2, 5, 9, 7,
+/* SCORE groups */
+static const unsigned int byt_score_uart1_pins[] = { 70, 71, 72, 73 };
+static const unsigned int byt_score_uart2_pins[] = { 74, 75, 76, 77 };
+static const struct byt_simple_func_mux byt_score_uart_mux[] = {
+ SIMPLE_FUNC("uart", 1),
};
-static unsigned const sus_pins[BYT_NGPIO_SUS] = {
+static const unsigned int byt_score_pwm0_pins[] = { 94 };
+static const unsigned int byt_score_pwm1_pins[] = { 95 };
+static const struct byt_simple_func_mux byt_score_pwm_mux[] = {
+ SIMPLE_FUNC("pwm", 1),
+};
+
+static const unsigned int byt_score_sio_spi_pins[] = { 66, 67, 68, 69 };
+static const struct byt_simple_func_mux byt_score_spi_mux[] = {
+ SIMPLE_FUNC("spi", 1),
+};
+
+static const unsigned int byt_score_i2c5_pins[] = { 88, 89 };
+static const unsigned int byt_score_i2c6_pins[] = { 90, 91 };
+static const unsigned int byt_score_i2c4_pins[] = { 86, 87 };
+static const unsigned int byt_score_i2c3_pins[] = { 84, 85 };
+static const unsigned int byt_score_i2c2_pins[] = { 82, 83 };
+static const unsigned int byt_score_i2c1_pins[] = { 80, 81 };
+static const unsigned int byt_score_i2c0_pins[] = { 78, 79 };
+static const struct byt_simple_func_mux byt_score_i2c_mux[] = {
+ SIMPLE_FUNC("i2c", 1),
+};
+
+static const unsigned int byt_score_ssp0_pins[] = { 8, 9, 10, 11 };
+static const unsigned int byt_score_ssp1_pins[] = { 12, 13, 14, 15 };
+static const unsigned int byt_score_ssp2_pins[] = { 62, 63, 64, 65 };
+static const struct byt_simple_func_mux byt_score_ssp_mux[] = {
+ SIMPLE_FUNC("ssp", 1),
+};
+
+static const unsigned int byt_score_sdcard_pins[] = {
+ 7, 33, 34, 35, 36, 37, 38, 39, 40, 41,
+};
+static const unsigned short byt_score_sdcard_mux_values[] = {
+ 2, 1, 1, 1, 1, 1, 1, 1, 1, 1,
+};
+static const struct byt_mixed_func_mux byt_score_sdcard_mux[] = {
+ MIXED_FUNC("sdcard", byt_score_sdcard_mux_values),
+};
+
+static const unsigned int byt_score_sdio_pins[] = { 27, 28, 29, 30, 31, 32 };
+static const struct byt_simple_func_mux byt_score_sdio_mux[] = {
+ SIMPLE_FUNC("sdio", 1),
+};
+
+static const unsigned int byt_score_emmc_pins[] = {
+ 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26,
+};
+static const struct byt_simple_func_mux byt_score_emmc_mux[] = {
+ SIMPLE_FUNC("emmc", 1),
+};
+
+static const unsigned int byt_score_ilb_lpc_pins[] = {
+ 42, 43, 44, 45, 46, 47, 48, 49, 50,
+};
+static const struct byt_simple_func_mux byt_score_lpc_mux[] = {
+ SIMPLE_FUNC("lpc", 1),
+};
+
+static const unsigned int byt_score_sata_pins[] = { 0, 1, 2 };
+static const struct byt_simple_func_mux byt_score_sata_mux[] = {
+ SIMPLE_FUNC("sata", 1),
+};
+
+static const unsigned int byt_score_plt_clk0_pins[] = { 96 };
+static const unsigned int byt_score_plt_clk1_pins[] = { 97 };
+static const unsigned int byt_score_plt_clk2_pins[] = { 98 };
+static const unsigned int byt_score_plt_clk4_pins[] = { 99 };
+static const unsigned int byt_score_plt_clk5_pins[] = { 100 };
+static const unsigned int byt_score_plt_clk3_pins[] = { 101 };
+static const struct byt_simple_func_mux byt_score_plt_clk_mux[] = {
+ SIMPLE_FUNC("plt_clk", 1),
+};
+
+static const unsigned int byt_score_smbus_pins[] = { 51, 52, 53 };
+static const struct byt_simple_func_mux byt_score_smbus_mux[] = {
+ SIMPLE_FUNC("smbus", 1),
+};
+
+static const struct byt_pingroup byt_score_groups[] = {
+ PIN_GROUP_SIMPLE("uart1_grp",
+ byt_score_uart1_pins, byt_score_uart_mux),
+ PIN_GROUP_SIMPLE("uart2_grp",
+ byt_score_uart2_pins, byt_score_uart_mux),
+ PIN_GROUP_SIMPLE("pwm0_grp",
+ byt_score_pwm0_pins, byt_score_pwm_mux),
+ PIN_GROUP_SIMPLE("pwm1_grp",
+ byt_score_pwm1_pins, byt_score_pwm_mux),
+ PIN_GROUP_SIMPLE("ssp2_grp",
+ byt_score_ssp2_pins, byt_score_pwm_mux),
+ PIN_GROUP_SIMPLE("sio_spi_grp",
+ byt_score_sio_spi_pins, byt_score_spi_mux),
+ PIN_GROUP_SIMPLE("i2c5_grp",
+ byt_score_i2c5_pins, byt_score_i2c_mux),
+ PIN_GROUP_SIMPLE("i2c6_grp",
+ byt_score_i2c6_pins, byt_score_i2c_mux),
+ PIN_GROUP_SIMPLE("i2c4_grp",
+ byt_score_i2c4_pins, byt_score_i2c_mux),
+ PIN_GROUP_SIMPLE("i2c3_grp",
+ byt_score_i2c3_pins, byt_score_i2c_mux),
+ PIN_GROUP_SIMPLE("i2c2_grp",
+ byt_score_i2c2_pins, byt_score_i2c_mux),
+ PIN_GROUP_SIMPLE("i2c1_grp",
+ byt_score_i2c1_pins, byt_score_i2c_mux),
+ PIN_GROUP_SIMPLE("i2c0_grp",
+ byt_score_i2c0_pins, byt_score_i2c_mux),
+ PIN_GROUP_SIMPLE("ssp0_grp",
+ byt_score_ssp0_pins, byt_score_ssp_mux),
+ PIN_GROUP_SIMPLE("ssp1_grp",
+ byt_score_ssp1_pins, byt_score_ssp_mux),
+ PIN_GROUP_MIXED("sdcard_grp",
+ byt_score_sdcard_pins, byt_score_sdcard_mux),
+ PIN_GROUP_SIMPLE("sdio_grp",
+ byt_score_sdio_pins, byt_score_sdio_mux),
+ PIN_GROUP_SIMPLE("emmc_grp",
+ byt_score_emmc_pins, byt_score_emmc_mux),
+ PIN_GROUP_SIMPLE("lpc_grp",
+ byt_score_ilb_lpc_pins, byt_score_lpc_mux),
+ PIN_GROUP_SIMPLE("sata_grp",
+ byt_score_sata_pins, byt_score_sata_mux),
+ PIN_GROUP_SIMPLE("plt_clk0_grp",
+ byt_score_plt_clk0_pins, byt_score_plt_clk_mux),
+ PIN_GROUP_SIMPLE("plt_clk1_grp",
+ byt_score_plt_clk1_pins, byt_score_plt_clk_mux),
+ PIN_GROUP_SIMPLE("plt_clk2_grp",
+ byt_score_plt_clk2_pins, byt_score_plt_clk_mux),
+ PIN_GROUP_SIMPLE("plt_clk3_grp",
+ byt_score_plt_clk3_pins, byt_score_plt_clk_mux),
+ PIN_GROUP_SIMPLE("plt_clk4_grp",
+ byt_score_plt_clk4_pins, byt_score_plt_clk_mux),
+ PIN_GROUP_SIMPLE("plt_clk5_grp",
+ byt_score_plt_clk5_pins, byt_score_plt_clk_mux),
+ PIN_GROUP_SIMPLE("smbus_grp",
+ byt_score_smbus_pins, byt_score_smbus_mux),
+};
+
+static const char * const byt_score_uart_groups[] = {
+ "uart1_grp", "uart2_grp",
+};
+static const char * const byt_score_pwm_groups[] = {
+ "pwm0_grp", "pwm1_grp",
+};
+static const char * const byt_score_ssp_groups[] = {
+ "ssp0_grp", "ssp1_grp", "ssp2_grp",
+};
+static const char * const byt_score_spi_groups[] = { "sio_spi_grp" };
+static const char * const byt_score_i2c_groups[] = {
+ "i2c0_grp", "i2c1_grp", "i2c2_grp", "i2c3_grp", "i2c4_grp", "i2c5_grp",
+ "i2c6_grp",
+};
+static const char * const byt_score_sdcard_groups[] = { "sdcard_grp" };
+static const char * const byt_score_sdio_groups[] = { "sdio_grp" };
+static const char * const byt_score_emmc_groups[] = { "emmc_grp" };
+static const char * const byt_score_lpc_groups[] = { "lpc_grp" };
+static const char * const byt_score_sata_groups[] = { "sata_grp" };
+static const char * const byt_score_plt_clk_groups[] = {
+ "plt_clk0_grp", "plt_clk1_grp", "plt_clk2_grp", "plt_clk3_grp",
+ "plt_clk4_grp", "plt_clk5_grp",
+};
+static const char * const byt_score_smbus_groups[] = { "smbus_grp" };
+static const char * const byt_score_gpio_groups[] = {
+ "uart1_grp", "uart2_grp", "pwm0_grp", "pwm1_grp", "ssp0_grp",
+ "ssp1_grp", "ssp2_grp", "sio_spi_grp", "i2c0_grp", "i2c1_grp",
+ "i2c2_grp", "i2c3_grp", "i2c4_grp", "i2c5_grp", "i2c6_grp",
+ "sdcard_grp", "sdio_grp", "emmc_grp", "lpc_grp", "sata_grp",
+ "plt_clk0_grp", "plt_clk1_grp", "plt_clk2_grp", "plt_clk3_grp",
+ "plt_clk4_grp", "plt_clk5_grp", "smbus_grp",
+
+};
+
+static const struct byt_function byt_score_functions[] = {
+ FUNCTION("uart", byt_score_uart_groups),
+ FUNCTION("pwm", byt_score_pwm_groups),
+ FUNCTION("ssp", byt_score_ssp_groups),
+ FUNCTION("spi", byt_score_spi_groups),
+ FUNCTION("i2c", byt_score_i2c_groups),
+ FUNCTION("sdcard", byt_score_sdcard_groups),
+ FUNCTION("sdio", byt_score_sdio_groups),
+ FUNCTION("emmc", byt_score_emmc_groups),
+ FUNCTION("lpc", byt_score_lpc_groups),
+ FUNCTION("sata", byt_score_sata_groups),
+ FUNCTION("plt_clk", byt_score_plt_clk_groups),
+ FUNCTION("smbus", byt_score_smbus_groups),
+ FUNCTION("gpio", byt_score_gpio_groups),
+};
+
+static const struct byt_community byt_score_communities[] = {
+ COMMUNITY(0, BYT_NGPIO_SCORE, byt_score_pins_map),
+};
+
+static const struct byt_pinctrl_soc_data byt_score_soc_data = {
+ .uid = BYT_SCORE_ACPI_UID,
+ .pins = byt_score_pins,
+ .npins = ARRAY_SIZE(byt_score_pins),
+ .groups = byt_score_groups,
+ .ngroups = ARRAY_SIZE(byt_score_groups),
+ .functions = byt_score_functions,
+ .nfunctions = ARRAY_SIZE(byt_score_functions),
+ .communities = byt_score_communities,
+ .ncommunities = ARRAY_SIZE(byt_score_communities),
+};
+
+/* SUS pins, aka GPIOS_<pin_no> or GPIO_S5[<pin_no>] */
+static const struct pinctrl_pin_desc byt_sus_pins[] = {
+ PINCTRL_PIN(0, "GPIO_S50"),
+ PINCTRL_PIN(1, "GPIO_S51"),
+ PINCTRL_PIN(2, "GPIO_S52"),
+ PINCTRL_PIN(3, "GPIO_S53"),
+ PINCTRL_PIN(4, "GPIO_S54"),
+ PINCTRL_PIN(5, "GPIO_S55"),
+ PINCTRL_PIN(6, "GPIO_S56"),
+ PINCTRL_PIN(7, "GPIO_S57"),
+ PINCTRL_PIN(8, "GPIO_S58"),
+ PINCTRL_PIN(9, "GPIO_S59"),
+ PINCTRL_PIN(10, "GPIO_S510"),
+ PINCTRL_PIN(11, "PMC_SUSPWRDNACK"),
+ PINCTRL_PIN(12, "PMC_SUSCLK0"),
+ PINCTRL_PIN(13, "GPIO_S513"),
+ PINCTRL_PIN(14, "USB_ULPI_RST"),
+ PINCTRL_PIN(15, "PMC_WAKE_PCIE0#"),
+ PINCTRL_PIN(16, "PMC_PWRBTN"),
+ PINCTRL_PIN(17, "GPIO_S517"),
+ PINCTRL_PIN(18, "PMC_SUS_STAT"),
+ PINCTRL_PIN(19, "USB_OC0"),
+ PINCTRL_PIN(20, "USB_OC1"),
+ PINCTRL_PIN(21, "PCU_SPI_CS1"),
+ PINCTRL_PIN(22, "GPIO_S522"),
+ PINCTRL_PIN(23, "GPIO_S523"),
+ PINCTRL_PIN(24, "GPIO_S524"),
+ PINCTRL_PIN(25, "GPIO_S525"),
+ PINCTRL_PIN(26, "GPIO_S526"),
+ PINCTRL_PIN(27, "GPIO_S527"),
+ PINCTRL_PIN(28, "GPIO_S528"),
+ PINCTRL_PIN(29, "GPIO_S529"),
+ PINCTRL_PIN(30, "GPIO_S530"),
+ PINCTRL_PIN(31, "USB_ULPI_CLK"),
+ PINCTRL_PIN(32, "USB_ULPI_DATA0"),
+ PINCTRL_PIN(33, "USB_ULPI_DATA1"),
+ PINCTRL_PIN(34, "USB_ULPI_DATA2"),
+ PINCTRL_PIN(35, "USB_ULPI_DATA3"),
+ PINCTRL_PIN(36, "USB_ULPI_DATA4"),
+ PINCTRL_PIN(37, "USB_ULPI_DATA5"),
+ PINCTRL_PIN(38, "USB_ULPI_DATA6"),
+ PINCTRL_PIN(39, "USB_ULPI_DATA7"),
+ PINCTRL_PIN(40, "USB_ULPI_DIR"),
+ PINCTRL_PIN(41, "USB_ULPI_NXT"),
+ PINCTRL_PIN(42, "USB_ULPI_STP"),
+ PINCTRL_PIN(43, "USB_ULPI_REFCLK"),
+};
+
+static const unsigned int byt_sus_pins_map[BYT_NGPIO_SUS] = {
29, 33, 30, 31, 32, 34, 36, 35, 38, 37,
18, 7, 11, 20, 17, 1, 8, 10, 19, 12,
0, 2, 23, 39, 28, 27, 22, 21, 24, 25,
@@ -113,57 +588,357 @@
52, 53, 59, 40,
};
-static struct pinctrl_gpio_range byt_ranges[] = {
- {
- .name = BYT_SCORE_ACPI_UID, /* match with acpi _UID in probe */
- .npins = BYT_NGPIO_SCORE,
- .pins = score_pins,
- },
- {
- .name = BYT_NCORE_ACPI_UID,
- .npins = BYT_NGPIO_NCORE,
- .pins = ncore_pins,
- },
- {
- .name = BYT_SUS_ACPI_UID,
- .npins = BYT_NGPIO_SUS,
- .pins = sus_pins,
- },
- {
- },
+static const unsigned int byt_sus_usb_over_current_pins[] = { 19, 20 };
+static const struct byt_simple_func_mux byt_sus_usb_oc_mux[] = {
+ SIMPLE_FUNC("usb", 0),
+ SIMPLE_FUNC("gpio", 1),
};
-struct byt_gpio_pin_context {
- u32 conf0;
- u32 val;
+static const unsigned int byt_sus_usb_ulpi_pins[] = {
+ 14, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43,
+};
+static const unsigned short byt_sus_usb_ulpi_mode_values[] = {
+ 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
+};
+static const unsigned short byt_sus_usb_ulpi_gpio_mode_values[] = {
+ 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
+};
+static const struct byt_mixed_func_mux byt_sus_usb_ulpi_mux[] = {
+ MIXED_FUNC("usb", byt_sus_usb_ulpi_mode_values),
+ MIXED_FUNC("gpio", byt_sus_usb_ulpi_gpio_mode_values),
};
-struct byt_gpio {
- struct gpio_chip chip;
- struct platform_device *pdev;
- raw_spinlock_t lock;
- void __iomem *reg_base;
- struct pinctrl_gpio_range *range;
- struct byt_gpio_pin_context *saved_context;
+static const unsigned int byt_sus_pcu_spi_pins[] = { 21 };
+static const struct byt_simple_func_mux byt_sus_pcu_spi_mux[] = {
+ SIMPLE_FUNC("spi", 0),
+ SIMPLE_FUNC("gpio", 1),
};
-static void __iomem *byt_gpio_reg(struct gpio_chip *chip, unsigned offset,
- int reg)
+static const struct byt_pingroup byt_sus_groups[] = {
+ PIN_GROUP_SIMPLE("usb_oc_grp",
+ byt_sus_usb_over_current_pins, byt_sus_usb_oc_mux),
+ PIN_GROUP_MIXED("usb_ulpi_grp",
+ byt_sus_usb_ulpi_pins, byt_sus_usb_ulpi_mux),
+ PIN_GROUP_SIMPLE("pcu_spi_grp",
+ byt_sus_pcu_spi_pins, byt_sus_pcu_spi_mux),
+};
+
+static const char * const byt_sus_usb_groups[] = {
+ "usb_oc_grp", "usb_ulpi_grp",
+};
+static const char * const byt_sus_spi_groups[] = { "pcu_spi_grp" };
+static const char * const byt_sus_gpio_groups[] = {
+ "usb_oc_grp", "usb_ulpi_grp", "pcu_spi_grp",
+};
+
+static const struct byt_function byt_sus_functions[] = {
+ FUNCTION("usb", byt_sus_usb_groups),
+ FUNCTION("spi", byt_sus_spi_groups),
+ FUNCTION("gpio", byt_sus_gpio_groups),
+};
+
+static const struct byt_community byt_sus_communities[] = {
+ COMMUNITY(0, BYT_NGPIO_SUS, byt_sus_pins_map),
+};
+
+static const struct byt_pinctrl_soc_data byt_sus_soc_data = {
+ .uid = BYT_SUS_ACPI_UID,
+ .pins = byt_sus_pins,
+ .npins = ARRAY_SIZE(byt_sus_pins),
+ .groups = byt_sus_groups,
+ .ngroups = ARRAY_SIZE(byt_sus_groups),
+ .functions = byt_sus_functions,
+ .nfunctions = ARRAY_SIZE(byt_sus_functions),
+ .communities = byt_sus_communities,
+ .ncommunities = ARRAY_SIZE(byt_sus_communities),
+};
+
+static const struct pinctrl_pin_desc byt_ncore_pins[] = {
+ PINCTRL_PIN(0, "GPIO_NCORE0"),
+ PINCTRL_PIN(1, "GPIO_NCORE1"),
+ PINCTRL_PIN(2, "GPIO_NCORE2"),
+ PINCTRL_PIN(3, "GPIO_NCORE3"),
+ PINCTRL_PIN(4, "GPIO_NCORE4"),
+ PINCTRL_PIN(5, "GPIO_NCORE5"),
+ PINCTRL_PIN(6, "GPIO_NCORE6"),
+ PINCTRL_PIN(7, "GPIO_NCORE7"),
+ PINCTRL_PIN(8, "GPIO_NCORE8"),
+ PINCTRL_PIN(9, "GPIO_NCORE9"),
+ PINCTRL_PIN(10, "GPIO_NCORE10"),
+ PINCTRL_PIN(11, "GPIO_NCORE11"),
+ PINCTRL_PIN(12, "GPIO_NCORE12"),
+ PINCTRL_PIN(13, "GPIO_NCORE13"),
+ PINCTRL_PIN(14, "GPIO_NCORE14"),
+ PINCTRL_PIN(15, "GPIO_NCORE15"),
+ PINCTRL_PIN(16, "GPIO_NCORE16"),
+ PINCTRL_PIN(17, "GPIO_NCORE17"),
+ PINCTRL_PIN(18, "GPIO_NCORE18"),
+ PINCTRL_PIN(19, "GPIO_NCORE19"),
+ PINCTRL_PIN(20, "GPIO_NCORE20"),
+ PINCTRL_PIN(21, "GPIO_NCORE21"),
+ PINCTRL_PIN(22, "GPIO_NCORE22"),
+ PINCTRL_PIN(23, "GPIO_NCORE23"),
+ PINCTRL_PIN(24, "GPIO_NCORE24"),
+ PINCTRL_PIN(25, "GPIO_NCORE25"),
+ PINCTRL_PIN(26, "GPIO_NCORE26"),
+ PINCTRL_PIN(27, "GPIO_NCORE27"),
+};
+
+static unsigned const byt_ncore_pins_map[BYT_NGPIO_NCORE] = {
+ 19, 18, 17, 20, 21, 22, 24, 25, 23, 16,
+ 14, 15, 12, 26, 27, 1, 4, 8, 11, 0,
+ 3, 6, 10, 13, 2, 5, 9, 7,
+};
+
+static const struct byt_community byt_ncore_communities[] = {
+ COMMUNITY(0, BYT_NGPIO_NCORE, byt_ncore_pins_map),
+};
+
+static const struct byt_pinctrl_soc_data byt_ncore_soc_data = {
+ .uid = BYT_NCORE_ACPI_UID,
+ .pins = byt_ncore_pins,
+ .npins = ARRAY_SIZE(byt_ncore_pins),
+ .communities = byt_ncore_communities,
+ .ncommunities = ARRAY_SIZE(byt_ncore_communities),
+};
+
+static const struct byt_pinctrl_soc_data *byt_soc_data[] = {
+ &byt_score_soc_data,
+ &byt_sus_soc_data,
+ &byt_ncore_soc_data,
+ NULL,
+};
+
+static struct byt_community *byt_get_community(struct byt_gpio *vg,
+ unsigned int pin)
{
- struct byt_gpio *vg = gpiochip_get_data(chip);
- u32 reg_offset;
+ struct byt_community *comm;
+ int i;
+ for (i = 0; i < vg->soc_data->ncommunities; i++) {
+ comm = vg->communities_copy + i;
+ if (pin < comm->pin_base + comm->npins && pin >= comm->pin_base)
+ return comm;
+ }
+
+ return NULL;
+}
+
+static void __iomem *byt_gpio_reg(struct byt_gpio *vg, unsigned int offset,
+ int reg)
+{
+ struct byt_community *comm = byt_get_community(vg, offset);
+ u32 reg_offset = 0;
+
+ if (!comm)
+ return NULL;
+
+ offset -= comm->pin_base;
if (reg == BYT_INT_STAT_REG)
reg_offset = (offset / 32) * 4;
else
- reg_offset = vg->range->pins[offset] * 16;
+ reg_offset = comm->pad_map[offset] * 16;
- return vg->reg_base + reg_offset + reg;
+ return comm->reg_base + reg_offset + reg;
}
-static void byt_gpio_clear_triggering(struct byt_gpio *vg, unsigned offset)
+static int byt_get_groups_count(struct pinctrl_dev *pctldev)
{
- void __iomem *reg = byt_gpio_reg(&vg->chip, offset, BYT_CONF0_REG);
+ struct byt_gpio *vg = pinctrl_dev_get_drvdata(pctldev);
+
+ return vg->soc_data->ngroups;
+}
+
+static const char *byt_get_group_name(struct pinctrl_dev *pctldev,
+ unsigned int selector)
+{
+ struct byt_gpio *vg = pinctrl_dev_get_drvdata(pctldev);
+
+ return vg->soc_data->groups[selector].name;
+}
+
+static int byt_get_group_pins(struct pinctrl_dev *pctldev,
+ unsigned int selector,
+ const unsigned int **pins,
+ unsigned int *num_pins)
+{
+ struct byt_gpio *vg = pinctrl_dev_get_drvdata(pctldev);
+
+ *pins = vg->soc_data->groups[selector].pins;
+ *num_pins = vg->soc_data->groups[selector].npins;
+
+ return 0;
+}
+
+static const struct pinctrl_ops byt_pinctrl_ops = {
+ .get_groups_count = byt_get_groups_count,
+ .get_group_name = byt_get_group_name,
+ .get_group_pins = byt_get_group_pins,
+};
+
+static int byt_get_functions_count(struct pinctrl_dev *pctldev)
+{
+ struct byt_gpio *vg = pinctrl_dev_get_drvdata(pctldev);
+
+ return vg->soc_data->nfunctions;
+}
+
+static const char *byt_get_function_name(struct pinctrl_dev *pctldev,
+ unsigned int selector)
+{
+ struct byt_gpio *vg = pinctrl_dev_get_drvdata(pctldev);
+
+ return vg->soc_data->functions[selector].name;
+}
+
+static int byt_get_function_groups(struct pinctrl_dev *pctldev,
+ unsigned int selector,
+ const char * const **groups,
+ unsigned int *num_groups)
+{
+ struct byt_gpio *vg = pinctrl_dev_get_drvdata(pctldev);
+
+ *groups = vg->soc_data->functions[selector].groups;
+ *num_groups = vg->soc_data->functions[selector].ngroups;
+
+ return 0;
+}
+
+static int byt_get_group_simple_mux(const struct byt_pingroup group,
+ const char *func_name,
+ unsigned short *func)
+{
+ int i;
+
+ for (i = 0; i < group.nfuncs; i++) {
+ if (!strcmp(group.simple_funcs[i].name, func_name)) {
+ *func = group.simple_funcs[i].func;
+ return 0;
+ }
+ }
+
+ return 1;
+}
+
+static int byt_get_group_mixed_mux(const struct byt_pingroup group,
+ const char *func_name,
+ const unsigned short **func)
+{
+ int i;
+
+ for (i = 0; i < group.nfuncs; i++) {
+ if (!strcmp(group.mixed_funcs[i].name, func_name)) {
+ *func = group.mixed_funcs[i].func_values;
+ return 0;
+ }
+ }
+
+ return 1;
+}
+
+static void byt_set_group_simple_mux(struct byt_gpio *vg,
+ const struct byt_pingroup group,
+ unsigned short func)
+{
+ unsigned long flags;
+ int i;
+
+ raw_spin_lock_irqsave(&vg->lock, flags);
+
+ for (i = 0; i < group.npins; i++) {
+ void __iomem *padcfg0;
+ u32 value;
+
+ padcfg0 = byt_gpio_reg(vg, group.pins[i], BYT_CONF0_REG);
+ if (!padcfg0) {
+ dev_warn(&vg->pdev->dev,
+ "Group %s, pin %i not muxed (no padcfg0)\n",
+ group.name, i);
+ continue;
+ }
+
+ value = readl(padcfg0);
+ value &= ~BYT_PIN_MUX;
+ value |= func;
+ writel(value, padcfg0);
+ }
+
+ raw_spin_unlock_irqrestore(&vg->lock, flags);
+}
+
+static void byt_set_group_mixed_mux(struct byt_gpio *vg,
+ const struct byt_pingroup group,
+ const unsigned short *func)
+{
+ unsigned long flags;
+ int i;
+
+ raw_spin_lock_irqsave(&vg->lock, flags);
+
+ for (i = 0; i < group.npins; i++) {
+ void __iomem *padcfg0;
+ u32 value;
+
+ padcfg0 = byt_gpio_reg(vg, group.pins[i], BYT_CONF0_REG);
+ if (!padcfg0) {
+ dev_warn(&vg->pdev->dev,
+ "Group %s, pin %i not muxed (no padcfg0)\n",
+ group.name, i);
+ continue;
+ }
+
+ value = readl(padcfg0);
+ value &= ~BYT_PIN_MUX;
+ value |= func[i];
+ writel(value, padcfg0);
+ }
+
+ raw_spin_unlock_irqrestore(&vg->lock, flags);
+}
+
+static int byt_set_mux(struct pinctrl_dev *pctldev, unsigned int func_selector,
+ unsigned int group_selector)
+{
+ struct byt_gpio *vg = pinctrl_dev_get_drvdata(pctldev);
+ const struct byt_function func = vg->soc_data->functions[func_selector];
+ const struct byt_pingroup group = vg->soc_data->groups[group_selector];
+ const unsigned short *mixed_func;
+ unsigned short simple_func;
+ int ret = 1;
+
+ if (group.has_simple_funcs)
+ ret = byt_get_group_simple_mux(group, func.name, &simple_func);
+ else
+ ret = byt_get_group_mixed_mux(group, func.name, &mixed_func);
+
+ if (ret)
+ byt_set_group_simple_mux(vg, group, BYT_DEFAULT_GPIO_MUX);
+ else if (group.has_simple_funcs)
+ byt_set_group_simple_mux(vg, group, simple_func);
+ else
+ byt_set_group_mixed_mux(vg, group, mixed_func);
+
+ return 0;
+}
+
+static u32 byt_get_gpio_mux(struct byt_gpio *vg, unsigned offset)
+{
+ /* SCORE pin 92-93 */
+ if (!strcmp(vg->soc_data->uid, BYT_SCORE_ACPI_UID) &&
+ offset >= 92 && offset <= 93)
+ return 1;
+
+ /* SUS pin 11-21 */
+ if (!strcmp(vg->soc_data->uid, BYT_SUS_ACPI_UID) &&
+ offset >= 11 && offset <= 21)
+ return 1;
+
+ return 0;
+}
+
+static void byt_gpio_clear_triggering(struct byt_gpio *vg, unsigned int offset)
+{
+ void __iomem *reg = byt_gpio_reg(vg, offset, BYT_CONF0_REG);
unsigned long flags;
u32 value;
@@ -174,25 +949,12 @@
raw_spin_unlock_irqrestore(&vg->lock, flags);
}
-static u32 byt_get_gpio_mux(struct byt_gpio *vg, unsigned offset)
+static int byt_gpio_request_enable(struct pinctrl_dev *pctl_dev,
+ struct pinctrl_gpio_range *range,
+ unsigned int offset)
{
- /* SCORE pin 92-93 */
- if (!strcmp(vg->range->name, BYT_SCORE_ACPI_UID) &&
- offset >= 92 && offset <= 93)
- return 1;
-
- /* SUS pin 11-21 */
- if (!strcmp(vg->range->name, BYT_SUS_ACPI_UID) &&
- offset >= 11 && offset <= 21)
- return 1;
-
- return 0;
-}
-
-static int byt_gpio_request(struct gpio_chip *chip, unsigned offset)
-{
- struct byt_gpio *vg = gpiochip_get_data(chip);
- void __iomem *reg = byt_gpio_reg(chip, offset, BYT_CONF0_REG);
+ struct byt_gpio *vg = pinctrl_dev_get_drvdata(pctl_dev);
+ void __iomem *reg = byt_gpio_reg(vg, offset, BYT_CONF0_REG);
u32 value, gpio_mux;
unsigned long flags;
@@ -225,53 +987,318 @@
return 0;
}
-static void byt_gpio_free(struct gpio_chip *chip, unsigned offset)
+static void byt_gpio_disable_free(struct pinctrl_dev *pctl_dev,
+ struct pinctrl_gpio_range *range,
+ unsigned int offset)
{
- struct byt_gpio *vg = gpiochip_get_data(chip);
+ struct byt_gpio *vg = pinctrl_dev_get_drvdata(pctl_dev);
byt_gpio_clear_triggering(vg, offset);
pm_runtime_put(&vg->pdev->dev);
}
-static int byt_irq_type(struct irq_data *d, unsigned type)
+static int byt_gpio_set_direction(struct pinctrl_dev *pctl_dev,
+ struct pinctrl_gpio_range *range,
+ unsigned int offset,
+ bool input)
{
- struct byt_gpio *vg = gpiochip_get_data(irq_data_get_irq_chip_data(d));
- u32 offset = irqd_to_hwirq(d);
- u32 value;
+ struct byt_gpio *vg = pinctrl_dev_get_drvdata(pctl_dev);
+ void __iomem *val_reg = byt_gpio_reg(vg, offset, BYT_VAL_REG);
+ void __iomem *conf_reg = byt_gpio_reg(vg, offset, BYT_CONF0_REG);
unsigned long flags;
- void __iomem *reg = byt_gpio_reg(&vg->chip, offset, BYT_CONF0_REG);
-
- if (offset >= vg->chip.ngpio)
- return -EINVAL;
+ u32 value;
raw_spin_lock_irqsave(&vg->lock, flags);
- value = readl(reg);
- WARN(value & BYT_DIRECT_IRQ_EN,
- "Bad pad config for io mode, force direct_irq_en bit clearing");
-
- /* For level trigges the BYT_TRIG_POS and BYT_TRIG_NEG bits
- * are used to indicate high and low level triggering
- */
- value &= ~(BYT_DIRECT_IRQ_EN | BYT_TRIG_POS | BYT_TRIG_NEG |
- BYT_TRIG_LVL);
-
- writel(value, reg);
-
- if (type & IRQ_TYPE_EDGE_BOTH)
- irq_set_handler_locked(d, handle_edge_irq);
- else if (type & IRQ_TYPE_LEVEL_MASK)
- irq_set_handler_locked(d, handle_level_irq);
+ value = readl(val_reg);
+ value &= ~BYT_DIR_MASK;
+ if (input)
+ value |= BYT_OUTPUT_EN;
+ else
+ /*
+ * Before making any direction modifications, do a check if gpio
+ * is set for direct IRQ. On baytrail, setting GPIO to output
+ * does not make sense, so let's at least warn the caller before
+ * they shoot themselves in the foot.
+ */
+ WARN(readl(conf_reg) & BYT_DIRECT_IRQ_EN,
+ "Potential Error: Setting GPIO with direct_irq_en to output");
+ writel(value, val_reg);
raw_spin_unlock_irqrestore(&vg->lock, flags);
return 0;
}
+static const struct pinmux_ops byt_pinmux_ops = {
+ .get_functions_count = byt_get_functions_count,
+ .get_function_name = byt_get_function_name,
+ .get_function_groups = byt_get_function_groups,
+ .set_mux = byt_set_mux,
+ .gpio_request_enable = byt_gpio_request_enable,
+ .gpio_disable_free = byt_gpio_disable_free,
+ .gpio_set_direction = byt_gpio_set_direction,
+};
+
+static void byt_get_pull_strength(u32 reg, u16 *strength)
+{
+ switch (reg & BYT_PULL_STR_MASK) {
+ case BYT_PULL_STR_2K:
+ *strength = 2000;
+ break;
+ case BYT_PULL_STR_10K:
+ *strength = 10000;
+ break;
+ case BYT_PULL_STR_20K:
+ *strength = 20000;
+ break;
+ case BYT_PULL_STR_40K:
+ *strength = 40000;
+ break;
+ }
+}
+
+static int byt_set_pull_strength(u32 *reg, u16 strength)
+{
+ *reg &= ~BYT_PULL_STR_MASK;
+
+ switch (strength) {
+ case 2000:
+ *reg |= BYT_PULL_STR_2K;
+ break;
+ case 10000:
+ *reg |= BYT_PULL_STR_10K;
+ break;
+ case 20000:
+ *reg |= BYT_PULL_STR_20K;
+ break;
+ case 40000:
+ *reg |= BYT_PULL_STR_40K;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int byt_pin_config_get(struct pinctrl_dev *pctl_dev, unsigned int offset,
+ unsigned long *config)
+{
+ struct byt_gpio *vg = pinctrl_dev_get_drvdata(pctl_dev);
+ enum pin_config_param param = pinconf_to_config_param(*config);
+ void __iomem *conf_reg = byt_gpio_reg(vg, offset, BYT_CONF0_REG);
+ void __iomem *val_reg = byt_gpio_reg(vg, offset, BYT_VAL_REG);
+ unsigned long flags;
+ u32 conf, pull, val, debounce;
+ u16 arg = 0;
+
+ raw_spin_lock_irqsave(&vg->lock, flags);
+ conf = readl(conf_reg);
+ pull = conf & BYT_PULL_ASSIGN_MASK;
+ val = readl(val_reg);
+ raw_spin_unlock_irqrestore(&vg->lock, flags);
+
+ switch (param) {
+ case PIN_CONFIG_BIAS_DISABLE:
+ if (pull)
+ return -EINVAL;
+ break;
+ case PIN_CONFIG_BIAS_PULL_DOWN:
+ /* Pull assignment is only applicable in input mode */
+ if ((val & BYT_INPUT_EN) || pull != BYT_PULL_ASSIGN_DOWN)
+ return -EINVAL;
+
+ byt_get_pull_strength(conf, &arg);
+
+ break;
+ case PIN_CONFIG_BIAS_PULL_UP:
+ /* Pull assignment is only applicable in input mode */
+ if ((val & BYT_INPUT_EN) || pull != BYT_PULL_ASSIGN_UP)
+ return -EINVAL;
+
+ byt_get_pull_strength(conf, &arg);
+
+ break;
+ case PIN_CONFIG_INPUT_DEBOUNCE:
+ if (!(conf & BYT_DEBOUNCE_EN))
+ return -EINVAL;
+
+ raw_spin_lock_irqsave(&vg->lock, flags);
+ debounce = readl(byt_gpio_reg(vg, offset, BYT_DEBOUNCE_REG));
+ raw_spin_unlock_irqrestore(&vg->lock, flags);
+
+ switch (debounce & BYT_DEBOUNCE_PULSE_MASK) {
+ case BYT_DEBOUNCE_PULSE_375US:
+ arg = 375;
+ break;
+ case BYT_DEBOUNCE_PULSE_750US:
+ arg = 750;
+ break;
+ case BYT_DEBOUNCE_PULSE_1500US:
+ arg = 1500;
+ break;
+ case BYT_DEBOUNCE_PULSE_3MS:
+ arg = 3000;
+ break;
+ case BYT_DEBOUNCE_PULSE_6MS:
+ arg = 6000;
+ break;
+ case BYT_DEBOUNCE_PULSE_12MS:
+ arg = 12000;
+ break;
+ case BYT_DEBOUNCE_PULSE_24MS:
+ arg = 24000;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ break;
+ default:
+ return -ENOTSUPP;
+ }
+
+ *config = pinconf_to_config_packed(param, arg);
+
+ return 0;
+}
+
+static int byt_pin_config_set(struct pinctrl_dev *pctl_dev,
+ unsigned int offset,
+ unsigned long *configs,
+ unsigned int num_configs)
+{
+ struct byt_gpio *vg = pinctrl_dev_get_drvdata(pctl_dev);
+ unsigned int param, arg;
+ void __iomem *conf_reg = byt_gpio_reg(vg, offset, BYT_CONF0_REG);
+ void __iomem *val_reg = byt_gpio_reg(vg, offset, BYT_VAL_REG);
+ unsigned long flags;
+ u32 conf, val, debounce;
+ int i, ret = 0;
+
+ raw_spin_lock_irqsave(&vg->lock, flags);
+
+ conf = readl(conf_reg);
+ val = readl(val_reg);
+
+ for (i = 0; i < num_configs; i++) {
+ param = pinconf_to_config_param(configs[i]);
+ arg = pinconf_to_config_argument(configs[i]);
+
+ switch (param) {
+ case PIN_CONFIG_BIAS_DISABLE:
+ conf &= ~BYT_PULL_ASSIGN_MASK;
+ break;
+ case PIN_CONFIG_BIAS_PULL_DOWN:
+ /* Set default strength value in case none is given */
+ if (arg == 1)
+ arg = 2000;
+
+ /*
+ * Pull assignment is only applicable in input mode. If
+ * chip is not in input mode, set it and warn about it.
+ */
+ if (val & BYT_INPUT_EN) {
+ val &= ~BYT_INPUT_EN;
+ writel(val, val_reg);
+ dev_warn(&vg->pdev->dev,
+ "pin %u forcibly set to input mode\n",
+ offset);
+ }
+
+ conf &= ~BYT_PULL_ASSIGN_MASK;
+ conf |= BYT_PULL_ASSIGN_DOWN;
+ ret = byt_set_pull_strength(&conf, arg);
+
+ break;
+ case PIN_CONFIG_BIAS_PULL_UP:
+ /* Set default strength value in case none is given */
+ if (arg == 1)
+ arg = 2000;
+
+ /*
+ * Pull assignment is only applicable in input mode. If
+ * chip is not in input mode, set it and warn about it.
+ */
+ if (val & BYT_INPUT_EN) {
+ val &= ~BYT_INPUT_EN;
+ writel(val, val_reg);
+ dev_warn(&vg->pdev->dev,
+ "pin %u forcibly set to input mode\n",
+ offset);
+ }
+
+ conf &= ~BYT_PULL_ASSIGN_MASK;
+ conf |= BYT_PULL_ASSIGN_UP;
+ ret = byt_set_pull_strength(&conf, arg);
+
+ break;
+ case PIN_CONFIG_INPUT_DEBOUNCE:
+ debounce = readl(byt_gpio_reg(vg, offset,
+ BYT_DEBOUNCE_REG));
+ conf &= ~BYT_DEBOUNCE_PULSE_MASK;
+
+ switch (arg) {
+ case 375:
+ conf |= BYT_DEBOUNCE_PULSE_375US;
+ break;
+ case 750:
+ conf |= BYT_DEBOUNCE_PULSE_750US;
+ break;
+ case 1500:
+ conf |= BYT_DEBOUNCE_PULSE_1500US;
+ break;
+ case 3000:
+ conf |= BYT_DEBOUNCE_PULSE_3MS;
+ break;
+ case 6000:
+ conf |= BYT_DEBOUNCE_PULSE_6MS;
+ break;
+ case 12000:
+ conf |= BYT_DEBOUNCE_PULSE_12MS;
+ break;
+ case 24000:
+ conf |= BYT_DEBOUNCE_PULSE_24MS;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+
+ break;
+ default:
+ ret = -ENOTSUPP;
+ }
+
+ if (ret)
+ break;
+ }
+
+ if (!ret)
+ writel(conf, conf_reg);
+
+ raw_spin_unlock_irqrestore(&vg->lock, flags);
+
+ return ret;
+}
+
+static const struct pinconf_ops byt_pinconf_ops = {
+ .is_generic = true,
+ .pin_config_get = byt_pin_config_get,
+ .pin_config_set = byt_pin_config_set,
+};
+
+static const struct pinctrl_desc byt_pinctrl_desc = {
+ .pctlops = &byt_pinctrl_ops,
+ .pmxops = &byt_pinmux_ops,
+ .confops = &byt_pinconf_ops,
+ .owner = THIS_MODULE,
+};
+
static int byt_gpio_get(struct gpio_chip *chip, unsigned offset)
{
- void __iomem *reg = byt_gpio_reg(chip, offset, BYT_VAL_REG);
struct byt_gpio *vg = gpiochip_get_data(chip);
+ void __iomem *reg = byt_gpio_reg(vg, offset, BYT_VAL_REG);
unsigned long flags;
u32 val;
@@ -285,69 +1312,58 @@
static void byt_gpio_set(struct gpio_chip *chip, unsigned offset, int value)
{
struct byt_gpio *vg = gpiochip_get_data(chip);
- void __iomem *reg = byt_gpio_reg(chip, offset, BYT_VAL_REG);
+ void __iomem *reg = byt_gpio_reg(vg, offset, BYT_VAL_REG);
unsigned long flags;
u32 old_val;
+ if (!reg)
+ return;
+
raw_spin_lock_irqsave(&vg->lock, flags);
-
old_val = readl(reg);
-
if (value)
writel(old_val | BYT_LEVEL, reg);
else
writel(old_val & ~BYT_LEVEL, reg);
-
raw_spin_unlock_irqrestore(&vg->lock, flags);
}
-static int byt_gpio_direction_input(struct gpio_chip *chip, unsigned offset)
+static int byt_gpio_get_direction(struct gpio_chip *chip, unsigned int offset)
{
struct byt_gpio *vg = gpiochip_get_data(chip);
- void __iomem *reg = byt_gpio_reg(chip, offset, BYT_VAL_REG);
+ void __iomem *reg = byt_gpio_reg(vg, offset, BYT_VAL_REG);
unsigned long flags;
u32 value;
+ if (!reg)
+ return -EINVAL;
+
raw_spin_lock_irqsave(&vg->lock, flags);
-
- value = readl(reg) | BYT_DIR_MASK;
- value &= ~BYT_INPUT_EN; /* active low */
- writel(value, reg);
-
+ value = readl(reg);
raw_spin_unlock_irqrestore(&vg->lock, flags);
- return 0;
+ if (!(value & BYT_OUTPUT_EN))
+ return GPIOF_DIR_OUT;
+ if (!(value & BYT_INPUT_EN))
+ return GPIOF_DIR_IN;
+
+ return -EINVAL;
+}
+
+static int byt_gpio_direction_input(struct gpio_chip *chip, unsigned int offset)
+{
+ return pinctrl_gpio_direction_input(chip->base + offset);
}
static int byt_gpio_direction_output(struct gpio_chip *chip,
- unsigned gpio, int value)
+ unsigned int offset, int value)
{
- struct byt_gpio *vg = gpiochip_get_data(chip);
- void __iomem *conf_reg = byt_gpio_reg(chip, gpio, BYT_CONF0_REG);
- void __iomem *reg = byt_gpio_reg(chip, gpio, BYT_VAL_REG);
- unsigned long flags;
- u32 reg_val;
+ int ret = pinctrl_gpio_direction_output(chip->base + offset);
- raw_spin_lock_irqsave(&vg->lock, flags);
+ if (ret)
+ return ret;
- /*
- * Before making any direction modifications, do a check if gpio
- * is set for direct IRQ. On baytrail, setting GPIO to output does
- * not make sense, so let's at least warn the caller before they shoot
- * themselves in the foot.
- */
- WARN(readl(conf_reg) & BYT_DIRECT_IRQ_EN,
- "Potential Error: Setting GPIO with direct_irq_en to output");
-
- reg_val = readl(reg) | BYT_DIR_MASK;
- reg_val &= ~(BYT_OUTPUT_EN | BYT_INPUT_EN);
-
- if (value)
- writel(reg_val | BYT_LEVEL, reg);
- else
- writel(reg_val & ~BYT_LEVEL, reg);
-
- raw_spin_unlock_irqrestore(&vg->lock, flags);
+ byt_gpio_set(chip, offset, value);
return 0;
}
@@ -356,20 +1372,45 @@
{
struct byt_gpio *vg = gpiochip_get_data(chip);
int i;
- u32 conf0, val, offs;
+ u32 conf0, val;
- for (i = 0; i < vg->chip.ngpio; i++) {
+ for (i = 0; i < vg->soc_data->npins; i++) {
+ const struct byt_community *comm;
const char *pull_str = NULL;
const char *pull = NULL;
+ void __iomem *reg;
unsigned long flags;
const char *label;
- offs = vg->range->pins[i] * 16;
+ unsigned int pin;
raw_spin_lock_irqsave(&vg->lock, flags);
- conf0 = readl(vg->reg_base + offs + BYT_CONF0_REG);
- val = readl(vg->reg_base + offs + BYT_VAL_REG);
+ pin = vg->soc_data->pins[i].number;
+ reg = byt_gpio_reg(vg, pin, BYT_CONF0_REG);
+ if (!reg) {
+ seq_printf(s,
+ "Could not retrieve pin %i conf0 reg\n",
+ pin);
+ raw_spin_unlock_irqrestore(&vg->lock, flags);
+ continue;
+ }
+ conf0 = readl(reg);
+
+ reg = byt_gpio_reg(vg, pin, BYT_VAL_REG);
+ if (!reg) {
+ seq_printf(s,
+ "Could not retrieve pin %i val reg\n", pin);
+ raw_spin_unlock_irqrestore(&vg->lock, flags);
+ continue;
+ }
+ val = readl(reg);
raw_spin_unlock_irqrestore(&vg->lock, flags);
+ comm = byt_get_community(vg, pin);
+ if (!comm) {
+ seq_printf(s,
+ "Could not get community for pin %i\n", pin);
+ continue;
+ }
label = gpiochip_is_requested(chip, i);
if (!label)
label = "Unrequested";
@@ -400,12 +1441,12 @@
seq_printf(s,
" gpio-%-3d (%-20.20s) %s %s %s pad-%-3d offset:0x%03x mux:%d %s%s%s",
- i,
+ pin,
label,
val & BYT_INPUT_EN ? " " : "in",
val & BYT_OUTPUT_EN ? " " : "out",
val & BYT_LEVEL ? "hi" : "lo",
- vg->range->pins[i], offs,
+ comm->pad_map[i], comm->pad_map[i] * 32,
conf0 & 0x7,
conf0 & BYT_TRIG_NEG ? " fall" : " ",
conf0 & BYT_TRIG_POS ? " rise" : " ",
@@ -423,27 +1464,17 @@
}
}
-static void byt_gpio_irq_handler(struct irq_desc *desc)
-{
- struct irq_data *data = irq_desc_get_irq_data(desc);
- struct byt_gpio *vg = gpiochip_get_data(irq_desc_get_handler_data(desc));
- struct irq_chip *chip = irq_data_get_irq_chip(data);
- u32 base, pin;
- void __iomem *reg;
- unsigned long pending;
- unsigned virq;
-
- /* check from GPIO controller which pin triggered the interrupt */
- for (base = 0; base < vg->chip.ngpio; base += 32) {
- reg = byt_gpio_reg(&vg->chip, base, BYT_INT_STAT_REG);
- pending = readl(reg);
- for_each_set_bit(pin, &pending, 32) {
- virq = irq_find_mapping(vg->chip.irqdomain, base + pin);
- generic_handle_irq(virq);
- }
- }
- chip->irq_eoi(data);
-}
+static const struct gpio_chip byt_gpio_chip = {
+ .owner = THIS_MODULE,
+ .request = gpiochip_generic_request,
+ .free = gpiochip_generic_free,
+ .get_direction = byt_gpio_get_direction,
+ .direction_input = byt_gpio_direction_input,
+ .direction_output = byt_gpio_direction_output,
+ .get = byt_gpio_get,
+ .set = byt_gpio_set,
+ .dbg_show = byt_gpio_dbg_show,
+};
static void byt_irq_ack(struct irq_data *d)
{
@@ -452,12 +1483,23 @@
unsigned offset = irqd_to_hwirq(d);
void __iomem *reg;
+ reg = byt_gpio_reg(vg, offset, BYT_INT_STAT_REG);
+ if (!reg)
+ return;
+
raw_spin_lock(&vg->lock);
- reg = byt_gpio_reg(&vg->chip, offset, BYT_INT_STAT_REG);
writel(BIT(offset % 32), reg);
raw_spin_unlock(&vg->lock);
}
+static void byt_irq_mask(struct irq_data *d)
+{
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
+ struct byt_gpio *vg = gpiochip_get_data(gc);
+
+ byt_gpio_clear_triggering(vg, irqd_to_hwirq(d));
+}
+
static void byt_irq_unmask(struct irq_data *d)
{
struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
@@ -467,7 +1509,9 @@
void __iomem *reg;
u32 value;
- reg = byt_gpio_reg(&vg->chip, offset, BYT_CONF0_REG);
+ reg = byt_gpio_reg(vg, offset, BYT_CONF0_REG);
+ if (!reg)
+ return;
raw_spin_lock_irqsave(&vg->lock, flags);
value = readl(reg);
@@ -493,23 +1537,81 @@
raw_spin_unlock_irqrestore(&vg->lock, flags);
}
-static void byt_irq_mask(struct irq_data *d)
+static int byt_irq_type(struct irq_data *d, unsigned int type)
{
- struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
- struct byt_gpio *vg = gpiochip_get_data(gc);
+ struct byt_gpio *vg = gpiochip_get_data(irq_data_get_irq_chip_data(d));
+ u32 offset = irqd_to_hwirq(d);
+ u32 value;
+ unsigned long flags;
+ void __iomem *reg = byt_gpio_reg(vg, offset, BYT_CONF0_REG);
- byt_gpio_clear_triggering(vg, irqd_to_hwirq(d));
+ if (!reg || offset >= vg->chip.ngpio)
+ return -EINVAL;
+
+ raw_spin_lock_irqsave(&vg->lock, flags);
+ value = readl(reg);
+
+ WARN(value & BYT_DIRECT_IRQ_EN,
+ "Bad pad config for io mode, force direct_irq_en bit clearing");
+
+ /* For level trigges the BYT_TRIG_POS and BYT_TRIG_NEG bits
+ * are used to indicate high and low level triggering
+ */
+ value &= ~(BYT_DIRECT_IRQ_EN | BYT_TRIG_POS | BYT_TRIG_NEG |
+ BYT_TRIG_LVL);
+
+ writel(value, reg);
+
+ if (type & IRQ_TYPE_EDGE_BOTH)
+ irq_set_handler_locked(d, handle_edge_irq);
+ else if (type & IRQ_TYPE_LEVEL_MASK)
+ irq_set_handler_locked(d, handle_level_irq);
+
+ raw_spin_unlock_irqrestore(&vg->lock, flags);
+
+ return 0;
}
static struct irq_chip byt_irqchip = {
- .name = "BYT-GPIO",
- .irq_ack = byt_irq_ack,
- .irq_mask = byt_irq_mask,
- .irq_unmask = byt_irq_unmask,
- .irq_set_type = byt_irq_type,
- .flags = IRQCHIP_SKIP_SET_WAKE,
+ .name = "BYT-GPIO",
+ .irq_ack = byt_irq_ack,
+ .irq_mask = byt_irq_mask,
+ .irq_unmask = byt_irq_unmask,
+ .irq_set_type = byt_irq_type,
+ .flags = IRQCHIP_SKIP_SET_WAKE,
};
+static void byt_gpio_irq_handler(struct irq_desc *desc)
+{
+ struct irq_data *data = irq_desc_get_irq_data(desc);
+ struct byt_gpio *vg = gpiochip_get_data(
+ irq_desc_get_handler_data(desc));
+ struct irq_chip *chip = irq_data_get_irq_chip(data);
+ u32 base, pin;
+ void __iomem *reg;
+ unsigned long pending;
+ unsigned int virq;
+
+ /* check from GPIO controller which pin triggered the interrupt */
+ for (base = 0; base < vg->chip.ngpio; base += 32) {
+ reg = byt_gpio_reg(vg, base, BYT_INT_STAT_REG);
+
+ if (!reg) {
+ dev_warn(&vg->pdev->dev,
+ "Pin %i: could not retrieve interrupt status register\n",
+ base);
+ continue;
+ }
+
+ pending = readl(reg);
+ for_each_set_bit(pin, &pending, 32) {
+ virq = irq_find_mapping(vg->chip.irqdomain, base + pin);
+ generic_handle_irq(virq);
+ }
+ }
+ chip->irq_eoi(data);
+}
+
static void byt_gpio_irq_init_hw(struct byt_gpio *vg)
{
void __iomem *reg;
@@ -521,8 +1623,18 @@
* do not use direct IRQ mode. This will prevent spurious
* interrupts from misconfigured pins.
*/
- for (i = 0; i < vg->chip.ngpio; i++) {
- value = readl(byt_gpio_reg(&vg->chip, i, BYT_CONF0_REG));
+ for (i = 0; i < vg->soc_data->npins; i++) {
+ unsigned int pin = vg->soc_data->pins[i].number;
+
+ reg = byt_gpio_reg(vg, pin, BYT_CONF0_REG);
+ if (!reg) {
+ dev_warn(&vg->pdev->dev,
+ "Pin %i: could not retrieve conf0 register\n",
+ i);
+ continue;
+ }
+
+ value = readl(reg);
if ((value & BYT_PIN_MUX) == byt_get_gpio_mux(vg, i) &&
!(value & BYT_DIRECT_IRQ_EN)) {
byt_gpio_clear_triggering(vg, i);
@@ -531,8 +1643,16 @@
}
/* clear interrupt status trigger registers */
- for (base = 0; base < vg->chip.ngpio; base += 32) {
- reg = byt_gpio_reg(&vg->chip, base, BYT_INT_STAT_REG);
+ for (base = 0; base < vg->soc_data->npins; base += 32) {
+ reg = byt_gpio_reg(vg, base, BYT_INT_STAT_REG);
+
+ if (!reg) {
+ dev_warn(&vg->pdev->dev,
+ "Pin %i: could not retrieve irq status reg\n",
+ base);
+ continue;
+ }
+
writel(0xffffffff, reg);
/* make sure trigger bits are cleared, if not then a pin
might be misconfigured in bios */
@@ -543,82 +1663,47 @@
}
}
-static int byt_gpio_probe(struct platform_device *pdev)
+static int byt_gpio_probe(struct byt_gpio *vg)
{
- struct byt_gpio *vg;
struct gpio_chip *gc;
- struct resource *mem_rc, *irq_rc;
- struct device *dev = &pdev->dev;
- struct acpi_device *acpi_dev;
- struct pinctrl_gpio_range *range;
- acpi_handle handle = ACPI_HANDLE(dev);
+ struct resource *irq_rc;
int ret;
- if (acpi_bus_get_device(handle, &acpi_dev))
- return -ENODEV;
-
- vg = devm_kzalloc(dev, sizeof(struct byt_gpio), GFP_KERNEL);
- if (!vg) {
- dev_err(&pdev->dev, "can't allocate byt_gpio chip data\n");
- return -ENOMEM;
- }
-
- for (range = byt_ranges; range->name; range++) {
- if (!strcmp(acpi_dev->pnp.unique_id, range->name)) {
- vg->chip.ngpio = range->npins;
- vg->range = range;
- break;
- }
- }
-
- if (!vg->chip.ngpio || !vg->range)
- return -ENODEV;
-
- vg->pdev = pdev;
- platform_set_drvdata(pdev, vg);
-
- mem_rc = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- vg->reg_base = devm_ioremap_resource(dev, mem_rc);
- if (IS_ERR(vg->reg_base))
- return PTR_ERR(vg->reg_base);
-
- raw_spin_lock_init(&vg->lock);
-
- gc = &vg->chip;
- gc->label = dev_name(&pdev->dev);
- gc->owner = THIS_MODULE;
- gc->request = byt_gpio_request;
- gc->free = byt_gpio_free;
- gc->direction_input = byt_gpio_direction_input;
- gc->direction_output = byt_gpio_direction_output;
- gc->get = byt_gpio_get;
- gc->set = byt_gpio_set;
- gc->dbg_show = byt_gpio_dbg_show;
- gc->base = -1;
- gc->can_sleep = false;
- gc->parent = dev;
+ /* Set up gpio chip */
+ vg->chip = byt_gpio_chip;
+ gc = &vg->chip;
+ gc->label = dev_name(&vg->pdev->dev);
+ gc->base = -1;
+ gc->can_sleep = false;
+ gc->parent = &vg->pdev->dev;
+ gc->ngpio = vg->soc_data->npins;
#ifdef CONFIG_PM_SLEEP
- vg->saved_context = devm_kcalloc(&pdev->dev, gc->ngpio,
+ vg->saved_context = devm_kcalloc(&vg->pdev->dev, gc->ngpio,
sizeof(*vg->saved_context), GFP_KERNEL);
#endif
-
ret = gpiochip_add_data(gc, vg);
if (ret) {
- dev_err(&pdev->dev, "failed adding byt-gpio chip\n");
+ dev_err(&vg->pdev->dev, "failed adding byt-gpio chip\n");
return ret;
}
+ ret = gpiochip_add_pin_range(&vg->chip, dev_name(&vg->pdev->dev),
+ 0, 0, vg->soc_data->npins);
+ if (ret) {
+ dev_err(&vg->pdev->dev, "failed to add GPIO pin range\n");
+ goto fail;
+ }
+
/* set up interrupts */
- irq_rc = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
+ irq_rc = platform_get_resource(vg->pdev, IORESOURCE_IRQ, 0);
if (irq_rc && irq_rc->start) {
byt_gpio_irq_init_hw(vg);
ret = gpiochip_irqchip_add(gc, &byt_irqchip, 0,
handle_simple_irq, IRQ_TYPE_NONE);
if (ret) {
- dev_err(dev, "failed to add irqchip\n");
- gpiochip_remove(gc);
- return ret;
+ dev_err(&vg->pdev->dev, "failed to add irqchip\n");
+ goto fail;
}
gpiochip_set_chained_irqchip(gc, &byt_irqchip,
@@ -626,7 +1711,120 @@
byt_gpio_irq_handler);
}
- pm_runtime_enable(dev);
+ return ret;
+
+fail:
+ gpiochip_remove(&vg->chip);
+
+ return ret;
+}
+
+static int byt_set_soc_data(struct byt_gpio *vg,
+ const struct byt_pinctrl_soc_data *soc_data)
+{
+ int i;
+
+ vg->soc_data = soc_data;
+ vg->communities_copy = devm_kcalloc(&vg->pdev->dev,
+ soc_data->ncommunities,
+ sizeof(*vg->communities_copy),
+ GFP_KERNEL);
+ if (!vg->communities_copy)
+ return -ENOMEM;
+
+ for (i = 0; i < soc_data->ncommunities; i++) {
+ struct byt_community *comm = vg->communities_copy + i;
+ struct resource *mem_rc;
+
+ *comm = vg->soc_data->communities[i];
+
+ mem_rc = platform_get_resource(vg->pdev, IORESOURCE_MEM, 0);
+ comm->reg_base = devm_ioremap_resource(&vg->pdev->dev, mem_rc);
+ if (IS_ERR(comm->reg_base))
+ return PTR_ERR(comm->reg_base);
+ }
+
+ return 0;
+}
+
+static const struct acpi_device_id byt_gpio_acpi_match[] = {
+ { "INT33B2", (kernel_ulong_t)byt_soc_data },
+ { "INT33FC", (kernel_ulong_t)byt_soc_data },
+ { }
+};
+MODULE_DEVICE_TABLE(acpi, byt_gpio_acpi_match);
+
+static int byt_pinctrl_probe(struct platform_device *pdev)
+{
+ const struct byt_pinctrl_soc_data *soc_data = NULL;
+ const struct byt_pinctrl_soc_data **soc_table;
+ const struct acpi_device_id *acpi_id;
+ struct acpi_device *acpi_dev;
+ struct byt_gpio *vg;
+ int i, ret;
+
+ acpi_dev = ACPI_COMPANION(&pdev->dev);
+ if (!acpi_dev)
+ return -ENODEV;
+
+ acpi_id = acpi_match_device(byt_gpio_acpi_match, &pdev->dev);
+ if (!acpi_id)
+ return -ENODEV;
+
+ soc_table = (const struct byt_pinctrl_soc_data **)acpi_id->driver_data;
+
+ for (i = 0; soc_table[i]; i++) {
+ if (!strcmp(acpi_dev->pnp.unique_id, soc_table[i]->uid)) {
+ soc_data = soc_table[i];
+ break;
+ }
+ }
+
+ if (!soc_data)
+ return -ENODEV;
+
+ vg = devm_kzalloc(&pdev->dev, sizeof(*vg), GFP_KERNEL);
+ if (!vg)
+ return -ENOMEM;
+
+ vg->pdev = pdev;
+ ret = byt_set_soc_data(vg, soc_data);
+ if (ret) {
+ dev_err(&pdev->dev, "failed to set soc data\n");
+ return ret;
+ }
+
+ vg->pctl_desc = byt_pinctrl_desc;
+ vg->pctl_desc.name = dev_name(&pdev->dev);
+ vg->pctl_desc.pins = vg->soc_data->pins;
+ vg->pctl_desc.npins = vg->soc_data->npins;
+
+ vg->pctl_dev = pinctrl_register(&vg->pctl_desc, &pdev->dev, vg);
+ if (IS_ERR(vg->pctl_dev)) {
+ dev_err(&pdev->dev, "failed to register pinctrl driver\n");
+ return PTR_ERR(vg->pctl_dev);
+ }
+
+ ret = byt_gpio_probe(vg);
+ if (ret) {
+ pinctrl_unregister(vg->pctl_dev);
+ return ret;
+ }
+
+ platform_set_drvdata(pdev, vg);
+ raw_spin_lock_init(&vg->lock);
+ pm_runtime_enable(&pdev->dev);
+
+ return 0;
+}
+
+static int byt_pinctrl_remove(struct platform_device *pdev)
+{
+ struct byt_gpio *vg = platform_get_drvdata(pdev);
+
+ pm_runtime_disable(&pdev->dev);
+ gpiochip_remove(&vg->chip);
+ pinctrl_unregister(vg->pctl_dev);
return 0;
}
@@ -638,15 +1836,22 @@
struct byt_gpio *vg = platform_get_drvdata(pdev);
int i;
- for (i = 0; i < vg->chip.ngpio; i++) {
+ for (i = 0; i < vg->soc_data->npins; i++) {
void __iomem *reg;
u32 value;
+ unsigned int pin = vg->soc_data->pins[i].number;
- reg = byt_gpio_reg(&vg->chip, i, BYT_CONF0_REG);
+ reg = byt_gpio_reg(vg, pin, BYT_CONF0_REG);
+ if (!reg) {
+ dev_warn(&vg->pdev->dev,
+ "Pin %i: could not retrieve conf0 register\n",
+ i);
+ continue;
+ }
value = readl(reg) & BYT_CONF0_RESTORE_MASK;
vg->saved_context[i].conf0 = value;
- reg = byt_gpio_reg(&vg->chip, i, BYT_VAL_REG);
+ reg = byt_gpio_reg(vg, pin, BYT_VAL_REG);
value = readl(reg) & BYT_VAL_RESTORE_MASK;
vg->saved_context[i].val = value;
}
@@ -660,11 +1865,18 @@
struct byt_gpio *vg = platform_get_drvdata(pdev);
int i;
- for (i = 0; i < vg->chip.ngpio; i++) {
+ for (i = 0; i < vg->soc_data->npins; i++) {
void __iomem *reg;
u32 value;
+ unsigned int pin = vg->soc_data->pins[i].number;
- reg = byt_gpio_reg(&vg->chip, i, BYT_CONF0_REG);
+ reg = byt_gpio_reg(vg, pin, BYT_CONF0_REG);
+ if (!reg) {
+ dev_warn(&vg->pdev->dev,
+ "Pin %i: could not retrieve conf0 register\n",
+ i);
+ continue;
+ }
value = readl(reg);
if ((value & BYT_CONF0_RESTORE_MASK) !=
vg->saved_context[i].conf0) {
@@ -674,7 +1886,7 @@
dev_info(dev, "restored pin %d conf0 %#08x", i, value);
}
- reg = byt_gpio_reg(&vg->chip, i, BYT_VAL_REG);
+ reg = byt_gpio_reg(vg, pin, BYT_VAL_REG);
value = readl(reg);
if ((value & BYT_VAL_RESTORE_MASK) !=
vg->saved_context[i].val) {
@@ -712,26 +1924,9 @@
NULL)
};
-static const struct acpi_device_id byt_gpio_acpi_match[] = {
- { "INT33B2", 0 },
- { "INT33FC", 0 },
- { }
-};
-MODULE_DEVICE_TABLE(acpi, byt_gpio_acpi_match);
-
-static int byt_gpio_remove(struct platform_device *pdev)
-{
- struct byt_gpio *vg = platform_get_drvdata(pdev);
-
- pm_runtime_disable(&pdev->dev);
- gpiochip_remove(&vg->chip);
-
- return 0;
-}
-
static struct platform_driver byt_gpio_driver = {
- .probe = byt_gpio_probe,
- .remove = byt_gpio_remove,
+ .probe = byt_pinctrl_probe,
+ .remove = byt_pinctrl_remove,
.driver = {
.name = "byt_gpio",
.pm = &byt_gpio_pm_ops,
diff --git a/drivers/pinctrl/intel/pinctrl-cherryview.c b/drivers/pinctrl/intel/pinctrl-cherryview.c
index 4251e07..ac4f564 100644
--- a/drivers/pinctrl/intel/pinctrl-cherryview.c
+++ b/drivers/pinctrl/intel/pinctrl-cherryview.c
@@ -1526,17 +1526,16 @@
pctrl->pctldesc.pins = pctrl->community->pins;
pctrl->pctldesc.npins = pctrl->community->npins;
- pctrl->pctldev = pinctrl_register(&pctrl->pctldesc, &pdev->dev, pctrl);
+ pctrl->pctldev = devm_pinctrl_register(&pdev->dev, &pctrl->pctldesc,
+ pctrl);
if (IS_ERR(pctrl->pctldev)) {
dev_err(&pdev->dev, "failed to register pinctrl driver\n");
return PTR_ERR(pctrl->pctldev);
}
ret = chv_gpio_probe(pctrl, irq);
- if (ret) {
- pinctrl_unregister(pctrl->pctldev);
+ if (ret)
return ret;
- }
platform_set_drvdata(pdev, pctrl);
@@ -1548,7 +1547,6 @@
struct chv_pinctrl *pctrl = platform_get_drvdata(pdev);
gpiochip_remove(&pctrl->chip);
- pinctrl_unregister(pctrl->pctldev);
return 0;
}
diff --git a/drivers/pinctrl/intel/pinctrl-intel.c b/drivers/pinctrl/intel/pinctrl-intel.c
index 6c2c816f..3584e50 100644
--- a/drivers/pinctrl/intel/pinctrl-intel.c
+++ b/drivers/pinctrl/intel/pinctrl-intel.c
@@ -1045,17 +1045,16 @@
pctrl->pctldesc.pins = pctrl->soc->pins;
pctrl->pctldesc.npins = pctrl->soc->npins;
- pctrl->pctldev = pinctrl_register(&pctrl->pctldesc, &pdev->dev, pctrl);
+ pctrl->pctldev = devm_pinctrl_register(&pdev->dev, &pctrl->pctldesc,
+ pctrl);
if (IS_ERR(pctrl->pctldev)) {
dev_err(&pdev->dev, "failed to register pinctrl driver\n");
return PTR_ERR(pctrl->pctldev);
}
ret = intel_gpio_probe(pctrl, irq);
- if (ret) {
- pinctrl_unregister(pctrl->pctldev);
+ if (ret)
return ret;
- }
platform_set_drvdata(pdev, pctrl);
@@ -1068,7 +1067,6 @@
struct intel_pinctrl *pctrl = platform_get_drvdata(pdev);
gpiochip_remove(&pctrl->chip);
- pinctrl_unregister(pctrl->pctldev);
return 0;
}
diff --git a/drivers/pinctrl/mediatek/pinctrl-mtk-common.c b/drivers/pinctrl/mediatek/pinctrl-mtk-common.c
index 6ab8c3c..207b13b 100644
--- a/drivers/pinctrl/mediatek/pinctrl-mtk-common.c
+++ b/drivers/pinctrl/mediatek/pinctrl-mtk-common.c
@@ -605,7 +605,7 @@
ret = mtk_pctrl_dt_subnode_to_map(pctldev, np, map,
&reserved_maps, num_maps);
if (ret < 0) {
- pinctrl_utils_dt_free_map(pctldev, *map, *num_maps);
+ pinctrl_utils_free_map(pctldev, *map, *num_maps);
of_node_put(np);
return ret;
}
@@ -644,7 +644,7 @@
static const struct pinctrl_ops mtk_pctrl_ops = {
.dt_node_to_map = mtk_pctrl_dt_node_to_map,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
.get_groups_count = mtk_pctrl_get_groups_count,
.get_group_name = mtk_pctrl_get_group_name,
.get_group_pins = mtk_pctrl_get_group_pins,
@@ -1396,17 +1396,16 @@
pctl->pctl_desc.pmxops = &mtk_pmx_ops;
pctl->dev = &pdev->dev;
- pctl->pctl_dev = pinctrl_register(&pctl->pctl_desc, &pdev->dev, pctl);
+ pctl->pctl_dev = devm_pinctrl_register(&pdev->dev, &pctl->pctl_desc,
+ pctl);
if (IS_ERR(pctl->pctl_dev)) {
dev_err(&pdev->dev, "couldn't register pinctrl driver\n");
return PTR_ERR(pctl->pctl_dev);
}
pctl->chip = devm_kzalloc(&pdev->dev, sizeof(*pctl->chip), GFP_KERNEL);
- if (!pctl->chip) {
- ret = -ENOMEM;
- goto pctrl_error;
- }
+ if (!pctl->chip)
+ return -ENOMEM;
*pctl->chip = mtk_gpio_chip;
pctl->chip->ngpio = pctl->devdata->npins;
@@ -1415,10 +1414,8 @@
pctl->chip->base = -1;
ret = gpiochip_add_data(pctl->chip, pctl);
- if (ret) {
- ret = -EINVAL;
- goto pctrl_error;
- }
+ if (ret)
+ return -EINVAL;
/* Register the GPIO to pin mappings. */
ret = gpiochip_add_pin_range(pctl->chip, dev_name(&pdev->dev),
@@ -1496,8 +1493,6 @@
chip_error:
gpiochip_remove(pctl->chip);
-pctrl_error:
- pinctrl_unregister(pctl->pctl_dev);
return ret;
}
diff --git a/drivers/pinctrl/meson/Makefile b/drivers/pinctrl/meson/Makefile
index c751d22..24434f1 100644
--- a/drivers/pinctrl/meson/Makefile
+++ b/drivers/pinctrl/meson/Makefile
@@ -1,2 +1,2 @@
-obj-y += pinctrl-meson8.o pinctrl-meson8b.o
+obj-y += pinctrl-meson8.o pinctrl-meson8b.o pinctrl-meson-gxbb.o
obj-y += pinctrl-meson.o
diff --git a/drivers/pinctrl/meson/pinctrl-meson-gxbb.c b/drivers/pinctrl/meson/pinctrl-meson-gxbb.c
new file mode 100644
index 0000000..eeabafb
--- /dev/null
+++ b/drivers/pinctrl/meson/pinctrl-meson-gxbb.c
@@ -0,0 +1,432 @@
+/*
+ * Pin controller and GPIO driver for Amlogic Meson GXBB.
+ *
+ * Copyright (C) 2016 Endless Mobile, Inc.
+ * Author: Carlo Caione <carlo@endlessm.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * version 2 as published by the Free Software Foundation.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <dt-bindings/gpio/meson-gxbb-gpio.h>
+#include "pinctrl-meson.h"
+
+#define EE_OFF 14
+
+static const struct pinctrl_pin_desc meson_gxbb_periphs_pins[] = {
+ MESON_PIN(GPIOZ_0, EE_OFF),
+ MESON_PIN(GPIOZ_1, EE_OFF),
+ MESON_PIN(GPIOZ_2, EE_OFF),
+ MESON_PIN(GPIOZ_3, EE_OFF),
+ MESON_PIN(GPIOZ_4, EE_OFF),
+ MESON_PIN(GPIOZ_5, EE_OFF),
+ MESON_PIN(GPIOZ_6, EE_OFF),
+ MESON_PIN(GPIOZ_7, EE_OFF),
+ MESON_PIN(GPIOZ_8, EE_OFF),
+ MESON_PIN(GPIOZ_9, EE_OFF),
+ MESON_PIN(GPIOZ_10, EE_OFF),
+ MESON_PIN(GPIOZ_11, EE_OFF),
+ MESON_PIN(GPIOZ_12, EE_OFF),
+ MESON_PIN(GPIOZ_13, EE_OFF),
+ MESON_PIN(GPIOZ_14, EE_OFF),
+ MESON_PIN(GPIOZ_15, EE_OFF),
+
+ MESON_PIN(GPIOH_0, EE_OFF),
+ MESON_PIN(GPIOH_1, EE_OFF),
+ MESON_PIN(GPIOH_2, EE_OFF),
+ MESON_PIN(GPIOH_3, EE_OFF),
+
+ MESON_PIN(BOOT_0, EE_OFF),
+ MESON_PIN(BOOT_1, EE_OFF),
+ MESON_PIN(BOOT_2, EE_OFF),
+ MESON_PIN(BOOT_3, EE_OFF),
+ MESON_PIN(BOOT_4, EE_OFF),
+ MESON_PIN(BOOT_5, EE_OFF),
+ MESON_PIN(BOOT_6, EE_OFF),
+ MESON_PIN(BOOT_7, EE_OFF),
+ MESON_PIN(BOOT_8, EE_OFF),
+ MESON_PIN(BOOT_9, EE_OFF),
+ MESON_PIN(BOOT_10, EE_OFF),
+ MESON_PIN(BOOT_11, EE_OFF),
+ MESON_PIN(BOOT_12, EE_OFF),
+ MESON_PIN(BOOT_13, EE_OFF),
+ MESON_PIN(BOOT_14, EE_OFF),
+ MESON_PIN(BOOT_15, EE_OFF),
+ MESON_PIN(BOOT_16, EE_OFF),
+ MESON_PIN(BOOT_17, EE_OFF),
+
+ MESON_PIN(CARD_0, EE_OFF),
+ MESON_PIN(CARD_1, EE_OFF),
+ MESON_PIN(CARD_2, EE_OFF),
+ MESON_PIN(CARD_3, EE_OFF),
+ MESON_PIN(CARD_4, EE_OFF),
+ MESON_PIN(CARD_5, EE_OFF),
+ MESON_PIN(CARD_6, EE_OFF),
+
+ MESON_PIN(GPIODV_0, EE_OFF),
+ MESON_PIN(GPIODV_1, EE_OFF),
+ MESON_PIN(GPIODV_2, EE_OFF),
+ MESON_PIN(GPIODV_3, EE_OFF),
+ MESON_PIN(GPIODV_4, EE_OFF),
+ MESON_PIN(GPIODV_5, EE_OFF),
+ MESON_PIN(GPIODV_6, EE_OFF),
+ MESON_PIN(GPIODV_7, EE_OFF),
+ MESON_PIN(GPIODV_8, EE_OFF),
+ MESON_PIN(GPIODV_9, EE_OFF),
+ MESON_PIN(GPIODV_10, EE_OFF),
+ MESON_PIN(GPIODV_11, EE_OFF),
+ MESON_PIN(GPIODV_12, EE_OFF),
+ MESON_PIN(GPIODV_13, EE_OFF),
+ MESON_PIN(GPIODV_14, EE_OFF),
+ MESON_PIN(GPIODV_15, EE_OFF),
+ MESON_PIN(GPIODV_16, EE_OFF),
+ MESON_PIN(GPIODV_17, EE_OFF),
+ MESON_PIN(GPIODV_19, EE_OFF),
+ MESON_PIN(GPIODV_20, EE_OFF),
+ MESON_PIN(GPIODV_21, EE_OFF),
+ MESON_PIN(GPIODV_22, EE_OFF),
+ MESON_PIN(GPIODV_23, EE_OFF),
+ MESON_PIN(GPIODV_24, EE_OFF),
+ MESON_PIN(GPIODV_25, EE_OFF),
+ MESON_PIN(GPIODV_26, EE_OFF),
+ MESON_PIN(GPIODV_27, EE_OFF),
+ MESON_PIN(GPIODV_28, EE_OFF),
+ MESON_PIN(GPIODV_29, EE_OFF),
+
+ MESON_PIN(GPIOY_0, EE_OFF),
+ MESON_PIN(GPIOY_1, EE_OFF),
+ MESON_PIN(GPIOY_2, EE_OFF),
+ MESON_PIN(GPIOY_3, EE_OFF),
+ MESON_PIN(GPIOY_4, EE_OFF),
+ MESON_PIN(GPIOY_5, EE_OFF),
+ MESON_PIN(GPIOY_6, EE_OFF),
+ MESON_PIN(GPIOY_7, EE_OFF),
+ MESON_PIN(GPIOY_8, EE_OFF),
+ MESON_PIN(GPIOY_9, EE_OFF),
+ MESON_PIN(GPIOY_10, EE_OFF),
+ MESON_PIN(GPIOY_11, EE_OFF),
+ MESON_PIN(GPIOY_12, EE_OFF),
+ MESON_PIN(GPIOY_13, EE_OFF),
+ MESON_PIN(GPIOY_14, EE_OFF),
+ MESON_PIN(GPIOY_15, EE_OFF),
+ MESON_PIN(GPIOY_16, EE_OFF),
+
+ MESON_PIN(GPIOX_0, EE_OFF),
+ MESON_PIN(GPIOX_1, EE_OFF),
+ MESON_PIN(GPIOX_2, EE_OFF),
+ MESON_PIN(GPIOX_3, EE_OFF),
+ MESON_PIN(GPIOX_4, EE_OFF),
+ MESON_PIN(GPIOX_5, EE_OFF),
+ MESON_PIN(GPIOX_6, EE_OFF),
+ MESON_PIN(GPIOX_7, EE_OFF),
+ MESON_PIN(GPIOX_8, EE_OFF),
+ MESON_PIN(GPIOX_9, EE_OFF),
+ MESON_PIN(GPIOX_10, EE_OFF),
+ MESON_PIN(GPIOX_11, EE_OFF),
+ MESON_PIN(GPIOX_12, EE_OFF),
+ MESON_PIN(GPIOX_13, EE_OFF),
+ MESON_PIN(GPIOX_14, EE_OFF),
+ MESON_PIN(GPIOX_15, EE_OFF),
+ MESON_PIN(GPIOX_16, EE_OFF),
+ MESON_PIN(GPIOX_17, EE_OFF),
+ MESON_PIN(GPIOX_18, EE_OFF),
+ MESON_PIN(GPIOX_19, EE_OFF),
+ MESON_PIN(GPIOX_20, EE_OFF),
+ MESON_PIN(GPIOX_21, EE_OFF),
+ MESON_PIN(GPIOX_22, EE_OFF),
+
+ MESON_PIN(GPIOCLK_0, EE_OFF),
+ MESON_PIN(GPIOCLK_1, EE_OFF),
+ MESON_PIN(GPIOCLK_2, EE_OFF),
+ MESON_PIN(GPIOCLK_3, EE_OFF),
+
+ MESON_PIN(GPIO_TEST_N, EE_OFF),
+};
+
+static const struct pinctrl_pin_desc meson_gxbb_aobus_pins[] = {
+ MESON_PIN(GPIOAO_0, 0),
+ MESON_PIN(GPIOAO_1, 0),
+ MESON_PIN(GPIOAO_2, 0),
+ MESON_PIN(GPIOAO_3, 0),
+ MESON_PIN(GPIOAO_4, 0),
+ MESON_PIN(GPIOAO_5, 0),
+ MESON_PIN(GPIOAO_6, 0),
+ MESON_PIN(GPIOAO_7, 0),
+ MESON_PIN(GPIOAO_8, 0),
+ MESON_PIN(GPIOAO_9, 0),
+ MESON_PIN(GPIOAO_10, 0),
+ MESON_PIN(GPIOAO_11, 0),
+ MESON_PIN(GPIOAO_12, 0),
+ MESON_PIN(GPIOAO_13, 0),
+};
+
+static const unsigned int uart_tx_ao_a_pins[] = { PIN(GPIOAO_0, 0) };
+static const unsigned int uart_rx_ao_a_pins[] = { PIN(GPIOAO_1, 0) };
+static const unsigned int uart_cts_ao_a_pins[] = { PIN(GPIOAO_2, 0) };
+static const unsigned int uart_rts_ao_a_pins[] = { PIN(GPIOAO_3, 0) };
+
+static struct meson_pmx_group meson_gxbb_periphs_groups[] = {
+ GPIO_GROUP(GPIOZ_0, EE_OFF),
+ GPIO_GROUP(GPIOZ_1, EE_OFF),
+ GPIO_GROUP(GPIOZ_2, EE_OFF),
+ GPIO_GROUP(GPIOZ_3, EE_OFF),
+ GPIO_GROUP(GPIOZ_4, EE_OFF),
+ GPIO_GROUP(GPIOZ_5, EE_OFF),
+ GPIO_GROUP(GPIOZ_6, EE_OFF),
+ GPIO_GROUP(GPIOZ_7, EE_OFF),
+ GPIO_GROUP(GPIOZ_8, EE_OFF),
+ GPIO_GROUP(GPIOZ_9, EE_OFF),
+ GPIO_GROUP(GPIOZ_10, EE_OFF),
+ GPIO_GROUP(GPIOZ_11, EE_OFF),
+ GPIO_GROUP(GPIOZ_12, EE_OFF),
+ GPIO_GROUP(GPIOZ_13, EE_OFF),
+ GPIO_GROUP(GPIOZ_14, EE_OFF),
+ GPIO_GROUP(GPIOZ_15, EE_OFF),
+
+ GPIO_GROUP(GPIOH_0, EE_OFF),
+ GPIO_GROUP(GPIOH_1, EE_OFF),
+ GPIO_GROUP(GPIOH_2, EE_OFF),
+ GPIO_GROUP(GPIOH_3, EE_OFF),
+
+ GPIO_GROUP(BOOT_0, EE_OFF),
+ GPIO_GROUP(BOOT_1, EE_OFF),
+ GPIO_GROUP(BOOT_2, EE_OFF),
+ GPIO_GROUP(BOOT_3, EE_OFF),
+ GPIO_GROUP(BOOT_4, EE_OFF),
+ GPIO_GROUP(BOOT_5, EE_OFF),
+ GPIO_GROUP(BOOT_6, EE_OFF),
+ GPIO_GROUP(BOOT_7, EE_OFF),
+ GPIO_GROUP(BOOT_8, EE_OFF),
+ GPIO_GROUP(BOOT_9, EE_OFF),
+ GPIO_GROUP(BOOT_10, EE_OFF),
+ GPIO_GROUP(BOOT_11, EE_OFF),
+ GPIO_GROUP(BOOT_12, EE_OFF),
+ GPIO_GROUP(BOOT_13, EE_OFF),
+ GPIO_GROUP(BOOT_14, EE_OFF),
+ GPIO_GROUP(BOOT_15, EE_OFF),
+ GPIO_GROUP(BOOT_16, EE_OFF),
+ GPIO_GROUP(BOOT_17, EE_OFF),
+
+ GPIO_GROUP(CARD_0, EE_OFF),
+ GPIO_GROUP(CARD_1, EE_OFF),
+ GPIO_GROUP(CARD_2, EE_OFF),
+ GPIO_GROUP(CARD_3, EE_OFF),
+ GPIO_GROUP(CARD_4, EE_OFF),
+ GPIO_GROUP(CARD_5, EE_OFF),
+ GPIO_GROUP(CARD_6, EE_OFF),
+
+ GPIO_GROUP(GPIODV_0, EE_OFF),
+ GPIO_GROUP(GPIODV_1, EE_OFF),
+ GPIO_GROUP(GPIODV_2, EE_OFF),
+ GPIO_GROUP(GPIODV_3, EE_OFF),
+ GPIO_GROUP(GPIODV_4, EE_OFF),
+ GPIO_GROUP(GPIODV_5, EE_OFF),
+ GPIO_GROUP(GPIODV_6, EE_OFF),
+ GPIO_GROUP(GPIODV_7, EE_OFF),
+ GPIO_GROUP(GPIODV_8, EE_OFF),
+ GPIO_GROUP(GPIODV_9, EE_OFF),
+ GPIO_GROUP(GPIODV_10, EE_OFF),
+ GPIO_GROUP(GPIODV_11, EE_OFF),
+ GPIO_GROUP(GPIODV_12, EE_OFF),
+ GPIO_GROUP(GPIODV_13, EE_OFF),
+ GPIO_GROUP(GPIODV_14, EE_OFF),
+ GPIO_GROUP(GPIODV_15, EE_OFF),
+ GPIO_GROUP(GPIODV_16, EE_OFF),
+ GPIO_GROUP(GPIODV_17, EE_OFF),
+ GPIO_GROUP(GPIODV_19, EE_OFF),
+ GPIO_GROUP(GPIODV_20, EE_OFF),
+ GPIO_GROUP(GPIODV_21, EE_OFF),
+ GPIO_GROUP(GPIODV_22, EE_OFF),
+ GPIO_GROUP(GPIODV_23, EE_OFF),
+ GPIO_GROUP(GPIODV_24, EE_OFF),
+ GPIO_GROUP(GPIODV_25, EE_OFF),
+ GPIO_GROUP(GPIODV_26, EE_OFF),
+ GPIO_GROUP(GPIODV_27, EE_OFF),
+ GPIO_GROUP(GPIODV_28, EE_OFF),
+ GPIO_GROUP(GPIODV_29, EE_OFF),
+
+ GPIO_GROUP(GPIOY_0, EE_OFF),
+ GPIO_GROUP(GPIOY_1, EE_OFF),
+ GPIO_GROUP(GPIOY_2, EE_OFF),
+ GPIO_GROUP(GPIOY_3, EE_OFF),
+ GPIO_GROUP(GPIOY_4, EE_OFF),
+ GPIO_GROUP(GPIOY_5, EE_OFF),
+ GPIO_GROUP(GPIOY_6, EE_OFF),
+ GPIO_GROUP(GPIOY_7, EE_OFF),
+ GPIO_GROUP(GPIOY_8, EE_OFF),
+ GPIO_GROUP(GPIOY_9, EE_OFF),
+ GPIO_GROUP(GPIOY_10, EE_OFF),
+ GPIO_GROUP(GPIOY_11, EE_OFF),
+ GPIO_GROUP(GPIOY_12, EE_OFF),
+ GPIO_GROUP(GPIOY_13, EE_OFF),
+ GPIO_GROUP(GPIOY_14, EE_OFF),
+ GPIO_GROUP(GPIOY_15, EE_OFF),
+ GPIO_GROUP(GPIOY_16, EE_OFF),
+
+ GPIO_GROUP(GPIOX_0, EE_OFF),
+ GPIO_GROUP(GPIOX_1, EE_OFF),
+ GPIO_GROUP(GPIOX_2, EE_OFF),
+ GPIO_GROUP(GPIOX_3, EE_OFF),
+ GPIO_GROUP(GPIOX_4, EE_OFF),
+ GPIO_GROUP(GPIOX_5, EE_OFF),
+ GPIO_GROUP(GPIOX_6, EE_OFF),
+ GPIO_GROUP(GPIOX_7, EE_OFF),
+ GPIO_GROUP(GPIOX_8, EE_OFF),
+ GPIO_GROUP(GPIOX_9, EE_OFF),
+ GPIO_GROUP(GPIOX_10, EE_OFF),
+ GPIO_GROUP(GPIOX_11, EE_OFF),
+ GPIO_GROUP(GPIOX_12, EE_OFF),
+ GPIO_GROUP(GPIOX_13, EE_OFF),
+ GPIO_GROUP(GPIOX_14, EE_OFF),
+ GPIO_GROUP(GPIOX_15, EE_OFF),
+ GPIO_GROUP(GPIOX_16, EE_OFF),
+ GPIO_GROUP(GPIOX_17, EE_OFF),
+ GPIO_GROUP(GPIOX_18, EE_OFF),
+ GPIO_GROUP(GPIOX_19, EE_OFF),
+ GPIO_GROUP(GPIOX_20, EE_OFF),
+ GPIO_GROUP(GPIOX_21, EE_OFF),
+ GPIO_GROUP(GPIOX_22, EE_OFF),
+
+ GPIO_GROUP(GPIOCLK_0, EE_OFF),
+ GPIO_GROUP(GPIOCLK_1, EE_OFF),
+ GPIO_GROUP(GPIOCLK_2, EE_OFF),
+ GPIO_GROUP(GPIOCLK_3, EE_OFF),
+
+ GPIO_GROUP(GPIO_TEST_N, EE_OFF),
+};
+
+static struct meson_pmx_group meson_gxbb_aobus_groups[] = {
+ GPIO_GROUP(GPIOAO_0, 0),
+ GPIO_GROUP(GPIOAO_1, 0),
+ GPIO_GROUP(GPIOAO_2, 0),
+ GPIO_GROUP(GPIOAO_3, 0),
+ GPIO_GROUP(GPIOAO_4, 0),
+ GPIO_GROUP(GPIOAO_5, 0),
+ GPIO_GROUP(GPIOAO_6, 0),
+ GPIO_GROUP(GPIOAO_7, 0),
+ GPIO_GROUP(GPIOAO_8, 0),
+ GPIO_GROUP(GPIOAO_9, 0),
+ GPIO_GROUP(GPIOAO_10, 0),
+ GPIO_GROUP(GPIOAO_11, 0),
+ GPIO_GROUP(GPIOAO_12, 0),
+ GPIO_GROUP(GPIOAO_13, 0),
+
+ /* bank AO */
+ GROUP(uart_tx_ao_a, 0, 12),
+ GROUP(uart_rx_ao_a, 0, 11),
+ GROUP(uart_cts_ao_a, 0, 10),
+ GROUP(uart_rts_ao_a, 0, 9),
+};
+
+static const char * const gpio_periphs_groups[] = {
+ "GPIOZ_0", "GPIOZ_1", "GPIOZ_2", "GPIOZ_3", "GPIOZ_4",
+ "GPIOZ_5", "GPIOZ_6", "GPIOZ_7", "GPIOZ_8", "GPIOZ_9",
+ "GPIOZ_10", "GPIOZ_11", "GPIOZ_12", "GPIOZ_13", "GPIOZ_14",
+ "GPIOZ_15",
+
+ "GPIOH_0", "GPIOH_1", "GPIOH_2", "GPIOH_3",
+
+ "BOOT_0", "BOOT_1", "BOOT_2", "BOOT_3", "BOOT_4",
+ "BOOT_5", "BOOT_6", "BOOT_7", "BOOT_8", "BOOT_9",
+ "BOOT_10", "BOOT_11", "BOOT_12", "BOOT_13", "BOOT_14",
+ "BOOT_15", "BOOT_16", "BOOT_17",
+
+ "CARD_0", "CARD_1", "CARD_2", "CARD_3", "CARD_4",
+ "CARD_5", "CARD_6",
+
+ "GPIODV_0", "GPIODV_1", "GPIODV_2", "GPIODV_3", "GPIODV_4",
+ "GPIODV_5", "GPIODV_6", "GPIODV_7", "GPIODV_8", "GPIODV_9",
+ "GPIODV_10", "GPIODV_11", "GPIODV_12", "GPIODV_13", "GPIODV_14",
+ "GPIODV_15", "GPIODV_16", "GPIODV_17", "GPIODV_18", "GPIODV_19",
+ "GPIODV_20", "GPIODV_21", "GPIODV_22", "GPIODV_23", "GPIODV_24",
+ "GPIODV_25", "GPIODV_26", "GPIODV_27", "GPIODV_28", "GPIODV_29",
+
+ "GPIOY_0", "GPIOY_1", "GPIOY_2", "GPIOY_3", "GPIOY_4",
+ "GPIOY_5", "GPIOY_6", "GPIOY_7", "GPIOY_8", "GPIOY_9",
+ "GPIOY_10", "GPIOY_11", "GPIOY_12", "GPIOY_13", "GPIOY_14",
+ "GPIOY_15", "GPIOY_16",
+
+ "GPIOX_0", "GPIOX_1", "GPIOX_2", "GPIOX_3", "GPIOX_4",
+ "GPIOX_5", "GPIOX_6", "GPIOX_7", "GPIOX_8", "GPIOX_9",
+ "GPIOX_10", "GPIOX_11", "GPIOX_12", "GPIOX_13", "GPIOX_14",
+ "GPIOX_15", "GPIOX_16", "GPIOX_17", "GPIOX_18", "GPIOX_19",
+ "GPIOX_20", "GPIOX_21", "GPIOX_22",
+
+ "GPIO_TEST_N",
+};
+
+static const char * const gpio_aobus_groups[] = {
+ "GPIOAO_0", "GPIOAO_1", "GPIOAO_2", "GPIOAO_3", "GPIOAO_4",
+ "GPIOAO_5", "GPIOAO_6", "GPIOAO_7", "GPIOAO_8", "GPIOAO_9",
+ "GPIOAO_10", "GPIOAO_11", "GPIOAO_12", "GPIOAO_13",
+};
+
+static const char * const uart_ao_groups[] = {
+ "uart_tx_ao_a", "uart_rx_ao_a", "uart_cts_ao_a", "uart_rts_ao_a"
+};
+
+static struct meson_pmx_func meson_gxbb_periphs_functions[] = {
+ FUNCTION(gpio_periphs),
+};
+
+static struct meson_pmx_func meson_gxbb_aobus_functions[] = {
+ FUNCTION(gpio_aobus),
+ FUNCTION(uart_ao),
+};
+
+static struct meson_bank meson_gxbb_periphs_banks[] = {
+ /* name first last pullen pull dir out in */
+ BANK("X", PIN(GPIOX_0, EE_OFF), PIN(GPIOX_22, EE_OFF), 4, 0, 4, 0, 12, 0, 13, 0, 14, 0),
+ BANK("Y", PIN(GPIOY_0, EE_OFF), PIN(GPIOY_16, EE_OFF), 1, 0, 1, 0, 3, 0, 4, 0, 5, 0),
+ BANK("DV", PIN(GPIODV_0, EE_OFF), PIN(GPIODV_29, EE_OFF), 0, 0, 0, 0, 0, 0, 1, 0, 2, 0),
+ BANK("H", PIN(GPIOH_0, EE_OFF), PIN(GPIOH_3, EE_OFF), 1, 20, 1, 20, 3, 20, 4, 20, 5, 20),
+ BANK("Z", PIN(GPIOZ_0, EE_OFF), PIN(GPIOZ_15, EE_OFF), 3, 0, 3, 0, 9, 0, 10, 0, 11, 0),
+ BANK("CARD", PIN(CARD_0, EE_OFF), PIN(CARD_6, EE_OFF), 2, 20, 2, 20, 6, 20, 7, 20, 8, 20),
+ BANK("BOOT", PIN(BOOT_0, EE_OFF), PIN(BOOT_17, EE_OFF), 2, 0, 2, 0, 6, 0, 7, 0, 8, 0),
+ BANK("CLK", PIN(GPIOCLK_0, EE_OFF), PIN(GPIOCLK_3, EE_OFF), 3, 28, 3, 28, 9, 28, 10, 28, 11, 28),
+};
+
+static struct meson_bank meson_gxbb_aobus_banks[] = {
+ /* name first last pullen pull dir out in */
+ BANK("AO", PIN(GPIOAO_0, 0), PIN(GPIOAO_13, 0), 0, 0, 0, 16, 0, 0, 0, 16, 1, 0),
+};
+
+static struct meson_domain_data meson_gxbb_periphs_domain_data = {
+ .name = "periphs-banks",
+ .banks = meson_gxbb_periphs_banks,
+ .num_banks = ARRAY_SIZE(meson_gxbb_periphs_banks),
+ .pin_base = 14,
+ .num_pins = 120,
+};
+
+static struct meson_domain_data meson_gxbb_aobus_domain_data = {
+ .name = "aobus-banks",
+ .banks = meson_gxbb_aobus_banks,
+ .num_banks = ARRAY_SIZE(meson_gxbb_aobus_banks),
+ .pin_base = 0,
+ .num_pins = 14,
+};
+
+struct meson_pinctrl_data meson_gxbb_periphs_pinctrl_data = {
+ .pins = meson_gxbb_periphs_pins,
+ .groups = meson_gxbb_periphs_groups,
+ .funcs = meson_gxbb_periphs_functions,
+ .domain_data = &meson_gxbb_periphs_domain_data,
+ .num_pins = ARRAY_SIZE(meson_gxbb_periphs_pins),
+ .num_groups = ARRAY_SIZE(meson_gxbb_periphs_groups),
+ .num_funcs = ARRAY_SIZE(meson_gxbb_periphs_functions),
+};
+
+struct meson_pinctrl_data meson_gxbb_aobus_pinctrl_data = {
+ .pins = meson_gxbb_aobus_pins,
+ .groups = meson_gxbb_aobus_groups,
+ .funcs = meson_gxbb_aobus_functions,
+ .domain_data = &meson_gxbb_aobus_domain_data,
+ .num_pins = ARRAY_SIZE(meson_gxbb_aobus_pins),
+ .num_groups = ARRAY_SIZE(meson_gxbb_aobus_groups),
+ .num_funcs = ARRAY_SIZE(meson_gxbb_aobus_functions),
+};
diff --git a/drivers/pinctrl/meson/pinctrl-meson.c b/drivers/pinctrl/meson/pinctrl-meson.c
index 0bdb8fd..11623c6 100644
--- a/drivers/pinctrl/meson/pinctrl-meson.c
+++ b/drivers/pinctrl/meson/pinctrl-meson.c
@@ -171,7 +171,7 @@
.get_group_name = meson_get_group_name,
.get_group_pins = meson_get_group_pins,
.dt_node_to_map = pinconf_generic_dt_node_to_map_all,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
.pin_dbg_show = meson_pin_dbg_show,
};
@@ -549,6 +549,14 @@
.compatible = "amlogic,meson8b-aobus-pinctrl",
.data = &meson8b_aobus_pinctrl_data,
},
+ {
+ .compatible = "amlogic,meson-gxbb-periphs-pinctrl",
+ .data = &meson_gxbb_periphs_pinctrl_data,
+ },
+ {
+ .compatible = "amlogic,meson-gxbb-aobus-pinctrl",
+ .data = &meson_gxbb_aobus_pinctrl_data,
+ },
{ },
};
@@ -713,7 +721,7 @@
pc->desc.pins = pc->data->pins;
pc->desc.npins = pc->data->num_pins;
- pc->pcdev = pinctrl_register(&pc->desc, pc->dev, pc);
+ pc->pcdev = devm_pinctrl_register(pc->dev, &pc->desc, pc);
if (IS_ERR(pc->pcdev)) {
dev_err(pc->dev, "can't register pinctrl device");
return PTR_ERR(pc->pcdev);
diff --git a/drivers/pinctrl/meson/pinctrl-meson.h b/drivers/pinctrl/meson/pinctrl-meson.h
index 9c93e0d..d89442e 100644
--- a/drivers/pinctrl/meson/pinctrl-meson.h
+++ b/drivers/pinctrl/meson/pinctrl-meson.h
@@ -199,3 +199,5 @@
extern struct meson_pinctrl_data meson8_aobus_pinctrl_data;
extern struct meson_pinctrl_data meson8b_cbus_pinctrl_data;
extern struct meson_pinctrl_data meson8b_aobus_pinctrl_data;
+extern struct meson_pinctrl_data meson_gxbb_periphs_pinctrl_data;
+extern struct meson_pinctrl_data meson_gxbb_aobus_pinctrl_data;
diff --git a/drivers/pinctrl/meson/pinctrl-meson8b.c b/drivers/pinctrl/meson/pinctrl-meson8b.c
index a100bcf..874f2ed 100644
--- a/drivers/pinctrl/meson/pinctrl-meson8b.c
+++ b/drivers/pinctrl/meson/pinctrl-meson8b.c
@@ -564,7 +564,7 @@
GROUP(eth_rx_clk, 6, 3),
GROUP(eth_txd0_1, 6, 4),
GROUP(eth_txd1_1, 6, 5),
- GROUP(eth_tx_en, 6, 0),
+ GROUP(eth_tx_en, 6, 6),
GROUP(eth_ref_clk, 6, 8),
GROUP(eth_mdc, 6, 9),
GROUP(eth_mdio_en, 6, 10),
diff --git a/drivers/pinctrl/mvebu/pinctrl-armada-370.c b/drivers/pinctrl/mvebu/pinctrl-armada-370.c
index 73dc1bc..9cc1cc3 100644
--- a/drivers/pinctrl/mvebu/pinctrl-armada-370.c
+++ b/drivers/pinctrl/mvebu/pinctrl-armada-370.c
@@ -417,18 +417,12 @@
return mvebu_pinctrl_probe(pdev);
}
-static int armada_370_pinctrl_remove(struct platform_device *pdev)
-{
- return mvebu_pinctrl_remove(pdev);
-}
-
static struct platform_driver armada_370_pinctrl_driver = {
.driver = {
.name = "armada-370-pinctrl",
.of_match_table = armada_370_pinctrl_of_match,
},
.probe = armada_370_pinctrl_probe,
- .remove = armada_370_pinctrl_remove,
};
module_platform_driver(armada_370_pinctrl_driver);
diff --git a/drivers/pinctrl/mvebu/pinctrl-armada-375.c b/drivers/pinctrl/mvebu/pinctrl-armada-375.c
index 54e9fbd..0706514 100644
--- a/drivers/pinctrl/mvebu/pinctrl-armada-375.c
+++ b/drivers/pinctrl/mvebu/pinctrl-armada-375.c
@@ -435,18 +435,12 @@
return mvebu_pinctrl_probe(pdev);
}
-static int armada_375_pinctrl_remove(struct platform_device *pdev)
-{
- return mvebu_pinctrl_remove(pdev);
-}
-
static struct platform_driver armada_375_pinctrl_driver = {
.driver = {
.name = "armada-375-pinctrl",
.of_match_table = of_match_ptr(armada_375_pinctrl_of_match),
},
.probe = armada_375_pinctrl_probe,
- .remove = armada_375_pinctrl_remove,
};
module_platform_driver(armada_375_pinctrl_driver);
diff --git a/drivers/pinctrl/mvebu/pinctrl-armada-38x.c b/drivers/pinctrl/mvebu/pinctrl-armada-38x.c
index 6ec82c6..4e84c8e 100644
--- a/drivers/pinctrl/mvebu/pinctrl-armada-38x.c
+++ b/drivers/pinctrl/mvebu/pinctrl-armada-38x.c
@@ -446,18 +446,12 @@
return mvebu_pinctrl_probe(pdev);
}
-static int armada_38x_pinctrl_remove(struct platform_device *pdev)
-{
- return mvebu_pinctrl_remove(pdev);
-}
-
static struct platform_driver armada_38x_pinctrl_driver = {
.driver = {
.name = "armada-38x-pinctrl",
.of_match_table = of_match_ptr(armada_38x_pinctrl_of_match),
},
.probe = armada_38x_pinctrl_probe,
- .remove = armada_38x_pinctrl_remove,
};
module_platform_driver(armada_38x_pinctrl_driver);
diff --git a/drivers/pinctrl/mvebu/pinctrl-armada-39x.c b/drivers/pinctrl/mvebu/pinctrl-armada-39x.c
index fcfe9b4..e288f8b 100644
--- a/drivers/pinctrl/mvebu/pinctrl-armada-39x.c
+++ b/drivers/pinctrl/mvebu/pinctrl-armada-39x.c
@@ -428,18 +428,12 @@
return mvebu_pinctrl_probe(pdev);
}
-static int armada_39x_pinctrl_remove(struct platform_device *pdev)
-{
- return mvebu_pinctrl_remove(pdev);
-}
-
static struct platform_driver armada_39x_pinctrl_driver = {
.driver = {
.name = "armada-39x-pinctrl",
.of_match_table = of_match_ptr(armada_39x_pinctrl_of_match),
},
.probe = armada_39x_pinctrl_probe,
- .remove = armada_39x_pinctrl_remove,
};
module_platform_driver(armada_39x_pinctrl_driver);
diff --git a/drivers/pinctrl/mvebu/pinctrl-armada-xp.c b/drivers/pinctrl/mvebu/pinctrl-armada-xp.c
index bf70e09..e4ea71a 100644
--- a/drivers/pinctrl/mvebu/pinctrl-armada-xp.c
+++ b/drivers/pinctrl/mvebu/pinctrl-armada-xp.c
@@ -502,18 +502,12 @@
return mvebu_pinctrl_probe(pdev);
}
-static int armada_xp_pinctrl_remove(struct platform_device *pdev)
-{
- return mvebu_pinctrl_remove(pdev);
-}
-
static struct platform_driver armada_xp_pinctrl_driver = {
.driver = {
.name = "armada-xp-pinctrl",
.of_match_table = armada_xp_pinctrl_of_match,
},
.probe = armada_xp_pinctrl_probe,
- .remove = armada_xp_pinctrl_remove,
.suspend = armada_xp_pinctrl_suspend,
.resume = armada_xp_pinctrl_resume,
};
diff --git a/drivers/pinctrl/mvebu/pinctrl-dove.c b/drivers/pinctrl/mvebu/pinctrl-dove.c
index 95bfd06..f93ae0d 100644
--- a/drivers/pinctrl/mvebu/pinctrl-dove.c
+++ b/drivers/pinctrl/mvebu/pinctrl-dove.c
@@ -840,12 +840,9 @@
static int dove_pinctrl_remove(struct platform_device *pdev)
{
- int ret;
-
- ret = mvebu_pinctrl_remove(pdev);
if (!IS_ERR(clk))
clk_disable_unprepare(clk);
- return ret;
+ return 0;
}
static struct platform_driver dove_pinctrl_driver = {
diff --git a/drivers/pinctrl/mvebu/pinctrl-kirkwood.c b/drivers/pinctrl/mvebu/pinctrl-kirkwood.c
index 0f07dc55..a78e9a4 100644
--- a/drivers/pinctrl/mvebu/pinctrl-kirkwood.c
+++ b/drivers/pinctrl/mvebu/pinctrl-kirkwood.c
@@ -481,18 +481,12 @@
return mvebu_pinctrl_probe(pdev);
}
-static int kirkwood_pinctrl_remove(struct platform_device *pdev)
-{
- return mvebu_pinctrl_remove(pdev);
-}
-
static struct platform_driver kirkwood_pinctrl_driver = {
.driver = {
.name = "kirkwood-pinctrl",
.of_match_table = kirkwood_pinctrl_of_match,
},
.probe = kirkwood_pinctrl_probe,
- .remove = kirkwood_pinctrl_remove,
};
module_platform_driver(kirkwood_pinctrl_driver);
diff --git a/drivers/pinctrl/mvebu/pinctrl-mvebu.c b/drivers/pinctrl/mvebu/pinctrl-mvebu.c
index 3ef798f..b6ec6db 100644
--- a/drivers/pinctrl/mvebu/pinctrl-mvebu.c
+++ b/drivers/pinctrl/mvebu/pinctrl-mvebu.c
@@ -711,7 +711,7 @@
return ret;
}
- pctl->pctldev = pinctrl_register(&pctl->desc, &pdev->dev, pctl);
+ pctl->pctldev = devm_pinctrl_register(&pdev->dev, &pctl->desc, pctl);
if (IS_ERR(pctl->pctldev)) {
dev_err(&pdev->dev, "unable to register pinctrl driver\n");
return PTR_ERR(pctl->pctldev);
@@ -725,10 +725,3 @@
return 0;
}
-
-int mvebu_pinctrl_remove(struct platform_device *pdev)
-{
- struct mvebu_pinctrl *pctl = platform_get_drvdata(pdev);
- pinctrl_unregister(pctl->pctldev);
- return 0;
-}
diff --git a/drivers/pinctrl/mvebu/pinctrl-mvebu.h b/drivers/pinctrl/mvebu/pinctrl-mvebu.h
index 65a98e6..b75a5f4 100644
--- a/drivers/pinctrl/mvebu/pinctrl-mvebu.h
+++ b/drivers/pinctrl/mvebu/pinctrl-mvebu.h
@@ -202,6 +202,5 @@
}
int mvebu_pinctrl_probe(struct platform_device *pdev);
-int mvebu_pinctrl_remove(struct platform_device *pdev);
#endif
diff --git a/drivers/pinctrl/mvebu/pinctrl-orion.c b/drivers/pinctrl/mvebu/pinctrl-orion.c
index 3b7122d..345c3df 100644
--- a/drivers/pinctrl/mvebu/pinctrl-orion.c
+++ b/drivers/pinctrl/mvebu/pinctrl-orion.c
@@ -239,18 +239,12 @@
return mvebu_pinctrl_probe(pdev);
}
-static int orion_pinctrl_remove(struct platform_device *pdev)
-{
- return mvebu_pinctrl_remove(pdev);
-}
-
static struct platform_driver orion_pinctrl_driver = {
.driver = {
.name = "orion-pinctrl",
.of_match_table = of_match_ptr(orion_pinctrl_of_match),
},
.probe = orion_pinctrl_probe,
- .remove = orion_pinctrl_remove,
};
module_platform_driver(orion_pinctrl_driver);
diff --git a/drivers/pinctrl/nomadik/pinctrl-abx500.c b/drivers/pinctrl/nomadik/pinctrl-abx500.c
index 1f7469c..7d343c2 100644
--- a/drivers/pinctrl/nomadik/pinctrl-abx500.c
+++ b/drivers/pinctrl/nomadik/pinctrl-abx500.c
@@ -937,7 +937,7 @@
ret = abx500_dt_subnode_to_map(pctldev, np, map,
&reserved_maps, num_maps);
if (ret < 0) {
- pinctrl_utils_dt_free_map(pctldev, *map, *num_maps);
+ pinctrl_utils_free_map(pctldev, *map, *num_maps);
return ret;
}
}
@@ -951,7 +951,7 @@
.get_group_pins = abx500_get_group_pins,
.pin_dbg_show = abx500_pin_dbg_show,
.dt_node_to_map = abx500_dt_node_to_map,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
static int abx500_pin_config_get(struct pinctrl_dev *pctldev,
@@ -1212,7 +1212,8 @@
abx500_pinctrl_desc.pins = pct->soc->pins;
abx500_pinctrl_desc.npins = pct->soc->npins;
- pct->pctldev = pinctrl_register(&abx500_pinctrl_desc, &pdev->dev, pct);
+ pct->pctldev = devm_pinctrl_register(&pdev->dev, &abx500_pinctrl_desc,
+ pct);
if (IS_ERR(pct->pctldev)) {
dev_err(&pdev->dev,
"could not register abx500 pinctrl driver\n");
diff --git a/drivers/pinctrl/nomadik/pinctrl-nomadik.c b/drivers/pinctrl/nomadik/pinctrl-nomadik.c
index c8969dd..ccbfc32 100644
--- a/drivers/pinctrl/nomadik/pinctrl-nomadik.c
+++ b/drivers/pinctrl/nomadik/pinctrl-nomadik.c
@@ -24,6 +24,7 @@
#include <linux/slab.h>
#include <linux/of_device.h>
#include <linux/of_address.h>
+#include <linux/bitops.h>
#include <linux/pinctrl/machine.h>
#include <linux/pinctrl/pinctrl.h>
#include <linux/pinctrl/pinmux.h>
@@ -292,15 +293,14 @@
static void __nmk_gpio_set_mode(struct nmk_gpio_chip *nmk_chip,
unsigned offset, int gpio_mode)
{
- u32 bit = 1 << offset;
u32 afunc, bfunc;
- afunc = readl(nmk_chip->addr + NMK_GPIO_AFSLA) & ~bit;
- bfunc = readl(nmk_chip->addr + NMK_GPIO_AFSLB) & ~bit;
+ afunc = readl(nmk_chip->addr + NMK_GPIO_AFSLA) & ~BIT(offset);
+ bfunc = readl(nmk_chip->addr + NMK_GPIO_AFSLB) & ~BIT(offset);
if (gpio_mode & NMK_GPIO_ALT_A)
- afunc |= bit;
+ afunc |= BIT(offset);
if (gpio_mode & NMK_GPIO_ALT_B)
- bfunc |= bit;
+ bfunc |= BIT(offset);
writel(afunc, nmk_chip->addr + NMK_GPIO_AFSLA);
writel(bfunc, nmk_chip->addr + NMK_GPIO_AFSLB);
}
@@ -308,55 +308,52 @@
static void __nmk_gpio_set_slpm(struct nmk_gpio_chip *nmk_chip,
unsigned offset, enum nmk_gpio_slpm mode)
{
- u32 bit = 1 << offset;
u32 slpm;
slpm = readl(nmk_chip->addr + NMK_GPIO_SLPC);
if (mode == NMK_GPIO_SLPM_NOCHANGE)
- slpm |= bit;
+ slpm |= BIT(offset);
else
- slpm &= ~bit;
+ slpm &= ~BIT(offset);
writel(slpm, nmk_chip->addr + NMK_GPIO_SLPC);
}
static void __nmk_gpio_set_pull(struct nmk_gpio_chip *nmk_chip,
unsigned offset, enum nmk_gpio_pull pull)
{
- u32 bit = 1 << offset;
u32 pdis;
pdis = readl(nmk_chip->addr + NMK_GPIO_PDIS);
if (pull == NMK_GPIO_PULL_NONE) {
- pdis |= bit;
- nmk_chip->pull_up &= ~bit;
+ pdis |= BIT(offset);
+ nmk_chip->pull_up &= ~BIT(offset);
} else {
- pdis &= ~bit;
+ pdis &= ~BIT(offset);
}
writel(pdis, nmk_chip->addr + NMK_GPIO_PDIS);
if (pull == NMK_GPIO_PULL_UP) {
- nmk_chip->pull_up |= bit;
- writel(bit, nmk_chip->addr + NMK_GPIO_DATS);
+ nmk_chip->pull_up |= BIT(offset);
+ writel(BIT(offset), nmk_chip->addr + NMK_GPIO_DATS);
} else if (pull == NMK_GPIO_PULL_DOWN) {
- nmk_chip->pull_up &= ~bit;
- writel(bit, nmk_chip->addr + NMK_GPIO_DATC);
+ nmk_chip->pull_up &= ~BIT(offset);
+ writel(BIT(offset), nmk_chip->addr + NMK_GPIO_DATC);
}
}
static void __nmk_gpio_set_lowemi(struct nmk_gpio_chip *nmk_chip,
unsigned offset, bool lowemi)
{
- u32 bit = BIT(offset);
- bool enabled = nmk_chip->lowemi & bit;
+ bool enabled = nmk_chip->lowemi & BIT(offset);
if (lowemi == enabled)
return;
if (lowemi)
- nmk_chip->lowemi |= bit;
+ nmk_chip->lowemi |= BIT(offset);
else
- nmk_chip->lowemi &= ~bit;
+ nmk_chip->lowemi &= ~BIT(offset);
writel_relaxed(nmk_chip->lowemi,
nmk_chip->addr + NMK_GPIO_LOWEMI);
@@ -365,22 +362,22 @@
static void __nmk_gpio_make_input(struct nmk_gpio_chip *nmk_chip,
unsigned offset)
{
- writel(1 << offset, nmk_chip->addr + NMK_GPIO_DIRC);
+ writel(BIT(offset), nmk_chip->addr + NMK_GPIO_DIRC);
}
static void __nmk_gpio_set_output(struct nmk_gpio_chip *nmk_chip,
unsigned offset, int val)
{
if (val)
- writel(1 << offset, nmk_chip->addr + NMK_GPIO_DATS);
+ writel(BIT(offset), nmk_chip->addr + NMK_GPIO_DATS);
else
- writel(1 << offset, nmk_chip->addr + NMK_GPIO_DATC);
+ writel(BIT(offset), nmk_chip->addr + NMK_GPIO_DATC);
}
static void __nmk_gpio_make_output(struct nmk_gpio_chip *nmk_chip,
unsigned offset, int val)
{
- writel(1 << offset, nmk_chip->addr + NMK_GPIO_DIRS);
+ writel(BIT(offset), nmk_chip->addr + NMK_GPIO_DIRS);
__nmk_gpio_set_output(nmk_chip, offset, val);
}
@@ -614,34 +611,7 @@
return NMK_GPIO_ALT_C;
}
-int nmk_gpio_get_mode(int gpio)
-{
- struct nmk_gpio_chip *nmk_chip;
- u32 afunc, bfunc, bit;
-
- nmk_chip = nmk_gpio_chips[gpio / NMK_GPIO_PER_CHIP];
- if (!nmk_chip)
- return -EINVAL;
-
- bit = 1 << (gpio % NMK_GPIO_PER_CHIP);
-
- clk_enable(nmk_chip->clk);
-
- afunc = readl(nmk_chip->addr + NMK_GPIO_AFSLA) & bit;
- bfunc = readl(nmk_chip->addr + NMK_GPIO_AFSLB) & bit;
-
- clk_disable(nmk_chip->clk);
-
- return (afunc ? NMK_GPIO_ALT_A : 0) | (bfunc ? NMK_GPIO_ALT_B : 0);
-}
-EXPORT_SYMBOL(nmk_gpio_get_mode);
-
-
/* IRQ functions */
-static inline int nmk_gpio_get_bitmask(int gpio)
-{
- return 1 << (gpio % NMK_GPIO_PER_CHIP);
-}
static void nmk_gpio_irq_ack(struct irq_data *d)
{
@@ -649,7 +619,7 @@
struct nmk_gpio_chip *nmk_chip = gpiochip_get_data(chip);
clk_enable(nmk_chip->clk);
- writel(nmk_gpio_get_bitmask(d->hwirq), nmk_chip->addr + NMK_GPIO_IC);
+ writel(BIT(d->hwirq), nmk_chip->addr + NMK_GPIO_IC);
clk_disable(nmk_chip->clk);
}
@@ -659,10 +629,9 @@
};
static void __nmk_gpio_irq_modify(struct nmk_gpio_chip *nmk_chip,
- int gpio, enum nmk_gpio_irq_type which,
+ int offset, enum nmk_gpio_irq_type which,
bool enable)
{
- u32 bitmask = nmk_gpio_get_bitmask(gpio);
u32 *rimscval;
u32 *fimscval;
u32 rimscreg;
@@ -681,24 +650,24 @@
}
/* we must individually set/clear the two edges */
- if (nmk_chip->edge_rising & bitmask) {
+ if (nmk_chip->edge_rising & BIT(offset)) {
if (enable)
- *rimscval |= bitmask;
+ *rimscval |= BIT(offset);
else
- *rimscval &= ~bitmask;
+ *rimscval &= ~BIT(offset);
writel(*rimscval, nmk_chip->addr + rimscreg);
}
- if (nmk_chip->edge_falling & bitmask) {
+ if (nmk_chip->edge_falling & BIT(offset)) {
if (enable)
- *fimscval |= bitmask;
+ *fimscval |= BIT(offset);
else
- *fimscval &= ~bitmask;
+ *fimscval &= ~BIT(offset);
writel(*fimscval, nmk_chip->addr + fimscreg);
}
}
static void __nmk_gpio_set_wake(struct nmk_gpio_chip *nmk_chip,
- int gpio, bool on)
+ int offset, bool on)
{
/*
* Ensure WAKEUP_ENABLE is on. No need to disable it if wakeup is
@@ -706,21 +675,19 @@
* wakeup is anyhow controlled by the RIMSC and FIMSC registers.
*/
if (nmk_chip->sleepmode && on) {
- __nmk_gpio_set_slpm(nmk_chip, gpio % NMK_GPIO_PER_CHIP,
+ __nmk_gpio_set_slpm(nmk_chip, offset,
NMK_GPIO_SLPM_WAKEUP_ENABLE);
}
- __nmk_gpio_irq_modify(nmk_chip, gpio, WAKE, on);
+ __nmk_gpio_irq_modify(nmk_chip, offset, WAKE, on);
}
static int nmk_gpio_irq_maskunmask(struct irq_data *d, bool enable)
{
struct nmk_gpio_chip *nmk_chip;
unsigned long flags;
- u32 bitmask;
nmk_chip = irq_data_get_irq_chip_data(d);
- bitmask = nmk_gpio_get_bitmask(d->hwirq);
if (!nmk_chip)
return -EINVAL;
@@ -730,7 +697,7 @@
__nmk_gpio_irq_modify(nmk_chip, d->hwirq, NORMAL, enable);
- if (!(nmk_chip->real_wake & bitmask))
+ if (!(nmk_chip->real_wake & BIT(d->hwirq)))
__nmk_gpio_set_wake(nmk_chip, d->hwirq, enable);
spin_unlock(&nmk_chip->lock);
@@ -754,12 +721,10 @@
{
struct nmk_gpio_chip *nmk_chip;
unsigned long flags;
- u32 bitmask;
nmk_chip = irq_data_get_irq_chip_data(d);
if (!nmk_chip)
return -EINVAL;
- bitmask = nmk_gpio_get_bitmask(d->hwirq);
clk_enable(nmk_chip->clk);
spin_lock_irqsave(&nmk_gpio_slpm_lock, flags);
@@ -769,9 +734,9 @@
__nmk_gpio_set_wake(nmk_chip, d->hwirq, on);
if (on)
- nmk_chip->real_wake |= bitmask;
+ nmk_chip->real_wake |= BIT(d->hwirq);
else
- nmk_chip->real_wake &= ~bitmask;
+ nmk_chip->real_wake &= ~BIT(d->hwirq);
spin_unlock(&nmk_chip->lock);
spin_unlock_irqrestore(&nmk_gpio_slpm_lock, flags);
@@ -786,10 +751,8 @@
bool wake = irqd_is_wakeup_set(d);
struct nmk_gpio_chip *nmk_chip;
unsigned long flags;
- u32 bitmask;
nmk_chip = irq_data_get_irq_chip_data(d);
- bitmask = nmk_gpio_get_bitmask(d->hwirq);
if (!nmk_chip)
return -EINVAL;
if (type & IRQ_TYPE_LEVEL_HIGH)
@@ -806,13 +769,13 @@
if (enabled || wake)
__nmk_gpio_irq_modify(nmk_chip, d->hwirq, WAKE, false);
- nmk_chip->edge_rising &= ~bitmask;
+ nmk_chip->edge_rising &= ~BIT(d->hwirq);
if (type & IRQ_TYPE_EDGE_RISING)
- nmk_chip->edge_rising |= bitmask;
+ nmk_chip->edge_rising |= BIT(d->hwirq);
- nmk_chip->edge_falling &= ~bitmask;
+ nmk_chip->edge_falling &= ~BIT(d->hwirq);
if (type & IRQ_TYPE_EDGE_FALLING)
- nmk_chip->edge_falling |= bitmask;
+ nmk_chip->edge_falling |= BIT(d->hwirq);
if (enabled)
__nmk_gpio_irq_modify(nmk_chip, d->hwirq, NORMAL, true);
@@ -884,13 +847,27 @@
/* I/O Functions */
+static int nmk_gpio_get_dir(struct gpio_chip *chip, unsigned offset)
+{
+ struct nmk_gpio_chip *nmk_chip = gpiochip_get_data(chip);
+ int dir;
+
+ clk_enable(nmk_chip->clk);
+
+ dir = !!(readl(nmk_chip->addr + NMK_GPIO_DIR) & BIT(offset));
+
+ clk_disable(nmk_chip->clk);
+
+ return dir;
+}
+
static int nmk_gpio_make_input(struct gpio_chip *chip, unsigned offset)
{
struct nmk_gpio_chip *nmk_chip = gpiochip_get_data(chip);
clk_enable(nmk_chip->clk);
- writel(1 << offset, nmk_chip->addr + NMK_GPIO_DIRC);
+ writel(BIT(offset), nmk_chip->addr + NMK_GPIO_DIRC);
clk_disable(nmk_chip->clk);
@@ -900,12 +877,11 @@
static int nmk_gpio_get_input(struct gpio_chip *chip, unsigned offset)
{
struct nmk_gpio_chip *nmk_chip = gpiochip_get_data(chip);
- u32 bit = 1 << offset;
int value;
clk_enable(nmk_chip->clk);
- value = (readl(nmk_chip->addr + NMK_GPIO_DAT) & bit) != 0;
+ value = !!(readl(nmk_chip->addr + NMK_GPIO_DAT) & BIT(offset));
clk_disable(nmk_chip->clk);
@@ -939,6 +915,19 @@
}
#ifdef CONFIG_DEBUG_FS
+static int nmk_gpio_get_mode(struct nmk_gpio_chip *nmk_chip, int offset)
+{
+ u32 afunc, bfunc;
+
+ clk_enable(nmk_chip->clk);
+
+ afunc = readl(nmk_chip->addr + NMK_GPIO_AFSLA) & BIT(offset);
+ bfunc = readl(nmk_chip->addr + NMK_GPIO_AFSLB) & BIT(offset);
+
+ clk_disable(nmk_chip->clk);
+
+ return (afunc ? NMK_GPIO_ALT_A : 0) | (bfunc ? NMK_GPIO_ALT_B : 0);
+}
#include <linux/seq_file.h>
@@ -952,7 +941,6 @@
bool is_out;
bool data_out;
bool pull;
- u32 bit = 1 << offset;
const char *modes[] = {
[NMK_GPIO_ALT_GPIO] = "gpio",
[NMK_GPIO_ALT_A] = "altA",
@@ -970,10 +958,10 @@
};
clk_enable(nmk_chip->clk);
- is_out = !!(readl(nmk_chip->addr + NMK_GPIO_DIR) & bit);
- pull = !(readl(nmk_chip->addr + NMK_GPIO_PDIS) & bit);
- data_out = !!(readl(nmk_chip->addr + NMK_GPIO_DAT) & bit);
- mode = nmk_gpio_get_mode(gpio);
+ is_out = !!(readl(nmk_chip->addr + NMK_GPIO_DIR) & BIT(offset));
+ pull = !(readl(nmk_chip->addr + NMK_GPIO_PDIS) & BIT(offset));
+ data_out = !!(readl(nmk_chip->addr + NMK_GPIO_DAT) & BIT(offset));
+ mode = nmk_gpio_get_mode(nmk_chip, offset);
if ((mode == NMK_GPIO_ALT_C) && pctldev)
mode = nmk_prcm_gpiocr_get_mode(pctldev, gpio);
@@ -1007,11 +995,10 @@
*/
if (irq > 0 && desc && desc->action) {
char *trigger;
- u32 bitmask = nmk_gpio_get_bitmask(gpio);
- if (nmk_chip->edge_rising & bitmask)
+ if (nmk_chip->edge_rising & BIT(offset))
trigger = "edge-rising";
- else if (nmk_chip->edge_falling & bitmask)
+ else if (nmk_chip->edge_falling & BIT(offset))
trigger = "edge-falling";
else
trigger = "edge-undefined";
@@ -1246,6 +1233,7 @@
chip = &nmk_chip->chip;
chip->request = gpiochip_generic_request;
chip->free = gpiochip_generic_free;
+ chip->get_direction = nmk_gpio_get_dir;
chip->direction_input = nmk_gpio_make_input;
chip->get = nmk_gpio_get_input;
chip->direction_output = nmk_gpio_make_output;
@@ -1612,7 +1600,7 @@
ret = nmk_pinctrl_dt_subnode_to_map(pctldev, np, map,
&reserved_maps, num_maps);
if (ret < 0) {
- pinctrl_utils_dt_free_map(pctldev, *map, *num_maps);
+ pinctrl_utils_free_map(pctldev, *map, *num_maps);
return ret;
}
}
@@ -1626,7 +1614,7 @@
.get_group_pins = nmk_get_group_pins,
.pin_dbg_show = nmk_pin_dbg_show,
.dt_node_to_map = nmk_pinctrl_dt_node_to_map,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
static int nmk_pmx_get_funcs_cnt(struct pinctrl_dev *pctldev)
@@ -2044,7 +2032,7 @@
nmk_pinctrl_desc.npins = npct->soc->npins;
npct->dev = &pdev->dev;
- npct->pctl = pinctrl_register(&nmk_pinctrl_desc, &pdev->dev, npct);
+ npct->pctl = devm_pinctrl_register(&pdev->dev, &nmk_pinctrl_desc, npct);
if (IS_ERR(npct->pctl)) {
dev_err(&pdev->dev, "could not register Nomadik pinctrl driver\n");
return PTR_ERR(npct->pctl);
diff --git a/drivers/pinctrl/pinconf-generic.c b/drivers/pinctrl/pinconf-generic.c
index 79e6159..d5bf9fa 100644
--- a/drivers/pinctrl/pinconf-generic.c
+++ b/drivers/pinctrl/pinconf-generic.c
@@ -386,7 +386,7 @@
return 0;
exit:
- pinctrl_utils_dt_free_map(pctldev, *map, *num_maps);
+ pinctrl_utils_free_map(pctldev, *map, *num_maps);
return ret;
}
EXPORT_SYMBOL_GPL(pinconf_generic_dt_node_to_map);
diff --git a/drivers/pinctrl/pinctrl-adi2.c b/drivers/pinctrl/pinctrl-adi2.c
index ecb57635..54569a7 100644
--- a/drivers/pinctrl/pinctrl-adi2.c
+++ b/drivers/pinctrl/pinctrl-adi2.c
@@ -1058,7 +1058,8 @@
adi_pinmux_desc.npins = pinctrl->soc->npins;
/* Now register the pin controller and all pins it handles */
- pinctrl->pctl = pinctrl_register(&adi_pinmux_desc, &pdev->dev, pinctrl);
+ pinctrl->pctl = devm_pinctrl_register(&pdev->dev, &adi_pinmux_desc,
+ pinctrl);
if (IS_ERR(pinctrl->pctl)) {
dev_err(&pdev->dev, "could not register pinctrl ADI2 driver\n");
return PTR_ERR(pinctrl->pctl);
@@ -1069,18 +1070,8 @@
return 0;
}
-static int adi_pinctrl_remove(struct platform_device *pdev)
-{
- struct adi_pinctrl *pinctrl = platform_get_drvdata(pdev);
-
- pinctrl_unregister(pinctrl->pctl);
-
- return 0;
-}
-
static struct platform_driver adi_pinctrl_driver = {
.probe = adi_pinctrl_probe,
- .remove = adi_pinctrl_remove,
.driver = {
.name = DRIVER_NAME,
},
diff --git a/drivers/pinctrl/pinctrl-amd.c b/drivers/pinctrl/pinctrl-amd.c
index 5c025f5..634b4d3 100644
--- a/drivers/pinctrl/pinctrl-amd.c
+++ b/drivers/pinctrl/pinctrl-amd.c
@@ -580,7 +580,7 @@
.get_group_pins = amd_get_group_pins,
#ifdef CONFIG_OF
.dt_node_to_map = pinconf_generic_dt_node_to_map_group,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
#endif
};
@@ -783,8 +783,8 @@
gpio_dev->ngroups = ARRAY_SIZE(kerncz_groups);
amd_pinctrl_desc.name = dev_name(&pdev->dev);
- gpio_dev->pctrl = pinctrl_register(&amd_pinctrl_desc,
- &pdev->dev, gpio_dev);
+ gpio_dev->pctrl = devm_pinctrl_register(&pdev->dev, &amd_pinctrl_desc,
+ gpio_dev);
if (IS_ERR(gpio_dev->pctrl)) {
dev_err(&pdev->dev, "Couldn't register pinctrl driver\n");
return PTR_ERR(gpio_dev->pctrl);
@@ -792,7 +792,7 @@
ret = gpiochip_add_data(&gpio_dev->gc, gpio_dev);
if (ret)
- goto out1;
+ return ret;
ret = gpiochip_add_pin_range(&gpio_dev->gc, dev_name(&pdev->dev),
0, 0, TOTAL_NUMBER_OF_PINS);
@@ -825,8 +825,6 @@
out2:
gpiochip_remove(&gpio_dev->gc);
-out1:
- pinctrl_unregister(gpio_dev->pctrl);
return ret;
}
@@ -837,13 +835,13 @@
gpio_dev = platform_get_drvdata(pdev);
gpiochip_remove(&gpio_dev->gc);
- pinctrl_unregister(gpio_dev->pctrl);
return 0;
}
static const struct acpi_device_id amd_gpio_acpi_match[] = {
{ "AMD0030", 0 },
+ { "AMDI0030", 0},
{ },
};
MODULE_DEVICE_TABLE(acpi, amd_gpio_acpi_match);
diff --git a/drivers/pinctrl/pinctrl-as3722.c b/drivers/pinctrl/pinctrl-as3722.c
index e844fdc..4e9fe78 100644
--- a/drivers/pinctrl/pinctrl-as3722.c
+++ b/drivers/pinctrl/pinctrl-as3722.c
@@ -201,7 +201,7 @@
.get_group_name = as3722_pinctrl_get_group_name,
.get_group_pins = as3722_pinctrl_get_group_pins,
.dt_node_to_map = pinconf_generic_dt_node_to_map_pin,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
static int as3722_pinctrl_get_funcs_count(struct pinctrl_dev *pctldev)
@@ -569,8 +569,8 @@
as3722_pinctrl_desc.name = dev_name(&pdev->dev);
as3722_pinctrl_desc.pins = as3722_pins_desc;
as3722_pinctrl_desc.npins = ARRAY_SIZE(as3722_pins_desc);
- as_pci->pctl = pinctrl_register(&as3722_pinctrl_desc,
- &pdev->dev, as_pci);
+ as_pci->pctl = devm_pinctrl_register(&pdev->dev, &as3722_pinctrl_desc,
+ as_pci);
if (IS_ERR(as_pci->pctl)) {
dev_err(&pdev->dev, "Couldn't register pinctrl driver\n");
return PTR_ERR(as_pci->pctl);
@@ -582,7 +582,7 @@
ret = gpiochip_add_data(&as_pci->gpio_chip, as_pci);
if (ret < 0) {
dev_err(&pdev->dev, "Couldn't register gpiochip, %d\n", ret);
- goto fail_chip_add;
+ return ret;
}
ret = gpiochip_add_pin_range(&as_pci->gpio_chip, dev_name(&pdev->dev),
@@ -596,8 +596,6 @@
fail_range_add:
gpiochip_remove(&as_pci->gpio_chip);
-fail_chip_add:
- pinctrl_unregister(as_pci->pctl);
return ret;
}
@@ -606,7 +604,6 @@
struct as3722_pctrl_info *as_pci = platform_get_drvdata(pdev);
gpiochip_remove(&as_pci->gpio_chip);
- pinctrl_unregister(as_pci->pctl);
return 0;
}
diff --git a/drivers/pinctrl/pinctrl-at91-pio4.c b/drivers/pinctrl/pinctrl-at91-pio4.c
index 2c447130..a025b40 100644
--- a/drivers/pinctrl/pinctrl-at91-pio4.c
+++ b/drivers/pinctrl/pinctrl-at91-pio4.c
@@ -579,7 +579,7 @@
}
if (ret < 0) {
- pinctrl_utils_dt_free_map(pctldev, *map, *num_maps);
+ pinctrl_utils_free_map(pctldev, *map, *num_maps);
dev_err(pctldev->dev, "can't create maps for node %s\n",
np_config->full_name);
}
@@ -592,7 +592,7 @@
.get_group_name = atmel_pctl_get_group_name,
.get_group_pins = atmel_pctl_get_group_pins,
.dt_node_to_map = atmel_pctl_dt_node_to_map,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
static int atmel_pmx_get_functions_count(struct pinctrl_dev *pctldev)
@@ -1036,18 +1036,19 @@
goto clk_prepare_enable_error;
}
- atmel_pioctrl->pinctrl_dev = pinctrl_register(&atmel_pinctrl_desc,
- &pdev->dev,
- atmel_pioctrl);
- if (!atmel_pioctrl->pinctrl_dev) {
+ atmel_pioctrl->pinctrl_dev = devm_pinctrl_register(&pdev->dev,
+ &atmel_pinctrl_desc,
+ atmel_pioctrl);
+ if (IS_ERR(atmel_pioctrl->pinctrl_dev)) {
+ ret = PTR_ERR(atmel_pioctrl->pinctrl_dev);
dev_err(dev, "pinctrl registration failed\n");
- goto pinctrl_register_error;
+ goto clk_unprep;
}
ret = gpiochip_add_data(atmel_pioctrl->gpio_chip, atmel_pioctrl);
if (ret) {
dev_err(dev, "failed to add gpiochip\n");
- goto gpiochip_add_error;
+ goto clk_unprep;
}
ret = gpiochip_add_pin_range(atmel_pioctrl->gpio_chip, dev_name(dev),
@@ -1061,15 +1062,15 @@
return 0;
-clk_prepare_enable_error:
- irq_domain_remove(atmel_pioctrl->irq_domain);
-pinctrl_register_error:
- clk_disable_unprepare(atmel_pioctrl->clk);
-gpiochip_add_error:
- pinctrl_unregister(atmel_pioctrl->pinctrl_dev);
gpiochip_add_pin_range_error:
gpiochip_remove(atmel_pioctrl->gpio_chip);
+clk_unprep:
+ clk_disable_unprepare(atmel_pioctrl->clk);
+
+clk_prepare_enable_error:
+ irq_domain_remove(atmel_pioctrl->irq_domain);
+
return ret;
}
@@ -1079,7 +1080,6 @@
irq_domain_remove(atmel_pioctrl->irq_domain);
clk_disable_unprepare(atmel_pioctrl->clk);
- pinctrl_unregister(atmel_pioctrl->pinctrl_dev);
gpiochip_remove(atmel_pioctrl->gpio_chip);
return 0;
diff --git a/drivers/pinctrl/pinctrl-at91.c b/drivers/pinctrl/pinctrl-at91.c
index 523b6b7..b7c0d6f 100644
--- a/drivers/pinctrl/pinctrl-at91.c
+++ b/drivers/pinctrl/pinctrl-at91.c
@@ -1252,7 +1252,8 @@
}
platform_set_drvdata(pdev, info);
- info->pctl = pinctrl_register(&at91_pinctrl_desc, &pdev->dev, info);
+ info->pctl = devm_pinctrl_register(&pdev->dev, &at91_pinctrl_desc,
+ info);
if (IS_ERR(info->pctl)) {
dev_err(&pdev->dev, "could not register AT91 pinctrl driver\n");
@@ -1269,15 +1270,6 @@
return 0;
}
-static int at91_pinctrl_remove(struct platform_device *pdev)
-{
- struct at91_pinctrl *info = platform_get_drvdata(pdev);
-
- pinctrl_unregister(info->pctl);
-
- return 0;
-}
-
static int at91_gpio_get_direction(struct gpio_chip *chip, unsigned offset)
{
struct at91_gpio_chip *at91_gpio = gpiochip_get_data(chip);
@@ -1660,7 +1652,7 @@
}
/* This structure is replicated for each GPIO block allocated at probe time */
-static struct gpio_chip at91_gpio_template = {
+static const struct gpio_chip at91_gpio_template = {
.request = gpiochip_generic_request,
.free = gpiochip_generic_free,
.get_direction = at91_gpio_get_direction,
@@ -1730,14 +1722,9 @@
goto err;
}
- ret = clk_prepare(at91_chip->clock);
- if (ret)
- goto clk_prepare_err;
-
- /* enable PIO controller's clock */
- ret = clk_enable(at91_chip->clock);
+ ret = clk_prepare_enable(at91_chip->clock);
if (ret) {
- dev_err(&pdev->dev, "failed to enable clock, ignoring.\n");
+ dev_err(&pdev->dev, "failed to prepare and enable clock, ignoring.\n");
goto clk_enable_err;
}
@@ -1797,10 +1784,8 @@
irq_setup_err:
gpiochip_remove(chip);
gpiochip_add_err:
- clk_disable(at91_chip->clock);
clk_enable_err:
- clk_unprepare(at91_chip->clock);
-clk_prepare_err:
+ clk_disable_unprepare(at91_chip->clock);
err:
dev_err(&pdev->dev, "Failure %i for GPIO %i\n", ret, alias_idx);
@@ -1821,7 +1806,6 @@
.of_match_table = at91_pinctrl_of_match,
},
.probe = at91_pinctrl_probe,
- .remove = at91_pinctrl_remove,
};
static struct platform_driver * const drivers[] = {
diff --git a/drivers/pinctrl/pinctrl-digicolor.c b/drivers/pinctrl/pinctrl-digicolor.c
index f1343d6..30ee564 100644
--- a/drivers/pinctrl/pinctrl-digicolor.c
+++ b/drivers/pinctrl/pinctrl-digicolor.c
@@ -84,7 +84,7 @@
.get_group_name = dc_get_group_name,
.get_group_pins = dc_get_group_pins,
.dt_node_to_map = pinconf_generic_dt_node_to_map_pin,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
static const char *const dc_functions[] = {
@@ -280,7 +280,7 @@
struct pinctrl_desc *pctl_desc;
char *pin_names;
int name_len = strlen("GP_xx") + 1;
- int i, j, ret;
+ int i, j;
pmap = devm_kzalloc(&pdev->dev, sizeof(*pmap), GFP_KERNEL);
if (!pmap)
@@ -326,26 +326,19 @@
pmap->dev = &pdev->dev;
- pmap->pctl = pinctrl_register(pctl_desc, &pdev->dev, pmap);
+ pmap->pctl = devm_pinctrl_register(&pdev->dev, pctl_desc, pmap);
if (IS_ERR(pmap->pctl)) {
dev_err(&pdev->dev, "pinctrl driver registration failed\n");
return PTR_ERR(pmap->pctl);
}
- ret = dc_gpiochip_add(pmap, pdev->dev.of_node);
- if (ret < 0) {
- pinctrl_unregister(pmap->pctl);
- return ret;
- }
-
- return 0;
+ return dc_gpiochip_add(pmap, pdev->dev.of_node);
}
static int dc_pinctrl_remove(struct platform_device *pdev)
{
struct dc_pinmap *pmap = platform_get_drvdata(pdev);
- pinctrl_unregister(pmap->pctl);
gpiochip_remove(&pmap->chip);
return 0;
diff --git a/drivers/pinctrl/pinctrl-lantiq.c b/drivers/pinctrl/pinctrl-lantiq.c
index fc38a85..a4d6474 100644
--- a/drivers/pinctrl/pinctrl-lantiq.c
+++ b/drivers/pinctrl/pinctrl-lantiq.c
@@ -336,7 +336,7 @@
desc->pmxops = <q_pmx_ops;
info->dev = &pdev->dev;
- info->pctrl = pinctrl_register(desc, &pdev->dev, info);
+ info->pctrl = devm_pinctrl_register(&pdev->dev, desc, info);
if (IS_ERR(info->pctrl)) {
dev_err(&pdev->dev, "failed to register LTQ pinmux driver\n");
return PTR_ERR(info->pctrl);
diff --git a/drivers/pinctrl/pinctrl-lpc18xx.c b/drivers/pinctrl/pinctrl-lpc18xx.c
index b1767f7..8a931c7 100644
--- a/drivers/pinctrl/pinctrl-lpc18xx.c
+++ b/drivers/pinctrl/pinctrl-lpc18xx.c
@@ -1252,7 +1252,7 @@
.get_group_name = lpc18xx_pctl_get_group_name,
.get_group_pins = lpc18xx_pctl_get_group_pins,
.dt_node_to_map = pinconf_generic_dt_node_to_map_pin,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
static struct pinctrl_desc lpc18xx_scu_desc = {
@@ -1355,7 +1355,7 @@
platform_set_drvdata(pdev, scu);
- scu->pctl = pinctrl_register(&lpc18xx_scu_desc, &pdev->dev, scu);
+ scu->pctl = devm_pinctrl_register(&pdev->dev, &lpc18xx_scu_desc, scu);
if (IS_ERR(scu->pctl)) {
dev_err(&pdev->dev, "Could not register pinctrl driver\n");
clk_disable_unprepare(scu->clk);
@@ -1369,7 +1369,6 @@
{
struct lpc18xx_scu_data *scu = platform_get_drvdata(pdev);
- pinctrl_unregister(scu->pctl);
clk_disable_unprepare(scu->clk);
return 0;
diff --git a/drivers/pinctrl/pinctrl-palmas.c b/drivers/pinctrl/pinctrl-palmas.c
index f7e1680..8edb3f8c 100644
--- a/drivers/pinctrl/pinctrl-palmas.c
+++ b/drivers/pinctrl/pinctrl-palmas.c
@@ -656,7 +656,7 @@
.get_group_name = palmas_pinctrl_get_group_name,
.get_group_pins = palmas_pinctrl_get_group_pins,
.dt_node_to_map = pinconf_generic_dt_node_to_map_pin,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
static int palmas_pinctrl_get_funcs_count(struct pinctrl_dev *pctldev)
@@ -1043,7 +1043,8 @@
palmas_pinctrl_desc.name = dev_name(&pdev->dev);
palmas_pinctrl_desc.pins = palmas_pins_desc;
palmas_pinctrl_desc.npins = ARRAY_SIZE(palmas_pins_desc);
- pci->pctl = pinctrl_register(&palmas_pinctrl_desc, &pdev->dev, pci);
+ pci->pctl = devm_pinctrl_register(&pdev->dev, &palmas_pinctrl_desc,
+ pci);
if (IS_ERR(pci->pctl)) {
dev_err(&pdev->dev, "Couldn't register pinctrl driver\n");
return PTR_ERR(pci->pctl);
@@ -1051,21 +1052,12 @@
return 0;
}
-static int palmas_pinctrl_remove(struct platform_device *pdev)
-{
- struct palmas_pctrl_chip_info *pci = platform_get_drvdata(pdev);
-
- pinctrl_unregister(pci->pctl);
- return 0;
-}
-
static struct platform_driver palmas_pinctrl_driver = {
.driver = {
.name = "palmas-pinctrl",
.of_match_table = palmas_pinctrl_of_match,
},
.probe = palmas_pinctrl_probe,
- .remove = palmas_pinctrl_remove,
};
module_platform_driver(palmas_pinctrl_driver);
diff --git a/drivers/pinctrl/pinctrl-pic32.c b/drivers/pinctrl/pinctrl-pic32.c
index 0b07d4b..31ceb95 100644
--- a/drivers/pinctrl/pinctrl-pic32.c
+++ b/drivers/pinctrl/pinctrl-pic32.c
@@ -1743,7 +1743,7 @@
.get_group_name = pic32_pinctrl_get_group_name,
.get_group_pins = pic32_pinctrl_get_group_pins,
.dt_node_to_map = pinconf_generic_dt_node_to_map_pin,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
static int pic32_pinmux_get_functions_count(struct pinctrl_dev *pctldev)
@@ -2194,7 +2194,8 @@
pic32_pinctrl_desc.custom_params = pic32_mpp_bindings;
pic32_pinctrl_desc.num_custom_params = ARRAY_SIZE(pic32_mpp_bindings);
- pctl->pctldev = pinctrl_register(&pic32_pinctrl_desc, &pdev->dev, pctl);
+ pctl->pctldev = devm_pinctrl_register(&pdev->dev, &pic32_pinctrl_desc,
+ pctl);
if (IS_ERR(pctl->pctldev)) {
dev_err(&pdev->dev, "Failed to register pinctrl device\n");
return PTR_ERR(pctl->pctldev);
diff --git a/drivers/pinctrl/pinctrl-pistachio.c b/drivers/pinctrl/pinctrl-pistachio.c
index 2673cd9..c6d410e 100644
--- a/drivers/pinctrl/pinctrl-pistachio.c
+++ b/drivers/pinctrl/pinctrl-pistachio.c
@@ -913,7 +913,7 @@
.get_group_name = pistachio_pinctrl_get_group_name,
.get_group_pins = pistachio_pinctrl_get_group_pins,
.dt_node_to_map = pinconf_generic_dt_node_to_map_pin,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
static int pistachio_pinmux_get_functions_count(struct pinctrl_dev *pctldev)
@@ -1457,8 +1457,8 @@
pistachio_pinctrl_desc.pins = pctl->pins;
pistachio_pinctrl_desc.npins = pctl->npins;
- pctl->pctldev = pinctrl_register(&pistachio_pinctrl_desc, &pdev->dev,
- pctl);
+ pctl->pctldev = devm_pinctrl_register(&pdev->dev, &pistachio_pinctrl_desc,
+ pctl);
if (IS_ERR(pctl->pctldev)) {
dev_err(&pdev->dev, "Failed to register pinctrl device\n");
return PTR_ERR(pctl->pctldev);
diff --git a/drivers/pinctrl/pinctrl-rockchip.c b/drivers/pinctrl/pinctrl-rockchip.c
index bf032b9..a91026e 100644
--- a/drivers/pinctrl/pinctrl-rockchip.c
+++ b/drivers/pinctrl/pinctrl-rockchip.c
@@ -99,6 +99,15 @@
};
/**
+ * enum type index corresponding to rockchip_pull_list arrays index.
+ */
+enum rockchip_pin_pull_type {
+ PULL_TYPE_IO_DEFAULT = 0,
+ PULL_TYPE_IO_1V8_ONLY,
+ PULL_TYPE_MAX
+};
+
+/**
* @drv_type: drive strength variant using rockchip_perpin_drv_type
* @offset: if initialized to -1 it will be autocalculated, by specifying
* an initial offset value the relevant source offset can be reset
@@ -123,6 +132,7 @@
* @bank_num: number of the bank, to account for holes
* @iomux: array describing the 4 iomux sources of the bank
* @drv: array describing the 4 drive strength sources of the bank
+ * @pull_type: array describing the 4 pull type sources of the bank
* @valid: are all necessary informations present
* @of_node: dt node of this bank
* @drvdata: common pinctrl basedata
@@ -143,6 +153,7 @@
u8 bank_num;
struct rockchip_iomux iomux[4];
struct rockchip_drv drv[4];
+ enum rockchip_pin_pull_type pull_type[4];
bool valid;
struct device_node *of_node;
struct rockchip_pinctrl *drvdata;
@@ -198,6 +209,31 @@
}, \
}
+#define PIN_BANK_DRV_FLAGS_PULL_FLAGS(id, pins, label, drv0, drv1, \
+ drv2, drv3, pull0, pull1, \
+ pull2, pull3) \
+ { \
+ .bank_num = id, \
+ .nr_pins = pins, \
+ .name = label, \
+ .iomux = { \
+ { .offset = -1 }, \
+ { .offset = -1 }, \
+ { .offset = -1 }, \
+ { .offset = -1 }, \
+ }, \
+ .drv = { \
+ { .drv_type = drv0, .offset = -1 }, \
+ { .drv_type = drv1, .offset = -1 }, \
+ { .drv_type = drv2, .offset = -1 }, \
+ { .drv_type = drv3, .offset = -1 }, \
+ }, \
+ .pull_type[0] = pull0, \
+ .pull_type[1] = pull1, \
+ .pull_type[2] = pull2, \
+ .pull_type[3] = pull3, \
+ }
+
#define PIN_BANK_IOMUX_DRV_FLAGS_OFFSET(id, pins, label, iom0, iom1, \
iom2, iom3, drv0, drv1, drv2, \
drv3, offset0, offset1, \
@@ -220,6 +256,34 @@
}, \
}
+#define PIN_BANK_IOMUX_FLAGS_DRV_FLAGS_OFFSET_PULL_FLAGS(id, pins, \
+ label, iom0, iom1, iom2, \
+ iom3, drv0, drv1, drv2, \
+ drv3, offset0, offset1, \
+ offset2, offset3, pull0, \
+ pull1, pull2, pull3) \
+ { \
+ .bank_num = id, \
+ .nr_pins = pins, \
+ .name = label, \
+ .iomux = { \
+ { .type = iom0, .offset = -1 }, \
+ { .type = iom1, .offset = -1 }, \
+ { .type = iom2, .offset = -1 }, \
+ { .type = iom3, .offset = -1 }, \
+ }, \
+ .drv = { \
+ { .drv_type = drv0, .offset = offset0 }, \
+ { .drv_type = drv1, .offset = offset1 }, \
+ { .drv_type = drv2, .offset = offset2 }, \
+ { .drv_type = drv3, .offset = offset3 }, \
+ }, \
+ .pull_type[0] = pull0, \
+ .pull_type[1] = pull1, \
+ .pull_type[2] = pull2, \
+ .pull_type[3] = pull3, \
+ }
+
/**
*/
struct rockchip_pin_ctrl {
@@ -1020,12 +1084,27 @@
return ret;
}
+static int rockchip_pull_list[PULL_TYPE_MAX][4] = {
+ {
+ PIN_CONFIG_BIAS_DISABLE,
+ PIN_CONFIG_BIAS_PULL_UP,
+ PIN_CONFIG_BIAS_PULL_DOWN,
+ PIN_CONFIG_BIAS_BUS_HOLD
+ },
+ {
+ PIN_CONFIG_BIAS_DISABLE,
+ PIN_CONFIG_BIAS_PULL_DOWN,
+ PIN_CONFIG_BIAS_DISABLE,
+ PIN_CONFIG_BIAS_PULL_UP
+ },
+};
+
static int rockchip_get_pull(struct rockchip_pin_bank *bank, int pin_num)
{
struct rockchip_pinctrl *info = bank->drvdata;
struct rockchip_pin_ctrl *ctrl = info->ctrl;
struct regmap *regmap;
- int reg, ret;
+ int reg, ret, pull_type;
u8 bit;
u32 data;
@@ -1048,22 +1127,11 @@
case RK3288:
case RK3368:
case RK3399:
+ pull_type = bank->pull_type[pin_num / 8];
data >>= bit;
data &= (1 << RK3188_PULL_BITS_PER_PIN) - 1;
- switch (data) {
- case 0:
- return PIN_CONFIG_BIAS_DISABLE;
- case 1:
- return PIN_CONFIG_BIAS_PULL_UP;
- case 2:
- return PIN_CONFIG_BIAS_PULL_DOWN;
- case 3:
- return PIN_CONFIG_BIAS_BUS_HOLD;
- }
-
- dev_err(info->dev, "unknown pull setting\n");
- return -EIO;
+ return rockchip_pull_list[pull_type][data];
default:
dev_err(info->dev, "unsupported pinctrl type\n");
return -EINVAL;
@@ -1076,7 +1144,7 @@
struct rockchip_pinctrl *info = bank->drvdata;
struct rockchip_pin_ctrl *ctrl = info->ctrl;
struct regmap *regmap;
- int reg, ret;
+ int reg, ret, i, pull_type;
unsigned long flags;
u8 bit;
u32 data, rmask;
@@ -1105,30 +1173,28 @@
case RK3288:
case RK3368:
case RK3399:
+ pull_type = bank->pull_type[pin_num / 8];
+ ret = -EINVAL;
+ for (i = 0; i < ARRAY_SIZE(rockchip_pull_list[pull_type]);
+ i++) {
+ if (rockchip_pull_list[pull_type][i] == pull) {
+ ret = i;
+ break;
+ }
+ }
+
+ if (ret < 0) {
+ dev_err(info->dev, "unsupported pull setting %d\n",
+ pull);
+ return ret;
+ }
+
spin_lock_irqsave(&bank->slock, flags);
/* enable the write to the equivalent lower bits */
data = ((1 << RK3188_PULL_BITS_PER_PIN) - 1) << (bit + 16);
rmask = data | (data >> 16);
-
- switch (pull) {
- case PIN_CONFIG_BIAS_DISABLE:
- break;
- case PIN_CONFIG_BIAS_PULL_UP:
- data |= (1 << bit);
- break;
- case PIN_CONFIG_BIAS_PULL_DOWN:
- data |= (2 << bit);
- break;
- case PIN_CONFIG_BIAS_BUS_HOLD:
- data |= (3 << bit);
- break;
- default:
- spin_unlock_irqrestore(&bank->slock, flags);
- dev_err(info->dev, "unsupported pull setting %d\n",
- pull);
- return -EINVAL;
- }
+ data |= (ret << bit);
ret = regmap_update_bits(regmap, reg, rmask, data);
@@ -1208,6 +1274,16 @@
return 0;
}
+static int rockchip_gpio_get_direction(struct gpio_chip *chip, unsigned offset)
+{
+ struct rockchip_pin_bank *bank = gpiochip_get_data(chip);
+ u32 data;
+
+ data = readl_relaxed(bank->reg_base + GPIO_SWPORT_DDR);
+
+ return !(data & BIT(offset));
+}
+
/*
* The calls to gpio_direction_output() and gpio_direction_input()
* leads to this function call (via the pinctrl_gpio_direction_{input|output}()
@@ -1636,7 +1712,7 @@
if (ret)
return ret;
- info->pctl_dev = pinctrl_register(ctrldesc, &pdev->dev, info);
+ info->pctl_dev = devm_pinctrl_register(&pdev->dev, ctrldesc, info);
if (IS_ERR(info->pctl_dev)) {
dev_err(&pdev->dev, "could not register pinctrl driver\n");
return PTR_ERR(info->pctl_dev);
@@ -1741,6 +1817,7 @@
.free = gpiochip_generic_free,
.set = rockchip_gpio_set,
.get = rockchip_gpio_get,
+ .get_direction = rockchip_gpio_get_direction,
.direction_input = rockchip_gpio_direction_input,
.direction_output = rockchip_gpio_direction_output,
.to_irq = rockchip_gpio_to_irq,
@@ -2541,19 +2618,24 @@
};
static struct rockchip_pin_bank rk3399_pin_banks[] = {
- PIN_BANK_IOMUX_DRV_FLAGS_OFFSET(0, 32, "gpio0", IOMUX_SOURCE_PMU,
- IOMUX_SOURCE_PMU,
- IOMUX_SOURCE_PMU,
- IOMUX_SOURCE_PMU,
- DRV_TYPE_IO_1V8_ONLY,
- DRV_TYPE_IO_1V8_ONLY,
- DRV_TYPE_IO_DEFAULT,
- DRV_TYPE_IO_DEFAULT,
- 0x0,
- 0x8,
- -1,
- -1
- ),
+ PIN_BANK_IOMUX_FLAGS_DRV_FLAGS_OFFSET_PULL_FLAGS(0, 32, "gpio0",
+ IOMUX_SOURCE_PMU,
+ IOMUX_SOURCE_PMU,
+ IOMUX_SOURCE_PMU,
+ IOMUX_SOURCE_PMU,
+ DRV_TYPE_IO_1V8_ONLY,
+ DRV_TYPE_IO_1V8_ONLY,
+ DRV_TYPE_IO_DEFAULT,
+ DRV_TYPE_IO_DEFAULT,
+ 0x0,
+ 0x8,
+ -1,
+ -1,
+ PULL_TYPE_IO_1V8_ONLY,
+ PULL_TYPE_IO_1V8_ONLY,
+ PULL_TYPE_IO_DEFAULT,
+ PULL_TYPE_IO_DEFAULT
+ ),
PIN_BANK_IOMUX_DRV_FLAGS_OFFSET(1, 32, "gpio1", IOMUX_SOURCE_PMU,
IOMUX_SOURCE_PMU,
IOMUX_SOURCE_PMU,
@@ -2567,11 +2649,15 @@
0x30,
0x38
),
- PIN_BANK_DRV_FLAGS(2, 32, "gpio2", DRV_TYPE_IO_1V8_OR_3V0,
- DRV_TYPE_IO_1V8_OR_3V0,
- DRV_TYPE_IO_1V8_ONLY,
- DRV_TYPE_IO_1V8_ONLY
- ),
+ PIN_BANK_DRV_FLAGS_PULL_FLAGS(2, 32, "gpio2", DRV_TYPE_IO_1V8_OR_3V0,
+ DRV_TYPE_IO_1V8_OR_3V0,
+ DRV_TYPE_IO_1V8_ONLY,
+ DRV_TYPE_IO_1V8_ONLY,
+ PULL_TYPE_IO_DEFAULT,
+ PULL_TYPE_IO_DEFAULT,
+ PULL_TYPE_IO_1V8_ONLY,
+ PULL_TYPE_IO_1V8_ONLY
+ ),
PIN_BANK_DRV_FLAGS(3, 32, "gpio3", DRV_TYPE_IO_3V3_ONLY,
DRV_TYPE_IO_3V3_ONLY,
DRV_TYPE_IO_3V3_ONLY,
diff --git a/drivers/pinctrl/pinctrl-st.c b/drivers/pinctrl/pinctrl-st.c
index cab66c6..d0ba968 100644
--- a/drivers/pinctrl/pinctrl-st.c
+++ b/drivers/pinctrl/pinctrl-st.c
@@ -1724,7 +1724,7 @@
pctl_desc->confops = &st_confops;
pctl_desc->name = dev_name(&pdev->dev);
- info->pctl = pinctrl_register(pctl_desc, &pdev->dev, info);
+ info->pctl = devm_pinctrl_register(&pdev->dev, pctl_desc, info);
if (IS_ERR(info->pctl)) {
dev_err(&pdev->dev, "Failed pinctrl registration\n");
return PTR_ERR(info->pctl);
diff --git a/drivers/pinctrl/pinctrl-tb10x.c b/drivers/pinctrl/pinctrl-tb10x.c
index 6546b9b..edfba50 100644
--- a/drivers/pinctrl/pinctrl-tb10x.c
+++ b/drivers/pinctrl/pinctrl-tb10x.c
@@ -582,7 +582,7 @@
.get_group_name = tb10x_get_group_name,
.get_group_pins = tb10x_get_group_pins,
.dt_node_to_map = tb10x_dt_node_to_map,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
static int tb10x_get_functions_count(struct pinctrl_dev *pctl)
@@ -806,7 +806,7 @@
}
}
- state->pctl = pinctrl_register(&tb10x_pindesc, dev, state);
+ state->pctl = devm_pinctrl_register(dev, &tb10x_pindesc, state);
if (IS_ERR(state->pctl)) {
dev_err(dev, "could not register TB10x pin driver\n");
ret = PTR_ERR(state->pctl);
@@ -824,7 +824,6 @@
{
struct tb10x_pinctrl *state = platform_get_drvdata(pdev);
- pinctrl_unregister(state->pctl);
mutex_destroy(&state->mutex);
return 0;
diff --git a/drivers/pinctrl/pinctrl-tz1090-pdc.c b/drivers/pinctrl/pinctrl-tz1090-pdc.c
index b89ad3c..e70e362 100644
--- a/drivers/pinctrl/pinctrl-tz1090-pdc.c
+++ b/drivers/pinctrl/pinctrl-tz1090-pdc.c
@@ -947,7 +947,8 @@
if (IS_ERR(pmx->regs))
return PTR_ERR(pmx->regs);
- pmx->pctl = pinctrl_register(&tz1090_pdc_pinctrl_desc, &pdev->dev, pmx);
+ pmx->pctl = devm_pinctrl_register(&pdev->dev, &tz1090_pdc_pinctrl_desc,
+ pmx);
if (IS_ERR(pmx->pctl)) {
dev_err(&pdev->dev, "Couldn't register pinctrl driver\n");
return PTR_ERR(pmx->pctl);
@@ -960,15 +961,6 @@
return 0;
}
-static int tz1090_pdc_pinctrl_remove(struct platform_device *pdev)
-{
- struct tz1090_pdc_pmx *pmx = platform_get_drvdata(pdev);
-
- pinctrl_unregister(pmx->pctl);
-
- return 0;
-}
-
static const struct of_device_id tz1090_pdc_pinctrl_of_match[] = {
{ .compatible = "img,tz1090-pdc-pinctrl", },
{ },
@@ -980,7 +972,6 @@
.of_match_table = tz1090_pdc_pinctrl_of_match,
},
.probe = tz1090_pdc_pinctrl_probe,
- .remove = tz1090_pdc_pinctrl_remove,
};
static int __init tz1090_pdc_pinctrl_init(void)
diff --git a/drivers/pinctrl/pinctrl-tz1090.c b/drivers/pinctrl/pinctrl-tz1090.c
index 5425299..04cbe53 100644
--- a/drivers/pinctrl/pinctrl-tz1090.c
+++ b/drivers/pinctrl/pinctrl-tz1090.c
@@ -1962,7 +1962,8 @@
if (IS_ERR(pmx->regs))
return PTR_ERR(pmx->regs);
- pmx->pctl = pinctrl_register(&tz1090_pinctrl_desc, &pdev->dev, pmx);
+ pmx->pctl = devm_pinctrl_register(&pdev->dev, &tz1090_pinctrl_desc,
+ pmx);
if (IS_ERR(pmx->pctl)) {
dev_err(&pdev->dev, "Couldn't register pinctrl driver\n");
return PTR_ERR(pmx->pctl);
@@ -1975,15 +1976,6 @@
return 0;
}
-static int tz1090_pinctrl_remove(struct platform_device *pdev)
-{
- struct tz1090_pmx *pmx = platform_get_drvdata(pdev);
-
- pinctrl_unregister(pmx->pctl);
-
- return 0;
-}
-
static const struct of_device_id tz1090_pinctrl_of_match[] = {
{ .compatible = "img,tz1090-pinctrl", },
{ },
@@ -1995,7 +1987,6 @@
.of_match_table = tz1090_pinctrl_of_match,
},
.probe = tz1090_pinctrl_probe,
- .remove = tz1090_pinctrl_remove,
};
static int __init tz1090_pinctrl_init(void)
diff --git a/drivers/pinctrl/pinctrl-u300.c b/drivers/pinctrl/pinctrl-u300.c
index c076021..d1af908a 100644
--- a/drivers/pinctrl/pinctrl-u300.c
+++ b/drivers/pinctrl/pinctrl-u300.c
@@ -1067,7 +1067,7 @@
if (IS_ERR(upmx->virtbase))
return PTR_ERR(upmx->virtbase);
- upmx->pctl = pinctrl_register(&u300_pmx_desc, &pdev->dev, upmx);
+ upmx->pctl = devm_pinctrl_register(&pdev->dev, &u300_pmx_desc, upmx);
if (IS_ERR(upmx->pctl)) {
dev_err(&pdev->dev, "could not register U300 pinmux driver\n");
return PTR_ERR(upmx->pctl);
@@ -1080,15 +1080,6 @@
return 0;
}
-static int u300_pmx_remove(struct platform_device *pdev)
-{
- struct u300_pmx *upmx = platform_get_drvdata(pdev);
-
- pinctrl_unregister(upmx->pctl);
-
- return 0;
-}
-
static const struct of_device_id u300_pinctrl_match[] = {
{ .compatible = "stericsson,pinctrl-u300" },
{},
@@ -1101,7 +1092,6 @@
.of_match_table = u300_pinctrl_match,
},
.probe = u300_pmx_probe,
- .remove = u300_pmx_remove,
};
static int __init u300_pmx_init(void)
diff --git a/drivers/pinctrl/pinctrl-utils.c b/drivers/pinctrl/pinctrl-utils.c
index d77693f..9189fba 100644
--- a/drivers/pinctrl/pinctrl-utils.c
+++ b/drivers/pinctrl/pinctrl-utils.c
@@ -122,7 +122,7 @@
}
EXPORT_SYMBOL_GPL(pinctrl_utils_add_config);
-void pinctrl_utils_dt_free_map(struct pinctrl_dev *pctldev,
+void pinctrl_utils_free_map(struct pinctrl_dev *pctldev,
struct pinctrl_map *map, unsigned num_maps)
{
int i;
@@ -139,4 +139,4 @@
}
kfree(map);
}
-EXPORT_SYMBOL_GPL(pinctrl_utils_dt_free_map);
+EXPORT_SYMBOL_GPL(pinctrl_utils_free_map);
diff --git a/drivers/pinctrl/pinctrl-utils.h b/drivers/pinctrl/pinctrl-utils.h
index d0ffe1c..8f9f2d2 100644
--- a/drivers/pinctrl/pinctrl-utils.h
+++ b/drivers/pinctrl/pinctrl-utils.h
@@ -37,7 +37,7 @@
int pinctrl_utils_add_config(struct pinctrl_dev *pctldev,
unsigned long **configs, unsigned *num_configs,
unsigned long config);
-void pinctrl_utils_dt_free_map(struct pinctrl_dev *pctldev,
+void pinctrl_utils_free_map(struct pinctrl_dev *pctldev,
struct pinctrl_map *map, unsigned num_maps);
#endif /* __PINCTRL_UTILS_H__ */
diff --git a/drivers/pinctrl/pinctrl-zynq.c b/drivers/pinctrl/pinctrl-zynq.c
index 76f1abd..8fdc60c 100644
--- a/drivers/pinctrl/pinctrl-zynq.c
+++ b/drivers/pinctrl/pinctrl-zynq.c
@@ -862,7 +862,7 @@
.get_group_name = zynq_pctrl_get_group_name,
.get_group_pins = zynq_pctrl_get_group_pins,
.dt_node_to_map = pinconf_generic_dt_node_to_map_all,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
/* pinmux */
@@ -1195,7 +1195,7 @@
pctrl->funcs = zynq_pmux_functions;
pctrl->nfuncs = ARRAY_SIZE(zynq_pmux_functions);
- pctrl->pctrl = pinctrl_register(&zynq_desc, &pdev->dev, pctrl);
+ pctrl->pctrl = devm_pinctrl_register(&pdev->dev, &zynq_desc, pctrl);
if (IS_ERR(pctrl->pctrl))
return PTR_ERR(pctrl->pctrl);
@@ -1206,15 +1206,6 @@
return 0;
}
-static int zynq_pinctrl_remove(struct platform_device *pdev)
-{
- struct zynq_pinctrl *pctrl = platform_get_drvdata(pdev);
-
- pinctrl_unregister(pctrl->pctrl);
-
- return 0;
-}
-
static const struct of_device_id zynq_pinctrl_of_match[] = {
{ .compatible = "xlnx,pinctrl-zynq" },
{ }
@@ -1227,7 +1218,6 @@
.of_match_table = zynq_pinctrl_of_match,
},
.probe = zynq_pinctrl_probe,
- .remove = zynq_pinctrl_remove,
};
static int __init zynq_pinctrl_init(void)
diff --git a/drivers/pinctrl/pxa/Kconfig b/drivers/pinctrl/pxa/Kconfig
index 990667f..c29bdcf 100644
--- a/drivers/pinctrl/pxa/Kconfig
+++ b/drivers/pinctrl/pxa/Kconfig
@@ -6,12 +6,20 @@
select PINCONF
select GENERIC_PINCONF
+config PINCTRL_PXA25X
+ tristate "Marvell PXA25x pin controller driver"
+ select PINCTRL_PXA
+ default y if PXA25x
+ help
+ This is the pinctrl, pinmux, pinconf driver for the Marvell
+ PXA2xx block found in the pxa25x platforms.
+
config PINCTRL_PXA27X
tristate "Marvell PXA27x pin controller driver"
select PINCTRL_PXA
default y if PXA27x
help
This is the pinctrl, pinmux, pinconf driver for the Marvell
- PXA2xx block found in the pxa25x and pxa27x platforms.
+ PXA2xx block found in the pxa27x platforms.
endif
diff --git a/drivers/pinctrl/pxa/Makefile b/drivers/pinctrl/pxa/Makefile
index f1d56af..ca2ade1 100644
--- a/drivers/pinctrl/pxa/Makefile
+++ b/drivers/pinctrl/pxa/Makefile
@@ -1,2 +1,3 @@
# Marvell PXA pin control drivers
+obj-$(CONFIG_PINCTRL_PXA25X) += pinctrl-pxa2xx.o pinctrl-pxa25x.o
obj-$(CONFIG_PINCTRL_PXA27X) += pinctrl-pxa2xx.o pinctrl-pxa27x.o
diff --git a/drivers/pinctrl/pxa/pinctrl-pxa25x.c b/drivers/pinctrl/pxa/pinctrl-pxa25x.c
new file mode 100644
index 0000000..b98ecb3
--- /dev/null
+++ b/drivers/pinctrl/pxa/pinctrl-pxa25x.c
@@ -0,0 +1,274 @@
+/*
+ * Marvell PXA25x family pin control
+ *
+ * Copyright (C) 2016 Robert Jarzmik
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ */
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/pinctrl/pinctrl.h>
+
+#include "pinctrl-pxa2xx.h"
+
+static const struct pxa_desc_pin pxa25x_pins[] = {
+ PXA_GPIO_ONLY_PIN(PXA_PINCTRL_PIN(0)),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(1),
+ PXA_FUNCTION(0, 1, "GP_RST")),
+ PXA_GPIO_ONLY_PIN(PXA_PINCTRL_PIN(2)),
+ PXA_GPIO_ONLY_PIN(PXA_PINCTRL_PIN(3)),
+ PXA_GPIO_ONLY_PIN(PXA_PINCTRL_PIN(4)),
+ PXA_GPIO_ONLY_PIN(PXA_PINCTRL_PIN(5)),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(6),
+ PXA_FUNCTION(1, 1, "MMCCLK")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(7),
+ PXA_FUNCTION(1, 1, "48_MHz")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(8),
+ PXA_FUNCTION(1, 1, "MMCCS0")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(9),
+ PXA_FUNCTION(1, 1, "MMCCS1")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(10),
+ PXA_FUNCTION(1, 1, "RTCCLK")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(11),
+ PXA_FUNCTION(1, 1, "3_6_MHz")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(12),
+ PXA_FUNCTION(1, 1, "32_kHz")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(13),
+ PXA_FUNCTION(1, 2, "MBGNT")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(14),
+ PXA_FUNCTION(0, 1, "MBREQ")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(15),
+ PXA_FUNCTION(1, 2, "nCS_1")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(16),
+ PXA_FUNCTION(1, 2, "PWM0")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(17),
+ PXA_FUNCTION(1, 2, "PWM1")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(18),
+ PXA_FUNCTION(0, 1, "RDY")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(19),
+ PXA_FUNCTION(0, 1, "DREQ[1]")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(20),
+ PXA_FUNCTION(0, 1, "DREQ[0]")),
+ PXA_GPIO_ONLY_PIN(PXA_PINCTRL_PIN(21)),
+ PXA_GPIO_ONLY_PIN(PXA_PINCTRL_PIN(22)),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(23),
+ PXA_FUNCTION(1, 2, "SCLK")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(24),
+ PXA_FUNCTION(1, 2, "SFRM")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(25),
+ PXA_FUNCTION(1, 2, "TXD")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(26),
+ PXA_FUNCTION(0, 1, "RXD")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(27),
+ PXA_FUNCTION(0, 1, "EXTCLK")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(28),
+ PXA_FUNCTION(0, 1, "BITCLK"),
+ PXA_FUNCTION(0, 2, "BITCLK"),
+ PXA_FUNCTION(1, 1, "BITCLK")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(29),
+ PXA_FUNCTION(0, 1, "SDATA_IN0"),
+ PXA_FUNCTION(0, 2, "SDATA_IN")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(30),
+ PXA_FUNCTION(1, 1, "SDATA_OUT"),
+ PXA_FUNCTION(1, 2, "SDATA_OUT")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(31),
+ PXA_FUNCTION(1, 1, "SYNC"),
+ PXA_FUNCTION(1, 2, "SYNC")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(32),
+ PXA_FUNCTION(0, 1, "SDATA_IN1"),
+ PXA_FUNCTION(1, 1, "SYSCLK")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(33),
+ PXA_FUNCTION(1, 2, "nCS[5]")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(34),
+ PXA_FUNCTION(0, 1, "FFRXD"),
+ PXA_FUNCTION(1, 2, "MMCCS0")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(35),
+ PXA_FUNCTION(0, 1, "CTS")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(36),
+ PXA_FUNCTION(0, 1, "DCD")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(37),
+ PXA_FUNCTION(0, 1, "DSR")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(38),
+ PXA_FUNCTION(0, 1, "RI")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(39),
+ PXA_FUNCTION(1, 1, "MMCC1"),
+ PXA_FUNCTION(1, 2, "FFTXD")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(40),
+ PXA_FUNCTION(1, 2, "DTR")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(41),
+ PXA_FUNCTION(1, 2, "RTS")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(42),
+ PXA_FUNCTION(0, 1, "BTRXD"),
+ PXA_FUNCTION(0, 3, "HWRXD")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(43),
+ PXA_FUNCTION(1, 2, "BTTXD"),
+ PXA_FUNCTION(1, 3, "HWTXD")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(44),
+ PXA_FUNCTION(0, 1, "BTCTS"),
+ PXA_FUNCTION(0, 3, "HWCTS")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(45),
+ PXA_FUNCTION(1, 2, "BTRTS"),
+ PXA_FUNCTION(1, 3, "HWRTS")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(46),
+ PXA_FUNCTION(0, 1, "ICP_RXD"),
+ PXA_FUNCTION(0, 2, "RXD")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(47),
+ PXA_FUNCTION(1, 1, "TXD"),
+ PXA_FUNCTION(1, 2, "ICP_TXD")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(48),
+ PXA_FUNCTION(1, 1, "HWTXD"),
+ PXA_FUNCTION(1, 2, "nPOE")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(49),
+ PXA_FUNCTION(0, 1, "HWRXD"),
+ PXA_FUNCTION(1, 2, "nPWE")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(50),
+ PXA_FUNCTION(0, 1, "HWCTS"),
+ PXA_FUNCTION(1, 2, "nPIOR")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(51),
+ PXA_FUNCTION(1, 1, "HWRTS"),
+ PXA_FUNCTION(1, 2, "nPIOW")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(52),
+ PXA_FUNCTION(1, 2, "nPCE[1]")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(53),
+ PXA_FUNCTION(1, 1, "MMCCLK"),
+ PXA_FUNCTION(1, 2, "nPCE[2]")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(54),
+ PXA_FUNCTION(1, 1, "MMCCLK"),
+ PXA_FUNCTION(1, 2, "nPSKTSEL")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(55),
+ PXA_FUNCTION(1, 2, "nPREG")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(56),
+ PXA_FUNCTION(0, 1, "nPWAIT")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(57),
+ PXA_FUNCTION(0, 1, "nIOIS16")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(58),
+ PXA_FUNCTION(1, 2, "LDD<0>")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(59),
+ PXA_FUNCTION(1, 2, "LDD<1>")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(60),
+ PXA_FUNCTION(1, 2, "LDD<2>")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(61),
+ PXA_FUNCTION(1, 2, "LDD<3>")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(62),
+ PXA_FUNCTION(1, 2, "LDD<4>")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(63),
+ PXA_FUNCTION(1, 2, "LDD<5>")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(64),
+ PXA_FUNCTION(1, 2, "LDD<6>")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(65),
+ PXA_FUNCTION(1, 2, "LDD<7>")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(66),
+ PXA_FUNCTION(0, 1, "MBREQ"),
+ PXA_FUNCTION(1, 2, "LDD<8>")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(67),
+ PXA_FUNCTION(1, 1, "MMCCS0"),
+ PXA_FUNCTION(1, 2, "LDD<9>")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(68),
+ PXA_FUNCTION(1, 1, "MMCCS1"),
+ PXA_FUNCTION(1, 2, "LDD<10>")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(69),
+ PXA_FUNCTION(1, 1, "MMCCLK"),
+ PXA_FUNCTION(1, 2, "LDD<11>")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(70),
+ PXA_FUNCTION(1, 1, "RTCCLK"),
+ PXA_FUNCTION(1, 2, "LDD<12>")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(71),
+ PXA_FUNCTION(1, 1, "3_6_MHz"),
+ PXA_FUNCTION(1, 2, "LDD<13>")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(72),
+ PXA_FUNCTION(1, 1, "32_kHz"),
+ PXA_FUNCTION(1, 2, "LDD<14>")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(73),
+ PXA_FUNCTION(1, 1, "MBGNT"),
+ PXA_FUNCTION(1, 2, "LDD<15>")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(74),
+ PXA_FUNCTION(1, 2, "LCD_FCLK")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(75),
+ PXA_FUNCTION(1, 2, "LCD_LCLK")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(76),
+ PXA_FUNCTION(1, 2, "LCD_PCLK")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(77),
+ PXA_FUNCTION(1, 2, "LCD_ACBIAS")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(78),
+ PXA_FUNCTION(1, 2, "nCS<2>")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(79),
+ PXA_FUNCTION(1, 2, "nCS<3>")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(80),
+ PXA_FUNCTION(1, 2, "nCS<4>")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(81),
+ PXA_FUNCTION(0, 1, "NSSPSCLK"),
+ PXA_FUNCTION(1, 1, "NSSPSCLK")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(82),
+ PXA_FUNCTION(0, 1, "NSSPSFRM"),
+ PXA_FUNCTION(1, 1, "NSSPSFRM")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(83),
+ PXA_FUNCTION(0, 2, "NSSPRXD"),
+ PXA_FUNCTION(1, 1, "NSSPTXD")),
+ PXA_GPIO_PIN(PXA_PINCTRL_PIN(84),
+ PXA_FUNCTION(0, 2, "NSSPRXD"),
+ PXA_FUNCTION(1, 1, "NSSPTXD")),
+};
+
+static int pxa25x_pinctrl_probe(struct platform_device *pdev)
+{
+ int ret, i;
+ void __iomem *base_af[8];
+ void __iomem *base_dir[4];
+ void __iomem *base_sleep[4];
+ struct resource *res;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ base_af[0] = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(base_af[0]))
+ return PTR_ERR(base_af[0]);
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 1);
+ base_dir[0] = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(base_dir[0]))
+ return PTR_ERR(base_dir[0]);
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 2);
+ base_dir[3] = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(base_dir[3]))
+ return PTR_ERR(base_dir[3]);
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 3);
+ base_sleep[0] = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(base_sleep[0]))
+ return PTR_ERR(base_sleep[0]);
+
+ for (i = 0; i < ARRAY_SIZE(base_af); i++)
+ base_af[i] = base_af[0] + sizeof(base_af[0]) * i;
+ for (i = 0; i < 3; i++)
+ base_dir[i] = base_dir[0] + sizeof(base_dir[0]) * i;
+ for (i = 0; i < ARRAY_SIZE(base_sleep); i++)
+ base_sleep[i] = base_sleep[0] + sizeof(base_af[0]) * i;
+
+ ret = pxa2xx_pinctrl_init(pdev, pxa25x_pins, ARRAY_SIZE(pxa25x_pins),
+ base_af, base_dir, base_sleep);
+ return ret;
+}
+
+static const struct of_device_id pxa25x_pinctrl_match[] = {
+ { .compatible = "marvell,pxa25x-pinctrl", },
+ {}
+};
+MODULE_DEVICE_TABLE(of, pxa25x_pinctrl_match);
+
+static struct platform_driver pxa25x_pinctrl_driver = {
+ .probe = pxa25x_pinctrl_probe,
+ .driver = {
+ .name = "pxa25x-pinctrl",
+ .of_match_table = pxa25x_pinctrl_match,
+ },
+};
+module_platform_driver(pxa25x_pinctrl_driver);
+
+MODULE_AUTHOR("Robert Jarzmik <robert.jarzmik@free.fr>");
+MODULE_DESCRIPTION("Marvell PXA25x pinctrl driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/pinctrl/pxa/pinctrl-pxa2xx.c b/drivers/pinctrl/pxa/pinctrl-pxa2xx.c
index f553313..866aa3c 100644
--- a/drivers/pinctrl/pxa/pinctrl-pxa2xx.c
+++ b/drivers/pinctrl/pxa/pinctrl-pxa2xx.c
@@ -57,7 +57,7 @@
static const struct pinctrl_ops pxa2xx_pctl_ops = {
#ifdef CONFIG_OF
.dt_node_to_map = pinconf_generic_dt_node_to_map_all,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
#endif
.get_groups_count = pxa2xx_pctrl_get_groups_count,
.get_group_name = pxa2xx_pctrl_get_group_name,
@@ -416,7 +416,7 @@
if (ret)
return ret;
- pctl->pctl_dev = pinctrl_register(&pctl->desc, &pdev->dev, pctl);
+ pctl->pctl_dev = devm_pinctrl_register(&pdev->dev, &pctl->desc, pctl);
if (IS_ERR(pctl->pctl_dev)) {
dev_err(&pdev->dev, "couldn't register pinctrl driver\n");
return PTR_ERR(pctl->pctl_dev);
diff --git a/drivers/pinctrl/qcom/pinctrl-msm.c b/drivers/pinctrl/qcom/pinctrl-msm.c
index 8777cf0..1a44e1d 100644
--- a/drivers/pinctrl/qcom/pinctrl-msm.c
+++ b/drivers/pinctrl/qcom/pinctrl-msm.c
@@ -101,7 +101,7 @@
.get_group_name = msm_get_group_name,
.get_group_pins = msm_get_group_pins,
.dt_node_to_map = pinconf_generic_dt_node_to_map_group,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
static int msm_get_functions_count(struct pinctrl_dev *pctldev)
@@ -898,17 +898,16 @@
msm_pinctrl_desc.name = dev_name(&pdev->dev);
msm_pinctrl_desc.pins = pctrl->soc->pins;
msm_pinctrl_desc.npins = pctrl->soc->npins;
- pctrl->pctrl = pinctrl_register(&msm_pinctrl_desc, &pdev->dev, pctrl);
+ pctrl->pctrl = devm_pinctrl_register(&pdev->dev, &msm_pinctrl_desc,
+ pctrl);
if (IS_ERR(pctrl->pctrl)) {
dev_err(&pdev->dev, "Couldn't register pinctrl driver\n");
return PTR_ERR(pctrl->pctrl);
}
ret = msm_gpio_init(pctrl);
- if (ret) {
- pinctrl_unregister(pctrl->pctrl);
+ if (ret)
return ret;
- }
platform_set_drvdata(pdev, pctrl);
@@ -923,7 +922,6 @@
struct msm_pinctrl *pctrl = platform_get_drvdata(pdev);
gpiochip_remove(&pctrl->chip);
- pinctrl_unregister(pctrl->pctrl);
unregister_restart_handler(&pctrl->restart_nb);
diff --git a/drivers/pinctrl/qcom/pinctrl-spmi-gpio.c b/drivers/pinctrl/qcom/pinctrl-spmi-gpio.c
index 4e12ded..686accb 100644
--- a/drivers/pinctrl/qcom/pinctrl-spmi-gpio.c
+++ b/drivers/pinctrl/qcom/pinctrl-spmi-gpio.c
@@ -212,7 +212,7 @@
.get_group_name = pmic_gpio_get_group_name,
.get_group_pins = pmic_gpio_get_group_pins,
.dt_node_to_map = pinconf_generic_dt_node_to_map_group,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
static int pmic_gpio_get_functions_count(struct pinctrl_dev *pctldev)
@@ -764,14 +764,14 @@
state->chip.of_gpio_n_cells = 2;
state->chip.can_sleep = false;
- state->ctrl = pinctrl_register(pctrldesc, dev, state);
+ state->ctrl = devm_pinctrl_register(dev, pctrldesc, state);
if (IS_ERR(state->ctrl))
return PTR_ERR(state->ctrl);
ret = gpiochip_add_data(&state->chip, state);
if (ret) {
dev_err(state->dev, "can't add gpio chip\n");
- goto err_chip;
+ return ret;
}
ret = gpiochip_add_pin_range(&state->chip, dev_name(dev), 0, 0, npins);
@@ -784,8 +784,6 @@
err_range:
gpiochip_remove(&state->chip);
-err_chip:
- pinctrl_unregister(state->ctrl);
return ret;
}
@@ -794,7 +792,6 @@
struct pmic_gpio_state *state = platform_get_drvdata(pdev);
gpiochip_remove(&state->chip);
- pinctrl_unregister(state->ctrl);
return 0;
}
diff --git a/drivers/pinctrl/qcom/pinctrl-spmi-mpp.c b/drivers/pinctrl/qcom/pinctrl-spmi-mpp.c
index 2a3e549..1735ffe 100644
--- a/drivers/pinctrl/qcom/pinctrl-spmi-mpp.c
+++ b/drivers/pinctrl/qcom/pinctrl-spmi-mpp.c
@@ -235,7 +235,7 @@
.get_group_name = pmic_mpp_get_group_name,
.get_group_pins = pmic_mpp_get_group_pins,
.dt_node_to_map = pinconf_generic_dt_node_to_map_group,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
static int pmic_mpp_get_functions_count(struct pinctrl_dev *pctldev)
@@ -877,14 +877,14 @@
state->chip.of_gpio_n_cells = 2;
state->chip.can_sleep = false;
- state->ctrl = pinctrl_register(pctrldesc, dev, state);
+ state->ctrl = devm_pinctrl_register(dev, pctrldesc, state);
if (IS_ERR(state->ctrl))
return PTR_ERR(state->ctrl);
ret = gpiochip_add_data(&state->chip, state);
if (ret) {
dev_err(state->dev, "can't add gpio chip\n");
- goto err_chip;
+ return ret;
}
ret = gpiochip_add_pin_range(&state->chip, dev_name(dev), 0, 0, npins);
@@ -897,8 +897,6 @@
err_range:
gpiochip_remove(&state->chip);
-err_chip:
- pinctrl_unregister(state->ctrl);
return ret;
}
@@ -907,7 +905,6 @@
struct pmic_mpp_state *state = platform_get_drvdata(pdev);
gpiochip_remove(&state->chip);
- pinctrl_unregister(state->ctrl);
return 0;
}
diff --git a/drivers/pinctrl/qcom/pinctrl-ssbi-gpio.c b/drivers/pinctrl/qcom/pinctrl-ssbi-gpio.c
index cd8580d..d3f5501d 100644
--- a/drivers/pinctrl/qcom/pinctrl-ssbi-gpio.c
+++ b/drivers/pinctrl/qcom/pinctrl-ssbi-gpio.c
@@ -200,7 +200,7 @@
.get_group_name = pm8xxx_get_group_name,
.get_group_pins = pm8xxx_get_group_pins,
.dt_node_to_map = pinconf_generic_dt_node_to_map_group,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
static int pm8xxx_get_functions_count(struct pinctrl_dev *pctldev)
@@ -729,7 +729,7 @@
pctrl->desc.custom_conf_items = pm8xxx_conf_items;
#endif
- pctrl->pctrl = pinctrl_register(&pctrl->desc, &pdev->dev, pctrl);
+ pctrl->pctrl = devm_pinctrl_register(&pdev->dev, &pctrl->desc, pctrl);
if (IS_ERR(pctrl->pctrl)) {
dev_err(&pdev->dev, "couldn't register pm8xxx gpio driver\n");
return PTR_ERR(pctrl->pctrl);
@@ -745,7 +745,7 @@
ret = gpiochip_add_data(&pctrl->chip, pctrl);
if (ret) {
dev_err(&pdev->dev, "failed register gpiochip\n");
- goto unregister_pinctrl;
+ return ret;
}
ret = gpiochip_add_pin_range(&pctrl->chip,
@@ -765,9 +765,6 @@
unregister_gpiochip:
gpiochip_remove(&pctrl->chip);
-unregister_pinctrl:
- pinctrl_unregister(pctrl->pctrl);
-
return ret;
}
@@ -777,8 +774,6 @@
gpiochip_remove(&pctrl->chip);
- pinctrl_unregister(pctrl->pctrl);
-
return 0;
}
diff --git a/drivers/pinctrl/qcom/pinctrl-ssbi-mpp.c b/drivers/pinctrl/qcom/pinctrl-ssbi-mpp.c
index 54a5402..9191727 100644
--- a/drivers/pinctrl/qcom/pinctrl-ssbi-mpp.c
+++ b/drivers/pinctrl/qcom/pinctrl-ssbi-mpp.c
@@ -277,7 +277,7 @@
.get_group_name = pm8xxx_get_group_name,
.get_group_pins = pm8xxx_get_group_pins,
.dt_node_to_map = pinconf_generic_dt_node_to_map_group,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
static int pm8xxx_get_functions_count(struct pinctrl_dev *pctldev)
@@ -820,7 +820,7 @@
pctrl->desc.custom_conf_items = pm8xxx_conf_items;
#endif
- pctrl->pctrl = pinctrl_register(&pctrl->desc, &pdev->dev, pctrl);
+ pctrl->pctrl = devm_pinctrl_register(&pdev->dev, &pctrl->desc, pctrl);
if (IS_ERR(pctrl->pctrl)) {
dev_err(&pdev->dev, "couldn't register pm8xxx mpp driver\n");
return PTR_ERR(pctrl->pctrl);
@@ -836,7 +836,7 @@
ret = gpiochip_add_data(&pctrl->chip, pctrl);
if (ret) {
dev_err(&pdev->dev, "failed register gpiochip\n");
- goto unregister_pinctrl;
+ return ret;
}
ret = gpiochip_add_pin_range(&pctrl->chip,
@@ -856,9 +856,6 @@
unregister_gpiochip:
gpiochip_remove(&pctrl->chip);
-unregister_pinctrl:
- pinctrl_unregister(pctrl->pctrl);
-
return ret;
}
@@ -868,8 +865,6 @@
gpiochip_remove(&pctrl->chip);
- pinctrl_unregister(pctrl->pctrl);
-
return 0;
}
diff --git a/drivers/pinctrl/samsung/pinctrl-exynos5440.c b/drivers/pinctrl/samsung/pinctrl-exynos5440.c
index 00ab63a..fb71fc3 100644
--- a/drivers/pinctrl/samsung/pinctrl-exynos5440.c
+++ b/drivers/pinctrl/samsung/pinctrl-exynos5440.c
@@ -107,6 +107,7 @@
* @nr_groups: number of pin groups available.
* @pmx_functions: list of pin functions parsed from device tree.
* @nr_functions: number of pin functions available.
+ * @range: gpio range to register with pinctrl
*/
struct exynos5440_pinctrl_priv_data {
void __iomem *reg_base;
@@ -117,6 +118,7 @@
unsigned int nr_groups;
const struct exynos5440_pmx_func *pmx_functions;
unsigned int nr_functions;
+ struct pinctrl_gpio_range range;
};
/**
@@ -742,7 +744,6 @@
struct pinctrl_desc *ctrldesc;
struct pinctrl_dev *pctl_dev;
struct pinctrl_pin_desc *pindesc, *pdesc;
- struct pinctrl_gpio_range grange;
char *pin_names;
int pin, ret;
@@ -788,18 +789,18 @@
if (ret)
return ret;
- pctl_dev = pinctrl_register(ctrldesc, &pdev->dev, priv);
+ pctl_dev = devm_pinctrl_register(&pdev->dev, ctrldesc, priv);
if (IS_ERR(pctl_dev)) {
dev_err(&pdev->dev, "could not register pinctrl driver\n");
return PTR_ERR(pctl_dev);
}
- grange.name = "exynos5440-pctrl-gpio-range";
- grange.id = 0;
- grange.base = 0;
- grange.npins = EXYNOS5440_MAX_PINS;
- grange.gc = priv->gc;
- pinctrl_add_gpio_range(pctl_dev, &grange);
+ priv->range.name = "exynos5440-pctrl-gpio-range";
+ priv->range.id = 0;
+ priv->range.base = 0;
+ priv->range.npins = EXYNOS5440_MAX_PINS;
+ priv->range.gc = priv->gc;
+ pinctrl_add_gpio_range(pctl_dev, &priv->range);
return 0;
}
diff --git a/drivers/pinctrl/samsung/pinctrl-samsung.c b/drivers/pinctrl/samsung/pinctrl-samsung.c
index 5cc97f8..ed0b708 100644
--- a/drivers/pinctrl/samsung/pinctrl-samsung.c
+++ b/drivers/pinctrl/samsung/pinctrl-samsung.c
@@ -884,7 +884,8 @@
if (ret)
return ret;
- drvdata->pctl_dev = pinctrl_register(ctrldesc, &pdev->dev, drvdata);
+ drvdata->pctl_dev = devm_pinctrl_register(&pdev->dev, ctrldesc,
+ drvdata);
if (IS_ERR(drvdata->pctl_dev)) {
dev_err(&pdev->dev, "could not register pinctrl driver\n");
return PTR_ERR(drvdata->pctl_dev);
diff --git a/drivers/pinctrl/sh-pfc/core.c b/drivers/pinctrl/sh-pfc/core.c
index ee0c1f2..9b9cee0 100644
--- a/drivers/pinctrl/sh-pfc/core.c
+++ b/drivers/pinctrl/sh-pfc/core.c
@@ -175,6 +175,21 @@
BUG();
}
+u32 sh_pfc_read_reg(struct sh_pfc *pfc, u32 reg, unsigned int width)
+{
+ return sh_pfc_read_raw_reg(sh_pfc_phys_to_virt(pfc, reg), width);
+}
+
+void sh_pfc_write_reg(struct sh_pfc *pfc, u32 reg, unsigned int width, u32 data)
+{
+ if (pfc->info->unlock_reg)
+ sh_pfc_write_raw_reg(
+ sh_pfc_phys_to_virt(pfc, pfc->info->unlock_reg), 32,
+ ~data);
+
+ sh_pfc_write_raw_reg(sh_pfc_phys_to_virt(pfc, reg), width, data);
+}
+
static void sh_pfc_config_reg_helper(struct sh_pfc *pfc,
const struct pinmux_cfg_reg *crp,
unsigned int in_pos,
@@ -585,12 +600,9 @@
static int sh_pfc_remove(struct platform_device *pdev)
{
- struct sh_pfc *pfc = platform_get_drvdata(pdev);
-
#ifdef CONFIG_PINCTRL_SH_PFC_GPIO
- sh_pfc_unregister_gpiochip(pfc);
+ sh_pfc_unregister_gpiochip(platform_get_drvdata(pdev));
#endif
- sh_pfc_unregister_pinctrl(pfc);
return 0;
}
diff --git a/drivers/pinctrl/sh-pfc/core.h b/drivers/pinctrl/sh-pfc/core.h
index 62f53b2..dc1b2ad 100644
--- a/drivers/pinctrl/sh-pfc/core.h
+++ b/drivers/pinctrl/sh-pfc/core.h
@@ -50,18 +50,19 @@
struct sh_pfc_chip *func;
#endif
- struct sh_pfc_pinctrl *pinctrl;
};
int sh_pfc_register_gpiochip(struct sh_pfc *pfc);
int sh_pfc_unregister_gpiochip(struct sh_pfc *pfc);
int sh_pfc_register_pinctrl(struct sh_pfc *pfc);
-int sh_pfc_unregister_pinctrl(struct sh_pfc *pfc);
u32 sh_pfc_read_raw_reg(void __iomem *mapped_reg, unsigned int reg_width);
void sh_pfc_write_raw_reg(void __iomem *mapped_reg, unsigned int reg_width,
u32 data);
+u32 sh_pfc_read_reg(struct sh_pfc *pfc, u32 reg, unsigned int width);
+void sh_pfc_write_reg(struct sh_pfc *pfc, u32 reg, unsigned int width,
+ u32 data);
int sh_pfc_get_pin_index(struct sh_pfc *pfc, unsigned int pin);
int sh_pfc_config_mux(struct sh_pfc *pfc, unsigned mark, int pinmux_type);
diff --git a/drivers/pinctrl/sh-pfc/pfc-r8a7790.c b/drivers/pinctrl/sh-pfc/pfc-r8a7790.c
index 0f4d48f..eed8daa 100644
--- a/drivers/pinctrl/sh-pfc/pfc-r8a7790.c
+++ b/drivers/pinctrl/sh-pfc/pfc-r8a7790.c
@@ -21,16 +21,21 @@
* Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
*/
+#include <linux/io.h>
#include <linux/kernel.h>
#include "core.h"
#include "sh_pfc.h"
+/*
+ * All pins assigned to GPIO bank 3 can be used for SD interfaces in
+ * which case they support both 3.3V and 1.8V signalling.
+ */
#define CPU_ALL_PORT(fn, sfx) \
PORT_GP_32(0, fn, sfx), \
PORT_GP_30(1, fn, sfx), \
PORT_GP_30(2, fn, sfx), \
- PORT_GP_32(3, fn, sfx), \
+ PORT_GP_CFG_32(3, fn, sfx, SH_PFC_PIN_CFG_IO_VOLTAGE), \
PORT_GP_32(4, fn, sfx), \
PORT_GP_32(5, fn, sfx)
@@ -4691,6 +4696,47 @@
"vin3_clk",
};
+#define IOCTRL6 0x8c
+
+static int r8a7790_get_io_voltage(struct sh_pfc *pfc, unsigned int pin)
+{
+ u32 data, mask;
+
+ if (WARN(pin < RCAR_GP_PIN(3, 0) || pin > RCAR_GP_PIN(3, 31), "invalid pin %#x", pin))
+ return -EINVAL;
+
+ data = ioread32(pfc->windows->virt + IOCTRL6),
+ /* Bits in IOCTRL6 are numbered in opposite order to pins */
+ mask = 0x80000000 >> (pin & 0x1f);
+
+ return (data & mask) ? 3300 : 1800;
+}
+
+static int r8a7790_set_io_voltage(struct sh_pfc *pfc, unsigned int pin, u16 mV)
+{
+ u32 data, mask;
+
+ if (WARN(pin < RCAR_GP_PIN(3, 0) || pin > RCAR_GP_PIN(3, 31), "invalid pin %#x", pin))
+ return -EINVAL;
+
+ if (mV != 1800 && mV != 3300)
+ return -EINVAL;
+
+ data = ioread32(pfc->windows->virt + IOCTRL6);
+ /* Bits in IOCTRL6 are numbered in opposite order to pins */
+ mask = 0x80000000 >> (pin & 0x1f);
+
+ if (mV == 3300)
+ data |= mask;
+ else
+ data &= ~mask;
+
+ iowrite32(~data, pfc->windows->virt); /* unlock reg */
+ iowrite32(data, pfc->windows->virt + IOCTRL6);
+
+ return 0;
+}
+
static const struct sh_pfc_function pinmux_functions[] = {
SH_PFC_FUNCTION(audio_clk),
SH_PFC_FUNCTION(avb),
@@ -5690,8 +5736,14 @@
{ },
};
+static const struct sh_pfc_soc_operations pinmux_ops = {
+ .get_io_voltage = r8a7790_get_io_voltage,
+ .set_io_voltage = r8a7790_set_io_voltage,
+};
+
const struct sh_pfc_soc_info r8a7790_pinmux_info = {
.name = "r8a77900_pfc",
+ .ops = &pinmux_ops,
.unlock_reg = 0xe6060000, /* PMMR */
.function = { PINMUX_FUNCTION_BEGIN, PINMUX_FUNCTION_END },
diff --git a/drivers/pinctrl/sh-pfc/pfc-r8a7794.c b/drivers/pinctrl/sh-pfc/pfc-r8a7794.c
index 38912cf..8bc2cf0 100644
--- a/drivers/pinctrl/sh-pfc/pfc-r8a7794.c
+++ b/drivers/pinctrl/sh-pfc/pfc-r8a7794.c
@@ -1682,6 +1682,179 @@
static const unsigned int avb_avtp_match_b_mux[] = {
AVB_AVTP_MATCH_B_MARK,
};
+/* - DU --------------------------------------------------------------------- */
+static const unsigned int du0_rgb666_pins[] = {
+ /* R[7:2], G[7:2], B[7:2] */
+ RCAR_GP_PIN(2, 7), RCAR_GP_PIN(2, 6), RCAR_GP_PIN(2, 5),
+ RCAR_GP_PIN(2, 4), RCAR_GP_PIN(2, 3), RCAR_GP_PIN(2, 2),
+ RCAR_GP_PIN(2, 15), RCAR_GP_PIN(2, 14), RCAR_GP_PIN(2, 13),
+ RCAR_GP_PIN(2, 12), RCAR_GP_PIN(2, 11), RCAR_GP_PIN(2, 10),
+ RCAR_GP_PIN(2, 23), RCAR_GP_PIN(2, 22), RCAR_GP_PIN(2, 21),
+ RCAR_GP_PIN(2, 20), RCAR_GP_PIN(2, 19), RCAR_GP_PIN(2, 18),
+};
+static const unsigned int du0_rgb666_mux[] = {
+ DU0_DR7_MARK, DU0_DR6_MARK, DU0_DR5_MARK, DU0_DR4_MARK,
+ DU0_DR3_MARK, DU0_DR2_MARK,
+ DU0_DG7_MARK, DU0_DG6_MARK, DU0_DG5_MARK, DU0_DG4_MARK,
+ DU0_DG3_MARK, DU0_DG2_MARK,
+ DU0_DB7_MARK, DU0_DB6_MARK, DU0_DB5_MARK, DU0_DB4_MARK,
+ DU0_DB3_MARK, DU0_DB2_MARK,
+};
+static const unsigned int du0_rgb888_pins[] = {
+ /* R[7:0], G[7:0], B[7:0] */
+ RCAR_GP_PIN(2, 7), RCAR_GP_PIN(2, 6), RCAR_GP_PIN(2, 5),
+ RCAR_GP_PIN(2, 4), RCAR_GP_PIN(2, 3), RCAR_GP_PIN(2, 2),
+ RCAR_GP_PIN(2, 1), RCAR_GP_PIN(2, 0),
+ RCAR_GP_PIN(2, 15), RCAR_GP_PIN(2, 14), RCAR_GP_PIN(2, 13),
+ RCAR_GP_PIN(2, 12), RCAR_GP_PIN(2, 11), RCAR_GP_PIN(2, 10),
+ RCAR_GP_PIN(2, 9), RCAR_GP_PIN(2, 8),
+ RCAR_GP_PIN(2, 23), RCAR_GP_PIN(2, 22), RCAR_GP_PIN(2, 21),
+ RCAR_GP_PIN(2, 20), RCAR_GP_PIN(2, 19), RCAR_GP_PIN(2, 18),
+ RCAR_GP_PIN(2, 17), RCAR_GP_PIN(2, 16),
+};
+static const unsigned int du0_rgb888_mux[] = {
+ DU0_DR7_MARK, DU0_DR6_MARK, DU0_DR5_MARK, DU0_DR4_MARK,
+ DU0_DR3_MARK, DU0_DR2_MARK, DU0_DR1_MARK, DU0_DR0_MARK,
+ DU0_DG7_MARK, DU0_DG6_MARK, DU0_DG5_MARK, DU0_DG4_MARK,
+ DU0_DG3_MARK, DU0_DG2_MARK, DU0_DG1_MARK, DU0_DG0_MARK,
+ DU0_DB7_MARK, DU0_DB6_MARK, DU0_DB5_MARK, DU0_DB4_MARK,
+ DU0_DB3_MARK, DU0_DB2_MARK, DU0_DB1_MARK, DU0_DB0_MARK,
+};
+static const unsigned int du0_clk0_out_pins[] = {
+ /* DOTCLKOUT0 */
+ RCAR_GP_PIN(2, 25),
+};
+static const unsigned int du0_clk0_out_mux[] = {
+ DU0_DOTCLKOUT0_MARK
+};
+static const unsigned int du0_clk1_out_pins[] = {
+ /* DOTCLKOUT1 */
+ RCAR_GP_PIN(2, 26),
+};
+static const unsigned int du0_clk1_out_mux[] = {
+ DU0_DOTCLKOUT1_MARK
+};
+static const unsigned int du0_clk_in_pins[] = {
+ /* CLKIN */
+ RCAR_GP_PIN(2, 24),
+};
+static const unsigned int du0_clk_in_mux[] = {
+ DU0_DOTCLKIN_MARK
+};
+static const unsigned int du0_sync_pins[] = {
+ /* EXVSYNC/VSYNC, EXHSYNC/HSYNC */
+ RCAR_GP_PIN(2, 28), RCAR_GP_PIN(2, 27),
+};
+static const unsigned int du0_sync_mux[] = {
+ DU0_EXVSYNC_DU0_VSYNC_MARK, DU0_EXHSYNC_DU0_HSYNC_MARK
+};
+static const unsigned int du0_oddf_pins[] = {
+ /* EXODDF/ODDF/DISP/CDE */
+ RCAR_GP_PIN(2, 29),
+};
+static const unsigned int du0_oddf_mux[] = {
+ DU0_EXODDF_DU0_ODDF_DISP_CDE_MARK,
+};
+static const unsigned int du0_cde_pins[] = {
+ /* CDE */
+ RCAR_GP_PIN(2, 31),
+};
+static const unsigned int du0_cde_mux[] = {
+ DU0_CDE_MARK,
+};
+static const unsigned int du0_disp_pins[] = {
+ /* DISP */
+ RCAR_GP_PIN(2, 30),
+};
+static const unsigned int du0_disp_mux[] = {
+ DU0_DISP_MARK
+};
+static const unsigned int du1_rgb666_pins[] = {
+ /* R[7:2], G[7:2], B[7:2] */
+ RCAR_GP_PIN(4, 7), RCAR_GP_PIN(4, 6), RCAR_GP_PIN(4, 5),
+ RCAR_GP_PIN(4, 4), RCAR_GP_PIN(4, 3), RCAR_GP_PIN(4, 2),
+ RCAR_GP_PIN(4, 15), RCAR_GP_PIN(4, 14), RCAR_GP_PIN(4, 13),
+ RCAR_GP_PIN(4, 12), RCAR_GP_PIN(4, 11), RCAR_GP_PIN(4, 10),
+ RCAR_GP_PIN(4, 23), RCAR_GP_PIN(4, 22), RCAR_GP_PIN(4, 21),
+ RCAR_GP_PIN(4, 20), RCAR_GP_PIN(4, 19), RCAR_GP_PIN(4, 18),
+};
+static const unsigned int du1_rgb666_mux[] = {
+ DU1_DR7_MARK, DU1_DR6_MARK, DU1_DR5_MARK, DU1_DR4_MARK,
+ DU1_DR3_MARK, DU1_DR2_MARK,
+ DU1_DG7_MARK, DU1_DG6_MARK, DU1_DG5_MARK, DU1_DG4_MARK,
+ DU1_DG3_MARK, DU1_DG2_MARK,
+ DU1_DB7_MARK, DU1_DB6_MARK, DU1_DB5_MARK, DU1_DB4_MARK,
+ DU1_DB3_MARK, DU1_DB2_MARK,
+};
+static const unsigned int du1_rgb888_pins[] = {
+ /* R[7:0], G[7:0], B[7:0] */
+ RCAR_GP_PIN(4, 7), RCAR_GP_PIN(4, 6), RCAR_GP_PIN(4, 5),
+ RCAR_GP_PIN(4, 4), RCAR_GP_PIN(4, 3), RCAR_GP_PIN(4, 2),
+ RCAR_GP_PIN(4, 1), RCAR_GP_PIN(4, 0),
+ RCAR_GP_PIN(4, 15), RCAR_GP_PIN(4, 14), RCAR_GP_PIN(4, 13),
+ RCAR_GP_PIN(4, 12), RCAR_GP_PIN(4, 11), RCAR_GP_PIN(4, 10),
+ RCAR_GP_PIN(4, 9), RCAR_GP_PIN(4, 8),
+ RCAR_GP_PIN(4, 23), RCAR_GP_PIN(4, 22), RCAR_GP_PIN(4, 21),
+ RCAR_GP_PIN(4, 20), RCAR_GP_PIN(4, 19), RCAR_GP_PIN(4, 18),
+ RCAR_GP_PIN(4, 17), RCAR_GP_PIN(4, 16),
+};
+static const unsigned int du1_rgb888_mux[] = {
+ DU1_DR7_MARK, DU1_DR6_MARK, DU1_DR5_MARK, DU1_DR4_MARK,
+ DU1_DR3_MARK, DU1_DR2_MARK, DU1_DR1_MARK, DU1_DR0_MARK,
+ DU1_DG7_MARK, DU1_DG6_MARK, DU1_DG5_MARK, DU1_DG4_MARK,
+ DU1_DG3_MARK, DU1_DG2_MARK, DU1_DG1_MARK, DU1_DG0_MARK,
+ DU1_DB7_MARK, DU1_DB6_MARK, DU1_DB5_MARK, DU1_DB4_MARK,
+ DU1_DB3_MARK, DU1_DB2_MARK, DU1_DB1_MARK, DU1_DB0_MARK,
+};
+static const unsigned int du1_clk0_out_pins[] = {
+ /* DOTCLKOUT0 */
+ RCAR_GP_PIN(4, 25),
+};
+static const unsigned int du1_clk0_out_mux[] = {
+ DU1_DOTCLKOUT0_MARK
+};
+static const unsigned int du1_clk1_out_pins[] = {
+ /* DOTCLKOUT1 */
+ RCAR_GP_PIN(4, 26),
+};
+static const unsigned int du1_clk1_out_mux[] = {
+ DU1_DOTCLKOUT1_MARK
+};
+static const unsigned int du1_clk_in_pins[] = {
+ /* DOTCLKIN */
+ RCAR_GP_PIN(4, 24),
+};
+static const unsigned int du1_clk_in_mux[] = {
+ DU1_DOTCLKIN_MARK
+};
+static const unsigned int du1_sync_pins[] = {
+ /* EXVSYNC/VSYNC, EXHSYNC/HSYNC */
+ RCAR_GP_PIN(4, 28), RCAR_GP_PIN(4, 27),
+};
+static const unsigned int du1_sync_mux[] = {
+ DU1_EXVSYNC_DU1_VSYNC_MARK, DU1_EXHSYNC_DU1_HSYNC_MARK
+};
+static const unsigned int du1_oddf_pins[] = {
+ /* EXODDF/ODDF/DISP/CDE */
+ RCAR_GP_PIN(4, 29),
+};
+static const unsigned int du1_oddf_mux[] = {
+ DU1_EXODDF_DU1_ODDF_DISP_CDE_MARK,
+};
+static const unsigned int du1_cde_pins[] = {
+ /* CDE */
+ RCAR_GP_PIN(4, 31),
+};
+static const unsigned int du1_cde_mux[] = {
+ DU1_CDE_MARK
+};
+static const unsigned int du1_disp_pins[] = {
+ /* DISP */
+ RCAR_GP_PIN(4, 30),
+};
+static const unsigned int du1_disp_mux[] = {
+ DU1_DISP_MARK
+};
/* - ETH -------------------------------------------------------------------- */
static const unsigned int eth_link_pins[] = {
/* LINK */
@@ -3364,6 +3537,24 @@
SH_PFC_PIN_GROUP(avb_avtp_match),
SH_PFC_PIN_GROUP(avb_avtp_capture_b),
SH_PFC_PIN_GROUP(avb_avtp_match_b),
+ SH_PFC_PIN_GROUP(du0_rgb666),
+ SH_PFC_PIN_GROUP(du0_rgb888),
+ SH_PFC_PIN_GROUP(du0_clk0_out),
+ SH_PFC_PIN_GROUP(du0_clk1_out),
+ SH_PFC_PIN_GROUP(du0_clk_in),
+ SH_PFC_PIN_GROUP(du0_sync),
+ SH_PFC_PIN_GROUP(du0_oddf),
+ SH_PFC_PIN_GROUP(du0_cde),
+ SH_PFC_PIN_GROUP(du0_disp),
+ SH_PFC_PIN_GROUP(du1_rgb666),
+ SH_PFC_PIN_GROUP(du1_rgb888),
+ SH_PFC_PIN_GROUP(du1_clk0_out),
+ SH_PFC_PIN_GROUP(du1_clk1_out),
+ SH_PFC_PIN_GROUP(du1_clk_in),
+ SH_PFC_PIN_GROUP(du1_sync),
+ SH_PFC_PIN_GROUP(du1_oddf),
+ SH_PFC_PIN_GROUP(du1_cde),
+ SH_PFC_PIN_GROUP(du1_disp),
SH_PFC_PIN_GROUP(eth_link),
SH_PFC_PIN_GROUP(eth_magic),
SH_PFC_PIN_GROUP(eth_mdio),
@@ -3622,6 +3813,30 @@
"avb_avtp_match_b",
};
+static const char * const du0_groups[] = {
+ "du0_rgb666",
+ "du0_rgb888",
+ "du0_clk0_out",
+ "du0_clk1_out",
+ "du0_clk_in",
+ "du0_sync",
+ "du0_oddf",
+ "du0_cde",
+ "du0_disp",
+};
+
+static const char * const du1_groups[] = {
+ "du1_rgb666",
+ "du1_rgb888",
+ "du1_clk0_out",
+ "du1_clk1_out",
+ "du1_clk_in",
+ "du1_sync",
+ "du1_oddf",
+ "du1_cde",
+ "du1_disp",
+};
+
static const char * const eth_groups[] = {
"eth_link",
"eth_magic",
@@ -3969,6 +4184,8 @@
static const struct sh_pfc_function pinmux_functions[] = {
SH_PFC_FUNCTION(audio_clk),
SH_PFC_FUNCTION(avb),
+ SH_PFC_FUNCTION(du0),
+ SH_PFC_FUNCTION(du1),
SH_PFC_FUNCTION(eth),
SH_PFC_FUNCTION(hscif0),
SH_PFC_FUNCTION(hscif1),
diff --git a/drivers/pinctrl/sh-pfc/pfc-r8a7795.c b/drivers/pinctrl/sh-pfc/pfc-r8a7795.c
index 5979dab..44632b1 100644
--- a/drivers/pinctrl/sh-pfc/pfc-r8a7795.c
+++ b/drivers/pinctrl/sh-pfc/pfc-r8a7795.c
@@ -14,14 +14,14 @@
#include "sh_pfc.h"
#define CPU_ALL_PORT(fn, sfx) \
- PORT_GP_16(0, fn, sfx), \
- PORT_GP_28(1, fn, sfx), \
- PORT_GP_15(2, fn, sfx), \
- PORT_GP_16(3, fn, sfx), \
- PORT_GP_18(4, fn, sfx), \
- PORT_GP_26(5, fn, sfx), \
- PORT_GP_32(6, fn, sfx), \
- PORT_GP_4(7, fn, sfx)
+ PORT_GP_CFG_16(0, fn, sfx, SH_PFC_PIN_CFG_DRIVE_STRENGTH), \
+ PORT_GP_CFG_28(1, fn, sfx, SH_PFC_PIN_CFG_DRIVE_STRENGTH), \
+ PORT_GP_CFG_15(2, fn, sfx, SH_PFC_PIN_CFG_DRIVE_STRENGTH), \
+ PORT_GP_CFG_16(3, fn, sfx, SH_PFC_PIN_CFG_DRIVE_STRENGTH), \
+ PORT_GP_CFG_18(4, fn, sfx, SH_PFC_PIN_CFG_DRIVE_STRENGTH), \
+ PORT_GP_CFG_26(5, fn, sfx, SH_PFC_PIN_CFG_DRIVE_STRENGTH), \
+ PORT_GP_CFG_32(6, fn, sfx, SH_PFC_PIN_CFG_DRIVE_STRENGTH), \
+ PORT_GP_CFG_4(7, fn, sfx, SH_PFC_PIN_CFG_DRIVE_STRENGTH)
/*
* F_() : just information
* FM() : macro for FN_xxx / xxx_MARK
@@ -4564,6 +4564,207 @@
{ },
};
+static const struct pinmux_drive_reg pinmux_drive_regs[] = {
+ { PINMUX_DRIVE_REG("DRVCTRL3", 0xe606030c) {
+ { RCAR_GP_PIN(2, 9), 8, 3 }, /* AVB_MDC */
+ { RCAR_GP_PIN(2, 10), 4, 3 }, /* AVB_MAGIC */
+ { RCAR_GP_PIN(2, 11), 0, 3 }, /* AVB_PHY_INT */
+ } },
+ { PINMUX_DRIVE_REG("DRVCTRL4", 0xe6060310) {
+ { RCAR_GP_PIN(2, 12), 28, 3 }, /* AVB_LINK */
+ { RCAR_GP_PIN(2, 13), 24, 3 }, /* AVB_AVTP_MATCH */
+ { RCAR_GP_PIN(2, 14), 20, 3 }, /* AVB_AVTP_CAPTURE */
+ { RCAR_GP_PIN(2, 0), 16, 3 }, /* IRQ0 */
+ { RCAR_GP_PIN(2, 1), 12, 3 }, /* IRQ1 */
+ { RCAR_GP_PIN(2, 2), 8, 3 }, /* IRQ2 */
+ { RCAR_GP_PIN(2, 3), 4, 3 }, /* IRQ3 */
+ { RCAR_GP_PIN(2, 4), 0, 3 }, /* IRQ4 */
+ } },
+ { PINMUX_DRIVE_REG("DRVCTRL5", 0xe6060314) {
+ { RCAR_GP_PIN(2, 5), 28, 3 }, /* IRQ5 */
+ { RCAR_GP_PIN(2, 6), 24, 3 }, /* PWM0 */
+ { RCAR_GP_PIN(2, 7), 20, 3 }, /* PWM1 */
+ { RCAR_GP_PIN(2, 8), 16, 3 }, /* PWM2 */
+ { RCAR_GP_PIN(1, 0), 12, 3 }, /* A0 */
+ { RCAR_GP_PIN(1, 1), 8, 3 }, /* A1 */
+ { RCAR_GP_PIN(1, 2), 4, 3 }, /* A2 */
+ { RCAR_GP_PIN(1, 3), 0, 3 }, /* A3 */
+ } },
+ { PINMUX_DRIVE_REG("DRVCTRL6", 0xe6060318) {
+ { RCAR_GP_PIN(1, 4), 28, 3 }, /* A4 */
+ { RCAR_GP_PIN(1, 5), 24, 3 }, /* A5 */
+ { RCAR_GP_PIN(1, 6), 20, 3 }, /* A6 */
+ { RCAR_GP_PIN(1, 7), 16, 3 }, /* A7 */
+ { RCAR_GP_PIN(1, 8), 12, 3 }, /* A8 */
+ { RCAR_GP_PIN(1, 9), 8, 3 }, /* A9 */
+ { RCAR_GP_PIN(1, 10), 4, 3 }, /* A10 */
+ { RCAR_GP_PIN(1, 11), 0, 3 }, /* A11 */
+ } },
+ { PINMUX_DRIVE_REG("DRVCTRL7", 0xe606031c) {
+ { RCAR_GP_PIN(1, 12), 28, 3 }, /* A12 */
+ { RCAR_GP_PIN(1, 13), 24, 3 }, /* A13 */
+ { RCAR_GP_PIN(1, 14), 20, 3 }, /* A14 */
+ { RCAR_GP_PIN(1, 15), 16, 3 }, /* A15 */
+ { RCAR_GP_PIN(1, 16), 12, 3 }, /* A16 */
+ { RCAR_GP_PIN(1, 17), 8, 3 }, /* A17 */
+ { RCAR_GP_PIN(1, 18), 4, 3 }, /* A18 */
+ { RCAR_GP_PIN(1, 19), 0, 3 }, /* A19 */
+ } },
+ { PINMUX_DRIVE_REG("DRVCTRL8", 0xe6060320) {
+ { RCAR_GP_PIN(1, 20), 24, 3 }, /* CS0 */
+ { RCAR_GP_PIN(1, 21), 20, 3 }, /* CS1_A26 */
+ { RCAR_GP_PIN(1, 22), 16, 3 }, /* BS */
+ { RCAR_GP_PIN(1, 23), 12, 3 }, /* RD */
+ { RCAR_GP_PIN(1, 24), 8, 3 }, /* RD_WR */
+ { RCAR_GP_PIN(1, 25), 4, 3 }, /* WE0 */
+ { RCAR_GP_PIN(1, 26), 0, 3 }, /* WE1 */
+ } },
+ { PINMUX_DRIVE_REG("DRVCTRL9", 0xe6060324) {
+ { RCAR_GP_PIN(1, 27), 28, 3 }, /* EX_WAIT0 */
+ { RCAR_GP_PIN(0, 0), 20, 3 }, /* D0 */
+ { RCAR_GP_PIN(0, 1), 16, 3 }, /* D1 */
+ { RCAR_GP_PIN(0, 2), 12, 3 }, /* D2 */
+ { RCAR_GP_PIN(0, 3), 8, 3 }, /* D3 */
+ { RCAR_GP_PIN(0, 4), 4, 3 }, /* D4 */
+ { RCAR_GP_PIN(0, 5), 0, 3 }, /* D5 */
+ } },
+ { PINMUX_DRIVE_REG("DRVCTRL10", 0xe6060328) {
+ { RCAR_GP_PIN(0, 6), 28, 3 }, /* D6 */
+ { RCAR_GP_PIN(0, 7), 24, 3 }, /* D7 */
+ { RCAR_GP_PIN(0, 8), 20, 3 }, /* D8 */
+ { RCAR_GP_PIN(0, 9), 16, 3 }, /* D9 */
+ { RCAR_GP_PIN(0, 10), 12, 3 }, /* D10 */
+ { RCAR_GP_PIN(0, 11), 8, 3 }, /* D11 */
+ { RCAR_GP_PIN(0, 12), 4, 3 }, /* D12 */
+ { RCAR_GP_PIN(0, 13), 0, 3 }, /* D13 */
+ } },
+ { PINMUX_DRIVE_REG("DRVCTRL11", 0xe606032c) {
+ { RCAR_GP_PIN(0, 14), 28, 3 }, /* D14 */
+ { RCAR_GP_PIN(0, 15), 24, 3 }, /* D15 */
+ { RCAR_GP_PIN(7, 0), 20, 3 }, /* AVS1 */
+ { RCAR_GP_PIN(7, 1), 16, 3 }, /* AVS2 */
+ { RCAR_GP_PIN(7, 2), 12, 3 }, /* HDMI0_CEC */
+ { RCAR_GP_PIN(7, 3), 8, 3 }, /* HDMI1_CEC */
+ } },
+ { PINMUX_DRIVE_REG("DRVCTRL13", 0xe6060334) {
+ { RCAR_GP_PIN(3, 0), 20, 3 }, /* SD0_CLK */
+ { RCAR_GP_PIN(3, 1), 16, 3 }, /* SD0_CMD */
+ { RCAR_GP_PIN(3, 2), 12, 3 }, /* SD0_DAT0 */
+ { RCAR_GP_PIN(3, 3), 8, 3 }, /* SD0_DAT1 */
+ { RCAR_GP_PIN(3, 4), 4, 3 }, /* SD0_DAT2 */
+ { RCAR_GP_PIN(3, 5), 0, 3 }, /* SD0_DAT3 */
+ } },
+ { PINMUX_DRIVE_REG("DRVCTRL14", 0xe6060338) {
+ { RCAR_GP_PIN(3, 6), 28, 3 }, /* SD1_CLK */
+ { RCAR_GP_PIN(3, 7), 24, 3 }, /* SD1_CMD */
+ { RCAR_GP_PIN(3, 8), 20, 3 }, /* SD1_DAT0 */
+ { RCAR_GP_PIN(3, 9), 16, 3 }, /* SD1_DAT1 */
+ { RCAR_GP_PIN(3, 10), 12, 3 }, /* SD1_DAT2 */
+ { RCAR_GP_PIN(3, 11), 8, 3 }, /* SD1_DAT3 */
+ { RCAR_GP_PIN(4, 0), 4, 3 }, /* SD2_CLK */
+ { RCAR_GP_PIN(4, 1), 0, 3 }, /* SD2_CMD */
+ } },
+ { PINMUX_DRIVE_REG("DRVCTRL15", 0xe606033c) {
+ { RCAR_GP_PIN(4, 2), 28, 3 }, /* SD2_DAT0 */
+ { RCAR_GP_PIN(4, 3), 24, 3 }, /* SD2_DAT1 */
+ { RCAR_GP_PIN(4, 4), 20, 3 }, /* SD2_DAT2 */
+ { RCAR_GP_PIN(4, 5), 16, 3 }, /* SD2_DAT3 */
+ { RCAR_GP_PIN(4, 6), 12, 3 }, /* SD2_DS */
+ { RCAR_GP_PIN(4, 7), 8, 3 }, /* SD3_CLK */
+ { RCAR_GP_PIN(4, 8), 4, 3 }, /* SD3_CMD */
+ { RCAR_GP_PIN(4, 9), 0, 3 }, /* SD3_DAT0 */
+ } },
+ { PINMUX_DRIVE_REG("DRVCTRL16", 0xe6060340) {
+ { RCAR_GP_PIN(4, 10), 28, 3 }, /* SD3_DAT1 */
+ { RCAR_GP_PIN(4, 11), 24, 3 }, /* SD3_DAT2 */
+ { RCAR_GP_PIN(4, 12), 20, 3 }, /* SD3_DAT3 */
+ { RCAR_GP_PIN(4, 13), 16, 3 }, /* SD3_DAT4 */
+ { RCAR_GP_PIN(4, 14), 12, 3 }, /* SD3_DAT5 */
+ { RCAR_GP_PIN(4, 15), 8, 3 }, /* SD3_DAT6 */
+ { RCAR_GP_PIN(4, 16), 4, 3 }, /* SD3_DAT7 */
+ { RCAR_GP_PIN(4, 17), 0, 3 }, /* SD3_DS */
+ } },
+ { PINMUX_DRIVE_REG("DRVCTRL17", 0xe6060344) {
+ { RCAR_GP_PIN(3, 12), 28, 3 }, /* SD0_CD */
+ { RCAR_GP_PIN(3, 13), 24, 3 }, /* SD0_WP */
+ { RCAR_GP_PIN(3, 14), 20, 3 }, /* SD1_CD */
+ { RCAR_GP_PIN(3, 15), 16, 3 }, /* SD1_WP */
+ { RCAR_GP_PIN(5, 0), 12, 3 }, /* SCK0 */
+ { RCAR_GP_PIN(5, 1), 8, 3 }, /* RX0 */
+ { RCAR_GP_PIN(5, 2), 4, 3 }, /* TX0 */
+ { RCAR_GP_PIN(5, 3), 0, 3 }, /* CTS0 */
+ } },
+ { PINMUX_DRIVE_REG("DRVCTRL18", 0xe6060348) {
+ { RCAR_GP_PIN(5, 4), 28, 3 }, /* RTS0_TANS */
+ { RCAR_GP_PIN(5, 5), 24, 3 }, /* RX1 */
+ { RCAR_GP_PIN(5, 6), 20, 3 }, /* TX1 */
+ { RCAR_GP_PIN(5, 7), 16, 3 }, /* CTS1 */
+ { RCAR_GP_PIN(5, 8), 12, 3 }, /* RTS1_TANS */
+ { RCAR_GP_PIN(5, 9), 8, 3 }, /* SCK2 */
+ { RCAR_GP_PIN(5, 10), 4, 3 }, /* TX2 */
+ { RCAR_GP_PIN(5, 11), 0, 3 }, /* RX2 */
+ } },
+ { PINMUX_DRIVE_REG("DRVCTRL19", 0xe606034c) {
+ { RCAR_GP_PIN(5, 12), 28, 3 }, /* HSCK0 */
+ { RCAR_GP_PIN(5, 13), 24, 3 }, /* HRX0 */
+ { RCAR_GP_PIN(5, 14), 20, 3 }, /* HTX0 */
+ { RCAR_GP_PIN(5, 15), 16, 3 }, /* HCTS0 */
+ { RCAR_GP_PIN(5, 16), 12, 3 }, /* HRTS0 */
+ { RCAR_GP_PIN(5, 17), 8, 3 }, /* MSIOF0_SCK */
+ { RCAR_GP_PIN(5, 18), 4, 3 }, /* MSIOF0_SYNC */
+ { RCAR_GP_PIN(5, 19), 0, 3 }, /* MSIOF0_SS1 */
+ } },
+ { PINMUX_DRIVE_REG("DRVCTRL20", 0xe6060350) {
+ { RCAR_GP_PIN(5, 20), 28, 3 }, /* MSIOF0_TXD */
+ { RCAR_GP_PIN(5, 21), 24, 3 }, /* MSIOF0_SS2 */
+ { RCAR_GP_PIN(5, 22), 20, 3 }, /* MSIOF0_RXD */
+ { RCAR_GP_PIN(5, 23), 16, 3 }, /* MLB_CLK */
+ { RCAR_GP_PIN(5, 24), 12, 3 }, /* MLB_SIG */
+ { RCAR_GP_PIN(5, 25), 8, 3 }, /* MLB_DAT */
+ { RCAR_GP_PIN(6, 0), 0, 3 }, /* SSI_SCK01239 */
+ } },
+ { PINMUX_DRIVE_REG("DRVCTRL21", 0xe6060354) {
+ { RCAR_GP_PIN(6, 1), 28, 3 }, /* SSI_WS01239 */
+ { RCAR_GP_PIN(6, 2), 24, 3 }, /* SSI_SDATA0 */
+ { RCAR_GP_PIN(6, 3), 20, 3 }, /* SSI_SDATA1 */
+ { RCAR_GP_PIN(6, 4), 16, 3 }, /* SSI_SDATA2 */
+ { RCAR_GP_PIN(6, 5), 12, 3 }, /* SSI_SCK34 */
+ { RCAR_GP_PIN(6, 6), 8, 3 }, /* SSI_WS34 */
+ { RCAR_GP_PIN(6, 7), 4, 3 }, /* SSI_SDATA3 */
+ { RCAR_GP_PIN(6, 8), 0, 3 }, /* SSI_SCK4 */
+ } },
+ { PINMUX_DRIVE_REG("DRVCTRL22", 0xe6060358) {
+ { RCAR_GP_PIN(6, 9), 28, 3 }, /* SSI_WS4 */
+ { RCAR_GP_PIN(6, 10), 24, 3 }, /* SSI_SDATA4 */
+ { RCAR_GP_PIN(6, 11), 20, 3 }, /* SSI_SCK5 */
+ { RCAR_GP_PIN(6, 12), 16, 3 }, /* SSI_WS5 */
+ { RCAR_GP_PIN(6, 13), 12, 3 }, /* SSI_SDATA5 */
+ { RCAR_GP_PIN(6, 14), 8, 3 }, /* SSI_SCK6 */
+ { RCAR_GP_PIN(6, 15), 4, 3 }, /* SSI_WS6 */
+ { RCAR_GP_PIN(6, 16), 0, 3 }, /* SSI_SDATA6 */
+ } },
+ { PINMUX_DRIVE_REG("DRVCTRL23", 0xe606035c) {
+ { RCAR_GP_PIN(6, 17), 28, 3 }, /* SSI_SCK78 */
+ { RCAR_GP_PIN(6, 18), 24, 3 }, /* SSI_WS78 */
+ { RCAR_GP_PIN(6, 19), 20, 3 }, /* SSI_SDATA7 */
+ { RCAR_GP_PIN(6, 20), 16, 3 }, /* SSI_SDATA8 */
+ { RCAR_GP_PIN(6, 21), 12, 3 }, /* SSI_SDATA9 */
+ { RCAR_GP_PIN(6, 22), 8, 3 }, /* AUDIO_CLKA */
+ { RCAR_GP_PIN(6, 23), 4, 3 }, /* AUDIO_CLKB */
+ { RCAR_GP_PIN(6, 24), 0, 3 }, /* USB0_PWEN */
+ } },
+ { PINMUX_DRIVE_REG("DRVCTRL24", 0xe6060360) {
+ { RCAR_GP_PIN(6, 25), 28, 3 }, /* USB0_OVC */
+ { RCAR_GP_PIN(6, 26), 24, 3 }, /* USB1_PWEN */
+ { RCAR_GP_PIN(6, 27), 20, 3 }, /* USB1_OVC */
+ { RCAR_GP_PIN(6, 28), 16, 3 }, /* USB30_PWEN */
+ { RCAR_GP_PIN(6, 29), 12, 3 }, /* USB30_OVC */
+ { RCAR_GP_PIN(6, 30), 8, 3 }, /* USB31_PWEN */
+ { RCAR_GP_PIN(6, 31), 4, 3 }, /* USB31_OVC */
+ } },
+ { },
+};
+
const struct sh_pfc_soc_info r8a7795_pinmux_info = {
.name = "r8a77950_pfc",
.unlock_reg = 0xe6060000, /* PMMR */
@@ -4578,6 +4779,7 @@
.nr_functions = ARRAY_SIZE(pinmux_functions),
.cfg_regs = pinmux_config_regs,
+ .drive_regs = pinmux_drive_regs,
.pinmux_data = pinmux_data,
.pinmux_data_size = ARRAY_SIZE(pinmux_data),
diff --git a/drivers/pinctrl/sh-pfc/pinctrl.c b/drivers/pinctrl/sh-pfc/pinctrl.c
index 87b0a59..fdb445d 100644
--- a/drivers/pinctrl/sh-pfc/pinctrl.c
+++ b/drivers/pinctrl/sh-pfc/pinctrl.c
@@ -476,6 +476,91 @@
.gpio_set_direction = sh_pfc_gpio_set_direction,
};
+static u32 sh_pfc_pinconf_find_drive_strength_reg(struct sh_pfc *pfc,
+ unsigned int pin, unsigned int *offset, unsigned int *size)
+{
+ const struct pinmux_drive_reg_field *field;
+ const struct pinmux_drive_reg *reg;
+ unsigned int i;
+
+ for (reg = pfc->info->drive_regs; reg->reg; ++reg) {
+ for (i = 0; i < ARRAY_SIZE(reg->fields); ++i) {
+ field = ®->fields[i];
+
+ if (field->size && field->pin == pin) {
+ *offset = field->offset;
+ *size = field->size;
+
+ return reg->reg;
+ }
+ }
+ }
+
+ return 0;
+}
+
+static int sh_pfc_pinconf_get_drive_strength(struct sh_pfc *pfc,
+ unsigned int pin)
+{
+ unsigned long flags;
+ unsigned int offset;
+ unsigned int size;
+ u32 reg;
+ u32 val;
+
+ reg = sh_pfc_pinconf_find_drive_strength_reg(pfc, pin, &offset, &size);
+ if (!reg)
+ return -EINVAL;
+
+ spin_lock_irqsave(&pfc->lock, flags);
+ val = sh_pfc_read_reg(pfc, reg, 32);
+ spin_unlock_irqrestore(&pfc->lock, flags);
+
+ val = (val >> offset) & GENMASK(size - 1, 0);
+
+ /* Convert the value to mA based on a full drive strength value of 24mA.
+ * We can make the full value configurable later if needed.
+ */
+ return (val + 1) * (size == 2 ? 6 : 3);
+}
+
+static int sh_pfc_pinconf_set_drive_strength(struct sh_pfc *pfc,
+ unsigned int pin, u16 strength)
+{
+ unsigned long flags;
+ unsigned int offset;
+ unsigned int size;
+ unsigned int step;
+ u32 reg;
+ u32 val;
+
+ reg = sh_pfc_pinconf_find_drive_strength_reg(pfc, pin, &offset, &size);
+ if (!reg)
+ return -EINVAL;
+
+ step = size == 2 ? 6 : 3;
+
+ if (strength < step || strength > 24)
+ return -EINVAL;
+
+ /* Convert the value from mA based on a full drive strength value of
+ * 24mA. We can make the full value configurable later if needed.
+ */
+ strength = strength / step - 1;
+
+ spin_lock_irqsave(&pfc->lock, flags);
+
+ val = sh_pfc_read_reg(pfc, reg, 32);
+ val &= ~GENMASK(offset + size - 1, offset);
+ val |= strength << offset;
+
+ sh_pfc_write_reg(pfc, reg, 32, val);
+
+ spin_unlock_irqrestore(&pfc->lock, flags);
+
+ return 0;
+}
+
/* Check whether the requested parameter is supported for a pin. */
static bool sh_pfc_pinconf_validate(struct sh_pfc *pfc, unsigned int _pin,
enum pin_config_param param)
@@ -493,6 +578,9 @@
case PIN_CONFIG_BIAS_PULL_DOWN:
return pin->configs & SH_PFC_PIN_CFG_PULL_DOWN;
+ case PIN_CONFIG_DRIVE_STRENGTH:
+ return pin->configs & SH_PFC_PIN_CFG_DRIVE_STRENGTH;
+
case PIN_CONFIG_POWER_SOURCE:
return pin->configs & SH_PFC_PIN_CFG_IO_VOLTAGE;
@@ -532,6 +620,17 @@
break;
}
+ case PIN_CONFIG_DRIVE_STRENGTH: {
+ int ret;
+
+ ret = sh_pfc_pinconf_get_drive_strength(pfc, _pin);
+ if (ret < 0)
+ return ret;
+
+ *config = ret;
+ break;
+ }
+
case PIN_CONFIG_POWER_SOURCE: {
int ret;
@@ -584,6 +683,18 @@
break;
+ case PIN_CONFIG_DRIVE_STRENGTH: {
+ unsigned int arg =
+ pinconf_to_config_argument(configs[i]);
+ int ret;
+
+ ret = sh_pfc_pinconf_set_drive_strength(pfc, _pin, arg);
+ if (ret < 0)
+ return ret;
+
+ break;
+ }
+
case PIN_CONFIG_POWER_SOURCE: {
unsigned int arg =
pinconf_to_config_argument(configs[i]);
@@ -678,7 +789,6 @@
return -ENOMEM;
pmx->pfc = pfc;
- pfc->pinctrl = pmx;
ret = sh_pfc_map_pins(pfc, pmx);
if (ret < 0)
@@ -692,19 +802,9 @@
pmx->pctl_desc.pins = pmx->pins;
pmx->pctl_desc.npins = pfc->info->nr_pins;
- pmx->pctl = pinctrl_register(&pmx->pctl_desc, pfc->dev, pmx);
+ pmx->pctl = devm_pinctrl_register(pfc->dev, &pmx->pctl_desc, pmx);
if (IS_ERR(pmx->pctl))
return PTR_ERR(pmx->pctl);
return 0;
}
-
-int sh_pfc_unregister_pinctrl(struct sh_pfc *pfc)
-{
- struct sh_pfc_pinctrl *pmx = pfc->pinctrl;
-
- pinctrl_unregister(pmx->pctl);
-
- pfc->pinctrl = NULL;
- return 0;
-}
diff --git a/drivers/pinctrl/sh-pfc/sh_pfc.h b/drivers/pinctrl/sh-pfc/sh_pfc.h
index a490834..656ea32 100644
--- a/drivers/pinctrl/sh-pfc/sh_pfc.h
+++ b/drivers/pinctrl/sh-pfc/sh_pfc.h
@@ -28,6 +28,7 @@
#define SH_PFC_PIN_CFG_PULL_UP (1 << 2)
#define SH_PFC_PIN_CFG_PULL_DOWN (1 << 3)
#define SH_PFC_PIN_CFG_IO_VOLTAGE (1 << 4)
+#define SH_PFC_PIN_CFG_DRIVE_STRENGTH (1 << 5)
#define SH_PFC_PIN_CFG_NO_GPIO (1 << 31)
struct sh_pfc_pin {
@@ -131,6 +132,21 @@
{ var_fw0, var_fwn, 0 }, \
.enum_ids = (const u16 [])
+struct pinmux_drive_reg_field {
+ u16 pin;
+ u8 offset;
+ u8 size;
+};
+
+struct pinmux_drive_reg {
+ u32 reg;
+ const struct pinmux_drive_reg_field fields[8];
+};
+
+#define PINMUX_DRIVE_REG(name, r) \
+ .reg = r, \
+ .fields =
+
struct pinmux_data_reg {
u32 reg;
u8 reg_width;
@@ -199,6 +215,7 @@
#endif
const struct pinmux_cfg_reg *cfg_regs;
+ const struct pinmux_drive_reg *drive_regs;
const struct pinmux_data_reg *data_regs;
const u16 *pinmux_data;
@@ -276,7 +293,7 @@
* - msel: Module selector
*/
#define PINMUX_IPSR_MSEL(ipsr, fn, msel) \
- PINMUX_DATA(fn##_MARK, FN_##msel, FN_##ipsr, FN_##fn)
+ PINMUX_DATA(fn##_MARK, FN_##msel, FN_##fn, FN_##ipsr)
/*
* Describe a pinmux configuration for a single-function pin with GPIO
diff --git a/drivers/pinctrl/spear/pinctrl-spear.c b/drivers/pinctrl/spear/pinctrl-spear.c
index 0afaf79..4db52ba 100644
--- a/drivers/pinctrl/spear/pinctrl-spear.c
+++ b/drivers/pinctrl/spear/pinctrl-spear.c
@@ -395,7 +395,7 @@
spear_pinctrl_desc.pins = machdata->pins;
spear_pinctrl_desc.npins = machdata->npins;
- pmx->pctl = pinctrl_register(&spear_pinctrl_desc, &pdev->dev, pmx);
+ pmx->pctl = devm_pinctrl_register(&pdev->dev, &spear_pinctrl_desc, pmx);
if (IS_ERR(pmx->pctl)) {
dev_err(&pdev->dev, "Couldn't register pinctrl driver\n");
return PTR_ERR(pmx->pctl);
@@ -403,12 +403,3 @@
return 0;
}
-
-int spear_pinctrl_remove(struct platform_device *pdev)
-{
- struct spear_pmx *pmx = platform_get_drvdata(pdev);
-
- pinctrl_unregister(pmx->pctl);
-
- return 0;
-}
diff --git a/drivers/pinctrl/spear/pinctrl-spear.h b/drivers/pinctrl/spear/pinctrl-spear.h
index 27c2cc8..aa5cf70 100644
--- a/drivers/pinctrl/spear/pinctrl-spear.h
+++ b/drivers/pinctrl/spear/pinctrl-spear.h
@@ -197,7 +197,6 @@
unsigned count, u16 reg);
int spear_pinctrl_probe(struct platform_device *pdev,
struct spear_pinctrl_machdata *machdata);
-int spear_pinctrl_remove(struct platform_device *pdev);
#define SPEAR_PIN_0_TO_101 \
PINCTRL_PIN(0, "PLGPIO0"), \
diff --git a/drivers/pinctrl/spear/pinctrl-spear1310.c b/drivers/pinctrl/spear/pinctrl-spear1310.c
index 92611bb..1821068 100644
--- a/drivers/pinctrl/spear/pinctrl-spear1310.c
+++ b/drivers/pinctrl/spear/pinctrl-spear1310.c
@@ -2704,18 +2704,12 @@
return spear_pinctrl_probe(pdev, &spear1310_machdata);
}
-static int spear1310_pinctrl_remove(struct platform_device *pdev)
-{
- return spear_pinctrl_remove(pdev);
-}
-
static struct platform_driver spear1310_pinctrl_driver = {
.driver = {
.name = DRIVER_NAME,
.of_match_table = spear1310_pinctrl_of_match,
},
.probe = spear1310_pinctrl_probe,
- .remove = spear1310_pinctrl_remove,
};
static int __init spear1310_pinctrl_init(void)
diff --git a/drivers/pinctrl/spear/pinctrl-spear1340.c b/drivers/pinctrl/spear/pinctrl-spear1340.c
index f842e9d..c01fb23 100644
--- a/drivers/pinctrl/spear/pinctrl-spear1340.c
+++ b/drivers/pinctrl/spear/pinctrl-spear1340.c
@@ -2020,18 +2020,12 @@
return spear_pinctrl_probe(pdev, &spear1340_machdata);
}
-static int spear1340_pinctrl_remove(struct platform_device *pdev)
-{
- return spear_pinctrl_remove(pdev);
-}
-
static struct platform_driver spear1340_pinctrl_driver = {
.driver = {
.name = DRIVER_NAME,
.of_match_table = spear1340_pinctrl_of_match,
},
.probe = spear1340_pinctrl_probe,
- .remove = spear1340_pinctrl_remove,
};
static int __init spear1340_pinctrl_init(void)
diff --git a/drivers/pinctrl/spear/pinctrl-spear300.c b/drivers/pinctrl/spear/pinctrl-spear300.c
index d998a2c..111148d 100644
--- a/drivers/pinctrl/spear/pinctrl-spear300.c
+++ b/drivers/pinctrl/spear/pinctrl-spear300.c
@@ -677,18 +677,12 @@
return 0;
}
-static int spear300_pinctrl_remove(struct platform_device *pdev)
-{
- return spear_pinctrl_remove(pdev);
-}
-
static struct platform_driver spear300_pinctrl_driver = {
.driver = {
.name = DRIVER_NAME,
.of_match_table = spear300_pinctrl_of_match,
},
.probe = spear300_pinctrl_probe,
- .remove = spear300_pinctrl_remove,
};
static int __init spear300_pinctrl_init(void)
diff --git a/drivers/pinctrl/spear/pinctrl-spear310.c b/drivers/pinctrl/spear/pinctrl-spear310.c
index 609b18a..a7b0000 100644
--- a/drivers/pinctrl/spear/pinctrl-spear310.c
+++ b/drivers/pinctrl/spear/pinctrl-spear310.c
@@ -400,18 +400,12 @@
return 0;
}
-static int spear310_pinctrl_remove(struct platform_device *pdev)
-{
- return spear_pinctrl_remove(pdev);
-}
-
static struct platform_driver spear310_pinctrl_driver = {
.driver = {
.name = DRIVER_NAME,
.of_match_table = spear310_pinctrl_of_match,
},
.probe = spear310_pinctrl_probe,
- .remove = spear310_pinctrl_remove,
};
static int __init spear310_pinctrl_init(void)
diff --git a/drivers/pinctrl/spear/pinctrl-spear320.c b/drivers/pinctrl/spear/pinctrl-spear320.c
index c071144..e2b3817 100644
--- a/drivers/pinctrl/spear/pinctrl-spear320.c
+++ b/drivers/pinctrl/spear/pinctrl-spear320.c
@@ -3441,18 +3441,12 @@
return 0;
}
-static int spear320_pinctrl_remove(struct platform_device *pdev)
-{
- return spear_pinctrl_remove(pdev);
-}
-
static struct platform_driver spear320_pinctrl_driver = {
.driver = {
.name = DRIVER_NAME,
.of_match_table = spear320_pinctrl_of_match,
},
.probe = spear320_pinctrl_probe,
- .remove = spear320_pinctrl_remove,
};
static int __init spear320_pinctrl_init(void)
diff --git a/drivers/pinctrl/stm32/pinctrl-stm32.c b/drivers/pinctrl/stm32/pinctrl-stm32.c
index 8deb566..ae9fab8 100644
--- a/drivers/pinctrl/stm32/pinctrl-stm32.c
+++ b/drivers/pinctrl/stm32/pinctrl-stm32.c
@@ -358,7 +358,7 @@
ret = stm32_pctrl_dt_subnode_to_map(pctldev, np, map,
&reserved_maps, num_maps);
if (ret < 0) {
- pinctrl_utils_dt_free_map(pctldev, *map, *num_maps);
+ pinctrl_utils_free_map(pctldev, *map, *num_maps);
return ret;
}
}
@@ -396,7 +396,7 @@
static const struct pinctrl_ops stm32_pctrl_ops = {
.dt_node_to_map = stm32_pctrl_dt_node_to_map,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
.get_groups_count = stm32_pctrl_get_groups_count,
.get_group_name = stm32_pctrl_get_group_name,
.get_group_pins = stm32_pctrl_get_group_pins,
@@ -454,6 +454,29 @@
clk_disable(bank->clk);
}
+static void stm32_pmx_get_mode(struct stm32_gpio_bank *bank,
+ int pin, u32 *mode, u32 *alt)
+{
+ u32 val;
+ int alt_shift = (pin % 8) * 4;
+ int alt_offset = STM32_GPIO_AFRL + (pin / 8) * 4;
+ unsigned long flags;
+
+ clk_enable(bank->clk);
+ spin_lock_irqsave(&bank->lock, flags);
+
+ val = readl_relaxed(bank->base + alt_offset);
+ val &= GENMASK(alt_shift + 3, alt_shift);
+ *alt = val >> alt_shift;
+
+ val = readl_relaxed(bank->base + STM32_GPIO_MODER);
+ val &= GENMASK(pin * 2 + 1, pin * 2);
+ *mode = val >> (pin * 2);
+
+ spin_unlock_irqrestore(&bank->lock, flags);
+ clk_disable(bank->clk);
+}
+
static int stm32_pmx_set_mux(struct pinctrl_dev *pctldev,
unsigned function,
unsigned group)
@@ -525,6 +548,24 @@
clk_disable(bank->clk);
}
+static u32 stm32_pconf_get_driving(struct stm32_gpio_bank *bank,
+ unsigned int offset)
+{
+ unsigned long flags;
+ u32 val;
+
+ clk_enable(bank->clk);
+ spin_lock_irqsave(&bank->lock, flags);
+
+ val = readl_relaxed(bank->base + STM32_GPIO_TYPER);
+ val &= BIT(offset);
+
+ spin_unlock_irqrestore(&bank->lock, flags);
+ clk_disable(bank->clk);
+
+ return (val >> offset);
+}
+
static void stm32_pconf_set_speed(struct stm32_gpio_bank *bank,
unsigned offset, u32 speed)
{
@@ -543,6 +584,24 @@
clk_disable(bank->clk);
}
+static u32 stm32_pconf_get_speed(struct stm32_gpio_bank *bank,
+ unsigned int offset)
+{
+ unsigned long flags;
+ u32 val;
+
+ clk_enable(bank->clk);
+ spin_lock_irqsave(&bank->lock, flags);
+
+ val = readl_relaxed(bank->base + STM32_GPIO_SPEEDR);
+ val &= GENMASK(offset * 2 + 1, offset * 2);
+
+ spin_unlock_irqrestore(&bank->lock, flags);
+ clk_disable(bank->clk);
+
+ return (val >> (offset * 2));
+}
+
static void stm32_pconf_set_bias(struct stm32_gpio_bank *bank,
unsigned offset, u32 bias)
{
@@ -561,6 +620,57 @@
clk_disable(bank->clk);
}
+static u32 stm32_pconf_get_bias(struct stm32_gpio_bank *bank,
+ unsigned int offset)
+{
+ unsigned long flags;
+ u32 val;
+
+ clk_enable(bank->clk);
+ spin_lock_irqsave(&bank->lock, flags);
+
+ val = readl_relaxed(bank->base + STM32_GPIO_PUPDR);
+ val &= GENMASK(offset * 2 + 1, offset * 2);
+
+ spin_unlock_irqrestore(&bank->lock, flags);
+ clk_disable(bank->clk);
+
+ return (val >> (offset * 2));
+}
+
+static bool stm32_pconf_input_get(struct stm32_gpio_bank *bank,
+ unsigned int offset)
+{
+ unsigned long flags;
+ u32 val;
+
+ clk_enable(bank->clk);
+ spin_lock_irqsave(&bank->lock, flags);
+
+ val = !!(readl_relaxed(bank->base + STM32_GPIO_IDR) & BIT(offset));
+
+ spin_unlock_irqrestore(&bank->lock, flags);
+ clk_disable(bank->clk);
+
+ return val;
+}
+
+static bool stm32_pconf_output_get(struct stm32_gpio_bank *bank,
+ unsigned int offset)
+{
+ unsigned long flags;
+ u32 val;
+
+ clk_enable(bank->clk);
+ spin_lock_irqsave(&bank->lock, flags);
+ val = !!(readl_relaxed(bank->base + STM32_GPIO_ODR) & BIT(offset));
+
+ spin_unlock_irqrestore(&bank->lock, flags);
+ clk_disable(bank->clk);
+
+ return val;
+}
+
static int stm32_pconf_parse_conf(struct pinctrl_dev *pctldev,
unsigned int pin, enum pin_config_param param,
enum pin_config_param arg)
@@ -634,9 +744,73 @@
return 0;
}
+static void stm32_pconf_dbg_show(struct pinctrl_dev *pctldev,
+ struct seq_file *s,
+ unsigned int pin)
+{
+ struct pinctrl_gpio_range *range;
+ struct stm32_gpio_bank *bank;
+ int offset;
+ u32 mode, alt, drive, speed, bias;
+ static const char * const modes[] = {
+ "input", "output", "alternate", "analog" };
+ static const char * const speeds[] = {
+ "low", "medium", "high", "very high" };
+ static const char * const biasing[] = {
+ "floating", "pull up", "pull down", "" };
+ bool val;
+
+ range = pinctrl_find_gpio_range_from_pin_nolock(pctldev, pin);
+ bank = gpio_range_to_bank(range);
+ offset = stm32_gpio_pin(pin);
+
+ stm32_pmx_get_mode(bank, offset, &mode, &alt);
+ bias = stm32_pconf_get_bias(bank, offset);
+
+ seq_printf(s, "%s ", modes[mode]);
+
+ switch (mode) {
+ /* input */
+ case 0:
+ val = stm32_pconf_input_get(bank, offset);
+ seq_printf(s, "- %s - %s",
+ val ? "high" : "low",
+ biasing[bias]);
+ break;
+
+ /* output */
+ case 1:
+ drive = stm32_pconf_get_driving(bank, offset);
+ speed = stm32_pconf_get_speed(bank, offset);
+ val = stm32_pconf_output_get(bank, offset);
+ seq_printf(s, "- %s - %s - %s - %s %s",
+ val ? "high" : "low",
+ drive ? "open drain" : "push pull",
+ biasing[bias],
+ speeds[speed], "speed");
+ break;
+
+ /* alternate */
+ case 2:
+ drive = stm32_pconf_get_driving(bank, offset);
+ speed = stm32_pconf_get_speed(bank, offset);
+ seq_printf(s, "%d - %s - %s - %s %s", alt,
+ drive ? "open drain" : "push pull",
+ biasing[bias],
+ speeds[speed], "speed");
+ break;
+
+ /* analog */
+ case 3:
+ break;
+ }
+}
+
+
static const struct pinconf_ops stm32_pconf_ops = {
.pin_config_group_get = stm32_pconf_group_get,
.pin_config_group_set = stm32_pconf_group_set,
+ .pin_config_dbg_show = stm32_pconf_dbg_show,
};
static int stm32_gpiolib_register_bank(struct stm32_pinctrl *pctl,
@@ -813,10 +987,11 @@
pctl->pctl_desc.pmxops = &stm32_pmx_ops;
pctl->dev = &pdev->dev;
- pctl->pctl_dev = pinctrl_register(&pctl->pctl_desc, &pdev->dev, pctl);
- if (!pctl->pctl_dev) {
+ pctl->pctl_dev = devm_pinctrl_register(&pdev->dev, &pctl->pctl_desc,
+ pctl);
+ if (IS_ERR(pctl->pctl_dev)) {
dev_err(&pdev->dev, "Failed pinctrl registration\n");
- return -EINVAL;
+ return PTR_ERR(pctl->pctl_dev);
}
for (i = 0; i < pctl->nbanks; i++)
diff --git a/drivers/pinctrl/sunxi/pinctrl-sunxi.c b/drivers/pinctrl/sunxi/pinctrl-sunxi.c
index 3b017db..54455af 100644
--- a/drivers/pinctrl/sunxi/pinctrl-sunxi.c
+++ b/drivers/pinctrl/sunxi/pinctrl-sunxi.c
@@ -933,18 +933,15 @@
pctrl_desc->pctlops = &sunxi_pctrl_ops;
pctrl_desc->pmxops = &sunxi_pmx_ops;
- pctl->pctl_dev = pinctrl_register(pctrl_desc,
- &pdev->dev, pctl);
+ pctl->pctl_dev = devm_pinctrl_register(&pdev->dev, pctrl_desc, pctl);
if (IS_ERR(pctl->pctl_dev)) {
dev_err(&pdev->dev, "couldn't register pinctrl driver\n");
return PTR_ERR(pctl->pctl_dev);
}
pctl->chip = devm_kzalloc(&pdev->dev, sizeof(*pctl->chip), GFP_KERNEL);
- if (!pctl->chip) {
- ret = -ENOMEM;
- goto pinctrl_error;
- }
+ if (!pctl->chip)
+ return -ENOMEM;
last_pin = pctl->desc->pins[pctl->desc->npins - 1].pin.number;
pctl->chip->owner = THIS_MODULE;
@@ -966,7 +963,7 @@
ret = gpiochip_add_data(pctl->chip, pctl);
if (ret)
- goto pinctrl_error;
+ return ret;
for (i = 0; i < pctl->desc->npins; i++) {
const struct sunxi_desc_pin *pin = pctl->desc->pins + i;
@@ -1044,7 +1041,5 @@
clk_disable_unprepare(clk);
gpiochip_error:
gpiochip_remove(pctl->chip);
-pinctrl_error:
- pinctrl_unregister(pctl->pctl_dev);
return ret;
}
diff --git a/drivers/pinctrl/tegra/pinctrl-tegra-xusb.c b/drivers/pinctrl/tegra/pinctrl-tegra-xusb.c
index 946cda3..6f68a9e 100644
--- a/drivers/pinctrl/tegra/pinctrl-tegra-xusb.c
+++ b/drivers/pinctrl/tegra/pinctrl-tegra-xusb.c
@@ -267,7 +267,7 @@
.get_group_name = tegra_xusb_padctl_get_group_name,
.get_group_pins = tegra_xusb_padctl_get_group_pins,
.dt_node_to_map = tegra_xusb_padctl_dt_node_to_map,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
static int tegra_xusb_padctl_get_functions_count(struct pinctrl_dev *pinctrl)
@@ -914,7 +914,8 @@
padctl->desc.confops = &tegra_xusb_padctl_pinconf_ops;
padctl->desc.owner = THIS_MODULE;
- padctl->pinctrl = pinctrl_register(&padctl->desc, &pdev->dev, padctl);
+ padctl->pinctrl = devm_pinctrl_register(&pdev->dev, &padctl->desc,
+ padctl);
if (IS_ERR(padctl->pinctrl)) {
dev_err(&pdev->dev, "failed to register pincontrol\n");
err = PTR_ERR(padctl->pinctrl);
@@ -924,7 +925,7 @@
phy = devm_phy_create(&pdev->dev, NULL, &pcie_phy_ops);
if (IS_ERR(phy)) {
err = PTR_ERR(phy);
- goto unregister;
+ goto reset;
}
padctl->phys[TEGRA_XUSB_PADCTL_PCIE] = phy;
@@ -933,7 +934,7 @@
phy = devm_phy_create(&pdev->dev, NULL, &sata_phy_ops);
if (IS_ERR(phy)) {
err = PTR_ERR(phy);
- goto unregister;
+ goto reset;
}
padctl->phys[TEGRA_XUSB_PADCTL_SATA] = phy;
@@ -944,13 +945,11 @@
if (IS_ERR(padctl->provider)) {
err = PTR_ERR(padctl->provider);
dev_err(&pdev->dev, "failed to register PHYs: %d\n", err);
- goto unregister;
+ goto reset;
}
return 0;
-unregister:
- pinctrl_unregister(padctl->pinctrl);
reset:
reset_control_assert(padctl->rst);
return err;
@@ -962,8 +961,6 @@
struct tegra_xusb_padctl *padctl = platform_get_drvdata(pdev);
int err;
- pinctrl_unregister(padctl->pinctrl);
-
err = reset_control_assert(padctl->rst);
if (err < 0)
dev_err(&pdev->dev, "failed to assert reset: %d\n", err);
diff --git a/drivers/pinctrl/tegra/pinctrl-tegra.c b/drivers/pinctrl/tegra/pinctrl-tegra.c
index 4938882..6e82b29 100644
--- a/drivers/pinctrl/tegra/pinctrl-tegra.c
+++ b/drivers/pinctrl/tegra/pinctrl-tegra.c
@@ -215,7 +215,7 @@
ret = tegra_pinctrl_dt_subnode_to_map(pctldev, np, map,
&reserved_maps, num_maps);
if (ret < 0) {
- pinctrl_utils_dt_free_map(pctldev, *map,
+ pinctrl_utils_free_map(pctldev, *map,
*num_maps);
of_node_put(np);
return ret;
@@ -233,7 +233,7 @@
.pin_dbg_show = tegra_pinctrl_pin_dbg_show,
#endif
.dt_node_to_map = tegra_pinctrl_dt_node_to_map,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
static int tegra_pinctrl_get_funcs_count(struct pinctrl_dev *pctldev)
@@ -417,7 +417,7 @@
return -ENOTSUPP;
}
- if (*reg < 0 || *bit > 31) {
+ if (*reg < 0 || *bit < 0) {
if (report_err) {
const char *prop = "unknown";
int i;
@@ -625,6 +625,22 @@
.owner = THIS_MODULE,
};
+static void tegra_pinctrl_clear_parked_bits(struct tegra_pmx *pmx)
+{
+ int i = 0;
+ const struct tegra_pingroup *g;
+ u32 val;
+
+ for (i = 0; i < pmx->soc->ngroups; ++i) {
+ if (pmx->soc->groups[i].parked_reg >= 0) {
+ g = &pmx->soc->groups[i];
+ val = pmx_readl(pmx, g->parked_bank, g->parked_reg);
+ val &= ~(1 << g->parked_bit);
+ pmx_writel(pmx, val, g->parked_bank, g->parked_reg);
+ }
+ }
+}
+
static bool gpio_node_has_range(void)
{
struct device_node *np;
@@ -719,12 +735,14 @@
return PTR_ERR(pmx->regs[i]);
}
- pmx->pctl = pinctrl_register(&tegra_pinctrl_desc, &pdev->dev, pmx);
+ pmx->pctl = devm_pinctrl_register(&pdev->dev, &tegra_pinctrl_desc, pmx);
if (IS_ERR(pmx->pctl)) {
dev_err(&pdev->dev, "Couldn't register pinctrl driver\n");
return PTR_ERR(pmx->pctl);
}
+ tegra_pinctrl_clear_parked_bits(pmx);
+
if (!gpio_node_has_range())
pinctrl_add_gpio_range(pmx->pctl, &tegra_pinctrl_gpio_range);
@@ -735,13 +753,3 @@
return 0;
}
EXPORT_SYMBOL_GPL(tegra_pinctrl_probe);
-
-int tegra_pinctrl_remove(struct platform_device *pdev)
-{
- struct tegra_pmx *pmx = platform_get_drvdata(pdev);
-
- pinctrl_unregister(pmx->pctl);
-
- return 0;
-}
-EXPORT_SYMBOL_GPL(tegra_pinctrl_remove);
diff --git a/drivers/pinctrl/tegra/pinctrl-tegra.h b/drivers/pinctrl/tegra/pinctrl-tegra.h
index 1615db7..d2ced17 100644
--- a/drivers/pinctrl/tegra/pinctrl-tegra.h
+++ b/drivers/pinctrl/tegra/pinctrl-tegra.h
@@ -93,6 +93,9 @@
* @tri_reg: Tri-state register offset.
* @tri_bank: Tri-state register bank.
* @tri_bit: Tri-state register bit.
+ * @parked_reg: Parked register offset. -1 if unsupported.
+ * @parked_bank: Parked register bank. 0 if unsupported.
+ * @parked_bit: Parked register bit. 0 if unsupported.
* @einput_bit: Enable-input register bit.
* @odrain_bit: Open-drain register bit.
* @lock_bit: Lock register bit.
@@ -135,13 +138,16 @@
s16 pupd_reg;
s16 tri_reg;
s16 drv_reg;
+ s16 parked_reg;
u32 mux_bank:2;
u32 pupd_bank:2;
u32 tri_bank:2;
u32 drv_bank:2;
+ u32 parked_bank:2;
s32 mux_bit:6;
s32 pupd_bit:6;
s32 tri_bit:6;
+ s32 parked_bit:6;
s32 einput_bit:6;
s32 odrain_bit:6;
s32 lock_bit:6;
@@ -189,6 +195,4 @@
int tegra_pinctrl_probe(struct platform_device *pdev,
const struct tegra_pinctrl_soc_data *soc_data);
-int tegra_pinctrl_remove(struct platform_device *pdev);
-
#endif
diff --git a/drivers/pinctrl/tegra/pinctrl-tegra114.c b/drivers/pinctrl/tegra/pinctrl-tegra114.c
index 05e49d5..4851d16 100644
--- a/drivers/pinctrl/tegra/pinctrl-tegra114.c
+++ b/drivers/pinctrl/tegra/pinctrl-tegra114.c
@@ -1578,6 +1578,7 @@
.lock_bit = 7, \
.ioreset_bit = PINGROUP_BIT_##ior(8), \
.rcv_sel_bit = PINGROUP_BIT_##rcv_sel(9), \
+ .parked_reg = -1, \
.drv_reg = -1, \
}
@@ -1598,6 +1599,7 @@
.rcv_sel_bit = -1, \
.drv_reg = DRV_PINGROUP_REG(r), \
.drv_bank = 0, \
+ .parked_reg = -1, \
.hsm_bit = hsm_b, \
.schmitt_bit = schmitt_b, \
.lpmd_bit = lpmd_b, \
@@ -1863,7 +1865,6 @@
.of_match_table = tegra114_pinctrl_of_match,
},
.probe = tegra114_pinctrl_probe,
- .remove = tegra_pinctrl_remove,
};
module_platform_driver(tegra114_pinctrl_driver);
diff --git a/drivers/pinctrl/tegra/pinctrl-tegra124.c b/drivers/pinctrl/tegra/pinctrl-tegra124.c
index 7cd44c7..a0ce723 100644
--- a/drivers/pinctrl/tegra/pinctrl-tegra124.c
+++ b/drivers/pinctrl/tegra/pinctrl-tegra124.c
@@ -1747,6 +1747,7 @@
.lock_bit = 7, \
.ioreset_bit = PINGROUP_BIT_##ior(8), \
.rcv_sel_bit = PINGROUP_BIT_##rcv_sel(9), \
+ .parked_reg = -1, \
.drv_reg = -1, \
}
@@ -1767,6 +1768,7 @@
.rcv_sel_bit = -1, \
.drv_reg = DRV_PINGROUP_REG(r), \
.drv_bank = 0, \
+ .parked_reg = -1, \
.hsm_bit = hsm_b, \
.schmitt_bit = schmitt_b, \
.lpmd_bit = lpmd_b, \
@@ -2075,7 +2077,6 @@
.of_match_table = tegra124_pinctrl_of_match,
},
.probe = tegra124_pinctrl_probe,
- .remove = tegra_pinctrl_remove,
};
module_platform_driver(tegra124_pinctrl_driver);
diff --git a/drivers/pinctrl/tegra/pinctrl-tegra20.c b/drivers/pinctrl/tegra/pinctrl-tegra20.c
index 4833db4..09bad69 100644
--- a/drivers/pinctrl/tegra/pinctrl-tegra20.c
+++ b/drivers/pinctrl/tegra/pinctrl-tegra20.c
@@ -1994,6 +1994,7 @@
.tri_reg = ((tri_r) - TRISTATE_REG_A), \
.tri_bank = 0, \
.tri_bit = tri_b, \
+ .parked_reg = -1, \
.einput_bit = -1, \
.odrain_bit = -1, \
.lock_bit = -1, \
@@ -2013,6 +2014,7 @@
.pupd_bank = 2, \
.pupd_bit = pupd_b, \
.drv_reg = -1, \
+ .parked_reg = -1, \
}
/* Pin groups for drive strength registers (configurable version) */
@@ -2028,6 +2030,7 @@
.tri_reg = -1, \
.drv_reg = ((r) - PINGROUP_REG_A), \
.drv_bank = 3, \
+ .parked_reg = -1, \
.hsm_bit = hsm_b, \
.schmitt_bit = schmitt_b, \
.lpmd_bit = lpmd_b, \
@@ -2242,7 +2245,6 @@
.of_match_table = tegra20_pinctrl_of_match,
},
.probe = tegra20_pinctrl_probe,
- .remove = tegra_pinctrl_remove,
};
module_platform_driver(tegra20_pinctrl_driver);
diff --git a/drivers/pinctrl/tegra/pinctrl-tegra210.c b/drivers/pinctrl/tegra/pinctrl-tegra210.c
index 252b464..2d856af 100644
--- a/drivers/pinctrl/tegra/pinctrl-tegra210.c
+++ b/drivers/pinctrl/tegra/pinctrl-tegra210.c
@@ -1310,6 +1310,9 @@
.lock_bit = 7, \
.ioreset_bit = -1, \
.rcv_sel_bit = PINGROUP_BIT_##e_io_hv(10), \
+ .parked_reg = PINGROUP_REG(r), \
+ .parked_bank = 1, \
+ .parked_bit = 5, \
.hsm_bit = PINGROUP_BIT_##hsm(9), \
.schmitt_bit = 12, \
.drvtype_bit = PINGROUP_BIT_##drvtype(13), \
@@ -1342,6 +1345,7 @@
.rcv_sel_bit = -1, \
.drv_reg = DRV_PINGROUP_REG(r), \
.drv_bank = 0, \
+ .parked_reg = -1, \
.hsm_bit = -1, \
.schmitt_bit = -1, \
.lpmd_bit = -1, \
@@ -1579,7 +1583,6 @@
.of_match_table = tegra210_pinctrl_of_match,
},
.probe = tegra210_pinctrl_probe,
- .remove = tegra_pinctrl_remove,
};
module_platform_driver(tegra210_pinctrl_driver);
diff --git a/drivers/pinctrl/tegra/pinctrl-tegra30.c b/drivers/pinctrl/tegra/pinctrl-tegra30.c
index 47b2fd8..fb7817f 100644
--- a/drivers/pinctrl/tegra/pinctrl-tegra30.c
+++ b/drivers/pinctrl/tegra/pinctrl-tegra30.c
@@ -2139,6 +2139,7 @@
.lock_bit = 7, \
.ioreset_bit = PINGROUP_BIT_##ior(8), \
.rcv_sel_bit = -1, \
+ .parked_reg = -1, \
.drv_reg = -1, \
}
@@ -2159,6 +2160,7 @@
.rcv_sel_bit = -1, \
.drv_reg = DRV_PINGROUP_REG(r), \
.drv_bank = 0, \
+ .parked_reg = -1, \
.hsm_bit = hsm_b, \
.schmitt_bit = schmitt_b, \
.lpmd_bit = lpmd_b, \
@@ -2498,7 +2500,6 @@
.of_match_table = tegra30_pinctrl_of_match,
},
.probe = tegra30_pinctrl_probe,
- .remove = tegra_pinctrl_remove,
};
module_platform_driver(tegra30_pinctrl_driver);
diff --git a/drivers/pinctrl/uniphier/pinctrl-uniphier-core.c b/drivers/pinctrl/uniphier/pinctrl-uniphier-core.c
index 589872c..9674009 100644
--- a/drivers/pinctrl/uniphier/pinctrl-uniphier-core.c
+++ b/drivers/pinctrl/uniphier/pinctrl-uniphier-core.c
@@ -115,7 +115,7 @@
.pin_dbg_show = uniphier_pctl_pin_dbg_show,
#endif
.dt_node_to_map = pinconf_generic_dt_node_to_map_all,
- .dt_free_map = pinctrl_utils_dt_free_map,
+ .dt_free_map = pinctrl_utils_free_map,
};
static int uniphier_conf_pin_bias_get(struct pinctrl_dev *pctldev,
@@ -665,7 +665,7 @@
desc->pmxops = &uniphier_pmxops;
desc->confops = &uniphier_confops;
- priv->pctldev = pinctrl_register(desc, dev, priv);
+ priv->pctldev = devm_pinctrl_register(dev, desc, priv);
if (IS_ERR(priv->pctldev)) {
dev_err(dev, "failed to register UniPhier pinctrl driver\n");
return PTR_ERR(priv->pctldev);
@@ -676,13 +676,3 @@
return 0;
}
EXPORT_SYMBOL_GPL(uniphier_pinctrl_probe);
-
-int uniphier_pinctrl_remove(struct platform_device *pdev)
-{
- struct uniphier_pinctrl_priv *priv = platform_get_drvdata(pdev);
-
- pinctrl_unregister(priv->pctldev);
-
- return 0;
-}
-EXPORT_SYMBOL_GPL(uniphier_pinctrl_remove);
diff --git a/drivers/pinctrl/uniphier/pinctrl-uniphier-ld4.c b/drivers/pinctrl/uniphier/pinctrl-uniphier-ld4.c
index a7056dc..4a0439c 100644
--- a/drivers/pinctrl/uniphier/pinctrl-uniphier-ld4.c
+++ b/drivers/pinctrl/uniphier/pinctrl-uniphier-ld4.c
@@ -878,7 +878,6 @@
static struct platform_driver ph1_ld4_pinctrl_driver = {
.probe = ph1_ld4_pinctrl_probe,
- .remove = uniphier_pinctrl_remove,
.driver = {
.name = DRIVER_NAME,
.of_match_table = ph1_ld4_pinctrl_match,
diff --git a/drivers/pinctrl/uniphier/pinctrl-uniphier-ld6b.c b/drivers/pinctrl/uniphier/pinctrl-uniphier-ld6b.c
index 1824831..150d339 100644
--- a/drivers/pinctrl/uniphier/pinctrl-uniphier-ld6b.c
+++ b/drivers/pinctrl/uniphier/pinctrl-uniphier-ld6b.c
@@ -1266,7 +1266,6 @@
static struct platform_driver ph1_ld6b_pinctrl_driver = {
.probe = ph1_ld6b_pinctrl_probe,
- .remove = uniphier_pinctrl_remove,
.driver = {
.name = DRIVER_NAME,
.of_match_table = ph1_ld6b_pinctrl_match,
diff --git a/drivers/pinctrl/uniphier/pinctrl-uniphier-pro4.c b/drivers/pinctrl/uniphier/pinctrl-uniphier-pro4.c
index ec8e92d..b1f09e6 100644
--- a/drivers/pinctrl/uniphier/pinctrl-uniphier-pro4.c
+++ b/drivers/pinctrl/uniphier/pinctrl-uniphier-pro4.c
@@ -1552,7 +1552,6 @@
static struct platform_driver ph1_pro4_pinctrl_driver = {
.probe = ph1_pro4_pinctrl_probe,
- .remove = uniphier_pinctrl_remove,
.driver = {
.name = DRIVER_NAME,
.of_match_table = ph1_pro4_pinctrl_match,
diff --git a/drivers/pinctrl/uniphier/pinctrl-uniphier-pro5.c b/drivers/pinctrl/uniphier/pinctrl-uniphier-pro5.c
index e3d648e..3087f76 100644
--- a/drivers/pinctrl/uniphier/pinctrl-uniphier-pro5.c
+++ b/drivers/pinctrl/uniphier/pinctrl-uniphier-pro5.c
@@ -1343,7 +1343,6 @@
static struct platform_driver ph1_pro5_pinctrl_driver = {
.probe = ph1_pro5_pinctrl_probe,
- .remove = uniphier_pinctrl_remove,
.driver = {
.name = DRIVER_NAME,
.of_match_table = ph1_pro5_pinctrl_match,
diff --git a/drivers/pinctrl/uniphier/pinctrl-uniphier-pxs2.c b/drivers/pinctrl/uniphier/pinctrl-uniphier-pxs2.c
index bc00d75..e868030 100644
--- a/drivers/pinctrl/uniphier/pinctrl-uniphier-pxs2.c
+++ b/drivers/pinctrl/uniphier/pinctrl-uniphier-pxs2.c
@@ -1261,7 +1261,6 @@
static struct platform_driver proxstream2_pinctrl_driver = {
.probe = proxstream2_pinctrl_probe,
- .remove = uniphier_pinctrl_remove,
.driver = {
.name = DRIVER_NAME,
.of_match_table = proxstream2_pinctrl_match,
diff --git a/drivers/pinctrl/uniphier/pinctrl-uniphier-sld8.c b/drivers/pinctrl/uniphier/pinctrl-uniphier-sld8.c
index c3700a3..ceb7a989 100644
--- a/drivers/pinctrl/uniphier/pinctrl-uniphier-sld8.c
+++ b/drivers/pinctrl/uniphier/pinctrl-uniphier-sld8.c
@@ -786,7 +786,6 @@
static struct platform_driver ph1_sld8_pinctrl_driver = {
.probe = ph1_sld8_pinctrl_probe,
- .remove = uniphier_pinctrl_remove,
.driver = {
.name = DRIVER_NAME,
.of_match_table = ph1_sld8_pinctrl_match,
diff --git a/drivers/pinctrl/uniphier/pinctrl-uniphier.h b/drivers/pinctrl/uniphier/pinctrl-uniphier.h
index e1e98b8..a21154f 100644
--- a/drivers/pinctrl/uniphier/pinctrl-uniphier.h
+++ b/drivers/pinctrl/uniphier/pinctrl-uniphier.h
@@ -212,6 +212,4 @@
struct pinctrl_desc *desc,
struct uniphier_pinctrl_socdata *socdata);
-int uniphier_pinctrl_remove(struct platform_device *pdev);
-
#endif /* __PINCTRL_UNIPHIER_H__ */
diff --git a/drivers/pinctrl/vt8500/pinctrl-wmt.c b/drivers/pinctrl/vt8500/pinctrl-wmt.c
index 5c261bf..cbc6386 100644
--- a/drivers/pinctrl/vt8500/pinctrl-wmt.c
+++ b/drivers/pinctrl/vt8500/pinctrl-wmt.c
@@ -583,7 +583,7 @@
data->dev = &pdev->dev;
- data->pctl_dev = pinctrl_register(&wmt_desc, &pdev->dev, data);
+ data->pctl_dev = devm_pinctrl_register(&pdev->dev, &wmt_desc, data);
if (IS_ERR(data->pctl_dev)) {
dev_err(&pdev->dev, "Failed to register pinctrl\n");
return PTR_ERR(data->pctl_dev);
@@ -592,7 +592,7 @@
err = gpiochip_add_data(&data->gpio_chip, data);
if (err) {
dev_err(&pdev->dev, "could not add GPIO chip\n");
- goto fail_gpio;
+ return err;
}
err = gpiochip_add_pin_range(&data->gpio_chip, dev_name(data->dev),
@@ -606,8 +606,6 @@
fail_range:
gpiochip_remove(&data->gpio_chip);
-fail_gpio:
- pinctrl_unregister(data->pctl_dev);
return err;
}
@@ -616,7 +614,6 @@
struct wmt_pinctrl_data *data = platform_get_drvdata(pdev);
gpiochip_remove(&data->gpio_chip);
- pinctrl_unregister(data->pctl_dev);
return 0;
}
diff --git a/drivers/platform/mips/Kconfig b/drivers/platform/mips/Kconfig
index 125e569..b3ae30a 100644
--- a/drivers/platform/mips/Kconfig
+++ b/drivers/platform/mips/Kconfig
@@ -15,10 +15,6 @@
if MIPS_PLATFORM_DEVICES
-config MIPS_ACPI
- bool
- default y if LOONGSON_MACH3X
-
config CPU_HWMON
tristate "Loongson CPU HWMon Driver"
depends on LOONGSON_MACH3X
diff --git a/drivers/platform/mips/Makefile b/drivers/platform/mips/Makefile
index 4341284..8dfd039 100644
--- a/drivers/platform/mips/Makefile
+++ b/drivers/platform/mips/Makefile
@@ -1,2 +1 @@
-obj-$(CONFIG_MIPS_ACPI) += acpi_init.o
obj-$(CONFIG_CPU_HWMON) += cpu_hwmon.o
diff --git a/drivers/platform/mips/cpu_hwmon.c b/drivers/platform/mips/cpu_hwmon.c
index 4993e19..4300a55 100644
--- a/drivers/platform/mips/cpu_hwmon.c
+++ b/drivers/platform/mips/cpu_hwmon.c
@@ -20,9 +20,9 @@
u32 reg;
reg = LOONGSON_CHIPTEMP(cpu);
- if (loongson_sysconf.cputype == Loongson_3A)
+ if ((read_c0_prid() & PRID_REV_MASK) == PRID_REV_LOONGSON3A_R1)
reg = (reg >> 8) & 0xff;
- else if (loongson_sysconf.cputype == Loongson_3B)
+ else
reg = ((reg >> 8) & 0xff) - 100;
return (int)reg * 1000;
diff --git a/drivers/platform/x86/wmi.c b/drivers/platform/x86/wmi.c
index eb391a2..ceeb8c1 100644
--- a/drivers/platform/x86/wmi.c
+++ b/drivers/platform/x86/wmi.c
@@ -37,6 +37,7 @@
#include <linux/acpi.h>
#include <linux/slab.h>
#include <linux/module.h>
+#include <linux/uuid.h>
ACPI_MODULE_NAME("wmi");
MODULE_AUTHOR("Carlos Corbacho");
@@ -115,100 +116,21 @@
* GUID parsing functions
*/
-/**
- * wmi_parse_hexbyte - Convert a ASCII hex number to a byte
- * @src: Pointer to at least 2 characters to convert.
- *
- * Convert a two character ASCII hex string to a number.
- *
- * Return: 0-255 Success, the byte was parsed correctly
- * -1 Error, an invalid character was supplied
- */
-static int wmi_parse_hexbyte(const u8 *src)
-{
- int h;
- int value;
-
- /* high part */
- h = value = hex_to_bin(src[0]);
- if (value < 0)
- return -1;
-
- /* low part */
- value = hex_to_bin(src[1]);
- if (value >= 0)
- return (h << 4) | value;
- return -1;
-}
-
-/**
- * wmi_swap_bytes - Rearrange GUID bytes to match GUID binary
- * @src: Memory block holding binary GUID (16 bytes)
- * @dest: Memory block to hold byte swapped binary GUID (16 bytes)
- *
- * Byte swap a binary GUID to match it's real GUID value
- */
-static void wmi_swap_bytes(u8 *src, u8 *dest)
-{
- int i;
-
- for (i = 0; i <= 3; i++)
- memcpy(dest + i, src + (3 - i), 1);
-
- for (i = 0; i <= 1; i++)
- memcpy(dest + 4 + i, src + (5 - i), 1);
-
- for (i = 0; i <= 1; i++)
- memcpy(dest + 6 + i, src + (7 - i), 1);
-
- memcpy(dest + 8, src + 8, 8);
-}
-
-/**
- * wmi_parse_guid - Convert GUID from ASCII to binary
- * @src: 36 char string of the form fa50ff2b-f2e8-45de-83fa-65417f2f49ba
- * @dest: Memory block to hold binary GUID (16 bytes)
- *
- * N.B. The GUID need not be NULL terminated.
- *
- * Return: 'true' @dest contains binary GUID
- * 'false' @dest contents are undefined
- */
-static bool wmi_parse_guid(const u8 *src, u8 *dest)
-{
- static const int size[] = { 4, 2, 2, 2, 6 };
- int i, j, v;
-
- if (src[8] != '-' || src[13] != '-' ||
- src[18] != '-' || src[23] != '-')
- return false;
-
- for (j = 0; j < 5; j++, src++) {
- for (i = 0; i < size[j]; i++, src += 2, *dest++ = v) {
- v = wmi_parse_hexbyte(src);
- if (v < 0)
- return false;
- }
- }
-
- return true;
-}
-
static bool find_guid(const char *guid_string, struct wmi_block **out)
{
- char tmp[16], guid_input[16];
+ uuid_le guid_input;
struct wmi_block *wblock;
struct guid_block *block;
struct list_head *p;
- wmi_parse_guid(guid_string, tmp);
- wmi_swap_bytes(tmp, guid_input);
+ if (uuid_le_to_bin(guid_string, &guid_input))
+ return false;
list_for_each(p, &wmi_block_list) {
wblock = list_entry(p, struct wmi_block, list);
block = &wblock->gblock;
- if (memcmp(block->guid, guid_input, 16) == 0) {
+ if (memcmp(block->guid, &guid_input, 16) == 0) {
if (out)
*out = wblock;
return true;
@@ -498,20 +420,20 @@
{
struct wmi_block *block;
acpi_status status = AE_NOT_EXIST;
- char tmp[16], guid_input[16];
+ uuid_le guid_input;
struct list_head *p;
if (!guid || !handler)
return AE_BAD_PARAMETER;
- wmi_parse_guid(guid, tmp);
- wmi_swap_bytes(tmp, guid_input);
+ if (uuid_le_to_bin(guid, &guid_input))
+ return AE_BAD_PARAMETER;
list_for_each(p, &wmi_block_list) {
acpi_status wmi_status;
block = list_entry(p, struct wmi_block, list);
- if (memcmp(block->gblock.guid, guid_input, 16) == 0) {
+ if (memcmp(block->gblock.guid, &guid_input, 16) == 0) {
if (block->handler &&
block->handler != wmi_notify_debug)
return AE_ALREADY_ACQUIRED;
@@ -539,20 +461,20 @@
{
struct wmi_block *block;
acpi_status status = AE_NOT_EXIST;
- char tmp[16], guid_input[16];
+ uuid_le guid_input;
struct list_head *p;
if (!guid)
return AE_BAD_PARAMETER;
- wmi_parse_guid(guid, tmp);
- wmi_swap_bytes(tmp, guid_input);
+ if (uuid_le_to_bin(guid, &guid_input))
+ return AE_BAD_PARAMETER;
list_for_each(p, &wmi_block_list) {
acpi_status wmi_status;
block = list_entry(p, struct wmi_block, list);
- if (memcmp(block->gblock.guid, guid_input, 16) == 0) {
+ if (memcmp(block->gblock.guid, &guid_input, 16) == 0) {
if (!block->handler ||
block->handler == wmi_notify_debug)
return AE_NULL_ENTRY;
diff --git a/drivers/pnp/pnpbios/Kconfig b/drivers/pnp/pnpbios/Kconfig
index 50c3dd0..a786086 100644
--- a/drivers/pnp/pnpbios/Kconfig
+++ b/drivers/pnp/pnpbios/Kconfig
@@ -3,7 +3,7 @@
#
config PNPBIOS
bool "Plug and Play BIOS support"
- depends on ISA && X86
+ depends on ISA && X86_32
default n
---help---
Linux uses the PNPBIOS as defined in "Plug and Play BIOS
diff --git a/drivers/power/ipaq_micro_battery.c b/drivers/power/ipaq_micro_battery.c
index 3f314b1..35b01c7 100644
--- a/drivers/power/ipaq_micro_battery.c
+++ b/drivers/power/ipaq_micro_battery.c
@@ -261,7 +261,7 @@
return 0;
ac_err:
- power_supply_unregister(micro_ac_power);
+ power_supply_unregister(micro_batt_power);
batt_err:
cancel_delayed_work_sync(&mb->update);
destroy_workqueue(mb->wq);
diff --git a/drivers/power/max8925_power.c b/drivers/power/max8925_power.c
index 57eb5c2..3b94620 100644
--- a/drivers/power/max8925_power.c
+++ b/drivers/power/max8925_power.c
@@ -540,14 +540,14 @@
info->usb = power_supply_register(&pdev->dev, &usb_desc, &psy_cfg);
if (IS_ERR(info->usb)) {
ret = PTR_ERR(info->usb);
- goto out_usb;
+ goto out_unregister_ac;
}
info->usb->dev.parent = &pdev->dev;
info->battery = power_supply_register(&pdev->dev, &battery_desc, NULL);
if (IS_ERR(info->battery)) {
ret = PTR_ERR(info->battery);
- goto out_battery;
+ goto out_unregister_usb;
}
info->battery->dev.parent = &pdev->dev;
@@ -560,9 +560,9 @@
max8925_init_charger(chip, info);
return 0;
-out_battery:
- power_supply_unregister(info->battery);
-out_usb:
+out_unregister_usb:
+ power_supply_unregister(info->usb);
+out_unregister_ac:
power_supply_unregister(info->ac);
out:
return ret;
diff --git a/drivers/power/reset/Kconfig b/drivers/power/reset/Kconfig
index 0a6408a..9bb2622 100644
--- a/drivers/power/reset/Kconfig
+++ b/drivers/power/reset/Kconfig
@@ -30,6 +30,14 @@
This driver supports restart for Atmel AT91SAM9 and SAMA5
SoCs
+config POWER_RESET_AT91_SAMA5D2_SHDWC
+ tristate "Atmel AT91 SAMA5D2-Compatible shutdown controller driver"
+ depends on ARCH_AT91 || COMPILE_TEST
+ default SOC_SAMA5
+ help
+ This driver supports the alternate shutdown controller for some Atmel
+ SAMA5 SoCs. It is present for example on SAMA5D2 SoC.
+
config POWER_RESET_AXXIA
bool "LSI Axxia reset driver"
depends on ARCH_AXXIA
diff --git a/drivers/power/reset/Makefile b/drivers/power/reset/Makefile
index 096fa67..ab7aa86 100644
--- a/drivers/power/reset/Makefile
+++ b/drivers/power/reset/Makefile
@@ -1,6 +1,7 @@
obj-$(CONFIG_POWER_RESET_AS3722) += as3722-poweroff.o
obj-$(CONFIG_POWER_RESET_AT91_POWEROFF) += at91-poweroff.o
obj-$(CONFIG_POWER_RESET_AT91_RESET) += at91-reset.o
+obj-$(CONFIG_POWER_RESET_AT91_SAMA5D2_SHDWC) += at91-sama5d2_shdwc.o
obj-$(CONFIG_POWER_RESET_AXXIA) += axxia-reset.o
obj-$(CONFIG_POWER_RESET_BRCMSTB) += brcmstb-reboot.o
obj-$(CONFIG_POWER_RESET_GPIO) += gpio-poweroff.o
diff --git a/drivers/power/reset/at91-sama5d2_shdwc.c b/drivers/power/reset/at91-sama5d2_shdwc.c
new file mode 100644
index 0000000..8a5ac97
--- /dev/null
+++ b/drivers/power/reset/at91-sama5d2_shdwc.c
@@ -0,0 +1,282 @@
+/*
+ * Atmel SAMA5D2-Compatible Shutdown Controller (SHDWC) driver.
+ * Found on some SoCs as the sama5d2 (obviously).
+ *
+ * Copyright (C) 2015 Atmel Corporation,
+ * Nicolas Ferre <nicolas.ferre@atmel.com>
+ *
+ * Evolved from driver at91-poweroff.c.
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2. This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ *
+ * TODO:
+ * - addition to status of other wake-up inputs [1 - 15]
+ * - Analog Comparator wake-up alarm
+ * - Serial RX wake-up alarm
+ * - low power debouncer
+ */
+
+#include <linux/clk.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/printk.h>
+
+#define SLOW_CLOCK_FREQ 32768
+
+#define AT91_SHDW_CR 0x00 /* Shut Down Control Register */
+#define AT91_SHDW_SHDW BIT(0) /* Shut Down command */
+#define AT91_SHDW_KEY (0xa5UL << 24) /* KEY Password */
+
+#define AT91_SHDW_MR 0x04 /* Shut Down Mode Register */
+#define AT91_SHDW_WKUPDBC_SHIFT 24
+#define AT91_SHDW_WKUPDBC_MASK GENMASK(31, 16)
+#define AT91_SHDW_WKUPDBC(x) (((x) << AT91_SHDW_WKUPDBC_SHIFT) \
+ & AT91_SHDW_WKUPDBC_MASK)
+
+#define AT91_SHDW_SR 0x08 /* Shut Down Status Register */
+#define AT91_SHDW_WKUPIS_SHIFT 16
+#define AT91_SHDW_WKUPIS_MASK GENMASK(31, 16)
+#define AT91_SHDW_WKUPIS(x) ((1 << (x)) << AT91_SHDW_WKUPIS_SHIFT \
+ & AT91_SHDW_WKUPIS_MASK)
+
+#define AT91_SHDW_WUIR 0x0c /* Shutdown Wake-up Inputs Register */
+#define AT91_SHDW_WKUPEN_MASK GENMASK(15, 0)
+#define AT91_SHDW_WKUPEN(x) ((1 << (x)) & AT91_SHDW_WKUPEN_MASK)
+#define AT91_SHDW_WKUPT_SHIFT 16
+#define AT91_SHDW_WKUPT_MASK GENMASK(31, 16)
+#define AT91_SHDW_WKUPT(x) ((1 << (x)) << AT91_SHDW_WKUPT_SHIFT \
+ & AT91_SHDW_WKUPT_MASK)
+
+#define SHDW_WK_PIN(reg, cfg) ((reg) & AT91_SHDW_WKUPIS((cfg)->wkup_pin_input))
+#define SHDW_RTCWK(reg, cfg) (((reg) >> ((cfg)->sr_rtcwk_shift)) & 0x1)
+#define SHDW_RTCWKEN(cfg) (1 << ((cfg)->mr_rtcwk_shift))
+
+#define DBC_PERIOD_US(x) DIV_ROUND_UP_ULL((1000000 * (x)), \
+ SLOW_CLOCK_FREQ)
+
+struct shdwc_config {
+ u8 wkup_pin_input;
+ u8 mr_rtcwk_shift;
+ u8 sr_rtcwk_shift;
+};
+
+struct shdwc {
+ struct shdwc_config *cfg;
+ void __iomem *at91_shdwc_base;
+};
+
+/*
+ * Hold configuration here, cannot be more than one instance of the driver
+ * since pm_power_off itself is global.
+ */
+static struct shdwc *at91_shdwc;
+static struct clk *sclk;
+
+static const unsigned long long sdwc_dbc_period[] = {
+ 0, 3, 32, 512, 4096, 32768,
+};
+
+static void __init at91_wakeup_status(struct platform_device *pdev)
+{
+ struct shdwc *shdw = platform_get_drvdata(pdev);
+ u32 reg;
+ char *reason = "unknown";
+
+ reg = readl(shdw->at91_shdwc_base + AT91_SHDW_SR);
+
+ dev_dbg(&pdev->dev, "%s: status = %#x\n", __func__, reg);
+
+ /* Simple power-on, just bail out */
+ if (!reg)
+ return;
+
+ if (SHDW_WK_PIN(reg, shdw->cfg))
+ reason = "WKUP pin";
+ else if (SHDW_RTCWK(reg, shdw->cfg))
+ reason = "RTC";
+
+ pr_info("AT91: Wake-Up source: %s\n", reason);
+}
+
+static void at91_poweroff(void)
+{
+ writel(AT91_SHDW_KEY | AT91_SHDW_SHDW,
+ at91_shdwc->at91_shdwc_base + AT91_SHDW_CR);
+}
+
+static u32 at91_shdwc_debouncer_value(struct platform_device *pdev,
+ u32 in_period_us)
+{
+ int i;
+ int max_idx = ARRAY_SIZE(sdwc_dbc_period) - 1;
+ unsigned long long period_us;
+ unsigned long long max_period_us = DBC_PERIOD_US(sdwc_dbc_period[max_idx]);
+
+ if (in_period_us > max_period_us) {
+ dev_warn(&pdev->dev,
+ "debouncer period %u too big, reduced to %llu us\n",
+ in_period_us, max_period_us);
+ return max_idx;
+ }
+
+ for (i = max_idx - 1; i > 0; i--) {
+ period_us = DBC_PERIOD_US(sdwc_dbc_period[i]);
+ dev_dbg(&pdev->dev, "%s: ref[%d] = %llu\n",
+ __func__, i, period_us);
+ if (in_period_us > period_us)
+ break;
+ }
+
+ return i + 1;
+}
+
+static u32 at91_shdwc_get_wakeup_input(struct platform_device *pdev,
+ struct device_node *np)
+{
+ struct device_node *cnp;
+ u32 wk_input_mask;
+ u32 wuir = 0;
+ u32 wk_input;
+
+ for_each_child_of_node(np, cnp) {
+ if (of_property_read_u32(cnp, "reg", &wk_input)) {
+ dev_warn(&pdev->dev, "reg property is missing for %s\n",
+ cnp->full_name);
+ continue;
+ }
+
+ wk_input_mask = 1 << wk_input;
+ if (!(wk_input_mask & AT91_SHDW_WKUPEN_MASK)) {
+ dev_warn(&pdev->dev,
+ "wake-up input %d out of bounds ignore\n",
+ wk_input);
+ continue;
+ }
+ wuir |= wk_input_mask;
+
+ if (of_property_read_bool(cnp, "atmel,wakeup-active-high"))
+ wuir |= AT91_SHDW_WKUPT(wk_input);
+
+ dev_dbg(&pdev->dev, "%s: (child %d) wuir = %#x\n",
+ __func__, wk_input, wuir);
+ }
+
+ return wuir;
+}
+
+static void at91_shdwc_dt_configure(struct platform_device *pdev)
+{
+ struct shdwc *shdw = platform_get_drvdata(pdev);
+ struct device_node *np = pdev->dev.of_node;
+ u32 mode = 0, tmp, input;
+
+ if (!np) {
+ dev_err(&pdev->dev, "device node not found\n");
+ return;
+ }
+
+ if (!of_property_read_u32(np, "debounce-delay-us", &tmp))
+ mode |= AT91_SHDW_WKUPDBC(at91_shdwc_debouncer_value(pdev, tmp));
+
+ if (of_property_read_bool(np, "atmel,wakeup-rtc-timer"))
+ mode |= SHDW_RTCWKEN(shdw->cfg);
+
+ dev_dbg(&pdev->dev, "%s: mode = %#x\n", __func__, mode);
+ writel(mode, shdw->at91_shdwc_base + AT91_SHDW_MR);
+
+ input = at91_shdwc_get_wakeup_input(pdev, np);
+ writel(input, shdw->at91_shdwc_base + AT91_SHDW_WUIR);
+}
+
+static const struct shdwc_config sama5d2_shdwc_config = {
+ .wkup_pin_input = 0,
+ .mr_rtcwk_shift = 17,
+ .sr_rtcwk_shift = 5,
+};
+
+static const struct of_device_id at91_shdwc_of_match[] = {
+ {
+ .compatible = "atmel,sama5d2-shdwc",
+ .data = &sama5d2_shdwc_config,
+ }, {
+ /*sentinel*/
+ }
+};
+MODULE_DEVICE_TABLE(of, at91_shdwc_of_match);
+
+static int __init at91_shdwc_probe(struct platform_device *pdev)
+{
+ struct resource *res;
+ const struct of_device_id *match;
+ int ret;
+
+ if (!pdev->dev.of_node)
+ return -ENODEV;
+
+ at91_shdwc = devm_kzalloc(&pdev->dev, sizeof(*at91_shdwc), GFP_KERNEL);
+ if (!at91_shdwc)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, at91_shdwc);
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ at91_shdwc->at91_shdwc_base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(at91_shdwc->at91_shdwc_base)) {
+ dev_err(&pdev->dev, "Could not map reset controller address\n");
+ return PTR_ERR(at91_shdwc->at91_shdwc_base);
+ }
+
+ match = of_match_node(at91_shdwc_of_match, pdev->dev.of_node);
+ at91_shdwc->cfg = (struct shdwc_config *)(match->data);
+
+ sclk = devm_clk_get(&pdev->dev, NULL);
+ if (IS_ERR(sclk))
+ return PTR_ERR(sclk);
+
+ ret = clk_prepare_enable(sclk);
+ if (ret) {
+ dev_err(&pdev->dev, "Could not enable slow clock\n");
+ return ret;
+ }
+
+ at91_wakeup_status(pdev);
+
+ at91_shdwc_dt_configure(pdev);
+
+ pm_power_off = at91_poweroff;
+
+ return 0;
+}
+
+static int __exit at91_shdwc_remove(struct platform_device *pdev)
+{
+ struct shdwc *shdw = platform_get_drvdata(pdev);
+
+ if (pm_power_off == at91_poweroff)
+ pm_power_off = NULL;
+
+ /* Reset values to disable wake-up features */
+ writel(0, shdw->at91_shdwc_base + AT91_SHDW_MR);
+ writel(0, shdw->at91_shdwc_base + AT91_SHDW_WUIR);
+
+ clk_disable_unprepare(sclk);
+
+ return 0;
+}
+
+static struct platform_driver at91_shdwc_driver = {
+ .remove = __exit_p(at91_shdwc_remove),
+ .driver = {
+ .name = "at91-shdwc",
+ .of_match_table = at91_shdwc_of_match,
+ },
+};
+module_platform_driver_probe(at91_shdwc_driver, at91_shdwc_probe);
+
+MODULE_AUTHOR("Nicolas Ferre <nicolas.ferre@atmel.com>");
+MODULE_DESCRIPTION("Atmel shutdown controller driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/power/sbs-battery.c b/drivers/power/sbs-battery.c
index d6226d6..768b9fc 100644
--- a/drivers/power/sbs-battery.c
+++ b/drivers/power/sbs-battery.c
@@ -382,8 +382,6 @@
if (ret & BATTERY_FULL_CHARGED)
val->intval = POWER_SUPPLY_STATUS_FULL;
- else if (ret & BATTERY_FULL_DISCHARGED)
- val->intval = POWER_SUPPLY_STATUS_NOT_CHARGING;
else if (ret & BATTERY_DISCHARGING)
val->intval = POWER_SUPPLY_STATUS_DISCHARGING;
else
@@ -702,8 +700,6 @@
if (ret & BATTERY_FULL_CHARGED)
ret = POWER_SUPPLY_STATUS_FULL;
- else if (ret & BATTERY_FULL_DISCHARGED)
- ret = POWER_SUPPLY_STATUS_NOT_CHARGING;
else if (ret & BATTERY_DISCHARGING)
ret = POWER_SUPPLY_STATUS_DISCHARGING;
else
diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
index 3d7d58a..db3958b 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -57,6 +57,8 @@
static const char * const rproc_crash_names[] = {
[RPROC_MMUFAULT] = "mmufault",
+ [RPROC_WATCHDOG] = "watchdog",
+ [RPROC_FATAL_ERROR] = "fatal error",
};
/* translate rproc_crash_type to string */
@@ -856,12 +858,8 @@
* copy this information to device memory.
*/
loaded_table = rproc_find_loaded_rsc_table(rproc, fw);
- if (!loaded_table) {
- ret = -EINVAL;
- goto clean_up;
- }
-
- memcpy(loaded_table, rproc->cached_table, tablesz);
+ if (loaded_table)
+ memcpy(loaded_table, rproc->cached_table, tablesz);
/* power up the remote processor */
ret = rproc->ops->start(rproc);
@@ -1030,8 +1028,9 @@
}
/**
- * rproc_boot() - boot a remote processor
+ * __rproc_boot() - boot a remote processor
* @rproc: handle of a remote processor
+ * @wait: wait for rproc registration completion
*
* Boot a remote processor (i.e. load its firmware, power it on, ...).
*
@@ -1040,7 +1039,7 @@
*
* Returns 0 on success, and an appropriate error value otherwise.
*/
-int rproc_boot(struct rproc *rproc)
+static int __rproc_boot(struct rproc *rproc, bool wait)
{
const struct firmware *firmware_p;
struct device *dev;
@@ -1088,6 +1087,10 @@
goto downref_rproc;
}
+ /* if rproc virtio is not yet configured, wait */
+ if (wait)
+ wait_for_completion(&rproc->firmware_loading_complete);
+
ret = rproc_fw_boot(rproc, firmware_p);
release_firmware(firmware_p);
@@ -1101,9 +1104,29 @@
mutex_unlock(&rproc->lock);
return ret;
}
+
+/**
+ * rproc_boot() - boot a remote processor
+ * @rproc: handle of a remote processor
+ */
+int rproc_boot(struct rproc *rproc)
+{
+ return __rproc_boot(rproc, true);
+}
EXPORT_SYMBOL(rproc_boot);
/**
+ * rproc_boot_nowait() - boot a remote processor
+ * @rproc: handle of a remote processor
+ *
+ * Same as rproc_boot() but don't wait for rproc registration completion
+ */
+int rproc_boot_nowait(struct rproc *rproc)
+{
+ return __rproc_boot(rproc, false);
+}
+
+/**
* rproc_shutdown() - power off the remote processor
* @rproc: the remote processor
*
diff --git a/drivers/remoteproc/remoteproc_internal.h b/drivers/remoteproc/remoteproc_internal.h
index 8041b95..57e1de5 100644
--- a/drivers/remoteproc/remoteproc_internal.h
+++ b/drivers/remoteproc/remoteproc_internal.h
@@ -48,6 +48,7 @@
/* from remoteproc_core.c */
void rproc_release(struct kref *kref);
irqreturn_t rproc_vq_interrupt(struct rproc *rproc, int vq_id);
+int rproc_boot_nowait(struct rproc *rproc);
/* from remoteproc_virtio.c */
int rproc_add_virtio_dev(struct rproc_vdev *rvdev, int id);
diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
index e44872f..cc91556 100644
--- a/drivers/remoteproc/remoteproc_virtio.c
+++ b/drivers/remoteproc/remoteproc_virtio.c
@@ -161,7 +161,7 @@
}
/* now that the vqs are all set, boot the remote processor */
- ret = rproc_boot(rproc);
+ ret = rproc_boot_nowait(rproc);
if (ret) {
dev_err(&rproc->dev, "rproc_boot() failed %d\n", ret);
goto error;
diff --git a/drivers/rpmsg/virtio_rpmsg_bus.c b/drivers/rpmsg/virtio_rpmsg_bus.c
index 1fcd27c..fe03b2a 100644
--- a/drivers/rpmsg/virtio_rpmsg_bus.c
+++ b/drivers/rpmsg/virtio_rpmsg_bus.c
@@ -436,17 +436,19 @@
};
/**
- * register_rpmsg_driver() - register an rpmsg driver with the rpmsg bus
+ * __register_rpmsg_driver() - register an rpmsg driver with the rpmsg bus
* @rpdrv: pointer to a struct rpmsg_driver
+ * @owner: owning module/driver
*
* Returns 0 on success, and an appropriate error value on failure.
*/
-int register_rpmsg_driver(struct rpmsg_driver *rpdrv)
+int __register_rpmsg_driver(struct rpmsg_driver *rpdrv, struct module *owner)
{
rpdrv->drv.bus = &rpmsg_bus;
+ rpdrv->drv.owner = owner;
return driver_register(&rpdrv->drv);
}
-EXPORT_SYMBOL(register_rpmsg_driver);
+EXPORT_SYMBOL(__register_rpmsg_driver);
/**
* unregister_rpmsg_driver() - unregister an rpmsg driver from the rpmsg bus
diff --git a/drivers/rtc/Kconfig b/drivers/rtc/Kconfig
index 3e84315..18639e0 100644
--- a/drivers/rtc/Kconfig
+++ b/drivers/rtc/Kconfig
@@ -573,24 +573,6 @@
This driver can also be built as a module. If so, the module
will be called rtc-em3027.
-config RTC_DRV_RV3029C2
- tristate "Micro Crystal RV3029"
- help
- If you say yes here you get support for the Micro Crystal
- RV3029 RTC chips.
-
- This driver can also be built as a module. If so, the module
- will be called rtc-rv3029c2.
-
-config RTC_DRV_RV3029_HWMON
- bool "HWMON support for RV3029"
- depends on RTC_DRV_RV3029C2 && HWMON
- depends on !(RTC_DRV_RV3029C2=y && HWMON=m)
- default y
- help
- Say Y here if you want to expose temperature sensor data on
- rtc-rv3029.
-
config RTC_DRV_RV8803
tristate "Micro Crystal RV8803"
help
@@ -634,6 +616,15 @@
This driver can also be built as a module. If so, the module
will be called rtc-m41t94.
+config RTC_DRV_DS1302
+ tristate "Dallas/Maxim DS1302"
+ depends on SPI
+ help
+ If you say yes here you get support for the Dallas DS1302 RTC chips.
+
+ This driver can also be built as a module. If so, the module
+ will be called rtc-ds1302.
+
config RTC_DRV_DS1305
tristate "Dallas/Maxim DS1305/DS1306"
help
@@ -777,6 +768,25 @@
This driver can also be built as a module. If so, the module
will be called rtc-pcf2127.
+config RTC_DRV_RV3029C2
+ tristate "Micro Crystal RV3029/3049"
+ depends on RTC_I2C_AND_SPI
+ help
+ If you say yes here you get support for the Micro Crystal
+ RV3029 and RV3049 RTC chips.
+
+ This driver can also be built as a module. If so, the module
+ will be called rtc-rv3029c2.
+
+config RTC_DRV_RV3029_HWMON
+ bool "HWMON support for RV3029/3049"
+ depends on RTC_DRV_RV3029C2 && HWMON
+ depends on !(RTC_DRV_RV3029C2=y && HWMON=m)
+ default y
+ help
+ Say Y here if you want to expose temperature sensor data on
+ rtc-rv3029.
+
comment "Platform RTC drivers"
# this 'CMOS' RTC driver is arch dependent because <asm-generic/rtc.h>
@@ -834,12 +844,6 @@
help
If you say yes here you get support for the Dallas DS1286 RTC chips.
-config RTC_DRV_DS1302
- tristate "Dallas DS1302"
- depends on SH_SECUREEDGE5410
- help
- If you say yes here you get support for the Dallas DS1302 RTC chips.
-
config RTC_DRV_DS1511
tristate "Dallas DS1511"
depends on HAS_IOMEM
diff --git a/drivers/rtc/rtc-at91sam9.c b/drivers/rtc/rtc-at91sam9.c
index 7206e2f..99732e6 100644
--- a/drivers/rtc/rtc-at91sam9.c
+++ b/drivers/rtc/rtc-at91sam9.c
@@ -268,7 +268,7 @@
static int at91_rtc_proc(struct device *dev, struct seq_file *seq)
{
struct sam9_rtc *rtc = dev_get_drvdata(dev);
- u32 mr = mr = rtt_readl(rtc, MR);
+ u32 mr = rtt_readl(rtc, MR);
seq_printf(seq, "update_IRQ\t: %s\n",
(mr & AT91_RTT_RTTINCIEN) ? "yes" : "no");
diff --git a/drivers/rtc/rtc-cmos.c b/drivers/rtc/rtc-cmos.c
index 84fb541..fbe9c72 100644
--- a/drivers/rtc/rtc-cmos.c
+++ b/drivers/rtc/rtc-cmos.c
@@ -401,7 +401,7 @@
return 0;
}
-#if defined(CONFIG_RTC_INTF_PROC) || defined(CONFIG_RTC_INTF_PROC_MODULE)
+#if IS_ENABLED(CONFIG_RTC_INTF_PROC)
static int cmos_procfs(struct device *dev, struct seq_file *seq)
{
diff --git a/drivers/rtc/rtc-da9052.c b/drivers/rtc/rtc-da9052.c
index 1ba4371..a20bcf0 100644
--- a/drivers/rtc/rtc-da9052.c
+++ b/drivers/rtc/rtc-da9052.c
@@ -302,6 +302,13 @@
if (ret != 0)
rtc_err(rtc, "Failed to disable TICKS: %d\n", ret);
+ device_init_wakeup(&pdev->dev, true);
+ rtc->rtc = devm_rtc_device_register(&pdev->dev, pdev->name,
+ &da9052_rtc_ops, THIS_MODULE);
+
+ if (IS_ERR(rtc->rtc))
+ return PTR_ERR(rtc->rtc);
+
ret = da9052_request_irq(rtc->da9052, DA9052_IRQ_ALARM, "ALM",
da9052_rtc_irq, rtc);
if (ret != 0) {
@@ -309,11 +316,7 @@
return ret;
}
- device_init_wakeup(&pdev->dev, true);
-
- rtc->rtc = devm_rtc_device_register(&pdev->dev, pdev->name,
- &da9052_rtc_ops, THIS_MODULE);
- return PTR_ERR_OR_ZERO(rtc->rtc);
+ return 0;
}
static struct platform_driver da9052_rtc_driver = {
diff --git a/drivers/rtc/rtc-ds1216.c b/drivers/rtc/rtc-ds1216.c
index 12dbd70..9c82b1d 100644
--- a/drivers/rtc/rtc-ds1216.c
+++ b/drivers/rtc/rtc-ds1216.c
@@ -11,8 +11,6 @@
#include <linux/bcd.h>
#include <linux/slab.h>
-#define DRV_VERSION "0.2"
-
struct ds1216_regs {
u8 tsec;
u8 sec;
@@ -176,5 +174,4 @@
MODULE_AUTHOR("Thomas Bogendoerfer <tsbogend@alpha.franken.de>");
MODULE_DESCRIPTION("DS1216 RTC driver");
MODULE_LICENSE("GPL");
-MODULE_VERSION(DRV_VERSION);
MODULE_ALIAS("platform:rtc-ds1216");
diff --git a/drivers/rtc/rtc-ds1286.c b/drivers/rtc/rtc-ds1286.c
index 8247a29..756e509f 100644
--- a/drivers/rtc/rtc-ds1286.c
+++ b/drivers/rtc/rtc-ds1286.c
@@ -20,8 +20,6 @@
#include <linux/io.h>
#include <linux/slab.h>
-#define DRV_VERSION "1.0"
-
struct ds1286_priv {
struct rtc_device *rtc;
u32 __iomem *rtcregs;
@@ -363,5 +361,4 @@
MODULE_AUTHOR("Thomas Bogendoerfer <tsbogend@alpha.franken.de>");
MODULE_DESCRIPTION("DS1286 RTC driver");
MODULE_LICENSE("GPL");
-MODULE_VERSION(DRV_VERSION);
MODULE_ALIAS("platform:rtc-ds1286");
diff --git a/drivers/rtc/rtc-ds1302.c b/drivers/rtc/rtc-ds1302.c
index 6bef7a5..f5dd09f 100644
--- a/drivers/rtc/rtc-ds1302.c
+++ b/drivers/rtc/rtc-ds1302.c
@@ -9,16 +9,16 @@
* this archive for more details.
*/
-#include <linux/init.h>
-#include <linux/module.h>
-#include <linux/kernel.h>
-#include <linux/platform_device.h>
-#include <linux/rtc.h>
-#include <linux/io.h>
#include <linux/bcd.h>
+#include <linux/init.h>
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/rtc.h>
+#include <linux/spi/spi.h>
#define DRV_NAME "rtc-ds1302"
-#define DRV_VERSION "0.1.1"
#define RTC_CMD_READ 0x81 /* Read command */
#define RTC_CMD_WRITE 0x80 /* Write command */
@@ -28,6 +28,8 @@
#define RTC_ADDR_RAM0 0x20 /* Address of RAM0 */
#define RTC_ADDR_TCR 0x08 /* Address of trickle charge register */
+#define RTC_CLCK_BURST 0x1F /* Address of clock burst */
+#define RTC_CLCK_LEN 0x08 /* Size of clock burst */
#define RTC_ADDR_CTRL 0x07 /* Address of control register */
#define RTC_ADDR_YEAR 0x06 /* Address of year register */
#define RTC_ADDR_DAY 0x05 /* Address of day of week register */
@@ -37,219 +39,181 @@
#define RTC_ADDR_MIN 0x01 /* Address of minute register */
#define RTC_ADDR_SEC 0x00 /* Address of second register */
-#ifdef CONFIG_SH_SECUREEDGE5410
-#include <asm/rtc.h>
-#include <mach/secureedge5410.h>
-
-#define RTC_RESET 0x1000
-#define RTC_IODATA 0x0800
-#define RTC_SCLK 0x0400
-
-#define set_dp(x) SECUREEDGE_WRITE_IOPORT(x, 0x1c00)
-#define get_dp() SECUREEDGE_READ_IOPORT()
-#define ds1302_set_tx()
-#define ds1302_set_rx()
-
-static inline int ds1302_hw_init(void)
+static int ds1302_rtc_set_time(struct device *dev, struct rtc_time *time)
{
- return 0;
+ struct spi_device *spi = dev_get_drvdata(dev);
+ u8 buf[1 + RTC_CLCK_LEN];
+ u8 *bp = buf;
+ int status;
+
+ /* Enable writing */
+ bp = buf;
+ *bp++ = RTC_ADDR_CTRL << 1 | RTC_CMD_WRITE;
+ *bp++ = RTC_CMD_WRITE_ENABLE;
+
+ status = spi_write_then_read(spi, buf, 2,
+ NULL, 0);
+ if (status)
+ return status;
+
+ /* Write registers starting at the first time/date address. */
+ bp = buf;
+ *bp++ = RTC_CLCK_BURST << 1 | RTC_CMD_WRITE;
+
+ *bp++ = bin2bcd(time->tm_sec);
+ *bp++ = bin2bcd(time->tm_min);
+ *bp++ = bin2bcd(time->tm_hour);
+ *bp++ = bin2bcd(time->tm_mday);
+ *bp++ = bin2bcd(time->tm_mon + 1);
+ *bp++ = time->tm_wday + 1;
+ *bp++ = bin2bcd(time->tm_year % 100);
+ *bp++ = RTC_CMD_WRITE_DISABLE;
+
+ /* use write-then-read since dma from stack is nonportable */
+ return spi_write_then_read(spi, buf, sizeof(buf),
+ NULL, 0);
}
-static inline void ds1302_reset(void)
+static int ds1302_rtc_get_time(struct device *dev, struct rtc_time *time)
{
- set_dp(get_dp() & ~(RTC_RESET | RTC_IODATA | RTC_SCLK));
-}
+ struct spi_device *spi = dev_get_drvdata(dev);
+ u8 addr = RTC_CLCK_BURST << 1 | RTC_CMD_READ;
+ u8 buf[RTC_CLCK_LEN - 1];
+ int status;
-static inline void ds1302_clock(void)
-{
- set_dp(get_dp() | RTC_SCLK); /* clock high */
- set_dp(get_dp() & ~RTC_SCLK); /* clock low */
-}
+ /* Use write-then-read to get all the date/time registers
+ * since dma from stack is nonportable
+ */
+ status = spi_write_then_read(spi, &addr, sizeof(addr),
+ buf, sizeof(buf));
+ if (status < 0)
+ return status;
-static inline void ds1302_start(void)
-{
- set_dp(get_dp() | RTC_RESET);
-}
+ /* Decode the registers */
+ time->tm_sec = bcd2bin(buf[RTC_ADDR_SEC]);
+ time->tm_min = bcd2bin(buf[RTC_ADDR_MIN]);
+ time->tm_hour = bcd2bin(buf[RTC_ADDR_HOUR]);
+ time->tm_wday = buf[RTC_ADDR_DAY] - 1;
+ time->tm_mday = bcd2bin(buf[RTC_ADDR_DATE]);
+ time->tm_mon = bcd2bin(buf[RTC_ADDR_MON]) - 1;
+ time->tm_year = bcd2bin(buf[RTC_ADDR_YEAR]) + 100;
-static inline void ds1302_stop(void)
-{
- set_dp(get_dp() & ~RTC_RESET);
-}
-
-static inline void ds1302_txbit(int bit)
-{
- set_dp((get_dp() & ~RTC_IODATA) | (bit ? RTC_IODATA : 0));
-}
-
-static inline int ds1302_rxbit(void)
-{
- return !!(get_dp() & RTC_IODATA);
-}
-
-#else
-#error "Add support for your platform"
-#endif
-
-static void ds1302_sendbits(unsigned int val)
-{
- int i;
-
- ds1302_set_tx();
-
- for (i = 8; (i); i--, val >>= 1) {
- ds1302_txbit(val & 0x1);
- ds1302_clock();
- }
-}
-
-static unsigned int ds1302_recvbits(void)
-{
- unsigned int val;
- int i;
-
- ds1302_set_rx();
-
- for (i = 0, val = 0; (i < 8); i++) {
- val |= (ds1302_rxbit() << i);
- ds1302_clock();
- }
-
- return val;
-}
-
-static unsigned int ds1302_readbyte(unsigned int addr)
-{
- unsigned int val;
-
- ds1302_reset();
-
- ds1302_start();
- ds1302_sendbits(((addr & 0x3f) << 1) | RTC_CMD_READ);
- val = ds1302_recvbits();
- ds1302_stop();
-
- return val;
-}
-
-static void ds1302_writebyte(unsigned int addr, unsigned int val)
-{
- ds1302_reset();
-
- ds1302_start();
- ds1302_sendbits(((addr & 0x3f) << 1) | RTC_CMD_WRITE);
- ds1302_sendbits(val);
- ds1302_stop();
-}
-
-static int ds1302_rtc_read_time(struct device *dev, struct rtc_time *tm)
-{
- tm->tm_sec = bcd2bin(ds1302_readbyte(RTC_ADDR_SEC));
- tm->tm_min = bcd2bin(ds1302_readbyte(RTC_ADDR_MIN));
- tm->tm_hour = bcd2bin(ds1302_readbyte(RTC_ADDR_HOUR));
- tm->tm_wday = bcd2bin(ds1302_readbyte(RTC_ADDR_DAY));
- tm->tm_mday = bcd2bin(ds1302_readbyte(RTC_ADDR_DATE));
- tm->tm_mon = bcd2bin(ds1302_readbyte(RTC_ADDR_MON)) - 1;
- tm->tm_year = bcd2bin(ds1302_readbyte(RTC_ADDR_YEAR));
-
- if (tm->tm_year < 70)
- tm->tm_year += 100;
-
- dev_dbg(dev, "%s: tm is secs=%d, mins=%d, hours=%d, "
- "mday=%d, mon=%d, year=%d, wday=%d\n",
- __func__,
- tm->tm_sec, tm->tm_min, tm->tm_hour,
- tm->tm_mday, tm->tm_mon + 1, tm->tm_year, tm->tm_wday);
-
- return rtc_valid_tm(tm);
-}
-
-static int ds1302_rtc_set_time(struct device *dev, struct rtc_time *tm)
-{
- ds1302_writebyte(RTC_ADDR_CTRL, RTC_CMD_WRITE_ENABLE);
- /* Stop RTC */
- ds1302_writebyte(RTC_ADDR_SEC, ds1302_readbyte(RTC_ADDR_SEC) | 0x80);
-
- ds1302_writebyte(RTC_ADDR_SEC, bin2bcd(tm->tm_sec));
- ds1302_writebyte(RTC_ADDR_MIN, bin2bcd(tm->tm_min));
- ds1302_writebyte(RTC_ADDR_HOUR, bin2bcd(tm->tm_hour));
- ds1302_writebyte(RTC_ADDR_DAY, bin2bcd(tm->tm_wday));
- ds1302_writebyte(RTC_ADDR_DATE, bin2bcd(tm->tm_mday));
- ds1302_writebyte(RTC_ADDR_MON, bin2bcd(tm->tm_mon + 1));
- ds1302_writebyte(RTC_ADDR_YEAR, bin2bcd(tm->tm_year % 100));
-
- /* Start RTC */
- ds1302_writebyte(RTC_ADDR_SEC, ds1302_readbyte(RTC_ADDR_SEC) & ~0x80);
-
- ds1302_writebyte(RTC_ADDR_CTRL, RTC_CMD_WRITE_DISABLE);
-
- return 0;
-}
-
-static int ds1302_rtc_ioctl(struct device *dev, unsigned int cmd,
- unsigned long arg)
-{
- switch (cmd) {
-#ifdef RTC_SET_CHARGE
- case RTC_SET_CHARGE:
- {
- int tcs_val;
-
- if (copy_from_user(&tcs_val, (int __user *)arg, sizeof(int)))
- return -EFAULT;
-
- ds1302_writebyte(RTC_ADDR_TCR, (0xa0 | tcs_val * 0xf));
- return 0;
- }
-#endif
- }
-
- return -ENOIOCTLCMD;
+ /* Time may not be set */
+ return rtc_valid_tm(time);
}
static struct rtc_class_ops ds1302_rtc_ops = {
- .read_time = ds1302_rtc_read_time,
+ .read_time = ds1302_rtc_get_time,
.set_time = ds1302_rtc_set_time,
- .ioctl = ds1302_rtc_ioctl,
};
-static int __init ds1302_rtc_probe(struct platform_device *pdev)
+static int ds1302_probe(struct spi_device *spi)
{
- struct rtc_device *rtc;
+ struct rtc_device *rtc;
+ u8 addr;
+ u8 buf[4];
+ u8 *bp = buf;
+ int status;
- if (ds1302_hw_init()) {
- dev_err(&pdev->dev, "Failed to init communication channel");
+ /* Sanity check board setup data. This may be hooked up
+ * in 3wire mode, but we don't care. Note that unless
+ * there's an inverter in place, this needs SPI_CS_HIGH!
+ */
+ if (spi->bits_per_word && (spi->bits_per_word != 8)) {
+ dev_err(&spi->dev, "bad word length\n");
+ return -EINVAL;
+ } else if (spi->max_speed_hz > 2000000) {
+ dev_err(&spi->dev, "speed is too high\n");
+ return -EINVAL;
+ } else if (spi->mode & SPI_CPHA) {
+ dev_err(&spi->dev, "bad mode\n");
return -EINVAL;
}
- /* Reset */
- ds1302_reset();
-
- /* Write a magic value to the DS1302 RAM, and see if it sticks. */
- ds1302_writebyte(RTC_ADDR_RAM0, 0x42);
- if (ds1302_readbyte(RTC_ADDR_RAM0) != 0x42) {
- dev_err(&pdev->dev, "Failed to probe");
- return -ENODEV;
+ addr = RTC_ADDR_CTRL << 1 | RTC_CMD_READ;
+ status = spi_write_then_read(spi, &addr, sizeof(addr), buf, 1);
+ if (status < 0) {
+ dev_err(&spi->dev, "control register read error %d\n",
+ status);
+ return status;
}
- rtc = devm_rtc_device_register(&pdev->dev, "ds1302",
- &ds1302_rtc_ops, THIS_MODULE);
- if (IS_ERR(rtc))
- return PTR_ERR(rtc);
+ if ((buf[0] & ~RTC_CMD_WRITE_DISABLE) != 0) {
+ status = spi_write_then_read(spi, &addr, sizeof(addr), buf, 1);
+ if (status < 0) {
+ dev_err(&spi->dev, "control register read error %d\n",
+ status);
+ return status;
+ }
- platform_set_drvdata(pdev, rtc);
+ if ((buf[0] & ~RTC_CMD_WRITE_DISABLE) != 0) {
+ dev_err(&spi->dev, "junk in control register\n");
+ return -ENODEV;
+ }
+ }
+ if (buf[0] == 0) {
+ bp = buf;
+ *bp++ = RTC_ADDR_CTRL << 1 | RTC_CMD_WRITE;
+ *bp++ = RTC_CMD_WRITE_DISABLE;
+
+ status = spi_write_then_read(spi, buf, 2, NULL, 0);
+ if (status < 0) {
+ dev_err(&spi->dev, "control register write error %d\n",
+ status);
+ return status;
+ }
+
+ addr = RTC_ADDR_CTRL << 1 | RTC_CMD_READ;
+ status = spi_write_then_read(spi, &addr, sizeof(addr), buf, 1);
+ if (status < 0) {
+ dev_err(&spi->dev,
+ "error %d reading control register\n",
+ status);
+ return status;
+ }
+
+ if (buf[0] != RTC_CMD_WRITE_DISABLE) {
+ dev_err(&spi->dev, "failed to detect chip\n");
+ return -ENODEV;
+ }
+ }
+
+ spi_set_drvdata(spi, spi);
+
+ rtc = devm_rtc_device_register(&spi->dev, "ds1302",
+ &ds1302_rtc_ops, THIS_MODULE);
+ if (IS_ERR(rtc)) {
+ status = PTR_ERR(rtc);
+ dev_err(&spi->dev, "error %d registering rtc\n", status);
+ return status;
+ }
return 0;
}
-static struct platform_driver ds1302_platform_driver = {
- .driver = {
- .name = DRV_NAME,
- },
+static int ds1302_remove(struct spi_device *spi)
+{
+ spi_set_drvdata(spi, NULL);
+ return 0;
+}
+
+#ifdef CONFIG_OF
+static const struct of_device_id ds1302_dt_ids[] = {
+ { .compatible = "maxim,ds1302", },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, ds1302_dt_ids);
+#endif
+
+static struct spi_driver ds1302_driver = {
+ .driver.name = "rtc-ds1302",
+ .driver.of_match_table = of_match_ptr(ds1302_dt_ids),
+ .probe = ds1302_probe,
+ .remove = ds1302_remove,
};
-module_platform_driver_probe(ds1302_platform_driver, ds1302_rtc_probe);
+module_spi_driver(ds1302_driver);
MODULE_DESCRIPTION("Dallas DS1302 RTC driver");
-MODULE_VERSION(DRV_VERSION);
MODULE_AUTHOR("Paul Mundt, David McCullough");
MODULE_LICENSE("GPL v2");
diff --git a/drivers/rtc/rtc-ds1307.c b/drivers/rtc/rtc-ds1307.c
index ecb7dba..821d9c0 100644
--- a/drivers/rtc/rtc-ds1307.c
+++ b/drivers/rtc/rtc-ds1307.c
@@ -275,9 +275,13 @@
{
u8 suboffset = 0;
- if (length <= I2C_SMBUS_BLOCK_MAX)
- return i2c_smbus_write_i2c_block_data(client,
+ if (length <= I2C_SMBUS_BLOCK_MAX) {
+ s32 retval = i2c_smbus_write_i2c_block_data(client,
command, length, values);
+ if (retval < 0)
+ return retval;
+ return length;
+ }
while (suboffset < length) {
s32 retval = i2c_smbus_write_i2c_block_data(client,
@@ -538,12 +542,8 @@
buf[5] = 0;
buf[6] = 0;
- /* optionally enable ALARM1 */
+ /* disable alarms */
buf[7] = control & ~(DS1337_BIT_A1IE | DS1337_BIT_A2IE);
- if (t->enabled) {
- dev_dbg(dev, "alarm IRQ armed\n");
- buf[7] |= DS1337_BIT_A1IE; /* only ALARM1 is used */
- }
buf[8] = status & ~(DS1337_BIT_A1I | DS1337_BIT_A2I);
ret = ds1307->write_block_data(client,
@@ -553,6 +553,13 @@
return ret;
}
+ /* optionally enable ALARM1 */
+ if (t->enabled) {
+ dev_dbg(dev, "alarm IRQ armed\n");
+ buf[7] |= DS1337_BIT_A1IE; /* only ALARM1 is used */
+ i2c_smbus_write_byte_data(client, DS1337_REG_CONTROL, buf[7]);
+ }
+
return 0;
}
@@ -1144,12 +1151,10 @@
[DS3231_CLK_SQW] = {
.name = "ds3231_clk_sqw",
.ops = &ds3231_clk_sqw_ops,
- .flags = CLK_IS_ROOT,
},
[DS3231_CLK_32KHZ] = {
.name = "ds3231_clk_32khz",
.ops = &ds3231_clk_32khz_ops,
- .flags = CLK_IS_ROOT,
},
};
diff --git a/drivers/rtc/rtc-ds1343.c b/drivers/rtc/rtc-ds1343.c
index 3d389bd..23fa9f0 100644
--- a/drivers/rtc/rtc-ds1343.c
+++ b/drivers/rtc/rtc-ds1343.c
@@ -24,7 +24,6 @@
#include <linux/pm_wakeirq.h>
#include <linux/slab.h>
-#define DS1343_DRV_VERSION "01.00"
#define DALLAS_MAXIM_DS1343 0
#define DALLAS_MAXIM_DS1344 1
@@ -747,4 +746,3 @@
MODULE_AUTHOR("Raghavendra Chandra Ganiga <ravi23ganiga@gmail.com>,"
"Ankur Srivastava <sankurece@gmail.com>");
MODULE_LICENSE("GPL v2");
-MODULE_VERSION(DS1343_DRV_VERSION);
diff --git a/drivers/rtc/rtc-ds1511.c b/drivers/rtc/rtc-ds1511.c
index da3d04c..1b2dcb5 100644
--- a/drivers/rtc/rtc-ds1511.c
+++ b/drivers/rtc/rtc-ds1511.c
@@ -25,8 +25,6 @@
#include <linux/io.h>
#include <linux/module.h>
-#define DRV_VERSION "0.6"
-
enum ds1511reg {
DS1511_SEC = 0x0,
DS1511_MIN = 0x1,
@@ -537,4 +535,3 @@
MODULE_AUTHOR("Andrew Sharp <andy.sharp@lsi.com>");
MODULE_DESCRIPTION("Dallas DS1511 RTC driver");
MODULE_LICENSE("GPL");
-MODULE_VERSION(DRV_VERSION);
diff --git a/drivers/rtc/rtc-ds1553.c b/drivers/rtc/rtc-ds1553.c
index 38422ab4..9961ec6 100644
--- a/drivers/rtc/rtc-ds1553.c
+++ b/drivers/rtc/rtc-ds1553.c
@@ -20,8 +20,6 @@
#include <linux/io.h>
#include <linux/module.h>
-#define DRV_VERSION "0.3"
-
#define RTC_REG_SIZE 0x2000
#define RTC_OFFSET 0x1ff0
@@ -359,4 +357,3 @@
MODULE_AUTHOR("Atsushi Nemoto <anemo@mba.ocn.ne.jp>");
MODULE_DESCRIPTION("Dallas DS1553 RTC driver");
MODULE_LICENSE("GPL");
-MODULE_VERSION(DRV_VERSION);
diff --git a/drivers/rtc/rtc-ds1672.c b/drivers/rtc/rtc-ds1672.c
index 92b1cbf..5c18ac7 100644
--- a/drivers/rtc/rtc-ds1672.c
+++ b/drivers/rtc/rtc-ds1672.c
@@ -13,8 +13,6 @@
#include <linux/rtc.h>
#include <linux/module.h>
-#define DRV_VERSION "0.4"
-
/* Registers */
#define DS1672_REG_CNT_BASE 0
@@ -165,8 +163,6 @@
if (!i2c_check_functionality(client->adapter, I2C_FUNC_I2C))
return -ENODEV;
- dev_info(&client->dev, "chip found, driver version " DRV_VERSION "\n");
-
rtc = devm_rtc_device_register(&client->dev, ds1672_driver.driver.name,
&ds1672_rtc_ops, THIS_MODULE);
@@ -213,4 +209,3 @@
MODULE_AUTHOR("Alessandro Zummo <a.zummo@towertech.it>");
MODULE_DESCRIPTION("Dallas/Maxim DS1672 timekeeper driver");
MODULE_LICENSE("GPL");
-MODULE_VERSION(DRV_VERSION);
diff --git a/drivers/rtc/rtc-ds1685.c b/drivers/rtc/rtc-ds1685.c
index 1e6cfc8..b3ce3c6 100644
--- a/drivers/rtc/rtc-ds1685.c
+++ b/drivers/rtc/rtc-ds1685.c
@@ -32,8 +32,6 @@
#include <linux/proc_fs.h>
#endif
-#define DRV_VERSION "0.42.0"
-
/* ----------------------------------------------------------------------- */
/* Standard read/write functions if platform does not provide overrides */
@@ -2213,6 +2211,7 @@
(ctrl4a | RTC_CTRL_4A_PAB));
/* Spin ... we do not switch back to bank0. */
+ while(1);
unreachable();
}
}
@@ -2224,5 +2223,4 @@
MODULE_AUTHOR("Matthias Fuchs <matthias.fuchs@esd-electronics.com>");
MODULE_DESCRIPTION("Dallas/Maxim DS1685/DS1687-series RTC driver");
MODULE_LICENSE("GPL");
-MODULE_VERSION(DRV_VERSION);
MODULE_ALIAS("platform:rtc-ds1685");
diff --git a/drivers/rtc/rtc-ds1742.c b/drivers/rtc/rtc-ds1742.c
index c5168b3..3abf1cb 100644
--- a/drivers/rtc/rtc-ds1742.c
+++ b/drivers/rtc/rtc-ds1742.c
@@ -24,8 +24,6 @@
#include <linux/io.h>
#include <linux/module.h>
-#define DRV_VERSION "0.4"
-
#define RTC_SIZE 8
#define RTC_CONTROL 0
@@ -239,5 +237,4 @@
MODULE_AUTHOR("Atsushi Nemoto <anemo@mba.ocn.ne.jp>");
MODULE_DESCRIPTION("Dallas DS1742 RTC driver");
MODULE_LICENSE("GPL");
-MODULE_VERSION(DRV_VERSION);
MODULE_ALIAS("platform:rtc-ds1742");
diff --git a/drivers/rtc/rtc-ds3232.c b/drivers/rtc/rtc-ds3232.c
index 7edc889..04fbd7f 100644
--- a/drivers/rtc/rtc-ds3232.c
+++ b/drivers/rtc/rtc-ds3232.c
@@ -369,6 +369,11 @@
if (ret)
return ret;
+ ds3232->rtc = devm_rtc_device_register(dev, name, &ds3232_rtc_ops,
+ THIS_MODULE);
+ if (IS_ERR(ds3232->rtc))
+ return PTR_ERR(ds3232->rtc);
+
if (ds3232->irq > 0) {
ret = devm_request_threaded_irq(dev, ds3232->irq, NULL,
ds3232_irq,
@@ -380,10 +385,8 @@
} else
device_init_wakeup(dev, 1);
}
- ds3232->rtc = devm_rtc_device_register(dev, name, &ds3232_rtc_ops,
- THIS_MODULE);
- return PTR_ERR_OR_ZERO(ds3232->rtc);
+ return 0;
}
#ifdef CONFIG_PM_SLEEP
diff --git a/drivers/rtc/rtc-ep93xx.c b/drivers/rtc/rtc-ep93xx.c
index a1628ad..6940382 100644
--- a/drivers/rtc/rtc-ep93xx.c
+++ b/drivers/rtc/rtc-ep93xx.c
@@ -28,8 +28,6 @@
#define EP93XX_RTC_SWCOMP_INT_MASK 0x0000ffff
#define EP93XX_RTC_SWCOMP_INT_SHIFT 0
-#define DRV_VERSION "0.3"
-
/*
* struct device dev.platform_data is used to store our private data
* because struct rtc_device does not have a variable to hold it.
@@ -184,5 +182,4 @@
MODULE_AUTHOR("Alessandro Zummo <a.zummo@towertech.it>");
MODULE_DESCRIPTION("EP93XX RTC driver");
MODULE_LICENSE("GPL");
-MODULE_VERSION(DRV_VERSION);
MODULE_ALIAS("platform:ep93xx-rtc");
diff --git a/drivers/rtc/rtc-gemini.c b/drivers/rtc/rtc-gemini.c
index f46b6d4..b57505e 100644
--- a/drivers/rtc/rtc-gemini.c
+++ b/drivers/rtc/rtc-gemini.c
@@ -28,7 +28,6 @@
#include <linux/module.h>
#define DRV_NAME "rtc-gemini"
-#define DRV_VERSION "0.2"
MODULE_AUTHOR("Hans Ulli Kroll <ulli.kroll@googlemail.com>");
MODULE_DESCRIPTION("RTC driver for Gemini SoC");
diff --git a/drivers/rtc/rtc-hym8563.c b/drivers/rtc/rtc-hym8563.c
index b1b4746..2072703 100644
--- a/drivers/rtc/rtc-hym8563.c
+++ b/drivers/rtc/rtc-hym8563.c
@@ -413,7 +413,7 @@
init.name = "hym8563-clkout";
init.ops = &hym8563_clkout_ops;
- init.flags = CLK_IS_ROOT;
+ init.flags = 0;
init.parent_names = NULL;
init.num_parents = 0;
hym8563->clkout_hw.init = &init;
diff --git a/drivers/rtc/rtc-isl12022.c b/drivers/rtc/rtc-isl12022.c
index 839d1fd..38586a0 100644
--- a/drivers/rtc/rtc-isl12022.c
+++ b/drivers/rtc/rtc-isl12022.c
@@ -20,8 +20,6 @@
#include <linux/of.h>
#include <linux/of_device.h>
-#define DRV_VERSION "0.1"
-
/* ISL register offsets */
#define ISL12022_REG_SC 0x00
#define ISL12022_REG_MN 0x01
@@ -258,8 +256,6 @@
if (!isl12022)
return -ENOMEM;
- dev_dbg(&client->dev, "chip found, driver version " DRV_VERSION "\n");
-
i2c_set_clientdata(client, isl12022);
isl12022->rtc = devm_rtc_device_register(&client->dev,
@@ -299,4 +295,3 @@
MODULE_AUTHOR("roman.fietze@telemotive.de");
MODULE_DESCRIPTION("ISL 12022 RTC driver");
MODULE_LICENSE("GPL");
-MODULE_VERSION(DRV_VERSION);
diff --git a/drivers/rtc/rtc-isl1208.c b/drivers/rtc/rtc-isl1208.c
index b57a304..2893785 100644
--- a/drivers/rtc/rtc-isl1208.c
+++ b/drivers/rtc/rtc-isl1208.c
@@ -15,8 +15,6 @@
#include <linux/bcd.h>
#include <linux/rtc.h>
-#define DRV_VERSION "0.3"
-
/* Register map */
/* rtc section */
#define ISL1208_REG_SC 0x00
@@ -632,9 +630,6 @@
if (isl1208_i2c_validate_client(client) < 0)
return -ENODEV;
- dev_info(&client->dev,
- "chip found, driver version " DRV_VERSION "\n");
-
if (client->irq > 0) {
rc = devm_request_threaded_irq(&client->dev, client->irq, NULL,
isl1208_rtc_interrupt,
@@ -706,4 +701,3 @@
MODULE_AUTHOR("Herbert Valerio Riedel <hvr@gnu.org>");
MODULE_DESCRIPTION("Intersil ISL1208 RTC driver");
MODULE_LICENSE("GPL");
-MODULE_VERSION(DRV_VERSION);
diff --git a/drivers/rtc/rtc-m41t80.c b/drivers/rtc/rtc-m41t80.c
index d107a8e..d1bf93a 100644
--- a/drivers/rtc/rtc-m41t80.c
+++ b/drivers/rtc/rtc-m41t80.c
@@ -32,41 +32,42 @@
#include <linux/watchdog.h>
#endif
-#define M41T80_REG_SSEC 0
-#define M41T80_REG_SEC 1
-#define M41T80_REG_MIN 2
-#define M41T80_REG_HOUR 3
-#define M41T80_REG_WDAY 4
-#define M41T80_REG_DAY 5
-#define M41T80_REG_MON 6
-#define M41T80_REG_YEAR 7
-#define M41T80_REG_ALARM_MON 0xa
-#define M41T80_REG_ALARM_DAY 0xb
-#define M41T80_REG_ALARM_HOUR 0xc
-#define M41T80_REG_ALARM_MIN 0xd
-#define M41T80_REG_ALARM_SEC 0xe
-#define M41T80_REG_FLAGS 0xf
-#define M41T80_REG_SQW 0x13
+#define M41T80_REG_SSEC 0x00
+#define M41T80_REG_SEC 0x01
+#define M41T80_REG_MIN 0x02
+#define M41T80_REG_HOUR 0x03
+#define M41T80_REG_WDAY 0x04
+#define M41T80_REG_DAY 0x05
+#define M41T80_REG_MON 0x06
+#define M41T80_REG_YEAR 0x07
+#define M41T80_REG_ALARM_MON 0x0a
+#define M41T80_REG_ALARM_DAY 0x0b
+#define M41T80_REG_ALARM_HOUR 0x0c
+#define M41T80_REG_ALARM_MIN 0x0d
+#define M41T80_REG_ALARM_SEC 0x0e
+#define M41T80_REG_FLAGS 0x0f
+#define M41T80_REG_SQW 0x13
#define M41T80_DATETIME_REG_SIZE (M41T80_REG_YEAR + 1)
#define M41T80_ALARM_REG_SIZE \
(M41T80_REG_ALARM_SEC + 1 - M41T80_REG_ALARM_MON)
-#define M41T80_SEC_ST (1 << 7) /* ST: Stop Bit */
-#define M41T80_ALMON_AFE (1 << 7) /* AFE: AF Enable Bit */
-#define M41T80_ALMON_SQWE (1 << 6) /* SQWE: SQW Enable Bit */
-#define M41T80_ALHOUR_HT (1 << 6) /* HT: Halt Update Bit */
-#define M41T80_FLAGS_AF (1 << 6) /* AF: Alarm Flag Bit */
-#define M41T80_FLAGS_BATT_LOW (1 << 4) /* BL: Battery Low Bit */
-#define M41T80_WATCHDOG_RB2 (1 << 7) /* RB: Watchdog resolution */
-#define M41T80_WATCHDOG_RB1 (1 << 1) /* RB: Watchdog resolution */
-#define M41T80_WATCHDOG_RB0 (1 << 0) /* RB: Watchdog resolution */
+#define M41T80_SEC_ST BIT(7) /* ST: Stop Bit */
+#define M41T80_ALMON_AFE BIT(7) /* AFE: AF Enable Bit */
+#define M41T80_ALMON_SQWE BIT(6) /* SQWE: SQW Enable Bit */
+#define M41T80_ALHOUR_HT BIT(6) /* HT: Halt Update Bit */
+#define M41T80_FLAGS_OF BIT(2) /* OF: Oscillator Failure Bit */
+#define M41T80_FLAGS_AF BIT(6) /* AF: Alarm Flag Bit */
+#define M41T80_FLAGS_BATT_LOW BIT(4) /* BL: Battery Low Bit */
+#define M41T80_WATCHDOG_RB2 BIT(7) /* RB: Watchdog resolution */
+#define M41T80_WATCHDOG_RB1 BIT(1) /* RB: Watchdog resolution */
+#define M41T80_WATCHDOG_RB0 BIT(0) /* RB: Watchdog resolution */
-#define M41T80_FEATURE_HT (1 << 0) /* Halt feature */
-#define M41T80_FEATURE_BL (1 << 1) /* Battery low indicator */
-#define M41T80_FEATURE_SQ (1 << 2) /* Squarewave feature */
-#define M41T80_FEATURE_WD (1 << 3) /* Extra watchdog resolution */
-#define M41T80_FEATURE_SQ_ALT (1 << 4) /* RSx bits are in reg 4 */
+#define M41T80_FEATURE_HT BIT(0) /* Halt feature */
+#define M41T80_FEATURE_BL BIT(1) /* Battery low indicator */
+#define M41T80_FEATURE_SQ BIT(2) /* Squarewave feature */
+#define M41T80_FEATURE_WD BIT(3) /* Extra watchdog resolution */
+#define M41T80_FEATURE_SQ_ALT BIT(4) /* RSx bits are in reg 4 */
static DEFINE_MUTEX(m41t80_rtc_mutex);
static const struct i2c_device_id m41t80_id[] = {
@@ -90,27 +91,65 @@
struct rtc_device *rtc;
};
+static irqreturn_t m41t80_handle_irq(int irq, void *dev_id)
+{
+ struct i2c_client *client = dev_id;
+ struct m41t80_data *m41t80 = i2c_get_clientdata(client);
+ struct mutex *lock = &m41t80->rtc->ops_lock;
+ unsigned long events = 0;
+ int flags, flags_afe;
+
+ mutex_lock(lock);
+
+ flags_afe = i2c_smbus_read_byte_data(client, M41T80_REG_ALARM_MON);
+ if (flags_afe < 0) {
+ mutex_unlock(lock);
+ return IRQ_NONE;
+ }
+
+ flags = i2c_smbus_read_byte_data(client, M41T80_REG_FLAGS);
+ if (flags <= 0) {
+ mutex_unlock(lock);
+ return IRQ_NONE;
+ }
+
+ if (flags & M41T80_FLAGS_AF) {
+ flags &= ~M41T80_FLAGS_AF;
+ flags_afe &= ~M41T80_ALMON_AFE;
+ events |= RTC_AF;
+ }
+
+ if (events) {
+ rtc_update_irq(m41t80->rtc, 1, events);
+ i2c_smbus_write_byte_data(client, M41T80_REG_FLAGS, flags);
+ i2c_smbus_write_byte_data(client, M41T80_REG_ALARM_MON,
+ flags_afe);
+ }
+
+ mutex_unlock(lock);
+
+ return IRQ_HANDLED;
+}
+
static int m41t80_get_datetime(struct i2c_client *client,
struct rtc_time *tm)
{
- u8 buf[M41T80_DATETIME_REG_SIZE], dt_addr[1] = { M41T80_REG_SEC };
- struct i2c_msg msgs[] = {
- {
- .addr = client->addr,
- .flags = 0,
- .len = 1,
- .buf = dt_addr,
- },
- {
- .addr = client->addr,
- .flags = I2C_M_RD,
- .len = M41T80_DATETIME_REG_SIZE - M41T80_REG_SEC,
- .buf = buf + M41T80_REG_SEC,
- },
- };
+ unsigned char buf[8];
+ int err, flags;
- if (i2c_transfer(client->adapter, msgs, 2) < 0) {
- dev_err(&client->dev, "read error\n");
+ flags = i2c_smbus_read_byte_data(client, M41T80_REG_FLAGS);
+ if (flags < 0)
+ return flags;
+
+ if (flags & M41T80_FLAGS_OF) {
+ dev_err(&client->dev, "Oscillator failure, data is invalid.\n");
+ return -EINVAL;
+ }
+
+ err = i2c_smbus_read_i2c_block_data(client, M41T80_REG_SSEC,
+ sizeof(buf), buf);
+ if (err < 0) {
+ dev_err(&client->dev, "Unable to read date\n");
return -EIO;
}
@@ -129,70 +168,42 @@
/* Sets the given date and time to the real time clock. */
static int m41t80_set_datetime(struct i2c_client *client, struct rtc_time *tm)
{
- u8 wbuf[1 + M41T80_DATETIME_REG_SIZE];
- u8 *buf = &wbuf[1];
- u8 dt_addr[1] = { M41T80_REG_SEC };
- struct i2c_msg msgs_in[] = {
- {
- .addr = client->addr,
- .flags = 0,
- .len = 1,
- .buf = dt_addr,
- },
- {
- .addr = client->addr,
- .flags = I2C_M_RD,
- .len = M41T80_DATETIME_REG_SIZE - M41T80_REG_SEC,
- .buf = buf + M41T80_REG_SEC,
- },
- };
- struct i2c_msg msgs[] = {
- {
- .addr = client->addr,
- .flags = 0,
- .len = 1 + M41T80_DATETIME_REG_SIZE,
- .buf = wbuf,
- },
- };
+ unsigned char buf[8];
+ int err, flags;
- /* Read current reg values into buf[1..7] */
- if (i2c_transfer(client->adapter, msgs_in, 2) < 0) {
- dev_err(&client->dev, "read error\n");
- return -EIO;
- }
-
- wbuf[0] = 0; /* offset into rtc's regs */
- /* Merge time-data and register flags into buf[0..7] */
- buf[M41T80_REG_SSEC] = 0;
- buf[M41T80_REG_SEC] =
- bin2bcd(tm->tm_sec) | (buf[M41T80_REG_SEC] & ~0x7f);
- buf[M41T80_REG_MIN] =
- bin2bcd(tm->tm_min) | (buf[M41T80_REG_MIN] & ~0x7f);
- buf[M41T80_REG_HOUR] =
- bin2bcd(tm->tm_hour) | (buf[M41T80_REG_HOUR] & ~0x3f);
- buf[M41T80_REG_WDAY] =
- (tm->tm_wday & 0x07) | (buf[M41T80_REG_WDAY] & ~0x07);
- buf[M41T80_REG_DAY] =
- bin2bcd(tm->tm_mday) | (buf[M41T80_REG_DAY] & ~0x3f);
- buf[M41T80_REG_MON] =
- bin2bcd(tm->tm_mon + 1) | (buf[M41T80_REG_MON] & ~0x1f);
-
- /* assume 20YY not 19YY */
- if (tm->tm_year < 100 || tm->tm_year > 199) {
- dev_err(&client->dev, "Year must be between 2000 and 2099. It's %d.\n",
- tm->tm_year + 1900);
+ if (tm->tm_year < 100 || tm->tm_year > 199)
return -EINVAL;
- }
- buf[M41T80_REG_YEAR] = bin2bcd(tm->tm_year % 100);
- if (i2c_transfer(client->adapter, msgs, 1) != 1) {
- dev_err(&client->dev, "write error\n");
+ buf[M41T80_REG_SSEC] = 0;
+ buf[M41T80_REG_SEC] = bin2bcd(tm->tm_sec);
+ buf[M41T80_REG_MIN] = bin2bcd(tm->tm_min);
+ buf[M41T80_REG_HOUR] = bin2bcd(tm->tm_hour);
+ buf[M41T80_REG_DAY] = bin2bcd(tm->tm_mday);
+ buf[M41T80_REG_MON] = bin2bcd(tm->tm_mon + 1);
+ buf[M41T80_REG_YEAR] = bin2bcd(tm->tm_year - 100);
+ buf[M41T80_REG_WDAY] = tm->tm_wday;
+
+ err = i2c_smbus_write_i2c_block_data(client, M41T80_REG_SSEC,
+ sizeof(buf), buf);
+ if (err < 0) {
+ dev_err(&client->dev, "Unable to write to date registers\n");
+ return err;
+ }
+
+ /* Clear the OF bit of Flags Register */
+ flags = i2c_smbus_read_byte_data(client, M41T80_REG_FLAGS);
+ if (flags < 0)
+ return flags;
+
+ if (i2c_smbus_write_byte_data(client, M41T80_REG_FLAGS,
+ flags & ~M41T80_FLAGS_OF)) {
+ dev_err(&client->dev, "Unable to write flags register\n");
return -EIO;
}
- return 0;
+
+ return err;
}
-#if defined(CONFIG_RTC_INTF_PROC) || defined(CONFIG_RTC_INTF_PROC_MODULE)
static int m41t80_rtc_proc(struct device *dev, struct seq_file *seq)
{
struct i2c_client *client = to_i2c_client(dev);
@@ -206,9 +217,6 @@
}
return 0;
}
-#else
-#define m41t80_rtc_proc NULL
-#endif
static int m41t80_rtc_read_time(struct device *dev, struct rtc_time *tm)
{
@@ -220,19 +228,117 @@
return m41t80_set_datetime(to_i2c_client(dev), tm);
}
-/*
- * XXX - m41t80 alarm functionality is reported broken.
- * until it is fixed, don't register alarm functions.
- */
+static int m41t80_alarm_irq_enable(struct device *dev, unsigned int enabled)
+{
+ struct i2c_client *client = to_i2c_client(dev);
+ int flags, retval;
+
+ flags = i2c_smbus_read_byte_data(client, M41T80_REG_ALARM_MON);
+ if (flags < 0)
+ return flags;
+
+ if (enabled)
+ flags |= M41T80_ALMON_AFE;
+ else
+ flags &= ~M41T80_ALMON_AFE;
+
+ retval = i2c_smbus_write_byte_data(client, M41T80_REG_ALARM_MON, flags);
+ if (retval < 0) {
+ dev_info(dev, "Unable to enable alarm IRQ %d\n", retval);
+ return retval;
+ }
+ return 0;
+}
+
+static int m41t80_set_alarm(struct device *dev, struct rtc_wkalrm *alrm)
+{
+ struct i2c_client *client = to_i2c_client(dev);
+ u8 alarmvals[5];
+ int ret, err;
+
+ alarmvals[0] = bin2bcd(alrm->time.tm_mon + 1);
+ alarmvals[1] = bin2bcd(alrm->time.tm_mday);
+ alarmvals[2] = bin2bcd(alrm->time.tm_hour);
+ alarmvals[3] = bin2bcd(alrm->time.tm_min);
+ alarmvals[4] = bin2bcd(alrm->time.tm_sec);
+
+ /* Clear AF and AFE flags */
+ ret = i2c_smbus_read_byte_data(client, M41T80_REG_ALARM_MON);
+ if (ret < 0)
+ return ret;
+ err = i2c_smbus_write_byte_data(client, M41T80_REG_ALARM_MON,
+ ret & ~(M41T80_ALMON_AFE));
+ if (err < 0) {
+ dev_err(dev, "Unable to clear AFE bit\n");
+ return err;
+ }
+
+ ret = i2c_smbus_read_byte_data(client, M41T80_REG_FLAGS);
+ if (ret < 0)
+ return ret;
+
+ err = i2c_smbus_write_byte_data(client, M41T80_REG_FLAGS,
+ ret & ~(M41T80_FLAGS_AF));
+ if (err < 0) {
+ dev_err(dev, "Unable to clear AF bit\n");
+ return err;
+ }
+
+ /* Write the alarm */
+ err = i2c_smbus_write_i2c_block_data(client, M41T80_REG_ALARM_MON,
+ 5, alarmvals);
+ if (err)
+ return err;
+
+ /* Enable the alarm interrupt */
+ if (alrm->enabled) {
+ alarmvals[0] |= M41T80_ALMON_AFE;
+ err = i2c_smbus_write_byte_data(client, M41T80_REG_ALARM_MON,
+ alarmvals[0]);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+static int m41t80_read_alarm(struct device *dev, struct rtc_wkalrm *alrm)
+{
+ struct i2c_client *client = to_i2c_client(dev);
+ u8 alarmvals[5];
+ int flags, ret;
+
+ ret = i2c_smbus_read_i2c_block_data(client, M41T80_REG_ALARM_MON,
+ 5, alarmvals);
+ if (ret != 5)
+ return ret < 0 ? ret : -EIO;
+
+ flags = i2c_smbus_read_byte_data(client, M41T80_REG_FLAGS);
+ if (flags < 0)
+ return flags;
+
+ alrm->time.tm_sec = bcd2bin(alarmvals[4] & 0x7f);
+ alrm->time.tm_min = bcd2bin(alarmvals[3] & 0x7f);
+ alrm->time.tm_hour = bcd2bin(alarmvals[2] & 0x3f);
+ alrm->time.tm_wday = -1;
+ alrm->time.tm_mday = bcd2bin(alarmvals[1] & 0x3f);
+ alrm->time.tm_mon = bcd2bin(alarmvals[0] & 0x3f);
+ alrm->time.tm_year = -1;
+
+ alrm->enabled = !!(alarmvals[0] & M41T80_ALMON_AFE);
+ alrm->pending = (flags & M41T80_FLAGS_AF) && alrm->enabled;
+
+ return 0;
+}
+
static struct rtc_class_ops m41t80_rtc_ops = {
.read_time = m41t80_rtc_read_time,
.set_time = m41t80_rtc_set_time,
.proc = m41t80_rtc_proc,
};
-#if defined(CONFIG_RTC_INTF_SYSFS) || defined(CONFIG_RTC_INTF_SYSFS_MODULE)
-static ssize_t m41t80_sysfs_show_flags(struct device *dev,
- struct device_attribute *attr, char *buf)
+static ssize_t flags_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
{
struct i2c_client *client = to_i2c_client(dev);
int val;
@@ -242,10 +348,10 @@
return val;
return sprintf(buf, "%#x\n", val);
}
-static DEVICE_ATTR(flags, S_IRUGO, m41t80_sysfs_show_flags, NULL);
+static DEVICE_ATTR_RO(flags);
-static ssize_t m41t80_sysfs_show_sqwfreq(struct device *dev,
- struct device_attribute *attr, char *buf)
+static ssize_t sqwfreq_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
{
struct i2c_client *client = to_i2c_client(dev);
struct m41t80_data *clientdata = i2c_get_clientdata(client);
@@ -272,14 +378,19 @@
}
return sprintf(buf, "%d\n", val);
}
-static ssize_t m41t80_sysfs_set_sqwfreq(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t count)
+
+static ssize_t sqwfreq_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
{
struct i2c_client *client = to_i2c_client(dev);
struct m41t80_data *clientdata = i2c_get_clientdata(client);
int almon, sqw, reg_sqw, rc;
- int val = simple_strtoul(buf, NULL, 0);
+ unsigned long val;
+
+ rc = kstrtoul(buf, 0, &val);
+ if (rc < 0)
+ return rc;
if (!(clientdata->features & M41T80_FEATURE_SQ))
return -EINVAL;
@@ -308,7 +419,7 @@
sqw = (sqw & 0x0f) | (val << 4);
rc = i2c_smbus_write_byte_data(client, M41T80_REG_ALARM_MON,
- almon & ~M41T80_ALMON_SQWE);
+ almon & ~M41T80_ALMON_SQWE);
if (rc < 0)
return rc;
@@ -318,35 +429,24 @@
return rc;
rc = i2c_smbus_write_byte_data(client, M41T80_REG_ALARM_MON,
- almon | M41T80_ALMON_SQWE);
- if (rc <0)
+ almon | M41T80_ALMON_SQWE);
+ if (rc < 0)
return rc;
}
return count;
}
-static DEVICE_ATTR(sqwfreq, S_IRUGO | S_IWUSR,
- m41t80_sysfs_show_sqwfreq, m41t80_sysfs_set_sqwfreq);
+static DEVICE_ATTR_RW(sqwfreq);
static struct attribute *attrs[] = {
&dev_attr_flags.attr,
&dev_attr_sqwfreq.attr,
NULL,
};
+
static struct attribute_group attr_group = {
.attrs = attrs,
};
-static int m41t80_sysfs_register(struct device *dev)
-{
- return sysfs_create_group(&dev->kobj, &attr_group);
-}
-#else
-static int m41t80_sysfs_register(struct device *dev)
-{
- return 0;
-}
-#endif
-
#ifdef CONFIG_RTC_DRV_M41T80_WDT
/*
*****************************************************************************
@@ -394,7 +494,7 @@
/*
* WDS = 1 (0x80), mulitplier = WD_TIMO, resolution = 1s (0x02)
*/
- i2c_data[1] = wdt_margin<<2 | 0x82;
+ i2c_data[1] = wdt_margin << 2 | 0x82;
/*
* M41T65 has three bits for watchdog resolution. Don't set bit 7, as
@@ -636,49 +736,76 @@
*
*****************************************************************************
*/
+
+static void m41t80_remove_sysfs_group(void *_dev)
+{
+ struct device *dev = _dev;
+
+ sysfs_remove_group(&dev->kobj, &attr_group);
+}
+
static int m41t80_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
+ struct i2c_adapter *adapter = to_i2c_adapter(client->dev.parent);
int rc = 0;
struct rtc_device *rtc = NULL;
struct rtc_time tm;
- struct m41t80_data *clientdata = NULL;
+ struct m41t80_data *m41t80_data = NULL;
- if (!i2c_check_functionality(client->adapter, I2C_FUNC_I2C
- | I2C_FUNC_SMBUS_BYTE_DATA))
+ if (!i2c_check_functionality(client->adapter, I2C_FUNC_SMBUS_I2C_BLOCK |
+ I2C_FUNC_SMBUS_BYTE_DATA)) {
+ dev_err(&adapter->dev, "doesn't support I2C_FUNC_SMBUS_BYTE_DATA | I2C_FUNC_SMBUS_I2C_BLOCK\n");
return -ENODEV;
+ }
- clientdata = devm_kzalloc(&client->dev, sizeof(*clientdata),
- GFP_KERNEL);
- if (!clientdata)
+ m41t80_data = devm_kzalloc(&client->dev, sizeof(*m41t80_data),
+ GFP_KERNEL);
+ if (!m41t80_data)
return -ENOMEM;
- clientdata->features = id->driver_data;
- i2c_set_clientdata(client, clientdata);
+ m41t80_data->features = id->driver_data;
+ i2c_set_clientdata(client, m41t80_data);
+
+ if (client->irq > 0) {
+ rc = devm_request_threaded_irq(&client->dev, client->irq,
+ NULL, m41t80_handle_irq,
+ IRQF_TRIGGER_LOW | IRQF_ONESHOT,
+ "m41t80", client);
+ if (rc) {
+ dev_warn(&client->dev, "unable to request IRQ, alarms disabled\n");
+ client->irq = 0;
+ } else {
+ m41t80_rtc_ops.read_alarm = m41t80_read_alarm;
+ m41t80_rtc_ops.set_alarm = m41t80_set_alarm;
+ m41t80_rtc_ops.alarm_irq_enable = m41t80_alarm_irq_enable;
+ /* Enable the wakealarm */
+ device_init_wakeup(&client->dev, true);
+ }
+ }
rtc = devm_rtc_device_register(&client->dev, client->name,
- &m41t80_rtc_ops, THIS_MODULE);
+ &m41t80_rtc_ops, THIS_MODULE);
if (IS_ERR(rtc))
return PTR_ERR(rtc);
- clientdata->rtc = rtc;
+ m41t80_data->rtc = rtc;
/* Make sure HT (Halt Update) bit is cleared */
rc = i2c_smbus_read_byte_data(client, M41T80_REG_ALARM_HOUR);
if (rc >= 0 && rc & M41T80_ALHOUR_HT) {
- if (clientdata->features & M41T80_FEATURE_HT) {
+ if (m41t80_data->features & M41T80_FEATURE_HT) {
m41t80_get_datetime(client, &tm);
dev_info(&client->dev, "HT bit was set!\n");
dev_info(&client->dev,
- "Power Down at "
- "%04i-%02i-%02i %02i:%02i:%02i\n",
+ "Power Down at %04i-%02i-%02i %02i:%02i:%02i\n",
tm.tm_year + 1900,
tm.tm_mon + 1, tm.tm_mday, tm.tm_hour,
tm.tm_min, tm.tm_sec);
}
rc = i2c_smbus_write_byte_data(client, M41T80_REG_ALARM_HOUR,
- rc & ~M41T80_ALHOUR_HT);
+ rc & ~M41T80_ALHOUR_HT);
}
if (rc < 0) {
@@ -691,18 +818,30 @@
if (rc >= 0 && rc & M41T80_SEC_ST)
rc = i2c_smbus_write_byte_data(client, M41T80_REG_SEC,
- rc & ~M41T80_SEC_ST);
+ rc & ~M41T80_SEC_ST);
if (rc < 0) {
dev_err(&client->dev, "Can't clear ST bit\n");
return rc;
}
- rc = m41t80_sysfs_register(&client->dev);
- if (rc)
+ /* Export sysfs entries */
+ rc = sysfs_create_group(&(&client->dev)->kobj, &attr_group);
+ if (rc) {
+ dev_err(&client->dev, "Failed to create sysfs group: %d\n", rc);
return rc;
+ }
+
+ rc = devm_add_action(&client->dev, m41t80_remove_sysfs_group,
+ &client->dev);
+ if (rc) {
+ m41t80_remove_sysfs_group(&client->dev);
+ dev_err(&client->dev,
+ "Failed to add sysfs cleanup action: %d\n", rc);
+ return rc;
+ }
#ifdef CONFIG_RTC_DRV_M41T80_WDT
- if (clientdata->features & M41T80_FEATURE_HT) {
+ if (m41t80_data->features & M41T80_FEATURE_HT) {
save_client = client;
rc = misc_register(&wdt_dev);
if (rc)
diff --git a/drivers/rtc/rtc-m48t35.c b/drivers/rtc/rtc-m48t35.c
index c62b512..810f4ea 100644
--- a/drivers/rtc/rtc-m48t35.c
+++ b/drivers/rtc/rtc-m48t35.c
@@ -22,8 +22,6 @@
#include <linux/io.h>
#include <linux/err.h>
-#define DRV_VERSION "1.0"
-
struct m48t35_rtc {
u8 pad[0x7ff8]; /* starts at 0x7ff8 */
u8 control;
@@ -190,5 +188,4 @@
MODULE_AUTHOR("Thomas Bogendoerfer <tsbogend@alpha.franken.de>");
MODULE_DESCRIPTION("M48T35 RTC driver");
MODULE_LICENSE("GPL");
-MODULE_VERSION(DRV_VERSION);
MODULE_ALIAS("platform:rtc-m48t35");
diff --git a/drivers/rtc/rtc-m48t86.c b/drivers/rtc/rtc-m48t86.c
index a17b7a3c..f72b91f 100644
--- a/drivers/rtc/rtc-m48t86.c
+++ b/drivers/rtc/rtc-m48t86.c
@@ -39,9 +39,6 @@
#define M48T86_REG_B_SET (1 << 7)
#define M48T86_REG_D_VRT (1 << 7)
-#define DRV_VERSION "0.1"
-
-
static int m48t86_rtc_read_time(struct device *dev, struct rtc_time *tm)
{
unsigned char reg;
@@ -178,5 +175,4 @@
MODULE_AUTHOR("Alessandro Zummo <a.zummo@towertech.it>");
MODULE_DESCRIPTION("M48T86 RTC driver");
MODULE_LICENSE("GPL");
-MODULE_VERSION(DRV_VERSION);
MODULE_ALIAS("platform:rtc-m48t86");
diff --git a/drivers/rtc/rtc-max6900.c b/drivers/rtc/rtc-max6900.c
index b2a7607..48b6b41 100644
--- a/drivers/rtc/rtc-max6900.c
+++ b/drivers/rtc/rtc-max6900.c
@@ -17,8 +17,6 @@
#include <linux/rtc.h>
#include <linux/delay.h>
-#define DRV_VERSION "0.2"
-
/*
* register indices
*/
@@ -218,8 +216,6 @@
if (!i2c_check_functionality(client->adapter, I2C_FUNC_I2C))
return -ENODEV;
- dev_info(&client->dev, "chip found, driver version " DRV_VERSION "\n");
-
rtc = devm_rtc_device_register(&client->dev, max6900_driver.driver.name,
&max6900_rtc_ops, THIS_MODULE);
if (IS_ERR(rtc))
@@ -249,4 +245,3 @@
MODULE_DESCRIPTION("Maxim MAX6900 RTC driver");
MODULE_AUTHOR("Dale Farnsworth <dale@farnsworth.org>");
MODULE_LICENSE("GPL");
-MODULE_VERSION(DRV_VERSION);
diff --git a/drivers/rtc/rtc-mc13xxx.c b/drivers/rtc/rtc-mc13xxx.c
index a658680..30b8ef6 100644
--- a/drivers/rtc/rtc-mc13xxx.c
+++ b/drivers/rtc/rtc-mc13xxx.c
@@ -250,18 +250,6 @@
return IRQ_HANDLED;
}
-static irqreturn_t mc13xxx_rtc_update_handler(int irq, void *dev)
-{
- struct mc13xxx_rtc *priv = dev;
- struct mc13xxx *mc13xxx = priv->mc13xxx;
-
- rtc_update_irq(priv->rtc, 1, RTC_IRQF | RTC_UF);
-
- mc13xxx_irq_ack(mc13xxx, irq);
-
- return IRQ_HANDLED;
-}
-
static const struct rtc_class_ops mc13xxx_rtc_ops = {
.read_time = mc13xxx_rtc_read_time,
.set_mmss64 = mc13xxx_rtc_set_mmss,
@@ -307,11 +295,6 @@
if (ret)
goto err_irq_request;
- ret = mc13xxx_irq_request(mc13xxx, MC13XXX_IRQ_1HZ,
- mc13xxx_rtc_update_handler, DRIVER_NAME, priv);
- if (ret)
- goto err_irq_request;
-
ret = mc13xxx_irq_request_nounmask(mc13xxx, MC13XXX_IRQ_TODA,
mc13xxx_rtc_alarm_handler, DRIVER_NAME, priv);
if (ret)
@@ -326,7 +309,6 @@
err_irq_request:
mc13xxx_irq_free(mc13xxx, MC13XXX_IRQ_TODA, priv);
- mc13xxx_irq_free(mc13xxx, MC13XXX_IRQ_1HZ, priv);
mc13xxx_irq_free(mc13xxx, MC13XXX_IRQ_RTCRST, priv);
mc13xxx_unlock(mc13xxx);
@@ -341,7 +323,6 @@
mc13xxx_lock(priv->mc13xxx);
mc13xxx_irq_free(priv->mc13xxx, MC13XXX_IRQ_TODA, priv);
- mc13xxx_irq_free(priv->mc13xxx, MC13XXX_IRQ_1HZ, priv);
mc13xxx_irq_free(priv->mc13xxx, MC13XXX_IRQ_RTCRST, priv);
mc13xxx_unlock(priv->mc13xxx);
diff --git a/drivers/rtc/rtc-mrst.c b/drivers/rtc/rtc-mrst.c
index 548ea6f..0094d9b 100644
--- a/drivers/rtc/rtc-mrst.c
+++ b/drivers/rtc/rtc-mrst.c
@@ -266,7 +266,7 @@
}
-#if defined(CONFIG_RTC_INTF_PROC) || defined(CONFIG_RTC_INTF_PROC_MODULE)
+#if IS_ENABLED(CONFIG_RTC_INTF_PROC)
static int mrst_procfs(struct device *dev, struct seq_file *seq)
{
diff --git a/drivers/rtc/rtc-mxc.c b/drivers/rtc/rtc-mxc.c
index 7bd89d9..359876a 100644
--- a/drivers/rtc/rtc-mxc.c
+++ b/drivers/rtc/rtc-mxc.c
@@ -240,9 +240,6 @@
mxc_rtc_irq_enable(&pdev->dev, RTC_ALM_BIT, 0);
}
- if (status & RTC_1HZ_BIT)
- events |= (RTC_UF | RTC_IRQF);
-
if (status & PIT_ALL_ON)
events |= (RTC_PF | RTC_IRQF);
diff --git a/drivers/rtc/rtc-pcf2123.c b/drivers/rtc/rtc-pcf2123.c
index da27738b..f22e060 100644
--- a/drivers/rtc/rtc-pcf2123.c
+++ b/drivers/rtc/rtc-pcf2123.c
@@ -46,8 +46,6 @@
#include <linux/module.h>
#include <linux/sysfs.h>
-#define DRV_VERSION "0.6"
-
/* REGISTERS */
#define PCF2123_REG_CTRL1 (0x00) /* Control Register 1 */
#define PCF2123_REG_CTRL2 (0x01) /* Control Register 2 */
@@ -395,7 +393,6 @@
}
}
- dev_info(&spi->dev, "chip found, driver version " DRV_VERSION "\n");
dev_info(&spi->dev, "spiclk %u KHz.\n",
(spi->max_speed_hz + 500) / 1000);
@@ -474,4 +471,3 @@
MODULE_AUTHOR("Chris Verges <chrisv@cyberswitching.com>");
MODULE_DESCRIPTION("NXP PCF2123 RTC driver");
MODULE_LICENSE("GPL");
-MODULE_VERSION(DRV_VERSION);
diff --git a/drivers/rtc/rtc-pcf8563.c b/drivers/rtc/rtc-pcf8563.c
index c8f95b8..b9ddbb0 100644
--- a/drivers/rtc/rtc-pcf8563.c
+++ b/drivers/rtc/rtc-pcf8563.c
@@ -23,8 +23,6 @@
#include <linux/of.h>
#include <linux/err.h>
-#define DRV_VERSION "0.4.4"
-
#define PCF8563_REG_ST1 0x00 /* status */
#define PCF8563_REG_ST2 0x01
#define PCF8563_BIT_AIE (1 << 1)
@@ -535,7 +533,7 @@
init.name = "pcf8563-clkout";
init.ops = &pcf8563_clkout_ops;
- init.flags = CLK_IS_ROOT;
+ init.flags = 0;
init.parent_names = NULL;
init.num_parents = 0;
pcf8563->clkout_hw.init = &init;
@@ -580,8 +578,6 @@
if (!pcf8563)
return -ENOMEM;
- dev_info(&client->dev, "chip found, driver version " DRV_VERSION "\n");
-
i2c_set_clientdata(client, pcf8563);
pcf8563->client = client;
device_set_wakeup_capable(&client->dev, 1);
@@ -662,4 +658,3 @@
MODULE_AUTHOR("Alessandro Zummo <a.zummo@towertech.it>");
MODULE_DESCRIPTION("Philips PCF8563/Epson RTC8564 RTC driver");
MODULE_LICENSE("GPL");
-MODULE_VERSION(DRV_VERSION);
diff --git a/drivers/rtc/rtc-rs5c313.c b/drivers/rtc/rtc-rs5c313.c
index 5f48167..89f38e3 100644
--- a/drivers/rtc/rtc-rs5c313.c
+++ b/drivers/rtc/rtc-rs5c313.c
@@ -50,7 +50,6 @@
#include <linux/io.h>
#define DRV_NAME "rs5c313"
-#define DRV_VERSION "1.13"
#ifdef CONFIG_SH_LANDISK
/*****************************************************/
@@ -407,7 +406,6 @@
module_init(rs5c313_rtc_init);
module_exit(rs5c313_rtc_exit);
-MODULE_VERSION(DRV_VERSION);
MODULE_AUTHOR("kogiidena , Nobuhiro Iwamatsu <iwamatsu@nigauri.org>");
MODULE_DESCRIPTION("Ricoh RS5C313 RTC device driver");
MODULE_LICENSE("GPL");
diff --git a/drivers/rtc/rtc-rs5c348.c b/drivers/rtc/rtc-rs5c348.c
index 1162fec..9a30698 100644
--- a/drivers/rtc/rtc-rs5c348.c
+++ b/drivers/rtc/rtc-rs5c348.c
@@ -25,8 +25,6 @@
#include <linux/spi/spi.h>
#include <linux/module.h>
-#define DRV_VERSION "0.2"
-
#define RS5C348_REG_SECS 0
#define RS5C348_REG_MINS 1
#define RS5C348_REG_HOURS 2
@@ -171,7 +169,6 @@
goto kfree_exit;
}
- dev_info(&spi->dev, "chip found, driver version " DRV_VERSION "\n");
dev_info(&spi->dev, "spiclk %u KHz.\n",
(spi->max_speed_hz + 500) / 1000);
@@ -230,5 +227,4 @@
MODULE_AUTHOR("Atsushi Nemoto <anemo@mba.ocn.ne.jp>");
MODULE_DESCRIPTION("Ricoh RS5C348 RTC driver");
MODULE_LICENSE("GPL");
-MODULE_VERSION(DRV_VERSION);
MODULE_ALIAS("spi:rtc-rs5c348");
diff --git a/drivers/rtc/rtc-rs5c372.c b/drivers/rtc/rtc-rs5c372.c
index 28871cd..ef86229 100644
--- a/drivers/rtc/rtc-rs5c372.c
+++ b/drivers/rtc/rtc-rs5c372.c
@@ -16,9 +16,6 @@
#include <linux/slab.h>
#include <linux/module.h>
-#define DRV_VERSION "0.6"
-
-
/*
* Ricoh has a family of I2C based RTCs, which differ only slightly from
* each other. Differences center on pinout (e.g. how many interrupts,
@@ -240,11 +237,11 @@
return 0;
}
-#if defined(CONFIG_RTC_INTF_PROC) || defined(CONFIG_RTC_INTF_PROC_MODULE)
+#if IS_ENABLED(CONFIG_RTC_INTF_PROC)
#define NEED_TRIM
#endif
-#if defined(CONFIG_RTC_INTF_SYSFS) || defined(CONFIG_RTC_INTF_SYSFS_MODULE)
+#if IS_ENABLED(CONFIG_RTC_INTF_SYSFS)
#define NEED_TRIM
#endif
@@ -412,7 +409,7 @@
return 0;
}
-#if defined(CONFIG_RTC_INTF_PROC) || defined(CONFIG_RTC_INTF_PROC_MODULE)
+#if IS_ENABLED(CONFIG_RTC_INTF_PROC)
static int rs5c372_rtc_proc(struct device *dev, struct seq_file *seq)
{
@@ -441,7 +438,7 @@
.alarm_irq_enable = rs5c_rtc_alarm_irq_enable,
};
-#if defined(CONFIG_RTC_INTF_SYSFS) || defined(CONFIG_RTC_INTF_SYSFS_MODULE)
+#if IS_ENABLED(CONFIG_RTC_INTF_SYSFS)
static ssize_t rs5c372_sysfs_show_trim(struct device *dev,
struct device_attribute *attr, char *buf)
@@ -509,9 +506,9 @@
int addr, i, ret = 0;
if (rs5c372->type == rtc_r2025sd) {
- if (!(rs5c372->regs[RS5C_REG_CTRL2] & R2025_CTRL2_XST))
+ if (rs5c372->regs[RS5C_REG_CTRL2] & R2025_CTRL2_XST)
return ret;
- rs5c372->regs[RS5C_REG_CTRL2] &= ~R2025_CTRL2_XST;
+ rs5c372->regs[RS5C_REG_CTRL2] |= R2025_CTRL2_XST;
} else {
if (!(rs5c372->regs[RS5C_REG_CTRL2] & RS5C_CTRL2_XSTP))
return ret;
@@ -640,7 +637,7 @@
if (rs5c372_get_datetime(client, &tm) < 0)
dev_warn(&client->dev, "clock needs to be set\n");
- dev_info(&client->dev, "%s found, %s, driver version " DRV_VERSION "\n",
+ dev_info(&client->dev, "%s found, %s\n",
({ char *s; switch (rs5c372->type) {
case rtc_r2025sd: s = "r2025sd"; break;
case rtc_r2221tl: s = "r2221tl"; break;
@@ -696,4 +693,3 @@
"Paul Mundt <lethal@linux-sh.org>");
MODULE_DESCRIPTION("Ricoh RS5C372 RTC driver");
MODULE_LICENSE("GPL");
-MODULE_VERSION(DRV_VERSION);
diff --git a/drivers/rtc/rtc-rv3029c2.c b/drivers/rtc/rtc-rv3029c2.c
index d0cbf08..1f9f7b4 100644
--- a/drivers/rtc/rtc-rv3029c2.c
+++ b/drivers/rtc/rtc-rv3029c2.c
@@ -1,5 +1,5 @@
/*
- * Micro Crystal RV-3029 rtc class driver
+ * Micro Crystal RV-3029 / RV-3049 rtc class driver
*
* Author: Gregory Hermant <gregory.hermant@calao-systems.com>
* Michael Buesch <m@bues.ch>
@@ -14,13 +14,14 @@
#include <linux/module.h>
#include <linux/i2c.h>
+#include <linux/spi/spi.h>
#include <linux/bcd.h>
#include <linux/rtc.h>
#include <linux/delay.h>
#include <linux/of.h>
#include <linux/hwmon.h>
#include <linux/hwmon-sysfs.h>
-
+#include <linux/regmap.h>
/* Register map */
/* control section */
@@ -75,6 +76,7 @@
#define RV3029_A_DW 0x14
#define RV3029_A_MO 0x15
#define RV3029_A_YR 0x16
+#define RV3029_A_AE_X BIT(7)
#define RV3029_ALARM_SECTION_LEN 0x07
/* timer section */
@@ -116,85 +118,84 @@
#define RV3029_USR2_RAM_PAGE 0x3C
#define RV3029_USR2_SECTION_LEN 0x04
-static int
-rv3029_i2c_read_regs(struct i2c_client *client, u8 reg, u8 *buf,
- unsigned len)
+struct rv3029_data {
+ struct device *dev;
+ struct rtc_device *rtc;
+ struct regmap *regmap;
+ int irq;
+};
+
+static int rv3029_read_regs(struct device *dev, u8 reg, u8 *buf,
+ unsigned int len)
{
- int ret;
+ struct rv3029_data *rv3029 = dev_get_drvdata(dev);
if ((reg > RV3029_USR1_RAM_PAGE + 7) ||
- (reg + len > RV3029_USR1_RAM_PAGE + 8))
+ (reg + len > RV3029_USR1_RAM_PAGE + 8))
return -EINVAL;
- ret = i2c_smbus_read_i2c_block_data(client, reg, len, buf);
- if (ret < 0)
- return ret;
- if (ret < len)
- return -EIO;
- return 0;
+ return regmap_bulk_read(rv3029->regmap, reg, buf, len);
}
-static int
-rv3029_i2c_write_regs(struct i2c_client *client, u8 reg, u8 const buf[],
- unsigned len)
+static int rv3029_write_regs(struct device *dev, u8 reg, u8 const buf[],
+ unsigned int len)
{
+ struct rv3029_data *rv3029 = dev_get_drvdata(dev);
+
if ((reg > RV3029_USR1_RAM_PAGE + 7) ||
- (reg + len > RV3029_USR1_RAM_PAGE + 8))
+ (reg + len > RV3029_USR1_RAM_PAGE + 8))
return -EINVAL;
- return i2c_smbus_write_i2c_block_data(client, reg, len, buf);
+ return regmap_bulk_write(rv3029->regmap, reg, buf, len);
}
-static int
-rv3029_i2c_update_bits(struct i2c_client *client, u8 reg, u8 mask, u8 set)
+static int rv3029_update_bits(struct device *dev, u8 reg, u8 mask, u8 set)
{
u8 buf;
int ret;
- ret = rv3029_i2c_read_regs(client, reg, &buf, 1);
+ ret = rv3029_read_regs(dev, reg, &buf, 1);
if (ret < 0)
return ret;
buf &= ~mask;
buf |= set & mask;
- ret = rv3029_i2c_write_regs(client, reg, &buf, 1);
+ ret = rv3029_write_regs(dev, reg, &buf, 1);
if (ret < 0)
return ret;
return 0;
}
-static int
-rv3029_i2c_get_sr(struct i2c_client *client, u8 *buf)
+static int rv3029_get_sr(struct device *dev, u8 *buf)
{
- int ret = rv3029_i2c_read_regs(client, RV3029_STATUS, buf, 1);
+ int ret = rv3029_read_regs(dev, RV3029_STATUS, buf, 1);
if (ret < 0)
return -EIO;
- dev_dbg(&client->dev, "status = 0x%.2x (%d)\n", buf[0], buf[0]);
+ dev_dbg(dev, "status = 0x%.2x (%d)\n", buf[0], buf[0]);
return 0;
}
-static int
-rv3029_i2c_set_sr(struct i2c_client *client, u8 val)
+static int rv3029_set_sr(struct device *dev, u8 val)
{
u8 buf[1];
int sr;
buf[0] = val;
- sr = rv3029_i2c_write_regs(client, RV3029_STATUS, buf, 1);
- dev_dbg(&client->dev, "status = 0x%.2x (%d)\n", buf[0], buf[0]);
+ sr = rv3029_write_regs(dev, RV3029_STATUS, buf, 1);
+ dev_dbg(dev, "status = 0x%.2x (%d)\n", buf[0], buf[0]);
if (sr < 0)
return -EIO;
return 0;
}
-static int rv3029_eeprom_busywait(struct i2c_client *client)
+static int rv3029_eeprom_busywait(struct device *dev)
{
int i, ret;
u8 sr;
for (i = 100; i > 0; i--) {
- ret = rv3029_i2c_get_sr(client, &sr);
+ ret = rv3029_get_sr(dev, &sr);
if (ret < 0)
break;
if (!(sr & RV3029_STATUS_EEBUSY))
@@ -202,28 +203,28 @@
usleep_range(1000, 10000);
}
if (i <= 0) {
- dev_err(&client->dev, "EEPROM busy wait timeout.\n");
+ dev_err(dev, "EEPROM busy wait timeout.\n");
return -ETIMEDOUT;
}
return ret;
}
-static int rv3029_eeprom_exit(struct i2c_client *client)
+static int rv3029_eeprom_exit(struct device *dev)
{
/* Re-enable eeprom refresh */
- return rv3029_i2c_update_bits(client, RV3029_ONOFF_CTRL,
- RV3029_ONOFF_CTRL_EERE,
- RV3029_ONOFF_CTRL_EERE);
+ return rv3029_update_bits(dev, RV3029_ONOFF_CTRL,
+ RV3029_ONOFF_CTRL_EERE,
+ RV3029_ONOFF_CTRL_EERE);
}
-static int rv3029_eeprom_enter(struct i2c_client *client)
+static int rv3029_eeprom_enter(struct device *dev)
{
int ret;
u8 sr;
/* Check whether we are in the allowed voltage range. */
- ret = rv3029_i2c_get_sr(client, &sr);
+ ret = rv3029_get_sr(dev, &sr);
if (ret < 0)
return ret;
if (sr & (RV3029_STATUS_VLOW1 | RV3029_STATUS_VLOW2)) {
@@ -232,129 +233,168 @@
*/
sr &= ~RV3029_STATUS_VLOW1;
sr &= ~RV3029_STATUS_VLOW2;
- ret = rv3029_i2c_set_sr(client, sr);
+ ret = rv3029_set_sr(dev, sr);
if (ret < 0)
return ret;
usleep_range(1000, 10000);
- ret = rv3029_i2c_get_sr(client, &sr);
+ ret = rv3029_get_sr(dev, &sr);
if (ret < 0)
return ret;
if (sr & (RV3029_STATUS_VLOW1 | RV3029_STATUS_VLOW2)) {
- dev_err(&client->dev,
+ dev_err(dev,
"Supply voltage is too low to safely access the EEPROM.\n");
return -ENODEV;
}
}
/* Disable eeprom refresh. */
- ret = rv3029_i2c_update_bits(client, RV3029_ONOFF_CTRL,
- RV3029_ONOFF_CTRL_EERE, 0);
+ ret = rv3029_update_bits(dev, RV3029_ONOFF_CTRL, RV3029_ONOFF_CTRL_EERE,
+ 0);
if (ret < 0)
return ret;
/* Wait for any previous eeprom accesses to finish. */
- ret = rv3029_eeprom_busywait(client);
+ ret = rv3029_eeprom_busywait(dev);
if (ret < 0)
- rv3029_eeprom_exit(client);
+ rv3029_eeprom_exit(dev);
return ret;
}
-static int rv3029_eeprom_read(struct i2c_client *client, u8 reg,
+static int rv3029_eeprom_read(struct device *dev, u8 reg,
u8 buf[], size_t len)
{
int ret, err;
- err = rv3029_eeprom_enter(client);
+ err = rv3029_eeprom_enter(dev);
if (err < 0)
return err;
- ret = rv3029_i2c_read_regs(client, reg, buf, len);
+ ret = rv3029_read_regs(dev, reg, buf, len);
- err = rv3029_eeprom_exit(client);
+ err = rv3029_eeprom_exit(dev);
if (err < 0)
return err;
return ret;
}
-static int rv3029_eeprom_write(struct i2c_client *client, u8 reg,
+static int rv3029_eeprom_write(struct device *dev, u8 reg,
u8 const buf[], size_t len)
{
int ret, err;
size_t i;
u8 tmp;
- err = rv3029_eeprom_enter(client);
+ err = rv3029_eeprom_enter(dev);
if (err < 0)
return err;
for (i = 0; i < len; i++, reg++) {
- ret = rv3029_i2c_read_regs(client, reg, &tmp, 1);
+ ret = rv3029_read_regs(dev, reg, &tmp, 1);
if (ret < 0)
break;
if (tmp != buf[i]) {
- ret = rv3029_i2c_write_regs(client, reg, &buf[i], 1);
+ ret = rv3029_write_regs(dev, reg, &buf[i], 1);
if (ret < 0)
break;
}
- ret = rv3029_eeprom_busywait(client);
+ ret = rv3029_eeprom_busywait(dev);
if (ret < 0)
break;
}
- err = rv3029_eeprom_exit(client);
+ err = rv3029_eeprom_exit(dev);
if (err < 0)
return err;
return ret;
}
-static int rv3029_eeprom_update_bits(struct i2c_client *client,
+static int rv3029_eeprom_update_bits(struct device *dev,
u8 reg, u8 mask, u8 set)
{
u8 buf;
int ret;
- ret = rv3029_eeprom_read(client, reg, &buf, 1);
+ ret = rv3029_eeprom_read(dev, reg, &buf, 1);
if (ret < 0)
return ret;
buf &= ~mask;
buf |= set & mask;
- ret = rv3029_eeprom_write(client, reg, &buf, 1);
+ ret = rv3029_eeprom_write(dev, reg, &buf, 1);
if (ret < 0)
return ret;
return 0;
}
-static int
-rv3029_i2c_read_time(struct i2c_client *client, struct rtc_time *tm)
+static irqreturn_t rv3029_handle_irq(int irq, void *dev_id)
+{
+ struct device *dev = dev_id;
+ struct rv3029_data *rv3029 = dev_get_drvdata(dev);
+ struct mutex *lock = &rv3029->rtc->ops_lock;
+ unsigned long events = 0;
+ u8 flags, controls;
+ int ret;
+
+ mutex_lock(lock);
+
+ ret = rv3029_read_regs(dev, RV3029_IRQ_CTRL, &controls, 1);
+ if (ret) {
+ dev_warn(dev, "Read IRQ Control Register error %d\n", ret);
+ mutex_unlock(lock);
+ return IRQ_NONE;
+ }
+
+ ret = rv3029_read_regs(dev, RV3029_IRQ_FLAGS, &flags, 1);
+ if (ret) {
+ dev_warn(dev, "Read IRQ Flags Register error %d\n", ret);
+ mutex_unlock(lock);
+ return IRQ_NONE;
+ }
+
+ if (flags & RV3029_IRQ_FLAGS_AF) {
+ flags &= ~RV3029_IRQ_FLAGS_AF;
+ controls &= ~RV3029_IRQ_CTRL_AIE;
+ events |= RTC_AF;
+ }
+
+ if (events) {
+ rtc_update_irq(rv3029->rtc, 1, events);
+ rv3029_write_regs(dev, RV3029_IRQ_FLAGS, &flags, 1);
+ rv3029_write_regs(dev, RV3029_IRQ_CTRL, &controls, 1);
+ }
+ mutex_unlock(lock);
+
+ return IRQ_HANDLED;
+}
+
+static int rv3029_read_time(struct device *dev, struct rtc_time *tm)
{
u8 buf[1];
int ret;
u8 regs[RV3029_WATCH_SECTION_LEN] = { 0, };
- ret = rv3029_i2c_get_sr(client, buf);
+ ret = rv3029_get_sr(dev, buf);
if (ret < 0) {
- dev_err(&client->dev, "%s: reading SR failed\n", __func__);
+ dev_err(dev, "%s: reading SR failed\n", __func__);
return -EIO;
}
- ret = rv3029_i2c_read_regs(client, RV3029_W_SEC, regs,
- RV3029_WATCH_SECTION_LEN);
+ ret = rv3029_read_regs(dev, RV3029_W_SEC, regs,
+ RV3029_WATCH_SECTION_LEN);
if (ret < 0) {
- dev_err(&client->dev, "%s: reading RTC section failed\n",
- __func__);
+ dev_err(dev, "%s: reading RTC section failed\n", __func__);
return ret;
}
- tm->tm_sec = bcd2bin(regs[RV3029_W_SEC-RV3029_W_SEC]);
- tm->tm_min = bcd2bin(regs[RV3029_W_MINUTES-RV3029_W_SEC]);
+ tm->tm_sec = bcd2bin(regs[RV3029_W_SEC - RV3029_W_SEC]);
+ tm->tm_min = bcd2bin(regs[RV3029_W_MINUTES - RV3029_W_SEC]);
/* HR field has a more complex interpretation */
{
- const u8 _hr = regs[RV3029_W_HOURS-RV3029_W_SEC];
+ const u8 _hr = regs[RV3029_W_HOURS - RV3029_W_SEC];
if (_hr & RV3029_REG_HR_12_24) {
/* 12h format */
@@ -365,77 +405,86 @@
tm->tm_hour = bcd2bin(_hr & 0x3f);
}
- tm->tm_mday = bcd2bin(regs[RV3029_W_DATE-RV3029_W_SEC]);
- tm->tm_mon = bcd2bin(regs[RV3029_W_MONTHS-RV3029_W_SEC]) - 1;
- tm->tm_year = bcd2bin(regs[RV3029_W_YEARS-RV3029_W_SEC]) + 100;
- tm->tm_wday = bcd2bin(regs[RV3029_W_DAYS-RV3029_W_SEC]) - 1;
+ tm->tm_mday = bcd2bin(regs[RV3029_W_DATE - RV3029_W_SEC]);
+ tm->tm_mon = bcd2bin(regs[RV3029_W_MONTHS - RV3029_W_SEC]) - 1;
+ tm->tm_year = bcd2bin(regs[RV3029_W_YEARS - RV3029_W_SEC]) + 100;
+ tm->tm_wday = bcd2bin(regs[RV3029_W_DAYS - RV3029_W_SEC]) - 1;
return 0;
}
-static int rv3029_rtc_read_time(struct device *dev, struct rtc_time *tm)
-{
- return rv3029_i2c_read_time(to_i2c_client(dev), tm);
-}
-
-static int
-rv3029_i2c_read_alarm(struct i2c_client *client, struct rtc_wkalrm *alarm)
+static int rv3029_read_alarm(struct device *dev, struct rtc_wkalrm *alarm)
{
struct rtc_time *const tm = &alarm->time;
int ret;
- u8 regs[8];
+ u8 regs[8], controls, flags;
- ret = rv3029_i2c_get_sr(client, regs);
+ ret = rv3029_get_sr(dev, regs);
if (ret < 0) {
- dev_err(&client->dev, "%s: reading SR failed\n", __func__);
+ dev_err(dev, "%s: reading SR failed\n", __func__);
return -EIO;
}
- ret = rv3029_i2c_read_regs(client, RV3029_A_SC, regs,
- RV3029_ALARM_SECTION_LEN);
+ ret = rv3029_read_regs(dev, RV3029_A_SC, regs,
+ RV3029_ALARM_SECTION_LEN);
if (ret < 0) {
- dev_err(&client->dev, "%s: reading alarm section failed\n",
- __func__);
+ dev_err(dev, "%s: reading alarm section failed\n", __func__);
return ret;
}
- tm->tm_sec = bcd2bin(regs[RV3029_A_SC-RV3029_A_SC] & 0x7f);
- tm->tm_min = bcd2bin(regs[RV3029_A_MN-RV3029_A_SC] & 0x7f);
- tm->tm_hour = bcd2bin(regs[RV3029_A_HR-RV3029_A_SC] & 0x3f);
- tm->tm_mday = bcd2bin(regs[RV3029_A_DT-RV3029_A_SC] & 0x3f);
- tm->tm_mon = bcd2bin(regs[RV3029_A_MO-RV3029_A_SC] & 0x1f) - 1;
- tm->tm_year = bcd2bin(regs[RV3029_A_YR-RV3029_A_SC] & 0x7f) + 100;
- tm->tm_wday = bcd2bin(regs[RV3029_A_DW-RV3029_A_SC] & 0x07) - 1;
+ ret = rv3029_read_regs(dev, RV3029_IRQ_CTRL, &controls, 1);
+ if (ret) {
+ dev_err(dev, "Read IRQ Control Register error %d\n", ret);
+ return ret;
+ }
+ ret = rv3029_read_regs(dev, RV3029_IRQ_FLAGS, &flags, 1);
+ if (ret < 0) {
+ dev_err(dev, "Read IRQ Flags Register error %d\n", ret);
+ return ret;
+ }
+
+ tm->tm_sec = bcd2bin(regs[RV3029_A_SC - RV3029_A_SC] & 0x7f);
+ tm->tm_min = bcd2bin(regs[RV3029_A_MN - RV3029_A_SC] & 0x7f);
+ tm->tm_hour = bcd2bin(regs[RV3029_A_HR - RV3029_A_SC] & 0x3f);
+ tm->tm_mday = bcd2bin(regs[RV3029_A_DT - RV3029_A_SC] & 0x3f);
+ tm->tm_mon = bcd2bin(regs[RV3029_A_MO - RV3029_A_SC] & 0x1f) - 1;
+ tm->tm_year = bcd2bin(regs[RV3029_A_YR - RV3029_A_SC] & 0x7f) + 100;
+ tm->tm_wday = bcd2bin(regs[RV3029_A_DW - RV3029_A_SC] & 0x07) - 1;
+
+ alarm->enabled = !!(controls & RV3029_IRQ_CTRL_AIE);
+ alarm->pending = (flags & RV3029_IRQ_FLAGS_AF) && alarm->enabled;
return 0;
}
-static int
-rv3029_rtc_read_alarm(struct device *dev, struct rtc_wkalrm *alarm)
-{
- return rv3029_i2c_read_alarm(to_i2c_client(dev), alarm);
-}
-
-static int rv3029_rtc_i2c_alarm_set_irq(struct i2c_client *client,
- int enable)
+static int rv3029_alarm_irq_enable(struct device *dev, unsigned int enable)
{
int ret;
+ u8 controls;
+
+ ret = rv3029_read_regs(dev, RV3029_IRQ_CTRL, &controls, 1);
+ if (ret < 0) {
+ dev_warn(dev, "Read IRQ Control Register error %d\n", ret);
+ return ret;
+ }
/* enable/disable AIE irq */
- ret = rv3029_i2c_update_bits(client, RV3029_IRQ_CTRL,
- RV3029_IRQ_CTRL_AIE,
- (enable ? RV3029_IRQ_CTRL_AIE : 0));
+ if (enable)
+ controls |= RV3029_IRQ_CTRL_AIE;
+ else
+ controls &= ~RV3029_IRQ_CTRL_AIE;
+
+ ret = rv3029_write_regs(dev, RV3029_IRQ_CTRL, &controls, 1);
if (ret < 0) {
- dev_err(&client->dev, "can't update INT reg\n");
+ dev_err(dev, "can't update INT reg\n");
return ret;
}
return 0;
}
-static int rv3029_rtc_i2c_set_alarm(struct i2c_client *client,
- struct rtc_wkalrm *alarm)
+static int rv3029_set_alarm(struct device *dev, struct rtc_wkalrm *alarm)
{
struct rtc_time *const tm = &alarm->time;
int ret;
@@ -449,57 +498,48 @@
if (tm->tm_year < 100)
return -EINVAL;
- ret = rv3029_i2c_get_sr(client, regs);
+ ret = rv3029_get_sr(dev, regs);
if (ret < 0) {
- dev_err(&client->dev, "%s: reading SR failed\n", __func__);
+ dev_err(dev, "%s: reading SR failed\n", __func__);
return -EIO;
}
- regs[RV3029_A_SC-RV3029_A_SC] = bin2bcd(tm->tm_sec & 0x7f);
- regs[RV3029_A_MN-RV3029_A_SC] = bin2bcd(tm->tm_min & 0x7f);
- regs[RV3029_A_HR-RV3029_A_SC] = bin2bcd(tm->tm_hour & 0x3f);
- regs[RV3029_A_DT-RV3029_A_SC] = bin2bcd(tm->tm_mday & 0x3f);
- regs[RV3029_A_MO-RV3029_A_SC] = bin2bcd((tm->tm_mon & 0x1f) - 1);
- regs[RV3029_A_DW-RV3029_A_SC] = bin2bcd((tm->tm_wday & 7) - 1);
- regs[RV3029_A_YR-RV3029_A_SC] = bin2bcd((tm->tm_year & 0x7f) - 100);
- ret = rv3029_i2c_write_regs(client, RV3029_A_SC, regs,
- RV3029_ALARM_SECTION_LEN);
+ /* Activate all the alarms with AE_x bit */
+ regs[RV3029_A_SC - RV3029_A_SC] = bin2bcd(tm->tm_sec) | RV3029_A_AE_X;
+ regs[RV3029_A_MN - RV3029_A_SC] = bin2bcd(tm->tm_min) | RV3029_A_AE_X;
+ regs[RV3029_A_HR - RV3029_A_SC] = (bin2bcd(tm->tm_hour) & 0x3f)
+ | RV3029_A_AE_X;
+ regs[RV3029_A_DT - RV3029_A_SC] = (bin2bcd(tm->tm_mday) & 0x3f)
+ | RV3029_A_AE_X;
+ regs[RV3029_A_MO - RV3029_A_SC] = (bin2bcd(tm->tm_mon + 1) & 0x1f)
+ | RV3029_A_AE_X;
+ regs[RV3029_A_DW - RV3029_A_SC] = (bin2bcd(tm->tm_wday + 1) & 0x7)
+ | RV3029_A_AE_X;
+ regs[RV3029_A_YR - RV3029_A_SC] = (bin2bcd(tm->tm_year - 100))
+ | RV3029_A_AE_X;
+
+ /* Write the alarm */
+ ret = rv3029_write_regs(dev, RV3029_A_SC, regs,
+ RV3029_ALARM_SECTION_LEN);
if (ret < 0)
return ret;
if (alarm->enabled) {
- /* clear AF flag */
- ret = rv3029_i2c_update_bits(client, RV3029_IRQ_FLAGS,
- RV3029_IRQ_FLAGS_AF, 0);
- if (ret < 0) {
- dev_err(&client->dev, "can't clear alarm flag\n");
- return ret;
- }
/* enable AIE irq */
- ret = rv3029_rtc_i2c_alarm_set_irq(client, 1);
+ ret = rv3029_alarm_irq_enable(dev, 1);
if (ret)
return ret;
-
- dev_dbg(&client->dev, "alarm IRQ armed\n");
} else {
/* disable AIE irq */
- ret = rv3029_rtc_i2c_alarm_set_irq(client, 0);
+ ret = rv3029_alarm_irq_enable(dev, 0);
if (ret)
return ret;
-
- dev_dbg(&client->dev, "alarm IRQ disabled\n");
}
return 0;
}
-static int rv3029_rtc_set_alarm(struct device *dev, struct rtc_wkalrm *alarm)
-{
- return rv3029_rtc_i2c_set_alarm(to_i2c_client(dev), alarm);
-}
-
-static int
-rv3029_i2c_set_time(struct i2c_client *client, struct rtc_time const *tm)
+static int rv3029_set_time(struct device *dev, struct rtc_time *tm)
{
u8 regs[8];
int ret;
@@ -512,39 +552,34 @@
if (tm->tm_year < 100)
return -EINVAL;
- regs[RV3029_W_SEC-RV3029_W_SEC] = bin2bcd(tm->tm_sec);
- regs[RV3029_W_MINUTES-RV3029_W_SEC] = bin2bcd(tm->tm_min);
- regs[RV3029_W_HOURS-RV3029_W_SEC] = bin2bcd(tm->tm_hour);
- regs[RV3029_W_DATE-RV3029_W_SEC] = bin2bcd(tm->tm_mday);
- regs[RV3029_W_MONTHS-RV3029_W_SEC] = bin2bcd(tm->tm_mon+1);
- regs[RV3029_W_DAYS-RV3029_W_SEC] = bin2bcd((tm->tm_wday & 7)+1);
- regs[RV3029_W_YEARS-RV3029_W_SEC] = bin2bcd(tm->tm_year - 100);
+ regs[RV3029_W_SEC - RV3029_W_SEC] = bin2bcd(tm->tm_sec);
+ regs[RV3029_W_MINUTES - RV3029_W_SEC] = bin2bcd(tm->tm_min);
+ regs[RV3029_W_HOURS - RV3029_W_SEC] = bin2bcd(tm->tm_hour);
+ regs[RV3029_W_DATE - RV3029_W_SEC] = bin2bcd(tm->tm_mday);
+ regs[RV3029_W_MONTHS - RV3029_W_SEC] = bin2bcd(tm->tm_mon + 1);
+ regs[RV3029_W_DAYS - RV3029_W_SEC] = bin2bcd(tm->tm_wday + 1) & 0x7;
+ regs[RV3029_W_YEARS - RV3029_W_SEC] = bin2bcd(tm->tm_year - 100);
- ret = rv3029_i2c_write_regs(client, RV3029_W_SEC, regs,
- RV3029_WATCH_SECTION_LEN);
+ ret = rv3029_write_regs(dev, RV3029_W_SEC, regs,
+ RV3029_WATCH_SECTION_LEN);
if (ret < 0)
return ret;
- ret = rv3029_i2c_get_sr(client, regs);
+ ret = rv3029_get_sr(dev, regs);
if (ret < 0) {
- dev_err(&client->dev, "%s: reading SR failed\n", __func__);
+ dev_err(dev, "%s: reading SR failed\n", __func__);
return ret;
}
/* clear PON bit */
- ret = rv3029_i2c_set_sr(client, (regs[0] & ~RV3029_STATUS_PON));
+ ret = rv3029_set_sr(dev, (regs[0] & ~RV3029_STATUS_PON));
if (ret < 0) {
- dev_err(&client->dev, "%s: reading SR failed\n", __func__);
+ dev_err(dev, "%s: reading SR failed\n", __func__);
return ret;
}
return 0;
}
-static int rv3029_rtc_set_time(struct device *dev, struct rtc_time *tm)
-{
- return rv3029_i2c_set_time(to_i2c_client(dev), tm);
-}
-
static const struct rv3029_trickle_tab_elem {
u32 r; /* resistance in ohms */
u8 conf; /* trickle config bits */
@@ -602,9 +637,9 @@
},
};
-static void rv3029_trickle_config(struct i2c_client *client)
+static void rv3029_trickle_config(struct device *dev)
{
- struct device_node *of_node = client->dev.of_node;
+ struct device_node *of_node = dev->of_node;
const struct rv3029_trickle_tab_elem *elem;
int i, err;
u32 ohms;
@@ -626,27 +661,25 @@
break;
}
trickle_set_bits = elem->conf;
- dev_info(&client->dev,
+ dev_info(dev,
"Trickle charger enabled at %d ohms resistance.\n",
elem->r);
}
- err = rv3029_eeprom_update_bits(client, RV3029_CONTROL_E2P_EECTRL,
+ err = rv3029_eeprom_update_bits(dev, RV3029_CONTROL_E2P_EECTRL,
RV3029_TRICKLE_MASK,
trickle_set_bits);
- if (err < 0) {
- dev_err(&client->dev,
- "Failed to update trickle charger config\n");
- }
+ if (err < 0)
+ dev_err(dev, "Failed to update trickle charger config\n");
}
#ifdef CONFIG_RTC_DRV_RV3029_HWMON
-static int rv3029_read_temp(struct i2c_client *client, int *temp_mC)
+static int rv3029_read_temp(struct device *dev, int *temp_mC)
{
int ret;
u8 temp;
- ret = rv3029_i2c_read_regs(client, RV3029_TEMP_PAGE, &temp, 1);
+ ret = rv3029_read_regs(dev, RV3029_TEMP_PAGE, &temp, 1);
if (ret < 0)
return ret;
@@ -659,10 +692,9 @@
struct device_attribute *attr,
char *buf)
{
- struct i2c_client *client = dev_get_drvdata(dev);
int ret, temp_mC;
- ret = rv3029_read_temp(client, &temp_mC);
+ ret = rv3029_read_temp(dev, &temp_mC);
if (ret < 0)
return ret;
@@ -674,7 +706,6 @@
const char *buf,
size_t count)
{
- struct i2c_client *client = dev_get_drvdata(dev);
unsigned long interval_ms;
int ret;
u8 th_set_bits = 0;
@@ -688,7 +719,7 @@
if (interval_ms >= 16000)
th_set_bits |= RV3029_EECTRL_THP;
}
- ret = rv3029_eeprom_update_bits(client, RV3029_CONTROL_E2P_EECTRL,
+ ret = rv3029_eeprom_update_bits(dev, RV3029_CONTROL_E2P_EECTRL,
RV3029_EECTRL_THE | RV3029_EECTRL_THP,
th_set_bits);
if (ret < 0)
@@ -701,11 +732,10 @@
struct device_attribute *attr,
char *buf)
{
- struct i2c_client *client = dev_get_drvdata(dev);
int ret, interval_ms;
u8 eectrl;
- ret = rv3029_eeprom_read(client, RV3029_CONTROL_E2P_EECTRL,
+ ret = rv3029_eeprom_read(dev, RV3029_CONTROL_E2P_EECTRL,
&eectrl, 1);
if (ret < 0)
return ret;
@@ -735,34 +765,109 @@
};
ATTRIBUTE_GROUPS(rv3029_hwmon);
-static void rv3029_hwmon_register(struct i2c_client *client)
+static void rv3029_hwmon_register(struct device *dev, const char *name)
{
+ struct rv3029_data *rv3029 = dev_get_drvdata(dev);
struct device *hwmon_dev;
- hwmon_dev = devm_hwmon_device_register_with_groups(
- &client->dev, client->name, client, rv3029_hwmon_groups);
+ hwmon_dev = devm_hwmon_device_register_with_groups(dev, name, rv3029,
+ rv3029_hwmon_groups);
if (IS_ERR(hwmon_dev)) {
- dev_warn(&client->dev,
- "unable to register hwmon device %ld\n",
- PTR_ERR(hwmon_dev));
+ dev_warn(dev, "unable to register hwmon device %ld\n",
+ PTR_ERR(hwmon_dev));
}
}
#else /* CONFIG_RTC_DRV_RV3029_HWMON */
-static void rv3029_hwmon_register(struct i2c_client *client)
+static void rv3029_hwmon_register(struct device *dev, const char *name)
{
}
#endif /* CONFIG_RTC_DRV_RV3029_HWMON */
-static const struct rtc_class_ops rv3029_rtc_ops = {
- .read_time = rv3029_rtc_read_time,
- .set_time = rv3029_rtc_set_time,
- .read_alarm = rv3029_rtc_read_alarm,
- .set_alarm = rv3029_rtc_set_alarm,
+static struct rtc_class_ops rv3029_rtc_ops = {
+ .read_time = rv3029_read_time,
+ .set_time = rv3029_set_time,
};
+static int rv3029_probe(struct device *dev, struct regmap *regmap, int irq,
+ const char *name)
+{
+ struct rv3029_data *rv3029;
+ int rc = 0;
+ u8 buf[1];
+
+ rv3029 = devm_kzalloc(dev, sizeof(*rv3029), GFP_KERNEL);
+ if (!rv3029)
+ return -ENOMEM;
+
+ rv3029->regmap = regmap;
+ rv3029->irq = irq;
+ rv3029->dev = dev;
+ dev_set_drvdata(dev, rv3029);
+
+ rc = rv3029_get_sr(dev, buf);
+ if (rc < 0) {
+ dev_err(dev, "reading status failed\n");
+ return rc;
+ }
+
+ rv3029_trickle_config(dev);
+ rv3029_hwmon_register(dev, name);
+
+ rv3029->rtc = devm_rtc_device_register(dev, name, &rv3029_rtc_ops,
+ THIS_MODULE);
+ if (IS_ERR(rv3029->rtc)) {
+ dev_err(dev, "unable to register the class device\n");
+ return PTR_ERR(rv3029->rtc);
+ }
+
+ if (rv3029->irq > 0) {
+ rc = devm_request_threaded_irq(dev, rv3029->irq,
+ NULL, rv3029_handle_irq,
+ IRQF_TRIGGER_LOW | IRQF_ONESHOT,
+ "rv3029", dev);
+ if (rc) {
+ dev_warn(dev, "unable to request IRQ, alarms disabled\n");
+ rv3029->irq = 0;
+ } else {
+ rv3029_rtc_ops.read_alarm = rv3029_read_alarm;
+ rv3029_rtc_ops.set_alarm = rv3029_set_alarm;
+ rv3029_rtc_ops.alarm_irq_enable = rv3029_alarm_irq_enable;
+ }
+ }
+
+ return 0;
+}
+
+#if IS_ENABLED(CONFIG_I2C)
+
+static int rv3029_i2c_probe(struct i2c_client *client,
+ const struct i2c_device_id *id)
+{
+ struct regmap *regmap;
+ static const struct regmap_config config = {
+ .reg_bits = 8,
+ .val_bits = 8,
+ };
+
+ if (!i2c_check_functionality(client->adapter, I2C_FUNC_SMBUS_I2C_BLOCK |
+ I2C_FUNC_SMBUS_BYTE)) {
+ dev_err(&client->dev, "Adapter does not support SMBUS_I2C_BLOCK or SMBUS_I2C_BYTE\n");
+ return -ENODEV;
+ }
+
+ regmap = devm_regmap_init_i2c(client, &config);
+ if (IS_ERR(regmap)) {
+ dev_err(&client->dev, "%s: regmap allocation failed: %ld\n",
+ __func__, PTR_ERR(regmap));
+ return PTR_ERR(regmap);
+ }
+
+ return rv3029_probe(&client->dev, regmap, client->irq, client->name);
+}
+
static struct i2c_device_id rv3029_id[] = {
{ "rv3029", 0 },
{ "rv3029c2", 0 },
@@ -770,47 +875,116 @@
};
MODULE_DEVICE_TABLE(i2c, rv3029_id);
-static int rv3029_probe(struct i2c_client *client,
- const struct i2c_device_id *id)
-{
- struct rtc_device *rtc;
- int rc = 0;
- u8 buf[1];
-
- if (!i2c_check_functionality(client->adapter, I2C_FUNC_SMBUS_EMUL))
- return -ENODEV;
-
- rc = rv3029_i2c_get_sr(client, buf);
- if (rc < 0) {
- dev_err(&client->dev, "reading status failed\n");
- return rc;
- }
-
- rv3029_trickle_config(client);
- rv3029_hwmon_register(client);
-
- rtc = devm_rtc_device_register(&client->dev, client->name,
- &rv3029_rtc_ops, THIS_MODULE);
-
- if (IS_ERR(rtc))
- return PTR_ERR(rtc);
-
- i2c_set_clientdata(client, rtc);
-
- return 0;
-}
-
static struct i2c_driver rv3029_driver = {
.driver = {
.name = "rtc-rv3029c2",
},
- .probe = rv3029_probe,
+ .probe = rv3029_i2c_probe,
.id_table = rv3029_id,
};
-module_i2c_driver(rv3029_driver);
+static int rv3029_register_driver(void)
+{
+ return i2c_add_driver(&rv3029_driver);
+}
+
+static void rv3029_unregister_driver(void)
+{
+ i2c_del_driver(&rv3029_driver);
+}
+
+#else
+
+static int rv3029_register_driver(void)
+{
+ return 0;
+}
+
+static void rv3029_unregister_driver(void)
+{
+}
+
+#endif
+
+#if IS_ENABLED(CONFIG_SPI_MASTER)
+
+static int rv3049_probe(struct spi_device *spi)
+{
+ static const struct regmap_config config = {
+ .reg_bits = 8,
+ .val_bits = 8,
+ };
+ struct regmap *regmap;
+
+ regmap = devm_regmap_init_spi(spi, &config);
+ if (IS_ERR(regmap)) {
+ dev_err(&spi->dev, "%s: regmap allocation failed: %ld\n",
+ __func__, PTR_ERR(regmap));
+ return PTR_ERR(regmap);
+ }
+
+ return rv3029_probe(&spi->dev, regmap, spi->irq, "rv3049");
+}
+
+static struct spi_driver rv3049_driver = {
+ .driver = {
+ .name = "rv3049",
+ },
+ .probe = rv3049_probe,
+};
+
+static int rv3049_register_driver(void)
+{
+ return spi_register_driver(&rv3049_driver);
+}
+
+static void rv3049_unregister_driver(void)
+{
+ spi_unregister_driver(&rv3049_driver);
+}
+
+#else
+
+static int rv3049_register_driver(void)
+{
+ return 0;
+}
+
+static void rv3049_unregister_driver(void)
+{
+}
+
+#endif
+
+static int __init rv30x9_init(void)
+{
+ int ret;
+
+ ret = rv3029_register_driver();
+ if (ret) {
+ pr_err("Failed to register rv3029 driver: %d\n", ret);
+ return ret;
+ }
+
+ ret = rv3049_register_driver();
+ if (ret) {
+ pr_err("Failed to register rv3049 driver: %d\n", ret);
+ rv3029_unregister_driver();
+ }
+
+ return ret;
+}
+module_init(rv30x9_init)
+
+static void __exit rv30x9_exit(void)
+{
+ rv3049_unregister_driver();
+ rv3029_unregister_driver();
+}
+module_exit(rv30x9_exit)
MODULE_AUTHOR("Gregory Hermant <gregory.hermant@calao-systems.com>");
MODULE_AUTHOR("Michael Buesch <m@bues.ch>");
-MODULE_DESCRIPTION("Micro Crystal RV3029 RTC driver");
+MODULE_DESCRIPTION("Micro Crystal RV3029/RV3049 RTC driver");
MODULE_LICENSE("GPL");
+MODULE_ALIAS("spi:rv3049");
diff --git a/drivers/rtc/rtc-rx8581.c b/drivers/rtc/rtc-rx8581.c
index 161e25d0..0c362a3 100644
--- a/drivers/rtc/rtc-rx8581.c
+++ b/drivers/rtc/rtc-rx8581.c
@@ -18,8 +18,6 @@
#include <linux/rtc.h>
#include <linux/log2.h>
-#define DRV_VERSION "0.1"
-
#define RX8581_REG_SC 0x00 /* Second in BCD */
#define RX8581_REG_MN 0x01 /* Minute in BCD */
#define RX8581_REG_HR 0x02 /* Hour in BCD */
@@ -292,8 +290,6 @@
rx8581->write_block_data = rx8581_write_block_data;
}
- dev_info(&client->dev, "chip found, driver version " DRV_VERSION "\n");
-
rx8581->rtc = devm_rtc_device_register(&client->dev,
rx8581_driver.driver.name, &rx8581_rtc_ops, THIS_MODULE);
@@ -325,4 +321,3 @@
MODULE_AUTHOR("Martyn Welch <martyn.welch@ge.com>");
MODULE_DESCRIPTION("Epson RX-8581 RTC driver");
MODULE_LICENSE("GPL");
-MODULE_VERSION(DRV_VERSION);
diff --git a/drivers/rtc/rtc-sh.c b/drivers/rtc/rtc-sh.c
index 2b81dd4..a45845a 100644
--- a/drivers/rtc/rtc-sh.c
+++ b/drivers/rtc/rtc-sh.c
@@ -30,7 +30,6 @@
#include <asm/rtc.h>
#define DRV_NAME "sh-rtc"
-#define DRV_VERSION "0.2.3"
#define RTC_REG(r) ((r) * rtc_reg_size)
@@ -790,7 +789,6 @@
module_platform_driver_probe(sh_rtc_platform_driver, sh_rtc_probe);
MODULE_DESCRIPTION("SuperH on-chip RTC driver");
-MODULE_VERSION(DRV_VERSION);
MODULE_AUTHOR("Paul Mundt <lethal@linux-sh.org>, "
"Jamie Lenehan <lenehan@twibble.org>, "
"Angelo Castello <angelo.castello@st.com>");
diff --git a/drivers/rtc/rtc-snvs.c b/drivers/rtc/rtc-snvs.c
index 950c5d0..0f11c2a 100644
--- a/drivers/rtc/rtc-snvs.c
+++ b/drivers/rtc/rtc-snvs.c
@@ -322,7 +322,7 @@
struct snvs_rtc_data *data = dev_get_drvdata(dev);
if (device_may_wakeup(dev))
- enable_irq_wake(data->irq);
+ return enable_irq_wake(data->irq);
return 0;
}
diff --git a/drivers/rtc/rtc-stk17ta8.c b/drivers/rtc/rtc-stk17ta8.c
index ba6a83b..a456cb6 100644
--- a/drivers/rtc/rtc-stk17ta8.c
+++ b/drivers/rtc/rtc-stk17ta8.c
@@ -23,8 +23,6 @@
#include <linux/io.h>
#include <linux/module.h>
-#define DRV_VERSION "0.1"
-
#define RTC_REG_SIZE 0x20000
#define RTC_OFFSET 0x1fff0
@@ -366,4 +364,3 @@
MODULE_AUTHOR("Thomas Hommel <thomas.hommel@ge.com>");
MODULE_DESCRIPTION("Simtek STK17TA8 RTC driver");
MODULE_LICENSE("GPL");
-MODULE_VERSION(DRV_VERSION);
diff --git a/drivers/rtc/rtc-stmp3xxx.c b/drivers/rtc/rtc-stmp3xxx.c
index ca54d03..e6aaaa5 100644
--- a/drivers/rtc/rtc-stmp3xxx.c
+++ b/drivers/rtc/rtc-stmp3xxx.c
@@ -107,14 +107,19 @@
static void stmp3xxx_wdt_register(struct platform_device *rtc_pdev)
{
+ int rc = -1;
struct platform_device *wdt_pdev =
platform_device_alloc("stmp3xxx_rtc_wdt", rtc_pdev->id);
if (wdt_pdev) {
wdt_pdev->dev.parent = &rtc_pdev->dev;
wdt_pdev->dev.platform_data = &wdt_pdata;
- platform_device_add(wdt_pdev);
+ rc = platform_device_add(wdt_pdev);
}
+
+ if (rc)
+ dev_err(&rtc_pdev->dev,
+ "failed to register stmp3xxx_rtc_wdt\n");
}
#else
static void stmp3xxx_wdt_register(struct platform_device *rtc_pdev)
diff --git a/drivers/rtc/rtc-tps6586x.c b/drivers/rtc/rtc-tps6586x.c
index e404faa..a3418a8 100644
--- a/drivers/rtc/rtc-tps6586x.c
+++ b/drivers/rtc/rtc-tps6586x.c
@@ -344,7 +344,7 @@
};
module_platform_driver(tps6586x_rtc_driver);
-MODULE_ALIAS("platform:rtc-tps6586x");
+MODULE_ALIAS("platform:tps6586x-rtc");
MODULE_DESCRIPTION("TI TPS6586x RTC driver");
MODULE_AUTHOR("Laxman dewangan <ldewangan@nvidia.com>");
MODULE_LICENSE("GPL v2");
diff --git a/drivers/rtc/rtc-x1205.c b/drivers/rtc/rtc-x1205.c
index 5638b7b..f08f18e 100644
--- a/drivers/rtc/rtc-x1205.c
+++ b/drivers/rtc/rtc-x1205.c
@@ -24,8 +24,6 @@
#include <linux/module.h>
#include <linux/bitops.h>
-#define DRV_VERSION "1.0.8"
-
/* offsets into CCR area */
#define CCR_SEC 0
@@ -634,8 +632,6 @@
if (x1205_validate_client(client) < 0)
return -ENODEV;
- dev_info(&client->dev, "chip found, driver version " DRV_VERSION "\n");
-
rtc = devm_rtc_device_register(&client->dev, x1205_driver.driver.name,
&x1205_rtc_ops, THIS_MODULE);
@@ -693,4 +689,3 @@
"Alessandro Zummo <a.zummo@towertech.it>");
MODULE_DESCRIPTION("Xicor/Intersil X1205 RTC driver");
MODULE_LICENSE("GPL");
-MODULE_VERSION(DRV_VERSION);
diff --git a/drivers/rtc/rtc-zynqmp.c b/drivers/rtc/rtc-zynqmp.c
index 8b28762..da18a8a 100644
--- a/drivers/rtc/rtc-zynqmp.c
+++ b/drivers/rtc/rtc-zynqmp.c
@@ -45,6 +45,7 @@
#define RTC_INT_SEC BIT(0)
#define RTC_INT_ALRM BIT(1)
#define RTC_OSC_EN BIT(24)
+#define RTC_BATT_EN BIT(31)
#define RTC_CALIB_DEF 0x198233
#define RTC_CALIB_MASK 0x1FFFFF
@@ -55,6 +56,7 @@
void __iomem *reg_base;
int alarm_irq;
int sec_irq;
+ int calibval;
};
static int xlnx_rtc_set_time(struct device *dev, struct rtc_time *tm)
@@ -62,21 +64,63 @@
struct xlnx_rtc_dev *xrtcdev = dev_get_drvdata(dev);
unsigned long new_time;
- new_time = rtc_tm_to_time64(tm);
+ /*
+ * The value written will be updated after 1 sec into the
+ * seconds read register, so we need to program time +1 sec
+ * to get the correct time on read.
+ */
+ new_time = rtc_tm_to_time64(tm) + 1;
if (new_time > RTC_SEC_MAX_VAL)
return -EINVAL;
+ /*
+ * Writing into calibration register will clear the Tick Counter and
+ * force the next second to be signaled exactly in 1 second period
+ */
+ xrtcdev->calibval &= RTC_CALIB_MASK;
+ writel(xrtcdev->calibval, (xrtcdev->reg_base + RTC_CALIB_WR));
+
writel(new_time, xrtcdev->reg_base + RTC_SET_TM_WR);
+ /*
+ * Clear the rtc interrupt status register after setting the
+ * time. During a read_time function, the code should read the
+ * RTC_INT_STATUS register and if bit 0 is still 0, it means
+ * that one second has not elapsed yet since RTC was set and
+ * the current time should be read from SET_TIME_READ register;
+ * otherwise, CURRENT_TIME register is read to report the time
+ */
+ writel(RTC_INT_SEC, xrtcdev->reg_base + RTC_INT_STS);
+
return 0;
}
static int xlnx_rtc_read_time(struct device *dev, struct rtc_time *tm)
{
+ u32 status;
+ unsigned long read_time;
struct xlnx_rtc_dev *xrtcdev = dev_get_drvdata(dev);
- rtc_time64_to_tm(readl(xrtcdev->reg_base + RTC_CUR_TM), tm);
+ status = readl(xrtcdev->reg_base + RTC_INT_STS);
+
+ if (status & RTC_INT_SEC) {
+ /*
+ * RTC has updated the CURRENT_TIME with the time written into
+ * SET_TIME_WRITE register.
+ */
+ rtc_time64_to_tm(readl(xrtcdev->reg_base + RTC_CUR_TM), tm);
+ } else {
+ /*
+ * Time written in SET_TIME_WRITE has not yet updated into
+ * the seconds read register, so read the time from the
+ * SET_TIME_WRITE instead of CURRENT_TIME register.
+ * Since we add +1 sec while writing, we need to -1 sec while
+ * reading.
+ */
+ read_time = readl(xrtcdev->reg_base + RTC_SET_TM_RD) - 1;
+ rtc_time64_to_tm(read_time, tm);
+ }
return rtc_valid_tm(tm);
}
@@ -120,16 +164,23 @@
return 0;
}
-static void xlnx_init_rtc(struct xlnx_rtc_dev *xrtcdev, u32 calibval)
+static void xlnx_init_rtc(struct xlnx_rtc_dev *xrtcdev)
{
+ u32 rtc_ctrl;
+
+ /* Enable RTC switch to battery when VCC_PSAUX is not available */
+ rtc_ctrl = readl(xrtcdev->reg_base + RTC_CTRL);
+ rtc_ctrl |= RTC_BATT_EN;
+ writel(rtc_ctrl, xrtcdev->reg_base + RTC_CTRL);
+
/*
* Based on crystal freq of 33.330 KHz
* set the seconds counter and enable, set fractions counter
* to default value suggested as per design spec
* to correct RTC delay in frequency over period of time.
*/
- calibval &= RTC_CALIB_MASK;
- writel(calibval, (xrtcdev->reg_base + RTC_CALIB_WR));
+ xrtcdev->calibval &= RTC_CALIB_MASK;
+ writel(xrtcdev->calibval, (xrtcdev->reg_base + RTC_CALIB_WR));
}
static const struct rtc_class_ops xlnx_rtc_ops = {
@@ -150,11 +201,9 @@
if (!(status & (RTC_INT_SEC | RTC_INT_ALRM)))
return IRQ_NONE;
- /* Clear interrupt */
- writel(status, xrtcdev->reg_base + RTC_INT_STS);
+ /* Clear RTC_INT_ALRM interrupt only */
+ writel(RTC_INT_ALRM, xrtcdev->reg_base + RTC_INT_STS);
- if (status & RTC_INT_SEC)
- rtc_update_irq(xrtcdev->rtc, 1, RTC_IRQF | RTC_UF);
if (status & RTC_INT_ALRM)
rtc_update_irq(xrtcdev->rtc, 1, RTC_IRQF | RTC_AF);
@@ -166,7 +215,6 @@
struct xlnx_rtc_dev *xrtcdev;
struct resource *res;
int ret;
- unsigned int calibvalue;
xrtcdev = devm_kzalloc(&pdev->dev, sizeof(*xrtcdev), GFP_KERNEL);
if (!xrtcdev)
@@ -207,11 +255,11 @@
}
ret = of_property_read_u32(pdev->dev.of_node, "calibration",
- &calibvalue);
+ &xrtcdev->calibval);
if (ret)
- calibvalue = RTC_CALIB_DEF;
+ xrtcdev->calibval = RTC_CALIB_DEF;
- xlnx_init_rtc(xrtcdev, calibvalue);
+ xlnx_init_rtc(xrtcdev);
device_init_wakeup(&pdev->dev, 1);
diff --git a/drivers/s390/char/con3215.c b/drivers/s390/char/con3215.c
index e7e078b..931d10e 100644
--- a/drivers/s390/char/con3215.c
+++ b/drivers/s390/char/con3215.c
@@ -289,7 +289,7 @@
spin_lock_irqsave(get_ccwdev_lock(raw->cdev), flags);
raw->flags &= ~RAW3215_TIMER_RUNS;
- if (!(raw->port.flags & ASYNC_SUSPENDED)) {
+ if (!tty_port_suspended(&raw->port)) {
raw3215_mk_write_req(raw);
raw3215_start_io(raw);
if ((raw->queued_read || raw->queued_write) &&
@@ -311,8 +311,7 @@
*/
static inline void raw3215_try_io(struct raw3215_info *raw)
{
- if (!(raw->port.flags & ASYNC_INITIALIZED) ||
- (raw->port.flags & ASYNC_SUSPENDED))
+ if (!tty_port_initialized(&raw->port) || tty_port_suspended(&raw->port))
return;
if (raw->queued_read != NULL)
raw3215_start_io(raw);
@@ -494,7 +493,7 @@
/* While console is frozen for suspend we have no other
* choice but to drop message from the buffer to make
* room for even more messages. */
- if (raw->port.flags & ASYNC_SUSPENDED) {
+ if (tty_port_suspended(&raw->port)) {
raw3215_drop_line(raw);
continue;
}
@@ -616,10 +615,10 @@
{
unsigned long flags;
- if (raw->port.flags & ASYNC_INITIALIZED)
+ if (tty_port_initialized(&raw->port))
return 0;
raw->line_pos = 0;
- raw->port.flags |= ASYNC_INITIALIZED;
+ tty_port_set_initialized(&raw->port, 1);
spin_lock_irqsave(get_ccwdev_lock(raw->cdev), flags);
raw3215_try_io(raw);
spin_unlock_irqrestore(get_ccwdev_lock(raw->cdev), flags);
@@ -635,8 +634,7 @@
DECLARE_WAITQUEUE(wait, current);
unsigned long flags;
- if (!(raw->port.flags & ASYNC_INITIALIZED) ||
- (raw->flags & RAW3215_FIXED))
+ if (!tty_port_initialized(&raw->port) || (raw->flags & RAW3215_FIXED))
return;
/* Wait for outstanding requests, then free irq */
spin_lock_irqsave(get_ccwdev_lock(raw->cdev), flags);
@@ -650,7 +648,7 @@
spin_lock_irqsave(get_ccwdev_lock(raw->cdev), flags);
remove_wait_queue(&raw->empty_wait, &wait);
set_current_state(TASK_RUNNING);
- raw->port.flags &= ~ASYNC_INITIALIZED;
+ tty_port_set_initialized(&raw->port, 1);
}
spin_unlock_irqrestore(get_ccwdev_lock(raw->cdev), flags);
}
@@ -773,7 +771,7 @@
raw = dev_get_drvdata(&cdev->dev);
spin_lock_irqsave(get_ccwdev_lock(raw->cdev), flags);
raw3215_make_room(raw, RAW3215_BUFFER_SIZE);
- raw->port.flags |= ASYNC_SUSPENDED;
+ tty_port_set_suspended(&raw->port, 1);
spin_unlock_irqrestore(get_ccwdev_lock(raw->cdev), flags);
return 0;
}
@@ -786,7 +784,7 @@
/* Allow I/O again and flush output buffer. */
raw = dev_get_drvdata(&cdev->dev);
spin_lock_irqsave(get_ccwdev_lock(raw->cdev), flags);
- raw->port.flags &= ~ASYNC_SUSPENDED;
+ tty_port_set_suspended(&raw->port, 0);
raw->flags |= RAW3215_FLUSHING;
raw3215_try_io(raw);
raw->flags &= ~RAW3215_FLUSHING;
@@ -859,7 +857,7 @@
unsigned long flags;
raw = raw3215[0]; /* console 3215 is the first one */
- if (raw->port.flags & ASYNC_SUSPENDED)
+ if (tty_port_suspended(&raw->port))
/* The console is still frozen for suspend. */
if (ccw_device_force_console(raw->cdev))
/* Forcing didn't work, no panic message .. */
diff --git a/drivers/s390/char/sclp_early.c b/drivers/s390/char/sclp_early.c
index 6804354..0ac520d 100644
--- a/drivers/s390/char/sclp_early.c
+++ b/drivers/s390/char/sclp_early.c
@@ -49,7 +49,9 @@
u8 _pad_117[119 - 117]; /* 117-118 */
u8 fac119; /* 119 */
u16 hcpua; /* 120-121 */
- u8 _pad_122[4096 - 122]; /* 122-4095 */
+ u8 _pad_122[124 - 122]; /* 122-123 */
+ u32 hmfai; /* 124-127 */
+ u8 _pad_128[4096 - 128]; /* 128-4095 */
} __packed __aligned(PAGE_SIZE);
static char sccb_early[PAGE_SIZE] __aligned(PAGE_SIZE) __initdata;
@@ -155,6 +157,8 @@
sclp.mtid = (sccb->fac42 & 0x80) ? (sccb->fac42 & 31) : 0;
sclp.mtid_cp = (sccb->fac42 & 0x80) ? (sccb->fac43 & 31) : 0;
sclp.mtid_prev = (sccb->fac42 & 0x80) ? (sccb->fac66 & 31) : 0;
+
+ sclp.hmfai = sccb->hmfai;
}
/*
diff --git a/drivers/s390/char/tty3270.c b/drivers/s390/char/tty3270.c
index 0a9f219..272cb6c 100644
--- a/drivers/s390/char/tty3270.c
+++ b/drivers/s390/char/tty3270.c
@@ -1860,7 +1860,7 @@
tp = tty->driver_data;
if (!tp)
return -ENODEV;
- if (tty->flags & (1 << TTY_IO_ERROR))
+ if (tty_io_error(tty))
return -EIO;
return kbd_ioctl(tp->kbd, cmd, arg);
}
@@ -1874,7 +1874,7 @@
tp = tty->driver_data;
if (!tp)
return -ENODEV;
- if (tty->flags & (1 << TTY_IO_ERROR))
+ if (tty_io_error(tty))
return -EIO;
return kbd_ioctl(tp->kbd, cmd, (unsigned long)compat_ptr(arg));
}
diff --git a/drivers/s390/scsi/zfcp_unit.c b/drivers/s390/scsi/zfcp_unit.c
index 157d3d2..9310a54 100644
--- a/drivers/s390/scsi/zfcp_unit.c
+++ b/drivers/s390/scsi/zfcp_unit.c
@@ -26,7 +26,8 @@
lun = scsilun_to_int((struct scsi_lun *) &unit->fcp_lun);
if (rport && rport->port_state == FC_PORTSTATE_ONLINE)
- scsi_scan_target(&rport->dev, 0, rport->scsi_target_id, lun, 1);
+ scsi_scan_target(&rport->dev, 0, rport->scsi_target_id, lun,
+ SCSI_SCAN_MANUAL);
}
static void zfcp_unit_scsi_scan_work(struct work_struct *work)
diff --git a/drivers/sbus/char/openprom.c b/drivers/sbus/char/openprom.c
index e077ebd..4612691 100644
--- a/drivers/sbus/char/openprom.c
+++ b/drivers/sbus/char/openprom.c
@@ -383,20 +383,12 @@
}
/* Copy in a whole string from userspace into kernelspace. */
-static int copyin_string(char __user *user, size_t len, char **ptr)
+static char * copyin_string(char __user *user, size_t len)
{
- char *tmp;
-
if ((ssize_t)len < 0 || (ssize_t)(len + 1) < 0)
- return -EINVAL;
+ return ERR_PTR(-EINVAL);
- tmp = memdup_user_nul(user, len);
- if (IS_ERR(tmp))
- return PTR_ERR(tmp);
-
- *ptr = tmp;
-
- return 0;
+ return memdup_user_nul(user, len);
}
/*
@@ -415,9 +407,9 @@
dp = get_node(op.op_nodeid, data);
- err = copyin_string(op.op_name, op.op_namelen, &str);
- if (err)
- return err;
+ str = copyin_string(op.op_name, op.op_namelen);
+ if (IS_ERR(str))
+ return PTR_ERR(str);
pval = of_get_property(dp, str, &len);
err = 0;
@@ -440,7 +432,7 @@
struct device_node *dp;
struct property *prop;
char *str;
- int err, len;
+ int len;
if (copy_from_user(&op, argp, sizeof(op)))
return -EFAULT;
@@ -449,9 +441,9 @@
if (!dp)
return -EINVAL;
- err = copyin_string(op.op_name, op.op_namelen, &str);
- if (err)
- return err;
+ str = copyin_string(op.op_name, op.op_namelen);
+ if (IS_ERR(str))
+ return PTR_ERR(str);
if (str[0] == '\0') {
prop = dp->properties;
@@ -494,14 +486,14 @@
if (!dp)
return -EINVAL;
- err = copyin_string(op.op_name, op.op_namelen, &str);
- if (err)
- return err;
+ str = copyin_string(op.op_name, op.op_namelen);
+ if (IS_ERR(str))
+ return PTR_ERR(str);
- err = copyin_string(op.op_buf, op.op_buflen, &tmp);
- if (err) {
+ tmp = copyin_string(op.op_buf, op.op_buflen);
+ if (IS_ERR(tmp)) {
kfree(str);
- return err;
+ return PTR_ERR(tmp);
}
err = of_set_property(dp, str, tmp, op.op_buflen);
diff --git a/drivers/scsi/Kconfig b/drivers/scsi/Kconfig
index e80768f..98e5d51 100644
--- a/drivers/scsi/Kconfig
+++ b/drivers/scsi/Kconfig
@@ -17,6 +17,7 @@
tristate "SCSI device support"
depends on BLOCK
select SCSI_DMA if HAS_DMA
+ select SG_POOL
---help---
If you want to use a SCSI hard disk, SCSI tape drive, SCSI CD-ROM or
any other SCSI device under Linux, say Y and make sure that you know
@@ -202,12 +203,12 @@
certain enclosure conditions to be reported and is not required.
config SCSI_CONSTANTS
- bool "Verbose SCSI error reporting (kernel size +=75K)"
+ bool "Verbose SCSI error reporting (kernel size += 36K)"
depends on SCSI
help
The error messages regarding your SCSI hardware will be easier to
understand if you say Y here; it will enlarge your kernel by about
- 75 KB. If in doubt, say Y.
+ 36 KB. If in doubt, say Y.
config SCSI_LOGGING
bool "SCSI logging facility"
@@ -813,17 +814,6 @@
To compile this driver as a module, choose M here: the
module will be called g_NCR5380_mmio.
-config SCSI_GENERIC_NCR53C400
- bool "Enable NCR53c400 extensions"
- depends on SCSI_GENERIC_NCR5380
- help
- This enables certain optimizations for the NCR53c400 SCSI cards.
- You might as well try it out. Note that this driver will only probe
- for the Trantor T130B in its default configuration; you might have
- to pass a command line option to the kernel at boot time if it does
- not detect your card. See the file
- <file:Documentation/scsi/g_NCR5380.txt> for details.
-
config SCSI_IPS
tristate "IBM ServeRAID support"
depends on PCI && SCSI
diff --git a/drivers/scsi/NCR5380.c b/drivers/scsi/NCR5380.c
index 3eff2a6..43908bb 100644
--- a/drivers/scsi/NCR5380.c
+++ b/drivers/scsi/NCR5380.c
@@ -29,29 +29,9 @@
* Ronald van Cuijlenborg, Alan Cox and others.
*/
-/*
- * Further development / testing that should be done :
- * 1. Cleanup the NCR5380_transfer_dma function and DMA operation complete
- * code so that everything does the same thing that's done at the
- * end of a pseudo-DMA read operation.
- *
- * 2. Fix REAL_DMA (interrupt driven, polled works fine) -
- * basically, transfer size needs to be reduced by one
- * and the last byte read as is done with PSEUDO_DMA.
- *
- * 4. Test SCSI-II tagged queueing (I have no devices which support
- * tagged queueing)
- */
+/* Ported to Atari by Roman Hodek and others. */
-#ifndef notyet
-#undef REAL_DMA
-#endif
-
-#ifdef BOARD_REQUIRES_NO_DELAY
-#define io_recovery_delay(x)
-#else
-#define io_recovery_delay(x) udelay(x)
-#endif
+/* Adapted for the Sun 3 by Sam Creasey. */
/*
* Design
@@ -126,17 +106,10 @@
* DIFFERENTIAL - if defined, NCR53c81 chips will use external differential
* transceivers.
*
- * DONT_USE_INTR - if defined, never use interrupts, even if we probe or
- * override-configure an IRQ.
- *
* PSEUDO_DMA - if defined, PSEUDO DMA is used during the data transfer phases.
*
* REAL_DMA - if defined, REAL DMA is used during the data transfer phases.
*
- * REAL_DMA_POLL - if defined, REAL DMA is used but the driver doesn't
- * rely on phase mismatch and EOP interrupts to determine end
- * of phase.
- *
* These macros MUST be defined :
*
* NCR5380_read(register) - read from the specified register
@@ -147,29 +120,29 @@
* specific implementation of the NCR5380
*
* Either real DMA *or* pseudo DMA may be implemented
- * REAL functions :
- * NCR5380_REAL_DMA should be defined if real DMA is to be used.
- * Note that the DMA setup functions should return the number of bytes
- * that they were able to program the controller for.
- *
- * Also note that generic i386/PC versions of these macros are
- * available as NCR5380_i386_dma_write_setup,
- * NCR5380_i386_dma_read_setup, and NCR5380_i386_dma_residual.
*
* NCR5380_dma_write_setup(instance, src, count) - initialize
* NCR5380_dma_read_setup(instance, dst, count) - initialize
* NCR5380_dma_residual(instance); - residual count
*
- * PSEUDO functions :
- * NCR5380_pwrite(instance, src, count)
- * NCR5380_pread(instance, dst, count);
- *
* The generic driver is initialized by calling NCR5380_init(instance),
* after setting the appropriate host specific fields and ID. If the
* driver wishes to autoprobe for an IRQ line, the NCR5380_probe_irq(instance,
* possible) function may be used.
*/
+#ifndef NCR5380_io_delay
+#define NCR5380_io_delay(x)
+#endif
+
+#ifndef NCR5380_acquire_dma_irq
+#define NCR5380_acquire_dma_irq(x) (1)
+#endif
+
+#ifndef NCR5380_release_dma_irq
+#define NCR5380_release_dma_irq(x)
+#endif
+
static int do_abort(struct Scsi_Host *);
static void do_reset(struct Scsi_Host *);
@@ -280,12 +253,20 @@
{0, NULL}
},
basrs[] = {
+ {BASR_END_DMA_TRANSFER, "END OF DMA"},
+ {BASR_DRQ, "DRQ"},
+ {BASR_PARITY_ERROR, "PARITY ERROR"},
+ {BASR_IRQ, "IRQ"},
+ {BASR_PHASE_MATCH, "PHASE MATCH"},
+ {BASR_BUSY_ERROR, "BUSY ERROR"},
{BASR_ATN, "ATN"},
{BASR_ACK, "ACK"},
{0, NULL}
},
icrs[] = {
{ICR_ASSERT_RST, "ASSERT RST"},
+ {ICR_ARBITRATION_PROGRESS, "ARB. IN PROGRESS"},
+ {ICR_ARBITRATION_LOST, "LOST ARB."},
{ICR_ASSERT_ACK, "ASSERT ACK"},
{ICR_ASSERT_BSY, "ASSERT BSY"},
{ICR_ASSERT_SEL, "ASSERT SEL"},
@@ -294,14 +275,14 @@
{0, NULL}
},
mrs[] = {
- {MR_BLOCK_DMA_MODE, "MODE BLOCK DMA"},
- {MR_TARGET, "MODE TARGET"},
- {MR_ENABLE_PAR_CHECK, "MODE PARITY CHECK"},
- {MR_ENABLE_PAR_INTR, "MODE PARITY INTR"},
- {MR_ENABLE_EOP_INTR, "MODE EOP INTR"},
- {MR_MONITOR_BSY, "MODE MONITOR BSY"},
- {MR_DMA_MODE, "MODE DMA"},
- {MR_ARBITRATE, "MODE ARBITRATION"},
+ {MR_BLOCK_DMA_MODE, "BLOCK DMA MODE"},
+ {MR_TARGET, "TARGET"},
+ {MR_ENABLE_PAR_CHECK, "PARITY CHECK"},
+ {MR_ENABLE_PAR_INTR, "PARITY INTR"},
+ {MR_ENABLE_EOP_INTR, "EOP INTR"},
+ {MR_MONITOR_BSY, "MONITOR BSY"},
+ {MR_DMA_MODE, "DMA MODE"},
+ {MR_ARBITRATE, "ARBITRATE"},
{0, NULL}
};
@@ -322,23 +303,23 @@
icr = NCR5380_read(INITIATOR_COMMAND_REG);
basr = NCR5380_read(BUS_AND_STATUS_REG);
- printk("STATUS_REG: %02x ", status);
+ printk(KERN_DEBUG "SR = 0x%02x : ", status);
for (i = 0; signals[i].mask; ++i)
if (status & signals[i].mask)
- printk(",%s", signals[i].name);
- printk("\nBASR: %02x ", basr);
+ printk(KERN_CONT "%s, ", signals[i].name);
+ printk(KERN_CONT "\nBASR = 0x%02x : ", basr);
for (i = 0; basrs[i].mask; ++i)
if (basr & basrs[i].mask)
- printk(",%s", basrs[i].name);
- printk("\nICR: %02x ", icr);
+ printk(KERN_CONT "%s, ", basrs[i].name);
+ printk(KERN_CONT "\nICR = 0x%02x : ", icr);
for (i = 0; icrs[i].mask; ++i)
if (icr & icrs[i].mask)
- printk(",%s", icrs[i].name);
- printk("\nMODE: %02x ", mr);
+ printk(KERN_CONT "%s, ", icrs[i].name);
+ printk(KERN_CONT "\nMR = 0x%02x : ", mr);
for (i = 0; mrs[i].mask; ++i)
if (mr & mrs[i].mask)
- printk(",%s", mrs[i].name);
- printk("\n");
+ printk(KERN_CONT "%s, ", mrs[i].name);
+ printk(KERN_CONT "\n");
}
static struct {
@@ -477,52 +458,18 @@
instance->base, instance->irq,
instance->can_queue, instance->cmd_per_lun,
instance->sg_tablesize, instance->this_id,
- hostdata->flags & FLAG_NO_DMA_FIXUP ? "NO_DMA_FIXUP " : "",
+ hostdata->flags & FLAG_DMA_FIXUP ? "DMA_FIXUP " : "",
hostdata->flags & FLAG_NO_PSEUDO_DMA ? "NO_PSEUDO_DMA " : "",
hostdata->flags & FLAG_TOSHIBA_DELAY ? "TOSHIBA_DELAY " : "",
-#ifdef AUTOPROBE_IRQ
- "AUTOPROBE_IRQ "
-#endif
#ifdef DIFFERENTIAL
"DIFFERENTIAL "
#endif
-#ifdef REAL_DMA
- "REAL_DMA "
-#endif
-#ifdef REAL_DMA_POLL
- "REAL_DMA_POLL "
-#endif
#ifdef PARITY
"PARITY "
#endif
-#ifdef PSEUDO_DMA
- "PSEUDO_DMA "
-#endif
"");
}
-#ifdef PSEUDO_DMA
-static int __maybe_unused NCR5380_write_info(struct Scsi_Host *instance,
- char *buffer, int length)
-{
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
-
- hostdata->spin_max_r = 0;
- hostdata->spin_max_w = 0;
- return 0;
-}
-
-static int __maybe_unused NCR5380_show_info(struct seq_file *m,
- struct Scsi_Host *instance)
-{
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
-
- seq_printf(m, "Highwater I/O busy spin counts: write %d, read %d\n",
- hostdata->spin_max_w, hostdata->spin_max_r);
- return 0;
-}
-#endif
-
/**
* NCR5380_init - initialise an NCR5380
* @instance: adapter to configure
@@ -543,6 +490,8 @@
int i;
unsigned long deadline;
+ instance->max_lun = 7;
+
hostdata->host = instance;
hostdata->id_mask = 1 << instance->this_id;
hostdata->id_higher_mask = 0;
@@ -551,9 +500,8 @@
hostdata->id_higher_mask |= i;
for (i = 0; i < 8; ++i)
hostdata->busy[i] = 0;
-#ifdef REAL_DMA
- hostdata->dmalen = 0;
-#endif
+ hostdata->dma_len = 0;
+
spin_lock_init(&hostdata->lock);
hostdata->connected = NULL;
hostdata->sensing = NULL;
@@ -719,6 +667,9 @@
cmd->result = 0;
+ if (!NCR5380_acquire_dma_irq(instance))
+ return SCSI_MLQUEUE_HOST_BUSY;
+
spin_lock_irqsave(&hostdata->lock, flags);
/*
@@ -743,6 +694,19 @@
return 0;
}
+static inline void maybe_release_dma_irq(struct Scsi_Host *instance)
+{
+ struct NCR5380_hostdata *hostdata = shost_priv(instance);
+
+ /* Caller does the locking needed to set & test these data atomically */
+ if (list_empty(&hostdata->disconnected) &&
+ list_empty(&hostdata->unissued) &&
+ list_empty(&hostdata->autosense) &&
+ !hostdata->connected &&
+ !hostdata->selecting)
+ NCR5380_release_dma_irq(instance);
+}
+
/**
* dequeue_next_cmd - dequeue a command for processing
* @instance: the scsi host instance
@@ -844,17 +808,14 @@
if (!NCR5380_select(instance, cmd)) {
dsprintk(NDEBUG_MAIN, instance, "main: select complete\n");
+ maybe_release_dma_irq(instance);
} else {
dsprintk(NDEBUG_MAIN | NDEBUG_QUEUES, instance,
"main: select failed, returning %p to queue\n", cmd);
requeue_cmd(instance, cmd);
}
}
- if (hostdata->connected
-#ifdef REAL_DMA
- && !hostdata->dmalen
-#endif
- ) {
+ if (hostdata->connected && !hostdata->dma_len) {
dsprintk(NDEBUG_MAIN, instance, "main: performing information transfer\n");
NCR5380_information_transfer(instance);
done = 0;
@@ -865,7 +826,88 @@
} while (!done);
}
-#ifndef DONT_USE_INTR
+/*
+ * NCR5380_dma_complete - finish DMA transfer
+ * @instance: the scsi host instance
+ *
+ * Called by the interrupt handler when DMA finishes or a phase
+ * mismatch occurs (which would end the DMA transfer).
+ */
+
+static void NCR5380_dma_complete(struct Scsi_Host *instance)
+{
+ struct NCR5380_hostdata *hostdata = shost_priv(instance);
+ int transferred;
+ unsigned char **data;
+ int *count;
+ int saved_data = 0, overrun = 0;
+ unsigned char p;
+
+ if (hostdata->read_overruns) {
+ p = hostdata->connected->SCp.phase;
+ if (p & SR_IO) {
+ udelay(10);
+ if ((NCR5380_read(BUS_AND_STATUS_REG) &
+ (BASR_PHASE_MATCH | BASR_ACK)) ==
+ (BASR_PHASE_MATCH | BASR_ACK)) {
+ saved_data = NCR5380_read(INPUT_DATA_REG);
+ overrun = 1;
+ dsprintk(NDEBUG_DMA, instance, "read overrun handled\n");
+ }
+ }
+ }
+
+#ifdef CONFIG_SUN3
+ if ((sun3scsi_dma_finish(rq_data_dir(hostdata->connected->request)))) {
+ pr_err("scsi%d: overrun in UDC counter -- not prepared to deal with this!\n",
+ instance->host_no);
+ BUG();
+ }
+
+ if ((NCR5380_read(BUS_AND_STATUS_REG) & (BASR_PHASE_MATCH | BASR_ACK)) ==
+ (BASR_PHASE_MATCH | BASR_ACK)) {
+ pr_err("scsi%d: BASR %02x\n", instance->host_no,
+ NCR5380_read(BUS_AND_STATUS_REG));
+ pr_err("scsi%d: bus stuck in data phase -- probably a single byte overrun!\n",
+ instance->host_no);
+ BUG();
+ }
+#endif
+
+ NCR5380_write(MODE_REG, MR_BASE);
+ NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
+ NCR5380_read(RESET_PARITY_INTERRUPT_REG);
+
+ transferred = hostdata->dma_len - NCR5380_dma_residual(instance);
+ hostdata->dma_len = 0;
+
+ data = (unsigned char **)&hostdata->connected->SCp.ptr;
+ count = &hostdata->connected->SCp.this_residual;
+ *data += transferred;
+ *count -= transferred;
+
+ if (hostdata->read_overruns) {
+ int cnt, toPIO;
+
+ if ((NCR5380_read(STATUS_REG) & PHASE_MASK) == p && (p & SR_IO)) {
+ cnt = toPIO = hostdata->read_overruns;
+ if (overrun) {
+ dsprintk(NDEBUG_DMA, instance,
+ "Got an input overrun, using saved byte\n");
+ *(*data)++ = saved_data;
+ (*count)--;
+ cnt--;
+ toPIO--;
+ }
+ if (toPIO > 0) {
+ dsprintk(NDEBUG_DMA, instance,
+ "Doing %d byte PIO to 0x%p\n", cnt, *data);
+ NCR5380_transfer_pio(instance, &p, &cnt, data);
+ *count -= toPIO - cnt;
+ }
+ }
+ }
+}
/**
* NCR5380_intr - generic NCR5380 irq handler
@@ -901,7 +943,7 @@
* the Busy Monitor interrupt is enabled together with DMA Mode.
*/
-static irqreturn_t NCR5380_intr(int irq, void *dev_id)
+static irqreturn_t __maybe_unused NCR5380_intr(int irq, void *dev_id)
{
struct Scsi_Host *instance = dev_id;
struct NCR5380_hostdata *hostdata = shost_priv(instance);
@@ -919,7 +961,6 @@
dsprintk(NDEBUG_INTR, instance, "IRQ %d, BASR 0x%02x, SR 0x%02x, MR 0x%02x\n",
irq, basr, sr, mr);
-#if defined(REAL_DMA)
if ((mr & MR_DMA_MODE) || (mr & MR_MONITOR_BSY)) {
/* Probably End of DMA, Phase Mismatch or Loss of BSY.
* We ack IRQ after clearing Mode Register. Workarounds
@@ -928,26 +969,14 @@
dsprintk(NDEBUG_INTR, instance, "interrupt in DMA mode\n");
- int transferred;
-
- if (!hostdata->connected)
- panic("scsi%d : DMA interrupt with no connected cmd\n",
- instance->hostno);
-
- transferred = hostdata->dmalen - NCR5380_dma_residual(instance);
- hostdata->connected->SCp.this_residual -= transferred;
- hostdata->connected->SCp.ptr += transferred;
- hostdata->dmalen = 0;
-
- /* FIXME: we need to poll briefly then defer a workqueue task ! */
- NCR5380_poll_politely(hostdata, BUS_AND_STATUS_REG, BASR_ACK, 0, 2 * HZ);
-
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- NCR5380_write(MODE_REG, MR_BASE);
- NCR5380_read(RESET_PARITY_INTERRUPT_REG);
- } else
-#endif /* REAL_DMA */
- if ((NCR5380_read(CURRENT_SCSI_DATA_REG) & hostdata->id_mask) &&
+ if (hostdata->connected) {
+ NCR5380_dma_complete(instance);
+ queue_work(hostdata->work_q, &hostdata->main_task);
+ } else {
+ NCR5380_write(MODE_REG, MR_BASE);
+ NCR5380_read(RESET_PARITY_INTERRUPT_REG);
+ }
+ } else if ((NCR5380_read(CURRENT_SCSI_DATA_REG) & hostdata->id_mask) &&
(sr & (SR_SEL | SR_IO | SR_BSY | SR_RST)) == (SR_SEL | SR_IO)) {
/* Probably reselected */
NCR5380_write(SELECT_ENABLE_REG, 0);
@@ -966,10 +995,16 @@
NCR5380_read(RESET_PARITY_INTERRUPT_REG);
dsprintk(NDEBUG_INTR, instance, "unknown interrupt\n");
+#ifdef SUN3_SCSI_VME
+ dregs->csr |= CSR_DMA_ENABLE;
+#endif
}
handled = 1;
} else {
shost_printk(KERN_NOTICE, instance, "interrupt without IRQ bit\n");
+#ifdef SUN3_SCSI_VME
+ dregs->csr |= CSR_DMA_ENABLE;
+#endif
}
spin_unlock_irqrestore(&hostdata->lock, flags);
@@ -977,8 +1012,6 @@
return IRQ_RETVAL(handled);
}
-#endif
-
/*
* Function : int NCR5380_select(struct Scsi_Host *instance,
* struct scsi_cmnd *cmd)
@@ -1217,14 +1250,6 @@
* was true but before BSY was false during selection, the information
* transfer phase should be a MESSAGE OUT phase so that we can send the
* IDENTIFY message.
- *
- * If SCSI-II tagged queuing is enabled, we also send a SIMPLE_QUEUE_TAG
- * message (2 bytes) with a tag ID that we increment with every command
- * until it wraps back to 0.
- *
- * XXX - it turns out that there are some broken SCSI-II devices,
- * which claim to support tagged queuing but fail when more than
- * some number of commands are issued at once.
*/
/* Wait for start of REQ/ACK handshake */
@@ -1247,9 +1272,6 @@
tmp[0] = IDENTIFY(((instance->irq == NO_IRQ) ? 0 : 1), cmd->device->lun);
len = 1;
- cmd->tag = 0;
-
- /* Send message(s) */
data = tmp;
phase = PHASE_MSGOUT;
NCR5380_transfer_pio(instance, &phase, &len, &data);
@@ -1259,6 +1281,10 @@
hostdata->connected = cmd;
hostdata->busy[cmd->device->id] |= 1 << cmd->device->lun;
+#ifdef SUN3_SCSI_VME
+ dregs->csr |= CSR_INTR;
+#endif
+
initialize_SCp(cmd);
cmd = NULL;
@@ -1495,7 +1521,6 @@
return -1;
}
-#if defined(REAL_DMA) || defined(PSEUDO_DMA) || defined (REAL_DMA_POLL)
/*
* Function : int NCR5380_transfer_dma (struct Scsi_Host *instance,
* unsigned char *phase, int *count, unsigned char **data)
@@ -1520,53 +1545,47 @@
unsigned char **data)
{
struct NCR5380_hostdata *hostdata = shost_priv(instance);
- register int c = *count;
- register unsigned char p = *phase;
- register unsigned char *d = *data;
+ int c = *count;
+ unsigned char p = *phase;
+ unsigned char *d = *data;
unsigned char tmp;
- int foo;
-#if defined(REAL_DMA_POLL)
- int cnt, toPIO;
- unsigned char saved_data = 0, overrun = 0, residue;
-#endif
+ int result = 0;
if ((tmp = (NCR5380_read(STATUS_REG) & PHASE_MASK)) != p) {
*phase = tmp;
return -1;
}
-#if defined(REAL_DMA) || defined(REAL_DMA_POLL)
+
+ hostdata->connected->SCp.phase = p;
+
if (p & SR_IO) {
- if (!(hostdata->flags & FLAG_NO_DMA_FIXUPS))
- c -= 2;
+ if (hostdata->read_overruns)
+ c -= hostdata->read_overruns;
+ else if (hostdata->flags & FLAG_DMA_FIXUP)
+ --c;
}
- hostdata->dma_len = (p & SR_IO) ? NCR5380_dma_read_setup(instance, d, c) : NCR5380_dma_write_setup(instance, d, c);
dsprintk(NDEBUG_DMA, instance, "initializing DMA %s: length %d, address %p\n",
- (p & SR_IO) ? "receive" : "send", c, *data);
+ (p & SR_IO) ? "receive" : "send", c, d);
+
+#ifdef CONFIG_SUN3
+ /* send start chain */
+ sun3scsi_dma_start(c, *data);
#endif
NCR5380_write(TARGET_COMMAND_REG, PHASE_SR_TO_TCR(p));
-
-#ifdef REAL_DMA
NCR5380_write(MODE_REG, MR_BASE | MR_DMA_MODE | MR_MONITOR_BSY |
MR_ENABLE_EOP_INTR);
-#elif defined(REAL_DMA_POLL)
- NCR5380_write(MODE_REG, MR_BASE | MR_DMA_MODE | MR_MONITOR_BSY);
-#else
- /*
- * Note : on my sample board, watch-dog timeouts occurred when interrupts
- * were not disabled for the duration of a single DMA transfer, from
- * before the setting of DMA mode to after transfer of the last byte.
- */
- if (hostdata->flags & FLAG_NO_DMA_FIXUP)
- NCR5380_write(MODE_REG, MR_BASE | MR_DMA_MODE | MR_MONITOR_BSY |
- MR_ENABLE_EOP_INTR);
- else
- NCR5380_write(MODE_REG, MR_BASE | MR_DMA_MODE | MR_MONITOR_BSY);
-#endif /* def REAL_DMA */
-
- dprintk(NDEBUG_DMA, "scsi%d : mode reg = 0x%X\n", instance->host_no, NCR5380_read(MODE_REG));
+ if (!(hostdata->flags & FLAG_LATE_DMA_SETUP)) {
+ /* On the Medusa, it is a must to initialize the DMA before
+ * starting the NCR. This is also the cleaner way for the TT.
+ */
+ if (p & SR_IO)
+ result = NCR5380_dma_recv_setup(instance, d, c);
+ else
+ result = NCR5380_dma_send_setup(instance, d, c);
+ }
/*
* On the PAS16 at least I/O recovery delays are not needed here.
@@ -1574,24 +1593,49 @@
*/
if (p & SR_IO) {
- io_recovery_delay(1);
+ NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
+ NCR5380_io_delay(1);
NCR5380_write(START_DMA_INITIATOR_RECEIVE_REG, 0);
} else {
- io_recovery_delay(1);
+ NCR5380_io_delay(1);
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE | ICR_ASSERT_DATA);
- io_recovery_delay(1);
+ NCR5380_io_delay(1);
NCR5380_write(START_DMA_SEND_REG, 0);
- io_recovery_delay(1);
+ NCR5380_io_delay(1);
}
-#if defined(REAL_DMA_POLL)
- do {
- tmp = NCR5380_read(BUS_AND_STATUS_REG);
- } while ((tmp & BASR_PHASE_MATCH) && !(tmp & (BASR_BUSY_ERROR | BASR_END_DMA_TRANSFER)));
+#ifdef CONFIG_SUN3
+#ifdef SUN3_SCSI_VME
+ dregs->csr |= CSR_DMA_ENABLE;
+#endif
+ sun3_dma_active = 1;
+#endif
+
+ if (hostdata->flags & FLAG_LATE_DMA_SETUP) {
+ /* On the Falcon, the DMA setup must be done after the last
+ * NCR access, else the DMA setup gets trashed!
+ */
+ if (p & SR_IO)
+ result = NCR5380_dma_recv_setup(instance, d, c);
+ else
+ result = NCR5380_dma_send_setup(instance, d, c);
+ }
+
+ /* On failure, NCR5380_dma_xxxx_setup() returns a negative int. */
+ if (result < 0)
+ return result;
+
+ /* For real DMA, result is the byte count. DMA interrupt is expected. */
+ if (result > 0) {
+ hostdata->dma_len = result;
+ return 0;
+ }
+
+ /* The result is zero iff pseudo DMA send/receive was completed. */
+ hostdata->dma_len = c;
/*
- * At this point, either we've completed DMA, or we have a phase mismatch,
- * or we've unexpectedly lost BUSY (which is a real error).
+ * A note regarding the DMA errata workarounds for early NMOS silicon.
*
* For DMA sends, we want to wait until the last byte has been
* transferred out over the bus before we turn off DMA mode. Alas, there
@@ -1618,79 +1662,16 @@
* properly, or the target switches to MESSAGE IN phase to signal a
* disconnection (either operation bringing the DMA to a clean halt).
* However, in order to handle scatter-receive, we must work around the
- * problem. The chosen fix is to DMA N-2 bytes, then check for the
+ * problem. The chosen fix is to DMA fewer bytes, then check for the
* condition before taking the NCR5380 out of DMA mode. One or two extra
* bytes are transferred via PIO as necessary to fill out the original
* request.
*/
- if (p & SR_IO) {
- if (!(hostdata->flags & FLAG_NO_DMA_FIXUPS)) {
- udelay(10);
- if ((NCR5380_read(BUS_AND_STATUS_REG) & (BASR_PHASE_MATCH | BASR_ACK)) ==
- (BASR_PHASE_MATCH | BASR_ACK)) {
- saved_data = NCR5380_read(INPUT_DATA_REGISTER);
- overrun = 1;
- }
- }
- } else {
- int limit = 100;
- while (((tmp = NCR5380_read(BUS_AND_STATUS_REG)) & BASR_ACK) || (NCR5380_read(STATUS_REG) & SR_REQ)) {
- if (!(tmp & BASR_PHASE_MATCH))
- break;
- if (--limit < 0)
- break;
- }
- }
-
- dsprintk(NDEBUG_DMA, "polled DMA transfer complete, basr 0x%02x, sr 0x%02x\n",
- tmp, NCR5380_read(STATUS_REG));
-
- NCR5380_write(MODE_REG, MR_BASE);
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
-
- residue = NCR5380_dma_residual(instance);
- c -= residue;
- *count -= c;
- *data += c;
- *phase = NCR5380_read(STATUS_REG) & PHASE_MASK;
-
- if (!(hostdata->flags & FLAG_NO_DMA_FIXUPS) &&
- *phase == p && (p & SR_IO) && residue == 0) {
- if (overrun) {
- dprintk(NDEBUG_DMA, "Got an input overrun, using saved byte\n");
- **data = saved_data;
- *data += 1;
- *count -= 1;
- cnt = toPIO = 1;
- } else {
- printk("No overrun??\n");
- cnt = toPIO = 2;
- }
- dprintk(NDEBUG_DMA, "Doing %d-byte PIO to 0x%X\n", cnt, *data);
- NCR5380_transfer_pio(instance, phase, &cnt, data);
- *count -= toPIO - cnt;
- }
-
- dprintk(NDEBUG_DMA, "Return with data ptr = 0x%X, count %d, last 0x%X, next 0x%X\n", *data, *count, *(*data + *count - 1), *(*data + *count));
- return 0;
-
-#elif defined(REAL_DMA)
- return 0;
-#else /* defined(REAL_DMA_POLL) */
- if (p & SR_IO) {
- foo = NCR5380_pread(instance, d,
- hostdata->flags & FLAG_NO_DMA_FIXUP ? c : c - 1);
- if (!foo && !(hostdata->flags & FLAG_NO_DMA_FIXUP)) {
+ if (hostdata->flags & FLAG_DMA_FIXUP) {
+ if (p & SR_IO) {
/*
- * We can't disable DMA mode after successfully transferring
- * what we plan to be the last byte, since that would open up
- * a race condition where if the target asserted REQ before
- * we got the DMA mode reset, the NCR5380 would have latched
- * an additional byte into the INPUT DATA register and we'd
- * have dropped it.
- *
- * The workaround was to transfer one fewer bytes than we
+ * The workaround was to transfer fewer bytes than we
* intended to with the pseudo-DMA read function, wait for
* the chip to latch the last byte, read it, and then disable
* pseudo-DMA mode.
@@ -1706,19 +1687,16 @@
if (NCR5380_poll_politely(instance, BUS_AND_STATUS_REG,
BASR_DRQ, BASR_DRQ, HZ) < 0) {
- foo = -1;
+ result = -1;
shost_printk(KERN_ERR, instance, "PDMA read: DRQ timeout\n");
}
if (NCR5380_poll_politely(instance, STATUS_REG,
SR_REQ, 0, HZ) < 0) {
- foo = -1;
+ result = -1;
shost_printk(KERN_ERR, instance, "PDMA read: !REQ timeout\n");
}
- d[c - 1] = NCR5380_read(INPUT_DATA_REG);
- }
- } else {
- foo = NCR5380_pwrite(instance, d, c);
- if (!foo && !(hostdata->flags & FLAG_NO_DMA_FIXUP)) {
+ d[*count - 1] = NCR5380_read(INPUT_DATA_REG);
+ } else {
/*
* Wait for the last byte to be sent. If REQ is being asserted for
* the byte we're interested, we'll ACK it and it will go false.
@@ -1726,21 +1704,15 @@
if (NCR5380_poll_politely2(instance,
BUS_AND_STATUS_REG, BASR_DRQ, BASR_DRQ,
BUS_AND_STATUS_REG, BASR_PHASE_MATCH, 0, HZ) < 0) {
- foo = -1;
+ result = -1;
shost_printk(KERN_ERR, instance, "PDMA write: DRQ and phase timeout\n");
}
}
}
- NCR5380_write(MODE_REG, MR_BASE);
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- NCR5380_read(RESET_PARITY_INTERRUPT_REG);
- *data = d + c;
- *count = 0;
- *phase = NCR5380_read(STATUS_REG) & PHASE_MASK;
- return foo;
-#endif /* def REAL_DMA */
+
+ NCR5380_dma_complete(instance);
+ return result;
}
-#endif /* defined(REAL_DMA) | defined(PSEUDO_DMA) */
/*
* Function : NCR5380_information_transfer (struct Scsi_Host *instance)
@@ -1770,6 +1742,10 @@
unsigned char phase, tmp, extended_msg[10], old_phase = 0xff;
struct scsi_cmnd *cmd;
+#ifdef SUN3_SCSI_VME
+ dregs->csr |= CSR_INTR;
+#endif
+
while ((cmd = hostdata->connected)) {
struct NCR5380_cmd *ncmd = scsi_cmd_priv(cmd);
@@ -1781,6 +1757,31 @@
old_phase = phase;
NCR5380_dprint_phase(NDEBUG_INFORMATION, instance);
}
+#ifdef CONFIG_SUN3
+ if (phase == PHASE_CMDOUT) {
+ void *d;
+ unsigned long count;
+
+ if (!cmd->SCp.this_residual && cmd->SCp.buffers_residual) {
+ count = cmd->SCp.buffer->length;
+ d = sg_virt(cmd->SCp.buffer);
+ } else {
+ count = cmd->SCp.this_residual;
+ d = cmd->SCp.ptr;
+ }
+
+ if (sun3_dma_setup_done != cmd &&
+ sun3scsi_dma_xfer_len(count, cmd) > 0) {
+ sun3scsi_dma_setup(instance, d, count,
+ rq_data_dir(cmd->request));
+ sun3_dma_setup_done = cmd;
+ }
+#ifdef SUN3_SCSI_VME
+ dregs->csr |= CSR_INTR;
+#endif
+ }
+#endif /* CONFIG_SUN3 */
+
if (sink && (phase != PHASE_MSGOUT)) {
NCR5380_write(TARGET_COMMAND_REG, PHASE_SR_TO_TCR(tmp));
@@ -1831,13 +1832,11 @@
* in an unconditional loop.
*/
-#if defined(PSEUDO_DMA) || defined(REAL_DMA_POLL)
transfersize = 0;
- if (!cmd->device->borken &&
- !(hostdata->flags & FLAG_NO_PSEUDO_DMA))
+ if (!cmd->device->borken)
transfersize = NCR5380_dma_xfer_len(instance, cmd, phase);
- if (transfersize) {
+ if (transfersize > 0) {
len = transfersize;
if (NCR5380_transfer_dma(instance, &phase,
&len, (unsigned char **)&cmd->SCp.ptr)) {
@@ -1853,11 +1852,8 @@
do_abort(instance);
cmd->result = DID_ERROR << 16;
/* XXX - need to source or sink data here, as appropriate */
- } else
- cmd->SCp.this_residual -= transfersize - len;
- } else
-#endif /* defined(PSEUDO_DMA) || defined(REAL_DMA_POLL) */
- {
+ }
+ } else {
/* Break up transfer into 3 ms chunks,
* presuming 6 accesses per handshake.
*/
@@ -1868,6 +1864,10 @@
(unsigned char **)&cmd->SCp.ptr);
cmd->SCp.this_residual -= transfersize - len;
}
+#ifdef CONFIG_SUN3
+ if (sun3_dma_setup_done == cmd)
+ sun3_dma_setup_done = NULL;
+#endif
return;
case PHASE_MSGIN:
len = 1;
@@ -1912,6 +1912,8 @@
/* Enable reselect interrupts */
NCR5380_write(SELECT_ENABLE_REG, hostdata->id_mask);
+
+ maybe_release_dma_irq(instance);
return;
case MESSAGE_REJECT:
/* Accept message by clearing ACK */
@@ -1944,6 +1946,9 @@
/* Enable reselect interrupts */
NCR5380_write(SELECT_ENABLE_REG, hostdata->id_mask);
+#ifdef SUN3_SCSI_VME
+ dregs->csr |= CSR_DMA_ENABLE;
+#endif
return;
/*
* The SCSI data pointer is *IMPLICITLY* saved on a disconnect
@@ -2047,6 +2052,7 @@
hostdata->connected = NULL;
cmd->result = DID_ERROR << 16;
complete_cmd(instance, cmd);
+ maybe_release_dma_irq(instance);
NCR5380_write(SELECT_ENABLE_REG, hostdata->id_mask);
return;
}
@@ -2094,10 +2100,8 @@
{
struct NCR5380_hostdata *hostdata = shost_priv(instance);
unsigned char target_mask;
- unsigned char lun, phase;
- int len;
+ unsigned char lun;
unsigned char msg[3];
- unsigned char *data;
struct NCR5380_cmd *ncmd;
struct scsi_cmnd *tmp;
@@ -2139,15 +2143,26 @@
return;
}
- len = 1;
- data = msg;
- phase = PHASE_MSGIN;
- NCR5380_transfer_pio(instance, &phase, &len, &data);
+#ifdef CONFIG_SUN3
+ /* acknowledge toggle to MSGIN */
+ NCR5380_write(TARGET_COMMAND_REG, PHASE_SR_TO_TCR(PHASE_MSGIN));
- if (len) {
- do_abort(instance);
- return;
+ /* peek at the byte without really hitting the bus */
+ msg[0] = NCR5380_read(CURRENT_SCSI_DATA_REG);
+#else
+ {
+ int len = 1;
+ unsigned char *data = msg;
+ unsigned char phase = PHASE_MSGIN;
+
+ NCR5380_transfer_pio(instance, &phase, &len, &data);
+
+ if (len) {
+ do_abort(instance);
+ return;
+ }
}
+#endif /* CONFIG_SUN3 */
if (!(msg[0] & 0x80)) {
shost_printk(KERN_ERR, instance, "expecting IDENTIFY message, got ");
@@ -2195,60 +2210,38 @@
return;
}
+#ifdef CONFIG_SUN3
+ {
+ void *d;
+ unsigned long count;
+
+ if (!tmp->SCp.this_residual && tmp->SCp.buffers_residual) {
+ count = tmp->SCp.buffer->length;
+ d = sg_virt(tmp->SCp.buffer);
+ } else {
+ count = tmp->SCp.this_residual;
+ d = tmp->SCp.ptr;
+ }
+
+ if (sun3_dma_setup_done != tmp &&
+ sun3scsi_dma_xfer_len(count, tmp) > 0) {
+ sun3scsi_dma_setup(instance, d, count,
+ rq_data_dir(tmp->request));
+ sun3_dma_setup_done = tmp;
+ }
+ }
+
+ NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE | ICR_ASSERT_ACK);
+#endif /* CONFIG_SUN3 */
+
/* Accept message by clearing ACK */
NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
hostdata->connected = tmp;
- dsprintk(NDEBUG_RESELECTION, instance, "nexus established, target %d, lun %llu, tag %d\n",
- scmd_id(tmp), tmp->device->lun, tmp->tag);
+ dsprintk(NDEBUG_RESELECTION, instance, "nexus established, target %d, lun %llu\n",
+ scmd_id(tmp), tmp->device->lun);
}
-/*
- * Function : void NCR5380_dma_complete (struct Scsi_Host *instance)
- *
- * Purpose : called by interrupt handler when DMA finishes or a phase
- * mismatch occurs (which would finish the DMA transfer).
- *
- * Inputs : instance - this instance of the NCR5380.
- *
- * Returns : pointer to the scsi_cmnd structure for which the I_T_L
- * nexus has been reestablished, on failure NULL is returned.
- */
-
-#ifdef REAL_DMA
-static void NCR5380_dma_complete(NCR5380_instance * instance) {
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
- int transferred;
-
- /*
- * XXX this might not be right.
- *
- * Wait for final byte to transfer, ie wait for ACK to go false.
- *
- * We should use the Last Byte Sent bit, unfortunately this is
- * not available on the 5380/5381 (only the various CMOS chips)
- *
- * FIXME: timeout, and need to handle long timeout/irq case
- */
-
- NCR5380_poll_politely(instance, BUS_AND_STATUS_REG, BASR_ACK, 0, 5*HZ);
-
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
-
- /*
- * The only places we should see a phase mismatch and have to send
- * data from the same set of pointers will be the data transfer
- * phases. So, residual, requested length are only important here.
- */
-
- if (!(hostdata->connected->SCp.phase & SR_CD)) {
- transferred = instance->dmalen - NCR5380_dma_residual();
- hostdata->connected->SCp.this_residual -= transferred;
- hostdata->connected->SCp.ptr += transferred;
- }
-}
-#endif /* def REAL_DMA */
-
/**
* list_find_cmd - test for presence of a command in a linked list
* @haystack: list of commands
@@ -2360,9 +2353,7 @@
if (hostdata->connected == cmd) {
dsprintk(NDEBUG_ABORT, instance, "abort: cmd %p is connected\n", cmd);
hostdata->connected = NULL;
-#ifdef REAL_DMA
hostdata->dma_len = 0;
-#endif
if (do_abort(instance)) {
set_host_byte(cmd, DID_ERROR);
complete_cmd(instance, cmd);
@@ -2388,6 +2379,7 @@
dsprintk(NDEBUG_ABORT, instance, "abort: successfully aborted %p\n", cmd);
queue_work(hostdata->work_q, &hostdata->main_task);
+ maybe_release_dma_irq(instance);
spin_unlock_irqrestore(&hostdata->lock, flags);
return result;
@@ -2445,7 +2437,7 @@
struct scsi_cmnd *cmd = NCR5380_to_scmd(ncmd);
set_host_byte(cmd, DID_RESET);
- cmd->scsi_done(cmd);
+ complete_cmd(instance, cmd);
}
INIT_LIST_HEAD(&hostdata->disconnected);
@@ -2465,11 +2457,10 @@
for (i = 0; i < 8; ++i)
hostdata->busy[i] = 0;
-#ifdef REAL_DMA
hostdata->dma_len = 0;
-#endif
queue_work(hostdata->work_q, &hostdata->main_task);
+ maybe_release_dma_irq(instance);
spin_unlock_irqrestore(&hostdata->lock, flags);
return SUCCESS;
diff --git a/drivers/scsi/NCR5380.h b/drivers/scsi/NCR5380.h
index a792886..c607287 100644
--- a/drivers/scsi/NCR5380.h
+++ b/drivers/scsi/NCR5380.h
@@ -199,13 +199,6 @@
#define PHASE_SR_TO_TCR(phase) ((phase) >> 2)
-/*
- * "Special" value for the (unsigned char) command tag, to indicate
- * I_T_L nexus instead of I_T_L_Q.
- */
-
-#define TAG_NONE 0xff
-
/*
* These are "special" values for the irq and dma_channel fields of the
* Scsi_Host structure
@@ -220,28 +213,17 @@
#define NO_IRQ 0
#endif
-#define FLAG_NO_DMA_FIXUP 1 /* No DMA errata workarounds */
+#define FLAG_DMA_FIXUP 1 /* Use DMA errata workarounds */
#define FLAG_NO_PSEUDO_DMA 8 /* Inhibit DMA */
#define FLAG_LATE_DMA_SETUP 32 /* Setup NCR before DMA H/W */
-#define FLAG_TAGGED_QUEUING 64 /* as X3T9.2 spelled it */
#define FLAG_TOSHIBA_DELAY 128 /* Allow for borken CD-ROMs */
-#ifdef SUPPORT_TAGS
-struct tag_alloc {
- DECLARE_BITMAP(allocated, MAX_TAGS);
- int nr_allocated;
- int queue_size;
-};
-#endif
-
struct NCR5380_hostdata {
NCR5380_implementation_fields; /* implementation specific */
struct Scsi_Host *host; /* Host backpointer */
unsigned char id_mask, id_higher_mask; /* 1 << id, all bits greater */
unsigned char busy[8]; /* index = target, bit = lun */
-#if defined(REAL_DMA) || defined(REAL_DMA_POLL)
int dma_len; /* requested length of DMA */
-#endif
unsigned char last_message; /* last message OUT */
struct scsi_cmnd *connected; /* currently connected cmnd */
struct scsi_cmnd *selecting; /* cmnd to be connected */
@@ -256,13 +238,6 @@
int read_overruns; /* number of bytes to cut from a
* transfer to handle chip overruns */
struct work_struct main_task;
-#ifdef SUPPORT_TAGS
- struct tag_alloc TagAlloc[8][8]; /* 8 targets and 8 LUNs */
-#endif
-#ifdef PSEUDO_DMA
- unsigned spin_max_r;
- unsigned spin_max_w;
-#endif
struct workqueue_struct *work_q;
unsigned long accesses_per_ms; /* chip register accesses per ms */
};
@@ -305,132 +280,20 @@
#define NCR5380_dprint_phase(flg, arg) do {} while (0)
#endif
-#if defined(AUTOPROBE_IRQ)
static int NCR5380_probe_irq(struct Scsi_Host *instance, int possible);
-#endif
static int NCR5380_init(struct Scsi_Host *instance, int flags);
static int NCR5380_maybe_reset_bus(struct Scsi_Host *);
static void NCR5380_exit(struct Scsi_Host *instance);
static void NCR5380_information_transfer(struct Scsi_Host *instance);
-#ifndef DONT_USE_INTR
static irqreturn_t NCR5380_intr(int irq, void *dev_id);
-#endif
static void NCR5380_main(struct work_struct *work);
static const char *NCR5380_info(struct Scsi_Host *instance);
static void NCR5380_reselect(struct Scsi_Host *instance);
static struct scsi_cmnd *NCR5380_select(struct Scsi_Host *, struct scsi_cmnd *);
-#if defined(PSEUDO_DMA) || defined(REAL_DMA) || defined(REAL_DMA_POLL)
static int NCR5380_transfer_dma(struct Scsi_Host *instance, unsigned char *phase, int *count, unsigned char **data);
-#endif
static int NCR5380_transfer_pio(struct Scsi_Host *instance, unsigned char *phase, int *count, unsigned char **data);
+static int NCR5380_poll_politely(struct Scsi_Host *, int, int, int, int);
+static int NCR5380_poll_politely2(struct Scsi_Host *, int, int, int, int, int, int, int);
-#if (defined(REAL_DMA) || defined(REAL_DMA_POLL))
-
-#if defined(i386) || defined(__alpha__)
-
-/**
- * NCR5380_pc_dma_setup - setup ISA DMA
- * @instance: adapter to set up
- * @ptr: block to transfer (virtual address)
- * @count: number of bytes to transfer
- * @mode: DMA controller mode to use
- *
- * Program the DMA controller ready to perform an ISA DMA transfer
- * on this chip.
- *
- * Locks: takes and releases the ISA DMA lock.
- */
-
-static __inline__ int NCR5380_pc_dma_setup(struct Scsi_Host *instance, unsigned char *ptr, unsigned int count, unsigned char mode)
-{
- unsigned limit;
- unsigned long bus_addr = virt_to_bus(ptr);
- unsigned long flags;
-
- if (instance->dma_channel <= 3) {
- if (count > 65536)
- count = 65536;
- limit = 65536 - (bus_addr & 0xFFFF);
- } else {
- if (count > 65536 * 2)
- count = 65536 * 2;
- limit = 65536 * 2 - (bus_addr & 0x1FFFF);
- }
-
- if (count > limit)
- count = limit;
-
- if ((count & 1) || (bus_addr & 1))
- panic("scsi%d : attempted unaligned DMA transfer\n", instance->host_no);
-
- flags=claim_dma_lock();
- disable_dma(instance->dma_channel);
- clear_dma_ff(instance->dma_channel);
- set_dma_addr(instance->dma_channel, bus_addr);
- set_dma_count(instance->dma_channel, count);
- set_dma_mode(instance->dma_channel, mode);
- enable_dma(instance->dma_channel);
- release_dma_lock(flags);
-
- return count;
-}
-
-/**
- * NCR5380_pc_dma_write_setup - setup ISA DMA write
- * @instance: adapter to set up
- * @ptr: block to transfer (virtual address)
- * @count: number of bytes to transfer
- *
- * Program the DMA controller ready to perform an ISA DMA write to the
- * SCSI controller.
- *
- * Locks: called routines take and release the ISA DMA lock.
- */
-
-static __inline__ int NCR5380_pc_dma_write_setup(struct Scsi_Host *instance, unsigned char *src, unsigned int count)
-{
- return NCR5380_pc_dma_setup(instance, src, count, DMA_MODE_WRITE);
-}
-
-/**
- * NCR5380_pc_dma_read_setup - setup ISA DMA read
- * @instance: adapter to set up
- * @ptr: block to transfer (virtual address)
- * @count: number of bytes to transfer
- *
- * Program the DMA controller ready to perform an ISA DMA read from the
- * SCSI controller.
- *
- * Locks: called routines take and release the ISA DMA lock.
- */
-
-static __inline__ int NCR5380_pc_dma_read_setup(struct Scsi_Host *instance, unsigned char *src, unsigned int count)
-{
- return NCR5380_pc_dma_setup(instance, src, count, DMA_MODE_READ);
-}
-
-/**
- * NCR5380_pc_dma_residual - return bytes left
- * @instance: adapter
- *
- * Reports the number of bytes left over after the DMA was terminated.
- *
- * Locks: takes and releases the ISA DMA lock.
- */
-
-static __inline__ int NCR5380_pc_dma_residual(struct Scsi_Host *instance)
-{
- unsigned long flags;
- int tmp;
-
- flags = claim_dma_lock();
- clear_dma_ff(instance->dma_channel);
- tmp = get_dma_residue(instance->dma_channel);
- release_dma_lock(flags);
-
- return tmp;
-}
-#endif /* defined(i386) || defined(__alpha__) */
-#endif /* defined(REAL_DMA) */
#endif /* __KERNEL__ */
#endif /* NCR5380_H */
diff --git a/drivers/scsi/aacraid/aachba.c b/drivers/scsi/aacraid/aachba.c
index 7dfd0fa..6678d1f 100644
--- a/drivers/scsi/aacraid/aachba.c
+++ b/drivers/scsi/aacraid/aachba.c
@@ -555,8 +555,6 @@
dev = (struct aac_dev *)scsicmd->device->host->hostdata;
cmd_fibcontext = aac_fib_alloc_tag(dev, scsicmd);
- if (!cmd_fibcontext)
- return -ENOMEM;
aac_fib_init(cmd_fibcontext);
dinfo = (struct aac_get_name *) fib_data(cmd_fibcontext);
@@ -1037,8 +1035,6 @@
dev = (struct aac_dev *)scsicmd->device->host->hostdata;
cmd_fibcontext = aac_fib_alloc_tag(dev, scsicmd);
- if (!cmd_fibcontext)
- return -ENOMEM;
aac_fib_init(cmd_fibcontext);
dinfo = (struct aac_get_serial *) fib_data(cmd_fibcontext);
@@ -1950,10 +1946,6 @@
* Alocate and initialize a Fib
*/
cmd_fibcontext = aac_fib_alloc_tag(dev, scsicmd);
- if (!cmd_fibcontext) {
- printk(KERN_WARNING "aac_read: fib allocation failed\n");
- return -1;
- }
status = aac_adapter_read(cmd_fibcontext, scsicmd, lba, count);
@@ -2048,16 +2040,6 @@
* Allocate and initialize a Fib then setup a BlockWrite command
*/
cmd_fibcontext = aac_fib_alloc_tag(dev, scsicmd);
- if (!cmd_fibcontext) {
- /* FIB temporarily unavailable,not catastrophic failure */
-
- /* scsicmd->result = DID_ERROR << 16;
- * scsicmd->scsi_done(scsicmd);
- * return 0;
- */
- printk(KERN_WARNING "aac_write: fib allocation failed\n");
- return -1;
- }
status = aac_adapter_write(cmd_fibcontext, scsicmd, lba, count, fua);
@@ -2283,8 +2265,6 @@
* Allocate and initialize a Fib
*/
cmd_fibcontext = aac_fib_alloc_tag(aac, scsicmd);
- if (!cmd_fibcontext)
- return SCSI_MLQUEUE_HOST_BUSY;
aac_fib_init(cmd_fibcontext);
@@ -3184,8 +3164,6 @@
* Allocate and initialize a Fib then setup a BlockWrite command
*/
cmd_fibcontext = aac_fib_alloc_tag(dev, scsicmd);
- if (!cmd_fibcontext)
- return -1;
status = aac_adapter_scsi(cmd_fibcontext, scsicmd);
diff --git a/drivers/scsi/aacraid/aacraid.h b/drivers/scsi/aacraid/aacraid.h
index efa493cf..8f90d9e 100644
--- a/drivers/scsi/aacraid/aacraid.h
+++ b/drivers/scsi/aacraid/aacraid.h
@@ -29,6 +29,7 @@
#define AAC_INT_MODE_MSI (1<<1)
#define AAC_INT_MODE_AIF (1<<2)
#define AAC_INT_MODE_SYNC (1<<3)
+#define AAC_INT_MODE_MSIX (1<<16)
#define AAC_INT_ENABLE_TYPE1_INTX 0xfffffffb
#define AAC_INT_ENABLE_TYPE1_MSIX 0xfffffffa
@@ -62,7 +63,7 @@
#define PMC_GLOBAL_INT_BIT0 0x00000001
#ifndef AAC_DRIVER_BUILD
-# define AAC_DRIVER_BUILD 41052
+# define AAC_DRIVER_BUILD 41066
# define AAC_DRIVER_BRANCH "-ms"
#endif
#define MAXIMUM_NUM_CONTAINERS 32
@@ -720,7 +721,7 @@
};
-#define Sa_MINIPORT_REVISION 1
+#define SA_INIT_NUM_MSIXVECTORS 1
#define sa_readw(AEP, CSR) readl(&((AEP)->regs.sa->CSR))
#define sa_readl(AEP, CSR) readl(&((AEP)->regs.sa->CSR))
@@ -2065,6 +2066,10 @@
#define AifEnAddJBOD 30 /* JBOD created */
#define AifEnDeleteJBOD 31 /* JBOD deleted */
+#define AifBuManagerEvent 42 /* Bu management*/
+#define AifBuCacheDataLoss 10
+#define AifBuCacheDataRecover 11
+
#define AifCmdJobProgress 2 /* Progress report */
#define AifJobCtrZero 101 /* Array Zero progress */
#define AifJobStsSuccess 1 /* Job completes */
diff --git a/drivers/scsi/aacraid/comminit.c b/drivers/scsi/aacraid/comminit.c
index 2b4e753..341ea32 100644
--- a/drivers/scsi/aacraid/comminit.c
+++ b/drivers/scsi/aacraid/comminit.c
@@ -37,6 +37,7 @@
#include <linux/spinlock.h>
#include <linux/slab.h>
#include <linux/blkdev.h>
+#include <linux/delay.h>
#include <linux/completion.h>
#include <linux/mm.h>
#include <scsi/scsi_host.h>
@@ -47,6 +48,20 @@
.irq_mod = 1
};
+static inline int aac_is_msix_mode(struct aac_dev *dev)
+{
+ u32 status;
+
+ status = src_readl(dev, MUnit.OMR);
+ return (status & AAC_INT_MODE_MSIX);
+}
+
+static inline void aac_change_to_intx(struct aac_dev *dev)
+{
+ aac_src_access_devreg(dev, AAC_DISABLE_MSIX);
+ aac_src_access_devreg(dev, AAC_ENABLE_INTX);
+}
+
static int aac_alloc_comm(struct aac_dev *dev, void **commaddr, unsigned long commsize, unsigned long commalign)
{
unsigned char *base;
@@ -91,7 +106,7 @@
init->InitStructRevision = cpu_to_le32(ADAPTER_INIT_STRUCT_REVISION);
if (dev->max_fib_size != sizeof(struct hw_fib))
init->InitStructRevision = cpu_to_le32(ADAPTER_INIT_STRUCT_REVISION_4);
- init->Sa_MSIXVectors = cpu_to_le32(Sa_MINIPORT_REVISION);
+ init->Sa_MSIXVectors = cpu_to_le32(SA_INIT_NUM_MSIXVECTORS);
init->fsrev = cpu_to_le32(dev->fsrev);
/*
@@ -378,21 +393,8 @@
msi_count = i;
} else {
dev->msi_enabled = 0;
- printk(KERN_ERR "%s%d: MSIX not supported!! Will try MSI 0x%x.\n",
- dev->name, dev->id, i);
- }
- }
-
- if (!dev->msi_enabled) {
- msi_count = 1;
- i = pci_enable_msi(dev->pdev);
-
- if (!i) {
- dev->msi_enabled = 1;
- dev->msi = 1;
- } else {
- printk(KERN_ERR "%s%d: MSI not supported!! Will try INTx 0x%x.\n",
- dev->name, dev->id, i);
+ dev_err(&dev->pdev->dev,
+ "MSIX not supported!! Will try INTX 0x%x.\n", i);
}
}
@@ -427,6 +429,15 @@
dev->comm_interface = AAC_COMM_PRODUCER;
dev->raw_io_interface = dev->raw_io_64 = 0;
+
+ /*
+ * Enable INTX mode, if not done already Enabled
+ */
+ if (aac_is_msix_mode(dev)) {
+ aac_change_to_intx(dev);
+ dev_info(&dev->pdev->dev, "Changed firmware to INTX mode");
+ }
+
if ((!aac_adapter_sync_cmd(dev, GET_ADAPTER_PROPERTIES,
0, 0, 0, 0, 0, 0,
status+0, status+1, status+2, status+3, NULL)) &&
diff --git a/drivers/scsi/aacraid/commsup.c b/drivers/scsi/aacraid/commsup.c
index 511bbc5..0aeecec 100644
--- a/drivers/scsi/aacraid/commsup.c
+++ b/drivers/scsi/aacraid/commsup.c
@@ -637,10 +637,10 @@
}
return -EFAULT;
}
- /* We used to udelay() here but that absorbed
- * a CPU when a timeout occured. Not very
- * useful. */
- cpu_relax();
+ /*
+ * Allow other processes / CPUS to use core
+ */
+ schedule();
}
} else if (down_interruptible(&fibptr->event_wait)) {
/* Do nothing ... satisfy
@@ -901,6 +901,31 @@
memset(cp, 0, 256);
}
+static inline int aac_aif_data(struct aac_aifcmd *aifcmd, uint32_t index)
+{
+ return le32_to_cpu(((__le32 *)aifcmd->data)[index]);
+}
+
+
+static void aac_handle_aif_bu(struct aac_dev *dev, struct aac_aifcmd *aifcmd)
+{
+ switch (aac_aif_data(aifcmd, 1)) {
+ case AifBuCacheDataLoss:
+ if (aac_aif_data(aifcmd, 2))
+ dev_info(&dev->pdev->dev, "Backup unit had cache data loss - [%d]\n",
+ aac_aif_data(aifcmd, 2));
+ else
+ dev_info(&dev->pdev->dev, "Backup Unit had cache data loss\n");
+ break;
+ case AifBuCacheDataRecover:
+ if (aac_aif_data(aifcmd, 2))
+ dev_info(&dev->pdev->dev, "DDR cache data recovered successfully - [%d]\n",
+ aac_aif_data(aifcmd, 2));
+ else
+ dev_info(&dev->pdev->dev, "DDR cache data recovered successfully\n");
+ break;
+ }
+}
/**
* aac_handle_aif - Handle a message from the firmware
@@ -1154,6 +1179,8 @@
ADD : DELETE;
break;
}
+ case AifBuManagerEvent:
+ aac_handle_aif_bu(dev, aifcmd);
break;
}
@@ -1996,6 +2023,10 @@
if (difference <= 0)
difference = 1;
set_current_state(TASK_INTERRUPTIBLE);
+
+ if (kthread_should_stop())
+ break;
+
schedule_timeout(difference);
if (kthread_should_stop())
diff --git a/drivers/scsi/aacraid/dpcsup.c b/drivers/scsi/aacraid/dpcsup.c
index d677b52..7e83620 100644
--- a/drivers/scsi/aacraid/dpcsup.c
+++ b/drivers/scsi/aacraid/dpcsup.c
@@ -392,9 +392,10 @@
if (likely(fib->callback && fib->callback_data)) {
fib->flags &= FIB_CONTEXT_FLAG_FASTRESP;
fib->callback(fib->callback_data, fib);
- } else {
- aac_fib_complete(fib);
- }
+ } else
+ dev_info(&dev->pdev->dev,
+ "Invalid callback_fib[%d] (*%p)(%p)\n",
+ index, fib->callback, fib->callback_data);
} else {
unsigned long flagv;
dprintk((KERN_INFO "event_wait up\n"));
diff --git a/drivers/scsi/aacraid/linit.c b/drivers/scsi/aacraid/linit.c
index ff6caab..a943bd2 100644
--- a/drivers/scsi/aacraid/linit.c
+++ b/drivers/scsi/aacraid/linit.c
@@ -1299,6 +1299,8 @@
else
shost->this_id = shost->max_id;
+ aac_intr_normal(aac, 0, 2, 0, NULL);
+
/*
* dmb - we may need to move the setting of these parms somewhere else once
* we get a fib that can report the actual numbers
@@ -1431,8 +1433,8 @@
/* After EEH recovery or suspend resume, max_msix count
* may change, therfore updating in init as well.
*/
- aac_adapter_start(dev);
dev->init->Sa_MSIXVectors = cpu_to_le32(dev->max_msix);
+ aac_adapter_start(dev);
}
return 0;
diff --git a/drivers/scsi/aacraid/src.c b/drivers/scsi/aacraid/src.c
index bc0203f..28f8b8a 100644
--- a/drivers/scsi/aacraid/src.c
+++ b/drivers/scsi/aacraid/src.c
@@ -135,7 +135,8 @@
if (mode & AAC_INT_MODE_AIF) {
/* handle AIF */
- aac_intr_normal(dev, 0, 2, 0, NULL);
+ if (dev->aif_thread && dev->fsa_dev)
+ aac_intr_normal(dev, 0, 2, 0, NULL);
if (dev->msi_enabled)
aac_src_access_devreg(dev, AAC_CLEAR_AIF_BIT);
mode = 0;
diff --git a/drivers/scsi/arm/cumana_1.c b/drivers/scsi/arm/cumana_1.c
index 221f18c..8e9cfe8 100644
--- a/drivers/scsi/arm/cumana_1.c
+++ b/drivers/scsi/arm/cumana_1.c
@@ -13,13 +13,14 @@
#include <scsi/scsi_host.h>
-#define PSEUDO_DMA
-
#define priv(host) ((struct NCR5380_hostdata *)(host)->hostdata)
#define NCR5380_read(reg) cumanascsi_read(instance, reg)
#define NCR5380_write(reg, value) cumanascsi_write(instance, reg, value)
#define NCR5380_dma_xfer_len(instance, cmd, phase) (cmd->transfersize)
+#define NCR5380_dma_recv_setup cumanascsi_pread
+#define NCR5380_dma_send_setup cumanascsi_pwrite
+#define NCR5380_dma_residual(instance) (0)
#define NCR5380_intr cumanascsi_intr
#define NCR5380_queue_command cumanascsi_queue_command
@@ -41,8 +42,8 @@
#define L(v) (((v)<<16)|((v) & 0x0000ffff))
#define H(v) (((v)>>16)|((v) & 0xffff0000))
-static inline int
-NCR5380_pwrite(struct Scsi_Host *host, unsigned char *addr, int len)
+static inline int cumanascsi_pwrite(struct Scsi_Host *host,
+ unsigned char *addr, int len)
{
unsigned long *laddr;
void __iomem *dma = priv(host)->dma + 0x2000;
@@ -101,11 +102,14 @@
}
end:
writeb(priv(host)->ctrl | 0x40, priv(host)->base + CTRL);
- return len;
+
+ if (len)
+ return -1;
+ return 0;
}
-static inline int
-NCR5380_pread(struct Scsi_Host *host, unsigned char *addr, int len)
+static inline int cumanascsi_pread(struct Scsi_Host *host,
+ unsigned char *addr, int len)
{
unsigned long *laddr;
void __iomem *dma = priv(host)->dma + 0x2000;
@@ -163,7 +167,10 @@
}
end:
writeb(priv(host)->ctrl | 0x40, priv(host)->base + CTRL);
- return len;
+
+ if (len)
+ return -1;
+ return 0;
}
static unsigned char cumanascsi_read(struct Scsi_Host *host, unsigned int reg)
@@ -239,7 +246,7 @@
host->irq = ec->irq;
- ret = NCR5380_init(host, 0);
+ ret = NCR5380_init(host, FLAG_DMA_FIXUP | FLAG_LATE_DMA_SETUP);
if (ret)
goto out_unmap;
diff --git a/drivers/scsi/arm/cumana_2.c b/drivers/scsi/arm/cumana_2.c
index faa1bee..edce5f3 100644
--- a/drivers/scsi/arm/cumana_2.c
+++ b/drivers/scsi/arm/cumana_2.c
@@ -365,7 +365,7 @@
.eh_abort_handler = fas216_eh_abort,
.can_queue = 1,
.this_id = 7,
- .sg_tablesize = SCSI_MAX_SG_CHAIN_SEGMENTS,
+ .sg_tablesize = SG_MAX_SEGMENTS,
.dma_boundary = IOMD_DMA_BOUNDARY,
.use_clustering = DISABLE_CLUSTERING,
.proc_name = "cumanascsi2",
diff --git a/drivers/scsi/arm/eesox.c b/drivers/scsi/arm/eesox.c
index a8ad688..e93e047 100644
--- a/drivers/scsi/arm/eesox.c
+++ b/drivers/scsi/arm/eesox.c
@@ -484,7 +484,7 @@
.eh_abort_handler = fas216_eh_abort,
.can_queue = 1,
.this_id = 7,
- .sg_tablesize = SCSI_MAX_SG_CHAIN_SEGMENTS,
+ .sg_tablesize = SG_MAX_SEGMENTS,
.dma_boundary = IOMD_DMA_BOUNDARY,
.use_clustering = DISABLE_CLUSTERING,
.proc_name = "eesox",
diff --git a/drivers/scsi/arm/oak.c b/drivers/scsi/arm/oak.c
index 1fab1d18..a396024 100644
--- a/drivers/scsi/arm/oak.c
+++ b/drivers/scsi/arm/oak.c
@@ -14,9 +14,6 @@
#include <scsi/scsi_host.h>
-/*#define PSEUDO_DMA*/
-#define DONT_USE_INTR
-
#define priv(host) ((struct NCR5380_hostdata *)(host)->hostdata)
#define NCR5380_read(reg) \
@@ -24,7 +21,10 @@
#define NCR5380_write(reg, value) \
writeb(value, priv(instance)->base + ((reg) << 2))
-#define NCR5380_dma_xfer_len(instance, cmd, phase) (cmd->transfersize)
+#define NCR5380_dma_xfer_len(instance, cmd, phase) (0)
+#define NCR5380_dma_recv_setup oakscsi_pread
+#define NCR5380_dma_send_setup oakscsi_pwrite
+#define NCR5380_dma_residual(instance) (0)
#define NCR5380_queue_command oakscsi_queue_command
#define NCR5380_info oakscsi_info
@@ -40,23 +40,23 @@
#define STAT ((128 + 16) << 2)
#define DATA ((128 + 8) << 2)
-static inline int NCR5380_pwrite(struct Scsi_Host *instance, unsigned char *addr,
- int len)
+static inline int oakscsi_pwrite(struct Scsi_Host *instance,
+ unsigned char *addr, int len)
{
void __iomem *base = priv(instance)->base;
printk("writing %p len %d\n",addr, len);
- if(!len) return -1;
while(1)
{
int status;
while (((status = readw(base + STAT)) & 0x100)==0);
}
+ return 0;
}
-static inline int NCR5380_pread(struct Scsi_Host *instance, unsigned char *addr,
- int len)
+static inline int oakscsi_pread(struct Scsi_Host *instance,
+ unsigned char *addr, int len)
{
void __iomem *base = priv(instance)->base;
printk("reading %p len %d\n", addr, len);
@@ -73,7 +73,7 @@
if(status & 0x200 || !timeout)
{
printk("status = %08X\n", status);
- return 1;
+ return -1;
}
}
@@ -143,7 +143,7 @@
host->irq = NO_IRQ;
host->n_io_port = 255;
- ret = NCR5380_init(host, 0);
+ ret = NCR5380_init(host, FLAG_DMA_FIXUP | FLAG_LATE_DMA_SETUP);
if (ret)
goto out_unmap;
diff --git a/drivers/scsi/arm/powertec.c b/drivers/scsi/arm/powertec.c
index 5e1b73e..79aa889 100644
--- a/drivers/scsi/arm/powertec.c
+++ b/drivers/scsi/arm/powertec.c
@@ -291,7 +291,7 @@
.can_queue = 8,
.this_id = 7,
- .sg_tablesize = SCSI_MAX_SG_CHAIN_SEGMENTS,
+ .sg_tablesize = SG_MAX_SEGMENTS,
.dma_boundary = IOMD_DMA_BOUNDARY,
.cmd_per_lun = 2,
.use_clustering = ENABLE_CLUSTERING,
diff --git a/drivers/scsi/atari_NCR5380.c b/drivers/scsi/atari_NCR5380.c
deleted file mode 100644
index 389825b..0000000
--- a/drivers/scsi/atari_NCR5380.c
+++ /dev/null
@@ -1,2676 +0,0 @@
-/*
- * NCR 5380 generic driver routines. These should make it *trivial*
- * to implement 5380 SCSI drivers under Linux with a non-trantor
- * architecture.
- *
- * Note that these routines also work with NR53c400 family chips.
- *
- * Copyright 1993, Drew Eckhardt
- * Visionary Computing
- * (Unix and Linux consulting and custom programming)
- * drew@colorado.edu
- * +1 (303) 666-5836
- *
- * For more information, please consult
- *
- * NCR 5380 Family
- * SCSI Protocol Controller
- * Databook
- *
- * NCR Microelectronics
- * 1635 Aeroplaza Drive
- * Colorado Springs, CO 80916
- * 1+ (719) 578-3400
- * 1+ (800) 334-5454
- */
-
-/* Ported to Atari by Roman Hodek and others. */
-
-/* Adapted for the sun3 by Sam Creasey. */
-
-/*
- * Design
- *
- * This is a generic 5380 driver. To use it on a different platform,
- * one simply writes appropriate system specific macros (ie, data
- * transfer - some PC's will use the I/O bus, 68K's must use
- * memory mapped) and drops this file in their 'C' wrapper.
- *
- * As far as command queueing, two queues are maintained for
- * each 5380 in the system - commands that haven't been issued yet,
- * and commands that are currently executing. This means that an
- * unlimited number of commands may be queued, letting
- * more commands propagate from the higher driver levels giving higher
- * throughput. Note that both I_T_L and I_T_L_Q nexuses are supported,
- * allowing multiple commands to propagate all the way to a SCSI-II device
- * while a command is already executing.
- *
- *
- * Issues specific to the NCR5380 :
- *
- * When used in a PIO or pseudo-dma mode, the NCR5380 is a braindead
- * piece of hardware that requires you to sit in a loop polling for
- * the REQ signal as long as you are connected. Some devices are
- * brain dead (ie, many TEXEL CD ROM drives) and won't disconnect
- * while doing long seek operations. [...] These
- * broken devices are the exception rather than the rule and I'd rather
- * spend my time optimizing for the normal case.
- *
- * Architecture :
- *
- * At the heart of the design is a coroutine, NCR5380_main,
- * which is started from a workqueue for each NCR5380 host in the
- * system. It attempts to establish I_T_L or I_T_L_Q nexuses by
- * removing the commands from the issue queue and calling
- * NCR5380_select() if a nexus is not established.
- *
- * Once a nexus is established, the NCR5380_information_transfer()
- * phase goes through the various phases as instructed by the target.
- * if the target goes into MSG IN and sends a DISCONNECT message,
- * the command structure is placed into the per instance disconnected
- * queue, and NCR5380_main tries to find more work. If the target is
- * idle for too long, the system will try to sleep.
- *
- * If a command has disconnected, eventually an interrupt will trigger,
- * calling NCR5380_intr() which will in turn call NCR5380_reselect
- * to reestablish a nexus. This will run main if necessary.
- *
- * On command termination, the done function will be called as
- * appropriate.
- *
- * SCSI pointers are maintained in the SCp field of SCSI command
- * structures, being initialized after the command is connected
- * in NCR5380_select, and set as appropriate in NCR5380_information_transfer.
- * Note that in violation of the standard, an implicit SAVE POINTERS operation
- * is done, since some BROKEN disks fail to issue an explicit SAVE POINTERS.
- */
-
-/*
- * Using this file :
- * This file a skeleton Linux SCSI driver for the NCR 5380 series
- * of chips. To use it, you write an architecture specific functions
- * and macros and include this file in your driver.
- *
- * These macros control options :
- * AUTOSENSE - if defined, REQUEST SENSE will be performed automatically
- * for commands that return with a CHECK CONDITION status.
- *
- * DIFFERENTIAL - if defined, NCR53c81 chips will use external differential
- * transceivers.
- *
- * REAL_DMA - if defined, REAL DMA is used during the data transfer phases.
- *
- * SUPPORT_TAGS - if defined, SCSI-2 tagged queuing is used where possible
- *
- * These macros MUST be defined :
- *
- * NCR5380_read(register) - read from the specified register
- *
- * NCR5380_write(register, value) - write to the specific register
- *
- * NCR5380_implementation_fields - additional fields needed for this
- * specific implementation of the NCR5380
- *
- * Either real DMA *or* pseudo DMA may be implemented
- * REAL functions :
- * NCR5380_REAL_DMA should be defined if real DMA is to be used.
- * Note that the DMA setup functions should return the number of bytes
- * that they were able to program the controller for.
- *
- * Also note that generic i386/PC versions of these macros are
- * available as NCR5380_i386_dma_write_setup,
- * NCR5380_i386_dma_read_setup, and NCR5380_i386_dma_residual.
- *
- * NCR5380_dma_write_setup(instance, src, count) - initialize
- * NCR5380_dma_read_setup(instance, dst, count) - initialize
- * NCR5380_dma_residual(instance); - residual count
- *
- * PSEUDO functions :
- * NCR5380_pwrite(instance, src, count)
- * NCR5380_pread(instance, dst, count);
- *
- * The generic driver is initialized by calling NCR5380_init(instance),
- * after setting the appropriate host specific fields and ID. If the
- * driver wishes to autoprobe for an IRQ line, the NCR5380_probe_irq(instance,
- * possible) function may be used.
- */
-
-static int do_abort(struct Scsi_Host *);
-static void do_reset(struct Scsi_Host *);
-
-#ifdef SUPPORT_TAGS
-
-/*
- * Functions for handling tagged queuing
- * =====================================
- *
- * ++roman (01/96): Now I've implemented SCSI-2 tagged queuing. Some notes:
- *
- * Using consecutive numbers for the tags is no good idea in my eyes. There
- * could be wrong re-usings if the counter (8 bit!) wraps and some early
- * command has been preempted for a long time. My solution: a bitfield for
- * remembering used tags.
- *
- * There's also the problem that each target has a certain queue size, but we
- * cannot know it in advance :-( We just see a QUEUE_FULL status being
- * returned. So, in this case, the driver internal queue size assumption is
- * reduced to the number of active tags if QUEUE_FULL is returned by the
- * target.
- *
- * We're also not allowed running tagged commands as long as an untagged
- * command is active. And REQUEST SENSE commands after a contingent allegiance
- * condition _must_ be untagged. To keep track whether an untagged command has
- * been issued, the host->busy array is still employed, as it is without
- * support for tagged queuing.
- *
- * One could suspect that there are possible race conditions between
- * is_lun_busy(), cmd_get_tag() and cmd_free_tag(). But I think this isn't the
- * case: is_lun_busy() and cmd_get_tag() are both called from NCR5380_main(),
- * which already guaranteed to be running at most once. It is also the only
- * place where tags/LUNs are allocated. So no other allocation can slip
- * between that pair, there could only happen a reselection, which can free a
- * tag, but that doesn't hurt. Only the sequence in cmd_free_tag() becomes
- * important: the tag bit must be cleared before 'nr_allocated' is decreased.
- */
-
-static void __init init_tags(struct NCR5380_hostdata *hostdata)
-{
- int target, lun;
- struct tag_alloc *ta;
-
- if (!(hostdata->flags & FLAG_TAGGED_QUEUING))
- return;
-
- for (target = 0; target < 8; ++target) {
- for (lun = 0; lun < 8; ++lun) {
- ta = &hostdata->TagAlloc[target][lun];
- bitmap_zero(ta->allocated, MAX_TAGS);
- ta->nr_allocated = 0;
- /* At the beginning, assume the maximum queue size we could
- * support (MAX_TAGS). This value will be decreased if the target
- * returns QUEUE_FULL status.
- */
- ta->queue_size = MAX_TAGS;
- }
- }
-}
-
-
-/* Check if we can issue a command to this LUN: First see if the LUN is marked
- * busy by an untagged command. If the command should use tagged queuing, also
- * check that there is a free tag and the target's queue won't overflow. This
- * function should be called with interrupts disabled to avoid race
- * conditions.
- */
-
-static int is_lun_busy(struct scsi_cmnd *cmd, int should_be_tagged)
-{
- u8 lun = cmd->device->lun;
- struct Scsi_Host *instance = cmd->device->host;
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
-
- if (hostdata->busy[cmd->device->id] & (1 << lun))
- return 1;
- if (!should_be_tagged ||
- !(hostdata->flags & FLAG_TAGGED_QUEUING) ||
- !cmd->device->tagged_supported)
- return 0;
- if (hostdata->TagAlloc[scmd_id(cmd)][lun].nr_allocated >=
- hostdata->TagAlloc[scmd_id(cmd)][lun].queue_size) {
- dsprintk(NDEBUG_TAGS, instance, "target %d lun %d: no free tags\n",
- scmd_id(cmd), lun);
- return 1;
- }
- return 0;
-}
-
-
-/* Allocate a tag for a command (there are no checks anymore, check_lun_busy()
- * must be called before!), or reserve the LUN in 'busy' if the command is
- * untagged.
- */
-
-static void cmd_get_tag(struct scsi_cmnd *cmd, int should_be_tagged)
-{
- u8 lun = cmd->device->lun;
- struct Scsi_Host *instance = cmd->device->host;
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
-
- /* If we or the target don't support tagged queuing, allocate the LUN for
- * an untagged command.
- */
- if (!should_be_tagged ||
- !(hostdata->flags & FLAG_TAGGED_QUEUING) ||
- !cmd->device->tagged_supported) {
- cmd->tag = TAG_NONE;
- hostdata->busy[cmd->device->id] |= (1 << lun);
- dsprintk(NDEBUG_TAGS, instance, "target %d lun %d now allocated by untagged command\n",
- scmd_id(cmd), lun);
- } else {
- struct tag_alloc *ta = &hostdata->TagAlloc[scmd_id(cmd)][lun];
-
- cmd->tag = find_first_zero_bit(ta->allocated, MAX_TAGS);
- set_bit(cmd->tag, ta->allocated);
- ta->nr_allocated++;
- dsprintk(NDEBUG_TAGS, instance, "using tag %d for target %d lun %d (%d tags allocated)\n",
- cmd->tag, scmd_id(cmd), lun, ta->nr_allocated);
- }
-}
-
-
-/* Mark the tag of command 'cmd' as free, or in case of an untagged command,
- * unlock the LUN.
- */
-
-static void cmd_free_tag(struct scsi_cmnd *cmd)
-{
- u8 lun = cmd->device->lun;
- struct Scsi_Host *instance = cmd->device->host;
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
-
- if (cmd->tag == TAG_NONE) {
- hostdata->busy[cmd->device->id] &= ~(1 << lun);
- dsprintk(NDEBUG_TAGS, instance, "target %d lun %d untagged cmd freed\n",
- scmd_id(cmd), lun);
- } else if (cmd->tag >= MAX_TAGS) {
- shost_printk(KERN_NOTICE, instance,
- "trying to free bad tag %d!\n", cmd->tag);
- } else {
- struct tag_alloc *ta = &hostdata->TagAlloc[scmd_id(cmd)][lun];
- clear_bit(cmd->tag, ta->allocated);
- ta->nr_allocated--;
- dsprintk(NDEBUG_TAGS, instance, "freed tag %d for target %d lun %d\n",
- cmd->tag, scmd_id(cmd), lun);
- }
-}
-
-
-static void free_all_tags(struct NCR5380_hostdata *hostdata)
-{
- int target, lun;
- struct tag_alloc *ta;
-
- if (!(hostdata->flags & FLAG_TAGGED_QUEUING))
- return;
-
- for (target = 0; target < 8; ++target) {
- for (lun = 0; lun < 8; ++lun) {
- ta = &hostdata->TagAlloc[target][lun];
- bitmap_zero(ta->allocated, MAX_TAGS);
- ta->nr_allocated = 0;
- }
- }
-}
-
-#endif /* SUPPORT_TAGS */
-
-/**
- * merge_contiguous_buffers - coalesce scatter-gather list entries
- * @cmd: command requesting IO
- *
- * Try to merge several scatter-gather buffers into one DMA transfer.
- * This is possible if the scatter buffers lie on physically
- * contiguous addresses. The first scatter-gather buffer's data are
- * assumed to be already transferred into cmd->SCp.this_residual.
- * Every buffer merged avoids an interrupt and a DMA setup operation.
- */
-
-static void merge_contiguous_buffers(struct scsi_cmnd *cmd)
-{
-#if !defined(CONFIG_SUN3)
- unsigned long endaddr;
-#if (NDEBUG & NDEBUG_MERGING)
- unsigned long oldlen = cmd->SCp.this_residual;
- int cnt = 1;
-#endif
-
- for (endaddr = virt_to_phys(cmd->SCp.ptr + cmd->SCp.this_residual - 1) + 1;
- cmd->SCp.buffers_residual &&
- virt_to_phys(sg_virt(&cmd->SCp.buffer[1])) == endaddr;) {
- dprintk(NDEBUG_MERGING, "VTOP(%p) == %08lx -> merging\n",
- page_address(sg_page(&cmd->SCp.buffer[1])), endaddr);
-#if (NDEBUG & NDEBUG_MERGING)
- ++cnt;
-#endif
- ++cmd->SCp.buffer;
- --cmd->SCp.buffers_residual;
- cmd->SCp.this_residual += cmd->SCp.buffer->length;
- endaddr += cmd->SCp.buffer->length;
- }
-#if (NDEBUG & NDEBUG_MERGING)
- if (oldlen != cmd->SCp.this_residual)
- dprintk(NDEBUG_MERGING, "merged %d buffers from %p, new length %08x\n",
- cnt, cmd->SCp.ptr, cmd->SCp.this_residual);
-#endif
-#endif /* !defined(CONFIG_SUN3) */
-}
-
-/**
- * initialize_SCp - init the scsi pointer field
- * @cmd: command block to set up
- *
- * Set up the internal fields in the SCSI command.
- */
-
-static inline void initialize_SCp(struct scsi_cmnd *cmd)
-{
- /*
- * Initialize the Scsi Pointer field so that all of the commands in the
- * various queues are valid.
- */
-
- if (scsi_bufflen(cmd)) {
- cmd->SCp.buffer = scsi_sglist(cmd);
- cmd->SCp.buffers_residual = scsi_sg_count(cmd) - 1;
- cmd->SCp.ptr = sg_virt(cmd->SCp.buffer);
- cmd->SCp.this_residual = cmd->SCp.buffer->length;
-
- merge_contiguous_buffers(cmd);
- } else {
- cmd->SCp.buffer = NULL;
- cmd->SCp.buffers_residual = 0;
- cmd->SCp.ptr = NULL;
- cmd->SCp.this_residual = 0;
- }
-
- cmd->SCp.Status = 0;
- cmd->SCp.Message = 0;
-}
-
-/**
- * NCR5380_poll_politely2 - wait for two chip register values
- * @instance: controller to poll
- * @reg1: 5380 register to poll
- * @bit1: Bitmask to check
- * @val1: Expected value
- * @reg2: Second 5380 register to poll
- * @bit2: Second bitmask to check
- * @val2: Second expected value
- * @wait: Time-out in jiffies
- *
- * Polls the chip in a reasonably efficient manner waiting for an
- * event to occur. After a short quick poll we begin to yield the CPU
- * (if possible). In irq contexts the time-out is arbitrarily limited.
- * Callers may hold locks as long as they are held in irq mode.
- *
- * Returns 0 if either or both event(s) occurred otherwise -ETIMEDOUT.
- */
-
-static int NCR5380_poll_politely2(struct Scsi_Host *instance,
- int reg1, int bit1, int val1,
- int reg2, int bit2, int val2, int wait)
-{
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
- unsigned long deadline = jiffies + wait;
- unsigned long n;
-
- /* Busy-wait for up to 10 ms */
- n = min(10000U, jiffies_to_usecs(wait));
- n *= hostdata->accesses_per_ms;
- n /= 2000;
- do {
- if ((NCR5380_read(reg1) & bit1) == val1)
- return 0;
- if ((NCR5380_read(reg2) & bit2) == val2)
- return 0;
- cpu_relax();
- } while (n--);
-
- if (irqs_disabled() || in_interrupt())
- return -ETIMEDOUT;
-
- /* Repeatedly sleep for 1 ms until deadline */
- while (time_is_after_jiffies(deadline)) {
- schedule_timeout_uninterruptible(1);
- if ((NCR5380_read(reg1) & bit1) == val1)
- return 0;
- if ((NCR5380_read(reg2) & bit2) == val2)
- return 0;
- }
-
- return -ETIMEDOUT;
-}
-
-static inline int NCR5380_poll_politely(struct Scsi_Host *instance,
- int reg, int bit, int val, int wait)
-{
- return NCR5380_poll_politely2(instance, reg, bit, val,
- reg, bit, val, wait);
-}
-
-#if NDEBUG
-static struct {
- unsigned char mask;
- const char *name;
-} signals[] = {
- {SR_DBP, "PARITY"},
- {SR_RST, "RST"},
- {SR_BSY, "BSY"},
- {SR_REQ, "REQ"},
- {SR_MSG, "MSG"},
- {SR_CD, "CD"},
- {SR_IO, "IO"},
- {SR_SEL, "SEL"},
- {0, NULL}
-},
-basrs[] = {
- {BASR_ATN, "ATN"},
- {BASR_ACK, "ACK"},
- {0, NULL}
-},
-icrs[] = {
- {ICR_ASSERT_RST, "ASSERT RST"},
- {ICR_ASSERT_ACK, "ASSERT ACK"},
- {ICR_ASSERT_BSY, "ASSERT BSY"},
- {ICR_ASSERT_SEL, "ASSERT SEL"},
- {ICR_ASSERT_ATN, "ASSERT ATN"},
- {ICR_ASSERT_DATA, "ASSERT DATA"},
- {0, NULL}
-},
-mrs[] = {
- {MR_BLOCK_DMA_MODE, "MODE BLOCK DMA"},
- {MR_TARGET, "MODE TARGET"},
- {MR_ENABLE_PAR_CHECK, "MODE PARITY CHECK"},
- {MR_ENABLE_PAR_INTR, "MODE PARITY INTR"},
- {MR_ENABLE_EOP_INTR, "MODE EOP INTR"},
- {MR_MONITOR_BSY, "MODE MONITOR BSY"},
- {MR_DMA_MODE, "MODE DMA"},
- {MR_ARBITRATE, "MODE ARBITRATION"},
- {0, NULL}
-};
-
-/**
- * NCR5380_print - print scsi bus signals
- * @instance: adapter state to dump
- *
- * Print the SCSI bus signals for debugging purposes
- */
-
-static void NCR5380_print(struct Scsi_Host *instance)
-{
- unsigned char status, data, basr, mr, icr, i;
-
- data = NCR5380_read(CURRENT_SCSI_DATA_REG);
- status = NCR5380_read(STATUS_REG);
- mr = NCR5380_read(MODE_REG);
- icr = NCR5380_read(INITIATOR_COMMAND_REG);
- basr = NCR5380_read(BUS_AND_STATUS_REG);
-
- printk("STATUS_REG: %02x ", status);
- for (i = 0; signals[i].mask; ++i)
- if (status & signals[i].mask)
- printk(",%s", signals[i].name);
- printk("\nBASR: %02x ", basr);
- for (i = 0; basrs[i].mask; ++i)
- if (basr & basrs[i].mask)
- printk(",%s", basrs[i].name);
- printk("\nICR: %02x ", icr);
- for (i = 0; icrs[i].mask; ++i)
- if (icr & icrs[i].mask)
- printk(",%s", icrs[i].name);
- printk("\nMODE: %02x ", mr);
- for (i = 0; mrs[i].mask; ++i)
- if (mr & mrs[i].mask)
- printk(",%s", mrs[i].name);
- printk("\n");
-}
-
-static struct {
- unsigned char value;
- const char *name;
-} phases[] = {
- {PHASE_DATAOUT, "DATAOUT"},
- {PHASE_DATAIN, "DATAIN"},
- {PHASE_CMDOUT, "CMDOUT"},
- {PHASE_STATIN, "STATIN"},
- {PHASE_MSGOUT, "MSGOUT"},
- {PHASE_MSGIN, "MSGIN"},
- {PHASE_UNKNOWN, "UNKNOWN"}
-};
-
-/**
- * NCR5380_print_phase - show SCSI phase
- * @instance: adapter to dump
- *
- * Print the current SCSI phase for debugging purposes
- */
-
-static void NCR5380_print_phase(struct Scsi_Host *instance)
-{
- unsigned char status;
- int i;
-
- status = NCR5380_read(STATUS_REG);
- if (!(status & SR_REQ))
- shost_printk(KERN_DEBUG, instance, "REQ not asserted, phase unknown.\n");
- else {
- for (i = 0; (phases[i].value != PHASE_UNKNOWN) &&
- (phases[i].value != (status & PHASE_MASK)); ++i)
- ;
- shost_printk(KERN_DEBUG, instance, "phase %s\n", phases[i].name);
- }
-}
-#endif
-
-/**
- * NCR58380_info - report driver and host information
- * @instance: relevant scsi host instance
- *
- * For use as the host template info() handler.
- */
-
-static const char *NCR5380_info(struct Scsi_Host *instance)
-{
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
-
- return hostdata->info;
-}
-
-static void prepare_info(struct Scsi_Host *instance)
-{
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
-
- snprintf(hostdata->info, sizeof(hostdata->info),
- "%s, io_port 0x%lx, n_io_port %d, "
- "base 0x%lx, irq %d, "
- "can_queue %d, cmd_per_lun %d, "
- "sg_tablesize %d, this_id %d, "
- "flags { %s%s}, "
- "options { %s} ",
- instance->hostt->name, instance->io_port, instance->n_io_port,
- instance->base, instance->irq,
- instance->can_queue, instance->cmd_per_lun,
- instance->sg_tablesize, instance->this_id,
- hostdata->flags & FLAG_TAGGED_QUEUING ? "TAGGED_QUEUING " : "",
- hostdata->flags & FLAG_TOSHIBA_DELAY ? "TOSHIBA_DELAY " : "",
-#ifdef DIFFERENTIAL
- "DIFFERENTIAL "
-#endif
-#ifdef REAL_DMA
- "REAL_DMA "
-#endif
-#ifdef PARITY
- "PARITY "
-#endif
-#ifdef SUPPORT_TAGS
- "SUPPORT_TAGS "
-#endif
- "");
-}
-
-/**
- * NCR5380_init - initialise an NCR5380
- * @instance: adapter to configure
- * @flags: control flags
- *
- * Initializes *instance and corresponding 5380 chip,
- * with flags OR'd into the initial flags value.
- *
- * Notes : I assume that the host, hostno, and id bits have been
- * set correctly. I don't care about the irq and other fields.
- *
- * Returns 0 for success
- */
-
-static int __init NCR5380_init(struct Scsi_Host *instance, int flags)
-{
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
- int i;
- unsigned long deadline;
-
- hostdata->host = instance;
- hostdata->id_mask = 1 << instance->this_id;
- hostdata->id_higher_mask = 0;
- for (i = hostdata->id_mask; i <= 0x80; i <<= 1)
- if (i > hostdata->id_mask)
- hostdata->id_higher_mask |= i;
- for (i = 0; i < 8; ++i)
- hostdata->busy[i] = 0;
-#ifdef SUPPORT_TAGS
- init_tags(hostdata);
-#endif
-#if defined (REAL_DMA)
- hostdata->dma_len = 0;
-#endif
- spin_lock_init(&hostdata->lock);
- hostdata->connected = NULL;
- hostdata->sensing = NULL;
- INIT_LIST_HEAD(&hostdata->autosense);
- INIT_LIST_HEAD(&hostdata->unissued);
- INIT_LIST_HEAD(&hostdata->disconnected);
-
- hostdata->flags = flags;
-
- INIT_WORK(&hostdata->main_task, NCR5380_main);
- hostdata->work_q = alloc_workqueue("ncr5380_%d",
- WQ_UNBOUND | WQ_MEM_RECLAIM,
- 1, instance->host_no);
- if (!hostdata->work_q)
- return -ENOMEM;
-
- prepare_info(instance);
-
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- NCR5380_write(MODE_REG, MR_BASE);
- NCR5380_write(TARGET_COMMAND_REG, 0);
- NCR5380_write(SELECT_ENABLE_REG, 0);
-
- /* Calibrate register polling loop */
- i = 0;
- deadline = jiffies + 1;
- do {
- cpu_relax();
- } while (time_is_after_jiffies(deadline));
- deadline += msecs_to_jiffies(256);
- do {
- NCR5380_read(STATUS_REG);
- ++i;
- cpu_relax();
- } while (time_is_after_jiffies(deadline));
- hostdata->accesses_per_ms = i / 256;
-
- return 0;
-}
-
-/**
- * NCR5380_maybe_reset_bus - Detect and correct bus wedge problems.
- * @instance: adapter to check
- *
- * If the system crashed, it may have crashed with a connected target and
- * the SCSI bus busy. Check for BUS FREE phase. If not, try to abort the
- * currently established nexus, which we know nothing about. Failing that
- * do a bus reset.
- *
- * Note that a bus reset will cause the chip to assert IRQ.
- *
- * Returns 0 if successful, otherwise -ENXIO.
- */
-
-static int NCR5380_maybe_reset_bus(struct Scsi_Host *instance)
-{
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
- int pass;
-
- for (pass = 1; (NCR5380_read(STATUS_REG) & SR_BSY) && pass <= 6; ++pass) {
- switch (pass) {
- case 1:
- case 3:
- case 5:
- shost_printk(KERN_ERR, instance, "SCSI bus busy, waiting up to five seconds\n");
- NCR5380_poll_politely(instance,
- STATUS_REG, SR_BSY, 0, 5 * HZ);
- break;
- case 2:
- shost_printk(KERN_ERR, instance, "bus busy, attempting abort\n");
- do_abort(instance);
- break;
- case 4:
- shost_printk(KERN_ERR, instance, "bus busy, attempting reset\n");
- do_reset(instance);
- /* Wait after a reset; the SCSI standard calls for
- * 250ms, we wait 500ms to be on the safe side.
- * But some Toshiba CD-ROMs need ten times that.
- */
- if (hostdata->flags & FLAG_TOSHIBA_DELAY)
- msleep(2500);
- else
- msleep(500);
- break;
- case 6:
- shost_printk(KERN_ERR, instance, "bus locked solid\n");
- return -ENXIO;
- }
- }
- return 0;
-}
-
-/**
- * NCR5380_exit - remove an NCR5380
- * @instance: adapter to remove
- *
- * Assumes that no more work can be queued (e.g. by NCR5380_intr).
- */
-
-static void NCR5380_exit(struct Scsi_Host *instance)
-{
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
-
- cancel_work_sync(&hostdata->main_task);
- destroy_workqueue(hostdata->work_q);
-}
-
-/**
- * complete_cmd - finish processing a command and return it to the SCSI ML
- * @instance: the host instance
- * @cmd: command to complete
- */
-
-static void complete_cmd(struct Scsi_Host *instance,
- struct scsi_cmnd *cmd)
-{
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
-
- dsprintk(NDEBUG_QUEUES, instance, "complete_cmd: cmd %p\n", cmd);
-
- if (hostdata->sensing == cmd) {
- /* Autosense processing ends here */
- if ((cmd->result & 0xff) != SAM_STAT_GOOD) {
- scsi_eh_restore_cmnd(cmd, &hostdata->ses);
- set_host_byte(cmd, DID_ERROR);
- } else
- scsi_eh_restore_cmnd(cmd, &hostdata->ses);
- hostdata->sensing = NULL;
- }
-
-#ifdef SUPPORT_TAGS
- cmd_free_tag(cmd);
-#else
- hostdata->busy[scmd_id(cmd)] &= ~(1 << cmd->device->lun);
-#endif
- cmd->scsi_done(cmd);
-}
-
-/**
- * NCR5380_queue_command - queue a command
- * @instance: the relevant SCSI adapter
- * @cmd: SCSI command
- *
- * cmd is added to the per-instance issue queue, with minor
- * twiddling done to the host specific fields of cmd. If the
- * main coroutine is not running, it is restarted.
- */
-
-static int NCR5380_queue_command(struct Scsi_Host *instance,
- struct scsi_cmnd *cmd)
-{
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
- struct NCR5380_cmd *ncmd = scsi_cmd_priv(cmd);
- unsigned long flags;
-
-#if (NDEBUG & NDEBUG_NO_WRITE)
- switch (cmd->cmnd[0]) {
- case WRITE_6:
- case WRITE_10:
- shost_printk(KERN_DEBUG, instance, "WRITE attempted with NDEBUG_NO_WRITE set\n");
- cmd->result = (DID_ERROR << 16);
- cmd->scsi_done(cmd);
- return 0;
- }
-#endif /* (NDEBUG & NDEBUG_NO_WRITE) */
-
- cmd->result = 0;
-
- /*
- * ++roman: Just disabling the NCR interrupt isn't sufficient here,
- * because also a timer int can trigger an abort or reset, which would
- * alter queues and touch the lock.
- */
- if (!NCR5380_acquire_dma_irq(instance))
- return SCSI_MLQUEUE_HOST_BUSY;
-
- spin_lock_irqsave(&hostdata->lock, flags);
-
- /*
- * Insert the cmd into the issue queue. Note that REQUEST SENSE
- * commands are added to the head of the queue since any command will
- * clear the contingent allegiance condition that exists and the
- * sense data is only guaranteed to be valid while the condition exists.
- */
-
- if (cmd->cmnd[0] == REQUEST_SENSE)
- list_add(&ncmd->list, &hostdata->unissued);
- else
- list_add_tail(&ncmd->list, &hostdata->unissued);
-
- spin_unlock_irqrestore(&hostdata->lock, flags);
-
- dsprintk(NDEBUG_QUEUES, instance, "command %p added to %s of queue\n",
- cmd, (cmd->cmnd[0] == REQUEST_SENSE) ? "head" : "tail");
-
- /* Kick off command processing */
- queue_work(hostdata->work_q, &hostdata->main_task);
- return 0;
-}
-
-static inline void maybe_release_dma_irq(struct Scsi_Host *instance)
-{
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
-
- /* Caller does the locking needed to set & test these data atomically */
- if (list_empty(&hostdata->disconnected) &&
- list_empty(&hostdata->unissued) &&
- list_empty(&hostdata->autosense) &&
- !hostdata->connected &&
- !hostdata->selecting)
- NCR5380_release_dma_irq(instance);
-}
-
-/**
- * dequeue_next_cmd - dequeue a command for processing
- * @instance: the scsi host instance
- *
- * Priority is given to commands on the autosense queue. These commands
- * need autosense because of a CHECK CONDITION result.
- *
- * Returns a command pointer if a command is found for a target that is
- * not already busy. Otherwise returns NULL.
- */
-
-static struct scsi_cmnd *dequeue_next_cmd(struct Scsi_Host *instance)
-{
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
- struct NCR5380_cmd *ncmd;
- struct scsi_cmnd *cmd;
-
- if (hostdata->sensing || list_empty(&hostdata->autosense)) {
- list_for_each_entry(ncmd, &hostdata->unissued, list) {
- cmd = NCR5380_to_scmd(ncmd);
- dsprintk(NDEBUG_QUEUES, instance, "dequeue: cmd=%p target=%d busy=0x%02x lun=%llu\n",
- cmd, scmd_id(cmd), hostdata->busy[scmd_id(cmd)], cmd->device->lun);
-
- if (
-#ifdef SUPPORT_TAGS
- !is_lun_busy(cmd, 1)
-#else
- !(hostdata->busy[scmd_id(cmd)] & (1 << cmd->device->lun))
-#endif
- ) {
- list_del(&ncmd->list);
- dsprintk(NDEBUG_QUEUES, instance,
- "dequeue: removed %p from issue queue\n", cmd);
- return cmd;
- }
- }
- } else {
- /* Autosense processing begins here */
- ncmd = list_first_entry(&hostdata->autosense,
- struct NCR5380_cmd, list);
- list_del(&ncmd->list);
- cmd = NCR5380_to_scmd(ncmd);
- dsprintk(NDEBUG_QUEUES, instance,
- "dequeue: removed %p from autosense queue\n", cmd);
- scsi_eh_prep_cmnd(cmd, &hostdata->ses, NULL, 0, ~0);
- hostdata->sensing = cmd;
- return cmd;
- }
- return NULL;
-}
-
-static void requeue_cmd(struct Scsi_Host *instance, struct scsi_cmnd *cmd)
-{
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
- struct NCR5380_cmd *ncmd = scsi_cmd_priv(cmd);
-
- if (hostdata->sensing == cmd) {
- scsi_eh_restore_cmnd(cmd, &hostdata->ses);
- list_add(&ncmd->list, &hostdata->autosense);
- hostdata->sensing = NULL;
- } else
- list_add(&ncmd->list, &hostdata->unissued);
-}
-
-/**
- * NCR5380_main - NCR state machines
- *
- * NCR5380_main is a coroutine that runs as long as more work can
- * be done on the NCR5380 host adapters in a system. Both
- * NCR5380_queue_command() and NCR5380_intr() will try to start it
- * in case it is not running.
- */
-
-static void NCR5380_main(struct work_struct *work)
-{
- struct NCR5380_hostdata *hostdata =
- container_of(work, struct NCR5380_hostdata, main_task);
- struct Scsi_Host *instance = hostdata->host;
- int done;
-
- /*
- * ++roman: Just disabling the NCR interrupt isn't sufficient here,
- * because also a timer int can trigger an abort or reset, which can
- * alter queues and touch the Falcon lock.
- */
-
- do {
- done = 1;
-
- spin_lock_irq(&hostdata->lock);
- while (!hostdata->connected && !hostdata->selecting) {
- struct scsi_cmnd *cmd = dequeue_next_cmd(instance);
-
- if (!cmd)
- break;
-
- dsprintk(NDEBUG_MAIN, instance, "main: dequeued %p\n", cmd);
-
- /*
- * Attempt to establish an I_T_L nexus here.
- * On success, instance->hostdata->connected is set.
- * On failure, we must add the command back to the
- * issue queue so we can keep trying.
- */
- /*
- * REQUEST SENSE commands are issued without tagged
- * queueing, even on SCSI-II devices because the
- * contingent allegiance condition exists for the
- * entire unit.
- */
- /* ++roman: ...and the standard also requires that
- * REQUEST SENSE command are untagged.
- */
-
-#ifdef SUPPORT_TAGS
- cmd_get_tag(cmd, cmd->cmnd[0] != REQUEST_SENSE);
-#endif
- if (!NCR5380_select(instance, cmd)) {
- dsprintk(NDEBUG_MAIN, instance, "main: select complete\n");
- maybe_release_dma_irq(instance);
- } else {
- dsprintk(NDEBUG_MAIN | NDEBUG_QUEUES, instance,
- "main: select failed, returning %p to queue\n", cmd);
- requeue_cmd(instance, cmd);
-#ifdef SUPPORT_TAGS
- cmd_free_tag(cmd);
-#endif
- }
- }
- if (hostdata->connected
-#ifdef REAL_DMA
- && !hostdata->dma_len
-#endif
- ) {
- dsprintk(NDEBUG_MAIN, instance, "main: performing information transfer\n");
- NCR5380_information_transfer(instance);
- done = 0;
- }
- spin_unlock_irq(&hostdata->lock);
- if (!done)
- cond_resched();
- } while (!done);
-}
-
-
-#ifdef REAL_DMA
-/*
- * Function : void NCR5380_dma_complete (struct Scsi_Host *instance)
- *
- * Purpose : Called by interrupt handler when DMA finishes or a phase
- * mismatch occurs (which would finish the DMA transfer).
- *
- * Inputs : instance - this instance of the NCR5380.
- */
-
-static void NCR5380_dma_complete(struct Scsi_Host *instance)
-{
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
- int transferred;
- unsigned char **data;
- int *count;
- int saved_data = 0, overrun = 0;
- unsigned char p;
-
- if (hostdata->read_overruns) {
- p = hostdata->connected->SCp.phase;
- if (p & SR_IO) {
- udelay(10);
- if ((NCR5380_read(BUS_AND_STATUS_REG) &
- (BASR_PHASE_MATCH|BASR_ACK)) ==
- (BASR_PHASE_MATCH|BASR_ACK)) {
- saved_data = NCR5380_read(INPUT_DATA_REG);
- overrun = 1;
- dsprintk(NDEBUG_DMA, instance, "read overrun handled\n");
- }
- }
- }
-
-#if defined(CONFIG_SUN3)
- if ((sun3scsi_dma_finish(rq_data_dir(hostdata->connected->request)))) {
- pr_err("scsi%d: overrun in UDC counter -- not prepared to deal with this!\n",
- instance->host_no);
- BUG();
- }
-
- /* make sure we're not stuck in a data phase */
- if ((NCR5380_read(BUS_AND_STATUS_REG) & (BASR_PHASE_MATCH | BASR_ACK)) ==
- (BASR_PHASE_MATCH | BASR_ACK)) {
- pr_err("scsi%d: BASR %02x\n", instance->host_no,
- NCR5380_read(BUS_AND_STATUS_REG));
- pr_err("scsi%d: bus stuck in data phase -- probably a single byte overrun!\n",
- instance->host_no);
- BUG();
- }
-#endif
-
- NCR5380_write(MODE_REG, MR_BASE);
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- NCR5380_read(RESET_PARITY_INTERRUPT_REG);
-
- transferred = hostdata->dma_len - NCR5380_dma_residual(instance);
- hostdata->dma_len = 0;
-
- data = (unsigned char **)&hostdata->connected->SCp.ptr;
- count = &hostdata->connected->SCp.this_residual;
- *data += transferred;
- *count -= transferred;
-
- if (hostdata->read_overruns) {
- int cnt, toPIO;
-
- if ((NCR5380_read(STATUS_REG) & PHASE_MASK) == p && (p & SR_IO)) {
- cnt = toPIO = hostdata->read_overruns;
- if (overrun) {
- dprintk(NDEBUG_DMA, "Got an input overrun, using saved byte\n");
- *(*data)++ = saved_data;
- (*count)--;
- cnt--;
- toPIO--;
- }
- dprintk(NDEBUG_DMA, "Doing %d-byte PIO to 0x%08lx\n", cnt, (long)*data);
- NCR5380_transfer_pio(instance, &p, &cnt, data);
- *count -= toPIO - cnt;
- }
- }
-}
-#endif /* REAL_DMA */
-
-
-/**
- * NCR5380_intr - generic NCR5380 irq handler
- * @irq: interrupt number
- * @dev_id: device info
- *
- * Handle interrupts, reestablishing I_T_L or I_T_L_Q nexuses
- * from the disconnected queue, and restarting NCR5380_main()
- * as required.
- *
- * The chip can assert IRQ in any of six different conditions. The IRQ flag
- * is then cleared by reading the Reset Parity/Interrupt Register (RPIR).
- * Three of these six conditions are latched in the Bus and Status Register:
- * - End of DMA (cleared by ending DMA Mode)
- * - Parity error (cleared by reading RPIR)
- * - Loss of BSY (cleared by reading RPIR)
- * Two conditions have flag bits that are not latched:
- * - Bus phase mismatch (non-maskable in DMA Mode, cleared by ending DMA Mode)
- * - Bus reset (non-maskable)
- * The remaining condition has no flag bit at all:
- * - Selection/reselection
- *
- * Hence, establishing the cause(s) of any interrupt is partly guesswork.
- * In "The DP8490 and DP5380 Comparison Guide", National Semiconductor
- * claimed that "the design of the [DP8490] interrupt logic ensures
- * interrupts will not be lost (they can be on the DP5380)."
- * The L5380/53C80 datasheet from LOGIC Devices has more details.
- *
- * Checking for bus reset by reading RST is futile because of interrupt
- * latency, but a bus reset will reset chip logic. Checking for parity error
- * is unnecessary because that interrupt is never enabled. A Loss of BSY
- * condition will clear DMA Mode. We can tell when this occurs because the
- * the Busy Monitor interrupt is enabled together with DMA Mode.
- */
-
-static irqreturn_t NCR5380_intr(int irq, void *dev_id)
-{
- struct Scsi_Host *instance = dev_id;
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
- int handled = 0;
- unsigned char basr;
- unsigned long flags;
-
- spin_lock_irqsave(&hostdata->lock, flags);
-
- basr = NCR5380_read(BUS_AND_STATUS_REG);
- if (basr & BASR_IRQ) {
- unsigned char mr = NCR5380_read(MODE_REG);
- unsigned char sr = NCR5380_read(STATUS_REG);
-
- dsprintk(NDEBUG_INTR, instance, "IRQ %d, BASR 0x%02x, SR 0x%02x, MR 0x%02x\n",
- irq, basr, sr, mr);
-
-#if defined(REAL_DMA)
- if ((mr & MR_DMA_MODE) || (mr & MR_MONITOR_BSY)) {
- /* Probably End of DMA, Phase Mismatch or Loss of BSY.
- * We ack IRQ after clearing Mode Register. Workarounds
- * for End of DMA errata need to happen in DMA Mode.
- */
-
- dsprintk(NDEBUG_INTR, instance, "interrupt in DMA mode\n");
-
- if (hostdata->connected) {
- NCR5380_dma_complete(instance);
- queue_work(hostdata->work_q, &hostdata->main_task);
- } else {
- NCR5380_write(MODE_REG, MR_BASE);
- NCR5380_read(RESET_PARITY_INTERRUPT_REG);
- }
- } else
-#endif /* REAL_DMA */
- if ((NCR5380_read(CURRENT_SCSI_DATA_REG) & hostdata->id_mask) &&
- (sr & (SR_SEL | SR_IO | SR_BSY | SR_RST)) == (SR_SEL | SR_IO)) {
- /* Probably reselected */
- NCR5380_write(SELECT_ENABLE_REG, 0);
- NCR5380_read(RESET_PARITY_INTERRUPT_REG);
-
- dsprintk(NDEBUG_INTR, instance, "interrupt with SEL and IO\n");
-
- if (!hostdata->connected) {
- NCR5380_reselect(instance);
- queue_work(hostdata->work_q, &hostdata->main_task);
- }
- if (!hostdata->connected)
- NCR5380_write(SELECT_ENABLE_REG, hostdata->id_mask);
- } else {
- /* Probably Bus Reset */
- NCR5380_read(RESET_PARITY_INTERRUPT_REG);
-
- dsprintk(NDEBUG_INTR, instance, "unknown interrupt\n");
-#ifdef SUN3_SCSI_VME
- dregs->csr |= CSR_DMA_ENABLE;
-#endif
- }
- handled = 1;
- } else {
- shost_printk(KERN_NOTICE, instance, "interrupt without IRQ bit\n");
-#ifdef SUN3_SCSI_VME
- dregs->csr |= CSR_DMA_ENABLE;
-#endif
- }
-
- spin_unlock_irqrestore(&hostdata->lock, flags);
-
- return IRQ_RETVAL(handled);
-}
-
-/*
- * Function : int NCR5380_select(struct Scsi_Host *instance,
- * struct scsi_cmnd *cmd)
- *
- * Purpose : establishes I_T_L or I_T_L_Q nexus for new or existing command,
- * including ARBITRATION, SELECTION, and initial message out for
- * IDENTIFY and queue messages.
- *
- * Inputs : instance - instantiation of the 5380 driver on which this
- * target lives, cmd - SCSI command to execute.
- *
- * Returns cmd if selection failed but should be retried,
- * NULL if selection failed and should not be retried, or
- * NULL if selection succeeded (hostdata->connected == cmd).
- *
- * Side effects :
- * If bus busy, arbitration failed, etc, NCR5380_select() will exit
- * with registers as they should have been on entry - ie
- * SELECT_ENABLE will be set appropriately, the NCR5380
- * will cease to drive any SCSI bus signals.
- *
- * If successful : I_T_L or I_T_L_Q nexus will be established,
- * instance->connected will be set to cmd.
- * SELECT interrupt will be disabled.
- *
- * If failed (no target) : cmd->scsi_done() will be called, and the
- * cmd->result host byte set to DID_BAD_TARGET.
- */
-
-static struct scsi_cmnd *NCR5380_select(struct Scsi_Host *instance,
- struct scsi_cmnd *cmd)
-{
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
- unsigned char tmp[3], phase;
- unsigned char *data;
- int len;
- int err;
-
- NCR5380_dprint(NDEBUG_ARBITRATION, instance);
- dsprintk(NDEBUG_ARBITRATION, instance, "starting arbitration, id = %d\n",
- instance->this_id);
-
- /*
- * Arbitration and selection phases are slow and involve dropping the
- * lock, so we have to watch out for EH. An exception handler may
- * change 'selecting' to NULL. This function will then return NULL
- * so that the caller will forget about 'cmd'. (During information
- * transfer phases, EH may change 'connected' to NULL.)
- */
- hostdata->selecting = cmd;
-
- /*
- * Set the phase bits to 0, otherwise the NCR5380 won't drive the
- * data bus during SELECTION.
- */
-
- NCR5380_write(TARGET_COMMAND_REG, 0);
-
- /*
- * Start arbitration.
- */
-
- NCR5380_write(OUTPUT_DATA_REG, hostdata->id_mask);
- NCR5380_write(MODE_REG, MR_ARBITRATE);
-
- /* The chip now waits for BUS FREE phase. Then after the 800 ns
- * Bus Free Delay, arbitration will begin.
- */
-
- spin_unlock_irq(&hostdata->lock);
- err = NCR5380_poll_politely2(instance, MODE_REG, MR_ARBITRATE, 0,
- INITIATOR_COMMAND_REG, ICR_ARBITRATION_PROGRESS,
- ICR_ARBITRATION_PROGRESS, HZ);
- spin_lock_irq(&hostdata->lock);
- if (!(NCR5380_read(MODE_REG) & MR_ARBITRATE)) {
- /* Reselection interrupt */
- goto out;
- }
- if (!hostdata->selecting) {
- /* Command was aborted */
- NCR5380_write(MODE_REG, MR_BASE);
- goto out;
- }
- if (err < 0) {
- NCR5380_write(MODE_REG, MR_BASE);
- shost_printk(KERN_ERR, instance,
- "select: arbitration timeout\n");
- goto out;
- }
- spin_unlock_irq(&hostdata->lock);
-
- /* The SCSI-2 arbitration delay is 2.4 us */
- udelay(3);
-
- /* Check for lost arbitration */
- if ((NCR5380_read(INITIATOR_COMMAND_REG) & ICR_ARBITRATION_LOST) ||
- (NCR5380_read(CURRENT_SCSI_DATA_REG) & hostdata->id_higher_mask) ||
- (NCR5380_read(INITIATOR_COMMAND_REG) & ICR_ARBITRATION_LOST)) {
- NCR5380_write(MODE_REG, MR_BASE);
- dsprintk(NDEBUG_ARBITRATION, instance, "lost arbitration, deasserting MR_ARBITRATE\n");
- spin_lock_irq(&hostdata->lock);
- goto out;
- }
-
- /* After/during arbitration, BSY should be asserted.
- * IBM DPES-31080 Version S31Q works now
- * Tnx to Thomas_Roesch@m2.maus.de for finding this! (Roman)
- */
- NCR5380_write(INITIATOR_COMMAND_REG,
- ICR_BASE | ICR_ASSERT_SEL | ICR_ASSERT_BSY);
-
- /*
- * Again, bus clear + bus settle time is 1.2us, however, this is
- * a minimum so we'll udelay ceil(1.2)
- */
-
- if (hostdata->flags & FLAG_TOSHIBA_DELAY)
- udelay(15);
- else
- udelay(2);
-
- spin_lock_irq(&hostdata->lock);
-
- /* NCR5380_reselect() clears MODE_REG after a reselection interrupt */
- if (!(NCR5380_read(MODE_REG) & MR_ARBITRATE))
- goto out;
-
- if (!hostdata->selecting) {
- NCR5380_write(MODE_REG, MR_BASE);
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- goto out;
- }
-
- dsprintk(NDEBUG_ARBITRATION, instance, "won arbitration\n");
-
- /*
- * Now that we have won arbitration, start Selection process, asserting
- * the host and target ID's on the SCSI bus.
- */
-
- NCR5380_write(OUTPUT_DATA_REG, hostdata->id_mask | (1 << scmd_id(cmd)));
-
- /*
- * Raise ATN while SEL is true before BSY goes false from arbitration,
- * since this is the only way to guarantee that we'll get a MESSAGE OUT
- * phase immediately after selection.
- */
-
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE | ICR_ASSERT_BSY |
- ICR_ASSERT_DATA | ICR_ASSERT_ATN | ICR_ASSERT_SEL);
- NCR5380_write(MODE_REG, MR_BASE);
-
- /*
- * Reselect interrupts must be turned off prior to the dropping of BSY,
- * otherwise we will trigger an interrupt.
- */
- NCR5380_write(SELECT_ENABLE_REG, 0);
-
- spin_unlock_irq(&hostdata->lock);
-
- /*
- * The initiator shall then wait at least two deskew delays and release
- * the BSY signal.
- */
- udelay(1); /* wingel -- wait two bus deskew delay >2*45ns */
-
- /* Reset BSY */
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE | ICR_ASSERT_DATA |
- ICR_ASSERT_ATN | ICR_ASSERT_SEL);
-
- /*
- * Something weird happens when we cease to drive BSY - looks
- * like the board/chip is letting us do another read before the
- * appropriate propagation delay has expired, and we're confusing
- * a BSY signal from ourselves as the target's response to SELECTION.
- *
- * A small delay (the 'C++' frontend breaks the pipeline with an
- * unnecessary jump, making it work on my 386-33/Trantor T128, the
- * tighter 'C' code breaks and requires this) solves the problem -
- * the 1 us delay is arbitrary, and only used because this delay will
- * be the same on other platforms and since it works here, it should
- * work there.
- *
- * wingel suggests that this could be due to failing to wait
- * one deskew delay.
- */
-
- udelay(1);
-
- dsprintk(NDEBUG_SELECTION, instance, "selecting target %d\n", scmd_id(cmd));
-
- /*
- * The SCSI specification calls for a 250 ms timeout for the actual
- * selection.
- */
-
- err = NCR5380_poll_politely(instance, STATUS_REG, SR_BSY, SR_BSY,
- msecs_to_jiffies(250));
-
- if ((NCR5380_read(STATUS_REG) & (SR_SEL | SR_IO)) == (SR_SEL | SR_IO)) {
- spin_lock_irq(&hostdata->lock);
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- NCR5380_reselect(instance);
- if (!hostdata->connected)
- NCR5380_write(SELECT_ENABLE_REG, hostdata->id_mask);
- shost_printk(KERN_ERR, instance, "reselection after won arbitration?\n");
- goto out;
- }
-
- if (err < 0) {
- spin_lock_irq(&hostdata->lock);
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- NCR5380_write(SELECT_ENABLE_REG, hostdata->id_mask);
- /* Can't touch cmd if it has been reclaimed by the scsi ML */
- if (hostdata->selecting) {
- cmd->result = DID_BAD_TARGET << 16;
- complete_cmd(instance, cmd);
- dsprintk(NDEBUG_SELECTION, instance, "target did not respond within 250ms\n");
- cmd = NULL;
- }
- goto out;
- }
-
- /*
- * No less than two deskew delays after the initiator detects the
- * BSY signal is true, it shall release the SEL signal and may
- * change the DATA BUS. -wingel
- */
-
- udelay(1);
-
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE | ICR_ASSERT_ATN);
-
- /*
- * Since we followed the SCSI spec, and raised ATN while SEL
- * was true but before BSY was false during selection, the information
- * transfer phase should be a MESSAGE OUT phase so that we can send the
- * IDENTIFY message.
- *
- * If SCSI-II tagged queuing is enabled, we also send a SIMPLE_QUEUE_TAG
- * message (2 bytes) with a tag ID that we increment with every command
- * until it wraps back to 0.
- *
- * XXX - it turns out that there are some broken SCSI-II devices,
- * which claim to support tagged queuing but fail when more than
- * some number of commands are issued at once.
- */
-
- /* Wait for start of REQ/ACK handshake */
-
- err = NCR5380_poll_politely(instance, STATUS_REG, SR_REQ, SR_REQ, HZ);
- spin_lock_irq(&hostdata->lock);
- if (err < 0) {
- shost_printk(KERN_ERR, instance, "select: REQ timeout\n");
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- NCR5380_write(SELECT_ENABLE_REG, hostdata->id_mask);
- goto out;
- }
- if (!hostdata->selecting) {
- do_abort(instance);
- goto out;
- }
-
- dsprintk(NDEBUG_SELECTION, instance, "target %d selected, going into MESSAGE OUT phase.\n",
- scmd_id(cmd));
- tmp[0] = IDENTIFY(1, cmd->device->lun);
-
-#ifdef SUPPORT_TAGS
- if (cmd->tag != TAG_NONE) {
- tmp[1] = hostdata->last_message = SIMPLE_QUEUE_TAG;
- tmp[2] = cmd->tag;
- len = 3;
- } else
- len = 1;
-#else
- len = 1;
- cmd->tag = 0;
-#endif /* SUPPORT_TAGS */
-
- /* Send message(s) */
- data = tmp;
- phase = PHASE_MSGOUT;
- NCR5380_transfer_pio(instance, &phase, &len, &data);
- dsprintk(NDEBUG_SELECTION, instance, "nexus established.\n");
- /* XXX need to handle errors here */
-
- hostdata->connected = cmd;
-#ifndef SUPPORT_TAGS
- hostdata->busy[cmd->device->id] |= 1 << cmd->device->lun;
-#endif
-#ifdef SUN3_SCSI_VME
- dregs->csr |= CSR_INTR;
-#endif
-
- initialize_SCp(cmd);
-
- cmd = NULL;
-
-out:
- if (!hostdata->selecting)
- return NULL;
- hostdata->selecting = NULL;
- return cmd;
-}
-
-/*
- * Function : int NCR5380_transfer_pio (struct Scsi_Host *instance,
- * unsigned char *phase, int *count, unsigned char **data)
- *
- * Purpose : transfers data in given phase using polled I/O
- *
- * Inputs : instance - instance of driver, *phase - pointer to
- * what phase is expected, *count - pointer to number of
- * bytes to transfer, **data - pointer to data pointer.
- *
- * Returns : -1 when different phase is entered without transferring
- * maximum number of bytes, 0 if all bytes are transferred or exit
- * is in same phase.
- *
- * Also, *phase, *count, *data are modified in place.
- *
- * XXX Note : handling for bus free may be useful.
- */
-
-/*
- * Note : this code is not as quick as it could be, however it
- * IS 100% reliable, and for the actual data transfer where speed
- * counts, we will always do a pseudo DMA or DMA transfer.
- */
-
-static int NCR5380_transfer_pio(struct Scsi_Host *instance,
- unsigned char *phase, int *count,
- unsigned char **data)
-{
- unsigned char p = *phase, tmp;
- int c = *count;
- unsigned char *d = *data;
-
- /*
- * The NCR5380 chip will only drive the SCSI bus when the
- * phase specified in the appropriate bits of the TARGET COMMAND
- * REGISTER match the STATUS REGISTER
- */
-
- NCR5380_write(TARGET_COMMAND_REG, PHASE_SR_TO_TCR(p));
-
- do {
- /*
- * Wait for assertion of REQ, after which the phase bits will be
- * valid
- */
-
- if (NCR5380_poll_politely(instance, STATUS_REG, SR_REQ, SR_REQ, HZ) < 0)
- break;
-
- dsprintk(NDEBUG_HANDSHAKE, instance, "REQ asserted\n");
-
- /* Check for phase mismatch */
- if ((NCR5380_read(STATUS_REG) & PHASE_MASK) != p) {
- dsprintk(NDEBUG_PIO, instance, "phase mismatch\n");
- NCR5380_dprint_phase(NDEBUG_PIO, instance);
- break;
- }
-
- /* Do actual transfer from SCSI bus to / from memory */
- if (!(p & SR_IO))
- NCR5380_write(OUTPUT_DATA_REG, *d);
- else
- *d = NCR5380_read(CURRENT_SCSI_DATA_REG);
-
- ++d;
-
- /*
- * The SCSI standard suggests that in MSGOUT phase, the initiator
- * should drop ATN on the last byte of the message phase
- * after REQ has been asserted for the handshake but before
- * the initiator raises ACK.
- */
-
- if (!(p & SR_IO)) {
- if (!((p & SR_MSG) && c > 1)) {
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE | ICR_ASSERT_DATA);
- NCR5380_dprint(NDEBUG_PIO, instance);
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE |
- ICR_ASSERT_DATA | ICR_ASSERT_ACK);
- } else {
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE |
- ICR_ASSERT_DATA | ICR_ASSERT_ATN);
- NCR5380_dprint(NDEBUG_PIO, instance);
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE |
- ICR_ASSERT_DATA | ICR_ASSERT_ATN | ICR_ASSERT_ACK);
- }
- } else {
- NCR5380_dprint(NDEBUG_PIO, instance);
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE | ICR_ASSERT_ACK);
- }
-
- if (NCR5380_poll_politely(instance,
- STATUS_REG, SR_REQ, 0, 5 * HZ) < 0)
- break;
-
- dsprintk(NDEBUG_HANDSHAKE, instance, "REQ negated, handshake complete\n");
-
-/*
- * We have several special cases to consider during REQ/ACK handshaking :
- * 1. We were in MSGOUT phase, and we are on the last byte of the
- * message. ATN must be dropped as ACK is dropped.
- *
- * 2. We are in a MSGIN phase, and we are on the last byte of the
- * message. We must exit with ACK asserted, so that the calling
- * code may raise ATN before dropping ACK to reject the message.
- *
- * 3. ACK and ATN are clear and the target may proceed as normal.
- */
- if (!(p == PHASE_MSGIN && c == 1)) {
- if (p == PHASE_MSGOUT && c > 1)
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE | ICR_ASSERT_ATN);
- else
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- }
- } while (--c);
-
- dsprintk(NDEBUG_PIO, instance, "residual %d\n", c);
-
- *count = c;
- *data = d;
- tmp = NCR5380_read(STATUS_REG);
- /* The phase read from the bus is valid if either REQ is (already)
- * asserted or if ACK hasn't been released yet. The latter applies if
- * we're in MSG IN, DATA IN or STATUS and all bytes have been received.
- */
- if ((tmp & SR_REQ) || ((tmp & SR_IO) && c == 0))
- *phase = tmp & PHASE_MASK;
- else
- *phase = PHASE_UNKNOWN;
-
- if (!c || (*phase == p))
- return 0;
- else
- return -1;
-}
-
-/**
- * do_reset - issue a reset command
- * @instance: adapter to reset
- *
- * Issue a reset sequence to the NCR5380 and try and get the bus
- * back into sane shape.
- *
- * This clears the reset interrupt flag because there may be no handler for
- * it. When the driver is initialized, the NCR5380_intr() handler has not yet
- * been installed. And when in EH we may have released the ST DMA interrupt.
- */
-
-static void do_reset(struct Scsi_Host *instance)
-{
- unsigned long flags;
-
- local_irq_save(flags);
- NCR5380_write(TARGET_COMMAND_REG,
- PHASE_SR_TO_TCR(NCR5380_read(STATUS_REG) & PHASE_MASK));
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE | ICR_ASSERT_RST);
- udelay(50);
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- (void)NCR5380_read(RESET_PARITY_INTERRUPT_REG);
- local_irq_restore(flags);
-}
-
-/**
- * do_abort - abort the currently established nexus by going to
- * MESSAGE OUT phase and sending an ABORT message.
- * @instance: relevant scsi host instance
- *
- * Returns 0 on success, -1 on failure.
- */
-
-static int do_abort(struct Scsi_Host *instance)
-{
- unsigned char *msgptr, phase, tmp;
- int len;
- int rc;
-
- /* Request message out phase */
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE | ICR_ASSERT_ATN);
-
- /*
- * Wait for the target to indicate a valid phase by asserting
- * REQ. Once this happens, we'll have either a MSGOUT phase
- * and can immediately send the ABORT message, or we'll have some
- * other phase and will have to source/sink data.
- *
- * We really don't care what value was on the bus or what value
- * the target sees, so we just handshake.
- */
-
- rc = NCR5380_poll_politely(instance, STATUS_REG, SR_REQ, SR_REQ, 10 * HZ);
- if (rc < 0)
- goto timeout;
-
- tmp = NCR5380_read(STATUS_REG) & PHASE_MASK;
-
- NCR5380_write(TARGET_COMMAND_REG, PHASE_SR_TO_TCR(tmp));
-
- if (tmp != PHASE_MSGOUT) {
- NCR5380_write(INITIATOR_COMMAND_REG,
- ICR_BASE | ICR_ASSERT_ATN | ICR_ASSERT_ACK);
- rc = NCR5380_poll_politely(instance, STATUS_REG, SR_REQ, 0, 3 * HZ);
- if (rc < 0)
- goto timeout;
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE | ICR_ASSERT_ATN);
- }
-
- tmp = ABORT;
- msgptr = &tmp;
- len = 1;
- phase = PHASE_MSGOUT;
- NCR5380_transfer_pio(instance, &phase, &len, &msgptr);
-
- /*
- * If we got here, and the command completed successfully,
- * we're about to go into bus free state.
- */
-
- return len ? -1 : 0;
-
-timeout:
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- return -1;
-}
-
-#if defined(REAL_DMA)
-/*
- * Function : int NCR5380_transfer_dma (struct Scsi_Host *instance,
- * unsigned char *phase, int *count, unsigned char **data)
- *
- * Purpose : transfers data in given phase using either real
- * or pseudo DMA.
- *
- * Inputs : instance - instance of driver, *phase - pointer to
- * what phase is expected, *count - pointer to number of
- * bytes to transfer, **data - pointer to data pointer.
- *
- * Returns : -1 when different phase is entered without transferring
- * maximum number of bytes, 0 if all bytes or transferred or exit
- * is in same phase.
- *
- * Also, *phase, *count, *data are modified in place.
- */
-
-
-static int NCR5380_transfer_dma(struct Scsi_Host *instance,
- unsigned char *phase, int *count,
- unsigned char **data)
-{
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
- register int c = *count;
- register unsigned char p = *phase;
-
-#if defined(CONFIG_SUN3)
- /* sanity check */
- if (!sun3_dma_setup_done) {
- pr_err("scsi%d: transfer_dma without setup!\n",
- instance->host_no);
- BUG();
- }
- hostdata->dma_len = c;
-
- dsprintk(NDEBUG_DMA, instance, "initializing DMA %s: length %d, address %p\n",
- (p & SR_IO) ? "receive" : "send", c, *data);
-
- /* netbsd turns off ints here, why not be safe and do it too */
-
- /* send start chain */
- sun3scsi_dma_start(c, *data);
-
- NCR5380_write(TARGET_COMMAND_REG, PHASE_SR_TO_TCR(p));
- NCR5380_write(MODE_REG, MR_BASE | MR_DMA_MODE | MR_MONITOR_BSY |
- MR_ENABLE_EOP_INTR);
- if (p & SR_IO) {
- NCR5380_write(INITIATOR_COMMAND_REG, 0);
- NCR5380_write(START_DMA_INITIATOR_RECEIVE_REG, 0);
- } else {
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_ASSERT_DATA);
- NCR5380_write(START_DMA_SEND_REG, 0);
- }
-
-#ifdef SUN3_SCSI_VME
- dregs->csr |= CSR_DMA_ENABLE;
-#endif
-
- sun3_dma_active = 1;
-
-#else /* !defined(CONFIG_SUN3) */
- register unsigned char *d = *data;
- unsigned char tmp;
-
- if ((tmp = (NCR5380_read(STATUS_REG) & PHASE_MASK)) != p) {
- *phase = tmp;
- return -1;
- }
-
- if (hostdata->read_overruns && (p & SR_IO))
- c -= hostdata->read_overruns;
-
- dsprintk(NDEBUG_DMA, instance, "initializing DMA %s: length %d, address %p\n",
- (p & SR_IO) ? "receive" : "send", c, d);
-
- NCR5380_write(TARGET_COMMAND_REG, PHASE_SR_TO_TCR(p));
- NCR5380_write(MODE_REG, MR_BASE | MR_DMA_MODE | MR_MONITOR_BSY |
- MR_ENABLE_EOP_INTR);
-
- if (!(hostdata->flags & FLAG_LATE_DMA_SETUP)) {
- /* On the Medusa, it is a must to initialize the DMA before
- * starting the NCR. This is also the cleaner way for the TT.
- */
- hostdata->dma_len = (p & SR_IO) ?
- NCR5380_dma_read_setup(instance, d, c) :
- NCR5380_dma_write_setup(instance, d, c);
- }
-
- if (p & SR_IO)
- NCR5380_write(START_DMA_INITIATOR_RECEIVE_REG, 0);
- else {
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE | ICR_ASSERT_DATA);
- NCR5380_write(START_DMA_SEND_REG, 0);
- }
-
- if (hostdata->flags & FLAG_LATE_DMA_SETUP) {
- /* On the Falcon, the DMA setup must be done after the last */
- /* NCR access, else the DMA setup gets trashed!
- */
- hostdata->dma_len = (p & SR_IO) ?
- NCR5380_dma_read_setup(instance, d, c) :
- NCR5380_dma_write_setup(instance, d, c);
- }
-#endif /* !defined(CONFIG_SUN3) */
-
- return 0;
-}
-#endif /* defined(REAL_DMA) */
-
-/*
- * Function : NCR5380_information_transfer (struct Scsi_Host *instance)
- *
- * Purpose : run through the various SCSI phases and do as the target
- * directs us to. Operates on the currently connected command,
- * instance->connected.
- *
- * Inputs : instance, instance for which we are doing commands
- *
- * Side effects : SCSI things happen, the disconnected queue will be
- * modified if a command disconnects, *instance->connected will
- * change.
- *
- * XXX Note : we need to watch for bus free or a reset condition here
- * to recover from an unexpected bus free condition.
- */
-
-static void NCR5380_information_transfer(struct Scsi_Host *instance)
-{
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
- unsigned char msgout = NOP;
- int sink = 0;
- int len;
- int transfersize;
- unsigned char *data;
- unsigned char phase, tmp, extended_msg[10], old_phase = 0xff;
- struct scsi_cmnd *cmd;
-
-#ifdef SUN3_SCSI_VME
- dregs->csr |= CSR_INTR;
-#endif
-
- while ((cmd = hostdata->connected)) {
- struct NCR5380_cmd *ncmd = scsi_cmd_priv(cmd);
-
- tmp = NCR5380_read(STATUS_REG);
- /* We only have a valid SCSI phase when REQ is asserted */
- if (tmp & SR_REQ) {
- phase = (tmp & PHASE_MASK);
- if (phase != old_phase) {
- old_phase = phase;
- NCR5380_dprint_phase(NDEBUG_INFORMATION, instance);
- }
-#if defined(CONFIG_SUN3)
- if (phase == PHASE_CMDOUT) {
-#if defined(REAL_DMA)
- void *d;
- unsigned long count;
-
- if (!cmd->SCp.this_residual && cmd->SCp.buffers_residual) {
- count = cmd->SCp.buffer->length;
- d = sg_virt(cmd->SCp.buffer);
- } else {
- count = cmd->SCp.this_residual;
- d = cmd->SCp.ptr;
- }
- /* this command setup for dma yet? */
- if ((count >= DMA_MIN_SIZE) && (sun3_dma_setup_done != cmd)) {
- if (cmd->request->cmd_type == REQ_TYPE_FS) {
- sun3scsi_dma_setup(instance, d, count,
- rq_data_dir(cmd->request));
- sun3_dma_setup_done = cmd;
- }
- }
-#endif
-#ifdef SUN3_SCSI_VME
- dregs->csr |= CSR_INTR;
-#endif
- }
-#endif /* CONFIG_SUN3 */
-
- if (sink && (phase != PHASE_MSGOUT)) {
- NCR5380_write(TARGET_COMMAND_REG, PHASE_SR_TO_TCR(tmp));
-
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE | ICR_ASSERT_ATN |
- ICR_ASSERT_ACK);
- while (NCR5380_read(STATUS_REG) & SR_REQ)
- ;
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE |
- ICR_ASSERT_ATN);
- sink = 0;
- continue;
- }
-
- switch (phase) {
- case PHASE_DATAOUT:
-#if (NDEBUG & NDEBUG_NO_DATAOUT)
- shost_printk(KERN_DEBUG, instance, "NDEBUG_NO_DATAOUT set, attempted DATAOUT aborted\n");
- sink = 1;
- do_abort(instance);
- cmd->result = DID_ERROR << 16;
- complete_cmd(instance, cmd);
- hostdata->connected = NULL;
- return;
-#endif
- case PHASE_DATAIN:
- /*
- * If there is no room left in the current buffer in the
- * scatter-gather list, move onto the next one.
- */
-
- if (!cmd->SCp.this_residual && cmd->SCp.buffers_residual) {
- ++cmd->SCp.buffer;
- --cmd->SCp.buffers_residual;
- cmd->SCp.this_residual = cmd->SCp.buffer->length;
- cmd->SCp.ptr = sg_virt(cmd->SCp.buffer);
- merge_contiguous_buffers(cmd);
- dsprintk(NDEBUG_INFORMATION, instance, "%d bytes and %d buffers left\n",
- cmd->SCp.this_residual,
- cmd->SCp.buffers_residual);
- }
-
- /*
- * The preferred transfer method is going to be
- * PSEUDO-DMA for systems that are strictly PIO,
- * since we can let the hardware do the handshaking.
- *
- * For this to work, we need to know the transfersize
- * ahead of time, since the pseudo-DMA code will sit
- * in an unconditional loop.
- */
-
- /* ++roman: I suggest, this should be
- * #if def(REAL_DMA)
- * instead of leaving REAL_DMA out.
- */
-
-#if defined(REAL_DMA)
-#if !defined(CONFIG_SUN3)
- transfersize = 0;
- if (!cmd->device->borken)
-#endif
- transfersize = NCR5380_dma_xfer_len(instance, cmd, phase);
-
- if (transfersize >= DMA_MIN_SIZE) {
- len = transfersize;
- cmd->SCp.phase = phase;
- if (NCR5380_transfer_dma(instance, &phase,
- &len, (unsigned char **)&cmd->SCp.ptr)) {
- /*
- * If the watchdog timer fires, all future
- * accesses to this device will use the
- * polled-IO.
- */
- scmd_printk(KERN_INFO, cmd,
- "switching to slow handshake\n");
- cmd->device->borken = 1;
- sink = 1;
- do_abort(instance);
- cmd->result = DID_ERROR << 16;
- /* XXX - need to source or sink data here, as appropriate */
- } else {
-#ifdef REAL_DMA
- /* ++roman: When using real DMA,
- * information_transfer() should return after
- * starting DMA since it has nothing more to
- * do.
- */
- return;
-#else
- cmd->SCp.this_residual -= transfersize - len;
-#endif
- }
- } else
-#endif /* defined(REAL_DMA) */
- {
- /* Break up transfer into 3 ms chunks,
- * presuming 6 accesses per handshake.
- */
- transfersize = min((unsigned long)cmd->SCp.this_residual,
- hostdata->accesses_per_ms / 2);
- len = transfersize;
- NCR5380_transfer_pio(instance, &phase, &len,
- (unsigned char **)&cmd->SCp.ptr);
- cmd->SCp.this_residual -= transfersize - len;
- }
-#if defined(CONFIG_SUN3) && defined(REAL_DMA)
- /* if we had intended to dma that command clear it */
- if (sun3_dma_setup_done == cmd)
- sun3_dma_setup_done = NULL;
-#endif
- return;
- case PHASE_MSGIN:
- len = 1;
- data = &tmp;
- NCR5380_transfer_pio(instance, &phase, &len, &data);
- cmd->SCp.Message = tmp;
-
- switch (tmp) {
- case ABORT:
- case COMMAND_COMPLETE:
- /* Accept message by clearing ACK */
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- dsprintk(NDEBUG_QUEUES, instance,
- "COMMAND COMPLETE %p target %d lun %llu\n",
- cmd, scmd_id(cmd), cmd->device->lun);
-
- hostdata->connected = NULL;
-#ifdef SUPPORT_TAGS
- cmd_free_tag(cmd);
- if (status_byte(cmd->SCp.Status) == QUEUE_FULL) {
- u8 lun = cmd->device->lun;
- struct tag_alloc *ta = &hostdata->TagAlloc[scmd_id(cmd)][lun];
-
- dsprintk(NDEBUG_TAGS, instance,
- "QUEUE_FULL %p target %d lun %d nr_allocated %d\n",
- cmd, scmd_id(cmd), lun, ta->nr_allocated);
- if (ta->queue_size > ta->nr_allocated)
- ta->queue_size = ta->nr_allocated;
- }
-#endif
-
- cmd->result &= ~0xffff;
- cmd->result |= cmd->SCp.Status;
- cmd->result |= cmd->SCp.Message << 8;
-
- if (cmd->cmnd[0] == REQUEST_SENSE)
- complete_cmd(instance, cmd);
- else {
- if (cmd->SCp.Status == SAM_STAT_CHECK_CONDITION ||
- cmd->SCp.Status == SAM_STAT_COMMAND_TERMINATED) {
- dsprintk(NDEBUG_QUEUES, instance, "autosense: adding cmd %p to tail of autosense queue\n",
- cmd);
- list_add_tail(&ncmd->list,
- &hostdata->autosense);
- } else
- complete_cmd(instance, cmd);
- }
-
- /*
- * Restore phase bits to 0 so an interrupted selection,
- * arbitration can resume.
- */
- NCR5380_write(TARGET_COMMAND_REG, 0);
-
- /* Enable reselect interrupts */
- NCR5380_write(SELECT_ENABLE_REG, hostdata->id_mask);
-
- maybe_release_dma_irq(instance);
- return;
- case MESSAGE_REJECT:
- /* Accept message by clearing ACK */
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- switch (hostdata->last_message) {
- case HEAD_OF_QUEUE_TAG:
- case ORDERED_QUEUE_TAG:
- case SIMPLE_QUEUE_TAG:
- /* The target obviously doesn't support tagged
- * queuing, even though it announced this ability in
- * its INQUIRY data ?!? (maybe only this LUN?) Ok,
- * clear 'tagged_supported' and lock the LUN, since
- * the command is treated as untagged further on.
- */
- cmd->device->tagged_supported = 0;
- hostdata->busy[cmd->device->id] |= (1 << cmd->device->lun);
- cmd->tag = TAG_NONE;
- dsprintk(NDEBUG_TAGS, instance, "target %d lun %llu rejected QUEUE_TAG message; tagged queuing disabled\n",
- scmd_id(cmd), cmd->device->lun);
- break;
- }
- break;
- case DISCONNECT:
- /* Accept message by clearing ACK */
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- hostdata->connected = NULL;
- list_add(&ncmd->list, &hostdata->disconnected);
- dsprintk(NDEBUG_INFORMATION | NDEBUG_QUEUES,
- instance, "connected command %p for target %d lun %llu moved to disconnected queue\n",
- cmd, scmd_id(cmd), cmd->device->lun);
-
- /*
- * Restore phase bits to 0 so an interrupted selection,
- * arbitration can resume.
- */
- NCR5380_write(TARGET_COMMAND_REG, 0);
-
- /* Enable reselect interrupts */
- NCR5380_write(SELECT_ENABLE_REG, hostdata->id_mask);
-#ifdef SUN3_SCSI_VME
- dregs->csr |= CSR_DMA_ENABLE;
-#endif
- return;
- /*
- * The SCSI data pointer is *IMPLICITLY* saved on a disconnect
- * operation, in violation of the SCSI spec so we can safely
- * ignore SAVE/RESTORE pointers calls.
- *
- * Unfortunately, some disks violate the SCSI spec and
- * don't issue the required SAVE_POINTERS message before
- * disconnecting, and we have to break spec to remain
- * compatible.
- */
- case SAVE_POINTERS:
- case RESTORE_POINTERS:
- /* Accept message by clearing ACK */
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- break;
- case EXTENDED_MESSAGE:
- /*
- * Start the message buffer with the EXTENDED_MESSAGE
- * byte, since spi_print_msg() wants the whole thing.
- */
- extended_msg[0] = EXTENDED_MESSAGE;
- /* Accept first byte by clearing ACK */
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
-
- spin_unlock_irq(&hostdata->lock);
-
- dsprintk(NDEBUG_EXTENDED, instance, "receiving extended message\n");
-
- len = 2;
- data = extended_msg + 1;
- phase = PHASE_MSGIN;
- NCR5380_transfer_pio(instance, &phase, &len, &data);
- dsprintk(NDEBUG_EXTENDED, instance, "length %d, code 0x%02x\n",
- (int)extended_msg[1],
- (int)extended_msg[2]);
-
- if (!len && extended_msg[1] > 0 &&
- extended_msg[1] <= sizeof(extended_msg) - 2) {
- /* Accept third byte by clearing ACK */
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- len = extended_msg[1] - 1;
- data = extended_msg + 3;
- phase = PHASE_MSGIN;
-
- NCR5380_transfer_pio(instance, &phase, &len, &data);
- dsprintk(NDEBUG_EXTENDED, instance, "message received, residual %d\n",
- len);
-
- switch (extended_msg[2]) {
- case EXTENDED_SDTR:
- case EXTENDED_WDTR:
- case EXTENDED_MODIFY_DATA_POINTER:
- case EXTENDED_EXTENDED_IDENTIFY:
- tmp = 0;
- }
- } else if (len) {
- shost_printk(KERN_ERR, instance, "error receiving extended message\n");
- tmp = 0;
- } else {
- shost_printk(KERN_NOTICE, instance, "extended message code %02x length %d is too long\n",
- extended_msg[2], extended_msg[1]);
- tmp = 0;
- }
-
- spin_lock_irq(&hostdata->lock);
- if (!hostdata->connected)
- return;
-
- /* Fall through to reject message */
-
- /*
- * If we get something weird that we aren't expecting,
- * reject it.
- */
- default:
- if (!tmp) {
- shost_printk(KERN_ERR, instance, "rejecting message ");
- spi_print_msg(extended_msg);
- printk("\n");
- } else if (tmp != EXTENDED_MESSAGE)
- scmd_printk(KERN_INFO, cmd,
- "rejecting unknown message %02x\n",
- tmp);
- else
- scmd_printk(KERN_INFO, cmd,
- "rejecting unknown extended message code %02x, length %d\n",
- extended_msg[1], extended_msg[0]);
-
- msgout = MESSAGE_REJECT;
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE | ICR_ASSERT_ATN);
- break;
- } /* switch (tmp) */
- break;
- case PHASE_MSGOUT:
- len = 1;
- data = &msgout;
- hostdata->last_message = msgout;
- NCR5380_transfer_pio(instance, &phase, &len, &data);
- if (msgout == ABORT) {
- hostdata->connected = NULL;
- cmd->result = DID_ERROR << 16;
- complete_cmd(instance, cmd);
- maybe_release_dma_irq(instance);
- NCR5380_write(SELECT_ENABLE_REG, hostdata->id_mask);
- return;
- }
- msgout = NOP;
- break;
- case PHASE_CMDOUT:
- len = cmd->cmd_len;
- data = cmd->cmnd;
- /*
- * XXX for performance reasons, on machines with a
- * PSEUDO-DMA architecture we should probably
- * use the dma transfer function.
- */
- NCR5380_transfer_pio(instance, &phase, &len, &data);
- break;
- case PHASE_STATIN:
- len = 1;
- data = &tmp;
- NCR5380_transfer_pio(instance, &phase, &len, &data);
- cmd->SCp.Status = tmp;
- break;
- default:
- shost_printk(KERN_ERR, instance, "unknown phase\n");
- NCR5380_dprint(NDEBUG_ANY, instance);
- } /* switch(phase) */
- } else {
- spin_unlock_irq(&hostdata->lock);
- NCR5380_poll_politely(instance, STATUS_REG, SR_REQ, SR_REQ, HZ);
- spin_lock_irq(&hostdata->lock);
- }
- }
-}
-
-/*
- * Function : void NCR5380_reselect (struct Scsi_Host *instance)
- *
- * Purpose : does reselection, initializing the instance->connected
- * field to point to the scsi_cmnd for which the I_T_L or I_T_L_Q
- * nexus has been reestablished,
- *
- * Inputs : instance - this instance of the NCR5380.
- */
-
-
-/* it might eventually prove necessary to do a dma setup on
- reselection, but it doesn't seem to be needed now -- sam */
-
-static void NCR5380_reselect(struct Scsi_Host *instance)
-{
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
- unsigned char target_mask;
- unsigned char lun;
-#ifdef SUPPORT_TAGS
- unsigned char tag;
-#endif
- unsigned char msg[3];
- int __maybe_unused len;
- unsigned char __maybe_unused *data, __maybe_unused phase;
- struct NCR5380_cmd *ncmd;
- struct scsi_cmnd *tmp;
-
- /*
- * Disable arbitration, etc. since the host adapter obviously
- * lost, and tell an interrupted NCR5380_select() to restart.
- */
-
- NCR5380_write(MODE_REG, MR_BASE);
-
- target_mask = NCR5380_read(CURRENT_SCSI_DATA_REG) & ~(hostdata->id_mask);
-
- dsprintk(NDEBUG_RESELECTION, instance, "reselect\n");
-
- /*
- * At this point, we have detected that our SCSI ID is on the bus,
- * SEL is true and BSY was false for at least one bus settle delay
- * (400 ns).
- *
- * We must assert BSY ourselves, until the target drops the SEL
- * signal.
- */
-
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE | ICR_ASSERT_BSY);
- if (NCR5380_poll_politely(instance,
- STATUS_REG, SR_SEL, 0, 2 * HZ) < 0) {
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- return;
- }
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
-
- /*
- * Wait for target to go into MSGIN.
- */
-
- if (NCR5380_poll_politely(instance,
- STATUS_REG, SR_REQ, SR_REQ, 2 * HZ) < 0) {
- do_abort(instance);
- return;
- }
-
-#if defined(CONFIG_SUN3) && defined(REAL_DMA)
- /* acknowledge toggle to MSGIN */
- NCR5380_write(TARGET_COMMAND_REG, PHASE_SR_TO_TCR(PHASE_MSGIN));
-
- /* peek at the byte without really hitting the bus */
- msg[0] = NCR5380_read(CURRENT_SCSI_DATA_REG);
-#else
- len = 1;
- data = msg;
- phase = PHASE_MSGIN;
- NCR5380_transfer_pio(instance, &phase, &len, &data);
-
- if (len) {
- do_abort(instance);
- return;
- }
-#endif
-
- if (!(msg[0] & 0x80)) {
- shost_printk(KERN_ERR, instance, "expecting IDENTIFY message, got ");
- spi_print_msg(msg);
- printk("\n");
- do_abort(instance);
- return;
- }
- lun = msg[0] & 0x07;
-
-#if defined(SUPPORT_TAGS) && !defined(CONFIG_SUN3)
- /* If the phase is still MSGIN, the target wants to send some more
- * messages. In case it supports tagged queuing, this is probably a
- * SIMPLE_QUEUE_TAG for the I_T_L_Q nexus.
- */
- tag = TAG_NONE;
- if (phase == PHASE_MSGIN && (hostdata->flags & FLAG_TAGGED_QUEUING)) {
- /* Accept previous IDENTIFY message by clearing ACK */
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- len = 2;
- data = msg + 1;
- if (!NCR5380_transfer_pio(instance, &phase, &len, &data) &&
- msg[1] == SIMPLE_QUEUE_TAG)
- tag = msg[2];
- dsprintk(NDEBUG_TAGS, instance, "reselect: target mask %02x, lun %d sent tag %d\n",
- target_mask, lun, tag);
- }
-#endif
-
- /*
- * Find the command corresponding to the I_T_L or I_T_L_Q nexus we
- * just reestablished, and remove it from the disconnected queue.
- */
-
- tmp = NULL;
- list_for_each_entry(ncmd, &hostdata->disconnected, list) {
- struct scsi_cmnd *cmd = NCR5380_to_scmd(ncmd);
-
- if (target_mask == (1 << scmd_id(cmd)) &&
- lun == (u8)cmd->device->lun
-#ifdef SUPPORT_TAGS
- && (tag == cmd->tag)
-#endif
- ) {
- list_del(&ncmd->list);
- tmp = cmd;
- break;
- }
- }
-
- if (tmp) {
- dsprintk(NDEBUG_RESELECTION | NDEBUG_QUEUES, instance,
- "reselect: removed %p from disconnected queue\n", tmp);
- } else {
-
-#ifdef SUPPORT_TAGS
- shost_printk(KERN_ERR, instance, "target bitmask 0x%02x lun %d tag %d not in disconnected queue.\n",
- target_mask, lun, tag);
-#else
- shost_printk(KERN_ERR, instance, "target bitmask 0x%02x lun %d not in disconnected queue.\n",
- target_mask, lun);
-#endif
- /*
- * Since we have an established nexus that we can't do anything
- * with, we must abort it.
- */
- do_abort(instance);
- return;
- }
-
-#if defined(CONFIG_SUN3) && defined(REAL_DMA)
- /* engage dma setup for the command we just saw */
- {
- void *d;
- unsigned long count;
-
- if (!tmp->SCp.this_residual && tmp->SCp.buffers_residual) {
- count = tmp->SCp.buffer->length;
- d = sg_virt(tmp->SCp.buffer);
- } else {
- count = tmp->SCp.this_residual;
- d = tmp->SCp.ptr;
- }
- /* setup this command for dma if not already */
- if ((count >= DMA_MIN_SIZE) && (sun3_dma_setup_done != tmp)) {
- sun3scsi_dma_setup(instance, d, count,
- rq_data_dir(tmp->request));
- sun3_dma_setup_done = tmp;
- }
- }
-
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE | ICR_ASSERT_ACK);
-#endif
-
- /* Accept message by clearing ACK */
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
-
-#if defined(SUPPORT_TAGS) && defined(CONFIG_SUN3)
- /* If the phase is still MSGIN, the target wants to send some more
- * messages. In case it supports tagged queuing, this is probably a
- * SIMPLE_QUEUE_TAG for the I_T_L_Q nexus.
- */
- tag = TAG_NONE;
- if (phase == PHASE_MSGIN && setup_use_tagged_queuing) {
- /* Accept previous IDENTIFY message by clearing ACK */
- NCR5380_write(INITIATOR_COMMAND_REG, ICR_BASE);
- len = 2;
- data = msg + 1;
- if (!NCR5380_transfer_pio(instance, &phase, &len, &data) &&
- msg[1] == SIMPLE_QUEUE_TAG)
- tag = msg[2];
- dsprintk(NDEBUG_TAGS, instance, "reselect: target mask %02x, lun %d sent tag %d\n"
- target_mask, lun, tag);
- }
-#endif
-
- hostdata->connected = tmp;
- dsprintk(NDEBUG_RESELECTION, instance, "nexus established, target %d, lun %llu, tag %d\n",
- scmd_id(tmp), tmp->device->lun, tmp->tag);
-}
-
-
-/**
- * list_find_cmd - test for presence of a command in a linked list
- * @haystack: list of commands
- * @needle: command to search for
- */
-
-static bool list_find_cmd(struct list_head *haystack,
- struct scsi_cmnd *needle)
-{
- struct NCR5380_cmd *ncmd;
-
- list_for_each_entry(ncmd, haystack, list)
- if (NCR5380_to_scmd(ncmd) == needle)
- return true;
- return false;
-}
-
-/**
- * list_remove_cmd - remove a command from linked list
- * @haystack: list of commands
- * @needle: command to remove
- */
-
-static bool list_del_cmd(struct list_head *haystack,
- struct scsi_cmnd *needle)
-{
- if (list_find_cmd(haystack, needle)) {
- struct NCR5380_cmd *ncmd = scsi_cmd_priv(needle);
-
- list_del(&ncmd->list);
- return true;
- }
- return false;
-}
-
-/**
- * NCR5380_abort - scsi host eh_abort_handler() method
- * @cmd: the command to be aborted
- *
- * Try to abort a given command by removing it from queues and/or sending
- * the target an abort message. This may not succeed in causing a target
- * to abort the command. Nonetheless, the low-level driver must forget about
- * the command because the mid-layer reclaims it and it may be re-issued.
- *
- * The normal path taken by a command is as follows. For EH we trace this
- * same path to locate and abort the command.
- *
- * unissued -> selecting -> [unissued -> selecting ->]... connected ->
- * [disconnected -> connected ->]...
- * [autosense -> connected ->] done
- *
- * If cmd was not found at all then presumably it has already been completed,
- * in which case return SUCCESS to try to avoid further EH measures.
- *
- * If the command has not completed yet, we must not fail to find it.
- * We have no option but to forget the aborted command (even if it still
- * lacks sense data). The mid-layer may re-issue a command that is in error
- * recovery (see scsi_send_eh_cmnd), but the logic and data structures in
- * this driver are such that a command can appear on one queue only.
- *
- * The lock protects driver data structures, but EH handlers also use it
- * to serialize their own execution and prevent their own re-entry.
- */
-
-static int NCR5380_abort(struct scsi_cmnd *cmd)
-{
- struct Scsi_Host *instance = cmd->device->host;
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
- unsigned long flags;
- int result = SUCCESS;
-
- spin_lock_irqsave(&hostdata->lock, flags);
-
-#if (NDEBUG & NDEBUG_ANY)
- scmd_printk(KERN_INFO, cmd, __func__);
-#endif
- NCR5380_dprint(NDEBUG_ANY, instance);
- NCR5380_dprint_phase(NDEBUG_ANY, instance);
-
- if (list_del_cmd(&hostdata->unissued, cmd)) {
- dsprintk(NDEBUG_ABORT, instance,
- "abort: removed %p from issue queue\n", cmd);
- cmd->result = DID_ABORT << 16;
- cmd->scsi_done(cmd); /* No tag or busy flag to worry about */
- goto out;
- }
-
- if (hostdata->selecting == cmd) {
- dsprintk(NDEBUG_ABORT, instance,
- "abort: cmd %p == selecting\n", cmd);
- hostdata->selecting = NULL;
- cmd->result = DID_ABORT << 16;
- complete_cmd(instance, cmd);
- goto out;
- }
-
- if (list_del_cmd(&hostdata->disconnected, cmd)) {
- dsprintk(NDEBUG_ABORT, instance,
- "abort: removed %p from disconnected list\n", cmd);
- /* Can't call NCR5380_select() and send ABORT because that
- * means releasing the lock. Need a bus reset.
- */
- set_host_byte(cmd, DID_ERROR);
- complete_cmd(instance, cmd);
- result = FAILED;
- goto out;
- }
-
- if (hostdata->connected == cmd) {
- dsprintk(NDEBUG_ABORT, instance, "abort: cmd %p is connected\n", cmd);
- hostdata->connected = NULL;
-#ifdef REAL_DMA
- hostdata->dma_len = 0;
-#endif
- if (do_abort(instance)) {
- set_host_byte(cmd, DID_ERROR);
- complete_cmd(instance, cmd);
- result = FAILED;
- goto out;
- }
- set_host_byte(cmd, DID_ABORT);
- complete_cmd(instance, cmd);
- goto out;
- }
-
- if (list_del_cmd(&hostdata->autosense, cmd)) {
- dsprintk(NDEBUG_ABORT, instance,
- "abort: removed %p from sense queue\n", cmd);
- set_host_byte(cmd, DID_ERROR);
- complete_cmd(instance, cmd);
- }
-
-out:
- if (result == FAILED)
- dsprintk(NDEBUG_ABORT, instance, "abort: failed to abort %p\n", cmd);
- else
- dsprintk(NDEBUG_ABORT, instance, "abort: successfully aborted %p\n", cmd);
-
- queue_work(hostdata->work_q, &hostdata->main_task);
- maybe_release_dma_irq(instance);
- spin_unlock_irqrestore(&hostdata->lock, flags);
-
- return result;
-}
-
-
-/**
- * NCR5380_bus_reset - reset the SCSI bus
- * @cmd: SCSI command undergoing EH
- *
- * Returns SUCCESS
- */
-
-static int NCR5380_bus_reset(struct scsi_cmnd *cmd)
-{
- struct Scsi_Host *instance = cmd->device->host;
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
- int i;
- unsigned long flags;
- struct NCR5380_cmd *ncmd;
-
- spin_lock_irqsave(&hostdata->lock, flags);
-
-#if (NDEBUG & NDEBUG_ANY)
- scmd_printk(KERN_INFO, cmd, __func__);
-#endif
- NCR5380_dprint(NDEBUG_ANY, instance);
- NCR5380_dprint_phase(NDEBUG_ANY, instance);
-
- do_reset(instance);
-
- /* reset NCR registers */
- NCR5380_write(MODE_REG, MR_BASE);
- NCR5380_write(TARGET_COMMAND_REG, 0);
- NCR5380_write(SELECT_ENABLE_REG, 0);
-
- /* After the reset, there are no more connected or disconnected commands
- * and no busy units; so clear the low-level status here to avoid
- * conflicts when the mid-level code tries to wake up the affected
- * commands!
- */
-
- if (list_del_cmd(&hostdata->unissued, cmd)) {
- cmd->result = DID_RESET << 16;
- cmd->scsi_done(cmd);
- }
-
- if (hostdata->selecting) {
- hostdata->selecting->result = DID_RESET << 16;
- complete_cmd(instance, hostdata->selecting);
- hostdata->selecting = NULL;
- }
-
- list_for_each_entry(ncmd, &hostdata->disconnected, list) {
- struct scsi_cmnd *cmd = NCR5380_to_scmd(ncmd);
-
- set_host_byte(cmd, DID_RESET);
- cmd->scsi_done(cmd);
- }
- INIT_LIST_HEAD(&hostdata->disconnected);
-
- list_for_each_entry(ncmd, &hostdata->autosense, list) {
- struct scsi_cmnd *cmd = NCR5380_to_scmd(ncmd);
-
- set_host_byte(cmd, DID_RESET);
- cmd->scsi_done(cmd);
- }
- INIT_LIST_HEAD(&hostdata->autosense);
-
- if (hostdata->connected) {
- set_host_byte(hostdata->connected, DID_RESET);
- complete_cmd(instance, hostdata->connected);
- hostdata->connected = NULL;
- }
-
-#ifdef SUPPORT_TAGS
- free_all_tags(hostdata);
-#endif
- for (i = 0; i < 8; ++i)
- hostdata->busy[i] = 0;
-#ifdef REAL_DMA
- hostdata->dma_len = 0;
-#endif
-
- queue_work(hostdata->work_q, &hostdata->main_task);
- maybe_release_dma_irq(instance);
- spin_unlock_irqrestore(&hostdata->lock, flags);
-
- return SUCCESS;
-}
diff --git a/drivers/scsi/atari_scsi.c b/drivers/scsi/atari_scsi.c
index 78d1b2963..a59ad94 100644
--- a/drivers/scsi/atari_scsi.c
+++ b/drivers/scsi/atari_scsi.c
@@ -14,55 +14,23 @@
*
*/
-
-/**************************************************************************/
-/* */
-/* Notes for Falcon SCSI: */
-/* ---------------------- */
-/* */
-/* Since the Falcon SCSI uses the ST-DMA chip, that is shared among */
-/* several device drivers, locking and unlocking the access to this */
-/* chip is required. But locking is not possible from an interrupt, */
-/* since it puts the process to sleep if the lock is not available. */
-/* This prevents "late" locking of the DMA chip, i.e. locking it just */
-/* before using it, since in case of disconnection-reconnection */
-/* commands, the DMA is started from the reselection interrupt. */
-/* */
-/* Two possible schemes for ST-DMA-locking would be: */
-/* 1) The lock is taken for each command separately and disconnecting */
-/* is forbidden (i.e. can_queue = 1). */
-/* 2) The DMA chip is locked when the first command comes in and */
-/* released when the last command is finished and all queues are */
-/* empty. */
-/* The first alternative would result in bad performance, since the */
-/* interleaving of commands would not be used. The second is unfair to */
-/* other drivers using the ST-DMA, because the queues will seldom be */
-/* totally empty if there is a lot of disk traffic. */
-/* */
-/* For this reasons I decided to employ a more elaborate scheme: */
-/* - First, we give up the lock every time we can (for fairness), this */
-/* means every time a command finishes and there are no other commands */
-/* on the disconnected queue. */
-/* - If there are others waiting to lock the DMA chip, we stop */
-/* issuing commands, i.e. moving them onto the issue queue. */
-/* Because of that, the disconnected queue will run empty in a */
-/* while. Instead we go to sleep on a 'fairness_queue'. */
-/* - If the lock is released, all processes waiting on the fairness */
-/* queue will be woken. The first of them tries to re-lock the DMA, */
-/* the others wait for the first to finish this task. After that, */
-/* they can all run on and do their commands... */
-/* This sounds complicated (and it is it :-(), but it seems to be a */
-/* good compromise between fairness and performance: As long as no one */
-/* else wants to work with the ST-DMA chip, SCSI can go along as */
-/* usual. If now someone else comes, this behaviour is changed to a */
-/* "fairness mode": just already initiated commands are finished and */
-/* then the lock is released. The other one waiting will probably win */
-/* the race for locking the DMA, since it was waiting for longer. And */
-/* after it has finished, SCSI can go ahead again. Finally: I hope I */
-/* have not produced any deadlock possibilities! */
-/* */
-/**************************************************************************/
-
+/*
+ * Notes for Falcon SCSI DMA
+ *
+ * The 5380 device is one of several that all share the DMA chip. Hence
+ * "locking" and "unlocking" access to this chip is required.
+ *
+ * Two possible schemes for ST DMA acquisition by atari_scsi are:
+ * 1) The lock is taken for each command separately (i.e. can_queue == 1).
+ * 2) The lock is taken when the first command arrives and released
+ * when the last command is finished (i.e. can_queue > 1).
+ *
+ * The first alternative limits SCSI bus utilization, since interleaving
+ * commands is not possible. The second gives better performance but is
+ * unfair to other drivers needing to use the ST DMA chip. In order to
+ * allow the IDE and floppy drivers equal access to the ST DMA chip
+ * the default is can_queue == 1.
+ */
#include <linux/module.h>
#include <linux/types.h>
@@ -83,13 +51,10 @@
#include <scsi/scsi_host.h>
-/* Definitions for the core NCR5380 driver. */
-
-#define REAL_DMA
-#define SUPPORT_TAGS
-#define MAX_TAGS 32
#define DMA_MIN_SIZE 32
+/* Definitions for the core NCR5380 driver. */
+
#define NCR5380_implementation_fields /* none */
#define NCR5380_read(reg) atari_scsi_reg_read(reg)
@@ -99,9 +64,9 @@
#define NCR5380_abort atari_scsi_abort
#define NCR5380_info atari_scsi_info
-#define NCR5380_dma_read_setup(instance, data, count) \
+#define NCR5380_dma_recv_setup(instance, data, count) \
atari_scsi_dma_setup(instance, data, count, 0)
-#define NCR5380_dma_write_setup(instance, data, count) \
+#define NCR5380_dma_send_setup(instance, data, count) \
atari_scsi_dma_setup(instance, data, count, 1)
#define NCR5380_dma_residual(instance) \
atari_scsi_dma_residual(instance)
@@ -159,14 +124,11 @@
return adr;
}
-#ifdef REAL_DMA
static void atari_scsi_fetch_restbytes(void);
-#endif
static unsigned char (*atari_scsi_reg_read)(unsigned char reg);
static void (*atari_scsi_reg_write)(unsigned char reg, unsigned char value);
-#ifdef REAL_DMA
static unsigned long atari_dma_residual, atari_dma_startaddr;
static short atari_dma_active;
/* pointer to the dribble buffer */
@@ -185,7 +147,6 @@
/* mask for address bits that can't be used with the ST-DMA */
static unsigned long atari_dma_stram_mask;
#define STRAM_ADDR(a) (((a) & atari_dma_stram_mask) == 0)
-#endif
static int setup_can_queue = -1;
module_param(setup_can_queue, int, 0);
@@ -193,16 +154,12 @@
module_param(setup_cmd_per_lun, int, 0);
static int setup_sg_tablesize = -1;
module_param(setup_sg_tablesize, int, 0);
-static int setup_use_tagged_queuing = -1;
-module_param(setup_use_tagged_queuing, int, 0);
static int setup_hostid = -1;
module_param(setup_hostid, int, 0);
static int setup_toshiba_delay = -1;
module_param(setup_toshiba_delay, int, 0);
-#if defined(REAL_DMA)
-
static int scsi_dma_is_ignored_buserr(unsigned char dma_stat)
{
int i;
@@ -255,12 +212,9 @@
}
#endif
-#endif
-
static irqreturn_t scsi_tt_intr(int irq, void *dev)
{
-#ifdef REAL_DMA
struct Scsi_Host *instance = dev;
struct NCR5380_hostdata *hostdata = shost_priv(instance);
int dma_stat;
@@ -342,8 +296,6 @@
tt_scsi_dma.dma_ctrl = 0;
}
-#endif /* REAL_DMA */
-
NCR5380_intr(irq, dev);
return IRQ_HANDLED;
@@ -352,7 +304,6 @@
static irqreturn_t scsi_falcon_intr(int irq, void *dev)
{
-#ifdef REAL_DMA
struct Scsi_Host *instance = dev;
struct NCR5380_hostdata *hostdata = shost_priv(instance);
int dma_stat;
@@ -405,15 +356,12 @@
atari_dma_orig_addr = NULL;
}
-#endif /* REAL_DMA */
-
NCR5380_intr(irq, dev);
return IRQ_HANDLED;
}
-#ifdef REAL_DMA
static void atari_scsi_fetch_restbytes(void)
{
int nr;
@@ -436,7 +384,6 @@
*dst++ = *src++;
}
}
-#endif /* REAL_DMA */
/* This function releases the lock on the DMA chip if there is no
@@ -464,6 +411,10 @@
if (IS_A_TT())
return 1;
+ if (stdma_is_locked_by(scsi_falcon_intr) &&
+ instance->hostt->can_queue > 1)
+ return 1;
+
if (in_interrupt())
return stdma_try_lock(scsi_falcon_intr, instance);
@@ -495,8 +446,7 @@
setup_sg_tablesize = ints[3];
if (ints[0] >= 4)
setup_hostid = ints[4];
- if (ints[0] >= 5)
- setup_use_tagged_queuing = ints[5];
+ /* ints[5] (use_tagged_queuing) is ignored */
/* ints[6] (use_pdma) is ignored */
if (ints[0] >= 7)
setup_toshiba_delay = ints[7];
@@ -508,8 +458,6 @@
#endif /* !MODULE */
-#if defined(REAL_DMA)
-
static unsigned long atari_scsi_dma_setup(struct Scsi_Host *instance,
void *data, unsigned long count,
int dir)
@@ -545,9 +493,6 @@
*/
dma_cache_maintenance(addr, count, dir);
- if (count == 0)
- printk(KERN_NOTICE "SCSI warning: DMA programmed for 0 bytes !\n");
-
if (IS_A_TT()) {
tt_scsi_dma.dma_ctrl = dir;
SCSI_DMA_WRITE_P(dma_addr, addr);
@@ -624,6 +569,9 @@
{
unsigned long possible_len, limit;
+ if (wanted_len < DMA_MIN_SIZE)
+ return 0;
+
if (IS_A_TT())
/* TT SCSI DMA can transfer arbitrary #bytes */
return wanted_len;
@@ -703,9 +651,6 @@
}
-#endif /* REAL_DMA */
-
-
/* NCR5380 register access functions
*
* There are separate functions for TT and Falcon, because the access
@@ -736,7 +681,7 @@
}
-#include "atari_NCR5380.c"
+#include "NCR5380.c"
static int atari_scsi_bus_reset(struct scsi_cmnd *cmd)
{
@@ -745,7 +690,6 @@
local_irq_save(flags);
-#ifdef REAL_DMA
/* Abort a maybe active DMA transfer */
if (IS_A_TT()) {
tt_scsi_dma.dma_ctrl = 0;
@@ -754,7 +698,6 @@
atari_dma_active = 0;
atari_dma_orig_addr = NULL;
}
-#endif
rv = NCR5380_bus_reset(cmd);
@@ -781,6 +724,7 @@
.eh_abort_handler = atari_scsi_abort,
.eh_bus_reset_handler = atari_scsi_bus_reset,
.this_id = 7,
+ .cmd_per_lun = 2,
.use_clustering = DISABLE_CLUSTERING,
.cmd_size = NCR5380_CMD_SIZE,
};
@@ -804,24 +748,11 @@
atari_scsi_reg_write = atari_scsi_falcon_reg_write;
}
- /* The values for CMD_PER_LUN and CAN_QUEUE are somehow arbitrary.
- * Higher values should work, too; try it!
- * (But cmd_per_lun costs memory!)
- *
- * But there seems to be a bug somewhere that requires CAN_QUEUE to be
- * 2*CMD_PER_LUN. At least on a TT, no spurious timeouts seen since
- * changed CMD_PER_LUN...
- *
- * Note: The Falcon currently uses 8/1 setting due to unsolved problems
- * with cmd_per_lun != 1
- */
if (ATARIHW_PRESENT(TT_SCSI)) {
atari_scsi_template.can_queue = 16;
- atari_scsi_template.cmd_per_lun = 8;
atari_scsi_template.sg_tablesize = SG_ALL;
} else {
- atari_scsi_template.can_queue = 8;
- atari_scsi_template.cmd_per_lun = 1;
+ atari_scsi_template.can_queue = 1;
atari_scsi_template.sg_tablesize = SG_NONE;
}
@@ -850,8 +781,6 @@
}
}
-
-#ifdef REAL_DMA
/* If running on a Falcon and if there's TT-Ram (i.e., more than one
* memory block, since there's always ST-Ram in a Falcon), then
* allocate a STRAM_BUFFER_SIZE byte dribble buffer for transfers
@@ -867,7 +796,6 @@
atari_dma_phys_buffer = atari_stram_to_phys(atari_dma_buffer);
atari_dma_orig_addr = 0;
}
-#endif
instance = scsi_host_alloc(&atari_scsi_template,
sizeof(struct NCR5380_hostdata));
@@ -879,9 +807,6 @@
instance->irq = irq->start;
host_flags |= IS_A_TT() ? 0 : FLAG_LATE_DMA_SETUP;
-#ifdef SUPPORT_TAGS
- host_flags |= setup_use_tagged_queuing > 0 ? FLAG_TAGGED_QUEUING : 0;
-#endif
host_flags |= setup_toshiba_delay > 0 ? FLAG_TOSHIBA_DELAY : 0;
error = NCR5380_init(instance, host_flags);
@@ -897,7 +822,7 @@
goto fail_irq;
}
tt_mfp.active_edge |= 0x80; /* SCSI int on L->H */
-#ifdef REAL_DMA
+
tt_scsi_dma.dma_ctrl = 0;
atari_dma_residual = 0;
@@ -919,17 +844,14 @@
hostdata->read_overruns = 4;
}
-#endif
} else {
/* Nothing to do for the interrupt: the ST-DMA is initialized
* already.
*/
-#ifdef REAL_DMA
atari_dma_residual = 0;
atari_dma_active = 0;
atari_dma_stram_mask = (ATARIHW_PRESENT(EXTD_DMA) ? 0x00000000
: 0xff000000);
-#endif
}
NCR5380_maybe_reset_bus(instance);
diff --git a/drivers/scsi/bfa/bfa_fcs.h b/drivers/scsi/bfa/bfa_fcs.h
index 06dc215..0f797a5 100644
--- a/drivers/scsi/bfa/bfa_fcs.h
+++ b/drivers/scsi/bfa/bfa_fcs.h
@@ -874,8 +874,8 @@
/*
* itnim callbacks
*/
-void bfa_fcb_itnim_alloc(struct bfad_s *bfad, struct bfa_fcs_itnim_s **itnim,
- struct bfad_itnim_s **itnim_drv);
+int bfa_fcb_itnim_alloc(struct bfad_s *bfad, struct bfa_fcs_itnim_s **itnim,
+ struct bfad_itnim_s **itnim_drv);
void bfa_fcb_itnim_free(struct bfad_s *bfad,
struct bfad_itnim_s *itnim_drv);
void bfa_fcb_itnim_online(struct bfad_itnim_s *itnim_drv);
diff --git a/drivers/scsi/bfa/bfa_fcs_fcpim.c b/drivers/scsi/bfa/bfa_fcs_fcpim.c
index 4f089d7..2e3b19e 100644
--- a/drivers/scsi/bfa/bfa_fcs_fcpim.c
+++ b/drivers/scsi/bfa/bfa_fcs_fcpim.c
@@ -588,12 +588,13 @@
struct bfa_fcs_lport_s *port = rport->port;
struct bfa_fcs_itnim_s *itnim;
struct bfad_itnim_s *itnim_drv;
+ int ret;
/*
* call bfad to allocate the itnim
*/
- bfa_fcb_itnim_alloc(port->fcs->bfad, &itnim, &itnim_drv);
- if (itnim == NULL) {
+ ret = bfa_fcb_itnim_alloc(port->fcs->bfad, &itnim, &itnim_drv);
+ if (ret) {
bfa_trc(port->fcs, rport->pwwn);
return NULL;
}
diff --git a/drivers/scsi/bfa/bfad_im.c b/drivers/scsi/bfa/bfad_im.c
index 6c805e1..02d8060 100644
--- a/drivers/scsi/bfa/bfad_im.c
+++ b/drivers/scsi/bfa/bfad_im.c
@@ -440,13 +440,13 @@
* BFA FCS itnim alloc callback, after successful PRLI
* Context: Interrupt
*/
-void
+int
bfa_fcb_itnim_alloc(struct bfad_s *bfad, struct bfa_fcs_itnim_s **itnim,
struct bfad_itnim_s **itnim_drv)
{
*itnim_drv = kzalloc(sizeof(struct bfad_itnim_s), GFP_ATOMIC);
if (*itnim_drv == NULL)
- return;
+ return -ENOMEM;
(*itnim_drv)->im = bfad->im;
*itnim = &(*itnim_drv)->fcs_itnim;
@@ -457,6 +457,7 @@
*/
INIT_WORK(&(*itnim_drv)->itnim_work, bfad_im_itnim_work_handler);
bfad->bfad_flags |= BFAD_RPORT_ONLINE;
+ return 0;
}
/*
diff --git a/drivers/scsi/bfa/bfi.h b/drivers/scsi/bfa/bfi.h
index 97600dc..5f698d0 100644
--- a/drivers/scsi/bfa/bfi.h
+++ b/drivers/scsi/bfa/bfi.h
@@ -356,7 +356,7 @@
u8 port0_mode; /* device mode for port 0 */
u8 port1_mode; /* device mode for port 1 */
u32 exec; /* exec vector */
- u32 bootenv; /* fimware boot env */
+ u32 bootenv; /* firmware boot env */
u32 rsvd_b[2];
struct bfi_ioc_fwver_s fwver;
u32 md5sum[BFI_IOC_MD5SUM_SZ];
diff --git a/drivers/scsi/bnx2fc/bnx2fc.h b/drivers/scsi/bnx2fc/bnx2fc.h
index 499e369..fdd4eb4 100644
--- a/drivers/scsi/bnx2fc/bnx2fc.h
+++ b/drivers/scsi/bnx2fc/bnx2fc.h
@@ -65,7 +65,7 @@
#include "bnx2fc_constants.h"
#define BNX2FC_NAME "bnx2fc"
-#define BNX2FC_VERSION "2.9.6"
+#define BNX2FC_VERSION "2.10.3"
#define PFX "bnx2fc: "
@@ -261,6 +261,7 @@
u8 vlan_enabled;
int vlan_id;
bool enabled;
+ u8 tm_timeout;
};
#define bnx2fc_from_ctlr(x) \
diff --git a/drivers/scsi/bnx2fc/bnx2fc_fcoe.c b/drivers/scsi/bnx2fc/bnx2fc_fcoe.c
index d7029ea..a188199 100644
--- a/drivers/scsi/bnx2fc/bnx2fc_fcoe.c
+++ b/drivers/scsi/bnx2fc/bnx2fc_fcoe.c
@@ -107,6 +107,26 @@
"\t\t0x10 - fcoe L2 fame related logs.\n"
"\t\t0xff - LOG all messages.");
+uint bnx2fc_devloss_tmo;
+module_param_named(devloss_tmo, bnx2fc_devloss_tmo, uint, S_IRUGO);
+MODULE_PARM_DESC(devloss_tmo, " Change devloss_tmo for the remote ports "
+ "attached via bnx2fc.");
+
+uint bnx2fc_max_luns = BNX2FC_MAX_LUN;
+module_param_named(max_luns, bnx2fc_max_luns, uint, S_IRUGO);
+MODULE_PARM_DESC(max_luns, " Change the default max_lun per SCSI host. Default "
+ "0xffff.");
+
+uint bnx2fc_queue_depth;
+module_param_named(queue_depth, bnx2fc_queue_depth, uint, S_IRUGO);
+MODULE_PARM_DESC(queue_depth, " Change the default queue depth of SCSI devices "
+ "attached via bnx2fc.");
+
+uint bnx2fc_log_fka;
+module_param_named(log_fka, bnx2fc_log_fka, uint, S_IRUGO|S_IWUSR);
+MODULE_PARM_DESC(log_fka, " Print message to kernel log when fcoe is "
+ "initiating a FIP keep alive when debug logging is enabled.");
+
static int bnx2fc_cpu_callback(struct notifier_block *nfb,
unsigned long action, void *hcpu);
/* notification function for CPU hotplug events */
@@ -692,7 +712,7 @@
int rc = 0;
shost->max_cmd_len = BNX2FC_MAX_CMD_LEN;
- shost->max_lun = BNX2FC_MAX_LUN;
+ shost->max_lun = bnx2fc_max_luns;
shost->max_id = BNX2FC_MAX_FCP_TGT;
shost->max_channel = 0;
if (lport->vport)
@@ -1061,6 +1081,20 @@
*/
static void bnx2fc_fip_send(struct fcoe_ctlr *fip, struct sk_buff *skb)
{
+ struct fip_header *fiph;
+ struct ethhdr *eth_hdr;
+ u16 op;
+ u8 sub;
+
+ fiph = (struct fip_header *) ((void *)skb->data + 2 * ETH_ALEN + 2);
+ eth_hdr = (struct ethhdr *)skb_mac_header(skb);
+ op = ntohs(fiph->fip_op);
+ sub = fiph->fip_subcode;
+
+ if (op == FIP_OP_CTRL && sub == FIP_SC_SOL && bnx2fc_log_fka)
+ BNX2FC_MISC_DBG("Sending FKA from %pM to %pM.\n",
+ eth_hdr->h_source, eth_hdr->h_dest);
+
skb->dev = bnx2fc_from_ctlr(fip)->netdev;
dev_queue_xmit(skb);
}
@@ -1102,6 +1136,9 @@
return -EIO;
}
+ if (bnx2fc_devloss_tmo)
+ fc_host_dev_loss_tmo(vn_port->host) = bnx2fc_devloss_tmo;
+
if (disabled) {
fc_vport_set_state(vport, FC_VPORT_DISABLED);
} else {
@@ -1495,6 +1532,9 @@
}
fc_host_port_type(lport->host) = FC_PORTTYPE_UNKNOWN;
+ if (bnx2fc_devloss_tmo)
+ fc_host_dev_loss_tmo(shost) = bnx2fc_devloss_tmo;
+
/* Allocate exchange manager */
if (!npiv)
rc = bnx2fc_em_config(lport, hba);
@@ -1999,6 +2039,8 @@
return;
}
+ pr_info(PFX "FCoE initialized for %s.\n", dev->netdev->name);
+
/* Add HBA to the adapter list */
mutex_lock(&bnx2fc_dev_lock);
list_add_tail(&hba->list, &adapter_list);
@@ -2293,6 +2335,7 @@
ctlr = bnx2fc_to_ctlr(interface);
cdev = fcoe_ctlr_to_ctlr_dev(ctlr);
interface->vlan_id = vlan_id;
+ interface->tm_timeout = BNX2FC_TM_TIMEOUT;
interface->timer_work_queue =
create_singlethread_workqueue("bnx2fc_timer_wq");
@@ -2612,6 +2655,15 @@
return NOTIFY_OK;
}
+static int bnx2fc_slave_configure(struct scsi_device *sdev)
+{
+ if (!bnx2fc_queue_depth)
+ return 0;
+
+ scsi_change_queue_depth(sdev, bnx2fc_queue_depth);
+ return 0;
+}
+
/**
* bnx2fc_mod_init - module init entry point
*
@@ -2858,6 +2910,50 @@
.bsg_request = fc_lport_bsg_request,
};
+/*
+ * Additional scsi_host attributes.
+ */
+static ssize_t
+bnx2fc_tm_timeout_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct Scsi_Host *shost = class_to_shost(dev);
+ struct fc_lport *lport = shost_priv(shost);
+ struct fcoe_port *port = lport_priv(lport);
+ struct bnx2fc_interface *interface = port->priv;
+
+ sprintf(buf, "%u\n", interface->tm_timeout);
+ return strlen(buf);
+}
+
+static ssize_t
+bnx2fc_tm_timeout_store(struct device *dev,
+ struct device_attribute *attr, const char *buf, size_t count)
+{
+ struct Scsi_Host *shost = class_to_shost(dev);
+ struct fc_lport *lport = shost_priv(shost);
+ struct fcoe_port *port = lport_priv(lport);
+ struct bnx2fc_interface *interface = port->priv;
+ int rval, val;
+
+ rval = kstrtouint(buf, 10, &val);
+ if (rval)
+ return rval;
+ if (val > 255)
+ return -ERANGE;
+
+ interface->tm_timeout = (u8)val;
+ return strlen(buf);
+}
+
+static DEVICE_ATTR(tm_timeout, S_IRUGO|S_IWUSR, bnx2fc_tm_timeout_show,
+ bnx2fc_tm_timeout_store);
+
+static struct device_attribute *bnx2fc_host_attrs[] = {
+ &dev_attr_tm_timeout,
+ NULL,
+};
+
/**
* scsi_host_template structure used while registering with SCSI-ml
*/
@@ -2877,6 +2973,8 @@
.sg_tablesize = BNX2FC_MAX_BDS_PER_CMD,
.max_sectors = 1024,
.track_queue_depth = 1,
+ .slave_configure = bnx2fc_slave_configure,
+ .shost_attrs = bnx2fc_host_attrs,
};
static struct libfc_function_template bnx2fc_libfc_fcn_templ = {
diff --git a/drivers/scsi/bnx2fc/bnx2fc_io.c b/drivers/scsi/bnx2fc/bnx2fc_io.c
index 2230dab..026f394 100644
--- a/drivers/scsi/bnx2fc/bnx2fc_io.c
+++ b/drivers/scsi/bnx2fc/bnx2fc_io.c
@@ -179,12 +179,24 @@
bnx2fc_unmap_sg_list(io_req);
io_req->sc_cmd = NULL;
+
+ /* Sanity checks before returning command to mid-layer */
if (!sc_cmd) {
printk(KERN_ERR PFX "scsi_done - sc_cmd NULL. "
"IO(0x%x) already cleaned up\n",
io_req->xid);
return;
}
+ if (!sc_cmd->device) {
+ pr_err(PFX "0x%x: sc_cmd->device is NULL.\n", io_req->xid);
+ return;
+ }
+ if (!sc_cmd->device->host) {
+ pr_err(PFX "0x%x: sc_cmd->device->host is NULL.\n",
+ io_req->xid);
+ return;
+ }
+
sc_cmd->result = err_code << 16;
BNX2FC_IO_DBG(io_req, "sc=%p, result=0x%x, retries=%d, allowed=%d\n",
@@ -770,7 +782,7 @@
spin_unlock_bh(&tgt->tgt_lock);
rc = wait_for_completion_timeout(&io_req->tm_done,
- BNX2FC_TM_TIMEOUT * HZ);
+ interface->tm_timeout * HZ);
spin_lock_bh(&tgt->tgt_lock);
io_req->wait_for_comp = 0;
diff --git a/drivers/scsi/bnx2i/bnx2i_iscsi.c b/drivers/scsi/bnx2i/bnx2i_iscsi.c
index 7289437..133901f 100644
--- a/drivers/scsi/bnx2i/bnx2i_iscsi.c
+++ b/drivers/scsi/bnx2i/bnx2i_iscsi.c
@@ -675,7 +675,7 @@
{
struct list_head *list;
struct list_head *tmp;
- struct bnx2i_endpoint *ep;
+ struct bnx2i_endpoint *ep = NULL;
read_lock_bh(&hba->ep_rdwr_lock);
list_for_each_safe(list, tmp, &hba->ep_ofld_list) {
@@ -703,7 +703,7 @@
{
struct list_head *list;
struct list_head *tmp;
- struct bnx2i_endpoint *ep;
+ struct bnx2i_endpoint *ep = NULL;
read_lock_bh(&hba->ep_rdwr_lock);
list_for_each_safe(list, tmp, &hba->ep_destroy_list) {
diff --git a/drivers/scsi/constants.c b/drivers/scsi/constants.c
index fa09d4b..83458f7 100644
--- a/drivers/scsi/constants.c
+++ b/drivers/scsi/constants.c
@@ -292,850 +292,30 @@
struct error_info {
unsigned short code12; /* 0x0302 looks better than 0x03,0x02 */
- const char * text;
+ unsigned short size;
};
/*
- * The canonical list of T10 Additional Sense Codes is available at:
- * http://www.t10.org/lists/asc-num.txt [most recent: 20141221]
+ * There are 700+ entries in this table. To save space, we don't store
+ * (code, pointer) pairs, which would make sizeof(struct
+ * error_info)==16 on 64 bits. Rather, the second element just stores
+ * the size (including \0) of the corresponding string, and we use the
+ * sum of these to get the appropriate offset into additional_text
+ * defined below. This approach saves 12 bytes per entry.
*/
-
static const struct error_info additional[] =
{
- {0x0000, "No additional sense information"},
- {0x0001, "Filemark detected"},
- {0x0002, "End-of-partition/medium detected"},
- {0x0003, "Setmark detected"},
- {0x0004, "Beginning-of-partition/medium detected"},
- {0x0005, "End-of-data detected"},
- {0x0006, "I/O process terminated"},
- {0x0007, "Programmable early warning detected"},
- {0x0011, "Audio play operation in progress"},
- {0x0012, "Audio play operation paused"},
- {0x0013, "Audio play operation successfully completed"},
- {0x0014, "Audio play operation stopped due to error"},
- {0x0015, "No current audio status to return"},
- {0x0016, "Operation in progress"},
- {0x0017, "Cleaning requested"},
- {0x0018, "Erase operation in progress"},
- {0x0019, "Locate operation in progress"},
- {0x001A, "Rewind operation in progress"},
- {0x001B, "Set capacity operation in progress"},
- {0x001C, "Verify operation in progress"},
- {0x001D, "ATA pass through information available"},
- {0x001E, "Conflicting SA creation request"},
- {0x001F, "Logical unit transitioning to another power condition"},
- {0x0020, "Extended copy information available"},
- {0x0021, "Atomic command aborted due to ACA"},
-
- {0x0100, "No index/sector signal"},
-
- {0x0200, "No seek complete"},
-
- {0x0300, "Peripheral device write fault"},
- {0x0301, "No write current"},
- {0x0302, "Excessive write errors"},
-
- {0x0400, "Logical unit not ready, cause not reportable"},
- {0x0401, "Logical unit is in process of becoming ready"},
- {0x0402, "Logical unit not ready, initializing command required"},
- {0x0403, "Logical unit not ready, manual intervention required"},
- {0x0404, "Logical unit not ready, format in progress"},
- {0x0405, "Logical unit not ready, rebuild in progress"},
- {0x0406, "Logical unit not ready, recalculation in progress"},
- {0x0407, "Logical unit not ready, operation in progress"},
- {0x0408, "Logical unit not ready, long write in progress"},
- {0x0409, "Logical unit not ready, self-test in progress"},
- {0x040A, "Logical unit not accessible, asymmetric access state "
- "transition"},
- {0x040B, "Logical unit not accessible, target port in standby state"},
- {0x040C, "Logical unit not accessible, target port in unavailable "
- "state"},
- {0x040D, "Logical unit not ready, structure check required"},
- {0x040E, "Logical unit not ready, security session in progress"},
- {0x0410, "Logical unit not ready, auxiliary memory not accessible"},
- {0x0411, "Logical unit not ready, notify (enable spinup) required"},
- {0x0412, "Logical unit not ready, offline"},
- {0x0413, "Logical unit not ready, SA creation in progress"},
- {0x0414, "Logical unit not ready, space allocation in progress"},
- {0x0415, "Logical unit not ready, robotics disabled"},
- {0x0416, "Logical unit not ready, configuration required"},
- {0x0417, "Logical unit not ready, calibration required"},
- {0x0418, "Logical unit not ready, a door is open"},
- {0x0419, "Logical unit not ready, operating in sequential mode"},
- {0x041A, "Logical unit not ready, start stop unit command in "
- "progress"},
- {0x041B, "Logical unit not ready, sanitize in progress"},
- {0x041C, "Logical unit not ready, additional power use not yet "
- "granted"},
- {0x041D, "Logical unit not ready, configuration in progress"},
- {0x041E, "Logical unit not ready, microcode activation required"},
- {0x041F, "Logical unit not ready, microcode download required"},
- {0x0420, "Logical unit not ready, logical unit reset required"},
- {0x0421, "Logical unit not ready, hard reset required"},
- {0x0422, "Logical unit not ready, power cycle required"},
-
- {0x0500, "Logical unit does not respond to selection"},
-
- {0x0600, "No reference position found"},
-
- {0x0700, "Multiple peripheral devices selected"},
-
- {0x0800, "Logical unit communication failure"},
- {0x0801, "Logical unit communication time-out"},
- {0x0802, "Logical unit communication parity error"},
- {0x0803, "Logical unit communication CRC error (Ultra-DMA/32)"},
- {0x0804, "Unreachable copy target"},
-
- {0x0900, "Track following error"},
- {0x0901, "Tracking servo failure"},
- {0x0902, "Focus servo failure"},
- {0x0903, "Spindle servo failure"},
- {0x0904, "Head select fault"},
- {0x0905, "Vibration induced tracking error"},
-
- {0x0A00, "Error log overflow"},
-
- {0x0B00, "Warning"},
- {0x0B01, "Warning - specified temperature exceeded"},
- {0x0B02, "Warning - enclosure degraded"},
- {0x0B03, "Warning - background self-test failed"},
- {0x0B04, "Warning - background pre-scan detected medium error"},
- {0x0B05, "Warning - background medium scan detected medium error"},
- {0x0B06, "Warning - non-volatile cache now volatile"},
- {0x0B07, "Warning - degraded power to non-volatile cache"},
- {0x0B08, "Warning - power loss expected"},
- {0x0B09, "Warning - device statistics notification active"},
-
- {0x0C00, "Write error"},
- {0x0C01, "Write error - recovered with auto reallocation"},
- {0x0C02, "Write error - auto reallocation failed"},
- {0x0C03, "Write error - recommend reassignment"},
- {0x0C04, "Compression check miscompare error"},
- {0x0C05, "Data expansion occurred during compression"},
- {0x0C06, "Block not compressible"},
- {0x0C07, "Write error - recovery needed"},
- {0x0C08, "Write error - recovery failed"},
- {0x0C09, "Write error - loss of streaming"},
- {0x0C0A, "Write error - padding blocks added"},
- {0x0C0B, "Auxiliary memory write error"},
- {0x0C0C, "Write error - unexpected unsolicited data"},
- {0x0C0D, "Write error - not enough unsolicited data"},
- {0x0C0E, "Multiple write errors"},
- {0x0C0F, "Defects in error window"},
- {0x0C10, "Incomplete multiple atomic write operations"},
-
- {0x0D00, "Error detected by third party temporary initiator"},
- {0x0D01, "Third party device failure"},
- {0x0D02, "Copy target device not reachable"},
- {0x0D03, "Incorrect copy target device type"},
- {0x0D04, "Copy target device data underrun"},
- {0x0D05, "Copy target device data overrun"},
-
- {0x0E00, "Invalid information unit"},
- {0x0E01, "Information unit too short"},
- {0x0E02, "Information unit too long"},
- {0x0E03, "Invalid field in command information unit"},
-
- {0x1000, "Id CRC or ECC error"},
- {0x1001, "Logical block guard check failed"},
- {0x1002, "Logical block application tag check failed"},
- {0x1003, "Logical block reference tag check failed"},
- {0x1004, "Logical block protection error on recover buffered data"},
- {0x1005, "Logical block protection method error"},
-
- {0x1100, "Unrecovered read error"},
- {0x1101, "Read retries exhausted"},
- {0x1102, "Error too long to correct"},
- {0x1103, "Multiple read errors"},
- {0x1104, "Unrecovered read error - auto reallocate failed"},
- {0x1105, "L-EC uncorrectable error"},
- {0x1106, "CIRC unrecovered error"},
- {0x1107, "Data re-synchronization error"},
- {0x1108, "Incomplete block read"},
- {0x1109, "No gap found"},
- {0x110A, "Miscorrected error"},
- {0x110B, "Unrecovered read error - recommend reassignment"},
- {0x110C, "Unrecovered read error - recommend rewrite the data"},
- {0x110D, "De-compression CRC error"},
- {0x110E, "Cannot decompress using declared algorithm"},
- {0x110F, "Error reading UPC/EAN number"},
- {0x1110, "Error reading ISRC number"},
- {0x1111, "Read error - loss of streaming"},
- {0x1112, "Auxiliary memory read error"},
- {0x1113, "Read error - failed retransmission request"},
- {0x1114, "Read error - lba marked bad by application client"},
- {0x1115, "Write after sanitize required"},
-
- {0x1200, "Address mark not found for id field"},
-
- {0x1300, "Address mark not found for data field"},
-
- {0x1400, "Recorded entity not found"},
- {0x1401, "Record not found"},
- {0x1402, "Filemark or setmark not found"},
- {0x1403, "End-of-data not found"},
- {0x1404, "Block sequence error"},
- {0x1405, "Record not found - recommend reassignment"},
- {0x1406, "Record not found - data auto-reallocated"},
- {0x1407, "Locate operation failure"},
-
- {0x1500, "Random positioning error"},
- {0x1501, "Mechanical positioning error"},
- {0x1502, "Positioning error detected by read of medium"},
-
- {0x1600, "Data synchronization mark error"},
- {0x1601, "Data sync error - data rewritten"},
- {0x1602, "Data sync error - recommend rewrite"},
- {0x1603, "Data sync error - data auto-reallocated"},
- {0x1604, "Data sync error - recommend reassignment"},
-
- {0x1700, "Recovered data with no error correction applied"},
- {0x1701, "Recovered data with retries"},
- {0x1702, "Recovered data with positive head offset"},
- {0x1703, "Recovered data with negative head offset"},
- {0x1704, "Recovered data with retries and/or circ applied"},
- {0x1705, "Recovered data using previous sector id"},
- {0x1706, "Recovered data without ECC - data auto-reallocated"},
- {0x1707, "Recovered data without ECC - recommend reassignment"},
- {0x1708, "Recovered data without ECC - recommend rewrite"},
- {0x1709, "Recovered data without ECC - data rewritten"},
-
- {0x1800, "Recovered data with error correction applied"},
- {0x1801, "Recovered data with error corr. & retries applied"},
- {0x1802, "Recovered data - data auto-reallocated"},
- {0x1803, "Recovered data with CIRC"},
- {0x1804, "Recovered data with L-EC"},
- {0x1805, "Recovered data - recommend reassignment"},
- {0x1806, "Recovered data - recommend rewrite"},
- {0x1807, "Recovered data with ECC - data rewritten"},
- {0x1808, "Recovered data with linking"},
-
- {0x1900, "Defect list error"},
- {0x1901, "Defect list not available"},
- {0x1902, "Defect list error in primary list"},
- {0x1903, "Defect list error in grown list"},
-
- {0x1A00, "Parameter list length error"},
-
- {0x1B00, "Synchronous data transfer error"},
-
- {0x1C00, "Defect list not found"},
- {0x1C01, "Primary defect list not found"},
- {0x1C02, "Grown defect list not found"},
-
- {0x1D00, "Miscompare during verify operation"},
- {0x1D01, "Miscompare verify of unmapped LBA"},
-
- {0x1E00, "Recovered id with ECC correction"},
-
- {0x1F00, "Partial defect list transfer"},
-
- {0x2000, "Invalid command operation code"},
- {0x2001, "Access denied - initiator pending-enrolled"},
- {0x2002, "Access denied - no access rights"},
- {0x2003, "Access denied - invalid mgmt id key"},
- {0x2004, "Illegal command while in write capable state"},
- {0x2005, "Obsolete"},
- {0x2006, "Illegal command while in explicit address mode"},
- {0x2007, "Illegal command while in implicit address mode"},
- {0x2008, "Access denied - enrollment conflict"},
- {0x2009, "Access denied - invalid LU identifier"},
- {0x200A, "Access denied - invalid proxy token"},
- {0x200B, "Access denied - ACL LUN conflict"},
- {0x200C, "Illegal command when not in append-only mode"},
-
- {0x2100, "Logical block address out of range"},
- {0x2101, "Invalid element address"},
- {0x2102, "Invalid address for write"},
- {0x2103, "Invalid write crossing layer jump"},
- {0x2104, "Unaligned write command"},
- {0x2105, "Write boundary violation"},
- {0x2106, "Attempt to read invalid data"},
- {0x2107, "Read boundary violation"},
-
- {0x2200, "Illegal function (use 20 00, 24 00, or 26 00)"},
-
- {0x2300, "Invalid token operation, cause not reportable"},
- {0x2301, "Invalid token operation, unsupported token type"},
- {0x2302, "Invalid token operation, remote token usage not supported"},
- {0x2303, "Invalid token operation, remote rod token creation not "
- "supported"},
- {0x2304, "Invalid token operation, token unknown"},
- {0x2305, "Invalid token operation, token corrupt"},
- {0x2306, "Invalid token operation, token revoked"},
- {0x2307, "Invalid token operation, token expired"},
- {0x2308, "Invalid token operation, token cancelled"},
- {0x2309, "Invalid token operation, token deleted"},
- {0x230A, "Invalid token operation, invalid token length"},
-
- {0x2400, "Invalid field in cdb"},
- {0x2401, "CDB decryption error"},
- {0x2402, "Obsolete"},
- {0x2403, "Obsolete"},
- {0x2404, "Security audit value frozen"},
- {0x2405, "Security working key frozen"},
- {0x2406, "Nonce not unique"},
- {0x2407, "Nonce timestamp out of range"},
- {0x2408, "Invalid XCDB"},
-
- {0x2500, "Logical unit not supported"},
-
- {0x2600, "Invalid field in parameter list"},
- {0x2601, "Parameter not supported"},
- {0x2602, "Parameter value invalid"},
- {0x2603, "Threshold parameters not supported"},
- {0x2604, "Invalid release of persistent reservation"},
- {0x2605, "Data decryption error"},
- {0x2606, "Too many target descriptors"},
- {0x2607, "Unsupported target descriptor type code"},
- {0x2608, "Too many segment descriptors"},
- {0x2609, "Unsupported segment descriptor type code"},
- {0x260A, "Unexpected inexact segment"},
- {0x260B, "Inline data length exceeded"},
- {0x260C, "Invalid operation for copy source or destination"},
- {0x260D, "Copy segment granularity violation"},
- {0x260E, "Invalid parameter while port is enabled"},
- {0x260F, "Invalid data-out buffer integrity check value"},
- {0x2610, "Data decryption key fail limit reached"},
- {0x2611, "Incomplete key-associated data set"},
- {0x2612, "Vendor specific key reference not found"},
-
- {0x2700, "Write protected"},
- {0x2701, "Hardware write protected"},
- {0x2702, "Logical unit software write protected"},
- {0x2703, "Associated write protect"},
- {0x2704, "Persistent write protect"},
- {0x2705, "Permanent write protect"},
- {0x2706, "Conditional write protect"},
- {0x2707, "Space allocation failed write protect"},
- {0x2708, "Zone is read only"},
-
- {0x2800, "Not ready to ready change, medium may have changed"},
- {0x2801, "Import or export element accessed"},
- {0x2802, "Format-layer may have changed"},
- {0x2803, "Import/export element accessed, medium changed"},
-
- {0x2900, "Power on, reset, or bus device reset occurred"},
- {0x2901, "Power on occurred"},
- {0x2902, "Scsi bus reset occurred"},
- {0x2903, "Bus device reset function occurred"},
- {0x2904, "Device internal reset"},
- {0x2905, "Transceiver mode changed to single-ended"},
- {0x2906, "Transceiver mode changed to lvd"},
- {0x2907, "I_T nexus loss occurred"},
-
- {0x2A00, "Parameters changed"},
- {0x2A01, "Mode parameters changed"},
- {0x2A02, "Log parameters changed"},
- {0x2A03, "Reservations preempted"},
- {0x2A04, "Reservations released"},
- {0x2A05, "Registrations preempted"},
- {0x2A06, "Asymmetric access state changed"},
- {0x2A07, "Implicit asymmetric access state transition failed"},
- {0x2A08, "Priority changed"},
- {0x2A09, "Capacity data has changed"},
- {0x2A0A, "Error history I_T nexus cleared"},
- {0x2A0B, "Error history snapshot released"},
- {0x2A0C, "Error recovery attributes have changed"},
- {0x2A0D, "Data encryption capabilities changed"},
- {0x2A10, "Timestamp changed"},
- {0x2A11, "Data encryption parameters changed by another i_t nexus"},
- {0x2A12, "Data encryption parameters changed by vendor specific "
- "event"},
- {0x2A13, "Data encryption key instance counter has changed"},
- {0x2A14, "SA creation capabilities data has changed"},
- {0x2A15, "Medium removal prevention preempted"},
-
- {0x2B00, "Copy cannot execute since host cannot disconnect"},
-
- {0x2C00, "Command sequence error"},
- {0x2C01, "Too many windows specified"},
- {0x2C02, "Invalid combination of windows specified"},
- {0x2C03, "Current program area is not empty"},
- {0x2C04, "Current program area is empty"},
- {0x2C05, "Illegal power condition request"},
- {0x2C06, "Persistent prevent conflict"},
- {0x2C07, "Previous busy status"},
- {0x2C08, "Previous task set full status"},
- {0x2C09, "Previous reservation conflict status"},
- {0x2C0A, "Partition or collection contains user objects"},
- {0x2C0B, "Not reserved"},
- {0x2C0C, "Orwrite generation does not match"},
- {0x2C0D, "Reset write pointer not allowed"},
- {0x2C0E, "Zone is offline"},
-
- {0x2D00, "Overwrite error on update in place"},
-
- {0x2E00, "Insufficient time for operation"},
- {0x2E01, "Command timeout before processing"},
- {0x2E02, "Command timeout during processing"},
- {0x2E03, "Command timeout during processing due to error recovery"},
-
- {0x2F00, "Commands cleared by another initiator"},
- {0x2F01, "Commands cleared by power loss notification"},
- {0x2F02, "Commands cleared by device server"},
- {0x2F03, "Some commands cleared by queuing layer event"},
-
- {0x3000, "Incompatible medium installed"},
- {0x3001, "Cannot read medium - unknown format"},
- {0x3002, "Cannot read medium - incompatible format"},
- {0x3003, "Cleaning cartridge installed"},
- {0x3004, "Cannot write medium - unknown format"},
- {0x3005, "Cannot write medium - incompatible format"},
- {0x3006, "Cannot format medium - incompatible medium"},
- {0x3007, "Cleaning failure"},
- {0x3008, "Cannot write - application code mismatch"},
- {0x3009, "Current session not fixated for append"},
- {0x300A, "Cleaning request rejected"},
- {0x300C, "WORM medium - overwrite attempted"},
- {0x300D, "WORM medium - integrity check"},
- {0x3010, "Medium not formatted"},
- {0x3011, "Incompatible volume type"},
- {0x3012, "Incompatible volume qualifier"},
- {0x3013, "Cleaning volume expired"},
-
- {0x3100, "Medium format corrupted"},
- {0x3101, "Format command failed"},
- {0x3102, "Zoned formatting failed due to spare linking"},
- {0x3103, "Sanitize command failed"},
-
- {0x3200, "No defect spare location available"},
- {0x3201, "Defect list update failure"},
-
- {0x3300, "Tape length error"},
-
- {0x3400, "Enclosure failure"},
-
- {0x3500, "Enclosure services failure"},
- {0x3501, "Unsupported enclosure function"},
- {0x3502, "Enclosure services unavailable"},
- {0x3503, "Enclosure services transfer failure"},
- {0x3504, "Enclosure services transfer refused"},
- {0x3505, "Enclosure services checksum error"},
-
- {0x3600, "Ribbon, ink, or toner failure"},
-
- {0x3700, "Rounded parameter"},
-
- {0x3800, "Event status notification"},
- {0x3802, "Esn - power management class event"},
- {0x3804, "Esn - media class event"},
- {0x3806, "Esn - device busy class event"},
- {0x3807, "Thin Provisioning soft threshold reached"},
-
- {0x3900, "Saving parameters not supported"},
-
- {0x3A00, "Medium not present"},
- {0x3A01, "Medium not present - tray closed"},
- {0x3A02, "Medium not present - tray open"},
- {0x3A03, "Medium not present - loadable"},
- {0x3A04, "Medium not present - medium auxiliary memory accessible"},
-
- {0x3B00, "Sequential positioning error"},
- {0x3B01, "Tape position error at beginning-of-medium"},
- {0x3B02, "Tape position error at end-of-medium"},
- {0x3B03, "Tape or electronic vertical forms unit not ready"},
- {0x3B04, "Slew failure"},
- {0x3B05, "Paper jam"},
- {0x3B06, "Failed to sense top-of-form"},
- {0x3B07, "Failed to sense bottom-of-form"},
- {0x3B08, "Reposition error"},
- {0x3B09, "Read past end of medium"},
- {0x3B0A, "Read past beginning of medium"},
- {0x3B0B, "Position past end of medium"},
- {0x3B0C, "Position past beginning of medium"},
- {0x3B0D, "Medium destination element full"},
- {0x3B0E, "Medium source element empty"},
- {0x3B0F, "End of medium reached"},
- {0x3B11, "Medium magazine not accessible"},
- {0x3B12, "Medium magazine removed"},
- {0x3B13, "Medium magazine inserted"},
- {0x3B14, "Medium magazine locked"},
- {0x3B15, "Medium magazine unlocked"},
- {0x3B16, "Mechanical positioning or changer error"},
- {0x3B17, "Read past end of user object"},
- {0x3B18, "Element disabled"},
- {0x3B19, "Element enabled"},
- {0x3B1A, "Data transfer device removed"},
- {0x3B1B, "Data transfer device inserted"},
- {0x3B1C, "Too many logical objects on partition to support "
- "operation"},
-
- {0x3D00, "Invalid bits in identify message"},
-
- {0x3E00, "Logical unit has not self-configured yet"},
- {0x3E01, "Logical unit failure"},
- {0x3E02, "Timeout on logical unit"},
- {0x3E03, "Logical unit failed self-test"},
- {0x3E04, "Logical unit unable to update self-test log"},
-
- {0x3F00, "Target operating conditions have changed"},
- {0x3F01, "Microcode has been changed"},
- {0x3F02, "Changed operating definition"},
- {0x3F03, "Inquiry data has changed"},
- {0x3F04, "Component device attached"},
- {0x3F05, "Device identifier changed"},
- {0x3F06, "Redundancy group created or modified"},
- {0x3F07, "Redundancy group deleted"},
- {0x3F08, "Spare created or modified"},
- {0x3F09, "Spare deleted"},
- {0x3F0A, "Volume set created or modified"},
- {0x3F0B, "Volume set deleted"},
- {0x3F0C, "Volume set deassigned"},
- {0x3F0D, "Volume set reassigned"},
- {0x3F0E, "Reported luns data has changed"},
- {0x3F0F, "Echo buffer overwritten"},
- {0x3F10, "Medium loadable"},
- {0x3F11, "Medium auxiliary memory accessible"},
- {0x3F12, "iSCSI IP address added"},
- {0x3F13, "iSCSI IP address removed"},
- {0x3F14, "iSCSI IP address changed"},
- {0x3F15, "Inspect referrals sense descriptors"},
- {0x3F16, "Microcode has been changed without reset"},
-/*
- * {0x40NN, "Ram failure"},
- * {0x40NN, "Diagnostic failure on component nn"},
- * {0x41NN, "Data path failure"},
- * {0x42NN, "Power-on or self-test failure"},
- */
- {0x4300, "Message error"},
-
- {0x4400, "Internal target failure"},
- {0x4401, "Persistent reservation information lost"},
- {0x4471, "ATA device failed set features"},
-
- {0x4500, "Select or reselect failure"},
-
- {0x4600, "Unsuccessful soft reset"},
-
- {0x4700, "Scsi parity error"},
- {0x4701, "Data phase CRC error detected"},
- {0x4702, "Scsi parity error detected during st data phase"},
- {0x4703, "Information unit iuCRC error detected"},
- {0x4704, "Asynchronous information protection error detected"},
- {0x4705, "Protocol service CRC error"},
- {0x4706, "Phy test function in progress"},
- {0x477f, "Some commands cleared by iSCSI Protocol event"},
-
- {0x4800, "Initiator detected error message received"},
-
- {0x4900, "Invalid message error"},
-
- {0x4A00, "Command phase error"},
-
- {0x4B00, "Data phase error"},
- {0x4B01, "Invalid target port transfer tag received"},
- {0x4B02, "Too much write data"},
- {0x4B03, "Ack/nak timeout"},
- {0x4B04, "Nak received"},
- {0x4B05, "Data offset error"},
- {0x4B06, "Initiator response timeout"},
- {0x4B07, "Connection lost"},
- {0x4B08, "Data-in buffer overflow - data buffer size"},
- {0x4B09, "Data-in buffer overflow - data buffer descriptor area"},
- {0x4B0A, "Data-in buffer error"},
- {0x4B0B, "Data-out buffer overflow - data buffer size"},
- {0x4B0C, "Data-out buffer overflow - data buffer descriptor area"},
- {0x4B0D, "Data-out buffer error"},
- {0x4B0E, "PCIe fabric error"},
- {0x4B0F, "PCIe completion timeout"},
- {0x4B10, "PCIe completer abort"},
- {0x4B11, "PCIe poisoned tlp received"},
- {0x4B12, "PCIe eCRC check failed"},
- {0x4B13, "PCIe unsupported request"},
- {0x4B14, "PCIe acs violation"},
- {0x4B15, "PCIe tlp prefix blocked"},
-
- {0x4C00, "Logical unit failed self-configuration"},
-/*
- * {0x4DNN, "Tagged overlapped commands (nn = queue tag)"},
- */
- {0x4E00, "Overlapped commands attempted"},
-
- {0x5000, "Write append error"},
- {0x5001, "Write append position error"},
- {0x5002, "Position error related to timing"},
-
- {0x5100, "Erase failure"},
- {0x5101, "Erase failure - incomplete erase operation detected"},
-
- {0x5200, "Cartridge fault"},
-
- {0x5300, "Media load or eject failed"},
- {0x5301, "Unload tape failure"},
- {0x5302, "Medium removal prevented"},
- {0x5303, "Medium removal prevented by data transfer element"},
- {0x5304, "Medium thread or unthread failure"},
- {0x5305, "Volume identifier invalid"},
- {0x5306, "Volume identifier missing"},
- {0x5307, "Duplicate volume identifier"},
- {0x5308, "Element status unknown"},
- {0x5309, "Data transfer device error - load failed"},
- {0x530a, "Data transfer device error - unload failed"},
- {0x530b, "Data transfer device error - unload missing"},
- {0x530c, "Data transfer device error - eject failed"},
- {0x530d, "Data transfer device error - library communication failed"},
-
- {0x5400, "Scsi to host system interface failure"},
-
- {0x5500, "System resource failure"},
- {0x5501, "System buffer full"},
- {0x5502, "Insufficient reservation resources"},
- {0x5503, "Insufficient resources"},
- {0x5504, "Insufficient registration resources"},
- {0x5505, "Insufficient access control resources"},
- {0x5506, "Auxiliary memory out of space"},
- {0x5507, "Quota error"},
- {0x5508, "Maximum number of supplemental decryption keys exceeded"},
- {0x5509, "Medium auxiliary memory not accessible"},
- {0x550A, "Data currently unavailable"},
- {0x550B, "Insufficient power for operation"},
- {0x550C, "Insufficient resources to create rod"},
- {0x550D, "Insufficient resources to create rod token"},
- {0x550E, "Insufficient zone resources"},
-
- {0x5700, "Unable to recover table-of-contents"},
-
- {0x5800, "Generation does not exist"},
-
- {0x5900, "Updated block read"},
-
- {0x5A00, "Operator request or state change input"},
- {0x5A01, "Operator medium removal request"},
- {0x5A02, "Operator selected write protect"},
- {0x5A03, "Operator selected write permit"},
-
- {0x5B00, "Log exception"},
- {0x5B01, "Threshold condition met"},
- {0x5B02, "Log counter at maximum"},
- {0x5B03, "Log list codes exhausted"},
-
- {0x5C00, "Rpl status change"},
- {0x5C01, "Spindles synchronized"},
- {0x5C02, "Spindles not synchronized"},
-
- {0x5D00, "Failure prediction threshold exceeded"},
- {0x5D01, "Media failure prediction threshold exceeded"},
- {0x5D02, "Logical unit failure prediction threshold exceeded"},
- {0x5D03, "Spare area exhaustion prediction threshold exceeded"},
- {0x5D10, "Hardware impending failure general hard drive failure"},
- {0x5D11, "Hardware impending failure drive error rate too high"},
- {0x5D12, "Hardware impending failure data error rate too high"},
- {0x5D13, "Hardware impending failure seek error rate too high"},
- {0x5D14, "Hardware impending failure too many block reassigns"},
- {0x5D15, "Hardware impending failure access times too high"},
- {0x5D16, "Hardware impending failure start unit times too high"},
- {0x5D17, "Hardware impending failure channel parametrics"},
- {0x5D18, "Hardware impending failure controller detected"},
- {0x5D19, "Hardware impending failure throughput performance"},
- {0x5D1A, "Hardware impending failure seek time performance"},
- {0x5D1B, "Hardware impending failure spin-up retry count"},
- {0x5D1C, "Hardware impending failure drive calibration retry count"},
- {0x5D20, "Controller impending failure general hard drive failure"},
- {0x5D21, "Controller impending failure drive error rate too high"},
- {0x5D22, "Controller impending failure data error rate too high"},
- {0x5D23, "Controller impending failure seek error rate too high"},
- {0x5D24, "Controller impending failure too many block reassigns"},
- {0x5D25, "Controller impending failure access times too high"},
- {0x5D26, "Controller impending failure start unit times too high"},
- {0x5D27, "Controller impending failure channel parametrics"},
- {0x5D28, "Controller impending failure controller detected"},
- {0x5D29, "Controller impending failure throughput performance"},
- {0x5D2A, "Controller impending failure seek time performance"},
- {0x5D2B, "Controller impending failure spin-up retry count"},
- {0x5D2C, "Controller impending failure drive calibration retry count"},
- {0x5D30, "Data channel impending failure general hard drive failure"},
- {0x5D31, "Data channel impending failure drive error rate too high"},
- {0x5D32, "Data channel impending failure data error rate too high"},
- {0x5D33, "Data channel impending failure seek error rate too high"},
- {0x5D34, "Data channel impending failure too many block reassigns"},
- {0x5D35, "Data channel impending failure access times too high"},
- {0x5D36, "Data channel impending failure start unit times too high"},
- {0x5D37, "Data channel impending failure channel parametrics"},
- {0x5D38, "Data channel impending failure controller detected"},
- {0x5D39, "Data channel impending failure throughput performance"},
- {0x5D3A, "Data channel impending failure seek time performance"},
- {0x5D3B, "Data channel impending failure spin-up retry count"},
- {0x5D3C, "Data channel impending failure drive calibration retry "
- "count"},
- {0x5D40, "Servo impending failure general hard drive failure"},
- {0x5D41, "Servo impending failure drive error rate too high"},
- {0x5D42, "Servo impending failure data error rate too high"},
- {0x5D43, "Servo impending failure seek error rate too high"},
- {0x5D44, "Servo impending failure too many block reassigns"},
- {0x5D45, "Servo impending failure access times too high"},
- {0x5D46, "Servo impending failure start unit times too high"},
- {0x5D47, "Servo impending failure channel parametrics"},
- {0x5D48, "Servo impending failure controller detected"},
- {0x5D49, "Servo impending failure throughput performance"},
- {0x5D4A, "Servo impending failure seek time performance"},
- {0x5D4B, "Servo impending failure spin-up retry count"},
- {0x5D4C, "Servo impending failure drive calibration retry count"},
- {0x5D50, "Spindle impending failure general hard drive failure"},
- {0x5D51, "Spindle impending failure drive error rate too high"},
- {0x5D52, "Spindle impending failure data error rate too high"},
- {0x5D53, "Spindle impending failure seek error rate too high"},
- {0x5D54, "Spindle impending failure too many block reassigns"},
- {0x5D55, "Spindle impending failure access times too high"},
- {0x5D56, "Spindle impending failure start unit times too high"},
- {0x5D57, "Spindle impending failure channel parametrics"},
- {0x5D58, "Spindle impending failure controller detected"},
- {0x5D59, "Spindle impending failure throughput performance"},
- {0x5D5A, "Spindle impending failure seek time performance"},
- {0x5D5B, "Spindle impending failure spin-up retry count"},
- {0x5D5C, "Spindle impending failure drive calibration retry count"},
- {0x5D60, "Firmware impending failure general hard drive failure"},
- {0x5D61, "Firmware impending failure drive error rate too high"},
- {0x5D62, "Firmware impending failure data error rate too high"},
- {0x5D63, "Firmware impending failure seek error rate too high"},
- {0x5D64, "Firmware impending failure too many block reassigns"},
- {0x5D65, "Firmware impending failure access times too high"},
- {0x5D66, "Firmware impending failure start unit times too high"},
- {0x5D67, "Firmware impending failure channel parametrics"},
- {0x5D68, "Firmware impending failure controller detected"},
- {0x5D69, "Firmware impending failure throughput performance"},
- {0x5D6A, "Firmware impending failure seek time performance"},
- {0x5D6B, "Firmware impending failure spin-up retry count"},
- {0x5D6C, "Firmware impending failure drive calibration retry count"},
- {0x5DFF, "Failure prediction threshold exceeded (false)"},
-
- {0x5E00, "Low power condition on"},
- {0x5E01, "Idle condition activated by timer"},
- {0x5E02, "Standby condition activated by timer"},
- {0x5E03, "Idle condition activated by command"},
- {0x5E04, "Standby condition activated by command"},
- {0x5E05, "Idle_b condition activated by timer"},
- {0x5E06, "Idle_b condition activated by command"},
- {0x5E07, "Idle_c condition activated by timer"},
- {0x5E08, "Idle_c condition activated by command"},
- {0x5E09, "Standby_y condition activated by timer"},
- {0x5E0A, "Standby_y condition activated by command"},
- {0x5E41, "Power state change to active"},
- {0x5E42, "Power state change to idle"},
- {0x5E43, "Power state change to standby"},
- {0x5E45, "Power state change to sleep"},
- {0x5E47, "Power state change to device control"},
-
- {0x6000, "Lamp failure"},
-
- {0x6100, "Video acquisition error"},
- {0x6101, "Unable to acquire video"},
- {0x6102, "Out of focus"},
-
- {0x6200, "Scan head positioning error"},
-
- {0x6300, "End of user area encountered on this track"},
- {0x6301, "Packet does not fit in available space"},
-
- {0x6400, "Illegal mode for this track"},
- {0x6401, "Invalid packet size"},
-
- {0x6500, "Voltage fault"},
-
- {0x6600, "Automatic document feeder cover up"},
- {0x6601, "Automatic document feeder lift up"},
- {0x6602, "Document jam in automatic document feeder"},
- {0x6603, "Document miss feed automatic in document feeder"},
-
- {0x6700, "Configuration failure"},
- {0x6701, "Configuration of incapable logical units failed"},
- {0x6702, "Add logical unit failed"},
- {0x6703, "Modification of logical unit failed"},
- {0x6704, "Exchange of logical unit failed"},
- {0x6705, "Remove of logical unit failed"},
- {0x6706, "Attachment of logical unit failed"},
- {0x6707, "Creation of logical unit failed"},
- {0x6708, "Assign failure occurred"},
- {0x6709, "Multiply assigned logical unit"},
- {0x670A, "Set target port groups command failed"},
- {0x670B, "ATA device feature not enabled"},
-
- {0x6800, "Logical unit not configured"},
- {0x6801, "Subsidiary logical unit not configured"},
-
- {0x6900, "Data loss on logical unit"},
- {0x6901, "Multiple logical unit failures"},
- {0x6902, "Parity/data mismatch"},
-
- {0x6A00, "Informational, refer to log"},
-
- {0x6B00, "State change has occurred"},
- {0x6B01, "Redundancy level got better"},
- {0x6B02, "Redundancy level got worse"},
-
- {0x6C00, "Rebuild failure occurred"},
-
- {0x6D00, "Recalculate failure occurred"},
-
- {0x6E00, "Command to logical unit failed"},
-
- {0x6F00, "Copy protection key exchange failure - authentication "
- "failure"},
- {0x6F01, "Copy protection key exchange failure - key not present"},
- {0x6F02, "Copy protection key exchange failure - key not established"},
- {0x6F03, "Read of scrambled sector without authentication"},
- {0x6F04, "Media region code is mismatched to logical unit region"},
- {0x6F05, "Drive region must be permanent/region reset count error"},
- {0x6F06, "Insufficient block count for binding nonce recording"},
- {0x6F07, "Conflict in binding nonce recording"},
-/*
- * {0x70NN, "Decompression exception short algorithm id of nn"},
- */
- {0x7100, "Decompression exception long algorithm id"},
-
- {0x7200, "Session fixation error"},
- {0x7201, "Session fixation error writing lead-in"},
- {0x7202, "Session fixation error writing lead-out"},
- {0x7203, "Session fixation error - incomplete track in session"},
- {0x7204, "Empty or partially written reserved track"},
- {0x7205, "No more track reservations allowed"},
- {0x7206, "RMZ extension is not allowed"},
- {0x7207, "No more test zone extensions are allowed"},
-
- {0x7300, "Cd control error"},
- {0x7301, "Power calibration area almost full"},
- {0x7302, "Power calibration area is full"},
- {0x7303, "Power calibration area error"},
- {0x7304, "Program memory area update failure"},
- {0x7305, "Program memory area is full"},
- {0x7306, "RMA/PMA is almost full"},
- {0x7310, "Current power calibration area almost full"},
- {0x7311, "Current power calibration area is full"},
- {0x7317, "RDZ is full"},
-
- {0x7400, "Security error"},
- {0x7401, "Unable to decrypt data"},
- {0x7402, "Unencrypted data encountered while decrypting"},
- {0x7403, "Incorrect data encryption key"},
- {0x7404, "Cryptographic integrity validation failed"},
- {0x7405, "Error decrypting data"},
- {0x7406, "Unknown signature verification key"},
- {0x7407, "Encryption parameters not useable"},
- {0x7408, "Digital signature validation failure"},
- {0x7409, "Encryption mode mismatch on read"},
- {0x740A, "Encrypted block not raw read enabled"},
- {0x740B, "Incorrect Encryption parameters"},
- {0x740C, "Unable to decrypt parameter list"},
- {0x740D, "Encryption algorithm disabled"},
- {0x7410, "SA creation parameter value invalid"},
- {0x7411, "SA creation parameter value rejected"},
- {0x7412, "Invalid SA usage"},
- {0x7421, "Data Encryption configuration prevented"},
- {0x7430, "SA creation parameter not supported"},
- {0x7440, "Authentication failed"},
- {0x7461, "External data encryption key manager access error"},
- {0x7462, "External data encryption key manager error"},
- {0x7463, "External data encryption key not found"},
- {0x7464, "External data encryption request not authorized"},
- {0x746E, "External data encryption control timeout"},
- {0x746F, "External data encryption control error"},
- {0x7471, "Logical unit access not authorized"},
- {0x7479, "Security conflict in translated device"},
-
- {0, NULL}
+#define SENSE_CODE(c, s) {c, sizeof(s)},
+#include "sense_codes.h"
+#undef SENSE_CODE
};
+static const char *additional_text =
+#define SENSE_CODE(c, s) s "\0"
+#include "sense_codes.h"
+#undef SENSE_CODE
+ ;
+
struct error_info2 {
unsigned char code1, code2_min, code2_max;
const char * str;
@@ -1197,11 +377,14 @@
{
int i;
unsigned short code = ((asc << 8) | ascq);
+ unsigned offset = 0;
*fmt = NULL;
- for (i = 0; additional[i].text; i++)
+ for (i = 0; i < ARRAY_SIZE(additional); i++) {
if (additional[i].code12 == code)
- return additional[i].text;
+ return additional_text + offset;
+ offset += additional[i].size;
+ }
for (i = 0; additional2[i].fmt; i++) {
if (additional2[i].code1 == asc &&
ascq >= additional2[i].code2_min &&
diff --git a/drivers/scsi/cxlflash/superpipe.c b/drivers/scsi/cxlflash/superpipe.c
index d8a5cb3..ce15070 100644
--- a/drivers/scsi/cxlflash/superpipe.c
+++ b/drivers/scsi/cxlflash/superpipe.c
@@ -1615,6 +1615,13 @@
* place at the same time and the failure was due to CXL services being
* unable to keep up.
*
+ * As this routine is called on ioctl context, it holds the ioctl r/w
+ * semaphore that is used to drain ioctls in recovery scenarios. The
+ * implementation to achieve the pacing described above (a local mutex)
+ * requires that the ioctl r/w semaphore be dropped and reacquired to
+ * avoid a 3-way deadlock when multiple process recoveries operate in
+ * parallel.
+ *
* Because a user can detect an error condition before the kernel, it is
* quite possible for this routine to act as the kernel's EEH detection
* source (MMIO read of mbox_r). Because of this, there is a window of
@@ -1642,9 +1649,17 @@
int rc = 0;
atomic_inc(&cfg->recovery_threads);
+ up_read(&cfg->ioctl_rwsem);
rc = mutex_lock_interruptible(mutex);
+ down_read(&cfg->ioctl_rwsem);
if (rc)
goto out;
+ rc = check_state(cfg);
+ if (rc) {
+ dev_err(dev, "%s: Failed state! rc=%d\n", __func__, rc);
+ rc = -ENODEV;
+ goto out;
+ }
dev_dbg(dev, "%s: reason 0x%016llX rctxid=%016llX\n",
__func__, recover->reason, rctxid);
diff --git a/drivers/scsi/device_handler/scsi_dh_alua.c b/drivers/scsi/device_handler/scsi_dh_alua.c
index a655cf2..752b5c9 100644
--- a/drivers/scsi/device_handler/scsi_dh_alua.c
+++ b/drivers/scsi/device_handler/scsi_dh_alua.c
@@ -190,15 +190,18 @@
ALUA_FAILOVER_RETRIES, NULL, req_flags);
}
-struct alua_port_group *alua_find_get_pg(char *id_str, size_t id_size,
- int group_id)
+static struct alua_port_group *alua_find_get_pg(char *id_str, size_t id_size,
+ int group_id)
{
struct alua_port_group *pg;
+ if (!id_str || !id_size || !strlen(id_str))
+ return NULL;
+
list_for_each_entry(pg, &port_group_list, node) {
if (pg->group_id != group_id)
continue;
- if (pg->device_id_len != id_size)
+ if (!pg->device_id_len || pg->device_id_len != id_size)
continue;
if (strncmp(pg->device_id_str, id_str, id_size))
continue;
@@ -219,8 +222,8 @@
* Allocate a new port_group structure for a given
* device.
*/
-struct alua_port_group *alua_alloc_pg(struct scsi_device *sdev,
- int group_id, int tpgs)
+static struct alua_port_group *alua_alloc_pg(struct scsi_device *sdev,
+ int group_id, int tpgs)
{
struct alua_port_group *pg, *tmp_pg;
@@ -232,14 +235,14 @@
sizeof(pg->device_id_str));
if (pg->device_id_len <= 0) {
/*
- * Internal error: TPGS supported but no device
- * identifcation found. Disable ALUA support.
+ * TPGS supported but no device identification found.
+ * Generate private device identification.
*/
- kfree(pg);
sdev_printk(KERN_INFO, sdev,
"%s: No device descriptors found\n",
ALUA_DH_NAME);
- return ERR_PTR(-ENXIO);
+ pg->device_id_str[0] = '\0';
+ pg->device_id_len = 0;
}
pg->group_id = group_id;
pg->tpgs = tpgs;
@@ -354,9 +357,15 @@
return SCSI_DH_NOMEM;
return SCSI_DH_DEV_UNSUPP;
}
- sdev_printk(KERN_INFO, sdev,
- "%s: device %s port group %x rel port %x\n",
- ALUA_DH_NAME, pg->device_id_str, group_id, rel_port);
+ if (pg->device_id_len)
+ sdev_printk(KERN_INFO, sdev,
+ "%s: device %s port group %x rel port %x\n",
+ ALUA_DH_NAME, pg->device_id_str,
+ group_id, rel_port);
+ else
+ sdev_printk(KERN_INFO, sdev,
+ "%s: port group %x rel port %x\n",
+ ALUA_DH_NAME, group_id, rel_port);
/* Check for existing port group references */
spin_lock(&h->pg_lock);
diff --git a/drivers/scsi/dmx3191d.c b/drivers/scsi/dmx3191d.c
index 6c14e68b..9b5a457 100644
--- a/drivers/scsi/dmx3191d.c
+++ b/drivers/scsi/dmx3191d.c
@@ -34,11 +34,14 @@
* Definitions for the generic 5380 driver.
*/
-#define DONT_USE_INTR
-
#define NCR5380_read(reg) inb(instance->io_port + reg)
#define NCR5380_write(reg, value) outb(value, instance->io_port + reg)
+#define NCR5380_dma_xfer_len(instance, cmd, phase) (0)
+#define NCR5380_dma_recv_setup(instance, dst, len) (0)
+#define NCR5380_dma_send_setup(instance, src, len) (0)
+#define NCR5380_dma_residual(instance) (0)
+
#define NCR5380_implementation_fields /* none */
#include "NCR5380.h"
@@ -62,7 +65,6 @@
.cmd_per_lun = 2,
.use_clustering = DISABLE_CLUSTERING,
.cmd_size = NCR5380_CMD_SIZE,
- .max_sectors = 128,
};
static int dmx3191d_probe_one(struct pci_dev *pdev,
@@ -93,7 +95,7 @@
*/
shost->irq = NO_IRQ;
- error = NCR5380_init(shost, FLAG_NO_PSEUDO_DMA);
+ error = NCR5380_init(shost, 0);
if (error)
goto out_host_put;
diff --git a/drivers/scsi/dtc.c b/drivers/scsi/dtc.c
index 6c736b0..459863f 100644
--- a/drivers/scsi/dtc.c
+++ b/drivers/scsi/dtc.c
@@ -1,6 +1,3 @@
-#define PSEUDO_DMA
-#define DONT_USE_INTR
-
/*
* DTC 3180/3280 driver, by
* Ray Van Tassle rayvt@comm.mot.com
@@ -54,7 +51,6 @@
#include <scsi/scsi_host.h>
#include "dtc.h"
-#define AUTOPROBE_IRQ
#include "NCR5380.h"
/*
@@ -229,7 +225,7 @@
instance->base = addr;
((struct NCR5380_hostdata *)(instance)->hostdata)->base = base;
- if (NCR5380_init(instance, FLAG_NO_DMA_FIXUP))
+ if (NCR5380_init(instance, FLAG_LATE_DMA_SETUP))
goto out_unregister;
NCR5380_maybe_reset_bus(instance);
@@ -244,9 +240,10 @@
if (instance->irq == 255)
instance->irq = NO_IRQ;
-#ifndef DONT_USE_INTR
/* With interrupts enabled, it will sometimes hang when doing heavy
* reads. So better not enable them until I finger it out. */
+ instance->irq = NO_IRQ;
+
if (instance->irq != NO_IRQ)
if (request_irq(instance->irq, dtc_intr, 0,
"dtc", instance)) {
@@ -258,11 +255,7 @@
printk(KERN_WARNING "scsi%d : interrupts not enabled. for better interactive performance,\n", instance->host_no);
printk(KERN_WARNING "scsi%d : please jumper the board for a free IRQ.\n", instance->host_no);
}
-#else
- if (instance->irq != NO_IRQ)
- printk(KERN_WARNING "scsi%d : interrupts not used. Might as well not jumper it.\n", instance->host_no);
- instance->irq = NO_IRQ;
-#endif
+
dprintk(NDEBUG_INIT, "scsi%d : irq = %d\n",
instance->host_no, instance->irq);
@@ -323,7 +316,8 @@
* timeout.
*/
-static inline int NCR5380_pread(struct Scsi_Host *instance, unsigned char *dst, int len)
+static inline int dtc_pread(struct Scsi_Host *instance,
+ unsigned char *dst, int len)
{
unsigned char *d = dst;
int i; /* For counting time spent in the poll-loop */
@@ -352,8 +346,6 @@
while (!(NCR5380_read(DTC_CONTROL_REG) & D_CR_ACCESS))
++i;
rtrc(0);
- if (i > hostdata->spin_max_r)
- hostdata->spin_max_r = i;
return (0);
}
@@ -370,7 +362,8 @@
* timeout.
*/
-static inline int NCR5380_pwrite(struct Scsi_Host *instance, unsigned char *src, int len)
+static inline int dtc_pwrite(struct Scsi_Host *instance,
+ unsigned char *src, int len)
{
int i;
struct NCR5380_hostdata *hostdata = shost_priv(instance);
@@ -400,8 +393,6 @@
rtrc(7);
/* Check for parity error here. fixme. */
rtrc(0);
- if (i > hostdata->spin_max_w)
- hostdata->spin_max_w = i;
return (0);
}
@@ -440,8 +431,6 @@
.detect = dtc_detect,
.release = dtc_release,
.proc_name = "dtc3x80",
- .show_info = dtc_show_info,
- .write_info = dtc_write_info,
.info = dtc_info,
.queuecommand = dtc_queue_command,
.eh_abort_handler = dtc_abort,
diff --git a/drivers/scsi/dtc.h b/drivers/scsi/dtc.h
index 56732cb..fcb0a8e 100644
--- a/drivers/scsi/dtc.h
+++ b/drivers/scsi/dtc.h
@@ -21,14 +21,17 @@
#define NCR5380_dma_xfer_len(instance, cmd, phase) \
dtc_dma_xfer_len(cmd)
+#define NCR5380_dma_recv_setup dtc_pread
+#define NCR5380_dma_send_setup dtc_pwrite
+#define NCR5380_dma_residual(instance) (0)
#define NCR5380_intr dtc_intr
#define NCR5380_queue_command dtc_queue_command
#define NCR5380_abort dtc_abort
#define NCR5380_bus_reset dtc_bus_reset
#define NCR5380_info dtc_info
-#define NCR5380_show_info dtc_show_info
-#define NCR5380_write_info dtc_write_info
+
+#define NCR5380_io_delay(x) udelay(x)
/* 15 12 11 10
1001 1100 0000 0000 */
diff --git a/drivers/scsi/eata_pio.c b/drivers/scsi/eata_pio.c
index ca8003f..4299fa4 100644
--- a/drivers/scsi/eata_pio.c
+++ b/drivers/scsi/eata_pio.c
@@ -729,6 +729,7 @@
break;
case 0x24:
SD(sh)->EATA_revision = 'z';
+ break;
default:
SD(sh)->EATA_revision = '?';
}
diff --git a/drivers/scsi/esas2r/esas2r_main.c b/drivers/scsi/esas2r/esas2r_main.c
index 33581ba..2aca4d1 100644
--- a/drivers/scsi/esas2r/esas2r_main.c
+++ b/drivers/scsi/esas2r/esas2r_main.c
@@ -246,7 +246,7 @@
.eh_target_reset_handler = esas2r_target_reset,
.can_queue = 128,
.this_id = -1,
- .sg_tablesize = SCSI_MAX_SG_SEGMENTS,
+ .sg_tablesize = SG_CHUNK_SIZE,
.cmd_per_lun =
ESAS2R_DEFAULT_CMD_PER_LUN,
.present = 0,
@@ -271,7 +271,7 @@
MODULE_PARM_DESC(num_sg_lists,
"Number of scatter/gather lists. Default 1024.");
-int sg_tablesize = SCSI_MAX_SG_SEGMENTS;
+int sg_tablesize = SG_CHUNK_SIZE;
module_param(sg_tablesize, int, 0);
MODULE_PARM_DESC(sg_tablesize,
"Maximum number of entries in a scatter/gather table.");
diff --git a/drivers/scsi/fnic/fnic.h b/drivers/scsi/fnic/fnic.h
index ce129e5..9ddc920 100644
--- a/drivers/scsi/fnic/fnic.h
+++ b/drivers/scsi/fnic/fnic.h
@@ -39,7 +39,7 @@
#define DRV_NAME "fnic"
#define DRV_DESCRIPTION "Cisco FCoE HBA Driver"
-#define DRV_VERSION "1.6.0.17a"
+#define DRV_VERSION "1.6.0.21"
#define PFX DRV_NAME ": "
#define DFX DRV_NAME "%d: "
diff --git a/drivers/scsi/fnic/fnic_scsi.c b/drivers/scsi/fnic/fnic_scsi.c
index f3032ca..d9fd2f8 100644
--- a/drivers/scsi/fnic/fnic_scsi.c
+++ b/drivers/scsi/fnic/fnic_scsi.c
@@ -439,7 +439,6 @@
int sg_count = 0;
unsigned long flags = 0;
unsigned long ptr;
- struct fc_rport_priv *rdata;
spinlock_t *io_lock = NULL;
int io_lock_acquired = 0;
@@ -455,14 +454,17 @@
return 0;
}
- rdata = lp->tt.rport_lookup(lp, rport->port_id);
- if (!rdata || (rdata->rp_state == RPORT_ST_DELETE)) {
- FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
- "returning IO as rport is removed\n");
- atomic64_inc(&fnic_stats->misc_stats.rport_not_ready);
- sc->result = DID_NO_CONNECT;
- done(sc);
- return 0;
+ if (rport) {
+ struct fc_rport_libfc_priv *rp = rport->dd_data;
+
+ if (!rp || rp->rp_state != RPORT_ST_READY) {
+ FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
+ "returning DID_NO_CONNECT for IO as rport is removed\n");
+ atomic64_inc(&fnic_stats->misc_stats.rport_not_ready);
+ sc->result = DID_NO_CONNECT<<16;
+ done(sc);
+ return 0;
+ }
}
if (lp->state != LPORT_ST_READY || !(lp->link_up))
@@ -1091,6 +1093,11 @@
atomic64_inc(
&term_stats->terminate_fw_timeouts);
break;
+ case FCPIO_ITMF_REJECTED:
+ FNIC_SCSI_DBG(KERN_INFO, fnic->lport->host,
+ "abort reject recd. id %d\n",
+ (int)(id & FNIC_TAG_MASK));
+ break;
case FCPIO_IO_NOT_FOUND:
if (CMD_FLAGS(sc) & FNIC_IO_ABTS_ISSUED)
atomic64_inc(&abts_stats->abort_io_not_found);
@@ -1111,9 +1118,15 @@
spin_unlock_irqrestore(io_lock, flags);
return;
}
- CMD_ABTS_STATUS(sc) = hdr_status;
+
CMD_FLAGS(sc) |= FNIC_IO_ABT_TERM_DONE;
+ /* If the status is IO not found consider it as success */
+ if (hdr_status == FCPIO_IO_NOT_FOUND)
+ CMD_ABTS_STATUS(sc) = FCPIO_SUCCESS;
+ else
+ CMD_ABTS_STATUS(sc) = hdr_status;
+
atomic64_dec(&fnic_stats->io_stats.active_ios);
if (atomic64_read(&fnic->io_cmpl_skip))
atomic64_dec(&fnic->io_cmpl_skip);
@@ -1926,21 +1939,31 @@
CMD_STATE(sc) = FNIC_IOREQ_ABTS_COMPLETE;
+ start_time = io_req->start_time;
/*
* firmware completed the abort, check the status,
- * free the io_req irrespective of failure or success
+ * free the io_req if successful. If abort fails,
+ * Device reset will clean the I/O.
*/
- if (CMD_ABTS_STATUS(sc) != FCPIO_SUCCESS)
+ if (CMD_ABTS_STATUS(sc) == FCPIO_SUCCESS)
+ CMD_SP(sc) = NULL;
+ else {
ret = FAILED;
-
- CMD_SP(sc) = NULL;
+ spin_unlock_irqrestore(io_lock, flags);
+ goto fnic_abort_cmd_end;
+ }
spin_unlock_irqrestore(io_lock, flags);
- start_time = io_req->start_time;
fnic_release_ioreq_buf(fnic, io_req, sc);
mempool_free(io_req, fnic->io_req_pool);
+ if (sc->scsi_done) {
+ /* Call SCSI completion function to complete the IO */
+ sc->result = (DID_ABORT << 16);
+ sc->scsi_done(sc);
+ }
+
fnic_abort_cmd_end:
FNIC_TRACE(fnic_abort_cmd, sc->device->host->host_no,
sc->request->tag, sc,
@@ -2018,7 +2041,9 @@
* successfully aborted, 1 otherwise
*/
static int fnic_clean_pending_aborts(struct fnic *fnic,
- struct scsi_cmnd *lr_sc)
+ struct scsi_cmnd *lr_sc,
+ bool new_sc)
+
{
int tag, abt_tag;
struct fnic_io_req *io_req;
@@ -2036,10 +2061,10 @@
spin_lock_irqsave(io_lock, flags);
sc = scsi_host_find_tag(fnic->lport->host, tag);
/*
- * ignore this lun reset cmd or cmds that do not belong to
- * this lun
+ * ignore this lun reset cmd if issued using new SC
+ * or cmds that do not belong to this lun
*/
- if (!sc || sc == lr_sc || sc->device != lun_dev) {
+ if (!sc || ((sc == lr_sc) && new_sc) || sc->device != lun_dev) {
spin_unlock_irqrestore(io_lock, flags);
continue;
}
@@ -2145,11 +2170,27 @@
goto clean_pending_aborts_end;
}
CMD_STATE(sc) = FNIC_IOREQ_ABTS_COMPLETE;
- CMD_SP(sc) = NULL;
+
+ /* original sc used for lr is handled by dev reset code */
+ if (sc != lr_sc)
+ CMD_SP(sc) = NULL;
spin_unlock_irqrestore(io_lock, flags);
- fnic_release_ioreq_buf(fnic, io_req, sc);
- mempool_free(io_req, fnic->io_req_pool);
+ /* original sc used for lr is handled by dev reset code */
+ if (sc != lr_sc) {
+ fnic_release_ioreq_buf(fnic, io_req, sc);
+ mempool_free(io_req, fnic->io_req_pool);
+ }
+
+ /*
+ * Any IO is returned during reset, it needs to call scsi_done
+ * to return the scsi_cmnd to upper layer.
+ */
+ if (sc->scsi_done) {
+ /* Set result to let upper SCSI layer retry */
+ sc->result = DID_RESET << 16;
+ sc->scsi_done(sc);
+ }
}
schedule_timeout(msecs_to_jiffies(2 * fnic->config.ed_tov));
@@ -2243,6 +2284,7 @@
int tag = 0;
DECLARE_COMPLETION_ONSTACK(tm_done);
int tag_gen_flag = 0; /*to track tags allocated by fnic driver*/
+ bool new_sc = 0;
/* Wait for rport to unblock */
fc_block_scsi_eh(sc);
@@ -2288,13 +2330,12 @@
* fix the way the EH ioctls work for real, but until
* that happens we fail these explicit requests here.
*/
- if (shost_use_blk_mq(sc->device->host))
- goto fnic_device_reset_end;
tag = fnic_scsi_host_start_tag(fnic, sc);
if (unlikely(tag == SCSI_NO_TAG))
goto fnic_device_reset_end;
tag_gen_flag = 1;
+ new_sc = 1;
}
io_lock = fnic_io_lock_hash(fnic, sc);
spin_lock_irqsave(io_lock, flags);
@@ -2429,7 +2470,7 @@
* the lun reset cmd. If all cmds get cleaned, the lun reset
* succeeds
*/
- if (fnic_clean_pending_aborts(fnic, sc)) {
+ if (fnic_clean_pending_aborts(fnic, sc, new_sc)) {
spin_lock_irqsave(io_lock, flags);
io_req = (struct fnic_io_req *)CMD_SP(sc);
FNIC_SCSI_DBG(KERN_DEBUG, fnic->lport->host,
diff --git a/drivers/scsi/g_NCR5380.c b/drivers/scsi/g_NCR5380.c
index 90091e6..516bd6c 100644
--- a/drivers/scsi/g_NCR5380.c
+++ b/drivers/scsi/g_NCR5380.c
@@ -18,50 +18,10 @@
*
* Added ISAPNP support for DTC436 adapters,
* Thomas Sailer, sailer@ife.ee.ethz.ch
- */
-
-/*
- * TODO : flesh out DMA support, find some one actually using this (I have
- * a memory mapped Trantor board that works fine)
- */
-
-/*
- * The card is detected and initialized in one of several ways :
- * 1. With command line overrides - NCR5380=port,irq may be
- * used on the LILO command line to override the defaults.
*
- * 2. With the GENERIC_NCR5380_OVERRIDE compile time define. This is
- * specified as an array of address, irq, dma, board tuples. Ie, for
- * one board at 0x350, IRQ5, no dma, I could say
- * -DGENERIC_NCR5380_OVERRIDE={{0xcc000, 5, DMA_NONE, BOARD_NCR5380}}
- *
- * -1 should be specified for no or DMA interrupt, -2 to autoprobe for an
- * IRQ line if overridden on the command line.
- *
- * 3. When included as a module, with arguments passed on the command line:
- * ncr_irq=xx the interrupt
- * ncr_addr=xx the port or base address (for port or memory
- * mapped, resp.)
- * ncr_dma=xx the DMA
- * ncr_5380=1 to set up for a NCR5380 board
- * ncr_53c400=1 to set up for a NCR53C400 board
- * e.g.
- * modprobe g_NCR5380 ncr_irq=5 ncr_addr=0x350 ncr_5380=1
- * for a port mapped NCR5380 board or
- * modprobe g_NCR5380 ncr_irq=255 ncr_addr=0xc8000 ncr_53c400=1
- * for a memory mapped NCR53C400 board with interrupts disabled.
- *
- * 255 should be specified for no or DMA interrupt, 254 to autoprobe for an
- * IRQ line if overridden on the command line.
- *
+ * See Documentation/scsi/g_NCR5380.txt for more info.
*/
-#define AUTOPROBE_IRQ
-
-#ifdef CONFIG_SCSI_GENERIC_NCR53C400
-#define PSEUDO_DMA
-#endif
-
#include <asm/io.h>
#include <linux/blkdev.h>
#include <linux/module.h>
@@ -270,7 +230,7 @@
#ifndef SCSI_G_NCR5380_MEM
int i;
int port_idx = -1;
- unsigned long region_size = 16;
+ unsigned long region_size;
#endif
static unsigned int __initdata ncr_53c400a_ports[] = {
0x280, 0x290, 0x300, 0x310, 0x330, 0x340, 0x348, 0x350, 0
@@ -290,6 +250,7 @@
#ifdef SCSI_G_NCR5380_MEM
unsigned long base;
void __iomem *iomem;
+ resource_size_t iomem_size;
#endif
if (ncr_irq)
@@ -350,25 +311,17 @@
flags = 0;
switch (overrides[current_override].board) {
case BOARD_NCR5380:
- flags = FLAG_NO_PSEUDO_DMA;
- break;
- case BOARD_NCR53C400:
-#ifdef PSEUDO_DMA
- flags = FLAG_NO_DMA_FIXUP;
-#endif
+ flags = FLAG_NO_PSEUDO_DMA | FLAG_DMA_FIXUP;
break;
case BOARD_NCR53C400A:
- flags = FLAG_NO_DMA_FIXUP;
ports = ncr_53c400a_ports;
magic = ncr_53c400a_magic;
break;
case BOARD_HP_C2502:
- flags = FLAG_NO_DMA_FIXUP;
ports = ncr_53c400a_ports;
magic = hp_c2502_magic;
break;
case BOARD_DTC3181E:
- flags = FLAG_NO_DMA_FIXUP;
ports = dtc_3181e_ports;
magic = ncr_53c400a_magic;
break;
@@ -381,20 +334,22 @@
/* Disable the adapter and look for a free io port */
magic_configure(-1, 0, magic);
+ region_size = 16;
+
if (overrides[current_override].NCR5380_map_name != PORT_AUTO)
for (i = 0; ports[i]; i++) {
- if (!request_region(ports[i], 16, "ncr53c80"))
+ if (!request_region(ports[i], region_size, "ncr53c80"))
continue;
if (overrides[current_override].NCR5380_map_name == ports[i])
break;
- release_region(ports[i], 16);
+ release_region(ports[i], region_size);
} else
for (i = 0; ports[i]; i++) {
- if (!request_region(ports[i], 16, "ncr53c80"))
+ if (!request_region(ports[i], region_size, "ncr53c80"))
continue;
if (inb(ports[i]) == 0xff)
break;
- release_region(ports[i], 16);
+ release_region(ports[i], region_size);
}
if (ports[i]) {
/* At this point we have our region reserved */
@@ -410,17 +365,19 @@
else
{
/* Not a 53C400A style setup - just grab */
- if(!(request_region(overrides[current_override].NCR5380_map_name, NCR5380_region_size, "ncr5380")))
+ region_size = 8;
+ if (!request_region(overrides[current_override].NCR5380_map_name,
+ region_size, "ncr5380"))
continue;
- region_size = NCR5380_region_size;
}
#else
base = overrides[current_override].NCR5380_map_name;
- if (!request_mem_region(base, NCR5380_region_size, "ncr5380"))
+ iomem_size = NCR53C400_region_size;
+ if (!request_mem_region(base, iomem_size, "ncr5380"))
continue;
- iomem = ioremap(base, NCR5380_region_size);
+ iomem = ioremap(base, iomem_size);
if (!iomem) {
- release_mem_region(base, NCR5380_region_size);
+ release_mem_region(base, iomem_size);
continue;
}
#endif
@@ -458,6 +415,7 @@
#else
instance->base = overrides[current_override].NCR5380_map_name;
hostdata->iomem = iomem;
+ hostdata->iomem_size = iomem_size;
switch (overrides[current_override].board) {
case BOARD_NCR53C400:
hostdata->c400_ctl_status = 0x100;
@@ -472,7 +430,7 @@
}
#endif
- if (NCR5380_init(instance, flags))
+ if (NCR5380_init(instance, flags | FLAG_LATE_DMA_SETUP))
goto out_unregister;
switch (overrides[current_override].board) {
@@ -524,7 +482,7 @@
release_region(overrides[current_override].NCR5380_map_name, region_size);
#else
iounmap(iomem);
- release_mem_region(base, NCR5380_region_size);
+ release_mem_region(base, iomem_size);
#endif
return count;
}
@@ -546,45 +504,18 @@
#ifndef SCSI_G_NCR5380_MEM
release_region(instance->io_port, instance->n_io_port);
#else
- iounmap(((struct NCR5380_hostdata *)instance->hostdata)->iomem);
- release_mem_region(instance->base, NCR5380_region_size);
+ {
+ struct NCR5380_hostdata *hostdata = shost_priv(instance);
+
+ iounmap(hostdata->iomem);
+ release_mem_region(instance->base, hostdata->iomem_size);
+ }
#endif
return 0;
}
-#ifdef BIOSPARAM
/**
- * generic_NCR5380_biosparam
- * @disk: disk to compute geometry for
- * @dev: device identifier for this disk
- * @ip: sizes to fill in
- *
- * Generates a BIOS / DOS compatible H-C-S mapping for the specified
- * device / size.
- *
- * XXX Most SCSI boards use this mapping, I could be incorrect. Someone
- * using hard disks on a trantor should verify that this mapping
- * corresponds to that used by the BIOS / ASPI driver by running the linux
- * fdisk program and matching the H_C_S coordinates to what DOS uses.
- *
- * Locks: none
- */
-
-static int
-generic_NCR5380_biosparam(struct scsi_device *sdev, struct block_device *bdev,
- sector_t capacity, int *ip)
-{
- ip[0] = 64;
- ip[1] = 32;
- ip[2] = capacity >> 11;
- return 0;
-}
-#endif
-
-#ifdef PSEUDO_DMA
-
-/**
- * NCR5380_pread - pseudo DMA read
+ * generic_NCR5380_pread - pseudo DMA read
* @instance: adapter to read from
* @dst: buffer to read into
* @len: buffer length
@@ -593,7 +524,8 @@
* controller
*/
-static inline int NCR5380_pread(struct Scsi_Host *instance, unsigned char *dst, int len)
+static inline int generic_NCR5380_pread(struct Scsi_Host *instance,
+ unsigned char *dst, int len)
{
struct NCR5380_hostdata *hostdata = shost_priv(instance);
int blocks = len / 128;
@@ -661,7 +593,7 @@
}
/**
- * NCR5380_write - pseudo DMA write
+ * generic_NCR5380_pwrite - pseudo DMA write
* @instance: adapter to read from
* @dst: buffer to read into
* @len: buffer length
@@ -670,7 +602,8 @@
* controller
*/
-static inline int NCR5380_pwrite(struct Scsi_Host *instance, unsigned char *src, int len)
+static inline int generic_NCR5380_pwrite(struct Scsi_Host *instance,
+ unsigned char *src, int len)
{
struct NCR5380_hostdata *hostdata = shost_priv(instance);
int blocks = len / 128;
@@ -738,10 +671,15 @@
return 0;
}
-static int generic_NCR5380_dma_xfer_len(struct scsi_cmnd *cmd)
+static int generic_NCR5380_dma_xfer_len(struct Scsi_Host *instance,
+ struct scsi_cmnd *cmd)
{
+ struct NCR5380_hostdata *hostdata = shost_priv(instance);
int transfersize = cmd->transfersize;
+ if (hostdata->flags & FLAG_NO_PSEUDO_DMA)
+ return 0;
+
/* Limit transfers to 32K, for xx400 & xx406
* pseudoDMA that transfers in 128 bytes blocks.
*/
@@ -756,8 +694,6 @@
return transfersize;
}
-#endif /* PSEUDO_DMA */
-
/*
* Include the NCR5380 core code that we build our driver around
*/
@@ -773,7 +709,6 @@
.queuecommand = generic_NCR5380_queue_command,
.eh_abort_handler = generic_NCR5380_abort,
.eh_bus_reset_handler = generic_NCR5380_bus_reset,
- .bios_param = NCR5380_BIOSPARAM,
.can_queue = 16,
.this_id = 7,
.sg_tablesize = SG_ALL,
diff --git a/drivers/scsi/g_NCR5380.h b/drivers/scsi/g_NCR5380.h
index 6f3d2ac..5951774 100644
--- a/drivers/scsi/g_NCR5380.h
+++ b/drivers/scsi/g_NCR5380.h
@@ -14,13 +14,6 @@
#ifndef GENERIC_NCR5380_H
#define GENERIC_NCR5380_H
-#ifdef CONFIG_SCSI_GENERIC_NCR53C400
-#define BIOSPARAM
-#define NCR5380_BIOSPARAM generic_NCR5380_biosparam
-#else
-#define NCR5380_BIOSPARAM NULL
-#endif
-
#define __STRVAL(x) #x
#define STRVAL(x) __STRVAL(x)
@@ -30,12 +23,6 @@
#define NCR5380_map_type int
#define NCR5380_map_name port
-#ifdef CONFIG_SCSI_GENERIC_NCR53C400
-#define NCR5380_region_size 16
-#else
-#define NCR5380_region_size 8
-#endif
-
#define NCR5380_read(reg) \
inb(instance->io_port + (reg))
#define NCR5380_write(reg, value) \
@@ -55,7 +42,7 @@
#define NCR5380_map_name base
#define NCR53C400_mem_base 0x3880
#define NCR53C400_host_buffer 0x3900
-#define NCR5380_region_size 0x3a00
+#define NCR53C400_region_size 0x3a00
#define NCR5380_read(reg) \
readb(((struct NCR5380_hostdata *)shost_priv(instance))->iomem + \
@@ -66,6 +53,7 @@
#define NCR5380_implementation_fields \
void __iomem *iomem; \
+ resource_size_t iomem_size; \
int c400_ctl_status; \
int c400_blk_cnt; \
int c400_host_buf;
@@ -73,16 +61,18 @@
#endif
#define NCR5380_dma_xfer_len(instance, cmd, phase) \
- generic_NCR5380_dma_xfer_len(cmd)
+ generic_NCR5380_dma_xfer_len(instance, cmd)
+#define NCR5380_dma_recv_setup generic_NCR5380_pread
+#define NCR5380_dma_send_setup generic_NCR5380_pwrite
+#define NCR5380_dma_residual(instance) (0)
#define NCR5380_intr generic_NCR5380_intr
#define NCR5380_queue_command generic_NCR5380_queue_command
#define NCR5380_abort generic_NCR5380_abort
#define NCR5380_bus_reset generic_NCR5380_bus_reset
-#define NCR5380_pread generic_NCR5380_pread
-#define NCR5380_pwrite generic_NCR5380_pwrite
#define NCR5380_info generic_NCR5380_info
-#define NCR5380_show_info generic_NCR5380_show_info
+
+#define NCR5380_io_delay(x) udelay(x)
#define BOARD_NCR5380 0
#define BOARD_NCR53C400 1
diff --git a/drivers/scsi/hisi_sas/hisi_sas.h b/drivers/scsi/hisi_sas/hisi_sas.h
index 29e89f3..d7cab72 100644
--- a/drivers/scsi/hisi_sas/hisi_sas.h
+++ b/drivers/scsi/hisi_sas/hisi_sas.h
@@ -23,7 +23,7 @@
#include <scsi/sas_ata.h>
#include <scsi/libsas.h>
-#define DRV_VERSION "v1.3"
+#define DRV_VERSION "v1.4"
#define HISI_SAS_MAX_PHYS 9
#define HISI_SAS_MAX_QUEUES 32
@@ -133,6 +133,9 @@
int (*hw_init)(struct hisi_hba *hisi_hba);
void (*setup_itct)(struct hisi_hba *hisi_hba,
struct hisi_sas_device *device);
+ int (*slot_index_alloc)(struct hisi_hba *hisi_hba, int *slot_idx,
+ struct domain_device *device);
+ struct hisi_sas_device *(*alloc_dev)(struct domain_device *device);
void (*sl_notify)(struct hisi_hba *hisi_hba, int phy_no);
int (*get_free_slot)(struct hisi_hba *hisi_hba, int *q, int *s);
void (*start_delivery)(struct hisi_hba *hisi_hba);
@@ -298,7 +301,7 @@
u8 atapi_cdb[ATAPI_CDB_LEN];
};
-#define HISI_SAS_SGE_PAGE_CNT SCSI_MAX_SG_SEGMENTS
+#define HISI_SAS_SGE_PAGE_CNT SG_CHUNK_SIZE
struct hisi_sas_sge_page {
struct hisi_sas_sge sge[HISI_SAS_SGE_PAGE_CNT];
};
diff --git a/drivers/scsi/hisi_sas/hisi_sas_main.c b/drivers/scsi/hisi_sas/hisi_sas_main.c
index 097ab4f..18dd5ea 100644
--- a/drivers/scsi/hisi_sas/hisi_sas_main.c
+++ b/drivers/scsi/hisi_sas/hisi_sas_main.c
@@ -227,7 +227,11 @@
} else
n_elem = task->num_scatter;
- rc = hisi_sas_slot_index_alloc(hisi_hba, &slot_idx);
+ if (hisi_hba->hw->slot_index_alloc)
+ rc = hisi_hba->hw->slot_index_alloc(hisi_hba, &slot_idx,
+ device);
+ else
+ rc = hisi_sas_slot_index_alloc(hisi_hba, &slot_idx);
if (rc)
goto err_out;
rc = hisi_hba->hw->get_free_slot(hisi_hba, &dlvry_queue,
@@ -417,7 +421,10 @@
struct hisi_sas_device *sas_dev;
struct device *dev = &hisi_hba->pdev->dev;
- sas_dev = hisi_sas_alloc_dev(device);
+ if (hisi_hba->hw->alloc_dev)
+ sas_dev = hisi_hba->hw->alloc_dev(device);
+ else
+ sas_dev = hisi_sas_alloc_dev(device);
if (!sas_dev) {
dev_err(dev, "fail alloc dev: max support %d devices\n",
HISI_SAS_MAX_DEVICES);
diff --git a/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c b/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c
index b733747..bd20c54 100644
--- a/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c
+++ b/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c
@@ -465,6 +465,62 @@
return readl(regs);
}
+/* This function needs to be protected from pre-emption. */
+static int
+slot_index_alloc_quirk_v2_hw(struct hisi_hba *hisi_hba, int *slot_idx,
+ struct domain_device *device)
+{
+ unsigned int index = 0;
+ void *bitmap = hisi_hba->slot_index_tags;
+ int sata_dev = dev_is_sata(device);
+
+ while (1) {
+ index = find_next_zero_bit(bitmap, hisi_hba->slot_index_count,
+ index);
+ if (index >= hisi_hba->slot_index_count)
+ return -SAS_QUEUE_FULL;
+ /*
+ * SAS IPTT bit0 should be 1
+ */
+ if (sata_dev || (index & 1))
+ break;
+ index++;
+ }
+
+ set_bit(index, bitmap);
+ *slot_idx = index;
+ return 0;
+}
+
+static struct
+hisi_sas_device *alloc_dev_quirk_v2_hw(struct domain_device *device)
+{
+ struct hisi_hba *hisi_hba = device->port->ha->lldd_ha;
+ struct hisi_sas_device *sas_dev = NULL;
+ int i, sata_dev = dev_is_sata(device);
+
+ spin_lock(&hisi_hba->lock);
+ for (i = 0; i < HISI_SAS_MAX_DEVICES; i++) {
+ /*
+ * SATA device id bit0 should be 0
+ */
+ if (sata_dev && (i & 1))
+ continue;
+ if (hisi_hba->devices[i].dev_type == SAS_PHY_UNUSED) {
+ hisi_hba->devices[i].device_id = i;
+ sas_dev = &hisi_hba->devices[i];
+ sas_dev->dev_status = HISI_SAS_DEV_NORMAL;
+ sas_dev->dev_type = device->dev_type;
+ sas_dev->hisi_hba = hisi_hba;
+ sas_dev->sas_device = device;
+ break;
+ }
+ }
+ spin_unlock(&hisi_hba->lock);
+
+ return sas_dev;
+}
+
static void config_phy_opt_mode_v2_hw(struct hisi_hba *hisi_hba, int phy_no)
{
u32 cfg = hisi_sas_phy_read32(hisi_hba, phy_no, PHY_CFG);
@@ -544,7 +600,7 @@
}
qw0 |= ((1 << ITCT_HDR_VALID_OFF) |
- (device->max_linkrate << ITCT_HDR_MCR_OFF) |
+ (device->linkrate << ITCT_HDR_MCR_OFF) |
(1 << ITCT_HDR_VLN_OFF) |
(port->id << ITCT_HDR_PORT_ID_OFF));
itct->qw0 = cpu_to_le64(qw0);
@@ -554,10 +610,11 @@
itct->sas_addr = __swab64(itct->sas_addr);
/* qw2 */
- itct->qw2 = cpu_to_le64((500ULL << ITCT_HDR_INLT_OFF) |
- (0xff00ULL << ITCT_HDR_BITLT_OFF) |
- (0xff00ULL << ITCT_HDR_MCTLT_OFF) |
- (0xff00ULL << ITCT_HDR_RTOLT_OFF));
+ if (!dev_is_sata(device))
+ itct->qw2 = cpu_to_le64((500ULL << ITCT_HDR_INLT_OFF) |
+ (0x1ULL << ITCT_HDR_BITLT_OFF) |
+ (0x32ULL << ITCT_HDR_MCTLT_OFF) |
+ (0x1ULL << ITCT_HDR_RTOLT_OFF));
}
static void free_device_v2_hw(struct hisi_hba *hisi_hba,
@@ -715,7 +772,7 @@
hisi_sas_write32(hisi_hba, HGC_SAS_TX_OPEN_FAIL_RETRY_CTRL, 0x7FF);
hisi_sas_write32(hisi_hba, OPENA_WT_CONTI_TIME, 0x1);
hisi_sas_write32(hisi_hba, I_T_NEXUS_LOSS_TIME, 0x1F4);
- hisi_sas_write32(hisi_hba, MAX_CON_TIME_LIMIT_TIME, 0x4E20);
+ hisi_sas_write32(hisi_hba, MAX_CON_TIME_LIMIT_TIME, 0x32);
hisi_sas_write32(hisi_hba, BUS_INACTIVE_LIMIT_TIME, 0x1);
hisi_sas_write32(hisi_hba, CFG_AGING_TIME, 0x1);
hisi_sas_write32(hisi_hba, HGC_ERR_STAT_EN, 0x1);
@@ -1573,6 +1630,9 @@
switch (cmd) {
case ATA_CMD_FPDMA_WRITE:
case ATA_CMD_FPDMA_READ:
+ case ATA_CMD_FPDMA_RECV:
+ case ATA_CMD_FPDMA_SEND:
+ case ATA_CMD_NCQ_NON_DATA:
return SATA_PROTOCOL_FPDMA;
case ATA_CMD_ID_ATA:
@@ -1993,22 +2053,23 @@
u32 ent_tmp, ent_msk, ent_int, port_id, link_rate, hard_phy_linkrate;
irqreturn_t res = IRQ_HANDLED;
u8 attached_sas_addr[SAS_ADDR_SIZE] = {0};
- int phy_no;
+ int phy_no, offset;
phy_no = sas_phy->id;
initial_fis = &hisi_hba->initial_fis[phy_no];
fis = &initial_fis->fis;
- ent_msk = hisi_sas_read32(hisi_hba, ENT_INT_SRC_MSK1);
- hisi_sas_write32(hisi_hba, ENT_INT_SRC_MSK1, ent_msk | 1 << phy_no);
+ offset = 4 * (phy_no / 4);
+ ent_msk = hisi_sas_read32(hisi_hba, ENT_INT_SRC_MSK1 + offset);
+ hisi_sas_write32(hisi_hba, ENT_INT_SRC_MSK1 + offset,
+ ent_msk | 1 << ((phy_no % 4) * 8));
- ent_int = hisi_sas_read32(hisi_hba, ENT_INT_SRC1);
- ent_tmp = ent_int;
+ ent_int = hisi_sas_read32(hisi_hba, ENT_INT_SRC1 + offset);
+ ent_tmp = ent_int & (1 << (ENT_INT_SRC1_D2H_FIS_CH1_OFF *
+ (phy_no % 4)));
ent_int >>= ENT_INT_SRC1_D2H_FIS_CH1_OFF * (phy_no % 4);
if ((ent_int & ENT_INT_SRC1_D2H_FIS_CH0_MSK) == 0) {
dev_warn(dev, "sata int: phy%d did not receive FIS\n", phy_no);
- hisi_sas_write32(hisi_hba, ENT_INT_SRC1, ent_tmp);
- hisi_sas_write32(hisi_hba, ENT_INT_SRC_MSK1, ent_msk);
res = IRQ_NONE;
goto end;
}
@@ -2056,8 +2117,8 @@
queue_work(hisi_hba->wq, &phy->phyup_ws);
end:
- hisi_sas_write32(hisi_hba, ENT_INT_SRC1, ent_tmp);
- hisi_sas_write32(hisi_hba, ENT_INT_SRC_MSK1, ent_msk);
+ hisi_sas_write32(hisi_hba, ENT_INT_SRC1 + offset, ent_tmp);
+ hisi_sas_write32(hisi_hba, ENT_INT_SRC_MSK1 + offset, ent_msk);
return res;
}
@@ -2165,6 +2226,8 @@
static const struct hisi_sas_hw hisi_sas_v2_hw = {
.hw_init = hisi_sas_v2_init,
.setup_itct = setup_itct_v2_hw,
+ .slot_index_alloc = slot_index_alloc_quirk_v2_hw,
+ .alloc_dev = alloc_dev_quirk_v2_hw,
.sl_notify = sl_notify_v2_hw,
.get_wideport_bitmap = get_wideport_bitmap_v2_hw,
.free_device = free_device_v2_hw,
diff --git a/drivers/scsi/hpsa.c b/drivers/scsi/hpsa.c
index 5be944c..ff8dcd5 100644
--- a/drivers/scsi/hpsa.c
+++ b/drivers/scsi/hpsa.c
@@ -60,7 +60,7 @@
* HPSA_DRIVER_VERSION must be 3 byte values (0-255) separated by '.'
* with an optional trailing '-' followed by a byte value (0-255).
*/
-#define HPSA_DRIVER_VERSION "3.4.14-0"
+#define HPSA_DRIVER_VERSION "3.4.16-0"
#define DRIVER_NAME "HP HPSA Driver (v " HPSA_DRIVER_VERSION ")"
#define HPSA "hpsa"
@@ -294,6 +294,9 @@
static inline int hpsa_scsi_do_report_phys_luns(struct ctlr_info *h,
struct ReportExtendedLUNdata *buf, int bufsize);
static int hpsa_luns_changed(struct ctlr_info *h);
+static bool hpsa_cmd_dev_match(struct ctlr_info *h, struct CommandList *c,
+ struct hpsa_scsi_dev_t *dev,
+ unsigned char *scsi3addr);
static inline struct ctlr_info *sdev_to_hba(struct scsi_device *sdev)
{
@@ -728,6 +731,29 @@
sn[12], sn[13], sn[14], sn[15]);
}
+static ssize_t sas_address_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct ctlr_info *h;
+ struct scsi_device *sdev;
+ struct hpsa_scsi_dev_t *hdev;
+ unsigned long flags;
+ u64 sas_address;
+
+ sdev = to_scsi_device(dev);
+ h = sdev_to_hba(sdev);
+ spin_lock_irqsave(&h->lock, flags);
+ hdev = sdev->hostdata;
+ if (!hdev || is_logical_device(hdev) || !hdev->expose_device) {
+ spin_unlock_irqrestore(&h->lock, flags);
+ return -ENODEV;
+ }
+ sas_address = hdev->sas_address;
+ spin_unlock_irqrestore(&h->lock, flags);
+
+ return snprintf(buf, PAGE_SIZE, "0x%016llx\n", sas_address);
+}
+
static ssize_t host_show_hp_ssd_smart_path_enabled(struct device *dev,
struct device_attribute *attr, char *buf)
{
@@ -840,6 +866,7 @@
static DEVICE_ATTR(lunid, S_IRUGO, lunid_show, NULL);
static DEVICE_ATTR(unique_id, S_IRUGO, unique_id_show, NULL);
static DEVICE_ATTR(rescan, S_IWUSR, NULL, host_store_rescan);
+static DEVICE_ATTR(sas_address, S_IRUGO, sas_address_show, NULL);
static DEVICE_ATTR(hp_ssd_smart_path_enabled, S_IRUGO,
host_show_hp_ssd_smart_path_enabled, NULL);
static DEVICE_ATTR(path_info, S_IRUGO, path_info_show, NULL);
@@ -865,6 +892,7 @@
&dev_attr_unique_id,
&dev_attr_hp_ssd_smart_path_enabled,
&dev_attr_path_info,
+ &dev_attr_sas_address,
NULL,
};
@@ -1637,9 +1665,8 @@
for (j = 0; j < ndevices; j++) {
if (dev[j] == NULL)
continue;
- if (dev[j]->devtype != TYPE_DISK)
- continue;
- if (dev[j]->devtype != TYPE_ZBC)
+ if (dev[j]->devtype != TYPE_DISK &&
+ dev[j]->devtype != TYPE_ZBC)
continue;
if (is_logical_device(dev[j]))
continue;
@@ -1684,9 +1711,8 @@
for (i = 0; i < ndevices; i++) {
if (dev[i] == NULL)
continue;
- if (dev[i]->devtype != TYPE_DISK)
- continue;
- if (dev[i]->devtype != TYPE_ZBC)
+ if (dev[i]->devtype != TYPE_DISK &&
+ dev[i]->devtype != TYPE_ZBC)
continue;
if (!is_logical_device(dev[i]))
continue;
@@ -1720,6 +1746,51 @@
return rc;
}
+static int hpsa_find_outstanding_commands_for_dev(struct ctlr_info *h,
+ struct hpsa_scsi_dev_t *dev)
+{
+ int i;
+ int count = 0;
+
+ for (i = 0; i < h->nr_cmds; i++) {
+ struct CommandList *c = h->cmd_pool + i;
+ int refcount = atomic_inc_return(&c->refcount);
+
+ if (refcount > 1 && hpsa_cmd_dev_match(h, c, dev,
+ dev->scsi3addr)) {
+ unsigned long flags;
+
+ spin_lock_irqsave(&h->lock, flags); /* Implied MB */
+ if (!hpsa_is_cmd_idle(c))
+ ++count;
+ spin_unlock_irqrestore(&h->lock, flags);
+ }
+
+ cmd_free(h, c);
+ }
+
+ return count;
+}
+
+static void hpsa_wait_for_outstanding_commands_for_dev(struct ctlr_info *h,
+ struct hpsa_scsi_dev_t *device)
+{
+ int cmds = 0;
+ int waits = 0;
+
+ while (1) {
+ cmds = hpsa_find_outstanding_commands_for_dev(h, device);
+ if (cmds == 0)
+ break;
+ if (++waits > 20)
+ break;
+ dev_warn(&h->pdev->dev,
+ "%s: removing device with %d outstanding commands!\n",
+ __func__, cmds);
+ msleep(1000);
+ }
+}
+
static void hpsa_remove_device(struct ctlr_info *h,
struct hpsa_scsi_dev_t *device)
{
@@ -1743,8 +1814,13 @@
hpsa_show_dev_msg(KERN_WARNING, h, device,
"didn't find device for removal.");
}
- } else /* HBA */
+ } else { /* HBA */
+
+ device->removed = 1;
+ hpsa_wait_for_outstanding_commands_for_dev(h, device);
+
hpsa_remove_sas_device(device);
+ }
}
static void adjust_hpsa_scsi_table(struct ctlr_info *h,
@@ -2146,7 +2222,8 @@
static int handle_ioaccel_mode2_error(struct ctlr_info *h,
struct CommandList *c,
struct scsi_cmnd *cmd,
- struct io_accel2_cmd *c2)
+ struct io_accel2_cmd *c2,
+ struct hpsa_scsi_dev_t *dev)
{
int data_len;
int retry = 0;
@@ -2210,8 +2287,27 @@
case IOACCEL2_STATUS_SR_NO_PATH_TO_DEVICE:
case IOACCEL2_STATUS_SR_INVALID_DEVICE:
case IOACCEL2_STATUS_SR_IOACCEL_DISABLED:
- /* We will get an event from ctlr to trigger rescan */
- retry = 1;
+ /*
+ * Did an HBA disk disappear? We will eventually
+ * get a state change event from the controller but
+ * in the meantime, we need to tell the OS that the
+ * HBA disk is no longer there and stop I/O
+ * from going down. This allows the potential re-insert
+ * of the disk to get the same device node.
+ */
+ if (dev->physical_device && dev->expose_device) {
+ cmd->result = DID_NO_CONNECT << 16;
+ dev->removed = 1;
+ h->drv_req_rescan = 1;
+ dev_warn(&h->pdev->dev,
+ "%s: device is gone!\n", __func__);
+ } else
+ /*
+ * Retry by sending down the RAID path.
+ * We will get an event from ctlr to
+ * trigger rescan regardless.
+ */
+ retry = 1;
break;
default:
retry = 1;
@@ -2335,13 +2431,15 @@
c2->error_data.serv_response ==
IOACCEL2_SERV_RESPONSE_FAILURE) {
if (c2->error_data.status ==
- IOACCEL2_STATUS_SR_IOACCEL_DISABLED)
+ IOACCEL2_STATUS_SR_IOACCEL_DISABLED) {
dev->offload_enabled = 0;
+ dev->offload_to_be_enabled = 0;
+ }
return hpsa_retry_cmd(h, c);
}
- if (handle_ioaccel_mode2_error(h, c, cmd, c2))
+ if (handle_ioaccel_mode2_error(h, c, cmd, c2, dev))
return hpsa_retry_cmd(h, c);
return hpsa_cmd_free_and_done(h, c, cmd);
@@ -2806,7 +2904,7 @@
goto out;
}
rc = hpsa_scsi_do_simple_cmd_with_retry(h, c,
- PCI_DMA_FROMDEVICE, NO_TIMEOUT);
+ PCI_DMA_FROMDEVICE, DEFAULT_TIMEOUT);
if (rc)
goto out;
ei = c->err_info;
@@ -2832,7 +2930,7 @@
/* fill_cmd can't fail here, no data buffer to map. */
(void) fill_cmd(c, reset_type, h, NULL, 0, 0,
scsi3addr, TYPE_MSG);
- rc = hpsa_scsi_do_simple_cmd(h, c, reply_queue, NO_TIMEOUT);
+ rc = hpsa_scsi_do_simple_cmd(h, c, reply_queue, DEFAULT_TIMEOUT);
if (rc) {
dev_warn(&h->pdev->dev, "Failed to send reset command\n");
goto out;
@@ -3080,7 +3178,7 @@
return -1;
}
rc = hpsa_scsi_do_simple_cmd_with_retry(h, c,
- PCI_DMA_FROMDEVICE, NO_TIMEOUT);
+ PCI_DMA_FROMDEVICE, DEFAULT_TIMEOUT);
if (rc)
goto out;
ei = c->err_info;
@@ -3123,7 +3221,7 @@
c->Request.CDB[9] = (bmic_device_index >> 8) & 0xff;
rc = hpsa_scsi_do_simple_cmd_with_retry(h, c,
- PCI_DMA_FROMDEVICE, NO_TIMEOUT);
+ PCI_DMA_FROMDEVICE, DEFAULT_TIMEOUT);
if (rc)
goto out;
ei = c->err_info;
@@ -3151,7 +3249,7 @@
goto out;
rc = hpsa_scsi_do_simple_cmd_with_retry(h, c,
- PCI_DMA_FROMDEVICE, NO_TIMEOUT);
+ PCI_DMA_FROMDEVICE, DEFAULT_TIMEOUT);
if (rc)
goto out;
ei = c->err_info;
@@ -3182,7 +3280,7 @@
c->Request.CDB[9] = (bmic_device_index >> 8) & 0xff;
hpsa_scsi_do_simple_cmd_with_retry(h, c, PCI_DMA_FROMDEVICE,
- NO_TIMEOUT);
+ DEFAULT_TIMEOUT);
ei = c->err_info;
if (ei->CommandStatus != 0 && ei->CommandStatus != CMD_DATA_UNDERRUN) {
hpsa_scsi_interpret_error(h, c);
@@ -3250,7 +3348,7 @@
c->Request.CDB[5] = 0;
rc = hpsa_scsi_do_simple_cmd_with_retry(h, c, PCI_DMA_FROMDEVICE,
- NO_TIMEOUT);
+ DEFAULT_TIMEOUT);
if (rc)
goto out;
@@ -3462,7 +3560,7 @@
if (extended_response)
c->Request.CDB[1] = extended_response;
rc = hpsa_scsi_do_simple_cmd_with_retry(h, c,
- PCI_DMA_FROMDEVICE, NO_TIMEOUT);
+ PCI_DMA_FROMDEVICE, DEFAULT_TIMEOUT);
if (rc)
goto out;
ei = c->err_info;
@@ -3569,7 +3667,8 @@
c = cmd_alloc(h);
(void) fill_cmd(c, TEST_UNIT_READY, h, NULL, 0, 0, scsi3addr, TYPE_CMD);
- rc = hpsa_scsi_do_simple_cmd(h, c, DEFAULT_REPLY_QUEUE, NO_TIMEOUT);
+ rc = hpsa_scsi_do_simple_cmd(h, c, DEFAULT_REPLY_QUEUE,
+ DEFAULT_TIMEOUT);
if (rc) {
cmd_free(h, c);
return 0;
@@ -3644,7 +3743,8 @@
c = cmd_alloc(h);
(void) fill_cmd(c, HPSA_ABORT_MSG, h, &tag, 0, 0, scsi3addr, TYPE_MSG);
- (void) hpsa_scsi_do_simple_cmd(h, c, DEFAULT_REPLY_QUEUE, NO_TIMEOUT);
+ (void) hpsa_scsi_do_simple_cmd(h, c, DEFAULT_REPLY_QUEUE,
+ DEFAULT_TIMEOUT);
/* no unmap needed here because no data xfer. */
ei = c->err_info;
switch (ei->CommandStatus) {
@@ -5234,6 +5334,12 @@
dev = cmd->device->hostdata;
if (!dev) {
+ cmd->result = NOT_READY << 16; /* host byte */
+ cmd->scsi_done(cmd);
+ return 0;
+ }
+
+ if (dev->removed) {
cmd->result = DID_NO_CONNECT << 16;
cmd->scsi_done(cmd);
return 0;
@@ -5414,7 +5520,7 @@
/* Send the Test Unit Ready, fill_cmd can't fail, no mapping */
(void) fill_cmd(c, TEST_UNIT_READY, h,
NULL, 0, 0, lunaddr, TYPE_CMD);
- rc = hpsa_scsi_do_simple_cmd(h, c, reply_queue, NO_TIMEOUT);
+ rc = hpsa_scsi_do_simple_cmd(h, c, reply_queue, DEFAULT_TIMEOUT);
if (rc)
return rc;
/* no unmap needed here because no data xfer. */
@@ -5638,7 +5744,7 @@
0, 0, scsi3addr, TYPE_MSG);
if (h->needs_abort_tags_swizzled)
swizzle_abort_tag(&c->Request.CDB[4]);
- (void) hpsa_scsi_do_simple_cmd(h, c, reply_queue, NO_TIMEOUT);
+ (void) hpsa_scsi_do_simple_cmd(h, c, reply_queue, DEFAULT_TIMEOUT);
hpsa_get_tag(h, abort, &taglower, &tagupper);
dev_dbg(&h->pdev->dev, "%s: Tag:0x%08x:%08x: do_simple_cmd(abort) completed.\n",
__func__, tagupper, taglower);
@@ -5803,7 +5909,7 @@
c = cmd_alloc(h);
setup_ioaccel2_abort_cmd(c, h, abort, reply_queue);
c2 = &h->ioaccel2_cmd_pool[c->cmdindex];
- (void) hpsa_scsi_do_simple_cmd(h, c, reply_queue, NO_TIMEOUT);
+ (void) hpsa_scsi_do_simple_cmd(h, c, reply_queue, DEFAULT_TIMEOUT);
hpsa_get_tag(h, abort, &taglower, &tagupper);
dev_dbg(&h->pdev->dev,
"%s: Tag:0x%08x:%08x: do_simple_cmd(ioaccel2 abort) completed.\n",
@@ -6348,7 +6454,8 @@
c->SG[0].Len = cpu_to_le32(iocommand.buf_size);
c->SG[0].Ext = cpu_to_le32(HPSA_SG_LAST); /* not chaining */
}
- rc = hpsa_scsi_do_simple_cmd(h, c, DEFAULT_REPLY_QUEUE, NO_TIMEOUT);
+ rc = hpsa_scsi_do_simple_cmd(h, c, DEFAULT_REPLY_QUEUE,
+ DEFAULT_TIMEOUT);
if (iocommand.buf_size > 0)
hpsa_pci_unmap(h->pdev, c, 1, PCI_DMA_BIDIRECTIONAL);
check_ioctl_unit_attention(h, c);
@@ -6480,7 +6587,8 @@
}
c->SG[--i].Ext = cpu_to_le32(HPSA_SG_LAST);
}
- status = hpsa_scsi_do_simple_cmd(h, c, DEFAULT_REPLY_QUEUE, NO_TIMEOUT);
+ status = hpsa_scsi_do_simple_cmd(h, c, DEFAULT_REPLY_QUEUE,
+ DEFAULT_TIMEOUT);
if (sg_used)
hpsa_pci_unmap(h->pdev, c, sg_used, PCI_DMA_BIDIRECTIONAL);
check_ioctl_unit_attention(h, c);
@@ -8254,8 +8362,10 @@
event_type = "configuration change";
/* Stop sending new RAID offload reqs via the IO accelerator */
scsi_block_requests(h->scsi_host);
- for (i = 0; i < h->ndevices; i++)
+ for (i = 0; i < h->ndevices; i++) {
h->dev[i]->offload_enabled = 0;
+ h->dev[i]->offload_to_be_enabled = 0;
+ }
hpsa_drain_accel_commands(h);
/* Set 'accelerator path config change' bit */
dev_warn(&h->pdev->dev,
@@ -8541,11 +8651,6 @@
if (rc)
goto clean6; /* sg, cmd, irq, shost, pci, lu, aer/h */
- /* hook into SCSI subsystem */
- rc = hpsa_scsi_add_host(h);
- if (rc)
- goto clean7; /* perf, sg, cmd, irq, shost, pci, lu, aer/h */
-
/* create the resubmit workqueue */
h->rescan_ctlr_wq = hpsa_create_controller_wq(h, "rescan");
if (!h->rescan_ctlr_wq) {
@@ -8642,6 +8747,11 @@
dev_info(&h->pdev->dev,
"Can't track change to report lun data\n");
+ /* hook into SCSI subsystem */
+ rc = hpsa_scsi_add_host(h);
+ if (rc)
+ goto clean7; /* perf, sg, cmd, irq, shost, pci, lu, aer/h */
+
/* Monitor the controller for firmware lockups */
h->heartbeat_sample_interval = HEARTBEAT_SAMPLE_INTERVAL;
INIT_DELAYED_WORK(&h->monitor_ctlr_work, hpsa_monitor_ctlr_worker);
@@ -8703,7 +8813,7 @@
goto out;
}
rc = hpsa_scsi_do_simple_cmd_with_retry(h, c,
- PCI_DMA_TODEVICE, NO_TIMEOUT);
+ PCI_DMA_TODEVICE, DEFAULT_TIMEOUT);
if (rc)
goto out;
if (c->err_info->CommandStatus != 0)
@@ -8742,7 +8852,7 @@
goto errout;
rc = hpsa_scsi_do_simple_cmd_with_retry(h, c,
- PCI_DMA_FROMDEVICE, NO_TIMEOUT);
+ PCI_DMA_FROMDEVICE, DEFAULT_TIMEOUT);
if ((rc != 0) || (c->err_info->CommandStatus != 0))
goto errout;
@@ -8754,7 +8864,7 @@
goto errout;
rc = hpsa_scsi_do_simple_cmd_with_retry(h, c,
- PCI_DMA_TODEVICE, NO_TIMEOUT);
+ PCI_DMA_TODEVICE, DEFAULT_TIMEOUT);
if ((rc != 0) || (c->err_info->CommandStatus != 0))
goto errout;
@@ -8764,7 +8874,7 @@
goto errout;
rc = hpsa_scsi_do_simple_cmd_with_retry(h, c,
- PCI_DMA_FROMDEVICE, NO_TIMEOUT);
+ PCI_DMA_FROMDEVICE, DEFAULT_TIMEOUT);
if ((rc != 0) || (c->err_info->CommandStatus != 0))
goto errout;
@@ -9602,6 +9712,7 @@
static int
hpsa_sas_get_enclosure_identifier(struct sas_rphy *rphy, u64 *identifier)
{
+ *identifier = 0;
return 0;
}
diff --git a/drivers/scsi/hpsa.h b/drivers/scsi/hpsa.h
index d06bb74..a1487e6 100644
--- a/drivers/scsi/hpsa.h
+++ b/drivers/scsi/hpsa.h
@@ -63,6 +63,7 @@
unsigned char scsi3addr[8]; /* as presented to the HW */
u8 physical_device : 1;
u8 expose_device;
+ u8 removed : 1; /* device is marked for death */
#define RAID_CTLR_LUNID "\0\0\0\0\0\0\0\0"
unsigned char device_id[16]; /* from inquiry pg. 0x83 */
u64 sas_address;
diff --git a/drivers/scsi/isci/request.c b/drivers/scsi/isci/request.c
index cfd0084..b709d2b 100644
--- a/drivers/scsi/isci/request.c
+++ b/drivers/scsi/isci/request.c
@@ -3169,7 +3169,10 @@
status = sci_io_request_construct_basic_sata(ireq);
if (qc && (qc->tf.command == ATA_CMD_FPDMA_WRITE ||
- qc->tf.command == ATA_CMD_FPDMA_READ)) {
+ qc->tf.command == ATA_CMD_FPDMA_READ ||
+ qc->tf.command == ATA_CMD_FPDMA_RECV ||
+ qc->tf.command == ATA_CMD_FPDMA_SEND ||
+ qc->tf.command == ATA_CMD_NCQ_NON_DATA)) {
fis->sector_count = qc->tag << 3;
ireq->tc->type.stp.ncq_tag = qc->tag;
}
diff --git a/drivers/scsi/iscsi_boot_sysfs.c b/drivers/scsi/iscsi_boot_sysfs.c
index 8f0ea97..d453667 100644
--- a/drivers/scsi/iscsi_boot_sysfs.c
+++ b/drivers/scsi/iscsi_boot_sysfs.c
@@ -306,6 +306,42 @@
.is_visible = iscsi_boot_ini_attr_is_visible,
};
+/* iBFT ACPI Table attributes */
+iscsi_boot_rd_attr(acpitbl_signature, signature, ISCSI_BOOT_ACPITBL_SIGNATURE);
+iscsi_boot_rd_attr(acpitbl_oem_id, oem_id, ISCSI_BOOT_ACPITBL_OEM_ID);
+iscsi_boot_rd_attr(acpitbl_oem_table_id, oem_table_id,
+ ISCSI_BOOT_ACPITBL_OEM_TABLE_ID);
+
+static struct attribute *acpitbl_attrs[] = {
+ &iscsi_boot_attr_acpitbl_signature.attr,
+ &iscsi_boot_attr_acpitbl_oem_id.attr,
+ &iscsi_boot_attr_acpitbl_oem_table_id.attr,
+ NULL
+};
+
+static umode_t iscsi_boot_acpitbl_attr_is_visible(struct kobject *kobj,
+ struct attribute *attr, int i)
+{
+ struct iscsi_boot_kobj *boot_kobj =
+ container_of(kobj, struct iscsi_boot_kobj, kobj);
+
+ if (attr == &iscsi_boot_attr_acpitbl_signature.attr)
+ return boot_kobj->is_visible(boot_kobj->data,
+ ISCSI_BOOT_ACPITBL_SIGNATURE);
+ if (attr == &iscsi_boot_attr_acpitbl_oem_id.attr)
+ return boot_kobj->is_visible(boot_kobj->data,
+ ISCSI_BOOT_ACPITBL_OEM_ID);
+ if (attr == &iscsi_boot_attr_acpitbl_oem_table_id.attr)
+ return boot_kobj->is_visible(boot_kobj->data,
+ ISCSI_BOOT_ACPITBL_OEM_TABLE_ID);
+ return 0;
+}
+
+static struct attribute_group iscsi_boot_acpitbl_attr_group = {
+ .attrs = acpitbl_attrs,
+ .is_visible = iscsi_boot_acpitbl_attr_is_visible,
+};
+
static struct iscsi_boot_kobj *
iscsi_boot_create_kobj(struct iscsi_boot_kset *boot_kset,
struct attribute_group *attr_group,
@@ -436,6 +472,32 @@
EXPORT_SYMBOL_GPL(iscsi_boot_create_ethernet);
/**
+ * iscsi_boot_create_acpitbl() - create boot acpi table sysfs dir
+ * @boot_kset: boot kset
+ * @index: not used
+ * @data: driver specific data
+ * @show: attr show function
+ * @is_visible: attr visibility function
+ * @release: release function
+ *
+ * Note: The boot sysfs lib will free the data passed in for the caller
+ * when all refs to the acpitbl kobject have been released.
+ */
+struct iscsi_boot_kobj *
+iscsi_boot_create_acpitbl(struct iscsi_boot_kset *boot_kset, int index,
+ void *data,
+ ssize_t (*show)(void *data, int type, char *buf),
+ umode_t (*is_visible)(void *data, int type),
+ void (*release)(void *data))
+{
+ return iscsi_boot_create_kobj(boot_kset,
+ &iscsi_boot_acpitbl_attr_group,
+ "acpi_header", index, data, show,
+ is_visible, release);
+}
+EXPORT_SYMBOL_GPL(iscsi_boot_create_acpitbl);
+
+/**
* iscsi_boot_create_kset() - creates root sysfs tree
* @set_name: name of root dir
*/
diff --git a/drivers/scsi/iscsi_tcp.c b/drivers/scsi/iscsi_tcp.c
index 2e4c82f..ace4f1f 100644
--- a/drivers/scsi/iscsi_tcp.c
+++ b/drivers/scsi/iscsi_tcp.c
@@ -131,10 +131,10 @@
struct iscsi_tcp_conn *tcp_conn;
read_descriptor_t rd_desc;
- read_lock(&sk->sk_callback_lock);
+ read_lock_bh(&sk->sk_callback_lock);
conn = sk->sk_user_data;
if (!conn) {
- read_unlock(&sk->sk_callback_lock);
+ read_unlock_bh(&sk->sk_callback_lock);
return;
}
tcp_conn = conn->dd_data;
@@ -154,7 +154,7 @@
/* If we had to (atomically) map a highmem page,
* unmap it now. */
iscsi_tcp_segment_unmap(&tcp_conn->in.segment);
- read_unlock(&sk->sk_callback_lock);
+ read_unlock_bh(&sk->sk_callback_lock);
}
static void iscsi_sw_tcp_state_change(struct sock *sk)
@@ -165,10 +165,10 @@
struct iscsi_session *session;
void (*old_state_change)(struct sock *);
- read_lock(&sk->sk_callback_lock);
+ read_lock_bh(&sk->sk_callback_lock);
conn = sk->sk_user_data;
if (!conn) {
- read_unlock(&sk->sk_callback_lock);
+ read_unlock_bh(&sk->sk_callback_lock);
return;
}
session = conn->session;
@@ -179,7 +179,7 @@
tcp_sw_conn = tcp_conn->dd_data;
old_state_change = tcp_sw_conn->old_state_change;
- read_unlock(&sk->sk_callback_lock);
+ read_unlock_bh(&sk->sk_callback_lock);
old_state_change(sk);
}
diff --git a/drivers/scsi/libiscsi.c b/drivers/scsi/libiscsi.c
index 6bffd91..c051694 100644
--- a/drivers/scsi/libiscsi.c
+++ b/drivers/scsi/libiscsi.c
@@ -2127,7 +2127,7 @@
struct iscsi_conn *conn;
struct iscsi_task *task;
struct iscsi_tm *hdr;
- int rc, age;
+ int age;
cls_session = starget_to_session(scsi_target(sc->device));
session = cls_session->dd_data;
@@ -2188,10 +2188,8 @@
hdr = &conn->tmhdr;
iscsi_prep_abort_task_pdu(task, hdr);
- if (iscsi_exec_task_mgmt_fn(conn, hdr, age, session->abort_timeout)) {
- rc = FAILED;
+ if (iscsi_exec_task_mgmt_fn(conn, hdr, age, session->abort_timeout))
goto failed;
- }
switch (conn->tmf_state) {
case TMF_SUCCESS:
@@ -2423,7 +2421,7 @@
*
* This will attempt to send a warm target reset.
*/
-int iscsi_eh_target_reset(struct scsi_cmnd *sc)
+static int iscsi_eh_target_reset(struct scsi_cmnd *sc)
{
struct iscsi_cls_session *cls_session;
struct iscsi_session *session;
@@ -2495,7 +2493,6 @@
mutex_unlock(&session->eh_mutex);
return rc;
}
-EXPORT_SYMBOL_GPL(iscsi_eh_target_reset);
/**
* iscsi_eh_recover_target - reset target and possibly the session
diff --git a/drivers/scsi/libsas/sas_ata.c b/drivers/scsi/libsas/sas_ata.c
index 9c706d8..935c430 100644
--- a/drivers/scsi/libsas/sas_ata.c
+++ b/drivers/scsi/libsas/sas_ata.c
@@ -205,7 +205,10 @@
task->task_done = sas_ata_task_done;
if (qc->tf.command == ATA_CMD_FPDMA_WRITE ||
- qc->tf.command == ATA_CMD_FPDMA_READ) {
+ qc->tf.command == ATA_CMD_FPDMA_READ ||
+ qc->tf.command == ATA_CMD_FPDMA_RECV ||
+ qc->tf.command == ATA_CMD_FPDMA_SEND ||
+ qc->tf.command == ATA_CMD_NCQ_NON_DATA) {
/* Need to zero out the tag libata assigned us */
qc->tf.nsect = 0;
}
@@ -548,7 +551,7 @@
static struct ata_port_info sata_port_info = {
.flags = ATA_FLAG_SATA | ATA_FLAG_PIO_DMA | ATA_FLAG_NCQ |
- ATA_FLAG_SAS_HOST,
+ ATA_FLAG_SAS_HOST | ATA_FLAG_FPDMA_AUX,
.pio_mask = ATA_PIO4,
.mwdma_mask = ATA_MWDMA2,
.udma_mask = ATA_UDMA6,
diff --git a/drivers/scsi/lpfc/lpfc.h b/drivers/scsi/lpfc/lpfc.h
index 90a3ca5..d5bd420 100644
--- a/drivers/scsi/lpfc/lpfc.h
+++ b/drivers/scsi/lpfc/lpfc.h
@@ -1,7 +1,7 @@
/*******************************************************************
* This file is part of the Emulex Linux Device Driver for *
* Fibre Channel Host Bus Adapters. *
- * Copyright (C) 2004-2015 Emulex. All rights reserved. *
+ * Copyright (C) 2004-2016 Emulex. All rights reserved. *
* EMULEX and SLI are trademarks of Emulex. *
* www.emulex.com *
* Portions Copyright (C) 2004-2005 Christoph Hellwig *
@@ -694,6 +694,7 @@
uint8_t wwnn[8];
uint8_t wwpn[8];
uint32_t RandomData[7];
+ uint32_t fcp_embed_io;
/* HBA Config Parameters */
uint32_t cfg_ack0;
@@ -757,7 +758,6 @@
uint32_t cfg_fdmi_on;
#define LPFC_FDMI_NO_SUPPORT 0 /* FDMI not supported */
#define LPFC_FDMI_SUPPORT 1 /* FDMI supported? */
-#define LPFC_FDMI_SMART_SAN 2 /* SmartSAN supported */
uint32_t cfg_enable_SmartSAN;
lpfc_vpd_t vpd; /* vital product data */
diff --git a/drivers/scsi/lpfc/lpfc_attr.c b/drivers/scsi/lpfc/lpfc_attr.c
index 343ae94..cfec2ec 100644
--- a/drivers/scsi/lpfc/lpfc_attr.c
+++ b/drivers/scsi/lpfc/lpfc_attr.c
@@ -1,7 +1,7 @@
/*******************************************************************
* This file is part of the Emulex Linux Device Driver for *
* Fibre Channel Host Bus Adapters. *
- * Copyright (C) 2004-2015 Emulex. All rights reserved. *
+ * Copyright (C) 2004-2016 Emulex. All rights reserved. *
* EMULEX and SLI are trademarks of Emulex. *
* www.emulex.com *
* Portions Copyright (C) 2004-2005 Christoph Hellwig *
@@ -4584,15 +4584,14 @@
# lpfc_fdmi_on: Controls FDMI support.
# 0 No FDMI support (default)
# 1 Traditional FDMI support
-# 2 Smart SAN support
-# If lpfc_enable_SmartSAN is set 1, the driver sets lpfc_fdmi_on to value 2
-# overwriting the current value. If lpfc_enable_SmartSAN is set 0, the
-# driver uses the current value of lpfc_fdmi_on provided it has value 0 or 1.
-# A value of 2 with lpfc_enable_SmartSAN set to 0 causes the driver to
-# set lpfc_fdmi_on back to 1.
-# Value range [0,2]. Default value is 0.
+# Traditional FDMI support means the driver will assume FDMI-2 support;
+# however, if that fails, it will fallback to FDMI-1.
+# If lpfc_enable_SmartSAN is set to 1, the driver ignores lpfc_fdmi_on.
+# If lpfc_enable_SmartSAN is set 0, the driver uses the current value of
+# lpfc_fdmi_on.
+# Value range [0,1]. Default value is 0.
*/
-LPFC_ATTR_R(fdmi_on, 0, 0, 2, "Enable FDMI support");
+LPFC_ATTR_R(fdmi_on, 0, 0, 1, "Enable FDMI support");
/*
# Specifies the maximum number of ELS cmds we can have outstanding (for
@@ -5150,7 +5149,6 @@
sysfs_remove_bin_file(&shost->shost_dev.kobj, &sysfs_ctlreg_attr);
}
-
/*
* Dynamic FC Host Attributes Support
*/
@@ -5857,14 +5855,6 @@
else
phba->cfg_poll = lpfc_poll;
- /* Ensure fdmi_on and enable_SmartSAN don't conflict */
- if (phba->cfg_enable_SmartSAN) {
- phba->cfg_fdmi_on = LPFC_FDMI_SMART_SAN;
- } else {
- if (phba->cfg_fdmi_on == LPFC_FDMI_SMART_SAN)
- phba->cfg_fdmi_on = LPFC_FDMI_SUPPORT;
- }
-
phba->cfg_soft_wwnn = 0L;
phba->cfg_soft_wwpn = 0L;
lpfc_sg_seg_cnt_init(phba, lpfc_sg_seg_cnt);
diff --git a/drivers/scsi/lpfc/lpfc_ct.c b/drivers/scsi/lpfc/lpfc_ct.c
index 79e261d..a38816e 100644
--- a/drivers/scsi/lpfc/lpfc_ct.c
+++ b/drivers/scsi/lpfc/lpfc_ct.c
@@ -1,7 +1,7 @@
/*******************************************************************
* This file is part of the Emulex Linux Device Driver for *
* Fibre Channel Host Bus Adapters. *
- * Copyright (C) 2004-2015 Emulex. All rights reserved. *
+ * Copyright (C) 2004-2016 Emulex. All rights reserved. *
* EMULEX and SLI are trademarks of Emulex. *
* www.emulex.com *
* *
@@ -2322,7 +2322,7 @@
ae = (struct lpfc_fdmi_attr_entry *)&ad->AttrValue;
memset(ae, 0, 256);
- strncpy(ae->un.AttrString, "Smart SAN Version 1.0",
+ strncpy(ae->un.AttrString, "Smart SAN Version 2.0",
sizeof(ae->un.AttrString));
len = strnlen(ae->un.AttrString,
sizeof(ae->un.AttrString));
@@ -2397,7 +2397,7 @@
uint32_t size;
ae = (struct lpfc_fdmi_attr_entry *)&ad->AttrValue;
- ae->un.AttrInt = cpu_to_be32(0);
+ ae->un.AttrInt = cpu_to_be32(1);
size = FOURBYTES + sizeof(uint32_t);
ad->AttrLen = cpu_to_be16(size);
ad->AttrType = cpu_to_be16(RPRT_SMART_SECURITY);
diff --git a/drivers/scsi/lpfc/lpfc_els.c b/drivers/scsi/lpfc/lpfc_els.c
index 7f5abb8..0498f57 100644
--- a/drivers/scsi/lpfc/lpfc_els.c
+++ b/drivers/scsi/lpfc/lpfc_els.c
@@ -1,7 +1,7 @@
/*******************************************************************
* This file is part of the Emulex Linux Device Driver for *
* Fibre Channel Host Bus Adapters. *
- * Copyright (C) 2004-2015 Emulex. All rights reserved. *
+ * Copyright (C) 2004-2016 Emulex. All rights reserved. *
* EMULEX and SLI are trademarks of Emulex. *
* www.emulex.com *
* Portions Copyright (C) 2004-2005 Christoph Hellwig *
@@ -690,16 +690,17 @@
fabric_param_changed = lpfc_check_clean_addr_bit(vport, sp);
if (fabric_param_changed) {
/* Reset FDMI attribute masks based on config parameter */
- if (phba->cfg_fdmi_on == LPFC_FDMI_NO_SUPPORT) {
- vport->fdmi_hba_mask = 0;
- vport->fdmi_port_mask = 0;
- } else {
+ if (phba->cfg_enable_SmartSAN ||
+ (phba->cfg_fdmi_on == LPFC_FDMI_SUPPORT)) {
/* Setup appropriate attribute masks */
vport->fdmi_hba_mask = LPFC_FDMI2_HBA_ATTR;
- if (phba->cfg_fdmi_on == LPFC_FDMI_SMART_SAN)
+ if (phba->cfg_enable_SmartSAN)
vport->fdmi_port_mask = LPFC_FDMI2_SMART_ATTR;
else
vport->fdmi_port_mask = LPFC_FDMI2_PORT_ATTR;
+ } else {
+ vport->fdmi_hba_mask = 0;
+ vport->fdmi_port_mask = 0;
}
}
@@ -1069,7 +1070,10 @@
lpfc_sli4_unreg_all_rpis(vport);
}
}
- lpfc_issue_reg_vfi(vport);
+
+ /* Do not register VFI if the driver aborted FLOGI */
+ if (!lpfc_error_lost_link(irsp))
+ lpfc_issue_reg_vfi(vport);
lpfc_nlp_put(ndlp);
goto out;
}
@@ -4705,6 +4709,144 @@
desc->length = cpu_to_be32(sizeof(desc->info));
}
+void
+lpfc_rdp_res_bbc_desc(struct fc_rdp_bbc_desc *desc, READ_LNK_VAR *stat,
+ struct lpfc_vport *vport)
+{
+ desc->tag = cpu_to_be32(RDP_BBC_DESC_TAG);
+
+ desc->bbc_info.port_bbc = cpu_to_be32(
+ vport->fc_sparam.cmn.bbCreditMsb |
+ vport->fc_sparam.cmn.bbCreditlsb << 8);
+ if (vport->phba->fc_topology != LPFC_TOPOLOGY_LOOP)
+ desc->bbc_info.attached_port_bbc = cpu_to_be32(
+ vport->phba->fc_fabparam.cmn.bbCreditMsb |
+ vport->phba->fc_fabparam.cmn.bbCreditlsb << 8);
+ else
+ desc->bbc_info.attached_port_bbc = 0;
+
+ desc->bbc_info.rtt = 0;
+ desc->length = cpu_to_be32(sizeof(desc->bbc_info));
+}
+
+void
+lpfc_rdp_res_oed_temp_desc(struct fc_rdp_oed_sfp_desc *desc, uint8_t *page_a2)
+{
+ uint32_t flags;
+
+ desc->tag = cpu_to_be32(RDP_OED_DESC_TAG);
+
+ desc->oed_info.hi_alarm =
+ cpu_to_be16(page_a2[SSF_TEMP_HIGH_ALARM]);
+ desc->oed_info.lo_alarm = cpu_to_be16(page_a2[SSF_TEMP_LOW_ALARM]);
+ desc->oed_info.hi_warning =
+ cpu_to_be16(page_a2[SSF_TEMP_HIGH_WARNING]);
+ desc->oed_info.lo_warning =
+ cpu_to_be16(page_a2[SSF_TEMP_LOW_WARNING]);
+ flags = 0xf; /* All four are valid */
+ flags |= ((0xf & RDP_OED_TEMPERATURE) << RDP_OED_TYPE_SHIFT);
+ desc->oed_info.function_flags = cpu_to_be32(flags);
+ desc->length = cpu_to_be32(sizeof(desc->oed_info));
+}
+
+void
+lpfc_rdp_res_oed_voltage_desc(struct fc_rdp_oed_sfp_desc *desc,
+ uint8_t *page_a2)
+{
+ uint32_t flags;
+
+ desc->tag = cpu_to_be32(RDP_OED_DESC_TAG);
+
+ desc->oed_info.hi_alarm =
+ cpu_to_be16(page_a2[SSF_VOLTAGE_HIGH_ALARM]);
+ desc->oed_info.lo_alarm = cpu_to_be16(page_a2[SSF_VOLTAGE_LOW_ALARM]);
+ desc->oed_info.hi_warning =
+ cpu_to_be16(page_a2[SSF_VOLTAGE_HIGH_WARNING]);
+ desc->oed_info.lo_warning =
+ cpu_to_be16(page_a2[SSF_VOLTAGE_LOW_WARNING]);
+ flags = 0xf; /* All four are valid */
+ flags |= ((0xf & RDP_OED_VOLTAGE) << RDP_OED_TYPE_SHIFT);
+ desc->oed_info.function_flags = cpu_to_be32(flags);
+ desc->length = cpu_to_be32(sizeof(desc->oed_info));
+}
+
+void
+lpfc_rdp_res_oed_txbias_desc(struct fc_rdp_oed_sfp_desc *desc,
+ uint8_t *page_a2)
+{
+ uint32_t flags;
+
+ desc->tag = cpu_to_be32(RDP_OED_DESC_TAG);
+
+ desc->oed_info.hi_alarm =
+ cpu_to_be16(page_a2[SSF_BIAS_HIGH_ALARM]);
+ desc->oed_info.lo_alarm = cpu_to_be16(page_a2[SSF_BIAS_LOW_ALARM]);
+ desc->oed_info.hi_warning =
+ cpu_to_be16(page_a2[SSF_BIAS_HIGH_WARNING]);
+ desc->oed_info.lo_warning =
+ cpu_to_be16(page_a2[SSF_BIAS_LOW_WARNING]);
+ flags = 0xf; /* All four are valid */
+ flags |= ((0xf & RDP_OED_TXBIAS) << RDP_OED_TYPE_SHIFT);
+ desc->oed_info.function_flags = cpu_to_be32(flags);
+ desc->length = cpu_to_be32(sizeof(desc->oed_info));
+}
+
+void
+lpfc_rdp_res_oed_txpower_desc(struct fc_rdp_oed_sfp_desc *desc,
+ uint8_t *page_a2)
+{
+ uint32_t flags;
+
+ desc->tag = cpu_to_be32(RDP_OED_DESC_TAG);
+
+ desc->oed_info.hi_alarm =
+ cpu_to_be16(page_a2[SSF_TXPOWER_HIGH_ALARM]);
+ desc->oed_info.lo_alarm = cpu_to_be16(page_a2[SSF_TXPOWER_LOW_ALARM]);
+ desc->oed_info.hi_warning =
+ cpu_to_be16(page_a2[SSF_TXPOWER_HIGH_WARNING]);
+ desc->oed_info.lo_warning =
+ cpu_to_be16(page_a2[SSF_TXPOWER_LOW_WARNING]);
+ flags = 0xf; /* All four are valid */
+ flags |= ((0xf & RDP_OED_TXPOWER) << RDP_OED_TYPE_SHIFT);
+ desc->oed_info.function_flags = cpu_to_be32(flags);
+ desc->length = cpu_to_be32(sizeof(desc->oed_info));
+}
+
+
+void
+lpfc_rdp_res_oed_rxpower_desc(struct fc_rdp_oed_sfp_desc *desc,
+ uint8_t *page_a2)
+{
+ uint32_t flags;
+
+ desc->tag = cpu_to_be32(RDP_OED_DESC_TAG);
+
+ desc->oed_info.hi_alarm =
+ cpu_to_be16(page_a2[SSF_RXPOWER_HIGH_ALARM]);
+ desc->oed_info.lo_alarm = cpu_to_be16(page_a2[SSF_RXPOWER_LOW_ALARM]);
+ desc->oed_info.hi_warning =
+ cpu_to_be16(page_a2[SSF_RXPOWER_HIGH_WARNING]);
+ desc->oed_info.lo_warning =
+ cpu_to_be16(page_a2[SSF_RXPOWER_LOW_WARNING]);
+ flags = 0xf; /* All four are valid */
+ flags |= ((0xf & RDP_OED_RXPOWER) << RDP_OED_TYPE_SHIFT);
+ desc->oed_info.function_flags = cpu_to_be32(flags);
+ desc->length = cpu_to_be32(sizeof(desc->oed_info));
+}
+
+void
+lpfc_rdp_res_opd_desc(struct fc_rdp_opd_sfp_desc *desc,
+ uint8_t *page_a0, struct lpfc_vport *vport)
+{
+ desc->tag = cpu_to_be32(RDP_OPD_DESC_TAG);
+ memcpy(desc->opd_info.vendor_name, &page_a0[SSF_VENDOR_NAME], 16);
+ memcpy(desc->opd_info.model_number, &page_a0[SSF_VENDOR_PN], 16);
+ memcpy(desc->opd_info.serial_number, &page_a0[SSF_VENDOR_SN], 16);
+ memcpy(desc->opd_info.revision, &page_a0[SSF_VENDOR_REV], 2);
+ memcpy(desc->opd_info.date, &page_a0[SSF_DATE_CODE], 8);
+ desc->length = cpu_to_be32(sizeof(desc->opd_info));
+}
+
int
lpfc_rdp_res_fec_desc(struct fc_fec_rdp_desc *desc, READ_LNK_VAR *stat)
{
@@ -4776,6 +4918,8 @@
if (rdp_cap == 0)
rdp_cap = RDP_CAP_UNKNOWN;
+ if (phba->cfg_link_speed != LPFC_USER_LINK_SPEED_AUTO)
+ rdp_cap |= RDP_CAP_USER_CONFIGURED;
desc->info.port_speed.capabilities = cpu_to_be16(rdp_cap);
desc->length = cpu_to_be32(sizeof(desc->info));
@@ -4875,6 +5019,19 @@
lpfc_rdp_res_diag_port_names(&rdp_res->diag_port_names_desc, phba);
lpfc_rdp_res_attach_port_names(&rdp_res->attached_port_names_desc,
vport, ndlp);
+ lpfc_rdp_res_bbc_desc(&rdp_res->bbc_desc, &rdp_context->link_stat,
+ vport);
+ lpfc_rdp_res_oed_temp_desc(&rdp_res->oed_temp_desc,
+ rdp_context->page_a2);
+ lpfc_rdp_res_oed_voltage_desc(&rdp_res->oed_voltage_desc,
+ rdp_context->page_a2);
+ lpfc_rdp_res_oed_txbias_desc(&rdp_res->oed_txbias_desc,
+ rdp_context->page_a2);
+ lpfc_rdp_res_oed_txpower_desc(&rdp_res->oed_txpower_desc,
+ rdp_context->page_a2);
+ lpfc_rdp_res_oed_rxpower_desc(&rdp_res->oed_rxpower_desc,
+ rdp_context->page_a2);
+ lpfc_rdp_res_opd_desc(&rdp_res->opd_desc, rdp_context->page_a0, vport);
fec_size = lpfc_rdp_res_fec_desc(&rdp_res->fec_desc,
&rdp_context->link_stat);
rdp_res->length = cpu_to_be32(fec_size + RDP_DESC_PAYLOAD_SIZE);
@@ -7849,8 +8006,9 @@
return;
}
- if ((phba->cfg_fdmi_on > LPFC_FDMI_NO_SUPPORT) &&
- (vport->load_flag & FC_ALLOW_FDMI))
+ if ((phba->cfg_enable_SmartSAN ||
+ (phba->cfg_fdmi_on == LPFC_FDMI_SUPPORT)) &&
+ (vport->load_flag & FC_ALLOW_FDMI))
lpfc_start_fdmi(vport);
}
diff --git a/drivers/scsi/lpfc/lpfc_hbadisc.c b/drivers/scsi/lpfc/lpfc_hbadisc.c
index 25b5dcd..ed22393 100644
--- a/drivers/scsi/lpfc/lpfc_hbadisc.c
+++ b/drivers/scsi/lpfc/lpfc_hbadisc.c
@@ -1,7 +1,7 @@
/*******************************************************************
* This file is part of the Emulex Linux Device Driver for *
* Fibre Channel Host Bus Adapters. *
- * Copyright (C) 2004-2015 Emulex. All rights reserved. *
+ * Copyright (C) 2004-2016 Emulex. All rights reserved. *
* EMULEX and SLI are trademarks of Emulex. *
* www.emulex.com *
* Portions Copyright (C) 2004-2005 Christoph Hellwig *
@@ -4545,7 +4545,8 @@
(!(vport->load_flag & FC_UNLOADING)) &&
(bf_get(lpfc_sli_intf_if_type,
&phba->sli4_hba.sli_intf) ==
- LPFC_SLI_INTF_IF_TYPE_2)) {
+ LPFC_SLI_INTF_IF_TYPE_2) &&
+ (atomic_read(&ndlp->kref.refcount) > 0)) {
mbox->context1 = lpfc_nlp_get(ndlp);
mbox->mbox_cmpl =
lpfc_sli4_unreg_rpi_cmpl_clr;
diff --git a/drivers/scsi/lpfc/lpfc_hw.h b/drivers/scsi/lpfc/lpfc_hw.h
index dd20412..39f0fd0 100644
--- a/drivers/scsi/lpfc/lpfc_hw.h
+++ b/drivers/scsi/lpfc/lpfc_hw.h
@@ -1,7 +1,7 @@
/*******************************************************************
* This file is part of the Emulex Linux Device Driver for *
* Fibre Channel Host Bus Adapters. *
- * Copyright (C) 2004-2015 Emulex. All rights reserved. *
+ * Copyright (C) 2004-2016 Emulex. All rights reserved. *
* EMULEX and SLI are trademarks of Emulex. *
* www.emulex.com *
* *
@@ -1134,9 +1134,10 @@
#define RDP_PS_16GB 0x0400
#define RDP_PS_32GB 0x0200
-#define RDP_CAP_UNKNOWN 0x0001
-#define RDP_PS_UNKNOWN 0x0002
-#define RDP_PS_NOT_ESTABLISHED 0x0001
+#define RDP_CAP_USER_CONFIGURED 0x0002
+#define RDP_CAP_UNKNOWN 0x0001
+#define RDP_PS_UNKNOWN 0x0002
+#define RDP_PS_NOT_ESTABLISHED 0x0001
struct fc_rdp_port_speed {
uint16_t capabilities;
@@ -1192,6 +1193,58 @@
struct fc_rdp_sfp_info sfp_info;
};
+/* Buffer Credit Descriptor */
+struct fc_rdp_bbc_info {
+ uint32_t port_bbc; /* FC_Port buffer-to-buffer credit */
+ uint32_t attached_port_bbc;
+ uint32_t rtt; /* Round trip time */
+};
+#define RDP_BBC_DESC_TAG 0x00010006
+struct fc_rdp_bbc_desc {
+ uint32_t tag;
+ uint32_t length;
+ struct fc_rdp_bbc_info bbc_info;
+};
+
+#define RDP_OED_TEMPERATURE 0x1
+#define RDP_OED_VOLTAGE 0x2
+#define RDP_OED_TXBIAS 0x3
+#define RDP_OED_TXPOWER 0x4
+#define RDP_OED_RXPOWER 0x5
+
+#define RDP_OED_TYPE_SHIFT 28
+/* Optical Element Data descriptor */
+struct fc_rdp_oed_info {
+ uint16_t hi_alarm;
+ uint16_t lo_alarm;
+ uint16_t hi_warning;
+ uint16_t lo_warning;
+ uint32_t function_flags;
+};
+#define RDP_OED_DESC_TAG 0x00010007
+struct fc_rdp_oed_sfp_desc {
+ uint32_t tag;
+ uint32_t length;
+ struct fc_rdp_oed_info oed_info;
+};
+
+/* Optical Product Data descriptor */
+struct fc_rdp_opd_sfp_info {
+ uint8_t vendor_name[16];
+ uint8_t model_number[16];
+ uint8_t serial_number[16];
+ uint8_t reserved[2];
+ uint8_t revision[2];
+ uint8_t date[8];
+};
+
+#define RDP_OPD_DESC_TAG 0x00010008
+struct fc_rdp_opd_sfp_desc {
+ uint32_t tag;
+ uint32_t length;
+ struct fc_rdp_opd_sfp_info opd_info;
+};
+
struct fc_rdp_req_frame {
uint32_t rdp_command; /* ELS command opcode (0x18)*/
uint32_t rdp_des_length; /* RDP Payload Word 1 */
@@ -1208,7 +1261,14 @@
struct fc_rdp_link_error_status_desc link_error_desc; /* Word 13-21 */
struct fc_rdp_port_name_desc diag_port_names_desc; /* Word 22-27 */
struct fc_rdp_port_name_desc attached_port_names_desc;/* Word 28-33 */
- struct fc_fec_rdp_desc fec_desc; /* FC Word 34 - 37 */
+ struct fc_rdp_bbc_desc bbc_desc; /* FC Word 34-38*/
+ struct fc_rdp_oed_sfp_desc oed_temp_desc; /* FC Word 39-43*/
+ struct fc_rdp_oed_sfp_desc oed_voltage_desc; /* FC word 44-48*/
+ struct fc_rdp_oed_sfp_desc oed_txbias_desc; /* FC word 49-53*/
+ struct fc_rdp_oed_sfp_desc oed_txpower_desc; /* FC word 54-58*/
+ struct fc_rdp_oed_sfp_desc oed_rxpower_desc; /* FC word 59-63*/
+ struct fc_rdp_opd_sfp_desc opd_desc; /* FC word 64-80*/
+ struct fc_fec_rdp_desc fec_desc; /* FC word 81-84*/
};
@@ -1216,7 +1276,10 @@
+ sizeof(struct fc_rdp_sfp_desc) \
+ sizeof(struct fc_rdp_port_speed_desc) \
+ sizeof(struct fc_rdp_link_error_status_desc) \
- + (sizeof(struct fc_rdp_port_name_desc) * 2))
+ + (sizeof(struct fc_rdp_port_name_desc) * 2) \
+ + sizeof(struct fc_rdp_bbc_desc) \
+ + (sizeof(struct fc_rdp_oed_sfp_desc) * 5) \
+ + sizeof(struct fc_rdp_opd_sfp_desc))
/******** FDMI ********/
diff --git a/drivers/scsi/lpfc/lpfc_hw4.h b/drivers/scsi/lpfc/lpfc_hw4.h
index 608f941..0c7070b 100644
--- a/drivers/scsi/lpfc/lpfc_hw4.h
+++ b/drivers/scsi/lpfc/lpfc_hw4.h
@@ -1,7 +1,7 @@
/*******************************************************************
* This file is part of the Emulex Linux Device Driver for *
* Fibre Channel Host Bus Adapters. *
- * Copyright (C) 2009-2015 Emulex. All rights reserved. *
+ * Copyright (C) 2009-2016 Emulex. All rights reserved. *
* EMULEX and SLI are trademarks of Emulex. *
* www.emulex.com *
* *
@@ -2557,7 +2557,26 @@
/* SFF-8472 Table 3.1a Diagnostics: Data Fields Address/Page A2 */
-#define SSF_AW_THRESHOLDS 0
+#define SSF_TEMP_HIGH_ALARM 0
+#define SSF_TEMP_LOW_ALARM 2
+#define SSF_TEMP_HIGH_WARNING 4
+#define SSF_TEMP_LOW_WARNING 6
+#define SSF_VOLTAGE_HIGH_ALARM 8
+#define SSF_VOLTAGE_LOW_ALARM 10
+#define SSF_VOLTAGE_HIGH_WARNING 12
+#define SSF_VOLTAGE_LOW_WARNING 14
+#define SSF_BIAS_HIGH_ALARM 16
+#define SSF_BIAS_LOW_ALARM 18
+#define SSF_BIAS_HIGH_WARNING 20
+#define SSF_BIAS_LOW_WARNING 22
+#define SSF_TXPOWER_HIGH_ALARM 24
+#define SSF_TXPOWER_LOW_ALARM 26
+#define SSF_TXPOWER_HIGH_WARNING 28
+#define SSF_TXPOWER_LOW_WARNING 30
+#define SSF_RXPOWER_HIGH_ALARM 32
+#define SSF_RXPOWER_LOW_ALARM 34
+#define SSF_RXPOWER_HIGH_WARNING 36
+#define SSF_RXPOWER_LOW_WARNING 38
#define SSF_EXT_CAL_CONSTANTS 56
#define SSF_CC_DMI 95
#define SFF_TEMPERATURE_B1 96
@@ -2865,6 +2884,9 @@
uint32_t word17;
uint32_t word18;
uint32_t word19;
+#define cfg_ext_embed_cb_SHIFT 0
+#define cfg_ext_embed_cb_MASK 0x00000001
+#define cfg_ext_embed_cb_WORD word19
};
struct lpfc_mbx_get_sli4_parameters {
@@ -3919,6 +3941,9 @@
union lpfc_wqe128 {
uint32_t words[32];
struct lpfc_wqe_generic generic;
+ struct fcp_icmnd64_wqe fcp_icmd;
+ struct fcp_iread64_wqe fcp_iread;
+ struct fcp_iwrite64_wqe fcp_iwrite;
struct xmit_seq64_wqe xmit_sequence;
struct gen_req64_wqe gen_req;
};
diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c
index f57d02c..b43f7ac 100644
--- a/drivers/scsi/lpfc/lpfc_init.c
+++ b/drivers/scsi/lpfc/lpfc_init.c
@@ -1,7 +1,7 @@
/*******************************************************************
* This file is part of the Emulex Linux Device Driver for *
* Fibre Channel Host Bus Adapters. *
- * Copyright (C) 2004-2015 Emulex. All rights reserved. *
+ * Copyright (C) 2004-2016 Emulex. All rights reserved. *
* EMULEX and SLI are trademarks of Emulex. *
* www.emulex.com *
* Portions Copyright (C) 2004-2005 Christoph Hellwig *
@@ -6158,11 +6158,12 @@
* any initial discovery should be completed.
*/
vport->load_flag |= FC_ALLOW_FDMI;
- if (phba->cfg_fdmi_on > LPFC_FDMI_NO_SUPPORT) {
+ if (phba->cfg_enable_SmartSAN ||
+ (phba->cfg_fdmi_on == LPFC_FDMI_SUPPORT)) {
/* Setup appropriate attribute masks */
vport->fdmi_hba_mask = LPFC_FDMI2_HBA_ATTR;
- if (phba->cfg_fdmi_on == LPFC_FDMI_SMART_SAN)
+ if (phba->cfg_enable_SmartSAN)
vport->fdmi_port_mask = LPFC_FDMI2_SMART_ATTR;
else
vport->fdmi_port_mask = LPFC_FDMI2_PORT_ATTR;
@@ -7264,8 +7265,15 @@
phba->sli4_hba.fcp_cq[idx] = qdesc;
/* Create Fast Path FCP WQs */
- qdesc = lpfc_sli4_queue_alloc(phba, phba->sli4_hba.wq_esize,
- phba->sli4_hba.wq_ecount);
+ if (phba->fcp_embed_io) {
+ qdesc = lpfc_sli4_queue_alloc(phba,
+ LPFC_WQE128_SIZE,
+ LPFC_WQE128_DEF_COUNT);
+ } else {
+ qdesc = lpfc_sli4_queue_alloc(phba,
+ phba->sli4_hba.wq_esize,
+ phba->sli4_hba.wq_ecount);
+ }
if (!qdesc) {
lpfc_printf_log(phba, KERN_ERR, LOG_INIT,
"0503 Failed allocate fast-path FCP "
@@ -9510,6 +9518,15 @@
if (sli4_params->sge_supp_len > LPFC_MAX_SGE_SIZE)
sli4_params->sge_supp_len = LPFC_MAX_SGE_SIZE;
+ /*
+ * Issue IOs with CDB embedded in WQE to minimized the number
+ * of DMAs the firmware has to do. Setting this to 1 also forces
+ * the driver to use 128 bytes WQEs for FCP IOs.
+ */
+ if (bf_get(cfg_ext_embed_cb, mbx_sli4_parameters))
+ phba->fcp_embed_io = 1;
+ else
+ phba->fcp_embed_io = 0;
return 0;
}
diff --git a/drivers/scsi/lpfc/lpfc_mbox.c b/drivers/scsi/lpfc/lpfc_mbox.c
index f87f90e..12dbe99 100644
--- a/drivers/scsi/lpfc/lpfc_mbox.c
+++ b/drivers/scsi/lpfc/lpfc_mbox.c
@@ -1,7 +1,7 @@
/*******************************************************************
* This file is part of the Emulex Linux Device Driver for *
* Fibre Channel Host Bus Adapters. *
- * Copyright (C) 2004-2015 Emulex. All rights reserved. *
+ * Copyright (C) 2004-2016 Emulex. All rights reserved. *
* EMULEX and SLI are trademarks of Emulex. *
* www.emulex.com *
* Portions Copyright (C) 2004-2005 Christoph Hellwig *
@@ -2145,10 +2145,12 @@
reg_vfi->wwn[1] = cpu_to_le32(reg_vfi->wwn[1]);
reg_vfi->e_d_tov = phba->fc_edtov;
reg_vfi->r_a_tov = phba->fc_ratov;
- reg_vfi->bde.addrHigh = putPaddrHigh(phys);
- reg_vfi->bde.addrLow = putPaddrLow(phys);
- reg_vfi->bde.tus.f.bdeSize = sizeof(vport->fc_sparam);
- reg_vfi->bde.tus.f.bdeFlags = BUFF_TYPE_BDE_64;
+ if (phys) {
+ reg_vfi->bde.addrHigh = putPaddrHigh(phys);
+ reg_vfi->bde.addrLow = putPaddrLow(phys);
+ reg_vfi->bde.tus.f.bdeSize = sizeof(vport->fc_sparam);
+ reg_vfi->bde.tus.f.bdeFlags = BUFF_TYPE_BDE_64;
+ }
bf_set(lpfc_reg_vfi_nport_id, reg_vfi, vport->fc_myDID);
/* Only FC supports upd bit */
diff --git a/drivers/scsi/lpfc/lpfc_mem.c b/drivers/scsi/lpfc/lpfc_mem.c
index 4fb3581..3fa6533 100644
--- a/drivers/scsi/lpfc/lpfc_mem.c
+++ b/drivers/scsi/lpfc/lpfc_mem.c
@@ -231,13 +231,15 @@
if (phba->lpfc_hbq_pool)
pci_pool_destroy(phba->lpfc_hbq_pool);
phba->lpfc_hbq_pool = NULL;
- mempool_destroy(phba->rrq_pool);
+
+ if (phba->rrq_pool)
+ mempool_destroy(phba->rrq_pool);
phba->rrq_pool = NULL;
/* Free NLP memory pool */
mempool_destroy(phba->nlp_mem_pool);
phba->nlp_mem_pool = NULL;
- if (phba->sli_rev == LPFC_SLI_REV4) {
+ if (phba->sli_rev == LPFC_SLI_REV4 && phba->active_rrq_pool) {
mempool_destroy(phba->active_rrq_pool);
phba->active_rrq_pool = NULL;
}
diff --git a/drivers/scsi/lpfc/lpfc_nportdisc.c b/drivers/scsi/lpfc/lpfc_nportdisc.c
index 193733e..56a3df4 100644
--- a/drivers/scsi/lpfc/lpfc_nportdisc.c
+++ b/drivers/scsi/lpfc/lpfc_nportdisc.c
@@ -1,7 +1,7 @@
/*******************************************************************
* This file is part of the Emulex Linux Device Driver for *
* Fibre Channel Host Bus Adapters. *
- * Copyright (C) 2004-2015 Emulex. All rights reserved. *
+ * Copyright (C) 2004-2016 Emulex. All rights reserved. *
* EMULEX and SLI are trademarks of Emulex. *
* www.emulex.com *
* Portions Copyright (C) 2004-2005 Christoph Hellwig *
@@ -1512,6 +1512,7 @@
if ((mb = phba->sli.mbox_active)) {
if ((mb->u.mb.mbxCommand == MBX_REG_LOGIN64) &&
(ndlp == (struct lpfc_nodelist *) mb->context2)) {
+ ndlp->nlp_flag &= ~NLP_REG_LOGIN_SEND;
lpfc_nlp_put(ndlp);
mb->context2 = NULL;
mb->mbox_cmpl = lpfc_sli_def_mbox_cmpl;
@@ -1527,6 +1528,7 @@
__lpfc_mbuf_free(phba, mp->virt, mp->phys);
kfree(mp);
}
+ ndlp->nlp_flag &= ~NLP_REG_LOGIN_SEND;
lpfc_nlp_put(ndlp);
list_del(&mb->list);
phba->sli.mboxq_cnt--;
diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c
index 2207726..70edf21 100644
--- a/drivers/scsi/lpfc/lpfc_sli.c
+++ b/drivers/scsi/lpfc/lpfc_sli.c
@@ -1,7 +1,7 @@
/*******************************************************************
* This file is part of the Emulex Linux Device Driver for *
* Fibre Channel Host Bus Adapters. *
- * Copyright (C) 2004-2015 Emulex. All rights reserved. *
+ * Copyright (C) 2004-2016 Emulex. All rights reserved. *
* EMULEX and SLI are trademarks of Emulex. *
* www.emulex.com *
* Portions Copyright (C) 2004-2005 Christoph Hellwig *
@@ -2000,10 +2000,9 @@
* @phba: Pointer to HBA context object.
* @tag: Tag of the hbq buffer.
*
- * This function is called with hbalock held. This function searches
- * for the hbq buffer associated with the given tag in the hbq buffer
- * list. If it finds the hbq buffer, it returns the hbq_buffer other wise
- * it returns NULL.
+ * This function searches for the hbq buffer associated with the given tag in
+ * the hbq buffer list. If it finds the hbq buffer, it returns the hbq_buffer
+ * otherwise it returns NULL.
**/
static struct hbq_dmabuf *
lpfc_sli_hbqbuf_find(struct lpfc_hba *phba, uint32_t tag)
@@ -2012,8 +2011,6 @@
struct hbq_dmabuf *hbq_buf;
uint32_t hbqno;
- lockdep_assert_held(&phba->hbalock);
-
hbqno = tag >> 16;
if (hbqno >= LPFC_MAX_HBQS)
return NULL;
@@ -2211,6 +2208,7 @@
rpi = pmb->u.mb.un.varWords[0];
vpi = pmb->u.mb.un.varRegLogin.vpi;
lpfc_unreg_login(phba, vpi, rpi, pmb);
+ pmb->vport = vport;
pmb->mbox_cmpl = lpfc_sli_def_mbox_cmpl;
rc = lpfc_sli_issue_mbox(phba, pmb, MBX_NOWAIT);
if (rc != MBX_NOT_FINISHED)
@@ -4688,6 +4686,7 @@
break;
}
+ phba->fcp_embed_io = 0; /* SLI4 FC support only */
rc = lpfc_sli_config_port(phba, mode);
@@ -6320,10 +6319,12 @@
mqe = &mboxq->u.mqe;
phba->sli_rev = bf_get(lpfc_mbx_rd_rev_sli_lvl, &mqe->un.read_rev);
- if (bf_get(lpfc_mbx_rd_rev_fcoe, &mqe->un.read_rev))
+ if (bf_get(lpfc_mbx_rd_rev_fcoe, &mqe->un.read_rev)) {
phba->hba_flag |= HBA_FCOE_MODE;
- else
+ phba->fcp_embed_io = 0; /* SLI4 FC support only */
+ } else {
phba->hba_flag &= ~HBA_FCOE_MODE;
+ }
if (bf_get(lpfc_mbx_rd_rev_cee_ver, &mqe->un.read_rev) ==
LPFC_DCBX_CEE_MODE)
@@ -8218,12 +8219,15 @@
else
command_type = ELS_COMMAND_NON_FIP;
+ if (phba->fcp_embed_io)
+ memset(wqe, 0, sizeof(union lpfc_wqe128));
/* Some of the fields are in the right position already */
memcpy(wqe, &iocbq->iocb, sizeof(union lpfc_wqe));
- abort_tag = (uint32_t) iocbq->iotag;
- xritag = iocbq->sli4_xritag;
wqe->generic.wqe_com.word7 = 0; /* The ct field has moved so reset */
wqe->generic.wqe_com.word10 = 0;
+
+ abort_tag = (uint32_t) iocbq->iotag;
+ xritag = iocbq->sli4_xritag;
/* words0-2 bpl convert bde */
if (iocbq->iocb.un.genreq64.bdl.bdeFlags == BUFF_TYPE_BLP_64) {
numBdes = iocbq->iocb.un.genreq64.bdl.bdeSize /
@@ -8372,11 +8376,9 @@
iocbq->iocb.ulpFCP2Rcvy);
bf_set(wqe_lnk, &wqe->fcp_iwrite.wqe_com, iocbq->iocb.ulpXS);
/* Always open the exchange */
- bf_set(wqe_xc, &wqe->fcp_iwrite.wqe_com, 0);
bf_set(wqe_iod, &wqe->fcp_iwrite.wqe_com, LPFC_WQE_IOD_WRITE);
bf_set(wqe_lenloc, &wqe->fcp_iwrite.wqe_com,
LPFC_WQE_LENLOC_WORD4);
- bf_set(wqe_ebde_cnt, &wqe->fcp_iwrite.wqe_com, 0);
bf_set(wqe_pu, &wqe->fcp_iwrite.wqe_com, iocbq->iocb.ulpPU);
bf_set(wqe_dbde, &wqe->fcp_iwrite.wqe_com, 1);
if (iocbq->iocb_flag & LPFC_IO_OAS) {
@@ -8387,6 +8389,35 @@
(phba->cfg_XLanePriority << 1));
}
}
+ /* Note, word 10 is already initialized to 0 */
+
+ if (phba->fcp_embed_io) {
+ struct lpfc_scsi_buf *lpfc_cmd;
+ struct sli4_sge *sgl;
+ union lpfc_wqe128 *wqe128;
+ struct fcp_cmnd *fcp_cmnd;
+ uint32_t *ptr;
+
+ /* 128 byte wqe support here */
+ wqe128 = (union lpfc_wqe128 *)wqe;
+
+ lpfc_cmd = iocbq->context1;
+ sgl = (struct sli4_sge *)lpfc_cmd->fcp_bpl;
+ fcp_cmnd = lpfc_cmd->fcp_cmnd;
+
+ /* Word 0-2 - FCP_CMND */
+ wqe128->generic.bde.tus.f.bdeFlags =
+ BUFF_TYPE_BDE_IMMED;
+ wqe128->generic.bde.tus.f.bdeSize = sgl->sge_len;
+ wqe128->generic.bde.addrHigh = 0;
+ wqe128->generic.bde.addrLow = 88; /* Word 22 */
+
+ bf_set(wqe_wqes, &wqe128->fcp_iwrite.wqe_com, 1);
+
+ /* Word 22-29 FCP CMND Payload */
+ ptr = &wqe128->words[22];
+ memcpy(ptr, fcp_cmnd, sizeof(struct fcp_cmnd));
+ }
break;
case CMD_FCP_IREAD64_CR:
/* word3 iocb=iotag wqe=payload_offset_len */
@@ -8401,11 +8432,9 @@
iocbq->iocb.ulpFCP2Rcvy);
bf_set(wqe_lnk, &wqe->fcp_iread.wqe_com, iocbq->iocb.ulpXS);
/* Always open the exchange */
- bf_set(wqe_xc, &wqe->fcp_iread.wqe_com, 0);
bf_set(wqe_iod, &wqe->fcp_iread.wqe_com, LPFC_WQE_IOD_READ);
bf_set(wqe_lenloc, &wqe->fcp_iread.wqe_com,
LPFC_WQE_LENLOC_WORD4);
- bf_set(wqe_ebde_cnt, &wqe->fcp_iread.wqe_com, 0);
bf_set(wqe_pu, &wqe->fcp_iread.wqe_com, iocbq->iocb.ulpPU);
bf_set(wqe_dbde, &wqe->fcp_iread.wqe_com, 1);
if (iocbq->iocb_flag & LPFC_IO_OAS) {
@@ -8416,6 +8445,35 @@
(phba->cfg_XLanePriority << 1));
}
}
+ /* Note, word 10 is already initialized to 0 */
+
+ if (phba->fcp_embed_io) {
+ struct lpfc_scsi_buf *lpfc_cmd;
+ struct sli4_sge *sgl;
+ union lpfc_wqe128 *wqe128;
+ struct fcp_cmnd *fcp_cmnd;
+ uint32_t *ptr;
+
+ /* 128 byte wqe support here */
+ wqe128 = (union lpfc_wqe128 *)wqe;
+
+ lpfc_cmd = iocbq->context1;
+ sgl = (struct sli4_sge *)lpfc_cmd->fcp_bpl;
+ fcp_cmnd = lpfc_cmd->fcp_cmnd;
+
+ /* Word 0-2 - FCP_CMND */
+ wqe128->generic.bde.tus.f.bdeFlags =
+ BUFF_TYPE_BDE_IMMED;
+ wqe128->generic.bde.tus.f.bdeSize = sgl->sge_len;
+ wqe128->generic.bde.addrHigh = 0;
+ wqe128->generic.bde.addrLow = 88; /* Word 22 */
+
+ bf_set(wqe_wqes, &wqe128->fcp_iread.wqe_com, 1);
+
+ /* Word 22-29 FCP CMND Payload */
+ ptr = &wqe128->words[22];
+ memcpy(ptr, fcp_cmnd, sizeof(struct fcp_cmnd));
+ }
break;
case CMD_FCP_ICMND64_CR:
/* word3 iocb=iotag wqe=payload_offset_len */
@@ -8427,13 +8485,11 @@
/* word3 iocb=IO_TAG wqe=reserved */
bf_set(wqe_pu, &wqe->fcp_icmd.wqe_com, 0);
/* Always open the exchange */
- bf_set(wqe_xc, &wqe->fcp_icmd.wqe_com, 0);
bf_set(wqe_dbde, &wqe->fcp_icmd.wqe_com, 1);
bf_set(wqe_iod, &wqe->fcp_icmd.wqe_com, LPFC_WQE_IOD_WRITE);
bf_set(wqe_qosd, &wqe->fcp_icmd.wqe_com, 1);
bf_set(wqe_lenloc, &wqe->fcp_icmd.wqe_com,
LPFC_WQE_LENLOC_NONE);
- bf_set(wqe_ebde_cnt, &wqe->fcp_icmd.wqe_com, 0);
bf_set(wqe_erp, &wqe->fcp_icmd.wqe_com,
iocbq->iocb.ulpFCP2Rcvy);
if (iocbq->iocb_flag & LPFC_IO_OAS) {
@@ -8444,6 +8500,35 @@
(phba->cfg_XLanePriority << 1));
}
}
+ /* Note, word 10 is already initialized to 0 */
+
+ if (phba->fcp_embed_io) {
+ struct lpfc_scsi_buf *lpfc_cmd;
+ struct sli4_sge *sgl;
+ union lpfc_wqe128 *wqe128;
+ struct fcp_cmnd *fcp_cmnd;
+ uint32_t *ptr;
+
+ /* 128 byte wqe support here */
+ wqe128 = (union lpfc_wqe128 *)wqe;
+
+ lpfc_cmd = iocbq->context1;
+ sgl = (struct sli4_sge *)lpfc_cmd->fcp_bpl;
+ fcp_cmnd = lpfc_cmd->fcp_cmnd;
+
+ /* Word 0-2 - FCP_CMND */
+ wqe128->generic.bde.tus.f.bdeFlags =
+ BUFF_TYPE_BDE_IMMED;
+ wqe128->generic.bde.tus.f.bdeSize = sgl->sge_len;
+ wqe128->generic.bde.addrHigh = 0;
+ wqe128->generic.bde.addrLow = 88; /* Word 22 */
+
+ bf_set(wqe_wqes, &wqe128->fcp_icmd.wqe_com, 1);
+
+ /* Word 22-29 FCP CMND Payload */
+ ptr = &wqe128->words[22];
+ memcpy(ptr, fcp_cmnd, sizeof(struct fcp_cmnd));
+ }
break;
case CMD_GEN_REQUEST64_CR:
/* For this command calculate the xmit length of the
@@ -8675,12 +8760,19 @@
struct lpfc_iocbq *piocb, uint32_t flag)
{
struct lpfc_sglq *sglq;
- union lpfc_wqe wqe;
+ union lpfc_wqe *wqe;
+ union lpfc_wqe128 wqe128;
struct lpfc_queue *wq;
struct lpfc_sli_ring *pring = &phba->sli.ring[ring_number];
lockdep_assert_held(&phba->hbalock);
+ /*
+ * The WQE can be either 64 or 128 bytes,
+ * so allocate space on the stack assuming the largest.
+ */
+ wqe = (union lpfc_wqe *)&wqe128;
+
if (piocb->sli4_xritag == NO_XRI) {
if (piocb->iocb.ulpCommand == CMD_ABORT_XRI_CN ||
piocb->iocb.ulpCommand == CMD_CLOSE_XRI_CN)
@@ -8727,7 +8819,7 @@
return IOCB_ERROR;
}
- if (lpfc_sli4_iocb2wqe(phba, piocb, &wqe))
+ if (lpfc_sli4_iocb2wqe(phba, piocb, wqe))
return IOCB_ERROR;
if ((piocb->iocb_flag & LPFC_IO_FCP) ||
@@ -8737,12 +8829,12 @@
} else {
wq = phba->sli4_hba.oas_wq;
}
- if (lpfc_sli4_wq_put(wq, &wqe))
+ if (lpfc_sli4_wq_put(wq, wqe))
return IOCB_ERROR;
} else {
if (unlikely(!phba->sli4_hba.els_wq))
return IOCB_ERROR;
- if (lpfc_sli4_wq_put(phba->sli4_hba.els_wq, &wqe))
+ if (lpfc_sli4_wq_put(phba->sli4_hba.els_wq, wqe))
return IOCB_ERROR;
}
lpfc_sli_ringtxcmpl_put(phba, pring, piocb);
@@ -8757,9 +8849,9 @@
* pointer from the lpfc_hba struct.
*
* Return codes:
- * IOCB_ERROR - Error
- * IOCB_SUCCESS - Success
- * IOCB_BUSY - Busy
+ * IOCB_ERROR - Error
+ * IOCB_SUCCESS - Success
+ * IOCB_BUSY - Busy
**/
int
__lpfc_sli_issue_iocb(struct lpfc_hba *phba, uint32_t ring_number,
diff --git a/drivers/scsi/lpfc/lpfc_version.h b/drivers/scsi/lpfc/lpfc_version.h
index 4dc2256..fa0d531 100644
--- a/drivers/scsi/lpfc/lpfc_version.h
+++ b/drivers/scsi/lpfc/lpfc_version.h
@@ -1,7 +1,7 @@
/*******************************************************************
* This file is part of the Emulex Linux Device Driver for *
* Fibre Channel Host Bus Adapters. *
- * Copyright (C) 2004-2015 Emulex. All rights reserved. *
+ * Copyright (C) 2004-2016 Emulex. All rights reserved. *
* EMULEX and SLI are trademarks of Emulex. *
* www.emulex.com *
* *
@@ -18,7 +18,7 @@
* included with this package. *
*******************************************************************/
-#define LPFC_DRIVER_VERSION "11.0.0.10."
+#define LPFC_DRIVER_VERSION "11.1.0.0."
#define LPFC_DRIVER_NAME "lpfc"
/* Used for SLI 2/3 */
@@ -30,4 +30,4 @@
#define LPFC_MODULE_DESC "Emulex LightPulse Fibre Channel SCSI driver " \
LPFC_DRIVER_VERSION
-#define LPFC_COPYRIGHT "Copyright(c) 2004-2015 Emulex. All rights reserved."
+#define LPFC_COPYRIGHT "Copyright(c) 2004-2016 Emulex. All rights reserved."
diff --git a/drivers/scsi/lpfc/lpfc_vport.c b/drivers/scsi/lpfc/lpfc_vport.c
index b3f85de..c27f4b7 100644
--- a/drivers/scsi/lpfc/lpfc_vport.c
+++ b/drivers/scsi/lpfc/lpfc_vport.c
@@ -1,7 +1,7 @@
/*******************************************************************
* This file is part of the Emulex Linux Device Driver for *
* Fibre Channel Host Bus Adapters. *
- * Copyright (C) 2004-2013 Emulex. All rights reserved. *
+ * Copyright (C) 2004-2016 Emulex. All rights reserved. *
* EMULEX and SLI are trademarks of Emulex. *
* www.emulex.com *
* Portions Copyright (C) 2004-2005 Christoph Hellwig *
@@ -395,7 +395,8 @@
/* At this point we are fully registered with SCSI Layer. */
vport->load_flag |= FC_ALLOW_FDMI;
- if (phba->cfg_fdmi_on > LPFC_FDMI_NO_SUPPORT) {
+ if (phba->cfg_enable_SmartSAN ||
+ (phba->cfg_fdmi_on == LPFC_FDMI_SUPPORT)) {
/* Setup appropriate attribute masks */
vport->fdmi_hba_mask = phba->pport->fdmi_hba_mask;
vport->fdmi_port_mask = phba->pport->fdmi_port_mask;
diff --git a/drivers/scsi/mac_scsi.c b/drivers/scsi/mac_scsi.c
index bb23813..a590089 100644
--- a/drivers/scsi/mac_scsi.c
+++ b/drivers/scsi/mac_scsi.c
@@ -28,24 +28,23 @@
/* Definitions for the core NCR5380 driver. */
-#define PSEUDO_DMA
-
-#define NCR5380_implementation_fields unsigned char *pdma_base
+#define NCR5380_implementation_fields unsigned char *pdma_base; \
+ int pdma_residual
#define NCR5380_read(reg) macscsi_read(instance, reg)
#define NCR5380_write(reg, value) macscsi_write(instance, reg, value)
-#define NCR5380_pread macscsi_pread
-#define NCR5380_pwrite macscsi_pwrite
-#define NCR5380_dma_xfer_len(instance, cmd, phase) (cmd->transfersize)
+#define NCR5380_dma_xfer_len(instance, cmd, phase) \
+ macscsi_dma_xfer_len(instance, cmd)
+#define NCR5380_dma_recv_setup macscsi_pread
+#define NCR5380_dma_send_setup macscsi_pwrite
+#define NCR5380_dma_residual(instance) (hostdata->pdma_residual)
#define NCR5380_intr macscsi_intr
#define NCR5380_queue_command macscsi_queue_command
#define NCR5380_abort macscsi_abort
#define NCR5380_bus_reset macscsi_bus_reset
#define NCR5380_info macscsi_info
-#define NCR5380_show_info macscsi_show_info
-#define NCR5380_write_info macscsi_write_info
#include "NCR5380.h"
@@ -57,8 +56,6 @@
module_param(setup_sg_tablesize, int, 0);
static int setup_use_pdma = -1;
module_param(setup_use_pdma, int, 0);
-static int setup_use_tagged_queuing = -1;
-module_param(setup_use_tagged_queuing, int, 0);
static int setup_hostid = -1;
module_param(setup_hostid, int, 0);
static int setup_toshiba_delay = -1;
@@ -97,8 +94,7 @@
setup_sg_tablesize = ints[3];
if (ints[0] >= 4)
setup_hostid = ints[4];
- if (ints[0] >= 5)
- setup_use_tagged_queuing = ints[5];
+ /* ints[5] (use_tagged_queuing) is ignored */
if (ints[0] >= 6)
setup_use_pdma = ints[6];
if (ints[0] >= 7)
@@ -109,19 +105,9 @@
__setup("mac5380=", mac_scsi_setup);
#endif /* !MODULE */
-#ifdef PSEUDO_DMA
-/*
- Pseudo-DMA: (Ove Edlund)
- The code attempts to catch bus errors that occur if one for example
- "trips over the cable".
- XXX: Since bus errors in the PDMA routines never happen on my
- computer, the bus error code is untested.
- If the code works as intended, a bus error results in Pseudo-DMA
- being disabled, meaning that the driver switches to slow handshake.
- If bus errors are NOT extremely rare, this has to be changed.
-*/
+/* Pseudo DMA asm originally by Ove Edlund */
-#define CP_IO_TO_MEM(s,d,len) \
+#define CP_IO_TO_MEM(s,d,n) \
__asm__ __volatile__ \
(" cmp.w #4,%2\n" \
" bls 8f\n" \
@@ -158,61 +144,73 @@
" 9: \n" \
".section .fixup,\"ax\"\n" \
" .even\n" \
- "90: moveq.l #1, %2\n" \
+ "91: moveq.l #1, %2\n" \
+ " jra 9b\n" \
+ "94: moveq.l #4, %2\n" \
" jra 9b\n" \
".previous\n" \
".section __ex_table,\"a\"\n" \
" .align 4\n" \
- " .long 1b,90b\n" \
- " .long 3b,90b\n" \
- " .long 31b,90b\n" \
- " .long 32b,90b\n" \
- " .long 33b,90b\n" \
- " .long 34b,90b\n" \
- " .long 35b,90b\n" \
- " .long 36b,90b\n" \
- " .long 37b,90b\n" \
- " .long 5b,90b\n" \
- " .long 7b,90b\n" \
+ " .long 1b,91b\n" \
+ " .long 3b,94b\n" \
+ " .long 31b,94b\n" \
+ " .long 32b,94b\n" \
+ " .long 33b,94b\n" \
+ " .long 34b,94b\n" \
+ " .long 35b,94b\n" \
+ " .long 36b,94b\n" \
+ " .long 37b,94b\n" \
+ " .long 5b,94b\n" \
+ " .long 7b,91b\n" \
".previous" \
- : "=a"(s), "=a"(d), "=d"(len) \
- : "0"(s), "1"(d), "2"(len) \
+ : "=a"(s), "=a"(d), "=d"(n) \
+ : "0"(s), "1"(d), "2"(n) \
: "d0")
static int macscsi_pread(struct Scsi_Host *instance,
unsigned char *dst, int len)
{
struct NCR5380_hostdata *hostdata = shost_priv(instance);
- unsigned char *d;
- unsigned char *s;
+ unsigned char *s = hostdata->pdma_base + (INPUT_DATA_REG << 4);
+ unsigned char *d = dst;
+ int n = len;
+ int transferred;
- s = hostdata->pdma_base + (INPUT_DATA_REG << 4);
- d = dst;
+ while (!NCR5380_poll_politely(instance, BUS_AND_STATUS_REG,
+ BASR_DRQ | BASR_PHASE_MATCH,
+ BASR_DRQ | BASR_PHASE_MATCH, HZ / 64)) {
+ CP_IO_TO_MEM(s, d, n);
- /* These conditions are derived from MacOS */
+ transferred = d - dst - n;
+ hostdata->pdma_residual = len - transferred;
- while (!(NCR5380_read(BUS_AND_STATUS_REG) & BASR_DRQ) &&
- !(NCR5380_read(STATUS_REG) & SR_REQ))
- ;
+ /* No bus error. */
+ if (n == 0)
+ return 0;
- if (!(NCR5380_read(BUS_AND_STATUS_REG) & BASR_DRQ) &&
- (NCR5380_read(BUS_AND_STATUS_REG) & BASR_PHASE_MATCH)) {
- pr_err("Error in macscsi_pread\n");
- return -1;
+ /* Target changed phase early? */
+ if (NCR5380_poll_politely2(instance, STATUS_REG, SR_REQ, SR_REQ,
+ BUS_AND_STATUS_REG, BASR_ACK, BASR_ACK, HZ / 64) < 0)
+ scmd_printk(KERN_ERR, hostdata->connected,
+ "%s: !REQ and !ACK\n", __func__);
+ if (!(NCR5380_read(BUS_AND_STATUS_REG) & BASR_PHASE_MATCH))
+ return 0;
+
+ dsprintk(NDEBUG_PSEUDO_DMA, instance,
+ "%s: bus error (%d/%d)\n", __func__, transferred, len);
+ NCR5380_dprint(NDEBUG_PSEUDO_DMA, instance);
+ d = dst + transferred;
+ n = len - transferred;
}
- CP_IO_TO_MEM(s, d, len);
-
- if (len != 0) {
- pr_notice("Bus error in macscsi_pread\n");
- return -1;
- }
-
- return 0;
+ scmd_printk(KERN_ERR, hostdata->connected,
+ "%s: phase mismatch or !DRQ\n", __func__);
+ NCR5380_dprint(NDEBUG_PSEUDO_DMA, instance);
+ return -1;
}
-#define CP_MEM_TO_IO(s,d,len) \
+#define CP_MEM_TO_IO(s,d,n) \
__asm__ __volatile__ \
(" cmp.w #4,%2\n" \
" bls 8f\n" \
@@ -249,59 +247,89 @@
" 9: \n" \
".section .fixup,\"ax\"\n" \
" .even\n" \
- "90: moveq.l #1, %2\n" \
+ "91: moveq.l #1, %2\n" \
+ " jra 9b\n" \
+ "94: moveq.l #4, %2\n" \
" jra 9b\n" \
".previous\n" \
".section __ex_table,\"a\"\n" \
" .align 4\n" \
- " .long 1b,90b\n" \
- " .long 3b,90b\n" \
- " .long 31b,90b\n" \
- " .long 32b,90b\n" \
- " .long 33b,90b\n" \
- " .long 34b,90b\n" \
- " .long 35b,90b\n" \
- " .long 36b,90b\n" \
- " .long 37b,90b\n" \
- " .long 5b,90b\n" \
- " .long 7b,90b\n" \
+ " .long 1b,91b\n" \
+ " .long 3b,94b\n" \
+ " .long 31b,94b\n" \
+ " .long 32b,94b\n" \
+ " .long 33b,94b\n" \
+ " .long 34b,94b\n" \
+ " .long 35b,94b\n" \
+ " .long 36b,94b\n" \
+ " .long 37b,94b\n" \
+ " .long 5b,94b\n" \
+ " .long 7b,91b\n" \
".previous" \
- : "=a"(s), "=a"(d), "=d"(len) \
- : "0"(s), "1"(d), "2"(len) \
+ : "=a"(s), "=a"(d), "=d"(n) \
+ : "0"(s), "1"(d), "2"(n) \
: "d0")
static int macscsi_pwrite(struct Scsi_Host *instance,
unsigned char *src, int len)
{
struct NCR5380_hostdata *hostdata = shost_priv(instance);
- unsigned char *s;
- unsigned char *d;
+ unsigned char *s = src;
+ unsigned char *d = hostdata->pdma_base + (OUTPUT_DATA_REG << 4);
+ int n = len;
+ int transferred;
- s = src;
- d = hostdata->pdma_base + (OUTPUT_DATA_REG << 4);
+ while (!NCR5380_poll_politely(instance, BUS_AND_STATUS_REG,
+ BASR_DRQ | BASR_PHASE_MATCH,
+ BASR_DRQ | BASR_PHASE_MATCH, HZ / 64)) {
+ CP_MEM_TO_IO(s, d, n);
- /* These conditions are derived from MacOS */
+ transferred = s - src - n;
+ hostdata->pdma_residual = len - transferred;
- while (!(NCR5380_read(BUS_AND_STATUS_REG) & BASR_DRQ) &&
- (!(NCR5380_read(STATUS_REG) & SR_REQ) ||
- (NCR5380_read(BUS_AND_STATUS_REG) & BASR_PHASE_MATCH)))
- ;
+ /* Target changed phase early? */
+ if (NCR5380_poll_politely2(instance, STATUS_REG, SR_REQ, SR_REQ,
+ BUS_AND_STATUS_REG, BASR_ACK, BASR_ACK, HZ / 64) < 0)
+ scmd_printk(KERN_ERR, hostdata->connected,
+ "%s: !REQ and !ACK\n", __func__);
+ if (!(NCR5380_read(BUS_AND_STATUS_REG) & BASR_PHASE_MATCH))
+ return 0;
- if (!(NCR5380_read(BUS_AND_STATUS_REG) & BASR_DRQ)) {
- pr_err("Error in macscsi_pwrite\n");
- return -1;
+ /* No bus error. */
+ if (n == 0) {
+ if (NCR5380_poll_politely(instance, TARGET_COMMAND_REG,
+ TCR_LAST_BYTE_SENT,
+ TCR_LAST_BYTE_SENT, HZ / 64) < 0)
+ scmd_printk(KERN_ERR, hostdata->connected,
+ "%s: Last Byte Sent timeout\n", __func__);
+ return 0;
+ }
+
+ dsprintk(NDEBUG_PSEUDO_DMA, instance,
+ "%s: bus error (%d/%d)\n", __func__, transferred, len);
+ NCR5380_dprint(NDEBUG_PSEUDO_DMA, instance);
+ s = src + transferred;
+ n = len - transferred;
}
- CP_MEM_TO_IO(s, d, len);
+ scmd_printk(KERN_ERR, hostdata->connected,
+ "%s: phase mismatch or !DRQ\n", __func__);
+ NCR5380_dprint(NDEBUG_PSEUDO_DMA, instance);
- if (len != 0) {
- pr_notice("Bus error in macscsi_pwrite\n");
- return -1;
- }
-
- return 0;
+ return -1;
}
-#endif
+
+static int macscsi_dma_xfer_len(struct Scsi_Host *instance,
+ struct scsi_cmnd *cmd)
+{
+ struct NCR5380_hostdata *hostdata = shost_priv(instance);
+
+ if (hostdata->flags & FLAG_NO_PSEUDO_DMA ||
+ cmd->SCp.this_residual < 16)
+ return 0;
+
+ return cmd->SCp.this_residual;
+}
#include "NCR5380.c"
@@ -311,8 +339,6 @@
static struct scsi_host_template mac_scsi_template = {
.module = THIS_MODULE,
.proc_name = DRV_MODULE_NAME,
- .show_info = macscsi_show_info,
- .write_info = macscsi_write_info,
.name = "Macintosh NCR5380 SCSI",
.info = macscsi_info,
.queuecommand = macscsi_queue_command,
@@ -320,7 +346,7 @@
.eh_bus_reset_handler = macscsi_bus_reset,
.can_queue = 16,
.this_id = 7,
- .sg_tablesize = SG_ALL,
+ .sg_tablesize = 1,
.cmd_per_lun = 2,
.use_clustering = DISABLE_CLUSTERING,
.cmd_size = NCR5380_CMD_SIZE,
@@ -338,9 +364,7 @@
if (!pio_mem)
return -ENODEV;
-#ifdef PSEUDO_DMA
pdma_mem = platform_get_resource(pdev, IORESOURCE_MEM, 1);
-#endif
irq = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
@@ -358,8 +382,6 @@
mac_scsi_template.sg_tablesize = setup_sg_tablesize;
if (setup_hostid >= 0)
mac_scsi_template.this_id = setup_hostid & 7;
- if (setup_use_pdma < 0)
- setup_use_pdma = 0;
instance = scsi_host_alloc(&mac_scsi_template,
sizeof(struct NCR5380_hostdata));
@@ -379,12 +401,9 @@
} else
host_flags |= FLAG_NO_PSEUDO_DMA;
-#ifdef SUPPORT_TAGS
- host_flags |= setup_use_tagged_queuing > 0 ? FLAG_TAGGED_QUEUING : 0;
-#endif
host_flags |= setup_toshiba_delay > 0 ? FLAG_TOSHIBA_DELAY : 0;
- error = NCR5380_init(instance, host_flags);
+ error = NCR5380_init(instance, host_flags | FLAG_LATE_DMA_SETUP);
if (error)
goto fail_init;
diff --git a/drivers/scsi/megaraid/megaraid_sas.h b/drivers/scsi/megaraid/megaraid_sas.h
index fce414a..ca86c88 100644
--- a/drivers/scsi/megaraid/megaraid_sas.h
+++ b/drivers/scsi/megaraid/megaraid_sas.h
@@ -35,8 +35,8 @@
/*
* MegaRAID SAS Driver meta data
*/
-#define MEGASAS_VERSION "06.810.09.00-rc1"
-#define MEGASAS_RELDATE "Jan. 28, 2016"
+#define MEGASAS_VERSION "06.811.02.00-rc1"
+#define MEGASAS_RELDATE "April 12, 2016"
/*
* Device IDs
@@ -1344,6 +1344,8 @@
#define SCAN_PD_CHANNEL 0x1
#define SCAN_VD_CHANNEL 0x2
+#define MEGASAS_KDUMP_QUEUE_DEPTH 100
+
enum MR_SCSI_CMD_TYPE {
READ_WRITE_LDIO = 0,
NON_READ_WRITE_LDIO = 1,
diff --git a/drivers/scsi/megaraid/megaraid_sas_base.c b/drivers/scsi/megaraid/megaraid_sas_base.c
index e6ebc7a..f4b0690 100644
--- a/drivers/scsi/megaraid/megaraid_sas_base.c
+++ b/drivers/scsi/megaraid/megaraid_sas_base.c
@@ -2670,17 +2670,6 @@
}
/**
- * megasas_reset_device - Device reset handler entry point
- */
-static int megasas_reset_device(struct scsi_cmnd *scmd)
-{
- /*
- * First wait for all commands to complete
- */
- return megasas_generic_reset(scmd);
-}
-
-/**
* megasas_reset_bus_host - Bus & host reset handler entry point
*/
static int megasas_reset_bus_host(struct scsi_cmnd *scmd)
@@ -2702,6 +2691,50 @@
}
/**
+ * megasas_task_abort - Issues task abort request to firmware
+ * (supported only for fusion adapters)
+ * @scmd: SCSI command pointer
+ */
+static int megasas_task_abort(struct scsi_cmnd *scmd)
+{
+ int ret;
+ struct megasas_instance *instance;
+
+ instance = (struct megasas_instance *)scmd->device->host->hostdata;
+
+ if (instance->ctrl_context)
+ ret = megasas_task_abort_fusion(scmd);
+ else {
+ sdev_printk(KERN_NOTICE, scmd->device, "TASK ABORT not supported\n");
+ ret = FAILED;
+ }
+
+ return ret;
+}
+
+/**
+ * megasas_reset_target: Issues target reset request to firmware
+ * (supported only for fusion adapters)
+ * @scmd: SCSI command pointer
+ */
+static int megasas_reset_target(struct scsi_cmnd *scmd)
+{
+ int ret;
+ struct megasas_instance *instance;
+
+ instance = (struct megasas_instance *)scmd->device->host->hostdata;
+
+ if (instance->ctrl_context)
+ ret = megasas_reset_target_fusion(scmd);
+ else {
+ sdev_printk(KERN_NOTICE, scmd->device, "TARGET RESET not supported\n");
+ ret = FAILED;
+ }
+
+ return ret;
+}
+
+/**
* megasas_bios_param - Returns disk geometry for a disk
* @sdev: device handle
* @bdev: block device
@@ -2969,8 +3002,8 @@
.slave_alloc = megasas_slave_alloc,
.slave_destroy = megasas_slave_destroy,
.queuecommand = megasas_queue_command,
- .eh_device_reset_handler = megasas_reset_device,
- .eh_bus_reset_handler = megasas_reset_bus_host,
+ .eh_target_reset_handler = megasas_reset_target,
+ .eh_abort_handler = megasas_task_abort,
.eh_host_reset_handler = megasas_reset_bus_host,
.eh_timed_out = megasas_reset_timer,
.shost_attrs = megaraid_host_attrs,
@@ -5152,7 +5185,7 @@
instance->instancet->enable_intr(instance);
- dev_err(&instance->pdev->dev, "INIT adapter done\n");
+ dev_info(&instance->pdev->dev, "INIT adapter done\n");
megasas_setup_jbod_map(instance);
@@ -5598,14 +5631,6 @@
host->max_lun = MEGASAS_MAX_LUN;
host->max_cmd_len = 16;
- /* Fusion only supports host reset */
- if (instance->ctrl_context) {
- host->hostt->eh_device_reset_handler = NULL;
- host->hostt->eh_bus_reset_handler = NULL;
- host->hostt->eh_target_reset_handler = megasas_reset_target_fusion;
- host->hostt->eh_abort_handler = megasas_task_abort_fusion;
- }
-
/*
* Notify the mid-layer about the new controller
*/
@@ -5761,13 +5786,6 @@
break;
}
- instance->system_info_buf = pci_zalloc_consistent(pdev,
- sizeof(struct MR_DRV_SYSTEM_INFO),
- &instance->system_info_h);
-
- if (!instance->system_info_buf)
- dev_info(&instance->pdev->dev, "Can't allocate system info buffer\n");
-
/* Crash dump feature related initialisation*/
instance->drv_buf_index = 0;
instance->drv_buf_alloc = 0;
@@ -5777,14 +5795,6 @@
spin_lock_init(&instance->crashdump_lock);
instance->crash_dump_buf = NULL;
- if (!reset_devices)
- instance->crash_dump_buf = pci_alloc_consistent(pdev,
- CRASH_DMA_BUF_SIZE,
- &instance->crash_dump_h);
- if (!instance->crash_dump_buf)
- dev_err(&pdev->dev, "Can't allocate Firmware "
- "crash dump DMA buffer\n");
-
megasas_poll_wait_aen = 0;
instance->flag_ieee = 0;
instance->ev = NULL;
@@ -5803,11 +5813,26 @@
goto fail_alloc_dma_buf;
}
- instance->pd_info = pci_alloc_consistent(pdev,
- sizeof(struct MR_PD_INFO), &instance->pd_info_h);
+ if (!reset_devices) {
+ instance->system_info_buf = pci_zalloc_consistent(pdev,
+ sizeof(struct MR_DRV_SYSTEM_INFO),
+ &instance->system_info_h);
+ if (!instance->system_info_buf)
+ dev_info(&instance->pdev->dev, "Can't allocate system info buffer\n");
- if (!instance->pd_info)
- dev_err(&instance->pdev->dev, "Failed to alloc mem for pd_info\n");
+ instance->pd_info = pci_alloc_consistent(pdev,
+ sizeof(struct MR_PD_INFO), &instance->pd_info_h);
+
+ if (!instance->pd_info)
+ dev_err(&instance->pdev->dev, "Failed to alloc mem for pd_info\n");
+
+ instance->crash_dump_buf = pci_alloc_consistent(pdev,
+ CRASH_DMA_BUF_SIZE,
+ &instance->crash_dump_h);
+ if (!instance->crash_dump_buf)
+ dev_err(&pdev->dev, "Can't allocate Firmware "
+ "crash dump DMA buffer\n");
+ }
/*
* Initialize locks and queues
@@ -7174,6 +7199,16 @@
int rval;
/*
+ * Booted in kdump kernel, minimize memory footprints by
+ * disabling few features
+ */
+ if (reset_devices) {
+ msix_vectors = 1;
+ rdpq_enable = 0;
+ dual_qdepth_disable = 1;
+ }
+
+ /*
* Announce driver version and other information
*/
pr_info("megasas: %s\n", MEGASAS_VERSION);
diff --git a/drivers/scsi/megaraid/megaraid_sas_fusion.c b/drivers/scsi/megaraid/megaraid_sas_fusion.c
index 98a848b..ec83754 100644
--- a/drivers/scsi/megaraid/megaraid_sas_fusion.c
+++ b/drivers/scsi/megaraid/megaraid_sas_fusion.c
@@ -257,6 +257,9 @@
if (!instance->is_rdpq)
instance->max_fw_cmds = min_t(u16, instance->max_fw_cmds, 1024);
+ if (reset_devices)
+ instance->max_fw_cmds = min(instance->max_fw_cmds,
+ (u16)MEGASAS_KDUMP_QUEUE_DEPTH);
/*
* Reduce the max supported cmds by 1. This is to ensure that the
* reply_q_sz (1 more than the max cmd that driver may send)
@@ -851,7 +854,7 @@
ret = 1;
goto fail_fw_init;
}
- dev_err(&instance->pdev->dev, "Init cmd success\n");
+ dev_info(&instance->pdev->dev, "Init cmd success\n");
ret = 0;
@@ -2759,6 +2762,7 @@
dev_warn(&instance->pdev->dev, "Found FW in FAULT state,"
" will reset adapter scsi%d.\n",
instance->host->host_no);
+ megasas_complete_cmd_dpc_fusion((unsigned long)instance);
retval = 1;
goto out;
}
@@ -2766,6 +2770,7 @@
if (reason == MFI_IO_TIMEOUT_OCR) {
dev_info(&instance->pdev->dev,
"MFI IO is timed out, initiating OCR\n");
+ megasas_complete_cmd_dpc_fusion((unsigned long)instance);
retval = 1;
goto out;
}
diff --git a/drivers/scsi/mpt3sas/mpi/mpi2.h b/drivers/scsi/mpt3sas/mpi/mpi2.h
index dfad5b8..a9a659f 100644
--- a/drivers/scsi/mpt3sas/mpi/mpi2.h
+++ b/drivers/scsi/mpt3sas/mpi/mpi2.h
@@ -8,7 +8,7 @@
* scatter/gather formats.
* Creation Date: June 21, 2006
*
- * mpi2.h Version: 02.00.39
+ * mpi2.h Version: 02.00.42
*
* NOTE: Names (typedefs, defines, etc.) beginning with an MPI25 or Mpi25
* prefix are for use only on MPI v2.5 products, and must not be used
@@ -100,6 +100,9 @@
* Added MPI2_DIAG_SBR_RELOAD.
* 03-19-15 02.00.38 Bumped MPI2_HEADER_VERSION_UNIT.
* 05-25-15 02.00.39 Bumped MPI2_HEADER_VERSION_UNIT.
+ * 08-25-15 02.00.40 Bumped MPI2_HEADER_VERSION_UNIT.
+ * 12-15-15 02.00.41 Bumped MPI_HEADER_VERSION_UNIT
+ * 01-01-16 02.00.42 Bumped MPI_HEADER_VERSION_UNIT
* --------------------------------------------------------------------------
*/
@@ -139,7 +142,7 @@
#define MPI2_VERSION_02_06 (0x0206)
/*Unit and Dev versioning for this MPI header set */
-#define MPI2_HEADER_VERSION_UNIT (0x27)
+#define MPI2_HEADER_VERSION_UNIT (0x2A)
#define MPI2_HEADER_VERSION_DEV (0x00)
#define MPI2_HEADER_VERSION_UNIT_MASK (0xFF00)
#define MPI2_HEADER_VERSION_UNIT_SHIFT (8)
diff --git a/drivers/scsi/mpt3sas/mpi/mpi2_cnfg.h b/drivers/scsi/mpt3sas/mpi/mpi2_cnfg.h
index 9cf09bf..95356a8 100644
--- a/drivers/scsi/mpt3sas/mpi/mpi2_cnfg.h
+++ b/drivers/scsi/mpt3sas/mpi/mpi2_cnfg.h
@@ -6,7 +6,7 @@
* Title: MPI Configuration messages and pages
* Creation Date: November 10, 2006
*
- * mpi2_cnfg.h Version: 02.00.33
+ * mpi2_cnfg.h Version: 02.00.35
*
* NOTE: Names (typedefs, defines, etc.) beginning with an MPI25 or Mpi25
* prefix are for use only on MPI v2.5 products, and must not be used
@@ -183,9 +183,12 @@
* Added MPI2_BIOSPAGE1_OPTIONS_ADVANCED_CONFIG.
* Added AdapterOrderAux fields to BIOS Page 3.
* 03-16-15 02.00.31 Updated for MPI v2.6.
+ * Added Flags field to IO Unit Page 7.
* Added new SAS Phy Event codes
* 05-25-15 02.00.33 Added more defines for the BiosOptions field of
* MPI2_CONFIG_PAGE_BIOS_1.
+ * 08-25-15 02.00.34 Bumped Header Version.
+ * 12-18-15 02.00.35 Added SATADeviceWaitTime to SAS IO Unit Page 4.
* --------------------------------------------------------------------------
*/
@@ -958,13 +961,16 @@
U8 Reserved3; /*0x17 */
U32 BoardPowerRequirement; /*0x18 */
U32 PCISlotPowerAllocation; /*0x1C */
- U32 Reserved6; /* 0x20 */
- U32 Reserved7; /* 0x24 */
+/* reserved prior to MPI v2.6 */
+ U8 Flags; /* 0x20 */
+ U8 Reserved6; /* 0x21 */
+ U16 Reserved7; /* 0x22 */
+ U32 Reserved8; /* 0x24 */
} MPI2_CONFIG_PAGE_IO_UNIT_7,
*PTR_MPI2_CONFIG_PAGE_IO_UNIT_7,
Mpi2IOUnitPage7_t, *pMpi2IOUnitPage7_t;
-#define MPI2_IOUNITPAGE7_PAGEVERSION (0x04)
+#define MPI2_IOUNITPAGE7_PAGEVERSION (0x05)
/*defines for IO Unit Page 7 CurrentPowerMode and PreviousPowerMode fields */
#define MPI25_IOUNITPAGE7_PM_INIT_MASK (0xC0)
@@ -1045,6 +1051,8 @@
#define MPI2_IOUNITPAGE7_BOARD_TEMP_FAHRENHEIT (0x01)
#define MPI2_IOUNITPAGE7_BOARD_TEMP_CELSIUS (0x02)
+/* defines for IO Unit Page 7 Flags field */
+#define MPI2_IOUNITPAGE7_FLAG_CABLE_POWER_EXC (0x01)
/*IO Unit Page 8 */
@@ -2271,7 +2279,7 @@
U8
BootDeviceWaitTime; /*0x24 */
U8
- Reserved4; /*0x25 */
+ SATADeviceWaitTime; /*0x25 */
U16
Reserved5; /*0x26 */
U8
diff --git a/drivers/scsi/mpt3sas/mpi/mpi2_init.h b/drivers/scsi/mpt3sas/mpi/mpi2_init.h
index c38f624..bba56b6 100644
--- a/drivers/scsi/mpt3sas/mpi/mpi2_init.h
+++ b/drivers/scsi/mpt3sas/mpi/mpi2_init.h
@@ -6,7 +6,7 @@
* Title: MPI SCSI initiator mode messages and structures
* Creation Date: June 23, 2006
*
- * mpi2_init.h Version: 02.00.17
+ * mpi2_init.h Version: 02.00.20
*
* NOTE: Names (typedefs, defines, etc.) beginning with an MPI25 or Mpi25
* prefix are for use only on MPI v2.5 products, and must not be used
@@ -51,6 +51,9 @@
* Added MPI26_SCSIIO_IOFLAGS_ESCAPE_PASSTHROUGH.
* Added MPI2_SEP_REQ_SLOTSTATUS_DEV_OFF and
* MPI2_SEP_REPLY_SLOTSTATUS_DEV_OFF.
+ * 08-26-15 02.00.18 Added SCSITASKMGMT_MSGFLAGS for Target Reset.
+ * 12-18-15 02.00.19 Added EEDPObservedValue added to SCSI IO Reply message.
+ * 01-04-16 02.00.20 Modified EEDP reported values in SCSI IO Reply message.
* --------------------------------------------------------------------------
*/
@@ -359,8 +362,14 @@
U16 TaskTag; /*0x20 */
U16 SCSIStatusQualifier; /* 0x22 */
U32 BidirectionalTransferCount; /*0x24 */
- U32 EEDPErrorOffset; /*0x28 *//*MPI 2.5 only; Reserved in MPI 2.0*/
- U32 Reserved6; /*0x2C */
+ /* MPI 2.5+ only; Reserved in MPI 2.0 */
+ U32 EEDPErrorOffset; /* 0x28 */
+ /* MPI 2.5+ only; Reserved in MPI 2.0 */
+ U16 EEDPObservedAppTag; /* 0x2C */
+ /* MPI 2.5+ only; Reserved in MPI 2.0 */
+ U16 EEDPObservedGuard; /* 0x2E */
+ /* MPI 2.5+ only; Reserved in MPI 2.0 */
+ U32 EEDPObservedRefTag; /* 0x30 */
} MPI2_SCSI_IO_REPLY, *PTR_MPI2_SCSI_IO_REPLY,
Mpi2SCSIIOReply_t, *pMpi2SCSIIOReply_t;
diff --git a/drivers/scsi/mpt3sas/mpi/mpi2_ioc.h b/drivers/scsi/mpt3sas/mpi/mpi2_ioc.h
index cf510ed..8bae305 100644
--- a/drivers/scsi/mpt3sas/mpi/mpi2_ioc.h
+++ b/drivers/scsi/mpt3sas/mpi/mpi2_ioc.h
@@ -6,7 +6,7 @@
* Title: MPI IOC, Port, Event, FW Download, and FW Upload messages
* Creation Date: October 11, 2006
*
- * mpi2_ioc.h Version: 02.00.26
+ * mpi2_ioc.h Version: 02.00.27
*
* NOTE: Names (typedefs, defines, etc.) beginning with an MPI25 or Mpi25
* prefix are for use only on MPI v2.5 products, and must not be used
@@ -134,9 +134,13 @@
* Added Encrypted Hash Extended Image.
* 12-05-13 02.00.24 Added MPI25_HASH_IMAGE_TYPE_BIOS.
* 11-18-14 02.00.25 Updated copyright information.
- * 03-16-15 02.00.26 Added MPI26_FW_HEADER_PID_FAMILY_3324_SAS and
+ * 03-16-15 02.00.26 Updated for MPI v2.6.
+ * Added MPI2_EVENT_ACTIVE_CABLE_EXCEPTION and
+ * MPI26_EVENT_DATA_ACTIVE_CABLE_EXCEPT.
+ * Added MPI26_FW_HEADER_PID_FAMILY_3324_SAS and
* MPI26_FW_HEADER_PID_FAMILY_3516_SAS.
* Added MPI26_CTRL_OP_SHUTDOWN.
+ * 08-25-15 02.00.27 Added IC ARCH Class based signature defines
* --------------------------------------------------------------------------
*/
@@ -168,7 +172,7 @@
U16 MsgVersion; /*0x0C */
U16 HeaderVersion; /*0x0E */
U32 Reserved5; /*0x10 */
- U16 Reserved6; /*0x14 */
+ U16 ConfigurationFlags; /* 0x14 */
U8 HostPageSize; /*0x16 */
U8 HostMSIxVectors; /*0x17 */
U16 Reserved8; /*0x18 */
@@ -516,6 +520,7 @@
#define MPI2_EVENT_TEMP_THRESHOLD (0x0027)
#define MPI2_EVENT_HOST_MESSAGE (0x0028)
#define MPI2_EVENT_POWER_PERFORMANCE_CHANGE (0x0029)
+#define MPI2_EVENT_ACTIVE_CABLE_EXCEPTION (0x0034)
#define MPI2_EVENT_MIN_PRODUCT_SPECIFIC (0x006E)
#define MPI2_EVENT_MAX_PRODUCT_SPECIFIC (0x007F)
@@ -580,7 +585,7 @@
} MPI2_EVENT_DATA_HOST_MESSAGE, *PTR_MPI2_EVENT_DATA_HOST_MESSAGE,
Mpi2EventDataHostMessage_t, *pMpi2EventDataHostMessage_t;
-/*Power Performance Change Event */
+/*Power Performance Change Event data */
typedef struct _MPI2_EVENT_DATA_POWER_PERF_CHANGE {
U8 CurrentPowerMode; /*0x00 */
@@ -605,6 +610,21 @@
#define MPI2_EVENT_PM_MODE_REDUCED_POWER (0x05)
#define MPI2_EVENT_PM_MODE_STANDBY (0x06)
+/* Active Cable Exception Event data */
+
+typedef struct _MPI26_EVENT_DATA_ACTIVE_CABLE_EXCEPT {
+ U32 ActiveCablePowerRequirement; /* 0x00 */
+ U8 ReasonCode; /* 0x04 */
+ U8 ReceptacleID; /* 0x05 */
+ U16 Reserved1; /* 0x06 */
+} MPI26_EVENT_DATA_ACTIVE_CABLE_EXCEPT,
+ *PTR_MPI26_EVENT_DATA_ACTIVE_CABLE_EXCEPT,
+ Mpi26EventDataActiveCableExcept_t,
+ *pMpi26EventDataActiveCableExcept_t;
+
+/* defines for ReasonCode field */
+#define MPI26_EVENT_ACTIVE_CABLE_INSUFFICIENT_POWER (0x00)
+
/*Hard Reset Received Event data */
typedef struct _MPI2_EVENT_DATA_HARD_RESET_RECEIVED {
@@ -1366,7 +1386,16 @@
/*Signature0 field */
#define MPI2_FW_HEADER_SIGNATURE0_OFFSET (0x04)
#define MPI2_FW_HEADER_SIGNATURE0 (0x5AFAA55A)
-#define MPI26_FW_HEADER_SIGNATURE0 (0x5AEAA55A)
+/* Last byte is defined by architecture */
+#define MPI26_FW_HEADER_SIGNATURE0_BASE (0x5AEAA500)
+#define MPI26_FW_HEADER_SIGNATURE0_ARC_0 (0x5A)
+#define MPI26_FW_HEADER_SIGNATURE0_ARC_1 (0x00)
+#define MPI26_FW_HEADER_SIGNATURE0_ARC_2 (0x01)
+/* legacy (0x5AEAA55A) */
+#define MPI26_FW_HEADER_SIGNATURE0 \
+ (MPI26_FW_HEADER_SIGNATURE0_BASE+MPI26_FW_HEADER_SIGNATURE0_ARC_0)
+#define MPI26_FW_HEADER_SIGNATURE0_3516 \
+ (MPI26_FW_HEADER_SIGNATURE0_BASE+MPI26_FW_HEADER_SIGNATURE0_ARC_1)
/*Signature1 field */
#define MPI2_FW_HEADER_SIGNATURE1_OFFSET (0x08)
@@ -1778,6 +1807,7 @@
#define MPI26_CTRL_OP_SAS_PHY_LINK_RESET (0x06)
#define MPI26_CTRL_OP_SAS_PHY_HARD_RESET (0x07)
#define MPI26_CTRL_OP_PHY_CLEAR_ERROR_LOG (0x08)
+#define MPI26_CTRL_OP_LINK_CLEAR_ERROR_LOG (0x09)
#define MPI26_CTRL_OP_SAS_SEND_PRIMITIVE (0x0A)
#define MPI26_CTRL_OP_FORCE_FULL_DISCOVERY (0x0B)
#define MPI26_CTRL_OP_REMOVE_DEVICE (0x0D)
diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c
index 8c44b9c..751f13e 100644
--- a/drivers/scsi/mpt3sas/mpt3sas_base.c
+++ b/drivers/scsi/mpt3sas/mpt3sas_base.c
@@ -57,6 +57,7 @@
#include <linux/dma-mapping.h>
#include <linux/io.h>
#include <linux/time.h>
+#include <linux/ktime.h>
#include <linux/kthread.h>
#include <linux/aer.h>
@@ -654,6 +655,9 @@
case MPI2_EVENT_TEMP_THRESHOLD:
desc = "Temperature Threshold";
break;
+ case MPI2_EVENT_ACTIVE_CABLE_EXCEPTION:
+ desc = "Active cable exception";
+ break;
}
if (!desc)
@@ -1100,18 +1104,16 @@
}
/**
- * mpt3sas_base_flush_reply_queues - flushing the MSIX reply queues
+ * mpt3sas_base_sync_reply_irqs - flush pending MSIX interrupts
* @ioc: per adapter object
- * Context: ISR conext
+ * Context: non ISR conext
*
- * Called when a Task Management request has completed. We want
- * to flush the other reply queues so all the outstanding IO has been
- * completed back to OS before we process the TM completetion.
+ * Called when a Task Management request has completed.
*
* Return nothing.
*/
void
-mpt3sas_base_flush_reply_queues(struct MPT3SAS_ADAPTER *ioc)
+mpt3sas_base_sync_reply_irqs(struct MPT3SAS_ADAPTER *ioc)
{
struct adapter_reply_queue *reply_q;
@@ -1122,12 +1124,13 @@
return;
list_for_each_entry(reply_q, &ioc->reply_queue_list, list) {
- if (ioc->shost_recovery)
+ if (ioc->shost_recovery || ioc->remove_host ||
+ ioc->pci_error_recovery)
return;
/* TMs are on msix_index == 0 */
if (reply_q->msix_index == 0)
continue;
- _base_interrupt(reply_q->vector, (void *)reply_q);
+ synchronize_irq(reply_q->vector);
}
}
@@ -3207,10 +3210,10 @@
sg_tablesize = MPT_MIN_PHYS_SEGMENTS;
else if (sg_tablesize > MPT_MAX_PHYS_SEGMENTS) {
sg_tablesize = min_t(unsigned short, sg_tablesize,
- SCSI_MAX_SG_CHAIN_SEGMENTS);
+ SG_MAX_SEGMENTS);
pr_warn(MPT3SAS_FMT
"sg_tablesize(%u) is bigger than kernel"
- " defined SCSI_MAX_SG_SEGMENTS(%u)\n", ioc->name,
+ " defined SG_CHUNK_SIZE(%u)\n", ioc->name,
sg_tablesize, MPT_MAX_PHYS_SEGMENTS);
}
ioc->shost->sg_tablesize = sg_tablesize;
@@ -4387,7 +4390,7 @@
Mpi2IOCInitRequest_t mpi_request;
Mpi2IOCInitReply_t mpi_reply;
int i, r = 0;
- struct timeval current_time;
+ ktime_t current_time;
u16 ioc_status;
u32 reply_post_free_array_sz = 0;
Mpi2IOCInitRDPQArrayEntry *reply_post_free_array = NULL;
@@ -4449,9 +4452,8 @@
/* This time stamp specifies number of milliseconds
* since epoch ~ midnight January 1, 1970.
*/
- do_gettimeofday(¤t_time);
- mpi_request.TimeStamp = cpu_to_le64((u64)current_time.tv_sec * 1000 +
- (current_time.tv_usec / 1000));
+ current_time = ktime_get_real();
+ mpi_request.TimeStamp = cpu_to_le64(ktime_to_ms(current_time));
if (ioc->logging_level & MPT_DEBUG_INIT) {
__le32 *mfp;
@@ -5424,6 +5426,8 @@
_base_unmask_events(ioc, MPI2_EVENT_IR_OPERATION_STATUS);
_base_unmask_events(ioc, MPI2_EVENT_LOG_ENTRY_ADDED);
_base_unmask_events(ioc, MPI2_EVENT_TEMP_THRESHOLD);
+ if (ioc->hba_mpi_version_belonged == MPI26_VERSION)
+ _base_unmask_events(ioc, MPI2_EVENT_ACTIVE_CABLE_EXCEPTION);
r = _base_make_ioc_operational(ioc, CAN_SLEEP);
if (r)
diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.h b/drivers/scsi/mpt3sas/mpt3sas_base.h
index 32580b5..892c9be 100644
--- a/drivers/scsi/mpt3sas/mpt3sas_base.h
+++ b/drivers/scsi/mpt3sas/mpt3sas_base.h
@@ -73,8 +73,8 @@
#define MPT3SAS_DRIVER_NAME "mpt3sas"
#define MPT3SAS_AUTHOR "Avago Technologies <MPT-FusionLinux.pdl@avagotech.com>"
#define MPT3SAS_DESCRIPTION "LSI MPT Fusion SAS 3.0 Device Driver"
-#define MPT3SAS_DRIVER_VERSION "12.100.00.00"
-#define MPT3SAS_MAJOR_VERSION 12
+#define MPT3SAS_DRIVER_VERSION "13.100.00.00"
+#define MPT3SAS_MAJOR_VERSION 13
#define MPT3SAS_MINOR_VERSION 100
#define MPT3SAS_BUILD_VERSION 0
#define MPT3SAS_RELEASE_VERSION 00
@@ -90,7 +90,7 @@
/*
* Set MPT3SAS_SG_DEPTH value based on user input.
*/
-#define MPT_MAX_PHYS_SEGMENTS SCSI_MAX_SG_SEGMENTS
+#define MPT_MAX_PHYS_SEGMENTS SG_CHUNK_SIZE
#define MPT_MIN_PHYS_SEGMENTS 16
#ifdef CONFIG_SCSI_MPT3SAS_MAX_SGE
@@ -112,6 +112,8 @@
#define MPT3SAS_SAS_QUEUE_DEPTH 254
#define MPT3SAS_RAID_QUEUE_DEPTH 128
+#define MPT3SAS_RAID_MAX_SECTORS 8192
+
#define MPT_NAME_LENGTH 32 /* generic length of strings */
#define MPT_STRING_LENGTH 64
@@ -1234,7 +1236,8 @@
void *mpt3sas_base_get_sense_buffer(struct MPT3SAS_ADAPTER *ioc, u16 smid);
__le32 mpt3sas_base_get_sense_buffer_dma(struct MPT3SAS_ADAPTER *ioc,
u16 smid);
-void mpt3sas_base_flush_reply_queues(struct MPT3SAS_ADAPTER *ioc);
+
+void mpt3sas_base_sync_reply_irqs(struct MPT3SAS_ADAPTER *ioc);
/* hi-priority queue */
u16 mpt3sas_base_get_smid_hpr(struct MPT3SAS_ADAPTER *ioc, u8 cb_idx);
diff --git a/drivers/scsi/mpt3sas/mpt3sas_scsih.c b/drivers/scsi/mpt3sas/mpt3sas_scsih.c
index e0e4920..6a4df5a 100644
--- a/drivers/scsi/mpt3sas/mpt3sas_scsih.c
+++ b/drivers/scsi/mpt3sas/mpt3sas_scsih.c
@@ -174,13 +174,13 @@
* struct fw_event_work - firmware event struct
* @list: link list framework
* @work: work object (ioc->fault_reset_work_q)
- * @cancel_pending_work: flag set during reset handling
* @ioc: per adapter object
* @device_handle: device handle
* @VF_ID: virtual function id
* @VP_ID: virtual port id
* @ignore: flag meaning this event has been marked to ignore
- * @event: firmware event MPI2_EVENT_XXX defined in mpt2_ioc.h
+ * @event: firmware event MPI2_EVENT_XXX defined in mpi2_ioc.h
+ * @refcount: kref for this event
* @event_data: reply event data payload follows
*
* This object stored on ioc->fw_event_list.
@@ -188,8 +188,6 @@
struct fw_event_work {
struct list_head list;
struct work_struct work;
- u8 cancel_pending_work;
- struct delayed_work delayed_work;
struct MPT3SAS_ADAPTER *ioc;
u16 device_handle;
@@ -1911,6 +1909,14 @@
(unsigned long long)raid_device->wwid,
raid_device->num_pds, ds);
+ if (shost->max_sectors > MPT3SAS_RAID_MAX_SECTORS) {
+ blk_queue_max_hw_sectors(sdev->request_queue,
+ MPT3SAS_RAID_MAX_SECTORS);
+ sdev_printk(KERN_INFO, sdev,
+ "Set queue's max_sector to: %u\n",
+ MPT3SAS_RAID_MAX_SECTORS);
+ }
+
scsih_change_queue_depth(sdev, qdepth);
/* raid transport support */
@@ -2118,7 +2124,6 @@
return 1;
if (ioc->tm_cmds.smid != smid)
return 1;
- mpt3sas_base_flush_reply_queues(ioc);
ioc->tm_cmds.status |= MPT3_CMD_COMPLETE;
mpi_reply = mpt3sas_base_get_reply_virt_addr(ioc, reply);
if (mpi_reply) {
@@ -2303,6 +2308,9 @@
}
}
+ /* sync IRQs in case those were busy during flush. */
+ mpt3sas_base_sync_reply_irqs(ioc);
+
if (ioc->tm_cmds.status & MPT3_CMD_REPLY_VALID) {
mpt3sas_trigger_master(ioc, MASTER_TRIGGER_TASK_MANAGMENT);
mpi_reply = ioc->tm_cmds.reply;
@@ -2804,12 +2812,12 @@
/*
* Wait on the fw_event to complete. If this returns 1, then
* the event was never executed, and we need a put for the
- * reference the delayed_work had on the fw_event.
+ * reference the work had on the fw_event.
*
* If it did execute, we wait for it to finish, and the put will
* happen from _firmware_event_work()
*/
- if (cancel_delayed_work_sync(&fw_event->delayed_work))
+ if (cancel_work_sync(&fw_event->work))
fw_event_work_put(fw_event);
fw_event_work_put(fw_event);
@@ -3961,7 +3969,7 @@
MPI2_SCSIIO_EEDPFLAGS_CHECK_REFTAG |
MPI2_SCSIIO_EEDPFLAGS_CHECK_GUARD;
mpi_request->CDB.EEDP32.PrimaryReferenceTag =
- cpu_to_be32(scsi_get_lba(scmd));
+ cpu_to_be32(scsi_prot_ref_tag(scmd));
break;
case SCSI_PROT_DIF_TYPE3:
@@ -7850,6 +7858,7 @@
Mpi2EventNotificationReply_t *mpi_reply;
u16 event;
u16 sz;
+ Mpi26EventDataActiveCableExcept_t *ActiveCableEventData;
/* events turned off due to host reset or driver unloading */
if (ioc->remove_host || ioc->pci_error_recovery)
@@ -7962,6 +7971,18 @@
(Mpi2EventDataTemperature_t *)
mpi_reply->EventData);
break;
+ case MPI2_EVENT_ACTIVE_CABLE_EXCEPTION:
+ ActiveCableEventData =
+ (Mpi26EventDataActiveCableExcept_t *) mpi_reply->EventData;
+ if (ActiveCableEventData->ReasonCode ==
+ MPI26_EVENT_ACTIVE_CABLE_INSUFFICIENT_POWER)
+ pr_info(MPT3SAS_FMT "Currently an active cable with ReceptacleID %d",
+ ioc->name, ActiveCableEventData->ReceptacleID);
+ pr_info("cannot be powered and devices connected to this active cable");
+ pr_info("will not be seen. This active cable");
+ pr_info("requires %d mW of power",
+ ActiveCableEventData->ActiveCablePowerRequirement);
+ break;
default: /* ignore the rest */
return 1;
diff --git a/drivers/scsi/mvsas/mv_init.c b/drivers/scsi/mvsas/mv_init.c
index c7c2505..8280046 100644
--- a/drivers/scsi/mvsas/mv_init.c
+++ b/drivers/scsi/mvsas/mv_init.c
@@ -704,24 +704,7 @@
.class_mask = 0,
.driver_data = chip_9445,
},
- {
- .vendor = PCI_VENDOR_ID_MARVELL_EXT,
- .device = 0x9485,
- .subvendor = PCI_ANY_ID,
- .subdevice = 0x9480,
- .class = 0,
- .class_mask = 0,
- .driver_data = chip_9485,
- },
- {
- .vendor = PCI_VENDOR_ID_MARVELL_EXT,
- .device = 0x9485,
- .subvendor = PCI_ANY_ID,
- .subdevice = 0x9485,
- .class = 0,
- .class_mask = 0,
- .driver_data = chip_9485,
- },
+ { PCI_VDEVICE(MARVELL_EXT, 0x9485), chip_9485 }, /* Marvell 9480/9485 (any vendor/model) */
{ PCI_VDEVICE(OCZ, 0x1021), chip_9485}, /* OCZ RevoDrive3 */
{ PCI_VDEVICE(OCZ, 0x1022), chip_9485}, /* OCZ RevoDrive3/zDriveR4 (exact model unknown) */
{ PCI_VDEVICE(OCZ, 0x1040), chip_9485}, /* OCZ RevoDrive3/zDriveR4 (exact model unknown) */
diff --git a/drivers/scsi/mvsas/mv_sas.c b/drivers/scsi/mvsas/mv_sas.c
index 83cd3ea..5b9fcff 100644
--- a/drivers/scsi/mvsas/mv_sas.c
+++ b/drivers/scsi/mvsas/mv_sas.c
@@ -429,7 +429,10 @@
if (qc) {
if (qc->tf.command == ATA_CMD_FPDMA_WRITE ||
- qc->tf.command == ATA_CMD_FPDMA_READ) {
+ qc->tf.command == ATA_CMD_FPDMA_READ ||
+ qc->tf.command == ATA_CMD_FPDMA_RECV ||
+ qc->tf.command == ATA_CMD_FPDMA_SEND ||
+ qc->tf.command == ATA_CMD_NCQ_NON_DATA) {
*tag = qc->tag;
return 1;
}
diff --git a/drivers/scsi/pas16.c b/drivers/scsi/pas16.c
index 512037e..2f689ae 100644
--- a/drivers/scsi/pas16.c
+++ b/drivers/scsi/pas16.c
@@ -1,5 +1,3 @@
-#define PSEUDO_DMA
-
/*
* This driver adapted from Drew Eckhardt's Trantor T128 driver
*
@@ -77,7 +75,6 @@
#include <scsi/scsi_host.h>
#include "pas16.h"
-#define AUTOPROBE_IRQ
#include "NCR5380.h"
@@ -377,7 +374,7 @@
instance->io_port = io_port;
- if (NCR5380_init(instance, 0))
+ if (NCR5380_init(instance, FLAG_DMA_FIXUP | FLAG_LATE_DMA_SETUP))
goto out_unregister;
NCR5380_maybe_reset_bus(instance);
@@ -460,7 +457,7 @@
}
/*
- * Function : int NCR5380_pread (struct Scsi_Host *instance,
+ * Function : int pas16_pread (struct Scsi_Host *instance,
* unsigned char *dst, int len)
*
* Purpose : Fast 5380 pseudo-dma read function, transfers len bytes to
@@ -472,14 +469,14 @@
* timeout.
*/
-static inline int NCR5380_pread (struct Scsi_Host *instance, unsigned char *dst,
- int len) {
+static inline int pas16_pread(struct Scsi_Host *instance,
+ unsigned char *dst, int len)
+{
register unsigned char *d = dst;
register unsigned short reg = (unsigned short) (instance->io_port +
P_DATA_REG_OFFSET);
register int i = len;
int ii = 0;
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
while ( !(inb(instance->io_port + P_STATUS_REG_OFFSET) & P_ST_RDY) )
++ii;
@@ -492,13 +489,11 @@
instance->host_no);
return -1;
}
- if (ii > hostdata->spin_max_r)
- hostdata->spin_max_r = ii;
return 0;
}
/*
- * Function : int NCR5380_pwrite (struct Scsi_Host *instance,
+ * Function : int pas16_pwrite (struct Scsi_Host *instance,
* unsigned char *src, int len)
*
* Purpose : Fast 5380 pseudo-dma write function, transfers len bytes from
@@ -510,13 +505,13 @@
* timeout.
*/
-static inline int NCR5380_pwrite (struct Scsi_Host *instance, unsigned char *src,
- int len) {
+static inline int pas16_pwrite(struct Scsi_Host *instance,
+ unsigned char *src, int len)
+{
register unsigned char *s = src;
register unsigned short reg = (instance->io_port + P_DATA_REG_OFFSET);
register int i = len;
int ii = 0;
- struct NCR5380_hostdata *hostdata = shost_priv(instance);
while ( !((inb(instance->io_port + P_STATUS_REG_OFFSET)) & P_ST_RDY) )
++ii;
@@ -529,8 +524,6 @@
instance->host_no);
return -1;
}
- if (ii > hostdata->spin_max_w)
- hostdata->spin_max_w = ii;
return 0;
}
@@ -550,8 +543,6 @@
.detect = pas16_detect,
.release = pas16_release,
.proc_name = "pas16",
- .show_info = pas16_show_info,
- .write_info = pas16_write_info,
.info = pas16_info,
.queuecommand = pas16_queue_command,
.eh_abort_handler = pas16_abort,
diff --git a/drivers/scsi/pas16.h b/drivers/scsi/pas16.h
index d375277..9fe7f33 100644
--- a/drivers/scsi/pas16.h
+++ b/drivers/scsi/pas16.h
@@ -103,14 +103,15 @@
#define NCR5380_write(reg, value) ( outb((value),PAS16_io_port(reg)) )
#define NCR5380_dma_xfer_len(instance, cmd, phase) (cmd->transfersize)
+#define NCR5380_dma_recv_setup pas16_pread
+#define NCR5380_dma_send_setup pas16_pwrite
+#define NCR5380_dma_residual(instance) (0)
#define NCR5380_intr pas16_intr
#define NCR5380_queue_command pas16_queue_command
#define NCR5380_abort pas16_abort
#define NCR5380_bus_reset pas16_bus_reset
#define NCR5380_info pas16_info
-#define NCR5380_show_info pas16_show_info
-#define NCR5380_write_info pas16_write_info
/* 15 14 12 10 7 5 3
1101 0100 1010 1000 */
diff --git a/drivers/scsi/pm8001/pm8001_init.c b/drivers/scsi/pm8001/pm8001_init.c
index 062ab34..6bd7bf4 100644
--- a/drivers/scsi/pm8001/pm8001_init.c
+++ b/drivers/scsi/pm8001/pm8001_init.c
@@ -418,8 +418,6 @@
if (pci_resource_flags(pdev, bar) & IORESOURCE_MEM) {
pm8001_ha->io_mem[logicalBar].membase =
pci_resource_start(pdev, bar);
- pm8001_ha->io_mem[logicalBar].membase &=
- (u32)PCI_BASE_ADDRESS_MEM_MASK;
pm8001_ha->io_mem[logicalBar].memsize =
pci_resource_len(pdev, bar);
pm8001_ha->io_mem[logicalBar].memvirtaddr =
diff --git a/drivers/scsi/pm8001/pm8001_sas.c b/drivers/scsi/pm8001/pm8001_sas.c
index 949198c..dc33dfa 100644
--- a/drivers/scsi/pm8001/pm8001_sas.c
+++ b/drivers/scsi/pm8001/pm8001_sas.c
@@ -280,7 +280,10 @@
struct ata_queued_cmd *qc = task->uldd_task;
if (qc) {
if (qc->tf.command == ATA_CMD_FPDMA_WRITE ||
- qc->tf.command == ATA_CMD_FPDMA_READ) {
+ qc->tf.command == ATA_CMD_FPDMA_READ ||
+ qc->tf.command == ATA_CMD_FPDMA_RECV ||
+ qc->tf.command == ATA_CMD_FPDMA_SEND ||
+ qc->tf.command == ATA_CMD_NCQ_NON_DATA) {
*tag = qc->tag;
return 1;
}
diff --git a/drivers/scsi/qla2xxx/qla_mr.c b/drivers/scsi/qla2xxx/qla_mr.c
index b5029e5..15dff70 100644
--- a/drivers/scsi/qla2xxx/qla_mr.c
+++ b/drivers/scsi/qla2xxx/qla_mr.c
@@ -6,6 +6,7 @@
*/
#include "qla_def.h"
#include <linux/delay.h>
+#include <linux/ktime.h>
#include <linux/pci.h>
#include <linux/ratelimit.h>
#include <linux/vmalloc.h>
@@ -1812,7 +1813,6 @@
struct host_system_info *phost_info;
struct register_host_info *preg_hsi;
struct new_utsname *p_sysid = NULL;
- struct timeval tv;
sp = qla2x00_get_sp(vha, fcport, GFP_KERNEL);
if (!sp)
@@ -1886,8 +1886,7 @@
p_sysid->domainname, DOMNAME_LENGTH);
strncpy(phost_info->hostdriver,
QLA2XXX_VERSION, VERSION_LENGTH);
- do_gettimeofday(&tv);
- preg_hsi->utc = (uint64_t)tv.tv_sec;
+ preg_hsi->utc = (uint64_t)ktime_get_real_seconds();
ql_dbg(ql_dbg_init, vha, 0x0149,
"ISP%04X: Host registration with firmware\n",
ha->pdev->device);
diff --git a/drivers/scsi/qla2xxx/qla_nx.c b/drivers/scsi/qla2xxx/qla_nx.c
index b6b4cfd..54380b4 100644
--- a/drivers/scsi/qla2xxx/qla_nx.c
+++ b/drivers/scsi/qla2xxx/qla_nx.c
@@ -1229,7 +1229,7 @@
if (buf == NULL) {
ql_log(ql_log_fatal, vha, 0x010c,
"Unable to allocate memory.\n");
- return -1;
+ return -ENOMEM;
}
for (i = 0; i < n; i++) {
diff --git a/drivers/scsi/scsi_common.c b/drivers/scsi/scsi_common.c
index ce79de8..b1383a7 100644
--- a/drivers/scsi/scsi_common.c
+++ b/drivers/scsi/scsi_common.c
@@ -293,3 +293,56 @@
return 0;
}
EXPORT_SYMBOL(scsi_set_sense_information);
+
+/**
+ * scsi_set_sense_field_pointer - set the field pointer sense key
+ * specific information in a formatted sense data buffer
+ * @buf: Where to build sense data
+ * @buf_len: buffer length
+ * @fp: field pointer to be set
+ * @bp: bit pointer to be set
+ * @cd: command/data bit
+ *
+ * Return value:
+ * 0 on success or EINVAL for invalid sense buffer length
+ */
+int scsi_set_sense_field_pointer(u8 *buf, int buf_len, u16 fp, u8 bp, bool cd)
+{
+ u8 *ucp, len;
+
+ if ((buf[0] & 0x7f) == 0x72) {
+ len = buf[7];
+ ucp = (char *)scsi_sense_desc_find(buf, len + 8, 2);
+ if (!ucp) {
+ buf[7] = len + 8;
+ ucp = buf + 8 + len;
+ }
+
+ if (buf_len < len + 8)
+ /* Not enough room for info */
+ return -EINVAL;
+
+ ucp[0] = 2;
+ ucp[1] = 6;
+ ucp[4] = 0x80; /* Valid bit */
+ if (cd)
+ ucp[4] |= 0x40;
+ if (bp < 0x8)
+ ucp[4] |= 0x8 | bp;
+ put_unaligned_be16(fp, &ucp[5]);
+ } else if ((buf[0] & 0x7f) == 0x70) {
+ len = buf[7];
+ if (len < 18)
+ buf[7] = 18;
+
+ buf[15] = 0x80;
+ if (cd)
+ buf[15] |= 0x40;
+ if (bp < 0x8)
+ buf[15] |= 0x8 | bp;
+ put_unaligned_be16(fp, &buf[16]);
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL(scsi_set_sense_field_pointer);
diff --git a/drivers/scsi/scsi_debug.c b/drivers/scsi/scsi_debug.c
index f3d69a98..0f9ba41 100644
--- a/drivers/scsi/scsi_debug.c
+++ b/drivers/scsi/scsi_debug.c
@@ -6,23 +6,15 @@
* anything out of the ordinary is seen.
* ^^^^^^^^^^^^^^^^^^^^^^^ Original ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
*
- * This version is more generic, simulating a variable number of disk
- * (or disk like devices) sharing a common amount of RAM. To be more
- * realistic, the simulated devices have the transport attributes of
- * SAS disks.
+ * Copyright (C) 2001 - 2016 Douglas Gilbert
*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
*
* For documentation see http://sg.danny.cz/sg/sdebug26.html
*
- * D. Gilbert (dpg) work for Magneto-Optical device test [20010421]
- * dpg: work for devfs large number of disks [20010809]
- * forked for lk 2.5 series [20011216, 20020101]
- * use vmalloc() more inquiry+mode_sense [20020302]
- * add timers for delayed responses [20020721]
- * Patrick Mansfield <patmans@us.ibm.com> max_luns+scsi_level [20021031]
- * Mike Anderson <andmike@us.ibm.com> sysfs work [20021118]
- * dpg: change style of boot options to "scsi_debug.num_tgts=2" and
- * module options to "modprobe scsi_debug num_tgts=2" [20021221]
*/
@@ -32,7 +24,7 @@
#include <linux/kernel.h>
#include <linux/errno.h>
-#include <linux/timer.h>
+#include <linux/jiffies.h>
#include <linux/slab.h>
#include <linux/types.h>
#include <linux/string.h>
@@ -49,6 +41,7 @@
#include <linux/interrupt.h>
#include <linux/atomic.h>
#include <linux/hrtimer.h>
+#include <linux/uuid.h>
#include <net/checksum.h>
@@ -66,8 +59,9 @@
#include "sd.h"
#include "scsi_logging.h"
-#define SCSI_DEBUG_VERSION "1.85"
-static const char *scsi_debug_version_date = "20141022";
+/* make sure inq_product_rev string corresponds to this version */
+#define SDEBUG_VERSION "1.86"
+static const char *sdebug_version_date = "20160430";
#define MY_NAME "scsi_debug"
@@ -102,7 +96,6 @@
/* Additional Sense Code Qualifier (ASCQ) */
#define ACK_NAK_TO 0x3
-
/* Default values for driver parameters */
#define DEF_NUM_HOST 1
#define DEF_NUM_TGTS 1
@@ -111,7 +104,7 @@
* (id 0) containing 1 logical unit (lun 0). That is 1 device.
*/
#define DEF_ATO 1
-#define DEF_DELAY 1 /* if > 0 unit is a jiffy */
+#define DEF_JDELAY 1 /* if > 0 unit is a jiffy */
#define DEF_DEV_SIZE_MB 8
#define DEF_DIF 0
#define DEF_DIX 0
@@ -131,9 +124,9 @@
#define DEF_OPTS 0
#define DEF_OPT_BLKS 1024
#define DEF_PHYSBLK_EXP 0
-#define DEF_PTYPE 0
+#define DEF_PTYPE TYPE_DISK
#define DEF_REMOVABLE false
-#define DEF_SCSI_LEVEL 6 /* INQUIRY, byte2 [6->SPC-4] */
+#define DEF_SCSI_LEVEL 7 /* INQUIRY, byte2 [6->SPC-4; 7->SPC-5] */
#define DEF_SECTOR_SIZE 512
#define DEF_UNMAP_ALIGNMENT 0
#define DEF_UNMAP_GRANULARITY 1
@@ -143,43 +136,54 @@
#define DEF_VPD_USE_HOSTNO 1
#define DEF_WRITESAME_LENGTH 0xFFFF
#define DEF_STRICT 0
-#define DELAY_OVERRIDDEN -9999
+#define DEF_STATISTICS false
+#define DEF_SUBMIT_QUEUES 1
+#define DEF_UUID_CTL 0
+#define JDELAY_OVERRIDDEN -9999
-/* bit mask values for scsi_debug_opts */
-#define SCSI_DEBUG_OPT_NOISE 1
-#define SCSI_DEBUG_OPT_MEDIUM_ERR 2
-#define SCSI_DEBUG_OPT_TIMEOUT 4
-#define SCSI_DEBUG_OPT_RECOVERED_ERR 8
-#define SCSI_DEBUG_OPT_TRANSPORT_ERR 16
-#define SCSI_DEBUG_OPT_DIF_ERR 32
-#define SCSI_DEBUG_OPT_DIX_ERR 64
-#define SCSI_DEBUG_OPT_MAC_TIMEOUT 128
-#define SCSI_DEBUG_OPT_SHORT_TRANSFER 0x100
-#define SCSI_DEBUG_OPT_Q_NOISE 0x200
-#define SCSI_DEBUG_OPT_ALL_TSF 0x400
-#define SCSI_DEBUG_OPT_RARE_TSF 0x800
-#define SCSI_DEBUG_OPT_N_WCE 0x1000
-#define SCSI_DEBUG_OPT_RESET_NOISE 0x2000
-#define SCSI_DEBUG_OPT_NO_CDB_NOISE 0x4000
-#define SCSI_DEBUG_OPT_ALL_NOISE (0x1 | 0x200 | 0x2000)
+#define SDEBUG_LUN_0_VAL 0
+
+/* bit mask values for sdebug_opts */
+#define SDEBUG_OPT_NOISE 1
+#define SDEBUG_OPT_MEDIUM_ERR 2
+#define SDEBUG_OPT_TIMEOUT 4
+#define SDEBUG_OPT_RECOVERED_ERR 8
+#define SDEBUG_OPT_TRANSPORT_ERR 16
+#define SDEBUG_OPT_DIF_ERR 32
+#define SDEBUG_OPT_DIX_ERR 64
+#define SDEBUG_OPT_MAC_TIMEOUT 128
+#define SDEBUG_OPT_SHORT_TRANSFER 0x100
+#define SDEBUG_OPT_Q_NOISE 0x200
+#define SDEBUG_OPT_ALL_TSF 0x400
+#define SDEBUG_OPT_RARE_TSF 0x800
+#define SDEBUG_OPT_N_WCE 0x1000
+#define SDEBUG_OPT_RESET_NOISE 0x2000
+#define SDEBUG_OPT_NO_CDB_NOISE 0x4000
+#define SDEBUG_OPT_ALL_NOISE (SDEBUG_OPT_NOISE | SDEBUG_OPT_Q_NOISE | \
+ SDEBUG_OPT_RESET_NOISE)
+#define SDEBUG_OPT_ALL_INJECTING (SDEBUG_OPT_RECOVERED_ERR | \
+ SDEBUG_OPT_TRANSPORT_ERR | \
+ SDEBUG_OPT_DIF_ERR | SDEBUG_OPT_DIX_ERR | \
+ SDEBUG_OPT_SHORT_TRANSFER)
/* When "every_nth" > 0 then modulo "every_nth" commands:
- * - a no response is simulated if SCSI_DEBUG_OPT_TIMEOUT is set
+ * - a missing response is simulated if SDEBUG_OPT_TIMEOUT is set
* - a RECOVERED_ERROR is simulated on successful read and write
- * commands if SCSI_DEBUG_OPT_RECOVERED_ERR is set.
+ * commands if SDEBUG_OPT_RECOVERED_ERR is set.
* - a TRANSPORT_ERROR is simulated on successful read and write
- * commands if SCSI_DEBUG_OPT_TRANSPORT_ERR is set.
+ * commands if SDEBUG_OPT_TRANSPORT_ERR is set.
*
* When "every_nth" < 0 then after "- every_nth" commands:
- * - a no response is simulated if SCSI_DEBUG_OPT_TIMEOUT is set
+ * - a missing response is simulated if SDEBUG_OPT_TIMEOUT is set
* - a RECOVERED_ERROR is simulated on successful read and write
- * commands if SCSI_DEBUG_OPT_RECOVERED_ERR is set.
+ * commands if SDEBUG_OPT_RECOVERED_ERR is set.
* - a TRANSPORT_ERROR is simulated on successful read and write
- * commands if SCSI_DEBUG_OPT_TRANSPORT_ERR is set.
- * This will continue until some other action occurs (e.g. the user
- * writing a new value (other than -1 or 1) to every_nth via sysfs).
+ * commands if _DEBUG_OPT_TRANSPORT_ERR is set.
+ * This will continue on every subsequent command until some other action
+ * occurs (e.g. the user * writing a new value (other than -1 or 1) to
+ * every_nth via sysfs).
*/
-/* As indicated in SAM-5 and SPC-4 Unit Attentions (UAs)are returned in
+/* As indicated in SAM-5 and SPC-4 Unit Attentions (UAs) are returned in
* priority order. In the subset implemented here lower numbers have higher
* priority. The UA numbers should be a sequence starting from 0 with
* SDEBUG_NUM_UAS being 1 higher than the highest numbered UA. */
@@ -192,11 +196,7 @@
#define SDEBUG_UA_MICROCODE_CHANGED_WO_RESET 6
#define SDEBUG_NUM_UAS 7
-/* for check_readiness() */
-#define UAS_ONLY 1 /* check for UAs only */
-#define UAS_TUR 0 /* if no UAs then check if media access possible */
-
-/* when 1==SCSI_DEBUG_OPT_MEDIUM_ERR, a medium error is simulated at this
+/* when 1==SDEBUG_OPT_MEDIUM_ERR, a medium error is simulated at this
* sector on read commands: */
#define OPT_MEDIUM_ERR_ADDR 0x1234 /* that's sector 4660 in decimal */
#define OPT_MEDIUM_ERR_NUM 10 /* number of consecutive medium errs */
@@ -205,21 +205,108 @@
* or "peripheral device" addressing (value 0) */
#define SAM2_LUN_ADDRESS_METHOD 0
-/* SCSI_DEBUG_CANQUEUE is the maximum number of commands that can be queued
- * (for response) at one time. Can be reduced by max_queue option. Command
- * responses are not queued when delay=0 and ndelay=0. The per-device
- * DEF_CMD_PER_LUN can be changed via sysfs:
- * /sys/class/scsi_device/<h:c:t:l>/device/queue_depth but cannot exceed
- * SCSI_DEBUG_CANQUEUE. */
-#define SCSI_DEBUG_CANQUEUE_WORDS 9 /* a WORD is bits in a long */
-#define SCSI_DEBUG_CANQUEUE (SCSI_DEBUG_CANQUEUE_WORDS * BITS_PER_LONG)
+/* SDEBUG_CANQUEUE is the maximum number of commands that can be queued
+ * (for response) per submit queue at one time. Can be reduced by max_queue
+ * option. Command responses are not queued when jdelay=0 and ndelay=0. The
+ * per-device DEF_CMD_PER_LUN can be changed via sysfs:
+ * /sys/class/scsi_device/<h:c:t:l>/device/queue_depth
+ * but cannot exceed SDEBUG_CANQUEUE .
+ */
+#define SDEBUG_CANQUEUE_WORDS 3 /* a WORD is bits in a long */
+#define SDEBUG_CANQUEUE (SDEBUG_CANQUEUE_WORDS * BITS_PER_LONG)
#define DEF_CMD_PER_LUN 255
-#if DEF_CMD_PER_LUN > SCSI_DEBUG_CANQUEUE
-#warning "Expect DEF_CMD_PER_LUN <= SCSI_DEBUG_CANQUEUE"
-#endif
+#define F_D_IN 1
+#define F_D_OUT 2
+#define F_D_OUT_MAYBE 4 /* WRITE SAME, NDOB bit */
+#define F_D_UNKN 8
+#define F_RL_WLUN_OK 0x10
+#define F_SKIP_UA 0x20
+#define F_DELAY_OVERR 0x40
+#define F_SA_LOW 0x80 /* cdb byte 1, bits 4 to 0 */
+#define F_SA_HIGH 0x100 /* as used by variable length cdbs */
+#define F_INV_OP 0x200
+#define F_FAKE_RW 0x400
+#define F_M_ACCESS 0x800 /* media access */
-/* SCSI opcodes (first byte of cdb) mapped onto these indexes */
+#define FF_RESPOND (F_RL_WLUN_OK | F_SKIP_UA | F_DELAY_OVERR)
+#define FF_DIRECT_IO (F_M_ACCESS | F_FAKE_RW)
+#define FF_SA (F_SA_HIGH | F_SA_LOW)
+
+#define SDEBUG_MAX_PARTS 4
+
+#define SDEBUG_MAX_CMD_LEN 32
+
+
+struct sdebug_dev_info {
+ struct list_head dev_list;
+ unsigned int channel;
+ unsigned int target;
+ u64 lun;
+ uuid_be lu_name;
+ struct sdebug_host_info *sdbg_host;
+ unsigned long uas_bm[1];
+ atomic_t num_in_q;
+ atomic_t stopped;
+ bool used;
+};
+
+struct sdebug_host_info {
+ struct list_head host_list;
+ struct Scsi_Host *shost;
+ struct device dev;
+ struct list_head dev_info_list;
+};
+
+#define to_sdebug_host(d) \
+ container_of(d, struct sdebug_host_info, dev)
+
+struct sdebug_defer {
+ struct hrtimer hrt;
+ struct execute_work ew;
+ int sqa_idx; /* index of sdebug_queue array */
+ int qc_idx; /* index of sdebug_queued_cmd array within sqa_idx */
+ int issuing_cpu;
+};
+
+struct sdebug_queued_cmd {
+ /* corresponding bit set in in_use_bm[] in owning struct sdebug_queue
+ * instance indicates this slot is in use.
+ */
+ struct sdebug_defer *sd_dp;
+ struct scsi_cmnd *a_cmnd;
+ unsigned int inj_recovered:1;
+ unsigned int inj_transport:1;
+ unsigned int inj_dif:1;
+ unsigned int inj_dix:1;
+ unsigned int inj_short:1;
+};
+
+struct sdebug_queue {
+ struct sdebug_queued_cmd qc_arr[SDEBUG_CANQUEUE];
+ unsigned long in_use_bm[SDEBUG_CANQUEUE_WORDS];
+ spinlock_t qc_lock;
+ atomic_t blocked; /* to temporarily stop more being queued */
+};
+
+static atomic_t sdebug_cmnd_count; /* number of incoming commands */
+static atomic_t sdebug_completions; /* count of deferred completions */
+static atomic_t sdebug_miss_cpus; /* submission + completion cpus differ */
+static atomic_t sdebug_a_tsf; /* 'almost task set full' counter */
+
+struct opcode_info_t {
+ u8 num_attached; /* 0 if this is it (i.e. a leaf); use 0xff */
+ /* for terminating element */
+ u8 opcode; /* if num_attached > 0, preferred */
+ u16 sa; /* service action */
+ u32 flags; /* OR-ed set of SDEB_F_* */
+ int (*pfp)(struct scsi_cmnd *, struct sdebug_dev_info *);
+ const struct opcode_info_t *arrp; /* num_attached elements or NULL */
+ u8 len_mask[16]; /* len=len_mask[0], then mask for cdb[1]... */
+ /* ignore cdb bytes after position 15 */
+};
+
+/* SCSI opcodes (first byte of cdb) of interest mapped onto these indexes */
enum sdeb_opcode_index {
SDEB_I_INVALID_OPCODE = 0,
SDEB_I_INQUIRY = 1,
@@ -254,6 +341,7 @@
SDEB_I_LAST_ELEMENT = 30, /* keep this last */
};
+
static const unsigned char opcode_ind_arr[256] = {
/* 0x0; 0x0->0x1f: 6 byte cdbs */
SDEB_I_TEST_UNIT_READY, SDEB_I_REZERO_UNIT, 0, SDEB_I_REQUEST_SENSE,
@@ -274,7 +362,7 @@
0, 0, 0, SDEB_I_XDWRITEREAD, 0, SDEB_I_MODE_SELECT, SDEB_I_RESERVE,
SDEB_I_RELEASE,
0, 0, SDEB_I_MODE_SENSE, 0, 0, 0, 0, 0,
-/* 0x60; 0x60->0x7d are reserved */
+/* 0x60; 0x60->0x7d are reserved, 0x7e is "extended cdb" */
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, SDEB_I_VARIABLE_LEN,
@@ -297,24 +385,6 @@
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
};
-#define F_D_IN 1
-#define F_D_OUT 2
-#define F_D_OUT_MAYBE 4 /* WRITE SAME, NDOB bit */
-#define F_D_UNKN 8
-#define F_RL_WLUN_OK 0x10
-#define F_SKIP_UA 0x20
-#define F_DELAY_OVERR 0x40
-#define F_SA_LOW 0x80 /* cdb byte 1, bits 4 to 0 */
-#define F_SA_HIGH 0x100 /* as used by variable length cdbs */
-#define F_INV_OP 0x200
-#define F_FAKE_RW 0x400
-#define F_M_ACCESS 0x800 /* media access */
-
-#define FF_RESPOND (F_RL_WLUN_OK | F_SKIP_UA | F_DELAY_OVERR)
-#define FF_DIRECT_IO (F_M_ACCESS | F_FAKE_RW)
-#define FF_SA (F_SA_HIGH | F_SA_LOW)
-
-struct sdebug_dev_info;
static int resp_inquiry(struct scsi_cmnd *, struct sdebug_dev_info *);
static int resp_report_luns(struct scsi_cmnd *, struct sdebug_dev_info *);
static int resp_requests(struct scsi_cmnd *, struct sdebug_dev_info *);
@@ -337,18 +407,6 @@
static int resp_comp_write(struct scsi_cmnd *, struct sdebug_dev_info *);
static int resp_write_buffer(struct scsi_cmnd *, struct sdebug_dev_info *);
-struct opcode_info_t {
- u8 num_attached; /* 0 if this is it (i.e. a leaf); use 0xff
- * for terminating element */
- u8 opcode; /* if num_attached > 0, preferred */
- u16 sa; /* service action */
- u32 flags; /* OR-ed set of SDEB_F_* */
- int (*pfp)(struct scsi_cmnd *, struct sdebug_dev_info *);
- const struct opcode_info_t *arrp; /* num_attached elements or NULL */
- u8 len_mask[16]; /* len=len_mask[0], then mask for cdb[1]... */
- /* ignore cdb bytes after position 15 */
-};
-
static const struct opcode_info_t msense_iarr[1] = {
{0, 0x1a, 0, F_D_IN, NULL, NULL,
{6, 0xe8, 0xff, 0xff, 0xff, 0xc7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0} },
@@ -509,61 +567,52 @@
{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0} },
};
-struct sdebug_scmd_extra_t {
- bool inj_recovered;
- bool inj_transport;
- bool inj_dif;
- bool inj_dix;
- bool inj_short;
-};
-
-static int scsi_debug_add_host = DEF_NUM_HOST;
-static int scsi_debug_ato = DEF_ATO;
-static int scsi_debug_delay = DEF_DELAY;
-static int scsi_debug_dev_size_mb = DEF_DEV_SIZE_MB;
-static int scsi_debug_dif = DEF_DIF;
-static int scsi_debug_dix = DEF_DIX;
-static int scsi_debug_dsense = DEF_D_SENSE;
-static int scsi_debug_every_nth = DEF_EVERY_NTH;
-static int scsi_debug_fake_rw = DEF_FAKE_RW;
-static unsigned int scsi_debug_guard = DEF_GUARD;
-static int scsi_debug_lowest_aligned = DEF_LOWEST_ALIGNED;
-static int scsi_debug_max_luns = DEF_MAX_LUNS;
-static int scsi_debug_max_queue = SCSI_DEBUG_CANQUEUE;
+static int sdebug_add_host = DEF_NUM_HOST;
+static int sdebug_ato = DEF_ATO;
+static int sdebug_jdelay = DEF_JDELAY; /* if > 0 then unit is jiffies */
+static int sdebug_dev_size_mb = DEF_DEV_SIZE_MB;
+static int sdebug_dif = DEF_DIF;
+static int sdebug_dix = DEF_DIX;
+static int sdebug_dsense = DEF_D_SENSE;
+static int sdebug_every_nth = DEF_EVERY_NTH;
+static int sdebug_fake_rw = DEF_FAKE_RW;
+static unsigned int sdebug_guard = DEF_GUARD;
+static int sdebug_lowest_aligned = DEF_LOWEST_ALIGNED;
+static int sdebug_max_luns = DEF_MAX_LUNS;
+static int sdebug_max_queue = SDEBUG_CANQUEUE; /* per submit queue */
static atomic_t retired_max_queue; /* if > 0 then was prior max_queue */
-static int scsi_debug_ndelay = DEF_NDELAY;
-static int scsi_debug_no_lun_0 = DEF_NO_LUN_0;
-static int scsi_debug_no_uld = 0;
-static int scsi_debug_num_parts = DEF_NUM_PARTS;
-static int scsi_debug_num_tgts = DEF_NUM_TGTS; /* targets per host */
-static int scsi_debug_opt_blks = DEF_OPT_BLKS;
-static int scsi_debug_opts = DEF_OPTS;
-static int scsi_debug_physblk_exp = DEF_PHYSBLK_EXP;
-static int scsi_debug_ptype = DEF_PTYPE; /* SCSI peripheral type (0==disk) */
-static int scsi_debug_scsi_level = DEF_SCSI_LEVEL;
-static int scsi_debug_sector_size = DEF_SECTOR_SIZE;
-static int scsi_debug_virtual_gb = DEF_VIRTUAL_GB;
-static int scsi_debug_vpd_use_hostno = DEF_VPD_USE_HOSTNO;
-static unsigned int scsi_debug_lbpu = DEF_LBPU;
-static unsigned int scsi_debug_lbpws = DEF_LBPWS;
-static unsigned int scsi_debug_lbpws10 = DEF_LBPWS10;
-static unsigned int scsi_debug_lbprz = DEF_LBPRZ;
-static unsigned int scsi_debug_unmap_alignment = DEF_UNMAP_ALIGNMENT;
-static unsigned int scsi_debug_unmap_granularity = DEF_UNMAP_GRANULARITY;
-static unsigned int scsi_debug_unmap_max_blocks = DEF_UNMAP_MAX_BLOCKS;
-static unsigned int scsi_debug_unmap_max_desc = DEF_UNMAP_MAX_DESC;
-static unsigned int scsi_debug_write_same_length = DEF_WRITESAME_LENGTH;
-static bool scsi_debug_removable = DEF_REMOVABLE;
-static bool scsi_debug_clustering;
-static bool scsi_debug_host_lock = DEF_HOST_LOCK;
-static bool scsi_debug_strict = DEF_STRICT;
+static int sdebug_ndelay = DEF_NDELAY; /* if > 0 then unit is nanoseconds */
+static int sdebug_no_lun_0 = DEF_NO_LUN_0;
+static int sdebug_no_uld;
+static int sdebug_num_parts = DEF_NUM_PARTS;
+static int sdebug_num_tgts = DEF_NUM_TGTS; /* targets per host */
+static int sdebug_opt_blks = DEF_OPT_BLKS;
+static int sdebug_opts = DEF_OPTS;
+static int sdebug_physblk_exp = DEF_PHYSBLK_EXP;
+static int sdebug_ptype = DEF_PTYPE; /* SCSI peripheral device type */
+static int sdebug_scsi_level = DEF_SCSI_LEVEL;
+static int sdebug_sector_size = DEF_SECTOR_SIZE;
+static int sdebug_virtual_gb = DEF_VIRTUAL_GB;
+static int sdebug_vpd_use_hostno = DEF_VPD_USE_HOSTNO;
+static unsigned int sdebug_lbpu = DEF_LBPU;
+static unsigned int sdebug_lbpws = DEF_LBPWS;
+static unsigned int sdebug_lbpws10 = DEF_LBPWS10;
+static unsigned int sdebug_lbprz = DEF_LBPRZ;
+static unsigned int sdebug_unmap_alignment = DEF_UNMAP_ALIGNMENT;
+static unsigned int sdebug_unmap_granularity = DEF_UNMAP_GRANULARITY;
+static unsigned int sdebug_unmap_max_blocks = DEF_UNMAP_MAX_BLOCKS;
+static unsigned int sdebug_unmap_max_desc = DEF_UNMAP_MAX_DESC;
+static unsigned int sdebug_write_same_length = DEF_WRITESAME_LENGTH;
+static int sdebug_uuid_ctl = DEF_UUID_CTL;
+static bool sdebug_removable = DEF_REMOVABLE;
+static bool sdebug_clustering;
+static bool sdebug_host_lock = DEF_HOST_LOCK;
+static bool sdebug_strict = DEF_STRICT;
static bool sdebug_any_injecting_opt;
-
-static atomic_t sdebug_cmnd_count;
-static atomic_t sdebug_completions;
-static atomic_t sdebug_a_tsf; /* counter of 'almost' TSFs */
-
-#define DEV_READONLY(TGT) (0)
+static bool sdebug_verbose;
+static bool have_dif_prot;
+static bool sdebug_statistics = DEF_STATISTICS;
+static bool sdebug_mq_active;
static unsigned int sdebug_store_sectors;
static sector_t sdebug_capacity; /* in sectors */
@@ -574,59 +623,10 @@
static int sdebug_cylinders_per; /* cylinders per surface */
static int sdebug_sectors_per; /* sectors per cylinder */
-#define SDEBUG_MAX_PARTS 4
-
-#define SCSI_DEBUG_MAX_CMD_LEN 32
-
-static unsigned int scsi_debug_lbp(void)
-{
- return ((0 == scsi_debug_fake_rw) &&
- (scsi_debug_lbpu | scsi_debug_lbpws | scsi_debug_lbpws10));
-}
-
-struct sdebug_dev_info {
- struct list_head dev_list;
- unsigned int channel;
- unsigned int target;
- u64 lun;
- struct sdebug_host_info *sdbg_host;
- unsigned long uas_bm[1];
- atomic_t num_in_q;
- char stopped; /* TODO: should be atomic */
- bool used;
-};
-
-struct sdebug_host_info {
- struct list_head host_list;
- struct Scsi_Host *shost;
- struct device dev;
- struct list_head dev_info_list;
-};
-
-#define to_sdebug_host(d) \
- container_of(d, struct sdebug_host_info, dev)
-
static LIST_HEAD(sdebug_host_list);
static DEFINE_SPINLOCK(sdebug_host_list_lock);
-
-struct sdebug_hrtimer { /* ... is derived from hrtimer */
- struct hrtimer hrt; /* must be first element */
- int qa_indx;
-};
-
-struct sdebug_queued_cmd {
- /* in_use flagged by a bit in queued_in_use_bm[] */
- struct timer_list *cmnd_timerp;
- struct tasklet_struct *tletp;
- struct sdebug_hrtimer *sd_hrtp;
- struct scsi_cmnd * a_cmnd;
-};
-static struct sdebug_queued_cmd queued_arr[SCSI_DEBUG_CANQUEUE];
-static unsigned long queued_in_use_bm[SCSI_DEBUG_CANQUEUE_WORDS];
-
-
-static unsigned char * fake_storep; /* ramdisk storage */
+static unsigned char *fake_storep; /* ramdisk storage */
static struct sd_dif_tuple *dif_storep; /* protection info */
static void *map_storep; /* provisioning map */
@@ -640,7 +640,9 @@
static int dix_reads;
static int dif_errors;
-static DEFINE_SPINLOCK(queued_arr_lock);
+static int submit_queues = DEF_SUBMIT_QUEUES; /* > 1 for multi-queue (mq) */
+static struct sdebug_queue *sdebug_q_arr; /* ptr to array of submit queues */
+
static DEFINE_RWLOCK(atomic_rw);
static char sdebug_proc_name[] = MY_NAME;
@@ -662,19 +664,22 @@
static const int device_qfull_result =
(DID_OK << 16) | (COMMAND_COMPLETE << 8) | SAM_STAT_TASK_SET_FULL;
-static unsigned char caching_pg[] = {0x8, 18, 0x14, 0, 0xff, 0xff, 0, 0,
- 0xff, 0xff, 0xff, 0xff, 0x80, 0x14, 0, 0,
- 0, 0, 0, 0};
-static unsigned char ctrl_m_pg[] = {0xa, 10, 2, 0, 0, 0, 0, 0,
- 0, 0, 0x2, 0x4b};
-static unsigned char iec_m_pg[] = {0x1c, 0xa, 0x08, 0, 0, 0, 0, 0,
- 0, 0, 0x0, 0x0};
+
+/* Only do the extra work involved in logical block provisioning if one or
+ * more of the lbpu, lbpws or lbpws10 parameters are given and we are doing
+ * real reads and writes (i.e. not skipping them for speed).
+ */
+static inline bool scsi_debug_lbp(void)
+{
+ return 0 == sdebug_fake_rw &&
+ (sdebug_lbpu || sdebug_lbpws || sdebug_lbpws10);
+}
static void *fake_store(unsigned long long lba)
{
lba = do_div(lba, sdebug_store_sectors);
- return fake_storep + lba * scsi_debug_sector_size;
+ return fake_storep + lba * sdebug_sector_size;
}
static struct sd_dif_tuple *dif_store(sector_t sector)
@@ -684,9 +689,6 @@
return dif_storep + sector;
}
-static int sdebug_add_adapter(void);
-static void sdebug_remove_adapter(void);
-
static void sdebug_max_tgts_luns(void)
{
struct sdebug_host_info *sdbg_host;
@@ -696,11 +698,11 @@
list_for_each_entry(sdbg_host, &sdebug_host_list, host_list) {
hpnt = sdbg_host->shost;
if ((hpnt->this_id >= 0) &&
- (scsi_debug_num_tgts > hpnt->this_id))
- hpnt->max_id = scsi_debug_num_tgts + 1;
+ (sdebug_num_tgts > hpnt->this_id))
+ hpnt->max_id = sdebug_num_tgts + 1;
else
- hpnt->max_id = scsi_debug_num_tgts;
- /* scsi_debug_max_luns; */
+ hpnt->max_id = sdebug_num_tgts;
+ /* sdebug_max_luns; */
hpnt->max_lun = SCSI_W_LUN_REPORT_LUNS + 1;
}
spin_unlock(&sdebug_host_list_lock);
@@ -709,9 +711,9 @@
enum sdeb_cmd_data {SDEB_IN_DATA = 0, SDEB_IN_CDB = 1};
/* Set in_bit to -1 to indicate no bit position of invalid field */
-static void
-mk_sense_invalid_fld(struct scsi_cmnd *scp, enum sdeb_cmd_data c_d,
- int in_byte, int in_bit)
+static void mk_sense_invalid_fld(struct scsi_cmnd *scp,
+ enum sdeb_cmd_data c_d,
+ int in_byte, int in_bit)
{
unsigned char *sbuff;
u8 sks[4];
@@ -725,8 +727,7 @@
}
asc = c_d ? INVALID_FIELD_IN_CDB : INVALID_FIELD_IN_PARAM_LIST;
memset(sbuff, 0, SCSI_SENSE_BUFFERSIZE);
- scsi_build_sense_buffer(scsi_debug_dsense, sbuff, ILLEGAL_REQUEST,
- asc, 0);
+ scsi_build_sense_buffer(sdebug_dsense, sbuff, ILLEGAL_REQUEST, asc, 0);
memset(sks, 0, sizeof(sks));
sks[0] = 0x80;
if (c_d)
@@ -736,7 +737,7 @@
sks[0] |= 0x7 & in_bit;
}
put_unaligned_be16(in_byte, sks + 1);
- if (scsi_debug_dsense) {
+ if (sdebug_dsense) {
sl = sbuff[7] + 8;
sbuff[7] = sl;
sbuff[sl] = 0x2;
@@ -744,7 +745,7 @@
memcpy(sbuff + sl + 4, sks, 3);
} else
memcpy(sbuff + 15, sks, 3);
- if (SCSI_DEBUG_OPT_NOISE & scsi_debug_opts)
+ if (sdebug_verbose)
sdev_printk(KERN_INFO, scp->device, "%s: [sense_key,asc,ascq"
"]: [0x5,0x%x,0x0] %c byte=%d, bit=%d\n",
my_name, asc, c_d ? 'C' : 'D', in_byte, in_bit);
@@ -762,23 +763,22 @@
}
memset(sbuff, 0, SCSI_SENSE_BUFFERSIZE);
- scsi_build_sense_buffer(scsi_debug_dsense, sbuff, key, asc, asq);
+ scsi_build_sense_buffer(sdebug_dsense, sbuff, key, asc, asq);
- if (SCSI_DEBUG_OPT_NOISE & scsi_debug_opts)
+ if (sdebug_verbose)
sdev_printk(KERN_INFO, scp->device,
"%s: [sense_key,asc,ascq]: [0x%x,0x%x,0x%x]\n",
my_name, key, asc, asq);
}
-static void
-mk_sense_invalid_opcode(struct scsi_cmnd *scp)
+static void mk_sense_invalid_opcode(struct scsi_cmnd *scp)
{
mk_sense_buffer(scp, ILLEGAL_REQUEST, INVALID_OPCODE, 0);
}
static int scsi_debug_ioctl(struct scsi_device *dev, int cmd, void __user *arg)
{
- if (SCSI_DEBUG_OPT_NOISE & scsi_debug_opts) {
+ if (sdebug_verbose) {
if (0x1261 == cmd)
sdev_printk(KERN_INFO, dev,
"%s: BLKFLSBUF [0x1261]\n", __func__);
@@ -810,11 +810,9 @@
spin_unlock(&sdebug_host_list_lock);
}
-static int check_readiness(struct scsi_cmnd *SCpnt, int uas_only,
- struct sdebug_dev_info * devip)
+static int make_ua(struct scsi_cmnd *scp, struct sdebug_dev_info *devip)
{
int k;
- bool debug = !!(SCSI_DEBUG_OPT_NOISE & scsi_debug_opts);
k = find_first_bit(devip->uas_bm, SDEBUG_NUM_UAS);
if (k != SDEBUG_NUM_UAS) {
@@ -822,40 +820,41 @@
switch (k) {
case SDEBUG_UA_POR:
- mk_sense_buffer(SCpnt, UNIT_ATTENTION,
- UA_RESET_ASC, POWER_ON_RESET_ASCQ);
- if (debug)
+ mk_sense_buffer(scp, UNIT_ATTENTION, UA_RESET_ASC,
+ POWER_ON_RESET_ASCQ);
+ if (sdebug_verbose)
cp = "power on reset";
break;
case SDEBUG_UA_BUS_RESET:
- mk_sense_buffer(SCpnt, UNIT_ATTENTION,
- UA_RESET_ASC, BUS_RESET_ASCQ);
- if (debug)
+ mk_sense_buffer(scp, UNIT_ATTENTION, UA_RESET_ASC,
+ BUS_RESET_ASCQ);
+ if (sdebug_verbose)
cp = "bus reset";
break;
case SDEBUG_UA_MODE_CHANGED:
- mk_sense_buffer(SCpnt, UNIT_ATTENTION,
- UA_CHANGED_ASC, MODE_CHANGED_ASCQ);
- if (debug)
+ mk_sense_buffer(scp, UNIT_ATTENTION, UA_CHANGED_ASC,
+ MODE_CHANGED_ASCQ);
+ if (sdebug_verbose)
cp = "mode parameters changed";
break;
case SDEBUG_UA_CAPACITY_CHANGED:
- mk_sense_buffer(SCpnt, UNIT_ATTENTION,
- UA_CHANGED_ASC, CAPACITY_CHANGED_ASCQ);
- if (debug)
+ mk_sense_buffer(scp, UNIT_ATTENTION, UA_CHANGED_ASC,
+ CAPACITY_CHANGED_ASCQ);
+ if (sdebug_verbose)
cp = "capacity data changed";
break;
case SDEBUG_UA_MICROCODE_CHANGED:
- mk_sense_buffer(SCpnt, UNIT_ATTENTION,
- TARGET_CHANGED_ASC, MICROCODE_CHANGED_ASCQ);
- if (debug)
+ mk_sense_buffer(scp, UNIT_ATTENTION,
+ TARGET_CHANGED_ASC,
+ MICROCODE_CHANGED_ASCQ);
+ if (sdebug_verbose)
cp = "microcode has been changed";
break;
case SDEBUG_UA_MICROCODE_CHANGED_WO_RESET:
- mk_sense_buffer(SCpnt, UNIT_ATTENTION,
+ mk_sense_buffer(scp, UNIT_ATTENTION,
TARGET_CHANGED_ASC,
MICROCODE_CHANGED_WO_RESET_ASCQ);
- if (debug)
+ if (sdebug_verbose)
cp = "microcode has been changed without reset";
break;
case SDEBUG_UA_LUNS_CHANGED:
@@ -864,40 +863,30 @@
* ASC/ASCQ REPORTED LUNS DATA HAS CHANGED on every LUN
* on the target, until a REPORT LUNS command is
* received. SPC-4 behavior is to report it only once.
- * NOTE: scsi_debug_scsi_level does not use the same
+ * NOTE: sdebug_scsi_level does not use the same
* values as struct scsi_device->scsi_level.
*/
- if (scsi_debug_scsi_level >= 6) /* SPC-4 and above */
+ if (sdebug_scsi_level >= 6) /* SPC-4 and above */
clear_luns_changed_on_target(devip);
- mk_sense_buffer(SCpnt, UNIT_ATTENTION,
+ mk_sense_buffer(scp, UNIT_ATTENTION,
TARGET_CHANGED_ASC,
LUNS_CHANGED_ASCQ);
- if (debug)
+ if (sdebug_verbose)
cp = "reported luns data has changed";
break;
default:
- pr_warn("%s: unexpected unit attention code=%d\n",
- __func__, k);
- if (debug)
+ pr_warn("unexpected unit attention code=%d\n", k);
+ if (sdebug_verbose)
cp = "unknown";
break;
}
clear_bit(k, devip->uas_bm);
- if (debug)
- sdev_printk(KERN_INFO, SCpnt->device,
+ if (sdebug_verbose)
+ sdev_printk(KERN_INFO, scp->device,
"%s reports: Unit attention: %s\n",
my_name, cp);
return check_condition_result;
}
- if ((UAS_TUR == uas_only) && devip->stopped) {
- mk_sense_buffer(SCpnt, NOT_READY, LOGICAL_UNIT_NOT_READY,
- 0x2);
- if (debug)
- sdev_printk(KERN_INFO, SCpnt->device,
- "%s reports: Not ready: %s\n", my_name,
- "initializing command required");
- return check_condition_result;
- }
return 0;
}
@@ -911,7 +900,7 @@
if (!sdb->length)
return 0;
if (!(scsi_bidi_cmnd(scp) || scp->sc_data_direction == DMA_FROM_DEVICE))
- return (DID_ERROR << 16);
+ return DID_ERROR << 16;
act_len = sg_copy_from_buffer(sdb->table.sgl, sdb->table.nents,
arr, arr_len);
@@ -935,13 +924,17 @@
static const char * inq_vendor_id = "Linux ";
static const char * inq_product_id = "scsi_debug ";
-static const char *inq_product_rev = "0184"; /* version less '.' */
+static const char *inq_product_rev = "0186"; /* version less '.' */
+/* Use some locally assigned NAAs for SAS addresses. */
+static const u64 naa3_comp_a = 0x3222222000000000ULL;
+static const u64 naa3_comp_b = 0x3333333000000000ULL;
+static const u64 naa3_comp_c = 0x3111111000000000ULL;
/* Device identification VPD page. Returns number of bytes placed in arr */
-static int inquiry_evpd_83(unsigned char * arr, int port_group_id,
- int target_dev_id, int dev_id_num,
- const char * dev_id_str,
- int dev_id_str_len)
+static int inquiry_vpd_83(unsigned char *arr, int port_group_id,
+ int target_dev_id, int dev_id_num,
+ const char *dev_id_str, int dev_id_str_len,
+ const uuid_be *lu_name)
{
int num, port_a;
char b[32];
@@ -958,19 +951,25 @@
arr[3] = num;
num += 4;
if (dev_id_num >= 0) {
- /* NAA-5, Logical unit identifier (binary) */
- arr[num++] = 0x1; /* binary (not necessarily sas) */
- arr[num++] = 0x3; /* PIV=0, lu, naa */
- arr[num++] = 0x0;
- arr[num++] = 0x8;
- arr[num++] = 0x53; /* naa-5 ieee company id=0x333333 (fake) */
- arr[num++] = 0x33;
- arr[num++] = 0x33;
- arr[num++] = 0x30;
- arr[num++] = (dev_id_num >> 24);
- arr[num++] = (dev_id_num >> 16) & 0xff;
- arr[num++] = (dev_id_num >> 8) & 0xff;
- arr[num++] = dev_id_num & 0xff;
+ if (sdebug_uuid_ctl) {
+ /* Locally assigned UUID */
+ arr[num++] = 0x1; /* binary (not necessarily sas) */
+ arr[num++] = 0xa; /* PIV=0, lu, naa */
+ arr[num++] = 0x0;
+ arr[num++] = 0x12;
+ arr[num++] = 0x10; /* uuid type=1, locally assigned */
+ arr[num++] = 0x0;
+ memcpy(arr + num, lu_name, 16);
+ num += 16;
+ } else {
+ /* NAA-3, Logical unit identifier (binary) */
+ arr[num++] = 0x1; /* binary (not necessarily sas) */
+ arr[num++] = 0x3; /* PIV=0, lu, naa */
+ arr[num++] = 0x0;
+ arr[num++] = 0x8;
+ put_unaligned_be64(naa3_comp_b + dev_id_num, arr + num);
+ num += 8;
+ }
/* Target relative port number */
arr[num++] = 0x61; /* proto=sas, binary */
arr[num++] = 0x94; /* PIV=1, target port, rel port */
@@ -981,47 +980,35 @@
arr[num++] = 0x0;
arr[num++] = 0x1; /* relative port A */
}
- /* NAA-5, Target port identifier */
+ /* NAA-3, Target port identifier */
arr[num++] = 0x61; /* proto=sas, binary */
arr[num++] = 0x93; /* piv=1, target port, naa */
arr[num++] = 0x0;
arr[num++] = 0x8;
- arr[num++] = 0x52; /* naa-5, company id=0x222222 (fake) */
- arr[num++] = 0x22;
- arr[num++] = 0x22;
- arr[num++] = 0x20;
- arr[num++] = (port_a >> 24);
- arr[num++] = (port_a >> 16) & 0xff;
- arr[num++] = (port_a >> 8) & 0xff;
- arr[num++] = port_a & 0xff;
- /* NAA-5, Target port group identifier */
+ put_unaligned_be64(naa3_comp_a + port_a, arr + num);
+ num += 8;
+ /* NAA-3, Target port group identifier */
arr[num++] = 0x61; /* proto=sas, binary */
arr[num++] = 0x95; /* piv=1, target port group id */
arr[num++] = 0x0;
arr[num++] = 0x4;
arr[num++] = 0;
arr[num++] = 0;
- arr[num++] = (port_group_id >> 8) & 0xff;
- arr[num++] = port_group_id & 0xff;
- /* NAA-5, Target device identifier */
+ put_unaligned_be16(port_group_id, arr + num);
+ num += 2;
+ /* NAA-3, Target device identifier */
arr[num++] = 0x61; /* proto=sas, binary */
arr[num++] = 0xa3; /* piv=1, target device, naa */
arr[num++] = 0x0;
arr[num++] = 0x8;
- arr[num++] = 0x52; /* naa-5, company id=0x222222 (fake) */
- arr[num++] = 0x22;
- arr[num++] = 0x22;
- arr[num++] = 0x20;
- arr[num++] = (target_dev_id >> 24);
- arr[num++] = (target_dev_id >> 16) & 0xff;
- arr[num++] = (target_dev_id >> 8) & 0xff;
- arr[num++] = target_dev_id & 0xff;
+ put_unaligned_be64(naa3_comp_a + target_dev_id, arr + num);
+ num += 8;
/* SCSI name string: Target device identifier */
arr[num++] = 0x63; /* proto=sas, UTF-8 */
arr[num++] = 0xa8; /* piv=1, target device, SCSI name string */
arr[num++] = 0x0;
arr[num++] = 24;
- memcpy(arr + num, "naa.52222220", 12);
+ memcpy(arr + num, "naa.32222220", 12);
num += 12;
snprintf(b, sizeof(b), "%08X", target_dev_id);
memcpy(arr + num, b, 8);
@@ -1031,7 +1018,6 @@
return num;
}
-
static unsigned char vpd84_data[] = {
/* from 4th byte */ 0x22,0x22,0x22,0x0,0xbb,0x0,
0x22,0x22,0x22,0x0,0xbb,0x1,
@@ -1039,14 +1025,14 @@
};
/* Software interface identification VPD page */
-static int inquiry_evpd_84(unsigned char * arr)
+static int inquiry_vpd_84(unsigned char *arr)
{
memcpy(arr, vpd84_data, sizeof(vpd84_data));
return sizeof(vpd84_data);
}
/* Management network addresses VPD page */
-static int inquiry_evpd_85(unsigned char * arr)
+static int inquiry_vpd_85(unsigned char *arr)
{
int num = 0;
const char * na1 = "https://www.kernel.org/config";
@@ -1081,7 +1067,7 @@
}
/* SCSI ports VPD page */
-static int inquiry_evpd_88(unsigned char * arr, int target_dev_id)
+static int inquiry_vpd_88(unsigned char *arr, int target_dev_id)
{
int num = 0;
int port_a, port_b;
@@ -1101,15 +1087,8 @@
arr[num++] = 0x93; /* PIV=1, target port, NAA */
arr[num++] = 0x0; /* reserved */
arr[num++] = 0x8; /* length */
- arr[num++] = 0x52; /* NAA-5, company_id=0x222222 (fake) */
- arr[num++] = 0x22;
- arr[num++] = 0x22;
- arr[num++] = 0x20;
- arr[num++] = (port_a >> 24);
- arr[num++] = (port_a >> 16) & 0xff;
- arr[num++] = (port_a >> 8) & 0xff;
- arr[num++] = port_a & 0xff;
-
+ put_unaligned_be64(naa3_comp_a + port_a, arr + num);
+ num += 8;
arr[num++] = 0x0; /* reserved */
arr[num++] = 0x0; /* reserved */
arr[num++] = 0x0;
@@ -1123,14 +1102,8 @@
arr[num++] = 0x93; /* PIV=1, target port, NAA */
arr[num++] = 0x0; /* reserved */
arr[num++] = 0x8; /* length */
- arr[num++] = 0x52; /* NAA-5, company_id=0x222222 (fake) */
- arr[num++] = 0x22;
- arr[num++] = 0x22;
- arr[num++] = 0x20;
- arr[num++] = (port_b >> 24);
- arr[num++] = (port_b >> 16) & 0xff;
- arr[num++] = (port_b >> 8) & 0xff;
- arr[num++] = port_b & 0xff;
+ put_unaligned_be64(naa3_comp_a + port_b, arr + num);
+ num += 8;
return num;
}
@@ -1181,7 +1154,7 @@
};
/* ATA Information VPD page */
-static int inquiry_evpd_89(unsigned char * arr)
+static int inquiry_vpd_89(unsigned char *arr)
{
memcpy(arr, vpd89_data, sizeof(vpd89_data));
return sizeof(vpd89_data);
@@ -1196,47 +1169,42 @@
};
/* Block limits VPD page (SBC-3) */
-static int inquiry_evpd_b0(unsigned char * arr)
+static int inquiry_vpd_b0(unsigned char *arr)
{
unsigned int gran;
memcpy(arr, vpdb0_data, sizeof(vpdb0_data));
/* Optimal transfer length granularity */
- gran = 1 << scsi_debug_physblk_exp;
- arr[2] = (gran >> 8) & 0xff;
- arr[3] = gran & 0xff;
+ gran = 1 << sdebug_physblk_exp;
+ put_unaligned_be16(gran, arr + 2);
/* Maximum Transfer Length */
- if (sdebug_store_sectors > 0x400) {
- arr[4] = (sdebug_store_sectors >> 24) & 0xff;
- arr[5] = (sdebug_store_sectors >> 16) & 0xff;
- arr[6] = (sdebug_store_sectors >> 8) & 0xff;
- arr[7] = sdebug_store_sectors & 0xff;
- }
+ if (sdebug_store_sectors > 0x400)
+ put_unaligned_be32(sdebug_store_sectors, arr + 4);
/* Optimal Transfer Length */
- put_unaligned_be32(scsi_debug_opt_blks, &arr[8]);
+ put_unaligned_be32(sdebug_opt_blks, &arr[8]);
- if (scsi_debug_lbpu) {
+ if (sdebug_lbpu) {
/* Maximum Unmap LBA Count */
- put_unaligned_be32(scsi_debug_unmap_max_blocks, &arr[16]);
+ put_unaligned_be32(sdebug_unmap_max_blocks, &arr[16]);
/* Maximum Unmap Block Descriptor Count */
- put_unaligned_be32(scsi_debug_unmap_max_desc, &arr[20]);
+ put_unaligned_be32(sdebug_unmap_max_desc, &arr[20]);
}
/* Unmap Granularity Alignment */
- if (scsi_debug_unmap_alignment) {
- put_unaligned_be32(scsi_debug_unmap_alignment, &arr[28]);
+ if (sdebug_unmap_alignment) {
+ put_unaligned_be32(sdebug_unmap_alignment, &arr[28]);
arr[28] |= 0x80; /* UGAVALID */
}
/* Optimal Unmap Granularity */
- put_unaligned_be32(scsi_debug_unmap_granularity, &arr[24]);
+ put_unaligned_be32(sdebug_unmap_granularity, &arr[24]);
/* Maximum WRITE SAME Length */
- put_unaligned_be64(scsi_debug_write_same_length, &arr[32]);
+ put_unaligned_be64(sdebug_write_same_length, &arr[32]);
return 0x3c; /* Mandatory page length for Logical Block Provisioning */
@@ -1244,7 +1212,7 @@
}
/* Block device characteristics VPD page (SBC-3) */
-static int inquiry_evpd_b1(unsigned char *arr)
+static int inquiry_vpd_b1(unsigned char *arr)
{
memset(arr, 0, 0x3c);
arr[0] = 0;
@@ -1255,24 +1223,22 @@
return 0x3c;
}
-/* Logical block provisioning VPD page (SBC-3) */
-static int inquiry_evpd_b2(unsigned char *arr)
+/* Logical block provisioning VPD page (SBC-4) */
+static int inquiry_vpd_b2(unsigned char *arr)
{
memset(arr, 0, 0x4);
arr[0] = 0; /* threshold exponent */
-
- if (scsi_debug_lbpu)
+ if (sdebug_lbpu)
arr[1] = 1 << 7;
-
- if (scsi_debug_lbpws)
+ if (sdebug_lbpws)
arr[1] |= 1 << 6;
-
- if (scsi_debug_lbpws10)
+ if (sdebug_lbpws10)
arr[1] |= 1 << 5;
-
- if (scsi_debug_lbprz)
- arr[1] |= 1 << 2;
-
+ if (sdebug_lbprz && scsi_debug_lbp())
+ arr[1] |= (sdebug_lbprz & 0x7) << 2; /* sbc4r07 and later */
+ /* anc_sup=0; dp=0 (no provisioning group descriptor) */
+ /* minimum_percentage=0; provisioning_type=0 (unknown) */
+ /* threshold_percentage=0 */
return 0x4;
}
@@ -1285,19 +1251,20 @@
unsigned char * arr;
unsigned char *cmd = scp->cmnd;
int alloc_len, n, ret;
- bool have_wlun;
+ bool have_wlun, is_disk;
- alloc_len = (cmd[3] << 8) + cmd[4];
+ alloc_len = get_unaligned_be16(cmd + 3);
arr = kzalloc(SDEBUG_MAX_INQ_ARR_SZ, GFP_ATOMIC);
if (! arr)
return DID_REQUEUE << 16;
- have_wlun = (scp->device->lun == SCSI_W_LUN_REPORT_LUNS);
+ is_disk = (sdebug_ptype == TYPE_DISK);
+ have_wlun = scsi_is_wlun(scp->device->lun);
if (have_wlun)
- pq_pdt = 0x1e; /* present, wlun */
- else if (scsi_debug_no_lun_0 && (0 == devip->lun))
- pq_pdt = 0x7f; /* not present, no device type */
+ pq_pdt = TYPE_WLUN; /* present, wlun */
+ else if (sdebug_no_lun_0 && (devip->lun == SDEBUG_LUN_0_VAL))
+ pq_pdt = 0x7f; /* not present, PQ=3, PDT=0x1f */
else
- pq_pdt = (scsi_debug_ptype & 0x1f);
+ pq_pdt = (sdebug_ptype & 0x1f);
arr[0] = pq_pdt;
if (0x2 & cmd[1]) { /* CMDDT bit set */
mk_sense_invalid_fld(scp, SDEB_IN_CDB, 1, 1);
@@ -1310,7 +1277,7 @@
port_group_id = (((host_no + 1) & 0x7f) << 8) +
(devip->channel & 0x7f);
- if (0 == scsi_debug_vpd_use_hostno)
+ if (sdebug_vpd_use_hostno == 0)
host_no = 0;
lu_id_num = have_wlun ? -1 : (((host_no + 1) * 2000) +
(devip->target * 1000) + devip->lun);
@@ -1328,11 +1295,12 @@
arr[n++] = 0x86; /* extended inquiry */
arr[n++] = 0x87; /* mode page policy */
arr[n++] = 0x88; /* SCSI ports */
- arr[n++] = 0x89; /* ATA information */
- arr[n++] = 0xb0; /* Block limits (SBC) */
- arr[n++] = 0xb1; /* Block characteristics (SBC) */
- if (scsi_debug_lbp()) /* Logical Block Prov. (SBC) */
- arr[n++] = 0xb2;
+ if (is_disk) { /* SBC only */
+ arr[n++] = 0x89; /* ATA information */
+ arr[n++] = 0xb0; /* Block limits */
+ arr[n++] = 0xb1; /* Block characteristics */
+ arr[n++] = 0xb2; /* Logical Block Prov */
+ }
arr[3] = n - 4; /* number of supported VPD pages */
} else if (0x80 == cmd[2]) { /* unit serial number */
arr[1] = cmd[2]; /*sanity */
@@ -1340,21 +1308,22 @@
memcpy(&arr[4], lu_id_str, len);
} else if (0x83 == cmd[2]) { /* device identification */
arr[1] = cmd[2]; /*sanity */
- arr[3] = inquiry_evpd_83(&arr[4], port_group_id,
- target_dev_id, lu_id_num,
- lu_id_str, len);
+ arr[3] = inquiry_vpd_83(&arr[4], port_group_id,
+ target_dev_id, lu_id_num,
+ lu_id_str, len,
+ &devip->lu_name);
} else if (0x84 == cmd[2]) { /* Software interface ident. */
arr[1] = cmd[2]; /*sanity */
- arr[3] = inquiry_evpd_84(&arr[4]);
+ arr[3] = inquiry_vpd_84(&arr[4]);
} else if (0x85 == cmd[2]) { /* Management network addresses */
arr[1] = cmd[2]; /*sanity */
- arr[3] = inquiry_evpd_85(&arr[4]);
+ arr[3] = inquiry_vpd_85(&arr[4]);
} else if (0x86 == cmd[2]) { /* extended inquiry */
arr[1] = cmd[2]; /*sanity */
arr[3] = 0x3c; /* number of following entries */
- if (scsi_debug_dif == SD_DIF_TYPE3_PROTECTION)
+ if (sdebug_dif == SD_DIF_TYPE3_PROTECTION)
arr[4] = 0x4; /* SPT: GRD_CHK:1 */
- else if (scsi_debug_dif)
+ else if (have_dif_prot)
arr[4] = 0x5; /* SPT: GRD_CHK:1, REF_CHK:1 */
else
arr[4] = 0x0; /* no protection stuff */
@@ -1368,39 +1337,38 @@
arr[10] = 0x82; /* mlus, per initiator port */
} else if (0x88 == cmd[2]) { /* SCSI Ports */
arr[1] = cmd[2]; /*sanity */
- arr[3] = inquiry_evpd_88(&arr[4], target_dev_id);
- } else if (0x89 == cmd[2]) { /* ATA information */
+ arr[3] = inquiry_vpd_88(&arr[4], target_dev_id);
+ } else if (is_disk && 0x89 == cmd[2]) { /* ATA information */
arr[1] = cmd[2]; /*sanity */
- n = inquiry_evpd_89(&arr[4]);
- arr[2] = (n >> 8);
- arr[3] = (n & 0xff);
- } else if (0xb0 == cmd[2]) { /* Block limits (SBC) */
+ n = inquiry_vpd_89(&arr[4]);
+ put_unaligned_be16(n, arr + 2);
+ } else if (is_disk && 0xb0 == cmd[2]) { /* Block limits */
arr[1] = cmd[2]; /*sanity */
- arr[3] = inquiry_evpd_b0(&arr[4]);
- } else if (0xb1 == cmd[2]) { /* Block characteristics (SBC) */
+ arr[3] = inquiry_vpd_b0(&arr[4]);
+ } else if (is_disk && 0xb1 == cmd[2]) { /* Block char. */
arr[1] = cmd[2]; /*sanity */
- arr[3] = inquiry_evpd_b1(&arr[4]);
- } else if (0xb2 == cmd[2]) { /* Logical Block Prov. (SBC) */
+ arr[3] = inquiry_vpd_b1(&arr[4]);
+ } else if (is_disk && 0xb2 == cmd[2]) { /* LB Prov. */
arr[1] = cmd[2]; /*sanity */
- arr[3] = inquiry_evpd_b2(&arr[4]);
+ arr[3] = inquiry_vpd_b2(&arr[4]);
} else {
mk_sense_invalid_fld(scp, SDEB_IN_CDB, 2, -1);
kfree(arr);
return check_condition_result;
}
- len = min(((arr[2] << 8) + arr[3]) + 4, alloc_len);
+ len = min(get_unaligned_be16(arr + 2) + 4, alloc_len);
ret = fill_from_dev_buffer(scp, arr,
min(len, SDEBUG_MAX_INQ_ARR_SZ));
kfree(arr);
return ret;
}
/* drops through here for a standard inquiry */
- arr[1] = scsi_debug_removable ? 0x80 : 0; /* Removable disk */
- arr[2] = scsi_debug_scsi_level;
+ arr[1] = sdebug_removable ? 0x80 : 0; /* Removable disk */
+ arr[2] = sdebug_scsi_level;
arr[3] = 2; /* response_data_format==2 */
arr[4] = SDEBUG_LONG_INQ_SZ - 5;
- arr[5] = scsi_debug_dif ? 1 : 0; /* PROTECT bit */
- if (0 == scsi_debug_vpd_use_hostno)
+ arr[5] = (int)have_dif_prot; /* PROTECT bit */
+ if (sdebug_vpd_use_hostno == 0)
arr[5] = 0x10; /* claim: implicit TGPS */
arr[6] = 0x10; /* claim: MultiP */
/* arr[6] |= 0x40; ... claim: EncServ (enclosure services) */
@@ -1409,21 +1377,26 @@
memcpy(&arr[16], inq_product_id, 16);
memcpy(&arr[32], inq_product_rev, 4);
/* version descriptors (2 bytes each) follow */
- arr[58] = 0x0; arr[59] = 0xa2; /* SAM-5 rev 4 */
- arr[60] = 0x4; arr[61] = 0x68; /* SPC-4 rev 37 */
+ put_unaligned_be16(0xc0, arr + 58); /* SAM-6 no version claimed */
+ put_unaligned_be16(0x5c0, arr + 60); /* SPC-5 no version claimed */
n = 62;
- if (scsi_debug_ptype == 0) {
- arr[n++] = 0x4; arr[n++] = 0xc5; /* SBC-4 rev 36 */
- } else if (scsi_debug_ptype == 1) {
- arr[n++] = 0x5; arr[n++] = 0x25; /* SSC-4 rev 3 */
+ if (is_disk) { /* SBC-4 no version claimed */
+ put_unaligned_be16(0x600, arr + n);
+ n += 2;
+ } else if (sdebug_ptype == TYPE_TAPE) { /* SSC-4 rev 3 */
+ put_unaligned_be16(0x525, arr + n);
+ n += 2;
}
- arr[n++] = 0x20; arr[n++] = 0xe6; /* SPL-3 rev 7 */
+ put_unaligned_be16(0x2100, arr + n); /* SPL-4 no version claimed */
ret = fill_from_dev_buffer(scp, arr,
min(alloc_len, SDEBUG_LONG_INQ_SZ));
kfree(arr);
return ret;
}
+static unsigned char iec_m_pg[] = {0x1c, 0xa, 0x08, 0, 0, 0, 0, 0,
+ 0, 0, 0x0, 0x0};
+
static int resp_requests(struct scsi_cmnd * scp,
struct sdebug_dev_info * devip)
{
@@ -1452,7 +1425,7 @@
}
} else {
memcpy(arr, sbuff, SCSI_SENSE_BUFFERSIZE);
- if (arr[0] >= 0x70 && dsense == scsi_debug_dsense)
+ if (arr[0] >= 0x70 && dsense == sdebug_dsense)
; /* have sense and formats match */
else if (arr[0] <= 0x70) {
if (dsense) {
@@ -1489,24 +1462,25 @@
struct sdebug_dev_info * devip)
{
unsigned char *cmd = scp->cmnd;
- int power_cond, start;
+ int power_cond, stop;
power_cond = (cmd[4] & 0xf0) >> 4;
if (power_cond) {
mk_sense_invalid_fld(scp, SDEB_IN_CDB, 4, 7);
return check_condition_result;
}
- start = cmd[4] & 1;
- if (start == devip->stopped)
- devip->stopped = !start;
+ stop = !(cmd[4] & 1);
+ atomic_xchg(&devip->stopped, stop);
return 0;
}
static sector_t get_sdebug_capacity(void)
{
- if (scsi_debug_virtual_gb > 0)
- return (sector_t)scsi_debug_virtual_gb *
- (1073741824 / scsi_debug_sector_size);
+ static const unsigned int gibibyte = 1073741824;
+
+ if (sdebug_virtual_gb > 0)
+ return (sector_t)sdebug_virtual_gb *
+ (gibibyte / sdebug_sector_size);
else
return sdebug_store_sectors;
}
@@ -1523,18 +1497,10 @@
memset(arr, 0, SDEBUG_READCAP_ARR_SZ);
if (sdebug_capacity < 0xffffffff) {
capac = (unsigned int)sdebug_capacity - 1;
- arr[0] = (capac >> 24);
- arr[1] = (capac >> 16) & 0xff;
- arr[2] = (capac >> 8) & 0xff;
- arr[3] = capac & 0xff;
- } else {
- arr[0] = 0xff;
- arr[1] = 0xff;
- arr[2] = 0xff;
- arr[3] = 0xff;
- }
- arr[6] = (scsi_debug_sector_size >> 8) & 0xff;
- arr[7] = scsi_debug_sector_size & 0xff;
+ put_unaligned_be32(capac, arr + 0);
+ } else
+ put_unaligned_be32(0xffffffff, arr + 0);
+ put_unaligned_be16(sdebug_sector_size, arr + 6);
return fill_from_dev_buffer(scp, arr, SDEBUG_READCAP_ARR_SZ);
}
@@ -1544,34 +1510,31 @@
{
unsigned char *cmd = scp->cmnd;
unsigned char arr[SDEBUG_READCAP16_ARR_SZ];
- unsigned long long capac;
- int k, alloc_len;
+ int alloc_len;
- alloc_len = ((cmd[10] << 24) + (cmd[11] << 16) + (cmd[12] << 8)
- + cmd[13]);
+ alloc_len = get_unaligned_be32(cmd + 10);
/* following just in case virtual_gb changed */
sdebug_capacity = get_sdebug_capacity();
memset(arr, 0, SDEBUG_READCAP16_ARR_SZ);
- capac = sdebug_capacity - 1;
- for (k = 0; k < 8; ++k, capac >>= 8)
- arr[7 - k] = capac & 0xff;
- arr[8] = (scsi_debug_sector_size >> 24) & 0xff;
- arr[9] = (scsi_debug_sector_size >> 16) & 0xff;
- arr[10] = (scsi_debug_sector_size >> 8) & 0xff;
- arr[11] = scsi_debug_sector_size & 0xff;
- arr[13] = scsi_debug_physblk_exp & 0xf;
- arr[14] = (scsi_debug_lowest_aligned >> 8) & 0x3f;
+ put_unaligned_be64((u64)(sdebug_capacity - 1), arr + 0);
+ put_unaligned_be32(sdebug_sector_size, arr + 8);
+ arr[13] = sdebug_physblk_exp & 0xf;
+ arr[14] = (sdebug_lowest_aligned >> 8) & 0x3f;
if (scsi_debug_lbp()) {
arr[14] |= 0x80; /* LBPME */
- if (scsi_debug_lbprz)
- arr[14] |= 0x40; /* LBPRZ */
+ /* from sbc4r07, this LBPRZ field is 1 bit, but the LBPRZ in
+ * the LB Provisioning VPD page is 3 bits. Note that lbprz=2
+ * in the wider field maps to 0 in this field.
+ */
+ if (sdebug_lbprz & 1) /* precisely what the draft requires */
+ arr[14] |= 0x40;
}
- arr[15] = scsi_debug_lowest_aligned & 0xff;
+ arr[15] = sdebug_lowest_aligned & 0xff;
- if (scsi_debug_dif) {
- arr[12] = (scsi_debug_dif - 1) << 1; /* P_TYPE */
+ if (have_dif_prot) {
+ arr[12] = (sdebug_dif - 1) << 1; /* P_TYPE */
arr[12] |= 1; /* PROT_EN */
}
@@ -1590,9 +1553,7 @@
int n, ret, alen, rlen;
int port_group_a, port_group_b, port_a, port_b;
- alen = ((cmd[6] << 24) + (cmd[7] << 16) + (cmd[8] << 8)
- + cmd[9]);
-
+ alen = get_unaligned_be32(cmd + 6);
arr = kzalloc(SDEBUG_MAX_TGTPGS_ARR_SZ, GFP_ATOMIC);
if (! arr)
return DID_REQUEUE << 16;
@@ -1605,49 +1566,46 @@
port_a = 0x1; /* relative port A */
port_b = 0x2; /* relative port B */
port_group_a = (((host_no + 1) & 0x7f) << 8) +
- (devip->channel & 0x7f);
+ (devip->channel & 0x7f);
port_group_b = (((host_no + 1) & 0x7f) << 8) +
- (devip->channel & 0x7f) + 0x80;
+ (devip->channel & 0x7f) + 0x80;
/*
* The asymmetric access state is cycled according to the host_id.
*/
n = 4;
- if (0 == scsi_debug_vpd_use_hostno) {
- arr[n++] = host_no % 3; /* Asymm access state */
- arr[n++] = 0x0F; /* claim: all states are supported */
+ if (sdebug_vpd_use_hostno == 0) {
+ arr[n++] = host_no % 3; /* Asymm access state */
+ arr[n++] = 0x0F; /* claim: all states are supported */
} else {
- arr[n++] = 0x0; /* Active/Optimized path */
- arr[n++] = 0x01; /* claim: only support active/optimized paths */
+ arr[n++] = 0x0; /* Active/Optimized path */
+ arr[n++] = 0x01; /* only support active/optimized paths */
}
- arr[n++] = (port_group_a >> 8) & 0xff;
- arr[n++] = port_group_a & 0xff;
+ put_unaligned_be16(port_group_a, arr + n);
+ n += 2;
arr[n++] = 0; /* Reserved */
arr[n++] = 0; /* Status code */
arr[n++] = 0; /* Vendor unique */
arr[n++] = 0x1; /* One port per group */
arr[n++] = 0; /* Reserved */
arr[n++] = 0; /* Reserved */
- arr[n++] = (port_a >> 8) & 0xff;
- arr[n++] = port_a & 0xff;
+ put_unaligned_be16(port_a, arr + n);
+ n += 2;
arr[n++] = 3; /* Port unavailable */
arr[n++] = 0x08; /* claim: only unavailalbe paths are supported */
- arr[n++] = (port_group_b >> 8) & 0xff;
- arr[n++] = port_group_b & 0xff;
+ put_unaligned_be16(port_group_b, arr + n);
+ n += 2;
arr[n++] = 0; /* Reserved */
arr[n++] = 0; /* Status code */
arr[n++] = 0; /* Vendor unique */
arr[n++] = 0x1; /* One port per group */
arr[n++] = 0; /* Reserved */
arr[n++] = 0; /* Reserved */
- arr[n++] = (port_b >> 8) & 0xff;
- arr[n++] = port_b & 0xff;
+ put_unaligned_be16(port_b, arr + n);
+ n += 2;
rlen = n - 4;
- arr[0] = (rlen >> 24) & 0xff;
- arr[1] = (rlen >> 16) & 0xff;
- arr[2] = (rlen >> 8) & 0xff;
- arr[3] = rlen & 0xff;
+ put_unaligned_be32(rlen, arr + 0);
/*
* Return the smallest value of either
@@ -1662,8 +1620,8 @@
return ret;
}
-static int
-resp_rsup_opcodes(struct scsi_cmnd *scp, struct sdebug_dev_info *devip)
+static int resp_rsup_opcodes(struct scsi_cmnd *scp,
+ struct sdebug_dev_info *devip)
{
bool rctd;
u8 reporting_opts, req_opcode, sdeb_i, supp;
@@ -1813,8 +1771,8 @@
return errsts;
}
-static int
-resp_rsup_tmfs(struct scsi_cmnd *scp, struct sdebug_dev_info *devip)
+static int resp_rsup_tmfs(struct scsi_cmnd *scp,
+ struct sdebug_dev_info *devip)
{
bool repd;
u32 alloc_len, len;
@@ -1871,17 +1829,19 @@
0, 0, 0, 0, 0x40, 0, 0, 0};
memcpy(p, format_pg, sizeof(format_pg));
- p[10] = (sdebug_sectors_per >> 8) & 0xff;
- p[11] = sdebug_sectors_per & 0xff;
- p[12] = (scsi_debug_sector_size >> 8) & 0xff;
- p[13] = scsi_debug_sector_size & 0xff;
- if (scsi_debug_removable)
+ put_unaligned_be16(sdebug_sectors_per, p + 10);
+ put_unaligned_be16(sdebug_sector_size, p + 12);
+ if (sdebug_removable)
p[20] |= 0x20; /* should agree with INQUIRY */
if (1 == pcontrol)
memset(p + 2, 0, sizeof(format_pg) - 2);
return sizeof(format_pg);
}
+static unsigned char caching_pg[] = {0x8, 18, 0x14, 0, 0xff, 0xff, 0, 0,
+ 0xff, 0xff, 0xff, 0xff, 0x80, 0x14, 0, 0,
+ 0, 0, 0, 0};
+
static int resp_caching_pg(unsigned char * p, int pcontrol, int target)
{ /* Caching page for mode_sense */
unsigned char ch_caching_pg[] = {/* 0x8, 18, */ 0x4, 0, 0, 0, 0, 0,
@@ -1889,7 +1849,7 @@
unsigned char d_caching_pg[] = {0x8, 18, 0x14, 0, 0xff, 0xff, 0, 0,
0xff, 0xff, 0xff, 0xff, 0x80, 0x14, 0, 0, 0, 0, 0, 0};
- if (SCSI_DEBUG_OPT_N_WCE & scsi_debug_opts)
+ if (SDEBUG_OPT_N_WCE & sdebug_opts)
caching_pg[2] &= ~0x4; /* set WCE=0 (default WCE=1) */
memcpy(p, caching_pg, sizeof(caching_pg));
if (1 == pcontrol)
@@ -1899,6 +1859,9 @@
return sizeof(caching_pg);
}
+static unsigned char ctrl_m_pg[] = {0xa, 10, 2, 0, 0, 0, 0, 0,
+ 0, 0, 0x2, 0x4b};
+
static int resp_ctrl_m_pg(unsigned char * p, int pcontrol, int target)
{ /* Control mode page for mode_sense */
unsigned char ch_ctrl_m_pg[] = {/* 0xa, 10, */ 0x6, 0, 0, 0, 0, 0,
@@ -1906,12 +1869,12 @@
unsigned char d_ctrl_m_pg[] = {0xa, 10, 2, 0, 0, 0, 0, 0,
0, 0, 0x2, 0x4b};
- if (scsi_debug_dsense)
+ if (sdebug_dsense)
ctrl_m_pg[2] |= 0x4;
else
ctrl_m_pg[2] &= ~0x4;
- if (scsi_debug_ato)
+ if (sdebug_ato)
ctrl_m_pg[5] |= 0x80; /* ATO=1 */
memcpy(p, ctrl_m_pg, sizeof(ctrl_m_pg));
@@ -1955,31 +1918,29 @@
{ /* SAS phy control and discover mode page for mode_sense */
unsigned char sas_pcd_m_pg[] = {0x59, 0x1, 0, 0x64, 0, 0x6, 0, 2,
0, 0, 0, 0, 0x10, 0x9, 0x8, 0x0,
- 0x52, 0x22, 0x22, 0x20, 0x0, 0x0, 0x0, 0x0,
- 0x51, 0x11, 0x11, 0x10, 0x0, 0x0, 0x0, 0x1,
+ 0, 0, 0, 0, 0, 0, 0, 0, /* insert SAS addr */
+ 0, 0, 0, 0, 0, 0, 0, 0, /* insert SAS addr */
0x2, 0, 0, 0, 0, 0, 0, 0,
0x88, 0x99, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0,
0, 1, 0, 0, 0x10, 0x9, 0x8, 0x0,
- 0x52, 0x22, 0x22, 0x20, 0x0, 0x0, 0x0, 0x0,
- 0x51, 0x11, 0x11, 0x10, 0x0, 0x0, 0x0, 0x1,
+ 0, 0, 0, 0, 0, 0, 0, 0, /* insert SAS addr */
+ 0, 0, 0, 0, 0, 0, 0, 0, /* insert SAS addr */
0x3, 0, 0, 0, 0, 0, 0, 0,
0x88, 0x99, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0,
};
int port_a, port_b;
+ put_unaligned_be64(naa3_comp_a, sas_pcd_m_pg + 16);
+ put_unaligned_be64(naa3_comp_c + 1, sas_pcd_m_pg + 24);
+ put_unaligned_be64(naa3_comp_a, sas_pcd_m_pg + 64);
+ put_unaligned_be64(naa3_comp_c + 1, sas_pcd_m_pg + 72);
port_a = target_dev_id + 1;
port_b = port_a + 1;
memcpy(p, sas_pcd_m_pg, sizeof(sas_pcd_m_pg));
- p[20] = (port_a >> 24);
- p[21] = (port_a >> 16) & 0xff;
- p[22] = (port_a >> 8) & 0xff;
- p[23] = port_a & 0xff;
- p[48 + 20] = (port_b >> 24);
- p[48 + 21] = (port_b >> 16) & 0xff;
- p[48 + 22] = (port_b >> 8) & 0xff;
- p[48 + 23] = port_b & 0xff;
+ put_unaligned_be32(port_a, p + 20);
+ put_unaligned_be32(port_b, p + 48 + 20);
if (1 == pcontrol)
memset(p + 4, 0, sizeof(sas_pcd_m_pg) - 4);
return sizeof(sas_pcd_m_pg);
@@ -1999,29 +1960,30 @@
#define SDEBUG_MAX_MSENSE_SZ 256
-static int
-resp_mode_sense(struct scsi_cmnd *scp, struct sdebug_dev_info *devip)
+static int resp_mode_sense(struct scsi_cmnd *scp,
+ struct sdebug_dev_info *devip)
{
- unsigned char dbd, llbaa;
int pcontrol, pcode, subpcode, bd_len;
unsigned char dev_spec;
- int k, alloc_len, msense_6, offset, len, target_dev_id;
+ int alloc_len, offset, len, target_dev_id;
int target = scp->device->id;
unsigned char * ap;
unsigned char arr[SDEBUG_MAX_MSENSE_SZ];
unsigned char *cmd = scp->cmnd;
+ bool dbd, llbaa, msense_6, is_disk, bad_pcode;
- dbd = !!(cmd[1] & 0x8);
+ dbd = !!(cmd[1] & 0x8); /* disable block descriptors */
pcontrol = (cmd[2] & 0xc0) >> 6;
pcode = cmd[2] & 0x3f;
subpcode = cmd[3];
msense_6 = (MODE_SENSE == cmd[0]);
- llbaa = msense_6 ? 0 : !!(cmd[1] & 0x10);
- if ((0 == scsi_debug_ptype) && (0 == dbd))
+ llbaa = msense_6 ? false : !!(cmd[1] & 0x10);
+ is_disk = (sdebug_ptype == TYPE_DISK);
+ if (is_disk && !dbd)
bd_len = llbaa ? 16 : 8;
else
bd_len = 0;
- alloc_len = msense_6 ? cmd[4] : ((cmd[7] << 8) | cmd[8]);
+ alloc_len = msense_6 ? cmd[4] : get_unaligned_be16(cmd + 7);
memset(arr, 0, SDEBUG_MAX_MSENSE_SZ);
if (0x3 == pcontrol) { /* Saving values not supported */
mk_sense_buffer(scp, ILLEGAL_REQUEST, SAVING_PARAMS_UNSUP, 0);
@@ -2029,9 +1991,9 @@
}
target_dev_id = ((devip->sdbg_host->shost->host_no + 1) * 2000) +
(devip->target * 1000) - 3;
- /* set DPOFUA bit for disks */
- if (0 == scsi_debug_ptype)
- dev_spec = (DEV_READONLY(target) ? 0x80 : 0x0) | 0x10;
+ /* for disks set DPOFUA bit and clear write protect (WP) bit */
+ if (is_disk)
+ dev_spec = 0x10; /* =0x90 if WP=1 implies read-only */
else
dev_spec = 0x0;
if (msense_6) {
@@ -2050,30 +2012,16 @@
sdebug_capacity = get_sdebug_capacity();
if (8 == bd_len) {
- if (sdebug_capacity > 0xfffffffe) {
- ap[0] = 0xff;
- ap[1] = 0xff;
- ap[2] = 0xff;
- ap[3] = 0xff;
- } else {
- ap[0] = (sdebug_capacity >> 24) & 0xff;
- ap[1] = (sdebug_capacity >> 16) & 0xff;
- ap[2] = (sdebug_capacity >> 8) & 0xff;
- ap[3] = sdebug_capacity & 0xff;
- }
- ap[6] = (scsi_debug_sector_size >> 8) & 0xff;
- ap[7] = scsi_debug_sector_size & 0xff;
+ if (sdebug_capacity > 0xfffffffe)
+ put_unaligned_be32(0xffffffff, ap + 0);
+ else
+ put_unaligned_be32(sdebug_capacity, ap + 0);
+ put_unaligned_be16(sdebug_sector_size, ap + 6);
offset += bd_len;
ap = arr + offset;
} else if (16 == bd_len) {
- unsigned long long capac = sdebug_capacity;
-
- for (k = 0; k < 8; ++k, capac >>= 8)
- ap[7 - k] = capac & 0xff;
- ap[12] = (scsi_debug_sector_size >> 24) & 0xff;
- ap[13] = (scsi_debug_sector_size >> 16) & 0xff;
- ap[14] = (scsi_debug_sector_size >> 8) & 0xff;
- ap[15] = scsi_debug_sector_size & 0xff;
+ put_unaligned_be64((u64)sdebug_capacity, ap + 0);
+ put_unaligned_be32(sdebug_sector_size, ap + 12);
offset += bd_len;
ap = arr + offset;
}
@@ -2083,6 +2031,8 @@
mk_sense_invalid_fld(scp, SDEB_IN_CDB, 3, -1);
return check_condition_result;
}
+ bad_pcode = false;
+
switch (pcode) {
case 0x1: /* Read-Write error recovery page, direct access */
len = resp_err_recov_pg(ap, pcontrol, target);
@@ -2093,12 +2043,18 @@
offset += len;
break;
case 0x3: /* Format device page, direct access */
- len = resp_format_pg(ap, pcontrol, target);
- offset += len;
+ if (is_disk) {
+ len = resp_format_pg(ap, pcontrol, target);
+ offset += len;
+ } else
+ bad_pcode = true;
break;
case 0x8: /* Caching page, direct access */
- len = resp_caching_pg(ap, pcontrol, target);
- offset += len;
+ if (is_disk) {
+ len = resp_caching_pg(ap, pcontrol, target);
+ offset += len;
+ } else
+ bad_pcode = true;
break;
case 0xa: /* Control Mode page, all devices */
len = resp_ctrl_m_pg(ap, pcontrol, target);
@@ -2127,8 +2083,12 @@
if ((0 == subpcode) || (0xff == subpcode)) {
len = resp_err_recov_pg(ap, pcontrol, target);
len += resp_disconnect_pg(ap + len, pcontrol, target);
- len += resp_format_pg(ap + len, pcontrol, target);
- len += resp_caching_pg(ap + len, pcontrol, target);
+ if (is_disk) {
+ len += resp_format_pg(ap + len, pcontrol,
+ target);
+ len += resp_caching_pg(ap + len, pcontrol,
+ target);
+ }
len += resp_ctrl_m_pg(ap + len, pcontrol, target);
len += resp_sas_sf_m_pg(ap + len, pcontrol, target);
if (0xff == subpcode) {
@@ -2137,29 +2097,31 @@
len += resp_sas_sha_m_spg(ap + len, pcontrol);
}
len += resp_iec_m_pg(ap + len, pcontrol, target);
+ offset += len;
} else {
mk_sense_invalid_fld(scp, SDEB_IN_CDB, 3, -1);
return check_condition_result;
}
- offset += len;
break;
default:
+ bad_pcode = true;
+ break;
+ }
+ if (bad_pcode) {
mk_sense_invalid_fld(scp, SDEB_IN_CDB, 2, 5);
return check_condition_result;
}
if (msense_6)
arr[0] = offset - 1;
- else {
- arr[0] = ((offset - 2) >> 8) & 0xff;
- arr[1] = (offset - 2) & 0xff;
- }
+ else
+ put_unaligned_be16((offset - 2), arr + 0);
return fill_from_dev_buffer(scp, arr, min(alloc_len, offset));
}
#define SDEBUG_MAX_MSELECT_SZ 512
-static int
-resp_mode_select(struct scsi_cmnd *scp, struct sdebug_dev_info *devip)
+static int resp_mode_select(struct scsi_cmnd *scp,
+ struct sdebug_dev_info *devip)
{
int pf, sp, ps, md_len, bd_len, off, spf, pg_len;
int param_len, res, mpage;
@@ -2170,21 +2132,20 @@
memset(arr, 0, sizeof(arr));
pf = cmd[1] & 0x10;
sp = cmd[1] & 0x1;
- param_len = mselect6 ? cmd[4] : ((cmd[7] << 8) + cmd[8]);
+ param_len = mselect6 ? cmd[4] : get_unaligned_be16(cmd + 7);
if ((0 == pf) || sp || (param_len > SDEBUG_MAX_MSELECT_SZ)) {
mk_sense_invalid_fld(scp, SDEB_IN_CDB, mselect6 ? 4 : 7, -1);
return check_condition_result;
}
res = fetch_to_dev_buffer(scp, arr, param_len);
if (-1 == res)
- return (DID_ERROR << 16);
- else if ((res < param_len) &&
- (SCSI_DEBUG_OPT_NOISE & scsi_debug_opts))
+ return DID_ERROR << 16;
+ else if (sdebug_verbose && (res < param_len))
sdev_printk(KERN_INFO, scp->device,
"%s: cdb indicated=%d, IO sent=%d bytes\n",
__func__, param_len, res);
- md_len = mselect6 ? (arr[0] + 1) : ((arr[0] << 8) + arr[1] + 2);
- bd_len = mselect6 ? arr[3] : ((arr[6] << 8) + arr[7]);
+ md_len = mselect6 ? (arr[0] + 1) : (get_unaligned_be16(arr + 0) + 2);
+ bd_len = mselect6 ? arr[3] : get_unaligned_be16(arr + 6);
if (md_len > 2) {
mk_sense_invalid_fld(scp, SDEB_IN_DATA, 0, -1);
return check_condition_result;
@@ -2197,7 +2158,7 @@
return check_condition_result;
}
spf = !!(arr[off] & 0x40);
- pg_len = spf ? ((arr[off + 2] << 8) + arr[off + 3] + 4) :
+ pg_len = spf ? (get_unaligned_be16(arr + off + 2) + 4) :
(arr[off + 1] + 2);
if ((pg_len + off) > param_len) {
mk_sense_buffer(scp, ILLEGAL_REQUEST,
@@ -2216,7 +2177,7 @@
if (ctrl_m_pg[1] == arr[off + 1]) {
memcpy(ctrl_m_pg + 2, arr + off + 2,
sizeof(ctrl_m_pg) - 2);
- scsi_debug_dsense = !!(ctrl_m_pg[2] & 0x4);
+ sdebug_dsense = !!(ctrl_m_pg[2] & 0x4);
goto set_mode_changed_ua;
}
break;
@@ -2279,7 +2240,7 @@
pcontrol = (cmd[2] & 0xc0) >> 6;
pcode = cmd[2] & 0x3f;
subpcode = cmd[3] & 0xff;
- alloc_len = (cmd[7] << 8) + cmd[8];
+ alloc_len = get_unaligned_be16(cmd + 7);
arr[0] = pcode;
if (0 == subpcode) {
switch (pcode) {
@@ -2336,7 +2297,7 @@
mk_sense_invalid_fld(scp, SDEB_IN_CDB, 3, -1);
return check_condition_result;
}
- len = min(((arr[2] << 8) + arr[3]) + 4, alloc_len);
+ len = min(get_unaligned_be16(arr + 2) + 4, alloc_len);
return fill_from_dev_buffer(scp, arr,
min(len, SDEBUG_MAX_INQ_ARR_SZ));
}
@@ -2358,8 +2319,8 @@
}
/* Returns number of bytes copied or -1 if error. */
-static int
-do_device_access(struct scsi_cmnd *scmd, u64 lba, u32 num, bool do_write)
+static int do_device_access(struct scsi_cmnd *scmd, u64 lba, u32 num,
+ bool do_write)
{
int ret;
u64 block, rest = 0;
@@ -2384,15 +2345,15 @@
rest = block + num - sdebug_store_sectors;
ret = sg_copy_buffer(sdb->table.sgl, sdb->table.nents,
- fake_storep + (block * scsi_debug_sector_size),
- (num - rest) * scsi_debug_sector_size, 0, do_write);
- if (ret != (num - rest) * scsi_debug_sector_size)
+ fake_storep + (block * sdebug_sector_size),
+ (num - rest) * sdebug_sector_size, 0, do_write);
+ if (ret != (num - rest) * sdebug_sector_size)
return ret;
if (rest) {
ret += sg_copy_buffer(sdb->table.sgl, sdb->table.nents,
- fake_storep, rest * scsi_debug_sector_size,
- (num - rest) * scsi_debug_sector_size, do_write);
+ fake_storep, rest * sdebug_sector_size,
+ (num - rest) * sdebug_sector_size, do_write);
}
return ret;
@@ -2401,13 +2362,12 @@
/* If fake_store(lba,num) compares equal to arr(num), then copy top half of
* arr into fake_store(lba,num) and return true. If comparison fails then
* return false. */
-static bool
-comp_write_worker(u64 lba, u32 num, const u8 *arr)
+static bool comp_write_worker(u64 lba, u32 num, const u8 *arr)
{
bool res;
u64 block, rest = 0;
u32 store_blks = sdebug_store_sectors;
- u32 lb_size = scsi_debug_sector_size;
+ u32 lb_size = sdebug_sector_size;
block = do_div(lba, store_blks);
if (block + num > store_blks)
@@ -2434,7 +2394,7 @@
{
__be16 csum;
- if (scsi_debug_guard)
+ if (sdebug_guard)
csum = (__force __be16)ip_compute_csum(buf, len);
else
csum = cpu_to_be16(crc_t10dif(buf, len));
@@ -2445,7 +2405,7 @@
static int dif_verify(struct sd_dif_tuple *sdt, const void *data,
sector_t sector, u32 ei_lba)
{
- __be16 csum = dif_compute_csum(data, scsi_debug_sector_size);
+ __be16 csum = dif_compute_csum(data, sdebug_sector_size);
if (sdt->guard_tag != csum) {
pr_err("GUARD check failed on sector %lu rcvd 0x%04x, data 0x%04x\n",
@@ -2454,13 +2414,13 @@
be16_to_cpu(csum));
return 0x01;
}
- if (scsi_debug_dif == SD_DIF_TYPE1_PROTECTION &&
+ if (sdebug_dif == SD_DIF_TYPE1_PROTECTION &&
be32_to_cpu(sdt->ref_tag) != (sector & 0xffffffff)) {
pr_err("REF check failed on sector %lu\n",
(unsigned long)sector);
return 0x03;
}
- if (scsi_debug_dif == SD_DIF_TYPE2_PROTECTION &&
+ if (sdebug_dif == SD_DIF_TYPE2_PROTECTION &&
be32_to_cpu(sdt->ref_tag) != ei_lba) {
pr_err("REF check failed on sector %lu\n",
(unsigned long)sector);
@@ -2541,10 +2501,10 @@
return 0;
}
-static int
-resp_read_dt0(struct scsi_cmnd *scp, struct sdebug_dev_info *devip)
+static int resp_read_dt0(struct scsi_cmnd *scp, struct sdebug_dev_info *devip)
{
u8 *cmd = scp->cmnd;
+ struct sdebug_queued_cmd *sqcp;
u64 lba;
u32 num;
u32 ei_lba;
@@ -2591,40 +2551,43 @@
check_prot = false;
break;
}
- if (check_prot) {
- if (scsi_debug_dif == SD_DIF_TYPE2_PROTECTION &&
+ if (unlikely(have_dif_prot && check_prot)) {
+ if (sdebug_dif == SD_DIF_TYPE2_PROTECTION &&
(cmd[1] & 0xe0)) {
mk_sense_invalid_opcode(scp);
return check_condition_result;
}
- if ((scsi_debug_dif == SD_DIF_TYPE1_PROTECTION ||
- scsi_debug_dif == SD_DIF_TYPE3_PROTECTION) &&
+ if ((sdebug_dif == SD_DIF_TYPE1_PROTECTION ||
+ sdebug_dif == SD_DIF_TYPE3_PROTECTION) &&
(cmd[1] & 0xe0) == 0)
sdev_printk(KERN_ERR, scp->device, "Unprotected RD "
"to DIF device\n");
}
- if (sdebug_any_injecting_opt) {
- struct sdebug_scmd_extra_t *ep = scsi_cmd_priv(scp);
+ if (unlikely(sdebug_any_injecting_opt)) {
+ sqcp = (struct sdebug_queued_cmd *)scp->host_scribble;
- if (ep->inj_short)
- num /= 2;
- }
+ if (sqcp) {
+ if (sqcp->inj_short)
+ num /= 2;
+ }
+ } else
+ sqcp = NULL;
/* inline check_device_access_params() */
- if (lba + num > sdebug_capacity) {
+ if (unlikely(lba + num > sdebug_capacity)) {
mk_sense_buffer(scp, ILLEGAL_REQUEST, LBA_OUT_OF_RANGE, 0);
return check_condition_result;
}
/* transfer length excessive (tie in to block limits VPD page) */
- if (num > sdebug_store_sectors) {
+ if (unlikely(num > sdebug_store_sectors)) {
/* needs work to find which cdb byte 'num' comes from */
mk_sense_buffer(scp, ILLEGAL_REQUEST, INVALID_FIELD_IN_CDB, 0);
return check_condition_result;
}
- if ((SCSI_DEBUG_OPT_MEDIUM_ERR & scsi_debug_opts) &&
- (lba <= (OPT_MEDIUM_ERR_ADDR + OPT_MEDIUM_ERR_NUM - 1)) &&
- ((lba + num) > OPT_MEDIUM_ERR_ADDR)) {
+ if (unlikely((SDEBUG_OPT_MEDIUM_ERR & sdebug_opts) &&
+ (lba <= (OPT_MEDIUM_ERR_ADDR + OPT_MEDIUM_ERR_NUM - 1)) &&
+ ((lba + num) > OPT_MEDIUM_ERR_ADDR))) {
/* claim unrecoverable read error */
mk_sense_buffer(scp, MEDIUM_ERROR, UNRECOVERED_READ_ERR, 0);
/* set info field and valid bit for fixed descriptor */
@@ -2641,7 +2604,7 @@
read_lock_irqsave(&atomic_rw, iflags);
/* DIX + T10 DIF */
- if (scsi_debug_dix && scsi_prot_sg_count(scp)) {
+ if (unlikely(sdebug_dix && scsi_prot_sg_count(scp))) {
int prot_ret = prot_verify_read(scp, lba, num, ei_lba);
if (prot_ret) {
@@ -2653,27 +2616,25 @@
ret = do_device_access(scp, lba, num, false);
read_unlock_irqrestore(&atomic_rw, iflags);
- if (ret == -1)
+ if (unlikely(ret == -1))
return DID_ERROR << 16;
scsi_in(scp)->resid = scsi_bufflen(scp) - ret;
- if (sdebug_any_injecting_opt) {
- struct sdebug_scmd_extra_t *ep = scsi_cmd_priv(scp);
-
- if (ep->inj_recovered) {
+ if (unlikely(sqcp)) {
+ if (sqcp->inj_recovered) {
mk_sense_buffer(scp, RECOVERED_ERROR,
THRESHOLD_EXCEEDED, 0);
return check_condition_result;
- } else if (ep->inj_transport) {
+ } else if (sqcp->inj_transport) {
mk_sense_buffer(scp, ABORTED_COMMAND,
TRANSPORT_PROBLEM, ACK_NAK_TO);
return check_condition_result;
- } else if (ep->inj_dif) {
+ } else if (sqcp->inj_dif) {
/* Logical block guard check failed */
mk_sense_buffer(scp, ABORTED_COMMAND, 0x10, 1);
return illegal_condition_result;
- } else if (ep->inj_dix) {
+ } else if (sqcp->inj_dix) {
mk_sense_buffer(scp, ILLEGAL_REQUEST, 0x10, 1);
return illegal_condition_result;
}
@@ -2750,13 +2711,13 @@
ret = dif_verify(sdt, daddr, sector, ei_lba);
if (ret) {
- dump_sector(daddr, scsi_debug_sector_size);
+ dump_sector(daddr, sdebug_sector_size);
goto out;
}
sector++;
ei_lba++;
- dpage_offset += scsi_debug_sector_size;
+ dpage_offset += sdebug_sector_size;
}
diter.consumed = dpage_offset;
sg_miter_stop(&diter);
@@ -2777,24 +2738,18 @@
static unsigned long lba_to_map_index(sector_t lba)
{
- if (scsi_debug_unmap_alignment) {
- lba += scsi_debug_unmap_granularity -
- scsi_debug_unmap_alignment;
- }
- sector_div(lba, scsi_debug_unmap_granularity);
-
+ if (sdebug_unmap_alignment)
+ lba += sdebug_unmap_granularity - sdebug_unmap_alignment;
+ sector_div(lba, sdebug_unmap_granularity);
return lba;
}
static sector_t map_index_to_lba(unsigned long index)
{
- sector_t lba = index * scsi_debug_unmap_granularity;
+ sector_t lba = index * sdebug_unmap_granularity;
- if (scsi_debug_unmap_alignment) {
- lba -= scsi_debug_unmap_granularity -
- scsi_debug_unmap_alignment;
- }
-
+ if (sdebug_unmap_alignment)
+ lba -= sdebug_unmap_granularity - sdebug_unmap_alignment;
return lba;
}
@@ -2815,7 +2770,6 @@
end = min_t(sector_t, sdebug_store_sectors, map_index_to_lba(next));
*num = end - lba;
-
return mapped;
}
@@ -2841,27 +2795,27 @@
unsigned long index = lba_to_map_index(lba);
if (lba == map_index_to_lba(index) &&
- lba + scsi_debug_unmap_granularity <= end &&
+ lba + sdebug_unmap_granularity <= end &&
index < map_size) {
clear_bit(index, map_storep);
- if (scsi_debug_lbprz) {
+ if (sdebug_lbprz) { /* for LBPRZ=2 return 0xff_s */
memset(fake_storep +
- lba * scsi_debug_sector_size, 0,
- scsi_debug_sector_size *
- scsi_debug_unmap_granularity);
+ lba * sdebug_sector_size,
+ (sdebug_lbprz & 1) ? 0 : 0xff,
+ sdebug_sector_size *
+ sdebug_unmap_granularity);
}
if (dif_storep) {
memset(dif_storep + lba, 0xff,
sizeof(*dif_storep) *
- scsi_debug_unmap_granularity);
+ sdebug_unmap_granularity);
}
}
lba = map_index_to_lba(index + 1);
}
}
-static int
-resp_write_dt0(struct scsi_cmnd *scp, struct sdebug_dev_info *devip)
+static int resp_write_dt0(struct scsi_cmnd *scp, struct sdebug_dev_info *devip)
{
u8 *cmd = scp->cmnd;
u64 lba;
@@ -2910,26 +2864,26 @@
check_prot = false;
break;
}
- if (check_prot) {
- if (scsi_debug_dif == SD_DIF_TYPE2_PROTECTION &&
+ if (unlikely(have_dif_prot && check_prot)) {
+ if (sdebug_dif == SD_DIF_TYPE2_PROTECTION &&
(cmd[1] & 0xe0)) {
mk_sense_invalid_opcode(scp);
return check_condition_result;
}
- if ((scsi_debug_dif == SD_DIF_TYPE1_PROTECTION ||
- scsi_debug_dif == SD_DIF_TYPE3_PROTECTION) &&
+ if ((sdebug_dif == SD_DIF_TYPE1_PROTECTION ||
+ sdebug_dif == SD_DIF_TYPE3_PROTECTION) &&
(cmd[1] & 0xe0) == 0)
sdev_printk(KERN_ERR, scp->device, "Unprotected WR "
"to DIF device\n");
}
/* inline check_device_access_params() */
- if (lba + num > sdebug_capacity) {
+ if (unlikely(lba + num > sdebug_capacity)) {
mk_sense_buffer(scp, ILLEGAL_REQUEST, LBA_OUT_OF_RANGE, 0);
return check_condition_result;
}
/* transfer length excessive (tie in to block limits VPD page) */
- if (num > sdebug_store_sectors) {
+ if (unlikely(num > sdebug_store_sectors)) {
/* needs work to find which cdb byte 'num' comes from */
mk_sense_buffer(scp, ILLEGAL_REQUEST, INVALID_FIELD_IN_CDB, 0);
return check_condition_result;
@@ -2938,7 +2892,7 @@
write_lock_irqsave(&atomic_rw, iflags);
/* DIX + T10 DIF */
- if (scsi_debug_dix && scsi_prot_sg_count(scp)) {
+ if (unlikely(sdebug_dix && scsi_prot_sg_count(scp))) {
int prot_ret = prot_verify_write(scp, lba, num, ei_lba);
if (prot_ret) {
@@ -2949,43 +2903,46 @@
}
ret = do_device_access(scp, lba, num, true);
- if (scsi_debug_lbp())
+ if (unlikely(scsi_debug_lbp()))
map_region(lba, num);
write_unlock_irqrestore(&atomic_rw, iflags);
- if (-1 == ret)
- return (DID_ERROR << 16);
- else if ((ret < (num * scsi_debug_sector_size)) &&
- (SCSI_DEBUG_OPT_NOISE & scsi_debug_opts))
+ if (unlikely(-1 == ret))
+ return DID_ERROR << 16;
+ else if (unlikely(sdebug_verbose &&
+ (ret < (num * sdebug_sector_size))))
sdev_printk(KERN_INFO, scp->device,
"%s: write: cdb indicated=%u, IO sent=%d bytes\n",
- my_name, num * scsi_debug_sector_size, ret);
+ my_name, num * sdebug_sector_size, ret);
- if (sdebug_any_injecting_opt) {
- struct sdebug_scmd_extra_t *ep = scsi_cmd_priv(scp);
+ if (unlikely(sdebug_any_injecting_opt)) {
+ struct sdebug_queued_cmd *sqcp =
+ (struct sdebug_queued_cmd *)scp->host_scribble;
- if (ep->inj_recovered) {
- mk_sense_buffer(scp, RECOVERED_ERROR,
- THRESHOLD_EXCEEDED, 0);
- return check_condition_result;
- } else if (ep->inj_dif) {
- /* Logical block guard check failed */
- mk_sense_buffer(scp, ABORTED_COMMAND, 0x10, 1);
- return illegal_condition_result;
- } else if (ep->inj_dix) {
- mk_sense_buffer(scp, ILLEGAL_REQUEST, 0x10, 1);
- return illegal_condition_result;
+ if (sqcp) {
+ if (sqcp->inj_recovered) {
+ mk_sense_buffer(scp, RECOVERED_ERROR,
+ THRESHOLD_EXCEEDED, 0);
+ return check_condition_result;
+ } else if (sqcp->inj_dif) {
+ /* Logical block guard check failed */
+ mk_sense_buffer(scp, ABORTED_COMMAND, 0x10, 1);
+ return illegal_condition_result;
+ } else if (sqcp->inj_dix) {
+ mk_sense_buffer(scp, ILLEGAL_REQUEST, 0x10, 1);
+ return illegal_condition_result;
+ }
}
}
return 0;
}
-static int
-resp_write_same(struct scsi_cmnd *scp, u64 lba, u32 num, u32 ei_lba,
- bool unmap, bool ndob)
+static int resp_write_same(struct scsi_cmnd *scp, u64 lba, u32 num,
+ u32 ei_lba, bool unmap, bool ndob)
{
unsigned long iflags;
unsigned long long i;
int ret;
+ u64 lba_off;
ret = check_device_access_params(scp, lba, num);
if (ret)
@@ -2998,31 +2955,29 @@
goto out;
}
+ lba_off = lba * sdebug_sector_size;
/* if ndob then zero 1 logical block, else fetch 1 logical block */
if (ndob) {
- memset(fake_storep + (lba * scsi_debug_sector_size), 0,
- scsi_debug_sector_size);
+ memset(fake_storep + lba_off, 0, sdebug_sector_size);
ret = 0;
} else
- ret = fetch_to_dev_buffer(scp, fake_storep +
- (lba * scsi_debug_sector_size),
- scsi_debug_sector_size);
+ ret = fetch_to_dev_buffer(scp, fake_storep + lba_off,
+ sdebug_sector_size);
if (-1 == ret) {
write_unlock_irqrestore(&atomic_rw, iflags);
- return (DID_ERROR << 16);
- } else if ((ret < (num * scsi_debug_sector_size)) &&
- (SCSI_DEBUG_OPT_NOISE & scsi_debug_opts))
+ return DID_ERROR << 16;
+ } else if (sdebug_verbose && (ret < (num * sdebug_sector_size)))
sdev_printk(KERN_INFO, scp->device,
"%s: %s: cdb indicated=%u, IO sent=%d bytes\n",
my_name, "write same",
- num * scsi_debug_sector_size, ret);
+ num * sdebug_sector_size, ret);
/* Copy first sector to remaining blocks */
for (i = 1 ; i < num ; i++)
- memcpy(fake_storep + ((lba + i) * scsi_debug_sector_size),
- fake_storep + (lba * scsi_debug_sector_size),
- scsi_debug_sector_size);
+ memcpy(fake_storep + ((lba + i) * sdebug_sector_size),
+ fake_storep + lba_off,
+ sdebug_sector_size);
if (scsi_debug_lbp())
map_region(lba, num);
@@ -3032,8 +2987,8 @@
return 0;
}
-static int
-resp_write_same_10(struct scsi_cmnd *scp, struct sdebug_dev_info *devip)
+static int resp_write_same_10(struct scsi_cmnd *scp,
+ struct sdebug_dev_info *devip)
{
u8 *cmd = scp->cmnd;
u32 lba;
@@ -3042,7 +2997,7 @@
bool unmap = false;
if (cmd[1] & 0x8) {
- if (scsi_debug_lbpws10 == 0) {
+ if (sdebug_lbpws10 == 0) {
mk_sense_invalid_fld(scp, SDEB_IN_CDB, 1, 3);
return check_condition_result;
} else
@@ -3050,15 +3005,15 @@
}
lba = get_unaligned_be32(cmd + 2);
num = get_unaligned_be16(cmd + 7);
- if (num > scsi_debug_write_same_length) {
+ if (num > sdebug_write_same_length) {
mk_sense_invalid_fld(scp, SDEB_IN_CDB, 7, -1);
return check_condition_result;
}
return resp_write_same(scp, lba, num, ei_lba, unmap, false);
}
-static int
-resp_write_same_16(struct scsi_cmnd *scp, struct sdebug_dev_info *devip)
+static int resp_write_same_16(struct scsi_cmnd *scp,
+ struct sdebug_dev_info *devip)
{
u8 *cmd = scp->cmnd;
u64 lba;
@@ -3068,7 +3023,7 @@
bool ndob = false;
if (cmd[1] & 0x8) { /* UNMAP */
- if (scsi_debug_lbpws == 0) {
+ if (sdebug_lbpws == 0) {
mk_sense_invalid_fld(scp, SDEB_IN_CDB, 1, 3);
return check_condition_result;
} else
@@ -3078,7 +3033,7 @@
ndob = true;
lba = get_unaligned_be64(cmd + 2);
num = get_unaligned_be32(cmd + 10);
- if (num > scsi_debug_write_same_length) {
+ if (num > sdebug_write_same_length) {
mk_sense_invalid_fld(scp, SDEB_IN_CDB, 10, -1);
return check_condition_result;
}
@@ -3088,8 +3043,8 @@
/* Note the mode field is in the same position as the (lower) service action
* field. For the Report supported operation codes command, SPC-4 suggests
* each mode of this command should be reported separately; for future. */
-static int
-resp_write_buffer(struct scsi_cmnd *scp, struct sdebug_dev_info *devip)
+static int resp_write_buffer(struct scsi_cmnd *scp,
+ struct sdebug_dev_info *devip)
{
u8 *cmd = scp->cmnd;
struct scsi_device *sdp = scp->device;
@@ -3134,15 +3089,15 @@
return 0;
}
-static int
-resp_comp_write(struct scsi_cmnd *scp, struct sdebug_dev_info *devip)
+static int resp_comp_write(struct scsi_cmnd *scp,
+ struct sdebug_dev_info *devip)
{
u8 *cmd = scp->cmnd;
u8 *arr;
u8 *fake_storep_hold;
u64 lba;
u32 dnum;
- u32 lb_size = scsi_debug_sector_size;
+ u32 lb_size = sdebug_sector_size;
u8 num;
unsigned long iflags;
int ret;
@@ -3152,13 +3107,13 @@
num = cmd[13]; /* 1 to a maximum of 255 logical blocks */
if (0 == num)
return 0; /* degenerate case, not an error */
- if (scsi_debug_dif == SD_DIF_TYPE2_PROTECTION &&
+ if (sdebug_dif == SD_DIF_TYPE2_PROTECTION &&
(cmd[1] & 0xe0)) {
mk_sense_invalid_opcode(scp);
return check_condition_result;
}
- if ((scsi_debug_dif == SD_DIF_TYPE1_PROTECTION ||
- scsi_debug_dif == SD_DIF_TYPE3_PROTECTION) &&
+ if ((sdebug_dif == SD_DIF_TYPE1_PROTECTION ||
+ sdebug_dif == SD_DIF_TYPE3_PROTECTION) &&
(cmd[1] & 0xe0) == 0)
sdev_printk(KERN_ERR, scp->device, "Unprotected WR "
"to DIF device\n");
@@ -3193,8 +3148,7 @@
if (ret == -1) {
retval = DID_ERROR << 16;
goto cleanup;
- } else if ((ret < (dnum * lb_size)) &&
- (SCSI_DEBUG_OPT_NOISE & scsi_debug_opts))
+ } else if (sdebug_verbose && (ret < (dnum * lb_size)))
sdev_printk(KERN_INFO, scp->device, "%s: compare_write: cdb "
"indicated=%u, IO sent=%d bytes\n", my_name,
dnum * lb_size, ret);
@@ -3217,8 +3171,7 @@
__be32 __reserved;
};
-static int
-resp_unmap(struct scsi_cmnd *scp, struct sdebug_dev_info *devip)
+static int resp_unmap(struct scsi_cmnd *scp, struct sdebug_dev_info *devip)
{
unsigned char *buf;
struct unmap_block_desc *desc;
@@ -3233,12 +3186,12 @@
BUG_ON(scsi_bufflen(scp) != payload_len);
descriptors = (payload_len - 8) / 16;
- if (descriptors > scsi_debug_unmap_max_desc) {
+ if (descriptors > sdebug_unmap_max_desc) {
mk_sense_invalid_fld(scp, SDEB_IN_CDB, 7, -1);
return check_condition_result;
}
- buf = kmalloc(scsi_bufflen(scp), GFP_ATOMIC);
+ buf = kzalloc(scsi_bufflen(scp), GFP_ATOMIC);
if (!buf) {
mk_sense_buffer(scp, ILLEGAL_REQUEST, INSUFF_RES_ASC,
INSUFF_RES_ASCQ);
@@ -3276,8 +3229,8 @@
#define SDEBUG_GET_LBA_STATUS_LEN 32
-static int
-resp_get_lba_status(struct scsi_cmnd *scp, struct sdebug_dev_info *devip)
+static int resp_get_lba_status(struct scsi_cmnd *scp,
+ struct sdebug_dev_info *devip)
{
u8 *cmd = scp->cmnd;
u64 lba;
@@ -3316,63 +3269,94 @@
return fill_from_dev_buffer(scp, arr, SDEBUG_GET_LBA_STATUS_LEN);
}
-#define SDEBUG_RLUN_ARR_SZ 256
-
-static int resp_report_luns(struct scsi_cmnd * scp,
- struct sdebug_dev_info * devip)
+/* Even though each pseudo target has a REPORT LUNS "well known logical unit"
+ * (W-LUN), the normal Linux scanning logic does not associate it with a
+ * device (e.g. /dev/sg7). The following magic will make that association:
+ * "cd /sys/class/scsi_host/host<n> ; echo '- - 49409' > scan"
+ * where <n> is a host number. If there are multiple targets in a host then
+ * the above will associate a W-LUN to each target. To only get a W-LUN
+ * for target 2, then use "echo '- 2 49409' > scan" .
+ */
+static int resp_report_luns(struct scsi_cmnd *scp,
+ struct sdebug_dev_info *devip)
{
- unsigned int alloc_len;
- int lun_cnt, i, upper, num, n, want_wlun, shortish;
- u64 lun;
unsigned char *cmd = scp->cmnd;
- int select_report = (int)cmd[2];
- struct scsi_lun *one_lun;
- unsigned char arr[SDEBUG_RLUN_ARR_SZ];
- unsigned char * max_addr;
+ unsigned int alloc_len;
+ unsigned char select_report;
+ u64 lun;
+ struct scsi_lun *lun_p;
+ u8 *arr;
+ unsigned int lun_cnt; /* normal LUN count (max: 256) */
+ unsigned int wlun_cnt; /* report luns W-LUN count */
+ unsigned int tlun_cnt; /* total LUN count */
+ unsigned int rlen; /* response length (in bytes) */
+ int i, res;
clear_luns_changed_on_target(devip);
- alloc_len = cmd[9] + (cmd[8] << 8) + (cmd[7] << 16) + (cmd[6] << 24);
- shortish = (alloc_len < 4);
- if (shortish || (select_report > 2)) {
- mk_sense_invalid_fld(scp, SDEB_IN_CDB, shortish ? 6 : 2, -1);
+
+ select_report = cmd[2];
+ alloc_len = get_unaligned_be32(cmd + 6);
+
+ if (alloc_len < 4) {
+ pr_err("alloc len too small %d\n", alloc_len);
+ mk_sense_invalid_fld(scp, SDEB_IN_CDB, 6, -1);
return check_condition_result;
}
- /* can produce response with up to 16k luns (lun 0 to lun 16383) */
- memset(arr, 0, SDEBUG_RLUN_ARR_SZ);
- lun_cnt = scsi_debug_max_luns;
- if (1 == select_report)
+
+ switch (select_report) {
+ case 0: /* all LUNs apart from W-LUNs */
+ lun_cnt = sdebug_max_luns;
+ wlun_cnt = 0;
+ break;
+ case 1: /* only W-LUNs */
lun_cnt = 0;
- else if (scsi_debug_no_lun_0 && (lun_cnt > 0))
+ wlun_cnt = 1;
+ break;
+ case 2: /* all LUNs */
+ lun_cnt = sdebug_max_luns;
+ wlun_cnt = 1;
+ break;
+ case 0x10: /* only administrative LUs */
+ case 0x11: /* see SPC-5 */
+ case 0x12: /* only subsiduary LUs owned by referenced LU */
+ default:
+ pr_debug("select report invalid %d\n", select_report);
+ mk_sense_invalid_fld(scp, SDEB_IN_CDB, 2, -1);
+ return check_condition_result;
+ }
+
+ if (sdebug_no_lun_0 && (lun_cnt > 0))
--lun_cnt;
- want_wlun = (select_report > 0) ? 1 : 0;
- num = lun_cnt + want_wlun;
- arr[2] = ((sizeof(struct scsi_lun) * num) >> 8) & 0xff;
- arr[3] = (sizeof(struct scsi_lun) * num) & 0xff;
- n = min((int)((SDEBUG_RLUN_ARR_SZ - 8) /
- sizeof(struct scsi_lun)), num);
- if (n < num) {
- want_wlun = 0;
- lun_cnt = n;
+
+ tlun_cnt = lun_cnt + wlun_cnt;
+
+ rlen = (tlun_cnt * sizeof(struct scsi_lun)) + 8;
+ arr = vmalloc(rlen);
+ if (!arr) {
+ mk_sense_buffer(scp, ILLEGAL_REQUEST, INSUFF_RES_ASC,
+ INSUFF_RES_ASCQ);
+ return check_condition_result;
}
- one_lun = (struct scsi_lun *) &arr[8];
- max_addr = arr + SDEBUG_RLUN_ARR_SZ;
- for (i = 0, lun = (scsi_debug_no_lun_0 ? 1 : 0);
- ((i < lun_cnt) && ((unsigned char *)(one_lun + i) < max_addr));
- i++, lun++) {
- upper = (lun >> 8) & 0x3f;
- if (upper)
- one_lun[i].scsi_lun[0] =
- (upper | (SAM2_LUN_ADDRESS_METHOD << 6));
- one_lun[i].scsi_lun[1] = lun & 0xff;
- }
- if (want_wlun) {
- one_lun[i].scsi_lun[0] = (SCSI_W_LUN_REPORT_LUNS >> 8) & 0xff;
- one_lun[i].scsi_lun[1] = SCSI_W_LUN_REPORT_LUNS & 0xff;
- i++;
- }
- alloc_len = (unsigned char *)(one_lun + i) - arr;
- return fill_from_dev_buffer(scp, arr,
- min((int)alloc_len, SDEBUG_RLUN_ARR_SZ));
+ memset(arr, 0, rlen);
+ pr_debug("select_report %d luns = %d wluns = %d no_lun0 %d\n",
+ select_report, lun_cnt, wlun_cnt, sdebug_no_lun_0);
+
+ /* luns start at byte 8 in response following the header */
+ lun_p = (struct scsi_lun *)&arr[8];
+
+ /* LUNs use single level peripheral device addressing method */
+ lun = sdebug_no_lun_0 ? 1 : 0;
+ for (i = 0; i < lun_cnt; i++)
+ int_to_scsilun(lun++, lun_p++);
+
+ if (wlun_cnt)
+ int_to_scsilun(SCSI_W_LUN_REPORT_LUNS, lun_p++);
+
+ put_unaligned_be32(rlen - 8, &arr[0]);
+
+ res = fill_from_dev_buffer(scp, arr, rlen);
+ vfree(arr);
+ return res;
}
static int resp_xdwriteread(struct scsi_cmnd *scp, unsigned long long lba,
@@ -3385,7 +3369,7 @@
struct sg_mapping_iter miter;
/* better not to use temporary buffer. */
- buf = kmalloc(scsi_bufflen(scp), GFP_ATOMIC);
+ buf = kzalloc(scsi_bufflen(scp), GFP_ATOMIC);
if (!buf) {
mk_sense_buffer(scp, ILLEGAL_REQUEST, INSUFF_RES_ASC,
INSUFF_RES_ASCQ);
@@ -3411,8 +3395,8 @@
return 0;
}
-static int
-resp_xdwriteread_10(struct scsi_cmnd *scp, struct sdebug_dev_info *devip)
+static int resp_xdwriteread_10(struct scsi_cmnd *scp,
+ struct sdebug_dev_info *devip)
{
u8 *cmd = scp->cmnd;
u64 lba;
@@ -3437,41 +3421,66 @@
return resp_xdwriteread(scp, lba, num, devip);
}
-/* When timer or tasklet goes off this function is called. */
-static void sdebug_q_cmd_complete(unsigned long indx)
+static struct sdebug_queue *get_queue(struct scsi_cmnd *cmnd)
{
- int qa_indx;
+ struct sdebug_queue *sqp = sdebug_q_arr;
+
+ if (sdebug_mq_active) {
+ u32 tag = blk_mq_unique_tag(cmnd->request);
+ u16 hwq = blk_mq_unique_tag_to_hwq(tag);
+
+ if (unlikely(hwq >= submit_queues)) {
+ pr_warn("Unexpected hwq=%d, apply modulo\n", hwq);
+ hwq %= submit_queues;
+ }
+ pr_debug("tag=%u, hwq=%d\n", tag, hwq);
+ return sqp + hwq;
+ } else
+ return sqp;
+}
+
+/* Queued (deferred) command completions converge here. */
+static void sdebug_q_cmd_complete(struct sdebug_defer *sd_dp)
+{
+ int qc_idx;
int retiring = 0;
unsigned long iflags;
+ struct sdebug_queue *sqp;
struct sdebug_queued_cmd *sqcp;
struct scsi_cmnd *scp;
struct sdebug_dev_info *devip;
- atomic_inc(&sdebug_completions);
- qa_indx = indx;
- if ((qa_indx < 0) || (qa_indx >= SCSI_DEBUG_CANQUEUE)) {
- pr_err("wild qa_indx=%d\n", qa_indx);
+ qc_idx = sd_dp->qc_idx;
+ sqp = sdebug_q_arr + sd_dp->sqa_idx;
+ if (sdebug_statistics) {
+ atomic_inc(&sdebug_completions);
+ if (raw_smp_processor_id() != sd_dp->issuing_cpu)
+ atomic_inc(&sdebug_miss_cpus);
+ }
+ if (unlikely((qc_idx < 0) || (qc_idx >= SDEBUG_CANQUEUE))) {
+ pr_err("wild qc_idx=%d\n", qc_idx);
return;
}
- spin_lock_irqsave(&queued_arr_lock, iflags);
- sqcp = &queued_arr[qa_indx];
+ spin_lock_irqsave(&sqp->qc_lock, iflags);
+ sqcp = &sqp->qc_arr[qc_idx];
scp = sqcp->a_cmnd;
- if (NULL == scp) {
- spin_unlock_irqrestore(&queued_arr_lock, iflags);
- pr_err("scp is NULL\n");
+ if (unlikely(scp == NULL)) {
+ spin_unlock_irqrestore(&sqp->qc_lock, iflags);
+ pr_err("scp is NULL, sqa_idx=%d, qc_idx=%d\n",
+ sd_dp->sqa_idx, qc_idx);
return;
}
devip = (struct sdebug_dev_info *)scp->device->hostdata;
- if (devip)
+ if (likely(devip))
atomic_dec(&devip->num_in_q);
else
pr_err("devip=NULL\n");
- if (atomic_read(&retired_max_queue) > 0)
+ if (unlikely(atomic_read(&retired_max_queue) > 0))
retiring = 1;
sqcp->a_cmnd = NULL;
- if (!test_and_clear_bit(qa_indx, queued_in_use_bm)) {
- spin_unlock_irqrestore(&queued_arr_lock, iflags);
+ if (unlikely(!test_and_clear_bit(qc_idx, sqp->in_use_bm))) {
+ spin_unlock_irqrestore(&sqp->qc_lock, iflags);
pr_err("Unexpected completion\n");
return;
}
@@ -3480,105 +3489,71 @@
int k, retval;
retval = atomic_read(&retired_max_queue);
- if (qa_indx >= retval) {
- spin_unlock_irqrestore(&queued_arr_lock, iflags);
+ if (qc_idx >= retval) {
+ spin_unlock_irqrestore(&sqp->qc_lock, iflags);
pr_err("index %d too large\n", retval);
return;
}
- k = find_last_bit(queued_in_use_bm, retval);
- if ((k < scsi_debug_max_queue) || (k == retval))
+ k = find_last_bit(sqp->in_use_bm, retval);
+ if ((k < sdebug_max_queue) || (k == retval))
atomic_set(&retired_max_queue, 0);
else
atomic_set(&retired_max_queue, k + 1);
}
- spin_unlock_irqrestore(&queued_arr_lock, iflags);
+ spin_unlock_irqrestore(&sqp->qc_lock, iflags);
scp->scsi_done(scp); /* callback to mid level */
}
/* When high resolution timer goes off this function is called. */
-static enum hrtimer_restart
-sdebug_q_cmd_hrt_complete(struct hrtimer *timer)
+static enum hrtimer_restart sdebug_q_cmd_hrt_complete(struct hrtimer *timer)
{
- int qa_indx;
- int retiring = 0;
- unsigned long iflags;
- struct sdebug_hrtimer *sd_hrtp = (struct sdebug_hrtimer *)timer;
- struct sdebug_queued_cmd *sqcp;
- struct scsi_cmnd *scp;
- struct sdebug_dev_info *devip;
-
- atomic_inc(&sdebug_completions);
- qa_indx = sd_hrtp->qa_indx;
- if ((qa_indx < 0) || (qa_indx >= SCSI_DEBUG_CANQUEUE)) {
- pr_err("wild qa_indx=%d\n", qa_indx);
- goto the_end;
- }
- spin_lock_irqsave(&queued_arr_lock, iflags);
- sqcp = &queued_arr[qa_indx];
- scp = sqcp->a_cmnd;
- if (NULL == scp) {
- spin_unlock_irqrestore(&queued_arr_lock, iflags);
- pr_err("scp is NULL\n");
- goto the_end;
- }
- devip = (struct sdebug_dev_info *)scp->device->hostdata;
- if (devip)
- atomic_dec(&devip->num_in_q);
- else
- pr_err("devip=NULL\n");
- if (atomic_read(&retired_max_queue) > 0)
- retiring = 1;
-
- sqcp->a_cmnd = NULL;
- if (!test_and_clear_bit(qa_indx, queued_in_use_bm)) {
- spin_unlock_irqrestore(&queued_arr_lock, iflags);
- pr_err("Unexpected completion\n");
- goto the_end;
- }
-
- if (unlikely(retiring)) { /* user has reduced max_queue */
- int k, retval;
-
- retval = atomic_read(&retired_max_queue);
- if (qa_indx >= retval) {
- spin_unlock_irqrestore(&queued_arr_lock, iflags);
- pr_err("index %d too large\n", retval);
- goto the_end;
- }
- k = find_last_bit(queued_in_use_bm, retval);
- if ((k < scsi_debug_max_queue) || (k == retval))
- atomic_set(&retired_max_queue, 0);
- else
- atomic_set(&retired_max_queue, k + 1);
- }
- spin_unlock_irqrestore(&queued_arr_lock, iflags);
- scp->scsi_done(scp); /* callback to mid level */
-the_end:
+ struct sdebug_defer *sd_dp = container_of(timer, struct sdebug_defer,
+ hrt);
+ sdebug_q_cmd_complete(sd_dp);
return HRTIMER_NORESTART;
}
-static struct sdebug_dev_info *
-sdebug_device_create(struct sdebug_host_info *sdbg_host, gfp_t flags)
+/* When work queue schedules work, it calls this function. */
+static void sdebug_q_cmd_wq_complete(struct work_struct *work)
+{
+ struct sdebug_defer *sd_dp = container_of(work, struct sdebug_defer,
+ ew.work);
+ sdebug_q_cmd_complete(sd_dp);
+}
+
+static bool got_shared_uuid;
+static uuid_be shared_uuid;
+
+static struct sdebug_dev_info *sdebug_device_create(
+ struct sdebug_host_info *sdbg_host, gfp_t flags)
{
struct sdebug_dev_info *devip;
devip = kzalloc(sizeof(*devip), flags);
if (devip) {
+ if (sdebug_uuid_ctl == 1)
+ uuid_be_gen(&devip->lu_name);
+ else if (sdebug_uuid_ctl == 2) {
+ if (got_shared_uuid)
+ devip->lu_name = shared_uuid;
+ else {
+ uuid_be_gen(&shared_uuid);
+ got_shared_uuid = true;
+ devip->lu_name = shared_uuid;
+ }
+ }
devip->sdbg_host = sdbg_host;
list_add_tail(&devip->dev_list, &sdbg_host->dev_info_list);
}
return devip;
}
-static struct sdebug_dev_info * devInfoReg(struct scsi_device * sdev)
+static struct sdebug_dev_info *find_build_dev_info(struct scsi_device *sdev)
{
- struct sdebug_host_info * sdbg_host;
- struct sdebug_dev_info * open_devip = NULL;
- struct sdebug_dev_info * devip =
- (struct sdebug_dev_info *)sdev->hostdata;
+ struct sdebug_host_info *sdbg_host;
+ struct sdebug_dev_info *open_devip = NULL;
+ struct sdebug_dev_info *devip;
- if (devip)
- return devip;
sdbg_host = *(struct sdebug_host_info **)shost_priv(sdev->host);
if (!sdbg_host) {
pr_err("Host info NULL\n");
@@ -3614,7 +3589,7 @@
static int scsi_debug_slave_alloc(struct scsi_device *sdp)
{
- if (SCSI_DEBUG_OPT_NOISE & scsi_debug_opts)
+ if (sdebug_verbose)
pr_info("slave_alloc <%u %u %u %llu>\n",
sdp->host->host_no, sdp->channel, sdp->id, sdp->lun);
queue_flag_set_unlocked(QUEUE_FLAG_BIDI, sdp->request_queue);
@@ -3623,19 +3598,22 @@
static int scsi_debug_slave_configure(struct scsi_device *sdp)
{
- struct sdebug_dev_info *devip;
+ struct sdebug_dev_info *devip =
+ (struct sdebug_dev_info *)sdp->hostdata;
- if (SCSI_DEBUG_OPT_NOISE & scsi_debug_opts)
+ if (sdebug_verbose)
pr_info("slave_configure <%u %u %u %llu>\n",
sdp->host->host_no, sdp->channel, sdp->id, sdp->lun);
- if (sdp->host->max_cmd_len != SCSI_DEBUG_MAX_CMD_LEN)
- sdp->host->max_cmd_len = SCSI_DEBUG_MAX_CMD_LEN;
- devip = devInfoReg(sdp);
- if (NULL == devip)
- return 1; /* no resources, will be marked offline */
+ if (sdp->host->max_cmd_len != SDEBUG_MAX_CMD_LEN)
+ sdp->host->max_cmd_len = SDEBUG_MAX_CMD_LEN;
+ if (devip == NULL) {
+ devip = find_build_dev_info(sdp);
+ if (devip == NULL)
+ return 1; /* no resources, will be marked offline */
+ }
sdp->hostdata = devip;
blk_queue_max_segment_size(sdp->request_queue, -1U);
- if (scsi_debug_no_uld)
+ if (sdebug_no_uld)
sdp->no_uld_attach = 1;
return 0;
}
@@ -3645,7 +3623,7 @@
struct sdebug_dev_info *devip =
(struct sdebug_dev_info *)sdp->hostdata;
- if (SCSI_DEBUG_OPT_NOISE & scsi_debug_opts)
+ if (sdebug_verbose)
pr_info("slave_destroy <%u %u %u %llu>\n",
sdp->host->host_no, sdp->channel, sdp->id, sdp->lun);
if (devip) {
@@ -3655,135 +3633,130 @@
}
}
-/* Returns 1 if cmnd found (deletes its timer or tasklet), else returns 0 */
-static int stop_queued_cmnd(struct scsi_cmnd *cmnd)
+static void stop_qc_helper(struct sdebug_defer *sd_dp)
+{
+ if (!sd_dp)
+ return;
+ if ((sdebug_jdelay > 0) || (sdebug_ndelay > 0))
+ hrtimer_cancel(&sd_dp->hrt);
+ else if (sdebug_jdelay < 0)
+ cancel_work_sync(&sd_dp->ew.work);
+}
+
+/* If @cmnd found deletes its timer or work queue and returns true; else
+ returns false */
+static bool stop_queued_cmnd(struct scsi_cmnd *cmnd)
{
unsigned long iflags;
- int k, qmax, r_qmax;
+ int j, k, qmax, r_qmax;
+ struct sdebug_queue *sqp;
struct sdebug_queued_cmd *sqcp;
struct sdebug_dev_info *devip;
+ struct sdebug_defer *sd_dp;
- spin_lock_irqsave(&queued_arr_lock, iflags);
- qmax = scsi_debug_max_queue;
- r_qmax = atomic_read(&retired_max_queue);
- if (r_qmax > qmax)
- qmax = r_qmax;
- for (k = 0; k < qmax; ++k) {
- if (test_bit(k, queued_in_use_bm)) {
- sqcp = &queued_arr[k];
- if (cmnd == sqcp->a_cmnd) {
+ for (j = 0, sqp = sdebug_q_arr; j < submit_queues; ++j, ++sqp) {
+ spin_lock_irqsave(&sqp->qc_lock, iflags);
+ qmax = sdebug_max_queue;
+ r_qmax = atomic_read(&retired_max_queue);
+ if (r_qmax > qmax)
+ qmax = r_qmax;
+ for (k = 0; k < qmax; ++k) {
+ if (test_bit(k, sqp->in_use_bm)) {
+ sqcp = &sqp->qc_arr[k];
+ if (cmnd != sqcp->a_cmnd)
+ continue;
+ /* found */
devip = (struct sdebug_dev_info *)
- cmnd->device->hostdata;
+ cmnd->device->hostdata;
if (devip)
atomic_dec(&devip->num_in_q);
sqcp->a_cmnd = NULL;
- spin_unlock_irqrestore(&queued_arr_lock,
- iflags);
- if (scsi_debug_ndelay > 0) {
- if (sqcp->sd_hrtp)
- hrtimer_cancel(
- &sqcp->sd_hrtp->hrt);
- } else if (scsi_debug_delay > 0) {
- if (sqcp->cmnd_timerp)
- del_timer_sync(
- sqcp->cmnd_timerp);
- } else if (scsi_debug_delay < 0) {
- if (sqcp->tletp)
- tasklet_kill(sqcp->tletp);
- }
- clear_bit(k, queued_in_use_bm);
- return 1;
+ sd_dp = sqcp->sd_dp;
+ spin_unlock_irqrestore(&sqp->qc_lock, iflags);
+ stop_qc_helper(sd_dp);
+ clear_bit(k, sqp->in_use_bm);
+ return true;
}
}
+ spin_unlock_irqrestore(&sqp->qc_lock, iflags);
}
- spin_unlock_irqrestore(&queued_arr_lock, iflags);
- return 0;
+ return false;
}
-/* Deletes (stops) timers or tasklets of all queued commands */
+/* Deletes (stops) timers or work queues of all queued commands */
static void stop_all_queued(void)
{
unsigned long iflags;
- int k;
+ int j, k;
+ struct sdebug_queue *sqp;
struct sdebug_queued_cmd *sqcp;
struct sdebug_dev_info *devip;
+ struct sdebug_defer *sd_dp;
- spin_lock_irqsave(&queued_arr_lock, iflags);
- for (k = 0; k < SCSI_DEBUG_CANQUEUE; ++k) {
- if (test_bit(k, queued_in_use_bm)) {
- sqcp = &queued_arr[k];
- if (sqcp->a_cmnd) {
+ for (j = 0, sqp = sdebug_q_arr; j < submit_queues; ++j, ++sqp) {
+ spin_lock_irqsave(&sqp->qc_lock, iflags);
+ for (k = 0; k < SDEBUG_CANQUEUE; ++k) {
+ if (test_bit(k, sqp->in_use_bm)) {
+ sqcp = &sqp->qc_arr[k];
+ if (sqcp->a_cmnd == NULL)
+ continue;
devip = (struct sdebug_dev_info *)
sqcp->a_cmnd->device->hostdata;
if (devip)
atomic_dec(&devip->num_in_q);
sqcp->a_cmnd = NULL;
- spin_unlock_irqrestore(&queued_arr_lock,
- iflags);
- if (scsi_debug_ndelay > 0) {
- if (sqcp->sd_hrtp)
- hrtimer_cancel(
- &sqcp->sd_hrtp->hrt);
- } else if (scsi_debug_delay > 0) {
- if (sqcp->cmnd_timerp)
- del_timer_sync(
- sqcp->cmnd_timerp);
- } else if (scsi_debug_delay < 0) {
- if (sqcp->tletp)
- tasklet_kill(sqcp->tletp);
- }
- clear_bit(k, queued_in_use_bm);
- spin_lock_irqsave(&queued_arr_lock, iflags);
+ sd_dp = sqcp->sd_dp;
+ spin_unlock_irqrestore(&sqp->qc_lock, iflags);
+ stop_qc_helper(sd_dp);
+ clear_bit(k, sqp->in_use_bm);
+ spin_lock_irqsave(&sqp->qc_lock, iflags);
}
}
+ spin_unlock_irqrestore(&sqp->qc_lock, iflags);
}
- spin_unlock_irqrestore(&queued_arr_lock, iflags);
}
/* Free queued command memory on heap */
static void free_all_queued(void)
{
- unsigned long iflags;
- int k;
+ int j, k;
+ struct sdebug_queue *sqp;
struct sdebug_queued_cmd *sqcp;
- spin_lock_irqsave(&queued_arr_lock, iflags);
- for (k = 0; k < SCSI_DEBUG_CANQUEUE; ++k) {
- sqcp = &queued_arr[k];
- kfree(sqcp->cmnd_timerp);
- sqcp->cmnd_timerp = NULL;
- kfree(sqcp->tletp);
- sqcp->tletp = NULL;
- kfree(sqcp->sd_hrtp);
- sqcp->sd_hrtp = NULL;
+ for (j = 0, sqp = sdebug_q_arr; j < submit_queues; ++j, ++sqp) {
+ for (k = 0; k < SDEBUG_CANQUEUE; ++k) {
+ sqcp = &sqp->qc_arr[k];
+ kfree(sqcp->sd_dp);
+ sqcp->sd_dp = NULL;
+ }
}
- spin_unlock_irqrestore(&queued_arr_lock, iflags);
}
static int scsi_debug_abort(struct scsi_cmnd *SCpnt)
{
+ bool ok;
+
++num_aborts;
if (SCpnt) {
- if (SCpnt->device &&
- (SCSI_DEBUG_OPT_ALL_NOISE & scsi_debug_opts))
- sdev_printk(KERN_INFO, SCpnt->device, "%s\n",
- __func__);
- stop_queued_cmnd(SCpnt);
+ ok = stop_queued_cmnd(SCpnt);
+ if (SCpnt->device && (SDEBUG_OPT_ALL_NOISE & sdebug_opts))
+ sdev_printk(KERN_INFO, SCpnt->device,
+ "%s: command%s found\n", __func__,
+ ok ? "" : " not");
}
return SUCCESS;
}
static int scsi_debug_device_reset(struct scsi_cmnd * SCpnt)
{
- struct sdebug_dev_info * devip;
-
++num_dev_resets;
if (SCpnt && SCpnt->device) {
struct scsi_device *sdp = SCpnt->device;
+ struct sdebug_dev_info *devip =
+ (struct sdebug_dev_info *)sdp->hostdata;
- if (SCSI_DEBUG_OPT_ALL_NOISE & scsi_debug_opts)
+ if (SDEBUG_OPT_ALL_NOISE & sdebug_opts)
sdev_printk(KERN_INFO, sdp, "%s\n", __func__);
- devip = devInfoReg(sdp);
if (devip)
set_bit(SDEBUG_UA_POR, devip->uas_bm);
}
@@ -3804,7 +3777,7 @@
sdp = SCpnt->device;
if (!sdp)
goto lie;
- if (SCSI_DEBUG_OPT_ALL_NOISE & scsi_debug_opts)
+ if (SDEBUG_OPT_ALL_NOISE & sdebug_opts)
sdev_printk(KERN_INFO, sdp, "%s\n", __func__);
hp = sdp->host;
if (!hp)
@@ -3819,7 +3792,7 @@
++k;
}
}
- if (SCSI_DEBUG_OPT_RESET_NOISE & scsi_debug_opts)
+ if (SDEBUG_OPT_RESET_NOISE & sdebug_opts)
sdev_printk(KERN_INFO, sdp,
"%s: %d device(s) found in target\n", __func__, k);
lie:
@@ -3838,7 +3811,7 @@
if (!(SCpnt && SCpnt->device))
goto lie;
sdp = SCpnt->device;
- if (SCSI_DEBUG_OPT_ALL_NOISE & scsi_debug_opts)
+ if (SDEBUG_OPT_ALL_NOISE & sdebug_opts)
sdev_printk(KERN_INFO, sdp, "%s\n", __func__);
hp = sdp->host;
if (hp) {
@@ -3852,7 +3825,7 @@
}
}
}
- if (SCSI_DEBUG_OPT_RESET_NOISE & scsi_debug_opts)
+ if (SDEBUG_OPT_RESET_NOISE & sdebug_opts)
sdev_printk(KERN_INFO, sdp,
"%s: %d device(s) found in host\n", __func__, k);
lie:
@@ -3866,7 +3839,7 @@
int k = 0;
++num_host_resets;
- if ((SCpnt->device) && (SCSI_DEBUG_OPT_ALL_NOISE & scsi_debug_opts))
+ if ((SCpnt->device) && (SDEBUG_OPT_ALL_NOISE & sdebug_opts))
sdev_printk(KERN_INFO, SCpnt->device, "%s\n", __func__);
spin_lock(&sdebug_host_list_lock);
list_for_each_entry(sdbg_host, &sdebug_host_list, host_list) {
@@ -3878,7 +3851,7 @@
}
spin_unlock(&sdebug_host_list_lock);
stop_all_queued();
- if (SCSI_DEBUG_OPT_RESET_NOISE & scsi_debug_opts)
+ if (SDEBUG_OPT_RESET_NOISE & sdebug_opts)
sdev_printk(KERN_INFO, SCpnt->device,
"%s: %d device(s) found\n", __func__, k);
return SUCCESS;
@@ -3893,22 +3866,22 @@
int heads_by_sects, start_sec, end_sec;
/* assume partition table already zeroed */
- if ((scsi_debug_num_parts < 1) || (store_size < 1048576))
+ if ((sdebug_num_parts < 1) || (store_size < 1048576))
return;
- if (scsi_debug_num_parts > SDEBUG_MAX_PARTS) {
- scsi_debug_num_parts = SDEBUG_MAX_PARTS;
+ if (sdebug_num_parts > SDEBUG_MAX_PARTS) {
+ sdebug_num_parts = SDEBUG_MAX_PARTS;
pr_warn("reducing partitions to %d\n", SDEBUG_MAX_PARTS);
}
num_sectors = (int)sdebug_store_sectors;
sectors_per_part = (num_sectors - sdebug_sectors_per)
- / scsi_debug_num_parts;
+ / sdebug_num_parts;
heads_by_sects = sdebug_heads * sdebug_sectors_per;
starts[0] = sdebug_sectors_per;
- for (k = 1; k < scsi_debug_num_parts; ++k)
+ for (k = 1; k < sdebug_num_parts; ++k)
starts[k] = ((k * sectors_per_part) / heads_by_sects)
* heads_by_sects;
- starts[scsi_debug_num_parts] = num_sectors;
- starts[scsi_debug_num_parts + 1] = 0;
+ starts[sdebug_num_parts] = num_sectors;
+ starts[sdebug_num_parts + 1] = 0;
ramp[510] = 0x55; /* magic partition markings */
ramp[511] = 0xAA;
@@ -3934,67 +3907,118 @@
}
}
-static int
-schedule_resp(struct scsi_cmnd *cmnd, struct sdebug_dev_info *devip,
- int scsi_result, int delta_jiff)
+static void block_unblock_all_queues(bool block)
+{
+ int j;
+ struct sdebug_queue *sqp;
+
+ for (j = 0, sqp = sdebug_q_arr; j < submit_queues; ++j, ++sqp)
+ atomic_set(&sqp->blocked, (int)block);
+}
+
+/* Adjust (by rounding down) the sdebug_cmnd_count so abs(every_nth)-1
+ * commands will be processed normally before triggers occur.
+ */
+static void tweak_cmnd_count(void)
+{
+ int count, modulo;
+
+ modulo = abs(sdebug_every_nth);
+ if (modulo < 2)
+ return;
+ block_unblock_all_queues(true);
+ count = atomic_read(&sdebug_cmnd_count);
+ atomic_set(&sdebug_cmnd_count, (count / modulo) * modulo);
+ block_unblock_all_queues(false);
+}
+
+static void clear_queue_stats(void)
+{
+ atomic_set(&sdebug_cmnd_count, 0);
+ atomic_set(&sdebug_completions, 0);
+ atomic_set(&sdebug_miss_cpus, 0);
+ atomic_set(&sdebug_a_tsf, 0);
+}
+
+static void setup_inject(struct sdebug_queue *sqp,
+ struct sdebug_queued_cmd *sqcp)
+{
+ if ((atomic_read(&sdebug_cmnd_count) % abs(sdebug_every_nth)) > 0)
+ return;
+ sqcp->inj_recovered = !!(SDEBUG_OPT_RECOVERED_ERR & sdebug_opts);
+ sqcp->inj_transport = !!(SDEBUG_OPT_TRANSPORT_ERR & sdebug_opts);
+ sqcp->inj_dif = !!(SDEBUG_OPT_DIF_ERR & sdebug_opts);
+ sqcp->inj_dix = !!(SDEBUG_OPT_DIX_ERR & sdebug_opts);
+ sqcp->inj_short = !!(SDEBUG_OPT_SHORT_TRANSFER & sdebug_opts);
+}
+
+/* Complete the processing of the thread that queued a SCSI command to this
+ * driver. It either completes the command by calling cmnd_done() or
+ * schedules a hr timer or work queue then returns 0. Returns
+ * SCSI_MLQUEUE_HOST_BUSY if temporarily out of resources.
+ */
+static int schedule_resp(struct scsi_cmnd *cmnd, struct sdebug_dev_info *devip,
+ int scsi_result, int delta_jiff)
{
unsigned long iflags;
int k, num_in_q, qdepth, inject;
- struct sdebug_queued_cmd *sqcp = NULL;
+ struct sdebug_queue *sqp;
+ struct sdebug_queued_cmd *sqcp;
struct scsi_device *sdp;
+ struct sdebug_defer *sd_dp;
- /* this should never happen */
- if (WARN_ON(!cmnd))
- return SCSI_MLQUEUE_HOST_BUSY;
-
- if (NULL == devip) {
- pr_warn("called devip == NULL\n");
- /* no particularly good error to report back */
- return SCSI_MLQUEUE_HOST_BUSY;
+ if (unlikely(devip == NULL)) {
+ if (scsi_result == 0)
+ scsi_result = DID_NO_CONNECT << 16;
+ goto respond_in_thread;
}
-
sdp = cmnd->device;
- if ((scsi_result) && (SCSI_DEBUG_OPT_NOISE & scsi_debug_opts))
+ if (unlikely(sdebug_verbose && scsi_result))
sdev_printk(KERN_INFO, sdp, "%s: non-zero result=0x%x\n",
__func__, scsi_result);
if (delta_jiff == 0)
goto respond_in_thread;
/* schedule the response at a later time if resources permit */
- spin_lock_irqsave(&queued_arr_lock, iflags);
+ sqp = get_queue(cmnd);
+ spin_lock_irqsave(&sqp->qc_lock, iflags);
+ if (unlikely(atomic_read(&sqp->blocked))) {
+ spin_unlock_irqrestore(&sqp->qc_lock, iflags);
+ return SCSI_MLQUEUE_HOST_BUSY;
+ }
num_in_q = atomic_read(&devip->num_in_q);
qdepth = cmnd->device->queue_depth;
inject = 0;
- if ((qdepth > 0) && (num_in_q >= qdepth)) {
+ if (unlikely((qdepth > 0) && (num_in_q >= qdepth))) {
if (scsi_result) {
- spin_unlock_irqrestore(&queued_arr_lock, iflags);
+ spin_unlock_irqrestore(&sqp->qc_lock, iflags);
goto respond_in_thread;
} else
scsi_result = device_qfull_result;
- } else if ((scsi_debug_every_nth != 0) &&
- (SCSI_DEBUG_OPT_RARE_TSF & scsi_debug_opts) &&
- (scsi_result == 0)) {
+ } else if (unlikely(sdebug_every_nth &&
+ (SDEBUG_OPT_RARE_TSF & sdebug_opts) &&
+ (scsi_result == 0))) {
if ((num_in_q == (qdepth - 1)) &&
(atomic_inc_return(&sdebug_a_tsf) >=
- abs(scsi_debug_every_nth))) {
+ abs(sdebug_every_nth))) {
atomic_set(&sdebug_a_tsf, 0);
inject = 1;
scsi_result = device_qfull_result;
}
}
- k = find_first_zero_bit(queued_in_use_bm, scsi_debug_max_queue);
- if (k >= scsi_debug_max_queue) {
- spin_unlock_irqrestore(&queued_arr_lock, iflags);
+ k = find_first_zero_bit(sqp->in_use_bm, sdebug_max_queue);
+ if (unlikely(k >= sdebug_max_queue)) {
+ spin_unlock_irqrestore(&sqp->qc_lock, iflags);
if (scsi_result)
goto respond_in_thread;
- else if (SCSI_DEBUG_OPT_ALL_TSF & scsi_debug_opts)
+ else if (SDEBUG_OPT_ALL_TSF & sdebug_opts)
scsi_result = device_qfull_result;
- if (SCSI_DEBUG_OPT_Q_NOISE & scsi_debug_opts)
+ if (SDEBUG_OPT_Q_NOISE & sdebug_opts)
sdev_printk(KERN_INFO, sdp,
"%s: max_queue=%d exceeded, %s\n",
- __func__, scsi_debug_max_queue,
+ __func__, sdebug_max_queue,
(scsi_result ? "status: TASK SET FULL" :
"report: host busy"));
if (scsi_result)
@@ -4002,55 +4026,56 @@
else
return SCSI_MLQUEUE_HOST_BUSY;
}
- __set_bit(k, queued_in_use_bm);
+ __set_bit(k, sqp->in_use_bm);
atomic_inc(&devip->num_in_q);
- sqcp = &queued_arr[k];
+ sqcp = &sqp->qc_arr[k];
sqcp->a_cmnd = cmnd;
+ cmnd->host_scribble = (unsigned char *)sqcp;
cmnd->result = scsi_result;
- spin_unlock_irqrestore(&queued_arr_lock, iflags);
- if (delta_jiff > 0) {
- if (NULL == sqcp->cmnd_timerp) {
- sqcp->cmnd_timerp = kmalloc(sizeof(struct timer_list),
- GFP_ATOMIC);
- if (NULL == sqcp->cmnd_timerp)
- return SCSI_MLQUEUE_HOST_BUSY;
- init_timer(sqcp->cmnd_timerp);
- }
- sqcp->cmnd_timerp->function = sdebug_q_cmd_complete;
- sqcp->cmnd_timerp->data = k;
- sqcp->cmnd_timerp->expires = get_jiffies_64() + delta_jiff;
- add_timer(sqcp->cmnd_timerp);
- } else if (scsi_debug_ndelay > 0) {
- ktime_t kt = ktime_set(0, scsi_debug_ndelay);
- struct sdebug_hrtimer *sd_hp = sqcp->sd_hrtp;
+ sd_dp = sqcp->sd_dp;
+ spin_unlock_irqrestore(&sqp->qc_lock, iflags);
+ if (unlikely(sdebug_every_nth && sdebug_any_injecting_opt))
+ setup_inject(sqp, sqcp);
+ if (delta_jiff > 0 || sdebug_ndelay > 0) {
+ ktime_t kt;
- if (NULL == sd_hp) {
- sd_hp = kmalloc(sizeof(*sd_hp), GFP_ATOMIC);
- if (NULL == sd_hp)
+ if (delta_jiff > 0) {
+ struct timespec ts;
+
+ jiffies_to_timespec(delta_jiff, &ts);
+ kt = ktime_set(ts.tv_sec, ts.tv_nsec);
+ } else
+ kt = ktime_set(0, sdebug_ndelay);
+ if (NULL == sd_dp) {
+ sd_dp = kzalloc(sizeof(*sd_dp), GFP_ATOMIC);
+ if (NULL == sd_dp)
return SCSI_MLQUEUE_HOST_BUSY;
- sqcp->sd_hrtp = sd_hp;
- hrtimer_init(&sd_hp->hrt, CLOCK_MONOTONIC,
- HRTIMER_MODE_REL);
- sd_hp->hrt.function = sdebug_q_cmd_hrt_complete;
- sd_hp->qa_indx = k;
+ sqcp->sd_dp = sd_dp;
+ hrtimer_init(&sd_dp->hrt, CLOCK_MONOTONIC,
+ HRTIMER_MODE_REL_PINNED);
+ sd_dp->hrt.function = sdebug_q_cmd_hrt_complete;
+ sd_dp->sqa_idx = sqp - sdebug_q_arr;
+ sd_dp->qc_idx = k;
}
- hrtimer_start(&sd_hp->hrt, kt, HRTIMER_MODE_REL);
- } else { /* delay < 0 */
- if (NULL == sqcp->tletp) {
- sqcp->tletp = kmalloc(sizeof(*sqcp->tletp),
- GFP_ATOMIC);
- if (NULL == sqcp->tletp)
+ if (sdebug_statistics)
+ sd_dp->issuing_cpu = raw_smp_processor_id();
+ hrtimer_start(&sd_dp->hrt, kt, HRTIMER_MODE_REL_PINNED);
+ } else { /* jdelay < 0, use work queue */
+ if (NULL == sd_dp) {
+ sd_dp = kzalloc(sizeof(*sqcp->sd_dp), GFP_ATOMIC);
+ if (NULL == sd_dp)
return SCSI_MLQUEUE_HOST_BUSY;
- tasklet_init(sqcp->tletp,
- sdebug_q_cmd_complete, k);
+ sqcp->sd_dp = sd_dp;
+ sd_dp->sqa_idx = sqp - sdebug_q_arr;
+ sd_dp->qc_idx = k;
+ INIT_WORK(&sd_dp->ew.work, sdebug_q_cmd_wq_complete);
}
- if (-1 == delta_jiff)
- tasklet_hi_schedule(sqcp->tletp);
- else
- tasklet_schedule(sqcp->tletp);
+ if (sdebug_statistics)
+ sd_dp->issuing_cpu = raw_smp_processor_id();
+ schedule_work(&sd_dp->ew.work);
}
- if ((SCSI_DEBUG_OPT_Q_NOISE & scsi_debug_opts) &&
- (scsi_result == device_qfull_result))
+ if (unlikely((SDEBUG_OPT_Q_NOISE & sdebug_opts) &&
+ (scsi_result == device_qfull_result)))
sdev_printk(KERN_INFO, sdp,
"%s: num_in_q=%d +1, %s%s\n", __func__,
num_in_q, (inject ? "<inject> " : ""),
@@ -4069,52 +4094,55 @@
as it can when the corresponding attribute in the
/sys/bus/pseudo/drivers/scsi_debug directory is changed.
*/
-module_param_named(add_host, scsi_debug_add_host, int, S_IRUGO | S_IWUSR);
-module_param_named(ato, scsi_debug_ato, int, S_IRUGO);
-module_param_named(clustering, scsi_debug_clustering, bool, S_IRUGO | S_IWUSR);
-module_param_named(delay, scsi_debug_delay, int, S_IRUGO | S_IWUSR);
-module_param_named(dev_size_mb, scsi_debug_dev_size_mb, int, S_IRUGO);
-module_param_named(dif, scsi_debug_dif, int, S_IRUGO);
-module_param_named(dix, scsi_debug_dix, int, S_IRUGO);
-module_param_named(dsense, scsi_debug_dsense, int, S_IRUGO | S_IWUSR);
-module_param_named(every_nth, scsi_debug_every_nth, int, S_IRUGO | S_IWUSR);
-module_param_named(fake_rw, scsi_debug_fake_rw, int, S_IRUGO | S_IWUSR);
-module_param_named(guard, scsi_debug_guard, uint, S_IRUGO);
-module_param_named(host_lock, scsi_debug_host_lock, bool, S_IRUGO | S_IWUSR);
-module_param_named(lbpu, scsi_debug_lbpu, int, S_IRUGO);
-module_param_named(lbpws, scsi_debug_lbpws, int, S_IRUGO);
-module_param_named(lbpws10, scsi_debug_lbpws10, int, S_IRUGO);
-module_param_named(lbprz, scsi_debug_lbprz, int, S_IRUGO);
-module_param_named(lowest_aligned, scsi_debug_lowest_aligned, int, S_IRUGO);
-module_param_named(max_luns, scsi_debug_max_luns, int, S_IRUGO | S_IWUSR);
-module_param_named(max_queue, scsi_debug_max_queue, int, S_IRUGO | S_IWUSR);
-module_param_named(ndelay, scsi_debug_ndelay, int, S_IRUGO | S_IWUSR);
-module_param_named(no_lun_0, scsi_debug_no_lun_0, int, S_IRUGO | S_IWUSR);
-module_param_named(no_uld, scsi_debug_no_uld, int, S_IRUGO);
-module_param_named(num_parts, scsi_debug_num_parts, int, S_IRUGO);
-module_param_named(num_tgts, scsi_debug_num_tgts, int, S_IRUGO | S_IWUSR);
-module_param_named(opt_blks, scsi_debug_opt_blks, int, S_IRUGO);
-module_param_named(opts, scsi_debug_opts, int, S_IRUGO | S_IWUSR);
-module_param_named(physblk_exp, scsi_debug_physblk_exp, int, S_IRUGO);
-module_param_named(ptype, scsi_debug_ptype, int, S_IRUGO | S_IWUSR);
-module_param_named(removable, scsi_debug_removable, bool, S_IRUGO | S_IWUSR);
-module_param_named(scsi_level, scsi_debug_scsi_level, int, S_IRUGO);
-module_param_named(sector_size, scsi_debug_sector_size, int, S_IRUGO);
-module_param_named(strict, scsi_debug_strict, bool, S_IRUGO | S_IWUSR);
-module_param_named(unmap_alignment, scsi_debug_unmap_alignment, int, S_IRUGO);
-module_param_named(unmap_granularity, scsi_debug_unmap_granularity, int, S_IRUGO);
-module_param_named(unmap_max_blocks, scsi_debug_unmap_max_blocks, int, S_IRUGO);
-module_param_named(unmap_max_desc, scsi_debug_unmap_max_desc, int, S_IRUGO);
-module_param_named(virtual_gb, scsi_debug_virtual_gb, int, S_IRUGO | S_IWUSR);
-module_param_named(vpd_use_hostno, scsi_debug_vpd_use_hostno, int,
+module_param_named(add_host, sdebug_add_host, int, S_IRUGO | S_IWUSR);
+module_param_named(ato, sdebug_ato, int, S_IRUGO);
+module_param_named(clustering, sdebug_clustering, bool, S_IRUGO | S_IWUSR);
+module_param_named(delay, sdebug_jdelay, int, S_IRUGO | S_IWUSR);
+module_param_named(dev_size_mb, sdebug_dev_size_mb, int, S_IRUGO);
+module_param_named(dif, sdebug_dif, int, S_IRUGO);
+module_param_named(dix, sdebug_dix, int, S_IRUGO);
+module_param_named(dsense, sdebug_dsense, int, S_IRUGO | S_IWUSR);
+module_param_named(every_nth, sdebug_every_nth, int, S_IRUGO | S_IWUSR);
+module_param_named(fake_rw, sdebug_fake_rw, int, S_IRUGO | S_IWUSR);
+module_param_named(guard, sdebug_guard, uint, S_IRUGO);
+module_param_named(host_lock, sdebug_host_lock, bool, S_IRUGO | S_IWUSR);
+module_param_named(lbpu, sdebug_lbpu, int, S_IRUGO);
+module_param_named(lbpws, sdebug_lbpws, int, S_IRUGO);
+module_param_named(lbpws10, sdebug_lbpws10, int, S_IRUGO);
+module_param_named(lbprz, sdebug_lbprz, int, S_IRUGO);
+module_param_named(lowest_aligned, sdebug_lowest_aligned, int, S_IRUGO);
+module_param_named(max_luns, sdebug_max_luns, int, S_IRUGO | S_IWUSR);
+module_param_named(max_queue, sdebug_max_queue, int, S_IRUGO | S_IWUSR);
+module_param_named(ndelay, sdebug_ndelay, int, S_IRUGO | S_IWUSR);
+module_param_named(no_lun_0, sdebug_no_lun_0, int, S_IRUGO | S_IWUSR);
+module_param_named(no_uld, sdebug_no_uld, int, S_IRUGO);
+module_param_named(num_parts, sdebug_num_parts, int, S_IRUGO);
+module_param_named(num_tgts, sdebug_num_tgts, int, S_IRUGO | S_IWUSR);
+module_param_named(opt_blks, sdebug_opt_blks, int, S_IRUGO);
+module_param_named(opts, sdebug_opts, int, S_IRUGO | S_IWUSR);
+module_param_named(physblk_exp, sdebug_physblk_exp, int, S_IRUGO);
+module_param_named(ptype, sdebug_ptype, int, S_IRUGO | S_IWUSR);
+module_param_named(removable, sdebug_removable, bool, S_IRUGO | S_IWUSR);
+module_param_named(scsi_level, sdebug_scsi_level, int, S_IRUGO);
+module_param_named(sector_size, sdebug_sector_size, int, S_IRUGO);
+module_param_named(statistics, sdebug_statistics, bool, S_IRUGO | S_IWUSR);
+module_param_named(strict, sdebug_strict, bool, S_IRUGO | S_IWUSR);
+module_param_named(submit_queues, submit_queues, int, S_IRUGO);
+module_param_named(unmap_alignment, sdebug_unmap_alignment, int, S_IRUGO);
+module_param_named(unmap_granularity, sdebug_unmap_granularity, int, S_IRUGO);
+module_param_named(unmap_max_blocks, sdebug_unmap_max_blocks, int, S_IRUGO);
+module_param_named(unmap_max_desc, sdebug_unmap_max_desc, int, S_IRUGO);
+module_param_named(virtual_gb, sdebug_virtual_gb, int, S_IRUGO | S_IWUSR);
+module_param_named(uuid_ctl, sdebug_uuid_ctl, int, S_IRUGO);
+module_param_named(vpd_use_hostno, sdebug_vpd_use_hostno, int,
S_IRUGO | S_IWUSR);
-module_param_named(write_same_length, scsi_debug_write_same_length, int,
+module_param_named(write_same_length, sdebug_write_same_length, int,
S_IRUGO | S_IWUSR);
MODULE_AUTHOR("Eric Youngdale + Douglas Gilbert");
MODULE_DESCRIPTION("SCSI debug adapter driver");
MODULE_LICENSE("GPL");
-MODULE_VERSION(SCSI_DEBUG_VERSION);
+MODULE_VERSION(SDEBUG_VERSION);
MODULE_PARM_DESC(add_host, "0..127 hosts allowed(def=1)");
MODULE_PARM_DESC(ato, "application tag ownership: 0=disk 1=host (def=1)");
@@ -4127,11 +4155,12 @@
MODULE_PARM_DESC(every_nth, "timeout every nth command(def=0)");
MODULE_PARM_DESC(fake_rw, "fake reads/writes instead of copying (def=0)");
MODULE_PARM_DESC(guard, "protection checksum: 0=crc, 1=ip (def=0)");
-MODULE_PARM_DESC(host_lock, "use host_lock around all commands (def=0)");
+MODULE_PARM_DESC(host_lock, "host_lock is ignored (def=0)");
MODULE_PARM_DESC(lbpu, "enable LBP, support UNMAP command (def=0)");
MODULE_PARM_DESC(lbpws, "enable LBP, support WRITE SAME(16) with UNMAP bit (def=0)");
MODULE_PARM_DESC(lbpws10, "enable LBP, support WRITE SAME(10) with UNMAP bit (def=0)");
-MODULE_PARM_DESC(lbprz, "unmapped blocks return 0 on read (def=1)");
+MODULE_PARM_DESC(lbprz,
+ "on read unmapped LBs return 0 when 1 (def), return 0xff when 2");
MODULE_PARM_DESC(lowest_aligned, "lowest aligned lba (def=0)");
MODULE_PARM_DESC(max_luns, "number of LUNs per target to simulate(def=1)");
MODULE_PARM_DESC(max_queue, "max number of queued commands (1 to max(def))");
@@ -4145,30 +4174,42 @@
MODULE_PARM_DESC(physblk_exp, "physical block exponent (def=0)");
MODULE_PARM_DESC(ptype, "SCSI peripheral type(def=0[disk])");
MODULE_PARM_DESC(removable, "claim to have removable media (def=0)");
-MODULE_PARM_DESC(scsi_level, "SCSI level to simulate(def=6[SPC-4])");
+MODULE_PARM_DESC(scsi_level, "SCSI level to simulate(def=7[SPC-5])");
MODULE_PARM_DESC(sector_size, "logical block size in bytes (def=512)");
+MODULE_PARM_DESC(statistics, "collect statistics on commands, queues (def=0)");
MODULE_PARM_DESC(strict, "stricter checks: reserved field in cdb (def=0)");
+MODULE_PARM_DESC(submit_queues, "support for block multi-queue (def=1)");
MODULE_PARM_DESC(unmap_alignment, "lowest aligned thin provisioning lba (def=0)");
MODULE_PARM_DESC(unmap_granularity, "thin provisioning granularity in blocks (def=1)");
MODULE_PARM_DESC(unmap_max_blocks, "max # of blocks can be unmapped in one cmd (def=0xffffffff)");
MODULE_PARM_DESC(unmap_max_desc, "max # of ranges that can be unmapped in one cmd (def=256)");
+MODULE_PARM_DESC(uuid_ctl,
+ "1->use uuid for lu name, 0->don't, 2->all use same (def=0)");
MODULE_PARM_DESC(virtual_gb, "virtual gigabyte (GiB) size (def=0 -> use dev_size_mb)");
MODULE_PARM_DESC(vpd_use_hostno, "0 -> dev ids ignore hostno (def=1 -> unique dev ids)");
MODULE_PARM_DESC(write_same_length, "Maximum blocks per WRITE SAME cmd (def=0xffff)");
-static char sdebug_info[256];
+#define SDEBUG_INFO_LEN 256
+static char sdebug_info[SDEBUG_INFO_LEN];
static const char * scsi_debug_info(struct Scsi_Host * shp)
{
- sprintf(sdebug_info, "scsi_debug, version %s [%s], "
- "dev_size_mb=%d, opts=0x%x", SCSI_DEBUG_VERSION,
- scsi_debug_version_date, scsi_debug_dev_size_mb,
- scsi_debug_opts);
+ int k;
+
+ k = scnprintf(sdebug_info, SDEBUG_INFO_LEN, "%s: version %s [%s]\n",
+ my_name, SDEBUG_VERSION, sdebug_version_date);
+ if (k >= (SDEBUG_INFO_LEN - 1))
+ return sdebug_info;
+ scnprintf(sdebug_info + k, SDEBUG_INFO_LEN - k,
+ " dev_size_mb=%d, opts=0x%x, submit_queues=%d, %s=%d",
+ sdebug_dev_size_mb, sdebug_opts, submit_queues,
+ "statistics", (int)sdebug_statistics);
return sdebug_info;
}
/* 'echo <val> > /proc/scsi/scsi_debug/<host_id>' writes to opts */
-static int scsi_debug_write_info(struct Scsi_Host *host, char *buffer, int length)
+static int scsi_debug_write_info(struct Scsi_Host *host, char *buffer,
+ int length)
{
char arr[16];
int opts;
@@ -4180,9 +4221,11 @@
arr[minLen] = '\0';
if (1 != sscanf(arr, "%d", &opts))
return -EINVAL;
- scsi_debug_opts = opts;
- if (scsi_debug_every_nth != 0)
- atomic_set(&sdebug_cmnd_count, 0);
+ sdebug_opts = opts;
+ sdebug_verbose = !!(SDEBUG_OPT_NOISE & opts);
+ sdebug_any_injecting_opt = !!(SDEBUG_OPT_ALL_INJECTING & opts);
+ if (sdebug_every_nth != 0)
+ tweak_cmnd_count();
return length;
}
@@ -4191,69 +4234,83 @@
* output are not atomics so might be inaccurate in a busy system. */
static int scsi_debug_show_info(struct seq_file *m, struct Scsi_Host *host)
{
- int f, l;
- char b[32];
+ int f, j, l;
+ struct sdebug_queue *sqp;
- if (scsi_debug_every_nth > 0)
- snprintf(b, sizeof(b), " (curr:%d)",
- ((SCSI_DEBUG_OPT_RARE_TSF & scsi_debug_opts) ?
- atomic_read(&sdebug_a_tsf) :
- atomic_read(&sdebug_cmnd_count)));
- else
- b[0] = '\0';
+ seq_printf(m, "scsi_debug adapter driver, version %s [%s]\n",
+ SDEBUG_VERSION, sdebug_version_date);
+ seq_printf(m, "num_tgts=%d, %ssize=%d MB, opts=0x%x, every_nth=%d\n",
+ sdebug_num_tgts, "shared (ram) ", sdebug_dev_size_mb,
+ sdebug_opts, sdebug_every_nth);
+ seq_printf(m, "delay=%d, ndelay=%d, max_luns=%d, sector_size=%d %s\n",
+ sdebug_jdelay, sdebug_ndelay, sdebug_max_luns,
+ sdebug_sector_size, "bytes");
+ seq_printf(m, "cylinders=%d, heads=%d, sectors=%d, command aborts=%d\n",
+ sdebug_cylinders_per, sdebug_heads, sdebug_sectors_per,
+ num_aborts);
+ seq_printf(m, "RESETs: device=%d, target=%d, bus=%d, host=%d\n",
+ num_dev_resets, num_target_resets, num_bus_resets,
+ num_host_resets);
+ seq_printf(m, "dix_reads=%d, dix_writes=%d, dif_errors=%d\n",
+ dix_reads, dix_writes, dif_errors);
+ seq_printf(m, "usec_in_jiffy=%lu, %s=%d, mq_active=%d\n",
+ TICK_NSEC / 1000, "statistics", sdebug_statistics,
+ sdebug_mq_active);
+ seq_printf(m, "cmnd_count=%d, completions=%d, %s=%d, a_tsf=%d\n",
+ atomic_read(&sdebug_cmnd_count),
+ atomic_read(&sdebug_completions),
+ "miss_cpus", atomic_read(&sdebug_miss_cpus),
+ atomic_read(&sdebug_a_tsf));
- seq_printf(m, "scsi_debug adapter driver, version %s [%s]\n"
- "num_tgts=%d, shared (ram) size=%d MB, opts=0x%x, "
- "every_nth=%d%s\n"
- "delay=%d, ndelay=%d, max_luns=%d, q_completions=%d\n"
- "sector_size=%d bytes, cylinders=%d, heads=%d, sectors=%d\n"
- "command aborts=%d; RESETs: device=%d, target=%d, bus=%d, "
- "host=%d\ndix_reads=%d dix_writes=%d dif_errors=%d "
- "usec_in_jiffy=%lu\n",
- SCSI_DEBUG_VERSION, scsi_debug_version_date,
- scsi_debug_num_tgts, scsi_debug_dev_size_mb, scsi_debug_opts,
- scsi_debug_every_nth, b, scsi_debug_delay, scsi_debug_ndelay,
- scsi_debug_max_luns, atomic_read(&sdebug_completions),
- scsi_debug_sector_size, sdebug_cylinders_per, sdebug_heads,
- sdebug_sectors_per, num_aborts, num_dev_resets,
- num_target_resets, num_bus_resets, num_host_resets,
- dix_reads, dix_writes, dif_errors, TICK_NSEC / 1000);
-
- f = find_first_bit(queued_in_use_bm, scsi_debug_max_queue);
- if (f != scsi_debug_max_queue) {
- l = find_last_bit(queued_in_use_bm, scsi_debug_max_queue);
- seq_printf(m, " %s BUSY: first,last bits set: %d,%d\n",
- "queued_in_use_bm", f, l);
+ seq_printf(m, "submit_queues=%d\n", submit_queues);
+ for (j = 0, sqp = sdebug_q_arr; j < submit_queues; ++j, ++sqp) {
+ seq_printf(m, " queue %d:\n", j);
+ f = find_first_bit(sqp->in_use_bm, sdebug_max_queue);
+ if (f != sdebug_max_queue) {
+ l = find_last_bit(sqp->in_use_bm, sdebug_max_queue);
+ seq_printf(m, " in_use_bm BUSY: %s: %d,%d\n",
+ "first,last bits", f, l);
+ }
}
return 0;
}
static ssize_t delay_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", scsi_debug_delay);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", sdebug_jdelay);
}
-/* Returns -EBUSY if delay is being changed and commands are queued */
+/* Returns -EBUSY if jdelay is being changed and commands are queued. The unit
+ * of delay is jiffies.
+ */
static ssize_t delay_store(struct device_driver *ddp, const char *buf,
size_t count)
{
- int delay, res;
+ int jdelay, res;
- if ((count > 0) && (1 == sscanf(buf, "%d", &delay))) {
+ if (count > 0 && sscanf(buf, "%d", &jdelay) == 1) {
res = count;
- if (scsi_debug_delay != delay) {
- unsigned long iflags;
- int k;
+ if (sdebug_jdelay != jdelay) {
+ int j, k;
+ struct sdebug_queue *sqp;
- spin_lock_irqsave(&queued_arr_lock, iflags);
- k = find_first_bit(queued_in_use_bm,
- scsi_debug_max_queue);
- if (k != scsi_debug_max_queue)
- res = -EBUSY; /* have queued commands */
- else {
- scsi_debug_delay = delay;
- scsi_debug_ndelay = 0;
+ block_unblock_all_queues(true);
+ for (j = 0, sqp = sdebug_q_arr; j < submit_queues;
+ ++j, ++sqp) {
+ k = find_first_bit(sqp->in_use_bm,
+ sdebug_max_queue);
+ if (k != sdebug_max_queue) {
+ res = -EBUSY; /* queued commands */
+ break;
+ }
}
- spin_unlock_irqrestore(&queued_arr_lock, iflags);
+ if (res > 0) {
+ /* make sure sdebug_defer instances get
+ * re-allocated for new delay variant */
+ free_all_queued();
+ sdebug_jdelay = jdelay;
+ sdebug_ndelay = 0;
+ }
+ block_unblock_all_queues(false);
}
return res;
}
@@ -4263,31 +4320,41 @@
static ssize_t ndelay_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", scsi_debug_ndelay);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", sdebug_ndelay);
}
/* Returns -EBUSY if ndelay is being changed and commands are queued */
-/* If > 0 and accepted then scsi_debug_delay is set to DELAY_OVERRIDDEN */
+/* If > 0 and accepted then sdebug_jdelay is set to JDELAY_OVERRIDDEN */
static ssize_t ndelay_store(struct device_driver *ddp, const char *buf,
- size_t count)
+ size_t count)
{
- unsigned long iflags;
- int ndelay, res, k;
+ int ndelay, res;
if ((count > 0) && (1 == sscanf(buf, "%d", &ndelay)) &&
- (ndelay >= 0) && (ndelay < 1000000000)) {
+ (ndelay >= 0) && (ndelay < (1000 * 1000 * 1000))) {
res = count;
- if (scsi_debug_ndelay != ndelay) {
- spin_lock_irqsave(&queued_arr_lock, iflags);
- k = find_first_bit(queued_in_use_bm,
- scsi_debug_max_queue);
- if (k != scsi_debug_max_queue)
- res = -EBUSY; /* have queued commands */
- else {
- scsi_debug_ndelay = ndelay;
- scsi_debug_delay = ndelay ? DELAY_OVERRIDDEN
- : DEF_DELAY;
+ if (sdebug_ndelay != ndelay) {
+ int j, k;
+ struct sdebug_queue *sqp;
+
+ block_unblock_all_queues(true);
+ for (j = 0, sqp = sdebug_q_arr; j < submit_queues;
+ ++j, ++sqp) {
+ k = find_first_bit(sqp->in_use_bm,
+ sdebug_max_queue);
+ if (k != sdebug_max_queue) {
+ res = -EBUSY; /* queued commands */
+ break;
+ }
}
- spin_unlock_irqrestore(&queued_arr_lock, iflags);
+ if (res > 0) {
+ /* make sure sdebug_defer instances get
+ * re-allocated for new delay variant */
+ free_all_queued();
+ sdebug_ndelay = ndelay;
+ sdebug_jdelay = ndelay ? JDELAY_OVERRIDDEN
+ : DEF_JDELAY;
+ }
+ block_unblock_all_queues(false);
}
return res;
}
@@ -4297,7 +4364,7 @@
static ssize_t opts_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "0x%x\n", scsi_debug_opts);
+ return scnprintf(buf, PAGE_SIZE, "0x%x\n", sdebug_opts);
}
static ssize_t opts_store(struct device_driver *ddp, const char *buf,
@@ -4317,26 +4384,17 @@
}
return -EINVAL;
opts_done:
- scsi_debug_opts = opts;
- if (SCSI_DEBUG_OPT_RECOVERED_ERR & opts)
- sdebug_any_injecting_opt = true;
- else if (SCSI_DEBUG_OPT_TRANSPORT_ERR & opts)
- sdebug_any_injecting_opt = true;
- else if (SCSI_DEBUG_OPT_DIF_ERR & opts)
- sdebug_any_injecting_opt = true;
- else if (SCSI_DEBUG_OPT_DIX_ERR & opts)
- sdebug_any_injecting_opt = true;
- else if (SCSI_DEBUG_OPT_SHORT_TRANSFER & opts)
- sdebug_any_injecting_opt = true;
- atomic_set(&sdebug_cmnd_count, 0);
- atomic_set(&sdebug_a_tsf, 0);
+ sdebug_opts = opts;
+ sdebug_verbose = !!(SDEBUG_OPT_NOISE & opts);
+ sdebug_any_injecting_opt = !!(SDEBUG_OPT_ALL_INJECTING & opts);
+ tweak_cmnd_count();
return count;
}
static DRIVER_ATTR_RW(opts);
static ssize_t ptype_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", scsi_debug_ptype);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", sdebug_ptype);
}
static ssize_t ptype_store(struct device_driver *ddp, const char *buf,
size_t count)
@@ -4344,7 +4402,7 @@
int n;
if ((count > 0) && (1 == sscanf(buf, "%d", &n)) && (n >= 0)) {
- scsi_debug_ptype = n;
+ sdebug_ptype = n;
return count;
}
return -EINVAL;
@@ -4353,7 +4411,7 @@
static ssize_t dsense_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", scsi_debug_dsense);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", sdebug_dsense);
}
static ssize_t dsense_store(struct device_driver *ddp, const char *buf,
size_t count)
@@ -4361,7 +4419,7 @@
int n;
if ((count > 0) && (1 == sscanf(buf, "%d", &n)) && (n >= 0)) {
- scsi_debug_dsense = n;
+ sdebug_dsense = n;
return count;
}
return -EINVAL;
@@ -4370,7 +4428,7 @@
static ssize_t fake_rw_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", scsi_debug_fake_rw);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", sdebug_fake_rw);
}
static ssize_t fake_rw_store(struct device_driver *ddp, const char *buf,
size_t count)
@@ -4379,11 +4437,11 @@
if ((count > 0) && (1 == sscanf(buf, "%d", &n)) && (n >= 0)) {
n = (n > 0);
- scsi_debug_fake_rw = (scsi_debug_fake_rw > 0);
- if (scsi_debug_fake_rw != n) {
+ sdebug_fake_rw = (sdebug_fake_rw > 0);
+ if (sdebug_fake_rw != n) {
if ((0 == n) && (NULL == fake_storep)) {
unsigned long sz =
- (unsigned long)scsi_debug_dev_size_mb *
+ (unsigned long)sdebug_dev_size_mb *
1048576;
fake_storep = vmalloc(sz);
@@ -4393,7 +4451,7 @@
}
memset(fake_storep, 0, sz);
}
- scsi_debug_fake_rw = n;
+ sdebug_fake_rw = n;
}
return count;
}
@@ -4403,7 +4461,7 @@
static ssize_t no_lun_0_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", scsi_debug_no_lun_0);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", sdebug_no_lun_0);
}
static ssize_t no_lun_0_store(struct device_driver *ddp, const char *buf,
size_t count)
@@ -4411,7 +4469,7 @@
int n;
if ((count > 0) && (1 == sscanf(buf, "%d", &n)) && (n >= 0)) {
- scsi_debug_no_lun_0 = n;
+ sdebug_no_lun_0 = n;
return count;
}
return -EINVAL;
@@ -4420,7 +4478,7 @@
static ssize_t num_tgts_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", scsi_debug_num_tgts);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", sdebug_num_tgts);
}
static ssize_t num_tgts_store(struct device_driver *ddp, const char *buf,
size_t count)
@@ -4428,7 +4486,7 @@
int n;
if ((count > 0) && (1 == sscanf(buf, "%d", &n)) && (n >= 0)) {
- scsi_debug_num_tgts = n;
+ sdebug_num_tgts = n;
sdebug_max_tgts_luns();
return count;
}
@@ -4438,19 +4496,19 @@
static ssize_t dev_size_mb_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", scsi_debug_dev_size_mb);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", sdebug_dev_size_mb);
}
static DRIVER_ATTR_RO(dev_size_mb);
static ssize_t num_parts_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", scsi_debug_num_parts);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", sdebug_num_parts);
}
static DRIVER_ATTR_RO(num_parts);
static ssize_t every_nth_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", scsi_debug_every_nth);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", sdebug_every_nth);
}
static ssize_t every_nth_store(struct device_driver *ddp, const char *buf,
size_t count)
@@ -4458,8 +4516,12 @@
int nth;
if ((count > 0) && (1 == sscanf(buf, "%d", &nth))) {
- scsi_debug_every_nth = nth;
- atomic_set(&sdebug_cmnd_count, 0);
+ sdebug_every_nth = nth;
+ if (nth && !sdebug_statistics) {
+ pr_info("every_nth needs statistics=1, set it\n");
+ sdebug_statistics = true;
+ }
+ tweak_cmnd_count();
return count;
}
return -EINVAL;
@@ -4468,7 +4530,7 @@
static ssize_t max_luns_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", scsi_debug_max_luns);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", sdebug_max_luns);
}
static ssize_t max_luns_store(struct device_driver *ddp, const char *buf,
size_t count)
@@ -4477,10 +4539,14 @@
bool changed;
if ((count > 0) && (1 == sscanf(buf, "%d", &n)) && (n >= 0)) {
- changed = (scsi_debug_max_luns != n);
- scsi_debug_max_luns = n;
+ if (n > 256) {
+ pr_warn("max_luns can be no more than 256\n");
+ return -EINVAL;
+ }
+ changed = (sdebug_max_luns != n);
+ sdebug_max_luns = n;
sdebug_max_tgts_luns();
- if (changed && (scsi_debug_scsi_level >= 5)) { /* >= SPC-3 */
+ if (changed && (sdebug_scsi_level >= 5)) { /* >= SPC-3 */
struct sdebug_host_info *sdhp;
struct sdebug_dev_info *dp;
@@ -4503,28 +4569,34 @@
static ssize_t max_queue_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", scsi_debug_max_queue);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", sdebug_max_queue);
}
/* N.B. max_queue can be changed while there are queued commands. In flight
* commands beyond the new max_queue will be completed. */
static ssize_t max_queue_store(struct device_driver *ddp, const char *buf,
size_t count)
{
- unsigned long iflags;
- int n, k;
+ int j, n, k, a;
+ struct sdebug_queue *sqp;
if ((count > 0) && (1 == sscanf(buf, "%d", &n)) && (n > 0) &&
- (n <= SCSI_DEBUG_CANQUEUE)) {
- spin_lock_irqsave(&queued_arr_lock, iflags);
- k = find_last_bit(queued_in_use_bm, SCSI_DEBUG_CANQUEUE);
- scsi_debug_max_queue = n;
- if (SCSI_DEBUG_CANQUEUE == k)
+ (n <= SDEBUG_CANQUEUE)) {
+ block_unblock_all_queues(true);
+ k = 0;
+ for (j = 0, sqp = sdebug_q_arr; j < submit_queues;
+ ++j, ++sqp) {
+ a = find_last_bit(sqp->in_use_bm, SDEBUG_CANQUEUE);
+ if (a > k)
+ k = a;
+ }
+ sdebug_max_queue = n;
+ if (k == SDEBUG_CANQUEUE)
atomic_set(&retired_max_queue, 0);
else if (k >= n)
atomic_set(&retired_max_queue, k + 1);
else
atomic_set(&retired_max_queue, 0);
- spin_unlock_irqrestore(&queued_arr_lock, iflags);
+ block_unblock_all_queues(false);
return count;
}
return -EINVAL;
@@ -4533,19 +4605,19 @@
static ssize_t no_uld_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", scsi_debug_no_uld);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", sdebug_no_uld);
}
static DRIVER_ATTR_RO(no_uld);
static ssize_t scsi_level_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", scsi_debug_scsi_level);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", sdebug_scsi_level);
}
static DRIVER_ATTR_RO(scsi_level);
static ssize_t virtual_gb_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", scsi_debug_virtual_gb);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", sdebug_virtual_gb);
}
static ssize_t virtual_gb_store(struct device_driver *ddp, const char *buf,
size_t count)
@@ -4554,8 +4626,8 @@
bool changed;
if ((count > 0) && (1 == sscanf(buf, "%d", &n)) && (n >= 0)) {
- changed = (scsi_debug_virtual_gb != n);
- scsi_debug_virtual_gb = n;
+ changed = (sdebug_virtual_gb != n);
+ sdebug_virtual_gb = n;
sdebug_capacity = get_sdebug_capacity();
if (changed) {
struct sdebug_host_info *sdhp;
@@ -4580,9 +4652,12 @@
static ssize_t add_host_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", scsi_debug_add_host);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", sdebug_add_host);
}
+static int sdebug_add_adapter(void);
+static void sdebug_remove_adapter(void);
+
static ssize_t add_host_store(struct device_driver *ddp, const char *buf,
size_t count)
{
@@ -4605,7 +4680,7 @@
static ssize_t vpd_use_hostno_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", scsi_debug_vpd_use_hostno);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", sdebug_vpd_use_hostno);
}
static ssize_t vpd_use_hostno_store(struct device_driver *ddp, const char *buf,
size_t count)
@@ -4613,40 +4688,68 @@
int n;
if ((count > 0) && (1 == sscanf(buf, "%d", &n)) && (n >= 0)) {
- scsi_debug_vpd_use_hostno = n;
+ sdebug_vpd_use_hostno = n;
return count;
}
return -EINVAL;
}
static DRIVER_ATTR_RW(vpd_use_hostno);
+static ssize_t statistics_show(struct device_driver *ddp, char *buf)
+{
+ return scnprintf(buf, PAGE_SIZE, "%d\n", (int)sdebug_statistics);
+}
+static ssize_t statistics_store(struct device_driver *ddp, const char *buf,
+ size_t count)
+{
+ int n;
+
+ if ((count > 0) && (sscanf(buf, "%d", &n) == 1) && (n >= 0)) {
+ if (n > 0)
+ sdebug_statistics = true;
+ else {
+ clear_queue_stats();
+ sdebug_statistics = false;
+ }
+ return count;
+ }
+ return -EINVAL;
+}
+static DRIVER_ATTR_RW(statistics);
+
static ssize_t sector_size_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%u\n", scsi_debug_sector_size);
+ return scnprintf(buf, PAGE_SIZE, "%u\n", sdebug_sector_size);
}
static DRIVER_ATTR_RO(sector_size);
+static ssize_t submit_queues_show(struct device_driver *ddp, char *buf)
+{
+ return scnprintf(buf, PAGE_SIZE, "%d\n", submit_queues);
+}
+static DRIVER_ATTR_RO(submit_queues);
+
static ssize_t dix_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", scsi_debug_dix);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", sdebug_dix);
}
static DRIVER_ATTR_RO(dix);
static ssize_t dif_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", scsi_debug_dif);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", sdebug_dif);
}
static DRIVER_ATTR_RO(dif);
static ssize_t guard_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%u\n", scsi_debug_guard);
+ return scnprintf(buf, PAGE_SIZE, "%u\n", sdebug_guard);
}
static DRIVER_ATTR_RO(guard);
static ssize_t ato_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", scsi_debug_ato);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", sdebug_ato);
}
static DRIVER_ATTR_RO(ato);
@@ -4669,7 +4772,7 @@
static ssize_t removable_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", scsi_debug_removable ? 1 : 0);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", sdebug_removable ? 1 : 0);
}
static ssize_t removable_store(struct device_driver *ddp, const char *buf,
size_t count)
@@ -4677,7 +4780,7 @@
int n;
if ((count > 0) && (1 == sscanf(buf, "%d", &n)) && (n >= 0)) {
- scsi_debug_removable = (n > 0);
+ sdebug_removable = (n > 0);
return count;
}
return -EINVAL;
@@ -4686,32 +4789,17 @@
static ssize_t host_lock_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", !!scsi_debug_host_lock);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", !!sdebug_host_lock);
}
-/* Returns -EBUSY if host_lock is being changed and commands are queued */
+/* N.B. sdebug_host_lock does nothing, kept for backward compatibility */
static ssize_t host_lock_store(struct device_driver *ddp, const char *buf,
size_t count)
{
- int n, res;
+ int n;
if ((count > 0) && (1 == sscanf(buf, "%d", &n)) && (n >= 0)) {
- bool new_host_lock = (n > 0);
-
- res = count;
- if (new_host_lock != scsi_debug_host_lock) {
- unsigned long iflags;
- int k;
-
- spin_lock_irqsave(&queued_arr_lock, iflags);
- k = find_first_bit(queued_in_use_bm,
- scsi_debug_max_queue);
- if (k != scsi_debug_max_queue)
- res = -EBUSY; /* have queued commands */
- else
- scsi_debug_host_lock = new_host_lock;
- spin_unlock_irqrestore(&queued_arr_lock, iflags);
- }
- return res;
+ sdebug_host_lock = (n > 0);
+ return count;
}
return -EINVAL;
}
@@ -4719,7 +4807,7 @@
static ssize_t strict_show(struct device_driver *ddp, char *buf)
{
- return scnprintf(buf, PAGE_SIZE, "%d\n", !!scsi_debug_strict);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", !!sdebug_strict);
}
static ssize_t strict_store(struct device_driver *ddp, const char *buf,
size_t count)
@@ -4727,13 +4815,19 @@
int n;
if ((count > 0) && (1 == sscanf(buf, "%d", &n)) && (n >= 0)) {
- scsi_debug_strict = (n > 0);
+ sdebug_strict = (n > 0);
return count;
}
return -EINVAL;
}
static DRIVER_ATTR_RW(strict);
+static ssize_t uuid_ctl_show(struct device_driver *ddp, char *buf)
+{
+ return scnprintf(buf, PAGE_SIZE, "%d\n", !!sdebug_uuid_ctl);
+}
+static DRIVER_ATTR_RO(uuid_ctl);
+
/* Note: The following array creates attribute files in the
/sys/bus/pseudo/drivers/scsi_debug directory. The advantage of these
@@ -4761,6 +4855,8 @@
&driver_attr_add_host.attr,
&driver_attr_vpd_use_hostno.attr,
&driver_attr_sector_size.attr,
+ &driver_attr_statistics.attr,
+ &driver_attr_submit_queues.attr,
&driver_attr_dix.attr,
&driver_attr_dif.attr,
&driver_attr_guard.attr,
@@ -4770,6 +4866,7 @@
&driver_attr_host_lock.attr,
&driver_attr_ndelay.attr,
&driver_attr_strict.attr,
+ &driver_attr_uuid_ctl.attr,
NULL,
};
ATTRIBUTE_GROUPS(sdebug_drv);
@@ -4783,33 +4880,33 @@
int k;
int ret;
- atomic_set(&sdebug_cmnd_count, 0);
- atomic_set(&sdebug_completions, 0);
atomic_set(&retired_max_queue, 0);
- if (scsi_debug_ndelay >= 1000000000) {
+ if (sdebug_ndelay >= 1000 * 1000 * 1000) {
pr_warn("ndelay must be less than 1 second, ignored\n");
- scsi_debug_ndelay = 0;
- } else if (scsi_debug_ndelay > 0)
- scsi_debug_delay = DELAY_OVERRIDDEN;
+ sdebug_ndelay = 0;
+ } else if (sdebug_ndelay > 0)
+ sdebug_jdelay = JDELAY_OVERRIDDEN;
- switch (scsi_debug_sector_size) {
+ switch (sdebug_sector_size) {
case 512:
case 1024:
case 2048:
case 4096:
break;
default:
- pr_err("invalid sector_size %d\n", scsi_debug_sector_size);
+ pr_err("invalid sector_size %d\n", sdebug_sector_size);
return -EINVAL;
}
- switch (scsi_debug_dif) {
+ switch (sdebug_dif) {
case SD_DIF_TYPE0_PROTECTION:
+ break;
case SD_DIF_TYPE1_PROTECTION:
case SD_DIF_TYPE2_PROTECTION:
case SD_DIF_TYPE3_PROTECTION:
+ have_dif_prot = true;
break;
default:
@@ -4817,39 +4914,53 @@
return -EINVAL;
}
- if (scsi_debug_guard > 1) {
+ if (sdebug_guard > 1) {
pr_err("guard must be 0 or 1\n");
return -EINVAL;
}
- if (scsi_debug_ato > 1) {
+ if (sdebug_ato > 1) {
pr_err("ato must be 0 or 1\n");
return -EINVAL;
}
- if (scsi_debug_physblk_exp > 15) {
- pr_err("invalid physblk_exp %u\n", scsi_debug_physblk_exp);
+ if (sdebug_physblk_exp > 15) {
+ pr_err("invalid physblk_exp %u\n", sdebug_physblk_exp);
+ return -EINVAL;
+ }
+ if (sdebug_max_luns > 256) {
+ pr_warn("max_luns can be no more than 256, use default\n");
+ sdebug_max_luns = DEF_MAX_LUNS;
+ }
+
+ if (sdebug_lowest_aligned > 0x3fff) {
+ pr_err("lowest_aligned too big: %u\n", sdebug_lowest_aligned);
return -EINVAL;
}
- if (scsi_debug_lowest_aligned > 0x3fff) {
- pr_err("lowest_aligned too big: %u\n",
- scsi_debug_lowest_aligned);
+ if (submit_queues < 1) {
+ pr_err("submit_queues must be 1 or more\n");
return -EINVAL;
}
+ sdebug_q_arr = kcalloc(submit_queues, sizeof(struct sdebug_queue),
+ GFP_KERNEL);
+ if (sdebug_q_arr == NULL)
+ return -ENOMEM;
+ for (k = 0; k < submit_queues; ++k)
+ spin_lock_init(&sdebug_q_arr[k].qc_lock);
- if (scsi_debug_dev_size_mb < 1)
- scsi_debug_dev_size_mb = 1; /* force minimum 1 MB ramdisk */
- sz = (unsigned long)scsi_debug_dev_size_mb * 1048576;
- sdebug_store_sectors = sz / scsi_debug_sector_size;
+ if (sdebug_dev_size_mb < 1)
+ sdebug_dev_size_mb = 1; /* force minimum 1 MB ramdisk */
+ sz = (unsigned long)sdebug_dev_size_mb * 1048576;
+ sdebug_store_sectors = sz / sdebug_sector_size;
sdebug_capacity = get_sdebug_capacity();
/* play around with geometry, don't waste too much on track 0 */
sdebug_heads = 8;
sdebug_sectors_per = 32;
- if (scsi_debug_dev_size_mb >= 256)
+ if (sdebug_dev_size_mb >= 256)
sdebug_heads = 64;
- else if (scsi_debug_dev_size_mb >= 16)
+ else if (sdebug_dev_size_mb >= 16)
sdebug_heads = 32;
sdebug_cylinders_per = (unsigned long)sdebug_capacity /
(sdebug_sectors_per * sdebug_heads);
@@ -4861,18 +4972,19 @@
(sdebug_sectors_per * sdebug_heads);
}
- if (0 == scsi_debug_fake_rw) {
+ if (sdebug_fake_rw == 0) {
fake_storep = vmalloc(sz);
if (NULL == fake_storep) {
pr_err("out of memory, 1\n");
- return -ENOMEM;
+ ret = -ENOMEM;
+ goto free_q_arr;
}
memset(fake_storep, 0, sz);
- if (scsi_debug_num_parts > 0)
+ if (sdebug_num_parts > 0)
sdebug_build_parts(fake_storep, sz);
}
- if (scsi_debug_dix) {
+ if (sdebug_dix) {
int dif_size;
dif_size = sdebug_store_sectors * sizeof(struct sd_dif_tuple);
@@ -4891,20 +5003,21 @@
/* Logical Block Provisioning */
if (scsi_debug_lbp()) {
- scsi_debug_unmap_max_blocks =
- clamp(scsi_debug_unmap_max_blocks, 0U, 0xffffffffU);
+ sdebug_unmap_max_blocks =
+ clamp(sdebug_unmap_max_blocks, 0U, 0xffffffffU);
- scsi_debug_unmap_max_desc =
- clamp(scsi_debug_unmap_max_desc, 0U, 256U);
+ sdebug_unmap_max_desc =
+ clamp(sdebug_unmap_max_desc, 0U, 256U);
- scsi_debug_unmap_granularity =
- clamp(scsi_debug_unmap_granularity, 1U, 0xffffffffU);
+ sdebug_unmap_granularity =
+ clamp(sdebug_unmap_granularity, 1U, 0xffffffffU);
- if (scsi_debug_unmap_alignment &&
- scsi_debug_unmap_granularity <=
- scsi_debug_unmap_alignment) {
+ if (sdebug_unmap_alignment &&
+ sdebug_unmap_granularity <=
+ sdebug_unmap_alignment) {
pr_err("ERR: unmap_granularity <= unmap_alignment\n");
- return -EINVAL;
+ ret = -EINVAL;
+ goto free_vm;
}
map_size = lba_to_map_index(sdebug_store_sectors - 1) + 1;
@@ -4921,7 +5034,7 @@
bitmap_zero(map_storep, map_size);
/* Map first 1KB for partition table */
- if (scsi_debug_num_parts)
+ if (sdebug_num_parts)
map_region(0, 2);
}
@@ -4942,8 +5055,8 @@
goto bus_unreg;
}
- host_to_add = scsi_debug_add_host;
- scsi_debug_add_host = 0;
+ host_to_add = sdebug_add_host;
+ sdebug_add_host = 0;
for (k = 0; k < host_to_add; k++) {
if (sdebug_add_adapter()) {
@@ -4952,8 +5065,8 @@
}
}
- if (SCSI_DEBUG_OPT_NOISE & scsi_debug_opts)
- pr_info("built %d host(s)\n", scsi_debug_add_host);
+ if (sdebug_verbose)
+ pr_info("built %d host(s)\n", sdebug_add_host);
return 0;
@@ -4965,13 +5078,14 @@
vfree(map_storep);
vfree(dif_storep);
vfree(fake_storep);
-
+free_q_arr:
+ kfree(sdebug_q_arr);
return ret;
}
static void __exit scsi_debug_exit(void)
{
- int k = scsi_debug_add_host;
+ int k = sdebug_add_host;
stop_all_queued();
free_all_queued();
@@ -4983,6 +5097,7 @@
vfree(dif_storep);
vfree(fake_storep);
+ kfree(sdebug_q_arr);
}
device_initcall(scsi_debug_init);
@@ -5011,7 +5126,7 @@
INIT_LIST_HEAD(&sdbg_host->dev_info_list);
- devs_per_host = scsi_debug_num_tgts * scsi_debug_max_luns;
+ devs_per_host = sdebug_num_tgts * sdebug_max_luns;
for (k = 0; k < devs_per_host; k++) {
sdbg_devinfo = sdebug_device_create(sdbg_host, GFP_KERNEL);
if (!sdbg_devinfo) {
@@ -5028,14 +5143,14 @@
sdbg_host->dev.bus = &pseudo_lld_bus;
sdbg_host->dev.parent = pseudo_primary;
sdbg_host->dev.release = &sdebug_release_adapter;
- dev_set_name(&sdbg_host->dev, "adapter%d", scsi_debug_add_host);
+ dev_set_name(&sdbg_host->dev, "adapter%d", sdebug_add_host);
error = device_register(&sdbg_host->dev);
if (error)
goto clean;
- ++scsi_debug_add_host;
+ ++sdebug_add_host;
return error;
clean:
@@ -5064,78 +5179,54 @@
if (!sdbg_host)
return;
- device_unregister(&sdbg_host->dev);
- --scsi_debug_add_host;
+ device_unregister(&sdbg_host->dev);
+ --sdebug_add_host;
}
-static int
-sdebug_change_qdepth(struct scsi_device *sdev, int qdepth)
+static int sdebug_change_qdepth(struct scsi_device *sdev, int qdepth)
{
int num_in_q = 0;
- unsigned long iflags;
struct sdebug_dev_info *devip;
- spin_lock_irqsave(&queued_arr_lock, iflags);
+ block_unblock_all_queues(true);
devip = (struct sdebug_dev_info *)sdev->hostdata;
if (NULL == devip) {
- spin_unlock_irqrestore(&queued_arr_lock, iflags);
+ block_unblock_all_queues(false);
return -ENODEV;
}
num_in_q = atomic_read(&devip->num_in_q);
- spin_unlock_irqrestore(&queued_arr_lock, iflags);
if (qdepth < 1)
qdepth = 1;
- /* allow to exceed max host queued_arr elements for testing */
- if (qdepth > SCSI_DEBUG_CANQUEUE + 10)
- qdepth = SCSI_DEBUG_CANQUEUE + 10;
+ /* allow to exceed max host qc_arr elements for testing */
+ if (qdepth > SDEBUG_CANQUEUE + 10)
+ qdepth = SDEBUG_CANQUEUE + 10;
scsi_change_queue_depth(sdev, qdepth);
- if (SCSI_DEBUG_OPT_Q_NOISE & scsi_debug_opts) {
- sdev_printk(KERN_INFO, sdev,
- "%s: qdepth=%d, num_in_q=%d\n",
+ if (SDEBUG_OPT_Q_NOISE & sdebug_opts) {
+ sdev_printk(KERN_INFO, sdev, "%s: qdepth=%d, num_in_q=%d\n",
__func__, qdepth, num_in_q);
}
+ block_unblock_all_queues(false);
return sdev->queue_depth;
}
-static int
-check_inject(struct scsi_cmnd *scp)
+static bool fake_timeout(struct scsi_cmnd *scp)
{
- struct sdebug_scmd_extra_t *ep = scsi_cmd_priv(scp);
-
- memset(ep, 0, sizeof(struct sdebug_scmd_extra_t));
-
- if (atomic_inc_return(&sdebug_cmnd_count) >=
- abs(scsi_debug_every_nth)) {
- atomic_set(&sdebug_cmnd_count, 0);
- if (scsi_debug_every_nth < -1)
- scsi_debug_every_nth = -1;
- if (SCSI_DEBUG_OPT_TIMEOUT & scsi_debug_opts)
- return 1; /* ignore command causing timeout */
- else if (SCSI_DEBUG_OPT_MAC_TIMEOUT & scsi_debug_opts &&
+ if (0 == (atomic_read(&sdebug_cmnd_count) % abs(sdebug_every_nth))) {
+ if (sdebug_every_nth < -1)
+ sdebug_every_nth = -1;
+ if (SDEBUG_OPT_TIMEOUT & sdebug_opts)
+ return true; /* ignore command causing timeout */
+ else if (SDEBUG_OPT_MAC_TIMEOUT & sdebug_opts &&
scsi_medium_access_command(scp))
- return 1; /* time out reads and writes */
- if (sdebug_any_injecting_opt) {
- int opts = scsi_debug_opts;
-
- if (SCSI_DEBUG_OPT_RECOVERED_ERR & opts)
- ep->inj_recovered = true;
- else if (SCSI_DEBUG_OPT_TRANSPORT_ERR & opts)
- ep->inj_transport = true;
- else if (SCSI_DEBUG_OPT_DIF_ERR & opts)
- ep->inj_dif = true;
- else if (SCSI_DEBUG_OPT_DIX_ERR & opts)
- ep->inj_dix = true;
- else if (SCSI_DEBUG_OPT_SHORT_TRANSFER & opts)
- ep->inj_short = true;
- }
+ return true; /* time out reads and writes */
}
- return 0;
+ return false;
}
-static int
-scsi_debug_queuecommand(struct scsi_cmnd *scp)
+static int scsi_debug_queuecommand(struct Scsi_Host *shost,
+ struct scsi_cmnd *scp)
{
u8 sdeb_i;
struct scsi_device *sdp = scp->device;
@@ -5146,15 +5237,16 @@
int (*r_pfp)(struct scsi_cmnd *, struct sdebug_dev_info *);
int k, na;
int errsts = 0;
- int errsts_no_connect = DID_NO_CONNECT << 16;
u32 flags;
u16 sa;
u8 opcode = cmd[0];
bool has_wlun_rl;
- bool debug = !!(SCSI_DEBUG_OPT_NOISE & scsi_debug_opts);
scsi_set_resid(scp, 0);
- if (debug && !(SCSI_DEBUG_OPT_NO_CDB_NOISE & scsi_debug_opts)) {
+ if (sdebug_statistics)
+ atomic_inc(&sdebug_cmnd_count);
+ if (unlikely(sdebug_verbose &&
+ !(SDEBUG_OPT_NO_CDB_NOISE & sdebug_opts))) {
char b[120];
int n, len, sb;
@@ -5167,19 +5259,25 @@
n += scnprintf(b + n, sb - n, "%02x ",
(u32)cmd[k]);
}
- sdev_printk(KERN_INFO, sdp, "%s: cmd %s\n", my_name, b);
+ if (sdebug_mq_active)
+ sdev_printk(KERN_INFO, sdp, "%s: tag=%u, cmd %s\n",
+ my_name, blk_mq_unique_tag(scp->request),
+ b);
+ else
+ sdev_printk(KERN_INFO, sdp, "%s: cmd %s\n", my_name,
+ b);
}
has_wlun_rl = (sdp->lun == SCSI_W_LUN_REPORT_LUNS);
- if ((sdp->lun >= scsi_debug_max_luns) && !has_wlun_rl)
- return schedule_resp(scp, NULL, errsts_no_connect, 0);
+ if (unlikely((sdp->lun >= sdebug_max_luns) && !has_wlun_rl))
+ goto err_out;
sdeb_i = opcode_ind_arr[opcode]; /* fully mapped */
oip = &opcode_info_arr[sdeb_i]; /* safe if table consistent */
devip = (struct sdebug_dev_info *)sdp->hostdata;
- if (!devip) {
- devip = devInfoReg(sdp);
+ if (unlikely(!devip)) {
+ devip = find_build_dev_info(sdp);
if (NULL == devip)
- return schedule_resp(scp, NULL, errsts_no_connect, 0);
+ goto err_out;
}
na = oip->num_attached;
r_pfp = oip->pfp;
@@ -5211,18 +5309,18 @@
}
} /* else (when na==0) we assume the oip is a match */
flags = oip->flags;
- if (F_INV_OP & flags) {
+ if (unlikely(F_INV_OP & flags)) {
mk_sense_invalid_opcode(scp);
goto check_cond;
}
- if (has_wlun_rl && !(F_RL_WLUN_OK & flags)) {
- if (debug)
- sdev_printk(KERN_INFO, sdp, "scsi_debug: Opcode: "
- "0x%x not supported for wlun\n", opcode);
+ if (unlikely(has_wlun_rl && !(F_RL_WLUN_OK & flags))) {
+ if (sdebug_verbose)
+ sdev_printk(KERN_INFO, sdp, "%s: Opcode 0x%x not%s\n",
+ my_name, opcode, " supported for wlun");
mk_sense_invalid_opcode(scp);
goto check_cond;
}
- if (scsi_debug_strict) { /* check cdb against mask */
+ if (unlikely(sdebug_strict)) { /* check cdb against mask */
u8 rem;
int j;
@@ -5238,52 +5336,40 @@
}
}
}
- if (!(F_SKIP_UA & flags) &&
- SDEBUG_NUM_UAS != find_first_bit(devip->uas_bm, SDEBUG_NUM_UAS)) {
- errsts = check_readiness(scp, UAS_ONLY, devip);
+ if (unlikely(!(F_SKIP_UA & flags) &&
+ find_first_bit(devip->uas_bm,
+ SDEBUG_NUM_UAS) != SDEBUG_NUM_UAS)) {
+ errsts = make_ua(scp, devip);
if (errsts)
goto check_cond;
}
- if ((F_M_ACCESS & flags) && devip->stopped) {
+ if (unlikely((F_M_ACCESS & flags) && atomic_read(&devip->stopped))) {
mk_sense_buffer(scp, NOT_READY, LOGICAL_UNIT_NOT_READY, 0x2);
- if (debug)
+ if (sdebug_verbose)
sdev_printk(KERN_INFO, sdp, "%s reports: Not ready: "
"%s\n", my_name, "initializing command "
"required");
errsts = check_condition_result;
goto fini;
}
- if (scsi_debug_fake_rw && (F_FAKE_RW & flags))
+ if (sdebug_fake_rw && (F_FAKE_RW & flags))
goto fini;
- if (scsi_debug_every_nth) {
- if (check_inject(scp))
+ if (unlikely(sdebug_every_nth)) {
+ if (fake_timeout(scp))
return 0; /* ignore command: make trouble */
}
- if (oip->pfp) /* if this command has a resp_* function, call it */
- errsts = oip->pfp(scp, devip);
+ if (likely(oip->pfp))
+ errsts = oip->pfp(scp, devip); /* calls a resp_* function */
else if (r_pfp) /* if leaf function ptr NULL, try the root's */
errsts = r_pfp(scp, devip);
fini:
return schedule_resp(scp, devip, errsts,
- ((F_DELAY_OVERR & flags) ? 0 : scsi_debug_delay));
+ ((F_DELAY_OVERR & flags) ? 0 : sdebug_jdelay));
check_cond:
return schedule_resp(scp, devip, check_condition_result, 0);
-}
-
-static int
-sdebug_queuecommand_lock_or_not(struct Scsi_Host *shost, struct scsi_cmnd *cmd)
-{
- if (scsi_debug_host_lock) {
- unsigned long iflags;
- int rc;
-
- spin_lock_irqsave(shost->host_lock, iflags);
- rc = scsi_debug_queuecommand(cmd);
- spin_unlock_irqrestore(shost->host_lock, iflags);
- return rc;
- } else
- return scsi_debug_queuecommand(cmd);
+err_out:
+ return schedule_resp(scp, NULL, DID_NO_CONNECT << 16, 0);
}
static struct scsi_host_template sdebug_driver_template = {
@@ -5296,36 +5382,34 @@
.slave_configure = scsi_debug_slave_configure,
.slave_destroy = scsi_debug_slave_destroy,
.ioctl = scsi_debug_ioctl,
- .queuecommand = sdebug_queuecommand_lock_or_not,
+ .queuecommand = scsi_debug_queuecommand,
.change_queue_depth = sdebug_change_qdepth,
.eh_abort_handler = scsi_debug_abort,
.eh_device_reset_handler = scsi_debug_device_reset,
.eh_target_reset_handler = scsi_debug_target_reset,
.eh_bus_reset_handler = scsi_debug_bus_reset,
.eh_host_reset_handler = scsi_debug_host_reset,
- .can_queue = SCSI_DEBUG_CANQUEUE,
+ .can_queue = SDEBUG_CANQUEUE,
.this_id = 7,
- .sg_tablesize = SCSI_MAX_SG_CHAIN_SEGMENTS,
+ .sg_tablesize = SG_MAX_SEGMENTS,
.cmd_per_lun = DEF_CMD_PER_LUN,
.max_sectors = -1U,
.use_clustering = DISABLE_CLUSTERING,
.module = THIS_MODULE,
.track_queue_depth = 1,
- .cmd_size = sizeof(struct sdebug_scmd_extra_t),
};
static int sdebug_driver_probe(struct device * dev)
{
int error = 0;
- int opts;
struct sdebug_host_info *sdbg_host;
struct Scsi_Host *hpnt;
- int host_prot;
+ int hprot;
sdbg_host = to_sdebug_host(dev);
- sdebug_driver_template.can_queue = scsi_debug_max_queue;
- if (scsi_debug_clustering)
+ sdebug_driver_template.can_queue = sdebug_max_queue;
+ if (sdebug_clustering)
sdebug_driver_template.use_clustering = ENABLE_CLUSTERING;
hpnt = scsi_host_alloc(&sdebug_driver_template, sizeof(sdbg_host));
if (NULL == hpnt) {
@@ -5333,72 +5417,75 @@
error = -ENODEV;
return error;
}
+ if (submit_queues > nr_cpu_ids) {
+ pr_warn("%s: trim submit_queues (was %d) to nr_cpu_ids=%d\n",
+ my_name, submit_queues, nr_cpu_ids);
+ submit_queues = nr_cpu_ids;
+ }
+ /* Decide whether to tell scsi subsystem that we want mq */
+ /* Following should give the same answer for each host */
+ sdebug_mq_active = shost_use_blk_mq(hpnt) && (submit_queues > 1);
+ if (sdebug_mq_active)
+ hpnt->nr_hw_queues = submit_queues;
sdbg_host->shost = hpnt;
*((struct sdebug_host_info **)hpnt->hostdata) = sdbg_host;
- if ((hpnt->this_id >= 0) && (scsi_debug_num_tgts > hpnt->this_id))
- hpnt->max_id = scsi_debug_num_tgts + 1;
+ if ((hpnt->this_id >= 0) && (sdebug_num_tgts > hpnt->this_id))
+ hpnt->max_id = sdebug_num_tgts + 1;
else
- hpnt->max_id = scsi_debug_num_tgts;
- /* = scsi_debug_max_luns; */
+ hpnt->max_id = sdebug_num_tgts;
+ /* = sdebug_max_luns; */
hpnt->max_lun = SCSI_W_LUN_REPORT_LUNS + 1;
- host_prot = 0;
+ hprot = 0;
- switch (scsi_debug_dif) {
+ switch (sdebug_dif) {
case SD_DIF_TYPE1_PROTECTION:
- host_prot = SHOST_DIF_TYPE1_PROTECTION;
- if (scsi_debug_dix)
- host_prot |= SHOST_DIX_TYPE1_PROTECTION;
+ hprot = SHOST_DIF_TYPE1_PROTECTION;
+ if (sdebug_dix)
+ hprot |= SHOST_DIX_TYPE1_PROTECTION;
break;
case SD_DIF_TYPE2_PROTECTION:
- host_prot = SHOST_DIF_TYPE2_PROTECTION;
- if (scsi_debug_dix)
- host_prot |= SHOST_DIX_TYPE2_PROTECTION;
+ hprot = SHOST_DIF_TYPE2_PROTECTION;
+ if (sdebug_dix)
+ hprot |= SHOST_DIX_TYPE2_PROTECTION;
break;
case SD_DIF_TYPE3_PROTECTION:
- host_prot = SHOST_DIF_TYPE3_PROTECTION;
- if (scsi_debug_dix)
- host_prot |= SHOST_DIX_TYPE3_PROTECTION;
+ hprot = SHOST_DIF_TYPE3_PROTECTION;
+ if (sdebug_dix)
+ hprot |= SHOST_DIX_TYPE3_PROTECTION;
break;
default:
- if (scsi_debug_dix)
- host_prot |= SHOST_DIX_TYPE0_PROTECTION;
+ if (sdebug_dix)
+ hprot |= SHOST_DIX_TYPE0_PROTECTION;
break;
}
- scsi_host_set_prot(hpnt, host_prot);
+ scsi_host_set_prot(hpnt, hprot);
- pr_info("host protection%s%s%s%s%s%s%s\n",
- (host_prot & SHOST_DIF_TYPE1_PROTECTION) ? " DIF1" : "",
- (host_prot & SHOST_DIF_TYPE2_PROTECTION) ? " DIF2" : "",
- (host_prot & SHOST_DIF_TYPE3_PROTECTION) ? " DIF3" : "",
- (host_prot & SHOST_DIX_TYPE0_PROTECTION) ? " DIX0" : "",
- (host_prot & SHOST_DIX_TYPE1_PROTECTION) ? " DIX1" : "",
- (host_prot & SHOST_DIX_TYPE2_PROTECTION) ? " DIX2" : "",
- (host_prot & SHOST_DIX_TYPE3_PROTECTION) ? " DIX3" : "");
+ if (have_dif_prot || sdebug_dix)
+ pr_info("host protection%s%s%s%s%s%s%s\n",
+ (hprot & SHOST_DIF_TYPE1_PROTECTION) ? " DIF1" : "",
+ (hprot & SHOST_DIF_TYPE2_PROTECTION) ? " DIF2" : "",
+ (hprot & SHOST_DIF_TYPE3_PROTECTION) ? " DIF3" : "",
+ (hprot & SHOST_DIX_TYPE0_PROTECTION) ? " DIX0" : "",
+ (hprot & SHOST_DIX_TYPE1_PROTECTION) ? " DIX1" : "",
+ (hprot & SHOST_DIX_TYPE2_PROTECTION) ? " DIX2" : "",
+ (hprot & SHOST_DIX_TYPE3_PROTECTION) ? " DIX3" : "");
- if (scsi_debug_guard == 1)
+ if (sdebug_guard == 1)
scsi_host_set_guard(hpnt, SHOST_DIX_GUARD_IP);
else
scsi_host_set_guard(hpnt, SHOST_DIX_GUARD_CRC);
- opts = scsi_debug_opts;
- if (SCSI_DEBUG_OPT_RECOVERED_ERR & opts)
- sdebug_any_injecting_opt = true;
- else if (SCSI_DEBUG_OPT_TRANSPORT_ERR & opts)
- sdebug_any_injecting_opt = true;
- else if (SCSI_DEBUG_OPT_DIF_ERR & opts)
- sdebug_any_injecting_opt = true;
- else if (SCSI_DEBUG_OPT_DIX_ERR & opts)
- sdebug_any_injecting_opt = true;
- else if (SCSI_DEBUG_OPT_SHORT_TRANSFER & opts)
- sdebug_any_injecting_opt = true;
-
+ sdebug_verbose = !!(SDEBUG_OPT_NOISE & sdebug_opts);
+ sdebug_any_injecting_opt = !!(SDEBUG_OPT_ALL_INJECTING & sdebug_opts);
+ if (sdebug_every_nth) /* need stats counters for every_nth */
+ sdebug_statistics = true;
error = scsi_add_host(hpnt, &sdbg_host->dev);
if (error) {
pr_err("scsi_add_host failed\n");
diff --git a/drivers/scsi/scsi_error.c b/drivers/scsi/scsi_error.c
index 984ddcb..a8b610e 100644
--- a/drivers/scsi/scsi_error.c
+++ b/drivers/scsi/scsi_error.c
@@ -452,7 +452,7 @@
* When a deferred error is detected the current command has
* not been executed and needs retrying.
*/
-static int scsi_check_sense(struct scsi_cmnd *scmd)
+int scsi_check_sense(struct scsi_cmnd *scmd)
{
struct scsi_device *sdev = scmd->device;
struct scsi_sense_hdr sshdr;
@@ -602,6 +602,7 @@
return SUCCESS;
}
}
+EXPORT_SYMBOL_GPL(scsi_check_sense);
static void scsi_handle_queue_ramp_up(struct scsi_device *sdev)
{
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index 8106515..b2e332a 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -14,8 +14,6 @@
#include <linux/completion.h>
#include <linux/kernel.h>
#include <linux/export.h>
-#include <linux/mempool.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/pci.h>
#include <linux/delay.h>
@@ -40,39 +38,6 @@
#include "scsi_logging.h"
-#define SG_MEMPOOL_NR ARRAY_SIZE(scsi_sg_pools)
-#define SG_MEMPOOL_SIZE 2
-
-struct scsi_host_sg_pool {
- size_t size;
- char *name;
- struct kmem_cache *slab;
- mempool_t *pool;
-};
-
-#define SP(x) { .size = x, "sgpool-" __stringify(x) }
-#if (SCSI_MAX_SG_SEGMENTS < 32)
-#error SCSI_MAX_SG_SEGMENTS is too small (must be 32 or greater)
-#endif
-static struct scsi_host_sg_pool scsi_sg_pools[] = {
- SP(8),
- SP(16),
-#if (SCSI_MAX_SG_SEGMENTS > 32)
- SP(32),
-#if (SCSI_MAX_SG_SEGMENTS > 64)
- SP(64),
-#if (SCSI_MAX_SG_SEGMENTS > 128)
- SP(128),
-#if (SCSI_MAX_SG_SEGMENTS > 256)
-#error SCSI_MAX_SG_SEGMENTS is too large (256 MAX)
-#endif
-#endif
-#endif
-#endif
- SP(SCSI_MAX_SG_SEGMENTS)
-};
-#undef SP
-
struct kmem_cache *scsi_sdb_cache;
/*
@@ -553,66 +518,6 @@
scsi_run_queue(sdev->request_queue);
}
-static inline unsigned int scsi_sgtable_index(unsigned short nents)
-{
- unsigned int index;
-
- BUG_ON(nents > SCSI_MAX_SG_SEGMENTS);
-
- if (nents <= 8)
- index = 0;
- else
- index = get_count_order(nents) - 3;
-
- return index;
-}
-
-static void scsi_sg_free(struct scatterlist *sgl, unsigned int nents)
-{
- struct scsi_host_sg_pool *sgp;
-
- sgp = scsi_sg_pools + scsi_sgtable_index(nents);
- mempool_free(sgl, sgp->pool);
-}
-
-static struct scatterlist *scsi_sg_alloc(unsigned int nents, gfp_t gfp_mask)
-{
- struct scsi_host_sg_pool *sgp;
-
- sgp = scsi_sg_pools + scsi_sgtable_index(nents);
- return mempool_alloc(sgp->pool, gfp_mask);
-}
-
-static void scsi_free_sgtable(struct scsi_data_buffer *sdb, bool mq)
-{
- if (mq && sdb->table.orig_nents <= SCSI_MAX_SG_SEGMENTS)
- return;
- __sg_free_table(&sdb->table, SCSI_MAX_SG_SEGMENTS, mq, scsi_sg_free);
-}
-
-static int scsi_alloc_sgtable(struct scsi_data_buffer *sdb, int nents, bool mq)
-{
- struct scatterlist *first_chunk = NULL;
- int ret;
-
- BUG_ON(!nents);
-
- if (mq) {
- if (nents <= SCSI_MAX_SG_SEGMENTS) {
- sdb->table.nents = sdb->table.orig_nents = nents;
- sg_init_table(sdb->table.sgl, nents);
- return 0;
- }
- first_chunk = sdb->table.sgl;
- }
-
- ret = __sg_alloc_table(&sdb->table, nents, SCSI_MAX_SG_SEGMENTS,
- first_chunk, GFP_ATOMIC, scsi_sg_alloc);
- if (unlikely(ret))
- scsi_free_sgtable(sdb, mq);
- return ret;
-}
-
static void scsi_uninit_cmd(struct scsi_cmnd *cmd)
{
if (cmd->request->cmd_type == REQ_TYPE_FS) {
@@ -625,12 +530,17 @@
static void scsi_mq_free_sgtables(struct scsi_cmnd *cmd)
{
+ struct scsi_data_buffer *sdb;
+
if (cmd->sdb.table.nents)
- scsi_free_sgtable(&cmd->sdb, true);
- if (cmd->request->next_rq && cmd->request->next_rq->special)
- scsi_free_sgtable(cmd->request->next_rq->special, true);
+ sg_free_table_chained(&cmd->sdb.table, true);
+ if (cmd->request->next_rq) {
+ sdb = cmd->request->next_rq->special;
+ if (sdb)
+ sg_free_table_chained(&sdb->table, true);
+ }
if (scsi_prot_sg_count(cmd))
- scsi_free_sgtable(cmd->prot_sdb, true);
+ sg_free_table_chained(&cmd->prot_sdb->table, true);
}
static void scsi_mq_uninit_cmd(struct scsi_cmnd *cmd)
@@ -669,19 +579,19 @@
static void scsi_release_buffers(struct scsi_cmnd *cmd)
{
if (cmd->sdb.table.nents)
- scsi_free_sgtable(&cmd->sdb, false);
+ sg_free_table_chained(&cmd->sdb.table, false);
memset(&cmd->sdb, 0, sizeof(cmd->sdb));
if (scsi_prot_sg_count(cmd))
- scsi_free_sgtable(cmd->prot_sdb, false);
+ sg_free_table_chained(&cmd->prot_sdb->table, false);
}
static void scsi_release_bidi_buffers(struct scsi_cmnd *cmd)
{
struct scsi_data_buffer *bidi_sdb = cmd->request->next_rq->special;
- scsi_free_sgtable(bidi_sdb, false);
+ sg_free_table_chained(&bidi_sdb->table, false);
kmem_cache_free(scsi_sdb_cache, bidi_sdb);
cmd->request->next_rq->special = NULL;
}
@@ -1085,8 +995,8 @@
/*
* If sg table allocation fails, requeue request later.
*/
- if (unlikely(scsi_alloc_sgtable(sdb, req->nr_phys_segments,
- req->mq_ctx != NULL)))
+ if (unlikely(sg_alloc_table_chained(&sdb->table, req->nr_phys_segments,
+ sdb->table.sgl)))
return BLKPREP_DEFER;
/*
@@ -1158,7 +1068,8 @@
ivecs = blk_rq_count_integrity_sg(rq->q, rq->bio);
- if (scsi_alloc_sgtable(prot_sdb, ivecs, is_mq)) {
+ if (sg_alloc_table_chained(&prot_sdb->table, ivecs,
+ prot_sdb->table.sgl)) {
error = BLKPREP_DEFER;
goto err_exit;
}
@@ -1932,7 +1843,7 @@
if (scsi_host_get_prot(shost)) {
cmd->prot_sdb = (void *)sg +
min_t(unsigned int,
- shost->sg_tablesize, SCSI_MAX_SG_SEGMENTS) *
+ shost->sg_tablesize, SG_CHUNK_SIZE) *
sizeof(struct scatterlist);
memset(cmd->prot_sdb, 0, sizeof(struct scsi_data_buffer));
@@ -2105,7 +2016,7 @@
* this limit is imposed by hardware restrictions
*/
blk_queue_max_segments(q, min_t(unsigned short, shost->sg_tablesize,
- SCSI_MAX_SG_CHAIN_SEGMENTS));
+ SG_MAX_SEGMENTS));
if (scsi_host_prot_dma(shost)) {
shost->sg_prot_tablesize =
@@ -2187,8 +2098,8 @@
unsigned int cmd_size, sgl_size, tbl_size;
tbl_size = shost->sg_tablesize;
- if (tbl_size > SCSI_MAX_SG_SEGMENTS)
- tbl_size = SCSI_MAX_SG_SEGMENTS;
+ if (tbl_size > SG_CHUNK_SIZE)
+ tbl_size = SG_CHUNK_SIZE;
sgl_size = tbl_size * sizeof(struct scatterlist);
cmd_size = sizeof(struct scsi_cmnd) + shost->hostt->cmd_size + sgl_size;
if (scsi_host_get_prot(shost))
@@ -2264,8 +2175,6 @@
int __init scsi_init_queue(void)
{
- int i;
-
scsi_sdb_cache = kmem_cache_create("scsi_data_buffer",
sizeof(struct scsi_data_buffer),
0, 0, NULL);
@@ -2274,53 +2183,12 @@
return -ENOMEM;
}
- for (i = 0; i < SG_MEMPOOL_NR; i++) {
- struct scsi_host_sg_pool *sgp = scsi_sg_pools + i;
- int size = sgp->size * sizeof(struct scatterlist);
-
- sgp->slab = kmem_cache_create(sgp->name, size, 0,
- SLAB_HWCACHE_ALIGN, NULL);
- if (!sgp->slab) {
- printk(KERN_ERR "SCSI: can't init sg slab %s\n",
- sgp->name);
- goto cleanup_sdb;
- }
-
- sgp->pool = mempool_create_slab_pool(SG_MEMPOOL_SIZE,
- sgp->slab);
- if (!sgp->pool) {
- printk(KERN_ERR "SCSI: can't init sg mempool %s\n",
- sgp->name);
- goto cleanup_sdb;
- }
- }
-
return 0;
-
-cleanup_sdb:
- for (i = 0; i < SG_MEMPOOL_NR; i++) {
- struct scsi_host_sg_pool *sgp = scsi_sg_pools + i;
- if (sgp->pool)
- mempool_destroy(sgp->pool);
- if (sgp->slab)
- kmem_cache_destroy(sgp->slab);
- }
- kmem_cache_destroy(scsi_sdb_cache);
-
- return -ENOMEM;
}
void scsi_exit_queue(void)
{
- int i;
-
kmem_cache_destroy(scsi_sdb_cache);
-
- for (i = 0; i < SG_MEMPOOL_NR; i++) {
- struct scsi_host_sg_pool *sgp = scsi_sg_pools + i;
- mempool_destroy(sgp->pool);
- kmem_cache_destroy(sgp->slab);
- }
}
/**
@@ -3196,6 +3064,7 @@
* - EUI-64 based 12-byte
* - NAA IEEE Registered
* - NAA IEEE Extended
+ * - T10 Vendor ID
* as longer descriptors reduce the likelyhood
* of identification clashes.
*/
@@ -3214,6 +3083,21 @@
goto next_desig;
switch (d[1] & 0xf) {
+ case 0x1:
+ /* T10 Vendor ID */
+ if (cur_id_size > d[3])
+ break;
+ /* Prefer anything */
+ if (cur_id_type > 0x01 && cur_id_type != 0xff)
+ break;
+ cur_id_size = d[3];
+ if (cur_id_size + 4 > id_len)
+ cur_id_size = id_len - 4;
+ cur_id_str = d + 4;
+ cur_id_type = d[1] & 0xf;
+ id_size = snprintf(id, id_len, "t10.%*pE",
+ cur_id_size, cur_id_str);
+ break;
case 0x2:
/* EUI-64 */
if (cur_id_size > d[3])
diff --git a/drivers/scsi/scsi_priv.h b/drivers/scsi/scsi_priv.h
index 27b4d0a..57a4b99 100644
--- a/drivers/scsi/scsi_priv.h
+++ b/drivers/scsi/scsi_priv.h
@@ -116,7 +116,7 @@
extern char scsi_scan_type[];
extern int scsi_complete_async_scans(void);
extern int scsi_scan_host_selected(struct Scsi_Host *, unsigned int,
- unsigned int, u64, int);
+ unsigned int, u64, enum scsi_scan_mode);
extern void scsi_forget_host(struct Scsi_Host *);
extern void scsi_rescan_device(struct device *);
diff --git a/drivers/scsi/scsi_proc.c b/drivers/scsi/scsi_proc.c
index 251598e..7a74b82 100644
--- a/drivers/scsi/scsi_proc.c
+++ b/drivers/scsi/scsi_proc.c
@@ -251,7 +251,8 @@
if (shost->transportt->user_scan)
error = shost->transportt->user_scan(shost, channel, id, lun);
else
- error = scsi_scan_host_selected(shost, channel, id, lun, 1);
+ error = scsi_scan_host_selected(shost, channel, id, lun,
+ SCSI_SCAN_MANUAL);
scsi_host_put(shost);
return error;
}
diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
index 97074c9..e0a78f5 100644
--- a/drivers/scsi/scsi_scan.c
+++ b/drivers/scsi/scsi_scan.c
@@ -96,10 +96,13 @@
#define SCSI_SCAN_TYPE_DEFAULT "sync"
#endif
-char scsi_scan_type[6] = SCSI_SCAN_TYPE_DEFAULT;
+char scsi_scan_type[7] = SCSI_SCAN_TYPE_DEFAULT;
-module_param_string(scan, scsi_scan_type, sizeof(scsi_scan_type), S_IRUGO);
-MODULE_PARM_DESC(scan, "sync, async or none");
+module_param_string(scan, scsi_scan_type, sizeof(scsi_scan_type),
+ S_IRUGO|S_IWUSR);
+MODULE_PARM_DESC(scan, "sync, async, manual, or none. "
+ "Setting to 'manual' disables automatic scanning, but allows "
+ "for manual device scan via the 'scan' sysfs attribute.");
static unsigned int scsi_inq_timeout = SCSI_TIMEOUT/HZ + 18;
@@ -316,6 +319,7 @@
struct Scsi_Host *shost = dev_to_shost(dev->parent);
unsigned long flags;
+ BUG_ON(starget->state == STARGET_DEL);
starget->state = STARGET_DEL;
transport_destroy_device(dev);
spin_lock_irqsave(shost->host_lock, flags);
@@ -1040,7 +1044,8 @@
* @lun: LUN of target device
* @bflagsp: store bflags here if not NULL
* @sdevp: probe the LUN corresponding to this scsi_device
- * @rescan: if nonzero skip some code only needed on first scan
+ * @rescan: if not equal to SCSI_SCAN_INITIAL skip some code only
+ * needed on first scan
* @hostdata: passed to scsi_alloc_sdev()
*
* Description:
@@ -1055,7 +1060,8 @@
**/
static int scsi_probe_and_add_lun(struct scsi_target *starget,
u64 lun, int *bflagsp,
- struct scsi_device **sdevp, int rescan,
+ struct scsi_device **sdevp,
+ enum scsi_scan_mode rescan,
void *hostdata)
{
struct scsi_device *sdev;
@@ -1069,7 +1075,7 @@
*/
sdev = scsi_device_lookup_by_target(starget, lun);
if (sdev) {
- if (rescan || !scsi_device_created(sdev)) {
+ if (rescan != SCSI_SCAN_INITIAL || !scsi_device_created(sdev)) {
SCSI_LOG_SCAN_BUS(3, sdev_printk(KERN_INFO, sdev,
"scsi scan: device exists on %s\n",
dev_name(&sdev->sdev_gendev)));
@@ -1205,7 +1211,8 @@
* Modifies sdevscan->lun.
**/
static void scsi_sequential_lun_scan(struct scsi_target *starget,
- int bflags, int scsi_level, int rescan)
+ int bflags, int scsi_level,
+ enum scsi_scan_mode rescan)
{
uint max_dev_lun;
u64 sparse_lun, lun;
@@ -1300,7 +1307,7 @@
* 1: could not scan with REPORT LUN
**/
static int scsi_report_lun_scan(struct scsi_target *starget, int bflags,
- int rescan)
+ enum scsi_scan_mode rescan)
{
char devname[64];
unsigned char scsi_cmd[MAX_COMMAND_SIZE];
@@ -1546,7 +1553,7 @@
EXPORT_SYMBOL(scsi_rescan_device);
static void __scsi_scan_target(struct device *parent, unsigned int channel,
- unsigned int id, u64 lun, int rescan)
+ unsigned int id, u64 lun, enum scsi_scan_mode rescan)
{
struct Scsi_Host *shost = dev_to_shost(parent);
int bflags = 0;
@@ -1604,7 +1611,10 @@
* @channel: channel to scan
* @id: target id to scan
* @lun: Specific LUN to scan or SCAN_WILD_CARD
- * @rescan: passed to LUN scanning routines
+ * @rescan: passed to LUN scanning routines; SCSI_SCAN_INITIAL for
+ * no rescan, SCSI_SCAN_RESCAN to rescan existing LUNs,
+ * and SCSI_SCAN_MANUAL to force scanning even if
+ * 'scan=manual' is set.
*
* Description:
* Scan the target id on @parent, @channel, and @id. Scan at least LUN 0,
@@ -1614,13 +1624,17 @@
* sequential scan of LUNs on the target id.
**/
void scsi_scan_target(struct device *parent, unsigned int channel,
- unsigned int id, u64 lun, int rescan)
+ unsigned int id, u64 lun, enum scsi_scan_mode rescan)
{
struct Scsi_Host *shost = dev_to_shost(parent);
if (strncmp(scsi_scan_type, "none", 4) == 0)
return;
+ if (rescan != SCSI_SCAN_MANUAL &&
+ strncmp(scsi_scan_type, "manual", 6) == 0)
+ return;
+
mutex_lock(&shost->scan_mutex);
if (!shost->async_scan)
scsi_complete_async_scans();
@@ -1634,7 +1648,8 @@
EXPORT_SYMBOL(scsi_scan_target);
static void scsi_scan_channel(struct Scsi_Host *shost, unsigned int channel,
- unsigned int id, u64 lun, int rescan)
+ unsigned int id, u64 lun,
+ enum scsi_scan_mode rescan)
{
uint order_id;
@@ -1665,7 +1680,8 @@
}
int scsi_scan_host_selected(struct Scsi_Host *shost, unsigned int channel,
- unsigned int id, u64 lun, int rescan)
+ unsigned int id, u64 lun,
+ enum scsi_scan_mode rescan)
{
SCSI_LOG_SCAN_BUS(3, shost_printk (KERN_INFO, shost,
"%s: <%u:%u:%llu>\n",
@@ -1844,7 +1860,8 @@
{
struct async_scan_data *data;
- if (strncmp(scsi_scan_type, "none", 4) == 0)
+ if (strncmp(scsi_scan_type, "none", 4) == 0 ||
+ strncmp(scsi_scan_type, "manual", 6) == 0)
return;
if (scsi_autopm_get_host(shost) < 0)
return;
diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
index 2b642b1..0734927 100644
--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -145,7 +145,8 @@
if (shost->transportt->user_scan)
res = shost->transportt->user_scan(shost, channel, id, lun);
else
- res = scsi_scan_host_selected(shost, channel, id, lun, 1);
+ res = scsi_scan_host_selected(shost, channel, id, lun,
+ SCSI_SCAN_MANUAL);
return res;
}
@@ -1366,18 +1367,18 @@
void scsi_remove_target(struct device *dev)
{
struct Scsi_Host *shost = dev_to_shost(dev->parent);
- struct scsi_target *starget, *last_target = NULL;
+ struct scsi_target *starget;
unsigned long flags;
restart:
spin_lock_irqsave(shost->host_lock, flags);
list_for_each_entry(starget, &shost->__targets, siblings) {
if (starget->state == STARGET_DEL ||
- starget == last_target)
+ starget->state == STARGET_REMOVE)
continue;
if (starget->dev.parent == dev || &starget->dev == dev) {
kref_get(&starget->reap_ref);
- last_target = starget;
+ starget->state = STARGET_REMOVE;
spin_unlock_irqrestore(shost->host_lock, flags);
__scsi_remove_target(starget);
scsi_target_reap(starget);
diff --git a/drivers/scsi/scsi_trace.c b/drivers/scsi/scsi_trace.c
index 08bb47b..0ff083b 100644
--- a/drivers/scsi/scsi_trace.c
+++ b/drivers/scsi/scsi_trace.c
@@ -17,6 +17,7 @@
*/
#include <linux/kernel.h>
#include <linux/trace_seq.h>
+#include <asm/unaligned.h>
#include <trace/events/scsi.h>
#define SERVICE_ACTION16(cdb) (cdb[1] & 0x1f)
@@ -231,6 +232,158 @@
}
static const char *
+scsi_trace_maintenance_in(struct trace_seq *p, unsigned char *cdb, int len)
+{
+ const char *ret = trace_seq_buffer_ptr(p), *cmd;
+ u32 alloc_len;
+
+ switch (SERVICE_ACTION16(cdb)) {
+ case MI_REPORT_IDENTIFYING_INFORMATION:
+ cmd = "REPORT_IDENTIFYING_INFORMATION";
+ break;
+ case MI_REPORT_TARGET_PGS:
+ cmd = "REPORT_TARGET_PORT_GROUPS";
+ break;
+ case MI_REPORT_ALIASES:
+ cmd = "REPORT_ALIASES";
+ break;
+ case MI_REPORT_SUPPORTED_OPERATION_CODES:
+ cmd = "REPORT_SUPPORTED_OPERATION_CODES";
+ break;
+ case MI_REPORT_SUPPORTED_TASK_MANAGEMENT_FUNCTIONS:
+ cmd = "REPORT_SUPPORTED_TASK_MANAGEMENT_FUNCTIONS";
+ break;
+ case MI_REPORT_PRIORITY:
+ cmd = "REPORT_PRIORITY";
+ break;
+ case MI_REPORT_TIMESTAMP:
+ cmd = "REPORT_TIMESTAMP";
+ break;
+ case MI_MANAGEMENT_PROTOCOL_IN:
+ cmd = "MANAGEMENT_PROTOCOL_IN";
+ break;
+ default:
+ trace_seq_puts(p, "UNKNOWN");
+ goto out;
+ }
+
+ alloc_len = get_unaligned_be32(&cdb[6]);
+
+ trace_seq_printf(p, "%s alloc_len=%u", cmd, alloc_len);
+
+out:
+ trace_seq_putc(p, 0);
+
+ return ret;
+}
+
+static const char *
+scsi_trace_maintenance_out(struct trace_seq *p, unsigned char *cdb, int len)
+{
+ const char *ret = trace_seq_buffer_ptr(p), *cmd;
+ u32 alloc_len;
+
+ switch (SERVICE_ACTION16(cdb)) {
+ case MO_SET_IDENTIFYING_INFORMATION:
+ cmd = "SET_IDENTIFYING_INFORMATION";
+ break;
+ case MO_SET_TARGET_PGS:
+ cmd = "SET_TARGET_PORT_GROUPS";
+ break;
+ case MO_CHANGE_ALIASES:
+ cmd = "CHANGE_ALIASES";
+ break;
+ case MO_SET_PRIORITY:
+ cmd = "SET_PRIORITY";
+ break;
+ case MO_SET_TIMESTAMP:
+ cmd = "SET_TIMESTAMP";
+ break;
+ case MO_MANAGEMENT_PROTOCOL_OUT:
+ cmd = "MANAGEMENT_PROTOCOL_OUT";
+ break;
+ default:
+ trace_seq_puts(p, "UNKNOWN");
+ goto out;
+ }
+
+ alloc_len = get_unaligned_be32(&cdb[6]);
+
+ trace_seq_printf(p, "%s alloc_len=%u", cmd, alloc_len);
+
+out:
+ trace_seq_putc(p, 0);
+
+ return ret;
+}
+
+static const char *
+scsi_trace_zbc_in(struct trace_seq *p, unsigned char *cdb, int len)
+{
+ const char *ret = trace_seq_buffer_ptr(p), *cmd;
+ u64 zone_id;
+ u32 alloc_len;
+ u8 options;
+
+ switch (SERVICE_ACTION16(cdb)) {
+ case ZI_REPORT_ZONES:
+ cmd = "REPORT_ZONES";
+ break;
+ default:
+ trace_seq_puts(p, "UNKNOWN");
+ goto out;
+ }
+
+ zone_id = get_unaligned_be64(&cdb[2]);
+ alloc_len = get_unaligned_be32(&cdb[10]);
+ options = cdb[14] & 0x3f;
+
+ trace_seq_printf(p, "%s zone=%llu alloc_len=%u options=%u partial=%u",
+ cmd, (unsigned long long)zone_id, alloc_len,
+ options, (cdb[14] >> 7) & 1);
+
+out:
+ trace_seq_putc(p, 0);
+
+ return ret;
+}
+
+static const char *
+scsi_trace_zbc_out(struct trace_seq *p, unsigned char *cdb, int len)
+{
+ const char *ret = trace_seq_buffer_ptr(p), *cmd;
+ u64 zone_id;
+
+ switch (SERVICE_ACTION16(cdb)) {
+ case ZO_CLOSE_ZONE:
+ cmd = "CLOSE_ZONE";
+ break;
+ case ZO_FINISH_ZONE:
+ cmd = "FINISH_ZONE";
+ break;
+ case ZO_OPEN_ZONE:
+ cmd = "OPEN_ZONE";
+ break;
+ case ZO_RESET_WRITE_POINTER:
+ cmd = "RESET_WRITE_POINTER";
+ break;
+ default:
+ trace_seq_puts(p, "UNKNOWN");
+ goto out;
+ }
+
+ zone_id = get_unaligned_be64(&cdb[2]);
+
+ trace_seq_printf(p, "%s zone=%llu all=%u", cmd,
+ (unsigned long long)zone_id, cdb[14] & 1);
+
+out:
+ trace_seq_putc(p, 0);
+
+ return ret;
+}
+
+static const char *
scsi_trace_varlen(struct trace_seq *p, unsigned char *cdb, int len)
{
switch (SERVICE_ACTION32(cdb)) {
@@ -282,6 +435,14 @@
return scsi_trace_service_action_in(p, cdb, len);
case VARIABLE_LENGTH_CMD:
return scsi_trace_varlen(p, cdb, len);
+ case MAINTENANCE_IN:
+ return scsi_trace_maintenance_in(p, cdb, len);
+ case MAINTENANCE_OUT:
+ return scsi_trace_maintenance_out(p, cdb, len);
+ case ZBC_IN:
+ return scsi_trace_zbc_in(p, cdb, len);
+ case ZBC_OUT:
+ return scsi_trace_zbc_out(p, cdb, len);
default:
return scsi_trace_misc(p, cdb, len);
}
diff --git a/drivers/scsi/scsi_transport_fc.c b/drivers/scsi/scsi_transport_fc.c
index 8a88226..0f3a386 100644
--- a/drivers/scsi/scsi_transport_fc.c
+++ b/drivers/scsi/scsi_transport_fc.c
@@ -2027,11 +2027,10 @@
kfree(vport);
}
-int scsi_is_fc_vport(const struct device *dev)
+static int scsi_is_fc_vport(const struct device *dev)
{
return dev->release == fc_vport_dev_release;
}
-EXPORT_SYMBOL(scsi_is_fc_vport);
static int fc_vport_match(struct attribute_container *cont,
struct device *dev)
@@ -2110,7 +2109,8 @@
if ((channel == rport->channel) &&
(id == rport->scsi_target_id)) {
spin_unlock_irqrestore(shost->host_lock, flags);
- scsi_scan_target(&rport->dev, channel, id, lun, 1);
+ scsi_scan_target(&rport->dev, channel, id, lun,
+ SCSI_SCAN_MANUAL);
return;
}
}
@@ -3277,7 +3277,8 @@
(rport->roles & FC_PORT_ROLE_FCP_TARGET) &&
!(i->f->disable_target_scan)) {
scsi_scan_target(&rport->dev, rport->channel,
- rport->scsi_target_id, SCAN_WILD_CARD, 1);
+ rport->scsi_target_id, SCAN_WILD_CARD,
+ SCSI_SCAN_RESCAN);
}
spin_lock_irqsave(shost->host_lock, flags);
diff --git a/drivers/scsi/scsi_transport_iscsi.c b/drivers/scsi/scsi_transport_iscsi.c
index 4414816..42bca61 100644
--- a/drivers/scsi/scsi_transport_iscsi.c
+++ b/drivers/scsi/scsi_transport_iscsi.c
@@ -1009,7 +1009,7 @@
kfree(fnode_sess);
}
-struct device_type iscsi_flashnode_sess_dev_type = {
+static struct device_type iscsi_flashnode_sess_dev_type = {
.name = "iscsi_flashnode_sess_dev_type",
.groups = iscsi_flashnode_sess_attr_groups,
.release = iscsi_flashnode_sess_release,
@@ -1195,13 +1195,13 @@
kfree(fnode_conn);
}
-struct device_type iscsi_flashnode_conn_dev_type = {
+static struct device_type iscsi_flashnode_conn_dev_type = {
.name = "iscsi_flashnode_conn_dev_type",
.groups = iscsi_flashnode_conn_attr_groups,
.release = iscsi_flashnode_conn_release,
};
-struct bus_type iscsi_flashnode_bus;
+static struct bus_type iscsi_flashnode_bus;
int iscsi_flashnode_bus_match(struct device *dev,
struct device_driver *drv)
@@ -1212,7 +1212,7 @@
}
EXPORT_SYMBOL_GPL(iscsi_flashnode_bus_match);
-struct bus_type iscsi_flashnode_bus = {
+static struct bus_type iscsi_flashnode_bus = {
.name = "iscsi_flashnode",
.match = &iscsi_flashnode_bus_match,
};
@@ -1324,11 +1324,10 @@
* 1 on success
* 0 on failure
*/
-int iscsi_is_flashnode_conn_dev(struct device *dev, void *data)
+static int iscsi_is_flashnode_conn_dev(struct device *dev, void *data)
{
return dev->bus == &iscsi_flashnode_bus;
}
-EXPORT_SYMBOL_GPL(iscsi_is_flashnode_conn_dev);
static int iscsi_destroy_flashnode_conn(struct iscsi_bus_flash_conn *fnode_conn)
{
@@ -1783,6 +1782,7 @@
unsigned int channel;
unsigned int id;
u64 lun;
+ enum scsi_scan_mode rescan;
};
static int iscsi_user_scan_session(struct device *dev, void *data)
@@ -1819,7 +1819,7 @@
(scan_data->id == SCAN_WILD_CARD ||
scan_data->id == id))
scsi_scan_target(&session->dev, 0, id,
- scan_data->lun, 1);
+ scan_data->lun, scan_data->rescan);
}
user_scan_exit:
@@ -1836,6 +1836,7 @@
scan_data.channel = channel;
scan_data.id = id;
scan_data.lun = lun;
+ scan_data.rescan = SCSI_SCAN_MANUAL;
return device_for_each_child(&shost->shost_gendev, &scan_data,
iscsi_user_scan_session);
@@ -1852,6 +1853,7 @@
scan_data.channel = 0;
scan_data.id = SCAN_WILD_CARD;
scan_data.lun = SCAN_WILD_CARD;
+ scan_data.rescan = SCSI_SCAN_RESCAN;
iscsi_user_scan_session(&session->dev, &scan_data);
atomic_dec(&ihost->nr_scans);
@@ -2067,13 +2069,10 @@
int iscsi_add_session(struct iscsi_cls_session *session, unsigned int target_id)
{
- struct Scsi_Host *shost = iscsi_session_to_shost(session);
- struct iscsi_cls_host *ihost;
unsigned long flags;
int id = 0;
int err;
- ihost = shost->shost_data;
session->sid = atomic_add_return(1, &iscsi_session_nr);
if (target_id == ISCSI_MAX_TARGET) {
diff --git a/drivers/scsi/scsi_transport_sas.c b/drivers/scsi/scsi_transport_sas.c
index b6f958193..3f0ff07 100644
--- a/drivers/scsi/scsi_transport_sas.c
+++ b/drivers/scsi/scsi_transport_sas.c
@@ -1614,7 +1614,8 @@
else
lun = 0;
- scsi_scan_target(&rphy->dev, 0, rphy->scsi_target_id, lun, 0);
+ scsi_scan_target(&rphy->dev, 0, rphy->scsi_target_id, lun,
+ SCSI_SCAN_INITIAL);
}
return 0;
@@ -1739,8 +1740,8 @@
if ((channel == SCAN_WILD_CARD || channel == 0) &&
(id == SCAN_WILD_CARD || id == rphy->scsi_target_id)) {
- scsi_scan_target(&rphy->dev, 0,
- rphy->scsi_target_id, lun, 1);
+ scsi_scan_target(&rphy->dev, 0, rphy->scsi_target_id,
+ lun, SCSI_SCAN_MANUAL);
}
}
mutex_unlock(&sas_host->lock);
diff --git a/drivers/scsi/sense_codes.h b/drivers/scsi/sense_codes.h
new file mode 100644
index 0000000..e4e1dcc
--- /dev/null
+++ b/drivers/scsi/sense_codes.h
@@ -0,0 +1,826 @@
+/*
+ * The canonical list of T10 Additional Sense Codes is available at:
+ * http://www.t10.org/lists/asc-num.txt [most recent: 20141221]
+ */
+
+SENSE_CODE(0x0000, "No additional sense information")
+SENSE_CODE(0x0001, "Filemark detected")
+SENSE_CODE(0x0002, "End-of-partition/medium detected")
+SENSE_CODE(0x0003, "Setmark detected")
+SENSE_CODE(0x0004, "Beginning-of-partition/medium detected")
+SENSE_CODE(0x0005, "End-of-data detected")
+SENSE_CODE(0x0006, "I/O process terminated")
+SENSE_CODE(0x0007, "Programmable early warning detected")
+SENSE_CODE(0x0011, "Audio play operation in progress")
+SENSE_CODE(0x0012, "Audio play operation paused")
+SENSE_CODE(0x0013, "Audio play operation successfully completed")
+SENSE_CODE(0x0014, "Audio play operation stopped due to error")
+SENSE_CODE(0x0015, "No current audio status to return")
+SENSE_CODE(0x0016, "Operation in progress")
+SENSE_CODE(0x0017, "Cleaning requested")
+SENSE_CODE(0x0018, "Erase operation in progress")
+SENSE_CODE(0x0019, "Locate operation in progress")
+SENSE_CODE(0x001A, "Rewind operation in progress")
+SENSE_CODE(0x001B, "Set capacity operation in progress")
+SENSE_CODE(0x001C, "Verify operation in progress")
+SENSE_CODE(0x001D, "ATA pass through information available")
+SENSE_CODE(0x001E, "Conflicting SA creation request")
+SENSE_CODE(0x001F, "Logical unit transitioning to another power condition")
+SENSE_CODE(0x0020, "Extended copy information available")
+SENSE_CODE(0x0021, "Atomic command aborted due to ACA")
+
+SENSE_CODE(0x0100, "No index/sector signal")
+
+SENSE_CODE(0x0200, "No seek complete")
+
+SENSE_CODE(0x0300, "Peripheral device write fault")
+SENSE_CODE(0x0301, "No write current")
+SENSE_CODE(0x0302, "Excessive write errors")
+
+SENSE_CODE(0x0400, "Logical unit not ready, cause not reportable")
+SENSE_CODE(0x0401, "Logical unit is in process of becoming ready")
+SENSE_CODE(0x0402, "Logical unit not ready, initializing command required")
+SENSE_CODE(0x0403, "Logical unit not ready, manual intervention required")
+SENSE_CODE(0x0404, "Logical unit not ready, format in progress")
+SENSE_CODE(0x0405, "Logical unit not ready, rebuild in progress")
+SENSE_CODE(0x0406, "Logical unit not ready, recalculation in progress")
+SENSE_CODE(0x0407, "Logical unit not ready, operation in progress")
+SENSE_CODE(0x0408, "Logical unit not ready, long write in progress")
+SENSE_CODE(0x0409, "Logical unit not ready, self-test in progress")
+SENSE_CODE(0x040A, "Logical unit not accessible, asymmetric access state transition")
+SENSE_CODE(0x040B, "Logical unit not accessible, target port in standby state")
+SENSE_CODE(0x040C, "Logical unit not accessible, target port in unavailable state")
+SENSE_CODE(0x040D, "Logical unit not ready, structure check required")
+SENSE_CODE(0x040E, "Logical unit not ready, security session in progress")
+SENSE_CODE(0x0410, "Logical unit not ready, auxiliary memory not accessible")
+SENSE_CODE(0x0411, "Logical unit not ready, notify (enable spinup) required")
+SENSE_CODE(0x0412, "Logical unit not ready, offline")
+SENSE_CODE(0x0413, "Logical unit not ready, SA creation in progress")
+SENSE_CODE(0x0414, "Logical unit not ready, space allocation in progress")
+SENSE_CODE(0x0415, "Logical unit not ready, robotics disabled")
+SENSE_CODE(0x0416, "Logical unit not ready, configuration required")
+SENSE_CODE(0x0417, "Logical unit not ready, calibration required")
+SENSE_CODE(0x0418, "Logical unit not ready, a door is open")
+SENSE_CODE(0x0419, "Logical unit not ready, operating in sequential mode")
+SENSE_CODE(0x041A, "Logical unit not ready, start stop unit command in progress")
+SENSE_CODE(0x041B, "Logical unit not ready, sanitize in progress")
+SENSE_CODE(0x041C, "Logical unit not ready, additional power use not yet granted")
+SENSE_CODE(0x041D, "Logical unit not ready, configuration in progress")
+SENSE_CODE(0x041E, "Logical unit not ready, microcode activation required")
+SENSE_CODE(0x041F, "Logical unit not ready, microcode download required")
+SENSE_CODE(0x0420, "Logical unit not ready, logical unit reset required")
+SENSE_CODE(0x0421, "Logical unit not ready, hard reset required")
+SENSE_CODE(0x0422, "Logical unit not ready, power cycle required")
+
+SENSE_CODE(0x0500, "Logical unit does not respond to selection")
+
+SENSE_CODE(0x0600, "No reference position found")
+
+SENSE_CODE(0x0700, "Multiple peripheral devices selected")
+
+SENSE_CODE(0x0800, "Logical unit communication failure")
+SENSE_CODE(0x0801, "Logical unit communication time-out")
+SENSE_CODE(0x0802, "Logical unit communication parity error")
+SENSE_CODE(0x0803, "Logical unit communication CRC error (Ultra-DMA/32)")
+SENSE_CODE(0x0804, "Unreachable copy target")
+
+SENSE_CODE(0x0900, "Track following error")
+SENSE_CODE(0x0901, "Tracking servo failure")
+SENSE_CODE(0x0902, "Focus servo failure")
+SENSE_CODE(0x0903, "Spindle servo failure")
+SENSE_CODE(0x0904, "Head select fault")
+SENSE_CODE(0x0905, "Vibration induced tracking error")
+
+SENSE_CODE(0x0A00, "Error log overflow")
+
+SENSE_CODE(0x0B00, "Warning")
+SENSE_CODE(0x0B01, "Warning - specified temperature exceeded")
+SENSE_CODE(0x0B02, "Warning - enclosure degraded")
+SENSE_CODE(0x0B03, "Warning - background self-test failed")
+SENSE_CODE(0x0B04, "Warning - background pre-scan detected medium error")
+SENSE_CODE(0x0B05, "Warning - background medium scan detected medium error")
+SENSE_CODE(0x0B06, "Warning - non-volatile cache now volatile")
+SENSE_CODE(0x0B07, "Warning - degraded power to non-volatile cache")
+SENSE_CODE(0x0B08, "Warning - power loss expected")
+SENSE_CODE(0x0B09, "Warning - device statistics notification active")
+
+SENSE_CODE(0x0C00, "Write error")
+SENSE_CODE(0x0C01, "Write error - recovered with auto reallocation")
+SENSE_CODE(0x0C02, "Write error - auto reallocation failed")
+SENSE_CODE(0x0C03, "Write error - recommend reassignment")
+SENSE_CODE(0x0C04, "Compression check miscompare error")
+SENSE_CODE(0x0C05, "Data expansion occurred during compression")
+SENSE_CODE(0x0C06, "Block not compressible")
+SENSE_CODE(0x0C07, "Write error - recovery needed")
+SENSE_CODE(0x0C08, "Write error - recovery failed")
+SENSE_CODE(0x0C09, "Write error - loss of streaming")
+SENSE_CODE(0x0C0A, "Write error - padding blocks added")
+SENSE_CODE(0x0C0B, "Auxiliary memory write error")
+SENSE_CODE(0x0C0C, "Write error - unexpected unsolicited data")
+SENSE_CODE(0x0C0D, "Write error - not enough unsolicited data")
+SENSE_CODE(0x0C0E, "Multiple write errors")
+SENSE_CODE(0x0C0F, "Defects in error window")
+SENSE_CODE(0x0C10, "Incomplete multiple atomic write operations")
+
+SENSE_CODE(0x0D00, "Error detected by third party temporary initiator")
+SENSE_CODE(0x0D01, "Third party device failure")
+SENSE_CODE(0x0D02, "Copy target device not reachable")
+SENSE_CODE(0x0D03, "Incorrect copy target device type")
+SENSE_CODE(0x0D04, "Copy target device data underrun")
+SENSE_CODE(0x0D05, "Copy target device data overrun")
+
+SENSE_CODE(0x0E00, "Invalid information unit")
+SENSE_CODE(0x0E01, "Information unit too short")
+SENSE_CODE(0x0E02, "Information unit too long")
+SENSE_CODE(0x0E03, "Invalid field in command information unit")
+
+SENSE_CODE(0x1000, "Id CRC or ECC error")
+SENSE_CODE(0x1001, "Logical block guard check failed")
+SENSE_CODE(0x1002, "Logical block application tag check failed")
+SENSE_CODE(0x1003, "Logical block reference tag check failed")
+SENSE_CODE(0x1004, "Logical block protection error on recover buffered data")
+SENSE_CODE(0x1005, "Logical block protection method error")
+
+SENSE_CODE(0x1100, "Unrecovered read error")
+SENSE_CODE(0x1101, "Read retries exhausted")
+SENSE_CODE(0x1102, "Error too long to correct")
+SENSE_CODE(0x1103, "Multiple read errors")
+SENSE_CODE(0x1104, "Unrecovered read error - auto reallocate failed")
+SENSE_CODE(0x1105, "L-EC uncorrectable error")
+SENSE_CODE(0x1106, "CIRC unrecovered error")
+SENSE_CODE(0x1107, "Data re-synchronization error")
+SENSE_CODE(0x1108, "Incomplete block read")
+SENSE_CODE(0x1109, "No gap found")
+SENSE_CODE(0x110A, "Miscorrected error")
+SENSE_CODE(0x110B, "Unrecovered read error - recommend reassignment")
+SENSE_CODE(0x110C, "Unrecovered read error - recommend rewrite the data")
+SENSE_CODE(0x110D, "De-compression CRC error")
+SENSE_CODE(0x110E, "Cannot decompress using declared algorithm")
+SENSE_CODE(0x110F, "Error reading UPC/EAN number")
+SENSE_CODE(0x1110, "Error reading ISRC number")
+SENSE_CODE(0x1111, "Read error - loss of streaming")
+SENSE_CODE(0x1112, "Auxiliary memory read error")
+SENSE_CODE(0x1113, "Read error - failed retransmission request")
+SENSE_CODE(0x1114, "Read error - lba marked bad by application client")
+SENSE_CODE(0x1115, "Write after sanitize required")
+
+SENSE_CODE(0x1200, "Address mark not found for id field")
+
+SENSE_CODE(0x1300, "Address mark not found for data field")
+
+SENSE_CODE(0x1400, "Recorded entity not found")
+SENSE_CODE(0x1401, "Record not found")
+SENSE_CODE(0x1402, "Filemark or setmark not found")
+SENSE_CODE(0x1403, "End-of-data not found")
+SENSE_CODE(0x1404, "Block sequence error")
+SENSE_CODE(0x1405, "Record not found - recommend reassignment")
+SENSE_CODE(0x1406, "Record not found - data auto-reallocated")
+SENSE_CODE(0x1407, "Locate operation failure")
+
+SENSE_CODE(0x1500, "Random positioning error")
+SENSE_CODE(0x1501, "Mechanical positioning error")
+SENSE_CODE(0x1502, "Positioning error detected by read of medium")
+
+SENSE_CODE(0x1600, "Data synchronization mark error")
+SENSE_CODE(0x1601, "Data sync error - data rewritten")
+SENSE_CODE(0x1602, "Data sync error - recommend rewrite")
+SENSE_CODE(0x1603, "Data sync error - data auto-reallocated")
+SENSE_CODE(0x1604, "Data sync error - recommend reassignment")
+
+SENSE_CODE(0x1700, "Recovered data with no error correction applied")
+SENSE_CODE(0x1701, "Recovered data with retries")
+SENSE_CODE(0x1702, "Recovered data with positive head offset")
+SENSE_CODE(0x1703, "Recovered data with negative head offset")
+SENSE_CODE(0x1704, "Recovered data with retries and/or circ applied")
+SENSE_CODE(0x1705, "Recovered data using previous sector id")
+SENSE_CODE(0x1706, "Recovered data without ECC - data auto-reallocated")
+SENSE_CODE(0x1707, "Recovered data without ECC - recommend reassignment")
+SENSE_CODE(0x1708, "Recovered data without ECC - recommend rewrite")
+SENSE_CODE(0x1709, "Recovered data without ECC - data rewritten")
+
+SENSE_CODE(0x1800, "Recovered data with error correction applied")
+SENSE_CODE(0x1801, "Recovered data with error corr. & retries applied")
+SENSE_CODE(0x1802, "Recovered data - data auto-reallocated")
+SENSE_CODE(0x1803, "Recovered data with CIRC")
+SENSE_CODE(0x1804, "Recovered data with L-EC")
+SENSE_CODE(0x1805, "Recovered data - recommend reassignment")
+SENSE_CODE(0x1806, "Recovered data - recommend rewrite")
+SENSE_CODE(0x1807, "Recovered data with ECC - data rewritten")
+SENSE_CODE(0x1808, "Recovered data with linking")
+
+SENSE_CODE(0x1900, "Defect list error")
+SENSE_CODE(0x1901, "Defect list not available")
+SENSE_CODE(0x1902, "Defect list error in primary list")
+SENSE_CODE(0x1903, "Defect list error in grown list")
+
+SENSE_CODE(0x1A00, "Parameter list length error")
+
+SENSE_CODE(0x1B00, "Synchronous data transfer error")
+
+SENSE_CODE(0x1C00, "Defect list not found")
+SENSE_CODE(0x1C01, "Primary defect list not found")
+SENSE_CODE(0x1C02, "Grown defect list not found")
+
+SENSE_CODE(0x1D00, "Miscompare during verify operation")
+SENSE_CODE(0x1D01, "Miscompare verify of unmapped LBA")
+
+SENSE_CODE(0x1E00, "Recovered id with ECC correction")
+
+SENSE_CODE(0x1F00, "Partial defect list transfer")
+
+SENSE_CODE(0x2000, "Invalid command operation code")
+SENSE_CODE(0x2001, "Access denied - initiator pending-enrolled")
+SENSE_CODE(0x2002, "Access denied - no access rights")
+SENSE_CODE(0x2003, "Access denied - invalid mgmt id key")
+SENSE_CODE(0x2004, "Illegal command while in write capable state")
+SENSE_CODE(0x2005, "Obsolete")
+SENSE_CODE(0x2006, "Illegal command while in explicit address mode")
+SENSE_CODE(0x2007, "Illegal command while in implicit address mode")
+SENSE_CODE(0x2008, "Access denied - enrollment conflict")
+SENSE_CODE(0x2009, "Access denied - invalid LU identifier")
+SENSE_CODE(0x200A, "Access denied - invalid proxy token")
+SENSE_CODE(0x200B, "Access denied - ACL LUN conflict")
+SENSE_CODE(0x200C, "Illegal command when not in append-only mode")
+
+SENSE_CODE(0x2100, "Logical block address out of range")
+SENSE_CODE(0x2101, "Invalid element address")
+SENSE_CODE(0x2102, "Invalid address for write")
+SENSE_CODE(0x2103, "Invalid write crossing layer jump")
+SENSE_CODE(0x2104, "Unaligned write command")
+SENSE_CODE(0x2105, "Write boundary violation")
+SENSE_CODE(0x2106, "Attempt to read invalid data")
+SENSE_CODE(0x2107, "Read boundary violation")
+
+SENSE_CODE(0x2200, "Illegal function (use 20 00, 24 00, or 26 00)")
+
+SENSE_CODE(0x2300, "Invalid token operation, cause not reportable")
+SENSE_CODE(0x2301, "Invalid token operation, unsupported token type")
+SENSE_CODE(0x2302, "Invalid token operation, remote token usage not supported")
+SENSE_CODE(0x2303, "Invalid token operation, remote rod token creation not supported")
+SENSE_CODE(0x2304, "Invalid token operation, token unknown")
+SENSE_CODE(0x2305, "Invalid token operation, token corrupt")
+SENSE_CODE(0x2306, "Invalid token operation, token revoked")
+SENSE_CODE(0x2307, "Invalid token operation, token expired")
+SENSE_CODE(0x2308, "Invalid token operation, token cancelled")
+SENSE_CODE(0x2309, "Invalid token operation, token deleted")
+SENSE_CODE(0x230A, "Invalid token operation, invalid token length")
+
+SENSE_CODE(0x2400, "Invalid field in cdb")
+SENSE_CODE(0x2401, "CDB decryption error")
+SENSE_CODE(0x2402, "Obsolete")
+SENSE_CODE(0x2403, "Obsolete")
+SENSE_CODE(0x2404, "Security audit value frozen")
+SENSE_CODE(0x2405, "Security working key frozen")
+SENSE_CODE(0x2406, "Nonce not unique")
+SENSE_CODE(0x2407, "Nonce timestamp out of range")
+SENSE_CODE(0x2408, "Invalid XCDB")
+
+SENSE_CODE(0x2500, "Logical unit not supported")
+
+SENSE_CODE(0x2600, "Invalid field in parameter list")
+SENSE_CODE(0x2601, "Parameter not supported")
+SENSE_CODE(0x2602, "Parameter value invalid")
+SENSE_CODE(0x2603, "Threshold parameters not supported")
+SENSE_CODE(0x2604, "Invalid release of persistent reservation")
+SENSE_CODE(0x2605, "Data decryption error")
+SENSE_CODE(0x2606, "Too many target descriptors")
+SENSE_CODE(0x2607, "Unsupported target descriptor type code")
+SENSE_CODE(0x2608, "Too many segment descriptors")
+SENSE_CODE(0x2609, "Unsupported segment descriptor type code")
+SENSE_CODE(0x260A, "Unexpected inexact segment")
+SENSE_CODE(0x260B, "Inline data length exceeded")
+SENSE_CODE(0x260C, "Invalid operation for copy source or destination")
+SENSE_CODE(0x260D, "Copy segment granularity violation")
+SENSE_CODE(0x260E, "Invalid parameter while port is enabled")
+SENSE_CODE(0x260F, "Invalid data-out buffer integrity check value")
+SENSE_CODE(0x2610, "Data decryption key fail limit reached")
+SENSE_CODE(0x2611, "Incomplete key-associated data set")
+SENSE_CODE(0x2612, "Vendor specific key reference not found")
+
+SENSE_CODE(0x2700, "Write protected")
+SENSE_CODE(0x2701, "Hardware write protected")
+SENSE_CODE(0x2702, "Logical unit software write protected")
+SENSE_CODE(0x2703, "Associated write protect")
+SENSE_CODE(0x2704, "Persistent write protect")
+SENSE_CODE(0x2705, "Permanent write protect")
+SENSE_CODE(0x2706, "Conditional write protect")
+SENSE_CODE(0x2707, "Space allocation failed write protect")
+SENSE_CODE(0x2708, "Zone is read only")
+
+SENSE_CODE(0x2800, "Not ready to ready change, medium may have changed")
+SENSE_CODE(0x2801, "Import or export element accessed")
+SENSE_CODE(0x2802, "Format-layer may have changed")
+SENSE_CODE(0x2803, "Import/export element accessed, medium changed")
+
+SENSE_CODE(0x2900, "Power on, reset, or bus device reset occurred")
+SENSE_CODE(0x2901, "Power on occurred")
+SENSE_CODE(0x2902, "Scsi bus reset occurred")
+SENSE_CODE(0x2903, "Bus device reset function occurred")
+SENSE_CODE(0x2904, "Device internal reset")
+SENSE_CODE(0x2905, "Transceiver mode changed to single-ended")
+SENSE_CODE(0x2906, "Transceiver mode changed to lvd")
+SENSE_CODE(0x2907, "I_T nexus loss occurred")
+
+SENSE_CODE(0x2A00, "Parameters changed")
+SENSE_CODE(0x2A01, "Mode parameters changed")
+SENSE_CODE(0x2A02, "Log parameters changed")
+SENSE_CODE(0x2A03, "Reservations preempted")
+SENSE_CODE(0x2A04, "Reservations released")
+SENSE_CODE(0x2A05, "Registrations preempted")
+SENSE_CODE(0x2A06, "Asymmetric access state changed")
+SENSE_CODE(0x2A07, "Implicit asymmetric access state transition failed")
+SENSE_CODE(0x2A08, "Priority changed")
+SENSE_CODE(0x2A09, "Capacity data has changed")
+SENSE_CODE(0x2A0A, "Error history I_T nexus cleared")
+SENSE_CODE(0x2A0B, "Error history snapshot released")
+SENSE_CODE(0x2A0C, "Error recovery attributes have changed")
+SENSE_CODE(0x2A0D, "Data encryption capabilities changed")
+SENSE_CODE(0x2A10, "Timestamp changed")
+SENSE_CODE(0x2A11, "Data encryption parameters changed by another i_t nexus")
+SENSE_CODE(0x2A12, "Data encryption parameters changed by vendor specific event")
+SENSE_CODE(0x2A13, "Data encryption key instance counter has changed")
+SENSE_CODE(0x2A14, "SA creation capabilities data has changed")
+SENSE_CODE(0x2A15, "Medium removal prevention preempted")
+
+SENSE_CODE(0x2B00, "Copy cannot execute since host cannot disconnect")
+
+SENSE_CODE(0x2C00, "Command sequence error")
+SENSE_CODE(0x2C01, "Too many windows specified")
+SENSE_CODE(0x2C02, "Invalid combination of windows specified")
+SENSE_CODE(0x2C03, "Current program area is not empty")
+SENSE_CODE(0x2C04, "Current program area is empty")
+SENSE_CODE(0x2C05, "Illegal power condition request")
+SENSE_CODE(0x2C06, "Persistent prevent conflict")
+SENSE_CODE(0x2C07, "Previous busy status")
+SENSE_CODE(0x2C08, "Previous task set full status")
+SENSE_CODE(0x2C09, "Previous reservation conflict status")
+SENSE_CODE(0x2C0A, "Partition or collection contains user objects")
+SENSE_CODE(0x2C0B, "Not reserved")
+SENSE_CODE(0x2C0C, "Orwrite generation does not match")
+SENSE_CODE(0x2C0D, "Reset write pointer not allowed")
+SENSE_CODE(0x2C0E, "Zone is offline")
+
+SENSE_CODE(0x2D00, "Overwrite error on update in place")
+
+SENSE_CODE(0x2E00, "Insufficient time for operation")
+SENSE_CODE(0x2E01, "Command timeout before processing")
+SENSE_CODE(0x2E02, "Command timeout during processing")
+SENSE_CODE(0x2E03, "Command timeout during processing due to error recovery")
+
+SENSE_CODE(0x2F00, "Commands cleared by another initiator")
+SENSE_CODE(0x2F01, "Commands cleared by power loss notification")
+SENSE_CODE(0x2F02, "Commands cleared by device server")
+SENSE_CODE(0x2F03, "Some commands cleared by queuing layer event")
+
+SENSE_CODE(0x3000, "Incompatible medium installed")
+SENSE_CODE(0x3001, "Cannot read medium - unknown format")
+SENSE_CODE(0x3002, "Cannot read medium - incompatible format")
+SENSE_CODE(0x3003, "Cleaning cartridge installed")
+SENSE_CODE(0x3004, "Cannot write medium - unknown format")
+SENSE_CODE(0x3005, "Cannot write medium - incompatible format")
+SENSE_CODE(0x3006, "Cannot format medium - incompatible medium")
+SENSE_CODE(0x3007, "Cleaning failure")
+SENSE_CODE(0x3008, "Cannot write - application code mismatch")
+SENSE_CODE(0x3009, "Current session not fixated for append")
+SENSE_CODE(0x300A, "Cleaning request rejected")
+SENSE_CODE(0x300C, "WORM medium - overwrite attempted")
+SENSE_CODE(0x300D, "WORM medium - integrity check")
+SENSE_CODE(0x3010, "Medium not formatted")
+SENSE_CODE(0x3011, "Incompatible volume type")
+SENSE_CODE(0x3012, "Incompatible volume qualifier")
+SENSE_CODE(0x3013, "Cleaning volume expired")
+
+SENSE_CODE(0x3100, "Medium format corrupted")
+SENSE_CODE(0x3101, "Format command failed")
+SENSE_CODE(0x3102, "Zoned formatting failed due to spare linking")
+SENSE_CODE(0x3103, "Sanitize command failed")
+
+SENSE_CODE(0x3200, "No defect spare location available")
+SENSE_CODE(0x3201, "Defect list update failure")
+
+SENSE_CODE(0x3300, "Tape length error")
+
+SENSE_CODE(0x3400, "Enclosure failure")
+
+SENSE_CODE(0x3500, "Enclosure services failure")
+SENSE_CODE(0x3501, "Unsupported enclosure function")
+SENSE_CODE(0x3502, "Enclosure services unavailable")
+SENSE_CODE(0x3503, "Enclosure services transfer failure")
+SENSE_CODE(0x3504, "Enclosure services transfer refused")
+SENSE_CODE(0x3505, "Enclosure services checksum error")
+
+SENSE_CODE(0x3600, "Ribbon, ink, or toner failure")
+
+SENSE_CODE(0x3700, "Rounded parameter")
+
+SENSE_CODE(0x3800, "Event status notification")
+SENSE_CODE(0x3802, "Esn - power management class event")
+SENSE_CODE(0x3804, "Esn - media class event")
+SENSE_CODE(0x3806, "Esn - device busy class event")
+SENSE_CODE(0x3807, "Thin Provisioning soft threshold reached")
+
+SENSE_CODE(0x3900, "Saving parameters not supported")
+
+SENSE_CODE(0x3A00, "Medium not present")
+SENSE_CODE(0x3A01, "Medium not present - tray closed")
+SENSE_CODE(0x3A02, "Medium not present - tray open")
+SENSE_CODE(0x3A03, "Medium not present - loadable")
+SENSE_CODE(0x3A04, "Medium not present - medium auxiliary memory accessible")
+
+SENSE_CODE(0x3B00, "Sequential positioning error")
+SENSE_CODE(0x3B01, "Tape position error at beginning-of-medium")
+SENSE_CODE(0x3B02, "Tape position error at end-of-medium")
+SENSE_CODE(0x3B03, "Tape or electronic vertical forms unit not ready")
+SENSE_CODE(0x3B04, "Slew failure")
+SENSE_CODE(0x3B05, "Paper jam")
+SENSE_CODE(0x3B06, "Failed to sense top-of-form")
+SENSE_CODE(0x3B07, "Failed to sense bottom-of-form")
+SENSE_CODE(0x3B08, "Reposition error")
+SENSE_CODE(0x3B09, "Read past end of medium")
+SENSE_CODE(0x3B0A, "Read past beginning of medium")
+SENSE_CODE(0x3B0B, "Position past end of medium")
+SENSE_CODE(0x3B0C, "Position past beginning of medium")
+SENSE_CODE(0x3B0D, "Medium destination element full")
+SENSE_CODE(0x3B0E, "Medium source element empty")
+SENSE_CODE(0x3B0F, "End of medium reached")
+SENSE_CODE(0x3B11, "Medium magazine not accessible")
+SENSE_CODE(0x3B12, "Medium magazine removed")
+SENSE_CODE(0x3B13, "Medium magazine inserted")
+SENSE_CODE(0x3B14, "Medium magazine locked")
+SENSE_CODE(0x3B15, "Medium magazine unlocked")
+SENSE_CODE(0x3B16, "Mechanical positioning or changer error")
+SENSE_CODE(0x3B17, "Read past end of user object")
+SENSE_CODE(0x3B18, "Element disabled")
+SENSE_CODE(0x3B19, "Element enabled")
+SENSE_CODE(0x3B1A, "Data transfer device removed")
+SENSE_CODE(0x3B1B, "Data transfer device inserted")
+SENSE_CODE(0x3B1C, "Too many logical objects on partition to support operation")
+
+SENSE_CODE(0x3D00, "Invalid bits in identify message")
+
+SENSE_CODE(0x3E00, "Logical unit has not self-configured yet")
+SENSE_CODE(0x3E01, "Logical unit failure")
+SENSE_CODE(0x3E02, "Timeout on logical unit")
+SENSE_CODE(0x3E03, "Logical unit failed self-test")
+SENSE_CODE(0x3E04, "Logical unit unable to update self-test log")
+
+SENSE_CODE(0x3F00, "Target operating conditions have changed")
+SENSE_CODE(0x3F01, "Microcode has been changed")
+SENSE_CODE(0x3F02, "Changed operating definition")
+SENSE_CODE(0x3F03, "Inquiry data has changed")
+SENSE_CODE(0x3F04, "Component device attached")
+SENSE_CODE(0x3F05, "Device identifier changed")
+SENSE_CODE(0x3F06, "Redundancy group created or modified")
+SENSE_CODE(0x3F07, "Redundancy group deleted")
+SENSE_CODE(0x3F08, "Spare created or modified")
+SENSE_CODE(0x3F09, "Spare deleted")
+SENSE_CODE(0x3F0A, "Volume set created or modified")
+SENSE_CODE(0x3F0B, "Volume set deleted")
+SENSE_CODE(0x3F0C, "Volume set deassigned")
+SENSE_CODE(0x3F0D, "Volume set reassigned")
+SENSE_CODE(0x3F0E, "Reported luns data has changed")
+SENSE_CODE(0x3F0F, "Echo buffer overwritten")
+SENSE_CODE(0x3F10, "Medium loadable")
+SENSE_CODE(0x3F11, "Medium auxiliary memory accessible")
+SENSE_CODE(0x3F12, "iSCSI IP address added")
+SENSE_CODE(0x3F13, "iSCSI IP address removed")
+SENSE_CODE(0x3F14, "iSCSI IP address changed")
+SENSE_CODE(0x3F15, "Inspect referrals sense descriptors")
+SENSE_CODE(0x3F16, "Microcode has been changed without reset")
+/*
+ * SENSE_CODE(0x40NN, "Ram failure")
+ * SENSE_CODE(0x40NN, "Diagnostic failure on component nn")
+ * SENSE_CODE(0x41NN, "Data path failure")
+ * SENSE_CODE(0x42NN, "Power-on or self-test failure")
+ */
+SENSE_CODE(0x4300, "Message error")
+
+SENSE_CODE(0x4400, "Internal target failure")
+SENSE_CODE(0x4401, "Persistent reservation information lost")
+SENSE_CODE(0x4471, "ATA device failed set features")
+
+SENSE_CODE(0x4500, "Select or reselect failure")
+
+SENSE_CODE(0x4600, "Unsuccessful soft reset")
+
+SENSE_CODE(0x4700, "Scsi parity error")
+SENSE_CODE(0x4701, "Data phase CRC error detected")
+SENSE_CODE(0x4702, "Scsi parity error detected during st data phase")
+SENSE_CODE(0x4703, "Information unit iuCRC error detected")
+SENSE_CODE(0x4704, "Asynchronous information protection error detected")
+SENSE_CODE(0x4705, "Protocol service CRC error")
+SENSE_CODE(0x4706, "Phy test function in progress")
+SENSE_CODE(0x477f, "Some commands cleared by iSCSI Protocol event")
+
+SENSE_CODE(0x4800, "Initiator detected error message received")
+
+SENSE_CODE(0x4900, "Invalid message error")
+
+SENSE_CODE(0x4A00, "Command phase error")
+
+SENSE_CODE(0x4B00, "Data phase error")
+SENSE_CODE(0x4B01, "Invalid target port transfer tag received")
+SENSE_CODE(0x4B02, "Too much write data")
+SENSE_CODE(0x4B03, "Ack/nak timeout")
+SENSE_CODE(0x4B04, "Nak received")
+SENSE_CODE(0x4B05, "Data offset error")
+SENSE_CODE(0x4B06, "Initiator response timeout")
+SENSE_CODE(0x4B07, "Connection lost")
+SENSE_CODE(0x4B08, "Data-in buffer overflow - data buffer size")
+SENSE_CODE(0x4B09, "Data-in buffer overflow - data buffer descriptor area")
+SENSE_CODE(0x4B0A, "Data-in buffer error")
+SENSE_CODE(0x4B0B, "Data-out buffer overflow - data buffer size")
+SENSE_CODE(0x4B0C, "Data-out buffer overflow - data buffer descriptor area")
+SENSE_CODE(0x4B0D, "Data-out buffer error")
+SENSE_CODE(0x4B0E, "PCIe fabric error")
+SENSE_CODE(0x4B0F, "PCIe completion timeout")
+SENSE_CODE(0x4B10, "PCIe completer abort")
+SENSE_CODE(0x4B11, "PCIe poisoned tlp received")
+SENSE_CODE(0x4B12, "PCIe eCRC check failed")
+SENSE_CODE(0x4B13, "PCIe unsupported request")
+SENSE_CODE(0x4B14, "PCIe acs violation")
+SENSE_CODE(0x4B15, "PCIe tlp prefix blocked")
+
+SENSE_CODE(0x4C00, "Logical unit failed self-configuration")
+/*
+ * SENSE_CODE(0x4DNN, "Tagged overlapped commands (nn = queue tag)")
+ */
+SENSE_CODE(0x4E00, "Overlapped commands attempted")
+
+SENSE_CODE(0x5000, "Write append error")
+SENSE_CODE(0x5001, "Write append position error")
+SENSE_CODE(0x5002, "Position error related to timing")
+
+SENSE_CODE(0x5100, "Erase failure")
+SENSE_CODE(0x5101, "Erase failure - incomplete erase operation detected")
+
+SENSE_CODE(0x5200, "Cartridge fault")
+
+SENSE_CODE(0x5300, "Media load or eject failed")
+SENSE_CODE(0x5301, "Unload tape failure")
+SENSE_CODE(0x5302, "Medium removal prevented")
+SENSE_CODE(0x5303, "Medium removal prevented by data transfer element")
+SENSE_CODE(0x5304, "Medium thread or unthread failure")
+SENSE_CODE(0x5305, "Volume identifier invalid")
+SENSE_CODE(0x5306, "Volume identifier missing")
+SENSE_CODE(0x5307, "Duplicate volume identifier")
+SENSE_CODE(0x5308, "Element status unknown")
+SENSE_CODE(0x5309, "Data transfer device error - load failed")
+SENSE_CODE(0x530a, "Data transfer device error - unload failed")
+SENSE_CODE(0x530b, "Data transfer device error - unload missing")
+SENSE_CODE(0x530c, "Data transfer device error - eject failed")
+SENSE_CODE(0x530d, "Data transfer device error - library communication failed")
+
+SENSE_CODE(0x5400, "Scsi to host system interface failure")
+
+SENSE_CODE(0x5500, "System resource failure")
+SENSE_CODE(0x5501, "System buffer full")
+SENSE_CODE(0x5502, "Insufficient reservation resources")
+SENSE_CODE(0x5503, "Insufficient resources")
+SENSE_CODE(0x5504, "Insufficient registration resources")
+SENSE_CODE(0x5505, "Insufficient access control resources")
+SENSE_CODE(0x5506, "Auxiliary memory out of space")
+SENSE_CODE(0x5507, "Quota error")
+SENSE_CODE(0x5508, "Maximum number of supplemental decryption keys exceeded")
+SENSE_CODE(0x5509, "Medium auxiliary memory not accessible")
+SENSE_CODE(0x550A, "Data currently unavailable")
+SENSE_CODE(0x550B, "Insufficient power for operation")
+SENSE_CODE(0x550C, "Insufficient resources to create rod")
+SENSE_CODE(0x550D, "Insufficient resources to create rod token")
+SENSE_CODE(0x550E, "Insufficient zone resources")
+
+SENSE_CODE(0x5700, "Unable to recover table-of-contents")
+
+SENSE_CODE(0x5800, "Generation does not exist")
+
+SENSE_CODE(0x5900, "Updated block read")
+
+SENSE_CODE(0x5A00, "Operator request or state change input")
+SENSE_CODE(0x5A01, "Operator medium removal request")
+SENSE_CODE(0x5A02, "Operator selected write protect")
+SENSE_CODE(0x5A03, "Operator selected write permit")
+
+SENSE_CODE(0x5B00, "Log exception")
+SENSE_CODE(0x5B01, "Threshold condition met")
+SENSE_CODE(0x5B02, "Log counter at maximum")
+SENSE_CODE(0x5B03, "Log list codes exhausted")
+
+SENSE_CODE(0x5C00, "Rpl status change")
+SENSE_CODE(0x5C01, "Spindles synchronized")
+SENSE_CODE(0x5C02, "Spindles not synchronized")
+
+SENSE_CODE(0x5D00, "Failure prediction threshold exceeded")
+SENSE_CODE(0x5D01, "Media failure prediction threshold exceeded")
+SENSE_CODE(0x5D02, "Logical unit failure prediction threshold exceeded")
+SENSE_CODE(0x5D03, "Spare area exhaustion prediction threshold exceeded")
+SENSE_CODE(0x5D10, "Hardware impending failure general hard drive failure")
+SENSE_CODE(0x5D11, "Hardware impending failure drive error rate too high")
+SENSE_CODE(0x5D12, "Hardware impending failure data error rate too high")
+SENSE_CODE(0x5D13, "Hardware impending failure seek error rate too high")
+SENSE_CODE(0x5D14, "Hardware impending failure too many block reassigns")
+SENSE_CODE(0x5D15, "Hardware impending failure access times too high")
+SENSE_CODE(0x5D16, "Hardware impending failure start unit times too high")
+SENSE_CODE(0x5D17, "Hardware impending failure channel parametrics")
+SENSE_CODE(0x5D18, "Hardware impending failure controller detected")
+SENSE_CODE(0x5D19, "Hardware impending failure throughput performance")
+SENSE_CODE(0x5D1A, "Hardware impending failure seek time performance")
+SENSE_CODE(0x5D1B, "Hardware impending failure spin-up retry count")
+SENSE_CODE(0x5D1C, "Hardware impending failure drive calibration retry count")
+SENSE_CODE(0x5D20, "Controller impending failure general hard drive failure")
+SENSE_CODE(0x5D21, "Controller impending failure drive error rate too high")
+SENSE_CODE(0x5D22, "Controller impending failure data error rate too high")
+SENSE_CODE(0x5D23, "Controller impending failure seek error rate too high")
+SENSE_CODE(0x5D24, "Controller impending failure too many block reassigns")
+SENSE_CODE(0x5D25, "Controller impending failure access times too high")
+SENSE_CODE(0x5D26, "Controller impending failure start unit times too high")
+SENSE_CODE(0x5D27, "Controller impending failure channel parametrics")
+SENSE_CODE(0x5D28, "Controller impending failure controller detected")
+SENSE_CODE(0x5D29, "Controller impending failure throughput performance")
+SENSE_CODE(0x5D2A, "Controller impending failure seek time performance")
+SENSE_CODE(0x5D2B, "Controller impending failure spin-up retry count")
+SENSE_CODE(0x5D2C, "Controller impending failure drive calibration retry count")
+SENSE_CODE(0x5D30, "Data channel impending failure general hard drive failure")
+SENSE_CODE(0x5D31, "Data channel impending failure drive error rate too high")
+SENSE_CODE(0x5D32, "Data channel impending failure data error rate too high")
+SENSE_CODE(0x5D33, "Data channel impending failure seek error rate too high")
+SENSE_CODE(0x5D34, "Data channel impending failure too many block reassigns")
+SENSE_CODE(0x5D35, "Data channel impending failure access times too high")
+SENSE_CODE(0x5D36, "Data channel impending failure start unit times too high")
+SENSE_CODE(0x5D37, "Data channel impending failure channel parametrics")
+SENSE_CODE(0x5D38, "Data channel impending failure controller detected")
+SENSE_CODE(0x5D39, "Data channel impending failure throughput performance")
+SENSE_CODE(0x5D3A, "Data channel impending failure seek time performance")
+SENSE_CODE(0x5D3B, "Data channel impending failure spin-up retry count")
+SENSE_CODE(0x5D3C, "Data channel impending failure drive calibration retry count")
+SENSE_CODE(0x5D40, "Servo impending failure general hard drive failure")
+SENSE_CODE(0x5D41, "Servo impending failure drive error rate too high")
+SENSE_CODE(0x5D42, "Servo impending failure data error rate too high")
+SENSE_CODE(0x5D43, "Servo impending failure seek error rate too high")
+SENSE_CODE(0x5D44, "Servo impending failure too many block reassigns")
+SENSE_CODE(0x5D45, "Servo impending failure access times too high")
+SENSE_CODE(0x5D46, "Servo impending failure start unit times too high")
+SENSE_CODE(0x5D47, "Servo impending failure channel parametrics")
+SENSE_CODE(0x5D48, "Servo impending failure controller detected")
+SENSE_CODE(0x5D49, "Servo impending failure throughput performance")
+SENSE_CODE(0x5D4A, "Servo impending failure seek time performance")
+SENSE_CODE(0x5D4B, "Servo impending failure spin-up retry count")
+SENSE_CODE(0x5D4C, "Servo impending failure drive calibration retry count")
+SENSE_CODE(0x5D50, "Spindle impending failure general hard drive failure")
+SENSE_CODE(0x5D51, "Spindle impending failure drive error rate too high")
+SENSE_CODE(0x5D52, "Spindle impending failure data error rate too high")
+SENSE_CODE(0x5D53, "Spindle impending failure seek error rate too high")
+SENSE_CODE(0x5D54, "Spindle impending failure too many block reassigns")
+SENSE_CODE(0x5D55, "Spindle impending failure access times too high")
+SENSE_CODE(0x5D56, "Spindle impending failure start unit times too high")
+SENSE_CODE(0x5D57, "Spindle impending failure channel parametrics")
+SENSE_CODE(0x5D58, "Spindle impending failure controller detected")
+SENSE_CODE(0x5D59, "Spindle impending failure throughput performance")
+SENSE_CODE(0x5D5A, "Spindle impending failure seek time performance")
+SENSE_CODE(0x5D5B, "Spindle impending failure spin-up retry count")
+SENSE_CODE(0x5D5C, "Spindle impending failure drive calibration retry count")
+SENSE_CODE(0x5D60, "Firmware impending failure general hard drive failure")
+SENSE_CODE(0x5D61, "Firmware impending failure drive error rate too high")
+SENSE_CODE(0x5D62, "Firmware impending failure data error rate too high")
+SENSE_CODE(0x5D63, "Firmware impending failure seek error rate too high")
+SENSE_CODE(0x5D64, "Firmware impending failure too many block reassigns")
+SENSE_CODE(0x5D65, "Firmware impending failure access times too high")
+SENSE_CODE(0x5D66, "Firmware impending failure start unit times too high")
+SENSE_CODE(0x5D67, "Firmware impending failure channel parametrics")
+SENSE_CODE(0x5D68, "Firmware impending failure controller detected")
+SENSE_CODE(0x5D69, "Firmware impending failure throughput performance")
+SENSE_CODE(0x5D6A, "Firmware impending failure seek time performance")
+SENSE_CODE(0x5D6B, "Firmware impending failure spin-up retry count")
+SENSE_CODE(0x5D6C, "Firmware impending failure drive calibration retry count")
+SENSE_CODE(0x5DFF, "Failure prediction threshold exceeded (false)")
+
+SENSE_CODE(0x5E00, "Low power condition on")
+SENSE_CODE(0x5E01, "Idle condition activated by timer")
+SENSE_CODE(0x5E02, "Standby condition activated by timer")
+SENSE_CODE(0x5E03, "Idle condition activated by command")
+SENSE_CODE(0x5E04, "Standby condition activated by command")
+SENSE_CODE(0x5E05, "Idle_b condition activated by timer")
+SENSE_CODE(0x5E06, "Idle_b condition activated by command")
+SENSE_CODE(0x5E07, "Idle_c condition activated by timer")
+SENSE_CODE(0x5E08, "Idle_c condition activated by command")
+SENSE_CODE(0x5E09, "Standby_y condition activated by timer")
+SENSE_CODE(0x5E0A, "Standby_y condition activated by command")
+SENSE_CODE(0x5E41, "Power state change to active")
+SENSE_CODE(0x5E42, "Power state change to idle")
+SENSE_CODE(0x5E43, "Power state change to standby")
+SENSE_CODE(0x5E45, "Power state change to sleep")
+SENSE_CODE(0x5E47, "Power state change to device control")
+
+SENSE_CODE(0x6000, "Lamp failure")
+
+SENSE_CODE(0x6100, "Video acquisition error")
+SENSE_CODE(0x6101, "Unable to acquire video")
+SENSE_CODE(0x6102, "Out of focus")
+
+SENSE_CODE(0x6200, "Scan head positioning error")
+
+SENSE_CODE(0x6300, "End of user area encountered on this track")
+SENSE_CODE(0x6301, "Packet does not fit in available space")
+
+SENSE_CODE(0x6400, "Illegal mode for this track")
+SENSE_CODE(0x6401, "Invalid packet size")
+
+SENSE_CODE(0x6500, "Voltage fault")
+
+SENSE_CODE(0x6600, "Automatic document feeder cover up")
+SENSE_CODE(0x6601, "Automatic document feeder lift up")
+SENSE_CODE(0x6602, "Document jam in automatic document feeder")
+SENSE_CODE(0x6603, "Document miss feed automatic in document feeder")
+
+SENSE_CODE(0x6700, "Configuration failure")
+SENSE_CODE(0x6701, "Configuration of incapable logical units failed")
+SENSE_CODE(0x6702, "Add logical unit failed")
+SENSE_CODE(0x6703, "Modification of logical unit failed")
+SENSE_CODE(0x6704, "Exchange of logical unit failed")
+SENSE_CODE(0x6705, "Remove of logical unit failed")
+SENSE_CODE(0x6706, "Attachment of logical unit failed")
+SENSE_CODE(0x6707, "Creation of logical unit failed")
+SENSE_CODE(0x6708, "Assign failure occurred")
+SENSE_CODE(0x6709, "Multiply assigned logical unit")
+SENSE_CODE(0x670A, "Set target port groups command failed")
+SENSE_CODE(0x670B, "ATA device feature not enabled")
+
+SENSE_CODE(0x6800, "Logical unit not configured")
+SENSE_CODE(0x6801, "Subsidiary logical unit not configured")
+
+SENSE_CODE(0x6900, "Data loss on logical unit")
+SENSE_CODE(0x6901, "Multiple logical unit failures")
+SENSE_CODE(0x6902, "Parity/data mismatch")
+
+SENSE_CODE(0x6A00, "Informational, refer to log")
+
+SENSE_CODE(0x6B00, "State change has occurred")
+SENSE_CODE(0x6B01, "Redundancy level got better")
+SENSE_CODE(0x6B02, "Redundancy level got worse")
+
+SENSE_CODE(0x6C00, "Rebuild failure occurred")
+
+SENSE_CODE(0x6D00, "Recalculate failure occurred")
+
+SENSE_CODE(0x6E00, "Command to logical unit failed")
+
+SENSE_CODE(0x6F00, "Copy protection key exchange failure - authentication failure")
+SENSE_CODE(0x6F01, "Copy protection key exchange failure - key not present")
+SENSE_CODE(0x6F02, "Copy protection key exchange failure - key not established")
+SENSE_CODE(0x6F03, "Read of scrambled sector without authentication")
+SENSE_CODE(0x6F04, "Media region code is mismatched to logical unit region")
+SENSE_CODE(0x6F05, "Drive region must be permanent/region reset count error")
+SENSE_CODE(0x6F06, "Insufficient block count for binding nonce recording")
+SENSE_CODE(0x6F07, "Conflict in binding nonce recording")
+/*
+ * SENSE_CODE(0x70NN, "Decompression exception short algorithm id of nn")
+ */
+SENSE_CODE(0x7100, "Decompression exception long algorithm id")
+
+SENSE_CODE(0x7200, "Session fixation error")
+SENSE_CODE(0x7201, "Session fixation error writing lead-in")
+SENSE_CODE(0x7202, "Session fixation error writing lead-out")
+SENSE_CODE(0x7203, "Session fixation error - incomplete track in session")
+SENSE_CODE(0x7204, "Empty or partially written reserved track")
+SENSE_CODE(0x7205, "No more track reservations allowed")
+SENSE_CODE(0x7206, "RMZ extension is not allowed")
+SENSE_CODE(0x7207, "No more test zone extensions are allowed")
+
+SENSE_CODE(0x7300, "Cd control error")
+SENSE_CODE(0x7301, "Power calibration area almost full")
+SENSE_CODE(0x7302, "Power calibration area is full")
+SENSE_CODE(0x7303, "Power calibration area error")
+SENSE_CODE(0x7304, "Program memory area update failure")
+SENSE_CODE(0x7305, "Program memory area is full")
+SENSE_CODE(0x7306, "RMA/PMA is almost full")
+SENSE_CODE(0x7310, "Current power calibration area almost full")
+SENSE_CODE(0x7311, "Current power calibration area is full")
+SENSE_CODE(0x7317, "RDZ is full")
+
+SENSE_CODE(0x7400, "Security error")
+SENSE_CODE(0x7401, "Unable to decrypt data")
+SENSE_CODE(0x7402, "Unencrypted data encountered while decrypting")
+SENSE_CODE(0x7403, "Incorrect data encryption key")
+SENSE_CODE(0x7404, "Cryptographic integrity validation failed")
+SENSE_CODE(0x7405, "Error decrypting data")
+SENSE_CODE(0x7406, "Unknown signature verification key")
+SENSE_CODE(0x7407, "Encryption parameters not useable")
+SENSE_CODE(0x7408, "Digital signature validation failure")
+SENSE_CODE(0x7409, "Encryption mode mismatch on read")
+SENSE_CODE(0x740A, "Encrypted block not raw read enabled")
+SENSE_CODE(0x740B, "Incorrect Encryption parameters")
+SENSE_CODE(0x740C, "Unable to decrypt parameter list")
+SENSE_CODE(0x740D, "Encryption algorithm disabled")
+SENSE_CODE(0x7410, "SA creation parameter value invalid")
+SENSE_CODE(0x7411, "SA creation parameter value rejected")
+SENSE_CODE(0x7412, "Invalid SA usage")
+SENSE_CODE(0x7421, "Data Encryption configuration prevented")
+SENSE_CODE(0x7430, "SA creation parameter not supported")
+SENSE_CODE(0x7440, "Authentication failed")
+SENSE_CODE(0x7461, "External data encryption key manager access error")
+SENSE_CODE(0x7462, "External data encryption key manager error")
+SENSE_CODE(0x7463, "External data encryption key not found")
+SENSE_CODE(0x7464, "External data encryption request not authorized")
+SENSE_CODE(0x746E, "External data encryption control timeout")
+SENSE_CODE(0x746F, "External data encryption control error")
+SENSE_CODE(0x7471, "Logical unit access not authorized")
+SENSE_CODE(0x7479, "Security conflict in translated device")
diff --git a/drivers/scsi/snic/snic.h b/drivers/scsi/snic/snic.h
index d7f5ba6..8ed778d 100644
--- a/drivers/scsi/snic/snic.h
+++ b/drivers/scsi/snic/snic.h
@@ -95,6 +95,8 @@
#define SNIC_DEV_RST_NOTSUP BIT(25)
#define SNIC_SCSI_CLEANUP BIT(26)
#define SNIC_HOST_RESET_ISSUED BIT(27)
+#define SNIC_HOST_RESET_CMD_TERM \
+ (SNIC_DEV_RST_NOTSUP | SNIC_SCSI_CLEANUP | SNIC_HOST_RESET_ISSUED)
#define SNIC_ABTS_TIMEOUT 30000 /* msec */
#define SNIC_LUN_RESET_TIMEOUT 30000 /* msec */
@@ -216,9 +218,10 @@
SNIC_MSIX_INTR_MAX,
};
+#define SNIC_INTRHDLR_NAMSZ (2 * IFNAMSIZ)
struct snic_msix_entry {
int requested;
- char devname[IFNAMSIZ];
+ char devname[SNIC_INTRHDLR_NAMSZ];
irqreturn_t (*isr)(int, void *);
void *devid;
};
diff --git a/drivers/scsi/snic/snic_ctl.c b/drivers/scsi/snic/snic_ctl.c
index ab0e06b..449b03f 100644
--- a/drivers/scsi/snic/snic_ctl.c
+++ b/drivers/scsi/snic/snic_ctl.c
@@ -39,17 +39,15 @@
{
struct snic *snic = container_of(work, struct snic, link_work);
- if (snic->config.xpt_type != SNIC_DAS) {
- SNIC_HOST_INFO(snic->shost, "Link Event Received.\n");
- SNIC_ASSERT_NOT_IMPL(1);
-
+ if (snic->config.xpt_type == SNIC_DAS)
return;
- }
snic->link_status = svnic_dev_link_status(snic->vdev);
snic->link_down_cnt = svnic_dev_link_down_cnt(snic->vdev);
SNIC_HOST_INFO(snic->shost, "Link Event: Link %s.\n",
((snic->link_status) ? "Up" : "Down"));
+
+ SNIC_ASSERT_NOT_IMPL(1);
}
diff --git a/drivers/scsi/snic/snic_debugfs.c b/drivers/scsi/snic/snic_debugfs.c
index 1686f01..d302803 100644
--- a/drivers/scsi/snic/snic_debugfs.c
+++ b/drivers/scsi/snic/snic_debugfs.c
@@ -264,12 +264,14 @@
"Aborts Fail : %lld\n"
"Aborts Driver Timeout : %lld\n"
"Abort FW Timeout : %lld\n"
- "Abort IO NOT Found : %lld\n",
+ "Abort IO NOT Found : %lld\n"
+ "Abort Queuing Failed : %lld\n",
(u64) atomic64_read(&stats->abts.num),
(u64) atomic64_read(&stats->abts.fail),
(u64) atomic64_read(&stats->abts.drv_tmo),
(u64) atomic64_read(&stats->abts.fw_tmo),
- (u64) atomic64_read(&stats->abts.io_not_found));
+ (u64) atomic64_read(&stats->abts.io_not_found),
+ (u64) atomic64_read(&stats->abts.q_fail));
/* Dump Reset Stats */
seq_printf(sfp,
@@ -316,7 +318,9 @@
seq_printf(sfp,
"Last ISR Time : %llu (%8lu.%8lu)\n"
"Last Ack Time : %llu (%8lu.%8lu)\n"
- "ISRs : %llu\n"
+ "Ack ISRs : %llu\n"
+ "IO Cmpl ISRs : %llu\n"
+ "Err Notify ISRs : %llu\n"
"Max CQ Entries : %lld\n"
"Data Count Mismatch : %lld\n"
"IOs w/ Timeout Status : %lld\n"
@@ -324,12 +328,17 @@
"IOs w/ SGL Invalid Stat : %lld\n"
"WQ Desc Alloc Fail : %lld\n"
"Queue Full : %lld\n"
+ "Queue Ramp Up : %lld\n"
+ "Queue Ramp Down : %lld\n"
+ "Queue Last Queue Depth : %lld\n"
"Target Not Ready : %lld\n",
(u64) stats->misc.last_isr_time,
last_isr_tms.tv_sec, last_isr_tms.tv_nsec,
(u64)stats->misc.last_ack_time,
last_ack_tms.tv_sec, last_ack_tms.tv_nsec,
- (u64) atomic64_read(&stats->misc.isr_cnt),
+ (u64) atomic64_read(&stats->misc.ack_isr_cnt),
+ (u64) atomic64_read(&stats->misc.cmpl_isr_cnt),
+ (u64) atomic64_read(&stats->misc.errnotify_isr_cnt),
(u64) atomic64_read(&stats->misc.max_cq_ents),
(u64) atomic64_read(&stats->misc.data_cnt_mismat),
(u64) atomic64_read(&stats->misc.io_tmo),
@@ -337,6 +346,9 @@
(u64) atomic64_read(&stats->misc.sgl_inval),
(u64) atomic64_read(&stats->misc.wq_alloc_fail),
(u64) atomic64_read(&stats->misc.qfull),
+ (u64) atomic64_read(&stats->misc.qsz_rampup),
+ (u64) atomic64_read(&stats->misc.qsz_rampdown),
+ (u64) atomic64_read(&stats->misc.last_qsz),
(u64) atomic64_read(&stats->misc.tgt_not_rdy));
return 0;
diff --git a/drivers/scsi/snic/snic_disc.c b/drivers/scsi/snic/snic_disc.c
index 5f63217..b0fefd6 100644
--- a/drivers/scsi/snic/snic_disc.c
+++ b/drivers/scsi/snic/snic_disc.c
@@ -171,7 +171,7 @@
tgt->channel,
tgt->scsi_tgt_id,
SCAN_WILD_CARD,
- 1);
+ SCSI_SCAN_RESCAN);
spin_lock_irqsave(shost->host_lock, flags);
tgt->flags &= ~SNIC_TGT_SCAN_PENDING;
@@ -480,10 +480,21 @@
snic_disc_start(struct snic *snic)
{
struct snic_disc *disc = &snic->disc;
+ unsigned long flags;
int ret = 0;
SNIC_SCSI_DBG(snic->shost, "Discovery Start.\n");
+ spin_lock_irqsave(&snic->snic_lock, flags);
+ if (snic->in_remove) {
+ spin_unlock_irqrestore(&snic->snic_lock, flags);
+ SNIC_ERR("snic driver removal in progress ...\n");
+ ret = 0;
+
+ return ret;
+ }
+ spin_unlock_irqrestore(&snic->snic_lock, flags);
+
mutex_lock(&disc->mutex);
if (disc->state == SNIC_DISC_PENDING) {
disc->req_cnt++;
@@ -533,6 +544,8 @@
struct list_head *cur, *nxt;
unsigned long flags;
+ scsi_flush_work(snic->shost);
+
mutex_lock(&snic->disc.mutex);
spin_lock_irqsave(snic->shost->host_lock, flags);
@@ -545,7 +558,7 @@
tgt = NULL;
}
spin_unlock_irqrestore(snic->shost->host_lock, flags);
-
- scsi_flush_work(snic->shost);
mutex_unlock(&snic->disc.mutex);
+
+ flush_workqueue(snic_glob->event_q);
} /* end of snic_tgt_del_all */
diff --git a/drivers/scsi/snic/snic_fwint.h b/drivers/scsi/snic/snic_fwint.h
index 2cfaf2d..c5f9e19 100644
--- a/drivers/scsi/snic/snic_fwint.h
+++ b/drivers/scsi/snic/snic_fwint.h
@@ -414,7 +414,7 @@
/* Payload 88 bytes = 128 - 24 - 16 */
#define SNIC_HOST_REQ_PAYLOAD ((int)(SNIC_HOST_REQ_LEN - \
sizeof(struct snic_io_hdr) - \
- (2 * sizeof(u64))))
+ (2 * sizeof(u64)) - sizeof(ulong)))
/*
* snic_host_req: host -> firmware request
@@ -448,6 +448,8 @@
/* hba reset */
struct snic_hba_reset reset;
} u;
+
+ ulong req_pa;
}; /* end of snic_host_req structure */
diff --git a/drivers/scsi/snic/snic_io.c b/drivers/scsi/snic/snic_io.c
index 993db7d..8e69548 100644
--- a/drivers/scsi/snic/snic_io.c
+++ b/drivers/scsi/snic/snic_io.c
@@ -48,7 +48,7 @@
SNIC_TRC(snic->shost->host_no, 0, 0,
((ulong)(buf->os_buf) - sizeof(struct snic_req_info)), 0, 0,
0);
- pci_unmap_single(snic->pdev, buf->dma_addr, buf->len, PCI_DMA_TODEVICE);
+
buf->os_buf = NULL;
}
@@ -137,13 +137,36 @@
return 0;
}
+static int
+snic_wqdesc_avail(struct snic *snic, int q_num, int req_type)
+{
+ int nr_wqdesc = snic->config.wq_enet_desc_count;
+
+ if (q_num > 0) {
+ /*
+ * Multi Queue case, additional care is required.
+ * Per WQ active requests need to be maintained.
+ */
+ SNIC_HOST_INFO(snic->shost, "desc_avail: Multi Queue case.\n");
+ SNIC_BUG_ON(q_num > 0);
+
+ return -1;
+ }
+
+ nr_wqdesc -= atomic64_read(&snic->s_stats.fw.actv_reqs);
+
+ return ((req_type == SNIC_REQ_HBA_RESET) ? nr_wqdesc : nr_wqdesc - 1);
+}
+
int
snic_queue_wq_desc(struct snic *snic, void *os_buf, u16 len)
{
dma_addr_t pa = 0;
unsigned long flags;
struct snic_fw_stats *fwstats = &snic->s_stats.fw;
+ struct snic_host_req *req = (struct snic_host_req *) os_buf;
long act_reqs;
+ long desc_avail = 0;
int q_num = 0;
snic_print_desc(__func__, os_buf, len);
@@ -156,11 +179,15 @@
return -ENOMEM;
}
+ req->req_pa = (ulong)pa;
+
q_num = snic_select_wq(snic);
spin_lock_irqsave(&snic->wq_lock[q_num], flags);
- if (!svnic_wq_desc_avail(snic->wq)) {
+ desc_avail = snic_wqdesc_avail(snic, q_num, req->hdr.type);
+ if (desc_avail <= 0) {
pci_unmap_single(snic->pdev, pa, len, PCI_DMA_TODEVICE);
+ req->req_pa = 0;
spin_unlock_irqrestore(&snic->wq_lock[q_num], flags);
atomic64_inc(&snic->s_stats.misc.wq_alloc_fail);
SNIC_DBG("host = %d, WQ is Full\n", snic->shost->host_no);
@@ -169,10 +196,13 @@
}
snic_queue_wq_eth_desc(&snic->wq[q_num], os_buf, pa, len, 0, 0, 1);
+ /*
+ * Update stats
+ * note: when multi queue enabled, fw actv_reqs should be per queue.
+ */
+ act_reqs = atomic64_inc_return(&fwstats->actv_reqs);
spin_unlock_irqrestore(&snic->wq_lock[q_num], flags);
- /* Update stats */
- act_reqs = atomic64_inc_return(&fwstats->actv_reqs);
if (act_reqs > atomic64_read(&fwstats->max_actv_reqs))
atomic64_set(&fwstats->max_actv_reqs, act_reqs);
@@ -318,11 +348,31 @@
"Req_free:rqi %p:ioreq %p:abt %p:dr %p\n",
rqi, rqi->req, rqi->abort_req, rqi->dr_req);
- if (rqi->abort_req)
- mempool_free(rqi->abort_req, snic->req_pool[SNIC_REQ_TM_CACHE]);
+ if (rqi->abort_req) {
+ if (rqi->abort_req->req_pa)
+ pci_unmap_single(snic->pdev,
+ rqi->abort_req->req_pa,
+ sizeof(struct snic_host_req),
+ PCI_DMA_TODEVICE);
- if (rqi->dr_req)
+ mempool_free(rqi->abort_req, snic->req_pool[SNIC_REQ_TM_CACHE]);
+ }
+
+ if (rqi->dr_req) {
+ if (rqi->dr_req->req_pa)
+ pci_unmap_single(snic->pdev,
+ rqi->dr_req->req_pa,
+ sizeof(struct snic_host_req),
+ PCI_DMA_TODEVICE);
+
mempool_free(rqi->dr_req, snic->req_pool[SNIC_REQ_TM_CACHE]);
+ }
+
+ if (rqi->req->req_pa)
+ pci_unmap_single(snic->pdev,
+ rqi->req->req_pa,
+ rqi->req_len,
+ PCI_DMA_TODEVICE);
mempool_free(rqi, snic->req_pool[rqi->rq_pool_type]);
}
diff --git a/drivers/scsi/snic/snic_isr.c b/drivers/scsi/snic/snic_isr.c
index a85fae2..f552003 100644
--- a/drivers/scsi/snic/snic_isr.c
+++ b/drivers/scsi/snic/snic_isr.c
@@ -38,7 +38,7 @@
unsigned long wq_work_done = 0;
snic->s_stats.misc.last_isr_time = jiffies;
- atomic64_inc(&snic->s_stats.misc.isr_cnt);
+ atomic64_inc(&snic->s_stats.misc.ack_isr_cnt);
wq_work_done = snic_wq_cmpl_handler(snic, -1);
svnic_intr_return_credits(&snic->intr[SNIC_MSIX_WQ],
@@ -56,7 +56,7 @@
unsigned long iocmpl_work_done = 0;
snic->s_stats.misc.last_isr_time = jiffies;
- atomic64_inc(&snic->s_stats.misc.isr_cnt);
+ atomic64_inc(&snic->s_stats.misc.cmpl_isr_cnt);
iocmpl_work_done = snic_fwcq_cmpl_handler(snic, -1);
svnic_intr_return_credits(&snic->intr[SNIC_MSIX_IO_CMPL],
@@ -73,7 +73,7 @@
struct snic *snic = data;
snic->s_stats.misc.last_isr_time = jiffies;
- atomic64_inc(&snic->s_stats.misc.isr_cnt);
+ atomic64_inc(&snic->s_stats.misc.errnotify_isr_cnt);
svnic_intr_return_all_credits(&snic->intr[SNIC_MSIX_ERR_NOTIFY]);
snic_log_q_error(snic);
diff --git a/drivers/scsi/snic/snic_main.c b/drivers/scsi/snic/snic_main.c
index 2b3c253..396b32d 100644
--- a/drivers/scsi/snic/snic_main.c
+++ b/drivers/scsi/snic/snic_main.c
@@ -98,11 +98,18 @@
static int
snic_change_queue_depth(struct scsi_device *sdev, int qdepth)
{
+ struct snic *snic = shost_priv(sdev->host);
int qsz = 0;
qsz = min_t(u32, qdepth, SNIC_MAX_QUEUE_DEPTH);
+ if (qsz < sdev->queue_depth)
+ atomic64_inc(&snic->s_stats.misc.qsz_rampdown);
+ else if (qsz > sdev->queue_depth)
+ atomic64_inc(&snic->s_stats.misc.qsz_rampup);
+
+ atomic64_set(&snic->s_stats.misc.last_qsz, sdev->queue_depth);
+
scsi_change_queue_depth(sdev, qsz);
- SNIC_INFO("QDepth Changed to %d\n", sdev->queue_depth);
return sdev->queue_depth;
}
@@ -624,19 +631,6 @@
goto err_free_tmreq_pool;
}
- /*
- * Initialization done with PCI system, hardware, firmware.
- * Add shost to SCSI
- */
- ret = snic_add_host(shost, pdev);
- if (ret) {
- SNIC_HOST_ERR(shost,
- "Adding scsi host Failed ... exiting. %d\n",
- ret);
-
- goto err_notify_unset;
- }
-
spin_lock_irqsave(&snic_glob->snic_list_lock, flags);
list_add_tail(&snic->list, &snic_glob->snic_list);
spin_unlock_irqrestore(&snic_glob->snic_list_lock, flags);
@@ -669,8 +663,6 @@
for (i = 0; i < snic->intr_count; i++)
svnic_intr_unmask(&snic->intr[i]);
- snic_set_state(snic, SNIC_ONLINE);
-
/* Get snic params */
ret = snic_get_conf(snic);
if (ret) {
@@ -681,6 +673,21 @@
goto err_get_conf;
}
+ /*
+ * Initialization done with PCI system, hardware, firmware.
+ * Add shost to SCSI
+ */
+ ret = snic_add_host(shost, pdev);
+ if (ret) {
+ SNIC_HOST_ERR(shost,
+ "Adding scsi host Failed ... exiting. %d\n",
+ ret);
+
+ goto err_get_conf;
+ }
+
+ snic_set_state(snic, SNIC_ONLINE);
+
ret = snic_disc_start(snic);
if (ret) {
SNIC_HOST_ERR(shost, "snic_probe:Discovery Failed w err = %d\n",
@@ -705,6 +712,8 @@
svnic_dev_disable(snic->vdev);
err_vdev_enable:
+ svnic_dev_notify_unset(snic->vdev);
+
for (i = 0; i < snic->wq_count; i++) {
int rc = 0;
@@ -718,9 +727,6 @@
}
snic_del_host(snic->shost);
-err_notify_unset:
- svnic_dev_notify_unset(snic->vdev);
-
err_free_tmreq_pool:
mempool_destroy(snic->req_pool[SNIC_REQ_TM_CACHE]);
diff --git a/drivers/scsi/snic/snic_scsi.c b/drivers/scsi/snic/snic_scsi.c
index 2c7b4c3..abada16 100644
--- a/drivers/scsi/snic/snic_scsi.c
+++ b/drivers/scsi/snic/snic_scsi.c
@@ -221,11 +221,15 @@
pa, /* sense buffer pa */
SCSI_SENSE_BUFFERSIZE);
+ atomic64_inc(&snic->s_stats.io.active);
ret = snic_queue_wq_desc(snic, rqi->req, rqi->req_len);
- if (ret)
+ if (ret) {
+ atomic64_dec(&snic->s_stats.io.active);
SNIC_HOST_ERR(snic->shost,
"QIcmnd: Queuing Icmnd Failed. ret = %d\n",
ret);
+ } else
+ snic_stats_update_active_ios(&snic->s_stats);
return ret;
} /* end of snic_queue_icmnd_req */
@@ -361,8 +365,7 @@
if (ret) {
SNIC_HOST_ERR(shost, "Failed to Q, Scsi Req w/ err %d.\n", ret);
ret = SCSI_MLQUEUE_HOST_BUSY;
- } else
- snic_stats_update_active_ios(&snic->s_stats);
+ }
atomic_dec(&snic->ios_inflight);
@@ -598,6 +601,12 @@
sc->device->lun, sc, sc->cmnd[0], snic_cmd_tag(sc),
CMD_FLAGS(sc), rqi);
+ if (CMD_FLAGS(sc) & SNIC_HOST_RESET_CMD_TERM) {
+ spin_unlock_irqrestore(io_lock, flags);
+
+ return;
+ }
+
SNIC_BUG_ON(rqi != (struct snic_req_info *)ctx);
WARN_ON_ONCE(req);
if (!rqi) {
@@ -779,6 +788,11 @@
io_lock = snic_io_lock_hash(snic, sc);
spin_lock_irqsave(io_lock, flags);
+ if (CMD_FLAGS(sc) & SNIC_HOST_RESET_CMD_TERM) {
+ spin_unlock_irqrestore(io_lock, flags);
+
+ return ret;
+ }
rqi = (struct snic_req_info *) CMD_SP(sc);
WARN_ON_ONCE(!rqi);
@@ -1001,10 +1015,11 @@
unsigned long flags, gflags;
int ret = 0;
- SNIC_HOST_INFO(snic->shost,
- "reset_cmpl:HBA Reset Completion received.\n");
-
snic_io_hdr_dec(&fwreq->hdr, &typ, &hdr_stat, &cmnd_id, &hid, &ctx);
+ SNIC_HOST_INFO(snic->shost,
+ "reset_cmpl:Tag %d ctx %lx cmpl status %s HBA Reset Completion received.\n",
+ cmnd_id, ctx, snic_io_status_to_str(hdr_stat));
+
SNIC_SCSI_DBG(snic->shost,
"reset_cmpl: type = %x, hdr_stat = %x, cmnd_id = %x, hid = %x, ctx = %lx\n",
typ, hdr_stat, cmnd_id, hid, ctx);
@@ -1012,6 +1027,9 @@
/* spl case, host reset issued through ioctl */
if (cmnd_id == SCSI_NO_TAG) {
rqi = (struct snic_req_info *) ctx;
+ SNIC_HOST_INFO(snic->shost,
+ "reset_cmpl:Tag %d ctx %lx cmpl stat %s\n",
+ cmnd_id, ctx, snic_io_status_to_str(hdr_stat));
sc = rqi->sc;
goto ioctl_hba_rst;
@@ -1038,6 +1056,10 @@
return ret;
}
+ SNIC_HOST_INFO(snic->shost,
+ "reset_cmpl: sc %p rqi %p Tag %d flags 0x%llx\n",
+ sc, rqi, cmnd_id, CMD_FLAGS(sc));
+
io_lock = snic_io_lock_hash(snic, sc);
spin_lock_irqsave(io_lock, flags);
@@ -1454,11 +1476,19 @@
case SNIC_STAT_IO_SUCCESS:
case SNIC_STAT_IO_NOT_FOUND:
ret = SUCCESS;
+ /*
+ * If abort path doesn't call scsi_done(),
+ * the # IO timeouts == 2, will cause the LUN offline.
+ * Call scsi_done to complete the IO.
+ */
+ sc->result = (DID_ERROR << 16);
+ sc->scsi_done(sc);
break;
default:
/* Firmware completed abort with error */
ret = FAILED;
+ rqi = NULL;
break;
}
@@ -1554,6 +1584,7 @@
/* Now Queue the abort command to firmware */
ret = snic_queue_abort_req(snic, rqi, sc, tmf);
if (ret) {
+ atomic64_inc(&snic->s_stats.abts.q_fail);
SNIC_HOST_ERR(snic->shost,
"send_abt_cmd: IO w/ Tag 0x%x fail w/ err %d flags 0x%llx\n",
tag, ret, CMD_FLAGS(sc));
@@ -1830,6 +1861,9 @@
snic_release_req_buf(snic, rqi, sc);
+ sc->result = (DID_ERROR << 16);
+ sc->scsi_done(sc);
+
ret = 0;
return ret;
@@ -2384,6 +2418,13 @@
"Completing Pending TM Req sc %p, state %s flags 0x%llx\n",
sc, snic_io_status_to_str(CMD_STATE(sc)), CMD_FLAGS(sc));
+ /*
+ * CASE : FW didn't post itmf completion due to PCIe Errors.
+ * Marking the abort status as Success to call scsi completion
+ * in snic_abort_finish()
+ */
+ CMD_ABTS_STATUS(sc) = SNIC_STAT_IO_SUCCESS;
+
rqi = (struct snic_req_info *) CMD_SP(sc);
if (!rqi)
return;
@@ -2459,8 +2500,9 @@
cleanup:
sc->result = DID_TRANSPORT_DISRUPTED << 16;
SNIC_HOST_INFO(snic->shost,
- "sc_clean: DID_TRANSPORT_DISRUPTED for sc %p. rqi %p duration %llu msecs\n",
- sc, rqi, (jiffies - st_time));
+ "sc_clean: DID_TRANSPORT_DISRUPTED for sc %p, Tag %d flags 0x%llx rqi %p duration %u msecs\n",
+ sc, sc->request->tag, CMD_FLAGS(sc), rqi,
+ jiffies_to_msecs(jiffies - st_time));
/* Update IO stats */
snic_stats_update_io_cmpl(&snic->s_stats);
diff --git a/drivers/scsi/snic/snic_stats.h b/drivers/scsi/snic/snic_stats.h
index 11e6148..fd1066b 100644
--- a/drivers/scsi/snic/snic_stats.h
+++ b/drivers/scsi/snic/snic_stats.h
@@ -42,6 +42,7 @@
atomic64_t drv_tmo; /* Abort Driver Timeouts */
atomic64_t fw_tmo; /* Abort Firmware Timeouts */
atomic64_t io_not_found;/* Abort IO Not Found */
+ atomic64_t q_fail; /* Abort Queuing Failed */
};
struct snic_reset_stats {
@@ -69,7 +70,9 @@
struct snic_misc_stats {
u64 last_isr_time;
u64 last_ack_time;
- atomic64_t isr_cnt;
+ atomic64_t ack_isr_cnt;
+ atomic64_t cmpl_isr_cnt;
+ atomic64_t errnotify_isr_cnt;
atomic64_t max_cq_ents; /* Max CQ Entries */
atomic64_t data_cnt_mismat; /* Data Count Mismatch */
atomic64_t io_tmo;
@@ -81,6 +84,9 @@
atomic64_t no_icmnd_itmf_cmpls;
atomic64_t io_under_run;
atomic64_t qfull;
+ atomic64_t qsz_rampup;
+ atomic64_t qsz_rampdown;
+ atomic64_t last_qsz;
atomic64_t tgt_not_rdy;
};
@@ -101,9 +107,9 @@
snic_stats_update_active_ios(struct snic_stats *s_stats)
{
struct snic_io_stats *io = &s_stats->io;
- u32 nr_active_ios;
+ int nr_active_ios;
- nr_active_ios = atomic64_inc_return(&io->active);
+ nr_active_ios = atomic64_read(&io->active);
if (atomic64_read(&io->max_active) < nr_active_ios)
atomic64_set(&io->max_active, nr_active_ios);
diff --git a/drivers/scsi/snic/vnic_dev.c b/drivers/scsi/snic/vnic_dev.c
index e0b5549..dad5fc6 100644
--- a/drivers/scsi/snic/vnic_dev.c
+++ b/drivers/scsi/snic/vnic_dev.c
@@ -263,12 +263,20 @@
int wait)
{
struct devcmd2_controller *dc2c = vdev->devcmd2;
- struct devcmd2_result *result = dc2c->result + dc2c->next_result;
+ struct devcmd2_result *result = NULL;
unsigned int i;
int delay;
int err;
u32 posted;
+ u32 fetch_idx;
u32 new_posted;
+ u8 color;
+
+ fetch_idx = ioread32(&dc2c->wq_ctrl->fetch_index);
+ if (fetch_idx == 0xFFFFFFFF) { /* check for hardware gone */
+ /* Hardware surprise removal: return error */
+ return -ENODEV;
+ }
posted = ioread32(&dc2c->wq_ctrl->posted_index);
@@ -278,6 +286,13 @@
}
new_posted = (posted + 1) % DEVCMD2_RING_SIZE;
+ if (new_posted == fetch_idx) {
+ pr_err("%s: wq is full while issuing devcmd2 command %d, fetch index: %u, posted index: %u\n",
+ pci_name(vdev->pdev), _CMD_N(cmd), fetch_idx, posted);
+
+ return -EBUSY;
+ }
+
dc2c->cmd_ring[posted].cmd = cmd;
dc2c->cmd_ring[posted].flags = 0;
@@ -299,14 +314,22 @@
if (dc2c->cmd_ring[posted].flags & DEVCMD2_FNORESULT)
return 0;
+ result = dc2c->result + dc2c->next_result;
+ color = dc2c->color;
+
+ /*
+ * Increment next_result, after posting the devcmd, irrespective of
+ * devcmd result, and it should be done only once.
+ */
+ dc2c->next_result++;
+ if (dc2c->next_result == dc2c->result_size) {
+ dc2c->next_result = 0;
+ dc2c->color = dc2c->color ? 0 : 1;
+ }
+
for (delay = 0; delay < wait; delay++) {
udelay(100);
- if (result->color == dc2c->color) {
- dc2c->next_result++;
- if (dc2c->next_result == dc2c->result_size) {
- dc2c->next_result = 0;
- dc2c->color = dc2c->color ? 0 : 1;
- }
+ if (result->color == color) {
if (result->error) {
err = (int) result->error;
if (err != ERR_ECMDUNKNOWN ||
@@ -317,13 +340,6 @@
return err;
}
if (_CMD_DIR(cmd) & _CMD_DIR_READ) {
- /*
- * Adding the rmb() prevents the compiler
- * and/or CPU from reordering the reads which
- * would potentially result in reading stale
- * values.
- */
- rmb();
for (i = 0; i < VNIC_DEVCMD_NARGS; i++)
vdev->args[i] = result->results[i];
}
diff --git a/drivers/scsi/st.c b/drivers/scsi/st.c
index dbf1882c..7af5226 100644
--- a/drivers/scsi/st.c
+++ b/drivers/scsi/st.c
@@ -1974,9 +1974,12 @@
transfer = (int)cmdstatp->uremainder64;
else
transfer = 0;
- if (STp->block_size == 0 &&
- cmdstatp->sense_hdr.sense_key == MEDIUM_ERROR)
- transfer = bytes;
+ if (cmdstatp->sense_hdr.sense_key == MEDIUM_ERROR) {
+ if (STp->block_size == 0)
+ transfer = bytes;
+ /* Some drives set ILI with MEDIUM ERROR */
+ cmdstatp->flags &= ~SENSE_ILI;
+ }
if (cmdstatp->flags & SENSE_ILI) { /* ILI */
if (STp->block_size == 0 &&
diff --git a/drivers/scsi/sun3_scsi.c b/drivers/scsi/sun3_scsi.c
index b9de487..3c4c070 100644
--- a/drivers/scsi/sun3_scsi.c
+++ b/drivers/scsi/sun3_scsi.c
@@ -36,14 +36,10 @@
#include <scsi/scsi_host.h>
#include "sun3_scsi.h"
-/* Definitions for the core NCR5380 driver. */
-
-#define REAL_DMA
-/* #define SUPPORT_TAGS */
/* minimum number of bytes to do dma on */
#define DMA_MIN_SIZE 129
-/* #define MAX_TAGS 32 */
+/* Definitions for the core NCR5380 driver. */
#define NCR5380_implementation_fields /* none */
@@ -55,14 +51,12 @@
#define NCR5380_abort sun3scsi_abort
#define NCR5380_info sun3scsi_info
-#define NCR5380_dma_read_setup(instance, data, count) \
- sun3scsi_dma_setup(instance, data, count, 0)
-#define NCR5380_dma_write_setup(instance, data, count) \
- sun3scsi_dma_setup(instance, data, count, 1)
+#define NCR5380_dma_recv_setup(instance, data, count) (count)
+#define NCR5380_dma_send_setup(instance, data, count) (count)
#define NCR5380_dma_residual(instance) \
sun3scsi_dma_residual(instance)
#define NCR5380_dma_xfer_len(instance, cmd, phase) \
- sun3scsi_dma_xfer_len(cmd->SCp.this_residual, cmd, !((phase) & SR_IO))
+ sun3scsi_dma_xfer_len(cmd->SCp.this_residual, cmd)
#define NCR5380_acquire_dma_irq(instance) (1)
#define NCR5380_release_dma_irq(instance)
@@ -78,10 +72,6 @@
module_param(setup_cmd_per_lun, int, 0);
static int setup_sg_tablesize = -1;
module_param(setup_sg_tablesize, int, 0);
-#ifdef SUPPORT_TAGS
-static int setup_use_tagged_queuing = -1;
-module_param(setup_use_tagged_queuing, int, 0);
-#endif
static int setup_hostid = -1;
module_param(setup_hostid, int, 0);
@@ -263,14 +253,13 @@
return last_residual;
}
-static inline unsigned long sun3scsi_dma_xfer_len(unsigned long wanted,
- struct scsi_cmnd *cmd,
- int write_flag)
+static inline unsigned long sun3scsi_dma_xfer_len(unsigned long wanted_len,
+ struct scsi_cmnd *cmd)
{
- if (cmd->request->cmd_type == REQ_TYPE_FS)
- return wanted;
- else
+ if (wanted_len < DMA_MIN_SIZE || cmd->request->cmd_type != REQ_TYPE_FS)
return 0;
+
+ return wanted_len;
}
static inline int sun3scsi_dma_start(unsigned long count, unsigned char *data)
@@ -408,7 +397,7 @@
}
-#include "atari_NCR5380.c"
+#include "NCR5380.c"
#ifdef SUN3_SCSI_VME
#define SUN3_SCSI_NAME "Sun3 NCR5380 VME SCSI"
@@ -516,10 +505,6 @@
instance->io_port = (unsigned long)ioaddr;
instance->irq = irq->start;
-#ifdef SUPPORT_TAGS
- host_flags |= setup_use_tagged_queuing > 0 ? FLAG_TAGGED_QUEUING : 0;
-#endif
-
error = NCR5380_init(instance, host_flags);
if (error)
goto fail_init;
@@ -527,15 +512,9 @@
error = request_irq(instance->irq, scsi_sun3_intr, 0,
"NCR5380", instance);
if (error) {
-#ifdef REAL_DMA
pr_err(PFX "scsi%d: IRQ %d not free, bailing out\n",
instance->host_no, instance->irq);
goto fail_irq;
-#else
- pr_warn(PFX "scsi%d: IRQ %d not free, interrupts disabled\n",
- instance->host_no, instance->irq);
- instance->irq = NO_IRQ;
-#endif
}
dregs->csr = 0;
@@ -565,8 +544,7 @@
return 0;
fail_host:
- if (instance->irq != NO_IRQ)
- free_irq(instance->irq, instance);
+ free_irq(instance->irq, instance);
fail_irq:
NCR5380_exit(instance);
fail_init:
@@ -583,8 +561,7 @@
struct Scsi_Host *instance = platform_get_drvdata(pdev);
scsi_remove_host(instance);
- if (instance->irq != NO_IRQ)
- free_irq(instance->irq, instance);
+ free_irq(instance->irq, instance);
NCR5380_exit(instance);
scsi_host_put(instance);
if (udc_regs)
diff --git a/drivers/scsi/t128.c b/drivers/scsi/t128.c
index 4615fda..8a8608a 100644
--- a/drivers/scsi/t128.c
+++ b/drivers/scsi/t128.c
@@ -1,5 +1,3 @@
-#define PSEUDO_DMA
-
/*
* Trantor T128/T128F/T228 driver
* Note : architecturally, the T100 and T130 are different and won't
@@ -76,7 +74,6 @@
#include <scsi/scsi_host.h>
#include "t128.h"
-#define AUTOPROBE_IRQ
#include "NCR5380.h"
static struct override {
@@ -210,7 +207,7 @@
instance->base = base;
((struct NCR5380_hostdata *)instance->hostdata)->base = p;
- if (NCR5380_init(instance, 0))
+ if (NCR5380_init(instance, FLAG_DMA_FIXUP | FLAG_LATE_DMA_SETUP))
goto out_unregister;
NCR5380_maybe_reset_bus(instance);
@@ -294,7 +291,7 @@
}
/*
- * Function : int NCR5380_pread (struct Scsi_Host *instance,
+ * Function : int t128_pread (struct Scsi_Host *instance,
* unsigned char *dst, int len)
*
* Purpose : Fast 5380 pseudo-dma read function, transfers len bytes to
@@ -306,8 +303,8 @@
* timeout.
*/
-static inline int
-NCR5380_pread(struct Scsi_Host *instance, unsigned char *dst, int len)
+static inline int t128_pread(struct Scsi_Host *instance,
+ unsigned char *dst, int len)
{
struct NCR5380_hostdata *hostdata = shost_priv(instance);
void __iomem *reg, *base = hostdata->base;
@@ -340,7 +337,7 @@
}
/*
- * Function : int NCR5380_pwrite (struct Scsi_Host *instance,
+ * Function : int t128_pwrite (struct Scsi_Host *instance,
* unsigned char *src, int len)
*
* Purpose : Fast 5380 pseudo-dma write function, transfers len bytes from
@@ -352,8 +349,8 @@
* timeout.
*/
-static inline int
-NCR5380_pwrite(struct Scsi_Host *instance, unsigned char *src, int len)
+static inline int t128_pwrite(struct Scsi_Host *instance,
+ unsigned char *src, int len)
{
struct NCR5380_hostdata *hostdata = shost_priv(instance);
void __iomem *reg, *base = hostdata->base;
@@ -394,8 +391,6 @@
.detect = t128_detect,
.release = t128_release,
.proc_name = "t128",
- .show_info = t128_show_info,
- .write_info = t128_write_info,
.info = t128_info,
.queuecommand = t128_queue_command,
.eh_abort_handler = t128_abort,
diff --git a/drivers/scsi/t128.h b/drivers/scsi/t128.h
index dd16d85..c95bcd83 100644
--- a/drivers/scsi/t128.h
+++ b/drivers/scsi/t128.h
@@ -77,14 +77,17 @@
#define NCR5380_write(reg, value) writeb((value),(T128_address(reg)))
#define NCR5380_dma_xfer_len(instance, cmd, phase) (cmd->transfersize)
+#define NCR5380_dma_recv_setup t128_pread
+#define NCR5380_dma_send_setup t128_pwrite
+#define NCR5380_dma_residual(instance) (0)
#define NCR5380_intr t128_intr
#define NCR5380_queue_command t128_queue_command
#define NCR5380_abort t128_abort
#define NCR5380_bus_reset t128_bus_reset
#define NCR5380_info t128_info
-#define NCR5380_show_info t128_show_info
-#define NCR5380_write_info t128_write_info
+
+#define NCR5380_io_delay(x) udelay(x)
/* 15 14 12 10 7 5 3
1101 0100 1010 1000 */
diff --git a/drivers/spi/Kconfig b/drivers/spi/Kconfig
index 9d8c84b..4b931ec 100644
--- a/drivers/spi/Kconfig
+++ b/drivers/spi/Kconfig
@@ -410,7 +410,6 @@
config SPI_OMAP24XX
tristate "McSPI driver for OMAP"
depends on HAS_DMA
- depends on ARM || ARM64 || AVR32 || HEXAGON || MIPS || SUPERH
depends on ARCH_OMAP2PLUS || COMPILE_TEST
help
SPI master controller for OMAP24XX and later Multichannel SPI
@@ -432,10 +431,23 @@
config SPI_ORION
tristate "Orion SPI master"
- depends on PLAT_ORION || COMPILE_TEST
+ depends on PLAT_ORION || ARCH_MVEBU || COMPILE_TEST
help
This enables using the SPI master controller on the Orion chips.
+config SPI_PIC32
+ tristate "Microchip PIC32 series SPI"
+ depends on MACH_PIC32 || COMPILE_TEST
+ help
+ SPI driver for Microchip PIC32 SPI master controller.
+
+config SPI_PIC32_SQI
+ tristate "Microchip PIC32 Quad SPI driver"
+ depends on MACH_PIC32 || COMPILE_TEST
+ depends on HAS_DMA
+ help
+ SPI driver for PIC32 Quad SPI controller.
+
config SPI_PL022
tristate "ARM AMBA PL022 SSP controller"
depends on ARM_AMBA
@@ -469,7 +481,6 @@
config SPI_ROCKCHIP
tristate "Rockchip SPI controller driver"
- depends on ARM || ARM64 || AVR32 || HEXAGON || MIPS || SUPERH
help
This selects a driver for Rockchip SPI controller.
@@ -569,7 +580,7 @@
config SPI_ST_SSC4
tristate "STMicroelectronics SPI SSC-based driver"
- depends on ARCH_STI
+ depends on ARCH_STI || COMPILE_TEST
help
STMicroelectronics SoCs support for SPI. If you say yes to
this option, support will be included for the SSC driven SPI.
@@ -656,7 +667,7 @@
config SPI_XLP
tristate "Netlogic XLP SPI controller driver"
- depends on CPU_XLP || COMPILE_TEST
+ depends on CPU_XLP || ARCH_VULCAN || COMPILE_TEST
help
Enable support for the SPI controller on the Netlogic XLP SoCs.
Currently supported XLP variants are XLP8XX, XLP3XX, XLP2XX, XLP9XX
diff --git a/drivers/spi/Makefile b/drivers/spi/Makefile
index fbb255c..3c74d00 100644
--- a/drivers/spi/Makefile
+++ b/drivers/spi/Makefile
@@ -62,6 +62,8 @@
obj-$(CONFIG_SPI_OMAP24XX) += spi-omap2-mcspi.o
obj-$(CONFIG_SPI_TI_QSPI) += spi-ti-qspi.o
obj-$(CONFIG_SPI_ORION) += spi-orion.o
+obj-$(CONFIG_SPI_PIC32) += spi-pic32.o
+obj-$(CONFIG_SPI_PIC32_SQI) += spi-pic32-sqi.o
obj-$(CONFIG_SPI_PL022) += spi-pl022.o
obj-$(CONFIG_SPI_PPC4xx) += spi-ppc4xx.o
spi-pxa2xx-platform-objs := spi-pxa2xx.o spi-pxa2xx-dma.o
diff --git a/drivers/spi/spi-axi-spi-engine.c b/drivers/spi/spi-axi-spi-engine.c
index c968ab2..2b1456e 100644
--- a/drivers/spi/spi-axi-spi-engine.c
+++ b/drivers/spi/spi-axi-spi-engine.c
@@ -525,7 +525,6 @@
if (ret)
goto err_ref_clk_disable;
- master->dev.parent = &pdev->dev;
master->dev.of_node = pdev->dev.of_node;
master->mode_bits = SPI_CPOL | SPI_CPHA | SPI_3WIRE;
master->bits_per_word_mask = SPI_BPW_MASK(8);
diff --git a/drivers/spi/spi-bcm53xx.c b/drivers/spi/spi-bcm53xx.c
index cc3f938..afb5169 100644
--- a/drivers/spi/spi-bcm53xx.c
+++ b/drivers/spi/spi-bcm53xx.c
@@ -10,6 +10,7 @@
#include "spi-bcm53xx.h"
#define BCM53XXSPI_MAX_SPI_BAUD 13500000 /* 216 MHz? */
+#define BCM53XXSPI_FLASH_WINDOW SZ_32M
/* The longest observed required wait was 19 ms */
#define BCM53XXSPI_SPE_TIMEOUT_MS 80
@@ -17,8 +18,10 @@
struct bcm53xxspi {
struct bcma_device *core;
struct spi_master *master;
+ void __iomem *mmio_base;
size_t read_offset;
+ bool bspi; /* Boot SPI mode with memory mapping */
};
static inline u32 bcm53xxspi_read(struct bcm53xxspi *b53spi, u16 offset)
@@ -32,6 +35,50 @@
bcma_write32(b53spi->core, offset, value);
}
+static void bcm53xxspi_disable_bspi(struct bcm53xxspi *b53spi)
+{
+ struct device *dev = &b53spi->core->dev;
+ unsigned long deadline;
+ u32 tmp;
+
+ if (!b53spi->bspi)
+ return;
+
+ tmp = bcm53xxspi_read(b53spi, B53SPI_BSPI_MAST_N_BOOT_CTRL);
+ if (tmp & 0x1)
+ return;
+
+ deadline = jiffies + usecs_to_jiffies(200);
+ do {
+ tmp = bcm53xxspi_read(b53spi, B53SPI_BSPI_BUSY_STATUS);
+ if (!(tmp & 0x1)) {
+ bcm53xxspi_write(b53spi, B53SPI_BSPI_MAST_N_BOOT_CTRL,
+ 0x1);
+ ndelay(200);
+ b53spi->bspi = false;
+ return;
+ }
+ udelay(1);
+ } while (!time_after_eq(jiffies, deadline));
+
+ dev_warn(dev, "Timeout disabling BSPI\n");
+}
+
+static void bcm53xxspi_enable_bspi(struct bcm53xxspi *b53spi)
+{
+ u32 tmp;
+
+ if (b53spi->bspi)
+ return;
+
+ tmp = bcm53xxspi_read(b53spi, B53SPI_BSPI_MAST_N_BOOT_CTRL);
+ if (!(tmp & 0x1))
+ return;
+
+ bcm53xxspi_write(b53spi, B53SPI_BSPI_MAST_N_BOOT_CTRL, 0x0);
+ b53spi->bspi = true;
+}
+
static inline unsigned int bcm53xxspi_calc_timeout(size_t len)
{
/* Do some magic calculation based on length and buad. Add 10% and 1. */
@@ -176,6 +223,8 @@
u8 *buf;
size_t left;
+ bcm53xxspi_disable_bspi(b53spi);
+
if (t->tx_buf) {
buf = (u8 *)t->tx_buf;
left = t->len;
@@ -206,6 +255,22 @@
return 0;
}
+static int bcm53xxspi_flash_read(struct spi_device *spi,
+ struct spi_flash_read_message *msg)
+{
+ struct bcm53xxspi *b53spi = spi_master_get_devdata(spi->master);
+ int ret = 0;
+
+ if (msg->from + msg->len > BCM53XXSPI_FLASH_WINDOW)
+ return -EINVAL;
+
+ bcm53xxspi_enable_bspi(b53spi);
+ memcpy_fromio(msg->buf, b53spi->mmio_base + msg->from, msg->len);
+ msg->retlen = msg->len;
+
+ return ret;
+}
+
/**************************************************
* BCMA
**************************************************/
@@ -222,6 +287,7 @@
static int bcm53xxspi_bcma_probe(struct bcma_device *core)
{
+ struct device *dev = &core->dev;
struct bcm53xxspi *b53spi;
struct spi_master *master;
int err;
@@ -231,7 +297,7 @@
return -ENOTSUPP;
}
- master = spi_alloc_master(&core->dev, sizeof(*b53spi));
+ master = spi_alloc_master(dev, sizeof(*b53spi));
if (!master)
return -ENOMEM;
@@ -239,11 +305,19 @@
b53spi->master = master;
b53spi->core = core;
+ if (core->addr_s[0])
+ b53spi->mmio_base = devm_ioremap(dev, core->addr_s[0],
+ BCM53XXSPI_FLASH_WINDOW);
+ b53spi->bspi = true;
+ bcm53xxspi_disable_bspi(b53spi);
+
master->transfer_one = bcm53xxspi_transfer_one;
+ if (b53spi->mmio_base)
+ master->spi_flash_read = bcm53xxspi_flash_read;
bcma_set_drvdata(core, b53spi);
- err = devm_spi_register_master(&core->dev, master);
+ err = devm_spi_register_master(dev, master);
if (err) {
spi_master_put(master);
bcma_set_drvdata(core, NULL);
diff --git a/drivers/spi/spi-cadence.c b/drivers/spi/spi-cadence.c
index 121a413..1c57ce6 100644
--- a/drivers/spi/spi-cadence.c
+++ b/drivers/spi/spi-cadence.c
@@ -19,44 +19,46 @@
#include <linux/of_irq.h>
#include <linux/of_address.h>
#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
#include <linux/spi/spi.h>
/* Name of this driver */
#define CDNS_SPI_NAME "cdns-spi"
/* Register offset definitions */
-#define CDNS_SPI_CR_OFFSET 0x00 /* Configuration Register, RW */
-#define CDNS_SPI_ISR_OFFSET 0x04 /* Interrupt Status Register, RO */
-#define CDNS_SPI_IER_OFFSET 0x08 /* Interrupt Enable Register, WO */
-#define CDNS_SPI_IDR_OFFSET 0x0c /* Interrupt Disable Register, WO */
-#define CDNS_SPI_IMR_OFFSET 0x10 /* Interrupt Enabled Mask Register, RO */
-#define CDNS_SPI_ER_OFFSET 0x14 /* Enable/Disable Register, RW */
-#define CDNS_SPI_DR_OFFSET 0x18 /* Delay Register, RW */
-#define CDNS_SPI_TXD_OFFSET 0x1C /* Data Transmit Register, WO */
-#define CDNS_SPI_RXD_OFFSET 0x20 /* Data Receive Register, RO */
-#define CDNS_SPI_SICR_OFFSET 0x24 /* Slave Idle Count Register, RW */
-#define CDNS_SPI_THLD_OFFSET 0x28 /* Transmit FIFO Watermark Register,RW */
+#define CDNS_SPI_CR 0x00 /* Configuration Register, RW */
+#define CDNS_SPI_ISR 0x04 /* Interrupt Status Register, RO */
+#define CDNS_SPI_IER 0x08 /* Interrupt Enable Register, WO */
+#define CDNS_SPI_IDR 0x0c /* Interrupt Disable Register, WO */
+#define CDNS_SPI_IMR 0x10 /* Interrupt Enabled Mask Register, RO */
+#define CDNS_SPI_ER 0x14 /* Enable/Disable Register, RW */
+#define CDNS_SPI_DR 0x18 /* Delay Register, RW */
+#define CDNS_SPI_TXD 0x1C /* Data Transmit Register, WO */
+#define CDNS_SPI_RXD 0x20 /* Data Receive Register, RO */
+#define CDNS_SPI_SICR 0x24 /* Slave Idle Count Register, RW */
+#define CDNS_SPI_THLD 0x28 /* Transmit FIFO Watermark Register,RW */
+#define SPI_AUTOSUSPEND_TIMEOUT 3000
/*
* SPI Configuration Register bit Masks
*
* This register contains various control bits that affect the operation
* of the SPI controller
*/
-#define CDNS_SPI_CR_MANSTRT_MASK 0x00010000 /* Manual TX Start */
-#define CDNS_SPI_CR_CPHA_MASK 0x00000004 /* Clock Phase Control */
-#define CDNS_SPI_CR_CPOL_MASK 0x00000002 /* Clock Polarity Control */
-#define CDNS_SPI_CR_SSCTRL_MASK 0x00003C00 /* Slave Select Mask */
-#define CDNS_SPI_CR_PERI_SEL_MASK 0x00000200 /* Peripheral Select Decode */
-#define CDNS_SPI_CR_BAUD_DIV_MASK 0x00000038 /* Baud Rate Divisor Mask */
-#define CDNS_SPI_CR_MSTREN_MASK 0x00000001 /* Master Enable Mask */
-#define CDNS_SPI_CR_MANSTRTEN_MASK 0x00008000 /* Manual TX Enable Mask */
-#define CDNS_SPI_CR_SSFORCE_MASK 0x00004000 /* Manual SS Enable Mask */
-#define CDNS_SPI_CR_BAUD_DIV_4_MASK 0x00000008 /* Default Baud Div Mask */
-#define CDNS_SPI_CR_DEFAULT_MASK (CDNS_SPI_CR_MSTREN_MASK | \
- CDNS_SPI_CR_SSCTRL_MASK | \
- CDNS_SPI_CR_SSFORCE_MASK | \
- CDNS_SPI_CR_BAUD_DIV_4_MASK)
+#define CDNS_SPI_CR_MANSTRT 0x00010000 /* Manual TX Start */
+#define CDNS_SPI_CR_CPHA 0x00000004 /* Clock Phase Control */
+#define CDNS_SPI_CR_CPOL 0x00000002 /* Clock Polarity Control */
+#define CDNS_SPI_CR_SSCTRL 0x00003C00 /* Slave Select Mask */
+#define CDNS_SPI_CR_PERI_SEL 0x00000200 /* Peripheral Select Decode */
+#define CDNS_SPI_CR_BAUD_DIV 0x00000038 /* Baud Rate Divisor Mask */
+#define CDNS_SPI_CR_MSTREN 0x00000001 /* Master Enable Mask */
+#define CDNS_SPI_CR_MANSTRTEN 0x00008000 /* Manual TX Enable Mask */
+#define CDNS_SPI_CR_SSFORCE 0x00004000 /* Manual SS Enable Mask */
+#define CDNS_SPI_CR_BAUD_DIV_4 0x00000008 /* Default Baud Div Mask */
+#define CDNS_SPI_CR_DEFAULT (CDNS_SPI_CR_MSTREN | \
+ CDNS_SPI_CR_SSCTRL | \
+ CDNS_SPI_CR_SSFORCE | \
+ CDNS_SPI_CR_BAUD_DIV_4)
/*
* SPI Configuration Register - Baud rate and slave select
@@ -77,21 +79,21 @@
* All the four interrupt registers (Status/Mask/Enable/Disable) have the same
* bit definitions.
*/
-#define CDNS_SPI_IXR_TXOW_MASK 0x00000004 /* SPI TX FIFO Overwater */
-#define CDNS_SPI_IXR_MODF_MASK 0x00000002 /* SPI Mode Fault */
-#define CDNS_SPI_IXR_RXNEMTY_MASK 0x00000010 /* SPI RX FIFO Not Empty */
-#define CDNS_SPI_IXR_DEFAULT_MASK (CDNS_SPI_IXR_TXOW_MASK | \
- CDNS_SPI_IXR_MODF_MASK)
-#define CDNS_SPI_IXR_TXFULL_MASK 0x00000008 /* SPI TX Full */
-#define CDNS_SPI_IXR_ALL_MASK 0x0000007F /* SPI all interrupts */
+#define CDNS_SPI_IXR_TXOW 0x00000004 /* SPI TX FIFO Overwater */
+#define CDNS_SPI_IXR_MODF 0x00000002 /* SPI Mode Fault */
+#define CDNS_SPI_IXR_RXNEMTY 0x00000010 /* SPI RX FIFO Not Empty */
+#define CDNS_SPI_IXR_DEFAULT (CDNS_SPI_IXR_TXOW | \
+ CDNS_SPI_IXR_MODF)
+#define CDNS_SPI_IXR_TXFULL 0x00000008 /* SPI TX Full */
+#define CDNS_SPI_IXR_ALL 0x0000007F /* SPI all interrupts */
/*
* SPI Enable Register bit Masks
*
* This register is used to enable or disable the SPI controller
*/
-#define CDNS_SPI_ER_ENABLE_MASK 0x00000001 /* SPI Enable Bit Mask */
-#define CDNS_SPI_ER_DISABLE_MASK 0x0 /* SPI Disable Bit Mask */
+#define CDNS_SPI_ER_ENABLE 0x00000001 /* SPI Enable Bit Mask */
+#define CDNS_SPI_ER_DISABLE 0x0 /* SPI Disable Bit Mask */
/* SPI FIFO depth in bytes */
#define CDNS_SPI_FIFO_DEPTH 128
@@ -149,56 +151,51 @@
*/
static void cdns_spi_init_hw(struct cdns_spi *xspi)
{
- u32 ctrl_reg = CDNS_SPI_CR_DEFAULT_MASK;
+ u32 ctrl_reg = CDNS_SPI_CR_DEFAULT;
if (xspi->is_decoded_cs)
- ctrl_reg |= CDNS_SPI_CR_PERI_SEL_MASK;
+ ctrl_reg |= CDNS_SPI_CR_PERI_SEL;
- cdns_spi_write(xspi, CDNS_SPI_ER_OFFSET,
- CDNS_SPI_ER_DISABLE_MASK);
- cdns_spi_write(xspi, CDNS_SPI_IDR_OFFSET,
- CDNS_SPI_IXR_ALL_MASK);
+ cdns_spi_write(xspi, CDNS_SPI_ER, CDNS_SPI_ER_DISABLE);
+ cdns_spi_write(xspi, CDNS_SPI_IDR, CDNS_SPI_IXR_ALL);
/* Clear the RX FIFO */
- while (cdns_spi_read(xspi, CDNS_SPI_ISR_OFFSET) &
- CDNS_SPI_IXR_RXNEMTY_MASK)
- cdns_spi_read(xspi, CDNS_SPI_RXD_OFFSET);
+ while (cdns_spi_read(xspi, CDNS_SPI_ISR) & CDNS_SPI_IXR_RXNEMTY)
+ cdns_spi_read(xspi, CDNS_SPI_RXD);
- cdns_spi_write(xspi, CDNS_SPI_ISR_OFFSET,
- CDNS_SPI_IXR_ALL_MASK);
- cdns_spi_write(xspi, CDNS_SPI_CR_OFFSET, ctrl_reg);
- cdns_spi_write(xspi, CDNS_SPI_ER_OFFSET,
- CDNS_SPI_ER_ENABLE_MASK);
+ cdns_spi_write(xspi, CDNS_SPI_ISR, CDNS_SPI_IXR_ALL);
+ cdns_spi_write(xspi, CDNS_SPI_CR, ctrl_reg);
+ cdns_spi_write(xspi, CDNS_SPI_ER, CDNS_SPI_ER_ENABLE);
}
/**
* cdns_spi_chipselect - Select or deselect the chip select line
* @spi: Pointer to the spi_device structure
- * @is_on: Select(0) or deselect (1) the chip select line
+ * @is_high: Select(0) or deselect (1) the chip select line
*/
static void cdns_spi_chipselect(struct spi_device *spi, bool is_high)
{
struct cdns_spi *xspi = spi_master_get_devdata(spi->master);
u32 ctrl_reg;
- ctrl_reg = cdns_spi_read(xspi, CDNS_SPI_CR_OFFSET);
+ ctrl_reg = cdns_spi_read(xspi, CDNS_SPI_CR);
if (is_high) {
/* Deselect the slave */
- ctrl_reg |= CDNS_SPI_CR_SSCTRL_MASK;
+ ctrl_reg |= CDNS_SPI_CR_SSCTRL;
} else {
/* Select the slave */
- ctrl_reg &= ~CDNS_SPI_CR_SSCTRL_MASK;
+ ctrl_reg &= ~CDNS_SPI_CR_SSCTRL;
if (!(xspi->is_decoded_cs))
ctrl_reg |= ((~(CDNS_SPI_SS0 << spi->chip_select)) <<
CDNS_SPI_SS_SHIFT) &
- CDNS_SPI_CR_SSCTRL_MASK;
+ CDNS_SPI_CR_SSCTRL;
else
ctrl_reg |= (spi->chip_select << CDNS_SPI_SS_SHIFT) &
- CDNS_SPI_CR_SSCTRL_MASK;
+ CDNS_SPI_CR_SSCTRL;
}
- cdns_spi_write(xspi, CDNS_SPI_CR_OFFSET, ctrl_reg);
+ cdns_spi_write(xspi, CDNS_SPI_CR, ctrl_reg);
}
/**
@@ -212,14 +209,15 @@
struct cdns_spi *xspi = spi_master_get_devdata(spi->master);
u32 ctrl_reg, new_ctrl_reg;
- new_ctrl_reg = ctrl_reg = cdns_spi_read(xspi, CDNS_SPI_CR_OFFSET);
+ new_ctrl_reg = cdns_spi_read(xspi, CDNS_SPI_CR);
+ ctrl_reg = new_ctrl_reg;
/* Set the SPI clock phase and clock polarity */
- new_ctrl_reg &= ~(CDNS_SPI_CR_CPHA_MASK | CDNS_SPI_CR_CPOL_MASK);
+ new_ctrl_reg &= ~(CDNS_SPI_CR_CPHA | CDNS_SPI_CR_CPOL);
if (spi->mode & SPI_CPHA)
- new_ctrl_reg |= CDNS_SPI_CR_CPHA_MASK;
+ new_ctrl_reg |= CDNS_SPI_CR_CPHA;
if (spi->mode & SPI_CPOL)
- new_ctrl_reg |= CDNS_SPI_CR_CPOL_MASK;
+ new_ctrl_reg |= CDNS_SPI_CR_CPOL;
if (new_ctrl_reg != ctrl_reg) {
/*
@@ -228,11 +226,9 @@
* polarity as it will cause the SPI slave to see spurious clock
* transitions. To workaround the issue toggle the ER register.
*/
- cdns_spi_write(xspi, CDNS_SPI_ER_OFFSET,
- CDNS_SPI_ER_DISABLE_MASK);
- cdns_spi_write(xspi, CDNS_SPI_CR_OFFSET, new_ctrl_reg);
- cdns_spi_write(xspi, CDNS_SPI_ER_OFFSET,
- CDNS_SPI_ER_ENABLE_MASK);
+ cdns_spi_write(xspi, CDNS_SPI_ER, CDNS_SPI_ER_DISABLE);
+ cdns_spi_write(xspi, CDNS_SPI_CR, new_ctrl_reg);
+ cdns_spi_write(xspi, CDNS_SPI_ER, CDNS_SPI_ER_ENABLE);
}
}
@@ -251,7 +247,7 @@
* controller.
*/
static void cdns_spi_config_clock_freq(struct spi_device *spi,
- struct spi_transfer *transfer)
+ struct spi_transfer *transfer)
{
struct cdns_spi *xspi = spi_master_get_devdata(spi->master);
u32 ctrl_reg, baud_rate_val;
@@ -259,7 +255,7 @@
frequency = clk_get_rate(xspi->ref_clk);
- ctrl_reg = cdns_spi_read(xspi, CDNS_SPI_CR_OFFSET);
+ ctrl_reg = cdns_spi_read(xspi, CDNS_SPI_CR);
/* Set the clock frequency */
if (xspi->speed_hz != transfer->speed_hz) {
@@ -269,12 +265,12 @@
(frequency / (2 << baud_rate_val)) > transfer->speed_hz)
baud_rate_val++;
- ctrl_reg &= ~CDNS_SPI_CR_BAUD_DIV_MASK;
+ ctrl_reg &= ~CDNS_SPI_CR_BAUD_DIV;
ctrl_reg |= baud_rate_val << CDNS_SPI_BAUD_DIV_SHIFT;
xspi->speed_hz = frequency / (2 << baud_rate_val);
}
- cdns_spi_write(xspi, CDNS_SPI_CR_OFFSET, ctrl_reg);
+ cdns_spi_write(xspi, CDNS_SPI_CR, ctrl_reg);
}
/**
@@ -313,10 +309,9 @@
while ((trans_cnt < CDNS_SPI_FIFO_DEPTH) &&
(xspi->tx_bytes > 0)) {
if (xspi->txbuf)
- cdns_spi_write(xspi, CDNS_SPI_TXD_OFFSET,
- *xspi->txbuf++);
+ cdns_spi_write(xspi, CDNS_SPI_TXD, *xspi->txbuf++);
else
- cdns_spi_write(xspi, CDNS_SPI_TXD_OFFSET, 0);
+ cdns_spi_write(xspi, CDNS_SPI_TXD, 0);
xspi->tx_bytes--;
trans_cnt++;
@@ -344,19 +339,18 @@
u32 intr_status, status;
status = IRQ_NONE;
- intr_status = cdns_spi_read(xspi, CDNS_SPI_ISR_OFFSET);
- cdns_spi_write(xspi, CDNS_SPI_ISR_OFFSET, intr_status);
+ intr_status = cdns_spi_read(xspi, CDNS_SPI_ISR);
+ cdns_spi_write(xspi, CDNS_SPI_ISR, intr_status);
- if (intr_status & CDNS_SPI_IXR_MODF_MASK) {
+ if (intr_status & CDNS_SPI_IXR_MODF) {
/* Indicate that transfer is completed, the SPI subsystem will
* identify the error as the remaining bytes to be
* transferred is non-zero
*/
- cdns_spi_write(xspi, CDNS_SPI_IDR_OFFSET,
- CDNS_SPI_IXR_DEFAULT_MASK);
+ cdns_spi_write(xspi, CDNS_SPI_IDR, CDNS_SPI_IXR_DEFAULT);
spi_finalize_current_transfer(master);
status = IRQ_HANDLED;
- } else if (intr_status & CDNS_SPI_IXR_TXOW_MASK) {
+ } else if (intr_status & CDNS_SPI_IXR_TXOW) {
unsigned long trans_cnt;
trans_cnt = xspi->rx_bytes - xspi->tx_bytes;
@@ -365,7 +359,7 @@
while (trans_cnt) {
u8 data;
- data = cdns_spi_read(xspi, CDNS_SPI_RXD_OFFSET);
+ data = cdns_spi_read(xspi, CDNS_SPI_RXD);
if (xspi->rxbuf)
*xspi->rxbuf++ = data;
@@ -378,8 +372,8 @@
cdns_spi_fill_tx_fifo(xspi);
} else {
/* Transfer is completed */
- cdns_spi_write(xspi, CDNS_SPI_IDR_OFFSET,
- CDNS_SPI_IXR_DEFAULT_MASK);
+ cdns_spi_write(xspi, CDNS_SPI_IDR,
+ CDNS_SPI_IXR_DEFAULT);
spi_finalize_current_transfer(master);
}
status = IRQ_HANDLED;
@@ -387,6 +381,7 @@
return status;
}
+
static int cdns_prepare_message(struct spi_master *master,
struct spi_message *msg)
{
@@ -421,8 +416,7 @@
cdns_spi_fill_tx_fifo(xspi);
- cdns_spi_write(xspi, CDNS_SPI_IER_OFFSET,
- CDNS_SPI_IXR_DEFAULT_MASK);
+ cdns_spi_write(xspi, CDNS_SPI_IER, CDNS_SPI_IXR_DEFAULT);
return transfer->len;
}
@@ -439,8 +433,7 @@
{
struct cdns_spi *xspi = spi_master_get_devdata(master);
- cdns_spi_write(xspi, CDNS_SPI_ER_OFFSET,
- CDNS_SPI_ER_ENABLE_MASK);
+ cdns_spi_write(xspi, CDNS_SPI_ER, CDNS_SPI_ER_ENABLE);
return 0;
}
@@ -458,8 +451,7 @@
{
struct cdns_spi *xspi = spi_master_get_devdata(master);
- cdns_spi_write(xspi, CDNS_SPI_ER_OFFSET,
- CDNS_SPI_ER_DISABLE_MASK);
+ cdns_spi_write(xspi, CDNS_SPI_ER, CDNS_SPI_ER_DISABLE);
return 0;
}
@@ -481,7 +473,7 @@
u32 num_cs;
master = spi_alloc_master(&pdev->dev, sizeof(*xspi));
- if (master == NULL)
+ if (!master)
return -ENOMEM;
xspi = spi_master_get_devdata(master);
@@ -521,6 +513,11 @@
goto clk_dis_apb;
}
+ pm_runtime_enable(&pdev->dev);
+ pm_runtime_use_autosuspend(&pdev->dev);
+ pm_runtime_set_autosuspend_delay(&pdev->dev, SPI_AUTOSUSPEND_TIMEOUT);
+ pm_runtime_set_active(&pdev->dev);
+
ret = of_property_read_u32(pdev->dev.of_node, "num-cs", &num_cs);
if (ret < 0)
master->num_chipselect = CDNS_SPI_DEFAULT_NUM_CS;
@@ -535,11 +532,14 @@
/* SPI controller initializations */
cdns_spi_init_hw(xspi);
+ pm_runtime_mark_last_busy(&pdev->dev);
+ pm_runtime_put_autosuspend(&pdev->dev);
+
irq = platform_get_irq(pdev, 0);
if (irq <= 0) {
ret = -ENXIO;
dev_err(&pdev->dev, "irq number is invalid\n");
- goto remove_master;
+ goto clk_dis_all;
}
ret = devm_request_irq(&pdev->dev, irq, cdns_spi_irq,
@@ -547,7 +547,7 @@
if (ret != 0) {
ret = -ENXIO;
dev_err(&pdev->dev, "request_irq failed\n");
- goto remove_master;
+ goto clk_dis_all;
}
master->prepare_transfer_hardware = cdns_prepare_transfer_hardware;
@@ -555,6 +555,7 @@
master->transfer_one = cdns_transfer_one;
master->unprepare_transfer_hardware = cdns_unprepare_transfer_hardware;
master->set_cs = cdns_spi_chipselect;
+ master->auto_runtime_pm = true;
master->mode_bits = SPI_CPOL | SPI_CPHA;
/* Set to default valid value */
@@ -572,6 +573,8 @@
return ret;
clk_dis_all:
+ pm_runtime_set_suspended(&pdev->dev);
+ pm_runtime_disable(&pdev->dev);
clk_disable_unprepare(xspi->ref_clk);
clk_dis_apb:
clk_disable_unprepare(xspi->pclk);
@@ -595,11 +598,12 @@
struct spi_master *master = platform_get_drvdata(pdev);
struct cdns_spi *xspi = spi_master_get_devdata(master);
- cdns_spi_write(xspi, CDNS_SPI_ER_OFFSET,
- CDNS_SPI_ER_DISABLE_MASK);
+ cdns_spi_write(xspi, CDNS_SPI_ER, CDNS_SPI_ER_DISABLE);
clk_disable_unprepare(xspi->ref_clk);
clk_disable_unprepare(xspi->pclk);
+ pm_runtime_set_suspended(&pdev->dev);
+ pm_runtime_disable(&pdev->dev);
spi_unregister_master(master);
@@ -613,21 +617,14 @@
* This function disables the SPI controller and
* changes the driver state to "suspend"
*
- * Return: Always 0
+ * Return: 0 on success and error value on error
*/
static int __maybe_unused cdns_spi_suspend(struct device *dev)
{
struct platform_device *pdev = to_platform_device(dev);
struct spi_master *master = platform_get_drvdata(pdev);
- struct cdns_spi *xspi = spi_master_get_devdata(master);
- spi_master_suspend(master);
-
- clk_disable_unprepare(xspi->ref_clk);
-
- clk_disable_unprepare(xspi->pclk);
-
- return 0;
+ return spi_master_suspend(master);
}
/**
@@ -642,8 +639,23 @@
{
struct platform_device *pdev = to_platform_device(dev);
struct spi_master *master = platform_get_drvdata(pdev);
+
+ return spi_master_resume(master);
+}
+
+/**
+ * cdns_spi_runtime_resume - Runtime resume method for the SPI driver
+ * @dev: Address of the platform_device structure
+ *
+ * This function enables the clocks
+ *
+ * Return: 0 on success and error value on error
+ */
+static int __maybe_unused cnds_runtime_resume(struct device *dev)
+{
+ struct spi_master *master = dev_get_drvdata(dev);
struct cdns_spi *xspi = spi_master_get_devdata(master);
- int ret = 0;
+ int ret;
ret = clk_prepare_enable(xspi->pclk);
if (ret) {
@@ -657,13 +669,33 @@
clk_disable(xspi->pclk);
return ret;
}
- spi_master_resume(master);
+ return 0;
+}
+
+/**
+ * cdns_spi_runtime_suspend - Runtime suspend method for the SPI driver
+ * @dev: Address of the platform_device structure
+ *
+ * This function disables the clocks
+ *
+ * Return: Always 0
+ */
+static int __maybe_unused cnds_runtime_suspend(struct device *dev)
+{
+ struct spi_master *master = dev_get_drvdata(dev);
+ struct cdns_spi *xspi = spi_master_get_devdata(master);
+
+ clk_disable_unprepare(xspi->ref_clk);
+ clk_disable_unprepare(xspi->pclk);
return 0;
}
-static SIMPLE_DEV_PM_OPS(cdns_spi_dev_pm_ops, cdns_spi_suspend,
- cdns_spi_resume);
+static const struct dev_pm_ops cdns_spi_dev_pm_ops = {
+ SET_RUNTIME_PM_OPS(cnds_runtime_suspend,
+ cnds_runtime_resume, NULL)
+ SET_SYSTEM_SLEEP_PM_OPS(cdns_spi_suspend, cdns_spi_resume)
+};
static const struct of_device_id cdns_spi_of_match[] = {
{ .compatible = "xlnx,zynq-spi-r1p6" },
diff --git a/drivers/spi/spi-davinci.c b/drivers/spi/spi-davinci.c
index fddb7a3..d36c11b 100644
--- a/drivers/spi/spi-davinci.c
+++ b/drivers/spi/spi-davinci.c
@@ -23,7 +23,6 @@
#include <linux/clk.h>
#include <linux/dmaengine.h>
#include <linux/dma-mapping.h>
-#include <linux/edma.h>
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/of_gpio.h>
@@ -33,8 +32,6 @@
#include <linux/platform_data/spi-davinci.h>
-#define SPI_NO_RESOURCE ((resource_size_t)-1)
-
#define CS_DEFAULT 0xFF
#define SPIFMT_PHASE_MASK BIT(16)
@@ -130,8 +127,6 @@
struct dma_chan *dma_rx;
struct dma_chan *dma_tx;
- int dma_rx_chnum;
- int dma_tx_chnum;
struct davinci_spi_platform_data pdata;
@@ -797,35 +792,19 @@
static int davinci_spi_request_dma(struct davinci_spi *dspi)
{
- dma_cap_mask_t mask;
struct device *sdev = dspi->bitbang.master->dev.parent;
- int r;
- dma_cap_zero(mask);
- dma_cap_set(DMA_SLAVE, mask);
+ dspi->dma_rx = dma_request_chan(sdev, "rx");
+ if (IS_ERR(dspi->dma_rx))
+ return PTR_ERR(dspi->dma_rx);
- dspi->dma_rx = dma_request_channel(mask, edma_filter_fn,
- &dspi->dma_rx_chnum);
- if (!dspi->dma_rx) {
- dev_err(sdev, "request RX DMA channel failed\n");
- r = -ENODEV;
- goto rx_dma_failed;
- }
-
- dspi->dma_tx = dma_request_channel(mask, edma_filter_fn,
- &dspi->dma_tx_chnum);
- if (!dspi->dma_tx) {
- dev_err(sdev, "request TX DMA channel failed\n");
- r = -ENODEV;
- goto tx_dma_failed;
+ dspi->dma_tx = dma_request_chan(sdev, "tx");
+ if (IS_ERR(dspi->dma_tx)) {
+ dma_release_channel(dspi->dma_rx);
+ return PTR_ERR(dspi->dma_tx);
}
return 0;
-
-tx_dma_failed:
- dma_release_channel(dspi->dma_rx);
-rx_dma_failed:
- return r;
}
#if defined(CONFIG_OF)
@@ -936,8 +915,6 @@
struct davinci_spi *dspi;
struct davinci_spi_platform_data *pdata;
struct resource *r;
- resource_size_t dma_rx_chan = SPI_NO_RESOURCE;
- resource_size_t dma_tx_chan = SPI_NO_RESOURCE;
int ret = 0;
u32 spipc0;
@@ -1044,27 +1021,15 @@
}
}
- r = platform_get_resource(pdev, IORESOURCE_DMA, 0);
- if (r)
- dma_rx_chan = r->start;
- r = platform_get_resource(pdev, IORESOURCE_DMA, 1);
- if (r)
- dma_tx_chan = r->start;
-
dspi->bitbang.txrx_bufs = davinci_spi_bufs;
- if (dma_rx_chan != SPI_NO_RESOURCE &&
- dma_tx_chan != SPI_NO_RESOURCE) {
- dspi->dma_rx_chnum = dma_rx_chan;
- dspi->dma_tx_chnum = dma_tx_chan;
- ret = davinci_spi_request_dma(dspi);
- if (ret)
- goto free_clk;
-
- dev_info(&pdev->dev, "DMA: supported\n");
- dev_info(&pdev->dev, "DMA: RX channel: %pa, TX channel: %pa, event queue: %d\n",
- &dma_rx_chan, &dma_tx_chan,
- pdata->dma_event_q);
+ ret = davinci_spi_request_dma(dspi);
+ if (ret == -EPROBE_DEFER) {
+ goto free_clk;
+ } else if (ret) {
+ dev_info(&pdev->dev, "DMA is not supported (%d)\n", ret);
+ dspi->dma_rx = NULL;
+ dspi->dma_tx = NULL;
}
dspi->get_rx = davinci_spi_rx_buf_u8;
@@ -1102,8 +1067,10 @@
return ret;
free_dma:
- dma_release_channel(dspi->dma_rx);
- dma_release_channel(dspi->dma_tx);
+ if (dspi->dma_rx) {
+ dma_release_channel(dspi->dma_rx);
+ dma_release_channel(dspi->dma_tx);
+ }
free_clk:
clk_disable_unprepare(dspi->clk);
free_master:
@@ -1134,6 +1101,11 @@
clk_disable_unprepare(dspi->clk);
spi_master_put(master);
+ if (dspi->dma_rx) {
+ dma_release_channel(dspi->dma_rx);
+ dma_release_channel(dspi->dma_tx);
+ }
+
return 0;
}
diff --git a/drivers/spi/spi-dln2.c b/drivers/spi/spi-dln2.c
index 3b7d91d..b62a99c 100644
--- a/drivers/spi/spi-dln2.c
+++ b/drivers/spi/spi-dln2.c
@@ -683,6 +683,7 @@
struct spi_master *master;
struct dln2_spi *dln2;
struct dln2_platform_data *pdata = dev_get_platdata(&pdev->dev);
+ struct device *dev = &pdev->dev;
int ret;
master = spi_alloc_master(&pdev->dev, sizeof(*dln2));
@@ -700,6 +701,7 @@
}
dln2->master = master;
+ dln2->master->dev.of_node = dev->of_node;
dln2->pdev = pdev;
dln2->port = pdata->port;
/* cs/mode can never be 0xff, so the first transfer will set them */
diff --git a/drivers/spi/spi-dw-pci.c b/drivers/spi/spi-dw-pci.c
index 332ccb0..ef7db75 100644
--- a/drivers/spi/spi-dw-pci.c
+++ b/drivers/spi/spi-dw-pci.c
@@ -67,7 +67,7 @@
dws->irq = pdev->irq;
/*
- * Specific handling for paltforms, like dma setup,
+ * Specific handling for platforms, like dma setup,
* clock rate, FIFO depth.
*/
if (desc) {
diff --git a/drivers/spi/spi-fsl-dspi.c b/drivers/spi/spi-fsl-dspi.c
index c1a2d74..9e9dadb 100644
--- a/drivers/spi/spi-fsl-dspi.c
+++ b/drivers/spi/spi-fsl-dspi.c
@@ -121,18 +121,22 @@
struct fsl_dspi_devtype_data {
enum dspi_trans_mode trans_mode;
+ u8 max_clock_factor;
};
static const struct fsl_dspi_devtype_data vf610_data = {
.trans_mode = DSPI_EOQ_MODE,
+ .max_clock_factor = 2,
};
static const struct fsl_dspi_devtype_data ls1021a_v1_data = {
.trans_mode = DSPI_TCFQ_MODE,
+ .max_clock_factor = 8,
};
static const struct fsl_dspi_devtype_data ls2085a_data = {
.trans_mode = DSPI_TCFQ_MODE,
+ .max_clock_factor = 8,
};
struct fsl_dspi {
@@ -726,6 +730,9 @@
}
clk_prepare_enable(dspi->clk);
+ master->max_speed_hz =
+ clk_get_rate(dspi->clk) / dspi->devtype_data->max_clock_factor;
+
init_waitqueue_head(&dspi->waitq);
platform_set_drvdata(pdev, master);
diff --git a/drivers/spi/spi-fsl-espi.c b/drivers/spi/spi-fsl-espi.c
index 7cb0c19..8d85a3c 100644
--- a/drivers/spi/spi-fsl-espi.c
+++ b/drivers/spi/spi-fsl-espi.c
@@ -245,7 +245,12 @@
if (ret)
return ret;
- wait_for_completion(&mpc8xxx_spi->done);
+ /* Won't hang up forever, SPI bus sometimes got lost interrupts... */
+ ret = wait_for_completion_timeout(&mpc8xxx_spi->done, 2 * HZ);
+ if (ret == 0)
+ dev_err(mpc8xxx_spi->dev,
+ "Transaction hanging up (left %d bytes)\n",
+ mpc8xxx_spi->count);
/* disable rx ints */
mpc8xxx_spi_write_reg(®_base->mask, 0);
@@ -539,16 +544,31 @@
if (events & SPIE_NE) {
u32 rx_data, tmp;
u8 rx_data_8;
+ int rx_nr_bytes = 4;
+ int ret;
/* Spin until RX is done */
- while (SPIE_RXCNT(events) < min(4, mspi->len)) {
- cpu_relax();
- events = mpc8xxx_spi_read_reg(®_base->event);
+ if (SPIE_RXCNT(events) < min(4, mspi->len)) {
+ ret = spin_event_timeout(
+ !(SPIE_RXCNT(events =
+ mpc8xxx_spi_read_reg(®_base->event)) <
+ min(4, mspi->len)),
+ 10000, 0); /* 10 msec */
+ if (!ret)
+ dev_err(mspi->dev,
+ "tired waiting for SPIE_RXCNT\n");
}
if (mspi->len >= 4) {
rx_data = mpc8xxx_spi_read_reg(®_base->receive);
+ } else if (mspi->len <= 0) {
+ dev_err(mspi->dev,
+ "unexpected RX(SPIE_NE) interrupt occurred,\n"
+ "(local rxlen %d bytes, reg rxlen %d bytes)\n",
+ min(4, mspi->len), SPIE_RXCNT(events));
+ rx_nr_bytes = 0;
} else {
+ rx_nr_bytes = mspi->len;
tmp = mspi->len;
rx_data = 0;
while (tmp--) {
@@ -559,7 +579,7 @@
rx_data <<= (4 - mspi->len) * 8;
}
- mspi->len -= 4;
+ mspi->len -= rx_nr_bytes;
if (mspi->rx)
mspi->get_rx(rx_data, mspi);
diff --git a/drivers/spi/spi-octeon.c b/drivers/spi/spi-octeon.c
index 07e4ce8..3b17009 100644
--- a/drivers/spi/spi-octeon.c
+++ b/drivers/spi/spi-octeon.c
@@ -175,6 +175,7 @@
static int octeon_spi_probe(struct platform_device *pdev)
{
struct resource *res_mem;
+ void __iomem *reg_base;
struct spi_master *master;
struct octeon_spi *p;
int err = -ENOENT;
@@ -186,19 +187,13 @@
platform_set_drvdata(pdev, master);
res_mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ reg_base = devm_ioremap_resource(&pdev->dev, res_mem);
+ if (IS_ERR(reg_base)) {
+ err = PTR_ERR(reg_base);
+ goto fail;
+ }
- if (res_mem == NULL) {
- dev_err(&pdev->dev, "found no memory resource\n");
- err = -ENXIO;
- goto fail;
- }
- if (!devm_request_mem_region(&pdev->dev, res_mem->start,
- resource_size(res_mem), res_mem->name)) {
- dev_err(&pdev->dev, "request_mem_region failed\n");
- goto fail;
- }
- p->register_base = (u64)devm_ioremap(&pdev->dev, res_mem->start,
- resource_size(res_mem));
+ p->register_base = (u64)reg_base;
master->num_chipselect = 4;
master->mode_bits = SPI_CPHA |
diff --git a/drivers/spi/spi-omap2-mcspi.c b/drivers/spi/spi-omap2-mcspi.c
index 0caa3c8..1d237e9 100644
--- a/drivers/spi/spi-omap2-mcspi.c
+++ b/drivers/spi/spi-omap2-mcspi.c
@@ -23,7 +23,6 @@
#include <linux/delay.h>
#include <linux/dma-mapping.h>
#include <linux/dmaengine.h>
-#include <linux/omap-dma.h>
#include <linux/pinctrl/consumer.h>
#include <linux/platform_device.h>
#include <linux/err.h>
@@ -103,9 +102,6 @@
struct dma_chan *dma_tx;
struct dma_chan *dma_rx;
- int dma_tx_sync_dev;
- int dma_rx_sync_dev;
-
struct completion dma_tx_completion;
struct completion dma_rx_completion;
@@ -964,8 +960,7 @@
struct spi_master *master = spi->master;
struct omap2_mcspi *mcspi;
struct omap2_mcspi_dma *mcspi_dma;
- dma_cap_mask_t mask;
- unsigned sig;
+ int ret = 0;
mcspi = spi_master_get_devdata(master);
mcspi_dma = mcspi->dma_channels + spi->chip_select;
@@ -973,34 +968,25 @@
init_completion(&mcspi_dma->dma_rx_completion);
init_completion(&mcspi_dma->dma_tx_completion);
- dma_cap_zero(mask);
- dma_cap_set(DMA_SLAVE, mask);
- sig = mcspi_dma->dma_rx_sync_dev;
-
- mcspi_dma->dma_rx =
- dma_request_slave_channel_compat(mask, omap_dma_filter_fn,
- &sig, &master->dev,
- mcspi_dma->dma_rx_ch_name);
- if (!mcspi_dma->dma_rx)
- goto no_dma;
-
- sig = mcspi_dma->dma_tx_sync_dev;
- mcspi_dma->dma_tx =
- dma_request_slave_channel_compat(mask, omap_dma_filter_fn,
- &sig, &master->dev,
- mcspi_dma->dma_tx_ch_name);
-
- if (!mcspi_dma->dma_tx) {
- dma_release_channel(mcspi_dma->dma_rx);
+ mcspi_dma->dma_rx = dma_request_chan(&master->dev,
+ mcspi_dma->dma_rx_ch_name);
+ if (IS_ERR(mcspi_dma->dma_rx)) {
+ ret = PTR_ERR(mcspi_dma->dma_rx);
mcspi_dma->dma_rx = NULL;
goto no_dma;
}
- return 0;
+ mcspi_dma->dma_tx = dma_request_chan(&master->dev,
+ mcspi_dma->dma_tx_ch_name);
+ if (IS_ERR(mcspi_dma->dma_tx)) {
+ ret = PTR_ERR(mcspi_dma->dma_tx);
+ mcspi_dma->dma_tx = NULL;
+ dma_release_channel(mcspi_dma->dma_rx);
+ mcspi_dma->dma_rx = NULL;
+ }
no_dma:
- dev_warn(&spi->dev, "not using DMA for McSPI\n");
- return -EAGAIN;
+ return ret;
}
static int omap2_mcspi_setup(struct spi_device *spi)
@@ -1039,8 +1025,9 @@
if (!mcspi_dma->dma_rx || !mcspi_dma->dma_tx) {
ret = omap2_mcspi_request_dma(spi);
- if (ret < 0 && ret != -EAGAIN)
- return ret;
+ if (ret)
+ dev_warn(&spi->dev, "not using DMA for McSPI (%d)\n",
+ ret);
}
ret = pm_runtime_get_sync(mcspi->dev);
@@ -1434,42 +1421,8 @@
}
for (i = 0; i < master->num_chipselect; i++) {
- char *dma_rx_ch_name = mcspi->dma_channels[i].dma_rx_ch_name;
- char *dma_tx_ch_name = mcspi->dma_channels[i].dma_tx_ch_name;
- struct resource *dma_res;
-
- sprintf(dma_rx_ch_name, "rx%d", i);
- if (!pdev->dev.of_node) {
- dma_res =
- platform_get_resource_byname(pdev,
- IORESOURCE_DMA,
- dma_rx_ch_name);
- if (!dma_res) {
- dev_dbg(&pdev->dev,
- "cannot get DMA RX channel\n");
- status = -ENODEV;
- break;
- }
-
- mcspi->dma_channels[i].dma_rx_sync_dev =
- dma_res->start;
- }
- sprintf(dma_tx_ch_name, "tx%d", i);
- if (!pdev->dev.of_node) {
- dma_res =
- platform_get_resource_byname(pdev,
- IORESOURCE_DMA,
- dma_tx_ch_name);
- if (!dma_res) {
- dev_dbg(&pdev->dev,
- "cannot get DMA TX channel\n");
- status = -ENODEV;
- break;
- }
-
- mcspi->dma_channels[i].dma_tx_sync_dev =
- dma_res->start;
- }
+ sprintf(mcspi->dma_channels[i].dma_rx_ch_name, "rx%d", i);
+ sprintf(mcspi->dma_channels[i].dma_tx_ch_name, "tx%d", i);
}
if (status < 0)
diff --git a/drivers/spi/spi-pic32-sqi.c b/drivers/spi/spi-pic32-sqi.c
new file mode 100644
index 0000000..ca3c8d9
--- /dev/null
+++ b/drivers/spi/spi-pic32-sqi.c
@@ -0,0 +1,727 @@
+/*
+ * PIC32 Quad SPI controller driver.
+ *
+ * Purna Chandra Mandal <purna.mandal@microchip.com>
+ * Copyright (c) 2016, Microchip Technology Inc.
+ *
+ * This program is free software; you can distribute it and/or modify it
+ * under the terms of the GNU General Public License (Version 2) as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+ * for more details.
+ */
+
+#include <linux/clk.h>
+#include <linux/dma-mapping.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/iopoll.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+#include <linux/spi/spi.h>
+
+/* SQI registers */
+#define PESQI_XIP_CONF1_REG 0x00
+#define PESQI_XIP_CONF2_REG 0x04
+#define PESQI_CONF_REG 0x08
+#define PESQI_CTRL_REG 0x0C
+#define PESQI_CLK_CTRL_REG 0x10
+#define PESQI_CMD_THRES_REG 0x14
+#define PESQI_INT_THRES_REG 0x18
+#define PESQI_INT_ENABLE_REG 0x1C
+#define PESQI_INT_STAT_REG 0x20
+#define PESQI_TX_DATA_REG 0x24
+#define PESQI_RX_DATA_REG 0x28
+#define PESQI_STAT1_REG 0x2C
+#define PESQI_STAT2_REG 0x30
+#define PESQI_BD_CTRL_REG 0x34
+#define PESQI_BD_CUR_ADDR_REG 0x38
+#define PESQI_BD_BASE_ADDR_REG 0x40
+#define PESQI_BD_STAT_REG 0x44
+#define PESQI_BD_POLL_CTRL_REG 0x48
+#define PESQI_BD_TX_DMA_STAT_REG 0x4C
+#define PESQI_BD_RX_DMA_STAT_REG 0x50
+#define PESQI_THRES_REG 0x54
+#define PESQI_INT_SIGEN_REG 0x58
+
+/* PESQI_CONF_REG fields */
+#define PESQI_MODE 0x7
+#define PESQI_MODE_BOOT 0
+#define PESQI_MODE_PIO 1
+#define PESQI_MODE_DMA 2
+#define PESQI_MODE_XIP 3
+#define PESQI_MODE_SHIFT 0
+#define PESQI_CPHA BIT(3)
+#define PESQI_CPOL BIT(4)
+#define PESQI_LSBF BIT(5)
+#define PESQI_RXLATCH BIT(7)
+#define PESQI_SERMODE BIT(8)
+#define PESQI_WP_EN BIT(9)
+#define PESQI_HOLD_EN BIT(10)
+#define PESQI_BURST_EN BIT(12)
+#define PESQI_CS_CTRL_HW BIT(15)
+#define PESQI_SOFT_RESET BIT(16)
+#define PESQI_LANES_SHIFT 20
+#define PESQI_SINGLE_LANE 0
+#define PESQI_DUAL_LANE 1
+#define PESQI_QUAD_LANE 2
+#define PESQI_CSEN_SHIFT 24
+#define PESQI_EN BIT(23)
+
+/* PESQI_CLK_CTRL_REG fields */
+#define PESQI_CLK_EN BIT(0)
+#define PESQI_CLK_STABLE BIT(1)
+#define PESQI_CLKDIV_SHIFT 8
+#define PESQI_CLKDIV 0xff
+
+/* PESQI_INT_THR/CMD_THR_REG */
+#define PESQI_TXTHR_MASK 0x1f
+#define PESQI_TXTHR_SHIFT 8
+#define PESQI_RXTHR_MASK 0x1f
+#define PESQI_RXTHR_SHIFT 0
+
+/* PESQI_INT_EN/INT_STAT/INT_SIG_EN_REG */
+#define PESQI_TXEMPTY BIT(0)
+#define PESQI_TXFULL BIT(1)
+#define PESQI_TXTHR BIT(2)
+#define PESQI_RXEMPTY BIT(3)
+#define PESQI_RXFULL BIT(4)
+#define PESQI_RXTHR BIT(5)
+#define PESQI_BDDONE BIT(9) /* BD processing complete */
+#define PESQI_PKTCOMP BIT(10) /* packet processing complete */
+#define PESQI_DMAERR BIT(11) /* error */
+
+/* PESQI_BD_CTRL_REG */
+#define PESQI_DMA_EN BIT(0) /* enable DMA engine */
+#define PESQI_POLL_EN BIT(1) /* enable polling */
+#define PESQI_BDP_START BIT(2) /* start BD processor */
+
+/* PESQI controller buffer descriptor */
+struct buf_desc {
+ u32 bd_ctrl; /* control */
+ u32 bd_status; /* reserved */
+ u32 bd_addr; /* DMA buffer addr */
+ u32 bd_nextp; /* next item in chain */
+};
+
+/* bd_ctrl */
+#define BD_BUFLEN 0x1ff
+#define BD_CBD_INT_EN BIT(16) /* Current BD is processed */
+#define BD_PKT_INT_EN BIT(17) /* All BDs of PKT processed */
+#define BD_LIFM BIT(18) /* last data of pkt */
+#define BD_LAST BIT(19) /* end of list */
+#define BD_DATA_RECV BIT(20) /* receive data */
+#define BD_DDR BIT(21) /* DDR mode */
+#define BD_DUAL BIT(22) /* Dual SPI */
+#define BD_QUAD BIT(23) /* Quad SPI */
+#define BD_LSBF BIT(25) /* LSB First */
+#define BD_STAT_CHECK BIT(27) /* Status poll */
+#define BD_DEVSEL_SHIFT 28 /* CS */
+#define BD_CS_DEASSERT BIT(30) /* de-assert CS after current BD */
+#define BD_EN BIT(31) /* BD owned by H/W */
+
+/**
+ * struct ring_desc - Representation of SQI ring descriptor
+ * @list: list element to add to free or used list.
+ * @bd: PESQI controller buffer descriptor
+ * @bd_dma: DMA address of PESQI controller buffer descriptor
+ * @xfer_len: transfer length
+ */
+struct ring_desc {
+ struct list_head list;
+ struct buf_desc *bd;
+ dma_addr_t bd_dma;
+ u32 xfer_len;
+};
+
+/* Global constants */
+#define PESQI_BD_BUF_LEN_MAX 256
+#define PESQI_BD_COUNT 256 /* max 64KB data per spi message */
+
+struct pic32_sqi {
+ void __iomem *regs;
+ struct clk *sys_clk;
+ struct clk *base_clk; /* drives spi clock */
+ struct spi_master *master;
+ int irq;
+ struct completion xfer_done;
+ struct ring_desc *ring;
+ void *bd;
+ dma_addr_t bd_dma;
+ struct list_head bd_list_free; /* free */
+ struct list_head bd_list_used; /* allocated */
+ struct spi_device *cur_spi;
+ u32 cur_speed;
+ u8 cur_mode;
+};
+
+static inline void pic32_setbits(void __iomem *reg, u32 set)
+{
+ writel(readl(reg) | set, reg);
+}
+
+static inline void pic32_clrbits(void __iomem *reg, u32 clr)
+{
+ writel(readl(reg) & ~clr, reg);
+}
+
+static int pic32_sqi_set_clk_rate(struct pic32_sqi *sqi, u32 sck)
+{
+ u32 val, div;
+
+ /* div = base_clk / (2 * spi_clk) */
+ div = clk_get_rate(sqi->base_clk) / (2 * sck);
+ div &= PESQI_CLKDIV;
+
+ val = readl(sqi->regs + PESQI_CLK_CTRL_REG);
+ /* apply new divider */
+ val &= ~(PESQI_CLK_STABLE | (PESQI_CLKDIV << PESQI_CLKDIV_SHIFT));
+ val |= div << PESQI_CLKDIV_SHIFT;
+ writel(val, sqi->regs + PESQI_CLK_CTRL_REG);
+
+ /* wait for stability */
+ return readl_poll_timeout(sqi->regs + PESQI_CLK_CTRL_REG, val,
+ val & PESQI_CLK_STABLE, 1, 5000);
+}
+
+static inline void pic32_sqi_enable_int(struct pic32_sqi *sqi)
+{
+ u32 mask = PESQI_DMAERR | PESQI_BDDONE | PESQI_PKTCOMP;
+
+ writel(mask, sqi->regs + PESQI_INT_ENABLE_REG);
+ /* INT_SIGEN works as interrupt-gate to INTR line */
+ writel(mask, sqi->regs + PESQI_INT_SIGEN_REG);
+}
+
+static inline void pic32_sqi_disable_int(struct pic32_sqi *sqi)
+{
+ writel(0, sqi->regs + PESQI_INT_ENABLE_REG);
+ writel(0, sqi->regs + PESQI_INT_SIGEN_REG);
+}
+
+static irqreturn_t pic32_sqi_isr(int irq, void *dev_id)
+{
+ struct pic32_sqi *sqi = dev_id;
+ u32 enable, status;
+
+ enable = readl(sqi->regs + PESQI_INT_ENABLE_REG);
+ status = readl(sqi->regs + PESQI_INT_STAT_REG);
+
+ /* check spurious interrupt */
+ if (!status)
+ return IRQ_NONE;
+
+ if (status & PESQI_DMAERR) {
+ enable = 0;
+ goto irq_done;
+ }
+
+ if (status & PESQI_TXTHR)
+ enable &= ~(PESQI_TXTHR | PESQI_TXFULL | PESQI_TXEMPTY);
+
+ if (status & PESQI_RXTHR)
+ enable &= ~(PESQI_RXTHR | PESQI_RXFULL | PESQI_RXEMPTY);
+
+ if (status & PESQI_BDDONE)
+ enable &= ~PESQI_BDDONE;
+
+ /* packet processing completed */
+ if (status & PESQI_PKTCOMP) {
+ /* mask all interrupts */
+ enable = 0;
+ /* complete trasaction */
+ complete(&sqi->xfer_done);
+ }
+
+irq_done:
+ /* interrupts are sticky, so mask when handled */
+ writel(enable, sqi->regs + PESQI_INT_ENABLE_REG);
+
+ return IRQ_HANDLED;
+}
+
+static struct ring_desc *ring_desc_get(struct pic32_sqi *sqi)
+{
+ struct ring_desc *rdesc;
+
+ if (list_empty(&sqi->bd_list_free))
+ return NULL;
+
+ rdesc = list_first_entry(&sqi->bd_list_free, struct ring_desc, list);
+ list_del(&rdesc->list);
+ list_add_tail(&rdesc->list, &sqi->bd_list_used);
+ return rdesc;
+}
+
+static void ring_desc_put(struct pic32_sqi *sqi, struct ring_desc *rdesc)
+{
+ list_del(&rdesc->list);
+ list_add(&rdesc->list, &sqi->bd_list_free);
+}
+
+static int pic32_sqi_one_transfer(struct pic32_sqi *sqi,
+ struct spi_message *mesg,
+ struct spi_transfer *xfer)
+{
+ struct spi_device *spi = mesg->spi;
+ struct scatterlist *sg, *sgl;
+ struct ring_desc *rdesc;
+ struct buf_desc *bd;
+ int nents, i;
+ u32 bd_ctrl;
+ u32 nbits;
+
+ /* Device selection */
+ bd_ctrl = spi->chip_select << BD_DEVSEL_SHIFT;
+
+ /* half-duplex: select transfer buffer, direction and lane */
+ if (xfer->rx_buf) {
+ bd_ctrl |= BD_DATA_RECV;
+ nbits = xfer->rx_nbits;
+ sgl = xfer->rx_sg.sgl;
+ nents = xfer->rx_sg.nents;
+ } else {
+ nbits = xfer->tx_nbits;
+ sgl = xfer->tx_sg.sgl;
+ nents = xfer->tx_sg.nents;
+ }
+
+ if (nbits & SPI_NBITS_QUAD)
+ bd_ctrl |= BD_QUAD;
+ else if (nbits & SPI_NBITS_DUAL)
+ bd_ctrl |= BD_DUAL;
+
+ /* LSB first */
+ if (spi->mode & SPI_LSB_FIRST)
+ bd_ctrl |= BD_LSBF;
+
+ /* ownership to hardware */
+ bd_ctrl |= BD_EN;
+
+ for_each_sg(sgl, sg, nents, i) {
+ /* get ring descriptor */
+ rdesc = ring_desc_get(sqi);
+ if (!rdesc)
+ break;
+
+ bd = rdesc->bd;
+
+ /* BD CTRL: length */
+ rdesc->xfer_len = sg_dma_len(sg);
+ bd->bd_ctrl = bd_ctrl;
+ bd->bd_ctrl |= rdesc->xfer_len;
+
+ /* BD STAT */
+ bd->bd_status = 0;
+
+ /* BD BUFFER ADDRESS */
+ bd->bd_addr = sg->dma_address;
+ }
+
+ return 0;
+}
+
+static int pic32_sqi_prepare_hardware(struct spi_master *master)
+{
+ struct pic32_sqi *sqi = spi_master_get_devdata(master);
+
+ /* enable spi interface */
+ pic32_setbits(sqi->regs + PESQI_CONF_REG, PESQI_EN);
+ /* enable spi clk */
+ pic32_setbits(sqi->regs + PESQI_CLK_CTRL_REG, PESQI_CLK_EN);
+
+ return 0;
+}
+
+static bool pic32_sqi_can_dma(struct spi_master *master,
+ struct spi_device *spi,
+ struct spi_transfer *x)
+{
+ /* Do DMA irrespective of transfer size */
+ return true;
+}
+
+static int pic32_sqi_one_message(struct spi_master *master,
+ struct spi_message *msg)
+{
+ struct spi_device *spi = msg->spi;
+ struct ring_desc *rdesc, *next;
+ struct spi_transfer *xfer;
+ struct pic32_sqi *sqi;
+ int ret = 0, mode;
+ u32 val;
+
+ sqi = spi_master_get_devdata(master);
+
+ reinit_completion(&sqi->xfer_done);
+ msg->actual_length = 0;
+
+ /* We can't handle spi_transfer specific "speed_hz", "bits_per_word"
+ * and "delay_usecs". But spi_device specific speed and mode change
+ * can be handled at best during spi chip-select switch.
+ */
+ if (sqi->cur_spi != spi) {
+ /* set spi speed */
+ if (sqi->cur_speed != spi->max_speed_hz) {
+ sqi->cur_speed = spi->max_speed_hz;
+ ret = pic32_sqi_set_clk_rate(sqi, spi->max_speed_hz);
+ if (ret)
+ dev_warn(&spi->dev, "set_clk, %d\n", ret);
+ }
+
+ /* set spi mode */
+ mode = spi->mode & (SPI_MODE_3 | SPI_LSB_FIRST);
+ if (sqi->cur_mode != mode) {
+ val = readl(sqi->regs + PESQI_CONF_REG);
+ val &= ~(PESQI_CPOL | PESQI_CPHA | PESQI_LSBF);
+ if (mode & SPI_CPOL)
+ val |= PESQI_CPOL;
+ if (mode & SPI_LSB_FIRST)
+ val |= PESQI_LSBF;
+ val |= PESQI_CPHA;
+ writel(val, sqi->regs + PESQI_CONF_REG);
+
+ sqi->cur_mode = mode;
+ }
+ sqi->cur_spi = spi;
+ }
+
+ /* prepare hardware desc-list(BD) for transfer(s) */
+ list_for_each_entry(xfer, &msg->transfers, transfer_list) {
+ ret = pic32_sqi_one_transfer(sqi, msg, xfer);
+ if (ret) {
+ dev_err(&spi->dev, "xfer %p err\n", xfer);
+ goto xfer_out;
+ }
+ }
+
+ /* BDs are prepared and chained. Now mark LAST_BD, CS_DEASSERT at last
+ * element of the list.
+ */
+ rdesc = list_last_entry(&sqi->bd_list_used, struct ring_desc, list);
+ rdesc->bd->bd_ctrl |= BD_LAST | BD_CS_DEASSERT |
+ BD_LIFM | BD_PKT_INT_EN;
+
+ /* set base address BD list for DMA engine */
+ rdesc = list_first_entry(&sqi->bd_list_used, struct ring_desc, list);
+ writel(rdesc->bd_dma, sqi->regs + PESQI_BD_BASE_ADDR_REG);
+
+ /* enable interrupt */
+ pic32_sqi_enable_int(sqi);
+
+ /* enable DMA engine */
+ val = PESQI_DMA_EN | PESQI_POLL_EN | PESQI_BDP_START;
+ writel(val, sqi->regs + PESQI_BD_CTRL_REG);
+
+ /* wait for xfer completion */
+ ret = wait_for_completion_timeout(&sqi->xfer_done, 5 * HZ);
+ if (ret <= 0) {
+ dev_err(&sqi->master->dev, "wait timedout/interrupted\n");
+ ret = -EIO;
+ msg->status = ret;
+ } else {
+ /* success */
+ msg->status = 0;
+ ret = 0;
+ }
+
+ /* disable DMA */
+ writel(0, sqi->regs + PESQI_BD_CTRL_REG);
+
+ pic32_sqi_disable_int(sqi);
+
+xfer_out:
+ list_for_each_entry_safe_reverse(rdesc, next,
+ &sqi->bd_list_used, list) {
+ /* Update total byte transferred */
+ msg->actual_length += rdesc->xfer_len;
+ /* release ring descr */
+ ring_desc_put(sqi, rdesc);
+ }
+ spi_finalize_current_message(spi->master);
+
+ return ret;
+}
+
+static int pic32_sqi_unprepare_hardware(struct spi_master *master)
+{
+ struct pic32_sqi *sqi = spi_master_get_devdata(master);
+
+ /* disable clk */
+ pic32_clrbits(sqi->regs + PESQI_CLK_CTRL_REG, PESQI_CLK_EN);
+ /* disable spi */
+ pic32_clrbits(sqi->regs + PESQI_CONF_REG, PESQI_EN);
+
+ return 0;
+}
+
+static int ring_desc_ring_alloc(struct pic32_sqi *sqi)
+{
+ struct ring_desc *rdesc;
+ struct buf_desc *bd;
+ int i;
+
+ /* allocate coherent DMAable memory for hardware buffer descriptors. */
+ sqi->bd = dma_zalloc_coherent(&sqi->master->dev,
+ sizeof(*bd) * PESQI_BD_COUNT,
+ &sqi->bd_dma, GFP_DMA32);
+ if (!sqi->bd) {
+ dev_err(&sqi->master->dev, "failed allocating dma buffer\n");
+ return -ENOMEM;
+ }
+
+ /* allocate software ring descriptors */
+ sqi->ring = kcalloc(PESQI_BD_COUNT, sizeof(*rdesc), GFP_KERNEL);
+ if (!sqi->ring) {
+ dma_free_coherent(&sqi->master->dev,
+ sizeof(*bd) * PESQI_BD_COUNT,
+ sqi->bd, sqi->bd_dma);
+ return -ENOMEM;
+ }
+
+ bd = (struct buf_desc *)sqi->bd;
+
+ INIT_LIST_HEAD(&sqi->bd_list_free);
+ INIT_LIST_HEAD(&sqi->bd_list_used);
+
+ /* initialize ring-desc */
+ for (i = 0, rdesc = sqi->ring; i < PESQI_BD_COUNT; i++, rdesc++) {
+ INIT_LIST_HEAD(&rdesc->list);
+ rdesc->bd = &bd[i];
+ rdesc->bd_dma = sqi->bd_dma + (void *)&bd[i] - (void *)bd;
+ list_add_tail(&rdesc->list, &sqi->bd_list_free);
+ }
+
+ /* Prepare BD: chain to next BD(s) */
+ for (i = 0, rdesc = sqi->ring; i < PESQI_BD_COUNT - 1; i++)
+ bd[i].bd_nextp = rdesc[i + 1].bd_dma;
+ bd[PESQI_BD_COUNT - 1].bd_nextp = 0;
+
+ return 0;
+}
+
+static void ring_desc_ring_free(struct pic32_sqi *sqi)
+{
+ dma_free_coherent(&sqi->master->dev,
+ sizeof(struct buf_desc) * PESQI_BD_COUNT,
+ sqi->bd, sqi->bd_dma);
+ kfree(sqi->ring);
+}
+
+static void pic32_sqi_hw_init(struct pic32_sqi *sqi)
+{
+ unsigned long flags;
+ u32 val;
+
+ /* Soft-reset of PESQI controller triggers interrupt.
+ * We are not yet ready to handle them so disable CPU
+ * interrupt for the time being.
+ */
+ local_irq_save(flags);
+
+ /* assert soft-reset */
+ writel(PESQI_SOFT_RESET, sqi->regs + PESQI_CONF_REG);
+
+ /* wait until clear */
+ readl_poll_timeout_atomic(sqi->regs + PESQI_CONF_REG, val,
+ !(val & PESQI_SOFT_RESET), 1, 5000);
+
+ /* disable all interrupts */
+ pic32_sqi_disable_int(sqi);
+
+ /* Now it is safe to enable back CPU interrupt */
+ local_irq_restore(flags);
+
+ /* tx and rx fifo interrupt threshold */
+ val = readl(sqi->regs + PESQI_CMD_THRES_REG);
+ val &= ~(PESQI_TXTHR_MASK << PESQI_TXTHR_SHIFT);
+ val &= ~(PESQI_RXTHR_MASK << PESQI_RXTHR_SHIFT);
+ val |= (1U << PESQI_TXTHR_SHIFT) | (1U << PESQI_RXTHR_SHIFT);
+ writel(val, sqi->regs + PESQI_CMD_THRES_REG);
+
+ val = readl(sqi->regs + PESQI_INT_THRES_REG);
+ val &= ~(PESQI_TXTHR_MASK << PESQI_TXTHR_SHIFT);
+ val &= ~(PESQI_RXTHR_MASK << PESQI_RXTHR_SHIFT);
+ val |= (1U << PESQI_TXTHR_SHIFT) | (1U << PESQI_RXTHR_SHIFT);
+ writel(val, sqi->regs + PESQI_INT_THRES_REG);
+
+ /* default configuration */
+ val = readl(sqi->regs + PESQI_CONF_REG);
+
+ /* set mode: DMA */
+ val &= ~PESQI_MODE;
+ val |= PESQI_MODE_DMA << PESQI_MODE_SHIFT;
+ writel(val, sqi->regs + PESQI_CONF_REG);
+
+ /* DATAEN - SQIID0-ID3 */
+ val |= PESQI_QUAD_LANE << PESQI_LANES_SHIFT;
+
+ /* burst/INCR4 enable */
+ val |= PESQI_BURST_EN;
+
+ /* CSEN - all CS */
+ val |= 3U << PESQI_CSEN_SHIFT;
+ writel(val, sqi->regs + PESQI_CONF_REG);
+
+ /* write poll count */
+ writel(0, sqi->regs + PESQI_BD_POLL_CTRL_REG);
+
+ sqi->cur_speed = 0;
+ sqi->cur_mode = -1;
+}
+
+static int pic32_sqi_probe(struct platform_device *pdev)
+{
+ struct spi_master *master;
+ struct pic32_sqi *sqi;
+ struct resource *reg;
+ int ret;
+
+ master = spi_alloc_master(&pdev->dev, sizeof(*sqi));
+ if (!master)
+ return -ENOMEM;
+
+ sqi = spi_master_get_devdata(master);
+ sqi->master = master;
+
+ reg = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ sqi->regs = devm_ioremap_resource(&pdev->dev, reg);
+ if (IS_ERR(sqi->regs)) {
+ ret = PTR_ERR(sqi->regs);
+ goto err_free_master;
+ }
+
+ /* irq */
+ sqi->irq = platform_get_irq(pdev, 0);
+ if (sqi->irq < 0) {
+ dev_err(&pdev->dev, "no irq found\n");
+ ret = sqi->irq;
+ goto err_free_master;
+ }
+
+ /* clocks */
+ sqi->sys_clk = devm_clk_get(&pdev->dev, "reg_ck");
+ if (IS_ERR(sqi->sys_clk)) {
+ ret = PTR_ERR(sqi->sys_clk);
+ dev_err(&pdev->dev, "no sys_clk ?\n");
+ goto err_free_master;
+ }
+
+ sqi->base_clk = devm_clk_get(&pdev->dev, "spi_ck");
+ if (IS_ERR(sqi->base_clk)) {
+ ret = PTR_ERR(sqi->base_clk);
+ dev_err(&pdev->dev, "no base clk ?\n");
+ goto err_free_master;
+ }
+
+ ret = clk_prepare_enable(sqi->sys_clk);
+ if (ret) {
+ dev_err(&pdev->dev, "sys clk enable failed\n");
+ goto err_free_master;
+ }
+
+ ret = clk_prepare_enable(sqi->base_clk);
+ if (ret) {
+ dev_err(&pdev->dev, "base clk enable failed\n");
+ clk_disable_unprepare(sqi->sys_clk);
+ goto err_free_master;
+ }
+
+ init_completion(&sqi->xfer_done);
+
+ /* initialize hardware */
+ pic32_sqi_hw_init(sqi);
+
+ /* allocate buffers & descriptors */
+ ret = ring_desc_ring_alloc(sqi);
+ if (ret) {
+ dev_err(&pdev->dev, "ring alloc failed\n");
+ goto err_disable_clk;
+ }
+
+ /* install irq handlers */
+ ret = request_irq(sqi->irq, pic32_sqi_isr, 0,
+ dev_name(&pdev->dev), sqi);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "request_irq(%d), failed\n", sqi->irq);
+ goto err_free_ring;
+ }
+
+ /* register master */
+ master->num_chipselect = 2;
+ master->max_speed_hz = clk_get_rate(sqi->base_clk);
+ master->dma_alignment = 32;
+ master->max_dma_len = PESQI_BD_BUF_LEN_MAX;
+ master->dev.of_node = of_node_get(pdev->dev.of_node);
+ master->mode_bits = SPI_MODE_3 | SPI_MODE_0 | SPI_TX_DUAL |
+ SPI_RX_DUAL | SPI_TX_QUAD | SPI_RX_QUAD;
+ master->flags = SPI_MASTER_HALF_DUPLEX;
+ master->can_dma = pic32_sqi_can_dma;
+ master->bits_per_word_mask = SPI_BPW_RANGE_MASK(8, 32);
+ master->transfer_one_message = pic32_sqi_one_message;
+ master->prepare_transfer_hardware = pic32_sqi_prepare_hardware;
+ master->unprepare_transfer_hardware = pic32_sqi_unprepare_hardware;
+
+ ret = devm_spi_register_master(&pdev->dev, master);
+ if (ret) {
+ dev_err(&master->dev, "failed registering spi master\n");
+ free_irq(sqi->irq, sqi);
+ goto err_free_ring;
+ }
+
+ platform_set_drvdata(pdev, sqi);
+
+ return 0;
+
+err_free_ring:
+ ring_desc_ring_free(sqi);
+
+err_disable_clk:
+ clk_disable_unprepare(sqi->base_clk);
+ clk_disable_unprepare(sqi->sys_clk);
+
+err_free_master:
+ spi_master_put(master);
+ return ret;
+}
+
+static int pic32_sqi_remove(struct platform_device *pdev)
+{
+ struct pic32_sqi *sqi = platform_get_drvdata(pdev);
+
+ /* release resources */
+ free_irq(sqi->irq, sqi);
+ ring_desc_ring_free(sqi);
+
+ /* disable clk */
+ clk_disable_unprepare(sqi->base_clk);
+ clk_disable_unprepare(sqi->sys_clk);
+
+ return 0;
+}
+
+static const struct of_device_id pic32_sqi_of_ids[] = {
+ {.compatible = "microchip,pic32mzda-sqi",},
+ {},
+};
+MODULE_DEVICE_TABLE(of, pic32_sqi_of_ids);
+
+static struct platform_driver pic32_sqi_driver = {
+ .driver = {
+ .name = "sqi-pic32",
+ .of_match_table = of_match_ptr(pic32_sqi_of_ids),
+ },
+ .probe = pic32_sqi_probe,
+ .remove = pic32_sqi_remove,
+};
+
+module_platform_driver(pic32_sqi_driver);
+
+MODULE_AUTHOR("Purna Chandra Mandal <purna.mandal@microchip.com>");
+MODULE_DESCRIPTION("Microchip SPI driver for PIC32 SQI controller.");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/spi/spi-pic32.c b/drivers/spi/spi-pic32.c
new file mode 100644
index 0000000..73db87f
--- /dev/null
+++ b/drivers/spi/spi-pic32.c
@@ -0,0 +1,878 @@
+/*
+ * Microchip PIC32 SPI controller driver.
+ *
+ * Purna Chandra Mandal <purna.mandal@microchip.com>
+ * Copyright (c) 2016, Microchip Technology Inc.
+ *
+ * This program is free software; you can distribute it and/or modify it
+ * under the terms of the GNU General Public License (Version 2) as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+ * for more details.
+ */
+
+#include <linux/clk.h>
+#include <linux/clkdev.h>
+#include <linux/delay.h>
+#include <linux/dmaengine.h>
+#include <linux/dma-mapping.h>
+#include <linux/highmem.h>
+#include <linux/module.h>
+#include <linux/io.h>
+#include <linux/interrupt.h>
+#include <linux/of.h>
+#include <linux/of_irq.h>
+#include <linux/of_gpio.h>
+#include <linux/of_address.h>
+#include <linux/platform_device.h>
+#include <linux/spi/spi.h>
+
+/* SPI controller registers */
+struct pic32_spi_regs {
+ u32 ctrl;
+ u32 ctrl_clr;
+ u32 ctrl_set;
+ u32 ctrl_inv;
+ u32 status;
+ u32 status_clr;
+ u32 status_set;
+ u32 status_inv;
+ u32 buf;
+ u32 dontuse[3];
+ u32 baud;
+ u32 dontuse2[3];
+ u32 ctrl2;
+ u32 ctrl2_clr;
+ u32 ctrl2_set;
+ u32 ctrl2_inv;
+};
+
+/* Bit fields of SPI Control Register */
+#define CTRL_RX_INT_SHIFT 0 /* Rx interrupt generation */
+#define RX_FIFO_EMTPY 0
+#define RX_FIFO_NOT_EMPTY 1 /* not empty */
+#define RX_FIFO_HALF_FULL 2 /* full by half or more */
+#define RX_FIFO_FULL 3 /* completely full */
+
+#define CTRL_TX_INT_SHIFT 2 /* TX interrupt generation */
+#define TX_FIFO_ALL_EMPTY 0 /* completely empty */
+#define TX_FIFO_EMTPY 1 /* empty */
+#define TX_FIFO_HALF_EMPTY 2 /* empty by half or more */
+#define TX_FIFO_NOT_FULL 3 /* atleast one empty */
+
+#define CTRL_MSTEN BIT(5) /* enable master mode */
+#define CTRL_CKP BIT(6) /* active low */
+#define CTRL_CKE BIT(8) /* Tx on falling edge */
+#define CTRL_SMP BIT(9) /* Rx at middle or end of tx */
+#define CTRL_BPW_MASK 0x03 /* bits per word/sample */
+#define CTRL_BPW_SHIFT 10
+#define PIC32_BPW_8 0
+#define PIC32_BPW_16 1
+#define PIC32_BPW_32 2
+#define CTRL_SIDL BIT(13) /* sleep when idle */
+#define CTRL_ON BIT(15) /* enable macro */
+#define CTRL_ENHBUF BIT(16) /* enable enhanced buffering */
+#define CTRL_MCLKSEL BIT(23) /* select clock source */
+#define CTRL_MSSEN BIT(28) /* macro driven /SS */
+#define CTRL_FRMEN BIT(31) /* enable framing mode */
+
+/* Bit fields of SPI Status Register */
+#define STAT_RF_EMPTY BIT(5) /* RX Fifo empty */
+#define STAT_RX_OV BIT(6) /* err, s/w needs to clear */
+#define STAT_TX_UR BIT(8) /* UR in Framed SPI modes */
+#define STAT_FRM_ERR BIT(12) /* Multiple Frame Sync pulse */
+#define STAT_TF_LVL_MASK 0x1F
+#define STAT_TF_LVL_SHIFT 16
+#define STAT_RF_LVL_MASK 0x1F
+#define STAT_RF_LVL_SHIFT 24
+
+/* Bit fields of SPI Baud Register */
+#define BAUD_MASK 0x1ff
+
+/* Bit fields of SPI Control2 Register */
+#define CTRL2_TX_UR_EN BIT(10) /* Enable int on Tx under-run */
+#define CTRL2_RX_OV_EN BIT(11) /* Enable int on Rx over-run */
+#define CTRL2_FRM_ERR_EN BIT(12) /* Enable frame err int */
+
+/* Minimum DMA transfer size */
+#define PIC32_DMA_LEN_MIN 64
+
+struct pic32_spi {
+ dma_addr_t dma_base;
+ struct pic32_spi_regs __iomem *regs;
+ int fault_irq;
+ int rx_irq;
+ int tx_irq;
+ u32 fifo_n_byte; /* FIFO depth in bytes */
+ struct clk *clk;
+ struct spi_master *master;
+ /* Current controller setting */
+ u32 speed_hz; /* spi-clk rate */
+ u32 mode;
+ u32 bits_per_word;
+ u32 fifo_n_elm; /* FIFO depth in words */
+#define PIC32F_DMA_PREP 0 /* DMA chnls configured */
+ unsigned long flags;
+ /* Current transfer state */
+ struct completion xfer_done;
+ /* PIO transfer specific */
+ const void *tx;
+ const void *tx_end;
+ const void *rx;
+ const void *rx_end;
+ int len;
+ void (*rx_fifo)(struct pic32_spi *);
+ void (*tx_fifo)(struct pic32_spi *);
+};
+
+static inline void pic32_spi_enable(struct pic32_spi *pic32s)
+{
+ writel(CTRL_ON | CTRL_SIDL, &pic32s->regs->ctrl_set);
+}
+
+static inline void pic32_spi_disable(struct pic32_spi *pic32s)
+{
+ writel(CTRL_ON | CTRL_SIDL, &pic32s->regs->ctrl_clr);
+
+ /* avoid SPI registers read/write at immediate next CPU clock */
+ ndelay(20);
+}
+
+static void pic32_spi_set_clk_rate(struct pic32_spi *pic32s, u32 spi_ck)
+{
+ u32 div;
+
+ /* div = (clk_in / 2 * spi_ck) - 1 */
+ div = DIV_ROUND_CLOSEST(clk_get_rate(pic32s->clk), 2 * spi_ck) - 1;
+
+ writel(div & BAUD_MASK, &pic32s->regs->baud);
+}
+
+static inline u32 pic32_rx_fifo_level(struct pic32_spi *pic32s)
+{
+ u32 sr = readl(&pic32s->regs->status);
+
+ return (sr >> STAT_RF_LVL_SHIFT) & STAT_RF_LVL_MASK;
+}
+
+static inline u32 pic32_tx_fifo_level(struct pic32_spi *pic32s)
+{
+ u32 sr = readl(&pic32s->regs->status);
+
+ return (sr >> STAT_TF_LVL_SHIFT) & STAT_TF_LVL_MASK;
+}
+
+/* Return the max entries we can fill into tx fifo */
+static u32 pic32_tx_max(struct pic32_spi *pic32s, int n_bytes)
+{
+ u32 tx_left, tx_room, rxtx_gap;
+
+ tx_left = (pic32s->tx_end - pic32s->tx) / n_bytes;
+ tx_room = pic32s->fifo_n_elm - pic32_tx_fifo_level(pic32s);
+
+ /*
+ * Another concern is about the tx/rx mismatch, we
+ * though to use (pic32s->fifo_n_byte - rxfl - txfl) as
+ * one maximum value for tx, but it doesn't cover the
+ * data which is out of tx/rx fifo and inside the
+ * shift registers. So a ctrl from sw point of
+ * view is taken.
+ */
+ rxtx_gap = ((pic32s->rx_end - pic32s->rx) -
+ (pic32s->tx_end - pic32s->tx)) / n_bytes;
+ return min3(tx_left, tx_room, (u32)(pic32s->fifo_n_elm - rxtx_gap));
+}
+
+/* Return the max entries we should read out of rx fifo */
+static u32 pic32_rx_max(struct pic32_spi *pic32s, int n_bytes)
+{
+ u32 rx_left = (pic32s->rx_end - pic32s->rx) / n_bytes;
+
+ return min_t(u32, rx_left, pic32_rx_fifo_level(pic32s));
+}
+
+#define BUILD_SPI_FIFO_RW(__name, __type, __bwl) \
+static void pic32_spi_rx_##__name(struct pic32_spi *pic32s) \
+{ \
+ __type v; \
+ u32 mx = pic32_rx_max(pic32s, sizeof(__type)); \
+ for (; mx; mx--) { \
+ v = read##__bwl(&pic32s->regs->buf); \
+ if (pic32s->rx_end - pic32s->len) \
+ *(__type *)(pic32s->rx) = v; \
+ pic32s->rx += sizeof(__type); \
+ } \
+} \
+ \
+static void pic32_spi_tx_##__name(struct pic32_spi *pic32s) \
+{ \
+ __type v; \
+ u32 mx = pic32_tx_max(pic32s, sizeof(__type)); \
+ for (; mx ; mx--) { \
+ v = (__type)~0U; \
+ if (pic32s->tx_end - pic32s->len) \
+ v = *(__type *)(pic32s->tx); \
+ write##__bwl(v, &pic32s->regs->buf); \
+ pic32s->tx += sizeof(__type); \
+ } \
+}
+
+BUILD_SPI_FIFO_RW(byte, u8, b);
+BUILD_SPI_FIFO_RW(word, u16, w);
+BUILD_SPI_FIFO_RW(dword, u32, l);
+
+static void pic32_err_stop(struct pic32_spi *pic32s, const char *msg)
+{
+ /* disable all interrupts */
+ disable_irq_nosync(pic32s->fault_irq);
+ disable_irq_nosync(pic32s->rx_irq);
+ disable_irq_nosync(pic32s->tx_irq);
+
+ /* Show err message and abort xfer with err */
+ dev_err(&pic32s->master->dev, "%s\n", msg);
+ if (pic32s->master->cur_msg)
+ pic32s->master->cur_msg->status = -EIO;
+ complete(&pic32s->xfer_done);
+}
+
+static irqreturn_t pic32_spi_fault_irq(int irq, void *dev_id)
+{
+ struct pic32_spi *pic32s = dev_id;
+ u32 status;
+
+ status = readl(&pic32s->regs->status);
+
+ /* Error handling */
+ if (status & (STAT_RX_OV | STAT_TX_UR)) {
+ writel(STAT_RX_OV, &pic32s->regs->status_clr);
+ writel(STAT_TX_UR, &pic32s->regs->status_clr);
+ pic32_err_stop(pic32s, "err_irq: fifo ov/ur-run\n");
+ return IRQ_HANDLED;
+ }
+
+ if (status & STAT_FRM_ERR) {
+ pic32_err_stop(pic32s, "err_irq: frame error");
+ return IRQ_HANDLED;
+ }
+
+ if (!pic32s->master->cur_msg) {
+ pic32_err_stop(pic32s, "err_irq: no mesg");
+ return IRQ_NONE;
+ }
+
+ return IRQ_NONE;
+}
+
+static irqreturn_t pic32_spi_rx_irq(int irq, void *dev_id)
+{
+ struct pic32_spi *pic32s = dev_id;
+
+ pic32s->rx_fifo(pic32s);
+
+ /* rx complete ? */
+ if (pic32s->rx_end == pic32s->rx) {
+ /* disable all interrupts */
+ disable_irq_nosync(pic32s->fault_irq);
+ disable_irq_nosync(pic32s->rx_irq);
+
+ /* complete current xfer */
+ complete(&pic32s->xfer_done);
+ }
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t pic32_spi_tx_irq(int irq, void *dev_id)
+{
+ struct pic32_spi *pic32s = dev_id;
+
+ pic32s->tx_fifo(pic32s);
+
+ /* tx complete? disable tx interrupt */
+ if (pic32s->tx_end == pic32s->tx)
+ disable_irq_nosync(pic32s->tx_irq);
+
+ return IRQ_HANDLED;
+}
+
+static void pic32_spi_dma_rx_notify(void *data)
+{
+ struct pic32_spi *pic32s = data;
+
+ complete(&pic32s->xfer_done);
+}
+
+static int pic32_spi_dma_transfer(struct pic32_spi *pic32s,
+ struct spi_transfer *xfer)
+{
+ struct spi_master *master = pic32s->master;
+ struct dma_async_tx_descriptor *desc_rx;
+ struct dma_async_tx_descriptor *desc_tx;
+ dma_cookie_t cookie;
+ int ret;
+
+ if (!master->dma_rx || !master->dma_tx)
+ return -ENODEV;
+
+ desc_rx = dmaengine_prep_slave_sg(master->dma_rx,
+ xfer->rx_sg.sgl,
+ xfer->rx_sg.nents,
+ DMA_FROM_DEVICE,
+ DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
+ if (!desc_rx) {
+ ret = -EINVAL;
+ goto err_dma;
+ }
+
+ desc_tx = dmaengine_prep_slave_sg(master->dma_tx,
+ xfer->tx_sg.sgl,
+ xfer->tx_sg.nents,
+ DMA_TO_DEVICE,
+ DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
+ if (!desc_tx) {
+ ret = -EINVAL;
+ goto err_dma;
+ }
+
+ /* Put callback on the RX transfer, that should finish last */
+ desc_rx->callback = pic32_spi_dma_rx_notify;
+ desc_rx->callback_param = pic32s;
+
+ cookie = dmaengine_submit(desc_rx);
+ ret = dma_submit_error(cookie);
+ if (ret)
+ goto err_dma;
+
+ cookie = dmaengine_submit(desc_tx);
+ ret = dma_submit_error(cookie);
+ if (ret)
+ goto err_dma_tx;
+
+ dma_async_issue_pending(master->dma_rx);
+ dma_async_issue_pending(master->dma_tx);
+
+ return 0;
+
+err_dma_tx:
+ dmaengine_terminate_all(master->dma_rx);
+err_dma:
+ return ret;
+}
+
+static int pic32_spi_dma_config(struct pic32_spi *pic32s, u32 dma_width)
+{
+ int buf_offset = offsetof(struct pic32_spi_regs, buf);
+ struct spi_master *master = pic32s->master;
+ struct dma_slave_config cfg;
+ int ret;
+
+ cfg.device_fc = true;
+ cfg.src_addr = pic32s->dma_base + buf_offset;
+ cfg.dst_addr = pic32s->dma_base + buf_offset;
+ cfg.src_maxburst = pic32s->fifo_n_elm / 2; /* fill one-half */
+ cfg.dst_maxburst = pic32s->fifo_n_elm / 2; /* drain one-half */
+ cfg.src_addr_width = dma_width;
+ cfg.dst_addr_width = dma_width;
+ /* tx channel */
+ cfg.slave_id = pic32s->tx_irq;
+ cfg.direction = DMA_MEM_TO_DEV;
+ ret = dmaengine_slave_config(master->dma_tx, &cfg);
+ if (ret) {
+ dev_err(&master->dev, "tx channel setup failed\n");
+ return ret;
+ }
+ /* rx channel */
+ cfg.slave_id = pic32s->rx_irq;
+ cfg.direction = DMA_DEV_TO_MEM;
+ ret = dmaengine_slave_config(master->dma_rx, &cfg);
+ if (ret)
+ dev_err(&master->dev, "rx channel setup failed\n");
+
+ return ret;
+}
+
+static int pic32_spi_set_word_size(struct pic32_spi *pic32s, u8 bits_per_word)
+{
+ enum dma_slave_buswidth dmawidth;
+ u32 buswidth, v;
+
+ switch (bits_per_word) {
+ case 8:
+ pic32s->rx_fifo = pic32_spi_rx_byte;
+ pic32s->tx_fifo = pic32_spi_tx_byte;
+ buswidth = PIC32_BPW_8;
+ dmawidth = DMA_SLAVE_BUSWIDTH_1_BYTE;
+ break;
+ case 16:
+ pic32s->rx_fifo = pic32_spi_rx_word;
+ pic32s->tx_fifo = pic32_spi_tx_word;
+ buswidth = PIC32_BPW_16;
+ dmawidth = DMA_SLAVE_BUSWIDTH_2_BYTES;
+ break;
+ case 32:
+ pic32s->rx_fifo = pic32_spi_rx_dword;
+ pic32s->tx_fifo = pic32_spi_tx_dword;
+ buswidth = PIC32_BPW_32;
+ dmawidth = DMA_SLAVE_BUSWIDTH_4_BYTES;
+ break;
+ default:
+ /* not supported */
+ return -EINVAL;
+ }
+
+ /* calculate maximum number of words fifos can hold */
+ pic32s->fifo_n_elm = DIV_ROUND_UP(pic32s->fifo_n_byte,
+ bits_per_word / 8);
+ /* set word size */
+ v = readl(&pic32s->regs->ctrl);
+ v &= ~(CTRL_BPW_MASK << CTRL_BPW_SHIFT);
+ v |= buswidth << CTRL_BPW_SHIFT;
+ writel(v, &pic32s->regs->ctrl);
+
+ /* re-configure dma width, if required */
+ if (test_bit(PIC32F_DMA_PREP, &pic32s->flags))
+ pic32_spi_dma_config(pic32s, dmawidth);
+
+ return 0;
+}
+
+static int pic32_spi_prepare_hardware(struct spi_master *master)
+{
+ struct pic32_spi *pic32s = spi_master_get_devdata(master);
+
+ pic32_spi_enable(pic32s);
+
+ return 0;
+}
+
+static int pic32_spi_prepare_message(struct spi_master *master,
+ struct spi_message *msg)
+{
+ struct pic32_spi *pic32s = spi_master_get_devdata(master);
+ struct spi_device *spi = msg->spi;
+ u32 val;
+
+ /* set device specific bits_per_word */
+ if (pic32s->bits_per_word != spi->bits_per_word) {
+ pic32_spi_set_word_size(pic32s, spi->bits_per_word);
+ pic32s->bits_per_word = spi->bits_per_word;
+ }
+
+ /* device specific speed change */
+ if (pic32s->speed_hz != spi->max_speed_hz) {
+ pic32_spi_set_clk_rate(pic32s, spi->max_speed_hz);
+ pic32s->speed_hz = spi->max_speed_hz;
+ }
+
+ /* device specific mode change */
+ if (pic32s->mode != spi->mode) {
+ val = readl(&pic32s->regs->ctrl);
+ /* active low */
+ if (spi->mode & SPI_CPOL)
+ val |= CTRL_CKP;
+ else
+ val &= ~CTRL_CKP;
+ /* tx on rising edge */
+ if (spi->mode & SPI_CPHA)
+ val &= ~CTRL_CKE;
+ else
+ val |= CTRL_CKE;
+
+ /* rx at end of tx */
+ val |= CTRL_SMP;
+ writel(val, &pic32s->regs->ctrl);
+ pic32s->mode = spi->mode;
+ }
+
+ return 0;
+}
+
+static bool pic32_spi_can_dma(struct spi_master *master,
+ struct spi_device *spi,
+ struct spi_transfer *xfer)
+{
+ struct pic32_spi *pic32s = spi_master_get_devdata(master);
+
+ /* skip using DMA on small size transfer to avoid overhead.*/
+ return (xfer->len >= PIC32_DMA_LEN_MIN) &&
+ test_bit(PIC32F_DMA_PREP, &pic32s->flags);
+}
+
+static int pic32_spi_one_transfer(struct spi_master *master,
+ struct spi_device *spi,
+ struct spi_transfer *transfer)
+{
+ struct pic32_spi *pic32s;
+ bool dma_issued = false;
+ int ret;
+
+ pic32s = spi_master_get_devdata(master);
+
+ /* handle transfer specific word size change */
+ if (transfer->bits_per_word &&
+ (transfer->bits_per_word != pic32s->bits_per_word)) {
+ ret = pic32_spi_set_word_size(pic32s, transfer->bits_per_word);
+ if (ret)
+ return ret;
+ pic32s->bits_per_word = transfer->bits_per_word;
+ }
+
+ /* handle transfer specific speed change */
+ if (transfer->speed_hz && (transfer->speed_hz != pic32s->speed_hz)) {
+ pic32_spi_set_clk_rate(pic32s, transfer->speed_hz);
+ pic32s->speed_hz = transfer->speed_hz;
+ }
+
+ reinit_completion(&pic32s->xfer_done);
+
+ /* transact by DMA mode */
+ if (transfer->rx_sg.nents && transfer->tx_sg.nents) {
+ ret = pic32_spi_dma_transfer(pic32s, transfer);
+ if (ret) {
+ dev_err(&spi->dev, "dma submit error\n");
+ return ret;
+ }
+
+ /* DMA issued */
+ dma_issued = true;
+ } else {
+ /* set current transfer information */
+ pic32s->tx = (const void *)transfer->tx_buf;
+ pic32s->rx = (const void *)transfer->rx_buf;
+ pic32s->tx_end = pic32s->tx + transfer->len;
+ pic32s->rx_end = pic32s->rx + transfer->len;
+ pic32s->len = transfer->len;
+
+ /* transact by interrupt driven PIO */
+ enable_irq(pic32s->fault_irq);
+ enable_irq(pic32s->rx_irq);
+ enable_irq(pic32s->tx_irq);
+ }
+
+ /* wait for completion */
+ ret = wait_for_completion_timeout(&pic32s->xfer_done, 2 * HZ);
+ if (ret <= 0) {
+ dev_err(&spi->dev, "wait error/timedout\n");
+ if (dma_issued) {
+ dmaengine_terminate_all(master->dma_rx);
+ dmaengine_terminate_all(master->dma_rx);
+ }
+ ret = -ETIMEDOUT;
+ } else {
+ ret = 0;
+ }
+
+ return ret;
+}
+
+static int pic32_spi_unprepare_message(struct spi_master *master,
+ struct spi_message *msg)
+{
+ /* nothing to do */
+ return 0;
+}
+
+static int pic32_spi_unprepare_hardware(struct spi_master *master)
+{
+ struct pic32_spi *pic32s = spi_master_get_devdata(master);
+
+ pic32_spi_disable(pic32s);
+
+ return 0;
+}
+
+/* This may be called multiple times by same spi dev */
+static int pic32_spi_setup(struct spi_device *spi)
+{
+ if (!spi->max_speed_hz) {
+ dev_err(&spi->dev, "No max speed HZ parameter\n");
+ return -EINVAL;
+ }
+
+ /* PIC32 spi controller can drive /CS during transfer depending
+ * on tx fifo fill-level. /CS will stay asserted as long as TX
+ * fifo is non-empty, else will be deasserted indicating
+ * completion of the ongoing transfer. This might result into
+ * unreliable/erroneous SPI transactions.
+ * To avoid that we will always handle /CS by toggling GPIO.
+ */
+ if (!gpio_is_valid(spi->cs_gpio))
+ return -EINVAL;
+
+ gpio_direction_output(spi->cs_gpio, !(spi->mode & SPI_CS_HIGH));
+
+ return 0;
+}
+
+static void pic32_spi_cleanup(struct spi_device *spi)
+{
+ /* de-activate cs-gpio */
+ gpio_direction_output(spi->cs_gpio, !(spi->mode & SPI_CS_HIGH));
+}
+
+static void pic32_spi_dma_prep(struct pic32_spi *pic32s, struct device *dev)
+{
+ struct spi_master *master = pic32s->master;
+ dma_cap_mask_t mask;
+
+ dma_cap_zero(mask);
+ dma_cap_set(DMA_SLAVE, mask);
+
+ master->dma_rx = dma_request_slave_channel_compat(mask, NULL, NULL,
+ dev, "spi-rx");
+ if (!master->dma_rx) {
+ dev_warn(dev, "RX channel not found.\n");
+ goto out_err;
+ }
+
+ master->dma_tx = dma_request_slave_channel_compat(mask, NULL, NULL,
+ dev, "spi-tx");
+ if (!master->dma_tx) {
+ dev_warn(dev, "TX channel not found.\n");
+ goto out_err;
+ }
+
+ if (pic32_spi_dma_config(pic32s, DMA_SLAVE_BUSWIDTH_1_BYTE))
+ goto out_err;
+
+ /* DMA chnls allocated and prepared */
+ set_bit(PIC32F_DMA_PREP, &pic32s->flags);
+
+ return;
+
+out_err:
+ if (master->dma_rx)
+ dma_release_channel(master->dma_rx);
+
+ if (master->dma_tx)
+ dma_release_channel(master->dma_tx);
+}
+
+static void pic32_spi_dma_unprep(struct pic32_spi *pic32s)
+{
+ if (!test_bit(PIC32F_DMA_PREP, &pic32s->flags))
+ return;
+
+ clear_bit(PIC32F_DMA_PREP, &pic32s->flags);
+ if (pic32s->master->dma_rx)
+ dma_release_channel(pic32s->master->dma_rx);
+
+ if (pic32s->master->dma_tx)
+ dma_release_channel(pic32s->master->dma_tx);
+}
+
+static void pic32_spi_hw_init(struct pic32_spi *pic32s)
+{
+ u32 ctrl;
+
+ /* disable hardware */
+ pic32_spi_disable(pic32s);
+
+ ctrl = readl(&pic32s->regs->ctrl);
+ /* enable enhanced fifo of 128bit deep */
+ ctrl |= CTRL_ENHBUF;
+ pic32s->fifo_n_byte = 16;
+
+ /* disable framing mode */
+ ctrl &= ~CTRL_FRMEN;
+
+ /* enable master mode while disabled */
+ ctrl |= CTRL_MSTEN;
+
+ /* set tx fifo threshold interrupt */
+ ctrl &= ~(0x3 << CTRL_TX_INT_SHIFT);
+ ctrl |= (TX_FIFO_HALF_EMPTY << CTRL_TX_INT_SHIFT);
+
+ /* set rx fifo threshold interrupt */
+ ctrl &= ~(0x3 << CTRL_RX_INT_SHIFT);
+ ctrl |= (RX_FIFO_NOT_EMPTY << CTRL_RX_INT_SHIFT);
+
+ /* select clk source */
+ ctrl &= ~CTRL_MCLKSEL;
+
+ /* set manual /CS mode */
+ ctrl &= ~CTRL_MSSEN;
+
+ writel(ctrl, &pic32s->regs->ctrl);
+
+ /* enable error reporting */
+ ctrl = CTRL2_TX_UR_EN | CTRL2_RX_OV_EN | CTRL2_FRM_ERR_EN;
+ writel(ctrl, &pic32s->regs->ctrl2_set);
+}
+
+static int pic32_spi_hw_probe(struct platform_device *pdev,
+ struct pic32_spi *pic32s)
+{
+ struct resource *mem;
+ int ret;
+
+ mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ pic32s->regs = devm_ioremap_resource(&pdev->dev, mem);
+ if (IS_ERR(pic32s->regs))
+ return PTR_ERR(pic32s->regs);
+
+ pic32s->dma_base = mem->start;
+
+ /* get irq resources: err-irq, rx-irq, tx-irq */
+ pic32s->fault_irq = platform_get_irq_byname(pdev, "fault");
+ if (pic32s->fault_irq < 0) {
+ dev_err(&pdev->dev, "fault-irq not found\n");
+ return pic32s->fault_irq;
+ }
+
+ pic32s->rx_irq = platform_get_irq_byname(pdev, "rx");
+ if (pic32s->rx_irq < 0) {
+ dev_err(&pdev->dev, "rx-irq not found\n");
+ return pic32s->rx_irq;
+ }
+
+ pic32s->tx_irq = platform_get_irq_byname(pdev, "tx");
+ if (pic32s->tx_irq < 0) {
+ dev_err(&pdev->dev, "tx-irq not found\n");
+ return pic32s->tx_irq;
+ }
+
+ /* get clock */
+ pic32s->clk = devm_clk_get(&pdev->dev, "mck0");
+ if (IS_ERR(pic32s->clk)) {
+ dev_err(&pdev->dev, "clk not found\n");
+ ret = PTR_ERR(pic32s->clk);
+ goto err_unmap_mem;
+ }
+
+ ret = clk_prepare_enable(pic32s->clk);
+ if (ret)
+ goto err_unmap_mem;
+
+ pic32_spi_hw_init(pic32s);
+
+ return 0;
+
+err_unmap_mem:
+ dev_err(&pdev->dev, "%s failed, err %d\n", __func__, ret);
+ return ret;
+}
+
+static int pic32_spi_probe(struct platform_device *pdev)
+{
+ struct spi_master *master;
+ struct pic32_spi *pic32s;
+ int ret;
+
+ master = spi_alloc_master(&pdev->dev, sizeof(*pic32s));
+ if (!master)
+ return -ENOMEM;
+
+ pic32s = spi_master_get_devdata(master);
+ pic32s->master = master;
+
+ ret = pic32_spi_hw_probe(pdev, pic32s);
+ if (ret)
+ goto err_master;
+
+ master->dev.of_node = of_node_get(pdev->dev.of_node);
+ master->mode_bits = SPI_MODE_3 | SPI_MODE_0 | SPI_CS_HIGH;
+ master->num_chipselect = 1; /* single chip-select */
+ master->max_speed_hz = clk_get_rate(pic32s->clk);
+ master->setup = pic32_spi_setup;
+ master->cleanup = pic32_spi_cleanup;
+ master->flags = SPI_MASTER_MUST_TX | SPI_MASTER_MUST_RX;
+ master->bits_per_word_mask = SPI_BPW_MASK(8) | SPI_BPW_MASK(16) |
+ SPI_BPW_MASK(32);
+ master->transfer_one = pic32_spi_one_transfer;
+ master->prepare_message = pic32_spi_prepare_message;
+ master->unprepare_message = pic32_spi_unprepare_message;
+ master->prepare_transfer_hardware = pic32_spi_prepare_hardware;
+ master->unprepare_transfer_hardware = pic32_spi_unprepare_hardware;
+
+ /* optional DMA support */
+ pic32_spi_dma_prep(pic32s, &pdev->dev);
+ if (test_bit(PIC32F_DMA_PREP, &pic32s->flags))
+ master->can_dma = pic32_spi_can_dma;
+
+ init_completion(&pic32s->xfer_done);
+ pic32s->mode = -1;
+
+ /* install irq handlers (with irq-disabled) */
+ irq_set_status_flags(pic32s->fault_irq, IRQ_NOAUTOEN);
+ ret = devm_request_irq(&pdev->dev, pic32s->fault_irq,
+ pic32_spi_fault_irq, IRQF_NO_THREAD,
+ dev_name(&pdev->dev), pic32s);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "request fault-irq %d\n", pic32s->rx_irq);
+ goto err_bailout;
+ }
+
+ /* receive interrupt handler */
+ irq_set_status_flags(pic32s->rx_irq, IRQ_NOAUTOEN);
+ ret = devm_request_irq(&pdev->dev, pic32s->rx_irq,
+ pic32_spi_rx_irq, IRQF_NO_THREAD,
+ dev_name(&pdev->dev), pic32s);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "request rx-irq %d\n", pic32s->rx_irq);
+ goto err_bailout;
+ }
+
+ /* transmit interrupt handler */
+ irq_set_status_flags(pic32s->tx_irq, IRQ_NOAUTOEN);
+ ret = devm_request_irq(&pdev->dev, pic32s->tx_irq,
+ pic32_spi_tx_irq, IRQF_NO_THREAD,
+ dev_name(&pdev->dev), pic32s);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "request tx-irq %d\n", pic32s->tx_irq);
+ goto err_bailout;
+ }
+
+ /* register master */
+ ret = devm_spi_register_master(&pdev->dev, master);
+ if (ret) {
+ dev_err(&master->dev, "failed registering spi master\n");
+ goto err_bailout;
+ }
+
+ platform_set_drvdata(pdev, pic32s);
+
+ return 0;
+
+err_bailout:
+ clk_disable_unprepare(pic32s->clk);
+err_master:
+ spi_master_put(master);
+ return ret;
+}
+
+static int pic32_spi_remove(struct platform_device *pdev)
+{
+ struct pic32_spi *pic32s;
+
+ pic32s = platform_get_drvdata(pdev);
+ pic32_spi_disable(pic32s);
+ clk_disable_unprepare(pic32s->clk);
+ pic32_spi_dma_unprep(pic32s);
+
+ return 0;
+}
+
+static const struct of_device_id pic32_spi_of_match[] = {
+ {.compatible = "microchip,pic32mzda-spi",},
+ {},
+};
+MODULE_DEVICE_TABLE(of, pic32_spi_of_match);
+
+static struct platform_driver pic32_spi_driver = {
+ .driver = {
+ .name = "spi-pic32",
+ .of_match_table = of_match_ptr(pic32_spi_of_match),
+ },
+ .probe = pic32_spi_probe,
+ .remove = pic32_spi_remove,
+};
+
+module_platform_driver(pic32_spi_driver);
+
+MODULE_AUTHOR("Purna Chandra Mandal <purna.mandal@microchip.com>");
+MODULE_DESCRIPTION("Microchip SPI driver for PIC32 SPI controller.");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/spi/spi-pxa2xx-dma.c b/drivers/spi/spi-pxa2xx-dma.c
index 365fc22..a18a03d 100644
--- a/drivers/spi/spi-pxa2xx-dma.c
+++ b/drivers/spi/spi-pxa2xx-dma.c
@@ -33,12 +33,10 @@
dmadev = drv_data->tx_chan->device->dev;
sgt = &drv_data->tx_sgt;
buf = drv_data->tx;
- drv_data->tx_map_len = len;
} else {
dmadev = drv_data->rx_chan->device->dev;
sgt = &drv_data->rx_sgt;
buf = drv_data->rx;
- drv_data->rx_map_len = len;
}
nents = DIV_ROUND_UP(len, SZ_2K);
@@ -55,11 +53,7 @@
for_each_sg(sgt->sgl, sg, sgt->nents, i) {
size_t bytes = min_t(size_t, len, SZ_2K);
- if (buf)
- sg_set_buf(sg, pbuf, bytes);
- else
- sg_set_buf(sg, drv_data->dummy, bytes);
-
+ sg_set_buf(sg, pbuf, bytes);
pbuf += bytes;
len -= bytes;
}
@@ -133,9 +127,6 @@
if (!error) {
pxa2xx_spi_unmap_dma_buffers(drv_data);
- drv_data->tx += drv_data->tx_map_len;
- drv_data->rx += drv_data->rx_map_len;
-
msg->actual_length += drv_data->len;
msg->state = pxa2xx_spi_next_transfer(drv_data);
} else {
@@ -267,19 +258,22 @@
int pxa2xx_spi_dma_prepare(struct driver_data *drv_data, u32 dma_burst)
{
struct dma_async_tx_descriptor *tx_desc, *rx_desc;
+ int err = 0;
tx_desc = pxa2xx_spi_dma_prepare_one(drv_data, DMA_MEM_TO_DEV);
if (!tx_desc) {
dev_err(&drv_data->pdev->dev,
"failed to get DMA TX descriptor\n");
- return -EBUSY;
+ err = -EBUSY;
+ goto err_tx;
}
rx_desc = pxa2xx_spi_dma_prepare_one(drv_data, DMA_DEV_TO_MEM);
if (!rx_desc) {
dev_err(&drv_data->pdev->dev,
"failed to get DMA RX descriptor\n");
- return -EBUSY;
+ err = -EBUSY;
+ goto err_rx;
}
/* We are ready when RX completes */
@@ -289,6 +283,12 @@
dmaengine_submit(rx_desc);
dmaengine_submit(tx_desc);
return 0;
+
+err_rx:
+ dmaengine_terminate_async(drv_data->tx_chan);
+err_tx:
+ pxa2xx_spi_unmap_dma_buffers(drv_data);
+ return err;
}
void pxa2xx_spi_dma_start(struct driver_data *drv_data)
@@ -308,10 +308,6 @@
dma_cap_zero(mask);
dma_cap_set(DMA_SLAVE, mask);
- drv_data->dummy = devm_kzalloc(dev, SZ_2K, GFP_KERNEL);
- if (!drv_data->dummy)
- return -ENOMEM;
-
drv_data->tx_chan = dma_request_slave_channel_compat(mask,
pdata->dma_filter, pdata->tx_param, dev, "tx");
if (!drv_data->tx_chan)
diff --git a/drivers/spi/spi-pxa2xx-pci.c b/drivers/spi/spi-pxa2xx-pci.c
index 520ed1d..5202de9 100644
--- a/drivers/spi/spi-pxa2xx-pci.c
+++ b/drivers/spi/spi-pxa2xx-pci.c
@@ -144,16 +144,16 @@
struct dw_dma_slave *slave = c->tx_param;
slave->dma_dev = &dma_dev->dev;
- slave->src_master = 1;
- slave->dst_master = 0;
+ slave->m_master = 0;
+ slave->p_master = 1;
}
if (c->rx_param) {
struct dw_dma_slave *slave = c->rx_param;
slave->dma_dev = &dma_dev->dev;
- slave->src_master = 1;
- slave->dst_master = 0;
+ slave->m_master = 0;
+ slave->p_master = 1;
}
spi_pdata.dma_filter = lpss_dma_filter;
@@ -173,8 +173,8 @@
ssp->type = c->type;
snprintf(buf, sizeof(buf), "pxa2xx-spi.%d", ssp->port_id);
- ssp->clk = clk_register_fixed_rate(&dev->dev, buf , NULL,
- CLK_IS_ROOT, c->max_clk_rate);
+ ssp->clk = clk_register_fixed_rate(&dev->dev, buf , NULL, 0,
+ c->max_clk_rate);
if (IS_ERR(ssp->clk))
return PTR_ERR(ssp->clk);
diff --git a/drivers/spi/spi-pxa2xx.c b/drivers/spi/spi-pxa2xx.c
index 86138e4..fe07c05 100644
--- a/drivers/spi/spi-pxa2xx.c
+++ b/drivers/spi/spi-pxa2xx.c
@@ -570,9 +570,8 @@
/* see if the next and current messages point
* to the same chip
*/
- if (next_msg && next_msg->spi != msg->spi)
- next_msg = NULL;
- if (!next_msg || msg->state == ERROR_STATE)
+ if ((next_msg && next_msg->spi != msg->spi) ||
+ msg->state == ERROR_STATE)
cs_deassert(drv_data);
}
@@ -928,6 +927,7 @@
u32 dma_thresh = drv_data->cur_chip->dma_threshold;
u32 dma_burst = drv_data->cur_chip->dma_burst_size;
u32 change_mask = pxa2xx_spi_get_ssrc1_change_mask(drv_data);
+ int err;
/* Get current state information */
message = drv_data->cur_msg;
@@ -1047,7 +1047,12 @@
/* Ensure we have the correct interrupt handler */
drv_data->transfer_handler = pxa2xx_spi_dma_transfer;
- pxa2xx_spi_dma_prepare(drv_data, dma_burst);
+ err = pxa2xx_spi_dma_prepare(drv_data, dma_burst);
+ if (err) {
+ message->status = err;
+ giveback(drv_data);
+ return;
+ }
/* Clear status and start DMA engine */
cr1 = chip->cr1 | dma_thresh | drv_data->dma_cr1;
@@ -1543,7 +1548,6 @@
drv_data->pdev = pdev;
drv_data->ssp = ssp;
- master->dev.parent = &pdev->dev;
master->dev.of_node = pdev->dev.of_node;
/* the spi->mode bits understood by this driver: */
master->mode_bits = SPI_CPOL | SPI_CPHA | SPI_CS_HIGH | SPI_LOOP;
@@ -1556,6 +1560,7 @@
master->unprepare_transfer_hardware = pxa2xx_spi_unprepare_transfer;
master->fw_translate_cs = pxa2xx_spi_fw_translate_cs;
master->auto_runtime_pm = true;
+ master->flags = SPI_MASTER_MUST_RX | SPI_MASTER_MUST_TX;
drv_data->ssp_type = ssp->type;
diff --git a/drivers/spi/spi-pxa2xx.h b/drivers/spi/spi-pxa2xx.h
index a1ef889..e6b0900 100644
--- a/drivers/spi/spi-pxa2xx.h
+++ b/drivers/spi/spi-pxa2xx.h
@@ -56,7 +56,6 @@
struct sg_table tx_sgt;
int rx_nents;
int tx_nents;
- void *dummy;
atomic_t dma_running;
/* Current message transfer state info */
@@ -69,8 +68,6 @@
void *rx;
void *rx_end;
int dma_mapped;
- size_t rx_map_len;
- size_t tx_map_len;
u8 n_bytes;
int (*write)(struct driver_data *drv_data);
int (*read)(struct driver_data *drv_data);
diff --git a/drivers/spi/spi-qup.c b/drivers/spi/spi-qup.c
index 810a7fa..c338ef1 100644
--- a/drivers/spi/spi-qup.c
+++ b/drivers/spi/spi-qup.c
@@ -937,6 +937,10 @@
config = readl(controller->base + QUP_CONFIG);
config |= QUP_CONFIG_CLOCK_AUTO_GATE;
writel_relaxed(config, controller->base + QUP_CONFIG);
+
+ clk_disable_unprepare(controller->cclk);
+ clk_disable_unprepare(controller->iclk);
+
return 0;
}
@@ -945,6 +949,15 @@
struct spi_master *master = dev_get_drvdata(device);
struct spi_qup *controller = spi_master_get_devdata(master);
u32 config;
+ int ret;
+
+ ret = clk_prepare_enable(controller->iclk);
+ if (ret)
+ return ret;
+
+ ret = clk_prepare_enable(controller->cclk);
+ if (ret)
+ return ret;
/* Disable clocks auto gaiting */
config = readl_relaxed(controller->base + QUP_CONFIG);
@@ -1017,6 +1030,8 @@
pm_runtime_put_noidle(&pdev->dev);
pm_runtime_disable(&pdev->dev);
+ spi_master_put(master);
+
return 0;
}
diff --git a/drivers/spi/spi-rockchip.c b/drivers/spi/spi-rockchip.c
index 6c6c001..cd89682 100644
--- a/drivers/spi/spi-rockchip.c
+++ b/drivers/spi/spi-rockchip.c
@@ -744,10 +744,8 @@
rs->dma_rx.ch = dma_request_chan(rs->dev, "rx");
if (IS_ERR(rs->dma_rx.ch)) {
if (PTR_ERR(rs->dma_rx.ch) == -EPROBE_DEFER) {
- dma_release_channel(rs->dma_tx.ch);
- rs->dma_tx.ch = NULL;
ret = -EPROBE_DEFER;
- goto err_get_fifo_len;
+ goto err_free_dma_tx;
}
dev_warn(rs->dev, "Failed to request RX DMA channel\n");
rs->dma_rx.ch = NULL;
@@ -775,10 +773,11 @@
err_register_master:
pm_runtime_disable(&pdev->dev);
- if (rs->dma_tx.ch)
- dma_release_channel(rs->dma_tx.ch);
if (rs->dma_rx.ch)
dma_release_channel(rs->dma_rx.ch);
+err_free_dma_tx:
+ if (rs->dma_tx.ch)
+ dma_release_channel(rs->dma_tx.ch);
err_get_fifo_len:
clk_disable_unprepare(rs->spiclk);
err_spiclk_enable:
diff --git a/drivers/spi/spi-st-ssc4.c b/drivers/spi/spi-st-ssc4.c
index f17c0ab..d5adf9f 100644
--- a/drivers/spi/spi-st-ssc4.c
+++ b/drivers/spi/spi-st-ssc4.c
@@ -345,12 +345,13 @@
spi_st->clk = devm_clk_get(&pdev->dev, "ssc");
if (IS_ERR(spi_st->clk)) {
dev_err(&pdev->dev, "Unable to request clock\n");
- return PTR_ERR(spi_st->clk);
+ ret = PTR_ERR(spi_st->clk);
+ goto put_master;
}
ret = spi_st_clk_enable(spi_st);
if (ret)
- return ret;
+ goto put_master;
init_completion(&spi_st->done);
@@ -408,7 +409,8 @@
clk_disable:
spi_st_clk_disable(spi_st);
-
+put_master:
+ spi_master_put(master);
return ret;
}
diff --git a/drivers/spi/spi-zynqmp-gqspi.c b/drivers/spi/spi-zynqmp-gqspi.c
index aab9b49..18aeace 100644
--- a/drivers/spi/spi-zynqmp-gqspi.c
+++ b/drivers/spi/spi-zynqmp-gqspi.c
@@ -360,7 +360,7 @@
ret = clk_enable(xqspi->refclk);
if (ret)
- goto clk_err;
+ return ret;
ret = clk_enable(xqspi->pclk);
if (ret)
@@ -369,6 +369,7 @@
zynqmp_gqspi_write(xqspi, GQSPI_EN_OFST, GQSPI_EN_MASK);
return 0;
clk_err:
+ clk_disable(xqspi->refclk);
return ret;
}
diff --git a/drivers/spi/spi.c b/drivers/spi/spi.c
index 0239b45..77e6e45 100644
--- a/drivers/spi/spi.c
+++ b/drivers/spi/spi.c
@@ -717,9 +717,11 @@
if (vmalloced_buf) {
desc_len = min_t(int, max_seg_size, PAGE_SIZE);
sgs = DIV_ROUND_UP(len + offset_in_page(buf), desc_len);
- } else {
+ } else if (virt_addr_valid(buf)) {
desc_len = min_t(int, max_seg_size, master->max_dma_len);
sgs = DIV_ROUND_UP(len, desc_len);
+ } else {
+ return -EINVAL;
}
ret = sg_alloc_table(sgt, sgs, GFP_KERNEL);
@@ -933,7 +935,7 @@
* spi_transfer_one_message - Default implementation of transfer_one_message()
*
* This is a standard implementation of transfer_one_message() for
- * drivers which impelment a transfer_one() operation. It provides
+ * drivers which implement a transfer_one() operation. It provides
* standard handling of delays and chip select management.
*/
static int spi_transfer_one_message(struct spi_master *master,
@@ -1764,6 +1766,7 @@
master->num_chipselect = 1;
master->dev.class = &spi_master_class;
master->dev.parent = dev;
+ pm_suspend_ignore_children(&master->dev, true);
spi_master_set_devdata(master, &master[1]);
return master;
diff --git a/drivers/spmi/spmi.c b/drivers/spmi/spmi.c
index 6b3da1b..2b9b094 100644
--- a/drivers/spmi/spmi.c
+++ b/drivers/spmi/spmi.c
@@ -25,6 +25,7 @@
#define CREATE_TRACE_POINTS
#include <trace/events/spmi.h>
+static bool is_registered;
static DEFINE_IDA(ctrl_ida);
static void spmi_dev_release(struct device *dev)
@@ -507,7 +508,7 @@
int ret;
/* Can't register until after driver model init */
- if (WARN_ON(!spmi_bus_type.p))
+ if (WARN_ON(!is_registered))
return -EAGAIN;
ret = device_add(&ctrl->dev);
@@ -576,7 +577,14 @@
static int __init spmi_init(void)
{
- return bus_register(&spmi_bus_type);
+ int ret;
+
+ ret = bus_register(&spmi_bus_type);
+ if (ret)
+ return ret;
+
+ is_registered = true;
+ return 0;
}
postcore_initcall(spmi_init);
diff --git a/drivers/staging/android/Kconfig b/drivers/staging/android/Kconfig
index bd90d20..6480f60 100644
--- a/drivers/staging/android/Kconfig
+++ b/drivers/staging/android/Kconfig
@@ -14,27 +14,13 @@
It is, in theory, a good memory allocator for low-memory devices,
because it can discard shared memory units when under memory pressure.
-config ANDROID_TIMED_OUTPUT
- bool "Timed output class driver"
- default y
-
-config ANDROID_TIMED_GPIO
- tristate "Android timed gpio driver"
- depends on GPIOLIB || COMPILE_TEST
- depends on ANDROID_TIMED_OUTPUT
- default n
- ---help---
- Unlike generic gpio is to allow programs to access and manipulate gpio
- registers from user space, timed output/gpio is a system to allow changing
- a gpio pin and restore it automatically after a specified timeout.
-
config ANDROID_LOW_MEMORY_KILLER
bool "Android Low Memory Killer"
---help---
Registers processes to be killed when low memory conditions, this is useful
as there is no particular swap space on android.
- The registered process will kills according to the priorities in android init
+ The registered process will kill according to the priorities in android init
scripts (/init.rc), and it defines priority values with minimum free memory size
for each priority.
@@ -52,6 +38,7 @@
bool "Software synchronization objects"
default n
depends on SYNC
+ depends on SYNC_FILE
---help---
A sync object driver that uses a 32bit counter to coordinate
synchronization. Useful when there is no hardware primitive backing
diff --git a/drivers/staging/android/Makefile b/drivers/staging/android/Makefile
index c7b6c99..980d6dc 100644
--- a/drivers/staging/android/Makefile
+++ b/drivers/staging/android/Makefile
@@ -3,8 +3,6 @@
obj-y += ion/
obj-$(CONFIG_ASHMEM) += ashmem.o
-obj-$(CONFIG_ANDROID_TIMED_OUTPUT) += timed_output.o
-obj-$(CONFIG_ANDROID_TIMED_GPIO) += timed_gpio.o
obj-$(CONFIG_ANDROID_LOW_MEMORY_KILLER) += lowmemorykiller.o
obj-$(CONFIG_SYNC) += sync.o sync_debug.o
obj-$(CONFIG_SW_SYNC) += sw_sync.o
diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c
index 8536567..a2cf93b 100644
--- a/drivers/staging/android/ion/ion.c
+++ b/drivers/staging/android/ion/ion.c
@@ -184,7 +184,7 @@
struct scatterlist *sg;
int i, ret;
- buffer = kzalloc(sizeof(struct ion_buffer), GFP_KERNEL);
+ buffer = kzalloc(sizeof(*buffer), GFP_KERNEL);
if (!buffer)
return ERR_PTR(-ENOMEM);
@@ -341,7 +341,7 @@
{
struct ion_handle *handle;
- handle = kzalloc(sizeof(struct ion_handle), GFP_KERNEL);
+ handle = kzalloc(sizeof(*handle), GFP_KERNEL);
if (!handle)
return ERR_PTR(-ENOMEM);
kref_init(&handle->ref);
@@ -396,7 +396,7 @@
return ret;
}
-int ion_handle_put(struct ion_handle *handle)
+static int ion_handle_put(struct ion_handle *handle)
{
struct ion_client *client = handle->client;
int ret;
@@ -438,8 +438,8 @@
return handle ? handle : ERR_PTR(-EINVAL);
}
-struct ion_handle *ion_handle_get_by_id(struct ion_client *client,
- int id)
+static struct ion_handle *ion_handle_get_by_id(struct ion_client *client,
+ int id)
{
struct ion_handle *handle;
@@ -827,7 +827,7 @@
}
task_unlock(current->group_leader);
- client = kzalloc(sizeof(struct ion_client), GFP_KERNEL);
+ client = kzalloc(sizeof(*client), GFP_KERNEL);
if (!client)
goto err_put_task_struct;
@@ -1035,7 +1035,7 @@
struct ion_buffer *buffer = vma->vm_private_data;
struct ion_vma_list *vma_list;
- vma_list = kmalloc(sizeof(struct ion_vma_list), GFP_KERNEL);
+ vma_list = kmalloc(sizeof(*vma_list), GFP_KERNEL);
if (!vma_list)
return;
vma_list->vma = vma;
@@ -1650,7 +1650,7 @@
struct ion_device *idev;
int ret;
- idev = kzalloc(sizeof(struct ion_device), GFP_KERNEL);
+ idev = kzalloc(sizeof(*idev), GFP_KERNEL);
if (!idev)
return ERR_PTR(-ENOMEM);
diff --git a/drivers/staging/android/ion/ion_chunk_heap.c b/drivers/staging/android/ion/ion_chunk_heap.c
index 0813163..e0553fe 100644
--- a/drivers/staging/android/ion/ion_chunk_heap.c
+++ b/drivers/staging/android/ion/ion_chunk_heap.c
@@ -55,7 +55,7 @@
if (allocated_size > chunk_heap->size - chunk_heap->allocated)
return -ENOMEM;
- table = kmalloc(sizeof(struct sg_table), GFP_KERNEL);
+ table = kmalloc(sizeof(*table), GFP_KERNEL);
if (!table)
return -ENOMEM;
ret = sg_alloc_table(table, num_chunks, GFP_KERNEL);
@@ -154,7 +154,7 @@
if (ret)
return ERR_PTR(ret);
- chunk_heap = kzalloc(sizeof(struct ion_chunk_heap), GFP_KERNEL);
+ chunk_heap = kzalloc(sizeof(*chunk_heap), GFP_KERNEL);
if (!chunk_heap)
return ERR_PTR(-ENOMEM);
diff --git a/drivers/staging/android/ion/ion_dummy_driver.c b/drivers/staging/android/ion/ion_dummy_driver.c
index 5678870..814a3c9 100644
--- a/drivers/staging/android/ion/ion_dummy_driver.c
+++ b/drivers/staging/android/ion/ion_dummy_driver.c
@@ -68,6 +68,8 @@
int i, err;
idev = ion_device_create(NULL);
+ if (IS_ERR(idev))
+ return PTR_ERR(idev);
heaps = kcalloc(dummy_ion_pdata.nr, sizeof(struct ion_heap *),
GFP_KERNEL);
if (!heaps)
diff --git a/drivers/staging/android/ion/ion_test.c b/drivers/staging/android/ion/ion_test.c
index 83a3af0..5a396a1 100644
--- a/drivers/staging/android/ion/ion_test.c
+++ b/drivers/staging/android/ion/ion_test.c
@@ -208,7 +208,7 @@
struct ion_test_data *data;
struct miscdevice *miscdev = file->private_data;
- data = kzalloc(sizeof(struct ion_test_data), GFP_KERNEL);
+ data = kzalloc(sizeof(*data), GFP_KERNEL);
if (!data)
return -ENOMEM;
diff --git a/drivers/staging/android/lowmemorykiller.c b/drivers/staging/android/lowmemorykiller.c
index 2509e5d..24d2745 100644
--- a/drivers/staging/android/lowmemorykiller.c
+++ b/drivers/staging/android/lowmemorykiller.c
@@ -131,7 +131,7 @@
if (!p)
continue;
- if (test_tsk_thread_flag(p, TIF_MEMDIE) &&
+ if (task_lmk_waiting(p) &&
time_before_eq(jiffies, lowmem_deathpending_timeout)) {
task_unlock(p);
rcu_read_unlock();
@@ -162,13 +162,8 @@
if (selected) {
task_lock(selected);
send_sig(SIGKILL, selected, 0);
- /*
- * FIXME: lowmemorykiller shouldn't abuse global OOM killer
- * infrastructure. There is no real reason why the selected
- * task should have access to the memory reserves.
- */
if (selected->mm)
- mark_oom_victim(selected);
+ task_set_lmk_waiting(selected);
task_unlock(selected);
lowmem_print(1, "Killing '%s' (%d), adj %hd,\n"
" to free %ldkB on behalf of '%s' (%d) because\n"
diff --git a/drivers/staging/android/sync.c b/drivers/staging/android/sync.c
index 3a8f210..1d14c83 100644
--- a/drivers/staging/android/sync.c
+++ b/drivers/staging/android/sync.c
@@ -16,10 +16,7 @@
#include <linux/debugfs.h>
#include <linux/export.h>
-#include <linux/file.h>
-#include <linux/fs.h>
#include <linux/kernel.h>
-#include <linux/poll.h>
#include <linux/sched.h>
#include <linux/seq_file.h>
#include <linux/slab.h>
@@ -32,7 +29,6 @@
#include "trace/sync.h"
static const struct fence_ops android_fence_ops;
-static const struct file_operations sync_file_fops;
struct sync_timeline *sync_timeline_create(const struct sync_timeline_ops *ops,
int size, const char *name)
@@ -136,170 +132,6 @@
}
EXPORT_SYMBOL(sync_pt_create);
-static struct sync_file *sync_file_alloc(int size, const char *name)
-{
- struct sync_file *sync_file;
-
- sync_file = kzalloc(size, GFP_KERNEL);
- if (!sync_file)
- return NULL;
-
- sync_file->file = anon_inode_getfile("sync_file", &sync_file_fops,
- sync_file, 0);
- if (IS_ERR(sync_file->file))
- goto err;
-
- kref_init(&sync_file->kref);
- strlcpy(sync_file->name, name, sizeof(sync_file->name));
-
- init_waitqueue_head(&sync_file->wq);
-
- return sync_file;
-
-err:
- kfree(sync_file);
- return NULL;
-}
-
-static void fence_check_cb_func(struct fence *f, struct fence_cb *cb)
-{
- struct sync_file_cb *check;
- struct sync_file *sync_file;
-
- check = container_of(cb, struct sync_file_cb, cb);
- sync_file = check->sync_file;
-
- if (atomic_dec_and_test(&sync_file->status))
- wake_up_all(&sync_file->wq);
-}
-
-/* TODO: implement a create which takes more that one fence */
-struct sync_file *sync_file_create(const char *name, struct fence *fence)
-{
- struct sync_file *sync_file;
-
- sync_file = sync_file_alloc(offsetof(struct sync_file, cbs[1]),
- name);
- if (!sync_file)
- return NULL;
-
- sync_file->num_fences = 1;
- atomic_set(&sync_file->status, 1);
-
- sync_file->cbs[0].fence = fence;
- sync_file->cbs[0].sync_file = sync_file;
- if (fence_add_callback(fence, &sync_file->cbs[0].cb,
- fence_check_cb_func))
- atomic_dec(&sync_file->status);
-
- sync_file_debug_add(sync_file);
-
- return sync_file;
-}
-EXPORT_SYMBOL(sync_file_create);
-
-struct sync_file *sync_file_fdget(int fd)
-{
- struct file *file = fget(fd);
-
- if (!file)
- return NULL;
-
- if (file->f_op != &sync_file_fops)
- goto err;
-
- return file->private_data;
-
-err:
- fput(file);
- return NULL;
-}
-EXPORT_SYMBOL(sync_file_fdget);
-
-void sync_file_put(struct sync_file *sync_file)
-{
- fput(sync_file->file);
-}
-EXPORT_SYMBOL(sync_file_put);
-
-void sync_file_install(struct sync_file *sync_file, int fd)
-{
- fd_install(fd, sync_file->file);
-}
-EXPORT_SYMBOL(sync_file_install);
-
-static void sync_file_add_pt(struct sync_file *sync_file, int *i,
- struct fence *fence)
-{
- sync_file->cbs[*i].fence = fence;
- sync_file->cbs[*i].sync_file = sync_file;
-
- if (!fence_add_callback(fence, &sync_file->cbs[*i].cb,
- fence_check_cb_func)) {
- fence_get(fence);
- (*i)++;
- }
-}
-
-struct sync_file *sync_file_merge(const char *name,
- struct sync_file *a, struct sync_file *b)
-{
- int num_fences = a->num_fences + b->num_fences;
- struct sync_file *sync_file;
- int i, i_a, i_b;
- unsigned long size = offsetof(struct sync_file, cbs[num_fences]);
-
- sync_file = sync_file_alloc(size, name);
- if (!sync_file)
- return NULL;
-
- atomic_set(&sync_file->status, num_fences);
-
- /*
- * Assume sync_file a and b are both ordered and have no
- * duplicates with the same context.
- *
- * If a sync_file can only be created with sync_file_merge
- * and sync_file_create, this is a reasonable assumption.
- */
- for (i = i_a = i_b = 0; i_a < a->num_fences && i_b < b->num_fences; ) {
- struct fence *pt_a = a->cbs[i_a].fence;
- struct fence *pt_b = b->cbs[i_b].fence;
-
- if (pt_a->context < pt_b->context) {
- sync_file_add_pt(sync_file, &i, pt_a);
-
- i_a++;
- } else if (pt_a->context > pt_b->context) {
- sync_file_add_pt(sync_file, &i, pt_b);
-
- i_b++;
- } else {
- if (pt_a->seqno - pt_b->seqno <= INT_MAX)
- sync_file_add_pt(sync_file, &i, pt_a);
- else
- sync_file_add_pt(sync_file, &i, pt_b);
-
- i_a++;
- i_b++;
- }
- }
-
- for (; i_a < a->num_fences; i_a++)
- sync_file_add_pt(sync_file, &i, a->cbs[i_a].fence);
-
- for (; i_b < b->num_fences; i_b++)
- sync_file_add_pt(sync_file, &i, b->cbs[i_b].fence);
-
- if (num_fences > i)
- atomic_sub(num_fences - i, &sync_file->status);
- sync_file->num_fences = i;
-
- sync_file_debug_add(sync_file);
- return sync_file;
-}
-EXPORT_SYMBOL(sync_file_merge);
-
static const char *android_fence_get_driver_name(struct fence *fence)
{
struct sync_timeline *parent = fence_parent(fence);
@@ -387,191 +219,3 @@
.fence_value_str = android_fence_value_str,
.timeline_value_str = android_fence_timeline_value_str,
};
-
-static void sync_file_free(struct kref *kref)
-{
- struct sync_file *sync_file = container_of(kref, struct sync_file,
- kref);
- int i;
-
- for (i = 0; i < sync_file->num_fences; ++i) {
- fence_remove_callback(sync_file->cbs[i].fence,
- &sync_file->cbs[i].cb);
- fence_put(sync_file->cbs[i].fence);
- }
-
- kfree(sync_file);
-}
-
-static int sync_file_release(struct inode *inode, struct file *file)
-{
- struct sync_file *sync_file = file->private_data;
-
- sync_file_debug_remove(sync_file);
-
- kref_put(&sync_file->kref, sync_file_free);
- return 0;
-}
-
-static unsigned int sync_file_poll(struct file *file, poll_table *wait)
-{
- struct sync_file *sync_file = file->private_data;
- int status;
-
- poll_wait(file, &sync_file->wq, wait);
-
- status = atomic_read(&sync_file->status);
-
- if (!status)
- return POLLIN;
- if (status < 0)
- return POLLERR;
- return 0;
-}
-
-static long sync_file_ioctl_merge(struct sync_file *sync_file,
- unsigned long arg)
-{
- int fd = get_unused_fd_flags(O_CLOEXEC);
- int err;
- struct sync_file *fence2, *fence3;
- struct sync_merge_data data;
-
- if (fd < 0)
- return fd;
-
- if (copy_from_user(&data, (void __user *)arg, sizeof(data))) {
- err = -EFAULT;
- goto err_put_fd;
- }
-
- fence2 = sync_file_fdget(data.fd2);
- if (!fence2) {
- err = -ENOENT;
- goto err_put_fd;
- }
-
- data.name[sizeof(data.name) - 1] = '\0';
- fence3 = sync_file_merge(data.name, sync_file, fence2);
- if (!fence3) {
- err = -ENOMEM;
- goto err_put_fence2;
- }
-
- data.fence = fd;
- if (copy_to_user((void __user *)arg, &data, sizeof(data))) {
- err = -EFAULT;
- goto err_put_fence3;
- }
-
- sync_file_install(fence3, fd);
- sync_file_put(fence2);
- return 0;
-
-err_put_fence3:
- sync_file_put(fence3);
-
-err_put_fence2:
- sync_file_put(fence2);
-
-err_put_fd:
- put_unused_fd(fd);
- return err;
-}
-
-static int sync_fill_fence_info(struct fence *fence, void *data, int size)
-{
- struct sync_fence_info *info = data;
-
- if (size < sizeof(*info))
- return -ENOMEM;
-
- strlcpy(info->obj_name, fence->ops->get_timeline_name(fence),
- sizeof(info->obj_name));
- strlcpy(info->driver_name, fence->ops->get_driver_name(fence),
- sizeof(info->driver_name));
- if (fence_is_signaled(fence))
- info->status = fence->status >= 0 ? 1 : fence->status;
- else
- info->status = 0;
- info->timestamp_ns = ktime_to_ns(fence->timestamp);
-
- return sizeof(*info);
-}
-
-static long sync_file_ioctl_fence_info(struct sync_file *sync_file,
- unsigned long arg)
-{
- struct sync_file_info *info;
- __u32 size;
- __u32 len = 0;
- int ret, i;
-
- if (copy_from_user(&size, (void __user *)arg, sizeof(size)))
- return -EFAULT;
-
- if (size < sizeof(struct sync_file_info))
- return -EINVAL;
-
- if (size > 4096)
- size = 4096;
-
- info = kzalloc(size, GFP_KERNEL);
- if (!info)
- return -ENOMEM;
-
- strlcpy(info->name, sync_file->name, sizeof(info->name));
- info->status = atomic_read(&sync_file->status);
- if (info->status >= 0)
- info->status = !info->status;
-
- len = sizeof(struct sync_file_info);
-
- for (i = 0; i < sync_file->num_fences; ++i) {
- struct fence *fence = sync_file->cbs[i].fence;
-
- ret = sync_fill_fence_info(fence, (u8 *)info + len, size - len);
-
- if (ret < 0)
- goto out;
-
- len += ret;
- }
-
- info->len = len;
-
- if (copy_to_user((void __user *)arg, info, len))
- ret = -EFAULT;
- else
- ret = 0;
-
-out:
- kfree(info);
-
- return ret;
-}
-
-static long sync_file_ioctl(struct file *file, unsigned int cmd,
- unsigned long arg)
-{
- struct sync_file *sync_file = file->private_data;
-
- switch (cmd) {
- case SYNC_IOC_MERGE:
- return sync_file_ioctl_merge(sync_file, arg);
-
- case SYNC_IOC_FENCE_INFO:
- return sync_file_ioctl_fence_info(sync_file, arg);
-
- default:
- return -ENOTTY;
- }
-}
-
-static const struct file_operations sync_file_fops = {
- .release = sync_file_release,
- .poll = sync_file_poll,
- .unlocked_ioctl = sync_file_ioctl,
- .compat_ioctl = sync_file_ioctl,
-};
-
diff --git a/drivers/staging/android/sync.h b/drivers/staging/android/sync.h
index d2a1734..b56885c 100644
--- a/drivers/staging/android/sync.h
+++ b/drivers/staging/android/sync.h
@@ -20,10 +20,10 @@
#include <linux/spinlock.h>
#include <linux/fence.h>
-#include "uapi/sync.h"
+#include <linux/sync_file.h>
+#include <uapi/linux/sync_file.h>
struct sync_timeline;
-struct sync_file;
/**
* struct sync_timeline_ops - sync object implementation ops
@@ -86,38 +86,6 @@
child_list_lock);
}
-struct sync_file_cb {
- struct fence_cb cb;
- struct fence *fence;
- struct sync_file *sync_file;
-};
-
-/**
- * struct sync_file - sync file to export to the userspace
- * @file: file representing this fence
- * @kref: reference count on fence.
- * @name: name of sync_file. Useful for debugging
- * @sync_file_list: membership in global file list
- * @num_fences number of sync_pts in the fence
- * @wq: wait queue for fence signaling
- * @status: 0: signaled, >0:active, <0: error
- * @cbs: sync_pts callback information
- */
-struct sync_file {
- struct file *file;
- struct kref kref;
- char name[32];
-#ifdef CONFIG_DEBUG_FS
- struct list_head sync_file_list;
-#endif
- int num_fences;
-
- wait_queue_head_t wq;
- atomic_t status;
-
- struct sync_file_cb cbs[];
-};
-
/*
* API for sync_timeline implementers
*/
@@ -167,61 +135,6 @@
*/
struct fence *sync_pt_create(struct sync_timeline *parent, int size);
-/**
- * sync_fence_create() - creates a sync fence
- * @name: name of fence to create
- * @fence: fence to add to the sync_fence
- *
- * Creates a sync_file containg @fence. Once this is called, the sync_file
- * takes ownership of @fence.
- */
-struct sync_file *sync_file_create(const char *name, struct fence *fence);
-
-/*
- * API for sync_file consumers
- */
-
-/**
- * sync_file_merge() - merge two sync_files
- * @name: name of new fence
- * @a: sync_file a
- * @b: sync_file b
- *
- * Creates a new sync_file which contains copies of all the fences in both
- * @a and @b. @a and @b remain valid, independent sync_file. Returns the
- * new merged sync_file or NULL in case of error.
- */
-struct sync_file *sync_file_merge(const char *name,
- struct sync_file *a, struct sync_file *b);
-
-/**
- * sync_file_fdget() - get a sync_file from an fd
- * @fd: fd referencing a fence
- *
- * Ensures @fd references a valid sync_file, increments the refcount of the
- * backing file. Returns the sync_file or NULL in case of error.
- */
-struct sync_file *sync_file_fdget(int fd);
-
-/**
- * sync_file_put() - puts a reference of a sync_file
- * @sync_file: sync_file to put
- *
- * Puts a reference on @sync_fence. If this is the last reference, the
- * sync_fil and all it's sync_pts will be freed
- */
-void sync_file_put(struct sync_file *sync_file);
-
-/**
- * sync_file_install() - installs a sync_file into a file descriptor
- * @sync_file: sync_file to install
- * @fd: file descriptor in which to install the fence
- *
- * Installs @sync_file into @fd. @fd's should be acquired through
- * get_unused_fd_flags(O_CLOEXEC).
- */
-void sync_file_install(struct sync_file *sync_file, int fd);
-
#ifdef CONFIG_DEBUG_FS
void sync_timeline_debug_add(struct sync_timeline *obj);
diff --git a/drivers/staging/android/sync_debug.c b/drivers/staging/android/sync_debug.c
index 5a7ec58..5f57499 100644
--- a/drivers/staging/android/sync_debug.c
+++ b/drivers/staging/android/sync_debug.c
@@ -26,6 +26,7 @@
#include <linux/uaccess.h>
#include <linux/anon_inodes.h>
#include <linux/time64.h>
+#include <linux/sync_file.h>
#include "sw_sync.h"
#ifdef CONFIG_DEBUG_FS
@@ -262,8 +263,7 @@
goto err;
}
- data.name[sizeof(data.name) - 1] = '\0';
- sync_file = sync_file_create(data.name, fence);
+ sync_file = sync_file_create(fence);
if (!sync_file) {
fence_put(fence);
err = -ENOMEM;
@@ -272,12 +272,12 @@
data.fence = fd;
if (copy_to_user((void __user *)arg, &data, sizeof(data))) {
- sync_file_put(sync_file);
+ fput(sync_file->file);
err = -EFAULT;
goto err;
}
- sync_file_install(sync_file, fd);
+ fd_install(fd, sync_file->file);
return 0;
diff --git a/drivers/staging/android/timed_gpio.c b/drivers/staging/android/timed_gpio.c
deleted file mode 100644
index 914fd10..0000000
--- a/drivers/staging/android/timed_gpio.c
+++ /dev/null
@@ -1,166 +0,0 @@
-/* drivers/misc/timed_gpio.c
- *
- * Copyright (C) 2008 Google, Inc.
- * Author: Mike Lockwood <lockwood@android.com>
- *
- * This software is licensed under the terms of the GNU General Public
- * License version 2, as published by the Free Software Foundation, and
- * may be copied, distributed, and modified under those terms.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- */
-
-#include <linux/module.h>
-#include <linux/platform_device.h>
-#include <linux/slab.h>
-#include <linux/hrtimer.h>
-#include <linux/err.h>
-#include <linux/gpio.h>
-#include <linux/ktime.h>
-
-#include "timed_output.h"
-#include "timed_gpio.h"
-
-struct timed_gpio_data {
- struct timed_output_dev dev;
- struct hrtimer timer;
- spinlock_t lock;
- unsigned gpio;
- int max_timeout;
- u8 active_low;
-};
-
-static enum hrtimer_restart gpio_timer_func(struct hrtimer *timer)
-{
- struct timed_gpio_data *data =
- container_of(timer, struct timed_gpio_data, timer);
-
- gpio_direction_output(data->gpio, data->active_low ? 1 : 0);
- return HRTIMER_NORESTART;
-}
-
-static int gpio_get_time(struct timed_output_dev *dev)
-{
- struct timed_gpio_data *data;
- ktime_t t;
-
- data = container_of(dev, struct timed_gpio_data, dev);
-
- if (!hrtimer_active(&data->timer))
- return 0;
-
- t = hrtimer_get_remaining(&data->timer);
-
- return ktime_to_ms(t);
-}
-
-static void gpio_enable(struct timed_output_dev *dev, int value)
-{
- struct timed_gpio_data *data =
- container_of(dev, struct timed_gpio_data, dev);
- unsigned long flags;
-
- spin_lock_irqsave(&data->lock, flags);
-
- /* cancel previous timer and set GPIO according to value */
- hrtimer_cancel(&data->timer);
- gpio_direction_output(data->gpio, data->active_low ? !value : !!value);
-
- if (value > 0) {
- if (value > data->max_timeout)
- value = data->max_timeout;
-
- hrtimer_start(&data->timer,
- ktime_set(value / 1000, (value % 1000) * 1000000),
- HRTIMER_MODE_REL);
- }
-
- spin_unlock_irqrestore(&data->lock, flags);
-}
-
-static int timed_gpio_probe(struct platform_device *pdev)
-{
- struct timed_gpio_platform_data *pdata = pdev->dev.platform_data;
- struct timed_gpio *cur_gpio;
- struct timed_gpio_data *gpio_data, *gpio_dat;
- int i, ret;
-
- if (!pdata)
- return -EBUSY;
-
- gpio_data = devm_kcalloc(&pdev->dev, pdata->num_gpios,
- sizeof(*gpio_data), GFP_KERNEL);
- if (!gpio_data)
- return -ENOMEM;
-
- for (i = 0; i < pdata->num_gpios; i++) {
- cur_gpio = &pdata->gpios[i];
- gpio_dat = &gpio_data[i];
-
- hrtimer_init(&gpio_dat->timer, CLOCK_MONOTONIC,
- HRTIMER_MODE_REL);
- gpio_dat->timer.function = gpio_timer_func;
- spin_lock_init(&gpio_dat->lock);
-
- gpio_dat->dev.name = cur_gpio->name;
- gpio_dat->dev.get_time = gpio_get_time;
- gpio_dat->dev.enable = gpio_enable;
- ret = gpio_request(cur_gpio->gpio, cur_gpio->name);
- if (ret < 0)
- goto err_out;
- ret = timed_output_dev_register(&gpio_dat->dev);
- if (ret < 0) {
- gpio_free(cur_gpio->gpio);
- goto err_out;
- }
-
- gpio_dat->gpio = cur_gpio->gpio;
- gpio_dat->max_timeout = cur_gpio->max_timeout;
- gpio_dat->active_low = cur_gpio->active_low;
- gpio_direction_output(gpio_dat->gpio, gpio_dat->active_low);
- }
-
- platform_set_drvdata(pdev, gpio_data);
-
- return 0;
-
-err_out:
- while (--i >= 0) {
- timed_output_dev_unregister(&gpio_data[i].dev);
- gpio_free(gpio_data[i].gpio);
- }
-
- return ret;
-}
-
-static int timed_gpio_remove(struct platform_device *pdev)
-{
- struct timed_gpio_platform_data *pdata = pdev->dev.platform_data;
- struct timed_gpio_data *gpio_data = platform_get_drvdata(pdev);
- int i;
-
- for (i = 0; i < pdata->num_gpios; i++) {
- timed_output_dev_unregister(&gpio_data[i].dev);
- gpio_free(gpio_data[i].gpio);
- }
-
- return 0;
-}
-
-static struct platform_driver timed_gpio_driver = {
- .probe = timed_gpio_probe,
- .remove = timed_gpio_remove,
- .driver = {
- .name = TIMED_GPIO_NAME,
- },
-};
-
-module_platform_driver(timed_gpio_driver);
-
-MODULE_AUTHOR("Mike Lockwood <lockwood@android.com>");
-MODULE_DESCRIPTION("timed gpio driver");
-MODULE_LICENSE("GPL");
diff --git a/drivers/staging/android/timed_gpio.h b/drivers/staging/android/timed_gpio.h
deleted file mode 100644
index d29e169..0000000
--- a/drivers/staging/android/timed_gpio.h
+++ /dev/null
@@ -1,33 +0,0 @@
-/* include/linux/timed_gpio.h
- *
- * Copyright (C) 2008 Google, Inc.
- *
- * This software is licensed under the terms of the GNU General Public
- * License version 2, as published by the Free Software Foundation, and
- * may be copied, distributed, and modified under those terms.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
-*/
-
-#ifndef _LINUX_TIMED_GPIO_H
-#define _LINUX_TIMED_GPIO_H
-
-#define TIMED_GPIO_NAME "timed-gpio"
-
-struct timed_gpio {
- const char *name;
- unsigned gpio;
- int max_timeout;
- u8 active_low;
-};
-
-struct timed_gpio_platform_data {
- int num_gpios;
- struct timed_gpio *gpios;
-};
-
-#endif
diff --git a/drivers/staging/android/timed_output.c b/drivers/staging/android/timed_output.c
deleted file mode 100644
index aff9cdb..0000000
--- a/drivers/staging/android/timed_output.c
+++ /dev/null
@@ -1,110 +0,0 @@
-/* drivers/misc/timed_output.c
- *
- * Copyright (C) 2009 Google, Inc.
- * Author: Mike Lockwood <lockwood@android.com>
- *
- * This software is licensed under the terms of the GNU General Public
- * License version 2, as published by the Free Software Foundation, and
- * may be copied, distributed, and modified under those terms.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- */
-
-#define pr_fmt(fmt) "timed_output: " fmt
-
-#include <linux/init.h>
-#include <linux/export.h>
-#include <linux/types.h>
-#include <linux/device.h>
-#include <linux/fs.h>
-#include <linux/err.h>
-
-#include "timed_output.h"
-
-static struct class *timed_output_class;
-static atomic_t device_count;
-
-static ssize_t enable_show(struct device *dev, struct device_attribute *attr,
- char *buf)
-{
- struct timed_output_dev *tdev = dev_get_drvdata(dev);
- int remaining = tdev->get_time(tdev);
-
- return sprintf(buf, "%d\n", remaining);
-}
-
-static ssize_t enable_store(struct device *dev, struct device_attribute *attr,
- const char *buf, size_t size)
-{
- struct timed_output_dev *tdev = dev_get_drvdata(dev);
- int value;
- int rc;
-
- rc = kstrtoint(buf, 0, &value);
- if (rc != 0)
- return -EINVAL;
-
- tdev->enable(tdev, value);
-
- return size;
-}
-static DEVICE_ATTR_RW(enable);
-
-static struct attribute *timed_output_attrs[] = {
- &dev_attr_enable.attr,
- NULL,
-};
-ATTRIBUTE_GROUPS(timed_output);
-
-static int create_timed_output_class(void)
-{
- if (!timed_output_class) {
- timed_output_class = class_create(THIS_MODULE, "timed_output");
- if (IS_ERR(timed_output_class))
- return PTR_ERR(timed_output_class);
- atomic_set(&device_count, 0);
- timed_output_class->dev_groups = timed_output_groups;
- }
-
- return 0;
-}
-
-int timed_output_dev_register(struct timed_output_dev *tdev)
-{
- int ret;
-
- if (!tdev || !tdev->name || !tdev->enable || !tdev->get_time)
- return -EINVAL;
-
- ret = create_timed_output_class();
- if (ret < 0)
- return ret;
-
- tdev->index = atomic_inc_return(&device_count);
- tdev->dev = device_create(timed_output_class, NULL,
- MKDEV(0, tdev->index), NULL, "%s", tdev->name);
- if (IS_ERR(tdev->dev))
- return PTR_ERR(tdev->dev);
-
- dev_set_drvdata(tdev->dev, tdev);
- tdev->state = 0;
- return 0;
-}
-EXPORT_SYMBOL_GPL(timed_output_dev_register);
-
-void timed_output_dev_unregister(struct timed_output_dev *tdev)
-{
- tdev->enable(tdev, 0);
- device_destroy(timed_output_class, MKDEV(0, tdev->index));
-}
-EXPORT_SYMBOL_GPL(timed_output_dev_unregister);
-
-static int __init timed_output_init(void)
-{
- return create_timed_output_class();
-}
-device_initcall(timed_output_init);
diff --git a/drivers/staging/android/timed_output.h b/drivers/staging/android/timed_output.h
deleted file mode 100644
index 13d2ca5..0000000
--- a/drivers/staging/android/timed_output.h
+++ /dev/null
@@ -1,37 +0,0 @@
-/* include/linux/timed_output.h
- *
- * Copyright (C) 2008 Google, Inc.
- *
- * This software is licensed under the terms of the GNU General Public
- * License version 2, as published by the Free Software Foundation, and
- * may be copied, distributed, and modified under those terms.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
-*/
-
-#ifndef _LINUX_TIMED_OUTPUT_H
-#define _LINUX_TIMED_OUTPUT_H
-
-struct timed_output_dev {
- const char *name;
-
- /* enable the output and set the timer */
- void (*enable)(struct timed_output_dev *sdev, int timeout);
-
- /* returns the current number of milliseconds remaining on the timer */
- int (*get_time)(struct timed_output_dev *sdev);
-
- /* private data */
- struct device *dev;
- int index;
- int state;
-};
-
-int timed_output_dev_register(struct timed_output_dev *dev);
-void timed_output_dev_unregister(struct timed_output_dev *dev);
-
-#endif
diff --git a/drivers/staging/android/uapi/sync.h b/drivers/staging/android/uapi/sync.h
deleted file mode 100644
index a0cf357..0000000
--- a/drivers/staging/android/uapi/sync.h
+++ /dev/null
@@ -1,84 +0,0 @@
-/*
- * Copyright (C) 2012 Google, Inc.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- */
-
-#ifndef _UAPI_LINUX_SYNC_H
-#define _UAPI_LINUX_SYNC_H
-
-#include <linux/ioctl.h>
-#include <linux/types.h>
-
-/**
- * struct sync_merge_data - data passed to merge ioctl
- * @fd2: file descriptor of second fence
- * @name: name of new fence
- * @fence: returns the fd of the new fence to userspace
- */
-struct sync_merge_data {
- __s32 fd2; /* fd of second fence */
- char name[32]; /* name of new fence */
- __s32 fence; /* fd on newly created fence */
-};
-
-/**
- * struct sync_fence_info - detailed fence information
- * @obj_name: name of parent sync_timeline
- * @driver_name: name of driver implementing the parent
- * @status: status of the fence 0:active 1:signaled <0:error
- * @timestamp_ns: timestamp of status change in nanoseconds
- */
-struct sync_fence_info {
- char obj_name[32];
- char driver_name[32];
- __s32 status;
- __u64 timestamp_ns;
-};
-
-/**
- * struct sync_file_info - data returned from fence info ioctl
- * @len: ioctl caller writes the size of the buffer its passing in.
- * ioctl returns length of sync_file_info returned to
- * userspace including pt_info.
- * @name: name of fence
- * @status: status of fence. 1: signaled 0:active <0:error
- * @sync_fence_info: array of sync_fence_info for every fence in the sync_file
- */
-struct sync_file_info {
- __u32 len;
- char name[32];
- __s32 status;
-
- __u8 sync_fence_info[0];
-};
-
-#define SYNC_IOC_MAGIC '>'
-
-/**
- * DOC: SYNC_IOC_MERGE - merge two fences
- *
- * Takes a struct sync_merge_data. Creates a new fence containing copies of
- * the sync_pts in both the calling fd and sync_merge_data.fd2. Returns the
- * new fence's fd in sync_merge_data.fence
- */
-#define SYNC_IOC_MERGE _IOWR(SYNC_IOC_MAGIC, 1, struct sync_merge_data)
-
-/**
- * DOC: SYNC_IOC_FENCE_INFO - get detailed information on a fence
- *
- * Takes a struct sync_file_info_data with extra space allocated for pt_info.
- * Caller should write the size of the buffer into len. On return, len is
- * updated to reflect the total size of the sync_file_info_data including
- * pt_info.
- *
- * pt_info is a buffer containing sync_pt_infos for every sync_pt in the fence.
- * To iterate over the sync_pt_infos, use the sync_pt_info.len field.
- */
-#define SYNC_IOC_FENCE_INFO _IOWR(SYNC_IOC_MAGIC, 2, struct sync_file_info)
-
-#endif /* _UAPI_LINUX_SYNC_H */
diff --git a/drivers/staging/board/armadillo800eva.c b/drivers/staging/board/armadillo800eva.c
index bb63ece..4de4fd0 100644
--- a/drivers/staging/board/armadillo800eva.c
+++ b/drivers/staging/board/armadillo800eva.c
@@ -87,10 +87,10 @@
static const struct board_staging_dev armadillo800eva_devices[] __initconst = {
{
- .pdev = &lcdc0_device,
- .clocks = lcdc0_clocks,
- .nclocks = ARRAY_SIZE(lcdc0_clocks),
- .domain = "/system-controller@e6180000/pm-domains/c5/a4lc@1"
+ .pdev = &lcdc0_device,
+ .clocks = lcdc0_clocks,
+ .nclocks = ARRAY_SIZE(lcdc0_clocks),
+ .domain = "/system-controller@e6180000/pm-domains/c5/a4lc@1"
},
};
diff --git a/drivers/staging/comedi/comedi_buf.c b/drivers/staging/comedi/comedi_buf.c
index 90c2801..c7d7682 100644
--- a/drivers/staging/comedi/comedi_buf.c
+++ b/drivers/staging/comedi/comedi_buf.c
@@ -80,14 +80,14 @@
static void __comedi_buf_alloc(struct comedi_device *dev,
struct comedi_subdevice *s,
- unsigned n_pages)
+ unsigned int n_pages)
{
struct comedi_async *async = s->async;
struct page **pages = NULL;
struct comedi_buf_map *bm;
struct comedi_buf_page *buf;
unsigned long flags;
- unsigned i;
+ unsigned int i;
if (!IS_ENABLED(CONFIG_HAS_DMA) && s->async_dma_dir != DMA_NONE) {
dev_err(dev->class_dev,
@@ -208,7 +208,7 @@
/* allocate new buffer */
if (new_size) {
- unsigned n_pages = new_size >> PAGE_SHIFT;
+ unsigned int n_pages = new_size >> PAGE_SHIFT;
__comedi_buf_alloc(dev, s, n_pages);
@@ -302,7 +302,7 @@
{
struct comedi_async *async = s->async;
unsigned int count = 0;
- const unsigned num_sample_bytes = comedi_bytes_per_sample(s);
+ const unsigned int num_sample_bytes = comedi_bytes_per_sample(s);
if (!s->munge || (async->cmd.flags & CMDF_RAWDATA)) {
async->munge_count += num_bytes;
@@ -395,7 +395,7 @@
unsigned int comedi_buf_read_n_available(struct comedi_subdevice *s)
{
struct comedi_async *async = s->async;
- unsigned num_bytes;
+ unsigned int num_bytes;
if (!async)
return 0;
diff --git a/drivers/staging/comedi/comedi_fops.c b/drivers/staging/comedi/comedi_fops.c
index 7c7b477..629080f 100644
--- a/drivers/staging/comedi/comedi_fops.c
+++ b/drivers/staging/comedi/comedi_fops.c
@@ -186,7 +186,7 @@
return cleared;
}
-static struct comedi_device *comedi_clear_board_minor(unsigned minor)
+static struct comedi_device *comedi_clear_board_minor(unsigned int minor)
{
struct comedi_device *dev;
@@ -209,8 +209,8 @@
}
}
-static struct comedi_subdevice
-*comedi_subdevice_from_minor(const struct comedi_device *dev, unsigned minor)
+static struct comedi_subdevice *
+comedi_subdevice_from_minor(const struct comedi_device *dev, unsigned int minor)
{
struct comedi_subdevice *s;
unsigned int i = minor - COMEDI_NUM_BOARD_MINORS;
@@ -223,7 +223,7 @@
return s;
}
-static struct comedi_device *comedi_dev_get_from_board_minor(unsigned minor)
+static struct comedi_device *comedi_dev_get_from_board_minor(unsigned int minor)
{
struct comedi_device *dev;
@@ -233,7 +233,8 @@
return dev;
}
-static struct comedi_device *comedi_dev_get_from_subdevice_minor(unsigned minor)
+static struct comedi_device *
+comedi_dev_get_from_subdevice_minor(unsigned int minor)
{
struct comedi_device *dev;
struct comedi_subdevice *s;
@@ -258,7 +259,7 @@
* reference incremented. Return NULL if no COMEDI device exists with the
* specified minor device number.
*/
-struct comedi_device *comedi_dev_get_from_minor(unsigned minor)
+struct comedi_device *comedi_dev_get_from_minor(unsigned int minor)
{
if (minor < COMEDI_NUM_BOARD_MINORS)
return comedi_dev_get_from_board_minor(minor);
@@ -342,7 +343,8 @@
}
static int resize_async_buffer(struct comedi_device *dev,
- struct comedi_subdevice *s, unsigned new_size)
+ struct comedi_subdevice *s,
+ unsigned int new_size)
{
struct comedi_async *async = s->async;
int retval;
@@ -616,19 +618,20 @@
ATTRIBUTE_GROUPS(comedi_dev);
static void __comedi_clear_subdevice_runflags(struct comedi_subdevice *s,
- unsigned bits)
+ unsigned int bits)
{
s->runflags &= ~bits;
}
static void __comedi_set_subdevice_runflags(struct comedi_subdevice *s,
- unsigned bits)
+ unsigned int bits)
{
s->runflags |= bits;
}
static void comedi_update_subdevice_runflags(struct comedi_subdevice *s,
- unsigned mask, unsigned bits)
+ unsigned int mask,
+ unsigned int bits)
{
unsigned long flags;
@@ -638,15 +641,15 @@
spin_unlock_irqrestore(&s->spin_lock, flags);
}
-static unsigned __comedi_get_subdevice_runflags(struct comedi_subdevice *s)
+static unsigned int __comedi_get_subdevice_runflags(struct comedi_subdevice *s)
{
return s->runflags;
}
-static unsigned comedi_get_subdevice_runflags(struct comedi_subdevice *s)
+static unsigned int comedi_get_subdevice_runflags(struct comedi_subdevice *s)
{
unsigned long flags;
- unsigned runflags;
+ unsigned int runflags;
spin_lock_irqsave(&s->spin_lock, flags);
runflags = __comedi_get_subdevice_runflags(s);
@@ -654,12 +657,12 @@
return runflags;
}
-static bool comedi_is_runflags_running(unsigned runflags)
+static bool comedi_is_runflags_running(unsigned int runflags)
{
return runflags & COMEDI_SRF_RUNNING;
}
-static bool comedi_is_runflags_in_error(unsigned runflags)
+static bool comedi_is_runflags_in_error(unsigned int runflags)
{
return runflags & COMEDI_SRF_ERROR;
}
@@ -673,7 +676,7 @@
*/
bool comedi_is_subdevice_running(struct comedi_subdevice *s)
{
- unsigned runflags = comedi_get_subdevice_runflags(s);
+ unsigned int runflags = comedi_get_subdevice_runflags(s);
return comedi_is_runflags_running(runflags);
}
@@ -681,14 +684,14 @@
static bool __comedi_is_subdevice_running(struct comedi_subdevice *s)
{
- unsigned runflags = __comedi_get_subdevice_runflags(s);
+ unsigned int runflags = __comedi_get_subdevice_runflags(s);
return comedi_is_runflags_running(runflags);
}
bool comedi_can_auto_free_spriv(struct comedi_subdevice *s)
{
- unsigned runflags = __comedi_get_subdevice_runflags(s);
+ unsigned int runflags = __comedi_get_subdevice_runflags(s);
return runflags & COMEDI_SRF_FREE_SPRIV;
}
@@ -2038,7 +2041,7 @@
static long comedi_unlocked_ioctl(struct file *file, unsigned int cmd,
unsigned long arg)
{
- unsigned minor = iminor(file_inode(file));
+ unsigned int minor = iminor(file_inode(file));
struct comedi_file *cfp = file->private_data;
struct comedi_device *dev = cfp->dev;
int rc;
@@ -2342,7 +2345,7 @@
add_wait_queue(&async->wait_head, &wait);
while (count == 0 && !retval) {
- unsigned runflags;
+ unsigned int runflags;
unsigned int wp, n1, n2;
set_current_state(TASK_INTERRUPTIBLE);
@@ -2485,7 +2488,8 @@
n = min_t(size_t, m, nbytes);
if (n == 0) {
- unsigned runflags = comedi_get_subdevice_runflags(s);
+ unsigned int runflags =
+ comedi_get_subdevice_runflags(s);
if (!comedi_is_runflags_running(runflags)) {
if (comedi_is_runflags_in_error(runflags))
@@ -2573,7 +2577,7 @@
static int comedi_open(struct inode *inode, struct file *file)
{
- const unsigned minor = iminor(inode);
+ const unsigned int minor = iminor(inode);
struct comedi_file *cfp;
struct comedi_device *dev = comedi_dev_get_from_minor(minor);
int rc;
@@ -2733,7 +2737,7 @@
{
struct comedi_device *dev;
struct device *csdev;
- unsigned i;
+ unsigned int i;
dev = kzalloc(sizeof(*dev), GFP_KERNEL);
if (!dev)
@@ -2791,7 +2795,7 @@
{
struct comedi_device *dev = s->device;
struct device *csdev;
- unsigned i;
+ unsigned int i;
mutex_lock(&comedi_subdevice_minor_table_lock);
for (i = 0; i < COMEDI_NUM_SUBDEVICE_MINORS; ++i) {
@@ -2841,7 +2845,7 @@
static void comedi_cleanup_board_minors(void)
{
struct comedi_device *dev;
- unsigned i;
+ unsigned int i;
for (i = 0; i < COMEDI_NUM_BOARD_MINORS; i++) {
dev = comedi_clear_board_minor(i);
diff --git a/drivers/staging/comedi/comedidev.h b/drivers/staging/comedi/comedidev.h
index 1158072..dcb6376 100644
--- a/drivers/staging/comedi/comedidev.h
+++ b/drivers/staging/comedi/comedidev.h
@@ -173,7 +173,7 @@
void *lock;
void *busy;
- unsigned runflags;
+ unsigned int runflags;
spinlock_t spin_lock; /* generic spin-lock for COMEDI and drivers */
unsigned int io_bits;
@@ -566,7 +566,7 @@
void comedi_event(struct comedi_device *dev, struct comedi_subdevice *s);
-struct comedi_device *comedi_dev_get_from_minor(unsigned minor);
+struct comedi_device *comedi_dev_get_from_minor(unsigned int minor);
int comedi_dev_put(struct comedi_device *dev);
bool comedi_is_subdevice_running(struct comedi_subdevice *s);
diff --git a/drivers/staging/comedi/drivers.c b/drivers/staging/comedi/drivers.c
index b63dd2e..44511d7 100644
--- a/drivers/staging/comedi/drivers.c
+++ b/drivers/staging/comedi/drivers.c
@@ -564,7 +564,7 @@
if (events == 0)
return events;
- if (events & COMEDI_CB_CANCEL_MASK)
+ if ((events & COMEDI_CB_CANCEL_MASK) && s->cancel)
s->cancel(dev, s);
comedi_event(dev, s);
@@ -575,38 +575,35 @@
static int insn_rw_emulate_bits(struct comedi_device *dev,
struct comedi_subdevice *s,
- struct comedi_insn *insn, unsigned int *data)
+ struct comedi_insn *insn,
+ unsigned int *data)
{
- struct comedi_insn new_insn;
+ struct comedi_insn _insn;
+ unsigned int chan = CR_CHAN(insn->chanspec);
+ unsigned int base_chan = (chan < 32) ? 0 : chan;
+ unsigned int _data[2];
int ret;
- static const unsigned channels_per_bitfield = 32;
- unsigned chan = CR_CHAN(insn->chanspec);
- const unsigned base_bitfield_channel =
- (chan < channels_per_bitfield) ? 0 : chan;
- unsigned int new_data[2];
-
- memset(new_data, 0, sizeof(new_data));
- memset(&new_insn, 0, sizeof(new_insn));
- new_insn.insn = INSN_BITS;
- new_insn.chanspec = base_bitfield_channel;
- new_insn.n = 2;
- new_insn.subdev = insn->subdev;
+ memset(_data, 0, sizeof(_data));
+ memset(&_insn, 0, sizeof(_insn));
+ _insn.insn = INSN_BITS;
+ _insn.chanspec = base_chan;
+ _insn.n = 2;
+ _insn.subdev = insn->subdev;
if (insn->insn == INSN_WRITE) {
if (!(s->subdev_flags & SDF_WRITABLE))
return -EINVAL;
- new_data[0] = 1 << (chan - base_bitfield_channel); /* mask */
- new_data[1] = data[0] ? (1 << (chan - base_bitfield_channel))
- : 0; /* bits */
+ _data[0] = 1 << (chan - base_chan); /* mask */
+ _data[1] = data[0] ? (1 << (chan - base_chan)) : 0; /* bits */
}
- ret = s->insn_bits(dev, s, &new_insn, new_data);
+ ret = s->insn_bits(dev, s, &_insn, _data);
if (ret < 0)
return ret;
if (insn->insn == INSN_READ)
- data[0] = (new_data[1] >> (chan - base_bitfield_channel)) & 1;
+ data[0] = (_data[1] >> (chan - base_chan)) & 1;
return 1;
}
@@ -628,6 +625,9 @@
"async subdevices must have a do_cmdtest() function\n");
return -EINVAL;
}
+ if (!s->cancel)
+ dev_warn(dev->class_dev,
+ "async subdevices should have a cancel() function\n");
async = kzalloc(sizeof(*async), GFP_KERNEL);
if (!async)
diff --git a/drivers/staging/comedi/drivers/amcc_s5933.h b/drivers/staging/comedi/drivers/amcc_s5933.h
index d4b8c01..f03e4c8 100644
--- a/drivers/staging/comedi/drivers/amcc_s5933.h
+++ b/drivers/staging/comedi/drivers/amcc_s5933.h
@@ -1,16 +1,14 @@
/*
- comedi/drivers/amcc_s5933.h
-
- Stuff for AMCC S5933 PCI Controller
-
- Author: Michal Dobes <dobes@tesnet.cz>
-
- Inspirated from general-purpose AMCC S5933 PCI Matchmaker driver
- made by Andrea Cisternino <acister@pcape1.pi.infn.it>
- and as result of espionage from MITE code made by David A. Schleef.
- Thanks to AMCC for their on-line documentation and bus master DMA
- example.
-*/
+ * Stuff for AMCC S5933 PCI Controller
+ *
+ * Author: Michal Dobes <dobes@tesnet.cz>
+ *
+ * Inspirated from general-purpose AMCC S5933 PCI Matchmaker driver
+ * made by Andrea Cisternino <acister@pcape1.pi.infn.it>
+ * and as result of espionage from MITE code made by David A. Schleef.
+ * Thanks to AMCC for their on-line documentation and bus master DMA
+ * example.
+ */
#ifndef _AMCC_S5933_H_
#define _AMCC_S5933_H_
@@ -58,7 +56,7 @@
#define INTCSR_INTR_ASSERTED 0x800000
/****************************************************************************/
-/* AMCC - PCI non-volatile ram command register (byte 3 of master control/status register) */
+/* AMCC - PCI non-volatile ram command register (byte 3 of AMCC_OP_REG_MCSR) */
/****************************************************************************/
#define MCSR_NV_LOAD_LOW_ADDR 0x0
#define MCSR_NV_LOAD_HIGH_ADDR 0x20
diff --git a/drivers/staging/comedi/drivers/amplc_dio200_common.c b/drivers/staging/comedi/drivers/amplc_dio200_common.c
index d1539e7..f6e4e98 100644
--- a/drivers/staging/comedi/drivers/amplc_dio200_common.c
+++ b/drivers/staging/comedi/drivers/amplc_dio200_common.c
@@ -101,7 +101,7 @@
};
struct dio200_subdev_intr {
- spinlock_t spinlock;
+ spinlock_t spinlock; /* protects the 'active' flag */
unsigned int ofs;
unsigned int valid_isns;
unsigned int enabled_isns;
@@ -221,7 +221,7 @@
struct dio200_subdev_intr *subpriv = s->private;
struct comedi_cmd *cmd = &s->async->cmd;
unsigned int n;
- unsigned isn_bits;
+ unsigned int isn_bits;
/* Determine interrupt sources to enable. */
isn_bits = 0;
@@ -284,9 +284,9 @@
{
const struct dio200_board *board = dev->board_ptr;
struct dio200_subdev_intr *subpriv = s->private;
- unsigned triggered;
- unsigned intstat;
- unsigned cur_enabled;
+ unsigned int triggered;
+ unsigned int intstat;
+ unsigned int cur_enabled;
unsigned long flags;
triggered = 0;
@@ -439,7 +439,7 @@
static int dio200_subdev_intr_init(struct comedi_device *dev,
struct comedi_subdevice *s,
unsigned int offset,
- unsigned valid_isns)
+ unsigned int valid_isns)
{
const struct dio200_board *board = dev->board_ptr;
struct dio200_subdev_intr *subpriv;
diff --git a/drivers/staging/comedi/drivers/amplc_pc263.c b/drivers/staging/comedi/drivers/amplc_pc263.c
index b1946ce..58b0b6b 100644
--- a/drivers/staging/comedi/drivers/amplc_pc263.c
+++ b/drivers/staging/comedi/drivers/amplc_pc263.c
@@ -1,46 +1,44 @@
/*
- comedi/drivers/amplc_pc263.c
- Driver for Amplicon PC263 and PCI263 relay boards.
+ * Driver for Amplicon PC263 relay board.
+ *
+ * Copyright (C) 2002 MEV Ltd. <http://www.mev.co.uk/>
+ *
+ * COMEDI - Linux Control and Measurement Device Interface
+ * Copyright (C) 2000 David A. Schleef <ds@schleef.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
- Copyright (C) 2002 MEV Ltd. <http://www.mev.co.uk/>
-
- COMEDI - Linux Control and Measurement Device Interface
- Copyright (C) 2000 David A. Schleef <ds@schleef.org>
-
- This program is free software; you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation; either version 2 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-*/
/*
-Driver: amplc_pc263
-Description: Amplicon PC263
-Author: Ian Abbott <abbotti@mev.co.uk>
-Devices: [Amplicon] PC263 (pc263)
-Updated: Fri, 12 Apr 2013 15:19:36 +0100
-Status: works
-
-Configuration options:
- [0] - I/O port base address
-
-The board appears as one subdevice, with 16 digital outputs, each
-connected to a reed-relay. Relay contacts are closed when output is 1.
-The state of the outputs can be read.
-*/
+ * Driver: amplc_pc263
+ * Description: Amplicon PC263
+ * Author: Ian Abbott <abbotti@mev.co.uk>
+ * Devices: [Amplicon] PC263 (pc263)
+ * Updated: Fri, 12 Apr 2013 15:19:36 +0100
+ * Status: works
+ *
+ * Configuration options:
+ * [0] - I/O port base address
+ *
+ * The board appears as one subdevice, with 16 digital outputs, each
+ * connected to a reed-relay. Relay contacts are closed when output is 1.
+ * The state of the outputs can be read.
+ */
#include <linux/module.h>
#include "../comedidev.h"
/* PC263 registers */
-
-/*
- * Board descriptions for Amplicon PC263.
- */
+#define PC263_DO_0_7_REG 0x00
+#define PC263_DO_8_15_REG 0x01
struct pc263_board {
const char *name;
@@ -58,8 +56,8 @@
unsigned int *data)
{
if (comedi_dio_update_state(s, data)) {
- outb(s->state & 0xff, dev->iobase);
- outb((s->state >> 8) & 0xff, dev->iobase + 1);
+ outb(s->state & 0xff, dev->iobase + PC263_DO_0_7_REG);
+ outb((s->state >> 8) & 0xff, dev->iobase + PC263_DO_8_15_REG);
}
data[1] = s->state;
@@ -80,28 +78,30 @@
if (ret)
return ret;
+ /* Digital Output subdevice */
s = &dev->subdevices[0];
- /* digital output subdevice */
- s->type = COMEDI_SUBD_DO;
- s->subdev_flags = SDF_WRITABLE;
- s->n_chan = 16;
- s->maxdata = 1;
- s->range_table = &range_digital;
- s->insn_bits = pc263_do_insn_bits;
+ s->type = COMEDI_SUBD_DO;
+ s->subdev_flags = SDF_WRITABLE;
+ s->n_chan = 16;
+ s->maxdata = 1;
+ s->range_table = &range_digital;
+ s->insn_bits = pc263_do_insn_bits;
+
/* read initial relay state */
- s->state = inb(dev->iobase) | (inb(dev->iobase + 1) << 8);
+ s->state = inb(dev->iobase + PC263_DO_0_7_REG) |
+ (inb(dev->iobase + PC263_DO_8_15_REG) << 8);
return 0;
}
static struct comedi_driver amplc_pc263_driver = {
- .driver_name = "amplc_pc263",
- .module = THIS_MODULE,
- .attach = pc263_attach,
- .detach = comedi_legacy_detach,
- .board_name = &pc263_boards[0].name,
- .offset = sizeof(struct pc263_board),
- .num_names = ARRAY_SIZE(pc263_boards),
+ .driver_name = "amplc_pc263",
+ .module = THIS_MODULE,
+ .attach = pc263_attach,
+ .detach = comedi_legacy_detach,
+ .board_name = &pc263_boards[0].name,
+ .offset = sizeof(struct pc263_board),
+ .num_names = ARRAY_SIZE(pc263_boards),
};
module_comedi_driver(amplc_pc263_driver);
diff --git a/drivers/staging/comedi/drivers/amplc_pci224.c b/drivers/staging/comedi/drivers/amplc_pci224.c
index cac011f..2e6decf 100644
--- a/drivers/staging/comedi/drivers/amplc_pci224.c
+++ b/drivers/staging/comedi/drivers/amplc_pci224.c
@@ -132,48 +132,53 @@
* DACCON values.
*/
/* (r/w) Scan trigger. */
-#define PCI224_DACCON_TRIG_MASK (7 << 0)
-#define PCI224_DACCON_TRIG_NONE (0 << 0) /* none */
-#define PCI224_DACCON_TRIG_SW (1 << 0) /* software trig */
-#define PCI224_DACCON_TRIG_EXTP (2 << 0) /* ext +ve edge */
-#define PCI224_DACCON_TRIG_EXTN (3 << 0) /* ext -ve edge */
-#define PCI224_DACCON_TRIG_Z2CT0 (4 << 0) /* Z2 CT0 out */
-#define PCI224_DACCON_TRIG_Z2CT1 (5 << 0) /* Z2 CT1 out */
-#define PCI224_DACCON_TRIG_Z2CT2 (6 << 0) /* Z2 CT2 out */
+#define PCI224_DACCON_TRIG(x) (((x) & 0x7) << 0)
+#define PCI224_DACCON_TRIG_MASK PCI224_DACCON_TRIG(7)
+#define PCI224_DACCON_TRIG_NONE PCI224_DACCON_TRIG(0) /* none */
+#define PCI224_DACCON_TRIG_SW PCI224_DACCON_TRIG(1) /* soft trig */
+#define PCI224_DACCON_TRIG_EXTP PCI224_DACCON_TRIG(2) /* ext + edge */
+#define PCI224_DACCON_TRIG_EXTN PCI224_DACCON_TRIG(3) /* ext - edge */
+#define PCI224_DACCON_TRIG_Z2CT0 PCI224_DACCON_TRIG(4) /* Z2 CT0 out */
+#define PCI224_DACCON_TRIG_Z2CT1 PCI224_DACCON_TRIG(5) /* Z2 CT1 out */
+#define PCI224_DACCON_TRIG_Z2CT2 PCI224_DACCON_TRIG(6) /* Z2 CT2 out */
/* (r/w) Polarity (PCI224 only, PCI234 always bipolar!). */
-#define PCI224_DACCON_POLAR_MASK (1 << 3)
-#define PCI224_DACCON_POLAR_UNI (0 << 3) /* range [0,Vref] */
-#define PCI224_DACCON_POLAR_BI (1 << 3) /* range [-Vref,Vref] */
+#define PCI224_DACCON_POLAR(x) (((x) & 0x1) << 3)
+#define PCI224_DACCON_POLAR_MASK PCI224_DACCON_POLAR(1)
+#define PCI224_DACCON_POLAR_UNI PCI224_DACCON_POLAR(0) /* [0,+V] */
+#define PCI224_DACCON_POLAR_BI PCI224_DACCON_POLAR(1) /* [-V,+V] */
/* (r/w) Internal Vref (PCI224 only, when LK1 in position 1-2). */
-#define PCI224_DACCON_VREF_MASK (3 << 4)
-#define PCI224_DACCON_VREF_1_25 (0 << 4) /* Vref = 1.25V */
-#define PCI224_DACCON_VREF_2_5 (1 << 4) /* Vref = 2.5V */
-#define PCI224_DACCON_VREF_5 (2 << 4) /* Vref = 5V */
-#define PCI224_DACCON_VREF_10 (3 << 4) /* Vref = 10V */
+#define PCI224_DACCON_VREF(x) (((x) & 0x3) << 4)
+#define PCI224_DACCON_VREF_MASK PCI224_DACCON_VREF(3)
+#define PCI224_DACCON_VREF_1_25 PCI224_DACCON_VREF(0) /* 1.25V */
+#define PCI224_DACCON_VREF_2_5 PCI224_DACCON_VREF(1) /* 2.5V */
+#define PCI224_DACCON_VREF_5 PCI224_DACCON_VREF(2) /* 5V */
+#define PCI224_DACCON_VREF_10 PCI224_DACCON_VREF(3) /* 10V */
/* (r/w) Wraparound mode enable (to play back stored waveform). */
-#define PCI224_DACCON_FIFOWRAP (1 << 7)
+#define PCI224_DACCON_FIFOWRAP BIT(7)
/* (r/w) FIFO enable. It MUST be set! */
-#define PCI224_DACCON_FIFOENAB (1 << 8)
+#define PCI224_DACCON_FIFOENAB BIT(8)
/* (r/w) FIFO interrupt trigger level (most values are not very useful). */
-#define PCI224_DACCON_FIFOINTR_MASK (7 << 9)
-#define PCI224_DACCON_FIFOINTR_EMPTY (0 << 9) /* when empty */
-#define PCI224_DACCON_FIFOINTR_NEMPTY (1 << 9) /* when not empty */
-#define PCI224_DACCON_FIFOINTR_NHALF (2 << 9) /* when not half full */
-#define PCI224_DACCON_FIFOINTR_HALF (3 << 9) /* when half full */
-#define PCI224_DACCON_FIFOINTR_NFULL (4 << 9) /* when not full */
-#define PCI224_DACCON_FIFOINTR_FULL (5 << 9) /* when full */
+#define PCI224_DACCON_FIFOINTR(x) (((x) & 0x7) << 9)
+#define PCI224_DACCON_FIFOINTR_MASK PCI224_DACCON_FIFOINTR(7)
+#define PCI224_DACCON_FIFOINTR_EMPTY PCI224_DACCON_FIFOINTR(0) /* empty */
+#define PCI224_DACCON_FIFOINTR_NEMPTY PCI224_DACCON_FIFOINTR(1) /* !empty */
+#define PCI224_DACCON_FIFOINTR_NHALF PCI224_DACCON_FIFOINTR(2) /* !half */
+#define PCI224_DACCON_FIFOINTR_HALF PCI224_DACCON_FIFOINTR(3) /* half */
+#define PCI224_DACCON_FIFOINTR_NFULL PCI224_DACCON_FIFOINTR(4) /* !full */
+#define PCI224_DACCON_FIFOINTR_FULL PCI224_DACCON_FIFOINTR(5) /* full */
/* (r-o) FIFO fill level. */
-#define PCI224_DACCON_FIFOFL_MASK (7 << 12)
-#define PCI224_DACCON_FIFOFL_EMPTY (1 << 12) /* 0 */
-#define PCI224_DACCON_FIFOFL_ONETOHALF (0 << 12) /* [1,2048] */
-#define PCI224_DACCON_FIFOFL_HALFTOFULL (4 << 12) /* [2049,4095] */
-#define PCI224_DACCON_FIFOFL_FULL (6 << 12) /* 4096 */
+#define PCI224_DACCON_FIFOFL(x) (((x) & 0x7) << 12)
+#define PCI224_DACCON_FIFOFL_MASK PCI224_DACCON_FIFOFL(7)
+#define PCI224_DACCON_FIFOFL_EMPTY PCI224_DACCON_FIFOFL(1) /* 0 */
+#define PCI224_DACCON_FIFOFL_ONETOHALF PCI224_DACCON_FIFOFL(0) /* 1-2048 */
+#define PCI224_DACCON_FIFOFL_HALFTOFULL PCI224_DACCON_FIFOFL(4) /* 2049-4095 */
+#define PCI224_DACCON_FIFOFL_FULL PCI224_DACCON_FIFOFL(6) /* 4096 */
/* (r-o) DAC busy flag. */
-#define PCI224_DACCON_BUSY (1 << 15)
+#define PCI224_DACCON_BUSY BIT(15)
/* (w-o) FIFO reset. */
-#define PCI224_DACCON_FIFORESET (1 << 12)
+#define PCI224_DACCON_FIFORESET BIT(12)
/* (w-o) Global reset (not sure what it does). */
-#define PCI224_DACCON_GLOBALRESET (1 << 13)
+#define PCI224_DACCON_GLOBALRESET BIT(13)
/*
* DAC FIFO size.
diff --git a/drivers/staging/comedi/drivers/amplc_pci230.c b/drivers/staging/comedi/drivers/amplc_pci230.c
index 907c39c..42945de 100644
--- a/drivers/staging/comedi/drivers/amplc_pci230.c
+++ b/drivers/staging/comedi/drivers/amplc_pci230.c
@@ -237,47 +237,50 @@
/*
* DACCON read-write values.
*/
-#define PCI230_DAC_OR_UNI (0 << 0) /* Output range unipolar */
-#define PCI230_DAC_OR_BIP (1 << 0) /* Output range bipolar */
-#define PCI230_DAC_OR_MASK (1 << 0)
+#define PCI230_DAC_OR(x) (((x) & 0x1) << 0)
+#define PCI230_DAC_OR_UNI PCI230_DAC_OR(0) /* Output unipolar */
+#define PCI230_DAC_OR_BIP PCI230_DAC_OR(1) /* Output bipolar */
+#define PCI230_DAC_OR_MASK PCI230_DAC_OR(1)
/*
* The following applies only if DAC FIFO support is enabled in the EXTFUNC
* register (and only for PCI230+ hardware version 2 onwards).
*/
-#define PCI230P2_DAC_FIFO_EN (1 << 8) /* FIFO enable */
+#define PCI230P2_DAC_FIFO_EN BIT(8) /* FIFO enable */
/*
* The following apply only if the DAC FIFO is enabled (and only for PCI230+
* hardware version 2 onwards).
*/
-#define PCI230P2_DAC_TRIG_NONE (0 << 2) /* No trigger */
-#define PCI230P2_DAC_TRIG_SW (1 << 2) /* Software trigger trigger */
-#define PCI230P2_DAC_TRIG_EXTP (2 << 2) /* EXTTRIG +ve edge trigger */
-#define PCI230P2_DAC_TRIG_EXTN (3 << 2) /* EXTTRIG -ve edge trigger */
-#define PCI230P2_DAC_TRIG_Z2CT0 (4 << 2) /* CT0-OUT +ve edge trigger */
-#define PCI230P2_DAC_TRIG_Z2CT1 (5 << 2) /* CT1-OUT +ve edge trigger */
-#define PCI230P2_DAC_TRIG_Z2CT2 (6 << 2) /* CT2-OUT +ve edge trigger */
-#define PCI230P2_DAC_TRIG_MASK (7 << 2)
-#define PCI230P2_DAC_FIFO_WRAP (1 << 7) /* FIFO wraparound mode */
-#define PCI230P2_DAC_INT_FIFO_EMPTY (0 << 9) /* FIFO interrupt empty */
-#define PCI230P2_DAC_INT_FIFO_NEMPTY (1 << 9)
-#define PCI230P2_DAC_INT_FIFO_NHALF (2 << 9) /* FIFO intr not half full */
-#define PCI230P2_DAC_INT_FIFO_HALF (3 << 9)
-#define PCI230P2_DAC_INT_FIFO_NFULL (4 << 9) /* FIFO interrupt not full */
-#define PCI230P2_DAC_INT_FIFO_FULL (5 << 9)
-#define PCI230P2_DAC_INT_FIFO_MASK (7 << 9)
+#define PCI230P2_DAC_TRIG(x) (((x) & 0x7) << 2)
+#define PCI230P2_DAC_TRIG_NONE PCI230P2_DAC_TRIG(0) /* none */
+#define PCI230P2_DAC_TRIG_SW PCI230P2_DAC_TRIG(1) /* soft trig */
+#define PCI230P2_DAC_TRIG_EXTP PCI230P2_DAC_TRIG(2) /* ext + edge */
+#define PCI230P2_DAC_TRIG_EXTN PCI230P2_DAC_TRIG(3) /* ext - edge */
+#define PCI230P2_DAC_TRIG_Z2CT0 PCI230P2_DAC_TRIG(4) /* Z2 CT0 out */
+#define PCI230P2_DAC_TRIG_Z2CT1 PCI230P2_DAC_TRIG(5) /* Z2 CT1 out */
+#define PCI230P2_DAC_TRIG_Z2CT2 PCI230P2_DAC_TRIG(6) /* Z2 CT2 out */
+#define PCI230P2_DAC_TRIG_MASK PCI230P2_DAC_TRIG(7)
+#define PCI230P2_DAC_FIFO_WRAP BIT(7) /* FIFO wraparound mode */
+#define PCI230P2_DAC_INT_FIFO(x) (((x) & 7) << 9)
+#define PCI230P2_DAC_INT_FIFO_EMPTY PCI230P2_DAC_INT_FIFO(0) /* empty */
+#define PCI230P2_DAC_INT_FIFO_NEMPTY PCI230P2_DAC_INT_FIFO(1) /* !empty */
+#define PCI230P2_DAC_INT_FIFO_NHALF PCI230P2_DAC_INT_FIFO(2) /* !half */
+#define PCI230P2_DAC_INT_FIFO_HALF PCI230P2_DAC_INT_FIFO(3) /* half */
+#define PCI230P2_DAC_INT_FIFO_NFULL PCI230P2_DAC_INT_FIFO(4) /* !full */
+#define PCI230P2_DAC_INT_FIFO_FULL PCI230P2_DAC_INT_FIFO(5) /* full */
+#define PCI230P2_DAC_INT_FIFO_MASK PCI230P2_DAC_INT_FIFO(7)
/*
* DACCON read-only values.
*/
-#define PCI230_DAC_BUSY (1 << 1) /* DAC busy. */
+#define PCI230_DAC_BUSY BIT(1) /* DAC busy. */
/*
* The following apply only if the DAC FIFO is enabled (and only for PCI230+
* hardware version 2 onwards).
*/
-#define PCI230P2_DAC_FIFO_UNDERRUN_LATCHED (1 << 5) /* Underrun error */
-#define PCI230P2_DAC_FIFO_EMPTY (1 << 13) /* FIFO empty */
-#define PCI230P2_DAC_FIFO_FULL (1 << 14) /* FIFO full */
-#define PCI230P2_DAC_FIFO_HALF (1 << 15) /* FIFO half full */
+#define PCI230P2_DAC_FIFO_UNDERRUN_LATCHED BIT(5) /* Underrun error */
+#define PCI230P2_DAC_FIFO_EMPTY BIT(13) /* FIFO empty */
+#define PCI230P2_DAC_FIFO_FULL BIT(14) /* FIFO full */
+#define PCI230P2_DAC_FIFO_HALF BIT(15) /* FIFO half full */
/*
* DACCON write-only, transient values.
@@ -286,8 +289,8 @@
* The following apply only if the DAC FIFO is enabled (and only for PCI230+
* hardware version 2 onwards).
*/
-#define PCI230P2_DAC_FIFO_UNDERRUN_CLEAR (1 << 5) /* Clear underrun */
-#define PCI230P2_DAC_FIFO_RESET (1 << 12) /* FIFO reset */
+#define PCI230P2_DAC_FIFO_UNDERRUN_CLEAR BIT(5) /* Clear underrun */
+#define PCI230P2_DAC_FIFO_RESET BIT(12) /* FIFO reset */
/*
* PCI230+ hardware version 2 DAC FIFO levels.
@@ -304,44 +307,48 @@
/*
* ADCCON read/write values.
*/
-#define PCI230_ADC_TRIG_NONE (0 << 0) /* No trigger */
-#define PCI230_ADC_TRIG_SW (1 << 0) /* Software trigger trigger */
-#define PCI230_ADC_TRIG_EXTP (2 << 0) /* EXTTRIG +ve edge trigger */
-#define PCI230_ADC_TRIG_EXTN (3 << 0) /* EXTTRIG -ve edge trigger */
-#define PCI230_ADC_TRIG_Z2CT0 (4 << 0) /* CT0-OUT +ve edge trigger */
-#define PCI230_ADC_TRIG_Z2CT1 (5 << 0) /* CT1-OUT +ve edge trigger */
-#define PCI230_ADC_TRIG_Z2CT2 (6 << 0) /* CT2-OUT +ve edge trigger */
-#define PCI230_ADC_TRIG_MASK (7 << 0)
-#define PCI230_ADC_IR_UNI (0 << 3) /* Input range unipolar */
-#define PCI230_ADC_IR_BIP (1 << 3) /* Input range bipolar */
-#define PCI230_ADC_IR_MASK (1 << 3)
-#define PCI230_ADC_IM_SE (0 << 4) /* Input mode single ended */
-#define PCI230_ADC_IM_DIF (1 << 4) /* Input mode differential */
-#define PCI230_ADC_IM_MASK (1 << 4)
-#define PCI230_ADC_FIFO_EN (1 << 8) /* FIFO enable */
-#define PCI230_ADC_INT_FIFO_EMPTY (0 << 9)
-#define PCI230_ADC_INT_FIFO_NEMPTY (1 << 9) /* FIFO interrupt not empty */
-#define PCI230_ADC_INT_FIFO_NHALF (2 << 9)
-#define PCI230_ADC_INT_FIFO_HALF (3 << 9) /* FIFO interrupt half full */
-#define PCI230_ADC_INT_FIFO_NFULL (4 << 9)
-#define PCI230_ADC_INT_FIFO_FULL (5 << 9) /* FIFO interrupt full */
-#define PCI230P_ADC_INT_FIFO_THRESH (7 << 9) /* FIFO interrupt threshold */
-#define PCI230_ADC_INT_FIFO_MASK (7 << 9)
+#define PCI230_ADC_TRIG(x) (((x) & 0x7) << 0)
+#define PCI230_ADC_TRIG_NONE PCI230_ADC_TRIG(0) /* none */
+#define PCI230_ADC_TRIG_SW PCI230_ADC_TRIG(1) /* soft trig */
+#define PCI230_ADC_TRIG_EXTP PCI230_ADC_TRIG(2) /* ext + edge */
+#define PCI230_ADC_TRIG_EXTN PCI230_ADC_TRIG(3) /* ext - edge */
+#define PCI230_ADC_TRIG_Z2CT0 PCI230_ADC_TRIG(4) /* Z2 CT0 out*/
+#define PCI230_ADC_TRIG_Z2CT1 PCI230_ADC_TRIG(5) /* Z2 CT1 out */
+#define PCI230_ADC_TRIG_Z2CT2 PCI230_ADC_TRIG(6) /* Z2 CT2 out */
+#define PCI230_ADC_TRIG_MASK PCI230_ADC_TRIG(7)
+#define PCI230_ADC_IR(x) (((x) & 0x1) << 3)
+#define PCI230_ADC_IR_UNI PCI230_ADC_IR(0) /* Input unipolar */
+#define PCI230_ADC_IR_BIP PCI230_ADC_IR(1) /* Input bipolar */
+#define PCI230_ADC_IR_MASK PCI230_ADC_IR(1)
+#define PCI230_ADC_IM(x) (((x) & 0x1) << 4)
+#define PCI230_ADC_IM_SE PCI230_ADC_IM(0) /* single ended */
+#define PCI230_ADC_IM_DIF PCI230_ADC_IM(1) /* differential */
+#define PCI230_ADC_IM_MASK PCI230_ADC_IM(1)
+#define PCI230_ADC_FIFO_EN BIT(8) /* FIFO enable */
+#define PCI230_ADC_INT_FIFO(x) (((x) & 0x7) << 9)
+#define PCI230_ADC_INT_FIFO_EMPTY PCI230_ADC_INT_FIFO(0) /* empty */
+#define PCI230_ADC_INT_FIFO_NEMPTY PCI230_ADC_INT_FIFO(1) /* !empty */
+#define PCI230_ADC_INT_FIFO_NHALF PCI230_ADC_INT_FIFO(2) /* !half */
+#define PCI230_ADC_INT_FIFO_HALF PCI230_ADC_INT_FIFO(3) /* half */
+#define PCI230_ADC_INT_FIFO_NFULL PCI230_ADC_INT_FIFO(4) /* !full */
+#define PCI230_ADC_INT_FIFO_FULL PCI230_ADC_INT_FIFO(5) /* full */
+#define PCI230P_ADC_INT_FIFO_THRESH PCI230_ADC_INT_FIFO(7) /* threshold */
+#define PCI230_ADC_INT_FIFO_MASK PCI230_ADC_INT_FIFO(7)
/*
* ADCCON write-only, transient values.
*/
-#define PCI230_ADC_FIFO_RESET (1 << 12) /* FIFO reset */
-#define PCI230_ADC_GLOB_RESET (1 << 13) /* Global reset */
+#define PCI230_ADC_FIFO_RESET BIT(12) /* FIFO reset */
+#define PCI230_ADC_GLOB_RESET BIT(13) /* Global reset */
/*
* ADCCON read-only values.
*/
-#define PCI230_ADC_BUSY (1 << 15) /* ADC busy */
-#define PCI230_ADC_FIFO_EMPTY (1 << 12) /* FIFO empty */
-#define PCI230_ADC_FIFO_FULL (1 << 13) /* FIFO full */
-#define PCI230_ADC_FIFO_HALF (1 << 14) /* FIFO half full */
-#define PCI230_ADC_FIFO_FULL_LATCHED (1 << 5) /* FIFO overrun occurred */
+#define PCI230_ADC_BUSY BIT(15) /* ADC busy */
+#define PCI230_ADC_FIFO_EMPTY BIT(12) /* FIFO empty */
+#define PCI230_ADC_FIFO_FULL BIT(13) /* FIFO full */
+#define PCI230_ADC_FIFO_HALF BIT(14) /* FIFO half full */
+#define PCI230_ADC_FIFO_FULL_LATCHED BIT(5) /* FIFO overrun occurred */
/*
* PCI230 ADC FIFO levels.
@@ -353,10 +360,10 @@
* PCI230+ EXTFUNC values.
*/
/* Route EXTTRIG pin to external gate inputs. */
-#define PCI230P_EXTFUNC_GAT_EXTTRIG (1 << 0)
+#define PCI230P_EXTFUNC_GAT_EXTTRIG BIT(0)
/* PCI230+ hardware version 2 values. */
/* Allow DAC FIFO to be enabled. */
-#define PCI230P2_EXTFUNC_DACFIFO (1 << 1)
+#define PCI230P2_EXTFUNC_DACFIFO BIT(1)
/*
* Counter/timer clock input configuration sources.
@@ -379,8 +386,12 @@
#define GAT_GND 1 /* GND (i.e. disabled) */
#define GAT_EXT 2 /* external gate input (PPCn on PCI230) */
#define GAT_NOUTNM2 3 /* inverted output of channel-2 modulo total */
-/* Macro to construct gate input configuration register value. */
-#define GAT_CONFIG(chan, src) ((((chan) & 3) << 3) | ((src) & 7))
+
+static inline unsigned int pci230_gat_config(unsigned int chan,
+ unsigned int src)
+{
+ return ((chan & 3) << 3) | (src & 7);
+}
/*
* Summary of CLK_OUTNM1 and GAT_NOUTNM2 connections for PCI230 and PCI260:
@@ -398,20 +409,20 @@
* Interrupt enables/status register values.
*/
#define PCI230_INT_DISABLE 0
-#define PCI230_INT_PPI_C0 (1 << 0)
-#define PCI230_INT_PPI_C3 (1 << 1)
-#define PCI230_INT_ADC (1 << 2)
-#define PCI230_INT_ZCLK_CT1 (1 << 5)
+#define PCI230_INT_PPI_C0 BIT(0)
+#define PCI230_INT_PPI_C3 BIT(1)
+#define PCI230_INT_ADC BIT(2)
+#define PCI230_INT_ZCLK_CT1 BIT(5)
/* For PCI230+ hardware version 2 when DAC FIFO enabled. */
-#define PCI230P2_INT_DAC (1 << 4)
+#define PCI230P2_INT_DAC BIT(4)
/*
* (Potentially) shared resources and their owners
*/
enum {
- RES_Z2CT0 = (1U << 0), /* Z2-CT0 */
- RES_Z2CT1 = (1U << 1), /* Z2-CT1 */
- RES_Z2CT2 = (1U << 2) /* Z2-CT2 */
+ RES_Z2CT0 = BIT(0), /* Z2-CT0 */
+ RES_Z2CT1 = BIT(1), /* Z2-CT1 */
+ RES_Z2CT2 = BIT(2) /* Z2-CT2 */
};
enum {
@@ -626,10 +637,10 @@
pci230_release_shared(dev, (unsigned char)~0, owner);
}
-static unsigned int pci230_divide_ns(uint64_t ns, unsigned int timebase,
+static unsigned int pci230_divide_ns(u64 ns, unsigned int timebase,
unsigned int flags)
{
- uint64_t div;
+ u64 div;
unsigned int rem;
div = ns;
@@ -652,7 +663,7 @@
* Given desired period in ns, returns the required internal clock source
* and gets the initial count.
*/
-static unsigned int pci230_choose_clk_count(uint64_t ns, unsigned int *count,
+static unsigned int pci230_choose_clk_count(u64 ns, unsigned int *count,
unsigned int flags)
{
unsigned int clk_src, cnt;
@@ -676,7 +687,7 @@
}
static void pci230_ct_setup_ns_mode(struct comedi_device *dev, unsigned int ct,
- unsigned int mode, uint64_t ns,
+ unsigned int mode, u64 ns,
unsigned int flags)
{
unsigned int clk_src;
@@ -1263,7 +1274,8 @@
irqflags);
}
/* Set CT1 gate high to start counting. */
- outb(GAT_CONFIG(1, GAT_VCC), dev->iobase + PCI230_ZGAT_SCE);
+ outb(pci230_gat_config(1, GAT_VCC),
+ dev->iobase + PCI230_ZGAT_SCE);
break;
case TRIG_INT:
async->inttrig = pci230_ao_inttrig_scan_begin;
@@ -1351,7 +1363,8 @@
* cmd->scan_begin_arg is sampling period in ns.
* Gate it off for now.
*/
- outb(GAT_CONFIG(1, GAT_GND), dev->iobase + PCI230_ZGAT_SCE);
+ outb(pci230_gat_config(1, GAT_GND),
+ dev->iobase + PCI230_ZGAT_SCE);
pci230_ct_setup_ns_mode(dev, 1, I8254_MODE3,
cmd->scan_begin_arg,
cmd->flags);
@@ -1792,9 +1805,9 @@
spin_lock_irqsave(&devpriv->ai_stop_spinlock, irqflags);
if (devpriv->ai_cmd_started) {
/* Trigger scan by waggling CT0 gate source. */
- zgat = GAT_CONFIG(0, GAT_GND);
+ zgat = pci230_gat_config(0, GAT_GND);
outb(zgat, dev->iobase + PCI230_ZGAT_SCE);
- zgat = GAT_CONFIG(0, GAT_VCC);
+ zgat = pci230_gat_config(0, GAT_VCC);
outb(zgat, dev->iobase + PCI230_ZGAT_SCE);
}
spin_unlock_irqrestore(&devpriv->ai_stop_spinlock, irqflags);
@@ -1926,20 +1939,20 @@
* Conversion timer CT2 needs to be gated by
* inverted output of monostable CT2.
*/
- zgat = GAT_CONFIG(2, GAT_NOUTNM2);
+ zgat = pci230_gat_config(2, GAT_NOUTNM2);
} else {
/*
* Conversion timer CT2 needs to be gated on
* continuously.
*/
- zgat = GAT_CONFIG(2, GAT_VCC);
+ zgat = pci230_gat_config(2, GAT_VCC);
}
outb(zgat, dev->iobase + PCI230_ZGAT_SCE);
if (cmd->scan_begin_src != TRIG_FOLLOW) {
/* Set monostable CT0 trigger source. */
switch (cmd->scan_begin_src) {
default:
- zgat = GAT_CONFIG(0, GAT_VCC);
+ zgat = pci230_gat_config(0, GAT_VCC);
break;
case TRIG_EXT:
/*
@@ -1950,21 +1963,21 @@
* input in order to use it as an external scan
* trigger.
*/
- zgat = GAT_CONFIG(0, GAT_EXT);
+ zgat = pci230_gat_config(0, GAT_EXT);
break;
case TRIG_TIMER:
/*
* Monostable CT0 triggered by rising edge on
* inverted output of CT1 (falling edge on CT1).
*/
- zgat = GAT_CONFIG(0, GAT_NOUTNM2);
+ zgat = pci230_gat_config(0, GAT_NOUTNM2);
break;
case TRIG_INT:
/*
* Monostable CT0 is triggered by inttrig
* function waggling the CT0 gate source.
*/
- zgat = GAT_CONFIG(0, GAT_VCC);
+ zgat = pci230_gat_config(0, GAT_VCC);
break;
}
outb(zgat, dev->iobase + PCI230_ZGAT_SCE);
@@ -1974,7 +1987,7 @@
* Scan period timer CT1 needs to be
* gated on to start counting.
*/
- zgat = GAT_CONFIG(1, GAT_VCC);
+ zgat = pci230_gat_config(1, GAT_VCC);
outb(zgat, dev->iobase + PCI230_ZGAT_SCE);
break;
case TRIG_INT:
@@ -2216,7 +2229,7 @@
* Note, counter/timer output 2 can be monitored on the
* connector: PCI230 pin 21, PCI260 pin 18.
*/
- zgat = GAT_CONFIG(2, GAT_GND);
+ zgat = pci230_gat_config(2, GAT_GND);
outb(zgat, dev->iobase + PCI230_ZGAT_SCE);
/* Set counter/timer 2 to the specified conversion period. */
pci230_ct_setup_ns_mode(dev, 2, I8254_MODE3, cmd->convert_arg,
@@ -2234,10 +2247,10 @@
* monostable to stop it triggering. The trigger
* source will be changed later.
*/
- zgat = GAT_CONFIG(0, GAT_VCC);
+ zgat = pci230_gat_config(0, GAT_VCC);
outb(zgat, dev->iobase + PCI230_ZGAT_SCE);
pci230_ct_setup_ns_mode(dev, 0, I8254_MODE1,
- ((uint64_t)cmd->convert_arg *
+ ((u64)cmd->convert_arg *
cmd->scan_end_arg),
CMDF_ROUND_UP);
if (cmd->scan_begin_src == TRIG_TIMER) {
@@ -2247,7 +2260,7 @@
*
* Set up CT1 but gate it off for now.
*/
- zgat = GAT_CONFIG(1, GAT_GND);
+ zgat = pci230_gat_config(1, GAT_GND);
outb(zgat, dev->iobase + PCI230_ZGAT_SCE);
pci230_ct_setup_ns_mode(dev, 1, I8254_MODE3,
cmd->scan_begin_arg,
diff --git a/drivers/staging/comedi/drivers/amplc_pci263.c b/drivers/staging/comedi/drivers/amplc_pci263.c
index b6768aa..8d4069b 100644
--- a/drivers/staging/comedi/drivers/amplc_pci263.c
+++ b/drivers/staging/comedi/drivers/amplc_pci263.c
@@ -1,49 +1,53 @@
/*
- comedi/drivers/amplc_pci263.c
- Driver for Amplicon PCI263 relay board.
+ * Driver for Amplicon PCI263 relay board.
+ *
+ * Copyright (C) 2002 MEV Ltd. <http://www.mev.co.uk/>
+ *
+ * COMEDI - Linux Control and Measurement Device Interface
+ * Copyright (C) 2000 David A. Schleef <ds@schleef.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
- Copyright (C) 2002 MEV Ltd. <http://www.mev.co.uk/>
-
- COMEDI - Linux Control and Measurement Device Interface
- Copyright (C) 2000 David A. Schleef <ds@schleef.org>
-
- This program is free software; you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation; either version 2 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-*/
/*
-Driver: amplc_pci263
-Description: Amplicon PCI263
-Author: Ian Abbott <abbotti@mev.co.uk>
-Devices: [Amplicon] PCI263 (amplc_pci263)
-Updated: Fri, 12 Apr 2013 15:19:36 +0100
-Status: works
-
-Configuration options: not applicable, uses PCI auto config
-
-The board appears as one subdevice, with 16 digital outputs, each
-connected to a reed-relay. Relay contacts are closed when output is 1.
-The state of the outputs can be read.
-*/
+ * Driver: amplc_pci263
+ * Description: Amplicon PCI263
+ * Author: Ian Abbott <abbotti@mev.co.uk>
+ * Devices: [Amplicon] PCI263 (amplc_pci263)
+ * Updated: Fri, 12 Apr 2013 15:19:36 +0100
+ * Status: works
+ *
+ * Configuration options: not applicable, uses PCI auto config
+ *
+ * The board appears as one subdevice, with 16 digital outputs, each
+ * connected to a reed-relay. Relay contacts are closed when output is 1.
+ * The state of the outputs can be read.
+ */
#include <linux/module.h>
#include "../comedi_pci.h"
+/* PCI263 registers */
+#define PCI263_DO_0_7_REG 0x00
+#define PCI263_DO_8_15_REG 0x01
+
static int pci263_do_insn_bits(struct comedi_device *dev,
struct comedi_subdevice *s,
struct comedi_insn *insn,
unsigned int *data)
{
if (comedi_dio_update_state(s, data)) {
- outb(s->state & 0xff, dev->iobase);
- outb((s->state >> 8) & 0xff, dev->iobase + 1);
+ outb(s->state & 0xff, dev->iobase + PCI263_DO_0_7_REG);
+ outb((s->state >> 8) & 0xff, dev->iobase + PCI263_DO_8_15_REG);
}
data[1] = s->state;
@@ -67,16 +71,18 @@
if (ret)
return ret;
+ /* Digital Output subdevice */
s = &dev->subdevices[0];
- /* digital output subdevice */
- s->type = COMEDI_SUBD_DO;
- s->subdev_flags = SDF_WRITABLE;
- s->n_chan = 16;
- s->maxdata = 1;
- s->range_table = &range_digital;
- s->insn_bits = pci263_do_insn_bits;
+ s->type = COMEDI_SUBD_DO;
+ s->subdev_flags = SDF_WRITABLE;
+ s->n_chan = 16;
+ s->maxdata = 1;
+ s->range_table = &range_digital;
+ s->insn_bits = pci263_do_insn_bits;
+
/* read initial relay state */
- s->state = inb(dev->iobase) | (inb(dev->iobase + 1) << 8);
+ s->state = inb(dev->iobase + PCI263_DO_0_7_REG) |
+ (inb(dev->iobase + PCI263_DO_8_15_REG) << 8);
return 0;
}
diff --git a/drivers/staging/comedi/drivers/c6xdigio.c b/drivers/staging/comedi/drivers/c6xdigio.c
index 1a109e3..8ee7325 100644
--- a/drivers/staging/comedi/drivers/c6xdigio.c
+++ b/drivers/staging/comedi/drivers/c6xdigio.c
@@ -47,8 +47,8 @@
*/
#define C6XDIGIO_DATA_REG 0x00
#define C6XDIGIO_DATA_CHAN(x) (((x) + 1) << 4)
-#define C6XDIGIO_DATA_PWM (1 << 5)
-#define C6XDIGIO_DATA_ENCODER (1 << 6)
+#define C6XDIGIO_DATA_PWM BIT(5)
+#define C6XDIGIO_DATA_ENCODER BIT(6)
#define C6XDIGIO_STATUS_REG 0x01
#define C6XDIGIO_CTRL_REG 0x02
diff --git a/drivers/staging/comedi/drivers/comedi_8254.h b/drivers/staging/comedi/drivers/comedi_8254.h
index f4610ea..a12c294 100644
--- a/drivers/staging/comedi/drivers/comedi_8254.h
+++ b/drivers/staging/comedi/drivers/comedi_8254.h
@@ -53,13 +53,15 @@
#define I8254_COUNTER2_REG 0x02
#define I8254_CTRL_REG 0x03
#define I8254_CTRL_SEL_CTR(x) ((x) << 6)
-#define I8254_CTRL_READBACK_COUNT ((3 << 6) | (1 << 4))
-#define I8254_CTRL_READBACK_STATUS ((3 << 6) | (1 << 5))
+#define I8254_CTRL_READBACK(x) (I8254_CTRL_SEL_CTR(3) | BIT(x))
+#define I8254_CTRL_READBACK_COUNT I8254_CTRL_READBACK(4)
+#define I8254_CTRL_READBACK_STATUS I8254_CTRL_READBACK(5)
#define I8254_CTRL_READBACK_SEL_CTR(x) (2 << (x))
-#define I8254_CTRL_LATCH (0 << 4)
-#define I8254_CTRL_LSB_ONLY (1 << 4)
-#define I8254_CTRL_MSB_ONLY (2 << 4)
-#define I8254_CTRL_LSB_MSB (3 << 4)
+#define I8254_CTRL_RW(x) (((x) & 0x3) << 4)
+#define I8254_CTRL_LATCH I8254_CTRL_RW(0)
+#define I8254_CTRL_LSB_ONLY I8254_CTRL_RW(1)
+#define I8254_CTRL_MSB_ONLY I8254_CTRL_RW(2)
+#define I8254_CTRL_LSB_MSB I8254_CTRL_RW(3)
/* counter maps zero to 0x10000 */
#define I8254_MAX_COUNT 0x10000
diff --git a/drivers/staging/comedi/drivers/daqboard2000.c b/drivers/staging/comedi/drivers/daqboard2000.c
index 57ab668..a536a15 100644
--- a/drivers/staging/comedi/drivers/daqboard2000.c
+++ b/drivers/staging/comedi/drivers/daqboard2000.c
@@ -26,7 +26,7 @@
* Much of the functionality of this driver was determined from reading
* the source code for the Windows driver.
*
- * The FPGA on the board requires fimware, which is available from
+ * The FPGA on the board requires firmware, which is available from
* http://www.comedi.org in the comedi_nonfree_firmware tarball.
*
* Configuration options: not applicable, uses PCI auto config
diff --git a/drivers/staging/comedi/drivers/das1800.c b/drivers/staging/comedi/drivers/das1800.c
index 94078118..e0a34c2 100644
--- a/drivers/staging/comedi/drivers/das1800.c
+++ b/drivers/staging/comedi/drivers/das1800.c
@@ -1,98 +1,82 @@
/*
- comedi/drivers/das1800.c
- Driver for Keitley das1700/das1800 series boards
- Copyright (C) 2000 Frank Mori Hess <fmhess@users.sourceforge.net>
+ * Comedi driver for Keithley DAS-1700/DAS-1800 series boards
+ * Copyright (C) 2000 Frank Mori Hess <fmhess@users.sourceforge.net>
+ *
+ * COMEDI - Linux Control and Measurement Device Interface
+ * Copyright (C) 2000 David A. Schleef <ds@schleef.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
- COMEDI - Linux Control and Measurement Device Interface
- Copyright (C) 2000 David A. Schleef <ds@schleef.org>
-
- This program is free software; you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation; either version 2 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-*/
/*
-Driver: das1800
-Description: Keithley Metrabyte DAS1800 (& compatibles)
-Author: Frank Mori Hess <fmhess@users.sourceforge.net>
-Devices: [Keithley Metrabyte] DAS-1701ST (das-1701st),
- DAS-1701ST-DA (das-1701st-da), DAS-1701/AO (das-1701ao),
- DAS-1702ST (das-1702st), DAS-1702ST-DA (das-1702st-da),
- DAS-1702HR (das-1702hr), DAS-1702HR-DA (das-1702hr-da),
- DAS-1702/AO (das-1702ao), DAS-1801ST (das-1801st),
- DAS-1801ST-DA (das-1801st-da), DAS-1801HC (das-1801hc),
- DAS-1801AO (das-1801ao), DAS-1802ST (das-1802st),
- DAS-1802ST-DA (das-1802st-da), DAS-1802HR (das-1802hr),
- DAS-1802HR-DA (das-1802hr-da), DAS-1802HC (das-1802hc),
- DAS-1802AO (das-1802ao)
-Status: works
-
-The waveform analog output on the 'ao' cards is not supported.
-If you need it, send me (Frank Hess) an email.
-
-Configuration options:
- [0] - I/O port base address
- [1] - IRQ (optional, required for timed or externally triggered conversions)
- [2] - DMA0 (optional, requires irq)
- [3] - DMA1 (optional, requires irq and dma0)
-*/
-/*
-
-This driver supports the following Keithley boards:
-
-das-1701st
-das-1701st-da
-das-1701ao
-das-1702st
-das-1702st-da
-das-1702hr
-das-1702hr-da
-das-1702ao
-das-1801st
-das-1801st-da
-das-1801hc
-das-1801ao
-das-1802st
-das-1802st-da
-das-1802hr
-das-1802hr-da
-das-1802hc
-das-1802ao
-
-Options:
- [0] - base io address
- [1] - irq (optional, required for timed or externally triggered conversions)
- [2] - dma0 (optional, requires irq)
- [3] - dma1 (optional, requires irq and dma0)
-
-irq can be omitted, although the cmd interface will not work without it.
-
-analog input cmd triggers supported:
- start_src: TRIG_NOW | TRIG_EXT
- scan_begin_src: TRIG_FOLLOW | TRIG_TIMER | TRIG_EXT
- scan_end_src: TRIG_COUNT
- convert_src: TRIG_TIMER | TRIG_EXT (TRIG_EXT requires scan_begin_src == TRIG_FOLLOW)
- stop_src: TRIG_COUNT | TRIG_EXT | TRIG_NONE
-
-scan_begin_src triggers TRIG_TIMER and TRIG_EXT use the card's
-'burst mode' which limits the valid conversion time to 64 microseconds
-(convert_arg <= 64000). This limitation does not apply if scan_begin_src
-is TRIG_FOLLOW.
-
-NOTES:
-Only the DAS-1801ST has been tested by me.
-Unipolar and bipolar ranges cannot be mixed in the channel/gain list.
-
-TODO:
- Make it automatically allocate irq and dma channels if they are not specified
- Add support for analog out on 'ao' cards
- read insn for analog out
-*/
+ * Driver: das1800
+ * Description: Keithley Metrabyte DAS1800 (& compatibles)
+ * Author: Frank Mori Hess <fmhess@users.sourceforge.net>
+ * Devices: [Keithley Metrabyte] DAS-1701ST (das-1701st),
+ * DAS-1701ST-DA (das-1701st-da), DAS-1701/AO (das-1701ao),
+ * DAS-1702ST (das-1702st), DAS-1702ST-DA (das-1702st-da),
+ * DAS-1702HR (das-1702hr), DAS-1702HR-DA (das-1702hr-da),
+ * DAS-1702/AO (das-1702ao), DAS-1801ST (das-1801st),
+ * DAS-1801ST-DA (das-1801st-da), DAS-1801HC (das-1801hc),
+ * DAS-1801AO (das-1801ao), DAS-1802ST (das-1802st),
+ * DAS-1802ST-DA (das-1802st-da), DAS-1802HR (das-1802hr),
+ * DAS-1802HR-DA (das-1802hr-da), DAS-1802HC (das-1802hc),
+ * DAS-1802AO (das-1802ao)
+ * Status: works
+ *
+ * Configuration options:
+ * [0] - I/O port base address
+ * [1] - IRQ (optional, required for analog input cmd support)
+ * [2] - DMA0 (optional, requires irq)
+ * [3] - DMA1 (optional, requires irq and dma0)
+ *
+ * analog input cmd triggers supported:
+ *
+ * start_src TRIG_NOW command starts immediately
+ * TRIG_EXT command starts on external pin TGIN
+ *
+ * scan_begin_src TRIG_FOLLOW paced/external scans start immediately
+ * TRIG_TIMER burst scans start periodically
+ * TRIG_EXT burst scans start on external pin XPCLK
+ *
+ * scan_end_src TRIG_COUNT scan ends after last channel
+ *
+ * convert_src TRIG_TIMER paced/burst conversions are timed
+ * TRIG_EXT conversions on external pin XPCLK
+ * (requires scan_begin_src == TRIG_FOLLOW)
+ *
+ * stop_src TRIG_COUNT command stops after stop_arg scans
+ * TRIG_EXT command stops on external pin TGIN
+ * TRIG_NONE command runs until canceled
+ *
+ * If TRIG_EXT is used for both the start_src and stop_src, the first TGIN
+ * trigger starts the command, and the second trigger will stop it. If only
+ * one is TRIG_EXT, the first trigger will either stop or start the command.
+ * The external pin TGIN is normally set for negative edge triggering. It
+ * can be set to positive edge with the CR_INVERT flag. If TRIG_EXT is used
+ * for both the start_src and stop_src they must have the same polarity.
+ *
+ * Minimum conversion speed is limited to 64 microseconds (convert_arg <= 64000)
+ * for 'burst' scans. This limitation does not apply for 'paced' scans. The
+ * maximum conversion speed is limited by the board (convert_arg >= ai_speed).
+ * Maximum conversion speeds are not always achievable depending on the
+ * board setup (see user manual).
+ *
+ * NOTES:
+ * Only the DAS-1801ST has been tested by me.
+ * Unipolar and bipolar ranges cannot be mixed in the channel/gain list.
+ *
+ * The waveform analog output on the 'ao' cards is not supported.
+ * If you need it, send me (Frank Hess) an email.
+ */
#include <linux/module.h>
#include <linux/interrupt.h>
@@ -107,7 +91,6 @@
/* misc. defines */
#define DAS1800_SIZE 16 /* uses 16 io addresses */
#define FIFO_SIZE 1024 /* 1024 sample fifo */
-#define UNIPOLAR 0x4 /* bit that determines whether input range is uni/bipolar */
#define DMA_BUF_SIZE 0x1ff00 /* size in bytes of dma buffers */
/* Registers for the das1800 */
@@ -125,6 +108,7 @@
#define CGSL 0x8
#define TGEN 0x10
#define TGSL 0x20
+#define TGPL 0x40
#define ATEN 0x80
#define DAS1800_CONTROL_B 0x5
#define DMA_CH5 0x1
@@ -133,7 +117,7 @@
#define DMA_CH5_CH6 0x5
#define DMA_CH6_CH7 0x6
#define DMA_CH7_CH5 0x7
-#define DMA_ENABLED 0x3 /* mask used to determine if dma is enabled */
+#define DMA_ENABLED 0x3
#define DMA_DUAL 0x4
#define IRQ3 0x8
#define IRQ5 0x10
@@ -151,319 +135,214 @@
#define SD 0x40
#define UB 0x80
#define DAS1800_STATUS 0x7
-/* bits that prevent interrupt status bits (and CVEN) from being cleared on write */
-#define CLEAR_INTR_MASK (CVEN_MASK | 0x1f)
#define INT 0x1
#define DMATC 0x2
#define CT0TC 0x8
#define OVF 0x10
#define FHF 0x20
#define FNE 0x40
-#define CVEN_MASK 0x40 /* masks CVEN on write */
#define CVEN 0x80
+#define CVEN_MASK 0x40
+#define CLEAR_INTR_MASK (CVEN_MASK | 0x1f)
#define DAS1800_BURST_LENGTH 0x8
#define DAS1800_BURST_RATE 0x9
#define DAS1800_QRAM_ADDRESS 0xa
#define DAS1800_COUNTER 0xc
-#define IOBASE2 0x400 /* offset of additional ioports used on 'ao' cards */
+#define IOBASE2 0x400
-enum {
- das1701st, das1701st_da, das1702st, das1702st_da, das1702hr,
- das1702hr_da,
- das1701ao, das1702ao, das1801st, das1801st_da, das1802st, das1802st_da,
- das1802hr, das1802hr_da, das1801hc, das1802hc, das1801ao, das1802ao
-};
-
-/* analog input ranges */
-static const struct comedi_lrange range_ai_das1801 = {
+static const struct comedi_lrange das1801_ai_range = {
8, {
- BIP_RANGE(5),
- BIP_RANGE(1),
- BIP_RANGE(0.1),
- BIP_RANGE(0.02),
- UNI_RANGE(5),
- UNI_RANGE(1),
- UNI_RANGE(0.1),
- UNI_RANGE(0.02)
+ BIP_RANGE(5), /* bipolar gain = 1 */
+ BIP_RANGE(1), /* bipolar gain = 10 */
+ BIP_RANGE(0.1), /* bipolar gain = 50 */
+ BIP_RANGE(0.02), /* bipolar gain = 250 */
+ UNI_RANGE(5), /* unipolar gain = 1 */
+ UNI_RANGE(1), /* unipolar gain = 10 */
+ UNI_RANGE(0.1), /* unipolar gain = 50 */
+ UNI_RANGE(0.02) /* unipolar gain = 250 */
}
};
-static const struct comedi_lrange range_ai_das1802 = {
+static const struct comedi_lrange das1802_ai_range = {
8, {
- BIP_RANGE(10),
- BIP_RANGE(5),
- BIP_RANGE(2.5),
- BIP_RANGE(1.25),
- UNI_RANGE(10),
- UNI_RANGE(5),
- UNI_RANGE(2.5),
- UNI_RANGE(1.25)
+ BIP_RANGE(10), /* bipolar gain = 1 */
+ BIP_RANGE(5), /* bipolar gain = 2 */
+ BIP_RANGE(2.5), /* bipolar gain = 4 */
+ BIP_RANGE(1.25), /* bipolar gain = 8 */
+ UNI_RANGE(10), /* unipolar gain = 1 */
+ UNI_RANGE(5), /* unipolar gain = 2 */
+ UNI_RANGE(2.5), /* unipolar gain = 4 */
+ UNI_RANGE(1.25) /* unipolar gain = 8 */
}
};
+/*
+ * The waveform analog outputs on the 'ao' boards are not currently
+ * supported. They have a comedi_lrange of:
+ * { 2, { BIP_RANGE(10), BIP_RANGE(5) } }
+ */
+
+enum das1800_boardid {
+ BOARD_DAS1701ST,
+ BOARD_DAS1701ST_DA,
+ BOARD_DAS1702ST,
+ BOARD_DAS1702ST_DA,
+ BOARD_DAS1702HR,
+ BOARD_DAS1702HR_DA,
+ BOARD_DAS1701AO,
+ BOARD_DAS1702AO,
+ BOARD_DAS1801ST,
+ BOARD_DAS1801ST_DA,
+ BOARD_DAS1802ST,
+ BOARD_DAS1802ST_DA,
+ BOARD_DAS1802HR,
+ BOARD_DAS1802HR_DA,
+ BOARD_DAS1801HC,
+ BOARD_DAS1802HC,
+ BOARD_DAS1801AO,
+ BOARD_DAS1802AO
+};
+
+/* board probe id values (hi byte of the digital input register) */
+#define DAS1800_ID_ST_DA 0x3
+#define DAS1800_ID_HR_DA 0x4
+#define DAS1800_ID_AO 0x5
+#define DAS1800_ID_HR 0x6
+#define DAS1800_ID_ST 0x7
+#define DAS1800_ID_HC 0x8
+
struct das1800_board {
const char *name;
- int ai_speed; /* max conversion period in nanoseconds */
- int resolution; /* bits of ai resolution */
- int qram_len; /* length of card's channel / gain queue */
- int common; /* supports AREF_COMMON flag */
- int do_n_chan; /* number of digital output channels */
- int ao_ability; /* 0 == no analog out, 1 == basic analog out, 2 == waveform analog out */
- int ao_n_chan; /* number of analog out channels */
- const struct comedi_lrange *range_ai; /* available input ranges */
+ unsigned char id;
+ unsigned int ai_speed;
+ unsigned int is_01_series:1;
};
-/* Warning: the maximum conversion speeds listed below are
- * not always achievable depending on board setup (see
- * user manual.)
- */
static const struct das1800_board das1800_boards[] = {
- {
- .name = "das-1701st",
- .ai_speed = 6250,
- .resolution = 12,
- .qram_len = 256,
- .common = 1,
- .do_n_chan = 4,
- .ao_ability = 0,
- .ao_n_chan = 0,
- .range_ai = &range_ai_das1801,
- },
- {
- .name = "das-1701st-da",
- .ai_speed = 6250,
- .resolution = 12,
- .qram_len = 256,
- .common = 1,
- .do_n_chan = 4,
- .ao_ability = 1,
- .ao_n_chan = 4,
- .range_ai = &range_ai_das1801,
- },
- {
- .name = "das-1702st",
- .ai_speed = 6250,
- .resolution = 12,
- .qram_len = 256,
- .common = 1,
- .do_n_chan = 4,
- .ao_ability = 0,
- .ao_n_chan = 0,
- .range_ai = &range_ai_das1802,
- },
- {
- .name = "das-1702st-da",
- .ai_speed = 6250,
- .resolution = 12,
- .qram_len = 256,
- .common = 1,
- .do_n_chan = 4,
- .ao_ability = 1,
- .ao_n_chan = 4,
- .range_ai = &range_ai_das1802,
- },
- {
- .name = "das-1702hr",
- .ai_speed = 20000,
- .resolution = 16,
- .qram_len = 256,
- .common = 1,
- .do_n_chan = 4,
- .ao_ability = 0,
- .ao_n_chan = 0,
- .range_ai = &range_ai_das1802,
- },
- {
- .name = "das-1702hr-da",
- .ai_speed = 20000,
- .resolution = 16,
- .qram_len = 256,
- .common = 1,
- .do_n_chan = 4,
- .ao_ability = 1,
- .ao_n_chan = 2,
- .range_ai = &range_ai_das1802,
- },
- {
- .name = "das-1701ao",
- .ai_speed = 6250,
- .resolution = 12,
- .qram_len = 256,
- .common = 1,
- .do_n_chan = 4,
- .ao_ability = 2,
- .ao_n_chan = 2,
- .range_ai = &range_ai_das1801,
- },
- {
- .name = "das-1702ao",
- .ai_speed = 6250,
- .resolution = 12,
- .qram_len = 256,
- .common = 1,
- .do_n_chan = 4,
- .ao_ability = 2,
- .ao_n_chan = 2,
- .range_ai = &range_ai_das1802,
- },
- {
- .name = "das-1801st",
- .ai_speed = 3000,
- .resolution = 12,
- .qram_len = 256,
- .common = 1,
- .do_n_chan = 4,
- .ao_ability = 0,
- .ao_n_chan = 0,
- .range_ai = &range_ai_das1801,
- },
- {
- .name = "das-1801st-da",
- .ai_speed = 3000,
- .resolution = 12,
- .qram_len = 256,
- .common = 1,
- .do_n_chan = 4,
- .ao_ability = 0,
- .ao_n_chan = 4,
- .range_ai = &range_ai_das1801,
- },
- {
- .name = "das-1802st",
- .ai_speed = 3000,
- .resolution = 12,
- .qram_len = 256,
- .common = 1,
- .do_n_chan = 4,
- .ao_ability = 0,
- .ao_n_chan = 0,
- .range_ai = &range_ai_das1802,
- },
- {
- .name = "das-1802st-da",
- .ai_speed = 3000,
- .resolution = 12,
- .qram_len = 256,
- .common = 1,
- .do_n_chan = 4,
- .ao_ability = 1,
- .ao_n_chan = 4,
- .range_ai = &range_ai_das1802,
- },
- {
- .name = "das-1802hr",
- .ai_speed = 10000,
- .resolution = 16,
- .qram_len = 256,
- .common = 1,
- .do_n_chan = 4,
- .ao_ability = 0,
- .ao_n_chan = 0,
- .range_ai = &range_ai_das1802,
- },
- {
- .name = "das-1802hr-da",
- .ai_speed = 10000,
- .resolution = 16,
- .qram_len = 256,
- .common = 1,
- .do_n_chan = 4,
- .ao_ability = 1,
- .ao_n_chan = 2,
- .range_ai = &range_ai_das1802,
- },
- {
- .name = "das-1801hc",
- .ai_speed = 3000,
- .resolution = 12,
- .qram_len = 64,
- .common = 0,
- .do_n_chan = 8,
- .ao_ability = 1,
- .ao_n_chan = 2,
- .range_ai = &range_ai_das1801,
- },
- {
- .name = "das-1802hc",
- .ai_speed = 3000,
- .resolution = 12,
- .qram_len = 64,
- .common = 0,
- .do_n_chan = 8,
- .ao_ability = 1,
- .ao_n_chan = 2,
- .range_ai = &range_ai_das1802,
- },
- {
- .name = "das-1801ao",
- .ai_speed = 3000,
- .resolution = 12,
- .qram_len = 256,
- .common = 1,
- .do_n_chan = 4,
- .ao_ability = 2,
- .ao_n_chan = 2,
- .range_ai = &range_ai_das1801,
- },
- {
- .name = "das-1802ao",
- .ai_speed = 3000,
- .resolution = 12,
- .qram_len = 256,
- .common = 1,
- .do_n_chan = 4,
- .ao_ability = 2,
- .ao_n_chan = 2,
- .range_ai = &range_ai_das1802,
- },
+ [BOARD_DAS1701ST] = {
+ .name = "das-1701st",
+ .id = DAS1800_ID_ST,
+ .ai_speed = 6250,
+ .is_01_series = 1,
+ },
+ [BOARD_DAS1701ST_DA] = {
+ .name = "das-1701st-da",
+ .id = DAS1800_ID_ST_DA,
+ .ai_speed = 6250,
+ .is_01_series = 1,
+ },
+ [BOARD_DAS1702ST] = {
+ .name = "das-1702st",
+ .id = DAS1800_ID_ST,
+ .ai_speed = 6250,
+ },
+ [BOARD_DAS1702ST_DA] = {
+ .name = "das-1702st-da",
+ .id = DAS1800_ID_ST_DA,
+ .ai_speed = 6250,
+ },
+ [BOARD_DAS1702HR] = {
+ .name = "das-1702hr",
+ .id = DAS1800_ID_HR,
+ .ai_speed = 20000,
+ },
+ [BOARD_DAS1702HR_DA] = {
+ .name = "das-1702hr-da",
+ .id = DAS1800_ID_HR_DA,
+ .ai_speed = 20000,
+ },
+ [BOARD_DAS1701AO] = {
+ .name = "das-1701ao",
+ .id = DAS1800_ID_AO,
+ .ai_speed = 6250,
+ .is_01_series = 1,
+ },
+ [BOARD_DAS1702AO] = {
+ .name = "das-1702ao",
+ .id = DAS1800_ID_AO,
+ .ai_speed = 6250,
+ },
+ [BOARD_DAS1801ST] = {
+ .name = "das-1801st",
+ .id = DAS1800_ID_ST,
+ .ai_speed = 3000,
+ .is_01_series = 1,
+ },
+ [BOARD_DAS1801ST_DA] = {
+ .name = "das-1801st-da",
+ .id = DAS1800_ID_ST_DA,
+ .ai_speed = 3000,
+ .is_01_series = 1,
+ },
+ [BOARD_DAS1802ST] = {
+ .name = "das-1802st",
+ .id = DAS1800_ID_ST,
+ .ai_speed = 3000,
+ },
+ [BOARD_DAS1802ST_DA] = {
+ .name = "das-1802st-da",
+ .id = DAS1800_ID_ST_DA,
+ .ai_speed = 3000,
+ },
+ [BOARD_DAS1802HR] = {
+ .name = "das-1802hr",
+ .id = DAS1800_ID_HR,
+ .ai_speed = 10000,
+ },
+ [BOARD_DAS1802HR_DA] = {
+ .name = "das-1802hr-da",
+ .id = DAS1800_ID_HR_DA,
+ .ai_speed = 10000,
+ },
+ [BOARD_DAS1801HC] = {
+ .name = "das-1801hc",
+ .id = DAS1800_ID_HC,
+ .ai_speed = 3000,
+ .is_01_series = 1,
+ },
+ [BOARD_DAS1802HC] = {
+ .name = "das-1802hc",
+ .id = DAS1800_ID_HC,
+ .ai_speed = 3000,
+ },
+ [BOARD_DAS1801AO] = {
+ .name = "das-1801ao",
+ .id = DAS1800_ID_AO,
+ .ai_speed = 3000,
+ .is_01_series = 1,
+ },
+ [BOARD_DAS1802AO] = {
+ .name = "das-1802ao",
+ .id = DAS1800_ID_AO,
+ .ai_speed = 3000,
+ },
};
struct das1800_private {
struct comedi_isadma *dma;
- int irq_dma_bits; /* bits for control register b */
- /* dma bits for control register b, stored so that dma can be
- * turned on and off */
+ int irq_dma_bits;
int dma_bits;
- uint16_t *fifo_buf; /* bounce buffer for analog input FIFO */
- unsigned long iobase2; /* secondary io address used for analog out on 'ao' boards */
- unsigned short ao_update_bits; /* remembers the last write to the
- * 'update' dac */
+ unsigned short *fifo_buf;
+ unsigned long iobase2;
+ bool ai_is_unipolar;
};
-/* analog out range for 'ao' boards */
-/*
-static const struct comedi_lrange range_ao_2 = {
- 2, {
- BIP_RANGE(10),
- BIP_RANGE(5)
- }
-};
-*/
-
-static inline uint16_t munge_bipolar_sample(const struct comedi_device *dev,
- uint16_t sample)
+static void das1800_ai_munge(struct comedi_device *dev,
+ struct comedi_subdevice *s,
+ void *data, unsigned int num_bytes,
+ unsigned int start_chan_index)
{
- const struct das1800_board *board = dev->board_ptr;
-
- sample += 1 << (board->resolution - 1);
- return sample;
-}
-
-static void munge_data(struct comedi_device *dev, uint16_t *array,
- unsigned int num_elements)
-{
+ struct das1800_private *devpriv = dev->private;
+ unsigned short *array = data;
+ unsigned int num_samples = comedi_bytes_to_samples(s, num_bytes);
unsigned int i;
- int unipolar;
- /* see if card is using a unipolar or bipolar range so we can munge data correctly */
- unipolar = inb(dev->iobase + DAS1800_CONTROL_C) & UB;
+ if (devpriv->ai_is_unipolar)
+ return;
- /* convert to unsigned type if we are in a bipolar mode */
- if (!unipolar) {
- for (i = 0; i < num_elements; i++)
- array[i] = munge_bipolar_sample(dev, array[i]);
- }
+ for (i = 0; i < num_samples; i++)
+ array[i] = comedi_offset_munge(s, array[i]);
}
static void das1800_handle_fifo_half_full(struct comedi_device *dev,
@@ -473,7 +352,6 @@
unsigned int nsamples = comedi_nsamples_left(s, FIFO_SIZE / 2);
insw(dev->iobase + DAS1800_FIFO, devpriv->fifo_buf, nsamples);
- munge_data(dev, devpriv->fifo_buf, nsamples);
comedi_buf_write_samples(s, devpriv->fifo_buf, nsamples);
}
@@ -482,14 +360,9 @@
{
struct comedi_cmd *cmd = &s->async->cmd;
unsigned short dpnt;
- int unipolar;
-
- unipolar = inb(dev->iobase + DAS1800_CONTROL_C) & UB;
while (inb(dev->iobase + DAS1800_STATUS) & FNE) {
dpnt = inw(dev->iobase + DAS1800_FIFO);
- /* convert to unsigned type */
- dpnt = munge_bipolar_sample(dev, dpnt);
comedi_buf_write_samples(s, &dpnt, 1);
if (cmd->stop_src == TRIG_COUNT &&
@@ -498,7 +371,6 @@
}
}
-/* Utility function used by das1800_flush_dma() and das1800_handle_dma() */
static void das1800_flush_dma_channel(struct comedi_device *dev,
struct comedi_subdevice *s,
struct comedi_isadma_desc *desc)
@@ -511,12 +383,9 @@
nsamples = comedi_bytes_to_samples(s, nbytes);
nsamples = comedi_nsamples_left(s, nsamples);
- munge_data(dev, desc->virt_addr, nsamples);
comedi_buf_write_samples(s, desc->virt_addr, nsamples);
}
-/* flushes remaining data from board when external trigger has stopped acquisition
- * and we are using dma transfers */
static void das1800_flush_dma(struct comedi_device *dev,
struct comedi_subdevice *s)
{
@@ -560,27 +429,30 @@
}
}
-static int das1800_cancel(struct comedi_device *dev, struct comedi_subdevice *s)
+static int das1800_ai_cancel(struct comedi_device *dev,
+ struct comedi_subdevice *s)
{
struct das1800_private *devpriv = dev->private;
struct comedi_isadma *dma = devpriv->dma;
struct comedi_isadma_desc *desc;
int i;
- outb(0x0, dev->iobase + DAS1800_STATUS); /* disable conversions */
- outb(0x0, dev->iobase + DAS1800_CONTROL_B); /* disable interrupts and dma */
- outb(0x0, dev->iobase + DAS1800_CONTROL_A); /* disable and clear fifo and stop triggering */
+ /* disable and stop conversions */
+ outb(0x0, dev->iobase + DAS1800_STATUS);
+ outb(0x0, dev->iobase + DAS1800_CONTROL_B);
+ outb(0x0, dev->iobase + DAS1800_CONTROL_A);
- for (i = 0; i < 2; i++) {
- desc = &dma->desc[i];
- if (desc->chan)
- comedi_isadma_disable(desc->chan);
+ if (dma) {
+ for (i = 0; i < 2; i++) {
+ desc = &dma->desc[i];
+ if (desc->chan)
+ comedi_isadma_disable(desc->chan);
+ }
}
return 0;
}
-/* the guts of the interrupt handler, that is shared with das1800_ai_poll */
static void das1800_ai_handler(struct comedi_device *dev)
{
struct das1800_private *devpriv = dev->private;
@@ -589,17 +461,16 @@
struct comedi_cmd *cmd = &async->cmd;
unsigned int status = inb(dev->iobase + DAS1800_STATUS);
- /* select adc for base address + 0 */
+ /* select adc register (spinlock is already held) */
outb(ADC, dev->iobase + DAS1800_SELECT);
- /* dma buffer full */
- if (devpriv->irq_dma_bits & DMA_ENABLED) {
- /* look for data from dma transfer even if dma terminal count hasn't happened yet */
+
+ /* get samples with dma, fifo, or polled as necessary */
+ if (devpriv->irq_dma_bits & DMA_ENABLED)
das1800_handle_dma(dev, s, status);
- } else if (status & FHF) { /* if fifo half full */
+ else if (status & FHF)
das1800_handle_fifo_half_full(dev, s);
- } else if (status & FNE) { /* if fifo not empty */
+ else if (status & FNE)
das1800_handle_fifo_not_empty(dev, s);
- }
/* if the card's fifo has overflowed */
if (status & OVF) {
@@ -615,7 +486,7 @@
if (status & CT0TC) {
/* clear CT0TC interrupt bit */
outb(CLEAR_INTR_MASK & ~CT0TC, dev->iobase + DAS1800_STATUS);
- /* make sure we get all remaining data from board before quitting */
+ /* get all remaining samples before quitting */
if (devpriv->irq_dma_bits & DMA_ENABLED)
das1800_flush_dma(dev, s);
else
@@ -634,9 +505,14 @@
{
unsigned long flags;
- /* prevent race with interrupt handler */
+ /*
+ * Protects the indirect addressing selected by DAS1800_SELECT
+ * in das1800_ai_handler() also prevents race with das1800_interrupt().
+ */
spin_lock_irqsave(&dev->spinlock, flags);
+
das1800_ai_handler(dev);
+
spin_unlock_irqrestore(&dev->spinlock, flags);
return comedi_buf_n_bytes_ready(s);
@@ -652,9 +528,12 @@
return IRQ_HANDLED;
}
- /* Prevent race with das1800_ai_poll() on multi processor systems.
- * Also protects indirect addressing in das1800_ai_handler */
+ /*
+ * Protects the indirect addressing selected by DAS1800_SELECT
+ * in das1800_ai_handler() also prevents race with das1800_ai_poll().
+ */
spin_lock(&dev->spinlock);
+
status = inb(dev->iobase + DAS1800_STATUS);
/* if interrupt was not caused by das-1800 */
@@ -671,46 +550,87 @@
return IRQ_HANDLED;
}
-/* converts requested conversion timing to timing compatible with
- * hardware, used only when card is in 'burst mode'
- */
-static unsigned int burst_convert_arg(unsigned int convert_arg, int flags)
+static int das1800_ai_fixup_paced_timing(struct comedi_device *dev,
+ struct comedi_cmd *cmd)
{
- unsigned int micro_sec;
+ unsigned int arg = cmd->convert_arg;
- /* in burst mode, the maximum conversion time is 64 microseconds */
- if (convert_arg > 64000)
- convert_arg = 64000;
+ /*
+ * Paced mode:
+ * scan_begin_src is TRIG_FOLLOW
+ * convert_src is TRIG_TIMER
+ *
+ * The convert_arg sets the pacer sample acquisition time.
+ * The max acquisition speed is limited to the boards
+ * 'ai_speed' (this was already verified). The min speed is
+ * limited by the cascaded 8254 timer.
+ */
+ comedi_8254_cascade_ns_to_timer(dev->pacer, &arg, cmd->flags);
+ return comedi_check_trigger_arg_is(&cmd->convert_arg, arg);
+}
- /* the conversion time must be an integral number of microseconds */
- switch (flags & CMDF_ROUND_MASK) {
+static int das1800_ai_fixup_burst_timing(struct comedi_device *dev,
+ struct comedi_cmd *cmd)
+{
+ unsigned int arg = cmd->convert_arg;
+ int err = 0;
+
+ /*
+ * Burst mode:
+ * scan_begin_src is TRIG_TIMER or TRIG_EXT
+ * convert_src is TRIG_TIMER
+ *
+ * The convert_arg sets burst sample acquisition time.
+ * The max acquisition speed is limited to the boards
+ * 'ai_speed' (this was already verified). The min speed is
+ * limiited to 64 microseconds,
+ */
+ err |= comedi_check_trigger_arg_max(&arg, 64000);
+
+ /* round to microseconds then verify */
+ switch (cmd->flags & CMDF_ROUND_MASK) {
case CMDF_ROUND_NEAREST:
default:
- micro_sec = (convert_arg + 500) / 1000;
+ arg = DIV_ROUND_CLOSEST(arg, 1000);
break;
case CMDF_ROUND_DOWN:
- micro_sec = convert_arg / 1000;
+ arg = arg / 1000;
break;
case CMDF_ROUND_UP:
- micro_sec = (convert_arg - 1) / 1000 + 1;
+ arg = DIV_ROUND_UP(arg, 1000);
break;
}
+ err |= comedi_check_trigger_arg_is(&cmd->convert_arg, arg * 1000);
- /* return number of nanoseconds */
- return micro_sec * 1000;
+ /*
+ * The pacer can be used to set the scan sample rate. The max scan
+ * speed is limited by the conversion speed and the number of channels
+ * to convert. The min speed is limited by the cascaded 8254 timer.
+ */
+ if (cmd->scan_begin_src == TRIG_TIMER) {
+ arg = cmd->convert_arg * cmd->chanlist_len;
+ err |= comedi_check_trigger_arg_min(&cmd->scan_begin_arg, arg);
+
+ arg = cmd->scan_begin_arg;
+ comedi_8254_cascade_ns_to_timer(dev->pacer, &arg, cmd->flags);
+ err |= comedi_check_trigger_arg_is(&cmd->scan_begin_arg, arg);
+ }
+
+ return err;
}
static int das1800_ai_check_chanlist(struct comedi_device *dev,
struct comedi_subdevice *s,
struct comedi_cmd *cmd)
{
- unsigned int unipolar0 = CR_RANGE(cmd->chanlist[0]) & UNIPOLAR;
+ unsigned int range = CR_RANGE(cmd->chanlist[0]);
+ bool unipolar0 = comedi_range_is_unipolar(s, range);
int i;
for (i = 1; i < cmd->chanlist_len; i++) {
- unsigned int unipolar = CR_RANGE(cmd->chanlist[i]) & UNIPOLAR;
+ range = CR_RANGE(cmd->chanlist[i]);
- if (unipolar != unipolar0) {
+ if (unipolar0 != comedi_range_is_unipolar(s, range)) {
dev_dbg(dev->class_dev,
"unipolar and bipolar ranges cannot be mixed in the chanlist\n");
return -EINVAL;
@@ -720,14 +640,12 @@
return 0;
}
-/* test analog input cmd */
-static int das1800_ai_do_cmdtest(struct comedi_device *dev,
- struct comedi_subdevice *s,
- struct comedi_cmd *cmd)
+static int das1800_ai_cmdtest(struct comedi_device *dev,
+ struct comedi_subdevice *s,
+ struct comedi_cmd *cmd)
{
const struct das1800_board *board = dev->board_ptr;
int err = 0;
- unsigned int arg;
/* Step 1 : check if triggers are trivially valid */
@@ -752,16 +670,23 @@
/* Step 2b : and mutually compatible */
+ /* burst scans must use timed conversions */
if (cmd->scan_begin_src != TRIG_FOLLOW &&
cmd->convert_src != TRIG_TIMER)
err |= -EINVAL;
+ /* the external pin TGIN must use the same polarity */
+ if (cmd->start_src == TRIG_EXT && cmd->stop_src == TRIG_EXT)
+ err |= comedi_check_trigger_arg_is(&cmd->start_arg,
+ cmd->stop_arg);
+
if (err)
return 2;
/* Step 3: check if arguments are trivially valid */
- err |= comedi_check_trigger_arg_is(&cmd->start_arg, 0);
+ if (cmd->start_arg == TRIG_NOW)
+ err |= comedi_check_trigger_arg_is(&cmd->start_arg, 0);
if (cmd->convert_src == TRIG_TIMER) {
err |= comedi_check_trigger_arg_min(&cmd->convert_arg,
@@ -786,31 +711,13 @@
if (err)
return 3;
- /* step 4: fix up any arguments */
+ /* Step 4: fix up any arguments */
- if (cmd->scan_begin_src == TRIG_FOLLOW &&
- cmd->convert_src == TRIG_TIMER) {
- /* we are not in burst mode */
- arg = cmd->convert_arg;
- comedi_8254_cascade_ns_to_timer(dev->pacer, &arg, cmd->flags);
- err |= comedi_check_trigger_arg_is(&cmd->convert_arg, arg);
- } else if (cmd->convert_src == TRIG_TIMER) {
- /* we are in burst mode */
- arg = burst_convert_arg(cmd->convert_arg, cmd->flags);
- err |= comedi_check_trigger_arg_is(&cmd->convert_arg, arg);
-
- if (cmd->scan_begin_src == TRIG_TIMER) {
- arg = cmd->convert_arg * cmd->chanlist_len;
- err |= comedi_check_trigger_arg_max(&cmd->
- scan_begin_arg,
- arg);
-
- arg = cmd->scan_begin_arg;
- comedi_8254_cascade_ns_to_timer(dev->pacer, &arg,
- cmd->flags);
- err |= comedi_check_trigger_arg_is(&cmd->scan_begin_arg,
- arg);
- }
+ if (cmd->convert_src == TRIG_TIMER) {
+ if (cmd->scan_begin_src == TRIG_FOLLOW)
+ err |= das1800_ai_fixup_paced_timing(dev, cmd);
+ else /* TRIG_TIMER or TRIG_EXT */
+ err |= das1800_ai_fixup_burst_timing(dev, cmd);
}
if (err)
@@ -826,74 +733,22 @@
return 0;
}
-/* returns appropriate bits for control register a, depending on command */
-static int control_a_bits(const struct comedi_cmd *cmd)
+static unsigned char das1800_ai_chanspec_bits(struct comedi_subdevice *s,
+ unsigned int chanspec)
{
- int control_a;
+ unsigned int range = CR_RANGE(chanspec);
+ unsigned int aref = CR_AREF(chanspec);
+ unsigned char bits;
- control_a = FFEN; /* enable fifo */
- if (cmd->stop_src == TRIG_EXT)
- control_a |= ATEN;
- switch (cmd->start_src) {
- case TRIG_EXT:
- control_a |= TGEN | CGSL;
- break;
- case TRIG_NOW:
- control_a |= CGEN;
- break;
- default:
- break;
- }
-
- return control_a;
-}
-
-/* returns appropriate bits for control register c, depending on command */
-static int control_c_bits(const struct comedi_cmd *cmd)
-{
- int control_c;
- int aref;
-
- /* set clock source to internal or external, select analog reference,
- * select unipolar / bipolar
- */
- aref = CR_AREF(cmd->chanlist[0]);
- control_c = UQEN; /* enable upper qram addresses */
+ bits = UQEN;
if (aref != AREF_DIFF)
- control_c |= SD;
+ bits |= SD;
if (aref == AREF_COMMON)
- control_c |= CMEN;
- /* if a unipolar range was selected */
- if (CR_RANGE(cmd->chanlist[0]) & UNIPOLAR)
- control_c |= UB;
- switch (cmd->scan_begin_src) {
- case TRIG_FOLLOW: /* not in burst mode */
- switch (cmd->convert_src) {
- case TRIG_TIMER:
- /* trig on cascaded counters */
- control_c |= IPCLK;
- break;
- case TRIG_EXT:
- /* trig on falling edge of external trigger */
- control_c |= XPCLK;
- break;
- default:
- break;
- }
- break;
- case TRIG_TIMER:
- /* burst mode with internal pacer clock */
- control_c |= BMDE | IPCLK;
- break;
- case TRIG_EXT:
- /* burst mode with external trigger */
- control_c |= BMDE | XPCLK;
- break;
- default:
- break;
- }
+ bits |= CMEN;
+ if (comedi_range_is_unipolar(s, range))
+ bits |= UB;
- return control_c;
+ return bits;
}
static unsigned int das1800_ai_transfer_size(struct comedi_device *dev,
@@ -934,13 +789,14 @@
{
struct das1800_private *devpriv = dev->private;
struct comedi_isadma *dma = devpriv->dma;
- struct comedi_isadma_desc *desc = &dma->desc[0];
+ struct comedi_isadma_desc *desc;
unsigned int bytes;
if ((devpriv->irq_dma_bits & DMA_ENABLED) == 0)
return;
dma->cur_dma = 0;
+ desc = &dma->desc[0];
/* determine a dma transfer size to fill buffer in 0.3 sec */
bytes = das1800_ai_transfer_size(dev, s, desc->maxsize, 300000000);
@@ -956,43 +812,48 @@
}
}
-/* programs channel/gain list into card */
-static void program_chanlist(struct comedi_device *dev,
- const struct comedi_cmd *cmd)
+static void das1800_ai_set_chanlist(struct comedi_device *dev,
+ unsigned int *chanlist, unsigned int len)
{
- int i, n, chan_range;
- unsigned long irq_flags;
- const int range_mask = 0x3; /* masks unipolar/bipolar bit off range */
- const int range_bitshift = 8;
+ unsigned long flags;
+ unsigned int i;
- n = cmd->chanlist_len;
- /* spinlock protects indirect addressing */
- spin_lock_irqsave(&dev->spinlock, irq_flags);
- outb(QRAM, dev->iobase + DAS1800_SELECT); /* select QRAM for baseAddress + 0x0 */
- outb(n - 1, dev->iobase + DAS1800_QRAM_ADDRESS); /*set QRAM address start */
+ /* protects the indirect addressing selected by DAS1800_SELECT */
+ spin_lock_irqsave(&dev->spinlock, flags);
+
+ /* select QRAM register and set start address */
+ outb(QRAM, dev->iobase + DAS1800_SELECT);
+ outb(len - 1, dev->iobase + DAS1800_QRAM_ADDRESS);
+
/* make channel / gain list */
- for (i = 0; i < n; i++) {
- chan_range =
- CR_CHAN(cmd->chanlist[i]) |
- ((CR_RANGE(cmd->chanlist[i]) & range_mask) <<
- range_bitshift);
- outw(chan_range, dev->iobase + DAS1800_QRAM);
+ for (i = 0; i < len; i++) {
+ unsigned int chan = CR_CHAN(chanlist[i]);
+ unsigned int range = CR_RANGE(chanlist[i]);
+ unsigned short val;
+
+ val = chan | ((range & 0x3) << 8);
+ outw(val, dev->iobase + DAS1800_QRAM);
}
- outb(n - 1, dev->iobase + DAS1800_QRAM_ADDRESS); /*finish write to QRAM */
- spin_unlock_irqrestore(&dev->spinlock, irq_flags);
+
+ /* finish write to QRAM */
+ outb(len - 1, dev->iobase + DAS1800_QRAM_ADDRESS);
+
+ spin_unlock_irqrestore(&dev->spinlock, flags);
}
-/* analog input do_cmd */
-static int das1800_ai_do_cmd(struct comedi_device *dev,
- struct comedi_subdevice *s)
+static int das1800_ai_cmd(struct comedi_device *dev,
+ struct comedi_subdevice *s)
{
struct das1800_private *devpriv = dev->private;
int control_a, control_c;
struct comedi_async *async = s->async;
const struct comedi_cmd *cmd = &async->cmd;
+ unsigned int range0 = CR_RANGE(cmd->chanlist[0]);
- /* disable dma on CMDF_WAKE_EOS, or CMDF_PRIORITY
- * (because dma in handler is unsafe at hard real-time priority) */
+ /*
+ * Disable dma on CMDF_WAKE_EOS, or CMDF_PRIORITY (because dma in
+ * handler is unsafe at hard real-time priority).
+ */
if (cmd->flags & (CMDF_WAKE_EOS | CMDF_PRIORITY))
devpriv->irq_dma_bits &= ~DMA_ENABLED;
else
@@ -1006,14 +867,42 @@
devpriv->irq_dma_bits |= FIMD;
}
- das1800_cancel(dev, s);
+ das1800_ai_cancel(dev, s);
- /* determine proper bits for control registers */
- control_a = control_a_bits(cmd);
- control_c = control_c_bits(cmd);
+ devpriv->ai_is_unipolar = comedi_range_is_unipolar(s, range0);
- /* setup card and start */
- program_chanlist(dev, cmd);
+ control_a = FFEN;
+ if (cmd->stop_src == TRIG_EXT)
+ control_a |= ATEN;
+ if (cmd->start_src == TRIG_EXT)
+ control_a |= TGEN | CGSL;
+ else /* TRIG_NOW */
+ control_a |= CGEN;
+ if (control_a & (ATEN | TGEN)) {
+ if ((cmd->start_arg & CR_INVERT) || (cmd->stop_arg & CR_INVERT))
+ control_a |= TGPL;
+ }
+
+ control_c = das1800_ai_chanspec_bits(s, cmd->chanlist[0]);
+ /* set clock source to internal or external */
+ if (cmd->scan_begin_src == TRIG_FOLLOW) {
+ /* not in burst mode */
+ if (cmd->convert_src == TRIG_TIMER) {
+ /* trig on cascaded counters */
+ control_c |= IPCLK;
+ } else { /* TRIG_EXT */
+ /* trig on falling edge of external trigger */
+ control_c |= XPCLK;
+ }
+ } else if (cmd->scan_begin_src == TRIG_TIMER) {
+ /* burst mode with internal pacer clock */
+ control_c |= BMDE | IPCLK;
+ } else { /* TRIG_EXT */
+ /* burst mode with external trigger */
+ control_c |= BMDE | XPCLK;
+ }
+
+ das1800_ai_set_chanlist(dev, cmd->chanlist, cmd->chanlist_len);
/* setup cascaded counters for conversion/scan frequency */
if ((cmd->scan_begin_src == TRIG_FOLLOW ||
@@ -1031,118 +920,117 @@
outb(control_c, dev->iobase + DAS1800_CONTROL_C);
/* set conversion rate and length for burst mode */
if (control_c & BMDE) {
- /* program conversion period with number of microseconds minus 1 */
- outb(cmd->convert_arg / 1000 - 1,
+ outb(cmd->convert_arg / 1000 - 1, /* microseconds - 1 */
dev->iobase + DAS1800_BURST_RATE);
outb(cmd->chanlist_len - 1, dev->iobase + DAS1800_BURST_LENGTH);
}
- outb(devpriv->irq_dma_bits, dev->iobase + DAS1800_CONTROL_B); /* enable irq/dma */
- outb(control_a, dev->iobase + DAS1800_CONTROL_A); /* enable fifo and triggering */
- outb(CVEN, dev->iobase + DAS1800_STATUS); /* enable conversions */
+
+ /* enable and start conversions */
+ outb(devpriv->irq_dma_bits, dev->iobase + DAS1800_CONTROL_B);
+ outb(control_a, dev->iobase + DAS1800_CONTROL_A);
+ outb(CVEN, dev->iobase + DAS1800_STATUS);
return 0;
}
-/* read analog input */
-static int das1800_ai_rinsn(struct comedi_device *dev,
- struct comedi_subdevice *s,
- struct comedi_insn *insn, unsigned int *data)
+static int das1800_ai_eoc(struct comedi_device *dev,
+ struct comedi_subdevice *s,
+ struct comedi_insn *insn,
+ unsigned long context)
{
- const struct das1800_board *board = dev->board_ptr;
- int i, n;
- int chan, range, aref, chan_range;
- int timeout = 1000;
+ unsigned char status;
+
+ status = inb(dev->iobase + DAS1800_STATUS);
+ if (status & FNE)
+ return 0;
+ return -EBUSY;
+}
+
+static int das1800_ai_insn_read(struct comedi_device *dev,
+ struct comedi_subdevice *s,
+ struct comedi_insn *insn,
+ unsigned int *data)
+{
+ unsigned int range = CR_RANGE(insn->chanspec);
+ bool is_unipolar = comedi_range_is_unipolar(s, range);
+ int ret = 0;
+ int n;
unsigned short dpnt;
- int conv_flags = 0;
- unsigned long irq_flags;
+ unsigned long flags;
- /* set up analog reference and unipolar / bipolar mode */
- aref = CR_AREF(insn->chanspec);
- conv_flags |= UQEN;
- if (aref != AREF_DIFF)
- conv_flags |= SD;
- if (aref == AREF_COMMON)
- conv_flags |= CMEN;
- /* if a unipolar range was selected */
- if (CR_RANGE(insn->chanspec) & UNIPOLAR)
- conv_flags |= UB;
-
- outb(conv_flags, dev->iobase + DAS1800_CONTROL_C); /* software conversion enabled */
+ outb(das1800_ai_chanspec_bits(s, insn->chanspec),
+ dev->iobase + DAS1800_CONTROL_C); /* software pacer */
outb(CVEN, dev->iobase + DAS1800_STATUS); /* enable conversions */
outb(0x0, dev->iobase + DAS1800_CONTROL_A); /* reset fifo */
outb(FFEN, dev->iobase + DAS1800_CONTROL_A);
- chan = CR_CHAN(insn->chanspec);
- /* mask of unipolar/bipolar bit from range */
- range = CR_RANGE(insn->chanspec) & 0x3;
- chan_range = chan | (range << 8);
- spin_lock_irqsave(&dev->spinlock, irq_flags);
- outb(QRAM, dev->iobase + DAS1800_SELECT); /* select QRAM for baseAddress + 0x0 */
- outb(0x0, dev->iobase + DAS1800_QRAM_ADDRESS); /* set QRAM address start */
- outw(chan_range, dev->iobase + DAS1800_QRAM);
- outb(0x0, dev->iobase + DAS1800_QRAM_ADDRESS); /*finish write to QRAM */
- outb(ADC, dev->iobase + DAS1800_SELECT); /* select ADC for baseAddress + 0x0 */
+ das1800_ai_set_chanlist(dev, &insn->chanspec, 1);
+
+ /* protects the indirect addressing selected by DAS1800_SELECT */
+ spin_lock_irqsave(&dev->spinlock, flags);
+
+ /* select ai fifo register */
+ outb(ADC, dev->iobase + DAS1800_SELECT);
for (n = 0; n < insn->n; n++) {
/* trigger conversion */
outb(0, dev->iobase + DAS1800_FIFO);
- for (i = 0; i < timeout; i++) {
- if (inb(dev->iobase + DAS1800_STATUS) & FNE)
- break;
- }
- if (i == timeout) {
- dev_err(dev->class_dev, "timeout\n");
- n = -ETIME;
- goto exit;
- }
+
+ ret = comedi_timeout(dev, s, insn, das1800_ai_eoc, 0);
+ if (ret)
+ break;
+
dpnt = inw(dev->iobase + DAS1800_FIFO);
- /* shift data to offset binary for bipolar ranges */
- if ((conv_flags & UB) == 0)
- dpnt += 1 << (board->resolution - 1);
+ if (!is_unipolar)
+ dpnt = comedi_offset_munge(s, dpnt);
data[n] = dpnt;
}
-exit:
- spin_unlock_irqrestore(&dev->spinlock, irq_flags);
+ spin_unlock_irqrestore(&dev->spinlock, flags);
- return n;
+ return ret ? ret : insn->n;
}
-/* writes to an analog output channel */
-static int das1800_ao_winsn(struct comedi_device *dev,
- struct comedi_subdevice *s,
- struct comedi_insn *insn, unsigned int *data)
+static int das1800_ao_insn_write(struct comedi_device *dev,
+ struct comedi_subdevice *s,
+ struct comedi_insn *insn,
+ unsigned int *data)
{
- const struct das1800_board *board = dev->board_ptr;
- struct das1800_private *devpriv = dev->private;
- int chan = CR_CHAN(insn->chanspec);
-/* int range = CR_RANGE(insn->chanspec); */
- int update_chan = board->ao_n_chan - 1;
- unsigned short output;
- unsigned long irq_flags;
+ unsigned int chan = CR_CHAN(insn->chanspec);
+ unsigned int update_chan = s->n_chan - 1;
+ unsigned long flags;
+ int i;
- /* card expects two's complement data */
- output = data[0] - (1 << (board->resolution - 1));
- /* if the write is to the 'update' channel, we need to remember its value */
- if (chan == update_chan)
- devpriv->ao_update_bits = output;
- /* write to channel */
- spin_lock_irqsave(&dev->spinlock, irq_flags);
- outb(DAC(chan), dev->iobase + DAS1800_SELECT); /* select dac channel for baseAddress + 0x0 */
- outw(output, dev->iobase + DAS1800_DAC);
- /* now we need to write to 'update' channel to update all dac channels */
- if (chan != update_chan) {
- outb(DAC(update_chan), dev->iobase + DAS1800_SELECT); /* select 'update' channel for baseAddress + 0x0 */
- outw(devpriv->ao_update_bits, dev->iobase + DAS1800_DAC);
+ /* protects the indirect addressing selected by DAS1800_SELECT */
+ spin_lock_irqsave(&dev->spinlock, flags);
+
+ for (i = 0; i < insn->n; i++) {
+ unsigned int val = data[i];
+
+ s->readback[chan] = val;
+
+ val = comedi_offset_munge(s, val);
+
+ /* load this channel (and update if it's the last channel) */
+ outb(DAC(chan), dev->iobase + DAS1800_SELECT);
+ outw(val, dev->iobase + DAS1800_DAC);
+
+ /* update all channels */
+ if (chan != update_chan) {
+ val = comedi_offset_munge(s, s->readback[update_chan]);
+
+ outb(DAC(update_chan), dev->iobase + DAS1800_SELECT);
+ outw(val, dev->iobase + DAS1800_DAC);
+ }
}
- spin_unlock_irqrestore(&dev->spinlock, irq_flags);
+ spin_unlock_irqrestore(&dev->spinlock, flags);
- return 1;
+ return insn->n;
}
-/* reads from digital input channels */
-static int das1800_di_rbits(struct comedi_device *dev,
- struct comedi_subdevice *s,
- struct comedi_insn *insn, unsigned int *data)
+static int das1800_di_insn_bits(struct comedi_device *dev,
+ struct comedi_subdevice *s,
+ struct comedi_insn *insn,
+ unsigned int *data)
{
data[1] = inb(dev->iobase + DAS1800_DIGITAL) & 0xf;
data[0] = 0;
@@ -1150,10 +1038,10 @@
return insn->n;
}
-static int das1800_do_wbits(struct comedi_device *dev,
- struct comedi_subdevice *s,
- struct comedi_insn *insn,
- unsigned int *data)
+static int das1800_do_insn_bits(struct comedi_device *dev,
+ struct comedi_subdevice *s,
+ struct comedi_insn *insn,
+ unsigned int *data)
{
if (comedi_dio_update_state(s, data))
outb(s->state, dev->iobase + DAS1800_DIGITAL);
@@ -1216,68 +1104,68 @@
comedi_isadma_free(devpriv->dma);
}
-static const struct das1800_board *das1800_probe(struct comedi_device *dev)
+static int das1800_probe(struct comedi_device *dev)
{
const struct das1800_board *board = dev->board_ptr;
- int index = board ? board - das1800_boards : -EINVAL;
- int id;
+ unsigned char id;
+
+ id = (inb(dev->iobase + DAS1800_DIGITAL) >> 4) & 0xf;
/*
* The dev->board_ptr will be set by comedi_device_attach() if the
* board name provided by the user matches a board->name in this
* driver. If so, this function sanity checks the id to verify that
* the board is correct.
- *
- * If the dev->board_ptr is not set, the user is trying to attach
- * an unspecified board to this driver. In this case the id is used
- * to 'probe' for the correct dev->board_ptr.
*/
- id = (inb(dev->iobase + DAS1800_DIGITAL) >> 4) & 0xf;
+ if (board) {
+ if (board->id == id)
+ return 0;
+ dev_err(dev->class_dev,
+ "probed id does not match board id (0x%x != 0x%x)\n",
+ id, board->id);
+ return -ENODEV;
+ }
+
+ /*
+ * If the dev->board_ptr is not set, the user is trying to attach
+ * an unspecified board to this driver. In this case the id is used
+ * to 'probe' for the dev->board_ptr.
+ */
switch (id) {
- case 0x3:
- if (index == das1801st_da || index == das1802st_da ||
- index == das1701st_da || index == das1702st_da)
- return board;
- index = das1801st;
+ case DAS1800_ID_ST_DA:
+ /* das-1701st-da, das-1702st-da, das-1801st-da, das-1802st-da */
+ board = &das1800_boards[BOARD_DAS1801ST_DA];
break;
- case 0x4:
- if (index == das1802hr_da || index == das1702hr_da)
- return board;
- index = das1802hr;
+ case DAS1800_ID_HR_DA:
+ /* das-1702hr-da, das-1802hr-da */
+ board = &das1800_boards[BOARD_DAS1802HR_DA];
break;
- case 0x5:
- if (index == das1801ao || index == das1802ao ||
- index == das1701ao || index == das1702ao)
- return board;
- index = das1801ao;
+ case DAS1800_ID_AO:
+ /* das-1701ao, das-1702ao, das-1801ao, das-1802ao */
+ board = &das1800_boards[BOARD_DAS1801AO];
break;
- case 0x6:
- if (index == das1802hr || index == das1702hr)
- return board;
- index = das1802hr;
+ case DAS1800_ID_HR:
+ /* das-1702hr, das-1802hr */
+ board = &das1800_boards[BOARD_DAS1802HR];
break;
- case 0x7:
- if (index == das1801st || index == das1802st ||
- index == das1701st || index == das1702st)
- return board;
- index = das1801st;
+ case DAS1800_ID_ST:
+ /* das-1701st, das-1702st, das-1801st, das-1802st */
+ board = &das1800_boards[BOARD_DAS1801ST];
break;
- case 0x8:
- if (index == das1801hc || index == das1802hc)
- return board;
- index = das1801hc;
+ case DAS1800_ID_HC:
+ /* das-1801hc, das-1802hc */
+ board = &das1800_boards[BOARD_DAS1801HC];
break;
default:
- dev_err(dev->class_dev,
- "Board model: probe returned 0x%x (unknown, please report)\n",
- id);
- return NULL;
+ dev_err(dev->class_dev, "invalid probe id 0x%x\n", id);
+ return -ENODEV;
}
- dev_err(dev->class_dev,
- "Board model (probed, not recommended): %s series\n",
- das1800_boards[index].name);
-
- return &das1800_boards[index];
+ dev->board_ptr = board;
+ dev->board_name = board->name;
+ dev_warn(dev->class_dev,
+ "probed id 0x%0x: %s series (not recommended)\n",
+ id, board->name);
+ return 0;
}
static int das1800_attach(struct comedi_device *dev,
@@ -1287,7 +1175,9 @@
struct das1800_private *devpriv;
struct comedi_subdevice *s;
unsigned int irq = it->options[1];
+ bool is_16bit;
int ret;
+ int i;
devpriv = comedi_alloc_devpriv(dev, sizeof(*devpriv));
if (!devpriv)
@@ -1297,16 +1187,15 @@
if (ret)
return ret;
- board = das1800_probe(dev);
- if (!board) {
- dev_err(dev->class_dev, "unable to determine board type\n");
- return -ENODEV;
- }
- dev->board_ptr = board;
- dev->board_name = board->name;
+ ret = das1800_probe(dev);
+ if (ret)
+ return ret;
+ board = dev->board_ptr;
- /* if it is an 'ao' board with fancy analog out then we need extra io ports */
- if (board->ao_ability == 2) {
+ is_16bit = board->id == DAS1800_ID_HR || board->id == DAS1800_ID_HR_DA;
+
+ /* waveform 'ao' boards have additional io ports */
+ if (board->id == DAS1800_ID_AO) {
unsigned long iobase2 = dev->iobase + IOBASE2;
ret = __comedi_request_region(dev, iobase2, DAS1800_SIZE);
@@ -1349,7 +1238,9 @@
if (dev->irq & it->options[2])
das1800_init_dma(dev, it);
- devpriv->fifo_buf = kmalloc_array(FIFO_SIZE, sizeof(uint16_t), GFP_KERNEL);
+ devpriv->fifo_buf = kmalloc_array(FIFO_SIZE,
+ sizeof(*devpriv->fifo_buf),
+ GFP_KERNEL);
if (!devpriv->fifo_buf)
return -ENOMEM;
@@ -1362,70 +1253,94 @@
if (ret)
return ret;
- /* analog input subdevice */
+ /*
+ * Analog Input subdevice
+ *
+ * The "hc" type boards have 64 analog input channels and a 64
+ * entry QRAM fifo.
+ *
+ * All the other board types have 16 on-board channels. Each channel
+ * can be expanded to 16 channels with the addition of an EXP-1800
+ * expansion board for a total of 256 channels. The QRAM fifo on
+ * these boards has 256 entries.
+ *
+ * From the datasheets it's not clear what the comedi channel to
+ * actual physical channel mapping is when EXP-1800 boards are used.
+ */
s = &dev->subdevices[0];
- s->type = COMEDI_SUBD_AI;
- s->subdev_flags = SDF_READABLE | SDF_DIFF | SDF_GROUND;
- if (board->common)
- s->subdev_flags |= SDF_COMMON;
- s->n_chan = board->qram_len;
- s->maxdata = (1 << board->resolution) - 1;
- s->range_table = board->range_ai;
- s->insn_read = das1800_ai_rinsn;
+ s->type = COMEDI_SUBD_AI;
+ s->subdev_flags = SDF_READABLE | SDF_DIFF | SDF_GROUND;
+ if (board->id != DAS1800_ID_HC)
+ s->subdev_flags |= SDF_COMMON;
+ s->n_chan = (board->id == DAS1800_ID_HC) ? 64 : 256;
+ s->maxdata = is_16bit ? 0xffff : 0x0fff;
+ s->range_table = board->is_01_series ? &das1801_ai_range
+ : &das1802_ai_range;
+ s->insn_read = das1800_ai_insn_read;
if (dev->irq) {
dev->read_subdev = s;
- s->subdev_flags |= SDF_CMD_READ;
- s->len_chanlist = s->n_chan;
- s->do_cmd = das1800_ai_do_cmd;
- s->do_cmdtest = das1800_ai_do_cmdtest;
- s->poll = das1800_ai_poll;
- s->cancel = das1800_cancel;
+ s->subdev_flags |= SDF_CMD_READ;
+ s->len_chanlist = s->n_chan;
+ s->do_cmd = das1800_ai_cmd;
+ s->do_cmdtest = das1800_ai_cmdtest;
+ s->poll = das1800_ai_poll;
+ s->cancel = das1800_ai_cancel;
+ s->munge = das1800_ai_munge;
}
- /* analog out */
+ /* Analog Output subdevice */
s = &dev->subdevices[1];
- if (board->ao_ability == 1) {
- s->type = COMEDI_SUBD_AO;
- s->subdev_flags = SDF_WRITABLE;
- s->n_chan = board->ao_n_chan;
- s->maxdata = (1 << board->resolution) - 1;
- s->range_table = &range_bipolar10;
- s->insn_write = das1800_ao_winsn;
+ if (board->id == DAS1800_ID_ST_DA || board->id == DAS1800_ID_HR_DA) {
+ s->type = COMEDI_SUBD_AO;
+ s->subdev_flags = SDF_WRITABLE;
+ s->n_chan = (board->id == DAS1800_ID_ST_DA) ? 4 : 2;
+ s->maxdata = is_16bit ? 0xffff : 0x0fff;
+ s->range_table = &range_bipolar10;
+ s->insn_write = das1800_ao_insn_write;
+
+ ret = comedi_alloc_subdev_readback(s);
+ if (ret)
+ return ret;
+
+ /* initialize all channels to 0V */
+ for (i = 0; i < s->n_chan; i++) {
+ /* spinlock is not necessary during the attach */
+ outb(DAC(i), dev->iobase + DAS1800_SELECT);
+ outw(0, dev->iobase + DAS1800_DAC);
+ }
+ } else if (board->id == DAS1800_ID_AO) {
+ /*
+ * 'ao' boards have waveform analog outputs that are not
+ * currently supported.
+ */
+ s->type = COMEDI_SUBD_UNUSED;
} else {
- s->type = COMEDI_SUBD_UNUSED;
+ s->type = COMEDI_SUBD_UNUSED;
}
- /* di */
+ /* Digital Input subdevice */
s = &dev->subdevices[2];
- s->type = COMEDI_SUBD_DI;
- s->subdev_flags = SDF_READABLE;
- s->n_chan = 4;
- s->maxdata = 1;
- s->range_table = &range_digital;
- s->insn_bits = das1800_di_rbits;
+ s->type = COMEDI_SUBD_DI;
+ s->subdev_flags = SDF_READABLE;
+ s->n_chan = 4;
+ s->maxdata = 1;
+ s->range_table = &range_digital;
+ s->insn_bits = das1800_di_insn_bits;
- /* do */
+ /* Digital Output subdevice */
s = &dev->subdevices[3];
- s->type = COMEDI_SUBD_DO;
- s->subdev_flags = SDF_WRITABLE;
- s->n_chan = board->do_n_chan;
- s->maxdata = 1;
- s->range_table = &range_digital;
- s->insn_bits = das1800_do_wbits;
+ s->type = COMEDI_SUBD_DO;
+ s->subdev_flags = SDF_WRITABLE;
+ s->n_chan = (board->id == DAS1800_ID_HC) ? 8 : 4;
+ s->maxdata = 1;
+ s->range_table = &range_digital;
+ s->insn_bits = das1800_do_insn_bits;
- das1800_cancel(dev, dev->read_subdev);
+ das1800_ai_cancel(dev, dev->read_subdev);
/* initialize digital out channels */
outb(0, dev->iobase + DAS1800_DIGITAL);
- /* initialize analog out channels */
- if (board->ao_ability == 1) {
- /* select 'update' dac channel for baseAddress + 0x0 */
- outb(DAC(board->ao_n_chan - 1),
- dev->iobase + DAS1800_SELECT);
- outw(devpriv->ao_update_bits, dev->iobase + DAS1800_DAC);
- }
-
return 0;
};
@@ -1454,5 +1369,5 @@
module_comedi_driver(das1800_driver);
MODULE_AUTHOR("Comedi http://www.comedi.org");
-MODULE_DESCRIPTION("Comedi low-level driver");
+MODULE_DESCRIPTION("Comedi driver for DAS1800 compatible ISA boards");
MODULE_LICENSE("GPL");
diff --git a/drivers/staging/comedi/drivers/dt282x.c b/drivers/staging/comedi/drivers/dt282x.c
index 40bf009..d5295bb 100644
--- a/drivers/staging/comedi/drivers/dt282x.c
+++ b/drivers/staging/comedi/drivers/dt282x.c
@@ -69,49 +69,61 @@
* Register map
*/
#define DT2821_ADCSR_REG 0x00
-#define DT2821_ADCSR_ADERR (1 << 15)
-#define DT2821_ADCSR_ADCLK (1 << 9)
-#define DT2821_ADCSR_MUXBUSY (1 << 8)
-#define DT2821_ADCSR_ADDONE (1 << 7)
-#define DT2821_ADCSR_IADDONE (1 << 6)
+#define DT2821_ADCSR_ADERR BIT(15)
+#define DT2821_ADCSR_ADCLK BIT(9)
+#define DT2821_ADCSR_MUXBUSY BIT(8)
+#define DT2821_ADCSR_ADDONE BIT(7)
+#define DT2821_ADCSR_IADDONE BIT(6)
#define DT2821_ADCSR_GS(x) (((x) & 0x3) << 4)
#define DT2821_ADCSR_CHAN(x) (((x) & 0xf) << 0)
#define DT2821_CHANCSR_REG 0x02
-#define DT2821_CHANCSR_LLE (1 << 15)
-#define DT2821_CHANCSR_PRESLA(x) (((x) & 0xf) >> 8)
+#define DT2821_CHANCSR_LLE BIT(15)
+#define DT2821_CHANCSR_TO_PRESLA(x) (((x) >> 8) & 0xf)
#define DT2821_CHANCSR_NUMB(x) ((((x) - 1) & 0xf) << 0)
#define DT2821_ADDAT_REG 0x04
#define DT2821_DACSR_REG 0x06
-#define DT2821_DACSR_DAERR (1 << 15)
+#define DT2821_DACSR_DAERR BIT(15)
#define DT2821_DACSR_YSEL(x) ((x) << 9)
-#define DT2821_DACSR_SSEL (1 << 8)
-#define DT2821_DACSR_DACRDY (1 << 7)
-#define DT2821_DACSR_IDARDY (1 << 6)
-#define DT2821_DACSR_DACLK (1 << 5)
-#define DT2821_DACSR_HBOE (1 << 1)
-#define DT2821_DACSR_LBOE (1 << 0)
+#define DT2821_DACSR_SSEL BIT(8)
+#define DT2821_DACSR_DACRDY BIT(7)
+#define DT2821_DACSR_IDARDY BIT(6)
+#define DT2821_DACSR_DACLK BIT(5)
+#define DT2821_DACSR_HBOE BIT(1)
+#define DT2821_DACSR_LBOE BIT(0)
#define DT2821_DADAT_REG 0x08
#define DT2821_DIODAT_REG 0x0a
#define DT2821_SUPCSR_REG 0x0c
-#define DT2821_SUPCSR_DMAD (1 << 15)
-#define DT2821_SUPCSR_ERRINTEN (1 << 14)
-#define DT2821_SUPCSR_CLRDMADNE (1 << 13)
-#define DT2821_SUPCSR_DDMA (1 << 12)
-#define DT2821_SUPCSR_DS_PIO (0 << 10)
-#define DT2821_SUPCSR_DS_AD_CLK (1 << 10)
-#define DT2821_SUPCSR_DS_DA_CLK (2 << 10)
-#define DT2821_SUPCSR_DS_AD_TRIG (3 << 10)
-#define DT2821_SUPCSR_BUFFB (1 << 9)
-#define DT2821_SUPCSR_SCDN (1 << 8)
-#define DT2821_SUPCSR_DACON (1 << 7)
-#define DT2821_SUPCSR_ADCINIT (1 << 6)
-#define DT2821_SUPCSR_DACINIT (1 << 5)
-#define DT2821_SUPCSR_PRLD (1 << 4)
-#define DT2821_SUPCSR_STRIG (1 << 3)
-#define DT2821_SUPCSR_XTRIG (1 << 2)
-#define DT2821_SUPCSR_XCLK (1 << 1)
-#define DT2821_SUPCSR_BDINIT (1 << 0)
+#define DT2821_SUPCSR_DMAD BIT(15)
+#define DT2821_SUPCSR_ERRINTEN BIT(14)
+#define DT2821_SUPCSR_CLRDMADNE BIT(13)
+#define DT2821_SUPCSR_DDMA BIT(12)
+#define DT2821_SUPCSR_DS(x) (((x) & 0x3) << 10)
+#define DT2821_SUPCSR_DS_PIO DT2821_SUPCSR_DS(0)
+#define DT2821_SUPCSR_DS_AD_CLK DT2821_SUPCSR_DS(1)
+#define DT2821_SUPCSR_DS_DA_CLK DT2821_SUPCSR_DS(2)
+#define DT2821_SUPCSR_DS_AD_TRIG DT2821_SUPCSR_DS(3)
+#define DT2821_SUPCSR_BUFFB BIT(9)
+#define DT2821_SUPCSR_SCDN BIT(8)
+#define DT2821_SUPCSR_DACON BIT(7)
+#define DT2821_SUPCSR_ADCINIT BIT(6)
+#define DT2821_SUPCSR_DACINIT BIT(5)
+#define DT2821_SUPCSR_PRLD BIT(4)
+#define DT2821_SUPCSR_STRIG BIT(3)
+#define DT2821_SUPCSR_XTRIG BIT(2)
+#define DT2821_SUPCSR_XCLK BIT(1)
+#define DT2821_SUPCSR_BDINIT BIT(0)
#define DT2821_TMRCTR_REG 0x0e
+#define DT2821_TMRCTR_PRESCALE(x) (((x) & 0xf) << 8)
+#define DT2821_TMRCTR_DIVIDER(x) ((255 - ((x) & 0xff)) << 0)
+
+/* Pacer Clock */
+#define DT2821_OSC_BASE 250 /* 4 MHz (in nanoseconds) */
+#define DT2821_PRESCALE(x) BIT(x)
+#define DT2821_PRESCALE_MAX 15
+#define DT2821_DIVIDER_MAX 255
+#define DT2821_OSC_MAX (DT2821_OSC_BASE * \
+ DT2821_PRESCALE(DT2821_PRESCALE_MAX) * \
+ DT2821_DIVIDER_MAX)
static const struct comedi_lrange range_dt282x_ai_lo_bipolar = {
4, {
@@ -364,10 +376,10 @@
{
unsigned int prescale, base, divider;
- for (prescale = 0; prescale < 16; prescale++) {
- if (prescale == 1)
+ for (prescale = 0; prescale <= DT2821_PRESCALE_MAX; prescale++) {
+ if (prescale == 1) /* 0 and 1 are both divide by 1 */
continue;
- base = 250 * (1 << prescale);
+ base = DT2821_OSC_BASE * DT2821_PRESCALE(prescale);
switch (flags & CMDF_ROUND_MASK) {
case CMDF_ROUND_NEAREST:
default:
@@ -380,15 +392,17 @@
divider = DIV_ROUND_UP(*ns, base);
break;
}
- if (divider < 256) {
- *ns = divider * base;
- return (prescale << 8) | (255 - divider);
- }
+ if (divider <= DT2821_DIVIDER_MAX)
+ break;
}
- base = 250 * (1 << 15);
- divider = 255;
+ if (divider > DT2821_DIVIDER_MAX) {
+ prescale = DT2821_PRESCALE_MAX;
+ divider = DT2821_DIVIDER_MAX;
+ base = DT2821_OSC_BASE * DT2821_PRESCALE(prescale);
+ }
*ns = divider * base;
- return (15 << 8) | (255 - divider);
+ return DT2821_TMRCTR_PRESCALE(prescale) |
+ DT2821_TMRCTR_DIVIDER(divider);
}
static void dt282x_munge(struct comedi_device *dev,
@@ -683,13 +697,8 @@
/* Step 3: check if arguments are trivially valid */
err |= comedi_check_trigger_arg_is(&cmd->start_arg, 0);
-
err |= comedi_check_trigger_arg_is(&cmd->scan_begin_arg, 0);
-
- err |= comedi_check_trigger_arg_min(&cmd->convert_arg, 4000);
-
-#define SLOWEST_TIMER (250*(1<<15)*255)
- err |= comedi_check_trigger_arg_max(&cmd->convert_arg, SLOWEST_TIMER);
+ err |= comedi_check_trigger_arg_max(&cmd->convert_arg, DT2821_OSC_MAX);
err |= comedi_check_trigger_arg_min(&cmd->convert_arg, board->ai_speed);
err |= comedi_check_trigger_arg_is(&cmd->scan_end_arg,
cmd->chanlist_len);
@@ -1084,20 +1093,6 @@
return 0;
}
-/*
- options:
- 0 i/o base
- 1 irq
- 2 dma1
- 3 dma2
- 4 0=single ended, 1=differential
- 5 ai 0=straight binary, 1=2's comp
- 6 ao0 0=straight binary, 1=2's comp
- 7 ao1 0=straight binary, 1=2's comp
- 8 ai 0=±10 V, 1=0-10 V, 2=±5 V, 3=0-5 V
- 9 ao0 0=±10 V, 1=0-10 V, 2=±5 V, 3=0-5 V, 4=±2.5 V
- 10 ao1 0=±10 V, 1=0-10 V, 2=±5 V, 3=0-5 V, 4=±2.5 V
- */
static int dt282x_attach(struct comedi_device *dev, struct comedi_devconfig *it)
{
const struct dt282x_board *board = dev->board_ptr;
diff --git a/drivers/staging/comedi/drivers/mite.c b/drivers/staging/comedi/drivers/mite.c
index 8f24702..b1c0860 100644
--- a/drivers/staging/comedi/drivers/mite.c
+++ b/drivers/staging/comedi/drivers/mite.c
@@ -46,232 +46,409 @@
#include <linux/module.h>
#include <linux/slab.h>
+#include <linux/log2.h>
#include "../comedi_pci.h"
#include "mite.h"
-#define TOP_OF_PAGE(x) ((x)|(~(PAGE_MASK)))
+/*
+ * Mite registers
+ */
+#define MITE_UNKNOWN_DMA_BURST_REG 0x28
+#define UNKNOWN_DMA_BURST_ENABLE_BITS 0x600
-struct mite_struct *mite_alloc(struct pci_dev *pcidev)
+#define MITE_PCI_CONFIG_OFFSET 0x300
+#define MITE_CSIGR 0x460 /* chip signature */
+#define CSIGR_TO_IOWINS(x) (((x) >> 29) & 0x7)
+#define CSIGR_TO_WINS(x) (((x) >> 24) & 0x1f)
+#define CSIGR_TO_WPDEP(x) (((x) >> 20) & 0x7)
+#define CSIGR_TO_DMAC(x) (((x) >> 16) & 0xf)
+#define CSIGR_TO_IMODE(x) (((x) >> 12) & 0x3) /* pci=0x3 */
+#define CSIGR_TO_MMODE(x) (((x) >> 8) & 0x3) /* minimite=1 */
+#define CSIGR_TO_TYPE(x) (((x) >> 4) & 0xf) /* mite=0, minimite=1 */
+#define CSIGR_TO_VER(x) (((x) >> 0) & 0xf)
+
+#define MITE_CHAN(x) (0x500 + 0x100 * (x))
+#define MITE_CHOR(x) (0x00 + MITE_CHAN(x)) /* channel operation */
+#define CHOR_DMARESET BIT(31)
+#define CHOR_SET_SEND_TC BIT(11)
+#define CHOR_CLR_SEND_TC BIT(10)
+#define CHOR_SET_LPAUSE BIT(9)
+#define CHOR_CLR_LPAUSE BIT(8)
+#define CHOR_CLRDONE BIT(7)
+#define CHOR_CLRRB BIT(6)
+#define CHOR_CLRLC BIT(5)
+#define CHOR_FRESET BIT(4)
+#define CHOR_ABORT BIT(3) /* stop without emptying fifo */
+#define CHOR_STOP BIT(2) /* stop after emptying fifo */
+#define CHOR_CONT BIT(1)
+#define CHOR_START BIT(0)
+#define MITE_CHCR(x) (0x04 + MITE_CHAN(x)) /* channel control */
+#define CHCR_SET_DMA_IE BIT(31)
+#define CHCR_CLR_DMA_IE BIT(30)
+#define CHCR_SET_LINKP_IE BIT(29)
+#define CHCR_CLR_LINKP_IE BIT(28)
+#define CHCR_SET_SAR_IE BIT(27)
+#define CHCR_CLR_SAR_IE BIT(26)
+#define CHCR_SET_DONE_IE BIT(25)
+#define CHCR_CLR_DONE_IE BIT(24)
+#define CHCR_SET_MRDY_IE BIT(23)
+#define CHCR_CLR_MRDY_IE BIT(22)
+#define CHCR_SET_DRDY_IE BIT(21)
+#define CHCR_CLR_DRDY_IE BIT(20)
+#define CHCR_SET_LC_IE BIT(19)
+#define CHCR_CLR_LC_IE BIT(18)
+#define CHCR_SET_CONT_RB_IE BIT(17)
+#define CHCR_CLR_CONT_RB_IE BIT(16)
+#define CHCR_FIFO(x) (((x) & 0x1) << 15)
+#define CHCR_FIFODIS CHCR_FIFO(1)
+#define CHCR_FIFO_ON CHCR_FIFO(0)
+#define CHCR_BURST(x) (((x) & 0x1) << 14)
+#define CHCR_BURSTEN CHCR_BURST(1)
+#define CHCR_NO_BURSTEN CHCR_BURST(0)
+#define CHCR_BYTE_SWAP_DEVICE BIT(6)
+#define CHCR_BYTE_SWAP_MEMORY BIT(4)
+#define CHCR_DIR(x) (((x) & 0x1) << 3)
+#define CHCR_DEV_TO_MEM CHCR_DIR(1)
+#define CHCR_MEM_TO_DEV CHCR_DIR(0)
+#define CHCR_MODE(x) (((x) & 0x7) << 0)
+#define CHCR_NORMAL CHCR_MODE(0)
+#define CHCR_CONTINUE CHCR_MODE(1)
+#define CHCR_RINGBUFF CHCR_MODE(2)
+#define CHCR_LINKSHORT CHCR_MODE(4)
+#define CHCR_LINKLONG CHCR_MODE(5)
+#define MITE_TCR(x) (0x08 + MITE_CHAN(x)) /* transfer count */
+#define MITE_MCR(x) (0x0c + MITE_CHAN(x)) /* memory config */
+#define MITE_MAR(x) (0x10 + MITE_CHAN(x)) /* memory address */
+#define MITE_DCR(x) (0x14 + MITE_CHAN(x)) /* device config */
+#define DCR_NORMAL BIT(29)
+#define MITE_DAR(x) (0x18 + MITE_CHAN(x)) /* device address */
+#define MITE_LKCR(x) (0x1c + MITE_CHAN(x)) /* link config */
+#define MITE_LKAR(x) (0x20 + MITE_CHAN(x)) /* link address */
+#define MITE_LLKAR(x) (0x24 + MITE_CHAN(x)) /* see tnt5002 manual */
+#define MITE_BAR(x) (0x28 + MITE_CHAN(x)) /* base address */
+#define MITE_BCR(x) (0x2c + MITE_CHAN(x)) /* base count */
+#define MITE_SAR(x) (0x30 + MITE_CHAN(x)) /* ? address */
+#define MITE_WSCR(x) (0x34 + MITE_CHAN(x)) /* ? */
+#define MITE_WSER(x) (0x38 + MITE_CHAN(x)) /* ? */
+#define MITE_CHSR(x) (0x3c + MITE_CHAN(x)) /* channel status */
+#define CHSR_INT BIT(31)
+#define CHSR_LPAUSES BIT(29)
+#define CHSR_SARS BIT(27)
+#define CHSR_DONE BIT(25)
+#define CHSR_MRDY BIT(23)
+#define CHSR_DRDY BIT(21)
+#define CHSR_LINKC BIT(19)
+#define CHSR_CONTS_RB BIT(17)
+#define CHSR_ERROR BIT(15)
+#define CHSR_SABORT BIT(14)
+#define CHSR_HABORT BIT(13)
+#define CHSR_STOPS BIT(12)
+#define CHSR_OPERR(x) (((x) & 0x3) << 10)
+#define CHSR_OPERR_MASK CHSR_OPERR(3)
+#define CHSR_OPERR_NOERROR CHSR_OPERR(0)
+#define CHSR_OPERR_FIFOERROR CHSR_OPERR(1)
+#define CHSR_OPERR_LINKERROR CHSR_OPERR(1) /* ??? */
+#define CHSR_XFERR BIT(9)
+#define CHSR_END BIT(8)
+#define CHSR_DRQ1 BIT(7)
+#define CHSR_DRQ0 BIT(6)
+#define CHSR_LERR(x) (((x) & 0x3) << 4)
+#define CHSR_LERR_MASK CHSR_LERR(3)
+#define CHSR_LBERR CHSR_LERR(1)
+#define CHSR_LRERR CHSR_LERR(2)
+#define CHSR_LOERR CHSR_LERR(3)
+#define CHSR_MERR(x) (((x) & 0x3) << 2)
+#define CHSR_MERR_MASK CHSR_MERR(3)
+#define CHSR_MBERR CHSR_MERR(1)
+#define CHSR_MRERR CHSR_MERR(2)
+#define CHSR_MOERR CHSR_MERR(3)
+#define CHSR_DERR(x) (((x) & 0x3) << 0)
+#define CHSR_DERR_MASK CHSR_DERR(3)
+#define CHSR_DBERR CHSR_DERR(1)
+#define CHSR_DRERR CHSR_DERR(2)
+#define CHSR_DOERR CHSR_DERR(3)
+#define MITE_FCR(x) (0x40 + MITE_CHAN(x)) /* fifo count */
+
+/* common bits for the memory/device/link config registers */
+#define CR_RL(x) (((x) & 0x7) << 21)
+#define CR_REQS(x) (((x) & 0x7) << 16)
+#define CR_REQS_MASK CR_REQS(7)
+#define CR_ASEQ(x) (((x) & 0x3) << 10)
+#define CR_ASEQDONT CR_ASEQ(0)
+#define CR_ASEQUP CR_ASEQ(1)
+#define CR_ASEQDOWN CR_ASEQ(2)
+#define CR_ASEQ_MASK CR_ASEQ(3)
+#define CR_PSIZE(x) (((x) & 0x3) << 8)
+#define CR_PSIZE8 CR_PSIZE(1)
+#define CR_PSIZE16 CR_PSIZE(2)
+#define CR_PSIZE32 CR_PSIZE(3)
+#define CR_PORT(x) (((x) & 0x3) << 6)
+#define CR_PORTCPU CR_PORT(0)
+#define CR_PORTIO CR_PORT(1)
+#define CR_PORTVXI CR_PORT(2)
+#define CR_PORTMXI CR_PORT(3)
+#define CR_AMDEVICE BIT(0)
+
+static unsigned int MITE_IODWBSR_1_WSIZE_bits(unsigned int size)
{
- struct mite_struct *mite;
- unsigned int i;
-
- mite = kzalloc(sizeof(*mite), GFP_KERNEL);
- if (mite) {
- spin_lock_init(&mite->lock);
- mite->pcidev = pcidev;
- for (i = 0; i < MAX_MITE_DMA_CHANNELS; ++i) {
- mite->channels[i].mite = mite;
- mite->channels[i].channel = i;
- mite->channels[i].done = 1;
- }
- }
- return mite;
-}
-EXPORT_SYMBOL_GPL(mite_alloc);
-
-static void dump_chip_signature(u32 csigr_bits)
-{
- pr_info("version = %i, type = %i, mite mode = %i, interface mode = %i\n",
- mite_csigr_version(csigr_bits), mite_csigr_type(csigr_bits),
- mite_csigr_mmode(csigr_bits), mite_csigr_imode(csigr_bits));
- pr_info("num channels = %i, write post fifo depth = %i, wins = %i, iowins = %i\n",
- mite_csigr_dmac(csigr_bits), mite_csigr_wpdep(csigr_bits),
- mite_csigr_wins(csigr_bits), mite_csigr_iowins(csigr_bits));
+ return (ilog2(size) - 1) & 0x1f;
}
-static unsigned mite_fifo_size(struct mite_struct *mite, unsigned channel)
+static unsigned int mite_retry_limit(unsigned int retry_limit)
{
- unsigned fcr_bits = readl(mite->mite_io_addr + MITE_FCR(channel));
- unsigned empty_count = (fcr_bits >> 16) & 0xff;
- unsigned full_count = fcr_bits & 0xff;
+ unsigned int value = 0;
+
+ if (retry_limit)
+ value = 1 + ilog2(retry_limit);
+ if (value > 0x7)
+ value = 0x7;
+ return CR_RL(value);
+}
+
+static unsigned int mite_drq_reqs(unsigned int drq_line)
+{
+ /* This also works on m-series when using channels (drq_line) 4 or 5. */
+ return CR_REQS((drq_line & 0x3) | 0x4);
+}
+
+static unsigned int mite_fifo_size(struct mite *mite, unsigned int channel)
+{
+ unsigned int fcr_bits = readl(mite->mmio + MITE_FCR(channel));
+ unsigned int empty_count = (fcr_bits >> 16) & 0xff;
+ unsigned int full_count = fcr_bits & 0xff;
return empty_count + full_count;
}
-int mite_setup2(struct comedi_device *dev,
- struct mite_struct *mite, bool use_win1)
+static u32 mite_device_bytes_transferred(struct mite_channel *mite_chan)
{
- unsigned long length;
- int i;
- u32 csigr_bits;
- unsigned unknown_dma_burst_bits;
+ struct mite *mite = mite_chan->mite;
- pci_set_master(mite->pcidev);
-
- mite->mite_io_addr = pci_ioremap_bar(mite->pcidev, 0);
- if (!mite->mite_io_addr) {
- dev_err(dev->class_dev,
- "Failed to remap mite io memory address\n");
- return -ENOMEM;
- }
- mite->mite_phys_addr = pci_resource_start(mite->pcidev, 0);
-
- dev->mmio = pci_ioremap_bar(mite->pcidev, 1);
- if (!dev->mmio) {
- dev_err(dev->class_dev,
- "Failed to remap daq io memory address\n");
- return -ENOMEM;
- }
- mite->daq_phys_addr = pci_resource_start(mite->pcidev, 1);
- length = pci_resource_len(mite->pcidev, 1);
-
- if (use_win1) {
- writel(0, mite->mite_io_addr + MITE_IODWBSR);
- dev_info(dev->class_dev,
- "using I/O Window Base Size register 1\n");
- writel(mite->daq_phys_addr | WENAB |
- MITE_IODWBSR_1_WSIZE_bits(length),
- mite->mite_io_addr + MITE_IODWBSR_1);
- writel(0, mite->mite_io_addr + MITE_IODWCR_1);
- } else {
- writel(mite->daq_phys_addr | WENAB,
- mite->mite_io_addr + MITE_IODWBSR);
- }
- /*
- * Make sure dma bursts work. I got this from running a bus analyzer
- * on a pxi-6281 and a pxi-6713. 6713 powered up with register value
- * of 0x61f and bursts worked. 6281 powered up with register value of
- * 0x1f and bursts didn't work. The NI windows driver reads the
- * register, then does a bitwise-or of 0x600 with it and writes it back.
- */
- unknown_dma_burst_bits =
- readl(mite->mite_io_addr + MITE_UNKNOWN_DMA_BURST_REG);
- unknown_dma_burst_bits |= UNKNOWN_DMA_BURST_ENABLE_BITS;
- writel(unknown_dma_burst_bits,
- mite->mite_io_addr + MITE_UNKNOWN_DMA_BURST_REG);
-
- csigr_bits = readl(mite->mite_io_addr + MITE_CSIGR);
- mite->num_channels = mite_csigr_dmac(csigr_bits);
- if (mite->num_channels > MAX_MITE_DMA_CHANNELS) {
- dev_warn(dev->class_dev,
- "mite: bug? chip claims to have %i dma channels. Setting to %i.\n",
- mite->num_channels, MAX_MITE_DMA_CHANNELS);
- mite->num_channels = MAX_MITE_DMA_CHANNELS;
- }
- dump_chip_signature(csigr_bits);
- for (i = 0; i < mite->num_channels; i++) {
- writel(CHOR_DMARESET, mite->mite_io_addr + MITE_CHOR(i));
- /* disable interrupts */
- writel(CHCR_CLR_DMA_IE | CHCR_CLR_LINKP_IE | CHCR_CLR_SAR_IE |
- CHCR_CLR_DONE_IE | CHCR_CLR_MRDY_IE | CHCR_CLR_DRDY_IE |
- CHCR_CLR_LC_IE | CHCR_CLR_CONT_RB_IE,
- mite->mite_io_addr + MITE_CHCR(i));
- }
- mite->fifo_size = mite_fifo_size(mite, 0);
- dev_info(dev->class_dev, "fifo size is %i.\n", mite->fifo_size);
- return 0;
+ return readl(mite->mmio + MITE_DAR(mite_chan->channel));
}
-EXPORT_SYMBOL_GPL(mite_setup2);
-void mite_detach(struct mite_struct *mite)
+/**
+ * mite_bytes_in_transit() - Returns the number of unread bytes in the fifo.
+ * @mite_chan: MITE dma channel.
+ */
+u32 mite_bytes_in_transit(struct mite_channel *mite_chan)
{
- if (!mite)
+ struct mite *mite = mite_chan->mite;
+
+ return readl(mite->mmio + MITE_FCR(mite_chan->channel)) & 0xff;
+}
+EXPORT_SYMBOL_GPL(mite_bytes_in_transit);
+
+/* returns lower bound for number of bytes transferred from device to memory */
+static u32 mite_bytes_written_to_memory_lb(struct mite_channel *mite_chan)
+{
+ u32 device_byte_count;
+
+ device_byte_count = mite_device_bytes_transferred(mite_chan);
+ return device_byte_count - mite_bytes_in_transit(mite_chan);
+}
+
+/* returns upper bound for number of bytes transferred from device to memory */
+static u32 mite_bytes_written_to_memory_ub(struct mite_channel *mite_chan)
+{
+ u32 in_transit_count;
+
+ in_transit_count = mite_bytes_in_transit(mite_chan);
+ return mite_device_bytes_transferred(mite_chan) - in_transit_count;
+}
+
+/* returns lower bound for number of bytes read from memory to device */
+static u32 mite_bytes_read_from_memory_lb(struct mite_channel *mite_chan)
+{
+ u32 device_byte_count;
+
+ device_byte_count = mite_device_bytes_transferred(mite_chan);
+ return device_byte_count + mite_bytes_in_transit(mite_chan);
+}
+
+/* returns upper bound for number of bytes read from memory to device */
+static u32 mite_bytes_read_from_memory_ub(struct mite_channel *mite_chan)
+{
+ u32 in_transit_count;
+
+ in_transit_count = mite_bytes_in_transit(mite_chan);
+ return mite_device_bytes_transferred(mite_chan) + in_transit_count;
+}
+
+static void mite_sync_input_dma(struct mite_channel *mite_chan,
+ struct comedi_subdevice *s)
+{
+ struct comedi_async *async = s->async;
+ int count;
+ unsigned int nbytes, old_alloc_count;
+
+ old_alloc_count = async->buf_write_alloc_count;
+ /* write alloc as much as we can */
+ comedi_buf_write_alloc(s, async->prealloc_bufsz);
+
+ nbytes = mite_bytes_written_to_memory_lb(mite_chan);
+ if ((int)(mite_bytes_written_to_memory_ub(mite_chan) -
+ old_alloc_count) > 0) {
+ dev_warn(s->device->class_dev,
+ "mite: DMA overwrite of free area\n");
+ async->events |= COMEDI_CB_OVERFLOW;
return;
-
- if (mite->mite_io_addr)
- iounmap(mite->mite_io_addr);
-
- kfree(mite);
-}
-EXPORT_SYMBOL_GPL(mite_detach);
-
-struct mite_dma_descriptor_ring *mite_alloc_ring(struct mite_struct *mite)
-{
- struct mite_dma_descriptor_ring *ring =
- kmalloc(sizeof(struct mite_dma_descriptor_ring), GFP_KERNEL);
-
- if (!ring)
- return NULL;
- ring->hw_dev = get_device(&mite->pcidev->dev);
- if (!ring->hw_dev) {
- kfree(ring);
- return NULL;
}
- ring->n_links = 0;
- ring->descriptors = NULL;
- ring->descriptors_dma_addr = 0;
- return ring;
-};
-EXPORT_SYMBOL_GPL(mite_alloc_ring);
-void mite_free_ring(struct mite_dma_descriptor_ring *ring)
-{
- if (ring) {
- if (ring->descriptors) {
- dma_free_coherent(ring->hw_dev,
- ring->n_links *
- sizeof(struct mite_dma_descriptor),
- ring->descriptors,
- ring->descriptors_dma_addr);
- }
- put_device(ring->hw_dev);
- kfree(ring);
- }
-};
-EXPORT_SYMBOL_GPL(mite_free_ring);
-
-struct mite_channel *mite_request_channel_in_range(struct mite_struct *mite,
- struct
- mite_dma_descriptor_ring
- *ring, unsigned min_channel,
- unsigned max_channel)
-{
- int i;
- unsigned long flags;
- struct mite_channel *channel = NULL;
-
+ count = nbytes - async->buf_write_count;
/*
- * spin lock so mite_release_channel can be called safely
- * from interrupts
+ * it's possible count will be negative due to conservative value
+ * returned by mite_bytes_written_to_memory_lb
*/
- spin_lock_irqsave(&mite->lock, flags);
- for (i = min_channel; i <= max_channel; ++i) {
- if (mite->channel_allocated[i] == 0) {
- mite->channel_allocated[i] = 1;
- channel = &mite->channels[i];
- channel->ring = ring;
- break;
- }
+ if (count > 0) {
+ comedi_buf_write_free(s, count);
+ comedi_inc_scan_progress(s, count);
+ async->events |= COMEDI_CB_BLOCK;
}
- spin_unlock_irqrestore(&mite->lock, flags);
- return channel;
}
-EXPORT_SYMBOL_GPL(mite_request_channel_in_range);
-void mite_release_channel(struct mite_channel *mite_chan)
+static void mite_sync_output_dma(struct mite_channel *mite_chan,
+ struct comedi_subdevice *s)
{
- struct mite_struct *mite = mite_chan->mite;
+ struct comedi_async *async = s->async;
+ struct comedi_cmd *cmd = &async->cmd;
+ u32 stop_count = cmd->stop_arg * comedi_bytes_per_scan(s);
+ unsigned int old_alloc_count = async->buf_read_alloc_count;
+ u32 nbytes_ub, nbytes_lb;
+ int count;
+ bool finite_regen = (cmd->stop_src == TRIG_NONE && stop_count != 0);
+
+ /* read alloc as much as we can */
+ comedi_buf_read_alloc(s, async->prealloc_bufsz);
+ nbytes_lb = mite_bytes_read_from_memory_lb(mite_chan);
+ if (cmd->stop_src == TRIG_COUNT && (int)(nbytes_lb - stop_count) > 0)
+ nbytes_lb = stop_count;
+ nbytes_ub = mite_bytes_read_from_memory_ub(mite_chan);
+ if (cmd->stop_src == TRIG_COUNT && (int)(nbytes_ub - stop_count) > 0)
+ nbytes_ub = stop_count;
+
+ if ((!finite_regen || stop_count > old_alloc_count) &&
+ ((int)(nbytes_ub - old_alloc_count) > 0)) {
+ dev_warn(s->device->class_dev, "mite: DMA underrun\n");
+ async->events |= COMEDI_CB_OVERFLOW;
+ return;
+ }
+
+ if (finite_regen) {
+ /*
+ * This is a special case where we continuously output a finite
+ * buffer. In this case, we do not free any of the memory,
+ * hence we expect that old_alloc_count will reach a maximum of
+ * stop_count bytes.
+ */
+ return;
+ }
+
+ count = nbytes_lb - async->buf_read_count;
+ if (count > 0) {
+ comedi_buf_read_free(s, count);
+ async->events |= COMEDI_CB_BLOCK;
+ }
+}
+
+/**
+ * mite_sync_dma() - Sync the MITE dma with the COMEDI async buffer.
+ * @mite_chan: MITE dma channel.
+ * @s: COMEDI subdevice.
+ */
+void mite_sync_dma(struct mite_channel *mite_chan, struct comedi_subdevice *s)
+{
+ if (mite_chan->dir == COMEDI_INPUT)
+ mite_sync_input_dma(mite_chan, s);
+ else
+ mite_sync_output_dma(mite_chan, s);
+}
+EXPORT_SYMBOL_GPL(mite_sync_dma);
+
+static unsigned int mite_get_status(struct mite_channel *mite_chan)
+{
+ struct mite *mite = mite_chan->mite;
+ unsigned int status;
unsigned long flags;
- /* spin lock to prevent races with mite_request_channel */
spin_lock_irqsave(&mite->lock, flags);
- if (mite->channel_allocated[mite_chan->channel]) {
- mite_dma_disarm(mite_chan);
- mite_dma_reset(mite_chan);
- /*
- * disable all channel's interrupts (do it after disarm/reset so
- * MITE_CHCR reg isn't changed while dma is still active!)
- */
- writel(CHCR_CLR_DMA_IE | CHCR_CLR_LINKP_IE |
- CHCR_CLR_SAR_IE | CHCR_CLR_DONE_IE |
- CHCR_CLR_MRDY_IE | CHCR_CLR_DRDY_IE |
- CHCR_CLR_LC_IE | CHCR_CLR_CONT_RB_IE,
- mite->mite_io_addr + MITE_CHCR(mite_chan->channel));
- mite->channel_allocated[mite_chan->channel] = 0;
- mite_chan->ring = NULL;
- mmiowb();
+ status = readl(mite->mmio + MITE_CHSR(mite_chan->channel));
+ if (status & CHSR_DONE) {
+ mite_chan->done = 1;
+ writel(CHOR_CLRDONE,
+ mite->mmio + MITE_CHOR(mite_chan->channel));
}
+ mmiowb();
spin_unlock_irqrestore(&mite->lock, flags);
+ return status;
}
-EXPORT_SYMBOL_GPL(mite_release_channel);
+/**
+ * mite_ack_linkc() - Check and ack the LINKC interrupt,
+ * @mite_chan: MITE dma channel.
+ * @s: COMEDI subdevice.
+ * @sync: flag to force a mite_sync_dma().
+ *
+ * This will also ack the DONE interrupt if active.
+ */
+void mite_ack_linkc(struct mite_channel *mite_chan,
+ struct comedi_subdevice *s,
+ bool sync)
+{
+ struct mite *mite = mite_chan->mite;
+ unsigned int status;
+
+ status = mite_get_status(mite_chan);
+ if (status & CHSR_LINKC) {
+ writel(CHOR_CLRLC, mite->mmio + MITE_CHOR(mite_chan->channel));
+ sync = true;
+ }
+ if (sync)
+ mite_sync_dma(mite_chan, s);
+
+ if (status & CHSR_XFERR) {
+ dev_err(s->device->class_dev,
+ "mite: transfer error %08x\n", status);
+ s->async->events |= COMEDI_CB_ERROR;
+ }
+}
+EXPORT_SYMBOL_GPL(mite_ack_linkc);
+
+/**
+ * mite_done() - Check is a MITE dma transfer is complete.
+ * @mite_chan: MITE dma channel.
+ *
+ * This will also ack the DONE interrupt if active.
+ */
+int mite_done(struct mite_channel *mite_chan)
+{
+ struct mite *mite = mite_chan->mite;
+ unsigned long flags;
+ int done;
+
+ mite_get_status(mite_chan);
+ spin_lock_irqsave(&mite->lock, flags);
+ done = mite_chan->done;
+ spin_unlock_irqrestore(&mite->lock, flags);
+ return done;
+}
+EXPORT_SYMBOL_GPL(mite_done);
+
+static void mite_dma_reset(struct mite_channel *mite_chan)
+{
+ writel(CHOR_DMARESET | CHOR_FRESET,
+ mite_chan->mite->mmio + MITE_CHOR(mite_chan->channel));
+}
+
+/**
+ * mite_dma_arm() - Start a MITE dma transfer.
+ * @mite_chan: MITE dma channel.
+ */
void mite_dma_arm(struct mite_channel *mite_chan)
{
- struct mite_struct *mite = mite_chan->mite;
- int chor;
+ struct mite *mite = mite_chan->mite;
unsigned long flags;
/*
@@ -279,122 +456,41 @@
* is done before writing to the mite to arm dma transfer
*/
smp_mb();
- /* arm */
- chor = CHOR_START;
spin_lock_irqsave(&mite->lock, flags);
mite_chan->done = 0;
- writel(chor, mite->mite_io_addr + MITE_CHOR(mite_chan->channel));
+ /* arm */
+ writel(CHOR_START, mite->mmio + MITE_CHOR(mite_chan->channel));
mmiowb();
spin_unlock_irqrestore(&mite->lock, flags);
- /* mite_dma_tcr(mite, channel); */
}
EXPORT_SYMBOL_GPL(mite_dma_arm);
-/**************************************/
-
-int mite_buf_change(struct mite_dma_descriptor_ring *ring,
- struct comedi_subdevice *s)
-{
- struct comedi_async *async = s->async;
- unsigned int n_links;
-
- if (ring->descriptors) {
- dma_free_coherent(ring->hw_dev,
- ring->n_links *
- sizeof(struct mite_dma_descriptor),
- ring->descriptors,
- ring->descriptors_dma_addr);
- }
- ring->descriptors = NULL;
- ring->descriptors_dma_addr = 0;
- ring->n_links = 0;
-
- if (async->prealloc_bufsz == 0)
- return 0;
-
- n_links = async->prealloc_bufsz >> PAGE_SHIFT;
-
- ring->descriptors =
- dma_alloc_coherent(ring->hw_dev,
- n_links * sizeof(struct mite_dma_descriptor),
- &ring->descriptors_dma_addr, GFP_KERNEL);
- if (!ring->descriptors) {
- dev_err(s->device->class_dev,
- "mite: ring buffer allocation failed\n");
- return -ENOMEM;
- }
- ring->n_links = n_links;
-
- return mite_init_ring_descriptors(ring, s, n_links << PAGE_SHIFT);
-}
-EXPORT_SYMBOL_GPL(mite_buf_change);
-
-/*
- * initializes the ring buffer descriptors to provide correct DMA transfer links
- * to the exact amount of memory required. When the ring buffer is allocated in
- * mite_buf_change, the default is to initialize the ring to refer to the entire
- * DMA data buffer. A command may call this function later to re-initialize and
- * shorten the amount of memory that will be transferred.
+/**
+ * mite_dma_disarm() - Stop a MITE dma transfer.
+ * @mite_chan: MITE dma channel.
*/
-int mite_init_ring_descriptors(struct mite_dma_descriptor_ring *ring,
- struct comedi_subdevice *s,
- unsigned int nbytes)
+void mite_dma_disarm(struct mite_channel *mite_chan)
{
- struct comedi_async *async = s->async;
- unsigned int n_full_links = nbytes >> PAGE_SHIFT;
- unsigned int remainder = nbytes % PAGE_SIZE;
- int i;
+ struct mite *mite = mite_chan->mite;
- dev_dbg(s->device->class_dev,
- "mite: init ring buffer to %u bytes\n", nbytes);
-
- if ((n_full_links + (remainder > 0 ? 1 : 0)) > ring->n_links) {
- dev_err(s->device->class_dev,
- "mite: ring buffer too small for requested init\n");
- return -ENOMEM;
- }
-
- /* We set the descriptors for all full links. */
- for (i = 0; i < n_full_links; ++i) {
- ring->descriptors[i].count = cpu_to_le32(PAGE_SIZE);
- ring->descriptors[i].addr =
- cpu_to_le32(async->buf_map->page_list[i].dma_addr);
- ring->descriptors[i].next =
- cpu_to_le32(ring->descriptors_dma_addr +
- (i + 1) * sizeof(struct mite_dma_descriptor));
- }
-
- /* the last link is either a remainder or was a full link. */
- if (remainder > 0) {
- /* set the lesser count for the remainder link */
- ring->descriptors[i].count = cpu_to_le32(remainder);
- ring->descriptors[i].addr =
- cpu_to_le32(async->buf_map->page_list[i].dma_addr);
- /* increment i so that assignment below refs last link */
- ++i;
- }
-
- /* Assign the last link->next to point back to the head of the list. */
- ring->descriptors[i - 1].next = cpu_to_le32(ring->descriptors_dma_addr);
-
- /*
- * barrier is meant to insure that all the writes to the dma descriptors
- * have completed before the dma controller is commanded to read them
- */
- smp_wmb();
- return 0;
+ /* disarm */
+ writel(CHOR_ABORT, mite->mmio + MITE_CHOR(mite_chan->channel));
}
-EXPORT_SYMBOL_GPL(mite_init_ring_descriptors);
+EXPORT_SYMBOL_GPL(mite_dma_disarm);
+/**
+ * mite_prep_dma() - Prepare a MITE dma channel for transfers.
+ * @mite_chan: MITE dma channel.
+ * @num_device_bits: device transfer size (8, 16, or 32-bits).
+ * @num_memory_bits: memory transfer size (8, 16, or 32-bits).
+ */
void mite_prep_dma(struct mite_channel *mite_chan,
unsigned int num_device_bits, unsigned int num_memory_bits)
{
- unsigned int chor, chcr, mcr, dcr, lkcr;
- struct mite_struct *mite = mite_chan->mite;
+ struct mite *mite = mite_chan->mite;
+ unsigned int chcr, mcr, dcr, lkcr;
- /* reset DMA and FIFO */
- chor = CHOR_DMARESET | CHOR_FRESET;
- writel(chor, mite->mite_io_addr + MITE_CHOR(mite_chan->channel));
+ mite_dma_reset(mite_chan);
/* short link chaining mode */
chcr = CHCR_SET_DMA_IE | CHCR_LINKSHORT | CHCR_SET_DONE_IE |
@@ -421,10 +517,10 @@
if (mite_chan->dir == COMEDI_INPUT)
chcr |= CHCR_DEV_TO_MEM;
- writel(chcr, mite->mite_io_addr + MITE_CHCR(mite_chan->channel));
+ writel(chcr, mite->mmio + MITE_CHCR(mite_chan->channel));
/* to/from memory */
- mcr = CR_RL(64) | CR_ASEQUP;
+ mcr = mite_retry_limit(64) | CR_ASEQUP;
switch (num_memory_bits) {
case 8:
mcr |= CR_PSIZE8;
@@ -439,11 +535,11 @@
pr_warn("bug! invalid mem bit width for dma transfer\n");
break;
}
- writel(mcr, mite->mite_io_addr + MITE_MCR(mite_chan->channel));
+ writel(mcr, mite->mmio + MITE_MCR(mite_chan->channel));
/* from/to device */
- dcr = CR_RL(64) | CR_ASEQUP;
- dcr |= CR_PORTIO | CR_AMDEVICE | CR_REQSDRQ(mite_chan->channel);
+ dcr = mite_retry_limit(64) | CR_ASEQUP;
+ dcr |= CR_PORTIO | CR_AMDEVICE | mite_drq_reqs(mite_chan->channel);
switch (num_device_bits) {
case 8:
dcr |= CR_PSIZE8;
@@ -458,223 +554,402 @@
pr_warn("bug! invalid dev bit width for dma transfer\n");
break;
}
- writel(dcr, mite->mite_io_addr + MITE_DCR(mite_chan->channel));
+ writel(dcr, mite->mmio + MITE_DCR(mite_chan->channel));
/* reset the DAR */
- writel(0, mite->mite_io_addr + MITE_DAR(mite_chan->channel));
+ writel(0, mite->mmio + MITE_DAR(mite_chan->channel));
/* the link is 32bits */
- lkcr = CR_RL(64) | CR_ASEQUP | CR_PSIZE32;
- writel(lkcr, mite->mite_io_addr + MITE_LKCR(mite_chan->channel));
+ lkcr = mite_retry_limit(64) | CR_ASEQUP | CR_PSIZE32;
+ writel(lkcr, mite->mmio + MITE_LKCR(mite_chan->channel));
/* starting address for link chaining */
- writel(mite_chan->ring->descriptors_dma_addr,
- mite->mite_io_addr + MITE_LKAR(mite_chan->channel));
+ writel(mite_chan->ring->dma_addr,
+ mite->mmio + MITE_LKAR(mite_chan->channel));
}
EXPORT_SYMBOL_GPL(mite_prep_dma);
-static u32 mite_device_bytes_transferred(struct mite_channel *mite_chan)
+static struct mite_channel *__mite_request_channel(struct mite *mite,
+ struct mite_ring *ring,
+ unsigned int min_channel,
+ unsigned int max_channel)
{
- struct mite_struct *mite = mite_chan->mite;
+ struct mite_channel *mite_chan = NULL;
+ unsigned long flags;
+ int i;
- return readl(mite->mite_io_addr + MITE_DAR(mite_chan->channel));
-}
-
-u32 mite_bytes_in_transit(struct mite_channel *mite_chan)
-{
- struct mite_struct *mite = mite_chan->mite;
-
- return readl(mite->mite_io_addr +
- MITE_FCR(mite_chan->channel)) & 0x000000FF;
-}
-EXPORT_SYMBOL_GPL(mite_bytes_in_transit);
-
-/* returns lower bound for number of bytes transferred from device to memory */
-u32 mite_bytes_written_to_memory_lb(struct mite_channel *mite_chan)
-{
- u32 device_byte_count;
-
- device_byte_count = mite_device_bytes_transferred(mite_chan);
- return device_byte_count - mite_bytes_in_transit(mite_chan);
-}
-EXPORT_SYMBOL_GPL(mite_bytes_written_to_memory_lb);
-
-/* returns upper bound for number of bytes transferred from device to memory */
-u32 mite_bytes_written_to_memory_ub(struct mite_channel *mite_chan)
-{
- u32 in_transit_count;
-
- in_transit_count = mite_bytes_in_transit(mite_chan);
- return mite_device_bytes_transferred(mite_chan) - in_transit_count;
-}
-EXPORT_SYMBOL_GPL(mite_bytes_written_to_memory_ub);
-
-/* returns lower bound for number of bytes read from memory to device */
-u32 mite_bytes_read_from_memory_lb(struct mite_channel *mite_chan)
-{
- u32 device_byte_count;
-
- device_byte_count = mite_device_bytes_transferred(mite_chan);
- return device_byte_count + mite_bytes_in_transit(mite_chan);
-}
-EXPORT_SYMBOL_GPL(mite_bytes_read_from_memory_lb);
-
-/* returns upper bound for number of bytes read from memory to device */
-u32 mite_bytes_read_from_memory_ub(struct mite_channel *mite_chan)
-{
- u32 in_transit_count;
-
- in_transit_count = mite_bytes_in_transit(mite_chan);
- return mite_device_bytes_transferred(mite_chan) + in_transit_count;
-}
-EXPORT_SYMBOL_GPL(mite_bytes_read_from_memory_ub);
-
-unsigned mite_dma_tcr(struct mite_channel *mite_chan)
-{
- struct mite_struct *mite = mite_chan->mite;
-
- return readl(mite->mite_io_addr + MITE_TCR(mite_chan->channel));
-}
-EXPORT_SYMBOL_GPL(mite_dma_tcr);
-
-void mite_dma_disarm(struct mite_channel *mite_chan)
-{
- struct mite_struct *mite = mite_chan->mite;
- unsigned chor;
-
- /* disarm */
- chor = CHOR_ABORT;
- writel(chor, mite->mite_io_addr + MITE_CHOR(mite_chan->channel));
-}
-EXPORT_SYMBOL_GPL(mite_dma_disarm);
-
-int mite_sync_input_dma(struct mite_channel *mite_chan,
- struct comedi_subdevice *s)
-{
- struct comedi_async *async = s->async;
- int count;
- unsigned int nbytes, old_alloc_count;
-
- old_alloc_count = async->buf_write_alloc_count;
- /* write alloc as much as we can */
- comedi_buf_write_alloc(s, async->prealloc_bufsz);
-
- nbytes = mite_bytes_written_to_memory_lb(mite_chan);
- if ((int)(mite_bytes_written_to_memory_ub(mite_chan) -
- old_alloc_count) > 0) {
- dev_warn(s->device->class_dev,
- "mite: DMA overwrite of free area\n");
- async->events |= COMEDI_CB_OVERFLOW;
- return -1;
- }
-
- count = nbytes - async->buf_write_count;
/*
- * it's possible count will be negative due to conservative value
- * returned by mite_bytes_written_to_memory_lb
+ * spin lock so mite_release_channel can be called safely
+ * from interrupts
*/
- if (count <= 0)
- return 0;
-
- comedi_buf_write_free(s, count);
- comedi_inc_scan_progress(s, count);
- async->events |= COMEDI_CB_BLOCK;
- return 0;
+ spin_lock_irqsave(&mite->lock, flags);
+ for (i = min_channel; i <= max_channel; ++i) {
+ mite_chan = &mite->channels[i];
+ if (!mite_chan->ring) {
+ mite_chan->ring = ring;
+ break;
+ }
+ mite_chan = NULL;
+ }
+ spin_unlock_irqrestore(&mite->lock, flags);
+ return mite_chan;
}
-EXPORT_SYMBOL_GPL(mite_sync_input_dma);
-int mite_sync_output_dma(struct mite_channel *mite_chan,
- struct comedi_subdevice *s)
+/**
+ * mite_request_channel_in_range() - Request a MITE dma channel.
+ * @mite: MITE device.
+ * @ring: MITE dma ring.
+ * @min_channel: minimum channel index to use.
+ * @max_channel: maximum channel index to use.
+ */
+struct mite_channel *mite_request_channel_in_range(struct mite *mite,
+ struct mite_ring *ring,
+ unsigned int min_channel,
+ unsigned int max_channel)
+{
+ return __mite_request_channel(mite, ring, min_channel, max_channel);
+}
+EXPORT_SYMBOL_GPL(mite_request_channel_in_range);
+
+/**
+ * mite_request_channel() - Request a MITE dma channel.
+ * @mite: MITE device.
+ * @ring: MITE dma ring.
+ */
+struct mite_channel *mite_request_channel(struct mite *mite,
+ struct mite_ring *ring)
+{
+ return __mite_request_channel(mite, ring, 0, mite->num_channels - 1);
+}
+EXPORT_SYMBOL_GPL(mite_request_channel);
+
+/**
+ * mite_release_channel() - Release a MITE dma channel.
+ * @mite_chan: MITE dma channel.
+ */
+void mite_release_channel(struct mite_channel *mite_chan)
+{
+ struct mite *mite = mite_chan->mite;
+ unsigned long flags;
+
+ /* spin lock to prevent races with mite_request_channel */
+ spin_lock_irqsave(&mite->lock, flags);
+ if (mite_chan->ring) {
+ mite_dma_disarm(mite_chan);
+ mite_dma_reset(mite_chan);
+ /*
+ * disable all channel's interrupts (do it after disarm/reset so
+ * MITE_CHCR reg isn't changed while dma is still active!)
+ */
+ writel(CHCR_CLR_DMA_IE | CHCR_CLR_LINKP_IE |
+ CHCR_CLR_SAR_IE | CHCR_CLR_DONE_IE |
+ CHCR_CLR_MRDY_IE | CHCR_CLR_DRDY_IE |
+ CHCR_CLR_LC_IE | CHCR_CLR_CONT_RB_IE,
+ mite->mmio + MITE_CHCR(mite_chan->channel));
+ mite_chan->ring = NULL;
+ mmiowb();
+ }
+ spin_unlock_irqrestore(&mite->lock, flags);
+}
+EXPORT_SYMBOL_GPL(mite_release_channel);
+
+/**
+ * mite_init_ring_descriptors() - Initialize a MITE dma ring descriptors.
+ * @ring: MITE dma ring.
+ * @s: COMEDI subdevice.
+ * @nbytes: the size of the dma ring (in bytes).
+ *
+ * Initializes the ring buffer descriptors to provide correct DMA transfer
+ * links to the exact amount of memory required. When the ring buffer is
+ * allocated by mite_buf_change(), the default is to initialize the ring
+ * to refer to the entire DMA data buffer. A command may call this function
+ * later to re-initialize and shorten the amount of memory that will be
+ * transferred.
+ */
+int mite_init_ring_descriptors(struct mite_ring *ring,
+ struct comedi_subdevice *s,
+ unsigned int nbytes)
{
struct comedi_async *async = s->async;
- struct comedi_cmd *cmd = &async->cmd;
- u32 stop_count = cmd->stop_arg * comedi_bytes_per_scan(s);
- unsigned int old_alloc_count = async->buf_read_alloc_count;
- u32 nbytes_ub, nbytes_lb;
- int count;
- bool finite_regen = (cmd->stop_src == TRIG_NONE && stop_count != 0);
+ struct mite_dma_desc *desc = NULL;
+ unsigned int n_full_links = nbytes >> PAGE_SHIFT;
+ unsigned int remainder = nbytes % PAGE_SIZE;
+ int i;
- /* read alloc as much as we can */
- comedi_buf_read_alloc(s, async->prealloc_bufsz);
- nbytes_lb = mite_bytes_read_from_memory_lb(mite_chan);
- if (cmd->stop_src == TRIG_COUNT && (int)(nbytes_lb - stop_count) > 0)
- nbytes_lb = stop_count;
- nbytes_ub = mite_bytes_read_from_memory_ub(mite_chan);
- if (cmd->stop_src == TRIG_COUNT && (int)(nbytes_ub - stop_count) > 0)
- nbytes_ub = stop_count;
+ dev_dbg(s->device->class_dev,
+ "mite: init ring buffer to %u bytes\n", nbytes);
- if ((!finite_regen || stop_count > old_alloc_count) &&
- ((int)(nbytes_ub - old_alloc_count) > 0)) {
- dev_warn(s->device->class_dev, "mite: DMA underrun\n");
- async->events |= COMEDI_CB_OVERFLOW;
- return -1;
+ if ((n_full_links + (remainder > 0 ? 1 : 0)) > ring->n_links) {
+ dev_err(s->device->class_dev,
+ "mite: ring buffer too small for requested init\n");
+ return -ENOMEM;
}
- if (finite_regen) {
- /*
- * This is a special case where we continuously output a finite
- * buffer. In this case, we do not free any of the memory,
- * hence we expect that old_alloc_count will reach a maximum of
- * stop_count bytes.
- */
- return 0;
+ /* We set the descriptors for all full links. */
+ for (i = 0; i < n_full_links; ++i) {
+ desc = &ring->descs[i];
+ desc->count = cpu_to_le32(PAGE_SIZE);
+ desc->addr = cpu_to_le32(async->buf_map->page_list[i].dma_addr);
+ desc->next = cpu_to_le32(ring->dma_addr +
+ (i + 1) * sizeof(*desc));
}
- count = nbytes_lb - async->buf_read_count;
- if (count <= 0)
- return 0;
-
- if (count) {
- comedi_buf_read_free(s, count);
- async->events |= COMEDI_CB_BLOCK;
+ /* the last link is either a remainder or was a full link. */
+ if (remainder > 0) {
+ desc = &ring->descs[i];
+ /* set the lesser count for the remainder link */
+ desc->count = cpu_to_le32(remainder);
+ desc->addr = cpu_to_le32(async->buf_map->page_list[i].dma_addr);
}
+
+ /* Assign the last link->next to point back to the head of the list. */
+ desc->next = cpu_to_le32(ring->dma_addr);
+
+ /*
+ * barrier is meant to insure that all the writes to the dma descriptors
+ * have completed before the dma controller is commanded to read them
+ */
+ smp_wmb();
return 0;
}
-EXPORT_SYMBOL_GPL(mite_sync_output_dma);
+EXPORT_SYMBOL_GPL(mite_init_ring_descriptors);
-unsigned mite_get_status(struct mite_channel *mite_chan)
+static void mite_free_dma_descs(struct mite_ring *ring)
{
- struct mite_struct *mite = mite_chan->mite;
- unsigned status;
- unsigned long flags;
+ struct mite_dma_desc *descs = ring->descs;
- spin_lock_irqsave(&mite->lock, flags);
- status = readl(mite->mite_io_addr + MITE_CHSR(mite_chan->channel));
- if (status & CHSR_DONE) {
- mite_chan->done = 1;
- writel(CHOR_CLRDONE,
- mite->mite_io_addr + MITE_CHOR(mite_chan->channel));
+ if (descs) {
+ dma_free_coherent(ring->hw_dev,
+ ring->n_links * sizeof(*descs),
+ descs, ring->dma_addr);
+ ring->descs = NULL;
+ ring->dma_addr = 0;
+ ring->n_links = 0;
}
- mmiowb();
- spin_unlock_irqrestore(&mite->lock, flags);
- return status;
}
-EXPORT_SYMBOL_GPL(mite_get_status);
-int mite_done(struct mite_channel *mite_chan)
+/**
+ * mite_buf_change() - COMEDI subdevice (*buf_change) for a MITE dma ring.
+ * @ring: MITE dma ring.
+ * @s: COMEDI subdevice.
+ */
+int mite_buf_change(struct mite_ring *ring, struct comedi_subdevice *s)
{
- struct mite_struct *mite = mite_chan->mite;
- unsigned long flags;
- int done;
+ struct comedi_async *async = s->async;
+ struct mite_dma_desc *descs;
+ unsigned int n_links;
- mite_get_status(mite_chan);
- spin_lock_irqsave(&mite->lock, flags);
- done = mite_chan->done;
- spin_unlock_irqrestore(&mite->lock, flags);
- return done;
+ mite_free_dma_descs(ring);
+
+ if (async->prealloc_bufsz == 0)
+ return 0;
+
+ n_links = async->prealloc_bufsz >> PAGE_SHIFT;
+
+ descs = dma_alloc_coherent(ring->hw_dev,
+ n_links * sizeof(*descs),
+ &ring->dma_addr, GFP_KERNEL);
+ if (!descs) {
+ dev_err(s->device->class_dev,
+ "mite: ring buffer allocation failed\n");
+ return -ENOMEM;
+ }
+ ring->descs = descs;
+ ring->n_links = n_links;
+
+ return mite_init_ring_descriptors(ring, s, n_links << PAGE_SHIFT);
}
-EXPORT_SYMBOL_GPL(mite_done);
+EXPORT_SYMBOL_GPL(mite_buf_change);
+
+/**
+ * mite_alloc_ring() - Allocate a MITE dma ring.
+ * @mite: MITE device.
+ */
+struct mite_ring *mite_alloc_ring(struct mite *mite)
+{
+ struct mite_ring *ring;
+
+ ring = kmalloc(sizeof(*ring), GFP_KERNEL);
+ if (!ring)
+ return NULL;
+ ring->hw_dev = get_device(&mite->pcidev->dev);
+ if (!ring->hw_dev) {
+ kfree(ring);
+ return NULL;
+ }
+ ring->n_links = 0;
+ ring->descs = NULL;
+ ring->dma_addr = 0;
+ return ring;
+}
+EXPORT_SYMBOL_GPL(mite_alloc_ring);
+
+/**
+ * mite_free_ring() - Free a MITE dma ring and its descriptors.
+ * @ring: MITE dma ring.
+ */
+void mite_free_ring(struct mite_ring *ring)
+{
+ if (ring) {
+ mite_free_dma_descs(ring);
+ put_device(ring->hw_dev);
+ kfree(ring);
+ }
+}
+EXPORT_SYMBOL_GPL(mite_free_ring);
+
+static int mite_setup(struct comedi_device *dev, struct mite *mite,
+ bool use_win1)
+{
+ resource_size_t daq_phys_addr;
+ unsigned long length;
+ int i;
+ u32 csigr_bits;
+ unsigned int unknown_dma_burst_bits;
+ unsigned int wpdep;
+
+ pci_set_master(mite->pcidev);
+
+ mite->mmio = pci_ioremap_bar(mite->pcidev, 0);
+ if (!mite->mmio)
+ return -ENOMEM;
+
+ dev->mmio = pci_ioremap_bar(mite->pcidev, 1);
+ if (!dev->mmio)
+ return -ENOMEM;
+ daq_phys_addr = pci_resource_start(mite->pcidev, 1);
+ length = pci_resource_len(mite->pcidev, 1);
+
+ if (use_win1) {
+ writel(0, mite->mmio + MITE_IODWBSR);
+ dev_dbg(dev->class_dev,
+ "mite: using I/O Window Base Size register 1\n");
+ writel(daq_phys_addr | WENAB |
+ MITE_IODWBSR_1_WSIZE_bits(length),
+ mite->mmio + MITE_IODWBSR_1);
+ writel(0, mite->mmio + MITE_IODWCR_1);
+ } else {
+ writel(daq_phys_addr | WENAB, mite->mmio + MITE_IODWBSR);
+ }
+ /*
+ * Make sure dma bursts work. I got this from running a bus analyzer
+ * on a pxi-6281 and a pxi-6713. 6713 powered up with register value
+ * of 0x61f and bursts worked. 6281 powered up with register value of
+ * 0x1f and bursts didn't work. The NI windows driver reads the
+ * register, then does a bitwise-or of 0x600 with it and writes it back.
+ *
+ * The bits 0x90180700 in MITE_UNKNOWN_DMA_BURST_REG can be
+ * written and read back. The bits 0x1f always read as 1.
+ * The rest always read as zero.
+ */
+ unknown_dma_burst_bits = readl(mite->mmio + MITE_UNKNOWN_DMA_BURST_REG);
+ unknown_dma_burst_bits |= UNKNOWN_DMA_BURST_ENABLE_BITS;
+ writel(unknown_dma_burst_bits, mite->mmio + MITE_UNKNOWN_DMA_BURST_REG);
+
+ csigr_bits = readl(mite->mmio + MITE_CSIGR);
+ mite->num_channels = CSIGR_TO_DMAC(csigr_bits);
+ if (mite->num_channels > MAX_MITE_DMA_CHANNELS) {
+ dev_warn(dev->class_dev,
+ "mite: bug? chip claims to have %i dma channels. Setting to %i.\n",
+ mite->num_channels, MAX_MITE_DMA_CHANNELS);
+ mite->num_channels = MAX_MITE_DMA_CHANNELS;
+ }
+
+ /* get the wpdep bits and convert it to the write port fifo depth */
+ wpdep = CSIGR_TO_WPDEP(csigr_bits);
+ if (wpdep)
+ wpdep = BIT(wpdep);
+
+ dev_dbg(dev->class_dev,
+ "mite: version = %i, type = %i, mite mode = %i, interface mode = %i\n",
+ CSIGR_TO_VER(csigr_bits), CSIGR_TO_TYPE(csigr_bits),
+ CSIGR_TO_MMODE(csigr_bits), CSIGR_TO_IMODE(csigr_bits));
+ dev_dbg(dev->class_dev,
+ "mite: num channels = %i, write post fifo depth = %i, wins = %i, iowins = %i\n",
+ CSIGR_TO_DMAC(csigr_bits), wpdep,
+ CSIGR_TO_WINS(csigr_bits), CSIGR_TO_IOWINS(csigr_bits));
+
+ for (i = 0; i < mite->num_channels; i++) {
+ writel(CHOR_DMARESET, mite->mmio + MITE_CHOR(i));
+ /* disable interrupts */
+ writel(CHCR_CLR_DMA_IE | CHCR_CLR_LINKP_IE | CHCR_CLR_SAR_IE |
+ CHCR_CLR_DONE_IE | CHCR_CLR_MRDY_IE | CHCR_CLR_DRDY_IE |
+ CHCR_CLR_LC_IE | CHCR_CLR_CONT_RB_IE,
+ mite->mmio + MITE_CHCR(i));
+ }
+ mite->fifo_size = mite_fifo_size(mite, 0);
+ dev_dbg(dev->class_dev, "mite: fifo size is %i.\n", mite->fifo_size);
+ return 0;
+}
+
+/**
+ * mite_attach() - Allocate and initialize a MITE device for a comedi driver.
+ * @dev: COMEDI device.
+ * @use_win1: flag to use I/O Window 1 instead of I/O Window 0.
+ *
+ * Called by a COMEDI drivers (*auto_attach).
+ *
+ * Returns a pointer to the MITE device on success, or NULL if the MITE cannot
+ * be allocated or remapped.
+ */
+struct mite *mite_attach(struct comedi_device *dev, bool use_win1)
+{
+ struct pci_dev *pcidev = comedi_to_pci_dev(dev);
+ struct mite *mite;
+ unsigned int i;
+ int ret;
+
+ mite = kzalloc(sizeof(*mite), GFP_KERNEL);
+ if (!mite)
+ return NULL;
+
+ spin_lock_init(&mite->lock);
+ mite->pcidev = pcidev;
+ for (i = 0; i < MAX_MITE_DMA_CHANNELS; ++i) {
+ mite->channels[i].mite = mite;
+ mite->channels[i].channel = i;
+ mite->channels[i].done = 1;
+ }
+
+ ret = mite_setup(dev, mite, use_win1);
+ if (ret) {
+ if (mite->mmio)
+ iounmap(mite->mmio);
+ kfree(mite);
+ return NULL;
+ }
+
+ return mite;
+}
+EXPORT_SYMBOL_GPL(mite_attach);
+
+/**
+ * mite_detach() - Unmap and free a MITE device for a comedi driver.
+ * @mite: MITE device.
+ *
+ * Called by a COMEDI drivers (*detach).
+ */
+void mite_detach(struct mite *mite)
+{
+ if (!mite)
+ return;
+
+ if (mite->mmio)
+ iounmap(mite->mmio);
+
+ kfree(mite);
+}
+EXPORT_SYMBOL_GPL(mite_detach);
static int __init mite_module_init(void)
{
return 0;
}
+module_init(mite_module_init);
static void __exit mite_module_exit(void)
{
}
-
-module_init(mite_module_init);
module_exit(mite_module_exit);
MODULE_AUTHOR("Comedi http://www.comedi.org");
diff --git a/drivers/staging/comedi/drivers/mite.h b/drivers/staging/comedi/drivers/mite.h
index 87534b0..b6349ae 100644
--- a/drivers/staging/comedi/drivers/mite.h
+++ b/drivers/staging/comedi/drivers/mite.h
@@ -19,8 +19,6 @@
#ifndef _MITE_H_
#define _MITE_H_
-#include <linux/io.h>
-#include <linux/log2.h>
#include <linux/spinlock.h>
#define MAX_MITE_DMA_CHANNELS 8
@@ -30,323 +28,74 @@
struct device;
struct pci_dev;
-struct mite_dma_descriptor {
+struct mite_dma_desc {
__le32 count;
__le32 addr;
__le32 next;
u32 dar;
};
-struct mite_dma_descriptor_ring {
+struct mite_ring {
struct device *hw_dev;
unsigned int n_links;
- struct mite_dma_descriptor *descriptors;
- dma_addr_t descriptors_dma_addr;
+ struct mite_dma_desc *descs;
+ dma_addr_t dma_addr;
};
struct mite_channel {
- struct mite_struct *mite;
- unsigned channel;
+ struct mite *mite;
+ unsigned int channel;
int dir;
int done;
- struct mite_dma_descriptor_ring *ring;
+ struct mite_ring *ring;
};
-struct mite_struct {
+struct mite {
struct pci_dev *pcidev;
- resource_size_t mite_phys_addr;
- void __iomem *mite_io_addr;
- resource_size_t daq_phys_addr;
+ void __iomem *mmio;
struct mite_channel channels[MAX_MITE_DMA_CHANNELS];
- short channel_allocated[MAX_MITE_DMA_CHANNELS];
int num_channels;
- unsigned fifo_size;
+ unsigned int fifo_size;
+ /* protects mite_channel from being released by the driver */
spinlock_t lock;
};
-struct mite_struct *mite_alloc(struct pci_dev *pcidev);
+u32 mite_bytes_in_transit(struct mite_channel *);
-int mite_setup2(struct comedi_device *, struct mite_struct *, bool use_win1);
+void mite_sync_dma(struct mite_channel *, struct comedi_subdevice *);
+void mite_ack_linkc(struct mite_channel *, struct comedi_subdevice *s,
+ bool sync);
+int mite_done(struct mite_channel *);
-static inline int mite_setup(struct comedi_device *dev,
- struct mite_struct *mite)
-{
- return mite_setup2(dev, mite, false);
-}
+void mite_dma_arm(struct mite_channel *);
+void mite_dma_disarm(struct mite_channel *);
-void mite_detach(struct mite_struct *mite);
-struct mite_dma_descriptor_ring *mite_alloc_ring(struct mite_struct *mite);
-void mite_free_ring(struct mite_dma_descriptor_ring *ring);
-struct mite_channel *
-mite_request_channel_in_range(struct mite_struct *mite,
- struct mite_dma_descriptor_ring *ring,
- unsigned min_channel, unsigned max_channel);
-static inline struct mite_channel *
-mite_request_channel(struct mite_struct *mite,
- struct mite_dma_descriptor_ring *ring)
-{
- return mite_request_channel_in_range(mite, ring, 0,
- mite->num_channels - 1);
-}
-
-void mite_release_channel(struct mite_channel *mite_chan);
-
-unsigned mite_dma_tcr(struct mite_channel *mite_chan);
-void mite_dma_arm(struct mite_channel *mite_chan);
-void mite_dma_disarm(struct mite_channel *mite_chan);
-int mite_sync_input_dma(struct mite_channel *mite_chan,
- struct comedi_subdevice *s);
-int mite_sync_output_dma(struct mite_channel *mite_chan,
- struct comedi_subdevice *s);
-u32 mite_bytes_written_to_memory_lb(struct mite_channel *mite_chan);
-u32 mite_bytes_written_to_memory_ub(struct mite_channel *mite_chan);
-u32 mite_bytes_read_from_memory_lb(struct mite_channel *mite_chan);
-u32 mite_bytes_read_from_memory_ub(struct mite_channel *mite_chan);
-u32 mite_bytes_in_transit(struct mite_channel *mite_chan);
-unsigned mite_get_status(struct mite_channel *mite_chan);
-int mite_done(struct mite_channel *mite_chan);
-
-void mite_prep_dma(struct mite_channel *mite_chan,
+void mite_prep_dma(struct mite_channel *,
unsigned int num_device_bits, unsigned int num_memory_bits);
-int mite_buf_change(struct mite_dma_descriptor_ring *ring,
- struct comedi_subdevice *s);
-int mite_init_ring_descriptors(struct mite_dma_descriptor_ring *ring,
- struct comedi_subdevice *s,
+
+struct mite_channel *mite_request_channel_in_range(struct mite *,
+ struct mite_ring *,
+ unsigned int min_channel,
+ unsigned int max_channel);
+struct mite_channel *mite_request_channel(struct mite *, struct mite_ring *);
+void mite_release_channel(struct mite_channel *);
+
+int mite_init_ring_descriptors(struct mite_ring *, struct comedi_subdevice *,
unsigned int nbytes);
+int mite_buf_change(struct mite_ring *, struct comedi_subdevice *);
-enum mite_registers {
- /*
- * The bits 0x90180700 in MITE_UNKNOWN_DMA_BURST_REG can be
- * written and read back. The bits 0x1f always read as 1.
- * The rest always read as zero.
- */
- MITE_UNKNOWN_DMA_BURST_REG = 0x28,
- MITE_IODWBSR = 0xc0, /* IO Device Window Base Size Register */
- MITE_IODWBSR_1 = 0xc4, /* IO Device Window Base Size Register 1 */
- MITE_IODWCR_1 = 0xf4,
- MITE_PCI_CONFIG_OFFSET = 0x300,
- MITE_CSIGR = 0x460 /* chip signature */
-};
+struct mite_ring *mite_alloc_ring(struct mite *);
+void mite_free_ring(struct mite_ring *);
-#define MITE_CHAN(x) (0x500 + 0x100 * (x))
-#define MITE_CHOR(x) (0x00 + MITE_CHAN(x)) /* channel operation */
-#define MITE_CHCR(x) (0x04 + MITE_CHAN(x)) /* channel control */
-#define MITE_TCR(x) (0x08 + MITE_CHAN(x)) /* transfer count */
-#define MITE_MCR(x) (0x0c + MITE_CHAN(x)) /* memory configuration */
-#define MITE_MAR(x) (0x10 + MITE_CHAN(x)) /* memory address */
-#define MITE_DCR(x) (0x14 + MITE_CHAN(x)) /* device configuration */
-#define MITE_DAR(x) (0x18 + MITE_CHAN(x)) /* device address */
-#define MITE_LKCR(x) (0x1c + MITE_CHAN(x)) /* link configuration */
-#define MITE_LKAR(x) (0x20 + MITE_CHAN(x)) /* link address */
-#define MITE_LLKAR(x) (0x24 + MITE_CHAN(x)) /* see tnt5002 manual */
-#define MITE_BAR(x) (0x28 + MITE_CHAN(x)) /* base address */
-#define MITE_BCR(x) (0x2c + MITE_CHAN(x)) /* base count */
-#define MITE_SAR(x) (0x30 + MITE_CHAN(x)) /* ? address */
-#define MITE_WSCR(x) (0x34 + MITE_CHAN(x)) /* ? */
-#define MITE_WSER(x) (0x38 + MITE_CHAN(x)) /* ? */
-#define MITE_CHSR(x) (0x3c + MITE_CHAN(x)) /* channel status */
-#define MITE_FCR(x) (0x40 + MITE_CHAN(x)) /* fifo count */
+struct mite *mite_attach(struct comedi_device *, bool use_win1);
+void mite_detach(struct mite *);
-enum MITE_IODWBSR_bits {
- WENAB = 0x80, /* window enable */
-};
-
-static inline unsigned MITE_IODWBSR_1_WSIZE_bits(unsigned size)
-{
- unsigned order = 0;
-
- BUG_ON(size == 0);
- order = ilog2(size);
- BUG_ON(order < 1);
- return (order - 1) & 0x1f;
-}
-
-enum MITE_UNKNOWN_DMA_BURST_bits {
- UNKNOWN_DMA_BURST_ENABLE_BITS = 0x600
-};
-
-static inline int mite_csigr_version(u32 csigr_bits)
-{
- return csigr_bits & 0xf;
-};
-
-static inline int mite_csigr_type(u32 csigr_bits)
-{ /* original mite = 0, minimite = 1 */
- return (csigr_bits >> 4) & 0xf;
-};
-
-static inline int mite_csigr_mmode(u32 csigr_bits)
-{ /* mite mode, minimite = 1 */
- return (csigr_bits >> 8) & 0x3;
-};
-
-static inline int mite_csigr_imode(u32 csigr_bits)
-{ /* cpu port interface mode, pci = 0x3 */
- return (csigr_bits >> 12) & 0x3;
-};
-
-static inline int mite_csigr_dmac(u32 csigr_bits)
-{ /* number of dma channels */
- return (csigr_bits >> 16) & 0xf;
-};
-
-static inline int mite_csigr_wpdep(u32 csigr_bits)
-{ /* write post fifo depth */
- unsigned int wpdep_bits = (csigr_bits >> 20) & 0x7;
-
- return (wpdep_bits) ? (1 << (wpdep_bits - 1)) : 0;
-}
-
-static inline int mite_csigr_wins(u32 csigr_bits)
-{
- return (csigr_bits >> 24) & 0x1f;
-};
-
-static inline int mite_csigr_iowins(u32 csigr_bits)
-{ /* number of io windows */
- return (csigr_bits >> 29) & 0x7;
-};
-
-enum MITE_MCR_bits {
- MCRPON = 0,
-};
-
-enum MITE_DCR_bits {
- DCR_NORMAL = (1 << 29),
- DCRPON = 0,
-};
-
-enum MITE_CHOR_bits {
- CHOR_DMARESET = (1 << 31),
- CHOR_SET_SEND_TC = (1 << 11),
- CHOR_CLR_SEND_TC = (1 << 10),
- CHOR_SET_LPAUSE = (1 << 9),
- CHOR_CLR_LPAUSE = (1 << 8),
- CHOR_CLRDONE = (1 << 7),
- CHOR_CLRRB = (1 << 6),
- CHOR_CLRLC = (1 << 5),
- CHOR_FRESET = (1 << 4),
- CHOR_ABORT = (1 << 3), /* stop without emptying fifo */
- CHOR_STOP = (1 << 2), /* stop after emptying fifo */
- CHOR_CONT = (1 << 1),
- CHOR_START = (1 << 0),
- CHOR_PON = (CHOR_CLR_SEND_TC | CHOR_CLR_LPAUSE),
-};
-
-enum MITE_CHCR_bits {
- CHCR_SET_DMA_IE = (1 << 31),
- CHCR_CLR_DMA_IE = (1 << 30),
- CHCR_SET_LINKP_IE = (1 << 29),
- CHCR_CLR_LINKP_IE = (1 << 28),
- CHCR_SET_SAR_IE = (1 << 27),
- CHCR_CLR_SAR_IE = (1 << 26),
- CHCR_SET_DONE_IE = (1 << 25),
- CHCR_CLR_DONE_IE = (1 << 24),
- CHCR_SET_MRDY_IE = (1 << 23),
- CHCR_CLR_MRDY_IE = (1 << 22),
- CHCR_SET_DRDY_IE = (1 << 21),
- CHCR_CLR_DRDY_IE = (1 << 20),
- CHCR_SET_LC_IE = (1 << 19),
- CHCR_CLR_LC_IE = (1 << 18),
- CHCR_SET_CONT_RB_IE = (1 << 17),
- CHCR_CLR_CONT_RB_IE = (1 << 16),
- CHCR_FIFODIS = (1 << 15),
- CHCR_FIFO_ON = 0,
- CHCR_BURSTEN = (1 << 14),
- CHCR_NO_BURSTEN = 0,
- CHCR_BYTE_SWAP_DEVICE = (1 << 6),
- CHCR_BYTE_SWAP_MEMORY = (1 << 4),
- CHCR_DIR = (1 << 3),
- CHCR_DEV_TO_MEM = CHCR_DIR,
- CHCR_MEM_TO_DEV = 0,
- CHCR_NORMAL = (0 << 0),
- CHCR_CONTINUE = (1 << 0),
- CHCR_RINGBUFF = (2 << 0),
- CHCR_LINKSHORT = (4 << 0),
- CHCR_LINKLONG = (5 << 0),
- CHCRPON =
- (CHCR_CLR_DMA_IE | CHCR_CLR_LINKP_IE | CHCR_CLR_SAR_IE |
- CHCR_CLR_DONE_IE | CHCR_CLR_MRDY_IE | CHCR_CLR_DRDY_IE |
- CHCR_CLR_LC_IE | CHCR_CLR_CONT_RB_IE),
-};
-
-enum ConfigRegister_bits {
- CR_REQS_MASK = 0x7 << 16,
- CR_ASEQDONT = 0x0 << 10,
- CR_ASEQUP = 0x1 << 10,
- CR_ASEQDOWN = 0x2 << 10,
- CR_ASEQ_MASK = 0x3 << 10,
- CR_PSIZE8 = (1 << 8),
- CR_PSIZE16 = (2 << 8),
- CR_PSIZE32 = (3 << 8),
- CR_PORTCPU = (0 << 6),
- CR_PORTIO = (1 << 6),
- CR_PORTVXI = (2 << 6),
- CR_PORTMXI = (3 << 6),
- CR_AMDEVICE = (1 << 0),
-};
-
-static inline int CR_REQS(int source)
-{
- return (source & 0x7) << 16;
-};
-
-static inline int CR_REQSDRQ(unsigned drq_line)
-{
- /* This also works on m-series when using channels (drq_line) 4 or 5. */
- return CR_REQS((drq_line & 0x3) | 0x4);
-}
-
-static inline int CR_RL(unsigned int retry_limit)
-{
- int value = 0;
-
- if (retry_limit)
- value = 1 + ilog2(retry_limit);
- if (value > 0x7)
- value = 0x7;
- return (value & 0x7) << 21;
-}
-
-enum CHSR_bits {
- CHSR_INT = (1 << 31),
- CHSR_LPAUSES = (1 << 29),
- CHSR_SARS = (1 << 27),
- CHSR_DONE = (1 << 25),
- CHSR_MRDY = (1 << 23),
- CHSR_DRDY = (1 << 21),
- CHSR_LINKC = (1 << 19),
- CHSR_CONTS_RB = (1 << 17),
- CHSR_ERROR = (1 << 15),
- CHSR_SABORT = (1 << 14),
- CHSR_HABORT = (1 << 13),
- CHSR_STOPS = (1 << 12),
- CHSR_OPERR_mask = (3 << 10),
- CHSR_OPERR_NOERROR = (0 << 10),
- CHSR_OPERR_FIFOERROR = (1 << 10),
- CHSR_OPERR_LINKERROR = (1 << 10), /* ??? */
- CHSR_XFERR = (1 << 9),
- CHSR_END = (1 << 8),
- CHSR_DRQ1 = (1 << 7),
- CHSR_DRQ0 = (1 << 6),
- CHSR_LxERR_mask = (3 << 4),
- CHSR_LBERR = (1 << 4),
- CHSR_LRERR = (2 << 4),
- CHSR_LOERR = (3 << 4),
- CHSR_MxERR_mask = (3 << 2),
- CHSR_MBERR = (1 << 2),
- CHSR_MRERR = (2 << 2),
- CHSR_MOERR = (3 << 2),
- CHSR_DxERR_mask = (3 << 0),
- CHSR_DBERR = (1 << 0),
- CHSR_DRERR = (2 << 0),
- CHSR_DOERR = (3 << 0),
-};
-
-static inline void mite_dma_reset(struct mite_channel *mite_chan)
-{
- writel(CHOR_DMARESET | CHOR_FRESET,
- mite_chan->mite->mite_io_addr + MITE_CHOR(mite_chan->channel));
-};
+/*
+ * Mite registers (used outside of the mite driver)
+ */
+#define MITE_IODWBSR 0xc0 /* IO Device Window Base Size */
+#define MITE_IODWBSR_1 0xc4 /* IO Device Window1 Base Size */
+#define WENAB BIT(7) /* window enable */
+#define MITE_IODWCR_1 0xf4
#endif
diff --git a/drivers/staging/comedi/drivers/ni_660x.c b/drivers/staging/comedi/drivers/ni_660x.c
index 46647c6..0dcb826 100644
--- a/drivers/staging/comedi/drivers/ni_660x.c
+++ b/drivers/staging/comedi/drivers/ni_660x.c
@@ -1,17 +1,16 @@
/*
- comedi/drivers/ni_660x.c
- Hardware driver for NI 660x devices
-
- This program is free software; you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation; either version 2 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-*/
+ * Hardware driver for NI 660x devices
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
/*
* Driver: ni_660x
@@ -42,91 +41,13 @@
#include "mite.h"
#include "ni_tio.h"
-enum ni_660x_constants {
- min_counter_pfi_chan = 8,
- max_dio_pfi_chan = 31,
- counters_per_chip = 4
-};
-
-#define NUM_PFI_CHANNELS 40
-/* really there are only up to 3 dma channels, but the register layout allows
-for 4 */
-#define MAX_DMA_CHANNEL 4
-
/* See Register-Level Programmer Manual page 3.1 */
enum ni_660x_register {
- NI660X_G0_INT_ACK,
- NI660X_G0_STATUS,
- NI660X_G1_INT_ACK,
- NI660X_G1_STATUS,
- NI660X_G01_STATUS,
- NI660X_G0_CMD,
- NI660X_STC_DIO_PARALLEL_INPUT,
- NI660X_G1_CMD,
- NI660X_G0_HW_SAVE,
- NI660X_G1_HW_SAVE,
+ /* see enum ni_gpct_register */
+ NI660X_STC_DIO_PARALLEL_INPUT = NITIO_NUM_REGS,
NI660X_STC_DIO_OUTPUT,
NI660X_STC_DIO_CONTROL,
- NI660X_G0_SW_SAVE,
- NI660X_G1_SW_SAVE,
- NI660X_G0_MODE,
- NI660X_G01_STATUS1,
- NI660X_G1_MODE,
NI660X_STC_DIO_SERIAL_INPUT,
- NI660X_G0_LOADA,
- NI660X_G01_STATUS2,
- NI660X_G0_LOADB,
- NI660X_G1_LOADA,
- NI660X_G1_LOADB,
- NI660X_G0_INPUT_SEL,
- NI660X_G1_INPUT_SEL,
- NI660X_G0_AUTO_INC,
- NI660X_G1_AUTO_INC,
- NI660X_G01_RESET,
- NI660X_G0_INT_ENA,
- NI660X_G1_INT_ENA,
- NI660X_G0_CNT_MODE,
- NI660X_G1_CNT_MODE,
- NI660X_G0_GATE2,
- NI660X_G1_GATE2,
- NI660X_G0_DMA_CFG,
- NI660X_G0_DMA_STATUS,
- NI660X_G1_DMA_CFG,
- NI660X_G1_DMA_STATUS,
- NI660X_G2_INT_ACK,
- NI660X_G2_STATUS,
- NI660X_G3_INT_ACK,
- NI660X_G3_STATUS,
- NI660X_G23_STATUS,
- NI660X_G2_CMD,
- NI660X_G3_CMD,
- NI660X_G2_HW_SAVE,
- NI660X_G3_HW_SAVE,
- NI660X_G2_SW_SAVE,
- NI660X_G3_SW_SAVE,
- NI660X_G2_MODE,
- NI660X_G23_STATUS1,
- NI660X_G3_MODE,
- NI660X_G2_LOADA,
- NI660X_G23_STATUS2,
- NI660X_G2_LOADB,
- NI660X_G3_LOADA,
- NI660X_G3_LOADB,
- NI660X_G2_INPUT_SEL,
- NI660X_G3_INPUT_SEL,
- NI660X_G2_AUTO_INC,
- NI660X_G3_AUTO_INC,
- NI660X_G23_RESET,
- NI660X_G2_INT_ENA,
- NI660X_G3_INT_ENA,
- NI660X_G2_CNT_MODE,
- NI660X_G3_CNT_MODE,
- NI660X_G3_GATE2,
- NI660X_G2_GATE2,
- NI660X_G2_DMA_CFG,
- NI660X_G2_DMA_STATUS,
- NI660X_G3_DMA_CFG,
- NI660X_G3_DMA_STATUS,
NI660X_DIO32_INPUT,
NI660X_DIO32_OUTPUT,
NI660X_CLK_CFG,
@@ -156,224 +77,134 @@
NI660X_NUM_REGS,
};
-static inline unsigned IOConfigReg(unsigned pfi_channel)
-{
- unsigned reg = NI660X_IO_CFG_0_1 + pfi_channel / 2;
+#define NI660X_CLK_CFG_COUNTER_SWAP BIT(21)
- BUG_ON(reg > NI660X_IO_CFG_38_39);
- return reg;
-}
+#define NI660X_GLOBAL_INT_COUNTER0 BIT(8)
+#define NI660X_GLOBAL_INT_COUNTER1 BIT(9)
+#define NI660X_GLOBAL_INT_COUNTER2 BIT(10)
+#define NI660X_GLOBAL_INT_COUNTER3 BIT(11)
+#define NI660X_GLOBAL_INT_CASCADE BIT(29)
+#define NI660X_GLOBAL_INT_GLOBAL_POL BIT(30)
+#define NI660X_GLOBAL_INT_GLOBAL BIT(31)
-enum ni_660x_register_width {
- DATA_1B,
- DATA_2B,
- DATA_4B
-};
+#define NI660X_DMA_CFG_SEL(_c, _s) (((_s) & 0x1f) << (8 * (_c)))
+#define NI660X_DMA_CFG_SEL_MASK(_c) NI660X_DMA_CFG_SEL((_c), 0x1f)
+#define NI660X_DMA_CFG_SEL_NONE(_c) NI660X_DMA_CFG_SEL((_c), 0x1f)
+#define NI660X_DMA_CFG_RESET(_c) NI660X_DMA_CFG_SEL((_c), 0x80)
-enum ni_660x_register_direction {
- NI_660x_READ,
- NI_660x_WRITE,
- NI_660x_READ_WRITE
-};
+#define NI660X_IO_CFG(x) (NI660X_IO_CFG_0_1 + ((x) / 2))
+#define NI660X_IO_CFG_OUT_SEL(_c, _s) (((_s) & 0x3) << (((_c) % 2) ? 0 : 8))
+#define NI660X_IO_CFG_OUT_SEL_MASK(_c) NI660X_IO_CFG_OUT_SEL((_c), 0x3)
+#define NI660X_IO_CFG_IN_SEL(_c, _s) (((_s) & 0x7) << (((_c) % 2) ? 4 : 12))
+#define NI660X_IO_CFG_IN_SEL_MASK(_c) NI660X_IO_CFG_IN_SEL((_c), 0x7)
-enum ni_660x_pfi_output_select {
- pfi_output_select_high_Z = 0,
- pfi_output_select_counter = 1,
- pfi_output_select_do = 2,
- num_pfi_output_selects
-};
-
-enum ni_660x_subdevices {
- NI_660X_DIO_SUBDEV = 1,
- NI_660X_GPCT_SUBDEV_0 = 2
-};
-static inline unsigned NI_660X_GPCT_SUBDEV(unsigned index)
-{
- return NI_660X_GPCT_SUBDEV_0 + index;
-}
-
-struct NI_660xRegisterData {
- const char *name; /* Register Name */
+struct ni_660x_register_data {
int offset; /* Offset from base address from GPCT chip */
- enum ni_660x_register_direction direction;
- enum ni_660x_register_width size; /* 1 byte, 2 bytes, or 4 bytes */
+ char size; /* 2 or 4 bytes */
};
-static const struct NI_660xRegisterData registerData[NI660X_NUM_REGS] = {
- {"G0 Interrupt Acknowledge", 0x004, NI_660x_WRITE, DATA_2B},
- {"G0 Status Register", 0x004, NI_660x_READ, DATA_2B},
- {"G1 Interrupt Acknowledge", 0x006, NI_660x_WRITE, DATA_2B},
- {"G1 Status Register", 0x006, NI_660x_READ, DATA_2B},
- {"G01 Status Register ", 0x008, NI_660x_READ, DATA_2B},
- {"G0 Command Register", 0x00C, NI_660x_WRITE, DATA_2B},
- {"STC DIO Parallel Input", 0x00E, NI_660x_READ, DATA_2B},
- {"G1 Command Register", 0x00E, NI_660x_WRITE, DATA_2B},
- {"G0 HW Save Register", 0x010, NI_660x_READ, DATA_4B},
- {"G1 HW Save Register", 0x014, NI_660x_READ, DATA_4B},
- {"STC DIO Output", 0x014, NI_660x_WRITE, DATA_2B},
- {"STC DIO Control", 0x016, NI_660x_WRITE, DATA_2B},
- {"G0 SW Save Register", 0x018, NI_660x_READ, DATA_4B},
- {"G1 SW Save Register", 0x01C, NI_660x_READ, DATA_4B},
- {"G0 Mode Register", 0x034, NI_660x_WRITE, DATA_2B},
- {"G01 Joint Status 1 Register", 0x036, NI_660x_READ, DATA_2B},
- {"G1 Mode Register", 0x036, NI_660x_WRITE, DATA_2B},
- {"STC DIO Serial Input", 0x038, NI_660x_READ, DATA_2B},
- {"G0 Load A Register", 0x038, NI_660x_WRITE, DATA_4B},
- {"G01 Joint Status 2 Register", 0x03A, NI_660x_READ, DATA_2B},
- {"G0 Load B Register", 0x03C, NI_660x_WRITE, DATA_4B},
- {"G1 Load A Register", 0x040, NI_660x_WRITE, DATA_4B},
- {"G1 Load B Register", 0x044, NI_660x_WRITE, DATA_4B},
- {"G0 Input Select Register", 0x048, NI_660x_WRITE, DATA_2B},
- {"G1 Input Select Register", 0x04A, NI_660x_WRITE, DATA_2B},
- {"G0 Autoincrement Register", 0x088, NI_660x_WRITE, DATA_2B},
- {"G1 Autoincrement Register", 0x08A, NI_660x_WRITE, DATA_2B},
- {"G01 Joint Reset Register", 0x090, NI_660x_WRITE, DATA_2B},
- {"G0 Interrupt Enable", 0x092, NI_660x_WRITE, DATA_2B},
- {"G1 Interrupt Enable", 0x096, NI_660x_WRITE, DATA_2B},
- {"G0 Counting Mode Register", 0x0B0, NI_660x_WRITE, DATA_2B},
- {"G1 Counting Mode Register", 0x0B2, NI_660x_WRITE, DATA_2B},
- {"G0 Second Gate Register", 0x0B4, NI_660x_WRITE, DATA_2B},
- {"G1 Second Gate Register", 0x0B6, NI_660x_WRITE, DATA_2B},
- {"G0 DMA Config Register", 0x0B8, NI_660x_WRITE, DATA_2B},
- {"G0 DMA Status Register", 0x0B8, NI_660x_READ, DATA_2B},
- {"G1 DMA Config Register", 0x0BA, NI_660x_WRITE, DATA_2B},
- {"G1 DMA Status Register", 0x0BA, NI_660x_READ, DATA_2B},
- {"G2 Interrupt Acknowledge", 0x104, NI_660x_WRITE, DATA_2B},
- {"G2 Status Register", 0x104, NI_660x_READ, DATA_2B},
- {"G3 Interrupt Acknowledge", 0x106, NI_660x_WRITE, DATA_2B},
- {"G3 Status Register", 0x106, NI_660x_READ, DATA_2B},
- {"G23 Status Register", 0x108, NI_660x_READ, DATA_2B},
- {"G2 Command Register", 0x10C, NI_660x_WRITE, DATA_2B},
- {"G3 Command Register", 0x10E, NI_660x_WRITE, DATA_2B},
- {"G2 HW Save Register", 0x110, NI_660x_READ, DATA_4B},
- {"G3 HW Save Register", 0x114, NI_660x_READ, DATA_4B},
- {"G2 SW Save Register", 0x118, NI_660x_READ, DATA_4B},
- {"G3 SW Save Register", 0x11C, NI_660x_READ, DATA_4B},
- {"G2 Mode Register", 0x134, NI_660x_WRITE, DATA_2B},
- {"G23 Joint Status 1 Register", 0x136, NI_660x_READ, DATA_2B},
- {"G3 Mode Register", 0x136, NI_660x_WRITE, DATA_2B},
- {"G2 Load A Register", 0x138, NI_660x_WRITE, DATA_4B},
- {"G23 Joint Status 2 Register", 0x13A, NI_660x_READ, DATA_2B},
- {"G2 Load B Register", 0x13C, NI_660x_WRITE, DATA_4B},
- {"G3 Load A Register", 0x140, NI_660x_WRITE, DATA_4B},
- {"G3 Load B Register", 0x144, NI_660x_WRITE, DATA_4B},
- {"G2 Input Select Register", 0x148, NI_660x_WRITE, DATA_2B},
- {"G3 Input Select Register", 0x14A, NI_660x_WRITE, DATA_2B},
- {"G2 Autoincrement Register", 0x188, NI_660x_WRITE, DATA_2B},
- {"G3 Autoincrement Register", 0x18A, NI_660x_WRITE, DATA_2B},
- {"G23 Joint Reset Register", 0x190, NI_660x_WRITE, DATA_2B},
- {"G2 Interrupt Enable", 0x192, NI_660x_WRITE, DATA_2B},
- {"G3 Interrupt Enable", 0x196, NI_660x_WRITE, DATA_2B},
- {"G2 Counting Mode Register", 0x1B0, NI_660x_WRITE, DATA_2B},
- {"G3 Counting Mode Register", 0x1B2, NI_660x_WRITE, DATA_2B},
- {"G3 Second Gate Register", 0x1B6, NI_660x_WRITE, DATA_2B},
- {"G2 Second Gate Register", 0x1B4, NI_660x_WRITE, DATA_2B},
- {"G2 DMA Config Register", 0x1B8, NI_660x_WRITE, DATA_2B},
- {"G2 DMA Status Register", 0x1B8, NI_660x_READ, DATA_2B},
- {"G3 DMA Config Register", 0x1BA, NI_660x_WRITE, DATA_2B},
- {"G3 DMA Status Register", 0x1BA, NI_660x_READ, DATA_2B},
- {"32 bit Digital Input", 0x414, NI_660x_READ, DATA_4B},
- {"32 bit Digital Output", 0x510, NI_660x_WRITE, DATA_4B},
- {"Clock Config Register", 0x73C, NI_660x_WRITE, DATA_4B},
- {"Global Interrupt Status Register", 0x754, NI_660x_READ, DATA_4B},
- {"DMA Configuration Register", 0x76C, NI_660x_WRITE, DATA_4B},
- {"Global Interrupt Config Register", 0x770, NI_660x_WRITE, DATA_4B},
- {"IO Config Register 0-1", 0x77C, NI_660x_READ_WRITE, DATA_2B},
- {"IO Config Register 2-3", 0x77E, NI_660x_READ_WRITE, DATA_2B},
- {"IO Config Register 4-5", 0x780, NI_660x_READ_WRITE, DATA_2B},
- {"IO Config Register 6-7", 0x782, NI_660x_READ_WRITE, DATA_2B},
- {"IO Config Register 8-9", 0x784, NI_660x_READ_WRITE, DATA_2B},
- {"IO Config Register 10-11", 0x786, NI_660x_READ_WRITE, DATA_2B},
- {"IO Config Register 12-13", 0x788, NI_660x_READ_WRITE, DATA_2B},
- {"IO Config Register 14-15", 0x78A, NI_660x_READ_WRITE, DATA_2B},
- {"IO Config Register 16-17", 0x78C, NI_660x_READ_WRITE, DATA_2B},
- {"IO Config Register 18-19", 0x78E, NI_660x_READ_WRITE, DATA_2B},
- {"IO Config Register 20-21", 0x790, NI_660x_READ_WRITE, DATA_2B},
- {"IO Config Register 22-23", 0x792, NI_660x_READ_WRITE, DATA_2B},
- {"IO Config Register 24-25", 0x794, NI_660x_READ_WRITE, DATA_2B},
- {"IO Config Register 26-27", 0x796, NI_660x_READ_WRITE, DATA_2B},
- {"IO Config Register 28-29", 0x798, NI_660x_READ_WRITE, DATA_2B},
- {"IO Config Register 30-31", 0x79A, NI_660x_READ_WRITE, DATA_2B},
- {"IO Config Register 32-33", 0x79C, NI_660x_READ_WRITE, DATA_2B},
- {"IO Config Register 34-35", 0x79E, NI_660x_READ_WRITE, DATA_2B},
- {"IO Config Register 36-37", 0x7A0, NI_660x_READ_WRITE, DATA_2B},
- {"IO Config Register 38-39", 0x7A2, NI_660x_READ_WRITE, DATA_2B}
+static const struct ni_660x_register_data ni_660x_reg_data[NI660X_NUM_REGS] = {
+ [NITIO_G0_INT_ACK] = { 0x004, 2 }, /* write */
+ [NITIO_G0_STATUS] = { 0x004, 2 }, /* read */
+ [NITIO_G1_INT_ACK] = { 0x006, 2 }, /* write */
+ [NITIO_G1_STATUS] = { 0x006, 2 }, /* read */
+ [NITIO_G01_STATUS] = { 0x008, 2 }, /* read */
+ [NITIO_G0_CMD] = { 0x00c, 2 }, /* write */
+ [NI660X_STC_DIO_PARALLEL_INPUT] = { 0x00e, 2 }, /* read */
+ [NITIO_G1_CMD] = { 0x00e, 2 }, /* write */
+ [NITIO_G0_HW_SAVE] = { 0x010, 4 }, /* read */
+ [NITIO_G1_HW_SAVE] = { 0x014, 4 }, /* read */
+ [NI660X_STC_DIO_OUTPUT] = { 0x014, 2 }, /* write */
+ [NI660X_STC_DIO_CONTROL] = { 0x016, 2 }, /* write */
+ [NITIO_G0_SW_SAVE] = { 0x018, 4 }, /* read */
+ [NITIO_G1_SW_SAVE] = { 0x01c, 4 }, /* read */
+ [NITIO_G0_MODE] = { 0x034, 2 }, /* write */
+ [NITIO_G01_STATUS1] = { 0x036, 2 }, /* read */
+ [NITIO_G1_MODE] = { 0x036, 2 }, /* write */
+ [NI660X_STC_DIO_SERIAL_INPUT] = { 0x038, 2 }, /* read */
+ [NITIO_G0_LOADA] = { 0x038, 4 }, /* write */
+ [NITIO_G01_STATUS2] = { 0x03a, 2 }, /* read */
+ [NITIO_G0_LOADB] = { 0x03c, 4 }, /* write */
+ [NITIO_G1_LOADA] = { 0x040, 4 }, /* write */
+ [NITIO_G1_LOADB] = { 0x044, 4 }, /* write */
+ [NITIO_G0_INPUT_SEL] = { 0x048, 2 }, /* write */
+ [NITIO_G1_INPUT_SEL] = { 0x04a, 2 }, /* write */
+ [NITIO_G0_AUTO_INC] = { 0x088, 2 }, /* write */
+ [NITIO_G1_AUTO_INC] = { 0x08a, 2 }, /* write */
+ [NITIO_G01_RESET] = { 0x090, 2 }, /* write */
+ [NITIO_G0_INT_ENA] = { 0x092, 2 }, /* write */
+ [NITIO_G1_INT_ENA] = { 0x096, 2 }, /* write */
+ [NITIO_G0_CNT_MODE] = { 0x0b0, 2 }, /* write */
+ [NITIO_G1_CNT_MODE] = { 0x0b2, 2 }, /* write */
+ [NITIO_G0_GATE2] = { 0x0b4, 2 }, /* write */
+ [NITIO_G1_GATE2] = { 0x0b6, 2 }, /* write */
+ [NITIO_G0_DMA_CFG] = { 0x0b8, 2 }, /* write */
+ [NITIO_G0_DMA_STATUS] = { 0x0b8, 2 }, /* read */
+ [NITIO_G1_DMA_CFG] = { 0x0ba, 2 }, /* write */
+ [NITIO_G1_DMA_STATUS] = { 0x0ba, 2 }, /* read */
+ [NITIO_G2_INT_ACK] = { 0x104, 2 }, /* write */
+ [NITIO_G2_STATUS] = { 0x104, 2 }, /* read */
+ [NITIO_G3_INT_ACK] = { 0x106, 2 }, /* write */
+ [NITIO_G3_STATUS] = { 0x106, 2 }, /* read */
+ [NITIO_G23_STATUS] = { 0x108, 2 }, /* read */
+ [NITIO_G2_CMD] = { 0x10c, 2 }, /* write */
+ [NITIO_G3_CMD] = { 0x10e, 2 }, /* write */
+ [NITIO_G2_HW_SAVE] = { 0x110, 4 }, /* read */
+ [NITIO_G3_HW_SAVE] = { 0x114, 4 }, /* read */
+ [NITIO_G2_SW_SAVE] = { 0x118, 4 }, /* read */
+ [NITIO_G3_SW_SAVE] = { 0x11c, 4 }, /* read */
+ [NITIO_G2_MODE] = { 0x134, 2 }, /* write */
+ [NITIO_G23_STATUS1] = { 0x136, 2 }, /* read */
+ [NITIO_G3_MODE] = { 0x136, 2 }, /* write */
+ [NITIO_G2_LOADA] = { 0x138, 4 }, /* write */
+ [NITIO_G23_STATUS2] = { 0x13a, 2 }, /* read */
+ [NITIO_G2_LOADB] = { 0x13c, 4 }, /* write */
+ [NITIO_G3_LOADA] = { 0x140, 4 }, /* write */
+ [NITIO_G3_LOADB] = { 0x144, 4 }, /* write */
+ [NITIO_G2_INPUT_SEL] = { 0x148, 2 }, /* write */
+ [NITIO_G3_INPUT_SEL] = { 0x14a, 2 }, /* write */
+ [NITIO_G2_AUTO_INC] = { 0x188, 2 }, /* write */
+ [NITIO_G3_AUTO_INC] = { 0x18a, 2 }, /* write */
+ [NITIO_G23_RESET] = { 0x190, 2 }, /* write */
+ [NITIO_G2_INT_ENA] = { 0x192, 2 }, /* write */
+ [NITIO_G3_INT_ENA] = { 0x196, 2 }, /* write */
+ [NITIO_G2_CNT_MODE] = { 0x1b0, 2 }, /* write */
+ [NITIO_G3_CNT_MODE] = { 0x1b2, 2 }, /* write */
+ [NITIO_G2_GATE2] = { 0x1b4, 2 }, /* write */
+ [NITIO_G3_GATE2] = { 0x1b6, 2 }, /* write */
+ [NITIO_G2_DMA_CFG] = { 0x1b8, 2 }, /* write */
+ [NITIO_G2_DMA_STATUS] = { 0x1b8, 2 }, /* read */
+ [NITIO_G3_DMA_CFG] = { 0x1ba, 2 }, /* write */
+ [NITIO_G3_DMA_STATUS] = { 0x1ba, 2 }, /* read */
+ [NI660X_DIO32_INPUT] = { 0x414, 4 }, /* read */
+ [NI660X_DIO32_OUTPUT] = { 0x510, 4 }, /* write */
+ [NI660X_CLK_CFG] = { 0x73c, 4 }, /* write */
+ [NI660X_GLOBAL_INT_STATUS] = { 0x754, 4 }, /* read */
+ [NI660X_DMA_CFG] = { 0x76c, 4 }, /* write */
+ [NI660X_GLOBAL_INT_CFG] = { 0x770, 4 }, /* write */
+ [NI660X_IO_CFG_0_1] = { 0x77c, 2 }, /* read/write */
+ [NI660X_IO_CFG_2_3] = { 0x77e, 2 }, /* read/write */
+ [NI660X_IO_CFG_4_5] = { 0x780, 2 }, /* read/write */
+ [NI660X_IO_CFG_6_7] = { 0x782, 2 }, /* read/write */
+ [NI660X_IO_CFG_8_9] = { 0x784, 2 }, /* read/write */
+ [NI660X_IO_CFG_10_11] = { 0x786, 2 }, /* read/write */
+ [NI660X_IO_CFG_12_13] = { 0x788, 2 }, /* read/write */
+ [NI660X_IO_CFG_14_15] = { 0x78a, 2 }, /* read/write */
+ [NI660X_IO_CFG_16_17] = { 0x78c, 2 }, /* read/write */
+ [NI660X_IO_CFG_18_19] = { 0x78e, 2 }, /* read/write */
+ [NI660X_IO_CFG_20_21] = { 0x790, 2 }, /* read/write */
+ [NI660X_IO_CFG_22_23] = { 0x792, 2 }, /* read/write */
+ [NI660X_IO_CFG_24_25] = { 0x794, 2 }, /* read/write */
+ [NI660X_IO_CFG_26_27] = { 0x796, 2 }, /* read/write */
+ [NI660X_IO_CFG_28_29] = { 0x798, 2 }, /* read/write */
+ [NI660X_IO_CFG_30_31] = { 0x79a, 2 }, /* read/write */
+ [NI660X_IO_CFG_32_33] = { 0x79c, 2 }, /* read/write */
+ [NI660X_IO_CFG_34_35] = { 0x79e, 2 }, /* read/write */
+ [NI660X_IO_CFG_36_37] = { 0x7a0, 2 }, /* read/write */
+ [NI660X_IO_CFG_38_39] = { 0x7a2, 2 } /* read/write */
};
-/* kind of ENABLE for the second counter */
-enum clock_config_register_bits {
- CounterSwap = 0x1 << 21
-};
-
-/* ioconfigreg */
-static inline unsigned ioconfig_bitshift(unsigned pfi_channel)
-{
- return (pfi_channel % 2) ? 0 : 8;
-}
-
-static inline unsigned pfi_output_select_mask(unsigned pfi_channel)
-{
- return 0x3 << ioconfig_bitshift(pfi_channel);
-}
-
-static inline unsigned pfi_output_select_bits(unsigned pfi_channel,
- unsigned output_select)
-{
- return (output_select & 0x3) << ioconfig_bitshift(pfi_channel);
-}
-
-static inline unsigned pfi_input_select_mask(unsigned pfi_channel)
-{
- return 0x7 << (4 + ioconfig_bitshift(pfi_channel));
-}
-
-static inline unsigned pfi_input_select_bits(unsigned pfi_channel,
- unsigned input_select)
-{
- return (input_select & 0x7) << (4 + ioconfig_bitshift(pfi_channel));
-}
-
-/* dma configuration register bits */
-static inline unsigned dma_select_mask(unsigned dma_channel)
-{
- BUG_ON(dma_channel >= MAX_DMA_CHANNEL);
- return 0x1f << (8 * dma_channel);
-}
-
-enum dma_selection {
- dma_selection_none = 0x1f,
-};
-
-static inline unsigned dma_select_bits(unsigned dma_channel, unsigned selection)
-{
- BUG_ON(dma_channel >= MAX_DMA_CHANNEL);
- return (selection << (8 * dma_channel)) & dma_select_mask(dma_channel);
-}
-
-static inline unsigned dma_reset_bit(unsigned dma_channel)
-{
- BUG_ON(dma_channel >= MAX_DMA_CHANNEL);
- return 0x80 << (8 * dma_channel);
-}
-
-enum global_interrupt_status_register_bits {
- Counter_0_Int_Bit = 0x100,
- Counter_1_Int_Bit = 0x200,
- Counter_2_Int_Bit = 0x400,
- Counter_3_Int_Bit = 0x800,
- Cascade_Int_Bit = 0x20000000,
- Global_Int_Bit = 0x80000000
-};
-
-enum global_interrupt_config_register_bits {
- Cascade_Int_Enable_Bit = 0x20000000,
- Global_Int_Polarity_Bit = 0x40000000,
- Global_Int_Enable_Bit = 0x80000000
-};
-
-/* Offset of the GPCT chips from the base-address of the card */
-/* First chip is at base-address + 0x00, etc. */
-static const unsigned GPCT_OFFSET[2] = { 0x0, 0x800 };
+#define NI660X_CHIP_OFFSET 0x800
enum ni_660x_boardid {
BOARD_PCI6601,
@@ -385,7 +216,7 @@
struct ni_660x_board {
const char *name;
- unsigned n_chips; /* total number of TIO chips */
+ unsigned int n_chips; /* total number of TIO chips */
};
static const struct ni_660x_board ni_660x_boards[] = {
@@ -411,280 +242,95 @@
},
};
-#define NI_660X_MAX_NUM_CHIPS 2
-#define NI_660X_MAX_NUM_COUNTERS (NI_660X_MAX_NUM_CHIPS * counters_per_chip)
+#define NI660X_NUM_PFI_CHANNELS 40
+
+/* there are only up to 3 dma channels, but the register layout allows for 4 */
+#define NI660X_MAX_DMA_CHANNEL 4
+
+#define NI660X_COUNTERS_PER_CHIP 4
+#define NI660X_MAX_CHIPS 2
+#define NI660X_MAX_COUNTERS (NI660X_MAX_CHIPS * \
+ NI660X_COUNTERS_PER_CHIP)
struct ni_660x_private {
- struct mite_struct *mite;
+ struct mite *mite;
struct ni_gpct_device *counter_dev;
- uint64_t pfi_direction_bits;
- struct mite_dma_descriptor_ring
- *mite_rings[NI_660X_MAX_NUM_CHIPS][counters_per_chip];
+ struct mite_ring *ring[NI660X_MAX_CHIPS][NI660X_COUNTERS_PER_CHIP];
+ /* protects mite channel request/release */
spinlock_t mite_channel_lock;
- /* interrupt_lock prevents races between interrupt and comedi_poll */
+ /* prevents races between interrupt and comedi_poll */
spinlock_t interrupt_lock;
- unsigned dma_configuration_soft_copies[NI_660X_MAX_NUM_CHIPS];
- spinlock_t soft_reg_copy_lock;
- unsigned short pfi_output_selects[NUM_PFI_CHANNELS];
+ unsigned int dma_cfg[NI660X_MAX_CHIPS];
+ unsigned int io_cfg[NI660X_NUM_PFI_CHANNELS];
+ u64 io_dir;
};
-static inline unsigned ni_660x_num_counters(struct comedi_device *dev)
+static void ni_660x_write(struct comedi_device *dev, unsigned int chip,
+ unsigned int bits, unsigned int reg)
{
- const struct ni_660x_board *board = dev->board_ptr;
+ unsigned int addr = (chip * NI660X_CHIP_OFFSET) +
+ ni_660x_reg_data[reg].offset;
- return board->n_chips * counters_per_chip;
-}
-
-static enum ni_660x_register ni_gpct_to_660x_register(enum ni_gpct_register reg)
-{
- switch (reg) {
- case NITIO_G0_AUTO_INC:
- return NI660X_G0_AUTO_INC;
- case NITIO_G1_AUTO_INC:
- return NI660X_G1_AUTO_INC;
- case NITIO_G2_AUTO_INC:
- return NI660X_G2_AUTO_INC;
- case NITIO_G3_AUTO_INC:
- return NI660X_G3_AUTO_INC;
- case NITIO_G0_CMD:
- return NI660X_G0_CMD;
- case NITIO_G1_CMD:
- return NI660X_G1_CMD;
- case NITIO_G2_CMD:
- return NI660X_G2_CMD;
- case NITIO_G3_CMD:
- return NI660X_G3_CMD;
- case NITIO_G0_HW_SAVE:
- return NI660X_G0_HW_SAVE;
- case NITIO_G1_HW_SAVE:
- return NI660X_G1_HW_SAVE;
- case NITIO_G2_HW_SAVE:
- return NI660X_G2_HW_SAVE;
- case NITIO_G3_HW_SAVE:
- return NI660X_G3_HW_SAVE;
- case NITIO_G0_SW_SAVE:
- return NI660X_G0_SW_SAVE;
- case NITIO_G1_SW_SAVE:
- return NI660X_G1_SW_SAVE;
- case NITIO_G2_SW_SAVE:
- return NI660X_G2_SW_SAVE;
- case NITIO_G3_SW_SAVE:
- return NI660X_G3_SW_SAVE;
- case NITIO_G0_MODE:
- return NI660X_G0_MODE;
- case NITIO_G1_MODE:
- return NI660X_G1_MODE;
- case NITIO_G2_MODE:
- return NI660X_G2_MODE;
- case NITIO_G3_MODE:
- return NI660X_G3_MODE;
- case NITIO_G0_LOADA:
- return NI660X_G0_LOADA;
- case NITIO_G1_LOADA:
- return NI660X_G1_LOADA;
- case NITIO_G2_LOADA:
- return NI660X_G2_LOADA;
- case NITIO_G3_LOADA:
- return NI660X_G3_LOADA;
- case NITIO_G0_LOADB:
- return NI660X_G0_LOADB;
- case NITIO_G1_LOADB:
- return NI660X_G1_LOADB;
- case NITIO_G2_LOADB:
- return NI660X_G2_LOADB;
- case NITIO_G3_LOADB:
- return NI660X_G3_LOADB;
- case NITIO_G0_INPUT_SEL:
- return NI660X_G0_INPUT_SEL;
- case NITIO_G1_INPUT_SEL:
- return NI660X_G1_INPUT_SEL;
- case NITIO_G2_INPUT_SEL:
- return NI660X_G2_INPUT_SEL;
- case NITIO_G3_INPUT_SEL:
- return NI660X_G3_INPUT_SEL;
- case NITIO_G01_STATUS:
- return NI660X_G01_STATUS;
- case NITIO_G23_STATUS:
- return NI660X_G23_STATUS;
- case NITIO_G01_RESET:
- return NI660X_G01_RESET;
- case NITIO_G23_RESET:
- return NI660X_G23_RESET;
- case NITIO_G01_STATUS1:
- return NI660X_G01_STATUS1;
- case NITIO_G23_STATUS1:
- return NI660X_G23_STATUS1;
- case NITIO_G01_STATUS2:
- return NI660X_G01_STATUS2;
- case NITIO_G23_STATUS2:
- return NI660X_G23_STATUS2;
- case NITIO_G0_CNT_MODE:
- return NI660X_G0_CNT_MODE;
- case NITIO_G1_CNT_MODE:
- return NI660X_G1_CNT_MODE;
- case NITIO_G2_CNT_MODE:
- return NI660X_G2_CNT_MODE;
- case NITIO_G3_CNT_MODE:
- return NI660X_G3_CNT_MODE;
- case NITIO_G0_GATE2:
- return NI660X_G0_GATE2;
- case NITIO_G1_GATE2:
- return NI660X_G1_GATE2;
- case NITIO_G2_GATE2:
- return NI660X_G2_GATE2;
- case NITIO_G3_GATE2:
- return NI660X_G3_GATE2;
- case NITIO_G0_DMA_CFG:
- return NI660X_G0_DMA_CFG;
- case NITIO_G0_DMA_STATUS:
- return NI660X_G0_DMA_STATUS;
- case NITIO_G1_DMA_CFG:
- return NI660X_G1_DMA_CFG;
- case NITIO_G1_DMA_STATUS:
- return NI660X_G1_DMA_STATUS;
- case NITIO_G2_DMA_CFG:
- return NI660X_G2_DMA_CFG;
- case NITIO_G2_DMA_STATUS:
- return NI660X_G2_DMA_STATUS;
- case NITIO_G3_DMA_CFG:
- return NI660X_G3_DMA_CFG;
- case NITIO_G3_DMA_STATUS:
- return NI660X_G3_DMA_STATUS;
- case NITIO_G0_INT_ACK:
- return NI660X_G0_INT_ACK;
- case NITIO_G1_INT_ACK:
- return NI660X_G1_INT_ACK;
- case NITIO_G2_INT_ACK:
- return NI660X_G2_INT_ACK;
- case NITIO_G3_INT_ACK:
- return NI660X_G3_INT_ACK;
- case NITIO_G0_STATUS:
- return NI660X_G0_STATUS;
- case NITIO_G1_STATUS:
- return NI660X_G1_STATUS;
- case NITIO_G2_STATUS:
- return NI660X_G2_STATUS;
- case NITIO_G3_STATUS:
- return NI660X_G3_STATUS;
- case NITIO_G0_INT_ENA:
- return NI660X_G0_INT_ENA;
- case NITIO_G1_INT_ENA:
- return NI660X_G1_INT_ENA;
- case NITIO_G2_INT_ENA:
- return NI660X_G2_INT_ENA;
- case NITIO_G3_INT_ENA:
- return NI660X_G3_INT_ENA;
- default:
- BUG();
- return 0;
- }
-}
-
-static inline void ni_660x_write_register(struct comedi_device *dev,
- unsigned chip, unsigned bits,
- enum ni_660x_register reg)
-{
- unsigned int addr = GPCT_OFFSET[chip] + registerData[reg].offset;
-
- switch (registerData[reg].size) {
- case DATA_2B:
+ if (ni_660x_reg_data[reg].size == 2)
writew(bits, dev->mmio + addr);
- break;
- case DATA_4B:
+ else
writel(bits, dev->mmio + addr);
- break;
- default:
- BUG();
- break;
- }
}
-static inline unsigned ni_660x_read_register(struct comedi_device *dev,
- unsigned chip,
- enum ni_660x_register reg)
+static unsigned int ni_660x_read(struct comedi_device *dev,
+ unsigned int chip, unsigned int reg)
{
- unsigned int addr = GPCT_OFFSET[chip] + registerData[reg].offset;
+ unsigned int addr = (chip * NI660X_CHIP_OFFSET) +
+ ni_660x_reg_data[reg].offset;
- switch (registerData[reg].size) {
- case DATA_2B:
+ if (ni_660x_reg_data[reg].size == 2)
return readw(dev->mmio + addr);
- case DATA_4B:
- return readl(dev->mmio + addr);
- default:
- BUG();
- break;
- }
- return 0;
+ return readl(dev->mmio + addr);
}
-static void ni_gpct_write_register(struct ni_gpct *counter, unsigned bits,
- enum ni_gpct_register reg)
+static void ni_660x_gpct_write(struct ni_gpct *counter, unsigned int bits,
+ enum ni_gpct_register reg)
{
struct comedi_device *dev = counter->counter_dev->dev;
- enum ni_660x_register ni_660x_register = ni_gpct_to_660x_register(reg);
- unsigned chip = counter->chip_index;
- ni_660x_write_register(dev, chip, bits, ni_660x_register);
+ ni_660x_write(dev, counter->chip_index, bits, reg);
}
-static unsigned ni_gpct_read_register(struct ni_gpct *counter,
+static unsigned int ni_660x_gpct_read(struct ni_gpct *counter,
enum ni_gpct_register reg)
{
struct comedi_device *dev = counter->counter_dev->dev;
- enum ni_660x_register ni_660x_register = ni_gpct_to_660x_register(reg);
- unsigned chip = counter->chip_index;
- return ni_660x_read_register(dev, chip, ni_660x_register);
-}
-
-static inline struct mite_dma_descriptor_ring *mite_ring(struct ni_660x_private
- *priv,
- struct ni_gpct
- *counter)
-{
- unsigned chip = counter->chip_index;
-
- return priv->mite_rings[chip][counter->counter_index];
+ return ni_660x_read(dev, counter->chip_index, reg);
}
static inline void ni_660x_set_dma_channel(struct comedi_device *dev,
- unsigned mite_channel,
+ unsigned int mite_channel,
struct ni_gpct *counter)
{
struct ni_660x_private *devpriv = dev->private;
- unsigned chip = counter->chip_index;
- unsigned long flags;
+ unsigned int chip = counter->chip_index;
- spin_lock_irqsave(&devpriv->soft_reg_copy_lock, flags);
- devpriv->dma_configuration_soft_copies[chip] &=
- ~dma_select_mask(mite_channel);
- devpriv->dma_configuration_soft_copies[chip] |=
- dma_select_bits(mite_channel, counter->counter_index);
- ni_660x_write_register(dev, chip,
- devpriv->dma_configuration_soft_copies[chip] |
- dma_reset_bit(mite_channel), NI660X_DMA_CFG);
+ devpriv->dma_cfg[chip] &= ~NI660X_DMA_CFG_SEL_MASK(mite_channel);
+ devpriv->dma_cfg[chip] |= NI660X_DMA_CFG_SEL(mite_channel,
+ counter->counter_index);
+ ni_660x_write(dev, chip, devpriv->dma_cfg[chip] |
+ NI660X_DMA_CFG_RESET(mite_channel),
+ NI660X_DMA_CFG);
mmiowb();
- spin_unlock_irqrestore(&devpriv->soft_reg_copy_lock, flags);
}
static inline void ni_660x_unset_dma_channel(struct comedi_device *dev,
- unsigned mite_channel,
+ unsigned int mite_channel,
struct ni_gpct *counter)
{
struct ni_660x_private *devpriv = dev->private;
- unsigned chip = counter->chip_index;
- unsigned long flags;
+ unsigned int chip = counter->chip_index;
- spin_lock_irqsave(&devpriv->soft_reg_copy_lock, flags);
- devpriv->dma_configuration_soft_copies[chip] &=
- ~dma_select_mask(mite_channel);
- devpriv->dma_configuration_soft_copies[chip] |=
- dma_select_bits(mite_channel, dma_selection_none);
- ni_660x_write_register(dev, chip,
- devpriv->dma_configuration_soft_copies[chip],
- NI660X_DMA_CFG);
+ devpriv->dma_cfg[chip] &= ~NI660X_DMA_CFG_SEL_MASK(mite_channel);
+ devpriv->dma_cfg[chip] |= NI660X_DMA_CFG_SEL_NONE(mite_channel);
+ ni_660x_write(dev, chip, devpriv->dma_cfg[chip], NI660X_DMA_CFG);
mmiowb();
- spin_unlock_irqrestore(&devpriv->soft_reg_copy_lock, flags);
}
static int ni_660x_request_mite_channel(struct comedi_device *dev,
@@ -692,13 +338,13 @@
enum comedi_io_direction direction)
{
struct ni_660x_private *devpriv = dev->private;
- unsigned long flags;
+ struct mite_ring *ring;
struct mite_channel *mite_chan;
+ unsigned long flags;
spin_lock_irqsave(&devpriv->mite_channel_lock, flags);
- BUG_ON(counter->mite_chan);
- mite_chan = mite_request_channel(devpriv->mite,
- mite_ring(devpriv, counter));
+ ring = devpriv->ring[counter->chip_index][counter->counter_index];
+ mite_chan = mite_request_channel(devpriv->mite, ring);
if (!mite_chan) {
spin_unlock_irqrestore(&devpriv->mite_channel_lock, flags);
dev_err(dev->class_dev,
@@ -757,7 +403,7 @@
static void set_tio_counterswap(struct comedi_device *dev, int chip)
{
- unsigned bits = 0;
+ unsigned int bits = 0;
/*
* See P. 3.5 of the Register-Level Programming manual.
@@ -766,9 +412,9 @@
* first chip.
*/
if (chip)
- bits = CounterSwap;
+ bits = NI660X_CLK_CFG_COUNTER_SWAP;
- ni_660x_write_register(dev, chip, bits, NI660X_CLK_CFG);
+ ni_660x_write(dev, chip, bits, NI660X_CLK_CFG);
}
static void ni_660x_handle_gpct_interrupt(struct comedi_device *dev,
@@ -785,17 +431,20 @@
struct comedi_device *dev = d;
struct ni_660x_private *devpriv = dev->private;
struct comedi_subdevice *s;
- unsigned i;
+ unsigned int i;
unsigned long flags;
if (!dev->attached)
return IRQ_NONE;
+ /* make sure dev->attached is checked before doing anything else */
+ smp_mb();
+
/* lock to avoid race with comedi_poll */
spin_lock_irqsave(&devpriv->interrupt_lock, flags);
- smp_mb();
- for (i = 0; i < ni_660x_num_counters(dev); ++i) {
- s = &dev->subdevices[NI_660X_GPCT_SUBDEV(i)];
- ni_660x_handle_gpct_interrupt(dev, s);
+ for (i = 0; i < dev->n_subdevices; ++i) {
+ s = &dev->subdevices[i];
+ if (s->type == COMEDI_SUBD_COUNTER)
+ ni_660x_handle_gpct_interrupt(dev, s);
}
spin_unlock_irqrestore(&devpriv->interrupt_lock, flags);
return IRQ_HANDLED;
@@ -810,7 +459,7 @@
/* lock to avoid race with comedi_poll */
spin_lock_irqsave(&devpriv->interrupt_lock, flags);
- mite_sync_input_dma(counter->mite_chan, s);
+ mite_sync_dma(counter->mite_chan, s);
spin_unlock_irqrestore(&devpriv->interrupt_lock, flags);
return comedi_buf_read_n_available(s);
}
@@ -820,9 +469,11 @@
{
struct ni_660x_private *devpriv = dev->private;
struct ni_gpct *counter = s->private;
+ struct mite_ring *ring;
int ret;
- ret = mite_buf_change(mite_ring(devpriv, counter), s);
+ ring = devpriv->ring[counter->chip_index][counter->counter_index];
+ ret = mite_buf_change(ring, s);
if (ret < 0)
return ret;
@@ -832,7 +483,7 @@
static int ni_660x_allocate_private(struct comedi_device *dev)
{
struct ni_660x_private *devpriv;
- unsigned i;
+ unsigned int i;
devpriv = comedi_alloc_devpriv(dev, sizeof(*devpriv));
if (!devpriv)
@@ -840,9 +491,8 @@
spin_lock_init(&devpriv->mite_channel_lock);
spin_lock_init(&devpriv->interrupt_lock);
- spin_lock_init(&devpriv->soft_reg_copy_lock);
- for (i = 0; i < NUM_PFI_CHANNELS; ++i)
- devpriv->pfi_output_selects[i] = pfi_output_select_counter;
+ for (i = 0; i < NI660X_NUM_PFI_CHANNELS; ++i)
+ devpriv->io_cfg[i] = NI_660X_PFI_OUTPUT_COUNTER;
return 0;
}
@@ -851,14 +501,13 @@
{
const struct ni_660x_board *board = dev->board_ptr;
struct ni_660x_private *devpriv = dev->private;
- unsigned i;
- unsigned j;
+ unsigned int i;
+ unsigned int j;
for (i = 0; i < board->n_chips; ++i) {
- for (j = 0; j < counters_per_chip; ++j) {
- devpriv->mite_rings[i][j] =
- mite_alloc_ring(devpriv->mite);
- if (!devpriv->mite_rings[i][j])
+ for (j = 0; j < NI660X_COUNTERS_PER_CHIP; ++j) {
+ devpriv->ring[i][j] = mite_alloc_ring(devpriv->mite);
+ if (!devpriv->ring[i][j])
return -ENOMEM;
}
}
@@ -869,120 +518,101 @@
{
const struct ni_660x_board *board = dev->board_ptr;
struct ni_660x_private *devpriv = dev->private;
- unsigned i;
- unsigned j;
+ unsigned int i;
+ unsigned int j;
for (i = 0; i < board->n_chips; ++i) {
- for (j = 0; j < counters_per_chip; ++j)
- mite_free_ring(devpriv->mite_rings[i][j]);
+ for (j = 0; j < NI660X_COUNTERS_PER_CHIP; ++j)
+ mite_free_ring(devpriv->ring[i][j]);
}
}
-static void init_tio_chip(struct comedi_device *dev, int chipset)
-{
- struct ni_660x_private *devpriv = dev->private;
- unsigned i;
-
- /* init dma configuration register */
- devpriv->dma_configuration_soft_copies[chipset] = 0;
- for (i = 0; i < MAX_DMA_CHANNEL; ++i) {
- devpriv->dma_configuration_soft_copies[chipset] |=
- dma_select_bits(i, dma_selection_none) & dma_select_mask(i);
- }
- ni_660x_write_register(dev, chipset,
- devpriv->dma_configuration_soft_copies[chipset],
- NI660X_DMA_CFG);
- for (i = 0; i < NUM_PFI_CHANNELS; ++i)
- ni_660x_write_register(dev, chipset, 0, IOConfigReg(i));
-}
-
static int ni_660x_dio_insn_bits(struct comedi_device *dev,
struct comedi_subdevice *s,
- struct comedi_insn *insn, unsigned int *data)
+ struct comedi_insn *insn,
+ unsigned int *data)
{
- unsigned base_bitfield_channel = CR_CHAN(insn->chanspec);
+ unsigned int shift = CR_CHAN(insn->chanspec);
+ unsigned int mask = data[0] << shift;
+ unsigned int bits = data[1] << shift;
- /* Check if we have to write some bits */
- if (data[0]) {
- s->state &= ~(data[0] << base_bitfield_channel);
- s->state |= (data[0] & data[1]) << base_bitfield_channel;
- /* Write out the new digital output lines */
- ni_660x_write_register(dev, 0, s->state, NI660X_DIO32_OUTPUT);
+ /*
+ * There are 40 channels in this subdevice but only 32 are usable
+ * as DIO. The shift adjusts the mask/bits to account for the base
+ * channel in insn->chanspec. The state update can then be handled
+ * normally for the 32 usable channels.
+ */
+ if (mask) {
+ s->state &= ~mask;
+ s->state |= (bits & mask);
+ ni_660x_write(dev, 0, s->state, NI660X_DIO32_OUTPUT);
}
- /* on return, data[1] contains the value of the digital
- * input and output lines. */
- data[1] = (ni_660x_read_register(dev, 0, NI660X_DIO32_INPUT) >>
- base_bitfield_channel);
+
+ /*
+ * Return the input channels, shifted back to account for the base
+ * channel.
+ */
+ data[1] = ni_660x_read(dev, 0, NI660X_DIO32_INPUT) >> shift;
return insn->n;
}
static void ni_660x_select_pfi_output(struct comedi_device *dev,
- unsigned pfi_channel,
- unsigned output_select)
+ unsigned int chan, unsigned int out_sel)
{
const struct ni_660x_board *board = dev->board_ptr;
- static const unsigned counter_4_7_first_pfi = 8;
- static const unsigned counter_4_7_last_pfi = 23;
- unsigned active_chipset = 0;
- unsigned idle_chipset = 0;
- unsigned active_bits;
- unsigned idle_bits;
+ unsigned int active_chip = 0;
+ unsigned int idle_chip = 0;
+ unsigned int bits;
if (board->n_chips > 1) {
- if (output_select == pfi_output_select_counter &&
- pfi_channel >= counter_4_7_first_pfi &&
- pfi_channel <= counter_4_7_last_pfi) {
- active_chipset = 1;
- idle_chipset = 0;
+ if (out_sel == NI_660X_PFI_OUTPUT_COUNTER &&
+ chan >= 8 && chan <= 23) {
+ /* counters 4-7 pfi channels */
+ active_chip = 1;
+ idle_chip = 0;
} else {
- active_chipset = 0;
- idle_chipset = 1;
+ /* counters 0-3 pfi channels */
+ active_chip = 0;
+ idle_chip = 1;
}
}
- if (idle_chipset != active_chipset) {
- idle_bits =
- ni_660x_read_register(dev, idle_chipset,
- IOConfigReg(pfi_channel));
- idle_bits &= ~pfi_output_select_mask(pfi_channel);
- idle_bits |=
- pfi_output_select_bits(pfi_channel,
- pfi_output_select_high_Z);
- ni_660x_write_register(dev, idle_chipset, idle_bits,
- IOConfigReg(pfi_channel));
+ if (idle_chip != active_chip) {
+ /* set the pfi channel to high-z on the inactive chip */
+ bits = ni_660x_read(dev, idle_chip, NI660X_IO_CFG(chan));
+ bits &= ~NI660X_IO_CFG_OUT_SEL_MASK(chan);
+ bits |= NI660X_IO_CFG_OUT_SEL(chan, 0); /* high-z */
+ ni_660x_write(dev, idle_chip, bits, NI660X_IO_CFG(chan));
}
- active_bits =
- ni_660x_read_register(dev, active_chipset,
- IOConfigReg(pfi_channel));
- active_bits &= ~pfi_output_select_mask(pfi_channel);
- active_bits |= pfi_output_select_bits(pfi_channel, output_select);
- ni_660x_write_register(dev, active_chipset, active_bits,
- IOConfigReg(pfi_channel));
+ /* set the pfi channel output on the active chip */
+ bits = ni_660x_read(dev, active_chip, NI660X_IO_CFG(chan));
+ bits &= ~NI660X_IO_CFG_OUT_SEL_MASK(chan);
+ bits |= NI660X_IO_CFG_OUT_SEL(chan, out_sel);
+ ni_660x_write(dev, active_chip, bits, NI660X_IO_CFG(chan));
}
-static int ni_660x_set_pfi_routing(struct comedi_device *dev, unsigned chan,
- unsigned source)
+static int ni_660x_set_pfi_routing(struct comedi_device *dev,
+ unsigned int chan, unsigned int source)
{
struct ni_660x_private *devpriv = dev->private;
- if (source > num_pfi_output_selects)
- return -EINVAL;
- if (source == pfi_output_select_high_Z)
- return -EINVAL;
- if (chan < min_counter_pfi_chan) {
- if (source == pfi_output_select_counter)
+ switch (source) {
+ case NI_660X_PFI_OUTPUT_COUNTER:
+ if (chan < 8)
return -EINVAL;
- } else if (chan > max_dio_pfi_chan) {
- if (source == pfi_output_select_do)
+ break;
+ case NI_660X_PFI_OUTPUT_DIO:
+ if (chan > 31)
return -EINVAL;
+ default:
+ return -EINVAL;
}
- devpriv->pfi_output_selects[chan] = source;
- if (devpriv->pfi_direction_bits & (((uint64_t) 1) << chan))
- ni_660x_select_pfi_output(dev, chan,
- devpriv->pfi_output_selects[chan]);
+ devpriv->io_cfg[chan] = source;
+ if (devpriv->io_dir & (1ULL << chan))
+ ni_660x_select_pfi_output(dev, chan, devpriv->io_cfg[chan]);
return 0;
}
@@ -993,25 +623,24 @@
{
struct ni_660x_private *devpriv = dev->private;
unsigned int chan = CR_CHAN(insn->chanspec);
- uint64_t bit = 1ULL << chan;
+ u64 bit = 1ULL << chan;
unsigned int val;
int ret;
switch (data[0]) {
case INSN_CONFIG_DIO_OUTPUT:
- devpriv->pfi_direction_bits |= bit;
- ni_660x_select_pfi_output(dev, chan,
- devpriv->pfi_output_selects[chan]);
+ devpriv->io_dir |= bit;
+ ni_660x_select_pfi_output(dev, chan, devpriv->io_cfg[chan]);
break;
case INSN_CONFIG_DIO_INPUT:
- devpriv->pfi_direction_bits &= ~bit;
- ni_660x_select_pfi_output(dev, chan, pfi_output_select_high_Z);
+ devpriv->io_dir &= ~bit;
+ ni_660x_select_pfi_output(dev, chan, 0); /* high-z */
break;
case INSN_CONFIG_DIO_QUERY:
- data[1] = (devpriv->pfi_direction_bits & bit) ? COMEDI_OUTPUT
- : COMEDI_INPUT;
+ data[1] = (devpriv->io_dir & bit) ? COMEDI_OUTPUT
+ : COMEDI_INPUT;
break;
case INSN_CONFIG_SET_ROUTING:
@@ -1021,14 +650,14 @@
break;
case INSN_CONFIG_GET_ROUTING:
- data[1] = devpriv->pfi_output_selects[chan];
+ data[1] = devpriv->io_cfg[chan];
break;
case INSN_CONFIG_FILTER:
- val = ni_660x_read_register(dev, 0, IOConfigReg(chan));
- val &= ~pfi_input_select_mask(chan);
- val |= pfi_input_select_bits(chan, data[1]);
- ni_660x_write_register(dev, 0, val, IOConfigReg(chan));
+ val = ni_660x_read(dev, 0, NI660X_IO_CFG(chan));
+ val &= ~NI660X_IO_CFG_IN_SEL_MASK(chan);
+ val |= NI660X_IO_CFG_IN_SEL(chan, data[1]);
+ ni_660x_write(dev, 0, val, NI660X_IO_CFG(chan));
break;
default:
@@ -1038,6 +667,33 @@
return insn->n;
}
+static void ni_660x_init_tio_chips(struct comedi_device *dev,
+ unsigned int n_chips)
+{
+ struct ni_660x_private *devpriv = dev->private;
+ unsigned int chip;
+ unsigned int chan;
+
+ /*
+ * We use the ioconfig registers to control dio direction, so zero
+ * output enables in stc dio control reg.
+ */
+ ni_660x_write(dev, 0, 0, NI660X_STC_DIO_CONTROL);
+
+ for (chip = 0; chip < n_chips; ++chip) {
+ /* init dma configuration register */
+ devpriv->dma_cfg[chip] = 0;
+ for (chan = 0; chan < NI660X_MAX_DMA_CHANNEL; ++chan)
+ devpriv->dma_cfg[chip] |= NI660X_DMA_CFG_SEL_NONE(chan);
+ ni_660x_write(dev, chip, devpriv->dma_cfg[chip],
+ NI660X_DMA_CFG);
+
+ /* init ioconfig registers */
+ for (chan = 0; chan < NI660X_NUM_PFI_CHANNELS; ++chan)
+ ni_660x_write(dev, chip, 0, NI660X_IO_CFG(chan));
+ }
+}
+
static int ni_660x_auto_attach(struct comedi_device *dev,
unsigned long context)
{
@@ -1045,9 +701,12 @@
const struct ni_660x_board *board = NULL;
struct ni_660x_private *devpriv;
struct comedi_subdevice *s;
+ struct ni_gpct_device *gpct_dev;
+ unsigned int n_counters;
+ int subdev;
int ret;
- unsigned i;
- unsigned global_interrupt_config_bits;
+ unsigned int i;
+ unsigned int global_interrupt_config_bits;
if (context < ARRAY_SIZE(ni_660x_boards))
board = &ni_660x_boards[context];
@@ -1065,91 +724,147 @@
return ret;
devpriv = dev->private;
- devpriv->mite = mite_alloc(pcidev);
+ devpriv->mite = mite_attach(dev, true); /* use win1 */
if (!devpriv->mite)
return -ENOMEM;
- ret = mite_setup2(dev, devpriv->mite, true);
- if (ret < 0)
- return ret;
-
ret = ni_660x_alloc_mite_rings(dev);
if (ret < 0)
return ret;
- ret = comedi_alloc_subdevices(dev, 2 + NI_660X_MAX_NUM_COUNTERS);
+ ni_660x_init_tio_chips(dev, board->n_chips);
+
+ n_counters = board->n_chips * NI660X_COUNTERS_PER_CHIP;
+ gpct_dev = ni_gpct_device_construct(dev,
+ ni_660x_gpct_write,
+ ni_660x_gpct_read,
+ ni_gpct_variant_660x,
+ n_counters);
+ if (!gpct_dev)
+ return -ENOMEM;
+ devpriv->counter_dev = gpct_dev;
+
+ ret = comedi_alloc_subdevices(dev, 2 + NI660X_MAX_COUNTERS);
if (ret)
return ret;
- s = &dev->subdevices[0];
+ subdev = 0;
+
+ s = &dev->subdevices[subdev++];
/* Old GENERAL-PURPOSE COUNTER/TIME (GPCT) subdevice, no longer used */
s->type = COMEDI_SUBD_UNUSED;
- s = &dev->subdevices[NI_660X_DIO_SUBDEV];
- /* DIGITAL I/O SUBDEVICE */
- s->type = COMEDI_SUBD_DIO;
- s->subdev_flags = SDF_READABLE | SDF_WRITABLE;
- s->n_chan = NUM_PFI_CHANNELS;
- s->maxdata = 1;
- s->range_table = &range_digital;
- s->insn_bits = ni_660x_dio_insn_bits;
- s->insn_config = ni_660x_dio_insn_config;
- /* we use the ioconfig registers to control dio direction, so zero
- output enables in stc dio control reg */
- ni_660x_write_register(dev, 0, 0, NI660X_STC_DIO_CONTROL);
+ /*
+ * Digital I/O subdevice
+ *
+ * There are 40 channels but only the first 32 can be digital I/Os.
+ * The last 8 are dedicated to counters 0 and 1.
+ *
+ * Counter 0-3 signals are from the first TIO chip.
+ * Counter 4-7 signals are from the second TIO chip.
+ *
+ * Comedi External
+ * PFI Chan DIO Chan Counter Signal
+ * ------- -------- --------------
+ * 0 0
+ * 1 1
+ * 2 2
+ * 3 3
+ * 4 4
+ * 5 5
+ * 6 6
+ * 7 7
+ * 8 8 CTR 7 OUT
+ * 9 9 CTR 7 AUX
+ * 10 10 CTR 7 GATE
+ * 11 11 CTR 7 SOURCE
+ * 12 12 CTR 6 OUT
+ * 13 13 CTR 6 AUX
+ * 14 14 CTR 6 GATE
+ * 15 15 CTR 6 SOURCE
+ * 16 16 CTR 5 OUT
+ * 17 17 CTR 5 AUX
+ * 18 18 CTR 5 GATE
+ * 19 19 CTR 5 SOURCE
+ * 20 20 CTR 4 OUT
+ * 21 21 CTR 4 AUX
+ * 22 22 CTR 4 GATE
+ * 23 23 CTR 4 SOURCE
+ * 24 24 CTR 3 OUT
+ * 25 25 CTR 3 AUX
+ * 26 26 CTR 3 GATE
+ * 27 27 CTR 3 SOURCE
+ * 28 28 CTR 2 OUT
+ * 29 29 CTR 2 AUX
+ * 30 30 CTR 2 GATE
+ * 31 31 CTR 2 SOURCE
+ * 32 CTR 1 OUT
+ * 33 CTR 1 AUX
+ * 34 CTR 1 GATE
+ * 35 CTR 1 SOURCE
+ * 36 CTR 0 OUT
+ * 37 CTR 0 AUX
+ * 38 CTR 0 GATE
+ * 39 CTR 0 SOURCE
+ */
+ s = &dev->subdevices[subdev++];
+ s->type = COMEDI_SUBD_DIO;
+ s->subdev_flags = SDF_READABLE | SDF_WRITABLE;
+ s->n_chan = NI660X_NUM_PFI_CHANNELS;
+ s->maxdata = 1;
+ s->range_table = &range_digital;
+ s->insn_bits = ni_660x_dio_insn_bits;
+ s->insn_config = ni_660x_dio_insn_config;
- devpriv->counter_dev = ni_gpct_device_construct(dev,
- &ni_gpct_write_register,
- &ni_gpct_read_register,
- ni_gpct_variant_660x,
- ni_660x_num_counters
- (dev));
- if (!devpriv->counter_dev)
- return -ENOMEM;
- for (i = 0; i < NI_660X_MAX_NUM_COUNTERS; ++i) {
- s = &dev->subdevices[NI_660X_GPCT_SUBDEV(i)];
- if (i < ni_660x_num_counters(dev)) {
- s->type = COMEDI_SUBD_COUNTER;
- s->subdev_flags = SDF_READABLE | SDF_WRITABLE |
+ /*
+ * Default the DIO channels as:
+ * chan 0-7: DIO inputs
+ * chan 8-39: counter signal inputs
+ */
+ for (i = 0; i < s->n_chan; ++i) {
+ unsigned int source = (i < 8) ? NI_660X_PFI_OUTPUT_DIO
+ : NI_660X_PFI_OUTPUT_COUNTER;
+
+ ni_660x_set_pfi_routing(dev, i, source);
+ ni_660x_select_pfi_output(dev, i, 0); /* high-z */
+ }
+
+ /* Counter subdevices (4 NI TIO General Purpose Counters per chip) */
+ for (i = 0; i < NI660X_MAX_COUNTERS; ++i) {
+ s = &dev->subdevices[subdev++];
+ if (i < n_counters) {
+ struct ni_gpct *counter = &gpct_dev->counters[i];
+
+ counter->chip_index = i / NI660X_COUNTERS_PER_CHIP;
+ counter->counter_index = i % NI660X_COUNTERS_PER_CHIP;
+
+ s->type = COMEDI_SUBD_COUNTER;
+ s->subdev_flags = SDF_READABLE | SDF_WRITABLE |
SDF_LSAMPL | SDF_CMD_READ;
- s->n_chan = 3;
- s->maxdata = 0xffffffff;
- s->insn_read = ni_tio_insn_read;
- s->insn_write = ni_tio_insn_write;
- s->insn_config = ni_tio_insn_config;
- s->do_cmd = &ni_660x_cmd;
- s->len_chanlist = 1;
- s->do_cmdtest = ni_tio_cmdtest;
- s->cancel = &ni_660x_cancel;
- s->poll = &ni_660x_input_poll;
+ s->n_chan = 3;
+ s->maxdata = 0xffffffff;
+ s->insn_read = ni_tio_insn_read;
+ s->insn_write = ni_tio_insn_write;
+ s->insn_config = ni_tio_insn_config;
+ s->len_chanlist = 1;
+ s->do_cmd = ni_660x_cmd;
+ s->do_cmdtest = ni_tio_cmdtest;
+ s->cancel = ni_660x_cancel;
+ s->poll = ni_660x_input_poll;
+ s->buf_change = ni_660x_buf_change;
s->async_dma_dir = DMA_BIDIRECTIONAL;
- s->buf_change = &ni_660x_buf_change;
- s->private = &devpriv->counter_dev->counters[i];
+ s->private = counter;
- devpriv->counter_dev->counters[i].chip_index =
- i / counters_per_chip;
- devpriv->counter_dev->counters[i].counter_index =
- i % counters_per_chip;
+ ni_tio_init_counter(counter);
} else {
- s->type = COMEDI_SUBD_UNUSED;
+ s->type = COMEDI_SUBD_UNUSED;
}
}
- for (i = 0; i < board->n_chips; ++i)
- init_tio_chip(dev, i);
- for (i = 0; i < ni_660x_num_counters(dev); ++i)
- ni_tio_init_counter(&devpriv->counter_dev->counters[i]);
-
- for (i = 0; i < NUM_PFI_CHANNELS; ++i) {
- if (i < min_counter_pfi_chan)
- ni_660x_set_pfi_routing(dev, i, pfi_output_select_do);
- else
- ni_660x_set_pfi_routing(dev, i,
- pfi_output_select_counter);
- ni_660x_select_pfi_output(dev, i, pfi_output_select_high_Z);
- }
- /* to be safe, set counterswap bits on tio chips after all the counter
- outputs have been set to high impedance mode */
+ /*
+ * To be safe, set counterswap bits on tio chips after all the counter
+ * outputs have been set to high impedance mode.
+ */
for (i = 0; i < board->n_chips; ++i)
set_tio_counterswap(dev, i);
@@ -1160,11 +875,11 @@
return ret;
}
dev->irq = pcidev->irq;
- global_interrupt_config_bits = Global_Int_Enable_Bit;
+ global_interrupt_config_bits = NI660X_GLOBAL_INT_GLOBAL;
if (board->n_chips > 1)
- global_interrupt_config_bits |= Cascade_Int_Enable_Bit;
- ni_660x_write_register(dev, 0, global_interrupt_config_bits,
- NI660X_GLOBAL_INT_CFG);
+ global_interrupt_config_bits |= NI660X_GLOBAL_INT_CASCADE;
+ ni_660x_write(dev, 0, global_interrupt_config_bits,
+ NI660X_GLOBAL_INT_CFG);
return 0;
}
@@ -1173,11 +888,12 @@
{
struct ni_660x_private *devpriv = dev->private;
- if (dev->irq)
+ if (dev->irq) {
+ ni_660x_write(dev, 0, 0, NI660X_GLOBAL_INT_CFG);
free_irq(dev->irq, dev);
+ }
if (devpriv) {
- if (devpriv->counter_dev)
- ni_gpct_device_destroy(devpriv->counter_dev);
+ ni_gpct_device_destroy(devpriv->counter_dev);
ni_660x_free_mite_rings(dev);
mite_detach(devpriv->mite);
}
@@ -1218,5 +934,5 @@
module_comedi_pci_driver(ni_660x_driver, ni_660x_pci_driver);
MODULE_AUTHOR("Comedi http://www.comedi.org");
-MODULE_DESCRIPTION("Comedi low-level driver");
+MODULE_DESCRIPTION("Comedi driver for NI 660x counter/timer boards");
MODULE_LICENSE("GPL");
diff --git a/drivers/staging/comedi/drivers/ni_labpc.h b/drivers/staging/comedi/drivers/ni_labpc.h
index 83f878a..be8d5cd 100644
--- a/drivers/staging/comedi/drivers/ni_labpc.h
+++ b/drivers/staging/comedi/drivers/ni_labpc.h
@@ -1,27 +1,22 @@
/*
- ni_labpc.h
-
- Header for ni_labpc.c and ni_labpc_cs.c
-
- Copyright (C) 2003 Frank Mori Hess <fmhess@users.sourceforge.net>
-
- This program is free software; you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation; either version 2 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-*/
+ * Header for ni_labpc ISA/PCMCIA/PCI drivers
+ *
+ * Copyright (C) 2003 Frank Mori Hess <fmhess@users.sourceforge.net>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
#ifndef _NI_LABPC_H
#define _NI_LABPC_H
-#define EEPROM_SIZE 256 /* 256 byte eeprom */
-#define NUM_AO_CHAN 2 /* boards have two analog output channels */
-
enum transfer_type { fifo_not_empty_transfer, fifo_half_full_transfer,
isa_dma_transfer
};
diff --git a/drivers/staging/comedi/drivers/ni_labpc_common.c b/drivers/staging/comedi/drivers/ni_labpc_common.c
index 863afb2..b0dfb8e 100644
--- a/drivers/staging/comedi/drivers/ni_labpc_common.c
+++ b/drivers/staging/comedi/drivers/ni_labpc_common.c
@@ -84,8 +84,10 @@
}
};
-/* functions that do inb/outb and readb/writeb so we can use
- * function pointers to decide which to use */
+/*
+ * functions that do inb/outb and readb/writeb so we can use
+ * function pointers to decide which to use
+ */
static unsigned int labpc_inb(struct comedi_device *dev, unsigned long reg)
{
return inb(dev->iobase + reg);
@@ -656,19 +658,24 @@
/* figure out what method we will use to transfer data */
if (devpriv->dma &&
- /* dma unsafe at RT priority,
- * and too much setup time for CMDF_WAKE_EOS */
- (cmd->flags & (CMDF_WAKE_EOS | CMDF_PRIORITY)) == 0)
+ (cmd->flags & (CMDF_WAKE_EOS | CMDF_PRIORITY)) == 0) {
+ /*
+ * dma unsafe at RT priority,
+ * and too much setup time for CMDF_WAKE_EOS
+ */
xfer = isa_dma_transfer;
- else if (/* pc-plus has no fifo-half full interrupt */
- board->is_labpc1200 &&
- /* wake-end-of-scan should interrupt on fifo not empty */
- (cmd->flags & CMDF_WAKE_EOS) == 0 &&
- /* make sure we are taking more than just a few points */
- (cmd->stop_src != TRIG_COUNT || devpriv->count > 256))
+ } else if (board->is_labpc1200 &&
+ (cmd->flags & CMDF_WAKE_EOS) == 0 &&
+ (cmd->stop_src != TRIG_COUNT || devpriv->count > 256)) {
+ /*
+ * pc-plus has no fifo-half full interrupt
+ * wake-end-of-scan should interrupt on fifo not empty
+ * make sure we are taking more than just a few points
+ */
xfer = fifo_half_full_transfer;
- else
+ } else {
xfer = fifo_not_empty_transfer;
+ }
devpriv->current_transfer = xfer;
labpc_ai_set_chan_and_gain(dev, mode, chan, range, aref);
@@ -679,9 +686,11 @@
/* manual says to set scan enable bit on second pass */
if (mode == MODE_MULT_CHAN_UP || mode == MODE_MULT_CHAN_DOWN) {
devpriv->cmd1 |= CMD1_SCANEN;
- /* need a brief delay before enabling scan, or scan
- * list will get screwed when you switch
- * between scan up to scan down mode - dunno why */
+ /*
+ * Need a brief delay before enabling scan, or scan
+ * list will get screwed when you switch between
+ * scan up to scan down mode - dunno why.
+ */
udelay(1);
devpriv->write_byte(dev, devpriv->cmd1, CMD1_REG);
}
@@ -728,8 +737,10 @@
devpriv->cmd4 = 0;
if (cmd->convert_src != TRIG_EXT)
devpriv->cmd4 |= CMD4_ECLKRCV;
- /* XXX should discard first scan when using interval scanning
- * since manual says it is not synced with scan clock */
+ /*
+ * XXX should discard first scan when using interval scanning
+ * since manual says it is not synced with scan clock.
+ */
if (!labpc_use_continuous_mode(cmd, mode)) {
devpriv->cmd4 |= CMD4_INTSCAN;
if (cmd->scan_begin_src == TRIG_EXT)
@@ -795,8 +806,10 @@
return 0;
}
-/* makes sure all data acquired by board is transferred to comedi (used
- * when acquisition is terminated by stop_src == TRIG_EXT). */
+/*
+ * Makes sure all data acquired by board is transferred to comedi (used
+ * when acquisition is terminated by stop_src == TRIG_EXT).
+ */
static void labpc_drain_dregs(struct comedi_device *dev)
{
struct labpc_private *devpriv = dev->private;
@@ -907,9 +920,11 @@
channel = CR_CHAN(insn->chanspec);
- /* turn off pacing of analog output channel */
- /* note: hardware bug in daqcard-1200 means pacing cannot
- * be independently enabled/disabled for its the two channels */
+ /*
+ * Turn off pacing of analog output channel.
+ * NOTE: hardware bug in daqcard-1200 means pacing cannot
+ * be independently enabled/disabled for its the two channels.
+ */
spin_lock_irqsave(&dev->spinlock, flags);
devpriv->cmd2 &= ~CMD2_LDAC(channel);
devpriv->write_byte(dev, devpriv->cmd2, CMD2_REG);
@@ -1261,7 +1276,7 @@
if (board->has_ao) {
s->type = COMEDI_SUBD_AO;
s->subdev_flags = SDF_READABLE | SDF_WRITABLE | SDF_GROUND;
- s->n_chan = NUM_AO_CHAN;
+ s->n_chan = 2;
s->maxdata = 0x0fff;
s->range_table = &range_labpc_ao;
s->insn_write = labpc_ao_insn_write;
@@ -1307,12 +1322,12 @@
s->type = COMEDI_SUBD_UNUSED;
}
- /* EEPROM */
+ /* EEPROM (256 bytes) */
s = &dev->subdevices[4];
if (board->is_labpc1200) {
s->type = COMEDI_SUBD_MEMORY;
s->subdev_flags = SDF_READABLE | SDF_WRITABLE | SDF_INTERNAL;
- s->n_chan = EEPROM_SIZE;
+ s->n_chan = 256;
s->maxdata = 0xff;
s->insn_write = labpc_eeprom_insn_write;
diff --git a/drivers/staging/comedi/drivers/ni_labpc_cs.c b/drivers/staging/comedi/drivers/ni_labpc_cs.c
index a1c69ac..3d4d0b9 100644
--- a/drivers/staging/comedi/drivers/ni_labpc_cs.c
+++ b/drivers/staging/comedi/drivers/ni_labpc_cs.c
@@ -1,57 +1,50 @@
/*
- comedi/drivers/ni_labpc_cs.c
- Driver for National Instruments daqcard-1200 boards
- Copyright (C) 2001, 2002, 2003 Frank Mori Hess <fmhess@users.sourceforge.net>
-
- PCMCIA crap is adapted from dummy_cs.c 1.31 2001/08/24 12:13:13
- from the pcmcia package.
- The initial developer of the pcmcia dummy_cs.c code is David A. Hinds
- <dahinds@users.sourceforge.net>. Portions created by David A. Hinds
- are Copyright (C) 1999 David A. Hinds.
-
- This program is free software; you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation; either version 2 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-*/
-/*
-Driver: ni_labpc_cs
-Description: National Instruments Lab-PC (& compatibles)
-Author: Frank Mori Hess <fmhess@users.sourceforge.net>
-Devices: [National Instruments] DAQCard-1200 (daqcard-1200)
-Status: works
-
-Thanks go to Fredrik Lingvall for much testing and perseverance in
-helping to debug daqcard-1200 support.
-
-The 1200 series boards have onboard calibration dacs for correcting
-analog input/output offsets and gains. The proper settings for these
-caldacs are stored on the board's eeprom. To read the caldac values
-from the eeprom and store them into a file that can be then be used by
-comedilib, use the comedi_calibrate program.
-
-Configuration options:
- none
-
-The daqcard-1200 has quirky chanlist requirements
-when scanning multiple channels. Multiple channel scan
-sequence must start at highest channel, then decrement down to
-channel 0. Chanlists consisting of all one channel
-are also legal, and allow you to pace conversions in bursts.
-
-*/
+ * Driver for National Instruments daqcard-1200 boards
+ * Copyright (C) 2001, 2002, 2003 Frank Mori Hess <fmhess@users.sourceforge.net>
+ *
+ * PCMCIA crap is adapted from dummy_cs.c 1.31 2001/08/24 12:13:13
+ * from the pcmcia package.
+ * The initial developer of the pcmcia dummy_cs.c code is David A. Hinds
+ * <dahinds@users.sourceforge.net>. Portions created by David A. Hinds
+ * are Copyright (C) 1999 David A. Hinds.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * General Public License for more details.
+ */
/*
-
-NI manuals:
-340988a (daqcard-1200)
-
-*/
+ * Driver: ni_labpc_cs
+ * Description: National Instruments Lab-PC (& compatibles)
+ * Author: Frank Mori Hess <fmhess@users.sourceforge.net>
+ * Devices: [National Instruments] DAQCard-1200 (daqcard-1200)
+ * Status: works
+ *
+ * Thanks go to Fredrik Lingvall for much testing and perseverance in
+ * helping to debug daqcard-1200 support.
+ *
+ * The 1200 series boards have onboard calibration dacs for correcting
+ * analog input/output offsets and gains. The proper settings for these
+ * caldacs are stored on the board's eeprom. To read the caldac values
+ * from the eeprom and store them into a file that can be then be used by
+ * comedilib, use the comedi_calibrate program.
+ *
+ * Configuration options: none
+ *
+ * The daqcard-1200 has quirky chanlist requirements when scanning multiple
+ * channels. Multiple channel scan sequence must start at highest channel,
+ * then decrement down to channel 0. Chanlists consisting of all one channel
+ * are also legal, and allow you to pace conversions in bursts.
+ *
+ * NI manuals:
+ * 340988a (daqcard-1200)
+ */
#include <linux/module.h>
diff --git a/drivers/staging/comedi/drivers/ni_labpc_pci.c b/drivers/staging/comedi/drivers/ni_labpc_pci.c
index 77d4038..cac0891 100644
--- a/drivers/staging/comedi/drivers/ni_labpc_pci.c
+++ b/drivers/staging/comedi/drivers/ni_labpc_pci.c
@@ -51,8 +51,8 @@
};
/* ripped from mite.h and mite_setup2() to avoid mite dependency */
-#define MITE_IODWBSR 0xc0 /* IO Device Window Base Size Register */
-#define WENAB (1 << 7) /* window enable */
+#define MITE_IODWBSR 0xc0 /* IO Device Window Base Size Register */
+#define WENAB BIT(7) /* window enable */
static int labpc_pci_mite_init(struct pci_dev *pcidev)
{
diff --git a/drivers/staging/comedi/drivers/ni_labpc_regs.h b/drivers/staging/comedi/drivers/ni_labpc_regs.h
index 2a274a3..8c52179 100644
--- a/drivers/staging/comedi/drivers/ni_labpc_regs.h
+++ b/drivers/staging/comedi/drivers/ni_labpc_regs.h
@@ -9,32 +9,32 @@
* Register map (all registers are 8-bit)
*/
#define STAT1_REG 0x00 /* R: Status 1 reg */
-#define STAT1_DAVAIL (1 << 0)
-#define STAT1_OVERRUN (1 << 1)
-#define STAT1_OVERFLOW (1 << 2)
-#define STAT1_CNTINT (1 << 3)
-#define STAT1_GATA0 (1 << 5)
-#define STAT1_EXTGATA0 (1 << 6)
+#define STAT1_DAVAIL BIT(0)
+#define STAT1_OVERRUN BIT(1)
+#define STAT1_OVERFLOW BIT(2)
+#define STAT1_CNTINT BIT(3)
+#define STAT1_GATA0 BIT(5)
+#define STAT1_EXTGATA0 BIT(6)
#define CMD1_REG 0x00 /* W: Command 1 reg */
#define CMD1_MA(x) (((x) & 0x7) << 0)
-#define CMD1_TWOSCMP (1 << 3)
+#define CMD1_TWOSCMP BIT(3)
#define CMD1_GAIN(x) (((x) & 0x7) << 4)
-#define CMD1_SCANEN (1 << 7)
+#define CMD1_SCANEN BIT(7)
#define CMD2_REG 0x01 /* W: Command 2 reg */
-#define CMD2_PRETRIG (1 << 0)
-#define CMD2_HWTRIG (1 << 1)
-#define CMD2_SWTRIG (1 << 2)
-#define CMD2_TBSEL (1 << 3)
-#define CMD2_2SDAC0 (1 << 4)
-#define CMD2_2SDAC1 (1 << 5)
-#define CMD2_LDAC(x) (1 << (6 + (x)))
+#define CMD2_PRETRIG BIT(0)
+#define CMD2_HWTRIG BIT(1)
+#define CMD2_SWTRIG BIT(2)
+#define CMD2_TBSEL BIT(3)
+#define CMD2_2SDAC0 BIT(4)
+#define CMD2_2SDAC1 BIT(5)
+#define CMD2_LDAC(x) BIT(6 + ((x) & 0x1))
#define CMD3_REG 0x02 /* W: Command 3 reg */
-#define CMD3_DMAEN (1 << 0)
-#define CMD3_DIOINTEN (1 << 1)
-#define CMD3_DMATCINTEN (1 << 2)
-#define CMD3_CNTINTEN (1 << 3)
-#define CMD3_ERRINTEN (1 << 4)
-#define CMD3_FIFOINTEN (1 << 5)
+#define CMD3_DMAEN BIT(0)
+#define CMD3_DIOINTEN BIT(1)
+#define CMD3_DMATCINTEN BIT(2)
+#define CMD3_CNTINTEN BIT(3)
+#define CMD3_ERRINTEN BIT(4)
+#define CMD3_FIFOINTEN BIT(5)
#define ADC_START_CONVERT_REG 0x03 /* W: Start Convert reg */
#define DAC_LSB_REG(x) (0x04 + 2 * (x)) /* W: DAC0/1 LSB reg */
#define DAC_MSB_REG(x) (0x05 + 2 * (x)) /* W: DAC0/1 MSB reg */
@@ -43,32 +43,32 @@
#define DMATC_CLEAR_REG 0x0a /* W: DMA Interrupt Clear reg */
#define TIMER_CLEAR_REG 0x0c /* W: Timer Interrupt Clear reg */
#define CMD6_REG 0x0e /* W: Command 6 reg */
-#define CMD6_NRSE (1 << 0)
-#define CMD6_ADCUNI (1 << 1)
-#define CMD6_DACUNI(x) (1 << (2 + (x)))
-#define CMD6_HFINTEN (1 << 5)
-#define CMD6_DQINTEN (1 << 6)
-#define CMD6_SCANUP (1 << 7)
+#define CMD6_NRSE BIT(0)
+#define CMD6_ADCUNI BIT(1)
+#define CMD6_DACUNI(x) BIT(2 + ((x) & 0x1))
+#define CMD6_HFINTEN BIT(5)
+#define CMD6_DQINTEN BIT(6)
+#define CMD6_SCANUP BIT(7)
#define CMD4_REG 0x0f /* W: Command 3 reg */
-#define CMD4_INTSCAN (1 << 0)
-#define CMD4_EOIRCV (1 << 1)
-#define CMD4_ECLKDRV (1 << 2)
-#define CMD4_SEDIFF (1 << 3)
-#define CMD4_ECLKRCV (1 << 4)
+#define CMD4_INTSCAN BIT(0)
+#define CMD4_EOIRCV BIT(1)
+#define CMD4_ECLKDRV BIT(2)
+#define CMD4_SEDIFF BIT(3)
+#define CMD4_ECLKRCV BIT(4)
#define DIO_BASE_REG 0x10 /* R/W: 8255 DIO base reg */
#define COUNTER_A_BASE_REG 0x14 /* R/W: 8253 Counter A base reg */
#define COUNTER_B_BASE_REG 0x18 /* R/W: 8253 Counter B base reg */
#define CMD5_REG 0x1c /* W: Command 5 reg */
-#define CMD5_WRTPRT (1 << 2)
-#define CMD5_DITHEREN (1 << 3)
-#define CMD5_CALDACLD (1 << 4)
-#define CMD5_SCLK (1 << 5)
-#define CMD5_SDATA (1 << 6)
-#define CMD5_EEPROMCS (1 << 7)
+#define CMD5_WRTPRT BIT(2)
+#define CMD5_DITHEREN BIT(3)
+#define CMD5_CALDACLD BIT(4)
+#define CMD5_SCLK BIT(5)
+#define CMD5_SDATA BIT(6)
+#define CMD5_EEPROMCS BIT(7)
#define STAT2_REG 0x1d /* R: Status 2 reg */
-#define STAT2_PROMOUT (1 << 0)
-#define STAT2_OUTA1 (1 << 1)
-#define STAT2_FIFONHF (1 << 2)
+#define STAT2_PROMOUT BIT(0)
+#define STAT2_OUTA1 BIT(1)
+#define STAT2_FIFONHF BIT(2)
#define INTERVAL_COUNT_REG 0x1e /* W: Interval Counter Data reg */
#define INTERVAL_STROBE_REG 0x1f /* W: Interval Counter Strobe reg */
diff --git a/drivers/staging/comedi/drivers/ni_mio_c_common.c b/drivers/staging/comedi/drivers/ni_mio_c_common.c
deleted file mode 100644
index e69de29..0000000
--- a/drivers/staging/comedi/drivers/ni_mio_c_common.c
+++ /dev/null
diff --git a/drivers/staging/comedi/drivers/ni_mio_common.c b/drivers/staging/comedi/drivers/ni_mio_common.c
index dcaf7e8..8dabb19 100644
--- a/drivers/staging/comedi/drivers/ni_mio_common.c
+++ b/drivers/staging/comedi/drivers/ni_mio_common.c
@@ -1,56 +1,53 @@
/*
- comedi/drivers/ni_mio_common.c
- Hardware driver for DAQ-STC based boards
-
- COMEDI - Linux Control and Measurement Device Interface
- Copyright (C) 1997-2001 David A. Schleef <ds@schleef.org>
- Copyright (C) 2002-2006 Frank Mori Hess <fmhess@users.sourceforge.net>
-
- This program is free software; you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation; either version 2 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-*/
+ * Hardware driver for DAQ-STC based boards
+ *
+ * COMEDI - Linux Control and Measurement Device Interface
+ * Copyright (C) 1997-2001 David A. Schleef <ds@schleef.org>
+ * Copyright (C) 2002-2006 Frank Mori Hess <fmhess@users.sourceforge.net>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
/*
- This file is meant to be included by another file, e.g.,
- ni_atmio.c or ni_pcimio.c.
-
- Interrupt support originally added by Truxton Fulton
- <trux@truxton.com>
-
- References (from ftp://ftp.natinst.com/support/manuals):
-
- 340747b.pdf AT-MIO E series Register Level Programmer Manual
- 341079b.pdf PCI E Series RLPM
- 340934b.pdf DAQ-STC reference manual
- 67xx and 611x registers (from ftp://ftp.ni.com/support/daq/mhddk/documentation/)
- release_ni611x.pdf
- release_ni67xx.pdf
- Other possibly relevant info:
-
- 320517c.pdf User manual (obsolete)
- 320517f.pdf User manual (new)
- 320889a.pdf delete
- 320906c.pdf maximum signal ratings
- 321066a.pdf about 16x
- 321791a.pdf discontinuation of at-mio-16e-10 rev. c
- 321808a.pdf about at-mio-16e-10 rev P
- 321837a.pdf discontinuation of at-mio-16de-10 rev d
- 321838a.pdf about at-mio-16de-10 rev N
-
- ISSUES:
-
- - the interrupt routine needs to be cleaned up
-
- 2006-02-07: S-Series PCI-6143: Support has been added but is not
- fully tested as yet. Terry Barnaby, BEAM Ltd.
-*/
+ * This file is meant to be included by another file, e.g.,
+ * ni_atmio.c or ni_pcimio.c.
+ *
+ * Interrupt support originally added by Truxton Fulton <trux@truxton.com>
+ *
+ * References (ftp://ftp.natinst.com/support/manuals):
+ * 340747b.pdf AT-MIO E series Register Level Programmer Manual
+ * 341079b.pdf PCI E Series RLPM
+ * 340934b.pdf DAQ-STC reference manual
+ *
+ * 67xx and 611x registers (ftp://ftp.ni.com/support/daq/mhddk/documentation/)
+ * release_ni611x.pdf
+ * release_ni67xx.pdf
+ *
+ * Other possibly relevant info:
+ * 320517c.pdf User manual (obsolete)
+ * 320517f.pdf User manual (new)
+ * 320889a.pdf delete
+ * 320906c.pdf maximum signal ratings
+ * 321066a.pdf about 16x
+ * 321791a.pdf discontinuation of at-mio-16e-10 rev. c
+ * 321808a.pdf about at-mio-16e-10 rev P
+ * 321837a.pdf discontinuation of at-mio-16de-10 rev d
+ * 321838a.pdf about at-mio-16de-10 rev N
+ *
+ * ISSUES:
+ * - the interrupt routine needs to be cleaned up
+ *
+ * 2006-02-07: S-Series PCI-6143: Support has been added but is not
+ * fully tested as yet. Terry Barnaby, BEAM Ltd.
+ */
#include <linux/interrupt.h>
#include <linux/sched.h>
@@ -216,19 +213,8 @@
NI_FREQ_OUT_SUBDEV,
NI_NUM_SUBDEVICES
};
-static inline unsigned NI_GPCT_SUBDEV(unsigned counter_index)
-{
- switch (counter_index) {
- case 0:
- return NI_GPCT0_SUBDEV;
- case 1:
- return NI_GPCT1_SUBDEV;
- default:
- break;
- }
- BUG();
- return NI_GPCT0_SUBDEV;
-}
+
+#define NI_GPCT_SUBDEV(x) (NI_GPCT0_SUBDEV + (x))
enum timebase_nanoseconds {
TIMEBASE_1_NS = 50,
@@ -242,7 +228,7 @@
static const int num_adc_stages_611x = 3;
-static void ni_writel(struct comedi_device *dev, uint32_t data, int reg)
+static void ni_writel(struct comedi_device *dev, unsigned int data, int reg)
{
if (dev->mmio)
writel(data, dev->mmio + reg);
@@ -250,7 +236,7 @@
outl(data, dev->iobase + reg);
}
-static void ni_writew(struct comedi_device *dev, uint16_t data, int reg)
+static void ni_writew(struct comedi_device *dev, unsigned int data, int reg)
{
if (dev->mmio)
writew(data, dev->mmio + reg);
@@ -258,7 +244,7 @@
outw(data, dev->iobase + reg);
}
-static void ni_writeb(struct comedi_device *dev, uint8_t data, int reg)
+static void ni_writeb(struct comedi_device *dev, unsigned int data, int reg)
{
if (dev->mmio)
writeb(data, dev->mmio + reg);
@@ -266,7 +252,7 @@
outb(data, dev->iobase + reg);
}
-static uint32_t ni_readl(struct comedi_device *dev, int reg)
+static unsigned int ni_readl(struct comedi_device *dev, int reg)
{
if (dev->mmio)
return readl(dev->mmio + reg);
@@ -274,7 +260,7 @@
return inl(dev->iobase + reg);
}
-static uint16_t ni_readw(struct comedi_device *dev, int reg)
+static unsigned int ni_readw(struct comedi_device *dev, int reg)
{
if (dev->mmio)
return readw(dev->mmio + reg);
@@ -282,7 +268,7 @@
return inw(dev->iobase + reg);
}
-static uint8_t ni_readb(struct comedi_device *dev, int reg)
+static unsigned int ni_readb(struct comedi_device *dev, int reg)
{
if (dev->mmio)
return readb(dev->mmio + reg);
@@ -457,7 +443,8 @@
}
}
-static void ni_stc_writew(struct comedi_device *dev, uint16_t data, int reg)
+static void ni_stc_writew(struct comedi_device *dev,
+ unsigned int data, int reg)
{
struct ni_private *devpriv = dev->private;
unsigned long flags;
@@ -476,7 +463,8 @@
}
}
-static void ni_stc_writel(struct comedi_device *dev, uint32_t data, int reg)
+static void ni_stc_writel(struct comedi_device *dev,
+ unsigned int data, int reg)
{
struct ni_private *devpriv = dev->private;
@@ -488,11 +476,11 @@
}
}
-static uint16_t ni_stc_readw(struct comedi_device *dev, int reg)
+static unsigned int ni_stc_readw(struct comedi_device *dev, int reg)
{
struct ni_private *devpriv = dev->private;
unsigned long flags;
- uint16_t val;
+ unsigned int val;
if (devpriv->is_m_series) {
val = m_series_stc_read(dev, reg);
@@ -509,10 +497,10 @@
return val;
}
-static uint32_t ni_stc_readl(struct comedi_device *dev, int reg)
+static unsigned int ni_stc_readl(struct comedi_device *dev, int reg)
{
struct ni_private *devpriv = dev->private;
- uint32_t val;
+ unsigned int val;
if (devpriv->is_m_series) {
val = m_series_stc_read(dev, reg);
@@ -524,7 +512,8 @@
}
static inline void ni_set_bitfield(struct comedi_device *dev, int reg,
- unsigned bit_mask, unsigned bit_values)
+ unsigned int bit_mask,
+ unsigned int bit_values)
{
struct ni_private *devpriv = dev->private;
unsigned long flags;
@@ -556,6 +545,11 @@
devpriv->g0_g1_select_reg |= bit_values & bit_mask;
ni_writeb(dev, devpriv->g0_g1_select_reg, reg);
break;
+ case NI_M_CDIO_DMA_SEL_REG:
+ devpriv->cdio_dma_select_reg &= ~bit_mask;
+ devpriv->cdio_dma_select_reg |= bit_values & bit_mask;
+ ni_writeb(dev, devpriv->cdio_dma_select_reg, reg);
+ break;
default:
dev_err(dev->class_dev, "called with invalid register %d\n",
reg);
@@ -566,116 +560,35 @@
}
#ifdef PCIDMA
+
+/* selects the MITE channel to use for DMA */
+#define NI_STC_DMA_CHAN_SEL(x) (((x) < 4) ? BIT(x) : \
+ ((x) == 4) ? 0x3 : \
+ ((x) == 5) ? 0x5 : 0x0)
+
/* DMA channel setup */
-static inline unsigned ni_stc_dma_channel_select_bitfield(unsigned channel)
-{
- if (channel < 4)
- return 1 << channel;
- if (channel == 4)
- return 0x3;
- if (channel == 5)
- return 0x5;
- BUG();
- return 0;
-}
-
-static inline void ni_set_ai_dma_channel(struct comedi_device *dev,
- unsigned channel)
-{
- unsigned bits = ni_stc_dma_channel_select_bitfield(channel);
-
- ni_set_bitfield(dev, NI_E_DMA_AI_AO_SEL_REG,
- NI_E_DMA_AI_SEL_MASK, NI_E_DMA_AI_SEL(bits));
-}
-
-static inline void ni_set_ai_dma_no_channel(struct comedi_device *dev)
-{
- ni_set_bitfield(dev, NI_E_DMA_AI_AO_SEL_REG, NI_E_DMA_AI_SEL_MASK, 0);
-}
-
-static inline void ni_set_ao_dma_channel(struct comedi_device *dev,
- unsigned channel)
-{
- unsigned bits = ni_stc_dma_channel_select_bitfield(channel);
-
- ni_set_bitfield(dev, NI_E_DMA_AI_AO_SEL_REG,
- NI_E_DMA_AO_SEL_MASK, NI_E_DMA_AO_SEL(bits));
-}
-
-static inline void ni_set_ao_dma_no_channel(struct comedi_device *dev)
-{
- ni_set_bitfield(dev, NI_E_DMA_AI_AO_SEL_REG, NI_E_DMA_AO_SEL_MASK, 0);
-}
-
-static inline void ni_set_gpct_dma_channel(struct comedi_device *dev,
- unsigned gpct_index,
- unsigned channel)
-{
- unsigned bits = ni_stc_dma_channel_select_bitfield(channel);
-
- ni_set_bitfield(dev, NI_E_DMA_G0_G1_SEL_REG,
- NI_E_DMA_G0_G1_SEL_MASK(gpct_index),
- NI_E_DMA_G0_G1_SEL(gpct_index, bits));
-}
-
-static inline void ni_set_gpct_dma_no_channel(struct comedi_device *dev,
- unsigned gpct_index)
-{
- ni_set_bitfield(dev, NI_E_DMA_G0_G1_SEL_REG,
- NI_E_DMA_G0_G1_SEL_MASK(gpct_index), 0);
-}
-
-static inline void ni_set_cdo_dma_channel(struct comedi_device *dev,
- unsigned mite_channel)
-{
- struct ni_private *devpriv = dev->private;
- unsigned long flags;
- unsigned bits;
-
- spin_lock_irqsave(&devpriv->soft_reg_copy_lock, flags);
- devpriv->cdio_dma_select_reg &= ~NI_M_CDIO_DMA_SEL_CDO_MASK;
- /*
- * XXX just guessing ni_stc_dma_channel_select_bitfield()
- * returns the right bits, under the assumption the cdio dma
- * selection works just like ai/ao/gpct.
- * Definitely works for dma channels 0 and 1.
- */
- bits = ni_stc_dma_channel_select_bitfield(mite_channel);
- devpriv->cdio_dma_select_reg |= NI_M_CDIO_DMA_SEL_CDO(bits);
- ni_writeb(dev, devpriv->cdio_dma_select_reg, NI_M_CDIO_DMA_SEL_REG);
- mmiowb();
- spin_unlock_irqrestore(&devpriv->soft_reg_copy_lock, flags);
-}
-
-static inline void ni_set_cdo_dma_no_channel(struct comedi_device *dev)
-{
- struct ni_private *devpriv = dev->private;
- unsigned long flags;
-
- spin_lock_irqsave(&devpriv->soft_reg_copy_lock, flags);
- devpriv->cdio_dma_select_reg &= ~NI_M_CDIO_DMA_SEL_CDO_MASK;
- ni_writeb(dev, devpriv->cdio_dma_select_reg, NI_M_CDIO_DMA_SEL_REG);
- mmiowb();
- spin_unlock_irqrestore(&devpriv->soft_reg_copy_lock, flags);
-}
-
static int ni_request_ai_mite_channel(struct comedi_device *dev)
{
struct ni_private *devpriv = dev->private;
+ struct mite_channel *mite_chan;
unsigned long flags;
+ unsigned int bits;
spin_lock_irqsave(&devpriv->mite_channel_lock, flags);
- BUG_ON(devpriv->ai_mite_chan);
- devpriv->ai_mite_chan =
- mite_request_channel(devpriv->mite, devpriv->ai_mite_ring);
- if (!devpriv->ai_mite_chan) {
+ mite_chan = mite_request_channel(devpriv->mite, devpriv->ai_mite_ring);
+ if (!mite_chan) {
spin_unlock_irqrestore(&devpriv->mite_channel_lock, flags);
dev_err(dev->class_dev,
"failed to reserve mite dma channel for analog input\n");
return -EBUSY;
}
- devpriv->ai_mite_chan->dir = COMEDI_INPUT;
- ni_set_ai_dma_channel(dev, devpriv->ai_mite_chan->channel);
+ mite_chan->dir = COMEDI_INPUT;
+ devpriv->ai_mite_chan = mite_chan;
+
+ bits = NI_STC_DMA_CHAN_SEL(mite_chan->channel);
+ ni_set_bitfield(dev, NI_E_DMA_AI_AO_SEL_REG,
+ NI_E_DMA_AI_SEL_MASK, NI_E_DMA_AI_SEL(bits));
+
spin_unlock_irqrestore(&devpriv->mite_channel_lock, flags);
return 0;
}
@@ -683,37 +596,42 @@
static int ni_request_ao_mite_channel(struct comedi_device *dev)
{
struct ni_private *devpriv = dev->private;
+ struct mite_channel *mite_chan;
unsigned long flags;
+ unsigned int bits;
spin_lock_irqsave(&devpriv->mite_channel_lock, flags);
- BUG_ON(devpriv->ao_mite_chan);
- devpriv->ao_mite_chan =
- mite_request_channel(devpriv->mite, devpriv->ao_mite_ring);
- if (!devpriv->ao_mite_chan) {
+ mite_chan = mite_request_channel(devpriv->mite, devpriv->ao_mite_ring);
+ if (!mite_chan) {
spin_unlock_irqrestore(&devpriv->mite_channel_lock, flags);
dev_err(dev->class_dev,
"failed to reserve mite dma channel for analog outut\n");
return -EBUSY;
}
- devpriv->ao_mite_chan->dir = COMEDI_OUTPUT;
- ni_set_ao_dma_channel(dev, devpriv->ao_mite_chan->channel);
+ mite_chan->dir = COMEDI_OUTPUT;
+ devpriv->ao_mite_chan = mite_chan;
+
+ bits = NI_STC_DMA_CHAN_SEL(mite_chan->channel);
+ ni_set_bitfield(dev, NI_E_DMA_AI_AO_SEL_REG,
+ NI_E_DMA_AO_SEL_MASK, NI_E_DMA_AO_SEL(bits));
+
spin_unlock_irqrestore(&devpriv->mite_channel_lock, flags);
return 0;
}
static int ni_request_gpct_mite_channel(struct comedi_device *dev,
- unsigned gpct_index,
+ unsigned int gpct_index,
enum comedi_io_direction direction)
{
struct ni_private *devpriv = dev->private;
- unsigned long flags;
+ struct ni_gpct *counter = &devpriv->counter_dev->counters[gpct_index];
struct mite_channel *mite_chan;
+ unsigned long flags;
+ unsigned int bits;
spin_lock_irqsave(&devpriv->mite_channel_lock, flags);
- BUG_ON(devpriv->counter_dev->counters[gpct_index].mite_chan);
- mite_chan =
- mite_request_channel(devpriv->mite,
- devpriv->gpct_mite_ring[gpct_index]);
+ mite_chan = mite_request_channel(devpriv->mite,
+ devpriv->gpct_mite_ring[gpct_index]);
if (!mite_chan) {
spin_unlock_irqrestore(&devpriv->mite_channel_lock, flags);
dev_err(dev->class_dev,
@@ -721,37 +639,50 @@
return -EBUSY;
}
mite_chan->dir = direction;
- ni_tio_set_mite_channel(&devpriv->counter_dev->counters[gpct_index],
- mite_chan);
- ni_set_gpct_dma_channel(dev, gpct_index, mite_chan->channel);
+ ni_tio_set_mite_channel(counter, mite_chan);
+
+ bits = NI_STC_DMA_CHAN_SEL(mite_chan->channel);
+ ni_set_bitfield(dev, NI_E_DMA_G0_G1_SEL_REG,
+ NI_E_DMA_G0_G1_SEL_MASK(gpct_index),
+ NI_E_DMA_G0_G1_SEL(gpct_index, bits));
+
spin_unlock_irqrestore(&devpriv->mite_channel_lock, flags);
return 0;
}
-#endif /* PCIDMA */
-
static int ni_request_cdo_mite_channel(struct comedi_device *dev)
{
-#ifdef PCIDMA
struct ni_private *devpriv = dev->private;
+ struct mite_channel *mite_chan;
unsigned long flags;
+ unsigned int bits;
spin_lock_irqsave(&devpriv->mite_channel_lock, flags);
- BUG_ON(devpriv->cdo_mite_chan);
- devpriv->cdo_mite_chan =
- mite_request_channel(devpriv->mite, devpriv->cdo_mite_ring);
- if (!devpriv->cdo_mite_chan) {
+ mite_chan = mite_request_channel(devpriv->mite, devpriv->cdo_mite_ring);
+ if (!mite_chan) {
spin_unlock_irqrestore(&devpriv->mite_channel_lock, flags);
dev_err(dev->class_dev,
"failed to reserve mite dma channel for correlated digital output\n");
return -EBUSY;
}
- devpriv->cdo_mite_chan->dir = COMEDI_OUTPUT;
- ni_set_cdo_dma_channel(dev, devpriv->cdo_mite_chan->channel);
+ mite_chan->dir = COMEDI_OUTPUT;
+ devpriv->cdo_mite_chan = mite_chan;
+
+ /*
+ * XXX just guessing NI_STC_DMA_CHAN_SEL()
+ * returns the right bits, under the assumption the cdio dma
+ * selection works just like ai/ao/gpct.
+ * Definitely works for dma channels 0 and 1.
+ */
+ bits = NI_STC_DMA_CHAN_SEL(mite_chan->channel);
+ ni_set_bitfield(dev, NI_M_CDIO_DMA_SEL_REG,
+ NI_M_CDIO_DMA_SEL_CDO_MASK,
+ NI_M_CDIO_DMA_SEL_CDO(bits));
+
spin_unlock_irqrestore(&devpriv->mite_channel_lock, flags);
-#endif /* PCIDMA */
return 0;
}
+#endif /* PCIDMA */
static void ni_release_ai_mite_channel(struct comedi_device *dev)
{
@@ -761,7 +692,8 @@
spin_lock_irqsave(&devpriv->mite_channel_lock, flags);
if (devpriv->ai_mite_chan) {
- ni_set_ai_dma_no_channel(dev);
+ ni_set_bitfield(dev, NI_E_DMA_AI_AO_SEL_REG,
+ NI_E_DMA_AI_SEL_MASK, 0);
mite_release_channel(devpriv->ai_mite_chan);
devpriv->ai_mite_chan = NULL;
}
@@ -777,7 +709,8 @@
spin_lock_irqsave(&devpriv->mite_channel_lock, flags);
if (devpriv->ao_mite_chan) {
- ni_set_ao_dma_no_channel(dev);
+ ni_set_bitfield(dev, NI_E_DMA_AI_AO_SEL_REG,
+ NI_E_DMA_AO_SEL_MASK, 0);
mite_release_channel(devpriv->ao_mite_chan);
devpriv->ao_mite_chan = NULL;
}
@@ -787,7 +720,7 @@
#ifdef PCIDMA
static void ni_release_gpct_mite_channel(struct comedi_device *dev,
- unsigned gpct_index)
+ unsigned int gpct_index)
{
struct ni_private *devpriv = dev->private;
unsigned long flags;
@@ -797,7 +730,8 @@
struct mite_channel *mite_chan =
devpriv->counter_dev->counters[gpct_index].mite_chan;
- ni_set_gpct_dma_no_channel(dev, gpct_index);
+ ni_set_bitfield(dev, NI_E_DMA_G0_G1_SEL_REG,
+ NI_E_DMA_G0_G1_SEL_MASK(gpct_index), 0);
ni_tio_set_mite_channel(&devpriv->
counter_dev->counters[gpct_index],
NULL);
@@ -805,30 +739,27 @@
}
spin_unlock_irqrestore(&devpriv->mite_channel_lock, flags);
}
-#endif /* PCIDMA */
static void ni_release_cdo_mite_channel(struct comedi_device *dev)
{
-#ifdef PCIDMA
struct ni_private *devpriv = dev->private;
unsigned long flags;
spin_lock_irqsave(&devpriv->mite_channel_lock, flags);
if (devpriv->cdo_mite_chan) {
- ni_set_cdo_dma_no_channel(dev);
+ ni_set_bitfield(dev, NI_M_CDIO_DMA_SEL_REG,
+ NI_M_CDIO_DMA_SEL_CDO_MASK, 0);
mite_release_channel(devpriv->cdo_mite_chan);
devpriv->cdo_mite_chan = NULL;
}
spin_unlock_irqrestore(&devpriv->mite_channel_lock, flags);
-#endif /* PCIDMA */
}
-#ifdef PCIDMA
static void ni_e_series_enable_second_irq(struct comedi_device *dev,
- unsigned gpct_index, short enable)
+ unsigned int gpct_index, short enable)
{
struct ni_private *devpriv = dev->private;
- uint16_t val = 0;
+ unsigned int val = 0;
int reg;
if (devpriv->is_m_series || gpct_index > 1)
@@ -875,8 +806,10 @@
ni_writeb(dev, 0, NI_M_STATIC_AI_CTRL_REG(0));
ni_writeb(dev, 1, NI_M_STATIC_AI_CTRL_REG(0));
#if 0
- /* the NI example code does 3 convert pulses for 625x boards,
- but that appears to be wrong in practice. */
+ /*
+ * The NI example code does 3 convert pulses for 625x
+ * boards, But that appears to be wrong in practice.
+ */
ni_stc_writew(dev, NISTC_AI_CMD1_CONVERT_PULSE,
NISTC_AI_CMD1_REG);
ni_stc_writew(dev, NISTC_AI_CMD1_CONVERT_PULSE,
@@ -888,8 +821,8 @@
}
}
-static inline void ni_ao_win_outw(struct comedi_device *dev, uint16_t data,
- int addr)
+static inline void ni_ao_win_outw(struct comedi_device *dev,
+ unsigned int data, int addr)
{
struct ni_private *devpriv = dev->private;
unsigned long flags;
@@ -900,8 +833,8 @@
spin_unlock_irqrestore(&devpriv->window_lock, flags);
}
-static inline void ni_ao_win_outl(struct comedi_device *dev, uint32_t data,
- int addr)
+static inline void ni_ao_win_outl(struct comedi_device *dev,
+ unsigned int data, int addr)
{
struct ni_private *devpriv = dev->private;
unsigned long flags;
@@ -925,20 +858,21 @@
return data;
}
-/* ni_set_bits( ) allows different parts of the ni_mio_common driver to
-* share registers (such as Interrupt_A_Register) without interfering with
-* each other.
-*
-* NOTE: the switch/case statements are optimized out for a constant argument
-* so this is actually quite fast--- If you must wrap another function around this
-* make it inline to avoid a large speed penalty.
-*
-* value should only be 1 or 0.
-*/
+/*
+ * ni_set_bits( ) allows different parts of the ni_mio_common driver to
+ * share registers (such as Interrupt_A_Register) without interfering with
+ * each other.
+ *
+ * NOTE: the switch/case statements are optimized out for a constant argument
+ * so this is actually quite fast--- If you must wrap another function around
+ * this make it inline to avoid a large speed penalty.
+ *
+ * value should only be 1 or 0.
+ */
static inline void ni_set_bits(struct comedi_device *dev, int reg,
- unsigned bits, unsigned value)
+ unsigned int bits, unsigned int value)
{
- unsigned bit_values;
+ unsigned int bit_values;
if (value)
bit_values = bits;
@@ -956,7 +890,7 @@
spin_lock_irqsave(&devpriv->mite_channel_lock, flags);
if (devpriv->ai_mite_chan)
- mite_sync_input_dma(devpriv->ai_mite_chan, s);
+ mite_sync_dma(devpriv->ai_mite_chan, s);
spin_unlock_irqrestore(&devpriv->mite_channel_lock, flags);
}
@@ -972,9 +906,8 @@
if (devpriv->ai_mite_chan) {
for (i = 0; i < timeout; i++) {
if ((ni_stc_readw(dev, NISTC_AI_STATUS1_REG) &
- NISTC_AI_STATUS1_FIFO_E)
- && mite_bytes_in_transit(devpriv->ai_mite_chan) ==
- 0)
+ NISTC_AI_STATUS1_FIFO_E) &&
+ mite_bytes_in_transit(devpriv->ai_mite_chan) == 0)
break;
udelay(5);
}
@@ -994,19 +927,6 @@
return retval;
}
-static void mite_handle_b_linkc(struct mite_struct *mite,
- struct comedi_device *dev)
-{
- struct ni_private *devpriv = dev->private;
- struct comedi_subdevice *s = dev->write_subdev;
- unsigned long flags;
-
- spin_lock_irqsave(&devpriv->mite_channel_lock, flags);
- if (devpriv->ao_mite_chan)
- mite_sync_output_dma(devpriv->ao_mite_chan, s);
- spin_unlock_irqrestore(&devpriv->mite_channel_lock, flags);
-}
-
static int ni_ao_wait_for_dma_load(struct comedi_device *dev)
{
static const int timeout = 10000;
@@ -1018,9 +938,11 @@
b_status = ni_stc_readw(dev, NISTC_AO_STATUS1_REG);
if (b_status & NISTC_AO_STATUS1_FIFO_HF)
break;
- /* if we poll too often, the pci bus activity seems
- to slow the dma transfer down */
- udelay(10);
+ /*
+ * If we poll too often, the pci bus activity seems
+ * to slow the dma transfer down.
+ */
+ usleep_range(10, 100);
}
if (i == timeout) {
dev_err(dev->class_dev, "timed out waiting for dma load\n");
@@ -1038,7 +960,7 @@
struct ni_private *devpriv = dev->private;
int i;
unsigned short d;
- u32 packed_data;
+ unsigned int packed_data;
for (i = 0; i < n; i++) {
comedi_buf_read_samples(s, &d, 1);
@@ -1128,7 +1050,7 @@
{
struct ni_private *devpriv = dev->private;
struct comedi_async *async = s->async;
- u32 dl;
+ unsigned int dl;
unsigned short data;
int i;
@@ -1148,7 +1070,10 @@
comedi_buf_write_samples(s, &data, 1);
}
} else if (devpriv->is_6143) {
- /* This just reads the FIFO assuming the data is present, no checks on the FIFO status are performed */
+ /*
+ * This just reads the FIFO assuming the data is present,
+ * no checks on the FIFO status are performed.
+ */
for (i = 0; i < n / 2; i++) {
dl = ni_readl(dev, NI6143_AI_FIFO_DATA_REG);
@@ -1192,16 +1117,13 @@
}
#endif
-/*
- Empties the AI fifo
-*/
+/* Empties the AI fifo */
static void ni_handle_fifo_dregs(struct comedi_device *dev)
{
struct ni_private *devpriv = dev->private;
struct comedi_subdevice *s = dev->read_subdev;
- u32 dl;
+ unsigned int dl;
unsigned short data;
- unsigned short fifo_empty;
int i;
if (devpriv->is_611x) {
@@ -1237,15 +1159,16 @@
}
} else {
- fifo_empty = ni_stc_readw(dev, NISTC_AI_STATUS1_REG) &
- NISTC_AI_STATUS1_FIFO_E;
- while (fifo_empty == 0) {
+ unsigned short fe; /* fifo empty */
+
+ fe = ni_stc_readw(dev, NISTC_AI_STATUS1_REG) &
+ NISTC_AI_STATUS1_FIFO_E;
+ while (fe == 0) {
for (i = 0;
i < ARRAY_SIZE(devpriv->ai_fifo_buffer); i++) {
- fifo_empty = ni_stc_readw(dev,
- NISTC_AI_STATUS1_REG) &
- NISTC_AI_STATUS1_FIFO_E;
- if (fifo_empty)
+ fe = ni_stc_readw(dev, NISTC_AI_STATUS1_REG) &
+ NISTC_AI_STATUS1_FIFO_E;
+ if (fe)
break;
devpriv->ai_fifo_buffer[i] =
ni_readw(dev, NI_E_AI_FIFO_DATA_REG);
@@ -1260,7 +1183,7 @@
struct ni_private *devpriv = dev->private;
struct comedi_subdevice *s = dev->read_subdev;
unsigned short data;
- u32 dl;
+ unsigned int dl;
if (!devpriv->is_611x)
return;
@@ -1278,7 +1201,7 @@
struct ni_private *devpriv = dev->private;
struct comedi_subdevice *s = dev->read_subdev;
unsigned short data;
- u32 dl;
+ unsigned int dl;
if (!devpriv->is_6143)
return;
@@ -1365,42 +1288,23 @@
ni_stc_writew(dev, ack, NISTC_INTA_ACK_REG);
}
-static void handle_a_interrupt(struct comedi_device *dev, unsigned short status,
- unsigned ai_mite_status)
+static void handle_a_interrupt(struct comedi_device *dev,
+ struct comedi_subdevice *s,
+ unsigned short status)
{
- struct comedi_subdevice *s = dev->read_subdev;
struct comedi_cmd *cmd = &s->async->cmd;
- /* 67xx boards don't have ai subdevice, but their gpct0 might generate an a interrupt */
- if (s->type == COMEDI_SUBD_UNUSED)
- return;
-
-#ifdef PCIDMA
- if (ai_mite_status & CHSR_LINKC)
- ni_sync_ai_dma(dev);
-
- if (ai_mite_status & ~(CHSR_INT | CHSR_LINKC | CHSR_DONE | CHSR_MRDY |
- CHSR_DRDY | CHSR_DRQ1 | CHSR_DRQ0 | CHSR_ERROR |
- CHSR_SABORT | CHSR_XFERR | CHSR_LxERR_mask)) {
- dev_err(dev->class_dev,
- "unknown mite interrupt (ai_mite_status=%08x)\n",
- ai_mite_status);
- s->async->events |= COMEDI_CB_ERROR;
- /* disable_irq(dev->irq); */
- }
-#endif
-
/* test for all uncommon interrupt events at the same time */
if (status & (NISTC_AI_STATUS1_ERR |
NISTC_AI_STATUS1_SC_TC | NISTC_AI_STATUS1_START1)) {
if (status == 0xffff) {
dev_err(dev->class_dev, "Card removed?\n");
- /* we probably aren't even running a command now,
- * so it's a good idea to be careful. */
- if (comedi_is_subdevice_running(s)) {
+ /*
+ * We probably aren't even running a command now,
+ * so it's a good idea to be careful.
+ */
+ if (comedi_is_subdevice_running(s))
s->async->events |= COMEDI_CB_ERROR;
- comedi_handle_events(dev, s);
- }
return;
}
if (status & NISTC_AI_STATUS1_ERR) {
@@ -1412,8 +1316,6 @@
s->async->events |= COMEDI_CB_ERROR;
if (status & NISTC_AI_STATUS1_OVER)
s->async->events |= COMEDI_CB_OVERFLOW;
-
- comedi_handle_events(dev, s);
return;
}
if (status & NISTC_AI_STATUS1_SC_TC) {
@@ -1425,8 +1327,11 @@
if (status & NISTC_AI_STATUS1_FIFO_HF) {
int i;
static const int timeout = 10;
- /* pcmcia cards (at least 6036) seem to stop producing interrupts if we
- *fail to get the fifo less than half full, so loop to be sure.*/
+ /*
+ * PCMCIA cards (at least 6036) seem to stop producing
+ * interrupts if we fail to get the fifo less than half
+ * full, so loop to be sure.
+ */
for (i = 0; i < timeout; ++i) {
ni_handle_fifo_half_full(dev);
if ((ni_stc_readw(dev, NISTC_AI_STATUS1_REG) &
@@ -1438,8 +1343,6 @@
if (status & NISTC_AI_STATUS1_STOP)
ni_handle_eos(dev, s);
-
- comedi_handle_events(dev, s);
}
static void ack_b_interrupt(struct comedi_device *dev, unsigned short b_status)
@@ -1465,29 +1368,9 @@
}
static void handle_b_interrupt(struct comedi_device *dev,
- unsigned short b_status, unsigned ao_mite_status)
+ struct comedi_subdevice *s,
+ unsigned short b_status)
{
- struct comedi_subdevice *s = dev->write_subdev;
- /* unsigned short ack=0; */
-
-#ifdef PCIDMA
- /* Currently, mite.c requires us to handle LINKC */
- if (ao_mite_status & CHSR_LINKC) {
- struct ni_private *devpriv = dev->private;
-
- mite_handle_b_linkc(devpriv->mite, dev);
- }
-
- if (ao_mite_status & ~(CHSR_INT | CHSR_LINKC | CHSR_DONE | CHSR_MRDY |
- CHSR_DRDY | CHSR_DRQ1 | CHSR_DRQ0 | CHSR_ERROR |
- CHSR_SABORT | CHSR_XFERR | CHSR_LxERR_mask)) {
- dev_err(dev->class_dev,
- "unknown mite interrupt (ao_mite_status=%08x)\n",
- ao_mite_status);
- s->async->events |= COMEDI_CB_ERROR;
- }
-#endif
-
if (b_status == 0xffff)
return;
if (b_status & NISTC_AO_STATUS1_OVERRUN) {
@@ -1515,8 +1398,6 @@
}
}
#endif
-
- comedi_handle_events(dev, s);
}
static void ni_ai_munge(struct comedi_device *dev, struct comedi_subdevice *s,
@@ -1606,8 +1487,11 @@
if (devpriv->is_611x || devpriv->is_6713) {
mite_prep_dma(devpriv->ao_mite_chan, 32, 32);
} else {
- /* doing 32 instead of 16 bit wide transfers from memory
- makes the mite do 32 bit pci transfers, doubling pci bandwidth. */
+ /*
+ * Doing 32 instead of 16 bit wide transfers from
+ * memory makes the mite do 32 bit pci transfers,
+ * doubling pci bandwidth.
+ */
mite_prep_dma(devpriv->ao_mite_chan, 16, 32);
}
mite_dma_arm(devpriv->ao_mite_chan);
@@ -1622,16 +1506,15 @@
#endif /* PCIDMA */
/*
- used for both cancel ioctl and board initialization
-
- this is pretty harsh for a cancel, but it works...
+ * used for both cancel ioctl and board initialization
+ *
+ * this is pretty harsh for a cancel, but it works...
*/
-
static int ni_ai_reset(struct comedi_device *dev, struct comedi_subdevice *s)
{
struct ni_private *devpriv = dev->private;
- unsigned ai_personal;
- unsigned ai_out_ctrl;
+ unsigned int ai_personal;
+ unsigned int ai_out_ctrl;
ni_release_ai_mite_channel(dev);
/* ai configuration */
@@ -1736,12 +1619,12 @@
unsigned int chan, range, aref;
unsigned int i;
unsigned int dither;
- unsigned range_code;
+ unsigned int range_code;
ni_stc_writew(dev, 1, NISTC_CFG_MEM_CLR_REG);
if ((list[0] & CR_ALT_SOURCE)) {
- unsigned bypass_bits;
+ unsigned int bypass_bits;
chan = CR_CHAN(list[0]);
range = CR_RANGE(list[0]);
@@ -1760,7 +1643,7 @@
ni_writel(dev, 0, NI_M_CFG_BYPASS_FIFO_REG);
}
for (i = 0; i < n_chan; i++) {
- unsigned config_bits = 0;
+ unsigned int config_bits = 0;
chan = CR_CHAN(list[i]);
aref = CR_AREF(list[i]);
@@ -1842,8 +1725,8 @@
return;
}
if (n_chan == 1 && !devpriv->is_611x && !devpriv->is_6143) {
- if (devpriv->changain_state
- && devpriv->changain_spec == list[0]) {
+ if (devpriv->changain_state &&
+ devpriv->changain_spec == list[0]) {
/* ready to go. */
return;
}
@@ -1857,8 +1740,8 @@
/* Set up Calibration mode if required */
if (devpriv->is_6143) {
- if ((list[0] & CR_ALT_SOURCE)
- && !devpriv->ai_calib_source_enabled) {
+ if ((list[0] & CR_ALT_SOURCE) &&
+ !devpriv->ai_calib_source_enabled) {
/* Strobe Relay enable bit */
ni_writew(dev, devpriv->ai_calib_source |
NI6143_CALIB_CHAN_RELAY_ON,
@@ -1866,9 +1749,10 @@
ni_writew(dev, devpriv->ai_calib_source,
NI6143_CALIB_CHAN_REG);
devpriv->ai_calib_source_enabled = 1;
- msleep_interruptible(100); /* Allow relays to change */
- } else if (!(list[0] & CR_ALT_SOURCE)
- && devpriv->ai_calib_source_enabled) {
+ /* Allow relays to change */
+ msleep_interruptible(100);
+ } else if (!(list[0] & CR_ALT_SOURCE) &&
+ devpriv->ai_calib_source_enabled) {
/* Strobe Relay disable bit */
ni_writew(dev, devpriv->ai_calib_source |
NI6143_CALIB_CHAN_RELAY_OFF,
@@ -1876,7 +1760,8 @@
ni_writew(dev, devpriv->ai_calib_source,
NI6143_CALIB_CHAN_REG);
devpriv->ai_calib_source_enabled = 0;
- msleep_interruptible(100); /* Allow relays to change */
+ /* Allow relays to change */
+ msleep_interruptible(100);
}
}
@@ -1949,7 +1834,7 @@
struct ni_private *devpriv = dev->private;
unsigned int mask = (s->maxdata + 1) >> 1;
int i, n;
- unsigned signbits;
+ unsigned int signbits;
unsigned int d;
unsigned long dl;
@@ -1997,7 +1882,11 @@
ni_stc_writew(dev, NISTC_AI_CMD1_CONVERT_PULSE,
NISTC_AI_CMD1_REG);
- /* The 6143 has 32-bit FIFOs. You need to strobe a bit to move a single 16bit stranded sample into the FIFO */
+ /*
+ * The 6143 has 32-bit FIFOs. You need to strobe a
+ * bit to move a single 16bit stranded sample into
+ * the FIFO.
+ */
dl = 0;
for (i = 0; i < NI_TIMEOUT; i++) {
if (ni_readl(dev, NI6143_AI_FIFO_STATUS_REG) &
@@ -2035,7 +1924,8 @@
data[n] = dl;
} else {
d = ni_readw(dev, NI_E_AI_FIFO_DATA_REG);
- d += signbits; /* subtle: needs to be short addition */
+ /* subtle: needs to be short addition */
+ d += signbits;
data[n] = d;
}
}
@@ -2043,8 +1933,8 @@
return insn->n;
}
-static int ni_ns_to_timer(const struct comedi_device *dev, unsigned nanosec,
- unsigned int flags)
+static int ni_ns_to_timer(const struct comedi_device *dev,
+ unsigned int nanosec, unsigned int flags)
{
struct ni_private *devpriv = dev->private;
int divider;
@@ -2064,14 +1954,14 @@
return divider - 1;
}
-static unsigned ni_timer_to_ns(const struct comedi_device *dev, int timer)
+static unsigned int ni_timer_to_ns(const struct comedi_device *dev, int timer)
{
struct ni_private *devpriv = dev->private;
return devpriv->clock_ns * (timer + 1);
}
-static void ni_cmd_set_mite_transfer(struct mite_dma_descriptor_ring *ring,
+static void ni_cmd_set_mite_transfer(struct mite_ring *ring,
struct comedi_subdevice *sdev,
const struct comedi_cmd *cmd,
unsigned int max_count) {
@@ -2102,8 +1992,8 @@
#endif
}
-static unsigned ni_min_ai_scan_period_ns(struct comedi_device *dev,
- unsigned num_channels)
+static unsigned int ni_min_ai_scan_period_ns(struct comedi_device *dev,
+ unsigned int num_channels)
{
const struct ni_board_struct *board = dev->board_ptr;
struct ni_private *devpriv = dev->private;
@@ -2294,7 +2184,7 @@
int start_stop_select = 0;
unsigned int stop_count;
int interrupt_a_enable = 0;
- unsigned ai_trig;
+ unsigned int ai_trig;
if (dev->irq == 0) {
dev_err(dev->class_dev, "cannot run command without an irq\n");
@@ -2307,8 +2197,10 @@
/* start configuration */
ni_stc_writew(dev, NISTC_RESET_AI_CFG_START, NISTC_RESET_REG);
- /* disable analog triggering for now, since it
- * interferes with the use of pfi0 */
+ /*
+ * Disable analog triggering for now, since it interferes
+ * with the use of pfi0.
+ */
devpriv->an_trig_etc_reg &= ~NISTC_ATRIG_ETC_ENA;
ni_stc_writew(dev, devpriv->an_trig_etc_reg, NISTC_ATRIG_ETC_REG);
@@ -2369,7 +2261,10 @@
if (stop_count == 0) {
devpriv->ai_cmd2 |= NISTC_AI_CMD2_END_ON_EOS;
interrupt_a_enable |= NISTC_INTA_ENA_AI_STOP;
- /* this is required to get the last sample for chanlist_len > 1, not sure why */
+ /*
+ * This is required to get the last sample for
+ * chanlist_len > 1, not sure why.
+ */
if (cmd->chanlist_len > 1)
start_stop_select |= NISTC_AI_STOP_POLARITY |
NISTC_AI_STOP_EDGE;
@@ -2489,7 +2384,7 @@
switch (devpriv->aimode) {
case AIMODE_HALF_FULL:
- /*generate FIFO interrupts and DMA requests on half-full */
+ /* FIFO interrupts and DMA requests on half-full */
#ifdef PCIDMA
ni_stc_writew(dev, NISTC_AI_MODE3_FIFO_MODE_HF_E,
NISTC_AI_MODE3_REG);
@@ -2880,9 +2775,11 @@
if (trig_num != cmd->start_arg)
return -EINVAL;
- /* Null trig at beginning prevent ao start trigger from executing more than
- once per command (and doing things like trying to allocate the ao dma channel
- multiple times) */
+ /*
+ * Null trig at beginning prevent ao start trigger from executing more
+ * than once per command (and doing things like trying to allocate the
+ * ao dma channel multiple times).
+ */
s->async->inttrig = NULL;
ni_set_bits(dev, NISTC_INTB_ENA_REG,
@@ -2951,7 +2848,7 @@
const struct comedi_cmd *cmd)
{
const struct ni_board_struct *board = dev->board_ptr;
- unsigned bits;
+ unsigned int bits;
ni_stc_writew(dev, NISTC_RESET_AO_CFG_START, NISTC_RESET_REG);
@@ -2999,6 +2896,7 @@
const struct comedi_cmd *cmd)
{
struct ni_private *devpriv = dev->private;
+ unsigned int trigsel;
ni_stc_writew(dev, NISTC_RESET_AO_CFG_START, NISTC_RESET_REG);
@@ -3012,39 +2910,20 @@
}
ni_stc_writew(dev, devpriv->ao_mode1, NISTC_AO_MODE1_REG);
- {
- unsigned int trigsel = devpriv->ao_trigger_select;
-
- switch (cmd->start_src) {
- case TRIG_INT:
- case TRIG_NOW:
- trigsel &= ~(NISTC_AO_TRIG_START1_POLARITY |
- NISTC_AO_TRIG_START1_SEL_MASK);
- trigsel |= NISTC_AO_TRIG_START1_EDGE |
- NISTC_AO_TRIG_START1_SYNC;
- break;
- case TRIG_EXT:
- trigsel = NISTC_AO_TRIG_START1_SEL(
- CR_CHAN(cmd->start_arg) + 1);
- if (cmd->start_arg & CR_INVERT)
- /*
- * 0=active high, 1=active low.
- * see daq-stc 3-24 (p186)
- */
- trigsel |= NISTC_AO_TRIG_START1_POLARITY;
- if (cmd->start_arg & CR_EDGE)
- /* 0=edge detection disabled, 1=enabled */
- trigsel |= NISTC_AO_TRIG_START1_EDGE;
- break;
- default:
- BUG();
- break;
- }
-
- devpriv->ao_trigger_select = trigsel;
- ni_stc_writew(dev, devpriv->ao_trigger_select,
- NISTC_AO_TRIG_SEL_REG);
+ if (cmd->start_src == TRIG_INT) {
+ trigsel = NISTC_AO_TRIG_START1_EDGE |
+ NISTC_AO_TRIG_START1_SYNC;
+ } else { /* TRIG_EXT */
+ trigsel = NISTC_AO_TRIG_START1_SEL(CR_CHAN(cmd->start_arg) + 1);
+ /* 0=active high, 1=active low. see daq-stc 3-24 (p186) */
+ if (cmd->start_arg & CR_INVERT)
+ trigsel |= NISTC_AO_TRIG_START1_POLARITY;
+ /* 0=edge detection disabled, 1=enabled */
+ if (cmd->start_arg & CR_EDGE)
+ trigsel |= NISTC_AO_TRIG_START1_EDGE;
}
+ ni_stc_writew(dev, trigsel, NISTC_AO_TRIG_SEL_REG);
+
/* AO_Delayed_START1 = 0, we do not support delayed start...yet */
/* sync */
@@ -3149,8 +3028,9 @@
NISTC_AO_MODE1_UPDATE_SRC_POLARITY
);
- switch (cmd->scan_begin_src) {
- case TRIG_TIMER:
+ if (cmd->scan_begin_src == TRIG_TIMER) {
+ unsigned int trigvar;
+
devpriv->ao_cmd2 &= ~NISTC_AO_CMD2_BC_GATE_ENA;
/*
@@ -3181,34 +3061,25 @@
* eseries/ni67xx and tMSeries.h for mseries.
*/
- {
- unsigned trigvar = ni_ns_to_timer(dev,
- cmd->scan_begin_arg,
- CMDF_ROUND_NEAREST);
+ trigvar = ni_ns_to_timer(dev, cmd->scan_begin_arg,
+ CMDF_ROUND_NEAREST);
- /*
- * Wait N TB3 ticks after the start trigger before
- * clocking(N must be >=2).
- */
- /* following line: 2-1 per STC */
- ni_stc_writel(dev, 1, NISTC_AO_UI_LOADA_REG);
- ni_stc_writew(dev, NISTC_AO_CMD1_UI_LOAD,
- NISTC_AO_CMD1_REG);
- /* following line: N-1 per STC */
- ni_stc_writel(dev, trigvar - 1, NISTC_AO_UI_LOADA_REG);
- }
- break;
- case TRIG_EXT:
+ /*
+ * Wait N TB3 ticks after the start trigger before
+ * clocking (N must be >=2).
+ */
+ /* following line: 2-1 per STC */
+ ni_stc_writel(dev, 1, NISTC_AO_UI_LOADA_REG);
+ ni_stc_writew(dev, NISTC_AO_CMD1_UI_LOAD, NISTC_AO_CMD1_REG);
+ /* following line: N-1 per STC */
+ ni_stc_writel(dev, trigvar - 1, NISTC_AO_UI_LOADA_REG);
+ } else { /* TRIG_EXT */
/* FIXME: assert scan_begin_arg != 0, ret failure otherwise */
devpriv->ao_cmd2 |= NISTC_AO_CMD2_BC_GATE_ENA;
devpriv->ao_mode1 |= NISTC_AO_MODE1_UPDATE_SRC(
CR_CHAN(cmd->scan_begin_arg));
if (cmd->scan_begin_arg & CR_INVERT)
devpriv->ao_mode1 |= NISTC_AO_MODE1_UPDATE_SRC_POLARITY;
- break;
- default:
- BUG();
- break;
}
ni_stc_writew(dev, devpriv->ao_cmd2, NISTC_AO_CMD2_REG);
@@ -3231,7 +3102,7 @@
{
struct ni_private *devpriv = dev->private;
const struct comedi_cmd *cmd = &s->async->cmd;
- unsigned bits = 0;
+ unsigned int bits = 0;
ni_stc_writew(dev, NISTC_RESET_AO_CFG_START, NISTC_RESET_REG);
@@ -3474,7 +3345,6 @@
devpriv->ao_mode3 = NISTC_AO_MODE3_LAST_GATE_DISABLE;
else
devpriv->ao_mode3 = 0;
- devpriv->ao_trigger_select = 0;
ni_stc_writew(dev, 0, NISTC_AO_PERSONAL_REG);
ni_stc_writew(dev, 0, NISTC_AO_CMD1_REG);
@@ -3550,6 +3420,7 @@
return insn->n;
}
+#ifdef PCIDMA
static int ni_m_series_dio_insn_config(struct comedi_device *dev,
struct comedi_subdevice *s,
struct comedi_insn *insn,
@@ -3652,13 +3523,11 @@
unsigned int trig_num)
{
struct comedi_cmd *cmd = &s->async->cmd;
- const unsigned timeout = 1000;
+ const unsigned int timeout = 1000;
int retval = 0;
- unsigned i;
-#ifdef PCIDMA
+ unsigned int i;
struct ni_private *devpriv = dev->private;
unsigned long flags;
-#endif
if (trig_num != cmd->start_arg)
return -EINVAL;
@@ -3668,7 +3537,6 @@
/* read alloc the entire buffer */
comedi_buf_read_alloc(s, s->async->prealloc_bufsz);
-#ifdef PCIDMA
spin_lock_irqsave(&devpriv->mite_channel_lock, flags);
if (devpriv->cdo_mite_chan) {
mite_prep_dma(devpriv->cdo_mite_chan, 32, 32);
@@ -3680,7 +3548,7 @@
spin_unlock_irqrestore(&devpriv->mite_channel_lock, flags);
if (retval < 0)
return retval;
-#endif
+
/*
* XXX not sure what interrupt C group does
* wait for dma to fill output fifo
@@ -3690,7 +3558,7 @@
if (ni_readl(dev, NI_M_CDIO_STATUS_REG) &
NI_M_CDIO_STATUS_CDO_FIFO_FULL)
break;
- udelay(10);
+ usleep_range(10, 100);
}
if (i == timeout) {
dev_err(dev->class_dev, "dma failed to fill cdo fifo!\n");
@@ -3708,7 +3576,7 @@
{
struct ni_private *devpriv = dev->private;
const struct comedi_cmd *cmd = &s->async->cmd;
- unsigned cdo_mode_bits;
+ unsigned int cdo_mode_bits;
int retval;
ni_writel(dev, NI_M_CDO_CMD_RESET, NI_M_CDIO_CMD_REG);
@@ -3759,28 +3627,14 @@
static void handle_cdio_interrupt(struct comedi_device *dev)
{
struct ni_private *devpriv = dev->private;
- unsigned cdio_status;
+ unsigned int cdio_status;
struct comedi_subdevice *s = &dev->subdevices[NI_DIO_SUBDEV];
-#ifdef PCIDMA
unsigned long flags;
-#endif
- if (!devpriv->is_m_series)
- return;
-#ifdef PCIDMA
spin_lock_irqsave(&devpriv->mite_channel_lock, flags);
- if (devpriv->cdo_mite_chan) {
- unsigned cdo_mite_status =
- mite_get_status(devpriv->cdo_mite_chan);
- if (cdo_mite_status & CHSR_LINKC) {
- writel(CHOR_CLRLC,
- devpriv->mite->mite_io_addr +
- MITE_CHOR(devpriv->cdo_mite_chan->channel));
- }
- mite_sync_output_dma(devpriv->cdo_mite_chan, s);
- }
+ if (devpriv->cdo_mite_chan)
+ mite_ack_linkc(devpriv->cdo_mite_chan, s, true);
spin_unlock_irqrestore(&devpriv->mite_channel_lock, flags);
-#endif
cdio_status = ni_readl(dev, NI_M_CDIO_STATUS_REG);
if (cdio_status & NI_M_CDIO_STATUS_CDO_ERROR) {
@@ -3796,6 +3650,7 @@
}
comedi_handle_events(dev, s);
}
+#endif /* PCIDMA */
static int ni_serial_hw_readwrite8(struct comedi_device *dev,
struct comedi_subdevice *s,
@@ -3813,7 +3668,7 @@
status1 = ni_stc_readw(dev, NISTC_STATUS1_REG);
if (status1 & NISTC_STATUS1_SERIO_IN_PROG) {
err = -EBUSY;
- goto Error;
+ goto error;
}
devpriv->dio_control |= NISTC_DIO_CTRL_HW_SER_START;
@@ -3829,7 +3684,7 @@
dev_err(dev->class_dev,
"SPI serial I/O didn't finish in time!\n");
err = -ETIME;
- goto Error;
+ goto error;
}
}
@@ -3842,7 +3697,7 @@
if (data_in)
*data_in = ni_stc_readw(dev, NISTC_DIO_SERIAL_IN_REG);
-Error:
+error:
ni_stc_writew(dev, devpriv->dio_control, NISTC_DIO_CTRL_REG);
return err;
@@ -3860,16 +3715,20 @@
udelay((devpriv->serial_interval_ns + 999) / 1000);
for (mask = 0x80; mask; mask >>= 1) {
- /* Output current bit; note that we cannot touch s->state
- because it is a per-subdevice field, and serial is
- a separate subdevice from DIO. */
+ /*
+ * Output current bit; note that we cannot touch s->state
+ * because it is a per-subdevice field, and serial is
+ * a separate subdevice from DIO.
+ */
devpriv->dio_output &= ~NISTC_DIO_SDOUT;
if (data_out & mask)
devpriv->dio_output |= NISTC_DIO_SDOUT;
ni_stc_writew(dev, devpriv->dio_output, NISTC_DIO_OUT_REG);
- /* Assert SDCLK (active low, inverted), wait for half of
- the delay, deassert SDCLK, and wait for the other half. */
+ /*
+ * Assert SDCLK (active low, inverted), wait for half of
+ * the delay, deassert SDCLK, and wait for the other half.
+ */
devpriv->dio_control |= NISTC_DIO_SDCLK;
ni_stc_writew(dev, devpriv->dio_control, NISTC_DIO_CTRL_REG);
@@ -3897,7 +3756,7 @@
unsigned int *data)
{
struct ni_private *devpriv = dev->private;
- unsigned clk_fout = devpriv->clock_and_fout;
+ unsigned int clk_fout = devpriv->clock_and_fout;
int err = insn->n;
unsigned char byte_out, byte_in = 0;
@@ -3916,8 +3775,10 @@
data[1] = SERIAL_DISABLED;
devpriv->serial_interval_ns = data[1];
} else if (data[1] <= SERIAL_600NS) {
- /* Warning: this clock speed is too fast to reliably
- control SCXI. */
+ /*
+ * Warning: this clock speed is too fast to reliably
+ * control SCXI.
+ */
devpriv->dio_control &= ~NISTC_DIO_CTRL_HW_SER_TIMEBASE;
clk_fout |= NISTC_CLK_FOUT_SLOW_TIMEBASE;
clk_fout &= ~NISTC_CLK_FOUT_DIO_SER_OUT_DIV2;
@@ -3933,10 +3794,12 @@
devpriv->dio_control |= NISTC_DIO_CTRL_HW_SER_TIMEBASE;
clk_fout |= NISTC_CLK_FOUT_SLOW_TIMEBASE |
NISTC_CLK_FOUT_DIO_SER_OUT_DIV2;
- /* Note: NISTC_CLK_FOUT_DIO_SER_OUT_DIV2 only affects
- 600ns/1.2us. If you turn divide_by_2 off with the
- slow clock, you will still get 10us, except then
- all your delays are wrong. */
+ /*
+ * Note: NISTC_CLK_FOUT_DIO_SER_OUT_DIV2 only affects
+ * 600ns/1.2us. If you turn divide_by_2 off with the
+ * slow clock, you will still get 10us, except then
+ * all your delays are wrong.
+ */
data[1] = SERIAL_10US;
devpriv->serial_interval_ns = data[1];
} else {
@@ -4046,15 +3909,11 @@
return regmap->mio_reg;
}
-static void ni_gpct_write_register(struct ni_gpct *counter, unsigned bits,
+static void ni_gpct_write_register(struct ni_gpct *counter, unsigned int bits,
enum ni_gpct_register reg)
{
struct comedi_device *dev = counter->counter_dev->dev;
unsigned int stc_register = ni_gpct_to_stc_register(dev, reg);
- static const unsigned gpct_interrupt_a_enable_mask =
- NISTC_INTA_ENA_G0_GATE | NISTC_INTA_ENA_G0_TC;
- static const unsigned gpct_interrupt_b_enable_mask =
- NISTC_INTB_ENA_G1_GATE | NISTC_INTB_ENA_G1_TC;
if (stc_register == 0)
return;
@@ -4082,25 +3941,22 @@
/* 16 bit registers */
case NITIO_G0_INT_ENA:
- BUG_ON(bits & ~gpct_interrupt_a_enable_mask);
ni_set_bitfield(dev, stc_register,
- gpct_interrupt_a_enable_mask, bits);
+ NISTC_INTA_ENA_G0_GATE | NISTC_INTA_ENA_G0_TC,
+ bits);
break;
case NITIO_G1_INT_ENA:
- BUG_ON(bits & ~gpct_interrupt_b_enable_mask);
ni_set_bitfield(dev, stc_register,
- gpct_interrupt_b_enable_mask, bits);
+ NISTC_INTB_ENA_G1_GATE | NISTC_INTB_ENA_G1_TC,
+ bits);
break;
- case NITIO_G01_RESET:
- BUG_ON(bits & ~(NISTC_RESET_G0 | NISTC_RESET_G1));
- /* fall-through */
default:
ni_stc_writew(dev, bits, stc_register);
}
}
-static unsigned ni_gpct_read_register(struct ni_gpct *counter,
- enum ni_gpct_register reg)
+static unsigned int ni_gpct_read_register(struct ni_gpct *counter,
+ enum ni_gpct_register reg)
{
struct comedi_device *dev = counter->counter_dev->dev;
unsigned int stc_register = ni_gpct_to_stc_register(dev, reg);
@@ -4227,7 +4083,7 @@
unsigned int *data)
{
struct ni_private *devpriv = dev->private;
- unsigned up_count, down_count;
+ unsigned int up_count, down_count;
switch (data[0]) {
case INSN_CONFIG_PWM_OUTPUT:
@@ -4287,7 +4143,7 @@
unsigned int *data)
{
struct ni_private *devpriv = dev->private;
- unsigned up_count, down_count;
+ unsigned int up_count, down_count;
switch (data[0]) {
case INSN_CONFIG_PWM_OUTPUT:
@@ -4343,13 +4199,13 @@
static int pack_mb88341(int addr, int val, int *bitstring)
{
/*
- Fujitsu MB 88341
- Note that address bits are reversed. Thanks to
- Ingo Keen for noticing this.
-
- Note also that the 88341 expects address values from
- 1-12, whereas we use channel numbers 0-11. The NI
- docs use 1-12, also, so be careful here.
+ * Fujitsu MB 88341
+ * Note that address bits are reversed. Thanks to
+ * Ingo Keen for noticing this.
+ *
+ * Note also that the 88341 expects address values from
+ * 1-12, whereas we use channel numbers 0-11. The NI
+ * docs use 1-12, also, so be careful here.
*/
addr++;
*bitstring = ((addr & 0x1) << 11) |
@@ -4495,12 +4351,12 @@
s->n_chan = n_chans;
if (diffbits) {
- unsigned int *maxdata_list;
+ unsigned int *maxdata_list = devpriv->caldac_maxdata_list;
if (n_chans > MAX_N_CALDACS)
dev_err(dev->class_dev,
"BUG! MAX_N_CALDACS too small\n");
- s->maxdata_list = maxdata_list = devpriv->caldac_maxdata_list;
+ s->maxdata_list = maxdata_list;
chan = 0;
for (i = 0; i < n_dacs; i++) {
type = board->caldac[i];
@@ -4574,8 +4430,8 @@
return 1;
}
-static unsigned ni_old_get_pfi_routing(struct comedi_device *dev,
- unsigned chan)
+static unsigned int ni_old_get_pfi_routing(struct comedi_device *dev,
+ unsigned int chan)
{
/* pre-m-series boards have fixed signals on pfi pins */
switch (chan) {
@@ -4607,7 +4463,7 @@
}
static int ni_old_set_pfi_routing(struct comedi_device *dev,
- unsigned chan, unsigned source)
+ unsigned int chan, unsigned int source)
{
/* pre-m-series boards have fixed signals on pfi pins */
if (source != ni_old_get_pfi_routing(dev, chan))
@@ -4615,21 +4471,21 @@
return 2;
}
-static unsigned ni_m_series_get_pfi_routing(struct comedi_device *dev,
- unsigned chan)
+static unsigned int ni_m_series_get_pfi_routing(struct comedi_device *dev,
+ unsigned int chan)
{
struct ni_private *devpriv = dev->private;
- const unsigned array_offset = chan / 3;
+ const unsigned int array_offset = chan / 3;
return NI_M_PFI_OUT_SEL_TO_SRC(chan,
devpriv->pfi_output_select_reg[array_offset]);
}
static int ni_m_series_set_pfi_routing(struct comedi_device *dev,
- unsigned chan, unsigned source)
+ unsigned int chan, unsigned int source)
{
struct ni_private *devpriv = dev->private;
- unsigned index = chan / 3;
+ unsigned int index = chan / 3;
unsigned short val = devpriv->pfi_output_select_reg[index];
if ((source & 0x1f) != source)
@@ -4643,7 +4499,8 @@
return 2;
}
-static unsigned ni_get_pfi_routing(struct comedi_device *dev, unsigned chan)
+static unsigned int ni_get_pfi_routing(struct comedi_device *dev,
+ unsigned int chan)
{
struct ni_private *devpriv = dev->private;
@@ -4652,8 +4509,8 @@
: ni_old_get_pfi_routing(dev, chan);
}
-static int ni_set_pfi_routing(struct comedi_device *dev, unsigned chan,
- unsigned source)
+static int ni_set_pfi_routing(struct comedi_device *dev,
+ unsigned int chan, unsigned int source)
{
struct ni_private *devpriv = dev->private;
@@ -4663,11 +4520,11 @@
}
static int ni_config_filter(struct comedi_device *dev,
- unsigned pfi_channel,
+ unsigned int pfi_channel,
enum ni_pfi_filter_select filter)
{
struct ni_private *devpriv = dev->private;
- unsigned bits;
+ unsigned int bits;
if (!devpriv->is_m_series)
return -ENOTSUPP;
@@ -4818,9 +4675,12 @@
unsigned int channel_select;
const unsigned int INTERNAL_REF = 0x1000;
- /* Set calibration adc source. Docs lie, reference select bits 8 to 11
+ /*
+ * Set calibration adc source. Docs lie, reference select bits 8 to 11
* do nothing. bit 12 seems to chooses internal reference voltage, bit
- * 13 causes the adc input to go overrange (maybe reads external reference?) */
+ * 13 causes the adc input to go overrange (maybe reads external
+ * reference?)
+ */
if (insn->chanspec & CR_ALT_SOURCE)
channel_select = INTERNAL_REF;
else
@@ -4875,27 +4735,28 @@
* Find best multiplier/divider to try and get the PLL running at 80 MHz
* given an arbitrary frequency input clock.
*/
-static int ni_mseries_get_pll_parameters(unsigned reference_period_ns,
- unsigned *freq_divider,
- unsigned *freq_multiplier,
- unsigned *actual_period_ns)
+static int ni_mseries_get_pll_parameters(unsigned int reference_period_ns,
+ unsigned int *freq_divider,
+ unsigned int *freq_multiplier,
+ unsigned int *actual_period_ns)
{
- unsigned div;
- unsigned best_div = 1;
- unsigned mult;
- unsigned best_mult = 1;
- static const unsigned pico_per_nano = 1000;
-
- const unsigned reference_picosec = reference_period_ns * pico_per_nano;
- /* m-series wants the phased-locked loop to output 80MHz, which is divided by 4 to
- * 20 MHz for most timing clocks */
- static const unsigned target_picosec = 12500;
- static const unsigned fudge_factor_80_to_20Mhz = 4;
+ unsigned int div;
+ unsigned int best_div = 1;
+ unsigned int mult;
+ unsigned int best_mult = 1;
+ static const unsigned int pico_per_nano = 1000;
+ const unsigned int reference_picosec = reference_period_ns *
+ pico_per_nano;
+ /*
+ * m-series wants the phased-locked loop to output 80MHz, which is
+ * divided by 4 to 20 MHz for most timing clocks
+ */
+ static const unsigned int target_picosec = 12500;
int best_period_picosec = 0;
for (div = 1; div <= NI_M_PLL_MAX_DIVISOR; ++div) {
for (mult = 1; mult <= NI_M_PLL_MAX_MULTIPLIER; ++mult) {
- unsigned new_period_ps =
+ unsigned int new_period_ps =
(reference_picosec * div) / mult;
if (abs(new_period_ps - target_picosec) <
abs(best_period_picosec - target_picosec)) {
@@ -4910,29 +4771,33 @@
*freq_divider = best_div;
*freq_multiplier = best_mult;
- *actual_period_ns = DIV_ROUND_CLOSEST(best_period_picosec *
- fudge_factor_80_to_20Mhz,
+ /* return the actual period (* fudge factor for 80 to 20 MHz) */
+ *actual_period_ns = DIV_ROUND_CLOSEST(best_period_picosec * 4,
pico_per_nano);
return 0;
}
static int ni_mseries_set_pll_master_clock(struct comedi_device *dev,
- unsigned source, unsigned period_ns)
+ unsigned int source,
+ unsigned int period_ns)
{
struct ni_private *devpriv = dev->private;
- static const unsigned min_period_ns = 50;
- static const unsigned max_period_ns = 1000;
- static const unsigned timeout = 1000;
- unsigned pll_control_bits;
- unsigned freq_divider;
- unsigned freq_multiplier;
- unsigned rtsi;
- unsigned i;
+ static const unsigned int min_period_ns = 50;
+ static const unsigned int max_period_ns = 1000;
+ static const unsigned int timeout = 1000;
+ unsigned int pll_control_bits;
+ unsigned int freq_divider;
+ unsigned int freq_multiplier;
+ unsigned int rtsi;
+ unsigned int i;
int retval;
if (source == NI_MIO_PLL_PXI10_CLOCK)
period_ns = 100;
- /* these limits are somewhat arbitrary, but NI advertises 1 to 20MHz range so we'll use that */
+ /*
+ * These limits are somewhat arbitrary, but NI advertises 1 to 20MHz
+ * range so we'll use that.
+ */
if (period_ns < min_period_ns || period_ns > max_period_ns) {
dev_err(dev->class_dev,
"%s: you must specify an input clock frequency between %i and %i nanosec for the phased-lock loop\n",
@@ -4982,7 +4847,7 @@
ni_writew(dev, pll_control_bits, NI_M_PLL_CTRL_REG);
devpriv->clock_source = source;
- /* it seems to typically take a few hundred microseconds for PLL to lock */
+ /* it takes a few hundred microseconds for PLL to lock */
for (i = 0; i < timeout; ++i) {
if (ni_readw(dev, NI_M_PLL_STATUS_REG) & NI_M_PLL_STATUS_LOCKED)
break;
@@ -4998,7 +4863,7 @@
}
static int ni_set_master_clock(struct comedi_device *dev,
- unsigned source, unsigned period_ns)
+ unsigned int source, unsigned int period_ns)
{
struct ni_private *devpriv = dev->private;
@@ -5043,7 +4908,7 @@
}
static int ni_valid_rtsi_output_source(struct comedi_device *dev,
- unsigned chan, unsigned source)
+ unsigned int chan, unsigned int source)
{
struct ni_private *devpriv = dev->private;
@@ -5078,7 +4943,7 @@
}
static int ni_set_rtsi_routing(struct comedi_device *dev,
- unsigned chan, unsigned src)
+ unsigned int chan, unsigned int src)
{
struct ni_private *devpriv = dev->private;
@@ -5098,7 +4963,8 @@
return 2;
}
-static unsigned ni_get_rtsi_routing(struct comedi_device *dev, unsigned chan)
+static unsigned int ni_get_rtsi_routing(struct comedi_device *dev,
+ unsigned int chan)
{
struct ni_private *devpriv = dev->private;
@@ -5262,10 +5128,10 @@
static irqreturn_t ni_E_interrupt(int irq, void *d)
{
struct comedi_device *dev = d;
+ struct comedi_subdevice *s_ai = dev->read_subdev;
+ struct comedi_subdevice *s_ao = dev->write_subdev;
unsigned short a_status;
unsigned short b_status;
- unsigned int ai_mite_status = 0;
- unsigned int ao_mite_status = 0;
unsigned long flags;
#ifdef PCIDMA
struct ni_private *devpriv = dev->private;
@@ -5273,7 +5139,7 @@
if (!dev->attached)
return IRQ_NONE;
- smp_mb(); /* make sure dev->attached is checked before handler does anything else. */
+ smp_mb(); /* make sure dev->attached is checked */
/* lock to avoid race with comedi_poll */
spin_lock_irqsave(&dev->spinlock, flags);
@@ -5284,34 +5150,33 @@
unsigned long flags_too;
spin_lock_irqsave(&devpriv->mite_channel_lock, flags_too);
- if (devpriv->ai_mite_chan) {
- ai_mite_status = mite_get_status(devpriv->ai_mite_chan);
- if (ai_mite_status & CHSR_LINKC)
- writel(CHOR_CLRLC,
- devpriv->mite->mite_io_addr +
- MITE_CHOR(devpriv->
- ai_mite_chan->channel));
- }
- if (devpriv->ao_mite_chan) {
- ao_mite_status = mite_get_status(devpriv->ao_mite_chan);
- if (ao_mite_status & CHSR_LINKC)
- writel(CHOR_CLRLC,
- devpriv->mite->mite_io_addr +
- MITE_CHOR(devpriv->
- ao_mite_chan->channel));
- }
+ if (s_ai && devpriv->ai_mite_chan)
+ mite_ack_linkc(devpriv->ai_mite_chan, s_ai, false);
+ if (s_ao && devpriv->ao_mite_chan)
+ mite_ack_linkc(devpriv->ao_mite_chan, s_ao, false);
spin_unlock_irqrestore(&devpriv->mite_channel_lock, flags_too);
}
#endif
ack_a_interrupt(dev, a_status);
ack_b_interrupt(dev, b_status);
- if ((a_status & NISTC_AI_STATUS1_INTA) || (ai_mite_status & CHSR_INT))
- handle_a_interrupt(dev, a_status, ai_mite_status);
- if ((b_status & NISTC_AO_STATUS1_INTB) || (ao_mite_status & CHSR_INT))
- handle_b_interrupt(dev, b_status, ao_mite_status);
+ if (s_ai) {
+ if (a_status & NISTC_AI_STATUS1_INTA)
+ handle_a_interrupt(dev, s_ai, a_status);
+ /* handle any interrupt or dma events */
+ comedi_handle_events(dev, s_ai);
+ }
+ if (s_ao) {
+ if (b_status & NISTC_AO_STATUS1_INTB)
+ handle_b_interrupt(dev, s_ao, b_status);
+ /* handle any interrupt or dma events */
+ comedi_handle_events(dev, s_ao);
+ }
handle_gpct_interrupt(dev, 0);
handle_gpct_interrupt(dev, 1);
- handle_cdio_interrupt(dev);
+#ifdef PCIDMA
+ if (devpriv->is_m_series)
+ handle_cdio_interrupt(dev);
+#endif
spin_unlock_irqrestore(&dev->spinlock, flags);
return IRQ_HANDLED;
@@ -5333,7 +5198,7 @@
}
static int ni_E_init(struct comedi_device *dev,
- unsigned interrupt_pin, unsigned irq_polarity)
+ unsigned int interrupt_pin, unsigned int irq_polarity)
{
const struct ni_board_struct *board = dev->board_ptr;
struct ni_private *devpriv = dev->private;
@@ -5450,6 +5315,7 @@
s->maxdata = 1;
s->range_table = &range_digital;
if (devpriv->is_m_series) {
+#ifdef PCIDMA
s->subdev_flags |= SDF_LSAMPL;
s->insn_bits = ni_m_series_dio_insn_bits;
s->insn_config = ni_m_series_dio_insn_config;
@@ -5469,6 +5335,7 @@
NI_M_CDI_CMD_RESET,
NI_M_CDIO_CMD_REG);
ni_writel(dev, s->io_bits, NI_M_DIO_DIR_REG);
+#endif /* PCIDMA */
} else {
s->insn_bits = ni_dio_insn_bits;
s->insn_config = ni_dio_insn_config;
@@ -5675,8 +5542,6 @@
{
struct ni_private *devpriv = dev->private;
- if (devpriv) {
- if (devpriv->counter_dev)
- ni_gpct_device_destroy(devpriv->counter_dev);
- }
+ if (devpriv)
+ ni_gpct_device_destroy(devpriv->counter_dev);
}
diff --git a/drivers/staging/comedi/drivers/ni_pcidio.c b/drivers/staging/comedi/drivers/ni_pcidio.c
index 7112c3f..02a5329 100644
--- a/drivers/staging/comedi/drivers/ni_pcidio.c
+++ b/drivers/staging/comedi/drivers/ni_pcidio.c
@@ -284,12 +284,12 @@
};
struct nidio96_private {
- struct mite_struct *mite;
+ struct mite *mite;
int boardtype;
int dio;
unsigned short OpModeBits;
struct mite_channel *di_mite_chan;
- struct mite_dma_descriptor_ring *di_mite_ring;
+ struct mite_ring *di_mite_ring;
spinlock_t mite_channel_lock;
};
@@ -324,8 +324,6 @@
spin_lock_irqsave(&devpriv->mite_channel_lock, flags);
if (devpriv->di_mite_chan) {
- mite_dma_disarm(devpriv->di_mite_chan);
- mite_dma_reset(devpriv->di_mite_chan);
mite_release_channel(devpriv->di_mite_chan);
devpriv->di_mite_chan = NULL;
writeb(primary_DMAChannel_bits(0) |
@@ -370,7 +368,7 @@
spin_lock_irqsave(&dev->spinlock, irq_flags);
spin_lock(&devpriv->mite_channel_lock);
if (devpriv->di_mite_chan)
- mite_sync_input_dma(devpriv->di_mite_chan, s);
+ mite_sync_dma(devpriv->di_mite_chan, s);
spin_unlock(&devpriv->mite_channel_lock);
count = comedi_buf_n_bytes_ready(s);
spin_unlock_irqrestore(&dev->spinlock, irq_flags);
@@ -383,12 +381,10 @@
struct nidio96_private *devpriv = dev->private;
struct comedi_subdevice *s = dev->read_subdev;
struct comedi_async *async = s->async;
- struct mite_struct *mite = devpriv->mite;
unsigned int auxdata;
int flags;
int status;
int work = 0;
- unsigned int m_status = 0;
/* interrupcions parasites */
if (!dev->attached) {
@@ -403,24 +399,9 @@
flags = readb(dev->mmio + Group_1_Flags);
spin_lock(&devpriv->mite_channel_lock);
- if (devpriv->di_mite_chan)
- m_status = mite_get_status(devpriv->di_mite_chan);
-
- if (m_status & CHSR_INT) {
- if (m_status & CHSR_LINKC) {
- writel(CHOR_CLRLC,
- mite->mite_io_addr +
- MITE_CHOR(devpriv->di_mite_chan->channel));
- mite_sync_input_dma(devpriv->di_mite_chan, s);
- /* XXX need to byteswap */
- }
- if (m_status & ~(CHSR_INT | CHSR_LINKC | CHSR_DONE | CHSR_DRDY |
- CHSR_DRQ1 | CHSR_MRDY)) {
- dev_dbg(dev->class_dev,
- "unknown mite interrupt, disabling IRQ\n");
- async->events |= COMEDI_CB_ERROR;
- disable_irq(dev->irq);
- }
+ if (devpriv->di_mite_chan) {
+ mite_ack_linkc(devpriv->di_mite_chan, s, false);
+ /* XXX need to byteswap sync'ed dma */
}
spin_unlock(&devpriv->mite_channel_lock);
@@ -916,14 +897,10 @@
spin_lock_init(&devpriv->mite_channel_lock);
- devpriv->mite = mite_alloc(pcidev);
+ devpriv->mite = mite_attach(dev, false); /* use win0 */
if (!devpriv->mite)
return -ENOMEM;
- ret = mite_setup(dev, devpriv->mite);
- if (ret < 0)
- return ret;
-
devpriv->di_mite_ring = mite_alloc_ring(devpriv->mite);
if (!devpriv->di_mite_ring)
return -ENOMEM;
diff --git a/drivers/staging/comedi/drivers/ni_pcimio.c b/drivers/staging/comedi/drivers/ni_pcimio.c
index 231e37d..344aa34 100644
--- a/drivers/staging/comedi/drivers/ni_pcimio.c
+++ b/drivers/staging/comedi/drivers/ni_pcimio.c
@@ -1061,6 +1061,8 @@
static void m_series_init_eeprom_buffer(struct comedi_device *dev)
{
struct ni_private *devpriv = dev->private;
+ struct mite *mite = devpriv->mite;
+ resource_size_t daq_phys_addr;
static const int Start_Cal_EEPROM = 0x400;
static const unsigned window_size = 10;
static const int serial_number_eeprom_offset = 0x4;
@@ -1070,15 +1072,17 @@
unsigned old_iodwcr1_bits;
int i;
- old_iodwbsr_bits = readl(devpriv->mite->mite_io_addr + MITE_IODWBSR);
- old_iodwbsr1_bits = readl(devpriv->mite->mite_io_addr + MITE_IODWBSR_1);
- old_iodwcr1_bits = readl(devpriv->mite->mite_io_addr + MITE_IODWCR_1);
- writel(0x0, devpriv->mite->mite_io_addr + MITE_IODWBSR);
- writel(((0x80 | window_size) | devpriv->mite->daq_phys_addr),
- devpriv->mite->mite_io_addr + MITE_IODWBSR_1);
- writel(0x1 | old_iodwcr1_bits,
- devpriv->mite->mite_io_addr + MITE_IODWCR_1);
- writel(0xf, devpriv->mite->mite_io_addr + 0x30);
+ /* IO Window 1 needs to be temporarily mapped to read the eeprom */
+ daq_phys_addr = pci_resource_start(mite->pcidev, 1);
+
+ old_iodwbsr_bits = readl(mite->mmio + MITE_IODWBSR);
+ old_iodwbsr1_bits = readl(mite->mmio + MITE_IODWBSR_1);
+ old_iodwcr1_bits = readl(mite->mmio + MITE_IODWCR_1);
+ writel(0x0, mite->mmio + MITE_IODWBSR);
+ writel(((0x80 | window_size) | daq_phys_addr),
+ mite->mmio + MITE_IODWBSR_1);
+ writel(0x1 | old_iodwcr1_bits, mite->mmio + MITE_IODWCR_1);
+ writel(0xf, mite->mmio + 0x30);
BUG_ON(serial_number_eeprom_length > sizeof(devpriv->serial_number));
for (i = 0; i < serial_number_eeprom_length; ++i) {
@@ -1090,10 +1094,10 @@
for (i = 0; i < M_SERIES_EEPROM_SIZE; ++i)
devpriv->eeprom_buffer[i] = ni_readb(dev, Start_Cal_EEPROM + i);
- writel(old_iodwbsr1_bits, devpriv->mite->mite_io_addr + MITE_IODWBSR_1);
- writel(old_iodwbsr_bits, devpriv->mite->mite_io_addr + MITE_IODWBSR);
- writel(old_iodwcr1_bits, devpriv->mite->mite_io_addr + MITE_IODWCR_1);
- writel(0x0, devpriv->mite->mite_io_addr + 0x30);
+ writel(old_iodwbsr1_bits, mite->mmio + MITE_IODWBSR_1);
+ writel(old_iodwbsr_bits, mite->mmio + MITE_IODWBSR);
+ writel(old_iodwcr1_bits, mite->mmio + MITE_IODWCR_1);
+ writel(0x0, mite->mmio + 0x30);
}
static void init_6143(struct comedi_device *dev)
@@ -1168,7 +1172,7 @@
return ret;
devpriv = dev->private;
- devpriv->mite = mite_alloc(pcidev);
+ devpriv->mite = mite_attach(dev, false); /* use win0 */
if (!devpriv->mite)
return -ENOMEM;
@@ -1193,10 +1197,6 @@
if (board->reg_type == ni_reg_6713)
devpriv->is_6713 = 1;
- ret = mite_setup(dev, devpriv->mite);
- if (ret < 0)
- return ret;
-
devpriv->ai_mite_ring = mite_alloc_ring(devpriv->mite);
if (!devpriv->ai_mite_ring)
return -ENOMEM;
diff --git a/drivers/staging/comedi/drivers/ni_stc.h b/drivers/staging/comedi/drivers/ni_stc.h
index 1d5af25..1966519 100644
--- a/drivers/staging/comedi/drivers/ni_stc.h
+++ b/drivers/staging/comedi/drivers/ni_stc.h
@@ -1,24 +1,23 @@
/*
- module/ni_stc.h
- Register descriptions for NI DAQ-STC chip
-
- COMEDI - Linux Control and Measurement Device Interface
- Copyright (C) 1998-9 David A. Schleef <ds@schleef.org>
-
- This program is free software; you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation; either version 2 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-*/
+ * Register descriptions for NI DAQ-STC chip
+ *
+ * COMEDI - Linux Control and Measurement Device Interface
+ * Copyright (C) 1998-9 David A. Schleef <ds@schleef.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
/*
- References:
- DAQ-STC Technical Reference Manual
+ * References:
+ * DAQ-STC Technical Reference Manual
*/
#ifndef _COMEDI_NI_STC_H
@@ -958,7 +957,7 @@
unsigned int ao_maxdata;
int ao_fifo_depth;
const struct comedi_lrange *ao_range_table;
- unsigned ao_speed;
+ unsigned int ao_speed;
int reg_type;
unsigned int has_8255:1;
@@ -1002,12 +1001,11 @@
unsigned short ao_mode3;
unsigned short ao_cmd1;
unsigned short ao_cmd2;
- unsigned short ao_trigger_select;
struct ni_gpct_device *counter_dev;
unsigned short an_trig_etc_reg;
- unsigned ai_offset[512];
+ unsigned int ai_offset[512];
unsigned long serial_interval_ns;
unsigned char serial_hw_mode;
@@ -1025,24 +1023,24 @@
unsigned short g0_g1_select_reg;
unsigned short cdio_dma_select_reg;
- unsigned clock_ns;
- unsigned clock_source;
+ unsigned int clock_ns;
+ unsigned int clock_source;
unsigned short pwm_up_count;
unsigned short pwm_down_count;
unsigned short ai_fifo_buffer[0x2000];
- uint8_t eeprom_buffer[M_SERIES_EEPROM_SIZE];
+ u8 eeprom_buffer[M_SERIES_EEPROM_SIZE];
__be32 serial_number;
- struct mite_struct *mite;
+ struct mite *mite;
struct mite_channel *ai_mite_chan;
struct mite_channel *ao_mite_chan;
struct mite_channel *cdo_mite_chan;
- struct mite_dma_descriptor_ring *ai_mite_ring;
- struct mite_dma_descriptor_ring *ao_mite_ring;
- struct mite_dma_descriptor_ring *cdo_mite_ring;
- struct mite_dma_descriptor_ring *gpct_mite_ring[NUM_GPCT];
+ struct mite_ring *ai_mite_ring;
+ struct mite_ring *ao_mite_ring;
+ struct mite_ring *cdo_mite_ring;
+ struct mite_ring *gpct_mite_ring[NUM_GPCT];
/* ni_pcimio board type flags (based on the boardinfo reg_type) */
unsigned int is_m_series:1;
diff --git a/drivers/staging/comedi/drivers/ni_tio.c b/drivers/staging/comedi/drivers/ni_tio.c
index b74e44e..7043eb0 100644
--- a/drivers/staging/comedi/drivers/ni_tio.c
+++ b/drivers/staging/comedi/drivers/ni_tio.c
@@ -1,19 +1,18 @@
/*
- comedi/drivers/ni_tio.c
- Support for NI general purpose counters
-
- Copyright (C) 2006 Frank Mori Hess <fmhess@users.sourceforge.net>
-
- This program is free software; you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation; either version 2 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-*/
+ * Support for NI general purpose counters
+ *
+ * Copyright (C) 2006 Frank Mori Hess <fmhess@users.sourceforge.net>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
/*
* Module: ni_tio
@@ -36,13 +35,10 @@
* DAQ 660x Register-Level Programmer Manual (NI 370505A-01)
* DAQ 6601/6602 User Manual (NI 322137B-01)
* 340934b.pdf DAQ-STC reference manual
+ *
+ * TODO: Support use of both banks X and Y
*/
-/*
-TODO:
- Support use of both banks X and Y
-*/
-
#include <linux/module.h>
#include <linux/slab.h>
@@ -115,20 +111,7 @@
#define NI_660X_LOGIC_LOW_GATE2_SEL 0x1f
#define NI_660X_MAX_UP_DOWN_PIN 7
-static inline unsigned GI_ALT_SYNC(enum ni_gpct_variant variant)
-{
- switch (variant) {
- case ni_gpct_variant_e_series:
- default:
- return 0;
- case ni_gpct_variant_m_series:
- return GI_M_ALT_SYNC;
- case ni_gpct_variant_660x:
- return GI_660X_ALT_SYNC;
- }
-}
-
-static inline unsigned GI_PRESCALE_X2(enum ni_gpct_variant variant)
+static inline unsigned int GI_PRESCALE_X2(enum ni_gpct_variant variant)
{
switch (variant) {
case ni_gpct_variant_e_series:
@@ -141,7 +124,7 @@
}
}
-static inline unsigned GI_PRESCALE_X8(enum ni_gpct_variant variant)
+static inline unsigned int GI_PRESCALE_X8(enum ni_gpct_variant variant)
{
switch (variant) {
case ni_gpct_variant_e_series:
@@ -154,19 +137,6 @@
}
}
-static inline unsigned GI_HW_ARM_SEL_MASK(enum ni_gpct_variant variant)
-{
- switch (variant) {
- case ni_gpct_variant_e_series:
- default:
- return 0;
- case ni_gpct_variant_m_series:
- return GI_M_HW_ARM_SEL_MASK;
- case ni_gpct_variant_660x:
- return GI_660X_HW_ARM_SEL_MASK;
- }
-}
-
static bool ni_tio_has_gate2_registers(const struct ni_gpct_device *counter_dev)
{
switch (counter_dev->variant) {
@@ -179,17 +149,45 @@
}
}
+/**
+ * ni_tio_write() - Write a TIO register using the driver provided callback.
+ * @counter: struct ni_gpct counter.
+ * @value: the value to write
+ * @reg: the register to write.
+ */
+void ni_tio_write(struct ni_gpct *counter, unsigned int value,
+ enum ni_gpct_register reg)
+{
+ if (reg < NITIO_NUM_REGS)
+ counter->counter_dev->write(counter, value, reg);
+}
+EXPORT_SYMBOL_GPL(ni_tio_write);
+
+/**
+ * ni_tio_read() - Read a TIO register using the driver provided callback.
+ * @counter: struct ni_gpct counter.
+ * @reg: the register to read.
+ */
+unsigned int ni_tio_read(struct ni_gpct *counter, enum ni_gpct_register reg)
+{
+ if (reg < NITIO_NUM_REGS)
+ return counter->counter_dev->read(counter, reg);
+ return 0;
+}
+EXPORT_SYMBOL_GPL(ni_tio_read);
+
static void ni_tio_reset_count_and_disarm(struct ni_gpct *counter)
{
- unsigned cidx = counter->counter_index;
+ unsigned int cidx = counter->counter_index;
- write_register(counter, GI_RESET(cidx), NITIO_RESET_REG(cidx));
+ ni_tio_write(counter, GI_RESET(cidx), NITIO_RESET_REG(cidx));
}
-static uint64_t ni_tio_clock_period_ps(const struct ni_gpct *counter,
- unsigned generic_clock_source)
+static int ni_tio_clock_period_ps(const struct ni_gpct *counter,
+ unsigned int generic_clock_source,
+ u64 *period_ps)
{
- uint64_t clock_period_ps;
+ u64 clock_period_ps;
switch (generic_clock_source & NI_GPCT_CLOCK_SRC_SELECT_MASK) {
case NI_GPCT_TIMEBASE_1_CLOCK_SRC_BITS:
@@ -222,19 +220,80 @@
clock_period_ps *= 8;
break;
default:
- BUG();
- break;
+ return -EINVAL;
}
- return clock_period_ps;
+ *period_ps = clock_period_ps;
+ return 0;
}
-static unsigned ni_tio_clock_src_modifiers(const struct ni_gpct *counter)
+static void ni_tio_set_bits_transient(struct ni_gpct *counter,
+ enum ni_gpct_register reg,
+ unsigned int mask, unsigned int value,
+ unsigned int transient)
{
struct ni_gpct_device *counter_dev = counter->counter_dev;
- unsigned cidx = counter->counter_index;
- const unsigned counting_mode_bits =
+ unsigned long flags;
+
+ if (reg < NITIO_NUM_REGS) {
+ spin_lock_irqsave(&counter_dev->regs_lock, flags);
+ counter_dev->regs[reg] &= ~mask;
+ counter_dev->regs[reg] |= (value & mask);
+ ni_tio_write(counter, counter_dev->regs[reg] | transient, reg);
+ mmiowb();
+ spin_unlock_irqrestore(&counter_dev->regs_lock, flags);
+ }
+}
+
+/**
+ * ni_tio_set_bits() - Safely write a counter register.
+ * @counter: struct ni_gpct counter.
+ * @reg: the register to write.
+ * @mask: the bits to change.
+ * @value: the new bits value.
+ *
+ * Used to write to, and update the software copy, a register whose bits may
+ * be twiddled in interrupt context, or whose software copy may be read in
+ * interrupt context.
+ */
+void ni_tio_set_bits(struct ni_gpct *counter, enum ni_gpct_register reg,
+ unsigned int mask, unsigned int value)
+{
+ ni_tio_set_bits_transient(counter, reg, mask, value, 0x0);
+}
+EXPORT_SYMBOL_GPL(ni_tio_set_bits);
+
+/**
+ * ni_tio_get_soft_copy() - Safely read the software copy of a counter register.
+ * @counter: struct ni_gpct counter.
+ * @reg: the register to read.
+ *
+ * Used to get the software copy of a register whose bits might be modified
+ * in interrupt context, or whose software copy might need to be read in
+ * interrupt context.
+ */
+unsigned int ni_tio_get_soft_copy(const struct ni_gpct *counter,
+ enum ni_gpct_register reg)
+{
+ struct ni_gpct_device *counter_dev = counter->counter_dev;
+ unsigned int value = 0;
+ unsigned long flags;
+
+ if (reg < NITIO_NUM_REGS) {
+ spin_lock_irqsave(&counter_dev->regs_lock, flags);
+ value = counter_dev->regs[reg];
+ spin_unlock_irqrestore(&counter_dev->regs_lock, flags);
+ }
+ return value;
+}
+EXPORT_SYMBOL_GPL(ni_tio_get_soft_copy);
+
+static unsigned int ni_tio_clock_src_modifiers(const struct ni_gpct *counter)
+{
+ struct ni_gpct_device *counter_dev = counter->counter_dev;
+ unsigned int cidx = counter->counter_index;
+ unsigned int counting_mode_bits =
ni_tio_get_soft_copy(counter, NITIO_CNT_MODE_REG(cidx));
- unsigned bits = 0;
+ unsigned int bits = 0;
if (ni_tio_get_soft_copy(counter, NITIO_INPUT_SEL_REG(cidx)) &
GI_SRC_POL_INVERT)
@@ -246,14 +305,15 @@
return bits;
}
-static unsigned ni_m_series_clock_src_select(const struct ni_gpct *counter)
+static int ni_m_series_clock_src_select(const struct ni_gpct *counter,
+ unsigned int *clk_src)
{
struct ni_gpct_device *counter_dev = counter->counter_dev;
- unsigned cidx = counter->counter_index;
- const unsigned second_gate_reg = NITIO_GATE2_REG(cidx);
- unsigned clock_source = 0;
- unsigned src;
- unsigned i;
+ unsigned int cidx = counter->counter_index;
+ unsigned int second_gate_reg = NITIO_GATE2_REG(cidx);
+ unsigned int clock_source = 0;
+ unsigned int src;
+ unsigned int i;
src = GI_BITS_TO_SRC(ni_tio_get_soft_copy(counter,
NITIO_INPUT_SEL_REG(cidx)));
@@ -304,19 +364,20 @@
}
if (i <= NI_M_MAX_PFI_CHAN)
break;
- BUG();
- break;
+ return -EINVAL;
}
clock_source |= ni_tio_clock_src_modifiers(counter);
- return clock_source;
+ *clk_src = clock_source;
+ return 0;
}
-static unsigned ni_660x_clock_src_select(const struct ni_gpct *counter)
+static int ni_660x_clock_src_select(const struct ni_gpct *counter,
+ unsigned int *clk_src)
{
- unsigned clock_source = 0;
- unsigned cidx = counter->counter_index;
- unsigned src;
- unsigned i;
+ unsigned int clock_source = 0;
+ unsigned int cidx = counter->counter_index;
+ unsigned int src;
+ unsigned int i;
src = GI_BITS_TO_SRC(ni_tio_get_soft_copy(counter,
NITIO_INPUT_SEL_REG(cidx)));
@@ -361,78 +422,88 @@
}
if (i <= NI_660X_MAX_SRC_PIN)
break;
- BUG();
- break;
+ return -EINVAL;
}
clock_source |= ni_tio_clock_src_modifiers(counter);
- return clock_source;
+ *clk_src = clock_source;
+ return 0;
}
-static unsigned ni_tio_generic_clock_src_select(const struct ni_gpct *counter)
+static int ni_tio_generic_clock_src_select(const struct ni_gpct *counter,
+ unsigned int *clk_src)
{
switch (counter->counter_dev->variant) {
case ni_gpct_variant_e_series:
case ni_gpct_variant_m_series:
default:
- return ni_m_series_clock_src_select(counter);
+ return ni_m_series_clock_src_select(counter, clk_src);
case ni_gpct_variant_660x:
- return ni_660x_clock_src_select(counter);
+ return ni_660x_clock_src_select(counter, clk_src);
}
}
-static void ni_tio_set_sync_mode(struct ni_gpct *counter, int force_alt_sync)
+static void ni_tio_set_sync_mode(struct ni_gpct *counter)
{
struct ni_gpct_device *counter_dev = counter->counter_dev;
- unsigned cidx = counter->counter_index;
- const unsigned counting_mode_reg = NITIO_CNT_MODE_REG(cidx);
- static const uint64_t min_normal_sync_period_ps = 25000;
- unsigned mode;
- uint64_t clock_period_ps;
+ unsigned int cidx = counter->counter_index;
+ static const u64 min_normal_sync_period_ps = 25000;
+ unsigned int mask = 0;
+ unsigned int bits = 0;
+ unsigned int reg;
+ unsigned int mode;
+ unsigned int clk_src;
+ u64 ps;
+ bool force_alt_sync;
- if (ni_tio_counting_mode_registers_present(counter_dev) == 0)
+ /* only m series and 660x variants have counting mode registers */
+ switch (counter_dev->variant) {
+ case ni_gpct_variant_e_series:
+ default:
return;
+ case ni_gpct_variant_m_series:
+ mask = GI_M_ALT_SYNC;
+ break;
+ case ni_gpct_variant_660x:
+ mask = GI_660X_ALT_SYNC;
+ break;
+ }
- mode = ni_tio_get_soft_copy(counter, counting_mode_reg);
+ reg = NITIO_CNT_MODE_REG(cidx);
+ mode = ni_tio_get_soft_copy(counter, reg);
switch (mode & GI_CNT_MODE_MASK) {
case GI_CNT_MODE_QUADX1:
case GI_CNT_MODE_QUADX2:
case GI_CNT_MODE_QUADX4:
case GI_CNT_MODE_SYNC_SRC:
- force_alt_sync = 1;
+ force_alt_sync = true;
break;
default:
+ force_alt_sync = false;
break;
}
- clock_period_ps = ni_tio_clock_period_ps(counter,
- ni_tio_generic_clock_src_select(counter));
+ ni_tio_generic_clock_src_select(counter, &clk_src);
+ ni_tio_clock_period_ps(counter, clk_src, &ps);
/*
* It's not clear what we should do if clock_period is unknown, so we
- * are not using the alt sync bit in that case, but allow the caller
- * to decide by using the force_alt_sync parameter.
+ * are not using the alt sync bit in that case.
*/
- if (force_alt_sync ||
- (clock_period_ps && clock_period_ps < min_normal_sync_period_ps)) {
- ni_tio_set_bits(counter, counting_mode_reg,
- GI_ALT_SYNC(counter_dev->variant),
- GI_ALT_SYNC(counter_dev->variant));
- } else {
- ni_tio_set_bits(counter, counting_mode_reg,
- GI_ALT_SYNC(counter_dev->variant),
- 0x0);
- }
+ if (force_alt_sync || (ps && ps < min_normal_sync_period_ps))
+ bits = mask;
+
+ ni_tio_set_bits(counter, reg, mask, bits);
}
-static int ni_tio_set_counter_mode(struct ni_gpct *counter, unsigned mode)
+static int ni_tio_set_counter_mode(struct ni_gpct *counter, unsigned int mode)
{
struct ni_gpct_device *counter_dev = counter->counter_dev;
- unsigned cidx = counter->counter_index;
- unsigned mode_reg_mask;
- unsigned mode_reg_values;
- unsigned input_select_bits = 0;
+ unsigned int cidx = counter->counter_index;
+ unsigned int mode_reg_mask;
+ unsigned int mode_reg_values;
+ unsigned int input_select_bits = 0;
/* these bits map directly on to the mode register */
- static const unsigned mode_reg_direct_mask =
+ static const unsigned int mode_reg_direct_mask =
NI_GPCT_GATE_ON_BOTH_EDGES_BIT | NI_GPCT_EDGE_GATE_MODE_MASK |
NI_GPCT_STOP_MODE_MASK | NI_GPCT_OUTPUT_MODE_MASK |
NI_GPCT_HARDWARE_DISARM_MASK | NI_GPCT_LOADING_ON_TC_BIT |
@@ -458,7 +529,7 @@
mode_reg_mask, mode_reg_values);
if (ni_tio_counting_mode_registers_present(counter_dev)) {
- unsigned bits = 0;
+ unsigned int bits = 0;
bits |= GI_CNT_MODE(mode >> NI_GPCT_COUNTING_MODE_SHIFT);
bits |= GI_INDEX_PHASE((mode >> NI_GPCT_INDEX_PHASE_BITSHIFT));
@@ -467,7 +538,7 @@
ni_tio_set_bits(counter, NITIO_CNT_MODE_REG(cidx),
GI_CNT_MODE_MASK | GI_INDEX_PHASE_MASK |
GI_INDEX_MODE, bits);
- ni_tio_set_sync_mode(counter, 0);
+ ni_tio_set_sync_mode(counter);
}
ni_tio_set_bits(counter, NITIO_CMD_REG(cidx), GI_CNT_DIR_MASK,
@@ -484,65 +555,68 @@
return 0;
}
-int ni_tio_arm(struct ni_gpct *counter, int arm, unsigned start_trigger)
+int ni_tio_arm(struct ni_gpct *counter, bool arm, unsigned int start_trigger)
{
struct ni_gpct_device *counter_dev = counter->counter_dev;
- unsigned cidx = counter->counter_index;
- unsigned command_transient_bits = 0;
+ unsigned int cidx = counter->counter_index;
+ unsigned int transient_bits = 0;
if (arm) {
- switch (start_trigger) {
- case NI_GPCT_ARM_IMMEDIATE:
- command_transient_bits |= GI_ARM;
- break;
- case NI_GPCT_ARM_PAIRED_IMMEDIATE:
- command_transient_bits |= GI_ARM | GI_ARM_COPY;
- break;
+ unsigned int mask = 0;
+ unsigned int bits = 0;
+
+ /* only m series and 660x have counting mode registers */
+ switch (counter_dev->variant) {
+ case ni_gpct_variant_e_series:
default:
break;
+ case ni_gpct_variant_m_series:
+ mask = GI_M_HW_ARM_SEL_MASK;
+ break;
+ case ni_gpct_variant_660x:
+ mask = GI_660X_HW_ARM_SEL_MASK;
+ break;
}
- if (ni_tio_counting_mode_registers_present(counter_dev)) {
- unsigned bits = 0;
- unsigned sel_mask;
- sel_mask = GI_HW_ARM_SEL_MASK(counter_dev->variant);
-
- switch (start_trigger) {
- case NI_GPCT_ARM_IMMEDIATE:
- case NI_GPCT_ARM_PAIRED_IMMEDIATE:
- break;
- default:
- if (start_trigger & NI_GPCT_ARM_UNKNOWN) {
- /*
- * pass-through the least significant
- * bits so we can figure out what
- * select later
- */
- bits |= GI_HW_ARM_ENA |
- (GI_HW_ARM_SEL(start_trigger) &
- sel_mask);
- } else {
- return -EINVAL;
- }
- break;
+ switch (start_trigger) {
+ case NI_GPCT_ARM_IMMEDIATE:
+ transient_bits |= GI_ARM;
+ break;
+ case NI_GPCT_ARM_PAIRED_IMMEDIATE:
+ transient_bits |= GI_ARM | GI_ARM_COPY;
+ break;
+ default:
+ /*
+ * for m series and 660x, pass-through the least
+ * significant bits so we can figure out what select
+ * later
+ */
+ if (mask && (start_trigger & NI_GPCT_ARM_UNKNOWN)) {
+ bits |= GI_HW_ARM_ENA |
+ (GI_HW_ARM_SEL(start_trigger) & mask);
+ } else {
+ return -EINVAL;
}
- ni_tio_set_bits(counter, NITIO_CNT_MODE_REG(cidx),
- GI_HW_ARM_ENA | sel_mask, bits);
+ break;
}
+
+ if (mask)
+ ni_tio_set_bits(counter, NITIO_CNT_MODE_REG(cidx),
+ GI_HW_ARM_ENA | mask, bits);
} else {
- command_transient_bits |= GI_DISARM;
+ transient_bits |= GI_DISARM;
}
ni_tio_set_bits_transient(counter, NITIO_CMD_REG(cidx),
- 0, 0, command_transient_bits);
+ 0, 0, transient_bits);
return 0;
}
EXPORT_SYMBOL_GPL(ni_tio_arm);
-static unsigned ni_660x_clk_src(unsigned int clock_source)
+static int ni_660x_clk_src(unsigned int clock_source, unsigned int *bits)
{
- unsigned clk_src = clock_source & NI_GPCT_CLOCK_SRC_SELECT_MASK;
- unsigned ni_660x_clock;
- unsigned i;
+ unsigned int clk_src = clock_source & NI_GPCT_CLOCK_SRC_SELECT_MASK;
+ unsigned int ni_660x_clock;
+ unsigned int i;
switch (clk_src) {
case NI_GPCT_TIMEBASE_1_CLOCK_SRC_BITS:
@@ -583,18 +657,17 @@
}
if (i <= NI_660X_MAX_SRC_PIN)
break;
- ni_660x_clock = 0;
- BUG();
- break;
+ return -EINVAL;
}
- return GI_SRC_SEL(ni_660x_clock);
+ *bits = GI_SRC_SEL(ni_660x_clock);
+ return 0;
}
-static unsigned ni_m_clk_src(unsigned int clock_source)
+static int ni_m_clk_src(unsigned int clock_source, unsigned int *bits)
{
- unsigned clk_src = clock_source & NI_GPCT_CLOCK_SRC_SELECT_MASK;
- unsigned ni_m_series_clock;
- unsigned i;
+ unsigned int clk_src = clock_source & NI_GPCT_CLOCK_SRC_SELECT_MASK;
+ unsigned int ni_m_series_clock;
+ unsigned int i;
switch (clk_src) {
case NI_GPCT_TIMEBASE_1_CLOCK_SRC_BITS:
@@ -641,21 +714,18 @@
}
if (i <= NI_M_MAX_PFI_CHAN)
break;
- pr_err("invalid clock source 0x%lx\n",
- (unsigned long)clock_source);
- BUG();
- ni_m_series_clock = 0;
- break;
+ return -EINVAL;
}
- return GI_SRC_SEL(ni_m_series_clock);
+ *bits = GI_SRC_SEL(ni_m_series_clock);
+ return 0;
};
static void ni_tio_set_source_subselect(struct ni_gpct *counter,
unsigned int clock_source)
{
struct ni_gpct_device *counter_dev = counter->counter_dev;
- unsigned cidx = counter->counter_index;
- const unsigned second_gate_reg = NITIO_GATE2_REG(cidx);
+ unsigned int cidx = counter->counter_index;
+ unsigned int second_gate_reg = NITIO_GATE2_REG(cidx);
if (counter_dev->variant != ni_gpct_variant_m_series)
return;
@@ -674,8 +744,8 @@
default:
return;
}
- write_register(counter, counter_dev->regs[second_gate_reg],
- second_gate_reg);
+ ni_tio_write(counter, counter_dev->regs[second_gate_reg],
+ second_gate_reg);
}
static int ni_tio_set_clock_src(struct ni_gpct *counter,
@@ -683,20 +753,28 @@
unsigned int period_ns)
{
struct ni_gpct_device *counter_dev = counter->counter_dev;
- unsigned cidx = counter->counter_index;
- unsigned bits = 0;
+ unsigned int cidx = counter->counter_index;
+ unsigned int bits = 0;
+ int ret;
- /* FIXME: validate clock source */
switch (counter_dev->variant) {
case ni_gpct_variant_660x:
- bits |= ni_660x_clk_src(clock_source);
+ ret = ni_660x_clk_src(clock_source, &bits);
break;
case ni_gpct_variant_e_series:
case ni_gpct_variant_m_series:
default:
- bits |= ni_m_clk_src(clock_source);
+ ret = ni_m_clk_src(clock_source, &bits);
break;
}
+ if (ret) {
+ struct comedi_device *dev = counter_dev->dev;
+
+ dev_err(dev->class_dev, "invalid clock source 0x%x\n",
+ clock_source);
+ return ret;
+ }
+
if (clock_source & NI_GPCT_INVERT_CLOCK_SRC_BIT)
bits |= GI_SRC_POL_INVERT;
ni_tio_set_bits(counter, NITIO_INPUT_SEL_REG(cidx),
@@ -722,28 +800,34 @@
GI_PRESCALE_X8(counter_dev->variant), bits);
}
counter->clock_period_ps = period_ns * 1000;
- ni_tio_set_sync_mode(counter, 0);
+ ni_tio_set_sync_mode(counter);
return 0;
}
-static void ni_tio_get_clock_src(struct ni_gpct *counter,
- unsigned int *clock_source,
- unsigned int *period_ns)
+static int ni_tio_get_clock_src(struct ni_gpct *counter,
+ unsigned int *clock_source,
+ unsigned int *period_ns)
{
- uint64_t temp64;
+ u64 temp64;
+ int ret;
- *clock_source = ni_tio_generic_clock_src_select(counter);
- temp64 = ni_tio_clock_period_ps(counter, *clock_source);
+ ret = ni_tio_generic_clock_src_select(counter, clock_source);
+ if (ret)
+ return ret;
+ ret = ni_tio_clock_period_ps(counter, *clock_source, &temp64);
+ if (ret)
+ return ret;
do_div(temp64, 1000); /* ps to ns */
*period_ns = temp64;
+ return 0;
}
static int ni_660x_set_gate(struct ni_gpct *counter, unsigned int gate_source)
{
unsigned int chan = CR_CHAN(gate_source);
- unsigned cidx = counter->counter_index;
- unsigned gate_sel;
- unsigned i;
+ unsigned int cidx = counter->counter_index;
+ unsigned int gate_sel;
+ unsigned int i;
switch (chan) {
case NI_GPCT_NEXT_SOURCE_GATE_SELECT:
@@ -782,9 +866,9 @@
static int ni_m_set_gate(struct ni_gpct *counter, unsigned int gate_source)
{
unsigned int chan = CR_CHAN(gate_source);
- unsigned cidx = counter->counter_index;
- unsigned gate_sel;
- unsigned i;
+ unsigned int cidx = counter->counter_index;
+ unsigned int gate_sel;
+ unsigned int i;
switch (chan) {
case NI_GPCT_TIMESTAMP_MUX_GATE_SELECT:
@@ -824,11 +908,11 @@
static int ni_660x_set_gate2(struct ni_gpct *counter, unsigned int gate_source)
{
struct ni_gpct_device *counter_dev = counter->counter_dev;
- unsigned cidx = counter->counter_index;
+ unsigned int cidx = counter->counter_index;
unsigned int chan = CR_CHAN(gate_source);
- unsigned gate2_reg = NITIO_GATE2_REG(cidx);
- unsigned gate2_sel;
- unsigned i;
+ unsigned int gate2_reg = NITIO_GATE2_REG(cidx);
+ unsigned int gate2_sel;
+ unsigned int i;
switch (chan) {
case NI_GPCT_SOURCE_PIN_i_GATE_SELECT:
@@ -863,17 +947,17 @@
counter_dev->regs[gate2_reg] |= GI_GATE2_MODE;
counter_dev->regs[gate2_reg] &= ~GI_GATE2_SEL_MASK;
counter_dev->regs[gate2_reg] |= GI_GATE2_SEL(gate2_sel);
- write_register(counter, counter_dev->regs[gate2_reg], gate2_reg);
+ ni_tio_write(counter, counter_dev->regs[gate2_reg], gate2_reg);
return 0;
}
static int ni_m_set_gate2(struct ni_gpct *counter, unsigned int gate_source)
{
struct ni_gpct_device *counter_dev = counter->counter_dev;
- unsigned cidx = counter->counter_index;
+ unsigned int cidx = counter->counter_index;
unsigned int chan = CR_CHAN(gate_source);
- unsigned gate2_reg = NITIO_GATE2_REG(cidx);
- unsigned gate2_sel;
+ unsigned int gate2_reg = NITIO_GATE2_REG(cidx);
+ unsigned int gate2_sel;
/*
* FIXME: We don't know what the m-series second gate codes are,
@@ -887,20 +971,20 @@
counter_dev->regs[gate2_reg] |= GI_GATE2_MODE;
counter_dev->regs[gate2_reg] &= ~GI_GATE2_SEL_MASK;
counter_dev->regs[gate2_reg] |= GI_GATE2_SEL(gate2_sel);
- write_register(counter, counter_dev->regs[gate2_reg], gate2_reg);
+ ni_tio_write(counter, counter_dev->regs[gate2_reg], gate2_reg);
return 0;
}
-int ni_tio_set_gate_src(struct ni_gpct *counter, unsigned gate_index,
- unsigned int gate_source)
+int ni_tio_set_gate_src(struct ni_gpct *counter,
+ unsigned int gate, unsigned int src)
{
struct ni_gpct_device *counter_dev = counter->counter_dev;
- unsigned cidx = counter->counter_index;
- unsigned int chan = CR_CHAN(gate_source);
- unsigned gate2_reg = NITIO_GATE2_REG(cidx);
- unsigned mode = 0;
+ unsigned int cidx = counter->counter_index;
+ unsigned int chan = CR_CHAN(src);
+ unsigned int gate2_reg = NITIO_GATE2_REG(cidx);
+ unsigned int mode = 0;
- switch (gate_index) {
+ switch (gate) {
case 0:
if (chan == NI_GPCT_DISABLED_GATE_SELECT) {
ni_tio_set_bits(counter, NITIO_MODE_REG(cidx),
@@ -908,9 +992,9 @@
GI_GATING_DISABLED);
return 0;
}
- if (gate_source & CR_INVERT)
+ if (src & CR_INVERT)
mode |= GI_GATE_POL_INVERT;
- if (gate_source & CR_EDGE)
+ if (src & CR_EDGE)
mode |= GI_RISING_EDGE_GATING;
else
mode |= GI_LEVEL_GATING;
@@ -921,9 +1005,9 @@
case ni_gpct_variant_e_series:
case ni_gpct_variant_m_series:
default:
- return ni_m_set_gate(counter, gate_source);
+ return ni_m_set_gate(counter, src);
case ni_gpct_variant_660x:
- return ni_660x_set_gate(counter, gate_source);
+ return ni_660x_set_gate(counter, src);
}
break;
case 1:
@@ -932,22 +1016,21 @@
if (chan == NI_GPCT_DISABLED_GATE_SELECT) {
counter_dev->regs[gate2_reg] &= ~GI_GATE2_MODE;
- write_register(counter, counter_dev->regs[gate2_reg],
- gate2_reg);
+ ni_tio_write(counter, counter_dev->regs[gate2_reg],
+ gate2_reg);
return 0;
}
- if (gate_source & CR_INVERT)
+ if (src & CR_INVERT)
counter_dev->regs[gate2_reg] |= GI_GATE2_POL_INVERT;
else
counter_dev->regs[gate2_reg] &= ~GI_GATE2_POL_INVERT;
switch (counter_dev->variant) {
case ni_gpct_variant_m_series:
- return ni_m_set_gate2(counter, gate_source);
+ return ni_m_set_gate2(counter, src);
case ni_gpct_variant_660x:
- return ni_660x_set_gate2(counter, gate_source);
+ return ni_660x_set_gate2(counter, src);
default:
- BUG();
- break;
+ return -EINVAL;
}
break;
default:
@@ -957,11 +1040,11 @@
}
EXPORT_SYMBOL_GPL(ni_tio_set_gate_src);
-static int ni_tio_set_other_src(struct ni_gpct *counter, unsigned index,
+static int ni_tio_set_other_src(struct ni_gpct *counter, unsigned int index,
unsigned int source)
{
struct ni_gpct_device *counter_dev = counter->counter_dev;
- unsigned cidx = counter->counter_index;
+ unsigned int cidx = counter->counter_index;
unsigned int abz_reg, shift, mask;
if (counter_dev->variant != ni_gpct_variant_m_series)
@@ -987,175 +1070,221 @@
counter_dev->regs[abz_reg] &= ~mask;
counter_dev->regs[abz_reg] |= (source << shift) & mask;
- write_register(counter, counter_dev->regs[abz_reg], abz_reg);
+ ni_tio_write(counter, counter_dev->regs[abz_reg], abz_reg);
return 0;
}
-static unsigned ni_660x_gate_to_generic_gate(unsigned gate)
+static int ni_660x_gate_to_generic_gate(unsigned int gate, unsigned int *src)
{
- unsigned i;
+ unsigned int source;
+ unsigned int i;
switch (gate) {
case NI_660X_SRC_PIN_I_GATE_SEL:
- return NI_GPCT_SOURCE_PIN_i_GATE_SELECT;
+ source = NI_GPCT_SOURCE_PIN_i_GATE_SELECT;
+ break;
case NI_660X_GATE_PIN_I_GATE_SEL:
- return NI_GPCT_GATE_PIN_i_GATE_SELECT;
+ source = NI_GPCT_GATE_PIN_i_GATE_SELECT;
+ break;
case NI_660X_NEXT_SRC_GATE_SEL:
- return NI_GPCT_NEXT_SOURCE_GATE_SELECT;
+ source = NI_GPCT_NEXT_SOURCE_GATE_SELECT;
+ break;
case NI_660X_NEXT_OUT_GATE_SEL:
- return NI_GPCT_NEXT_OUT_GATE_SELECT;
+ source = NI_GPCT_NEXT_OUT_GATE_SELECT;
+ break;
case NI_660X_LOGIC_LOW_GATE_SEL:
- return NI_GPCT_LOGIC_LOW_GATE_SELECT;
+ source = NI_GPCT_LOGIC_LOW_GATE_SELECT;
+ break;
default:
for (i = 0; i <= NI_660X_MAX_RTSI_CHAN; ++i) {
- if (gate == NI_660X_RTSI_GATE_SEL(i))
- return NI_GPCT_RTSI_GATE_SELECT(i);
+ if (gate == NI_660X_RTSI_GATE_SEL(i)) {
+ source = NI_GPCT_RTSI_GATE_SELECT(i);
+ break;
+ }
}
+ if (i <= NI_660X_MAX_RTSI_CHAN)
+ break;
for (i = 0; i <= NI_660X_MAX_GATE_PIN; ++i) {
- if (gate == NI_660X_PIN_GATE_SEL(i))
- return NI_GPCT_GATE_PIN_GATE_SELECT(i);
+ if (gate == NI_660X_PIN_GATE_SEL(i)) {
+ source = NI_GPCT_GATE_PIN_GATE_SELECT(i);
+ break;
+ }
}
- BUG();
- break;
+ if (i <= NI_660X_MAX_GATE_PIN)
+ break;
+ return -EINVAL;
}
+ *src = source;
return 0;
};
-static unsigned ni_m_gate_to_generic_gate(unsigned gate)
+static int ni_m_gate_to_generic_gate(unsigned int gate, unsigned int *src)
{
- unsigned i;
+ unsigned int source;
+ unsigned int i;
switch (gate) {
case NI_M_TIMESTAMP_MUX_GATE_SEL:
- return NI_GPCT_TIMESTAMP_MUX_GATE_SELECT;
+ source = NI_GPCT_TIMESTAMP_MUX_GATE_SELECT;
+ break;
case NI_M_AI_START2_GATE_SEL:
- return NI_GPCT_AI_START2_GATE_SELECT;
+ source = NI_GPCT_AI_START2_GATE_SELECT;
+ break;
case NI_M_PXI_STAR_TRIGGER_GATE_SEL:
- return NI_GPCT_PXI_STAR_TRIGGER_GATE_SELECT;
+ source = NI_GPCT_PXI_STAR_TRIGGER_GATE_SELECT;
+ break;
case NI_M_NEXT_OUT_GATE_SEL:
- return NI_GPCT_NEXT_OUT_GATE_SELECT;
+ source = NI_GPCT_NEXT_OUT_GATE_SELECT;
+ break;
case NI_M_AI_START1_GATE_SEL:
- return NI_GPCT_AI_START1_GATE_SELECT;
+ source = NI_GPCT_AI_START1_GATE_SELECT;
+ break;
case NI_M_NEXT_SRC_GATE_SEL:
- return NI_GPCT_NEXT_SOURCE_GATE_SELECT;
+ source = NI_GPCT_NEXT_SOURCE_GATE_SELECT;
+ break;
case NI_M_ANALOG_TRIG_OUT_GATE_SEL:
- return NI_GPCT_ANALOG_TRIGGER_OUT_GATE_SELECT;
+ source = NI_GPCT_ANALOG_TRIGGER_OUT_GATE_SELECT;
+ break;
case NI_M_LOGIC_LOW_GATE_SEL:
- return NI_GPCT_LOGIC_LOW_GATE_SELECT;
+ source = NI_GPCT_LOGIC_LOW_GATE_SELECT;
+ break;
default:
for (i = 0; i <= NI_M_MAX_RTSI_CHAN; ++i) {
- if (gate == NI_M_RTSI_GATE_SEL(i))
- return NI_GPCT_RTSI_GATE_SELECT(i);
+ if (gate == NI_M_RTSI_GATE_SEL(i)) {
+ source = NI_GPCT_RTSI_GATE_SELECT(i);
+ break;
+ }
}
+ if (i <= NI_M_MAX_RTSI_CHAN)
+ break;
for (i = 0; i <= NI_M_MAX_PFI_CHAN; ++i) {
- if (gate == NI_M_PFI_GATE_SEL(i))
- return NI_GPCT_PFI_GATE_SELECT(i);
+ if (gate == NI_M_PFI_GATE_SEL(i)) {
+ source = NI_GPCT_PFI_GATE_SELECT(i);
+ break;
+ }
}
- BUG();
- break;
+ if (i <= NI_M_MAX_PFI_CHAN)
+ break;
+ return -EINVAL;
}
+ *src = source;
return 0;
};
-static unsigned ni_660x_gate2_to_generic_gate(unsigned gate)
+static int ni_660x_gate2_to_generic_gate(unsigned int gate, unsigned int *src)
{
- unsigned i;
+ unsigned int source;
+ unsigned int i;
switch (gate) {
case NI_660X_SRC_PIN_I_GATE2_SEL:
- return NI_GPCT_SOURCE_PIN_i_GATE_SELECT;
+ source = NI_GPCT_SOURCE_PIN_i_GATE_SELECT;
+ break;
case NI_660X_UD_PIN_I_GATE2_SEL:
- return NI_GPCT_UP_DOWN_PIN_i_GATE_SELECT;
+ source = NI_GPCT_UP_DOWN_PIN_i_GATE_SELECT;
+ break;
case NI_660X_NEXT_SRC_GATE2_SEL:
- return NI_GPCT_NEXT_SOURCE_GATE_SELECT;
+ source = NI_GPCT_NEXT_SOURCE_GATE_SELECT;
+ break;
case NI_660X_NEXT_OUT_GATE2_SEL:
- return NI_GPCT_NEXT_OUT_GATE_SELECT;
+ source = NI_GPCT_NEXT_OUT_GATE_SELECT;
+ break;
case NI_660X_SELECTED_GATE2_SEL:
- return NI_GPCT_SELECTED_GATE_GATE_SELECT;
+ source = NI_GPCT_SELECTED_GATE_GATE_SELECT;
+ break;
case NI_660X_LOGIC_LOW_GATE2_SEL:
- return NI_GPCT_LOGIC_LOW_GATE_SELECT;
+ source = NI_GPCT_LOGIC_LOW_GATE_SELECT;
+ break;
default:
for (i = 0; i <= NI_660X_MAX_RTSI_CHAN; ++i) {
- if (gate == NI_660X_RTSI_GATE2_SEL(i))
- return NI_GPCT_RTSI_GATE_SELECT(i);
+ if (gate == NI_660X_RTSI_GATE2_SEL(i)) {
+ source = NI_GPCT_RTSI_GATE_SELECT(i);
+ break;
+ }
}
+ if (i <= NI_660X_MAX_RTSI_CHAN)
+ break;
for (i = 0; i <= NI_660X_MAX_UP_DOWN_PIN; ++i) {
- if (gate == NI_660X_UD_PIN_GATE2_SEL(i))
- return NI_GPCT_UP_DOWN_PIN_GATE_SELECT(i);
+ if (gate == NI_660X_UD_PIN_GATE2_SEL(i)) {
+ source = NI_GPCT_UP_DOWN_PIN_GATE_SELECT(i);
+ break;
+ }
}
- BUG();
- break;
+ if (i <= NI_660X_MAX_UP_DOWN_PIN)
+ break;
+ return -EINVAL;
}
+ *src = source;
return 0;
};
-static unsigned ni_m_gate2_to_generic_gate(unsigned gate)
+static int ni_m_gate2_to_generic_gate(unsigned int gate, unsigned int *src)
{
/*
* FIXME: the second gate sources for the m series are undocumented,
* so we just return the raw bits for now.
*/
- switch (gate) {
- default:
- return gate;
- }
+ *src = gate;
return 0;
};
-static int ni_tio_get_gate_src(struct ni_gpct *counter, unsigned gate_index,
+static int ni_tio_get_gate_src(struct ni_gpct *counter, unsigned int gate_index,
unsigned int *gate_source)
{
struct ni_gpct_device *counter_dev = counter->counter_dev;
- unsigned cidx = counter->counter_index;
- unsigned mode = ni_tio_get_soft_copy(counter, NITIO_MODE_REG(cidx));
- unsigned gate2_reg = NITIO_GATE2_REG(cidx);
- unsigned gate;
+ unsigned int cidx = counter->counter_index;
+ unsigned int mode;
+ unsigned int reg;
+ unsigned int gate;
+ int ret;
+
+ mode = ni_tio_get_soft_copy(counter, NITIO_MODE_REG(cidx));
+ if (((mode & GI_GATING_MODE_MASK) == GI_GATING_DISABLED) ||
+ (gate_index == 1 &&
+ !(counter_dev->regs[NITIO_GATE2_REG(cidx)] & GI_GATE2_MODE))) {
+ *gate_source = NI_GPCT_DISABLED_GATE_SELECT;
+ return 0;
+ }
switch (gate_index) {
case 0:
- if ((mode & GI_GATING_MODE_MASK) == GI_GATING_DISABLED) {
- *gate_source = NI_GPCT_DISABLED_GATE_SELECT;
- return 0;
- }
-
- gate = GI_BITS_TO_GATE(ni_tio_get_soft_copy(counter,
- NITIO_INPUT_SEL_REG(cidx)));
+ reg = NITIO_INPUT_SEL_REG(cidx);
+ gate = GI_BITS_TO_GATE(ni_tio_get_soft_copy(counter, reg));
switch (counter_dev->variant) {
case ni_gpct_variant_e_series:
case ni_gpct_variant_m_series:
default:
- *gate_source = ni_m_gate_to_generic_gate(gate);
+ ret = ni_m_gate_to_generic_gate(gate, gate_source);
break;
case ni_gpct_variant_660x:
- *gate_source = ni_660x_gate_to_generic_gate(gate);
+ ret = ni_660x_gate_to_generic_gate(gate, gate_source);
break;
}
+ if (ret)
+ return ret;
if (mode & GI_GATE_POL_INVERT)
*gate_source |= CR_INVERT;
if ((mode & GI_GATING_MODE_MASK) != GI_LEVEL_GATING)
*gate_source |= CR_EDGE;
break;
case 1:
- if ((mode & GI_GATING_MODE_MASK) == GI_GATING_DISABLED ||
- !(counter_dev->regs[gate2_reg] & GI_GATE2_MODE)) {
- *gate_source = NI_GPCT_DISABLED_GATE_SELECT;
- return 0;
- }
-
- gate = GI_BITS_TO_GATE2(counter_dev->regs[gate2_reg]);
+ reg = NITIO_GATE2_REG(cidx);
+ gate = GI_BITS_TO_GATE2(counter_dev->regs[reg]);
switch (counter_dev->variant) {
case ni_gpct_variant_e_series:
case ni_gpct_variant_m_series:
default:
- *gate_source = ni_m_gate2_to_generic_gate(gate);
+ ret = ni_m_gate2_to_generic_gate(gate, gate_source);
break;
case ni_gpct_variant_660x:
- *gate_source = ni_660x_gate2_to_generic_gate(gate);
+ ret = ni_660x_gate2_to_generic_gate(gate, gate_source);
break;
}
- if (counter_dev->regs[gate2_reg] & GI_GATE2_POL_INVERT)
+ if (ret)
+ return ret;
+ if (counter_dev->regs[reg] & GI_GATE2_POL_INVERT)
*gate_source |= CR_INVERT;
/* second gate can't have edge/level mode set independently */
if ((mode & GI_GATING_MODE_MASK) != GI_LEVEL_GATING)
@@ -1173,45 +1302,52 @@
unsigned int *data)
{
struct ni_gpct *counter = s->private;
- unsigned cidx = counter->counter_index;
- unsigned status;
+ unsigned int cidx = counter->counter_index;
+ unsigned int status;
+ int ret = 0;
switch (data[0]) {
case INSN_CONFIG_SET_COUNTER_MODE:
- return ni_tio_set_counter_mode(counter, data[1]);
+ ret = ni_tio_set_counter_mode(counter, data[1]);
+ break;
case INSN_CONFIG_ARM:
- return ni_tio_arm(counter, 1, data[1]);
+ ret = ni_tio_arm(counter, true, data[1]);
+ break;
case INSN_CONFIG_DISARM:
- ni_tio_arm(counter, 0, 0);
- return 0;
+ ret = ni_tio_arm(counter, false, 0);
+ break;
case INSN_CONFIG_GET_COUNTER_STATUS:
data[1] = 0;
- status = read_register(counter, NITIO_SHARED_STATUS_REG(cidx));
+ status = ni_tio_read(counter, NITIO_SHARED_STATUS_REG(cidx));
if (status & GI_ARMED(cidx)) {
data[1] |= COMEDI_COUNTER_ARMED;
if (status & GI_COUNTING(cidx))
data[1] |= COMEDI_COUNTER_COUNTING;
}
data[2] = COMEDI_COUNTER_ARMED | COMEDI_COUNTER_COUNTING;
- return 0;
+ break;
case INSN_CONFIG_SET_CLOCK_SRC:
- return ni_tio_set_clock_src(counter, data[1], data[2]);
+ ret = ni_tio_set_clock_src(counter, data[1], data[2]);
+ break;
case INSN_CONFIG_GET_CLOCK_SRC:
- ni_tio_get_clock_src(counter, &data[1], &data[2]);
- return 0;
+ ret = ni_tio_get_clock_src(counter, &data[1], &data[2]);
+ break;
case INSN_CONFIG_SET_GATE_SRC:
- return ni_tio_set_gate_src(counter, data[1], data[2]);
+ ret = ni_tio_set_gate_src(counter, data[1], data[2]);
+ break;
case INSN_CONFIG_GET_GATE_SRC:
- return ni_tio_get_gate_src(counter, data[1], &data[2]);
+ ret = ni_tio_get_gate_src(counter, data[1], &data[2]);
+ break;
case INSN_CONFIG_SET_OTHER_SRC:
- return ni_tio_set_other_src(counter, data[1], data[2]);
+ ret = ni_tio_set_other_src(counter, data[1], data[2]);
+ break;
case INSN_CONFIG_RESET:
ni_tio_reset_count_and_disarm(counter);
- return 0;
- default:
break;
+ default:
+ return -EINVAL;
}
- return -EINVAL;
+ return ret ? ret : insn->n;
}
EXPORT_SYMBOL_GPL(ni_tio_insn_config);
@@ -1219,7 +1355,7 @@
struct comedi_subdevice *s)
{
struct ni_gpct *counter = s->private;
- unsigned cidx = counter->counter_index;
+ unsigned int cidx = counter->counter_index;
unsigned int val;
ni_tio_set_bits(counter, NITIO_CMD_REG(cidx), GI_SAVE_TRACE, 0);
@@ -1235,9 +1371,9 @@
* will be correct since the count value will definitely have latched
* by then.
*/
- val = read_register(counter, NITIO_SW_SAVE_REG(cidx));
- if (val != read_register(counter, NITIO_SW_SAVE_REG(cidx)))
- val = read_register(counter, NITIO_SW_SAVE_REG(cidx));
+ val = ni_tio_read(counter, NITIO_SW_SAVE_REG(cidx));
+ if (val != ni_tio_read(counter, NITIO_SW_SAVE_REG(cidx)))
+ val = ni_tio_read(counter, NITIO_SW_SAVE_REG(cidx));
return val;
}
@@ -1250,7 +1386,7 @@
struct ni_gpct *counter = s->private;
struct ni_gpct_device *counter_dev = counter->counter_dev;
unsigned int channel = CR_CHAN(insn->chanspec);
- unsigned cidx = counter->counter_index;
+ unsigned int cidx = counter->counter_index;
int i;
for (i = 0; i < insn->n; i++) {
@@ -1270,11 +1406,10 @@
}
EXPORT_SYMBOL_GPL(ni_tio_insn_read);
-static unsigned ni_tio_next_load_register(struct ni_gpct *counter)
+static unsigned int ni_tio_next_load_register(struct ni_gpct *counter)
{
- unsigned cidx = counter->counter_index;
- const unsigned bits =
- read_register(counter, NITIO_SHARED_STATUS_REG(cidx));
+ unsigned int cidx = counter->counter_index;
+ unsigned int bits = ni_tio_read(counter, NITIO_SHARED_STATUS_REG(cidx));
return (bits & GI_NEXT_LOAD_SRC(cidx))
? NITIO_LOADB_REG(cidx)
@@ -1288,9 +1423,9 @@
{
struct ni_gpct *counter = s->private;
struct ni_gpct_device *counter_dev = counter->counter_dev;
- const unsigned channel = CR_CHAN(insn->chanspec);
- unsigned cidx = counter->counter_index;
- unsigned load_reg;
+ unsigned int channel = CR_CHAN(insn->chanspec);
+ unsigned int cidx = counter->counter_index;
+ unsigned int load_reg;
if (insn->n < 1)
return 0;
@@ -1306,19 +1441,19 @@
* load register is already selected.
*/
load_reg = ni_tio_next_load_register(counter);
- write_register(counter, data[0], load_reg);
+ ni_tio_write(counter, data[0], load_reg);
ni_tio_set_bits_transient(counter, NITIO_CMD_REG(cidx),
0, 0, GI_LOAD);
/* restore load reg */
- write_register(counter, counter_dev->regs[load_reg], load_reg);
+ ni_tio_write(counter, counter_dev->regs[load_reg], load_reg);
break;
case 1:
counter_dev->regs[NITIO_LOADA_REG(cidx)] = data[0];
- write_register(counter, data[0], NITIO_LOADA_REG(cidx));
+ ni_tio_write(counter, data[0], NITIO_LOADA_REG(cidx));
break;
case 2:
counter_dev->regs[NITIO_LOADB_REG(cidx)] = data[0];
- write_register(counter, data[0], NITIO_LOADB_REG(cidx));
+ ni_tio_write(counter, data[0], NITIO_LOADB_REG(cidx));
break;
default:
return -EINVAL;
@@ -1330,13 +1465,13 @@
void ni_tio_init_counter(struct ni_gpct *counter)
{
struct ni_gpct_device *counter_dev = counter->counter_dev;
- unsigned cidx = counter->counter_index;
+ unsigned int cidx = counter->counter_index;
ni_tio_reset_count_and_disarm(counter);
/* initialize counter registers */
counter_dev->regs[NITIO_AUTO_INC_REG(cidx)] = 0x0;
- write_register(counter, 0x0, NITIO_AUTO_INC_REG(cidx));
+ ni_tio_write(counter, 0x0, NITIO_AUTO_INC_REG(cidx));
ni_tio_set_bits(counter, NITIO_CMD_REG(cidx),
~0, GI_SYNC_GATE);
@@ -1344,10 +1479,10 @@
ni_tio_set_bits(counter, NITIO_MODE_REG(cidx), ~0, 0);
counter_dev->regs[NITIO_LOADA_REG(cidx)] = 0x0;
- write_register(counter, 0x0, NITIO_LOADA_REG(cidx));
+ ni_tio_write(counter, 0x0, NITIO_LOADA_REG(cidx));
counter_dev->regs[NITIO_LOADB_REG(cidx)] = 0x0;
- write_register(counter, 0x0, NITIO_LOADB_REG(cidx));
+ ni_tio_write(counter, 0x0, NITIO_LOADB_REG(cidx));
ni_tio_set_bits(counter, NITIO_INPUT_SEL_REG(cidx), ~0, 0);
@@ -1356,7 +1491,7 @@
if (ni_tio_has_gate2_registers(counter_dev)) {
counter_dev->regs[NITIO_GATE2_REG(cidx)] = 0x0;
- write_register(counter, 0x0, NITIO_GATE2_REG(cidx));
+ ni_tio_write(counter, 0x0, NITIO_GATE2_REG(cidx));
}
ni_tio_set_bits(counter, NITIO_DMA_CFG_REG(cidx), ~0, 0x0);
@@ -1367,17 +1502,17 @@
struct ni_gpct_device *
ni_gpct_device_construct(struct comedi_device *dev,
- void (*write_register)(struct ni_gpct *counter,
- unsigned bits,
- enum ni_gpct_register reg),
- unsigned (*read_register)(struct ni_gpct *counter,
- enum ni_gpct_register reg),
+ void (*write)(struct ni_gpct *counter,
+ unsigned int value,
+ enum ni_gpct_register reg),
+ unsigned int (*read)(struct ni_gpct *counter,
+ enum ni_gpct_register reg),
enum ni_gpct_variant variant,
- unsigned num_counters)
+ unsigned int num_counters)
{
struct ni_gpct_device *counter_dev;
struct ni_gpct *counter;
- unsigned i;
+ unsigned int i;
if (num_counters == 0)
return NULL;
@@ -1387,8 +1522,8 @@
return NULL;
counter_dev->dev = dev;
- counter_dev->write_register = write_register;
- counter_dev->read_register = read_register;
+ counter_dev->write = write;
+ counter_dev->read = read;
counter_dev->variant = variant;
spin_lock_init(&counter_dev->regs_lock);
@@ -1413,7 +1548,7 @@
void ni_gpct_device_destroy(struct ni_gpct_device *counter_dev)
{
- if (!counter_dev->counters)
+ if (!counter_dev)
return;
kfree(counter_dev->counters);
kfree(counter_dev);
diff --git a/drivers/staging/comedi/drivers/ni_tio.h b/drivers/staging/comedi/drivers/ni_tio.h
index 25aedd0..4978358 100644
--- a/drivers/staging/comedi/drivers/ni_tio.h
+++ b/drivers/staging/comedi/drivers/ni_tio.h
@@ -1,29 +1,24 @@
/*
- drivers/ni_tio.h
- Header file for NI general purpose counter support code (ni_tio.c)
-
- COMEDI - Linux Control and Measurement Device Interface
-
- This program is free software; you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation; either version 2 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-*/
+ * Header file for NI general purpose counter support code (ni_tio.c)
+ *
+ * COMEDI - Linux Control and Measurement Device Interface
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
#ifndef _COMEDI_NI_TIO_H
#define _COMEDI_NI_TIO_H
#include "../comedidev.h"
-/* forward declarations */
-struct mite_struct;
-struct ni_gpct_device;
-
enum ni_gpct_register {
NITIO_G0_AUTO_INC,
NITIO_G1_AUTO_INC,
@@ -106,35 +101,34 @@
struct ni_gpct {
struct ni_gpct_device *counter_dev;
- unsigned counter_index;
- unsigned chip_index;
- uint64_t clock_period_ps; /* clock period in picoseconds */
+ unsigned int counter_index;
+ unsigned int chip_index;
+ u64 clock_period_ps; /* clock period in picoseconds */
struct mite_channel *mite_chan;
- spinlock_t lock;
+ spinlock_t lock; /* protects 'mite_chan' */
};
struct ni_gpct_device {
struct comedi_device *dev;
- void (*write_register)(struct ni_gpct *counter, unsigned bits,
- enum ni_gpct_register reg);
- unsigned (*read_register)(struct ni_gpct *counter,
- enum ni_gpct_register reg);
+ void (*write)(struct ni_gpct *, unsigned int value,
+ enum ni_gpct_register);
+ unsigned int (*read)(struct ni_gpct *, enum ni_gpct_register);
enum ni_gpct_variant variant;
struct ni_gpct *counters;
- unsigned num_counters;
- unsigned regs[NITIO_NUM_REGS];
- spinlock_t regs_lock;
+ unsigned int num_counters;
+ unsigned int regs[NITIO_NUM_REGS];
+ spinlock_t regs_lock; /* protects 'regs' */
};
struct ni_gpct_device *
ni_gpct_device_construct(struct comedi_device *,
- void (*write_register)(struct ni_gpct *,
- unsigned bits,
- enum ni_gpct_register),
- unsigned (*read_register)(struct ni_gpct *,
- enum ni_gpct_register),
+ void (*write)(struct ni_gpct *,
+ unsigned int value,
+ enum ni_gpct_register),
+ unsigned int (*read)(struct ni_gpct *,
+ enum ni_gpct_register),
enum ni_gpct_variant,
- unsigned num_counters);
+ unsigned int num_counters);
void ni_gpct_device_destroy(struct ni_gpct_device *);
void ni_tio_init_counter(struct ni_gpct *);
int ni_tio_insn_read(struct comedi_device *, struct comedi_subdevice *,
diff --git a/drivers/staging/comedi/drivers/ni_tio_internal.h b/drivers/staging/comedi/drivers/ni_tio_internal.h
index 2bceae4..b15b108 100644
--- a/drivers/staging/comedi/drivers/ni_tio_internal.h
+++ b/drivers/staging/comedi/drivers/ni_tio_internal.h
@@ -1,20 +1,19 @@
/*
- drivers/ni_tio_internal.h
- Header file for NI general purpose counter support code (ni_tio.c and
- ni_tiocmd.c)
-
- COMEDI - Linux Control and Measurement Device Interface
-
- This program is free software; you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation; either version 2 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-*/
+ * Header file for NI general purpose counter support code (ni_tio.c and
+ * ni_tiocmd.c)
+ *
+ * COMEDI - Linux Control and Measurement Device Interface
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
#ifndef _COMEDI_NI_TIO_INTERNAL_H
#define _COMEDI_NI_TIO_INTERNAL_H
@@ -24,68 +23,73 @@
#define NITIO_AUTO_INC_REG(x) (NITIO_G0_AUTO_INC + (x))
#define GI_AUTO_INC_MASK 0xff
#define NITIO_CMD_REG(x) (NITIO_G0_CMD + (x))
-#define GI_ARM (1 << 0)
-#define GI_SAVE_TRACE (1 << 1)
-#define GI_LOAD (1 << 2)
-#define GI_DISARM (1 << 4)
+#define GI_ARM BIT(0)
+#define GI_SAVE_TRACE BIT(1)
+#define GI_LOAD BIT(2)
+#define GI_DISARM BIT(4)
#define GI_CNT_DIR(x) (((x) & 0x3) << 5)
-#define GI_CNT_DIR_MASK (3 << 5)
-#define GI_WRITE_SWITCH (1 << 7)
-#define GI_SYNC_GATE (1 << 8)
-#define GI_LITTLE_BIG_ENDIAN (1 << 9)
-#define GI_BANK_SWITCH_START (1 << 10)
-#define GI_BANK_SWITCH_MODE (1 << 11)
-#define GI_BANK_SWITCH_ENABLE (1 << 12)
-#define GI_ARM_COPY (1 << 13)
-#define GI_SAVE_TRACE_COPY (1 << 14)
-#define GI_DISARM_COPY (1 << 15)
+#define GI_CNT_DIR_MASK GI_CNT_DIR(3)
+#define GI_WRITE_SWITCH BIT(7)
+#define GI_SYNC_GATE BIT(8)
+#define GI_LITTLE_BIG_ENDIAN BIT(9)
+#define GI_BANK_SWITCH_START BIT(10)
+#define GI_BANK_SWITCH_MODE BIT(11)
+#define GI_BANK_SWITCH_ENABLE BIT(12)
+#define GI_ARM_COPY BIT(13)
+#define GI_SAVE_TRACE_COPY BIT(14)
+#define GI_DISARM_COPY BIT(15)
#define NITIO_HW_SAVE_REG(x) (NITIO_G0_HW_SAVE + (x))
#define NITIO_SW_SAVE_REG(x) (NITIO_G0_SW_SAVE + (x))
#define NITIO_MODE_REG(x) (NITIO_G0_MODE + (x))
-#define GI_GATING_DISABLED (0 << 0)
-#define GI_LEVEL_GATING (1 << 0)
-#define GI_RISING_EDGE_GATING (2 << 0)
-#define GI_FALLING_EDGE_GATING (3 << 0)
-#define GI_GATING_MODE_MASK (3 << 0)
-#define GI_GATE_ON_BOTH_EDGES (1 << 2)
-#define GI_EDGE_GATE_STARTS_STOPS (0 << 3)
-#define GI_EDGE_GATE_STOPS_STARTS (1 << 3)
-#define GI_EDGE_GATE_STARTS (2 << 3)
-#define GI_EDGE_GATE_NO_STARTS_OR_STOPS (3 << 3)
-#define GI_EDGE_GATE_MODE_MASK (3 << 3)
-#define GI_STOP_ON_GATE (0 << 5)
-#define GI_STOP_ON_GATE_OR_TC (1 << 5)
-#define GI_STOP_ON_GATE_OR_SECOND_TC (2 << 5)
-#define GI_STOP_MODE_MASK (3 << 5)
-#define GI_LOAD_SRC_SEL (1 << 7)
-#define GI_OUTPUT_TC_PULSE (1 << 8)
-#define GI_OUTPUT_TC_TOGGLE (2 << 8)
-#define GI_OUTPUT_TC_OR_GATE_TOGGLE (3 << 8)
-#define GI_OUTPUT_MODE_MASK (3 << 8)
-#define GI_NO_HARDWARE_DISARM (0 << 10)
-#define GI_DISARM_AT_TC (1 << 10)
-#define GI_DISARM_AT_GATE (2 << 10)
-#define GI_DISARM_AT_TC_OR_GATE (3 << 10)
-#define GI_COUNTING_ONCE_MASK (3 << 10)
-#define GI_LOADING_ON_TC (1 << 12)
-#define GI_GATE_POL_INVERT (1 << 13)
-#define GI_LOADING_ON_GATE (1 << 14)
-#define GI_RELOAD_SRC_SWITCHING (1 << 15)
+#define GI_GATING_MODE(x) (((x) & 0x3) << 0)
+#define GI_GATING_DISABLED GI_GATING_MODE(0)
+#define GI_LEVEL_GATING GI_GATING_MODE(1)
+#define GI_RISING_EDGE_GATING GI_GATING_MODE(2)
+#define GI_FALLING_EDGE_GATING GI_GATING_MODE(3)
+#define GI_GATING_MODE_MASK GI_GATING_MODE(3)
+#define GI_GATE_ON_BOTH_EDGES BIT(2)
+#define GI_EDGE_GATE_MODE(x) (((x) & 0x3) << 3)
+#define GI_EDGE_GATE_STARTS_STOPS GI_EDGE_GATE_MODE(0)
+#define GI_EDGE_GATE_STOPS_STARTS GI_EDGE_GATE_MODE(1)
+#define GI_EDGE_GATE_STARTS GI_EDGE_GATE_MODE(2)
+#define GI_EDGE_GATE_NO_STARTS_OR_STOPS GI_EDGE_GATE_MODE(3)
+#define GI_EDGE_GATE_MODE_MASK GI_EDGE_GATE_MODE(3)
+#define GI_STOP_MODE(x) (((x) & 0x3) << 5)
+#define GI_STOP_ON_GATE GI_STOP_MODE(0)
+#define GI_STOP_ON_GATE_OR_TC GI_STOP_MODE(1)
+#define GI_STOP_ON_GATE_OR_SECOND_TC GI_STOP_MODE(2)
+#define GI_STOP_MODE_MASK GI_STOP_MODE(3)
+#define GI_LOAD_SRC_SEL BIT(7)
+#define GI_OUTPUT_MODE(x) (((x) & 0x3) << 8)
+#define GI_OUTPUT_TC_PULSE GI_OUTPUT_MODE(1)
+#define GI_OUTPUT_TC_TOGGLE GI_OUTPUT_MODE(2)
+#define GI_OUTPUT_TC_OR_GATE_TOGGLE GI_OUTPUT_MODE(3)
+#define GI_OUTPUT_MODE_MASK GI_OUTPUT_MODE(3)
+#define GI_COUNTING_ONCE(x) (((x) & 0x3) << 10)
+#define GI_NO_HARDWARE_DISARM GI_COUNTING_ONCE(0)
+#define GI_DISARM_AT_TC GI_COUNTING_ONCE(1)
+#define GI_DISARM_AT_GATE GI_COUNTING_ONCE(2)
+#define GI_DISARM_AT_TC_OR_GATE GI_COUNTING_ONCE(3)
+#define GI_COUNTING_ONCE_MASK GI_COUNTING_ONCE(3)
+#define GI_LOADING_ON_TC BIT(12)
+#define GI_GATE_POL_INVERT BIT(13)
+#define GI_LOADING_ON_GATE BIT(14)
+#define GI_RELOAD_SRC_SWITCHING BIT(15)
#define NITIO_LOADA_REG(x) (NITIO_G0_LOADA + (x))
#define NITIO_LOADB_REG(x) (NITIO_G0_LOADB + (x))
#define NITIO_INPUT_SEL_REG(x) (NITIO_G0_INPUT_SEL + (x))
-#define GI_READ_ACKS_IRQ (1 << 0)
-#define GI_WRITE_ACKS_IRQ (1 << 1)
+#define GI_READ_ACKS_IRQ BIT(0)
+#define GI_WRITE_ACKS_IRQ BIT(1)
#define GI_BITS_TO_SRC(x) (((x) >> 2) & 0x1f)
#define GI_SRC_SEL(x) (((x) & 0x1f) << 2)
-#define GI_SRC_SEL_MASK (0x1f << 2)
+#define GI_SRC_SEL_MASK GI_SRC_SEL(0x1f)
#define GI_BITS_TO_GATE(x) (((x) >> 7) & 0x1f)
#define GI_GATE_SEL(x) (((x) & 0x1f) << 7)
-#define GI_GATE_SEL_MASK (0x1f << 7)
-#define GI_GATE_SEL_LOAD_SRC (1 << 12)
-#define GI_OR_GATE (1 << 13)
-#define GI_OUTPUT_POL_INVERT (1 << 14)
-#define GI_SRC_POL_INVERT (1 << 15)
+#define GI_GATE_SEL_MASK GI_GATE_SEL(0x1f)
+#define GI_GATE_SEL_LOAD_SRC BIT(12)
+#define GI_OR_GATE BIT(13)
+#define GI_OUTPUT_POL_INVERT BIT(14)
+#define GI_SRC_POL_INVERT BIT(15)
#define NITIO_CNT_MODE_REG(x) (NITIO_G0_CNT_MODE + (x))
#define GI_CNT_MODE(x) (((x) & 0x7) << 0)
#define GI_CNT_MODE_NORMAL GI_CNT_MODE(0)
@@ -94,152 +98,84 @@
#define GI_CNT_MODE_QUADX4 GI_CNT_MODE(3)
#define GI_CNT_MODE_TWO_PULSE GI_CNT_MODE(4)
#define GI_CNT_MODE_SYNC_SRC GI_CNT_MODE(6)
-#define GI_CNT_MODE_MASK (7 << 0)
-#define GI_INDEX_MODE (1 << 4)
+#define GI_CNT_MODE_MASK GI_CNT_MODE(7)
+#define GI_INDEX_MODE BIT(4)
#define GI_INDEX_PHASE(x) (((x) & 0x3) << 5)
-#define GI_INDEX_PHASE_MASK (3 << 5)
-#define GI_HW_ARM_ENA (1 << 7)
+#define GI_INDEX_PHASE_MASK GI_INDEX_PHASE(3)
+#define GI_HW_ARM_ENA BIT(7)
#define GI_HW_ARM_SEL(x) ((x) << 8)
-#define GI_660X_HW_ARM_SEL_MASK (0x7 << 8)
-#define GI_M_HW_ARM_SEL_MASK (0x1f << 8)
-#define GI_660X_PRESCALE_X8 (1 << 12)
-#define GI_M_PRESCALE_X8 (1 << 13)
-#define GI_660X_ALT_SYNC (1 << 13)
-#define GI_M_ALT_SYNC (1 << 14)
-#define GI_660X_PRESCALE_X2 (1 << 14)
-#define GI_M_PRESCALE_X2 (1 << 15)
+#define GI_660X_HW_ARM_SEL_MASK GI_HW_ARM_SEL(0x7)
+#define GI_M_HW_ARM_SEL_MASK GI_HW_ARM_SEL(0x1f)
+#define GI_660X_PRESCALE_X8 BIT(12)
+#define GI_M_PRESCALE_X8 BIT(13)
+#define GI_660X_ALT_SYNC BIT(13)
+#define GI_M_ALT_SYNC BIT(14)
+#define GI_660X_PRESCALE_X2 BIT(14)
+#define GI_M_PRESCALE_X2 BIT(15)
#define NITIO_GATE2_REG(x) (NITIO_G0_GATE2 + (x))
-#define GI_GATE2_MODE (1 << 0)
+#define GI_GATE2_MODE BIT(0)
#define GI_BITS_TO_GATE2(x) (((x) >> 7) & 0x1f)
#define GI_GATE2_SEL(x) (((x) & 0x1f) << 7)
-#define GI_GATE2_SEL_MASK (0x1f << 7)
-#define GI_GATE2_POL_INVERT (1 << 13)
-#define GI_GATE2_SUBSEL (1 << 14)
-#define GI_SRC_SUBSEL (1 << 15)
+#define GI_GATE2_SEL_MASK GI_GATE2_SEL(0x1f)
+#define GI_GATE2_POL_INVERT BIT(13)
+#define GI_GATE2_SUBSEL BIT(14)
+#define GI_SRC_SUBSEL BIT(15)
#define NITIO_SHARED_STATUS_REG(x) (NITIO_G01_STATUS + ((x) / 2))
-#define GI_SAVE(x) (((x) % 2) ? (1 << 1) : (1 << 0))
-#define GI_COUNTING(x) (((x) % 2) ? (1 << 3) : (1 << 2))
-#define GI_NEXT_LOAD_SRC(x) (((x) % 2) ? (1 << 5) : (1 << 4))
-#define GI_STALE_DATA(x) (((x) % 2) ? (1 << 7) : (1 << 6))
-#define GI_ARMED(x) (((x) % 2) ? (1 << 9) : (1 << 8))
-#define GI_NO_LOAD_BETWEEN_GATES(x) (((x) % 2) ? (1 << 11) : (1 << 10))
-#define GI_TC_ERROR(x) (((x) % 2) ? (1 << 13) : (1 << 12))
-#define GI_GATE_ERROR(x) (((x) % 2) ? (1 << 15) : (1 << 14))
+#define GI_SAVE(x) (((x) % 2) ? BIT(1) : BIT(0))
+#define GI_COUNTING(x) (((x) % 2) ? BIT(3) : BIT(2))
+#define GI_NEXT_LOAD_SRC(x) (((x) % 2) ? BIT(5) : BIT(4))
+#define GI_STALE_DATA(x) (((x) % 2) ? BIT(7) : BIT(6))
+#define GI_ARMED(x) (((x) % 2) ? BIT(9) : BIT(8))
+#define GI_NO_LOAD_BETWEEN_GATES(x) (((x) % 2) ? BIT(11) : BIT(10))
+#define GI_TC_ERROR(x) (((x) % 2) ? BIT(13) : BIT(12))
+#define GI_GATE_ERROR(x) (((x) % 2) ? BIT(15) : BIT(14))
#define NITIO_RESET_REG(x) (NITIO_G01_RESET + ((x) / 2))
-#define GI_RESET(x) (1 << (2 + ((x) % 2)))
+#define GI_RESET(x) BIT(2 + ((x) % 2))
#define NITIO_STATUS1_REG(x) (NITIO_G01_STATUS1 + ((x) / 2))
#define NITIO_STATUS2_REG(x) (NITIO_G01_STATUS2 + ((x) / 2))
-#define GI_OUTPUT(x) (((x) % 2) ? (1 << 1) : (1 << 0))
-#define GI_HW_SAVE(x) (((x) % 2) ? (1 << 13) : (1 << 12))
-#define GI_PERMANENT_STALE(x) (((x) % 2) ? (1 << 15) : (1 << 14))
+#define GI_OUTPUT(x) (((x) % 2) ? BIT(1) : BIT(0))
+#define GI_HW_SAVE(x) (((x) % 2) ? BIT(13) : BIT(12))
+#define GI_PERMANENT_STALE(x) (((x) % 2) ? BIT(15) : BIT(14))
#define NITIO_DMA_CFG_REG(x) (NITIO_G0_DMA_CFG + (x))
-#define GI_DMA_ENABLE (1 << 0)
-#define GI_DMA_WRITE (1 << 1)
-#define GI_DMA_INT_ENA (1 << 2)
-#define GI_DMA_RESET (1 << 3)
-#define GI_DMA_BANKSW_ERROR (1 << 4)
+#define GI_DMA_ENABLE BIT(0)
+#define GI_DMA_WRITE BIT(1)
+#define GI_DMA_INT_ENA BIT(2)
+#define GI_DMA_RESET BIT(3)
+#define GI_DMA_BANKSW_ERROR BIT(4)
#define NITIO_DMA_STATUS_REG(x) (NITIO_G0_DMA_STATUS + (x))
-#define GI_DMA_READBANK (1 << 13)
-#define GI_DRQ_ERROR (1 << 14)
-#define GI_DRQ_STATUS (1 << 15)
+#define GI_DMA_READBANK BIT(13)
+#define GI_DRQ_ERROR BIT(14)
+#define GI_DRQ_STATUS BIT(15)
#define NITIO_ABZ_REG(x) (NITIO_G0_ABZ + (x))
#define NITIO_INT_ACK_REG(x) (NITIO_G0_INT_ACK + (x))
-#define GI_GATE_ERROR_CONFIRM(x) (((x) % 2) ? (1 << 1) : (1 << 5))
-#define GI_TC_ERROR_CONFIRM(x) (((x) % 2) ? (1 << 2) : (1 << 6))
-#define GI_TC_INTERRUPT_ACK (1 << 14)
-#define GI_GATE_INTERRUPT_ACK (1 << 15)
+#define GI_GATE_ERROR_CONFIRM(x) (((x) % 2) ? BIT(1) : BIT(5))
+#define GI_TC_ERROR_CONFIRM(x) (((x) % 2) ? BIT(2) : BIT(6))
+#define GI_TC_INTERRUPT_ACK BIT(14)
+#define GI_GATE_INTERRUPT_ACK BIT(15)
#define NITIO_STATUS_REG(x) (NITIO_G0_STATUS + (x))
-#define GI_GATE_INTERRUPT (1 << 2)
-#define GI_TC (1 << 3)
-#define GI_INTERRUPT (1 << 15)
+#define GI_GATE_INTERRUPT BIT(2)
+#define GI_TC BIT(3)
+#define GI_INTERRUPT BIT(15)
#define NITIO_INT_ENA_REG(x) (NITIO_G0_INT_ENA + (x))
-#define GI_TC_INTERRUPT_ENABLE(x) (((x) % 2) ? (1 << 9) : (1 << 6))
-#define GI_GATE_INTERRUPT_ENABLE(x) (((x) % 2) ? (1 << 10) : (1 << 8))
+#define GI_TC_INTERRUPT_ENABLE(x) (((x) % 2) ? BIT(9) : BIT(6))
+#define GI_GATE_INTERRUPT_ENABLE(x) (((x) % 2) ? BIT(10) : BIT(8))
-static inline void write_register(struct ni_gpct *counter, unsigned bits,
- enum ni_gpct_register reg)
+void ni_tio_write(struct ni_gpct *, unsigned int value, enum ni_gpct_register);
+unsigned int ni_tio_read(struct ni_gpct *, enum ni_gpct_register);
+
+static inline bool
+ni_tio_counting_mode_registers_present(const struct ni_gpct_device *counter_dev)
{
- BUG_ON(reg >= NITIO_NUM_REGS);
- counter->counter_dev->write_register(counter, bits, reg);
+ /* m series and 660x variants have counting mode registers */
+ return counter_dev->variant != ni_gpct_variant_e_series;
}
-static inline unsigned read_register(struct ni_gpct *counter,
- enum ni_gpct_register reg)
-{
- BUG_ON(reg >= NITIO_NUM_REGS);
- return counter->counter_dev->read_register(counter, reg);
-}
+void ni_tio_set_bits(struct ni_gpct *, enum ni_gpct_register reg,
+ unsigned int mask, unsigned int value);
+unsigned int ni_tio_get_soft_copy(const struct ni_gpct *,
+ enum ni_gpct_register reg);
-static inline int ni_tio_counting_mode_registers_present(const struct
- ni_gpct_device
- *counter_dev)
-{
- switch (counter_dev->variant) {
- case ni_gpct_variant_e_series:
- return 0;
- case ni_gpct_variant_m_series:
- case ni_gpct_variant_660x:
- return 1;
- default:
- BUG();
- break;
- }
- return 0;
-}
-
-static inline void ni_tio_set_bits_transient(struct ni_gpct *counter,
- enum ni_gpct_register
- register_index, unsigned bit_mask,
- unsigned bit_values,
- unsigned transient_bit_values)
-{
- struct ni_gpct_device *counter_dev = counter->counter_dev;
- unsigned long flags;
-
- BUG_ON(register_index >= NITIO_NUM_REGS);
- spin_lock_irqsave(&counter_dev->regs_lock, flags);
- counter_dev->regs[register_index] &= ~bit_mask;
- counter_dev->regs[register_index] |= (bit_values & bit_mask);
- write_register(counter,
- counter_dev->regs[register_index] | transient_bit_values,
- register_index);
- mmiowb();
- spin_unlock_irqrestore(&counter_dev->regs_lock, flags);
-}
-
-/* ni_tio_set_bits( ) is for safely writing to registers whose bits may be
- * twiddled in interrupt context, or whose software copy may be read in
- * interrupt context.
- */
-static inline void ni_tio_set_bits(struct ni_gpct *counter,
- enum ni_gpct_register register_index,
- unsigned bit_mask, unsigned bit_values)
-{
- ni_tio_set_bits_transient(counter, register_index, bit_mask, bit_values,
- 0x0);
-}
-
-/* ni_tio_get_soft_copy( ) is for safely reading the software copy of a register
-whose bits might be modified in interrupt context, or whose software copy
-might need to be read in interrupt context.
-*/
-static inline unsigned ni_tio_get_soft_copy(const struct ni_gpct *counter,
- enum ni_gpct_register
- register_index)
-{
- struct ni_gpct_device *counter_dev = counter->counter_dev;
- unsigned long flags;
- unsigned value;
-
- BUG_ON(register_index >= NITIO_NUM_REGS);
- spin_lock_irqsave(&counter_dev->regs_lock, flags);
- value = counter_dev->regs[register_index];
- spin_unlock_irqrestore(&counter_dev->regs_lock, flags);
- return value;
-}
-
-int ni_tio_arm(struct ni_gpct *counter, int arm, unsigned start_trigger);
-int ni_tio_set_gate_src(struct ni_gpct *counter, unsigned gate_index,
- unsigned int gate_source);
+int ni_tio_arm(struct ni_gpct *, bool arm, unsigned int start_trigger);
+int ni_tio_set_gate_src(struct ni_gpct *, unsigned int gate, unsigned int src);
#endif /* _COMEDI_NI_TIO_INTERNAL_H */
diff --git a/drivers/staging/comedi/drivers/ni_tiocmd.c b/drivers/staging/comedi/drivers/ni_tiocmd.c
index 823e479..9007c57 100644
--- a/drivers/staging/comedi/drivers/ni_tiocmd.c
+++ b/drivers/staging/comedi/drivers/ni_tiocmd.c
@@ -1,19 +1,18 @@
/*
- comedi/drivers/ni_tiocmd.c
- Command support for NI general purpose counters
-
- Copyright (C) 2006 Frank Mori Hess <fmhess@users.sourceforge.net>
-
- This program is free software; you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation; either version 2 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-*/
+ * Command support for NI general purpose counters
+ *
+ * Copyright (C) 2006 Frank Mori Hess <fmhess@users.sourceforge.net>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
/*
* Module: ni_tiocmd
@@ -36,13 +35,10 @@
* DAQ 660x Register-Level Programmer Manual (NI 370505A-01)
* DAQ 6601/6602 User Manual (NI 322137B-01)
* 340934b.pdf DAQ-STC reference manual
+ *
+ * TODO: Support use of both banks X and Y
*/
-/*
-TODO:
- Support use of both banks X and Y
-*/
-
#include <linux/module.h>
#include "ni_tio_internal.h"
#include "mite.h"
@@ -51,9 +47,9 @@
bool enable, bool read)
{
struct ni_gpct_device *counter_dev = counter->counter_dev;
- unsigned cidx = counter->counter_index;
- unsigned mask;
- unsigned bits;
+ unsigned int cidx = counter->counter_index;
+ unsigned int mask;
+ unsigned int bits;
mask = GI_READ_ACKS_IRQ | GI_WRITE_ACKS_IRQ;
bits = 0;
@@ -103,7 +99,7 @@
spin_unlock_irqrestore(&counter->lock, flags);
if (ret < 0)
return ret;
- ret = ni_tio_arm(counter, 1, NI_GPCT_ARM_IMMEDIATE);
+ ret = ni_tio_arm(counter, true, NI_GPCT_ARM_IMMEDIATE);
s->async->inttrig = NULL;
return ret;
@@ -113,7 +109,7 @@
{
struct ni_gpct *counter = s->private;
struct ni_gpct_device *counter_dev = counter->counter_dev;
- unsigned cidx = counter->counter_index;
+ unsigned int cidx = counter->counter_index;
struct comedi_async *async = s->async;
struct comedi_cmd *cmd = &async->cmd;
int ret = 0;
@@ -129,9 +125,6 @@
case ni_gpct_variant_e_series:
mite_prep_dma(counter->mite_chan, 16, 32);
break;
- default:
- BUG();
- break;
}
ni_tio_set_bits(counter, NITIO_CMD_REG(cidx), GI_SAVE_TRACE, 0);
ni_tio_configure_dma(counter, true, true);
@@ -143,9 +136,9 @@
mite_dma_arm(counter->mite_chan);
if (cmd->start_src == TRIG_NOW)
- ret = ni_tio_arm(counter, 1, NI_GPCT_ARM_IMMEDIATE);
+ ret = ni_tio_arm(counter, true, NI_GPCT_ARM_IMMEDIATE);
else if (cmd->start_src == TRIG_EXT)
- ret = ni_tio_arm(counter, 1, cmd->start_arg);
+ ret = ni_tio_arm(counter, true, cmd->start_arg);
}
return ret;
}
@@ -163,9 +156,9 @@
{
struct comedi_cmd *cmd = &s->async->cmd;
struct ni_gpct *counter = s->private;
- unsigned cidx = counter->counter_index;
+ unsigned int cidx = counter->counter_index;
int set_gate_source = 0;
- unsigned gate_source;
+ unsigned int gate_source;
int retval = 0;
if (cmd->scan_begin_src == TRIG_EXT) {
@@ -289,10 +282,10 @@
int ni_tio_cancel(struct ni_gpct *counter)
{
- unsigned cidx = counter->counter_index;
+ unsigned int cidx = counter->counter_index;
unsigned long flags;
- ni_tio_arm(counter, 0, 0);
+ ni_tio_arm(counter, false, 0);
spin_lock_irqsave(&counter->lock, flags);
if (counter->mite_chan)
mite_dma_disarm(counter->mite_chan);
@@ -305,9 +298,6 @@
}
EXPORT_SYMBOL_GPL(ni_tio_cancel);
- /* During buffered input counter operation for e-series, the gate
- interrupt is acked automatically by the dma controller, due to the
- Gi_Read/Write_Acknowledges_IRQ bits in the input select register. */
static int should_ack_gate(struct ni_gpct *counter)
{
unsigned long flags;
@@ -315,12 +305,19 @@
switch (counter->counter_dev->variant) {
case ni_gpct_variant_m_series:
- /* not sure if 660x really supports gate
- interrupts (the bits are not listed
- in register-level manual) */
case ni_gpct_variant_660x:
+ /*
+ * not sure if 660x really supports gate interrupts
+ * (the bits are not listed in register-level manual)
+ */
return 1;
case ni_gpct_variant_e_series:
+ /*
+ * During buffered input counter operation for e-series,
+ * the gate interrupt is acked automatically by the dma
+ * controller, due to the Gi_Read/Write_Acknowledges_IRQ
+ * bits in the input select register.
+ */
spin_lock_irqsave(&counter->lock, flags);
{
if (!counter->mite_chan ||
@@ -338,15 +335,14 @@
static void ni_tio_acknowledge_and_confirm(struct ni_gpct *counter,
int *gate_error,
int *tc_error,
- int *perm_stale_data,
- int *stale_data)
+ int *perm_stale_data)
{
- unsigned cidx = counter->counter_index;
- const unsigned short gxx_status = read_register(counter,
+ unsigned int cidx = counter->counter_index;
+ const unsigned short gxx_status = ni_tio_read(counter,
NITIO_SHARED_STATUS_REG(cidx));
- const unsigned short gi_status = read_register(counter,
+ const unsigned short gi_status = ni_tio_read(counter,
NITIO_STATUS_REG(cidx));
- unsigned ack = 0;
+ unsigned int ack = 0;
if (gate_error)
*gate_error = 0;
@@ -354,15 +350,15 @@
*tc_error = 0;
if (perm_stale_data)
*perm_stale_data = 0;
- if (stale_data)
- *stale_data = 0;
if (gxx_status & GI_GATE_ERROR(cidx)) {
ack |= GI_GATE_ERROR_CONFIRM(cidx);
if (gate_error) {
- /*660x don't support automatic acknowledgment
- of gate interrupt via dma read/write
- and report bogus gate errors */
+ /*
+ * 660x don't support automatic acknowledgment
+ * of gate interrupt via dma read/write
+ * and report bogus gate errors
+ */
if (counter->counter_dev->variant !=
ni_gpct_variant_660x)
*gate_error = 1;
@@ -380,14 +376,10 @@
ack |= GI_GATE_INTERRUPT_ACK;
}
if (ack)
- write_register(counter, ack, NITIO_INT_ACK_REG(cidx));
+ ni_tio_write(counter, ack, NITIO_INT_ACK_REG(cidx));
if (ni_tio_get_soft_copy(counter, NITIO_MODE_REG(cidx)) &
GI_LOADING_ON_GATE) {
- if (gxx_status & GI_STALE_DATA(cidx)) {
- if (stale_data)
- *stale_data = 1;
- }
- if (read_register(counter, NITIO_STATUS2_REG(cidx)) &
+ if (ni_tio_read(counter, NITIO_STATUS2_REG(cidx)) &
GI_PERMANENT_STALE(cidx)) {
dev_info(counter->counter_dev->dev->class_dev,
"%s: Gi_Permanent_Stale_Data detected.\n",
@@ -400,22 +392,21 @@
void ni_tio_acknowledge(struct ni_gpct *counter)
{
- ni_tio_acknowledge_and_confirm(counter, NULL, NULL, NULL, NULL);
+ ni_tio_acknowledge_and_confirm(counter, NULL, NULL, NULL);
}
EXPORT_SYMBOL_GPL(ni_tio_acknowledge);
void ni_tio_handle_interrupt(struct ni_gpct *counter,
struct comedi_subdevice *s)
{
- unsigned cidx = counter->counter_index;
- unsigned gpct_mite_status;
+ unsigned int cidx = counter->counter_index;
unsigned long flags;
int gate_error;
int tc_error;
int perm_stale_data;
ni_tio_acknowledge_and_confirm(counter, &gate_error, &tc_error,
- &perm_stale_data, NULL);
+ &perm_stale_data);
if (gate_error) {
dev_notice(counter->counter_dev->dev->class_dev,
"%s: Gi_Gate_Error detected.\n", __func__);
@@ -426,7 +417,7 @@
switch (counter->counter_dev->variant) {
case ni_gpct_variant_m_series:
case ni_gpct_variant_660x:
- if (read_register(counter, NITIO_DMA_STATUS_REG(cidx)) &
+ if (ni_tio_read(counter, NITIO_DMA_STATUS_REG(cidx)) &
GI_DRQ_ERROR) {
dev_notice(counter->counter_dev->dev->class_dev,
"%s: Gi_DRQ_Error detected.\n", __func__);
@@ -437,16 +428,8 @@
break;
}
spin_lock_irqsave(&counter->lock, flags);
- if (!counter->mite_chan) {
- spin_unlock_irqrestore(&counter->lock, flags);
- return;
- }
- gpct_mite_status = mite_get_status(counter->mite_chan);
- if (gpct_mite_status & CHSR_LINKC)
- writel(CHOR_CLRLC,
- counter->mite_chan->mite->mite_io_addr +
- MITE_CHOR(counter->mite_chan->channel));
- mite_sync_input_dma(counter->mite_chan, s);
+ if (counter->mite_chan)
+ mite_ack_linkc(counter->mite_chan, s, true);
spin_unlock_irqrestore(&counter->lock, flags);
}
EXPORT_SYMBOL_GPL(ni_tio_handle_interrupt);
diff --git a/drivers/staging/comedi/drivers/plx9052.h b/drivers/staging/comedi/drivers/plx9052.h
index fbcf250..2892e65 100644
--- a/drivers/staging/comedi/drivers/plx9052.h
+++ b/drivers/staging/comedi/drivers/plx9052.h
@@ -1,22 +1,21 @@
/*
- comedi/drivers/plx9052.h
- Definitions for the PLX-9052 PCI interface chip
-
- Copyright (C) 2002 MEV Ltd. <http://www.mev.co.uk/>
-
- COMEDI - Linux Control and Measurement Device Interface
- Copyright (C) 2000 David A. Schleef <ds@schleef.org>
-
- This program is free software; you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation; either version 2 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-*/
+ * Definitions for the PLX-9052 PCI interface chip
+ *
+ * Copyright (C) 2002 MEV Ltd. <http://www.mev.co.uk/>
+ *
+ * COMEDI - Linux Control and Measurement Device Interface
+ * Copyright (C) 2000 David A. Schleef <ds@schleef.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
#ifndef _PLX9052_H_
#define _PLX9052_H_
@@ -25,55 +24,56 @@
* INTCSR - Interrupt Control/Status register
*/
#define PLX9052_INTCSR 0x4c
-#define PLX9052_INTCSR_LI1ENAB (1 << 0) /* LI1 enabled */
-#define PLX9052_INTCSR_LI1POL (1 << 1) /* LI1 active high */
-#define PLX9052_INTCSR_LI1STAT (1 << 2) /* LI1 active */
-#define PLX9052_INTCSR_LI2ENAB (1 << 3) /* LI2 enabled */
-#define PLX9052_INTCSR_LI2POL (1 << 4) /* LI2 active high */
-#define PLX9052_INTCSR_LI2STAT (1 << 5) /* LI2 active */
-#define PLX9052_INTCSR_PCIENAB (1 << 6) /* PCIINT enabled */
-#define PLX9052_INTCSR_SOFTINT (1 << 7) /* generate soft int */
-#define PLX9052_INTCSR_LI1SEL (1 << 8) /* LI1 edge */
-#define PLX9052_INTCSR_LI2SEL (1 << 9) /* LI2 edge */
-#define PLX9052_INTCSR_LI1CLRINT (1 << 10) /* LI1 clear int */
-#define PLX9052_INTCSR_LI2CLRINT (1 << 11) /* LI2 clear int */
-#define PLX9052_INTCSR_ISAMODE (1 << 12) /* ISA interface mode */
+#define PLX9052_INTCSR_LI1ENAB BIT(0) /* LI1 enabled */
+#define PLX9052_INTCSR_LI1POL BIT(1) /* LI1 active high */
+#define PLX9052_INTCSR_LI1STAT BIT(2) /* LI1 active */
+#define PLX9052_INTCSR_LI2ENAB BIT(3) /* LI2 enabled */
+#define PLX9052_INTCSR_LI2POL BIT(4) /* LI2 active high */
+#define PLX9052_INTCSR_LI2STAT BIT(5) /* LI2 active */
+#define PLX9052_INTCSR_PCIENAB BIT(6) /* PCIINT enabled */
+#define PLX9052_INTCSR_SOFTINT BIT(7) /* generate soft int */
+#define PLX9052_INTCSR_LI1SEL BIT(8) /* LI1 edge */
+#define PLX9052_INTCSR_LI2SEL BIT(9) /* LI2 edge */
+#define PLX9052_INTCSR_LI1CLRINT BIT(10) /* LI1 clear int */
+#define PLX9052_INTCSR_LI2CLRINT BIT(11) /* LI2 clear int */
+#define PLX9052_INTCSR_ISAMODE BIT(12) /* ISA interface mode */
/*
* CNTRL - User I/O, Direct Slave Response, Serial EEPROM, and
* Initialization Control register
*/
#define PLX9052_CNTRL 0x50
-#define PLX9052_CNTRL_WAITO (1 << 0) /* UIO0 or WAITO# select */
-#define PLX9052_CNTRL_UIO0_DIR (1 << 1) /* UIO0 direction */
-#define PLX9052_CNTRL_UIO0_DATA (1 << 2) /* UIO0 data */
-#define PLX9052_CNTRL_LLOCKO (1 << 3) /* UIO1 or LLOCKo# select */
-#define PLX9052_CNTRL_UIO1_DIR (1 << 4) /* UIO1 direction */
-#define PLX9052_CNTRL_UIO1_DATA (1 << 5) /* UIO1 data */
-#define PLX9052_CNTRL_CS2 (1 << 6) /* UIO2 or CS2# select */
-#define PLX9052_CNTRL_UIO2_DIR (1 << 7) /* UIO2 direction */
-#define PLX9052_CNTRL_UIO2_DATA (1 << 8) /* UIO2 data */
-#define PLX9052_CNTRL_CS3 (1 << 9) /* UIO3 or CS3# select */
-#define PLX9052_CNTRL_UIO3_DIR (1 << 10) /* UIO3 direction */
-#define PLX9052_CNTRL_UIO3_DATA (1 << 11) /* UIO3 data */
-#define PLX9052_CNTRL_PCIBAR01 (0 << 12) /* bar 0 (mem) and 1 (I/O) */
-#define PLX9052_CNTRL_PCIBAR0 (1 << 12) /* bar 0 (mem) only */
-#define PLX9052_CNTRL_PCIBAR1 (2 << 12) /* bar 1 (I/O) only */
-#define PLX9052_CNTRL_PCI2_1_FEATURES (1 << 14) /* PCI r2.1 features enabled */
-#define PLX9052_CNTRL_PCI_R_W_FLUSH (1 << 15) /* read w/write flush mode */
-#define PLX9052_CNTRL_PCI_R_NO_FLUSH (1 << 16) /* read no flush mode */
-#define PLX9052_CNTRL_PCI_R_NO_WRITE (1 << 17) /* read no write mode */
-#define PLX9052_CNTRL_PCI_W_RELEASE (1 << 18) /* write release bus mode */
-#define PLX9052_CNTRL_RETRY_CLKS(x) (((x) & 0xf) << 19) /* slave retry clks */
-#define PLX9052_CNTRL_LOCK_ENAB (1 << 23) /* slave LOCK# enable */
+#define PLX9052_CNTRL_WAITO BIT(0) /* UIO0 or WAITO# select */
+#define PLX9052_CNTRL_UIO0_DIR BIT(1) /* UIO0 direction */
+#define PLX9052_CNTRL_UIO0_DATA BIT(2) /* UIO0 data */
+#define PLX9052_CNTRL_LLOCKO BIT(3) /* UIO1 or LLOCKo# select */
+#define PLX9052_CNTRL_UIO1_DIR BIT(4) /* UIO1 direction */
+#define PLX9052_CNTRL_UIO1_DATA BIT(5) /* UIO1 data */
+#define PLX9052_CNTRL_CS2 BIT(6) /* UIO2 or CS2# select */
+#define PLX9052_CNTRL_UIO2_DIR BIT(7) /* UIO2 direction */
+#define PLX9052_CNTRL_UIO2_DATA BIT(8) /* UIO2 data */
+#define PLX9052_CNTRL_CS3 BIT(9) /* UIO3 or CS3# select */
+#define PLX9052_CNTRL_UIO3_DIR BIT(10) /* UIO3 direction */
+#define PLX9052_CNTRL_UIO3_DATA BIT(11) /* UIO3 data */
+#define PLX9052_CNTRL_PCIBAR(x) (((x) & 0x3) << 12)
+#define PLX9052_CNTRL_PCIBAR01 PLX9052_CNTRL_PCIBAR(0) /* mem and IO */
+#define PLX9052_CNTRL_PCIBAR0 PLX9052_CNTRL_PCIBAR(1) /* mem only */
+#define PLX9052_CNTRL_PCIBAR1 PLX9052_CNTRL_PCIBAR(2) /* IO only */
+#define PLX9052_CNTRL_PCI2_1_FEATURES BIT(14) /* PCI v2.1 features enabled */
+#define PLX9052_CNTRL_PCI_R_W_FLUSH BIT(15) /* read w/write flush mode */
+#define PLX9052_CNTRL_PCI_R_NO_FLUSH BIT(16) /* read no flush mode */
+#define PLX9052_CNTRL_PCI_R_NO_WRITE BIT(17) /* read no write mode */
+#define PLX9052_CNTRL_PCI_W_RELEASE BIT(18) /* write release bus mode */
+#define PLX9052_CNTRL_RETRY_CLKS(x) (((x) & 0xf) << 19) /* retry clks */
+#define PLX9052_CNTRL_LOCK_ENAB BIT(23) /* slave LOCK# enable */
#define PLX9052_CNTRL_EEPROM_MASK (0x1f << 24) /* EEPROM bits */
-#define PLX9052_CNTRL_EEPROM_CLK (1 << 24) /* EEPROM clock */
-#define PLX9052_CNTRL_EEPROM_CS (1 << 25) /* EEPROM chip select */
-#define PLX9052_CNTRL_EEPROM_DOUT (1 << 26) /* EEPROM write bit */
-#define PLX9052_CNTRL_EEPROM_DIN (1 << 27) /* EEPROM read bit */
-#define PLX9052_CNTRL_EEPROM_PRESENT (1 << 28) /* EEPROM present */
-#define PLX9052_CNTRL_RELOAD_CFG (1 << 29) /* reload configuration */
-#define PLX9052_CNTRL_PCI_RESET (1 << 30) /* PCI adapter reset */
-#define PLX9052_CNTRL_MASK_REV (1 << 31) /* mask revision */
+#define PLX9052_CNTRL_EEPROM_CLK BIT(24) /* EEPROM clock */
+#define PLX9052_CNTRL_EEPROM_CS BIT(25) /* EEPROM chip select */
+#define PLX9052_CNTRL_EEPROM_DOUT BIT(26) /* EEPROM write bit */
+#define PLX9052_CNTRL_EEPROM_DIN BIT(27) /* EEPROM read bit */
+#define PLX9052_CNTRL_EEPROM_PRESENT BIT(28) /* EEPROM present */
+#define PLX9052_CNTRL_RELOAD_CFG BIT(29) /* reload configuration */
+#define PLX9052_CNTRL_PCI_RESET BIT(30) /* PCI adapter reset */
+#define PLX9052_CNTRL_MASK_REV BIT(31) /* mask revision */
#endif /* _PLX9052_H_ */
diff --git a/drivers/staging/comedi/drivers/plx9080.h b/drivers/staging/comedi/drivers/plx9080.h
index f5cd6d5..8d1aee00b1 100644
--- a/drivers/staging/comedi/drivers/plx9080.h
+++ b/drivers/staging/comedi/drivers/plx9080.h
@@ -88,7 +88,7 @@
/* direct slave LLOCKo# enable */
MARB_DS_LLOCK_ENABLE = 0x00400000,
MARB_PCI_REQUEST_MODE = 0x00800000,
- MARB_PCIv21_MODE = 0x01000000, /* pci specification v2.1 mode */
+ MARB_PCIV21_MODE = 0x01000000, /* pci specification v2.1 mode */
MARB_PCI_READ_NO_WRITE_MODE = 0x02000000,
MARB_PCI_READ_WITH_WRITE_FLUSH_MODE = 0x04000000,
/* gate local bus latency timer with BREQ */
diff --git a/drivers/staging/comedi/drivers/z8536.h b/drivers/staging/comedi/drivers/z8536.h
index 7be5310..47eadbf 100644
--- a/drivers/staging/comedi/drivers/z8536.h
+++ b/drivers/staging/comedi/drivers/z8536.h
@@ -24,11 +24,12 @@
#define Z8536_CFG_CTRL_PCE_CT3E BIT(4) /* Port C & C/T 3 Enable */
#define Z8536_CFG_CTRL_PLC BIT(3) /* Port A/B Link Control */
#define Z8536_CFG_CTRL_PAE BIT(2) /* Port A Enable */
-#define Z8536_CFG_CTRL_LC_INDEP (0 << 0)/* C/Ts Independent */
-#define Z8536_CFG_CTRL_LC_GATE (1 << 0)/* C/T 1 Out Gates C/T 2 */
-#define Z8536_CFG_CTRL_LC_TRIG (2 << 0)/* C/T 1 Out Triggers C/T 2 */
-#define Z8536_CFG_CTRL_LC_CLK (3 << 0)/* C/T 1 Out Clocks C/T 2 */
-#define Z8536_CFG_CTRL_LC_MASK (3 << 0)/* C/T Link Control mask */
+#define Z8536_CFG_CTRL_LC(x) (((x) & 0x3) << 0) /* Link Control */
+#define Z8536_CFG_CTRL_LC_INDEP Z8536_CFG_CTRL_LC(0)/* Independent */
+#define Z8536_CFG_CTRL_LC_GATE Z8536_CFG_CTRL_LC(1)/* 1 Gates 2 */
+#define Z8536_CFG_CTRL_LC_TRIG Z8536_CFG_CTRL_LC(2)/* 1 Triggers 2 */
+#define Z8536_CFG_CTRL_LC_CLK Z8536_CFG_CTRL_LC(3)/* 1 Clocks 2 */
+#define Z8536_CFG_CTRL_LC_MASK Z8536_CFG_CTRL_LC(3)
/* Interrupt Vector registers */
#define Z8536_PA_INT_VECT_REG 0x02
@@ -43,15 +44,16 @@
#define Z8536_CT2_CMDSTAT_REG 0x0b
#define Z8536_CT3_CMDSTAT_REG 0x0c
#define Z8536_CT_CMDSTAT_REG(x) (0x0a + (x))
-#define Z8536_CMD_NULL (0 << 5)/* Null Code */
-#define Z8536_CMD_CLR_IP_IUS (1 << 5)/* Clear IP & IUS */
-#define Z8536_CMD_SET_IUS (2 << 5)/* Set IUS */
-#define Z8536_CMD_CLR_IUS (3 << 5)/* Clear IUS */
-#define Z8536_CMD_SET_IP (4 << 5)/* Set IP */
-#define Z8536_CMD_CLR_IP (5 << 5)/* Clear IP */
-#define Z8536_CMD_SET_IE (6 << 5)/* Set IE */
-#define Z8536_CMD_CLR_IE (7 << 5)/* Clear IE */
-#define Z8536_CMD_MASK (7 << 5)
+#define Z8536_CMD(x) (((x) & 0x7) << 5)
+#define Z8536_CMD_NULL Z8536_CMD(0) /* Null Code */
+#define Z8536_CMD_CLR_IP_IUS Z8536_CMD(1) /* Clear IP & IUS */
+#define Z8536_CMD_SET_IUS Z8536_CMD(2) /* Set IUS */
+#define Z8536_CMD_CLR_IUS Z8536_CMD(3) /* Clear IUS */
+#define Z8536_CMD_SET_IP Z8536_CMD(4) /* Set IP */
+#define Z8536_CMD_CLR_IP Z8536_CMD(5) /* Clear IP */
+#define Z8536_CMD_SET_IE Z8536_CMD(6) /* Set IE */
+#define Z8536_CMD_CLR_IE Z8536_CMD(7) /* Clear IE */
+#define Z8536_CMD_MASK Z8536_CMD(7)
#define Z8536_STAT_IUS BIT(7) /* Interrupt Under Service */
#define Z8536_STAT_IE BIT(6) /* Interrupt Enable */
@@ -105,46 +107,51 @@
#define Z8536_CT_MODE_ETE BIT(4) /* External Trigger Enable */
#define Z8536_CT_MODE_EGE BIT(3) /* External Gate Enable */
#define Z8536_CT_MODE_REB BIT(2) /* Retrigger Enable Bit */
-#define Z8536_CT_MODE_DCS_PULSE (0 << 0)/* Duty Cycle - Pulse */
-#define Z8536_CT_MODE_DCS_ONESHOT (1 << 0)/* Duty Cycle - One-Shot */
-#define Z8536_CT_MODE_DCS_SQRWAVE (2 << 0)/* Duty Cycle - Square Wave */
-#define Z8536_CT_MODE_DCS_DO_NOT_USE (3 << 0)/* Duty Cycle - Do Not Use */
-#define Z8536_CT_MODE_DCS_MASK (3 << 0)/* Duty Cycle mask */
+#define Z8536_CT_MODE_DCS(x) (((x) & 0x3) << 0) /* Duty Cycle */
+#define Z8536_CT_MODE_DCS_PULSE Z8536_CT_MODE_DCS(0) /* Pulse */
+#define Z8536_CT_MODE_DCS_ONESHOT Z8536_CT_MODE_DCS(1) /* One-Shot */
+#define Z8536_CT_MODE_DCS_SQRWAVE Z8536_CT_MODE_DCS(2) /* Square Wave */
+#define Z8536_CT_MODE_DCS_DO_NOT_USE Z8536_CT_MODE_DCS(3) /* Do Not Use */
+#define Z8536_CT_MODE_DCS_MASK Z8536_CT_MODE_DCS(3)
/* Port A/B Mode Specification registers */
#define Z8536_PA_MODE_REG 0x20
#define Z8536_PB_MODE_REG 0x28
-#define Z8536_PAB_MODE_PTS_BIT (0 << 6)/* Bit Port */
-#define Z8536_PAB_MODE_PTS_INPUT (1 << 6)/* Input Port */
-#define Z8536_PAB_MODE_PTS_OUTPUT (2 << 6)/* Output Port */
-#define Z8536_PAB_MODE_PTS_BIDIR (3 << 6)/* Bidirectional Port */
-#define Z8536_PAB_MODE_PTS_MASK (3 << 6)/* Port Type Select mask */
+#define Z8536_PAB_MODE_PTS(x) (((x) & 0x3) << 6) /* Port type */
+#define Z8536_PAB_MODE_PTS_BIT Z8536_PAB_MODE_PTS(0 << 6)/* Bit */
+#define Z8536_PAB_MODE_PTS_INPUT Z8536_PAB_MODE_PTS(1 << 6)/* Input */
+#define Z8536_PAB_MODE_PTS_OUTPUT Z8536_PAB_MODE_PTS(2 << 6)/* Output */
+#define Z8536_PAB_MODE_PTS_BIDIR Z8536_PAB_MODE_PTS(3 << 6)/* Bidir */
+#define Z8536_PAB_MODE_PTS_MASK Z8536_PAB_MODE_PTS(3 << 6)
#define Z8536_PAB_MODE_ITB BIT(5) /* Interrupt on Two Bytes */
#define Z8536_PAB_MODE_SB BIT(4) /* Single Buffered mode */
#define Z8536_PAB_MODE_IMO BIT(3) /* Interrupt on Match Only */
-#define Z8536_PAB_MODE_PMS_DISABLE (0 << 1)/* Disable Pattern Match */
-#define Z8536_PAB_MODE_PMS_AND (1 << 1)/* "AND" mode */
-#define Z8536_PAB_MODE_PMS_OR (2 << 1)/* "OR" mode */
-#define Z8536_PAB_MODE_PMS_OR_PEV (3 << 1)/* "OR-Priority" mode */
-#define Z8536_PAB_MODE_PMS_MASK (3 << 1)/* Pattern Mode mask */
+#define Z8536_PAB_MODE_PMS(x) (((x) & 0x3) << 1) /* Pattern Mode */
+#define Z8536_PAB_MODE_PMS_DISABLE Z8536_PAB_MODE_PMS(0)/* Disabled */
+#define Z8536_PAB_MODE_PMS_AND Z8536_PAB_MODE_PMS(1)/* "AND" */
+#define Z8536_PAB_MODE_PMS_OR Z8536_PAB_MODE_PMS(2)/* "OR" */
+#define Z8536_PAB_MODE_PMS_OR_PEV Z8536_PAB_MODE_PMS(3)/* "OR-Priority" */
+#define Z8536_PAB_MODE_PMS_MASK Z8536_PAB_MODE_PMS(3)
#define Z8536_PAB_MODE_LPM BIT(0) /* Latch on Pattern Match */
#define Z8536_PAB_MODE_DTE BIT(0) /* Deskew Timer Enabled */
/* Port A/B Handshake Specification registers */
#define Z8536_PA_HANDSHAKE_REG 0x21
#define Z8536_PB_HANDSHAKE_REG 0x29
-#define Z8536_PAB_HANDSHAKE_HST_INTER (0 << 6)/* Interlocked Handshake */
-#define Z8536_PAB_HANDSHAKE_HST_STROBED (1 << 6)/* Strobed Handshake */
-#define Z8536_PAB_HANDSHAKE_HST_PULSED (2 << 6)/* Pulsed Handshake */
-#define Z8536_PAB_HANDSHAKE_HST_3WIRE (3 << 6)/* Three-Wire Handshake */
-#define Z8536_PAB_HANDSHAKE_HST_MASK (3 << 6)/* Handshake Type mask */
-#define Z8536_PAB_HANDSHAKE_RWS_DISABLE (0 << 3)/* Req/Wait Disabled */
-#define Z8536_PAB_HANDSHAKE_RWS_OUTWAIT (1 << 3)/* Output Wait */
-#define Z8536_PAB_HANDSHAKE_RWS_INWAIT (3 << 3)/* Input Wait */
-#define Z8536_PAB_HANDSHAKE_RWS_SPREQ (4 << 3)/* Special Request */
-#define Z8536_PAB_HANDSHAKE_RWS_OUTREQ (5 << 4)/* Output Request */
-#define Z8536_PAB_HANDSHAKE_RWS_INREQ (7 << 3)/* Input Request */
-#define Z8536_PAB_HANDSHAKE_RWS_MASK (7 << 3)/* Req/Wait mask */
+#define Z8536_PAB_HANDSHAKE_HST(x) (((x) & 0x3) << 6) /* Handshake Type */
+#define Z8536_PAB_HANDSHAKE_HST_INTER Z8536_PAB_HANDSHAKE_HST(0)/*Interlock*/
+#define Z8536_PAB_HANDSHAKE_HST_STROBED Z8536_PAB_HANDSHAKE_HST(1)/* Strobed */
+#define Z8536_PAB_HANDSHAKE_HST_PULSED Z8536_PAB_HANDSHAKE_HST(2)/* Pulsed */
+#define Z8536_PAB_HANDSHAKE_HST_3WIRE Z8536_PAB_HANDSHAKE_HST(3)/* 3-Wire */
+#define Z8536_PAB_HANDSHAKE_HST_MASK Z8536_PAB_HANDSHAKE_HST(3)
+#define Z8536_PAB_HANDSHAKE_RWS(x) (((x) & 0x7) << 3) /* Req/Wait */
+#define Z8536_PAB_HANDSHAKE_RWS_DISABLE Z8536_PAB_HANDSHAKE_RWS(0)/* Disabled */
+#define Z8536_PAB_HANDSHAKE_RWS_OUTWAIT Z8536_PAB_HANDSHAKE_RWS(1)/* Out Wait */
+#define Z8536_PAB_HANDSHAKE_RWS_INWAIT Z8536_PAB_HANDSHAKE_RWS(3)/* In Wait */
+#define Z8536_PAB_HANDSHAKE_RWS_SPREQ Z8536_PAB_HANDSHAKE_RWS(4)/* Special */
+#define Z8536_PAB_HANDSHAKE_RWS_OUTREQ Z8536_PAB_HANDSHAKE_RWS(5)/* Out Req */
+#define Z8536_PAB_HANDSHAKE_RWS_INREQ Z8536_PAB_HANDSHAKE_RWS(7)/* In Req */
+#define Z8536_PAB_HANDSHAKE_RWS_MASK Z8536_PAB_HANDSHAKE_RWS(7)
#define Z8536_PAB_HANDSHAKE_DESKEW(x) ((x) << 0)/* Deskew Time */
#define Z8536_PAB_HANDSHAKE_DESKEW_MASK (3 << 0)/* Deskew Time mask */
diff --git a/drivers/staging/dgnc/dgnc_cls.c b/drivers/staging/dgnc/dgnc_cls.c
index 0ff3139..46c050c 100644
--- a/drivers/staging/dgnc/dgnc_cls.c
+++ b/drivers/staging/dgnc/dgnc_cls.c
@@ -1168,7 +1168,7 @@
/* Clear out UART and FIFO */
readb(&ch->ch_cls_uart->txrx);
- writeb((UART_FCR_ENABLE_FIFO|UART_FCR_CLEAR_RCVR|UART_FCR_CLEAR_XMIT),
+ writeb(UART_FCR_ENABLE_FIFO | UART_FCR_CLEAR_RCVR | UART_FCR_CLEAR_XMIT,
&ch->ch_cls_uart->isr_fcr);
udelay(10);
diff --git a/drivers/staging/dgnc/dgnc_driver.c b/drivers/staging/dgnc/dgnc_driver.c
index 4eb410e..af2e835 100644
--- a/drivers/staging/dgnc/dgnc_driver.c
+++ b/drivers/staging/dgnc/dgnc_driver.c
@@ -48,7 +48,7 @@
/*
* File operations permitted on Control/Management major.
*/
-static const struct file_operations dgnc_BoardFops = {
+static const struct file_operations dgnc_board_fops = {
.owner = THIS_MODULE,
.unlocked_ioctl = dgnc_mgmt_ioctl,
.open = dgnc_mgmt_open,
@@ -58,11 +58,11 @@
/*
* Globals
*/
-uint dgnc_NumBoards;
-struct dgnc_board *dgnc_Board[MAXBOARDS];
+uint dgnc_num_boards;
+struct dgnc_board *dgnc_board[MAXBOARDS];
DEFINE_SPINLOCK(dgnc_global_lock);
DEFINE_SPINLOCK(dgnc_poll_lock); /* Poll scheduling lock */
-uint dgnc_Major;
+uint dgnc_major;
int dgnc_poll_tick = 20; /* Poll interval - 20 ms */
/*
@@ -92,7 +92,7 @@
unsigned int is_pci_express;
};
-static struct board_id dgnc_Ids[] = {
+static struct board_id dgnc_ids[] = {
{ PCI_DEVICE_CLASSIC_4_PCI_NAME, 4, 0 },
{ PCI_DEVICE_CLASSIC_4_422_PCI_NAME, 4, 0 },
{ PCI_DEVICE_CLASSIC_8_PCI_NAME, 8, 0 },
@@ -140,14 +140,14 @@
if (sysfiles)
dgnc_remove_driver_sysfiles(&dgnc_driver);
- device_destroy(dgnc_class, MKDEV(dgnc_Major, 0));
+ device_destroy(dgnc_class, MKDEV(dgnc_major, 0));
class_destroy(dgnc_class);
- unregister_chrdev(dgnc_Major, "dgnc");
+ unregister_chrdev(dgnc_major, "dgnc");
- for (i = 0; i < dgnc_NumBoards; ++i) {
- dgnc_remove_ports_sysfiles(dgnc_Board[i]);
- dgnc_tty_uninit(dgnc_Board[i]);
- dgnc_cleanup_board(dgnc_Board[i]);
+ for (i = 0; i < dgnc_num_boards; ++i) {
+ dgnc_remove_ports_sysfiles(dgnc_board[i]);
+ dgnc_tty_uninit(dgnc_board[i]);
+ dgnc_cleanup_board(dgnc_board[i]);
}
dgnc_tty_post_uninit();
@@ -217,12 +217,12 @@
*
* Register management/dpa devices
*/
- rc = register_chrdev(0, "dgnc", &dgnc_BoardFops);
+ rc = register_chrdev(0, "dgnc", &dgnc_board_fops);
if (rc < 0) {
pr_err(DRVSTR ": Can't register dgnc driver device (%d)\n", rc);
return rc;
}
- dgnc_Major = rc;
+ dgnc_major = rc;
dgnc_class = class_create(THIS_MODULE, "dgnc_mgmt");
if (IS_ERR(dgnc_class)) {
@@ -232,7 +232,7 @@
}
dev = device_create(dgnc_class, NULL,
- MKDEV(dgnc_Major, 0),
+ MKDEV(dgnc_major, 0),
NULL, "dgnc_mgmt");
if (IS_ERR(dev)) {
rc = PTR_ERR(dev);
@@ -262,11 +262,11 @@
return 0;
failed_tty:
- device_destroy(dgnc_class, MKDEV(dgnc_Major, 0));
+ device_destroy(dgnc_class, MKDEV(dgnc_major, 0));
failed_device:
class_destroy(dgnc_class);
failed_class:
- unregister_chrdev(dgnc_Major, "dgnc");
+ unregister_chrdev(dgnc_major, "dgnc");
return rc;
}
@@ -283,7 +283,7 @@
rc = dgnc_found_board(pdev, ent->driver_data);
if (rc == 0)
- dgnc_NumBoards++;
+ dgnc_num_boards++;
return rc;
}
@@ -346,7 +346,7 @@
}
}
- dgnc_Board[brd->boardnum] = NULL;
+ dgnc_board[brd->boardnum] = NULL;
kfree(brd);
}
@@ -365,8 +365,8 @@
unsigned long flags;
/* get the board structure and prep it */
- dgnc_Board[dgnc_NumBoards] = kzalloc(sizeof(*brd), GFP_KERNEL);
- brd = dgnc_Board[dgnc_NumBoards];
+ dgnc_board[dgnc_num_boards] = kzalloc(sizeof(*brd), GFP_KERNEL);
+ brd = dgnc_board[dgnc_num_boards];
if (!brd)
return -ENOMEM;
@@ -382,15 +382,15 @@
/* store the info for the board we've found */
brd->magic = DGNC_BOARD_MAGIC;
- brd->boardnum = dgnc_NumBoards;
+ brd->boardnum = dgnc_num_boards;
brd->vendor = dgnc_pci_tbl[id].vendor;
brd->device = dgnc_pci_tbl[id].device;
brd->pdev = pdev;
brd->pci_bus = pdev->bus->number;
brd->pci_slot = PCI_SLOT(pdev->devfn);
- brd->name = dgnc_Ids[id].name;
- brd->maxports = dgnc_Ids[id].maxports;
- if (dgnc_Ids[i].is_pci_express)
+ brd->name = dgnc_ids[id].name;
+ brd->maxports = dgnc_ids[id].maxports;
+ if (dgnc_ids[i].is_pci_express)
brd->bd_flags |= BD_IS_PCI_EXPRESS;
brd->dpastatus = BD_NOFEP;
init_waitqueue_head(&brd->state_wait);
@@ -642,8 +642,8 @@
unsigned long new_time;
/* Go thru each board, kicking off a tasklet for each if needed */
- for (i = 0; i < dgnc_NumBoards; i++) {
- brd = dgnc_Board[i];
+ for (i = 0; i < dgnc_num_boards; i++) {
+ brd = dgnc_board[i];
spin_lock_irqsave(&brd->bd_lock, flags);
diff --git a/drivers/staging/dgnc/dgnc_driver.h b/drivers/staging/dgnc/dgnc_driver.h
index e4be81b..95ec729 100644
--- a/drivers/staging/dgnc/dgnc_driver.h
+++ b/drivers/staging/dgnc/dgnc_driver.h
@@ -202,18 +202,13 @@
* to our channels.
*/
- struct tty_driver SerialDriver;
- char SerialName[200];
- struct tty_driver PrintDriver;
- char PrintName[200];
+ struct tty_driver *serial_driver;
+ char serial_name[200];
+ struct tty_driver *print_driver;
+ char print_name[200];
- bool dgnc_Major_Serial_Registered;
- bool dgnc_Major_TransparentPrint_Registered;
-
- uint dgnc_Serial_Major;
- uint dgnc_TransparentPrint_Major;
-
- uint TtyRefCnt;
+ bool dgnc_major_serial_registered;
+ bool dgnc_major_transparent_print_registered;
u16 dpatype; /* The board "type",
* as defined by DPA
@@ -399,12 +394,12 @@
/*
* Our Global Variables.
*/
-extern uint dgnc_Major; /* Our driver/mgmt major */
+extern uint dgnc_major; /* Our driver/mgmt major */
extern int dgnc_poll_tick; /* Poll interval - 20 ms */
extern spinlock_t dgnc_global_lock; /* Driver global spinlock */
extern spinlock_t dgnc_poll_lock; /* Poll scheduling lock */
-extern uint dgnc_NumBoards; /* Total number of boards */
-extern struct dgnc_board *dgnc_Board[MAXBOARDS]; /* Array of board
+extern uint dgnc_num_boards; /* Total number of boards */
+extern struct dgnc_board *dgnc_board[MAXBOARDS]; /* Array of board
* structs
*/
diff --git a/drivers/staging/dgnc/dgnc_mgmt.c b/drivers/staging/dgnc/dgnc_mgmt.c
index ba29a8d..683c098 100644
--- a/drivers/staging/dgnc/dgnc_mgmt.c
+++ b/drivers/staging/dgnc/dgnc_mgmt.c
@@ -111,7 +111,7 @@
spin_lock_irqsave(&dgnc_global_lock, flags);
memset(&ddi, 0, sizeof(ddi));
- ddi.dinfo_nboards = dgnc_NumBoards;
+ ddi.dinfo_nboards = dgnc_num_boards;
sprintf(ddi.dinfo_version, "%s", DG_PART);
spin_unlock_irqrestore(&dgnc_global_lock, flags);
@@ -131,27 +131,27 @@
if (copy_from_user(&brd, uarg, sizeof(int)))
return -EFAULT;
- if (brd < 0 || brd >= dgnc_NumBoards)
+ if (brd < 0 || brd >= dgnc_num_boards)
return -ENODEV;
memset(&di, 0, sizeof(di));
di.info_bdnum = brd;
- spin_lock_irqsave(&dgnc_Board[brd]->bd_lock, flags);
+ spin_lock_irqsave(&dgnc_board[brd]->bd_lock, flags);
- di.info_bdtype = dgnc_Board[brd]->dpatype;
- di.info_bdstate = dgnc_Board[brd]->dpastatus;
+ di.info_bdtype = dgnc_board[brd]->dpatype;
+ di.info_bdstate = dgnc_board[brd]->dpastatus;
di.info_ioport = 0;
- di.info_physaddr = (ulong)dgnc_Board[brd]->membase;
- di.info_physsize = (ulong)dgnc_Board[brd]->membase
- - dgnc_Board[brd]->membase_end;
- if (dgnc_Board[brd]->state != BOARD_FAILED)
- di.info_nports = dgnc_Board[brd]->nasync;
+ di.info_physaddr = (ulong)dgnc_board[brd]->membase;
+ di.info_physsize = (ulong)dgnc_board[brd]->membase
+ - dgnc_board[brd]->membase_end;
+ if (dgnc_board[brd]->state != BOARD_FAILED)
+ di.info_nports = dgnc_board[brd]->nasync;
else
di.info_nports = 0;
- spin_unlock_irqrestore(&dgnc_Board[brd]->bd_lock, flags);
+ spin_unlock_irqrestore(&dgnc_board[brd]->bd_lock, flags);
if (copy_to_user(uarg, &di, sizeof(di)))
return -EFAULT;
@@ -174,14 +174,14 @@
channel = ni.channel;
/* Verify boundaries on board */
- if (board >= dgnc_NumBoards)
+ if (board >= dgnc_num_boards)
return -ENODEV;
/* Verify boundaries on channel */
- if (channel >= dgnc_Board[board]->nasync)
+ if (channel >= dgnc_board[board]->nasync)
return -ENODEV;
- ch = dgnc_Board[board]->channels[channel];
+ ch = dgnc_board[board]->channels[channel];
if (!ch || ch->magic != DGNC_CHANNEL_MAGIC)
return -ENODEV;
diff --git a/drivers/staging/dgnc/dgnc_neo.c b/drivers/staging/dgnc/dgnc_neo.c
index 31ac437..ba57e95 100644
--- a/drivers/staging/dgnc/dgnc_neo.c
+++ b/drivers/staging/dgnc/dgnc_neo.c
@@ -77,8 +77,6 @@
.send_immediate_char = neo_send_immediate_char
};
-static uint dgnc_offset_table[8] = { 0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40, 0x80 };
-
/*
* This function allows calls to ensure that all outstanding
* PCI writes have been completed, by doing a PCI read against
@@ -116,7 +114,8 @@
writeb(efr, &ch->ch_neo_uart->efr);
/* Turn on table D, with 8 char hi/low watermarks */
- writeb((UART_17158_FCTR_TRGD | UART_17158_FCTR_RTS_4DELAY), &ch->ch_neo_uart->fctr);
+ writeb(UART_17158_FCTR_TRGD | UART_17158_FCTR_RTS_4DELAY,
+ &ch->ch_neo_uart->fctr);
/* Feed the UART our trigger levels */
writeb(8, &ch->ch_neo_uart->tfifo);
@@ -150,7 +149,8 @@
/* Turn on UART enhanced bits */
writeb(efr, &ch->ch_neo_uart->efr);
- writeb((UART_17158_FCTR_TRGD | UART_17158_FCTR_RTS_4DELAY), &ch->ch_neo_uart->fctr);
+ writeb(UART_17158_FCTR_TRGD | UART_17158_FCTR_RTS_4DELAY,
+ &ch->ch_neo_uart->fctr);
ch->ch_r_watermark = 4;
writeb(32, &ch->ch_neo_uart->rfifo);
@@ -187,7 +187,8 @@
/* Turn on UART enhanced bits */
writeb(efr, &ch->ch_neo_uart->efr);
- writeb((UART_17158_FCTR_TRGD | UART_17158_FCTR_RTS_8DELAY), &ch->ch_neo_uart->fctr);
+ writeb(UART_17158_FCTR_TRGD | UART_17158_FCTR_RTS_8DELAY,
+ &ch->ch_neo_uart->fctr);
ch->ch_r_watermark = 4;
writeb(32, &ch->ch_neo_uart->rfifo);
@@ -225,7 +226,8 @@
writeb(efr, &ch->ch_neo_uart->efr);
/* Turn on table D, with 8 char hi/low watermarks */
- writeb((UART_17158_FCTR_TRGD | UART_17158_FCTR_RTS_8DELAY), &ch->ch_neo_uart->fctr);
+ writeb(UART_17158_FCTR_TRGD | UART_17158_FCTR_RTS_8DELAY,
+ &ch->ch_neo_uart->fctr);
writeb(8, &ch->ch_neo_uart->tfifo);
ch->ch_t_tlevel = 8;
@@ -265,7 +267,8 @@
writeb(efr, &ch->ch_neo_uart->efr);
/* Turn on table D, with 8 char hi/low watermarks */
- writeb((UART_17158_FCTR_TRGD | UART_17158_FCTR_RTS_8DELAY), &ch->ch_neo_uart->fctr);
+ writeb(UART_17158_FCTR_TRGD | UART_17158_FCTR_RTS_8DELAY,
+ &ch->ch_neo_uart->fctr);
ch->ch_r_watermark = 0;
@@ -302,7 +305,8 @@
writeb(efr, &ch->ch_neo_uart->efr);
/* Turn on table D, with 8 char hi/low watermarks */
- writeb((UART_17158_FCTR_TRGD | UART_17158_FCTR_RTS_8DELAY), &ch->ch_neo_uart->fctr);
+ writeb(UART_17158_FCTR_TRGD | UART_17158_FCTR_RTS_8DELAY,
+ &ch->ch_neo_uart->fctr);
ch->ch_r_watermark = 0;
@@ -321,7 +325,8 @@
static inline void neo_set_new_start_stop_chars(struct channel_t *ch)
{
/* if hardware flow control is set, then skip this whole thing */
- if (ch->ch_digi.digi_flags & (CTSPACE | RTSPACE) || ch->ch_c_cflag & CRTSCTS)
+ if (ch->ch_digi.digi_flags & (CTSPACE | RTSPACE) ||
+ ch->ch_c_cflag & CRTSCTS)
return;
/* Tell UART what start/stop chars it should be looking for */
@@ -351,8 +356,8 @@
/* Turn break off, and unset some variables */
if (ch->ch_flags & CH_BREAK_SENDING) {
- if (time_after_eq(jiffies, ch->ch_stop_sending_break)
- || force) {
+ if (force ||
+ time_after_eq(jiffies, ch->ch_stop_sending_break)) {
unsigned char temp = readb(&ch->ch_neo_uart->lcr);
writeb((temp & ~UART_LCR_SBC), &ch->ch_neo_uart->lcr);
@@ -374,14 +379,8 @@
unsigned char cause;
unsigned long flags;
- if (!brd || brd->magic != DGNC_BOARD_MAGIC)
- return;
-
- if (port >= brd->maxports)
- return;
-
ch = brd->channels[port];
- if (ch->magic != DGNC_CHANNEL_MAGIC)
+ if (!ch || ch->magic != DGNC_CHANNEL_MAGIC)
return;
/* Here we try to figure out what caused the interrupt to happen */
@@ -393,7 +392,8 @@
break;
/*
- * Yank off the upper 2 bits, which just show that the FIFO's are enabled.
+ * Yank off the upper 2 bits,
+ * which just show that the FIFO's are enabled.
*/
isr &= ~(UART_17158_IIR_FIFO_ENABLED);
@@ -666,7 +666,8 @@
};
/* Only use the TXPrint baud rate if the terminal unit is NOT open */
- if (!(ch->ch_tun.un_flags & UN_ISOPEN) && (un->un_type == DGNC_PRINT))
+ if (!(ch->ch_tun.un_flags & UN_ISOPEN) &&
+ (un->un_type == DGNC_PRINT))
baud = C_BAUD(ch->ch_pun.un_tty) & 0xff;
else
baud = C_BAUD(ch->ch_tun.un_tty) & 0xff;
@@ -679,7 +680,8 @@
jindex = baud;
- if ((iindex >= 0) && (iindex < 4) && (jindex >= 0) && (jindex < 16))
+ if ((iindex >= 0) && (iindex < 4) &&
+ (jindex >= 0) && (jindex < 16))
baud = bauds[iindex][jindex];
else
baud = 0;
@@ -787,7 +789,8 @@
neo_set_cts_flow_control(ch);
} else if (ch->ch_c_iflag & IXON) {
/* If start/stop is set to disable, then we should disable flow control */
- if ((ch->ch_startc == _POSIX_VDISABLE) || (ch->ch_stopc == _POSIX_VDISABLE))
+ if ((ch->ch_startc == _POSIX_VDISABLE) ||
+ (ch->ch_stopc == _POSIX_VDISABLE))
neo_set_no_output_flow_control(ch);
else
neo_set_ixon_flow_control(ch);
@@ -799,7 +802,8 @@
neo_set_rts_flow_control(ch);
} else if (ch->ch_c_iflag & IXOFF) {
/* If start/stop is set to disable, then we should disable flow control */
- if ((ch->ch_startc == _POSIX_VDISABLE) || (ch->ch_stopc == _POSIX_VDISABLE))
+ if ((ch->ch_startc == _POSIX_VDISABLE) ||
+ (ch->ch_stopc == _POSIX_VDISABLE))
neo_set_no_input_flow_control(ch);
else
neo_set_ixoff_flow_control(ch);
@@ -910,9 +914,7 @@
struct dgnc_board *brd = voidbrd;
struct channel_t *ch;
int port = 0;
- int type = 0;
- int current_port;
- u32 tmp;
+ int type;
u32 uart_poll;
unsigned long flags;
unsigned long flags2;
@@ -947,29 +949,12 @@
/* At this point, we have at least SOMETHING to service, dig further... */
- current_port = 0;
-
/* Loop on each port */
while ((uart_poll & 0xff) != 0) {
- tmp = uart_poll;
+ type = uart_poll >> (8 + (port * 3));
+ type &= 0x7;
- /* Check current port to see if it has interrupt pending */
- if ((tmp & dgnc_offset_table[current_port]) != 0) {
- port = current_port;
- type = tmp >> (8 + (port * 3));
- type &= 0x7;
- } else {
- current_port++;
- continue;
- }
-
- /* Remove this port + type from uart_poll */
- uart_poll &= ~(dgnc_offset_table[port]);
-
- if (!type) {
- /* If no type, just ignore it, and move onto next port */
- continue;
- }
+ uart_poll &= ~(0x01 << port);
/* Switch on type of interrupt we have */
switch (type) {
@@ -981,7 +966,7 @@
/* Verify the port is in range. */
if (port >= brd->nasync)
- continue;
+ break;
ch = brd->channels[port];
neo_copy_data_from_uart_to_queue(ch);
@@ -991,14 +976,14 @@
dgnc_check_queue_flow_control(ch);
spin_unlock_irqrestore(&ch->ch_lock, flags2);
- continue;
+ break;
case UART_17158_RX_LINE_STATUS:
/*
* RXRDY and RX LINE Status (logic OR of LSR[4:1])
*/
neo_parse_lsr(brd, port);
- continue;
+ break;
case UART_17158_TXRDY:
/*
@@ -1014,14 +999,14 @@
* it should be, I was getting things like RXDY too. Weird.
*/
neo_parse_isr(brd, port);
- continue;
+ break;
case UART_17158_MSR:
/*
* MSR or flow control was seen.
*/
neo_parse_isr(brd, port);
- continue;
+ break;
default:
/*
@@ -1030,8 +1015,10 @@
* these once and awhile.
* Its harmless, just ignore it and move on.
*/
- continue;
+ break;
}
+
+ port++;
}
/*
@@ -1172,7 +1159,8 @@
linestatus = 0;
/* Copy data from uart to the queue */
- memcpy_fromio(ch->ch_rqueue + head, &ch->ch_neo_uart->txrxburst, n);
+ memcpy_fromio(ch->ch_rqueue + head,
+ &ch->ch_neo_uart->txrxburst, n);
/*
* Since RX_FIFO_DATA_ERROR was 0, we are guaranteed
@@ -1225,7 +1213,8 @@
* we don't miss our TX FIFO emptys.
*/
if (linestatus & (UART_LSR_THRE | UART_17158_TX_AND_FIFO_CLR)) {
- linestatus &= ~(UART_LSR_THRE | UART_17158_TX_AND_FIFO_CLR);
+ linestatus &= ~(UART_LSR_THRE |
+ UART_17158_TX_AND_FIFO_CLR);
ch->ch_flags |= (CH_TX_FIFO_EMPTY | CH_TX_FIFO_LWM);
}
@@ -1255,7 +1244,8 @@
qleft++;
}
- memcpy_fromio(ch->ch_rqueue + head, &ch->ch_neo_uart->txrxburst, 1);
+ memcpy_fromio(ch->ch_rqueue + head,
+ &ch->ch_neo_uart->txrxburst, 1);
ch->ch_equeue[head] = (unsigned char)linestatus;
/* Ditch any remaining linestatus value. */
@@ -1328,7 +1318,8 @@
if (!ch || ch->magic != DGNC_CHANNEL_MAGIC)
return;
- writeb((UART_FCR_ENABLE_FIFO | UART_FCR_CLEAR_XMIT), &ch->ch_neo_uart->isr_fcr);
+ writeb((UART_FCR_ENABLE_FIFO | UART_FCR_CLEAR_XMIT),
+ &ch->ch_neo_uart->isr_fcr);
neo_pci_posting_flush(ch->ch_bd);
for (i = 0; i < 10; i++) {
@@ -1356,7 +1347,8 @@
if (!ch || ch->magic != DGNC_CHANNEL_MAGIC)
return;
- writeb((UART_FCR_ENABLE_FIFO | UART_FCR_CLEAR_RCVR), &ch->ch_neo_uart->isr_fcr);
+ writeb(UART_FCR_ENABLE_FIFO | UART_FCR_CLEAR_RCVR,
+ &ch->ch_neo_uart->isr_fcr);
neo_pci_posting_flush(ch->ch_bd);
for (i = 0; i < 10; i++) {
@@ -1427,7 +1419,8 @@
ch->ch_tun.un_flags |= (UN_EMPTY);
}
- writeb(ch->ch_wqueue[ch->ch_w_tail], &ch->ch_neo_uart->txrx);
+ writeb(ch->ch_wqueue[ch->ch_w_tail],
+ &ch->ch_neo_uart->txrx);
ch->ch_w_tail++;
ch->ch_w_tail &= WQUEUEMASK;
ch->ch_txcount++;
@@ -1494,7 +1487,8 @@
ch->ch_tun.un_flags |= (UN_EMPTY);
}
- memcpy_toio(&ch->ch_neo_uart->txrxburst, ch->ch_wqueue + tail, s);
+ memcpy_toio(&ch->ch_neo_uart->txrxburst,
+ ch->ch_wqueue + tail, s);
/* Add and flip queue if needed */
tail = (tail + s) & WQUEUEMASK;
@@ -1628,7 +1622,8 @@
/* Clear out UART and FIFO */
readb(&ch->ch_neo_uart->txrx);
- writeb((UART_FCR_ENABLE_FIFO | UART_FCR_CLEAR_RCVR | UART_FCR_CLEAR_XMIT), &ch->ch_neo_uart->isr_fcr);
+ writeb(UART_FCR_ENABLE_FIFO | UART_FCR_CLEAR_RCVR | UART_FCR_CLEAR_XMIT,
+ &ch->ch_neo_uart->isr_fcr);
readb(&ch->ch_neo_uart->lsr);
readb(&ch->ch_neo_uart->msr);
@@ -1725,7 +1720,8 @@
neo_pci_posting_flush(ch->ch_bd);
}
-static unsigned int neo_read_eeprom(unsigned char __iomem *base, unsigned int address)
+static unsigned int neo_read_eeprom(unsigned char __iomem *base,
+ unsigned int address)
{
unsigned int enable;
unsigned int bits;
@@ -1783,10 +1779,15 @@
brd->vpd[(i * 2) + 1] = (a >> 8) & 0xff;
}
- if (((brd->vpd[0x08] != 0x82) /* long resource name tag */
- && (brd->vpd[0x10] != 0x82)) /* long resource name tag (PCI-66 files)*/
- || (brd->vpd[0x7F] != 0x78)) { /* small resource end tag */
-
+ /*
+ * brd->vpd has different name tags by below index.
+ * 0x08 : long resource name tag
+ * 0x10 : long resource name tage (PCI-66 files)
+ * 0x7F : small resource end tag
+ */
+ if (((brd->vpd[0x08] != 0x82) &&
+ (brd->vpd[0x10] != 0x82)) ||
+ (brd->vpd[0x7F] != 0x78)) {
memset(brd->vpd, '\0', NEO_VPD_IMAGESIZE);
} else {
/* Search for the serial number */
diff --git a/drivers/staging/dgnc/dgnc_sysfs.c b/drivers/staging/dgnc/dgnc_sysfs.c
index 74a0725..b8d41c5 100644
--- a/drivers/staging/dgnc/dgnc_sysfs.c
+++ b/drivers/staging/dgnc/dgnc_sysfs.c
@@ -33,7 +33,7 @@
static ssize_t dgnc_driver_boards_show(struct device_driver *ddp, char *buf)
{
- return snprintf(buf, PAGE_SIZE, "%d\n", dgnc_NumBoards);
+ return snprintf(buf, PAGE_SIZE, "%d\n", dgnc_num_boards);
}
static DRIVER_ATTR(boards, S_IRUSR, dgnc_driver_boards_show, NULL);
@@ -189,19 +189,21 @@
DGNC_VERIFY_BOARD(p, bd);
for (i = 0; i < bd->nasync; i++) {
- if (bd->channels[i]->ch_open_count) {
+ struct channel_t *ch = bd->channels[i];
+
+ if (ch->ch_open_count) {
count += snprintf(buf + count, PAGE_SIZE - count,
"%d %s %s %s %s %s %s\n",
- bd->channels[i]->ch_portnum,
- (bd->channels[i]->ch_mostat & UART_MCR_RTS) ? "RTS" : "",
- (bd->channels[i]->ch_mistat & UART_MSR_CTS) ? "CTS" : "",
- (bd->channels[i]->ch_mostat & UART_MCR_DTR) ? "DTR" : "",
- (bd->channels[i]->ch_mistat & UART_MSR_DSR) ? "DSR" : "",
- (bd->channels[i]->ch_mistat & UART_MSR_DCD) ? "DCD" : "",
- (bd->channels[i]->ch_mistat & UART_MSR_RI) ? "RI" : "");
+ ch->ch_portnum,
+ (ch->ch_mostat & UART_MCR_RTS) ? "RTS" : "",
+ (ch->ch_mistat & UART_MSR_CTS) ? "CTS" : "",
+ (ch->ch_mostat & UART_MCR_DTR) ? "DTR" : "",
+ (ch->ch_mistat & UART_MSR_DSR) ? "DSR" : "",
+ (ch->ch_mistat & UART_MSR_DCD) ? "DCD" : "",
+ (ch->ch_mistat & UART_MSR_RI) ? "RI" : "");
} else {
count += snprintf(buf + count, PAGE_SIZE - count,
- "%d\n", bd->channels[i]->ch_portnum);
+ "%d\n", ch->ch_portnum);
}
}
return count;
diff --git a/drivers/staging/dgnc/dgnc_tty.c b/drivers/staging/dgnc/dgnc_tty.c
index bcd2bdf..4eeecc9 100644
--- a/drivers/staging/dgnc/dgnc_tty.c
+++ b/drivers/staging/dgnc/dgnc_tty.c
@@ -176,57 +176,42 @@
*/
int dgnc_tty_register(struct dgnc_board *brd)
{
- int rc = 0;
+ int rc;
- brd->SerialDriver.magic = TTY_DRIVER_MAGIC;
+ brd->serial_driver = tty_alloc_driver(brd->maxports,
+ TTY_DRIVER_REAL_RAW |
+ TTY_DRIVER_DYNAMIC_DEV |
+ TTY_DRIVER_HARDWARE_BREAK);
- snprintf(brd->SerialName, MAXTTYNAMELEN, "tty_dgnc_%d_", brd->boardnum);
+ if (IS_ERR(brd->serial_driver))
+ return PTR_ERR(brd->serial_driver);
- brd->SerialDriver.name = brd->SerialName;
- brd->SerialDriver.name_base = 0;
- brd->SerialDriver.major = 0;
- brd->SerialDriver.minor_start = 0;
- brd->SerialDriver.num = brd->maxports;
- brd->SerialDriver.type = TTY_DRIVER_TYPE_SERIAL;
- brd->SerialDriver.subtype = SERIAL_TYPE_NORMAL;
- brd->SerialDriver.init_termios = DgncDefaultTermios;
- brd->SerialDriver.driver_name = DRVSTR;
- brd->SerialDriver.flags = (TTY_DRIVER_REAL_RAW |
- TTY_DRIVER_DYNAMIC_DEV |
- TTY_DRIVER_HARDWARE_BREAK);
+ snprintf(brd->serial_name, MAXTTYNAMELEN, "tty_dgnc_%d_", brd->boardnum);
- /*
- * The kernel wants space to store pointers to
- * tty_struct's and termios's.
- */
- brd->SerialDriver.ttys = kcalloc(brd->maxports,
- sizeof(*brd->SerialDriver.ttys),
- GFP_KERNEL);
- if (!brd->SerialDriver.ttys)
- return -ENOMEM;
-
- kref_init(&brd->SerialDriver.kref);
- brd->SerialDriver.termios = kcalloc(brd->maxports,
- sizeof(*brd->SerialDriver.termios),
- GFP_KERNEL);
- if (!brd->SerialDriver.termios)
- return -ENOMEM;
+ brd->serial_driver->name = brd->serial_name;
+ brd->serial_driver->name_base = 0;
+ brd->serial_driver->major = 0;
+ brd->serial_driver->minor_start = 0;
+ brd->serial_driver->type = TTY_DRIVER_TYPE_SERIAL;
+ brd->serial_driver->subtype = SERIAL_TYPE_NORMAL;
+ brd->serial_driver->init_termios = DgncDefaultTermios;
+ brd->serial_driver->driver_name = DRVSTR;
/*
* Entry points for driver. Called by the kernel from
* tty_io.c and n_tty.c.
*/
- tty_set_operations(&brd->SerialDriver, &dgnc_tty_ops);
+ tty_set_operations(brd->serial_driver, &dgnc_tty_ops);
- if (!brd->dgnc_Major_Serial_Registered) {
+ if (!brd->dgnc_major_serial_registered) {
/* Register tty devices */
- rc = tty_register_driver(&brd->SerialDriver);
+ rc = tty_register_driver(brd->serial_driver);
if (rc < 0) {
dev_dbg(&brd->pdev->dev,
"Can't register tty device (%d)\n", rc);
- return rc;
+ goto free_serial_driver;
}
- brd->dgnc_Major_Serial_Registered = true;
+ brd->dgnc_major_serial_registered = true;
}
/*
@@ -234,60 +219,55 @@
* again, separately so we don't get the LD confused about what major
* we are when we get into the dgnc_tty_open() routine.
*/
- brd->PrintDriver.magic = TTY_DRIVER_MAGIC;
- snprintf(brd->PrintName, MAXTTYNAMELEN, "pr_dgnc_%d_", brd->boardnum);
+ brd->print_driver = tty_alloc_driver(brd->maxports,
+ TTY_DRIVER_REAL_RAW |
+ TTY_DRIVER_DYNAMIC_DEV |
+ TTY_DRIVER_HARDWARE_BREAK);
- brd->PrintDriver.name = brd->PrintName;
- brd->PrintDriver.name_base = 0;
- brd->PrintDriver.major = brd->SerialDriver.major;
- brd->PrintDriver.minor_start = 0x80;
- brd->PrintDriver.num = brd->maxports;
- brd->PrintDriver.type = TTY_DRIVER_TYPE_SERIAL;
- brd->PrintDriver.subtype = SERIAL_TYPE_NORMAL;
- brd->PrintDriver.init_termios = DgncDefaultTermios;
- brd->PrintDriver.driver_name = DRVSTR;
- brd->PrintDriver.flags = (TTY_DRIVER_REAL_RAW |
- TTY_DRIVER_DYNAMIC_DEV |
- TTY_DRIVER_HARDWARE_BREAK);
+ if (IS_ERR(brd->print_driver)) {
+ rc = PTR_ERR(brd->print_driver);
+ goto unregister_serial_driver;
+ }
- /*
- * The kernel wants space to store pointers to
- * tty_struct's and termios's. Must be separated from
- * the Serial Driver so we don't get confused
- */
- brd->PrintDriver.ttys = kcalloc(brd->maxports,
- sizeof(*brd->PrintDriver.ttys),
- GFP_KERNEL);
- if (!brd->PrintDriver.ttys)
- return -ENOMEM;
- kref_init(&brd->PrintDriver.kref);
- brd->PrintDriver.termios = kcalloc(brd->maxports,
- sizeof(*brd->PrintDriver.termios),
- GFP_KERNEL);
- if (!brd->PrintDriver.termios)
- return -ENOMEM;
+ snprintf(brd->print_name, MAXTTYNAMELEN, "pr_dgnc_%d_", brd->boardnum);
+
+ brd->print_driver->name = brd->print_name;
+ brd->print_driver->name_base = 0;
+ brd->print_driver->major = brd->serial_driver->major;
+ brd->print_driver->minor_start = 0x80;
+ brd->print_driver->type = TTY_DRIVER_TYPE_SERIAL;
+ brd->print_driver->subtype = SERIAL_TYPE_NORMAL;
+ brd->print_driver->init_termios = DgncDefaultTermios;
+ brd->print_driver->driver_name = DRVSTR;
/*
* Entry points for driver. Called by the kernel from
* tty_io.c and n_tty.c.
*/
- tty_set_operations(&brd->PrintDriver, &dgnc_tty_ops);
+ tty_set_operations(brd->print_driver, &dgnc_tty_ops);
- if (!brd->dgnc_Major_TransparentPrint_Registered) {
+ if (!brd->dgnc_major_transparent_print_registered) {
/* Register Transparent Print devices */
- rc = tty_register_driver(&brd->PrintDriver);
+ rc = tty_register_driver(brd->print_driver);
if (rc < 0) {
dev_dbg(&brd->pdev->dev,
"Can't register Transparent Print device(%d)\n",
rc);
- return rc;
+ goto free_print_driver;
}
- brd->dgnc_Major_TransparentPrint_Registered = true;
+ brd->dgnc_major_transparent_print_registered = true;
}
- dgnc_BoardsByMajor[brd->SerialDriver.major] = brd;
- brd->dgnc_Serial_Major = brd->SerialDriver.major;
- brd->dgnc_TransparentPrint_Major = brd->PrintDriver.major;
+ dgnc_BoardsByMajor[brd->serial_driver->major] = brd;
+
+ return 0;
+
+free_print_driver:
+ put_tty_driver(brd->print_driver);
+unregister_serial_driver:
+ tty_unregister_driver(brd->serial_driver);
+free_serial_driver:
+ put_tty_driver(brd->serial_driver);
return rc;
}
@@ -364,12 +344,12 @@
{
struct device *classp;
- classp = tty_register_device(&brd->SerialDriver, i,
+ classp = tty_register_device(brd->serial_driver, i,
&ch->ch_bd->pdev->dev);
ch->ch_tun.un_sysfs = classp;
dgnc_create_tty_sysfs(&ch->ch_tun, classp);
- classp = tty_register_device(&brd->PrintDriver, i,
+ classp = tty_register_device(brd->print_driver, i,
&ch->ch_bd->pdev->dev);
ch->ch_pun.un_sysfs = classp;
dgnc_create_tty_sysfs(&ch->ch_pun, classp);
@@ -407,40 +387,32 @@
{
int i = 0;
- if (brd->dgnc_Major_Serial_Registered) {
- dgnc_BoardsByMajor[brd->SerialDriver.major] = NULL;
- brd->dgnc_Serial_Major = 0;
+ if (brd->dgnc_major_serial_registered) {
+ dgnc_BoardsByMajor[brd->serial_driver->major] = NULL;
for (i = 0; i < brd->nasync; i++) {
if (brd->channels[i])
dgnc_remove_tty_sysfs(brd->channels[i]->
ch_tun.un_sysfs);
- tty_unregister_device(&brd->SerialDriver, i);
+ tty_unregister_device(brd->serial_driver, i);
}
- tty_unregister_driver(&brd->SerialDriver);
- brd->dgnc_Major_Serial_Registered = false;
+ tty_unregister_driver(brd->serial_driver);
+ brd->dgnc_major_serial_registered = false;
}
- if (brd->dgnc_Major_TransparentPrint_Registered) {
- dgnc_BoardsByMajor[brd->PrintDriver.major] = NULL;
- brd->dgnc_TransparentPrint_Major = 0;
+ if (brd->dgnc_major_transparent_print_registered) {
+ dgnc_BoardsByMajor[brd->print_driver->major] = NULL;
for (i = 0; i < brd->nasync; i++) {
if (brd->channels[i])
dgnc_remove_tty_sysfs(brd->channels[i]->
ch_pun.un_sysfs);
- tty_unregister_device(&brd->PrintDriver, i);
+ tty_unregister_device(brd->print_driver, i);
}
- tty_unregister_driver(&brd->PrintDriver);
- brd->dgnc_Major_TransparentPrint_Registered = false;
+ tty_unregister_driver(brd->print_driver);
+ brd->dgnc_major_transparent_print_registered = false;
}
- kfree(brd->SerialDriver.ttys);
- brd->SerialDriver.ttys = NULL;
- kfree(brd->SerialDriver.termios);
- brd->SerialDriver.termios = NULL;
- kfree(brd->PrintDriver.ttys);
- brd->PrintDriver.ttys = NULL;
- kfree(brd->PrintDriver.termios);
- brd->PrintDriver.termios = NULL;
+ put_tty_driver(brd->serial_driver);
+ put_tty_driver(brd->print_driver);
}
/*
@@ -606,6 +578,8 @@
* or the amount of data the card actually has pending...
*/
while (n) {
+ unsigned char *ch_pos = ch->ch_equeue + tail;
+
s = ((head >= tail) ? head : RQUEUESIZE) - tail;
s = min(s, n);
@@ -620,29 +594,20 @@
*/
if (I_PARMRK(tp) || I_BRKINT(tp) || I_INPCK(tp)) {
for (i = 0; i < s; i++) {
- if (*(ch->ch_equeue + tail + i) & UART_LSR_BI)
- tty_insert_flip_char(tp->port,
- *(ch->ch_rqueue + tail + i),
- TTY_BREAK);
- else if (*(ch->ch_equeue + tail + i) &
- UART_LSR_PE)
- tty_insert_flip_char(tp->port,
- *(ch->ch_rqueue + tail + i),
- TTY_PARITY);
- else if (*(ch->ch_equeue + tail + i) &
- UART_LSR_FE)
- tty_insert_flip_char(tp->port,
- *(ch->ch_rqueue + tail + i),
- TTY_FRAME);
- else
- tty_insert_flip_char(tp->port,
- *(ch->ch_rqueue + tail + i),
- TTY_NORMAL);
+ unsigned char ch = *(ch_pos + i);
+ char flag = TTY_NORMAL;
+
+ if (ch & UART_LSR_BI)
+ flag = TTY_BREAK;
+ else if (ch & UART_LSR_PE)
+ flag = TTY_PARITY;
+ else if (ch & UART_LSR_FE)
+ flag = TTY_FRAME;
+
+ tty_insert_flip_char(tp->port, ch, flag);
}
} else {
- tty_insert_flip_string(tp->port,
- ch->ch_rqueue + tail,
- s);
+ tty_insert_flip_string(tp->port, ch_pos, s);
}
tail += s;
@@ -1117,6 +1082,14 @@
if (!ch->ch_wqueue)
ch->ch_wqueue = kzalloc(WQUEUESIZE, GFP_KERNEL);
+ if (!ch->ch_rqueue || !ch->ch_equeue || !ch->ch_wqueue) {
+ kfree(ch->ch_rqueue);
+ kfree(ch->ch_equeue);
+ kfree(ch->ch_wqueue);
+
+ return -ENOMEM;
+ }
+
spin_lock_irqsave(&ch->ch_lock, flags);
ch->ch_flags &= ~(CH_OPENING);
@@ -1255,7 +1228,7 @@
if (file->f_flags & O_NONBLOCK)
break;
- if (tty->flags & (1 << TTY_IO_ERROR)) {
+ if (tty_io_error(tty)) {
retval = -EIO;
break;
}
@@ -1539,19 +1512,8 @@
*/
static int dgnc_maxcps_room(struct tty_struct *tty, int bytes_available)
{
- struct channel_t *ch = NULL;
- struct un_t *un = NULL;
-
- if (!tty)
- return bytes_available;
-
- un = tty->driver_data;
- if (!un || un->magic != DGNC_UNIT_MAGIC)
- return bytes_available;
-
- ch = un->un_ch;
- if (!ch || ch->magic != DGNC_CHANNEL_MAGIC)
- return bytes_available;
+ struct un_t *un = tty->driver_data;
+ struct channel_t *ch = un->un_ch;
/*
* If its not the Transparent print device, return
@@ -2058,17 +2020,7 @@
static int dgnc_get_modem_info(struct channel_t *ch,
unsigned int __user *value)
{
- int result;
-
- if (!ch || ch->magic != DGNC_CHANNEL_MAGIC)
- return -ENXIO;
-
- result = dgnc_get_mstat(ch);
-
- if (result < 0)
- return -ENXIO;
-
- return put_user(result, value);
+ return put_user(dgnc_get_mstat(ch), value);
}
/*
@@ -2529,6 +2481,7 @@
unsigned long arg)
{
struct dgnc_board *bd;
+ struct board_ops *ch_bd_ops;
struct channel_t *ch;
struct un_t *un;
int rc;
@@ -2550,6 +2503,8 @@
if (!bd || bd->magic != DGNC_BOARD_MAGIC)
return -ENODEV;
+ ch_bd_ops = bd->bd_ops;
+
spin_lock_irqsave(&ch->ch_lock, flags);
if (un->un_open_count <= 0) {
@@ -2574,7 +2529,7 @@
if (rc)
return rc;
- rc = ch->ch_bd->bd_ops->drain(tty, 0);
+ rc = ch_bd_ops->drain(tty, 0);
if (rc)
return -EINTR;
@@ -2582,7 +2537,7 @@
spin_lock_irqsave(&ch->ch_lock, flags);
if (((cmd == TCSBRK) && (!arg)) || (cmd == TCSBRKP))
- ch->ch_bd->bd_ops->send_break(ch, 250);
+ ch_bd_ops->send_break(ch, 250);
spin_unlock_irqrestore(&ch->ch_lock, flags);
@@ -2599,13 +2554,13 @@
if (rc)
return rc;
- rc = ch->ch_bd->bd_ops->drain(tty, 0);
+ rc = ch_bd_ops->drain(tty, 0);
if (rc)
return -EINTR;
spin_lock_irqsave(&ch->ch_lock, flags);
- ch->ch_bd->bd_ops->send_break(ch, 250);
+ ch_bd_ops->send_break(ch, 250);
spin_unlock_irqrestore(&ch->ch_lock, flags);
@@ -2617,13 +2572,13 @@
if (rc)
return rc;
- rc = ch->ch_bd->bd_ops->drain(tty, 0);
+ rc = ch_bd_ops->drain(tty, 0);
if (rc)
return -EINTR;
spin_lock_irqsave(&ch->ch_lock, flags);
- ch->ch_bd->bd_ops->send_break(ch, 250);
+ ch_bd_ops->send_break(ch, 250);
spin_unlock_irqrestore(&ch->ch_lock, flags);
@@ -2652,7 +2607,7 @@
spin_lock_irqsave(&ch->ch_lock, flags);
tty->termios.c_cflag = ((tty->termios.c_cflag & ~CLOCAL) |
(arg ? CLOCAL : 0));
- ch->ch_bd->bd_ops->param(tty);
+ ch_bd_ops->param(tty);
spin_unlock_irqrestore(&ch->ch_lock, flags);
return 0;
@@ -2689,7 +2644,7 @@
if ((arg == TCIFLUSH) || (arg == TCIOFLUSH)) {
ch->ch_r_head = ch->ch_r_tail;
- ch->ch_bd->bd_ops->flush_uart_read(ch);
+ ch_bd_ops->flush_uart_read(ch);
/* Force queue flow control to be released, if needed */
dgnc_check_queue_flow_control(ch);
}
@@ -2697,9 +2652,9 @@
if ((arg == TCOFLUSH) || (arg == TCIOFLUSH)) {
if (!(un->un_type == DGNC_PRINT)) {
ch->ch_w_head = ch->ch_w_tail;
- ch->ch_bd->bd_ops->flush_uart_write(ch);
+ ch_bd_ops->flush_uart_write(ch);
- if (ch->ch_tun.un_flags & (UN_LOW|UN_EMPTY)) {
+ if (ch->ch_tun.un_flags & (UN_LOW | UN_EMPTY)) {
ch->ch_tun.un_flags &=
~(UN_LOW | UN_EMPTY);
wake_up_interruptible(&ch->ch_tun.un_flags_wait);
@@ -2731,14 +2686,14 @@
/* flush rx */
ch->ch_flags &= ~CH_STOP;
ch->ch_r_head = ch->ch_r_tail;
- ch->ch_bd->bd_ops->flush_uart_read(ch);
+ ch_bd_ops->flush_uart_read(ch);
/* Force queue flow control to be released, if needed */
dgnc_check_queue_flow_control(ch);
}
/* now wait for all the output to drain */
spin_unlock_irqrestore(&ch->ch_lock, flags);
- rc = ch->ch_bd->bd_ops->drain(tty, 0);
+ rc = ch_bd_ops->drain(tty, 0);
if (rc)
return -EINTR;
@@ -2748,7 +2703,7 @@
case TCSETAW:
spin_unlock_irqrestore(&ch->ch_lock, flags);
- rc = ch->ch_bd->bd_ops->drain(tty, 0);
+ rc = ch_bd_ops->drain(tty, 0);
if (rc)
return -EINTR;
@@ -2771,7 +2726,7 @@
/* set information for ditty */
if (cmd == (DIGI_SETAW)) {
spin_unlock_irqrestore(&ch->ch_lock, flags);
- rc = ch->ch_bd->bd_ops->drain(tty, 0);
+ rc = ch_bd_ops->drain(tty, 0);
if (rc)
return -EINTR;
@@ -2804,7 +2759,7 @@
else
ch->ch_flags &= ~(CH_LOOPBACK);
- ch->ch_bd->bd_ops->param(tty);
+ ch_bd_ops->param(tty);
spin_unlock_irqrestore(&ch->ch_lock, flags);
return 0;
}
@@ -2824,7 +2779,7 @@
return rc;
spin_lock_irqsave(&ch->ch_lock, flags);
dgnc_set_custom_speed(ch, new_rate);
- ch->ch_bd->bd_ops->param(tty);
+ ch_bd_ops->param(tty);
spin_unlock_irqrestore(&ch->ch_lock, flags);
return 0;
}
@@ -2845,7 +2800,7 @@
if (rc)
return rc;
spin_lock_irqsave(&ch->ch_lock, flags);
- ch->ch_bd->bd_ops->send_immediate_char(ch, c);
+ ch_bd_ops->send_immediate_char(ch, c);
spin_unlock_irqrestore(&ch->ch_lock, flags);
return 0;
}
@@ -2933,13 +2888,13 @@
/*
* Is the UART empty? Add that value to whats in our TX queue.
*/
- count = buf.txbuf + ch->ch_bd->bd_ops->get_uart_bytes_left(ch);
+ count = buf.txbuf + ch_bd_ops->get_uart_bytes_left(ch);
/*
* Figure out how much data the RealPort Server believes should
* be in our TX queue.
*/
- tdist = (buf.tIn - buf.tOut) & 0xffff;
+ tdist = (buf.tx_in - buf.tx_out) & 0xffff;
/*
* If we have more data than the RealPort Server believes we
diff --git a/drivers/staging/dgnc/digi.h b/drivers/staging/dgnc/digi.h
index 523a2d3..5b983e6 100644
--- a/drivers/staging/dgnc/digi.h
+++ b/drivers/staging/dgnc/digi.h
@@ -109,8 +109,8 @@
struct digi_getbuffer /* Struct for holding buffer use counts */
{
- unsigned long tIn;
- unsigned long tOut;
+ unsigned long tx_in;
+ unsigned long tx_out;
unsigned long rxbuf;
unsigned long txbuf;
unsigned long txdone;
diff --git a/drivers/staging/emxx_udc/emxx_udc.c b/drivers/staging/emxx_udc/emxx_udc.c
index e8cacae..3bd9175 100644
--- a/drivers/staging/emxx_udc/emxx_udc.c
+++ b/drivers/staging/emxx_udc/emxx_udc.c
@@ -418,9 +418,9 @@
{
struct fc_regs *preg = udc->p_regs;
- _nbu2ss_bitclr(&preg->EP_DCR[ep->epnum-1].EP_DCR1, DCR1_EPn_REQEN);
+ _nbu2ss_bitclr(&preg->EP_DCR[ep->epnum - 1].EP_DCR1, DCR1_EPn_REQEN);
mdelay(DMA_DISABLE_TIME); /* DCR1_EPn_REQEN Clear */
- _nbu2ss_bitclr(&preg->EP_REGS[ep->epnum-1].EP_DMA_CTRL, EPn_DMA_EN);
+ _nbu2ss_bitclr(&preg->EP_REGS[ep->epnum - 1].EP_DMA_CTRL, EPn_DMA_EN);
}
/*-------------------------------------------------------------------------*/
@@ -909,7 +909,7 @@
/* Copy of every four bytes */
for (i = 0; i < iWordLength; i++) {
pBuf32->dw =
- _nbu2ss_readl(&preg->EP_REGS[ep->epnum-1].EP_READ);
+ _nbu2ss_readl(&preg->EP_REGS[ep->epnum - 1].EP_READ);
pBuf32++;
}
result = iWordLength * sizeof(u32);
@@ -919,7 +919,7 @@
if (data > 0) {
/*---------------------------------------------------------*/
/* Copy of fraction byte */
- Temp32.dw = _nbu2ss_readl(&preg->EP_REGS[ep->epnum-1].EP_READ);
+ Temp32.dw = _nbu2ss_readl(&preg->EP_REGS[ep->epnum - 1].EP_READ);
for (i = 0 ; i < data ; i++)
pBuf32->byte.DATA[i] = Temp32.byte.DATA[i];
result += data;
@@ -1128,7 +1128,7 @@
if (iWordLength > 0) {
for (i = 0; i < iWordLength; i++) {
_nbu2ss_writel(
- &preg->EP_REGS[ep->epnum-1].EP_WRITE
+ &preg->EP_REGS[ep->epnum - 1].EP_WRITE
, pBuf32->dw
);
@@ -1290,7 +1290,7 @@
if (ep->epnum > 0) {
length = _nbu2ss_readl(
- &ep->udc->p_regs->EP_REGS[ep->epnum-1].EP_LEN_DCNT);
+ &ep->udc->p_regs->EP_REGS[ep->epnum - 1].EP_LEN_DCNT);
length &= EPn_LDATA;
if (length < ep->ep.maxpacket)
@@ -1463,7 +1463,7 @@
bit_data = EP0_STL;
} else {
- data = _nbu2ss_readl(&preg->EP_REGS[epnum-1].EP_CONTROL);
+ data = _nbu2ss_readl(&preg->EP_REGS[epnum - 1].EP_CONTROL);
if ((data & EPn_EN) == 0)
return -1;
@@ -1558,7 +1558,7 @@
; limit_cnt++) {
regdata = _nbu2ss_readl(
- &preg->EP_REGS[ep->epnum-1].EP_STATUS);
+ &preg->EP_REGS[ep->epnum - 1].EP_STATUS);
if ((regdata & EPn_IN_DATA) == 0)
break;
@@ -1983,7 +1983,7 @@
if (req->zero && ((req->req.actual % ep->ep.maxpacket) == 0)) {
status =
- _nbu2ss_readl(&preg->EP_REGS[ep->epnum-1].EP_STATUS);
+ _nbu2ss_readl(&preg->EP_REGS[ep->epnum - 1].EP_STATUS);
if ((status & EPn_IN_FULL) == 0) {
/*-----------------------------------------*/
@@ -2894,7 +2894,7 @@
data = _nbu2ss_readl(&preg->EP0_LENGTH) & EP0_LDATA;
} else {
- data = _nbu2ss_readl(&preg->EP_REGS[ep->epnum-1].EP_LEN_DCNT)
+ data = _nbu2ss_readl(&preg->EP_REGS[ep->epnum - 1].EP_LEN_DCNT)
& EPn_LDATA;
}
@@ -3051,7 +3051,7 @@
}
/*-------------------------------------------------------------------------*/
-static int nbu2ss_gad_vbus_draw(struct usb_gadget *pgadget, unsigned mA)
+static int nbu2ss_gad_vbus_draw(struct usb_gadget *pgadget, unsigned int mA)
{
struct nbu2ss_udc *udc;
unsigned long flags;
@@ -3101,7 +3101,7 @@
/*-------------------------------------------------------------------------*/
static int nbu2ss_gad_ioctl(
struct usb_gadget *pgadget,
- unsigned code,
+ unsigned int code,
unsigned long param)
{
return 0;
diff --git a/drivers/staging/emxx_udc/emxx_udc.h b/drivers/staging/emxx_udc/emxx_udc.h
index 4a2cc38..39769e3 100644
--- a/drivers/staging/emxx_udc/emxx_udc.h
+++ b/drivers/staging/emxx_udc/emxx_udc.h
@@ -97,7 +97,7 @@
#define BIT30 0x40000000
#define BIT31 0x80000000
-#define TEST_FORCE_ENABLE (BIT18+BIT16)
+#define TEST_FORCE_ENABLE (BIT18 + BIT16)
#define INT_SEL BIT10
#define CONSTFS BIT09
@@ -125,15 +125,15 @@
/*------- (0x0008) USB Address Register */
#define USB_ADDR 0x007F0000
#define SOF_STATUS BIT15
-#define UFRAME (BIT14+BIT13+BIT12)
+#define UFRAME (BIT14 + BIT13 + BIT12)
#define FRAME 0x000007FF
#define USB_ADRS_SHIFT 16
/*------- (0x000C) UTMI Characteristic 1 Register */
-#define SQUSET (BIT07+BIT06+BIT05+BIT04)
+#define SQUSET (BIT07 + BIT06 + BIT05 + BIT04)
-#define USB_SQUSET (BIT06+BIT05+BIT04)
+#define USB_SQUSET (BIT06 + BIT05 + BIT04)
/*------- (0x0010) TEST Control Register */
#define FORCEHS BIT02
@@ -196,7 +196,7 @@
#define RSUM_EN BIT01
#define USB_INT_EN_BIT \
- (EP0_EN|SPEED_MODE_EN|USB_RST_EN|SPND_EN|RSUM_EN)
+ (EP0_EN | SPEED_MODE_EN | USB_RST_EN | SPND_EN | RSUM_EN)
/*------- (0x0028) EP0 Control Register */
#define EP0_STGSEL BIT18
@@ -205,9 +205,9 @@
#define EP0_PIDCLR BIT09
#define EP0_BCLR BIT08
#define EP0_DEND BIT07
-#define EP0_DW (BIT06+BIT05)
+#define EP0_DW (BIT06 + BIT05)
#define EP0_DW4 0
-#define EP0_DW3 (BIT06+BIT05)
+#define EP0_DW3 (BIT06 + BIT05)
#define EP0_DW2 BIT06
#define EP0_DW1 BIT05
@@ -238,7 +238,7 @@
#define STG_START_INT BIT01
#define SETUP_INT BIT00
-#define EP0_STATUS_RW_BIT (BIT16|BIT15|BIT11|0xFF)
+#define EP0_STATUS_RW_BIT (BIT16 | BIT15 | BIT11 | 0xFF)
/*------- (0x0030) EP0 Interrupt Enable Register */
#define EP0_PERR_NAK_EN BIT16
@@ -256,7 +256,7 @@
#define SETUP_EN BIT00
#define EP0_INT_EN_BIT \
- (EP0_OUT_OR_EN|EP0_OUT_EN|EP0_IN_EN|STG_END_EN|SETUP_EN)
+ (EP0_OUT_OR_EN | EP0_OUT_EN | EP0_IN_EN | STG_END_EN | SETUP_EN)
/*------- (0x0034) EP0 Length Register */
#define EP0_LDATA 0x0000007F
@@ -270,7 +270,7 @@
#define EPn_BUF_SINGLE BIT30
#define EPn_DIR0 BIT26
-#define EPn_MODE (BIT25+BIT24)
+#define EPn_MODE (BIT25 + BIT24)
#define EPn_BULK 0
#define EPn_INTERRUPT BIT24
#define EPn_ISO BIT25
@@ -283,9 +283,9 @@
#define EPn_BCLR BIT09
#define EPn_CBCLR BIT08
#define EPn_DEND BIT07
-#define EPn_DW (BIT06+BIT05)
+#define EPn_DW (BIT06 + BIT05)
#define EPn_DW4 0
-#define EPn_DW3 (BIT06+BIT05)
+#define EPn_DW3 (BIT06 + BIT05)
#define EPn_DW2 BIT06
#define EPn_DW1 BIT05
@@ -324,7 +324,7 @@
#define EPn_IN_EMPTY BIT00 /* R */
#define EPn_INT_EN \
- (EPn_OUT_END_INT|EPn_OUT_INT|EPn_IN_END_INT|EPn_IN_INT)
+ (EPn_OUT_END_INT | EPn_OUT_INT | EPn_IN_END_INT | EPn_IN_INT)
/*------- (0x0048:) EPn Interrupt Enable Register */
#define EPn_OUT_END_EN BIT23 /* RW */
@@ -368,7 +368,7 @@
#define ARBITER_CTR BIT31 /* RW */
#define MCYCLE_RST BIT12 /* RW */
-#define ENDIAN_CTR (BIT09+BIT08) /* RW */
+#define ENDIAN_CTR (BIT09 + BIT08) /* RW */
#define ENDIAN_BYTE_SWAP BIT09
#define ENDIAN_HALF_WORD_SWAP ENDIAN_CTR
@@ -376,7 +376,7 @@
#define HTRANS_MODE BIT04 /* RW */
#define WBURST_TYPE BIT02 /* RW */
-#define BURST_TYPE (BIT01+BIT00) /* RW */
+#define BURST_TYPE (BIT01 + BIT00) /* RW */
#define BURST_MAX_16 0
#define BURST_MAX_8 BIT00
#define BURST_MAX_4 BIT01
@@ -412,7 +412,7 @@
#define EPC_RST BIT00 /* RW */
/*------- (0x1014) USBF_EPTEST Register */
-#define LINESTATE (BIT09+BIT08) /* R */
+#define LINESTATE (BIT09 + BIT08) /* R */
#define DM_LEVEL BIT09 /* R */
#define DP_LEVEL BIT08 /* R */
@@ -485,7 +485,7 @@
struct ep_regs EP_REGS[REG_EP_NUM]; /* Endpoint Register */
- u8 Reserved220[0x1000-0x220]; /* (0x0220:0x0FFF) Reserved */
+ u8 Reserved220[0x1000 - 0x220]; /* (0x0220:0x0FFF) Reserved */
u32 AHBSCTR; /* (0x1000) AHBSCTR */
u32 AHBMCTR; /* (0x1004) AHBMCTR */
@@ -494,16 +494,16 @@
u32 EPCTR; /* (0x1010) EPCTR */
u32 USBF_EPTEST; /* (0x1014) USBF_EPTEST */
- u8 Reserved1018[0x20-0x18]; /* (0x1018:0x101F) Reserved */
+ u8 Reserved1018[0x20 - 0x18]; /* (0x1018:0x101F) Reserved */
u32 USBSSVER; /* (0x1020) USBSSVER */
u32 USBSSCONF; /* (0x1024) USBSSCONF */
- u8 Reserved1028[0x110-0x28]; /* (0x1028:0x110F) Reserved */
+ u8 Reserved1028[0x110 - 0x28]; /* (0x1028:0x110F) Reserved */
struct ep_dcr EP_DCR[REG_EP_NUM]; /* */
- u8 Reserved1200[0x1000-0x200]; /* Reserved */
+ u8 Reserved1200[0x1000 - 0x200]; /* Reserved */
} __aligned(32);
#define EP0_PACKETSIZE 64
diff --git a/drivers/staging/fbtft/fb_agm1264k-fl.c b/drivers/staging/fbtft/fb_agm1264k-fl.c
index ba9fc44..82b46cd 100644
--- a/drivers/staging/fbtft/fb_agm1264k-fl.c
+++ b/drivers/staging/fbtft/fb_agm1264k-fl.c
@@ -414,7 +414,7 @@
while (len--) {
u8 i, data;
- data = *(u8 *) buf++;
+ data = *(u8 *)buf++;
/* set data bus */
for (i = 0; i < 8; ++i)
diff --git a/drivers/staging/fbtft/fbtft-io.c b/drivers/staging/fbtft/fbtft-io.c
index a6f091f..4dcea2e 100644
--- a/drivers/staging/fbtft/fbtft-io.c
+++ b/drivers/staging/fbtft/fbtft-io.c
@@ -141,7 +141,7 @@
"%s(len=%d): ", __func__, len);
while (len--) {
- data = *(u8 *) buf;
+ data = *(u8 *)buf;
/* Start writing by pulling down /WR */
gpio_set_value(par->gpio.wr, 0);
@@ -170,7 +170,7 @@
gpio_set_value(par->gpio.wr, 1);
#ifndef DO_NOT_OPTIMIZE_FBTFT_WRITE_GPIO
- prev_data = *(u8 *) buf;
+ prev_data = *(u8 *)buf;
#endif
buf++;
}
@@ -191,7 +191,7 @@
"%s(len=%d): ", __func__, len);
while (len) {
- data = *(u16 *) buf;
+ data = *(u16 *)buf;
/* Start writing by pulling down /WR */
gpio_set_value(par->gpio.wr, 0);
@@ -220,7 +220,7 @@
gpio_set_value(par->gpio.wr, 1);
#ifndef DO_NOT_OPTIMIZE_FBTFT_WRITE_GPIO
- prev_data = *(u16 *) buf;
+ prev_data = *(u16 *)buf;
#endif
buf += 2;
len -= 2;
diff --git a/drivers/staging/fbtft/fbtft_device.c b/drivers/staging/fbtft/fbtft_device.c
index 241d7c6..e4a355a 100644
--- a/drivers/staging/fbtft/fbtft_device.c
+++ b/drivers/staging/fbtft/fbtft_device.c
@@ -1254,7 +1254,7 @@
"%s(len=%d): ", __func__, len);
while (len) {
- data = *(u16 *) buf;
+ data = *(u16 *)buf;
/* Start writing by pulling down /WR */
gpio_set_value(par->gpio.wr, 0);
@@ -1283,7 +1283,7 @@
gpio_set_value(par->gpio.wr, 1);
#ifndef DO_NOT_OPTIMIZE_FBTFT_WRITE_GPIO
- prev_data = *(u16 *) buf;
+ prev_data = *(u16 *)buf;
#endif
buf += 2;
len -= 2;
@@ -1436,7 +1436,7 @@
}
strncpy(fbtft_device_param_gpios[i].name, p_name,
FBTFT_GPIO_NAME_SIZE - 1);
- fbtft_device_param_gpios[i++].gpio = (int) val;
+ fbtft_device_param_gpios[i++].gpio = (int)val;
if (i == MAX_GPIOS) {
pr_err("gpios parameter: exceeded max array size: %d\n",
MAX_GPIOS);
diff --git a/drivers/staging/fsl-mc/README.txt b/drivers/staging/fsl-mc/README.txt
index 8214102..179536a 100644
--- a/drivers/staging/fsl-mc/README.txt
+++ b/drivers/staging/fsl-mc/README.txt
@@ -11,11 +11,11 @@
-Overview of DPAA2 objects
-DPAA2 Linux driver architecture overview
-bus driver
- -dprc driver
+ -DPRC driver
-allocator
- -dpio driver
+ -DPIO driver
-Ethernet
- -mac
+ -MAC
DPAA2 Overview
--------------
@@ -37,6 +37,9 @@
The MC provides memory-mapped I/O command interfaces (MC portals)
which DPAA2 software drivers use to operate on DPAA2 objects:
+The diagram below shows an overview of the DPAA2 resource management
+architecture:
+
+--------------------------------------+
| OS |
| DPAA2 drivers |
@@ -77,13 +80,13 @@
Overview of DPAA2 Objects
-------------------------
-The section provides a brief overview of some key objects
-in the DPAA2 hardware. A simple scenario is described illustrating
-the objects involved in creating a network interfaces.
+The section provides a brief overview of some key DPAA2 objects.
+A simple scenario is described illustrating the objects involved
+in creating a network interfaces.
-DPRC (Datapath Resource Container)
- A DPRC is an container object that holds all the other
+ A DPRC is a container object that holds all the other
types of DPAA2 objects. In the example diagram below there
are 8 objects of 5 types (DPMCP, DPIO, DPBP, DPNI, and DPMAC)
in the container.
@@ -101,23 +104,23 @@
| |
+---------------------------------------------------------+
- From the point of view of an OS, a DPRC is bus-like. Like
- a plug-and-play bus, such as PCI, DPRC commands can be used to
- enumerate the contents of the DPRC, discover the hardware
- objects present (including mappable regions and interrupts).
+ From the point of view of an OS, a DPRC behaves similar to a plug and
+ play bus, like PCI. DPRC commands can be used to enumerate the contents
+ of the DPRC, discover the hardware objects present (including mappable
+ regions and interrupts).
- dprc.1 (bus)
+ DPRC.1 (bus)
|
+--+--------+-------+-------+-------+
| | | | |
- dpmcp.1 dpio.1 dpbp.1 dpni.1 dpmac.1
- dpmcp.2 dpio.2
- dpmcp.3
+ DPMCP.1 DPIO.1 DPBP.1 DPNI.1 DPMAC.1
+ DPMCP.2 DPIO.2
+ DPMCP.3
Hardware objects can be created and destroyed dynamically, providing
the ability to hot plug/unplug objects in and out of the DPRC.
- A DPRC has a mappable mmio region (an MC portal) that can be used
+ A DPRC has a mappable MMIO region (an MC portal) that can be used
to send MC commands. It has an interrupt for status events (like
hotplug).
@@ -137,10 +140,11 @@
A typical Ethernet NIC is monolithic-- the NIC device contains TX/RX
queuing mechanisms, configuration mechanisms, buffer management,
physical ports, and interrupts. DPAA2 uses a more granular approach
- utilizing multiple hardware objects. Each object has specialized
- functions, and are used together by software to provide Ethernet network
- interface functionality. This approach provides efficient use of finite
- hardware resources, flexibility, and performance advantages.
+ utilizing multiple hardware objects. Each object provides specialized
+ functions. Groups of these objects are used by software to provide
+ Ethernet network interface functionality. This approach provides
+ efficient use of finite hardware resources, flexibility, and
+ performance advantages.
The diagram below shows the objects needed for a simple
network interface configuration on a system with 2 CPUs.
@@ -168,46 +172,52 @@
Below the objects are described. For each object a brief description
is provided along with a summary of the kinds of operations the object
- supports and a summary of key resources of the object (mmio regions
- and irqs).
+ supports and a summary of key resources of the object (MMIO regions
+ and IRQs).
-DPMAC (Datapath Ethernet MAC): represents an Ethernet MAC, a
hardware device that connects to an Ethernet PHY and allows
physical transmission and reception of Ethernet frames.
- -mmio regions: none
- -irqs: dpni link change
+ -MMIO regions: none
+ -IRQs: DPNI link change
-commands: set link up/down, link config, get stats,
- irq config, enable, reset
+ IRQ config, enable, reset
-DPNI (Datapath Network Interface): contains TX/RX queues,
- network interface configuration, and rx buffer pool configuration
- mechanisms.
- -mmio regions: none
- -irqs: link state
+ network interface configuration, and RX buffer pool configuration
+ mechanisms. The TX/RX queues are in memory and are identified by
+ queue number.
+ -MMIO regions: none
+ -IRQs: link state
-commands: port config, offload config, queue config,
- parse/classify config, irq config, enable, reset
+ parse/classify config, IRQ config, enable, reset
-DPIO (Datapath I/O): provides interfaces to enqueue and dequeue
- packets and do hardware buffer pool management operations. For
- optimum performance there is typically DPIO per CPU. This allows
- each CPU to perform simultaneous enqueue/dequeue operations.
- -mmio regions: queue operations, buffer mgmt
- -irqs: data availability, congestion notification, buffer
+ packets and do hardware buffer pool management operations. The DPAA2
+ architecture separates the mechanism to access queues (the DPIO object)
+ from the queues themselves. The DPIO provides an MMIO interface to
+ enqueue/dequeue packets. To enqueue something a descriptor is written
+ to the DPIO MMIO region, which includes the target queue number.
+ There will typically be one DPIO assigned to each CPU. This allows all
+ CPUs to simultaneously perform enqueue/dequeued operations. DPIOs are
+ expected to be shared by different DPAA2 drivers.
+ -MMIO regions: queue operations, buffer management
+ -IRQs: data availability, congestion notification, buffer
pool depletion
- -commands: irq config, enable, reset
+ -commands: IRQ config, enable, reset
-DPBP (Datapath Buffer Pool): represents a hardware buffer
pool.
- -mmio regions: none
- -irqs: none
+ -MMIO regions: none
+ -IRQs: none
-commands: enable, reset
-DPMCP (Datapath MC Portal): provides an MC command portal.
Used by drivers to send commands to the MC to manage
objects.
- -mmio regions: MC command portal
- -irqs: command completion
- -commands: irq config, enable, reset
+ -MMIO regions: MC command portal
+ -IRQs: command completion
+ -commands: IRQ config, enable, reset
Object Connections
------------------
@@ -268,22 +278,22 @@
| Stack |
+------------+ +------------+
| Allocator |. . . . . . . | Ethernet |
- |(dpmcp,dpbp)| | (dpni) |
+ |(DPMCP,DPBP)| | (DPNI) |
+-.----------+ +---+---+----+
. . ^ |
. . <data avail, | |<enqueue,
. . tx confirm> | | dequeue>
+-------------+ . | |
| DPRC driver | . +---+---V----+ +---------+
- | (dprc) | . . . . . .| DPIO driver| | MAC |
- +----------+--+ | (dpio) | | (dpmac) |
+ | (DPRC) | . . . . . .| DPIO driver| | MAC |
+ +----------+--+ | (DPIO) | | (DPMAC) |
| +------+-----+ +-----+---+
|<dev add/remove> | |
| | |
+----+--------------+ | +--+---+
- | mc-bus driver | | | PHY |
+ | MC-bus driver | | | PHY |
| | | |driver|
- | /fsl-mc@80c000000 | | +--+---+
+ | /soc/fsl-mc | | +--+---+
+-------------------+ | |
| |
================================ HARDWARE =========|=================|======
@@ -298,25 +308,27 @@
A brief description of each driver is provided below.
- mc-bus driver
+ MC-bus driver
-------------
- The mc-bus driver is a platform driver and is probed from an
- "/fsl-mc@xxxx" node in the device tree passed in by boot firmware.
- It is responsible for bootstrapping the DPAA2 kernel infrastructure.
+ The MC-bus driver is a platform driver and is probed from a
+ node in the device tree (compatible "fsl,qoriq-mc") passed in by boot
+ firmware. It is responsible for bootstrapping the DPAA2 kernel
+ infrastructure.
Key functions include:
-registering a new bus type named "fsl-mc" with the kernel,
and implementing bus call-backs (e.g. match/uevent/dev_groups)
- -implemeting APIs for DPAA2 driver registration and for device
+ -implementing APIs for DPAA2 driver registration and for device
add/remove
- -creates an MSI irq domain
- -do a device add of the 'root' DPRC device, which is needed
- to bootstrap things
+ -creates an MSI IRQ domain
+ -doing a 'device add' to expose the 'root' DPRC, in turn triggering
+ a bind of the root DPRC to the DPRC driver
DPRC driver
-----------
- The dprc-driver is bound DPRC objects and does runtime management
+ The DPRC driver is bound to DPRC objects and does runtime management
of a bus instance. It performs the initial bus scan of the DPRC
- and handles interrupts for container events such as hot plug.
+ and handles interrupts for container events such as hot plug by
+ re-scanning the DPRC.
Allocator
----------
@@ -334,14 +346,20 @@
DPIO driver
-----------
The DPIO driver is bound to DPIO objects and provides services that allow
- other drivers such as the Ethernet driver to receive and transmit data.
+ other drivers such as the Ethernet driver to enqueue and dequeue data for
+ their respective objects.
Key services include:
-data availability notifications
-hardware queuing operations (enqueue and dequeue of data)
-hardware buffer pool management
+ To transmit a packet the Ethernet driver puts data on a queue and
+ invokes a DPIO API. For receive, the Ethernet driver registers
+ a data availability notification callback. To dequeue a packet
+ a DPIO API is used.
+
There is typically one DPIO object per physical CPU for optimum
- performance, allowing each CPU to simultaneously enqueue
+ performance, allowing different CPUs to simultaneously enqueue
and dequeue data.
The DPIO driver operates on behalf of all DPAA2 drivers
@@ -362,3 +380,7 @@
by the appropriate PHY driver via an mdio bus. The MAC driver
plays a role of being a proxy between the PHY driver and the
MC. It does this proxy via the MC commands to a DPMAC object.
+ If the PHY driver signals a link change, the MAC driver notifies
+ the MC via a DPMAC command. If a network interface is brought
+ up or down, the MC notifies the DPMAC driver via an interrupt and
+ the driver can take appropriate action.
diff --git a/drivers/staging/fsl-mc/TODO b/drivers/staging/fsl-mc/TODO
index 3894368..54a8bc6 100644
--- a/drivers/staging/fsl-mc/TODO
+++ b/drivers/staging/fsl-mc/TODO
@@ -1,21 +1,8 @@
-* Decide if multiple root fsl-mc buses will be supported per Linux instance,
- and if so add support for this.
-
* Add at least one device driver for a DPAA2 object (child device of the
fsl-mc bus). Most likely candidate for this is adding DPAA2 Ethernet
driver support, which depends on drivers for several objects: DPNI,
DPIO, DPMAC. Other pre-requisites include:
- * interrupt support. for meaningful driver support we need
- interrupts, and thus need message interrupt support by the bus
- driver.
- -Note: this has dependencies on generic MSI support work
- in process upstream, see [1] and [2].
-
- * Management Complex (MC) command serialization. locking mechanisms
- are needed by drivers to serialize commands sent to the MC, including
- from atomic context.
-
* MC firmware uprev. The MC firmware upon which the fsl-mc
bus driver and DPAA2 object drivers are based is continuing
to evolve, so minor updates are needed to keep in sync with binary
diff --git a/drivers/staging/fsl-mc/bus/dpbp.c b/drivers/staging/fsl-mc/bus/dpbp.c
index 2d97173..c31fe1b 100644
--- a/drivers/staging/fsl-mc/bus/dpbp.c
+++ b/drivers/staging/fsl-mc/bus/dpbp.c
@@ -293,7 +293,7 @@
cmd.params[0] |= mc_enc(0, 8, irq_index);
cmd.params[0] |= mc_enc(32, 32, irq_cfg->val);
cmd.params[1] |= mc_enc(0, 64, irq_cfg->addr);
- cmd.params[2] |= mc_enc(0, 32, irq_cfg->user_irq_id);
+ cmd.params[2] |= mc_enc(0, 32, irq_cfg->irq_num);
/* send command to mc*/
return mc_send_command(mc_io, &cmd);
@@ -334,7 +334,7 @@
/* retrieve response parameters */
irq_cfg->val = (u32)mc_dec(cmd.params[0], 0, 32);
irq_cfg->addr = (u64)mc_dec(cmd.params[1], 0, 64);
- irq_cfg->user_irq_id = (int)mc_dec(cmd.params[2], 0, 32);
+ irq_cfg->irq_num = (int)mc_dec(cmd.params[2], 0, 32);
*type = (int)mc_dec(cmd.params[2], 32, 32);
return 0;
}
@@ -502,6 +502,7 @@
/* prepare command */
cmd.header = mc_encode_cmd_header(DPBP_CMDID_GET_IRQ_STATUS,
cmd_flags, token);
+ cmd.params[0] |= mc_enc(0, 32, *status);
cmd.params[0] |= mc_enc(32, 8, irq_index);
/* send command to mc*/
@@ -580,3 +581,75 @@
return 0;
}
EXPORT_SYMBOL(dpbp_get_attributes);
+
+/**
+ * dpbp_set_notifications() - Set notifications towards software
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPBP object
+ * @cfg: notifications configuration
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpbp_set_notifications(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ struct dpbp_notification_cfg *cfg)
+{
+ struct mc_command cmd = { 0 };
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPBP_CMDID_SET_NOTIFICATIONS,
+ cmd_flags,
+ token);
+
+ cmd.params[0] |= mc_enc(0, 32, cfg->depletion_entry);
+ cmd.params[0] |= mc_enc(32, 32, cfg->depletion_exit);
+ cmd.params[1] |= mc_enc(0, 32, cfg->surplus_entry);
+ cmd.params[1] |= mc_enc(32, 32, cfg->surplus_exit);
+ cmd.params[2] |= mc_enc(0, 16, cfg->options);
+ cmd.params[3] |= mc_enc(0, 64, cfg->message_ctx);
+ cmd.params[4] |= mc_enc(0, 64, cfg->message_iova);
+
+ /* send command to mc*/
+ return mc_send_command(mc_io, &cmd);
+}
+
+/**
+ * dpbp_get_notifications() - Get the notifications configuration
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPBP object
+ * @cfg: notifications configuration
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpbp_get_notifications(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ struct dpbp_notification_cfg *cfg)
+{
+ struct mc_command cmd = { 0 };
+ int err;
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPBP_CMDID_GET_NOTIFICATIONS,
+ cmd_flags,
+ token);
+
+ /* send command to mc*/
+ err = mc_send_command(mc_io, &cmd);
+ if (err)
+ return err;
+
+ /* retrieve response parameters */
+ cfg->depletion_entry = (u32)mc_dec(cmd.params[0], 0, 32);
+ cfg->depletion_exit = (u32)mc_dec(cmd.params[0], 32, 32);
+ cfg->surplus_entry = (u32)mc_dec(cmd.params[1], 0, 32);
+ cfg->surplus_exit = (u32)mc_dec(cmd.params[1], 32, 32);
+ cfg->options = (u16)mc_dec(cmd.params[2], 0, 16);
+ cfg->message_ctx = (u64)mc_dec(cmd.params[3], 0, 64);
+ cfg->message_iova = (u64)mc_dec(cmd.params[4], 0, 64);
+
+ return 0;
+}
diff --git a/drivers/staging/fsl-mc/bus/dpmcp-cmd.h b/drivers/staging/fsl-mc/bus/dpmcp-cmd.h
index a87e9f8..c9b52dd 100644
--- a/drivers/staging/fsl-mc/bus/dpmcp-cmd.h
+++ b/drivers/staging/fsl-mc/bus/dpmcp-cmd.h
@@ -32,9 +32,9 @@
#ifndef _FSL_DPMCP_CMD_H
#define _FSL_DPMCP_CMD_H
-/* DPMCP Version */
-#define DPMCP_VER_MAJOR 2
-#define DPMCP_VER_MINOR 1
+/* Minimal supported DPMCP Version */
+#define DPMCP_MIN_VER_MAJOR 3
+#define DPMCP_MIN_VER_MINOR 0
/* Command IDs */
#define DPMCP_CMDID_CLOSE 0x800
@@ -52,6 +52,5 @@
#define DPMCP_CMDID_SET_IRQ_MASK 0x014
#define DPMCP_CMDID_GET_IRQ_MASK 0x015
#define DPMCP_CMDID_GET_IRQ_STATUS 0x016
-#define DPMCP_CMDID_CLEAR_IRQ_STATUS 0x017
#endif /* _FSL_DPMCP_CMD_H */
diff --git a/drivers/staging/fsl-mc/bus/dpmcp.c b/drivers/staging/fsl-mc/bus/dpmcp.c
index b0248f5..fd6dd4e 100644
--- a/drivers/staging/fsl-mc/bus/dpmcp.c
+++ b/drivers/staging/fsl-mc/bus/dpmcp.c
@@ -213,7 +213,7 @@
cmd.params[0] |= mc_enc(0, 8, irq_index);
cmd.params[0] |= mc_enc(32, 32, irq_cfg->val);
cmd.params[1] |= mc_enc(0, 64, irq_cfg->paddr);
- cmd.params[2] |= mc_enc(0, 32, irq_cfg->user_irq_id);
+ cmd.params[2] |= mc_enc(0, 32, irq_cfg->irq_num);
/* send command to mc*/
return mc_send_command(mc_io, &cmd);
@@ -254,7 +254,7 @@
/* retrieve response parameters */
irq_cfg->val = (u32)mc_dec(cmd.params[0], 0, 32);
irq_cfg->paddr = (u64)mc_dec(cmd.params[1], 0, 64);
- irq_cfg->user_irq_id = (int)mc_dec(cmd.params[2], 0, 32);
+ irq_cfg->irq_num = (int)mc_dec(cmd.params[2], 0, 32);
*type = (int)mc_dec(cmd.params[2], 32, 32);
return 0;
}
@@ -435,37 +435,6 @@
}
/**
- * dpmcp_clear_irq_status() - Clear a pending interrupt's status
- *
- * @mc_io: Pointer to MC portal's I/O object
- * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
- * @token: Token of DPMCP object
- * @irq_index: The interrupt index to configure
- * @status: Bits to clear (W1C) - one bit per cause:
- * 0 = don't change
- * 1 = clear status bit
- *
- * Return: '0' on Success; Error code otherwise.
- */
-int dpmcp_clear_irq_status(struct fsl_mc_io *mc_io,
- u32 cmd_flags,
- u16 token,
- u8 irq_index,
- u32 status)
-{
- struct mc_command cmd = { 0 };
-
- /* prepare command */
- cmd.header = mc_encode_cmd_header(DPMCP_CMDID_CLEAR_IRQ_STATUS,
- cmd_flags, token);
- cmd.params[0] |= mc_enc(0, 32, status);
- cmd.params[0] |= mc_enc(32, 8, irq_index);
-
- /* send command to mc*/
- return mc_send_command(mc_io, &cmd);
-}
-
-/**
* dpmcp_get_attributes - Retrieve DPMCP attributes.
*
* @mc_io: Pointer to MC portal's I/O object
diff --git a/drivers/staging/fsl-mc/bus/dpmcp.h b/drivers/staging/fsl-mc/bus/dpmcp.h
index 6df351f..fe79d4d 100644
--- a/drivers/staging/fsl-mc/bus/dpmcp.h
+++ b/drivers/staging/fsl-mc/bus/dpmcp.h
@@ -82,12 +82,12 @@
* struct dpmcp_irq_cfg - IRQ configuration
* @paddr: Address that must be written to signal a message-based interrupt
* @val: Value to write into irq_addr address
- * @user_irq_id: A user defined number associated with this IRQ
+ * @irq_num: A user defined number associated with this IRQ
*/
struct dpmcp_irq_cfg {
uint64_t paddr;
uint32_t val;
- int user_irq_id;
+ int irq_num;
};
int dpmcp_set_irq(struct fsl_mc_io *mc_io,
@@ -133,12 +133,6 @@
uint8_t irq_index,
uint32_t *status);
-int dpmcp_clear_irq_status(struct fsl_mc_io *mc_io,
- uint32_t cmd_flags,
- uint16_t token,
- uint8_t irq_index,
- uint32_t status);
-
/**
* struct dpmcp_attr - Structure representing DPMCP attributes
* @id: DPMCP object ID
diff --git a/drivers/staging/fsl-mc/bus/dprc-cmd.h b/drivers/staging/fsl-mc/bus/dprc-cmd.h
index 6552c20..9b854fa 100644
--- a/drivers/staging/fsl-mc/bus/dprc-cmd.h
+++ b/drivers/staging/fsl-mc/bus/dprc-cmd.h
@@ -40,9 +40,9 @@
#ifndef _FSL_DPRC_CMD_H
#define _FSL_DPRC_CMD_H
-/* DPRC Version */
-#define DPRC_VER_MAJOR 4
-#define DPRC_VER_MINOR 0
+/* Minimal supported DPRC Version */
+#define DPRC_MIN_VER_MAJOR 5
+#define DPRC_MIN_VER_MINOR 0
/* Command IDs */
#define DPRC_CMDID_CLOSE 0x800
diff --git a/drivers/staging/fsl-mc/bus/dprc-driver.c b/drivers/staging/fsl-mc/bus/dprc-driver.c
index 31488a7..7fc4717 100644
--- a/drivers/staging/fsl-mc/bus/dprc-driver.c
+++ b/drivers/staging/fsl-mc/bus/dprc-driver.c
@@ -312,6 +312,15 @@
continue;
}
+ /*
+ * add a quirk for all versions of dpsec < 4.0...none
+ * are coherent regardless of what the MC reports.
+ */
+ if ((strcmp(obj_desc->type, "dpseci") == 0) &&
+ (obj_desc->ver_major < 4))
+ obj_desc->flags |=
+ DPRC_OBJ_FLAG_NO_MEM_SHAREABILITY;
+
irq_count += obj_desc->irq_count;
dev_dbg(&mc_bus_dev->dev,
"Discovered object: type %s, id %d\n",
@@ -423,6 +432,7 @@
if (WARN_ON(!msi_desc || msi_desc->irq != (u32)irq_num))
goto out;
+ status = 0;
error = dprc_get_irq_status(mc_io, 0, mc_dev->mc_handle, 0,
&status);
if (error < 0) {
@@ -692,6 +702,25 @@
goto error_cleanup_msi_domain;
}
+ error = dprc_get_attributes(mc_dev->mc_io, 0, mc_dev->mc_handle,
+ &mc_bus->dprc_attr);
+ if (error < 0) {
+ dev_err(&mc_dev->dev, "dprc_get_attributes() failed: %d\n",
+ error);
+ goto error_cleanup_open;
+ }
+
+ if (mc_bus->dprc_attr.version.major < DPRC_MIN_VER_MAJOR ||
+ (mc_bus->dprc_attr.version.major == DPRC_MIN_VER_MAJOR &&
+ mc_bus->dprc_attr.version.minor < DPRC_MIN_VER_MINOR)) {
+ dev_err(&mc_dev->dev,
+ "ERROR: DPRC version %d.%d not supported\n",
+ mc_bus->dprc_attr.version.major,
+ mc_bus->dprc_attr.version.minor);
+ error = -ENOTSUPP;
+ goto error_cleanup_open;
+ }
+
mutex_init(&mc_bus->scan_mutex);
/*
@@ -779,9 +808,7 @@
static const struct fsl_mc_device_match_id match_id_table[] = {
{
.vendor = FSL_MC_VENDOR_FREESCALE,
- .obj_type = "dprc",
- .ver_major = DPRC_VER_MAJOR,
- .ver_minor = DPRC_VER_MINOR},
+ .obj_type = "dprc"},
{.vendor = 0x0},
};
diff --git a/drivers/staging/fsl-mc/bus/dprc.c b/drivers/staging/fsl-mc/bus/dprc.c
index 381b9a9..a2c4737 100644
--- a/drivers/staging/fsl-mc/bus/dprc.c
+++ b/drivers/staging/fsl-mc/bus/dprc.c
@@ -265,7 +265,7 @@
/* retrieve response parameters */
irq_cfg->val = mc_dec(cmd.params[0], 0, 32);
irq_cfg->paddr = mc_dec(cmd.params[1], 0, 64);
- irq_cfg->user_irq_id = mc_dec(cmd.params[2], 0, 32);
+ irq_cfg->irq_num = mc_dec(cmd.params[2], 0, 32);
*type = mc_dec(cmd.params[2], 32, 32);
return 0;
@@ -296,7 +296,7 @@
cmd.params[0] |= mc_enc(32, 8, irq_index);
cmd.params[0] |= mc_enc(0, 32, irq_cfg->val);
cmd.params[1] |= mc_enc(0, 64, irq_cfg->paddr);
- cmd.params[2] |= mc_enc(0, 32, irq_cfg->user_irq_id);
+ cmd.params[2] |= mc_enc(0, 32, irq_cfg->irq_num);
/* send command to mc*/
return mc_send_command(mc_io, &cmd);
@@ -466,6 +466,7 @@
/* prepare command */
cmd.header = mc_encode_cmd_header(DPRC_CMDID_GET_IRQ_STATUS,
cmd_flags, token);
+ cmd.params[0] |= mc_enc(0, 32, *status);
cmd.params[0] |= mc_enc(32, 8, irq_index);
/* send command to mc*/
@@ -948,6 +949,7 @@
obj_desc->state = mc_dec(cmd.params[1], 32, 32);
obj_desc->ver_major = mc_dec(cmd.params[2], 0, 16);
obj_desc->ver_minor = mc_dec(cmd.params[2], 16, 16);
+ obj_desc->flags = mc_dec(cmd.params[2], 32, 16);
obj_desc->type[0] = mc_dec(cmd.params[3], 0, 8);
obj_desc->type[1] = mc_dec(cmd.params[3], 8, 8);
obj_desc->type[2] = mc_dec(cmd.params[3], 16, 8);
@@ -1042,6 +1044,7 @@
obj_desc->state = (u32)mc_dec(cmd.params[1], 32, 32);
obj_desc->ver_major = (u16)mc_dec(cmd.params[2], 0, 16);
obj_desc->ver_minor = (u16)mc_dec(cmd.params[2], 16, 16);
+ obj_desc->flags = mc_dec(cmd.params[2], 32, 16);
obj_desc->type[0] = (char)mc_dec(cmd.params[3], 0, 8);
obj_desc->type[1] = (char)mc_dec(cmd.params[3], 8, 8);
obj_desc->type[2] = (char)mc_dec(cmd.params[3], 16, 8);
@@ -1108,7 +1111,7 @@
cmd.params[0] |= mc_enc(32, 8, irq_index);
cmd.params[0] |= mc_enc(0, 32, irq_cfg->val);
cmd.params[1] |= mc_enc(0, 64, irq_cfg->paddr);
- cmd.params[2] |= mc_enc(0, 32, irq_cfg->user_irq_id);
+ cmd.params[2] |= mc_enc(0, 32, irq_cfg->irq_num);
cmd.params[2] |= mc_enc(32, 32, obj_id);
cmd.params[3] |= mc_enc(0, 8, obj_type[0]);
cmd.params[3] |= mc_enc(8, 8, obj_type[1]);
@@ -1189,7 +1192,7 @@
/* retrieve response parameters */
irq_cfg->val = (u32)mc_dec(cmd.params[0], 0, 32);
irq_cfg->paddr = (u64)mc_dec(cmd.params[1], 0, 64);
- irq_cfg->user_irq_id = (int)mc_dec(cmd.params[2], 0, 32);
+ irq_cfg->irq_num = (int)mc_dec(cmd.params[2], 0, 32);
*type = (int)mc_dec(cmd.params[2], 32, 32);
return 0;
@@ -1437,14 +1440,8 @@
* @endpoint1: Endpoint 1 configuration parameters
* @endpoint2: Endpoint 2 configuration parameters
* @cfg: Connection configuration. The connection configuration is ignored for
- * connections made to DPMAC objects, where rate is set according to
- * MAC configuration.
- * The committed rate is the guaranteed rate for the connection.
- * The maximum rate is an upper limit allowed for the connection; it is
- * expected to be equal or higher than the committed rate.
- * When committed and maximum rates are both zero, the connection is set
- * to "best effort" mode, having lower priority compared to connections
- * with committed or maximum rates.
+ * connections made to DPMAC objects, where rate is retrieved from the
+ * MAC configuration.
*
* Return: '0' on Success; Error code otherwise.
*/
@@ -1555,7 +1552,10 @@
* @token: Token of DPRC object
* @endpoint1: Endpoint 1 configuration parameters
* @endpoint2: Returned endpoint 2 configuration parameters
-* @state: Returned link state: 1 - link is up, 0 - link is down
+* @state: Returned link state:
+* 1 - link is up;
+* 0 - link is down;
+* -1 - no connection (endpoint2 information is irrelevant)
*
* Return: '0' on Success; -ENAVAIL if connection does not exist.
*/
diff --git a/drivers/staging/fsl-mc/bus/mc-allocator.c b/drivers/staging/fsl-mc/bus/mc-allocator.c
index 86f8543..fb08f22 100644
--- a/drivers/staging/fsl-mc/bus/mc-allocator.c
+++ b/drivers/staging/fsl-mc/bus/mc-allocator.c
@@ -39,7 +39,6 @@
struct fsl_mc_resource *resource;
struct fsl_mc_device *mc_bus_dev = &mc_bus->mc_dev;
int error = -EINVAL;
- bool mutex_locked = false;
if (WARN_ON(pool_type < 0 || pool_type >= FSL_MC_NUM_POOL_TYPES))
goto out;
@@ -55,13 +54,12 @@
goto out;
mutex_lock(&res_pool->mutex);
- mutex_locked = true;
if (WARN_ON(res_pool->max_count < 0))
- goto out;
+ goto out_unlock;
if (WARN_ON(res_pool->free_count < 0 ||
res_pool->free_count > res_pool->max_count))
- goto out;
+ goto out_unlock;
resource = devm_kzalloc(&mc_bus_dev->dev, sizeof(*resource),
GFP_KERNEL);
@@ -69,7 +67,7 @@
error = -ENOMEM;
dev_err(&mc_bus_dev->dev,
"Failed to allocate memory for fsl_mc_resource\n");
- goto out;
+ goto out_unlock;
}
resource->type = pool_type;
@@ -82,10 +80,9 @@
res_pool->free_count++;
res_pool->max_count++;
error = 0;
+out_unlock:
+ mutex_unlock(&res_pool->mutex);
out:
- if (mutex_locked)
- mutex_unlock(&res_pool->mutex);
-
return error;
}
@@ -106,7 +103,6 @@
struct fsl_mc_resource_pool *res_pool;
struct fsl_mc_resource *resource;
int error = -EINVAL;
- bool mutex_locked = false;
if (WARN_ON(!FSL_MC_IS_ALLOCATABLE(mc_dev->obj_desc.type)))
goto out;
@@ -122,13 +118,12 @@
goto out;
mutex_lock(&res_pool->mutex);
- mutex_locked = true;
if (WARN_ON(res_pool->max_count <= 0))
- goto out;
+ goto out_unlock;
if (WARN_ON(res_pool->free_count <= 0 ||
res_pool->free_count > res_pool->max_count))
- goto out;
+ goto out_unlock;
/*
* If the device is currently allocated, its resource is not
@@ -139,7 +134,7 @@
dev_err(&mc_bus_dev->dev,
"Device %s cannot be removed from resource pool\n",
dev_name(&mc_dev->dev));
- goto out;
+ goto out_unlock;
}
list_del(&resource->node);
@@ -150,10 +145,9 @@
devm_kfree(&mc_bus_dev->dev, resource);
mc_dev->resource = NULL;
error = 0;
+out_unlock:
+ mutex_unlock(&res_pool->mutex);
out:
- if (mutex_locked)
- mutex_unlock(&res_pool->mutex);
-
return error;
}
@@ -188,21 +182,19 @@
struct fsl_mc_resource *resource;
struct fsl_mc_device *mc_bus_dev = &mc_bus->mc_dev;
int error = -EINVAL;
- bool mutex_locked = false;
BUILD_BUG_ON(ARRAY_SIZE(fsl_mc_pool_type_strings) !=
FSL_MC_NUM_POOL_TYPES);
*new_resource = NULL;
if (WARN_ON(pool_type < 0 || pool_type >= FSL_MC_NUM_POOL_TYPES))
- goto error;
+ goto out;
res_pool = &mc_bus->resource_pools[pool_type];
if (WARN_ON(res_pool->mc_bus != mc_bus))
- goto error;
+ goto out;
mutex_lock(&res_pool->mutex);
- mutex_locked = true;
resource = list_first_entry_or_null(&res_pool->free_list,
struct fsl_mc_resource, node);
@@ -212,28 +204,26 @@
dev_err(&mc_bus_dev->dev,
"No more resources of type %s left\n",
fsl_mc_pool_type_strings[pool_type]);
- goto error;
+ goto out_unlock;
}
if (WARN_ON(resource->type != pool_type))
- goto error;
+ goto out_unlock;
if (WARN_ON(resource->parent_pool != res_pool))
- goto error;
+ goto out_unlock;
if (WARN_ON(res_pool->free_count <= 0 ||
res_pool->free_count > res_pool->max_count))
- goto error;
+ goto out_unlock;
list_del(&resource->node);
INIT_LIST_HEAD(&resource->node);
res_pool->free_count--;
+ error = 0;
+out_unlock:
mutex_unlock(&res_pool->mutex);
*new_resource = resource;
- return 0;
-error:
- if (mutex_locked)
- mutex_unlock(&res_pool->mutex);
-
+out:
return error;
}
EXPORT_SYMBOL_GPL(fsl_mc_resource_allocate);
@@ -241,26 +231,23 @@
void fsl_mc_resource_free(struct fsl_mc_resource *resource)
{
struct fsl_mc_resource_pool *res_pool;
- bool mutex_locked = false;
res_pool = resource->parent_pool;
if (WARN_ON(resource->type != res_pool->type))
- goto out;
+ return;
mutex_lock(&res_pool->mutex);
- mutex_locked = true;
if (WARN_ON(res_pool->free_count < 0 ||
res_pool->free_count >= res_pool->max_count))
- goto out;
+ goto out_unlock;
if (WARN_ON(!list_empty(&resource->node)))
- goto out;
+ goto out_unlock;
list_add_tail(&resource->node, &res_pool->free_list);
res_pool->free_count++;
-out:
- if (mutex_locked)
- mutex_unlock(&res_pool->mutex);
+out_unlock:
+ mutex_unlock(&res_pool->mutex);
}
EXPORT_SYMBOL_GPL(fsl_mc_resource_free);
@@ -306,10 +293,22 @@
if (error < 0)
return error;
+ error = -EINVAL;
dpmcp_dev = resource->data;
if (WARN_ON(!dpmcp_dev))
goto error_cleanup_resource;
+ if (dpmcp_dev->obj_desc.ver_major < DPMCP_MIN_VER_MAJOR ||
+ (dpmcp_dev->obj_desc.ver_major == DPMCP_MIN_VER_MAJOR &&
+ dpmcp_dev->obj_desc.ver_minor < DPMCP_MIN_VER_MINOR)) {
+ dev_err(&dpmcp_dev->dev,
+ "ERROR: Version %d.%d of DPMCP not supported.\n",
+ dpmcp_dev->obj_desc.ver_major,
+ dpmcp_dev->obj_desc.ver_minor);
+ error = -ENOTSUPP;
+ goto error_cleanup_resource;
+ }
+
if (WARN_ON(dpmcp_dev->obj_desc.region_count == 0))
goto error_cleanup_resource;
@@ -722,20 +721,14 @@
{
.vendor = FSL_MC_VENDOR_FREESCALE,
.obj_type = "dpbp",
- .ver_major = DPBP_VER_MAJOR,
- .ver_minor = DPBP_VER_MINOR
},
{
.vendor = FSL_MC_VENDOR_FREESCALE,
.obj_type = "dpmcp",
- .ver_major = DPMCP_VER_MAJOR,
- .ver_minor = DPMCP_VER_MINOR
},
{
.vendor = FSL_MC_VENDOR_FREESCALE,
.obj_type = "dpcon",
- .ver_major = DPCON_VER_MAJOR,
- .ver_minor = DPCON_VER_MINOR
},
{.vendor = 0x0},
};
diff --git a/drivers/staging/fsl-mc/bus/mc-bus.c b/drivers/staging/fsl-mc/bus/mc-bus.c
index b594556..4053643 100644
--- a/drivers/staging/fsl-mc/bus/mc-bus.c
+++ b/drivers/staging/fsl-mc/bus/mc-bus.c
@@ -40,8 +40,6 @@
struct fsl_mc_device *mc_dev = to_fsl_mc_device(dev);
struct fsl_mc_driver *mc_drv = to_fsl_mc_driver(drv);
bool found = false;
- bool major_version_mismatch = false;
- bool minor_version_mismatch = false;
if (WARN_ON(!fsl_mc_bus_exists()))
goto out;
@@ -64,32 +62,12 @@
for (id = mc_drv->match_id_table; id->vendor != 0x0; id++) {
if (id->vendor == mc_dev->obj_desc.vendor &&
strcmp(id->obj_type, mc_dev->obj_desc.type) == 0) {
- if (id->ver_major == mc_dev->obj_desc.ver_major) {
- found = true;
- if (id->ver_minor != mc_dev->obj_desc.ver_minor)
- minor_version_mismatch = true;
- } else {
- major_version_mismatch = true;
- }
+ found = true;
break;
}
}
- if (major_version_mismatch) {
- dev_warn(dev,
- "Major version mismatch: driver version %u.%u, MC object version %u.%u\n",
- id->ver_major, id->ver_minor,
- mc_dev->obj_desc.ver_major,
- mc_dev->obj_desc.ver_minor);
- } else if (minor_version_mismatch) {
- dev_warn(dev,
- "Minor version mismatch: driver version %u.%u, MC object version %u.%u\n",
- id->ver_major, id->ver_minor,
- mc_dev->obj_desc.ver_major,
- mc_dev->obj_desc.ver_minor);
- }
-
out:
dev_dbg(dev, "%smatched\n", found ? "" : "not ");
return found;
@@ -251,11 +229,10 @@
return dev == root_dprc_dev;
}
-static int get_dprc_icid(struct fsl_mc_io *mc_io,
- int container_id, u16 *icid)
+static int get_dprc_attr(struct fsl_mc_io *mc_io,
+ int container_id, struct dprc_attributes *attr)
{
u16 dprc_handle;
- struct dprc_attributes attr;
int error;
error = dprc_open(mc_io, 0, container_id, &dprc_handle);
@@ -264,15 +241,14 @@
return error;
}
- memset(&attr, 0, sizeof(attr));
- error = dprc_get_attributes(mc_io, 0, dprc_handle, &attr);
+ memset(attr, 0, sizeof(struct dprc_attributes));
+ error = dprc_get_attributes(mc_io, 0, dprc_handle, attr);
if (error < 0) {
dev_err(mc_io->dev, "dprc_get_attributes() failed: %d\n",
error);
goto common_cleanup;
}
- *icid = attr.icid;
error = 0;
common_cleanup:
@@ -280,6 +256,34 @@
return error;
}
+static int get_dprc_icid(struct fsl_mc_io *mc_io,
+ int container_id, u16 *icid)
+{
+ struct dprc_attributes attr;
+ int error;
+
+ error = get_dprc_attr(mc_io, container_id, &attr);
+ if (error == 0)
+ *icid = attr.icid;
+
+ return error;
+}
+
+static int get_dprc_version(struct fsl_mc_io *mc_io,
+ int container_id, u16 *major, u16 *minor)
+{
+ struct dprc_attributes attr;
+ int error;
+
+ error = get_dprc_attr(mc_io, container_id, &attr);
+ if (error == 0) {
+ *major = attr.version.major;
+ *minor = attr.version.minor;
+ }
+
+ return error;
+}
+
static int translate_mc_addr(struct fsl_mc_device *mc_dev,
enum dprc_region_type mc_region_type,
u64 mc_offset, phys_addr_t *phys_addr)
@@ -376,6 +380,8 @@
regions[i].end = regions[i].start + region_desc.size - 1;
regions[i].name = "fsl-mc object MMIO region";
regions[i].flags = IORESOURCE_IO;
+ if (region_desc.flags & DPRC_REGION_CACHEABLE)
+ regions[i].flags |= IORESOURCE_CACHEABLE;
}
mc_dev->regions = regions;
@@ -491,6 +497,10 @@
goto error_cleanup_dev;
}
+ /* Objects are coherent, unless 'no shareability' flag set. */
+ if (!(obj_desc->flags & DPRC_OBJ_FLAG_NO_MEM_SHAREABILITY))
+ arch_setup_dma_ops(&mc_dev->dev, 0, 0, NULL, true);
+
/*
* The device-specific probe callback will get invoked by device_add()
*/
@@ -722,20 +732,6 @@
"Freescale Management Complex Firmware version: %u.%u.%u\n",
mc_version.major, mc_version.minor, mc_version.revision);
- if (mc_version.major < MC_VER_MAJOR) {
- dev_err(&pdev->dev,
- "ERROR: MC firmware version not supported by driver (driver version: %u.%u)\n",
- MC_VER_MAJOR, MC_VER_MINOR);
- error = -ENOTSUPP;
- goto error_cleanup_mc_io;
- }
-
- if (mc_version.major > MC_VER_MAJOR) {
- dev_warn(&pdev->dev,
- "WARNING: driver may not support newer MC firmware features (driver version: %u.%u)\n",
- MC_VER_MAJOR, MC_VER_MINOR);
- }
-
error = get_mc_addr_translation_ranges(&pdev->dev,
&mc->translation_ranges,
&mc->num_translation_ranges);
@@ -749,11 +745,15 @@
goto error_cleanup_mc_io;
}
+ memset(&obj_desc, 0, sizeof(struct dprc_obj_desc));
+ error = get_dprc_version(mc_io, container_id,
+ &obj_desc.ver_major, &obj_desc.ver_minor);
+ if (error < 0)
+ goto error_cleanup_mc_io;
+
obj_desc.vendor = FSL_MC_VENDOR_FREESCALE;
strcpy(obj_desc.type, "dprc");
obj_desc.id = container_id;
- obj_desc.ver_major = DPRC_VER_MAJOR;
- obj_desc.ver_minor = DPRC_VER_MINOR;
obj_desc.irq_count = 1;
obj_desc.region_count = 0;
diff --git a/drivers/staging/fsl-mc/bus/mc-msi.c b/drivers/staging/fsl-mc/bus/mc-msi.c
index 3a8258f..e202b2b 100644
--- a/drivers/staging/fsl-mc/bus/mc-msi.c
+++ b/drivers/staging/fsl-mc/bus/mc-msi.c
@@ -37,10 +37,8 @@
/*
* set_desc should not be set by the caller
*/
- if (WARN_ON(ops->set_desc))
- return;
-
- ops->set_desc = fsl_mc_msi_set_desc;
+ if (ops->set_desc == NULL)
+ ops->set_desc = fsl_mc_msi_set_desc;
}
static void __fsl_mc_msi_write_msg(struct fsl_mc_device *mc_bus_dev,
@@ -65,7 +63,7 @@
irq_cfg.paddr = ((u64)msi_desc->msg.address_hi << 32) |
msi_desc->msg.address_lo;
irq_cfg.val = msi_desc->msg.data;
- irq_cfg.user_irq_id = msi_desc->irq;
+ irq_cfg.irq_num = msi_desc->irq;
if (owner_mc_dev == mc_bus_dev) {
/*
@@ -129,10 +127,8 @@
/*
* irq_write_msi_msg should not be set by the caller
*/
- if (WARN_ON(chip->irq_write_msi_msg))
- return;
-
- chip->irq_write_msi_msg = fsl_mc_msi_write_msg;
+ if (chip->irq_write_msi_msg == NULL)
+ chip->irq_write_msi_msg = fsl_mc_msi_write_msg;
}
/**
diff --git a/drivers/staging/fsl-mc/include/dpbp-cmd.h b/drivers/staging/fsl-mc/include/dpbp-cmd.h
index efa9bf3..c57b454 100644
--- a/drivers/staging/fsl-mc/include/dpbp-cmd.h
+++ b/drivers/staging/fsl-mc/include/dpbp-cmd.h
@@ -34,7 +34,7 @@
/* DPBP Version */
#define DPBP_VER_MAJOR 2
-#define DPBP_VER_MINOR 1
+#define DPBP_VER_MINOR 2
/* Command IDs */
#define DPBP_CMDID_CLOSE 0x800
@@ -57,4 +57,6 @@
#define DPBP_CMDID_GET_IRQ_STATUS 0x016
#define DPBP_CMDID_CLEAR_IRQ_STATUS 0x017
+#define DPBP_CMDID_SET_NOTIFICATIONS 0x01b0
+#define DPBP_CMDID_GET_NOTIFICATIONS 0x01b1
#endif /* _FSL_DPBP_CMD_H */
diff --git a/drivers/staging/fsl-mc/include/dpbp.h b/drivers/staging/fsl-mc/include/dpbp.h
index 37ed951..e14e85a 100644
--- a/drivers/staging/fsl-mc/include/dpbp.h
+++ b/drivers/staging/fsl-mc/include/dpbp.h
@@ -85,12 +85,12 @@
* struct dpbp_irq_cfg - IRQ configuration
* @addr: Address that must be written to signal a message-based interrupt
* @val: Value to write into irq_addr address
- * @user_irq_id: A user defined number associated with this IRQ
+ * @irq_num: A user defined number associated with this IRQ
*/
struct dpbp_irq_cfg {
u64 addr;
u32 val;
- int user_irq_id;
+ int irq_num;
};
int dpbp_set_irq(struct fsl_mc_io *mc_io,
@@ -168,6 +168,53 @@
u16 token,
struct dpbp_attr *attr);
+/**
+ * DPBP notifications options
+ */
+
+/**
+ * BPSCN write will attempt to allocate into a cache (coherent write)
+ */
+#define DPBP_NOTIF_OPT_COHERENT_WRITE 0x00000001
+
+/**
+ * struct dpbp_notification_cfg - Structure representing DPBP notifications
+ * towards software
+ * @depletion_entry: below this threshold the pool is "depleted";
+ * set it to '0' to disable it
+ * @depletion_exit: greater than or equal to this threshold the pool exit its
+ * "depleted" state
+ * @surplus_entry: above this threshold the pool is in "surplus" state;
+ * set it to '0' to disable it
+ * @surplus_exit: less than or equal to this threshold the pool exit its
+ * "surplus" state
+ * @message_iova: MUST be given if either 'depletion_entry' or 'surplus_entry'
+ * is not '0' (enable); I/O virtual address (must be in DMA-able memory),
+ * must be 16B aligned.
+ * @message_ctx: The context that will be part of the BPSCN message and will
+ * be written to 'message_iova'
+ * @options: Mask of available options; use 'DPBP_NOTIF_OPT_<X>' values
+ */
+struct dpbp_notification_cfg {
+ u32 depletion_entry;
+ u32 depletion_exit;
+ u32 surplus_entry;
+ u32 surplus_exit;
+ u64 message_iova;
+ u64 message_ctx;
+ u16 options;
+};
+
+int dpbp_set_notifications(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ struct dpbp_notification_cfg *cfg);
+
+int dpbp_get_notifications(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ struct dpbp_notification_cfg *cfg);
+
/** @} */
#endif /* __FSL_DPBP_H */
diff --git a/drivers/staging/fsl-mc/include/dprc.h b/drivers/staging/fsl-mc/include/dprc.h
index 94c4927..593b2bb 100644
--- a/drivers/staging/fsl-mc/include/dprc.h
+++ b/drivers/staging/fsl-mc/include/dprc.h
@@ -94,11 +94,6 @@
*/
#define DPRC_CFG_OPT_TOPOLOGY_CHANGES_ALLOWED 0x00000008
-/* IOMMU bypass - indicates whether objects of this container are permitted
- * to bypass the IOMMU.
- */
-#define DPRC_CFG_OPT_IOMMU_BYPASS 0x00000010
-
/* AIOP - Indicates that container belongs to AIOP. */
#define DPRC_CFG_OPT_AIOP 0x00000020
@@ -173,12 +168,12 @@
* struct dprc_irq_cfg - IRQ configuration
* @paddr: Address that must be written to signal a message-based interrupt
* @val: Value to write into irq_addr address
- * @user_irq_id: A user defined number associated with this IRQ
+ * @irq_num: A user defined number associated with this IRQ
*/
struct dprc_irq_cfg {
phys_addr_t paddr;
u32 val;
- int user_irq_id;
+ int irq_num;
};
int dprc_set_irq(struct fsl_mc_io *mc_io,
@@ -353,6 +348,14 @@
#define DPRC_OBJ_STATE_PLUGGED 0x00000002
/**
+ * Shareability flag - Object flag indicating no memory shareability.
+ * the object generates memory accesses that are non coherent with other
+ * masters;
+ * user is responsible for proper memory handling through IOMMU configuration.
+ */
+#define DPRC_OBJ_FLAG_NO_MEM_SHAREABILITY 0x0001
+
+/**
* struct dprc_obj_desc - Object descriptor, returned from dprc_get_obj()
* @type: Type of object: NULL terminated string
* @id: ID of logical object resource
@@ -363,6 +366,7 @@
* @region_count: Number of mappable regions supported by the object
* @state: Object state: combination of DPRC_OBJ_STATE_ states
* @label: Object label
+ * @flags: Object's flags
*/
struct dprc_obj_desc {
char type[16];
@@ -374,6 +378,7 @@
u8 region_count;
u32 state;
char label[16];
+ u16 flags;
};
int dprc_get_obj(struct fsl_mc_io *mc_io,
diff --git a/drivers/staging/fsl-mc/include/mc-private.h b/drivers/staging/fsl-mc/include/mc-private.h
index ee5f1d2..cab1ae9 100644
--- a/drivers/staging/fsl-mc/include/mc-private.h
+++ b/drivers/staging/fsl-mc/include/mc-private.h
@@ -94,12 +94,14 @@
* from the physical DPRC.
* @irq_resources: Pointer to array of IRQ objects for the IRQ pool
* @scan_mutex: Serializes bus scanning
+ * @dprc_attr: DPRC attributes
*/
struct fsl_mc_bus {
struct fsl_mc_device mc_dev;
struct fsl_mc_resource_pool resource_pools[FSL_MC_NUM_POOL_TYPES];
struct fsl_mc_device_irq *irq_resources;
struct mutex scan_mutex; /* serializes bus scanning */
+ struct dprc_attributes dprc_attr;
};
#define to_fsl_mc_bus(_mc_dev) \
diff --git a/drivers/staging/fwserial/dma_fifo.c b/drivers/staging/fwserial/dma_fifo.c
index 4cd3ed3..8b23a55 100644
--- a/drivers/staging/fwserial/dma_fifo.c
+++ b/drivers/staging/fwserial/dma_fifo.c
@@ -35,7 +35,7 @@
/*
* private helper fn to determine if check is in open interval (lo,hi)
*/
-static bool addr_check(unsigned check, unsigned lo, unsigned hi)
+static bool addr_check(unsigned int check, unsigned int lo, unsigned int hi)
{
return check - (lo + 1) < (hi - 1) - lo;
}
@@ -64,7 +64,7 @@
* The 'apparent' size will be rounded up to next greater aligned size.
* Returns 0 if no error, otherwise an error code
*/
-int dma_fifo_alloc(struct dma_fifo *fifo, int size, unsigned align,
+int dma_fifo_alloc(struct dma_fifo *fifo, int size, unsigned int align,
int tx_limit, int open_limit, gfp_t gfp_mask)
{
int capacity;
@@ -190,7 +190,7 @@
*/
int dma_fifo_out_pend(struct dma_fifo *fifo, struct dma_pending *pended)
{
- unsigned len, n, ofs, l, limit;
+ unsigned int len, n, ofs, l, limit;
if (!fifo->data)
return -ENOENT;
@@ -210,7 +210,7 @@
n = len;
ofs = fifo->out % fifo->capacity;
l = fifo->capacity - ofs;
- limit = min_t(unsigned, l, fifo->tx_limit);
+ limit = min_t(unsigned int, l, fifo->tx_limit);
if (n > limit) {
n = limit;
fifo->out += limit;
diff --git a/drivers/staging/fwserial/dma_fifo.h b/drivers/staging/fwserial/dma_fifo.h
index 4109882..37a91c6 100644
--- a/drivers/staging/fwserial/dma_fifo.h
+++ b/drivers/staging/fwserial/dma_fifo.h
@@ -45,9 +45,9 @@
#define DMA_FIFO_GUARD 3 /* # of cache lines to reserve for the guard area */
struct dma_fifo {
- unsigned in;
- unsigned out; /* updated when dma is pended */
- unsigned done; /* updated upon dma completion */
+ unsigned int in;
+ unsigned int out; /* updated when dma is pended */
+ unsigned int done; /* updated upon dma completion */
struct {
unsigned corrupt:1;
};
@@ -55,7 +55,7 @@
int guard; /* ofs of guard area */
int capacity; /* size + reserved */
int avail; /* # of unused bytes in fifo */
- unsigned align; /* must be power of 2 */
+ unsigned int align; /* must be power of 2 */
int tx_limit; /* max # of bytes per dma transaction */
int open_limit; /* max # of outstanding allowed */
int open; /* # of outstanding dma transactions */
@@ -66,9 +66,9 @@
struct dma_pending {
struct list_head link;
void *data;
- unsigned len;
- unsigned next;
- unsigned out;
+ unsigned int len;
+ unsigned int next;
+ unsigned int out;
};
static inline void dp_mark_completed(struct dma_pending *dp)
@@ -82,7 +82,7 @@
}
void dma_fifo_init(struct dma_fifo *fifo);
-int dma_fifo_alloc(struct dma_fifo *fifo, int size, unsigned align,
+int dma_fifo_alloc(struct dma_fifo *fifo, int size, unsigned int align,
int tx_limit, int open_limit, gfp_t gfp_mask);
void dma_fifo_free(struct dma_fifo *fifo);
void dma_fifo_reset(struct dma_fifo *fifo);
diff --git a/drivers/staging/fwserial/fwserial.c b/drivers/staging/fwserial/fwserial.c
index 9b23b5c..c241c0a 100644
--- a/drivers/staging/fwserial/fwserial.c
+++ b/drivers/staging/fwserial/fwserial.c
@@ -132,7 +132,7 @@
#ifdef FWTTY_PROFILING
-static void fwtty_profile_fifo(struct fwtty_port *port, unsigned *stat)
+static void fwtty_profile_fifo(struct fwtty_port *port, unsigned int *stat)
{
spin_lock_bh(&port->lock);
fwtty_profile_data(stat, dma_fifo_avail(&port->tx_fifo));
@@ -143,7 +143,7 @@
{
/* for each stat, print sum of 0 to 2^k, then individually */
int k = 4;
- unsigned sum;
+ unsigned int sum;
int j;
char t[10];
@@ -303,9 +303,10 @@
* Note: in loopback, the port->lock is being held. Only use functions that
* don't attempt to reclaim the port->lock.
*/
-static void fwtty_update_port_status(struct fwtty_port *port, unsigned status)
+static void fwtty_update_port_status(struct fwtty_port *port,
+ unsigned int status)
{
- unsigned delta;
+ unsigned int delta;
struct tty_struct *tty;
/* simulated LSR/MSR status from remote */
@@ -396,9 +397,9 @@
*
* Note: caller must be holding port lock
*/
-static unsigned __fwtty_port_line_status(struct fwtty_port *port)
+static unsigned int __fwtty_port_line_status(struct fwtty_port *port)
{
- unsigned status = 0;
+ unsigned int status = 0;
/* TODO: add module param to tie RNG to DTR as well */
@@ -424,7 +425,7 @@
{
struct fwtty_peer *peer;
int err = -ENOENT;
- unsigned status = __fwtty_port_line_status(port);
+ unsigned int status = __fwtty_port_line_status(port);
rcu_read_lock();
peer = rcu_dereference(port->peer);
@@ -454,7 +455,7 @@
static void fwtty_throttle_port(struct fwtty_port *port)
{
struct tty_struct *tty;
- unsigned old;
+ unsigned int old;
tty = tty_port_tty_get(&port->port);
if (!tty)
@@ -540,7 +541,7 @@
static int fwtty_rx(struct fwtty_port *port, unsigned char *data, size_t len)
{
int c, n = len;
- unsigned lsr;
+ unsigned int lsr;
int err = 0;
fwtty_dbg(port, "%d\n", n);
@@ -635,7 +636,7 @@
if (addr != port->rx_handler.offset || len != 4) {
rcode = RCODE_ADDRESS_ERROR;
} else {
- fwtty_update_port_status(port, *(unsigned *)data);
+ fwtty_update_port_status(port, *(unsigned int *)data);
rcode = RCODE_COMPLETE;
}
break;
@@ -828,7 +829,7 @@
rcu_read_unlock();
}
-static struct fwtty_port *fwtty_port_get(unsigned index)
+static struct fwtty_port *fwtty_port_get(unsigned int index)
{
struct fwtty_port *port;
@@ -934,9 +935,9 @@
return rc;
}
-static unsigned set_termios(struct fwtty_port *port, struct tty_struct *tty)
+static unsigned int set_termios(struct fwtty_port *port, struct tty_struct *tty)
{
- unsigned baud, frame;
+ unsigned int baud, frame;
baud = tty_termios_baud_rate(&tty->termios);
tty_termios_encode_baud_rate(&tty->termios, baud, baud);
@@ -988,7 +989,7 @@
struct tty_struct *tty)
{
struct fwtty_port *port = to_port(tty_port, port);
- unsigned baud;
+ unsigned int baud;
int err;
set_bit(TTY_IO_ERROR, &tty->flags);
@@ -1264,7 +1265,7 @@
return 0;
}
-static int fwtty_ioctl(struct tty_struct *tty, unsigned cmd,
+static int fwtty_ioctl(struct tty_struct *tty, unsigned int cmd,
unsigned long arg)
{
struct fwtty_port *port = tty->driver_data;
@@ -1297,7 +1298,7 @@
static void fwtty_set_termios(struct tty_struct *tty, struct ktermios *old)
{
struct fwtty_port *port = tty->driver_data;
- unsigned baud;
+ unsigned int baud;
spin_lock_bh(&port->lock);
baud = set_termios(port, tty);
@@ -1305,7 +1306,7 @@
if ((baud == 0) && (old->c_cflag & CBAUD)) {
port->mctrl &= ~(TIOCM_DTR | TIOCM_RTS);
} else if ((baud != 0) && !(old->c_cflag & CBAUD)) {
- if (C_CRTSCTS(tty) || !test_bit(TTY_THROTTLED, &tty->flags))
+ if (C_CRTSCTS(tty) || !tty_throttled(tty))
port->mctrl |= TIOCM_DTR | TIOCM_RTS;
else
port->mctrl |= TIOCM_DTR;
@@ -1369,7 +1370,7 @@
static int fwtty_tiocmget(struct tty_struct *tty)
{
struct fwtty_port *port = tty->driver_data;
- unsigned tiocm;
+ unsigned int tiocm;
spin_lock_bh(&port->lock);
tiocm = (port->mctrl & MCTRL_MASK) | (port->mstatus & ~MCTRL_MASK);
@@ -1380,7 +1381,8 @@
return tiocm;
}
-static int fwtty_tiocmset(struct tty_struct *tty, unsigned set, unsigned clear)
+static int fwtty_tiocmset(struct tty_struct *tty,
+ unsigned int set, unsigned int clear)
{
struct fwtty_port *port = tty->driver_data;
@@ -1699,7 +1701,7 @@
dma_fifo_change_tx_limit(&port->tx_fifo, port->max_payload);
spin_unlock_bh(&peer->port->lock);
- if (port->port.console && port->fwcon_ops->notify != NULL)
+ if (port->port.console && port->fwcon_ops->notify)
(*port->fwcon_ops->notify)(FWCON_NOTIFY_ATTACH, port->con_data);
fwtty_info(&peer->unit, "peer (guid:%016llx) connected on %s\n",
@@ -1806,7 +1808,7 @@
RCU_INIT_POINTER(port->peer, NULL);
spin_unlock_bh(&port->lock);
- if (port->port.console && port->fwcon_ops->notify != NULL)
+ if (port->port.console && port->fwcon_ops->notify)
(*port->fwcon_ops->notify)(FWCON_NOTIFY_DETACH, port->con_data);
}
diff --git a/drivers/staging/fwserial/fwserial.h b/drivers/staging/fwserial/fwserial.h
index 6fa9365..30b2481 100644
--- a/drivers/staging/fwserial/fwserial.h
+++ b/drivers/staging/fwserial/fwserial.h
@@ -22,7 +22,7 @@
#ifdef FWTTY_PROFILING
#define DISTRIBUTION_MAX_SIZE 8192
#define DISTRIBUTION_MAX_INDEX (ilog2(DISTRIBUTION_MAX_SIZE) + 1)
-static inline void fwtty_profile_data(unsigned stat[], unsigned val)
+static inline void fwtty_profile_data(unsigned int stat[], unsigned int val)
{
int n = (val) ? min(ilog2(val) + 1, DISTRIBUTION_MAX_INDEX) : 0;
++stat[n];
@@ -78,7 +78,7 @@
u64 guid;
int generation;
int node_id;
- unsigned speed;
+ unsigned int speed;
int max_payload;
u64 mgmt_addr;
@@ -160,17 +160,17 @@
#define VIRT_CABLE_PLUG_TIMEOUT (60 * HZ)
struct stats {
- unsigned xchars;
- unsigned dropped;
- unsigned tx_stall;
- unsigned fifo_errs;
- unsigned sent;
- unsigned lost;
- unsigned throttled;
- unsigned reads[DISTRIBUTION_MAX_INDEX + 1];
- unsigned writes[DISTRIBUTION_MAX_INDEX + 1];
- unsigned txns[DISTRIBUTION_MAX_INDEX + 1];
- unsigned unthrottle[DISTRIBUTION_MAX_INDEX + 1];
+ unsigned int xchars;
+ unsigned int dropped;
+ unsigned int tx_stall;
+ unsigned int fifo_errs;
+ unsigned int sent;
+ unsigned int lost;
+ unsigned int throttled;
+ unsigned int reads[DISTRIBUTION_MAX_INDEX + 1];
+ unsigned int writes[DISTRIBUTION_MAX_INDEX + 1];
+ unsigned int txns[DISTRIBUTION_MAX_INDEX + 1];
+ unsigned int unthrottle[DISTRIBUTION_MAX_INDEX + 1];
};
struct fwconsole_ops {
@@ -237,7 +237,7 @@
struct fwtty_port {
struct tty_port port;
struct device *device;
- unsigned index;
+ unsigned int index;
struct fw_serial *serial;
struct fw_address_handler rx_handler;
@@ -246,21 +246,21 @@
wait_queue_head_t wait_tx;
struct delayed_work emit_breaks;
- unsigned cps;
+ unsigned int cps;
unsigned long break_last;
struct work_struct hangup;
- unsigned mstatus;
+ unsigned int mstatus;
spinlock_t lock;
- unsigned mctrl;
+ unsigned int mctrl;
struct delayed_work drain;
struct dma_fifo tx_fifo;
int max_payload;
- unsigned status_mask;
- unsigned ignore_mask;
- unsigned break_ctl:1,
+ unsigned int status_mask;
+ unsigned int ignore_mask;
+ unsigned int break_ctl:1,
write_only:1,
overrun:1,
loopback:1;
@@ -349,7 +349,7 @@
* being used for isochronous traffic)
* 2) isochronous arbitration always wins.
*/
-static inline int link_speed_to_max_payload(unsigned speed)
+static inline int link_speed_to_max_payload(unsigned int speed)
{
/* Max async payload is 4096 - see IEEE 1394-2008 tables 6-4, 16-18 */
return min(512 << speed, 4096);
diff --git a/drivers/staging/gdm724x/gdm_mux.c b/drivers/staging/gdm724x/gdm_mux.c
index 6bedd66..4009691 100644
--- a/drivers/staging/gdm724x/gdm_mux.c
+++ b/drivers/staging/gdm724x/gdm_mux.c
@@ -278,8 +278,9 @@
}
}
-static int gdm_mux_recv(void *priv_dev, int (*cb)(void *data, int len,
- int tty_index, struct tty_dev *tty_dev, int complete))
+static int gdm_mux_recv(void *priv_dev,
+ int (*cb)(void *data, int len, int tty_index,
+ struct tty_dev *tty_dev, int complete))
{
struct mux_dev *mux_dev = priv_dev;
struct usb_device *usbdev = mux_dev->usbdev;
diff --git a/drivers/staging/gdm724x/gdm_usb.c b/drivers/staging/gdm724x/gdm_usb.c
index 9db9b903..d650d77 100644
--- a/drivers/staging/gdm724x/gdm_usb.c
+++ b/drivers/staging/gdm724x/gdm_usb.c
@@ -708,7 +708,7 @@
#define SDU_PARAM_LEN 12
static int gdm_usb_sdu_send(void *priv_dev, void *data, int len,
- unsigned int dftEpsId, unsigned int epsId,
+ unsigned int dft_eps_ID, unsigned int eps_ID,
void (*cb)(void *data), void *cb_data,
int dev_idx, int nic_type)
{
@@ -746,8 +746,8 @@
}
sdu->len = gdm_cpu_to_dev16(&udev->gdm_ed, send_len);
- sdu->dftEpsId = gdm_cpu_to_dev32(&udev->gdm_ed, dftEpsId);
- sdu->bearer_ID = gdm_cpu_to_dev32(&udev->gdm_ed, epsId);
+ sdu->dft_eps_ID = gdm_cpu_to_dev32(&udev->gdm_ed, dft_eps_ID);
+ sdu->bearer_ID = gdm_cpu_to_dev32(&udev->gdm_ed, eps_ID);
sdu->nic_type = gdm_cpu_to_dev32(&udev->gdm_ed, nic_type);
t_sdu->len = send_len + HCI_HEADER_SIZE;
diff --git a/drivers/staging/gdm724x/hci_packet.h b/drivers/staging/gdm724x/hci_packet.h
index 7fba8a6..dbc4446 100644
--- a/drivers/staging/gdm724x/hci_packet.h
+++ b/drivers/staging/gdm724x/hci_packet.h
@@ -58,7 +58,7 @@
struct sdu {
u16 cmd_evt;
u16 len;
- u32 dftEpsId;
+ u32 dft_eps_ID;
u32 bearer_ID;
u32 nic_type;
u8 data[0];
diff --git a/drivers/staging/gdm724x/netlink_k.c b/drivers/staging/gdm724x/netlink_k.c
index 9d83477..a0232e8 100644
--- a/drivers/staging/gdm724x/netlink_k.c
+++ b/drivers/staging/gdm724x/netlink_k.c
@@ -88,7 +88,8 @@
}
struct sock *netlink_init(int unit,
- void (*cb)(struct net_device *dev, u16 type, void *msg, int len))
+ void (*cb)(struct net_device *dev, u16 type,
+ void *msg, int len))
{
struct sock *sock;
struct netlink_kernel_cfg cfg = {
diff --git a/drivers/staging/gs_fpgaboot/gs_fpgaboot.c b/drivers/staging/gs_fpgaboot/gs_fpgaboot.c
index 7b7c978..a221f26 100644
--- a/drivers/staging/gs_fpgaboot/gs_fpgaboot.c
+++ b/drivers/staging/gs_fpgaboot/gs_fpgaboot.c
@@ -93,7 +93,6 @@
return 0;
}
-
/*
* read first 13 bytes to check bitstream magic number
*/
@@ -201,7 +200,7 @@
#endif /* DEBUG_FPGA */
if (!xl_supported_prog_bus_width(bus_bytes)) {
pr_err("unsupported program bus width %d\n",
- bus_bytes);
+ bus_bytes);
return -1;
}
@@ -222,7 +221,7 @@
pr_info("device init done\n");
for (i = 0; i < size; i += bus_bytes)
- xl_shift_bytes_out(bus_bytes, bitdata+i);
+ xl_shift_bytes_out(bus_bytes, bitdata + i);
pr_info("program done\n");
@@ -277,7 +276,7 @@
static int init_driver(void)
{
firmware_pdev = platform_device_register_simple("fpgaboot", -1,
- NULL, 0);
+ NULL, 0);
return PTR_ERR_OR_ZERO(firmware_pdev);
}
@@ -331,7 +330,6 @@
kfree(fimage);
return -1;
-
}
static int __init gs_fpgaboot_init(void)
diff --git a/drivers/staging/gs_fpgaboot/gs_fpgaboot.h b/drivers/staging/gs_fpgaboot/gs_fpgaboot.h
index f41f4cc..8cc3255 100644
--- a/drivers/staging/gs_fpgaboot/gs_fpgaboot.h
+++ b/drivers/staging/gs_fpgaboot/gs_fpgaboot.h
@@ -51,6 +51,6 @@
char part[MAX_STR];
char date[MAX_STR];
char time[MAX_STR];
- int32_t lendata;
+ int lendata;
char *fpgadata;
};
diff --git a/drivers/staging/gs_fpgaboot/io.c b/drivers/staging/gs_fpgaboot/io.c
index 819db53..c939119 100644
--- a/drivers/staging/gs_fpgaboot/io.c
+++ b/drivers/staging/gs_fpgaboot/io.c
@@ -35,7 +35,6 @@
static inline void byte1_out(unsigned char data);
static inline void xl_cclk_b(int32_t i);
-
/* Assert and Deassert CCLK */
void xl_shift_cclk(int count)
{
diff --git a/drivers/staging/i4l/act2000/act2000_isa.c b/drivers/staging/i4l/act2000/act2000_isa.c
index b5fad29..f0eb844 100644
--- a/drivers/staging/i4l/act2000/act2000_isa.c
+++ b/drivers/staging/i4l/act2000/act2000_isa.c
@@ -31,7 +31,8 @@
int serial = 0;
found = 0;
- if ((reg = inb(portbase + ISA_COR)) != 0xff) {
+ reg = inb(portbase + ISA_COR);
+ if (reg != 0xff) {
outb(reg | ISA_COR_RESET, portbase + ISA_COR);
mdelay(10);
outb(reg, portbase + ISA_COR);
@@ -232,7 +233,7 @@
{
u_char c;
- if (test_and_set_bit(ACT2000_LOCK_RX, (void *) &card->ilock) != 0)
+ if (test_and_set_bit(ACT2000_LOCK_RX, (void *)&card->ilock) != 0)
return;
while (!act2000_isa_readb(card, &c)) {
if (card->idat.isa.rcvidx < 8) {
@@ -247,7 +248,7 @@
card->idat.isa.rcvignore = 1;
printk(KERN_WARNING
"act2000_isa_receive: no memory\n");
- test_and_clear_bit(ACT2000_LOCK_RX, (void *) &card->ilock);
+ test_and_clear_bit(ACT2000_LOCK_RX, (void *)&card->ilock);
return;
}
memcpy(skb_put(card->idat.isa.rcvskb, 8), card->idat.isa.rcvhdr, 8);
@@ -287,7 +288,7 @@
(card->idat.isa.rcvidx < card->idat.isa.rcvlen)))
act2000_schedule_poll(card);
}
- test_and_clear_bit(ACT2000_LOCK_RX, (void *) &card->ilock);
+ test_and_clear_bit(ACT2000_LOCK_RX, (void *)&card->ilock);
}
void
@@ -298,12 +299,13 @@
actcapi_msg *msg;
int l;
- if (test_and_set_bit(ACT2000_LOCK_TX, (void *) &card->ilock) != 0)
+ if (test_and_set_bit(ACT2000_LOCK_TX, (void *)&card->ilock) != 0)
return;
while (1) {
spin_lock_irqsave(&card->lock, flags);
if (!(card->sbuf)) {
- if ((card->sbuf = skb_dequeue(&card->sndq))) {
+ card->sbuf = skb_dequeue(&card->sndq);
+ if (card->sbuf) {
card->ack_msg = card->sbuf->data;
msg = (actcapi_msg *)card->sbuf->data;
if ((msg->hdr.cmd.cmd == 0x86) &&
@@ -317,7 +319,7 @@
spin_unlock_irqrestore(&card->lock, flags);
if (!(card->sbuf)) {
/* No more data to send */
- test_and_clear_bit(ACT2000_LOCK_TX, (void *) &card->ilock);
+ test_and_clear_bit(ACT2000_LOCK_TX, (void *)&card->ilock);
return;
}
skb = card->sbuf;
@@ -325,7 +327,7 @@
while (skb->len) {
if (act2000_isa_writeb(card, *(skb->data))) {
/* Fifo is full, but more data to send */
- test_and_clear_bit(ACT2000_LOCK_TX, (void *) &card->ilock);
+ test_and_clear_bit(ACT2000_LOCK_TX, (void *)&card->ilock);
/* Schedule myself */
act2000_schedule_tx(card);
return;
@@ -356,7 +358,6 @@
static int
act2000_isa_getid(act2000_card *card)
{
-
act2000_fwid fid;
u_char *p = (u_char *)&fid;
int count = 0;
@@ -378,7 +379,8 @@
printk(KERN_WARNING "act2000: Wrong Firmware-ID!\n");
return -EPROTO;
}
- if ((p = strchr(fid.revision, '\n')))
+ p = strchr(fid.revision, '\n');
+ if (p)
*p = '\0';
printk(KERN_INFO "act2000: Firmware-ID: %s\n", fid.revision);
if (card->flags & ACT2000_FLAGS_IVALID) {
@@ -439,5 +441,5 @@
}
kfree(buf);
msleep_interruptible(500);
- return (act2000_isa_getid(card));
+ return act2000_isa_getid(card);
}
diff --git a/drivers/staging/i4l/pcbit/capi.h b/drivers/staging/i4l/pcbit/capi.h
index 635f634..6f6f4dd 100644
--- a/drivers/staging/i4l/pcbit/capi.h
+++ b/drivers/staging/i4l/pcbit/capi.h
@@ -17,7 +17,7 @@
#define REQ_DISPLAY 0x04
#define REQ_USER_TO_USER 0x08
-#define AppInfoMask REQ_CAUSE | REQ_DISPLAY | REQ_USER_TO_USER
+#define AppInfoMask (REQ_CAUSE | REQ_DISPLAY | REQ_USER_TO_USER)
/* Connection Setup */
extern int capi_conn_req(const char *calledPN, struct sk_buff **buf,
diff --git a/drivers/staging/i4l/pcbit/drv.c b/drivers/staging/i4l/pcbit/drv.c
index 4172e22..c5270e2 100644
--- a/drivers/staging/i4l/pcbit/drv.c
+++ b/drivers/staging/i4l/pcbit/drv.c
@@ -284,7 +284,7 @@
default:
printk(KERN_DEBUG "pcbit_command: unknown command\n");
break;
- };
+ }
return 0;
}
@@ -699,8 +699,8 @@
*/
static char statbuf[STATBUF_LEN];
-static int stat_st = 0;
-static int stat_end = 0;
+static int stat_st;
+static int stat_end;
static int pcbit_stat(u_char __user *buf, int len, int driver, int channel)
{
@@ -968,7 +968,7 @@
default:
printk("error: unknown ioctl\n");
break;
- };
+ }
return 0;
}
diff --git a/drivers/staging/i4l/pcbit/edss1.c b/drivers/staging/i4l/pcbit/edss1.c
index b2262ba..e72c164 100644
--- a/drivers/staging/i4l/pcbit/edss1.c
+++ b/drivers/staging/i4l/pcbit/edss1.c
@@ -254,7 +254,7 @@
dev = chan2dev(chan);
- if (dev == NULL) {
+ if (!dev) {
printk(KERN_WARNING "pcbit: timer for unknown device\n");
return;
}
diff --git a/drivers/staging/i4l/pcbit/layer2.h b/drivers/staging/i4l/pcbit/layer2.h
index be1327b..6b9063e 100644
--- a/drivers/staging/i4l/pcbit/layer2.h
+++ b/drivers/staging/i4l/pcbit/layer2.h
@@ -109,7 +109,7 @@
#define SCHED_READ 0x01
#define SCHED_WRITE 0x02
-#define SET_RUN_TIMEOUT 2 * HZ /* 2 seconds */
+#define SET_RUN_TIMEOUT (2 * HZ) /* 2 seconds */
struct frame_buf {
ulong msg;
diff --git a/drivers/staging/iio/accel/Kconfig b/drivers/staging/iio/accel/Kconfig
index fa67da9..f066aa3 100644
--- a/drivers/staging/iio/accel/Kconfig
+++ b/drivers/staging/iio/accel/Kconfig
@@ -27,18 +27,6 @@
To compile this driver as a module, say M here: the module will be
called adis16203.
-config ADIS16204
- tristate "Analog Devices ADIS16204 Programmable High-g Digital Impact Sensor and Recorder"
- depends on SPI
- select IIO_ADIS_LIB
- select IIO_ADIS_LIB_BUFFER if IIO_BUFFER
- help
- Say Y here to build support for Analog Devices adis16204 Programmable
- High-g Digital Impact Sensor and Recorder.
-
- To compile this driver as a module, say M here: the module will be
- called adis16204.
-
config ADIS16209
tristate "Analog Devices ADIS16209 Dual-Axis Digital Inclinometer and Accelerometer"
depends on SPI
@@ -51,17 +39,6 @@
To compile this driver as a module, say M here: the module will be
called adis16209.
-config ADIS16220
- tristate "Analog Devices ADIS16220 Programmable Digital Vibration Sensor"
- depends on SPI
- select IIO_ADIS_LIB
- help
- Say Y here to build support for Analog Devices adis16220 programmable
- digital vibration sensor.
-
- To compile this driver as a module, say M here: the module will be
- called adis16220.
-
config ADIS16240
tristate "Analog Devices ADIS16240 Programmable Impact Sensor and Recorder"
depends on SPI
diff --git a/drivers/staging/iio/accel/Makefile b/drivers/staging/iio/accel/Makefile
index 1ed137f..415329c 100644
--- a/drivers/staging/iio/accel/Makefile
+++ b/drivers/staging/iio/accel/Makefile
@@ -8,15 +8,9 @@
adis16203-y := adis16203_core.o
obj-$(CONFIG_ADIS16203) += adis16203.o
-adis16204-y := adis16204_core.o
-obj-$(CONFIG_ADIS16204) += adis16204.o
-
adis16209-y := adis16209_core.o
obj-$(CONFIG_ADIS16209) += adis16209.o
-adis16220-y := adis16220_core.o
-obj-$(CONFIG_ADIS16220) += adis16220.o
-
adis16240-y := adis16240_core.o
obj-$(CONFIG_ADIS16240) += adis16240.o
diff --git a/drivers/staging/iio/accel/adis16201.h b/drivers/staging/iio/accel/adis16201.h
index e6b8c9a..64844ad 100644
--- a/drivers/staging/iio/accel/adis16201.h
+++ b/drivers/staging/iio/accel/adis16201.h
@@ -3,51 +3,129 @@
#define ADIS16201_STARTUP_DELAY 220 /* ms */
-#define ADIS16201_FLASH_CNT 0x00 /* Flash memory write count */
-#define ADIS16201_SUPPLY_OUT 0x02 /* Output, power supply */
-#define ADIS16201_XACCL_OUT 0x04 /* Output, x-axis accelerometer */
-#define ADIS16201_YACCL_OUT 0x06 /* Output, y-axis accelerometer */
-#define ADIS16201_AUX_ADC 0x08 /* Output, auxiliary ADC input */
-#define ADIS16201_TEMP_OUT 0x0A /* Output, temperature */
-#define ADIS16201_XINCL_OUT 0x0C /* Output, x-axis inclination */
-#define ADIS16201_YINCL_OUT 0x0E /* Output, y-axis inclination */
-#define ADIS16201_XACCL_OFFS 0x10 /* Calibration, x-axis acceleration offset */
-#define ADIS16201_YACCL_OFFS 0x12 /* Calibration, y-axis acceleration offset */
-#define ADIS16201_XACCL_SCALE 0x14 /* x-axis acceleration scale factor */
-#define ADIS16201_YACCL_SCALE 0x16 /* y-axis acceleration scale factor */
-#define ADIS16201_XINCL_OFFS 0x18 /* Calibration, x-axis inclination offset */
-#define ADIS16201_YINCL_OFFS 0x1A /* Calibration, y-axis inclination offset */
-#define ADIS16201_XINCL_SCALE 0x1C /* x-axis inclination scale factor */
-#define ADIS16201_YINCL_SCALE 0x1E /* y-axis inclination scale factor */
-#define ADIS16201_ALM_MAG1 0x20 /* Alarm 1 amplitude threshold */
-#define ADIS16201_ALM_MAG2 0x22 /* Alarm 2 amplitude threshold */
-#define ADIS16201_ALM_SMPL1 0x24 /* Alarm 1, sample period */
-#define ADIS16201_ALM_SMPL2 0x26 /* Alarm 2, sample period */
-#define ADIS16201_ALM_CTRL 0x28 /* Alarm control */
-#define ADIS16201_AUX_DAC 0x30 /* Auxiliary DAC data */
-#define ADIS16201_GPIO_CTRL 0x32 /* General-purpose digital input/output control */
-#define ADIS16201_MSC_CTRL 0x34 /* Miscellaneous control */
-#define ADIS16201_SMPL_PRD 0x36 /* Internal sample period (rate) control */
-#define ADIS16201_AVG_CNT 0x38 /* Operation, filter configuration */
-#define ADIS16201_SLP_CNT 0x3A /* Operation, sleep mode control */
-#define ADIS16201_DIAG_STAT 0x3C /* Diagnostics, system status register */
-#define ADIS16201_GLOB_CMD 0x3E /* Operation, system command register */
+/* Flash memory write count */
+#define ADIS16201_FLASH_CNT 0x00
+
+/* Output, power supply */
+#define ADIS16201_SUPPLY_OUT 0x02
+
+/* Output, x-axis accelerometer */
+#define ADIS16201_XACCL_OUT 0x04
+
+/* Output, y-axis accelerometer */
+#define ADIS16201_YACCL_OUT 0x06
+
+/* Output, auxiliary ADC input */
+#define ADIS16201_AUX_ADC 0x08
+
+/* Output, temperature */
+#define ADIS16201_TEMP_OUT 0x0A
+
+/* Output, x-axis inclination */
+#define ADIS16201_XINCL_OUT 0x0C
+
+/* Output, y-axis inclination */
+#define ADIS16201_YINCL_OUT 0x0E
+
+/* Calibration, x-axis acceleration offset */
+#define ADIS16201_XACCL_OFFS 0x10
+
+/* Calibration, y-axis acceleration offset */
+#define ADIS16201_YACCL_OFFS 0x12
+
+/* x-axis acceleration scale factor */
+#define ADIS16201_XACCL_SCALE 0x14
+
+/* y-axis acceleration scale factor */
+#define ADIS16201_YACCL_SCALE 0x16
+
+/* Calibration, x-axis inclination offset */
+#define ADIS16201_XINCL_OFFS 0x18
+
+/* Calibration, y-axis inclination offset */
+#define ADIS16201_YINCL_OFFS 0x1A
+
+/* x-axis inclination scale factor */
+#define ADIS16201_XINCL_SCALE 0x1C
+
+/* y-axis inclination scale factor */
+#define ADIS16201_YINCL_SCALE 0x1E
+
+/* Alarm 1 amplitude threshold */
+#define ADIS16201_ALM_MAG1 0x20
+
+/* Alarm 2 amplitude threshold */
+#define ADIS16201_ALM_MAG2 0x22
+
+/* Alarm 1, sample period */
+#define ADIS16201_ALM_SMPL1 0x24
+
+/* Alarm 2, sample period */
+#define ADIS16201_ALM_SMPL2 0x26
+
+/* Alarm control */
+#define ADIS16201_ALM_CTRL 0x28
+
+/* Auxiliary DAC data */
+#define ADIS16201_AUX_DAC 0x30
+
+/* General-purpose digital input/output control */
+#define ADIS16201_GPIO_CTRL 0x32
+
+/* Miscellaneous control */
+#define ADIS16201_MSC_CTRL 0x34
+
+/* Internal sample period (rate) control */
+#define ADIS16201_SMPL_PRD 0x36
+
+/* Operation, filter configuration */
+#define ADIS16201_AVG_CNT 0x38
+
+/* Operation, sleep mode control */
+#define ADIS16201_SLP_CNT 0x3A
+
+/* Diagnostics, system status register */
+#define ADIS16201_DIAG_STAT 0x3C
+
+/* Operation, system command register */
+#define ADIS16201_GLOB_CMD 0x3E
/* MSC_CTRL */
-#define ADIS16201_MSC_CTRL_SELF_TEST_EN BIT(8) /* Self-test enable */
-#define ADIS16201_MSC_CTRL_DATA_RDY_EN BIT(2) /* Data-ready enable: 1 = enabled, 0 = disabled */
-#define ADIS16201_MSC_CTRL_ACTIVE_HIGH BIT(1) /* Data-ready polarity: 1 = active high, 0 = active low */
-#define ADIS16201_MSC_CTRL_DATA_RDY_DIO1 BIT(0) /* Data-ready line selection: 1 = DIO1, 0 = DIO0 */
+
+/* Self-test enable */
+#define ADIS16201_MSC_CTRL_SELF_TEST_EN BIT(8)
+
+/* Data-ready enable: 1 = enabled, 0 = disabled */
+#define ADIS16201_MSC_CTRL_DATA_RDY_EN BIT(2)
+
+/* Data-ready polarity: 1 = active high, 0 = active low */
+#define ADIS16201_MSC_CTRL_ACTIVE_HIGH BIT(1)
+
+/* Data-ready line selection: 1 = DIO1, 0 = DIO0 */
+#define ADIS16201_MSC_CTRL_DATA_RDY_DIO1 BIT(0)
/* DIAG_STAT */
-#define ADIS16201_DIAG_STAT_ALARM2 BIT(9) /* Alarm 2 status: 1 = alarm active, 0 = alarm inactive */
-#define ADIS16201_DIAG_STAT_ALARM1 BIT(8) /* Alarm 1 status: 1 = alarm active, 0 = alarm inactive */
-#define ADIS16201_DIAG_STAT_SPI_FAIL_BIT 3 /* SPI communications failure */
-#define ADIS16201_DIAG_STAT_FLASH_UPT_BIT 2 /* Flash update failure */
-#define ADIS16201_DIAG_STAT_POWER_HIGH_BIT 1 /* Power supply above 3.625 V */
-#define ADIS16201_DIAG_STAT_POWER_LOW_BIT 0 /* Power supply below 3.15 V */
+
+/* Alarm 2 status: 1 = alarm active, 0 = alarm inactive */
+#define ADIS16201_DIAG_STAT_ALARM2 BIT(9)
+
+/* Alarm 1 status: 1 = alarm active, 0 = alarm inactive */
+#define ADIS16201_DIAG_STAT_ALARM1 BIT(8)
+
+/* SPI communications failure */
+#define ADIS16201_DIAG_STAT_SPI_FAIL_BIT 3
+
+/* Flash update failure */
+#define ADIS16201_DIAG_STAT_FLASH_UPT_BIT 2
+
+/* Power supply above 3.625 V */
+#define ADIS16201_DIAG_STAT_POWER_HIGH_BIT 1
+
+/* Power supply below 3.15 V */
+#define ADIS16201_DIAG_STAT_POWER_LOW_BIT 0
/* GLOB_CMD */
+
#define ADIS16201_GLOB_CMD_SW_RESET BIT(7)
#define ADIS16201_GLOB_CMD_FACTORY_CAL BIT(1)
diff --git a/drivers/staging/iio/accel/adis16201_core.c b/drivers/staging/iio/accel/adis16201_core.c
index 06c0b75..6f3f8ff 100644
--- a/drivers/staging/iio/accel/adis16201_core.c
+++ b/drivers/staging/iio/accel/adis16201_core.c
@@ -167,6 +167,7 @@
.diag_stat_reg = ADIS16201_DIAG_STAT,
.self_test_mask = ADIS16201_MSC_CTRL_SELF_TEST_EN,
+ .self_test_no_autoclear = true,
.startup_delay = ADIS16201_STARTUP_DELAY,
.status_error_msgs = adis16201_status_error_msgs,
diff --git a/drivers/staging/iio/accel/adis16203.h b/drivers/staging/iio/accel/adis16203.h
index 6426e38..b483e4e 100644
--- a/drivers/staging/iio/accel/adis16203.h
+++ b/drivers/staging/iio/accel/adis16203.h
@@ -3,45 +3,111 @@
#define ADIS16203_STARTUP_DELAY 220 /* ms */
-#define ADIS16203_FLASH_CNT 0x00 /* Flash memory write count */
-#define ADIS16203_SUPPLY_OUT 0x02 /* Output, power supply */
-#define ADIS16203_AUX_ADC 0x08 /* Output, auxiliary ADC input */
-#define ADIS16203_TEMP_OUT 0x0A /* Output, temperature */
-#define ADIS16203_XINCL_OUT 0x0C /* Output, x-axis inclination */
-#define ADIS16203_YINCL_OUT 0x0E /* Output, y-axis inclination */
-#define ADIS16203_INCL_NULL 0x18 /* Incline null calibration */
-#define ADIS16203_ALM_MAG1 0x20 /* Alarm 1 amplitude threshold */
-#define ADIS16203_ALM_MAG2 0x22 /* Alarm 2 amplitude threshold */
-#define ADIS16203_ALM_SMPL1 0x24 /* Alarm 1, sample period */
-#define ADIS16203_ALM_SMPL2 0x26 /* Alarm 2, sample period */
-#define ADIS16203_ALM_CTRL 0x28 /* Alarm control */
-#define ADIS16203_AUX_DAC 0x30 /* Auxiliary DAC data */
-#define ADIS16203_GPIO_CTRL 0x32 /* General-purpose digital input/output control */
-#define ADIS16203_MSC_CTRL 0x34 /* Miscellaneous control */
-#define ADIS16203_SMPL_PRD 0x36 /* Internal sample period (rate) control */
-#define ADIS16203_AVG_CNT 0x38 /* Operation, filter configuration */
-#define ADIS16203_SLP_CNT 0x3A /* Operation, sleep mode control */
-#define ADIS16203_DIAG_STAT 0x3C /* Diagnostics, system status register */
-#define ADIS16203_GLOB_CMD 0x3E /* Operation, system command register */
+/* Flash memory write count */
+#define ADIS16203_FLASH_CNT 0x00
+
+/* Output, power supply */
+#define ADIS16203_SUPPLY_OUT 0x02
+
+/* Output, auxiliary ADC input */
+#define ADIS16203_AUX_ADC 0x08
+
+/* Output, temperature */
+#define ADIS16203_TEMP_OUT 0x0A
+
+/* Output, x-axis inclination */
+#define ADIS16203_XINCL_OUT 0x0C
+
+/* Output, y-axis inclination */
+#define ADIS16203_YINCL_OUT 0x0E
+
+/* Incline null calibration */
+#define ADIS16203_INCL_NULL 0x18
+
+/* Alarm 1 amplitude threshold */
+#define ADIS16203_ALM_MAG1 0x20
+
+/* Alarm 2 amplitude threshold */
+#define ADIS16203_ALM_MAG2 0x22
+
+/* Alarm 1, sample period */
+#define ADIS16203_ALM_SMPL1 0x24
+
+/* Alarm 2, sample period */
+#define ADIS16203_ALM_SMPL2 0x26
+
+/* Alarm control */
+#define ADIS16203_ALM_CTRL 0x28
+
+/* Auxiliary DAC data */
+#define ADIS16203_AUX_DAC 0x30
+
+/* General-purpose digital input/output control */
+#define ADIS16203_GPIO_CTRL 0x32
+
+/* Miscellaneous control */
+#define ADIS16203_MSC_CTRL 0x34
+
+/* Internal sample period (rate) control */
+#define ADIS16203_SMPL_PRD 0x36
+
+/* Operation, filter configuration */
+#define ADIS16203_AVG_CNT 0x38
+
+/* Operation, sleep mode control */
+#define ADIS16203_SLP_CNT 0x3A
+
+/* Diagnostics, system status register */
+#define ADIS16203_DIAG_STAT 0x3C
+
+/* Operation, system command register */
+#define ADIS16203_GLOB_CMD 0x3E
/* MSC_CTRL */
-#define ADIS16203_MSC_CTRL_PWRUP_SELF_TEST BIT(10) /* Self-test at power-on: 1 = disabled, 0 = enabled */
-#define ADIS16203_MSC_CTRL_REVERSE_ROT_EN BIT(9) /* Reverses rotation of both inclination outputs */
-#define ADIS16203_MSC_CTRL_SELF_TEST_EN BIT(8) /* Self-test enable */
-#define ADIS16203_MSC_CTRL_DATA_RDY_EN BIT(2) /* Data-ready enable: 1 = enabled, 0 = disabled */
-#define ADIS16203_MSC_CTRL_ACTIVE_HIGH BIT(1) /* Data-ready polarity: 1 = active high, 0 = active low */
-#define ADIS16203_MSC_CTRL_DATA_RDY_DIO1 BIT(0) /* Data-ready line selection: 1 = DIO1, 0 = DIO0 */
+
+/* Self-test at power-on: 1 = disabled, 0 = enabled */
+#define ADIS16203_MSC_CTRL_PWRUP_SELF_TEST BIT(10)
+
+/* Reverses rotation of both inclination outputs */
+#define ADIS16203_MSC_CTRL_REVERSE_ROT_EN BIT(9)
+
+/* Self-test enable */
+#define ADIS16203_MSC_CTRL_SELF_TEST_EN BIT(8)
+
+/* Data-ready enable: 1 = enabled, 0 = disabled */
+#define ADIS16203_MSC_CTRL_DATA_RDY_EN BIT(2)
+
+/* Data-ready polarity: 1 = active high, 0 = active low */
+#define ADIS16203_MSC_CTRL_ACTIVE_HIGH BIT(1)
+
+/* Data-ready line selection: 1 = DIO1, 0 = DIO0 */
+#define ADIS16203_MSC_CTRL_DATA_RDY_DIO1 BIT(0)
/* DIAG_STAT */
-#define ADIS16203_DIAG_STAT_ALARM2 BIT(9) /* Alarm 2 status: 1 = alarm active, 0 = alarm inactive */
-#define ADIS16203_DIAG_STAT_ALARM1 BIT(8) /* Alarm 1 status: 1 = alarm active, 0 = alarm inactive */
-#define ADIS16203_DIAG_STAT_SELFTEST_FAIL_BIT 5 /* Self-test diagnostic error flag */
-#define ADIS16203_DIAG_STAT_SPI_FAIL_BIT 3 /* SPI communications failure */
-#define ADIS16203_DIAG_STAT_FLASH_UPT_BIT 2 /* Flash update failure */
-#define ADIS16203_DIAG_STAT_POWER_HIGH_BIT 1 /* Power supply above 3.625 V */
-#define ADIS16203_DIAG_STAT_POWER_LOW_BIT 0 /* Power supply below 3.15 V */
+
+/* Alarm 2 status: 1 = alarm active, 0 = alarm inactive */
+#define ADIS16203_DIAG_STAT_ALARM2 BIT(9)
+
+/* Alarm 1 status: 1 = alarm active, 0 = alarm inactive */
+#define ADIS16203_DIAG_STAT_ALARM1 BIT(8)
+
+/* Self-test diagnostic error flag */
+#define ADIS16203_DIAG_STAT_SELFTEST_FAIL_BIT 5
+
+/* SPI communications failure */
+#define ADIS16203_DIAG_STAT_SPI_FAIL_BIT 3
+
+/* Flash update failure */
+#define ADIS16203_DIAG_STAT_FLASH_UPT_BIT 2
+
+/* Power supply above 3.625 V */
+#define ADIS16203_DIAG_STAT_POWER_HIGH_BIT 1
+
+/* Power supply below 3.15 V */
+#define ADIS16203_DIAG_STAT_POWER_LOW_BIT 0
/* GLOB_CMD */
+
#define ADIS16203_GLOB_CMD_SW_RESET BIT(7)
#define ADIS16203_GLOB_CMD_CLEAR_STAT BIT(4)
#define ADIS16203_GLOB_CMD_FACTORY_CAL BIT(1)
diff --git a/drivers/staging/iio/accel/adis16203_core.c b/drivers/staging/iio/accel/adis16203_core.c
index de5b84a..c706717 100644
--- a/drivers/staging/iio/accel/adis16203_core.c
+++ b/drivers/staging/iio/accel/adis16203_core.c
@@ -134,6 +134,7 @@
.diag_stat_reg = ADIS16203_DIAG_STAT,
.self_test_mask = ADIS16203_MSC_CTRL_SELF_TEST_EN,
+ .self_test_no_autoclear = true,
.startup_delay = ADIS16203_STARTUP_DELAY,
.status_error_msgs = adis16203_status_error_msgs,
diff --git a/drivers/staging/iio/accel/adis16204.h b/drivers/staging/iio/accel/adis16204.h
deleted file mode 100644
index 0b23f0b..0000000
--- a/drivers/staging/iio/accel/adis16204.h
+++ /dev/null
@@ -1,68 +0,0 @@
-#ifndef SPI_ADIS16204_H_
-#define SPI_ADIS16204_H_
-
-#define ADIS16204_STARTUP_DELAY 220 /* ms */
-
-#define ADIS16204_FLASH_CNT 0x00 /* Flash memory write count */
-#define ADIS16204_SUPPLY_OUT 0x02 /* Output, power supply */
-#define ADIS16204_XACCL_OUT 0x04 /* Output, x-axis accelerometer */
-#define ADIS16204_YACCL_OUT 0x06 /* Output, y-axis accelerometer */
-#define ADIS16204_AUX_ADC 0x08 /* Output, auxiliary ADC input */
-#define ADIS16204_TEMP_OUT 0x0A /* Output, temperature */
-#define ADIS16204_X_PEAK_OUT 0x0C /* Twos complement */
-#define ADIS16204_Y_PEAK_OUT 0x0E /* Twos complement */
-#define ADIS16204_XACCL_NULL 0x10 /* Calibration, x-axis acceleration offset null */
-#define ADIS16204_YACCL_NULL 0x12 /* Calibration, y-axis acceleration offset null */
-#define ADIS16204_XACCL_SCALE 0x14 /* X-axis scale factor calibration register */
-#define ADIS16204_YACCL_SCALE 0x16 /* Y-axis scale factor calibration register */
-#define ADIS16204_XY_RSS_OUT 0x18 /* XY combined acceleration (RSS) */
-#define ADIS16204_XY_PEAK_OUT 0x1A /* Peak, XY combined output (RSS) */
-#define ADIS16204_CAP_BUF_1 0x1C /* Capture buffer output register 1 */
-#define ADIS16204_CAP_BUF_2 0x1E /* Capture buffer output register 2 */
-#define ADIS16204_ALM_MAG1 0x20 /* Alarm 1 amplitude threshold */
-#define ADIS16204_ALM_MAG2 0x22 /* Alarm 2 amplitude threshold */
-#define ADIS16204_ALM_CTRL 0x28 /* Alarm control */
-#define ADIS16204_CAPT_PNTR 0x2A /* Capture register address pointer */
-#define ADIS16204_AUX_DAC 0x30 /* Auxiliary DAC data */
-#define ADIS16204_GPIO_CTRL 0x32 /* General-purpose digital input/output control */
-#define ADIS16204_MSC_CTRL 0x34 /* Miscellaneous control */
-#define ADIS16204_SMPL_PRD 0x36 /* Internal sample period (rate) control */
-#define ADIS16204_AVG_CNT 0x38 /* Operation, filter configuration */
-#define ADIS16204_SLP_CNT 0x3A /* Operation, sleep mode control */
-#define ADIS16204_DIAG_STAT 0x3C /* Diagnostics, system status register */
-#define ADIS16204_GLOB_CMD 0x3E /* Operation, system command register */
-
-/* MSC_CTRL */
-#define ADIS16204_MSC_CTRL_PWRUP_SELF_TEST BIT(10) /* Self-test at power-on: 1 = disabled, 0 = enabled */
-#define ADIS16204_MSC_CTRL_SELF_TEST_EN BIT(8) /* Self-test enable */
-#define ADIS16204_MSC_CTRL_DATA_RDY_EN BIT(2) /* Data-ready enable: 1 = enabled, 0 = disabled */
-#define ADIS16204_MSC_CTRL_ACTIVE_HIGH BIT(1) /* Data-ready polarity: 1 = active high, 0 = active low */
-#define ADIS16204_MSC_CTRL_DATA_RDY_DIO2 BIT(0) /* Data-ready line selection: 1 = DIO2, 0 = DIO1 */
-
-/* DIAG_STAT */
-#define ADIS16204_DIAG_STAT_ALARM2 BIT(9) /* Alarm 2 status: 1 = alarm active, 0 = alarm inactive */
-#define ADIS16204_DIAG_STAT_ALARM1 BIT(8) /* Alarm 1 status: 1 = alarm active, 0 = alarm inactive */
-#define ADIS16204_DIAG_STAT_SELFTEST_FAIL_BIT 5 /* Self-test diagnostic error flag: 1 = error condition,
- 0 = normal operation */
-#define ADIS16204_DIAG_STAT_SPI_FAIL_BIT 3 /* SPI communications failure */
-#define ADIS16204_DIAG_STAT_FLASH_UPT_BIT 2 /* Flash update failure */
-#define ADIS16204_DIAG_STAT_POWER_HIGH_BIT 1 /* Power supply above 3.625 V */
-#define ADIS16204_DIAG_STAT_POWER_LOW_BIT 0 /* Power supply below 2.975 V */
-
-/* GLOB_CMD */
-#define ADIS16204_GLOB_CMD_SW_RESET BIT(7)
-#define ADIS16204_GLOB_CMD_CLEAR_STAT BIT(4)
-#define ADIS16204_GLOB_CMD_FACTORY_CAL BIT(1)
-
-#define ADIS16204_ERROR_ACTIVE BIT(14)
-
-enum adis16204_scan {
- ADIS16204_SCAN_ACC_X,
- ADIS16204_SCAN_ACC_Y,
- ADIS16204_SCAN_ACC_XY,
- ADIS16204_SCAN_SUPPLY,
- ADIS16204_SCAN_AUX_ADC,
- ADIS16204_SCAN_TEMP,
-};
-
-#endif /* SPI_ADIS16204_H_ */
diff --git a/drivers/staging/iio/accel/adis16204_core.c b/drivers/staging/iio/accel/adis16204_core.c
deleted file mode 100644
index 20a9df6..0000000
--- a/drivers/staging/iio/accel/adis16204_core.c
+++ /dev/null
@@ -1,253 +0,0 @@
-/*
- * ADIS16204 Programmable High-g Digital Impact Sensor and Recorder
- *
- * Copyright 2010 Analog Devices Inc.
- *
- * Licensed under the GPL-2 or later.
- */
-
-#include <linux/interrupt.h>
-#include <linux/irq.h>
-#include <linux/delay.h>
-#include <linux/mutex.h>
-#include <linux/device.h>
-#include <linux/kernel.h>
-#include <linux/spi/spi.h>
-#include <linux/slab.h>
-#include <linux/sysfs.h>
-#include <linux/list.h>
-#include <linux/module.h>
-
-#include <linux/iio/iio.h>
-#include <linux/iio/sysfs.h>
-#include <linux/iio/buffer.h>
-#include <linux/iio/imu/adis.h>
-
-#include "adis16204.h"
-
-/* Unique to this driver currently */
-
-static const u8 adis16204_addresses[][2] = {
- [ADIS16204_SCAN_ACC_X] = { ADIS16204_XACCL_NULL, ADIS16204_X_PEAK_OUT },
- [ADIS16204_SCAN_ACC_Y] = { ADIS16204_YACCL_NULL, ADIS16204_Y_PEAK_OUT },
- [ADIS16204_SCAN_ACC_XY] = { 0, ADIS16204_XY_PEAK_OUT },
-};
-
-static int adis16204_read_raw(struct iio_dev *indio_dev,
- struct iio_chan_spec const *chan,
- int *val, int *val2,
- long mask)
-{
- struct adis *st = iio_priv(indio_dev);
- int ret;
- int bits;
- u8 addr;
- s16 val16;
- int addrind;
-
- switch (mask) {
- case IIO_CHAN_INFO_RAW:
- return adis_single_conversion(indio_dev, chan,
- ADIS16204_ERROR_ACTIVE, val);
- case IIO_CHAN_INFO_SCALE:
- switch (chan->type) {
- case IIO_VOLTAGE:
- if (chan->channel == 0) {
- *val = 1;
- *val2 = 220000; /* 1.22 mV */
- } else {
- *val = 0;
- *val2 = 610000; /* 0.61 mV */
- }
- return IIO_VAL_INT_PLUS_MICRO;
- case IIO_TEMP:
- *val = -470; /* 0.47 C */
- *val2 = 0;
- return IIO_VAL_INT_PLUS_MICRO;
- case IIO_ACCEL:
- *val = 0;
- switch (chan->channel2) {
- case IIO_MOD_X:
- case IIO_MOD_ROOT_SUM_SQUARED_X_Y:
- *val2 = IIO_G_TO_M_S_2(17125); /* 17.125 mg */
- break;
- case IIO_MOD_Y:
- case IIO_MOD_Z:
- *val2 = IIO_G_TO_M_S_2(8407); /* 8.407 mg */
- break;
- }
- return IIO_VAL_INT_PLUS_MICRO;
- default:
- return -EINVAL;
- }
- break;
- case IIO_CHAN_INFO_OFFSET:
- *val = 25000 / -470 - 1278; /* 25 C = 1278 */
- return IIO_VAL_INT;
- case IIO_CHAN_INFO_CALIBBIAS:
- case IIO_CHAN_INFO_PEAK:
- if (mask == IIO_CHAN_INFO_CALIBBIAS) {
- bits = 12;
- addrind = 0;
- } else { /* PEAK_SEPARATE */
- bits = 14;
- addrind = 1;
- }
- mutex_lock(&indio_dev->mlock);
- addr = adis16204_addresses[chan->scan_index][addrind];
- ret = adis_read_reg_16(st, addr, &val16);
- if (ret) {
- mutex_unlock(&indio_dev->mlock);
- return ret;
- }
- val16 &= (1 << bits) - 1;
- val16 = (s16)(val16 << (16 - bits)) >> (16 - bits);
- *val = val16;
- mutex_unlock(&indio_dev->mlock);
- return IIO_VAL_INT;
- }
- return -EINVAL;
-}
-
-static int adis16204_write_raw(struct iio_dev *indio_dev,
- struct iio_chan_spec const *chan,
- int val,
- int val2,
- long mask)
-{
- struct adis *st = iio_priv(indio_dev);
- int bits;
- s16 val16;
- u8 addr;
-
- switch (mask) {
- case IIO_CHAN_INFO_CALIBBIAS:
- switch (chan->type) {
- case IIO_ACCEL:
- bits = 12;
- break;
- default:
- return -EINVAL;
- }
- val16 = val & ((1 << bits) - 1);
- addr = adis16204_addresses[chan->scan_index][1];
- return adis_write_reg_16(st, addr, val16);
- }
- return -EINVAL;
-}
-
-static const struct iio_chan_spec adis16204_channels[] = {
- ADIS_SUPPLY_CHAN(ADIS16204_SUPPLY_OUT, ADIS16204_SCAN_SUPPLY, 0, 12),
- ADIS_AUX_ADC_CHAN(ADIS16204_AUX_ADC, ADIS16204_SCAN_AUX_ADC, 0, 12),
- ADIS_TEMP_CHAN(ADIS16204_TEMP_OUT, ADIS16204_SCAN_TEMP, 0, 12),
- ADIS_ACCEL_CHAN(X, ADIS16204_XACCL_OUT, ADIS16204_SCAN_ACC_X,
- BIT(IIO_CHAN_INFO_CALIBBIAS) | BIT(IIO_CHAN_INFO_PEAK),
- 0, 14),
- ADIS_ACCEL_CHAN(Y, ADIS16204_YACCL_OUT, ADIS16204_SCAN_ACC_Y,
- BIT(IIO_CHAN_INFO_CALIBBIAS) | BIT(IIO_CHAN_INFO_PEAK),
- 0, 14),
- ADIS_ACCEL_CHAN(ROOT_SUM_SQUARED_X_Y, ADIS16204_XY_RSS_OUT,
- ADIS16204_SCAN_ACC_XY, BIT(IIO_CHAN_INFO_PEAK), 0, 14),
- IIO_CHAN_SOFT_TIMESTAMP(5),
-};
-
-static const struct iio_info adis16204_info = {
- .read_raw = &adis16204_read_raw,
- .write_raw = &adis16204_write_raw,
- .update_scan_mode = adis_update_scan_mode,
- .driver_module = THIS_MODULE,
-};
-
-static const char * const adis16204_status_error_msgs[] = {
- [ADIS16204_DIAG_STAT_SELFTEST_FAIL_BIT] = "Self test failure",
- [ADIS16204_DIAG_STAT_SPI_FAIL_BIT] = "SPI failure",
- [ADIS16204_DIAG_STAT_FLASH_UPT_BIT] = "Flash update failed",
- [ADIS16204_DIAG_STAT_POWER_HIGH_BIT] = "Power supply above 3.625V",
- [ADIS16204_DIAG_STAT_POWER_LOW_BIT] = "Power supply below 2.975V",
-};
-
-static const struct adis_data adis16204_data = {
- .read_delay = 20,
- .msc_ctrl_reg = ADIS16204_MSC_CTRL,
- .glob_cmd_reg = ADIS16204_GLOB_CMD,
- .diag_stat_reg = ADIS16204_DIAG_STAT,
-
- .self_test_mask = ADIS16204_MSC_CTRL_SELF_TEST_EN,
- .startup_delay = ADIS16204_STARTUP_DELAY,
-
- .status_error_msgs = adis16204_status_error_msgs,
- .status_error_mask = BIT(ADIS16204_DIAG_STAT_SELFTEST_FAIL_BIT) |
- BIT(ADIS16204_DIAG_STAT_SPI_FAIL_BIT) |
- BIT(ADIS16204_DIAG_STAT_FLASH_UPT_BIT) |
- BIT(ADIS16204_DIAG_STAT_POWER_HIGH_BIT) |
- BIT(ADIS16204_DIAG_STAT_POWER_LOW_BIT),
-};
-
-static int adis16204_probe(struct spi_device *spi)
-{
- int ret;
- struct adis *st;
- struct iio_dev *indio_dev;
-
- /* setup the industrialio driver allocated elements */
- indio_dev = devm_iio_device_alloc(&spi->dev, sizeof(*st));
- if (!indio_dev)
- return -ENOMEM;
- st = iio_priv(indio_dev);
- /* this is only used for removal purposes */
- spi_set_drvdata(spi, indio_dev);
-
- indio_dev->name = spi->dev.driver->name;
- indio_dev->dev.parent = &spi->dev;
- indio_dev->info = &adis16204_info;
- indio_dev->channels = adis16204_channels;
- indio_dev->num_channels = ARRAY_SIZE(adis16204_channels);
- indio_dev->modes = INDIO_DIRECT_MODE;
-
- ret = adis_init(st, indio_dev, spi, &adis16204_data);
- if (ret)
- return ret;
-
- ret = adis_setup_buffer_and_trigger(st, indio_dev, NULL);
- if (ret)
- return ret;
-
- /* Get the device into a sane initial state */
- ret = adis_initial_startup(st);
- if (ret)
- goto error_cleanup_buffer_trigger;
- ret = iio_device_register(indio_dev);
- if (ret)
- goto error_cleanup_buffer_trigger;
-
- return 0;
-
-error_cleanup_buffer_trigger:
- adis_cleanup_buffer_and_trigger(st, indio_dev);
- return ret;
-}
-
-static int adis16204_remove(struct spi_device *spi)
-{
- struct iio_dev *indio_dev = spi_get_drvdata(spi);
- struct adis *st = iio_priv(indio_dev);
-
- iio_device_unregister(indio_dev);
- adis_cleanup_buffer_and_trigger(st, indio_dev);
-
- return 0;
-}
-
-static struct spi_driver adis16204_driver = {
- .driver = {
- .name = "adis16204",
- },
- .probe = adis16204_probe,
- .remove = adis16204_remove,
-};
-module_spi_driver(adis16204_driver);
-
-MODULE_AUTHOR("Barry Song <21cnbao@gmail.com>");
-MODULE_DESCRIPTION("ADIS16204 High-g Digital Impact Sensor and Recorder");
-MODULE_LICENSE("GPL v2");
-MODULE_ALIAS("spi:adis16204");
diff --git a/drivers/staging/iio/accel/adis16209.h b/drivers/staging/iio/accel/adis16209.h
index 813698d..315f1c0 100644
--- a/drivers/staging/iio/accel/adis16209.h
+++ b/drivers/staging/iio/accel/adis16209.h
@@ -5,88 +5,127 @@
/* Flash memory write count */
#define ADIS16209_FLASH_CNT 0x00
+
/* Output, power supply */
#define ADIS16209_SUPPLY_OUT 0x02
+
/* Output, x-axis accelerometer */
#define ADIS16209_XACCL_OUT 0x04
+
/* Output, y-axis accelerometer */
#define ADIS16209_YACCL_OUT 0x06
+
/* Output, auxiliary ADC input */
#define ADIS16209_AUX_ADC 0x08
+
/* Output, temperature */
#define ADIS16209_TEMP_OUT 0x0A
+
/* Output, x-axis inclination */
#define ADIS16209_XINCL_OUT 0x0C
+
/* Output, y-axis inclination */
#define ADIS16209_YINCL_OUT 0x0E
+
/* Output, +/-180 vertical rotational position */
#define ADIS16209_ROT_OUT 0x10
+
/* Calibration, x-axis acceleration offset null */
#define ADIS16209_XACCL_NULL 0x12
+
/* Calibration, y-axis acceleration offset null */
#define ADIS16209_YACCL_NULL 0x14
+
/* Calibration, x-axis inclination offset null */
#define ADIS16209_XINCL_NULL 0x16
+
/* Calibration, y-axis inclination offset null */
#define ADIS16209_YINCL_NULL 0x18
+
/* Calibration, vertical rotation offset null */
#define ADIS16209_ROT_NULL 0x1A
+
/* Alarm 1 amplitude threshold */
#define ADIS16209_ALM_MAG1 0x20
+
/* Alarm 2 amplitude threshold */
#define ADIS16209_ALM_MAG2 0x22
+
/* Alarm 1, sample period */
#define ADIS16209_ALM_SMPL1 0x24
+
/* Alarm 2, sample period */
#define ADIS16209_ALM_SMPL2 0x26
+
/* Alarm control */
#define ADIS16209_ALM_CTRL 0x28
+
/* Auxiliary DAC data */
#define ADIS16209_AUX_DAC 0x30
+
/* General-purpose digital input/output control */
#define ADIS16209_GPIO_CTRL 0x32
+
/* Miscellaneous control */
#define ADIS16209_MSC_CTRL 0x34
+
/* Internal sample period (rate) control */
#define ADIS16209_SMPL_PRD 0x36
+
/* Operation, filter configuration */
#define ADIS16209_AVG_CNT 0x38
+
/* Operation, sleep mode control */
#define ADIS16209_SLP_CNT 0x3A
+
/* Diagnostics, system status register */
#define ADIS16209_DIAG_STAT 0x3C
+
/* Operation, system command register */
#define ADIS16209_GLOB_CMD 0x3E
/* MSC_CTRL */
+
/* Self-test at power-on: 1 = disabled, 0 = enabled */
#define ADIS16209_MSC_CTRL_PWRUP_SELF_TEST BIT(10)
+
/* Self-test enable */
#define ADIS16209_MSC_CTRL_SELF_TEST_EN BIT(8)
+
/* Data-ready enable: 1 = enabled, 0 = disabled */
#define ADIS16209_MSC_CTRL_DATA_RDY_EN BIT(2)
+
/* Data-ready polarity: 1 = active high, 0 = active low */
#define ADIS16209_MSC_CTRL_ACTIVE_HIGH BIT(1)
+
/* Data-ready line selection: 1 = DIO2, 0 = DIO1 */
#define ADIS16209_MSC_CTRL_DATA_RDY_DIO2 BIT(0)
/* DIAG_STAT */
+
/* Alarm 2 status: 1 = alarm active, 0 = alarm inactive */
#define ADIS16209_DIAG_STAT_ALARM2 BIT(9)
+
/* Alarm 1 status: 1 = alarm active, 0 = alarm inactive */
#define ADIS16209_DIAG_STAT_ALARM1 BIT(8)
+
/* Self-test diagnostic error flag: 1 = error condition, 0 = normal operation */
#define ADIS16209_DIAG_STAT_SELFTEST_FAIL_BIT 5
+
/* SPI communications failure */
#define ADIS16209_DIAG_STAT_SPI_FAIL_BIT 3
+
/* Flash update failure */
#define ADIS16209_DIAG_STAT_FLASH_UPT_BIT 2
+
/* Power supply above 3.625 V */
#define ADIS16209_DIAG_STAT_POWER_HIGH_BIT 1
+
/* Power supply below 3.15 V */
#define ADIS16209_DIAG_STAT_POWER_LOW_BIT 0
/* GLOB_CMD */
+
#define ADIS16209_GLOB_CMD_SW_RESET BIT(7)
#define ADIS16209_GLOB_CMD_CLEAR_STAT BIT(4)
#define ADIS16209_GLOB_CMD_FACTORY_CAL BIT(1)
diff --git a/drivers/staging/iio/accel/adis16209_core.c b/drivers/staging/iio/accel/adis16209_core.c
index 8b42bf8..8dbad58 100644
--- a/drivers/staging/iio/accel/adis16209_core.c
+++ b/drivers/staging/iio/accel/adis16209_core.c
@@ -168,6 +168,7 @@
.diag_stat_reg = ADIS16209_DIAG_STAT,
.self_test_mask = ADIS16209_MSC_CTRL_SELF_TEST_EN,
+ .self_test_no_autoclear = true,
.startup_delay = ADIS16209_STARTUP_DELAY,
.status_error_msgs = adis16209_status_error_msgs,
diff --git a/drivers/staging/iio/accel/adis16220.h b/drivers/staging/iio/accel/adis16220.h
deleted file mode 100644
index eab8633..0000000
--- a/drivers/staging/iio/accel/adis16220.h
+++ /dev/null
@@ -1,140 +0,0 @@
-#ifndef SPI_ADIS16220_H_
-#define SPI_ADIS16220_H_
-
-#include <linux/iio/imu/adis.h>
-
-#define ADIS16220_STARTUP_DELAY 220 /* ms */
-
-/* Flash memory write count */
-#define ADIS16220_FLASH_CNT 0x00
-/* Control, acceleration offset adjustment control */
-#define ADIS16220_ACCL_NULL 0x02
-/* Control, AIN1 offset adjustment control */
-#define ADIS16220_AIN1_NULL 0x04
-/* Control, AIN2 offset adjustment control */
-#define ADIS16220_AIN2_NULL 0x06
-/* Output, power supply during capture */
-#define ADIS16220_CAPT_SUPPLY 0x0A
-/* Output, temperature during capture */
-#define ADIS16220_CAPT_TEMP 0x0C
-/* Output, peak acceleration during capture */
-#define ADIS16220_CAPT_PEAKA 0x0E
-/* Output, peak AIN1 level during capture */
-#define ADIS16220_CAPT_PEAK1 0x10
-/* Output, peak AIN2 level during capture */
-#define ADIS16220_CAPT_PEAK2 0x12
-/* Output, capture buffer for acceleration */
-#define ADIS16220_CAPT_BUFA 0x14
-/* Output, capture buffer for AIN1 */
-#define ADIS16220_CAPT_BUF1 0x16
-/* Output, capture buffer for AIN2 */
-#define ADIS16220_CAPT_BUF2 0x18
-/* Control, capture buffer address pointer */
-#define ADIS16220_CAPT_PNTR 0x1A
-/* Control, capture control register */
-#define ADIS16220_CAPT_CTRL 0x1C
-/* Control, capture period (automatic mode) */
-#define ADIS16220_CAPT_PRD 0x1E
-/* Control, Alarm A, acceleration peak threshold */
-#define ADIS16220_ALM_MAGA 0x20
-/* Control, Alarm 1, AIN1 peak threshold */
-#define ADIS16220_ALM_MAG1 0x22
-/* Control, Alarm 2, AIN2 peak threshold */
-#define ADIS16220_ALM_MAG2 0x24
-/* Control, Alarm S, peak threshold */
-#define ADIS16220_ALM_MAGS 0x26
-/* Control, alarm configuration register */
-#define ADIS16220_ALM_CTRL 0x28
-/* Control, general I/O configuration */
-#define ADIS16220_GPIO_CTRL 0x32
-/* Control, self-test control, AIN configuration */
-#define ADIS16220_MSC_CTRL 0x34
-/* Control, digital I/O configuration */
-#define ADIS16220_DIO_CTRL 0x36
-/* Control, filter configuration */
-#define ADIS16220_AVG_CNT 0x38
-/* Status, system status */
-#define ADIS16220_DIAG_STAT 0x3C
-/* Control, system commands */
-#define ADIS16220_GLOB_CMD 0x3E
-/* Status, self-test response */
-#define ADIS16220_ST_DELTA 0x40
-/* Lot Identification Code 1 */
-#define ADIS16220_LOT_ID1 0x52
-/* Lot Identification Code 2 */
-#define ADIS16220_LOT_ID2 0x54
-/* Product identifier; convert to decimal = 16220 */
-#define ADIS16220_PROD_ID 0x56
-/* Serial number */
-#define ADIS16220_SERIAL_NUM 0x58
-
-#define ADIS16220_CAPTURE_SIZE 2048
-
-/* MSC_CTRL */
-#define ADIS16220_MSC_CTRL_SELF_TEST_EN BIT(8)
-#define ADIS16220_MSC_CTRL_POWER_SUP_COM_AIN1 BIT(1)
-#define ADIS16220_MSC_CTRL_POWER_SUP_COM_AIN2 BIT(0)
-
-/* DIO_CTRL */
-#define ADIS16220_MSC_CTRL_DIO2_BUSY_IND (BIT(5) | BIT(4))
-#define ADIS16220_MSC_CTRL_DIO1_BUSY_IND (BIT(3) | BIT(2))
-#define ADIS16220_MSC_CTRL_DIO2_ACT_HIGH BIT(1)
-#define ADIS16220_MSC_CTRL_DIO1_ACT_HIGH BIT(0)
-
-/* DIAG_STAT */
-/* AIN2 sample > ALM_MAG2 */
-#define ADIS16220_DIAG_STAT_ALM_MAG2 BIT(14)
-/* AIN1 sample > ALM_MAG1 */
-#define ADIS16220_DIAG_STAT_ALM_MAG1 BIT(13)
-/* Acceleration sample > ALM_MAGA */
-#define ADIS16220_DIAG_STAT_ALM_MAGA BIT(12)
-/* Error condition programmed into ALM_MAGS[11:0] and ALM_CTRL[5:4] is true */
-#define ADIS16220_DIAG_STAT_ALM_MAGS BIT(11)
-/* |Peak value in AIN2 data capture| > ALM_MAG2 */
-#define ADIS16220_DIAG_STAT_PEAK_AIN2 BIT(10)
-/* |Peak value in AIN1 data capture| > ALM_MAG1 */
-#define ADIS16220_DIAG_STAT_PEAK_AIN1 BIT(9)
-/* |Peak value in acceleration data capture| > ALM_MAGA */
-#define ADIS16220_DIAG_STAT_PEAK_ACCEL BIT(8)
-/* Data ready, capture complete */
-#define ADIS16220_DIAG_STAT_DATA_RDY BIT(7)
-#define ADIS16220_DIAG_STAT_FLASH_CHK BIT(6)
-#define ADIS16220_DIAG_STAT_SELF_TEST BIT(5)
-/* Capture period violation/interruption */
-#define ADIS16220_DIAG_STAT_VIOLATION_BIT 4
-/* SPI communications failure */
-#define ADIS16220_DIAG_STAT_SPI_FAIL_BIT 3
-/* Flash update failure */
-#define ADIS16220_DIAG_STAT_FLASH_UPT_BIT 2
-/* Power supply above 3.625 V */
-#define ADIS16220_DIAG_STAT_POWER_HIGH_BIT 1
-/* Power supply below 3.15 V */
-#define ADIS16220_DIAG_STAT_POWER_LOW_BIT 0
-
-/* GLOB_CMD */
-#define ADIS16220_GLOB_CMD_SW_RESET BIT(7)
-#define ADIS16220_GLOB_CMD_SELF_TEST BIT(2)
-#define ADIS16220_GLOB_CMD_PWR_DOWN BIT(1)
-
-#define ADIS16220_MAX_TX 2048
-#define ADIS16220_MAX_RX 2048
-
-#define ADIS16220_SPI_BURST (u32)(1000 * 1000)
-#define ADIS16220_SPI_FAST (u32)(2000 * 1000)
-
-/**
- * struct adis16220_state - device instance specific data
- * @adis: adis device
- * @tx: transmit buffer
- * @rx: receive buffer
- * @buf_lock: mutex to protect tx and rx
- **/
-struct adis16220_state {
- struct adis adis;
-
- struct mutex buf_lock;
- u8 tx[ADIS16220_MAX_TX] ____cacheline_aligned;
- u8 rx[ADIS16220_MAX_RX];
-};
-
-#endif /* SPI_ADIS16220_H_ */
diff --git a/drivers/staging/iio/accel/adis16220_core.c b/drivers/staging/iio/accel/adis16220_core.c
deleted file mode 100644
index d016521..0000000
--- a/drivers/staging/iio/accel/adis16220_core.c
+++ /dev/null
@@ -1,494 +0,0 @@
-/*
- * ADIS16220 Programmable Digital Vibration Sensor driver
- *
- * Copyright 2010 Analog Devices Inc.
- *
- * Licensed under the GPL-2 or later.
- */
-
-#include <linux/delay.h>
-#include <linux/mutex.h>
-#include <linux/device.h>
-#include <linux/kernel.h>
-#include <linux/spi/spi.h>
-#include <linux/slab.h>
-#include <linux/sysfs.h>
-#include <linux/module.h>
-
-#include <linux/iio/iio.h>
-#include <linux/iio/sysfs.h>
-
-#include "adis16220.h"
-
-static ssize_t adis16220_read_16bit(struct device *dev,
- struct device_attribute *attr,
- char *buf)
-{
- struct iio_dev_attr *this_attr = to_iio_dev_attr(attr);
- struct iio_dev *indio_dev = dev_to_iio_dev(dev);
- struct adis16220_state *st = iio_priv(indio_dev);
- ssize_t ret;
- u16 val;
-
- /* Take the iio_dev status lock */
- mutex_lock(&indio_dev->mlock);
- ret = adis_read_reg_16(&st->adis, this_attr->address, &val);
- mutex_unlock(&indio_dev->mlock);
- if (ret)
- return ret;
- return sprintf(buf, "%u\n", val);
-}
-
-static ssize_t adis16220_write_16bit(struct device *dev,
- struct device_attribute *attr,
- const char *buf,
- size_t len)
-{
- struct iio_dev *indio_dev = dev_to_iio_dev(dev);
- struct iio_dev_attr *this_attr = to_iio_dev_attr(attr);
- struct adis16220_state *st = iio_priv(indio_dev);
- int ret;
- u16 val;
-
- ret = kstrtou16(buf, 10, &val);
- if (ret)
- goto error_ret;
- ret = adis_write_reg_16(&st->adis, this_attr->address, val);
-
-error_ret:
- return ret ? ret : len;
-}
-
-static int adis16220_capture(struct iio_dev *indio_dev)
-{
- struct adis16220_state *st = iio_priv(indio_dev);
- int ret;
-
- /* initiates a manual data capture */
- ret = adis_write_reg_16(&st->adis, ADIS16220_GLOB_CMD, 0xBF08);
- if (ret)
- dev_err(&indio_dev->dev, "problem beginning capture");
-
- usleep_range(10000, 11000); /* delay for capture to finish */
-
- return ret;
-}
-
-static ssize_t adis16220_write_capture(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t len)
-{
- struct iio_dev *indio_dev = dev_to_iio_dev(dev);
- bool val;
- int ret;
-
- ret = strtobool(buf, &val);
- if (ret)
- return ret;
- if (!val)
- return -EINVAL;
- ret = adis16220_capture(indio_dev);
- if (ret)
- return ret;
-
- return len;
-}
-
-static ssize_t adis16220_capture_buffer_read(struct iio_dev *indio_dev,
- char *buf,
- loff_t off,
- size_t count,
- int addr)
-{
- struct adis16220_state *st = iio_priv(indio_dev);
- struct spi_transfer xfers[] = {
- {
- .tx_buf = st->tx,
- .bits_per_word = 8,
- .len = 2,
- .cs_change = 1,
- .delay_usecs = 25,
- }, {
- .tx_buf = st->tx,
- .rx_buf = st->rx,
- .bits_per_word = 8,
- .cs_change = 1,
- .delay_usecs = 25,
- },
- };
- int ret;
- int i;
-
- if (unlikely(!count))
- return count;
-
- if ((off >= ADIS16220_CAPTURE_SIZE) || (count & 1) || (off & 1))
- return -EINVAL;
-
- if (off + count > ADIS16220_CAPTURE_SIZE)
- count = ADIS16220_CAPTURE_SIZE - off;
-
- /* write the begin position of capture buffer */
- ret = adis_write_reg_16(&st->adis,
- ADIS16220_CAPT_PNTR,
- off > 1);
- if (ret)
- return -EIO;
-
- /* read count/2 values from capture buffer */
- mutex_lock(&st->buf_lock);
-
- for (i = 0; i < count; i += 2) {
- st->tx[i] = ADIS_READ_REG(addr);
- st->tx[i + 1] = 0;
- }
- xfers[1].len = count;
-
- ret = spi_sync_transfer(st->adis.spi, xfers, ARRAY_SIZE(xfers));
- if (ret) {
- mutex_unlock(&st->buf_lock);
- return -EIO;
- }
-
- memcpy(buf, st->rx, count);
-
- mutex_unlock(&st->buf_lock);
- return count;
-}
-
-static ssize_t adis16220_accel_bin_read(struct file *filp, struct kobject *kobj,
- struct bin_attribute *attr,
- char *buf,
- loff_t off,
- size_t count)
-{
- struct iio_dev *indio_dev = dev_to_iio_dev(kobj_to_dev(kobj));
-
- return adis16220_capture_buffer_read(indio_dev, buf,
- off, count,
- ADIS16220_CAPT_BUFA);
-}
-
-static struct bin_attribute accel_bin = {
- .attr = {
- .name = "accel_bin",
- .mode = S_IRUGO,
- },
- .read = adis16220_accel_bin_read,
- .size = ADIS16220_CAPTURE_SIZE,
-};
-
-static ssize_t adis16220_adc1_bin_read(struct file *filp, struct kobject *kobj,
- struct bin_attribute *attr,
- char *buf, loff_t off,
- size_t count)
-{
- struct iio_dev *indio_dev = dev_to_iio_dev(kobj_to_dev(kobj));
-
- return adis16220_capture_buffer_read(indio_dev, buf,
- off, count,
- ADIS16220_CAPT_BUF1);
-}
-
-static struct bin_attribute adc1_bin = {
- .attr = {
- .name = "in0_bin",
- .mode = S_IRUGO,
- },
- .read = adis16220_adc1_bin_read,
- .size = ADIS16220_CAPTURE_SIZE,
-};
-
-static ssize_t adis16220_adc2_bin_read(struct file *filp, struct kobject *kobj,
- struct bin_attribute *attr,
- char *buf, loff_t off,
- size_t count)
-{
- struct iio_dev *indio_dev = dev_to_iio_dev(kobj_to_dev(kobj));
-
- return adis16220_capture_buffer_read(indio_dev, buf,
- off, count,
- ADIS16220_CAPT_BUF2);
-}
-
-static struct bin_attribute adc2_bin = {
- .attr = {
- .name = "in1_bin",
- .mode = S_IRUGO,
- },
- .read = adis16220_adc2_bin_read,
- .size = ADIS16220_CAPTURE_SIZE,
-};
-
-#define IIO_DEV_ATTR_CAPTURE(_store) \
- IIO_DEVICE_ATTR(capture, S_IWUSR, NULL, _store, 0)
-
-static IIO_DEV_ATTR_CAPTURE(adis16220_write_capture);
-
-#define IIO_DEV_ATTR_CAPTURE_COUNT(_mode, _show, _store, _addr) \
- IIO_DEVICE_ATTR(capture_count, _mode, _show, _store, _addr)
-
-static IIO_DEV_ATTR_CAPTURE_COUNT(S_IWUSR | S_IRUGO,
- adis16220_read_16bit,
- adis16220_write_16bit,
- ADIS16220_CAPT_PNTR);
-
-enum adis16220_channel {
- in_supply, in_1, in_2, accel, temp
-};
-
-struct adis16220_address_spec {
- u8 addr;
- u8 bits;
- bool sign;
-};
-
-/* Address / bits / signed */
-static const struct adis16220_address_spec adis16220_addresses[][3] = {
- [in_supply] = { { ADIS16220_CAPT_SUPPLY, 12, 0 }, },
- [in_1] = { { ADIS16220_CAPT_BUF1, 16, 1 },
- { ADIS16220_AIN1_NULL, 16, 1 },
- { ADIS16220_CAPT_PEAK1, 16, 1 }, },
- [in_2] = { { ADIS16220_CAPT_BUF2, 16, 1 },
- { ADIS16220_AIN2_NULL, 16, 1 },
- { ADIS16220_CAPT_PEAK2, 16, 1 }, },
- [accel] = { { ADIS16220_CAPT_BUFA, 16, 1 },
- { ADIS16220_ACCL_NULL, 16, 1 },
- { ADIS16220_CAPT_PEAKA, 16, 1 }, },
- [temp] = { { ADIS16220_CAPT_TEMP, 12, 0 }, }
-};
-
-static int adis16220_read_raw(struct iio_dev *indio_dev,
- struct iio_chan_spec const *chan,
- int *val, int *val2,
- long mask)
-{
- struct adis16220_state *st = iio_priv(indio_dev);
- const struct adis16220_address_spec *addr;
- int ret = -EINVAL;
- int addrind = 0;
- u16 uval;
- s16 sval;
- u8 bits;
-
- switch (mask) {
- case IIO_CHAN_INFO_RAW:
- addrind = 0;
- break;
- case IIO_CHAN_INFO_OFFSET:
- if (chan->type == IIO_TEMP) {
- *val = 25000 / -470 - 1278; /* 25 C = 1278 */
- return IIO_VAL_INT;
- }
- addrind = 1;
- break;
- case IIO_CHAN_INFO_PEAK:
- addrind = 2;
- break;
- case IIO_CHAN_INFO_SCALE:
- switch (chan->type) {
- case IIO_TEMP:
- *val = -470; /* -0.47 C */
- *val2 = 0;
- return IIO_VAL_INT_PLUS_MICRO;
- case IIO_ACCEL:
- *val2 = IIO_G_TO_M_S_2(19073); /* 19.073 g */
- return IIO_VAL_INT_PLUS_MICRO;
- case IIO_VOLTAGE:
- if (chan->channel == 0) {
- *val = 1;
- *val2 = 220700; /* 1.2207 mV */
- } else {
- /* Should really be dependent on VDD */
- *val2 = 305180; /* 305.18 uV */
- }
- return IIO_VAL_INT_PLUS_MICRO;
- default:
- return -EINVAL;
- }
- default:
- return -EINVAL;
- }
- addr = &adis16220_addresses[chan->address][addrind];
- if (addr->sign) {
- ret = adis_read_reg_16(&st->adis, addr->addr, &sval);
- if (ret)
- return ret;
- bits = addr->bits;
- sval &= (1 << bits) - 1;
- sval = (s16)(sval << (16 - bits)) >> (16 - bits);
- *val = sval;
- return IIO_VAL_INT;
- }
- ret = adis_read_reg_16(&st->adis, addr->addr, &uval);
- if (ret)
- return ret;
- bits = addr->bits;
- uval &= (1 << bits) - 1;
- *val = uval;
- return IIO_VAL_INT;
-}
-
-static const struct iio_chan_spec adis16220_channels[] = {
- {
- .type = IIO_VOLTAGE,
- .indexed = 1,
- .channel = 0,
- .extend_name = "supply",
- .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) |
- BIT(IIO_CHAN_INFO_SCALE),
- .address = in_supply,
- }, {
- .type = IIO_ACCEL,
- .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) |
- BIT(IIO_CHAN_INFO_OFFSET) |
- BIT(IIO_CHAN_INFO_SCALE) |
- BIT(IIO_CHAN_INFO_PEAK),
- .address = accel,
- }, {
- .type = IIO_TEMP,
- .indexed = 1,
- .channel = 0,
- .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) |
- BIT(IIO_CHAN_INFO_OFFSET) |
- BIT(IIO_CHAN_INFO_SCALE),
- .address = temp,
- }, {
- .type = IIO_VOLTAGE,
- .indexed = 1,
- .channel = 1,
- .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) |
- BIT(IIO_CHAN_INFO_OFFSET) |
- BIT(IIO_CHAN_INFO_SCALE),
- .address = in_1,
- }, {
- .type = IIO_VOLTAGE,
- .indexed = 1,
- .channel = 2,
- .info_mask_separate = BIT(IIO_CHAN_INFO_RAW),
- .address = in_2,
- }
-};
-
-static struct attribute *adis16220_attributes[] = {
- &iio_dev_attr_capture.dev_attr.attr,
- &iio_dev_attr_capture_count.dev_attr.attr,
- NULL
-};
-
-static const struct attribute_group adis16220_attribute_group = {
- .attrs = adis16220_attributes,
-};
-
-static const struct iio_info adis16220_info = {
- .attrs = &adis16220_attribute_group,
- .driver_module = THIS_MODULE,
- .read_raw = &adis16220_read_raw,
-};
-
-static const char * const adis16220_status_error_msgs[] = {
- [ADIS16220_DIAG_STAT_VIOLATION_BIT] = "Capture period violation/interruption",
- [ADIS16220_DIAG_STAT_SPI_FAIL_BIT] = "SPI failure",
- [ADIS16220_DIAG_STAT_FLASH_UPT_BIT] = "Flash update failed",
- [ADIS16220_DIAG_STAT_POWER_HIGH_BIT] = "Power supply above 3.625V",
- [ADIS16220_DIAG_STAT_POWER_LOW_BIT] = "Power supply below 3.15V",
-};
-
-static const struct adis_data adis16220_data = {
- .read_delay = 35,
- .write_delay = 35,
- .msc_ctrl_reg = ADIS16220_MSC_CTRL,
- .glob_cmd_reg = ADIS16220_GLOB_CMD,
- .diag_stat_reg = ADIS16220_DIAG_STAT,
-
- .self_test_mask = ADIS16220_MSC_CTRL_SELF_TEST_EN,
- .startup_delay = ADIS16220_STARTUP_DELAY,
-
- .status_error_msgs = adis16220_status_error_msgs,
- .status_error_mask = BIT(ADIS16220_DIAG_STAT_VIOLATION_BIT) |
- BIT(ADIS16220_DIAG_STAT_SPI_FAIL_BIT) |
- BIT(ADIS16220_DIAG_STAT_FLASH_UPT_BIT) |
- BIT(ADIS16220_DIAG_STAT_POWER_HIGH_BIT) |
- BIT(ADIS16220_DIAG_STAT_POWER_LOW_BIT),
-};
-
-static int adis16220_probe(struct spi_device *spi)
-{
- int ret;
- struct adis16220_state *st;
- struct iio_dev *indio_dev;
-
- /* setup the industrialio driver allocated elements */
- indio_dev = devm_iio_device_alloc(&spi->dev, sizeof(*st));
- if (!indio_dev)
- return -ENOMEM;
-
- st = iio_priv(indio_dev);
- /* this is only used for removal purposes */
- spi_set_drvdata(spi, indio_dev);
-
- indio_dev->name = spi->dev.driver->name;
- indio_dev->dev.parent = &spi->dev;
- indio_dev->info = &adis16220_info;
- indio_dev->modes = INDIO_DIRECT_MODE;
- indio_dev->channels = adis16220_channels;
- indio_dev->num_channels = ARRAY_SIZE(adis16220_channels);
-
- ret = devm_iio_device_register(&spi->dev, indio_dev);
- if (ret)
- return ret;
-
- ret = sysfs_create_bin_file(&indio_dev->dev.kobj, &accel_bin);
- if (ret)
- return ret;
-
- ret = sysfs_create_bin_file(&indio_dev->dev.kobj, &adc1_bin);
- if (ret)
- goto error_rm_accel_bin;
-
- ret = sysfs_create_bin_file(&indio_dev->dev.kobj, &adc2_bin);
- if (ret)
- goto error_rm_adc1_bin;
-
- ret = adis_init(&st->adis, indio_dev, spi, &adis16220_data);
- if (ret)
- goto error_rm_adc2_bin;
- /* Get the device into a sane initial state */
- ret = adis_initial_startup(&st->adis);
- if (ret)
- goto error_rm_adc2_bin;
- return 0;
-
-error_rm_adc2_bin:
- sysfs_remove_bin_file(&indio_dev->dev.kobj, &adc2_bin);
-error_rm_adc1_bin:
- sysfs_remove_bin_file(&indio_dev->dev.kobj, &adc1_bin);
-error_rm_accel_bin:
- sysfs_remove_bin_file(&indio_dev->dev.kobj, &accel_bin);
- return ret;
-}
-
-static int adis16220_remove(struct spi_device *spi)
-{
- struct iio_dev *indio_dev = spi_get_drvdata(spi);
-
- sysfs_remove_bin_file(&indio_dev->dev.kobj, &adc2_bin);
- sysfs_remove_bin_file(&indio_dev->dev.kobj, &adc1_bin);
- sysfs_remove_bin_file(&indio_dev->dev.kobj, &accel_bin);
-
- return 0;
-}
-
-static struct spi_driver adis16220_driver = {
- .driver = {
- .name = "adis16220",
- },
- .probe = adis16220_probe,
- .remove = adis16220_remove,
-};
-module_spi_driver(adis16220_driver);
-
-MODULE_AUTHOR("Barry Song <21cnbao@gmail.com>");
-MODULE_DESCRIPTION("Analog Devices ADIS16220 Digital Vibration Sensor");
-MODULE_LICENSE("GPL v2");
-MODULE_ALIAS("spi:adis16220");
diff --git a/drivers/staging/iio/accel/adis16240.h b/drivers/staging/iio/accel/adis16240.h
index 66b5ad2..b2cb37b 100644
--- a/drivers/staging/iio/accel/adis16240.h
+++ b/drivers/staging/iio/accel/adis16240.h
@@ -5,110 +5,160 @@
/* Flash memory write count */
#define ADIS16240_FLASH_CNT 0x00
+
/* Output, power supply */
#define ADIS16240_SUPPLY_OUT 0x02
+
/* Output, x-axis accelerometer */
#define ADIS16240_XACCL_OUT 0x04
+
/* Output, y-axis accelerometer */
#define ADIS16240_YACCL_OUT 0x06
+
/* Output, z-axis accelerometer */
#define ADIS16240_ZACCL_OUT 0x08
+
/* Output, auxiliary ADC input */
#define ADIS16240_AUX_ADC 0x0A
+
/* Output, temperature */
#define ADIS16240_TEMP_OUT 0x0C
+
/* Output, x-axis acceleration peak */
#define ADIS16240_XPEAK_OUT 0x0E
+
/* Output, y-axis acceleration peak */
#define ADIS16240_YPEAK_OUT 0x10
+
/* Output, z-axis acceleration peak */
#define ADIS16240_ZPEAK_OUT 0x12
+
/* Output, sum-of-squares acceleration peak */
#define ADIS16240_XYZPEAK_OUT 0x14
+
/* Output, Capture Buffer 1, X and Y acceleration */
#define ADIS16240_CAPT_BUF1 0x16
+
/* Output, Capture Buffer 2, Z acceleration */
#define ADIS16240_CAPT_BUF2 0x18
+
/* Diagnostic, error flags */
#define ADIS16240_DIAG_STAT 0x1A
+
/* Diagnostic, event counter */
#define ADIS16240_EVNT_CNTR 0x1C
+
/* Diagnostic, check sum value from firmware test */
#define ADIS16240_CHK_SUM 0x1E
+
/* Calibration, x-axis acceleration offset adjustment */
#define ADIS16240_XACCL_OFF 0x20
+
/* Calibration, y-axis acceleration offset adjustment */
#define ADIS16240_YACCL_OFF 0x22
+
/* Calibration, z-axis acceleration offset adjustment */
#define ADIS16240_ZACCL_OFF 0x24
+
/* Clock, hour and minute */
#define ADIS16240_CLK_TIME 0x2E
+
/* Clock, month and day */
#define ADIS16240_CLK_DATE 0x30
+
/* Clock, year */
#define ADIS16240_CLK_YEAR 0x32
+
/* Wake-up setting, hour and minute */
#define ADIS16240_WAKE_TIME 0x34
+
/* Wake-up setting, month and day */
#define ADIS16240_WAKE_DATE 0x36
+
/* Alarm 1 amplitude threshold */
#define ADIS16240_ALM_MAG1 0x38
+
/* Alarm 2 amplitude threshold */
#define ADIS16240_ALM_MAG2 0x3A
+
/* Alarm control */
#define ADIS16240_ALM_CTRL 0x3C
+
/* Capture, external trigger control */
#define ADIS16240_XTRIG_CTRL 0x3E
+
/* Capture, address pointer */
#define ADIS16240_CAPT_PNTR 0x40
+
/* Capture, configuration and control */
#define ADIS16240_CAPT_CTRL 0x42
+
/* General-purpose digital input/output control */
#define ADIS16240_GPIO_CTRL 0x44
+
/* Miscellaneous control */
#define ADIS16240_MSC_CTRL 0x46
+
/* Internal sample period (rate) control */
#define ADIS16240_SMPL_PRD 0x48
+
/* System command */
#define ADIS16240_GLOB_CMD 0x4A
/* MSC_CTRL */
+
/* Enables sum-of-squares output (XYZPEAK_OUT) */
#define ADIS16240_MSC_CTRL_XYZPEAK_OUT_EN BIT(15)
+
/* Enables peak tracking output (XPEAK_OUT, YPEAK_OUT, and ZPEAK_OUT) */
#define ADIS16240_MSC_CTRL_X_Y_ZPEAK_OUT_EN BIT(14)
+
/* Self-test enable: 1 = apply electrostatic force, 0 = disabled */
#define ADIS16240_MSC_CTRL_SELF_TEST_EN BIT(8)
+
/* Data-ready enable: 1 = enabled, 0 = disabled */
#define ADIS16240_MSC_CTRL_DATA_RDY_EN BIT(2)
+
/* Data-ready polarity: 1 = active high, 0 = active low */
#define ADIS16240_MSC_CTRL_ACTIVE_HIGH BIT(1)
+
/* Data-ready line selection: 1 = DIO2, 0 = DIO1 */
#define ADIS16240_MSC_CTRL_DATA_RDY_DIO2 BIT(0)
/* DIAG_STAT */
+
/* Alarm 2 status: 1 = alarm active, 0 = alarm inactive */
#define ADIS16240_DIAG_STAT_ALARM2 BIT(9)
+
/* Alarm 1 status: 1 = alarm active, 0 = alarm inactive */
#define ADIS16240_DIAG_STAT_ALARM1 BIT(8)
+
/* Capture buffer full: 1 = capture buffer is full */
#define ADIS16240_DIAG_STAT_CPT_BUF_FUL BIT(7)
+
/* Flash test, checksum flag: 1 = mismatch, 0 = match */
#define ADIS16240_DIAG_STAT_CHKSUM BIT(6)
+
/* Power-on, self-test flag: 1 = failure, 0 = pass */
#define ADIS16240_DIAG_STAT_PWRON_FAIL_BIT 5
+
/* Power-on self-test: 1 = in-progress, 0 = complete */
#define ADIS16240_DIAG_STAT_PWRON_BUSY BIT(4)
+
/* SPI communications failure */
#define ADIS16240_DIAG_STAT_SPI_FAIL_BIT 3
+
/* Flash update failure */
#define ADIS16240_DIAG_STAT_FLASH_UPT_BIT 2
+
/* Power supply above 3.625 V */
#define ADIS16240_DIAG_STAT_POWER_HIGH_BIT 1
+
/* Power supply below 3.15 V */
#define ADIS16240_DIAG_STAT_POWER_LOW_BIT 0
/* GLOB_CMD */
+
#define ADIS16240_GLOB_CMD_RESUME BIT(8)
#define ADIS16240_GLOB_CMD_SW_RESET BIT(7)
#define ADIS16240_GLOB_CMD_STANDBY BIT(2)
diff --git a/drivers/staging/iio/accel/adis16240_core.c b/drivers/staging/iio/accel/adis16240_core.c
index 1b5b685..d5b99e6 100644
--- a/drivers/staging/iio/accel/adis16240_core.c
+++ b/drivers/staging/iio/accel/adis16240_core.c
@@ -29,13 +29,13 @@
static ssize_t adis16240_spi_read_signed(struct device *dev,
struct device_attribute *attr,
char *buf,
- unsigned bits)
+ unsigned int bits)
{
struct iio_dev *indio_dev = dev_to_iio_dev(dev);
struct adis *st = iio_priv(indio_dev);
int ret;
s16 val = 0;
- unsigned shift = 16 - bits;
+ unsigned int shift = 16 - bits;
struct iio_dev_attr *this_attr = to_iio_dev_attr(attr);
ret = adis_read_reg_16(st,
@@ -222,6 +222,7 @@
.diag_stat_reg = ADIS16240_DIAG_STAT,
.self_test_mask = ADIS16240_MSC_CTRL_SELF_TEST_EN,
+ .self_test_no_autoclear = true,
.startup_delay = ADIS16240_STARTUP_DELAY,
.status_error_msgs = adis16240_status_error_msgs,
diff --git a/drivers/staging/iio/adc/ad7192.c b/drivers/staging/iio/adc/ad7192.c
index f843f19..1cf6b79 100644
--- a/drivers/staging/iio/adc/ad7192.c
+++ b/drivers/staging/iio/adc/ad7192.c
@@ -35,10 +35,10 @@
#define AD7192_REG_DATA 3 /* Data Register (RO, 24/32-bit) */
#define AD7192_REG_ID 4 /* ID Register (RO, 8-bit) */
#define AD7192_REG_GPOCON 5 /* GPOCON Register (RO, 8-bit) */
-#define AD7192_REG_OFFSET 6 /* Offset Register (RW, 16-bit
- * (AD7792)/24-bit (AD7192)) */
-#define AD7192_REG_FULLSALE 7 /* Full-Scale Register
- * (RW, 16-bit (AD7792)/24-bit (AD7192)) */
+#define AD7192_REG_OFFSET 6 /* Offset Register (RW, 16-bit */
+ /* (AD7792)/24-bit (AD7192)) */
+#define AD7192_REG_FULLSALE 7 /* Full-Scale Register */
+ /* (RW, 16-bit (AD7792)/24-bit (AD7192)) */
/* Communications Register Bit Designations (AD7192_REG_COMM) */
#define AD7192_COMM_WEN BIT(7) /* Write Enable */
@@ -80,13 +80,13 @@
#define AD7192_MODE_CAL_SYS_FULL 7 /* System Full-Scale Calibration */
/* Mode Register: AD7192_MODE_CLKSRC options */
-#define AD7192_CLK_EXT_MCLK1_2 0 /* External 4.92 MHz Clock connected
- * from MCLK1 to MCLK2 */
+#define AD7192_CLK_EXT_MCLK1_2 0 /* External 4.92 MHz Clock connected*/
+ /* from MCLK1 to MCLK2 */
#define AD7192_CLK_EXT_MCLK2 1 /* External Clock applied to MCLK2 */
-#define AD7192_CLK_INT 2 /* Internal 4.92 MHz Clock not
- * available at the MCLK2 pin */
-#define AD7192_CLK_INT_CO 3 /* Internal 4.92 MHz Clock available
- * at the MCLK2 pin */
+#define AD7192_CLK_INT 2 /* Internal 4.92 MHz Clock not */
+ /* available at the MCLK2 pin */
+#define AD7192_CLK_INT_CO 3 /* Internal 4.92 MHz Clock available*/
+ /* at the MCLK2 pin */
/* Configuration Register Bit Designations (AD7192_REG_CONF) */
@@ -349,11 +349,9 @@
if (lval == 0)
return -EINVAL;
- mutex_lock(&indio_dev->mlock);
- if (iio_buffer_enabled(indio_dev)) {
- mutex_unlock(&indio_dev->mlock);
- return -EBUSY;
- }
+ ret = iio_device_claim_direct_mode(indio_dev);
+ if (ret)
+ return ret;
div = st->mclk / (lval * st->f_order * 1024);
if (div < 1 || div > 1023) {
@@ -366,7 +364,7 @@
ad_sd_write_reg(&st->sd, AD7192_REG_MODE, 3, st->mode);
out:
- mutex_unlock(&indio_dev->mlock);
+ iio_device_release_direct_mode(indio_dev);
return ret ? ret : len;
}
@@ -434,11 +432,9 @@
if (ret < 0)
return ret;
- mutex_lock(&indio_dev->mlock);
- if (iio_buffer_enabled(indio_dev)) {
- mutex_unlock(&indio_dev->mlock);
- return -EBUSY;
- }
+ ret = iio_device_claim_direct_mode(indio_dev);
+ if (ret)
+ return ret;
switch ((u32)this_attr->address) {
case AD7192_REG_GPOCON:
@@ -461,7 +457,7 @@
ret = -EINVAL;
}
- mutex_unlock(&indio_dev->mlock);
+ iio_device_release_direct_mode(indio_dev);
return ret ? ret : len;
}
@@ -555,11 +551,9 @@
int ret, i;
unsigned int tmp;
- mutex_lock(&indio_dev->mlock);
- if (iio_buffer_enabled(indio_dev)) {
- mutex_unlock(&indio_dev->mlock);
- return -EBUSY;
- }
+ ret = iio_device_claim_direct_mode(indio_dev);
+ if (ret)
+ return ret;
switch (mask) {
case IIO_CHAN_INFO_SCALE:
@@ -582,7 +576,7 @@
ret = -EINVAL;
}
- mutex_unlock(&indio_dev->mlock);
+ iio_device_release_direct_mode(indio_dev);
return ret;
}
diff --git a/drivers/staging/iio/adc/ad7280a.c b/drivers/staging/iio/adc/ad7280a.c
index 62e5eca..a06b46c 100644
--- a/drivers/staging/iio/adc/ad7280a.c
+++ b/drivers/staging/iio/adc/ad7280a.c
@@ -155,7 +155,7 @@
}
}
-static unsigned char ad7280_calc_crc8(unsigned char *crc_tab, unsigned val)
+static unsigned char ad7280_calc_crc8(unsigned char *crc_tab, unsigned int val)
{
unsigned char crc;
@@ -165,7 +165,7 @@
return crc ^ (val & 0xFF);
}
-static int ad7280_check_crc(struct ad7280_state *st, unsigned val)
+static int ad7280_check_crc(struct ad7280_state *st, unsigned int val)
{
unsigned char crc = ad7280_calc_crc8(st->crc_tab, val >> 10);
@@ -191,7 +191,7 @@
usleep_range(250, 500);
}
-static int __ad7280_read32(struct ad7280_state *st, unsigned *val)
+static int __ad7280_read32(struct ad7280_state *st, unsigned int *val)
{
int ret;
struct spi_transfer t = {
@@ -211,10 +211,10 @@
return 0;
}
-static int ad7280_write(struct ad7280_state *st, unsigned devaddr,
- unsigned addr, bool all, unsigned val)
+static int ad7280_write(struct ad7280_state *st, unsigned int devaddr,
+ unsigned int addr, bool all, unsigned int val)
{
- unsigned reg = devaddr << 27 | addr << 21 |
+ unsigned int reg = devaddr << 27 | addr << 21 |
(val & 0xFF) << 13 | all << 12;
reg |= ad7280_calc_crc8(st->crc_tab, reg >> 11) << 3 | 0x2;
@@ -223,11 +223,11 @@
return spi_write(st->spi, &st->buf[0], 4);
}
-static int ad7280_read(struct ad7280_state *st, unsigned devaddr,
- unsigned addr)
+static int ad7280_read(struct ad7280_state *st, unsigned int devaddr,
+ unsigned int addr)
{
int ret;
- unsigned tmp;
+ unsigned int tmp;
/* turns off the read operation on all parts */
ret = ad7280_write(st, AD7280A_DEVADDR_MASTER, AD7280A_CONTROL_HB, 1,
@@ -261,11 +261,11 @@
return (tmp >> 13) & 0xFF;
}
-static int ad7280_read_channel(struct ad7280_state *st, unsigned devaddr,
- unsigned addr)
+static int ad7280_read_channel(struct ad7280_state *st, unsigned int devaddr,
+ unsigned int addr)
{
int ret;
- unsigned tmp;
+ unsigned int tmp;
ret = ad7280_write(st, devaddr, AD7280A_READ, 0, addr << 2);
if (ret)
@@ -299,11 +299,11 @@
return (tmp >> 11) & 0xFFF;
}
-static int ad7280_read_all_channels(struct ad7280_state *st, unsigned cnt,
- unsigned *array)
+static int ad7280_read_all_channels(struct ad7280_state *st, unsigned int cnt,
+ unsigned int *array)
{
int i, ret;
- unsigned tmp, sum = 0;
+ unsigned int tmp, sum = 0;
ret = ad7280_write(st, AD7280A_DEVADDR_MASTER, AD7280A_READ, 1,
AD7280A_CELL_VOLTAGE_1 << 2);
@@ -338,7 +338,7 @@
static int ad7280_chain_setup(struct ad7280_state *st)
{
- unsigned val, n;
+ unsigned int val, n;
int ret;
ret = ad7280_write(st, AD7280A_DEVADDR_MASTER, AD7280A_CONTROL_LB, 1,
@@ -401,7 +401,7 @@
struct iio_dev_attr *this_attr = to_iio_dev_attr(attr);
bool readin;
int ret;
- unsigned devaddr, ch;
+ unsigned int devaddr, ch;
ret = strtobool(buf, &readin);
if (ret)
@@ -431,7 +431,7 @@
struct ad7280_state *st = iio_priv(indio_dev);
struct iio_dev_attr *this_attr = to_iio_dev_attr(attr);
int ret;
- unsigned msecs;
+ unsigned int msecs;
mutex_lock(&indio_dev->mlock);
ret = ad7280_read(st, this_attr->address >> 8,
@@ -602,7 +602,7 @@
struct iio_dev *indio_dev = dev_to_iio_dev(dev);
struct ad7280_state *st = iio_priv(indio_dev);
struct iio_dev_attr *this_attr = to_iio_dev_attr(attr);
- unsigned val;
+ unsigned int val;
switch ((u32)this_attr->address) {
case AD7280A_CELL_OVERVOLTAGE:
@@ -683,7 +683,7 @@
{
struct iio_dev *indio_dev = private;
struct ad7280_state *st = iio_priv(indio_dev);
- unsigned *channels;
+ unsigned int *channels;
int i, ret;
channels = kcalloc(st->scan_cnt, sizeof(*channels), GFP_KERNEL);
diff --git a/drivers/staging/iio/adc/ad7280a.h b/drivers/staging/iio/adc/ad7280a.h
index 732347a..ccfb90d 100644
--- a/drivers/staging/iio/adc/ad7280a.h
+++ b/drivers/staging/iio/adc/ad7280a.h
@@ -29,10 +29,10 @@
#define AD7280A_ALERT_REMOVE_AUX4_AUX5 BIT(1)
struct ad7280_platform_data {
- unsigned acquisition_time;
- unsigned conversion_averaging;
- unsigned chain_last_alert_ignore;
- bool thermistor_term_en;
+ unsigned int acquisition_time;
+ unsigned int conversion_averaging;
+ unsigned int chain_last_alert_ignore;
+ bool thermistor_term_en;
};
#endif /* IIO_ADC_AD7280_H_ */
diff --git a/drivers/staging/iio/adc/ad7606.h b/drivers/staging/iio/adc/ad7606.h
index cca9469..39f5044 100644
--- a/drivers/staging/iio/adc/ad7606.h
+++ b/drivers/staging/iio/adc/ad7606.h
@@ -28,16 +28,16 @@
*/
struct ad7606_platform_data {
- unsigned default_os;
- unsigned default_range;
- unsigned gpio_convst;
- unsigned gpio_reset;
- unsigned gpio_range;
- unsigned gpio_os0;
- unsigned gpio_os1;
- unsigned gpio_os2;
- unsigned gpio_frstdata;
- unsigned gpio_stby;
+ unsigned int default_os;
+ unsigned int default_range;
+ unsigned int gpio_convst;
+ unsigned int gpio_reset;
+ unsigned int gpio_range;
+ unsigned int gpio_os0;
+ unsigned int gpio_os1;
+ unsigned int gpio_os2;
+ unsigned int gpio_frstdata;
+ unsigned int gpio_stby;
};
/**
@@ -52,7 +52,7 @@
const char *name;
u16 int_vref_mv;
const struct iio_chan_spec *channels;
- unsigned num_channels;
+ unsigned int num_channels;
};
/**
@@ -67,8 +67,8 @@
struct work_struct poll_work;
wait_queue_head_t wq_data_avail;
const struct ad7606_bus_ops *bops;
- unsigned range;
- unsigned oversampling;
+ unsigned int range;
+ unsigned int oversampling;
bool done;
void __iomem *base_address;
@@ -86,7 +86,7 @@
};
struct iio_dev *ad7606_probe(struct device *dev, int irq,
- void __iomem *base_address, unsigned id,
+ void __iomem *base_address, unsigned int id,
const struct ad7606_bus_ops *bops);
int ad7606_remove(struct iio_dev *indio_dev, int irq);
int ad7606_reset(struct ad7606_state *st);
diff --git a/drivers/staging/iio/adc/ad7606_core.c b/drivers/staging/iio/adc/ad7606_core.c
index fe6caee..f79ee61 100644
--- a/drivers/staging/iio/adc/ad7606_core.c
+++ b/drivers/staging/iio/adc/ad7606_core.c
@@ -36,7 +36,7 @@
return -ENODEV;
}
-static int ad7606_scan_direct(struct iio_dev *indio_dev, unsigned ch)
+static int ad7606_scan_direct(struct iio_dev *indio_dev, unsigned int ch)
{
struct ad7606_state *st = iio_priv(indio_dev);
int ret;
@@ -88,12 +88,12 @@
switch (m) {
case IIO_CHAN_INFO_RAW:
- mutex_lock(&indio_dev->mlock);
- if (iio_buffer_enabled(indio_dev))
- ret = -EBUSY;
- else
- ret = ad7606_scan_direct(indio_dev, chan->address);
- mutex_unlock(&indio_dev->mlock);
+ ret = iio_device_claim_direct_mode(indio_dev);
+ if (ret)
+ return ret;
+
+ ret = ad7606_scan_direct(indio_dev, chan->address);
+ iio_device_release_direct_mode(indio_dev);
if (ret < 0)
return ret;
@@ -155,7 +155,7 @@
return sprintf(buf, "%u\n", st->oversampling);
}
-static int ad7606_oversampling_get_index(unsigned val)
+static int ad7606_oversampling_get_index(unsigned int val)
{
unsigned char supported[] = {0, 2, 4, 8, 16, 32, 64};
int i;
@@ -446,7 +446,7 @@
struct iio_dev *ad7606_probe(struct device *dev, int irq,
void __iomem *base_address,
- unsigned id,
+ unsigned int id,
const struct ad7606_bus_ops *bops)
{
struct ad7606_platform_data *pdata = dev->platform_data;
diff --git a/drivers/staging/iio/adc/ad7606_spi.c b/drivers/staging/iio/adc/ad7606_spi.c
index d873a51..825da07 100644
--- a/drivers/staging/iio/adc/ad7606_spi.c
+++ b/drivers/staging/iio/adc/ad7606_spi.c
@@ -21,7 +21,8 @@
{
struct spi_device *spi = to_spi_device(dev);
int i, ret;
- unsigned short *data = buf;
+ unsigned short *data;
+ __be16 *bdata = buf;
ret = spi_read(spi, buf, count * 2);
if (ret < 0) {
@@ -30,7 +31,7 @@
}
for (i = 0; i < count; i++)
- data[i] = be16_to_cpu(data[i]);
+ data[i] = be16_to_cpu(bdata[i]);
return 0;
}
diff --git a/drivers/staging/iio/adc/ad7780.c b/drivers/staging/iio/adc/ad7780.c
index 1439cfd..c9a0c2a 100644
--- a/drivers/staging/iio/adc/ad7780.c
+++ b/drivers/staging/iio/adc/ad7780.c
@@ -63,7 +63,7 @@
enum ad_sigma_delta_mode mode)
{
struct ad7780_state *st = ad_sigma_delta_to_ad7780(sigma_delta);
- unsigned val;
+ unsigned int val;
switch (mode) {
case AD_SD_MODE_SINGLE:
diff --git a/drivers/staging/iio/frequency/ad9832.c b/drivers/staging/iio/frequency/ad9832.c
index 18b27a1..358400b 100644
--- a/drivers/staging/iio/frequency/ad9832.c
+++ b/drivers/staging/iio/frequency/ad9832.c
@@ -31,7 +31,7 @@
}
static int ad9832_write_frequency(struct ad9832_state *st,
- unsigned addr, unsigned long fout)
+ unsigned int addr, unsigned long fout)
{
unsigned long regval;
diff --git a/drivers/staging/iio/impedance-analyzer/ad5933.c b/drivers/staging/iio/impedance-analyzer/ad5933.c
index d1218d8..9f43976 100644
--- a/drivers/staging/iio/impedance-analyzer/ad5933.c
+++ b/drivers/staging/iio/impedance-analyzer/ad5933.c
@@ -12,20 +12,16 @@
#include <linux/sysfs.h>
#include <linux/i2c.h>
#include <linux/regulator/consumer.h>
-#include <linux/slab.h>
#include <linux/types.h>
#include <linux/err.h>
#include <linux/delay.h>
#include <linux/module.h>
-#include <asm/div64.h>
#include <linux/iio/iio.h>
#include <linux/iio/sysfs.h>
#include <linux/iio/buffer.h>
#include <linux/iio/kfifo_buf.h>
-#include "ad5933.h"
-
/* AD5933/AD5934 Registers */
#define AD5933_REG_CONTROL_HB 0x80 /* R/W, 2 bytes */
#define AD5933_REG_CONTROL_LB 0x81 /* R/W, 2 bytes */
@@ -86,6 +82,18 @@
#define AD5933_POLL_TIME_ms 10
#define AD5933_INIT_EXCITATION_TIME_ms 100
+/**
+ * struct ad5933_platform_data - platform specific data
+ * @ext_clk_Hz: the external clock frequency in Hz, if not set
+ * the driver uses the internal clock (16.776 MHz)
+ * @vref_mv: the external reference voltage in millivolt
+ */
+
+struct ad5933_platform_data {
+ unsigned long ext_clk_Hz;
+ unsigned short vref_mv;
+};
+
struct ad5933_state {
struct i2c_client *client;
struct regulator *reg;
@@ -93,14 +101,14 @@
unsigned long mclk_hz;
unsigned char ctrl_hb;
unsigned char ctrl_lb;
- unsigned range_avail[4];
+ unsigned int range_avail[4];
unsigned short vref_mv;
unsigned short settling_cycles;
unsigned short freq_points;
- unsigned freq_start;
- unsigned freq_inc;
- unsigned state;
- unsigned poll_time_jiffies;
+ unsigned int freq_start;
+ unsigned int freq_inc;
+ unsigned int state;
+ unsigned int poll_time_jiffies;
};
static struct ad5933_platform_data ad5933_default_pdata = {
@@ -214,7 +222,7 @@
}
static int ad5933_set_freq(struct ad5933_state *st,
- unsigned reg, unsigned long freq)
+ unsigned int reg, unsigned long freq)
{
unsigned long long freqreg;
union {
@@ -274,7 +282,7 @@
static void ad5933_calc_out_ranges(struct ad5933_state *st)
{
int i;
- unsigned normalized_3v3[4] = {1980, 198, 383, 970};
+ unsigned int normalized_3v3[4] = {1980, 198, 383, 970};
for (i = 0; i < 4; i++)
st->range_avail[i] = normalized_3v3[i] * st->vref_mv / 3300;
@@ -307,10 +315,10 @@
freqreg = be32_to_cpu(dat.d32) & 0xFFFFFF;
- freqreg = (u64) freqreg * (u64) (st->mclk_hz / 4);
+ freqreg = (u64)freqreg * (u64)(st->mclk_hz / 4);
do_div(freqreg, 1 << 27);
- return sprintf(buf, "%d\n", (int) freqreg);
+ return sprintf(buf, "%d\n", (int)freqreg);
}
static ssize_t ad5933_store_frequency(struct device *dev,
@@ -358,7 +366,7 @@
int ret = 0, len = 0;
mutex_lock(&indio_dev->mlock);
- switch ((u32) this_attr->address) {
+ switch ((u32)this_attr->address) {
case AD5933_OUT_RANGE:
len = sprintf(buf, "%u\n",
st->range_avail[(st->ctrl_hb >> 1) & 0x3]);
@@ -409,7 +417,7 @@
}
mutex_lock(&indio_dev->mlock);
- switch ((u32) this_attr->address) {
+ switch ((u32)this_attr->address) {
case AD5933_OUT_RANGE:
for (i = 0; i < 4; i++)
if (val == st->range_avail[i]) {
@@ -683,8 +691,9 @@
}
if (status & AD5933_STAT_SWEEP_DONE) {
- /* last sample received - power down do nothing until
- * the ring enable is toggled */
+ /* last sample received - power down do
+ * nothing until the ring enable is toggled
+ */
ad5933_cmd(st, AD5933_CTRL_POWER_DOWN);
} else {
/* we just received a valid datum, move on to the next */
@@ -699,7 +708,7 @@
const struct i2c_device_id *id)
{
int ret, voltage_uv = 0;
- struct ad5933_platform_data *pdata = client->dev.platform_data;
+ struct ad5933_platform_data *pdata = dev_get_platdata(&client->dev);
struct ad5933_state *st;
struct iio_dev *indio_dev;
diff --git a/drivers/staging/iio/impedance-analyzer/ad5933.h b/drivers/staging/iio/impedance-analyzer/ad5933.h
deleted file mode 100644
index b140e42..0000000
--- a/drivers/staging/iio/impedance-analyzer/ad5933.h
+++ /dev/null
@@ -1,28 +0,0 @@
-/*
- * AD5933 AD5934 Impedance Converter, Network Analyzer
- *
- * Copyright 2011 Analog Devices Inc.
- *
- * Licensed under the GPL-2.
- */
-
-#ifndef IIO_ADC_AD5933_H_
-#define IIO_ADC_AD5933_H_
-
-/*
- * TODO: struct ad5933_platform_data needs to go into include/linux/iio
- */
-
-/**
- * struct ad5933_platform_data - platform specific data
- * @ext_clk_Hz: the external clock frequency in Hz, if not set
- * the driver uses the internal clock (16.776 MHz)
- * @vref_mv: the external reference voltage in millivolt
- */
-
-struct ad5933_platform_data {
- unsigned long ext_clk_Hz;
- unsigned short vref_mv;
-};
-
-#endif /* IIO_ADC_AD5933_H_ */
diff --git a/drivers/staging/iio/light/isl29028.c b/drivers/staging/iio/light/isl29028.c
index 6e2ba45..2e3b1d6 100644
--- a/drivers/staging/iio/light/isl29028.c
+++ b/drivers/staging/iio/light/isl29028.c
@@ -69,7 +69,6 @@
};
struct isl29028_chip {
- struct device *dev;
struct mutex lock;
struct regmap *regmap;
@@ -166,20 +165,21 @@
static int isl29028_read_als_ir(struct isl29028_chip *chip, int *als_ir)
{
+ struct device *dev = regmap_get_device(chip->regmap);
unsigned int lsb;
unsigned int msb;
int ret;
ret = regmap_read(chip->regmap, ISL29028_REG_ALSIR_L, &lsb);
if (ret < 0) {
- dev_err(chip->dev,
+ dev_err(dev,
"Error in reading register ALSIR_L err %d\n", ret);
return ret;
}
ret = regmap_read(chip->regmap, ISL29028_REG_ALSIR_U, &msb);
if (ret < 0) {
- dev_err(chip->dev,
+ dev_err(dev,
"Error in reading register ALSIR_U err %d\n", ret);
return ret;
}
@@ -190,12 +190,13 @@
static int isl29028_read_proxim(struct isl29028_chip *chip, int *prox)
{
+ struct device *dev = regmap_get_device(chip->regmap);
unsigned int data;
int ret;
ret = regmap_read(chip->regmap, ISL29028_REG_PROX_DATA, &data);
if (ret < 0) {
- dev_err(chip->dev, "Error in reading register %d, error %d\n",
+ dev_err(dev, "Error in reading register %d, error %d\n",
ISL29028_REG_PROX_DATA, ret);
return ret;
}
@@ -218,13 +219,14 @@
static int isl29028_als_get(struct isl29028_chip *chip, int *als_data)
{
+ struct device *dev = regmap_get_device(chip->regmap);
int ret;
int als_ir_data;
if (chip->als_ir_mode != MODE_ALS) {
ret = isl29028_set_als_ir_mode(chip, MODE_ALS);
if (ret < 0) {
- dev_err(chip->dev,
+ dev_err(dev,
"Error in enabling ALS mode err %d\n", ret);
return ret;
}
@@ -251,12 +253,13 @@
static int isl29028_ir_get(struct isl29028_chip *chip, int *ir_data)
{
+ struct device *dev = regmap_get_device(chip->regmap);
int ret;
if (chip->als_ir_mode != MODE_IR) {
ret = isl29028_set_als_ir_mode(chip, MODE_IR);
if (ret < 0) {
- dev_err(chip->dev,
+ dev_err(dev,
"Error in enabling IR mode err %d\n", ret);
return ret;
}
@@ -271,25 +274,26 @@
int val, int val2, long mask)
{
struct isl29028_chip *chip = iio_priv(indio_dev);
+ struct device *dev = regmap_get_device(chip->regmap);
int ret = -EINVAL;
mutex_lock(&chip->lock);
switch (chan->type) {
case IIO_PROXIMITY:
if (mask != IIO_CHAN_INFO_SAMP_FREQ) {
- dev_err(chip->dev,
+ dev_err(dev,
"proximity: mask value 0x%08lx not supported\n",
mask);
break;
}
if (val < 1 || val > 100) {
- dev_err(chip->dev,
+ dev_err(dev,
"Samp_freq %d is not in range[1:100]\n", val);
break;
}
ret = isl29028_set_proxim_sampling(chip, val);
if (ret < 0) {
- dev_err(chip->dev,
+ dev_err(dev,
"Setting proximity samp_freq fail, err %d\n",
ret);
break;
@@ -299,19 +303,19 @@
case IIO_LIGHT:
if (mask != IIO_CHAN_INFO_SCALE) {
- dev_err(chip->dev,
+ dev_err(dev,
"light: mask value 0x%08lx not supported\n",
mask);
break;
}
if ((val != 125) && (val != 2000)) {
- dev_err(chip->dev,
+ dev_err(dev,
"lux scale %d is invalid [125, 2000]\n", val);
break;
}
ret = isl29028_set_als_scale(chip, val);
if (ret < 0) {
- dev_err(chip->dev,
+ dev_err(dev,
"Setting lux scale fail with error %d\n", ret);
break;
}
@@ -319,7 +323,7 @@
break;
default:
- dev_err(chip->dev, "Unsupported channel type\n");
+ dev_err(dev, "Unsupported channel type\n");
break;
}
mutex_unlock(&chip->lock);
@@ -331,6 +335,7 @@
int *val, int *val2, long mask)
{
struct isl29028_chip *chip = iio_priv(indio_dev);
+ struct device *dev = regmap_get_device(chip->regmap);
int ret = -EINVAL;
mutex_lock(&chip->lock);
@@ -370,7 +375,7 @@
break;
default:
- dev_err(chip->dev, "mask value 0x%08lx not supported\n", mask);
+ dev_err(dev, "mask value 0x%08lx not supported\n", mask);
break;
}
mutex_unlock(&chip->lock);
@@ -417,6 +422,7 @@
static int isl29028_chip_init(struct isl29028_chip *chip)
{
+ struct device *dev = regmap_get_device(chip->regmap);
int ret;
chip->enable_prox = false;
@@ -426,35 +432,33 @@
ret = regmap_write(chip->regmap, ISL29028_REG_TEST1_MODE, 0x0);
if (ret < 0) {
- dev_err(chip->dev, "%s(): write to reg %d failed, err = %d\n",
+ dev_err(dev, "%s(): write to reg %d failed, err = %d\n",
__func__, ISL29028_REG_TEST1_MODE, ret);
return ret;
}
ret = regmap_write(chip->regmap, ISL29028_REG_TEST2_MODE, 0x0);
if (ret < 0) {
- dev_err(chip->dev, "%s(): write to reg %d failed, err = %d\n",
+ dev_err(dev, "%s(): write to reg %d failed, err = %d\n",
__func__, ISL29028_REG_TEST2_MODE, ret);
return ret;
}
ret = regmap_write(chip->regmap, ISL29028_REG_CONFIGURE, 0x0);
if (ret < 0) {
- dev_err(chip->dev, "%s(): write to reg %d failed, err = %d\n",
+ dev_err(dev, "%s(): write to reg %d failed, err = %d\n",
__func__, ISL29028_REG_CONFIGURE, ret);
return ret;
}
ret = isl29028_set_proxim_sampling(chip, chip->prox_sampling);
if (ret < 0) {
- dev_err(chip->dev, "setting the proximity, err = %d\n",
- ret);
+ dev_err(dev, "setting the proximity, err = %d\n", ret);
return ret;
}
ret = isl29028_set_als_scale(chip, chip->lux_scale);
if (ret < 0)
- dev_err(chip->dev,
- "setting als scale failed, err = %d\n", ret);
+ dev_err(dev, "setting als scale failed, err = %d\n", ret);
return ret;
}
@@ -496,19 +500,19 @@
chip = iio_priv(indio_dev);
i2c_set_clientdata(client, indio_dev);
- chip->dev = &client->dev;
mutex_init(&chip->lock);
chip->regmap = devm_regmap_init_i2c(client, &isl29028_regmap_config);
if (IS_ERR(chip->regmap)) {
ret = PTR_ERR(chip->regmap);
- dev_err(chip->dev, "regmap initialization failed: %d\n", ret);
+ dev_err(&client->dev, "regmap initialization failed: %d\n",
+ ret);
return ret;
}
ret = isl29028_chip_init(chip);
if (ret < 0) {
- dev_err(chip->dev, "chip initialization failed: %d\n", ret);
+ dev_err(&client->dev, "chip initialization failed: %d\n", ret);
return ret;
}
@@ -520,7 +524,8 @@
indio_dev->modes = INDIO_DIRECT_MODE;
ret = devm_iio_device_register(indio_dev->dev.parent, indio_dev);
if (ret < 0) {
- dev_err(chip->dev, "iio registration fails with error %d\n",
+ dev_err(&client->dev,
+ "iio registration fails with error %d\n",
ret);
return ret;
}
diff --git a/drivers/staging/iio/light/tsl2x7x_core.c b/drivers/staging/iio/light/tsl2x7x_core.c
index 5f308ba..d553c8e 100644
--- a/drivers/staging/iio/light/tsl2x7x_core.c
+++ b/drivers/staging/iio/light/tsl2x7x_core.c
@@ -187,9 +187,11 @@
const struct tsl2x7x_chip_info *chip_info;
const struct iio_info *info;
s64 event_timestamp;
- /* This structure is intentionally large to accommodate
- * updates via sysfs. */
- /* Sized to 9 = max 8 segments + 1 termination segment */
+ /*
+ * This structure is intentionally large to accommodate
+ * updates via sysfs.
+ * Sized to 9 = max 8 segments + 1 termination segment
+ */
struct tsl2x7x_lux tsl2x7x_device_lux[TSL2X7X_MAX_LUX_TABLE_SIZE];
};
@@ -349,13 +351,13 @@
if (chip->tsl2x7x_chip_status != TSL2X7X_CHIP_WORKING) {
/* device is not enabled */
dev_err(&chip->client->dev, "%s: device is not enabled\n",
- __func__);
+ __func__);
ret = -EBUSY;
goto out_unlock;
}
ret = tsl2x7x_i2c_read(chip->client,
- (TSL2X7X_CMD_REG | TSL2X7X_STATUS), &buf[0]);
+ (TSL2X7X_CMD_REG | TSL2X7X_STATUS), &buf[0]);
if (ret < 0) {
dev_err(&chip->client->dev,
"%s: Failed to read STATUS Reg\n", __func__);
@@ -371,8 +373,8 @@
for (i = 0; i < 4; i++) {
ret = tsl2x7x_i2c_read(chip->client,
- (TSL2X7X_CMD_REG | (TSL2X7X_ALS_CHAN0LO + i)),
- &buf[i]);
+ (TSL2X7X_CMD_REG |
+ (TSL2X7X_ALS_CHAN0LO + i)), &buf[i]);
if (ret < 0) {
dev_err(&chip->client->dev,
"failed to read. err=%x\n", ret);
@@ -382,9 +384,9 @@
/* clear any existing interrupt status */
ret = i2c_smbus_write_byte(chip->client,
- (TSL2X7X_CMD_REG |
- TSL2X7X_CMD_SPL_FN |
- TSL2X7X_CMD_ALS_INT_CLR));
+ (TSL2X7X_CMD_REG |
+ TSL2X7X_CMD_SPL_FN |
+ TSL2X7X_CMD_ALS_INT_CLR));
if (ret < 0) {
dev_err(&chip->client->dev,
"i2c_write_command failed - err = %d\n", ret);
@@ -411,7 +413,7 @@
/* calculate ratio */
ratio = (ch1 << 15) / ch0;
/* convert to unscaled lux using the pointer to the table */
- p = (struct tsl2x7x_lux *) chip->tsl2x7x_device_lux;
+ p = (struct tsl2x7x_lux *)chip->tsl2x7x_device_lux;
while (p->ratio != 0 && p->ratio < ratio)
p++;
@@ -488,7 +490,7 @@
}
ret = tsl2x7x_i2c_read(chip->client,
- (TSL2X7X_CMD_REG | TSL2X7X_STATUS), &status);
+ (TSL2X7X_CMD_REG | TSL2X7X_STATUS), &status);
if (ret < 0) {
dev_err(&chip->client->dev, "i2c err=%d\n", ret);
goto prox_poll_err;
@@ -515,8 +517,8 @@
for (i = 0; i < 2; i++) {
ret = tsl2x7x_i2c_read(chip->client,
- (TSL2X7X_CMD_REG |
- (TSL2X7X_PRX_LO + i)), &chdata[i]);
+ (TSL2X7X_CMD_REG |
+ (TSL2X7X_PRX_LO + i)), &chdata[i]);
if (ret < 0)
goto prox_poll_err;
}
@@ -542,19 +544,19 @@
{
/* If Operational settings defined elsewhere.. */
if (chip->pdata && chip->pdata->platform_default_settings)
- memcpy(&(chip->tsl2x7x_settings),
- chip->pdata->platform_default_settings,
- sizeof(tsl2x7x_default_settings));
+ memcpy(&chip->tsl2x7x_settings,
+ chip->pdata->platform_default_settings,
+ sizeof(tsl2x7x_default_settings));
else
- memcpy(&(chip->tsl2x7x_settings),
- &tsl2x7x_default_settings,
- sizeof(tsl2x7x_default_settings));
+ memcpy(&chip->tsl2x7x_settings,
+ &tsl2x7x_default_settings,
+ sizeof(tsl2x7x_default_settings));
/* Load up the proper lux table. */
if (chip->pdata && chip->pdata->platform_lux_table[0].ratio != 0)
memcpy(chip->tsl2x7x_device_lux,
- chip->pdata->platform_lux_table,
- sizeof(chip->pdata->platform_lux_table));
+ chip->pdata->platform_lux_table,
+ sizeof(chip->pdata->platform_lux_table));
else
memcpy(chip->tsl2x7x_device_lux,
(struct tsl2x7x_lux *)tsl2x7x_default_lux_table_group[chip->id],
@@ -576,7 +578,7 @@
int lux_val;
ret = i2c_smbus_write_byte(chip->client,
- (TSL2X7X_CMD_REG | TSL2X7X_CNTRL));
+ (TSL2X7X_CMD_REG | TSL2X7X_CNTRL));
if (ret < 0) {
dev_err(&chip->client->dev,
"failed to write CNTRL register, ret=%d\n", ret);
@@ -592,7 +594,7 @@
}
ret = i2c_smbus_write_byte(chip->client,
- (TSL2X7X_CMD_REG | TSL2X7X_CNTRL));
+ (TSL2X7X_CMD_REG | TSL2X7X_CNTRL));
if (ret < 0) {
dev_err(&chip->client->dev,
"failed to write ctrl reg: ret=%d\n", ret);
@@ -609,7 +611,7 @@
lux_val = tsl2x7x_get_lux(indio_dev);
if (lux_val < 0) {
dev_err(&chip->client->dev,
- "%s: failed to get lux\n", __func__);
+ "%s: failed to get lux\n", __func__);
return lux_val;
}
@@ -620,9 +622,9 @@
chip->tsl2x7x_settings.als_gain_trim = gain_trim_val;
dev_info(&chip->client->dev,
- "%s als_calibrate completed\n", chip->client->name);
+ "%s als_calibrate completed\n", chip->client->name);
- return (int) gain_trim_val;
+ return (int)gain_trim_val;
}
static int tsl2x7x_chip_on(struct iio_dev *indio_dev)
@@ -695,23 +697,28 @@
chip->als_saturation = als_count * 922; /* 90% of full scale */
chip->als_time_scale = (als_time + 25) / 50;
- /* TSL2X7X Specific power-on / adc enable sequence
- * Power on the device 1st. */
+ /*
+ * TSL2X7X Specific power-on / adc enable sequence
+ * Power on the device 1st.
+ */
utmp = TSL2X7X_CNTL_PWR_ON;
ret = i2c_smbus_write_byte_data(chip->client,
- TSL2X7X_CMD_REG | TSL2X7X_CNTRL, utmp);
+ TSL2X7X_CMD_REG | TSL2X7X_CNTRL, utmp);
if (ret < 0) {
dev_err(&chip->client->dev,
"%s: failed on CNTRL reg.\n", __func__);
return ret;
}
- /* Use the following shadow copy for our delay before enabling ADC.
- * Write all the registers. */
+ /*
+ * Use the following shadow copy for our delay before enabling ADC.
+ * Write all the registers.
+ */
for (i = 0, dev_reg = chip->tsl2x7x_config;
i < TSL2X7X_MAX_CONFIG_REG; i++) {
ret = i2c_smbus_write_byte_data(chip->client,
- TSL2X7X_CMD_REG + i, *dev_reg++);
+ TSL2X7X_CMD_REG + i,
+ *dev_reg++);
if (ret < 0) {
dev_err(&chip->client->dev,
"failed on write to reg %d.\n", i);
@@ -721,13 +728,15 @@
mdelay(3); /* Power-on settling time */
- /* NOW enable the ADC
- * initialize the desired mode of operation */
+ /*
+ * NOW enable the ADC
+ * initialize the desired mode of operation
+ */
utmp = TSL2X7X_CNTL_PWR_ON |
TSL2X7X_CNTL_ADC_ENBL |
TSL2X7X_CNTL_PROX_DET_ENBL;
ret = i2c_smbus_write_byte_data(chip->client,
- TSL2X7X_CMD_REG | TSL2X7X_CNTRL, utmp);
+ TSL2X7X_CMD_REG | TSL2X7X_CNTRL, utmp);
if (ret < 0) {
dev_err(&chip->client->dev,
"%s: failed on 2nd CTRL reg.\n", __func__);
@@ -741,12 +750,13 @@
reg_val = TSL2X7X_CNTL_PWR_ON | TSL2X7X_CNTL_ADC_ENBL;
if ((chip->tsl2x7x_settings.interrupts_en == 0x20) ||
- (chip->tsl2x7x_settings.interrupts_en == 0x30))
+ (chip->tsl2x7x_settings.interrupts_en == 0x30))
reg_val |= TSL2X7X_CNTL_PROX_DET_ENBL;
reg_val |= chip->tsl2x7x_settings.interrupts_en;
ret = i2c_smbus_write_byte_data(chip->client,
- (TSL2X7X_CMD_REG | TSL2X7X_CNTRL), reg_val);
+ (TSL2X7X_CMD_REG |
+ TSL2X7X_CNTRL), reg_val);
if (ret < 0)
dev_err(&chip->client->dev,
"%s: failed in tsl2x7x_IOCTL_INT_SET.\n",
@@ -754,8 +764,9 @@
/* Clear out any initial interrupts */
ret = i2c_smbus_write_byte(chip->client,
- TSL2X7X_CMD_REG | TSL2X7X_CMD_SPL_FN |
- TSL2X7X_CMD_PROXALS_INT_CLR);
+ TSL2X7X_CMD_REG |
+ TSL2X7X_CMD_SPL_FN |
+ TSL2X7X_CMD_PROXALS_INT_CLR);
if (ret < 0) {
dev_err(&chip->client->dev,
"%s: Failed to clear Int status\n",
@@ -776,7 +787,7 @@
chip->tsl2x7x_chip_status = TSL2X7X_CHIP_SUSPENDED;
ret = i2c_smbus_write_byte_data(chip->client,
- TSL2X7X_CMD_REG | TSL2X7X_CNTRL, 0x00);
+ TSL2X7X_CMD_REG | TSL2X7X_CNTRL, 0x00);
if (chip->pdata && chip->pdata->power_off)
chip->pdata->power_off(chip->client);
@@ -819,7 +830,7 @@
static
void tsl2x7x_prox_calculate(int *data, int length,
- struct tsl2x7x_prox_stat *statP)
+ struct tsl2x7x_prox_stat *statP)
{
int i;
int sample_sum;
@@ -843,7 +854,7 @@
tmp = data[i] - statP->mean;
sample_sum += tmp * tmp;
}
- statP->stddev = int_sqrt((long)sample_sum)/length;
+ statP->stddev = int_sqrt((long)sample_sum) / length;
}
/**
@@ -886,20 +897,21 @@
tsl2x7x_get_prox(indio_dev);
prox_history[i] = chip->prox_data;
dev_info(&chip->client->dev, "2 i=%d prox data= %d\n",
- i, chip->prox_data);
+ i, chip->prox_data);
}
tsl2x7x_chip_off(indio_dev);
calP = &prox_stat_data[PROX_STAT_CAL];
tsl2x7x_prox_calculate(prox_history,
- chip->tsl2x7x_settings.prox_max_samples_cal, calP);
+ chip->tsl2x7x_settings.prox_max_samples_cal,
+ calP);
chip->tsl2x7x_settings.prox_thres_high = (calP->max << 1) - calP->mean;
dev_info(&chip->client->dev, " cal min=%d mean=%d max=%d\n",
- calP->min, calP->mean, calP->max);
+ calP->min, calP->mean, calP->max);
dev_info(&chip->client->dev,
- "%s proximity threshold set to %d\n",
- chip->client->name, chip->tsl2x7x_settings.prox_thres_high);
+ "%s proximity threshold set to %d\n",
+ chip->client->name, chip->tsl2x7x_settings.prox_thres_high);
/* back to the way they were */
chip->tsl2x7x_settings.interrupts_en = tmp_irq_settings;
@@ -908,7 +920,8 @@
}
static ssize_t tsl2x7x_power_state_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+ struct device_attribute *attr,
+ char *buf)
{
struct tsl2X7X_chip *chip = iio_priv(dev_to_iio_dev(dev));
@@ -916,7 +929,8 @@
}
static ssize_t tsl2x7x_power_state_store(struct device *dev,
- struct device_attribute *attr, const char *buf, size_t len)
+ struct device_attribute *attr,
+ const char *buf, size_t len)
{
struct iio_dev *indio_dev = dev_to_iio_dev(dev);
bool value;
@@ -933,7 +947,8 @@
}
static ssize_t tsl2x7x_gain_available_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+ struct device_attribute *attr,
+ char *buf)
{
struct tsl2X7X_chip *chip = iio_priv(dev_to_iio_dev(dev));
@@ -950,13 +965,15 @@
}
static ssize_t tsl2x7x_prox_gain_available_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+ struct device_attribute *attr,
+ char *buf)
{
return snprintf(buf, PAGE_SIZE, "%s\n", "1 2 4 8");
}
static ssize_t tsl2x7x_als_time_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+ struct device_attribute *attr,
+ char *buf)
{
struct tsl2X7X_chip *chip = iio_priv(dev_to_iio_dev(dev));
int y, z;
@@ -970,7 +987,8 @@
}
static ssize_t tsl2x7x_als_time_store(struct device *dev,
- struct device_attribute *attr, const char *buf, size_t len)
+ struct device_attribute *attr,
+ const char *buf, size_t len)
{
struct iio_dev *indio_dev = dev_to_iio_dev(dev);
struct tsl2X7X_chip *chip = iio_priv(indio_dev);
@@ -986,7 +1004,7 @@
TSL2X7X_MAX_TIMER_CNT - (u8)result.fract;
dev_info(&chip->client->dev, "%s: als time = %d",
- __func__, chip->tsl2x7x_settings.als_time);
+ __func__, chip->tsl2x7x_settings.als_time);
tsl2x7x_invoke_change(indio_dev);
@@ -997,7 +1015,8 @@
".00272 - .696");
static ssize_t tsl2x7x_als_cal_target_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+ struct device_attribute *attr,
+ char *buf)
{
struct tsl2X7X_chip *chip = iio_priv(dev_to_iio_dev(dev));
@@ -1006,7 +1025,8 @@
}
static ssize_t tsl2x7x_als_cal_target_store(struct device *dev,
- struct device_attribute *attr, const char *buf, size_t len)
+ struct device_attribute *attr,
+ const char *buf, size_t len)
{
struct iio_dev *indio_dev = dev_to_iio_dev(dev);
struct tsl2X7X_chip *chip = iio_priv(indio_dev);
@@ -1025,7 +1045,8 @@
/* persistence settings */
static ssize_t tsl2x7x_als_persistence_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+ struct device_attribute *attr,
+ char *buf)
{
struct tsl2X7X_chip *chip = iio_priv(dev_to_iio_dev(dev));
int y, z, filter_delay;
@@ -1041,7 +1062,8 @@
}
static ssize_t tsl2x7x_als_persistence_store(struct device *dev,
- struct device_attribute *attr, const char *buf, size_t len)
+ struct device_attribute *attr,
+ const char *buf, size_t len)
{
struct iio_dev *indio_dev = dev_to_iio_dev(dev);
struct tsl2X7X_chip *chip = iio_priv(indio_dev);
@@ -1063,7 +1085,7 @@
chip->tsl2x7x_settings.persistence |= (filter_delay & 0x0F);
dev_info(&chip->client->dev, "%s: als persistence = %d",
- __func__, filter_delay);
+ __func__, filter_delay);
tsl2x7x_invoke_change(indio_dev);
@@ -1071,7 +1093,8 @@
}
static ssize_t tsl2x7x_prox_persistence_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+ struct device_attribute *attr,
+ char *buf)
{
struct tsl2X7X_chip *chip = iio_priv(dev_to_iio_dev(dev));
int y, z, filter_delay;
@@ -1087,7 +1110,8 @@
}
static ssize_t tsl2x7x_prox_persistence_store(struct device *dev,
- struct device_attribute *attr, const char *buf, size_t len)
+ struct device_attribute *attr,
+ const char *buf, size_t len)
{
struct iio_dev *indio_dev = dev_to_iio_dev(dev);
struct tsl2X7X_chip *chip = iio_priv(indio_dev);
@@ -1109,7 +1133,7 @@
chip->tsl2x7x_settings.persistence |= ((filter_delay << 4) & 0xF0);
dev_info(&chip->client->dev, "%s: prox persistence = %d",
- __func__, filter_delay);
+ __func__, filter_delay);
tsl2x7x_invoke_change(indio_dev);
@@ -1117,7 +1141,8 @@
}
static ssize_t tsl2x7x_do_calibrate(struct device *dev,
- struct device_attribute *attr, const char *buf, size_t len)
+ struct device_attribute *attr,
+ const char *buf, size_t len)
{
struct iio_dev *indio_dev = dev_to_iio_dev(dev);
bool value;
@@ -1134,7 +1159,8 @@
}
static ssize_t tsl2x7x_luxtable_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+ struct device_attribute *attr,
+ char *buf)
{
struct tsl2X7X_chip *chip = iio_priv(dev_to_iio_dev(dev));
int i = 0;
@@ -1146,8 +1172,10 @@
chip->tsl2x7x_device_lux[i].ch0,
chip->tsl2x7x_device_lux[i].ch1);
if (chip->tsl2x7x_device_lux[i].ratio == 0) {
- /* We just printed the first "0" entry.
- * Now get rid of the extra "," and break. */
+ /*
+ * We just printed the first "0" entry.
+ * Now get rid of the extra "," and break.
+ */
offset--;
break;
}
@@ -1159,11 +1187,12 @@
}
static ssize_t tsl2x7x_luxtable_store(struct device *dev,
- struct device_attribute *attr, const char *buf, size_t len)
+ struct device_attribute *attr,
+ const char *buf, size_t len)
{
struct iio_dev *indio_dev = dev_to_iio_dev(dev);
struct tsl2X7X_chip *chip = iio_priv(indio_dev);
- int value[ARRAY_SIZE(chip->tsl2x7x_device_lux)*3 + 1];
+ int value[ARRAY_SIZE(chip->tsl2x7x_device_lux) * 3 + 1];
int n;
get_options(buf, ARRAY_SIZE(value), value);
@@ -1175,7 +1204,7 @@
*/
n = value[0];
if ((n % 3) || n < 6 ||
- n > ((ARRAY_SIZE(chip->tsl2x7x_device_lux) - 1) * 3)) {
+ n > ((ARRAY_SIZE(chip->tsl2x7x_device_lux) - 1) * 3)) {
dev_info(dev, "LUX TABLE INPUT ERROR 1 Value[0]=%d\n", n);
return -EINVAL;
}
@@ -1198,7 +1227,8 @@
}
static ssize_t tsl2x7x_do_prox_calibrate(struct device *dev,
- struct device_attribute *attr, const char *buf, size_t len)
+ struct device_attribute *attr,
+ const char *buf, size_t len)
{
struct iio_dev *indio_dev = dev_to_iio_dev(dev);
bool value;
@@ -1391,10 +1421,10 @@
}
static int tsl2x7x_write_raw(struct iio_dev *indio_dev,
- struct iio_chan_spec const *chan,
- int val,
- int val2,
- long mask)
+ struct iio_chan_spec const *chan,
+ int val,
+ int val2,
+ long mask)
{
struct tsl2X7X_chip *chip = iio_priv(indio_dev);
@@ -1529,7 +1559,7 @@
u8 value;
value = i2c_smbus_read_byte_data(chip->client,
- TSL2X7X_CMD_REG | TSL2X7X_STATUS);
+ TSL2X7X_CMD_REG | TSL2X7X_STATUS);
/* What type of interrupt do we need to process */
if (value & TSL2X7X_STA_PRX_INTR) {
@@ -1545,16 +1575,16 @@
if (value & TSL2X7X_STA_ALS_INTR) {
tsl2x7x_get_lux(indio_dev); /* freshen data for ABI */
iio_push_event(indio_dev,
- IIO_UNMOD_EVENT_CODE(IIO_LIGHT,
- 0,
- IIO_EV_TYPE_THRESH,
- IIO_EV_DIR_EITHER),
- timestamp);
+ IIO_UNMOD_EVENT_CODE(IIO_LIGHT,
+ 0,
+ IIO_EV_TYPE_THRESH,
+ IIO_EV_DIR_EITHER),
+ timestamp);
}
/* Clear interrupt now that we have handled it. */
ret = i2c_smbus_write_byte(chip->client,
- TSL2X7X_CMD_REG | TSL2X7X_CMD_SPL_FN |
- TSL2X7X_CMD_PROXALS_INT_CLR);
+ TSL2X7X_CMD_REG | TSL2X7X_CMD_SPL_FN |
+ TSL2X7X_CMD_PROXALS_INT_CLR);
if (ret < 0)
dev_err(&chip->client->dev,
"Failed to clear irq from event handler. err = %d\n",
@@ -1616,6 +1646,7 @@
&dev_attr_in_intensity0_thresh_period.attr,
NULL,
};
+
static struct attribute *tsl2X7X_PRX_event_attrs[] = {
&dev_attr_in_proximity0_thresh_period.attr,
NULL,
@@ -1857,7 +1888,7 @@
};
static int tsl2x7x_probe(struct i2c_client *clientp,
- const struct i2c_device_id *id)
+ const struct i2c_device_id *id)
{
int ret;
unsigned char device_id;
@@ -1873,14 +1904,14 @@
i2c_set_clientdata(clientp, indio_dev);
ret = tsl2x7x_i2c_read(chip->client,
- TSL2X7X_CHIPID, &device_id);
+ TSL2X7X_CHIPID, &device_id);
if (ret < 0)
return ret;
if ((!tsl2x7x_device_id(&device_id, id->driver_data)) ||
- (tsl2x7x_device_id(&device_id, id->driver_data) == -EINVAL)) {
+ (tsl2x7x_device_id(&device_id, id->driver_data) == -EINVAL)) {
dev_info(&chip->client->dev,
- "%s: i2c device found does not match expected id\n",
+ "%s: i2c device found does not match expected id\n",
__func__);
return -EINVAL;
}
@@ -1892,8 +1923,10 @@
return ret;
}
- /* ALS and PROX functions can be invoked via user space poll
- * or H/W interrupt. If busy return last sample. */
+ /*
+ * ALS and PROX functions can be invoked via user space poll
+ * or H/W interrupt. If busy return last sample.
+ */
mutex_init(&chip->als_mutex);
mutex_init(&chip->prox_mutex);
diff --git a/drivers/staging/iio/meter/ade7753.c b/drivers/staging/iio/meter/ade7753.c
index 6928710..4b5f05f 100644
--- a/drivers/staging/iio/meter/ade7753.c
+++ b/drivers/staging/iio/meter/ade7753.c
@@ -333,7 +333,8 @@
if (enable)
irqen |= BIT(3); /* Enables an interrupt when a data is
- present in the waveform register */
+ * present in the waveform register
+ */
else
irqen &= ~BIT(3);
@@ -528,7 +529,6 @@
return iio_device_register(indio_dev);
}
-/* fixme, confirm ordering in this function */
static int ade7753_remove(struct spi_device *spi)
{
struct iio_dev *indio_dev = spi_get_drvdata(spi);
diff --git a/drivers/staging/iio/meter/ade7754.c b/drivers/staging/iio/meter/ade7754.c
index f4188e1..c46bef64 100644
--- a/drivers/staging/iio/meter/ade7754.c
+++ b/drivers/staging/iio/meter/ade7754.c
@@ -351,7 +351,8 @@
if (enable)
irqen |= BIT(14); /* Enables an interrupt when a data is
- present in the waveform register */
+ * present in the waveform register
+ */
else
irqen &= ~BIT(14);
@@ -558,7 +559,6 @@
return ret;
}
-/* fixme, confirm ordering in this function */
static int ade7754_remove(struct spi_device *spi)
{
struct iio_dev *indio_dev = spi_get_drvdata(spi);
diff --git a/drivers/staging/iio/meter/ade7758.h b/drivers/staging/iio/meter/ade7758.h
index f6739e2..1d04ec9 100644
--- a/drivers/staging/iio/meter/ade7758.h
+++ b/drivers/staging/iio/meter/ade7758.h
@@ -129,6 +129,7 @@
unsigned char tx_buf[8];
};
+
#ifdef CONFIG_IIO_BUFFER
/* At the moment triggers are only used for ring buffer
* filling. This may change!
@@ -138,25 +139,22 @@
int ade7758_probe_trigger(struct iio_dev *indio_dev);
ssize_t ade7758_read_data_from_ring(struct device *dev,
- struct device_attribute *attr,
- char *buf);
-
+ struct device_attribute *attr, char *buf);
int ade7758_configure_ring(struct iio_dev *indio_dev);
void ade7758_unconfigure_ring(struct iio_dev *indio_dev);
int ade7758_set_irq(struct device *dev, bool enable);
-int ade7758_spi_write_reg_8(struct device *dev,
- u8 reg_address, u8 val);
-int ade7758_spi_read_reg_8(struct device *dev,
- u8 reg_address, u8 *val);
+int ade7758_spi_write_reg_8(struct device *dev, u8 reg_address, u8 val);
+int ade7758_spi_read_reg_8(struct device *dev, u8 reg_address, u8 *val);
#else /* CONFIG_IIO_BUFFER */
static inline void ade7758_remove_trigger(struct iio_dev *indio_dev)
{
}
+
static inline int ade7758_probe_trigger(struct iio_dev *indio_dev)
{
return 0;
@@ -166,16 +164,20 @@
{
return 0;
}
+
static inline void ade7758_unconfigure_ring(struct iio_dev *indio_dev)
{
}
+
static inline int ade7758_initialize_ring(struct iio_ring_buffer *ring)
{
return 0;
}
+
static inline void ade7758_uninitialize_ring(struct iio_dev *indio_dev)
{
}
+
#endif /* CONFIG_IIO_BUFFER */
#endif
diff --git a/drivers/staging/iio/meter/ade7758_core.c b/drivers/staging/iio/meter/ade7758_core.c
index 40f5afa..ebb8a19 100644
--- a/drivers/staging/iio/meter/ade7758_core.c
+++ b/drivers/staging/iio/meter/ade7758_core.c
@@ -24,9 +24,7 @@
#include "meter.h"
#include "ade7758.h"
-int ade7758_spi_write_reg_8(struct device *dev,
- u8 reg_address,
- u8 val)
+int ade7758_spi_write_reg_8(struct device *dev, u8 reg_address, u8 val)
{
int ret;
struct iio_dev *indio_dev = dev_to_iio_dev(dev);
@@ -42,9 +40,8 @@
return ret;
}
-static int ade7758_spi_write_reg_16(struct device *dev,
- u8 reg_address,
- u16 value)
+static int ade7758_spi_write_reg_16(struct device *dev, u8 reg_address,
+ u16 value)
{
int ret;
struct iio_dev *indio_dev = dev_to_iio_dev(dev);
@@ -68,9 +65,8 @@
return ret;
}
-static int ade7758_spi_write_reg_24(struct device *dev,
- u8 reg_address,
- u32 value)
+static int ade7758_spi_write_reg_24(struct device *dev, u8 reg_address,
+ u32 value)
{
int ret;
struct iio_dev *indio_dev = dev_to_iio_dev(dev);
@@ -95,9 +91,7 @@
return ret;
}
-int ade7758_spi_read_reg_8(struct device *dev,
- u8 reg_address,
- u8 *val)
+int ade7758_spi_read_reg_8(struct device *dev, u8 reg_address, u8 *val)
{
struct iio_dev *indio_dev = dev_to_iio_dev(dev);
struct ade7758_state *st = iio_priv(indio_dev);
@@ -124,7 +118,7 @@
ret = spi_sync_transfer(st->us, xfers, ARRAY_SIZE(xfers));
if (ret) {
dev_err(&st->us->dev, "problem when reading 8 bit register 0x%02X",
- reg_address);
+ reg_address);
goto error_ret;
}
*val = st->rx[0];
@@ -134,9 +128,8 @@
return ret;
}
-static int ade7758_spi_read_reg_16(struct device *dev,
- u8 reg_address,
- u16 *val)
+static int ade7758_spi_read_reg_16(struct device *dev, u8 reg_address,
+ u16 *val)
{
struct iio_dev *indio_dev = dev_to_iio_dev(dev);
struct ade7758_state *st = iio_priv(indio_dev);
@@ -156,7 +149,6 @@
},
};
-
mutex_lock(&st->buf_lock);
st->tx[0] = ADE7758_READ_REG(reg_address);
st->tx[1] = 0;
@@ -165,7 +157,7 @@
ret = spi_sync_transfer(st->us, xfers, ARRAY_SIZE(xfers));
if (ret) {
dev_err(&st->us->dev, "problem when reading 16 bit register 0x%02X",
- reg_address);
+ reg_address);
goto error_ret;
}
@@ -176,9 +168,8 @@
return ret;
}
-static int ade7758_spi_read_reg_24(struct device *dev,
- u8 reg_address,
- u32 *val)
+static int ade7758_spi_read_reg_24(struct device *dev, u8 reg_address,
+ u32 *val)
{
struct iio_dev *indio_dev = dev_to_iio_dev(dev);
struct ade7758_state *st = iio_priv(indio_dev);
@@ -207,7 +198,7 @@
ret = spi_sync_transfer(st->us, xfers, ARRAY_SIZE(xfers));
if (ret) {
dev_err(&st->us->dev, "problem when reading 24 bit register 0x%02X",
- reg_address);
+ reg_address);
goto error_ret;
}
*val = (st->rx[0] << 16) | (st->rx[1] << 8) | st->rx[2];
@@ -218,8 +209,7 @@
}
static ssize_t ade7758_read_8bit(struct device *dev,
- struct device_attribute *attr,
- char *buf)
+ struct device_attribute *attr, char *buf)
{
int ret;
u8 val = 0;
@@ -233,8 +223,7 @@
}
static ssize_t ade7758_read_16bit(struct device *dev,
- struct device_attribute *attr,
- char *buf)
+ struct device_attribute *attr, char *buf)
{
int ret;
u16 val = 0;
@@ -248,8 +237,7 @@
}
static ssize_t ade7758_read_24bit(struct device *dev,
- struct device_attribute *attr,
- char *buf)
+ struct device_attribute *attr, char *buf)
{
int ret;
u32 val = 0;
@@ -263,9 +251,8 @@
}
static ssize_t ade7758_write_8bit(struct device *dev,
- struct device_attribute *attr,
- const char *buf,
- size_t len)
+ struct device_attribute *attr,
+ const char *buf, size_t len)
{
struct iio_dev_attr *this_attr = to_iio_dev_attr(attr);
int ret;
@@ -281,9 +268,8 @@
}
static ssize_t ade7758_write_16bit(struct device *dev,
- struct device_attribute *attr,
- const char *buf,
- size_t len)
+ struct device_attribute *attr,
+ const char *buf, size_t len)
{
struct iio_dev_attr *this_attr = to_iio_dev_attr(attr);
int ret;
@@ -427,7 +413,8 @@
if (enable)
irqen |= BIT(16); /* Enables an interrupt when a data is
- present in the waveform register */
+ * present in the waveform register
+ */
else
irqen &= ~BIT(16);
@@ -479,16 +466,13 @@
}
static ssize_t ade7758_read_frequency(struct device *dev,
- struct device_attribute *attr,
- char *buf)
+ struct device_attribute *attr, char *buf)
{
int ret;
u8 t;
int sps;
- ret = ade7758_spi_read_reg_8(dev,
- ADE7758_WAVMODE,
- &t);
+ ret = ade7758_spi_read_reg_8(dev, ADE7758_WAVMODE, &t);
if (ret)
return ret;
@@ -499,9 +483,8 @@
}
static ssize_t ade7758_write_frequency(struct device *dev,
- struct device_attribute *attr,
- const char *buf,
- size_t len)
+ struct device_attribute *attr,
+ const char *buf, size_t len)
{
struct iio_dev *indio_dev = dev_to_iio_dev(dev);
u16 val;
@@ -532,18 +515,14 @@
goto out;
}
- ret = ade7758_spi_read_reg_8(dev,
- ADE7758_WAVMODE,
- ®);
+ ret = ade7758_spi_read_reg_8(dev, ADE7758_WAVMODE, ®);
if (ret)
goto out;
reg &= ~(5 << 3);
reg |= t << 5;
- ret = ade7758_spi_write_reg_8(dev,
- ADE7758_WAVMODE,
- reg);
+ ret = ade7758_spi_write_reg_8(dev, ADE7758_WAVMODE, reg);
out:
mutex_unlock(&indio_dev->mlock);
diff --git a/drivers/staging/iio/meter/ade7758_ring.c b/drivers/staging/iio/meter/ade7758_ring.c
index 9a24e02..a6b76d4 100644
--- a/drivers/staging/iio/meter/ade7758_ring.c
+++ b/drivers/staging/iio/meter/ade7758_ring.c
@@ -33,7 +33,7 @@
return ret;
}
-static int ade7758_write_waveform_type(struct device *dev, unsigned type)
+static int ade7758_write_waveform_type(struct device *dev, unsigned int type)
{
int ret;
u8 reg;
@@ -85,7 +85,7 @@
**/
static int ade7758_ring_preenable(struct iio_dev *indio_dev)
{
- unsigned channel;
+ unsigned int channel;
if (bitmap_empty(indio_dev->active_scan_mask, indio_dev->masklength))
return -EINVAL;
diff --git a/drivers/staging/iio/meter/ade7759.c b/drivers/staging/iio/meter/ade7759.c
index 684e612..80144d4 100644
--- a/drivers/staging/iio/meter/ade7759.c
+++ b/drivers/staging/iio/meter/ade7759.c
@@ -289,7 +289,8 @@
if (enable)
irqen |= BIT(3); /* Enables an interrupt when a data is
- present in the waveform register */
+ * present in the waveform register
+ */
else
irqen &= ~BIT(3);
@@ -476,7 +477,6 @@
return iio_device_register(indio_dev);
}
-/* fixme, confirm ordering in this function */
static int ade7759_remove(struct spi_device *spi)
{
struct iio_dev *indio_dev = spi_get_drvdata(spi);
diff --git a/drivers/staging/iio/meter/ade7854.c b/drivers/staging/iio/meter/ade7854.c
index 9e439af..75e8685 100644
--- a/drivers/staging/iio/meter/ade7854.c
+++ b/drivers/staging/iio/meter/ade7854.c
@@ -421,7 +421,8 @@
if (enable)
irqen |= BIT(17); /* 1: interrupt enabled when all periodical
- (at 8 kHz rate) DSP computations finish. */
+ * (at 8 kHz rate) DSP computations finish.
+ */
else
irqen &= ~BIT(17);
diff --git a/drivers/staging/iio/resolver/ad2s1210.h b/drivers/staging/iio/resolver/ad2s1210.h
index c7158f6..e9b2147 100644
--- a/drivers/staging/iio/resolver/ad2s1210.h
+++ b/drivers/staging/iio/resolver/ad2s1210.h
@@ -12,9 +12,9 @@
#define _AD2S1210_H
struct ad2s1210_platform_data {
- unsigned sample;
- unsigned a[2];
- unsigned res[2];
- bool gpioin;
+ unsigned int sample;
+ unsigned int a[2];
+ unsigned int res[2];
+ bool gpioin;
};
#endif /* _AD2S1210_H */
diff --git a/drivers/staging/iio/trigger/iio-trig-bfin-timer.c b/drivers/staging/iio/trigger/iio-trig-bfin-timer.c
index 035dd45..38dca69 100644
--- a/drivers/staging/iio/trigger/iio-trig-bfin-timer.c
+++ b/drivers/staging/iio/trigger/iio-trig-bfin-timer.c
@@ -55,12 +55,12 @@
};
struct bfin_tmr_state {
- struct iio_trigger *trig;
- struct bfin_timer *t;
- unsigned timer_num;
- bool output_enable;
- unsigned int duty;
- int irq;
+ struct iio_trigger *trig;
+ struct bfin_timer *t;
+ unsigned int timer_num;
+ bool output_enable;
+ unsigned int duty;
+ int irq;
};
static int iio_bfin_tmr_set_state(struct iio_trigger *trig, bool state)
@@ -178,7 +178,7 @@
static int iio_bfin_tmr_trigger_probe(struct platform_device *pdev)
{
- struct iio_bfin_timer_trigger_pdata *pdata = pdev->dev.platform_data;
+ struct iio_bfin_timer_trigger_pdata *pdata;
struct bfin_tmr_state *st;
unsigned int config;
int ret;
@@ -221,6 +221,7 @@
config = PWM_OUT | PERIOD_CNT | IRQ_ENA;
+ pdata = dev_get_platdata(&pdev->dev);
if (pdata && pdata->output_enable) {
unsigned long long val;
diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs.h b/drivers/staging/lustre/include/linux/libcfs/libcfs.h
index 40af75c..4141afb 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs.h
@@ -60,41 +60,12 @@
#define LNET_ACCEPTOR_MAX_RESERVED_PORT 1023
/*
- * libcfs pseudo device operations
- *
- * It's just draft now.
- */
-
-struct cfs_psdev_file {
- unsigned long off;
- void *private_data;
- unsigned long reserved1;
- unsigned long reserved2;
-};
-
-struct cfs_psdev_ops {
- int (*p_open)(unsigned long, void *);
- int (*p_close)(unsigned long, void *);
- int (*p_read)(struct cfs_psdev_file *, char *, unsigned long);
- int (*p_write)(struct cfs_psdev_file *, char *, unsigned long);
- int (*p_ioctl)(struct cfs_psdev_file *, unsigned long, void __user *);
-};
-
-/*
- * Drop into debugger, if possible. Implementation is provided by platform.
- */
-
-void cfs_enter_debugger(void);
-
-/*
* Defined by platform
*/
-int unshare_fs_struct(void);
sigset_t cfs_block_allsigs(void);
sigset_t cfs_block_sigs(unsigned long sigs);
sigset_t cfs_block_sigsinv(unsigned long sigs);
void cfs_restore_sigs(sigset_t);
-int cfs_signal_pending(void);
void cfs_clear_sigpending(void);
/*
@@ -117,7 +88,25 @@
#include "libcfs_workitem.h"
#include "libcfs_hash.h"
#include "libcfs_fail.h"
-#include "libcfs_crypto.h"
+
+struct libcfs_ioctl_handler {
+ struct list_head item;
+ int (*handle_ioctl)(unsigned int cmd, struct libcfs_ioctl_hdr *hdr);
+};
+
+#define DECLARE_IOCTL_HANDLER(ident, func) \
+ struct libcfs_ioctl_handler ident = { \
+ .item = LIST_HEAD_INIT(ident.item), \
+ .handle_ioctl = func \
+ }
+
+int libcfs_register_ioctl(struct libcfs_ioctl_handler *hand);
+int libcfs_deregister_ioctl(struct libcfs_ioctl_handler *hand);
+
+int libcfs_ioctl_getdata(struct libcfs_ioctl_hdr **hdr_pp,
+ const struct libcfs_ioctl_hdr __user *uparam);
+int libcfs_ioctl_data_adjust(struct libcfs_ioctl_data *data);
+int libcfs_ioctl(unsigned long cmd, void __user *arg);
/* container_of depends on "likely" which is defined in libcfs_private.h */
static inline void *__container_of(void *ptr, unsigned long shift)
@@ -143,8 +132,6 @@
extern char lnet_upcall[1024];
extern char lnet_debug_log_upcall[1024];
-extern struct cfs_psdev_ops libcfs_psdev_ops;
-
extern struct cfs_wi_sched *cfs_sched_rehash;
struct lnet_debugfs_symlink_def {
diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
index 9e62c59..81d8079 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h
@@ -203,6 +203,85 @@
*/
int cfs_cpu_ht_nsiblings(int cpu);
+/*
+ * allocate per-cpu-partition data, returned value is an array of pointers,
+ * variable can be indexed by CPU ID.
+ * cptab != NULL: size of array is number of CPU partitions
+ * cptab == NULL: size of array is number of HW cores
+ */
+void *cfs_percpt_alloc(struct cfs_cpt_table *cptab, unsigned int size);
+/*
+ * destory per-cpu-partition variable
+ */
+void cfs_percpt_free(void *vars);
+int cfs_percpt_number(void *vars);
+
+#define cfs_percpt_for_each(var, i, vars) \
+ for (i = 0; i < cfs_percpt_number(vars) && \
+ ((var) = (vars)[i]) != NULL; i++)
+
+/*
+ * percpu partition lock
+ *
+ * There are some use-cases like this in Lustre:
+ * . each CPU partition has it's own private data which is frequently changed,
+ * and mostly by the local CPU partition.
+ * . all CPU partitions share some global data, these data are rarely changed.
+ *
+ * LNet is typical example.
+ * CPU partition lock is designed for this kind of use-cases:
+ * . each CPU partition has it's own private lock
+ * . change on private data just needs to take the private lock
+ * . read on shared data just needs to take _any_ of private locks
+ * . change on shared data needs to take _all_ private locks,
+ * which is slow and should be really rare.
+ */
+enum {
+ CFS_PERCPT_LOCK_EX = -1, /* negative */
+};
+
+struct cfs_percpt_lock {
+ /* cpu-partition-table for this lock */
+ struct cfs_cpt_table *pcl_cptab;
+ /* exclusively locked */
+ unsigned int pcl_locked;
+ /* private lock table */
+ spinlock_t **pcl_locks;
+};
+
+/* return number of private locks */
+#define cfs_percpt_lock_num(pcl) cfs_cpt_number(pcl->pcl_cptab)
+
+/*
+ * create a cpu-partition lock based on CPU partition table \a cptab,
+ * each private lock has extra \a psize bytes padding data
+ */
+struct cfs_percpt_lock *cfs_percpt_lock_create(struct cfs_cpt_table *cptab,
+ struct lock_class_key *keys);
+/* destroy a cpu-partition lock */
+void cfs_percpt_lock_free(struct cfs_percpt_lock *pcl);
+
+/* lock private lock \a index of \a pcl */
+void cfs_percpt_lock(struct cfs_percpt_lock *pcl, int index);
+
+/* unlock private lock \a index of \a pcl */
+void cfs_percpt_unlock(struct cfs_percpt_lock *pcl, int index);
+
+#define CFS_PERCPT_LOCK_KEYS 256
+
+/* NB: don't allocate keys dynamically, lockdep needs them to be in ".data" */
+#define cfs_percpt_lock_alloc(cptab) \
+({ \
+ static struct lock_class_key ___keys[CFS_PERCPT_LOCK_KEYS]; \
+ struct cfs_percpt_lock *___lk; \
+ \
+ if (cfs_cpt_number(cptab) > CFS_PERCPT_LOCK_KEYS) \
+ ___lk = cfs_percpt_lock_create(cptab, NULL); \
+ else \
+ ___lk = cfs_percpt_lock_create(cptab, ___keys); \
+ ___lk; \
+})
+
/**
* iterate over all CPU partitions in \a cptab
*/
diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h
index e866369..02be7d7 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h
@@ -46,7 +46,8 @@
CFS_HASH_ALG_SHA384,
CFS_HASH_ALG_SHA512,
CFS_HASH_ALG_CRC32C,
- CFS_HASH_ALG_MAX
+ CFS_HASH_ALG_MAX,
+ CFS_HASH_ALG_UNKNOWN = 0xff
};
static struct cfs_crypto_hash_type hash_types[] = {
@@ -59,11 +60,22 @@
[CFS_HASH_ALG_SHA256] = { "sha256", 0, 32 },
[CFS_HASH_ALG_SHA384] = { "sha384", 0, 48 },
[CFS_HASH_ALG_SHA512] = { "sha512", 0, 64 },
+ [CFS_HASH_ALG_MAX] = { NULL, 0, 64 },
};
-/** Return pointer to type of hash for valid hash algorithm identifier */
+/* Maximum size of hash_types[].cht_size */
+#define CFS_CRYPTO_HASH_DIGESTSIZE_MAX 64
+
+/**
+ * Return hash algorithm information for the specified algorithm identifier
+ *
+ * Hash information includes algorithm name, initial seed, hash size.
+ *
+ * \retval cfs_crypto_hash_type for valid ID (CFS_HASH_ALG_*)
+ * \retval NULL for unknown algorithm identifier
+ */
static inline const struct cfs_crypto_hash_type *
- cfs_crypto_hash_type(unsigned char hash_alg)
+cfs_crypto_hash_type(enum cfs_crypto_hash_alg hash_alg)
{
struct cfs_crypto_hash_type *ht;
@@ -75,8 +87,16 @@
return NULL;
}
-/** Return hash name for valid hash algorithm identifier or "unknown" */
-static inline const char *cfs_crypto_hash_name(unsigned char hash_alg)
+/**
+ * Return hash name for hash algorithm identifier
+ *
+ * \param[in] hash_alg hash alrgorithm id (CFS_HASH_ALG_*)
+ *
+ * \retval string name of known hash algorithm
+ * \retval "unknown" if hash algorithm is unknown
+ */
+static inline const char *
+cfs_crypto_hash_name(enum cfs_crypto_hash_alg hash_alg)
{
const struct cfs_crypto_hash_type *ht;
@@ -86,8 +106,15 @@
return "unknown";
}
-/** Return digest size for valid algorithm identifier or 0 */
-static inline int cfs_crypto_hash_digestsize(unsigned char hash_alg)
+/**
+ * Return digest size for hash algorithm type
+ *
+ * \param[in] hash_alg hash alrgorithm id (CFS_HASH_ALG_*)
+ *
+ * \retval hash algorithm digest size in bytes
+ * \retval 0 if hash algorithm type is unknown
+ */
+static inline int cfs_crypto_hash_digestsize(enum cfs_crypto_hash_alg hash_alg)
{
const struct cfs_crypto_hash_type *ht;
@@ -97,36 +124,24 @@
return 0;
}
-/** Return hash identifier for valid hash algorithm name or 0xFF */
+/**
+ * Find hash algorithm ID for the specified algorithm name
+ *
+ * \retval hash algorithm ID for valid ID (CFS_HASH_ALG_*)
+ * \retval CFS_HASH_ALG_UNKNOWN for unknown algorithm name
+ */
static inline unsigned char cfs_crypto_hash_alg(const char *algname)
{
- unsigned char i;
+ enum cfs_crypto_hash_alg hash_alg;
- for (i = 0; i < CFS_HASH_ALG_MAX; i++)
- if (!strcmp(hash_types[i].cht_name, algname))
- break;
- return (i == CFS_HASH_ALG_MAX ? 0xFF : i);
+ for (hash_alg = 0; hash_alg < CFS_HASH_ALG_MAX; hash_alg++)
+ if (strcmp(hash_types[hash_alg].cht_name, algname) == 0)
+ return hash_alg;
+
+ return CFS_HASH_ALG_UNKNOWN;
}
-/** Calculate hash digest for buffer.
- * @param alg id of hash algorithm
- * @param buf buffer of data
- * @param buf_len buffer len
- * @param key initial value for algorithm, if it is NULL,
- * default initial value should be used.
- * @param key_len len of initial value
- * @param hash [out] pointer to hash, if it is NULL, hash_len is
- * set to valid digest size in bytes, retval -ENOSPC.
- * @param hash_len [in,out] size of hash buffer
- * @returns status of operation
- * @retval -EINVAL if buf, buf_len, hash_len or alg_id is invalid
- * @retval -ENODEV if this algorithm is unsupported
- * @retval -ENOSPC if pointer to hash is NULL, or hash_len less than
- * digest size
- * @retval 0 for success
- * @retval < 0 other errors from lower layers.
- */
-int cfs_crypto_hash_digest(unsigned char alg,
+int cfs_crypto_hash_digest(enum cfs_crypto_hash_alg hash_alg,
const void *buf, unsigned int buf_len,
unsigned char *key, unsigned int key_len,
unsigned char *hash, unsigned int *hash_len);
@@ -134,66 +149,17 @@
/* cfs crypto hash descriptor */
struct cfs_crypto_hash_desc;
-/** Allocate and initialize descriptor for hash algorithm.
- * @param alg algorithm id
- * @param key initial value for algorithm, if it is NULL,
- * default initial value should be used.
- * @param key_len len of initial value
- * @returns pointer to descriptor of hash instance
- * @retval ERR_PTR(error) when errors occurred.
- */
-struct cfs_crypto_hash_desc*
- cfs_crypto_hash_init(unsigned char alg,
- unsigned char *key, unsigned int key_len);
-
-/** Update digest by part of data.
- * @param desc hash descriptor
- * @param page data page
- * @param offset data offset
- * @param len data len
- * @returns status of operation
- * @retval 0 for success.
- */
+struct cfs_crypto_hash_desc *
+cfs_crypto_hash_init(enum cfs_crypto_hash_alg hash_alg,
+ unsigned char *key, unsigned int key_len);
int cfs_crypto_hash_update_page(struct cfs_crypto_hash_desc *desc,
struct page *page, unsigned int offset,
unsigned int len);
-
-/** Update digest by part of data.
- * @param desc hash descriptor
- * @param buf pointer to data buffer
- * @param buf_len size of data at buffer
- * @returns status of operation
- * @retval 0 for success.
- */
int cfs_crypto_hash_update(struct cfs_crypto_hash_desc *desc, const void *buf,
unsigned int buf_len);
-
-/** Finalize hash calculation, copy hash digest to buffer, destroy hash
- * descriptor.
- * @param desc hash descriptor
- * @param hash buffer pointer to store hash digest
- * @param hash_len pointer to hash buffer size, if NULL
- * destroy hash descriptor
- * @returns status of operation
- * @retval -ENOSPC if hash is NULL, or *hash_len less than
- * digest size
- * @retval 0 for success
- * @retval < 0 other errors from lower layers.
- */
int cfs_crypto_hash_final(struct cfs_crypto_hash_desc *desc,
unsigned char *hash, unsigned int *hash_len);
-/**
- * Register crypto hash algorithms
- */
int cfs_crypto_register(void);
-
-/**
- * Unregister
- */
void cfs_crypto_unregister(void);
-
-/** Return hash speed in Mbytes per second for valid hash algorithm
- * identifier. If test was unsuccessful -1 would be returned.
- */
-int cfs_crypto_hash_speed(unsigned char hash_alg);
+int cfs_crypto_hash_speed(enum cfs_crypto_hash_alg hash_alg);
#endif
diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_debug.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_debug.h
index 98430e7..455c54d 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs_debug.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_debug.h
@@ -85,7 +85,6 @@
#define PH_FLAG_FIRST_RECORD 1
/* Debugging subsystems (32 bits, non-overlapping) */
-/* keep these in sync with lnet/utils/debug.c and lnet/libcfs/debug.c */
#define S_UNDEFINED 0x00000001
#define S_MDC 0x00000002
#define S_MDS 0x00000004
@@ -118,10 +117,14 @@
#define S_MGS 0x20000000
#define S_FID 0x40000000 /* b_new_cmd */
#define S_FLD 0x80000000 /* b_new_cmd */
-/* keep these in sync with lnet/utils/debug.c and lnet/libcfs/debug.c */
+
+#define LIBCFS_DEBUG_SUBSYS_NAMES { \
+ "undefined", "mdc", "mds", "osc", "ost", "class", "log", \
+ "llite", "rpc", "mgmt", "lnet", "lnd", "pinger", "filter", "", \
+ "echo", "ldlm", "lov", "lquota", "osd", "lfsck", "", "", "lmv", \
+ "", "sec", "gss", "", "mgc", "mgs", "fid", "fld", NULL }
/* Debugging masks (32 bits, non-overlapping) */
-/* keep these in sync with lnet/utils/debug.c and lnet/libcfs/debug.c */
#define D_TRACE 0x00000001 /* ENTRY/EXIT markers */
#define D_INODE 0x00000002
#define D_SUPER 0x00000004
@@ -151,9 +154,14 @@
#define D_QUOTA 0x04000000
#define D_SEC 0x08000000
#define D_LFSCK 0x10000000 /* For both OI scrub and LFSCK */
-/* keep these in sync with lnet/{utils,libcfs}/debug.c */
+#define D_HSM 0x20000000
-#define D_HSM D_TRACE
+#define LIBCFS_DEBUG_MASKS_NAMES { \
+ "trace", "inode", "super", "ext2", "malloc", "cache", "info", \
+ "ioctl", "neterror", "net", "warning", "buffs", "other", \
+ "dentry", "nettrace", "page", "dlmtrace", "error", "emerg", \
+ "ha", "rpctrace", "vfstrace", "reada", "mmap", "config", \
+ "console", "quota", "sec", "lfsck", "hsm", NULL }
#define D_CANTMASK (D_ERROR | D_EMERG | D_WARNING | D_CONSOLE)
diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h
index aa69c6a..2e008bf 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h
@@ -38,6 +38,7 @@
extern unsigned long cfs_fail_loc;
extern unsigned int cfs_fail_val;
+extern int cfs_fail_err;
extern wait_queue_head_t cfs_race_waitq;
extern int cfs_race_state;
@@ -70,9 +71,14 @@
#define CFS_FAIL_RAND 0x08000000 /* fail 1/N of the times */
#define CFS_FAIL_USR1 0x04000000 /* user flag */
-#define CFS_FAIL_PRECHECK(id) (cfs_fail_loc && \
- (cfs_fail_loc & CFS_FAIL_MASK_LOC) == \
- ((id) & CFS_FAIL_MASK_LOC))
+#define CFS_FAULT 0x02000000 /* match any CFS_FAULT_CHECK */
+
+static inline bool CFS_FAIL_PRECHECK(__u32 id)
+{
+ return cfs_fail_loc != 0 &&
+ ((cfs_fail_loc & CFS_FAIL_MASK_LOC) == (id & CFS_FAIL_MASK_LOC) ||
+ (cfs_fail_loc & id & CFS_FAULT));
+}
static inline int cfs_fail_check_set(__u32 id, __u32 value,
int set, int quiet)
@@ -144,6 +150,9 @@
#define CFS_FAIL_TIMEOUT_MS_ORSET(id, value, ms) \
cfs_fail_timeout_set(id, value, ms, CFS_FAIL_LOC_ORSET)
+#define CFS_FAULT_CHECK(id) \
+ CFS_FAIL_CHECK(CFS_FAULT | (id))
+
/* The idea here is to synchronise two threads to force a race. The
* first thread that calls this with a matching fail_loc is put to
* sleep. The next thread that calls with the same fail_loc wakes up
diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h
index c3f2332..119986b 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h
@@ -245,7 +245,7 @@
/** # of iterators (caller of cfs_hash_for_each_*) */
__u32 hs_iterators;
/** rehash workitem */
- cfs_workitem_t hs_rehash_wi;
+ struct cfs_workitem hs_rehash_wi;
/** refcount on this hash table */
atomic_t hs_refcount;
/** rehash buckets-table */
@@ -262,7 +262,7 @@
/** bits when we found the max depth */
unsigned int hs_dep_bits;
/** workitem to output max depth */
- cfs_workitem_t hs_dep_wi;
+ struct cfs_workitem hs_dep_wi;
#endif
/** name of htable */
char hs_name[0];
diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_ioctl.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_ioctl.h
index 5ca99bd..4b9102b 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs_ioctl.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_ioctl.h
@@ -34,13 +34,16 @@
* libcfs/include/libcfs/libcfs_ioctl.h
*
* Low-level ioctl data structures. Kernel ioctl functions declared here,
- * and user space functions are in libcfsutil_ioctl.h.
+ * and user space functions are in libcfs/util/ioctl.h.
*
*/
#ifndef __LIBCFS_IOCTL_H__
#define __LIBCFS_IOCTL_H__
+#include <linux/types.h>
+#include <linux/ioctl.h>
+
#define LIBCFS_IOCTL_VERSION 0x0001000a
#define LIBCFS_IOCTL_VERSION2 0x0001000b
@@ -49,6 +52,9 @@
__u32 ioc_version;
};
+/** max size to copy from userspace */
+#define LIBCFS_IOC_DATA_MAX (128 * 1024)
+
struct libcfs_ioctl_data {
struct libcfs_ioctl_hdr ioc_hdr;
@@ -73,67 +79,48 @@
char ioc_bulk[0];
};
-#define ioc_priority ioc_u32[0]
-
struct libcfs_debug_ioctl_data {
struct libcfs_ioctl_hdr hdr;
unsigned int subs;
unsigned int debug;
};
-#define LIBCFS_IOC_INIT(data) \
-do { \
- memset(&data, 0, sizeof(data)); \
- data.ioc_version = LIBCFS_IOCTL_VERSION; \
- data.ioc_len = sizeof(data); \
-} while (0)
+/* 'f' ioctls are defined in lustre_ioctl.h and lustre_user.h except for: */
+#define LIBCFS_IOC_DEBUG_MASK _IOWR('f', 250, long)
+#define IOCTL_LIBCFS_TYPE long
-struct libcfs_ioctl_handler {
- struct list_head item;
- int (*handle_ioctl)(unsigned int cmd, struct libcfs_ioctl_hdr *hdr);
-};
-
-#define DECLARE_IOCTL_HANDLER(ident, func) \
- struct libcfs_ioctl_handler ident = { \
- /* .item = */ LIST_HEAD_INIT(ident.item), \
- /* .handle_ioctl = */ func \
- }
-
-/* FIXME check conflict with lustre_lib.h */
-#define LIBCFS_IOC_DEBUG_MASK _IOWR('f', 250, long)
-
-#define IOC_LIBCFS_TYPE 'e'
-#define IOC_LIBCFS_MIN_NR 30
+#define IOC_LIBCFS_TYPE ('e')
+#define IOC_LIBCFS_MIN_NR 30
/* libcfs ioctls */
-#define IOC_LIBCFS_PANIC _IOWR('e', 30, long)
-#define IOC_LIBCFS_CLEAR_DEBUG _IOWR('e', 31, long)
-#define IOC_LIBCFS_MARK_DEBUG _IOWR('e', 32, long)
-#define IOC_LIBCFS_MEMHOG _IOWR('e', 36, long)
+/* IOC_LIBCFS_PANIC obsolete in 2.8.0, was _IOWR('e', 30, IOCTL_LIBCFS_TYPE) */
+#define IOC_LIBCFS_CLEAR_DEBUG _IOWR('e', 31, IOCTL_LIBCFS_TYPE)
+#define IOC_LIBCFS_MARK_DEBUG _IOWR('e', 32, IOCTL_LIBCFS_TYPE)
+/* IOC_LIBCFS_MEMHOG obsolete in 2.8.0, was _IOWR('e', 36, IOCTL_LIBCFS_TYPE) */
/* lnet ioctls */
-#define IOC_LIBCFS_GET_NI _IOWR('e', 50, long)
-#define IOC_LIBCFS_FAIL_NID _IOWR('e', 51, long)
-#define IOC_LIBCFS_NOTIFY_ROUTER _IOWR('e', 55, long)
-#define IOC_LIBCFS_UNCONFIGURE _IOWR('e', 56, long)
-/* #define IOC_LIBCFS_PORTALS_COMPATIBILITY _IOWR('e', 57, long) */
-#define IOC_LIBCFS_LNET_DIST _IOWR('e', 58, long)
-#define IOC_LIBCFS_CONFIGURE _IOWR('e', 59, long)
-#define IOC_LIBCFS_TESTPROTOCOMPAT _IOWR('e', 60, long)
-#define IOC_LIBCFS_PING _IOWR('e', 61, long)
-/* #define IOC_LIBCFS_DEBUG_PEER _IOWR('e', 62, long) */
-#define IOC_LIBCFS_LNETST _IOWR('e', 63, long)
-#define IOC_LIBCFS_LNET_FAULT _IOWR('e', 64, long)
+#define IOC_LIBCFS_GET_NI _IOWR('e', 50, IOCTL_LIBCFS_TYPE)
+#define IOC_LIBCFS_FAIL_NID _IOWR('e', 51, IOCTL_LIBCFS_TYPE)
+#define IOC_LIBCFS_NOTIFY_ROUTER _IOWR('e', 55, IOCTL_LIBCFS_TYPE)
+#define IOC_LIBCFS_UNCONFIGURE _IOWR('e', 56, IOCTL_LIBCFS_TYPE)
+/* IOC_LIBCFS_PORTALS_COMPATIBILITY _IOWR('e', 57, IOCTL_LIBCFS_TYPE) */
+#define IOC_LIBCFS_LNET_DIST _IOWR('e', 58, IOCTL_LIBCFS_TYPE)
+#define IOC_LIBCFS_CONFIGURE _IOWR('e', 59, IOCTL_LIBCFS_TYPE)
+#define IOC_LIBCFS_TESTPROTOCOMPAT _IOWR('e', 60, IOCTL_LIBCFS_TYPE)
+#define IOC_LIBCFS_PING _IOWR('e', 61, IOCTL_LIBCFS_TYPE)
+/* IOC_LIBCFS_DEBUG_PEER _IOWR('e', 62, IOCTL_LIBCFS_TYPE) */
+#define IOC_LIBCFS_LNETST _IOWR('e', 63, IOCTL_LIBCFS_TYPE)
+#define IOC_LIBCFS_LNET_FAULT _IOWR('e', 64, IOCTL_LIBCFS_TYPE)
/* lnd ioctls */
-#define IOC_LIBCFS_REGISTER_MYNID _IOWR('e', 70, long)
-#define IOC_LIBCFS_CLOSE_CONNECTION _IOWR('e', 71, long)
-#define IOC_LIBCFS_PUSH_CONNECTION _IOWR('e', 72, long)
-#define IOC_LIBCFS_GET_CONN _IOWR('e', 73, long)
-#define IOC_LIBCFS_DEL_PEER _IOWR('e', 74, long)
-#define IOC_LIBCFS_ADD_PEER _IOWR('e', 75, long)
-#define IOC_LIBCFS_GET_PEER _IOWR('e', 76, long)
+#define IOC_LIBCFS_REGISTER_MYNID _IOWR('e', 70, IOCTL_LIBCFS_TYPE)
+#define IOC_LIBCFS_CLOSE_CONNECTION _IOWR('e', 71, IOCTL_LIBCFS_TYPE)
+#define IOC_LIBCFS_PUSH_CONNECTION _IOWR('e', 72, IOCTL_LIBCFS_TYPE)
+#define IOC_LIBCFS_GET_CONN _IOWR('e', 73, IOCTL_LIBCFS_TYPE)
+#define IOC_LIBCFS_DEL_PEER _IOWR('e', 74, IOCTL_LIBCFS_TYPE)
+#define IOC_LIBCFS_ADD_PEER _IOWR('e', 75, IOCTL_LIBCFS_TYPE)
+#define IOC_LIBCFS_GET_PEER _IOWR('e', 76, IOCTL_LIBCFS_TYPE)
/* ioctl 77 is free for use */
-#define IOC_LIBCFS_ADD_INTERFACE _IOWR('e', 78, long)
-#define IOC_LIBCFS_DEL_INTERFACE _IOWR('e', 79, long)
-#define IOC_LIBCFS_GET_INTERFACE _IOWR('e', 80, long)
+#define IOC_LIBCFS_ADD_INTERFACE _IOWR('e', 78, IOCTL_LIBCFS_TYPE)
+#define IOC_LIBCFS_DEL_INTERFACE _IOWR('e', 79, IOCTL_LIBCFS_TYPE)
+#define IOC_LIBCFS_GET_INTERFACE _IOWR('e', 80, IOCTL_LIBCFS_TYPE)
/*
* DLC Specific IOCTL numbers.
@@ -155,76 +142,4 @@
#define IOC_LIBCFS_GET_LNET_STATS _IOWR(IOC_LIBCFS_TYPE, 91, IOCTL_CONFIG_SIZE)
#define IOC_LIBCFS_MAX_NR 91
-static inline int libcfs_ioctl_packlen(struct libcfs_ioctl_data *data)
-{
- int len = sizeof(*data);
-
- len += cfs_size_round(data->ioc_inllen1);
- len += cfs_size_round(data->ioc_inllen2);
- return len;
-}
-
-static inline bool libcfs_ioctl_is_invalid(struct libcfs_ioctl_data *data)
-{
- if (data->ioc_hdr.ioc_len > (1 << 30)) {
- CERROR("LIBCFS ioctl: ioc_len larger than 1<<30\n");
- return 1;
- }
- if (data->ioc_inllen1 > (1<<30)) {
- CERROR("LIBCFS ioctl: ioc_inllen1 larger than 1<<30\n");
- return 1;
- }
- if (data->ioc_inllen2 > (1<<30)) {
- CERROR("LIBCFS ioctl: ioc_inllen2 larger than 1<<30\n");
- return 1;
- }
- if (data->ioc_inlbuf1 && !data->ioc_inllen1) {
- CERROR("LIBCFS ioctl: inlbuf1 pointer but 0 length\n");
- return 1;
- }
- if (data->ioc_inlbuf2 && !data->ioc_inllen2) {
- CERROR("LIBCFS ioctl: inlbuf2 pointer but 0 length\n");
- return 1;
- }
- if (data->ioc_pbuf1 && !data->ioc_plen1) {
- CERROR("LIBCFS ioctl: pbuf1 pointer but 0 length\n");
- return 1;
- }
- if (data->ioc_pbuf2 && !data->ioc_plen2) {
- CERROR("LIBCFS ioctl: pbuf2 pointer but 0 length\n");
- return 1;
- }
- if (data->ioc_plen1 && !data->ioc_pbuf1) {
- CERROR("LIBCFS ioctl: plen1 nonzero but no pbuf1 pointer\n");
- return 1;
- }
- if (data->ioc_plen2 && !data->ioc_pbuf2) {
- CERROR("LIBCFS ioctl: plen2 nonzero but no pbuf2 pointer\n");
- return 1;
- }
- if ((__u32)libcfs_ioctl_packlen(data) != data->ioc_hdr.ioc_len) {
- CERROR("LIBCFS ioctl: packlen != ioc_len\n");
- return 1;
- }
- if (data->ioc_inllen1 &&
- data->ioc_bulk[data->ioc_inllen1 - 1] != '\0') {
- CERROR("LIBCFS ioctl: inlbuf1 not 0 terminated\n");
- return 1;
- }
- if (data->ioc_inllen2 &&
- data->ioc_bulk[cfs_size_round(data->ioc_inllen1) +
- data->ioc_inllen2 - 1] != '\0') {
- CERROR("LIBCFS ioctl: inlbuf2 not 0 terminated\n");
- return 1;
- }
- return 0;
-}
-
-int libcfs_register_ioctl(struct libcfs_ioctl_handler *hand);
-int libcfs_deregister_ioctl(struct libcfs_ioctl_handler *hand);
-int libcfs_ioctl_getdata_len(const struct libcfs_ioctl_hdr __user *arg,
- __u32 *buf_len);
-int libcfs_ioctl_popdata(void __user *arg, void *buf, int size);
-int libcfs_ioctl_data_adjust(struct libcfs_ioctl_data *data);
-
#endif /* __LIBCFS_IOCTL_H__ */
diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_prim.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_prim.h
index 082fe6d..ac4e8cf 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs_prim.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_prim.h
@@ -40,20 +40,31 @@
#ifndef __LIBCFS_PRIM_H__
#define __LIBCFS_PRIM_H__
-void add_wait_queue_exclusive_head(wait_queue_head_t *, wait_queue_t *);
-
/*
* Memory
*/
-#ifndef memory_pressure_get
-#define memory_pressure_get() (0)
+#if BITS_PER_LONG == 32
+/* limit to lowmem on 32-bit systems */
+#define NUM_CACHEPAGES \
+ min(totalram_pages, 1UL << (30 - PAGE_SHIFT) * 3 / 4)
+#else
+#define NUM_CACHEPAGES totalram_pages
#endif
-#ifndef memory_pressure_set
-#define memory_pressure_set() do {} while (0)
-#endif
-#ifndef memory_pressure_clr
-#define memory_pressure_clr() do {} while (0)
-#endif
+
+static inline unsigned int memory_pressure_get(void)
+{
+ return current->flags & PF_MEMALLOC;
+}
+
+static inline void memory_pressure_set(void)
+{
+ current->flags |= PF_MEMALLOC;
+}
+
+static inline void memory_pressure_clr(void)
+{
+ current->flags &= ~PF_MEMALLOC;
+}
static inline int cfs_memory_pressure_get_and_set(void)
{
diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h
index 1333543..2fd2a96 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h
@@ -182,25 +182,6 @@
int libcfs_debug_mark_buffer(const char *text);
/*
- * allocate per-cpu-partition data, returned value is an array of pointers,
- * variable can be indexed by CPU ID.
- * cptable != NULL: size of array is number of CPU partitions
- * cptable == NULL: size of array is number of HW cores
- */
-void *cfs_percpt_alloc(struct cfs_cpt_table *cptab, unsigned int size);
-/*
- * destroy per-cpu-partition variable
- */
-void cfs_percpt_free(void *vars);
-int cfs_percpt_number(void *vars);
-void *cfs_percpt_current(void *vars);
-void *cfs_percpt_index(void *vars, int idx);
-
-#define cfs_percpt_for_each(var, i, vars) \
- for (i = 0; i < cfs_percpt_number(vars) && \
- ((var) = (vars)[i]) != NULL; i++)
-
-/*
* allocate a variable array, returned value is an array of pointers.
* Caller can specify length of array by count.
*/
@@ -302,62 +283,6 @@
#define CFS_ALLOC_PTR(ptr) LIBCFS_ALLOC(ptr, sizeof(*(ptr)))
#define CFS_FREE_PTR(ptr) LIBCFS_FREE(ptr, sizeof(*(ptr)))
-/*
- * percpu partition lock
- *
- * There are some use-cases like this in Lustre:
- * . each CPU partition has it's own private data which is frequently changed,
- * and mostly by the local CPU partition.
- * . all CPU partitions share some global data, these data are rarely changed.
- *
- * LNet is typical example.
- * CPU partition lock is designed for this kind of use-cases:
- * . each CPU partition has it's own private lock
- * . change on private data just needs to take the private lock
- * . read on shared data just needs to take _any_ of private locks
- * . change on shared data needs to take _all_ private locks,
- * which is slow and should be really rare.
- */
-
-enum {
- CFS_PERCPT_LOCK_EX = -1, /* negative */
-};
-
-struct cfs_percpt_lock {
- /* cpu-partition-table for this lock */
- struct cfs_cpt_table *pcl_cptab;
- /* exclusively locked */
- unsigned int pcl_locked;
- /* private lock table */
- spinlock_t **pcl_locks;
-};
-
-/* return number of private locks */
-static inline int
-cfs_percpt_lock_num(struct cfs_percpt_lock *pcl)
-{
- return cfs_cpt_number(pcl->pcl_cptab);
-}
-
-/*
- * create a cpu-partition lock based on CPU partition table \a cptab,
- * each private lock has extra \a psize bytes padding data
- */
-struct cfs_percpt_lock *cfs_percpt_lock_alloc(struct cfs_cpt_table *cptab);
-/* destroy a cpu-partition lock */
-void cfs_percpt_lock_free(struct cfs_percpt_lock *pcl);
-
-/* lock private lock \a index of \a pcl */
-void cfs_percpt_lock(struct cfs_percpt_lock *pcl, int index);
-/* unlock private lock \a index of \a pcl */
-void cfs_percpt_unlock(struct cfs_percpt_lock *pcl, int index);
-/* create percpt (atomic) refcount based on @cptab */
-atomic_t **cfs_percpt_atomic_alloc(struct cfs_cpt_table *cptab, int val);
-/* destroy percpt refcount */
-void cfs_percpt_atomic_free(atomic_t **refs);
-/* return sum of all percpu refs */
-int cfs_percpt_atomic_summary(atomic_t **refs);
-
/** Compile-time assertion.
* Check an invariant described by a constant expression at compile time by
diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_workitem.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_workitem.h
index 5cc64f3..f9b20c5 100644
--- a/drivers/staging/lustre/include/linux/libcfs/libcfs_workitem.h
+++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_workitem.h
@@ -73,7 +73,7 @@
struct cfs_workitem;
typedef int (*cfs_wi_action_t) (struct cfs_workitem *);
-typedef struct cfs_workitem {
+struct cfs_workitem {
/** chain on runq or rerunq */
struct list_head wi_list;
/** working function */
@@ -84,10 +84,10 @@
unsigned short wi_running:1;
/** scheduled */
unsigned short wi_scheduled:1;
-} cfs_workitem_t;
+};
static inline void
-cfs_wi_init(cfs_workitem_t *wi, void *data, cfs_wi_action_t action)
+cfs_wi_init(struct cfs_workitem *wi, void *data, cfs_wi_action_t action)
{
INIT_LIST_HEAD(&wi->wi_list);
@@ -97,9 +97,9 @@
wi->wi_action = action;
}
-void cfs_wi_schedule(struct cfs_wi_sched *sched, cfs_workitem_t *wi);
-int cfs_wi_deschedule(struct cfs_wi_sched *sched, cfs_workitem_t *wi);
-void cfs_wi_exit(struct cfs_wi_sched *sched, cfs_workitem_t *wi);
+void cfs_wi_schedule(struct cfs_wi_sched *sched, struct cfs_workitem *wi);
+int cfs_wi_deschedule(struct cfs_wi_sched *sched, struct cfs_workitem *wi);
+void cfs_wi_exit(struct cfs_wi_sched *sched, struct cfs_workitem *wi);
int cfs_wi_startup(void);
void cfs_wi_shutdown(void);
diff --git a/drivers/staging/lustre/include/linux/libcfs/linux/libcfs.h b/drivers/staging/lustre/include/linux/libcfs/linux/libcfs.h
index d94b266..a268ef7 100644
--- a/drivers/staging/lustre/include/linux/libcfs/linux/libcfs.h
+++ b/drivers/staging/lustre/include/linux/libcfs/linux/libcfs.h
@@ -60,6 +60,7 @@
#include <linux/moduleparam.h>
#include <linux/mutex.h>
#include <linux/notifier.h>
+#include <linux/pagemap.h>
#include <linux/random.h>
#include <linux/rbtree.h>
#include <linux/rwsem.h>
@@ -83,7 +84,6 @@
#include <stdarg.h>
#include "linux-cpu.h"
#include "linux-time.h"
-#include "linux-mem.h"
#define LUSTRE_TRACE_SIZE (THREAD_SIZE >> 5)
diff --git a/drivers/staging/lustre/include/linux/libcfs/linux/linux-cpu.h b/drivers/staging/lustre/include/linux/libcfs/linux/linux-cpu.h
index c04979a..f63cb47 100644
--- a/drivers/staging/lustre/include/linux/libcfs/linux/linux-cpu.h
+++ b/drivers/staging/lustre/include/linux/libcfs/linux/linux-cpu.h
@@ -23,7 +23,7 @@
* This file is part of Lustre, http://www.lustre.org/
* Lustre is a trademark of Sun Microsystems, Inc.
*
- * libcfs/include/libcfs/linux/linux-mem.h
+ * libcfs/include/libcfs/linux/linux-cpu.h
*
* Basic library routines.
*
diff --git a/drivers/staging/lustre/include/linux/libcfs/linux/linux-mem.h b/drivers/staging/lustre/include/linux/libcfs/linux/linux-mem.h
deleted file mode 100644
index 837eb22..0000000
--- a/drivers/staging/lustre/include/linux/libcfs/linux/linux-mem.h
+++ /dev/null
@@ -1,80 +0,0 @@
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.sun.com/software/products/lustre/docs/GPLv2.pdf
- *
- * Please contact Sun Microsystems, Inc., 4150 Network Circle, Santa Clara,
- * CA 95054 USA or visit www.sun.com if you need additional information or
- * have any questions.
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2011, 2012, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- *
- * libcfs/include/libcfs/linux/linux-mem.h
- *
- * Basic library routines.
- */
-
-#ifndef __LIBCFS_LINUX_CFS_MEM_H__
-#define __LIBCFS_LINUX_CFS_MEM_H__
-
-#ifndef __LIBCFS_LIBCFS_H__
-#error Do not #include this file directly. #include <linux/libcfs/libcfs.h> instead
-#endif
-
-#include <linux/mm.h>
-#include <linux/vmalloc.h>
-#include <linux/pagemap.h>
-#include <linux/slab.h>
-#include <linux/memcontrol.h>
-#include <linux/mm_inline.h>
-
-#ifndef HAVE_LIBCFS_CPT
-/* Need this for cfs_cpt_table */
-#include "../libcfs_cpu.h"
-#endif
-
-#define CFS_PAGE_MASK (~((__u64)PAGE_SIZE-1))
-#define page_index(p) ((p)->index)
-
-#define memory_pressure_get() (current->flags & PF_MEMALLOC)
-#define memory_pressure_set() do { current->flags |= PF_MEMALLOC; } while (0)
-#define memory_pressure_clr() do { current->flags &= ~PF_MEMALLOC; } while (0)
-
-#if BITS_PER_LONG == 32
-/* limit to lowmem on 32-bit systems */
-#define NUM_CACHEPAGES \
- min(totalram_pages, 1UL << (30 - PAGE_SHIFT) * 3 / 4)
-#else
-#define NUM_CACHEPAGES totalram_pages
-#endif
-
-#define DECL_MMSPACE mm_segment_t __oldfs
-#define MMSPACE_OPEN \
- do { __oldfs = get_fs(); set_fs(get_ds()); } while (0)
-#define MMSPACE_CLOSE set_fs(__oldfs)
-
-#endif /* __LINUX_CFS_MEM_H__ */
diff --git a/drivers/staging/lustre/include/linux/libcfs/linux/linux-time.h b/drivers/staging/lustre/include/linux/libcfs/linux/linux-time.h
index ed8764b..7656b09 100644
--- a/drivers/staging/lustre/include/linux/libcfs/linux/linux-time.h
+++ b/drivers/staging/lustre/include/linux/libcfs/linux/linux-time.h
@@ -70,12 +70,12 @@
static inline long cfs_time_seconds(int seconds)
{
- return ((long)seconds) * HZ;
+ return ((long)seconds) * msecs_to_jiffies(MSEC_PER_SEC);
}
static inline long cfs_duration_sec(long d)
{
- return d / HZ;
+ return d / msecs_to_jiffies(MSEC_PER_SEC);
}
#define cfs_time_current_64 get_jiffies_64
diff --git a/drivers/staging/lustre/include/linux/lnet/lib-dlc.h b/drivers/staging/lustre/include/linux/lnet/lib-dlc.h
index 84a19e9..6ce9acc 100644
--- a/drivers/staging/lustre/include/linux/lnet/lib-dlc.h
+++ b/drivers/staging/lustre/include/linux/lnet/lib-dlc.h
@@ -37,10 +37,37 @@
#define LNET_MAX_SHOW_NUM_CPT 128
#define LNET_UNDEFINED_HOPS ((__u32) -1)
+struct lnet_ioctl_config_lnd_cmn_tunables {
+ __u32 lct_version;
+ __u32 lct_peer_timeout;
+ __u32 lct_peer_tx_credits;
+ __u32 lct_peer_rtr_credits;
+ __u32 lct_max_tx_credits;
+};
+
+struct lnet_ioctl_config_o2iblnd_tunables {
+ __u32 lnd_version;
+ __u32 lnd_peercredits_hiw;
+ __u32 lnd_map_on_demand;
+ __u32 lnd_concurrent_sends;
+ __u32 lnd_fmr_pool_size;
+ __u32 lnd_fmr_flush_trigger;
+ __u32 lnd_fmr_cache;
+ __u32 pad;
+};
+
+struct lnet_ioctl_config_lnd_tunables {
+ struct lnet_ioctl_config_lnd_cmn_tunables lt_cmn;
+ union {
+ struct lnet_ioctl_config_o2iblnd_tunables lt_o2ib;
+ } lt_tun_u;
+};
+
struct lnet_ioctl_net_config {
char ni_interfaces[LNET_MAX_INTERFACES][LNET_MAX_STR_LEN];
__u32 ni_status;
__u32 ni_cpts[LNET_MAX_SHOW_NUM_CPT];
+ char cfg_bulk[0];
};
#define LNET_TINY_BUF_IDX 0
@@ -81,7 +108,7 @@
__s32 net_peer_rtr_credits;
__s32 net_max_tx_credits;
__u32 net_cksum_algo;
- __u32 net_pad;
+ __u32 net_interface_count;
} cfg_net;
struct {
__u32 buf_enable;
diff --git a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
index dfc0208..513a822 100644
--- a/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
+++ b/drivers/staging/lustre/include/linux/lnet/lib-lnet.h
@@ -463,10 +463,6 @@
void lnet_destroy_routes(void);
int lnet_get_route(int idx, __u32 *net, __u32 *hops,
lnet_nid_t *gateway, __u32 *alive, __u32 *priority);
-int lnet_get_net_config(int idx, __u32 *cpt_count, __u64 *nid,
- int *peer_timeout, int *peer_tx_credits,
- int *peer_rtr_cr, int *max_tx_credits,
- struct lnet_ioctl_net_config *net_config);
int lnet_get_rtr_pool_cfg(int idx, struct lnet_ioctl_pool_cfg *pool_cfg);
void lnet_router_debugfs_init(void);
@@ -478,9 +474,8 @@
void lnet_rtrpools_disable(void);
void lnet_rtrpools_free(int keep_pools);
lnet_remotenet_t *lnet_find_net_locked(__u32 net);
-int lnet_dyn_add_ni(lnet_pid_t requested_pid, char *nets,
- __s32 peer_timeout, __s32 peer_cr, __s32 peer_buf_cr,
- __s32 credits);
+int lnet_dyn_add_ni(lnet_pid_t requested_pid,
+ struct lnet_ioctl_config_data *conf);
int lnet_dyn_del_ni(__u32 net);
int lnet_clear_lazy_portal(struct lnet_ni *ni, int portal, char *reason);
diff --git a/drivers/staging/lustre/include/linux/lnet/lib-types.h b/drivers/staging/lustre/include/linux/lnet/lib-types.h
index 29c72f8..24c4a08 100644
--- a/drivers/staging/lustre/include/linux/lnet/lib-types.h
+++ b/drivers/staging/lustre/include/linux/lnet/lib-types.h
@@ -273,6 +273,8 @@
int **ni_refs; /* percpt reference count */
time64_t ni_last_alive;/* when I was last alive */
lnet_ni_status_t *ni_status; /* my health status */
+ /* per NI LND tunables */
+ struct lnet_ioctl_config_lnd_tunables *ni_lnd_tunables;
/* equivalent interfaces to use */
char *ni_interfaces[LNET_MAX_INTERFACES];
} lnet_ni_t;
diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
index 0d32e65..6c59f2f 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c
@@ -335,8 +335,8 @@
peer->ibp_nid = nid;
peer->ibp_error = 0;
peer->ibp_last_alive = 0;
- peer->ibp_max_frags = IBLND_CFG_RDMA_FRAGS;
- peer->ibp_queue_depth = *kiblnd_tunables.kib_peertxcredits;
+ peer->ibp_max_frags = kiblnd_cfg_rdma_frags(peer->ibp_ni);
+ peer->ibp_queue_depth = ni->ni_peertxcredits;
atomic_set(&peer->ibp_refcount, 1); /* 1 ref for caller */
INIT_LIST_HEAD(&peer->ibp_list); /* not in the peer table yet */
@@ -1283,65 +1283,86 @@
}
}
-struct ib_mr *kiblnd_find_rd_dma_mr(kib_hca_dev_t *hdev, kib_rdma_desc_t *rd,
+struct ib_mr *kiblnd_find_rd_dma_mr(struct lnet_ni *ni, kib_rdma_desc_t *rd,
int negotiated_nfrags)
{
- __u16 nfrags = (negotiated_nfrags != -1) ?
- negotiated_nfrags : *kiblnd_tunables.kib_map_on_demand;
+ kib_net_t *net = ni->ni_data;
+ kib_hca_dev_t *hdev = net->ibn_dev->ibd_hdev;
+ struct lnet_ioctl_config_o2iblnd_tunables *tunables;
+ __u16 nfrags;
+ int mod;
+
+ tunables = &ni->ni_lnd_tunables->lt_tun_u.lt_o2ib;
+ mod = tunables->lnd_map_on_demand;
+ nfrags = (negotiated_nfrags != -1) ? negotiated_nfrags : mod;
LASSERT(hdev->ibh_mrs);
- if (*kiblnd_tunables.kib_map_on_demand > 0 &&
- nfrags <= rd->rd_nfrags)
+ if (mod > 0 && nfrags <= rd->rd_nfrags)
return NULL;
return hdev->ibh_mrs;
}
-static void kiblnd_destroy_fmr_pool(kib_fmr_pool_t *pool)
+static void kiblnd_destroy_fmr_pool(kib_fmr_pool_t *fpo)
{
- LASSERT(!pool->fpo_map_count);
+ LASSERT(!fpo->fpo_map_count);
- if (pool->fpo_fmr_pool)
- ib_destroy_fmr_pool(pool->fpo_fmr_pool);
+ if (fpo->fpo_is_fmr) {
+ if (fpo->fmr.fpo_fmr_pool)
+ ib_destroy_fmr_pool(fpo->fmr.fpo_fmr_pool);
+ } else {
+ struct kib_fast_reg_descriptor *frd, *tmp;
+ int i = 0;
- if (pool->fpo_hdev)
- kiblnd_hdev_decref(pool->fpo_hdev);
+ list_for_each_entry_safe(frd, tmp, &fpo->fast_reg.fpo_pool_list,
+ frd_list) {
+ list_del(&frd->frd_list);
+ ib_dereg_mr(frd->frd_mr);
+ LIBCFS_FREE(frd, sizeof(*frd));
+ i++;
+ }
+ if (i < fpo->fast_reg.fpo_pool_size)
+ CERROR("FastReg pool still has %d regions registered\n",
+ fpo->fast_reg.fpo_pool_size - i);
+ }
- LIBCFS_FREE(pool, sizeof(*pool));
+ if (fpo->fpo_hdev)
+ kiblnd_hdev_decref(fpo->fpo_hdev);
+
+ LIBCFS_FREE(fpo, sizeof(*fpo));
}
static void kiblnd_destroy_fmr_pool_list(struct list_head *head)
{
- kib_fmr_pool_t *pool;
+ kib_fmr_pool_t *fpo, *tmp;
- while (!list_empty(head)) {
- pool = list_entry(head->next, kib_fmr_pool_t, fpo_list);
- list_del(&pool->fpo_list);
- kiblnd_destroy_fmr_pool(pool);
+ list_for_each_entry_safe(fpo, tmp, head, fpo_list) {
+ list_del(&fpo->fpo_list);
+ kiblnd_destroy_fmr_pool(fpo);
}
}
-static int kiblnd_fmr_pool_size(int ncpts)
+static int
+kiblnd_fmr_pool_size(struct lnet_ioctl_config_o2iblnd_tunables *tunables,
+ int ncpts)
{
- int size = *kiblnd_tunables.kib_fmr_pool_size / ncpts;
+ int size = tunables->lnd_fmr_pool_size / ncpts;
return max(IBLND_FMR_POOL, size);
}
-static int kiblnd_fmr_flush_trigger(int ncpts)
+static int
+kiblnd_fmr_flush_trigger(struct lnet_ioctl_config_o2iblnd_tunables *tunables,
+ int ncpts)
{
- int size = *kiblnd_tunables.kib_fmr_flush_trigger / ncpts;
+ int size = tunables->lnd_fmr_flush_trigger / ncpts;
return max(IBLND_FMR_POOL_FLUSH, size);
}
-static int kiblnd_create_fmr_pool(kib_fmr_poolset_t *fps,
- kib_fmr_pool_t **pp_fpo)
+static int kiblnd_alloc_fmr_pool(kib_fmr_poolset_t *fps, kib_fmr_pool_t *fpo)
{
- /* FMR pool for RDMA */
- kib_dev_t *dev = fps->fps_net->ibn_dev;
- kib_fmr_pool_t *fpo;
struct ib_fmr_pool_param param = {
.max_pages_per_fmr = LNET_MAX_PAYLOAD / PAGE_SIZE,
.page_shift = PAGE_SHIFT,
@@ -1351,7 +1372,78 @@
.dirty_watermark = fps->fps_flush_trigger,
.flush_function = NULL,
.flush_arg = NULL,
- .cache = !!*kiblnd_tunables.kib_fmr_cache};
+ .cache = !!fps->fps_cache };
+ int rc = 0;
+
+ fpo->fmr.fpo_fmr_pool = ib_create_fmr_pool(fpo->fpo_hdev->ibh_pd,
+ ¶m);
+ if (IS_ERR(fpo->fmr.fpo_fmr_pool)) {
+ rc = PTR_ERR(fpo->fmr.fpo_fmr_pool);
+ if (rc != -ENOSYS)
+ CERROR("Failed to create FMR pool: %d\n", rc);
+ else
+ CERROR("FMRs are not supported\n");
+ }
+
+ return rc;
+}
+
+static int kiblnd_alloc_freg_pool(kib_fmr_poolset_t *fps, kib_fmr_pool_t *fpo)
+{
+ struct kib_fast_reg_descriptor *frd, *tmp;
+ int i, rc;
+
+ INIT_LIST_HEAD(&fpo->fast_reg.fpo_pool_list);
+ fpo->fast_reg.fpo_pool_size = 0;
+ for (i = 0; i < fps->fps_pool_size; i++) {
+ LIBCFS_CPT_ALLOC(frd, lnet_cpt_table(), fps->fps_cpt,
+ sizeof(*frd));
+ if (!frd) {
+ CERROR("Failed to allocate a new fast_reg descriptor\n");
+ rc = -ENOMEM;
+ goto out;
+ }
+
+ frd->frd_mr = ib_alloc_mr(fpo->fpo_hdev->ibh_pd,
+ IB_MR_TYPE_MEM_REG,
+ LNET_MAX_PAYLOAD / PAGE_SIZE);
+ if (IS_ERR(frd->frd_mr)) {
+ rc = PTR_ERR(frd->frd_mr);
+ CERROR("Failed to allocate ib_alloc_mr: %d\n", rc);
+ frd->frd_mr = NULL;
+ goto out_middle;
+ }
+
+ frd->frd_valid = true;
+
+ list_add_tail(&frd->frd_list, &fpo->fast_reg.fpo_pool_list);
+ fpo->fast_reg.fpo_pool_size++;
+ }
+
+ return 0;
+
+out_middle:
+ if (frd->frd_mr)
+ ib_dereg_mr(frd->frd_mr);
+ LIBCFS_FREE(frd, sizeof(*frd));
+
+out:
+ list_for_each_entry_safe(frd, tmp, &fpo->fast_reg.fpo_pool_list,
+ frd_list) {
+ list_del(&frd->frd_list);
+ ib_dereg_mr(frd->frd_mr);
+ LIBCFS_FREE(frd, sizeof(*frd));
+ }
+
+ return rc;
+}
+
+static int kiblnd_create_fmr_pool(kib_fmr_poolset_t *fps,
+ kib_fmr_pool_t **pp_fpo)
+{
+ kib_dev_t *dev = fps->fps_net->ibn_dev;
+ struct ib_device_attr *dev_attr;
+ kib_fmr_pool_t *fpo;
int rc;
LIBCFS_CPT_ALLOC(fpo, lnet_cpt_table(), fps->fps_cpt, sizeof(*fpo));
@@ -1359,22 +1451,41 @@
return -ENOMEM;
fpo->fpo_hdev = kiblnd_current_hdev(dev);
+ dev_attr = &fpo->fpo_hdev->ibh_ibdev->attrs;
- fpo->fpo_fmr_pool = ib_create_fmr_pool(fpo->fpo_hdev->ibh_pd, ¶m);
- if (IS_ERR(fpo->fpo_fmr_pool)) {
- rc = PTR_ERR(fpo->fpo_fmr_pool);
- CERROR("Failed to create FMR pool: %d\n", rc);
-
- kiblnd_hdev_decref(fpo->fpo_hdev);
- LIBCFS_FREE(fpo, sizeof(*fpo));
- return rc;
+ /* Check for FMR or FastReg support */
+ fpo->fpo_is_fmr = 0;
+ if (fpo->fpo_hdev->ibh_ibdev->alloc_fmr &&
+ fpo->fpo_hdev->ibh_ibdev->dealloc_fmr &&
+ fpo->fpo_hdev->ibh_ibdev->map_phys_fmr &&
+ fpo->fpo_hdev->ibh_ibdev->unmap_fmr) {
+ LCONSOLE_INFO("Using FMR for registration\n");
+ fpo->fpo_is_fmr = 1;
+ } else if (dev_attr->device_cap_flags & IB_DEVICE_MEM_MGT_EXTENSIONS) {
+ LCONSOLE_INFO("Using FastReg for registration\n");
+ } else {
+ rc = -ENOSYS;
+ LCONSOLE_ERROR_MSG(rc, "IB device does not support FMRs nor FastRegs, can't register memory\n");
+ goto out_fpo;
}
+ if (fpo->fpo_is_fmr)
+ rc = kiblnd_alloc_fmr_pool(fps, fpo);
+ else
+ rc = kiblnd_alloc_freg_pool(fps, fpo);
+ if (rc)
+ goto out_fpo;
+
fpo->fpo_deadline = cfs_time_shift(IBLND_POOL_DEADLINE);
- fpo->fpo_owner = fps;
+ fpo->fpo_owner = fps;
*pp_fpo = fpo;
return 0;
+
+out_fpo:
+ kiblnd_hdev_decref(fpo->fpo_hdev);
+ LIBCFS_FREE(fpo, sizeof(*fpo));
+ return rc;
}
static void kiblnd_fail_fmr_poolset(kib_fmr_poolset_t *fps,
@@ -1407,9 +1518,10 @@
}
}
-static int kiblnd_init_fmr_poolset(kib_fmr_poolset_t *fps, int cpt,
- kib_net_t *net, int pool_size,
- int flush_trigger)
+static int
+kiblnd_init_fmr_poolset(kib_fmr_poolset_t *fps, int cpt, int ncpts,
+ kib_net_t *net,
+ struct lnet_ioctl_config_o2iblnd_tunables *tunables)
{
kib_fmr_pool_t *fpo;
int rc;
@@ -1418,8 +1530,11 @@
fps->fps_net = net;
fps->fps_cpt = cpt;
- fps->fps_pool_size = pool_size;
- fps->fps_flush_trigger = flush_trigger;
+
+ fps->fps_pool_size = kiblnd_fmr_pool_size(tunables, ncpts);
+ fps->fps_flush_trigger = kiblnd_fmr_flush_trigger(tunables, ncpts);
+ fps->fps_cache = tunables->lnd_fmr_cache;
+
spin_lock_init(&fps->fps_lock);
INIT_LIST_HEAD(&fps->fps_pool_list);
INIT_LIST_HEAD(&fps->fps_failed_pool_list);
@@ -1440,25 +1555,64 @@
return cfs_time_aftereq(now, fpo->fpo_deadline);
}
+static int
+kiblnd_map_tx_pages(kib_tx_t *tx, kib_rdma_desc_t *rd)
+{
+ __u64 *pages = tx->tx_pages;
+ kib_hca_dev_t *hdev;
+ int npages;
+ int size;
+ int i;
+
+ hdev = tx->tx_pool->tpo_hdev;
+
+ for (i = 0, npages = 0; i < rd->rd_nfrags; i++) {
+ for (size = 0; size < rd->rd_frags[i].rf_nob;
+ size += hdev->ibh_page_size) {
+ pages[npages++] = (rd->rd_frags[i].rf_addr &
+ hdev->ibh_page_mask) + size;
+ }
+ }
+
+ return npages;
+}
+
void kiblnd_fmr_pool_unmap(kib_fmr_t *fmr, int status)
{
LIST_HEAD(zombies);
kib_fmr_pool_t *fpo = fmr->fmr_pool;
- kib_fmr_poolset_t *fps = fpo->fpo_owner;
+ kib_fmr_poolset_t *fps;
unsigned long now = cfs_time_current();
kib_fmr_pool_t *tmp;
int rc;
- rc = ib_fmr_pool_unmap(fmr->fmr_pfmr);
- LASSERT(!rc);
+ if (!fpo)
+ return;
- if (status) {
- rc = ib_flush_fmr_pool(fpo->fpo_fmr_pool);
- LASSERT(!rc);
+ fps = fpo->fpo_owner;
+ if (fpo->fpo_is_fmr) {
+ if (fmr->fmr_pfmr) {
+ rc = ib_fmr_pool_unmap(fmr->fmr_pfmr);
+ LASSERT(!rc);
+ fmr->fmr_pfmr = NULL;
+ }
+
+ if (status) {
+ rc = ib_flush_fmr_pool(fpo->fmr.fpo_fmr_pool);
+ LASSERT(!rc);
+ }
+ } else {
+ struct kib_fast_reg_descriptor *frd = fmr->fmr_frd;
+
+ if (frd) {
+ frd->frd_valid = false;
+ spin_lock(&fps->fps_lock);
+ list_add_tail(&frd->frd_list, &fpo->fast_reg.fpo_pool_list);
+ spin_unlock(&fps->fps_lock);
+ fmr->fmr_frd = NULL;
+ }
}
-
fmr->fmr_pool = NULL;
- fmr->fmr_pfmr = NULL;
spin_lock(&fps->fps_lock);
fpo->fpo_map_count--; /* decref the pool */
@@ -1479,11 +1633,15 @@
kiblnd_destroy_fmr_pool_list(&zombies);
}
-int kiblnd_fmr_pool_map(kib_fmr_poolset_t *fps, __u64 *pages, int npages,
- __u64 iov, kib_fmr_t *fmr)
+int kiblnd_fmr_pool_map(kib_fmr_poolset_t *fps, kib_tx_t *tx,
+ kib_rdma_desc_t *rd, __u32 nob, __u64 iov,
+ kib_fmr_t *fmr)
{
- struct ib_pool_fmr *pfmr;
+ __u64 *pages = tx->tx_pages;
+ bool is_rx = (rd != tx->tx_rd);
+ bool tx_pages_mapped = 0;
kib_fmr_pool_t *fpo;
+ int npages = 0;
__u64 version;
int rc;
@@ -1493,21 +1651,95 @@
list_for_each_entry(fpo, &fps->fps_pool_list, fpo_list) {
fpo->fpo_deadline = cfs_time_shift(IBLND_POOL_DEADLINE);
fpo->fpo_map_count++;
- spin_unlock(&fps->fps_lock);
- pfmr = ib_fmr_pool_map_phys(fpo->fpo_fmr_pool,
- pages, npages, iov);
- if (likely(!IS_ERR(pfmr))) {
- fmr->fmr_pool = fpo;
- fmr->fmr_pfmr = pfmr;
- return 0;
+ if (fpo->fpo_is_fmr) {
+ struct ib_pool_fmr *pfmr;
+
+ spin_unlock(&fps->fps_lock);
+
+ if (!tx_pages_mapped) {
+ npages = kiblnd_map_tx_pages(tx, rd);
+ tx_pages_mapped = 1;
+ }
+
+ pfmr = ib_fmr_pool_map_phys(fpo->fmr.fpo_fmr_pool,
+ pages, npages, iov);
+ if (likely(!IS_ERR(pfmr))) {
+ fmr->fmr_key = is_rx ? pfmr->fmr->rkey :
+ pfmr->fmr->lkey;
+ fmr->fmr_frd = NULL;
+ fmr->fmr_pfmr = pfmr;
+ fmr->fmr_pool = fpo;
+ return 0;
+ }
+ rc = PTR_ERR(pfmr);
+ } else {
+ if (!list_empty(&fpo->fast_reg.fpo_pool_list)) {
+ struct kib_fast_reg_descriptor *frd;
+ struct ib_reg_wr *wr;
+ struct ib_mr *mr;
+ int n;
+
+ frd = list_first_entry(&fpo->fast_reg.fpo_pool_list,
+ struct kib_fast_reg_descriptor,
+ frd_list);
+ list_del(&frd->frd_list);
+ spin_unlock(&fps->fps_lock);
+
+ mr = frd->frd_mr;
+
+ if (!frd->frd_valid) {
+ __u32 key = is_rx ? mr->rkey : mr->lkey;
+ struct ib_send_wr *inv_wr;
+
+ inv_wr = &frd->frd_inv_wr;
+ memset(inv_wr, 0, sizeof(*inv_wr));
+ inv_wr->opcode = IB_WR_LOCAL_INV;
+ inv_wr->wr_id = IBLND_WID_MR;
+ inv_wr->ex.invalidate_rkey = key;
+
+ /* Bump the key */
+ key = ib_inc_rkey(key);
+ ib_update_fast_reg_key(mr, key);
+ }
+
+ n = ib_map_mr_sg(mr, tx->tx_frags,
+ tx->tx_nfrags, NULL, PAGE_SIZE);
+ if (unlikely(n != tx->tx_nfrags)) {
+ CERROR("Failed to map mr %d/%d elements\n",
+ n, tx->tx_nfrags);
+ return n < 0 ? n : -EINVAL;
+ }
+
+ mr->iova = iov;
+
+ /* Prepare FastReg WR */
+ wr = &frd->frd_fastreg_wr;
+ memset(wr, 0, sizeof(*wr));
+ wr->wr.opcode = IB_WR_REG_MR;
+ wr->wr.wr_id = IBLND_WID_MR;
+ wr->wr.num_sge = 0;
+ wr->wr.send_flags = 0;
+ wr->mr = mr;
+ wr->key = is_rx ? mr->rkey : mr->lkey;
+ wr->access = (IB_ACCESS_LOCAL_WRITE |
+ IB_ACCESS_REMOTE_WRITE);
+
+ fmr->fmr_key = is_rx ? mr->rkey : mr->lkey;
+ fmr->fmr_frd = frd;
+ fmr->fmr_pfmr = NULL;
+ fmr->fmr_pool = fpo;
+ return 0;
+ }
+ spin_unlock(&fps->fps_lock);
+ rc = -EBUSY;
}
spin_lock(&fps->fps_lock);
fpo->fpo_map_count--;
- if (PTR_ERR(pfmr) != -EAGAIN) {
+ if (rc != -EAGAIN) {
spin_unlock(&fps->fps_lock);
- return PTR_ERR(pfmr);
+ return rc;
}
/* EAGAIN and ... */
@@ -1932,25 +2164,28 @@
}
}
-static int kiblnd_net_init_pools(kib_net_t *net, __u32 *cpts, int ncpts)
+static int kiblnd_net_init_pools(kib_net_t *net, lnet_ni_t *ni, __u32 *cpts,
+ int ncpts)
{
+ struct lnet_ioctl_config_o2iblnd_tunables *tunables;
unsigned long flags;
int cpt;
- int rc = 0;
+ int rc;
int i;
+ tunables = &ni->ni_lnd_tunables->lt_tun_u.lt_o2ib;
+
read_lock_irqsave(&kiblnd_data.kib_global_lock, flags);
- if (!*kiblnd_tunables.kib_map_on_demand) {
+ if (!tunables->lnd_map_on_demand) {
read_unlock_irqrestore(&kiblnd_data.kib_global_lock, flags);
goto create_tx_pool;
}
read_unlock_irqrestore(&kiblnd_data.kib_global_lock, flags);
- if (*kiblnd_tunables.kib_fmr_pool_size <
- *kiblnd_tunables.kib_ntx / 4) {
+ if (tunables->lnd_fmr_pool_size < *kiblnd_tunables.kib_ntx / 4) {
CERROR("Can't set fmr pool size (%d) < ntx / 4(%d)\n",
- *kiblnd_tunables.kib_fmr_pool_size,
+ tunables->lnd_fmr_pool_size,
*kiblnd_tunables.kib_ntx / 4);
rc = -EINVAL;
goto failed;
@@ -1965,8 +2200,11 @@
/*
* premapping can fail if ibd_nmr > 1, so we always create
* FMR pool and map-on-demand if premapping failed
+ *
+ * cfs_precpt_alloc is creating an array of struct kib_fmr_poolset
+ * The number of struct kib_fmr_poolsets create is equal to the
+ * number of CPTs that exist, i.e net->ibn_fmr_ps[cpt].
*/
-
net->ibn_fmr_ps = cfs_percpt_alloc(lnet_cpt_table(),
sizeof(kib_fmr_poolset_t));
if (!net->ibn_fmr_ps) {
@@ -1977,9 +2215,8 @@
for (i = 0; i < ncpts; i++) {
cpt = !cpts ? i : cpts[i];
- rc = kiblnd_init_fmr_poolset(net->ibn_fmr_ps[cpt], cpt, net,
- kiblnd_fmr_pool_size(ncpts),
- kiblnd_fmr_flush_trigger(ncpts));
+ rc = kiblnd_init_fmr_poolset(net->ibn_fmr_ps[cpt], cpt, ncpts,
+ net, tunables);
if (rc) {
CERROR("Can't initialize FMR pool for CPT %d: %d\n",
cpt, rc);
@@ -1991,6 +2228,11 @@
LASSERT(i == ncpts);
create_tx_pool:
+ /*
+ * cfs_precpt_alloc is creating an array of struct kib_tx_poolset
+ * The number of struct kib_tx_poolsets create is equal to the
+ * number of CPTs that exist, i.e net->ibn_tx_ps[cpt].
+ */
net->ibn_tx_ps = cfs_percpt_alloc(lnet_cpt_table(),
sizeof(kib_tx_poolset_t));
if (!net->ibn_tx_ps) {
@@ -2694,10 +2936,9 @@
net->ibn_incarnation = tv.tv_sec * USEC_PER_SEC +
tv.tv_nsec / NSEC_PER_USEC;
- ni->ni_peertimeout = *kiblnd_tunables.kib_peertimeout;
- ni->ni_maxtxcredits = *kiblnd_tunables.kib_credits;
- ni->ni_peertxcredits = *kiblnd_tunables.kib_peertxcredits;
- ni->ni_peerrtrcredits = *kiblnd_tunables.kib_peerrtrcredits;
+ rc = kiblnd_tunables_setup(ni);
+ if (rc)
+ goto net_failed;
if (ni->ni_interfaces[0]) {
/* Use the IPoIB interface specified in 'networks=' */
@@ -2736,7 +2977,7 @@
if (rc)
goto failed;
- rc = kiblnd_net_init_pools(net, ni->ni_cpts, ni->ni_ncpts);
+ rc = kiblnd_net_init_pools(net, ni, ni->ni_cpts, ni->ni_ncpts);
if (rc) {
CERROR("Failed to initialize NI pools: %d\n", rc);
goto failed;
@@ -2779,8 +3020,6 @@
static int __init ko2iblnd_init(void)
{
- int rc;
-
CLASSERT(sizeof(kib_msg_t) <= IBLND_MSG_SIZE);
CLASSERT(offsetof(kib_msg_t,
ibm_u.get.ibgm_rd.rd_frags[IBLND_MAX_RDMA_FRAGS])
@@ -2789,9 +3028,7 @@
ibm_u.putack.ibpam_rd.rd_frags[IBLND_MAX_RDMA_FRAGS])
<= IBLND_MSG_SIZE);
- rc = kiblnd_tunables_init();
- if (rc)
- return rc;
+ kiblnd_tunables_init();
lnet_register_lnd(&the_o2iblnd);
diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
index bfcbdd1..b22984f 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.h
@@ -87,22 +87,10 @@
int *kib_timeout; /* comms timeout (seconds) */
int *kib_keepalive; /* keepalive timeout (seconds) */
int *kib_ntx; /* # tx descs */
- int *kib_credits; /* # concurrent sends */
- int *kib_peertxcredits; /* # concurrent sends to 1 peer */
- int *kib_peerrtrcredits; /* # per-peer router buffer credits */
- int *kib_peercredits_hiw; /* # when eagerly to return credits */
- int *kib_peertimeout; /* seconds to consider peer dead */
char **kib_default_ipif; /* default IPoIB interface */
int *kib_retry_count;
int *kib_rnr_retry_count;
- int *kib_concurrent_sends; /* send work queue sizing */
int *kib_ib_mtu; /* IB MTU */
- int *kib_map_on_demand; /* map-on-demand if RD has more */
- /* fragments than this value, 0 */
- /* disable map-on-demand */
- int *kib_fmr_pool_size; /* # FMRs in pool */
- int *kib_fmr_flush_trigger; /* When to trigger FMR flush */
- int *kib_fmr_cache; /* enable FMR pool cache? */
int *kib_require_priv_port; /* accept only privileged ports */
int *kib_use_priv_port; /* use privileged port for active connect */
int *kib_nscheds; /* # threads on each CPT */
@@ -116,43 +104,21 @@
#define IBLND_CREDITS_DEFAULT 8 /* default # of peer credits */
#define IBLND_CREDITS_MAX ((typeof(((kib_msg_t *) 0)->ibm_credits)) - 1) /* Max # of peer credits */
-#define IBLND_MSG_QUEUE_SIZE(v) ((v) == IBLND_MSG_VERSION_1 ? \
- IBLND_MSG_QUEUE_SIZE_V1 : \
- *kiblnd_tunables.kib_peertxcredits) /* # messages/RDMAs in-flight */
-#define IBLND_CREDITS_HIGHWATER(v) ((v) == IBLND_MSG_VERSION_1 ? \
- IBLND_CREDIT_HIGHWATER_V1 : \
- *kiblnd_tunables.kib_peercredits_hiw) /* when eagerly to return credits */
+/* when eagerly to return credits */
+#define IBLND_CREDITS_HIGHWATER(t, v) ((v) == IBLND_MSG_VERSION_1 ? \
+ IBLND_CREDIT_HIGHWATER_V1 : \
+ t->lnd_peercredits_hiw)
#define kiblnd_rdma_create_id(cb, dev, ps, qpt) rdma_create_id(&init_net, \
cb, dev, \
ps, qpt)
-static inline int
-kiblnd_concurrent_sends_v1(void)
-{
- if (*kiblnd_tunables.kib_concurrent_sends > IBLND_MSG_QUEUE_SIZE_V1 * 2)
- return IBLND_MSG_QUEUE_SIZE_V1 * 2;
-
- if (*kiblnd_tunables.kib_concurrent_sends < IBLND_MSG_QUEUE_SIZE_V1 / 2)
- return IBLND_MSG_QUEUE_SIZE_V1 / 2;
-
- return *kiblnd_tunables.kib_concurrent_sends;
-}
-
-#define IBLND_CONCURRENT_SENDS(v) ((v) == IBLND_MSG_VERSION_1 ? \
- kiblnd_concurrent_sends_v1() : \
- *kiblnd_tunables.kib_concurrent_sends)
/* 2 OOB shall suffice for 1 keepalive and 1 returning credits */
#define IBLND_OOB_CAPABLE(v) ((v) != IBLND_MSG_VERSION_1)
#define IBLND_OOB_MSGS(v) (IBLND_OOB_CAPABLE(v) ? 2 : 0)
#define IBLND_MSG_SIZE (4 << 10) /* max size of queued messages (inc hdr) */
#define IBLND_MAX_RDMA_FRAGS LNET_MAX_IOV /* max # of fragments supported */
-#define IBLND_CFG_RDMA_FRAGS (*kiblnd_tunables.kib_map_on_demand ? \
- *kiblnd_tunables.kib_map_on_demand : \
- IBLND_MAX_RDMA_FRAGS) /* max # of fragments configured by user */
-#define IBLND_RDMA_FRAGS(v) ((v) == IBLND_MSG_VERSION_1 ? \
- IBLND_MAX_RDMA_FRAGS : IBLND_CFG_RDMA_FRAGS)
/************************/
/* derived constants... */
@@ -171,7 +137,8 @@
/* WRs and CQEs (per connection) */
#define IBLND_RECV_WRS(c) IBLND_RX_MSGS(c)
#define IBLND_SEND_WRS(c) \
- ((c->ibc_max_frags + 1) * IBLND_CONCURRENT_SENDS(c->ibc_version))
+ ((c->ibc_max_frags + 1) * kiblnd_concurrent_sends(c->ibc_version, \
+ c->ibc_peer->ibp_ni))
#define IBLND_CQ_ENTRIES(c) (IBLND_RECV_WRS(c) + IBLND_SEND_WRS(c))
struct kib_hca_dev;
@@ -286,24 +253,44 @@
int fps_cpt; /* CPT id */
int fps_pool_size;
int fps_flush_trigger;
+ int fps_cache;
int fps_increasing; /* is allocating new pool */
unsigned long fps_next_retry; /* time stamp for retry if*/
/* failed to allocate */
} kib_fmr_poolset_t;
+struct kib_fast_reg_descriptor { /* For fast registration */
+ struct list_head frd_list;
+ struct ib_send_wr frd_inv_wr;
+ struct ib_reg_wr frd_fastreg_wr;
+ struct ib_mr *frd_mr;
+ bool frd_valid;
+};
+
typedef struct {
struct list_head fpo_list; /* chain on pool list */
struct kib_hca_dev *fpo_hdev; /* device for this pool */
kib_fmr_poolset_t *fpo_owner; /* owner of this pool */
- struct ib_fmr_pool *fpo_fmr_pool; /* IB FMR pool */
+ union {
+ struct {
+ struct ib_fmr_pool *fpo_fmr_pool; /* IB FMR pool */
+ } fmr;
+ struct { /* For fast registration */
+ struct list_head fpo_pool_list;
+ int fpo_pool_size;
+ } fast_reg;
+ };
unsigned long fpo_deadline; /* deadline of this pool */
int fpo_failed; /* fmr pool is failed */
int fpo_map_count; /* # of mapped FMR */
+ int fpo_is_fmr;
} kib_fmr_pool_t;
typedef struct {
- struct ib_pool_fmr *fmr_pfmr; /* IB pool fmr */
- kib_fmr_pool_t *fmr_pool; /* pool of FMR */
+ kib_fmr_pool_t *fmr_pool; /* pool of FMR */
+ struct ib_pool_fmr *fmr_pfmr; /* IB pool fmr */
+ struct kib_fast_reg_descriptor *fmr_frd;
+ u32 fmr_key;
} kib_fmr_t;
typedef struct kib_net {
@@ -615,6 +602,48 @@
void kiblnd_hdev_destroy(kib_hca_dev_t *hdev);
+int kiblnd_msg_queue_size(int version, struct lnet_ni *ni);
+
+/* max # of fragments configured by user */
+static inline int
+kiblnd_cfg_rdma_frags(struct lnet_ni *ni)
+{
+ struct lnet_ioctl_config_o2iblnd_tunables *tunables;
+ int mod;
+
+ tunables = &ni->ni_lnd_tunables->lt_tun_u.lt_o2ib;
+ mod = tunables->lnd_map_on_demand;
+ return mod ? mod : IBLND_MAX_RDMA_FRAGS;
+}
+
+static inline int
+kiblnd_rdma_frags(int version, struct lnet_ni *ni)
+{
+ return version == IBLND_MSG_VERSION_1 ?
+ IBLND_MAX_RDMA_FRAGS :
+ kiblnd_cfg_rdma_frags(ni);
+}
+
+static inline int
+kiblnd_concurrent_sends(int version, struct lnet_ni *ni)
+{
+ struct lnet_ioctl_config_o2iblnd_tunables *tunables;
+ int concurrent_sends;
+
+ tunables = &ni->ni_lnd_tunables->lt_tun_u.lt_o2ib;
+ concurrent_sends = tunables->lnd_concurrent_sends;
+
+ if (version == IBLND_MSG_VERSION_1) {
+ if (concurrent_sends > IBLND_MSG_QUEUE_SIZE_V1 * 2)
+ return IBLND_MSG_QUEUE_SIZE_V1 * 2;
+
+ if (concurrent_sends < IBLND_MSG_QUEUE_SIZE_V1 / 2)
+ return IBLND_MSG_QUEUE_SIZE_V1 / 2;
+ }
+
+ return concurrent_sends;
+}
+
static inline void
kiblnd_hdev_addref_locked(kib_hca_dev_t *hdev)
{
@@ -737,10 +766,14 @@
static inline int
kiblnd_need_noop(kib_conn_t *conn)
{
+ struct lnet_ioctl_config_o2iblnd_tunables *tunables;
+ lnet_ni_t *ni = conn->ibc_peer->ibp_ni;
+
LASSERT(conn->ibc_state >= IBLND_CONN_ESTABLISHED);
+ tunables = &ni->ni_lnd_tunables->lt_tun_u.lt_o2ib;
if (conn->ibc_outstanding_credits <
- IBLND_CREDITS_HIGHWATER(conn->ibc_version) &&
+ IBLND_CREDITS_HIGHWATER(tunables, conn->ibc_version) &&
!kiblnd_send_keepalive(conn))
return 0; /* No need to send NOOP */
@@ -799,7 +832,8 @@
#define IBLND_WID_TX 1
#define IBLND_WID_RX 2
#define IBLND_WID_RDMA 3
-#define IBLND_WID_MASK 3UL
+#define IBLND_WID_MR 4
+#define IBLND_WID_MASK 7UL
static inline __u64
kiblnd_ptr2wreqid(void *ptr, int type)
@@ -947,20 +981,20 @@
#define KIBLND_CONN_PARAM(e) ((e)->param.conn.private_data)
#define KIBLND_CONN_PARAM_LEN(e) ((e)->param.conn.private_data_len)
-struct ib_mr *kiblnd_find_rd_dma_mr(kib_hca_dev_t *hdev,
- kib_rdma_desc_t *rd,
+struct ib_mr *kiblnd_find_rd_dma_mr(struct lnet_ni *ni, kib_rdma_desc_t *rd,
int negotiated_nfrags);
void kiblnd_map_rx_descs(kib_conn_t *conn);
void kiblnd_unmap_rx_descs(kib_conn_t *conn);
void kiblnd_pool_free_node(kib_pool_t *pool, struct list_head *node);
struct list_head *kiblnd_pool_alloc_node(kib_poolset_t *ps);
-int kiblnd_fmr_pool_map(kib_fmr_poolset_t *fps, __u64 *pages,
- int npages, __u64 iov, kib_fmr_t *fmr);
+int kiblnd_fmr_pool_map(kib_fmr_poolset_t *fps, kib_tx_t *tx,
+ kib_rdma_desc_t *rd, __u32 nob, __u64 iov,
+ kib_fmr_t *fmr);
void kiblnd_fmr_pool_unmap(kib_fmr_t *fmr, int status);
-int kiblnd_tunables_init(void);
-void kiblnd_tunables_fini(void);
+int kiblnd_tunables_setup(struct lnet_ni *ni);
+void kiblnd_tunables_init(void);
int kiblnd_connd(void *arg);
int kiblnd_scheduler(void *arg);
diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
index 2323e8d..bbfee53 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c
@@ -561,36 +561,23 @@
}
static int
-kiblnd_fmr_map_tx(kib_net_t *net, kib_tx_t *tx, kib_rdma_desc_t *rd, int nob)
+kiblnd_fmr_map_tx(kib_net_t *net, kib_tx_t *tx, kib_rdma_desc_t *rd, __u32 nob)
{
kib_hca_dev_t *hdev;
- __u64 *pages = tx->tx_pages;
kib_fmr_poolset_t *fps;
- int npages;
- int size;
int cpt;
int rc;
- int i;
LASSERT(tx->tx_pool);
LASSERT(tx->tx_pool->tpo_pool.po_owner);
hdev = tx->tx_pool->tpo_hdev;
-
- for (i = 0, npages = 0; i < rd->rd_nfrags; i++) {
- for (size = 0; size < rd->rd_frags[i].rf_nob;
- size += hdev->ibh_page_size) {
- pages[npages++] = (rd->rd_frags[i].rf_addr &
- hdev->ibh_page_mask) + size;
- }
- }
-
cpt = tx->tx_pool->tpo_pool.po_owner->ps_cpt;
fps = net->ibn_fmr_ps[cpt];
- rc = kiblnd_fmr_pool_map(fps, pages, npages, 0, &tx->fmr);
+ rc = kiblnd_fmr_pool_map(fps, tx, rd, nob, 0, &tx->fmr);
if (rc) {
- CERROR("Can't map %d pages: %d\n", npages, rc);
+ CERROR("Can't map %u bytes: %d\n", nob, rc);
return rc;
}
@@ -598,8 +585,7 @@
* If rd is not tx_rd, it's going to get sent to a peer, who will need
* the rkey
*/
- rd->rd_key = (rd != tx->tx_rd) ? tx->fmr.fmr_pfmr->fmr->rkey :
- tx->fmr.fmr_pfmr->fmr->lkey;
+ rd->rd_key = tx->fmr.fmr_key;
rd->rd_frags[0].rf_addr &= ~hdev->ibh_page_mask;
rd->rd_frags[0].rf_nob = nob;
rd->rd_nfrags = 1;
@@ -613,10 +599,8 @@
LASSERT(net);
- if (net->ibn_fmr_ps && tx->fmr.fmr_pfmr) {
+ if (net->ibn_fmr_ps)
kiblnd_fmr_pool_unmap(&tx->fmr, tx->tx_status);
- tx->fmr.fmr_pfmr = NULL;
- }
if (tx->tx_nfrags) {
kiblnd_dma_unmap_sg(tx->tx_pool->tpo_hdev->ibh_ibdev,
@@ -628,8 +612,8 @@
static int kiblnd_map_tx(lnet_ni_t *ni, kib_tx_t *tx, kib_rdma_desc_t *rd,
int nfrags)
{
- kib_hca_dev_t *hdev = tx->tx_pool->tpo_hdev;
kib_net_t *net = ni->ni_data;
+ kib_hca_dev_t *hdev = net->ibn_dev->ibd_hdev;
struct ib_mr *mr = NULL;
__u32 nob;
int i;
@@ -652,7 +636,7 @@
nob += rd->rd_frags[i].rf_nob;
}
- mr = kiblnd_find_rd_dma_mr(hdev, rd, tx->tx_conn ?
+ mr = kiblnd_find_rd_dma_mr(ni, rd, tx->tx_conn ?
tx->tx_conn->ibc_max_frags : -1);
if (mr) {
/* found pre-mapping MR */
@@ -704,7 +688,7 @@
fragnob = min(fragnob, (int)PAGE_SIZE - page_offset);
sg_set_page(sg, page, fragnob, page_offset);
- sg++;
+ sg = sg_next(sg);
if (offset + fragnob < iov->iov_len) {
offset += fragnob;
@@ -748,7 +732,7 @@
sg_set_page(sg, kiov->kiov_page, fragnob,
kiov->kiov_offset + offset);
- sg++;
+ sg = sg_next(sg);
offset = 0;
kiov++;
@@ -765,6 +749,7 @@
{
kib_msg_t *msg = tx->tx_msg;
kib_peer_t *peer = conn->ibc_peer;
+ struct lnet_ni *ni = peer->ibp_ni;
int ver = conn->ibc_version;
int rc;
int done;
@@ -780,7 +765,7 @@
LASSERT(conn->ibc_credits >= 0);
LASSERT(conn->ibc_credits <= conn->ibc_queue_depth);
- if (conn->ibc_nsends_posted == IBLND_CONCURRENT_SENDS(ver)) {
+ if (conn->ibc_nsends_posted == kiblnd_concurrent_sends(ver, ni)) {
/* tx completions outstanding... */
CDEBUG(D_NET, "%s: posted enough\n",
libcfs_nid2str(peer->ibp_nid));
@@ -851,14 +836,26 @@
/* close_conn will launch failover */
rc = -ENETDOWN;
} else {
- struct ib_send_wr *wrq = &tx->tx_wrq[tx->tx_nwrq - 1].wr;
+ struct kib_fast_reg_descriptor *frd = tx->fmr.fmr_frd;
+ struct ib_send_wr *bad = &tx->tx_wrq[tx->tx_nwrq - 1].wr;
+ struct ib_send_wr *wrq = &tx->tx_wrq[0].wr;
- LASSERTF(wrq->wr_id == kiblnd_ptr2wreqid(tx, IBLND_WID_TX),
+ if (frd) {
+ if (!frd->frd_valid) {
+ wrq = &frd->frd_inv_wr;
+ wrq->next = &frd->frd_fastreg_wr.wr;
+ } else {
+ wrq = &frd->frd_fastreg_wr.wr;
+ }
+ frd->frd_fastreg_wr.wr.next = &tx->tx_wrq[0].wr;
+ }
+
+ LASSERTF(bad->wr_id == kiblnd_ptr2wreqid(tx, IBLND_WID_TX),
"bad wr_id %llx, opc %d, flags %d, peer: %s\n",
- wrq->wr_id, wrq->opcode, wrq->send_flags,
- libcfs_nid2str(conn->ibc_peer->ibp_nid));
- wrq = NULL;
- rc = ib_post_send(conn->ibc_cmid->qp, &tx->tx_wrq->wr, &wrq);
+ bad->wr_id, bad->opcode, bad->send_flags,
+ libcfs_nid2str(conn->ibc_peer->ibp_nid));
+ bad = NULL;
+ rc = ib_post_send(conn->ibc_cmid->qp, wrq, &bad);
}
conn->ibc_last_send = jiffies;
@@ -919,7 +916,7 @@
spin_lock(&conn->ibc_lock);
- LASSERT(conn->ibc_nsends_posted <= IBLND_CONCURRENT_SENDS(ver));
+ LASSERT(conn->ibc_nsends_posted <= kiblnd_concurrent_sends(ver, ni));
LASSERT(!IBLND_OOB_CAPABLE(ver) ||
conn->ibc_noops_posted <= IBLND_OOB_MSGS(ver));
LASSERT(conn->ibc_reserved_credits >= 0);
@@ -1066,7 +1063,7 @@
kib_msg_t *ibmsg = tx->tx_msg;
kib_rdma_desc_t *srcrd = tx->tx_rd;
struct ib_sge *sge = &tx->tx_sge[0];
- struct ib_rdma_wr *wrq = &tx->tx_wrq[0], *next;
+ struct ib_rdma_wr *wrq, *next;
int rc = resid;
int srcidx = 0;
int dstidx = 0;
@@ -2333,11 +2330,11 @@
}
if (reqmsg->ibm_u.connparams.ibcp_queue_depth >
- IBLND_MSG_QUEUE_SIZE(version)) {
+ kiblnd_msg_queue_size(version, ni)) {
CERROR("Can't accept conn from %s, queue depth too large: %d (<=%d wanted)\n",
libcfs_nid2str(nid),
reqmsg->ibm_u.connparams.ibcp_queue_depth,
- IBLND_MSG_QUEUE_SIZE(version));
+ kiblnd_msg_queue_size(version, ni));
if (version == IBLND_MSG_VERSION)
rej.ibr_why = IBLND_REJECT_MSG_QUEUE_SIZE;
@@ -2346,24 +2343,24 @@
}
if (reqmsg->ibm_u.connparams.ibcp_max_frags >
- IBLND_RDMA_FRAGS(version)) {
+ kiblnd_rdma_frags(version, ni)) {
CWARN("Can't accept conn from %s (version %x): max_frags %d too large (%d wanted)\n",
libcfs_nid2str(nid), version,
reqmsg->ibm_u.connparams.ibcp_max_frags,
- IBLND_RDMA_FRAGS(version));
+ kiblnd_rdma_frags(version, ni));
if (version >= IBLND_MSG_VERSION)
rej.ibr_why = IBLND_REJECT_RDMA_FRAGS;
goto failed;
} else if (reqmsg->ibm_u.connparams.ibcp_max_frags <
- IBLND_RDMA_FRAGS(version) && !net->ibn_fmr_ps) {
+ kiblnd_rdma_frags(version, ni) && !net->ibn_fmr_ps) {
CWARN("Can't accept conn from %s (version %x): max_frags %d incompatible without FMR pool (%d wanted)\n",
libcfs_nid2str(nid), version,
reqmsg->ibm_u.connparams.ibcp_max_frags,
- IBLND_RDMA_FRAGS(version));
+ kiblnd_rdma_frags(version, ni));
- if (version >= IBLND_MSG_VERSION)
+ if (version == IBLND_MSG_VERSION)
rej.ibr_why = IBLND_REJECT_RDMA_FRAGS;
goto failed;
@@ -2528,8 +2525,8 @@
lnet_ni_decref(ni);
rej.ibr_version = version;
- rej.ibr_cp.ibcp_queue_depth = IBLND_MSG_QUEUE_SIZE(version);
- rej.ibr_cp.ibcp_max_frags = IBLND_RDMA_FRAGS(version);
+ rej.ibr_cp.ibcp_queue_depth = kiblnd_msg_queue_size(version, ni);
+ rej.ibr_cp.ibcp_max_frags = kiblnd_rdma_frags(version, ni);
kiblnd_reject(cmid, &rej);
return -ECONNREFUSED;
@@ -2580,12 +2577,15 @@
reason = "Unknown";
break;
- case IBLND_REJECT_RDMA_FRAGS:
+ case IBLND_REJECT_RDMA_FRAGS: {
+ struct lnet_ioctl_config_lnd_tunables *tunables;
+
if (!cp) {
reason = "can't negotiate max frags";
goto out;
}
- if (!*kiblnd_tunables.kib_map_on_demand) {
+ tunables = peer->ibp_ni->ni_lnd_tunables;
+ if (!tunables->lt_tun_u.lt_o2ib.lnd_map_on_demand) {
reason = "map_on_demand must be enabled";
goto out;
}
@@ -2597,7 +2597,7 @@
peer->ibp_max_frags = frag_num;
reason = "rdma fragments";
break;
-
+ }
case IBLND_REJECT_MSG_QUEUE_SIZE:
if (!cp) {
reason = "can't negotiate queue depth";
@@ -3430,6 +3430,12 @@
default:
LBUG();
+ case IBLND_WID_MR:
+ if (wc->status != IB_WC_SUCCESS &&
+ wc->status != IB_WC_WR_FLUSH_ERR)
+ CNETERR("FastReg failed: %d\n", wc->status);
+ break;
+
case IBLND_WID_RDMA:
/*
* We only get RDMA completion notification if it fails. All
diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c
index b4607da..f8fdd4a 100644
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c
@@ -152,74 +152,135 @@
.kib_timeout = &timeout,
.kib_keepalive = &keepalive,
.kib_ntx = &ntx,
- .kib_credits = &credits,
- .kib_peertxcredits = &peer_credits,
- .kib_peercredits_hiw = &peer_credits_hiw,
- .kib_peerrtrcredits = &peer_buffer_credits,
- .kib_peertimeout = &peer_timeout,
.kib_default_ipif = &ipif_name,
.kib_retry_count = &retry_count,
.kib_rnr_retry_count = &rnr_retry_count,
- .kib_concurrent_sends = &concurrent_sends,
.kib_ib_mtu = &ib_mtu,
- .kib_map_on_demand = &map_on_demand,
- .kib_fmr_pool_size = &fmr_pool_size,
- .kib_fmr_flush_trigger = &fmr_flush_trigger,
- .kib_fmr_cache = &fmr_cache,
.kib_require_priv_port = &require_privileged_port,
.kib_use_priv_port = &use_privileged_port,
.kib_nscheds = &nscheds
};
-int
-kiblnd_tunables_init(void)
+static struct lnet_ioctl_config_o2iblnd_tunables default_tunables;
+
+/* # messages/RDMAs in-flight */
+int kiblnd_msg_queue_size(int version, lnet_ni_t *ni)
{
+ if (version == IBLND_MSG_VERSION_1)
+ return IBLND_MSG_QUEUE_SIZE_V1;
+ else if (ni)
+ return ni->ni_peertxcredits;
+ else
+ return peer_credits;
+}
+
+int kiblnd_tunables_setup(struct lnet_ni *ni)
+{
+ struct lnet_ioctl_config_o2iblnd_tunables *tunables;
+
+ /*
+ * if there was no tunables specified, setup the tunables to be
+ * defaulted
+ */
+ if (!ni->ni_lnd_tunables) {
+ LIBCFS_ALLOC(ni->ni_lnd_tunables,
+ sizeof(*ni->ni_lnd_tunables));
+ if (!ni->ni_lnd_tunables)
+ return -ENOMEM;
+
+ memcpy(&ni->ni_lnd_tunables->lt_tun_u.lt_o2ib,
+ &default_tunables, sizeof(*tunables));
+ }
+ tunables = &ni->ni_lnd_tunables->lt_tun_u.lt_o2ib;
+
+ /* Current API version */
+ tunables->lnd_version = 0;
+
if (kiblnd_translate_mtu(*kiblnd_tunables.kib_ib_mtu) < 0) {
CERROR("Invalid ib_mtu %d, expected 256/512/1024/2048/4096\n",
*kiblnd_tunables.kib_ib_mtu);
return -EINVAL;
}
- if (*kiblnd_tunables.kib_peertxcredits < IBLND_CREDITS_DEFAULT)
- *kiblnd_tunables.kib_peertxcredits = IBLND_CREDITS_DEFAULT;
+ if (!ni->ni_peertimeout)
+ ni->ni_peertimeout = peer_timeout;
- if (*kiblnd_tunables.kib_peertxcredits > IBLND_CREDITS_MAX)
- *kiblnd_tunables.kib_peertxcredits = IBLND_CREDITS_MAX;
+ if (!ni->ni_maxtxcredits)
+ ni->ni_maxtxcredits = credits;
- if (*kiblnd_tunables.kib_peertxcredits > *kiblnd_tunables.kib_credits)
- *kiblnd_tunables.kib_peertxcredits = *kiblnd_tunables.kib_credits;
+ if (!ni->ni_peertxcredits)
+ ni->ni_peertxcredits = peer_credits;
- if (*kiblnd_tunables.kib_peercredits_hiw < *kiblnd_tunables.kib_peertxcredits / 2)
- *kiblnd_tunables.kib_peercredits_hiw = *kiblnd_tunables.kib_peertxcredits / 2;
+ if (!ni->ni_peerrtrcredits)
+ ni->ni_peerrtrcredits = peer_buffer_credits;
- if (*kiblnd_tunables.kib_peercredits_hiw >= *kiblnd_tunables.kib_peertxcredits)
- *kiblnd_tunables.kib_peercredits_hiw = *kiblnd_tunables.kib_peertxcredits - 1;
+ if (ni->ni_peertxcredits < IBLND_CREDITS_DEFAULT)
+ ni->ni_peertxcredits = IBLND_CREDITS_DEFAULT;
- if (*kiblnd_tunables.kib_map_on_demand < 0 ||
- *kiblnd_tunables.kib_map_on_demand > IBLND_MAX_RDMA_FRAGS)
- *kiblnd_tunables.kib_map_on_demand = 0; /* disable map-on-demand */
+ if (ni->ni_peertxcredits > IBLND_CREDITS_MAX)
+ ni->ni_peertxcredits = IBLND_CREDITS_MAX;
- if (*kiblnd_tunables.kib_map_on_demand == 1)
- *kiblnd_tunables.kib_map_on_demand = 2; /* don't make sense to create map if only one fragment */
+ if (ni->ni_peertxcredits > credits)
+ ni->ni_peertxcredits = credits;
- if (!*kiblnd_tunables.kib_concurrent_sends) {
- if (*kiblnd_tunables.kib_map_on_demand > 0 &&
- *kiblnd_tunables.kib_map_on_demand <= IBLND_MAX_RDMA_FRAGS / 8)
- *kiblnd_tunables.kib_concurrent_sends = (*kiblnd_tunables.kib_peertxcredits) * 2;
- else
- *kiblnd_tunables.kib_concurrent_sends = (*kiblnd_tunables.kib_peertxcredits);
+ if (!tunables->lnd_peercredits_hiw)
+ tunables->lnd_peercredits_hiw = peer_credits_hiw;
+
+ if (tunables->lnd_peercredits_hiw < ni->ni_peertxcredits / 2)
+ tunables->lnd_peercredits_hiw = ni->ni_peertxcredits / 2;
+
+ if (tunables->lnd_peercredits_hiw >= ni->ni_peertxcredits)
+ tunables->lnd_peercredits_hiw = ni->ni_peertxcredits - 1;
+
+ if (tunables->lnd_map_on_demand < 0 ||
+ tunables->lnd_map_on_demand > IBLND_MAX_RDMA_FRAGS) {
+ /* disable map-on-demand */
+ tunables->lnd_map_on_demand = 0;
}
- if (*kiblnd_tunables.kib_concurrent_sends > *kiblnd_tunables.kib_peertxcredits * 2)
- *kiblnd_tunables.kib_concurrent_sends = *kiblnd_tunables.kib_peertxcredits * 2;
+ if (tunables->lnd_map_on_demand == 1) {
+ /* don't make sense to create map if only one fragment */
+ tunables->lnd_map_on_demand = 2;
+ }
- if (*kiblnd_tunables.kib_concurrent_sends < *kiblnd_tunables.kib_peertxcredits / 2)
- *kiblnd_tunables.kib_concurrent_sends = *kiblnd_tunables.kib_peertxcredits / 2;
+ if (!tunables->lnd_concurrent_sends) {
+ if (tunables->lnd_map_on_demand > 0 &&
+ tunables->lnd_map_on_demand <= IBLND_MAX_RDMA_FRAGS / 8) {
+ tunables->lnd_concurrent_sends =
+ ni->ni_peertxcredits * 2;
+ } else {
+ tunables->lnd_concurrent_sends = ni->ni_peertxcredits;
+ }
+ }
- if (*kiblnd_tunables.kib_concurrent_sends < *kiblnd_tunables.kib_peertxcredits) {
+ if (tunables->lnd_concurrent_sends > ni->ni_peertxcredits * 2)
+ tunables->lnd_concurrent_sends = ni->ni_peertxcredits * 2;
+
+ if (tunables->lnd_concurrent_sends < ni->ni_peertxcredits / 2)
+ tunables->lnd_concurrent_sends = ni->ni_peertxcredits / 2;
+
+ if (tunables->lnd_concurrent_sends < ni->ni_peertxcredits) {
CWARN("Concurrent sends %d is lower than message queue size: %d, performance may drop slightly.\n",
- *kiblnd_tunables.kib_concurrent_sends, *kiblnd_tunables.kib_peertxcredits);
+ tunables->lnd_concurrent_sends, ni->ni_peertxcredits);
}
+ if (!tunables->lnd_fmr_pool_size)
+ tunables->lnd_fmr_pool_size = fmr_pool_size;
+ if (!tunables->lnd_fmr_flush_trigger)
+ tunables->lnd_fmr_flush_trigger = fmr_flush_trigger;
+ if (!tunables->lnd_fmr_cache)
+ tunables->lnd_fmr_cache = fmr_cache;
+
return 0;
}
+
+void kiblnd_tunables_init(void)
+{
+ default_tunables.lnd_version = 0;
+ default_tunables.lnd_peercredits_hiw = peer_credits_hiw,
+ default_tunables.lnd_map_on_demand = map_on_demand;
+ default_tunables.lnd_concurrent_sends = concurrent_sends;
+ default_tunables.lnd_fmr_pool_size = fmr_pool_size;
+ default_tunables.lnd_fmr_flush_trigger = fmr_flush_trigger;
+ default_tunables.lnd_fmr_cache = fmr_cache;
+}
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
index cca7b2f..406c0e7 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c
@@ -2582,7 +2582,6 @@
}
read_unlock(&ksocknal_data.ksnd_global_lock);
- return;
}
void
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c
index d4ce06d..964b4e3 100644
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c
@@ -675,7 +675,6 @@
sock->sk->sk_user_data = conn;
sock->sk->sk_data_ready = ksocknal_data_ready;
sock->sk->sk_write_space = ksocknal_write_space;
- return;
}
void
@@ -695,8 +694,6 @@
* sk_user_data is NULL.
*/
sock->sk->sk_user_data = NULL;
-
- return ;
}
int
diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c
index c3d628b..8c260c3 100644
--- a/drivers/staging/lustre/lnet/libcfs/debug.c
+++ b/drivers/staging/lustre/lnet/libcfs/debug.c
@@ -232,130 +232,24 @@
static const char *
libcfs_debug_subsys2str(int subsys)
{
- switch (1 << subsys) {
- default:
+ static const char *libcfs_debug_subsystems[] = LIBCFS_DEBUG_SUBSYS_NAMES;
+
+ if (subsys >= ARRAY_SIZE(libcfs_debug_subsystems))
return NULL;
- case S_UNDEFINED:
- return "undefined";
- case S_MDC:
- return "mdc";
- case S_MDS:
- return "mds";
- case S_OSC:
- return "osc";
- case S_OST:
- return "ost";
- case S_CLASS:
- return "class";
- case S_LOG:
- return "log";
- case S_LLITE:
- return "llite";
- case S_RPC:
- return "rpc";
- case S_LNET:
- return "lnet";
- case S_LND:
- return "lnd";
- case S_PINGER:
- return "pinger";
- case S_FILTER:
- return "filter";
- case S_ECHO:
- return "echo";
- case S_LDLM:
- return "ldlm";
- case S_LOV:
- return "lov";
- case S_LQUOTA:
- return "lquota";
- case S_OSD:
- return "osd";
- case S_LFSCK:
- return "lfsck";
- case S_LMV:
- return "lmv";
- case S_SEC:
- return "sec";
- case S_GSS:
- return "gss";
- case S_MGC:
- return "mgc";
- case S_MGS:
- return "mgs";
- case S_FID:
- return "fid";
- case S_FLD:
- return "fld";
- }
+
+ return libcfs_debug_subsystems[subsys];
}
/* libcfs_debug_token2mask() expects the returned string in lower-case */
static const char *
libcfs_debug_dbg2str(int debug)
{
- switch (1 << debug) {
- default:
+ static const char *libcfs_debug_masks[] = LIBCFS_DEBUG_MASKS_NAMES;
+
+ if (debug >= ARRAY_SIZE(libcfs_debug_masks))
return NULL;
- case D_TRACE:
- return "trace";
- case D_INODE:
- return "inode";
- case D_SUPER:
- return "super";
- case D_EXT2:
- return "ext2";
- case D_MALLOC:
- return "malloc";
- case D_CACHE:
- return "cache";
- case D_INFO:
- return "info";
- case D_IOCTL:
- return "ioctl";
- case D_NETERROR:
- return "neterror";
- case D_NET:
- return "net";
- case D_WARNING:
- return "warning";
- case D_BUFFS:
- return "buffs";
- case D_OTHER:
- return "other";
- case D_DENTRY:
- return "dentry";
- case D_NETTRACE:
- return "nettrace";
- case D_PAGE:
- return "page";
- case D_DLMTRACE:
- return "dlmtrace";
- case D_ERROR:
- return "error";
- case D_EMERG:
- return "emerg";
- case D_HA:
- return "ha";
- case D_RPCTRACE:
- return "rpctrace";
- case D_VFSTRACE:
- return "vfstrace";
- case D_READA:
- return "reada";
- case D_MMAP:
- return "mmap";
- case D_CONFIG:
- return "config";
- case D_CONSOLE:
- return "console";
- case D_QUOTA:
- return "quota";
- case D_SEC:
- return "sec";
- case D_LFSCK:
- return "lfsck";
- }
+
+ return libcfs_debug_masks[debug];
}
int
diff --git a/drivers/staging/lustre/lnet/libcfs/fail.c b/drivers/staging/lustre/lnet/libcfs/fail.c
index dadaf76..086e690 100644
--- a/drivers/staging/lustre/lnet/libcfs/fail.c
+++ b/drivers/staging/lustre/lnet/libcfs/fail.c
@@ -41,6 +41,9 @@
unsigned int cfs_fail_val;
EXPORT_SYMBOL(cfs_fail_val);
+int cfs_fail_err;
+EXPORT_SYMBOL(cfs_fail_err);
+
DECLARE_WAIT_QUEUE_HEAD(cfs_race_waitq);
EXPORT_SYMBOL(cfs_race_waitq);
diff --git a/drivers/staging/lustre/lnet/libcfs/hash.c b/drivers/staging/lustre/lnet/libcfs/hash.c
index f60feb3..cc45ed8 100644
--- a/drivers/staging/lustre/lnet/libcfs/hash.c
+++ b/drivers/staging/lustre/lnet/libcfs/hash.c
@@ -942,10 +942,10 @@
* @flags - CFS_HASH_REHASH enable synamic hash resizing
* - CFS_HASH_SORT enable chained hash sort
*/
-static int cfs_hash_rehash_worker(cfs_workitem_t *wi);
+static int cfs_hash_rehash_worker(struct cfs_workitem *wi);
#if CFS_HASH_DEBUG_LEVEL >= CFS_HASH_DEBUG_1
-static int cfs_hash_dep_print(cfs_workitem_t *wi)
+static int cfs_hash_dep_print(struct cfs_workitem *wi)
{
struct cfs_hash *hs = container_of(wi, struct cfs_hash, hs_dep_wi);
int dep;
@@ -1847,7 +1847,7 @@
}
static int
-cfs_hash_rehash_worker(cfs_workitem_t *wi)
+cfs_hash_rehash_worker(struct cfs_workitem *wi)
{
struct cfs_hash *hs = container_of(wi, struct cfs_hash, hs_rehash_wi);
struct cfs_hash_bucket **bkts;
diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c b/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c
index 2de9eea..83543f9 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c
@@ -49,7 +49,8 @@
* reason we always allocate cacheline-aligned memory block.
*/
struct cfs_percpt_lock *
-cfs_percpt_lock_alloc(struct cfs_cpt_table *cptab)
+cfs_percpt_lock_create(struct cfs_cpt_table *cptab,
+ struct lock_class_key *keys)
{
struct cfs_percpt_lock *pcl;
spinlock_t *lock;
@@ -67,12 +68,18 @@
return NULL;
}
- cfs_percpt_for_each(lock, i, pcl->pcl_locks)
+ if (!keys)
+ CWARN("Cannot setup class key for percpt lock, you may see recursive locking warnings which are actually fake.\n");
+
+ cfs_percpt_for_each(lock, i, pcl->pcl_locks) {
spin_lock_init(lock);
+ if (keys != NULL)
+ lockdep_set_class(lock, &keys[i]);
+ }
return pcl;
}
-EXPORT_SYMBOL(cfs_percpt_lock_alloc);
+EXPORT_SYMBOL(cfs_percpt_lock_create);
/**
* lock a CPU partition
@@ -142,44 +149,3 @@
}
}
EXPORT_SYMBOL(cfs_percpt_unlock);
-
-/** free cpu-partition refcount */
-void
-cfs_percpt_atomic_free(atomic_t **refs)
-{
- cfs_percpt_free(refs);
-}
-EXPORT_SYMBOL(cfs_percpt_atomic_free);
-
-/** allocate cpu-partition refcount with initial value @init_val */
-atomic_t **
-cfs_percpt_atomic_alloc(struct cfs_cpt_table *cptab, int init_val)
-{
- atomic_t **refs;
- atomic_t *ref;
- int i;
-
- refs = cfs_percpt_alloc(cptab, sizeof(*ref));
- if (!refs)
- return NULL;
-
- cfs_percpt_for_each(ref, i, refs)
- atomic_set(ref, init_val);
- return refs;
-}
-EXPORT_SYMBOL(cfs_percpt_atomic_alloc);
-
-/** return sum of cpu-partition refs */
-int
-cfs_percpt_atomic_summary(atomic_t **refs)
-{
- atomic_t *ref;
- int i;
- int val = 0;
-
- cfs_percpt_for_each(ref, i, refs)
- val += atomic_read(ref);
-
- return val;
-}
-EXPORT_SYMBOL(cfs_percpt_atomic_summary);
diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_mem.c b/drivers/staging/lustre/lnet/libcfs/libcfs_mem.c
index c5a6951..d0e81bb 100644
--- a/drivers/staging/lustre/lnet/libcfs/libcfs_mem.c
+++ b/drivers/staging/lustre/lnet/libcfs/libcfs_mem.c
@@ -115,34 +115,6 @@
EXPORT_SYMBOL(cfs_percpt_number);
/*
- * return memory block shadowed from current CPU
- */
-void *
-cfs_percpt_current(void *vars)
-{
- struct cfs_var_array *arr;
- int cpt;
-
- arr = container_of(vars, struct cfs_var_array, va_ptrs[0]);
- cpt = cfs_cpt_current(arr->va_cptab, 0);
- if (cpt < 0)
- return NULL;
-
- return arr->va_ptrs[cpt];
-}
-
-void *
-cfs_percpt_index(void *vars, int idx)
-{
- struct cfs_var_array *arr;
-
- arr = container_of(vars, struct cfs_var_array, va_ptrs[0]);
-
- LASSERT(idx >= 0 && idx < arr->va_count);
- return arr->va_ptrs[idx];
-}
-
-/*
* free variable array, see more detail in cfs_array_alloc
*/
void
diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c
index 389fb9e..b52518c5 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c
@@ -755,8 +755,13 @@
struct cfs_cpu_partition *part;
int n;
- if (cpt >= ncpt)
- goto failed;
+ /*
+ * Each emulated NUMA node has all allowed CPUs in
+ * the mask.
+ * End loop when all partitions have assigned CPUs.
+ */
+ if (cpt == ncpt)
+ break;
part = &cptab->ctb_parts[cpt];
diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c
index 8c9377e..84f9b7b 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c
@@ -30,13 +30,34 @@
#include <crypto/hash.h>
#include <linux/scatterlist.h>
#include "../../../include/linux/libcfs/libcfs.h"
+#include "../../../include/linux/libcfs/libcfs_crypto.h"
#include "linux-crypto.h"
+
/**
- * Array of hash algorithm speed in MByte per second
+ * Array of hash algorithm speed in MByte per second
*/
static int cfs_crypto_hash_speeds[CFS_HASH_ALG_MAX];
-static int cfs_crypto_hash_alloc(unsigned char alg_id,
+/**
+ * Initialize the state descriptor for the specified hash algorithm.
+ *
+ * An internal routine to allocate the hash-specific state in \a hdesc for
+ * use with cfs_crypto_hash_digest() to compute the hash of a single message,
+ * though possibly in multiple chunks. The descriptor internal state should
+ * be freed with cfs_crypto_hash_final().
+ *
+ * \param[in] hash_alg hash algorithm id (CFS_HASH_ALG_*)
+ * \param[out] type pointer to the hash description in hash_types[]
+ * array
+ * \param[in,out] hdesc hash state descriptor to be initialized
+ * \param[in] key initial hash value/state, NULL to use default
+ * value
+ * \param[in] key_len length of \a key
+ *
+ * \retval 0 on success
+ * \retval negative errno on failure
+ */
+static int cfs_crypto_hash_alloc(enum cfs_crypto_hash_alg hash_alg,
const struct cfs_crypto_hash_type **type,
struct ahash_request **req,
unsigned char *key,
@@ -45,11 +66,11 @@
struct crypto_ahash *tfm;
int err = 0;
- *type = cfs_crypto_hash_type(alg_id);
+ *type = cfs_crypto_hash_type(hash_alg);
if (!*type) {
CWARN("Unsupported hash algorithm id = %d, max id is %d\n",
- alg_id, CFS_HASH_ALG_MAX);
+ hash_alg, CFS_HASH_ALG_MAX);
return -EINVAL;
}
tfm = crypto_alloc_ahash((*type)->cht_name, 0, CRYPTO_ALG_ASYNC);
@@ -70,12 +91,6 @@
ahash_request_set_callback(*req, 0, NULL, NULL);
- /** Shash have different logic for initialization then digest
- * shash: crypto_hash_setkey, crypto_hash_init
- * digest: crypto_digest_init, crypto_digest_setkey
- * Skip this function for digest, because we use shash logic at
- * cfs_crypto_hash_alloc.
- */
if (key)
err = crypto_ahash_setkey(tfm, key, key_len);
else if ((*type)->cht_key != 0)
@@ -90,7 +105,7 @@
CDEBUG(D_INFO, "Using crypto hash: %s (%s) speed %d MB/s\n",
crypto_ahash_alg_name(tfm), crypto_ahash_driver_name(tfm),
- cfs_crypto_hash_speeds[alg_id]);
+ cfs_crypto_hash_speeds[hash_alg]);
err = crypto_ahash_init(*req);
if (err) {
@@ -100,7 +115,33 @@
return err;
}
-int cfs_crypto_hash_digest(unsigned char alg_id,
+/**
+ * Calculate hash digest for the passed buffer.
+ *
+ * This should be used when computing the hash on a single contiguous buffer.
+ * It combines the hash initialization, computation, and cleanup.
+ *
+ * \param[in] hash_alg id of hash algorithm (CFS_HASH_ALG_*)
+ * \param[in] buf data buffer on which to compute hash
+ * \param[in] buf_len length of \a buf in bytes
+ * \param[in] key initial value/state for algorithm,
+ * if \a key = NULL use default initial value
+ * \param[in] key_len length of \a key in bytes
+ * \param[out] hash pointer to computed hash value,
+ * if \a hash = NULL then \a hash_len is to digest
+ * size in bytes, retval -ENOSPC
+ * \param[in,out] hash_len size of \a hash buffer
+ *
+ * \retval -EINVAL \a buf, \a buf_len, \a hash_len,
+ * \a hash_alg invalid
+ * \retval -ENOENT \a hash_alg is unsupported
+ * \retval -ENOSPC \a hash is NULL, or \a hash_len less than
+ * digest size
+ * \retval 0 for success
+ * \retval negative errno for other errors from lower
+ * layers.
+ */
+int cfs_crypto_hash_digest(enum cfs_crypto_hash_alg hash_alg,
const void *buf, unsigned int buf_len,
unsigned char *key, unsigned int key_len,
unsigned char *hash, unsigned int *hash_len)
@@ -113,7 +154,7 @@
if (!buf || buf_len == 0 || !hash_len)
return -EINVAL;
- err = cfs_crypto_hash_alloc(alg_id, &type, &req, key, key_len);
+ err = cfs_crypto_hash_alloc(hash_alg, &type, &req, key, key_len);
if (err != 0)
return err;
@@ -134,15 +175,32 @@
}
EXPORT_SYMBOL(cfs_crypto_hash_digest);
+/**
+ * Allocate and initialize desriptor for hash algorithm.
+ *
+ * This should be used to initialize a hash descriptor for multiple calls
+ * to a single hash function when computing the hash across multiple
+ * separate buffers or pages using cfs_crypto_hash_update{,_page}().
+ *
+ * The hash descriptor should be freed with cfs_crypto_hash_final().
+ *
+ * \param[in] hash_alg algorithm id (CFS_HASH_ALG_*)
+ * \param[in] key initial value/state for algorithm, if \a key = NULL
+ * use default initial value
+ * \param[in] key_len length of \a key in bytes
+ *
+ * \retval pointer to descriptor of hash instance
+ * \retval ERR_PTR(errno) in case of error
+ */
struct cfs_crypto_hash_desc *
- cfs_crypto_hash_init(unsigned char alg_id,
- unsigned char *key, unsigned int key_len)
+cfs_crypto_hash_init(enum cfs_crypto_hash_alg hash_alg,
+ unsigned char *key, unsigned int key_len)
{
struct ahash_request *req;
int err;
const struct cfs_crypto_hash_type *type;
- err = cfs_crypto_hash_alloc(alg_id, &type, &req, key, key_len);
+ err = cfs_crypto_hash_alloc(hash_alg, &type, &req, key, key_len);
if (err)
return ERR_PTR(err);
@@ -150,6 +208,17 @@
}
EXPORT_SYMBOL(cfs_crypto_hash_init);
+/**
+ * Update hash digest computed on data within the given \a page
+ *
+ * \param[in] hdesc hash state descriptor
+ * \param[in] page data page on which to compute the hash
+ * \param[in] offset offset within \a page at which to start hash
+ * \param[in] len length of data on which to compute hash
+ *
+ * \retval 0 for success
+ * \retval negative errno on failure
+ */
int cfs_crypto_hash_update_page(struct cfs_crypto_hash_desc *hdesc,
struct page *page, unsigned int offset,
unsigned int len)
@@ -158,13 +227,23 @@
struct scatterlist sl;
sg_init_table(&sl, 1);
- sg_set_page(&sl, page, len, offset & ~CFS_PAGE_MASK);
+ sg_set_page(&sl, page, len, offset & ~PAGE_MASK);
ahash_request_set_crypt(req, &sl, NULL, sl.length);
return crypto_ahash_update(req);
}
EXPORT_SYMBOL(cfs_crypto_hash_update_page);
+/**
+ * Update hash digest computed on the specified data
+ *
+ * \param[in] hdesc hash state descriptor
+ * \param[in] buf data buffer on which to compute the hash
+ * \param[in] buf_len length of \buf on which to compute hash
+ *
+ * \retval 0 for success
+ * \retval negative errno on failure
+ */
int cfs_crypto_hash_update(struct cfs_crypto_hash_desc *hdesc,
const void *buf, unsigned int buf_len)
{
@@ -178,7 +257,18 @@
}
EXPORT_SYMBOL(cfs_crypto_hash_update);
-/* If hash_len pointer is NULL - destroy descriptor. */
+/**
+ * Finish hash calculation, copy hash digest to buffer, clean up hash descriptor
+ *
+ * \param[in] hdesc hash descriptor
+ * \param[out] hash pointer to hash buffer to store hash digest
+ * \param[in,out] hash_len pointer to hash buffer size, if \a hdesc = NULL
+ * only free \a hdesc instead of computing the hash
+ *
+ * \retval 0 for success
+ * \retval -EOVERFLOW if hash_len is too small for the hash digest
+ * \retval negative errno for other errors from lower layers
+ */
int cfs_crypto_hash_final(struct cfs_crypto_hash_desc *hdesc,
unsigned char *hash, unsigned int *hash_len)
{
@@ -186,99 +276,153 @@
struct ahash_request *req = (void *)hdesc;
int size = crypto_ahash_digestsize(crypto_ahash_reqtfm(req));
- if (!hash_len) {
- crypto_free_ahash(crypto_ahash_reqtfm(req));
- ahash_request_free(req);
- return 0;
+ if (!hash || !hash_len) {
+ err = 0;
+ goto free_ahash;
}
- if (!hash || *hash_len < size) {
- *hash_len = size;
- return -ENOSPC;
+ if (*hash_len < size) {
+ err = -EOVERFLOW;
+ goto free_ahash;
}
+
ahash_request_set_crypt(req, NULL, hash, 0);
err = crypto_ahash_final(req);
-
- if (err < 0) {
- /* May be caller can fix error */
- return err;
- }
+ if (!err)
+ *hash_len = size;
+free_ahash:
crypto_free_ahash(crypto_ahash_reqtfm(req));
ahash_request_free(req);
return err;
}
EXPORT_SYMBOL(cfs_crypto_hash_final);
-static void cfs_crypto_performance_test(unsigned char alg_id,
- const unsigned char *buf,
- unsigned int buf_len)
+/**
+ * Compute the speed of specified hash function
+ *
+ * Run a speed test on the given hash algorithm on buffer of the given size.
+ * The speed is stored internally in the cfs_crypto_hash_speeds[] array, and
+ * is available through the cfs_crypto_hash_speed() function.
+ *
+ * \param[in] hash_alg hash algorithm id (CFS_HASH_ALG_*)
+ * \param[in] buf data buffer on which to compute the hash
+ * \param[in] buf_len length of \buf on which to compute hash
+ */
+static void cfs_crypto_performance_test(enum cfs_crypto_hash_alg hash_alg)
{
+ int buf_len = max(PAGE_SIZE, 1048576UL);
+ void *buf;
unsigned long start, end;
int bcount, err = 0;
- int sec = 1; /* do test only 1 sec */
- unsigned char hash[64];
- unsigned int hash_len = 64;
+ struct page *page;
+ unsigned char hash[CFS_CRYPTO_HASH_DIGESTSIZE_MAX];
+ unsigned int hash_len = sizeof(hash);
- for (start = jiffies, end = start + sec * HZ, bcount = 0;
- time_before(jiffies, end); bcount++) {
- err = cfs_crypto_hash_digest(alg_id, buf, buf_len, NULL, 0,
- hash, &hash_len);
+ page = alloc_page(GFP_KERNEL);
+ if (!page) {
+ err = -ENOMEM;
+ goto out_err;
+ }
+
+ buf = kmap(page);
+ memset(buf, 0xAD, PAGE_SIZE);
+ kunmap(page);
+
+ for (start = jiffies, end = start + msecs_to_jiffies(MSEC_PER_SEC),
+ bcount = 0; time_before(jiffies, end); bcount++) {
+ struct cfs_crypto_hash_desc *hdesc;
+ int i;
+
+ hdesc = cfs_crypto_hash_init(hash_alg, NULL, 0);
+ if (IS_ERR(hdesc)) {
+ err = PTR_ERR(hdesc);
+ break;
+ }
+
+ for (i = 0; i < buf_len / PAGE_SIZE; i++) {
+ err = cfs_crypto_hash_update_page(hdesc, page, 0,
+ PAGE_SIZE);
+ if (err)
+ break;
+ }
+
+ err = cfs_crypto_hash_final(hdesc, hash, &hash_len);
if (err)
break;
}
end = jiffies;
-
+ __free_page(page);
+out_err:
if (err) {
- cfs_crypto_hash_speeds[alg_id] = -1;
- CDEBUG(D_INFO, "Crypto hash algorithm %s, err = %d\n",
- cfs_crypto_hash_name(alg_id), err);
+ cfs_crypto_hash_speeds[hash_alg] = err;
+ CDEBUG(D_INFO, "Crypto hash algorithm %s test error: rc = %d\n",
+ cfs_crypto_hash_name(hash_alg), err);
} else {
unsigned long tmp;
tmp = ((bcount * buf_len / jiffies_to_msecs(end - start)) *
1000) / (1024 * 1024);
- cfs_crypto_hash_speeds[alg_id] = (int)tmp;
+ cfs_crypto_hash_speeds[hash_alg] = (int)tmp;
+ CDEBUG(D_CONFIG, "Crypto hash algorithm %s speed = %d MB/s\n",
+ cfs_crypto_hash_name(hash_alg),
+ cfs_crypto_hash_speeds[hash_alg]);
}
- CDEBUG(D_INFO, "Crypto hash algorithm %s speed = %d MB/s\n",
- cfs_crypto_hash_name(alg_id), cfs_crypto_hash_speeds[alg_id]);
}
-int cfs_crypto_hash_speed(unsigned char hash_alg)
+/**
+ * hash speed in Mbytes per second for valid hash algorithm
+ *
+ * Return the performance of the specified \a hash_alg that was previously
+ * computed using cfs_crypto_performance_test().
+ *
+ * \param[in] hash_alg hash algorithm id (CFS_HASH_ALG_*)
+ *
+ * \retval positive speed of the hash function in MB/s
+ * \retval -ENOENT if \a hash_alg is unsupported
+ * \retval negative errno if \a hash_alg speed is unavailable
+ */
+int cfs_crypto_hash_speed(enum cfs_crypto_hash_alg hash_alg)
{
if (hash_alg < CFS_HASH_ALG_MAX)
return cfs_crypto_hash_speeds[hash_alg];
- return -1;
+ return -ENOENT;
}
EXPORT_SYMBOL(cfs_crypto_hash_speed);
/**
- * Do performance test for all hash algorithms.
+ * Run the performance test for all hash algorithms.
+ *
+ * Run the cfs_crypto_performance_test() benchmark for all of the available
+ * hash functions using a 1MB buffer size. This is a reasonable buffer size
+ * for Lustre RPCs, even if the actual RPC size is larger or smaller.
+ *
+ * Since the setup cost and computation speed of various hash algorithms is
+ * a function of the buffer size (and possibly internal contention of offload
+ * engines), this speed only represents an estimate of the actual speed under
+ * actual usage, but is reasonable for comparing available algorithms.
+ *
+ * The actual speeds are available via cfs_crypto_hash_speed() for later
+ * comparison.
+ *
+ * \retval 0 on success
+ * \retval -ENOMEM if no memory is available for test buffer
*/
static int cfs_crypto_test_hashes(void)
{
- unsigned char i;
- unsigned char *data;
- unsigned int j;
- /* Data block size for testing hash. Maximum
- * kmalloc size for 2.6.18 kernel is 128K
- */
- unsigned int data_len = 1 * 128 * 1024;
+ enum cfs_crypto_hash_alg hash_alg;
- data = kmalloc(data_len, 0);
- if (!data)
- return -ENOMEM;
+ for (hash_alg = 0; hash_alg < CFS_HASH_ALG_MAX; hash_alg++)
+ cfs_crypto_performance_test(hash_alg);
- for (j = 0; j < data_len; j++)
- data[j] = j & 0xff;
-
- for (i = 0; i < CFS_HASH_ALG_MAX; i++)
- cfs_crypto_performance_test(i, data, data_len);
-
- kfree(data);
return 0;
}
static int adler32;
+/**
+ * Register available hash functions
+ *
+ * \retval 0
+ */
int cfs_crypto_register(void)
{
request_module("crc32c");
@@ -290,6 +434,9 @@
return 0;
}
+/**
+ * Unregister previously registered hash functions
+ */
void cfs_crypto_unregister(void)
{
if (adler32 == 0)
diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-module.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-module.c
index ebc60ac..d89f71e 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux/linux-module.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-module.c
@@ -40,10 +40,75 @@
#define LNET_MINOR 240
+static inline size_t libcfs_ioctl_packlen(struct libcfs_ioctl_data *data)
+{
+ size_t len = sizeof(*data);
+
+ len += cfs_size_round(data->ioc_inllen1);
+ len += cfs_size_round(data->ioc_inllen2);
+ return len;
+}
+
+static inline bool libcfs_ioctl_is_invalid(struct libcfs_ioctl_data *data)
+{
+ if (data->ioc_hdr.ioc_len > BIT(30)) {
+ CERROR("LIBCFS ioctl: ioc_len larger than 1<<30\n");
+ return true;
+ }
+ if (data->ioc_inllen1 > BIT(30)) {
+ CERROR("LIBCFS ioctl: ioc_inllen1 larger than 1<<30\n");
+ return true;
+ }
+ if (data->ioc_inllen2 > BIT(30)) {
+ CERROR("LIBCFS ioctl: ioc_inllen2 larger than 1<<30\n");
+ return true;
+ }
+ if (data->ioc_inlbuf1 && !data->ioc_inllen1) {
+ CERROR("LIBCFS ioctl: inlbuf1 pointer but 0 length\n");
+ return true;
+ }
+ if (data->ioc_inlbuf2 && !data->ioc_inllen2) {
+ CERROR("LIBCFS ioctl: inlbuf2 pointer but 0 length\n");
+ return true;
+ }
+ if (data->ioc_pbuf1 && !data->ioc_plen1) {
+ CERROR("LIBCFS ioctl: pbuf1 pointer but 0 length\n");
+ return true;
+ }
+ if (data->ioc_pbuf2 && !data->ioc_plen2) {
+ CERROR("LIBCFS ioctl: pbuf2 pointer but 0 length\n");
+ return true;
+ }
+ if (data->ioc_plen1 && !data->ioc_pbuf1) {
+ CERROR("LIBCFS ioctl: plen1 nonzero but no pbuf1 pointer\n");
+ return true;
+ }
+ if (data->ioc_plen2 && !data->ioc_pbuf2) {
+ CERROR("LIBCFS ioctl: plen2 nonzero but no pbuf2 pointer\n");
+ return true;
+ }
+ if ((__u32)libcfs_ioctl_packlen(data) != data->ioc_hdr.ioc_len) {
+ CERROR("LIBCFS ioctl: packlen != ioc_len\n");
+ return true;
+ }
+ if (data->ioc_inllen1 &&
+ data->ioc_bulk[data->ioc_inllen1 - 1] != '\0') {
+ CERROR("LIBCFS ioctl: inlbuf1 not 0 terminated\n");
+ return true;
+ }
+ if (data->ioc_inllen2 &&
+ data->ioc_bulk[cfs_size_round(data->ioc_inllen1) +
+ data->ioc_inllen2 - 1] != '\0') {
+ CERROR("LIBCFS ioctl: inlbuf2 not 0 terminated\n");
+ return true;
+ }
+ return false;
+}
+
int libcfs_ioctl_data_adjust(struct libcfs_ioctl_data *data)
{
if (libcfs_ioctl_is_invalid(data)) {
- CERROR("LNET: ioctl not correctly formatted\n");
+ CERROR("libcfs ioctl: parameter not correctly formatted\n");
return -EINVAL;
}
@@ -57,68 +122,47 @@
return 0;
}
-int libcfs_ioctl_getdata_len(const struct libcfs_ioctl_hdr __user *arg,
- __u32 *len)
+int libcfs_ioctl_getdata(struct libcfs_ioctl_hdr **hdr_pp,
+ const struct libcfs_ioctl_hdr __user *uhdr)
{
struct libcfs_ioctl_hdr hdr;
+ int err = 0;
- if (copy_from_user(&hdr, arg, sizeof(hdr)))
+ if (copy_from_user(&hdr, uhdr, sizeof(hdr)))
return -EFAULT;
if (hdr.ioc_version != LIBCFS_IOCTL_VERSION &&
hdr.ioc_version != LIBCFS_IOCTL_VERSION2) {
- CERROR("LNET: version mismatch expected %#x, got %#x\n",
+ CERROR("libcfs ioctl: version mismatch expected %#x, got %#x\n",
LIBCFS_IOCTL_VERSION, hdr.ioc_version);
return -EINVAL;
}
- *len = hdr.ioc_len;
-
- return 0;
-}
-
-int libcfs_ioctl_popdata(void __user *arg, void *data, int size)
-{
- if (copy_to_user(arg, data, size))
- return -EFAULT;
- return 0;
-}
-
-static int
-libcfs_psdev_open(struct inode *inode, struct file *file)
-{
- int rc = 0;
-
- if (!inode)
+ if (hdr.ioc_len < sizeof(struct libcfs_ioctl_data)) {
+ CERROR("libcfs ioctl: user buffer too small for ioctl\n");
return -EINVAL;
- if (libcfs_psdev_ops.p_open)
- rc = libcfs_psdev_ops.p_open(0, NULL);
- else
- return -EPERM;
- return rc;
-}
+ }
-/* called when closing /dev/device */
-static int
-libcfs_psdev_release(struct inode *inode, struct file *file)
-{
- int rc = 0;
-
- if (!inode)
+ if (hdr.ioc_len > LIBCFS_IOC_DATA_MAX) {
+ CERROR("libcfs ioctl: user buffer is too large %d/%d\n",
+ hdr.ioc_len, LIBCFS_IOC_DATA_MAX);
return -EINVAL;
- if (libcfs_psdev_ops.p_close)
- rc = libcfs_psdev_ops.p_close(0, NULL);
- else
- rc = -EPERM;
- return rc;
+ }
+
+ LIBCFS_ALLOC(*hdr_pp, hdr.ioc_len);
+ if (!*hdr_pp)
+ return -ENOMEM;
+
+ if (copy_from_user(*hdr_pp, uhdr, hdr.ioc_len)) {
+ LIBCFS_FREE(*hdr_pp, hdr.ioc_len);
+ err = -EFAULT;
+ }
+ return err;
}
-static long libcfs_ioctl(struct file *file,
- unsigned int cmd, unsigned long arg)
+static long
+libcfs_psdev_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
{
- struct cfs_psdev_file pfile;
- int rc = 0;
-
if (!capable(CAP_SYS_ADMIN))
return -EACCES;
@@ -130,26 +174,12 @@
return -EINVAL;
}
- /* Handle platform-dependent IOC requests */
- switch (cmd) {
- case IOC_LIBCFS_PANIC:
- if (!capable(CFS_CAP_SYS_BOOT))
- return -EPERM;
- panic("debugctl-invoked panic");
- return 0;
- }
-
- if (libcfs_psdev_ops.p_ioctl)
- rc = libcfs_psdev_ops.p_ioctl(&pfile, cmd, (void __user *)arg);
- else
- rc = -EPERM;
- return rc;
+ return libcfs_ioctl(cmd, (void __user *)arg);
}
static const struct file_operations libcfs_fops = {
- .unlocked_ioctl = libcfs_ioctl,
- .open = libcfs_psdev_open,
- .release = libcfs_psdev_release,
+ .owner = THIS_MODULE,
+ .unlocked_ioctl = libcfs_psdev_ioctl,
};
struct miscdevice libcfs_dev = {
diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-prim.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-prim.c
index 8908446..bbe19a6 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux/linux-prim.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-prim.c
@@ -46,30 +46,6 @@
#include <linux/kgdb.h>
#endif
-/**
- * wait_queue_t of Linux (version < 2.6.34) is a FIFO list for exclusively
- * waiting threads, which is not always desirable because all threads will
- * be waken up again and again, even user only needs a few of them to be
- * active most time. This is not good for performance because cache can
- * be polluted by different threads.
- *
- * LIFO list can resolve this problem because we always wakeup the most
- * recent active thread by default.
- *
- * NB: please don't call non-exclusive & exclusive wait on the same
- * waitq if add_wait_queue_exclusive_head is used.
- */
-void
-add_wait_queue_exclusive_head(wait_queue_head_t *waitq, wait_queue_t *link)
-{
- unsigned long flags;
-
- spin_lock_irqsave(&waitq->lock, flags);
- __add_wait_queue_exclusive(waitq, link);
- spin_unlock_irqrestore(&waitq->lock, flags);
-}
-EXPORT_SYMBOL(add_wait_queue_exclusive_head);
-
sigset_t
cfs_block_allsigs(void)
{
@@ -128,13 +104,6 @@
}
EXPORT_SYMBOL(cfs_restore_sigs);
-int
-cfs_signal_pending(void)
-{
- return signal_pending(current);
-}
-EXPORT_SYMBOL(cfs_signal_pending);
-
void
cfs_clear_sigpending(void)
{
diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c
index cdc640b..f2d0411 100644
--- a/drivers/staging/lustre/lnet/libcfs/module.c
+++ b/drivers/staging/lustre/lnet/libcfs/module.c
@@ -54,9 +54,6 @@
# define DEBUG_SUBSYSTEM S_LNET
-#define LNET_MAX_IOCTL_BUF_LEN (sizeof(struct lnet_ioctl_net_config) + \
- sizeof(struct lnet_ioctl_config_data))
-
#include "../../include/linux/libcfs/libcfs.h"
#include <asm/div64.h>
@@ -68,20 +65,6 @@
static struct dentry *lnet_debugfs_root;
-/* called when opening /dev/device */
-static int libcfs_psdev_open(unsigned long flags, void *args)
-{
- try_module_get(THIS_MODULE);
- return 0;
-}
-
-/* called when closing /dev/device */
-static int libcfs_psdev_release(unsigned long flags, void *args)
-{
- module_put(THIS_MODULE);
- return 0;
-}
-
static DECLARE_RWSEM(ioctl_list_sem);
static LIST_HEAD(ioctl_list);
@@ -115,39 +98,47 @@
}
EXPORT_SYMBOL(libcfs_deregister_ioctl);
-static int libcfs_ioctl_handle(struct cfs_psdev_file *pfile, unsigned long cmd,
- void __user *arg, struct libcfs_ioctl_hdr *hdr)
+int libcfs_ioctl(unsigned long cmd, void __user *uparam)
{
struct libcfs_ioctl_data *data = NULL;
- int err = -EINVAL;
+ struct libcfs_ioctl_hdr *hdr;
+ int err;
- /*
- * The libcfs_ioctl_data_adjust() function performs adjustment
- * operations on the libcfs_ioctl_data structure to make
- * it usable by the code. This doesn't need to be called
- * for new data structures added.
- */
+ /* 'cmd' and permissions get checked in our arch-specific caller */
+ err = libcfs_ioctl_getdata(&hdr, uparam);
+ if (err) {
+ CDEBUG_LIMIT(D_ERROR,
+ "libcfs ioctl: data header error %d\n", err);
+ return err;
+ }
+
if (hdr->ioc_version == LIBCFS_IOCTL_VERSION) {
+ /*
+ * The libcfs_ioctl_data_adjust() function performs adjustment
+ * operations on the libcfs_ioctl_data structure to make
+ * it usable by the code. This doesn't need to be called
+ * for new data structures added.
+ */
data = container_of(hdr, struct libcfs_ioctl_data, ioc_hdr);
err = libcfs_ioctl_data_adjust(data);
if (err)
- return err;
+ goto out;
}
+ CDEBUG(D_IOCTL, "libcfs ioctl cmd %lu\n", cmd);
switch (cmd) {
case IOC_LIBCFS_CLEAR_DEBUG:
libcfs_debug_clear_buffer();
- return 0;
- /*
- * case IOC_LIBCFS_PANIC:
- * Handled in arch/cfs_module.c
- */
+ break;
+
case IOC_LIBCFS_MARK_DEBUG:
- if (!data->ioc_inlbuf1 ||
- data->ioc_inlbuf1[data->ioc_inllen1 - 1] != '\0')
- return -EINVAL;
+ if (!data || !data->ioc_inlbuf1 ||
+ data->ioc_inlbuf1[data->ioc_inllen1 - 1] != '\0') {
+ err = -EINVAL;
+ goto out;
+ }
libcfs_debug_mark_buffer(data->ioc_inlbuf1);
- return 0;
+ break;
default: {
struct libcfs_ioctl_handler *hand;
@@ -156,67 +147,23 @@
down_read(&ioctl_list_sem);
list_for_each_entry(hand, &ioctl_list, item) {
err = hand->handle_ioctl(cmd, hdr);
- if (err != -EINVAL) {
- if (err == 0)
- err = libcfs_ioctl_popdata(arg,
- hdr, hdr->ioc_len);
- break;
+ if (err == -EINVAL)
+ continue;
+
+ if (!err) {
+ if (copy_to_user(uparam, hdr, hdr->ioc_len))
+ err = -EFAULT;
}
+ break;
}
up_read(&ioctl_list_sem);
- break;
+ break; }
}
- }
-
- return err;
-}
-
-static int libcfs_ioctl(struct cfs_psdev_file *pfile, unsigned long cmd,
- void __user *arg)
-{
- struct libcfs_ioctl_hdr *hdr;
- int err = 0;
- __u32 buf_len;
-
- err = libcfs_ioctl_getdata_len(arg, &buf_len);
- if (err)
- return err;
-
- /*
- * do a check here to restrict the size of the memory
- * to allocate to guard against DoS attacks.
- */
- if (buf_len > LNET_MAX_IOCTL_BUF_LEN) {
- CERROR("LNET: user buffer exceeds kernel buffer\n");
- return -EINVAL;
- }
-
- LIBCFS_ALLOC_GFP(hdr, buf_len, GFP_KERNEL);
- if (!hdr)
- return -ENOMEM;
-
- /* 'cmd' and permissions get checked in our arch-specific caller */
- if (copy_from_user(hdr, arg, buf_len)) {
- CERROR("LNET ioctl: data error\n");
- err = -EFAULT;
- goto out;
- }
-
- err = libcfs_ioctl_handle(pfile, cmd, arg, hdr);
-
out:
- LIBCFS_FREE(hdr, buf_len);
+ LIBCFS_FREE(hdr, hdr->ioc_len);
return err;
}
-struct cfs_psdev_ops libcfs_psdev_ops = {
- libcfs_psdev_open,
- libcfs_psdev_release,
- NULL,
- NULL,
- libcfs_ioctl
-};
-
int lprocfs_call_handler(void *data, int write, loff_t *ppos,
void __user *buffer, size_t *lenp,
int (*handler)(void *data, int write, loff_t pos,
@@ -478,6 +425,13 @@
.proc_handler = &proc_dointvec
},
{
+ .procname = "fail_err",
+ .data = &cfs_fail_err,
+ .maxlen = sizeof(cfs_fail_err),
+ .mode = 0644,
+ .proc_handler = &proc_dointvec,
+ },
+ {
}
};
diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c
index 244eb89..7739b94 100644
--- a/drivers/staging/lustre/lnet/libcfs/tracefile.c
+++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c
@@ -707,10 +707,9 @@
struct cfs_trace_page *tage;
struct cfs_trace_page *tmp;
char *buf;
+ mm_segment_t __oldfs;
int rc;
- DECL_MMSPACE;
-
cfs_tracefile_write_lock();
filp = filp_open(filename, O_CREAT | O_EXCL | O_WRONLY | O_LARGEFILE,
@@ -729,11 +728,12 @@
rc = 0;
goto close;
}
+ __oldfs = get_fs();
+ set_fs(get_ds());
/* ok, for now, just write the pages. in the future we'll be building
* iobufs with the pages and calling generic_direct_IO
*/
- MMSPACE_OPEN;
list_for_each_entry_safe(tage, tmp, &pc.pc_pages, linkage) {
__LASSERT_TAGE_INVARIANT(tage);
@@ -752,7 +752,7 @@
list_del(&tage->linkage);
cfs_tage_free(tage);
}
- MMSPACE_CLOSE;
+ set_fs(__oldfs);
rc = vfs_fsync(filp, 1);
if (rc)
pr_err("sync returns %d\n", rc);
@@ -986,13 +986,12 @@
struct tracefiled_ctl *tctl = arg;
struct cfs_trace_page *tage;
struct cfs_trace_page *tmp;
+ mm_segment_t __oldfs;
struct file *filp;
char *buf;
int last_loop = 0;
int rc;
- DECL_MMSPACE;
-
/* we're started late enough that we pick up init's fs context */
/* this is so broken in uml? what on earth is going on? */
@@ -1025,8 +1024,8 @@
__LASSERT(list_empty(&pc.pc_pages));
goto end_loop;
}
-
- MMSPACE_OPEN;
+ __oldfs = get_fs();
+ set_fs(get_ds());
list_for_each_entry_safe(tage, tmp, &pc.pc_pages, linkage) {
static loff_t f_pos;
@@ -1051,7 +1050,7 @@
break;
}
}
- MMSPACE_CLOSE;
+ set_fs(__oldfs);
filp_close(filp, NULL);
put_pages_on_daemon_list(&pc);
diff --git a/drivers/staging/lustre/lnet/libcfs/workitem.c b/drivers/staging/lustre/lnet/libcfs/workitem.c
index c72fe00..92236ae 100644
--- a/drivers/staging/lustre/lnet/libcfs/workitem.c
+++ b/drivers/staging/lustre/lnet/libcfs/workitem.c
@@ -111,7 +111,7 @@
* 1. when it returns no one shall try to schedule the workitem.
*/
void
-cfs_wi_exit(struct cfs_wi_sched *sched, cfs_workitem_t *wi)
+cfs_wi_exit(struct cfs_wi_sched *sched, struct cfs_workitem *wi)
{
LASSERT(!in_interrupt()); /* because we use plain spinlock */
LASSERT(!sched->ws_stopping);
@@ -138,7 +138,7 @@
* cancel schedule request of workitem \a wi
*/
int
-cfs_wi_deschedule(struct cfs_wi_sched *sched, cfs_workitem_t *wi)
+cfs_wi_deschedule(struct cfs_wi_sched *sched, struct cfs_workitem *wi)
{
int rc;
@@ -179,7 +179,7 @@
* be added, and even dynamic creation of serialised queues might be supported.
*/
void
-cfs_wi_schedule(struct cfs_wi_sched *sched, cfs_workitem_t *wi)
+cfs_wi_schedule(struct cfs_wi_sched *sched, struct cfs_workitem *wi)
{
LASSERT(!in_interrupt()); /* because we use plain spinlock */
LASSERT(!sched->ws_stopping);
@@ -229,12 +229,12 @@
while (!sched->ws_stopping) {
int nloops = 0;
int rc;
- cfs_workitem_t *wi;
+ struct cfs_workitem *wi;
while (!list_empty(&sched->ws_runq) &&
nloops < CFS_WI_RESCHED) {
- wi = list_entry(sched->ws_runq.next, cfs_workitem_t,
- wi_list);
+ wi = list_entry(sched->ws_runq.next,
+ struct cfs_workitem, wi_list);
LASSERT(wi->wi_scheduled && !wi->wi_running);
list_del_init(&wi->wi_list);
diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c
index 8764755..fe0dbe7 100644
--- a/drivers/staging/lustre/lnet/lnet/api-ni.c
+++ b/drivers/staging/lustre/lnet/lnet/api-ni.c
@@ -1215,9 +1215,9 @@
}
static int
-lnet_startup_lndni(struct lnet_ni *ni, __s32 peer_timeout,
- __s32 peer_cr, __s32 peer_buf_cr, __s32 credits)
+lnet_startup_lndni(struct lnet_ni *ni, struct lnet_ioctl_config_data *conf)
{
+ struct lnet_ioctl_config_lnd_tunables *lnd_tunables = NULL;
int rc = -EINVAL;
int lnd_type;
lnd_t *lnd;
@@ -1275,6 +1275,21 @@
ni->ni_lnd = lnd;
+ if (conf && conf->cfg_hdr.ioc_len > sizeof(*conf))
+ lnd_tunables = (struct lnet_ioctl_config_lnd_tunables *)conf->cfg_bulk;
+
+ if (lnd_tunables) {
+ LIBCFS_ALLOC(ni->ni_lnd_tunables,
+ sizeof(*ni->ni_lnd_tunables));
+ if (!ni->ni_lnd_tunables) {
+ mutex_unlock(&the_lnet.ln_lnd_mutex);
+ rc = -ENOMEM;
+ goto failed0;
+ }
+ memcpy(ni->ni_lnd_tunables, lnd_tunables,
+ sizeof(*ni->ni_lnd_tunables));
+ }
+
rc = lnd->lnd_startup(ni);
mutex_unlock(&the_lnet.ln_lnd_mutex);
@@ -1292,20 +1307,28 @@
* If given some LND tunable parameters, parse those now to
* override the values in the NI structure.
*/
- if (peer_buf_cr >= 0)
- ni->ni_peerrtrcredits = peer_buf_cr;
- if (peer_timeout >= 0)
- ni->ni_peertimeout = peer_timeout;
+ if (conf && conf->cfg_config_u.cfg_net.net_peer_rtr_credits >= 0) {
+ ni->ni_peerrtrcredits =
+ conf->cfg_config_u.cfg_net.net_peer_rtr_credits;
+ }
+ if (conf && conf->cfg_config_u.cfg_net.net_peer_timeout >= 0) {
+ ni->ni_peertimeout =
+ conf->cfg_config_u.cfg_net.net_peer_timeout;
+ }
/*
* TODO
* Note: For now, don't allow the user to change
* peertxcredits as this number is used in the
* IB LND to control queue depth.
- * if (peer_cr != -1)
- * ni->ni_peertxcredits = peer_cr;
+ *
+ * if (conf && conf->cfg_config_u.cfg_net.net_peer_tx_credits != -1)
+ * ni->ni_peertxcredits =
+ * conf->cfg_config_u.cfg_net.net_peer_tx_credits;
*/
- if (credits >= 0)
- ni->ni_maxtxcredits = credits;
+ if (conf && conf->cfg_config_u.cfg_net.net_max_tx_credits >= 0) {
+ ni->ni_maxtxcredits =
+ conf->cfg_config_u.cfg_net.net_max_tx_credits;
+ }
LASSERT(ni->ni_peertimeout <= 0 || lnd->lnd_query);
@@ -1367,7 +1390,7 @@
while (!list_empty(nilist)) {
ni = list_entry(nilist->next, lnet_ni_t, ni_list);
list_del(&ni->ni_list);
- rc = lnet_startup_lndni(ni, -1, -1, -1, -1);
+ rc = lnet_startup_lndni(ni, NULL);
if (rc < 0)
goto failed;
@@ -1641,25 +1664,20 @@
* parameters
*
* \param[in] ni network interface structure
- * \param[out] cpt_count the number of cpts the ni is on
- * \param[out] nid Network Interface ID
- * \param[out] peer_timeout NI peer timeout
- * \param[out] peer_tx_crdits NI peer transmit credits
- * \param[out] peer_rtr_credits NI peer router credits
- * \param[out] max_tx_credits NI max transmit credit
- * \param[out] net_config Network configuration
+ * \param[out] config NI configuration
*/
static void
-lnet_fill_ni_info(struct lnet_ni *ni, __u32 *cpt_count, __u64 *nid,
- int *peer_timeout, int *peer_tx_credits,
- int *peer_rtr_credits, int *max_tx_credits,
- struct lnet_ioctl_net_config *net_config)
+lnet_fill_ni_info(struct lnet_ni *ni, struct lnet_ioctl_config_data *config)
{
+ struct lnet_ioctl_config_lnd_tunables *lnd_cfg = NULL;
+ struct lnet_ioctl_net_config *net_config;
+ size_t min_size, tunable_size = 0;
int i;
- if (!ni)
+ if (!ni || !config)
return;
+ net_config = (struct lnet_ioctl_net_config *) config->cfg_bulk;
if (!net_config)
return;
@@ -1675,11 +1693,11 @@
sizeof(net_config->ni_interfaces[i]));
}
- *nid = ni->ni_nid;
- *peer_timeout = ni->ni_peertimeout;
- *peer_tx_credits = ni->ni_peertxcredits;
- *peer_rtr_credits = ni->ni_peerrtrcredits;
- *max_tx_credits = ni->ni_maxtxcredits;
+ config->cfg_nid = ni->ni_nid;
+ config->cfg_config_u.cfg_net.net_peer_timeout = ni->ni_peertimeout;
+ config->cfg_config_u.cfg_net.net_max_tx_credits = ni->ni_maxtxcredits;
+ config->cfg_config_u.cfg_net.net_peer_tx_credits = ni->ni_peertxcredits;
+ config->cfg_config_u.cfg_net.net_peer_rtr_credits = ni->ni_peerrtrcredits;
net_config->ni_status = ni->ni_status->ns_status;
@@ -1689,18 +1707,40 @@
for (i = 0; i < num_cpts; i++)
net_config->ni_cpts[i] = ni->ni_cpts[i];
- *cpt_count = num_cpts;
+ config->cfg_ncpts = num_cpts;
+ }
+
+ /*
+ * See if user land tools sent in a newer and larger version
+ * of struct lnet_tunables than what the kernel uses.
+ */
+ min_size = sizeof(*config) + sizeof(*net_config);
+
+ if (config->cfg_hdr.ioc_len > min_size)
+ tunable_size = config->cfg_hdr.ioc_len - min_size;
+
+ /* Don't copy to much data to user space */
+ min_size = min(tunable_size, sizeof(*ni->ni_lnd_tunables));
+ lnd_cfg = (struct lnet_ioctl_config_lnd_tunables *)net_config->cfg_bulk;
+
+ if (ni->ni_lnd_tunables && lnd_cfg && min_size) {
+ memcpy(lnd_cfg, ni->ni_lnd_tunables, min_size);
+ config->cfg_config_u.cfg_net.net_interface_count = 1;
+
+ /* Tell user land that kernel side has less data */
+ if (tunable_size > sizeof(*ni->ni_lnd_tunables)) {
+ min_size = tunable_size - sizeof(ni->ni_lnd_tunables);
+ config->cfg_hdr.ioc_len -= min_size;
+ }
}
}
-int
-lnet_get_net_config(int idx, __u32 *cpt_count, __u64 *nid, int *peer_timeout,
- int *peer_tx_credits, int *peer_rtr_credits,
- int *max_tx_credits,
- struct lnet_ioctl_net_config *net_config)
+static int
+lnet_get_net_config(struct lnet_ioctl_config_data *config)
{
struct lnet_ni *ni;
struct list_head *tmp;
+ int idx = config->cfg_count;
int cpt, i = 0;
int rc = -ENOENT;
@@ -1712,9 +1752,7 @@
ni = list_entry(tmp, lnet_ni_t, ni_list);
lnet_ni_lock(ni);
- lnet_fill_ni_info(ni, cpt_count, nid, peer_timeout,
- peer_tx_credits, peer_rtr_credits,
- max_tx_credits, net_config);
+ lnet_fill_ni_info(ni, config);
lnet_ni_unlock(ni);
rc = 0;
break;
@@ -1725,10 +1763,9 @@
}
int
-lnet_dyn_add_ni(lnet_pid_t requested_pid, char *nets,
- __s32 peer_timeout, __s32 peer_cr, __s32 peer_buf_cr,
- __s32 credits)
+lnet_dyn_add_ni(lnet_pid_t requested_pid, struct lnet_ioctl_config_data *conf)
{
+ char *nets = conf->cfg_config_u.cfg_net.net_intf;
lnet_ping_info_t *pinfo;
lnet_handle_md_t md_handle;
struct lnet_ni *ni;
@@ -1773,8 +1810,7 @@
list_del_init(&ni->ni_list);
- rc = lnet_startup_lndni(ni, peer_timeout, peer_cr,
- peer_buf_cr, credits);
+ rc = lnet_startup_lndni(ni, conf);
if (rc)
goto failed1;
@@ -1864,6 +1900,10 @@
int rc;
unsigned long secs_passed;
+ BUILD_BUG_ON(LIBCFS_IOC_DATA_MAX <
+ sizeof(struct lnet_ioctl_net_config) +
+ sizeof(struct lnet_ioctl_config_data));
+
switch (cmd) {
case IOC_LIBCFS_GET_NI:
rc = LNetGetId(data->ioc_count, &id);
@@ -1918,27 +1958,14 @@
&config->cfg_config_u.cfg_route.rtr_priority);
case IOC_LIBCFS_GET_NET: {
- struct lnet_ioctl_net_config *net_config;
- size_t total = sizeof(*config) + sizeof(*net_config);
-
+ size_t total = sizeof(*config) +
+ sizeof(struct lnet_ioctl_net_config);
config = arg;
if (config->cfg_hdr.ioc_len < total)
return -EINVAL;
- net_config = (struct lnet_ioctl_net_config *)
- config->cfg_bulk;
- if (!net_config)
- return -EINVAL;
-
- return lnet_get_net_config(config->cfg_count,
- &config->cfg_ncpts,
- &config->cfg_nid,
- &config->cfg_config_u.cfg_net.net_peer_timeout,
- &config->cfg_config_u.cfg_net.net_peer_tx_credits,
- &config->cfg_config_u.cfg_net.net_peer_rtr_credits,
- &config->cfg_config_u.cfg_net.net_max_tx_credits,
- net_config);
+ return lnet_get_net_config(config);
}
case IOC_LIBCFS_GET_LNET_STATS: {
diff --git a/drivers/staging/lustre/lnet/lnet/config.c b/drivers/staging/lustre/lnet/lnet/config.c
index 449069c..480cc9c 100644
--- a/drivers/staging/lustre/lnet/lnet/config.c
+++ b/drivers/staging/lustre/lnet/lnet/config.c
@@ -107,6 +107,9 @@
if (ni->ni_cpts)
cfs_expr_list_values_free(ni->ni_cpts, ni->ni_ncpts);
+ if (ni->ni_lnd_tunables)
+ LIBCFS_FREE(ni->ni_lnd_tunables, sizeof(*ni->ni_lnd_tunables));
+
for (i = 0; i < LNET_MAX_INTERFACES && ni->ni_interfaces[i]; i++) {
LIBCFS_FREE(ni->ni_interfaces[i],
strlen(ni->ni_interfaces[i]) + 1);
diff --git a/drivers/staging/lustre/lnet/lnet/lib-move.c b/drivers/staging/lustre/lnet/lnet/lib-move.c
index f19aa93..c5d5bed 100644
--- a/drivers/staging/lustre/lnet/lnet/lib-move.c
+++ b/drivers/staging/lustre/lnet/lnet/lib-move.c
@@ -407,7 +407,7 @@
LASSERT(niov > 0);
LASSERT(nkiov > 0);
this_nob = min(iov->iov_len - iovoffset,
- (__kernel_size_t) kiov->kiov_len - kiovoffset);
+ (__kernel_size_t)kiov->kiov_len - kiovoffset);
this_nob = min(this_nob, nob);
if (!addr)
@@ -477,7 +477,7 @@
do {
LASSERT(nkiov > 0);
LASSERT(niov > 0);
- this_nob = min((__kernel_size_t) kiov->kiov_len - kiovoffset,
+ this_nob = min((__kernel_size_t)kiov->kiov_len - kiovoffset,
iov->iov_len - iovoffset);
this_nob = min(this_nob, nob);
@@ -996,7 +996,7 @@
LASSERT(msg2->msg_txpeer->lp_ni == ni);
LASSERT(msg2->msg_tx_delayed);
- (void) lnet_post_send_locked(msg2, 1);
+ (void)lnet_post_send_locked(msg2, 1);
}
}
@@ -1019,7 +1019,7 @@
LASSERT(msg2->msg_txpeer == txpeer);
LASSERT(msg2->msg_tx_delayed);
- (void) lnet_post_send_locked(msg2, 1);
+ (void)lnet_post_send_locked(msg2, 1);
}
}
@@ -1142,7 +1142,7 @@
lnet_msg_t, msg_list);
list_del(&msg2->msg_list);
- (void) lnet_post_routed_recv_locked(msg2, 1);
+ (void)lnet_post_routed_recv_locked(msg2, 1);
}
}
if (rxpeer) {
diff --git a/drivers/staging/lustre/lnet/lnet/module.c b/drivers/staging/lustre/lnet/lnet/module.c
index 93037c1..246b5c1 100644
--- a/drivers/staging/lustre/lnet/lnet/module.c
+++ b/drivers/staging/lustre/lnet/lnet/module.c
@@ -108,12 +108,7 @@
rc = -EINVAL;
goto out_unlock;
}
- rc = lnet_dyn_add_ni(LNET_PID_LUSTRE,
- conf->cfg_config_u.cfg_net.net_intf,
- conf->cfg_config_u.cfg_net.net_peer_timeout,
- conf->cfg_config_u.cfg_net.net_peer_tx_credits,
- conf->cfg_config_u.cfg_net.net_peer_rtr_credits,
- conf->cfg_config_u.cfg_net.net_max_tx_credits);
+ rc = lnet_dyn_add_ni(LNET_PID_LUSTRE, conf);
out_unlock:
mutex_unlock(&lnet_config_mutex);
diff --git a/drivers/staging/lustre/lnet/selftest/brw_test.c b/drivers/staging/lustre/lnet/selftest/brw_test.c
index dcb6e50..a63d86c 100644
--- a/drivers/staging/lustre/lnet/selftest/brw_test.c
+++ b/drivers/staging/lustre/lnet/selftest/brw_test.c
@@ -49,10 +49,10 @@
MODULE_PARM_DESC(brw_inject_errors, "# data errors to inject randomly, zero by default");
static void
-brw_client_fini(sfw_test_instance_t *tsi)
+brw_client_fini(struct sfw_test_instance *tsi)
{
- srpc_bulk_t *bulk;
- sfw_test_unit_t *tsu;
+ struct srpc_bulk *bulk;
+ struct sfw_test_unit *tsu;
LASSERT(tsi->tsi_is_client);
@@ -67,21 +67,21 @@
}
static int
-brw_client_init(sfw_test_instance_t *tsi)
+brw_client_init(struct sfw_test_instance *tsi)
{
- sfw_session_t *sn = tsi->tsi_batch->bat_session;
+ struct sfw_session *sn = tsi->tsi_batch->bat_session;
int flags;
int npg;
int len;
int opc;
- srpc_bulk_t *bulk;
- sfw_test_unit_t *tsu;
+ struct srpc_bulk *bulk;
+ struct sfw_test_unit *tsu;
LASSERT(sn);
LASSERT(tsi->tsi_is_client);
if (!(sn->sn_features & LST_FEAT_BULK_LEN)) {
- test_bulk_req_t *breq = &tsi->tsi_u.bulk_v0;
+ struct test_bulk_req *breq = &tsi->tsi_u.bulk_v0;
opc = breq->blk_opc;
flags = breq->blk_flags;
@@ -91,9 +91,8 @@
* but we have to keep it for compatibility
*/
len = npg * PAGE_SIZE;
-
} else {
- test_bulk_req_v1_t *breq = &tsi->tsi_u.bulk_v1;
+ struct test_bulk_req_v1 *breq = &tsi->tsi_u.bulk_v1;
/*
* I should never get this step if it's unknown feature
@@ -225,7 +224,7 @@
}
static void
-brw_fill_bulk(srpc_bulk_t *bk, int pattern, __u64 magic)
+brw_fill_bulk(struct srpc_bulk *bk, int pattern, __u64 magic)
{
int i;
struct page *pg;
@@ -237,7 +236,7 @@
}
static int
-brw_check_bulk(srpc_bulk_t *bk, int pattern, __u64 magic)
+brw_check_bulk(struct srpc_bulk *bk, int pattern, __u64 magic)
{
int i;
struct page *pg;
@@ -255,14 +254,14 @@
}
static int
-brw_client_prep_rpc(sfw_test_unit_t *tsu,
- lnet_process_id_t dest, srpc_client_rpc_t **rpcpp)
+brw_client_prep_rpc(struct sfw_test_unit *tsu,
+ lnet_process_id_t dest, struct srpc_client_rpc **rpcpp)
{
- srpc_bulk_t *bulk = tsu->tsu_private;
- sfw_test_instance_t *tsi = tsu->tsu_instance;
- sfw_session_t *sn = tsi->tsi_batch->bat_session;
- srpc_client_rpc_t *rpc;
- srpc_brw_reqst_t *req;
+ struct srpc_bulk *bulk = tsu->tsu_private;
+ struct sfw_test_instance *tsi = tsu->tsu_instance;
+ struct sfw_session *sn = tsi->tsi_batch->bat_session;
+ struct srpc_client_rpc *rpc;
+ struct srpc_brw_reqst *req;
int flags;
int npg;
int len;
@@ -273,15 +272,14 @@
LASSERT(bulk);
if (!(sn->sn_features & LST_FEAT_BULK_LEN)) {
- test_bulk_req_t *breq = &tsi->tsi_u.bulk_v0;
+ struct test_bulk_req *breq = &tsi->tsi_u.bulk_v0;
opc = breq->blk_opc;
flags = breq->blk_flags;
npg = breq->blk_npg;
len = npg * PAGE_SIZE;
-
} else {
- test_bulk_req_v1_t *breq = &tsi->tsi_u.bulk_v1;
+ struct test_bulk_req_v1 *breq = &tsi->tsi_u.bulk_v1;
/*
* I should never get this step if it's unknown feature
@@ -299,7 +297,7 @@
if (rc)
return rc;
- memcpy(&rpc->crpc_bulk, bulk, offsetof(srpc_bulk_t, bk_iovs[npg]));
+ memcpy(&rpc->crpc_bulk, bulk, offsetof(struct srpc_bulk, bk_iovs[npg]));
if (opc == LST_BRW_WRITE)
brw_fill_bulk(&rpc->crpc_bulk, flags, BRW_MAGIC);
else
@@ -315,21 +313,21 @@
}
static void
-brw_client_done_rpc(sfw_test_unit_t *tsu, srpc_client_rpc_t *rpc)
+brw_client_done_rpc(struct sfw_test_unit *tsu, struct srpc_client_rpc *rpc)
{
__u64 magic = BRW_MAGIC;
- sfw_test_instance_t *tsi = tsu->tsu_instance;
- sfw_session_t *sn = tsi->tsi_batch->bat_session;
- srpc_msg_t *msg = &rpc->crpc_replymsg;
- srpc_brw_reply_t *reply = &msg->msg_body.brw_reply;
- srpc_brw_reqst_t *reqst = &rpc->crpc_reqstmsg.msg_body.brw_reqst;
+ struct sfw_test_instance *tsi = tsu->tsu_instance;
+ struct sfw_session *sn = tsi->tsi_batch->bat_session;
+ struct srpc_msg *msg = &rpc->crpc_replymsg;
+ struct srpc_brw_reply *reply = &msg->msg_body.brw_reply;
+ struct srpc_brw_reqst *reqst = &rpc->crpc_reqstmsg.msg_body.brw_reqst;
LASSERT(sn);
if (rpc->crpc_status) {
CERROR("BRW RPC to %s failed with %d\n",
libcfs_id2str(rpc->crpc_dest), rpc->crpc_status);
- if (!tsi->tsi_stopping) /* rpc could have been aborted */
+ if (!tsi->tsi_stopping) /* rpc could have been aborted */
atomic_inc(&sn->sn_brw_errors);
return;
}
@@ -363,7 +361,7 @@
static void
brw_server_rpc_done(struct srpc_server_rpc *rpc)
{
- srpc_bulk_t *blk = rpc->srpc_bulk;
+ struct srpc_bulk *blk = rpc->srpc_bulk;
if (!blk)
return;
@@ -384,9 +382,9 @@
brw_bulk_ready(struct srpc_server_rpc *rpc, int status)
{
__u64 magic = BRW_MAGIC;
- srpc_brw_reply_t *reply = &rpc->srpc_replymsg.msg_body.brw_reply;
- srpc_brw_reqst_t *reqst;
- srpc_msg_t *reqstmsg;
+ struct srpc_brw_reply *reply = &rpc->srpc_replymsg.msg_body.brw_reply;
+ struct srpc_brw_reqst *reqst;
+ struct srpc_msg *reqstmsg;
LASSERT(rpc->srpc_bulk);
LASSERT(rpc->srpc_reqstbuf);
@@ -420,10 +418,10 @@
brw_server_handle(struct srpc_server_rpc *rpc)
{
struct srpc_service *sv = rpc->srpc_scd->scd_svc;
- srpc_msg_t *replymsg = &rpc->srpc_replymsg;
- srpc_msg_t *reqstmsg = &rpc->srpc_reqstbuf->buf_msg;
- srpc_brw_reply_t *reply = &replymsg->msg_body.brw_reply;
- srpc_brw_reqst_t *reqst = &reqstmsg->msg_body.brw_reqst;
+ struct srpc_msg *replymsg = &rpc->srpc_replymsg;
+ struct srpc_msg *reqstmsg = &rpc->srpc_reqstbuf->buf_msg;
+ struct srpc_brw_reply *reply = &replymsg->msg_body.brw_reply;
+ struct srpc_brw_reqst *reqst = &reqstmsg->msg_body.brw_reqst;
int npg;
int rc;
@@ -459,7 +457,7 @@
if (!(reqstmsg->msg_ses_feats & LST_FEAT_BULK_LEN)) {
/* compat with old version */
- if (reqst->brw_len & ~CFS_PAGE_MASK) {
+ if (reqst->brw_len & ~PAGE_MASK) {
reply->brw_status = EINVAL;
return 0;
}
@@ -490,7 +488,8 @@
return 0;
}
-sfw_test_client_ops_t brw_test_client;
+struct sfw_test_client_ops brw_test_client;
+
void brw_init_test_client(void)
{
brw_test_client.tso_init = brw_client_init;
@@ -499,7 +498,8 @@
brw_test_client.tso_done_rpc = brw_client_done_rpc;
};
-srpc_service_t brw_test_service;
+struct srpc_service brw_test_service;
+
void brw_init_test_service(void)
{
brw_test_service.sv_id = SRPC_SERVICE_BRW;
diff --git a/drivers/staging/lustre/lnet/selftest/conctl.c b/drivers/staging/lustre/lnet/selftest/conctl.c
index 79ee6c0..408c614 100644
--- a/drivers/staging/lustre/lnet/selftest/conctl.c
+++ b/drivers/staging/lustre/lnet/selftest/conctl.c
@@ -51,9 +51,9 @@
char *name;
int rc;
- if (!args->lstio_ses_idp || /* address for output sid */
- !args->lstio_ses_key || /* no key is specified */
- !args->lstio_ses_namep || /* session name */
+ if (!args->lstio_ses_idp || /* address for output sid */
+ !args->lstio_ses_key || /* no key is specified */
+ !args->lstio_ses_namep || /* session name */
args->lstio_ses_nmlen <= 0 ||
args->lstio_ses_nmlen > LST_NAME_SIZE)
return -EINVAL;
@@ -95,11 +95,11 @@
{
/* no checking of key */
- if (!args->lstio_ses_idp || /* address for output sid */
- !args->lstio_ses_keyp || /* address for output key */
- !args->lstio_ses_featp || /* address for output features */
- !args->lstio_ses_ndinfo || /* address for output ndinfo */
- !args->lstio_ses_namep || /* address for output name */
+ if (!args->lstio_ses_idp || /* address for output sid */
+ !args->lstio_ses_keyp || /* address for output key */
+ !args->lstio_ses_featp || /* address for output features */
+ !args->lstio_ses_ndinfo || /* address for output ndinfo */
+ !args->lstio_ses_namep || /* address for output name */
args->lstio_ses_nmlen <= 0 ||
args->lstio_ses_nmlen > LST_NAME_SIZE)
return -EINVAL;
@@ -125,7 +125,7 @@
if (!args->lstio_dbg_resultp)
return -EINVAL;
- if (args->lstio_dbg_namep && /* name of batch/group */
+ if (args->lstio_dbg_namep && /* name of batch/group */
(args->lstio_dbg_nmlen <= 0 ||
args->lstio_dbg_nmlen > LST_NAME_SIZE))
return -EINVAL;
@@ -326,7 +326,7 @@
if (args->lstio_grp_key != console_session.ses_key)
return -EACCES;
- if (!args->lstio_grp_idsp || /* array of ids */
+ if (!args->lstio_grp_idsp || /* array of ids */
args->lstio_grp_count <= 0 ||
!args->lstio_grp_resultp ||
!args->lstio_grp_featp ||
@@ -394,13 +394,13 @@
args->lstio_grp_nmlen > LST_NAME_SIZE)
return -EINVAL;
- if (!args->lstio_grp_entp && /* output: group entry */
- !args->lstio_grp_dentsp) /* output: node entry */
+ if (!args->lstio_grp_entp && /* output: group entry */
+ !args->lstio_grp_dentsp) /* output: node entry */
return -EINVAL;
- if (args->lstio_grp_dentsp) { /* have node entry */
- if (!args->lstio_grp_idxp || /* node index */
- !args->lstio_grp_ndentp) /* # of node entry */
+ if (args->lstio_grp_dentsp) { /* have node entry */
+ if (!args->lstio_grp_idxp || /* node index */
+ !args->lstio_grp_ndentp) /* # of node entry */
return -EINVAL;
if (copy_from_user(&ndent, args->lstio_grp_ndentp,
@@ -612,18 +612,18 @@
if (args->lstio_bat_key != console_session.ses_key)
return -EACCES;
- if (!args->lstio_bat_namep || /* batch name */
+ if (!args->lstio_bat_namep || /* batch name */
args->lstio_bat_nmlen <= 0 ||
args->lstio_bat_nmlen > LST_NAME_SIZE)
return -EINVAL;
- if (!args->lstio_bat_entp && /* output: batch entry */
- !args->lstio_bat_dentsp) /* output: node entry */
+ if (!args->lstio_bat_entp && /* output: batch entry */
+ !args->lstio_bat_dentsp) /* output: node entry */
return -EINVAL;
- if (args->lstio_bat_dentsp) { /* have node entry */
- if (!args->lstio_bat_idxp || /* node index */
- !args->lstio_bat_ndentp) /* # of node entry */
+ if (args->lstio_bat_dentsp) { /* have node entry */
+ if (!args->lstio_bat_idxp || /* node index */
+ !args->lstio_bat_ndentp) /* # of node entry */
return -EINVAL;
if (copy_from_user(&index, args->lstio_bat_idxp,
@@ -722,18 +722,18 @@
if (!args->lstio_tes_resultp ||
!args->lstio_tes_retp ||
- !args->lstio_tes_bat_name || /* no specified batch */
+ !args->lstio_tes_bat_name || /* no specified batch */
args->lstio_tes_bat_nmlen <= 0 ||
args->lstio_tes_bat_nmlen > LST_NAME_SIZE ||
- !args->lstio_tes_sgrp_name || /* no source group */
+ !args->lstio_tes_sgrp_name || /* no source group */
args->lstio_tes_sgrp_nmlen <= 0 ||
args->lstio_tes_sgrp_nmlen > LST_NAME_SIZE ||
- !args->lstio_tes_dgrp_name || /* no target group */
+ !args->lstio_tes_dgrp_name || /* no target group */
args->lstio_tes_dgrp_nmlen <= 0 ||
args->lstio_tes_dgrp_nmlen > LST_NAME_SIZE)
return -EINVAL;
- if (!args->lstio_tes_loop || /* negative is infinite */
+ if (!args->lstio_tes_loop || /* negative is infinite */
args->lstio_tes_concur <= 0 ||
args->lstio_tes_dist <= 0 ||
args->lstio_tes_span <= 0)
@@ -743,7 +743,7 @@
if (args->lstio_tes_param &&
(args->lstio_tes_param_len <= 0 ||
args->lstio_tes_param_len >
- PAGE_SIZE - sizeof(lstcon_test_t)))
+ PAGE_SIZE - sizeof(struct lstcon_test)))
return -EINVAL;
LIBCFS_ALLOC(batch_name, args->lstio_tes_bat_nmlen + 1);
diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.c b/drivers/staging/lustre/lnet/selftest/conrpc.c
index 35a227d..6f68758 100644
--- a/drivers/staging/lustre/lnet/selftest/conrpc.c
+++ b/drivers/staging/lustre/lnet/selftest/conrpc.c
@@ -46,13 +46,13 @@
#include "conrpc.h"
#include "console.h"
-void lstcon_rpc_stat_reply(lstcon_rpc_trans_t *, srpc_msg_t *,
- lstcon_node_t *, lstcon_trans_stat_t *);
+void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *,
+ struct lstcon_node *, lstcon_trans_stat_t *);
static void
-lstcon_rpc_done(srpc_client_rpc_t *rpc)
+lstcon_rpc_done(struct srpc_client_rpc *rpc)
{
- lstcon_rpc_t *crpc = (lstcon_rpc_t *)rpc->crpc_priv;
+ struct lstcon_rpc *crpc = (struct lstcon_rpc *)rpc->crpc_priv;
LASSERT(crpc && rpc == crpc->crp_rpc);
LASSERT(crpc->crp_posted && !crpc->crp_finished);
@@ -90,8 +90,8 @@
}
static int
-lstcon_rpc_init(lstcon_node_t *nd, int service, unsigned feats,
- int bulk_npg, int bulk_len, int embedded, lstcon_rpc_t *crpc)
+lstcon_rpc_init(struct lstcon_node *nd, int service, unsigned feats,
+ int bulk_npg, int bulk_len, int embedded, struct lstcon_rpc *crpc)
{
crpc->crp_rpc = sfw_create_rpc(nd->nd_id, service,
feats, bulk_npg, bulk_len,
@@ -115,16 +115,16 @@
}
static int
-lstcon_rpc_prep(lstcon_node_t *nd, int service, unsigned feats,
- int bulk_npg, int bulk_len, lstcon_rpc_t **crpcpp)
+lstcon_rpc_prep(struct lstcon_node *nd, int service, unsigned feats,
+ int bulk_npg, int bulk_len, struct lstcon_rpc **crpcpp)
{
- lstcon_rpc_t *crpc = NULL;
+ struct lstcon_rpc *crpc = NULL;
int rc;
spin_lock(&console_session.ses_rpc_lock);
crpc = list_first_entry_or_null(&console_session.ses_rpc_freelist,
- lstcon_rpc_t, crp_link);
+ struct lstcon_rpc, crp_link);
if (crpc)
list_del_init(&crpc->crp_link);
@@ -148,9 +148,9 @@
}
void
-lstcon_rpc_put(lstcon_rpc_t *crpc)
+lstcon_rpc_put(struct lstcon_rpc *crpc)
{
- srpc_bulk_t *bulk = &crpc->crp_rpc->crpc_bulk;
+ struct srpc_bulk *bulk = &crpc->crp_rpc->crpc_bulk;
int i;
LASSERT(list_empty(&crpc->crp_link));
@@ -183,9 +183,9 @@
}
static void
-lstcon_rpc_post(lstcon_rpc_t *crpc)
+lstcon_rpc_post(struct lstcon_rpc *crpc)
{
- lstcon_rpc_trans_t *trans = crpc->crp_trans;
+ struct lstcon_rpc_trans *trans = crpc->crp_trans;
LASSERT(trans);
@@ -236,9 +236,9 @@
int
lstcon_rpc_trans_prep(struct list_head *translist, int transop,
- lstcon_rpc_trans_t **transpp)
+ struct lstcon_rpc_trans **transpp)
{
- lstcon_rpc_trans_t *trans;
+ struct lstcon_rpc_trans *trans;
if (translist) {
list_for_each_entry(trans, translist, tas_link) {
@@ -278,26 +278,26 @@
}
void
-lstcon_rpc_trans_addreq(lstcon_rpc_trans_t *trans, lstcon_rpc_t *crpc)
+lstcon_rpc_trans_addreq(struct lstcon_rpc_trans *trans, struct lstcon_rpc *crpc)
{
list_add_tail(&crpc->crp_link, &trans->tas_rpcs_list);
crpc->crp_trans = trans;
}
void
-lstcon_rpc_trans_abort(lstcon_rpc_trans_t *trans, int error)
+lstcon_rpc_trans_abort(struct lstcon_rpc_trans *trans, int error)
{
- srpc_client_rpc_t *rpc;
- lstcon_rpc_t *crpc;
- lstcon_node_t *nd;
+ struct srpc_client_rpc *rpc;
+ struct lstcon_rpc *crpc;
+ struct lstcon_node *nd;
list_for_each_entry(crpc, &trans->tas_rpcs_list, crp_link) {
rpc = crpc->crp_rpc;
spin_lock(&rpc->crpc_lock);
- if (!crpc->crp_posted || /* not posted */
- crpc->crp_stamp) { /* rpc done or aborted already */
+ if (!crpc->crp_posted || /* not posted */
+ crpc->crp_stamp) { /* rpc done or aborted already */
if (!crpc->crp_stamp) {
crpc->crp_stamp = cfs_time_current();
crpc->crp_status = -EINTR;
@@ -326,7 +326,7 @@
}
static int
-lstcon_rpc_trans_check(lstcon_rpc_trans_t *trans)
+lstcon_rpc_trans_check(struct lstcon_rpc_trans *trans)
{
if (console_session.ses_shutdown &&
!list_empty(&trans->tas_olink)) /* Not an end session RPC */
@@ -336,9 +336,9 @@
}
int
-lstcon_rpc_trans_postwait(lstcon_rpc_trans_t *trans, int timeout)
+lstcon_rpc_trans_postwait(struct lstcon_rpc_trans *trans, int timeout)
{
- lstcon_rpc_t *crpc;
+ struct lstcon_rpc *crpc;
int rc;
if (list_empty(&trans->tas_rpcs_list))
@@ -386,11 +386,11 @@
}
static int
-lstcon_rpc_get_reply(lstcon_rpc_t *crpc, srpc_msg_t **msgpp)
+lstcon_rpc_get_reply(struct lstcon_rpc *crpc, struct srpc_msg **msgpp)
{
- lstcon_node_t *nd = crpc->crp_node;
- srpc_client_rpc_t *rpc = crpc->crp_rpc;
- srpc_generic_reply_t *rep;
+ struct lstcon_node *nd = crpc->crp_node;
+ struct srpc_client_rpc *rpc = crpc->crp_rpc;
+ struct srpc_generic_reply *rep;
LASSERT(nd && rpc);
LASSERT(crpc->crp_stamp);
@@ -423,10 +423,10 @@
}
void
-lstcon_rpc_trans_stat(lstcon_rpc_trans_t *trans, lstcon_trans_stat_t *stat)
+lstcon_rpc_trans_stat(struct lstcon_rpc_trans *trans, lstcon_trans_stat_t *stat)
{
- lstcon_rpc_t *crpc;
- srpc_msg_t *rep;
+ struct lstcon_rpc *crpc;
+ struct srpc_msg *rep;
int error;
LASSERT(stat);
@@ -466,17 +466,17 @@
}
int
-lstcon_rpc_trans_interpreter(lstcon_rpc_trans_t *trans,
+lstcon_rpc_trans_interpreter(struct lstcon_rpc_trans *trans,
struct list_head __user *head_up,
lstcon_rpc_readent_func_t readent)
{
struct list_head tmp;
struct list_head __user *next;
lstcon_rpc_ent_t *ent;
- srpc_generic_reply_t *rep;
- lstcon_rpc_t *crpc;
- srpc_msg_t *msg;
- lstcon_node_t *nd;
+ struct srpc_generic_reply *rep;
+ struct lstcon_rpc *crpc;
+ struct srpc_msg *msg;
+ struct lstcon_node *nd;
long dur;
struct timeval tv;
int error;
@@ -520,7 +520,7 @@
continue;
/* RPC is done */
- rep = (srpc_generic_reply_t *)&msg->msg_body.reply;
+ rep = (struct srpc_generic_reply *)&msg->msg_body.reply;
if (copy_to_user(&ent->rpe_sid, &rep->sid, sizeof(lst_sid_t)) ||
copy_to_user(&ent->rpe_fwk_errno, &rep->status,
@@ -531,7 +531,6 @@
continue;
error = readent(trans->tas_opc, msg, ent);
-
if (error)
return error;
}
@@ -540,11 +539,11 @@
}
void
-lstcon_rpc_trans_destroy(lstcon_rpc_trans_t *trans)
+lstcon_rpc_trans_destroy(struct lstcon_rpc_trans *trans)
{
- srpc_client_rpc_t *rpc;
- lstcon_rpc_t *crpc;
- lstcon_rpc_t *tmp;
+ struct srpc_client_rpc *rpc;
+ struct lstcon_rpc *crpc;
+ struct lstcon_rpc *tmp;
int count = 0;
list_for_each_entry_safe(crpc, tmp, &trans->tas_rpcs_list, crp_link) {
@@ -563,10 +562,10 @@
}
/*
- * rpcs can be still not callbacked (even LNetMDUnlink is called)
- * because huge timeout for inaccessible network, don't make
- * user wait for them, just abandon them, they will be recycled
- * in callback
+ * rpcs can be still not callbacked (even LNetMDUnlink is
+ * called) because huge timeout for inaccessible network,
+ * don't make user wait for them, just abandon them, they
+ * will be recycled in callback
*/
LASSERT(crpc->crp_status);
@@ -593,11 +592,11 @@
}
int
-lstcon_sesrpc_prep(lstcon_node_t *nd, int transop,
- unsigned feats, lstcon_rpc_t **crpc)
+lstcon_sesrpc_prep(struct lstcon_node *nd, int transop,
+ unsigned feats, struct lstcon_rpc **crpc)
{
- srpc_mksn_reqst_t *msrq;
- srpc_rmsn_reqst_t *rsrq;
+ struct srpc_mksn_reqst *msrq;
+ struct srpc_rmsn_reqst *rsrq;
int rc;
switch (transop) {
@@ -632,9 +631,9 @@
}
int
-lstcon_dbgrpc_prep(lstcon_node_t *nd, unsigned feats, lstcon_rpc_t **crpc)
+lstcon_dbgrpc_prep(struct lstcon_node *nd, unsigned feats, struct lstcon_rpc **crpc)
{
- srpc_debug_reqst_t *drq;
+ struct srpc_debug_reqst *drq;
int rc;
rc = lstcon_rpc_prep(nd, SRPC_SERVICE_DEBUG, feats, 0, 0, crpc);
@@ -650,11 +649,11 @@
}
int
-lstcon_batrpc_prep(lstcon_node_t *nd, int transop, unsigned feats,
- lstcon_tsb_hdr_t *tsb, lstcon_rpc_t **crpc)
+lstcon_batrpc_prep(struct lstcon_node *nd, int transop, unsigned feats,
+ struct lstcon_tsb_hdr *tsb, struct lstcon_rpc **crpc)
{
- lstcon_batch_t *batch;
- srpc_batch_reqst_t *brq;
+ struct lstcon_batch *batch;
+ struct srpc_batch_reqst *brq;
int rc;
rc = lstcon_rpc_prep(nd, SRPC_SERVICE_BATCH, feats, 0, 0, crpc);
@@ -676,16 +675,16 @@
LASSERT(!tsb->tsb_index);
- batch = (lstcon_batch_t *)tsb;
+ batch = (struct lstcon_batch *)tsb;
brq->bar_arg = batch->bat_arg;
return 0;
}
int
-lstcon_statrpc_prep(lstcon_node_t *nd, unsigned feats, lstcon_rpc_t **crpc)
+lstcon_statrpc_prep(struct lstcon_node *nd, unsigned feats, struct lstcon_rpc **crpc)
{
- srpc_stat_reqst_t *srq;
+ struct srpc_stat_reqst *srq;
int rc;
rc = lstcon_rpc_prep(nd, SRPC_SERVICE_QUERY_STAT, feats, 0, 0, crpc);
@@ -716,12 +715,12 @@
}
static int
-lstcon_dstnodes_prep(lstcon_group_t *grp, int idx,
+lstcon_dstnodes_prep(struct lstcon_group *grp, int idx,
int dist, int span, int nkiov, lnet_kiov_t *kiov)
{
lnet_process_id_packed_t *pid;
- lstcon_ndlink_t *ndl;
- lstcon_node_t *nd;
+ struct lstcon_ndlink *ndl;
+ struct lstcon_node *nd;
int start;
int end;
int i = 0;
@@ -770,9 +769,9 @@
}
static int
-lstcon_pingrpc_prep(lst_test_ping_param_t *param, srpc_test_reqst_t *req)
+lstcon_pingrpc_prep(lst_test_ping_param_t *param, struct srpc_test_reqst *req)
{
- test_ping_req_t *prq = &req->tsr_u.ping;
+ struct test_ping_req *prq = &req->tsr_u.ping;
prq->png_size = param->png_size;
prq->png_flags = param->png_flags;
@@ -781,9 +780,9 @@
}
static int
-lstcon_bulkrpc_v0_prep(lst_test_bulk_param_t *param, srpc_test_reqst_t *req)
+lstcon_bulkrpc_v0_prep(lst_test_bulk_param_t *param, struct srpc_test_reqst *req)
{
- test_bulk_req_t *brq = &req->tsr_u.bulk_v0;
+ struct test_bulk_req *brq = &req->tsr_u.bulk_v0;
brq->blk_opc = param->blk_opc;
brq->blk_npg = (param->blk_size + PAGE_SIZE - 1) /
@@ -794,9 +793,9 @@
}
static int
-lstcon_bulkrpc_v1_prep(lst_test_bulk_param_t *param, srpc_test_reqst_t *req)
+lstcon_bulkrpc_v1_prep(lst_test_bulk_param_t *param, struct srpc_test_reqst *req)
{
- test_bulk_req_v1_t *brq = &req->tsr_u.bulk_v1;
+ struct test_bulk_req_v1 *brq = &req->tsr_u.bulk_v1;
brq->blk_opc = param->blk_opc;
brq->blk_flags = param->blk_flags;
@@ -807,13 +806,13 @@
}
int
-lstcon_testrpc_prep(lstcon_node_t *nd, int transop, unsigned feats,
- lstcon_test_t *test, lstcon_rpc_t **crpc)
+lstcon_testrpc_prep(struct lstcon_node *nd, int transop, unsigned feats,
+ struct lstcon_test *test, struct lstcon_rpc **crpc)
{
- lstcon_group_t *sgrp = test->tes_src_grp;
- lstcon_group_t *dgrp = test->tes_dst_grp;
- srpc_test_reqst_t *trq;
- srpc_bulk_t *bulk;
+ struct lstcon_group *sgrp = test->tes_src_grp;
+ struct lstcon_group *dgrp = test->tes_dst_grp;
+ struct srpc_test_reqst *trq;
+ struct srpc_bulk *bulk;
int i;
int npg = 0;
int nob = 0;
@@ -841,7 +840,6 @@
trq->tsr_ndest = 0;
trq->tsr_loop = nmax * test->tes_dist * test->tes_concur;
-
} else {
bulk = &(*crpc)->crp_rpc->crpc_bulk;
@@ -917,10 +915,10 @@
}
static int
-lstcon_sesnew_stat_reply(lstcon_rpc_trans_t *trans,
- lstcon_node_t *nd, srpc_msg_t *reply)
+lstcon_sesnew_stat_reply(struct lstcon_rpc_trans *trans,
+ struct lstcon_node *nd, struct srpc_msg *reply)
{
- srpc_mksn_reply_t *mksn_rep = &reply->msg_body.mksn_reply;
+ struct srpc_mksn_reply *mksn_rep = &reply->msg_body.mksn_reply;
int status = mksn_rep->mksn_status;
if (!status &&
@@ -940,7 +938,7 @@
if (!trans->tas_feats_updated) {
spin_lock(&console_session.ses_rpc_lock);
- if (!trans->tas_feats_updated) { /* recheck with lock */
+ if (!trans->tas_feats_updated) { /* recheck with lock */
trans->tas_feats_updated = 1;
trans->tas_features = reply->msg_ses_feats;
}
@@ -964,14 +962,14 @@
}
void
-lstcon_rpc_stat_reply(lstcon_rpc_trans_t *trans, srpc_msg_t *msg,
- lstcon_node_t *nd, lstcon_trans_stat_t *stat)
+lstcon_rpc_stat_reply(struct lstcon_rpc_trans *trans, struct srpc_msg *msg,
+ struct lstcon_node *nd, lstcon_trans_stat_t *stat)
{
- srpc_rmsn_reply_t *rmsn_rep;
- srpc_debug_reply_t *dbg_rep;
- srpc_batch_reply_t *bat_rep;
- srpc_test_reply_t *test_rep;
- srpc_stat_reply_t *stat_rep;
+ struct srpc_rmsn_reply *rmsn_rep;
+ struct srpc_debug_reply *dbg_rep;
+ struct srpc_batch_reply *bat_rep;
+ struct srpc_test_reply *test_rep;
+ struct srpc_stat_reply *stat_rep;
int rc = 0;
switch (trans->tas_opc) {
@@ -1085,12 +1083,12 @@
lstcon_rpc_trans_ndlist(struct list_head *ndlist,
struct list_head *translist, int transop,
void *arg, lstcon_rpc_cond_func_t condition,
- lstcon_rpc_trans_t **transpp)
+ struct lstcon_rpc_trans **transpp)
{
- lstcon_rpc_trans_t *trans;
- lstcon_ndlink_t *ndl;
- lstcon_node_t *nd;
- lstcon_rpc_t *rpc;
+ struct lstcon_rpc_trans *trans;
+ struct lstcon_ndlink *ndl;
+ struct lstcon_node *nd;
+ struct lstcon_rpc *rpc;
unsigned feats;
int rc;
@@ -1130,14 +1128,16 @@
case LST_TRANS_TSBCLIADD:
case LST_TRANS_TSBSRVADD:
rc = lstcon_testrpc_prep(nd, transop, feats,
- (lstcon_test_t *)arg, &rpc);
+ (struct lstcon_test *)arg,
+ &rpc);
break;
case LST_TRANS_TSBRUN:
case LST_TRANS_TSBSTOP:
case LST_TRANS_TSBCLIQRY:
case LST_TRANS_TSBSRVQRY:
rc = lstcon_batrpc_prep(nd, transop, feats,
- (lstcon_tsb_hdr_t *)arg, &rpc);
+ (struct lstcon_tsb_hdr *)arg,
+ &rpc);
break;
case LST_TRANS_STATQRY:
rc = lstcon_statrpc_prep(nd, feats, &rpc);
@@ -1170,17 +1170,18 @@
lstcon_rpc_pinger(void *arg)
{
struct stt_timer *ptimer = (struct stt_timer *)arg;
- lstcon_rpc_trans_t *trans;
- lstcon_rpc_t *crpc;
- srpc_msg_t *rep;
- srpc_debug_reqst_t *drq;
- lstcon_ndlink_t *ndl;
- lstcon_node_t *nd;
+ struct lstcon_rpc_trans *trans;
+ struct lstcon_rpc *crpc;
+ struct srpc_msg *rep;
+ struct srpc_debug_reqst *drq;
+ struct lstcon_ndlink *ndl;
+ struct lstcon_node *nd;
int intv;
int count = 0;
int rc;
- /* RPC pinger is a special case of transaction,
+ /*
+ * RPC pinger is a special case of transaction,
* it's called by timer at 8 seconds interval.
*/
mutex_lock(&console_session.ses_mutex);
@@ -1326,9 +1327,9 @@
void
lstcon_rpc_cleanup_wait(void)
{
- lstcon_rpc_trans_t *trans;
- lstcon_rpc_t *crpc;
- lstcon_rpc_t *temp;
+ struct lstcon_rpc_trans *trans;
+ struct lstcon_rpc *crpc;
+ struct lstcon_rpc *temp;
struct list_head *pacer;
struct list_head zlist;
@@ -1338,7 +1339,7 @@
while (!list_empty(&console_session.ses_trans_list)) {
list_for_each(pacer, &console_session.ses_trans_list) {
- trans = list_entry(pacer, lstcon_rpc_trans_t,
+ trans = list_entry(pacer, struct lstcon_rpc_trans,
tas_link);
CDEBUG(D_NET, "Session closed, wakeup transaction %s\n",
@@ -1370,7 +1371,7 @@
list_for_each_entry_safe(crpc, temp, &zlist, crp_link) {
list_del(&crpc->crp_link);
- LIBCFS_FREE(crpc, sizeof(lstcon_rpc_t));
+ LIBCFS_FREE(crpc, sizeof(struct lstcon_rpc));
}
}
diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.h b/drivers/staging/lustre/lnet/selftest/conrpc.h
index 3e7839d..90c3385 100644
--- a/drivers/staging/lustre/lnet/selftest/conrpc.h
+++ b/drivers/staging/lustre/lnet/selftest/conrpc.h
@@ -63,9 +63,9 @@
struct lstcon_test;
struct lstcon_node;
-typedef struct lstcon_rpc {
+struct lstcon_rpc {
struct list_head crp_link; /* chain on rpc transaction */
- srpc_client_rpc_t *crp_rpc; /* client rpc */
+ struct srpc_client_rpc *crp_rpc; /* client rpc */
struct lstcon_node *crp_node; /* destination node */
struct lstcon_rpc_trans *crp_trans; /* conrpc transaction */
@@ -76,9 +76,9 @@
unsigned int crp_embedded:1;
int crp_status; /* console rpc errors */
unsigned long crp_stamp; /* replied time stamp */
-} lstcon_rpc_t;
+};
-typedef struct lstcon_rpc_trans {
+struct lstcon_rpc_trans {
struct list_head tas_olink; /* link chain on owner list */
struct list_head tas_link; /* link chain on global list */
int tas_opc; /* operation code of transaction */
@@ -87,7 +87,7 @@
wait_queue_head_t tas_waitq; /* wait queue head */
atomic_t tas_remaining; /* # of un-scheduled rpcs */
struct list_head tas_rpcs_list; /* queued requests */
-} lstcon_rpc_trans_t;
+};
#define LST_TRANS_PRIVATE 0x1000
@@ -106,35 +106,35 @@
#define LST_TRANS_STATQRY 0x21
typedef int (*lstcon_rpc_cond_func_t)(int, struct lstcon_node *, void *);
-typedef int (*lstcon_rpc_readent_func_t)(int, srpc_msg_t *,
+typedef int (*lstcon_rpc_readent_func_t)(int, struct srpc_msg *,
lstcon_rpc_ent_t __user *);
int lstcon_sesrpc_prep(struct lstcon_node *nd, int transop,
- unsigned version, lstcon_rpc_t **crpc);
+ unsigned version, struct lstcon_rpc **crpc);
int lstcon_dbgrpc_prep(struct lstcon_node *nd,
- unsigned version, lstcon_rpc_t **crpc);
+ unsigned version, struct lstcon_rpc **crpc);
int lstcon_batrpc_prep(struct lstcon_node *nd, int transop, unsigned version,
- struct lstcon_tsb_hdr *tsb, lstcon_rpc_t **crpc);
+ struct lstcon_tsb_hdr *tsb, struct lstcon_rpc **crpc);
int lstcon_testrpc_prep(struct lstcon_node *nd, int transop, unsigned version,
- struct lstcon_test *test, lstcon_rpc_t **crpc);
+ struct lstcon_test *test, struct lstcon_rpc **crpc);
int lstcon_statrpc_prep(struct lstcon_node *nd, unsigned version,
- lstcon_rpc_t **crpc);
-void lstcon_rpc_put(lstcon_rpc_t *crpc);
+ struct lstcon_rpc **crpc);
+void lstcon_rpc_put(struct lstcon_rpc *crpc);
int lstcon_rpc_trans_prep(struct list_head *translist,
- int transop, lstcon_rpc_trans_t **transpp);
+ int transop, struct lstcon_rpc_trans **transpp);
int lstcon_rpc_trans_ndlist(struct list_head *ndlist,
struct list_head *translist, int transop,
void *arg, lstcon_rpc_cond_func_t condition,
- lstcon_rpc_trans_t **transpp);
-void lstcon_rpc_trans_stat(lstcon_rpc_trans_t *trans,
+ struct lstcon_rpc_trans **transpp);
+void lstcon_rpc_trans_stat(struct lstcon_rpc_trans *trans,
lstcon_trans_stat_t *stat);
-int lstcon_rpc_trans_interpreter(lstcon_rpc_trans_t *trans,
+int lstcon_rpc_trans_interpreter(struct lstcon_rpc_trans *trans,
struct list_head __user *head_up,
lstcon_rpc_readent_func_t readent);
-void lstcon_rpc_trans_abort(lstcon_rpc_trans_t *trans, int error);
-void lstcon_rpc_trans_destroy(lstcon_rpc_trans_t *trans);
-void lstcon_rpc_trans_addreq(lstcon_rpc_trans_t *trans, lstcon_rpc_t *req);
-int lstcon_rpc_trans_postwait(lstcon_rpc_trans_t *trans, int timeout);
+void lstcon_rpc_trans_abort(struct lstcon_rpc_trans *trans, int error);
+void lstcon_rpc_trans_destroy(struct lstcon_rpc_trans *trans);
+void lstcon_rpc_trans_addreq(struct lstcon_rpc_trans *trans, struct lstcon_rpc *req);
+int lstcon_rpc_trans_postwait(struct lstcon_rpc_trans *trans, int timeout);
int lstcon_rpc_pinger_start(void);
void lstcon_rpc_pinger_stop(void);
void lstcon_rpc_cleanup_wait(void);
diff --git a/drivers/staging/lustre/lnet/selftest/console.c b/drivers/staging/lustre/lnet/selftest/console.c
index 1a923ea..a03e52d 100644
--- a/drivers/staging/lustre/lnet/selftest/console.c
+++ b/drivers/staging/lustre/lnet/selftest/console.c
@@ -61,7 +61,7 @@
struct lstcon_session console_session;
static void
-lstcon_node_get(lstcon_node_t *nd)
+lstcon_node_get(struct lstcon_node *nd)
{
LASSERT(nd->nd_ref >= 1);
@@ -69,9 +69,9 @@
}
static int
-lstcon_node_find(lnet_process_id_t id, lstcon_node_t **ndpp, int create)
+lstcon_node_find(lnet_process_id_t id, struct lstcon_node **ndpp, int create)
{
- lstcon_ndlink_t *ndl;
+ struct lstcon_ndlink *ndl;
unsigned int idx = LNET_NIDADDR(id.nid) % LST_GLOBAL_HASHSIZE;
LASSERT(id.nid != LNET_NID_ANY);
@@ -90,11 +90,11 @@
if (!create)
return -ENOENT;
- LIBCFS_ALLOC(*ndpp, sizeof(lstcon_node_t) + sizeof(lstcon_ndlink_t));
+ LIBCFS_ALLOC(*ndpp, sizeof(struct lstcon_node) + sizeof(struct lstcon_ndlink));
if (!*ndpp)
return -ENOMEM;
- ndl = (lstcon_ndlink_t *)(*ndpp + 1);
+ ndl = (struct lstcon_ndlink *)(*ndpp + 1);
ndl->ndl_node = *ndpp;
@@ -103,7 +103,7 @@
ndl->ndl_node->nd_stamp = cfs_time_current();
ndl->ndl_node->nd_state = LST_NODE_UNKNOWN;
ndl->ndl_node->nd_timeout = 0;
- memset(&ndl->ndl_node->nd_ping, 0, sizeof(lstcon_rpc_t));
+ memset(&ndl->ndl_node->nd_ping, 0, sizeof(struct lstcon_rpc));
/*
* queued in global hash & list, no refcount is taken by
@@ -117,16 +117,16 @@
}
static void
-lstcon_node_put(lstcon_node_t *nd)
+lstcon_node_put(struct lstcon_node *nd)
{
- lstcon_ndlink_t *ndl;
+ struct lstcon_ndlink *ndl;
LASSERT(nd->nd_ref > 0);
if (--nd->nd_ref > 0)
return;
- ndl = (lstcon_ndlink_t *)(nd + 1);
+ ndl = (struct lstcon_ndlink *)(nd + 1);
LASSERT(!list_empty(&ndl->ndl_link));
LASSERT(!list_empty(&ndl->ndl_hlink));
@@ -135,16 +135,16 @@
list_del(&ndl->ndl_link);
list_del(&ndl->ndl_hlink);
- LIBCFS_FREE(nd, sizeof(lstcon_node_t) + sizeof(lstcon_ndlink_t));
+ LIBCFS_FREE(nd, sizeof(struct lstcon_node) + sizeof(struct lstcon_ndlink));
}
static int
lstcon_ndlink_find(struct list_head *hash,
- lnet_process_id_t id, lstcon_ndlink_t **ndlpp, int create)
+ lnet_process_id_t id, struct lstcon_ndlink **ndlpp, int create)
{
unsigned int idx = LNET_NIDADDR(id.nid) % LST_NODE_HASHSIZE;
- lstcon_ndlink_t *ndl;
- lstcon_node_t *nd;
+ struct lstcon_ndlink *ndl;
+ struct lstcon_node *nd;
int rc;
if (id.nid == LNET_NID_ANY)
@@ -168,7 +168,7 @@
if (rc)
return rc;
- LIBCFS_ALLOC(ndl, sizeof(lstcon_ndlink_t));
+ LIBCFS_ALLOC(ndl, sizeof(struct lstcon_ndlink));
if (!ndl) {
lstcon_node_put(nd);
return -ENOMEM;
@@ -184,7 +184,7 @@
}
static void
-lstcon_ndlink_release(lstcon_ndlink_t *ndl)
+lstcon_ndlink_release(struct lstcon_ndlink *ndl)
{
LASSERT(list_empty(&ndl->ndl_link));
LASSERT(!list_empty(&ndl->ndl_hlink));
@@ -196,12 +196,12 @@
}
static int
-lstcon_group_alloc(char *name, lstcon_group_t **grpp)
+lstcon_group_alloc(char *name, struct lstcon_group **grpp)
{
- lstcon_group_t *grp;
+ struct lstcon_group *grp;
int i;
- LIBCFS_ALLOC(grp, offsetof(lstcon_group_t,
+ LIBCFS_ALLOC(grp, offsetof(struct lstcon_group,
grp_ndl_hash[LST_NODE_HASHSIZE]));
if (!grp)
return -ENOMEM;
@@ -209,7 +209,7 @@
grp->grp_ref = 1;
if (name) {
if (strlen(name) > sizeof(grp->grp_name) - 1) {
- LIBCFS_FREE(grp, offsetof(lstcon_group_t,
+ LIBCFS_FREE(grp, offsetof(struct lstcon_group,
grp_ndl_hash[LST_NODE_HASHSIZE]));
return -E2BIG;
}
@@ -229,18 +229,18 @@
}
static void
-lstcon_group_addref(lstcon_group_t *grp)
+lstcon_group_addref(struct lstcon_group *grp)
{
grp->grp_ref++;
}
-static void lstcon_group_ndlink_release(lstcon_group_t *, lstcon_ndlink_t *);
+static void lstcon_group_ndlink_release(struct lstcon_group *, struct lstcon_ndlink *);
static void
-lstcon_group_drain(lstcon_group_t *grp, int keep)
+lstcon_group_drain(struct lstcon_group *grp, int keep)
{
- lstcon_ndlink_t *ndl;
- lstcon_ndlink_t *tmp;
+ struct lstcon_ndlink *ndl;
+ struct lstcon_ndlink *tmp;
list_for_each_entry_safe(ndl, tmp, &grp->grp_ndl_list, ndl_link) {
if (!(ndl->ndl_node->nd_state & keep))
@@ -249,7 +249,7 @@
}
static void
-lstcon_group_decref(lstcon_group_t *grp)
+lstcon_group_decref(struct lstcon_group *grp)
{
int i;
@@ -264,20 +264,20 @@
for (i = 0; i < LST_NODE_HASHSIZE; i++)
LASSERT(list_empty(&grp->grp_ndl_hash[i]));
- LIBCFS_FREE(grp, offsetof(lstcon_group_t,
+ LIBCFS_FREE(grp, offsetof(struct lstcon_group,
grp_ndl_hash[LST_NODE_HASHSIZE]));
}
static int
-lstcon_group_find(const char *name, lstcon_group_t **grpp)
+lstcon_group_find(const char *name, struct lstcon_group **grpp)
{
- lstcon_group_t *grp;
+ struct lstcon_group *grp;
list_for_each_entry(grp, &console_session.ses_grp_list, grp_link) {
if (strncmp(grp->grp_name, name, LST_NAME_SIZE))
continue;
- lstcon_group_addref(grp); /* +1 ref for caller */
+ lstcon_group_addref(grp); /* +1 ref for caller */
*grpp = grp;
return 0;
}
@@ -286,8 +286,8 @@
}
static int
-lstcon_group_ndlink_find(lstcon_group_t *grp, lnet_process_id_t id,
- lstcon_ndlink_t **ndlpp, int create)
+lstcon_group_ndlink_find(struct lstcon_group *grp, lnet_process_id_t id,
+ struct lstcon_ndlink **ndlpp, int create)
{
int rc;
@@ -305,7 +305,7 @@
}
static void
-lstcon_group_ndlink_release(lstcon_group_t *grp, lstcon_ndlink_t *ndl)
+lstcon_group_ndlink_release(struct lstcon_group *grp, struct lstcon_ndlink *ndl)
{
list_del_init(&ndl->ndl_link);
lstcon_ndlink_release(ndl);
@@ -313,8 +313,8 @@
}
static void
-lstcon_group_ndlink_move(lstcon_group_t *old,
- lstcon_group_t *new, lstcon_ndlink_t *ndl)
+lstcon_group_ndlink_move(struct lstcon_group *old,
+ struct lstcon_group *new, struct lstcon_ndlink *ndl)
{
unsigned int idx = LNET_NIDADDR(ndl->ndl_node->nd_id.nid) %
LST_NODE_HASHSIZE;
@@ -329,21 +329,21 @@
}
static void
-lstcon_group_move(lstcon_group_t *old, lstcon_group_t *new)
+lstcon_group_move(struct lstcon_group *old, struct lstcon_group *new)
{
- lstcon_ndlink_t *ndl;
+ struct lstcon_ndlink *ndl;
while (!list_empty(&old->grp_ndl_list)) {
ndl = list_entry(old->grp_ndl_list.next,
- lstcon_ndlink_t, ndl_link);
+ struct lstcon_ndlink, ndl_link);
lstcon_group_ndlink_move(old, new, ndl);
}
}
static int
-lstcon_sesrpc_condition(int transop, lstcon_node_t *nd, void *arg)
+lstcon_sesrpc_condition(int transop, struct lstcon_node *nd, void *arg)
{
- lstcon_group_t *grp = (lstcon_group_t *)arg;
+ struct lstcon_group *grp = (struct lstcon_group *)arg;
switch (transop) {
case LST_TRANS_SESNEW:
@@ -370,10 +370,10 @@
}
static int
-lstcon_sesrpc_readent(int transop, srpc_msg_t *msg,
+lstcon_sesrpc_readent(int transop, struct srpc_msg *msg,
lstcon_rpc_ent_t __user *ent_up)
{
- srpc_debug_reply_t *rep;
+ struct srpc_debug_reply *rep;
switch (transop) {
case LST_TRANS_SESNEW:
@@ -399,13 +399,13 @@
}
static int
-lstcon_group_nodes_add(lstcon_group_t *grp,
+lstcon_group_nodes_add(struct lstcon_group *grp,
int count, lnet_process_id_t __user *ids_up,
unsigned *featp, struct list_head __user *result_up)
{
- lstcon_rpc_trans_t *trans;
- lstcon_ndlink_t *ndl;
- lstcon_group_t *tmp;
+ struct lstcon_rpc_trans *trans;
+ struct lstcon_ndlink *ndl;
+ struct lstcon_group *tmp;
lnet_process_id_t id;
int i;
int rc;
@@ -466,13 +466,13 @@
}
static int
-lstcon_group_nodes_remove(lstcon_group_t *grp,
+lstcon_group_nodes_remove(struct lstcon_group *grp,
int count, lnet_process_id_t __user *ids_up,
struct list_head __user *result_up)
{
- lstcon_rpc_trans_t *trans;
- lstcon_ndlink_t *ndl;
- lstcon_group_t *tmp;
+ struct lstcon_rpc_trans *trans;
+ struct lstcon_ndlink *ndl;
+ struct lstcon_group *tmp;
lnet_process_id_t id;
int rc;
int i;
@@ -523,7 +523,7 @@
int
lstcon_group_add(char *name)
{
- lstcon_group_t *grp;
+ struct lstcon_group *grp;
int rc;
rc = lstcon_group_find(name, &grp) ? 0 : -EEXIST;
@@ -548,7 +548,7 @@
lstcon_nodes_add(char *name, int count, lnet_process_id_t __user *ids_up,
unsigned *featp, struct list_head __user *result_up)
{
- lstcon_group_t *grp;
+ struct lstcon_group *grp;
int rc;
LASSERT(count > 0);
@@ -578,8 +578,8 @@
int
lstcon_group_del(char *name)
{
- lstcon_rpc_trans_t *trans;
- lstcon_group_t *grp;
+ struct lstcon_rpc_trans *trans;
+ struct lstcon_group *grp;
int rc;
rc = lstcon_group_find(name, &grp);
@@ -621,7 +621,7 @@
int
lstcon_group_clean(char *name, int args)
{
- lstcon_group_t *grp = NULL;
+ struct lstcon_group *grp = NULL;
int rc;
rc = lstcon_group_find(name, &grp);
@@ -654,7 +654,7 @@
lstcon_nodes_remove(char *name, int count, lnet_process_id_t __user *ids_up,
struct list_head __user *result_up)
{
- lstcon_group_t *grp = NULL;
+ struct lstcon_group *grp = NULL;
int rc;
rc = lstcon_group_find(name, &grp);
@@ -683,8 +683,8 @@
int
lstcon_group_refresh(char *name, struct list_head __user *result_up)
{
- lstcon_rpc_trans_t *trans;
- lstcon_group_t *grp;
+ struct lstcon_rpc_trans *trans;
+ struct lstcon_group *grp;
int rc;
rc = lstcon_group_find(name, &grp);
@@ -725,7 +725,7 @@
int
lstcon_group_list(int index, int len, char __user *name_up)
{
- lstcon_group_t *grp;
+ struct lstcon_group *grp;
LASSERT(index >= 0);
LASSERT(name_up);
@@ -733,7 +733,7 @@
list_for_each_entry(grp, &console_session.ses_grp_list, grp_link) {
if (!index--) {
return copy_to_user(name_up, grp->grp_name, len) ?
- -EFAULT : 0;
+ -EFAULT : 0;
}
}
@@ -744,8 +744,8 @@
lstcon_nodes_getent(struct list_head *head, int *index_p,
int *count_p, lstcon_node_ent_t __user *dents_up)
{
- lstcon_ndlink_t *ndl;
- lstcon_node_t *nd;
+ struct lstcon_ndlink *ndl;
+ struct lstcon_node *nd;
int count = 0;
int index = 0;
@@ -786,8 +786,8 @@
lstcon_node_ent_t __user *dents_up)
{
lstcon_ndlist_ent_t *gentp;
- lstcon_group_t *grp;
- lstcon_ndlink_t *ndl;
+ struct lstcon_group *grp;
+ struct lstcon_ndlink *ndl;
int rc;
rc = lstcon_group_find(name, &grp);
@@ -828,9 +828,9 @@
}
static int
-lstcon_batch_find(const char *name, lstcon_batch_t **batpp)
+lstcon_batch_find(const char *name, struct lstcon_batch **batpp)
{
- lstcon_batch_t *bat;
+ struct lstcon_batch *bat;
list_for_each_entry(bat, &console_session.ses_bat_list, bat_link) {
if (!strncmp(bat->bat_name, name, LST_NAME_SIZE)) {
@@ -845,7 +845,7 @@
int
lstcon_batch_add(char *name)
{
- lstcon_batch_t *bat;
+ struct lstcon_batch *bat;
int i;
int rc;
@@ -855,7 +855,7 @@
return rc;
}
- LIBCFS_ALLOC(bat, sizeof(lstcon_batch_t));
+ LIBCFS_ALLOC(bat, sizeof(struct lstcon_batch));
if (!bat) {
CERROR("Can't allocate descriptor for batch %s\n", name);
return -ENOMEM;
@@ -865,7 +865,7 @@
sizeof(struct list_head) * LST_NODE_HASHSIZE);
if (!bat->bat_cli_hash) {
CERROR("Can't allocate hash for batch %s\n", name);
- LIBCFS_FREE(bat, sizeof(lstcon_batch_t));
+ LIBCFS_FREE(bat, sizeof(struct lstcon_batch));
return -ENOMEM;
}
@@ -875,7 +875,7 @@
if (!bat->bat_srv_hash) {
CERROR("Can't allocate hash for batch %s\n", name);
LIBCFS_FREE(bat->bat_cli_hash, LST_NODE_HASHSIZE);
- LIBCFS_FREE(bat, sizeof(lstcon_batch_t));
+ LIBCFS_FREE(bat, sizeof(struct lstcon_batch));
return -ENOMEM;
}
@@ -883,7 +883,7 @@
if (strlen(name) > sizeof(bat->bat_name) - 1) {
LIBCFS_FREE(bat->bat_srv_hash, LST_NODE_HASHSIZE);
LIBCFS_FREE(bat->bat_cli_hash, LST_NODE_HASHSIZE);
- LIBCFS_FREE(bat, sizeof(lstcon_batch_t));
+ LIBCFS_FREE(bat, sizeof(struct lstcon_batch));
return -E2BIG;
}
strncpy(bat->bat_name, name, sizeof(bat->bat_name));
@@ -911,7 +911,7 @@
int
lstcon_batch_list(int index, int len, char __user *name_up)
{
- lstcon_batch_t *bat;
+ struct lstcon_batch *bat;
LASSERT(name_up);
LASSERT(index >= 0);
@@ -934,9 +934,9 @@
lstcon_test_batch_ent_t *entp;
struct list_head *clilst;
struct list_head *srvlst;
- lstcon_test_t *test = NULL;
- lstcon_batch_t *bat;
- lstcon_ndlink_t *ndl;
+ struct lstcon_test *test = NULL;
+ struct lstcon_batch *bat;
+ struct lstcon_ndlink *ndl;
int rc;
rc = lstcon_batch_find(name, &bat);
@@ -977,7 +977,6 @@
if (!test) {
entp->u.tbe_batch.bae_ntest = bat->bat_ntest;
entp->u.tbe_batch.bae_state = bat->bat_state;
-
} else {
entp->u.tbe_test.tse_type = test->tes_type;
entp->u.tbe_test.tse_loop = test->tes_loop;
@@ -999,7 +998,7 @@
}
static int
-lstcon_batrpc_condition(int transop, lstcon_node_t *nd, void *arg)
+lstcon_batrpc_condition(int transop, struct lstcon_node *nd, void *arg)
{
switch (transop) {
case LST_TRANS_TSBRUN:
@@ -1021,10 +1020,10 @@
}
static int
-lstcon_batch_op(lstcon_batch_t *bat, int transop,
+lstcon_batch_op(struct lstcon_batch *bat, int transop,
struct list_head __user *result_up)
{
- lstcon_rpc_trans_t *trans;
+ struct lstcon_rpc_trans *trans;
int rc;
rc = lstcon_rpc_trans_ndlist(&bat->bat_cli_list,
@@ -1047,7 +1046,7 @@
int
lstcon_batch_run(char *name, int timeout, struct list_head __user *result_up)
{
- lstcon_batch_t *bat;
+ struct lstcon_batch *bat;
int rc;
if (lstcon_batch_find(name, &bat)) {
@@ -1069,7 +1068,7 @@
int
lstcon_batch_stop(char *name, int force, struct list_head __user *result_up)
{
- lstcon_batch_t *bat;
+ struct lstcon_batch *bat;
int rc;
if (lstcon_batch_find(name, &bat)) {
@@ -1089,17 +1088,17 @@
}
static void
-lstcon_batch_destroy(lstcon_batch_t *bat)
+lstcon_batch_destroy(struct lstcon_batch *bat)
{
- lstcon_ndlink_t *ndl;
- lstcon_test_t *test;
+ struct lstcon_ndlink *ndl;
+ struct lstcon_test *test;
int i;
list_del(&bat->bat_link);
while (!list_empty(&bat->bat_test_list)) {
test = list_entry(bat->bat_test_list.next,
- lstcon_test_t, tes_link);
+ struct lstcon_test, tes_link);
LASSERT(list_empty(&test->tes_trans_list));
list_del(&test->tes_link);
@@ -1107,7 +1106,7 @@
lstcon_group_decref(test->tes_src_grp);
lstcon_group_decref(test->tes_dst_grp);
- LIBCFS_FREE(test, offsetof(lstcon_test_t,
+ LIBCFS_FREE(test, offsetof(struct lstcon_test,
tes_param[test->tes_paramlen]));
}
@@ -1115,7 +1114,7 @@
while (!list_empty(&bat->bat_cli_list)) {
ndl = list_entry(bat->bat_cli_list.next,
- lstcon_ndlink_t, ndl_link);
+ struct lstcon_ndlink, ndl_link);
list_del_init(&ndl->ndl_link);
lstcon_ndlink_release(ndl);
@@ -1123,7 +1122,7 @@
while (!list_empty(&bat->bat_srv_list)) {
ndl = list_entry(bat->bat_srv_list.next,
- lstcon_ndlink_t, ndl_link);
+ struct lstcon_ndlink, ndl_link);
list_del_init(&ndl->ndl_link);
lstcon_ndlink_release(ndl);
@@ -1138,19 +1137,19 @@
sizeof(struct list_head) * LST_NODE_HASHSIZE);
LIBCFS_FREE(bat->bat_srv_hash,
sizeof(struct list_head) * LST_NODE_HASHSIZE);
- LIBCFS_FREE(bat, sizeof(lstcon_batch_t));
+ LIBCFS_FREE(bat, sizeof(struct lstcon_batch));
}
static int
-lstcon_testrpc_condition(int transop, lstcon_node_t *nd, void *arg)
+lstcon_testrpc_condition(int transop, struct lstcon_node *nd, void *arg)
{
- lstcon_test_t *test;
- lstcon_batch_t *batch;
- lstcon_ndlink_t *ndl;
+ struct lstcon_test *test;
+ struct lstcon_batch *batch;
+ struct lstcon_ndlink *ndl;
struct list_head *hash;
struct list_head *head;
- test = (lstcon_test_t *)arg;
+ test = (struct lstcon_test *)arg;
LASSERT(test);
batch = test->tes_batch;
@@ -1186,10 +1185,10 @@
}
static int
-lstcon_test_nodes_add(lstcon_test_t *test, struct list_head __user *result_up)
+lstcon_test_nodes_add(struct lstcon_test *test, struct list_head __user *result_up)
{
- lstcon_rpc_trans_t *trans;
- lstcon_group_t *grp;
+ struct lstcon_rpc_trans *trans;
+ struct lstcon_group *grp;
int transop;
int rc;
@@ -1237,7 +1236,7 @@
}
static int
-lstcon_verify_batch(const char *name, lstcon_batch_t **batch)
+lstcon_verify_batch(const char *name, struct lstcon_batch **batch)
{
int rc;
@@ -1256,10 +1255,10 @@
}
static int
-lstcon_verify_group(const char *name, lstcon_group_t **grp)
+lstcon_verify_group(const char *name, struct lstcon_group **grp)
{
int rc;
- lstcon_ndlink_t *ndl;
+ struct lstcon_ndlink *ndl;
rc = lstcon_group_find(name, grp);
if (rc) {
@@ -1284,11 +1283,11 @@
void *param, int paramlen, int *retp,
struct list_head __user *result_up)
{
- lstcon_test_t *test = NULL;
+ struct lstcon_test *test = NULL;
int rc;
- lstcon_group_t *src_grp = NULL;
- lstcon_group_t *dst_grp = NULL;
- lstcon_batch_t *batch = NULL;
+ struct lstcon_group *src_grp = NULL;
+ struct lstcon_group *dst_grp = NULL;
+ struct lstcon_batch *batch = NULL;
/*
* verify that a batch of the given name exists, and the groups
@@ -1310,7 +1309,7 @@
if (dst_grp->grp_userland)
*retp = 1;
- LIBCFS_ALLOC(test, offsetof(lstcon_test_t, tes_param[paramlen]));
+ LIBCFS_ALLOC(test, offsetof(struct lstcon_test, tes_param[paramlen]));
if (!test) {
CERROR("Can't allocate test descriptor\n");
rc = -ENOMEM;
@@ -1357,7 +1356,7 @@
return rc;
out:
if (test)
- LIBCFS_FREE(test, offsetof(lstcon_test_t, tes_param[paramlen]));
+ LIBCFS_FREE(test, offsetof(struct lstcon_test, tes_param[paramlen]));
if (dst_grp)
lstcon_group_decref(dst_grp);
@@ -1369,9 +1368,9 @@
}
static int
-lstcon_test_find(lstcon_batch_t *batch, int idx, lstcon_test_t **testpp)
+lstcon_test_find(struct lstcon_batch *batch, int idx, struct lstcon_test **testpp)
{
- lstcon_test_t *test;
+ struct lstcon_test *test;
list_for_each_entry(test, &batch->bat_test_list, tes_link) {
if (idx == test->tes_hdr.tsb_index) {
@@ -1384,10 +1383,10 @@
}
static int
-lstcon_tsbrpc_readent(int transop, srpc_msg_t *msg,
+lstcon_tsbrpc_readent(int transop, struct srpc_msg *msg,
lstcon_rpc_ent_t __user *ent_up)
{
- srpc_batch_reply_t *rep = &msg->msg_body.bat_reply;
+ struct srpc_batch_reply *rep = &msg->msg_body.bat_reply;
LASSERT(transop == LST_TRANS_TSBCLIQRY ||
transop == LST_TRANS_TSBSRVQRY);
@@ -1404,12 +1403,12 @@
lstcon_test_batch_query(char *name, int testidx, int client,
int timeout, struct list_head __user *result_up)
{
- lstcon_rpc_trans_t *trans;
+ struct lstcon_rpc_trans *trans;
struct list_head *translist;
struct list_head *ndlist;
- lstcon_tsb_hdr_t *hdr;
- lstcon_batch_t *batch;
- lstcon_test_t *test = NULL;
+ struct lstcon_tsb_hdr *hdr;
+ struct lstcon_batch *batch;
+ struct lstcon_test *test = NULL;
int transop;
int rc;
@@ -1423,7 +1422,6 @@
translist = &batch->bat_trans_list;
ndlist = &batch->bat_cli_list;
hdr = &batch->bat_hdr;
-
} else {
/* query specified test only */
rc = lstcon_test_find(batch, testidx, &test);
@@ -1448,7 +1446,8 @@
lstcon_rpc_trans_postwait(trans, timeout);
- if (!testidx && /* query a batch, not a test */
+ /* query a batch, not a test */
+ if (!testidx &&
!lstcon_rpc_stat_failure(lstcon_trans_stat(), 0) &&
!lstcon_tsbqry_stat_run(lstcon_trans_stat(), 0)) {
/* all RPCs finished, and no active test */
@@ -1463,10 +1462,10 @@
}
static int
-lstcon_statrpc_readent(int transop, srpc_msg_t *msg,
+lstcon_statrpc_readent(int transop, struct srpc_msg *msg,
lstcon_rpc_ent_t __user *ent_up)
{
- srpc_stat_reply_t *rep = &msg->msg_body.stat_reply;
+ struct srpc_stat_reply *rep = &msg->msg_body.stat_reply;
sfw_counters_t __user *sfwk_stat;
srpc_counters_t __user *srpc_stat;
lnet_counters_t __user *lnet_stat;
@@ -1491,7 +1490,7 @@
int timeout, struct list_head __user *result_up)
{
struct list_head head;
- lstcon_rpc_trans_t *trans;
+ struct lstcon_rpc_trans *trans;
int rc;
INIT_LIST_HEAD(&head);
@@ -1516,7 +1515,7 @@
lstcon_group_stat(char *grp_name, int timeout,
struct list_head __user *result_up)
{
- lstcon_group_t *grp;
+ struct lstcon_group *grp;
int rc;
rc = lstcon_group_find(grp_name, &grp);
@@ -1536,8 +1535,8 @@
lstcon_nodes_stat(int count, lnet_process_id_t __user *ids_up,
int timeout, struct list_head __user *result_up)
{
- lstcon_ndlink_t *ndl;
- lstcon_group_t *tmp;
+ struct lstcon_ndlink *ndl;
+ struct lstcon_group *tmp;
lnet_process_id_t id;
int i;
int rc;
@@ -1581,7 +1580,7 @@
struct list_head *translist,
int timeout, struct list_head __user *result_up)
{
- lstcon_rpc_trans_t *trans;
+ struct lstcon_rpc_trans *trans;
int rc;
rc = lstcon_rpc_trans_ndlist(ndlist, translist, LST_TRANS_SESQRY,
@@ -1611,7 +1610,7 @@
lstcon_batch_debug(int timeout, char *name,
int client, struct list_head __user *result_up)
{
- lstcon_batch_t *bat;
+ struct lstcon_batch *bat;
int rc;
rc = lstcon_batch_find(name, &bat);
@@ -1629,7 +1628,7 @@
lstcon_group_debug(int timeout, char *name,
struct list_head __user *result_up)
{
- lstcon_group_t *grp;
+ struct lstcon_group *grp;
int rc;
rc = lstcon_group_find(name, &grp);
@@ -1649,8 +1648,8 @@
struct list_head __user *result_up)
{
lnet_process_id_t id;
- lstcon_ndlink_t *ndl;
- lstcon_group_t *grp;
+ struct lstcon_ndlink *ndl;
+ struct lstcon_group *grp;
int i;
int rc;
@@ -1749,7 +1748,7 @@
if (strlen(name) > sizeof(console_session.ses_name) - 1)
return -E2BIG;
- strncpy(console_session.ses_name, name,
+ strlcpy(console_session.ses_name, name,
sizeof(console_session.ses_name));
rc = lstcon_batch_add(LST_DEFAULT_BATCH);
@@ -1758,7 +1757,7 @@
rc = lstcon_rpc_pinger_start();
if (rc) {
- lstcon_batch_t *bat = NULL;
+ struct lstcon_batch *bat = NULL;
lstcon_batch_find(LST_DEFAULT_BATCH, &bat);
lstcon_batch_destroy(bat);
@@ -1782,7 +1781,7 @@
char __user *name_up, int len)
{
lstcon_ndlist_ent_t *entp;
- lstcon_ndlink_t *ndl;
+ struct lstcon_ndlink *ndl;
int rc = 0;
if (console_session.ses_state != LST_SESSION_ACTIVE)
@@ -1813,9 +1812,9 @@
int
lstcon_session_end(void)
{
- lstcon_rpc_trans_t *trans;
- lstcon_group_t *grp;
- lstcon_batch_t *bat;
+ struct lstcon_rpc_trans *trans;
+ struct lstcon_group *grp;
+ struct lstcon_batch *bat;
int rc = 0;
LASSERT(console_session.ses_state == LST_SESSION_ACTIVE);
@@ -1849,7 +1848,7 @@
/* destroy all batches */
while (!list_empty(&console_session.ses_bat_list)) {
bat = list_entry(console_session.ses_bat_list.next,
- lstcon_batch_t, bat_link);
+ struct lstcon_batch, bat_link);
lstcon_batch_destroy(bat);
}
@@ -1857,7 +1856,7 @@
/* destroy all groups */
while (!list_empty(&console_session.ses_grp_list)) {
grp = list_entry(console_session.ses_grp_list.next,
- lstcon_group_t, grp_link);
+ struct lstcon_group, grp_link);
LASSERT(grp->grp_ref == 1);
lstcon_group_decref(grp);
@@ -1906,12 +1905,12 @@
static int
lstcon_acceptor_handle(struct srpc_server_rpc *rpc)
{
- srpc_msg_t *rep = &rpc->srpc_replymsg;
- srpc_msg_t *req = &rpc->srpc_reqstbuf->buf_msg;
- srpc_join_reqst_t *jreq = &req->msg_body.join_reqst;
- srpc_join_reply_t *jrep = &rep->msg_body.join_reply;
- lstcon_group_t *grp = NULL;
- lstcon_ndlink_t *ndl;
+ struct srpc_msg *rep = &rpc->srpc_replymsg;
+ struct srpc_msg *req = &rpc->srpc_reqstbuf->buf_msg;
+ struct srpc_join_reqst *jreq = &req->msg_body.join_reqst;
+ struct srpc_join_reply *jrep = &rep->msg_body.join_reply;
+ struct lstcon_group *grp = NULL;
+ struct lstcon_ndlink *ndl;
int rc = 0;
sfw_unpack_message(req);
@@ -1987,7 +1986,8 @@
return rc;
}
-static srpc_service_t lstcon_acceptor_service;
+static struct srpc_service lstcon_acceptor_service;
+
static void lstcon_init_acceptor_service(void)
{
/* initialize selftest console acceptor service table */
diff --git a/drivers/staging/lustre/lnet/selftest/console.h b/drivers/staging/lustre/lnet/selftest/console.h
index 554f582..becd22e 100644
--- a/drivers/staging/lustre/lnet/selftest/console.h
+++ b/drivers/staging/lustre/lnet/selftest/console.h
@@ -50,22 +50,25 @@
#include "selftest.h"
#include "conrpc.h"
-typedef struct lstcon_node {
+/* node descriptor */
+struct lstcon_node {
lnet_process_id_t nd_id; /* id of the node */
int nd_ref; /* reference count */
int nd_state; /* state of the node */
int nd_timeout; /* session timeout */
unsigned long nd_stamp; /* timestamp of last replied RPC */
struct lstcon_rpc nd_ping; /* ping rpc */
-} lstcon_node_t; /* node descriptor */
+};
-typedef struct {
+/* node link descriptor */
+struct lstcon_ndlink {
struct list_head ndl_link; /* chain on list */
struct list_head ndl_hlink; /* chain on hash */
- lstcon_node_t *ndl_node; /* pointer to node */
-} lstcon_ndlink_t; /* node link descriptor */
+ struct lstcon_node *ndl_node; /* pointer to node */
+};
-typedef struct {
+/* (alias of nodes) group descriptor */
+struct lstcon_group {
struct list_head grp_link; /* chain on global group list
*/
int grp_ref; /* reference count */
@@ -76,18 +79,19 @@
struct list_head grp_trans_list; /* transaction list */
struct list_head grp_ndl_list; /* nodes list */
struct list_head grp_ndl_hash[0]; /* hash table for nodes */
-} lstcon_group_t; /* (alias of nodes) group descriptor */
+};
#define LST_BATCH_IDLE 0xB0 /* idle batch */
#define LST_BATCH_RUNNING 0xB1 /* running batch */
-typedef struct lstcon_tsb_hdr {
+struct lstcon_tsb_hdr {
lst_bid_t tsb_id; /* batch ID */
int tsb_index; /* test index */
-} lstcon_tsb_hdr_t;
+};
-typedef struct {
- lstcon_tsb_hdr_t bat_hdr; /* test_batch header */
+/* (tests ) batch descriptor */
+struct lstcon_batch {
+ struct lstcon_tsb_hdr bat_hdr; /* test_batch header */
struct list_head bat_link; /* chain on session's batches list */
int bat_ntest; /* # of test */
int bat_state; /* state of the batch */
@@ -95,20 +99,21 @@
* for run, force for stop */
char bat_name[LST_NAME_SIZE];/* name of batch */
- struct list_head bat_test_list; /* list head of tests (lstcon_test_t)
+ struct list_head bat_test_list; /* list head of tests (struct lstcon_test)
*/
struct list_head bat_trans_list; /* list head of transaction */
struct list_head bat_cli_list; /* list head of client nodes
- * (lstcon_node_t) */
+ * (struct lstcon_node) */
struct list_head *bat_cli_hash; /* hash table of client nodes */
struct list_head bat_srv_list; /* list head of server nodes */
struct list_head *bat_srv_hash; /* hash table of server nodes */
-} lstcon_batch_t; /* (tests ) batch descriptor */
+};
-typedef struct lstcon_test {
- lstcon_tsb_hdr_t tes_hdr; /* test batch header */
+/* a single test descriptor */
+struct lstcon_test {
+ struct lstcon_tsb_hdr tes_hdr; /* test batch header */
struct list_head tes_link; /* chain on batch's tests list */
- lstcon_batch_t *tes_batch; /* pointer to batch */
+ struct lstcon_batch *tes_batch; /* pointer to batch */
int tes_type; /* type of the test, i.e: bulk, ping */
int tes_stop_onerr; /* stop on error */
@@ -120,12 +125,12 @@
int tes_cliidx; /* client index, used for RPC creating */
struct list_head tes_trans_list; /* transaction list */
- lstcon_group_t *tes_src_grp; /* group run the test */
- lstcon_group_t *tes_dst_grp; /* target group */
+ struct lstcon_group *tes_src_grp; /* group run the test */
+ struct lstcon_group *tes_dst_grp; /* target group */
int tes_paramlen; /* test parameter length */
char tes_param[0]; /* test parameter */
-} lstcon_test_t; /* a single test descriptor */
+};
#define LST_GLOBAL_HASHSIZE 503 /* global nodes hash table size */
#define LST_NODE_HASHSIZE 239 /* node hash table (for batch or group) */
@@ -152,7 +157,7 @@
unsigned ses_expired:1; /* console is timedout */
__u64 ses_id_cookie; /* batch id cookie */
char ses_name[LST_NAME_SIZE];/* session name */
- lstcon_rpc_trans_t *ses_ping; /* session pinger */
+ struct lstcon_rpc_trans *ses_ping; /* session pinger */
struct stt_timer ses_ping_timer; /* timer for pinger */
lstcon_trans_stat_t ses_trans_stat; /* transaction stats */
diff --git a/drivers/staging/lustre/lnet/selftest/framework.c b/drivers/staging/lustre/lnet/selftest/framework.c
index e2c5323..30e4f71 100644
--- a/drivers/staging/lustre/lnet/selftest/framework.c
+++ b/drivers/staging/lustre/lnet/selftest/framework.c
@@ -109,19 +109,19 @@
struct list_head fw_tests; /* registered test cases */
atomic_t fw_nzombies; /* # zombie sessions */
spinlock_t fw_lock; /* serialise */
- sfw_session_t *fw_session; /* _the_ session */
+ struct sfw_session *fw_session; /* _the_ session */
int fw_shuttingdown; /* shutdown in progress */
struct srpc_server_rpc *fw_active_srpc;/* running RPC */
} sfw_data;
/* forward ref's */
-int sfw_stop_batch(sfw_batch_t *tsb, int force);
-void sfw_destroy_session(sfw_session_t *sn);
+int sfw_stop_batch(struct sfw_batch *tsb, int force);
+void sfw_destroy_session(struct sfw_session *sn);
-static inline sfw_test_case_t *
+static inline struct sfw_test_case *
sfw_find_test_case(int id)
{
- sfw_test_case_t *tsc;
+ struct sfw_test_case *tsc;
LASSERT(id <= SRPC_SERVICE_MAX_ID);
LASSERT(id > SRPC_FRAMEWORK_SERVICE_MAX_ID);
@@ -135,9 +135,9 @@
}
static int
-sfw_register_test(srpc_service_t *service, sfw_test_client_ops_t *cliops)
+sfw_register_test(struct srpc_service *service, struct sfw_test_client_ops *cliops)
{
- sfw_test_case_t *tsc;
+ struct sfw_test_case *tsc;
if (sfw_find_test_case(service->sv_id)) {
CERROR("Failed to register test %s (%d)\n",
@@ -145,7 +145,7 @@
return -EEXIST;
}
- LIBCFS_ALLOC(tsc, sizeof(sfw_test_case_t));
+ LIBCFS_ALLOC(tsc, sizeof(struct sfw_test_case));
if (!tsc)
return -ENOMEM;
@@ -159,7 +159,7 @@
static void
sfw_add_session_timer(void)
{
- sfw_session_t *sn = sfw_data.fw_session;
+ struct sfw_session *sn = sfw_data.fw_session;
struct stt_timer *timer = &sn->sn_timer;
LASSERT(!sfw_data.fw_shuttingdown);
@@ -177,7 +177,7 @@
static int
sfw_del_session_timer(void)
{
- sfw_session_t *sn = sfw_data.fw_session;
+ struct sfw_session *sn = sfw_data.fw_session;
if (!sn || !sn->sn_timer_active)
return 0;
@@ -196,10 +196,10 @@
sfw_deactivate_session(void)
__must_hold(&sfw_data.fw_lock)
{
- sfw_session_t *sn = sfw_data.fw_session;
+ struct sfw_session *sn = sfw_data.fw_session;
int nactive = 0;
- sfw_batch_t *tsb;
- sfw_test_case_t *tsc;
+ struct sfw_batch *tsb;
+ struct sfw_test_case *tsc;
if (!sn)
return;
@@ -226,7 +226,7 @@
}
if (nactive)
- return; /* wait for active batches to stop */
+ return; /* wait for active batches to stop */
list_del_init(&sn->sn_list);
spin_unlock(&sfw_data.fw_lock);
@@ -239,7 +239,7 @@
static void
sfw_session_expired(void *data)
{
- sfw_session_t *sn = data;
+ struct sfw_session *sn = data;
spin_lock(&sfw_data.fw_lock);
@@ -257,12 +257,12 @@
}
static inline void
-sfw_init_session(sfw_session_t *sn, lst_sid_t sid,
+sfw_init_session(struct sfw_session *sn, lst_sid_t sid,
unsigned features, const char *name)
{
struct stt_timer *timer = &sn->sn_timer;
- memset(sn, 0, sizeof(sfw_session_t));
+ memset(sn, 0, sizeof(struct sfw_session));
INIT_LIST_HEAD(&sn->sn_list);
INIT_LIST_HEAD(&sn->sn_batches);
atomic_set(&sn->sn_refcount, 1); /* +1 for caller */
@@ -298,7 +298,7 @@
}
static void
-sfw_client_rpc_fini(srpc_client_rpc_t *rpc)
+sfw_client_rpc_fini(struct srpc_client_rpc *rpc)
{
LASSERT(!rpc->crpc_bulk.bk_niov);
LASSERT(list_empty(&rpc->crpc_list));
@@ -318,11 +318,11 @@
spin_unlock(&sfw_data.fw_lock);
}
-static sfw_batch_t *
+static struct sfw_batch *
sfw_find_batch(lst_bid_t bid)
{
- sfw_session_t *sn = sfw_data.fw_session;
- sfw_batch_t *bat;
+ struct sfw_session *sn = sfw_data.fw_session;
+ struct sfw_batch *bat;
LASSERT(sn);
@@ -334,11 +334,11 @@
return NULL;
}
-static sfw_batch_t *
+static struct sfw_batch *
sfw_bid2batch(lst_bid_t bid)
{
- sfw_session_t *sn = sfw_data.fw_session;
- sfw_batch_t *bat;
+ struct sfw_session *sn = sfw_data.fw_session;
+ struct sfw_batch *bat;
LASSERT(sn);
@@ -346,7 +346,7 @@
if (bat)
return bat;
- LIBCFS_ALLOC(bat, sizeof(sfw_batch_t));
+ LIBCFS_ALLOC(bat, sizeof(struct sfw_batch));
if (!bat)
return NULL;
@@ -361,11 +361,11 @@
}
static int
-sfw_get_stats(srpc_stat_reqst_t *request, srpc_stat_reply_t *reply)
+sfw_get_stats(struct srpc_stat_reqst *request, struct srpc_stat_reply *reply)
{
- sfw_session_t *sn = sfw_data.fw_session;
+ struct sfw_session *sn = sfw_data.fw_session;
sfw_counters_t *cnt = &reply->str_fw;
- sfw_batch_t *bat;
+ struct sfw_batch *bat;
reply->str_sid = !sn ? LST_INVALID_SID : sn->sn_id;
@@ -402,10 +402,10 @@
}
int
-sfw_make_session(srpc_mksn_reqst_t *request, srpc_mksn_reply_t *reply)
+sfw_make_session(struct srpc_mksn_reqst *request, struct srpc_mksn_reply *reply)
{
- sfw_session_t *sn = sfw_data.fw_session;
- srpc_msg_t *msg = container_of(request, srpc_msg_t,
+ struct sfw_session *sn = sfw_data.fw_session;
+ struct srpc_msg *msg = container_of(request, struct srpc_msg,
msg_body.mksn_reqst);
int cplen = 0;
@@ -438,7 +438,7 @@
/*
* reject the request if it requires unknown features
* NB: old version will always accept all features because it's not
- * aware of srpc_msg_t::msg_ses_feats, it's a defect but it's also
+ * aware of srpc_msg::msg_ses_feats, it's a defect but it's also
* harmless because it will return zero feature to console, and it's
* console's responsibility to make sure all nodes in a session have
* same feature mask.
@@ -449,7 +449,7 @@
}
/* brand new or create by force */
- LIBCFS_ALLOC(sn, sizeof(sfw_session_t));
+ LIBCFS_ALLOC(sn, sizeof(struct sfw_session));
if (!sn) {
CERROR("dropping RPC mksn under memory pressure\n");
return -ENOMEM;
@@ -473,9 +473,9 @@
}
static int
-sfw_remove_session(srpc_rmsn_reqst_t *request, srpc_rmsn_reply_t *reply)
+sfw_remove_session(struct srpc_rmsn_reqst *request, struct srpc_rmsn_reply *reply)
{
- sfw_session_t *sn = sfw_data.fw_session;
+ struct sfw_session *sn = sfw_data.fw_session;
reply->rmsn_sid = !sn ? LST_INVALID_SID : sn->sn_id;
@@ -505,9 +505,9 @@
}
static int
-sfw_debug_session(srpc_debug_reqst_t *request, srpc_debug_reply_t *reply)
+sfw_debug_session(struct srpc_debug_reqst *request, struct srpc_debug_reply *reply)
{
- sfw_session_t *sn = sfw_data.fw_session;
+ struct sfw_session *sn = sfw_data.fw_session;
if (!sn) {
reply->dbg_status = ESRCH;
@@ -526,10 +526,10 @@
}
static void
-sfw_test_rpc_fini(srpc_client_rpc_t *rpc)
+sfw_test_rpc_fini(struct srpc_client_rpc *rpc)
{
- sfw_test_unit_t *tsu = rpc->crpc_priv;
- sfw_test_instance_t *tsi = tsu->tsu_instance;
+ struct sfw_test_unit *tsu = rpc->crpc_priv;
+ struct sfw_test_instance *tsi = tsu->tsu_instance;
/* Called with hold of tsi->tsi_lock */
LASSERT(list_empty(&rpc->crpc_list));
@@ -537,7 +537,7 @@
}
static inline int
-sfw_test_buffers(sfw_test_instance_t *tsi)
+sfw_test_buffers(struct sfw_test_instance *tsi)
{
struct sfw_test_case *tsc;
struct srpc_service *svc;
@@ -614,10 +614,10 @@
}
static void
-sfw_destroy_test_instance(sfw_test_instance_t *tsi)
+sfw_destroy_test_instance(struct sfw_test_instance *tsi)
{
- srpc_client_rpc_t *rpc;
- sfw_test_unit_t *tsu;
+ struct srpc_client_rpc *rpc;
+ struct sfw_test_unit *tsu;
if (!tsi->tsi_is_client)
goto clean;
@@ -630,14 +630,14 @@
while (!list_empty(&tsi->tsi_units)) {
tsu = list_entry(tsi->tsi_units.next,
- sfw_test_unit_t, tsu_list);
+ struct sfw_test_unit, tsu_list);
list_del(&tsu->tsu_list);
LIBCFS_FREE(tsu, sizeof(*tsu));
}
while (!list_empty(&tsi->tsi_free_rpcs)) {
rpc = list_entry(tsi->tsi_free_rpcs.next,
- srpc_client_rpc_t, crpc_list);
+ struct srpc_client_rpc, crpc_list);
list_del(&rpc->crpc_list);
LIBCFS_FREE(rpc, srpc_client_rpc_size(rpc));
}
@@ -648,34 +648,34 @@
}
static void
-sfw_destroy_batch(sfw_batch_t *tsb)
+sfw_destroy_batch(struct sfw_batch *tsb)
{
- sfw_test_instance_t *tsi;
+ struct sfw_test_instance *tsi;
LASSERT(!sfw_batch_active(tsb));
LASSERT(list_empty(&tsb->bat_list));
while (!list_empty(&tsb->bat_tests)) {
tsi = list_entry(tsb->bat_tests.next,
- sfw_test_instance_t, tsi_list);
+ struct sfw_test_instance, tsi_list);
list_del_init(&tsi->tsi_list);
sfw_destroy_test_instance(tsi);
}
- LIBCFS_FREE(tsb, sizeof(sfw_batch_t));
+ LIBCFS_FREE(tsb, sizeof(struct sfw_batch));
}
void
-sfw_destroy_session(sfw_session_t *sn)
+sfw_destroy_session(struct sfw_session *sn)
{
- sfw_batch_t *batch;
+ struct sfw_batch *batch;
LASSERT(list_empty(&sn->sn_list));
LASSERT(sn != sfw_data.fw_session);
while (!list_empty(&sn->sn_batches)) {
batch = list_entry(sn->sn_batches.next,
- sfw_batch_t, bat_list);
+ struct sfw_batch, bat_list);
list_del_init(&batch->bat_list);
sfw_destroy_batch(batch);
}
@@ -685,28 +685,28 @@
}
static void
-sfw_unpack_addtest_req(srpc_msg_t *msg)
+sfw_unpack_addtest_req(struct srpc_msg *msg)
{
- srpc_test_reqst_t *req = &msg->msg_body.tes_reqst;
+ struct srpc_test_reqst *req = &msg->msg_body.tes_reqst;
LASSERT(msg->msg_type == SRPC_MSG_TEST_REQST);
LASSERT(req->tsr_is_client);
if (msg->msg_magic == SRPC_MSG_MAGIC)
- return; /* no flipping needed */
+ return; /* no flipping needed */
LASSERT(msg->msg_magic == __swab32(SRPC_MSG_MAGIC));
if (req->tsr_service == SRPC_SERVICE_BRW) {
if (!(msg->msg_ses_feats & LST_FEAT_BULK_LEN)) {
- test_bulk_req_t *bulk = &req->tsr_u.bulk_v0;
+ struct test_bulk_req *bulk = &req->tsr_u.bulk_v0;
__swab32s(&bulk->blk_opc);
__swab32s(&bulk->blk_npg);
__swab32s(&bulk->blk_flags);
} else {
- test_bulk_req_v1_t *bulk = &req->tsr_u.bulk_v1;
+ struct test_bulk_req_v1 *bulk = &req->tsr_u.bulk_v1;
__swab16s(&bulk->blk_opc);
__swab16s(&bulk->blk_flags);
@@ -718,7 +718,7 @@
}
if (req->tsr_service == SRPC_SERVICE_PING) {
- test_ping_req_t *ping = &req->tsr_u.ping;
+ struct test_ping_req *ping = &req->tsr_u.ping;
__swab32s(&ping->png_size);
__swab32s(&ping->png_flags);
@@ -729,14 +729,14 @@
}
static int
-sfw_add_test_instance(sfw_batch_t *tsb, struct srpc_server_rpc *rpc)
+sfw_add_test_instance(struct sfw_batch *tsb, struct srpc_server_rpc *rpc)
{
- srpc_msg_t *msg = &rpc->srpc_reqstbuf->buf_msg;
- srpc_test_reqst_t *req = &msg->msg_body.tes_reqst;
- srpc_bulk_t *bk = rpc->srpc_bulk;
+ struct srpc_msg *msg = &rpc->srpc_reqstbuf->buf_msg;
+ struct srpc_test_reqst *req = &msg->msg_body.tes_reqst;
+ struct srpc_bulk *bk = rpc->srpc_bulk;
int ndest = req->tsr_ndest;
- sfw_test_unit_t *tsu;
- sfw_test_instance_t *tsi;
+ struct sfw_test_unit *tsu;
+ struct sfw_test_instance *tsi;
int i;
int rc;
@@ -789,13 +789,13 @@
int j;
dests = page_address(bk->bk_iovs[i / SFW_ID_PER_PAGE].kiov_page);
- LASSERT(dests); /* my pages are within KVM always */
+ LASSERT(dests); /* my pages are within KVM always */
id = dests[i % SFW_ID_PER_PAGE];
if (msg->msg_magic != SRPC_MSG_MAGIC)
sfw_unpack_id(id);
for (j = 0; j < tsi->tsi_concur; j++) {
- LIBCFS_ALLOC(tsu, sizeof(sfw_test_unit_t));
+ LIBCFS_ALLOC(tsu, sizeof(struct sfw_test_unit));
if (!tsu) {
rc = -ENOMEM;
CERROR("Can't allocate tsu for %d\n",
@@ -824,11 +824,11 @@
}
static void
-sfw_test_unit_done(sfw_test_unit_t *tsu)
+sfw_test_unit_done(struct sfw_test_unit *tsu)
{
- sfw_test_instance_t *tsi = tsu->tsu_instance;
- sfw_batch_t *tsb = tsi->tsi_batch;
- sfw_session_t *sn = tsb->bat_session;
+ struct sfw_test_instance *tsi = tsu->tsu_instance;
+ struct sfw_batch *tsb = tsi->tsi_batch;
+ struct sfw_session *sn = tsb->bat_session;
LASSERT(sfw_test_active(tsi));
@@ -844,8 +844,8 @@
spin_lock(&sfw_data.fw_lock);
- if (!atomic_dec_and_test(&tsb->bat_nactive) ||/* tsb still active */
- sn == sfw_data.fw_session) { /* sn also active */
+ if (!atomic_dec_and_test(&tsb->bat_nactive) || /* tsb still active */
+ sn == sfw_data.fw_session) { /* sn also active */
spin_unlock(&sfw_data.fw_lock);
return;
}
@@ -866,10 +866,10 @@
}
static void
-sfw_test_rpc_done(srpc_client_rpc_t *rpc)
+sfw_test_rpc_done(struct srpc_client_rpc *rpc)
{
- sfw_test_unit_t *tsu = rpc->crpc_priv;
- sfw_test_instance_t *tsi = tsu->tsu_instance;
+ struct sfw_test_unit *tsu = rpc->crpc_priv;
+ struct sfw_test_instance *tsi = tsu->tsu_instance;
int done = 0;
tsi->tsi_ops->tso_done_rpc(tsu, rpc);
@@ -900,19 +900,19 @@
}
int
-sfw_create_test_rpc(sfw_test_unit_t *tsu, lnet_process_id_t peer,
+sfw_create_test_rpc(struct sfw_test_unit *tsu, lnet_process_id_t peer,
unsigned features, int nblk, int blklen,
- srpc_client_rpc_t **rpcpp)
+ struct srpc_client_rpc **rpcpp)
{
- srpc_client_rpc_t *rpc = NULL;
- sfw_test_instance_t *tsi = tsu->tsu_instance;
+ struct srpc_client_rpc *rpc = NULL;
+ struct sfw_test_instance *tsi = tsu->tsu_instance;
spin_lock(&tsi->tsi_lock);
LASSERT(sfw_test_active(tsi));
/* pick request from buffer */
rpc = list_first_entry_or_null(&tsi->tsi_free_rpcs,
- srpc_client_rpc_t, crpc_list);
+ struct srpc_client_rpc, crpc_list);
if (rpc) {
LASSERT(nblk == rpc->crpc_bulk.bk_niov);
list_del_init(&rpc->crpc_list);
@@ -942,11 +942,11 @@
}
static int
-sfw_run_test(swi_workitem_t *wi)
+sfw_run_test(struct swi_workitem *wi)
{
- sfw_test_unit_t *tsu = wi->swi_workitem.wi_data;
- sfw_test_instance_t *tsi = tsu->tsu_instance;
- srpc_client_rpc_t *rpc = NULL;
+ struct sfw_test_unit *tsu = wi->swi_workitem.wi_data;
+ struct sfw_test_instance *tsi = tsu->tsu_instance;
+ struct srpc_client_rpc *rpc = NULL;
LASSERT(wi == &tsu->tsu_worker);
@@ -991,11 +991,11 @@
}
static int
-sfw_run_batch(sfw_batch_t *tsb)
+sfw_run_batch(struct sfw_batch *tsb)
{
- swi_workitem_t *wi;
- sfw_test_unit_t *tsu;
- sfw_test_instance_t *tsi;
+ struct swi_workitem *wi;
+ struct sfw_test_unit *tsu;
+ struct sfw_test_instance *tsi;
if (sfw_batch_active(tsb)) {
CDEBUG(D_NET, "Batch already active: %llu (%d)\n",
@@ -1026,10 +1026,10 @@
}
int
-sfw_stop_batch(sfw_batch_t *tsb, int force)
+sfw_stop_batch(struct sfw_batch *tsb, int force)
{
- sfw_test_instance_t *tsi;
- srpc_client_rpc_t *rpc;
+ struct sfw_test_instance *tsi;
+ struct srpc_client_rpc *rpc;
if (!sfw_batch_active(tsb)) {
CDEBUG(D_NET, "Batch %llu inactive\n", tsb->bat_id.bat_id);
@@ -1068,9 +1068,9 @@
}
static int
-sfw_query_batch(sfw_batch_t *tsb, int testidx, srpc_batch_reply_t *reply)
+sfw_query_batch(struct sfw_batch *tsb, int testidx, struct srpc_batch_reply *reply)
{
- sfw_test_instance_t *tsi;
+ struct sfw_test_instance *tsi;
if (testidx < 0)
return -EINVAL;
@@ -1115,11 +1115,11 @@
static int
sfw_add_test(struct srpc_server_rpc *rpc)
{
- sfw_session_t *sn = sfw_data.fw_session;
- srpc_test_reply_t *reply = &rpc->srpc_replymsg.msg_body.tes_reply;
- srpc_test_reqst_t *request;
+ struct sfw_session *sn = sfw_data.fw_session;
+ struct srpc_test_reply *reply = &rpc->srpc_replymsg.msg_body.tes_reply;
+ struct srpc_test_reqst *request;
int rc;
- sfw_batch_t *bat;
+ struct sfw_batch *bat;
request = &rpc->srpc_reqstbuf->buf_msg.msg_body.tes_reqst;
reply->tsr_sid = !sn ? LST_INVALID_SID : sn->sn_id;
@@ -1183,11 +1183,11 @@
}
static int
-sfw_control_batch(srpc_batch_reqst_t *request, srpc_batch_reply_t *reply)
+sfw_control_batch(struct srpc_batch_reqst *request, struct srpc_batch_reply *reply)
{
- sfw_session_t *sn = sfw_data.fw_session;
+ struct sfw_session *sn = sfw_data.fw_session;
int rc = 0;
- sfw_batch_t *bat;
+ struct sfw_batch *bat;
reply->bar_sid = !sn ? LST_INVALID_SID : sn->sn_id;
@@ -1227,8 +1227,8 @@
sfw_handle_server_rpc(struct srpc_server_rpc *rpc)
{
struct srpc_service *sv = rpc->srpc_scd->scd_svc;
- srpc_msg_t *reply = &rpc->srpc_replymsg;
- srpc_msg_t *request = &rpc->srpc_reqstbuf->buf_msg;
+ struct srpc_msg *reply = &rpc->srpc_replymsg;
+ struct srpc_msg *request = &rpc->srpc_reqstbuf->buf_msg;
unsigned features = LST_FEATS_MASK;
int rc = 0;
@@ -1244,7 +1244,7 @@
/* Remove timer to avoid racing with it or expiring active session */
if (sfw_del_session_timer()) {
- CERROR("Dropping RPC (%s) from %s: racing with expiry timer.",
+ CERROR("dropping RPC %s from %s: racing with expiry timer\n",
sv->sv_name, libcfs_id2str(rpc->srpc_peer));
spin_unlock(&sfw_data.fw_lock);
return -EAGAIN;
@@ -1261,7 +1261,7 @@
if (sv->sv_id != SRPC_SERVICE_MAKE_SESSION &&
sv->sv_id != SRPC_SERVICE_DEBUG) {
- sfw_session_t *sn = sfw_data.fw_session;
+ struct sfw_session *sn = sfw_data.fw_session;
if (sn &&
sn->sn_features != request->msg_ses_feats) {
@@ -1273,7 +1273,7 @@
}
} else if (request->msg_ses_feats & ~LST_FEATS_MASK) {
- /**
+ /*
* NB: at this point, old version will ignore features and
* create new session anyway, so console should be able
* to handle this
@@ -1377,12 +1377,12 @@
return rc;
}
-srpc_client_rpc_t *
+struct srpc_client_rpc *
sfw_create_rpc(lnet_process_id_t peer, int service,
unsigned features, int nbulkiov, int bulklen,
- void (*done)(srpc_client_rpc_t *), void *priv)
+ void (*done)(struct srpc_client_rpc *), void *priv)
{
- srpc_client_rpc_t *rpc = NULL;
+ struct srpc_client_rpc *rpc = NULL;
spin_lock(&sfw_data.fw_lock);
@@ -1391,7 +1391,7 @@
if (!nbulkiov && !list_empty(&sfw_data.fw_zombie_rpcs)) {
rpc = list_entry(sfw_data.fw_zombie_rpcs.next,
- srpc_client_rpc_t, crpc_list);
+ struct srpc_client_rpc, crpc_list);
list_del(&rpc->crpc_list);
srpc_init_client_rpc(rpc, peer, service, 0, 0,
@@ -1415,7 +1415,7 @@
}
void
-sfw_unpack_message(srpc_msg_t *msg)
+sfw_unpack_message(struct srpc_msg *msg)
{
if (msg->msg_magic == SRPC_MSG_MAGIC)
return; /* no flipping needed */
@@ -1424,7 +1424,7 @@
LASSERT(msg->msg_magic == __swab32(SRPC_MSG_MAGIC));
if (msg->msg_type == SRPC_MSG_STAT_REQST) {
- srpc_stat_reqst_t *req = &msg->msg_body.stat_reqst;
+ struct srpc_stat_reqst *req = &msg->msg_body.stat_reqst;
__swab32s(&req->str_type);
__swab64s(&req->str_rpyid);
@@ -1433,7 +1433,7 @@
}
if (msg->msg_type == SRPC_MSG_STAT_REPLY) {
- srpc_stat_reply_t *rep = &msg->msg_body.stat_reply;
+ struct srpc_stat_reply *rep = &msg->msg_body.stat_reply;
__swab32s(&rep->str_status);
sfw_unpack_sid(rep->str_sid);
@@ -1444,7 +1444,7 @@
}
if (msg->msg_type == SRPC_MSG_MKSN_REQST) {
- srpc_mksn_reqst_t *req = &msg->msg_body.mksn_reqst;
+ struct srpc_mksn_reqst *req = &msg->msg_body.mksn_reqst;
__swab64s(&req->mksn_rpyid);
__swab32s(&req->mksn_force);
@@ -1453,7 +1453,7 @@
}
if (msg->msg_type == SRPC_MSG_MKSN_REPLY) {
- srpc_mksn_reply_t *rep = &msg->msg_body.mksn_reply;
+ struct srpc_mksn_reply *rep = &msg->msg_body.mksn_reply;
__swab32s(&rep->mksn_status);
__swab32s(&rep->mksn_timeout);
@@ -1462,7 +1462,7 @@
}
if (msg->msg_type == SRPC_MSG_RMSN_REQST) {
- srpc_rmsn_reqst_t *req = &msg->msg_body.rmsn_reqst;
+ struct srpc_rmsn_reqst *req = &msg->msg_body.rmsn_reqst;
__swab64s(&req->rmsn_rpyid);
sfw_unpack_sid(req->rmsn_sid);
@@ -1470,7 +1470,7 @@
}
if (msg->msg_type == SRPC_MSG_RMSN_REPLY) {
- srpc_rmsn_reply_t *rep = &msg->msg_body.rmsn_reply;
+ struct srpc_rmsn_reply *rep = &msg->msg_body.rmsn_reply;
__swab32s(&rep->rmsn_status);
sfw_unpack_sid(rep->rmsn_sid);
@@ -1478,7 +1478,7 @@
}
if (msg->msg_type == SRPC_MSG_DEBUG_REQST) {
- srpc_debug_reqst_t *req = &msg->msg_body.dbg_reqst;
+ struct srpc_debug_reqst *req = &msg->msg_body.dbg_reqst;
__swab64s(&req->dbg_rpyid);
__swab32s(&req->dbg_flags);
@@ -1487,7 +1487,7 @@
}
if (msg->msg_type == SRPC_MSG_DEBUG_REPLY) {
- srpc_debug_reply_t *rep = &msg->msg_body.dbg_reply;
+ struct srpc_debug_reply *rep = &msg->msg_body.dbg_reply;
__swab32s(&rep->dbg_nbatch);
__swab32s(&rep->dbg_timeout);
@@ -1496,7 +1496,7 @@
}
if (msg->msg_type == SRPC_MSG_BATCH_REQST) {
- srpc_batch_reqst_t *req = &msg->msg_body.bat_reqst;
+ struct srpc_batch_reqst *req = &msg->msg_body.bat_reqst;
__swab32s(&req->bar_opc);
__swab64s(&req->bar_rpyid);
@@ -1508,7 +1508,7 @@
}
if (msg->msg_type == SRPC_MSG_BATCH_REPLY) {
- srpc_batch_reply_t *rep = &msg->msg_body.bat_reply;
+ struct srpc_batch_reply *rep = &msg->msg_body.bat_reply;
__swab32s(&rep->bar_status);
sfw_unpack_sid(rep->bar_sid);
@@ -1516,7 +1516,7 @@
}
if (msg->msg_type == SRPC_MSG_TEST_REQST) {
- srpc_test_reqst_t *req = &msg->msg_body.tes_reqst;
+ struct srpc_test_reqst *req = &msg->msg_body.tes_reqst;
__swab64s(&req->tsr_rpyid);
__swab64s(&req->tsr_bulkid);
@@ -1530,7 +1530,7 @@
}
if (msg->msg_type == SRPC_MSG_TEST_REPLY) {
- srpc_test_reply_t *rep = &msg->msg_body.tes_reply;
+ struct srpc_test_reply *rep = &msg->msg_body.tes_reply;
__swab32s(&rep->tsr_status);
sfw_unpack_sid(rep->tsr_sid);
@@ -1538,7 +1538,7 @@
}
if (msg->msg_type == SRPC_MSG_JOIN_REQST) {
- srpc_join_reqst_t *req = &msg->msg_body.join_reqst;
+ struct srpc_join_reqst *req = &msg->msg_body.join_reqst;
__swab64s(&req->join_rpyid);
sfw_unpack_sid(req->join_sid);
@@ -1546,7 +1546,7 @@
}
if (msg->msg_type == SRPC_MSG_JOIN_REPLY) {
- srpc_join_reply_t *rep = &msg->msg_body.join_reply;
+ struct srpc_join_reply *rep = &msg->msg_body.join_reply;
__swab32s(&rep->join_status);
__swab32s(&rep->join_timeout);
@@ -1558,7 +1558,7 @@
}
void
-sfw_abort_rpc(srpc_client_rpc_t *rpc)
+sfw_abort_rpc(struct srpc_client_rpc *rpc)
{
LASSERT(atomic_read(&rpc->crpc_refcount) > 0);
LASSERT(rpc->crpc_service <= SRPC_FRAMEWORK_SERVICE_MAX_ID);
@@ -1569,7 +1569,7 @@
}
void
-sfw_post_rpc(srpc_client_rpc_t *rpc)
+sfw_post_rpc(struct srpc_client_rpc *rpc)
{
spin_lock(&rpc->crpc_lock);
@@ -1584,7 +1584,7 @@
spin_unlock(&rpc->crpc_lock);
}
-static srpc_service_t sfw_services[] = {
+static struct srpc_service sfw_services[] = {
{
/* sv_id */ SRPC_SERVICE_DEBUG,
/* sv_name */ "debug",
@@ -1628,8 +1628,8 @@
int i;
int rc;
int error;
- srpc_service_t *sv;
- sfw_test_case_t *tsc;
+ struct srpc_service *sv;
+ struct sfw_test_case *tsc;
if (session_timeout < 0) {
CERROR("Session timeout must be non-negative: %d\n",
@@ -1721,8 +1721,8 @@
void
sfw_shutdown(void)
{
- srpc_service_t *sv;
- sfw_test_case_t *tsc;
+ struct srpc_service *sv;
+ struct sfw_test_case *tsc;
int i;
spin_lock(&sfw_data.fw_lock);
@@ -1759,10 +1759,10 @@
}
while (!list_empty(&sfw_data.fw_zombie_rpcs)) {
- srpc_client_rpc_t *rpc;
+ struct srpc_client_rpc *rpc;
rpc = list_entry(sfw_data.fw_zombie_rpcs.next,
- srpc_client_rpc_t, crpc_list);
+ struct srpc_client_rpc, crpc_list);
list_del(&rpc->crpc_list);
LIBCFS_FREE(rpc, srpc_client_rpc_size(rpc));
@@ -1778,7 +1778,7 @@
while (!list_empty(&sfw_data.fw_tests)) {
tsc = list_entry(sfw_data.fw_tests.next,
- sfw_test_case_t, tsc_list);
+ struct sfw_test_case, tsc_list);
srpc_wait_service_shutdown(tsc->tsc_srv_service);
diff --git a/drivers/staging/lustre/lnet/selftest/ping_test.c b/drivers/staging/lustre/lnet/selftest/ping_test.c
index 81a4504..ad26fe9 100644
--- a/drivers/staging/lustre/lnet/selftest/ping_test.c
+++ b/drivers/staging/lustre/lnet/selftest/ping_test.c
@@ -56,9 +56,9 @@
static struct lst_ping_data lst_ping_data;
static int
-ping_client_init(sfw_test_instance_t *tsi)
+ping_client_init(struct sfw_test_instance *tsi)
{
- sfw_session_t *sn = tsi->tsi_batch->bat_session;
+ struct sfw_session *sn = tsi->tsi_batch->bat_session;
LASSERT(tsi->tsi_is_client);
LASSERT(sn && !(sn->sn_features & ~LST_FEATS_MASK));
@@ -70,9 +70,9 @@
}
static void
-ping_client_fini(sfw_test_instance_t *tsi)
+ping_client_fini(struct sfw_test_instance *tsi)
{
- sfw_session_t *sn = tsi->tsi_batch->bat_session;
+ struct sfw_session *sn = tsi->tsi_batch->bat_session;
int errors;
LASSERT(sn);
@@ -86,12 +86,12 @@
}
static int
-ping_client_prep_rpc(sfw_test_unit_t *tsu,
- lnet_process_id_t dest, srpc_client_rpc_t **rpc)
+ping_client_prep_rpc(struct sfw_test_unit *tsu, lnet_process_id_t dest,
+ struct srpc_client_rpc **rpc)
{
- srpc_ping_reqst_t *req;
- sfw_test_instance_t *tsi = tsu->tsu_instance;
- sfw_session_t *sn = tsi->tsi_batch->bat_session;
+ struct srpc_ping_reqst *req;
+ struct sfw_test_instance *tsi = tsu->tsu_instance;
+ struct sfw_session *sn = tsi->tsi_batch->bat_session;
struct timespec64 ts;
int rc;
@@ -118,18 +118,18 @@
}
static void
-ping_client_done_rpc(sfw_test_unit_t *tsu, srpc_client_rpc_t *rpc)
+ping_client_done_rpc(struct sfw_test_unit *tsu, struct srpc_client_rpc *rpc)
{
- sfw_test_instance_t *tsi = tsu->tsu_instance;
- sfw_session_t *sn = tsi->tsi_batch->bat_session;
- srpc_ping_reqst_t *reqst = &rpc->crpc_reqstmsg.msg_body.ping_reqst;
- srpc_ping_reply_t *reply = &rpc->crpc_replymsg.msg_body.ping_reply;
+ struct sfw_test_instance *tsi = tsu->tsu_instance;
+ struct sfw_session *sn = tsi->tsi_batch->bat_session;
+ struct srpc_ping_reqst *reqst = &rpc->crpc_reqstmsg.msg_body.ping_reqst;
+ struct srpc_ping_reply *reply = &rpc->crpc_replymsg.msg_body.ping_reply;
struct timespec64 ts;
LASSERT(sn);
if (rpc->crpc_status) {
- if (!tsi->tsi_stopping) /* rpc could have been aborted */
+ if (!tsi->tsi_stopping) /* rpc could have been aborted */
atomic_inc(&sn->sn_ping_errors);
CERROR("Unable to ping %s (%d): %d\n",
libcfs_id2str(rpc->crpc_dest),
@@ -171,10 +171,10 @@
ping_server_handle(struct srpc_server_rpc *rpc)
{
struct srpc_service *sv = rpc->srpc_scd->scd_svc;
- srpc_msg_t *reqstmsg = &rpc->srpc_reqstbuf->buf_msg;
- srpc_msg_t *replymsg = &rpc->srpc_replymsg;
- srpc_ping_reqst_t *req = &reqstmsg->msg_body.ping_reqst;
- srpc_ping_reply_t *rep = &rpc->srpc_replymsg.msg_body.ping_reply;
+ struct srpc_msg *reqstmsg = &rpc->srpc_reqstbuf->buf_msg;
+ struct srpc_msg *replymsg = &rpc->srpc_replymsg;
+ struct srpc_ping_reqst *req = &reqstmsg->msg_body.ping_reqst;
+ struct srpc_ping_reply *rep = &rpc->srpc_replymsg.msg_body.ping_reply;
LASSERT(sv->sv_id == SRPC_SERVICE_PING);
@@ -210,7 +210,8 @@
return 0;
}
-sfw_test_client_ops_t ping_test_client;
+struct sfw_test_client_ops ping_test_client;
+
void ping_init_test_client(void)
{
ping_test_client.tso_init = ping_client_init;
@@ -219,7 +220,8 @@
ping_test_client.tso_done_rpc = ping_client_done_rpc;
}
-srpc_service_t ping_test_service;
+struct srpc_service ping_test_service;
+
void ping_init_test_service(void)
{
ping_test_service.sv_id = SRPC_SERVICE_PING;
diff --git a/drivers/staging/lustre/lnet/selftest/rpc.c b/drivers/staging/lustre/lnet/selftest/rpc.c
index 7d7748d..3c45a7c 100644
--- a/drivers/staging/lustre/lnet/selftest/rpc.c
+++ b/drivers/staging/lustre/lnet/selftest/rpc.c
@@ -46,19 +46,19 @@
#include "selftest.h"
-typedef enum {
+enum srpc_state {
SRPC_STATE_NONE,
SRPC_STATE_NI_INIT,
SRPC_STATE_EQ_INIT,
SRPC_STATE_RUNNING,
SRPC_STATE_STOPPING,
-} srpc_state_t;
+};
static struct smoketest_rpc {
spinlock_t rpc_glock; /* global lock */
- srpc_service_t *rpc_services[SRPC_SERVICE_MAX_ID + 1];
+ struct srpc_service *rpc_services[SRPC_SERVICE_MAX_ID + 1];
lnet_handle_eq_t rpc_lnet_eq; /* _the_ LNet event queue */
- srpc_state_t rpc_state;
+ enum srpc_state rpc_state;
srpc_counters_t rpc_counters;
__u64 rpc_matchbits; /* matchbits counter */
} srpc_data;
@@ -71,7 +71,7 @@
}
/* forward ref's */
-int srpc_handle_rpc(swi_workitem_t *wi);
+int srpc_handle_rpc(struct swi_workitem *wi);
void srpc_get_counters(srpc_counters_t *cnt)
{
@@ -88,7 +88,7 @@
}
static int
-srpc_add_bulk_page(srpc_bulk_t *bk, struct page *pg, int i, int nob)
+srpc_add_bulk_page(struct srpc_bulk *bk, struct page *pg, int i, int nob)
{
nob = min_t(int, nob, PAGE_SIZE);
@@ -102,7 +102,7 @@
}
void
-srpc_free_bulk(srpc_bulk_t *bk)
+srpc_free_bulk(struct srpc_bulk *bk)
{
int i;
struct page *pg;
@@ -117,25 +117,25 @@
__free_page(pg);
}
- LIBCFS_FREE(bk, offsetof(srpc_bulk_t, bk_iovs[bk->bk_niov]));
+ LIBCFS_FREE(bk, offsetof(struct srpc_bulk, bk_iovs[bk->bk_niov]));
}
-srpc_bulk_t *
+struct srpc_bulk *
srpc_alloc_bulk(int cpt, unsigned bulk_npg, unsigned bulk_len, int sink)
{
- srpc_bulk_t *bk;
+ struct srpc_bulk *bk;
int i;
LASSERT(bulk_npg > 0 && bulk_npg <= LNET_MAX_IOV);
LIBCFS_CPT_ALLOC(bk, lnet_cpt_table(), cpt,
- offsetof(srpc_bulk_t, bk_iovs[bulk_npg]));
+ offsetof(struct srpc_bulk, bk_iovs[bulk_npg]));
if (!bk) {
CERROR("Can't allocate descriptor for %d pages\n", bulk_npg);
return NULL;
}
- memset(bk, 0, offsetof(srpc_bulk_t, bk_iovs[bulk_npg]));
+ memset(bk, 0, offsetof(struct srpc_bulk, bk_iovs[bulk_npg]));
bk->bk_sink = sink;
bk->bk_len = bulk_len;
bk->bk_niov = bulk_npg;
@@ -256,7 +256,7 @@
svc->sv_shuttingdown = 0;
svc->sv_cpt_data = cfs_percpt_alloc(lnet_cpt_table(),
- sizeof(struct srpc_service_cd));
+ sizeof(*svc->sv_cpt_data));
if (!svc->sv_cpt_data)
return -ENOMEM;
@@ -338,7 +338,7 @@
}
int
-srpc_remove_service(srpc_service_t *sv)
+srpc_remove_service(struct srpc_service *sv)
{
int id = sv->sv_id;
@@ -357,7 +357,7 @@
static int
srpc_post_passive_rdma(int portal, int local, __u64 matchbits, void *buf,
int len, int options, lnet_process_id_t peer,
- lnet_handle_md_t *mdh, srpc_event_t *ev)
+ lnet_handle_md_t *mdh, struct srpc_event *ev)
{
int rc;
lnet_md_t md;
@@ -396,7 +396,7 @@
static int
srpc_post_active_rdma(int portal, __u64 matchbits, void *buf, int len,
int options, lnet_process_id_t peer, lnet_nid_t self,
- lnet_handle_md_t *mdh, srpc_event_t *ev)
+ lnet_handle_md_t *mdh, struct srpc_event *ev)
{
int rc;
lnet_md_t md;
@@ -449,7 +449,7 @@
static int
srpc_post_passive_rqtbuf(int service, int local, void *buf, int len,
- lnet_handle_md_t *mdh, srpc_event_t *ev)
+ lnet_handle_md_t *mdh, struct srpc_event *ev)
{
lnet_process_id_t any = { 0 };
@@ -697,7 +697,7 @@
/* called with sv->sv_lock held */
static void
-srpc_service_recycle_buffer(struct srpc_service_cd *scd, srpc_buffer_t *buf)
+srpc_service_recycle_buffer(struct srpc_service_cd *scd, struct srpc_buffer *buf)
__must_hold(&scd->scd_lock)
{
if (!scd->scd_svc->sv_shuttingdown && scd->scd_buf_adjust >= 0) {
@@ -755,11 +755,11 @@
}
void
-srpc_shutdown_service(srpc_service_t *sv)
+srpc_shutdown_service(struct srpc_service *sv)
{
struct srpc_service_cd *scd;
struct srpc_server_rpc *rpc;
- srpc_buffer_t *buf;
+ struct srpc_buffer *buf;
int i;
CDEBUG(D_NET, "Shutting down service: id %d, name %s\n",
@@ -792,9 +792,9 @@
}
static int
-srpc_send_request(srpc_client_rpc_t *rpc)
+srpc_send_request(struct srpc_client_rpc *rpc)
{
- srpc_event_t *ev = &rpc->crpc_reqstev;
+ struct srpc_event *ev = &rpc->crpc_reqstev;
int rc;
ev->ev_fired = 0;
@@ -803,7 +803,7 @@
rc = srpc_post_active_rdma(srpc_serv_portal(rpc->crpc_service),
rpc->crpc_service, &rpc->crpc_reqstmsg,
- sizeof(srpc_msg_t), LNET_MD_OP_PUT,
+ sizeof(struct srpc_msg), LNET_MD_OP_PUT,
rpc->crpc_dest, LNET_NID_ANY,
&rpc->crpc_reqstmdh, ev);
if (rc) {
@@ -814,9 +814,9 @@
}
static int
-srpc_prepare_reply(srpc_client_rpc_t *rpc)
+srpc_prepare_reply(struct srpc_client_rpc *rpc)
{
- srpc_event_t *ev = &rpc->crpc_replyev;
+ struct srpc_event *ev = &rpc->crpc_replyev;
__u64 *id = &rpc->crpc_reqstmsg.msg_body.reqst.rpyid;
int rc;
@@ -827,7 +827,8 @@
*id = srpc_next_id();
rc = srpc_post_passive_rdma(SRPC_RDMA_PORTAL, 0, *id,
- &rpc->crpc_replymsg, sizeof(srpc_msg_t),
+ &rpc->crpc_replymsg,
+ sizeof(struct srpc_msg),
LNET_MD_OP_PUT, rpc->crpc_dest,
&rpc->crpc_replymdh, ev);
if (rc) {
@@ -838,10 +839,10 @@
}
static int
-srpc_prepare_bulk(srpc_client_rpc_t *rpc)
+srpc_prepare_bulk(struct srpc_client_rpc *rpc)
{
- srpc_bulk_t *bk = &rpc->crpc_bulk;
- srpc_event_t *ev = &rpc->crpc_bulkev;
+ struct srpc_bulk *bk = &rpc->crpc_bulk;
+ struct srpc_event *ev = &rpc->crpc_bulkev;
__u64 *id = &rpc->crpc_reqstmsg.msg_body.reqst.bulkid;
int rc;
int opt;
@@ -873,8 +874,8 @@
static int
srpc_do_bulk(struct srpc_server_rpc *rpc)
{
- srpc_event_t *ev = &rpc->srpc_ev;
- srpc_bulk_t *bk = rpc->srpc_bulk;
+ struct srpc_event *ev = &rpc->srpc_ev;
+ struct srpc_bulk *bk = rpc->srpc_bulk;
__u64 id = rpc->srpc_reqstbuf->buf_msg.msg_body.reqst.bulkid;
int rc;
int opt;
@@ -903,7 +904,7 @@
{
struct srpc_service_cd *scd = rpc->srpc_scd;
struct srpc_service *sv = scd->scd_svc;
- srpc_buffer_t *buffer;
+ struct srpc_buffer *buffer;
LASSERT(status || rpc->srpc_wi.swi_state == SWI_STATE_DONE);
@@ -948,7 +949,7 @@
if (!sv->sv_shuttingdown && !list_empty(&scd->scd_buf_blocked)) {
buffer = list_entry(scd->scd_buf_blocked.next,
- srpc_buffer_t, buf_list);
+ struct srpc_buffer, buf_list);
list_del(&buffer->buf_list);
srpc_init_server_rpc(rpc, scd, buffer);
@@ -963,12 +964,12 @@
/* handles an incoming RPC */
int
-srpc_handle_rpc(swi_workitem_t *wi)
+srpc_handle_rpc(struct swi_workitem *wi)
{
struct srpc_server_rpc *rpc = wi->swi_workitem.wi_data;
struct srpc_service_cd *scd = rpc->srpc_scd;
struct srpc_service *sv = scd->scd_svc;
- srpc_event_t *ev = &rpc->srpc_ev;
+ struct srpc_event *ev = &rpc->srpc_ev;
int rc = 0;
LASSERT(wi == &rpc->srpc_wi);
@@ -995,8 +996,8 @@
default:
LBUG();
case SWI_STATE_NEWBORN: {
- srpc_msg_t *msg;
- srpc_generic_reply_t *reply;
+ struct srpc_msg *msg;
+ struct srpc_generic_reply *reply;
msg = &rpc->srpc_reqstbuf->buf_msg;
reply = &rpc->srpc_replymsg.msg_body.reply;
@@ -1077,7 +1078,7 @@
static void
srpc_client_rpc_expired(void *data)
{
- srpc_client_rpc_t *rpc = data;
+ struct srpc_client_rpc *rpc = data;
CWARN("Client RPC expired: service %d, peer %s, timeout %d.\n",
rpc->crpc_service, libcfs_id2str(rpc->crpc_dest),
@@ -1096,7 +1097,7 @@
}
static void
-srpc_add_client_rpc_timer(srpc_client_rpc_t *rpc)
+srpc_add_client_rpc_timer(struct srpc_client_rpc *rpc)
{
struct stt_timer *timer = &rpc->crpc_timer;
@@ -1117,7 +1118,7 @@
* running on any CPU.
*/
static void
-srpc_del_client_rpc_timer(srpc_client_rpc_t *rpc)
+srpc_del_client_rpc_timer(struct srpc_client_rpc *rpc)
{
/* timer not planted or already exploded */
if (!rpc->crpc_timeout)
@@ -1138,9 +1139,9 @@
}
static void
-srpc_client_rpc_done(srpc_client_rpc_t *rpc, int status)
+srpc_client_rpc_done(struct srpc_client_rpc *rpc, int status)
{
- swi_workitem_t *wi = &rpc->crpc_wi;
+ struct swi_workitem *wi = &rpc->crpc_wi;
LASSERT(status || wi->swi_state == SWI_STATE_DONE);
@@ -1175,11 +1176,11 @@
/* sends an outgoing RPC */
int
-srpc_send_rpc(swi_workitem_t *wi)
+srpc_send_rpc(struct swi_workitem *wi)
{
int rc = 0;
- srpc_client_rpc_t *rpc;
- srpc_msg_t *reply;
+ struct srpc_client_rpc *rpc;
+ struct srpc_msg *reply;
int do_bulk;
LASSERT(wi);
@@ -1237,7 +1238,7 @@
wi->swi_state = SWI_STATE_REQUEST_SENT;
/* perhaps more events, fall thru */
case SWI_STATE_REQUEST_SENT: {
- srpc_msg_type_t type = srpc_service2reply(rpc->crpc_service);
+ enum srpc_msg_type type = srpc_service2reply(rpc->crpc_service);
if (!rpc->crpc_replyev.ev_fired)
break;
@@ -1308,15 +1309,15 @@
return 0;
}
-srpc_client_rpc_t *
+struct srpc_client_rpc *
srpc_create_client_rpc(lnet_process_id_t peer, int service,
int nbulkiov, int bulklen,
- void (*rpc_done)(srpc_client_rpc_t *),
- void (*rpc_fini)(srpc_client_rpc_t *), void *priv)
+ void (*rpc_done)(struct srpc_client_rpc *),
+ void (*rpc_fini)(struct srpc_client_rpc *), void *priv)
{
- srpc_client_rpc_t *rpc;
+ struct srpc_client_rpc *rpc;
- LIBCFS_ALLOC(rpc, offsetof(srpc_client_rpc_t,
+ LIBCFS_ALLOC(rpc, offsetof(struct srpc_client_rpc,
crpc_bulk.bk_iovs[nbulkiov]));
if (!rpc)
return NULL;
@@ -1328,12 +1329,12 @@
/* called with rpc->crpc_lock held */
void
-srpc_abort_rpc(srpc_client_rpc_t *rpc, int why)
+srpc_abort_rpc(struct srpc_client_rpc *rpc, int why)
{
LASSERT(why);
- if (rpc->crpc_aborted || /* already aborted */
- rpc->crpc_closed) /* callback imminent */
+ if (rpc->crpc_aborted || /* already aborted */
+ rpc->crpc_closed) /* callback imminent */
return;
CDEBUG(D_NET, "Aborting RPC: service %d, peer %s, state %s, why %d\n",
@@ -1347,7 +1348,7 @@
/* called with rpc->crpc_lock held */
void
-srpc_post_rpc(srpc_client_rpc_t *rpc)
+srpc_post_rpc(struct srpc_client_rpc *rpc)
{
LASSERT(!rpc->crpc_aborted);
LASSERT(srpc_data.rpc_state == SRPC_STATE_RUNNING);
@@ -1363,7 +1364,7 @@
int
srpc_send_reply(struct srpc_server_rpc *rpc)
{
- srpc_event_t *ev = &rpc->srpc_ev;
+ struct srpc_event *ev = &rpc->srpc_ev;
struct srpc_msg *msg = &rpc->srpc_replymsg;
struct srpc_buffer *buffer = rpc->srpc_reqstbuf;
struct srpc_service_cd *scd = rpc->srpc_scd;
@@ -1401,7 +1402,7 @@
rpc->srpc_peer, rpc->srpc_self,
&rpc->srpc_replymdh, ev);
if (rc)
- ev->ev_fired = 1; /* no more event expected */
+ ev->ev_fired = 1; /* no more event expected */
return rc;
}
@@ -1410,13 +1411,13 @@
srpc_lnet_ev_handler(lnet_event_t *ev)
{
struct srpc_service_cd *scd;
- srpc_event_t *rpcev = ev->md.user_ptr;
- srpc_client_rpc_t *crpc;
+ struct srpc_event *rpcev = ev->md.user_ptr;
+ struct srpc_client_rpc *crpc;
struct srpc_server_rpc *srpc;
- srpc_buffer_t *buffer;
- srpc_service_t *sv;
- srpc_msg_t *msg;
- srpc_msg_type_t type;
+ struct srpc_buffer *buffer;
+ struct srpc_service *sv;
+ struct srpc_msg *msg;
+ enum srpc_msg_type type;
LASSERT(!in_interrupt());
@@ -1486,7 +1487,7 @@
LASSERT(ev->type != LNET_EVENT_UNLINK ||
sv->sv_shuttingdown);
- buffer = container_of(ev->md.start, srpc_buffer_t, buf_msg);
+ buffer = container_of(ev->md.start, struct srpc_buffer, buf_msg);
buffer->buf_peer = ev->initiator;
buffer->buf_self = ev->target.nid;
@@ -1509,7 +1510,7 @@
scd->scd_buf_err = 0;
}
- if (!scd->scd_buf_err && /* adding buffer is enabled */
+ if (!scd->scd_buf_err && /* adding buffer is enabled */
!scd->scd_buf_adjust &&
scd->scd_buf_nposted < scd->scd_buf_low) {
scd->scd_buf_adjust = max(scd->scd_buf_total / 2,
@@ -1663,7 +1664,7 @@
spin_lock(&srpc_data.rpc_glock);
for (i = 0; i <= SRPC_SERVICE_MAX_ID; i++) {
- srpc_service_t *sv = srpc_data.rpc_services[i];
+ struct srpc_service *sv = srpc_data.rpc_services[i];
LASSERTF(!sv, "service not empty: id %d, name %s\n",
i, sv->sv_name);
diff --git a/drivers/staging/lustre/lnet/selftest/rpc.h b/drivers/staging/lustre/lnet/selftest/rpc.h
index a79c315..c9b904c 100644
--- a/drivers/staging/lustre/lnet/selftest/rpc.h
+++ b/drivers/staging/lustre/lnet/selftest/rpc.h
@@ -44,7 +44,7 @@
*
* XXX: *REPLY == *REQST + 1
*/
-typedef enum {
+enum srpc_msg_type {
SRPC_MSG_MKSN_REQST = 0,
SRPC_MSG_MKSN_REPLY = 1,
SRPC_MSG_RMSN_REQST = 2,
@@ -63,7 +63,7 @@
SRPC_MSG_PING_REPLY = 15,
SRPC_MSG_JOIN_REQST = 16,
SRPC_MSG_JOIN_REPLY = 17,
-} srpc_msg_type_t;
+};
/* CAVEAT EMPTOR:
* All srpc_*_reqst_t's 1st field must be matchbits of reply buffer,
@@ -72,122 +72,122 @@
* All srpc_*_reply_t's 1st field must be a __u32 status, and 2nd field
* session id if needed.
*/
-typedef struct {
+struct srpc_generic_reqst {
__u64 rpyid; /* reply buffer matchbits */
__u64 bulkid; /* bulk buffer matchbits */
-} WIRE_ATTR srpc_generic_reqst_t;
+} WIRE_ATTR;
-typedef struct {
+struct srpc_generic_reply {
__u32 status;
lst_sid_t sid;
-} WIRE_ATTR srpc_generic_reply_t;
+} WIRE_ATTR;
/* FRAMEWORK RPCs */
-typedef struct {
+struct srpc_mksn_reqst {
__u64 mksn_rpyid; /* reply buffer matchbits */
lst_sid_t mksn_sid; /* session id */
__u32 mksn_force; /* use brute force */
char mksn_name[LST_NAME_SIZE];
-} WIRE_ATTR srpc_mksn_reqst_t; /* make session request */
+} WIRE_ATTR; /* make session request */
-typedef struct {
+struct srpc_mksn_reply {
__u32 mksn_status; /* session status */
lst_sid_t mksn_sid; /* session id */
__u32 mksn_timeout; /* session timeout */
char mksn_name[LST_NAME_SIZE];
-} WIRE_ATTR srpc_mksn_reply_t; /* make session reply */
+} WIRE_ATTR; /* make session reply */
-typedef struct {
+struct srpc_rmsn_reqst {
__u64 rmsn_rpyid; /* reply buffer matchbits */
lst_sid_t rmsn_sid; /* session id */
-} WIRE_ATTR srpc_rmsn_reqst_t; /* remove session request */
+} WIRE_ATTR; /* remove session request */
-typedef struct {
+struct srpc_rmsn_reply {
__u32 rmsn_status;
lst_sid_t rmsn_sid; /* session id */
-} WIRE_ATTR srpc_rmsn_reply_t; /* remove session reply */
+} WIRE_ATTR; /* remove session reply */
-typedef struct {
+struct srpc_join_reqst {
__u64 join_rpyid; /* reply buffer matchbits */
lst_sid_t join_sid; /* session id to join */
char join_group[LST_NAME_SIZE]; /* group name */
-} WIRE_ATTR srpc_join_reqst_t;
+} WIRE_ATTR;
-typedef struct {
+struct srpc_join_reply {
__u32 join_status; /* returned status */
lst_sid_t join_sid; /* session id */
__u32 join_timeout; /* # seconds' inactivity to
* expire */
char join_session[LST_NAME_SIZE]; /* session name */
-} WIRE_ATTR srpc_join_reply_t;
+} WIRE_ATTR;
-typedef struct {
+struct srpc_debug_reqst {
__u64 dbg_rpyid; /* reply buffer matchbits */
lst_sid_t dbg_sid; /* session id */
__u32 dbg_flags; /* bitmap of debug */
-} WIRE_ATTR srpc_debug_reqst_t;
+} WIRE_ATTR;
-typedef struct {
+struct srpc_debug_reply {
__u32 dbg_status; /* returned code */
lst_sid_t dbg_sid; /* session id */
__u32 dbg_timeout; /* session timeout */
__u32 dbg_nbatch; /* # of batches in the node */
char dbg_name[LST_NAME_SIZE]; /* session name */
-} WIRE_ATTR srpc_debug_reply_t;
+} WIRE_ATTR;
#define SRPC_BATCH_OPC_RUN 1
#define SRPC_BATCH_OPC_STOP 2
#define SRPC_BATCH_OPC_QUERY 3
-typedef struct {
+struct srpc_batch_reqst {
__u64 bar_rpyid; /* reply buffer matchbits */
lst_sid_t bar_sid; /* session id */
lst_bid_t bar_bid; /* batch id */
__u32 bar_opc; /* create/start/stop batch */
__u32 bar_testidx; /* index of test */
__u32 bar_arg; /* parameters */
-} WIRE_ATTR srpc_batch_reqst_t;
+} WIRE_ATTR;
-typedef struct {
+struct srpc_batch_reply {
__u32 bar_status; /* status of request */
lst_sid_t bar_sid; /* session id */
__u32 bar_active; /* # of active tests in batch/test */
__u32 bar_time; /* remained time */
-} WIRE_ATTR srpc_batch_reply_t;
+} WIRE_ATTR;
-typedef struct {
+struct srpc_stat_reqst {
__u64 str_rpyid; /* reply buffer matchbits */
lst_sid_t str_sid; /* session id */
__u32 str_type; /* type of stat */
-} WIRE_ATTR srpc_stat_reqst_t;
+} WIRE_ATTR;
-typedef struct {
+struct srpc_stat_reply {
__u32 str_status;
lst_sid_t str_sid;
sfw_counters_t str_fw;
srpc_counters_t str_rpc;
lnet_counters_t str_lnet;
-} WIRE_ATTR srpc_stat_reply_t;
+} WIRE_ATTR;
-typedef struct {
+struct test_bulk_req {
__u32 blk_opc; /* bulk operation code */
__u32 blk_npg; /* # of pages */
__u32 blk_flags; /* reserved flags */
-} WIRE_ATTR test_bulk_req_t;
+} WIRE_ATTR;
-typedef struct {
+struct test_bulk_req_v1 {
__u16 blk_opc; /* bulk operation code */
__u16 blk_flags; /* data check flags */
__u32 blk_len; /* data length */
__u32 blk_offset; /* reserved: offset */
-} WIRE_ATTR test_bulk_req_v1_t;
+} WIRE_ATTR;
-typedef struct {
+struct test_ping_req {
__u32 png_size; /* size of ping message */
__u32 png_flags; /* reserved flags */
-} WIRE_ATTR test_ping_req_t;
+} WIRE_ATTR;
-typedef struct {
+struct srpc_test_reqst {
__u64 tsr_rpyid; /* reply buffer matchbits */
__u64 tsr_bulkid; /* bulk buffer matchbits */
lst_sid_t tsr_sid; /* session id */
@@ -201,82 +201,82 @@
__u32 tsr_ndest; /* # of dest nodes */
union {
- test_ping_req_t ping;
- test_bulk_req_t bulk_v0;
- test_bulk_req_v1_t bulk_v1;
- } tsr_u;
-} WIRE_ATTR srpc_test_reqst_t;
+ struct test_ping_req ping;
+ struct test_bulk_req bulk_v0;
+ struct test_bulk_req_v1 bulk_v1;
+ } tsr_u;
+} WIRE_ATTR;
-typedef struct {
+struct srpc_test_reply {
__u32 tsr_status; /* returned code */
lst_sid_t tsr_sid;
-} WIRE_ATTR srpc_test_reply_t;
+} WIRE_ATTR;
/* TEST RPCs */
-typedef struct {
+struct srpc_ping_reqst {
__u64 pnr_rpyid;
__u32 pnr_magic;
__u32 pnr_seq;
__u64 pnr_time_sec;
__u64 pnr_time_usec;
-} WIRE_ATTR srpc_ping_reqst_t;
+} WIRE_ATTR;
-typedef struct {
+struct srpc_ping_reply {
__u32 pnr_status;
__u32 pnr_magic;
__u32 pnr_seq;
-} WIRE_ATTR srpc_ping_reply_t;
+} WIRE_ATTR;
-typedef struct {
+struct srpc_brw_reqst {
__u64 brw_rpyid; /* reply buffer matchbits */
__u64 brw_bulkid; /* bulk buffer matchbits */
__u32 brw_rw; /* read or write */
__u32 brw_len; /* bulk data len */
__u32 brw_flags; /* bulk data patterns */
-} WIRE_ATTR srpc_brw_reqst_t; /* bulk r/w request */
+} WIRE_ATTR; /* bulk r/w request */
-typedef struct {
+struct srpc_brw_reply {
__u32 brw_status;
-} WIRE_ATTR srpc_brw_reply_t; /* bulk r/w reply */
+} WIRE_ATTR; /* bulk r/w reply */
#define SRPC_MSG_MAGIC 0xeeb0f00d
#define SRPC_MSG_VERSION 1
-typedef struct srpc_msg {
+struct srpc_msg {
__u32 msg_magic; /* magic number */
__u32 msg_version; /* message version number */
- __u32 msg_type; /* type of message body: srpc_msg_type_t */
+ __u32 msg_type; /* type of message body: srpc_msg_type */
__u32 msg_reserved0;
__u32 msg_reserved1;
__u32 msg_ses_feats; /* test session features */
union {
- srpc_generic_reqst_t reqst;
- srpc_generic_reply_t reply;
+ struct srpc_generic_reqst reqst;
+ struct srpc_generic_reply reply;
- srpc_mksn_reqst_t mksn_reqst;
- srpc_mksn_reply_t mksn_reply;
- srpc_rmsn_reqst_t rmsn_reqst;
- srpc_rmsn_reply_t rmsn_reply;
- srpc_debug_reqst_t dbg_reqst;
- srpc_debug_reply_t dbg_reply;
- srpc_batch_reqst_t bat_reqst;
- srpc_batch_reply_t bat_reply;
- srpc_stat_reqst_t stat_reqst;
- srpc_stat_reply_t stat_reply;
- srpc_test_reqst_t tes_reqst;
- srpc_test_reply_t tes_reply;
- srpc_join_reqst_t join_reqst;
- srpc_join_reply_t join_reply;
+ struct srpc_mksn_reqst mksn_reqst;
+ struct srpc_mksn_reply mksn_reply;
+ struct srpc_rmsn_reqst rmsn_reqst;
+ struct srpc_rmsn_reply rmsn_reply;
+ struct srpc_debug_reqst dbg_reqst;
+ struct srpc_debug_reply dbg_reply;
+ struct srpc_batch_reqst bat_reqst;
+ struct srpc_batch_reply bat_reply;
+ struct srpc_stat_reqst stat_reqst;
+ struct srpc_stat_reply stat_reply;
+ struct srpc_test_reqst tes_reqst;
+ struct srpc_test_reply tes_reply;
+ struct srpc_join_reqst join_reqst;
+ struct srpc_join_reply join_reply;
- srpc_ping_reqst_t ping_reqst;
- srpc_ping_reply_t ping_reply;
- srpc_brw_reqst_t brw_reqst;
- srpc_brw_reply_t brw_reply;
+ struct srpc_ping_reqst ping_reqst;
+ struct srpc_ping_reply ping_reply;
+ struct srpc_brw_reqst brw_reqst;
+ struct srpc_brw_reply brw_reply;
} msg_body;
-} WIRE_ATTR srpc_msg_t;
+} WIRE_ATTR;
static inline void
-srpc_unpack_msg_hdr(srpc_msg_t *msg)
+srpc_unpack_msg_hdr(struct srpc_msg *msg)
{
if (msg->msg_magic == SRPC_MSG_MAGIC)
return; /* no flipping needed */
diff --git a/drivers/staging/lustre/lnet/selftest/selftest.h b/drivers/staging/lustre/lnet/selftest/selftest.h
index e689ca1..4eac1c9 100644
--- a/drivers/staging/lustre/lnet/selftest/selftest.h
+++ b/drivers/staging/lustre/lnet/selftest/selftest.h
@@ -93,7 +93,7 @@
/* all reply/bulk RDMAs go to this portal */
#define SRPC_RDMA_PORTAL 52
-static inline srpc_msg_type_t
+static inline enum srpc_msg_type
srpc_service2request(int service)
{
switch (service) {
@@ -128,13 +128,13 @@
}
}
-static inline srpc_msg_type_t
+static inline enum srpc_msg_type
srpc_service2reply(int service)
{
return srpc_service2request(service) + 1;
}
-typedef enum {
+enum srpc_event_type {
SRPC_BULK_REQ_RCVD = 1, /* passive bulk request(PUT sink/GET source)
* received */
SRPC_BULK_PUT_SENT = 2, /* active bulk PUT sent (source) */
@@ -143,57 +143,58 @@
SRPC_REPLY_SENT = 5, /* outgoing reply sent */
SRPC_REQUEST_RCVD = 6, /* incoming request received */
SRPC_REQUEST_SENT = 7, /* outgoing request sent */
-} srpc_event_type_t;
+};
/* RPC event */
-typedef struct {
- srpc_event_type_t ev_type; /* what's up */
+struct srpc_event {
+ enum srpc_event_type ev_type; /* what's up */
lnet_event_kind_t ev_lnet; /* LNet event type */
int ev_fired; /* LNet event fired? */
int ev_status; /* LNet event status */
void *ev_data; /* owning server/client RPC */
-} srpc_event_t;
+};
-typedef struct {
+/* bulk descriptor */
+struct srpc_bulk {
int bk_len; /* len of bulk data */
lnet_handle_md_t bk_mdh;
int bk_sink; /* sink/source */
int bk_niov; /* # iov in bk_iovs */
lnet_kiov_t bk_iovs[0];
-} srpc_bulk_t; /* bulk descriptor */
+};
/* message buffer descriptor */
-typedef struct srpc_buffer {
+struct srpc_buffer {
struct list_head buf_list; /* chain on srpc_service::*_msgq */
- srpc_msg_t buf_msg;
+ struct srpc_msg buf_msg;
lnet_handle_md_t buf_mdh;
lnet_nid_t buf_self;
lnet_process_id_t buf_peer;
-} srpc_buffer_t;
+};
struct swi_workitem;
typedef int (*swi_action_t) (struct swi_workitem *);
-typedef struct swi_workitem {
+struct swi_workitem {
struct cfs_wi_sched *swi_sched;
- cfs_workitem_t swi_workitem;
+ struct cfs_workitem swi_workitem;
swi_action_t swi_action;
int swi_state;
-} swi_workitem_t;
+};
/* server-side state of a RPC */
struct srpc_server_rpc {
/* chain on srpc_service::*_rpcq */
struct list_head srpc_list;
struct srpc_service_cd *srpc_scd;
- swi_workitem_t srpc_wi;
- srpc_event_t srpc_ev; /* bulk/reply event */
+ struct swi_workitem srpc_wi;
+ struct srpc_event srpc_ev; /* bulk/reply event */
lnet_nid_t srpc_self;
lnet_process_id_t srpc_peer;
- srpc_msg_t srpc_replymsg;
+ struct srpc_msg srpc_replymsg;
lnet_handle_md_t srpc_replymdh;
- srpc_buffer_t *srpc_reqstbuf;
- srpc_bulk_t *srpc_bulk;
+ struct srpc_buffer *srpc_reqstbuf;
+ struct srpc_bulk *srpc_bulk;
unsigned int srpc_aborted; /* being given up */
int srpc_status;
@@ -201,14 +202,14 @@
};
/* client-side state of a RPC */
-typedef struct srpc_client_rpc {
+struct srpc_client_rpc {
struct list_head crpc_list; /* chain on user's lists */
spinlock_t crpc_lock; /* serialize */
int crpc_service;
atomic_t crpc_refcount;
int crpc_timeout; /* # seconds to wait for reply */
struct stt_timer crpc_timer;
- swi_workitem_t crpc_wi;
+ struct swi_workitem crpc_wi;
lnet_process_id_t crpc_dest;
void (*crpc_done)(struct srpc_client_rpc *);
@@ -221,20 +222,20 @@
unsigned int crpc_closed:1; /* completed */
/* RPC events */
- srpc_event_t crpc_bulkev; /* bulk event */
- srpc_event_t crpc_reqstev; /* request event */
- srpc_event_t crpc_replyev; /* reply event */
+ struct srpc_event crpc_bulkev; /* bulk event */
+ struct srpc_event crpc_reqstev; /* request event */
+ struct srpc_event crpc_replyev; /* reply event */
/* bulk, request(reqst), and reply exchanged on wire */
- srpc_msg_t crpc_reqstmsg;
- srpc_msg_t crpc_replymsg;
+ struct srpc_msg crpc_reqstmsg;
+ struct srpc_msg crpc_replymsg;
lnet_handle_md_t crpc_reqstmdh;
lnet_handle_md_t crpc_replymdh;
- srpc_bulk_t crpc_bulk;
-} srpc_client_rpc_t;
+ struct srpc_bulk crpc_bulk;
+};
#define srpc_client_rpc_size(rpc) \
-offsetof(srpc_client_rpc_t, crpc_bulk.bk_iovs[(rpc)->crpc_bulk.bk_niov])
+offsetof(struct srpc_client_rpc, crpc_bulk.bk_iovs[(rpc)->crpc_bulk.bk_niov])
#define srpc_client_rpc_addref(rpc) \
do { \
@@ -266,13 +267,13 @@
/** backref to service */
struct srpc_service *scd_svc;
/** event buffer */
- srpc_event_t scd_ev;
+ struct srpc_event scd_ev;
/** free RPC descriptors */
struct list_head scd_rpc_free;
/** in-flight RPCs */
struct list_head scd_rpc_active;
/** workitem for posting buffer */
- swi_workitem_t scd_buf_wi;
+ struct swi_workitem scd_buf_wi;
/** CPT id */
int scd_cpt;
/** error code for scd_buf_wi */
@@ -306,7 +307,7 @@
#define SFW_FRWK_WI_MIN 16
#define SFW_FRWK_WI_MAX 256
-typedef struct srpc_service {
+struct srpc_service {
int sv_id; /* service id */
const char *sv_name; /* human readable name */
int sv_wi_total; /* total server workitems */
@@ -320,9 +321,9 @@
*/
int (*sv_handler)(struct srpc_server_rpc *);
int (*sv_bulk_ready)(struct srpc_server_rpc *, int);
-} srpc_service_t;
+};
-typedef struct {
+struct sfw_session {
struct list_head sn_list; /* chain on fw_zombie_sessions */
lst_sid_t sn_id; /* unique identifier */
unsigned int sn_timeout; /* # seconds' inactivity to expire */
@@ -335,37 +336,37 @@
atomic_t sn_brw_errors;
atomic_t sn_ping_errors;
unsigned long sn_started;
-} sfw_session_t;
+};
#define sfw_sid_equal(sid0, sid1) ((sid0).ses_nid == (sid1).ses_nid && \
(sid0).ses_stamp == (sid1).ses_stamp)
-typedef struct {
+struct sfw_batch {
struct list_head bat_list; /* chain on sn_batches */
lst_bid_t bat_id; /* batch id */
int bat_error; /* error code of batch */
- sfw_session_t *bat_session; /* batch's session */
+ struct sfw_session *bat_session; /* batch's session */
atomic_t bat_nactive; /* # of active tests */
struct list_head bat_tests; /* test instances */
-} sfw_batch_t;
+};
-typedef struct {
+struct sfw_test_client_ops {
int (*tso_init)(struct sfw_test_instance *tsi); /* initialize test
* client */
void (*tso_fini)(struct sfw_test_instance *tsi); /* finalize test
* client */
int (*tso_prep_rpc)(struct sfw_test_unit *tsu,
lnet_process_id_t dest,
- srpc_client_rpc_t **rpc); /* prep a tests rpc */
+ struct srpc_client_rpc **rpc); /* prep a tests rpc */
void (*tso_done_rpc)(struct sfw_test_unit *tsu,
- srpc_client_rpc_t *rpc); /* done a test rpc */
-} sfw_test_client_ops_t;
+ struct srpc_client_rpc *rpc); /* done a test rpc */
+};
-typedef struct sfw_test_instance {
+struct sfw_test_instance {
struct list_head tsi_list; /* chain on batch */
int tsi_service; /* test type */
- sfw_batch_t *tsi_batch; /* batch */
- sfw_test_client_ops_t *tsi_ops; /* test client operation
+ struct sfw_batch *tsi_batch; /* batch */
+ struct sfw_test_client_ops *tsi_ops; /* test client operation
*/
/* public parameter for all test units */
@@ -384,11 +385,11 @@
struct list_head tsi_active_rpcs; /* active rpcs */
union {
- test_ping_req_t ping; /* ping parameter */
- test_bulk_req_t bulk_v0; /* bulk parameter */
- test_bulk_req_v1_t bulk_v1; /* bulk v1 parameter */
+ struct test_ping_req ping; /* ping parameter */
+ struct test_bulk_req bulk_v0; /* bulk parameter */
+ struct test_bulk_req_v1 bulk_v1; /* bulk v1 parameter */
} tsi_u;
-} sfw_test_instance_t;
+};
/* XXX: trailing (PAGE_SIZE % sizeof(lnet_process_id_t)) bytes at the end of
* pages are not used */
@@ -397,57 +398,58 @@
#define SFW_MAX_NDESTS (LNET_MAX_IOV * SFW_ID_PER_PAGE)
#define sfw_id_pages(n) (((n) + SFW_ID_PER_PAGE - 1) / SFW_ID_PER_PAGE)
-typedef struct sfw_test_unit {
+struct sfw_test_unit {
struct list_head tsu_list; /* chain on lst_test_instance */
lnet_process_id_t tsu_dest; /* id of dest node */
int tsu_loop; /* loop count of the test */
- sfw_test_instance_t *tsu_instance; /* pointer to test instance */
+ struct sfw_test_instance *tsu_instance; /* pointer to test instance */
void *tsu_private; /* private data */
- swi_workitem_t tsu_worker; /* workitem of the test unit */
-} sfw_test_unit_t;
+ struct swi_workitem tsu_worker; /* workitem of the test unit */
+};
-typedef struct sfw_test_case {
+struct sfw_test_case {
struct list_head tsc_list; /* chain on fw_tests */
- srpc_service_t *tsc_srv_service; /* test service */
- sfw_test_client_ops_t *tsc_cli_ops; /* ops of test client */
-} sfw_test_case_t;
+ struct srpc_service *tsc_srv_service; /* test service */
+ struct sfw_test_client_ops *tsc_cli_ops; /* ops of test client */
+};
-srpc_client_rpc_t *
+struct srpc_client_rpc *
sfw_create_rpc(lnet_process_id_t peer, int service,
unsigned features, int nbulkiov, int bulklen,
- void (*done)(srpc_client_rpc_t *), void *priv);
-int sfw_create_test_rpc(sfw_test_unit_t *tsu,
+ void (*done)(struct srpc_client_rpc *), void *priv);
+int sfw_create_test_rpc(struct sfw_test_unit *tsu,
lnet_process_id_t peer, unsigned features,
- int nblk, int blklen, srpc_client_rpc_t **rpc);
-void sfw_abort_rpc(srpc_client_rpc_t *rpc);
-void sfw_post_rpc(srpc_client_rpc_t *rpc);
-void sfw_client_rpc_done(srpc_client_rpc_t *rpc);
-void sfw_unpack_message(srpc_msg_t *msg);
+ int nblk, int blklen, struct srpc_client_rpc **rpc);
+void sfw_abort_rpc(struct srpc_client_rpc *rpc);
+void sfw_post_rpc(struct srpc_client_rpc *rpc);
+void sfw_client_rpc_done(struct srpc_client_rpc *rpc);
+void sfw_unpack_message(struct srpc_msg *msg);
void sfw_free_pages(struct srpc_server_rpc *rpc);
-void sfw_add_bulk_page(srpc_bulk_t *bk, struct page *pg, int i);
+void sfw_add_bulk_page(struct srpc_bulk *bk, struct page *pg, int i);
int sfw_alloc_pages(struct srpc_server_rpc *rpc, int cpt, int npages, int len,
int sink);
-int sfw_make_session(srpc_mksn_reqst_t *request, srpc_mksn_reply_t *reply);
+int sfw_make_session(struct srpc_mksn_reqst *request,
+ struct srpc_mksn_reply *reply);
-srpc_client_rpc_t *
+struct srpc_client_rpc *
srpc_create_client_rpc(lnet_process_id_t peer, int service,
int nbulkiov, int bulklen,
- void (*rpc_done)(srpc_client_rpc_t *),
- void (*rpc_fini)(srpc_client_rpc_t *), void *priv);
-void srpc_post_rpc(srpc_client_rpc_t *rpc);
-void srpc_abort_rpc(srpc_client_rpc_t *rpc, int why);
-void srpc_free_bulk(srpc_bulk_t *bk);
-srpc_bulk_t *srpc_alloc_bulk(int cpt, unsigned bulk_npg, unsigned bulk_len,
- int sink);
-int srpc_send_rpc(swi_workitem_t *wi);
+ void (*rpc_done)(struct srpc_client_rpc *),
+ void (*rpc_fini)(struct srpc_client_rpc *), void *priv);
+void srpc_post_rpc(struct srpc_client_rpc *rpc);
+void srpc_abort_rpc(struct srpc_client_rpc *rpc, int why);
+void srpc_free_bulk(struct srpc_bulk *bk);
+struct srpc_bulk *srpc_alloc_bulk(int cpt, unsigned bulk_npg,
+ unsigned bulk_len, int sink);
+int srpc_send_rpc(struct swi_workitem *wi);
int srpc_send_reply(struct srpc_server_rpc *rpc);
-int srpc_add_service(srpc_service_t *sv);
-int srpc_remove_service(srpc_service_t *sv);
-void srpc_shutdown_service(srpc_service_t *sv);
-void srpc_abort_service(srpc_service_t *sv);
-int srpc_finish_service(srpc_service_t *sv);
-int srpc_service_add_buffers(srpc_service_t *sv, int nbuffer);
-void srpc_service_remove_buffers(srpc_service_t *sv, int nbuffer);
+int srpc_add_service(struct srpc_service *sv);
+int srpc_remove_service(struct srpc_service *sv);
+void srpc_shutdown_service(struct srpc_service *sv);
+void srpc_abort_service(struct srpc_service *sv);
+int srpc_finish_service(struct srpc_service *sv);
+int srpc_service_add_buffers(struct srpc_service *sv, int nbuffer);
+void srpc_service_remove_buffers(struct srpc_service *sv, int nbuffer);
void srpc_get_counters(srpc_counters_t *cnt);
void srpc_set_counters(const srpc_counters_t *cnt);
@@ -461,15 +463,17 @@
}
static inline int
-swi_wi_action(cfs_workitem_t *wi)
+swi_wi_action(struct cfs_workitem *wi)
{
- swi_workitem_t *swi = container_of(wi, swi_workitem_t, swi_workitem);
+ struct swi_workitem *swi;
+
+ swi = container_of(wi, struct swi_workitem, swi_workitem);
return swi->swi_action(swi);
}
static inline void
-swi_init_workitem(swi_workitem_t *swi, void *data,
+swi_init_workitem(struct swi_workitem *swi, void *data,
swi_action_t action, struct cfs_wi_sched *sched)
{
swi->swi_sched = sched;
@@ -479,19 +483,19 @@
}
static inline void
-swi_schedule_workitem(swi_workitem_t *wi)
+swi_schedule_workitem(struct swi_workitem *wi)
{
cfs_wi_schedule(wi->swi_sched, &wi->swi_workitem);
}
static inline void
-swi_exit_workitem(swi_workitem_t *swi)
+swi_exit_workitem(struct swi_workitem *swi)
{
cfs_wi_exit(swi->swi_sched, &swi->swi_workitem);
}
static inline int
-swi_deschedule_workitem(swi_workitem_t *swi)
+swi_deschedule_workitem(struct swi_workitem *swi)
{
return cfs_wi_deschedule(swi->swi_sched, &swi->swi_workitem);
}
@@ -502,7 +506,7 @@
void srpc_shutdown(void);
static inline void
-srpc_destroy_client_rpc(srpc_client_rpc_t *rpc)
+srpc_destroy_client_rpc(struct srpc_client_rpc *rpc)
{
LASSERT(rpc);
LASSERT(!srpc_event_pending(rpc));
@@ -515,14 +519,14 @@
}
static inline void
-srpc_init_client_rpc(srpc_client_rpc_t *rpc, lnet_process_id_t peer,
+srpc_init_client_rpc(struct srpc_client_rpc *rpc, lnet_process_id_t peer,
int service, int nbulkiov, int bulklen,
- void (*rpc_done)(srpc_client_rpc_t *),
- void (*rpc_fini)(srpc_client_rpc_t *), void *priv)
+ void (*rpc_done)(struct srpc_client_rpc *),
+ void (*rpc_fini)(struct srpc_client_rpc *), void *priv)
{
LASSERT(nbulkiov <= LNET_MAX_IOV);
- memset(rpc, 0, offsetof(srpc_client_rpc_t,
+ memset(rpc, 0, offsetof(struct srpc_client_rpc,
crpc_bulk.bk_iovs[nbulkiov]));
INIT_LIST_HEAD(&rpc->crpc_list);
@@ -592,7 +596,7 @@
} while (0)
static inline void
-srpc_wait_service_shutdown(srpc_service_t *sv)
+srpc_wait_service_shutdown(struct srpc_service *sv)
{
int i = 2;
@@ -607,16 +611,16 @@
}
}
-extern sfw_test_client_ops_t brw_test_client;
+extern struct sfw_test_client_ops brw_test_client;
void brw_init_test_client(void);
-extern srpc_service_t brw_test_service;
+extern struct srpc_service brw_test_service;
void brw_init_test_service(void);
-extern sfw_test_client_ops_t ping_test_client;
+extern struct sfw_test_client_ops ping_test_client;
void ping_init_test_client(void);
-extern srpc_service_t ping_test_service;
+extern struct srpc_service ping_test_service;
void ping_init_test_service(void);
#endif /* __SELFTEST_SELFTEST_H__ */
diff --git a/drivers/staging/lustre/lnet/selftest/timer.c b/drivers/staging/lustre/lnet/selftest/timer.c
index 8be5252..b6c4aae 100644
--- a/drivers/staging/lustre/lnet/selftest/timer.c
+++ b/drivers/staging/lustre/lnet/selftest/timer.c
@@ -49,7 +49,7 @@
* sorted by increasing expiry time. The number of slots is 2**7 (128),
* to cover a time period of 1024 seconds into the future before wrapping.
*/
-#define STTIMER_MINPOLL 3 /* log2 min poll interval (8 s) */
+#define STTIMER_MINPOLL 3 /* log2 min poll interval (8 s) */
#define STTIMER_SLOTTIME (1 << STTIMER_MINPOLL)
#define STTIMER_SLOTTIMEMASK (~(STTIMER_SLOTTIME - 1))
#define STTIMER_NSLOTS (1 << 7)
@@ -170,20 +170,22 @@
static int
stt_timer_main(void *arg)
{
+ int rc = 0;
+
cfs_block_allsigs();
while (!stt_data.stt_shuttingdown) {
stt_check_timers(&stt_data.stt_prev_slot);
- wait_event_timeout(stt_data.stt_waitq,
- stt_data.stt_shuttingdown,
- cfs_time_seconds(STTIMER_SLOTTIME));
+ rc = wait_event_timeout(stt_data.stt_waitq,
+ stt_data.stt_shuttingdown,
+ cfs_time_seconds(STTIMER_SLOTTIME));
}
spin_lock(&stt_data.stt_lock);
stt_data.stt_nthreads--;
spin_unlock(&stt_data.stt_lock);
- return 0;
+ return rc;
}
static int
diff --git a/drivers/staging/lustre/lustre/fid/fid_request.c b/drivers/staging/lustre/lustre/fid/fid_request.c
index 39269c3..3a4df62 100644
--- a/drivers/staging/lustre/lustre/fid/fid_request.c
+++ b/drivers/staging/lustre/lustre/fid/fid_request.c
@@ -66,6 +66,7 @@
unsigned int debug_mask;
int rc;
+ LASSERT(exp && !IS_ERR(exp));
req = ptlrpc_request_alloc_pack(class_exp2cliimp(exp), &RQF_SEQ_QUERY,
LUSTRE_MDS_VERSION, SEQ_QUERY);
if (!req)
@@ -101,19 +102,22 @@
req->rq_no_delay = req->rq_no_resend = 1;
debug_mask = D_CONSOLE;
} else {
- if (seq->lcs_type == LUSTRE_SEQ_METADATA)
+ if (seq->lcs_type == LUSTRE_SEQ_METADATA) {
+ req->rq_reply_portal = MDC_REPLY_PORTAL;
req->rq_request_portal = SEQ_METADATA_PORTAL;
- else
+ } else {
+ req->rq_reply_portal = OSC_REPLY_PORTAL;
req->rq_request_portal = SEQ_DATA_PORTAL;
+ }
debug_mask = D_INFO;
}
ptlrpc_at_set_req_timeout(req);
- if (seq->lcs_type == LUSTRE_SEQ_METADATA)
+ if (opc != SEQ_ALLOC_SUPER && seq->lcs_type == LUSTRE_SEQ_METADATA)
mdc_get_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL);
rc = ptlrpc_queue_wait(req);
- if (seq->lcs_type == LUSTRE_SEQ_METADATA)
+ if (opc != SEQ_ALLOC_SUPER && seq->lcs_type == LUSTRE_SEQ_METADATA)
mdc_put_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL);
if (rc)
goto out_req;
diff --git a/drivers/staging/lustre/lustre/fld/fld_cache.c b/drivers/staging/lustre/lustre/fld/fld_cache.c
index 062f388..5a04e99 100644
--- a/drivers/staging/lustre/lustre/fld/fld_cache.c
+++ b/drivers/staging/lustre/lustre/fld/fld_cache.c
@@ -178,8 +178,9 @@
if (n_range->lsr_end <= c_range->lsr_end) {
*n_range = *c_range;
fld_cache_entry_delete(cache, f_curr);
- } else
+ } else {
n_range->lsr_start = c_range->lsr_end;
+ }
}
/* we could have overlap over next
diff --git a/drivers/staging/lustre/lustre/fld/fld_internal.h b/drivers/staging/lustre/lustre/fld/fld_internal.h
index e8a3caf..75d6a48 100644
--- a/drivers/staging/lustre/lustre/fld/fld_internal.h
+++ b/drivers/staging/lustre/lustre/fld/fld_internal.h
@@ -101,12 +101,6 @@
unsigned int fci_no_shrink:1;
};
-enum fld_op {
- FLD_CREATE = 0,
- FLD_DELETE = 1,
- FLD_LOOKUP = 2
-};
-
enum {
/* 4M of FLD cache will not hurt client a lot. */
FLD_SERVER_CACHE_SIZE = (4 * 0x100000),
@@ -126,7 +120,8 @@
extern struct lu_fld_hash fld_hash[];
int fld_client_rpc(struct obd_export *exp,
- struct lu_seq_range *range, __u32 fld_op);
+ struct lu_seq_range *range, __u32 fld_op,
+ struct ptlrpc_request **reqp);
extern struct lprocfs_vars fld_client_debugfs_list[];
diff --git a/drivers/staging/lustre/lustre/fld/fld_request.c b/drivers/staging/lustre/lustre/fld/fld_request.c
index a3d122d..304c0ec 100644
--- a/drivers/staging/lustre/lustre/fld/fld_request.c
+++ b/drivers/staging/lustre/lustre/fld/fld_request.c
@@ -64,9 +64,9 @@
{
int rc;
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
rc = list_empty(&mcw->mcw_entry);
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
return rc;
};
@@ -75,15 +75,15 @@
struct mdc_cache_waiter mcw;
struct l_wait_info lwi = { 0 };
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
if (cli->cl_r_in_flight >= cli->cl_max_rpcs_in_flight) {
list_add_tail(&mcw.mcw_entry, &cli->cl_cache_waiters);
init_waitqueue_head(&mcw.mcw_waitq);
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
l_wait_event(mcw.mcw_waitq, fld_req_avail(cli, &mcw), &lwi);
} else {
cli->cl_r_in_flight++;
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
}
}
@@ -92,10 +92,9 @@
struct list_head *l, *tmp;
struct mdc_cache_waiter *mcw;
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
cli->cl_r_in_flight--;
list_for_each_safe(l, tmp, &cli->cl_cache_waiters) {
-
if (cli->cl_r_in_flight >= cli->cl_max_rpcs_in_flight) {
/* No free request slots anymore */
break;
@@ -106,7 +105,7 @@
cli->cl_r_in_flight++;
wake_up(&mcw->mcw_waitq);
}
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
}
static int fld_rrb_hash(struct lu_client_fld *fld, u64 seq)
@@ -392,55 +391,82 @@
EXPORT_SYMBOL(fld_client_fini);
int fld_client_rpc(struct obd_export *exp,
- struct lu_seq_range *range, __u32 fld_op)
+ struct lu_seq_range *range, __u32 fld_op,
+ struct ptlrpc_request **reqp)
{
- struct ptlrpc_request *req;
+ struct ptlrpc_request *req = NULL;
struct lu_seq_range *prange;
__u32 *op;
- int rc;
+ int rc = 0;
struct obd_import *imp;
LASSERT(exp);
imp = class_exp2cliimp(exp);
- req = ptlrpc_request_alloc_pack(imp, &RQF_FLD_QUERY, LUSTRE_MDS_VERSION,
- FLD_QUERY);
- if (!req)
- return -ENOMEM;
+ switch (fld_op) {
+ case FLD_QUERY:
+ req = ptlrpc_request_alloc_pack(imp, &RQF_FLD_QUERY,
+ LUSTRE_MDS_VERSION, FLD_QUERY);
+ if (!req)
+ return -ENOMEM;
- op = req_capsule_client_get(&req->rq_pill, &RMF_FLD_OPC);
- *op = fld_op;
+ /*
+ * XXX: only needed when talking to old server(< 2.6), it should
+ * be removed when < 2.6 server is not supported
+ */
+ op = req_capsule_client_get(&req->rq_pill, &RMF_FLD_OPC);
+ *op = FLD_LOOKUP;
+
+ if (imp->imp_connect_flags_orig & OBD_CONNECT_MDS_MDS)
+ req->rq_allow_replay = 1;
+ break;
+ case FLD_READ:
+ req = ptlrpc_request_alloc_pack(imp, &RQF_FLD_READ,
+ LUSTRE_MDS_VERSION, FLD_READ);
+ if (!req)
+ return -ENOMEM;
+
+ req_capsule_set_size(&req->rq_pill, &RMF_GENERIC_DATA,
+ RCL_SERVER, PAGE_SIZE);
+ break;
+ default:
+ rc = -EINVAL;
+ break;
+ }
+ if (rc)
+ return rc;
prange = req_capsule_client_get(&req->rq_pill, &RMF_FLD_MDFLD);
*prange = *range;
-
ptlrpc_request_set_replen(req);
req->rq_request_portal = FLD_REQUEST_PORTAL;
req->rq_reply_portal = MDC_REPLY_PORTAL;
ptlrpc_at_set_req_timeout(req);
- if (fld_op == FLD_LOOKUP &&
- imp->imp_connect_flags_orig & OBD_CONNECT_MDS_MDS)
- req->rq_allow_replay = 1;
-
- if (fld_op != FLD_LOOKUP)
- mdc_get_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL);
fld_enter_request(&exp->exp_obd->u.cli);
rc = ptlrpc_queue_wait(req);
fld_exit_request(&exp->exp_obd->u.cli);
- if (fld_op != FLD_LOOKUP)
- mdc_put_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL);
if (rc)
goto out_req;
- prange = req_capsule_server_get(&req->rq_pill, &RMF_FLD_MDFLD);
- if (!prange) {
- rc = -EFAULT;
- goto out_req;
+ if (fld_op == FLD_QUERY) {
+ prange = req_capsule_server_get(&req->rq_pill, &RMF_FLD_MDFLD);
+ if (!prange) {
+ rc = -EFAULT;
+ goto out_req;
+ }
+ *range = *prange;
}
- *range = *prange;
+
out_req:
- ptlrpc_req_finished(req);
+ if (rc || !reqp) {
+ ptlrpc_req_finished(req);
+ req = NULL;
+ }
+
+ if (reqp)
+ *reqp = req;
+
return rc;
}
@@ -468,7 +494,7 @@
res.lsr_start = seq;
fld_range_set_type(&res, flags);
- rc = fld_client_rpc(target->ft_exp, &res, FLD_LOOKUP);
+ rc = fld_client_rpc(target->ft_exp, &res, FLD_QUERY, NULL);
if (rc == 0) {
*mds = res.lsr_index;
diff --git a/drivers/staging/lustre/lustre/include/cl_object.h b/drivers/staging/lustre/lustre/include/cl_object.h
index fb971de..d4c33dd 100644
--- a/drivers/staging/lustre/lustre/include/cl_object.h
+++ b/drivers/staging/lustre/lustre/include/cl_object.h
@@ -82,7 +82,6 @@
* - i_mutex
* - PG_locked
* - cl_object_header::coh_page_guard
- * - cl_object_header::coh_lock_guard
* - lu_site::ls_guard
*
* See the top comment in cl_object.c for the description of overall locking and
@@ -98,9 +97,12 @@
* super-class definitions.
*/
#include "lu_object.h"
+#include <linux/atomic.h>
#include "linux/lustre_compat25.h"
#include <linux/mutex.h>
#include <linux/radix-tree.h>
+#include <linux/spinlock.h>
+#include <linux/wait.h>
struct inode;
@@ -138,7 +140,7 @@
* cl_req_slice_add().
*
* \see osc_req_init(), lov_req_init(), lovsub_req_init()
- * \see ccc_req_init()
+ * \see vvp_req_init()
*/
int (*cdo_req_init)(const struct lu_env *env, struct cl_device *dev,
struct cl_req *req);
@@ -147,7 +149,7 @@
/**
* Device in the client stack.
*
- * \see ccc_device, lov_device, lovsub_device, osc_device
+ * \see vvp_device, lov_device, lovsub_device, osc_device
*/
struct cl_device {
/** Super-class. */
@@ -243,7 +245,7 @@
* be discarded from the memory, all its sub-objects are torn-down and
* destroyed too.
*
- * \see ccc_object, lov_object, lovsub_object, osc_object
+ * \see vvp_object, lov_object, lovsub_object, osc_object
*/
struct cl_object {
/** super class */
@@ -322,7 +324,7 @@
* to be used instead of newly created.
*/
int (*coo_page_init)(const struct lu_env *env, struct cl_object *obj,
- struct cl_page *page, struct page *vmpage);
+ struct cl_page *page, pgoff_t index);
/**
* Initialize lock slice for this layer. Called top-to-bottom through
* every object layer when a new cl_lock is instantiated. Layer
@@ -383,11 +385,17 @@
* object. Layers are supposed to fill parts of \a lvb that will be
* shipped to the glimpse originator as a glimpse result.
*
- * \see ccc_object_glimpse(), lovsub_object_glimpse(),
+ * \see vvp_object_glimpse(), lovsub_object_glimpse(),
* \see osc_object_glimpse()
*/
int (*coo_glimpse)(const struct lu_env *env,
const struct cl_object *obj, struct ost_lvb *lvb);
+ /**
+ * Object prune method. Called when the layout is going to change on
+ * this object, therefore each layer has to clean up their cache,
+ * mainly pages and locks.
+ */
+ int (*coo_prune)(const struct lu_env *env, struct cl_object *obj);
};
/**
@@ -398,22 +406,6 @@
* here.
*/
struct lu_object_header coh_lu;
- /** \name locks
- * \todo XXX move locks below to the separate cache-lines, they are
- * mostly useless otherwise.
- */
- /** @{ */
- /** Lock protecting page tree. */
- spinlock_t coh_page_guard;
- /** Lock protecting lock list. */
- spinlock_t coh_lock_guard;
- /** @} locks */
- /** Radix tree of cl_page's, cached for this object. */
- struct radix_tree_root coh_tree;
- /** # of pages in radix tree. */
- unsigned long coh_pages;
- /** List of cl_lock's granted for this object. */
- struct list_head coh_locks;
/**
* Parent object. It is assumed that an object has a well-defined
@@ -460,10 +452,6 @@
co_lu.lo_linkage)
/** @} cl_object */
-#ifndef pgoff_t
-#define pgoff_t unsigned long
-#endif
-
#define CL_PAGE_EOF ((pgoff_t)~0ull)
/** \addtogroup cl_page cl_page
@@ -727,16 +715,10 @@
atomic_t cp_ref;
/** An object this page is a part of. Immutable after creation. */
struct cl_object *cp_obj;
- /** Logical page index within the object. Immutable after creation. */
- pgoff_t cp_index;
/** List of slices. Immutable after creation. */
struct list_head cp_layers;
- /** Parent page, NULL for top-level page. Immutable after creation. */
- struct cl_page *cp_parent;
- /** Lower-layer page. NULL for bottommost page. Immutable after
- * creation.
- */
- struct cl_page *cp_child;
+ /** vmpage */
+ struct page *cp_vmpage;
/**
* Page state. This field is const to avoid accidental update, it is
* modified only internally within cl_page.c. Protected by a VM lock.
@@ -787,10 +769,11 @@
/**
* Per-layer part of cl_page.
*
- * \see ccc_page, lov_page, osc_page
+ * \see vvp_page, lov_page, osc_page
*/
struct cl_page_slice {
struct cl_page *cpl_page;
+ pgoff_t cpl_index;
/**
* Object slice corresponding to this page slice. Immutable after
* creation.
@@ -804,16 +787,9 @@
/**
* Lock mode. For the client extent locks.
*
- * \warning: cl_lock_mode_match() assumes particular ordering here.
* \ingroup cl_lock
*/
enum cl_lock_mode {
- /**
- * Mode of a lock that protects no data, and exists only as a
- * placeholder. This is used for `glimpse' requests. A phantom lock
- * might get promoted to real lock at some point.
- */
- CLM_PHANTOM,
CLM_READ,
CLM_WRITE,
CLM_GROUP
@@ -846,11 +822,6 @@
*/
/**
- * \return the underlying VM page. Optional.
- */
- struct page *(*cpo_vmpage)(const struct lu_env *env,
- const struct cl_page_slice *slice);
- /**
* Called when \a io acquires this page into the exclusive
* ownership. When this method returns, it is guaranteed that the is
* not owned by other io, and no transfer is going on against
@@ -897,14 +868,6 @@
void (*cpo_export)(const struct lu_env *env,
const struct cl_page_slice *slice, int uptodate);
/**
- * Unmaps page from the user space (if it is mapped).
- *
- * \see cl_page_unmap()
- * \see vvp_page_unmap()
- */
- int (*cpo_unmap)(const struct lu_env *env,
- const struct cl_page_slice *slice, struct cl_io *io);
- /**
* Checks whether underlying VM page is locked (in the suitable
* sense). Used for assertions.
*
@@ -957,7 +920,7 @@
*/
int (*cpo_is_under_lock)(const struct lu_env *env,
const struct cl_page_slice *slice,
- struct cl_io *io);
+ struct cl_io *io, pgoff_t *max);
/**
* Optional debugging helper. Prints given page slice.
@@ -1027,26 +990,6 @@
*/
int (*cpo_make_ready)(const struct lu_env *env,
const struct cl_page_slice *slice);
- /**
- * Announce that this page is to be written out
- * opportunistically, that is, page is dirty, it is not
- * necessary to start write-out transfer right now, but
- * eventually page has to be written out.
- *
- * Main caller of this is the write path (see
- * vvp_io_commit_write()), using this method to build a
- * "transfer cache" from which large transfers are then
- * constructed by the req-formation engine.
- *
- * \todo XXX it would make sense to add page-age tracking
- * semantics here, and to oblige the req-formation engine to
- * send the page out not later than it is too old.
- *
- * \see cl_page_cache_add()
- */
- int (*cpo_cache_add)(const struct lu_env *env,
- const struct cl_page_slice *slice,
- struct cl_io *io);
} io[CRT_NR];
/**
* Tell transfer engine that only [to, from] part of a page should be
@@ -1098,9 +1041,8 @@
*/
#define CL_PAGE_DEBUG(mask, env, page, format, ...) \
do { \
- LIBCFS_DEBUG_MSG_DATA_DECL(msgdata, mask, NULL); \
- \
if (cfs_cdebug_show(mask, DEBUG_SUBSYSTEM)) { \
+ LIBCFS_DEBUG_MSG_DATA_DECL(msgdata, mask, NULL); \
cl_page_print(env, &msgdata, lu_cdebug_printer, page); \
CDEBUG(mask, format, ## __VA_ARGS__); \
} \
@@ -1111,9 +1053,8 @@
*/
#define CL_PAGE_HEADER(mask, env, page, format, ...) \
do { \
- LIBCFS_DEBUG_MSG_DATA_DECL(msgdata, mask, NULL); \
- \
if (cfs_cdebug_show(mask, DEBUG_SUBSYSTEM)) { \
+ LIBCFS_DEBUG_MSG_DATA_DECL(msgdata, mask, NULL); \
cl_page_header_print(env, &msgdata, lu_cdebug_printer, page); \
CDEBUG(mask, format, ## __VA_ARGS__); \
} \
@@ -1130,6 +1071,12 @@
#define cl_page_in_use(pg) __page_in_use(pg, 1)
#define cl_page_in_use_noref(pg) __page_in_use(pg, 0)
+static inline struct page *cl_page_vmpage(struct cl_page *page)
+{
+ LASSERT(page->cp_vmpage);
+ return page->cp_vmpage;
+}
+
/** @} cl_page */
/** \addtogroup cl_lock cl_lock
@@ -1150,12 +1097,6 @@
* (struct cl_lock) and a list of layers (struct cl_lock_slice), linked to
* cl_lock::cll_layers list through cl_lock_slice::cls_linkage.
*
- * All locks for a given object are linked into cl_object_header::coh_locks
- * list (protected by cl_object_header::coh_lock_guard spin-lock) through
- * cl_lock::cll_linkage. Currently this list is not sorted in any way. We can
- * sort it in starting lock offset, or use altogether different data structure
- * like a tree.
- *
* Typical cl_lock consists of the two layers:
*
* - vvp_lock (vvp specific data), and
@@ -1177,111 +1118,29 @@
*
* LIFE CYCLE
*
- * cl_lock is reference counted. When reference counter drops to 0, lock is
- * placed in the cache, except when lock is in CLS_FREEING state. CLS_FREEING
- * lock is destroyed when last reference is released. Referencing between
- * top-lock and its sub-locks is described in the lov documentation module.
+ * cl_lock is a cacheless data container for the requirements of locks to
+ * complete the IO. cl_lock is created before I/O starts and destroyed when the
+ * I/O is complete.
*
- * STATE MACHINE
- *
- * Also, cl_lock is a state machine. This requires some clarification. One of
- * the goals of client IO re-write was to make IO path non-blocking, or at
- * least to make it easier to make it non-blocking in the future. Here
- * `non-blocking' means that when a system call (read, write, truncate)
- * reaches a situation where it has to wait for a communication with the
- * server, it should --instead of waiting-- remember its current state and
- * switch to some other work. E.g,. instead of waiting for a lock enqueue,
- * client should proceed doing IO on the next stripe, etc. Obviously this is
- * rather radical redesign, and it is not planned to be fully implemented at
- * this time, instead we are putting some infrastructure in place, that would
- * make it easier to do asynchronous non-blocking IO easier in the
- * future. Specifically, where old locking code goes to sleep (waiting for
- * enqueue, for example), new code returns cl_lock_transition::CLO_WAIT. When
- * enqueue reply comes, its completion handler signals that lock state-machine
- * is ready to transit to the next state. There is some generic code in
- * cl_lock.c that sleeps, waiting for these signals. As a result, for users of
- * this cl_lock.c code, it looks like locking is done in normal blocking
- * fashion, and it the same time it is possible to switch to the non-blocking
- * locking (simply by returning cl_lock_transition::CLO_WAIT from cl_lock.c
- * functions).
- *
- * For a description of state machine states and transitions see enum
- * cl_lock_state.
- *
- * There are two ways to restrict a set of states which lock might move to:
- *
- * - placing a "hold" on a lock guarantees that lock will not be moved
- * into cl_lock_state::CLS_FREEING state until hold is released. Hold
- * can be only acquired on a lock that is not in
- * cl_lock_state::CLS_FREEING. All holds on a lock are counted in
- * cl_lock::cll_holds. Hold protects lock from cancellation and
- * destruction. Requests to cancel and destroy a lock on hold will be
- * recorded, but only honored when last hold on a lock is released;
- *
- * - placing a "user" on a lock guarantees that lock will not leave
- * cl_lock_state::CLS_NEW, cl_lock_state::CLS_QUEUING,
- * cl_lock_state::CLS_ENQUEUED and cl_lock_state::CLS_HELD set of
- * states, once it enters this set. That is, if a user is added onto a
- * lock in a state not from this set, it doesn't immediately enforce
- * lock to move to this set, but once lock enters this set it will
- * remain there until all users are removed. Lock users are counted in
- * cl_lock::cll_users.
- *
- * User is used to assure that lock is not canceled or destroyed while
- * it is being enqueued, or actively used by some IO.
- *
- * Currently, a user always comes with a hold (cl_lock_invariant()
- * checks that a number of holds is not less than a number of users).
- *
- * CONCURRENCY
- *
- * This is how lock state-machine operates. struct cl_lock contains a mutex
- * cl_lock::cll_guard that protects struct fields.
- *
- * - mutex is taken, and cl_lock::cll_state is examined.
- *
- * - for every state there are possible target states where lock can move
- * into. They are tried in order. Attempts to move into next state are
- * done by _try() functions in cl_lock.c:cl_{enqueue,unlock,wait}_try().
- *
- * - if the transition can be performed immediately, state is changed,
- * and mutex is released.
- *
- * - if the transition requires blocking, _try() function returns
- * cl_lock_transition::CLO_WAIT. Caller unlocks mutex and goes to
- * sleep, waiting for possibility of lock state change. It is woken
- * up when some event occurs, that makes lock state change possible
- * (e.g., the reception of the reply from the server), and repeats
- * the loop.
- *
- * Top-lock and sub-lock has separate mutexes and the latter has to be taken
- * first to avoid dead-lock.
- *
- * To see an example of interaction of all these issues, take a look at the
- * lov_cl.c:lov_lock_enqueue() function. It is called as a part of
- * cl_enqueue_try(), and tries to advance top-lock to ENQUEUED state, by
- * advancing state-machines of its sub-locks (lov_lock_enqueue_one()). Note
- * also, that it uses trylock to grab sub-lock mutex to avoid dead-lock. It
- * also has to handle CEF_ASYNC enqueue, when sub-locks enqueues have to be
- * done in parallel, rather than one after another (this is used for glimpse
- * locks, that cannot dead-lock).
+ * cl_lock depends on LDLM lock to fulfill lock semantics. LDLM lock is attached
+ * to cl_lock at OSC layer. LDLM lock is still cacheable.
*
* INTERFACE AND USAGE
*
- * struct cl_lock_operations provide a number of call-backs that are invoked
- * when events of interest occurs. Layers can intercept and handle glimpse,
- * blocking, cancel ASTs and a reception of the reply from the server.
+ * Two major methods are supported for cl_lock: clo_enqueue and clo_cancel. A
+ * cl_lock is enqueued by cl_lock_request(), which will call clo_enqueue()
+ * methods for each layer to enqueue the lock. At the LOV layer, if a cl_lock
+ * consists of multiple sub cl_locks, each sub locks will be enqueued
+ * correspondingly. At OSC layer, the lock enqueue request will tend to reuse
+ * cached LDLM lock; otherwise a new LDLM lock will have to be requested from
+ * OST side.
*
- * One important difference with the old client locking model is that new
- * client has a representation for the top-lock, whereas in the old code only
- * sub-locks existed as real data structures and file-level locks are
- * represented by "request sets" that are created and destroyed on each and
- * every lock creation.
+ * cl_lock_cancel() must be called to release a cl_lock after use. clo_cancel()
+ * method will be called for each layer to release the resource held by this
+ * lock. At OSC layer, the reference count of LDLM lock, which is held at
+ * clo_enqueue time, is released.
*
- * Top-locks are cached, and can be found in the cache by the system calls. It
- * is possible that top-lock is in cache, but some of its sub-locks were
- * canceled and destroyed. In that case top-lock has to be enqueued again
- * before it can be used.
+ * LDLM lock can only be canceled if there is no cl_lock using it.
*
* Overall process of the locking during IO operation is as following:
*
@@ -1294,7 +1153,7 @@
*
* - when all locks are acquired, IO is performed;
*
- * - locks are released into cache.
+ * - locks are released after IO is complete.
*
* Striping introduces major additional complexity into locking. The
* fundamental problem is that it is generally unsafe to actively use (hold)
@@ -1316,16 +1175,6 @@
* buf is a part of memory mapped Lustre file, a lock or locks protecting buf
* has to be held together with the usual lock on [offset, offset + count].
*
- * As multi-stripe locks have to be allowed, it makes sense to cache them, so
- * that, for example, a sequence of O_APPEND writes can proceed quickly
- * without going down to the individual stripes to do lock matching. On the
- * other hand, multi-stripe locks shouldn't be used by normal read/write
- * calls. To achieve this, every layer can implement ->clo_fits_into() method,
- * that is called by lock matching code (cl_lock_lookup()), and that can be
- * used to selectively disable matching of certain locks for certain IOs. For
- * example, lov layer implements lov_lock_fits_into() that allow multi-stripe
- * locks to be matched only for truncates and O_APPEND writes.
- *
* Interaction with DLM
*
* In the expected setup, cl_lock is ultimately backed up by a collection of
@@ -1356,295 +1205,27 @@
__u32 cld_enq_flags;
};
-#define DDESCR "%s(%d):[%lu, %lu]"
+#define DDESCR "%s(%d):[%lu, %lu]:%x"
#define PDESCR(descr) \
cl_lock_mode_name((descr)->cld_mode), (descr)->cld_mode, \
- (descr)->cld_start, (descr)->cld_end
+ (descr)->cld_start, (descr)->cld_end, (descr)->cld_enq_flags
const char *cl_lock_mode_name(const enum cl_lock_mode mode);
/**
- * Lock state-machine states.
- *
- * \htmlonly
- * <pre>
- *
- * Possible state transitions:
- *
- * +------------------>NEW
- * | |
- * | | cl_enqueue_try()
- * | |
- * | cl_unuse_try() V
- * | +--------------QUEUING (*)
- * | | |
- * | | | cl_enqueue_try()
- * | | |
- * | | cl_unuse_try() V
- * sub-lock | +-------------ENQUEUED (*)
- * canceled | | |
- * | | | cl_wait_try()
- * | | |
- * | | (R)
- * | | |
- * | | V
- * | | HELD<---------+
- * | | | |
- * | | | | cl_use_try()
- * | | cl_unuse_try() | |
- * | | | |
- * | | V ---+
- * | +------------>INTRANSIT (D) <--+
- * | | |
- * | cl_unuse_try() | | cached lock found
- * | | | cl_use_try()
- * | | |
- * | V |
- * +------------------CACHED---------+
- * |
- * (C)
- * |
- * V
- * FREEING
- *
- * Legend:
- *
- * In states marked with (*) transition to the same state (i.e., a loop
- * in the diagram) is possible.
- *
- * (R) is the point where Receive call-back is invoked: it allows layers
- * to handle arrival of lock reply.
- *
- * (C) is the point where Cancellation call-back is invoked.
- *
- * (D) is the transit state which means the lock is changing.
- *
- * Transition to FREEING state is possible from any other state in the
- * diagram in case of unrecoverable error.
- * </pre>
- * \endhtmlonly
- *
- * These states are for individual cl_lock object. Top-lock and its sub-locks
- * can be in the different states. Another way to say this is that we have
- * nested state-machines.
- *
- * Separate QUEUING and ENQUEUED states are needed to support non-blocking
- * operation for locks with multiple sub-locks. Imagine lock on a file F, that
- * intersects 3 stripes S0, S1, and S2. To enqueue F client has to send
- * enqueue to S0, wait for its completion, then send enqueue for S1, wait for
- * its completion and at last enqueue lock for S2, and wait for its
- * completion. In that case, top-lock is in QUEUING state while S0, S1 are
- * handled, and is in ENQUEUED state after enqueue to S2 has been sent (note
- * that in this case, sub-locks move from state to state, and top-lock remains
- * in the same state).
- */
-enum cl_lock_state {
- /**
- * Lock that wasn't yet enqueued
- */
- CLS_NEW,
- /**
- * Enqueue is in progress, blocking for some intermediate interaction
- * with the other side.
- */
- CLS_QUEUING,
- /**
- * Lock is fully enqueued, waiting for server to reply when it is
- * granted.
- */
- CLS_ENQUEUED,
- /**
- * Lock granted, actively used by some IO.
- */
- CLS_HELD,
- /**
- * This state is used to mark the lock is being used, or unused.
- * We need this state because the lock may have several sublocks,
- * so it's impossible to have an atomic way to bring all sublocks
- * into CLS_HELD state at use case, or all sublocks to CLS_CACHED
- * at unuse case.
- * If a thread is referring to a lock, and it sees the lock is in this
- * state, it must wait for the lock.
- * See state diagram for details.
- */
- CLS_INTRANSIT,
- /**
- * Lock granted, not used.
- */
- CLS_CACHED,
- /**
- * Lock is being destroyed.
- */
- CLS_FREEING,
- CLS_NR
-};
-
-enum cl_lock_flags {
- /**
- * lock has been cancelled. This flag is never cleared once set (by
- * cl_lock_cancel0()).
- */
- CLF_CANCELLED = 1 << 0,
- /** cancellation is pending for this lock. */
- CLF_CANCELPEND = 1 << 1,
- /** destruction is pending for this lock. */
- CLF_DOOMED = 1 << 2,
- /** from enqueue RPC reply upcall. */
- CLF_FROM_UPCALL = 1 << 3,
-};
-
-/**
- * Lock closure.
- *
- * Lock closure is a collection of locks (both top-locks and sub-locks) that
- * might be updated in a result of an operation on a certain lock (which lock
- * this is a closure of).
- *
- * Closures are needed to guarantee dead-lock freedom in the presence of
- *
- * - nested state-machines (top-lock state-machine composed of sub-lock
- * state-machines), and
- *
- * - shared sub-locks.
- *
- * Specifically, many operations, such as lock enqueue, wait, unlock,
- * etc. start from a top-lock, and then operate on a sub-locks of this
- * top-lock, holding a top-lock mutex. When sub-lock state changes as a result
- * of such operation, this change has to be propagated to all top-locks that
- * share this sub-lock. Obviously, no natural lock ordering (e.g.,
- * top-to-bottom or bottom-to-top) captures this scenario, so try-locking has
- * to be used. Lock closure systematizes this try-and-repeat logic.
- */
-struct cl_lock_closure {
- /**
- * Lock that is mutexed when closure construction is started. When
- * closure in is `wait' mode (cl_lock_closure::clc_wait), mutex on
- * origin is released before waiting.
- */
- struct cl_lock *clc_origin;
- /**
- * List of enclosed locks, so far. Locks are linked here through
- * cl_lock::cll_inclosure.
- */
- struct list_head clc_list;
- /**
- * True iff closure is in a `wait' mode. This determines what
- * cl_lock_enclosure() does when a lock L to be added to the closure
- * is currently mutexed by some other thread.
- *
- * If cl_lock_closure::clc_wait is not set, then closure construction
- * fails with CLO_REPEAT immediately.
- *
- * In wait mode, cl_lock_enclosure() waits until next attempt to build
- * a closure might succeed. To this end it releases an origin mutex
- * (cl_lock_closure::clc_origin), that has to be the only lock mutex
- * owned by the current thread, and then waits on L mutex (by grabbing
- * it and immediately releasing), before returning CLO_REPEAT to the
- * caller.
- */
- int clc_wait;
- /** Number of locks in the closure. */
- int clc_nr;
-};
-
-/**
* Layered client lock.
*/
struct cl_lock {
- /** Reference counter. */
- atomic_t cll_ref;
/** List of slices. Immutable after creation. */
struct list_head cll_layers;
- /**
- * Linkage into cl_lock::cll_descr::cld_obj::coh_locks list. Protected
- * by cl_lock::cll_descr::cld_obj::coh_lock_guard.
- */
- struct list_head cll_linkage;
- /**
- * Parameters of this lock. Protected by
- * cl_lock::cll_descr::cld_obj::coh_lock_guard nested within
- * cl_lock::cll_guard. Modified only on lock creation and in
- * cl_lock_modify().
- */
+ /** lock attribute, extent, cl_object, etc. */
struct cl_lock_descr cll_descr;
- /** Protected by cl_lock::cll_guard. */
- enum cl_lock_state cll_state;
- /** signals state changes. */
- wait_queue_head_t cll_wq;
- /**
- * Recursive lock, most fields in cl_lock{} are protected by this.
- *
- * Locking rules: this mutex is never held across network
- * communication, except when lock is being canceled.
- *
- * Lock ordering: a mutex of a sub-lock is taken first, then a mutex
- * on a top-lock. Other direction is implemented through a
- * try-lock-repeat loop. Mutices of unrelated locks can be taken only
- * by try-locking.
- *
- * \see osc_lock_enqueue_wait(), lov_lock_cancel(), lov_sublock_wait().
- */
- struct mutex cll_guard;
- struct task_struct *cll_guarder;
- int cll_depth;
-
- /**
- * the owner for INTRANSIT state
- */
- struct task_struct *cll_intransit_owner;
- int cll_error;
- /**
- * Number of holds on a lock. A hold prevents a lock from being
- * canceled and destroyed. Protected by cl_lock::cll_guard.
- *
- * \see cl_lock_hold(), cl_lock_unhold(), cl_lock_release()
- */
- int cll_holds;
- /**
- * Number of lock users. Valid in cl_lock_state::CLS_HELD state
- * only. Lock user pins lock in CLS_HELD state. Protected by
- * cl_lock::cll_guard.
- *
- * \see cl_wait(), cl_unuse().
- */
- int cll_users;
- /**
- * Flag bit-mask. Values from enum cl_lock_flags. Updates are
- * protected by cl_lock::cll_guard.
- */
- unsigned long cll_flags;
- /**
- * A linkage into a list of locks in a closure.
- *
- * \see cl_lock_closure
- */
- struct list_head cll_inclosure;
- /**
- * Confict lock at queuing time.
- */
- struct cl_lock *cll_conflict;
- /**
- * A list of references to this lock, for debugging.
- */
- struct lu_ref cll_reference;
- /**
- * A list of holds on this lock, for debugging.
- */
- struct lu_ref cll_holders;
- /**
- * A reference for cl_lock::cll_descr::cld_obj. For debugging.
- */
- struct lu_ref_link cll_obj_ref;
-#ifdef CONFIG_LOCKDEP
- /* "dep_map" name is assumed by lockdep.h macros. */
- struct lockdep_map dep_map;
-#endif
};
/**
* Per-layer part of cl_lock
*
- * \see ccc_lock, lov_lock, lovsub_lock, osc_lock
+ * \see vvp_lock, lov_lock, lovsub_lock, osc_lock
*/
struct cl_lock_slice {
struct cl_lock *cls_lock;
@@ -1658,174 +1239,36 @@
};
/**
- * Possible (non-error) return values of ->clo_{enqueue,wait,unlock}().
- *
- * NOTE: lov_subresult() depends on ordering here.
- */
-enum cl_lock_transition {
- /** operation cannot be completed immediately. Wait for state change. */
- CLO_WAIT = 1,
- /** operation had to release lock mutex, restart. */
- CLO_REPEAT = 2,
- /** lower layer re-enqueued. */
- CLO_REENQUEUED = 3,
-};
-
-/**
*
* \see vvp_lock_ops, lov_lock_ops, lovsub_lock_ops, osc_lock_ops
*/
struct cl_lock_operations {
- /**
- * \name statemachine
- *
- * State machine transitions. These 3 methods are called to transfer
- * lock from one state to another, as described in the commentary
- * above enum #cl_lock_state.
- *
- * \retval 0 this layer has nothing more to do to before
- * transition to the target state happens;
- *
- * \retval CLO_REPEAT method had to release and re-acquire cl_lock
- * mutex, repeat invocation of transition method
- * across all layers;
- *
- * \retval CLO_WAIT this layer cannot move to the target state
- * immediately, as it has to wait for certain event
- * (e.g., the communication with the server). It
- * is guaranteed, that when the state transfer
- * becomes possible, cl_lock::cll_wq wait-queue
- * is signaled. Caller can wait for this event by
- * calling cl_lock_state_wait();
- *
- * \retval -ve failure, abort state transition, move the lock
- * into cl_lock_state::CLS_FREEING state, and set
- * cl_lock::cll_error.
- *
- * Once all layers voted to agree to transition (by returning 0), lock
- * is moved into corresponding target state. All state transition
- * methods are optional.
- */
/** @{ */
/**
* Attempts to enqueue the lock. Called top-to-bottom.
*
- * \see ccc_lock_enqueue(), lov_lock_enqueue(), lovsub_lock_enqueue(),
+ * \retval 0 this layer has enqueued the lock successfully
+ * \retval >0 this layer has enqueued the lock, but need to wait on
+ * @anchor for resources
+ * \retval -ve failure
+ *
+ * \see vvp_lock_enqueue(), lov_lock_enqueue(), lovsub_lock_enqueue(),
* \see osc_lock_enqueue()
*/
int (*clo_enqueue)(const struct lu_env *env,
const struct cl_lock_slice *slice,
- struct cl_io *io, __u32 enqflags);
+ struct cl_io *io, struct cl_sync_io *anchor);
/**
- * Attempts to wait for enqueue result. Called top-to-bottom.
- *
- * \see ccc_lock_wait(), lov_lock_wait(), osc_lock_wait()
- */
- int (*clo_wait)(const struct lu_env *env,
- const struct cl_lock_slice *slice);
- /**
- * Attempts to unlock the lock. Called bottom-to-top. In addition to
- * usual return values of lock state-machine methods, this can return
- * -ESTALE to indicate that lock cannot be returned to the cache, and
- * has to be re-initialized.
- * unuse is a one-shot operation, so it must NOT return CLO_WAIT.
- *
- * \see ccc_lock_unuse(), lov_lock_unuse(), osc_lock_unuse()
- */
- int (*clo_unuse)(const struct lu_env *env,
- const struct cl_lock_slice *slice);
- /**
- * Notifies layer that cached lock is started being used.
- *
- * \pre lock->cll_state == CLS_CACHED
- *
- * \see lov_lock_use(), osc_lock_use()
- */
- int (*clo_use)(const struct lu_env *env,
- const struct cl_lock_slice *slice);
- /** @} statemachine */
- /**
- * A method invoked when lock state is changed (as a result of state
- * transition). This is used, for example, to track when the state of
- * a sub-lock changes, to propagate this change to the corresponding
- * top-lock. Optional
- *
- * \see lovsub_lock_state()
- */
- void (*clo_state)(const struct lu_env *env,
- const struct cl_lock_slice *slice,
- enum cl_lock_state st);
- /**
- * Returns true, iff given lock is suitable for the given io, idea
- * being, that there are certain "unsafe" locks, e.g., ones acquired
- * for O_APPEND writes, that we don't want to re-use for a normal
- * write, to avoid the danger of cascading evictions. Optional. Runs
- * under cl_object_header::coh_lock_guard.
- *
- * XXX this should take more information about lock needed by
- * io. Probably lock description or something similar.
- *
- * \see lov_fits_into()
- */
- int (*clo_fits_into)(const struct lu_env *env,
- const struct cl_lock_slice *slice,
- const struct cl_lock_descr *need,
- const struct cl_io *io);
- /**
- * \name ast
- * Asynchronous System Traps. All of then are optional, all are
- * executed bottom-to-top.
- */
- /** @{ */
-
- /**
- * Cancellation callback. Cancel a lock voluntarily, or under
- * the request of server.
+ * Cancel a lock, release its DLM lock ref, while does not cancel the
+ * DLM lock
*/
void (*clo_cancel)(const struct lu_env *env,
const struct cl_lock_slice *slice);
- /**
- * Lock weighting ast. Executed to estimate how precious this lock
- * is. The sum of results across all layers is used to determine
- * whether lock worth keeping in cache given present memory usage.
- *
- * \see osc_lock_weigh(), vvp_lock_weigh(), lovsub_lock_weigh().
- */
- unsigned long (*clo_weigh)(const struct lu_env *env,
- const struct cl_lock_slice *slice);
- /** @} ast */
-
- /**
- * \see lovsub_lock_closure()
- */
- int (*clo_closure)(const struct lu_env *env,
- const struct cl_lock_slice *slice,
- struct cl_lock_closure *closure);
- /**
- * Executed bottom-to-top when lock description changes (e.g., as a
- * result of server granting more generous lock than was requested).
- *
- * \see lovsub_lock_modify()
- */
- int (*clo_modify)(const struct lu_env *env,
- const struct cl_lock_slice *slice,
- const struct cl_lock_descr *updated);
- /**
- * Notifies layers (bottom-to-top) that lock is going to be
- * destroyed. Responsibility of layers is to prevent new references on
- * this lock from being acquired once this method returns.
- *
- * This can be called multiple times due to the races.
- *
- * \see cl_lock_delete()
- * \see osc_lock_delete(), lovsub_lock_delete()
- */
- void (*clo_delete)(const struct lu_env *env,
- const struct cl_lock_slice *slice);
+ /** @} */
/**
* Destructor. Frees resources and the slice.
*
- * \see ccc_lock_fini(), lov_lock_fini(), lovsub_lock_fini(),
+ * \see vvp_lock_fini(), lov_lock_fini(), lovsub_lock_fini(),
* \see osc_lock_fini()
*/
void (*clo_fini)(const struct lu_env *env, struct cl_lock_slice *slice);
@@ -2016,7 +1459,7 @@
* This is usually embedded into layer session data, rather than allocated
* dynamically.
*
- * \see vvp_io, lov_io, osc_io, ccc_io
+ * \see vvp_io, lov_io, osc_io
*/
struct cl_io_slice {
struct cl_io *cis_io;
@@ -2031,6 +1474,8 @@
struct list_head cis_linkage;
};
+typedef void (*cl_commit_cbt)(const struct lu_env *, struct cl_io *,
+ struct cl_page *);
/**
* Per-layer io operations.
* \see vvp_io_ops, lov_io_ops, lovsub_io_ops, osc_io_ops
@@ -2114,7 +1559,7 @@
void (*cio_fini)(const struct lu_env *env,
const struct cl_io_slice *slice);
} op[CIT_OP_NR];
- struct {
+
/**
* Submit pages from \a queue->c2_qin for IO, and move
* successfully submitted pages into \a queue->c2_qout. Return
@@ -2127,7 +1572,15 @@
const struct cl_io_slice *slice,
enum cl_req_type crt,
struct cl_2queue *queue);
- } req_op[CRT_NR];
+ /**
+ * Queue async page for write.
+ * The difference between cio_submit and cio_queue is that
+ * cio_submit is for urgent request.
+ */
+ int (*cio_commit_async)(const struct lu_env *env,
+ const struct cl_io_slice *slice,
+ struct cl_page_list *queue, int from, int to,
+ cl_commit_cbt cb);
/**
* Read missing page.
*
@@ -2140,31 +1593,6 @@
const struct cl_io_slice *slice,
const struct cl_page_slice *page);
/**
- * Prepare write of a \a page. Called bottom-to-top by a top-level
- * cl_io_operations::op[CIT_WRITE]::cio_start() to prepare page for
- * get data from user-level buffer.
- *
- * \pre io->ci_type == CIT_WRITE
- *
- * \see vvp_io_prepare_write(), lov_io_prepare_write(),
- * osc_io_prepare_write().
- */
- int (*cio_prepare_write)(const struct lu_env *env,
- const struct cl_io_slice *slice,
- const struct cl_page_slice *page,
- unsigned from, unsigned to);
- /**
- *
- * \pre io->ci_type == CIT_WRITE
- *
- * \see vvp_io_commit_write(), lov_io_commit_write(),
- * osc_io_commit_write().
- */
- int (*cio_commit_write)(const struct lu_env *env,
- const struct cl_io_slice *slice,
- const struct cl_page_slice *page,
- unsigned from, unsigned to);
- /**
* Optional debugging helper. Print given io slice.
*/
int (*cio_print)(const struct lu_env *env, void *cookie,
@@ -2216,9 +1644,13 @@
*/
CEF_AGL = 0x00000020,
/**
+ * enqueue a lock to test DLM lock existence.
+ */
+ CEF_PEEK = 0x00000040,
+ /**
* mask of enq_flags.
*/
- CEF_MASK = 0x0000003f,
+ CEF_MASK = 0x0000007f,
};
/**
@@ -2228,12 +1660,12 @@
struct cl_io_lock_link {
/** linkage into one of cl_lockset lists. */
struct list_head cill_linkage;
- struct cl_lock_descr cill_descr;
- struct cl_lock *cill_lock;
+ struct cl_lock cill_lock;
/** optional destructor */
void (*cill_fini)(const struct lu_env *env,
struct cl_io_lock_link *link);
};
+#define cill_descr cill_lock.cll_descr
/**
* Lock-set represents a collection of locks, that io needs at a
@@ -2267,8 +1699,6 @@
struct cl_lockset {
/** locks to be acquired. */
struct list_head cls_todo;
- /** locks currently being processed. */
- struct list_head cls_curr;
/** locks acquired. */
struct list_head cls_done;
};
@@ -2632,9 +2062,7 @@
* and top-locks (and top-pages) are accounted here.
*/
struct cache_stats cs_pages;
- struct cache_stats cs_locks;
atomic_t cs_pages_state[CPS_NR];
- atomic_t cs_locks_state[CLS_NR];
};
int cl_site_init(struct cl_site *s, struct cl_device *top);
@@ -2725,7 +2153,7 @@
}
void cl_page_slice_add(struct cl_page *page, struct cl_page_slice *slice,
- struct cl_object *obj,
+ struct cl_object *obj, pgoff_t index,
const struct cl_page_operations *ops);
void cl_lock_slice_add(struct cl_lock *lock, struct cl_lock_slice *slice,
struct cl_object *obj,
@@ -2758,7 +2186,7 @@
struct ost_lvb *lvb);
int cl_conf_set(const struct lu_env *env, struct cl_object *obj,
const struct cl_object_conf *conf);
-void cl_object_prune(const struct lu_env *env, struct cl_object *obj);
+int cl_object_prune(const struct lu_env *env, struct cl_object *obj);
void cl_object_kill(const struct lu_env *env, struct cl_object *obj);
/**
@@ -2772,7 +2200,7 @@
static inline void cl_object_page_init(struct cl_object *clob, int size)
{
clob->co_slice_off = cl_object_header(clob)->coh_page_bufsize;
- cl_object_header(clob)->coh_page_bufsize += ALIGN(size, 8);
+ cl_object_header(clob)->coh_page_bufsize += cfs_size_round(size);
}
static inline void *cl_object_page_slice(struct cl_object *clob,
@@ -2781,6 +2209,16 @@
return (void *)((char *)page + clob->co_slice_off);
}
+/**
+ * Return refcount of cl_object.
+ */
+static inline int cl_object_refc(struct cl_object *clob)
+{
+ struct lu_object_header *header = clob->co_lu.lo_header;
+
+ return atomic_read(&header->loh_ref);
+}
+
/** @} cl_object */
/** \defgroup cl_page cl_page
@@ -2794,28 +2232,20 @@
};
/* callback of cl_page_gang_lookup() */
-typedef int (*cl_page_gang_cb_t) (const struct lu_env *, struct cl_io *,
- struct cl_page *, void *);
-int cl_page_gang_lookup(const struct lu_env *env, struct cl_object *obj,
- struct cl_io *io, pgoff_t start, pgoff_t end,
- cl_page_gang_cb_t cb, void *cbdata);
-struct cl_page *cl_page_lookup(struct cl_object_header *hdr, pgoff_t index);
struct cl_page *cl_page_find(const struct lu_env *env, struct cl_object *obj,
pgoff_t idx, struct page *vmpage,
enum cl_page_type type);
-struct cl_page *cl_page_find_sub(const struct lu_env *env,
- struct cl_object *obj,
- pgoff_t idx, struct page *vmpage,
- struct cl_page *parent);
+struct cl_page *cl_page_alloc(const struct lu_env *env,
+ struct cl_object *o, pgoff_t ind,
+ struct page *vmpage,
+ enum cl_page_type type);
void cl_page_get(struct cl_page *page);
void cl_page_put(const struct lu_env *env, struct cl_page *page);
void cl_page_print(const struct lu_env *env, void *cookie, lu_printer_t printer,
const struct cl_page *pg);
void cl_page_header_print(const struct lu_env *env, void *cookie,
lu_printer_t printer, const struct cl_page *pg);
-struct page *cl_page_vmpage(const struct lu_env *env, struct cl_page *page);
struct cl_page *cl_vmpage_page(struct page *vmpage, struct cl_object *obj);
-struct cl_page *cl_page_top(struct cl_page *page);
const struct cl_page_slice *cl_page_at(const struct cl_page *page,
const struct lu_device_type *dtype);
@@ -2872,12 +2302,10 @@
void cl_page_discard(const struct lu_env *env, struct cl_io *io,
struct cl_page *pg);
void cl_page_delete(const struct lu_env *env, struct cl_page *pg);
-int cl_page_unmap(const struct lu_env *env, struct cl_io *io,
- struct cl_page *pg);
int cl_page_is_vmlocked(const struct lu_env *env, const struct cl_page *pg);
void cl_page_export(const struct lu_env *env, struct cl_page *pg, int uptodate);
int cl_page_is_under_lock(const struct lu_env *env, struct cl_io *io,
- struct cl_page *page);
+ struct cl_page *page, pgoff_t *max_index);
loff_t cl_offset(const struct cl_object *obj, pgoff_t idx);
pgoff_t cl_index(const struct cl_object *obj, loff_t offset);
int cl_page_size(const struct cl_object *obj);
@@ -2890,138 +2318,66 @@
const struct cl_lock_descr *descr);
/* @} helper */
+/**
+ * Data structure managing a client's cached pages. A count of
+ * "unstable" pages is maintained, and an LRU of clean pages is
+ * maintained. "unstable" pages are pages pinned by the ptlrpc
+ * layer for recovery purposes.
+ */
+struct cl_client_cache {
+ /**
+ * # of users (OSCs)
+ */
+ atomic_t ccc_users;
+ /**
+ * # of threads are doing shrinking
+ */
+ unsigned int ccc_lru_shrinkers;
+ /**
+ * # of LRU entries available
+ */
+ atomic_t ccc_lru_left;
+ /**
+ * List of entities(OSCs) for this LRU cache
+ */
+ struct list_head ccc_lru;
+ /**
+ * Max # of LRU entries
+ */
+ unsigned long ccc_lru_max;
+ /**
+ * Lock to protect ccc_lru list
+ */
+ spinlock_t ccc_lru_lock;
+ /**
+ * # of unstable pages for this mount point
+ */
+ atomic_t ccc_unstable_nr;
+ /**
+ * Waitq for awaiting unstable pages to reach zero.
+ * Used at umounting time and signaled on BRW commit
+ */
+ wait_queue_head_t ccc_unstable_waitq;
+
+};
+
/** @} cl_page */
/** \defgroup cl_lock cl_lock
* @{
*/
-struct cl_lock *cl_lock_hold(const struct lu_env *env, const struct cl_io *io,
- const struct cl_lock_descr *need,
- const char *scope, const void *source);
-struct cl_lock *cl_lock_peek(const struct lu_env *env, const struct cl_io *io,
- const struct cl_lock_descr *need,
- const char *scope, const void *source);
-struct cl_lock *cl_lock_request(const struct lu_env *env, struct cl_io *io,
- const struct cl_lock_descr *need,
- const char *scope, const void *source);
-struct cl_lock *cl_lock_at_pgoff(const struct lu_env *env,
- struct cl_object *obj, pgoff_t index,
- struct cl_lock *except, int pending,
- int canceld);
-static inline struct cl_lock *cl_lock_at_page(const struct lu_env *env,
- struct cl_object *obj,
- struct cl_page *page,
- struct cl_lock *except,
- int pending, int canceld)
-{
- LASSERT(cl_object_header(obj) == cl_object_header(page->cp_obj));
- return cl_lock_at_pgoff(env, obj, page->cp_index, except,
- pending, canceld);
-}
-
+int cl_lock_request(const struct lu_env *env, struct cl_io *io,
+ struct cl_lock *lock);
+int cl_lock_init(const struct lu_env *env, struct cl_lock *lock,
+ const struct cl_io *io);
+void cl_lock_fini(const struct lu_env *env, struct cl_lock *lock);
const struct cl_lock_slice *cl_lock_at(const struct cl_lock *lock,
const struct lu_device_type *dtype);
-
-void cl_lock_get(struct cl_lock *lock);
-void cl_lock_get_trust(struct cl_lock *lock);
-void cl_lock_put(const struct lu_env *env, struct cl_lock *lock);
-void cl_lock_hold_add(const struct lu_env *env, struct cl_lock *lock,
- const char *scope, const void *source);
-void cl_lock_hold_release(const struct lu_env *env, struct cl_lock *lock,
- const char *scope, const void *source);
-void cl_lock_unhold(const struct lu_env *env, struct cl_lock *lock,
- const char *scope, const void *source);
-void cl_lock_release(const struct lu_env *env, struct cl_lock *lock,
- const char *scope, const void *source);
-void cl_lock_user_add(const struct lu_env *env, struct cl_lock *lock);
-void cl_lock_user_del(const struct lu_env *env, struct cl_lock *lock);
-
-int cl_lock_is_intransit(struct cl_lock *lock);
-
-int cl_lock_enqueue_wait(const struct lu_env *env, struct cl_lock *lock,
- int keep_mutex);
-
-/** \name statemachine statemachine
- * Interface to lock state machine consists of 3 parts:
- *
- * - "try" functions that attempt to effect a state transition. If state
- * transition is not possible right now (e.g., if it has to wait for some
- * asynchronous event to occur), these functions return
- * cl_lock_transition::CLO_WAIT.
- *
- * - "non-try" functions that implement synchronous blocking interface on
- * top of non-blocking "try" functions. These functions repeatedly call
- * corresponding "try" versions, and if state transition is not possible
- * immediately, wait for lock state change.
- *
- * - methods from cl_lock_operations, called by "try" functions. Lock can
- * be advanced to the target state only when all layers voted that they
- * are ready for this transition. "Try" functions call methods under lock
- * mutex. If a layer had to release a mutex, it re-acquires it and returns
- * cl_lock_transition::CLO_REPEAT, causing "try" function to call all
- * layers again.
- *
- * TRY NON-TRY METHOD FINAL STATE
- *
- * cl_enqueue_try() cl_enqueue() cl_lock_operations::clo_enqueue() CLS_ENQUEUED
- *
- * cl_wait_try() cl_wait() cl_lock_operations::clo_wait() CLS_HELD
- *
- * cl_unuse_try() cl_unuse() cl_lock_operations::clo_unuse() CLS_CACHED
- *
- * cl_use_try() NONE cl_lock_operations::clo_use() CLS_HELD
- *
- * @{
- */
-
-int cl_wait(const struct lu_env *env, struct cl_lock *lock);
-void cl_unuse(const struct lu_env *env, struct cl_lock *lock);
-int cl_enqueue_try(const struct lu_env *env, struct cl_lock *lock,
- struct cl_io *io, __u32 flags);
-int cl_unuse_try(const struct lu_env *env, struct cl_lock *lock);
-int cl_wait_try(const struct lu_env *env, struct cl_lock *lock);
-int cl_use_try(const struct lu_env *env, struct cl_lock *lock, int atomic);
-
-/** @} statemachine */
-
-void cl_lock_signal(const struct lu_env *env, struct cl_lock *lock);
-int cl_lock_state_wait(const struct lu_env *env, struct cl_lock *lock);
-void cl_lock_state_set(const struct lu_env *env, struct cl_lock *lock,
- enum cl_lock_state state);
-int cl_queue_match(const struct list_head *queue,
- const struct cl_lock_descr *need);
-
-void cl_lock_mutex_get(const struct lu_env *env, struct cl_lock *lock);
-void cl_lock_mutex_put(const struct lu_env *env, struct cl_lock *lock);
-int cl_lock_is_mutexed(struct cl_lock *lock);
-int cl_lock_nr_mutexed(const struct lu_env *env);
-int cl_lock_discard_pages(const struct lu_env *env, struct cl_lock *lock);
-int cl_lock_ext_match(const struct cl_lock_descr *has,
- const struct cl_lock_descr *need);
-int cl_lock_descr_match(const struct cl_lock_descr *has,
- const struct cl_lock_descr *need);
-int cl_lock_mode_match(enum cl_lock_mode has, enum cl_lock_mode need);
-int cl_lock_modify(const struct lu_env *env, struct cl_lock *lock,
- const struct cl_lock_descr *desc);
-
-void cl_lock_closure_init(const struct lu_env *env,
- struct cl_lock_closure *closure,
- struct cl_lock *origin, int wait);
-void cl_lock_closure_fini(struct cl_lock_closure *closure);
-int cl_lock_closure_build(const struct lu_env *env, struct cl_lock *lock,
- struct cl_lock_closure *closure);
-void cl_lock_disclosure(const struct lu_env *env,
- struct cl_lock_closure *closure);
-int cl_lock_enclosure(const struct lu_env *env, struct cl_lock *lock,
- struct cl_lock_closure *closure);
-
+void cl_lock_release(const struct lu_env *env, struct cl_lock *lock);
+int cl_lock_enqueue(const struct lu_env *env, struct cl_io *io,
+ struct cl_lock *lock, struct cl_sync_io *anchor);
void cl_lock_cancel(const struct lu_env *env, struct cl_lock *lock);
-void cl_lock_delete(const struct lu_env *env, struct cl_lock *lock);
-void cl_lock_error(const struct lu_env *env, struct cl_lock *lock, int error);
-void cl_locks_prune(const struct lu_env *env, struct cl_object *obj, int wait);
-
-unsigned long cl_lock_weigh(const struct lu_env *env, struct cl_lock *lock);
/** @} cl_lock */
@@ -3050,15 +2406,14 @@
struct cl_lock_descr *descr);
int cl_io_read_page(const struct lu_env *env, struct cl_io *io,
struct cl_page *page);
-int cl_io_prepare_write(const struct lu_env *env, struct cl_io *io,
- struct cl_page *page, unsigned from, unsigned to);
-int cl_io_commit_write(const struct lu_env *env, struct cl_io *io,
- struct cl_page *page, unsigned from, unsigned to);
int cl_io_submit_rw(const struct lu_env *env, struct cl_io *io,
enum cl_req_type iot, struct cl_2queue *queue);
int cl_io_submit_sync(const struct lu_env *env, struct cl_io *io,
enum cl_req_type iot, struct cl_2queue *queue,
long timeout);
+int cl_io_commit_async(const struct lu_env *env, struct cl_io *io,
+ struct cl_page_list *queue, int from, int to,
+ cl_commit_cbt cb);
int cl_io_is_going(const struct lu_env *env);
/**
@@ -3114,6 +2469,12 @@
return list_entry(plist->pl_pages.prev, struct cl_page, cp_batch);
}
+static inline struct cl_page *cl_page_list_first(struct cl_page_list *plist)
+{
+ LASSERT(plist->pl_nr > 0);
+ return list_entry(plist->pl_pages.next, struct cl_page, cp_batch);
+}
+
/**
* Iterate over pages in a page list.
*/
@@ -3130,9 +2491,14 @@
void cl_page_list_add(struct cl_page_list *plist, struct cl_page *page);
void cl_page_list_move(struct cl_page_list *dst, struct cl_page_list *src,
struct cl_page *page);
+void cl_page_list_move_head(struct cl_page_list *dst, struct cl_page_list *src,
+ struct cl_page *page);
void cl_page_list_splice(struct cl_page_list *list, struct cl_page_list *head);
+void cl_page_list_del(const struct lu_env *env, struct cl_page_list *plist,
+ struct cl_page *page);
void cl_page_list_disown(const struct lu_env *env,
struct cl_io *io, struct cl_page_list *plist);
+void cl_page_list_fini(const struct lu_env *env, struct cl_page_list *plist);
void cl_2queue_init(struct cl_2queue *queue);
void cl_2queue_disown(const struct lu_env *env,
@@ -3177,13 +2543,18 @@
atomic_t csi_barrier;
/** completion to be signaled when transfer is complete. */
wait_queue_head_t csi_waitq;
+ /** callback to invoke when this IO is finished */
+ void (*csi_end_io)(const struct lu_env *,
+ struct cl_sync_io *);
};
-void cl_sync_io_init(struct cl_sync_io *anchor, int nrpages);
-int cl_sync_io_wait(const struct lu_env *env, struct cl_io *io,
- struct cl_page_list *queue, struct cl_sync_io *anchor,
+void cl_sync_io_init(struct cl_sync_io *anchor, int nr,
+ void (*end)(const struct lu_env *, struct cl_sync_io *));
+int cl_sync_io_wait(const struct lu_env *env, struct cl_sync_io *anchor,
long timeout);
-void cl_sync_io_note(struct cl_sync_io *anchor, int ioret);
+void cl_sync_io_note(const struct lu_env *env, struct cl_sync_io *anchor,
+ int ioret);
+void cl_sync_io_end(const struct lu_env *env, struct cl_sync_io *anchor);
/** @} cl_sync_io */
@@ -3241,6 +2612,9 @@
void cl_env_reexit(void *cookie);
void cl_env_implant(struct lu_env *env, int *refcheck);
void cl_env_unplant(struct lu_env *env, int *refcheck);
+unsigned int cl_env_cache_purge(unsigned int nr);
+struct lu_env *cl_env_percpu_get(void);
+void cl_env_percpu_put(struct lu_env *env);
/** @} cl_env */
diff --git a/drivers/staging/lustre/lustre/include/lclient.h b/drivers/staging/lustre/lustre/include/lclient.h
deleted file mode 100644
index 5d839a9..0000000
--- a/drivers/staging/lustre/lustre/include/lclient.h
+++ /dev/null
@@ -1,408 +0,0 @@
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.sun.com/software/products/lustre/docs/GPLv2.pdf
- *
- * Please contact Sun Microsystems, Inc., 4150 Network Circle, Santa Clara,
- * CA 95054 USA or visit www.sun.com if you need additional information or
- * have any questions.
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2011, 2012, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- *
- * Definitions shared between vvp and liblustre, and other clients in the
- * future.
- *
- * Author: Oleg Drokin <oleg.drokin@sun.com>
- * Author: Nikita Danilov <nikita.danilov@sun.com>
- */
-
-#ifndef LCLIENT_H
-#define LCLIENT_H
-
-blkcnt_t dirty_cnt(struct inode *inode);
-
-int cl_glimpse_size0(struct inode *inode, int agl);
-int cl_glimpse_lock(const struct lu_env *env, struct cl_io *io,
- struct inode *inode, struct cl_object *clob, int agl);
-
-static inline int cl_glimpse_size(struct inode *inode)
-{
- return cl_glimpse_size0(inode, 0);
-}
-
-static inline int cl_agl(struct inode *inode)
-{
- return cl_glimpse_size0(inode, 1);
-}
-
-/**
- * Locking policy for setattr.
- */
-enum ccc_setattr_lock_type {
- /** Locking is done by server */
- SETATTR_NOLOCK,
- /** Extent lock is enqueued */
- SETATTR_EXTENT_LOCK,
- /** Existing local extent lock is used */
- SETATTR_MATCH_LOCK
-};
-
-/**
- * IO state private to vvp or slp layers.
- */
-struct ccc_io {
- /** super class */
- struct cl_io_slice cui_cl;
- struct cl_io_lock_link cui_link;
- /**
- * I/O vector information to or from which read/write is going.
- */
- struct iov_iter *cui_iter;
- /**
- * Total size for the left IO.
- */
- size_t cui_tot_count;
-
- union {
- struct {
- enum ccc_setattr_lock_type cui_local_lock;
- } setattr;
- } u;
- /**
- * True iff io is processing glimpse right now.
- */
- int cui_glimpse;
- /**
- * Layout version when this IO is initialized
- */
- __u32 cui_layout_gen;
- /**
- * File descriptor against which IO is done.
- */
- struct ll_file_data *cui_fd;
- struct kiocb *cui_iocb;
-};
-
-/**
- * True, if \a io is a normal io, False for splice_{read,write}.
- * must be implemented in arch specific code.
- */
-int cl_is_normalio(const struct lu_env *env, const struct cl_io *io);
-
-extern struct lu_context_key ccc_key;
-extern struct lu_context_key ccc_session_key;
-
-struct ccc_thread_info {
- struct cl_lock_descr cti_descr;
- struct cl_io cti_io;
- struct cl_attr cti_attr;
-};
-
-static inline struct ccc_thread_info *ccc_env_info(const struct lu_env *env)
-{
- struct ccc_thread_info *info;
-
- info = lu_context_key_get(&env->le_ctx, &ccc_key);
- LASSERT(info);
- return info;
-}
-
-static inline struct cl_attr *ccc_env_thread_attr(const struct lu_env *env)
-{
- struct cl_attr *attr = &ccc_env_info(env)->cti_attr;
-
- memset(attr, 0, sizeof(*attr));
- return attr;
-}
-
-static inline struct cl_io *ccc_env_thread_io(const struct lu_env *env)
-{
- struct cl_io *io = &ccc_env_info(env)->cti_io;
-
- memset(io, 0, sizeof(*io));
- return io;
-}
-
-struct ccc_session {
- struct ccc_io cs_ios;
-};
-
-static inline struct ccc_session *ccc_env_session(const struct lu_env *env)
-{
- struct ccc_session *ses;
-
- ses = lu_context_key_get(env->le_ses, &ccc_session_key);
- LASSERT(ses);
- return ses;
-}
-
-static inline struct ccc_io *ccc_env_io(const struct lu_env *env)
-{
- return &ccc_env_session(env)->cs_ios;
-}
-
-/**
- * ccc-private object state.
- */
-struct ccc_object {
- struct cl_object_header cob_header;
- struct cl_object cob_cl;
- struct inode *cob_inode;
-
- /**
- * A list of dirty pages pending IO in the cache. Used by
- * SOM. Protected by ll_inode_info::lli_lock.
- *
- * \see ccc_page::cpg_pending_linkage
- */
- struct list_head cob_pending_list;
-
- /**
- * Access this counter is protected by inode->i_sem. Now that
- * the lifetime of transient pages must be covered by inode sem,
- * we don't need to hold any lock..
- */
- int cob_transient_pages;
- /**
- * Number of outstanding mmaps on this file.
- *
- * \see ll_vm_open(), ll_vm_close().
- */
- atomic_t cob_mmap_cnt;
-
- /**
- * various flags
- * cob_discard_page_warned
- * if pages belonging to this object are discarded when a client
- * is evicted, some debug info will be printed, this flag will be set
- * during processing the first discarded page, then avoid flooding
- * debug message for lots of discarded pages.
- *
- * \see ll_dirty_page_discard_warn.
- */
- unsigned int cob_discard_page_warned:1;
-};
-
-/**
- * ccc-private page state.
- */
-struct ccc_page {
- struct cl_page_slice cpg_cl;
- int cpg_defer_uptodate;
- int cpg_ra_used;
- int cpg_write_queued;
- /**
- * Non-empty iff this page is already counted in
- * ccc_object::cob_pending_list. Protected by
- * ccc_object::cob_pending_guard. This list is only used as a flag,
- * that is, never iterated through, only checked for list_empty(), but
- * having a list is useful for debugging.
- */
- struct list_head cpg_pending_linkage;
- /** VM page */
- struct page *cpg_page;
-};
-
-static inline struct ccc_page *cl2ccc_page(const struct cl_page_slice *slice)
-{
- return container_of(slice, struct ccc_page, cpg_cl);
-}
-
-struct ccc_device {
- struct cl_device cdv_cl;
- struct super_block *cdv_sb;
- struct cl_device *cdv_next;
-};
-
-struct ccc_lock {
- struct cl_lock_slice clk_cl;
-};
-
-struct ccc_req {
- struct cl_req_slice crq_cl;
-};
-
-void *ccc_key_init (const struct lu_context *ctx,
- struct lu_context_key *key);
-void ccc_key_fini (const struct lu_context *ctx,
- struct lu_context_key *key, void *data);
-void *ccc_session_key_init(const struct lu_context *ctx,
- struct lu_context_key *key);
-void ccc_session_key_fini(const struct lu_context *ctx,
- struct lu_context_key *key, void *data);
-
-int ccc_device_init (const struct lu_env *env,
- struct lu_device *d,
- const char *name, struct lu_device *next);
-struct lu_device *ccc_device_fini (const struct lu_env *env,
- struct lu_device *d);
-struct lu_device *ccc_device_alloc(const struct lu_env *env,
- struct lu_device_type *t,
- struct lustre_cfg *cfg,
- const struct lu_device_operations *luops,
- const struct cl_device_operations *clops);
-struct lu_device *ccc_device_free (const struct lu_env *env,
- struct lu_device *d);
-struct lu_object *ccc_object_alloc(const struct lu_env *env,
- const struct lu_object_header *hdr,
- struct lu_device *dev,
- const struct cl_object_operations *clops,
- const struct lu_object_operations *luops);
-
-int ccc_req_init(const struct lu_env *env, struct cl_device *dev,
- struct cl_req *req);
-void ccc_umount(const struct lu_env *env, struct cl_device *dev);
-int ccc_global_init(struct lu_device_type *device_type);
-void ccc_global_fini(struct lu_device_type *device_type);
-int ccc_object_init0(const struct lu_env *env, struct ccc_object *vob,
- const struct cl_object_conf *conf);
-int ccc_object_init(const struct lu_env *env, struct lu_object *obj,
- const struct lu_object_conf *conf);
-void ccc_object_free(const struct lu_env *env, struct lu_object *obj);
-int ccc_lock_init(const struct lu_env *env, struct cl_object *obj,
- struct cl_lock *lock, const struct cl_io *io,
- const struct cl_lock_operations *lkops);
-int ccc_object_glimpse(const struct lu_env *env,
- const struct cl_object *obj, struct ost_lvb *lvb);
-struct page *ccc_page_vmpage(const struct lu_env *env,
- const struct cl_page_slice *slice);
-int ccc_page_is_under_lock(const struct lu_env *env,
- const struct cl_page_slice *slice, struct cl_io *io);
-int ccc_fail(const struct lu_env *env, const struct cl_page_slice *slice);
-int ccc_transient_page_prep(const struct lu_env *env,
- const struct cl_page_slice *slice,
- struct cl_io *io);
-void ccc_lock_delete(const struct lu_env *env,
- const struct cl_lock_slice *slice);
-void ccc_lock_fini(const struct lu_env *env, struct cl_lock_slice *slice);
-int ccc_lock_enqueue(const struct lu_env *env,
- const struct cl_lock_slice *slice,
- struct cl_io *io, __u32 enqflags);
-int ccc_lock_use(const struct lu_env *env, const struct cl_lock_slice *slice);
-int ccc_lock_unuse(const struct lu_env *env, const struct cl_lock_slice *slice);
-int ccc_lock_wait(const struct lu_env *env, const struct cl_lock_slice *slice);
-int ccc_lock_fits_into(const struct lu_env *env,
- const struct cl_lock_slice *slice,
- const struct cl_lock_descr *need,
- const struct cl_io *io);
-void ccc_lock_state(const struct lu_env *env,
- const struct cl_lock_slice *slice,
- enum cl_lock_state state);
-
-int ccc_io_one_lock_index(const struct lu_env *env, struct cl_io *io,
- __u32 enqflags, enum cl_lock_mode mode,
- pgoff_t start, pgoff_t end);
-int ccc_io_one_lock(const struct lu_env *env, struct cl_io *io,
- __u32 enqflags, enum cl_lock_mode mode,
- loff_t start, loff_t end);
-void ccc_io_end(const struct lu_env *env, const struct cl_io_slice *ios);
-void ccc_io_advance(const struct lu_env *env, const struct cl_io_slice *ios,
- size_t nob);
-void ccc_io_update_iov(const struct lu_env *env, struct ccc_io *cio,
- struct cl_io *io);
-int ccc_prep_size(const struct lu_env *env, struct cl_object *obj,
- struct cl_io *io, loff_t start, size_t count, int *exceed);
-void ccc_req_completion(const struct lu_env *env,
- const struct cl_req_slice *slice, int ioret);
-void ccc_req_attr_set(const struct lu_env *env,
- const struct cl_req_slice *slice,
- const struct cl_object *obj,
- struct cl_req_attr *oa, u64 flags);
-
-struct lu_device *ccc2lu_dev (struct ccc_device *vdv);
-struct lu_object *ccc2lu (struct ccc_object *vob);
-struct ccc_device *lu2ccc_dev (const struct lu_device *d);
-struct ccc_device *cl2ccc_dev (const struct cl_device *d);
-struct ccc_object *lu2ccc (const struct lu_object *obj);
-struct ccc_object *cl2ccc (const struct cl_object *obj);
-struct ccc_lock *cl2ccc_lock (const struct cl_lock_slice *slice);
-struct ccc_io *cl2ccc_io (const struct lu_env *env,
- const struct cl_io_slice *slice);
-struct ccc_req *cl2ccc_req (const struct cl_req_slice *slice);
-struct page *cl2vm_page (const struct cl_page_slice *slice);
-struct inode *ccc_object_inode(const struct cl_object *obj);
-struct ccc_object *cl_inode2ccc (struct inode *inode);
-
-int cl_setattr_ost(struct inode *inode, const struct iattr *attr);
-
-int ccc_object_invariant(const struct cl_object *obj);
-int cl_file_inode_init(struct inode *inode, struct lustre_md *md);
-void cl_inode_fini(struct inode *inode);
-int cl_local_size(struct inode *inode);
-
-__u16 ll_dirent_type_get(struct lu_dirent *ent);
-__u64 cl_fid_build_ino(const struct lu_fid *fid, int api32);
-__u32 cl_fid_build_gen(const struct lu_fid *fid);
-
-# define CLOBINVRNT(env, clob, expr) \
- ((void)sizeof(env), (void)sizeof(clob), (void)sizeof(!!(expr)))
-
-int cl_init_ea_size(struct obd_export *md_exp, struct obd_export *dt_exp);
-int cl_ocd_update(struct obd_device *host,
- struct obd_device *watched,
- enum obd_notify_event ev, void *owner, void *data);
-
-struct ccc_grouplock {
- struct lu_env *cg_env;
- struct cl_io *cg_io;
- struct cl_lock *cg_lock;
- unsigned long cg_gid;
-};
-
-int cl_get_grouplock(struct cl_object *obj, unsigned long gid, int nonblock,
- struct ccc_grouplock *cg);
-void cl_put_grouplock(struct ccc_grouplock *cg);
-
-/**
- * New interfaces to get and put lov_stripe_md from lov layer. This violates
- * layering because lov_stripe_md is supposed to be a private data in lov.
- *
- * NB: If you find you have to use these interfaces for your new code, please
- * think about it again. These interfaces may be removed in the future for
- * better layering.
- */
-struct lov_stripe_md *lov_lsm_get(struct cl_object *clobj);
-void lov_lsm_put(struct cl_object *clobj, struct lov_stripe_md *lsm);
-int lov_read_and_clear_async_rc(struct cl_object *clob);
-
-struct lov_stripe_md *ccc_inode_lsm_get(struct inode *inode);
-void ccc_inode_lsm_put(struct inode *inode, struct lov_stripe_md *lsm);
-
-/**
- * Data structure managing a client's cached clean pages. An LRU of
- * pages is maintained, along with other statistics.
- */
-struct cl_client_cache {
- atomic_t ccc_users; /* # of users (OSCs) of this data */
- struct list_head ccc_lru; /* LRU list of cached clean pages */
- spinlock_t ccc_lru_lock; /* lock for list */
- atomic_t ccc_lru_left; /* # of LRU entries available */
- unsigned long ccc_lru_max; /* Max # of LRU entries possible */
- unsigned int ccc_lru_shrinkers; /* # of threads reclaiming */
-};
-
-#endif /*LCLIENT_H */
diff --git a/drivers/staging/lustre/lustre/include/linux/obd.h b/drivers/staging/lustre/lustre/include/linux/obd.h
deleted file mode 100644
index 3907bf4..0000000
--- a/drivers/staging/lustre/lustre/include/linux/obd.h
+++ /dev/null
@@ -1,125 +0,0 @@
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.sun.com/software/products/lustre/docs/GPLv2.pdf
- *
- * Please contact Sun Microsystems, Inc., 4150 Network Circle, Santa Clara,
- * CA 95054 USA or visit www.sun.com if you need additional information or
- * have any questions.
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 1999, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2011, 2012, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- */
-
-#ifndef __LINUX_OBD_H
-#define __LINUX_OBD_H
-
-#ifndef __OBD_H
-#error Do not #include this file directly. #include <obd.h> instead
-#endif
-
-#include "../obd_support.h"
-
-#include <linux/fs.h>
-#include <linux/list.h>
-#include <linux/sched.h> /* for struct task_struct, for current.h */
-#include <linux/mount.h>
-
-#include "../lustre_intent.h"
-
-struct ll_iattr {
- struct iattr iattr;
- unsigned int ia_attr_flags;
-};
-
-#define CLIENT_OBD_LIST_LOCK_DEBUG 1
-
-struct client_obd_lock {
- spinlock_t lock;
-
- unsigned long time;
- struct task_struct *task;
- const char *func;
- int line;
-};
-
-static inline void __client_obd_list_lock(struct client_obd_lock *lock,
- const char *func, int line)
-{
- unsigned long cur = jiffies;
-
- while (1) {
- if (spin_trylock(&lock->lock)) {
- LASSERT(!lock->task);
- lock->task = current;
- lock->func = func;
- lock->line = line;
- lock->time = jiffies;
- break;
- }
-
- if (time_before(cur + 5 * HZ, jiffies) &&
- time_before(lock->time + 5 * HZ, jiffies)) {
- struct task_struct *task = lock->task;
-
- if (!task)
- continue;
-
- LCONSOLE_WARN("%s:%d: lock %p was acquired by <%s:%d:%s:%d> for %lu seconds.\n",
- current->comm, current->pid,
- lock, task->comm, task->pid,
- lock->func, lock->line,
- (jiffies - lock->time) / HZ);
- LCONSOLE_WARN("====== for current process =====\n");
- dump_stack();
- LCONSOLE_WARN("====== end =======\n");
- set_current_state(TASK_UNINTERRUPTIBLE);
- schedule_timeout(1000 * HZ);
- }
- cpu_relax();
- }
-}
-
-#define client_obd_list_lock(lock) \
- __client_obd_list_lock(lock, __func__, __LINE__)
-
-static inline void client_obd_list_unlock(struct client_obd_lock *lock)
-{
- LASSERT(lock->task);
- lock->task = NULL;
- lock->time = jiffies;
- spin_unlock(&lock->lock);
-}
-
-static inline void client_obd_list_lock_init(struct client_obd_lock *lock)
-{
- spin_lock_init(&lock->lock);
-}
-
-static inline void client_obd_list_lock_done(struct client_obd_lock *lock)
-{}
-
-#endif /* __LINUX_OBD_H */
diff --git a/drivers/staging/lustre/lustre/include/lu_object.h b/drivers/staging/lustre/lustre/include/lu_object.h
index 242bb1e..28165121 100644
--- a/drivers/staging/lustre/lustre/include/lu_object.h
+++ b/drivers/staging/lustre/lustre/include/lu_object.h
@@ -198,7 +198,6 @@
* Operations specific for particular lu_object.
*/
struct lu_object_operations {
-
/**
* Allocate lower-layer parts of the object by calling
* lu_device_operations::ldo_object_alloc() of the corresponding
@@ -656,21 +655,21 @@
* @{
*/
-int lu_site_init (struct lu_site *s, struct lu_device *d);
-void lu_site_fini (struct lu_site *s);
-int lu_site_init_finish (struct lu_site *s);
-void lu_stack_fini (const struct lu_env *env, struct lu_device *top);
-void lu_device_get (struct lu_device *d);
-void lu_device_put (struct lu_device *d);
-int lu_device_init (struct lu_device *d, struct lu_device_type *t);
-void lu_device_fini (struct lu_device *d);
-int lu_object_header_init(struct lu_object_header *h);
+int lu_site_init(struct lu_site *s, struct lu_device *d);
+void lu_site_fini(struct lu_site *s);
+int lu_site_init_finish(struct lu_site *s);
+void lu_stack_fini(const struct lu_env *env, struct lu_device *top);
+void lu_device_get(struct lu_device *d);
+void lu_device_put(struct lu_device *d);
+int lu_device_init(struct lu_device *d, struct lu_device_type *t);
+void lu_device_fini(struct lu_device *d);
+int lu_object_header_init(struct lu_object_header *h);
void lu_object_header_fini(struct lu_object_header *h);
-int lu_object_init (struct lu_object *o,
- struct lu_object_header *h, struct lu_device *d);
-void lu_object_fini (struct lu_object *o);
-void lu_object_add_top (struct lu_object_header *h, struct lu_object *o);
-void lu_object_add (struct lu_object *before, struct lu_object *o);
+int lu_object_init(struct lu_object *o,
+ struct lu_object_header *h, struct lu_device *d);
+void lu_object_fini(struct lu_object *o);
+void lu_object_add_top(struct lu_object_header *h, struct lu_object *o);
+void lu_object_add(struct lu_object *before, struct lu_object *o);
/**
* Helpers to initialize and finalize device types.
@@ -781,9 +780,8 @@
*/
#define LU_OBJECT_DEBUG(mask, env, object, format, ...) \
do { \
- LIBCFS_DEBUG_MSG_DATA_DECL(msgdata, mask, NULL); \
- \
if (cfs_cdebug_show(mask, DEBUG_SUBSYSTEM)) { \
+ LIBCFS_DEBUG_MSG_DATA_DECL(msgdata, mask, NULL); \
lu_object_print(env, &msgdata, lu_cdebug_printer, object);\
CDEBUG(mask, format, ## __VA_ARGS__); \
} \
@@ -794,9 +792,8 @@
*/
#define LU_OBJECT_HEADER(mask, env, object, format, ...) \
do { \
- LIBCFS_DEBUG_MSG_DATA_DECL(msgdata, mask, NULL); \
- \
if (cfs_cdebug_show(mask, DEBUG_SUBSYSTEM)) { \
+ LIBCFS_DEBUG_MSG_DATA_DECL(msgdata, mask, NULL); \
lu_object_header_print(env, &msgdata, lu_cdebug_printer,\
(object)->lo_header); \
lu_cdebug_printer(env, &msgdata, "\n"); \
@@ -1007,6 +1004,10 @@
*/
LCT_LOCAL = 1 << 7,
/**
+ * session for server thread
+ **/
+ LCT_SERVER_SESSION = BIT(8),
+ /**
* Set when at least one of keys, having values in this context has
* non-NULL lu_context_key::lct_exit() method. This is used to
* optimize lu_context_exit() call.
@@ -1118,7 +1119,7 @@
{ \
type *value; \
\
- CLASSERT(PAGE_SIZE >= sizeof (*value)); \
+ CLASSERT(PAGE_SIZE >= sizeof(*value)); \
\
value = kzalloc(sizeof(*value), GFP_NOFS); \
if (!value) \
@@ -1154,12 +1155,12 @@
(key)->lct_owner = THIS_MODULE; \
} while (0)
-int lu_context_key_register(struct lu_context_key *key);
-void lu_context_key_degister(struct lu_context_key *key);
-void *lu_context_key_get (const struct lu_context *ctx,
- const struct lu_context_key *key);
-void lu_context_key_quiesce (struct lu_context_key *key);
-void lu_context_key_revive (struct lu_context_key *key);
+int lu_context_key_register(struct lu_context_key *key);
+void lu_context_key_degister(struct lu_context_key *key);
+void *lu_context_key_get(const struct lu_context *ctx,
+ const struct lu_context_key *key);
+void lu_context_key_quiesce(struct lu_context_key *key);
+void lu_context_key_revive(struct lu_context_key *key);
/*
* LU_KEY_INIT_GENERIC() has to be a macro to correctly determine an
@@ -1216,21 +1217,21 @@
LU_TYPE_START(mod, __VA_ARGS__); \
LU_TYPE_STOP(mod, __VA_ARGS__)
-int lu_context_init (struct lu_context *ctx, __u32 tags);
-void lu_context_fini (struct lu_context *ctx);
-void lu_context_enter (struct lu_context *ctx);
-void lu_context_exit (struct lu_context *ctx);
-int lu_context_refill(struct lu_context *ctx);
+int lu_context_init(struct lu_context *ctx, __u32 tags);
+void lu_context_fini(struct lu_context *ctx);
+void lu_context_enter(struct lu_context *ctx);
+void lu_context_exit(struct lu_context *ctx);
+int lu_context_refill(struct lu_context *ctx);
/*
* Helper functions to operate on multiple keys. These are used by the default
* device type operations, defined by LU_TYPE_INIT_FINI().
*/
-int lu_context_key_register_many(struct lu_context_key *k, ...);
+int lu_context_key_register_many(struct lu_context_key *k, ...);
void lu_context_key_degister_many(struct lu_context_key *k, ...);
-void lu_context_key_revive_many (struct lu_context_key *k, ...);
-void lu_context_key_quiesce_many (struct lu_context_key *k, ...);
+void lu_context_key_revive_many(struct lu_context_key *k, ...);
+void lu_context_key_quiesce_many(struct lu_context_key *k, ...);
/**
* Environment.
@@ -1246,9 +1247,9 @@
struct lu_context *le_ses;
};
-int lu_env_init (struct lu_env *env, __u32 tags);
-void lu_env_fini (struct lu_env *env);
-int lu_env_refill(struct lu_env *env);
+int lu_env_init(struct lu_env *env, __u32 tags);
+void lu_env_fini(struct lu_env *env);
+int lu_env_refill(struct lu_env *env);
/** @} lu_context */
diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h
index 5aae1d0..9c53c17 100644
--- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h
+++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h
@@ -183,6 +183,12 @@
__u32 lsr_flags;
};
+struct lu_seq_range_array {
+ __u32 lsra_count;
+ __u32 lsra_padding;
+ struct lu_seq_range lsra_lsr[0];
+};
+
#define LU_SEQ_RANGE_MDT 0x0
#define LU_SEQ_RANGE_OST 0x1
#define LU_SEQ_RANGE_ANY 0x3
@@ -578,7 +584,7 @@
if (fid_seq_is_mdt0(ostid->oi.oi_seq))
return FID_SEQ_OST_MDT0;
- if (fid_seq_is_default(ostid->oi.oi_seq))
+ if (unlikely(fid_seq_is_default(ostid->oi.oi_seq)))
return FID_SEQ_LOV_DEFAULT;
if (fid_is_idif(&ostid->oi_fid))
@@ -590,9 +596,12 @@
/* extract OST objid from a wire ost_id (id/seq) pair */
static inline __u64 ostid_id(const struct ost_id *ostid)
{
- if (fid_seq_is_mdt0(ostid_seq(ostid)))
+ if (fid_seq_is_mdt0(ostid->oi.oi_seq))
return ostid->oi.oi_id & IDIF_OID_MASK;
+ if (unlikely(fid_seq_is_default(ostid->oi.oi_seq)))
+ return ostid->oi.oi_id;
+
if (fid_is_idif(&ostid->oi_fid))
return fid_idif_id(fid_seq(&ostid->oi_fid),
fid_oid(&ostid->oi_fid), 0);
@@ -636,12 +645,22 @@
*/
static inline void ostid_set_id(struct ost_id *oi, __u64 oid)
{
- if (fid_seq_is_mdt0(ostid_seq(oi))) {
+ if (fid_seq_is_mdt0(oi->oi.oi_seq)) {
if (oid >= IDIF_MAX_OID) {
CERROR("Bad %llu to set " DOSTID "\n", oid, POSTID(oi));
return;
}
oi->oi.oi_id = oid;
+ } else if (fid_is_idif(&oi->oi_fid)) {
+ if (oid >= IDIF_MAX_OID) {
+ CERROR("Bad %llu to set "DOSTID"\n",
+ oid, POSTID(oi));
+ return;
+ }
+ oi->oi_fid.f_seq = fid_idif_seq(oid,
+ fid_idif_ost_idx(&oi->oi_fid));
+ oi->oi_fid.f_oid = oid;
+ oi->oi_fid.f_ver = oid >> 48;
} else {
if (oid > OBIF_MAX_OID) {
CERROR("Bad %llu to set " DOSTID "\n", oid, POSTID(oi));
@@ -651,25 +670,31 @@
}
}
-static inline void ostid_inc_id(struct ost_id *oi)
+static inline int fid_set_id(struct lu_fid *fid, __u64 oid)
{
- if (fid_seq_is_mdt0(ostid_seq(oi))) {
- if (unlikely(ostid_id(oi) + 1 > IDIF_MAX_OID)) {
- CERROR("Bad inc "DOSTID"\n", POSTID(oi));
- return;
- }
- oi->oi.oi_id++;
- } else {
- oi->oi_fid.f_oid++;
+ if (unlikely(fid_seq_is_igif(fid->f_seq))) {
+ CERROR("bad IGIF, "DFID"\n", PFID(fid));
+ return -EBADF;
}
-}
-static inline void ostid_dec_id(struct ost_id *oi)
-{
- if (fid_seq_is_mdt0(ostid_seq(oi)))
- oi->oi.oi_id--;
- else
- oi->oi_fid.f_oid--;
+ if (fid_is_idif(fid)) {
+ if (oid >= IDIF_MAX_OID) {
+ CERROR("Too large OID %#llx to set IDIF "DFID"\n",
+ (unsigned long long)oid, PFID(fid));
+ return -EBADF;
+ }
+ fid->f_seq = fid_idif_seq(oid, fid_idif_ost_idx(fid));
+ fid->f_oid = oid;
+ fid->f_ver = oid >> 48;
+ } else {
+ if (oid > OBIF_MAX_OID) {
+ CERROR("Too large OID %#llx to set REG "DFID"\n",
+ (unsigned long long)oid, PFID(fid));
+ return -EBADF;
+ }
+ fid->f_oid = oid;
+ }
+ return 0;
}
/**
@@ -684,30 +709,34 @@
static inline int ostid_to_fid(struct lu_fid *fid, struct ost_id *ostid,
__u32 ost_idx)
{
+ __u64 seq = ostid_seq(ostid);
+
if (ost_idx > 0xffff) {
CERROR("bad ost_idx, "DOSTID" ost_idx:%u\n", POSTID(ostid),
ost_idx);
return -EBADF;
}
- if (fid_seq_is_mdt0(ostid_seq(ostid))) {
+ if (fid_seq_is_mdt0(seq)) {
+ __u64 oid = ostid_id(ostid);
+
/* This is a "legacy" (old 1.x/2.early) OST object in "group 0"
* that we map into the IDIF namespace. It allows up to 2^48
* objects per OST, as this is the object namespace that has
* been in production for years. This can handle create rates
* of 1M objects/s/OST for 9 years, or combinations thereof.
*/
- if (ostid_id(ostid) >= IDIF_MAX_OID) {
+ if (oid >= IDIF_MAX_OID) {
CERROR("bad MDT0 id, " DOSTID " ost_idx:%u\n",
POSTID(ostid), ost_idx);
return -EBADF;
}
- fid->f_seq = fid_idif_seq(ostid_id(ostid), ost_idx);
+ fid->f_seq = fid_idif_seq(oid, ost_idx);
/* truncate to 32 bits by assignment */
- fid->f_oid = ostid_id(ostid);
+ fid->f_oid = oid;
/* in theory, not currently used */
- fid->f_ver = ostid_id(ostid) >> 48;
- } else /* if (fid_seq_is_idif(seq) || fid_seq_is_norm(seq)) */ {
+ fid->f_ver = oid >> 48;
+ } else if (likely(!fid_seq_is_default(seq))) {
/* This is either an IDIF object, which identifies objects across
* all OSTs, or a regular FID. The IDIF namespace maps legacy
* OST objects into the FID namespace. In both cases, we just
@@ -1001,8 +1030,9 @@
size = (sizeof(struct lu_dirent) + namelen + align) & ~align;
size += sizeof(struct luda_type);
- } else
+ } else {
size = sizeof(struct lu_dirent) + namelen;
+ }
return (size + 7) & ~7;
}
@@ -1256,6 +1286,9 @@
#define OBD_CONNECT_PINGLESS 0x4000000000000ULL/* pings not required */
#define OBD_CONNECT_FLOCK_DEAD 0x8000000000000ULL/* flock deadlock detection */
#define OBD_CONNECT_DISP_STRIPE 0x10000000000000ULL/*create stripe disposition*/
+#define OBD_CONNECT_OPEN_BY_FID 0x20000000000000ULL /* open by fid won't pack
+ * name in request
+ */
/* XXX README XXX:
* Please DO NOT add flag values here before first ensuring that this same
@@ -1428,6 +1461,8 @@
*/
OBD_FL_RECOV_RESEND = 0x00080000, /* recoverable resent */
OBD_FL_NOSPC_BLK = 0x00100000, /* no more block space on OST */
+ OBD_FL_FLUSH = 0x00200000, /* flush pages on the OST */
+ OBD_FL_SHORT_IO = 0x00400000, /* short io request */
/* Note that while these checksum values are currently separate bits,
* in 2.x we can actually allow all values from 1-31 if we wanted.
@@ -1525,6 +1560,11 @@
oi->oi.oi_seq = seq;
}
+static inline void lmm_oi_set_id(struct ost_id *oi, __u64 oid)
+{
+ oi->oi.oi_id = oid;
+}
+
static inline __u64 lmm_oi_id(struct ost_id *oi)
{
return oi->oi.oi_id;
@@ -1732,6 +1772,11 @@
#define OBD_BRW_MEMALLOC 0x800 /* Client runs in the "kswapd" context */
#define OBD_BRW_OVER_USRQUOTA 0x1000 /* Running out of user quota */
#define OBD_BRW_OVER_GRPQUOTA 0x2000 /* Running out of group quota */
+#define OBD_BRW_SOFT_SYNC 0x4000 /* This flag notifies the server
+ * that the client is running low on
+ * space for unstable pages; asking
+ * it to sync quickly
+ */
#define OBD_OBJECT_EOF 0xffffffffffffffffULL
@@ -2436,6 +2481,7 @@
void lustre_swab_mdt_rec_reint(struct mdt_rec_reint *rr);
+/* lmv structures */
struct lmv_desc {
__u32 ld_tgt_count; /* how many MDS's */
__u32 ld_active_tgt_count; /* how many active */
@@ -2460,7 +2506,6 @@
struct lu_fid mea_ids[0];
};
-/* lmv structures */
#define MEA_MAGIC_LAST_CHAR 0xb2221ca1
#define MEA_MAGIC_ALL_CHARS 0xb222a11c
#define MEA_MAGIC_HASH_SEGMENT 0xb222a11b
@@ -2470,9 +2515,10 @@
#define MAX_HASH_HIGHEST_BIT 0x1000000000000000ULL
enum fld_rpc_opc {
- FLD_QUERY = 900,
+ FLD_QUERY = 900,
+ FLD_READ = 901,
FLD_LAST_OPC,
- FLD_FIRST_OPC = FLD_QUERY
+ FLD_FIRST_OPC = FLD_QUERY
};
enum seq_rpc_opc {
@@ -2486,6 +2532,12 @@
SEQ_ALLOC_META = 1
};
+enum fld_op {
+ FLD_CREATE = 0,
+ FLD_DELETE = 1,
+ FLD_LOOKUP = 2,
+};
+
/*
* LOV data structures
*/
@@ -2582,6 +2634,8 @@
__u64 gid;
};
+#define LDLM_GID_ANY ((__u64)-1)
+
static inline int ldlm_extent_overlap(struct ldlm_extent *ex1,
struct ldlm_extent *ex2)
{
@@ -3304,7 +3358,7 @@
char gf_path[0];
} __packed;
-void lustre_swab_fid2path (struct getinfo_fid2path *gf);
+void lustre_swab_fid2path(struct getinfo_fid2path *gf);
enum {
LAYOUT_INTENT_ACCESS = 0,
diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_user.h b/drivers/staging/lustre/lustre/include/lustre/lustre_user.h
index 276906e..59ba48a 100644
--- a/drivers/staging/lustre/lustre/include/lustre/lustre_user.h
+++ b/drivers/staging/lustre/lustre/include/lustre/lustre_user.h
@@ -193,37 +193,37 @@
* *INFO - set/get lov_user_mds_data
*/
/* see <lustre_lib.h> for ioctl numberss 101-150 */
-#define LL_IOC_GETFLAGS _IOR ('f', 151, long)
-#define LL_IOC_SETFLAGS _IOW ('f', 152, long)
-#define LL_IOC_CLRFLAGS _IOW ('f', 153, long)
+#define LL_IOC_GETFLAGS _IOR('f', 151, long)
+#define LL_IOC_SETFLAGS _IOW('f', 152, long)
+#define LL_IOC_CLRFLAGS _IOW('f', 153, long)
/* LL_IOC_LOV_SETSTRIPE: See also OBD_IOC_LOV_SETSTRIPE */
-#define LL_IOC_LOV_SETSTRIPE _IOW ('f', 154, long)
+#define LL_IOC_LOV_SETSTRIPE _IOW('f', 154, long)
/* LL_IOC_LOV_GETSTRIPE: See also OBD_IOC_LOV_GETSTRIPE */
-#define LL_IOC_LOV_GETSTRIPE _IOW ('f', 155, long)
+#define LL_IOC_LOV_GETSTRIPE _IOW('f', 155, long)
/* LL_IOC_LOV_SETEA: See also OBD_IOC_LOV_SETEA */
-#define LL_IOC_LOV_SETEA _IOW ('f', 156, long)
-#define LL_IOC_RECREATE_OBJ _IOW ('f', 157, long)
-#define LL_IOC_RECREATE_FID _IOW ('f', 157, struct lu_fid)
-#define LL_IOC_GROUP_LOCK _IOW ('f', 158, long)
-#define LL_IOC_GROUP_UNLOCK _IOW ('f', 159, long)
+#define LL_IOC_LOV_SETEA _IOW('f', 156, long)
+#define LL_IOC_RECREATE_OBJ _IOW('f', 157, long)
+#define LL_IOC_RECREATE_FID _IOW('f', 157, struct lu_fid)
+#define LL_IOC_GROUP_LOCK _IOW('f', 158, long)
+#define LL_IOC_GROUP_UNLOCK _IOW('f', 159, long)
/* LL_IOC_QUOTACHECK: See also OBD_IOC_QUOTACHECK */
-#define LL_IOC_QUOTACHECK _IOW ('f', 160, int)
+#define LL_IOC_QUOTACHECK _IOW('f', 160, int)
/* LL_IOC_POLL_QUOTACHECK: See also OBD_IOC_POLL_QUOTACHECK */
-#define LL_IOC_POLL_QUOTACHECK _IOR ('f', 161, struct if_quotacheck *)
+#define LL_IOC_POLL_QUOTACHECK _IOR('f', 161, struct if_quotacheck *)
/* LL_IOC_QUOTACTL: See also OBD_IOC_QUOTACTL */
#define LL_IOC_QUOTACTL _IOWR('f', 162, struct if_quotactl)
#define IOC_OBD_STATFS _IOWR('f', 164, struct obd_statfs *)
#define IOC_LOV_GETINFO _IOWR('f', 165, struct lov_user_mds_data *)
-#define LL_IOC_FLUSHCTX _IOW ('f', 166, long)
-#define LL_IOC_RMTACL _IOW ('f', 167, long)
-#define LL_IOC_GETOBDCOUNT _IOR ('f', 168, long)
+#define LL_IOC_FLUSHCTX _IOW('f', 166, long)
+#define LL_IOC_RMTACL _IOW('f', 167, long)
+#define LL_IOC_GETOBDCOUNT _IOR('f', 168, long)
#define LL_IOC_LLOOP_ATTACH _IOWR('f', 169, long)
#define LL_IOC_LLOOP_DETACH _IOWR('f', 170, long)
#define LL_IOC_LLOOP_INFO _IOWR('f', 171, struct lu_fid)
#define LL_IOC_LLOOP_DETACH_BYDEV _IOWR('f', 172, long)
-#define LL_IOC_PATH2FID _IOR ('f', 173, long)
+#define LL_IOC_PATH2FID _IOR('f', 173, long)
#define LL_IOC_GET_CONNECT_FLAGS _IOWR('f', 174, __u64 *)
-#define LL_IOC_GET_MDTIDX _IOR ('f', 175, int)
+#define LL_IOC_GET_MDTIDX _IOR('f', 175, int)
/* see <lustre_lib.h> for ioctl numbers 177-210 */
@@ -676,7 +676,12 @@
#define CLF_UNLINK_HSM_EXISTS 0x0002 /* File has something in HSM */
/* HSM cleaning needed */
/* Flags for rename */
-#define CLF_RENAME_LAST 0x0001 /* rename unlink last hardlink of target */
+#define CLF_RENAME_LAST 0x0001 /* rename unlink last hardlink of
+ * target
+ */
+#define CLF_RENAME_LAST_EXISTS 0x0002 /* rename unlink last hardlink of target
+ * has an archive in backend
+ */
/* Flags for HSM */
/* 12b used (from high weight to low weight):
@@ -833,9 +838,8 @@
__u64 idv_flags; /* See LL_DV_xxx */
};
-#define LL_DV_NOFLUSH 0x01 /* Do not take READ EXTENT LOCK before sampling
- * version. Dirty caches are left unchanged.
- */
+#define LL_DV_RD_FLUSH BIT(0) /* Flush dirty pages from clients */
+#define LL_DV_WR_FLUSH BIT(1) /* Flush all caching pages from clients */
#ifndef offsetof
# define offsetof(typ, memb) ((unsigned long)((char *)&(((typ *)0)->memb)))
@@ -1095,12 +1099,12 @@
__u32 padding1;
char hal_fsname[0]; /* null-terminated */
/* struct hsm_action_item[hal_count] follows, aligned on 8-byte
- * boundaries. See hai_zero
+ * boundaries. See hai_first
*/
} __packed;
#ifndef HAVE_CFS_SIZE_ROUND
-static inline int cfs_size_round (int val)
+static inline int cfs_size_round(int val)
{
return (val + 7) & (~0x7);
}
@@ -1109,7 +1113,7 @@
#endif
/* Return pointer to first hai in action list */
-static inline struct hsm_action_item *hai_zero(struct hsm_action_list *hal)
+static inline struct hsm_action_item *hai_first(struct hsm_action_list *hal)
{
return (struct hsm_action_item *)(hal->hal_fsname +
cfs_size_round(strlen(hal-> \
@@ -1131,7 +1135,7 @@
struct hsm_action_item *hai;
sz = sizeof(*hal) + cfs_size_round(strlen(hal->hal_fsname) + 1);
- hai = hai_zero(hal);
+ hai = hai_first(hal);
for (i = 0; i < hal->hal_count; i++, hai = hai_next(hai))
sz += cfs_size_round(hai->hai_len);
diff --git a/drivers/staging/lustre/lustre/include/lustre_cfg.h b/drivers/staging/lustre/lustre/include/lustre_cfg.h
index bb16ae9..e229e91 100644
--- a/drivers/staging/lustre/lustre/include/lustre_cfg.h
+++ b/drivers/staging/lustre/lustre/include/lustre_cfg.h
@@ -161,7 +161,7 @@
int offset;
int bufcount;
- LASSERT (index >= 0);
+ LASSERT(index >= 0);
bufcount = lcfg->lcfg_bufcount;
if (index >= bufcount)
diff --git a/drivers/staging/lustre/lustre/include/lustre_disk.h b/drivers/staging/lustre/lustre/include/lustre_disk.h
index 95fd360..b36821f 100644
--- a/drivers/staging/lustre/lustre/include/lustre_disk.h
+++ b/drivers/staging/lustre/lustre/include/lustre_disk.h
@@ -130,7 +130,6 @@
struct lustre_mount_data *lsi_lmd; /* mount command info */
struct ll_sb_info *lsi_llsbi; /* add'l client sbi info */
struct dt_device *lsi_dt_dev; /* dt device to access disk fs*/
- struct vfsmount *lsi_srv_mnt; /* the one server mount */
atomic_t lsi_mounts; /* references to the srv_mnt */
char lsi_svname[MTI_NAME_MAXLEN];
char lsi_osd_obdname[64];
@@ -158,7 +157,6 @@
struct lustre_mount_info {
char *lmi_name;
struct super_block *lmi_sb;
- struct vfsmount *lmi_mnt;
struct list_head lmi_list_chain;
};
diff --git a/drivers/staging/lustre/lustre/include/lustre_dlm.h b/drivers/staging/lustre/lustre/include/lustre_dlm.h
index 8b0364f..9cade14 100644
--- a/drivers/staging/lustre/lustre/include/lustre_dlm.h
+++ b/drivers/staging/lustre/lustre/include/lustre_dlm.h
@@ -71,6 +71,7 @@
*/
enum ldlm_error {
ELDLM_OK = 0,
+ ELDLM_LOCK_MATCHED = 1,
ELDLM_LOCK_CHANGED = 300,
ELDLM_LOCK_ABORTED = 301,
@@ -269,7 +270,7 @@
struct completion pl_kobj_unregister;
};
-typedef int (*ldlm_cancel_for_recovery)(struct ldlm_lock *lock);
+typedef int (*ldlm_cancel_cbt)(struct ldlm_lock *lock);
/**
* LVB operations.
@@ -446,8 +447,11 @@
/** Limit of parallel AST RPC count. */
unsigned ns_max_parallel_ast;
- /** Callback to cancel locks before replaying it during recovery. */
- ldlm_cancel_for_recovery ns_cancel_for_recovery;
+ /**
+ * Callback to check if a lock is good to be canceled by ELC or
+ * during recovery.
+ */
+ ldlm_cancel_cbt ns_cancel;
/** LDLM lock stats */
struct lprocfs_stats *ns_stats;
@@ -479,9 +483,9 @@
}
static inline void ns_register_cancel(struct ldlm_namespace *ns,
- ldlm_cancel_for_recovery arg)
+ ldlm_cancel_cbt arg)
{
- ns->ns_cancel_for_recovery = arg;
+ ns->ns_cancel = arg;
}
struct ldlm_lock;
diff --git a/drivers/staging/lustre/lustre/include/lustre_dlm_flags.h b/drivers/staging/lustre/lustre/include/lustre_dlm_flags.h
index 7f2ba2f..e7e0c21 100644
--- a/drivers/staging/lustre/lustre/include/lustre_dlm_flags.h
+++ b/drivers/staging/lustre/lustre/include/lustre_dlm_flags.h
@@ -37,17 +37,11 @@
/** l_flags bits marked as "gone" bits */
#define LDLM_FL_GONE_MASK 0x0006004000000000ULL
-/** l_flags bits marked as "hide_lock" bits */
-#define LDLM_FL_HIDE_LOCK_MASK 0x0000206400000000ULL
-
/** l_flags bits marked as "inherit" bits */
#define LDLM_FL_INHERIT_MASK 0x0000000000800000ULL
-/** l_flags bits marked as "local_only" bits */
-#define LDLM_FL_LOCAL_ONLY_MASK 0x00FFFFFF00000000ULL
-
-/** l_flags bits marked as "on_wire" bits */
-#define LDLM_FL_ON_WIRE_MASK 0x00000000C08F932FULL
+/** l_flags bits marked as "off_wire" bits */
+#define LDLM_FL_OFF_WIRE_MASK 0x00FFFFFF00000000ULL
/** extent, mode, or resource changed */
#define LDLM_FL_LOCK_CHANGED 0x0000000000000001ULL /* bit 0 */
@@ -204,7 +198,7 @@
#define ldlm_set_cancel(_l) LDLM_SET_FLAG((_l), 1ULL << 36)
#define ldlm_clear_cancel(_l) LDLM_CLEAR_FLAG((_l), 1ULL << 36)
-/** whatever it might mean */
+/** whatever it might mean -- never transmitted? */
#define LDLM_FL_LOCAL_ONLY 0x0000002000000000ULL /* bit 37 */
#define ldlm_is_local_only(_l) LDLM_TEST_FLAG((_l), 1ULL << 37)
#define ldlm_set_local_only(_l) LDLM_SET_FLAG((_l), 1ULL << 37)
@@ -287,18 +281,18 @@
* has canceled this lock and is waiting for rpc_lock which is taken by
* the first operation. LDLM_FL_BL_AST is set by ldlm_callback_handler() in
* the lock to prevent the Early Lock Cancel (ELC) code from cancelling it.
- *
- * LDLM_FL_BL_DONE is to be set by ldlm_cancel_callback() when lock cache is
- * dropped to let ldlm_callback_handler() return EINVAL to the server. It
- * is used when ELC RPC is already prepared and is waiting for rpc_lock,
- * too late to send a separate CANCEL RPC.
*/
#define LDLM_FL_BL_AST 0x0000400000000000ULL /* bit 46 */
#define ldlm_is_bl_ast(_l) LDLM_TEST_FLAG((_l), 1ULL << 46)
#define ldlm_set_bl_ast(_l) LDLM_SET_FLAG((_l), 1ULL << 46)
#define ldlm_clear_bl_ast(_l) LDLM_CLEAR_FLAG((_l), 1ULL << 46)
-/** whatever it might mean */
+/**
+ * Set by ldlm_cancel_callback() when lock cache is dropped to let
+ * ldlm_callback_handler() return EINVAL to the server. It is used when
+ * ELC RPC is already prepared and is waiting for rpc_lock, too late to
+ * send a separate CANCEL RPC.
+ */
#define LDLM_FL_BL_DONE 0x0000800000000000ULL /* bit 47 */
#define ldlm_is_bl_done(_l) LDLM_TEST_FLAG((_l), 1ULL << 47)
#define ldlm_set_bl_done(_l) LDLM_SET_FLAG((_l), 1ULL << 47)
@@ -381,104 +375,16 @@
/** test for ldlm_lock flag bit set */
#define LDLM_TEST_FLAG(_l, _b) (((_l)->l_flags & (_b)) != 0)
+/** multi-bit test: are any of mask bits set? */
+#define LDLM_HAVE_MASK(_l, _m) ((_l)->l_flags & LDLM_FL_##_m##_MASK)
+
/** set a ldlm_lock flag bit */
#define LDLM_SET_FLAG(_l, _b) ((_l)->l_flags |= (_b))
/** clear a ldlm_lock flag bit */
#define LDLM_CLEAR_FLAG(_l, _b) ((_l)->l_flags &= ~(_b))
-/** Mask of flags inherited from parent lock when doing intents. */
-#define LDLM_INHERIT_FLAGS LDLM_FL_INHERIT_MASK
-
-/** Mask of Flags sent in AST lock_flags to map into the receiving lock. */
-#define LDLM_AST_FLAGS LDLM_FL_AST_MASK
-
/** @} subgroup */
/** @} group */
-#ifdef WIRESHARK_COMPILE
-static int hf_lustre_ldlm_fl_lock_changed = -1;
-static int hf_lustre_ldlm_fl_block_granted = -1;
-static int hf_lustre_ldlm_fl_block_conv = -1;
-static int hf_lustre_ldlm_fl_block_wait = -1;
-static int hf_lustre_ldlm_fl_ast_sent = -1;
-static int hf_lustre_ldlm_fl_replay = -1;
-static int hf_lustre_ldlm_fl_intent_only = -1;
-static int hf_lustre_ldlm_fl_has_intent = -1;
-static int hf_lustre_ldlm_fl_flock_deadlock = -1;
-static int hf_lustre_ldlm_fl_discard_data = -1;
-static int hf_lustre_ldlm_fl_no_timeout = -1;
-static int hf_lustre_ldlm_fl_block_nowait = -1;
-static int hf_lustre_ldlm_fl_test_lock = -1;
-static int hf_lustre_ldlm_fl_cancel_on_block = -1;
-static int hf_lustre_ldlm_fl_deny_on_contention = -1;
-static int hf_lustre_ldlm_fl_ast_discard_data = -1;
-static int hf_lustre_ldlm_fl_fail_loc = -1;
-static int hf_lustre_ldlm_fl_skipped = -1;
-static int hf_lustre_ldlm_fl_cbpending = -1;
-static int hf_lustre_ldlm_fl_wait_noreproc = -1;
-static int hf_lustre_ldlm_fl_cancel = -1;
-static int hf_lustre_ldlm_fl_local_only = -1;
-static int hf_lustre_ldlm_fl_failed = -1;
-static int hf_lustre_ldlm_fl_canceling = -1;
-static int hf_lustre_ldlm_fl_local = -1;
-static int hf_lustre_ldlm_fl_lvb_ready = -1;
-static int hf_lustre_ldlm_fl_kms_ignore = -1;
-static int hf_lustre_ldlm_fl_cp_reqd = -1;
-static int hf_lustre_ldlm_fl_cleaned = -1;
-static int hf_lustre_ldlm_fl_atomic_cb = -1;
-static int hf_lustre_ldlm_fl_bl_ast = -1;
-static int hf_lustre_ldlm_fl_bl_done = -1;
-static int hf_lustre_ldlm_fl_no_lru = -1;
-static int hf_lustre_ldlm_fl_fail_notified = -1;
-static int hf_lustre_ldlm_fl_destroyed = -1;
-static int hf_lustre_ldlm_fl_server_lock = -1;
-static int hf_lustre_ldlm_fl_res_locked = -1;
-static int hf_lustre_ldlm_fl_waited = -1;
-static int hf_lustre_ldlm_fl_ns_srv = -1;
-static int hf_lustre_ldlm_fl_excl = -1;
-const value_string lustre_ldlm_flags_vals[] = {
- {LDLM_FL_LOCK_CHANGED, "LDLM_FL_LOCK_CHANGED"},
- {LDLM_FL_BLOCK_GRANTED, "LDLM_FL_BLOCK_GRANTED"},
- {LDLM_FL_BLOCK_CONV, "LDLM_FL_BLOCK_CONV"},
- {LDLM_FL_BLOCK_WAIT, "LDLM_FL_BLOCK_WAIT"},
- {LDLM_FL_AST_SENT, "LDLM_FL_AST_SENT"},
- {LDLM_FL_REPLAY, "LDLM_FL_REPLAY"},
- {LDLM_FL_INTENT_ONLY, "LDLM_FL_INTENT_ONLY"},
- {LDLM_FL_HAS_INTENT, "LDLM_FL_HAS_INTENT"},
- {LDLM_FL_FLOCK_DEADLOCK, "LDLM_FL_FLOCK_DEADLOCK"},
- {LDLM_FL_DISCARD_DATA, "LDLM_FL_DISCARD_DATA"},
- {LDLM_FL_NO_TIMEOUT, "LDLM_FL_NO_TIMEOUT"},
- {LDLM_FL_BLOCK_NOWAIT, "LDLM_FL_BLOCK_NOWAIT"},
- {LDLM_FL_TEST_LOCK, "LDLM_FL_TEST_LOCK"},
- {LDLM_FL_CANCEL_ON_BLOCK, "LDLM_FL_CANCEL_ON_BLOCK"},
- {LDLM_FL_DENY_ON_CONTENTION, "LDLM_FL_DENY_ON_CONTENTION"},
- {LDLM_FL_AST_DISCARD_DATA, "LDLM_FL_AST_DISCARD_DATA"},
- {LDLM_FL_FAIL_LOC, "LDLM_FL_FAIL_LOC"},
- {LDLM_FL_SKIPPED, "LDLM_FL_SKIPPED"},
- {LDLM_FL_CBPENDING, "LDLM_FL_CBPENDING"},
- {LDLM_FL_WAIT_NOREPROC, "LDLM_FL_WAIT_NOREPROC"},
- {LDLM_FL_CANCEL, "LDLM_FL_CANCEL"},
- {LDLM_FL_LOCAL_ONLY, "LDLM_FL_LOCAL_ONLY"},
- {LDLM_FL_FAILED, "LDLM_FL_FAILED"},
- {LDLM_FL_CANCELING, "LDLM_FL_CANCELING"},
- {LDLM_FL_LOCAL, "LDLM_FL_LOCAL"},
- {LDLM_FL_LVB_READY, "LDLM_FL_LVB_READY"},
- {LDLM_FL_KMS_IGNORE, "LDLM_FL_KMS_IGNORE"},
- {LDLM_FL_CP_REQD, "LDLM_FL_CP_REQD"},
- {LDLM_FL_CLEANED, "LDLM_FL_CLEANED"},
- {LDLM_FL_ATOMIC_CB, "LDLM_FL_ATOMIC_CB"},
- {LDLM_FL_BL_AST, "LDLM_FL_BL_AST"},
- {LDLM_FL_BL_DONE, "LDLM_FL_BL_DONE"},
- {LDLM_FL_NO_LRU, "LDLM_FL_NO_LRU"},
- {LDLM_FL_FAIL_NOTIFIED, "LDLM_FL_FAIL_NOTIFIED"},
- {LDLM_FL_DESTROYED, "LDLM_FL_DESTROYED"},
- {LDLM_FL_SERVER_LOCK, "LDLM_FL_SERVER_LOCK"},
- {LDLM_FL_RES_LOCKED, "LDLM_FL_RES_LOCKED"},
- {LDLM_FL_WAITED, "LDLM_FL_WAITED"},
- {LDLM_FL_NS_SRV, "LDLM_FL_NS_SRV"},
- {LDLM_FL_EXCL, "LDLM_FL_EXCL"},
- { 0, NULL }
-};
-#endif /* WIRESHARK_COMPILE */
#endif /* LDLM_ALL_FLAGS_MASK */
diff --git a/drivers/staging/lustre/lustre/include/lustre_fid.h b/drivers/staging/lustre/lustre/include/lustre_fid.h
index ab4a923..12e8b58 100644
--- a/drivers/staging/lustre/lustre/include/lustre_fid.h
+++ b/drivers/staging/lustre/lustre/include/lustre_fid.h
@@ -308,10 +308,10 @@
fid_seq_is_root(seq) || fid_seq_is_dot(seq);
}
-static inline void lu_last_id_fid(struct lu_fid *fid, __u64 seq)
+static inline void lu_last_id_fid(struct lu_fid *fid, __u64 seq, __u32 ost_idx)
{
if (fid_seq_is_mdt0(seq)) {
- fid->f_seq = fid_idif_seq(0, 0);
+ fid->f_seq = fid_idif_seq(0, ost_idx);
} else {
LASSERTF(fid_seq_is_norm(seq) || fid_seq_is_echo(seq) ||
fid_seq_is_idif(seq), "%#llx\n", seq);
@@ -498,19 +498,6 @@
}
}
-static inline void ostid_res_name_to_id(struct ost_id *oi,
- struct ldlm_res_id *name)
-{
- if (fid_seq_is_mdt0(name->name[LUSTRE_RES_ID_SEQ_OFF])) {
- /* old resid */
- ostid_set_seq(oi, name->name[LUSTRE_RES_ID_VER_OID_OFF]);
- ostid_set_id(oi, name->name[LUSTRE_RES_ID_SEQ_OFF]);
- } else {
- /* new resid */
- fid_extract_from_res_name(&oi->oi_fid, name);
- }
-}
-
/**
* Return true if the resource is for the object identified by this id & group.
*/
@@ -546,7 +533,8 @@
}
static inline void ost_fid_from_resid(struct lu_fid *fid,
- const struct ldlm_res_id *name)
+ const struct ldlm_res_id *name,
+ int ost_idx)
{
if (fid_seq_is_mdt0(name->name[LUSTRE_RES_ID_VER_OID_OFF])) {
/* old resid */
@@ -554,7 +542,7 @@
ostid_set_seq(&oi, name->name[LUSTRE_RES_ID_VER_OID_OFF]);
ostid_set_id(&oi, name->name[LUSTRE_RES_ID_SEQ_OFF]);
- ostid_to_fid(fid, &oi, 0);
+ ostid_to_fid(fid, &oi, ost_idx);
} else {
/* new resid */
fid_extract_from_res_name(fid, name);
diff --git a/drivers/staging/lustre/lustre/include/lustre_import.h b/drivers/staging/lustre/lustre/include/lustre_import.h
index dac2d84..8325c82 100644
--- a/drivers/staging/lustre/lustre/include/lustre_import.h
+++ b/drivers/staging/lustre/lustre/include/lustre_import.h
@@ -109,7 +109,7 @@
"RECOVER", "FULL", "EVICTED",
};
- LASSERT (state <= LUSTRE_IMP_EVICTED);
+ LASSERT(state <= LUSTRE_IMP_EVICTED);
return import_state_names[state];
}
diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h b/drivers/staging/lustre/lustre/include/lustre_lib.h
index f2223d5..00b9767 100644
--- a/drivers/staging/lustre/lustre/include/lustre_lib.h
+++ b/drivers/staging/lustre/lustre/include/lustre_lib.h
@@ -280,16 +280,16 @@
#define OBD_IOC_DATA_TYPE long
#define OBD_IOC_CREATE _IOWR('f', 101, OBD_IOC_DATA_TYPE)
-#define OBD_IOC_DESTROY _IOW ('f', 104, OBD_IOC_DATA_TYPE)
+#define OBD_IOC_DESTROY _IOW('f', 104, OBD_IOC_DATA_TYPE)
#define OBD_IOC_PREALLOCATE _IOWR('f', 105, OBD_IOC_DATA_TYPE)
-#define OBD_IOC_SETATTR _IOW ('f', 107, OBD_IOC_DATA_TYPE)
+#define OBD_IOC_SETATTR _IOW('f', 107, OBD_IOC_DATA_TYPE)
#define OBD_IOC_GETATTR _IOWR ('f', 108, OBD_IOC_DATA_TYPE)
#define OBD_IOC_READ _IOWR('f', 109, OBD_IOC_DATA_TYPE)
#define OBD_IOC_WRITE _IOWR('f', 110, OBD_IOC_DATA_TYPE)
#define OBD_IOC_STATFS _IOWR('f', 113, OBD_IOC_DATA_TYPE)
-#define OBD_IOC_SYNC _IOW ('f', 114, OBD_IOC_DATA_TYPE)
+#define OBD_IOC_SYNC _IOW('f', 114, OBD_IOC_DATA_TYPE)
#define OBD_IOC_READ2 _IOWR('f', 115, OBD_IOC_DATA_TYPE)
#define OBD_IOC_FORMAT _IOWR('f', 116, OBD_IOC_DATA_TYPE)
#define OBD_IOC_PARTITION _IOWR('f', 117, OBD_IOC_DATA_TYPE)
@@ -308,13 +308,13 @@
#define OBD_IOC_GETDTNAME OBD_IOC_GETNAME
#define OBD_IOC_LOV_GET_CONFIG _IOWR('f', 132, OBD_IOC_DATA_TYPE)
-#define OBD_IOC_CLIENT_RECOVER _IOW ('f', 133, OBD_IOC_DATA_TYPE)
-#define OBD_IOC_PING_TARGET _IOW ('f', 136, OBD_IOC_DATA_TYPE)
+#define OBD_IOC_CLIENT_RECOVER _IOW('f', 133, OBD_IOC_DATA_TYPE)
+#define OBD_IOC_PING_TARGET _IOW('f', 136, OBD_IOC_DATA_TYPE)
#define OBD_IOC_DEC_FS_USE_COUNT _IO ('f', 139)
-#define OBD_IOC_NO_TRANSNO _IOW ('f', 140, OBD_IOC_DATA_TYPE)
-#define OBD_IOC_SET_READONLY _IOW ('f', 141, OBD_IOC_DATA_TYPE)
-#define OBD_IOC_ABORT_RECOVERY _IOR ('f', 142, OBD_IOC_DATA_TYPE)
+#define OBD_IOC_NO_TRANSNO _IOW('f', 140, OBD_IOC_DATA_TYPE)
+#define OBD_IOC_SET_READONLY _IOW('f', 141, OBD_IOC_DATA_TYPE)
+#define OBD_IOC_ABORT_RECOVERY _IOR('f', 142, OBD_IOC_DATA_TYPE)
#define OBD_IOC_ROOT_SQUASH _IOWR('f', 143, OBD_IOC_DATA_TYPE)
@@ -324,27 +324,27 @@
#define OBD_IOC_CLOSE_UUID _IOWR ('f', 147, OBD_IOC_DATA_TYPE)
-#define OBD_IOC_CHANGELOG_SEND _IOW ('f', 148, OBD_IOC_DATA_TYPE)
+#define OBD_IOC_CHANGELOG_SEND _IOW('f', 148, OBD_IOC_DATA_TYPE)
#define OBD_IOC_GETDEVICE _IOWR ('f', 149, OBD_IOC_DATA_TYPE)
#define OBD_IOC_FID2PATH _IOWR ('f', 150, OBD_IOC_DATA_TYPE)
/* see also <lustre/lustre_user.h> for ioctls 151-153 */
/* OBD_IOC_LOV_SETSTRIPE: See also LL_IOC_LOV_SETSTRIPE */
-#define OBD_IOC_LOV_SETSTRIPE _IOW ('f', 154, OBD_IOC_DATA_TYPE)
+#define OBD_IOC_LOV_SETSTRIPE _IOW('f', 154, OBD_IOC_DATA_TYPE)
/* OBD_IOC_LOV_GETSTRIPE: See also LL_IOC_LOV_GETSTRIPE */
-#define OBD_IOC_LOV_GETSTRIPE _IOW ('f', 155, OBD_IOC_DATA_TYPE)
+#define OBD_IOC_LOV_GETSTRIPE _IOW('f', 155, OBD_IOC_DATA_TYPE)
/* OBD_IOC_LOV_SETEA: See also LL_IOC_LOV_SETEA */
-#define OBD_IOC_LOV_SETEA _IOW ('f', 156, OBD_IOC_DATA_TYPE)
+#define OBD_IOC_LOV_SETEA _IOW('f', 156, OBD_IOC_DATA_TYPE)
/* see <lustre/lustre_user.h> for ioctls 157-159 */
/* OBD_IOC_QUOTACHECK: See also LL_IOC_QUOTACHECK */
-#define OBD_IOC_QUOTACHECK _IOW ('f', 160, int)
+#define OBD_IOC_QUOTACHECK _IOW('f', 160, int)
/* OBD_IOC_POLL_QUOTACHECK: See also LL_IOC_POLL_QUOTACHECK */
-#define OBD_IOC_POLL_QUOTACHECK _IOR ('f', 161, struct if_quotacheck *)
+#define OBD_IOC_POLL_QUOTACHECK _IOR('f', 161, struct if_quotacheck *)
/* OBD_IOC_QUOTACTL: See also LL_IOC_QUOTACTL */
#define OBD_IOC_QUOTACTL _IOWR('f', 162, struct if_quotactl)
/* see also <lustre/lustre_user.h> for ioctls 163-176 */
-#define OBD_IOC_CHANGELOG_REG _IOW ('f', 177, struct obd_ioctl_data)
-#define OBD_IOC_CHANGELOG_DEREG _IOW ('f', 178, struct obd_ioctl_data)
-#define OBD_IOC_CHANGELOG_CLEAR _IOW ('f', 179, struct obd_ioctl_data)
+#define OBD_IOC_CHANGELOG_REG _IOW('f', 177, struct obd_ioctl_data)
+#define OBD_IOC_CHANGELOG_DEREG _IOW('f', 178, struct obd_ioctl_data)
+#define OBD_IOC_CHANGELOG_CLEAR _IOW('f', 179, struct obd_ioctl_data)
#define OBD_IOC_RECORD _IOWR('f', 180, OBD_IOC_DATA_TYPE)
#define OBD_IOC_ENDRECORD _IOWR('f', 181, OBD_IOC_DATA_TYPE)
#define OBD_IOC_PARSE _IOWR('f', 182, OBD_IOC_DATA_TYPE)
@@ -352,7 +352,7 @@
#define OBD_IOC_PROCESS_CFG _IOWR('f', 184, OBD_IOC_DATA_TYPE)
#define OBD_IOC_DUMP_LOG _IOWR('f', 185, OBD_IOC_DATA_TYPE)
#define OBD_IOC_CLEAR_LOG _IOWR('f', 186, OBD_IOC_DATA_TYPE)
-#define OBD_IOC_PARAM _IOW ('f', 187, OBD_IOC_DATA_TYPE)
+#define OBD_IOC_PARAM _IOW('f', 187, OBD_IOC_DATA_TYPE)
#define OBD_IOC_POOL _IOWR('f', 188, OBD_IOC_DATA_TYPE)
#define OBD_IOC_REPLACE_NIDS _IOWR('f', 189, OBD_IOC_DATA_TYPE)
@@ -522,6 +522,28 @@
sigmask(SIGTERM) | sigmask(SIGQUIT) | \
sigmask(SIGALRM))
+/**
+ * wait_queue_t of Linux (version < 2.6.34) is a FIFO list for exclusively
+ * waiting threads, which is not always desirable because all threads will
+ * be waken up again and again, even user only needs a few of them to be
+ * active most time. This is not good for performance because cache can
+ * be polluted by different threads.
+ *
+ * LIFO list can resolve this problem because we always wakeup the most
+ * recent active thread by default.
+ *
+ * NB: please don't call non-exclusive & exclusive wait on the same
+ * waitq if add_wait_queue_exclusive_head is used.
+ */
+#define add_wait_queue_exclusive_head(waitq, link) \
+{ \
+ unsigned long flags; \
+ \
+ spin_lock_irqsave(&((waitq)->lock), flags); \
+ __add_wait_queue_exclusive(waitq, link); \
+ spin_unlock_irqrestore(&((waitq)->lock), flags); \
+}
+
/*
* wait for @condition to become true, but no longer than timeout, specified
* by @info.
@@ -578,7 +600,7 @@
\
if (condition) \
break; \
- if (cfs_signal_pending()) { \
+ if (signal_pending(current)) { \
if (info->lwi_on_signal && \
(__timeout == 0 || __allow_intr)) { \
if (info->lwi_on_signal != LWI_ON_SIGNAL_NOOP) \
diff --git a/drivers/staging/lustre/lustre/include/lustre_mdc.h b/drivers/staging/lustre/lustre/include/lustre_mdc.h
index af77eb3..f267ff8 100644
--- a/drivers/staging/lustre/lustre/include/lustre_mdc.h
+++ b/drivers/staging/lustre/lustre/include/lustre_mdc.h
@@ -64,9 +64,27 @@
struct ptlrpc_request;
struct obd_device;
+/**
+ * Serializes in-flight MDT-modifying RPC requests to preserve idempotency.
+ *
+ * This mutex is used to implement execute-once semantics on the MDT.
+ * The MDT stores the last transaction ID and result for every client in
+ * its last_rcvd file. If the client doesn't get a reply, it can safely
+ * resend the request and the MDT will reconstruct the reply being aware
+ * that the request has already been executed. Without this lock,
+ * execution status of concurrent in-flight requests would be
+ * overwritten.
+ *
+ * This design limits the extent to which we can keep a full pipeline of
+ * in-flight requests from a single client. This limitation could be
+ * overcome by allowing multiple slots per client in the last_rcvd file.
+ */
struct mdc_rpc_lock {
+ /** Lock protecting in-flight RPC concurrency. */
struct mutex rpcl_mutex;
+ /** Intent associated with currently executing request. */
struct lookup_intent *rpcl_it;
+ /** Used for MDS/RPC load testing purposes. */
int rpcl_fakes;
};
diff --git a/drivers/staging/lustre/lustre/include/lustre_net.h b/drivers/staging/lustre/lustre/include/lustre_net.h
index 69586a5..a7973d5 100644
--- a/drivers/staging/lustre/lustre/include/lustre_net.h
+++ b/drivers/staging/lustre/lustre/include/lustre_net.h
@@ -1327,7 +1327,9 @@
/* allow the req to be sent if the import is in recovery
* status
*/
- rq_allow_replay:1;
+ rq_allow_replay:1,
+ /* bulk request, sent to server, but uncommitted */
+ rq_unstable:1;
unsigned int rq_nr_resend;
diff --git a/drivers/staging/lustre/lustre/include/lustre_param.h b/drivers/staging/lustre/lustre/include/lustre_param.h
index 383fe6f..a42cf90 100644
--- a/drivers/staging/lustre/lustre/include/lustre_param.h
+++ b/drivers/staging/lustre/lustre/include/lustre_param.h
@@ -89,6 +89,7 @@
/* Prefixes for parameters handled by obd's proc methods (XXX_process_config) */
#define PARAM_OST "ost."
+#define PARAM_OSD "osd."
#define PARAM_OSC "osc."
#define PARAM_MDT "mdt."
#define PARAM_MDD "mdd."
diff --git a/drivers/staging/lustre/lustre/include/lustre_req_layout.h b/drivers/staging/lustre/lustre/include/lustre_req_layout.h
index b2e67fc..0aac439 100644
--- a/drivers/staging/lustre/lustre/include/lustre_req_layout.h
+++ b/drivers/staging/lustre/lustre/include/lustre_req_layout.h
@@ -137,6 +137,7 @@
/* fid/fld req_format */
extern struct req_format RQF_SEQ_QUERY;
extern struct req_format RQF_FLD_QUERY;
+extern struct req_format RQF_FLD_READ;
/* MDS req_format */
extern struct req_format RQF_MDS_CONNECT;
extern struct req_format RQF_MDS_DISCONNECT;
@@ -199,7 +200,7 @@
extern struct req_format RQF_OST_BRW_WRITE;
extern struct req_format RQF_OST_STATFS;
extern struct req_format RQF_OST_SET_GRANT_INFO;
-extern struct req_format RQF_OST_GET_INFO_GENERIC;
+extern struct req_format RQF_OST_GET_INFO;
extern struct req_format RQF_OST_GET_INFO_LAST_ID;
extern struct req_format RQF_OST_GET_INFO_LAST_FID;
extern struct req_format RQF_OST_SET_INFO_LAST_FID;
diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h
index 4264d97..2d926e0 100644
--- a/drivers/staging/lustre/lustre/include/obd.h
+++ b/drivers/staging/lustre/lustre/include/obd.h
@@ -37,7 +37,7 @@
#ifndef __OBD_H
#define __OBD_H
-#include "linux/obd.h"
+#include <linux/spinlock.h>
#define IOC_OSC_TYPE 'h'
#define IOC_OSC_MIN_NR 20
@@ -54,6 +54,7 @@
#include "lustre_export.h"
#include "lustre_fid.h"
#include "lustre_fld.h"
+#include "lustre_intent.h"
#define MAX_OBD_DEVICES 8192
@@ -165,9 +166,6 @@
obd_enqueue_update_f oi_cb_up;
};
-void lov_stripe_lock(struct lov_stripe_md *md);
-void lov_stripe_unlock(struct lov_stripe_md *md);
-
struct obd_type {
struct list_head typ_chain;
struct obd_ops *typ_dt_ops;
@@ -293,14 +291,10 @@
* blocking everywhere, but we don't want to slow down fast-path of
* our main platform.)
*
- * Exact type of ->cl_loi_list_lock is defined in arch/obd.h together
- * with client_obd_list_{un,}lock() and
- * client_obd_list_lock_{init,done}() functions.
- *
* NB by Jinshan: though field names are still _loi_, but actually
* osc_object{}s are in the list.
*/
- struct client_obd_lock cl_loi_list_lock;
+ spinlock_t cl_loi_list_lock;
struct list_head cl_loi_ready_list;
struct list_head cl_loi_hp_ready_list;
struct list_head cl_loi_write_list;
@@ -327,7 +321,8 @@
atomic_t cl_lru_shrinkers;
atomic_t cl_lru_in_list;
struct list_head cl_lru_list; /* lru page list */
- struct client_obd_lock cl_lru_list_lock; /* page list protector */
+ spinlock_t cl_lru_list_lock; /* page list protector */
+ atomic_t cl_unstable_count;
/* number of in flight destroy rpcs is limited to max_rpcs_in_flight */
atomic_t cl_destroy_in_flight;
@@ -364,6 +359,7 @@
/* ptlrpc work for writeback in ptlrpcd context */
void *cl_writeback_work;
+ void *cl_lru_work;
/* hash tables for osc_quota_info */
struct cfs_hash *cl_quota_hash[MAXQUOTAS];
};
@@ -391,45 +387,9 @@
struct rw_semaphore op_rw_sem; /* to protect ost_pool use */
};
-/* Round-robin allocator data */
-struct lov_qos_rr {
- __u32 lqr_start_idx; /* start index of new inode */
- __u32 lqr_offset_idx; /* aliasing for start_idx */
- int lqr_start_count; /* reseed counter */
- struct ost_pool lqr_pool; /* round-robin optimized list */
- unsigned long lqr_dirty:1; /* recalc round-robin list */
-};
-
/* allow statfs data caching for 1 second */
#define OBD_STATFS_CACHE_SECONDS 1
-struct lov_statfs_data {
- struct obd_info lsd_oi;
- struct obd_statfs lsd_statfs;
-};
-
-/* Stripe placement optimization */
-struct lov_qos {
- struct list_head lq_oss_list; /* list of OSSs that targets use */
- struct rw_semaphore lq_rw_sem;
- __u32 lq_active_oss_count;
- unsigned int lq_prio_free; /* priority for free space */
- unsigned int lq_threshold_rr;/* priority for rr */
- struct lov_qos_rr lq_rr; /* round robin qos data */
- unsigned long lq_dirty:1, /* recalc qos data */
- lq_same_space:1,/* the ost's all have approx.
- * the same space avail
- */
- lq_reset:1, /* zero current penalties */
- lq_statfs_in_progress:1; /* statfs op in
- progress */
- /* qos statfs data */
- struct lov_statfs_data *lq_statfs_data;
- wait_queue_head_t lq_statfs_waitq; /* waitqueue to notify statfs
- * requests completion
- */
-};
-
struct lov_tgt_desc {
struct list_head ltd_kill;
struct obd_uuid ltd_uuid;
@@ -442,25 +402,6 @@
ltd_reap:1; /* should this target be deleted */
};
-/* Pool metadata */
-#define pool_tgt_size(_p) _p->pool_obds.op_size
-#define pool_tgt_count(_p) _p->pool_obds.op_count
-#define pool_tgt_array(_p) _p->pool_obds.op_array
-#define pool_tgt_rw_sem(_p) _p->pool_obds.op_rw_sem
-
-struct pool_desc {
- char pool_name[LOV_MAXPOOLNAME + 1]; /* name of pool */
- struct ost_pool pool_obds; /* pool members */
- atomic_t pool_refcount; /* pool ref. counter */
- struct lov_qos_rr pool_rr; /* round robin qos */
- struct hlist_node pool_hash; /* access by poolname */
- struct list_head pool_list; /* serial access */
- struct dentry *pool_debugfs_entry; /* file in debugfs */
- struct obd_device *pool_lobd; /* obd of the lov/lod to which
- * this pool belongs
- */
-};
-
struct lov_obd {
struct lov_desc desc;
struct lov_tgt_desc **lov_tgts; /* sparse array */
@@ -468,8 +409,6 @@
struct mutex lov_lock;
struct obd_connect_data lov_ocd;
atomic_t lov_refcount;
- __u32 lov_tgt_count; /* how many OBD's */
- __u32 lov_active_tgt_count; /* how many active */
__u32 lov_death_row;/* tgts scheduled to be deleted */
__u32 lov_tgt_size; /* size of tgts array */
int lov_connects;
@@ -479,7 +418,7 @@
struct dentry *lov_pool_debugfs_entry;
enum lustre_sec_part lov_sp_me;
- /* Cached LRU pages from upper layer */
+ /* Cached LRU and unstable data from upper layer */
void *lov_cache;
struct rw_semaphore lov_notify_lock;
@@ -511,7 +450,7 @@
struct obd_uuid cluuid;
struct obd_export *exp;
- struct mutex init_mutex;
+ struct mutex lmv_init_mutex;
int connected;
int max_easize;
int max_def_easize;
diff --git a/drivers/staging/lustre/lustre/include/obd_cksum.h b/drivers/staging/lustre/lustre/include/obd_cksum.h
index 637fa22..f6c18df 100644
--- a/drivers/staging/lustre/lustre/include/obd_cksum.h
+++ b/drivers/staging/lustre/lustre/include/obd_cksum.h
@@ -35,6 +35,7 @@
#ifndef __OBD_CKSUM
#define __OBD_CKSUM
#include "../../include/linux/libcfs/libcfs.h"
+#include "../../include/linux/libcfs/libcfs_crypto.h"
#include "lustre/lustre_idl.h"
static inline unsigned char cksum_obd2cfs(enum cksum_type cksum_type)
diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h
index 706869f..32863bc 100644
--- a/drivers/staging/lustre/lustre/include/obd_class.h
+++ b/drivers/staging/lustre/lustre/include/obd_class.h
@@ -477,7 +477,7 @@
struct lu_context session_ctx;
struct lu_env env;
- lu_context_init(&session_ctx, LCT_SESSION);
+ lu_context_init(&session_ctx, LCT_SESSION | LCT_SERVER_SESSION);
session_ctx.lc_thread = NULL;
lu_context_enter(&session_ctx);
@@ -490,8 +490,9 @@
obd->obd_lu_dev = d;
d->ld_obd = obd;
rc = 0;
- } else
+ } else {
rc = PTR_ERR(d);
+ }
}
lu_context_exit(&session_ctx);
lu_context_fini(&session_ctx);
diff --git a/drivers/staging/lustre/lustre/include/obd_support.h b/drivers/staging/lustre/lustre/include/obd_support.h
index f8ee3a3..60034d39 100644
--- a/drivers/staging/lustre/lustre/include/obd_support.h
+++ b/drivers/staging/lustre/lustre/include/obd_support.h
@@ -58,6 +58,7 @@
extern int at_extra;
extern unsigned int obd_sync_filter;
extern unsigned int obd_max_dirty_pages;
+extern atomic_t obd_unstable_pages;
extern atomic_t obd_dirty_pages;
extern atomic_t obd_dirty_transit_pages;
extern char obd_jobid_var[];
@@ -289,6 +290,7 @@
#define OBD_FAIL_OST_ENOINO 0x229
#define OBD_FAIL_OST_DQACQ_NET 0x230
#define OBD_FAIL_OST_STATFS_EINPROGRESS 0x231
+#define OBD_FAIL_OST_SET_INFO_NET 0x232
#define OBD_FAIL_LDLM 0x300
#define OBD_FAIL_LDLM_NAMESPACE_NEW 0x301
@@ -319,6 +321,7 @@
#define OBD_FAIL_LDLM_AGL_DELAY 0x31a
#define OBD_FAIL_LDLM_AGL_NOLOCK 0x31b
#define OBD_FAIL_LDLM_OST_LVB 0x31c
+#define OBD_FAIL_LDLM_ENQUEUE_HANG 0x31d
/* LOCKLESS IO */
#define OBD_FAIL_LDLM_SET_CONTENTION 0x385
@@ -426,6 +429,7 @@
#define OBD_FAIL_FLD 0x1100
#define OBD_FAIL_FLD_QUERY_NET 0x1101
+#define OBD_FAIL_FLD_READ_NET 0x1102
#define OBD_FAIL_SEC_CTX 0x1200
#define OBD_FAIL_SEC_CTX_INIT_NET 0x1201
diff --git a/drivers/staging/lustre/lustre/lclient/glimpse.c b/drivers/staging/lustre/lustre/lclient/glimpse.c
deleted file mode 100644
index c4e8a08..0000000
--- a/drivers/staging/lustre/lustre/lclient/glimpse.c
+++ /dev/null
@@ -1,270 +0,0 @@
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.sun.com/software/products/lustre/docs/GPLv2.pdf
- *
- * Please contact Sun Microsystems, Inc., 4150 Network Circle, Santa Clara,
- * CA 95054 USA or visit www.sun.com if you need additional information or
- * have any questions.
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2011, 2012, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- *
- * glimpse code shared between vvp and liblustre (and other Lustre clients in
- * the future).
- *
- * Author: Nikita Danilov <nikita.danilov@sun.com>
- * Author: Oleg Drokin <oleg.drokin@sun.com>
- */
-
-#include "../../include/linux/libcfs/libcfs.h"
-#include "../include/obd_class.h"
-#include "../include/obd_support.h"
-#include "../include/obd.h"
-
-#include "../include/lustre_dlm.h"
-#include "../include/lustre_lite.h"
-#include "../include/lustre_mdc.h"
-#include <linux/pagemap.h>
-#include <linux/file.h>
-
-#include "../include/cl_object.h"
-#include "../include/lclient.h"
-#include "../llite/llite_internal.h"
-
-static const struct cl_lock_descr whole_file = {
- .cld_start = 0,
- .cld_end = CL_PAGE_EOF,
- .cld_mode = CLM_READ
-};
-
-/*
- * Check whether file has possible unwriten pages.
- *
- * \retval 1 file is mmap-ed or has dirty pages
- * 0 otherwise
- */
-blkcnt_t dirty_cnt(struct inode *inode)
-{
- blkcnt_t cnt = 0;
- struct ccc_object *vob = cl_inode2ccc(inode);
- void *results[1];
-
- if (inode->i_mapping)
- cnt += radix_tree_gang_lookup_tag(&inode->i_mapping->page_tree,
- results, 0, 1,
- PAGECACHE_TAG_DIRTY);
- if (cnt == 0 && atomic_read(&vob->cob_mmap_cnt) > 0)
- cnt = 1;
-
- return (cnt > 0) ? 1 : 0;
-}
-
-int cl_glimpse_lock(const struct lu_env *env, struct cl_io *io,
- struct inode *inode, struct cl_object *clob, int agl)
-{
- struct cl_lock_descr *descr = &ccc_env_info(env)->cti_descr;
- struct cl_inode_info *lli = cl_i2info(inode);
- const struct lu_fid *fid = lu_object_fid(&clob->co_lu);
- struct ccc_io *cio = ccc_env_io(env);
- struct cl_lock *lock;
- int result;
-
- result = 0;
- if (!(lli->lli_flags & LLIF_MDS_SIZE_LOCK)) {
- CDEBUG(D_DLMTRACE, "Glimpsing inode "DFID"\n", PFID(fid));
- if (lli->lli_has_smd) {
- /* NOTE: this looks like DLM lock request, but it may
- * not be one. Due to CEF_ASYNC flag (translated
- * to LDLM_FL_HAS_INTENT by osc), this is
- * glimpse request, that won't revoke any
- * conflicting DLM locks held. Instead,
- * ll_glimpse_callback() will be called on each
- * client holding a DLM lock against this file,
- * and resulting size will be returned for each
- * stripe. DLM lock on [0, EOF] is acquired only
- * if there were no conflicting locks. If there
- * were conflicting locks, enqueuing or waiting
- * fails with -ENAVAIL, but valid inode
- * attributes are returned anyway.
- */
- *descr = whole_file;
- descr->cld_obj = clob;
- descr->cld_mode = CLM_PHANTOM;
- descr->cld_enq_flags = CEF_ASYNC | CEF_MUST;
- if (agl)
- descr->cld_enq_flags |= CEF_AGL;
- cio->cui_glimpse = 1;
- /*
- * CEF_ASYNC is used because glimpse sub-locks cannot
- * deadlock (because they never conflict with other
- * locks) and, hence, can be enqueued out-of-order.
- *
- * CEF_MUST protects glimpse lock from conversion into
- * a lockless mode.
- */
- lock = cl_lock_request(env, io, descr, "glimpse",
- current);
- cio->cui_glimpse = 0;
-
- if (!lock)
- return 0;
-
- if (IS_ERR(lock))
- return PTR_ERR(lock);
-
- LASSERT(agl == 0);
- result = cl_wait(env, lock);
- if (result == 0) {
- cl_merge_lvb(env, inode);
- if (cl_isize_read(inode) > 0 &&
- inode->i_blocks == 0) {
- /*
- * LU-417: Add dirty pages block count
- * lest i_blocks reports 0, some "cp" or
- * "tar" may think it's a completely
- * sparse file and skip it.
- */
- inode->i_blocks = dirty_cnt(inode);
- }
- cl_unuse(env, lock);
- }
- cl_lock_release(env, lock, "glimpse", current);
- } else {
- CDEBUG(D_DLMTRACE, "No objects for inode\n");
- cl_merge_lvb(env, inode);
- }
- }
-
- return result;
-}
-
-static int cl_io_get(struct inode *inode, struct lu_env **envout,
- struct cl_io **ioout, int *refcheck)
-{
- struct lu_env *env;
- struct cl_io *io;
- struct cl_inode_info *lli = cl_i2info(inode);
- struct cl_object *clob = lli->lli_clob;
- int result;
-
- if (S_ISREG(cl_inode_mode(inode))) {
- env = cl_env_get(refcheck);
- if (!IS_ERR(env)) {
- io = ccc_env_thread_io(env);
- io->ci_obj = clob;
- *envout = env;
- *ioout = io;
- result = 1;
- } else
- result = PTR_ERR(env);
- } else
- result = 0;
- return result;
-}
-
-int cl_glimpse_size0(struct inode *inode, int agl)
-{
- /*
- * We don't need ast_flags argument to cl_glimpse_size(), because
- * osc_lock_enqueue() takes care of the possible deadlock that said
- * argument was introduced to avoid.
- */
- /*
- * XXX but note that ll_file_seek() passes LDLM_FL_BLOCK_NOWAIT to
- * cl_glimpse_size(), which doesn't make sense: glimpse locks are not
- * blocking anyway.
- */
- struct lu_env *env = NULL;
- struct cl_io *io = NULL;
- int result;
- int refcheck;
-
- result = cl_io_get(inode, &env, &io, &refcheck);
- if (result > 0) {
-again:
- io->ci_verify_layout = 1;
- result = cl_io_init(env, io, CIT_MISC, io->ci_obj);
- if (result > 0)
- /*
- * nothing to do for this io. This currently happens
- * when stripe sub-object's are not yet created.
- */
- result = io->ci_result;
- else if (result == 0)
- result = cl_glimpse_lock(env, io, inode, io->ci_obj,
- agl);
-
- OBD_FAIL_TIMEOUT(OBD_FAIL_GLIMPSE_DELAY, 2);
- cl_io_fini(env, io);
- if (unlikely(io->ci_need_restart))
- goto again;
- cl_env_put(env, &refcheck);
- }
- return result;
-}
-
-int cl_local_size(struct inode *inode)
-{
- struct lu_env *env = NULL;
- struct cl_io *io = NULL;
- struct ccc_thread_info *cti;
- struct cl_object *clob;
- struct cl_lock_descr *descr;
- struct cl_lock *lock;
- int result;
- int refcheck;
-
- if (!cl_i2info(inode)->lli_has_smd)
- return 0;
-
- result = cl_io_get(inode, &env, &io, &refcheck);
- if (result <= 0)
- return result;
-
- clob = io->ci_obj;
- result = cl_io_init(env, io, CIT_MISC, clob);
- if (result > 0)
- result = io->ci_result;
- else if (result == 0) {
- cti = ccc_env_info(env);
- descr = &cti->cti_descr;
-
- *descr = whole_file;
- descr->cld_obj = clob;
- lock = cl_lock_peek(env, io, descr, "localsize", current);
- if (lock) {
- cl_merge_lvb(env, inode);
- cl_unuse(env, lock);
- cl_lock_release(env, lock, "localsize", current);
- result = 0;
- } else
- result = -ENODATA;
- }
- cl_io_fini(env, io);
- cl_env_put(env, &refcheck);
- return result;
-}
diff --git a/drivers/staging/lustre/lustre/lclient/lcommon_cl.c b/drivers/staging/lustre/lustre/lclient/lcommon_cl.c
deleted file mode 100644
index 96141d1..0000000
--- a/drivers/staging/lustre/lustre/lclient/lcommon_cl.c
+++ /dev/null
@@ -1,1203 +0,0 @@
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.sun.com/software/products/lustre/docs/GPLv2.pdf
- *
- * Please contact Sun Microsystems, Inc., 4150 Network Circle, Santa Clara,
- * CA 95054 USA or visit www.sun.com if you need additional information or
- * have any questions.
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2011, 2015, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- *
- * cl code shared between vvp and liblustre (and other Lustre clients in the
- * future).
- *
- * Author: Nikita Danilov <nikita.danilov@sun.com>
- */
-
-#define DEBUG_SUBSYSTEM S_LLITE
-
-#include "../../include/linux/libcfs/libcfs.h"
-# include <linux/fs.h>
-# include <linux/sched.h>
-# include <linux/mm.h>
-# include <linux/quotaops.h>
-# include <linux/highmem.h>
-# include <linux/pagemap.h>
-# include <linux/rbtree.h>
-
-#include "../include/obd.h"
-#include "../include/obd_support.h"
-#include "../include/lustre_fid.h"
-#include "../include/lustre_lite.h"
-#include "../include/lustre_dlm.h"
-#include "../include/lustre_ver.h"
-#include "../include/lustre_mdc.h"
-#include "../include/cl_object.h"
-
-#include "../include/lclient.h"
-
-#include "../llite/llite_internal.h"
-
-static const struct cl_req_operations ccc_req_ops;
-
-/*
- * ccc_ prefix stands for "Common Client Code".
- */
-
-static struct kmem_cache *ccc_lock_kmem;
-static struct kmem_cache *ccc_object_kmem;
-static struct kmem_cache *ccc_thread_kmem;
-static struct kmem_cache *ccc_session_kmem;
-static struct kmem_cache *ccc_req_kmem;
-
-static struct lu_kmem_descr ccc_caches[] = {
- {
- .ckd_cache = &ccc_lock_kmem,
- .ckd_name = "ccc_lock_kmem",
- .ckd_size = sizeof(struct ccc_lock)
- },
- {
- .ckd_cache = &ccc_object_kmem,
- .ckd_name = "ccc_object_kmem",
- .ckd_size = sizeof(struct ccc_object)
- },
- {
- .ckd_cache = &ccc_thread_kmem,
- .ckd_name = "ccc_thread_kmem",
- .ckd_size = sizeof(struct ccc_thread_info),
- },
- {
- .ckd_cache = &ccc_session_kmem,
- .ckd_name = "ccc_session_kmem",
- .ckd_size = sizeof(struct ccc_session)
- },
- {
- .ckd_cache = &ccc_req_kmem,
- .ckd_name = "ccc_req_kmem",
- .ckd_size = sizeof(struct ccc_req)
- },
- {
- .ckd_cache = NULL
- }
-};
-
-/*****************************************************************************
- *
- * Vvp device and device type functions.
- *
- */
-
-void *ccc_key_init(const struct lu_context *ctx, struct lu_context_key *key)
-{
- struct ccc_thread_info *info;
-
- info = kmem_cache_zalloc(ccc_thread_kmem, GFP_NOFS);
- if (!info)
- info = ERR_PTR(-ENOMEM);
- return info;
-}
-
-void ccc_key_fini(const struct lu_context *ctx,
- struct lu_context_key *key, void *data)
-{
- struct ccc_thread_info *info = data;
-
- kmem_cache_free(ccc_thread_kmem, info);
-}
-
-void *ccc_session_key_init(const struct lu_context *ctx,
- struct lu_context_key *key)
-{
- struct ccc_session *session;
-
- session = kmem_cache_zalloc(ccc_session_kmem, GFP_NOFS);
- if (!session)
- session = ERR_PTR(-ENOMEM);
- return session;
-}
-
-void ccc_session_key_fini(const struct lu_context *ctx,
- struct lu_context_key *key, void *data)
-{
- struct ccc_session *session = data;
-
- kmem_cache_free(ccc_session_kmem, session);
-}
-
-struct lu_context_key ccc_key = {
- .lct_tags = LCT_CL_THREAD,
- .lct_init = ccc_key_init,
- .lct_fini = ccc_key_fini
-};
-
-struct lu_context_key ccc_session_key = {
- .lct_tags = LCT_SESSION,
- .lct_init = ccc_session_key_init,
- .lct_fini = ccc_session_key_fini
-};
-
-/* type constructor/destructor: ccc_type_{init,fini,start,stop}(). */
-/* LU_TYPE_INIT_FINI(ccc, &ccc_key, &ccc_session_key); */
-
-int ccc_device_init(const struct lu_env *env, struct lu_device *d,
- const char *name, struct lu_device *next)
-{
- struct ccc_device *vdv;
- int rc;
-
- vdv = lu2ccc_dev(d);
- vdv->cdv_next = lu2cl_dev(next);
-
- LASSERT(d->ld_site && next->ld_type);
- next->ld_site = d->ld_site;
- rc = next->ld_type->ldt_ops->ldto_device_init(
- env, next, next->ld_type->ldt_name, NULL);
- if (rc == 0) {
- lu_device_get(next);
- lu_ref_add(&next->ld_reference, "lu-stack", &lu_site_init);
- }
- return rc;
-}
-
-struct lu_device *ccc_device_fini(const struct lu_env *env,
- struct lu_device *d)
-{
- return cl2lu_dev(lu2ccc_dev(d)->cdv_next);
-}
-
-struct lu_device *ccc_device_alloc(const struct lu_env *env,
- struct lu_device_type *t,
- struct lustre_cfg *cfg,
- const struct lu_device_operations *luops,
- const struct cl_device_operations *clops)
-{
- struct ccc_device *vdv;
- struct lu_device *lud;
- struct cl_site *site;
- int rc;
-
- vdv = kzalloc(sizeof(*vdv), GFP_NOFS);
- if (!vdv)
- return ERR_PTR(-ENOMEM);
-
- lud = &vdv->cdv_cl.cd_lu_dev;
- cl_device_init(&vdv->cdv_cl, t);
- ccc2lu_dev(vdv)->ld_ops = luops;
- vdv->cdv_cl.cd_ops = clops;
-
- site = kzalloc(sizeof(*site), GFP_NOFS);
- if (site) {
- rc = cl_site_init(site, &vdv->cdv_cl);
- if (rc == 0)
- rc = lu_site_init_finish(&site->cs_lu);
- else {
- LASSERT(!lud->ld_site);
- CERROR("Cannot init lu_site, rc %d.\n", rc);
- kfree(site);
- }
- } else
- rc = -ENOMEM;
- if (rc != 0) {
- ccc_device_free(env, lud);
- lud = ERR_PTR(rc);
- }
- return lud;
-}
-
-struct lu_device *ccc_device_free(const struct lu_env *env,
- struct lu_device *d)
-{
- struct ccc_device *vdv = lu2ccc_dev(d);
- struct cl_site *site = lu2cl_site(d->ld_site);
- struct lu_device *next = cl2lu_dev(vdv->cdv_next);
-
- if (d->ld_site) {
- cl_site_fini(site);
- kfree(site);
- }
- cl_device_fini(lu2cl_dev(d));
- kfree(vdv);
- return next;
-}
-
-int ccc_req_init(const struct lu_env *env, struct cl_device *dev,
- struct cl_req *req)
-{
- struct ccc_req *vrq;
- int result;
-
- vrq = kmem_cache_zalloc(ccc_req_kmem, GFP_NOFS);
- if (vrq) {
- cl_req_slice_add(req, &vrq->crq_cl, dev, &ccc_req_ops);
- result = 0;
- } else
- result = -ENOMEM;
- return result;
-}
-
-/**
- * An `emergency' environment used by ccc_inode_fini() when cl_env_get()
- * fails. Access to this environment is serialized by ccc_inode_fini_guard
- * mutex.
- */
-static struct lu_env *ccc_inode_fini_env;
-
-/**
- * A mutex serializing calls to slp_inode_fini() under extreme memory
- * pressure, when environments cannot be allocated.
- */
-static DEFINE_MUTEX(ccc_inode_fini_guard);
-static int dummy_refcheck;
-
-int ccc_global_init(struct lu_device_type *device_type)
-{
- int result;
-
- result = lu_kmem_init(ccc_caches);
- if (result)
- return result;
-
- result = lu_device_type_init(device_type);
- if (result)
- goto out_kmem;
-
- ccc_inode_fini_env = cl_env_alloc(&dummy_refcheck,
- LCT_REMEMBER|LCT_NOREF);
- if (IS_ERR(ccc_inode_fini_env)) {
- result = PTR_ERR(ccc_inode_fini_env);
- goto out_device;
- }
-
- ccc_inode_fini_env->le_ctx.lc_cookie = 0x4;
- return 0;
-out_device:
- lu_device_type_fini(device_type);
-out_kmem:
- lu_kmem_fini(ccc_caches);
- return result;
-}
-
-void ccc_global_fini(struct lu_device_type *device_type)
-{
- if (ccc_inode_fini_env) {
- cl_env_put(ccc_inode_fini_env, &dummy_refcheck);
- ccc_inode_fini_env = NULL;
- }
- lu_device_type_fini(device_type);
- lu_kmem_fini(ccc_caches);
-}
-
-/*****************************************************************************
- *
- * Object operations.
- *
- */
-
-struct lu_object *ccc_object_alloc(const struct lu_env *env,
- const struct lu_object_header *unused,
- struct lu_device *dev,
- const struct cl_object_operations *clops,
- const struct lu_object_operations *luops)
-{
- struct ccc_object *vob;
- struct lu_object *obj;
-
- vob = kmem_cache_zalloc(ccc_object_kmem, GFP_NOFS);
- if (vob) {
- struct cl_object_header *hdr;
-
- obj = ccc2lu(vob);
- hdr = &vob->cob_header;
- cl_object_header_init(hdr);
- lu_object_init(obj, &hdr->coh_lu, dev);
- lu_object_add_top(&hdr->coh_lu, obj);
-
- vob->cob_cl.co_ops = clops;
- obj->lo_ops = luops;
- } else
- obj = NULL;
- return obj;
-}
-
-int ccc_object_init0(const struct lu_env *env,
- struct ccc_object *vob,
- const struct cl_object_conf *conf)
-{
- vob->cob_inode = conf->coc_inode;
- vob->cob_transient_pages = 0;
- cl_object_page_init(&vob->cob_cl, sizeof(struct ccc_page));
- return 0;
-}
-
-int ccc_object_init(const struct lu_env *env, struct lu_object *obj,
- const struct lu_object_conf *conf)
-{
- struct ccc_device *dev = lu2ccc_dev(obj->lo_dev);
- struct ccc_object *vob = lu2ccc(obj);
- struct lu_object *below;
- struct lu_device *under;
- int result;
-
- under = &dev->cdv_next->cd_lu_dev;
- below = under->ld_ops->ldo_object_alloc(env, obj->lo_header, under);
- if (below) {
- const struct cl_object_conf *cconf;
-
- cconf = lu2cl_conf(conf);
- INIT_LIST_HEAD(&vob->cob_pending_list);
- lu_object_add(obj, below);
- result = ccc_object_init0(env, vob, cconf);
- } else
- result = -ENOMEM;
- return result;
-}
-
-void ccc_object_free(const struct lu_env *env, struct lu_object *obj)
-{
- struct ccc_object *vob = lu2ccc(obj);
-
- lu_object_fini(obj);
- lu_object_header_fini(obj->lo_header);
- kmem_cache_free(ccc_object_kmem, vob);
-}
-
-int ccc_lock_init(const struct lu_env *env,
- struct cl_object *obj, struct cl_lock *lock,
- const struct cl_io *unused,
- const struct cl_lock_operations *lkops)
-{
- struct ccc_lock *clk;
- int result;
-
- CLOBINVRNT(env, obj, ccc_object_invariant(obj));
-
- clk = kmem_cache_zalloc(ccc_lock_kmem, GFP_NOFS);
- if (clk) {
- cl_lock_slice_add(lock, &clk->clk_cl, obj, lkops);
- result = 0;
- } else
- result = -ENOMEM;
- return result;
-}
-
-int ccc_object_glimpse(const struct lu_env *env,
- const struct cl_object *obj, struct ost_lvb *lvb)
-{
- struct inode *inode = ccc_object_inode(obj);
-
- lvb->lvb_mtime = cl_inode_mtime(inode);
- lvb->lvb_atime = cl_inode_atime(inode);
- lvb->lvb_ctime = cl_inode_ctime(inode);
- /*
- * LU-417: Add dirty pages block count lest i_blocks reports 0, some
- * "cp" or "tar" on remote node may think it's a completely sparse file
- * and skip it.
- */
- if (lvb->lvb_size > 0 && lvb->lvb_blocks == 0)
- lvb->lvb_blocks = dirty_cnt(inode);
- return 0;
-}
-
-static void ccc_object_size_lock(struct cl_object *obj)
-{
- struct inode *inode = ccc_object_inode(obj);
-
- ll_inode_size_lock(inode);
- cl_object_attr_lock(obj);
-}
-
-static void ccc_object_size_unlock(struct cl_object *obj)
-{
- struct inode *inode = ccc_object_inode(obj);
-
- cl_object_attr_unlock(obj);
- ll_inode_size_unlock(inode);
-}
-
-/*****************************************************************************
- *
- * Page operations.
- *
- */
-
-struct page *ccc_page_vmpage(const struct lu_env *env,
- const struct cl_page_slice *slice)
-{
- return cl2vm_page(slice);
-}
-
-int ccc_page_is_under_lock(const struct lu_env *env,
- const struct cl_page_slice *slice,
- struct cl_io *io)
-{
- struct ccc_io *cio = ccc_env_io(env);
- struct cl_lock_descr *desc = &ccc_env_info(env)->cti_descr;
- struct cl_page *page = slice->cpl_page;
-
- int result;
-
- if (io->ci_type == CIT_READ || io->ci_type == CIT_WRITE ||
- io->ci_type == CIT_FAULT) {
- if (cio->cui_fd->fd_flags & LL_FILE_GROUP_LOCKED)
- result = -EBUSY;
- else {
- desc->cld_start = page->cp_index;
- desc->cld_end = page->cp_index;
- desc->cld_obj = page->cp_obj;
- desc->cld_mode = CLM_READ;
- result = cl_queue_match(&io->ci_lockset.cls_done,
- desc) ? -EBUSY : 0;
- }
- } else
- result = 0;
- return result;
-}
-
-int ccc_fail(const struct lu_env *env, const struct cl_page_slice *slice)
-{
- /*
- * Cached read?
- */
- LBUG();
- return 0;
-}
-
-int ccc_transient_page_prep(const struct lu_env *env,
- const struct cl_page_slice *slice,
- struct cl_io *unused)
-{
- /* transient page should always be sent. */
- return 0;
-}
-
-/*****************************************************************************
- *
- * Lock operations.
- *
- */
-
-void ccc_lock_delete(const struct lu_env *env,
- const struct cl_lock_slice *slice)
-{
- CLOBINVRNT(env, slice->cls_obj, ccc_object_invariant(slice->cls_obj));
-}
-
-void ccc_lock_fini(const struct lu_env *env, struct cl_lock_slice *slice)
-{
- struct ccc_lock *clk = cl2ccc_lock(slice);
-
- kmem_cache_free(ccc_lock_kmem, clk);
-}
-
-int ccc_lock_enqueue(const struct lu_env *env,
- const struct cl_lock_slice *slice,
- struct cl_io *unused, __u32 enqflags)
-{
- CLOBINVRNT(env, slice->cls_obj, ccc_object_invariant(slice->cls_obj));
- return 0;
-}
-
-int ccc_lock_use(const struct lu_env *env, const struct cl_lock_slice *slice)
-{
- CLOBINVRNT(env, slice->cls_obj, ccc_object_invariant(slice->cls_obj));
- return 0;
-}
-
-int ccc_lock_unuse(const struct lu_env *env, const struct cl_lock_slice *slice)
-{
- CLOBINVRNT(env, slice->cls_obj, ccc_object_invariant(slice->cls_obj));
- return 0;
-}
-
-int ccc_lock_wait(const struct lu_env *env, const struct cl_lock_slice *slice)
-{
- CLOBINVRNT(env, slice->cls_obj, ccc_object_invariant(slice->cls_obj));
- return 0;
-}
-
-/**
- * Implementation of cl_lock_operations::clo_fits_into() methods for ccc
- * layer. This function is executed every time io finds an existing lock in
- * the lock cache while creating new lock. This function has to decide whether
- * cached lock "fits" into io.
- *
- * \param slice lock to be checked
- * \param io IO that wants a lock.
- *
- * \see lov_lock_fits_into().
- */
-int ccc_lock_fits_into(const struct lu_env *env,
- const struct cl_lock_slice *slice,
- const struct cl_lock_descr *need,
- const struct cl_io *io)
-{
- const struct cl_lock *lock = slice->cls_lock;
- const struct cl_lock_descr *descr = &lock->cll_descr;
- const struct ccc_io *cio = ccc_env_io(env);
- int result;
-
- /*
- * Work around DLM peculiarity: it assumes that glimpse
- * (LDLM_FL_HAS_INTENT) lock is always LCK_PR, and returns reads lock
- * when asked for LCK_PW lock with LDLM_FL_HAS_INTENT flag set. Make
- * sure that glimpse doesn't get CLM_WRITE top-lock, so that it
- * doesn't enqueue CLM_WRITE sub-locks.
- */
- if (cio->cui_glimpse)
- result = descr->cld_mode != CLM_WRITE;
-
- /*
- * Also, don't match incomplete write locks for read, otherwise read
- * would enqueue missing sub-locks in the write mode.
- */
- else if (need->cld_mode != descr->cld_mode)
- result = lock->cll_state >= CLS_ENQUEUED;
- else
- result = 1;
- return result;
-}
-
-/**
- * Implements cl_lock_operations::clo_state() method for ccc layer, invoked
- * whenever lock state changes. Transfers object attributes, that might be
- * updated as a result of lock acquiring into inode.
- */
-void ccc_lock_state(const struct lu_env *env,
- const struct cl_lock_slice *slice,
- enum cl_lock_state state)
-{
- struct cl_lock *lock = slice->cls_lock;
-
- /*
- * Refresh inode attributes when the lock is moving into CLS_HELD
- * state, and only when this is a result of real enqueue, rather than
- * of finding lock in the cache.
- */
- if (state == CLS_HELD && lock->cll_state < CLS_HELD) {
- struct cl_object *obj;
- struct inode *inode;
-
- obj = slice->cls_obj;
- inode = ccc_object_inode(obj);
-
- /* vmtruncate() sets the i_size
- * under both a DLM lock and the
- * ll_inode_size_lock(). If we don't get the
- * ll_inode_size_lock() here we can match the DLM lock and
- * reset i_size. generic_file_write can then trust the
- * stale i_size when doing appending writes and effectively
- * cancel the result of the truncate. Getting the
- * ll_inode_size_lock() after the enqueue maintains the DLM
- * -> ll_inode_size_lock() acquiring order.
- */
- if (lock->cll_descr.cld_start == 0 &&
- lock->cll_descr.cld_end == CL_PAGE_EOF)
- cl_merge_lvb(env, inode);
- }
-}
-
-/*****************************************************************************
- *
- * io operations.
- *
- */
-
-int ccc_io_one_lock_index(const struct lu_env *env, struct cl_io *io,
- __u32 enqflags, enum cl_lock_mode mode,
- pgoff_t start, pgoff_t end)
-{
- struct ccc_io *cio = ccc_env_io(env);
- struct cl_lock_descr *descr = &cio->cui_link.cill_descr;
- struct cl_object *obj = io->ci_obj;
-
- CLOBINVRNT(env, obj, ccc_object_invariant(obj));
-
- CDEBUG(D_VFSTRACE, "lock: %d [%lu, %lu]\n", mode, start, end);
-
- memset(&cio->cui_link, 0, sizeof(cio->cui_link));
-
- if (cio->cui_fd && (cio->cui_fd->fd_flags & LL_FILE_GROUP_LOCKED)) {
- descr->cld_mode = CLM_GROUP;
- descr->cld_gid = cio->cui_fd->fd_grouplock.cg_gid;
- } else {
- descr->cld_mode = mode;
- }
- descr->cld_obj = obj;
- descr->cld_start = start;
- descr->cld_end = end;
- descr->cld_enq_flags = enqflags;
-
- cl_io_lock_add(env, io, &cio->cui_link);
- return 0;
-}
-
-void ccc_io_update_iov(const struct lu_env *env,
- struct ccc_io *cio, struct cl_io *io)
-{
- size_t size = io->u.ci_rw.crw_count;
-
- if (!cl_is_normalio(env, io) || !cio->cui_iter)
- return;
-
- iov_iter_truncate(cio->cui_iter, size);
-}
-
-int ccc_io_one_lock(const struct lu_env *env, struct cl_io *io,
- __u32 enqflags, enum cl_lock_mode mode,
- loff_t start, loff_t end)
-{
- struct cl_object *obj = io->ci_obj;
-
- return ccc_io_one_lock_index(env, io, enqflags, mode,
- cl_index(obj, start), cl_index(obj, end));
-}
-
-void ccc_io_end(const struct lu_env *env, const struct cl_io_slice *ios)
-{
- CLOBINVRNT(env, ios->cis_io->ci_obj,
- ccc_object_invariant(ios->cis_io->ci_obj));
-}
-
-void ccc_io_advance(const struct lu_env *env,
- const struct cl_io_slice *ios,
- size_t nob)
-{
- struct ccc_io *cio = cl2ccc_io(env, ios);
- struct cl_io *io = ios->cis_io;
- struct cl_object *obj = ios->cis_io->ci_obj;
-
- CLOBINVRNT(env, obj, ccc_object_invariant(obj));
-
- if (!cl_is_normalio(env, io))
- return;
-
- iov_iter_reexpand(cio->cui_iter, cio->cui_tot_count -= nob);
-}
-
-/**
- * Helper function that if necessary adjusts file size (inode->i_size), when
- * position at the offset \a pos is accessed. File size can be arbitrary stale
- * on a Lustre client, but client at least knows KMS. If accessed area is
- * inside [0, KMS], set file size to KMS, otherwise glimpse file size.
- *
- * Locking: cl_isize_lock is used to serialize changes to inode size and to
- * protect consistency between inode size and cl_object
- * attributes. cl_object_size_lock() protects consistency between cl_attr's of
- * top-object and sub-objects.
- */
-int ccc_prep_size(const struct lu_env *env, struct cl_object *obj,
- struct cl_io *io, loff_t start, size_t count, int *exceed)
-{
- struct cl_attr *attr = ccc_env_thread_attr(env);
- struct inode *inode = ccc_object_inode(obj);
- loff_t pos = start + count - 1;
- loff_t kms;
- int result;
-
- /*
- * Consistency guarantees: following possibilities exist for the
- * relation between region being accessed and real file size at this
- * moment:
- *
- * (A): the region is completely inside of the file;
- *
- * (B-x): x bytes of region are inside of the file, the rest is
- * outside;
- *
- * (C): the region is completely outside of the file.
- *
- * This classification is stable under DLM lock already acquired by
- * the caller, because to change the class, other client has to take
- * DLM lock conflicting with our lock. Also, any updates to ->i_size
- * by other threads on this client are serialized by
- * ll_inode_size_lock(). This guarantees that short reads are handled
- * correctly in the face of concurrent writes and truncates.
- */
- ccc_object_size_lock(obj);
- result = cl_object_attr_get(env, obj, attr);
- if (result == 0) {
- kms = attr->cat_kms;
- if (pos > kms) {
- /*
- * A glimpse is necessary to determine whether we
- * return a short read (B) or some zeroes at the end
- * of the buffer (C)
- */
- ccc_object_size_unlock(obj);
- result = cl_glimpse_lock(env, io, inode, obj, 0);
- if (result == 0 && exceed) {
- /* If objective page index exceed end-of-file
- * page index, return directly. Do not expect
- * kernel will check such case correctly.
- * linux-2.6.18-128.1.1 miss to do that.
- * --bug 17336
- */
- loff_t size = cl_isize_read(inode);
- loff_t cur_index = start >> PAGE_SHIFT;
- loff_t size_index = (size - 1) >>
- PAGE_SHIFT;
-
- if ((size == 0 && cur_index != 0) ||
- size_index < cur_index)
- *exceed = 1;
- }
- return result;
- }
- /*
- * region is within kms and, hence, within real file
- * size (A). We need to increase i_size to cover the
- * read region so that generic_file_read() will do its
- * job, but that doesn't mean the kms size is
- * _correct_, it is only the _minimum_ size. If
- * someone does a stat they will get the correct size
- * which will always be >= the kms value here.
- * b=11081
- */
- if (cl_isize_read(inode) < kms) {
- cl_isize_write_nolock(inode, kms);
- CDEBUG(D_VFSTRACE,
- DFID" updating i_size %llu\n",
- PFID(lu_object_fid(&obj->co_lu)),
- (__u64)cl_isize_read(inode));
-
- }
- }
- ccc_object_size_unlock(obj);
- return result;
-}
-
-/*****************************************************************************
- *
- * Transfer operations.
- *
- */
-
-void ccc_req_completion(const struct lu_env *env,
- const struct cl_req_slice *slice, int ioret)
-{
- struct ccc_req *vrq;
-
- if (ioret > 0)
- cl_stats_tally(slice->crs_dev, slice->crs_req->crq_type, ioret);
-
- vrq = cl2ccc_req(slice);
- kmem_cache_free(ccc_req_kmem, vrq);
-}
-
-/**
- * Implementation of struct cl_req_operations::cro_attr_set() for ccc
- * layer. ccc is responsible for
- *
- * - o_[mac]time
- *
- * - o_mode
- *
- * - o_parent_seq
- *
- * - o_[ug]id
- *
- * - o_parent_oid
- *
- * - o_parent_ver
- *
- * - o_ioepoch,
- *
- */
-void ccc_req_attr_set(const struct lu_env *env,
- const struct cl_req_slice *slice,
- const struct cl_object *obj,
- struct cl_req_attr *attr, u64 flags)
-{
- struct inode *inode;
- struct obdo *oa;
- u32 valid_flags;
-
- oa = attr->cra_oa;
- inode = ccc_object_inode(obj);
- valid_flags = OBD_MD_FLTYPE;
-
- if (slice->crs_req->crq_type == CRT_WRITE) {
- if (flags & OBD_MD_FLEPOCH) {
- oa->o_valid |= OBD_MD_FLEPOCH;
- oa->o_ioepoch = cl_i2info(inode)->lli_ioepoch;
- valid_flags |= OBD_MD_FLMTIME | OBD_MD_FLCTIME |
- OBD_MD_FLUID | OBD_MD_FLGID;
- }
- }
- obdo_from_inode(oa, inode, valid_flags & flags);
- obdo_set_parent_fid(oa, &cl_i2info(inode)->lli_fid);
- memcpy(attr->cra_jobid, cl_i2info(inode)->lli_jobid,
- JOBSTATS_JOBID_SIZE);
-}
-
-static const struct cl_req_operations ccc_req_ops = {
- .cro_attr_set = ccc_req_attr_set,
- .cro_completion = ccc_req_completion
-};
-
-int cl_setattr_ost(struct inode *inode, const struct iattr *attr)
-{
- struct lu_env *env;
- struct cl_io *io;
- int result;
- int refcheck;
-
- env = cl_env_get(&refcheck);
- if (IS_ERR(env))
- return PTR_ERR(env);
-
- io = ccc_env_thread_io(env);
- io->ci_obj = cl_i2info(inode)->lli_clob;
-
- io->u.ci_setattr.sa_attr.lvb_atime = LTIME_S(attr->ia_atime);
- io->u.ci_setattr.sa_attr.lvb_mtime = LTIME_S(attr->ia_mtime);
- io->u.ci_setattr.sa_attr.lvb_ctime = LTIME_S(attr->ia_ctime);
- io->u.ci_setattr.sa_attr.lvb_size = attr->ia_size;
- io->u.ci_setattr.sa_valid = attr->ia_valid;
-
-again:
- if (cl_io_init(env, io, CIT_SETATTR, io->ci_obj) == 0) {
- struct ccc_io *cio = ccc_env_io(env);
-
- if (attr->ia_valid & ATTR_FILE)
- /* populate the file descriptor for ftruncate to honor
- * group lock - see LU-787
- */
- cio->cui_fd = cl_iattr2fd(inode, attr);
-
- result = cl_io_loop(env, io);
- } else {
- result = io->ci_result;
- }
- cl_io_fini(env, io);
- if (unlikely(io->ci_need_restart))
- goto again;
- /* HSM import case: file is released, cannot be restored
- * no need to fail except if restore registration failed
- * with -ENODATA
- */
- if (result == -ENODATA && io->ci_restore_needed &&
- io->ci_result != -ENODATA)
- result = 0;
- cl_env_put(env, &refcheck);
- return result;
-}
-
-/*****************************************************************************
- *
- * Type conversions.
- *
- */
-
-struct lu_device *ccc2lu_dev(struct ccc_device *vdv)
-{
- return &vdv->cdv_cl.cd_lu_dev;
-}
-
-struct ccc_device *lu2ccc_dev(const struct lu_device *d)
-{
- return container_of0(d, struct ccc_device, cdv_cl.cd_lu_dev);
-}
-
-struct ccc_device *cl2ccc_dev(const struct cl_device *d)
-{
- return container_of0(d, struct ccc_device, cdv_cl);
-}
-
-struct lu_object *ccc2lu(struct ccc_object *vob)
-{
- return &vob->cob_cl.co_lu;
-}
-
-struct ccc_object *lu2ccc(const struct lu_object *obj)
-{
- return container_of0(obj, struct ccc_object, cob_cl.co_lu);
-}
-
-struct ccc_object *cl2ccc(const struct cl_object *obj)
-{
- return container_of0(obj, struct ccc_object, cob_cl);
-}
-
-struct ccc_lock *cl2ccc_lock(const struct cl_lock_slice *slice)
-{
- return container_of(slice, struct ccc_lock, clk_cl);
-}
-
-struct ccc_io *cl2ccc_io(const struct lu_env *env,
- const struct cl_io_slice *slice)
-{
- struct ccc_io *cio;
-
- cio = container_of(slice, struct ccc_io, cui_cl);
- LASSERT(cio == ccc_env_io(env));
- return cio;
-}
-
-struct ccc_req *cl2ccc_req(const struct cl_req_slice *slice)
-{
- return container_of0(slice, struct ccc_req, crq_cl);
-}
-
-struct page *cl2vm_page(const struct cl_page_slice *slice)
-{
- return cl2ccc_page(slice)->cpg_page;
-}
-
-/*****************************************************************************
- *
- * Accessors.
- *
- */
-int ccc_object_invariant(const struct cl_object *obj)
-{
- struct inode *inode = ccc_object_inode(obj);
- struct cl_inode_info *lli = cl_i2info(inode);
-
- return (S_ISREG(cl_inode_mode(inode)) ||
- /* i_mode of unlinked inode is zeroed. */
- cl_inode_mode(inode) == 0) && lli->lli_clob == obj;
-}
-
-struct inode *ccc_object_inode(const struct cl_object *obj)
-{
- return cl2ccc(obj)->cob_inode;
-}
-
-/**
- * Initialize or update CLIO structures for regular files when new
- * meta-data arrives from the server.
- *
- * \param inode regular file inode
- * \param md new file metadata from MDS
- * - allocates cl_object if necessary,
- * - updated layout, if object was already here.
- */
-int cl_file_inode_init(struct inode *inode, struct lustre_md *md)
-{
- struct lu_env *env;
- struct cl_inode_info *lli;
- struct cl_object *clob;
- struct lu_site *site;
- struct lu_fid *fid;
- struct cl_object_conf conf = {
- .coc_inode = inode,
- .u = {
- .coc_md = md
- }
- };
- int result = 0;
- int refcheck;
-
- LASSERT(md->body->valid & OBD_MD_FLID);
- LASSERT(S_ISREG(cl_inode_mode(inode)));
-
- env = cl_env_get(&refcheck);
- if (IS_ERR(env))
- return PTR_ERR(env);
-
- site = cl_i2sbi(inode)->ll_site;
- lli = cl_i2info(inode);
- fid = &lli->lli_fid;
- LASSERT(fid_is_sane(fid));
-
- if (!lli->lli_clob) {
- /* clob is slave of inode, empty lli_clob means for new inode,
- * there is no clob in cache with the given fid, so it is
- * unnecessary to perform lookup-alloc-lookup-insert, just
- * alloc and insert directly.
- */
- LASSERT(inode->i_state & I_NEW);
- conf.coc_lu.loc_flags = LOC_F_NEW;
- clob = cl_object_find(env, lu2cl_dev(site->ls_top_dev),
- fid, &conf);
- if (!IS_ERR(clob)) {
- /*
- * No locking is necessary, as new inode is
- * locked by I_NEW bit.
- */
- lli->lli_clob = clob;
- lli->lli_has_smd = lsm_has_objects(md->lsm);
- lu_object_ref_add(&clob->co_lu, "inode", inode);
- } else
- result = PTR_ERR(clob);
- } else {
- result = cl_conf_set(env, lli->lli_clob, &conf);
- }
-
- cl_env_put(env, &refcheck);
-
- if (result != 0)
- CERROR("Failure to initialize cl object "DFID": %d\n",
- PFID(fid), result);
- return result;
-}
-
-/**
- * Wait for others drop their references of the object at first, then we drop
- * the last one, which will lead to the object be destroyed immediately.
- * Must be called after cl_object_kill() against this object.
- *
- * The reason we want to do this is: destroying top object will wait for sub
- * objects being destroyed first, so we can't let bottom layer (e.g. from ASTs)
- * to initiate top object destroying which may deadlock. See bz22520.
- */
-static void cl_object_put_last(struct lu_env *env, struct cl_object *obj)
-{
- struct lu_object_header *header = obj->co_lu.lo_header;
- wait_queue_t waiter;
-
- if (unlikely(atomic_read(&header->loh_ref) != 1)) {
- struct lu_site *site = obj->co_lu.lo_dev->ld_site;
- struct lu_site_bkt_data *bkt;
-
- bkt = lu_site_bkt_from_fid(site, &header->loh_fid);
-
- init_waitqueue_entry(&waiter, current);
- add_wait_queue(&bkt->lsb_marche_funebre, &waiter);
-
- while (1) {
- set_current_state(TASK_UNINTERRUPTIBLE);
- if (atomic_read(&header->loh_ref) == 1)
- break;
- schedule();
- }
-
- set_current_state(TASK_RUNNING);
- remove_wait_queue(&bkt->lsb_marche_funebre, &waiter);
- }
-
- cl_object_put(env, obj);
-}
-
-void cl_inode_fini(struct inode *inode)
-{
- struct lu_env *env;
- struct cl_inode_info *lli = cl_i2info(inode);
- struct cl_object *clob = lli->lli_clob;
- int refcheck;
- int emergency;
-
- if (clob) {
- void *cookie;
-
- cookie = cl_env_reenter();
- env = cl_env_get(&refcheck);
- emergency = IS_ERR(env);
- if (emergency) {
- mutex_lock(&ccc_inode_fini_guard);
- LASSERT(ccc_inode_fini_env);
- cl_env_implant(ccc_inode_fini_env, &refcheck);
- env = ccc_inode_fini_env;
- }
- /*
- * cl_object cache is a slave to inode cache (which, in turn
- * is a slave to dentry cache), don't keep cl_object in memory
- * when its master is evicted.
- */
- cl_object_kill(env, clob);
- lu_object_ref_del(&clob->co_lu, "inode", inode);
- cl_object_put_last(env, clob);
- lli->lli_clob = NULL;
- if (emergency) {
- cl_env_unplant(ccc_inode_fini_env, &refcheck);
- mutex_unlock(&ccc_inode_fini_guard);
- } else
- cl_env_put(env, &refcheck);
- cl_env_reexit(cookie);
- }
-}
-
-/**
- * return IF_* type for given lu_dirent entry.
- * IF_* flag shld be converted to particular OS file type in
- * platform llite module.
- */
-__u16 ll_dirent_type_get(struct lu_dirent *ent)
-{
- __u16 type = 0;
- struct luda_type *lt;
- int len = 0;
-
- if (le32_to_cpu(ent->lde_attrs) & LUDA_TYPE) {
- const unsigned align = sizeof(struct luda_type) - 1;
-
- len = le16_to_cpu(ent->lde_namelen);
- len = (len + align) & ~align;
- lt = (void *)ent->lde_name + len;
- type = IFTODT(le16_to_cpu(lt->lt_type));
- }
- return type;
-}
-
-/**
- * build inode number from passed @fid
- */
-__u64 cl_fid_build_ino(const struct lu_fid *fid, int api32)
-{
- if (BITS_PER_LONG == 32 || api32)
- return fid_flatten32(fid);
- else
- return fid_flatten(fid);
-}
-
-/**
- * build inode generation from passed @fid. If our FID overflows the 32-bit
- * inode number then return a non-zero generation to distinguish them.
- */
-__u32 cl_fid_build_gen(const struct lu_fid *fid)
-{
- __u32 gen;
-
- if (fid_is_igif(fid)) {
- gen = lu_igif_gen(fid);
- return gen;
- }
-
- gen = fid_flatten(fid) >> 32;
- return gen;
-}
-
-/* lsm is unreliable after hsm implementation as layout can be changed at
- * any time. This is only to support old, non-clio-ized interfaces. It will
- * cause deadlock if clio operations are called with this extra layout refcount
- * because in case the layout changed during the IO, ll_layout_refresh() will
- * have to wait for the refcount to become zero to destroy the older layout.
- *
- * Notice that the lsm returned by this function may not be valid unless called
- * inside layout lock - MDS_INODELOCK_LAYOUT.
- */
-struct lov_stripe_md *ccc_inode_lsm_get(struct inode *inode)
-{
- return lov_lsm_get(cl_i2info(inode)->lli_clob);
-}
-
-inline void ccc_inode_lsm_put(struct inode *inode, struct lov_stripe_md *lsm)
-{
- lov_lsm_put(cl_i2info(inode)->lli_clob, lsm);
-}
diff --git a/drivers/staging/lustre/lustre/lclient/lcommon_misc.c b/drivers/staging/lustre/lustre/lclient/lcommon_misc.c
deleted file mode 100644
index d80bcedd..0000000
--- a/drivers/staging/lustre/lustre/lclient/lcommon_misc.c
+++ /dev/null
@@ -1,200 +0,0 @@
-/*
- * GPL HEADER START
- *
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 only,
- * as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License version 2 for more details (a copy is included
- * in the LICENSE file that accompanied this code).
- *
- * You should have received a copy of the GNU General Public License
- * version 2 along with this program; If not, see
- * http://www.sun.com/software/products/lustre/docs/GPLv2.pdf
- *
- * Please contact Sun Microsystems, Inc., 4150 Network Circle, Santa Clara,
- * CA 95054 USA or visit www.sun.com if you need additional information or
- * have any questions.
- *
- * GPL HEADER END
- */
-/*
- * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.
- * Use is subject to license terms.
- *
- * Copyright (c) 2011, 2012, Intel Corporation.
- */
-/*
- * This file is part of Lustre, http://www.lustre.org/
- * Lustre is a trademark of Sun Microsystems, Inc.
- *
- * cl code shared between vvp and liblustre (and other Lustre clients in the
- * future).
- *
- */
-#include "../include/obd_class.h"
-#include "../include/obd_support.h"
-#include "../include/obd.h"
-#include "../include/cl_object.h"
-#include "../include/lclient.h"
-
-#include "../include/lustre_lite.h"
-
-/* Initialize the default and maximum LOV EA and cookie sizes. This allows
- * us to make MDS RPCs with large enough reply buffers to hold the
- * maximum-sized (= maximum striped) EA and cookie without having to
- * calculate this (via a call into the LOV + OSCs) each time we make an RPC.
- */
-int cl_init_ea_size(struct obd_export *md_exp, struct obd_export *dt_exp)
-{
- struct lov_stripe_md lsm = { .lsm_magic = LOV_MAGIC_V3 };
- __u32 valsize = sizeof(struct lov_desc);
- int rc, easize, def_easize, cookiesize;
- struct lov_desc desc;
- __u16 stripes, def_stripes;
-
- rc = obd_get_info(NULL, dt_exp, sizeof(KEY_LOVDESC), KEY_LOVDESC,
- &valsize, &desc, NULL);
- if (rc)
- return rc;
-
- stripes = min_t(__u32, desc.ld_tgt_count, LOV_MAX_STRIPE_COUNT);
- lsm.lsm_stripe_count = stripes;
- easize = obd_size_diskmd(dt_exp, &lsm);
-
- def_stripes = min_t(__u32, desc.ld_default_stripe_count,
- LOV_MAX_STRIPE_COUNT);
- lsm.lsm_stripe_count = def_stripes;
- def_easize = obd_size_diskmd(dt_exp, &lsm);
-
- cookiesize = stripes * sizeof(struct llog_cookie);
-
- /* default cookiesize is 0 because from 2.4 server doesn't send
- * llog cookies to client.
- */
- CDEBUG(D_HA,
- "updating def/max_easize: %d/%d def/max_cookiesize: 0/%d\n",
- def_easize, easize, cookiesize);
-
- rc = md_init_ea_size(md_exp, easize, def_easize, cookiesize, 0);
- return rc;
-}
-
-/**
- * This function is used as an upcall-callback hooked by liblustre and llite
- * clients into obd_notify() listeners chain to handle notifications about
- * change of import connect_flags. See llu_fsswop_mount() and
- * lustre_common_fill_super().
- */
-int cl_ocd_update(struct obd_device *host,
- struct obd_device *watched,
- enum obd_notify_event ev, void *owner, void *data)
-{
- struct lustre_client_ocd *lco;
- struct client_obd *cli;
- __u64 flags;
- int result;
-
- if (!strcmp(watched->obd_type->typ_name, LUSTRE_OSC_NAME)) {
- cli = &watched->u.cli;
- lco = owner;
- flags = cli->cl_import->imp_connect_data.ocd_connect_flags;
- CDEBUG(D_SUPER, "Changing connect_flags: %#llx -> %#llx\n",
- lco->lco_flags, flags);
- mutex_lock(&lco->lco_lock);
- lco->lco_flags &= flags;
- /* for each osc event update ea size */
- if (lco->lco_dt_exp)
- cl_init_ea_size(lco->lco_md_exp, lco->lco_dt_exp);
-
- mutex_unlock(&lco->lco_lock);
- result = 0;
- } else {
- CERROR("unexpected notification from %s %s!\n",
- watched->obd_type->typ_name,
- watched->obd_name);
- result = -EINVAL;
- }
- return result;
-}
-
-#define GROUPLOCK_SCOPE "grouplock"
-
-int cl_get_grouplock(struct cl_object *obj, unsigned long gid, int nonblock,
- struct ccc_grouplock *cg)
-{
- struct lu_env *env;
- struct cl_io *io;
- struct cl_lock *lock;
- struct cl_lock_descr *descr;
- __u32 enqflags;
- int refcheck;
- int rc;
-
- env = cl_env_get(&refcheck);
- if (IS_ERR(env))
- return PTR_ERR(env);
-
- io = ccc_env_thread_io(env);
- io->ci_obj = obj;
- io->ci_ignore_layout = 1;
-
- rc = cl_io_init(env, io, CIT_MISC, io->ci_obj);
- if (rc) {
- /* Does not make sense to take GL for released layout */
- if (rc > 0)
- rc = -ENOTSUPP;
- cl_env_put(env, &refcheck);
- return rc;
- }
-
- descr = &ccc_env_info(env)->cti_descr;
- descr->cld_obj = obj;
- descr->cld_start = 0;
- descr->cld_end = CL_PAGE_EOF;
- descr->cld_gid = gid;
- descr->cld_mode = CLM_GROUP;
-
- enqflags = CEF_MUST | (nonblock ? CEF_NONBLOCK : 0);
- descr->cld_enq_flags = enqflags;
-
- lock = cl_lock_request(env, io, descr, GROUPLOCK_SCOPE, current);
- if (IS_ERR(lock)) {
- cl_io_fini(env, io);
- cl_env_put(env, &refcheck);
- return PTR_ERR(lock);
- }
-
- cg->cg_env = cl_env_get(&refcheck);
- cg->cg_io = io;
- cg->cg_lock = lock;
- cg->cg_gid = gid;
- LASSERT(cg->cg_env == env);
-
- cl_env_unplant(env, &refcheck);
- return 0;
-}
-
-void cl_put_grouplock(struct ccc_grouplock *cg)
-{
- struct lu_env *env = cg->cg_env;
- struct cl_io *io = cg->cg_io;
- struct cl_lock *lock = cg->cg_lock;
- int refcheck;
-
- LASSERT(cg->cg_env);
- LASSERT(cg->cg_gid);
-
- cl_env_implant(env, &refcheck);
- cl_env_put(env, &refcheck);
-
- cl_unuse(env, lock);
- cl_lock_release(env, lock, GROUPLOCK_SCOPE, current);
- cl_io_fini(env, io);
- cl_env_put(env, NULL);
-}
diff --git a/drivers/staging/lustre/lustre/ldlm/l_lock.c b/drivers/staging/lustre/lustre/ldlm/l_lock.c
index e5d1344..621323f 100644
--- a/drivers/staging/lustre/lustre/ldlm/l_lock.c
+++ b/drivers/staging/lustre/lustre/ldlm/l_lock.c
@@ -54,7 +54,7 @@
lock_res(lock->l_resource);
- lock->l_flags |= LDLM_FL_RES_LOCKED;
+ ldlm_set_res_locked(lock);
return lock->l_resource;
}
EXPORT_SYMBOL(lock_res_and_lock);
@@ -65,7 +65,7 @@
void unlock_res_and_lock(struct ldlm_lock *lock)
{
/* on server-side resource of lock doesn't change */
- lock->l_flags &= ~LDLM_FL_RES_LOCKED;
+ ldlm_clear_res_locked(lock);
unlock_res(lock->l_resource);
spin_unlock(&lock->l_lock);
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
index a803e20..cf1f178 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
@@ -75,12 +75,12 @@
* just after we finish and take our lock into account in its
* calculation of the kms
*/
- lock->l_flags |= LDLM_FL_KMS_IGNORE;
+ ldlm_set_kms_ignore(lock);
list_for_each(tmp, &res->lr_granted) {
lck = list_entry(tmp, struct ldlm_lock, l_res_link);
- if (lck->l_flags & LDLM_FL_KMS_IGNORE)
+ if (ldlm_is_kms_ignore(lck))
continue;
if (lck->l_policy_data.l_extent.end >= old_kms)
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c
index b88b786..349bfcc 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c
@@ -101,8 +101,7 @@
LASSERT(hlist_unhashed(&lock->l_exp_flock_hash));
list_del_init(&lock->l_res_link);
- if (flags == LDLM_FL_WAIT_NOREPROC &&
- !(lock->l_flags & LDLM_FL_FAILED)) {
+ if (flags == LDLM_FL_WAIT_NOREPROC && !ldlm_is_failed(lock)) {
/* client side - set a flag to prevent sending a CANCEL */
lock->l_flags |= LDLM_FL_LOCAL_ONLY | LDLM_FL_CBPENDING;
@@ -436,7 +435,7 @@
lock_res_and_lock(lock);
/* client side - set flag to prevent lock from being put on LRU list */
- lock->l_flags |= LDLM_FL_CBPENDING;
+ ldlm_set_cbpending(lock);
unlock_res_and_lock(lock);
}
@@ -520,30 +519,29 @@
granted:
OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT, 10);
- if (lock->l_flags & LDLM_FL_DESTROYED) {
- LDLM_DEBUG(lock, "client-side enqueue waking up: destroyed");
- return 0;
- }
-
- if (lock->l_flags & LDLM_FL_FAILED) {
+ if (ldlm_is_failed(lock)) {
LDLM_DEBUG(lock, "client-side enqueue waking up: failed");
return -EIO;
}
- if (rc) {
- LDLM_DEBUG(lock, "client-side enqueue waking up: failed (%d)",
- rc);
- return rc;
- }
-
LDLM_DEBUG(lock, "client-side enqueue granted");
lock_res_and_lock(lock);
+ /*
+ * Protect against race where lock could have been just destroyed
+ * due to overlap in ldlm_process_flock_lock().
+ */
+ if (ldlm_is_destroyed(lock)) {
+ unlock_res_and_lock(lock);
+ LDLM_DEBUG(lock, "client-side enqueue waking up: destroyed");
+ return 0;
+ }
+
/* ldlm_lock_enqueue() has already placed lock on the granted list. */
list_del_init(&lock->l_res_link);
- if (lock->l_flags & LDLM_FL_FLOCK_DEADLOCK) {
+ if (ldlm_is_flock_deadlock(lock)) {
LDLM_DEBUG(lock, "client-side enqueue deadlock received");
rc = -EDEADLK;
} else if (flags & LDLM_FL_TEST_LOCK) {
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
index e21373e..32f227f 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h
@@ -95,9 +95,10 @@
LDLM_CANCEL_PASSED = 1 << 1, /* Cancel passed number of locks. */
LDLM_CANCEL_SHRINK = 1 << 2, /* Cancel locks from shrinker. */
LDLM_CANCEL_LRUR = 1 << 3, /* Cancel locks from lru resize. */
- LDLM_CANCEL_NO_WAIT = 1 << 4 /* Cancel locks w/o blocking (neither
- * sending nor waiting for any rpcs)
- */
+ LDLM_CANCEL_NO_WAIT = 1 << 4, /* Cancel locks w/o blocking (neither
+ * sending nor waiting for any rpcs)
+ */
+ LDLM_CANCEL_LRUR_NO_WAIT = 1 << 5, /* LRUR + NO_WAIT */
};
int ldlm_cancel_lru(struct ldlm_namespace *ns, int nr,
@@ -145,7 +146,8 @@
void ldlm_lock_decref_internal_nolock(struct ldlm_lock *, __u32 mode);
int ldlm_run_ast_work(struct ldlm_namespace *ns, struct list_head *rpc_list,
enum ldlm_desc_ast_t ast_type);
-int ldlm_lock_remove_from_lru(struct ldlm_lock *lock);
+int ldlm_lock_remove_from_lru_check(struct ldlm_lock *lock, time_t last_use);
+#define ldlm_lock_remove_from_lru(lock) ldlm_lock_remove_from_lru_check(lock, 0)
int ldlm_lock_remove_from_lru_nolock(struct ldlm_lock *lock);
void ldlm_lock_destroy_nolock(struct ldlm_lock *lock);
@@ -216,8 +218,6 @@
LDLM_POLICY_SKIP_LOCK
};
-typedef enum ldlm_policy_res ldlm_policy_res_t;
-
#define LDLM_POOL_SYSFS_PRINT_int(v) sprintf(buf, "%d\n", v)
#define LDLM_POOL_SYSFS_SET_int(a, b) { a = b; }
#define LDLM_POOL_SYSFS_PRINT_u64(v) sprintf(buf, "%lld\n", v)
@@ -305,9 +305,10 @@
int ret = 0;
lock_res_and_lock(lock);
- if (((lock->l_req_mode == lock->l_granted_mode) &&
- !(lock->l_flags & LDLM_FL_CP_REQD)) ||
- (lock->l_flags & (LDLM_FL_FAILED | LDLM_FL_CANCEL)))
+ if ((lock->l_req_mode == lock->l_granted_mode) &&
+ !ldlm_is_cp_reqd(lock))
+ ret = 1;
+ else if (ldlm_is_failed(lock) || ldlm_is_cancel(lock))
ret = 1;
unlock_res_and_lock(lock);
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
index 7dd7df5..b4ffbe2 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
@@ -314,7 +314,7 @@
INIT_LIST_HEAD(&cli->cl_loi_hp_ready_list);
INIT_LIST_HEAD(&cli->cl_loi_write_list);
INIT_LIST_HEAD(&cli->cl_loi_read_list);
- client_obd_list_lock_init(&cli->cl_loi_list_lock);
+ spin_lock_init(&cli->cl_loi_list_lock);
atomic_set(&cli->cl_pending_w_pages, 0);
atomic_set(&cli->cl_pending_r_pages, 0);
cli->cl_r_in_flight = 0;
@@ -333,7 +333,8 @@
atomic_set(&cli->cl_lru_busy, 0);
atomic_set(&cli->cl_lru_in_list, 0);
INIT_LIST_HEAD(&cli->cl_lru_list);
- client_obd_list_lock_init(&cli->cl_lru_list_lock);
+ spin_lock_init(&cli->cl_lru_list_lock);
+ atomic_set(&cli->cl_unstable_count, 0);
init_waitqueue_head(&cli->cl_destroy_waitq);
atomic_set(&cli->cl_destroy_in_flight, 0);
@@ -355,6 +356,12 @@
cli->cl_max_pages_per_rpc = min_t(int, PTLRPC_MAX_BRW_PAGES,
LNET_MTU >> PAGE_SHIFT);
+ /*
+ * set cl_chunkbits default value to PAGE_CACHE_SHIFT,
+ * it will be updated at OSC connection time.
+ */
+ cli->cl_chunkbits = PAGE_SHIFT;
+
if (!strcmp(name, LUSTRE_MDC_NAME)) {
cli->cl_max_rpcs_in_flight = MDC_MAX_RIF_DEFAULT;
} else if (totalram_pages >> (20 - PAGE_SHIFT) <= 128 /* MB */) {
@@ -429,7 +436,6 @@
ldlm_put_ref();
err:
return rc;
-
}
EXPORT_SYMBOL(client_obd_setup);
@@ -438,6 +444,7 @@
ldlm_namespace_free_post(obddev->obd_namespace);
obddev->obd_namespace = NULL;
+ obd_cleanup_client_import(obddev);
LASSERT(!obddev->u.cli.cl_import);
ldlm_put_ref();
@@ -748,6 +755,7 @@
switch (error) {
case ELDLM_OK:
+ case ELDLM_LOCK_MATCHED:
result = 0;
break;
case ELDLM_LOCK_CHANGED:
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
index ecd65a7..bff94ea 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
@@ -185,7 +185,7 @@
"final lock_put on destroyed lock, freeing it.");
res = lock->l_resource;
- LASSERT(lock->l_flags & LDLM_FL_DESTROYED);
+ LASSERT(ldlm_is_destroyed(lock));
LASSERT(list_empty(&lock->l_res_link));
LASSERT(list_empty(&lock->l_pending_chain));
@@ -229,15 +229,25 @@
/**
* Removes LDLM lock \a lock from LRU. Obtains the LRU lock first.
+ *
+ * If \a last_use is non-zero, it will remove the lock from LRU only if
+ * it matches lock's l_last_used.
+ *
+ * \retval 0 if \a last_use is set, the lock is not in LRU list or \a last_use
+ * doesn't match lock's l_last_used;
+ * otherwise, the lock hasn't been in the LRU list.
+ * \retval 1 the lock was in LRU list and removed.
*/
-int ldlm_lock_remove_from_lru(struct ldlm_lock *lock)
+int ldlm_lock_remove_from_lru_check(struct ldlm_lock *lock, time_t last_use)
{
struct ldlm_namespace *ns = ldlm_lock_to_ns(lock);
- int rc;
+ int rc = 0;
spin_lock(&ns->ns_lock);
- rc = ldlm_lock_remove_from_lru_nolock(lock);
+ if (last_use == 0 || last_use == lock->l_last_used)
+ rc = ldlm_lock_remove_from_lru_nolock(lock);
spin_unlock(&ns->ns_lock);
+
return rc;
}
@@ -252,8 +262,7 @@
LASSERT(list_empty(&lock->l_lru));
LASSERT(lock->l_resource->lr_type != LDLM_FLOCK);
list_add_tail(&lock->l_lru, &ns->ns_unused_list);
- if (lock->l_flags & LDLM_FL_SKIPPED)
- lock->l_flags &= ~LDLM_FL_SKIPPED;
+ ldlm_clear_skipped(lock);
LASSERT(ns->ns_nr_unused >= 0);
ns->ns_nr_unused++;
}
@@ -318,11 +327,11 @@
LBUG();
}
- if (lock->l_flags & LDLM_FL_DESTROYED) {
+ if (ldlm_is_destroyed(lock)) {
LASSERT(list_empty(&lock->l_lru));
return 0;
}
- lock->l_flags |= LDLM_FL_DESTROYED;
+ ldlm_set_destroyed(lock);
if (lock->l_export && lock->l_export->exp_lock_hash) {
/* NB: it's safe to call cfs_hash_del() even lock isn't
@@ -544,7 +553,7 @@
/* It's unlikely but possible that someone marked the lock as
* destroyed after we did handle2object on it
*/
- if (flags == 0 && ((lock->l_flags & LDLM_FL_DESTROYED) == 0)) {
+ if (flags == 0 && !ldlm_is_destroyed(lock)) {
lu_ref_add(&lock->l_reference, "handle", current);
return lock;
}
@@ -554,21 +563,22 @@
LASSERT(lock->l_resource);
lu_ref_add_atomic(&lock->l_reference, "handle", current);
- if (unlikely(lock->l_flags & LDLM_FL_DESTROYED)) {
+ if (unlikely(ldlm_is_destroyed(lock))) {
unlock_res_and_lock(lock);
CDEBUG(D_INFO, "lock already destroyed: lock %p\n", lock);
LDLM_LOCK_PUT(lock);
return NULL;
}
- if (flags && (lock->l_flags & flags)) {
- unlock_res_and_lock(lock);
- LDLM_LOCK_PUT(lock);
- return NULL;
- }
+ if (flags) {
+ if (lock->l_flags & flags) {
+ unlock_res_and_lock(lock);
+ LDLM_LOCK_PUT(lock);
+ return NULL;
+ }
- if (flags)
lock->l_flags |= flags;
+ }
unlock_res_and_lock(lock);
return lock;
@@ -599,14 +609,14 @@
static void ldlm_add_bl_work_item(struct ldlm_lock *lock, struct ldlm_lock *new,
struct list_head *work_list)
{
- if ((lock->l_flags & LDLM_FL_AST_SENT) == 0) {
+ if (!ldlm_is_ast_sent(lock)) {
LDLM_DEBUG(lock, "lock incompatible; sending blocking AST.");
- lock->l_flags |= LDLM_FL_AST_SENT;
+ ldlm_set_ast_sent(lock);
/* If the enqueuing client said so, tell the AST recipient to
* discard dirty data, rather than writing back.
*/
- if (new->l_flags & LDLM_FL_AST_DISCARD_DATA)
- lock->l_flags |= LDLM_FL_DISCARD_DATA;
+ if (ldlm_is_ast_discard_data(new))
+ ldlm_set_discard_data(lock);
LASSERT(list_empty(&lock->l_bl_ast));
list_add(&lock->l_bl_ast, work_list);
LDLM_LOCK_GET(lock);
@@ -621,8 +631,8 @@
static void ldlm_add_cp_work_item(struct ldlm_lock *lock,
struct list_head *work_list)
{
- if ((lock->l_flags & LDLM_FL_CP_REQD) == 0) {
- lock->l_flags |= LDLM_FL_CP_REQD;
+ if (!ldlm_is_cp_reqd(lock)) {
+ ldlm_set_cp_reqd(lock);
LDLM_DEBUG(lock, "lock granted; sending completion AST.");
LASSERT(list_empty(&lock->l_cp_ast));
list_add(&lock->l_cp_ast, work_list);
@@ -657,7 +667,7 @@
struct ldlm_lock *lock;
lock = ldlm_handle2lock(lockh);
- LASSERT(lock);
+ LASSERTF(lock, "Non-existing lock: %llx\n", lockh->cookie);
ldlm_lock_addref_internal(lock, mode);
LDLM_LOCK_PUT(lock);
}
@@ -704,7 +714,7 @@
if (lock) {
lock_res_and_lock(lock);
if (lock->l_readers != 0 || lock->l_writers != 0 ||
- !(lock->l_flags & LDLM_FL_CBPENDING)) {
+ !ldlm_is_cbpending(lock)) {
ldlm_lock_addref_internal_nolock(lock, mode);
result = 0;
}
@@ -770,17 +780,17 @@
ldlm_lock_decref_internal_nolock(lock, mode);
- if (lock->l_flags & LDLM_FL_LOCAL &&
+ if (ldlm_is_local(lock) &&
!lock->l_readers && !lock->l_writers) {
/* If this is a local lock on a server namespace and this was
* the last reference, cancel the lock.
*/
CDEBUG(D_INFO, "forcing cancel of local lock\n");
- lock->l_flags |= LDLM_FL_CBPENDING;
+ ldlm_set_cbpending(lock);
}
if (!lock->l_readers && !lock->l_writers &&
- (lock->l_flags & LDLM_FL_CBPENDING)) {
+ ldlm_is_cbpending(lock)) {
/* If we received a blocked AST and this was the last reference,
* run the callback.
*/
@@ -791,16 +801,14 @@
ldlm_lock_remove_from_lru(lock);
unlock_res_and_lock(lock);
- if (lock->l_flags & LDLM_FL_FAIL_LOC)
+ if (ldlm_is_fail_loc(lock))
OBD_RACE(OBD_FAIL_LDLM_CP_BL_RACE);
- if ((lock->l_flags & LDLM_FL_ATOMIC_CB) ||
+ if (ldlm_is_atomic_cb(lock) ||
ldlm_bl_to_thread_lock(ns, NULL, lock) != 0)
ldlm_handle_bl_callback(ns, NULL, lock);
} else if (!lock->l_readers && !lock->l_writers &&
- !(lock->l_flags & LDLM_FL_NO_LRU) &&
- !(lock->l_flags & LDLM_FL_BL_AST)) {
-
+ !ldlm_is_no_lru(lock) && !ldlm_is_bl_ast(lock)) {
LDLM_DEBUG(lock, "add lock into lru list");
/* If this is a client-side namespace and this was the last
@@ -809,7 +817,7 @@
ldlm_lock_add_to_lru(lock);
unlock_res_and_lock(lock);
- if (lock->l_flags & LDLM_FL_FAIL_LOC)
+ if (ldlm_is_fail_loc(lock))
OBD_RACE(OBD_FAIL_LDLM_CP_BL_RACE);
/* Call ldlm_cancel_lru() only if EARLY_CANCEL and LRU RESIZE
@@ -853,7 +861,7 @@
LDLM_DEBUG(lock, "ldlm_lock_decref(%s)", ldlm_lockname[mode]);
lock_res_and_lock(lock);
- lock->l_flags |= LDLM_FL_CBPENDING;
+ ldlm_set_cbpending(lock);
unlock_res_and_lock(lock);
ldlm_lock_decref_internal(lock, mode);
LDLM_LOCK_PUT(lock);
@@ -971,7 +979,7 @@
ldlm_resource_dump(D_INFO, res);
LDLM_DEBUG(lock, "About to add lock:");
- if (lock->l_flags & LDLM_FL_DESTROYED) {
+ if (ldlm_is_destroyed(lock)) {
CDEBUG(D_OTHER, "Lock destroyed, not adding to resource\n");
return;
}
@@ -1073,10 +1081,9 @@
* whose parents already hold a lock so forward progress
* can still happen.
*/
- if (lock->l_flags & LDLM_FL_CBPENDING &&
- !(flags & LDLM_FL_CBPENDING))
+ if (ldlm_is_cbpending(lock) && !(flags & LDLM_FL_CBPENDING))
continue;
- if (!unref && lock->l_flags & LDLM_FL_CBPENDING &&
+ if (!unref && ldlm_is_cbpending(lock) &&
lock->l_readers == 0 && lock->l_writers == 0)
continue;
@@ -1092,6 +1099,7 @@
if (unlikely(match == LCK_GROUP) &&
lock->l_resource->lr_type == LDLM_EXTENT &&
+ policy->l_extent.gid != LDLM_GID_ANY &&
lock->l_policy_data.l_extent.gid != policy->l_extent.gid)
continue;
@@ -1104,11 +1112,10 @@
policy->l_inodebits.bits))
continue;
- if (!unref && (lock->l_flags & LDLM_FL_GONE_MASK))
+ if (!unref && LDLM_HAVE_MASK(lock, GONE))
continue;
- if ((flags & LDLM_FL_LOCAL_ONLY) &&
- !(lock->l_flags & LDLM_FL_LOCAL))
+ if ((flags & LDLM_FL_LOCAL_ONLY) && !ldlm_is_local(lock))
continue;
if (flags & LDLM_FL_TEST_LOCK) {
@@ -1142,7 +1149,7 @@
*/
void ldlm_lock_allow_match_locked(struct ldlm_lock *lock)
{
- lock->l_flags |= LDLM_FL_LVB_READY;
+ ldlm_set_lvb_ready(lock);
wake_up_all(&lock->l_waitq);
}
EXPORT_SYMBOL(ldlm_lock_allow_match_locked);
@@ -1243,8 +1250,7 @@
if (lock) {
ldlm_lock2handle(lock, lockh);
- if ((flags & LDLM_FL_LVB_READY) &&
- (!(lock->l_flags & LDLM_FL_LVB_READY))) {
+ if ((flags & LDLM_FL_LVB_READY) && !ldlm_is_lvb_ready(lock)) {
__u64 wait_flags = LDLM_FL_LVB_READY |
LDLM_FL_DESTROYED | LDLM_FL_FAIL_NOTIFIED;
struct l_wait_info lwi;
@@ -1271,7 +1277,7 @@
l_wait_event(lock->l_waitq,
lock->l_flags & wait_flags,
&lwi);
- if (!(lock->l_flags & LDLM_FL_LVB_READY)) {
+ if (!ldlm_is_lvb_ready(lock)) {
if (flags & LDLM_FL_TEST_LOCK)
LDLM_LOCK_RELEASE(lock);
else
@@ -1325,10 +1331,10 @@
lock = ldlm_handle2lock(lockh);
if (lock) {
lock_res_and_lock(lock);
- if (lock->l_flags & LDLM_FL_GONE_MASK)
+ if (LDLM_HAVE_MASK(lock, GONE))
goto out;
- if (lock->l_flags & LDLM_FL_CBPENDING &&
+ if (ldlm_is_cbpending(lock) &&
lock->l_readers == 0 && lock->l_writers == 0)
goto out;
@@ -1542,7 +1548,8 @@
/* Some flags from the enqueue want to make it into the AST, via the
* lock's l_flags.
*/
- lock->l_flags |= *flags & LDLM_FL_AST_DISCARD_DATA;
+ if (*flags & LDLM_FL_AST_DISCARD_DATA)
+ ldlm_set_ast_discard_data(lock);
/*
* This distinction between local lock trees is very important; a client
@@ -1581,7 +1588,7 @@
lock_res_and_lock(lock);
list_del_init(&lock->l_bl_ast);
- LASSERT(lock->l_flags & LDLM_FL_AST_SENT);
+ LASSERT(ldlm_is_ast_sent(lock));
LASSERT(lock->l_bl_ast_run == 0);
LASSERT(lock->l_blocking_lock);
lock->l_bl_ast_run++;
@@ -1628,12 +1635,12 @@
/* nobody should touch l_cp_ast */
lock_res_and_lock(lock);
list_del_init(&lock->l_cp_ast);
- LASSERT(lock->l_flags & LDLM_FL_CP_REQD);
+ LASSERT(ldlm_is_cp_reqd(lock));
/* save l_completion_ast since it can be changed by
* mds_intent_policy(), see bug 14225
*/
completion_callback = lock->l_completion_ast;
- lock->l_flags &= ~LDLM_FL_CP_REQD;
+ ldlm_clear_cp_reqd(lock);
unlock_res_and_lock(lock);
if (completion_callback)
@@ -1778,8 +1785,8 @@
void ldlm_cancel_callback(struct ldlm_lock *lock)
{
check_res_locked(lock->l_resource);
- if (!(lock->l_flags & LDLM_FL_CANCEL)) {
- lock->l_flags |= LDLM_FL_CANCEL;
+ if (!ldlm_is_cancel(lock)) {
+ ldlm_set_cancel(lock);
if (lock->l_blocking_ast) {
unlock_res_and_lock(lock);
lock->l_blocking_ast(lock, NULL, lock->l_ast_data,
@@ -1789,7 +1796,7 @@
LDLM_DEBUG(lock, "no blocking ast");
}
}
- lock->l_flags |= LDLM_FL_BL_DONE;
+ ldlm_set_bl_done(lock);
}
/**
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
index ebe9042..ab739f0 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c
@@ -124,10 +124,10 @@
LDLM_DEBUG(lock, "client blocking AST callback handler");
lock_res_and_lock(lock);
- lock->l_flags |= LDLM_FL_CBPENDING;
+ ldlm_set_cbpending(lock);
- if (lock->l_flags & LDLM_FL_CANCEL_ON_BLOCK)
- lock->l_flags |= LDLM_FL_CANCEL;
+ if (ldlm_is_cancel_on_block(lock))
+ ldlm_set_cancel(lock);
do_ast = !lock->l_readers && !lock->l_writers;
unlock_res_and_lock(lock);
@@ -172,7 +172,7 @@
set_current_state(TASK_INTERRUPTIBLE);
schedule_timeout(to);
if (lock->l_granted_mode == lock->l_req_mode ||
- lock->l_flags & LDLM_FL_DESTROYED)
+ ldlm_is_destroyed(lock))
break;
}
}
@@ -215,7 +215,7 @@
}
lock_res_and_lock(lock);
- if ((lock->l_flags & LDLM_FL_DESTROYED) ||
+ if (ldlm_is_destroyed(lock) ||
lock->l_granted_mode == lock->l_req_mode) {
/* bug 11300: the lock has already been granted */
unlock_res_and_lock(lock);
@@ -291,7 +291,7 @@
out:
if (rc < 0) {
lock_res_and_lock(lock);
- lock->l_flags |= LDLM_FL_FAILED;
+ ldlm_set_failed(lock);
unlock_res_and_lock(lock);
wake_up(&lock->l_waitq);
}
@@ -360,8 +360,7 @@
struct ldlm_bl_pool *blp = ldlm_state->ldlm_bl_pool;
spin_lock(&blp->blp_lock);
- if (blwi->blwi_lock &&
- blwi->blwi_lock->l_flags & LDLM_FL_DISCARD_DATA) {
+ if (blwi->blwi_lock && ldlm_is_discard_data(blwi->blwi_lock)) {
/* add LDLM_FL_DISCARD_DATA requests to the priority list */
list_add_tail(&blwi->blwi_entry, &blp->blp_prio_list);
} else {
@@ -626,23 +625,22 @@
return 0;
}
- if ((lock->l_flags & LDLM_FL_FAIL_LOC) &&
+ if (ldlm_is_fail_loc(lock) &&
lustre_msg_get_opc(req->rq_reqmsg) == LDLM_BL_CALLBACK)
OBD_RACE(OBD_FAIL_LDLM_CP_BL_RACE);
/* Copy hints/flags (e.g. LDLM_FL_DISCARD_DATA) from AST. */
lock_res_and_lock(lock);
lock->l_flags |= ldlm_flags_from_wire(dlm_req->lock_flags &
- LDLM_AST_FLAGS);
+ LDLM_FL_AST_MASK);
if (lustre_msg_get_opc(req->rq_reqmsg) == LDLM_BL_CALLBACK) {
/* If somebody cancels lock and cache is already dropped,
* or lock is failed before cp_ast received on client,
* we can tell the server we have no lock. Otherwise, we
* should send cancel after dropping the cache.
*/
- if (((lock->l_flags & LDLM_FL_CANCELING) &&
- (lock->l_flags & LDLM_FL_BL_DONE)) ||
- (lock->l_flags & LDLM_FL_FAILED)) {
+ if ((ldlm_is_canceling(lock) && ldlm_is_bl_done(lock)) ||
+ ldlm_is_failed(lock)) {
LDLM_DEBUG(lock, "callback on lock %#llx - lock disappeared\n",
dlm_req->lock_handle[0].cookie);
unlock_res_and_lock(lock);
@@ -656,7 +654,7 @@
* Let ldlm_cancel_lru() be fast.
*/
ldlm_lock_remove_from_lru(lock);
- lock->l_flags |= LDLM_FL_BL_AST;
+ ldlm_set_bl_ast(lock);
}
unlock_res_and_lock(lock);
@@ -674,7 +672,7 @@
case LDLM_BL_CALLBACK:
CDEBUG(D_INODE, "blocking ast\n");
req_capsule_extend(&req->rq_pill, &RQF_LDLM_BL_CALLBACK);
- if (!(lock->l_flags & LDLM_FL_CANCEL_ON_BLOCK)) {
+ if (!ldlm_is_cancel_on_block(lock)) {
rc = ldlm_callback_reply(req, 0);
if (req->rq_no_reply || rc)
ldlm_callback_errmsg(req, "Normal process", rc,
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c
index 74e193e..107314e 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c
@@ -153,7 +153,7 @@
long delay;
int result;
- if (lock->l_flags & (LDLM_FL_DESTROYED | LDLM_FL_FAILED)) {
+ if (ldlm_is_destroyed(lock) || ldlm_is_failed(lock)) {
LDLM_DEBUG(lock, "client-side enqueue: destroyed");
result = -EIO;
} else {
@@ -252,7 +252,7 @@
lwd.lwd_lock = lock;
- if (lock->l_flags & LDLM_FL_NO_TIMEOUT) {
+ if (ldlm_is_no_timeout(lock)) {
LDLM_DEBUG(lock, "waiting indefinitely because of NO_TIMEOUT");
lwi = LWI_INTR(interrupted_completion_wait, &lwd);
} else {
@@ -269,7 +269,7 @@
if (OBD_FAIL_CHECK_RESET(OBD_FAIL_LDLM_INTR_CP_AST,
OBD_FAIL_LDLM_CP_BL_RACE | OBD_FAIL_ONCE)) {
- lock->l_flags |= LDLM_FL_FAIL_LOC;
+ ldlm_set_fail_loc(lock);
rc = -EINTR;
} else {
/* Go to sleep until the lock is granted or cancelled. */
@@ -296,7 +296,7 @@
lock_res_and_lock(lock);
/* Check that lock is not granted or failed, we might race. */
if ((lock->l_req_mode != lock->l_granted_mode) &&
- !(lock->l_flags & LDLM_FL_FAILED)) {
+ !ldlm_is_failed(lock)) {
/* Make sure that this lock will not be found by raced
* bl_ast and -EINVAL reply is sent to server anyways.
* bug 17645
@@ -347,7 +347,6 @@
struct ldlm_lock *lock;
struct ldlm_reply *reply;
int cleanup_phase = 1;
- int size = 0;
lock = ldlm_handle2lock(lockh);
/* ldlm_cli_enqueue is holding a reference on this lock. */
@@ -375,8 +374,8 @@
goto cleanup;
}
- if (lvb_len != 0) {
- LASSERT(lvb);
+ if (lvb_len > 0) {
+ int size = 0;
size = req_capsule_get_size(&req->rq_pill, &RMF_DLM_LVB,
RCL_SERVER);
@@ -390,12 +389,13 @@
rc = -EINVAL;
goto cleanup;
}
+ lvb_len = size;
}
if (rc == ELDLM_LOCK_ABORTED) {
- if (lvb_len != 0)
+ if (lvb_len > 0 && lvb)
rc = ldlm_fill_lvb(lock, &req->rq_pill, RCL_SERVER,
- lvb, size);
+ lvb, lvb_len);
if (rc == 0)
rc = ELDLM_LOCK_ABORTED;
goto cleanup;
@@ -421,7 +421,7 @@
*flags = ldlm_flags_from_wire(reply->lock_flags);
lock->l_flags |= ldlm_flags_from_wire(reply->lock_flags &
- LDLM_INHERIT_FLAGS);
+ LDLM_FL_INHERIT_MASK);
/* move NO_TIMEOUT flag to the lock to force ldlm_lock_match()
* to wait with no timeout as well
*/
@@ -489,7 +489,7 @@
/* If the lock has already been granted by a completion AST, don't
* clobber the LVB with an older one.
*/
- if (lvb_len != 0) {
+ if (lvb_len > 0) {
/* We must lock or a racing completion might update lvb without
* letting us know and we'll clobber the correct value.
* Cannot unlock after the check either, as that still leaves
@@ -498,7 +498,7 @@
lock_res_and_lock(lock);
if (lock->l_req_mode != lock->l_granted_mode)
rc = ldlm_fill_lvb(lock, &req->rq_pill, RCL_SERVER,
- lock->l_lvb_data, size);
+ lock->l_lvb_data, lvb_len);
unlock_res_and_lock(lock);
if (rc < 0) {
cleanup_phase = 1;
@@ -518,7 +518,7 @@
}
}
- if (lvb_len && lvb) {
+ if (lvb_len > 0 && lvb) {
/* Copy the LVB here, and not earlier, because the completion
* AST (if any) can override what we got in the reply
*/
@@ -601,7 +601,7 @@
avail = ldlm_capsule_handles_avail(pill, RCL_CLIENT, canceloff);
flags = ns_connect_lru_resize(ns) ?
- LDLM_CANCEL_LRUR : LDLM_CANCEL_AGED;
+ LDLM_CANCEL_LRUR_NO_WAIT : LDLM_CANCEL_AGED;
to_free = !ns_connect_lru_resize(ns) &&
opc == LDLM_ENQUEUE ? 1 : 0;
@@ -821,12 +821,11 @@
LDLM_DEBUG(lock, "client-side cancel");
/* Set this flag to prevent others from getting new references*/
lock_res_and_lock(lock);
- lock->l_flags |= LDLM_FL_CBPENDING;
+ ldlm_set_cbpending(lock);
local_only = !!(lock->l_flags &
(LDLM_FL_LOCAL_ONLY|LDLM_FL_CANCEL_ON_BLOCK));
ldlm_cancel_callback(lock);
- rc = (lock->l_flags & LDLM_FL_BL_AST) ?
- LDLM_FL_BL_AST : LDLM_FL_CANCELING;
+ rc = ldlm_is_bl_ast(lock) ? LDLM_FL_BL_AST : LDLM_FL_CANCELING;
unlock_res_and_lock(lock);
if (local_only) {
@@ -1131,31 +1130,30 @@
* dirty data, to close a file, ...) or waiting for any RPCs in-flight (e.g.
* readahead requests, ...)
*/
-static ldlm_policy_res_t ldlm_cancel_no_wait_policy(struct ldlm_namespace *ns,
- struct ldlm_lock *lock,
- int unused, int added,
- int count)
+static enum ldlm_policy_res
+ldlm_cancel_no_wait_policy(struct ldlm_namespace *ns, struct ldlm_lock *lock,
+ int unused, int added, int count)
{
- ldlm_policy_res_t result = LDLM_POLICY_CANCEL_LOCK;
- ldlm_cancel_for_recovery cb = ns->ns_cancel_for_recovery;
-
- lock_res_and_lock(lock);
+ enum ldlm_policy_res result = LDLM_POLICY_CANCEL_LOCK;
/* don't check added & count since we want to process all locks
- * from unused list
+ * from unused list.
+ * It's fine to not take lock to access lock->l_resource since
+ * the lock has already been granted so it won't change.
*/
switch (lock->l_resource->lr_type) {
case LDLM_EXTENT:
case LDLM_IBITS:
- if (cb && cb(lock))
+ if (ns->ns_cancel && ns->ns_cancel(lock) != 0)
break;
default:
result = LDLM_POLICY_SKIP_LOCK;
- lock->l_flags |= LDLM_FL_SKIPPED;
+ lock_res_and_lock(lock);
+ ldlm_set_skipped(lock);
+ unlock_res_and_lock(lock);
break;
}
- unlock_res_and_lock(lock);
return result;
}
@@ -1168,10 +1166,10 @@
*
* \retval LDLM_POLICY_CANCEL_LOCK cancel lock from LRU
*/
-static ldlm_policy_res_t ldlm_cancel_lrur_policy(struct ldlm_namespace *ns,
- struct ldlm_lock *lock,
- int unused, int added,
- int count)
+static enum ldlm_policy_res ldlm_cancel_lrur_policy(struct ldlm_namespace *ns,
+ struct ldlm_lock *lock,
+ int unused, int added,
+ int count)
{
unsigned long cur = cfs_time_current();
struct ldlm_pool *pl = &ns->ns_pool;
@@ -1196,8 +1194,13 @@
/* Stop when SLV is not yet come from server or lv is smaller than
* it is.
*/
- return (slv == 0 || lv < slv) ?
- LDLM_POLICY_KEEP_LOCK : LDLM_POLICY_CANCEL_LOCK;
+ if (slv == 0 || lv < slv)
+ return LDLM_POLICY_KEEP_LOCK;
+
+ if (ns->ns_cancel && ns->ns_cancel(lock) == 0)
+ return LDLM_POLICY_KEEP_LOCK;
+
+ return LDLM_POLICY_CANCEL_LOCK;
}
/**
@@ -1209,10 +1212,10 @@
*
* \retval LDLM_POLICY_CANCEL_LOCK cancel lock from LRU
*/
-static ldlm_policy_res_t ldlm_cancel_passed_policy(struct ldlm_namespace *ns,
- struct ldlm_lock *lock,
- int unused, int added,
- int count)
+static enum ldlm_policy_res ldlm_cancel_passed_policy(struct ldlm_namespace *ns,
+ struct ldlm_lock *lock,
+ int unused, int added,
+ int count)
{
/* Stop LRU processing when we reach past @count or have checked all
* locks in LRU.
@@ -1230,16 +1233,35 @@
*
* \retval LDLM_POLICY_CANCEL_LOCK cancel lock from LRU
*/
-static ldlm_policy_res_t ldlm_cancel_aged_policy(struct ldlm_namespace *ns,
- struct ldlm_lock *lock,
- int unused, int added,
- int count)
+static enum ldlm_policy_res ldlm_cancel_aged_policy(struct ldlm_namespace *ns,
+ struct ldlm_lock *lock,
+ int unused, int added,
+ int count)
{
- /* Stop LRU processing if young lock is found and we reach past count */
- return ((added >= count) &&
- time_before(cfs_time_current(),
- cfs_time_add(lock->l_last_used, ns->ns_max_age))) ?
- LDLM_POLICY_KEEP_LOCK : LDLM_POLICY_CANCEL_LOCK;
+ if ((added >= count) &&
+ time_before(cfs_time_current(),
+ cfs_time_add(lock->l_last_used, ns->ns_max_age)))
+ return LDLM_POLICY_KEEP_LOCK;
+
+ if (ns->ns_cancel && ns->ns_cancel(lock) == 0)
+ return LDLM_POLICY_KEEP_LOCK;
+
+ return LDLM_POLICY_CANCEL_LOCK;
+}
+
+static enum ldlm_policy_res
+ldlm_cancel_lrur_no_wait_policy(struct ldlm_namespace *ns,
+ struct ldlm_lock *lock,
+ int unused, int added,
+ int count)
+{
+ enum ldlm_policy_res result;
+
+ result = ldlm_cancel_lrur_policy(ns, lock, unused, added, count);
+ if (result == LDLM_POLICY_KEEP_LOCK)
+ return result;
+
+ return ldlm_cancel_no_wait_policy(ns, lock, unused, added, count);
}
/**
@@ -1251,10 +1273,9 @@
*
* \retval LDLM_POLICY_CANCEL_LOCK cancel lock from LRU
*/
-static ldlm_policy_res_t ldlm_cancel_default_policy(struct ldlm_namespace *ns,
- struct ldlm_lock *lock,
- int unused, int added,
- int count)
+static enum ldlm_policy_res
+ldlm_cancel_default_policy(struct ldlm_namespace *ns, struct ldlm_lock *lock,
+ int unused, int added, int count)
{
/* Stop LRU processing when we reach past count or have checked all
* locks in LRU.
@@ -1263,7 +1284,8 @@
LDLM_POLICY_KEEP_LOCK : LDLM_POLICY_CANCEL_LOCK;
}
-typedef ldlm_policy_res_t (*ldlm_cancel_lru_policy_t)(struct ldlm_namespace *,
+typedef enum ldlm_policy_res (*ldlm_cancel_lru_policy_t)(
+ struct ldlm_namespace *,
struct ldlm_lock *, int,
int, int);
@@ -1281,6 +1303,8 @@
return ldlm_cancel_lrur_policy;
else if (flags & LDLM_CANCEL_PASSED)
return ldlm_cancel_passed_policy;
+ else if (flags & LDLM_CANCEL_LRUR_NO_WAIT)
+ return ldlm_cancel_lrur_no_wait_policy;
} else {
if (flags & LDLM_CANCEL_AGED)
return ldlm_cancel_aged_policy;
@@ -1329,6 +1353,7 @@
ldlm_cancel_lru_policy_t pf;
struct ldlm_lock *lock, *next;
int added = 0, unused, remained;
+ int no_wait = flags & (LDLM_CANCEL_NO_WAIT | LDLM_CANCEL_LRUR_NO_WAIT);
spin_lock(&ns->ns_lock);
unused = ns->ns_nr_unused;
@@ -1341,7 +1366,8 @@
LASSERT(pf);
while (!list_empty(&ns->ns_unused_list)) {
- ldlm_policy_res_t result;
+ enum ldlm_policy_res result;
+ time_t last_use = 0;
/* all unused locks */
if (remained-- <= 0)
@@ -1354,17 +1380,20 @@
list_for_each_entry_safe(lock, next, &ns->ns_unused_list,
l_lru) {
/* No locks which got blocking requests. */
- LASSERT(!(lock->l_flags & LDLM_FL_BL_AST));
+ LASSERT(!ldlm_is_bl_ast(lock));
- if (flags & LDLM_CANCEL_NO_WAIT &&
- lock->l_flags & LDLM_FL_SKIPPED)
+ if (no_wait && ldlm_is_skipped(lock))
/* already processed */
continue;
+ last_use = lock->l_last_used;
+ if (last_use == cfs_time_current())
+ continue;
+
/* Somebody is already doing CANCEL. No need for this
* lock in LRU, do not traverse it again.
*/
- if (!(lock->l_flags & LDLM_FL_CANCELING))
+ if (!ldlm_is_canceling(lock))
break;
ldlm_lock_remove_from_lru_nolock(lock);
@@ -1407,12 +1436,14 @@
lock_res_and_lock(lock);
/* Check flags again under the lock. */
- if ((lock->l_flags & LDLM_FL_CANCELING) ||
- (ldlm_lock_remove_from_lru(lock) == 0)) {
+ if (ldlm_is_canceling(lock) ||
+ (ldlm_lock_remove_from_lru_check(lock, last_use) == 0)) {
/* Another thread is removing lock from LRU, or
* somebody is already doing CANCEL, or there
* is a blocking request which will send cancel
- * by itself, or the lock is no longer unused.
+ * by itself, or the lock is no longer unused or
+ * the lock has been used since the pf() call and
+ * pages could be put under it.
*/
unlock_res_and_lock(lock);
lu_ref_del(&lock->l_reference,
@@ -1429,7 +1460,7 @@
* where while we are doing cancel here, server is also
* silently cancelling this lock.
*/
- lock->l_flags &= ~LDLM_FL_CANCEL_ON_BLOCK;
+ ldlm_clear_cancel_on_block(lock);
/* Setting the CBPENDING flag is a little misleading,
* but prevents an important race; namely, once
@@ -1526,8 +1557,7 @@
/* If somebody is already doing CANCEL, or blocking AST came,
* skip this lock.
*/
- if (lock->l_flags & LDLM_FL_BL_AST ||
- lock->l_flags & LDLM_FL_CANCELING)
+ if (ldlm_is_bl_ast(lock) || ldlm_is_canceling(lock))
continue;
if (lockmode_compat(lock->l_granted_mode, mode))
@@ -1771,7 +1801,6 @@
cfs_hash_for_each_nolock(ns->ns_rs_hash,
ldlm_res_iter_helper, &helper);
-
}
/* non-blocking function to manipulate a lock whose cb_data is being put away.
@@ -1887,7 +1916,7 @@
int flags;
/* Bug 11974: Do not replay a lock which is actively being canceled */
- if (lock->l_flags & LDLM_FL_CANCELING) {
+ if (ldlm_is_canceling(lock)) {
LDLM_DEBUG(lock, "Not replaying canceled lock:");
return 0;
}
@@ -1896,7 +1925,7 @@
* server might have long dropped it, but notification of that event was
* lost by network. (and server granted conflicting lock already)
*/
- if (lock->l_flags & LDLM_FL_CANCEL_ON_BLOCK) {
+ if (ldlm_is_cancel_on_block(lock)) {
LDLM_DEBUG(lock, "Not replaying reply-less lock:");
ldlm_lock_cancel(lock);
return 0;
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
index 9dede87..e99c89c 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c
@@ -124,9 +124,15 @@
}
rc = ldebugfs_add_vars(ldlm_debugfs_dir, ldlm_debugfs_list, NULL);
+ if (rc) {
+ CERROR("LProcFS failed in ldlm-init\n");
+ goto err_svc;
+ }
return 0;
+err_svc:
+ ldebugfs_remove(&ldlm_svc_debugfs_dir);
err_ns:
ldebugfs_remove(&ldlm_ns_debugfs_dir);
err_type:
@@ -758,12 +764,12 @@
list_for_each(tmp, q) {
lock = list_entry(tmp, struct ldlm_lock,
l_res_link);
- if (lock->l_flags & LDLM_FL_CLEANED) {
+ if (ldlm_is_cleaned(lock)) {
lock = NULL;
continue;
}
LDLM_LOCK_GET(lock);
- lock->l_flags |= LDLM_FL_CLEANED;
+ ldlm_set_cleaned(lock);
break;
}
@@ -775,13 +781,13 @@
/* Set CBPENDING so nothing in the cancellation path
* can match this lock.
*/
- lock->l_flags |= LDLM_FL_CBPENDING;
- lock->l_flags |= LDLM_FL_FAILED;
+ ldlm_set_cbpending(lock);
+ ldlm_set_failed(lock);
lock->l_flags |= flags;
/* ... without sending a CANCEL message for local_only. */
if (local_only)
- lock->l_flags |= LDLM_FL_LOCAL_ONLY;
+ ldlm_set_local_only(lock);
if (local_only && (lock->l_readers || lock->l_writers)) {
/* This is a little bit gross, but much better than the
@@ -1275,7 +1281,7 @@
LDLM_DEBUG(lock, "About to add this lock:\n");
- if (lock->l_flags & LDLM_FL_DESTROYED) {
+ if (ldlm_is_destroyed(lock)) {
CDEBUG(D_OTHER, "Lock destroyed, not adding to resource\n");
return;
}
@@ -1400,3 +1406,4 @@
LDLM_DEBUG_LIMIT(level, lock, "###");
}
}
+EXPORT_SYMBOL(ldlm_resource_dump);
diff --git a/drivers/staging/lustre/lustre/llite/Makefile b/drivers/staging/lustre/lustre/llite/Makefile
index 9ac29e7..2ce10ff 100644
--- a/drivers/staging/lustre/lustre/llite/Makefile
+++ b/drivers/staging/lustre/lustre/llite/Makefile
@@ -4,7 +4,8 @@
rw.o namei.o symlink.o llite_mmap.o \
xattr.o xattr_cache.o remote_perm.o llite_rmtacl.o \
rw26.o super25.o statahead.o \
- ../lclient/glimpse.o ../lclient/lcommon_cl.o ../lclient/lcommon_misc.o \
- vvp_dev.o vvp_page.o vvp_lock.o vvp_io.o vvp_object.o lproc_llite.o
+ glimpse.o lcommon_cl.o lcommon_misc.o \
+ vvp_dev.o vvp_page.o vvp_lock.o vvp_io.o vvp_object.o vvp_req.o \
+ lproc_llite.o
llite_lloop-y := lloop.o
diff --git a/drivers/staging/lustre/lustre/llite/dcache.c b/drivers/staging/lustre/lustre/llite/dcache.c
index dd1c827..1b6f82a 100644
--- a/drivers/staging/lustre/lustre/llite/dcache.c
+++ b/drivers/staging/lustre/lustre/llite/dcache.c
@@ -108,11 +108,8 @@
static inline int return_if_equal(struct ldlm_lock *lock, void *data)
{
- if ((lock->l_flags &
- (LDLM_FL_CANCELING | LDLM_FL_DISCARD_DATA)) ==
- (LDLM_FL_CANCELING | LDLM_FL_DISCARD_DATA))
- return LDLM_ITER_CONTINUE;
- return LDLM_ITER_STOP;
+ return (ldlm_is_canceling(lock) && ldlm_is_discard_data(lock)) ?
+ LDLM_ITER_CONTINUE : LDLM_ITER_STOP;
}
/* find any ldlm lock of the inode in mdc and lov
@@ -253,8 +250,8 @@
{
struct dentry *dentry;
- CDEBUG(D_INODE, "marking dentries for ino %lu/%u(%p) invalid\n",
- inode->i_ino, inode->i_generation, inode);
+ CDEBUG(D_INODE, "marking dentries for ino "DFID"(%p) invalid\n",
+ PFID(ll_inode2fid(inode)), inode);
ll_lock_dcache(inode);
hlist_for_each_entry(dentry, &inode->i_dentry, d_u.d_alias) {
@@ -289,8 +286,8 @@
if (it->d.lustre.it_lock_mode && inode) {
struct ll_sb_info *sbi = ll_i2sbi(inode);
- CDEBUG(D_DLMTRACE, "setting l_data to inode %p (%lu/%u)\n",
- inode, inode->i_ino, inode->i_generation);
+ CDEBUG(D_DLMTRACE, "setting l_data to inode "DFID"(%p)\n",
+ PFID(ll_inode2fid(inode)), inode);
ll_set_lock_data(sbi->ll_md_exp, inode, it, NULL);
}
diff --git a/drivers/staging/lustre/lustre/llite/dir.c b/drivers/staging/lustre/lustre/llite/dir.c
index 7a0a67f..4b00d1a 100644
--- a/drivers/staging/lustre/lustre/llite/dir.c
+++ b/drivers/staging/lustre/lustre/llite/dir.c
@@ -158,11 +158,16 @@
int i;
int rc;
- CDEBUG(D_VFSTRACE, "VFS Op:inode=%lu/%u(%p) hash %llu\n",
- inode->i_ino, inode->i_generation, inode, hash);
+ CDEBUG(D_VFSTRACE, "VFS Op:inode="DFID"(%p) hash %llu\n",
+ PFID(ll_inode2fid(inode)), inode, hash);
LASSERT(max_pages > 0 && max_pages <= MD_MAX_BRW_PAGES);
+ op_data = ll_prep_md_op_data(NULL, inode, NULL, NULL, 0, 0,
+ LUSTRE_OPC_ANY, NULL);
+ if (IS_ERR(op_data))
+ return PTR_ERR(op_data);
+
page_pool = kcalloc(max_pages, sizeof(page), GFP_NOFS);
if (page_pool) {
page_pool[0] = page0;
@@ -177,8 +182,6 @@
page_pool[npages] = page;
}
- op_data = ll_prep_md_op_data(NULL, inode, NULL, NULL, 0, 0,
- LUSTRE_OPC_ANY, NULL);
op_data->op_npages = npages;
op_data->op_offset = hash;
rc = md_readpage(exp, op_data, page_pool, &request);
@@ -190,7 +193,7 @@
body = req_capsule_server_get(&request->rq_pill, &RMF_MDT_BODY);
/* Checked by mdc_readpage() */
if (body->valid & OBD_MD_FLSIZE)
- cl_isize_write(inode, body->size);
+ i_size_write(inode, body->size);
nrdpgs = (request->rq_bulk->bd_nob_transferred+PAGE_SIZE-1)
>> PAGE_SHIFT;
@@ -372,8 +375,8 @@
return ERR_PTR(rc);
}
- CDEBUG(D_INODE, "setting lr_lvb_inode to inode %p (%lu/%u)\n",
- dir, dir->i_ino, dir->i_generation);
+ CDEBUG(D_INODE, "setting lr_lvb_inode to inode "DFID"(%p)\n",
+ PFID(ll_inode2fid(dir)), dir);
md_set_lock_data(ll_i2sbi(dir)->ll_md_exp,
&it.d.lustre.it_lock_handle, dir, NULL);
} else {
@@ -468,6 +471,28 @@
goto out_unlock;
}
+/**
+ * return IF_* type for given lu_dirent entry.
+ * IF_* flag shld be converted to particular OS file type in
+ * platform llite module.
+ */
+static __u16 ll_dirent_type_get(struct lu_dirent *ent)
+{
+ __u16 type = 0;
+ struct luda_type *lt;
+ int len = 0;
+
+ if (le32_to_cpu(ent->lde_attrs) & LUDA_TYPE) {
+ const unsigned int align = sizeof(struct luda_type) - 1;
+
+ len = le16_to_cpu(ent->lde_namelen);
+ len = (len + align) & ~align;
+ lt = (void *)ent->lde_name + len;
+ type = IFTODT(le16_to_cpu(lt->lt_type));
+ }
+ return type;
+}
+
int ll_dir_read(struct inode *inode, struct dir_context *ctx)
{
struct ll_inode_info *info = ll_i2info(inode);
@@ -589,15 +614,16 @@
struct inode *inode = file_inode(filp);
struct ll_file_data *lfd = LUSTRE_FPRIVATE(filp);
struct ll_sb_info *sbi = ll_i2sbi(inode);
+ __u64 pos = lfd ? lfd->lfd_pos : 0;
int hash64 = sbi->ll_flags & LL_SBI_64BIT_HASH;
int api32 = ll_need_32bit_api(sbi);
int rc;
- CDEBUG(D_VFSTRACE, "VFS Op:inode=%lu/%u(%p) pos %lu/%llu 32bit_api %d\n",
- inode->i_ino, inode->i_generation,
- inode, (unsigned long)lfd->lfd_pos, i_size_read(inode), api32);
+ CDEBUG(D_VFSTRACE, "VFS Op:inode="DFID"(%p) pos %lu/%llu 32bit_api %d\n",
+ PFID(ll_inode2fid(inode)), inode, (unsigned long)pos,
+ i_size_read(inode), api32);
- if (lfd->lfd_pos == MDS_DIR_END_OFF) {
+ if (pos == MDS_DIR_END_OFF) {
/*
* end-of-file.
*/
@@ -605,9 +631,10 @@
goto out;
}
- ctx->pos = lfd->lfd_pos;
+ ctx->pos = pos;
rc = ll_dir_read(inode, ctx);
- lfd->lfd_pos = ctx->pos;
+ if (lfd)
+ lfd->lfd_pos = ctx->pos;
if (ctx->pos == MDS_DIR_END_OFF) {
if (api32)
ctx->pos = LL_DIR_END_OFF_32BIT;
@@ -804,9 +831,8 @@
rc = md_getattr(sbi->ll_md_exp, op_data, &req);
ll_finish_md_op_data(op_data);
if (rc < 0) {
- CDEBUG(D_INFO, "md_getattr failed on inode %lu/%u: rc %d\n",
- inode->i_ino,
- inode->i_generation, rc);
+ CDEBUG(D_INFO, "md_getattr failed on inode "DFID": rc %d\n",
+ PFID(ll_inode2fid(inode)), rc);
goto out;
}
@@ -916,7 +942,7 @@
}
/* Read current file data version */
- rc = ll_data_version(inode, &data_version, 1);
+ rc = ll_data_version(inode, &data_version, LL_DV_RD_FLUSH);
iput(inode);
if (rc != 0) {
CDEBUG(D_HSM, "Could not read file data version of "
@@ -936,6 +962,9 @@
}
progress:
+ /* On error, the request should be considered as completed */
+ if (hpk.hpk_errval > 0)
+ hpk.hpk_flags |= HP_FLAG_COMPLETED;
rc = obd_iocontrol(LL_IOC_HSM_PROGRESS, sbi->ll_md_exp, sizeof(hpk),
&hpk, NULL);
@@ -997,8 +1026,7 @@
goto progress;
}
- rc = ll_data_version(inode, &data_version,
- copy->hc_hai.hai_action == HSMA_ARCHIVE);
+ rc = ll_data_version(inode, &data_version, LL_DV_RD_FLUSH);
iput(inode);
if (rc) {
CDEBUG(D_HSM, "Could not read file data version. Request could not be confirmed.\n");
@@ -1033,7 +1061,6 @@
/* hpk_errval must be >= 0 */
hpk.hpk_errval = EBUSY;
}
-
}
progress:
@@ -1242,8 +1269,8 @@
struct obd_ioctl_data *data;
int rc = 0;
- CDEBUG(D_VFSTRACE, "VFS Op:inode=%lu/%u(%p), cmd=%#x\n",
- inode->i_ino, inode->i_generation, inode, cmd);
+ CDEBUG(D_VFSTRACE, "VFS Op:inode="DFID"(%p), cmd=%#x\n",
+ PFID(ll_inode2fid(inode)), inode, cmd);
/* asm-ppc{,64} declares TCGETS, et. al. as type 't' not 'T' */
if (_IOC_TYPE(cmd) == 'T' || _IOC_TYPE(cmd) == 't') /* tty ioctls */
@@ -1362,7 +1389,6 @@
lmv_out_free:
obd_ioctl_freedata(buf, len);
return rc;
-
}
case LL_IOC_LOV_SETSTRIPE: {
struct lov_user_md_v3 lumv3;
@@ -1474,8 +1500,9 @@
cmd == LL_IOC_MDC_GETINFO)) {
rc = 0;
goto skip_lmm;
- } else
+ } else {
goto out_req;
+ }
}
if (cmd == IOC_MDC_GETFILESTRIPE ||
@@ -1688,15 +1715,16 @@
return ll_flush_ctx(inode);
#ifdef CONFIG_FS_POSIX_ACL
case LL_IOC_RMTACL: {
- if (sbi->ll_flags & LL_SBI_RMT_CLIENT && is_root_inode(inode)) {
- struct ll_file_data *fd = LUSTRE_FPRIVATE(file);
+ if (sbi->ll_flags & LL_SBI_RMT_CLIENT && is_root_inode(inode)) {
+ struct ll_file_data *fd = LUSTRE_FPRIVATE(file);
- rc = rct_add(&sbi->ll_rct, current_pid(), arg);
- if (!rc)
- fd->fd_flags |= LL_FILE_RMTACL;
- return rc;
- } else
- return 0;
+ rc = rct_add(&sbi->ll_rct, current_pid(), arg);
+ if (!rc)
+ fd->fd_flags |= LL_FILE_RMTACL;
+ return rc;
+ } else {
+ return 0;
+ }
}
#endif
case LL_IOC_GETOBDCOUNT: {
@@ -1817,6 +1845,9 @@
return rc;
}
case LL_IOC_HSM_CT_START:
+ if (!capable(CFS_CAP_SYS_ADMIN))
+ return -EPERM;
+
rc = copy_and_ioctl(cmd, sbi->ll_md_exp, (void __user *)arg,
sizeof(struct lustre_kernelcomm));
return rc;
diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c
index cf619af..f47f2ac 100644
--- a/drivers/staging/lustre/lustre/llite/file.c
+++ b/drivers/staging/lustre/lustre/llite/file.c
@@ -45,6 +45,7 @@
#include "../include/lustre_lite.h"
#include <linux/pagemap.h>
#include <linux/file.h>
+#include <linux/mount.h>
#include "llite_internal.h"
#include "../include/lustre/ll_fiemap.h"
@@ -87,8 +88,7 @@
op_data->op_attr.ia_ctime = inode->i_ctime;
op_data->op_attr.ia_size = i_size_read(inode);
op_data->op_attr_blocks = inode->i_blocks;
- ((struct ll_iattr *)&op_data->op_attr)->ia_attr_flags =
- ll_inode_to_ext_flags(inode->i_flags);
+ op_data->op_attr_flags = ll_inode_to_ext_flags(inode->i_flags);
op_data->op_ioepoch = ll_i2info(inode)->lli_ioepoch;
if (fh)
op_data->op_handle = *fh;
@@ -170,13 +170,15 @@
*/
rc = ll_som_update(inode, op_data);
if (rc) {
- CERROR("inode %lu mdc Size-on-MDS update failed: rc = %d\n",
- inode->i_ino, rc);
+ CERROR("%s: inode "DFID" mdc Size-on-MDS update failed: rc = %d\n",
+ ll_i2mdexp(inode)->exp_obd->obd_name,
+ PFID(ll_inode2fid(inode)), rc);
rc = 0;
}
} else if (rc) {
- CERROR("inode %lu mdc close failed: rc = %d\n",
- inode->i_ino, rc);
+ CERROR("%s: inode "DFID" mdc close failed: rc = %d\n",
+ ll_i2mdexp(inode)->exp_obd->obd_name,
+ PFID(ll_inode2fid(inode)), rc);
}
/* DATA_MODIFIED flag was successfully sent on close, cancel data
@@ -278,7 +280,7 @@
/* clear group lock, if present */
if (unlikely(fd->fd_flags & LL_FILE_GROUP_LOCKED))
- ll_put_grouplock(inode, file, fd->fd_grouplock.cg_gid);
+ ll_put_grouplock(inode, file, fd->fd_grouplock.lg_gid);
if (fd->fd_lease_och) {
bool lease_broken;
@@ -343,8 +345,8 @@
struct ll_inode_info *lli = ll_i2info(inode);
int rc;
- CDEBUG(D_VFSTRACE, "VFS Op:inode=%lu/%u(%p)\n", inode->i_ino,
- inode->i_generation, inode);
+ CDEBUG(D_VFSTRACE, "VFS Op:inode="DFID"(%p)\n",
+ PFID(ll_inode2fid(inode)), inode);
#ifdef CONFIG_FS_POSIX_ACL
if (sbi->ll_flags & LL_SBI_RMT_CLIENT && is_root_inode(inode)) {
@@ -543,8 +545,8 @@
struct ll_file_data *fd;
int rc = 0, opendir_set = 0;
- CDEBUG(D_VFSTRACE, "VFS Op:inode=%lu/%u(%p), flags %o\n", inode->i_ino,
- inode->i_generation, inode, file->f_flags);
+ CDEBUG(D_VFSTRACE, "VFS Op:inode="DFID"(%p), flags %o\n",
+ PFID(ll_inode2fid(inode)), inode, file->f_flags);
it = file->private_data; /* XXX: compat macro */
file->private_data = NULL; /* prevent ll_local_open assertion */
@@ -677,7 +679,9 @@
if (rc)
goto out_och_free;
- LASSERT(it_disposition(it, DISP_ENQ_OPEN_REF));
+ LASSERTF(it_disposition(it, DISP_ENQ_OPEN_REF),
+ "inode %p: disposition %x, status %d\n", inode,
+ it_disposition(it, ~0), it->d.lustre.it_status);
rc = ll_local_open(file, it, fd, *och_p);
if (rc)
@@ -875,16 +879,19 @@
return och;
out_close:
- rc2 = ll_close_inode_openhandle(sbi->ll_md_exp, inode, och, NULL);
- if (rc2)
- CERROR("Close openhandle returned %d\n", rc2);
-
- /* cancel open lock */
+ /* Cancel open lock */
if (it.d.lustre.it_lock_mode != 0) {
ldlm_lock_decref_and_cancel(&och->och_lease_handle,
it.d.lustre.it_lock_mode);
it.d.lustre.it_lock_mode = 0;
+ och->och_lease_handle.cookie = 0ULL;
}
+ rc2 = ll_close_inode_openhandle(sbi->ll_md_exp, inode, och, NULL);
+ if (rc2 < 0)
+ CERROR("%s: error closing file "DFID": %d\n",
+ ll_get_fsname(inode->i_sb, NULL, 0),
+ PFID(&ll_i2info(inode)->lli_fid), rc2);
+ och = NULL; /* och has been freed in ll_close_inode_openhandle() */
out_release_it:
ll_intent_release(&it);
out:
@@ -908,7 +915,7 @@
lock_res_and_lock(lock);
cancelled = ldlm_is_cancel(lock);
unlock_res_and_lock(lock);
- ldlm_lock_put(lock);
+ LDLM_LOCK_PUT(lock);
}
CDEBUG(D_INODE, "lease for " DFID " broken? %d\n",
@@ -926,7 +933,7 @@
/* Fills the obdo with the attributes for the lsm */
static int ll_lsm_getattr(struct lov_stripe_md *lsm, struct obd_export *exp,
- struct obdo *obdo, __u64 ioepoch, int sync)
+ struct obdo *obdo, __u64 ioepoch, int dv_flags)
{
struct ptlrpc_request_set *set;
struct obd_info oinfo = { };
@@ -945,9 +952,11 @@
OBD_MD_FLMTIME | OBD_MD_FLCTIME |
OBD_MD_FLGROUP | OBD_MD_FLEPOCH |
OBD_MD_FLDATAVERSION;
- if (sync) {
+ if (dv_flags & (LL_DV_WR_FLUSH | LL_DV_RD_FLUSH)) {
oinfo.oi_oa->o_valid |= OBD_MD_FLFLAGS;
oinfo.oi_oa->o_flags |= OBD_FL_SRVLOCK;
+ if (dv_flags & LL_DV_WR_FLUSH)
+ oinfo.oi_oa->o_flags |= OBD_FL_FLUSH;
}
set = ptlrpc_prep_set();
@@ -960,11 +969,16 @@
rc = ptlrpc_set_wait(set);
ptlrpc_set_destroy(set);
}
- if (rc == 0)
+ if (rc == 0) {
oinfo.oi_oa->o_valid &= (OBD_MD_FLBLOCKS | OBD_MD_FLBLKSZ |
OBD_MD_FLATIME | OBD_MD_FLMTIME |
OBD_MD_FLCTIME | OBD_MD_FLSIZE |
- OBD_MD_FLDATAVERSION);
+ OBD_MD_FLDATAVERSION | OBD_MD_FLFLAGS);
+ if (dv_flags & LL_DV_WR_FLUSH &&
+ !(oinfo.oi_oa->o_valid & OBD_MD_FLFLAGS &&
+ oinfo.oi_oa->o_flags & OBD_FL_FLUSH))
+ return -ENOTSUPP;
+ }
return rc;
}
@@ -980,7 +994,7 @@
lsm = ccc_inode_lsm_get(inode);
rc = ll_lsm_getattr(lsm, ll_i2dtexp(inode),
- obdo, ioepoch, sync);
+ obdo, ioepoch, sync ? LL_DV_RD_FLUSH : 0);
if (rc == 0) {
struct ost_id *oi = lsm ? &lsm->lsm_oi : &obdo->o_oi;
@@ -994,50 +1008,57 @@
return rc;
}
-int ll_merge_lvb(const struct lu_env *env, struct inode *inode)
+int ll_merge_attr(const struct lu_env *env, struct inode *inode)
{
struct ll_inode_info *lli = ll_i2info(inode);
struct cl_object *obj = lli->lli_clob;
- struct cl_attr *attr = ccc_env_thread_attr(env);
- struct ost_lvb lvb;
+ struct cl_attr *attr = vvp_env_thread_attr(env);
+ s64 atime;
+ s64 mtime;
+ s64 ctime;
int rc = 0;
ll_inode_size_lock(inode);
+
/* merge timestamps the most recently obtained from mds with
* timestamps obtained from osts
*/
- LTIME_S(inode->i_atime) = lli->lli_lvb.lvb_atime;
- LTIME_S(inode->i_mtime) = lli->lli_lvb.lvb_mtime;
- LTIME_S(inode->i_ctime) = lli->lli_lvb.lvb_ctime;
+ LTIME_S(inode->i_atime) = lli->lli_atime;
+ LTIME_S(inode->i_mtime) = lli->lli_mtime;
+ LTIME_S(inode->i_ctime) = lli->lli_ctime;
- lvb.lvb_size = i_size_read(inode);
- lvb.lvb_blocks = inode->i_blocks;
- lvb.lvb_mtime = LTIME_S(inode->i_mtime);
- lvb.lvb_atime = LTIME_S(inode->i_atime);
- lvb.lvb_ctime = LTIME_S(inode->i_ctime);
+ mtime = LTIME_S(inode->i_mtime);
+ atime = LTIME_S(inode->i_atime);
+ ctime = LTIME_S(inode->i_ctime);
cl_object_attr_lock(obj);
rc = cl_object_attr_get(env, obj, attr);
cl_object_attr_unlock(obj);
- if (rc == 0) {
- if (lvb.lvb_atime < attr->cat_atime)
- lvb.lvb_atime = attr->cat_atime;
- if (lvb.lvb_ctime < attr->cat_ctime)
- lvb.lvb_ctime = attr->cat_ctime;
- if (lvb.lvb_mtime < attr->cat_mtime)
- lvb.lvb_mtime = attr->cat_mtime;
+ if (rc != 0)
+ goto out_size_unlock;
- CDEBUG(D_VFSTRACE, DFID " updating i_size %llu\n",
- PFID(&lli->lli_fid), attr->cat_size);
- cl_isize_write_nolock(inode, attr->cat_size);
+ if (atime < attr->cat_atime)
+ atime = attr->cat_atime;
- inode->i_blocks = attr->cat_blocks;
+ if (ctime < attr->cat_ctime)
+ ctime = attr->cat_ctime;
- LTIME_S(inode->i_mtime) = lvb.lvb_mtime;
- LTIME_S(inode->i_atime) = lvb.lvb_atime;
- LTIME_S(inode->i_ctime) = lvb.lvb_ctime;
- }
+ if (mtime < attr->cat_mtime)
+ mtime = attr->cat_mtime;
+
+ CDEBUG(D_VFSTRACE, DFID " updating i_size %llu\n",
+ PFID(&lli->lli_fid), attr->cat_size);
+
+ i_size_write(inode, attr->cat_size);
+
+ inode->i_blocks = attr->cat_blocks;
+
+ LTIME_S(inode->i_mtime) = mtime;
+ LTIME_S(inode->i_atime) = atime;
+ LTIME_S(inode->i_ctime) = ctime;
+
+out_size_unlock:
ll_inode_size_unlock(inode);
return rc;
@@ -1120,47 +1141,48 @@
struct cl_io *io;
ssize_t result;
+ CDEBUG(D_VFSTRACE, "file: %s, type: %d ppos: %llu, count: %zd\n",
+ file->f_path.dentry->d_name.name, iot, *ppos, count);
+
restart:
- io = ccc_env_thread_io(env);
+ io = vvp_env_thread_io(env);
ll_io_init(io, file, iot == CIT_WRITE);
if (cl_io_rw_init(env, io, iot, *ppos, count) == 0) {
struct vvp_io *vio = vvp_env_io(env);
- struct ccc_io *cio = ccc_env_io(env);
int write_mutex_locked = 0;
- cio->cui_fd = LUSTRE_FPRIVATE(file);
- vio->cui_io_subtype = args->via_io_subtype;
+ vio->vui_fd = LUSTRE_FPRIVATE(file);
+ vio->vui_io_subtype = args->via_io_subtype;
- switch (vio->cui_io_subtype) {
+ switch (vio->vui_io_subtype) {
case IO_NORMAL:
- cio->cui_iter = args->u.normal.via_iter;
- cio->cui_iocb = args->u.normal.via_iocb;
+ vio->vui_iter = args->u.normal.via_iter;
+ vio->vui_iocb = args->u.normal.via_iocb;
if ((iot == CIT_WRITE) &&
- !(cio->cui_fd->fd_flags & LL_FILE_GROUP_LOCKED)) {
+ !(vio->vui_fd->fd_flags & LL_FILE_GROUP_LOCKED)) {
if (mutex_lock_interruptible(&lli->
lli_write_mutex)) {
result = -ERESTARTSYS;
goto out;
}
write_mutex_locked = 1;
- } else if (iot == CIT_READ) {
- down_read(&lli->lli_trunc_sem);
}
+ down_read(&lli->lli_trunc_sem);
break;
case IO_SPLICE:
- vio->u.splice.cui_pipe = args->u.splice.via_pipe;
- vio->u.splice.cui_flags = args->u.splice.via_flags;
+ vio->u.splice.vui_pipe = args->u.splice.via_pipe;
+ vio->u.splice.vui_flags = args->u.splice.via_flags;
break;
default:
- CERROR("Unknown IO type - %u\n", vio->cui_io_subtype);
+ CERROR("Unknown IO type - %u\n", vio->vui_io_subtype);
LBUG();
}
result = cl_io_loop(env, io);
+ if (args->via_io_subtype == IO_NORMAL)
+ up_read(&lli->lli_trunc_sem);
if (write_mutex_locked)
mutex_unlock(&lli->lli_write_mutex);
- else if (args->via_io_subtype == IO_NORMAL && iot == CIT_READ)
- up_read(&lli->lli_trunc_sem);
} else {
/* cl_io_rw_init() handled IO */
result = io->ci_result;
@@ -1197,6 +1219,7 @@
fd->fd_write_failed = true;
}
}
+ CDEBUG(D_VFSTRACE, "iot: %d, result: %zd\n", iot, result);
return result;
}
@@ -1212,7 +1235,7 @@
if (IS_ERR(env))
return PTR_ERR(env);
- args = vvp_env_args(env, IO_NORMAL);
+ args = ll_env_args(env, IO_NORMAL);
args->u.normal.via_iter = to;
args->u.normal.via_iocb = iocb;
@@ -1236,7 +1259,7 @@
if (IS_ERR(env))
return PTR_ERR(env);
- args = vvp_env_args(env, IO_NORMAL);
+ args = ll_env_args(env, IO_NORMAL);
args->u.normal.via_iter = from;
args->u.normal.via_iocb = iocb;
@@ -1262,7 +1285,7 @@
if (IS_ERR(env))
return PTR_ERR(env);
- args = vvp_env_args(env, IO_SPLICE);
+ args = ll_env_args(env, IO_SPLICE);
args->u.splice.via_pipe = pipe;
args->u.splice.via_flags = flags;
@@ -1354,7 +1377,8 @@
}
int ll_lov_setstripe_ea_info(struct inode *inode, struct dentry *dentry,
- int flags, struct lov_user_md *lum, int lum_size)
+ __u64 flags, struct lov_user_md *lum,
+ int lum_size)
{
struct lov_stripe_md *lsm = NULL;
struct lookup_intent oit = {.it_op = IT_OPEN, .it_flags = flags};
@@ -1363,8 +1387,8 @@
lsm = ccc_inode_lsm_get(inode);
if (lsm) {
ccc_inode_lsm_put(inode, lsm);
- CDEBUG(D_IOCTL, "stripe already exists for ino %lu\n",
- inode->i_ino);
+ CDEBUG(D_IOCTL, "stripe already exists for inode "DFID"\n",
+ PFID(ll_inode2fid(inode)));
rc = -EEXIST;
goto out;
}
@@ -1478,7 +1502,7 @@
static int ll_lov_setea(struct inode *inode, struct file *file,
unsigned long arg)
{
- int flags = MDS_OPEN_HAS_OBJS | FMODE_WRITE;
+ __u64 flags = MDS_OPEN_HAS_OBJS | FMODE_WRITE;
struct lov_user_md *lump;
int lum_size = sizeof(struct lov_user_md) +
sizeof(struct lov_user_ost_data);
@@ -1512,7 +1536,7 @@
struct lov_user_md_v1 __user *lumv1p = (void __user *)arg;
struct lov_user_md_v3 __user *lumv3p = (void __user *)arg;
int lum_size, rc;
- int flags = FMODE_WRITE;
+ __u64 flags = FMODE_WRITE;
/* first try with v1 which is smaller than v3 */
lum_size = sizeof(struct lov_user_md_v1);
@@ -1561,7 +1585,7 @@
{
struct ll_inode_info *lli = ll_i2info(inode);
struct ll_file_data *fd = LUSTRE_FPRIVATE(file);
- struct ccc_grouplock grouplock;
+ struct ll_grouplock grouplock;
int rc;
if (arg == 0) {
@@ -1575,14 +1599,14 @@
spin_lock(&lli->lli_lock);
if (fd->fd_flags & LL_FILE_GROUP_LOCKED) {
CWARN("group lock already existed with gid %lu\n",
- fd->fd_grouplock.cg_gid);
+ fd->fd_grouplock.lg_gid);
spin_unlock(&lli->lli_lock);
return -EINVAL;
}
- LASSERT(!fd->fd_grouplock.cg_lock);
+ LASSERT(!fd->fd_grouplock.lg_lock);
spin_unlock(&lli->lli_lock);
- rc = cl_get_grouplock(cl_i2info(inode)->lli_clob,
+ rc = cl_get_grouplock(ll_i2info(inode)->lli_clob,
arg, (file->f_flags & O_NONBLOCK), &grouplock);
if (rc)
return rc;
@@ -1608,7 +1632,7 @@
{
struct ll_inode_info *lli = ll_i2info(inode);
struct ll_file_data *fd = LUSTRE_FPRIVATE(file);
- struct ccc_grouplock grouplock;
+ struct ll_grouplock grouplock;
spin_lock(&lli->lli_lock);
if (!(fd->fd_flags & LL_FILE_GROUP_LOCKED)) {
@@ -1616,11 +1640,11 @@
CWARN("no group lock held\n");
return -EINVAL;
}
- LASSERT(fd->fd_grouplock.cg_lock);
+ LASSERT(fd->fd_grouplock.lg_lock);
- if (fd->fd_grouplock.cg_gid != arg) {
+ if (fd->fd_grouplock.lg_gid != arg) {
CWARN("group lock %lu doesn't match current id %lu\n",
- arg, fd->fd_grouplock.cg_gid);
+ arg, fd->fd_grouplock.lg_gid);
spin_unlock(&lli->lli_lock);
return -EINVAL;
}
@@ -1861,11 +1885,12 @@
* This value is computed using stripe object version on OST.
* Version is computed using server side locking.
*
- * @param extent_lock Take extent lock. Not needed if a process is already
- * holding the OST object group locks.
+ * @param sync if do sync on the OST side;
+ * 0: no sync
+ * LL_DV_RD_FLUSH: flush dirty pages, LCK_PR on OSTs
+ * LL_DV_WR_FLUSH: drop all caching pages, LCK_PW on OSTs
*/
-int ll_data_version(struct inode *inode, __u64 *data_version,
- int extent_lock)
+int ll_data_version(struct inode *inode, __u64 *data_version, int flags)
{
struct lov_stripe_md *lsm = NULL;
struct ll_sb_info *sbi = ll_i2sbi(inode);
@@ -1887,7 +1912,7 @@
goto out;
}
- rc = ll_lsm_getattr(lsm, sbi->ll_dt_exp, obdo, 0, extent_lock);
+ rc = ll_lsm_getattr(lsm, sbi->ll_dt_exp, obdo, 0, flags);
if (rc == 0) {
if (!(obdo->o_valid & OBD_MD_FLDATAVERSION))
rc = -EOPNOTSUPP;
@@ -1923,7 +1948,7 @@
}
/* Grab latest data_version and [am]time values */
- rc = ll_data_version(inode, &data_version, 1);
+ rc = ll_data_version(inode, &data_version, LL_DV_WR_FLUSH);
if (rc != 0)
goto out;
@@ -1933,7 +1958,7 @@
goto out;
}
- ll_merge_lvb(env, inode);
+ ll_merge_attr(env, inode);
cl_env_nested_put(&nest, env);
/* Release the file.
@@ -2227,8 +2252,8 @@
struct ll_file_data *fd = LUSTRE_FPRIVATE(file);
int flags, rc;
- CDEBUG(D_VFSTRACE, "VFS Op:inode=%lu/%u(%p),cmd=%x\n", inode->i_ino,
- inode->i_generation, inode, cmd);
+ CDEBUG(D_VFSTRACE, "VFS Op:inode="DFID"(%p),cmd=%x\n",
+ PFID(ll_inode2fid(inode)), inode, cmd);
ll_stats_ops_tally(ll_i2sbi(inode), LPROC_LL_IOCTL, 1);
/* asm-ppc{,64} declares TCGETS, et. al. as type 't' not 'T' */
@@ -2331,9 +2356,8 @@
if (copy_from_user(&idv, (char __user *)arg, sizeof(idv)))
return -EFAULT;
- rc = ll_data_version(inode, &idv.idv_version,
- !(idv.idv_flags & LL_DV_NOFLUSH));
-
+ idv.idv_flags &= LL_DV_RD_FLUSH | LL_DV_WR_FLUSH;
+ rc = ll_data_version(inode, &idv.idv_version, idv.idv_flags);
if (rc == 0 && copy_to_user((char __user *)arg, &idv,
sizeof(idv)))
return -EFAULT;
@@ -2499,7 +2523,7 @@
rc = och->och_flags &
(FMODE_READ | FMODE_WRITE);
unlock_res_and_lock(lock);
- ldlm_lock_put(lock);
+ LDLM_LOCK_PUT(lock);
}
}
mutex_unlock(&lli->lli_och_mutex);
@@ -2537,9 +2561,8 @@
retval = offset + ((origin == SEEK_END) ? i_size_read(inode) :
(origin == SEEK_CUR) ? file->f_pos : 0);
- CDEBUG(D_VFSTRACE, "VFS Op:inode=%lu/%u(%p), to=%llu=%#llx(%d)\n",
- inode->i_ino, inode->i_generation, inode, retval, retval,
- origin);
+ CDEBUG(D_VFSTRACE, "VFS Op:inode="DFID"(%p), to=%llu=%#llx(%d)\n",
+ PFID(ll_inode2fid(inode)), inode, retval, retval, origin);
ll_stats_ops_tally(ll_i2sbi(inode), LPROC_LL_LLSEEK, 1);
if (origin == SEEK_END || origin == SEEK_HOLE || origin == SEEK_DATA) {
@@ -2603,8 +2626,8 @@
if (IS_ERR(env))
return PTR_ERR(env);
- io = ccc_env_thread_io(env);
- io->ci_obj = cl_i2info(inode)->lli_clob;
+ io = vvp_env_thread_io(env);
+ io->ci_obj = ll_i2info(inode)->lli_clob;
io->ci_ignore_layout = ignore_layout;
/* initialize parameters for sync */
@@ -2634,8 +2657,8 @@
struct ptlrpc_request *req;
int rc, err;
- CDEBUG(D_VFSTRACE, "VFS Op:inode=%lu/%u(%p)\n", inode->i_ino,
- inode->i_generation, inode);
+ CDEBUG(D_VFSTRACE, "VFS Op:inode="DFID"(%p)\n",
+ PFID(ll_inode2fid(inode)), inode);
ll_stats_ops_tally(ll_i2sbi(inode), LPROC_LL_FSYNC, 1);
rc = filemap_write_and_wait_range(inode->i_mapping, start, end);
@@ -2693,8 +2716,8 @@
int rc;
int rc2 = 0;
- CDEBUG(D_VFSTRACE, "VFS Op:inode=%lu file_lock=%p\n",
- inode->i_ino, file_lock);
+ CDEBUG(D_VFSTRACE, "VFS Op:inode="DFID" file_lock=%p\n",
+ PFID(ll_inode2fid(inode)), file_lock);
ll_stats_ops_tally(ll_i2sbi(inode), LPROC_LL_FLOCK, 1);
@@ -2777,9 +2800,9 @@
if (IS_ERR(op_data))
return PTR_ERR(op_data);
- CDEBUG(D_DLMTRACE, "inode=%lu, pid=%u, flags=%#llx, mode=%u, start=%llu, end=%llu\n",
- inode->i_ino, flock.l_flock.pid, flags, einfo.ei_mode,
- flock.l_flock.start, flock.l_flock.end);
+ CDEBUG(D_DLMTRACE, "inode="DFID", pid=%u, flags=%#llx, mode=%u, start=%llu, end=%llu\n",
+ PFID(ll_inode2fid(inode)), flock.l_flock.pid, flags,
+ einfo.ei_mode, flock.l_flock.start, flock.l_flock.end);
rc = md_enqueue(sbi->ll_md_exp, &einfo, NULL,
op_data, &lockh, &flock, 0, NULL /* req */, flags);
@@ -2901,8 +2924,8 @@
struct obd_export *exp;
int rc = 0;
- CDEBUG(D_VFSTRACE, "VFS Op:inode=%lu/%u(%p),name=%pd\n",
- inode->i_ino, inode->i_generation, inode, dentry);
+ CDEBUG(D_VFSTRACE, "VFS Op:inode="DFID"(%p),name=%pd\n",
+ PFID(ll_inode2fid(inode)), inode, dentry);
exp = ll_i2mdexp(inode);
@@ -2998,9 +3021,9 @@
/* if object isn't regular file, don't validate size */
if (!S_ISREG(inode->i_mode)) {
- LTIME_S(inode->i_atime) = ll_i2info(inode)->lli_lvb.lvb_atime;
- LTIME_S(inode->i_mtime) = ll_i2info(inode)->lli_lvb.lvb_mtime;
- LTIME_S(inode->i_ctime) = ll_i2info(inode)->lli_lvb.lvb_ctime;
+ LTIME_S(inode->i_atime) = ll_i2info(inode)->lli_atime;
+ LTIME_S(inode->i_mtime) = ll_i2info(inode)->lli_mtime;
+ LTIME_S(inode->i_ctime) = ll_i2info(inode)->lli_ctime;
} else {
/* In case of restore, the MDT has the right size and has
* already send it back without granting the layout lock,
@@ -3124,8 +3147,8 @@
return rc;
}
- CDEBUG(D_VFSTRACE, "VFS Op:inode=%lu/%u(%p), inode mode %x mask %o\n",
- inode->i_ino, inode->i_generation, inode, inode->i_mode, mask);
+ CDEBUG(D_VFSTRACE, "VFS Op:inode="DFID"(%p), inode mode %x mask %o\n",
+ PFID(ll_inode2fid(inode)), inode, inode->i_mode, mask);
if (ll_i2sbi(inode)->ll_flags & LL_SBI_RMT_CLIENT)
return lustre_check_remote_perm(inode, mask);
@@ -3335,10 +3358,10 @@
int rc;
CDEBUG(D_INODE, DFID" LVB_READY=%d l_lvb_data=%p l_lvb_len=%d\n",
- PFID(ll_inode2fid(inode)), !!(lock->l_flags & LDLM_FL_LVB_READY),
+ PFID(ll_inode2fid(inode)), ldlm_is_lvb_ready(lock),
lock->l_lvb_data, lock->l_lvb_len);
- if (lock->l_lvb_data && (lock->l_flags & LDLM_FL_LVB_READY))
+ if (lock->l_lvb_data && ldlm_is_lvb_ready(lock))
return 0;
/* if layout lock was granted right away, the layout is returned
@@ -3415,14 +3438,14 @@
LASSERT(lock);
LASSERT(ldlm_has_layout(lock));
- LDLM_DEBUG(lock, "File %p/"DFID" being reconfigured: %d",
- inode, PFID(&lli->lli_fid), reconf);
+ LDLM_DEBUG(lock, "File "DFID"(%p) being reconfigured: %d",
+ PFID(&lli->lli_fid), inode, reconf);
/* in case this is a caching lock and reinstate with new inode */
md_set_lock_data(sbi->ll_md_exp, &lockh->cookie, inode, NULL);
lock_res_and_lock(lock);
- lvb_ready = !!(lock->l_flags & LDLM_FL_LVB_READY);
+ lvb_ready = ldlm_is_lvb_ready(lock);
unlock_res_and_lock(lock);
/* checking lvb_ready is racy but this is okay. The worst case is
* that multi processes may configure the file on the same time.
@@ -3487,9 +3510,9 @@
/* wait for IO to complete if it's still being used. */
if (wait_layout) {
- CDEBUG(D_INODE, "%s: %p/" DFID " wait for layout reconf.\n",
+ CDEBUG(D_INODE, "%s: "DFID"(%p) wait for layout reconf\n",
ll_get_fsname(inode->i_sb, NULL, 0),
- inode, PFID(&lli->lli_fid));
+ PFID(&lli->lli_fid), inode);
memset(&conf, 0, sizeof(conf));
conf.coc_opc = OBJECT_CONF_WAIT;
@@ -3498,7 +3521,8 @@
if (rc == 0)
rc = -EAGAIN;
- CDEBUG(D_INODE, "file: " DFID " waiting layout return: %d.\n",
+ CDEBUG(D_INODE, "%s: file="DFID" waiting layout return: %d.\n",
+ ll_get_fsname(inode->i_sb, NULL, 0),
PFID(&lli->lli_fid), rc);
}
return rc;
@@ -3571,9 +3595,9 @@
it.it_op = IT_LAYOUT;
lockh.cookie = 0ULL;
- LDLM_DEBUG_NOLOCK("%s: requeue layout lock for file %p/" DFID "",
- ll_get_fsname(inode->i_sb, NULL, 0), inode,
- PFID(&lli->lli_fid));
+ LDLM_DEBUG_NOLOCK("%s: requeue layout lock for file "DFID"(%p)",
+ ll_get_fsname(inode->i_sb, NULL, 0),
+ PFID(&lli->lli_fid), inode);
rc = md_enqueue(sbi->ll_md_exp, &einfo, &it, op_data, &lockh,
NULL, 0, NULL, 0);
@@ -3601,7 +3625,7 @@
/**
* This function send a restore request to the MDT
*/
-int ll_layout_restore(struct inode *inode)
+int ll_layout_restore(struct inode *inode, loff_t offset, __u64 length)
{
struct hsm_user_request *hur;
int len, rc;
@@ -3617,9 +3641,10 @@
hur->hur_request.hr_flags = 0;
memcpy(&hur->hur_user_item[0].hui_fid, &ll_i2info(inode)->lli_fid,
sizeof(hur->hur_user_item[0].hui_fid));
- hur->hur_user_item[0].hui_extent.length = -1;
+ hur->hur_user_item[0].hui_extent.offset = offset;
+ hur->hur_user_item[0].hui_extent.length = length;
hur->hur_request.hr_itemcount = 1;
- rc = obd_iocontrol(LL_IOC_HSM_REQUEST, cl_i2sbi(inode)->ll_md_exp,
+ rc = obd_iocontrol(LL_IOC_HSM_REQUEST, ll_i2sbi(inode)->ll_md_exp,
len, hur, NULL);
kfree(hur);
return rc;
diff --git a/drivers/staging/lustre/lustre/llite/glimpse.c b/drivers/staging/lustre/lustre/llite/glimpse.c
new file mode 100644
index 0000000..d8ea754
--- /dev/null
+++ b/drivers/staging/lustre/lustre/llite/glimpse.c
@@ -0,0 +1,255 @@
+/*
+ * GPL HEADER START
+ *
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 only,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License version 2 for more details (a copy is included
+ * in the LICENSE file that accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 along with this program; If not, see
+ * http://www.sun.com/software/products/lustre/docs/GPLv2.pdf
+ *
+ * Please contact Sun Microsystems, Inc., 4150 Network Circle, Santa Clara,
+ * CA 95054 USA or visit www.sun.com if you need additional information or
+ * have any questions.
+ *
+ * GPL HEADER END
+ */
+/*
+ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.
+ * Use is subject to license terms.
+ *
+ * Copyright (c) 2011, 2012, Intel Corporation.
+ */
+/*
+ * This file is part of Lustre, http://www.lustre.org/
+ * Lustre is a trademark of Sun Microsystems, Inc.
+ *
+ * glimpse code shared between vvp and liblustre (and other Lustre clients in
+ * the future).
+ *
+ * Author: Nikita Danilov <nikita.danilov@sun.com>
+ * Author: Oleg Drokin <oleg.drokin@sun.com>
+ */
+
+#include "../../include/linux/libcfs/libcfs.h"
+#include "../include/obd_class.h"
+#include "../include/obd_support.h"
+#include "../include/obd.h"
+
+#include "../include/lustre_dlm.h"
+#include "../include/lustre_lite.h"
+#include "../include/lustre_mdc.h"
+#include <linux/pagemap.h>
+#include <linux/file.h>
+
+#include "../include/cl_object.h"
+#include "../llite/llite_internal.h"
+
+static const struct cl_lock_descr whole_file = {
+ .cld_start = 0,
+ .cld_end = CL_PAGE_EOF,
+ .cld_mode = CLM_READ
+};
+
+/*
+ * Check whether file has possible unwriten pages.
+ *
+ * \retval 1 file is mmap-ed or has dirty pages
+ * 0 otherwise
+ */
+blkcnt_t dirty_cnt(struct inode *inode)
+{
+ blkcnt_t cnt = 0;
+ struct vvp_object *vob = cl_inode2vvp(inode);
+ void *results[1];
+
+ if (inode->i_mapping)
+ cnt += radix_tree_gang_lookup_tag(&inode->i_mapping->page_tree,
+ results, 0, 1,
+ PAGECACHE_TAG_DIRTY);
+ if (cnt == 0 && atomic_read(&vob->vob_mmap_cnt) > 0)
+ cnt = 1;
+
+ return (cnt > 0) ? 1 : 0;
+}
+
+int cl_glimpse_lock(const struct lu_env *env, struct cl_io *io,
+ struct inode *inode, struct cl_object *clob, int agl)
+{
+ struct ll_inode_info *lli = ll_i2info(inode);
+ const struct lu_fid *fid = lu_object_fid(&clob->co_lu);
+ int result;
+
+ result = 0;
+ if (!(lli->lli_flags & LLIF_MDS_SIZE_LOCK)) {
+ CDEBUG(D_DLMTRACE, "Glimpsing inode " DFID "\n", PFID(fid));
+ if (lli->lli_has_smd) {
+ struct cl_lock *lock = vvp_env_lock(env);
+ struct cl_lock_descr *descr = &lock->cll_descr;
+
+ /* NOTE: this looks like DLM lock request, but it may
+ * not be one. Due to CEF_ASYNC flag (translated
+ * to LDLM_FL_HAS_INTENT by osc), this is
+ * glimpse request, that won't revoke any
+ * conflicting DLM locks held. Instead,
+ * ll_glimpse_callback() will be called on each
+ * client holding a DLM lock against this file,
+ * and resulting size will be returned for each
+ * stripe. DLM lock on [0, EOF] is acquired only
+ * if there were no conflicting locks. If there
+ * were conflicting locks, enqueuing or waiting
+ * fails with -ENAVAIL, but valid inode
+ * attributes are returned anyway.
+ */
+ *descr = whole_file;
+ descr->cld_obj = clob;
+ descr->cld_mode = CLM_READ;
+ descr->cld_enq_flags = CEF_ASYNC | CEF_MUST;
+ if (agl)
+ descr->cld_enq_flags |= CEF_AGL;
+ /*
+ * CEF_ASYNC is used because glimpse sub-locks cannot
+ * deadlock (because they never conflict with other
+ * locks) and, hence, can be enqueued out-of-order.
+ *
+ * CEF_MUST protects glimpse lock from conversion into
+ * a lockless mode.
+ */
+ result = cl_lock_request(env, io, lock);
+ if (result < 0)
+ return result;
+
+ if (!agl) {
+ ll_merge_attr(env, inode);
+ if (i_size_read(inode) > 0 &&
+ inode->i_blocks == 0) {
+ /*
+ * LU-417: Add dirty pages block count
+ * lest i_blocks reports 0, some "cp" or
+ * "tar" may think it's a completely
+ * sparse file and skip it.
+ */
+ inode->i_blocks = dirty_cnt(inode);
+ }
+ }
+ cl_lock_release(env, lock);
+ } else {
+ CDEBUG(D_DLMTRACE, "No objects for inode\n");
+ ll_merge_attr(env, inode);
+ }
+ }
+
+ return result;
+}
+
+static int cl_io_get(struct inode *inode, struct lu_env **envout,
+ struct cl_io **ioout, int *refcheck)
+{
+ struct lu_env *env;
+ struct cl_io *io;
+ struct ll_inode_info *lli = ll_i2info(inode);
+ struct cl_object *clob = lli->lli_clob;
+ int result;
+
+ if (S_ISREG(inode->i_mode)) {
+ env = cl_env_get(refcheck);
+ if (!IS_ERR(env)) {
+ io = vvp_env_thread_io(env);
+ io->ci_obj = clob;
+ *envout = env;
+ *ioout = io;
+ result = 1;
+ } else {
+ result = PTR_ERR(env);
+ }
+ } else {
+ result = 0;
+ }
+ return result;
+}
+
+int cl_glimpse_size0(struct inode *inode, int agl)
+{
+ /*
+ * We don't need ast_flags argument to cl_glimpse_size(), because
+ * osc_lock_enqueue() takes care of the possible deadlock that said
+ * argument was introduced to avoid.
+ */
+ /*
+ * XXX but note that ll_file_seek() passes LDLM_FL_BLOCK_NOWAIT to
+ * cl_glimpse_size(), which doesn't make sense: glimpse locks are not
+ * blocking anyway.
+ */
+ struct lu_env *env = NULL;
+ struct cl_io *io = NULL;
+ int result;
+ int refcheck;
+
+ result = cl_io_get(inode, &env, &io, &refcheck);
+ if (result > 0) {
+again:
+ io->ci_verify_layout = 1;
+ result = cl_io_init(env, io, CIT_MISC, io->ci_obj);
+ if (result > 0)
+ /*
+ * nothing to do for this io. This currently happens
+ * when stripe sub-object's are not yet created.
+ */
+ result = io->ci_result;
+ else if (result == 0)
+ result = cl_glimpse_lock(env, io, inode, io->ci_obj,
+ agl);
+
+ OBD_FAIL_TIMEOUT(OBD_FAIL_GLIMPSE_DELAY, 2);
+ cl_io_fini(env, io);
+ if (unlikely(io->ci_need_restart))
+ goto again;
+ cl_env_put(env, &refcheck);
+ }
+ return result;
+}
+
+int cl_local_size(struct inode *inode)
+{
+ struct lu_env *env = NULL;
+ struct cl_io *io = NULL;
+ struct cl_object *clob;
+ int result;
+ int refcheck;
+
+ if (!ll_i2info(inode)->lli_has_smd)
+ return 0;
+
+ result = cl_io_get(inode, &env, &io, &refcheck);
+ if (result <= 0)
+ return result;
+
+ clob = io->ci_obj;
+ result = cl_io_init(env, io, CIT_MISC, clob);
+ if (result > 0) {
+ result = io->ci_result;
+ } else if (result == 0) {
+ struct cl_lock *lock = vvp_env_lock(env);
+
+ lock->cll_descr = whole_file;
+ lock->cll_descr.cld_enq_flags = CEF_PEEK;
+ lock->cll_descr.cld_obj = clob;
+ result = cl_lock_request(env, io, lock);
+ if (result == 0) {
+ ll_merge_attr(env, inode);
+ cl_lock_release(env, lock);
+ }
+ }
+ cl_io_fini(env, io);
+ cl_env_put(env, &refcheck);
+ return result;
+}
diff --git a/drivers/staging/lustre/lustre/llite/lcommon_cl.c b/drivers/staging/lustre/lustre/llite/lcommon_cl.c
new file mode 100644
index 0000000..6c00715
--- /dev/null
+++ b/drivers/staging/lustre/lustre/llite/lcommon_cl.c
@@ -0,0 +1,327 @@
+/*
+ * GPL HEADER START
+ *
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 only,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License version 2 for more details (a copy is included
+ * in the LICENSE file that accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 along with this program; If not, see
+ * http://www.sun.com/software/products/lustre/docs/GPLv2.pdf
+ *
+ * Please contact Sun Microsystems, Inc., 4150 Network Circle, Santa Clara,
+ * CA 95054 USA or visit www.sun.com if you need additional information or
+ * have any questions.
+ *
+ * GPL HEADER END
+ */
+/*
+ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.
+ * Use is subject to license terms.
+ *
+ * Copyright (c) 2011, 2015, Intel Corporation.
+ */
+/*
+ * This file is part of Lustre, http://www.lustre.org/
+ * Lustre is a trademark of Sun Microsystems, Inc.
+ *
+ * cl code shared between vvp and liblustre (and other Lustre clients in the
+ * future).
+ *
+ * Author: Nikita Danilov <nikita.danilov@sun.com>
+ */
+
+#define DEBUG_SUBSYSTEM S_LLITE
+
+#include "../../include/linux/libcfs/libcfs.h"
+# include <linux/fs.h>
+# include <linux/sched.h>
+# include <linux/mm.h>
+# include <linux/quotaops.h>
+# include <linux/highmem.h>
+# include <linux/pagemap.h>
+# include <linux/rbtree.h>
+
+#include "../include/obd.h"
+#include "../include/obd_support.h"
+#include "../include/lustre_fid.h"
+#include "../include/lustre_lite.h"
+#include "../include/lustre_dlm.h"
+#include "../include/lustre_ver.h"
+#include "../include/lustre_mdc.h"
+#include "../include/cl_object.h"
+
+#include "../llite/llite_internal.h"
+
+/*
+ * ccc_ prefix stands for "Common Client Code".
+ */
+
+/*****************************************************************************
+ *
+ * Vvp device and device type functions.
+ *
+ */
+
+/**
+ * An `emergency' environment used by cl_inode_fini() when cl_env_get()
+ * fails. Access to this environment is serialized by cl_inode_fini_guard
+ * mutex.
+ */
+struct lu_env *cl_inode_fini_env;
+int cl_inode_fini_refcheck;
+
+/**
+ * A mutex serializing calls to slp_inode_fini() under extreme memory
+ * pressure, when environments cannot be allocated.
+ */
+static DEFINE_MUTEX(cl_inode_fini_guard);
+
+int cl_setattr_ost(struct inode *inode, const struct iattr *attr)
+{
+ struct lu_env *env;
+ struct cl_io *io;
+ int result;
+ int refcheck;
+
+ env = cl_env_get(&refcheck);
+ if (IS_ERR(env))
+ return PTR_ERR(env);
+
+ io = vvp_env_thread_io(env);
+ io->ci_obj = ll_i2info(inode)->lli_clob;
+
+ io->u.ci_setattr.sa_attr.lvb_atime = LTIME_S(attr->ia_atime);
+ io->u.ci_setattr.sa_attr.lvb_mtime = LTIME_S(attr->ia_mtime);
+ io->u.ci_setattr.sa_attr.lvb_ctime = LTIME_S(attr->ia_ctime);
+ io->u.ci_setattr.sa_attr.lvb_size = attr->ia_size;
+ io->u.ci_setattr.sa_valid = attr->ia_valid;
+
+again:
+ if (cl_io_init(env, io, CIT_SETATTR, io->ci_obj) == 0) {
+ struct vvp_io *vio = vvp_env_io(env);
+
+ if (attr->ia_valid & ATTR_FILE)
+ /* populate the file descriptor for ftruncate to honor
+ * group lock - see LU-787
+ */
+ vio->vui_fd = LUSTRE_FPRIVATE(attr->ia_file);
+
+ result = cl_io_loop(env, io);
+ } else {
+ result = io->ci_result;
+ }
+ cl_io_fini(env, io);
+ if (unlikely(io->ci_need_restart))
+ goto again;
+ /* HSM import case: file is released, cannot be restored
+ * no need to fail except if restore registration failed
+ * with -ENODATA
+ */
+ if (result == -ENODATA && io->ci_restore_needed &&
+ io->ci_result != -ENODATA)
+ result = 0;
+ cl_env_put(env, &refcheck);
+ return result;
+}
+
+/**
+ * Initialize or update CLIO structures for regular files when new
+ * meta-data arrives from the server.
+ *
+ * \param inode regular file inode
+ * \param md new file metadata from MDS
+ * - allocates cl_object if necessary,
+ * - updated layout, if object was already here.
+ */
+int cl_file_inode_init(struct inode *inode, struct lustre_md *md)
+{
+ struct lu_env *env;
+ struct ll_inode_info *lli;
+ struct cl_object *clob;
+ struct lu_site *site;
+ struct lu_fid *fid;
+ struct cl_object_conf conf = {
+ .coc_inode = inode,
+ .u = {
+ .coc_md = md
+ }
+ };
+ int result = 0;
+ int refcheck;
+
+ LASSERT(md->body->valid & OBD_MD_FLID);
+ LASSERT(S_ISREG(inode->i_mode));
+
+ env = cl_env_get(&refcheck);
+ if (IS_ERR(env))
+ return PTR_ERR(env);
+
+ site = ll_i2sbi(inode)->ll_site;
+ lli = ll_i2info(inode);
+ fid = &lli->lli_fid;
+ LASSERT(fid_is_sane(fid));
+
+ if (!lli->lli_clob) {
+ /* clob is slave of inode, empty lli_clob means for new inode,
+ * there is no clob in cache with the given fid, so it is
+ * unnecessary to perform lookup-alloc-lookup-insert, just
+ * alloc and insert directly.
+ */
+ LASSERT(inode->i_state & I_NEW);
+ conf.coc_lu.loc_flags = LOC_F_NEW;
+ clob = cl_object_find(env, lu2cl_dev(site->ls_top_dev),
+ fid, &conf);
+ if (!IS_ERR(clob)) {
+ /*
+ * No locking is necessary, as new inode is
+ * locked by I_NEW bit.
+ */
+ lli->lli_clob = clob;
+ lli->lli_has_smd = lsm_has_objects(md->lsm);
+ lu_object_ref_add(&clob->co_lu, "inode", inode);
+ } else {
+ result = PTR_ERR(clob);
+ }
+ } else {
+ result = cl_conf_set(env, lli->lli_clob, &conf);
+ }
+
+ cl_env_put(env, &refcheck);
+
+ if (result != 0)
+ CERROR("Failure to initialize cl object " DFID ": %d\n",
+ PFID(fid), result);
+ return result;
+}
+
+/**
+ * Wait for others drop their references of the object at first, then we drop
+ * the last one, which will lead to the object be destroyed immediately.
+ * Must be called after cl_object_kill() against this object.
+ *
+ * The reason we want to do this is: destroying top object will wait for sub
+ * objects being destroyed first, so we can't let bottom layer (e.g. from ASTs)
+ * to initiate top object destroying which may deadlock. See bz22520.
+ */
+static void cl_object_put_last(struct lu_env *env, struct cl_object *obj)
+{
+ struct lu_object_header *header = obj->co_lu.lo_header;
+ wait_queue_t waiter;
+
+ if (unlikely(atomic_read(&header->loh_ref) != 1)) {
+ struct lu_site *site = obj->co_lu.lo_dev->ld_site;
+ struct lu_site_bkt_data *bkt;
+
+ bkt = lu_site_bkt_from_fid(site, &header->loh_fid);
+
+ init_waitqueue_entry(&waiter, current);
+ add_wait_queue(&bkt->lsb_marche_funebre, &waiter);
+
+ while (1) {
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ if (atomic_read(&header->loh_ref) == 1)
+ break;
+ schedule();
+ }
+
+ set_current_state(TASK_RUNNING);
+ remove_wait_queue(&bkt->lsb_marche_funebre, &waiter);
+ }
+
+ cl_object_put(env, obj);
+}
+
+void cl_inode_fini(struct inode *inode)
+{
+ struct lu_env *env;
+ struct ll_inode_info *lli = ll_i2info(inode);
+ struct cl_object *clob = lli->lli_clob;
+ int refcheck;
+ int emergency;
+
+ if (clob) {
+ void *cookie;
+
+ cookie = cl_env_reenter();
+ env = cl_env_get(&refcheck);
+ emergency = IS_ERR(env);
+ if (emergency) {
+ mutex_lock(&cl_inode_fini_guard);
+ LASSERT(cl_inode_fini_env);
+ cl_env_implant(cl_inode_fini_env, &refcheck);
+ env = cl_inode_fini_env;
+ }
+ /*
+ * cl_object cache is a slave to inode cache (which, in turn
+ * is a slave to dentry cache), don't keep cl_object in memory
+ * when its master is evicted.
+ */
+ cl_object_kill(env, clob);
+ lu_object_ref_del(&clob->co_lu, "inode", inode);
+ cl_object_put_last(env, clob);
+ lli->lli_clob = NULL;
+ if (emergency) {
+ cl_env_unplant(cl_inode_fini_env, &refcheck);
+ mutex_unlock(&cl_inode_fini_guard);
+ } else {
+ cl_env_put(env, &refcheck);
+ }
+ cl_env_reexit(cookie);
+ }
+}
+
+/**
+ * build inode number from passed @fid
+ */
+__u64 cl_fid_build_ino(const struct lu_fid *fid, int api32)
+{
+ if (BITS_PER_LONG == 32 || api32)
+ return fid_flatten32(fid);
+ else
+ return fid_flatten(fid);
+}
+
+/**
+ * build inode generation from passed @fid. If our FID overflows the 32-bit
+ * inode number then return a non-zero generation to distinguish them.
+ */
+__u32 cl_fid_build_gen(const struct lu_fid *fid)
+{
+ __u32 gen;
+
+ if (fid_is_igif(fid)) {
+ gen = lu_igif_gen(fid);
+ return gen;
+ }
+
+ gen = fid_flatten(fid) >> 32;
+ return gen;
+}
+
+/* lsm is unreliable after hsm implementation as layout can be changed at
+ * any time. This is only to support old, non-clio-ized interfaces. It will
+ * cause deadlock if clio operations are called with this extra layout refcount
+ * because in case the layout changed during the IO, ll_layout_refresh() will
+ * have to wait for the refcount to become zero to destroy the older layout.
+ *
+ * Notice that the lsm returned by this function may not be valid unless called
+ * inside layout lock - MDS_INODELOCK_LAYOUT.
+ */
+struct lov_stripe_md *ccc_inode_lsm_get(struct inode *inode)
+{
+ return lov_lsm_get(ll_i2info(inode)->lli_clob);
+}
+
+inline void ccc_inode_lsm_put(struct inode *inode, struct lov_stripe_md *lsm)
+{
+ lov_lsm_put(ll_i2info(inode)->lli_clob, lsm);
+}
diff --git a/drivers/staging/lustre/lustre/llite/lcommon_misc.c b/drivers/staging/lustre/lustre/llite/lcommon_misc.c
new file mode 100644
index 0000000..12f3e71
--- /dev/null
+++ b/drivers/staging/lustre/lustre/llite/lcommon_misc.c
@@ -0,0 +1,201 @@
+/*
+ * GPL HEADER START
+ *
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 only,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License version 2 for more details (a copy is included
+ * in the LICENSE file that accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 along with this program; If not, see
+ * http://www.sun.com/software/products/lustre/docs/GPLv2.pdf
+ *
+ * Please contact Sun Microsystems, Inc., 4150 Network Circle, Santa Clara,
+ * CA 95054 USA or visit www.sun.com if you need additional information or
+ * have any questions.
+ *
+ * GPL HEADER END
+ */
+/*
+ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.
+ * Use is subject to license terms.
+ *
+ * Copyright (c) 2011, 2012, Intel Corporation.
+ */
+/*
+ * This file is part of Lustre, http://www.lustre.org/
+ * Lustre is a trademark of Sun Microsystems, Inc.
+ *
+ * cl code shared between vvp and liblustre (and other Lustre clients in the
+ * future).
+ *
+ */
+#include "../include/obd_class.h"
+#include "../include/obd_support.h"
+#include "../include/obd.h"
+#include "../include/cl_object.h"
+
+#include "../include/lustre_lite.h"
+#include "llite_internal.h"
+
+/* Initialize the default and maximum LOV EA and cookie sizes. This allows
+ * us to make MDS RPCs with large enough reply buffers to hold the
+ * maximum-sized (= maximum striped) EA and cookie without having to
+ * calculate this (via a call into the LOV + OSCs) each time we make an RPC.
+ */
+int cl_init_ea_size(struct obd_export *md_exp, struct obd_export *dt_exp)
+{
+ struct lov_stripe_md lsm = { .lsm_magic = LOV_MAGIC_V3 };
+ __u32 valsize = sizeof(struct lov_desc);
+ int rc, easize, def_easize, cookiesize;
+ struct lov_desc desc;
+ __u16 stripes, def_stripes;
+
+ rc = obd_get_info(NULL, dt_exp, sizeof(KEY_LOVDESC), KEY_LOVDESC,
+ &valsize, &desc, NULL);
+ if (rc)
+ return rc;
+
+ stripes = min_t(__u32, desc.ld_tgt_count, LOV_MAX_STRIPE_COUNT);
+ lsm.lsm_stripe_count = stripes;
+ easize = obd_size_diskmd(dt_exp, &lsm);
+
+ def_stripes = min_t(__u32, desc.ld_default_stripe_count,
+ LOV_MAX_STRIPE_COUNT);
+ lsm.lsm_stripe_count = def_stripes;
+ def_easize = obd_size_diskmd(dt_exp, &lsm);
+
+ cookiesize = stripes * sizeof(struct llog_cookie);
+
+ /* default cookiesize is 0 because from 2.4 server doesn't send
+ * llog cookies to client.
+ */
+ CDEBUG(D_HA,
+ "updating def/max_easize: %d/%d def/max_cookiesize: 0/%d\n",
+ def_easize, easize, cookiesize);
+
+ rc = md_init_ea_size(md_exp, easize, def_easize, cookiesize, 0);
+ return rc;
+}
+
+/**
+ * This function is used as an upcall-callback hooked by liblustre and llite
+ * clients into obd_notify() listeners chain to handle notifications about
+ * change of import connect_flags. See llu_fsswop_mount() and
+ * lustre_common_fill_super().
+ */
+int cl_ocd_update(struct obd_device *host,
+ struct obd_device *watched,
+ enum obd_notify_event ev, void *owner, void *data)
+{
+ struct lustre_client_ocd *lco;
+ struct client_obd *cli;
+ __u64 flags;
+ int result;
+
+ if (!strcmp(watched->obd_type->typ_name, LUSTRE_OSC_NAME)) {
+ cli = &watched->u.cli;
+ lco = owner;
+ flags = cli->cl_import->imp_connect_data.ocd_connect_flags;
+ CDEBUG(D_SUPER, "Changing connect_flags: %#llx -> %#llx\n",
+ lco->lco_flags, flags);
+ mutex_lock(&lco->lco_lock);
+ lco->lco_flags &= flags;
+ /* for each osc event update ea size */
+ if (lco->lco_dt_exp)
+ cl_init_ea_size(lco->lco_md_exp, lco->lco_dt_exp);
+
+ mutex_unlock(&lco->lco_lock);
+ result = 0;
+ } else {
+ CERROR("unexpected notification from %s %s!\n",
+ watched->obd_type->typ_name,
+ watched->obd_name);
+ result = -EINVAL;
+ }
+ return result;
+}
+
+#define GROUPLOCK_SCOPE "grouplock"
+
+int cl_get_grouplock(struct cl_object *obj, unsigned long gid, int nonblock,
+ struct ll_grouplock *cg)
+{
+ struct lu_env *env;
+ struct cl_io *io;
+ struct cl_lock *lock;
+ struct cl_lock_descr *descr;
+ __u32 enqflags;
+ int refcheck;
+ int rc;
+
+ env = cl_env_get(&refcheck);
+ if (IS_ERR(env))
+ return PTR_ERR(env);
+
+ io = vvp_env_thread_io(env);
+ io->ci_obj = obj;
+ io->ci_ignore_layout = 1;
+
+ rc = cl_io_init(env, io, CIT_MISC, io->ci_obj);
+ if (rc != 0) {
+ cl_io_fini(env, io);
+ cl_env_put(env, &refcheck);
+ /* Does not make sense to take GL for released layout */
+ if (rc > 0)
+ rc = -ENOTSUPP;
+ return rc;
+ }
+
+ lock = vvp_env_lock(env);
+ descr = &lock->cll_descr;
+ descr->cld_obj = obj;
+ descr->cld_start = 0;
+ descr->cld_end = CL_PAGE_EOF;
+ descr->cld_gid = gid;
+ descr->cld_mode = CLM_GROUP;
+
+ enqflags = CEF_MUST | (nonblock ? CEF_NONBLOCK : 0);
+ descr->cld_enq_flags = enqflags;
+
+ rc = cl_lock_request(env, io, lock);
+ if (rc < 0) {
+ cl_io_fini(env, io);
+ cl_env_put(env, &refcheck);
+ return rc;
+ }
+
+ cg->lg_env = cl_env_get(&refcheck);
+ cg->lg_io = io;
+ cg->lg_lock = lock;
+ cg->lg_gid = gid;
+ LASSERT(cg->lg_env == env);
+
+ cl_env_unplant(env, &refcheck);
+ return 0;
+}
+
+void cl_put_grouplock(struct ll_grouplock *cg)
+{
+ struct lu_env *env = cg->lg_env;
+ struct cl_io *io = cg->lg_io;
+ struct cl_lock *lock = cg->lg_lock;
+ int refcheck;
+
+ LASSERT(cg->lg_env);
+ LASSERT(cg->lg_gid);
+
+ cl_env_implant(env, &refcheck);
+ cl_env_put(env, &refcheck);
+
+ cl_lock_release(env, lock);
+ cl_io_fini(env, io);
+ cl_env_put(env, NULL);
+}
diff --git a/drivers/staging/lustre/lustre/llite/llite_close.c b/drivers/staging/lustre/lustre/llite/llite_close.c
index a55ac4d..2df551d 100644
--- a/drivers/staging/lustre/lustre/llite/llite_close.c
+++ b/drivers/staging/lustre/lustre/llite/llite_close.c
@@ -46,31 +46,31 @@
#include "llite_internal.h"
/** records that a write is in flight */
-void vvp_write_pending(struct ccc_object *club, struct ccc_page *page)
+void vvp_write_pending(struct vvp_object *club, struct vvp_page *page)
{
- struct ll_inode_info *lli = ll_i2info(club->cob_inode);
+ struct ll_inode_info *lli = ll_i2info(club->vob_inode);
spin_lock(&lli->lli_lock);
lli->lli_flags |= LLIF_SOM_DIRTY;
- if (page && list_empty(&page->cpg_pending_linkage))
- list_add(&page->cpg_pending_linkage, &club->cob_pending_list);
+ if (page && list_empty(&page->vpg_pending_linkage))
+ list_add(&page->vpg_pending_linkage, &club->vob_pending_list);
spin_unlock(&lli->lli_lock);
}
/** records that a write has completed */
-void vvp_write_complete(struct ccc_object *club, struct ccc_page *page)
+void vvp_write_complete(struct vvp_object *club, struct vvp_page *page)
{
- struct ll_inode_info *lli = ll_i2info(club->cob_inode);
+ struct ll_inode_info *lli = ll_i2info(club->vob_inode);
int rc = 0;
spin_lock(&lli->lli_lock);
- if (page && !list_empty(&page->cpg_pending_linkage)) {
- list_del_init(&page->cpg_pending_linkage);
+ if (page && !list_empty(&page->vpg_pending_linkage)) {
+ list_del_init(&page->vpg_pending_linkage);
rc = 1;
}
spin_unlock(&lli->lli_lock);
if (rc)
- ll_queue_done_writing(club->cob_inode, 0);
+ ll_queue_done_writing(club->vob_inode, 0);
}
/** Queues DONE_WRITING if
@@ -80,25 +80,25 @@
void ll_queue_done_writing(struct inode *inode, unsigned long flags)
{
struct ll_inode_info *lli = ll_i2info(inode);
- struct ccc_object *club = cl2ccc(ll_i2info(inode)->lli_clob);
+ struct vvp_object *club = cl2vvp(ll_i2info(inode)->lli_clob);
spin_lock(&lli->lli_lock);
lli->lli_flags |= flags;
if ((lli->lli_flags & LLIF_DONE_WRITING) &&
- list_empty(&club->cob_pending_list)) {
+ list_empty(&club->vob_pending_list)) {
struct ll_close_queue *lcq = ll_i2sbi(inode)->ll_lcq;
if (lli->lli_flags & LLIF_MDS_SIZE_LOCK)
- CWARN("ino %lu/%u(flags %u) som valid it just after recovery\n",
- inode->i_ino, inode->i_generation,
- lli->lli_flags);
+ CWARN("%s: file "DFID"(flags %u) Size-on-MDS valid, done writing allowed and no diry pages\n",
+ ll_get_fsname(inode->i_sb, NULL, 0),
+ PFID(ll_inode2fid(inode)), lli->lli_flags);
/* DONE_WRITING is allowed and inode has no dirty page. */
spin_lock(&lcq->lcq_lock);
LASSERT(list_empty(&lli->lli_close_list));
- CDEBUG(D_INODE, "adding inode %lu/%u to close list\n",
- inode->i_ino, inode->i_generation);
+ CDEBUG(D_INODE, "adding inode "DFID" to close list\n",
+ PFID(ll_inode2fid(inode)));
list_add_tail(&lli->lli_close_list, &lcq->lcq_head);
/* Avoid a concurrent insertion into the close thread queue:
@@ -124,9 +124,9 @@
op_data->op_flags |= MF_SOM_CHANGE;
/* Check if Size-on-MDS attributes are valid. */
if (lli->lli_flags & LLIF_MDS_SIZE_LOCK)
- CERROR("ino %lu/%u(flags %u) som valid it just after recovery\n",
- inode->i_ino, inode->i_generation,
- lli->lli_flags);
+ CERROR("%s: inode "DFID"(flags %u) MDS holds lock on Size-on-MDS attributes\n",
+ ll_get_fsname(inode->i_sb, NULL, 0),
+ PFID(ll_inode2fid(inode)), lli->lli_flags);
if (!cl_local_size(inode)) {
/* Send Size-on-MDS Attributes if valid. */
@@ -140,10 +140,10 @@
struct obd_client_handle **och, unsigned long flags)
{
struct ll_inode_info *lli = ll_i2info(inode);
- struct ccc_object *club = cl2ccc(ll_i2info(inode)->lli_clob);
+ struct vvp_object *club = cl2vvp(ll_i2info(inode)->lli_clob);
spin_lock(&lli->lli_lock);
- if (!(list_empty(&club->cob_pending_list))) {
+ if (!(list_empty(&club->vob_pending_list))) {
if (!(lli->lli_flags & LLIF_EPOCH_PENDING)) {
LASSERT(*och);
LASSERT(!lli->lli_pending_och);
@@ -198,7 +198,7 @@
}
}
- LASSERT(list_empty(&club->cob_pending_list));
+ LASSERT(list_empty(&club->vob_pending_list));
lli->lli_flags &= ~LLIF_SOM_DIRTY;
spin_unlock(&lli->lli_lock);
ll_done_writing_attr(inode, op_data);
@@ -221,9 +221,9 @@
LASSERT(op_data);
if (lli->lli_flags & LLIF_MDS_SIZE_LOCK)
- CERROR("ino %lu/%u(flags %u) som valid it just after recovery\n",
- inode->i_ino, inode->i_generation,
- lli->lli_flags);
+ CERROR("%s: inode "DFID"(flags %u) MDS holds lock on Size-on-MDS attributes\n",
+ ll_get_fsname(inode->i_sb, NULL, 0),
+ PFID(ll_inode2fid(inode)), lli->lli_flags);
oa = kmem_cache_zalloc(obdo_cachep, GFP_NOFS);
if (!oa) {
@@ -241,9 +241,9 @@
if (rc) {
oa->o_valid = 0;
if (rc != -ENOENT)
- CERROR("inode_getattr failed (%d): unable to send a Size-on-MDS attribute update for inode %lu/%u\n",
- rc, inode->i_ino,
- inode->i_generation);
+ CERROR("%s: inode_getattr failed - unable to send a Size-on-MDS attribute update for inode "DFID": rc = %d\n",
+ ll_get_fsname(inode->i_sb, NULL, 0),
+ PFID(ll_inode2fid(inode)), rc);
} else {
CDEBUG(D_INODE, "Size-on-MDS update on "DFID"\n",
PFID(&lli->lli_fid));
@@ -302,9 +302,11 @@
* OSTs and send setattr to back to MDS.
*/
rc = ll_som_update(inode, op_data);
- else if (rc)
- CERROR("inode %lu mdc done_writing failed: rc = %d\n",
- inode->i_ino, rc);
+ else if (rc) {
+ CERROR("%s: inode "DFID" mdc done_writing failed: rc = %d\n",
+ ll_get_fsname(inode->i_sb, NULL, 0),
+ PFID(ll_inode2fid(inode)), rc);
+ }
out:
ll_finish_md_op_data(op_data);
if (och) {
@@ -323,8 +325,9 @@
lli = list_entry(lcq->lcq_head.next, struct ll_inode_info,
lli_close_list);
list_del_init(&lli->lli_close_list);
- } else if (atomic_read(&lcq->lcq_stop))
+ } else if (atomic_read(&lcq->lcq_stop)) {
lli = ERR_PTR(-EALREADY);
+ }
spin_unlock(&lcq->lcq_lock);
return lli;
@@ -348,8 +351,8 @@
break;
inode = ll_info2i(lli);
- CDEBUG(D_INFO, "done_writing for inode %lu/%u\n",
- inode->i_ino, inode->i_generation);
+ CDEBUG(D_INFO, "done_writing for inode "DFID"\n",
+ PFID(ll_inode2fid(inode)));
ll_done_writing(inode);
iput(inode);
}
diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
index 65a6ace..ce1f949 100644
--- a/drivers/staging/lustre/lustre/llite/llite_internal.h
+++ b/drivers/staging/lustre/lustre/llite/llite_internal.h
@@ -43,11 +43,11 @@
/* for struct cl_lock_descr and struct cl_io */
#include "../include/cl_object.h"
-#include "../include/lclient.h"
#include "../include/lustre_mdc.h"
#include "../include/lustre_intent.h"
#include <linux/compat.h>
#include <linux/posix_acl_xattr.h>
+#include "vvp_internal.h"
#ifndef FMODE_EXEC
#define FMODE_EXEC 0
@@ -99,6 +99,13 @@
*/
};
+struct ll_grouplock {
+ struct lu_env *lg_env;
+ struct cl_io *lg_io;
+ struct cl_lock *lg_lock;
+ unsigned long lg_gid;
+};
+
enum lli_flags {
/* MDS has an authority for the Size-on-MDS attributes. */
LLIF_MDS_SIZE_LOCK = (1 << 0),
@@ -161,7 +168,9 @@
struct inode lli_vfs_inode;
/* the most recent timestamps obtained from mds */
- struct ost_lvb lli_lvb;
+ s64 lli_atime;
+ s64 lli_mtime;
+ s64 lli_ctime;
spinlock_t lli_agl_lock;
/* Try to make the d::member and f::member are aligned. Before using
@@ -328,6 +337,7 @@
RA_STAT_EOF,
RA_STAT_MAX_IN_FLIGHT,
RA_STAT_WRONG_GRAB_PAGE,
+ RA_STAT_FAILED_REACH_END,
_NR_RA_STAT,
};
@@ -481,6 +491,12 @@
struct lprocfs_stats *ll_stats; /* lprocfs stats counter */
+ /*
+ * Used to track "unstable" pages on a client, and maintain a
+ * LRU list of clean pages. An "unstable" page is defined as
+ * any page which is sent to a server as part of a bulk request,
+ * but is uncommitted to stable storage.
+ */
struct cl_client_cache ll_cache;
struct lprocfs_stats *ll_ra_stats;
@@ -525,13 +541,6 @@
struct completion ll_kobj_unregister;
};
-struct ll_ra_read {
- pgoff_t lrr_start;
- pgoff_t lrr_count;
- struct task_struct *lrr_reader;
- struct list_head lrr_linkage;
-};
-
/*
* per file-descriptor read-ahead data.
*/
@@ -590,12 +599,6 @@
*/
unsigned long ras_request_index;
/*
- * list of struct ll_ra_read's one per read(2) call current in
- * progress against this file descriptor. Used by read-ahead code,
- * protected by ->ras_lock.
- */
- struct list_head ras_read_beads;
- /*
* The following 3 items are used for detecting the stride I/O
* mode.
* In stride I/O mode,
@@ -622,7 +625,7 @@
struct lustre_handle;
struct ll_file_data {
struct ll_readahead_state fd_ras;
- struct ccc_grouplock fd_grouplock;
+ struct ll_grouplock fd_grouplock;
__u64 lfd_pos;
__u32 fd_flags;
fmode_t fd_omode;
@@ -663,8 +666,16 @@
#endif
}
-void ll_ra_read_in(struct file *f, struct ll_ra_read *rar);
-void ll_ra_read_ex(struct file *f, struct ll_ra_read *rar);
+void ll_ras_enter(struct file *f);
+
+/* llite/lcommon_misc.c */
+int cl_init_ea_size(struct obd_export *md_exp, struct obd_export *dt_exp);
+int cl_ocd_update(struct obd_device *host,
+ struct obd_device *watched,
+ enum obd_notify_event ev, void *owner, void *data);
+int cl_get_grouplock(struct cl_object *obj, unsigned long gid, int nonblock,
+ struct ll_grouplock *cg);
+void cl_put_grouplock(struct ll_grouplock *cg);
/* llite/lproc_llite.c */
int ldebugfs_register_mountpoint(struct dentry *parent,
@@ -697,15 +708,15 @@
struct dentry *ll_splice_alias(struct inode *inode, struct dentry *de);
/* llite/rw.c */
-int ll_prepare_write(struct file *, struct page *, unsigned from, unsigned to);
-int ll_commit_write(struct file *, struct page *, unsigned from, unsigned to);
int ll_writepage(struct page *page, struct writeback_control *wbc);
int ll_writepages(struct address_space *, struct writeback_control *wbc);
int ll_readpage(struct file *file, struct page *page);
void ll_readahead_init(struct inode *inode, struct ll_readahead_state *ras);
int ll_readahead(const struct lu_env *env, struct cl_io *io,
- struct ll_readahead_state *ras, struct address_space *mapping,
- struct cl_page_list *queue, int flags);
+ struct cl_page_list *queue, struct ll_readahead_state *ras,
+ bool hit);
+struct ll_cl_context *ll_cl_init(struct file *file, struct page *vmpage);
+void ll_cl_fini(struct ll_cl_context *lcc);
extern const struct address_space_operations ll_aops;
@@ -740,7 +751,7 @@
int ll_inode_permission(struct inode *inode, int mask);
int ll_lov_setstripe_ea_info(struct inode *inode, struct dentry *dentry,
- int flags, struct lov_user_md *lum,
+ __u64 flags, struct lov_user_md *lum,
int lum_size);
int ll_lov_getstripe_ea_info(struct inode *inode, const char *filename,
struct lov_mds_md **lmm, int *lmm_size,
@@ -750,9 +761,9 @@
int ll_dir_getstripe(struct inode *inode, struct lov_mds_md **lmmp,
int *lmm_size, struct ptlrpc_request **request);
int ll_fsync(struct file *file, loff_t start, loff_t end, int data);
-int ll_merge_lvb(const struct lu_env *env, struct inode *inode);
+int ll_merge_attr(const struct lu_env *env, struct inode *inode);
int ll_fid2path(struct inode *inode, void __user *arg);
-int ll_data_version(struct inode *inode, __u64 *data_version, int extent_lock);
+int ll_data_version(struct inode *inode, __u64 *data_version, int flags);
int ll_hsm_release(struct inode *inode);
/* llite/dcache.c */
@@ -824,65 +835,8 @@
atomic_t lcq_stop;
};
-struct ccc_object *cl_inode2ccc(struct inode *inode);
-
-void vvp_write_pending (struct ccc_object *club, struct ccc_page *page);
-void vvp_write_complete(struct ccc_object *club, struct ccc_page *page);
-
-/* specific architecture can implement only part of this list */
-enum vvp_io_subtype {
- /** normal IO */
- IO_NORMAL,
- /** io started from splice_{read|write} */
- IO_SPLICE
-};
-
-/* IO subtypes */
-struct vvp_io {
- /** io subtype */
- enum vvp_io_subtype cui_io_subtype;
-
- union {
- struct {
- struct pipe_inode_info *cui_pipe;
- unsigned int cui_flags;
- } splice;
- struct vvp_fault_io {
- /**
- * Inode modification time that is checked across DLM
- * lock request.
- */
- time64_t ft_mtime;
- struct vm_area_struct *ft_vma;
- /**
- * locked page returned from vvp_io
- */
- struct page *ft_vmpage;
- struct vm_fault_api {
- /**
- * kernel fault info
- */
- struct vm_fault *ft_vmf;
- /**
- * fault API used bitflags for return code.
- */
- unsigned int ft_flags;
- /**
- * check that flags are from filemap_fault
- */
- bool ft_flags_valid;
- } fault;
- } fault;
- } u;
- /**
- * Read-ahead state used by read and page-fault IO contexts.
- */
- struct ll_ra_read cui_bead;
- /**
- * Set when cui_bead has been initialized.
- */
- int cui_ra_window_set;
-};
+void vvp_write_pending(struct vvp_object *club, struct vvp_page *page);
+void vvp_write_complete(struct vvp_object *club, struct vvp_page *page);
/**
* IO arguments for various VFS I/O interfaces.
@@ -911,54 +865,32 @@
int lcc_refcheck;
};
-struct vvp_thread_info {
- struct vvp_io_args vti_args;
- struct ra_io_arg vti_ria;
- struct ll_cl_context vti_io_ctx;
+struct ll_thread_info {
+ struct vvp_io_args lti_args;
+ struct ra_io_arg lti_ria;
+ struct ll_cl_context lti_io_ctx;
};
-static inline struct vvp_thread_info *vvp_env_info(const struct lu_env *env)
+extern struct lu_context_key ll_thread_key;
+static inline struct ll_thread_info *ll_env_info(const struct lu_env *env)
{
- extern struct lu_context_key vvp_key;
- struct vvp_thread_info *info;
+ struct ll_thread_info *lti;
- info = lu_context_key_get(&env->le_ctx, &vvp_key);
- LASSERT(info);
- return info;
+ lti = lu_context_key_get(&env->le_ctx, &ll_thread_key);
+ LASSERT(lti);
+ return lti;
}
-static inline struct vvp_io_args *vvp_env_args(const struct lu_env *env,
- enum vvp_io_subtype type)
+static inline struct vvp_io_args *ll_env_args(const struct lu_env *env,
+ enum vvp_io_subtype type)
{
- struct vvp_io_args *ret = &vvp_env_info(env)->vti_args;
+ struct vvp_io_args *via = &ll_env_info(env)->lti_args;
- ret->via_io_subtype = type;
+ via->via_io_subtype = type;
- return ret;
+ return via;
}
-struct vvp_session {
- struct vvp_io vs_ios;
-};
-
-static inline struct vvp_session *vvp_env_session(const struct lu_env *env)
-{
- extern struct lu_context_key vvp_session_key;
- struct vvp_session *ses;
-
- ses = lu_context_key_get(env->le_ses, &vvp_session_key);
- LASSERT(ses);
- return ses;
-}
-
-static inline struct vvp_io *vvp_env_io(const struct lu_env *env)
-{
- return &vvp_env_session(env)->vs_ios;
-}
-
-int vvp_global_init(void);
-void vvp_global_fini(void);
-
void ll_queue_done_writing(struct inode *inode, unsigned long flags);
void ll_close_thread_shutdown(struct ll_close_queue *lcq);
int ll_close_thread_start(struct ll_close_queue **lcq_ret);
@@ -981,6 +913,10 @@
if (!mapping)
return;
+ /*
+ * truncate_complete_page() calls
+ * a_ops->invalidatepage()->cl_page_delete()->vvp_page_delete().
+ */
ll_teardown_mmaps(mapping, offset, offset + PAGE_SIZE);
truncate_complete_page(mapping, vmpage);
}
@@ -1055,9 +991,6 @@
int ll_update_remote_perm(struct inode *inode, struct mdt_remote_perm *perm);
int lustre_check_remote_perm(struct inode *inode, int mask);
-/* llite/llite_cl.c */
-extern struct lu_device_type vvp_device_type;
-
/**
* Common IO arguments for various VFS I/O interfaces.
*/
@@ -1069,7 +1002,7 @@
struct ll_readahead_state *ras, unsigned long index,
unsigned hit);
void ll_ra_count_put(struct ll_sb_info *sbi, unsigned long len);
-void ll_ra_stats_inc(struct address_space *mapping, enum ra_stat which);
+void ll_ra_stats_inc(struct inode *inode, enum ra_stat which);
/* llite/llite_rmtacl.c */
#ifdef CONFIG_FS_POSIX_ACL
@@ -1163,6 +1096,22 @@
int only_unplug);
void ll_stop_statahead(struct inode *dir, void *key);
+blkcnt_t dirty_cnt(struct inode *inode);
+
+int cl_glimpse_size0(struct inode *inode, int agl);
+int cl_glimpse_lock(const struct lu_env *env, struct cl_io *io,
+ struct inode *inode, struct cl_object *clob, int agl);
+
+static inline int cl_glimpse_size(struct inode *inode)
+{
+ return cl_glimpse_size0(inode, 0);
+}
+
+static inline int cl_agl(struct inode *inode)
+{
+ return cl_glimpse_size0(inode, 1);
+}
+
static inline int ll_glimpse_size(struct inode *inode)
{
struct ll_inode_info *lli = ll_i2info(inode);
@@ -1285,43 +1234,6 @@
void *ll_iocontrol_register(llioc_callback_t cb, int count, unsigned int *cmd);
void ll_iocontrol_unregister(void *magic);
-/* lclient compat stuff */
-#define cl_inode_info ll_inode_info
-#define cl_i2info(info) ll_i2info(info)
-#define cl_inode_mode(inode) ((inode)->i_mode)
-#define cl_i2sbi ll_i2sbi
-
-static inline struct ll_file_data *cl_iattr2fd(struct inode *inode,
- const struct iattr *attr)
-{
- LASSERT(attr->ia_valid & ATTR_FILE);
- return LUSTRE_FPRIVATE(attr->ia_file);
-}
-
-static inline void cl_isize_write_nolock(struct inode *inode, loff_t kms)
-{
- LASSERT(mutex_is_locked(&ll_i2info(inode)->lli_size_mutex));
- i_size_write(inode, kms);
-}
-
-static inline void cl_isize_write(struct inode *inode, loff_t kms)
-{
- ll_inode_size_lock(inode);
- i_size_write(inode, kms);
- ll_inode_size_unlock(inode);
-}
-
-#define cl_isize_read(inode) i_size_read(inode)
-
-static inline int cl_merge_lvb(const struct lu_env *env, struct inode *inode)
-{
- return ll_merge_lvb(env, inode);
-}
-
-#define cl_inode_atime(inode) LTIME_S((inode)->i_atime)
-#define cl_inode_ctime(inode) LTIME_S((inode)->i_ctime)
-#define cl_inode_mtime(inode) LTIME_S((inode)->i_mtime)
-
int cl_sync_file_range(struct inode *inode, loff_t start, loff_t end,
enum cl_fsync_mode mode, int ignore_layout);
@@ -1350,7 +1262,7 @@
int opc = (crt == CRT_READ) ? LPROC_LL_OSC_READ :
LPROC_LL_OSC_WRITE;
- ll_stats_ops_tally(ll_s2sbi(cl2ccc_dev(dev)->cdv_sb), opc, rc);
+ ll_stats_ops_tally(ll_s2sbi(cl2vvp_dev(dev)->vdv_sb), opc, rc);
}
ssize_t ll_direct_rw_pages(const struct lu_env *env, struct cl_io *io,
@@ -1382,18 +1294,16 @@
*/
if (it->d.lustre.it_remote_lock_mode) {
handle.cookie = it->d.lustre.it_remote_lock_handle;
- CDEBUG(D_DLMTRACE, "setting l_data to inode %p(%lu/%u) for remote lock %#llx\n",
- inode,
- inode->i_ino, inode->i_generation,
+ CDEBUG(D_DLMTRACE, "setting l_data to inode "DFID"%p for remote lock %#llx\n",
+ PFID(ll_inode2fid(inode)), inode,
handle.cookie);
md_set_lock_data(exp, &handle.cookie, inode, NULL);
}
handle.cookie = it->d.lustre.it_lock_handle;
- CDEBUG(D_DLMTRACE, "setting l_data to inode %p (%lu/%u) for lock %#llx\n",
- inode, inode->i_ino,
- inode->i_generation, handle.cookie);
+ CDEBUG(D_DLMTRACE, "setting l_data to inode "DFID"%p for lock %#llx\n",
+ PFID(ll_inode2fid(inode)), inode, handle.cookie);
md_set_lock_data(exp, &handle.cookie, inode,
&it->d.lustre.it_lock_bits);
@@ -1471,9 +1381,25 @@
int ll_layout_conf(struct inode *inode, const struct cl_object_conf *conf);
int ll_layout_refresh(struct inode *inode, __u32 *gen);
-int ll_layout_restore(struct inode *inode);
+int ll_layout_restore(struct inode *inode, loff_t start, __u64 length);
int ll_xattr_init(void);
void ll_xattr_fini(void);
+int ll_page_sync_io(const struct lu_env *env, struct cl_io *io,
+ struct cl_page *page, enum cl_req_type crt);
+
+/* lcommon_cl.c */
+int cl_setattr_ost(struct inode *inode, const struct iattr *attr);
+
+extern struct lu_env *cl_inode_fini_env;
+extern int cl_inode_fini_refcheck;
+
+int cl_file_inode_init(struct inode *inode, struct lustre_md *md);
+void cl_inode_fini(struct inode *inode);
+int cl_local_size(struct inode *inode);
+
+__u64 cl_fid_build_ino(const struct lu_fid *fid, int api32);
+__u32 cl_fid_build_gen(const struct lu_fid *fid);
+
#endif /* LLITE_INTERNAL_H */
diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index b57a992..96c7e9f 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -85,18 +85,18 @@
si_meminfo(&si);
pages = si.totalram - si.totalhigh;
- if (pages >> (20 - PAGE_SHIFT) < 512)
- lru_page_max = pages / 2;
- else
- lru_page_max = (pages / 4) * 3;
+ lru_page_max = pages / 2;
- /* initialize lru data */
+ /* initialize ll_cache data */
atomic_set(&sbi->ll_cache.ccc_users, 0);
sbi->ll_cache.ccc_lru_max = lru_page_max;
atomic_set(&sbi->ll_cache.ccc_lru_left, lru_page_max);
spin_lock_init(&sbi->ll_cache.ccc_lru_lock);
INIT_LIST_HEAD(&sbi->ll_cache.ccc_lru);
+ atomic_set(&sbi->ll_cache.ccc_unstable_nr, 0);
+ init_waitqueue_head(&sbi->ll_cache.ccc_unstable_waitq);
+
sbi->ll_ra_info.ra_max_pages_per_file = min(pages / 32,
SBI_DEFAULT_READAHEAD_MAX);
sbi->ll_ra_info.ra_max_pages = sbi->ll_ra_info.ra_max_pages_per_file;
@@ -169,12 +169,6 @@
return -ENOMEM;
}
- if (llite_root) {
- err = ldebugfs_register_mountpoint(llite_root, sb, dt, md);
- if (err < 0)
- CERROR("could not register mount in <debugfs>/lustre/llite\n");
- }
-
/* indicate the features supported by this client */
data->ocd_connect_flags = OBD_CONNECT_IBITS | OBD_CONNECT_NODEVOH |
OBD_CONNECT_ATTRFID |
@@ -337,10 +331,8 @@
else
sbi->ll_md_brw_size = PAGE_SIZE;
- if (data->ocd_connect_flags & OBD_CONNECT_LAYOUTLOCK) {
- LCONSOLE_INFO("Layout lock feature supported.\n");
+ if (data->ocd_connect_flags & OBD_CONNECT_LAYOUTLOCK)
sbi->ll_flags |= LL_SBI_LAYOUT_LOCK;
- }
if (data->ocd_ibits_known & MDS_INODELOCK_XATTR) {
if (!(data->ocd_connect_flags & OBD_CONNECT_MAX_EASIZE)) {
@@ -453,7 +445,7 @@
/* make root inode
* XXX: move this to after cbd setup?
*/
- valid = OBD_MD_FLGETATTR | OBD_MD_FLBLOCKS;
+ valid = OBD_MD_FLGETATTR | OBD_MD_FLBLOCKS | OBD_MD_FLMODEASIZE;
if (sbi->ll_flags & LL_SBI_RMT_CLIENT)
valid |= OBD_MD_FLRMTPERM;
else if (sbi->ll_flags & LL_SBI_ACL)
@@ -555,6 +547,15 @@
kfree(data);
kfree(osfs);
+ if (llite_root) {
+ err = ldebugfs_register_mountpoint(llite_root, sb, dt, md);
+ if (err < 0) {
+ CERROR("%s: could not register mount in debugfs: "
+ "rc = %d\n", ll_get_fsname(sb, NULL, 0), err);
+ err = 0;
+ }
+ }
+
return err;
out_root:
iput(root);
@@ -573,7 +574,6 @@
out:
kfree(data);
kfree(osfs);
- ldebugfs_unregister_mountpoint(sbi);
return err;
}
@@ -897,10 +897,8 @@
cfg->cfg_callback = class_config_llog_handler;
/* set up client obds */
err = lustre_process_log(sb, profilenm, cfg);
- if (err < 0) {
- CERROR("Unable to process log: %d\n", err);
+ if (err < 0)
goto out_free;
- }
/* Profile set with LCFG_MOUNTOPT so we can find our mdc and osc obds */
lprof = class_get_profile(profilenm);
@@ -947,7 +945,7 @@
struct lustre_sb_info *lsi = s2lsi(sb);
struct ll_sb_info *sbi = ll_s2sbi(sb);
char *profilenm = get_profile_name(sb);
- int next, force = 1;
+ int ccc_count, next, force = 1, rc = 0;
CDEBUG(D_VFSTRACE, "VFS Op: sb %p - %s\n", sb, profilenm);
@@ -963,6 +961,19 @@
force = obd->obd_force;
}
+ /* Wait for unstable pages to be committed to stable storage */
+ if (!force) {
+ struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL);
+
+ rc = l_wait_event(sbi->ll_cache.ccc_unstable_waitq,
+ !atomic_read(&sbi->ll_cache.ccc_unstable_nr),
+ &lwi);
+ }
+
+ ccc_count = atomic_read(&sbi->ll_cache.ccc_unstable_nr);
+ if (!force && rc != -EINTR)
+ LASSERTF(!ccc_count, "count: %i\n", ccc_count);
+
/* We need to set force before the lov_disconnect in
* lustre_common_put_super, since l_d cleans up osc's as well.
*/
@@ -999,6 +1010,8 @@
lustre_common_put_super(sb);
+ cl_env_cache_purge(~0);
+
module_put(THIS_MODULE);
} /* client_put_super */
@@ -1032,8 +1045,8 @@
struct ll_inode_info *lli = ll_i2info(inode);
struct ll_sb_info *sbi = ll_i2sbi(inode);
- CDEBUG(D_VFSTRACE, "VFS Op:inode=%lu/%u(%p)\n", inode->i_ino,
- inode->i_generation, inode);
+ CDEBUG(D_VFSTRACE, "VFS Op:inode="DFID"(%p)\n",
+ PFID(ll_inode2fid(inode)), inode);
if (S_ISDIR(inode->i_mode)) {
/* these should have been cleared in ll_file_release */
@@ -1180,9 +1193,11 @@
* from OSTs and send setattr to back to MDS.
*/
rc = ll_som_update(inode, op_data);
- else if (rc)
- CERROR("inode %lu mdc truncate failed: rc = %d\n",
- inode->i_ino, rc);
+ else if (rc) {
+ CERROR("%s: inode "DFID" mdc truncate failed: rc = %d\n",
+ ll_i2sbi(inode)->ll_md_exp->exp_obd->obd_name,
+ PFID(ll_inode2fid(inode)), rc);
+ }
return rc;
}
@@ -1210,12 +1225,9 @@
bool file_is_released = false;
int rc = 0, rc1 = 0;
- CDEBUG(D_VFSTRACE,
- "%s: setattr inode %p/fid:" DFID
- " from %llu to %llu, valid %x, hsm_import %d\n",
- ll_get_fsname(inode->i_sb, NULL, 0), inode,
- PFID(&lli->lli_fid), i_size_read(inode), attr->ia_size,
- attr->ia_valid, hsm_import);
+ CDEBUG(D_VFSTRACE, "%s: setattr inode "DFID"(%p) from %llu to %llu, valid %x, hsm_import %d\n",
+ ll_get_fsname(inode->i_sb, NULL, 0), PFID(&lli->lli_fid), inode,
+ i_size_read(inode), attr->ia_size, attr->ia_valid, hsm_import);
if (attr->ia_valid & ATTR_SIZE) {
/* Check new size against VFS/VM file size limit and rlimit */
@@ -1265,14 +1277,6 @@
LTIME_S(attr->ia_mtime), LTIME_S(attr->ia_ctime),
(s64)ktime_get_real_seconds());
- /* If we are changing file size, file content is modified, flag it. */
- if (attr->ia_valid & ATTR_SIZE) {
- attr->ia_valid |= MDS_OPEN_OWNEROVERRIDE;
- spin_lock(&lli->lli_lock);
- lli->lli_flags |= LLIF_DATA_MODIFIED;
- spin_unlock(&lli->lli_lock);
- }
-
/* We always do an MDS RPC, even if we're only changing the size;
* only the MDS knows whether truncate() should fail with -ETXTBUSY
*/
@@ -1284,13 +1288,6 @@
if (!S_ISDIR(inode->i_mode))
inode_unlock(inode);
- memcpy(&op_data->op_attr, attr, sizeof(*attr));
-
- /* Open epoch for truncate. */
- if (exp_connect_som(ll_i2mdexp(inode)) &&
- (attr->ia_valid & (ATTR_SIZE | ATTR_MTIME | ATTR_MTIME_SET)))
- op_data->op_flags = MF_EPOCH_OPEN;
-
/* truncate on a released file must failed with -ENODATA,
* so size must not be set on MDS for released file
* but other attributes must be set
@@ -1304,29 +1301,40 @@
if (lsm && lsm->lsm_pattern & LOV_PATTERN_F_RELEASED)
file_is_released = true;
ccc_inode_lsm_put(inode, lsm);
+
+ if (!hsm_import && attr->ia_valid & ATTR_SIZE) {
+ if (file_is_released) {
+ rc = ll_layout_restore(inode, 0, attr->ia_size);
+ if (rc < 0)
+ goto out;
+
+ file_is_released = false;
+ ll_layout_refresh(inode, &gen);
+ }
+
+ /*
+ * If we are changing file size, file content is
+ * modified, flag it.
+ */
+ attr->ia_valid |= MDS_OPEN_OWNEROVERRIDE;
+ spin_lock(&lli->lli_lock);
+ lli->lli_flags |= LLIF_DATA_MODIFIED;
+ spin_unlock(&lli->lli_lock);
+ op_data->op_bias |= MDS_DATA_MODIFIED;
+ }
}
- /* if not in HSM import mode, clear size attr for released file
- * we clear the attribute send to MDT in op_data, not the original
- * received from caller in attr which is used later to
- * decide return code
- */
- if (file_is_released && (attr->ia_valid & ATTR_SIZE) && !hsm_import)
- op_data->op_attr.ia_valid &= ~ATTR_SIZE;
+ memcpy(&op_data->op_attr, attr, sizeof(*attr));
+
+ /* Open epoch for truncate. */
+ if (exp_connect_som(ll_i2mdexp(inode)) && !hsm_import &&
+ (attr->ia_valid & (ATTR_SIZE | ATTR_MTIME | ATTR_MTIME_SET)))
+ op_data->op_flags = MF_EPOCH_OPEN;
rc = ll_md_setattr(dentry, op_data, &mod);
if (rc)
goto out;
- /* truncate failed (only when non HSM import), others succeed */
- if (file_is_released) {
- if ((attr->ia_valid & ATTR_SIZE) && !hsm_import)
- rc = -ENODATA;
- else
- rc = 0;
- goto out;
- }
-
/* RPC to MDT is sent, cancel data modification flag */
if (op_data->op_bias & MDS_DATA_MODIFIED) {
spin_lock(&lli->lli_lock);
@@ -1335,7 +1343,7 @@
}
ll_ioepoch_open(lli, op_data->op_ioepoch);
- if (!S_ISREG(inode->i_mode)) {
+ if (!S_ISREG(inode->i_mode) || file_is_released) {
rc = 0;
goto out;
}
@@ -1552,7 +1560,7 @@
if (body->valid & OBD_MD_FLATIME) {
if (body->atime > LTIME_S(inode->i_atime))
LTIME_S(inode->i_atime) = body->atime;
- lli->lli_lvb.lvb_atime = body->atime;
+ lli->lli_atime = body->atime;
}
if (body->valid & OBD_MD_FLMTIME) {
if (body->mtime > LTIME_S(inode->i_mtime)) {
@@ -1561,12 +1569,12 @@
body->mtime);
LTIME_S(inode->i_mtime) = body->mtime;
}
- lli->lli_lvb.lvb_mtime = body->mtime;
+ lli->lli_mtime = body->mtime;
}
if (body->valid & OBD_MD_FLCTIME) {
if (body->ctime > LTIME_S(inode->i_ctime))
LTIME_S(inode->i_ctime) = body->ctime;
- lli->lli_lvb.lvb_ctime = body->ctime;
+ lli->lli_ctime = body->ctime;
}
if (body->valid & OBD_MD_FLMODE)
inode->i_mode = (inode->i_mode & S_IFMT)|(body->mode & ~S_IFMT);
@@ -1593,12 +1601,12 @@
/* FID shouldn't be changed! */
if (fid_is_sane(&lli->lli_fid)) {
LASSERTF(lu_fid_eq(&lli->lli_fid, &body->fid1),
- "Trying to change FID "DFID
- " to the "DFID", inode %lu/%u(%p)\n",
+ "Trying to change FID "DFID" to the "DFID", inode "DFID"(%p)\n",
PFID(&lli->lli_fid), PFID(&body->fid1),
- inode->i_ino, inode->i_generation, inode);
- } else
+ PFID(ll_inode2fid(inode)), inode);
+ } else {
lli->lli_fid = body->fid1;
+ }
}
LASSERT(fid_seq(&lli->lli_fid) != 0);
@@ -1622,8 +1630,10 @@
if (lli->lli_flags & (LLIF_DONE_WRITING |
LLIF_EPOCH_PENDING |
LLIF_SOM_DIRTY)) {
- CERROR("ino %lu flags %u still has size authority! do not trust the size got from MDS\n",
- inode->i_ino, lli->lli_flags);
+ CERROR("%s: inode "DFID" flags %u still has size authority! do not trust the size got from MDS\n",
+ sbi->ll_md_exp->exp_obd->obd_name,
+ PFID(ll_inode2fid(inode)),
+ lli->lli_flags);
} else {
/* Use old size assignment to avoid
* deadlock bz14138 & bz14326
@@ -1699,7 +1709,7 @@
void ll_delete_inode(struct inode *inode)
{
- struct cl_inode_info *lli = cl_i2info(inode);
+ struct ll_inode_info *lli = ll_i2info(inode);
if (S_ISREG(inode->i_mode) && lli->lli_clob)
/* discard all dirty pages before truncating them, required by
@@ -1715,8 +1725,8 @@
spin_lock_irq(&inode->i_data.tree_lock);
spin_unlock_irq(&inode->i_data.tree_lock);
LASSERTF(inode->i_data.nrpages == 0,
- "inode=%lu/%u(%p) nrpages=%lu, see http://jira.whamcloud.com/browse/LU-118\n",
- inode->i_ino, inode->i_generation, inode,
+ "inode="DFID"(%p) nrpages=%lu, see http://jira.whamcloud.com/browse/LU-118\n",
+ PFID(ll_inode2fid(inode)), inode,
inode->i_data.nrpages);
}
/* Workaround end */
@@ -1747,7 +1757,9 @@
rc = md_getattr(sbi->ll_md_exp, op_data, &req);
ll_finish_md_op_data(op_data);
if (rc) {
- CERROR("failure %d inode %lu\n", rc, inode->i_ino);
+ CERROR("%s: failure inode "DFID": rc = %d\n",
+ sbi->ll_md_exp->exp_obd->obd_name,
+ PFID(ll_inode2fid(inode)), rc);
return -abs(rc);
}
@@ -1772,7 +1784,7 @@
if (IS_ERR(op_data))
return PTR_ERR(op_data);
- ((struct ll_iattr *)&op_data->op_attr)->ia_attr_flags = flags;
+ op_data->op_attr_flags = flags;
op_data->op_attr.ia_valid |= ATTR_ATTR_FLAG;
rc = md_setattr(sbi->ll_md_exp, op_data,
NULL, 0, NULL, 0, &req, NULL);
@@ -2066,11 +2078,11 @@
}
memcpy(&type, data->ioc_inlbuf1, sizeof(__u32));
- if (type & LL_STATFS_LMV)
+ if (type & LL_STATFS_LMV) {
exp = sbi->ll_md_exp;
- else if (type & LL_STATFS_LOV)
+ } else if (type & LL_STATFS_LOV) {
exp = sbi->ll_dt_exp;
- else {
+ } else {
rc = -ENODEV;
goto out_statfs;
}
@@ -2271,7 +2283,7 @@
{
char *buf, *path = NULL;
struct dentry *dentry = NULL;
- struct ccc_object *obj = cl_inode2ccc(page->mapping->host);
+ struct vvp_object *obj = cl_inode2vvp(page->mapping->host);
/* this can be called inside spin lock so use GFP_ATOMIC. */
buf = (char *)__get_free_page(GFP_ATOMIC);
@@ -2285,7 +2297,7 @@
"%s: dirty page discard: %s/fid: " DFID "/%s may get corrupted (rc %d)\n",
ll_get_fsname(page->mapping->host->i_sb, NULL, 0),
s2lsi(page->mapping->host->i_sb)->lsi_lmd->lmd_dev,
- PFID(&obj->cob_header.coh_lu.loh_fid),
+ PFID(&obj->vob_header.coh_lu.loh_fid),
(path && !IS_ERR(path)) ? path : "", ioret);
if (dentry)
diff --git a/drivers/staging/lustre/lustre/llite/llite_mmap.c b/drivers/staging/lustre/lustre/llite/llite_mmap.c
index 5b484e6..88ef1ca 100644
--- a/drivers/staging/lustre/lustre/llite/llite_mmap.c
+++ b/drivers/staging/lustre/lustre/llite/llite_mmap.c
@@ -57,10 +57,10 @@
struct vm_area_struct *vma, unsigned long addr,
size_t count)
{
- policy->l_extent.start = ((addr - vma->vm_start) & CFS_PAGE_MASK) +
+ policy->l_extent.start = ((addr - vma->vm_start) & PAGE_MASK) +
(vma->vm_pgoff << PAGE_SHIFT);
policy->l_extent.end = (policy->l_extent.start + count - 1) |
- ~CFS_PAGE_MASK;
+ ~PAGE_MASK;
}
struct vm_area_struct *our_vma(struct mm_struct *mm, unsigned long addr,
@@ -123,7 +123,8 @@
*env_ret = env;
- io = ccc_env_thread_io(env);
+restart:
+ io = vvp_env_thread_io(env);
io->ci_obj = ll_i2info(inode)->lli_clob;
LASSERT(io->ci_obj);
@@ -146,17 +147,20 @@
rc = cl_io_init(env, io, CIT_FAULT, io->ci_obj);
if (rc == 0) {
- struct ccc_io *cio = ccc_env_io(env);
+ struct vvp_io *vio = vvp_env_io(env);
struct ll_file_data *fd = LUSTRE_FPRIVATE(file);
- LASSERT(cio->cui_cl.cis_io == io);
+ LASSERT(vio->vui_cl.cis_io == io);
/* mmap lock must be MANDATORY it has to cache pages. */
io->ci_lockreq = CILR_MANDATORY;
- cio->cui_fd = fd;
+ vio->vui_fd = fd;
} else {
LASSERT(rc < 0);
cl_io_fini(env, io);
+ if (io->ci_need_restart)
+ goto restart;
+
cl_env_nested_put(nest, env);
io = ERR_PTR(rc);
}
@@ -200,7 +204,7 @@
* Otherwise, we could add dirty pages into osc cache
* while truncate is on-going.
*/
- inode = ccc_object_inode(io->ci_obj);
+ inode = vvp_object_inode(io->ci_obj);
lli = ll_i2info(inode);
down_read(&lli->lli_trunc_sem);
@@ -307,17 +311,17 @@
vio = vvp_env_io(env);
vio->u.fault.ft_vma = vma;
vio->u.fault.ft_vmpage = NULL;
- vio->u.fault.fault.ft_vmf = vmf;
- vio->u.fault.fault.ft_flags = 0;
- vio->u.fault.fault.ft_flags_valid = false;
+ vio->u.fault.ft_vmf = vmf;
+ vio->u.fault.ft_flags = 0;
+ vio->u.fault.ft_flags_valid = false;
result = cl_io_loop(env, io);
/* ft_flags are only valid if we reached
* the call to filemap_fault
*/
- if (vio->u.fault.fault.ft_flags_valid)
- fault_ret = vio->u.fault.fault.ft_flags;
+ if (vio->u.fault.ft_flags_valid)
+ fault_ret = vio->u.fault.ft_flags;
vmpage = vio->u.fault.ft_vmpage;
if (result != 0 && vmpage) {
@@ -390,9 +394,11 @@
result = ll_page_mkwrite0(vma, vmf->page, &retry);
if (!printed && ++count > 16) {
- CWARN("app(%s): the page %lu of file %lu is under heavy contention.\n",
+ const struct dentry *de = vma->vm_file->f_path.dentry;
+
+ CWARN("app(%s): the page %lu of file "DFID" is under heavy contention\n",
current->comm, vmf->pgoff,
- file_inode(vma->vm_file)->i_ino);
+ PFID(ll_inode2fid(de->d_inode)));
printed = true;
}
} while (retry);
@@ -422,16 +428,16 @@
/**
* To avoid cancel the locks covering mmapped region for lock cache pressure,
- * we track the mapped vma count in ccc_object::cob_mmap_cnt.
+ * we track the mapped vma count in vvp_object::vob_mmap_cnt.
*/
static void ll_vm_open(struct vm_area_struct *vma)
{
struct inode *inode = file_inode(vma->vm_file);
- struct ccc_object *vob = cl_inode2ccc(inode);
+ struct vvp_object *vob = cl_inode2vvp(inode);
LASSERT(vma->vm_file);
- LASSERT(atomic_read(&vob->cob_mmap_cnt) >= 0);
- atomic_inc(&vob->cob_mmap_cnt);
+ LASSERT(atomic_read(&vob->vob_mmap_cnt) >= 0);
+ atomic_inc(&vob->vob_mmap_cnt);
}
/**
@@ -440,11 +446,11 @@
static void ll_vm_close(struct vm_area_struct *vma)
{
struct inode *inode = file_inode(vma->vm_file);
- struct ccc_object *vob = cl_inode2ccc(inode);
+ struct vvp_object *vob = cl_inode2vvp(inode);
LASSERT(vma->vm_file);
- atomic_dec(&vob->cob_mmap_cnt);
- LASSERT(atomic_read(&vob->cob_mmap_cnt) >= 0);
+ atomic_dec(&vob->vob_mmap_cnt);
+ LASSERT(atomic_read(&vob->vob_mmap_cnt) >= 0);
}
/* XXX put nice comment here. talk about __free_pte -> dirty pages and
diff --git a/drivers/staging/lustre/lustre/llite/llite_nfs.c b/drivers/staging/lustre/lustre/llite/llite_nfs.c
index 193aab8..c1eef61 100644
--- a/drivers/staging/lustre/lustre/llite/llite_nfs.c
+++ b/drivers/staging/lustre/lustre/llite/llite_nfs.c
@@ -119,7 +119,7 @@
rc = md_getattr(sbi->ll_md_exp, op_data, &req);
kfree(op_data);
if (rc) {
- CERROR("can't get object attrs, fid "DFID", rc %d\n",
+ CDEBUG(D_INFO, "can't get object attrs, fid "DFID", rc %d\n",
PFID(fid), rc);
return ERR_PTR(rc);
}
@@ -191,8 +191,9 @@
int fileid_len = sizeof(struct lustre_nfs_fid) / 4;
struct lustre_nfs_fid *nfs_fid = (void *)fh;
- CDEBUG(D_INFO, "encoding for (%lu," DFID ") maxlen=%d minlen=%d\n",
- inode->i_ino, PFID(ll_inode2fid(inode)), *plen, fileid_len);
+ CDEBUG(D_INFO, "%s: encoding for ("DFID") maxlen=%d minlen=%d\n",
+ ll_get_fsname(inode->i_sb, NULL, 0),
+ PFID(ll_inode2fid(inode)), *plen, fileid_len);
if (*plen < fileid_len) {
*plen = fileid_len;
@@ -298,8 +299,9 @@
sbi = ll_s2sbi(dir->i_sb);
- CDEBUG(D_INFO, "getting parent for (%lu," DFID ")\n",
- dir->i_ino, PFID(ll_inode2fid(dir)));
+ CDEBUG(D_INFO, "%s: getting parent for ("DFID")\n",
+ ll_get_fsname(dir->i_sb, NULL, 0),
+ PFID(ll_inode2fid(dir)));
rc = ll_get_default_mdsize(sbi, &lmmsize);
if (rc != 0)
@@ -314,15 +316,20 @@
rc = md_getattr_name(sbi->ll_md_exp, op_data, &req);
ll_finish_md_op_data(op_data);
if (rc) {
- CERROR("failure %d inode %lu get parent\n", rc, dir->i_ino);
+ CERROR("%s: failure inode "DFID" get parent: rc = %d\n",
+ ll_get_fsname(dir->i_sb, NULL, 0),
+ PFID(ll_inode2fid(dir)), rc);
return ERR_PTR(rc);
}
body = req_capsule_server_get(&req->rq_pill, &RMF_MDT_BODY);
- LASSERT(body->valid & OBD_MD_FLID);
-
- CDEBUG(D_INFO, "parent for " DFID " is " DFID "\n",
- PFID(ll_inode2fid(dir)), PFID(&body->fid1));
-
+ /*
+ * LU-3952: MDT may lost the FID of its parent, we should not crash
+ * the NFS server, ll_iget_for_nfs() will handle the error.
+ */
+ if (body->valid & OBD_MD_FLID) {
+ CDEBUG(D_INFO, "parent for " DFID " is " DFID "\n",
+ PFID(ll_inode2fid(dir)), PFID(&body->fid1));
+ }
result = ll_iget_for_nfs(dir->i_sb, &body->fid1, NULL);
ptlrpc_req_finished(req);
diff --git a/drivers/staging/lustre/lustre/llite/lloop.c b/drivers/staging/lustre/lustre/llite/lloop.c
index f169c0d..813a9a3 100644
--- a/drivers/staging/lustre/lustre/llite/lloop.c
+++ b/drivers/staging/lustre/lustre/llite/lloop.c
@@ -274,8 +274,9 @@
if (lo->lo_biotail) {
lo->lo_biotail->bi_next = bio;
lo->lo_biotail = bio;
- } else
+ } else {
lo->lo_bio = lo->lo_biotail = bio;
+ }
spin_unlock_irqrestore(&lo->lo_lock, flags);
atomic_inc(&lo->lo_pending);
diff --git a/drivers/staging/lustre/lustre/llite/lproc_llite.c b/drivers/staging/lustre/lustre/llite/lproc_llite.c
index 27ab126..55d62eb 100644
--- a/drivers/staging/lustre/lustre/llite/lproc_llite.c
+++ b/drivers/staging/lustre/lustre/llite/lproc_llite.c
@@ -254,7 +254,6 @@
pages_number *= 1 << (20 - PAGE_SHIFT); /* MB -> pages */
if (pages_number > totalram_pages / 2) {
-
CERROR("can't set file readahead more than %lu MB\n",
totalram_pages >> (20 - PAGE_SHIFT + 1)); /*1/2 of RAM*/
return -ERANGE;
@@ -393,6 +392,8 @@
struct super_block *sb = ((struct seq_file *)file->private_data)->private;
struct ll_sb_info *sbi = ll_s2sbi(sb);
struct cl_client_cache *cache = &sbi->ll_cache;
+ struct lu_env *env;
+ int refcheck;
int mult, rc, pages_number;
int diff = 0;
int nrpages = 0;
@@ -430,6 +431,10 @@
goto out;
}
+ env = cl_env_get(&refcheck);
+ if (IS_ERR(env))
+ return 0;
+
diff = -diff;
while (diff > 0) {
int tmp;
@@ -455,19 +460,20 @@
break;
if (!sbi->ll_dt_exp) { /* being initialized */
- rc = -ENODEV;
- break;
+ rc = 0;
+ goto out;
}
/* difficult - have to ask OSCs to drop LRU slots. */
tmp = diff << 1;
- rc = obd_set_info_async(NULL, sbi->ll_dt_exp,
+ rc = obd_set_info_async(env, sbi->ll_dt_exp,
sizeof(KEY_CACHE_LRU_SHRINK),
KEY_CACHE_LRU_SHRINK,
sizeof(tmp), &tmp, NULL);
if (rc < 0)
break;
}
+ cl_env_put(env, &refcheck);
out:
if (rc >= 0) {
@@ -818,6 +824,23 @@
}
LUSTRE_RW_ATTR(xattr_cache);
+static ssize_t unstable_stats_show(struct kobject *kobj,
+ struct attribute *attr,
+ char *buf)
+{
+ struct ll_sb_info *sbi = container_of(kobj, struct ll_sb_info,
+ ll_kobj);
+ struct cl_client_cache *cache = &sbi->ll_cache;
+ int pages, mb;
+
+ pages = atomic_read(&cache->ccc_unstable_nr);
+ mb = (pages * PAGE_SIZE) >> 20;
+
+ return sprintf(buf, "unstable_pages: %8d\n"
+ "unstable_mb: %8d\n", pages, mb);
+}
+LUSTRE_RO_ATTR(unstable_stats);
+
static struct lprocfs_vars lprocfs_llite_obd_vars[] = {
/* { "mntpt_path", ll_rd_path, 0, 0 }, */
{ "site", &ll_site_stats_fops, NULL, 0 },
@@ -853,6 +876,7 @@
&lustre_attr_max_easize.attr,
&lustre_attr_default_easize.attr,
&lustre_attr_xattr_cache.attr,
+ &lustre_attr_unstable_stats.attr,
NULL,
};
@@ -953,6 +977,7 @@
[RA_STAT_EOF] = "read-ahead to EOF",
[RA_STAT_MAX_IN_FLIGHT] = "hit max r-a issue",
[RA_STAT_WRONG_GRAB_PAGE] = "wrong page from grab_cache_page",
+ [RA_STAT_FAILED_REACH_END] = "failed to reach end"
};
int ldebugfs_register_mountpoint(struct dentry *parent,
diff --git a/drivers/staging/lustre/lustre/llite/namei.c b/drivers/staging/lustre/lustre/llite/namei.c
index f8f98e4..5eba0eb 100644
--- a/drivers/staging/lustre/lustre/llite/namei.c
+++ b/drivers/staging/lustre/lustre/llite/namei.c
@@ -128,12 +128,14 @@
if (rc != 0) {
iget_failed(inode);
inode = NULL;
- } else
+ } else {
unlock_new_inode(inode);
- } else if (!(inode->i_state & (I_FREEING | I_CLEAR)))
+ }
+ } else if (!(inode->i_state & (I_FREEING | I_CLEAR))) {
ll_update_inode(inode, md);
- CDEBUG(D_VFSTRACE, "got inode: %p for "DFID"\n",
- inode, PFID(&md->body->fid1));
+ CDEBUG(D_VFSTRACE, "got inode: "DFID"(%p)\n",
+ PFID(&md->body->fid1), inode);
+ }
}
return inode;
}
@@ -188,7 +190,7 @@
break;
/* Invalidate all dentries associated with this inode */
- LASSERT(lock->l_flags & LDLM_FL_CANCELING);
+ LASSERT(ldlm_is_canceling(lock));
if (!fid_res_name_eq(ll_inode2fid(inode),
&lock->l_resource->lr_name)) {
@@ -255,8 +257,8 @@
}
if ((bits & MDS_INODELOCK_UPDATE) && S_ISDIR(inode->i_mode)) {
- CDEBUG(D_INODE, "invalidating inode %lu\n",
- inode->i_ino);
+ CDEBUG(D_INODE, "invalidating inode "DFID"\n",
+ PFID(ll_inode2fid(inode)));
truncate_inode_pages(inode->i_mapping, 0);
ll_invalidate_negative_children(inode);
}
@@ -476,9 +478,8 @@
if (dentry->d_name.len > ll_i2sbi(parent)->ll_namelen)
return ERR_PTR(-ENAMETOOLONG);
- CDEBUG(D_VFSTRACE, "VFS Op:name=%pd,dir=%lu/%u(%p),intent=%s\n",
- dentry, parent->i_ino,
- parent->i_generation, parent, LL_IT2STR(it));
+ CDEBUG(D_VFSTRACE, "VFS Op:name=%pd, dir="DFID"(%p),intent=%s\n",
+ dentry, PFID(ll_inode2fid(parent)), parent, LL_IT2STR(it));
if (d_mountpoint(dentry))
CERROR("Tell Peter, lookup on mtpt, it %s\n", LL_IT2STR(it));
@@ -553,9 +554,8 @@
struct lookup_intent *itp, it = { .it_op = IT_GETATTR };
struct dentry *de;
- CDEBUG(D_VFSTRACE, "VFS Op:name=%pd,dir=%lu/%u(%p),flags=%u\n",
- dentry, parent->i_ino,
- parent->i_generation, parent, flags);
+ CDEBUG(D_VFSTRACE, "VFS Op:name=%pd, dir="DFID"(%p),flags=%u\n",
+ dentry, PFID(ll_inode2fid(parent)), parent, flags);
/* Optimize away (CREATE && !OPEN). Let .create handle the race. */
if ((flags & LOOKUP_CREATE) && !(flags & LOOKUP_OPEN))
@@ -586,10 +586,9 @@
long long lookup_flags = LOOKUP_OPEN;
int rc = 0;
- CDEBUG(D_VFSTRACE,
- "VFS Op:name=%pd,dir=%lu/%u(%p),file %p,open_flags %x,mode %x opened %d\n",
- dentry, dir->i_ino,
- dir->i_generation, dir, file, open_flags, mode, *opened);
+ CDEBUG(D_VFSTRACE, "VFS Op:name=%pd, dir="DFID"(%p),file %p,open_flags %x,mode %x opened %d\n",
+ dentry, PFID(ll_inode2fid(dir)), dir, file, open_flags, mode,
+ *opened);
it = kzalloc(sizeof(*it), GFP_NOFS);
if (!it)
@@ -680,8 +679,8 @@
* lock on the inode. Since we finally have an inode pointer,
* stuff it in the lock.
*/
- CDEBUG(D_DLMTRACE, "setting l_ast_data to inode %p (%lu/%u)\n",
- inode, inode->i_ino, inode->i_generation);
+ CDEBUG(D_DLMTRACE, "setting l_ast_data to inode "DFID"(%p)\n",
+ PFID(ll_inode2fid(dir)), inode);
ll_set_lock_data(sbi->ll_md_exp, inode, it, NULL);
out:
ptlrpc_req_finished(request);
@@ -708,9 +707,8 @@
struct inode *inode;
int rc = 0;
- CDEBUG(D_VFSTRACE, "VFS Op:name=%pd,dir=%lu/%u(%p),intent=%s\n",
- dentry, dir->i_ino,
- dir->i_generation, dir, LL_IT2STR(it));
+ CDEBUG(D_VFSTRACE, "VFS Op:name=%pd, dir="DFID"(%p), intent=%s\n",
+ dentry, PFID(ll_inode2fid(dir)), dir, LL_IT2STR(it));
rc = it_open_error(DISP_OPEN_CREATE, it);
if (rc)
@@ -733,8 +731,9 @@
LASSERT(body);
if (body->valid & OBD_MD_FLMTIME &&
body->mtime > LTIME_S(inode->i_mtime)) {
- CDEBUG(D_INODE, "setting ino %lu mtime from %lu to %llu\n",
- inode->i_ino, LTIME_S(inode->i_mtime), body->mtime);
+ CDEBUG(D_INODE, "setting fid "DFID" mtime from %lu to %llu\n",
+ PFID(ll_inode2fid(inode)), LTIME_S(inode->i_mtime),
+ body->mtime);
LTIME_S(inode->i_mtime) = body->mtime;
}
if (body->valid & OBD_MD_FLCTIME &&
@@ -791,9 +790,9 @@
{
int err;
- CDEBUG(D_VFSTRACE, "VFS Op:name=%pd,dir=%lu/%u(%p) mode %o dev %x\n",
- dchild, dir->i_ino, dir->i_generation, dir,
- mode, old_encode_dev(rdev));
+ CDEBUG(D_VFSTRACE, "VFS Op:name=%pd, dir="DFID"(%p) mode %o dev %x\n",
+ dchild, PFID(ll_inode2fid(dir)), dir, mode,
+ old_encode_dev(rdev));
if (!IS_POSIXACL(dir) || !exp_connect_umask(ll_i2mdexp(dir)))
mode &= ~current_umask();
@@ -831,9 +830,8 @@
{
int rc;
- CDEBUG(D_VFSTRACE, "VFS Op:name=%pd,dir=%lu/%u(%p),flags=%u, excl=%d\n",
- dentry, dir->i_ino,
- dir->i_generation, dir, mode, want_excl);
+ CDEBUG(D_VFSTRACE, "VFS Op:name=%pd, dir="DFID"(%p), flags=%u, excl=%d\n",
+ dentry, PFID(ll_inode2fid(dir)), dir, mode, want_excl);
rc = ll_mknod(dir, dentry, mode, 0);
@@ -845,12 +843,6 @@
return rc;
}
-static inline void ll_get_child_fid(struct dentry *child, struct lu_fid *fid)
-{
- if (d_really_is_positive(child))
- *fid = *ll_inode2fid(d_inode(child));
-}
-
int ll_objects_destroy(struct ptlrpc_request *request, struct inode *dir)
{
struct mdt_body *body;
@@ -927,23 +919,25 @@
* is any lock existing. They will recycle dentries and inodes based upon locks
* too. b=20433
*/
-static int ll_unlink(struct inode *dir, struct dentry *dentry)
+static int ll_unlink(struct inode *dir, struct dentry *dchild)
{
struct ptlrpc_request *request = NULL;
struct md_op_data *op_data;
int rc;
CDEBUG(D_VFSTRACE, "VFS Op:name=%pd,dir=%lu/%u(%p)\n",
- dentry, dir->i_ino, dir->i_generation, dir);
+ dchild, dir->i_ino, dir->i_generation, dir);
op_data = ll_prep_md_op_data(NULL, dir, NULL,
- dentry->d_name.name,
- dentry->d_name.len,
+ dchild->d_name.name,
+ dchild->d_name.len,
0, LUSTRE_OPC_ANY, NULL);
if (IS_ERR(op_data))
return PTR_ERR(op_data);
- ll_get_child_fid(dentry, &op_data->op_fid3);
+ if (dchild && dchild->d_inode)
+ op_data->op_fid3 = *ll_inode2fid(dchild->d_inode);
+
op_data->op_fid2 = op_data->op_fid3;
rc = md_unlink(ll_i2sbi(dir)->ll_md_exp, op_data, &request);
ll_finish_md_op_data(op_data);
@@ -963,8 +957,8 @@
{
int err;
- CDEBUG(D_VFSTRACE, "VFS Op:name=%pd,dir=%lu/%u(%p)\n",
- dentry, dir->i_ino, dir->i_generation, dir);
+ CDEBUG(D_VFSTRACE, "VFS Op:name=%pd, dir"DFID"(%p)\n",
+ dentry, PFID(ll_inode2fid(dir)), dir);
if (!IS_POSIXACL(dir) || !exp_connect_umask(ll_i2mdexp(dir)))
mode &= ~current_umask();
@@ -977,23 +971,25 @@
return err;
}
-static int ll_rmdir(struct inode *dir, struct dentry *dentry)
+static int ll_rmdir(struct inode *dir, struct dentry *dchild)
{
struct ptlrpc_request *request = NULL;
struct md_op_data *op_data;
int rc;
- CDEBUG(D_VFSTRACE, "VFS Op:name=%pd,dir=%lu/%u(%p)\n",
- dentry, dir->i_ino, dir->i_generation, dir);
+ CDEBUG(D_VFSTRACE, "VFS Op:name=%pd, dir="DFID"(%p)\n",
+ dchild, PFID(ll_inode2fid(dir)), dir);
op_data = ll_prep_md_op_data(NULL, dir, NULL,
- dentry->d_name.name,
- dentry->d_name.len,
+ dchild->d_name.name,
+ dchild->d_name.len,
S_IFDIR, LUSTRE_OPC_ANY, NULL);
if (IS_ERR(op_data))
return PTR_ERR(op_data);
- ll_get_child_fid(dentry, &op_data->op_fid3);
+ if (dchild && dchild->d_inode)
+ op_data->op_fid3 = *ll_inode2fid(dchild->d_inode);
+
op_data->op_fid2 = op_data->op_fid3;
rc = md_unlink(ll_i2sbi(dir)->ll_md_exp, op_data, &request);
ll_finish_md_op_data(op_data);
@@ -1011,9 +1007,8 @@
{
int err;
- CDEBUG(D_VFSTRACE, "VFS Op:name=%pd,dir=%lu/%u(%p),target=%.*s\n",
- dentry, dir->i_ino, dir->i_generation,
- dir, 3000, oldname);
+ CDEBUG(D_VFSTRACE, "VFS Op:name=%pd, dir="DFID"(%p),target=%.*s\n",
+ dentry, PFID(ll_inode2fid(dir)), dir, 3000, oldname);
err = ll_new_node(dir, dentry, oldname, S_IFLNK | S_IRWXUGO,
0, LUSTRE_OPC_SYMLINK);
@@ -1033,10 +1028,9 @@
struct md_op_data *op_data;
int err;
- CDEBUG(D_VFSTRACE,
- "VFS Op: inode=%lu/%u(%p), dir=%lu/%u(%p), target=%pd\n",
- src->i_ino, src->i_generation, src, dir->i_ino,
- dir->i_generation, dir, new_dentry);
+ CDEBUG(D_VFSTRACE, "VFS Op: inode="DFID"(%p), dir="DFID"(%p), target=%pd\n",
+ PFID(ll_inode2fid(src)), src, PFID(ll_inode2fid(dir)), dir,
+ new_dentry);
op_data = ll_prep_md_op_data(NULL, src, dir, new_dentry->d_name.name,
new_dentry->d_name.len,
@@ -1056,42 +1050,45 @@
return err;
}
-static int ll_rename(struct inode *old_dir, struct dentry *old_dentry,
- struct inode *new_dir, struct dentry *new_dentry)
+static int ll_rename(struct inode *src, struct dentry *src_dchild,
+ struct inode *tgt, struct dentry *tgt_dchild)
{
struct ptlrpc_request *request = NULL;
- struct ll_sb_info *sbi = ll_i2sbi(old_dir);
+ struct ll_sb_info *sbi = ll_i2sbi(src);
struct md_op_data *op_data;
int err;
CDEBUG(D_VFSTRACE,
- "VFS Op:oldname=%pd,src_dir=%lu/%u(%p),newname=%pd,tgt_dir=%lu/%u(%p)\n",
- old_dentry, old_dir->i_ino, old_dir->i_generation, old_dir,
- new_dentry, new_dir->i_ino, new_dir->i_generation, new_dir);
+ "VFS Op:oldname=%pd, src_dir="DFID"(%p), newname=%pd, tgt_dir="DFID"(%p)\n",
+ src_dchild, PFID(ll_inode2fid(src)), src,
+ tgt_dchild, PFID(ll_inode2fid(tgt)), tgt);
- op_data = ll_prep_md_op_data(NULL, old_dir, new_dir, NULL, 0, 0,
+ op_data = ll_prep_md_op_data(NULL, src, tgt, NULL, 0, 0,
LUSTRE_OPC_ANY, NULL);
if (IS_ERR(op_data))
return PTR_ERR(op_data);
- ll_get_child_fid(old_dentry, &op_data->op_fid3);
- ll_get_child_fid(new_dentry, &op_data->op_fid4);
+ if (src_dchild && src_dchild->d_inode)
+ op_data->op_fid3 = *ll_inode2fid(src_dchild->d_inode);
+ if (tgt_dchild && tgt_dchild->d_inode)
+ op_data->op_fid4 = *ll_inode2fid(tgt_dchild->d_inode);
+
err = md_rename(sbi->ll_md_exp, op_data,
- old_dentry->d_name.name,
- old_dentry->d_name.len,
- new_dentry->d_name.name,
- new_dentry->d_name.len, &request);
+ src_dchild->d_name.name,
+ src_dchild->d_name.len,
+ tgt_dchild->d_name.name,
+ tgt_dchild->d_name.len, &request);
ll_finish_md_op_data(op_data);
if (!err) {
- ll_update_times(request, old_dir);
- ll_update_times(request, new_dir);
+ ll_update_times(request, src);
+ ll_update_times(request, tgt);
ll_stats_ops_tally(sbi, LPROC_LL_RENAME, 1);
- err = ll_objects_destroy(request, old_dir);
+ err = ll_objects_destroy(request, src);
}
ptlrpc_req_finished(request);
if (!err)
- d_move(old_dentry, new_dentry);
+ d_move(src_dchild, tgt_dchild);
return err;
}
diff --git a/drivers/staging/lustre/lustre/llite/rw.c b/drivers/staging/lustre/lustre/llite/rw.c
index edab6c5..3363977 100644
--- a/drivers/staging/lustre/lustre/llite/rw.c
+++ b/drivers/staging/lustre/lustre/llite/rw.c
@@ -63,7 +63,7 @@
* Finalizes cl-data before exiting typical address_space operation. Dual to
* ll_cl_init().
*/
-static void ll_cl_fini(struct ll_cl_context *lcc)
+void ll_cl_fini(struct ll_cl_context *lcc)
{
struct lu_env *env = lcc->lcc_env;
struct cl_io *io = lcc->lcc_io;
@@ -84,200 +84,59 @@
* Initializes common cl-data at the typical address_space operation entry
* point.
*/
-static struct ll_cl_context *ll_cl_init(struct file *file,
- struct page *vmpage, int create)
+struct ll_cl_context *ll_cl_init(struct file *file, struct page *vmpage)
{
struct ll_cl_context *lcc;
struct lu_env *env;
struct cl_io *io;
struct cl_object *clob;
- struct ccc_io *cio;
+ struct vvp_io *vio;
int refcheck;
int result = 0;
- clob = ll_i2info(vmpage->mapping->host)->lli_clob;
+ clob = ll_i2info(file_inode(file))->lli_clob;
LASSERT(clob);
env = cl_env_get(&refcheck);
if (IS_ERR(env))
return ERR_CAST(env);
- lcc = &vvp_env_info(env)->vti_io_ctx;
+ lcc = &ll_env_info(env)->lti_io_ctx;
memset(lcc, 0, sizeof(*lcc));
lcc->lcc_env = env;
lcc->lcc_refcheck = refcheck;
lcc->lcc_cookie = current;
- cio = ccc_env_io(env);
- io = cio->cui_cl.cis_io;
- if (!io && create) {
- struct inode *inode = vmpage->mapping->host;
- loff_t pos;
-
- if (inode_trylock(inode)) {
- inode_unlock((inode));
-
- /* this is too bad. Someone is trying to write the
- * page w/o holding inode mutex. This means we can
- * add dirty pages into cache during truncate
- */
- CERROR("Proc %s is dirtying page w/o inode lock, this will break truncate\n",
- current->comm);
- dump_stack();
- LBUG();
- return ERR_PTR(-EIO);
- }
-
- /*
- * Loop-back driver calls ->prepare_write().
- * methods directly, bypassing file system ->write() operation,
- * so cl_io has to be created here.
- */
- io = ccc_env_thread_io(env);
- ll_io_init(io, file, 1);
-
- /* No lock at all for this kind of IO - we can't do it because
- * we have held page lock, it would cause deadlock.
- * XXX: This causes poor performance to loop device - One page
- * per RPC.
- * In order to get better performance, users should use
- * lloop driver instead.
- */
- io->ci_lockreq = CILR_NEVER;
-
- pos = vmpage->index << PAGE_SHIFT;
-
- /* Create a temp IO to serve write. */
- result = cl_io_rw_init(env, io, CIT_WRITE, pos, PAGE_SIZE);
- if (result == 0) {
- cio->cui_fd = LUSTRE_FPRIVATE(file);
- cio->cui_iter = NULL;
- result = cl_io_iter_init(env, io);
- if (result == 0) {
- result = cl_io_lock(env, io);
- if (result == 0)
- result = cl_io_start(env, io);
- }
- } else
- result = io->ci_result;
- }
-
+ vio = vvp_env_io(env);
+ io = vio->vui_cl.cis_io;
lcc->lcc_io = io;
if (!io)
result = -EIO;
- if (result == 0) {
+
+ if (result == 0 && vmpage) {
struct cl_page *page;
LASSERT(io->ci_state == CIS_IO_GOING);
- LASSERT(cio->cui_fd == LUSTRE_FPRIVATE(file));
+ LASSERT(vio->vui_fd == LUSTRE_FPRIVATE(file));
page = cl_page_find(env, clob, vmpage->index, vmpage,
CPT_CACHEABLE);
if (!IS_ERR(page)) {
lcc->lcc_page = page;
lu_ref_add(&page->cp_reference, "cl_io", io);
result = 0;
- } else
+ } else {
result = PTR_ERR(page);
+ }
}
if (result) {
ll_cl_fini(lcc);
lcc = ERR_PTR(result);
}
- CDEBUG(D_VFSTRACE, "%lu@"DFID" -> %d %p %p\n",
- vmpage->index, PFID(lu_object_fid(&clob->co_lu)), result,
- env, io);
return lcc;
}
-static struct ll_cl_context *ll_cl_get(void)
-{
- struct ll_cl_context *lcc;
- struct lu_env *env;
- int refcheck;
-
- env = cl_env_get(&refcheck);
- LASSERT(!IS_ERR(env));
- lcc = &vvp_env_info(env)->vti_io_ctx;
- LASSERT(env == lcc->lcc_env);
- LASSERT(current == lcc->lcc_cookie);
- cl_env_put(env, &refcheck);
-
- /* env has got in ll_cl_init, so it is still usable. */
- return lcc;
-}
-
-/**
- * ->prepare_write() address space operation called by generic_file_write()
- * for every page during write.
- */
-int ll_prepare_write(struct file *file, struct page *vmpage, unsigned from,
- unsigned to)
-{
- struct ll_cl_context *lcc;
- int result;
-
- lcc = ll_cl_init(file, vmpage, 1);
- if (!IS_ERR(lcc)) {
- struct lu_env *env = lcc->lcc_env;
- struct cl_io *io = lcc->lcc_io;
- struct cl_page *page = lcc->lcc_page;
-
- cl_page_assume(env, io, page);
-
- result = cl_io_prepare_write(env, io, page, from, to);
- if (result == 0) {
- /*
- * Add a reference, so that page is not evicted from
- * the cache until ->commit_write() is called.
- */
- cl_page_get(page);
- lu_ref_add(&page->cp_reference, "prepare_write",
- current);
- } else {
- cl_page_unassume(env, io, page);
- ll_cl_fini(lcc);
- }
- /* returning 0 in prepare assumes commit must be called
- * afterwards
- */
- } else {
- result = PTR_ERR(lcc);
- }
- return result;
-}
-
-int ll_commit_write(struct file *file, struct page *vmpage, unsigned from,
- unsigned to)
-{
- struct ll_cl_context *lcc;
- struct lu_env *env;
- struct cl_io *io;
- struct cl_page *page;
- int result = 0;
-
- lcc = ll_cl_get();
- env = lcc->lcc_env;
- page = lcc->lcc_page;
- io = lcc->lcc_io;
-
- LASSERT(cl_page_is_owned(page, io));
- LASSERT(from <= to);
- if (from != to) /* handle short write case. */
- result = cl_io_commit_write(env, io, page, from, to);
- if (cl_page_is_owned(page, io))
- cl_page_unassume(env, io, page);
-
- /*
- * Release reference acquired by ll_prepare_write().
- */
- lu_ref_del(&page->cp_reference, "prepare_write", current);
- cl_page_put(env, page);
- ll_cl_fini(lcc);
- return result;
-}
-
static void ll_ra_stats_inc_sbi(struct ll_sb_info *sbi, enum ra_stat which);
/**
@@ -301,7 +160,7 @@
*/
static unsigned long ll_ra_count_get(struct ll_sb_info *sbi,
struct ra_io_arg *ria,
- unsigned long pages)
+ unsigned long pages, unsigned long min)
{
struct ll_ra_info *ra = &sbi->ll_ra_info;
long ret;
@@ -341,6 +200,11 @@
}
out:
+ if (ret < min) {
+ /* override ra limit for maximum performance */
+ atomic_add(min - ret, &ra->ra_cur_pages);
+ ret = min;
+ }
return ret;
}
@@ -357,9 +221,9 @@
lprocfs_counter_incr(sbi->ll_ra_stats, which);
}
-void ll_ra_stats_inc(struct address_space *mapping, enum ra_stat which)
+void ll_ra_stats_inc(struct inode *inode, enum ra_stat which)
{
- struct ll_sb_info *sbi = ll_i2sbi(mapping->host);
+ struct ll_sb_info *sbi = ll_i2sbi(inode);
ll_ra_stats_inc_sbi(sbi, which);
}
@@ -388,61 +252,42 @@
return start <= index && index <= end;
}
-static struct ll_readahead_state *ll_ras_get(struct file *f)
+void ll_ras_enter(struct file *f)
{
- struct ll_file_data *fd;
-
- fd = LUSTRE_FPRIVATE(f);
- return &fd->fd_ras;
-}
-
-void ll_ra_read_in(struct file *f, struct ll_ra_read *rar)
-{
- struct ll_readahead_state *ras;
-
- ras = ll_ras_get(f);
+ struct ll_file_data *fd = LUSTRE_FPRIVATE(f);
+ struct ll_readahead_state *ras = &fd->fd_ras;
spin_lock(&ras->ras_lock);
ras->ras_requests++;
ras->ras_request_index = 0;
ras->ras_consecutive_requests++;
- rar->lrr_reader = current;
-
- list_add(&rar->lrr_linkage, &ras->ras_read_beads);
- spin_unlock(&ras->ras_lock);
-}
-
-void ll_ra_read_ex(struct file *f, struct ll_ra_read *rar)
-{
- struct ll_readahead_state *ras;
-
- ras = ll_ras_get(f);
-
- spin_lock(&ras->ras_lock);
- list_del_init(&rar->lrr_linkage);
spin_unlock(&ras->ras_lock);
}
static int cl_read_ahead_page(const struct lu_env *env, struct cl_io *io,
struct cl_page_list *queue, struct cl_page *page,
- struct page *vmpage)
+ struct cl_object *clob, pgoff_t *max_index)
{
- struct ccc_page *cp;
+ struct page *vmpage = page->cp_vmpage;
+ struct vvp_page *vpg;
int rc;
rc = 0;
cl_page_assume(env, io, page);
lu_ref_add(&page->cp_reference, "ra", current);
- cp = cl2ccc_page(cl_page_at(page, &vvp_device_type));
- if (!cp->cpg_defer_uptodate && !PageUptodate(vmpage)) {
- rc = cl_page_is_under_lock(env, io, page);
- if (rc == -EBUSY) {
- cp->cpg_defer_uptodate = 1;
- cp->cpg_ra_used = 0;
+ vpg = cl2vvp_page(cl_object_page_slice(clob, page));
+ if (!vpg->vpg_defer_uptodate && !PageUptodate(vmpage)) {
+ CDEBUG(D_READA, "page index %lu, max_index: %lu\n",
+ vvp_index(vpg), *max_index);
+ if (*max_index == 0 || vvp_index(vpg) > *max_index)
+ rc = cl_page_is_under_lock(env, io, page, max_index);
+ if (rc == 0) {
+ vpg->vpg_defer_uptodate = 1;
+ vpg->vpg_ra_used = 0;
cl_page_list_add(queue, page);
rc = 1;
} else {
- cl_page_delete(env, page);
+ cl_page_discard(env, io, page);
rc = -ENOLCK;
}
} else {
@@ -466,24 +311,25 @@
*/
static int ll_read_ahead_page(const struct lu_env *env, struct cl_io *io,
struct cl_page_list *queue,
- pgoff_t index, struct address_space *mapping)
+ pgoff_t index, pgoff_t *max_index)
{
+ struct cl_object *clob = io->ci_obj;
+ struct inode *inode = vvp_object_inode(clob);
struct page *vmpage;
- struct cl_object *clob = ll_i2info(mapping->host)->lli_clob;
struct cl_page *page;
enum ra_stat which = _NR_RA_STAT; /* keep gcc happy */
int rc = 0;
const char *msg = NULL;
- vmpage = grab_cache_page_nowait(mapping, index);
+ vmpage = grab_cache_page_nowait(inode->i_mapping, index);
if (vmpage) {
/* Check if vmpage was truncated or reclaimed */
- if (vmpage->mapping == mapping) {
+ if (vmpage->mapping == inode->i_mapping) {
page = cl_page_find(env, clob, vmpage->index,
vmpage, CPT_CACHEABLE);
if (!IS_ERR(page)) {
rc = cl_read_ahead_page(env, io, queue,
- page, vmpage);
+ page, clob, max_index);
if (rc == -ENOLCK) {
which = RA_STAT_FAILED_MATCH;
msg = "lock match failed";
@@ -504,7 +350,7 @@
msg = "g_c_p_n failed";
}
if (msg) {
- ll_ra_stats_inc(mapping, which);
+ ll_ra_stats_inc(inode, which);
CDEBUG(D_READA, "%s\n", msg);
}
return rc;
@@ -616,11 +462,12 @@
struct cl_io *io, struct cl_page_list *queue,
struct ra_io_arg *ria,
unsigned long *reserved_pages,
- struct address_space *mapping,
unsigned long *ra_end)
{
- int rc, count = 0, stride_ria;
- unsigned long page_idx;
+ int rc, count = 0;
+ bool stride_ria;
+ pgoff_t page_idx;
+ pgoff_t max_index = 0;
LASSERT(ria);
RIA_DEBUG(ria);
@@ -631,12 +478,13 @@
if (ras_inside_ra_window(page_idx, ria)) {
/* If the page is inside the read-ahead window*/
rc = ll_read_ahead_page(env, io, queue,
- page_idx, mapping);
+ page_idx, &max_index);
if (rc == 1) {
(*reserved_pages)--;
count++;
- } else if (rc == -ENOLCK)
+ } else if (rc == -ENOLCK) {
break;
+ }
} else if (stride_ria) {
/* If it is not in the read-ahead window, and it is
* read-ahead mode, then check whether it should skip
@@ -666,25 +514,22 @@
}
int ll_readahead(const struct lu_env *env, struct cl_io *io,
- struct ll_readahead_state *ras, struct address_space *mapping,
- struct cl_page_list *queue, int flags)
+ struct cl_page_list *queue, struct ll_readahead_state *ras,
+ bool hit)
{
struct vvp_io *vio = vvp_env_io(env);
- struct vvp_thread_info *vti = vvp_env_info(env);
- struct cl_attr *attr = ccc_env_thread_attr(env);
+ struct ll_thread_info *lti = ll_env_info(env);
+ struct cl_attr *attr = vvp_env_thread_attr(env);
unsigned long start = 0, end = 0, reserved;
- unsigned long ra_end, len;
+ unsigned long ra_end, len, mlen = 0;
struct inode *inode;
- struct ll_ra_read *bead;
- struct ra_io_arg *ria = &vti->vti_ria;
- struct ll_inode_info *lli;
+ struct ra_io_arg *ria = <i->lti_ria;
struct cl_object *clob;
int ret = 0;
__u64 kms;
- inode = mapping->host;
- lli = ll_i2info(inode);
- clob = lli->lli_clob;
+ clob = io->ci_obj;
+ inode = vvp_object_inode(clob);
memset(ria, 0, sizeof(*ria));
@@ -696,22 +541,20 @@
return ret;
kms = attr->cat_kms;
if (kms == 0) {
- ll_ra_stats_inc(mapping, RA_STAT_ZERO_LEN);
+ ll_ra_stats_inc(inode, RA_STAT_ZERO_LEN);
return 0;
}
spin_lock(&ras->ras_lock);
- if (vio->cui_ra_window_set)
- bead = &vio->cui_bead;
- else
- bead = NULL;
/* Enlarge the RA window to encompass the full read */
- if (bead && ras->ras_window_start + ras->ras_window_len <
- bead->lrr_start + bead->lrr_count) {
- ras->ras_window_len = bead->lrr_start + bead->lrr_count -
+ if (vio->vui_ra_valid &&
+ ras->ras_window_start + ras->ras_window_len <
+ vio->vui_ra_start + vio->vui_ra_count) {
+ ras->ras_window_len = vio->vui_ra_start + vio->vui_ra_count -
ras->ras_window_start;
}
+
/* Reserve a part of the read-ahead window that we'll be issuing */
if (ras->ras_window_len) {
start = ras->ras_next_readahead;
@@ -755,29 +598,48 @@
spin_unlock(&ras->ras_lock);
if (end == 0) {
- ll_ra_stats_inc(mapping, RA_STAT_ZERO_WINDOW);
+ ll_ra_stats_inc(inode, RA_STAT_ZERO_WINDOW);
return 0;
}
len = ria_page_count(ria);
- if (len == 0)
+ if (len == 0) {
+ ll_ra_stats_inc(inode, RA_STAT_ZERO_WINDOW);
return 0;
+ }
- reserved = ll_ra_count_get(ll_i2sbi(inode), ria, len);
+ CDEBUG(D_READA, DFID ": ria: %lu/%lu, bead: %lu/%lu, hit: %d\n",
+ PFID(lu_object_fid(&clob->co_lu)),
+ ria->ria_start, ria->ria_end,
+ vio->vui_ra_valid ? vio->vui_ra_start : 0,
+ vio->vui_ra_valid ? vio->vui_ra_count : 0,
+ hit);
+
+ /* at least to extend the readahead window to cover current read */
+ if (!hit && vio->vui_ra_valid &&
+ vio->vui_ra_start + vio->vui_ra_count > ria->ria_start) {
+ /* to the end of current read window. */
+ mlen = vio->vui_ra_start + vio->vui_ra_count - ria->ria_start;
+ /* trim to RPC boundary */
+ start = ria->ria_start & (PTLRPC_MAX_BRW_PAGES - 1);
+ mlen = min(mlen, PTLRPC_MAX_BRW_PAGES - start);
+ }
+
+ reserved = ll_ra_count_get(ll_i2sbi(inode), ria, len, mlen);
if (reserved < len)
- ll_ra_stats_inc(mapping, RA_STAT_MAX_IN_FLIGHT);
+ ll_ra_stats_inc(inode, RA_STAT_MAX_IN_FLIGHT);
- CDEBUG(D_READA, "reserved page %lu ra_cur %d ra_max %lu\n", reserved,
+ CDEBUG(D_READA, "reserved pages %lu/%lu/%lu, ra_cur %d, ra_max %lu\n",
+ reserved, len, mlen,
atomic_read(&ll_i2sbi(inode)->ll_ra_info.ra_cur_pages),
ll_i2sbi(inode)->ll_ra_info.ra_max_pages);
- ret = ll_read_ahead_pages(env, io, queue,
- ria, &reserved, mapping, &ra_end);
+ ret = ll_read_ahead_pages(env, io, queue, ria, &reserved, &ra_end);
if (reserved != 0)
ll_ra_count_put(ll_i2sbi(inode), reserved);
if (ra_end == end + 1 && ra_end == (kms >> PAGE_SHIFT))
- ll_ra_stats_inc(mapping, RA_STAT_EOF);
+ ll_ra_stats_inc(inode, RA_STAT_EOF);
/* if we didn't get to the end of the region we reserved from
* the ras we need to go back and update the ras so that the
@@ -789,6 +651,7 @@
ra_end, end, ria->ria_end);
if (ra_end != end + 1) {
+ ll_ra_stats_inc(inode, RA_STAT_FAILED_REACH_END);
spin_lock(&ras->ras_lock);
if (ra_end < ras->ras_next_readahead &&
index_in_window(ra_end, ras->ras_window_start, 0,
@@ -836,7 +699,6 @@
spin_lock_init(&ras->ras_lock);
ras_reset(inode, ras, 0);
ras->ras_requests = 0;
- INIT_LIST_HEAD(&ras->ras_read_beads);
}
/*
@@ -1059,15 +921,18 @@
ras->ras_last_readpage = index;
ras_set_start(inode, ras, index);
- if (stride_io_mode(ras))
+ if (stride_io_mode(ras)) {
/* Since stride readahead is sensitive to the offset
* of read-ahead, so we use original offset here,
* instead of ras_window_start, which is RPC aligned
*/
ras->ras_next_readahead = max(index, ras->ras_next_readahead);
- else
- ras->ras_next_readahead = max(ras->ras_window_start,
- ras->ras_next_readahead);
+ } else {
+ if (ras->ras_next_readahead < ras->ras_window_start)
+ ras->ras_next_readahead = ras->ras_window_start;
+ if (!hit)
+ ras->ras_next_readahead = index + 1;
+ }
RAS_CDEBUG(ras);
/* Trigger RA in the mmap case where ras_consecutive_requests
@@ -1129,7 +994,7 @@
clob = ll_i2info(inode)->lli_clob;
LASSERT(clob);
- io = ccc_env_thread_io(env);
+ io = vvp_env_thread_io(env);
io->ci_obj = clob;
io->ci_ignore_layout = 1;
result = cl_io_init(env, io, CIT_MISC, clob);
@@ -1240,8 +1105,9 @@
if (wbc->range_cyclic || (range_whole && wbc->nr_to_write > 0)) {
if (end == OBD_OBJECT_EOF)
- end = i_size_read(inode);
- mapping->writeback_index = (end >> PAGE_SHIFT) + 1;
+ mapping->writeback_index = 0;
+ else
+ mapping->writeback_index = (end >> PAGE_SHIFT) + 1;
}
return result;
}
@@ -1251,7 +1117,7 @@
struct ll_cl_context *lcc;
int result;
- lcc = ll_cl_init(file, vmpage, 0);
+ lcc = ll_cl_init(file, vmpage);
if (!IS_ERR(lcc)) {
struct lu_env *env = lcc->lcc_env;
struct cl_io *io = lcc->lcc_io;
@@ -1273,3 +1139,28 @@
}
return result;
}
+
+int ll_page_sync_io(const struct lu_env *env, struct cl_io *io,
+ struct cl_page *page, enum cl_req_type crt)
+{
+ struct cl_2queue *queue;
+ int result;
+
+ LASSERT(io->ci_type == CIT_READ || io->ci_type == CIT_WRITE);
+
+ queue = &io->ci_queue;
+ cl_2queue_init_page(queue, page);
+
+ result = cl_io_submit_sync(env, io, crt, queue, 0);
+ LASSERT(cl_page_is_owned(page, io));
+
+ if (crt == CRT_READ)
+ /*
+ * in CRT_WRITE case page is left locked even in case of
+ * error.
+ */
+ cl_page_list_disown(env, io, &queue->c2_qin);
+ cl_2queue_fini(env, queue);
+
+ return result;
+}
diff --git a/drivers/staging/lustre/lustre/llite/rw26.c b/drivers/staging/lustre/lustre/llite/rw26.c
index 0c3459c..c12a048 100644
--- a/drivers/staging/lustre/lustre/llite/rw26.c
+++ b/drivers/staging/lustre/lustre/llite/rw26.c
@@ -95,15 +95,12 @@
if (obj) {
page = cl_vmpage_page(vmpage, obj);
if (page) {
- lu_ref_add(&page->cp_reference,
- "delete", vmpage);
cl_page_delete(env, page);
- lu_ref_del(&page->cp_reference,
- "delete", vmpage);
cl_page_put(env, page);
}
- } else
+ } else {
LASSERT(vmpage->private == 0);
+ }
cl_env_put(env, &refcheck);
}
}
@@ -111,12 +108,12 @@
static int ll_releasepage(struct page *vmpage, gfp_t gfp_mask)
{
- struct cl_env_nest nest;
struct lu_env *env;
+ void *cookie;
struct cl_object *obj;
struct cl_page *page;
struct address_space *mapping;
- int result;
+ int result = 0;
LASSERT(PageLocked(vmpage));
if (PageWriteback(vmpage) || PageDirty(vmpage))
@@ -130,55 +127,44 @@
if (!obj)
return 1;
- /* 1 for page allocator, 1 for cl_page and 1 for page cache */
+ /* 1 for caller, 1 for cl_page and 1 for page cache */
if (page_count(vmpage) > 3)
return 0;
- /* TODO: determine what gfp should be used by @gfp_mask. */
- env = cl_env_nested_get(&nest);
- if (IS_ERR(env))
- /* If we can't allocate an env we won't call cl_page_put()
- * later on which further means it's impossible to drop
- * page refcount by cl_page, so ask kernel to not free
- * this page.
- */
- return 0;
-
page = cl_vmpage_page(vmpage, obj);
- result = !page;
- if (page) {
- if (!cl_page_in_use(page)) {
- result = 1;
- cl_page_delete(env, page);
- }
- cl_page_put(env, page);
+ if (!page)
+ return 1;
+
+ cookie = cl_env_reenter();
+ env = cl_env_percpu_get();
+ LASSERT(!IS_ERR(env));
+
+ if (!cl_page_in_use(page)) {
+ result = 1;
+ cl_page_delete(env, page);
}
- cl_env_nested_put(&nest, env);
+
+ /* To use percpu env array, the call path can not be rescheduled;
+ * otherwise percpu array will be messed if ll_releaspage() called
+ * again on the same CPU.
+ *
+ * If this page holds the last refc of cl_object, the following
+ * call path may cause reschedule:
+ * cl_page_put -> cl_page_free -> cl_object_put ->
+ * lu_object_put -> lu_object_free -> lov_delete_raid0.
+ *
+ * However, the kernel can't get rid of this inode until all pages have
+ * been cleaned up. Now that we hold page lock here, it's pretty safe
+ * that we won't get into object delete path.
+ */
+ LASSERT(cl_object_refc(obj) > 1);
+ cl_page_put(env, page);
+
+ cl_env_percpu_put(env);
+ cl_env_reexit(cookie);
return result;
}
-static int ll_set_page_dirty(struct page *vmpage)
-{
-#if 0
- struct cl_page *page = vvp_vmpage_page_transient(vmpage);
- struct vvp_object *obj = cl_inode2vvp(vmpage->mapping->host);
- struct vvp_page *cpg;
-
- /*
- * XXX should page method be called here?
- */
- LASSERT(&obj->co_cl == page->cp_obj);
- cpg = cl2vvp_page(cl_page_at(page, &vvp_device_type));
- /*
- * XXX cannot do much here, because page is possibly not locked:
- * sys_munmap()->...
- * ->unmap_page_range()->zap_pte_range()->set_page_dirty().
- */
- vvp_write_pending(obj, cpg);
-#endif
- return __set_page_dirty_nobuffers(vmpage);
-}
-
#define MAX_DIRECTIO_SIZE (2*1024*1024*1024UL)
static inline int ll_get_user_pages(int rw, unsigned long user_addr,
@@ -266,7 +252,7 @@
* write directly
*/
if (clp->cp_type == CPT_CACHEABLE) {
- struct page *vmpage = cl_page_vmpage(env, clp);
+ struct page *vmpage = cl_page_vmpage(clp);
struct page *src_page;
struct page *dst_page;
void *src;
@@ -364,7 +350,7 @@
struct cl_io *io;
struct file *file = iocb->ki_filp;
struct inode *inode = file->f_mapping->host;
- struct ccc_object *obj = cl_inode2ccc(inode);
+ struct vvp_object *obj = cl_inode2vvp(inode);
loff_t file_offset = iocb->ki_pos;
ssize_t count = iov_iter_count(iter);
ssize_t tot_bytes = 0, result = 0;
@@ -376,22 +362,21 @@
return -EBADF;
/* FIXME: io smaller than PAGE_SIZE is broken on ia64 ??? */
- if ((file_offset & ~CFS_PAGE_MASK) || (count & ~CFS_PAGE_MASK))
+ if ((file_offset & ~PAGE_MASK) || (count & ~PAGE_MASK))
return -EINVAL;
- CDEBUG(D_VFSTRACE,
- "VFS Op:inode=%lu/%u(%p), size=%zd (max %lu), offset=%lld=%llx, pages %zd (max %lu)\n",
- inode->i_ino, inode->i_generation, inode, count, MAX_DIO_SIZE,
+ CDEBUG(D_VFSTRACE, "VFS Op:inode="DFID"(%p), size=%zd (max %lu), offset=%lld=%llx, pages %zd (max %lu)\n",
+ PFID(ll_inode2fid(inode)), inode, count, MAX_DIO_SIZE,
file_offset, file_offset, count >> PAGE_SHIFT,
MAX_DIO_SIZE >> PAGE_SHIFT);
/* Check that all user buffers are aligned as well */
- if (iov_iter_alignment(iter) & ~CFS_PAGE_MASK)
+ if (iov_iter_alignment(iter) & ~PAGE_MASK)
return -EINVAL;
env = cl_env_get(&refcheck);
LASSERT(!IS_ERR(env));
- io = ccc_env_io(env)->cui_cl.cis_io;
+ io = vvp_env_io(env)->vui_cl.cis_io;
LASSERT(io);
/* 0. Need locking between buffered and direct access. and race with
@@ -401,7 +386,7 @@
if (iov_iter_rw(iter) == READ)
inode_lock(inode);
- LASSERT(obj->cob_transient_pages == 0);
+ LASSERT(obj->vob_transient_pages == 0);
while (iov_iter_count(iter)) {
struct page **pages;
size_t offs;
@@ -435,8 +420,8 @@
size > (PAGE_SIZE / sizeof(*pages)) *
PAGE_SIZE) {
size = ((((size / 2) - 1) |
- ~CFS_PAGE_MASK) + 1) &
- CFS_PAGE_MASK;
+ ~PAGE_MASK) + 1) &
+ PAGE_MASK;
CDEBUG(D_VFSTRACE, "DIO size now %lu\n",
size);
continue;
@@ -449,62 +434,213 @@
file_offset += result;
}
out:
- LASSERT(obj->cob_transient_pages == 0);
+ LASSERT(obj->vob_transient_pages == 0);
if (iov_iter_rw(iter) == READ)
inode_unlock(inode);
if (tot_bytes > 0) {
- if (iov_iter_rw(iter) == WRITE) {
- struct lov_stripe_md *lsm;
+ struct vvp_io *vio = vvp_env_io(env);
- lsm = ccc_inode_lsm_get(inode);
- LASSERT(lsm);
- lov_stripe_lock(lsm);
- obd_adjust_kms(ll_i2dtexp(inode), lsm, file_offset, 0);
- lov_stripe_unlock(lsm);
- ccc_inode_lsm_put(inode, lsm);
- }
+ /* no commit async for direct IO */
+ vio->u.write.vui_written += tot_bytes;
}
cl_env_put(env, &refcheck);
- return tot_bytes ? : result;
+ return tot_bytes ? tot_bytes : result;
+}
+
+/**
+ * Prepare partially written-to page for a write.
+ */
+static int ll_prepare_partial_page(const struct lu_env *env, struct cl_io *io,
+ struct cl_page *pg)
+{
+ struct cl_attr *attr = vvp_env_thread_attr(env);
+ struct cl_object *obj = io->ci_obj;
+ struct vvp_page *vpg = cl_object_page_slice(obj, pg);
+ loff_t offset = cl_offset(obj, vvp_index(vpg));
+ int result;
+
+ cl_object_attr_lock(obj);
+ result = cl_object_attr_get(env, obj, attr);
+ cl_object_attr_unlock(obj);
+ if (result == 0) {
+ /*
+ * If are writing to a new page, no need to read old data.
+ * The extent locking will have updated the KMS, and for our
+ * purposes here we can treat it like i_size.
+ */
+ if (attr->cat_kms <= offset) {
+ char *kaddr = kmap_atomic(vpg->vpg_page);
+
+ memset(kaddr, 0, cl_page_size(obj));
+ kunmap_atomic(kaddr);
+ } else if (vpg->vpg_defer_uptodate) {
+ vpg->vpg_ra_used = 1;
+ } else {
+ result = ll_page_sync_io(env, io, pg, CRT_READ);
+ }
+ }
+ return result;
}
static int ll_write_begin(struct file *file, struct address_space *mapping,
loff_t pos, unsigned len, unsigned flags,
struct page **pagep, void **fsdata)
{
+ struct ll_cl_context *lcc;
+ struct lu_env *env;
+ struct cl_io *io;
+ struct cl_page *page;
+ struct cl_object *clob = ll_i2info(mapping->host)->lli_clob;
pgoff_t index = pos >> PAGE_SHIFT;
- struct page *page;
- int rc;
- unsigned from = pos & (PAGE_SIZE - 1);
+ struct page *vmpage = NULL;
+ unsigned int from = pos & (PAGE_SIZE - 1);
+ unsigned int to = from + len;
+ int result = 0;
- page = grab_cache_page_write_begin(mapping, index, flags);
- if (!page)
- return -ENOMEM;
+ CDEBUG(D_VFSTRACE, "Writing %lu of %d to %d bytes\n", index, from, len);
- *pagep = page;
-
- rc = ll_prepare_write(file, page, from, from + len);
- if (rc) {
- unlock_page(page);
- put_page(page);
+ lcc = ll_cl_init(file, NULL);
+ if (IS_ERR(lcc)) {
+ result = PTR_ERR(lcc);
+ goto out;
}
- return rc;
+
+ env = lcc->lcc_env;
+ io = lcc->lcc_io;
+
+ /* To avoid deadlock, try to lock page first. */
+ vmpage = grab_cache_page_nowait(mapping, index);
+ if (unlikely(!vmpage || PageDirty(vmpage) || PageWriteback(vmpage))) {
+ struct vvp_io *vio = vvp_env_io(env);
+ struct cl_page_list *plist = &vio->u.write.vui_queue;
+
+ /* if the page is already in dirty cache, we have to commit
+ * the pages right now; otherwise, it may cause deadlock
+ * because it holds page lock of a dirty page and request for
+ * more grants. It's okay for the dirty page to be the first
+ * one in commit page list, though.
+ */
+ if (vmpage && plist->pl_nr > 0) {
+ unlock_page(vmpage);
+ put_page(vmpage);
+ vmpage = NULL;
+ }
+
+ /* commit pages and then wait for page lock */
+ result = vvp_io_write_commit(env, io);
+ if (result < 0)
+ goto out;
+
+ if (!vmpage) {
+ vmpage = grab_cache_page_write_begin(mapping, index,
+ flags);
+ if (!vmpage) {
+ result = -ENOMEM;
+ goto out;
+ }
+ }
+ }
+
+ page = cl_page_find(env, clob, vmpage->index, vmpage, CPT_CACHEABLE);
+ if (IS_ERR(page)) {
+ result = PTR_ERR(page);
+ goto out;
+ }
+
+ lcc->lcc_page = page;
+ lu_ref_add(&page->cp_reference, "cl_io", io);
+
+ cl_page_assume(env, io, page);
+ if (!PageUptodate(vmpage)) {
+ /*
+ * We're completely overwriting an existing page,
+ * so _don't_ set it up to date until commit_write
+ */
+ if (from == 0 && to == PAGE_SIZE) {
+ CL_PAGE_HEADER(D_PAGE, env, page, "full page write\n");
+ POISON_PAGE(vmpage, 0x11);
+ } else {
+ /* TODO: can be optimized at OSC layer to check if it
+ * is a lockless IO. In that case, it's not necessary
+ * to read the data.
+ */
+ result = ll_prepare_partial_page(env, io, page);
+ if (result == 0)
+ SetPageUptodate(vmpage);
+ }
+ }
+ if (result < 0)
+ cl_page_unassume(env, io, page);
+out:
+ if (result < 0) {
+ if (vmpage) {
+ unlock_page(vmpage);
+ put_page(vmpage);
+ }
+ if (!IS_ERR(lcc))
+ ll_cl_fini(lcc);
+ } else {
+ *pagep = vmpage;
+ *fsdata = lcc;
+ }
+ return result;
}
static int ll_write_end(struct file *file, struct address_space *mapping,
loff_t pos, unsigned len, unsigned copied,
- struct page *page, void *fsdata)
+ struct page *vmpage, void *fsdata)
{
+ struct ll_cl_context *lcc = fsdata;
+ struct lu_env *env;
+ struct cl_io *io;
+ struct vvp_io *vio;
+ struct cl_page *page;
unsigned from = pos & (PAGE_SIZE - 1);
- int rc;
+ bool unplug = false;
+ int result = 0;
- rc = ll_commit_write(file, page, from, from + copied);
- unlock_page(page);
- put_page(page);
+ put_page(vmpage);
- return rc ?: copied;
+ env = lcc->lcc_env;
+ page = lcc->lcc_page;
+ io = lcc->lcc_io;
+ vio = vvp_env_io(env);
+
+ LASSERT(cl_page_is_owned(page, io));
+ if (copied > 0) {
+ struct cl_page_list *plist = &vio->u.write.vui_queue;
+
+ lcc->lcc_page = NULL; /* page will be queued */
+
+ /* Add it into write queue */
+ cl_page_list_add(plist, page);
+ if (plist->pl_nr == 1) /* first page */
+ vio->u.write.vui_from = from;
+ else
+ LASSERT(from == 0);
+ vio->u.write.vui_to = from + copied;
+
+ /* We may have one full RPC, commit it soon */
+ if (plist->pl_nr >= PTLRPC_MAX_BRW_PAGES)
+ unplug = true;
+
+ CL_PAGE_DEBUG(D_VFSTRACE, env, page,
+ "queued page: %d.\n", plist->pl_nr);
+ } else {
+ cl_page_disown(env, io, page);
+
+ /* page list is not contiguous now, commit it now */
+ unplug = true;
+ }
+
+ if (unplug ||
+ file->f_flags & O_SYNC || IS_SYNC(file_inode(file)))
+ result = vvp_io_write_commit(env, io);
+
+ ll_cl_fini(lcc);
+ return result >= 0 ? copied : result;
}
#ifdef CONFIG_MIGRATION
@@ -523,7 +659,7 @@
.direct_IO = ll_direct_IO_26,
.writepage = ll_writepage,
.writepages = ll_writepages,
- .set_page_dirty = ll_set_page_dirty,
+ .set_page_dirty = __set_page_dirty_nobuffers,
.write_begin = ll_write_begin,
.write_end = ll_write_end,
.invalidatepage = ll_invalidatepage,
diff --git a/drivers/staging/lustre/lustre/llite/statahead.c b/drivers/staging/lustre/lustre/llite/statahead.c
index 99ffd15..6322f88 100644
--- a/drivers/staging/lustre/lustre/llite/statahead.c
+++ b/drivers/staging/lustre/lustre/llite/statahead.c
@@ -661,8 +661,9 @@
if (rc)
goto out;
- CDEBUG(D_DLMTRACE, "setting l_data to inode %p (%lu/%u)\n",
- child, child->i_ino, child->i_generation);
+ CDEBUG(D_DLMTRACE, "%s: setting l_data to inode "DFID"%p\n",
+ ll_get_fsname(child->i_sb, NULL, 0),
+ PFID(ll_inode2fid(child)), child);
ll_set_lock_data(ll_i2sbi(dir)->ll_md_exp, child, it, NULL);
entry->se_inode = child;
@@ -1591,13 +1592,11 @@
*dentryp = alias;
} else if (d_inode(*dentryp) != inode) {
/* revalidate, but inode is recreated */
- CDEBUG(D_READA,
- "stale dentry %pd inode %lu/%u, statahead inode %lu/%u\n",
- *dentryp,
- d_inode(*dentryp)->i_ino,
- d_inode(*dentryp)->i_generation,
- inode->i_ino,
- inode->i_generation);
+ CDEBUG(D_READA, "%s: stale dentry %pd inode "DFID", statahead inode "DFID"\n",
+ ll_get_fsname(d_inode(*dentryp)->i_sb, NULL, 0),
+ *dentryp,
+ PFID(ll_inode2fid(d_inode(*dentryp))),
+ PFID(ll_inode2fid(inode)));
ll_sai_unplug(sai, entry);
return -ESTALE;
} else {
diff --git a/drivers/staging/lustre/lustre/llite/super25.c b/drivers/staging/lustre/lustre/llite/super25.c
index 61856d3..415750b 100644
--- a/drivers/staging/lustre/lustre/llite/super25.c
+++ b/drivers/staging/lustre/lustre/llite/super25.c
@@ -164,9 +164,18 @@
if (rc != 0)
goto out_sysfs;
+ cl_inode_fini_env = cl_env_alloc(&cl_inode_fini_refcheck,
+ LCT_REMEMBER | LCT_NOREF);
+ if (IS_ERR(cl_inode_fini_env)) {
+ rc = PTR_ERR(cl_inode_fini_env);
+ goto out_vvp;
+ }
+
+ cl_inode_fini_env->le_ctx.lc_cookie = 0x4;
+
rc = ll_xattr_init();
if (rc != 0)
- goto out_vvp;
+ goto out_inode_fini_env;
lustre_register_client_fill_super(ll_fill_super);
lustre_register_kill_super_cb(ll_kill_super);
@@ -174,6 +183,8 @@
return 0;
+out_inode_fini_env:
+ cl_env_put(cl_inode_fini_env, &cl_inode_fini_refcheck);
out_vvp:
vvp_global_fini();
out_sysfs:
@@ -198,6 +209,7 @@
kset_unregister(llite_kset);
ll_xattr_fini();
+ cl_env_put(cl_inode_fini_env, &cl_inode_fini_refcheck);
vvp_global_fini();
kmem_cache_destroy(ll_inode_cachep);
diff --git a/drivers/staging/lustre/lustre/llite/symlink.c b/drivers/staging/lustre/lustre/llite/symlink.c
index 46d03ea..3fc736c 100644
--- a/drivers/staging/lustre/lustre/llite/symlink.c
+++ b/drivers/staging/lustre/lustre/llite/symlink.c
@@ -77,7 +77,9 @@
ll_finish_md_op_data(op_data);
if (rc) {
if (rc != -ENOENT)
- CERROR("inode %lu: rc = %d\n", inode->i_ino, rc);
+ CERROR("%s: inode "DFID": rc = %d\n",
+ ll_get_fsname(inode->i_sb, NULL, 0),
+ PFID(ll_inode2fid(inode)), rc);
goto failed;
}
@@ -90,8 +92,10 @@
LASSERT(symlen != 0);
if (body->eadatasize != symlen) {
- CERROR("inode %lu: symlink length %d not expected %d\n",
- inode->i_ino, body->eadatasize - 1, symlen - 1);
+ CERROR("%s: inode "DFID": symlink length %d not expected %d\n",
+ ll_get_fsname(inode->i_sb, NULL, 0),
+ PFID(ll_inode2fid(inode)), body->eadatasize - 1,
+ symlen - 1);
rc = -EPROTO;
goto failed;
}
diff --git a/drivers/staging/lustre/lustre/llite/vvp_dev.c b/drivers/staging/lustre/lustre/llite/vvp_dev.c
index 282b70b..47101de 100644
--- a/drivers/staging/lustre/lustre/llite/vvp_dev.c
+++ b/drivers/staging/lustre/lustre/llite/vvp_dev.c
@@ -36,6 +36,7 @@
* cl_device and cl_device_type implementation for VVP layer.
*
* Author: Nikita Danilov <nikita.danilov@sun.com>
+ * Author: Jinshan Xiong <jinshan.xiong@intel.com>
*/
#define DEBUG_SUBSYSTEM S_LLITE
@@ -56,13 +57,33 @@
* "llite_" (var. "ll_") prefix.
*/
-static struct kmem_cache *vvp_thread_kmem;
+static struct kmem_cache *ll_thread_kmem;
+struct kmem_cache *vvp_lock_kmem;
+struct kmem_cache *vvp_object_kmem;
+struct kmem_cache *vvp_req_kmem;
static struct kmem_cache *vvp_session_kmem;
+static struct kmem_cache *vvp_thread_kmem;
+
static struct lu_kmem_descr vvp_caches[] = {
{
- .ckd_cache = &vvp_thread_kmem,
- .ckd_name = "vvp_thread_kmem",
- .ckd_size = sizeof(struct vvp_thread_info),
+ .ckd_cache = &ll_thread_kmem,
+ .ckd_name = "ll_thread_kmem",
+ .ckd_size = sizeof(struct ll_thread_info),
+ },
+ {
+ .ckd_cache = &vvp_lock_kmem,
+ .ckd_name = "vvp_lock_kmem",
+ .ckd_size = sizeof(struct vvp_lock),
+ },
+ {
+ .ckd_cache = &vvp_object_kmem,
+ .ckd_name = "vvp_object_kmem",
+ .ckd_size = sizeof(struct vvp_object),
+ },
+ {
+ .ckd_cache = &vvp_req_kmem,
+ .ckd_name = "vvp_req_kmem",
+ .ckd_size = sizeof(struct vvp_req),
},
{
.ckd_cache = &vvp_session_kmem,
@@ -70,29 +91,40 @@
.ckd_size = sizeof(struct vvp_session)
},
{
+ .ckd_cache = &vvp_thread_kmem,
+ .ckd_name = "vvp_thread_kmem",
+ .ckd_size = sizeof(struct vvp_thread_info),
+ },
+ {
.ckd_cache = NULL
}
};
-static void *vvp_key_init(const struct lu_context *ctx,
- struct lu_context_key *key)
+static void *ll_thread_key_init(const struct lu_context *ctx,
+ struct lu_context_key *key)
{
struct vvp_thread_info *info;
- info = kmem_cache_zalloc(vvp_thread_kmem, GFP_NOFS);
+ info = kmem_cache_zalloc(ll_thread_kmem, GFP_NOFS);
if (!info)
info = ERR_PTR(-ENOMEM);
return info;
}
-static void vvp_key_fini(const struct lu_context *ctx,
- struct lu_context_key *key, void *data)
+static void ll_thread_key_fini(const struct lu_context *ctx,
+ struct lu_context_key *key, void *data)
{
struct vvp_thread_info *info = data;
- kmem_cache_free(vvp_thread_kmem, info);
+ kmem_cache_free(ll_thread_kmem, info);
}
+struct lu_context_key ll_thread_key = {
+ .lct_tags = LCT_CL_THREAD,
+ .lct_init = ll_thread_key_init,
+ .lct_fini = ll_thread_key_fini
+};
+
static void *vvp_session_key_init(const struct lu_context *ctx,
struct lu_context_key *key)
{
@@ -112,34 +144,127 @@
kmem_cache_free(vvp_session_kmem, session);
}
-struct lu_context_key vvp_key = {
- .lct_tags = LCT_CL_THREAD,
- .lct_init = vvp_key_init,
- .lct_fini = vvp_key_fini
-};
-
struct lu_context_key vvp_session_key = {
.lct_tags = LCT_SESSION,
.lct_init = vvp_session_key_init,
.lct_fini = vvp_session_key_fini
};
+void *vvp_thread_key_init(const struct lu_context *ctx,
+ struct lu_context_key *key)
+{
+ struct vvp_thread_info *vti;
+
+ vti = kmem_cache_zalloc(vvp_thread_kmem, GFP_NOFS);
+ if (!vti)
+ vti = ERR_PTR(-ENOMEM);
+ return vti;
+}
+
+void vvp_thread_key_fini(const struct lu_context *ctx,
+ struct lu_context_key *key, void *data)
+{
+ struct vvp_thread_info *vti = data;
+
+ kmem_cache_free(vvp_thread_kmem, vti);
+}
+
+struct lu_context_key vvp_thread_key = {
+ .lct_tags = LCT_CL_THREAD,
+ .lct_init = vvp_thread_key_init,
+ .lct_fini = vvp_thread_key_fini
+};
+
/* type constructor/destructor: vvp_type_{init,fini,start,stop}(). */
-LU_TYPE_INIT_FINI(vvp, &ccc_key, &ccc_session_key, &vvp_key, &vvp_session_key);
+LU_TYPE_INIT_FINI(vvp, &vvp_thread_key, &ll_thread_key, &vvp_session_key);
static const struct lu_device_operations vvp_lu_ops = {
.ldo_object_alloc = vvp_object_alloc
};
static const struct cl_device_operations vvp_cl_ops = {
- .cdo_req_init = ccc_req_init
+ .cdo_req_init = vvp_req_init
};
+static struct lu_device *vvp_device_free(const struct lu_env *env,
+ struct lu_device *d)
+{
+ struct vvp_device *vdv = lu2vvp_dev(d);
+ struct cl_site *site = lu2cl_site(d->ld_site);
+ struct lu_device *next = cl2lu_dev(vdv->vdv_next);
+
+ if (d->ld_site) {
+ cl_site_fini(site);
+ kfree(site);
+ }
+ cl_device_fini(lu2cl_dev(d));
+ kfree(vdv);
+ return next;
+}
+
static struct lu_device *vvp_device_alloc(const struct lu_env *env,
struct lu_device_type *t,
struct lustre_cfg *cfg)
{
- return ccc_device_alloc(env, t, cfg, &vvp_lu_ops, &vvp_cl_ops);
+ struct vvp_device *vdv;
+ struct lu_device *lud;
+ struct cl_site *site;
+ int rc;
+
+ vdv = kzalloc(sizeof(*vdv), GFP_NOFS);
+ if (!vdv)
+ return ERR_PTR(-ENOMEM);
+
+ lud = &vdv->vdv_cl.cd_lu_dev;
+ cl_device_init(&vdv->vdv_cl, t);
+ vvp2lu_dev(vdv)->ld_ops = &vvp_lu_ops;
+ vdv->vdv_cl.cd_ops = &vvp_cl_ops;
+
+ site = kzalloc(sizeof(*site), GFP_NOFS);
+ if (site) {
+ rc = cl_site_init(site, &vdv->vdv_cl);
+ if (rc == 0) {
+ rc = lu_site_init_finish(&site->cs_lu);
+ } else {
+ LASSERT(!lud->ld_site);
+ CERROR("Cannot init lu_site, rc %d.\n", rc);
+ kfree(site);
+ }
+ } else {
+ rc = -ENOMEM;
+ }
+ if (rc != 0) {
+ vvp_device_free(env, lud);
+ lud = ERR_PTR(rc);
+ }
+ return lud;
+}
+
+static int vvp_device_init(const struct lu_env *env, struct lu_device *d,
+ const char *name, struct lu_device *next)
+{
+ struct vvp_device *vdv;
+ int rc;
+
+ vdv = lu2vvp_dev(d);
+ vdv->vdv_next = lu2cl_dev(next);
+
+ LASSERT(d->ld_site && next->ld_type);
+ next->ld_site = d->ld_site;
+ rc = next->ld_type->ldt_ops->ldto_device_init(env, next,
+ next->ld_type->ldt_name,
+ NULL);
+ if (rc == 0) {
+ lu_device_get(next);
+ lu_ref_add(&next->ld_reference, "lu-stack", &lu_site_init);
+ }
+ return rc;
+}
+
+static struct lu_device *vvp_device_fini(const struct lu_env *env,
+ struct lu_device *d)
+{
+ return cl2lu_dev(lu2vvp_dev(d)->vdv_next);
}
static const struct lu_device_type_operations vvp_device_type_ops = {
@@ -150,9 +275,9 @@
.ldto_stop = vvp_type_stop,
.ldto_device_alloc = vvp_device_alloc,
- .ldto_device_free = ccc_device_free,
- .ldto_device_init = ccc_device_init,
- .ldto_device_fini = ccc_device_fini
+ .ldto_device_free = vvp_device_free,
+ .ldto_device_init = vvp_device_init,
+ .ldto_device_fini = vvp_device_fini,
};
struct lu_device_type vvp_device_type = {
@@ -168,20 +293,27 @@
*/
int vvp_global_init(void)
{
- int result;
+ int rc;
- result = lu_kmem_init(vvp_caches);
- if (result == 0) {
- result = ccc_global_init(&vvp_device_type);
- if (result != 0)
- lu_kmem_fini(vvp_caches);
- }
- return result;
+ rc = lu_kmem_init(vvp_caches);
+ if (rc != 0)
+ return rc;
+
+ rc = lu_device_type_init(&vvp_device_type);
+ if (rc != 0)
+ goto out_kmem;
+
+ return 0;
+
+out_kmem:
+ lu_kmem_fini(vvp_caches);
+
+ return rc;
}
void vvp_global_fini(void)
{
- ccc_global_fini(&vvp_device_type);
+ lu_device_type_fini(&vvp_device_type);
lu_kmem_fini(vvp_caches);
}
@@ -205,13 +337,14 @@
cl = cl_type_setup(env, NULL, &vvp_device_type,
sbi->ll_dt_exp->exp_obd->obd_lu_dev);
if (!IS_ERR(cl)) {
- cl2ccc_dev(cl)->cdv_sb = sb;
+ cl2vvp_dev(cl)->vdv_sb = sb;
sbi->ll_cl = cl;
sbi->ll_site = cl2lu_dev(cl)->ld_site;
}
cl_env_put(env, &refcheck);
- } else
+ } else {
rc = PTR_ERR(env);
+ }
return rc;
}
@@ -356,23 +489,18 @@
return ~0ULL;
clob = vvp_pgcache_obj(env, dev, &id);
if (clob) {
- struct cl_object_header *hdr;
- int nr;
- struct cl_page *pg;
+ struct inode *inode = vvp_object_inode(clob);
+ struct page *vmpage;
+ int nr;
- /* got an object. Find next page. */
- hdr = cl_object_header(clob);
-
- spin_lock(&hdr->coh_page_guard);
- nr = radix_tree_gang_lookup(&hdr->coh_tree,
- (void **)&pg,
- id.vpi_index, 1);
+ nr = find_get_pages_contig(inode->i_mapping,
+ id.vpi_index, 1, &vmpage);
if (nr > 0) {
- id.vpi_index = pg->cp_index;
+ id.vpi_index = vmpage->index;
/* Cant support over 16T file */
- nr = !(pg->cp_index > 0xffffffff);
+ nr = !(vmpage->index > 0xffffffff);
+ put_page(vmpage);
}
- spin_unlock(&hdr->coh_page_guard);
lu_object_ref_del(&clob->co_lu, "dump", current);
cl_object_put(env, clob);
@@ -398,21 +526,20 @@
static void vvp_pgcache_page_show(const struct lu_env *env,
struct seq_file *seq, struct cl_page *page)
{
- struct ccc_page *cpg;
+ struct vvp_page *vpg;
struct page *vmpage;
int has_flags;
- cpg = cl2ccc_page(cl_page_at(page, &vvp_device_type));
- vmpage = cpg->cpg_page;
- seq_printf(seq, " %5i | %p %p %s %s %s %s | %p %lu/%u(%p) %lu %u [",
+ vpg = cl2vvp_page(cl_page_at(page, &vvp_device_type));
+ vmpage = vpg->vpg_page;
+ seq_printf(seq, " %5i | %p %p %s %s %s %s | %p "DFID"(%p) %lu %u [",
0 /* gen */,
- cpg, page,
+ vpg, page,
"none",
- cpg->cpg_write_queued ? "wq" : "- ",
- cpg->cpg_defer_uptodate ? "du" : "- ",
+ vpg->vpg_write_queued ? "wq" : "- ",
+ vpg->vpg_defer_uptodate ? "du" : "- ",
PageWriteback(vmpage) ? "wb" : "-",
- vmpage, vmpage->mapping->host->i_ino,
- vmpage->mapping->host->i_generation,
+ vmpage, PFID(ll_inode2fid(vmpage->mapping->host)),
vmpage->mapping->host, vmpage->index,
page_count(vmpage));
has_flags = 0;
@@ -431,8 +558,6 @@
struct ll_sb_info *sbi;
struct cl_object *clob;
struct lu_env *env;
- struct cl_page *page;
- struct cl_object_header *hdr;
struct vvp_pgcache_id id;
int refcheck;
int result;
@@ -444,27 +569,38 @@
sbi = f->private;
clob = vvp_pgcache_obj(env, &sbi->ll_cl->cd_lu_dev, &id);
if (clob) {
- hdr = cl_object_header(clob);
+ struct inode *inode = vvp_object_inode(clob);
+ struct cl_page *page = NULL;
+ struct page *vmpage;
- spin_lock(&hdr->coh_page_guard);
- page = cl_page_lookup(hdr, id.vpi_index);
- spin_unlock(&hdr->coh_page_guard);
+ result = find_get_pages_contig(inode->i_mapping,
+ id.vpi_index, 1,
+ &vmpage);
+ if (result > 0) {
+ lock_page(vmpage);
+ page = cl_vmpage_page(vmpage, clob);
+ unlock_page(vmpage);
+ put_page(vmpage);
+ }
- seq_printf(f, "%8x@"DFID": ",
- id.vpi_index, PFID(&hdr->coh_lu.loh_fid));
+ seq_printf(f, "%8x@" DFID ": ", id.vpi_index,
+ PFID(lu_object_fid(&clob->co_lu)));
if (page) {
vvp_pgcache_page_show(env, f, page);
cl_page_put(env, page);
- } else
+ } else {
seq_puts(f, "missing\n");
+ }
lu_object_ref_del(&clob->co_lu, "dump", current);
cl_object_put(env, clob);
- } else
+ } else {
seq_printf(f, "%llx missing\n", pos);
+ }
cl_env_put(env, &refcheck);
result = 0;
- } else
+ } else {
result = PTR_ERR(env);
+ }
return result;
}
diff --git a/drivers/staging/lustre/lustre/llite/vvp_internal.h b/drivers/staging/lustre/lustre/llite/vvp_internal.h
index bb39337..27b9b0a 100644
--- a/drivers/staging/lustre/lustre/llite/vvp_internal.h
+++ b/drivers/staging/lustre/lustre/llite/vvp_internal.h
@@ -41,21 +41,337 @@
#ifndef VVP_INTERNAL_H
#define VVP_INTERNAL_H
+#include "../include/lustre/lustre_idl.h"
#include "../include/cl_object.h"
-#include "llite_internal.h"
-int vvp_io_init(const struct lu_env *env,
- struct cl_object *obj, struct cl_io *io);
-int vvp_lock_init(const struct lu_env *env,
- struct cl_object *obj, struct cl_lock *lock,
- const struct cl_io *io);
+enum obd_notify_event;
+struct inode;
+struct lov_stripe_md;
+struct lustre_md;
+struct obd_capa;
+struct obd_device;
+struct obd_export;
+struct page;
+
+/* specific architecture can implement only part of this list */
+enum vvp_io_subtype {
+ /** normal IO */
+ IO_NORMAL,
+ /** io started from splice_{read|write} */
+ IO_SPLICE
+};
+
+/**
+ * IO state private to IO state private to VVP layer.
+ */
+struct vvp_io {
+ /** super class */
+ struct cl_io_slice vui_cl;
+ struct cl_io_lock_link vui_link;
+ /**
+ * I/O vector information to or from which read/write is going.
+ */
+ struct iov_iter *vui_iter;
+ /**
+ * Total size for the left IO.
+ */
+ size_t vui_tot_count;
+
+ union {
+ struct vvp_fault_io {
+ /**
+ * Inode modification time that is checked across DLM
+ * lock request.
+ */
+ time64_t ft_mtime;
+ struct vm_area_struct *ft_vma;
+ /**
+ * locked page returned from vvp_io
+ */
+ struct page *ft_vmpage;
+ /**
+ * kernel fault info
+ */
+ struct vm_fault *ft_vmf;
+ /**
+ * fault API used bitflags for return code.
+ */
+ unsigned int ft_flags;
+ /**
+ * check that flags are from filemap_fault
+ */
+ bool ft_flags_valid;
+ } fault;
+ struct {
+ struct pipe_inode_info *vui_pipe;
+ unsigned int vui_flags;
+ } splice;
+ struct {
+ struct cl_page_list vui_queue;
+ unsigned long vui_written;
+ int vui_from;
+ int vui_to;
+ } write;
+ } u;
+
+ enum vvp_io_subtype vui_io_subtype;
+
+ /**
+ * Layout version when this IO is initialized
+ */
+ __u32 vui_layout_gen;
+ /**
+ * File descriptor against which IO is done.
+ */
+ struct ll_file_data *vui_fd;
+ struct kiocb *vui_iocb;
+
+ /* Readahead state. */
+ pgoff_t vui_ra_start;
+ pgoff_t vui_ra_count;
+ /* Set when vui_ra_{start,count} have been initialized. */
+ bool vui_ra_valid;
+};
+
+extern struct lu_device_type vvp_device_type;
+
+extern struct lu_context_key vvp_session_key;
+extern struct lu_context_key vvp_thread_key;
+
+extern struct kmem_cache *vvp_lock_kmem;
+extern struct kmem_cache *vvp_object_kmem;
+extern struct kmem_cache *vvp_req_kmem;
+
+struct vvp_thread_info {
+ struct cl_lock vti_lock;
+ struct cl_lock_descr vti_descr;
+ struct cl_io vti_io;
+ struct cl_attr vti_attr;
+};
+
+static inline struct vvp_thread_info *vvp_env_info(const struct lu_env *env)
+{
+ struct vvp_thread_info *vti;
+
+ vti = lu_context_key_get(&env->le_ctx, &vvp_thread_key);
+ LASSERT(vti);
+
+ return vti;
+}
+
+static inline struct cl_lock *vvp_env_lock(const struct lu_env *env)
+{
+ struct cl_lock *lock = &vvp_env_info(env)->vti_lock;
+
+ memset(lock, 0, sizeof(*lock));
+ return lock;
+}
+
+static inline struct cl_attr *vvp_env_thread_attr(const struct lu_env *env)
+{
+ struct cl_attr *attr = &vvp_env_info(env)->vti_attr;
+
+ memset(attr, 0, sizeof(*attr));
+
+ return attr;
+}
+
+static inline struct cl_io *vvp_env_thread_io(const struct lu_env *env)
+{
+ struct cl_io *io = &vvp_env_info(env)->vti_io;
+
+ memset(io, 0, sizeof(*io));
+
+ return io;
+}
+
+struct vvp_session {
+ struct vvp_io cs_ios;
+};
+
+static inline struct vvp_session *vvp_env_session(const struct lu_env *env)
+{
+ struct vvp_session *ses;
+
+ ses = lu_context_key_get(env->le_ses, &vvp_session_key);
+ LASSERT(ses);
+
+ return ses;
+}
+
+static inline struct vvp_io *vvp_env_io(const struct lu_env *env)
+{
+ return &vvp_env_session(env)->cs_ios;
+}
+
+/**
+ * ccc-private object state.
+ */
+struct vvp_object {
+ struct cl_object_header vob_header;
+ struct cl_object vob_cl;
+ struct inode *vob_inode;
+
+ /**
+ * A list of dirty pages pending IO in the cache. Used by
+ * SOM. Protected by ll_inode_info::lli_lock.
+ *
+ * \see vvp_page::vpg_pending_linkage
+ */
+ struct list_head vob_pending_list;
+
+ /**
+ * Access this counter is protected by inode->i_sem. Now that
+ * the lifetime of transient pages must be covered by inode sem,
+ * we don't need to hold any lock..
+ */
+ int vob_transient_pages;
+ /**
+ * Number of outstanding mmaps on this file.
+ *
+ * \see ll_vm_open(), ll_vm_close().
+ */
+ atomic_t vob_mmap_cnt;
+
+ /**
+ * various flags
+ * vob_discard_page_warned
+ * if pages belonging to this object are discarded when a client
+ * is evicted, some debug info will be printed, this flag will be set
+ * during processing the first discarded page, then avoid flooding
+ * debug message for lots of discarded pages.
+ *
+ * \see ll_dirty_page_discard_warn.
+ */
+ unsigned int vob_discard_page_warned:1;
+};
+
+/**
+ * VVP-private page state.
+ */
+struct vvp_page {
+ struct cl_page_slice vpg_cl;
+ int vpg_defer_uptodate;
+ int vpg_ra_used;
+ int vpg_write_queued;
+ /**
+ * Non-empty iff this page is already counted in
+ * vvp_object::vob_pending_list. This list is only used as a flag,
+ * that is, never iterated through, only checked for list_empty(), but
+ * having a list is useful for debugging.
+ */
+ struct list_head vpg_pending_linkage;
+ /** VM page */
+ struct page *vpg_page;
+};
+
+static inline struct vvp_page *cl2vvp_page(const struct cl_page_slice *slice)
+{
+ return container_of(slice, struct vvp_page, vpg_cl);
+}
+
+static inline pgoff_t vvp_index(struct vvp_page *vvp)
+{
+ return vvp->vpg_cl.cpl_index;
+}
+
+struct vvp_device {
+ struct cl_device vdv_cl;
+ struct super_block *vdv_sb;
+ struct cl_device *vdv_next;
+};
+
+struct vvp_lock {
+ struct cl_lock_slice vlk_cl;
+};
+
+struct vvp_req {
+ struct cl_req_slice vrq_cl;
+};
+
+void *ccc_key_init(const struct lu_context *ctx,
+ struct lu_context_key *key);
+void ccc_key_fini(const struct lu_context *ctx,
+ struct lu_context_key *key, void *data);
+
+void ccc_umount(const struct lu_env *env, struct cl_device *dev);
+
+static inline struct lu_device *vvp2lu_dev(struct vvp_device *vdv)
+{
+ return &vdv->vdv_cl.cd_lu_dev;
+}
+
+static inline struct vvp_device *lu2vvp_dev(const struct lu_device *d)
+{
+ return container_of0(d, struct vvp_device, vdv_cl.cd_lu_dev);
+}
+
+static inline struct vvp_device *cl2vvp_dev(const struct cl_device *d)
+{
+ return container_of0(d, struct vvp_device, vdv_cl);
+}
+
+static inline struct vvp_object *cl2vvp(const struct cl_object *obj)
+{
+ return container_of0(obj, struct vvp_object, vob_cl);
+}
+
+static inline struct vvp_object *lu2vvp(const struct lu_object *obj)
+{
+ return container_of0(obj, struct vvp_object, vob_cl.co_lu);
+}
+
+static inline struct inode *vvp_object_inode(const struct cl_object *obj)
+{
+ return cl2vvp(obj)->vob_inode;
+}
+
+int vvp_object_invariant(const struct cl_object *obj);
+struct vvp_object *cl_inode2vvp(struct inode *inode);
+
+static inline struct page *cl2vm_page(const struct cl_page_slice *slice)
+{
+ return cl2vvp_page(slice)->vpg_page;
+}
+
+static inline struct vvp_lock *cl2vvp_lock(const struct cl_lock_slice *slice)
+{
+ return container_of(slice, struct vvp_lock, vlk_cl);
+}
+
+# define CLOBINVRNT(env, clob, expr) \
+ ((void)sizeof(env), (void)sizeof(clob), (void)sizeof(!!(expr)))
+
+/**
+ * New interfaces to get and put lov_stripe_md from lov layer. This violates
+ * layering because lov_stripe_md is supposed to be a private data in lov.
+ *
+ * NB: If you find you have to use these interfaces for your new code, please
+ * think about it again. These interfaces may be removed in the future for
+ * better layering.
+ */
+struct lov_stripe_md *lov_lsm_get(struct cl_object *clobj);
+void lov_lsm_put(struct cl_object *clobj, struct lov_stripe_md *lsm);
+int lov_read_and_clear_async_rc(struct cl_object *clob);
+
+struct lov_stripe_md *ccc_inode_lsm_get(struct inode *inode);
+void ccc_inode_lsm_put(struct inode *inode, struct lov_stripe_md *lsm);
+
+int vvp_io_init(const struct lu_env *env, struct cl_object *obj,
+ struct cl_io *io);
+int vvp_io_write_commit(const struct lu_env *env, struct cl_io *io);
+int vvp_lock_init(const struct lu_env *env, struct cl_object *obj,
+ struct cl_lock *lock, const struct cl_io *io);
int vvp_page_init(const struct lu_env *env, struct cl_object *obj,
- struct cl_page *page, struct page *vmpage);
+ struct cl_page *page, pgoff_t index);
+int vvp_req_init(const struct lu_env *env, struct cl_device *dev,
+ struct cl_req *req);
struct lu_object *vvp_object_alloc(const struct lu_env *env,
const struct lu_object_header *hdr,
struct lu_device *dev);
-struct ccc_object *cl_inode2ccc(struct inode *inode);
+int vvp_global_init(void);
+void vvp_global_fini(void);
extern const struct file_operations vvp_dump_pgcache_file_ops;
diff --git a/drivers/staging/lustre/lustre/llite/vvp_io.c b/drivers/staging/lustre/lustre/llite/vvp_io.c
index 85a8359..5bf9592 100644
--- a/drivers/staging/lustre/lustre/llite/vvp_io.c
+++ b/drivers/staging/lustre/lustre/llite/vvp_io.c
@@ -44,21 +44,30 @@
#include "../include/obd.h"
#include "../include/lustre_lite.h"
+#include "llite_internal.h"
#include "vvp_internal.h"
-static struct vvp_io *cl2vvp_io(const struct lu_env *env,
- const struct cl_io_slice *slice);
+struct vvp_io *cl2vvp_io(const struct lu_env *env,
+ const struct cl_io_slice *slice)
+{
+ struct vvp_io *vio;
+
+ vio = container_of(slice, struct vvp_io, vui_cl);
+ LASSERT(vio == vvp_env_io(env));
+
+ return vio;
+}
/**
* True, if \a io is a normal io, False for splice_{read,write}
*/
-int cl_is_normalio(const struct lu_env *env, const struct cl_io *io)
+static int cl_is_normalio(const struct lu_env *env, const struct cl_io *io)
{
struct vvp_io *vio = vvp_env_io(env);
LASSERT(io->ci_type == CIT_READ || io->ci_type == CIT_WRITE);
- return vio->cui_io_subtype == IO_NORMAL;
+ return vio->vui_io_subtype == IO_NORMAL;
}
/**
@@ -71,7 +80,7 @@
struct inode *inode)
{
struct ll_inode_info *lli = ll_i2info(inode);
- struct ccc_io *cio = ccc_env_io(env);
+ struct vvp_io *vio = vvp_env_io(env);
bool rc = true;
switch (io->ci_type) {
@@ -80,7 +89,7 @@
/* don't need lock here to check lli_layout_gen as we have held
* extent lock and GROUP lock has to hold to swap layout
*/
- if (ll_layout_version_get(lli) != cio->cui_layout_gen) {
+ if (ll_layout_version_get(lli) != vio->vui_layout_gen) {
io->ci_need_restart = 1;
/* this will return application a short read/write */
io->ci_continue = 0;
@@ -95,20 +104,187 @@
return rc;
}
+static void vvp_object_size_lock(struct cl_object *obj)
+{
+ struct inode *inode = vvp_object_inode(obj);
+
+ ll_inode_size_lock(inode);
+ cl_object_attr_lock(obj);
+}
+
+static void vvp_object_size_unlock(struct cl_object *obj)
+{
+ struct inode *inode = vvp_object_inode(obj);
+
+ cl_object_attr_unlock(obj);
+ ll_inode_size_unlock(inode);
+}
+
+/**
+ * Helper function that if necessary adjusts file size (inode->i_size), when
+ * position at the offset \a pos is accessed. File size can be arbitrary stale
+ * on a Lustre client, but client at least knows KMS. If accessed area is
+ * inside [0, KMS], set file size to KMS, otherwise glimpse file size.
+ *
+ * Locking: cl_isize_lock is used to serialize changes to inode size and to
+ * protect consistency between inode size and cl_object
+ * attributes. cl_object_size_lock() protects consistency between cl_attr's of
+ * top-object and sub-objects.
+ */
+static int vvp_prep_size(const struct lu_env *env, struct cl_object *obj,
+ struct cl_io *io, loff_t start, size_t count,
+ int *exceed)
+{
+ struct cl_attr *attr = vvp_env_thread_attr(env);
+ struct inode *inode = vvp_object_inode(obj);
+ loff_t pos = start + count - 1;
+ loff_t kms;
+ int result;
+
+ /*
+ * Consistency guarantees: following possibilities exist for the
+ * relation between region being accessed and real file size at this
+ * moment:
+ *
+ * (A): the region is completely inside of the file;
+ *
+ * (B-x): x bytes of region are inside of the file, the rest is
+ * outside;
+ *
+ * (C): the region is completely outside of the file.
+ *
+ * This classification is stable under DLM lock already acquired by
+ * the caller, because to change the class, other client has to take
+ * DLM lock conflicting with our lock. Also, any updates to ->i_size
+ * by other threads on this client are serialized by
+ * ll_inode_size_lock(). This guarantees that short reads are handled
+ * correctly in the face of concurrent writes and truncates.
+ */
+ vvp_object_size_lock(obj);
+ result = cl_object_attr_get(env, obj, attr);
+ if (result == 0) {
+ kms = attr->cat_kms;
+ if (pos > kms) {
+ /*
+ * A glimpse is necessary to determine whether we
+ * return a short read (B) or some zeroes at the end
+ * of the buffer (C)
+ */
+ vvp_object_size_unlock(obj);
+ result = cl_glimpse_lock(env, io, inode, obj, 0);
+ if (result == 0 && exceed) {
+ /* If objective page index exceed end-of-file
+ * page index, return directly. Do not expect
+ * kernel will check such case correctly.
+ * linux-2.6.18-128.1.1 miss to do that.
+ * --bug 17336
+ */
+ loff_t size = i_size_read(inode);
+ loff_t cur_index = start >> PAGE_SHIFT;
+ loff_t size_index = (size - 1) >> PAGE_SHIFT;
+
+ if ((size == 0 && cur_index != 0) ||
+ size_index < cur_index)
+ *exceed = 1;
+ }
+ return result;
+ }
+ /*
+ * region is within kms and, hence, within real file
+ * size (A). We need to increase i_size to cover the
+ * read region so that generic_file_read() will do its
+ * job, but that doesn't mean the kms size is
+ * _correct_, it is only the _minimum_ size. If
+ * someone does a stat they will get the correct size
+ * which will always be >= the kms value here.
+ * b=11081
+ */
+ if (i_size_read(inode) < kms) {
+ i_size_write(inode, kms);
+ CDEBUG(D_VFSTRACE, DFID " updating i_size %llu\n",
+ PFID(lu_object_fid(&obj->co_lu)),
+ (__u64)i_size_read(inode));
+ }
+ }
+
+ vvp_object_size_unlock(obj);
+
+ return result;
+}
+
/*****************************************************************************
*
* io operations.
*
*/
+static int vvp_io_one_lock_index(const struct lu_env *env, struct cl_io *io,
+ __u32 enqflags, enum cl_lock_mode mode,
+ pgoff_t start, pgoff_t end)
+{
+ struct vvp_io *vio = vvp_env_io(env);
+ struct cl_lock_descr *descr = &vio->vui_link.cill_descr;
+ struct cl_object *obj = io->ci_obj;
+
+ CLOBINVRNT(env, obj, vvp_object_invariant(obj));
+
+ CDEBUG(D_VFSTRACE, "lock: %d [%lu, %lu]\n", mode, start, end);
+
+ memset(&vio->vui_link, 0, sizeof(vio->vui_link));
+
+ if (vio->vui_fd && (vio->vui_fd->fd_flags & LL_FILE_GROUP_LOCKED)) {
+ descr->cld_mode = CLM_GROUP;
+ descr->cld_gid = vio->vui_fd->fd_grouplock.lg_gid;
+ } else {
+ descr->cld_mode = mode;
+ }
+ descr->cld_obj = obj;
+ descr->cld_start = start;
+ descr->cld_end = end;
+ descr->cld_enq_flags = enqflags;
+
+ cl_io_lock_add(env, io, &vio->vui_link);
+ return 0;
+}
+
+static int vvp_io_one_lock(const struct lu_env *env, struct cl_io *io,
+ __u32 enqflags, enum cl_lock_mode mode,
+ loff_t start, loff_t end)
+{
+ struct cl_object *obj = io->ci_obj;
+
+ return vvp_io_one_lock_index(env, io, enqflags, mode,
+ cl_index(obj, start), cl_index(obj, end));
+}
+
+static int vvp_io_write_iter_init(const struct lu_env *env,
+ const struct cl_io_slice *ios)
+{
+ struct vvp_io *vio = cl2vvp_io(env, ios);
+
+ cl_page_list_init(&vio->u.write.vui_queue);
+ vio->u.write.vui_written = 0;
+ vio->u.write.vui_from = 0;
+ vio->u.write.vui_to = PAGE_SIZE;
+
+ return 0;
+}
+
+static void vvp_io_write_iter_fini(const struct lu_env *env,
+ const struct cl_io_slice *ios)
+{
+ struct vvp_io *vio = cl2vvp_io(env, ios);
+
+ LASSERT(vio->u.write.vui_queue.pl_nr == 0);
+}
+
static int vvp_io_fault_iter_init(const struct lu_env *env,
const struct cl_io_slice *ios)
{
struct vvp_io *vio = cl2vvp_io(env, ios);
- struct inode *inode = ccc_object_inode(ios->cis_obj);
+ struct inode *inode = vvp_object_inode(ios->cis_obj);
- LASSERT(inode ==
- file_inode(cl2ccc_io(env, ios)->cui_fd->fd_file));
+ LASSERT(inode == file_inode(vio->vui_fd->fd_file));
vio->u.fault.ft_mtime = inode->i_mtime.tv_sec;
return 0;
}
@@ -117,15 +293,16 @@
{
struct cl_io *io = ios->cis_io;
struct cl_object *obj = io->ci_obj;
- struct ccc_io *cio = cl2ccc_io(env, ios);
+ struct vvp_io *vio = cl2vvp_io(env, ios);
+ struct inode *inode = vvp_object_inode(obj);
- CLOBINVRNT(env, obj, ccc_object_invariant(obj));
+ CLOBINVRNT(env, obj, vvp_object_invariant(obj));
CDEBUG(D_VFSTRACE, DFID
" ignore/verify layout %d/%d, layout version %d restore needed %d\n",
PFID(lu_object_fid(&obj->co_lu)),
io->ci_ignore_layout, io->ci_verify_layout,
- cio->cui_layout_gen, io->ci_restore_needed);
+ vio->vui_layout_gen, io->ci_restore_needed);
if (io->ci_restore_needed == 1) {
int rc;
@@ -133,7 +310,7 @@
/* file was detected release, we need to restore it
* before finishing the io
*/
- rc = ll_layout_restore(ccc_object_inode(obj));
+ rc = ll_layout_restore(inode, 0, OBD_OBJECT_EOF);
/* if restore registration failed, no restart,
* we will return -ENODATA
*/
@@ -159,16 +336,16 @@
__u32 gen = 0;
/* check layout version */
- ll_layout_refresh(ccc_object_inode(obj), &gen);
- io->ci_need_restart = cio->cui_layout_gen != gen;
+ ll_layout_refresh(inode, &gen);
+ io->ci_need_restart = vio->vui_layout_gen != gen;
if (io->ci_need_restart) {
CDEBUG(D_VFSTRACE,
DFID" layout changed from %d to %d.\n",
PFID(lu_object_fid(&obj->co_lu)),
- cio->cui_layout_gen, gen);
+ vio->vui_layout_gen, gen);
/* today successful restore is the only possible case */
/* restore was done, clear restoring state */
- ll_i2info(ccc_object_inode(obj))->lli_flags &=
+ ll_i2info(vvp_object_inode(obj))->lli_flags &=
~LLIF_FILE_RESTORING;
}
}
@@ -180,7 +357,7 @@
struct cl_io *io = ios->cis_io;
struct cl_page *page = io->u.ci_fault.ft_page;
- CLOBINVRNT(env, io->ci_obj, ccc_object_invariant(io->ci_obj));
+ CLOBINVRNT(env, io->ci_obj, vvp_object_invariant(io->ci_obj));
if (page) {
lu_ref_del(&page->cp_reference, "fault", io);
@@ -203,16 +380,16 @@
}
static int vvp_mmap_locks(const struct lu_env *env,
- struct ccc_io *vio, struct cl_io *io)
+ struct vvp_io *vio, struct cl_io *io)
{
- struct ccc_thread_info *cti = ccc_env_info(env);
+ struct vvp_thread_info *cti = vvp_env_info(env);
struct mm_struct *mm = current->mm;
struct vm_area_struct *vma;
- struct cl_lock_descr *descr = &cti->cti_descr;
+ struct cl_lock_descr *descr = &cti->vti_descr;
ldlm_policy_data_t policy;
unsigned long addr;
ssize_t count;
- int result;
+ int result = 0;
struct iov_iter i;
struct iovec iov;
@@ -221,21 +398,21 @@
if (!cl_is_normalio(env, io))
return 0;
- if (!vio->cui_iter) /* nfs or loop back device write */
+ if (!vio->vui_iter) /* nfs or loop back device write */
return 0;
/* No MM (e.g. NFS)? No vmas too. */
if (!mm)
return 0;
- iov_for_each(iov, i, *(vio->cui_iter)) {
+ iov_for_each(iov, i, *vio->vui_iter) {
addr = (unsigned long)iov.iov_base;
count = iov.iov_len;
if (count == 0)
continue;
- count += addr & (~CFS_PAGE_MASK);
- addr &= CFS_PAGE_MASK;
+ count += addr & (~PAGE_MASK);
+ addr &= PAGE_MASK;
down_read(&mm->mmap_sem);
while ((vma = our_vma(mm, addr, count)) != NULL) {
@@ -244,10 +421,10 @@
if (ll_file_nolock(vma->vm_file)) {
/*
- * For no lock case, a lockless lock will be
- * generated.
+ * For no lock case is not allowed for mmap
*/
- flags = CEF_NEVER;
+ result = -EINVAL;
+ break;
}
/*
@@ -269,10 +446,8 @@
descr->cld_mode, descr->cld_start,
descr->cld_end);
- if (result < 0) {
- up_read(&mm->mmap_sem);
- return result;
- }
+ if (result < 0)
+ break;
if (vma->vm_end - addr >= count)
break;
@@ -281,26 +456,55 @@
addr = vma->vm_end;
}
up_read(&mm->mmap_sem);
+ if (result < 0)
+ break;
}
- return 0;
+ return result;
+}
+
+static void vvp_io_advance(const struct lu_env *env,
+ const struct cl_io_slice *ios,
+ size_t nob)
+{
+ struct vvp_io *vio = cl2vvp_io(env, ios);
+ struct cl_io *io = ios->cis_io;
+ struct cl_object *obj = ios->cis_io->ci_obj;
+
+ CLOBINVRNT(env, obj, vvp_object_invariant(obj));
+
+ if (!cl_is_normalio(env, io))
+ return;
+
+ iov_iter_reexpand(vio->vui_iter, vio->vui_tot_count -= nob);
+}
+
+static void vvp_io_update_iov(const struct lu_env *env,
+ struct vvp_io *vio, struct cl_io *io)
+{
+ size_t size = io->u.ci_rw.crw_count;
+
+ if (!cl_is_normalio(env, io) || !vio->vui_iter)
+ return;
+
+ iov_iter_truncate(vio->vui_iter, size);
}
static int vvp_io_rw_lock(const struct lu_env *env, struct cl_io *io,
enum cl_lock_mode mode, loff_t start, loff_t end)
{
- struct ccc_io *cio = ccc_env_io(env);
+ struct vvp_io *vio = vvp_env_io(env);
int result;
int ast_flags = 0;
LASSERT(io->ci_type == CIT_READ || io->ci_type == CIT_WRITE);
- ccc_io_update_iov(env, cio, io);
+ vvp_io_update_iov(env, vio, io);
if (io->u.ci_rw.crw_nonblock)
ast_flags |= CEF_NONBLOCK;
- result = vvp_mmap_locks(env, cio, io);
+ result = vvp_mmap_locks(env, vio, io);
if (result == 0)
- result = ccc_io_one_lock(env, io, ast_flags, mode, start, end);
+ result = vvp_io_one_lock(env, io, ast_flags, mode, start, end);
return result;
}
@@ -325,9 +529,11 @@
/*
* XXX LDLM_FL_CBPENDING
*/
- return ccc_io_one_lock_index
- (env, io, 0, vvp_mode_from_vma(vio->u.fault.ft_vma),
- io->u.ci_fault.ft_index, io->u.ci_fault.ft_index);
+ return vvp_io_one_lock_index(env,
+ io, 0,
+ vvp_mode_from_vma(vio->u.fault.ft_vma),
+ io->u.ci_fault.ft_index,
+ io->u.ci_fault.ft_index);
}
static int vvp_io_write_lock(const struct lu_env *env,
@@ -354,14 +560,13 @@
}
/**
- * Implementation of cl_io_operations::cio_lock() method for CIT_SETATTR io.
+ * Implementation of cl_io_operations::vio_lock() method for CIT_SETATTR io.
*
* Handles "lockless io" mode when extent locking is done by server.
*/
static int vvp_io_setattr_lock(const struct lu_env *env,
const struct cl_io_slice *ios)
{
- struct ccc_io *cio = ccc_env_io(env);
struct cl_io *io = ios->cis_io;
__u64 new_size;
__u32 enqflags = 0;
@@ -378,8 +583,8 @@
return 0;
new_size = 0;
}
- cio->u.setattr.cui_local_lock = SETATTR_EXTENT_LOCK;
- return ccc_io_one_lock(env, io, enqflags, CLM_WRITE,
+
+ return vvp_io_one_lock(env, io, enqflags, CLM_WRITE,
new_size, OBD_OBJECT_EOF);
}
@@ -413,7 +618,7 @@
{
struct cl_io *io = ios->cis_io;
struct cl_object *obj = io->ci_obj;
- struct cl_attr *attr = ccc_env_thread_attr(env);
+ struct cl_attr *attr = vvp_env_thread_attr(env);
int result;
unsigned valid = CAT_CTIME;
@@ -437,7 +642,7 @@
const struct cl_io_slice *ios)
{
struct cl_io *io = ios->cis_io;
- struct inode *inode = ccc_object_inode(io->ci_obj);
+ struct inode *inode = vvp_object_inode(io->ci_obj);
int result = 0;
inode_lock(inode);
@@ -453,7 +658,7 @@
const struct cl_io_slice *ios)
{
struct cl_io *io = ios->cis_io;
- struct inode *inode = ccc_object_inode(io->ci_obj);
+ struct inode *inode = vvp_object_inode(io->ci_obj);
if (cl_io_is_trunc(io))
/* Truncate in memory pages - they must be clean pages
@@ -474,27 +679,25 @@
const struct cl_io_slice *ios)
{
struct vvp_io *vio = cl2vvp_io(env, ios);
- struct ccc_io *cio = cl2ccc_io(env, ios);
struct cl_io *io = ios->cis_io;
struct cl_object *obj = io->ci_obj;
- struct inode *inode = ccc_object_inode(obj);
- struct ll_ra_read *bead = &vio->cui_bead;
- struct file *file = cio->cui_fd->fd_file;
+ struct inode *inode = vvp_object_inode(obj);
+ struct file *file = vio->vui_fd->fd_file;
int result;
loff_t pos = io->u.ci_rd.rd.crw_pos;
long cnt = io->u.ci_rd.rd.crw_count;
- long tot = cio->cui_tot_count;
+ long tot = vio->vui_tot_count;
int exceed = 0;
- CLOBINVRNT(env, obj, ccc_object_invariant(obj));
+ CLOBINVRNT(env, obj, vvp_object_invariant(obj));
CDEBUG(D_VFSTRACE, "read: -> [%lli, %lli)\n", pos, pos + cnt);
if (!can_populate_pages(env, io, inode))
return 0;
- result = ccc_prep_size(env, obj, io, pos, tot, &exceed);
+ result = vvp_prep_size(env, obj, io, pos, tot, &exceed);
if (result != 0)
return result;
else if (exceed != 0)
@@ -505,30 +708,27 @@
inode->i_ino, cnt, pos, i_size_read(inode));
/* turn off the kernel's read-ahead */
- cio->cui_fd->fd_file->f_ra.ra_pages = 0;
+ vio->vui_fd->fd_file->f_ra.ra_pages = 0;
/* initialize read-ahead window once per syscall */
- if (!vio->cui_ra_window_set) {
- vio->cui_ra_window_set = 1;
- bead->lrr_start = cl_index(obj, pos);
- /*
- * XXX: explicit PAGE_SIZE
- */
- bead->lrr_count = cl_index(obj, tot + PAGE_SIZE - 1);
- ll_ra_read_in(file, bead);
+ if (!vio->vui_ra_valid) {
+ vio->vui_ra_valid = true;
+ vio->vui_ra_start = cl_index(obj, pos);
+ vio->vui_ra_count = cl_index(obj, tot + PAGE_SIZE - 1);
+ ll_ras_enter(file);
}
/* BUG: 5972 */
file_accessed(file);
- switch (vio->cui_io_subtype) {
+ switch (vio->vui_io_subtype) {
case IO_NORMAL:
- LASSERT(cio->cui_iocb->ki_pos == pos);
- result = generic_file_read_iter(cio->cui_iocb, cio->cui_iter);
+ LASSERT(vio->vui_iocb->ki_pos == pos);
+ result = generic_file_read_iter(vio->vui_iocb, vio->vui_iter);
break;
case IO_SPLICE:
result = generic_file_splice_read(file, &pos,
- vio->u.splice.cui_pipe, cnt,
- vio->u.splice.cui_flags);
+ vio->u.splice.vui_pipe, cnt,
+ vio->u.splice.vui_flags);
/* LU-1109: do splice read stripe by stripe otherwise if it
* may make nfsd stuck if this read occupied all internal pipe
* buffers.
@@ -536,7 +736,7 @@
io->ci_continue = 0;
break;
default:
- CERROR("Wrong IO type %u\n", vio->cui_io_subtype);
+ CERROR("Wrong IO type %u\n", vio->vui_io_subtype);
LBUG();
}
@@ -546,30 +746,201 @@
io->ci_continue = 0;
io->ci_nob += result;
ll_rw_stats_tally(ll_i2sbi(inode), current->pid,
- cio->cui_fd, pos, result, READ);
+ vio->vui_fd, pos, result, READ);
result = 0;
}
return result;
}
-static void vvp_io_read_fini(const struct lu_env *env, const struct cl_io_slice *ios)
+static int vvp_io_commit_sync(const struct lu_env *env, struct cl_io *io,
+ struct cl_page_list *plist, int from, int to)
{
- struct vvp_io *vio = cl2vvp_io(env, ios);
- struct ccc_io *cio = cl2ccc_io(env, ios);
+ struct cl_2queue *queue = &io->ci_queue;
+ struct cl_page *page;
+ unsigned int bytes = 0;
+ int rc = 0;
- if (vio->cui_ra_window_set)
- ll_ra_read_ex(cio->cui_fd->fd_file, &vio->cui_bead);
+ if (plist->pl_nr == 0)
+ return 0;
- vvp_io_fini(env, ios);
+ if (from > 0 || to != PAGE_SIZE) {
+ page = cl_page_list_first(plist);
+ if (plist->pl_nr == 1) {
+ cl_page_clip(env, page, from, to);
+ } else {
+ if (from > 0)
+ cl_page_clip(env, page, from, PAGE_SIZE);
+ if (to != PAGE_SIZE) {
+ page = cl_page_list_last(plist);
+ cl_page_clip(env, page, 0, to);
+ }
+ }
+ }
+
+ cl_2queue_init(queue);
+ cl_page_list_splice(plist, &queue->c2_qin);
+ rc = cl_io_submit_sync(env, io, CRT_WRITE, queue, 0);
+
+ /* plist is not sorted any more */
+ cl_page_list_splice(&queue->c2_qin, plist);
+ cl_page_list_splice(&queue->c2_qout, plist);
+ cl_2queue_fini(env, queue);
+
+ if (rc == 0) {
+ /* calculate bytes */
+ bytes = plist->pl_nr << PAGE_SHIFT;
+ bytes -= from + PAGE_SIZE - to;
+
+ while (plist->pl_nr > 0) {
+ page = cl_page_list_first(plist);
+ cl_page_list_del(env, plist, page);
+
+ cl_page_clip(env, page, 0, PAGE_SIZE);
+
+ SetPageUptodate(cl_page_vmpage(page));
+ cl_page_disown(env, io, page);
+
+ /* held in ll_cl_init() */
+ lu_ref_del(&page->cp_reference, "cl_io", io);
+ cl_page_put(env, page);
+ }
+ }
+
+ return bytes > 0 ? bytes : rc;
+}
+
+static void write_commit_callback(const struct lu_env *env, struct cl_io *io,
+ struct cl_page *page)
+{
+ struct vvp_page *vpg;
+ struct page *vmpage = page->cp_vmpage;
+ struct cl_object *clob = cl_io_top(io)->ci_obj;
+
+ SetPageUptodate(vmpage);
+ set_page_dirty(vmpage);
+
+ vpg = cl2vvp_page(cl_object_page_slice(clob, page));
+ vvp_write_pending(cl2vvp(clob), vpg);
+
+ cl_page_disown(env, io, page);
+
+ /* held in ll_cl_init() */
+ lu_ref_del(&page->cp_reference, "cl_io", io);
+ cl_page_put(env, page);
+}
+
+/* make sure the page list is contiguous */
+static bool page_list_sanity_check(struct cl_object *obj,
+ struct cl_page_list *plist)
+{
+ struct cl_page *page;
+ pgoff_t index = CL_PAGE_EOF;
+
+ cl_page_list_for_each(page, plist) {
+ struct vvp_page *vpg = cl_object_page_slice(obj, page);
+
+ if (index == CL_PAGE_EOF) {
+ index = vvp_index(vpg);
+ continue;
+ }
+
+ ++index;
+ if (index == vvp_index(vpg))
+ continue;
+
+ return false;
+ }
+ return true;
+}
+
+/* Return how many bytes have queued or written */
+int vvp_io_write_commit(const struct lu_env *env, struct cl_io *io)
+{
+ struct cl_object *obj = io->ci_obj;
+ struct inode *inode = vvp_object_inode(obj);
+ struct vvp_io *vio = vvp_env_io(env);
+ struct cl_page_list *queue = &vio->u.write.vui_queue;
+ struct cl_page *page;
+ int rc = 0;
+ int bytes = 0;
+ unsigned int npages = vio->u.write.vui_queue.pl_nr;
+
+ if (npages == 0)
+ return 0;
+
+ CDEBUG(D_VFSTRACE, "commit async pages: %d, from %d, to %d\n",
+ npages, vio->u.write.vui_from, vio->u.write.vui_to);
+
+ LASSERT(page_list_sanity_check(obj, queue));
+
+ /* submit IO with async write */
+ rc = cl_io_commit_async(env, io, queue,
+ vio->u.write.vui_from, vio->u.write.vui_to,
+ write_commit_callback);
+ npages -= queue->pl_nr; /* already committed pages */
+ if (npages > 0) {
+ /* calculate how many bytes were written */
+ bytes = npages << PAGE_SHIFT;
+
+ /* first page */
+ bytes -= vio->u.write.vui_from;
+ if (queue->pl_nr == 0) /* last page */
+ bytes -= PAGE_SIZE - vio->u.write.vui_to;
+ LASSERTF(bytes > 0, "bytes = %d, pages = %d\n", bytes, npages);
+
+ vio->u.write.vui_written += bytes;
+
+ CDEBUG(D_VFSTRACE, "Committed %d pages %d bytes, tot: %ld\n",
+ npages, bytes, vio->u.write.vui_written);
+
+ /* the first page must have been written. */
+ vio->u.write.vui_from = 0;
+ }
+ LASSERT(page_list_sanity_check(obj, queue));
+ LASSERT(ergo(rc == 0, queue->pl_nr == 0));
+
+ /* out of quota, try sync write */
+ if (rc == -EDQUOT && !cl_io_is_mkwrite(io)) {
+ rc = vvp_io_commit_sync(env, io, queue,
+ vio->u.write.vui_from,
+ vio->u.write.vui_to);
+ if (rc > 0) {
+ vio->u.write.vui_written += rc;
+ rc = 0;
+ }
+ }
+
+ /* update inode size */
+ ll_merge_attr(env, inode);
+
+ /* Now the pages in queue were failed to commit, discard them
+ * unless they were dirtied before.
+ */
+ while (queue->pl_nr > 0) {
+ page = cl_page_list_first(queue);
+ cl_page_list_del(env, queue, page);
+
+ if (!PageDirty(cl_page_vmpage(page)))
+ cl_page_discard(env, io, page);
+
+ cl_page_disown(env, io, page);
+
+ /* held in ll_cl_init() */
+ lu_ref_del(&page->cp_reference, "cl_io", io);
+ cl_page_put(env, page);
+ }
+ cl_page_list_fini(env, queue);
+
+ return rc;
}
static int vvp_io_write_start(const struct lu_env *env,
const struct cl_io_slice *ios)
{
- struct ccc_io *cio = cl2ccc_io(env, ios);
+ struct vvp_io *vio = cl2vvp_io(env, ios);
struct cl_io *io = ios->cis_io;
struct cl_object *obj = io->ci_obj;
- struct inode *inode = ccc_object_inode(obj);
+ struct inode *inode = vvp_object_inode(obj);
ssize_t result = 0;
loff_t pos = io->u.ci_wr.wr.crw_pos;
size_t cnt = io->u.ci_wr.wr.crw_count;
@@ -582,25 +953,41 @@
* PARALLEL IO This has to be changed for parallel IO doing
* out-of-order writes.
*/
+ ll_merge_attr(env, inode);
pos = io->u.ci_wr.wr.crw_pos = i_size_read(inode);
- cio->cui_iocb->ki_pos = pos;
+ vio->vui_iocb->ki_pos = pos;
} else {
- LASSERT(cio->cui_iocb->ki_pos == pos);
+ LASSERT(vio->vui_iocb->ki_pos == pos);
}
CDEBUG(D_VFSTRACE, "write: [%lli, %lli)\n", pos, pos + (long long)cnt);
- if (!cio->cui_iter) /* from a temp io in ll_cl_init(). */
+ if (!vio->vui_iter) /* from a temp io in ll_cl_init(). */
result = 0;
else
- result = generic_file_write_iter(cio->cui_iocb, cio->cui_iter);
+ result = generic_file_write_iter(vio->vui_iocb, vio->vui_iter);
if (result > 0) {
+ result = vvp_io_write_commit(env, io);
+ if (vio->u.write.vui_written > 0) {
+ result = vio->u.write.vui_written;
+ io->ci_nob += result;
+
+ CDEBUG(D_VFSTRACE, "write: nob %zd, result: %zd\n",
+ io->ci_nob, result);
+ }
+ }
+ if (result > 0) {
+ struct ll_inode_info *lli = ll_i2info(inode);
+
+ spin_lock(&lli->lli_lock);
+ lli->lli_flags |= LLIF_DATA_MODIFIED;
+ spin_unlock(&lli->lli_lock);
+
if (result < cnt)
io->ci_continue = 0;
- io->ci_nob += result;
ll_rw_stats_tally(ll_i2sbi(inode), current->pid,
- cio->cui_fd, pos, result, WRITE);
+ vio->vui_fd, pos, result, WRITE);
result = 0;
}
return result;
@@ -608,10 +995,10 @@
static int vvp_io_kernel_fault(struct vvp_fault_io *cfio)
{
- struct vm_fault *vmf = cfio->fault.ft_vmf;
+ struct vm_fault *vmf = cfio->ft_vmf;
- cfio->fault.ft_flags = filemap_fault(cfio->ft_vma, vmf);
- cfio->fault.ft_flags_valid = 1;
+ cfio->ft_flags = filemap_fault(cfio->ft_vma, vmf);
+ cfio->ft_flags_valid = 1;
if (vmf->page) {
CDEBUG(D_PAGE,
@@ -619,39 +1006,51 @@
vmf->page, vmf->page->mapping, vmf->page->index,
(long)vmf->page->flags, page_count(vmf->page),
page_private(vmf->page), vmf->virtual_address);
- if (unlikely(!(cfio->fault.ft_flags & VM_FAULT_LOCKED))) {
+ if (unlikely(!(cfio->ft_flags & VM_FAULT_LOCKED))) {
lock_page(vmf->page);
- cfio->fault.ft_flags |= VM_FAULT_LOCKED;
+ cfio->ft_flags |= VM_FAULT_LOCKED;
}
cfio->ft_vmpage = vmf->page;
return 0;
}
- if (cfio->fault.ft_flags & (VM_FAULT_SIGBUS | VM_FAULT_SIGSEGV)) {
+ if (cfio->ft_flags & (VM_FAULT_SIGBUS | VM_FAULT_SIGSEGV)) {
CDEBUG(D_PAGE, "got addr %p - SIGBUS\n", vmf->virtual_address);
return -EFAULT;
}
- if (cfio->fault.ft_flags & VM_FAULT_OOM) {
+ if (cfio->ft_flags & VM_FAULT_OOM) {
CDEBUG(D_PAGE, "got addr %p - OOM\n", vmf->virtual_address);
return -ENOMEM;
}
- if (cfio->fault.ft_flags & VM_FAULT_RETRY)
+ if (cfio->ft_flags & VM_FAULT_RETRY)
return -EAGAIN;
- CERROR("Unknown error in page fault %d!\n", cfio->fault.ft_flags);
+ CERROR("Unknown error in page fault %d!\n", cfio->ft_flags);
return -EINVAL;
}
+static void mkwrite_commit_callback(const struct lu_env *env, struct cl_io *io,
+ struct cl_page *page)
+{
+ struct vvp_page *vpg;
+ struct cl_object *clob = cl_io_top(io)->ci_obj;
+
+ set_page_dirty(page->cp_vmpage);
+
+ vpg = cl2vvp_page(cl_object_page_slice(clob, page));
+ vvp_write_pending(cl2vvp(clob), vpg);
+}
+
static int vvp_io_fault_start(const struct lu_env *env,
const struct cl_io_slice *ios)
{
struct vvp_io *vio = cl2vvp_io(env, ios);
struct cl_io *io = ios->cis_io;
struct cl_object *obj = io->ci_obj;
- struct inode *inode = ccc_object_inode(obj);
+ struct inode *inode = vvp_object_inode(obj);
struct cl_fault_io *fio = &io->u.ci_fault;
struct vvp_fault_io *cfio = &vio->u.fault;
loff_t offset;
@@ -659,7 +1058,7 @@
struct page *vmpage = NULL;
struct cl_page *page;
loff_t size;
- pgoff_t last; /* last page in a file data region */
+ pgoff_t last_index;
if (fio->ft_executable &&
inode->i_mtime.tv_sec != vio->u.fault.ft_mtime)
@@ -670,7 +1069,7 @@
/* offset of the last byte on the page */
offset = cl_offset(obj, fio->ft_index + 1) - 1;
LASSERT(cl_index(obj, offset) == fio->ft_index);
- result = ccc_prep_size(env, obj, io, 0, offset + 1, NULL);
+ result = vvp_prep_size(env, obj, io, 0, offset + 1, NULL);
if (result != 0)
return result;
@@ -705,15 +1104,15 @@
goto out;
}
+ last_index = cl_index(obj, size - 1);
+
if (fio->ft_mkwrite) {
- pgoff_t last_index;
/*
* Capture the size while holding the lli_trunc_sem from above
* we want to make sure that we complete the mkwrite action
* while holding this lock. We need to make sure that we are
* not past the end of the file.
*/
- last_index = cl_index(obj, size - 1);
if (last_index < fio->ft_index) {
CDEBUG(D_PAGE,
"llite: mkwrite and truncate race happened: %p: 0x%lx 0x%lx\n",
@@ -745,25 +1144,32 @@
*/
if (fio->ft_mkwrite) {
wait_on_page_writeback(vmpage);
- if (set_page_dirty(vmpage)) {
- struct ccc_page *cp;
+ if (!PageDirty(vmpage)) {
+ struct cl_page_list *plist = &io->ci_queue.c2_qin;
+ struct vvp_page *vpg = cl_object_page_slice(obj, page);
+ int to = PAGE_SIZE;
/* vvp_page_assume() calls wait_on_page_writeback(). */
cl_page_assume(env, io, page);
- cp = cl2ccc_page(cl_page_at(page, &vvp_device_type));
- vvp_write_pending(cl2ccc(obj), cp);
+ cl_page_list_init(plist);
+ cl_page_list_add(plist, page);
+
+ /* size fixup */
+ if (last_index == vvp_index(vpg))
+ to = size & ~PAGE_MASK;
/* Do not set Dirty bit here so that in case IO is
* started before the page is really made dirty, we
* still have chance to detect it.
*/
- result = cl_page_cache_add(env, io, page, CRT_WRITE);
+ result = cl_io_commit_async(env, io, plist, 0, to,
+ mkwrite_commit_callback);
LASSERT(cl_page_is_owned(page, io));
+ cl_page_list_fini(env, plist);
vmpage = NULL;
if (result < 0) {
- cl_page_unmap(env, io, page);
cl_page_discard(env, io, page);
cl_page_disown(env, io, page);
@@ -773,20 +1179,20 @@
if (result == -EDQUOT)
result = -ENOSPC;
goto out;
- } else
+ } else {
cl_page_disown(env, io, page);
+ }
}
}
- last = cl_index(obj, size - 1);
/*
* The ft_index is only used in the case of
* a mkwrite action. We need to check
* our assertions are correct, since
* we should have caught this above
*/
- LASSERT(!fio->ft_mkwrite || fio->ft_index <= last);
- if (fio->ft_index == last)
+ LASSERT(!fio->ft_mkwrite || fio->ft_index <= last_index);
+ if (fio->ft_index == last_index)
/*
* Last page is mapped partially.
*/
@@ -801,7 +1207,9 @@
/* return unlocked vmpage to avoid deadlocking */
if (vmpage)
unlock_page(vmpage);
- cfio->fault.ft_flags &= ~VM_FAULT_LOCKED;
+
+ cfio->ft_flags &= ~VM_FAULT_LOCKED;
+
return result;
}
@@ -820,293 +1228,58 @@
const struct cl_page_slice *slice)
{
struct cl_io *io = ios->cis_io;
- struct cl_object *obj = slice->cpl_obj;
- struct ccc_page *cp = cl2ccc_page(slice);
+ struct vvp_page *vpg = cl2vvp_page(slice);
struct cl_page *page = slice->cpl_page;
- struct inode *inode = ccc_object_inode(obj);
+ struct inode *inode = vvp_object_inode(slice->cpl_obj);
struct ll_sb_info *sbi = ll_i2sbi(inode);
- struct ll_file_data *fd = cl2ccc_io(env, ios)->cui_fd;
+ struct ll_file_data *fd = cl2vvp_io(env, ios)->vui_fd;
struct ll_readahead_state *ras = &fd->fd_ras;
- struct page *vmpage = cp->cpg_page;
struct cl_2queue *queue = &io->ci_queue;
- int rc;
-
- CLOBINVRNT(env, obj, ccc_object_invariant(obj));
- LASSERT(slice->cpl_obj == obj);
if (sbi->ll_ra_info.ra_max_pages_per_file &&
sbi->ll_ra_info.ra_max_pages)
- ras_update(sbi, inode, ras, page->cp_index,
- cp->cpg_defer_uptodate);
+ ras_update(sbi, inode, ras, vvp_index(vpg),
+ vpg->vpg_defer_uptodate);
- /* Sanity check whether the page is protected by a lock. */
- rc = cl_page_is_under_lock(env, io, page);
- if (rc != -EBUSY) {
- CL_PAGE_HEADER(D_WARNING, env, page, "%s: %d\n",
- rc == -ENODATA ? "without a lock" :
- "match failed", rc);
- if (rc != -ENODATA)
- return rc;
- }
-
- if (cp->cpg_defer_uptodate) {
- cp->cpg_ra_used = 1;
+ if (vpg->vpg_defer_uptodate) {
+ vpg->vpg_ra_used = 1;
cl_page_export(env, page, 1);
}
/*
* Add page into the queue even when it is marked uptodate above.
* this will unlock it automatically as part of cl_page_list_disown().
*/
+
cl_page_list_add(&queue->c2_qin, page);
if (sbi->ll_ra_info.ra_max_pages_per_file &&
sbi->ll_ra_info.ra_max_pages)
- ll_readahead(env, io, ras,
- vmpage->mapping, &queue->c2_qin, fd->fd_flags);
+ ll_readahead(env, io, &queue->c2_qin, ras,
+ vpg->vpg_defer_uptodate);
return 0;
}
-static int vvp_page_sync_io(const struct lu_env *env, struct cl_io *io,
- struct cl_page *page, struct ccc_page *cp,
- enum cl_req_type crt)
+void vvp_io_end(const struct lu_env *env, const struct cl_io_slice *ios)
{
- struct cl_2queue *queue;
- int result;
-
- LASSERT(io->ci_type == CIT_READ || io->ci_type == CIT_WRITE);
-
- queue = &io->ci_queue;
- cl_2queue_init_page(queue, page);
-
- result = cl_io_submit_sync(env, io, crt, queue, 0);
- LASSERT(cl_page_is_owned(page, io));
-
- if (crt == CRT_READ)
- /*
- * in CRT_WRITE case page is left locked even in case of
- * error.
- */
- cl_page_list_disown(env, io, &queue->c2_qin);
- cl_2queue_fini(env, queue);
-
- return result;
-}
-
-/**
- * Prepare partially written-to page for a write.
- */
-static int vvp_io_prepare_partial(const struct lu_env *env, struct cl_io *io,
- struct cl_object *obj, struct cl_page *pg,
- struct ccc_page *cp,
- unsigned from, unsigned to)
-{
- struct cl_attr *attr = ccc_env_thread_attr(env);
- loff_t offset = cl_offset(obj, pg->cp_index);
- int result;
-
- cl_object_attr_lock(obj);
- result = cl_object_attr_get(env, obj, attr);
- cl_object_attr_unlock(obj);
- if (result == 0) {
- /*
- * If are writing to a new page, no need to read old data.
- * The extent locking will have updated the KMS, and for our
- * purposes here we can treat it like i_size.
- */
- if (attr->cat_kms <= offset) {
- char *kaddr = kmap_atomic(cp->cpg_page);
-
- memset(kaddr, 0, cl_page_size(obj));
- kunmap_atomic(kaddr);
- } else if (cp->cpg_defer_uptodate)
- cp->cpg_ra_used = 1;
- else
- result = vvp_page_sync_io(env, io, pg, cp, CRT_READ);
- /*
- * In older implementations, obdo_refresh_inode is called here
- * to update the inode because the write might modify the
- * object info at OST. However, this has been proven useless,
- * since LVB functions will be called when user space program
- * tries to retrieve inode attribute. Also, see bug 15909 for
- * details. -jay
- */
- if (result == 0)
- cl_page_export(env, pg, 1);
- }
- return result;
-}
-
-static int vvp_io_prepare_write(const struct lu_env *env,
- const struct cl_io_slice *ios,
- const struct cl_page_slice *slice,
- unsigned from, unsigned to)
-{
- struct cl_object *obj = slice->cpl_obj;
- struct ccc_page *cp = cl2ccc_page(slice);
- struct cl_page *pg = slice->cpl_page;
- struct page *vmpage = cp->cpg_page;
-
- int result;
-
- LINVRNT(cl_page_is_vmlocked(env, pg));
- LASSERT(vmpage->mapping->host == ccc_object_inode(obj));
-
- result = 0;
-
- CL_PAGE_HEADER(D_PAGE, env, pg, "preparing: [%d, %d]\n", from, to);
- if (!PageUptodate(vmpage)) {
- /*
- * We're completely overwriting an existing page, so _don't_
- * set it up to date until commit_write
- */
- if (from == 0 && to == PAGE_SIZE) {
- CL_PAGE_HEADER(D_PAGE, env, pg, "full page write\n");
- POISON_PAGE(page, 0x11);
- } else
- result = vvp_io_prepare_partial(env, ios->cis_io, obj,
- pg, cp, from, to);
- } else
- CL_PAGE_HEADER(D_PAGE, env, pg, "uptodate\n");
- return result;
-}
-
-static int vvp_io_commit_write(const struct lu_env *env,
- const struct cl_io_slice *ios,
- const struct cl_page_slice *slice,
- unsigned from, unsigned to)
-{
- struct cl_object *obj = slice->cpl_obj;
- struct cl_io *io = ios->cis_io;
- struct ccc_page *cp = cl2ccc_page(slice);
- struct cl_page *pg = slice->cpl_page;
- struct inode *inode = ccc_object_inode(obj);
- struct ll_sb_info *sbi = ll_i2sbi(inode);
- struct ll_inode_info *lli = ll_i2info(inode);
- struct page *vmpage = cp->cpg_page;
-
- int result;
- int tallyop;
- loff_t size;
-
- LINVRNT(cl_page_is_vmlocked(env, pg));
- LASSERT(vmpage->mapping->host == inode);
-
- LU_OBJECT_HEADER(D_INODE, env, &obj->co_lu, "committing page write\n");
- CL_PAGE_HEADER(D_PAGE, env, pg, "committing: [%d, %d]\n", from, to);
-
- /*
- * queue a write for some time in the future the first time we
- * dirty the page.
- *
- * This is different from what other file systems do: they usually
- * just mark page (and some of its buffers) dirty and rely on
- * balance_dirty_pages() to start a write-back. Lustre wants write-back
- * to be started earlier for the following reasons:
- *
- * (1) with a large number of clients we need to limit the amount
- * of cached data on the clients a lot;
- *
- * (2) large compute jobs generally want compute-only then io-only
- * and the IO should complete as quickly as possible;
- *
- * (3) IO is batched up to the RPC size and is async until the
- * client max cache is hit
- * (/sys/fs/lustre/osc/OSC.../max_dirty_mb)
- *
- */
- if (!PageDirty(vmpage)) {
- tallyop = LPROC_LL_DIRTY_MISSES;
- result = cl_page_cache_add(env, io, pg, CRT_WRITE);
- if (result == 0) {
- /* page was added into cache successfully. */
- set_page_dirty(vmpage);
- vvp_write_pending(cl2ccc(obj), cp);
- } else if (result == -EDQUOT) {
- pgoff_t last_index = i_size_read(inode) >> PAGE_SHIFT;
- bool need_clip = true;
-
- /*
- * Client ran out of disk space grant. Possible
- * strategies are:
- *
- * (a) do a sync write, renewing grant;
- *
- * (b) stop writing on this stripe, switch to the
- * next one.
- *
- * (b) is a part of "parallel io" design that is the
- * ultimate goal. (a) is what "old" client did, and
- * what the new code continues to do for the time
- * being.
- */
- if (last_index > pg->cp_index) {
- to = PAGE_SIZE;
- need_clip = false;
- } else if (last_index == pg->cp_index) {
- int size_to = i_size_read(inode) & ~CFS_PAGE_MASK;
-
- if (to < size_to)
- to = size_to;
- }
- if (need_clip)
- cl_page_clip(env, pg, 0, to);
- result = vvp_page_sync_io(env, io, pg, cp, CRT_WRITE);
- if (result)
- CERROR("Write page %lu of inode %p failed %d\n",
- pg->cp_index, inode, result);
- }
- } else {
- tallyop = LPROC_LL_DIRTY_HITS;
- result = 0;
- }
- ll_stats_ops_tally(sbi, tallyop, 1);
-
- /* Inode should be marked DIRTY even if no new page was marked DIRTY
- * because page could have been not flushed between 2 modifications.
- * It is important the file is marked DIRTY as soon as the I/O is done
- * Indeed, when cache is flushed, file could be already closed and it
- * is too late to warn the MDT.
- * It is acceptable that file is marked DIRTY even if I/O is dropped
- * for some reasons before being flushed to OST.
- */
- if (result == 0) {
- spin_lock(&lli->lli_lock);
- lli->lli_flags |= LLIF_DATA_MODIFIED;
- spin_unlock(&lli->lli_lock);
- }
-
- size = cl_offset(obj, pg->cp_index) + to;
-
- ll_inode_size_lock(inode);
- if (result == 0) {
- if (size > i_size_read(inode)) {
- cl_isize_write_nolock(inode, size);
- CDEBUG(D_VFSTRACE, DFID" updating i_size %lu\n",
- PFID(lu_object_fid(&obj->co_lu)),
- (unsigned long)size);
- }
- cl_page_export(env, pg, 1);
- } else {
- if (size > i_size_read(inode))
- cl_page_discard(env, io, pg);
- }
- ll_inode_size_unlock(inode);
- return result;
+ CLOBINVRNT(env, ios->cis_io->ci_obj,
+ vvp_object_invariant(ios->cis_io->ci_obj));
}
static const struct cl_io_operations vvp_io_ops = {
.op = {
[CIT_READ] = {
- .cio_fini = vvp_io_read_fini,
+ .cio_fini = vvp_io_fini,
.cio_lock = vvp_io_read_lock,
.cio_start = vvp_io_read_start,
- .cio_advance = ccc_io_advance
+ .cio_advance = vvp_io_advance,
},
[CIT_WRITE] = {
.cio_fini = vvp_io_fini,
+ .cio_iter_init = vvp_io_write_iter_init,
+ .cio_iter_fini = vvp_io_write_iter_fini,
.cio_lock = vvp_io_write_lock,
.cio_start = vvp_io_write_start,
- .cio_advance = ccc_io_advance
+ .cio_advance = vvp_io_advance,
},
[CIT_SETATTR] = {
.cio_fini = vvp_io_setattr_fini,
@@ -1120,7 +1293,7 @@
.cio_iter_init = vvp_io_fault_iter_init,
.cio_lock = vvp_io_fault_lock,
.cio_start = vvp_io_fault_start,
- .cio_end = ccc_io_end
+ .cio_end = vvp_io_end,
},
[CIT_FSYNC] = {
.cio_start = vvp_io_fsync_start,
@@ -1131,29 +1304,26 @@
}
},
.cio_read_page = vvp_io_read_page,
- .cio_prepare_write = vvp_io_prepare_write,
- .cio_commit_write = vvp_io_commit_write
};
int vvp_io_init(const struct lu_env *env, struct cl_object *obj,
struct cl_io *io)
{
struct vvp_io *vio = vvp_env_io(env);
- struct ccc_io *cio = ccc_env_io(env);
- struct inode *inode = ccc_object_inode(obj);
+ struct inode *inode = vvp_object_inode(obj);
int result;
- CLOBINVRNT(env, obj, ccc_object_invariant(obj));
+ CLOBINVRNT(env, obj, vvp_object_invariant(obj));
CDEBUG(D_VFSTRACE, DFID
" ignore/verify layout %d/%d, layout version %d restore needed %d\n",
PFID(lu_object_fid(&obj->co_lu)),
io->ci_ignore_layout, io->ci_verify_layout,
- cio->cui_layout_gen, io->ci_restore_needed);
+ vio->vui_layout_gen, io->ci_restore_needed);
- CL_IO_SLICE_CLEAN(cio, cui_cl);
- cl_io_slice_add(io, &cio->cui_cl, obj, &vvp_io_ops);
- vio->cui_ra_window_set = 0;
+ CL_IO_SLICE_CLEAN(vio, vui_cl);
+ cl_io_slice_add(io, &vio->vui_cl, obj, &vvp_io_ops);
+ vio->vui_ra_valid = false;
result = 0;
if (io->ci_type == CIT_READ || io->ci_type == CIT_WRITE) {
size_t count;
@@ -1166,7 +1336,7 @@
if (count == 0)
result = 1;
else
- cio->cui_tot_count = count;
+ vio->vui_tot_count = count;
/* for read/write, we store the jobid in the inode, and
* it'll be fetched by osc when building RPC.
@@ -1192,7 +1362,7 @@
* because it might not grant layout lock in IT_OPEN.
*/
if (result == 0 && !io->ci_ignore_layout) {
- result = ll_layout_refresh(inode, &cio->cui_layout_gen);
+ result = ll_layout_refresh(inode, &vio->vui_layout_gen);
if (result == -ENOENT)
/* If the inode on MDS has been removed, but the objects
* on OSTs haven't been destroyed (async unlink), layout
@@ -1208,11 +1378,3 @@
return result;
}
-
-static struct vvp_io *cl2vvp_io(const struct lu_env *env,
- const struct cl_io_slice *slice)
-{
- /* Calling just for assertion */
- cl2ccc_io(env, slice);
- return vvp_env_io(env);
-}
diff --git a/drivers/staging/lustre/lustre/llite/vvp_lock.c b/drivers/staging/lustre/lustre/llite/vvp_lock.c
index ff09480..f5bd6c2 100644
--- a/drivers/staging/lustre/lustre/llite/vvp_lock.c
+++ b/drivers/staging/lustre/lustre/llite/vvp_lock.c
@@ -40,7 +40,7 @@
#define DEBUG_SUBSYSTEM S_LLITE
-#include "../include/obd.h"
+#include "../include/obd_support.h"
#include "../include/lustre_lite.h"
#include "vvp_internal.h"
@@ -51,36 +51,41 @@
*
*/
-/**
- * Estimates lock value for the purpose of managing the lock cache during
- * memory shortages.
- *
- * Locks for memory mapped files are almost infinitely precious, others are
- * junk. "Mapped locks" are heavy, but not infinitely heavy, so that they are
- * ordered within themselves by weights assigned from other layers.
- */
-static unsigned long vvp_lock_weigh(const struct lu_env *env,
- const struct cl_lock_slice *slice)
+static void vvp_lock_fini(const struct lu_env *env, struct cl_lock_slice *slice)
{
- struct ccc_object *cob = cl2ccc(slice->cls_obj);
+ struct vvp_lock *vlk = cl2vvp_lock(slice);
- return atomic_read(&cob->cob_mmap_cnt) > 0 ? ~0UL >> 2 : 0;
+ kmem_cache_free(vvp_lock_kmem, vlk);
+}
+
+static int vvp_lock_enqueue(const struct lu_env *env,
+ const struct cl_lock_slice *slice,
+ struct cl_io *unused, struct cl_sync_io *anchor)
+{
+ CLOBINVRNT(env, slice->cls_obj, vvp_object_invariant(slice->cls_obj));
+
+ return 0;
}
static const struct cl_lock_operations vvp_lock_ops = {
- .clo_delete = ccc_lock_delete,
- .clo_fini = ccc_lock_fini,
- .clo_enqueue = ccc_lock_enqueue,
- .clo_wait = ccc_lock_wait,
- .clo_use = ccc_lock_use,
- .clo_unuse = ccc_lock_unuse,
- .clo_fits_into = ccc_lock_fits_into,
- .clo_state = ccc_lock_state,
- .clo_weigh = vvp_lock_weigh
+ .clo_fini = vvp_lock_fini,
+ .clo_enqueue = vvp_lock_enqueue,
};
int vvp_lock_init(const struct lu_env *env, struct cl_object *obj,
- struct cl_lock *lock, const struct cl_io *io)
+ struct cl_lock *lock, const struct cl_io *unused)
{
- return ccc_lock_init(env, obj, lock, io, &vvp_lock_ops);
+ struct vvp_lock *vlk;
+ int result;
+
+ CLOBINVRNT(env, obj, vvp_object_invariant(obj));
+
+ vlk = kmem_cache_zalloc(vvp_lock_kmem, GFP_NOFS);
+ if (vlk) {
+ cl_lock_slice_add(lock, &vlk->vlk_cl, obj, &vvp_lock_ops);
+ result = 0;
+ } else {
+ result = -ENOMEM;
+ }
+ return result;
}
diff --git a/drivers/staging/lustre/lustre/llite/vvp_object.c b/drivers/staging/lustre/lustre/llite/vvp_object.c
index 03c887d..18c9df7 100644
--- a/drivers/staging/lustre/lustre/llite/vvp_object.c
+++ b/drivers/staging/lustre/lustre/llite/vvp_object.c
@@ -45,6 +45,7 @@
#include "../include/obd.h"
#include "../include/lustre_lite.h"
+#include "llite_internal.h"
#include "vvp_internal.h"
/*****************************************************************************
@@ -53,16 +54,25 @@
*
*/
+int vvp_object_invariant(const struct cl_object *obj)
+{
+ struct inode *inode = vvp_object_inode(obj);
+ struct ll_inode_info *lli = ll_i2info(inode);
+
+ return (S_ISREG(inode->i_mode) || inode->i_mode == 0) &&
+ lli->lli_clob == obj;
+}
+
static int vvp_object_print(const struct lu_env *env, void *cookie,
lu_printer_t p, const struct lu_object *o)
{
- struct ccc_object *obj = lu2ccc(o);
- struct inode *inode = obj->cob_inode;
+ struct vvp_object *obj = lu2vvp(o);
+ struct inode *inode = obj->vob_inode;
struct ll_inode_info *lli;
(*p)(env, cookie, "(%s %d %d) inode: %p ",
- list_empty(&obj->cob_pending_list) ? "-" : "+",
- obj->cob_transient_pages, atomic_read(&obj->cob_mmap_cnt),
+ list_empty(&obj->vob_pending_list) ? "-" : "+",
+ obj->vob_transient_pages, atomic_read(&obj->vob_mmap_cnt),
inode);
if (inode) {
lli = ll_i2info(inode);
@@ -77,7 +87,7 @@
static int vvp_attr_get(const struct lu_env *env, struct cl_object *obj,
struct cl_attr *attr)
{
- struct inode *inode = ccc_object_inode(obj);
+ struct inode *inode = vvp_object_inode(obj);
/*
* lov overwrites most of these fields in
@@ -99,7 +109,7 @@
static int vvp_attr_set(const struct lu_env *env, struct cl_object *obj,
const struct cl_attr *attr, unsigned valid)
{
- struct inode *inode = ccc_object_inode(obj);
+ struct inode *inode = vvp_object_inode(obj);
if (valid & CAT_UID)
inode->i_uid = make_kuid(&init_user_ns, attr->cat_uid);
@@ -112,7 +122,7 @@
if (valid & CAT_CTIME)
inode->i_ctime.tv_sec = attr->cat_ctime;
if (0 && valid & CAT_SIZE)
- cl_isize_write_nolock(inode, attr->cat_size);
+ i_size_write(inode, attr->cat_size);
/* not currently necessary */
if (0 && valid & (CAT_UID|CAT_GID|CAT_SIZE))
mark_inode_dirty(inode);
@@ -165,6 +175,40 @@
return 0;
}
+static int vvp_prune(const struct lu_env *env, struct cl_object *obj)
+{
+ struct inode *inode = vvp_object_inode(obj);
+ int rc;
+
+ rc = cl_sync_file_range(inode, 0, OBD_OBJECT_EOF, CL_FSYNC_LOCAL, 1);
+ if (rc < 0) {
+ CDEBUG(D_VFSTRACE, DFID ": writeback failed: %d\n",
+ PFID(lu_object_fid(&obj->co_lu)), rc);
+ return rc;
+ }
+
+ truncate_inode_pages(inode->i_mapping, 0);
+ return 0;
+}
+
+static int vvp_object_glimpse(const struct lu_env *env,
+ const struct cl_object *obj, struct ost_lvb *lvb)
+{
+ struct inode *inode = vvp_object_inode(obj);
+
+ lvb->lvb_mtime = LTIME_S(inode->i_mtime);
+ lvb->lvb_atime = LTIME_S(inode->i_atime);
+ lvb->lvb_ctime = LTIME_S(inode->i_ctime);
+ /*
+ * LU-417: Add dirty pages block count lest i_blocks reports 0, some
+ * "cp" or "tar" on remote node may think it's a completely sparse file
+ * and skip it.
+ */
+ if (lvb->lvb_size > 0 && lvb->lvb_blocks == 0)
+ lvb->lvb_blocks = dirty_cnt(inode);
+ return 0;
+}
+
static const struct cl_object_operations vvp_ops = {
.coo_page_init = vvp_page_init,
.coo_lock_init = vvp_lock_init,
@@ -172,29 +216,94 @@
.coo_attr_get = vvp_attr_get,
.coo_attr_set = vvp_attr_set,
.coo_conf_set = vvp_conf_set,
- .coo_glimpse = ccc_object_glimpse
+ .coo_prune = vvp_prune,
+ .coo_glimpse = vvp_object_glimpse
};
+static int vvp_object_init0(const struct lu_env *env,
+ struct vvp_object *vob,
+ const struct cl_object_conf *conf)
+{
+ vob->vob_inode = conf->coc_inode;
+ vob->vob_transient_pages = 0;
+ cl_object_page_init(&vob->vob_cl, sizeof(struct vvp_page));
+ return 0;
+}
+
+static int vvp_object_init(const struct lu_env *env, struct lu_object *obj,
+ const struct lu_object_conf *conf)
+{
+ struct vvp_device *dev = lu2vvp_dev(obj->lo_dev);
+ struct vvp_object *vob = lu2vvp(obj);
+ struct lu_object *below;
+ struct lu_device *under;
+ int result;
+
+ under = &dev->vdv_next->cd_lu_dev;
+ below = under->ld_ops->ldo_object_alloc(env, obj->lo_header, under);
+ if (below) {
+ const struct cl_object_conf *cconf;
+
+ cconf = lu2cl_conf(conf);
+ INIT_LIST_HEAD(&vob->vob_pending_list);
+ lu_object_add(obj, below);
+ result = vvp_object_init0(env, vob, cconf);
+ } else {
+ result = -ENOMEM;
+ }
+
+ return result;
+}
+
+static void vvp_object_free(const struct lu_env *env, struct lu_object *obj)
+{
+ struct vvp_object *vob = lu2vvp(obj);
+
+ lu_object_fini(obj);
+ lu_object_header_fini(obj->lo_header);
+ kmem_cache_free(vvp_object_kmem, vob);
+}
+
static const struct lu_object_operations vvp_lu_obj_ops = {
- .loo_object_init = ccc_object_init,
- .loo_object_free = ccc_object_free,
- .loo_object_print = vvp_object_print
+ .loo_object_init = vvp_object_init,
+ .loo_object_free = vvp_object_free,
+ .loo_object_print = vvp_object_print,
};
-struct ccc_object *cl_inode2ccc(struct inode *inode)
+struct vvp_object *cl_inode2vvp(struct inode *inode)
{
- struct cl_inode_info *lli = cl_i2info(inode);
+ struct ll_inode_info *lli = ll_i2info(inode);
struct cl_object *obj = lli->lli_clob;
struct lu_object *lu;
lu = lu_object_locate(obj->co_lu.lo_header, &vvp_device_type);
LASSERT(lu);
- return lu2ccc(lu);
+ return lu2vvp(lu);
}
struct lu_object *vvp_object_alloc(const struct lu_env *env,
- const struct lu_object_header *hdr,
+ const struct lu_object_header *unused,
struct lu_device *dev)
{
- return ccc_object_alloc(env, hdr, dev, &vvp_ops, &vvp_lu_obj_ops);
+ struct vvp_object *vob;
+ struct lu_object *obj;
+
+ vob = kmem_cache_zalloc(vvp_object_kmem, GFP_NOFS);
+ if (vob) {
+ struct cl_object_header *hdr;
+
+ obj = &vob->vob_cl.co_lu;
+ hdr = &vob->vob_header;
+ cl_object_header_init(hdr);
+ hdr->coh_page_bufsize = cfs_size_round(sizeof(struct cl_page));
+
+ lu_object_init(obj, &hdr->coh_lu, dev);
+ lu_object_add_top(&hdr->coh_lu, obj);
+
+ vob->vob_cl.co_ops = &vvp_ops;
+ obj->lo_ops = &vvp_lu_obj_ops;
+ } else {
+ obj = NULL;
+ }
+ return obj;
}
diff --git a/drivers/staging/lustre/lustre/llite/vvp_page.c b/drivers/staging/lustre/lustre/llite/vvp_page.c
index 33ca3eb..6cd2af7 100644
--- a/drivers/staging/lustre/lustre/llite/vvp_page.c
+++ b/drivers/staging/lustre/lustre/llite/vvp_page.c
@@ -41,9 +41,16 @@
#define DEBUG_SUBSYSTEM S_LLITE
-#include "../include/obd.h"
+#include <linux/atomic.h>
+#include <linux/bitops.h>
+#include <linux/mm.h>
+#include <linux/mutex.h>
+#include <linux/page-flags.h>
+#include <linux/pagemap.h>
+
#include "../include/lustre_lite.h"
+#include "llite_internal.h"
#include "vvp_internal.h"
/*****************************************************************************
@@ -52,9 +59,9 @@
*
*/
-static void vvp_page_fini_common(struct ccc_page *cp)
+static void vvp_page_fini_common(struct vvp_page *vpg)
{
- struct page *vmpage = cp->cpg_page;
+ struct page *vmpage = vpg->vpg_page;
LASSERT(vmpage);
put_page(vmpage);
@@ -63,23 +70,23 @@
static void vvp_page_fini(const struct lu_env *env,
struct cl_page_slice *slice)
{
- struct ccc_page *cp = cl2ccc_page(slice);
- struct page *vmpage = cp->cpg_page;
+ struct vvp_page *vpg = cl2vvp_page(slice);
+ struct page *vmpage = vpg->vpg_page;
/*
* vmpage->private was already cleared when page was moved into
* VPG_FREEING state.
*/
LASSERT((struct cl_page *)vmpage->private != slice->cpl_page);
- vvp_page_fini_common(cp);
+ vvp_page_fini_common(vpg);
}
static int vvp_page_own(const struct lu_env *env,
const struct cl_page_slice *slice, struct cl_io *io,
int nonblock)
{
- struct ccc_page *vpg = cl2ccc_page(slice);
- struct page *vmpage = vpg->cpg_page;
+ struct vvp_page *vpg = cl2vvp_page(slice);
+ struct page *vmpage = vpg->vpg_page;
LASSERT(vmpage);
if (nonblock) {
@@ -96,6 +103,7 @@
lock_page(vmpage);
wait_on_page_writeback(vmpage);
+
return 0;
}
@@ -136,41 +144,15 @@
struct cl_io *unused)
{
struct page *vmpage = cl2vm_page(slice);
- struct address_space *mapping;
- struct ccc_page *cpg = cl2ccc_page(slice);
+ struct vvp_page *vpg = cl2vvp_page(slice);
LASSERT(vmpage);
LASSERT(PageLocked(vmpage));
- mapping = vmpage->mapping;
+ if (vpg->vpg_defer_uptodate && !vpg->vpg_ra_used)
+ ll_ra_stats_inc(vmpage->mapping->host, RA_STAT_DISCARDED);
- if (cpg->cpg_defer_uptodate && !cpg->cpg_ra_used)
- ll_ra_stats_inc(mapping, RA_STAT_DISCARDED);
-
- /*
- * truncate_complete_page() calls
- * a_ops->invalidatepage()->cl_page_delete()->vvp_page_delete().
- */
- truncate_complete_page(mapping, vmpage);
-}
-
-static int vvp_page_unmap(const struct lu_env *env,
- const struct cl_page_slice *slice,
- struct cl_io *unused)
-{
- struct page *vmpage = cl2vm_page(slice);
- __u64 offset;
-
- LASSERT(vmpage);
- LASSERT(PageLocked(vmpage));
-
- offset = vmpage->index << PAGE_SHIFT;
-
- /*
- * XXX is it safe to call this with the page lock held?
- */
- ll_teardown_mmaps(vmpage->mapping, offset, offset + PAGE_SIZE);
- return 0;
+ ll_invalidate_page(vmpage);
}
static void vvp_page_delete(const struct lu_env *env,
@@ -179,12 +161,20 @@
struct page *vmpage = cl2vm_page(slice);
struct inode *inode = vmpage->mapping->host;
struct cl_object *obj = slice->cpl_obj;
+ struct cl_page *page = slice->cpl_page;
+ int refc;
LASSERT(PageLocked(vmpage));
- LASSERT((struct cl_page *)vmpage->private == slice->cpl_page);
- LASSERT(inode == ccc_object_inode(obj));
+ LASSERT((struct cl_page *)vmpage->private == page);
+ LASSERT(inode == vvp_object_inode(obj));
- vvp_write_complete(cl2ccc(obj), cl2ccc_page(slice));
+ vvp_write_complete(cl2vvp(obj), cl2vvp_page(slice));
+
+ /* Drop the reference count held in vvp_page_init */
+ refc = atomic_dec_return(&page->cp_ref);
+ LASSERTF(refc >= 1, "page = %p, refc = %d\n", page, refc);
+
+ ClearPageUptodate(vmpage);
ClearPagePrivate(vmpage);
vmpage->private = 0;
/*
@@ -237,7 +227,7 @@
if (!pg->cp_sync_io)
set_page_writeback(vmpage);
- vvp_write_pending(cl2ccc(slice->cpl_obj), cl2ccc_page(slice));
+ vvp_write_pending(cl2vvp(slice->cpl_obj), cl2vvp_page(slice));
return 0;
}
@@ -250,11 +240,11 @@
*/
static void vvp_vmpage_error(struct inode *inode, struct page *vmpage, int ioret)
{
- struct ccc_object *obj = cl_inode2ccc(inode);
+ struct vvp_object *obj = cl_inode2vvp(inode);
if (ioret == 0) {
ClearPageError(vmpage);
- obj->cob_discard_page_warned = 0;
+ obj->vob_discard_page_warned = 0;
} else {
SetPageError(vmpage);
if (ioret == -ENOSPC)
@@ -263,8 +253,8 @@
set_bit(AS_EIO, &inode->i_mapping->flags);
if ((ioret == -ESHUTDOWN || ioret == -EINTR) &&
- obj->cob_discard_page_warned == 0) {
- obj->cob_discard_page_warned = 1;
+ obj->vob_discard_page_warned == 0) {
+ obj->vob_discard_page_warned = 1;
ll_dirty_page_discard_warn(vmpage, ioret);
}
}
@@ -274,22 +264,23 @@
const struct cl_page_slice *slice,
int ioret)
{
- struct ccc_page *cp = cl2ccc_page(slice);
- struct page *vmpage = cp->cpg_page;
- struct cl_page *page = cl_page_top(slice->cpl_page);
- struct inode *inode = ccc_object_inode(page->cp_obj);
+ struct vvp_page *vpg = cl2vvp_page(slice);
+ struct page *vmpage = vpg->vpg_page;
+ struct cl_page *page = slice->cpl_page;
+ struct inode *inode = vvp_object_inode(page->cp_obj);
LASSERT(PageLocked(vmpage));
CL_PAGE_HEADER(D_PAGE, env, page, "completing READ with %d\n", ioret);
- if (cp->cpg_defer_uptodate)
+ if (vpg->vpg_defer_uptodate)
ll_ra_count_put(ll_i2sbi(inode), 1);
if (ioret == 0) {
- if (!cp->cpg_defer_uptodate)
+ if (!vpg->vpg_defer_uptodate)
cl_page_export(env, page, 1);
- } else
- cp->cpg_defer_uptodate = 0;
+ } else {
+ vpg->vpg_defer_uptodate = 0;
+ }
if (!page->cp_sync_io)
unlock_page(vmpage);
@@ -299,9 +290,9 @@
const struct cl_page_slice *slice,
int ioret)
{
- struct ccc_page *cp = cl2ccc_page(slice);
+ struct vvp_page *vpg = cl2vvp_page(slice);
struct cl_page *pg = slice->cpl_page;
- struct page *vmpage = cp->cpg_page;
+ struct page *vmpage = vpg->vpg_page;
CL_PAGE_HEADER(D_PAGE, env, pg, "completing WRITE with %d\n", ioret);
@@ -315,8 +306,8 @@
* and then re-add the page into pending transfer queue. -jay
*/
- cp->cpg_write_queued = 0;
- vvp_write_complete(cl2ccc(slice->cpl_obj), cp);
+ vpg->vpg_write_queued = 0;
+ vvp_write_complete(cl2vvp(slice->cpl_obj), vpg);
if (pg->cp_sync_io) {
LASSERT(PageLocked(vmpage));
@@ -327,7 +318,7 @@
* Only mark the page error only when it's an async write
* because applications won't wait for IO to finish.
*/
- vvp_vmpage_error(ccc_object_inode(pg->cp_obj), vmpage, ioret);
+ vvp_vmpage_error(vvp_object_inode(pg->cp_obj), vmpage, ioret);
end_page_writeback(vmpage);
}
@@ -359,7 +350,7 @@
LASSERT(pg->cp_state == CPS_CACHED);
/* This actually clears the dirty bit in the radix tree. */
set_page_writeback(vmpage);
- vvp_write_pending(cl2ccc(slice->cpl_obj), cl2ccc_page(slice));
+ vvp_write_pending(cl2vvp(slice->cpl_obj), cl2vvp_page(slice));
CL_PAGE_HEADER(D_PAGE, env, pg, "readied\n");
} else if (pg->cp_state == CPS_PAGEOUT) {
/* is it possible for osc_flush_async_page() to already
@@ -375,24 +366,51 @@
return result;
}
+static int vvp_page_is_under_lock(const struct lu_env *env,
+ const struct cl_page_slice *slice,
+ struct cl_io *io, pgoff_t *max_index)
+{
+ if (io->ci_type == CIT_READ || io->ci_type == CIT_WRITE ||
+ io->ci_type == CIT_FAULT) {
+ struct vvp_io *vio = vvp_env_io(env);
+
+ if (unlikely(vio->vui_fd->fd_flags & LL_FILE_GROUP_LOCKED))
+ *max_index = CL_PAGE_EOF;
+ }
+ return 0;
+}
+
static int vvp_page_print(const struct lu_env *env,
const struct cl_page_slice *slice,
void *cookie, lu_printer_t printer)
{
- struct ccc_page *vp = cl2ccc_page(slice);
- struct page *vmpage = vp->cpg_page;
+ struct vvp_page *vpg = cl2vvp_page(slice);
+ struct page *vmpage = vpg->vpg_page;
(*printer)(env, cookie, LUSTRE_VVP_NAME "-page@%p(%d:%d:%d) vm@%p ",
- vp, vp->cpg_defer_uptodate, vp->cpg_ra_used,
- vp->cpg_write_queued, vmpage);
+ vpg, vpg->vpg_defer_uptodate, vpg->vpg_ra_used,
+ vpg->vpg_write_queued, vmpage);
if (vmpage) {
(*printer)(env, cookie, "%lx %d:%d %lx %lu %slru",
(long)vmpage->flags, page_count(vmpage),
page_mapcount(vmpage), vmpage->private,
- page_index(vmpage),
+ vmpage->index,
list_empty(&vmpage->lru) ? "not-" : "");
}
+
(*printer)(env, cookie, "\n");
+
+ return 0;
+}
+
+static int vvp_page_fail(const struct lu_env *env,
+ const struct cl_page_slice *slice)
+{
+ /*
+ * Cached read?
+ */
+ LBUG();
+
return 0;
}
@@ -401,32 +419,38 @@
.cpo_assume = vvp_page_assume,
.cpo_unassume = vvp_page_unassume,
.cpo_disown = vvp_page_disown,
- .cpo_vmpage = ccc_page_vmpage,
.cpo_discard = vvp_page_discard,
.cpo_delete = vvp_page_delete,
- .cpo_unmap = vvp_page_unmap,
.cpo_export = vvp_page_export,
.cpo_is_vmlocked = vvp_page_is_vmlocked,
.cpo_fini = vvp_page_fini,
.cpo_print = vvp_page_print,
- .cpo_is_under_lock = ccc_page_is_under_lock,
+ .cpo_is_under_lock = vvp_page_is_under_lock,
.io = {
[CRT_READ] = {
.cpo_prep = vvp_page_prep_read,
.cpo_completion = vvp_page_completion_read,
- .cpo_make_ready = ccc_fail,
+ .cpo_make_ready = vvp_page_fail,
},
[CRT_WRITE] = {
.cpo_prep = vvp_page_prep_write,
.cpo_completion = vvp_page_completion_write,
.cpo_make_ready = vvp_page_make_ready,
- }
- }
+ },
+ },
};
+static int vvp_transient_page_prep(const struct lu_env *env,
+ const struct cl_page_slice *slice,
+ struct cl_io *unused)
+{
+ /* transient page should always be sent. */
+ return 0;
+}
+
static void vvp_transient_page_verify(const struct cl_page *page)
{
- struct inode *inode = ccc_object_inode(page->cp_obj);
+ struct inode *inode = vvp_object_inode(page->cp_obj);
LASSERT(!inode_trylock(inode));
}
@@ -477,7 +501,7 @@
static int vvp_transient_page_is_vmlocked(const struct lu_env *env,
const struct cl_page_slice *slice)
{
- struct inode *inode = ccc_object_inode(slice->cpl_obj);
+ struct inode *inode = vvp_object_inode(slice->cpl_obj);
int locked;
locked = !inode_trylock(inode);
@@ -497,13 +521,13 @@
static void vvp_transient_page_fini(const struct lu_env *env,
struct cl_page_slice *slice)
{
- struct ccc_page *cp = cl2ccc_page(slice);
+ struct vvp_page *vpg = cl2vvp_page(slice);
struct cl_page *clp = slice->cpl_page;
- struct ccc_object *clobj = cl2ccc(clp->cp_obj);
+ struct vvp_object *clobj = cl2vvp(clp->cp_obj);
- vvp_page_fini_common(cp);
- LASSERT(!inode_trylock(clobj->cob_inode));
- clobj->cob_transient_pages--;
+ vvp_page_fini_common(vpg);
+ LASSERT(!inode_trylock(clobj->vob_inode));
+ clobj->vob_transient_pages--;
}
static const struct cl_page_operations vvp_transient_page_ops = {
@@ -512,45 +536,48 @@
.cpo_unassume = vvp_transient_page_unassume,
.cpo_disown = vvp_transient_page_disown,
.cpo_discard = vvp_transient_page_discard,
- .cpo_vmpage = ccc_page_vmpage,
.cpo_fini = vvp_transient_page_fini,
.cpo_is_vmlocked = vvp_transient_page_is_vmlocked,
.cpo_print = vvp_page_print,
- .cpo_is_under_lock = ccc_page_is_under_lock,
+ .cpo_is_under_lock = vvp_page_is_under_lock,
.io = {
[CRT_READ] = {
- .cpo_prep = ccc_transient_page_prep,
+ .cpo_prep = vvp_transient_page_prep,
.cpo_completion = vvp_transient_page_completion,
},
[CRT_WRITE] = {
- .cpo_prep = ccc_transient_page_prep,
+ .cpo_prep = vvp_transient_page_prep,
.cpo_completion = vvp_transient_page_completion,
}
}
};
int vvp_page_init(const struct lu_env *env, struct cl_object *obj,
- struct cl_page *page, struct page *vmpage)
+ struct cl_page *page, pgoff_t index)
{
- struct ccc_page *cpg = cl_object_page_slice(obj, page);
+ struct vvp_page *vpg = cl_object_page_slice(obj, page);
+ struct page *vmpage = page->cp_vmpage;
- CLOBINVRNT(env, obj, ccc_object_invariant(obj));
+ CLOBINVRNT(env, obj, vvp_object_invariant(obj));
- cpg->cpg_page = vmpage;
+ vpg->vpg_page = vmpage;
get_page(vmpage);
- INIT_LIST_HEAD(&cpg->cpg_pending_linkage);
+ INIT_LIST_HEAD(&vpg->vpg_pending_linkage);
if (page->cp_type == CPT_CACHEABLE) {
+ /* in cache, decref in vvp_page_delete */
+ atomic_inc(&page->cp_ref);
SetPagePrivate(vmpage);
vmpage->private = (unsigned long)page;
- cl_page_slice_add(page, &cpg->cpg_cl, obj, &vvp_page_ops);
+ cl_page_slice_add(page, &vpg->vpg_cl, obj, index,
+ &vvp_page_ops);
} else {
- struct ccc_object *clobj = cl2ccc(obj);
+ struct vvp_object *clobj = cl2vvp(obj);
- LASSERT(!inode_trylock(clobj->cob_inode));
- cl_page_slice_add(page, &cpg->cpg_cl, obj,
+ LASSERT(!inode_trylock(clobj->vob_inode));
+ cl_page_slice_add(page, &vpg->vpg_cl, obj, index,
&vvp_transient_page_ops);
- clobj->cob_transient_pages++;
+ clobj->vob_transient_pages++;
}
return 0;
}
diff --git a/drivers/staging/lustre/lustre/llite/vvp_req.c b/drivers/staging/lustre/lustre/llite/vvp_req.c
new file mode 100644
index 0000000..fb88629
--- /dev/null
+++ b/drivers/staging/lustre/lustre/llite/vvp_req.c
@@ -0,0 +1,121 @@
+/*
+ * GPL HEADER START
+ *
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 only,
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License version 2 for more details (a copy is included
+ * in the LICENSE file that accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 along with this program; If not, see
+ * http://www.gnu.org/licenses/gpl-2.0.html
+ *
+ * GPL HEADER END
+ */
+/*
+ * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.
+ * Use is subject to license terms.
+ *
+ * Copyright (c) 2011, 2014, Intel Corporation.
+ */
+
+#define DEBUG_SUBSYSTEM S_LLITE
+
+#include "../include/lustre/lustre_idl.h"
+#include "../include/cl_object.h"
+#include "../include/obd.h"
+#include "../include/obd_support.h"
+#include "../include/lustre_lite.h"
+#include "llite_internal.h"
+#include "vvp_internal.h"
+
+static inline struct vvp_req *cl2vvp_req(const struct cl_req_slice *slice)
+{
+ return container_of0(slice, struct vvp_req, vrq_cl);
+}
+
+/**
+ * Implementation of struct cl_req_operations::cro_attr_set() for VVP
+ * layer. VVP is responsible for
+ *
+ * - o_[mac]time
+ *
+ * - o_mode
+ *
+ * - o_parent_seq
+ *
+ * - o_[ug]id
+ *
+ * - o_parent_oid
+ *
+ * - o_parent_ver
+ *
+ * - o_ioepoch,
+ *
+ */
+void vvp_req_attr_set(const struct lu_env *env,
+ const struct cl_req_slice *slice,
+ const struct cl_object *obj,
+ struct cl_req_attr *attr, u64 flags)
+{
+ struct inode *inode;
+ struct obdo *oa;
+ u32 valid_flags;
+
+ oa = attr->cra_oa;
+ inode = vvp_object_inode(obj);
+ valid_flags = OBD_MD_FLTYPE;
+
+ if (slice->crs_req->crq_type == CRT_WRITE) {
+ if (flags & OBD_MD_FLEPOCH) {
+ oa->o_valid |= OBD_MD_FLEPOCH;
+ oa->o_ioepoch = ll_i2info(inode)->lli_ioepoch;
+ valid_flags |= OBD_MD_FLMTIME | OBD_MD_FLCTIME |
+ OBD_MD_FLUID | OBD_MD_FLGID;
+ }
+ }
+ obdo_from_inode(oa, inode, valid_flags & flags);
+ obdo_set_parent_fid(oa, &ll_i2info(inode)->lli_fid);
+ memcpy(attr->cra_jobid, ll_i2info(inode)->lli_jobid,
+ JOBSTATS_JOBID_SIZE);
+}
+
+void vvp_req_completion(const struct lu_env *env,
+ const struct cl_req_slice *slice, int ioret)
+{
+ struct vvp_req *vrq;
+
+ if (ioret > 0)
+ cl_stats_tally(slice->crs_dev, slice->crs_req->crq_type, ioret);
+
+ vrq = cl2vvp_req(slice);
+ kmem_cache_free(vvp_req_kmem, vrq);
+}
+
+static const struct cl_req_operations vvp_req_ops = {
+ .cro_attr_set = vvp_req_attr_set,
+ .cro_completion = vvp_req_completion
+};
+
+int vvp_req_init(const struct lu_env *env, struct cl_device *dev,
+ struct cl_req *req)
+{
+ struct vvp_req *vrq;
+ int result;
+
+ vrq = kmem_cache_zalloc(vvp_req_kmem, GFP_NOFS);
+ if (vrq) {
+ cl_req_slice_add(req, &vrq->vrq_cl, dev, &vvp_req_ops);
+ result = 0;
+ } else {
+ result = -ENOMEM;
+ }
+ return result;
+}
diff --git a/drivers/staging/lustre/lustre/llite/xattr.c b/drivers/staging/lustre/lustre/llite/xattr.c
index c671f22..ed4de04 100644
--- a/drivers/staging/lustre/lustre/llite/xattr.c
+++ b/drivers/staging/lustre/lustre/llite/xattr.c
@@ -181,8 +181,9 @@
size = rc;
pv = (const char *)new_value;
- } else
+ } else {
return -EOPNOTSUPP;
+ }
valid |= rce_ops2valid(rce->rce_ops);
}
@@ -218,8 +219,8 @@
LASSERT(inode);
LASSERT(name);
- CDEBUG(D_VFSTRACE, "VFS Op:inode=%lu/%u(%p), xattr %s\n",
- inode->i_ino, inode->i_generation, inode, name);
+ CDEBUG(D_VFSTRACE, "VFS Op:inode="DFID"(%p), xattr %s\n",
+ PFID(ll_inode2fid(inode)), inode, name);
ll_stats_ops_tally(ll_i2sbi(inode), LPROC_LL_SETXATTR, 1);
@@ -243,12 +244,12 @@
lump->lmm_stripe_offset = -1;
if (lump && S_ISREG(inode->i_mode)) {
- int flags = FMODE_WRITE;
+ __u64 it_flags = FMODE_WRITE;
int lum_size = (lump->lmm_magic == LOV_USER_MAGIC_V1) ?
sizeof(*lump) : sizeof(struct lov_user_md_v3);
- rc = ll_lov_setstripe_ea_info(inode, dentry, flags, lump,
- lum_size);
+ rc = ll_lov_setstripe_ea_info(inode, dentry, it_flags,
+ lump, lum_size);
/* b10667: rc always be 0 here for now */
rc = 0;
} else if (S_ISDIR(inode->i_mode)) {
@@ -272,8 +273,8 @@
LASSERT(inode);
LASSERT(name);
- CDEBUG(D_VFSTRACE, "VFS Op:inode=%lu/%u(%p), xattr %s\n",
- inode->i_ino, inode->i_generation, inode, name);
+ CDEBUG(D_VFSTRACE, "VFS Op:inode="DFID"(%p), xattr %s\n",
+ PFID(ll_inode2fid(inode)), inode, name);
ll_stats_ops_tally(ll_i2sbi(inode), LPROC_LL_REMOVEXATTR, 1);
return ll_setxattr_common(inode, name, NULL, 0, 0,
@@ -292,8 +293,8 @@
struct rmtacl_ctl_entry *rce = NULL;
struct ll_inode_info *lli = ll_i2info(inode);
- CDEBUG(D_VFSTRACE, "VFS Op:inode=%lu/%u(%p)\n",
- inode->i_ino, inode->i_generation, inode);
+ CDEBUG(D_VFSTRACE, "VFS Op:inode="DFID"(%p)\n",
+ PFID(ll_inode2fid(inode)), inode);
/* listxattr have slightly different behavior from of ext3:
* without 'user_xattr' ext3 will list all xattr names but
@@ -338,7 +339,6 @@
*/
if (xattr_type == XATTR_ACL_ACCESS_T &&
!(sbi->ll_flags & LL_SBI_RMT_CLIENT)) {
-
struct posix_acl *acl;
spin_lock(&lli->lli_lock);
@@ -423,8 +423,7 @@
if (rce && rce->rce_ops == RMT_LSETFACL) {
ext_acl_xattr_header *acl;
- acl = lustre_posix_acl_xattr_2ext(
- (posix_acl_xattr_header *)buffer, rc);
+ acl = lustre_posix_acl_xattr_2ext(buffer, rc);
if (IS_ERR(acl)) {
rc = PTR_ERR(acl);
goto out;
@@ -457,8 +456,8 @@
LASSERT(inode);
LASSERT(name);
- CDEBUG(D_VFSTRACE, "VFS Op:inode=%lu/%u(%p), xattr %s\n",
- inode->i_ino, inode->i_generation, inode, name);
+ CDEBUG(D_VFSTRACE, "VFS Op:inode="DFID"(%p), xattr %s\n",
+ PFID(ll_inode2fid(inode)), inode, name);
ll_stats_ops_tally(ll_i2sbi(inode), LPROC_LL_GETXATTR, 1);
@@ -552,8 +551,8 @@
LASSERT(inode);
- CDEBUG(D_VFSTRACE, "VFS Op:inode=%lu/%u(%p)\n",
- inode->i_ino, inode->i_generation, inode);
+ CDEBUG(D_VFSTRACE, "VFS Op:inode="DFID"(%p)\n",
+ PFID(ll_inode2fid(inode)), inode);
ll_stats_ops_tally(ll_i2sbi(inode), LPROC_LL_LISTXATTR, 1);
diff --git a/drivers/staging/lustre/lustre/llite/xattr_cache.c b/drivers/staging/lustre/lustre/llite/xattr_cache.c
index 3480ce2..d7e17ab 100644
--- a/drivers/staging/lustre/lustre/llite/xattr_cache.c
+++ b/drivers/staging/lustre/lustre/llite/xattr_cache.c
@@ -229,7 +229,6 @@
*/
static int ll_xattr_cache_destroy_locked(struct ll_inode_info *lli)
{
-
if (!ll_xattr_cache_valid(lli))
return 0;
diff --git a/drivers/staging/lustre/lustre/lmv/lmv_internal.h b/drivers/staging/lustre/lustre/lmv/lmv_internal.h
index 8a00871..7007e4c 100644
--- a/drivers/staging/lustre/lustre/lmv/lmv_internal.h
+++ b/drivers/staging/lustre/lustre/lmv/lmv_internal.h
@@ -42,9 +42,6 @@
#define LMV_MAX_TGT_COUNT 128
-#define lmv_init_lock(lmv) mutex_lock(&lmv->init_mutex)
-#define lmv_init_unlock(lmv) mutex_unlock(&lmv->init_mutex)
-
#define LL_IT2STR(it) \
((it) ? ldlm_it2str((it)->it_op) : "0")
diff --git a/drivers/staging/lustre/lustre/lmv/lmv_obd.c b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
index 9abb7c2..9e31f6b 100644
--- a/drivers/staging/lustre/lustre/lmv/lmv_obd.c
+++ b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
@@ -132,8 +132,9 @@
static struct obd_uuid *lmv_get_uuid(struct obd_export *exp)
{
struct lmv_obd *lmv = &exp->exp_obd->u.lmv;
+ struct lmv_tgt_desc *tgt = lmv->tgts[0];
- return obd_get_uuid(lmv->tgts[0]->ltd_exp);
+ return tgt ? obd_get_uuid(tgt->ltd_exp) : NULL;
}
static int lmv_notify(struct obd_device *obd, struct obd_device *watched,
@@ -249,7 +250,6 @@
static void lmv_set_timeouts(struct obd_device *obd)
{
- struct lmv_tgt_desc *tgt;
struct lmv_obd *lmv;
int i;
@@ -261,8 +261,10 @@
return;
for (i = 0; i < lmv->desc.ld_tgt_count; i++) {
+ struct lmv_tgt_desc *tgt = lmv->tgts[i];
+
tgt = lmv->tgts[i];
- if (!tgt || !tgt->ltd_exp || tgt->ltd_active == 0)
+ if (!tgt || !tgt->ltd_exp || !tgt->ltd_active)
continue;
obd_set_info_async(NULL, tgt->ltd_exp, sizeof(KEY_INTERMDS),
@@ -302,13 +304,14 @@
return 0;
for (i = 0; i < lmv->desc.ld_tgt_count; i++) {
- if (!lmv->tgts[i] || !lmv->tgts[i]->ltd_exp ||
- lmv->tgts[i]->ltd_active == 0) {
+ struct lmv_tgt_desc *tgt = lmv->tgts[i];
+
+ if (!tgt || !tgt->ltd_exp || !tgt->ltd_active) {
CWARN("%s: NULL export for %d\n", obd->obd_name, i);
continue;
}
- rc = md_init_ea_size(lmv->tgts[i]->ltd_exp, easize, def_easize,
+ rc = md_init_ea_size(tgt->ltd_exp, easize, def_easize,
cookiesize, def_cookiesize);
if (rc) {
CERROR("%s: obd_init_ea_size() failed on MDT target %d: rc = %d\n",
@@ -425,7 +428,7 @@
CDEBUG(D_CONFIG, "Target uuid: %s. index %d\n", uuidp->uuid, index);
- lmv_init_lock(lmv);
+ mutex_lock(&lmv->lmv_init_mutex);
if (lmv->desc.ld_tgt_count == 0) {
struct obd_device *mdc_obd;
@@ -433,7 +436,7 @@
mdc_obd = class_find_client_obd(uuidp, LUSTRE_MDC_NAME,
&obd->obd_uuid);
if (!mdc_obd) {
- lmv_init_unlock(lmv);
+ mutex_unlock(&lmv->lmv_init_mutex);
CERROR("%s: Target %s not attached: rc = %d\n",
obd->obd_name, uuidp->uuid, -EINVAL);
return -EINVAL;
@@ -445,7 +448,7 @@
CERROR("%s: UUID %s already assigned at LOV target index %d: rc = %d\n",
obd->obd_name,
obd_uuid2str(&tgt->ltd_uuid), index, -EEXIST);
- lmv_init_unlock(lmv);
+ mutex_unlock(&lmv->lmv_init_mutex);
return -EEXIST;
}
@@ -459,7 +462,7 @@
newsize <<= 1;
newtgts = kcalloc(newsize, sizeof(*newtgts), GFP_NOFS);
if (!newtgts) {
- lmv_init_unlock(lmv);
+ mutex_unlock(&lmv->lmv_init_mutex);
return -ENOMEM;
}
@@ -481,7 +484,7 @@
tgt = kzalloc(sizeof(*tgt), GFP_NOFS);
if (!tgt) {
- lmv_init_unlock(lmv);
+ mutex_unlock(&lmv->lmv_init_mutex);
return -ENOMEM;
}
@@ -507,7 +510,7 @@
}
}
- lmv_init_unlock(lmv);
+ mutex_unlock(&lmv->lmv_init_mutex);
return rc;
}
@@ -522,18 +525,27 @@
if (lmv->connected)
return 0;
- lmv_init_lock(lmv);
+ mutex_lock(&lmv->lmv_init_mutex);
if (lmv->connected) {
- lmv_init_unlock(lmv);
+ mutex_unlock(&lmv->lmv_init_mutex);
return 0;
}
if (lmv->desc.ld_tgt_count == 0) {
- lmv_init_unlock(lmv);
+ mutex_unlock(&lmv->lmv_init_mutex);
CERROR("%s: no targets configured.\n", obd->obd_name);
return -EINVAL;
}
+ LASSERT(lmv->tgts);
+
+ if (!lmv->tgts[0]) {
+ mutex_unlock(&lmv->lmv_init_mutex);
+ CERROR("%s: no target configured for index 0.\n",
+ obd->obd_name);
+ return -EINVAL;
+ }
+
CDEBUG(D_CONFIG, "Time to connect %s to %s\n",
lmv->cluuid.uuid, obd->obd_name);
@@ -551,7 +563,7 @@
lmv->connected = 1;
easize = lmv_get_easize(lmv);
lmv_init_ea_size(obd->obd_self_export, easize, 0, 0, 0);
- lmv_init_unlock(lmv);
+ mutex_unlock(&lmv->lmv_init_mutex);
return 0;
out_disc:
@@ -572,7 +584,7 @@
}
}
class_disconnect(lmv->exp);
- lmv_init_unlock(lmv);
+ mutex_unlock(&lmv->lmv_init_mutex);
return rc;
}
@@ -796,6 +808,11 @@
/* unregister request (call from llapi_hsm_copytool_fini) */
for (i = 0; i < lmv->desc.ld_tgt_count; i++) {
+ struct lmv_tgt_desc *tgt = lmv->tgts[i];
+
+ if (!tgt || !tgt->ltd_exp)
+ continue;
+
/* best effort: try to clean as much as possible
* (continue on error)
*/
@@ -825,20 +842,28 @@
* except if it because of inactive target.
*/
for (i = 0; i < lmv->desc.ld_tgt_count; i++) {
- err = obd_iocontrol(cmd, lmv->tgts[i]->ltd_exp, len, lk, uarg);
+ struct lmv_tgt_desc *tgt = lmv->tgts[i];
+
+ if (!tgt || !tgt->ltd_exp)
+ continue;
+
+ err = obd_iocontrol(cmd, tgt->ltd_exp, len, lk, uarg);
if (err) {
- if (lmv->tgts[i]->ltd_active) {
+ if (tgt->ltd_active) {
/* permanent error */
CERROR("error: iocontrol MDC %s on MDTidx %d cmd %x: err = %d\n",
- lmv->tgts[i]->ltd_uuid.uuid,
- i, cmd, err);
+ tgt->ltd_uuid.uuid, i, cmd, err);
rc = err;
lk->lk_flags |= LK_FLG_STOP;
/* unregister from previous MDS */
- for (j = 0; j < i; j++)
- obd_iocontrol(cmd,
- lmv->tgts[j]->ltd_exp,
- len, lk, uarg);
+ for (j = 0; j < i; j++) {
+ tgt = lmv->tgts[j];
+
+ if (!tgt || !tgt->ltd_exp)
+ continue;
+ obd_iocontrol(cmd, tgt->ltd_exp, len,
+ lk, uarg);
+ }
return rc;
}
/* else: transient error.
@@ -877,6 +902,7 @@
{
struct obd_device *obddev = class_exp2obd(exp);
struct lmv_obd *lmv = &obddev->u.lmv;
+ struct lmv_tgt_desc *tgt = NULL;
int i = 0;
int rc = 0;
int set = 0;
@@ -896,10 +922,11 @@
if (index >= count)
return -ENODEV;
- if (!lmv->tgts[index] || lmv->tgts[index]->ltd_active == 0)
+ tgt = lmv->tgts[index];
+ if (!tgt || !tgt->ltd_active)
return -ENODATA;
- mdc_obd = class_exp2obd(lmv->tgts[index]->ltd_exp);
+ mdc_obd = class_exp2obd(tgt->ltd_exp);
if (!mdc_obd)
return -EINVAL;
@@ -909,7 +936,7 @@
(int)sizeof(struct obd_uuid))))
return -EFAULT;
- rc = obd_statfs(NULL, lmv->tgts[index]->ltd_exp, &stat_buf,
+ rc = obd_statfs(NULL, tgt->ltd_exp, &stat_buf,
cfs_time_shift_64(-OBD_STATFS_CACHE_SECONDS),
0);
if (rc)
@@ -922,11 +949,10 @@
}
case OBD_IOC_QUOTACTL: {
struct if_quotactl *qctl = karg;
- struct lmv_tgt_desc *tgt = NULL;
struct obd_quotactl *oqctl;
if (qctl->qc_valid == QC_MDTIDX) {
- if (qctl->qc_idx < 0 || count <= qctl->qc_idx)
+ if (count <= qctl->qc_idx)
return -EINVAL;
tgt = lmv->tgts[qctl->qc_idx];
@@ -975,18 +1001,18 @@
if (icc->icc_mdtindex >= count)
return -ENODEV;
- if (!lmv->tgts[icc->icc_mdtindex] ||
- !lmv->tgts[icc->icc_mdtindex]->ltd_exp ||
- lmv->tgts[icc->icc_mdtindex]->ltd_active == 0)
+ tgt = lmv->tgts[icc->icc_mdtindex];
+ if (!tgt || !tgt->ltd_exp || !tgt->ltd_active)
return -ENODEV;
- rc = obd_iocontrol(cmd, lmv->tgts[icc->icc_mdtindex]->ltd_exp,
- sizeof(*icc), icc, NULL);
+ rc = obd_iocontrol(cmd, tgt->ltd_exp, sizeof(*icc), icc, NULL);
break;
}
case LL_IOC_GET_CONNECT_FLAGS: {
- if (!lmv->tgts[0])
+ tgt = lmv->tgts[0];
+
+ if (!tgt || !tgt->ltd_exp)
return -ENODATA;
- rc = obd_iocontrol(cmd, lmv->tgts[0]->ltd_exp, len, karg, uarg);
+ rc = obd_iocontrol(cmd, tgt->ltd_exp, len, karg, uarg);
break;
}
case OBD_IOC_FID2PATH: {
@@ -997,7 +1023,6 @@
case LL_IOC_HSM_STATE_SET:
case LL_IOC_HSM_ACTION: {
struct md_op_data *op_data = karg;
- struct lmv_tgt_desc *tgt;
tgt = lmv_find_target(lmv, &op_data->op_fid1);
if (IS_ERR(tgt))
@@ -1011,7 +1036,6 @@
}
case LL_IOC_HSM_PROGRESS: {
const struct hsm_progress_kernel *hpk = karg;
- struct lmv_tgt_desc *tgt;
tgt = lmv_find_target(lmv, &hpk->hpk_fid);
if (IS_ERR(tgt))
@@ -1021,7 +1045,6 @@
}
case LL_IOC_HSM_REQUEST: {
struct hsm_user_request *hur = karg;
- struct lmv_tgt_desc *tgt;
unsigned int reqcount = hur->hur_request.hr_itemcount;
if (reqcount == 0)
@@ -1044,7 +1067,11 @@
int rc1;
struct hsm_user_request *req;
- nr = lmv_hsm_req_count(lmv, hur, lmv->tgts[i]);
+ tgt = lmv->tgts[i];
+ if (!tgt || !tgt->ltd_exp)
+ continue;
+
+ nr = lmv_hsm_req_count(lmv, hur, tgt);
if (nr == 0) /* nothing for this MDS */
continue;
@@ -1056,10 +1083,10 @@
if (!req)
return -ENOMEM;
- lmv_hsm_req_build(lmv, hur, lmv->tgts[i], req);
+ lmv_hsm_req_build(lmv, hur, tgt, req);
- rc1 = obd_iocontrol(cmd, lmv->tgts[i]->ltd_exp,
- reqlen, req, uarg);
+ rc1 = obd_iocontrol(cmd, tgt->ltd_exp, reqlen,
+ req, uarg);
if (rc1 != 0 && rc == 0)
rc = rc1;
kvfree(req);
@@ -1103,27 +1130,27 @@
struct obd_device *mdc_obd;
int err;
- if (!lmv->tgts[i] || !lmv->tgts[i]->ltd_exp)
+ tgt = lmv->tgts[i];
+ if (!tgt || !tgt->ltd_exp)
continue;
/* ll_umount_begin() sets force flag but for lmv, not
* mdc. Let's pass it through
*/
- mdc_obd = class_exp2obd(lmv->tgts[i]->ltd_exp);
+ mdc_obd = class_exp2obd(tgt->ltd_exp);
mdc_obd->obd_force = obddev->obd_force;
- err = obd_iocontrol(cmd, lmv->tgts[i]->ltd_exp, len,
- karg, uarg);
+ err = obd_iocontrol(cmd, tgt->ltd_exp, len, karg, uarg);
if (err == -ENODATA && cmd == OBD_IOC_POLL_QUOTACHECK) {
return err;
} else if (err) {
- if (lmv->tgts[i]->ltd_active) {
+ if (tgt->ltd_active) {
CERROR("error: iocontrol MDC %s on MDTidx %d cmd %x: err = %d\n",
- lmv->tgts[i]->ltd_uuid.uuid,
- i, cmd, err);
+ tgt->ltd_uuid.uuid, i, cmd, err);
if (!rc)
rc = err;
}
- } else
+ } else {
set = 1;
+ }
}
if (!set && !rc)
rc = -EIO;
@@ -1269,7 +1296,7 @@
lmv->lmv_placement = PLACEMENT_CHAR_POLICY;
spin_lock_init(&lmv->lmv_lock);
- mutex_init(&lmv->init_mutex);
+ mutex_init(&lmv->lmv_init_mutex);
lprocfs_lmv_init_vars(&lvars);
@@ -2071,7 +2098,7 @@
dp = (struct lu_dirpage *)((char *)dp + LU_PAGE_SIZE);
/* Check if we've reached the end of the CFS_PAGE. */
- if (!((unsigned long)dp & ~CFS_PAGE_MASK))
+ if (!((unsigned long)dp & ~PAGE_MASK))
break;
/* Save the hash and flags of this lu_dirpage. */
@@ -2268,7 +2295,6 @@
lmv = &obd->u.lmv;
if (keylen >= strlen("remote_flag") && !strcmp(key, "remote_flag")) {
- struct lmv_tgt_desc *tgt;
int i;
rc = lmv_check_connect(obd);
@@ -2277,7 +2303,8 @@
LASSERT(*vallen == sizeof(__u32));
for (i = 0; i < lmv->desc.ld_tgt_count; i++) {
- tgt = lmv->tgts[i];
+ struct lmv_tgt_desc *tgt = lmv->tgts[i];
+
/*
* All tgts should be connected when this gets called.
*/
@@ -2466,12 +2493,13 @@
LASSERT(fid);
for (i = 0; i < lmv->desc.ld_tgt_count; i++) {
- if (!lmv->tgts[i] || !lmv->tgts[i]->ltd_exp ||
- lmv->tgts[i]->ltd_active == 0)
+ struct lmv_tgt_desc *tgt = lmv->tgts[i];
+
+ if (!tgt || !tgt->ltd_exp || !tgt->ltd_active)
continue;
- err = md_cancel_unused(lmv->tgts[i]->ltd_exp, fid,
- policy, mode, flags, opaque);
+ err = md_cancel_unused(tgt->ltd_exp, fid, policy, mode, flags,
+ opaque);
if (!rc)
rc = err;
}
@@ -2482,9 +2510,13 @@
__u64 *bits)
{
struct lmv_obd *lmv = &exp->exp_obd->u.lmv;
+ struct lmv_tgt_desc *tgt = lmv->tgts[0];
int rc;
- rc = md_set_lock_data(lmv->tgts[0]->ltd_exp, lockh, data, bits);
+ if (!tgt || !tgt->ltd_exp)
+ return -EINVAL;
+
+ rc = md_set_lock_data(tgt->ltd_exp, lockh, data, bits);
return rc;
}
@@ -2509,12 +2541,13 @@
* one fid was created in.
*/
for (i = 0; i < lmv->desc.ld_tgt_count; i++) {
- if (!lmv->tgts[i] || !lmv->tgts[i]->ltd_exp ||
- lmv->tgts[i]->ltd_active == 0)
+ struct lmv_tgt_desc *tgt = lmv->tgts[i];
+
+ if (!tgt || !tgt->ltd_exp || !tgt->ltd_active)
continue;
- rc = md_lock_match(lmv->tgts[i]->ltd_exp, flags, fid,
- type, policy, mode, lockh);
+ rc = md_lock_match(tgt->ltd_exp, flags, fid, type, policy, mode,
+ lockh);
if (rc)
return rc;
}
@@ -2529,18 +2562,24 @@
struct lustre_md *md)
{
struct lmv_obd *lmv = &exp->exp_obd->u.lmv;
+ struct lmv_tgt_desc *tgt = lmv->tgts[0];
- return md_get_lustre_md(lmv->tgts[0]->ltd_exp, req, dt_exp, md_exp, md);
+ if (!tgt || !tgt->ltd_exp)
+ return -EINVAL;
+ return md_get_lustre_md(tgt->ltd_exp, req, dt_exp, md_exp, md);
}
static int lmv_free_lustre_md(struct obd_export *exp, struct lustre_md *md)
{
struct obd_device *obd = exp->exp_obd;
struct lmv_obd *lmv = &obd->u.lmv;
+ struct lmv_tgt_desc *tgt = lmv->tgts[0];
if (md->mea)
obd_free_memmd(exp, (void *)&md->mea);
- return md_free_lustre_md(lmv->tgts[0]->ltd_exp, md);
+ if (!tgt || !tgt->ltd_exp)
+ return -EINVAL;
+ return md_free_lustre_md(tgt->ltd_exp, md);
}
static int lmv_set_open_replay_data(struct obd_export *exp,
@@ -2649,7 +2688,8 @@
int rc = 0, i;
__u64 curspace, curinodes;
- if (!lmv->desc.ld_tgt_count || !tgt->ltd_active) {
+ if (!tgt || !tgt->ltd_exp || !tgt->ltd_active ||
+ !lmv->desc.ld_tgt_count) {
CERROR("master lmv inactive\n");
return -EIO;
}
@@ -2665,12 +2705,8 @@
tgt = lmv->tgts[i];
- if (!tgt || !tgt->ltd_exp || tgt->ltd_active == 0)
+ if (!tgt || !tgt->ltd_exp || !tgt->ltd_active)
continue;
- if (!tgt->ltd_active) {
- CDEBUG(D_HA, "mdt %d is inactive.\n", i);
- continue;
- }
err = obd_quotactl(tgt->ltd_exp, oqctl);
if (err) {
diff --git a/drivers/staging/lustre/lustre/lov/lov_cl_internal.h b/drivers/staging/lustre/lustre/lov/lov_cl_internal.h
index 7dd3162..ac9744e 100644
--- a/drivers/staging/lustre/lustre/lov/lov_cl_internal.h
+++ b/drivers/staging/lustre/lustre/lov/lov_cl_internal.h
@@ -73,19 +73,6 @@
* - top-page keeps a reference to its sub-page, and destroys it when it
* is destroyed.
*
- * - sub-lock keep a reference to its top-locks. Top-lock keeps a
- * reference (and a hold, see cl_lock_hold()) on its sub-locks when it
- * actively using them (that is, in cl_lock_state::CLS_QUEUING,
- * cl_lock_state::CLS_ENQUEUED, cl_lock_state::CLS_HELD states). When
- * moving into cl_lock_state::CLS_CACHED state, top-lock releases a
- * hold. From this moment top-lock has only a 'weak' reference to its
- * sub-locks. This reference is protected by top-lock
- * cl_lock::cll_guard, and will be automatically cleared by the sub-lock
- * when the latter is destroyed. When a sub-lock is canceled, a
- * reference to it is removed from the top-lock array, and top-lock is
- * moved into CLS_NEW state. It is guaranteed that all sub-locks exist
- * while their top-lock is in CLS_HELD or CLS_CACHED states.
- *
* - IO's are not reference counted.
*
* To implement a connection between top and sub entities, lov layer is split
@@ -281,24 +268,17 @@
};
/**
- * Flags that top-lock can set on each of its sub-locks.
- */
-enum lov_sub_flags {
- /** Top-lock acquired a hold (cl_lock_hold()) on a sub-lock. */
- LSF_HELD = 1 << 0
-};
-
-/**
* State lov_lock keeps for each sub-lock.
*/
struct lov_lock_sub {
/** sub-lock itself */
- struct lovsub_lock *sub_lock;
- /** An array of per-sub-lock flags, taken from enum lov_sub_flags */
- unsigned sub_flags;
+ struct cl_lock sub_lock;
+ /** Set if the sublock has ever been enqueued, meaning it may
+ * hold resources of underlying layers
+ */
+ unsigned int sub_is_enqueued:1,
+ sub_initialized:1;
int sub_stripe;
- struct cl_lock_descr sub_descr;
- struct cl_lock_descr sub_got;
};
/**
@@ -308,59 +288,8 @@
struct cl_lock_slice lls_cl;
/** Number of sub-locks in this lock */
int lls_nr;
- /**
- * Number of existing sub-locks.
- */
- unsigned lls_nr_filled;
- /**
- * Set when sub-lock was canceled, while top-lock was being
- * used, or unused.
- */
- unsigned int lls_cancel_race:1;
- /**
- * An array of sub-locks
- *
- * There are two issues with managing sub-locks:
- *
- * - sub-locks are concurrently canceled, and
- *
- * - sub-locks are shared with other top-locks.
- *
- * To manage cancellation, top-lock acquires a hold on a sublock
- * (lov_sublock_adopt()) when the latter is inserted into
- * lov_lock::lls_sub[]. This hold is released (lov_sublock_release())
- * when top-lock is going into CLS_CACHED state or destroyed. Hold
- * prevents sub-lock from cancellation.
- *
- * Sub-lock sharing means, among other things, that top-lock that is
- * in the process of creation (i.e., not yet inserted into lock list)
- * is already accessible to other threads once at least one of its
- * sub-locks is created, see lov_lock_sub_init().
- *
- * Sub-lock can be in one of the following states:
- *
- * - doesn't exist, lov_lock::lls_sub[]::sub_lock == NULL. Such
- * sub-lock was either never created (top-lock is in CLS_NEW
- * state), or it was created, then canceled, then destroyed
- * (lov_lock_unlink() cleared sub-lock pointer in the top-lock).
- *
- * - sub-lock exists and is on
- * hold. (lov_lock::lls_sub[]::sub_flags & LSF_HELD). This is a
- * normal state of a sub-lock in CLS_HELD and CLS_CACHED states
- * of a top-lock.
- *
- * - sub-lock exists, but is not held by the top-lock. This
- * happens after top-lock released a hold on sub-locks before
- * going into cache (lov_lock_unuse()).
- *
- * \todo To support wide-striping, array has to be replaced with a set
- * of queues to avoid scanning.
- */
- struct lov_lock_sub *lls_sub;
- /**
- * Original description with which lock was enqueued.
- */
- struct cl_lock_descr lls_orig;
+ /** sublock array */
+ struct lov_lock_sub lls_sub[0];
};
struct lov_page {
@@ -444,8 +373,9 @@
struct cl_lock_descr lti_ldescr;
struct ost_lvb lti_lvb;
struct cl_2queue lti_cl2q;
- struct cl_lock_closure lti_closure;
+ struct cl_page_list lti_plist;
wait_queue_t lti_waiter;
+ struct cl_attr lti_attr;
};
/**
@@ -611,14 +541,13 @@
const struct cl_lock_descr *d, int idx);
int lov_page_init(const struct lu_env *env, struct cl_object *ob,
- struct cl_page *page, struct page *vmpage);
+ struct cl_page *page, pgoff_t index);
int lovsub_page_init(const struct lu_env *env, struct cl_object *ob,
- struct cl_page *page, struct page *vmpage);
-
+ struct cl_page *page, pgoff_t index);
int lov_page_init_empty(const struct lu_env *env, struct cl_object *obj,
- struct cl_page *page, struct page *vmpage);
+ struct cl_page *page, pgoff_t index);
int lov_page_init_raid0(const struct lu_env *env, struct cl_object *obj,
- struct cl_page *page, struct page *vmpage);
+ struct cl_page *page, pgoff_t index);
struct lu_object *lov_object_alloc(const struct lu_env *env,
const struct lu_object_header *hdr,
struct lu_device *dev);
@@ -631,6 +560,7 @@
struct lovsub_lock *sub);
struct lov_io_sub *lov_page_subio(const struct lu_env *env, struct lov_io *lio,
const struct cl_page_slice *slice);
+int lov_page_stripe(const struct cl_page *page);
#define lov_foreach_target(lov, var) \
for (var = 0; var < lov_targets_nr(lov); ++var)
@@ -789,11 +719,6 @@
return container_of0(slice, struct lovsub_req, lsrq_cl);
}
-static inline struct cl_page *lov_sub_page(const struct cl_page_slice *slice)
-{
- return slice->cpl_page->cp_child;
-}
-
static inline struct lov_io *cl2lov_io(const struct lu_env *env,
const struct cl_io_slice *ios)
{
diff --git a/drivers/staging/lustre/lustre/lov/lov_dev.c b/drivers/staging/lustre/lustre/lov/lov_dev.c
index 532ef87..dae8e89 100644
--- a/drivers/staging/lustre/lustre/lov/lov_dev.c
+++ b/drivers/staging/lustre/lustre/lov/lov_dev.c
@@ -143,9 +143,7 @@
struct lov_thread_info *info;
info = kmem_cache_zalloc(lov_thread_kmem, GFP_NOFS);
- if (info)
- INIT_LIST_HEAD(&info->lti_closure.clc_list);
- else
+ if (!info)
info = ERR_PTR(-ENOMEM);
return info;
}
@@ -155,7 +153,6 @@
{
struct lov_thread_info *info = data;
- LINVRNT(list_empty(&info->lti_closure.clc_list));
kmem_cache_free(lov_thread_kmem, info);
}
@@ -265,8 +262,9 @@
if (lr) {
cl_req_slice_add(req, &lr->lr_cl, dev, &lov_req_ops);
result = 0;
- } else
+ } else {
result = -ENOMEM;
+ }
return result;
}
@@ -335,14 +333,15 @@
cl_page_list_init(&em->emrg_page_list);
em->emrg_env = cl_env_alloc(&em->emrg_refcheck,
LCT_REMEMBER | LCT_NOREF);
- if (!IS_ERR(em->emrg_env))
+ if (!IS_ERR(em->emrg_env)) {
em->emrg_env->le_ctx.lc_cookie = 0x2;
- else {
+ } else {
result = PTR_ERR(em->emrg_env);
em->emrg_env = NULL;
}
- } else
+ } else {
result = -ENOMEM;
+ }
}
if (result != 0) {
lov_emerg_free(emerg, nr);
diff --git a/drivers/staging/lustre/lustre/lov/lov_ea.c b/drivers/staging/lustre/lustre/lov/lov_ea.c
index b652940..460f0fa 100644
--- a/drivers/staging/lustre/lustre/lov/lov_ea.c
+++ b/drivers/staging/lustre/lustre/lov/lov_ea.c
@@ -48,11 +48,6 @@
#include "lov_internal.h"
-struct lovea_unpack_args {
- struct lov_stripe_md *lsm;
- int cursor;
-};
-
static int lsm_lmm_verify_common(struct lov_mds_md *lmm, int lmm_bytes,
__u16 stripe_count)
{
diff --git a/drivers/staging/lustre/lustre/lov/lov_internal.h b/drivers/staging/lustre/lustre/lov/lov_internal.h
index 590f932..eef9afa 100644
--- a/drivers/staging/lustre/lustre/lov/lov_internal.h
+++ b/drivers/staging/lustre/lustre/lov/lov_internal.h
@@ -72,6 +72,21 @@
})
#endif
+#define pool_tgt_size(p) ((p)->pool_obds.op_size)
+#define pool_tgt_count(p) ((p)->pool_obds.op_count)
+#define pool_tgt_array(p) ((p)->pool_obds.op_array)
+#define pool_tgt_rw_sem(p) ((p)->pool_obds.op_rw_sem)
+
+struct pool_desc {
+ char pool_name[LOV_MAXPOOLNAME + 1];
+ struct ost_pool pool_obds;
+ atomic_t pool_refcount;
+ struct hlist_node pool_hash; /* access by poolname */
+ struct list_head pool_list; /* serial access */
+ struct dentry *pool_debugfs_entry; /* file in debugfs */
+ struct obd_device *pool_lobd; /* owner */
+};
+
struct lov_request {
struct obd_info rq_oi;
struct lov_request_set *rq_rqset;
@@ -88,7 +103,6 @@
};
struct lov_request_set {
- struct ldlm_enqueue_info *set_ei;
struct obd_info *set_oi;
atomic_t set_refcount;
struct obd_export *set_exp;
@@ -102,10 +116,8 @@
atomic_t set_finish_checked;
struct llog_cookie *set_cookies;
int set_cookie_sent;
- struct obd_trans_info *set_oti;
struct list_head set_list;
wait_queue_head_t set_waitq;
- spinlock_t set_lock;
};
extern struct kmem_cache *lov_oinfo_slab;
@@ -114,12 +126,6 @@
void lov_finish_set(struct lov_request_set *set);
-static inline void lov_get_reqset(struct lov_request_set *set)
-{
- LASSERT(atomic_read(&set->set_refcount) > 0);
- atomic_inc(&set->set_refcount);
-}
-
static inline void lov_put_reqset(struct lov_request_set *set)
{
if (atomic_dec_and_test(&set->set_refcount))
@@ -146,10 +152,8 @@
u64 start, u64 end,
u64 *obd_start, u64 *obd_end);
int lov_stripe_number(struct lov_stripe_md *lsm, u64 lov_off);
-
-/* lov_qos.c */
-#define LOV_USES_ASSIGNED_STRIPE 0
-#define LOV_USES_DEFAULT_STRIPE 1
+pgoff_t lov_stripe_pgoff(struct lov_stripe_md *lsm, pgoff_t stripe_index,
+ int stripe);
/* lov_request.c */
int lov_update_common_set(struct lov_request_set *set,
@@ -176,6 +180,8 @@
int lov_statfs_interpret(struct ptlrpc_request_set *rqset, void *data, int rc);
/* lov_obd.c */
+void lov_stripe_lock(struct lov_stripe_md *md);
+void lov_stripe_unlock(struct lov_stripe_md *md);
void lov_fix_desc(struct lov_desc *desc);
void lov_fix_desc_stripe_size(__u64 *val);
void lov_fix_desc_stripe_count(__u32 *val);
@@ -231,8 +237,6 @@
int lov_pool_del(struct obd_device *obd, char *poolname);
int lov_pool_add(struct obd_device *obd, char *poolname, char *ostname);
int lov_pool_remove(struct obd_device *obd, char *poolname, char *ostname);
-struct pool_desc *lov_find_pool(struct lov_obd *lov, char *poolname);
-int lov_check_index_in_pool(__u32 idx, struct pool_desc *pool);
void lov_pool_putref(struct pool_desc *pool);
static inline struct lov_stripe_md *lsm_addref(struct lov_stripe_md *lsm)
diff --git a/drivers/staging/lustre/lustre/lov/lov_io.c b/drivers/staging/lustre/lustre/lov/lov_io.c
index 4296aac..86cb3f8 100644
--- a/drivers/staging/lustre/lustre/lov/lov_io.c
+++ b/drivers/staging/lustre/lustre/lov/lov_io.c
@@ -225,8 +225,9 @@
if (!sub->sub_io_initialized) {
sub->sub_stripe = stripe;
rc = lov_io_sub_init(env, lio, sub);
- } else
+ } else {
rc = 0;
+ }
if (rc == 0)
lov_sub_enter(sub);
else
@@ -245,13 +246,15 @@
*
*/
-static int lov_page_stripe(const struct cl_page *page)
+int lov_page_stripe(const struct cl_page *page)
{
struct lovsub_object *subobj;
+ const struct cl_page_slice *slice;
- subobj = lu2lovsub(
- lu_object_locate(page->cp_child->cp_obj->co_lu.lo_header,
- &lovsub_device_type));
+ slice = cl_page_at(page, &lovsub_device_type);
+ LASSERT(slice->cpl_obj);
+
+ subobj = cl2lovsub(slice->cpl_obj);
return subobj->lso_index;
}
@@ -274,10 +277,11 @@
static int lov_io_subio_init(const struct lu_env *env, struct lov_io *lio,
struct cl_io *io)
{
- struct lov_stripe_md *lsm = lio->lis_object->lo_lsm;
+ struct lov_stripe_md *lsm;
int result;
LASSERT(lio->lis_object);
+ lsm = lio->lis_object->lo_lsm;
/*
* Need to be optimized, we can't afford to allocate a piece of memory
@@ -292,8 +296,9 @@
lio->lis_single_subio_index = -1;
lio->lis_active_subios = 0;
result = 0;
- } else
+ } else {
result = -ENOMEM;
+ }
return result;
}
@@ -411,8 +416,9 @@
lov_sub_put(sub);
CDEBUG(D_VFSTRACE, "shrink: %d [%llu, %llu)\n",
stripe, start, end);
- } else
+ } else {
rc = PTR_ERR(sub);
+ }
if (!rc)
list_add_tail(&sub->sub_linkage, &lio->lis_active);
@@ -436,7 +442,6 @@
/* fast path for common case. */
if (lio->lis_nr_subios != 1 && !cl_io_is_append(io)) {
-
lov_do_div64(start, ssize);
next = (start + 1) * ssize;
if (next <= start * ssize)
@@ -543,13 +548,6 @@
LASSERT(rc == 0);
}
-static struct cl_page_list *lov_io_submit_qin(struct lov_device *ld,
- struct cl_page_list *qin,
- int idx, int alloc)
-{
- return alloc ? &qin[idx] : &ld->ld_emrg[idx]->emrg_page_list;
-}
-
/**
* lov implementation of cl_operations::cio_submit() method. It takes a list
* of pages in \a queue, splits it into per-stripe sub-lists, invokes
@@ -569,25 +567,17 @@
const struct cl_io_slice *ios,
enum cl_req_type crt, struct cl_2queue *queue)
{
- struct lov_io *lio = cl2lov_io(env, ios);
- struct lov_object *obj = lio->lis_object;
- struct lov_device *ld = lu2lov_dev(lov2cl(obj)->co_lu.lo_dev);
- struct cl_page_list *qin = &queue->c2_qin;
- struct cl_2queue *cl2q = &lov_env_info(env)->lti_cl2q;
- struct cl_page_list *stripes_qin = NULL;
+ struct cl_page_list *qin = &queue->c2_qin;
+ struct lov_io *lio = cl2lov_io(env, ios);
+ struct lov_io_sub *sub;
+ struct cl_page_list *plist = &lov_env_info(env)->lti_plist;
struct cl_page *page;
- struct cl_page *tmp;
int stripe;
-#define QIN(stripe) lov_io_submit_qin(ld, stripes_qin, stripe, alloc)
-
int rc = 0;
- int alloc =
- !(current->flags & PF_MEMALLOC);
if (lio->lis_active_subios == 1) {
int idx = lio->lis_single_subio_index;
- struct lov_io_sub *sub;
LASSERT(idx < lio->lis_nr_subios);
sub = lov_sub_get(env, lio, idx);
@@ -600,119 +590,120 @@
}
LASSERT(lio->lis_subs);
- if (alloc) {
- stripes_qin =
- libcfs_kvzalloc(sizeof(*stripes_qin) *
- lio->lis_nr_subios,
- GFP_NOFS);
- if (!stripes_qin)
- return -ENOMEM;
- for (stripe = 0; stripe < lio->lis_nr_subios; stripe++)
- cl_page_list_init(&stripes_qin[stripe]);
- } else {
- /*
- * If we get here, it means pageout & swap doesn't help.
- * In order to not make things worse, even don't try to
- * allocate the memory with __GFP_NOWARN. -jay
- */
- mutex_lock(&ld->ld_mutex);
- lio->lis_mem_frozen = 1;
- }
+ cl_page_list_init(plist);
+ while (qin->pl_nr > 0) {
+ struct cl_2queue *cl2q = &lov_env_info(env)->lti_cl2q;
- cl_2queue_init(cl2q);
- cl_page_list_for_each_safe(page, tmp, qin) {
+ cl_2queue_init(cl2q);
+
+ page = cl_page_list_first(qin);
+ cl_page_list_move(&cl2q->c2_qin, qin, page);
+
stripe = lov_page_stripe(page);
- cl_page_list_move(QIN(stripe), qin, page);
- }
+ while (qin->pl_nr > 0) {
+ page = cl_page_list_first(qin);
+ if (stripe != lov_page_stripe(page))
+ break;
- for (stripe = 0; stripe < lio->lis_nr_subios; stripe++) {
- struct lov_io_sub *sub;
- struct cl_page_list *sub_qin = QIN(stripe);
+ cl_page_list_move(&cl2q->c2_qin, qin, page);
+ }
- if (list_empty(&sub_qin->pl_pages))
- continue;
-
- cl_page_list_splice(sub_qin, &cl2q->c2_qin);
sub = lov_sub_get(env, lio, stripe);
if (!IS_ERR(sub)) {
rc = cl_io_submit_rw(sub->sub_env, sub->sub_io,
crt, cl2q);
lov_sub_put(sub);
- } else
+ } else {
rc = PTR_ERR(sub);
- cl_page_list_splice(&cl2q->c2_qin, &queue->c2_qin);
+ }
+
+ cl_page_list_splice(&cl2q->c2_qin, plist);
cl_page_list_splice(&cl2q->c2_qout, &queue->c2_qout);
+ cl_2queue_fini(env, cl2q);
+
if (rc != 0)
break;
}
- for (stripe = 0; stripe < lio->lis_nr_subios; stripe++) {
- struct cl_page_list *sub_qin = QIN(stripe);
+ cl_page_list_splice(plist, qin);
+ cl_page_list_fini(env, plist);
- if (list_empty(&sub_qin->pl_pages))
- continue;
+ return rc;
+}
- cl_page_list_splice(sub_qin, qin);
+static int lov_io_commit_async(const struct lu_env *env,
+ const struct cl_io_slice *ios,
+ struct cl_page_list *queue, int from, int to,
+ cl_commit_cbt cb)
+{
+ struct cl_page_list *plist = &lov_env_info(env)->lti_plist;
+ struct lov_io *lio = cl2lov_io(env, ios);
+ struct lov_io_sub *sub;
+ struct cl_page *page;
+ int rc = 0;
+
+ if (lio->lis_active_subios == 1) {
+ int idx = lio->lis_single_subio_index;
+
+ LASSERT(idx < lio->lis_nr_subios);
+ sub = lov_sub_get(env, lio, idx);
+ LASSERT(!IS_ERR(sub));
+ LASSERT(sub->sub_io == &lio->lis_single_subio);
+ rc = cl_io_commit_async(sub->sub_env, sub->sub_io, queue,
+ from, to, cb);
+ lov_sub_put(sub);
+ return rc;
}
- if (alloc) {
- kvfree(stripes_qin);
- } else {
- int i;
+ LASSERT(lio->lis_subs);
- for (i = 0; i < lio->lis_nr_subios; i++) {
- struct cl_io *cio = lio->lis_subs[i].sub_io;
+ cl_page_list_init(plist);
+ while (queue->pl_nr > 0) {
+ int stripe_to = to;
+ int stripe;
- if (cio && cio == &ld->ld_emrg[i]->emrg_subio)
- lov_io_sub_fini(env, lio, &lio->lis_subs[i]);
+ LASSERT(plist->pl_nr == 0);
+ page = cl_page_list_first(queue);
+ cl_page_list_move(plist, queue, page);
+
+ stripe = lov_page_stripe(page);
+ while (queue->pl_nr > 0) {
+ page = cl_page_list_first(queue);
+ if (stripe != lov_page_stripe(page))
+ break;
+
+ cl_page_list_move(plist, queue, page);
}
- lio->lis_mem_frozen = 0;
- mutex_unlock(&ld->ld_mutex);
+
+ if (queue->pl_nr > 0) /* still has more pages */
+ stripe_to = PAGE_SIZE;
+
+ sub = lov_sub_get(env, lio, stripe);
+ if (!IS_ERR(sub)) {
+ rc = cl_io_commit_async(sub->sub_env, sub->sub_io,
+ plist, from, stripe_to, cb);
+ lov_sub_put(sub);
+ } else {
+ rc = PTR_ERR(sub);
+ break;
+ }
+
+ if (plist->pl_nr > 0) /* short write */
+ break;
+
+ from = 0;
+ }
+
+ /* for error case, add the page back into the qin list */
+ LASSERT(ergo(rc == 0, plist->pl_nr == 0));
+ while (plist->pl_nr > 0) {
+ /* error occurred, add the uncommitted pages back into queue */
+ page = cl_page_list_last(plist);
+ cl_page_list_move_head(queue, plist, page);
}
return rc;
-#undef QIN
-}
-
-static int lov_io_prepare_write(const struct lu_env *env,
- const struct cl_io_slice *ios,
- const struct cl_page_slice *slice,
- unsigned from, unsigned to)
-{
- struct lov_io *lio = cl2lov_io(env, ios);
- struct cl_page *sub_page = lov_sub_page(slice);
- struct lov_io_sub *sub;
- int result;
-
- sub = lov_page_subio(env, lio, slice);
- if (!IS_ERR(sub)) {
- result = cl_io_prepare_write(sub->sub_env, sub->sub_io,
- sub_page, from, to);
- lov_sub_put(sub);
- } else
- result = PTR_ERR(sub);
- return result;
-}
-
-static int lov_io_commit_write(const struct lu_env *env,
- const struct cl_io_slice *ios,
- const struct cl_page_slice *slice,
- unsigned from, unsigned to)
-{
- struct lov_io *lio = cl2lov_io(env, ios);
- struct cl_page *sub_page = lov_sub_page(slice);
- struct lov_io_sub *sub;
- int result;
-
- sub = lov_page_subio(env, lio, slice);
- if (!IS_ERR(sub)) {
- result = cl_io_commit_write(sub->sub_env, sub->sub_io,
- sub_page, from, to);
- lov_sub_put(sub);
- } else
- result = PTR_ERR(sub);
- return result;
}
static int lov_io_fault_start(const struct lu_env *env,
@@ -803,16 +794,8 @@
.cio_fini = lov_io_fini
}
},
- .req_op = {
- [CRT_READ] = {
- .cio_submit = lov_io_submit
- },
- [CRT_WRITE] = {
- .cio_submit = lov_io_submit
- }
- },
- .cio_prepare_write = lov_io_prepare_write,
- .cio_commit_write = lov_io_commit_write
+ .cio_submit = lov_io_submit,
+ .cio_commit_async = lov_io_commit_async,
};
/*****************************************************************************
@@ -880,15 +863,8 @@
.cio_fini = lov_empty_io_fini
}
},
- .req_op = {
- [CRT_READ] = {
- .cio_submit = LOV_EMPTY_IMPOSSIBLE
- },
- [CRT_WRITE] = {
- .cio_submit = LOV_EMPTY_IMPOSSIBLE
- }
- },
- .cio_commit_write = LOV_EMPTY_IMPOSSIBLE
+ .cio_submit = LOV_EMPTY_IMPOSSIBLE,
+ .cio_commit_async = LOV_EMPTY_IMPOSSIBLE
};
int lov_io_init_raid0(const struct lu_env *env, struct cl_object *obj,
@@ -943,7 +919,7 @@
}
io->ci_result = result < 0 ? result : 0;
- return result != 0;
+ return result;
}
int lov_io_init_released(const struct lu_env *env, struct cl_object *obj,
@@ -986,7 +962,7 @@
}
io->ci_result = result < 0 ? result : 0;
- return result != 0;
+ return result;
}
/** @} lov */
diff --git a/drivers/staging/lustre/lustre/lov/lov_lock.c b/drivers/staging/lustre/lustre/lov/lov_lock.c
index ae854bc..1b203d1 100644
--- a/drivers/staging/lustre/lustre/lov/lov_lock.c
+++ b/drivers/staging/lustre/lustre/lov/lov_lock.c
@@ -46,11 +46,6 @@
* @{
*/
-static struct cl_lock_closure *lov_closure_get(const struct lu_env *env,
- struct cl_lock *parent);
-
-static int lov_lock_unuse(const struct lu_env *env,
- const struct cl_lock_slice *slice);
/*****************************************************************************
*
* Lov lock operations.
@@ -58,7 +53,7 @@
*/
static struct lov_sublock_env *lov_sublock_env_get(const struct lu_env *env,
- struct cl_lock *parent,
+ const struct cl_lock *parent,
struct lov_lock_sub *lls)
{
struct lov_sublock_env *subenv;
@@ -100,185 +95,26 @@
lov_sub_put(subenv->lse_sub);
}
-static void lov_sublock_adopt(const struct lu_env *env, struct lov_lock *lck,
- struct cl_lock *sublock, int idx,
- struct lov_lock_link *link)
+static int lov_sublock_init(const struct lu_env *env,
+ const struct cl_lock *parent,
+ struct lov_lock_sub *lls)
{
- struct lovsub_lock *lsl;
- struct cl_lock *parent = lck->lls_cl.cls_lock;
- int rc;
+ struct lov_sublock_env *subenv;
+ int result;
- LASSERT(cl_lock_is_mutexed(parent));
- LASSERT(cl_lock_is_mutexed(sublock));
-
- lsl = cl2sub_lock(sublock);
- /*
- * check that sub-lock doesn't have lock link to this top-lock.
- */
- LASSERT(!lov_lock_link_find(env, lck, lsl));
- LASSERT(idx < lck->lls_nr);
-
- lck->lls_sub[idx].sub_lock = lsl;
- lck->lls_nr_filled++;
- LASSERT(lck->lls_nr_filled <= lck->lls_nr);
- list_add_tail(&link->lll_list, &lsl->lss_parents);
- link->lll_idx = idx;
- link->lll_super = lck;
- cl_lock_get(parent);
- lu_ref_add(&parent->cll_reference, "lov-child", sublock);
- lck->lls_sub[idx].sub_flags |= LSF_HELD;
- cl_lock_user_add(env, sublock);
-
- rc = lov_sublock_modify(env, lck, lsl, &sublock->cll_descr, idx);
- LASSERT(rc == 0); /* there is no way this can fail, currently */
-}
-
-static struct cl_lock *lov_sublock_alloc(const struct lu_env *env,
- const struct cl_io *io,
- struct lov_lock *lck,
- int idx, struct lov_lock_link **out)
-{
- struct cl_lock *sublock;
- struct cl_lock *parent;
- struct lov_lock_link *link;
-
- LASSERT(idx < lck->lls_nr);
-
- link = kmem_cache_zalloc(lov_lock_link_kmem, GFP_NOFS);
- if (link) {
- struct lov_sublock_env *subenv;
- struct lov_lock_sub *lls;
- struct cl_lock_descr *descr;
-
- parent = lck->lls_cl.cls_lock;
- lls = &lck->lls_sub[idx];
- descr = &lls->sub_got;
-
- subenv = lov_sublock_env_get(env, parent, lls);
- if (!IS_ERR(subenv)) {
- /* CAVEAT: Don't try to add a field in lov_lock_sub
- * to remember the subio. This is because lock is able
- * to be cached, but this is not true for IO. This
- * further means a sublock might be referenced in
- * different io context. -jay
- */
-
- sublock = cl_lock_hold(subenv->lse_env, subenv->lse_io,
- descr, "lov-parent", parent);
- lov_sublock_env_put(subenv);
- } else {
- /* error occurs. */
- sublock = (void *)subenv;
- }
-
- if (!IS_ERR(sublock))
- *out = link;
- else
- kmem_cache_free(lov_lock_link_kmem, link);
- } else
- sublock = ERR_PTR(-ENOMEM);
- return sublock;
-}
-
-static void lov_sublock_unlock(const struct lu_env *env,
- struct lovsub_lock *lsl,
- struct cl_lock_closure *closure,
- struct lov_sublock_env *subenv)
-{
- lov_sublock_env_put(subenv);
- lsl->lss_active = NULL;
- cl_lock_disclosure(env, closure);
-}
-
-static int lov_sublock_lock(const struct lu_env *env,
- struct lov_lock *lck,
- struct lov_lock_sub *lls,
- struct cl_lock_closure *closure,
- struct lov_sublock_env **lsep)
-{
- struct lovsub_lock *sublock;
- struct cl_lock *child;
- int result = 0;
-
- LASSERT(list_empty(&closure->clc_list));
-
- sublock = lls->sub_lock;
- child = sublock->lss_cl.cls_lock;
- result = cl_lock_closure_build(env, child, closure);
- if (result == 0) {
- struct cl_lock *parent = closure->clc_origin;
-
- LASSERT(cl_lock_is_mutexed(child));
- sublock->lss_active = parent;
-
- if (unlikely((child->cll_state == CLS_FREEING) ||
- (child->cll_flags & CLF_CANCELLED))) {
- struct lov_lock_link *link;
- /*
- * we could race with lock deletion which temporarily
- * put the lock in freeing state, bug 19080.
- */
- LASSERT(!(lls->sub_flags & LSF_HELD));
-
- link = lov_lock_link_find(env, lck, sublock);
- LASSERT(link);
- lov_lock_unlink(env, link, sublock);
- lov_sublock_unlock(env, sublock, closure, NULL);
- lck->lls_cancel_race = 1;
- result = CLO_REPEAT;
- } else if (lsep) {
- struct lov_sublock_env *subenv;
-
- subenv = lov_sublock_env_get(env, parent, lls);
- if (IS_ERR(subenv)) {
- lov_sublock_unlock(env, sublock,
- closure, NULL);
- result = PTR_ERR(subenv);
- } else {
- *lsep = subenv;
- }
- }
+ subenv = lov_sublock_env_get(env, parent, lls);
+ if (!IS_ERR(subenv)) {
+ result = cl_lock_init(subenv->lse_env, &lls->sub_lock,
+ subenv->lse_io);
+ lov_sublock_env_put(subenv);
+ } else {
+ /* error occurs. */
+ result = PTR_ERR(subenv);
}
return result;
}
/**
- * Updates the result of a top-lock operation from a result of sub-lock
- * sub-operations. Top-operations like lov_lock_{enqueue,use,unuse}() iterate
- * over sub-locks and lov_subresult() is used to calculate return value of a
- * top-operation. To this end, possible return values of sub-operations are
- * ordered as
- *
- * - 0 success
- * - CLO_WAIT wait for event
- * - CLO_REPEAT repeat top-operation
- * - -ne fundamental error
- *
- * Top-level return code can only go down through this list. CLO_REPEAT
- * overwrites CLO_WAIT, because lock mutex was released and sleeping condition
- * has to be rechecked by the upper layer.
- */
-static int lov_subresult(int result, int rc)
-{
- int result_rank;
- int rc_rank;
-
- LASSERTF(result <= 0 || result == CLO_REPEAT || result == CLO_WAIT,
- "result = %d\n", result);
- LASSERTF(rc <= 0 || rc == CLO_REPEAT || rc == CLO_WAIT,
- "rc = %d\n", rc);
- CLASSERT(CLO_WAIT < CLO_REPEAT);
-
- /* calculate ranks in the ordering above */
- result_rank = result < 0 ? 1 + CLO_REPEAT : result;
- rc_rank = rc < 0 ? 1 + CLO_REPEAT : rc;
-
- if (result_rank < rc_rank)
- result = rc;
- return result;
-}
-
-/**
* Creates sub-locks for a given lov_lock for the first time.
*
* Goes through all sub-objects of top-object, and creates sub-locks on every
@@ -286,8 +122,9 @@
* fact that top-lock (that is being created) can be accessed concurrently
* through already created sub-locks (possibly shared with other top-locks).
*/
-static int lov_lock_sub_init(const struct lu_env *env,
- struct lov_lock *lck, const struct cl_io *io)
+static struct lov_lock *lov_lock_sub_init(const struct lu_env *env,
+ const struct cl_object *obj,
+ struct cl_lock *lock)
{
int result = 0;
int i;
@@ -297,241 +134,86 @@
u64 file_start;
u64 file_end;
- struct lov_object *loo = cl2lov(lck->lls_cl.cls_obj);
+ struct lov_object *loo = cl2lov(obj);
struct lov_layout_raid0 *r0 = lov_r0(loo);
- struct cl_lock *parent = lck->lls_cl.cls_lock;
+ struct lov_lock *lovlck;
- lck->lls_orig = parent->cll_descr;
- file_start = cl_offset(lov2cl(loo), parent->cll_descr.cld_start);
- file_end = cl_offset(lov2cl(loo), parent->cll_descr.cld_end + 1) - 1;
+ file_start = cl_offset(lov2cl(loo), lock->cll_descr.cld_start);
+ file_end = cl_offset(lov2cl(loo), lock->cll_descr.cld_end + 1) - 1;
for (i = 0, nr = 0; i < r0->lo_nr; i++) {
/*
* XXX for wide striping smarter algorithm is desirable,
* breaking out of the loop, early.
*/
- if (likely(r0->lo_sub[i]) &&
+ if (likely(r0->lo_sub[i]) && /* spare layout */
lov_stripe_intersects(loo->lo_lsm, i,
file_start, file_end, &start, &end))
nr++;
}
LASSERT(nr > 0);
- lck->lls_sub = libcfs_kvzalloc(nr * sizeof(lck->lls_sub[0]), GFP_NOFS);
- if (!lck->lls_sub)
- return -ENOMEM;
+ lovlck = libcfs_kvzalloc(offsetof(struct lov_lock, lls_sub[nr]),
+ GFP_NOFS);
+ if (!lovlck)
+ return ERR_PTR(-ENOMEM);
- lck->lls_nr = nr;
- /*
- * First, fill in sub-lock descriptions in
- * lck->lls_sub[].sub_descr. They are used by lov_sublock_alloc()
- * (called below in this function, and by lov_lock_enqueue()) to
- * create sub-locks. At this moment, no other thread can access
- * top-lock.
- */
+ lovlck->lls_nr = nr;
for (i = 0, nr = 0; i < r0->lo_nr; ++i) {
if (likely(r0->lo_sub[i]) &&
lov_stripe_intersects(loo->lo_lsm, i,
file_start, file_end, &start, &end)) {
+ struct lov_lock_sub *lls = &lovlck->lls_sub[nr];
struct cl_lock_descr *descr;
- descr = &lck->lls_sub[nr].sub_descr;
+ descr = &lls->sub_lock.cll_descr;
LASSERT(!descr->cld_obj);
descr->cld_obj = lovsub2cl(r0->lo_sub[i]);
descr->cld_start = cl_index(descr->cld_obj, start);
descr->cld_end = cl_index(descr->cld_obj, end);
- descr->cld_mode = parent->cll_descr.cld_mode;
- descr->cld_gid = parent->cll_descr.cld_gid;
- descr->cld_enq_flags = parent->cll_descr.cld_enq_flags;
- /* XXX has no effect */
- lck->lls_sub[nr].sub_got = *descr;
- lck->lls_sub[nr].sub_stripe = i;
+ descr->cld_mode = lock->cll_descr.cld_mode;
+ descr->cld_gid = lock->cll_descr.cld_gid;
+ descr->cld_enq_flags = lock->cll_descr.cld_enq_flags;
+ lls->sub_stripe = i;
+
+ /* initialize sub lock */
+ result = lov_sublock_init(env, lock, lls);
+ if (result < 0)
+ break;
+
+ lls->sub_initialized = 1;
nr++;
}
}
- LASSERT(nr == lck->lls_nr);
+ LASSERT(ergo(result == 0, nr == lovlck->lls_nr));
- /*
- * Some sub-locks can be missing at this point. This is not a problem,
- * because enqueue will create them anyway. Main duty of this function
- * is to fill in sub-lock descriptions in a race free manner.
- */
- return result;
-}
+ if (result != 0) {
+ for (i = 0; i < nr; ++i) {
+ if (!lovlck->lls_sub[i].sub_initialized)
+ break;
-static int lov_sublock_release(const struct lu_env *env, struct lov_lock *lck,
- int i, int deluser, int rc)
-{
- struct cl_lock *parent = lck->lls_cl.cls_lock;
-
- LASSERT(cl_lock_is_mutexed(parent));
-
- if (lck->lls_sub[i].sub_flags & LSF_HELD) {
- struct cl_lock *sublock;
- int dying;
-
- sublock = lck->lls_sub[i].sub_lock->lss_cl.cls_lock;
- LASSERT(cl_lock_is_mutexed(sublock));
-
- lck->lls_sub[i].sub_flags &= ~LSF_HELD;
- if (deluser)
- cl_lock_user_del(env, sublock);
- /*
- * If the last hold is released, and cancellation is pending
- * for a sub-lock, release parent mutex, to avoid keeping it
- * while sub-lock is being paged out.
- */
- dying = (sublock->cll_descr.cld_mode == CLM_PHANTOM ||
- sublock->cll_descr.cld_mode == CLM_GROUP ||
- (sublock->cll_flags & (CLF_CANCELPEND|CLF_DOOMED))) &&
- sublock->cll_holds == 1;
- if (dying)
- cl_lock_mutex_put(env, parent);
- cl_lock_unhold(env, sublock, "lov-parent", parent);
- if (dying) {
- cl_lock_mutex_get(env, parent);
- rc = lov_subresult(rc, CLO_REPEAT);
+ cl_lock_fini(env, &lovlck->lls_sub[i].sub_lock);
}
- /*
- * From now on lck->lls_sub[i].sub_lock is a "weak" pointer,
- * not backed by a reference on a
- * sub-lock. lovsub_lock_delete() will clear
- * lck->lls_sub[i].sub_lock under semaphores, just before
- * sub-lock is destroyed.
- */
+ kvfree(lovlck);
+ lovlck = ERR_PTR(result);
}
- return rc;
-}
-static void lov_sublock_hold(const struct lu_env *env, struct lov_lock *lck,
- int i)
-{
- struct cl_lock *parent = lck->lls_cl.cls_lock;
-
- LASSERT(cl_lock_is_mutexed(parent));
-
- if (!(lck->lls_sub[i].sub_flags & LSF_HELD)) {
- struct cl_lock *sublock;
-
- sublock = lck->lls_sub[i].sub_lock->lss_cl.cls_lock;
- LASSERT(cl_lock_is_mutexed(sublock));
- LASSERT(sublock->cll_state != CLS_FREEING);
-
- lck->lls_sub[i].sub_flags |= LSF_HELD;
-
- cl_lock_get_trust(sublock);
- cl_lock_hold_add(env, sublock, "lov-parent", parent);
- cl_lock_user_add(env, sublock);
- cl_lock_put(env, sublock);
- }
+ return lovlck;
}
static void lov_lock_fini(const struct lu_env *env,
struct cl_lock_slice *slice)
{
- struct lov_lock *lck;
+ struct lov_lock *lovlck;
int i;
- lck = cl2lov_lock(slice);
- LASSERT(lck->lls_nr_filled == 0);
- if (lck->lls_sub) {
- for (i = 0; i < lck->lls_nr; ++i)
- /*
- * No sub-locks exists at this point, as sub-lock has
- * a reference on its parent.
- */
- LASSERT(!lck->lls_sub[i].sub_lock);
- kvfree(lck->lls_sub);
+ lovlck = cl2lov_lock(slice);
+ for (i = 0; i < lovlck->lls_nr; ++i) {
+ LASSERT(!lovlck->lls_sub[i].sub_is_enqueued);
+ if (lovlck->lls_sub[i].sub_initialized)
+ cl_lock_fini(env, &lovlck->lls_sub[i].sub_lock);
}
- kmem_cache_free(lov_lock_kmem, lck);
-}
-
-static int lov_lock_enqueue_wait(const struct lu_env *env,
- struct lov_lock *lck,
- struct cl_lock *sublock)
-{
- struct cl_lock *lock = lck->lls_cl.cls_lock;
- int result;
-
- LASSERT(cl_lock_is_mutexed(lock));
-
- cl_lock_mutex_put(env, lock);
- result = cl_lock_enqueue_wait(env, sublock, 0);
- cl_lock_mutex_get(env, lock);
- return result ?: CLO_REPEAT;
-}
-
-/**
- * Tries to advance a state machine of a given sub-lock toward enqueuing of
- * the top-lock.
- *
- * \retval 0 if state-transition can proceed
- * \retval -ve otherwise.
- */
-static int lov_lock_enqueue_one(const struct lu_env *env, struct lov_lock *lck,
- struct cl_lock *sublock,
- struct cl_io *io, __u32 enqflags, int last)
-{
- int result;
-
- /* first, try to enqueue a sub-lock ... */
- result = cl_enqueue_try(env, sublock, io, enqflags);
- if ((sublock->cll_state == CLS_ENQUEUED) && !(enqflags & CEF_AGL)) {
- /* if it is enqueued, try to `wait' on it---maybe it's already
- * granted
- */
- result = cl_wait_try(env, sublock);
- if (result == CLO_REENQUEUED)
- result = CLO_WAIT;
- }
- /*
- * If CEF_ASYNC flag is set, then all sub-locks can be enqueued in
- * parallel, otherwise---enqueue has to wait until sub-lock is granted
- * before proceeding to the next one.
- */
- if ((result == CLO_WAIT) && (sublock->cll_state <= CLS_HELD) &&
- (enqflags & CEF_ASYNC) && (!last || (enqflags & CEF_AGL)))
- result = 0;
- return result;
-}
-
-/**
- * Helper function for lov_lock_enqueue() that creates missing sub-lock.
- */
-static int lov_sublock_fill(const struct lu_env *env, struct cl_lock *parent,
- struct cl_io *io, struct lov_lock *lck, int idx)
-{
- struct lov_lock_link *link = NULL;
- struct cl_lock *sublock;
- int result;
-
- LASSERT(parent->cll_depth == 1);
- cl_lock_mutex_put(env, parent);
- sublock = lov_sublock_alloc(env, io, lck, idx, &link);
- if (!IS_ERR(sublock))
- cl_lock_mutex_get(env, sublock);
- cl_lock_mutex_get(env, parent);
-
- if (!IS_ERR(sublock)) {
- cl_lock_get_trust(sublock);
- if (parent->cll_state == CLS_QUEUING &&
- !lck->lls_sub[idx].sub_lock) {
- lov_sublock_adopt(env, lck, sublock, idx, link);
- } else {
- kmem_cache_free(lov_lock_link_kmem, link);
- /* other thread allocated sub-lock, or enqueue is no
- * longer going on
- */
- cl_lock_mutex_put(env, parent);
- cl_lock_unhold(env, sublock, "lov-parent", parent);
- cl_lock_mutex_get(env, parent);
- }
- cl_lock_mutex_put(env, sublock);
- cl_lock_put(env, sublock);
- result = CLO_REPEAT;
- } else
- result = PTR_ERR(sublock);
- return result;
+ kvfree(lovlck);
}
/**
@@ -543,529 +225,59 @@
*/
static int lov_lock_enqueue(const struct lu_env *env,
const struct cl_lock_slice *slice,
- struct cl_io *io, __u32 enqflags)
+ struct cl_io *io, struct cl_sync_io *anchor)
{
- struct cl_lock *lock = slice->cls_lock;
- struct lov_lock *lck = cl2lov_lock(slice);
- struct cl_lock_closure *closure = lov_closure_get(env, lock);
+ struct cl_lock *lock = slice->cls_lock;
+ struct lov_lock *lovlck = cl2lov_lock(slice);
int i;
- int result;
- enum cl_lock_state minstate;
+ int rc = 0;
- for (result = 0, minstate = CLS_FREEING, i = 0; i < lck->lls_nr; ++i) {
- int rc;
- struct lovsub_lock *sub;
- struct lov_lock_sub *lls;
- struct cl_lock *sublock;
+ for (i = 0; i < lovlck->lls_nr; ++i) {
+ struct lov_lock_sub *lls = &lovlck->lls_sub[i];
struct lov_sublock_env *subenv;
- if (lock->cll_state != CLS_QUEUING) {
- /*
- * Lock might have left QUEUING state if previous
- * iteration released its mutex. Stop enqueing in this
- * case and let the upper layer to decide what to do.
- */
- LASSERT(i > 0 && result != 0);
+ subenv = lov_sublock_env_get(env, lock, lls);
+ if (IS_ERR(subenv)) {
+ rc = PTR_ERR(subenv);
break;
}
-
- lls = &lck->lls_sub[i];
- sub = lls->sub_lock;
- /*
- * Sub-lock might have been canceled, while top-lock was
- * cached.
- */
- if (!sub) {
- result = lov_sublock_fill(env, lock, io, lck, i);
- /* lov_sublock_fill() released @lock mutex,
- * restart.
- */
+ rc = cl_lock_enqueue(subenv->lse_env, subenv->lse_io,
+ &lls->sub_lock, anchor);
+ lov_sublock_env_put(subenv);
+ if (rc != 0)
break;
- }
- sublock = sub->lss_cl.cls_lock;
- rc = lov_sublock_lock(env, lck, lls, closure, &subenv);
- if (rc == 0) {
- lov_sublock_hold(env, lck, i);
- rc = lov_lock_enqueue_one(subenv->lse_env, lck, sublock,
- subenv->lse_io, enqflags,
- i == lck->lls_nr - 1);
- minstate = min(minstate, sublock->cll_state);
- if (rc == CLO_WAIT) {
- switch (sublock->cll_state) {
- case CLS_QUEUING:
- /* take recursive mutex, the lock is
- * released in lov_lock_enqueue_wait.
- */
- cl_lock_mutex_get(env, sublock);
- lov_sublock_unlock(env, sub, closure,
- subenv);
- rc = lov_lock_enqueue_wait(env, lck,
- sublock);
- break;
- case CLS_CACHED:
- cl_lock_get(sublock);
- /* take recursive mutex of sublock */
- cl_lock_mutex_get(env, sublock);
- /* need to release all locks in closure
- * otherwise it may deadlock. LU-2683.
- */
- lov_sublock_unlock(env, sub, closure,
- subenv);
- /* sublock and parent are held. */
- rc = lov_sublock_release(env, lck, i,
- 1, rc);
- cl_lock_mutex_put(env, sublock);
- cl_lock_put(env, sublock);
- break;
- default:
- lov_sublock_unlock(env, sub, closure,
- subenv);
- break;
- }
- } else {
- LASSERT(!sublock->cll_conflict);
- lov_sublock_unlock(env, sub, closure, subenv);
- }
- }
- result = lov_subresult(result, rc);
- if (result != 0)
- break;
+
+ lls->sub_is_enqueued = 1;
}
- cl_lock_closure_fini(closure);
- return result ?: minstate >= CLS_ENQUEUED ? 0 : CLO_WAIT;
-}
-
-static int lov_lock_unuse(const struct lu_env *env,
- const struct cl_lock_slice *slice)
-{
- struct lov_lock *lck = cl2lov_lock(slice);
- struct cl_lock_closure *closure = lov_closure_get(env, slice->cls_lock);
- int i;
- int result;
-
- for (result = 0, i = 0; i < lck->lls_nr; ++i) {
- int rc;
- struct lovsub_lock *sub;
- struct cl_lock *sublock;
- struct lov_lock_sub *lls;
- struct lov_sublock_env *subenv;
-
- /* top-lock state cannot change concurrently, because single
- * thread (one that released the last hold) carries unlocking
- * to the completion.
- */
- LASSERT(slice->cls_lock->cll_state == CLS_INTRANSIT);
- lls = &lck->lls_sub[i];
- sub = lls->sub_lock;
- if (!sub)
- continue;
-
- sublock = sub->lss_cl.cls_lock;
- rc = lov_sublock_lock(env, lck, lls, closure, &subenv);
- if (rc == 0) {
- if (lls->sub_flags & LSF_HELD) {
- LASSERT(sublock->cll_state == CLS_HELD ||
- sublock->cll_state == CLS_ENQUEUED);
- rc = cl_unuse_try(subenv->lse_env, sublock);
- rc = lov_sublock_release(env, lck, i, 0, rc);
- }
- lov_sublock_unlock(env, sub, closure, subenv);
- }
- result = lov_subresult(result, rc);
- }
-
- if (result == 0 && lck->lls_cancel_race) {
- lck->lls_cancel_race = 0;
- result = -ESTALE;
- }
- cl_lock_closure_fini(closure);
- return result;
+ return rc;
}
static void lov_lock_cancel(const struct lu_env *env,
const struct cl_lock_slice *slice)
{
- struct lov_lock *lck = cl2lov_lock(slice);
- struct cl_lock_closure *closure = lov_closure_get(env, slice->cls_lock);
+ struct cl_lock *lock = slice->cls_lock;
+ struct lov_lock *lovlck = cl2lov_lock(slice);
int i;
- int result;
- for (result = 0, i = 0; i < lck->lls_nr; ++i) {
- int rc;
- struct lovsub_lock *sub;
- struct cl_lock *sublock;
- struct lov_lock_sub *lls;
+ for (i = 0; i < lovlck->lls_nr; ++i) {
+ struct lov_lock_sub *lls = &lovlck->lls_sub[i];
+ struct cl_lock *sublock = &lls->sub_lock;
struct lov_sublock_env *subenv;
- /* top-lock state cannot change concurrently, because single
- * thread (one that released the last hold) carries unlocking
- * to the completion.
- */
- lls = &lck->lls_sub[i];
- sub = lls->sub_lock;
- if (!sub)
+ if (!lls->sub_is_enqueued)
continue;
- sublock = sub->lss_cl.cls_lock;
- rc = lov_sublock_lock(env, lck, lls, closure, &subenv);
- if (rc == 0) {
- if (!(lls->sub_flags & LSF_HELD)) {
- lov_sublock_unlock(env, sub, closure, subenv);
- continue;
- }
-
- switch (sublock->cll_state) {
- case CLS_HELD:
- rc = cl_unuse_try(subenv->lse_env, sublock);
- lov_sublock_release(env, lck, i, 0, 0);
- break;
- default:
- lov_sublock_release(env, lck, i, 1, 0);
- break;
- }
- lov_sublock_unlock(env, sub, closure, subenv);
- }
-
- if (rc == CLO_REPEAT) {
- --i;
- continue;
- }
-
- result = lov_subresult(result, rc);
- }
-
- if (result)
- CL_LOCK_DEBUG(D_ERROR, env, slice->cls_lock,
- "lov_lock_cancel fails with %d.\n", result);
-
- cl_lock_closure_fini(closure);
-}
-
-static int lov_lock_wait(const struct lu_env *env,
- const struct cl_lock_slice *slice)
-{
- struct lov_lock *lck = cl2lov_lock(slice);
- struct cl_lock_closure *closure = lov_closure_get(env, slice->cls_lock);
- enum cl_lock_state minstate;
- int reenqueued;
- int result;
- int i;
-
-again:
- for (result = 0, minstate = CLS_FREEING, i = 0, reenqueued = 0;
- i < lck->lls_nr; ++i) {
- int rc;
- struct lovsub_lock *sub;
- struct cl_lock *sublock;
- struct lov_lock_sub *lls;
- struct lov_sublock_env *subenv;
-
- lls = &lck->lls_sub[i];
- sub = lls->sub_lock;
- sublock = sub->lss_cl.cls_lock;
- rc = lov_sublock_lock(env, lck, lls, closure, &subenv);
- if (rc == 0) {
- LASSERT(sublock->cll_state >= CLS_ENQUEUED);
- if (sublock->cll_state < CLS_HELD)
- rc = cl_wait_try(env, sublock);
-
- minstate = min(minstate, sublock->cll_state);
- lov_sublock_unlock(env, sub, closure, subenv);
- }
- if (rc == CLO_REENQUEUED) {
- reenqueued++;
- rc = 0;
- }
- result = lov_subresult(result, rc);
- if (result != 0)
- break;
- }
- /* Each sublock only can be reenqueued once, so will not loop
- * forever.
- */
- if (result == 0 && reenqueued != 0)
- goto again;
- cl_lock_closure_fini(closure);
- return result ?: minstate >= CLS_HELD ? 0 : CLO_WAIT;
-}
-
-static int lov_lock_use(const struct lu_env *env,
- const struct cl_lock_slice *slice)
-{
- struct lov_lock *lck = cl2lov_lock(slice);
- struct cl_lock_closure *closure = lov_closure_get(env, slice->cls_lock);
- int result;
- int i;
-
- LASSERT(slice->cls_lock->cll_state == CLS_INTRANSIT);
-
- for (result = 0, i = 0; i < lck->lls_nr; ++i) {
- int rc;
- struct lovsub_lock *sub;
- struct cl_lock *sublock;
- struct lov_lock_sub *lls;
- struct lov_sublock_env *subenv;
-
- LASSERT(slice->cls_lock->cll_state == CLS_INTRANSIT);
-
- lls = &lck->lls_sub[i];
- sub = lls->sub_lock;
- if (!sub) {
- /*
- * Sub-lock might have been canceled, while top-lock was
- * cached.
- */
- result = -ESTALE;
- break;
- }
-
- sublock = sub->lss_cl.cls_lock;
- rc = lov_sublock_lock(env, lck, lls, closure, &subenv);
- if (rc == 0) {
- LASSERT(sublock->cll_state != CLS_FREEING);
- lov_sublock_hold(env, lck, i);
- if (sublock->cll_state == CLS_CACHED) {
- rc = cl_use_try(subenv->lse_env, sublock, 0);
- if (rc != 0)
- rc = lov_sublock_release(env, lck,
- i, 1, rc);
- } else if (sublock->cll_state == CLS_NEW) {
- /* Sub-lock might have been canceled, while
- * top-lock was cached.
- */
- result = -ESTALE;
- lov_sublock_release(env, lck, i, 1, result);
- }
- lov_sublock_unlock(env, sub, closure, subenv);
- }
- result = lov_subresult(result, rc);
- if (result != 0)
- break;
- }
-
- if (lck->lls_cancel_race) {
- /*
- * If there is unlocking happened at the same time, then
- * sublock_lock state should be FREEING, and lov_sublock_lock
- * should return CLO_REPEAT. In this case, it should return
- * ESTALE, and up layer should reset the lock state to be NEW.
- */
- lck->lls_cancel_race = 0;
- LASSERT(result != 0);
- result = -ESTALE;
- }
- cl_lock_closure_fini(closure);
- return result;
-}
-
-/**
- * Check if the extent region \a descr is covered by \a child against the
- * specific \a stripe.
- */
-static int lov_lock_stripe_is_matching(const struct lu_env *env,
- struct lov_object *lov, int stripe,
- const struct cl_lock_descr *child,
- const struct cl_lock_descr *descr)
-{
- struct lov_stripe_md *lsm = lov->lo_lsm;
- u64 start;
- u64 end;
- int result;
-
- if (lov_r0(lov)->lo_nr == 1)
- return cl_lock_ext_match(child, descr);
-
- /*
- * For a multi-stripes object:
- * - make sure the descr only covers child's stripe, and
- * - check if extent is matching.
- */
- start = cl_offset(&lov->lo_cl, descr->cld_start);
- end = cl_offset(&lov->lo_cl, descr->cld_end + 1) - 1;
- result = 0;
- /* glimpse should work on the object with LOV EA hole. */
- if (end - start <= lsm->lsm_stripe_size) {
- int idx;
-
- idx = lov_stripe_number(lsm, start);
- if (idx == stripe ||
- unlikely(!lov_r0(lov)->lo_sub[idx])) {
- idx = lov_stripe_number(lsm, end);
- if (idx == stripe ||
- unlikely(!lov_r0(lov)->lo_sub[idx]))
- result = 1;
+ lls->sub_is_enqueued = 0;
+ subenv = lov_sublock_env_get(env, lock, lls);
+ if (!IS_ERR(subenv)) {
+ cl_lock_cancel(subenv->lse_env, sublock);
+ lov_sublock_env_put(subenv);
+ } else {
+ CL_LOCK_DEBUG(D_ERROR, env, slice->cls_lock,
+ "lov_lock_cancel fails with %ld.\n",
+ PTR_ERR(subenv));
}
}
-
- if (result != 0) {
- struct cl_lock_descr *subd = &lov_env_info(env)->lti_ldescr;
- u64 sub_start;
- u64 sub_end;
-
- subd->cld_obj = NULL; /* don't need sub object at all */
- subd->cld_mode = descr->cld_mode;
- subd->cld_gid = descr->cld_gid;
- result = lov_stripe_intersects(lsm, stripe, start, end,
- &sub_start, &sub_end);
- LASSERT(result);
- subd->cld_start = cl_index(child->cld_obj, sub_start);
- subd->cld_end = cl_index(child->cld_obj, sub_end);
- result = cl_lock_ext_match(child, subd);
- }
- return result;
-}
-
-/**
- * An implementation of cl_lock_operations::clo_fits_into() method.
- *
- * Checks whether a lock (given by \a slice) is suitable for \a
- * io. Multi-stripe locks can be used only for "quick" io, like truncate, or
- * O_APPEND write.
- *
- * \see ccc_lock_fits_into().
- */
-static int lov_lock_fits_into(const struct lu_env *env,
- const struct cl_lock_slice *slice,
- const struct cl_lock_descr *need,
- const struct cl_io *io)
-{
- struct lov_lock *lov = cl2lov_lock(slice);
- struct lov_object *obj = cl2lov(slice->cls_obj);
- int result;
-
- LASSERT(cl_object_same(need->cld_obj, slice->cls_obj));
- LASSERT(lov->lls_nr > 0);
-
- /* for top lock, it's necessary to match enq flags otherwise it will
- * run into problem if a sublock is missing and reenqueue.
- */
- if (need->cld_enq_flags != lov->lls_orig.cld_enq_flags)
- return 0;
-
- if (need->cld_mode == CLM_GROUP)
- /*
- * always allow to match group lock.
- */
- result = cl_lock_ext_match(&lov->lls_orig, need);
- else if (lov->lls_nr == 1) {
- struct cl_lock_descr *got = &lov->lls_sub[0].sub_got;
-
- result = lov_lock_stripe_is_matching(env,
- cl2lov(slice->cls_obj),
- lov->lls_sub[0].sub_stripe,
- got, need);
- } else if (io->ci_type != CIT_SETATTR && io->ci_type != CIT_MISC &&
- !cl_io_is_append(io) && need->cld_mode != CLM_PHANTOM)
- /*
- * Multi-stripe locks are only suitable for `quick' IO and for
- * glimpse.
- */
- result = 0;
- else
- /*
- * Most general case: multi-stripe existing lock, and
- * (potentially) multi-stripe @need lock. Check that @need is
- * covered by @lov's sub-locks.
- *
- * For now, ignore lock expansions made by the server, and
- * match against original lock extent.
- */
- result = cl_lock_ext_match(&lov->lls_orig, need);
- CDEBUG(D_DLMTRACE, DDESCR"/"DDESCR" %d %d/%d: %d\n",
- PDESCR(&lov->lls_orig), PDESCR(&lov->lls_sub[0].sub_got),
- lov->lls_sub[0].sub_stripe, lov->lls_nr, lov_r0(obj)->lo_nr,
- result);
- return result;
-}
-
-void lov_lock_unlink(const struct lu_env *env,
- struct lov_lock_link *link, struct lovsub_lock *sub)
-{
- struct lov_lock *lck = link->lll_super;
- struct cl_lock *parent = lck->lls_cl.cls_lock;
-
- LASSERT(cl_lock_is_mutexed(parent));
- LASSERT(cl_lock_is_mutexed(sub->lss_cl.cls_lock));
-
- list_del_init(&link->lll_list);
- LASSERT(lck->lls_sub[link->lll_idx].sub_lock == sub);
- /* yank this sub-lock from parent's array */
- lck->lls_sub[link->lll_idx].sub_lock = NULL;
- LASSERT(lck->lls_nr_filled > 0);
- lck->lls_nr_filled--;
- lu_ref_del(&parent->cll_reference, "lov-child", sub->lss_cl.cls_lock);
- cl_lock_put(env, parent);
- kmem_cache_free(lov_lock_link_kmem, link);
-}
-
-struct lov_lock_link *lov_lock_link_find(const struct lu_env *env,
- struct lov_lock *lck,
- struct lovsub_lock *sub)
-{
- struct lov_lock_link *scan;
-
- LASSERT(cl_lock_is_mutexed(sub->lss_cl.cls_lock));
-
- list_for_each_entry(scan, &sub->lss_parents, lll_list) {
- if (scan->lll_super == lck)
- return scan;
- }
- return NULL;
-}
-
-/**
- * An implementation of cl_lock_operations::clo_delete() method. This is
- * invoked for "top-to-bottom" delete, when lock destruction starts from the
- * top-lock, e.g., as a result of inode destruction.
- *
- * Unlinks top-lock from all its sub-locks. Sub-locks are not deleted there:
- * this is done separately elsewhere:
- *
- * - for inode destruction, lov_object_delete() calls cl_object_kill() for
- * each sub-object, purging its locks;
- *
- * - in other cases (e.g., a fatal error with a top-lock) sub-locks are
- * left in the cache.
- */
-static void lov_lock_delete(const struct lu_env *env,
- const struct cl_lock_slice *slice)
-{
- struct lov_lock *lck = cl2lov_lock(slice);
- struct cl_lock_closure *closure = lov_closure_get(env, slice->cls_lock);
- struct lov_lock_link *link;
- int rc;
- int i;
-
- LASSERT(slice->cls_lock->cll_state == CLS_FREEING);
-
- for (i = 0; i < lck->lls_nr; ++i) {
- struct lov_lock_sub *lls = &lck->lls_sub[i];
- struct lovsub_lock *lsl = lls->sub_lock;
-
- if (!lsl) /* already removed */
- continue;
-
- rc = lov_sublock_lock(env, lck, lls, closure, NULL);
- if (rc == CLO_REPEAT) {
- --i;
- continue;
- }
-
- LASSERT(rc == 0);
- LASSERT(lsl->lss_cl.cls_lock->cll_state < CLS_FREEING);
-
- if (lls->sub_flags & LSF_HELD)
- lov_sublock_release(env, lck, i, 1, 0);
-
- link = lov_lock_link_find(env, lck, lsl);
- LASSERT(link);
- lov_lock_unlink(env, link, lsl);
- LASSERT(!lck->lls_sub[i].sub_lock);
-
- lov_sublock_unlock(env, lsl, closure, NULL);
- }
-
- cl_lock_closure_fini(closure);
}
static int lov_lock_print(const struct lu_env *env, void *cookie,
@@ -1079,12 +291,8 @@
struct lov_lock_sub *sub;
sub = &lck->lls_sub[i];
- (*p)(env, cookie, " %d %x: ", i, sub->sub_flags);
- if (sub->sub_lock)
- cl_lock_print(env, cookie, p,
- sub->sub_lock->lss_cl.cls_lock);
- else
- (*p)(env, cookie, "---\n");
+ (*p)(env, cookie, " %d %x: ", i, sub->sub_is_enqueued);
+ cl_lock_print(env, cookie, p, &sub->sub_lock);
}
return 0;
}
@@ -1092,12 +300,7 @@
static const struct cl_lock_operations lov_lock_ops = {
.clo_fini = lov_lock_fini,
.clo_enqueue = lov_lock_enqueue,
- .clo_wait = lov_lock_wait,
- .clo_use = lov_lock_use,
- .clo_unuse = lov_lock_unuse,
.clo_cancel = lov_lock_cancel,
- .clo_fits_into = lov_lock_fits_into,
- .clo_delete = lov_lock_delete,
.clo_print = lov_lock_print
};
@@ -1105,14 +308,13 @@
struct cl_lock *lock, const struct cl_io *io)
{
struct lov_lock *lck;
- int result;
+ int result = 0;
- lck = kmem_cache_zalloc(lov_lock_kmem, GFP_NOFS);
- if (lck) {
+ lck = lov_lock_sub_init(env, obj, lock);
+ if (!IS_ERR(lck))
cl_lock_slice_add(lock, &lck->lls_cl, obj, &lov_lock_ops);
- result = lov_lock_sub_init(env, lck, io);
- } else
- result = -ENOMEM;
+ else
+ result = PTR_ERR(lck);
return result;
}
@@ -1147,21 +349,9 @@
lck = kmem_cache_zalloc(lov_lock_kmem, GFP_NOFS);
if (lck) {
cl_lock_slice_add(lock, &lck->lls_cl, obj, &lov_empty_lock_ops);
- lck->lls_orig = lock->cll_descr;
result = 0;
}
return result;
}
-static struct cl_lock_closure *lov_closure_get(const struct lu_env *env,
- struct cl_lock *parent)
-{
- struct cl_lock_closure *closure;
-
- closure = &lov_env_info(env)->lti_closure;
- LASSERT(list_empty(&closure->clc_list));
- cl_lock_closure_init(env, closure, parent, 1);
- return closure;
-}
-
/** @} lov */
diff --git a/drivers/staging/lustre/lustre/lov/lov_merge.c b/drivers/staging/lustre/lustre/lov/lov_merge.c
index 029cd4d..56ef41d 100644
--- a/drivers/staging/lustre/lustre/lov/lov_merge.c
+++ b/drivers/staging/lustre/lustre/lov/lov_merge.c
@@ -154,6 +154,7 @@
valid &= src->o_valid;
if (*set) {
+ tgt->o_valid &= valid;
if (valid & OBD_MD_FLSIZE) {
/* this handles sparse files properly */
u64 lov_size;
@@ -172,12 +173,22 @@
tgt->o_mtime = src->o_mtime;
if (valid & OBD_MD_FLDATAVERSION)
tgt->o_data_version += src->o_data_version;
+
+ /* handle flags */
+ if (valid & OBD_MD_FLFLAGS)
+ tgt->o_flags &= src->o_flags;
+ else
+ tgt->o_flags = 0;
} else {
memcpy(tgt, src, sizeof(*tgt));
tgt->o_oi = lsm->lsm_oi;
+ tgt->o_valid = valid;
if (valid & OBD_MD_FLSIZE)
tgt->o_size = lov_stripe_size(lsm, src->o_size,
stripeno);
+ tgt->o_flags = 0;
+ if (valid & OBD_MD_FLFLAGS)
+ tgt->o_flags = src->o_flags;
}
/* data_version needs to be valid on all stripes to be correct! */
diff --git a/drivers/staging/lustre/lustre/lov/lov_obd.c b/drivers/staging/lustre/lustre/lov/lov_obd.c
index 5daa7fa..e15ef2e 100644
--- a/drivers/staging/lustre/lustre/lov/lov_obd.c
+++ b/drivers/staging/lustre/lustre/lov/lov_obd.c
@@ -54,7 +54,6 @@
#include "../include/lprocfs_status.h"
#include "../include/lustre_param.h"
#include "../include/cl_object.h"
-#include "../include/lclient.h" /* for cl_client_lru */
#include "../include/lustre/ll_fiemap.h"
#include "../include/lustre_fid.h"
@@ -124,7 +123,6 @@
static int lov_notify(struct obd_device *obd, struct obd_device *watched,
enum obd_notify_event ev, void *data);
-#define MAX_STRING_SIZE 128
int lov_connect_obd(struct obd_device *obd, __u32 index, int activate,
struct obd_connect_data *data)
{
@@ -965,7 +963,6 @@
CERROR("Unknown command: %d\n", lcfg->lcfg_command);
rc = -EINVAL;
goto out;
-
}
}
out:
@@ -1734,6 +1731,27 @@
unsigned int buffer_size = FIEMAP_BUFFER_SIZE;
if (!lsm_has_objects(lsm)) {
+ if (lsm && lsm_is_released(lsm) && (fm_key->fiemap.fm_start <
+ fm_key->oa.o_size)) {
+ /*
+ * released file, return a minimal FIEMAP if
+ * request fits in file-size.
+ */
+ fiemap->fm_mapped_extents = 1;
+ fiemap->fm_extents[0].fe_logical =
+ fm_key->fiemap.fm_start;
+ if (fm_key->fiemap.fm_start + fm_key->fiemap.fm_length <
+ fm_key->oa.o_size) {
+ fiemap->fm_extents[0].fe_length =
+ fm_key->fiemap.fm_length;
+ } else {
+ fiemap->fm_extents[0].fe_length =
+ fm_key->oa.o_size - fm_key->fiemap.fm_start;
+ fiemap->fm_extents[0].fe_flags |=
+ (FIEMAP_EXTENT_UNKNOWN |
+ FIEMAP_EXTENT_LAST);
+ }
+ }
rc = 0;
goto out;
}
@@ -2173,7 +2191,6 @@
LASSERT(md->lsm_lock_owner == 0);
md->lsm_lock_owner = current_pid();
}
-EXPORT_SYMBOL(lov_stripe_lock);
void lov_stripe_unlock(struct lov_stripe_md *md)
__releases(&md->lsm_lock)
@@ -2182,7 +2199,6 @@
md->lsm_lock_owner = 0;
spin_unlock(&md->lsm_lock);
}
-EXPORT_SYMBOL(lov_stripe_unlock);
static int lov_quotactl(struct obd_device *obd, struct obd_export *exp,
struct obd_quotactl *oqctl)
diff --git a/drivers/staging/lustre/lustre/lov/lov_object.c b/drivers/staging/lustre/lustre/lov/lov_object.c
index 1f8ed95..561d493 100644
--- a/drivers/staging/lustre/lustre/lov/lov_object.c
+++ b/drivers/staging/lustre/lustre/lov/lov_object.c
@@ -67,7 +67,7 @@
int (*llo_print)(const struct lu_env *env, void *cookie,
lu_printer_t p, const struct lu_object *o);
int (*llo_page_init)(const struct lu_env *env, struct cl_object *obj,
- struct cl_page *page, struct page *vmpage);
+ struct cl_page *page, pgoff_t index);
int (*llo_lock_init)(const struct lu_env *env,
struct cl_object *obj, struct cl_lock *lock,
const struct cl_io *io);
@@ -193,6 +193,18 @@
return result;
}
+static int lov_page_slice_fixup(struct lov_object *lov,
+ struct cl_object *stripe)
+{
+ struct cl_object_header *hdr = cl_object_header(&lov->lo_cl);
+ struct cl_object *o;
+
+ cl_object_for_each(o, stripe)
+ o->co_slice_off += hdr->coh_page_bufsize;
+
+ return cl_object_header(stripe)->coh_page_bufsize;
+}
+
static int lov_init_raid0(const struct lu_env *env,
struct lov_device *dev, struct lov_object *lov,
const struct cl_object_conf *conf,
@@ -222,6 +234,8 @@
r0->lo_sub = libcfs_kvzalloc(r0->lo_nr * sizeof(r0->lo_sub[0]),
GFP_NOFS);
if (r0->lo_sub) {
+ int psz = 0;
+
result = 0;
subconf->coc_inode = conf->coc_inode;
spin_lock_init(&r0->lo_sub_lock);
@@ -254,13 +268,24 @@
if (result == -EAGAIN) { /* try again */
--i;
result = 0;
+ continue;
}
} else {
result = PTR_ERR(stripe);
}
+
+ if (result == 0) {
+ int sz = lov_page_slice_fixup(lov, stripe);
+
+ LASSERT(ergo(psz > 0, psz == sz));
+ psz = sz;
+ }
}
- } else
+ if (result == 0)
+ cl_object_header(&lov->lo_cl)->coh_page_bufsize += psz;
+ } else {
result = -ENOMEM;
+ }
out:
return result;
}
@@ -286,8 +311,6 @@
LASSERT(lov->lo_type == LLT_EMPTY || lov->lo_type == LLT_RELEASED);
lov_layout_wait(env, lov);
-
- cl_object_prune(env, &lov->lo_cl);
return 0;
}
@@ -355,7 +378,7 @@
struct lovsub_object *los = r0->lo_sub[i];
if (los) {
- cl_locks_prune(env, &los->lso_cl, 1);
+ cl_object_prune(env, &los->lso_cl);
/*
* If top-level object is to be evicted from
* the cache, so are its sub-objects.
@@ -364,7 +387,6 @@
}
}
}
- cl_object_prune(env, &lov->lo_cl);
return 0;
}
@@ -666,7 +688,6 @@
const struct lov_layout_operations *old_ops;
const struct lov_layout_operations *new_ops;
- struct cl_object_header *hdr = cl_object_header(&lov->lo_cl);
void *cookie;
struct lu_env *env;
int refcheck;
@@ -691,13 +712,15 @@
old_ops = &lov_dispatch[lov->lo_type];
new_ops = &lov_dispatch[llt];
+ result = cl_object_prune(env, &lov->lo_cl);
+ if (result != 0)
+ goto out;
+
result = old_ops->llo_delete(env, lov, &lov->u);
if (result == 0) {
old_ops->llo_fini(env, lov, &lov->u);
LASSERT(atomic_read(&lov->lo_active_ios) == 0);
- LASSERT(!hdr->coh_tree.rnode);
- LASSERT(hdr->coh_pages == 0);
lov->lo_type = LLT_EMPTY;
result = new_ops->llo_init(env,
@@ -713,6 +736,7 @@
}
}
+out:
cl_env_put(env, &refcheck);
cl_env_reexit(cookie);
return result;
@@ -793,7 +817,8 @@
goto out;
}
- lov->lo_layout_invalid = lov_layout_change(env, lov, conf);
+ result = lov_layout_change(env, lov, conf);
+ lov->lo_layout_invalid = result != 0;
out:
lov_conf_unlock(lov);
@@ -825,10 +850,10 @@
}
int lov_page_init(const struct lu_env *env, struct cl_object *obj,
- struct cl_page *page, struct page *vmpage)
+ struct cl_page *page, pgoff_t index)
{
- return LOV_2DISPATCH_NOLOCK(cl2lov(obj),
- llo_page_init, env, obj, page, vmpage);
+ return LOV_2DISPATCH_NOLOCK(cl2lov(obj), llo_page_init, env, obj, page,
+ index);
}
/**
@@ -911,8 +936,9 @@
* for object with different layouts.
*/
obj->lo_ops = &lov_lu_obj_ops;
- } else
+ } else {
obj = NULL;
+ }
return obj;
}
diff --git a/drivers/staging/lustre/lustre/lov/lov_offset.c b/drivers/staging/lustre/lustre/lov/lov_offset.c
index ae83eb0..9302f06 100644
--- a/drivers/staging/lustre/lustre/lov/lov_offset.c
+++ b/drivers/staging/lustre/lustre/lov/lov_offset.c
@@ -66,6 +66,18 @@
return lov_size;
}
+/**
+ * Compute file level page index by stripe level page offset
+ */
+pgoff_t lov_stripe_pgoff(struct lov_stripe_md *lsm, pgoff_t stripe_index,
+ int stripe)
+{
+ loff_t offset;
+
+ offset = lov_stripe_size(lsm, stripe_index << PAGE_SHIFT, stripe);
+ return offset >> PAGE_SHIFT;
+}
+
/* we have an offset in file backed by an lov and want to find out where
* that offset lands in our given stripe of the file. for the easy
* case where the offset is within the stripe, we just have to scale the
diff --git a/drivers/staging/lustre/lustre/lov/lov_pack.c b/drivers/staging/lustre/lustre/lov/lov_pack.c
index 3925633..0215ea5 100644
--- a/drivers/staging/lustre/lustre/lov/lov_pack.c
+++ b/drivers/staging/lustre/lustre/lov/lov_pack.c
@@ -136,7 +136,6 @@
CERROR("bad mem LOV MAGIC: 0x%08X != 0x%08X nor 0x%08X\n",
lmm_magic, LOV_MAGIC_V1, LOV_MAGIC_V3);
return -EINVAL;
-
}
if (lsm) {
@@ -444,8 +443,7 @@
if (lum.lmm_magic == LOV_USER_MAGIC) {
/* User request for v1, we need skip lmm_pool_name */
if (lmmk->lmm_magic == LOV_MAGIC_V3) {
- memmove((char *)(&lmmk->lmm_stripe_count) +
- sizeof(lmmk->lmm_stripe_count),
+ memmove(((struct lov_mds_md_v1 *)lmmk)->lmm_objects,
((struct lov_mds_md_v3 *)lmmk)->lmm_objects,
lmmk->lmm_stripe_count *
sizeof(struct lov_ost_data_v1));
@@ -457,9 +455,9 @@
}
/* User wasn't expecting this many OST entries */
- if (lum.lmm_stripe_count == 0)
+ if (lum.lmm_stripe_count == 0) {
lmm_size = lum_size;
- else if (lum.lmm_stripe_count < lmmk->lmm_stripe_count) {
+ } else if (lum.lmm_stripe_count < lmmk->lmm_stripe_count) {
rc = -EOVERFLOW;
goto out_set;
}
diff --git a/drivers/staging/lustre/lustre/lov/lov_page.c b/drivers/staging/lustre/lustre/lov/lov_page.c
index fdcaf80..0306f00 100644
--- a/drivers/staging/lustre/lustre/lov/lov_page.c
+++ b/drivers/staging/lustre/lustre/lov/lov_page.c
@@ -36,6 +36,7 @@
* Implementation of cl_page for LOV layer.
*
* Author: Nikita Danilov <nikita.danilov@sun.com>
+ * Author: Jinshan Xiong <jinshan.xiong@intel.com>
*/
#define DEBUG_SUBSYSTEM S_LOV
@@ -52,116 +53,66 @@
*
*/
-static int lov_page_invariant(const struct cl_page_slice *slice)
+/**
+ * Adjust the stripe index by layout of raid0. @max_index is the maximum
+ * page index covered by an underlying DLM lock.
+ * This function converts max_index from stripe level to file level, and make
+ * sure it's not beyond one stripe.
+ */
+static int lov_raid0_page_is_under_lock(const struct lu_env *env,
+ const struct cl_page_slice *slice,
+ struct cl_io *unused,
+ pgoff_t *max_index)
{
- const struct cl_page *page = slice->cpl_page;
- const struct cl_page *sub = lov_sub_page(slice);
+ struct lov_object *loo = cl2lov(slice->cpl_obj);
+ struct lov_layout_raid0 *r0 = lov_r0(loo);
+ pgoff_t index = *max_index;
+ unsigned int pps; /* pages per stripe */
- return ergo(sub,
- page->cp_child == sub &&
- sub->cp_parent == page &&
- page->cp_state == sub->cp_state);
-}
+ CDEBUG(D_READA, "*max_index = %lu, nr = %d\n", index, r0->lo_nr);
+ if (index == 0) /* the page is not covered by any lock */
+ return 0;
-static void lov_page_fini(const struct lu_env *env,
- struct cl_page_slice *slice)
-{
- struct cl_page *sub = lov_sub_page(slice);
+ if (r0->lo_nr == 1) /* single stripe file */
+ return 0;
- LINVRNT(lov_page_invariant(slice));
-
- if (sub) {
- LASSERT(sub->cp_state == CPS_FREEING);
- lu_ref_del(&sub->cp_reference, "lov", sub->cp_parent);
- sub->cp_parent = NULL;
- slice->cpl_page->cp_child = NULL;
- cl_page_put(env, sub);
+ /* max_index is stripe level, convert it into file level */
+ if (index != CL_PAGE_EOF) {
+ int stripeno = lov_page_stripe(slice->cpl_page);
+ *max_index = lov_stripe_pgoff(loo->lo_lsm, index, stripeno);
}
-}
-static int lov_page_own(const struct lu_env *env,
- const struct cl_page_slice *slice, struct cl_io *io,
- int nonblock)
-{
- struct lov_io *lio = lov_env_io(env);
- struct lov_io_sub *sub;
+ /* calculate the end of current stripe */
+ pps = loo->lo_lsm->lsm_stripe_size >> PAGE_SHIFT;
+ index = ((slice->cpl_index + pps) & ~(pps - 1)) - 1;
- LINVRNT(lov_page_invariant(slice));
- LINVRNT(!cl2lov_page(slice)->lps_invalid);
-
- sub = lov_page_subio(env, lio, slice);
- if (!IS_ERR(sub)) {
- lov_sub_page(slice)->cp_owner = sub->sub_io;
- lov_sub_put(sub);
- } else
- LBUG(); /* Arrgh */
+ /* never exceed the end of the stripe */
+ *max_index = min_t(pgoff_t, *max_index, index);
return 0;
}
-static void lov_page_assume(const struct lu_env *env,
- const struct cl_page_slice *slice, struct cl_io *io)
-{
- lov_page_own(env, slice, io, 0);
-}
-
-static int lov_page_cache_add(const struct lu_env *env,
- const struct cl_page_slice *slice,
- struct cl_io *io)
-{
- struct lov_io *lio = lov_env_io(env);
- struct lov_io_sub *sub;
- int rc = 0;
-
- LINVRNT(lov_page_invariant(slice));
- LINVRNT(!cl2lov_page(slice)->lps_invalid);
-
- sub = lov_page_subio(env, lio, slice);
- if (!IS_ERR(sub)) {
- rc = cl_page_cache_add(sub->sub_env, sub->sub_io,
- slice->cpl_page->cp_child, CRT_WRITE);
- lov_sub_put(sub);
- } else {
- rc = PTR_ERR(sub);
- CL_PAGE_DEBUG(D_ERROR, env, slice->cpl_page, "rc = %d\n", rc);
- }
- return rc;
-}
-
-static int lov_page_print(const struct lu_env *env,
- const struct cl_page_slice *slice,
- void *cookie, lu_printer_t printer)
+static int lov_raid0_page_print(const struct lu_env *env,
+ const struct cl_page_slice *slice,
+ void *cookie, lu_printer_t printer)
{
struct lov_page *lp = cl2lov_page(slice);
- return (*printer)(env, cookie, LUSTRE_LOV_NAME"-page@%p\n", lp);
+ return (*printer)(env, cookie, LUSTRE_LOV_NAME "-page@%p, raid0\n", lp);
}
-static const struct cl_page_operations lov_page_ops = {
- .cpo_fini = lov_page_fini,
- .cpo_own = lov_page_own,
- .cpo_assume = lov_page_assume,
- .io = {
- [CRT_WRITE] = {
- .cpo_cache_add = lov_page_cache_add
- }
- },
- .cpo_print = lov_page_print
+static const struct cl_page_operations lov_raid0_page_ops = {
+ .cpo_is_under_lock = lov_raid0_page_is_under_lock,
+ .cpo_print = lov_raid0_page_print
};
-static void lov_empty_page_fini(const struct lu_env *env,
- struct cl_page_slice *slice)
-{
- LASSERT(!slice->cpl_page->cp_child);
-}
-
int lov_page_init_raid0(const struct lu_env *env, struct cl_object *obj,
- struct cl_page *page, struct page *vmpage)
+ struct cl_page *page, pgoff_t index)
{
struct lov_object *loo = cl2lov(obj);
struct lov_layout_raid0 *r0 = lov_r0(loo);
struct lov_io *lio = lov_env_io(env);
- struct cl_page *subpage;
struct cl_object *subobj;
+ struct cl_object *o;
struct lov_io_sub *sub;
struct lov_page *lpg = cl_object_page_slice(obj, page);
loff_t offset;
@@ -169,59 +120,57 @@
int stripe;
int rc;
- offset = cl_offset(obj, page->cp_index);
+ offset = cl_offset(obj, index);
stripe = lov_stripe_number(loo->lo_lsm, offset);
LASSERT(stripe < r0->lo_nr);
rc = lov_stripe_offset(loo->lo_lsm, offset, stripe, &suboff);
LASSERT(rc == 0);
- lpg->lps_invalid = 1;
- cl_page_slice_add(page, &lpg->lps_cl, obj, &lov_page_ops);
+ cl_page_slice_add(page, &lpg->lps_cl, obj, index, &lov_raid0_page_ops);
sub = lov_sub_get(env, lio, stripe);
- if (IS_ERR(sub)) {
- rc = PTR_ERR(sub);
- goto out;
- }
+ if (IS_ERR(sub))
+ return PTR_ERR(sub);
subobj = lovsub2cl(r0->lo_sub[stripe]);
- subpage = cl_page_find_sub(sub->sub_env, subobj,
- cl_index(subobj, suboff), vmpage, page);
+ list_for_each_entry(o, &subobj->co_lu.lo_header->loh_layers,
+ co_lu.lo_linkage) {
+ if (o->co_ops->coo_page_init) {
+ rc = o->co_ops->coo_page_init(sub->sub_env, o, page,
+ cl_index(subobj, suboff));
+ if (rc != 0)
+ break;
+ }
+ }
lov_sub_put(sub);
- if (IS_ERR(subpage)) {
- rc = PTR_ERR(subpage);
- goto out;
- }
- if (likely(subpage->cp_parent == page)) {
- lu_ref_add(&subpage->cp_reference, "lov", page);
- lpg->lps_invalid = 0;
- rc = 0;
- } else {
- CL_PAGE_DEBUG(D_ERROR, env, page, "parent page\n");
- CL_PAGE_DEBUG(D_ERROR, env, subpage, "child page\n");
- LASSERT(0);
- }
-
-out:
return rc;
}
+static int lov_empty_page_print(const struct lu_env *env,
+ const struct cl_page_slice *slice,
+ void *cookie, lu_printer_t printer)
+{
+ struct lov_page *lp = cl2lov_page(slice);
+
+ return (*printer)(env, cookie, LUSTRE_LOV_NAME "-page@%p, empty.\n",
+ lp);
+}
+
static const struct cl_page_operations lov_empty_page_ops = {
- .cpo_fini = lov_empty_page_fini,
- .cpo_print = lov_page_print
+ .cpo_print = lov_empty_page_print
};
int lov_page_init_empty(const struct lu_env *env, struct cl_object *obj,
- struct cl_page *page, struct page *vmpage)
+ struct cl_page *page, pgoff_t index)
{
struct lov_page *lpg = cl_object_page_slice(obj, page);
void *addr;
- cl_page_slice_add(page, &lpg->lps_cl, obj, &lov_empty_page_ops);
- addr = kmap(vmpage);
+ cl_page_slice_add(page, &lpg->lps_cl, obj, index, &lov_empty_page_ops);
+ addr = kmap(page->cp_vmpage);
memset(addr, 0, cl_page_size(obj));
- kunmap(vmpage);
+ kunmap(page->cp_vmpage);
cl_page_export(env, page, 1);
return 0;
}
diff --git a/drivers/staging/lustre/lustre/lov/lov_pool.c b/drivers/staging/lustre/lustre/lov/lov_pool.c
index 9ae1d6f..690292e 100644
--- a/drivers/staging/lustre/lustre/lov/lov_pool.c
+++ b/drivers/staging/lustre/lustre/lov/lov_pool.c
@@ -65,7 +65,6 @@
LASSERT(hlist_unhashed(&pool->pool_hash));
LASSERT(list_empty(&pool->pool_list));
LASSERT(!pool->pool_debugfs_entry);
- lov_ost_pool_free(&(pool->pool_rr.lqr_pool));
lov_ost_pool_free(&(pool->pool_obds));
kfree(pool);
}
@@ -424,11 +423,6 @@
if (rc)
goto out_err;
- memset(&(new_pool->pool_rr), 0, sizeof(struct lov_qos_rr));
- rc = lov_ost_pool_init(&new_pool->pool_rr.lqr_pool, 0);
- if (rc)
- goto out_free_pool_obds;
-
INIT_HLIST_NODE(&new_pool->pool_hash);
/* get ref for debugfs file */
@@ -469,13 +463,10 @@
list_del_init(&new_pool->pool_list);
lov->lov_pool_count--;
spin_unlock(&obd->obd_dev_lock);
-
ldebugfs_remove(&new_pool->pool_debugfs_entry);
-
- lov_ost_pool_free(&new_pool->pool_rr.lqr_pool);
-out_free_pool_obds:
lov_ost_pool_free(&new_pool->pool_obds);
kfree(new_pool);
+
return rc;
}
@@ -543,8 +534,6 @@
if (rc)
goto out;
- pool->pool_rr.lqr_dirty = 1;
-
CDEBUG(D_CONFIG, "Added %s to "LOV_POOLNAMEF" as member %d\n",
ostname, poolname, pool_tgt_count(pool));
@@ -589,8 +578,6 @@
lov_ost_pool_remove(&pool->pool_obds, lov_idx);
- pool->pool_rr.lqr_dirty = 1;
-
CDEBUG(D_CONFIG, "%s removed from "LOV_POOLNAMEF"\n", ostname,
poolname);
@@ -599,50 +586,3 @@
lov_pool_putref(pool);
return rc;
}
-
-int lov_check_index_in_pool(__u32 idx, struct pool_desc *pool)
-{
- int i, rc;
-
- /* caller may no have a ref on pool if it got the pool
- * without calling lov_find_pool() (e.g. go through the lov pool
- * list)
- */
- lov_pool_getref(pool);
-
- down_read(&pool_tgt_rw_sem(pool));
-
- for (i = 0; i < pool_tgt_count(pool); i++) {
- if (pool_tgt_array(pool)[i] == idx) {
- rc = 0;
- goto out;
- }
- }
- rc = -ENOENT;
-out:
- up_read(&pool_tgt_rw_sem(pool));
-
- lov_pool_putref(pool);
- return rc;
-}
-
-struct pool_desc *lov_find_pool(struct lov_obd *lov, char *poolname)
-{
- struct pool_desc *pool;
-
- pool = NULL;
- if (poolname[0] != '\0') {
- pool = cfs_hash_lookup(lov->lov_pools_hash_body, poolname);
- if (!pool)
- CWARN("Request for an unknown pool ("LOV_POOLNAMEF")\n",
- poolname);
- if (pool && (pool_tgt_count(pool) == 0)) {
- CWARN("Request for an empty pool ("LOV_POOLNAMEF")\n",
- poolname);
- /* pool is ignored, so we remove ref on it */
- lov_pool_putref(pool);
- pool = NULL;
- }
- }
- return pool;
-}
diff --git a/drivers/staging/lustre/lustre/lov/lov_request.c b/drivers/staging/lustre/lustre/lov/lov_request.c
index 7178a02..1be4b92 100644
--- a/drivers/staging/lustre/lustre/lov/lov_request.c
+++ b/drivers/staging/lustre/lustre/lov/lov_request.c
@@ -52,7 +52,6 @@
INIT_LIST_HEAD(&set->set_list);
atomic_set(&set->set_refcount, 1);
init_waitqueue_head(&set->set_waitq);
- spin_lock_init(&set->set_lock);
}
void lov_finish_set(struct lov_request_set *set)
@@ -235,7 +234,6 @@
if (tmp_oa)
kmem_cache_free(obdo_cachep, tmp_oa);
return rc;
-
}
int lov_fini_getattr_set(struct lov_request_set *set)
@@ -363,7 +361,6 @@
set->set_oi = oinfo;
set->set_oi->oi_md = lsm;
set->set_oi->oi_oa = src_oa;
- set->set_oti = oti;
if (oti && src_oa->o_valid & OBD_MD_FLCOOKIE)
set->set_cookies = oti->oti_logcookies;
@@ -480,7 +477,6 @@
lov_init_set(set);
set->set_exp = exp;
- set->set_oti = oti;
set->set_oi = oinfo;
if (oti && oinfo->oi_oa->o_valid & OBD_MD_FLCOOKIE)
set->set_cookies = oti->oti_logcookies;
@@ -716,12 +712,15 @@
struct lov_request *req;
if (!lov->lov_tgts[i] ||
- (!lov_check_and_wait_active(lov, i) &&
- (oinfo->oi_flags & OBD_STATFS_NODELAY))) {
+ (oinfo->oi_flags & OBD_STATFS_NODELAY &&
+ !lov->lov_tgts[i]->ltd_active)) {
CDEBUG(D_HA, "lov idx %d inactive\n", i);
continue;
}
+ if (!lov->lov_tgts[i]->ltd_active)
+ lov_check_and_wait_active(lov, i);
+
/* skip targets that have been explicitly disabled by the
* administrator
*/
diff --git a/drivers/staging/lustre/lustre/lov/lovsub_dev.c b/drivers/staging/lustre/lustre/lov/lovsub_dev.c
index c335c02..35f6b1d 100644
--- a/drivers/staging/lustre/lustre/lov/lovsub_dev.c
+++ b/drivers/staging/lustre/lustre/lov/lovsub_dev.c
@@ -151,8 +151,9 @@
if (lsr) {
cl_req_slice_add(req, &lsr->lsrq_cl, dev, &lovsub_req_ops);
result = 0;
- } else
+ } else {
result = -ENOMEM;
+ }
return result;
}
@@ -182,10 +183,12 @@
d = lovsub2lu_dev(lsd);
d->ld_ops = &lovsub_lu_ops;
lsd->acid_cl.cd_ops = &lovsub_cl_ops;
- } else
+ } else {
d = ERR_PTR(result);
- } else
+ }
+ } else {
d = ERR_PTR(-ENOMEM);
+ }
return d;
}
diff --git a/drivers/staging/lustre/lustre/lov/lovsub_lock.c b/drivers/staging/lustre/lustre/lov/lovsub_lock.c
index 3bb0c90..e92edfb 100644
--- a/drivers/staging/lustre/lustre/lov/lovsub_lock.c
+++ b/drivers/staging/lustre/lustre/lov/lovsub_lock.c
@@ -62,391 +62,8 @@
kmem_cache_free(lovsub_lock_kmem, lsl);
}
-static void lovsub_parent_lock(const struct lu_env *env, struct lov_lock *lov)
-{
- struct cl_lock *parent;
-
- parent = lov->lls_cl.cls_lock;
- cl_lock_get(parent);
- lu_ref_add(&parent->cll_reference, "lovsub-parent", current);
- cl_lock_mutex_get(env, parent);
-}
-
-static void lovsub_parent_unlock(const struct lu_env *env, struct lov_lock *lov)
-{
- struct cl_lock *parent;
-
- parent = lov->lls_cl.cls_lock;
- cl_lock_mutex_put(env, lov->lls_cl.cls_lock);
- lu_ref_del(&parent->cll_reference, "lovsub-parent", current);
- cl_lock_put(env, parent);
-}
-
-/**
- * Implements cl_lock_operations::clo_state() method for lovsub layer, which
- * method is called whenever sub-lock state changes. Propagates state change
- * to the top-locks.
- */
-static void lovsub_lock_state(const struct lu_env *env,
- const struct cl_lock_slice *slice,
- enum cl_lock_state state)
-{
- struct lovsub_lock *sub = cl2lovsub_lock(slice);
- struct lov_lock_link *scan;
-
- LASSERT(cl_lock_is_mutexed(slice->cls_lock));
-
- list_for_each_entry(scan, &sub->lss_parents, lll_list) {
- struct lov_lock *lov = scan->lll_super;
- struct cl_lock *parent = lov->lls_cl.cls_lock;
-
- if (sub->lss_active != parent) {
- lovsub_parent_lock(env, lov);
- cl_lock_signal(env, parent);
- lovsub_parent_unlock(env, lov);
- }
- }
-}
-
-/**
- * Implementation of cl_lock_operation::clo_weigh() estimating lock weight by
- * asking parent lock.
- */
-static unsigned long lovsub_lock_weigh(const struct lu_env *env,
- const struct cl_lock_slice *slice)
-{
- struct lovsub_lock *lock = cl2lovsub_lock(slice);
- struct lov_lock *lov;
- unsigned long dumbbell;
-
- LASSERT(cl_lock_is_mutexed(slice->cls_lock));
-
- if (!list_empty(&lock->lss_parents)) {
- /*
- * It is not clear whether all parents have to be asked and
- * their estimations summed, or it is enough to ask one. For
- * the current usages, one is always enough.
- */
- lov = container_of(lock->lss_parents.next,
- struct lov_lock_link, lll_list)->lll_super;
-
- lovsub_parent_lock(env, lov);
- dumbbell = cl_lock_weigh(env, lov->lls_cl.cls_lock);
- lovsub_parent_unlock(env, lov);
- } else
- dumbbell = 0;
-
- return dumbbell;
-}
-
-/**
- * Maps start/end offsets within a stripe, to offsets within a file.
- */
-static void lovsub_lock_descr_map(const struct cl_lock_descr *in,
- struct lov_object *lov,
- int stripe, struct cl_lock_descr *out)
-{
- pgoff_t size; /* stripe size in pages */
- pgoff_t skip; /* how many pages in every stripe are occupied by
- * "other" stripes
- */
- pgoff_t start;
- pgoff_t end;
-
- start = in->cld_start;
- end = in->cld_end;
-
- if (lov->lo_lsm->lsm_stripe_count > 1) {
- size = cl_index(lov2cl(lov), lov->lo_lsm->lsm_stripe_size);
- skip = (lov->lo_lsm->lsm_stripe_count - 1) * size;
-
- /* XXX overflow check here? */
- start += start/size * skip + stripe * size;
-
- if (end != CL_PAGE_EOF) {
- end += end/size * skip + stripe * size;
- /*
- * And check for overflow...
- */
- if (end < in->cld_end)
- end = CL_PAGE_EOF;
- }
- }
- out->cld_start = start;
- out->cld_end = end;
-}
-
-/**
- * Adjusts parent lock extent when a sub-lock is attached to a parent. This is
- * called in two ways:
- *
- * - as part of receive call-back, when server returns granted extent to
- * the client, and
- *
- * - when top-lock finds existing sub-lock in the cache.
- *
- * Note, that lock mode is not propagated to the parent: i.e., if CLM_READ
- * top-lock matches CLM_WRITE sub-lock, top-lock is still CLM_READ.
- */
-int lov_sublock_modify(const struct lu_env *env, struct lov_lock *lov,
- struct lovsub_lock *sublock,
- const struct cl_lock_descr *d, int idx)
-{
- struct cl_lock *parent;
- struct lovsub_object *subobj;
- struct cl_lock_descr *pd;
- struct cl_lock_descr *parent_descr;
- int result;
-
- parent = lov->lls_cl.cls_lock;
- parent_descr = &parent->cll_descr;
- LASSERT(cl_lock_mode_match(d->cld_mode, parent_descr->cld_mode));
-
- subobj = cl2lovsub(sublock->lss_cl.cls_obj);
- pd = &lov_env_info(env)->lti_ldescr;
-
- pd->cld_obj = parent_descr->cld_obj;
- pd->cld_mode = parent_descr->cld_mode;
- pd->cld_gid = parent_descr->cld_gid;
- lovsub_lock_descr_map(d, subobj->lso_super, subobj->lso_index, pd);
- lov->lls_sub[idx].sub_got = *d;
- /*
- * Notify top-lock about modification, if lock description changes
- * materially.
- */
- if (!cl_lock_ext_match(parent_descr, pd))
- result = cl_lock_modify(env, parent, pd);
- else
- result = 0;
- return result;
-}
-
-static int lovsub_lock_modify(const struct lu_env *env,
- const struct cl_lock_slice *s,
- const struct cl_lock_descr *d)
-{
- struct lovsub_lock *lock = cl2lovsub_lock(s);
- struct lov_lock_link *scan;
- struct lov_lock *lov;
- int result = 0;
-
- LASSERT(cl_lock_mode_match(d->cld_mode,
- s->cls_lock->cll_descr.cld_mode));
- list_for_each_entry(scan, &lock->lss_parents, lll_list) {
- int rc;
-
- lov = scan->lll_super;
- lovsub_parent_lock(env, lov);
- rc = lov_sublock_modify(env, lov, lock, d, scan->lll_idx);
- lovsub_parent_unlock(env, lov);
- result = result ?: rc;
- }
- return result;
-}
-
-static int lovsub_lock_closure(const struct lu_env *env,
- const struct cl_lock_slice *slice,
- struct cl_lock_closure *closure)
-{
- struct lovsub_lock *sub;
- struct cl_lock *parent;
- struct lov_lock_link *scan;
- int result;
-
- LASSERT(cl_lock_is_mutexed(slice->cls_lock));
-
- sub = cl2lovsub_lock(slice);
- result = 0;
-
- list_for_each_entry(scan, &sub->lss_parents, lll_list) {
- parent = scan->lll_super->lls_cl.cls_lock;
- result = cl_lock_closure_build(env, parent, closure);
- if (result != 0)
- break;
- }
- return result;
-}
-
-/**
- * A helper function for lovsub_lock_delete() that deals with a given parent
- * top-lock.
- */
-static int lovsub_lock_delete_one(const struct lu_env *env,
- struct cl_lock *child, struct lov_lock *lov)
-{
- struct cl_lock *parent;
- int result;
-
- parent = lov->lls_cl.cls_lock;
- if (parent->cll_error)
- return 0;
-
- result = 0;
- switch (parent->cll_state) {
- case CLS_ENQUEUED:
- /* See LU-1355 for the case that a glimpse lock is
- * interrupted by signal
- */
- LASSERT(parent->cll_flags & CLF_CANCELLED);
- break;
- case CLS_QUEUING:
- case CLS_FREEING:
- cl_lock_signal(env, parent);
- break;
- case CLS_INTRANSIT:
- /*
- * Here lies a problem: a sub-lock is canceled while top-lock
- * is being unlocked. Top-lock cannot be moved into CLS_NEW
- * state, because unlocking has to succeed eventually by
- * placing lock into CLS_CACHED (or failing it), see
- * cl_unuse_try(). Nor can top-lock be left in CLS_CACHED
- * state, because lov maintains an invariant that all
- * sub-locks exist in CLS_CACHED (this allows cached top-lock
- * to be reused immediately). Nor can we wait for top-lock
- * state to change, because this can be synchronous to the
- * current thread.
- *
- * We know for sure that lov_lock_unuse() will be called at
- * least one more time to finish un-using, so leave a mark on
- * the top-lock, that will be seen by the next call to
- * lov_lock_unuse().
- */
- if (cl_lock_is_intransit(parent))
- lov->lls_cancel_race = 1;
- break;
- case CLS_CACHED:
- /*
- * if a sub-lock is canceled move its top-lock into CLS_NEW
- * state to preserve an invariant that a top-lock in
- * CLS_CACHED is immediately ready for re-use (i.e., has all
- * sub-locks), and so that next attempt to re-use the top-lock
- * enqueues missing sub-lock.
- */
- cl_lock_state_set(env, parent, CLS_NEW);
- /* fall through */
- case CLS_NEW:
- /*
- * if last sub-lock is canceled, destroy the top-lock (which
- * is now `empty') proactively.
- */
- if (lov->lls_nr_filled == 0) {
- /* ... but unfortunately, this cannot be done easily,
- * as cancellation of a top-lock might acquire mutices
- * of its other sub-locks, violating lock ordering,
- * see cl_lock_{cancel,delete}() preconditions.
- *
- * To work around this, the mutex of this sub-lock is
- * released, top-lock is destroyed, and sub-lock mutex
- * acquired again. The list of parents has to be
- * re-scanned from the beginning after this.
- *
- * Only do this if no mutices other than on @child and
- * @parent are held by the current thread.
- *
- * TODO: The lock modal here is too complex, because
- * the lock may be canceled and deleted by voluntarily:
- * cl_lock_request
- * -> osc_lock_enqueue_wait
- * -> osc_lock_cancel_wait
- * -> cl_lock_delete
- * -> lovsub_lock_delete
- * -> cl_lock_cancel/delete
- * -> ...
- *
- * The better choice is to spawn a kernel thread for
- * this purpose. -jay
- */
- if (cl_lock_nr_mutexed(env) == 2) {
- cl_lock_mutex_put(env, child);
- cl_lock_cancel(env, parent);
- cl_lock_delete(env, parent);
- result = 1;
- }
- }
- break;
- case CLS_HELD:
- CL_LOCK_DEBUG(D_ERROR, env, parent, "Delete CLS_HELD lock\n");
- default:
- CERROR("Impossible state: %d\n", parent->cll_state);
- LBUG();
- break;
- }
-
- return result;
-}
-
-/**
- * An implementation of cl_lock_operations::clo_delete() method. This is
- * invoked in "bottom-to-top" delete, when lock destruction starts from the
- * sub-lock (e.g, as a result of ldlm lock LRU policy).
- */
-static void lovsub_lock_delete(const struct lu_env *env,
- const struct cl_lock_slice *slice)
-{
- struct cl_lock *child = slice->cls_lock;
- struct lovsub_lock *sub = cl2lovsub_lock(slice);
- int restart;
-
- LASSERT(cl_lock_is_mutexed(child));
-
- /*
- * Destruction of a sub-lock might take multiple iterations, because
- * when the last sub-lock of a given top-lock is deleted, top-lock is
- * canceled proactively, and this requires to release sub-lock
- * mutex. Once sub-lock mutex has been released, list of its parents
- * has to be re-scanned from the beginning.
- */
- do {
- struct lov_lock *lov;
- struct lov_lock_link *scan;
- struct lov_lock_link *temp;
- struct lov_lock_sub *subdata;
-
- restart = 0;
- list_for_each_entry_safe(scan, temp,
- &sub->lss_parents, lll_list) {
- lov = scan->lll_super;
- subdata = &lov->lls_sub[scan->lll_idx];
- lovsub_parent_lock(env, lov);
- subdata->sub_got = subdata->sub_descr;
- lov_lock_unlink(env, scan, sub);
- restart = lovsub_lock_delete_one(env, child, lov);
- lovsub_parent_unlock(env, lov);
-
- if (restart) {
- cl_lock_mutex_get(env, child);
- break;
- }
- }
- } while (restart);
-}
-
-static int lovsub_lock_print(const struct lu_env *env, void *cookie,
- lu_printer_t p, const struct cl_lock_slice *slice)
-{
- struct lovsub_lock *sub = cl2lovsub_lock(slice);
- struct lov_lock *lov;
- struct lov_lock_link *scan;
-
- list_for_each_entry(scan, &sub->lss_parents, lll_list) {
- lov = scan->lll_super;
- (*p)(env, cookie, "[%d %p ", scan->lll_idx, lov);
- if (lov)
- cl_lock_descr_print(env, cookie, p,
- &lov->lls_cl.cls_lock->cll_descr);
- (*p)(env, cookie, "] ");
- }
- return 0;
-}
-
static const struct cl_lock_operations lovsub_lock_ops = {
.clo_fini = lovsub_lock_fini,
- .clo_state = lovsub_lock_state,
- .clo_delete = lovsub_lock_delete,
- .clo_modify = lovsub_lock_modify,
- .clo_closure = lovsub_lock_closure,
- .clo_weigh = lovsub_lock_weigh,
- .clo_print = lovsub_lock_print
};
int lovsub_lock_init(const struct lu_env *env, struct cl_object *obj,
@@ -460,8 +77,9 @@
INIT_LIST_HEAD(&lsk->lss_parents);
cl_lock_slice_add(lock, &lsk->lss_cl, obj, &lovsub_lock_ops);
result = 0;
- } else
+ } else {
result = -ENOMEM;
+ }
return result;
}
diff --git a/drivers/staging/lustre/lustre/lov/lovsub_object.c b/drivers/staging/lustre/lustre/lov/lovsub_object.c
index 6c5430d..bcaae1e 100644
--- a/drivers/staging/lustre/lustre/lov/lovsub_object.c
+++ b/drivers/staging/lustre/lustre/lov/lovsub_object.c
@@ -67,10 +67,10 @@
lu_object_add(obj, below);
cl_object_page_init(lu2cl(obj), sizeof(struct lovsub_page));
result = 0;
- } else
+ } else {
result = -ENOMEM;
+ }
return result;
-
}
static void lovsub_object_free(const struct lu_env *env, struct lu_object *obj)
@@ -154,8 +154,9 @@
lu_object_add_top(&hdr->coh_lu, obj);
los->lso_cl.co_ops = &lovsub_ops;
obj->lo_ops = &lovsub_lu_obj_ops;
- } else
+ } else {
obj = NULL;
+ }
return obj;
}
diff --git a/drivers/staging/lustre/lustre/lov/lovsub_page.c b/drivers/staging/lustre/lustre/lov/lovsub_page.c
index 2d94553..9badedc 100644
--- a/drivers/staging/lustre/lustre/lov/lovsub_page.c
+++ b/drivers/staging/lustre/lustre/lov/lovsub_page.c
@@ -60,11 +60,11 @@
};
int lovsub_page_init(const struct lu_env *env, struct cl_object *obj,
- struct cl_page *page, struct page *unused)
+ struct cl_page *page, pgoff_t index)
{
struct lovsub_page *lsb = cl_object_page_slice(obj, page);
- cl_page_slice_add(page, &lsb->lsb_cl, obj, &lovsub_page_ops);
+ cl_page_slice_add(page, &lsb->lsb_cl, obj, index, &lovsub_page_ops);
return 0;
}
diff --git a/drivers/staging/lustre/lustre/mdc/lproc_mdc.c b/drivers/staging/lustre/lustre/mdc/lproc_mdc.c
index 38f267a..5c7a15d 100644
--- a/drivers/staging/lustre/lustre/mdc/lproc_mdc.c
+++ b/drivers/staging/lustre/lustre/mdc/lproc_mdc.c
@@ -49,9 +49,9 @@
obd_kobj);
struct client_obd *cli = &dev->u.cli;
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
len = sprintf(buf, "%u\n", cli->cl_max_rpcs_in_flight);
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
return len;
}
@@ -74,9 +74,9 @@
if (val < 1 || val > MDC_MAX_RIF_MAX)
return -ERANGE;
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
cli->cl_max_rpcs_in_flight = val;
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
return count;
}
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_lib.c b/drivers/staging/lustre/lustre/mdc/mdc_lib.c
index b3bfdcb..856c54e 100644
--- a/drivers/staging/lustre/lustre/mdc/mdc_lib.c
+++ b/drivers/staging/lustre/lustre/mdc/mdc_lib.c
@@ -279,8 +279,7 @@
rec->sa_atime = LTIME_S(op_data->op_attr.ia_atime);
rec->sa_mtime = LTIME_S(op_data->op_attr.ia_mtime);
rec->sa_ctime = LTIME_S(op_data->op_attr.ia_ctime);
- rec->sa_attr_flags =
- ((struct ll_iattr *)&op_data->op_attr)->ia_attr_flags;
+ rec->sa_attr_flags = op_data->op_attr_flags;
if ((op_data->op_attr.ia_valid & ATTR_GID) &&
in_group_p(op_data->op_attr.ia_gid))
rec->sa_suppgid =
@@ -439,7 +438,6 @@
char *tmp = req_capsule_client_get(&req->rq_pill, &RMF_NAME);
LOGL0(op_data->op_name, op_data->op_namelen, tmp);
-
}
}
@@ -455,7 +453,7 @@
lock = ldlm_handle2lock(&op_data->op_lease_handle);
if (lock) {
data->cd_handle = lock->l_remote_handle;
- ldlm_lock_put(lock);
+ LDLM_LOCK_PUT(lock);
}
ldlm_cli_cancel(&op_data->op_lease_handle, LCF_LOCAL);
@@ -481,9 +479,9 @@
{
int rc;
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
rc = list_empty(&mcw->mcw_entry);
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
return rc;
};
@@ -497,23 +495,23 @@
struct mdc_cache_waiter mcw;
struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL);
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
if (cli->cl_r_in_flight >= cli->cl_max_rpcs_in_flight) {
list_add_tail(&mcw.mcw_entry, &cli->cl_cache_waiters);
init_waitqueue_head(&mcw.mcw_waitq);
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
rc = l_wait_event(mcw.mcw_waitq, mdc_req_avail(cli, &mcw),
&lwi);
if (rc) {
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
if (list_empty(&mcw.mcw_entry))
cli->cl_r_in_flight--;
list_del_init(&mcw.mcw_entry);
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
}
} else {
cli->cl_r_in_flight++;
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
}
return rc;
}
@@ -523,7 +521,7 @@
struct list_head *l, *tmp;
struct mdc_cache_waiter *mcw;
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
cli->cl_r_in_flight--;
list_for_each_safe(l, tmp, &cli->cl_cache_waiters) {
if (cli->cl_r_in_flight >= cli->cl_max_rpcs_in_flight) {
@@ -538,5 +536,5 @@
}
/* Empty waiting list? Decrease reqs in-flight number */
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
}
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_locks.c b/drivers/staging/lustre/lustre/mdc/mdc_locks.c
index 958a164..3b1bc91 100644
--- a/drivers/staging/lustre/lustre/mdc/mdc_locks.c
+++ b/drivers/staging/lustre/lustre/mdc/mdc_locks.c
@@ -869,7 +869,9 @@
* (explicits or automatically generated by Kernel to clean
* current FLocks upon exit) that can't be trashed
*/
- if ((rc == -EINTR) || (rc == -ETIMEDOUT))
+ if (((rc == -EINTR) || (rc == -ETIMEDOUT)) &&
+ (einfo->ei_type == LDLM_FLOCK) &&
+ (einfo->ei_mode == LCK_NL))
goto resend;
return rc;
}
@@ -963,7 +965,6 @@
if (fid_is_sane(&op_data->op_fid2) &&
it->it_create_mode & M_CHECK_STALE &&
it->it_op != IT_GETATTR) {
-
/* Also: did we find the same inode? */
/* sever can return one of two fids:
* op_fid2 - new allocated fid - if file is created.
diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c
index b91d3ff..86b7445 100644
--- a/drivers/staging/lustre/lustre/mdc/mdc_request.c
+++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c
@@ -142,9 +142,8 @@
CDEBUG(D_NET, "mode: %o\n", body->mode);
+ mdc_update_max_ea_from_body(exp, body);
if (body->eadatasize != 0) {
- mdc_update_max_ea_from_body(exp, body);
-
eadata = req_capsule_server_sized_get(pill, &RMF_MDT_MD,
body->eadatasize);
if (!eadata)
@@ -1169,7 +1168,7 @@
goto out;
}
- mdc_pack_body(req, NULL, OBD_MD_FLRMTPERM, 0, 0, 0);
+ mdc_pack_body(req, NULL, OBD_MD_FLRMTPERM, 0, -1, 0);
/* Copy hsm_progress struct */
req_hpk = req_capsule_client_get(&req->rq_pill, &RMF_MDS_HSM_PROGRESS);
@@ -1203,7 +1202,7 @@
goto out;
}
- mdc_pack_body(req, NULL, OBD_MD_FLRMTPERM, 0, 0, 0);
+ mdc_pack_body(req, NULL, OBD_MD_FLRMTPERM, 0, -1, 0);
/* Copy hsm_progress struct */
archive_mask = req_capsule_client_get(&req->rq_pill,
@@ -1278,7 +1277,7 @@
goto out;
}
- mdc_pack_body(req, NULL, OBD_MD_FLRMTPERM, 0, 0, 0);
+ mdc_pack_body(req, NULL, OBD_MD_FLRMTPERM, 0, -1, 0);
ptlrpc_request_set_replen(req);
@@ -1395,7 +1394,7 @@
return rc;
}
- mdc_pack_body(req, NULL, OBD_MD_FLRMTPERM, 0, 0, 0);
+ mdc_pack_body(req, NULL, OBD_MD_FLRMTPERM, 0, -1, 0);
/* Copy hsm_request struct */
req_hr = req_capsule_client_get(&req->rq_pill, &RMF_MDS_HSM_REQUEST);
@@ -1952,7 +1951,7 @@
__swab32s(&h->hal_count);
__swab32s(&h->hal_archive_id);
__swab64s(&h->hal_flags);
- hai = hai_zero(h);
+ hai = hai_first(h);
for (i = 0; i < h->hal_count; i++, hai = hai_next(hai))
lustre_swab_hai(hai);
}
@@ -2249,7 +2248,7 @@
* recovery, non zero value will be return if the lock can be canceled,
* or zero returned for not
*/
-static int mdc_cancel_for_recovery(struct ldlm_lock *lock)
+static int mdc_cancel_weight(struct ldlm_lock *lock)
{
if (lock->l_resource->lr_type != LDLM_IBITS)
return 0;
@@ -2314,12 +2313,14 @@
return -ENOMEM;
mdc_init_rpc_lock(cli->cl_rpc_lock);
- ptlrpcd_addref();
+ rc = ptlrpcd_addref();
+ if (rc < 0)
+ goto err_rpc_lock;
cli->cl_close_lock = kzalloc(sizeof(*cli->cl_close_lock), GFP_NOFS);
if (!cli->cl_close_lock) {
rc = -ENOMEM;
- goto err_rpc_lock;
+ goto err_ptlrpcd_decref;
}
mdc_init_rpc_lock(cli->cl_close_lock);
@@ -2331,7 +2332,7 @@
sptlrpc_lprocfs_cliobd_attach(obd);
ptlrpc_lprocfs_register_obd(obd);
- ns_register_cancel(obd->obd_namespace, mdc_cancel_for_recovery);
+ ns_register_cancel(obd->obd_namespace, mdc_cancel_weight);
obd->obd_namespace->ns_lvbo = &inode_lvbo;
@@ -2345,9 +2346,10 @@
err_close_lock:
kfree(cli->cl_close_lock);
+err_ptlrpcd_decref:
+ ptlrpcd_decref();
err_rpc_lock:
kfree(cli->cl_rpc_lock);
- ptlrpcd_decref();
return rc;
}
diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c b/drivers/staging/lustre/lustre/mgc/mgc_request.c
index 3924b09..2311a43 100644
--- a/drivers/staging/lustre/lustre/mgc/mgc_request.c
+++ b/drivers/staging/lustre/lustre/mgc/mgc_request.c
@@ -502,8 +502,12 @@
*/
down_read(&cld->cld_mgcexp->exp_obd->u.cli.cl_sem);
if (cld->cld_mgcexp->exp_obd->u.cli.cl_conn_count != 0) {
+ int rc;
+
CDEBUG(D_MGC, "updating log %s\n", cld->cld_logname);
- mgc_process_log(cld->cld_mgcexp->exp_obd, cld);
+ rc = mgc_process_log(cld->cld_mgcexp->exp_obd, cld);
+ if (rc && rc != -ENOENT)
+ CERROR("failed processing log: %d\n", rc);
} else {
CDEBUG(D_MGC, "disconnecting, won't update log %s\n",
cld->cld_logname);
@@ -734,7 +738,9 @@
struct task_struct *task;
int rc;
- ptlrpcd_addref();
+ rc = ptlrpcd_addref();
+ if (rc < 0)
+ goto err_noref;
rc = client_obd_setup(obd, lcfg);
if (rc)
@@ -773,6 +779,7 @@
client_obd_cleanup(obd);
err_decref:
ptlrpcd_decref();
+err_noref:
return rc;
}
@@ -1720,7 +1727,6 @@
CERROR("Unknown command: %d\n", lcfg->lcfg_command);
rc = -EINVAL;
goto out;
-
}
}
out:
diff --git a/drivers/staging/lustre/lustre/obdclass/cl_io.c b/drivers/staging/lustre/lustre/obdclass/cl_io.c
index f5128b4..583fb5f 100644
--- a/drivers/staging/lustre/lustre/obdclass/cl_io.c
+++ b/drivers/staging/lustre/lustre/obdclass/cl_io.c
@@ -36,6 +36,7 @@
* Client IO.
*
* Author: Nikita Danilov <nikita.danilov@sun.com>
+ * Author: Jinshan Xiong <jinshan.xiong@intel.com>
*/
#define DEBUG_SUBSYSTEM S_CLASS
@@ -132,6 +133,7 @@
case CIT_WRITE:
break;
case CIT_FAULT:
+ break;
case CIT_FSYNC:
LASSERT(!io->ci_need_restart);
break;
@@ -159,7 +161,6 @@
io->ci_type = iot;
INIT_LIST_HEAD(&io->ci_lockset.cls_todo);
- INIT_LIST_HEAD(&io->ci_lockset.cls_curr);
INIT_LIST_HEAD(&io->ci_lockset.cls_done);
INIT_LIST_HEAD(&io->ci_layers);
@@ -241,37 +242,7 @@
const struct cl_lock_descr *d1)
{
return lu_fid_cmp(lu_object_fid(&d0->cld_obj->co_lu),
- lu_object_fid(&d1->cld_obj->co_lu)) ?:
- __diff_normalize(d0->cld_start, d1->cld_start);
-}
-
-static int cl_lock_descr_cmp(const struct cl_lock_descr *d0,
- const struct cl_lock_descr *d1)
-{
- int ret;
-
- ret = lu_fid_cmp(lu_object_fid(&d0->cld_obj->co_lu),
- lu_object_fid(&d1->cld_obj->co_lu));
- if (ret)
- return ret;
- if (d0->cld_end < d1->cld_start)
- return -1;
- if (d0->cld_start > d0->cld_end)
- return 1;
- return 0;
-}
-
-static void cl_lock_descr_merge(struct cl_lock_descr *d0,
- const struct cl_lock_descr *d1)
-{
- d0->cld_start = min(d0->cld_start, d1->cld_start);
- d0->cld_end = max(d0->cld_end, d1->cld_end);
-
- if (d1->cld_mode == CLM_WRITE && d0->cld_mode != CLM_WRITE)
- d0->cld_mode = CLM_WRITE;
-
- if (d1->cld_mode == CLM_GROUP && d0->cld_mode != CLM_GROUP)
- d0->cld_mode = CLM_GROUP;
+ lu_object_fid(&d1->cld_obj->co_lu));
}
/*
@@ -320,33 +291,35 @@
} while (!done);
}
-/**
- * Check whether \a queue contains locks matching \a need.
- *
- * \retval +ve there is a matching lock in the \a queue
- * \retval 0 there are no matching locks in the \a queue
- */
-int cl_queue_match(const struct list_head *queue,
- const struct cl_lock_descr *need)
+static void cl_lock_descr_merge(struct cl_lock_descr *d0,
+ const struct cl_lock_descr *d1)
{
- struct cl_io_lock_link *scan;
+ d0->cld_start = min(d0->cld_start, d1->cld_start);
+ d0->cld_end = max(d0->cld_end, d1->cld_end);
- list_for_each_entry(scan, queue, cill_linkage) {
- if (cl_lock_descr_match(&scan->cill_descr, need))
- return 1;
- }
- return 0;
+ if (d1->cld_mode == CLM_WRITE && d0->cld_mode != CLM_WRITE)
+ d0->cld_mode = CLM_WRITE;
+
+ if (d1->cld_mode == CLM_GROUP && d0->cld_mode != CLM_GROUP)
+ d0->cld_mode = CLM_GROUP;
}
-EXPORT_SYMBOL(cl_queue_match);
-static int cl_queue_merge(const struct list_head *queue,
- const struct cl_lock_descr *need)
+static int cl_lockset_merge(const struct cl_lockset *set,
+ const struct cl_lock_descr *need)
{
struct cl_io_lock_link *scan;
- list_for_each_entry(scan, queue, cill_linkage) {
- if (cl_lock_descr_cmp(&scan->cill_descr, need))
+ list_for_each_entry(scan, &set->cls_todo, cill_linkage) {
+ if (!cl_object_same(scan->cill_descr.cld_obj, need->cld_obj))
continue;
+
+ /* Merge locks for the same object because ldlm lock server
+ * may expand the lock extent, otherwise there is a deadlock
+ * case if two conflicted locks are queueud for the same object
+ * and lock server expands one lock to overlap the another.
+ * The side effect is that it can generate a multi-stripe lock
+ * that may cause casacading problem
+ */
cl_lock_descr_merge(&scan->cill_descr, need);
CDEBUG(D_VFSTRACE, "lock: %d: [%lu, %lu]\n",
scan->cill_descr.cld_mode, scan->cill_descr.cld_start,
@@ -356,87 +329,20 @@
return 0;
}
-static int cl_lockset_match(const struct cl_lockset *set,
- const struct cl_lock_descr *need)
-{
- return cl_queue_match(&set->cls_curr, need) ||
- cl_queue_match(&set->cls_done, need);
-}
-
-static int cl_lockset_merge(const struct cl_lockset *set,
- const struct cl_lock_descr *need)
-{
- return cl_queue_merge(&set->cls_todo, need) ||
- cl_lockset_match(set, need);
-}
-
-static int cl_lockset_lock_one(const struct lu_env *env,
- struct cl_io *io, struct cl_lockset *set,
- struct cl_io_lock_link *link)
-{
- struct cl_lock *lock;
- int result;
-
- lock = cl_lock_request(env, io, &link->cill_descr, "io", io);
-
- if (!IS_ERR(lock)) {
- link->cill_lock = lock;
- list_move(&link->cill_linkage, &set->cls_curr);
- if (!(link->cill_descr.cld_enq_flags & CEF_ASYNC)) {
- result = cl_wait(env, lock);
- if (result == 0)
- list_move(&link->cill_linkage, &set->cls_done);
- } else
- result = 0;
- } else
- result = PTR_ERR(lock);
- return result;
-}
-
-static void cl_lock_link_fini(const struct lu_env *env, struct cl_io *io,
- struct cl_io_lock_link *link)
-{
- struct cl_lock *lock = link->cill_lock;
-
- list_del_init(&link->cill_linkage);
- if (lock) {
- cl_lock_release(env, lock, "io", io);
- link->cill_lock = NULL;
- }
- if (link->cill_fini)
- link->cill_fini(env, link);
-}
-
static int cl_lockset_lock(const struct lu_env *env, struct cl_io *io,
struct cl_lockset *set)
{
struct cl_io_lock_link *link;
struct cl_io_lock_link *temp;
- struct cl_lock *lock;
int result;
result = 0;
list_for_each_entry_safe(link, temp, &set->cls_todo, cill_linkage) {
- if (!cl_lockset_match(set, &link->cill_descr)) {
- /* XXX some locking to guarantee that locks aren't
- * expanded in between.
- */
- result = cl_lockset_lock_one(env, io, set, link);
- if (result != 0)
- break;
- } else
- cl_lock_link_fini(env, io, link);
- }
- if (result == 0) {
- list_for_each_entry_safe(link, temp,
- &set->cls_curr, cill_linkage) {
- lock = link->cill_lock;
- result = cl_wait(env, lock);
- if (result == 0)
- list_move(&link->cill_linkage, &set->cls_done);
- else
- break;
- }
+ result = cl_lock_request(env, io, &link->cill_lock);
+ if (result < 0)
+ break;
+
+ list_move(&link->cill_linkage, &set->cls_done);
}
return result;
}
@@ -492,16 +398,19 @@
set = &io->ci_lockset;
- list_for_each_entry_safe(link, temp, &set->cls_todo, cill_linkage)
- cl_lock_link_fini(env, io, link);
-
- list_for_each_entry_safe(link, temp, &set->cls_curr, cill_linkage)
- cl_lock_link_fini(env, io, link);
+ list_for_each_entry_safe(link, temp, &set->cls_todo, cill_linkage) {
+ list_del_init(&link->cill_linkage);
+ if (link->cill_fini)
+ link->cill_fini(env, link);
+ }
list_for_each_entry_safe(link, temp, &set->cls_done, cill_linkage) {
- cl_unuse(env, link->cill_lock);
- cl_lock_link_fini(env, io, link);
+ list_del_init(&link->cill_linkage);
+ cl_lock_release(env, &link->cill_lock);
+ if (link->cill_fini)
+ link->cill_fini(env, link);
}
+
cl_io_for_each_reverse(scan, io) {
if (scan->cis_iop->op[io->ci_type].cio_unlock)
scan->cis_iop->op[io->ci_type].cio_unlock(env, scan);
@@ -595,9 +504,9 @@
{
int result;
- if (cl_lockset_merge(&io->ci_lockset, &link->cill_descr))
+ if (cl_lockset_merge(&io->ci_lockset, &link->cill_descr)) {
result = 1;
- else {
+ } else {
list_add(&link->cill_linkage, &io->ci_lockset.cls_todo);
result = 0;
}
@@ -627,8 +536,9 @@
result = cl_io_lock_add(env, io, link);
if (result) /* lock match */
link->cill_fini(env, link);
- } else
+ } else {
result = -ENOMEM;
+ }
return result;
}
@@ -692,42 +602,6 @@
}
/**
- * True iff \a page is within \a io range.
- */
-static int cl_page_in_io(const struct cl_page *page, const struct cl_io *io)
-{
- int result = 1;
- loff_t start;
- loff_t end;
- pgoff_t idx;
-
- idx = page->cp_index;
- switch (io->ci_type) {
- case CIT_READ:
- case CIT_WRITE:
- /*
- * check that [start, end) and [pos, pos + count) extents
- * overlap.
- */
- if (!cl_io_is_append(io)) {
- const struct cl_io_rw_common *crw = &(io->u.ci_rw);
-
- start = cl_offset(page->cp_obj, idx);
- end = cl_offset(page->cp_obj, idx + 1);
- result = crw->crw_pos < end &&
- start < crw->crw_pos + crw->crw_count;
- }
- break;
- case CIT_FAULT:
- result = io->u.ci_fault.ft_index == idx;
- break;
- default:
- LBUG();
- }
- return result;
-}
-
-/**
* Called by read io, when page has to be read from the server.
*
* \see cl_io_operations::cio_read_page()
@@ -742,7 +616,6 @@
LINVRNT(io->ci_type == CIT_READ || io->ci_type == CIT_FAULT);
LINVRNT(cl_page_is_owned(page, io));
LINVRNT(io->ci_state == CIS_IO_GOING || io->ci_state == CIS_LOCKED);
- LINVRNT(cl_page_in_io(page, io));
LINVRNT(cl_io_invariant(io));
queue = &io->ci_queue;
@@ -769,7 +642,7 @@
break;
}
}
- if (result == 0)
+ if (result == 0 && queue->c2_qin.pl_nr > 0)
result = cl_io_submit_rw(env, io, CRT_READ, queue);
/*
* Unlock unsent pages in case of error.
@@ -781,77 +654,29 @@
EXPORT_SYMBOL(cl_io_read_page);
/**
- * Called by write io to prepare page to receive data from user buffer.
+ * Commit a list of contiguous pages into writeback cache.
*
- * \see cl_io_operations::cio_prepare_write()
+ * \returns 0 if all pages committed, or errcode if error occurred.
+ * \see cl_io_operations::cio_commit_async()
*/
-int cl_io_prepare_write(const struct lu_env *env, struct cl_io *io,
- struct cl_page *page, unsigned from, unsigned to)
+int cl_io_commit_async(const struct lu_env *env, struct cl_io *io,
+ struct cl_page_list *queue, int from, int to,
+ cl_commit_cbt cb)
{
const struct cl_io_slice *scan;
int result = 0;
- LINVRNT(io->ci_type == CIT_WRITE);
- LINVRNT(cl_page_is_owned(page, io));
- LINVRNT(io->ci_state == CIS_IO_GOING || io->ci_state == CIS_LOCKED);
- LINVRNT(cl_io_invariant(io));
- LASSERT(cl_page_in_io(page, io));
-
- cl_io_for_each_reverse(scan, io) {
- if (scan->cis_iop->cio_prepare_write) {
- const struct cl_page_slice *slice;
-
- slice = cl_io_slice_page(scan, page);
- result = scan->cis_iop->cio_prepare_write(env, scan,
- slice,
- from, to);
- if (result != 0)
- break;
- }
- }
- return result;
-}
-EXPORT_SYMBOL(cl_io_prepare_write);
-
-/**
- * Called by write io after user data were copied into a page.
- *
- * \see cl_io_operations::cio_commit_write()
- */
-int cl_io_commit_write(const struct lu_env *env, struct cl_io *io,
- struct cl_page *page, unsigned from, unsigned to)
-{
- const struct cl_io_slice *scan;
- int result = 0;
-
- LINVRNT(io->ci_type == CIT_WRITE);
- LINVRNT(io->ci_state == CIS_IO_GOING || io->ci_state == CIS_LOCKED);
- LINVRNT(cl_io_invariant(io));
- /*
- * XXX Uh... not nice. Top level cl_io_commit_write() call (vvp->lov)
- * already called cl_page_cache_add(), moving page into CPS_CACHED
- * state. Better (and more general) way of dealing with such situation
- * is needed.
- */
- LASSERT(cl_page_is_owned(page, io) || page->cp_parent);
- LASSERT(cl_page_in_io(page, io));
-
cl_io_for_each(scan, io) {
- if (scan->cis_iop->cio_commit_write) {
- const struct cl_page_slice *slice;
-
- slice = cl_io_slice_page(scan, page);
- result = scan->cis_iop->cio_commit_write(env, scan,
- slice,
- from, to);
- if (result != 0)
- break;
- }
+ if (!scan->cis_iop->cio_commit_async)
+ continue;
+ result = scan->cis_iop->cio_commit_async(env, scan, queue,
+ from, to, cb);
+ if (result != 0)
+ break;
}
- LINVRNT(result <= 0);
return result;
}
-EXPORT_SYMBOL(cl_io_commit_write);
+EXPORT_SYMBOL(cl_io_commit_async);
/**
* Submits a list of pages for immediate io.
@@ -869,13 +694,10 @@
const struct cl_io_slice *scan;
int result = 0;
- LINVRNT(crt < ARRAY_SIZE(scan->cis_iop->req_op));
-
cl_io_for_each(scan, io) {
- if (!scan->cis_iop->req_op[crt].cio_submit)
+ if (!scan->cis_iop->cio_submit)
continue;
- result = scan->cis_iop->req_op[crt].cio_submit(env, scan, crt,
- queue);
+ result = scan->cis_iop->cio_submit(env, scan, crt, queue);
if (result != 0)
break;
}
@@ -887,6 +709,9 @@
}
EXPORT_SYMBOL(cl_io_submit_rw);
+static void cl_page_list_assume(const struct lu_env *env,
+ struct cl_io *io, struct cl_page_list *plist);
+
/**
* Submit a sync_io and wait for the IO to be finished, or error happens.
* If \a timeout is zero, it means to wait for the IO unconditionally.
@@ -904,7 +729,7 @@
pg->cp_sync_io = anchor;
}
- cl_sync_io_init(anchor, queue->c2_qin.pl_nr);
+ cl_sync_io_init(anchor, queue->c2_qin.pl_nr, &cl_sync_io_end);
rc = cl_io_submit_rw(env, io, iot, queue);
if (rc == 0) {
/*
@@ -915,12 +740,12 @@
*/
cl_page_list_for_each(pg, &queue->c2_qin) {
pg->cp_sync_io = NULL;
- cl_sync_io_note(anchor, 1);
+ cl_sync_io_note(env, anchor, 1);
}
/* wait for the IO to be finished. */
- rc = cl_sync_io_wait(env, io, &queue->c2_qout,
- anchor, timeout);
+ rc = cl_sync_io_wait(env, anchor, timeout);
+ cl_page_list_assume(env, io, &queue->c2_qout);
} else {
LASSERT(list_empty(&queue->c2_qout.pl_pages));
cl_page_list_for_each(pg, &queue->c2_qin)
@@ -931,26 +756,6 @@
EXPORT_SYMBOL(cl_io_submit_sync);
/**
- * Cancel an IO which has been submitted by cl_io_submit_rw.
- */
-static int cl_io_cancel(const struct lu_env *env, struct cl_io *io,
- struct cl_page_list *queue)
-{
- struct cl_page *page;
- int result = 0;
-
- CERROR("Canceling ongoing page transmission\n");
- cl_page_list_for_each(page, queue) {
- int rc;
-
- LINVRNT(cl_page_in_io(page, io));
- rc = cl_page_cancel(env, page);
- result = result ?: rc;
- }
- return result;
-}
-
-/**
* Main io loop.
*
* Pumps io through iterations calling
@@ -1072,8 +877,8 @@
/**
* Removes a page from a page list.
*/
-static void cl_page_list_del(const struct lu_env *env,
- struct cl_page_list *plist, struct cl_page *page)
+void cl_page_list_del(const struct lu_env *env, struct cl_page_list *plist,
+ struct cl_page *page)
{
LASSERT(plist->pl_nr > 0);
LINVRNT(plist->pl_owner == current);
@@ -1086,6 +891,7 @@
lu_ref_del_at(&page->cp_reference, &page->cp_queue_ref, "queue", plist);
cl_page_put(env, page);
}
+EXPORT_SYMBOL(cl_page_list_del);
/**
* Moves a page from one page list to another.
@@ -1106,6 +912,24 @@
EXPORT_SYMBOL(cl_page_list_move);
/**
+ * Moves a page from one page list to the head of another list.
+ */
+void cl_page_list_move_head(struct cl_page_list *dst, struct cl_page_list *src,
+ struct cl_page *page)
+{
+ LASSERT(src->pl_nr > 0);
+ LINVRNT(dst->pl_owner == current);
+ LINVRNT(src->pl_owner == current);
+
+ list_move(&page->cp_batch, &dst->pl_pages);
+ --src->pl_nr;
+ ++dst->pl_nr;
+ lu_ref_set_at(&page->cp_reference, &page->cp_queue_ref, "queue",
+ src, dst);
+}
+EXPORT_SYMBOL(cl_page_list_move_head);
+
+/**
* splice the cl_page_list, just as list head does
*/
void cl_page_list_splice(struct cl_page_list *list, struct cl_page_list *head)
@@ -1162,8 +986,7 @@
/**
* Releases pages from queue.
*/
-static void cl_page_list_fini(const struct lu_env *env,
- struct cl_page_list *plist)
+void cl_page_list_fini(const struct lu_env *env, struct cl_page_list *plist)
{
struct cl_page *page;
struct cl_page *temp;
@@ -1174,6 +997,7 @@
cl_page_list_del(env, plist, page);
LASSERT(plist->pl_nr == 0);
}
+EXPORT_SYMBOL(cl_page_list_fini);
/**
* Assumes all pages in a queue.
@@ -1260,7 +1084,7 @@
/**
* Returns top-level io.
*
- * \see cl_object_top(), cl_page_top().
+ * \see cl_object_top()
*/
struct cl_io *cl_io_top(struct cl_io *io)
{
@@ -1323,19 +1147,14 @@
int result;
result = 0;
- page = cl_page_top(page);
- do {
- list_for_each_entry(slice, &page->cp_layers, cpl_linkage) {
- dev = lu2cl_dev(slice->cpl_obj->co_lu.lo_dev);
- if (dev->cd_ops->cdo_req_init) {
- result = dev->cd_ops->cdo_req_init(env,
- dev, req);
- if (result != 0)
- break;
- }
+ list_for_each_entry(slice, &page->cp_layers, cpl_linkage) {
+ dev = lu2cl_dev(slice->cpl_obj->co_lu.lo_dev);
+ if (dev->cd_ops->cdo_req_init) {
+ result = dev->cd_ops->cdo_req_init(env, dev, req);
+ if (result != 0)
+ break;
}
- page = page->cp_child;
- } while (page && result == 0);
+ }
return result;
}
@@ -1384,14 +1203,16 @@
if (req->crq_o) {
req->crq_nrobjs = nr_objects;
result = cl_req_init(env, req, page);
- } else
+ } else {
result = -ENOMEM;
+ }
if (result != 0) {
cl_req_completion(env, req, result);
req = ERR_PTR(result);
}
- } else
+ } else {
req = ERR_PTR(-ENOMEM);
+ }
return req;
}
EXPORT_SYMBOL(cl_req_alloc);
@@ -1406,8 +1227,6 @@
struct cl_req_obj *rqo;
int i;
- page = cl_page_top(page);
-
LASSERT(list_empty(&page->cp_flight));
LASSERT(!page->cp_req);
@@ -1438,8 +1257,6 @@
{
struct cl_req *req = page->cp_req;
- page = cl_page_top(page);
-
LASSERT(!list_empty(&page->cp_flight));
LASSERT(req->crq_nrpages > 0);
@@ -1511,25 +1328,39 @@
}
EXPORT_SYMBOL(cl_req_attr_set);
+/* cl_sync_io_callback assumes the caller must call cl_sync_io_wait() to
+ * wait for the IO to finish.
+ */
+void cl_sync_io_end(const struct lu_env *env, struct cl_sync_io *anchor)
+{
+ wake_up_all(&anchor->csi_waitq);
+
+ /* it's safe to nuke or reuse anchor now */
+ atomic_set(&anchor->csi_barrier, 0);
+}
+EXPORT_SYMBOL(cl_sync_io_end);
/**
- * Initialize synchronous io wait anchor, for transfer of \a nrpages pages.
+ * Initialize synchronous io wait anchor
*/
-void cl_sync_io_init(struct cl_sync_io *anchor, int nrpages)
+void cl_sync_io_init(struct cl_sync_io *anchor, int nr,
+ void (*end)(const struct lu_env *, struct cl_sync_io *))
{
+ memset(anchor, 0, sizeof(*anchor));
init_waitqueue_head(&anchor->csi_waitq);
- atomic_set(&anchor->csi_sync_nr, nrpages);
- atomic_set(&anchor->csi_barrier, nrpages > 0);
+ atomic_set(&anchor->csi_sync_nr, nr);
+ atomic_set(&anchor->csi_barrier, nr > 0);
anchor->csi_sync_rc = 0;
+ anchor->csi_end_io = end;
+ LASSERT(end);
}
EXPORT_SYMBOL(cl_sync_io_init);
/**
- * Wait until all transfer completes. Transfer completion routine has to call
- * cl_sync_io_note() for every page.
+ * Wait until all IO completes. Transfer completion routine has to call
+ * cl_sync_io_note() for every entity.
*/
-int cl_sync_io_wait(const struct lu_env *env, struct cl_io *io,
- struct cl_page_list *queue, struct cl_sync_io *anchor,
+int cl_sync_io_wait(const struct lu_env *env, struct cl_sync_io *anchor,
long timeout)
{
struct l_wait_info lwi = LWI_TIMEOUT_INTR(cfs_time_seconds(timeout),
@@ -1542,11 +1373,9 @@
atomic_read(&anchor->csi_sync_nr) == 0,
&lwi);
if (rc < 0) {
- CERROR("SYNC IO failed with error: %d, try to cancel %d remaining pages\n",
+ CERROR("IO failed: %d, still wait for %d remaining entries\n",
rc, atomic_read(&anchor->csi_sync_nr));
- (void)cl_io_cancel(env, io, queue);
-
lwi = (struct l_wait_info) { 0 };
(void)l_wait_event(anchor->csi_waitq,
atomic_read(&anchor->csi_sync_nr) == 0,
@@ -1555,14 +1384,12 @@
rc = anchor->csi_sync_rc;
}
LASSERT(atomic_read(&anchor->csi_sync_nr) == 0);
- cl_page_list_assume(env, io, queue);
/* wait until cl_sync_io_note() has done wakeup */
while (unlikely(atomic_read(&anchor->csi_barrier) != 0)) {
cpu_relax();
}
- POISON(anchor, 0x5a, sizeof(*anchor));
return rc;
}
EXPORT_SYMBOL(cl_sync_io_wait);
@@ -1570,7 +1397,8 @@
/**
* Indicate that transfer of a single page completed.
*/
-void cl_sync_io_note(struct cl_sync_io *anchor, int ioret)
+void cl_sync_io_note(const struct lu_env *env, struct cl_sync_io *anchor,
+ int ioret)
{
if (anchor->csi_sync_rc == 0 && ioret < 0)
anchor->csi_sync_rc = ioret;
@@ -1581,9 +1409,9 @@
*/
LASSERT(atomic_read(&anchor->csi_sync_nr) > 0);
if (atomic_dec_and_test(&anchor->csi_sync_nr)) {
- wake_up_all(&anchor->csi_waitq);
- /* it's safe to nuke or reuse anchor now */
- atomic_set(&anchor->csi_barrier, 0);
+ LASSERT(anchor->csi_end_io);
+ anchor->csi_end_io(env, anchor);
+ /* Can't access anchor any more */
}
}
EXPORT_SYMBOL(cl_sync_io_note);
diff --git a/drivers/staging/lustre/lustre/obdclass/cl_lock.c b/drivers/staging/lustre/lustre/obdclass/cl_lock.c
index aec644eb..26a576b 100644
--- a/drivers/staging/lustre/lustre/obdclass/cl_lock.c
+++ b/drivers/staging/lustre/lustre/obdclass/cl_lock.c
@@ -36,6 +36,7 @@
* Client Extent Lock.
*
* Author: Nikita Danilov <nikita.danilov@sun.com>
+ * Author: Jinshan Xiong <jinshan.xiong@intel.com>
*/
#define DEBUG_SUBSYSTEM S_CLASS
@@ -47,138 +48,18 @@
#include "../include/cl_object.h"
#include "cl_internal.h"
-/** Lock class of cl_lock::cll_guard */
-static struct lock_class_key cl_lock_guard_class;
-static struct kmem_cache *cl_lock_kmem;
-
-static struct lu_kmem_descr cl_lock_caches[] = {
- {
- .ckd_cache = &cl_lock_kmem,
- .ckd_name = "cl_lock_kmem",
- .ckd_size = sizeof (struct cl_lock)
- },
- {
- .ckd_cache = NULL
- }
-};
-
-#define CS_LOCK_INC(o, item)
-#define CS_LOCK_DEC(o, item)
-#define CS_LOCKSTATE_INC(o, state)
-#define CS_LOCKSTATE_DEC(o, state)
-
-/**
- * Basic lock invariant that is maintained at all times. Caller either has a
- * reference to \a lock, or somehow assures that \a lock cannot be freed.
- *
- * \see cl_lock_invariant()
- */
-static int cl_lock_invariant_trusted(const struct lu_env *env,
- const struct cl_lock *lock)
-{
- return ergo(lock->cll_state == CLS_FREEING, lock->cll_holds == 0) &&
- atomic_read(&lock->cll_ref) >= lock->cll_holds &&
- lock->cll_holds >= lock->cll_users &&
- lock->cll_holds >= 0 &&
- lock->cll_users >= 0 &&
- lock->cll_depth >= 0;
-}
-
-/**
- * Stronger lock invariant, checking that caller has a reference on a lock.
- *
- * \see cl_lock_invariant_trusted()
- */
-static int cl_lock_invariant(const struct lu_env *env,
- const struct cl_lock *lock)
-{
- int result;
-
- result = atomic_read(&lock->cll_ref) > 0 &&
- cl_lock_invariant_trusted(env, lock);
- if (!result && env)
- CL_LOCK_DEBUG(D_ERROR, env, lock, "invariant broken\n");
- return result;
-}
-
-/**
- * Returns lock "nesting": 0 for a top-lock and 1 for a sub-lock.
- */
-static enum clt_nesting_level cl_lock_nesting(const struct cl_lock *lock)
-{
- return cl_object_header(lock->cll_descr.cld_obj)->coh_nesting;
-}
-
-/**
- * Returns a set of counters for this lock, depending on a lock nesting.
- */
-static struct cl_thread_counters *cl_lock_counters(const struct lu_env *env,
- const struct cl_lock *lock)
-{
- struct cl_thread_info *info;
- enum clt_nesting_level nesting;
-
- info = cl_env_info(env);
- nesting = cl_lock_nesting(lock);
- LASSERT(nesting < ARRAY_SIZE(info->clt_counters));
- return &info->clt_counters[nesting];
-}
-
static void cl_lock_trace0(int level, const struct lu_env *env,
const char *prefix, const struct cl_lock *lock,
const char *func, const int line)
{
struct cl_object_header *h = cl_object_header(lock->cll_descr.cld_obj);
- CDEBUG(level, "%s: %p@(%d %p %d %d %d %d %d %lx)(%p/%d/%d) at %s():%d\n",
- prefix, lock, atomic_read(&lock->cll_ref),
- lock->cll_guarder, lock->cll_depth,
- lock->cll_state, lock->cll_error, lock->cll_holds,
- lock->cll_users, lock->cll_flags,
- env, h->coh_nesting, cl_lock_nr_mutexed(env),
- func, line);
+ CDEBUG(level, "%s: %p (%p/%d) at %s():%d\n",
+ prefix, lock, env, h->coh_nesting, func, line);
}
-
-#define cl_lock_trace(level, env, prefix, lock) \
+#define cl_lock_trace(level, env, prefix, lock) \
cl_lock_trace0(level, env, prefix, lock, __func__, __LINE__)
-#define RETIP ((unsigned long)__builtin_return_address(0))
-
-#ifdef CONFIG_LOCKDEP
-static struct lock_class_key cl_lock_key;
-
-static void cl_lock_lockdep_init(struct cl_lock *lock)
-{
- lockdep_set_class_and_name(lock, &cl_lock_key, "EXT");
-}
-
-static void cl_lock_lockdep_acquire(const struct lu_env *env,
- struct cl_lock *lock, __u32 enqflags)
-{
- cl_lock_counters(env, lock)->ctc_nr_locks_acquired++;
- lock_map_acquire(&lock->dep_map);
-}
-
-static void cl_lock_lockdep_release(const struct lu_env *env,
- struct cl_lock *lock)
-{
- cl_lock_counters(env, lock)->ctc_nr_locks_acquired--;
- lock_release(&lock->dep_map, 0, RETIP);
-}
-
-#else /* !CONFIG_LOCKDEP */
-
-static void cl_lock_lockdep_init(struct cl_lock *lock)
-{}
-static void cl_lock_lockdep_acquire(const struct lu_env *env,
- struct cl_lock *lock, __u32 enqflags)
-{}
-static void cl_lock_lockdep_release(const struct lu_env *env,
- struct cl_lock *lock)
-{}
-
-#endif /* !CONFIG_LOCKDEP */
-
/**
* Adds lock slice to the compound lock.
*
@@ -199,62 +80,10 @@
}
EXPORT_SYMBOL(cl_lock_slice_add);
-/**
- * Returns true iff a lock with the mode \a has provides at least the same
- * guarantees as a lock with the mode \a need.
- */
-int cl_lock_mode_match(enum cl_lock_mode has, enum cl_lock_mode need)
+void cl_lock_fini(const struct lu_env *env, struct cl_lock *lock)
{
- LINVRNT(need == CLM_READ || need == CLM_WRITE ||
- need == CLM_PHANTOM || need == CLM_GROUP);
- LINVRNT(has == CLM_READ || has == CLM_WRITE ||
- has == CLM_PHANTOM || has == CLM_GROUP);
- CLASSERT(CLM_PHANTOM < CLM_READ);
- CLASSERT(CLM_READ < CLM_WRITE);
- CLASSERT(CLM_WRITE < CLM_GROUP);
+ cl_lock_trace(D_DLMTRACE, env, "destroy lock", lock);
- if (has != CLM_GROUP)
- return need <= has;
- else
- return need == has;
-}
-EXPORT_SYMBOL(cl_lock_mode_match);
-
-/**
- * Returns true iff extent portions of lock descriptions match.
- */
-int cl_lock_ext_match(const struct cl_lock_descr *has,
- const struct cl_lock_descr *need)
-{
- return
- has->cld_start <= need->cld_start &&
- has->cld_end >= need->cld_end &&
- cl_lock_mode_match(has->cld_mode, need->cld_mode) &&
- (has->cld_mode != CLM_GROUP || has->cld_gid == need->cld_gid);
-}
-EXPORT_SYMBOL(cl_lock_ext_match);
-
-/**
- * Returns true iff a lock with the description \a has provides at least the
- * same guarantees as a lock with the description \a need.
- */
-int cl_lock_descr_match(const struct cl_lock_descr *has,
- const struct cl_lock_descr *need)
-{
- return
- cl_object_same(has->cld_obj, need->cld_obj) &&
- cl_lock_ext_match(has, need);
-}
-EXPORT_SYMBOL(cl_lock_descr_match);
-
-static void cl_lock_free(const struct lu_env *env, struct cl_lock *lock)
-{
- struct cl_object *obj = lock->cll_descr.cld_obj;
-
- LINVRNT(!cl_lock_is_mutexed(lock));
-
- cl_lock_trace(D_DLMTRACE, env, "free lock", lock);
- might_sleep();
while (!list_empty(&lock->cll_layers)) {
struct cl_lock_slice *slice;
@@ -263,350 +92,36 @@
list_del_init(lock->cll_layers.next);
slice->cls_ops->clo_fini(env, slice);
}
- CS_LOCK_DEC(obj, total);
- CS_LOCKSTATE_DEC(obj, lock->cll_state);
- lu_object_ref_del_at(&obj->co_lu, &lock->cll_obj_ref, "cl_lock", lock);
- cl_object_put(env, obj);
- lu_ref_fini(&lock->cll_reference);
- lu_ref_fini(&lock->cll_holders);
- mutex_destroy(&lock->cll_guard);
- kmem_cache_free(cl_lock_kmem, lock);
+ POISON(lock, 0x5a, sizeof(*lock));
}
+EXPORT_SYMBOL(cl_lock_fini);
-/**
- * Releases a reference on a lock.
- *
- * When last reference is released, lock is returned to the cache, unless it
- * is in cl_lock_state::CLS_FREEING state, in which case it is destroyed
- * immediately.
- *
- * \see cl_object_put(), cl_page_put()
- */
-void cl_lock_put(const struct lu_env *env, struct cl_lock *lock)
+int cl_lock_init(const struct lu_env *env, struct cl_lock *lock,
+ const struct cl_io *io)
{
- struct cl_object *obj;
+ struct cl_object *obj = lock->cll_descr.cld_obj;
+ struct cl_object *scan;
+ int result = 0;
- LINVRNT(cl_lock_invariant(env, lock));
- obj = lock->cll_descr.cld_obj;
- LINVRNT(obj);
+ /* Make sure cl_lock::cll_descr is initialized. */
+ LASSERT(obj);
- CDEBUG(D_TRACE, "releasing reference: %d %p %lu\n",
- atomic_read(&lock->cll_ref), lock, RETIP);
-
- if (atomic_dec_and_test(&lock->cll_ref)) {
- if (lock->cll_state == CLS_FREEING) {
- LASSERT(list_empty(&lock->cll_linkage));
- cl_lock_free(env, lock);
- }
- CS_LOCK_DEC(obj, busy);
- }
-}
-EXPORT_SYMBOL(cl_lock_put);
-
-/**
- * Acquires an additional reference to a lock.
- *
- * This can be called only by caller already possessing a reference to \a
- * lock.
- *
- * \see cl_object_get(), cl_page_get()
- */
-void cl_lock_get(struct cl_lock *lock)
-{
- LINVRNT(cl_lock_invariant(NULL, lock));
- CDEBUG(D_TRACE, "acquiring reference: %d %p %lu\n",
- atomic_read(&lock->cll_ref), lock, RETIP);
- atomic_inc(&lock->cll_ref);
-}
-EXPORT_SYMBOL(cl_lock_get);
-
-/**
- * Acquires a reference to a lock.
- *
- * This is much like cl_lock_get(), except that this function can be used to
- * acquire initial reference to the cached lock. Caller has to deal with all
- * possible races. Use with care!
- *
- * \see cl_page_get_trust()
- */
-void cl_lock_get_trust(struct cl_lock *lock)
-{
- CDEBUG(D_TRACE, "acquiring trusted reference: %d %p %lu\n",
- atomic_read(&lock->cll_ref), lock, RETIP);
- if (atomic_inc_return(&lock->cll_ref) == 1)
- CS_LOCK_INC(lock->cll_descr.cld_obj, busy);
-}
-EXPORT_SYMBOL(cl_lock_get_trust);
-
-/**
- * Helper function destroying the lock that wasn't completely initialized.
- *
- * Other threads can acquire references to the top-lock through its
- * sub-locks. Hence, it cannot be cl_lock_free()-ed immediately.
- */
-static void cl_lock_finish(const struct lu_env *env, struct cl_lock *lock)
-{
- cl_lock_mutex_get(env, lock);
- cl_lock_cancel(env, lock);
- cl_lock_delete(env, lock);
- cl_lock_mutex_put(env, lock);
- cl_lock_put(env, lock);
-}
-
-static struct cl_lock *cl_lock_alloc(const struct lu_env *env,
- struct cl_object *obj,
- const struct cl_io *io,
- const struct cl_lock_descr *descr)
-{
- struct cl_lock *lock;
- struct lu_object_header *head;
-
- lock = kmem_cache_zalloc(cl_lock_kmem, GFP_NOFS);
- if (lock) {
- atomic_set(&lock->cll_ref, 1);
- lock->cll_descr = *descr;
- lock->cll_state = CLS_NEW;
- cl_object_get(obj);
- lu_object_ref_add_at(&obj->co_lu, &lock->cll_obj_ref, "cl_lock",
- lock);
- INIT_LIST_HEAD(&lock->cll_layers);
- INIT_LIST_HEAD(&lock->cll_linkage);
- INIT_LIST_HEAD(&lock->cll_inclosure);
- lu_ref_init(&lock->cll_reference);
- lu_ref_init(&lock->cll_holders);
- mutex_init(&lock->cll_guard);
- lockdep_set_class(&lock->cll_guard, &cl_lock_guard_class);
- init_waitqueue_head(&lock->cll_wq);
- head = obj->co_lu.lo_header;
- CS_LOCKSTATE_INC(obj, CLS_NEW);
- CS_LOCK_INC(obj, total);
- CS_LOCK_INC(obj, create);
- cl_lock_lockdep_init(lock);
- list_for_each_entry(obj, &head->loh_layers, co_lu.lo_linkage) {
- int err;
-
- err = obj->co_ops->coo_lock_init(env, obj, lock, io);
- if (err != 0) {
- cl_lock_finish(env, lock);
- lock = ERR_PTR(err);
- break;
- }
- }
- } else
- lock = ERR_PTR(-ENOMEM);
- return lock;
-}
-
-/**
- * Transfer the lock into INTRANSIT state and return the original state.
- *
- * \pre state: CLS_CACHED, CLS_HELD or CLS_ENQUEUED
- * \post state: CLS_INTRANSIT
- * \see CLS_INTRANSIT
- */
-static enum cl_lock_state cl_lock_intransit(const struct lu_env *env,
- struct cl_lock *lock)
-{
- enum cl_lock_state state = lock->cll_state;
-
- LASSERT(cl_lock_is_mutexed(lock));
- LASSERT(state != CLS_INTRANSIT);
- LASSERTF(state >= CLS_ENQUEUED && state <= CLS_CACHED,
- "Malformed lock state %d.\n", state);
-
- cl_lock_state_set(env, lock, CLS_INTRANSIT);
- lock->cll_intransit_owner = current;
- cl_lock_hold_add(env, lock, "intransit", current);
- return state;
-}
-
-/**
- * Exit the intransit state and restore the lock state to the original state
- */
-static void cl_lock_extransit(const struct lu_env *env, struct cl_lock *lock,
- enum cl_lock_state state)
-{
- LASSERT(cl_lock_is_mutexed(lock));
- LASSERT(lock->cll_state == CLS_INTRANSIT);
- LASSERT(state != CLS_INTRANSIT);
- LASSERT(lock->cll_intransit_owner == current);
-
- lock->cll_intransit_owner = NULL;
- cl_lock_state_set(env, lock, state);
- cl_lock_unhold(env, lock, "intransit", current);
-}
-
-/**
- * Checking whether the lock is intransit state
- */
-int cl_lock_is_intransit(struct cl_lock *lock)
-{
- LASSERT(cl_lock_is_mutexed(lock));
- return lock->cll_state == CLS_INTRANSIT &&
- lock->cll_intransit_owner != current;
-}
-EXPORT_SYMBOL(cl_lock_is_intransit);
-/**
- * Returns true iff lock is "suitable" for given io. E.g., locks acquired by
- * truncate and O_APPEND cannot be reused for read/non-append-write, as they
- * cover multiple stripes and can trigger cascading timeouts.
- */
-static int cl_lock_fits_into(const struct lu_env *env,
- const struct cl_lock *lock,
- const struct cl_lock_descr *need,
- const struct cl_io *io)
-{
- const struct cl_lock_slice *slice;
-
- LINVRNT(cl_lock_invariant_trusted(env, lock));
- list_for_each_entry(slice, &lock->cll_layers, cls_linkage) {
- if (slice->cls_ops->clo_fits_into &&
- !slice->cls_ops->clo_fits_into(env, slice, need, io))
- return 0;
- }
- return 1;
-}
-
-static struct cl_lock *cl_lock_lookup(const struct lu_env *env,
- struct cl_object *obj,
- const struct cl_io *io,
- const struct cl_lock_descr *need)
-{
- struct cl_lock *lock;
- struct cl_object_header *head;
-
- head = cl_object_header(obj);
- assert_spin_locked(&head->coh_lock_guard);
- CS_LOCK_INC(obj, lookup);
- list_for_each_entry(lock, &head->coh_locks, cll_linkage) {
- int matched;
-
- matched = cl_lock_ext_match(&lock->cll_descr, need) &&
- lock->cll_state < CLS_FREEING &&
- lock->cll_error == 0 &&
- !(lock->cll_flags & CLF_CANCELLED) &&
- cl_lock_fits_into(env, lock, need, io);
- CDEBUG(D_DLMTRACE, "has: "DDESCR"(%d) need: "DDESCR": %d\n",
- PDESCR(&lock->cll_descr), lock->cll_state, PDESCR(need),
- matched);
- if (matched) {
- cl_lock_get_trust(lock);
- CS_LOCK_INC(obj, hit);
- return lock;
+ INIT_LIST_HEAD(&lock->cll_layers);
+ list_for_each_entry(scan, &obj->co_lu.lo_header->loh_layers,
+ co_lu.lo_linkage) {
+ result = scan->co_ops->coo_lock_init(env, scan, lock, io);
+ if (result != 0) {
+ cl_lock_fini(env, lock);
+ break;
}
}
- return NULL;
+
+ return result;
}
+EXPORT_SYMBOL(cl_lock_init);
/**
- * Returns a lock matching description \a need.
- *
- * This is the main entry point into the cl_lock caching interface. First, a
- * cache (implemented as a per-object linked list) is consulted. If lock is
- * found there, it is returned immediately. Otherwise new lock is allocated
- * and returned. In any case, additional reference to lock is acquired.
- *
- * \see cl_object_find(), cl_page_find()
- */
-static struct cl_lock *cl_lock_find(const struct lu_env *env,
- const struct cl_io *io,
- const struct cl_lock_descr *need)
-{
- struct cl_object_header *head;
- struct cl_object *obj;
- struct cl_lock *lock;
-
- obj = need->cld_obj;
- head = cl_object_header(obj);
-
- spin_lock(&head->coh_lock_guard);
- lock = cl_lock_lookup(env, obj, io, need);
- spin_unlock(&head->coh_lock_guard);
-
- if (!lock) {
- lock = cl_lock_alloc(env, obj, io, need);
- if (!IS_ERR(lock)) {
- struct cl_lock *ghost;
-
- spin_lock(&head->coh_lock_guard);
- ghost = cl_lock_lookup(env, obj, io, need);
- if (!ghost) {
- cl_lock_get_trust(lock);
- list_add_tail(&lock->cll_linkage,
- &head->coh_locks);
- spin_unlock(&head->coh_lock_guard);
- CS_LOCK_INC(obj, busy);
- } else {
- spin_unlock(&head->coh_lock_guard);
- /*
- * Other threads can acquire references to the
- * top-lock through its sub-locks. Hence, it
- * cannot be cl_lock_free()-ed immediately.
- */
- cl_lock_finish(env, lock);
- lock = ghost;
- }
- }
- }
- return lock;
-}
-
-/**
- * Returns existing lock matching given description. This is similar to
- * cl_lock_find() except that no new lock is created, and returned lock is
- * guaranteed to be in enum cl_lock_state::CLS_HELD state.
- */
-struct cl_lock *cl_lock_peek(const struct lu_env *env, const struct cl_io *io,
- const struct cl_lock_descr *need,
- const char *scope, const void *source)
-{
- struct cl_object_header *head;
- struct cl_object *obj;
- struct cl_lock *lock;
-
- obj = need->cld_obj;
- head = cl_object_header(obj);
-
- do {
- spin_lock(&head->coh_lock_guard);
- lock = cl_lock_lookup(env, obj, io, need);
- spin_unlock(&head->coh_lock_guard);
- if (!lock)
- return NULL;
-
- cl_lock_mutex_get(env, lock);
- if (lock->cll_state == CLS_INTRANSIT)
- /* Don't care return value. */
- cl_lock_state_wait(env, lock);
- if (lock->cll_state == CLS_FREEING) {
- cl_lock_mutex_put(env, lock);
- cl_lock_put(env, lock);
- lock = NULL;
- }
- } while (!lock);
-
- cl_lock_hold_add(env, lock, scope, source);
- cl_lock_user_add(env, lock);
- if (lock->cll_state == CLS_CACHED)
- cl_use_try(env, lock, 1);
- if (lock->cll_state == CLS_HELD) {
- cl_lock_mutex_put(env, lock);
- cl_lock_lockdep_acquire(env, lock, 0);
- cl_lock_put(env, lock);
- } else {
- cl_unuse_try(env, lock);
- cl_lock_unhold(env, lock, scope, source);
- cl_lock_mutex_put(env, lock);
- cl_lock_put(env, lock);
- lock = NULL;
- }
-
- return lock;
-}
-EXPORT_SYMBOL(cl_lock_peek);
-
-/**
- * Returns a slice within a lock, corresponding to the given layer in the
+ * Returns a slice with a lock, corresponding to the given layer in the
* device stack.
*
* \see cl_page_at()
@@ -616,8 +131,6 @@
{
const struct cl_lock_slice *slice;
- LINVRNT(cl_lock_invariant_trusted(NULL, lock));
-
list_for_each_entry(slice, &lock->cll_layers, cls_linkage) {
if (slice->cls_obj->co_lu.lo_dev->ld_type == dtype)
return slice;
@@ -626,1537 +139,96 @@
}
EXPORT_SYMBOL(cl_lock_at);
-static void cl_lock_mutex_tail(const struct lu_env *env, struct cl_lock *lock)
-{
- struct cl_thread_counters *counters;
-
- counters = cl_lock_counters(env, lock);
- lock->cll_depth++;
- counters->ctc_nr_locks_locked++;
- lu_ref_add(&counters->ctc_locks_locked, "cll_guard", lock);
- cl_lock_trace(D_TRACE, env, "got mutex", lock);
-}
-
-/**
- * Locks cl_lock object.
- *
- * This is used to manipulate cl_lock fields, and to serialize state
- * transitions in the lock state machine.
- *
- * \post cl_lock_is_mutexed(lock)
- *
- * \see cl_lock_mutex_put()
- */
-void cl_lock_mutex_get(const struct lu_env *env, struct cl_lock *lock)
-{
- LINVRNT(cl_lock_invariant(env, lock));
-
- if (lock->cll_guarder == current) {
- LINVRNT(cl_lock_is_mutexed(lock));
- LINVRNT(lock->cll_depth > 0);
- } else {
- struct cl_object_header *hdr;
- struct cl_thread_info *info;
- int i;
-
- LINVRNT(lock->cll_guarder != current);
- hdr = cl_object_header(lock->cll_descr.cld_obj);
- /*
- * Check that mutices are taken in the bottom-to-top order.
- */
- info = cl_env_info(env);
- for (i = 0; i < hdr->coh_nesting; ++i)
- LASSERT(info->clt_counters[i].ctc_nr_locks_locked == 0);
- mutex_lock_nested(&lock->cll_guard, hdr->coh_nesting);
- lock->cll_guarder = current;
- LINVRNT(lock->cll_depth == 0);
- }
- cl_lock_mutex_tail(env, lock);
-}
-EXPORT_SYMBOL(cl_lock_mutex_get);
-
-/**
- * Try-locks cl_lock object.
- *
- * \retval 0 \a lock was successfully locked
- *
- * \retval -EBUSY \a lock cannot be locked right now
- *
- * \post ergo(result == 0, cl_lock_is_mutexed(lock))
- *
- * \see cl_lock_mutex_get()
- */
-static int cl_lock_mutex_try(const struct lu_env *env, struct cl_lock *lock)
-{
- int result;
-
- LINVRNT(cl_lock_invariant_trusted(env, lock));
-
- result = 0;
- if (lock->cll_guarder == current) {
- LINVRNT(lock->cll_depth > 0);
- cl_lock_mutex_tail(env, lock);
- } else if (mutex_trylock(&lock->cll_guard)) {
- LINVRNT(lock->cll_depth == 0);
- lock->cll_guarder = current;
- cl_lock_mutex_tail(env, lock);
- } else
- result = -EBUSY;
- return result;
-}
-
-/**
- {* Unlocks cl_lock object.
- *
- * \pre cl_lock_is_mutexed(lock)
- *
- * \see cl_lock_mutex_get()
- */
-void cl_lock_mutex_put(const struct lu_env *env, struct cl_lock *lock)
-{
- struct cl_thread_counters *counters;
-
- LINVRNT(cl_lock_invariant(env, lock));
- LINVRNT(cl_lock_is_mutexed(lock));
- LINVRNT(lock->cll_guarder == current);
- LINVRNT(lock->cll_depth > 0);
-
- counters = cl_lock_counters(env, lock);
- LINVRNT(counters->ctc_nr_locks_locked > 0);
-
- cl_lock_trace(D_TRACE, env, "put mutex", lock);
- lu_ref_del(&counters->ctc_locks_locked, "cll_guard", lock);
- counters->ctc_nr_locks_locked--;
- if (--lock->cll_depth == 0) {
- lock->cll_guarder = NULL;
- mutex_unlock(&lock->cll_guard);
- }
-}
-EXPORT_SYMBOL(cl_lock_mutex_put);
-
-/**
- * Returns true iff lock's mutex is owned by the current thread.
- */
-int cl_lock_is_mutexed(struct cl_lock *lock)
-{
- return lock->cll_guarder == current;
-}
-EXPORT_SYMBOL(cl_lock_is_mutexed);
-
-/**
- * Returns number of cl_lock mutices held by the current thread (environment).
- */
-int cl_lock_nr_mutexed(const struct lu_env *env)
-{
- struct cl_thread_info *info;
- int i;
- int locked;
-
- /*
- * NOTE: if summation across all nesting levels (currently 2) proves
- * too expensive, a summary counter can be added to
- * struct cl_thread_info.
- */
- info = cl_env_info(env);
- for (i = 0, locked = 0; i < ARRAY_SIZE(info->clt_counters); ++i)
- locked += info->clt_counters[i].ctc_nr_locks_locked;
- return locked;
-}
-EXPORT_SYMBOL(cl_lock_nr_mutexed);
-
-static void cl_lock_cancel0(const struct lu_env *env, struct cl_lock *lock)
-{
- LINVRNT(cl_lock_is_mutexed(lock));
- LINVRNT(cl_lock_invariant(env, lock));
- if (!(lock->cll_flags & CLF_CANCELLED)) {
- const struct cl_lock_slice *slice;
-
- lock->cll_flags |= CLF_CANCELLED;
- list_for_each_entry_reverse(slice, &lock->cll_layers,
- cls_linkage) {
- if (slice->cls_ops->clo_cancel)
- slice->cls_ops->clo_cancel(env, slice);
- }
- }
-}
-
-static void cl_lock_delete0(const struct lu_env *env, struct cl_lock *lock)
-{
- struct cl_object_header *head;
- const struct cl_lock_slice *slice;
-
- LINVRNT(cl_lock_is_mutexed(lock));
- LINVRNT(cl_lock_invariant(env, lock));
-
- if (lock->cll_state < CLS_FREEING) {
- bool in_cache;
-
- LASSERT(lock->cll_state != CLS_INTRANSIT);
- cl_lock_state_set(env, lock, CLS_FREEING);
-
- head = cl_object_header(lock->cll_descr.cld_obj);
-
- spin_lock(&head->coh_lock_guard);
- in_cache = !list_empty(&lock->cll_linkage);
- if (in_cache)
- list_del_init(&lock->cll_linkage);
- spin_unlock(&head->coh_lock_guard);
-
- if (in_cache) /* coh_locks cache holds a refcount. */
- cl_lock_put(env, lock);
-
- /*
- * From now on, no new references to this lock can be acquired
- * by cl_lock_lookup().
- */
- list_for_each_entry_reverse(slice, &lock->cll_layers,
- cls_linkage) {
- if (slice->cls_ops->clo_delete)
- slice->cls_ops->clo_delete(env, slice);
- }
- /*
- * From now on, no new references to this lock can be acquired
- * by layer-specific means (like a pointer from struct
- * ldlm_lock in osc, or a pointer from top-lock to sub-lock in
- * lov).
- *
- * Lock will be finally freed in cl_lock_put() when last of
- * existing references goes away.
- */
- }
-}
-
-/**
- * Mod(ifie)s cl_lock::cll_holds counter for a given lock. Also, for a
- * top-lock (nesting == 0) accounts for this modification in the per-thread
- * debugging counters. Sub-lock holds can be released by a thread different
- * from one that acquired it.
- */
-static void cl_lock_hold_mod(const struct lu_env *env, struct cl_lock *lock,
- int delta)
-{
- struct cl_thread_counters *counters;
- enum clt_nesting_level nesting;
-
- lock->cll_holds += delta;
- nesting = cl_lock_nesting(lock);
- if (nesting == CNL_TOP) {
- counters = &cl_env_info(env)->clt_counters[CNL_TOP];
- counters->ctc_nr_held += delta;
- LASSERT(counters->ctc_nr_held >= 0);
- }
-}
-
-/**
- * Mod(ifie)s cl_lock::cll_users counter for a given lock. See
- * cl_lock_hold_mod() for the explanation of the debugging code.
- */
-static void cl_lock_used_mod(const struct lu_env *env, struct cl_lock *lock,
- int delta)
-{
- struct cl_thread_counters *counters;
- enum clt_nesting_level nesting;
-
- lock->cll_users += delta;
- nesting = cl_lock_nesting(lock);
- if (nesting == CNL_TOP) {
- counters = &cl_env_info(env)->clt_counters[CNL_TOP];
- counters->ctc_nr_used += delta;
- LASSERT(counters->ctc_nr_used >= 0);
- }
-}
-
-void cl_lock_hold_release(const struct lu_env *env, struct cl_lock *lock,
- const char *scope, const void *source)
-{
- LINVRNT(cl_lock_is_mutexed(lock));
- LINVRNT(cl_lock_invariant(env, lock));
- LASSERT(lock->cll_holds > 0);
-
- cl_lock_trace(D_DLMTRACE, env, "hold release lock", lock);
- lu_ref_del(&lock->cll_holders, scope, source);
- cl_lock_hold_mod(env, lock, -1);
- if (lock->cll_holds == 0) {
- CL_LOCK_ASSERT(lock->cll_state != CLS_HELD, env, lock);
- if (lock->cll_descr.cld_mode == CLM_PHANTOM ||
- lock->cll_descr.cld_mode == CLM_GROUP ||
- lock->cll_state != CLS_CACHED)
- /*
- * If lock is still phantom or grouplock when user is
- * done with it---destroy the lock.
- */
- lock->cll_flags |= CLF_CANCELPEND|CLF_DOOMED;
- if (lock->cll_flags & CLF_CANCELPEND) {
- lock->cll_flags &= ~CLF_CANCELPEND;
- cl_lock_cancel0(env, lock);
- }
- if (lock->cll_flags & CLF_DOOMED) {
- /* no longer doomed: it's dead... Jim. */
- lock->cll_flags &= ~CLF_DOOMED;
- cl_lock_delete0(env, lock);
- }
- }
-}
-EXPORT_SYMBOL(cl_lock_hold_release);
-
-/**
- * Waits until lock state is changed.
- *
- * This function is called with cl_lock mutex locked, atomically releases
- * mutex and goes to sleep, waiting for a lock state change (signaled by
- * cl_lock_signal()), and re-acquires the mutex before return.
- *
- * This function is used to wait until lock state machine makes some progress
- * and to emulate synchronous operations on top of asynchronous lock
- * interface.
- *
- * \retval -EINTR wait was interrupted
- *
- * \retval 0 wait wasn't interrupted
- *
- * \pre cl_lock_is_mutexed(lock)
- *
- * \see cl_lock_signal()
- */
-int cl_lock_state_wait(const struct lu_env *env, struct cl_lock *lock)
-{
- wait_queue_t waiter;
- sigset_t blocked;
- int result;
-
- LINVRNT(cl_lock_is_mutexed(lock));
- LINVRNT(cl_lock_invariant(env, lock));
- LASSERT(lock->cll_depth == 1);
- LASSERT(lock->cll_state != CLS_FREEING); /* too late to wait */
-
- cl_lock_trace(D_DLMTRACE, env, "state wait lock", lock);
- result = lock->cll_error;
- if (result == 0) {
- /* To avoid being interrupted by the 'non-fatal' signals
- * (SIGCHLD, for instance), we'd block them temporarily.
- * LU-305
- */
- blocked = cfs_block_sigsinv(LUSTRE_FATAL_SIGS);
-
- init_waitqueue_entry(&waiter, current);
- add_wait_queue(&lock->cll_wq, &waiter);
- set_current_state(TASK_INTERRUPTIBLE);
- cl_lock_mutex_put(env, lock);
-
- LASSERT(cl_lock_nr_mutexed(env) == 0);
-
- /* Returning ERESTARTSYS instead of EINTR so syscalls
- * can be restarted if signals are pending here
- */
- result = -ERESTARTSYS;
- if (likely(!OBD_FAIL_CHECK(OBD_FAIL_LOCK_STATE_WAIT_INTR))) {
- schedule();
- if (!cfs_signal_pending())
- result = 0;
- }
-
- cl_lock_mutex_get(env, lock);
- set_current_state(TASK_RUNNING);
- remove_wait_queue(&lock->cll_wq, &waiter);
-
- /* Restore old blocked signals */
- cfs_restore_sigs(blocked);
- }
- return result;
-}
-EXPORT_SYMBOL(cl_lock_state_wait);
-
-static void cl_lock_state_signal(const struct lu_env *env, struct cl_lock *lock,
- enum cl_lock_state state)
-{
- const struct cl_lock_slice *slice;
-
- LINVRNT(cl_lock_is_mutexed(lock));
- LINVRNT(cl_lock_invariant(env, lock));
-
- list_for_each_entry(slice, &lock->cll_layers, cls_linkage)
- if (slice->cls_ops->clo_state)
- slice->cls_ops->clo_state(env, slice, state);
- wake_up_all(&lock->cll_wq);
-}
-
-/**
- * Notifies waiters that lock state changed.
- *
- * Wakes up all waiters sleeping in cl_lock_state_wait(), also notifies all
- * layers about state change by calling cl_lock_operations::clo_state()
- * top-to-bottom.
- */
-void cl_lock_signal(const struct lu_env *env, struct cl_lock *lock)
-{
- cl_lock_trace(D_DLMTRACE, env, "state signal lock", lock);
- cl_lock_state_signal(env, lock, lock->cll_state);
-}
-EXPORT_SYMBOL(cl_lock_signal);
-
-/**
- * Changes lock state.
- *
- * This function is invoked to notify layers that lock state changed, possible
- * as a result of an asynchronous event such as call-back reception.
- *
- * \post lock->cll_state == state
- *
- * \see cl_lock_operations::clo_state()
- */
-void cl_lock_state_set(const struct lu_env *env, struct cl_lock *lock,
- enum cl_lock_state state)
-{
- LASSERT(lock->cll_state <= state ||
- (lock->cll_state == CLS_CACHED &&
- (state == CLS_HELD || /* lock found in cache */
- state == CLS_NEW || /* sub-lock canceled */
- state == CLS_INTRANSIT)) ||
- /* lock is in transit state */
- lock->cll_state == CLS_INTRANSIT);
-
- if (lock->cll_state != state) {
- CS_LOCKSTATE_DEC(lock->cll_descr.cld_obj, lock->cll_state);
- CS_LOCKSTATE_INC(lock->cll_descr.cld_obj, state);
-
- cl_lock_state_signal(env, lock, state);
- lock->cll_state = state;
- }
-}
-EXPORT_SYMBOL(cl_lock_state_set);
-
-static int cl_unuse_try_internal(const struct lu_env *env, struct cl_lock *lock)
-{
- const struct cl_lock_slice *slice;
- int result;
-
- do {
- result = 0;
-
- LINVRNT(cl_lock_is_mutexed(lock));
- LINVRNT(cl_lock_invariant(env, lock));
- LASSERT(lock->cll_state == CLS_INTRANSIT);
-
- result = -ENOSYS;
- list_for_each_entry_reverse(slice, &lock->cll_layers,
- cls_linkage) {
- if (slice->cls_ops->clo_unuse) {
- result = slice->cls_ops->clo_unuse(env, slice);
- if (result != 0)
- break;
- }
- }
- LASSERT(result != -ENOSYS);
- } while (result == CLO_REPEAT);
-
- return result;
-}
-
-/**
- * Yanks lock from the cache (cl_lock_state::CLS_CACHED state) by calling
- * cl_lock_operations::clo_use() top-to-bottom to notify layers.
- * @atomic = 1, it must unuse the lock to recovery the lock to keep the
- * use process atomic
- */
-int cl_use_try(const struct lu_env *env, struct cl_lock *lock, int atomic)
-{
- const struct cl_lock_slice *slice;
- int result;
- enum cl_lock_state state;
-
- cl_lock_trace(D_DLMTRACE, env, "use lock", lock);
-
- LASSERT(lock->cll_state == CLS_CACHED);
- if (lock->cll_error)
- return lock->cll_error;
-
- result = -ENOSYS;
- state = cl_lock_intransit(env, lock);
- list_for_each_entry(slice, &lock->cll_layers, cls_linkage) {
- if (slice->cls_ops->clo_use) {
- result = slice->cls_ops->clo_use(env, slice);
- if (result != 0)
- break;
- }
- }
- LASSERT(result != -ENOSYS);
-
- LASSERTF(lock->cll_state == CLS_INTRANSIT, "Wrong state %d.\n",
- lock->cll_state);
-
- if (result == 0) {
- state = CLS_HELD;
- } else {
- if (result == -ESTALE) {
- /*
- * ESTALE means sublock being cancelled
- * at this time, and set lock state to
- * be NEW here and ask the caller to repeat.
- */
- state = CLS_NEW;
- result = CLO_REPEAT;
- }
-
- /* @atomic means back-off-on-failure. */
- if (atomic) {
- int rc;
-
- rc = cl_unuse_try_internal(env, lock);
- /* Vet the results. */
- if (rc < 0 && result > 0)
- result = rc;
- }
-
- }
- cl_lock_extransit(env, lock, state);
- return result;
-}
-EXPORT_SYMBOL(cl_use_try);
-
-/**
- * Helper for cl_enqueue_try() that calls ->clo_enqueue() across all layers
- * top-to-bottom.
- */
-static int cl_enqueue_kick(const struct lu_env *env,
- struct cl_lock *lock,
- struct cl_io *io, __u32 flags)
-{
- int result;
- const struct cl_lock_slice *slice;
-
- result = -ENOSYS;
- list_for_each_entry(slice, &lock->cll_layers, cls_linkage) {
- if (slice->cls_ops->clo_enqueue) {
- result = slice->cls_ops->clo_enqueue(env,
- slice, io, flags);
- if (result != 0)
- break;
- }
- }
- LASSERT(result != -ENOSYS);
- return result;
-}
-
-/**
- * Tries to enqueue a lock.
- *
- * This function is called repeatedly by cl_enqueue() until either lock is
- * enqueued, or error occurs. This function does not block waiting for
- * networking communication to complete.
- *
- * \post ergo(result == 0, lock->cll_state == CLS_ENQUEUED ||
- * lock->cll_state == CLS_HELD)
- *
- * \see cl_enqueue() cl_lock_operations::clo_enqueue()
- * \see cl_lock_state::CLS_ENQUEUED
- */
-int cl_enqueue_try(const struct lu_env *env, struct cl_lock *lock,
- struct cl_io *io, __u32 flags)
-{
- int result;
-
- cl_lock_trace(D_DLMTRACE, env, "enqueue lock", lock);
- do {
- LINVRNT(cl_lock_is_mutexed(lock));
-
- result = lock->cll_error;
- if (result != 0)
- break;
-
- switch (lock->cll_state) {
- case CLS_NEW:
- cl_lock_state_set(env, lock, CLS_QUEUING);
- /* fall-through */
- case CLS_QUEUING:
- /* kick layers. */
- result = cl_enqueue_kick(env, lock, io, flags);
- /* For AGL case, the cl_lock::cll_state may
- * become CLS_HELD already.
- */
- if (result == 0 && lock->cll_state == CLS_QUEUING)
- cl_lock_state_set(env, lock, CLS_ENQUEUED);
- break;
- case CLS_INTRANSIT:
- LASSERT(cl_lock_is_intransit(lock));
- result = CLO_WAIT;
- break;
- case CLS_CACHED:
- /* yank lock from the cache. */
- result = cl_use_try(env, lock, 0);
- break;
- case CLS_ENQUEUED:
- case CLS_HELD:
- result = 0;
- break;
- default:
- case CLS_FREEING:
- /*
- * impossible, only held locks with increased
- * ->cll_holds can be enqueued, and they cannot be
- * freed.
- */
- LBUG();
- }
- } while (result == CLO_REPEAT);
- return result;
-}
-EXPORT_SYMBOL(cl_enqueue_try);
-
-/**
- * Cancel the conflicting lock found during previous enqueue.
- *
- * \retval 0 conflicting lock has been canceled.
- * \retval -ve error code.
- */
-int cl_lock_enqueue_wait(const struct lu_env *env,
- struct cl_lock *lock,
- int keep_mutex)
-{
- struct cl_lock *conflict;
- int rc = 0;
-
- LASSERT(cl_lock_is_mutexed(lock));
- LASSERT(lock->cll_state == CLS_QUEUING);
- LASSERT(lock->cll_conflict);
-
- conflict = lock->cll_conflict;
- lock->cll_conflict = NULL;
-
- cl_lock_mutex_put(env, lock);
- LASSERT(cl_lock_nr_mutexed(env) == 0);
-
- cl_lock_mutex_get(env, conflict);
- cl_lock_trace(D_DLMTRACE, env, "enqueue wait", conflict);
- cl_lock_cancel(env, conflict);
- cl_lock_delete(env, conflict);
-
- while (conflict->cll_state != CLS_FREEING) {
- rc = cl_lock_state_wait(env, conflict);
- if (rc != 0)
- break;
- }
- cl_lock_mutex_put(env, conflict);
- lu_ref_del(&conflict->cll_reference, "cancel-wait", lock);
- cl_lock_put(env, conflict);
-
- if (keep_mutex)
- cl_lock_mutex_get(env, lock);
-
- LASSERT(rc <= 0);
- return rc;
-}
-EXPORT_SYMBOL(cl_lock_enqueue_wait);
-
-static int cl_enqueue_locked(const struct lu_env *env, struct cl_lock *lock,
- struct cl_io *io, __u32 enqflags)
-{
- int result;
-
- LINVRNT(cl_lock_is_mutexed(lock));
- LINVRNT(cl_lock_invariant(env, lock));
- LASSERT(lock->cll_holds > 0);
-
- cl_lock_user_add(env, lock);
- do {
- result = cl_enqueue_try(env, lock, io, enqflags);
- if (result == CLO_WAIT) {
- if (lock->cll_conflict)
- result = cl_lock_enqueue_wait(env, lock, 1);
- else
- result = cl_lock_state_wait(env, lock);
- if (result == 0)
- continue;
- }
- break;
- } while (1);
- if (result != 0)
- cl_unuse_try(env, lock);
- LASSERT(ergo(result == 0 && !(enqflags & CEF_AGL),
- lock->cll_state == CLS_ENQUEUED ||
- lock->cll_state == CLS_HELD));
- return result;
-}
-
-/**
- * Tries to unlock a lock.
- *
- * This function is called to release underlying resource:
- * 1. for top lock, the resource is sublocks it held;
- * 2. for sublock, the resource is the reference to dlmlock.
- *
- * cl_unuse_try is a one-shot operation, so it must NOT return CLO_WAIT.
- *
- * \see cl_unuse() cl_lock_operations::clo_unuse()
- * \see cl_lock_state::CLS_CACHED
- */
-int cl_unuse_try(const struct lu_env *env, struct cl_lock *lock)
-{
- int result;
- enum cl_lock_state state = CLS_NEW;
-
- cl_lock_trace(D_DLMTRACE, env, "unuse lock", lock);
-
- if (lock->cll_users > 1) {
- cl_lock_user_del(env, lock);
- return 0;
- }
-
- /* Only if the lock is in CLS_HELD or CLS_ENQUEUED state, it can hold
- * underlying resources.
- */
- if (!(lock->cll_state == CLS_HELD || lock->cll_state == CLS_ENQUEUED)) {
- cl_lock_user_del(env, lock);
- return 0;
- }
-
- /*
- * New lock users (->cll_users) are not protecting unlocking
- * from proceeding. From this point, lock eventually reaches
- * CLS_CACHED, is reinitialized to CLS_NEW or fails into
- * CLS_FREEING.
- */
- state = cl_lock_intransit(env, lock);
-
- result = cl_unuse_try_internal(env, lock);
- LASSERT(lock->cll_state == CLS_INTRANSIT);
- LASSERT(result != CLO_WAIT);
- cl_lock_user_del(env, lock);
- if (result == 0 || result == -ESTALE) {
- /*
- * Return lock back to the cache. This is the only
- * place where lock is moved into CLS_CACHED state.
- *
- * If one of ->clo_unuse() methods returned -ESTALE, lock
- * cannot be placed into cache and has to be
- * re-initialized. This happens e.g., when a sub-lock was
- * canceled while unlocking was in progress.
- */
- if (state == CLS_HELD && result == 0)
- state = CLS_CACHED;
- else
- state = CLS_NEW;
- cl_lock_extransit(env, lock, state);
-
- /*
- * Hide -ESTALE error.
- * If the lock is a glimpse lock, and it has multiple
- * stripes. Assuming that one of its sublock returned -ENAVAIL,
- * and other sublocks are matched write locks. In this case,
- * we can't set this lock to error because otherwise some of
- * its sublocks may not be canceled. This causes some dirty
- * pages won't be written to OSTs. -jay
- */
- result = 0;
- } else {
- CERROR("result = %d, this is unlikely!\n", result);
- state = CLS_NEW;
- cl_lock_extransit(env, lock, state);
- }
- return result ?: lock->cll_error;
-}
-EXPORT_SYMBOL(cl_unuse_try);
-
-static void cl_unuse_locked(const struct lu_env *env, struct cl_lock *lock)
-{
- int result;
-
- result = cl_unuse_try(env, lock);
- if (result)
- CL_LOCK_DEBUG(D_ERROR, env, lock, "unuse return %d\n", result);
-}
-
-/**
- * Unlocks a lock.
- */
-void cl_unuse(const struct lu_env *env, struct cl_lock *lock)
-{
- cl_lock_mutex_get(env, lock);
- cl_unuse_locked(env, lock);
- cl_lock_mutex_put(env, lock);
- cl_lock_lockdep_release(env, lock);
-}
-EXPORT_SYMBOL(cl_unuse);
-
-/**
- * Tries to wait for a lock.
- *
- * This function is called repeatedly by cl_wait() until either lock is
- * granted, or error occurs. This function does not block waiting for network
- * communication to complete.
- *
- * \see cl_wait() cl_lock_operations::clo_wait()
- * \see cl_lock_state::CLS_HELD
- */
-int cl_wait_try(const struct lu_env *env, struct cl_lock *lock)
-{
- const struct cl_lock_slice *slice;
- int result;
-
- cl_lock_trace(D_DLMTRACE, env, "wait lock try", lock);
- do {
- LINVRNT(cl_lock_is_mutexed(lock));
- LINVRNT(cl_lock_invariant(env, lock));
- LASSERTF(lock->cll_state == CLS_QUEUING ||
- lock->cll_state == CLS_ENQUEUED ||
- lock->cll_state == CLS_HELD ||
- lock->cll_state == CLS_INTRANSIT,
- "lock state: %d\n", lock->cll_state);
- LASSERT(lock->cll_users > 0);
- LASSERT(lock->cll_holds > 0);
-
- result = lock->cll_error;
- if (result != 0)
- break;
-
- if (cl_lock_is_intransit(lock)) {
- result = CLO_WAIT;
- break;
- }
-
- if (lock->cll_state == CLS_HELD)
- /* nothing to do */
- break;
-
- result = -ENOSYS;
- list_for_each_entry(slice, &lock->cll_layers, cls_linkage) {
- if (slice->cls_ops->clo_wait) {
- result = slice->cls_ops->clo_wait(env, slice);
- if (result != 0)
- break;
- }
- }
- LASSERT(result != -ENOSYS);
- if (result == 0) {
- LASSERT(lock->cll_state != CLS_INTRANSIT);
- cl_lock_state_set(env, lock, CLS_HELD);
- }
- } while (result == CLO_REPEAT);
- return result;
-}
-EXPORT_SYMBOL(cl_wait_try);
-
-/**
- * Waits until enqueued lock is granted.
- *
- * \pre current thread or io owns a hold on the lock
- * \pre ergo(result == 0, lock->cll_state == CLS_ENQUEUED ||
- * lock->cll_state == CLS_HELD)
- *
- * \post ergo(result == 0, lock->cll_state == CLS_HELD)
- */
-int cl_wait(const struct lu_env *env, struct cl_lock *lock)
-{
- int result;
-
- cl_lock_mutex_get(env, lock);
-
- LINVRNT(cl_lock_invariant(env, lock));
- LASSERTF(lock->cll_state == CLS_ENQUEUED || lock->cll_state == CLS_HELD,
- "Wrong state %d\n", lock->cll_state);
- LASSERT(lock->cll_holds > 0);
-
- do {
- result = cl_wait_try(env, lock);
- if (result == CLO_WAIT) {
- result = cl_lock_state_wait(env, lock);
- if (result == 0)
- continue;
- }
- break;
- } while (1);
- if (result < 0) {
- cl_unuse_try(env, lock);
- cl_lock_lockdep_release(env, lock);
- }
- cl_lock_trace(D_DLMTRACE, env, "wait lock", lock);
- cl_lock_mutex_put(env, lock);
- LASSERT(ergo(result == 0, lock->cll_state == CLS_HELD));
- return result;
-}
-EXPORT_SYMBOL(cl_wait);
-
-/**
- * Executes cl_lock_operations::clo_weigh(), and sums results to estimate lock
- * value.
- */
-unsigned long cl_lock_weigh(const struct lu_env *env, struct cl_lock *lock)
-{
- const struct cl_lock_slice *slice;
- unsigned long pound;
- unsigned long ounce;
-
- LINVRNT(cl_lock_is_mutexed(lock));
- LINVRNT(cl_lock_invariant(env, lock));
-
- pound = 0;
- list_for_each_entry_reverse(slice, &lock->cll_layers, cls_linkage) {
- if (slice->cls_ops->clo_weigh) {
- ounce = slice->cls_ops->clo_weigh(env, slice);
- pound += ounce;
- if (pound < ounce) /* over-weight^Wflow */
- pound = ~0UL;
- }
- }
- return pound;
-}
-EXPORT_SYMBOL(cl_lock_weigh);
-
-/**
- * Notifies layers that lock description changed.
- *
- * The server can grant client a lock different from one that was requested
- * (e.g., larger in extent). This method is called when actually granted lock
- * description becomes known to let layers to accommodate for changed lock
- * description.
- *
- * \see cl_lock_operations::clo_modify()
- */
-int cl_lock_modify(const struct lu_env *env, struct cl_lock *lock,
- const struct cl_lock_descr *desc)
-{
- const struct cl_lock_slice *slice;
- struct cl_object *obj = lock->cll_descr.cld_obj;
- struct cl_object_header *hdr = cl_object_header(obj);
- int result;
-
- cl_lock_trace(D_DLMTRACE, env, "modify lock", lock);
- /* don't allow object to change */
- LASSERT(obj == desc->cld_obj);
- LINVRNT(cl_lock_is_mutexed(lock));
- LINVRNT(cl_lock_invariant(env, lock));
-
- list_for_each_entry_reverse(slice, &lock->cll_layers, cls_linkage) {
- if (slice->cls_ops->clo_modify) {
- result = slice->cls_ops->clo_modify(env, slice, desc);
- if (result != 0)
- return result;
- }
- }
- CL_LOCK_DEBUG(D_DLMTRACE, env, lock, " -> "DDESCR"@"DFID"\n",
- PDESCR(desc), PFID(lu_object_fid(&desc->cld_obj->co_lu)));
- /*
- * Just replace description in place. Nothing more is needed for
- * now. If locks were indexed according to their extent and/or mode,
- * that index would have to be updated here.
- */
- spin_lock(&hdr->coh_lock_guard);
- lock->cll_descr = *desc;
- spin_unlock(&hdr->coh_lock_guard);
- return 0;
-}
-EXPORT_SYMBOL(cl_lock_modify);
-
-/**
- * Initializes lock closure with a given origin.
- *
- * \see cl_lock_closure
- */
-void cl_lock_closure_init(const struct lu_env *env,
- struct cl_lock_closure *closure,
- struct cl_lock *origin, int wait)
-{
- LINVRNT(cl_lock_is_mutexed(origin));
- LINVRNT(cl_lock_invariant(env, origin));
-
- INIT_LIST_HEAD(&closure->clc_list);
- closure->clc_origin = origin;
- closure->clc_wait = wait;
- closure->clc_nr = 0;
-}
-EXPORT_SYMBOL(cl_lock_closure_init);
-
-/**
- * Builds a closure of \a lock.
- *
- * Building of a closure consists of adding initial lock (\a lock) into it,
- * and calling cl_lock_operations::clo_closure() methods of \a lock. These
- * methods might call cl_lock_closure_build() recursively again, adding more
- * locks to the closure, etc.
- *
- * \see cl_lock_closure
- */
-int cl_lock_closure_build(const struct lu_env *env, struct cl_lock *lock,
- struct cl_lock_closure *closure)
-{
- const struct cl_lock_slice *slice;
- int result;
-
- LINVRNT(cl_lock_is_mutexed(closure->clc_origin));
- LINVRNT(cl_lock_invariant(env, closure->clc_origin));
-
- result = cl_lock_enclosure(env, lock, closure);
- if (result == 0) {
- list_for_each_entry(slice, &lock->cll_layers, cls_linkage) {
- if (slice->cls_ops->clo_closure) {
- result = slice->cls_ops->clo_closure(env, slice,
- closure);
- if (result != 0)
- break;
- }
- }
- }
- if (result != 0)
- cl_lock_disclosure(env, closure);
- return result;
-}
-EXPORT_SYMBOL(cl_lock_closure_build);
-
-/**
- * Adds new lock to a closure.
- *
- * Try-locks \a lock and if succeeded, adds it to the closure (never more than
- * once). If try-lock failed, returns CLO_REPEAT, after optionally waiting
- * until next try-lock is likely to succeed.
- */
-int cl_lock_enclosure(const struct lu_env *env, struct cl_lock *lock,
- struct cl_lock_closure *closure)
-{
- int result = 0;
-
- cl_lock_trace(D_DLMTRACE, env, "enclosure lock", lock);
- if (!cl_lock_mutex_try(env, lock)) {
- /*
- * If lock->cll_inclosure is not empty, lock is already in
- * this closure.
- */
- if (list_empty(&lock->cll_inclosure)) {
- cl_lock_get_trust(lock);
- lu_ref_add(&lock->cll_reference, "closure", closure);
- list_add(&lock->cll_inclosure, &closure->clc_list);
- closure->clc_nr++;
- } else
- cl_lock_mutex_put(env, lock);
- result = 0;
- } else {
- cl_lock_disclosure(env, closure);
- if (closure->clc_wait) {
- cl_lock_get_trust(lock);
- lu_ref_add(&lock->cll_reference, "closure-w", closure);
- cl_lock_mutex_put(env, closure->clc_origin);
-
- LASSERT(cl_lock_nr_mutexed(env) == 0);
- cl_lock_mutex_get(env, lock);
- cl_lock_mutex_put(env, lock);
-
- cl_lock_mutex_get(env, closure->clc_origin);
- lu_ref_del(&lock->cll_reference, "closure-w", closure);
- cl_lock_put(env, lock);
- }
- result = CLO_REPEAT;
- }
- return result;
-}
-EXPORT_SYMBOL(cl_lock_enclosure);
-
-/** Releases mutices of enclosed locks. */
-void cl_lock_disclosure(const struct lu_env *env,
- struct cl_lock_closure *closure)
-{
- struct cl_lock *scan;
- struct cl_lock *temp;
-
- cl_lock_trace(D_DLMTRACE, env, "disclosure lock", closure->clc_origin);
- list_for_each_entry_safe(scan, temp, &closure->clc_list,
- cll_inclosure) {
- list_del_init(&scan->cll_inclosure);
- cl_lock_mutex_put(env, scan);
- lu_ref_del(&scan->cll_reference, "closure", closure);
- cl_lock_put(env, scan);
- closure->clc_nr--;
- }
- LASSERT(closure->clc_nr == 0);
-}
-EXPORT_SYMBOL(cl_lock_disclosure);
-
-/** Finalizes a closure. */
-void cl_lock_closure_fini(struct cl_lock_closure *closure)
-{
- LASSERT(closure->clc_nr == 0);
- LASSERT(list_empty(&closure->clc_list));
-}
-EXPORT_SYMBOL(cl_lock_closure_fini);
-
-/**
- * Destroys this lock. Notifies layers (bottom-to-top) that lock is being
- * destroyed, then destroy the lock. If there are holds on the lock, postpone
- * destruction until all holds are released. This is called when a decision is
- * made to destroy the lock in the future. E.g., when a blocking AST is
- * received on it, or fatal communication error happens.
- *
- * Caller must have a reference on this lock to prevent a situation, when
- * deleted lock lingers in memory for indefinite time, because nobody calls
- * cl_lock_put() to finish it.
- *
- * \pre atomic_read(&lock->cll_ref) > 0
- * \pre ergo(cl_lock_nesting(lock) == CNL_TOP,
- * cl_lock_nr_mutexed(env) == 1)
- * [i.e., if a top-lock is deleted, mutices of no other locks can be
- * held, as deletion of sub-locks might require releasing a top-lock
- * mutex]
- *
- * \see cl_lock_operations::clo_delete()
- * \see cl_lock::cll_holds
- */
-void cl_lock_delete(const struct lu_env *env, struct cl_lock *lock)
-{
- LINVRNT(cl_lock_is_mutexed(lock));
- LINVRNT(cl_lock_invariant(env, lock));
- LASSERT(ergo(cl_lock_nesting(lock) == CNL_TOP,
- cl_lock_nr_mutexed(env) == 1));
-
- cl_lock_trace(D_DLMTRACE, env, "delete lock", lock);
- if (lock->cll_holds == 0)
- cl_lock_delete0(env, lock);
- else
- lock->cll_flags |= CLF_DOOMED;
-}
-EXPORT_SYMBOL(cl_lock_delete);
-
-/**
- * Mark lock as irrecoverably failed, and mark it for destruction. This
- * happens when, e.g., server fails to grant a lock to us, or networking
- * time-out happens.
- *
- * \pre atomic_read(&lock->cll_ref) > 0
- *
- * \see clo_lock_delete()
- * \see cl_lock::cll_holds
- */
-void cl_lock_error(const struct lu_env *env, struct cl_lock *lock, int error)
-{
- LINVRNT(cl_lock_is_mutexed(lock));
- LINVRNT(cl_lock_invariant(env, lock));
-
- if (lock->cll_error == 0 && error != 0) {
- cl_lock_trace(D_DLMTRACE, env, "set lock error", lock);
- lock->cll_error = error;
- cl_lock_signal(env, lock);
- cl_lock_cancel(env, lock);
- cl_lock_delete(env, lock);
- }
-}
-EXPORT_SYMBOL(cl_lock_error);
-
-/**
- * Cancels this lock. Notifies layers
- * (bottom-to-top) that lock is being cancelled, then destroy the lock. If
- * there are holds on the lock, postpone cancellation until
- * all holds are released.
- *
- * Cancellation notification is delivered to layers at most once.
- *
- * \see cl_lock_operations::clo_cancel()
- * \see cl_lock::cll_holds
- */
void cl_lock_cancel(const struct lu_env *env, struct cl_lock *lock)
{
- LINVRNT(cl_lock_is_mutexed(lock));
- LINVRNT(cl_lock_invariant(env, lock));
+ const struct cl_lock_slice *slice;
cl_lock_trace(D_DLMTRACE, env, "cancel lock", lock);
- if (lock->cll_holds == 0)
- cl_lock_cancel0(env, lock);
- else
- lock->cll_flags |= CLF_CANCELPEND;
+ list_for_each_entry_reverse(slice, &lock->cll_layers, cls_linkage) {
+ if (slice->cls_ops->clo_cancel)
+ slice->cls_ops->clo_cancel(env, slice);
+ }
}
EXPORT_SYMBOL(cl_lock_cancel);
/**
- * Finds an existing lock covering given index and optionally different from a
- * given \a except lock.
+ * Enqueue a lock.
+ * \param anchor: if we need to wait for resources before getting the lock,
+ * use @anchor for the purpose.
+ * \retval 0 enqueue successfully
+ * \retval <0 error code
*/
-struct cl_lock *cl_lock_at_pgoff(const struct lu_env *env,
- struct cl_object *obj, pgoff_t index,
- struct cl_lock *except,
- int pending, int canceld)
+int cl_lock_enqueue(const struct lu_env *env, struct cl_io *io,
+ struct cl_lock *lock, struct cl_sync_io *anchor)
{
- struct cl_object_header *head;
- struct cl_lock *scan;
- struct cl_lock *lock;
- struct cl_lock_descr *need;
+ const struct cl_lock_slice *slice;
+ int rc = -ENOSYS;
- head = cl_object_header(obj);
- need = &cl_env_info(env)->clt_descr;
- lock = NULL;
+ list_for_each_entry(slice, &lock->cll_layers, cls_linkage) {
+ if (!slice->cls_ops->clo_enqueue)
+ continue;
- need->cld_mode = CLM_READ; /* CLM_READ matches both READ & WRITE, but
- * not PHANTOM
- */
- need->cld_start = need->cld_end = index;
- need->cld_enq_flags = 0;
-
- spin_lock(&head->coh_lock_guard);
- /* It is fine to match any group lock since there could be only one
- * with a uniq gid and it conflicts with all other lock modes too
- */
- list_for_each_entry(scan, &head->coh_locks, cll_linkage) {
- if (scan != except &&
- (scan->cll_descr.cld_mode == CLM_GROUP ||
- cl_lock_ext_match(&scan->cll_descr, need)) &&
- scan->cll_state >= CLS_HELD &&
- scan->cll_state < CLS_FREEING &&
- /*
- * This check is racy as the lock can be canceled right
- * after it is done, but this is fine, because page exists
- * already.
- */
- (canceld || !(scan->cll_flags & CLF_CANCELLED)) &&
- (pending || !(scan->cll_flags & CLF_CANCELPEND))) {
- /* Don't increase cs_hit here since this
- * is just a helper function.
- */
- cl_lock_get_trust(scan);
- lock = scan;
+ rc = slice->cls_ops->clo_enqueue(env, slice, io, anchor);
+ if (rc != 0)
break;
}
- }
- spin_unlock(&head->coh_lock_guard);
- return lock;
+ return rc;
}
-EXPORT_SYMBOL(cl_lock_at_pgoff);
-
-/**
- * Calculate the page offset at the layer of @lock.
- * At the time of this writing, @page is top page and @lock is sub lock.
- */
-static pgoff_t pgoff_at_lock(struct cl_page *page, struct cl_lock *lock)
-{
- struct lu_device_type *dtype;
- const struct cl_page_slice *slice;
-
- dtype = lock->cll_descr.cld_obj->co_lu.lo_dev->ld_type;
- slice = cl_page_at(page, dtype);
- return slice->cpl_page->cp_index;
-}
-
-/**
- * Check if page @page is covered by an extra lock or discard it.
- */
-static int check_and_discard_cb(const struct lu_env *env, struct cl_io *io,
- struct cl_page *page, void *cbdata)
-{
- struct cl_thread_info *info = cl_env_info(env);
- struct cl_lock *lock = cbdata;
- pgoff_t index = pgoff_at_lock(page, lock);
-
- if (index >= info->clt_fn_index) {
- struct cl_lock *tmp;
-
- /* refresh non-overlapped index */
- tmp = cl_lock_at_pgoff(env, lock->cll_descr.cld_obj, index,
- lock, 1, 0);
- if (tmp) {
- /* Cache the first-non-overlapped index so as to skip
- * all pages within [index, clt_fn_index). This
- * is safe because if tmp lock is canceled, it will
- * discard these pages.
- */
- info->clt_fn_index = tmp->cll_descr.cld_end + 1;
- if (tmp->cll_descr.cld_end == CL_PAGE_EOF)
- info->clt_fn_index = CL_PAGE_EOF;
- cl_lock_put(env, tmp);
- } else if (cl_page_own(env, io, page) == 0) {
- /* discard the page */
- cl_page_unmap(env, io, page);
- cl_page_discard(env, io, page);
- cl_page_disown(env, io, page);
- } else {
- LASSERT(page->cp_state == CPS_FREEING);
- }
- }
-
- info->clt_next_index = index + 1;
- return CLP_GANG_OKAY;
-}
-
-static int discard_cb(const struct lu_env *env, struct cl_io *io,
- struct cl_page *page, void *cbdata)
-{
- struct cl_thread_info *info = cl_env_info(env);
- struct cl_lock *lock = cbdata;
-
- LASSERT(lock->cll_descr.cld_mode >= CLM_WRITE);
- KLASSERT(ergo(page->cp_type == CPT_CACHEABLE,
- !PageWriteback(cl_page_vmpage(env, page))));
- KLASSERT(ergo(page->cp_type == CPT_CACHEABLE,
- !PageDirty(cl_page_vmpage(env, page))));
-
- info->clt_next_index = pgoff_at_lock(page, lock) + 1;
- if (cl_page_own(env, io, page) == 0) {
- /* discard the page */
- cl_page_unmap(env, io, page);
- cl_page_discard(env, io, page);
- cl_page_disown(env, io, page);
- } else {
- LASSERT(page->cp_state == CPS_FREEING);
- }
-
- return CLP_GANG_OKAY;
-}
-
-/**
- * Discard pages protected by the given lock. This function traverses radix
- * tree to find all covering pages and discard them. If a page is being covered
- * by other locks, it should remain in cache.
- *
- * If error happens on any step, the process continues anyway (the reasoning
- * behind this being that lock cancellation cannot be delayed indefinitely).
- */
-int cl_lock_discard_pages(const struct lu_env *env, struct cl_lock *lock)
-{
- struct cl_thread_info *info = cl_env_info(env);
- struct cl_io *io = &info->clt_io;
- struct cl_lock_descr *descr = &lock->cll_descr;
- cl_page_gang_cb_t cb;
- int res;
- int result;
-
- LINVRNT(cl_lock_invariant(env, lock));
-
- io->ci_obj = cl_object_top(descr->cld_obj);
- io->ci_ignore_layout = 1;
- result = cl_io_init(env, io, CIT_MISC, io->ci_obj);
- if (result != 0)
- goto out;
-
- cb = descr->cld_mode == CLM_READ ? check_and_discard_cb : discard_cb;
- info->clt_fn_index = info->clt_next_index = descr->cld_start;
- do {
- res = cl_page_gang_lookup(env, descr->cld_obj, io,
- info->clt_next_index, descr->cld_end,
- cb, (void *)lock);
- if (info->clt_next_index > descr->cld_end)
- break;
-
- if (res == CLP_GANG_RESCHED)
- cond_resched();
- } while (res != CLP_GANG_OKAY);
-out:
- cl_io_fini(env, io);
- return result;
-}
-EXPORT_SYMBOL(cl_lock_discard_pages);
-
-/**
- * Eliminate all locks for a given object.
- *
- * Caller has to guarantee that no lock is in active use.
- *
- * \param cancel when this is set, cl_locks_prune() cancels locks before
- * destroying.
- */
-void cl_locks_prune(const struct lu_env *env, struct cl_object *obj, int cancel)
-{
- struct cl_object_header *head;
- struct cl_lock *lock;
-
- head = cl_object_header(obj);
- /*
- * If locks are destroyed without cancellation, all pages must be
- * already destroyed (as otherwise they will be left unprotected).
- */
- LASSERT(ergo(!cancel,
- !head->coh_tree.rnode && head->coh_pages == 0));
-
- spin_lock(&head->coh_lock_guard);
- while (!list_empty(&head->coh_locks)) {
- lock = container_of(head->coh_locks.next,
- struct cl_lock, cll_linkage);
- cl_lock_get_trust(lock);
- spin_unlock(&head->coh_lock_guard);
- lu_ref_add(&lock->cll_reference, "prune", current);
-
-again:
- cl_lock_mutex_get(env, lock);
- if (lock->cll_state < CLS_FREEING) {
- LASSERT(lock->cll_users <= 1);
- if (unlikely(lock->cll_users == 1)) {
- struct l_wait_info lwi = { 0 };
-
- cl_lock_mutex_put(env, lock);
- l_wait_event(lock->cll_wq,
- lock->cll_users == 0,
- &lwi);
- goto again;
- }
-
- if (cancel)
- cl_lock_cancel(env, lock);
- cl_lock_delete(env, lock);
- }
- cl_lock_mutex_put(env, lock);
- lu_ref_del(&lock->cll_reference, "prune", current);
- cl_lock_put(env, lock);
- spin_lock(&head->coh_lock_guard);
- }
- spin_unlock(&head->coh_lock_guard);
-}
-EXPORT_SYMBOL(cl_locks_prune);
-
-static struct cl_lock *cl_lock_hold_mutex(const struct lu_env *env,
- const struct cl_io *io,
- const struct cl_lock_descr *need,
- const char *scope, const void *source)
-{
- struct cl_lock *lock;
-
- while (1) {
- lock = cl_lock_find(env, io, need);
- if (IS_ERR(lock))
- break;
- cl_lock_mutex_get(env, lock);
- if (lock->cll_state < CLS_FREEING &&
- !(lock->cll_flags & CLF_CANCELLED)) {
- cl_lock_hold_mod(env, lock, 1);
- lu_ref_add(&lock->cll_holders, scope, source);
- lu_ref_add(&lock->cll_reference, scope, source);
- break;
- }
- cl_lock_mutex_put(env, lock);
- cl_lock_put(env, lock);
- }
- return lock;
-}
-
-/**
- * Returns a lock matching \a need description with a reference and a hold on
- * it.
- *
- * This is much like cl_lock_find(), except that cl_lock_hold() additionally
- * guarantees that lock is not in the CLS_FREEING state on return.
- */
-struct cl_lock *cl_lock_hold(const struct lu_env *env, const struct cl_io *io,
- const struct cl_lock_descr *need,
- const char *scope, const void *source)
-{
- struct cl_lock *lock;
-
- lock = cl_lock_hold_mutex(env, io, need, scope, source);
- if (!IS_ERR(lock))
- cl_lock_mutex_put(env, lock);
- return lock;
-}
-EXPORT_SYMBOL(cl_lock_hold);
+EXPORT_SYMBOL(cl_lock_enqueue);
/**
* Main high-level entry point of cl_lock interface that finds existing or
* enqueues new lock matching given description.
*/
-struct cl_lock *cl_lock_request(const struct lu_env *env, struct cl_io *io,
- const struct cl_lock_descr *need,
- const char *scope, const void *source)
+int cl_lock_request(const struct lu_env *env, struct cl_io *io,
+ struct cl_lock *lock)
{
- struct cl_lock *lock;
- int rc;
- __u32 enqflags = need->cld_enq_flags;
+ struct cl_sync_io *anchor = NULL;
+ __u32 enq_flags = lock->cll_descr.cld_enq_flags;
+ int rc;
- do {
- lock = cl_lock_hold_mutex(env, io, need, scope, source);
- if (IS_ERR(lock))
- break;
+ rc = cl_lock_init(env, lock, io);
+ if (rc < 0)
+ return rc;
- rc = cl_enqueue_locked(env, lock, io, enqflags);
- if (rc == 0) {
- if (cl_lock_fits_into(env, lock, need, io)) {
- if (!(enqflags & CEF_AGL)) {
- cl_lock_mutex_put(env, lock);
- cl_lock_lockdep_acquire(env, lock,
- enqflags);
- break;
- }
- rc = 1;
- }
- cl_unuse_locked(env, lock);
- }
- cl_lock_trace(D_DLMTRACE, env,
- rc <= 0 ? "enqueue failed" : "agl succeed", lock);
- cl_lock_hold_release(env, lock, scope, source);
- cl_lock_mutex_put(env, lock);
- lu_ref_del(&lock->cll_reference, scope, source);
- cl_lock_put(env, lock);
- if (rc > 0) {
- LASSERT(enqflags & CEF_AGL);
- lock = NULL;
- } else if (rc != 0) {
- lock = ERR_PTR(rc);
- }
- } while (rc == 0);
- return lock;
+ if ((enq_flags & CEF_ASYNC) && !(enq_flags & CEF_AGL)) {
+ anchor = &cl_env_info(env)->clt_anchor;
+ cl_sync_io_init(anchor, 1, cl_sync_io_end);
+ }
+
+ rc = cl_lock_enqueue(env, io, lock, anchor);
+
+ if (anchor) {
+ int rc2;
+
+ /* drop the reference count held at initialization time */
+ cl_sync_io_note(env, anchor, 0);
+ rc2 = cl_sync_io_wait(env, anchor, 0);
+ if (rc2 < 0 && rc == 0)
+ rc = rc2;
+ }
+
+ if (rc < 0)
+ cl_lock_release(env, lock);
+
+ return rc;
}
EXPORT_SYMBOL(cl_lock_request);
/**
- * Adds a hold to a known lock.
- */
-void cl_lock_hold_add(const struct lu_env *env, struct cl_lock *lock,
- const char *scope, const void *source)
-{
- LINVRNT(cl_lock_is_mutexed(lock));
- LINVRNT(cl_lock_invariant(env, lock));
- LASSERT(lock->cll_state != CLS_FREEING);
-
- cl_lock_hold_mod(env, lock, 1);
- cl_lock_get(lock);
- lu_ref_add(&lock->cll_holders, scope, source);
- lu_ref_add(&lock->cll_reference, scope, source);
-}
-EXPORT_SYMBOL(cl_lock_hold_add);
-
-/**
- * Releases a hold and a reference on a lock, on which caller acquired a
- * mutex.
- */
-void cl_lock_unhold(const struct lu_env *env, struct cl_lock *lock,
- const char *scope, const void *source)
-{
- LINVRNT(cl_lock_invariant(env, lock));
- cl_lock_hold_release(env, lock, scope, source);
- lu_ref_del(&lock->cll_reference, scope, source);
- cl_lock_put(env, lock);
-}
-EXPORT_SYMBOL(cl_lock_unhold);
-
-/**
* Releases a hold and a reference on a lock, obtained by cl_lock_hold().
*/
-void cl_lock_release(const struct lu_env *env, struct cl_lock *lock,
- const char *scope, const void *source)
+void cl_lock_release(const struct lu_env *env, struct cl_lock *lock)
{
- LINVRNT(cl_lock_invariant(env, lock));
cl_lock_trace(D_DLMTRACE, env, "release lock", lock);
- cl_lock_mutex_get(env, lock);
- cl_lock_hold_release(env, lock, scope, source);
- cl_lock_mutex_put(env, lock);
- lu_ref_del(&lock->cll_reference, scope, source);
- cl_lock_put(env, lock);
+ cl_lock_cancel(env, lock);
+ cl_lock_fini(env, lock);
}
EXPORT_SYMBOL(cl_lock_release);
-void cl_lock_user_add(const struct lu_env *env, struct cl_lock *lock)
-{
- LINVRNT(cl_lock_is_mutexed(lock));
- LINVRNT(cl_lock_invariant(env, lock));
-
- cl_lock_used_mod(env, lock, 1);
-}
-EXPORT_SYMBOL(cl_lock_user_add);
-
-void cl_lock_user_del(const struct lu_env *env, struct cl_lock *lock)
-{
- LINVRNT(cl_lock_is_mutexed(lock));
- LINVRNT(cl_lock_invariant(env, lock));
- LASSERT(lock->cll_users > 0);
-
- cl_lock_used_mod(env, lock, -1);
- if (lock->cll_users == 0)
- wake_up_all(&lock->cll_wq);
-}
-EXPORT_SYMBOL(cl_lock_user_del);
-
const char *cl_lock_mode_name(const enum cl_lock_mode mode)
{
static const char *names[] = {
- [CLM_PHANTOM] = "P",
[CLM_READ] = "R",
[CLM_WRITE] = "W",
[CLM_GROUP] = "G"
@@ -2189,10 +261,8 @@
lu_printer_t printer, const struct cl_lock *lock)
{
const struct cl_lock_slice *slice;
- (*printer)(env, cookie, "lock@%p[%d %d %d %d %d %08lx] ",
- lock, atomic_read(&lock->cll_ref),
- lock->cll_state, lock->cll_error, lock->cll_holds,
- lock->cll_users, lock->cll_flags);
+
+ (*printer)(env, cookie, "lock@%p", lock);
cl_lock_descr_print(env, cookie, printer, &lock->cll_descr);
(*printer)(env, cookie, " {\n");
@@ -2207,13 +277,3 @@
(*printer)(env, cookie, "} lock@%p\n", lock);
}
EXPORT_SYMBOL(cl_lock_print);
-
-int cl_lock_init(void)
-{
- return lu_kmem_init(cl_lock_caches);
-}
-
-void cl_lock_fini(void)
-{
- lu_kmem_fini(cl_lock_caches);
-}
diff --git a/drivers/staging/lustre/lustre/obdclass/cl_object.c b/drivers/staging/lustre/lustre/obdclass/cl_object.c
index 43e299d..5940f30 100644
--- a/drivers/staging/lustre/lustre/obdclass/cl_object.c
+++ b/drivers/staging/lustre/lustre/obdclass/cl_object.c
@@ -36,6 +36,7 @@
* Client Lustre Object.
*
* Author: Nikita Danilov <nikita.danilov@sun.com>
+ * Author: Jinshan Xiong <jinshan.xiong@intel.com>
*/
/*
@@ -43,8 +44,6 @@
*
* i_mutex
* PG_locked
- * ->coh_page_guard
- * ->coh_lock_guard
* ->coh_attr_guard
* ->ls_guard
*/
@@ -63,10 +62,6 @@
static struct kmem_cache *cl_env_kmem;
-/** Lock class of cl_object_header::coh_page_guard */
-static struct lock_class_key cl_page_guard_class;
-/** Lock class of cl_object_header::coh_lock_guard */
-static struct lock_class_key cl_lock_guard_class;
/** Lock class of cl_object_header::coh_attr_guard */
static struct lock_class_key cl_attr_guard_class;
@@ -81,17 +76,9 @@
result = lu_object_header_init(&h->coh_lu);
if (result == 0) {
- spin_lock_init(&h->coh_page_guard);
- spin_lock_init(&h->coh_lock_guard);
spin_lock_init(&h->coh_attr_guard);
- lockdep_set_class(&h->coh_page_guard, &cl_page_guard_class);
- lockdep_set_class(&h->coh_lock_guard, &cl_lock_guard_class);
lockdep_set_class(&h->coh_attr_guard, &cl_attr_guard_class);
- h->coh_pages = 0;
- /* XXX hard coded GFP_* mask. */
- INIT_RADIX_TREE(&h->coh_tree, GFP_ATOMIC);
- INIT_LIST_HEAD(&h->coh_locks);
- h->coh_page_bufsize = ALIGN(sizeof(struct cl_page), 8);
+ h->coh_page_bufsize = 0;
}
return result;
}
@@ -145,7 +132,7 @@
/**
* Returns the top-object for a given \a o.
*
- * \see cl_page_top(), cl_io_top()
+ * \see cl_io_top()
*/
struct cl_object *cl_object_top(struct cl_object *o)
{
@@ -315,6 +302,29 @@
EXPORT_SYMBOL(cl_conf_set);
/**
+ * Prunes caches of pages and locks for this object.
+ */
+int cl_object_prune(const struct lu_env *env, struct cl_object *obj)
+{
+ struct lu_object_header *top;
+ struct cl_object *o;
+ int result;
+
+ top = obj->co_lu.lo_header;
+ result = 0;
+ list_for_each_entry(o, &top->loh_layers, co_lu.lo_linkage) {
+ if (o->co_ops->coo_prune) {
+ result = o->co_ops->coo_prune(env, o);
+ if (result != 0)
+ break;
+ }
+ }
+
+ return result;
+}
+EXPORT_SYMBOL(cl_object_prune);
+
+/**
* Helper function removing all object locks, and marking object for
* deletion. All object pages must have been deleted at this point.
*
@@ -323,34 +333,12 @@
*/
void cl_object_kill(const struct lu_env *env, struct cl_object *obj)
{
- struct cl_object_header *hdr;
-
- hdr = cl_object_header(obj);
- LASSERT(!hdr->coh_tree.rnode);
- LASSERT(hdr->coh_pages == 0);
+ struct cl_object_header *hdr = cl_object_header(obj);
set_bit(LU_OBJECT_HEARD_BANSHEE, &hdr->coh_lu.loh_flags);
- /*
- * Destroy all locks. Object destruction (including cl_inode_fini())
- * cannot cancel the locks, because in the case of a local client,
- * where client and server share the same thread running
- * prune_icache(), this can dead-lock with ldlm_cancel_handler()
- * waiting on __wait_on_freeing_inode().
- */
- cl_locks_prune(env, obj, 0);
}
EXPORT_SYMBOL(cl_object_kill);
-/**
- * Prunes caches of pages and locks for this object.
- */
-void cl_object_prune(const struct lu_env *env, struct cl_object *obj)
-{
- cl_pages_prune(env, obj);
- cl_locks_prune(env, obj, 1);
-}
-EXPORT_SYMBOL(cl_object_prune);
-
void cache_stats_init(struct cache_stats *cs, const char *name)
{
int i;
@@ -383,6 +371,8 @@
return 0;
}
+static void cl_env_percpu_refill(void);
+
/**
* Initialize client site.
*
@@ -397,11 +387,9 @@
result = lu_site_init(&s->cs_lu, &d->cd_lu_dev);
if (result == 0) {
cache_stats_init(&s->cs_pages, "pages");
- cache_stats_init(&s->cs_locks, "locks");
for (i = 0; i < ARRAY_SIZE(s->cs_pages_state); ++i)
atomic_set(&s->cs_pages_state[0], 0);
- for (i = 0; i < ARRAY_SIZE(s->cs_locks_state); ++i)
- atomic_set(&s->cs_locks_state[i], 0);
+ cl_env_percpu_refill();
}
return result;
}
@@ -435,15 +423,6 @@
[CPS_PAGEIN] = "r",
[CPS_FREEING] = "f"
};
- static const char *lstate[] = {
- [CLS_NEW] = "n",
- [CLS_QUEUING] = "q",
- [CLS_ENQUEUED] = "e",
- [CLS_HELD] = "h",
- [CLS_INTRANSIT] = "t",
- [CLS_CACHED] = "c",
- [CLS_FREEING] = "f"
- };
/*
lookup hit total busy create
pages: ...... ...... ...... ...... ...... [...... ...... ...... ......]
@@ -457,12 +436,6 @@
seq_printf(m, "%s: %u ", pstate[i],
atomic_read(&site->cs_pages_state[i]));
seq_printf(m, "]\n");
- cache_stats_print(&site->cs_locks, m, 0);
- seq_printf(m, " [");
- for (i = 0; i < ARRAY_SIZE(site->cs_locks_state); ++i)
- seq_printf(m, "%s: %u ", lstate[i],
- atomic_read(&site->cs_locks_state[i]));
- seq_printf(m, "]\n");
cache_stats_print(&cl_env_stats, m, 0);
seq_printf(m, "\n");
return 0;
@@ -492,6 +465,13 @@
* bz20044, bz22683.
*/
+static LIST_HEAD(cl_envs);
+static unsigned int cl_envs_cached_nr;
+static unsigned int cl_envs_cached_max = 128; /* XXX: prototype: arbitrary limit
+ * for now.
+ */
+static DEFINE_SPINLOCK(cl_envs_guard);
+
struct cl_env {
void *ce_magic;
struct lu_env ce_lu;
@@ -674,8 +654,9 @@
lu_context_enter(&cle->ce_ses);
env->le_ses = &cle->ce_ses;
cl_env_init0(cle, debug);
- } else
+ } else {
lu_env_fini(env);
+ }
}
if (rc != 0) {
kmem_cache_free(cl_env_kmem, cle);
@@ -684,8 +665,9 @@
CL_ENV_INC(create);
CL_ENV_INC(total);
}
- } else
+ } else {
env = ERR_PTR(-ENOMEM);
+ }
return env;
}
@@ -697,6 +679,39 @@
kmem_cache_free(cl_env_kmem, cle);
}
+static struct lu_env *cl_env_obtain(void *debug)
+{
+ struct cl_env *cle;
+ struct lu_env *env;
+
+ spin_lock(&cl_envs_guard);
+ LASSERT(equi(cl_envs_cached_nr == 0, list_empty(&cl_envs)));
+ if (cl_envs_cached_nr > 0) {
+ int rc;
+
+ cle = container_of(cl_envs.next, struct cl_env, ce_linkage);
+ list_del_init(&cle->ce_linkage);
+ cl_envs_cached_nr--;
+ spin_unlock(&cl_envs_guard);
+
+ env = &cle->ce_lu;
+ rc = lu_env_refill(env);
+ if (rc == 0) {
+ cl_env_init0(cle, debug);
+ lu_context_enter(&env->le_ctx);
+ lu_context_enter(&cle->ce_ses);
+ } else {
+ cl_env_fini(cle);
+ env = ERR_PTR(rc);
+ }
+ } else {
+ spin_unlock(&cl_envs_guard);
+ env = cl_env_new(lu_context_tags_default,
+ lu_session_tags_default, debug);
+ }
+ return env;
+}
+
static inline struct cl_env *cl_env_container(struct lu_env *env)
{
return container_of(env, struct cl_env, ce_lu);
@@ -727,6 +742,8 @@
* Returns lu_env: if there already is an environment associated with the
* current thread, it is returned, otherwise, new environment is allocated.
*
+ * Allocations are amortized through the global cache of environments.
+ *
* \param refcheck pointer to a counter used to detect environment leaks. In
* the usual case cl_env_get() and cl_env_put() are called in the same lexical
* scope and pointer to the same integer is passed as \a refcheck. This is
@@ -740,10 +757,7 @@
env = cl_env_peek(refcheck);
if (!env) {
- env = cl_env_new(lu_context_tags_default,
- lu_session_tags_default,
- __builtin_return_address(0));
-
+ env = cl_env_obtain(__builtin_return_address(0));
if (!IS_ERR(env)) {
struct cl_env *cle;
@@ -787,6 +801,32 @@
}
/**
+ * Finalizes and frees a given number of cached environments. This is done to
+ * (1) free some memory (not currently hooked into VM), or (2) release
+ * references to modules.
+ */
+unsigned int cl_env_cache_purge(unsigned int nr)
+{
+ struct cl_env *cle;
+
+ spin_lock(&cl_envs_guard);
+ for (; !list_empty(&cl_envs) && nr > 0; --nr) {
+ cle = container_of(cl_envs.next, struct cl_env, ce_linkage);
+ list_del_init(&cle->ce_linkage);
+ LASSERT(cl_envs_cached_nr > 0);
+ cl_envs_cached_nr--;
+ spin_unlock(&cl_envs_guard);
+
+ cl_env_fini(cle);
+ spin_lock(&cl_envs_guard);
+ }
+ LASSERT(equi(cl_envs_cached_nr == 0, list_empty(&cl_envs)));
+ spin_unlock(&cl_envs_guard);
+ return nr;
+}
+EXPORT_SYMBOL(cl_env_cache_purge);
+
+/**
* Release an environment.
*
* Decrement \a env reference counter. When counter drops to 0, nothing in
@@ -808,7 +848,22 @@
cl_env_detach(cle);
cle->ce_debug = NULL;
cl_env_exit(cle);
- cl_env_fini(cle);
+ /*
+ * Don't bother to take a lock here.
+ *
+ * Return environment to the cache only when it was allocated
+ * with the standard tags.
+ */
+ if (cl_envs_cached_nr < cl_envs_cached_max &&
+ (env->le_ctx.lc_tags & ~LCT_HAS_EXIT) == LCT_CL_THREAD &&
+ (env->le_ses->lc_tags & ~LCT_HAS_EXIT) == LCT_SESSION) {
+ spin_lock(&cl_envs_guard);
+ list_add(&cle->ce_linkage, &cl_envs);
+ cl_envs_cached_nr++;
+ spin_unlock(&cl_envs_guard);
+ } else {
+ cl_env_fini(cle);
+ }
}
}
EXPORT_SYMBOL(cl_env_put);
@@ -914,6 +969,104 @@
}
EXPORT_SYMBOL(cl_lvb2attr);
+static struct cl_env cl_env_percpu[NR_CPUS];
+
+static int cl_env_percpu_init(void)
+{
+ struct cl_env *cle;
+ int tags = LCT_REMEMBER | LCT_NOREF;
+ int i, j;
+ int rc = 0;
+
+ for_each_possible_cpu(i) {
+ struct lu_env *env;
+
+ cle = &cl_env_percpu[i];
+ env = &cle->ce_lu;
+
+ INIT_LIST_HEAD(&cle->ce_linkage);
+ cle->ce_magic = &cl_env_init0;
+ rc = lu_env_init(env, LCT_CL_THREAD | tags);
+ if (rc == 0) {
+ rc = lu_context_init(&cle->ce_ses, LCT_SESSION | tags);
+ if (rc == 0) {
+ lu_context_enter(&cle->ce_ses);
+ env->le_ses = &cle->ce_ses;
+ } else {
+ lu_env_fini(env);
+ }
+ }
+ if (rc != 0)
+ break;
+ }
+ if (rc != 0) {
+ /* Indices 0 to i (excluding i) were correctly initialized,
+ * thus we must uninitialize up to i, the rest are undefined.
+ */
+ for (j = 0; j < i; j++) {
+ cle = &cl_env_percpu[i];
+ lu_context_exit(&cle->ce_ses);
+ lu_context_fini(&cle->ce_ses);
+ lu_env_fini(&cle->ce_lu);
+ }
+ }
+
+ return rc;
+}
+
+static void cl_env_percpu_fini(void)
+{
+ int i;
+
+ for_each_possible_cpu(i) {
+ struct cl_env *cle = &cl_env_percpu[i];
+
+ lu_context_exit(&cle->ce_ses);
+ lu_context_fini(&cle->ce_ses);
+ lu_env_fini(&cle->ce_lu);
+ }
+}
+
+static void cl_env_percpu_refill(void)
+{
+ int i;
+
+ for_each_possible_cpu(i)
+ lu_env_refill(&cl_env_percpu[i].ce_lu);
+}
+
+void cl_env_percpu_put(struct lu_env *env)
+{
+ struct cl_env *cle;
+ int cpu;
+
+ cpu = smp_processor_id();
+ cle = cl_env_container(env);
+ LASSERT(cle == &cl_env_percpu[cpu]);
+
+ cle->ce_ref--;
+ LASSERT(cle->ce_ref == 0);
+
+ CL_ENV_DEC(busy);
+ cl_env_detach(cle);
+ cle->ce_debug = NULL;
+
+ put_cpu();
+}
+EXPORT_SYMBOL(cl_env_percpu_put);
+
+struct lu_env *cl_env_percpu_get(void)
+{
+ struct cl_env *cle;
+
+ cle = &cl_env_percpu[get_cpu()];
+ cl_env_init0(cle, __builtin_return_address(0));
+
+ cl_env_attach(cle);
+ return &cle->ce_lu;
+}
+EXPORT_SYMBOL(cl_env_percpu_get);
+
/*****************************************************************************
*
* Temporary prototype thing: mirror obd-devices into cl devices.
@@ -944,8 +1097,9 @@
CERROR("can't init device '%s', %d\n", typename, rc);
d = ERR_PTR(rc);
}
- } else
+ } else {
CERROR("Cannot allocate device: '%s'\n", typename);
+ }
return lu2cl_dev(d);
}
EXPORT_SYMBOL(cl_type_setup);
@@ -959,12 +1113,6 @@
}
EXPORT_SYMBOL(cl_stack_fini);
-int cl_lock_init(void);
-void cl_lock_fini(void);
-
-int cl_page_init(void);
-void cl_page_fini(void);
-
static struct lu_context_key cl_key;
struct cl_thread_info *cl_env_info(const struct lu_env *env)
@@ -1059,17 +1207,13 @@
if (result)
goto out_kmem;
- result = cl_lock_init();
+ result = cl_env_percpu_init();
if (result)
+ /* no cl_env_percpu_fini on error */
goto out_context;
- result = cl_page_init();
- if (result)
- goto out_lock;
-
return 0;
-out_lock:
- cl_lock_fini();
+
out_context:
lu_context_key_degister(&cl_key);
out_kmem:
@@ -1084,8 +1228,7 @@
*/
void cl_global_fini(void)
{
- cl_lock_fini();
- cl_page_fini();
+ cl_env_percpu_fini();
lu_context_key_degister(&cl_key);
lu_kmem_fini(cl_object_caches);
cl_env_store_fini();
diff --git a/drivers/staging/lustre/lustre/obdclass/cl_page.c b/drivers/staging/lustre/lustre/obdclass/cl_page.c
index 3945800..b754f51 100644
--- a/drivers/staging/lustre/lustre/obdclass/cl_page.c
+++ b/drivers/staging/lustre/lustre/obdclass/cl_page.c
@@ -36,6 +36,7 @@
* Client Lustre Page.
*
* Author: Nikita Danilov <nikita.danilov@sun.com>
+ * Author: Jinshan Xiong <jinshan.xiong@intel.com>
*/
#define DEBUG_SUBSYSTEM S_CLASS
@@ -48,8 +49,7 @@
#include "../include/cl_object.h"
#include "cl_internal.h"
-static void cl_page_delete0(const struct lu_env *env, struct cl_page *pg,
- int radix);
+static void cl_page_delete0(const struct lu_env *env, struct cl_page *pg);
# define PASSERT(env, page, expr) \
do { \
@@ -63,24 +63,11 @@
((void)sizeof(env), (void)sizeof(page), (void)sizeof !!(exp))
/**
- * Internal version of cl_page_top, it should be called if the page is
- * known to be not freed, says with page referenced, or radix tree lock held,
- * or page owned.
- */
-static struct cl_page *cl_page_top_trusted(struct cl_page *page)
-{
- while (page->cp_parent)
- page = page->cp_parent;
- return page;
-}
-
-/**
* Internal version of cl_page_get().
*
* This function can be used to obtain initial reference to previously
* unreferenced cached object. It can be called only if concurrent page
- * reclamation is somehow prevented, e.g., by locking page radix-tree
- * (cl_object_header::hdr->coh_page_guard), or by keeping a lock on a VM page,
+ * reclamation is somehow prevented, e.g., by keeping a lock on a VM page,
* associated with \a page.
*
* Use with care! Not exported.
@@ -103,143 +90,13 @@
{
const struct cl_page_slice *slice;
- page = cl_page_top_trusted((struct cl_page *)page);
- do {
- list_for_each_entry(slice, &page->cp_layers, cpl_linkage) {
- if (slice->cpl_obj->co_lu.lo_dev->ld_type == dtype)
- return slice;
- }
- page = page->cp_child;
- } while (page);
+ list_for_each_entry(slice, &page->cp_layers, cpl_linkage) {
+ if (slice->cpl_obj->co_lu.lo_dev->ld_type == dtype)
+ return slice;
+ }
return NULL;
}
-/**
- * Returns a page with given index in the given object, or NULL if no page is
- * found. Acquires a reference on \a page.
- *
- * Locking: called under cl_object_header::coh_page_guard spin-lock.
- */
-struct cl_page *cl_page_lookup(struct cl_object_header *hdr, pgoff_t index)
-{
- struct cl_page *page;
-
- assert_spin_locked(&hdr->coh_page_guard);
-
- page = radix_tree_lookup(&hdr->coh_tree, index);
- if (page)
- cl_page_get_trust(page);
- return page;
-}
-EXPORT_SYMBOL(cl_page_lookup);
-
-/**
- * Returns a list of pages by a given [start, end] of \a obj.
- *
- * \param resched If not NULL, then we give up before hogging CPU for too
- * long and set *resched = 1, in that case caller should implement a retry
- * logic.
- *
- * Gang tree lookup (radix_tree_gang_lookup()) optimization is absolutely
- * crucial in the face of [offset, EOF] locks.
- *
- * Return at least one page in @queue unless there is no covered page.
- */
-int cl_page_gang_lookup(const struct lu_env *env, struct cl_object *obj,
- struct cl_io *io, pgoff_t start, pgoff_t end,
- cl_page_gang_cb_t cb, void *cbdata)
-{
- struct cl_object_header *hdr;
- struct cl_page *page;
- struct cl_page **pvec;
- const struct cl_page_slice *slice;
- const struct lu_device_type *dtype;
- pgoff_t idx;
- unsigned int nr;
- unsigned int i;
- unsigned int j;
- int res = CLP_GANG_OKAY;
- int tree_lock = 1;
-
- idx = start;
- hdr = cl_object_header(obj);
- pvec = cl_env_info(env)->clt_pvec;
- dtype = cl_object_top(obj)->co_lu.lo_dev->ld_type;
- spin_lock(&hdr->coh_page_guard);
- while ((nr = radix_tree_gang_lookup(&hdr->coh_tree, (void **)pvec,
- idx, CLT_PVEC_SIZE)) > 0) {
- int end_of_region = 0;
-
- idx = pvec[nr - 1]->cp_index + 1;
- for (i = 0, j = 0; i < nr; ++i) {
- page = pvec[i];
- pvec[i] = NULL;
-
- LASSERT(page->cp_type == CPT_CACHEABLE);
- if (page->cp_index > end) {
- end_of_region = 1;
- break;
- }
- if (page->cp_state == CPS_FREEING)
- continue;
-
- slice = cl_page_at_trusted(page, dtype);
- /*
- * Pages for lsm-less file has no underneath sub-page
- * for osc, in case of ...
- */
- PASSERT(env, page, slice);
-
- page = slice->cpl_page;
- /*
- * Can safely call cl_page_get_trust() under
- * radix-tree spin-lock.
- *
- * XXX not true, because @page is from object another
- * than @hdr and protected by different tree lock.
- */
- cl_page_get_trust(page);
- lu_ref_add_atomic(&page->cp_reference,
- "gang_lookup", current);
- pvec[j++] = page;
- }
-
- /*
- * Here a delicate locking dance is performed. Current thread
- * holds a reference to a page, but has to own it before it
- * can be placed into queue. Owning implies waiting, so
- * radix-tree lock is to be released. After a wait one has to
- * check that pages weren't truncated (cl_page_own() returns
- * error in the latter case).
- */
- spin_unlock(&hdr->coh_page_guard);
- tree_lock = 0;
-
- for (i = 0; i < j; ++i) {
- page = pvec[i];
- if (res == CLP_GANG_OKAY)
- res = (*cb)(env, io, page, cbdata);
- lu_ref_del(&page->cp_reference,
- "gang_lookup", current);
- cl_page_put(env, page);
- }
- if (nr < CLT_PVEC_SIZE || end_of_region)
- break;
-
- if (res == CLP_GANG_OKAY && need_resched())
- res = CLP_GANG_RESCHED;
- if (res != CLP_GANG_OKAY)
- break;
-
- spin_lock(&hdr->coh_page_guard);
- tree_lock = 1;
- }
- if (tree_lock)
- spin_unlock(&hdr->coh_page_guard);
- return res;
-}
-EXPORT_SYMBOL(cl_page_gang_lookup);
-
static void cl_page_free(const struct lu_env *env, struct cl_page *page)
{
struct cl_object *obj = page->cp_obj;
@@ -247,17 +104,16 @@
PASSERT(env, page, list_empty(&page->cp_batch));
PASSERT(env, page, !page->cp_owner);
PASSERT(env, page, !page->cp_req);
- PASSERT(env, page, !page->cp_parent);
PASSERT(env, page, page->cp_state == CPS_FREEING);
- might_sleep();
while (!list_empty(&page->cp_layers)) {
struct cl_page_slice *slice;
slice = list_entry(page->cp_layers.next,
struct cl_page_slice, cpl_linkage);
list_del_init(page->cp_layers.next);
- slice->cpl_ops->cpo_fini(env, slice);
+ if (unlikely(slice->cpl_ops->cpo_fini))
+ slice->cpl_ops->cpo_fini(env, slice);
}
lu_object_ref_del_at(&obj->co_lu, &page->cp_obj_ref, "cl_page", page);
cl_object_put(env, obj);
@@ -276,10 +132,10 @@
*(enum cl_page_state *)&page->cp_state = state;
}
-static struct cl_page *cl_page_alloc(const struct lu_env *env,
- struct cl_object *o, pgoff_t ind,
- struct page *vmpage,
- enum cl_page_type type)
+struct cl_page *cl_page_alloc(const struct lu_env *env,
+ struct cl_object *o, pgoff_t ind,
+ struct page *vmpage,
+ enum cl_page_type type)
{
struct cl_page *page;
struct lu_object_header *head;
@@ -289,13 +145,11 @@
int result = 0;
atomic_set(&page->cp_ref, 1);
- if (type == CPT_CACHEABLE) /* for radix tree */
- atomic_inc(&page->cp_ref);
page->cp_obj = o;
cl_object_get(o);
lu_object_ref_add_at(&o->co_lu, &page->cp_obj_ref, "cl_page",
page);
- page->cp_index = ind;
+ page->cp_vmpage = vmpage;
cl_page_state_set_trust(page, CPS_CACHED);
page->cp_type = type;
INIT_LIST_HEAD(&page->cp_layers);
@@ -306,10 +160,10 @@
head = o->co_lu.lo_header;
list_for_each_entry(o, &head->loh_layers, co_lu.lo_linkage) {
if (o->co_ops->coo_page_init) {
- result = o->co_ops->coo_page_init(env, o,
- page, vmpage);
+ result = o->co_ops->coo_page_init(env, o, page,
+ ind);
if (result != 0) {
- cl_page_delete0(env, page, 0);
+ cl_page_delete0(env, page);
cl_page_free(env, page);
page = ERR_PTR(result);
break;
@@ -321,6 +175,7 @@
}
return page;
}
+EXPORT_SYMBOL(cl_page_alloc);
/**
* Returns a cl_page with index \a idx at the object \a o, and associated with
@@ -333,16 +188,13 @@
*
* \see cl_object_find(), cl_lock_find()
*/
-static struct cl_page *cl_page_find0(const struct lu_env *env,
- struct cl_object *o,
- pgoff_t idx, struct page *vmpage,
- enum cl_page_type type,
- struct cl_page *parent)
+struct cl_page *cl_page_find(const struct lu_env *env,
+ struct cl_object *o,
+ pgoff_t idx, struct page *vmpage,
+ enum cl_page_type type)
{
struct cl_page *page = NULL;
- struct cl_page *ghost = NULL;
struct cl_object_header *hdr;
- int err;
LASSERT(type == CPT_CACHEABLE || type == CPT_TRANSIENT);
might_sleep();
@@ -368,120 +220,25 @@
* reference on it.
*/
page = cl_vmpage_page(vmpage, o);
- PINVRNT(env, page,
- ergo(page,
- cl_page_vmpage(env, page) == vmpage &&
- (void *)radix_tree_lookup(&hdr->coh_tree,
- idx) == page));
- }
- if (page)
- return page;
+ if (page)
+ return page;
+ }
/* allocate and initialize cl_page */
page = cl_page_alloc(env, o, idx, vmpage, type);
- if (IS_ERR(page))
- return page;
-
- if (type == CPT_TRANSIENT) {
- if (parent) {
- LASSERT(!page->cp_parent);
- page->cp_parent = parent;
- parent->cp_child = page;
- }
- return page;
- }
-
- /*
- * XXX optimization: use radix_tree_preload() here, and change tree
- * gfp mask to GFP_KERNEL in cl_object_header_init().
- */
- spin_lock(&hdr->coh_page_guard);
- err = radix_tree_insert(&hdr->coh_tree, idx, page);
- if (err != 0) {
- ghost = page;
- /*
- * Noted by Jay: a lock on \a vmpage protects cl_page_find()
- * from this race, but
- *
- * 0. it's better to have cl_page interface "locally
- * consistent" so that its correctness can be reasoned
- * about without appealing to the (obscure world of) VM
- * locking.
- *
- * 1. handling this race allows ->coh_tree to remain
- * consistent even when VM locking is somehow busted,
- * which is very useful during diagnosing and debugging.
- */
- page = ERR_PTR(err);
- CL_PAGE_DEBUG(D_ERROR, env, ghost,
- "fail to insert into radix tree: %d\n", err);
- } else {
- if (parent) {
- LASSERT(!page->cp_parent);
- page->cp_parent = parent;
- parent->cp_child = page;
- }
- hdr->coh_pages++;
- }
- spin_unlock(&hdr->coh_page_guard);
-
- if (unlikely(ghost)) {
- cl_page_delete0(env, ghost, 0);
- cl_page_free(env, ghost);
- }
return page;
}
-
-struct cl_page *cl_page_find(const struct lu_env *env, struct cl_object *o,
- pgoff_t idx, struct page *vmpage,
- enum cl_page_type type)
-{
- return cl_page_find0(env, o, idx, vmpage, type, NULL);
-}
EXPORT_SYMBOL(cl_page_find);
-struct cl_page *cl_page_find_sub(const struct lu_env *env, struct cl_object *o,
- pgoff_t idx, struct page *vmpage,
- struct cl_page *parent)
-{
- return cl_page_find0(env, o, idx, vmpage, parent->cp_type, parent);
-}
-EXPORT_SYMBOL(cl_page_find_sub);
-
static inline int cl_page_invariant(const struct cl_page *pg)
{
- struct cl_object_header *header;
- struct cl_page *parent;
- struct cl_page *child;
- struct cl_io *owner;
-
/*
* Page invariant is protected by a VM lock.
*/
LINVRNT(cl_page_is_vmlocked(NULL, pg));
- header = cl_object_header(pg->cp_obj);
- parent = pg->cp_parent;
- child = pg->cp_child;
- owner = pg->cp_owner;
-
- return cl_page_in_use(pg) &&
- ergo(parent, parent->cp_child == pg) &&
- ergo(child, child->cp_parent == pg) &&
- ergo(child, pg->cp_obj != child->cp_obj) &&
- ergo(parent, pg->cp_obj != parent->cp_obj) &&
- ergo(owner && parent,
- parent->cp_owner == pg->cp_owner->ci_parent) &&
- ergo(owner && child, child->cp_owner->ci_parent == owner) &&
- /*
- * Either page is early in initialization (has neither child
- * nor parent yet), or it is in the object radix tree.
- */
- ergo(pg->cp_state < CPS_FREEING && pg->cp_type == CPT_CACHEABLE,
- (void *)radix_tree_lookup(&header->coh_tree,
- pg->cp_index) == pg ||
- (!child && !parent));
+ return cl_page_in_use_noref(pg);
}
static void cl_page_state_set0(const struct lu_env *env,
@@ -534,13 +291,9 @@
old = page->cp_state;
PASSERT(env, page, allowed_transitions[old][state]);
CL_PAGE_HEADER(D_TRACE, env, page, "%d -> %d\n", old, state);
- for (; page; page = page->cp_child) {
- PASSERT(env, page, page->cp_state == old);
- PASSERT(env, page,
- equi(state == CPS_OWNED, page->cp_owner));
-
- cl_page_state_set_trust(page, state);
- }
+ PASSERT(env, page, page->cp_state == old);
+ PASSERT(env, page, equi(state == CPS_OWNED, page->cp_owner));
+ cl_page_state_set_trust(page, state);
}
static void cl_page_state_set(const struct lu_env *env,
@@ -574,8 +327,6 @@
*/
void cl_page_put(const struct lu_env *env, struct cl_page *page)
{
- PASSERT(env, page, atomic_read(&page->cp_ref) > !!page->cp_parent);
-
CL_PAGE_HEADER(D_TRACE, env, page, "%d\n",
atomic_read(&page->cp_ref));
@@ -595,34 +346,10 @@
EXPORT_SYMBOL(cl_page_put);
/**
- * Returns a VM page associated with a given cl_page.
- */
-struct page *cl_page_vmpage(const struct lu_env *env, struct cl_page *page)
-{
- const struct cl_page_slice *slice;
-
- /*
- * Find uppermost layer with ->cpo_vmpage() method, and return its
- * result.
- */
- page = cl_page_top(page);
- do {
- list_for_each_entry(slice, &page->cp_layers, cpl_linkage) {
- if (slice->cpl_ops->cpo_vmpage)
- return slice->cpl_ops->cpo_vmpage(env, slice);
- }
- page = page->cp_child;
- } while (page);
- LBUG(); /* ->cpo_vmpage() has to be defined somewhere in the stack */
-}
-EXPORT_SYMBOL(cl_page_vmpage);
-
-/**
* Returns a cl_page associated with a VM page, and given cl_object.
*/
struct cl_page *cl_vmpage_page(struct page *vmpage, struct cl_object *obj)
{
- struct cl_page *top;
struct cl_page *page;
KLASSERT(PageLocked(vmpage));
@@ -633,36 +360,15 @@
* bottom-to-top pass.
*/
- /*
- * This loop assumes that ->private points to the top-most page. This
- * can be rectified easily.
- */
- top = (struct cl_page *)vmpage->private;
- if (!top)
- return NULL;
-
- for (page = top; page; page = page->cp_child) {
- if (cl_object_same(page->cp_obj, obj)) {
- cl_page_get_trust(page);
- break;
- }
+ page = (struct cl_page *)vmpage->private;
+ if (page) {
+ cl_page_get_trust(page);
+ LASSERT(page->cp_type == CPT_CACHEABLE);
}
- LASSERT(ergo(page, page->cp_type == CPT_CACHEABLE));
return page;
}
EXPORT_SYMBOL(cl_vmpage_page);
-/**
- * Returns the top-page for a given page.
- *
- * \see cl_object_top(), cl_io_top()
- */
-struct cl_page *cl_page_top(struct cl_page *page)
-{
- return cl_page_top_trusted(page);
-}
-EXPORT_SYMBOL(cl_page_top);
-
const struct cl_page_slice *cl_page_at(const struct cl_page *page,
const struct lu_device_type *dtype)
{
@@ -682,26 +388,43 @@
int (*__method)_proto; \
\
__result = 0; \
- __page = cl_page_top(__page); \
- do { \
- list_for_each_entry(__scan, &__page->cp_layers, \
- cpl_linkage) { \
- __method = *(void **)((char *)__scan->cpl_ops + \
- __op); \
- if (__method) { \
- __result = (*__method)(__env, __scan, \
- ## __VA_ARGS__); \
- if (__result != 0) \
- break; \
- } \
- } \
- __page = __page->cp_child; \
- } while (__page && __result == 0); \
+ list_for_each_entry(__scan, &__page->cp_layers, cpl_linkage) { \
+ __method = *(void **)((char *)__scan->cpl_ops + __op); \
+ if (__method) { \
+ __result = (*__method)(__env, __scan, ## __VA_ARGS__); \
+ if (__result != 0) \
+ break; \
+ } \
+ } \
if (__result > 0) \
__result = 0; \
__result; \
})
+#define CL_PAGE_INVOKE_REVERSE(_env, _page, _op, _proto, ...) \
+({ \
+ const struct lu_env *__env = (_env); \
+ struct cl_page *__page = (_page); \
+ const struct cl_page_slice *__scan; \
+ int __result; \
+ ptrdiff_t __op = (_op); \
+ int (*__method)_proto; \
+ \
+ __result = 0; \
+ list_for_each_entry_reverse(__scan, &__page->cp_layers, \
+ cpl_linkage) { \
+ __method = *(void **)((char *)__scan->cpl_ops + __op); \
+ if (__method) { \
+ __result = (*__method)(__env, __scan, ## __VA_ARGS__); \
+ if (__result != 0) \
+ break; \
+ } \
+ } \
+ if (__result > 0) \
+ __result = 0; \
+ __result; \
+})
+
#define CL_PAGE_INVOID(_env, _page, _op, _proto, ...) \
do { \
const struct lu_env *__env = (_env); \
@@ -710,18 +433,11 @@
ptrdiff_t __op = (_op); \
void (*__method)_proto; \
\
- __page = cl_page_top(__page); \
- do { \
- list_for_each_entry(__scan, &__page->cp_layers, \
- cpl_linkage) { \
- __method = *(void **)((char *)__scan->cpl_ops + \
- __op); \
- if (__method) \
- (*__method)(__env, __scan, \
- ## __VA_ARGS__); \
- } \
- __page = __page->cp_child; \
- } while (__page); \
+ list_for_each_entry(__scan, &__page->cp_layers, cpl_linkage) { \
+ __method = *(void **)((char *)__scan->cpl_ops + __op); \
+ if (__method) \
+ (*__method)(__env, __scan, ## __VA_ARGS__); \
+ } \
} while (0)
#define CL_PAGE_INVOID_REVERSE(_env, _page, _op, _proto, ...) \
@@ -732,20 +448,11 @@
ptrdiff_t __op = (_op); \
void (*__method)_proto; \
\
- /* get to the bottom page. */ \
- while (__page->cp_child) \
- __page = __page->cp_child; \
- do { \
- list_for_each_entry_reverse(__scan, &__page->cp_layers, \
- cpl_linkage) { \
- __method = *(void **)((char *)__scan->cpl_ops + \
- __op); \
- if (__method) \
- (*__method)(__env, __scan, \
- ## __VA_ARGS__); \
- } \
- __page = __page->cp_parent; \
- } while (__page); \
+ list_for_each_entry_reverse(__scan, &__page->cp_layers, cpl_linkage) { \
+ __method = *(void **)((char *)__scan->cpl_ops + __op); \
+ if (__method) \
+ (*__method)(__env, __scan, ## __VA_ARGS__); \
+ } \
} while (0)
static int cl_page_invoke(const struct lu_env *env,
@@ -771,20 +478,17 @@
static void cl_page_owner_clear(struct cl_page *page)
{
- for (page = cl_page_top(page); page; page = page->cp_child) {
- if (page->cp_owner) {
- LASSERT(page->cp_owner->ci_owned_nr > 0);
- page->cp_owner->ci_owned_nr--;
- page->cp_owner = NULL;
- page->cp_task = NULL;
- }
+ if (page->cp_owner) {
+ LASSERT(page->cp_owner->ci_owned_nr > 0);
+ page->cp_owner->ci_owned_nr--;
+ page->cp_owner = NULL;
+ page->cp_task = NULL;
}
}
static void cl_page_owner_set(struct cl_page *page)
{
- for (page = cl_page_top(page); page; page = page->cp_child)
- page->cp_owner->ci_owned_nr++;
+ page->cp_owner->ci_owned_nr++;
}
void cl_page_disown0(const struct lu_env *env,
@@ -794,7 +498,7 @@
state = pg->cp_state;
PINVRNT(env, pg, state == CPS_OWNED || state == CPS_FREEING);
- PINVRNT(env, pg, cl_page_invariant(pg));
+ PINVRNT(env, pg, cl_page_invariant(pg) || state == CPS_FREEING);
cl_page_owner_clear(pg);
if (state == CPS_OWNED)
@@ -815,8 +519,9 @@
*/
int cl_page_is_owned(const struct cl_page *pg, const struct cl_io *io)
{
+ struct cl_io *top = cl_io_top((struct cl_io *)io);
LINVRNT(cl_object_same(pg->cp_obj, io->ci_obj));
- return pg->cp_state == CPS_OWNED && pg->cp_owner == io;
+ return pg->cp_state == CPS_OWNED && pg->cp_owner == top;
}
EXPORT_SYMBOL(cl_page_is_owned);
@@ -847,7 +552,6 @@
PINVRNT(env, pg, !cl_page_is_owned(pg, io));
- pg = cl_page_top(pg);
io = cl_io_top(io);
if (pg->cp_state == CPS_FREEING) {
@@ -861,7 +565,7 @@
if (result == 0) {
PASSERT(env, pg, !pg->cp_owner);
PASSERT(env, pg, !pg->cp_req);
- pg->cp_owner = io;
+ pg->cp_owner = cl_io_top(io);
pg->cp_task = current;
cl_page_owner_set(pg);
if (pg->cp_state != CPS_FREEING) {
@@ -914,12 +618,11 @@
{
PINVRNT(env, pg, cl_object_same(pg->cp_obj, io->ci_obj));
- pg = cl_page_top(pg);
io = cl_io_top(io);
cl_page_invoid(env, io, pg, CL_PAGE_OP(cpo_assume));
PASSERT(env, pg, !pg->cp_owner);
- pg->cp_owner = io;
+ pg->cp_owner = cl_io_top(io);
pg->cp_task = current;
cl_page_owner_set(pg);
cl_page_state_set(env, pg, CPS_OWNED);
@@ -943,7 +646,6 @@
PINVRNT(env, pg, cl_page_is_owned(pg, io));
PINVRNT(env, pg, cl_page_invariant(pg));
- pg = cl_page_top(pg);
io = cl_io_top(io);
cl_page_owner_clear(pg);
cl_page_state_set(env, pg, CPS_CACHED);
@@ -968,9 +670,9 @@
void cl_page_disown(const struct lu_env *env,
struct cl_io *io, struct cl_page *pg)
{
- PINVRNT(env, pg, cl_page_is_owned(pg, io));
+ PINVRNT(env, pg, cl_page_is_owned(pg, io) ||
+ pg->cp_state == CPS_FREEING);
- pg = cl_page_top(pg);
io = cl_io_top(io);
cl_page_disown0(env, io, pg);
}
@@ -1001,12 +703,8 @@
* pages, e.g,. in a error handling cl_page_find()->cl_page_delete0()
* path. Doesn't check page invariant.
*/
-static void cl_page_delete0(const struct lu_env *env, struct cl_page *pg,
- int radix)
+static void cl_page_delete0(const struct lu_env *env, struct cl_page *pg)
{
- struct cl_page *tmp = pg;
-
- PASSERT(env, pg, pg == cl_page_top(pg));
PASSERT(env, pg, pg->cp_state != CPS_FREEING);
/*
@@ -1014,41 +712,11 @@
*/
cl_page_owner_clear(pg);
- /*
- * unexport the page firstly before freeing it so that
- * the page content is considered to be invalid.
- * We have to do this because a CPS_FREEING cl_page may
- * be NOT under the protection of a cl_lock.
- * Afterwards, if this page is found by other threads, then this
- * page will be forced to reread.
- */
- cl_page_export(env, pg, 0);
cl_page_state_set0(env, pg, CPS_FREEING);
- CL_PAGE_INVOID(env, pg, CL_PAGE_OP(cpo_delete),
- (const struct lu_env *, const struct cl_page_slice *));
-
- if (tmp->cp_type == CPT_CACHEABLE) {
- if (!radix)
- /* !radix means that @pg is not yet in the radix tree,
- * skip removing it.
- */
- tmp = pg->cp_child;
- for (; tmp; tmp = tmp->cp_child) {
- void *value;
- struct cl_object_header *hdr;
-
- hdr = cl_object_header(tmp->cp_obj);
- spin_lock(&hdr->coh_page_guard);
- value = radix_tree_delete(&hdr->coh_tree,
- tmp->cp_index);
- PASSERT(env, tmp, value == tmp);
- PASSERT(env, tmp, hdr->coh_pages > 0);
- hdr->coh_pages--;
- spin_unlock(&hdr->coh_page_guard);
- cl_page_put(env, tmp);
- }
- }
+ CL_PAGE_INVOID_REVERSE(env, pg, CL_PAGE_OP(cpo_delete),
+ (const struct lu_env *,
+ const struct cl_page_slice *));
}
/**
@@ -1070,7 +738,6 @@
* Once page reaches cl_page_state::CPS_FREEING, all remaining references will
* drain after some time, at which point page will be recycled.
*
- * \pre pg == cl_page_top(pg)
* \pre VM page is locked
* \post pg->cp_state == CPS_FREEING
*
@@ -1079,30 +746,11 @@
void cl_page_delete(const struct lu_env *env, struct cl_page *pg)
{
PINVRNT(env, pg, cl_page_invariant(pg));
- cl_page_delete0(env, pg, 1);
+ cl_page_delete0(env, pg);
}
EXPORT_SYMBOL(cl_page_delete);
/**
- * Unmaps page from user virtual memory.
- *
- * Calls cl_page_operations::cpo_unmap() through all layers top-to-bottom. The
- * layer responsible for VM interaction has to unmap page from user space
- * virtual memory.
- *
- * \see cl_page_operations::cpo_unmap()
- */
-int cl_page_unmap(const struct lu_env *env,
- struct cl_io *io, struct cl_page *pg)
-{
- PINVRNT(env, pg, cl_page_is_owned(pg, io));
- PINVRNT(env, pg, cl_page_invariant(pg));
-
- return cl_page_invoke(env, io, pg, CL_PAGE_OP(cpo_unmap));
-}
-EXPORT_SYMBOL(cl_page_unmap);
-
-/**
* Marks page up-to-date.
*
* Call cl_page_operations::cpo_export() through all layers top-to-bottom. The
@@ -1129,7 +777,6 @@
int result;
const struct cl_page_slice *slice;
- pg = cl_page_top_trusted((struct cl_page *)pg);
slice = container_of(pg->cp_layers.next,
const struct cl_page_slice, cpl_linkage);
PASSERT(env, pg, slice->cpl_ops->cpo_is_vmlocked);
@@ -1241,7 +888,7 @@
cl_page_put(env, pg);
if (anchor)
- cl_sync_io_note(anchor, ioret);
+ cl_sync_io_note(env, anchor, ioret);
}
EXPORT_SYMBOL(cl_page_completion);
@@ -1276,44 +923,6 @@
EXPORT_SYMBOL(cl_page_make_ready);
/**
- * Notify layers that high level io decided to place this page into a cache
- * for future transfer.
- *
- * The layer implementing transfer engine (osc) has to register this page in
- * its queues.
- *
- * \pre cl_page_is_owned(pg, io)
- * \post cl_page_is_owned(pg, io)
- *
- * \see cl_page_operations::cpo_cache_add()
- */
-int cl_page_cache_add(const struct lu_env *env, struct cl_io *io,
- struct cl_page *pg, enum cl_req_type crt)
-{
- const struct cl_page_slice *scan;
- int result = 0;
-
- PINVRNT(env, pg, crt < CRT_NR);
- PINVRNT(env, pg, cl_page_is_owned(pg, io));
- PINVRNT(env, pg, cl_page_invariant(pg));
-
- if (crt >= CRT_NR)
- return -EINVAL;
-
- list_for_each_entry(scan, &pg->cp_layers, cpl_linkage) {
- if (!scan->cpl_ops->io[crt].cpo_cache_add)
- continue;
-
- result = scan->cpl_ops->io[crt].cpo_cache_add(env, scan, io);
- if (result != 0)
- break;
- }
- CL_PAGE_HEADER(D_TRACE, env, pg, "%d %d\n", crt, result);
- return result;
-}
-EXPORT_SYMBOL(cl_page_cache_add);
-
-/**
* Called if a pge is being written back by kernel's intention.
*
* \pre cl_page_is_owned(pg, io)
@@ -1344,68 +953,21 @@
* \see cl_page_operations::cpo_is_under_lock()
*/
int cl_page_is_under_lock(const struct lu_env *env, struct cl_io *io,
- struct cl_page *page)
+ struct cl_page *page, pgoff_t *max_index)
{
int rc;
PINVRNT(env, page, cl_page_invariant(page));
- rc = CL_PAGE_INVOKE(env, page, CL_PAGE_OP(cpo_is_under_lock),
- (const struct lu_env *,
- const struct cl_page_slice *, struct cl_io *),
- io);
- PASSERT(env, page, rc != 0);
+ rc = CL_PAGE_INVOKE_REVERSE(env, page, CL_PAGE_OP(cpo_is_under_lock),
+ (const struct lu_env *,
+ const struct cl_page_slice *,
+ struct cl_io *, pgoff_t *),
+ io, max_index);
return rc;
}
EXPORT_SYMBOL(cl_page_is_under_lock);
-static int page_prune_cb(const struct lu_env *env, struct cl_io *io,
- struct cl_page *page, void *cbdata)
-{
- cl_page_own(env, io, page);
- cl_page_unmap(env, io, page);
- cl_page_discard(env, io, page);
- cl_page_disown(env, io, page);
- return CLP_GANG_OKAY;
-}
-
-/**
- * Purges all cached pages belonging to the object \a obj.
- */
-int cl_pages_prune(const struct lu_env *env, struct cl_object *clobj)
-{
- struct cl_thread_info *info;
- struct cl_object *obj = cl_object_top(clobj);
- struct cl_io *io;
- int result;
-
- info = cl_env_info(env);
- io = &info->clt_io;
-
- /*
- * initialize the io. This is ugly since we never do IO in this
- * function, we just make cl_page_list functions happy. -jay
- */
- io->ci_obj = obj;
- io->ci_ignore_layout = 1;
- result = cl_io_init(env, io, CIT_MISC, obj);
- if (result != 0) {
- cl_io_fini(env, io);
- return io->ci_result;
- }
-
- do {
- result = cl_page_gang_lookup(env, obj, io, 0, CL_PAGE_EOF,
- page_prune_cb, NULL);
- if (result == CLP_GANG_RESCHED)
- cond_resched();
- } while (result != CLP_GANG_OKAY);
-
- cl_io_fini(env, io);
- return result;
-}
-EXPORT_SYMBOL(cl_pages_prune);
-
/**
* Tells transfer engine that only part of a page is to be transmitted.
*
@@ -1431,9 +993,8 @@
lu_printer_t printer, const struct cl_page *pg)
{
(*printer)(env, cookie,
- "page@%p[%d %p:%lu ^%p_%p %d %d %d %p %p %#x]\n",
+ "page@%p[%d %p %d %d %d %p %p %#x]\n",
pg, atomic_read(&pg->cp_ref), pg->cp_obj,
- pg->cp_index, pg->cp_parent, pg->cp_child,
pg->cp_state, pg->cp_error, pg->cp_type,
pg->cp_owner, pg->cp_req, pg->cp_flags);
}
@@ -1445,11 +1006,7 @@
void cl_page_print(const struct lu_env *env, void *cookie,
lu_printer_t printer, const struct cl_page *pg)
{
- struct cl_page *scan;
-
- for (scan = cl_page_top((struct cl_page *)pg); scan;
- scan = scan->cp_child)
- cl_page_header_print(env, cookie, printer, scan);
+ cl_page_header_print(env, cookie, printer, pg);
CL_PAGE_INVOKE(env, (struct cl_page *)pg, CL_PAGE_OP(cpo_print),
(const struct lu_env *env,
const struct cl_page_slice *slice,
@@ -1509,21 +1066,13 @@
* \see cl_lock_slice_add(), cl_req_slice_add(), cl_io_slice_add()
*/
void cl_page_slice_add(struct cl_page *page, struct cl_page_slice *slice,
- struct cl_object *obj,
+ struct cl_object *obj, pgoff_t index,
const struct cl_page_operations *ops)
{
list_add_tail(&slice->cpl_linkage, &page->cp_layers);
slice->cpl_obj = obj;
+ slice->cpl_index = index;
slice->cpl_ops = ops;
slice->cpl_page = page;
}
EXPORT_SYMBOL(cl_page_slice_add);
-
-int cl_page_init(void)
-{
- return 0;
-}
-
-void cl_page_fini(void)
-{
-}
diff --git a/drivers/staging/lustre/lustre/obdclass/class_obd.c b/drivers/staging/lustre/lustre/obdclass/class_obd.c
index c2cf015..f48816a 100644
--- a/drivers/staging/lustre/lustre/obdclass/class_obd.c
+++ b/drivers/staging/lustre/lustre/obdclass/class_obd.c
@@ -60,6 +60,8 @@
EXPORT_SYMBOL(obd_dump_on_eviction);
unsigned int obd_max_dirty_pages = 256;
EXPORT_SYMBOL(obd_max_dirty_pages);
+atomic_t obd_unstable_pages;
+EXPORT_SYMBOL(obd_unstable_pages);
atomic_t obd_dirty_pages;
EXPORT_SYMBOL(obd_dirty_pages);
unsigned int obd_timeout = OBD_TIMEOUT_DEFAULT; /* seconds */
@@ -335,7 +337,6 @@
err = 0;
goto out;
}
-
}
if (data->ioc_dev == OBD_DEV_BY_DEVNAME) {
@@ -461,7 +462,7 @@
CWARN("LPD64 wrong length! strlen(%s)=%d != 2\n", buf, len);
ret = -EINVAL;
}
- if ((u64val & ~CFS_PAGE_MASK) >= PAGE_SIZE) {
+ if ((u64val & ~PAGE_MASK) >= PAGE_SIZE) {
CWARN("mask failed: u64val %llu >= %llu\n", u64val,
(__u64)PAGE_SIZE);
ret = -EINVAL;
diff --git a/drivers/staging/lustre/lustre/obdclass/debug.c b/drivers/staging/lustre/lustre/obdclass/debug.c
index 43a7f7a..e4edfb2 100644
--- a/drivers/staging/lustre/lustre/obdclass/debug.c
+++ b/drivers/staging/lustre/lustre/obdclass/debug.c
@@ -68,8 +68,8 @@
LASSERT(addr);
- ne_off = le64_to_cpu (off);
- id = le64_to_cpu (id);
+ ne_off = le64_to_cpu(off);
+ id = le64_to_cpu(id);
if (memcmp(addr, (char *)&ne_off, LPDS)) {
CDEBUG(D_ERROR, "%s: id %#llx offset %llu off: %#llx != %#llx\n",
who, id, off, *(__u64 *)addr, ne_off);
diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c b/drivers/staging/lustre/lustre/obdclass/genops.c
index cf97b8f0..d95f11d 100644
--- a/drivers/staging/lustre/lustre/obdclass/genops.c
+++ b/drivers/staging/lustre/lustre/obdclass/genops.c
@@ -604,7 +604,6 @@
out:
obd_cleanup_caches();
return -ENOMEM;
-
}
/* map connection to client */
diff --git a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c
index 8eddf20..2cd4522 100644
--- a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c
+++ b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c
@@ -158,9 +158,7 @@
{
int err;
- err = copy_to_user(arg, data, len);
- if (err)
- err = -EFAULT;
+ err = copy_to_user(arg, data, len) ? -EFAULT : 0;
return err;
}
EXPORT_SYMBOL(obd_ioctl_popdata);
diff --git a/drivers/staging/lustre/lustre/obdclass/llog.c b/drivers/staging/lustre/lustre/obdclass/llog.c
index 992573e..79194d8 100644
--- a/drivers/staging/lustre/lustre/obdclass/llog.c
+++ b/drivers/staging/lustre/lustre/obdclass/llog.c
@@ -265,7 +265,6 @@
for (rec = (struct llog_rec_hdr *)buf;
(char *)rec < buf + LLOG_CHUNK_SIZE;
rec = (struct llog_rec_hdr *)((char *)rec + rec->lrh_len)) {
-
CDEBUG(D_OTHER, "processing rec 0x%p type %#x\n",
rec, rec->lrh_type);
diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
index d93f42f..5a1eae1 100644
--- a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
+++ b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
@@ -49,7 +49,7 @@
static const char * const obd_connect_names[] = {
"read_only",
"lov_index",
- "unused",
+ "connect_from_mds",
"write_grant",
"server_lock",
"version",
@@ -122,6 +122,56 @@
}
EXPORT_SYMBOL(obd_connect_flags2str);
+static void obd_connect_data_seqprint(struct seq_file *m,
+ struct obd_connect_data *ocd)
+{
+ int flags;
+
+ LASSERT(ocd);
+ flags = ocd->ocd_connect_flags;
+
+ seq_printf(m, " connect_data:\n"
+ " flags: %llx\n"
+ " instance: %u\n",
+ ocd->ocd_connect_flags,
+ ocd->ocd_instance);
+ if (flags & OBD_CONNECT_VERSION)
+ seq_printf(m, " target_version: %u.%u.%u.%u\n",
+ OBD_OCD_VERSION_MAJOR(ocd->ocd_version),
+ OBD_OCD_VERSION_MINOR(ocd->ocd_version),
+ OBD_OCD_VERSION_PATCH(ocd->ocd_version),
+ OBD_OCD_VERSION_FIX(ocd->ocd_version));
+ if (flags & OBD_CONNECT_MDS)
+ seq_printf(m, " mdt_index: %d\n", ocd->ocd_group);
+ if (flags & OBD_CONNECT_GRANT)
+ seq_printf(m, " initial_grant: %d\n", ocd->ocd_grant);
+ if (flags & OBD_CONNECT_INDEX)
+ seq_printf(m, " target_index: %u\n", ocd->ocd_index);
+ if (flags & OBD_CONNECT_BRW_SIZE)
+ seq_printf(m, " max_brw_size: %d\n", ocd->ocd_brw_size);
+ if (flags & OBD_CONNECT_IBITS)
+ seq_printf(m, " ibits_known: %llx\n",
+ ocd->ocd_ibits_known);
+ if (flags & OBD_CONNECT_GRANT_PARAM)
+ seq_printf(m, " grant_block_size: %d\n"
+ " grant_inode_size: %d\n"
+ " grant_extent_overhead: %d\n",
+ ocd->ocd_blocksize,
+ ocd->ocd_inodespace,
+ ocd->ocd_grant_extent);
+ if (flags & OBD_CONNECT_TRANSNO)
+ seq_printf(m, " first_transno: %llx\n",
+ ocd->ocd_transno);
+ if (flags & OBD_CONNECT_CKSUM)
+ seq_printf(m, " cksum_types: %#x\n",
+ ocd->ocd_cksum_types);
+ if (flags & OBD_CONNECT_MAX_EASIZE)
+ seq_printf(m, " max_easize: %d\n", ocd->ocd_max_easize);
+ if (flags & OBD_CONNECT_MAXBYTES)
+ seq_printf(m, " max_object_bytes: %llx\n",
+ ocd->ocd_maxbytes);
+}
+
int lprocfs_read_frac_helper(char *buffer, unsigned long count, long val,
int mult)
{
@@ -624,6 +674,7 @@
struct obd_device *obd = data;
struct obd_import *imp;
struct obd_import_conn *conn;
+ struct obd_connect_data *ocd;
int j;
int k;
int rw = 0;
@@ -635,9 +686,9 @@
return rc;
imp = obd->u.cli.cl_import;
+ ocd = &imp->imp_connect_data;
- seq_printf(m,
- "import:\n"
+ seq_printf(m, "import:\n"
" name: %s\n"
" target: %s\n"
" state: %s\n"
@@ -649,9 +700,9 @@
imp->imp_connect_data.ocd_instance);
obd_connect_seq_flags2str(m, imp->imp_connect_data.ocd_connect_flags,
", ");
- seq_printf(m,
- " ]\n"
- " import_flags: [ ");
+ seq_printf(m, " ]\n");
+ obd_connect_data_seqprint(m, ocd);
+ seq_printf(m, " import_flags: [ ");
obd_import_flags2str(imp, m);
seq_printf(m,
@@ -694,8 +745,9 @@
do_div(sum, ret.lc_count);
ret.lc_sum = sum;
- } else
+ } else {
ret.lc_sum = 0;
+ }
seq_printf(m,
" rpcs:\n"
" inflight: %u\n"
@@ -1471,10 +1523,10 @@
void lprocfs_oh_tally_log2(struct obd_histogram *oh, unsigned int value)
{
- unsigned int val;
+ unsigned int val = 0;
- for (val = 0; ((1 << val) < value) && (val <= OBD_HIST_MAX); val++)
- ;
+ if (likely(value != 0))
+ val = min(fls(value - 1), OBD_HIST_MAX);
lprocfs_oh_tally(oh, val);
}
diff --git a/drivers/staging/lustre/lustre/obdclass/lu_object.c b/drivers/staging/lustre/lustre/obdclass/lu_object.c
index 978568a..e043857 100644
--- a/drivers/staging/lustre/lustre/obdclass/lu_object.c
+++ b/drivers/staging/lustre/lustre/obdclass/lu_object.c
@@ -55,6 +55,7 @@
#include "../include/lustre_disk.h"
#include "../include/lustre_fid.h"
#include "../include/lu_object.h"
+#include "../include/cl_object.h"
#include "../include/lu_ref.h"
#include <linux/list.h>
@@ -103,7 +104,6 @@
if (!cfs_hash_bd_dec_and_lock(site->ls_obj_hash, &bd, &top->loh_ref)) {
if (lu_object_is_dying(top)) {
-
/*
* somebody may be waiting for this, currently only
* used for cl_object, see cl_object_put_last().
@@ -357,7 +357,6 @@
if (count > 0 && --count == 0)
break;
-
}
cfs_hash_bd_unlock(s->ls_obj_hash, &bd, 1);
cond_resched();
@@ -715,8 +714,9 @@
obj = lu_object_locate(top->lo_header, dev->ld_type);
if (!obj)
lu_object_put(env, top);
- } else
+ } else {
obj = top;
+ }
return obj;
}
EXPORT_SYMBOL(lu_object_find_slice);
@@ -935,7 +935,7 @@
* Initialize site \a s, with \a d as the top level device.
*/
#define LU_SITE_BITS_MIN 12
-#define LU_SITE_BITS_MAX 24
+#define LU_SITE_BITS_MAX 19
/**
* total 256 buckets, we don't want too many buckets because:
* - consume too much memory
@@ -1468,6 +1468,7 @@
/*
* XXX layering violation.
*/
+ cl_env_cache_purge(~0);
key->lct_tags |= LCT_QUIESCENT;
/*
* XXX memory barrier has to go here.
diff --git a/drivers/staging/lustre/lustre/obdclass/lustre_peer.c b/drivers/staging/lustre/lustre/obdclass/lustre_peer.c
index 5f81246..b1abe02 100644
--- a/drivers/staging/lustre/lustre/obdclass/lustre_peer.c
+++ b/drivers/staging/lustre/lustre/obdclass/lustre_peer.c
@@ -163,8 +163,9 @@
break;
}
}
- } else
+ } else {
list_splice_init(&g_uuid_list, &deathrow);
+ }
spin_unlock(&g_uuid_lock);
if (uuid && list_empty(&deathrow)) {
diff --git a/drivers/staging/lustre/lustre/obdclass/obd_config.c b/drivers/staging/lustre/lustre/obdclass/obd_config.c
index 5395e99..cb1d65c 100644
--- a/drivers/staging/lustre/lustre/obdclass/obd_config.c
+++ b/drivers/staging/lustre/lustre/obdclass/obd_config.c
@@ -606,7 +606,7 @@
return rc;
}
-LIST_HEAD(lustre_profile_list);
+static LIST_HEAD(lustre_profile_list);
struct lustre_profile *class_get_profile(const char *prof)
{
@@ -961,7 +961,6 @@
default: {
err = obd_process_config(obd, sizeof(*lcfg), lcfg);
goto out;
-
}
}
out:
@@ -1001,7 +1000,13 @@
for (i = 1; i < lcfg->lcfg_bufcount; i++) {
key = lustre_cfg_buf(lcfg, i);
/* Strip off prefix */
- class_match_param(key, prefix, &key);
+ if (class_match_param(key, prefix, &key)) {
+ /*
+ * If the prefix doesn't match, return error so we
+ * can pass it down the stack
+ */
+ return -ENOSYS;
+ }
sval = strchr(key, '=');
if (!sval || (*(sval + 1) == 0)) {
CERROR("Can't parse param %s (missing '=')\n", key);
@@ -1034,18 +1039,14 @@
j++;
}
if (!matched) {
- /* If the prefix doesn't match, return error so we
- * can pass it down the stack
- */
- if (strnchr(key, keylen, '.'))
- return -ENOSYS;
- CERROR("%s: unknown param %s\n",
+ CERROR("%.*s: %s unknown param %s\n",
+ (int)strlen(prefix) - 1, prefix,
(char *)lustre_cfg_string(lcfg, 0), key);
/* rc = -EINVAL; continue parsing other params */
skip++;
} else if (rc < 0) {
- CERROR("writing proc entry %s err %d\n",
- var->name, rc);
+ CERROR("%s: error writing proc entry '%s': rc = %d\n",
+ prefix, var->name, rc);
rc = 0;
} else {
CDEBUG(D_CONFIG, "%s.%.*s: Set parameter %.*s=%s\n",
@@ -1350,6 +1351,7 @@
lustre_cfg_string(lcfg, i));
}
}
+ ptr += snprintf(ptr, end - ptr, "\n");
/* return consumed bytes */
rc = ptr - buf;
return rc;
@@ -1368,7 +1370,7 @@
if (rec->lrh_type == OBD_CFG_REC) {
class_config_parse_rec(rec, outstr, 256);
- LCONSOLE(D_WARNING, " %s\n", outstr);
+ LCONSOLE(D_WARNING, " %s", outstr);
} else {
LCONSOLE(D_WARNING, "unhandled lrh_type: %#x\n", rec->lrh_type);
rc = -EINVAL;
diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
index d3e28a3..e0c90ad 100644
--- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c
+++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
@@ -102,7 +102,7 @@
LCONSOLE_ERROR_MSG(0x15b, "%s: The configuration from log '%s' failed from the MGS (%d). Make sure this client and the MGS are running compatible versions of Lustre.\n",
mgc->obd_name, logname, rc);
- if (rc)
+ else if (rc)
LCONSOLE_ERROR_MSG(0x15c, "%s: The configuration from log '%s' failed (%d). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information.\n",
mgc->obd_name, logname,
rc);
@@ -307,7 +307,8 @@
while (class_parse_nid(ptr, &nid, &ptr) == 0) {
rc = do_lcfg(mgcname, nid,
LCFG_ADD_UUID, niduuid, NULL, NULL, NULL);
- i++;
+ if (!rc)
+ i++;
/* Stop at the first failover nid */
if (*ptr == ':')
break;
@@ -345,16 +346,18 @@
sprintf(niduuid, "%s_%x", mgcname, i);
j = 0;
while (class_parse_nid_quiet(ptr, &nid, &ptr) == 0) {
- j++;
- rc = do_lcfg(mgcname, nid,
- LCFG_ADD_UUID, niduuid, NULL, NULL, NULL);
+ rc = do_lcfg(mgcname, nid, LCFG_ADD_UUID, niduuid,
+ NULL, NULL, NULL);
+ if (!rc)
+ ++j;
if (*ptr == ':')
break;
}
if (j > 0) {
rc = do_lcfg(mgcname, 0, LCFG_ADD_CONN,
niduuid, NULL, NULL, NULL);
- i++;
+ if (!rc)
+ i++;
} else {
/* at ":/fsname" */
break;
diff --git a/drivers/staging/lustre/lustre/obdclass/obdo.c b/drivers/staging/lustre/lustre/obdclass/obdo.c
index e6436cb..748e33f 100644
--- a/drivers/staging/lustre/lustre/obdclass/obdo.c
+++ b/drivers/staging/lustre/lustre/obdclass/obdo.c
@@ -185,8 +185,7 @@
op_data->op_attr.ia_valid |= ATTR_BLOCKS;
}
if (valid & OBD_MD_FLFLAGS) {
- ((struct ll_iattr *)&op_data->op_attr)->ia_attr_flags =
- oa->o_flags;
+ op_data->op_attr_flags = oa->o_flags;
op_data->op_attr.ia_valid |= ATTR_ATTR_FLAG;
}
}
diff --git a/drivers/staging/lustre/lustre/obdecho/echo_client.c b/drivers/staging/lustre/lustre/obdecho/echo_client.c
index 1e83669..91ef06f 100644
--- a/drivers/staging/lustre/lustre/obdecho/echo_client.c
+++ b/drivers/staging/lustre/lustre/obdecho/echo_client.c
@@ -81,7 +81,6 @@
struct echo_page {
struct cl_page_slice ep_cl;
struct mutex ep_lock;
- struct page *ep_vmpage;
};
struct echo_lock {
@@ -164,15 +163,13 @@
static int cl_echo_object_brw(struct echo_object *eco, int rw, u64 offset,
struct page **pages, int npages, int async);
-static struct echo_thread_info *echo_env_info(const struct lu_env *env);
-
struct echo_thread_info {
struct echo_object_conf eti_conf;
struct lustre_md eti_md;
struct cl_2queue eti_queue;
struct cl_io eti_io;
- struct cl_lock_descr eti_descr;
+ struct cl_lock eti_lock;
struct lu_fid eti_fid;
struct lu_fid eti_fid2;
};
@@ -219,12 +216,6 @@
*
* @{
*/
-static struct page *echo_page_vmpage(const struct lu_env *env,
- const struct cl_page_slice *slice)
-{
- return cl2echo_page(slice)->ep_vmpage;
-}
-
static int echo_page_own(const struct lu_env *env,
const struct cl_page_slice *slice,
struct cl_io *io, int nonblock)
@@ -273,12 +264,10 @@
static void echo_page_fini(const struct lu_env *env,
struct cl_page_slice *slice)
{
- struct echo_page *ep = cl2echo_page(slice);
struct echo_object *eco = cl2echo_obj(slice->cpl_obj);
- struct page *vmpage = ep->ep_vmpage;
atomic_dec(&eco->eo_npages);
- put_page(vmpage);
+ put_page(slice->cpl_page->cp_vmpage);
}
static int echo_page_prep(const struct lu_env *env,
@@ -295,7 +284,8 @@
struct echo_page *ep = cl2echo_page(slice);
(*printer)(env, cookie, LUSTRE_ECHO_CLIENT_NAME"-page@%p %d vm@%p\n",
- ep, mutex_is_locked(&ep->ep_lock), ep->ep_vmpage);
+ ep, mutex_is_locked(&ep->ep_lock),
+ slice->cpl_page->cp_vmpage);
return 0;
}
@@ -303,7 +293,6 @@
.cpo_own = echo_page_own,
.cpo_disown = echo_page_disown,
.cpo_discard = echo_page_discard,
- .cpo_vmpage = echo_page_vmpage,
.cpo_fini = echo_page_fini,
.cpo_print = echo_page_print,
.cpo_is_vmlocked = echo_page_is_vmlocked,
@@ -336,26 +325,8 @@
kmem_cache_free(echo_lock_kmem, ecl);
}
-static void echo_lock_delete(const struct lu_env *env,
- const struct cl_lock_slice *slice)
-{
- struct echo_lock *ecl = cl2echo_lock(slice);
-
- LASSERT(list_empty(&ecl->el_chain));
-}
-
-static int echo_lock_fits_into(const struct lu_env *env,
- const struct cl_lock_slice *slice,
- const struct cl_lock_descr *need,
- const struct cl_io *unused)
-{
- return 1;
-}
-
static struct cl_lock_operations echo_lock_ops = {
.clo_fini = echo_lock_fini,
- .clo_delete = echo_lock_delete,
- .clo_fits_into = echo_lock_fits_into
};
/** @} echo_lock */
@@ -367,15 +338,14 @@
* @{
*/
static int echo_page_init(const struct lu_env *env, struct cl_object *obj,
- struct cl_page *page, struct page *vmpage)
+ struct cl_page *page, pgoff_t index)
{
struct echo_page *ep = cl_object_page_slice(obj, page);
struct echo_object *eco = cl2echo_obj(obj);
- ep->ep_vmpage = vmpage;
- get_page(vmpage);
+ get_page(page->cp_vmpage);
mutex_init(&ep->ep_lock);
- cl_page_slice_add(page, &ep->ep_cl, obj, &echo_page_ops);
+ cl_page_slice_add(page, &ep->ep_cl, obj, index, &echo_page_ops);
atomic_inc(&eco->eo_npages);
return 0;
}
@@ -568,6 +538,8 @@
obj = &echo_obj2cl(eco)->co_lu;
cl_object_header_init(hdr);
+ hdr->coh_page_bufsize = cfs_size_round(sizeof(struct cl_page));
+
lu_object_init(obj, &hdr->coh_lu, dev);
lu_object_add_top(&hdr->coh_lu, obj);
@@ -694,8 +666,7 @@
struct obd_device *obd = NULL; /* to keep compiler happy */
struct obd_device *tgt;
const char *tgt_type_name;
- int rc;
- int cleanup = 0;
+ int rc, err;
ed = kzalloc(sizeof(*ed), GFP_NOFS);
if (!ed) {
@@ -703,16 +674,14 @@
goto out;
}
- cleanup = 1;
cd = &ed->ed_cl;
rc = cl_device_init(cd, t);
if (rc)
- goto out;
+ goto out_free;
cd->cd_lu_dev.ld_ops = &echo_device_lu_ops;
cd->cd_ops = &echo_device_cl_ops;
- cleanup = 2;
obd = class_name2obd(lustre_cfg_string(cfg, 0));
LASSERT(obd);
LASSERT(env);
@@ -722,28 +691,25 @@
CERROR("Can not find tgt device %s\n",
lustre_cfg_string(cfg, 1));
rc = -ENODEV;
- goto out;
+ goto out_device_fini;
}
next = tgt->obd_lu_dev;
if (!strcmp(tgt->obd_type->typ_name, LUSTRE_MDT_NAME)) {
CERROR("echo MDT client must be run on server\n");
rc = -EOPNOTSUPP;
- goto out;
+ goto out_device_fini;
}
rc = echo_site_init(env, ed);
if (rc)
- goto out;
-
- cleanup = 3;
+ goto out_device_fini;
rc = echo_client_setup(env, obd, cfg);
if (rc)
- goto out;
+ goto out_site_fini;
ed->ed_ec = &obd->u.echo_client;
- cleanup = 4;
/* if echo client is to be stacked upon ost device, the next is
* NULL since ost is not a clio device so far
@@ -755,7 +721,7 @@
if (next) {
if (next->ld_site) {
rc = -EBUSY;
- goto out;
+ goto out_cleanup;
}
next->ld_site = &ed->ed_site->cs_lu;
@@ -763,7 +729,7 @@
next->ld_type->ldt_name,
NULL);
if (rc)
- goto out;
+ goto out_cleanup;
} else {
LASSERT(strcmp(tgt_type_name, LUSTRE_OST_NAME) == 0);
@@ -771,27 +737,19 @@
ed->ed_next = next;
return &cd->cd_lu_dev;
+
+out_cleanup:
+ err = echo_client_cleanup(obd);
+ if (err)
+ CERROR("Cleanup obd device %s error(%d)\n",
+ obd->obd_name, err);
+out_site_fini:
+ echo_site_fini(env, ed);
+out_device_fini:
+ cl_device_fini(&ed->ed_cl);
+out_free:
+ kfree(ed);
out:
- switch (cleanup) {
- case 4: {
- int rc2;
-
- rc2 = echo_client_cleanup(obd);
- if (rc2)
- CERROR("Cleanup obd device %s error(%d)\n",
- obd->obd_name, rc2);
- }
-
- case 3:
- echo_site_fini(env, ed);
- case 2:
- cl_device_fini(&ed->ed_cl);
- case 1:
- kfree(ed);
- case 0:
- default:
- break;
- }
return ERR_PTR(rc);
}
@@ -819,16 +777,7 @@
{
struct cl_lock *clk = echo_lock2cl(ecl);
- cl_lock_get(clk);
- cl_unuse(env, clk);
- cl_lock_release(env, clk, "ec enqueue", ecl->el_object);
- if (!still_used) {
- cl_lock_mutex_get(env, clk);
- cl_lock_cancel(env, clk);
- cl_lock_delete(env, clk);
- cl_lock_mutex_put(env, clk);
- }
- cl_lock_put(env, clk);
+ cl_lock_release(env, clk);
}
static struct lu_device *echo_device_free(const struct lu_env *env,
@@ -1022,9 +971,11 @@
info = echo_env_info(env);
io = &info->eti_io;
- descr = &info->eti_descr;
+ lck = &info->eti_lock;
obj = echo_obj2cl(eco);
+ memset(lck, 0, sizeof(*lck));
+ descr = &lck->cll_descr;
descr->cld_obj = obj;
descr->cld_start = cl_index(obj, start);
descr->cld_end = cl_index(obj, end);
@@ -1032,25 +983,20 @@
descr->cld_enq_flags = enqflags;
io->ci_obj = obj;
- lck = cl_lock_request(env, io, descr, "ec enqueue", eco);
- if (lck) {
+ rc = cl_lock_request(env, io, lck);
+ if (rc == 0) {
struct echo_client_obd *ec = eco->eo_dev->ed_ec;
struct echo_lock *el;
- rc = cl_wait(env, lck);
- if (rc == 0) {
- el = cl2echo_lock(cl_lock_at(lck, &echo_device_type));
- spin_lock(&ec->ec_lock);
- if (list_empty(&el->el_chain)) {
- list_add(&el->el_chain, &ec->ec_locks);
- el->el_cookie = ++ec->ec_unique;
- }
- atomic_inc(&el->el_refcount);
- *cookie = el->el_cookie;
- spin_unlock(&ec->ec_lock);
- } else {
- cl_lock_release(env, lck, "ec enqueue", current);
+ el = cl2echo_lock(cl_lock_at(lck, &echo_device_type));
+ spin_lock(&ec->ec_lock);
+ if (list_empty(&el->el_chain)) {
+ list_add(&el->el_chain, &ec->ec_locks);
+ el->el_cookie = ++ec->ec_unique;
}
+ atomic_inc(&el->el_refcount);
+ *cookie = el->el_cookie;
+ spin_unlock(&ec->ec_lock);
}
return rc;
}
@@ -1085,22 +1031,17 @@
return 0;
}
-static int cl_echo_async_brw(const struct lu_env *env, struct cl_io *io,
- enum cl_req_type unused, struct cl_2queue *queue)
+static void echo_commit_callback(const struct lu_env *env, struct cl_io *io,
+ struct cl_page *page)
{
- struct cl_page *clp;
- struct cl_page *temp;
- int result = 0;
+ struct echo_thread_info *info;
+ struct cl_2queue *queue;
- cl_page_list_for_each_safe(clp, temp, &queue->c2_qin) {
- int rc;
+ info = echo_env_info(env);
+ LASSERT(io == &info->eti_io);
- rc = cl_page_cache_add(env, io, clp, CRT_WRITE);
- if (rc == 0)
- continue;
- result = result ?: rc;
- }
- return result;
+ queue = &info->eti_queue;
+ cl_page_list_add(&queue->c2_qout, page);
}
static int cl_echo_object_brw(struct echo_object *eco, int rw, u64 offset,
@@ -1119,7 +1060,7 @@
int rc;
int i;
- LASSERT((offset & ~CFS_PAGE_MASK) == 0);
+ LASSERT((offset & ~PAGE_MASK) == 0);
LASSERT(ed->ed_next);
env = cl_env_get(&refcheck);
if (IS_ERR(env))
@@ -1179,7 +1120,9 @@
async = async && (typ == CRT_WRITE);
if (async)
- rc = cl_echo_async_brw(env, io, typ, queue);
+ rc = cl_io_commit_async(env, io, &queue->c2_qin,
+ 0, PAGE_SIZE,
+ echo_commit_callback);
else
rc = cl_io_submit_sync(env, io, typ, queue, 0);
CDEBUG(D_INFO, "echo_client %s write returns %d\n",
@@ -1387,7 +1330,7 @@
LASSERT(rw == OBD_BRW_WRITE || rw == OBD_BRW_READ);
if (count <= 0 ||
- (count & (~CFS_PAGE_MASK)) != 0)
+ (count & (~PAGE_MASK)) != 0)
return -EINVAL;
/* XXX think again with misaligned I/O */
@@ -1409,7 +1352,6 @@
for (i = 0, pgp = pga, off = offset;
i < npages;
i++, pgp++, off += PAGE_SIZE) {
-
LASSERT(!pgp->pg); /* for cleanup */
rc = -ENOMEM;
@@ -1470,7 +1412,7 @@
u64 npages, tot_pages;
int i, ret = 0, brw_flags = 0;
- if (count <= 0 || (count & (~CFS_PAGE_MASK)) != 0)
+ if (count <= 0 || (count & (~PAGE_MASK)) != 0)
return -EINVAL;
npages = batch >> PAGE_SHIFT;
@@ -1886,7 +1828,6 @@
static void /*__exit*/ obdecho_exit(void)
{
echo_client_exit();
-
}
MODULE_AUTHOR("OpenSFS, Inc. <http://www.lustre.org/>");
diff --git a/drivers/staging/lustre/lustre/osc/lproc_osc.c b/drivers/staging/lustre/lustre/osc/lproc_osc.c
index a3358c3..33a1132 100644
--- a/drivers/staging/lustre/lustre/osc/lproc_osc.c
+++ b/drivers/staging/lustre/lustre/osc/lproc_osc.c
@@ -121,9 +121,9 @@
atomic_add(added, &osc_pool_req_count);
}
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
cli->cl_max_rpcs_in_flight = val;
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
return count;
}
@@ -139,9 +139,9 @@
long val;
int mult;
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
val = cli->cl_dirty_max;
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
mult = 1 << 20;
return lprocfs_read_frac_helper(buf, PAGE_SIZE, val, mult);
@@ -169,10 +169,10 @@
pages_number > totalram_pages / 4) /* 1/4 of RAM */
return -ERANGE;
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
cli->cl_dirty_max = (u32)(pages_number << PAGE_SHIFT);
osc_wake_cache_waiters(cli);
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
return count;
}
@@ -222,8 +222,16 @@
return -ERANGE;
rc = atomic_read(&cli->cl_lru_in_list) - pages_number;
- if (rc > 0)
- (void)osc_lru_shrink(cli, rc);
+ if (rc > 0) {
+ struct lu_env *env;
+ int refcheck;
+
+ env = cl_env_get(&refcheck);
+ if (!IS_ERR(env)) {
+ (void)osc_lru_shrink(env, cli, rc, true);
+ cl_env_put(env, &refcheck);
+ }
+ }
return count;
}
@@ -239,9 +247,9 @@
struct client_obd *cli = &dev->u.cli;
int len;
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
len = sprintf(buf, "%lu\n", cli->cl_dirty);
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
return len;
}
@@ -256,9 +264,9 @@
struct client_obd *cli = &dev->u.cli;
int len;
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
len = sprintf(buf, "%lu\n", cli->cl_avail_grant);
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
return len;
}
@@ -279,12 +287,12 @@
return rc;
/* this is only for shrinking grant */
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
if (val >= cli->cl_avail_grant) {
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
return -EINVAL;
}
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
if (cli->cl_import->imp_state == LUSTRE_IMP_FULL)
rc = osc_shrink_grant_to_target(cli, val);
@@ -303,9 +311,9 @@
struct client_obd *cli = &dev->u.cli;
int len;
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
len = sprintf(buf, "%lu\n", cli->cl_lost_grant);
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
return len;
}
@@ -577,14 +585,31 @@
if (val == 0 || val > ocd->ocd_brw_size >> PAGE_SHIFT) {
return -ERANGE;
}
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
cli->cl_max_pages_per_rpc = val;
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
return count;
}
LUSTRE_RW_ATTR(max_pages_per_rpc);
+static ssize_t unstable_stats_show(struct kobject *kobj,
+ struct attribute *attr,
+ char *buf)
+{
+ struct obd_device *dev = container_of(kobj, struct obd_device,
+ obd_kobj);
+ struct client_obd *cli = &dev->u.cli;
+ int pages, mb;
+
+ pages = atomic_read(&cli->cl_unstable_count);
+ mb = (pages * PAGE_SIZE) >> 20;
+
+ return sprintf(buf, "unstable_pages: %8d\n"
+ "unstable_mb: %8d\n", pages, mb);
+}
+LUSTRE_RO_ATTR(unstable_stats);
+
LPROC_SEQ_FOPS_RO_TYPE(osc, connect_flags);
LPROC_SEQ_FOPS_RO_TYPE(osc, server_uuid);
LPROC_SEQ_FOPS_RO_TYPE(osc, conn_uuid);
@@ -623,7 +648,7 @@
ktime_get_real_ts64(&now);
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
seq_printf(seq, "snapshot_time: %llu.%9lu (secs.usecs)\n",
(s64)now.tv_sec, (unsigned long)now.tv_nsec);
@@ -707,7 +732,7 @@
break;
}
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
return 0;
}
@@ -794,6 +819,7 @@
&lustre_attr_max_pages_per_rpc.attr,
&lustre_attr_max_rpcs_in_flight.attr,
&lustre_attr_resend_count.attr,
+ &lustre_attr_unstable_stats.attr,
NULL,
};
diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c
index 5f25bf8..5a14bea 100644
--- a/drivers/staging/lustre/lustre/osc/osc_cache.c
+++ b/drivers/staging/lustre/lustre/osc/osc_cache.c
@@ -76,6 +76,8 @@
*buf++ = ext->oe_rw ? 'r' : 'w';
if (ext->oe_intree)
*buf++ = 'i';
+ if (ext->oe_sync)
+ *buf++ = 'S';
if (ext->oe_srvlock)
*buf++ = 's';
if (ext->oe_hp)
@@ -121,9 +123,13 @@
__ext->oe_grants, __ext->oe_nr_pages, \
list_empty_marker(&__ext->oe_pages), \
waitqueue_active(&__ext->oe_waitq) ? '+' : '-', \
- __ext->oe_osclock, __ext->oe_mppr, __ext->oe_owner, \
+ __ext->oe_dlmlock, __ext->oe_mppr, __ext->oe_owner, \
/* ----- part 4 ----- */ \
## __VA_ARGS__); \
+ if (lvl == D_ERROR && __ext->oe_dlmlock) \
+ LDLM_ERROR(__ext->oe_dlmlock, "extent: %p\n", __ext); \
+ else \
+ LDLM_DEBUG(__ext->oe_dlmlock, "extent: %p\n", __ext); \
} while (0)
#undef EASSERTF
@@ -240,20 +246,25 @@
goto out;
}
- if (!ext->oe_osclock && ext->oe_grants > 0) {
+ if (ext->oe_sync && ext->oe_grants > 0) {
rc = 90;
goto out;
}
- if (ext->oe_osclock) {
- struct cl_lock_descr *descr;
+ if (ext->oe_dlmlock) {
+ struct ldlm_extent *extent;
- descr = &ext->oe_osclock->cll_descr;
- if (!(descr->cld_start <= ext->oe_start &&
- descr->cld_end >= ext->oe_max_end)) {
+ extent = &ext->oe_dlmlock->l_policy_data.l_extent;
+ if (!(extent->start <= cl_offset(osc2cl(obj), ext->oe_start) &&
+ extent->end >= cl_offset(osc2cl(obj), ext->oe_max_end))) {
rc = 100;
goto out;
}
+
+ if (!(ext->oe_dlmlock->l_granted_mode & (LCK_PW | LCK_GROUP))) {
+ rc = 102;
+ goto out;
+ }
}
if (ext->oe_nr_pages > ext->oe_mppr) {
@@ -276,7 +287,7 @@
page_count = 0;
list_for_each_entry(oap, &ext->oe_pages, oap_pending_item) {
- pgoff_t index = oap2cl_page(oap)->cp_index;
+ pgoff_t index = osc_index(oap2osc(oap));
++page_count;
if (index > ext->oe_end || index < ext->oe_start) {
rc = 110;
@@ -359,7 +370,7 @@
ext->oe_state = OES_INV;
INIT_LIST_HEAD(&ext->oe_pages);
init_waitqueue_head(&ext->oe_waitq);
- ext->oe_osclock = NULL;
+ ext->oe_dlmlock = NULL;
return ext;
}
@@ -385,9 +396,11 @@
LASSERT(ext->oe_state == OES_INV);
LASSERT(!ext->oe_intree);
- if (ext->oe_osclock) {
- cl_lock_put(env, ext->oe_osclock);
- ext->oe_osclock = NULL;
+ if (ext->oe_dlmlock) {
+ lu_ref_add(&ext->oe_dlmlock->l_reference,
+ "osc_extent", ext);
+ LDLM_LOCK_PUT(ext->oe_dlmlock);
+ ext->oe_dlmlock = NULL;
}
osc_extent_free(ext);
}
@@ -543,7 +556,7 @@
if (cur->oe_max_end != victim->oe_max_end)
return -ERANGE;
- LASSERT(cur->oe_osclock == victim->oe_osclock);
+ LASSERT(cur->oe_dlmlock == victim->oe_dlmlock);
ppc_bits = osc_cli(obj)->cl_chunkbits - PAGE_SHIFT;
chunk_start = cur->oe_start >> ppc_bits;
chunk_end = cur->oe_end >> ppc_bits;
@@ -624,10 +637,10 @@
static struct osc_extent *osc_extent_find(const struct lu_env *env,
struct osc_object *obj, pgoff_t index,
int *grants)
-
{
struct client_obd *cli = osc_cli(obj);
- struct cl_lock *lock;
+ struct osc_lock *olck;
+ struct cl_lock_descr *descr;
struct osc_extent *cur;
struct osc_extent *ext;
struct osc_extent *conflict = NULL;
@@ -644,8 +657,12 @@
if (!cur)
return ERR_PTR(-ENOMEM);
- lock = cl_lock_at_pgoff(env, osc2cl(obj), index, NULL, 1, 0);
- LASSERT(lock->cll_descr.cld_mode >= CLM_WRITE);
+ olck = osc_env_io(env)->oi_write_osclock;
+ LASSERTF(olck, "page %lu is not covered by lock\n", index);
+ LASSERT(olck->ols_state == OLS_GRANTED);
+
+ descr = &olck->ols_cl.cls_lock->cll_descr;
+ LASSERT(descr->cld_mode >= CLM_WRITE);
LASSERT(cli->cl_chunkbits >= PAGE_SHIFT);
ppc_bits = cli->cl_chunkbits - PAGE_SHIFT;
@@ -657,19 +674,23 @@
max_pages = cli->cl_max_pages_per_rpc;
LASSERT((max_pages & ~chunk_mask) == 0);
max_end = index - (index % max_pages) + max_pages - 1;
- max_end = min_t(pgoff_t, max_end, lock->cll_descr.cld_end);
+ max_end = min_t(pgoff_t, max_end, descr->cld_end);
/* initialize new extent by parameters so far */
cur->oe_max_end = max_end;
cur->oe_start = index & chunk_mask;
cur->oe_end = ((index + ~chunk_mask + 1) & chunk_mask) - 1;
- if (cur->oe_start < lock->cll_descr.cld_start)
- cur->oe_start = lock->cll_descr.cld_start;
+ if (cur->oe_start < descr->cld_start)
+ cur->oe_start = descr->cld_start;
if (cur->oe_end > max_end)
cur->oe_end = max_end;
- cur->oe_osclock = lock;
cur->oe_grants = 0;
cur->oe_mppr = max_pages;
+ if (olck->ols_dlmlock) {
+ LASSERT(olck->ols_hold);
+ cur->oe_dlmlock = LDLM_LOCK_GET(olck->ols_dlmlock);
+ lu_ref_add(&olck->ols_dlmlock->l_reference, "osc_extent", cur);
+ }
/* grants has been allocated by caller */
LASSERTF(*grants >= chunksize + cli->cl_extent_tax,
@@ -691,7 +712,7 @@
break;
/* if covering by different locks, no chance to match */
- if (lock != ext->oe_osclock) {
+ if (olck->ols_dlmlock != ext->oe_dlmlock) {
EASSERTF(!overlapped(ext, cur), ext,
EXTSTR"\n", EXTPARA(cur));
@@ -795,7 +816,7 @@
if (found) {
LASSERT(!conflict);
if (!IS_ERR(found)) {
- LASSERT(found->oe_osclock == cur->oe_osclock);
+ LASSERT(found->oe_dlmlock == cur->oe_dlmlock);
OSC_EXTENT_DUMP(D_CACHE, found,
"found caching ext for %lu.\n", index);
}
@@ -810,7 +831,7 @@
found = osc_extent_hold(cur);
osc_extent_insert(obj, cur);
OSC_EXTENT_DUMP(D_CACHE, cur, "add into tree %lu/%lu.\n",
- index, lock->cll_descr.cld_end);
+ index, descr->cld_end);
}
osc_object_unlock(obj);
@@ -856,6 +877,8 @@
ext->oe_rc = rc ?: ext->oe_nr_pages;
EASSERT(ergo(rc == 0, ext->oe_state == OES_RPC), ext);
+
+ osc_lru_add_batch(cli, &ext->oe_pages);
list_for_each_entry_safe(oap, tmp, &ext->oe_pages, oap_pending_item) {
list_del_init(&oap->oap_rpc_item);
list_del_init(&oap->oap_pending_item);
@@ -877,10 +900,9 @@
* span a whole chunk on the OST side, or our accounting goes
* wrong. Should match the code in filter_grant_check.
*/
- int offset = oap->oap_page_off & ~CFS_PAGE_MASK;
- int count = oap->oap_count + (offset & (blocksize - 1));
- int end = (offset + oap->oap_count) & (blocksize - 1);
-
+ int offset = last_off & ~PAGE_MASK;
+ int count = last_count + (offset & (blocksize - 1));
+ int end = (offset + last_count) & (blocksize - 1);
if (end)
count += blocksize - end;
@@ -943,7 +965,7 @@
"%s: wait ext to %d timedout, recovery in progress?\n",
osc_export(obj)->exp_obd->obd_name, state);
- lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL);
+ lwi = LWI_INTR(NULL, NULL);
rc = l_wait_event(ext->oe_waitq, extent_wait_cb(ext, state),
&lwi);
}
@@ -990,19 +1012,19 @@
/* discard all pages with index greater then trunc_index */
list_for_each_entry_safe(oap, tmp, &ext->oe_pages, oap_pending_item) {
- struct cl_page *sub = oap2cl_page(oap);
- struct cl_page *page = cl_page_top(sub);
+ pgoff_t index = osc_index(oap2osc(oap));
+ struct cl_page *page = oap2cl_page(oap);
LASSERT(list_empty(&oap->oap_rpc_item));
/* only discard the pages with their index greater than
* trunc_index, and ...
*/
- if (sub->cp_index < trunc_index ||
- (sub->cp_index == trunc_index && partial)) {
+ if (index < trunc_index ||
+ (index == trunc_index && partial)) {
/* accounting how many pages remaining in the chunk
* so that we can calculate grants correctly. */
- if (sub->cp_index >> ppc_bits == trunc_chunk)
+ if (index >> ppc_bits == trunc_chunk)
++pages_in_chunk;
continue;
}
@@ -1013,7 +1035,6 @@
lu_ref_add(&page->cp_reference, "truncate", current);
if (cl_page_own(env, io, page) == 0) {
- cl_page_unmap(env, io, page);
cl_page_discard(env, io, page);
cl_page_disown(env, io, page);
} else {
@@ -1126,7 +1147,9 @@
last->oap_count = osc_refresh_count(env, last, OBD_BRW_WRITE);
LASSERT(last->oap_count > 0);
LASSERT(last->oap_page_off + last->oap_count <= PAGE_SIZE);
+ spin_lock(&last->oap_lock);
last->oap_async_flags |= ASYNC_COUNT_STABLE;
+ spin_unlock(&last->oap_lock);
}
/* for the rest of pages, we don't need to call osf_refresh_count()
@@ -1135,7 +1158,9 @@
list_for_each_entry(oap, &ext->oe_pages, oap_pending_item) {
if (!(oap->oap_async_flags & ASYNC_COUNT_STABLE)) {
oap->oap_count = PAGE_SIZE - oap->oap_page_off;
+ spin_lock(&last->oap_lock);
oap->oap_async_flags |= ASYNC_COUNT_STABLE;
+ spin_unlock(&last->oap_lock);
}
}
@@ -1256,7 +1281,7 @@
int cmd)
{
struct osc_page *opg = oap2osc_page(oap);
- struct cl_page *page = cl_page_top(oap2cl_page(oap));
+ struct cl_page *page = oap2cl_page(oap);
int result;
LASSERT(cmd == OBD_BRW_WRITE); /* no cached reads */
@@ -1271,7 +1296,7 @@
struct osc_async_page *oap, int cmd)
{
struct osc_page *opg = oap2osc_page(oap);
- struct cl_page *page = oap2cl_page(oap);
+ pgoff_t index = osc_index(oap2osc(oap));
struct cl_object *obj;
struct cl_attr *attr = &osc_env_info(env)->oti_attr;
@@ -1288,10 +1313,10 @@
if (result < 0)
return result;
kms = attr->cat_kms;
- if (cl_offset(obj, page->cp_index) >= kms)
+ if (cl_offset(obj, index) >= kms)
/* catch race with truncate */
return 0;
- else if (cl_offset(obj, page->cp_index + 1) > kms)
+ else if (cl_offset(obj, index + 1) > kms)
/* catch sub-page write at end of file */
return kms % PAGE_SIZE;
else
@@ -1302,14 +1327,16 @@
int cmd, int rc)
{
struct osc_page *opg = oap2osc_page(oap);
- struct cl_page *page = cl_page_top(oap2cl_page(oap));
+ struct cl_page *page = oap2cl_page(oap);
struct osc_object *obj = cl2osc(opg->ops_cl.cpl_obj);
enum cl_req_type crt;
int srvlock;
cmd &= ~OBD_BRW_NOQUOTA;
- LASSERT(equi(page->cp_state == CPS_PAGEIN, cmd == OBD_BRW_READ));
- LASSERT(equi(page->cp_state == CPS_PAGEOUT, cmd == OBD_BRW_WRITE));
+ LASSERTF(equi(page->cp_state == CPS_PAGEIN, cmd == OBD_BRW_READ),
+ "cp_state:%u, cmd:%d\n", page->cp_state, cmd);
+ LASSERTF(equi(page->cp_state == CPS_PAGEOUT, cmd == OBD_BRW_WRITE),
+ "cp_state:%u, cmd:%d\n", page->cp_state, cmd);
LASSERT(opg->ops_transfer_pinned);
/*
@@ -1358,22 +1385,28 @@
return 0;
}
-#define OSC_DUMP_GRANT(cli, fmt, args...) do { \
+#define OSC_DUMP_GRANT(lvl, cli, fmt, args...) do { \
struct client_obd *__tmp = (cli); \
- CDEBUG(D_CACHE, "%s: { dirty: %ld/%ld dirty_pages: %d/%d " \
- "dropped: %ld avail: %ld, reserved: %ld, flight: %d } " fmt, \
+ CDEBUG(lvl, "%s: grant { dirty: %ld/%ld dirty_pages: %d/%d " \
+ "unstable_pages: %d/%d dropped: %ld avail: %ld, " \
+ "reserved: %ld, flight: %d } lru {in list: %d, " \
+ "left: %d, waiters: %d }" fmt, \
__tmp->cl_import->imp_obd->obd_name, \
__tmp->cl_dirty, __tmp->cl_dirty_max, \
atomic_read(&obd_dirty_pages), obd_max_dirty_pages, \
+ atomic_read(&obd_unstable_pages), obd_max_dirty_pages, \
__tmp->cl_lost_grant, __tmp->cl_avail_grant, \
- __tmp->cl_reserved_grant, __tmp->cl_w_in_flight, ##args); \
+ __tmp->cl_reserved_grant, __tmp->cl_w_in_flight, \
+ atomic_read(&__tmp->cl_lru_in_list), \
+ atomic_read(&__tmp->cl_lru_busy), \
+ atomic_read(&__tmp->cl_lru_shrinkers), ##args); \
} while (0)
/* caller must hold loi_list_lock */
static void osc_consume_write_grant(struct client_obd *cli,
struct brw_page *pga)
{
- assert_spin_locked(&cli->cl_loi_list_lock.lock);
+ assert_spin_locked(&cli->cl_loi_list_lock);
LASSERT(!(pga->flag & OBD_BRW_FROM_GRANT));
atomic_inc(&obd_dirty_pages);
cli->cl_dirty += PAGE_SIZE;
@@ -1389,7 +1422,7 @@
static void osc_release_write_grant(struct client_obd *cli,
struct brw_page *pga)
{
- assert_spin_locked(&cli->cl_loi_list_lock.lock);
+ assert_spin_locked(&cli->cl_loi_list_lock);
if (!(pga->flag & OBD_BRW_FROM_GRANT)) {
return;
}
@@ -1408,7 +1441,7 @@
* To avoid sleeping with object lock held, it's good for us allocate enough
* grants before entering into critical section.
*
- * client_obd_list_lock held by caller
+ * spin_lock held by caller
*/
static int osc_reserve_grant(struct client_obd *cli, unsigned int bytes)
{
@@ -1442,11 +1475,11 @@
static void osc_unreserve_grant(struct client_obd *cli,
unsigned int reserved, unsigned int unused)
{
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
__osc_unreserve_grant(cli, reserved, unused);
if (unused > 0)
osc_wake_cache_waiters(cli);
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
}
/**
@@ -1467,7 +1500,7 @@
{
int grant = (1 << cli->cl_chunkbits) + cli->cl_extent_tax;
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
atomic_sub(nr_pages, &obd_dirty_pages);
cli->cl_dirty -= nr_pages << PAGE_SHIFT;
cli->cl_lost_grant += lost_grant;
@@ -1479,7 +1512,7 @@
cli->cl_avail_grant += grant;
}
osc_wake_cache_waiters(cli);
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
CDEBUG(D_CACHE, "lost %u grant: %lu avail: %lu dirty: %lu\n",
lost_grant, cli->cl_lost_grant,
cli->cl_avail_grant, cli->cl_dirty);
@@ -1491,9 +1524,9 @@
*/
static void osc_exit_cache(struct client_obd *cli, struct osc_async_page *oap)
{
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
osc_release_write_grant(cli, &oap->oap_brw_page);
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
}
/**
@@ -1506,14 +1539,15 @@
{
int rc;
- OSC_DUMP_GRANT(cli, "need:%d.\n", bytes);
+ OSC_DUMP_GRANT(D_CACHE, cli, "need:%d.\n", bytes);
rc = osc_reserve_grant(cli, bytes);
if (rc < 0)
return 0;
if (cli->cl_dirty + PAGE_SIZE <= cli->cl_dirty_max &&
- atomic_read(&obd_dirty_pages) + 1 <= obd_max_dirty_pages) {
+ atomic_read(&obd_unstable_pages) + 1 +
+ atomic_read(&obd_dirty_pages) <= obd_max_dirty_pages) {
osc_consume_write_grant(cli, &oap->oap_brw_page);
if (transient) {
cli->cl_dirty_transit += PAGE_SIZE;
@@ -1532,9 +1566,9 @@
{
int rc;
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
rc = list_empty(&ocw->ocw_entry);
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
return rc;
}
@@ -1551,12 +1585,13 @@
struct osc_object *osc = oap->oap_obj;
struct lov_oinfo *loi = osc->oo_oinfo;
struct osc_cache_waiter ocw;
- struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL);
+ struct l_wait_info lwi = LWI_TIMEOUT_INTR(cfs_time_seconds(600), NULL,
+ LWI_ON_SIGNAL_NOOP, NULL);
int rc = -EDQUOT;
- OSC_DUMP_GRANT(cli, "need:%d.\n", bytes);
+ OSC_DUMP_GRANT(D_CACHE, cli, "need:%d.\n", bytes);
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
/* force the caller to try sync io. this can jump the list
* of queued writes and create a discontiguous rpc stream
@@ -1587,7 +1622,7 @@
while (cli->cl_dirty > 0 || cli->cl_w_in_flight > 0) {
list_add_tail(&ocw.ocw_entry, &cli->cl_cache_waiters);
ocw.ocw_rc = 0;
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
osc_io_unplug_async(env, cli, NULL);
@@ -1596,10 +1631,17 @@
rc = l_wait_event(ocw.ocw_waitq, ocw_granted(cli, &ocw), &lwi);
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
- /* l_wait_event is interrupted by signal */
+ /* l_wait_event is interrupted by signal, or timed out */
if (rc < 0) {
+ if (rc == -ETIMEDOUT) {
+ OSC_DUMP_GRANT(D_ERROR, cli,
+ "try to reserve %d.\n", bytes);
+ osc_extent_tree_dump(D_ERROR, osc);
+ rc = -EDQUOT;
+ }
+
list_del_init(&ocw.ocw_entry);
goto out;
}
@@ -1615,8 +1657,8 @@
}
}
out:
- client_obd_list_unlock(&cli->cl_loi_list_lock);
- OSC_DUMP_GRANT(cli, "returned %d.\n", rc);
+ spin_unlock(&cli->cl_loi_list_lock);
+ OSC_DUMP_GRANT(D_CACHE, cli, "returned %d.\n", rc);
return rc;
}
@@ -1633,8 +1675,8 @@
ocw->ocw_rc = -EDQUOT;
/* we can't dirty more */
if ((cli->cl_dirty + PAGE_SIZE > cli->cl_dirty_max) ||
- (atomic_read(&obd_dirty_pages) + 1 >
- obd_max_dirty_pages)) {
+ (atomic_read(&obd_unstable_pages) + 1 +
+ atomic_read(&obd_dirty_pages) > obd_max_dirty_pages)) {
CDEBUG(D_CACHE, "no dirty room: dirty: %ld osc max %ld, sys max %d\n",
cli->cl_dirty,
cli->cl_dirty_max, obd_max_dirty_pages);
@@ -1776,9 +1818,9 @@
{
int is_ready;
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
is_ready = __osc_list_maint(cli, osc);
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
return is_ready;
}
@@ -1799,13 +1841,101 @@
ar->ar_force_sync = 1;
ar->ar_min_xid = ptlrpc_sample_next_xid();
return;
-
}
if (ar->ar_force_sync && (xid >= ar->ar_min_xid))
ar->ar_force_sync = 0;
}
+/**
+ * Performs "unstable" page accounting. This function balances the
+ * increment operations performed in osc_inc_unstable_pages. It is
+ * registered as the RPC request callback, and is executed when the
+ * bulk RPC is committed on the server. Thus at this point, the pages
+ * involved in the bulk transfer are no longer considered unstable.
+ */
+void osc_dec_unstable_pages(struct ptlrpc_request *req)
+{
+ struct client_obd *cli = &req->rq_import->imp_obd->u.cli;
+ struct ptlrpc_bulk_desc *desc = req->rq_bulk;
+ int page_count = desc->bd_iov_count;
+ int i;
+
+ /* No unstable page tracking */
+ if (!cli->cl_cache)
+ return;
+
+ LASSERT(page_count >= 0);
+
+ for (i = 0; i < page_count; i++)
+ dec_zone_page_state(desc->bd_iov[i].kiov_page, NR_UNSTABLE_NFS);
+
+ atomic_sub(page_count, &cli->cl_cache->ccc_unstable_nr);
+ LASSERT(atomic_read(&cli->cl_cache->ccc_unstable_nr) >= 0);
+
+ atomic_sub(page_count, &cli->cl_unstable_count);
+ LASSERT(atomic_read(&cli->cl_unstable_count) >= 0);
+
+ atomic_sub(page_count, &obd_unstable_pages);
+ LASSERT(atomic_read(&obd_unstable_pages) >= 0);
+
+ spin_lock(&req->rq_lock);
+ req->rq_committed = 1;
+ req->rq_unstable = 0;
+ spin_unlock(&req->rq_lock);
+
+ wake_up_all(&cli->cl_cache->ccc_unstable_waitq);
+}
+
+/* "unstable" page accounting. See: osc_dec_unstable_pages. */
+void osc_inc_unstable_pages(struct ptlrpc_request *req)
+{
+ struct client_obd *cli = &req->rq_import->imp_obd->u.cli;
+ struct ptlrpc_bulk_desc *desc = req->rq_bulk;
+ long page_count = desc->bd_iov_count;
+ int i;
+
+ /* No unstable page tracking */
+ if (!cli->cl_cache)
+ return;
+
+ LASSERT(page_count >= 0);
+
+ for (i = 0; i < page_count; i++)
+ inc_zone_page_state(desc->bd_iov[i].kiov_page, NR_UNSTABLE_NFS);
+
+ LASSERT(atomic_read(&cli->cl_cache->ccc_unstable_nr) >= 0);
+ atomic_add(page_count, &cli->cl_cache->ccc_unstable_nr);
+
+ LASSERT(atomic_read(&cli->cl_unstable_count) >= 0);
+ atomic_add(page_count, &cli->cl_unstable_count);
+
+ LASSERT(atomic_read(&obd_unstable_pages) >= 0);
+ atomic_add(page_count, &obd_unstable_pages);
+
+ spin_lock(&req->rq_lock);
+
+ /*
+ * If the request has already been committed (i.e. brw_commit
+ * called via rq_commit_cb), we need to undo the unstable page
+ * increments we just performed because rq_commit_cb wont be
+ * called again. Otherwise, just set the commit callback so the
+ * unstable page accounting is properly updated when the request
+ * is committed
+ */
+ if (req->rq_committed) {
+ /* Drop lock before calling osc_dec_unstable_pages */
+ spin_unlock(&req->rq_lock);
+ osc_dec_unstable_pages(req);
+ spin_lock(&req->rq_lock);
+ } else {
+ req->rq_unstable = 1;
+ req->rq_commit_cb = osc_dec_unstable_pages;
+ }
+
+ spin_unlock(&req->rq_lock);
+}
+
/* this must be called holding the loi list lock to give coverage to exit_cache,
* async_flag maintenance, and oap_request
*/
@@ -1817,6 +1947,9 @@
__u64 xid = 0;
if (oap->oap_request) {
+ if (!rc)
+ osc_inc_unstable_pages(oap->oap_request);
+
xid = ptlrpc_req_xid(oap->oap_request);
ptlrpc_req_finished(oap->oap_request);
oap->oap_request = NULL;
@@ -1829,10 +1962,10 @@
oap->oap_interrupted = 0;
if (oap->oap_cmd & OBD_BRW_WRITE && xid > 0) {
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
osc_process_ar(&cli->cl_ar, xid, rc);
osc_process_ar(&loi->loi_ar, xid, rc);
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
}
rc = osc_completion(env, oap, oap->oap_cmd, rc);
@@ -2133,9 +2266,8 @@
}
cl_object_get(obj);
- client_obd_list_unlock(&cli->cl_loi_list_lock);
- lu_object_ref_add_at(&obj->co_lu, &link, "check",
- current);
+ spin_unlock(&cli->cl_loi_list_lock);
+ lu_object_ref_add_at(&obj->co_lu, &link, "check", current);
/* attempt some read/write balancing by alternating between
* reads and writes in an object. The makes_rpc checks here
@@ -2178,11 +2310,10 @@
osc_object_unlock(osc);
osc_list_maint(cli, osc);
- lu_object_ref_del_at(&obj->co_lu, &link, "check",
- current);
+ lu_object_ref_del_at(&obj->co_lu, &link, "check", current);
cl_object_put(env, obj);
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
}
}
@@ -2199,9 +2330,9 @@
* potential stack overrun problem. LU-2859
*/
atomic_inc(&cli->cl_lru_shrinkers);
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
osc_check_rpcs(env, cli);
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
atomic_dec(&cli->cl_lru_shrinkers);
} else {
CDEBUG(D_CACHE, "Queue writeback work for client %p.\n", cli);
@@ -2238,7 +2369,7 @@
oap->oap_page = page;
oap->oap_obj_off = offset;
- LASSERT(!(offset & ~CFS_PAGE_MASK));
+ LASSERT(!(offset & ~PAGE_MASK));
if (!client_is_remote(exp) && capable(CFS_CAP_SYS_RESOURCE))
oap->oap_brw_flags = OBD_BRW_NOQUOTA;
@@ -2306,16 +2437,23 @@
return rc;
}
+ if (osc_over_unstable_soft_limit(cli))
+ brw_flags |= OBD_BRW_SOFT_SYNC;
+
oap->oap_cmd = cmd;
oap->oap_page_off = ops->ops_from;
oap->oap_count = ops->ops_to - ops->ops_from;
+ /*
+ * No need to hold a lock here,
+ * since this page is not in any list yet.
+ */
oap->oap_async_flags = 0;
oap->oap_brw_flags = brw_flags;
OSC_IO_DEBUG(osc, "oap %p page %p added for cmd %d\n",
oap, oap->oap_page, oap->oap_cmd & OBD_BRW_RWMASK);
- index = oap2cl_page(oap)->cp_index;
+ index = osc_index(oap2osc(oap));
/* Add this page into extent by the following steps:
* 1. if there exists an active extent for this IO, mostly this page
@@ -2334,9 +2472,9 @@
grants = 0;
/* it doesn't need any grant to dirty this page */
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
rc = osc_enter_cache_try(cli, oap, grants, 0);
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
if (rc == 0) { /* try failed */
grants = 0;
need_release = 1;
@@ -2427,21 +2565,21 @@
LASSERT(oap->oap_magic == OAP_MAGIC);
CDEBUG(D_INFO, "teardown oap %p page %p at index %lu.\n",
- oap, ops, oap2cl_page(oap)->cp_index);
+ oap, ops, osc_index(oap2osc(oap)));
osc_object_lock(obj);
if (!list_empty(&oap->oap_rpc_item)) {
CDEBUG(D_CACHE, "oap %p is not in cache.\n", oap);
rc = -EBUSY;
} else if (!list_empty(&oap->oap_pending_item)) {
- ext = osc_extent_lookup(obj, oap2cl_page(oap)->cp_index);
+ ext = osc_extent_lookup(obj, osc_index(oap2osc(oap)));
/* only truncated pages are allowed to be taken out.
* See osc_extent_truncate() and osc_cache_truncate_start()
* for details.
*/
if (ext && ext->oe_state != OES_TRUNC) {
OSC_EXTENT_DUMP(D_ERROR, ext, "trunc at %lu.\n",
- oap2cl_page(oap)->cp_index);
+ osc_index(oap2osc(oap)));
rc = -EBUSY;
}
}
@@ -2464,7 +2602,7 @@
struct osc_extent *ext = NULL;
struct osc_object *obj = cl2osc(ops->ops_cl.cpl_obj);
struct cl_page *cp = ops->ops_cl.cpl_page;
- pgoff_t index = cp->cp_index;
+ pgoff_t index = osc_index(ops);
struct osc_async_page *oap = &ops->ops_oap;
bool unplug = false;
int rc = 0;
@@ -2479,8 +2617,7 @@
switch (ext->oe_state) {
case OES_RPC:
case OES_LOCK_DONE:
- CL_PAGE_DEBUG(D_ERROR, env, cl_page_top(cp),
- "flush an in-rpc page?\n");
+ CL_PAGE_DEBUG(D_ERROR, env, cp, "flush an in-rpc page?\n");
LASSERT(0);
break;
case OES_LOCKING:
@@ -2506,7 +2643,7 @@
break;
}
- rc = cl_page_prep(env, io, cl_page_top(cp), CRT_WRITE);
+ rc = cl_page_prep(env, io, cp, CRT_WRITE);
if (rc)
goto out;
@@ -2550,7 +2687,7 @@
struct osc_extent *ext;
struct osc_extent *found = NULL;
struct list_head *plist;
- pgoff_t index = oap2cl_page(oap)->cp_index;
+ pgoff_t index = osc_index(ops);
int rc = -EBUSY;
int cmd;
@@ -2613,12 +2750,12 @@
pgoff_t end = 0;
list_for_each_entry(oap, list, oap_pending_item) {
- struct cl_page *cp = oap2cl_page(oap);
+ pgoff_t index = osc_index(oap2osc(oap));
- if (cp->cp_index > end)
- end = cp->cp_index;
- if (cp->cp_index < start)
- start = cp->cp_index;
+ if (index > end)
+ end = index;
+ if (index < start)
+ start = index;
++page_count;
mppr <<= (page_count > mppr);
}
@@ -2633,6 +2770,7 @@
}
ext->oe_rw = !!(cmd & OBD_BRW_READ);
+ ext->oe_sync = 1;
ext->oe_urgent = 1;
ext->oe_start = start;
ext->oe_end = ext->oe_max_end = end;
@@ -2988,7 +3126,200 @@
result = rc;
}
- OSC_IO_DEBUG(obj, "cache page out.\n");
+ OSC_IO_DEBUG(obj, "pageout [%lu, %lu], %d.\n", start, end, result);
+ return result;
+}
+
+/**
+ * Returns a list of pages by a given [start, end] of \a obj.
+ *
+ * \param resched If not NULL, then we give up before hogging CPU for too
+ * long and set *resched = 1, in that case caller should implement a retry
+ * logic.
+ *
+ * Gang tree lookup (radix_tree_gang_lookup()) optimization is absolutely
+ * crucial in the face of [offset, EOF] locks.
+ *
+ * Return at least one page in @queue unless there is no covered page.
+ */
+int osc_page_gang_lookup(const struct lu_env *env, struct cl_io *io,
+ struct osc_object *osc, pgoff_t start, pgoff_t end,
+ osc_page_gang_cbt cb, void *cbdata)
+{
+ struct osc_page *ops;
+ void **pvec;
+ pgoff_t idx;
+ unsigned int nr;
+ unsigned int i;
+ unsigned int j;
+ int res = CLP_GANG_OKAY;
+ bool tree_lock = true;
+
+ idx = start;
+ pvec = osc_env_info(env)->oti_pvec;
+ spin_lock(&osc->oo_tree_lock);
+ while ((nr = radix_tree_gang_lookup(&osc->oo_tree, pvec,
+ idx, OTI_PVEC_SIZE)) > 0) {
+ struct cl_page *page;
+ bool end_of_region = false;
+
+ for (i = 0, j = 0; i < nr; ++i) {
+ ops = pvec[i];
+ pvec[i] = NULL;
+
+ idx = osc_index(ops);
+ if (idx > end) {
+ end_of_region = true;
+ break;
+ }
+
+ page = ops->ops_cl.cpl_page;
+ LASSERT(page->cp_type == CPT_CACHEABLE);
+ if (page->cp_state == CPS_FREEING)
+ continue;
+
+ cl_page_get(page);
+ lu_ref_add_atomic(&page->cp_reference,
+ "gang_lookup", current);
+ pvec[j++] = ops;
+ }
+ ++idx;
+
+ /*
+ * Here a delicate locking dance is performed. Current thread
+ * holds a reference to a page, but has to own it before it
+ * can be placed into queue. Owning implies waiting, so
+ * radix-tree lock is to be released. After a wait one has to
+ * check that pages weren't truncated (cl_page_own() returns
+ * error in the latter case).
+ */
+ spin_unlock(&osc->oo_tree_lock);
+ tree_lock = false;
+
+ for (i = 0; i < j; ++i) {
+ ops = pvec[i];
+ if (res == CLP_GANG_OKAY)
+ res = (*cb)(env, io, ops, cbdata);
+
+ page = ops->ops_cl.cpl_page;
+ lu_ref_del(&page->cp_reference, "gang_lookup", current);
+ cl_page_put(env, page);
+ }
+ if (nr < OTI_PVEC_SIZE || end_of_region)
+ break;
+
+ if (res == CLP_GANG_OKAY && need_resched())
+ res = CLP_GANG_RESCHED;
+ if (res != CLP_GANG_OKAY)
+ break;
+
+ spin_lock(&osc->oo_tree_lock);
+ tree_lock = true;
+ }
+ if (tree_lock)
+ spin_unlock(&osc->oo_tree_lock);
+ return res;
+}
+
+/**
+ * Check if page @page is covered by an extra lock or discard it.
+ */
+static int check_and_discard_cb(const struct lu_env *env, struct cl_io *io,
+ struct osc_page *ops, void *cbdata)
+{
+ struct osc_thread_info *info = osc_env_info(env);
+ struct osc_object *osc = cbdata;
+ pgoff_t index;
+
+ index = osc_index(ops);
+ if (index >= info->oti_fn_index) {
+ struct ldlm_lock *tmp;
+ struct cl_page *page = ops->ops_cl.cpl_page;
+
+ /* refresh non-overlapped index */
+ tmp = osc_dlmlock_at_pgoff(env, osc, index, 0, 0);
+ if (tmp) {
+ __u64 end = tmp->l_policy_data.l_extent.end;
+ /* Cache the first-non-overlapped index so as to skip
+ * all pages within [index, oti_fn_index). This is safe
+ * because if tmp lock is canceled, it will discard
+ * these pages.
+ */
+ info->oti_fn_index = cl_index(osc2cl(osc), end + 1);
+ if (end == OBD_OBJECT_EOF)
+ info->oti_fn_index = CL_PAGE_EOF;
+ LDLM_LOCK_PUT(tmp);
+ } else if (cl_page_own(env, io, page) == 0) {
+ /* discard the page */
+ cl_page_discard(env, io, page);
+ cl_page_disown(env, io, page);
+ } else {
+ LASSERT(page->cp_state == CPS_FREEING);
+ }
+ }
+
+ info->oti_next_index = index + 1;
+ return CLP_GANG_OKAY;
+}
+
+static int discard_cb(const struct lu_env *env, struct cl_io *io,
+ struct osc_page *ops, void *cbdata)
+{
+ struct osc_thread_info *info = osc_env_info(env);
+ struct cl_page *page = ops->ops_cl.cpl_page;
+
+ /* page is top page. */
+ info->oti_next_index = osc_index(ops) + 1;
+ if (cl_page_own(env, io, page) == 0) {
+ KLASSERT(ergo(page->cp_type == CPT_CACHEABLE,
+ !PageDirty(cl_page_vmpage(page))));
+
+ /* discard the page */
+ cl_page_discard(env, io, page);
+ cl_page_disown(env, io, page);
+ } else {
+ LASSERT(page->cp_state == CPS_FREEING);
+ }
+
+ return CLP_GANG_OKAY;
+}
+
+/**
+ * Discard pages protected by the given lock. This function traverses radix
+ * tree to find all covering pages and discard them. If a page is being covered
+ * by other locks, it should remain in cache.
+ *
+ * If error happens on any step, the process continues anyway (the reasoning
+ * behind this being that lock cancellation cannot be delayed indefinitely).
+ */
+int osc_lock_discard_pages(const struct lu_env *env, struct osc_object *osc,
+ pgoff_t start, pgoff_t end, enum cl_lock_mode mode)
+{
+ struct osc_thread_info *info = osc_env_info(env);
+ struct cl_io *io = &info->oti_io;
+ osc_page_gang_cbt cb;
+ int res;
+ int result;
+
+ io->ci_obj = cl_object_top(osc2cl(osc));
+ io->ci_ignore_layout = 1;
+ result = cl_io_init(env, io, CIT_MISC, io->ci_obj);
+ if (result != 0)
+ goto out;
+
+ cb = mode == CLM_READ ? check_and_discard_cb : discard_cb;
+ info->oti_fn_index = info->oti_next_index = start;
+ do {
+ res = osc_page_gang_lookup(env, io, osc,
+ info->oti_next_index, end, cb, osc);
+ if (info->oti_next_index > end)
+ break;
+
+ if (res == CLP_GANG_RESCHED)
+ cond_resched();
+ } while (res != CLP_GANG_OKAY);
+out:
+ cl_io_fini(env, io);
return result;
}
diff --git a/drivers/staging/lustre/lustre/osc/osc_cl_internal.h b/drivers/staging/lustre/lustre/osc/osc_cl_internal.h
index d55d04d..ae19d39 100644
--- a/drivers/staging/lustre/lustre/osc/osc_cl_internal.h
+++ b/drivers/staging/lustre/lustre/osc/osc_cl_internal.h
@@ -51,7 +51,6 @@
#include "../include/obd.h"
/* osc_build_res_name() */
#include "../include/cl_object.h"
-#include "../include/lclient.h"
#include "osc_internal.h"
/** \defgroup osc osc
@@ -68,6 +67,9 @@
struct cl_io_slice oi_cl;
/** true if this io is lockless. */
int oi_lockless;
+ /** how many LRU pages are reserved for this IO */
+ int oi_lru_reserved;
+
/** active extents, we know how many bytes is going to be written,
* so having an active extent will prevent it from being fragmented
*/
@@ -77,6 +79,8 @@
*/
struct osc_extent *oi_trunc;
+ /** write osc_lock for this IO, used by osc_extent_find(). */
+ struct osc_lock *oi_write_osclock;
struct obd_info oi_info;
struct obdo oi_oa;
struct osc_async_cbargs {
@@ -100,7 +104,7 @@
struct osc_io os_io;
};
-#define OTI_PVEC_SIZE 64
+#define OTI_PVEC_SIZE 256
struct osc_thread_info {
struct ldlm_res_id oti_resname;
ldlm_policy_data_t oti_policy;
@@ -109,7 +113,13 @@
struct lustre_handle oti_handle;
struct cl_page_list oti_plist;
struct cl_io oti_io;
- struct cl_page *oti_pvec[OTI_PVEC_SIZE];
+ void *oti_pvec[OTI_PVEC_SIZE];
+ /**
+ * Fields used by cl_lock_discard_pages().
+ */
+ pgoff_t oti_next_index;
+ pgoff_t oti_fn_index; /* first non-overlapped index */
+ struct cl_sync_io oti_anchor;
};
struct osc_object {
@@ -125,7 +135,7 @@
*/
struct list_head oo_inflight[CRT_NR];
/**
- * Lock, protecting ccc_object::cob_inflight, because a seat-belt is
+ * Lock, protecting osc_page::ops_inflight, because a seat-belt is
* locked during take-off and landing.
*/
spinlock_t oo_seatbelt;
@@ -159,6 +169,17 @@
* oo_{read|write}_pages soon.
*/
spinlock_t oo_lock;
+
+ /**
+ * Radix tree for caching pages
+ */
+ struct radix_tree_root oo_tree;
+ spinlock_t oo_tree_lock;
+ unsigned long oo_npages;
+
+ /* Protect osc_lock this osc_object has */
+ spinlock_t oo_ol_spin;
+ struct list_head oo_ol_list;
};
static inline void osc_object_lock(struct osc_object *obj)
@@ -198,8 +219,6 @@
OLS_ENQUEUED,
OLS_UPCALL_RECEIVED,
OLS_GRANTED,
- OLS_RELEASED,
- OLS_BLOCKED,
OLS_CANCELLED
};
@@ -208,10 +227,8 @@
*
* Interaction with DLM.
*
- * CLIO enqueues all DLM locks through ptlrpcd (that is, in "async" mode).
- *
* Once receive upcall is invoked, osc_lock remembers a handle of DLM lock in
- * osc_lock::ols_handle and a pointer to that lock in osc_lock::ols_lock.
+ * osc_lock::ols_handle and a pointer to that lock in osc_lock::ols_dlmlock.
*
* This pointer is protected through a reference, acquired by
* osc_lock_upcall0(). Also, an additional reference is acquired by
@@ -249,26 +266,27 @@
*/
struct osc_lock {
struct cl_lock_slice ols_cl;
+ /** Internal lock to protect states, etc. */
+ spinlock_t ols_lock;
+ /** Owner sleeps on this channel for state change */
+ struct cl_sync_io *ols_owner;
+ /** waiting list for this lock to be cancelled */
+ struct list_head ols_waiting_list;
+ /** wait entry of ols_waiting_list */
+ struct list_head ols_wait_entry;
+ /** list entry for osc_object::oo_ol_list */
+ struct list_head ols_nextlock_oscobj;
+
/** underlying DLM lock */
- struct ldlm_lock *ols_lock;
- /** lock value block */
- struct ost_lvb ols_lvb;
+ struct ldlm_lock *ols_dlmlock;
/** DLM flags with which osc_lock::ols_lock was enqueued */
__u64 ols_flags;
/** osc_lock::ols_lock handle */
struct lustre_handle ols_handle;
struct ldlm_enqueue_info ols_einfo;
enum osc_lock_state ols_state;
-
- /**
- * How many pages are using this lock for io, currently only used by
- * read-ahead. If non-zero, the underlying dlm lock won't be cancelled
- * during recovery to avoid deadlock. see bz16774.
- *
- * \see osc_page::ops_lock
- * \see osc_page_addref_lock(), osc_page_putref_lock()
- */
- atomic_t ols_pageref;
+ /** lock value block */
+ struct ost_lvb ols_lvb;
/**
* true, if ldlm_lock_addref() was called against
@@ -299,16 +317,6 @@
*/
ols_locklessable:1,
/**
- * set by osc_lock_use() to wait until blocking AST enters into
- * osc_ldlm_blocking_ast0(), so that cl_lock mutex can be used for
- * further synchronization.
- */
- ols_ast_wait:1,
- /**
- * If the data of this lock has been flushed to server side.
- */
- ols_flush:1,
- /**
* if set, the osc_lock is a glimpse lock. For glimpse locks, we treat
* the EVAVAIL error as tolerable, this will make upper logic happy
* to wait all glimpse locks to each OSTs to be completed.
@@ -321,15 +329,6 @@
* For async glimpse lock.
*/
ols_agl:1;
- /**
- * IO that owns this lock. This field is used for a dead-lock
- * avoidance by osc_lock_enqueue_wait().
- *
- * XXX: unfortunately, the owner of a osc_lock is not unique,
- * the lock may have multiple users, if the lock is granted and
- * then matched.
- */
- struct osc_io *ols_owner;
};
/**
@@ -369,18 +368,15 @@
* Set if the page must be transferred with OBD_BRW_SRVLOCK.
*/
ops_srvlock:1;
- union {
- /**
- * lru page list. ops_inflight and ops_lru are exclusive so
- * that they can share the same data.
- */
- struct list_head ops_lru;
- /**
- * Linkage into a per-osc_object list of pages in flight. For
- * debugging.
- */
- struct list_head ops_inflight;
- };
+ /**
+ * lru page list. See osc_lru_{del|use}() in osc_page.c for usage.
+ */
+ struct list_head ops_lru;
+ /**
+ * Linkage into a per-osc_object list of pages in flight. For
+ * debugging.
+ */
+ struct list_head ops_inflight;
/**
* Thread that submitted this page for transfer. For debugging.
*/
@@ -389,16 +385,6 @@
* Submit time - the time when the page is starting RPC. For debugging.
*/
unsigned long ops_submit_time;
-
- /**
- * A lock of which we hold a reference covers this page. Only used by
- * read-ahead: for a readahead page, we hold it's covering lock to
- * prevent it from being canceled during recovery.
- *
- * \see osc_lock::ols_pageref
- * \see osc_page_addref_lock(), osc_page_putref_lock().
- */
- struct cl_lock *ops_lock;
};
extern struct kmem_cache *osc_lock_kmem;
@@ -417,21 +403,22 @@
int osc_lock_init(const struct lu_env *env,
struct cl_object *obj, struct cl_lock *lock,
const struct cl_io *io);
-int osc_io_init (const struct lu_env *env,
- struct cl_object *obj, struct cl_io *io);
-int osc_req_init (const struct lu_env *env, struct cl_device *dev,
- struct cl_req *req);
+int osc_io_init(const struct lu_env *env,
+ struct cl_object *obj, struct cl_io *io);
+int osc_req_init(const struct lu_env *env, struct cl_device *dev,
+ struct cl_req *req);
struct lu_object *osc_object_alloc(const struct lu_env *env,
const struct lu_object_header *hdr,
struct lu_device *dev);
int osc_page_init(const struct lu_env *env, struct cl_object *obj,
- struct cl_page *page, struct page *vmpage);
+ struct cl_page *page, pgoff_t ind);
-void osc_index2policy (ldlm_policy_data_t *policy, const struct cl_object *obj,
- pgoff_t start, pgoff_t end);
-int osc_lvb_print (const struct lu_env *env, void *cookie,
- lu_printer_t p, const struct ost_lvb *lvb);
+void osc_index2policy(ldlm_policy_data_t *policy, const struct cl_object *obj,
+ pgoff_t start, pgoff_t end);
+int osc_lvb_print(const struct lu_env *env, void *cookie,
+ lu_printer_t p, const struct ost_lvb *lvb);
+void osc_lru_add_batch(struct client_obd *cli, struct list_head *list);
void osc_page_submit(const struct lu_env *env, struct osc_page *opg,
enum cl_req_type crt, int brw_flags);
int osc_cancel_async_page(const struct lu_env *env, struct osc_page *ops);
@@ -441,6 +428,8 @@
struct page *page, loff_t offset);
int osc_queue_async_io(const struct lu_env *env, struct cl_io *io,
struct osc_page *ops);
+int osc_page_cache_add(const struct lu_env *env,
+ const struct cl_page_slice *slice, struct cl_io *io);
int osc_teardown_async_page(const struct lu_env *env, struct osc_object *obj,
struct osc_page *ops);
int osc_flush_async_page(const struct lu_env *env, struct cl_io *io,
@@ -457,12 +446,13 @@
pgoff_t start, pgoff_t end);
void osc_io_unplug(const struct lu_env *env, struct client_obd *cli,
struct osc_object *osc);
+int lru_queue_work(const struct lu_env *env, void *data);
-void osc_object_set_contended (struct osc_object *obj);
+void osc_object_set_contended(struct osc_object *obj);
void osc_object_clear_contended(struct osc_object *obj);
-int osc_object_is_contended (struct osc_object *obj);
+int osc_object_is_contended(struct osc_object *obj);
-int osc_lock_is_lockless (const struct osc_lock *olck);
+int osc_lock_is_lockless(const struct osc_lock *olck);
/*****************************************************************************
*
@@ -558,6 +548,11 @@
return container_of0(oap, struct osc_page, ops_oap);
}
+static inline pgoff_t osc_index(struct osc_page *opg)
+{
+ return opg->ops_cl.cpl_index;
+}
+
static inline struct cl_page *oap2cl_page(struct osc_async_page *oap)
{
return oap2osc(oap)->ops_cl.cpl_page;
@@ -608,7 +603,7 @@
*
* LOCKING ORDER
* =============
- * page lock -> client_obd_list_lock -> object lock(osc_object::oo_lock)
+ * page lock -> cl_loi_list_lock -> object lock(osc_object::oo_lock)
*/
struct osc_extent {
/** red-black tree node */
@@ -627,6 +622,8 @@
unsigned int oe_intree:1,
/** 0 is write, 1 is read */
oe_rw:1,
+ /** sync extent, queued by osc_queue_sync_pages() */
+ oe_sync:1,
oe_srvlock:1,
oe_memalloc:1,
/** an ACTIVE extent is going to be truncated, so when this extent
@@ -675,7 +672,7 @@
*/
wait_queue_head_t oe_waitq;
/** lock covering this extent */
- struct cl_lock *oe_osclock;
+ struct ldlm_lock *oe_dlmlock;
/** terminator of this extent. Must be true if this extent is in IO. */
struct task_struct *oe_owner;
/** return value of writeback. If somebody is waiting for this extent,
@@ -690,6 +687,14 @@
int sent, int rc);
void osc_extent_release(const struct lu_env *env, struct osc_extent *ext);
+int osc_lock_discard_pages(const struct lu_env *env, struct osc_object *osc,
+ pgoff_t start, pgoff_t end, enum cl_lock_mode mode);
+
+typedef int (*osc_page_gang_cbt)(const struct lu_env *, struct cl_io *,
+ struct osc_page *, void *);
+int osc_page_gang_lookup(const struct lu_env *env, struct cl_io *io,
+ struct osc_object *osc, pgoff_t start, pgoff_t end,
+ osc_page_gang_cbt cb, void *cbdata);
/** @} osc */
#endif /* OSC_CL_INTERNAL_H */
diff --git a/drivers/staging/lustre/lustre/osc/osc_internal.h b/drivers/staging/lustre/lustre/osc/osc_internal.h
index ea695c2..7fad827 100644
--- a/drivers/staging/lustre/lustre/osc/osc_internal.h
+++ b/drivers/staging/lustre/lustre/osc/osc_internal.h
@@ -83,6 +83,12 @@
#define oap_count oap_brw_page.count
#define oap_brw_flags oap_brw_page.flag
+static inline struct osc_async_page *brw_page2oap(struct brw_page *pga)
+{
+ return (struct osc_async_page *)container_of(pga, struct osc_async_page,
+ oap_brw_page);
+}
+
struct osc_cache_waiter {
struct list_head ocw_entry;
wait_queue_head_t ocw_waitq;
@@ -102,12 +108,14 @@
extern struct ptlrpc_request_set *PTLRPCD_SET;
+typedef int (*osc_enqueue_upcall_f)(void *cookie, struct lustre_handle *lockh,
+ int rc);
+
int osc_enqueue_base(struct obd_export *exp, struct ldlm_res_id *res_id,
__u64 *flags, ldlm_policy_data_t *policy,
struct ost_lvb *lvb, int kms_valid,
- obd_enqueue_update_f upcall,
+ osc_enqueue_upcall_f upcall,
void *cookie, struct ldlm_enqueue_info *einfo,
- struct lustre_handle *lockh,
struct ptlrpc_request_set *rqset, int async, int agl);
int osc_cancel_base(struct lustre_handle *lockh, __u32 mode);
@@ -130,9 +138,11 @@
int osc_process_config_base(struct obd_device *obd, struct lustre_cfg *cfg);
int osc_build_rpc(const struct lu_env *env, struct client_obd *cli,
struct list_head *ext_list, int cmd);
-int osc_lru_shrink(struct client_obd *cli, int target);
+int osc_lru_shrink(const struct lu_env *env, struct client_obd *cli,
+ int target, bool force);
+int osc_lru_reclaim(struct client_obd *cli);
-extern spinlock_t osc_ast_guard;
+unsigned long osc_ldlm_weigh_ast(struct ldlm_lock *dlmlock);
int osc_setup(struct obd_device *obd, struct lustre_cfg *lcfg);
@@ -173,8 +183,6 @@
return container_of0(d->obd_lu_dev, struct osc_device, od_cl.cd_lu_dev);
}
-int osc_dlm_lock_pageref(struct ldlm_lock *dlm);
-
extern struct kmem_cache *osc_quota_kmem;
struct osc_quota_info {
/** linkage for quota hash table */
@@ -192,5 +200,12 @@
int osc_quotacheck(struct obd_device *unused, struct obd_export *exp,
struct obd_quotactl *oqctl);
int osc_quota_poll_check(struct obd_export *exp, struct if_quotacheck *qchk);
+void osc_inc_unstable_pages(struct ptlrpc_request *req);
+void osc_dec_unstable_pages(struct ptlrpc_request *req);
+int osc_over_unstable_soft_limit(struct client_obd *cli);
+
+struct ldlm_lock *osc_dlmlock_at_pgoff(const struct lu_env *env,
+ struct osc_object *obj, pgoff_t index,
+ int pending, int canceling);
#endif /* OSC_INTERNAL_H */
diff --git a/drivers/staging/lustre/lustre/osc/osc_io.c b/drivers/staging/lustre/lustre/osc/osc_io.c
index 6bd0a45..d534b0e 100644
--- a/drivers/staging/lustre/lustre/osc/osc_io.c
+++ b/drivers/staging/lustre/lustre/osc/osc_io.c
@@ -68,11 +68,15 @@
return oio;
}
-static struct osc_page *osc_cl_page_osc(struct cl_page *page)
+static struct osc_page *osc_cl_page_osc(struct cl_page *page,
+ struct osc_object *osc)
{
const struct cl_page_slice *slice;
- slice = cl_page_at(page, &osc_device_type);
+ if (osc)
+ slice = cl_object_page_slice(&osc->oo_cl, page);
+ else
+ slice = cl_page_at(page, &osc_device_type);
LASSERT(slice);
return cl2osc_page(slice);
@@ -137,7 +141,7 @@
io = page->cp_owner;
LASSERT(io);
- opg = osc_cl_page_osc(page);
+ opg = osc_cl_page_osc(page, osc);
oap = &opg->ops_oap;
LASSERT(osc == oap->oap_obj);
@@ -164,8 +168,10 @@
}
cl_page_list_move(qout, qin, page);
+ spin_lock(&oap->oap_lock);
oap->oap_async_flags = ASYNC_URGENT|ASYNC_READY;
oap->oap_async_flags |= ASYNC_COUNT_STABLE;
+ spin_unlock(&oap->oap_lock);
osc_page_submit(env, opg, crt, brw_flags);
list_add_tail(&oap->oap_pending_item, &list);
@@ -185,6 +191,13 @@
return qout->pl_nr > 0 ? 0 : result;
}
+/**
+ * This is called when a page is accessed within file in a way that creates
+ * new page, if one were missing (i.e., if there were a hole at that place in
+ * the file, or accessed page is beyond the current file size).
+ *
+ * Expand stripe KMS if necessary.
+ */
static void osc_page_touch_at(const struct lu_env *env,
struct cl_object *obj, pgoff_t idx, unsigned to)
{
@@ -208,7 +221,8 @@
kms > loi->loi_kms ? "" : "not ", loi->loi_kms, kms,
loi->loi_lvb.lvb_size);
- valid = 0;
+ attr->cat_mtime = attr->cat_ctime = LTIME_S(CURRENT_TIME);
+ valid = CAT_MTIME | CAT_CTIME;
if (kms > loi->loi_kms) {
attr->cat_kms = kms;
valid |= CAT_KMS;
@@ -221,93 +235,130 @@
cl_object_attr_unlock(obj);
}
-/**
- * This is called when a page is accessed within file in a way that creates
- * new page, if one were missing (i.e., if there were a hole at that place in
- * the file, or accessed page is beyond the current file size). Examples:
- * ->commit_write() and ->nopage() methods.
- *
- * Expand stripe KMS if necessary.
- */
-static void osc_page_touch(const struct lu_env *env,
- struct osc_page *opage, unsigned to)
+static int osc_io_commit_async(const struct lu_env *env,
+ const struct cl_io_slice *ios,
+ struct cl_page_list *qin, int from, int to,
+ cl_commit_cbt cb)
{
- struct cl_page *page = opage->ops_cl.cpl_page;
- struct cl_object *obj = opage->ops_cl.cpl_obj;
-
- osc_page_touch_at(env, obj, page->cp_index, to);
-}
-
-/**
- * Implements cl_io_operations::cio_prepare_write() method for osc layer.
- *
- * \retval -EIO transfer initiated against this osc will most likely fail
- * \retval 0 transfer initiated against this osc will most likely succeed.
- *
- * The reason for this check is to immediately return an error to the caller
- * in the case of a deactivated import. Note, that import can be deactivated
- * later, while pages, dirtied by this IO, are still in the cache, but this is
- * irrelevant, because that would still return an error to the application (if
- * it does fsync), but many applications don't do fsync because of performance
- * issues, and we wanted to return an -EIO at write time to notify the
- * application.
- */
-static int osc_io_prepare_write(const struct lu_env *env,
- const struct cl_io_slice *ios,
- const struct cl_page_slice *slice,
- unsigned from, unsigned to)
-{
- struct osc_device *dev = lu2osc_dev(slice->cpl_obj->co_lu.lo_dev);
- struct obd_import *imp = class_exp2cliimp(dev->od_exp);
+ struct cl_io *io = ios->cis_io;
struct osc_io *oio = cl2osc_io(env, ios);
+ struct osc_object *osc = cl2osc(ios->cis_obj);
+ struct cl_page *page;
+ struct cl_page *last_page;
+ struct osc_page *opg;
int result = 0;
- /*
- * This implements OBD_BRW_CHECK logic from old client.
- */
+ LASSERT(qin->pl_nr > 0);
- if (!imp || imp->imp_invalid)
- result = -EIO;
- if (result == 0 && oio->oi_lockless)
- /* this page contains `invalid' data, but who cares?
- * nobody can access the invalid data.
- * in osc_io_commit_write(), we're going to write exact
- * [from, to) bytes of this page to OST. -jay
+ /* Handle partial page cases */
+ last_page = cl_page_list_last(qin);
+ if (oio->oi_lockless) {
+ page = cl_page_list_first(qin);
+ if (page == last_page) {
+ cl_page_clip(env, page, from, to);
+ } else {
+ if (from != 0)
+ cl_page_clip(env, page, from, PAGE_SIZE);
+ if (to != PAGE_SIZE)
+ cl_page_clip(env, last_page, 0, to);
+ }
+ }
+
+ while (qin->pl_nr > 0) {
+ struct osc_async_page *oap;
+
+ page = cl_page_list_first(qin);
+ opg = osc_cl_page_osc(page, osc);
+ oap = &opg->ops_oap;
+
+ if (!list_empty(&oap->oap_rpc_item)) {
+ CDEBUG(D_CACHE, "Busy oap %p page %p for submit.\n",
+ oap, opg);
+ result = -EBUSY;
+ break;
+ }
+
+ /* The page may be already in dirty cache. */
+ if (list_empty(&oap->oap_pending_item)) {
+ result = osc_page_cache_add(env, &opg->ops_cl, io);
+ if (result != 0)
+ break;
+ }
+
+ osc_page_touch_at(env, osc2cl(osc), osc_index(opg),
+ page == last_page ? to : PAGE_SIZE);
+
+ cl_page_list_del(env, qin, page);
+
+ (*cb)(env, io, page);
+ /* Can't access page any more. Page can be in transfer and
+ * complete at any time.
*/
- cl_page_export(env, slice->cpl_page, 1);
+ }
+ /* for sync write, kernel will wait for this page to be flushed before
+ * osc_io_end() is called, so release it earlier.
+ * for mkwrite(), it's known there is no further pages.
+ */
+ if (cl_io_is_sync_write(io) && oio->oi_active) {
+ osc_extent_release(env, oio->oi_active);
+ oio->oi_active = NULL;
+ }
+
+ CDEBUG(D_INFO, "%d %d\n", qin->pl_nr, result);
return result;
}
-static int osc_io_commit_write(const struct lu_env *env,
- const struct cl_io_slice *ios,
- const struct cl_page_slice *slice,
- unsigned from, unsigned to)
+static int osc_io_rw_iter_init(const struct lu_env *env,
+ const struct cl_io_slice *ios)
{
- struct osc_io *oio = cl2osc_io(env, ios);
- struct osc_page *opg = cl2osc_page(slice);
- struct osc_object *obj = cl2osc(opg->ops_cl.cpl_obj);
- struct osc_async_page *oap = &opg->ops_oap;
+ struct cl_io *io = ios->cis_io;
+ struct osc_io *oio = osc_env_io(env);
+ struct osc_object *osc = cl2osc(ios->cis_obj);
+ struct client_obd *cli = osc_cli(osc);
+ unsigned long c;
+ unsigned int npages;
+ unsigned int max_pages;
- LASSERT(to > 0);
- /*
- * XXX instead of calling osc_page_touch() here and in
- * osc_io_fault_start() it might be more logical to introduce
- * cl_page_touch() method, that generic cl_io_commit_write() and page
- * fault code calls.
- */
- osc_page_touch(env, cl2osc_page(slice), to);
- if (!client_is_remote(osc_export(obj)) &&
- capable(CFS_CAP_SYS_RESOURCE))
- oap->oap_brw_flags |= OBD_BRW_NOQUOTA;
+ if (cl_io_is_append(io))
+ return 0;
- if (oio->oi_lockless)
- /* see osc_io_prepare_write() for lockless io handling. */
- cl_page_clip(env, slice->cpl_page, from, to);
+ npages = io->u.ci_rw.crw_count >> PAGE_SHIFT;
+ if (io->u.ci_rw.crw_pos & ~PAGE_MASK)
+ ++npages;
+
+ max_pages = cli->cl_max_pages_per_rpc * cli->cl_max_rpcs_in_flight;
+ if (npages > max_pages)
+ npages = max_pages;
+
+ c = atomic_read(cli->cl_lru_left);
+ if (c < npages && osc_lru_reclaim(cli) > 0)
+ c = atomic_read(cli->cl_lru_left);
+ while (c >= npages) {
+ if (c == atomic_cmpxchg(cli->cl_lru_left, c, c - npages)) {
+ oio->oi_lru_reserved = npages;
+ break;
+ }
+ c = atomic_read(cli->cl_lru_left);
+ }
return 0;
}
+static void osc_io_rw_iter_fini(const struct lu_env *env,
+ const struct cl_io_slice *ios)
+{
+ struct osc_io *oio = osc_env_io(env);
+ struct osc_object *osc = cl2osc(ios->cis_obj);
+ struct client_obd *cli = osc_cli(osc);
+
+ if (oio->oi_lru_reserved > 0) {
+ atomic_add(oio->oi_lru_reserved, cli->cl_lru_left);
+ oio->oi_lru_reserved = 0;
+ }
+ oio->oi_write_osclock = NULL;
+}
+
static int osc_io_fault_start(const struct lu_env *env,
const struct cl_io_slice *ios)
{
@@ -342,31 +393,21 @@
* Checks that there are no pages being written in the extent being truncated.
*/
static int trunc_check_cb(const struct lu_env *env, struct cl_io *io,
- struct cl_page *page, void *cbdata)
+ struct osc_page *ops, void *cbdata)
{
- const struct cl_page_slice *slice;
- struct osc_page *ops;
+ struct cl_page *page = ops->ops_cl.cpl_page;
struct osc_async_page *oap;
__u64 start = *(__u64 *)cbdata;
- slice = cl_page_at(page, &osc_device_type);
- LASSERT(slice);
- ops = cl2osc_page(slice);
oap = &ops->ops_oap;
-
if (oap->oap_cmd & OBD_BRW_WRITE &&
!list_empty(&oap->oap_pending_item))
CL_PAGE_DEBUG(D_ERROR, env, page, "exists %llu/%s.\n",
start, current->comm);
- {
- struct page *vmpage = cl_page_vmpage(env, page);
-
- if (PageLocked(vmpage))
- CDEBUG(D_CACHE, "page %p index %lu locked for %d.\n",
- ops, page->cp_index,
- (oap->oap_cmd & OBD_BRW_RWMASK));
- }
+ if (PageLocked(page->cp_vmpage))
+ CDEBUG(D_CACHE, "page %p index %lu locked for %d.\n",
+ ops, osc_index(ops), oap->oap_cmd & OBD_BRW_RWMASK);
return CLP_GANG_OKAY;
}
@@ -385,8 +426,9 @@
/*
* Complain if there are pages in the truncated region.
*/
- cl_page_gang_lookup(env, clob, io, start + partial, CL_PAGE_EOF,
- trunc_check_cb, (void *)&size);
+ osc_page_gang_lookup(env, io, cl2osc(clob),
+ start + partial, CL_PAGE_EOF,
+ trunc_check_cb, (void *)&size);
}
static int osc_io_setattr_start(const struct lu_env *env,
@@ -650,6 +692,8 @@
.cio_fini = osc_io_fini
},
[CIT_WRITE] = {
+ .cio_iter_init = osc_io_rw_iter_init,
+ .cio_iter_fini = osc_io_rw_iter_fini,
.cio_start = osc_io_write_start,
.cio_end = osc_io_end,
.cio_fini = osc_io_fini
@@ -672,16 +716,8 @@
.cio_fini = osc_io_fini
}
},
- .req_op = {
- [CRT_READ] = {
- .cio_submit = osc_io_submit
- },
- [CRT_WRITE] = {
- .cio_submit = osc_io_submit
- }
- },
- .cio_prepare_write = osc_io_prepare_write,
- .cio_commit_write = osc_io_commit_write
+ .cio_submit = osc_io_submit,
+ .cio_commit_async = osc_io_commit_async
};
/*****************************************************************************
@@ -718,8 +754,7 @@
struct lov_oinfo *oinfo;
struct cl_req *clerq;
struct cl_page *apage; /* _some_ page in @clerq */
- struct cl_lock *lock; /* _some_ lock protecting @apage */
- struct osc_lock *olck;
+ struct ldlm_lock *lock; /* _some_ lock protecting @apage */
struct osc_page *opg;
struct obdo *oa;
struct ost_lvb *lvb;
@@ -753,31 +788,32 @@
LASSERT(!list_empty(&clerq->crq_pages));
apage = container_of(clerq->crq_pages.next,
struct cl_page, cp_flight);
- opg = osc_cl_page_osc(apage);
- apage = opg->ops_cl.cpl_page; /* now apage is a sub-page */
- lock = cl_lock_at_page(env, apage->cp_obj, apage, NULL, 1, 1);
- if (!lock) {
- struct cl_object_header *head;
- struct cl_lock *scan;
+ opg = osc_cl_page_osc(apage, NULL);
+ lock = osc_dlmlock_at_pgoff(env, cl2osc(obj), osc_index(opg),
+ 1, 1);
+ if (!lock && !opg->ops_srvlock) {
+ struct ldlm_resource *res;
+ struct ldlm_res_id *resname;
- head = cl_object_header(apage->cp_obj);
- list_for_each_entry(scan, &head->coh_locks, cll_linkage)
- CL_LOCK_DEBUG(D_ERROR, env, scan,
- "no cover page!\n");
- CL_PAGE_DEBUG(D_ERROR, env, apage,
- "dump uncover page!\n");
+ CL_PAGE_DEBUG(D_ERROR, env, apage, "uncovered page!\n");
+
+ resname = &osc_env_info(env)->oti_resname;
+ ostid_build_res_name(&oinfo->loi_oi, resname);
+ res = ldlm_resource_get(
+ osc_export(cl2osc(obj))->exp_obd->obd_namespace,
+ NULL, resname, LDLM_EXTENT, 0);
+ ldlm_resource_dump(D_ERROR, res);
+
dump_stack();
LBUG();
}
- olck = osc_lock_at(lock);
- LASSERT(ergo(opg->ops_srvlock, !olck->ols_lock));
/* check for lockless io. */
- if (olck->ols_lock) {
- oa->o_handle = olck->ols_lock->l_remote_handle;
+ if (lock) {
+ oa->o_handle = lock->l_remote_handle;
oa->o_valid |= OBD_MD_FLHANDLE;
+ LDLM_LOCK_PUT(lock);
}
- cl_lock_put(env, lock);
}
}
@@ -807,8 +843,9 @@
if (or) {
cl_req_slice_add(req, &or->or_cl, dev, &osc_req_ops);
result = 0;
- } else
+ } else {
result = -ENOMEM;
+ }
return result;
}
diff --git a/drivers/staging/lustre/lustre/osc/osc_lock.c b/drivers/staging/lustre/lustre/osc/osc_lock.c
index 013df978..16f9cd9 100644
--- a/drivers/staging/lustre/lustre/osc/osc_lock.c
+++ b/drivers/staging/lustre/lustre/osc/osc_lock.c
@@ -36,6 +36,7 @@
* Implementation of cl_lock for OSC layer.
*
* Author: Nikita Danilov <nikita.danilov@sun.com>
+ * Author: Jinshan Xiong <jinshan.xiong@intel.com>
*/
#define DEBUG_SUBSYSTEM S_OSC
@@ -50,8 +51,6 @@
* @{
*/
-#define _PAGEREF_MAGIC (-10000000)
-
/*****************************************************************************
*
* Type conversions.
@@ -62,7 +61,6 @@
static const struct cl_lock_operations osc_lock_lockless_ops;
static void osc_lock_to_lockless(const struct lu_env *env,
struct osc_lock *ols, int force);
-static int osc_lock_has_pages(struct osc_lock *olck);
int osc_lock_is_lockless(const struct osc_lock *olck)
{
@@ -90,11 +88,11 @@
static int osc_lock_invariant(struct osc_lock *ols)
{
struct ldlm_lock *lock = osc_handle_ptr(&ols->ols_handle);
- struct ldlm_lock *olock = ols->ols_lock;
+ struct ldlm_lock *olock = ols->ols_dlmlock;
int handle_used = lustre_handle_is_used(&ols->ols_handle);
if (ergo(osc_lock_is_lockless(ols),
- ols->ols_locklessable && !ols->ols_lock))
+ ols->ols_locklessable && !ols->ols_dlmlock))
return 1;
/*
@@ -111,7 +109,7 @@
ergo(!lock, !olock)))
return 0;
/*
- * Check that ->ols_handle and ->ols_lock are consistent, but
+ * Check that ->ols_handle and ->ols_dlmlock are consistent, but
* take into account that they are set at the different time.
*/
if (!ergo(ols->ols_state == OLS_CANCELLED,
@@ -122,7 +120,7 @@
* ast.
*/
if (!ergo(olock && ols->ols_state < OLS_CANCELLED,
- ((olock->l_flags & LDLM_FL_DESTROYED) == 0)))
+ !ldlm_is_destroyed(olock)))
return 0;
if (!ergo(ols->ols_state == OLS_GRANTED,
@@ -138,117 +136,13 @@
*
*/
-/**
- * Breaks a link between osc_lock and dlm_lock.
- */
-static void osc_lock_detach(const struct lu_env *env, struct osc_lock *olck)
-{
- struct ldlm_lock *dlmlock;
-
- spin_lock(&osc_ast_guard);
- dlmlock = olck->ols_lock;
- if (!dlmlock) {
- spin_unlock(&osc_ast_guard);
- return;
- }
-
- olck->ols_lock = NULL;
- /* wb(); --- for all who checks (ols->ols_lock != NULL) before
- * call to osc_lock_detach()
- */
- dlmlock->l_ast_data = NULL;
- olck->ols_handle.cookie = 0ULL;
- spin_unlock(&osc_ast_guard);
-
- lock_res_and_lock(dlmlock);
- if (dlmlock->l_granted_mode == dlmlock->l_req_mode) {
- struct cl_object *obj = olck->ols_cl.cls_obj;
- struct cl_attr *attr = &osc_env_info(env)->oti_attr;
- __u64 old_kms;
-
- cl_object_attr_lock(obj);
- /* Must get the value under the lock to avoid possible races. */
- old_kms = cl2osc(obj)->oo_oinfo->loi_kms;
- /* Update the kms. Need to loop all granted locks.
- * Not a problem for the client
- */
- attr->cat_kms = ldlm_extent_shift_kms(dlmlock, old_kms);
-
- cl_object_attr_set(env, obj, attr, CAT_KMS);
- cl_object_attr_unlock(obj);
- }
- unlock_res_and_lock(dlmlock);
-
- /* release a reference taken in osc_lock_upcall0(). */
- LASSERT(olck->ols_has_ref);
- lu_ref_del(&dlmlock->l_reference, "osc_lock", olck);
- LDLM_LOCK_RELEASE(dlmlock);
- olck->ols_has_ref = 0;
-}
-
-static int osc_lock_unhold(struct osc_lock *ols)
-{
- int result = 0;
-
- if (ols->ols_hold) {
- ols->ols_hold = 0;
- result = osc_cancel_base(&ols->ols_handle,
- ols->ols_einfo.ei_mode);
- }
- return result;
-}
-
-static int osc_lock_unuse(const struct lu_env *env,
- const struct cl_lock_slice *slice)
-{
- struct osc_lock *ols = cl2osc_lock(slice);
-
- LINVRNT(osc_lock_invariant(ols));
-
- switch (ols->ols_state) {
- case OLS_NEW:
- LASSERT(!ols->ols_hold);
- LASSERT(ols->ols_agl);
- return 0;
- case OLS_UPCALL_RECEIVED:
- osc_lock_unhold(ols);
- case OLS_ENQUEUED:
- LASSERT(!ols->ols_hold);
- osc_lock_detach(env, ols);
- ols->ols_state = OLS_NEW;
- return 0;
- case OLS_GRANTED:
- LASSERT(!ols->ols_glimpse);
- LASSERT(ols->ols_hold);
- /*
- * Move lock into OLS_RELEASED state before calling
- * osc_cancel_base() so that possible synchronous cancellation
- * sees that lock is released.
- */
- ols->ols_state = OLS_RELEASED;
- return osc_lock_unhold(ols);
- default:
- CERROR("Impossible state: %d\n", ols->ols_state);
- LBUG();
- }
-}
-
static void osc_lock_fini(const struct lu_env *env,
struct cl_lock_slice *slice)
{
struct osc_lock *ols = cl2osc_lock(slice);
LINVRNT(osc_lock_invariant(ols));
- /*
- * ->ols_hold can still be true at this point if, for example, a
- * thread that requested a lock was killed (and released a reference
- * to the lock), before reply from a server was received. In this case
- * lock is destroyed immediately after upcall.
- */
- osc_lock_unhold(ols);
- LASSERT(!ols->ols_lock);
- LASSERT(atomic_read(&ols->ols_pageref) == 0 ||
- atomic_read(&ols->ols_pageref) == _PAGEREF_MAGIC);
+ LASSERT(!ols->ols_dlmlock);
kmem_cache_free(osc_lock_kmem, ols);
}
@@ -275,55 +169,12 @@
result |= LDLM_FL_HAS_INTENT;
if (enqflags & CEF_DISCARD_DATA)
result |= LDLM_FL_AST_DISCARD_DATA;
+ if (enqflags & CEF_PEEK)
+ result |= LDLM_FL_TEST_LOCK;
return result;
}
/**
- * Global spin-lock protecting consistency of ldlm_lock::l_ast_data
- * pointers. Initialized in osc_init().
- */
-spinlock_t osc_ast_guard;
-
-static struct osc_lock *osc_ast_data_get(struct ldlm_lock *dlm_lock)
-{
- struct osc_lock *olck;
-
- lock_res_and_lock(dlm_lock);
- spin_lock(&osc_ast_guard);
- olck = dlm_lock->l_ast_data;
- if (olck) {
- struct cl_lock *lock = olck->ols_cl.cls_lock;
- /*
- * If osc_lock holds a reference on ldlm lock, return it even
- * when cl_lock is in CLS_FREEING state. This way
- *
- * osc_ast_data_get(dlmlock) == NULL
- *
- * guarantees that all osc references on dlmlock were
- * released. osc_dlm_blocking_ast0() relies on that.
- */
- if (lock->cll_state < CLS_FREEING || olck->ols_has_ref) {
- cl_lock_get_trust(lock);
- lu_ref_add_atomic(&lock->cll_reference,
- "ast", current);
- } else
- olck = NULL;
- }
- spin_unlock(&osc_ast_guard);
- unlock_res_and_lock(dlm_lock);
- return olck;
-}
-
-static void osc_ast_data_put(const struct lu_env *env, struct osc_lock *olck)
-{
- struct cl_lock *lock;
-
- lock = olck->ols_cl.cls_lock;
- lu_ref_del(&lock->cll_reference, "ast", current);
- cl_lock_put(env, lock);
-}
-
-/**
* Updates object attributes from a lock value block (lvb) received together
* with the DLM lock reply from the server. Copy of osc_update_enqueue()
* logic.
@@ -333,35 +184,30 @@
*
* Called under lock and resource spin-locks.
*/
-static void osc_lock_lvb_update(const struct lu_env *env, struct osc_lock *olck,
- int rc)
+static void osc_lock_lvb_update(const struct lu_env *env,
+ struct osc_object *osc,
+ struct ldlm_lock *dlmlock,
+ struct ost_lvb *lvb)
{
- struct ost_lvb *lvb;
- struct cl_object *obj;
- struct lov_oinfo *oinfo;
- struct cl_attr *attr;
+ struct cl_object *obj = osc2cl(osc);
+ struct lov_oinfo *oinfo = osc->oo_oinfo;
+ struct cl_attr *attr = &osc_env_info(env)->oti_attr;
unsigned valid;
- if (!(olck->ols_flags & LDLM_FL_LVB_READY))
- return;
-
- lvb = &olck->ols_lvb;
- obj = olck->ols_cl.cls_obj;
- oinfo = cl2osc(obj)->oo_oinfo;
- attr = &osc_env_info(env)->oti_attr;
valid = CAT_BLOCKS | CAT_ATIME | CAT_CTIME | CAT_MTIME | CAT_SIZE;
+ if (!lvb)
+ lvb = dlmlock->l_lvb_data;
+
cl_lvb2attr(attr, lvb);
cl_object_attr_lock(obj);
- if (rc == 0) {
- struct ldlm_lock *dlmlock;
+ if (dlmlock) {
__u64 size;
- dlmlock = olck->ols_lock;
-
- /* re-grab LVB from a dlm lock under DLM spin-locks. */
- *lvb = *(struct ost_lvb *)dlmlock->l_lvb_data;
+ check_res_locked(dlmlock->l_resource);
+ LASSERT(lvb == dlmlock->l_lvb_data);
size = lvb->lvb_size;
+
/* Extend KMS up to the end of this lock and no further
* A lock on [x,y] means a KMS of up to y + 1 bytes!
*/
@@ -378,102 +224,67 @@
dlmlock->l_policy_data.l_extent.end);
}
ldlm_lock_allow_match_locked(dlmlock);
- } else if (rc == -ENAVAIL && olck->ols_glimpse) {
- CDEBUG(D_INODE, "glimpsed, setting rss=%llu; leaving kms=%llu\n",
- lvb->lvb_size, oinfo->loi_kms);
- } else
- valid = 0;
+ }
- if (valid != 0)
- cl_object_attr_set(env, obj, attr, valid);
-
+ cl_object_attr_set(env, obj, attr, valid);
cl_object_attr_unlock(obj);
}
-/**
- * Called when a lock is granted, from an upcall (when server returned a
- * granted lock), or from completion AST, when server returned a blocked lock.
- *
- * Called under lock and resource spin-locks, that are released temporarily
- * here.
- */
-static void osc_lock_granted(const struct lu_env *env, struct osc_lock *olck,
- struct ldlm_lock *dlmlock, int rc)
+static void osc_lock_granted(const struct lu_env *env, struct osc_lock *oscl,
+ struct lustre_handle *lockh, bool lvb_update)
{
- struct ldlm_extent *ext;
- struct cl_lock *lock;
- struct cl_lock_descr *descr;
+ struct ldlm_lock *dlmlock;
- LASSERT(dlmlock->l_granted_mode == dlmlock->l_req_mode);
+ dlmlock = ldlm_handle2lock_long(lockh, 0);
+ LASSERT(dlmlock);
- if (olck->ols_state < OLS_GRANTED) {
- lock = olck->ols_cl.cls_lock;
- ext = &dlmlock->l_policy_data.l_extent;
- descr = &osc_env_info(env)->oti_descr;
- descr->cld_obj = lock->cll_descr.cld_obj;
+ /* lock reference taken by ldlm_handle2lock_long() is
+ * owned by osc_lock and released in osc_lock_detach()
+ */
+ lu_ref_add(&dlmlock->l_reference, "osc_lock", oscl);
+ oscl->ols_has_ref = 1;
- /* XXX check that ->l_granted_mode is valid. */
+ LASSERT(!oscl->ols_dlmlock);
+ oscl->ols_dlmlock = dlmlock;
+
+ /* This may be a matched lock for glimpse request, do not hold
+ * lock reference in that case.
+ */
+ if (!oscl->ols_glimpse) {
+ /* hold a refc for non glimpse lock which will
+ * be released in osc_lock_cancel()
+ */
+ lustre_handle_copy(&oscl->ols_handle, lockh);
+ ldlm_lock_addref(lockh, oscl->ols_einfo.ei_mode);
+ oscl->ols_hold = 1;
+ }
+
+ /* Lock must have been granted. */
+ lock_res_and_lock(dlmlock);
+ if (dlmlock->l_granted_mode == dlmlock->l_req_mode) {
+ struct ldlm_extent *ext = &dlmlock->l_policy_data.l_extent;
+ struct cl_lock_descr *descr = &oscl->ols_cl.cls_lock->cll_descr;
+
+ /* extend the lock extent, otherwise it will have problem when
+ * we decide whether to grant a lockless lock.
+ */
descr->cld_mode = osc_ldlm2cl_lock(dlmlock->l_granted_mode);
descr->cld_start = cl_index(descr->cld_obj, ext->start);
descr->cld_end = cl_index(descr->cld_obj, ext->end);
descr->cld_gid = ext->gid;
- /*
- * tell upper layers the extent of the lock that was actually
- * granted
- */
- olck->ols_state = OLS_GRANTED;
- osc_lock_lvb_update(env, olck, rc);
- /* release DLM spin-locks to allow cl_lock_{modify,signal}()
- * to take a semaphore on a parent lock. This is safe, because
- * spin-locks are needed to protect consistency of
- * dlmlock->l_*_mode and LVB, and we have finished processing
- * them.
- */
- unlock_res_and_lock(dlmlock);
- cl_lock_modify(env, lock, descr);
- cl_lock_signal(env, lock);
- LINVRNT(osc_lock_invariant(olck));
- lock_res_and_lock(dlmlock);
+ /* no lvb update for matched lock */
+ if (lvb_update) {
+ LASSERT(oscl->ols_flags & LDLM_FL_LVB_READY);
+ osc_lock_lvb_update(env, cl2osc(oscl->ols_cl.cls_obj),
+ dlmlock, NULL);
+ }
+ LINVRNT(osc_lock_invariant(oscl));
}
-}
-
-static void osc_lock_upcall0(const struct lu_env *env, struct osc_lock *olck)
-
-{
- struct ldlm_lock *dlmlock;
-
- dlmlock = ldlm_handle2lock_long(&olck->ols_handle, 0);
- LASSERT(dlmlock);
-
- lock_res_and_lock(dlmlock);
- spin_lock(&osc_ast_guard);
- LASSERT(dlmlock->l_ast_data == olck);
- LASSERT(!olck->ols_lock);
- olck->ols_lock = dlmlock;
- spin_unlock(&osc_ast_guard);
-
- /*
- * Lock might be not yet granted. In this case, completion ast
- * (osc_ldlm_completion_ast()) comes later and finishes lock
- * granting.
- */
- if (dlmlock->l_granted_mode == dlmlock->l_req_mode)
- osc_lock_granted(env, olck, dlmlock, 0);
unlock_res_and_lock(dlmlock);
- /*
- * osc_enqueue_interpret() decrefs asynchronous locks, counter
- * this.
- */
- ldlm_lock_addref(&olck->ols_handle, olck->ols_einfo.ei_mode);
- olck->ols_hold = 1;
-
- /* lock reference taken by ldlm_handle2lock_long() is owned by
- * osc_lock and released in osc_lock_detach()
- */
- lu_ref_add(&dlmlock->l_reference, "osc_lock", olck);
- olck->ols_has_ref = 1;
+ LASSERT(oscl->ols_state != OLS_GRANTED);
+ oscl->ols_state = OLS_GRANTED;
}
/**
@@ -481,143 +292,124 @@
* received from a server, or after osc_enqueue_base() matched a local DLM
* lock.
*/
-static int osc_lock_upcall(void *cookie, int errcode)
+static int osc_lock_upcall(void *cookie, struct lustre_handle *lockh,
+ int errcode)
{
- struct osc_lock *olck = cookie;
- struct cl_lock_slice *slice = &olck->ols_cl;
- struct cl_lock *lock = slice->cls_lock;
+ struct osc_lock *oscl = cookie;
+ struct cl_lock_slice *slice = &oscl->ols_cl;
+ struct lu_env *env;
+ struct cl_env_nest nest;
+ int rc;
+
+ env = cl_env_nested_get(&nest);
+ /* should never happen, similar to osc_ldlm_blocking_ast(). */
+ LASSERT(!IS_ERR(env));
+
+ rc = ldlm_error2errno(errcode);
+ if (oscl->ols_state == OLS_ENQUEUED) {
+ oscl->ols_state = OLS_UPCALL_RECEIVED;
+ } else if (oscl->ols_state == OLS_CANCELLED) {
+ rc = -EIO;
+ } else {
+ CERROR("Impossible state: %d\n", oscl->ols_state);
+ LBUG();
+ }
+
+ if (rc == 0)
+ osc_lock_granted(env, oscl, lockh, errcode == ELDLM_OK);
+
+ /* Error handling, some errors are tolerable. */
+ if (oscl->ols_locklessable && rc == -EUSERS) {
+ /* This is a tolerable error, turn this lock into
+ * lockless lock.
+ */
+ osc_object_set_contended(cl2osc(slice->cls_obj));
+ LASSERT(slice->cls_ops == &osc_lock_ops);
+
+ /* Change this lock to ldlmlock-less lock. */
+ osc_lock_to_lockless(env, oscl, 1);
+ oscl->ols_state = OLS_GRANTED;
+ rc = 0;
+ } else if (oscl->ols_glimpse && rc == -ENAVAIL) {
+ LASSERT(oscl->ols_flags & LDLM_FL_LVB_READY);
+ osc_lock_lvb_update(env, cl2osc(slice->cls_obj),
+ NULL, &oscl->ols_lvb);
+ /* Hide the error. */
+ rc = 0;
+ }
+
+ if (oscl->ols_owner)
+ cl_sync_io_note(env, oscl->ols_owner, rc);
+ cl_env_nested_put(&nest, env);
+
+ return rc;
+}
+
+static int osc_lock_upcall_agl(void *cookie, struct lustre_handle *lockh,
+ int errcode)
+{
+ struct osc_object *osc = cookie;
+ struct ldlm_lock *dlmlock;
struct lu_env *env;
struct cl_env_nest nest;
env = cl_env_nested_get(&nest);
- if (!IS_ERR(env)) {
- int rc;
+ LASSERT(!IS_ERR(env));
- cl_lock_mutex_get(env, lock);
-
- LASSERT(lock->cll_state >= CLS_QUEUING);
- if (olck->ols_state == OLS_ENQUEUED) {
- olck->ols_state = OLS_UPCALL_RECEIVED;
- rc = ldlm_error2errno(errcode);
- } else if (olck->ols_state == OLS_CANCELLED) {
- rc = -EIO;
- } else {
- CERROR("Impossible state: %d\n", olck->ols_state);
- LBUG();
- }
- if (rc) {
- struct ldlm_lock *dlmlock;
-
- dlmlock = ldlm_handle2lock(&olck->ols_handle);
- if (dlmlock) {
- lock_res_and_lock(dlmlock);
- spin_lock(&osc_ast_guard);
- LASSERT(!olck->ols_lock);
- dlmlock->l_ast_data = NULL;
- olck->ols_handle.cookie = 0ULL;
- spin_unlock(&osc_ast_guard);
- ldlm_lock_fail_match_locked(dlmlock);
- unlock_res_and_lock(dlmlock);
- LDLM_LOCK_PUT(dlmlock);
- }
- } else {
- if (olck->ols_glimpse)
- olck->ols_glimpse = 0;
- osc_lock_upcall0(env, olck);
- }
-
- /* Error handling, some errors are tolerable. */
- if (olck->ols_locklessable && rc == -EUSERS) {
- /* This is a tolerable error, turn this lock into
- * lockless lock.
- */
- osc_object_set_contended(cl2osc(slice->cls_obj));
- LASSERT(slice->cls_ops == &osc_lock_ops);
-
- /* Change this lock to ldlmlock-less lock. */
- osc_lock_to_lockless(env, olck, 1);
- olck->ols_state = OLS_GRANTED;
- rc = 0;
- } else if (olck->ols_glimpse && rc == -ENAVAIL) {
- osc_lock_lvb_update(env, olck, rc);
- cl_lock_delete(env, lock);
- /* Hide the error. */
- rc = 0;
- }
-
- if (rc == 0) {
- /* For AGL case, the RPC sponsor may exits the cl_lock
- * processing without wait() called before related OSC
- * lock upcall(). So update the lock status according
- * to the enqueue result inside AGL upcall().
- */
- if (olck->ols_agl) {
- lock->cll_flags |= CLF_FROM_UPCALL;
- cl_wait_try(env, lock);
- lock->cll_flags &= ~CLF_FROM_UPCALL;
- if (!olck->ols_glimpse)
- olck->ols_agl = 0;
- }
- cl_lock_signal(env, lock);
- /* del user for lock upcall cookie */
- cl_unuse_try(env, lock);
- } else {
- /* del user for lock upcall cookie */
- cl_lock_user_del(env, lock);
- cl_lock_error(env, lock, rc);
- }
-
- /* release cookie reference, acquired by osc_lock_enqueue() */
- cl_lock_hold_release(env, lock, "upcall", lock);
- cl_lock_mutex_put(env, lock);
-
- lu_ref_del(&lock->cll_reference, "upcall", lock);
- /* This maybe the last reference, so must be called after
- * cl_lock_mutex_put().
- */
- cl_lock_put(env, lock);
-
- cl_env_nested_put(&nest, env);
- } else {
- /* should never happen, similar to osc_ldlm_blocking_ast(). */
- LBUG();
+ if (errcode == ELDLM_LOCK_MATCHED) {
+ errcode = ELDLM_OK;
+ goto out;
}
- return errcode;
+
+ if (errcode != ELDLM_OK)
+ goto out;
+
+ dlmlock = ldlm_handle2lock(lockh);
+ LASSERT(dlmlock);
+
+ lock_res_and_lock(dlmlock);
+ LASSERT(dlmlock->l_granted_mode == dlmlock->l_req_mode);
+
+ /* there is no osc_lock associated with AGL lock */
+ osc_lock_lvb_update(env, osc, dlmlock, NULL);
+
+ unlock_res_and_lock(dlmlock);
+ LDLM_LOCK_PUT(dlmlock);
+
+out:
+ cl_object_put(env, osc2cl(osc));
+ cl_env_nested_put(&nest, env);
+ return ldlm_error2errno(errcode);
}
-/**
- * Core of osc_dlm_blocking_ast() logic.
- */
-static void osc_lock_blocking(const struct lu_env *env,
- struct ldlm_lock *dlmlock,
- struct osc_lock *olck, int blocking)
+static int osc_lock_flush(struct osc_object *obj, pgoff_t start, pgoff_t end,
+ enum cl_lock_mode mode, int discard)
{
- struct cl_lock *lock = olck->ols_cl.cls_lock;
+ struct lu_env *env;
+ struct cl_env_nest nest;
+ int rc = 0;
+ int rc2 = 0;
- LASSERT(olck->ols_lock == dlmlock);
- CLASSERT(OLS_BLOCKED < OLS_CANCELLED);
- LASSERT(!osc_lock_is_lockless(olck));
+ env = cl_env_nested_get(&nest);
+ if (IS_ERR(env))
+ return PTR_ERR(env);
- /*
- * Lock might be still addref-ed here, if e.g., blocking ast
- * is sent for a failed lock.
- */
- osc_lock_unhold(olck);
+ if (mode == CLM_WRITE) {
+ rc = osc_cache_writeback_range(env, obj, start, end, 1,
+ discard);
+ CDEBUG(D_CACHE, "object %p: [%lu -> %lu] %d pages were %s.\n",
+ obj, start, end, rc,
+ discard ? "discarded" : "written back");
+ if (rc > 0)
+ rc = 0;
+ }
- if (blocking && olck->ols_state < OLS_BLOCKED)
- /*
- * Move osc_lock into OLS_BLOCKED before canceling the lock,
- * because it recursively re-enters osc_lock_blocking(), with
- * the state set to OLS_CANCELLED.
- */
- olck->ols_state = OLS_BLOCKED;
- /*
- * cancel and destroy lock at least once no matter how blocking ast is
- * entered (see comment above osc_ldlm_blocking_ast() for use
- * cases). cl_lock_cancel() and cl_lock_delete() are idempotent.
- */
- cl_lock_cancel(env, lock);
- cl_lock_delete(env, lock);
+ rc2 = osc_lock_discard_pages(env, obj, start, end, mode);
+ if (rc == 0 && rc2 < 0)
+ rc = rc2;
+
+ cl_env_nested_put(&nest, env);
+ return rc;
}
/**
@@ -628,65 +420,63 @@
struct ldlm_lock *dlmlock,
void *data, int flag)
{
- struct osc_lock *olck;
- struct cl_lock *lock;
- int result;
- int cancel;
+ struct cl_object *obj = NULL;
+ int result = 0;
+ int discard;
+ enum cl_lock_mode mode = CLM_READ;
- LASSERT(flag == LDLM_CB_BLOCKING || flag == LDLM_CB_CANCELING);
+ LASSERT(flag == LDLM_CB_CANCELING);
- cancel = 0;
- olck = osc_ast_data_get(dlmlock);
- if (olck) {
- lock = olck->ols_cl.cls_lock;
- cl_lock_mutex_get(env, lock);
- LINVRNT(osc_lock_invariant(olck));
- if (olck->ols_ast_wait) {
- /* wake up osc_lock_use() */
- cl_lock_signal(env, lock);
- olck->ols_ast_wait = 0;
- }
- /*
- * Lock might have been canceled while this thread was
- * sleeping for lock mutex, but olck is pinned in memory.
+ lock_res_and_lock(dlmlock);
+ if (dlmlock->l_granted_mode != dlmlock->l_req_mode) {
+ dlmlock->l_ast_data = NULL;
+ unlock_res_and_lock(dlmlock);
+ return 0;
+ }
+
+ discard = ldlm_is_discard_data(dlmlock);
+ if (dlmlock->l_granted_mode & (LCK_PW | LCK_GROUP))
+ mode = CLM_WRITE;
+
+ if (dlmlock->l_ast_data) {
+ obj = osc2cl(dlmlock->l_ast_data);
+ dlmlock->l_ast_data = NULL;
+
+ cl_object_get(obj);
+ }
+
+ unlock_res_and_lock(dlmlock);
+
+ /* if l_ast_data is NULL, the dlmlock was enqueued by AGL or
+ * the object has been destroyed.
+ */
+ if (obj) {
+ struct ldlm_extent *extent = &dlmlock->l_policy_data.l_extent;
+ struct cl_attr *attr = &osc_env_info(env)->oti_attr;
+ __u64 old_kms;
+
+ /* Destroy pages covered by the extent of the DLM lock */
+ result = osc_lock_flush(cl2osc(obj),
+ cl_index(obj, extent->start),
+ cl_index(obj, extent->end),
+ mode, discard);
+
+ /* losing a lock, update kms */
+ lock_res_and_lock(dlmlock);
+ cl_object_attr_lock(obj);
+ /* Must get the value under the lock to avoid race. */
+ old_kms = cl2osc(obj)->oo_oinfo->loi_kms;
+ /* Update the kms. Need to loop all granted locks.
+ * Not a problem for the client
*/
- if (olck == dlmlock->l_ast_data) {
- /*
- * NOTE: DLM sends blocking AST's for failed locks
- * (that are still in pre-OLS_GRANTED state)
- * too, and they have to be canceled otherwise
- * DLM lock is never destroyed and stuck in
- * the memory.
- *
- * Alternatively, ldlm_cli_cancel() can be
- * called here directly for osc_locks with
- * ols_state < OLS_GRANTED to maintain an
- * invariant that ->clo_cancel() is only called
- * for locks that were granted.
- */
- LASSERT(data == olck);
- osc_lock_blocking(env, dlmlock,
- olck, flag == LDLM_CB_BLOCKING);
- } else
- cancel = 1;
- cl_lock_mutex_put(env, lock);
- osc_ast_data_put(env, olck);
- } else
- /*
- * DLM lock exists, but there is no cl_lock attached to it.
- * This is a `normal' race. cl_object and its cl_lock's can be
- * removed by memory pressure, together with all pages.
- */
- cancel = (flag == LDLM_CB_BLOCKING);
+ attr->cat_kms = ldlm_extent_shift_kms(dlmlock, old_kms);
- if (cancel) {
- struct lustre_handle *lockh;
+ cl_object_attr_set(env, obj, attr, CAT_KMS);
+ cl_object_attr_unlock(obj);
+ unlock_res_and_lock(dlmlock);
- lockh = &osc_env_info(env)->oti_handle;
- ldlm_lock2handle(dlmlock, lockh);
- result = ldlm_cli_cancel(lockh, LCF_ASYNC);
- } else
- result = 0;
+ cl_object_put(env, obj);
+ }
return result;
}
@@ -736,107 +526,52 @@
struct ldlm_lock_desc *new, void *data,
int flag)
{
- struct lu_env *env;
- struct cl_env_nest nest;
- int result;
+ int result = 0;
- /*
- * This can be called in the context of outer IO, e.g.,
- *
- * cl_enqueue()->...
- * ->osc_enqueue_base()->...
- * ->ldlm_prep_elc_req()->...
- * ->ldlm_cancel_callback()->...
- * ->osc_ldlm_blocking_ast()
- *
- * new environment has to be created to not corrupt outer context.
- */
- env = cl_env_nested_get(&nest);
- if (!IS_ERR(env)) {
- result = osc_dlm_blocking_ast0(env, dlmlock, data, flag);
- cl_env_nested_put(&nest, env);
- } else {
- result = PTR_ERR(env);
- /*
- * XXX This should never happen, as cl_lock is
- * stuck. Pre-allocated environment a la vvp_inode_fini_env
- * should be used.
- */
- LBUG();
- }
- if (result != 0) {
+ switch (flag) {
+ case LDLM_CB_BLOCKING: {
+ struct lustre_handle lockh;
+
+ ldlm_lock2handle(dlmlock, &lockh);
+ result = ldlm_cli_cancel(&lockh, LCF_ASYNC);
if (result == -ENODATA)
result = 0;
- else
- CERROR("BAST failed: %d\n", result);
+ break;
+ }
+ case LDLM_CB_CANCELING: {
+ struct lu_env *env;
+ struct cl_env_nest nest;
+
+ /*
+ * This can be called in the context of outer IO, e.g.,
+ *
+ * osc_enqueue_base()->...
+ * ->ldlm_prep_elc_req()->...
+ * ->ldlm_cancel_callback()->...
+ * ->osc_ldlm_blocking_ast()
+ *
+ * new environment has to be created to not corrupt outer
+ * context.
+ */
+ env = cl_env_nested_get(&nest);
+ if (IS_ERR(env)) {
+ result = PTR_ERR(env);
+ break;
+ }
+
+ result = osc_dlm_blocking_ast0(env, dlmlock, data, flag);
+ cl_env_nested_put(&nest, env);
+ break;
+ }
+ default:
+ LBUG();
}
return result;
}
-static int osc_ldlm_completion_ast(struct ldlm_lock *dlmlock,
- __u64 flags, void *data)
-{
- struct cl_env_nest nest;
- struct lu_env *env;
- struct osc_lock *olck;
- struct cl_lock *lock;
- int result;
- int dlmrc;
-
- /* first, do dlm part of the work */
- dlmrc = ldlm_completion_ast_async(dlmlock, flags, data);
- /* then, notify cl_lock */
- env = cl_env_nested_get(&nest);
- if (!IS_ERR(env)) {
- olck = osc_ast_data_get(dlmlock);
- if (olck) {
- lock = olck->ols_cl.cls_lock;
- cl_lock_mutex_get(env, lock);
- /*
- * ldlm_handle_cp_callback() copied LVB from request
- * to lock->l_lvb_data, store it in osc_lock.
- */
- LASSERT(dlmlock->l_lvb_data);
- lock_res_and_lock(dlmlock);
- olck->ols_lvb = *(struct ost_lvb *)dlmlock->l_lvb_data;
- if (!olck->ols_lock) {
- /*
- * upcall (osc_lock_upcall()) hasn't yet been
- * called. Do nothing now, upcall will bind
- * olck to dlmlock and signal the waiters.
- *
- * This maintains an invariant that osc_lock
- * and ldlm_lock are always bound when
- * osc_lock is in OLS_GRANTED state.
- */
- } else if (dlmlock->l_granted_mode ==
- dlmlock->l_req_mode) {
- osc_lock_granted(env, olck, dlmlock, dlmrc);
- }
- unlock_res_and_lock(dlmlock);
-
- if (dlmrc != 0) {
- CL_LOCK_DEBUG(D_ERROR, env, lock,
- "dlmlock returned %d\n", dlmrc);
- cl_lock_error(env, lock, dlmrc);
- }
- cl_lock_mutex_put(env, lock);
- osc_ast_data_put(env, olck);
- result = 0;
- } else
- result = -ELDLM_NO_LOCK_DATA;
- cl_env_nested_put(&nest, env);
- } else
- result = PTR_ERR(env);
- return dlmrc ?: result;
-}
-
static int osc_ldlm_glimpse_ast(struct ldlm_lock *dlmlock, void *data)
{
struct ptlrpc_request *req = data;
- struct osc_lock *olck;
- struct cl_lock *lock;
- struct cl_object *obj;
struct cl_env_nest nest;
struct lu_env *env;
struct ost_lvb *lvb;
@@ -847,14 +582,16 @@
env = cl_env_nested_get(&nest);
if (!IS_ERR(env)) {
- /* osc_ast_data_get() has to go after environment is
- * allocated, because osc_ast_data() acquires a
- * reference to a lock, and it can only be released in
- * environment.
- */
- olck = osc_ast_data_get(dlmlock);
- if (olck) {
- lock = olck->ols_cl.cls_lock;
+ struct cl_object *obj = NULL;
+
+ lock_res_and_lock(dlmlock);
+ if (dlmlock->l_ast_data) {
+ obj = osc2cl(dlmlock->l_ast_data);
+ cl_object_get(obj);
+ }
+ unlock_res_and_lock(dlmlock);
+
+ if (obj) {
/* Do not grab the mutex of cl_lock for glimpse.
* See LU-1274 for details.
* BTW, it's okay for cl_lock to be cancelled during
@@ -869,7 +606,6 @@
result = req_capsule_server_pack(cap);
if (result == 0) {
lvb = req_capsule_server_get(cap, &RMF_DLM_LVB);
- obj = lock->cll_descr.cld_obj;
result = cl_object_glimpse(env, obj, lvb);
}
if (!exp_connect_lvb_type(req->rq_export))
@@ -877,7 +613,7 @@
&RMF_DLM_LVB,
sizeof(struct ost_lvb_v1),
RCL_SERVER);
- osc_ast_data_put(env, olck);
+ cl_object_put(env, obj);
} else {
/*
* These errors are normal races, so we don't want to
@@ -888,44 +624,123 @@
result = -ELDLM_NO_LOCK_DATA;
}
cl_env_nested_put(&nest, env);
- } else
+ } else {
result = PTR_ERR(env);
+ }
req->rq_status = result;
return result;
}
-static unsigned long osc_lock_weigh(const struct lu_env *env,
- const struct cl_lock_slice *slice)
+static int weigh_cb(const struct lu_env *env, struct cl_io *io,
+ struct osc_page *ops, void *cbdata)
{
+ struct cl_page *page = ops->ops_cl.cpl_page;
+
+ if (cl_page_is_vmlocked(env, page) ||
+ PageDirty(page->cp_vmpage) || PageWriteback(page->cp_vmpage)
+ ) {
+ (*(unsigned long *)cbdata)++;
+ return CLP_GANG_ABORT;
+ }
+
+ return CLP_GANG_OKAY;
+}
+
+static unsigned long osc_lock_weight(const struct lu_env *env,
+ struct osc_object *oscobj,
+ struct ldlm_extent *extent)
+{
+ struct cl_io *io = &osc_env_info(env)->oti_io;
+ struct cl_object *obj = cl_object_top(&oscobj->oo_cl);
+ unsigned long npages = 0;
+ int result;
+
+ io->ci_obj = obj;
+ io->ci_ignore_layout = 1;
+ result = cl_io_init(env, io, CIT_MISC, io->ci_obj);
+ if (result != 0)
+ return result;
+
+ do {
+ result = osc_page_gang_lookup(env, io, oscobj,
+ cl_index(obj, extent->start),
+ cl_index(obj, extent->end),
+ weigh_cb, (void *)&npages);
+ if (result == CLP_GANG_ABORT)
+ break;
+ if (result == CLP_GANG_RESCHED)
+ cond_resched();
+ } while (result != CLP_GANG_OKAY);
+ cl_io_fini(env, io);
+
+ return npages;
+}
+
+/**
+ * Get the weight of dlm lock for early cancellation.
+ */
+unsigned long osc_ldlm_weigh_ast(struct ldlm_lock *dlmlock)
+{
+ struct cl_env_nest nest;
+ struct lu_env *env;
+ struct osc_object *obj;
+ struct osc_lock *oscl;
+ unsigned long weight;
+ bool found = false;
+
+ might_sleep();
/*
- * don't need to grab coh_page_guard since we don't care the exact #
- * of pages..
+ * osc_ldlm_weigh_ast has a complex context since it might be called
+ * because of lock canceling, or from user's input. We have to make
+ * a new environment for it. Probably it is implementation safe to use
+ * the upper context because cl_lock_put don't modify environment
+ * variables. But just in case ..
*/
- return cl_object_header(slice->cls_obj)->coh_pages;
+ env = cl_env_nested_get(&nest);
+ if (IS_ERR(env))
+ /* Mostly because lack of memory, do not eliminate this lock */
+ return 1;
+
+ LASSERT(dlmlock->l_resource->lr_type == LDLM_EXTENT);
+ obj = dlmlock->l_ast_data;
+ if (obj) {
+ weight = 1;
+ goto out;
+ }
+
+ spin_lock(&obj->oo_ol_spin);
+ list_for_each_entry(oscl, &obj->oo_ol_list, ols_nextlock_oscobj) {
+ if (oscl->ols_dlmlock && oscl->ols_dlmlock != dlmlock)
+ continue;
+ found = true;
+ }
+ spin_unlock(&obj->oo_ol_spin);
+ if (found) {
+ /*
+ * If the lock is being used by an IO, definitely not cancel it.
+ */
+ weight = 1;
+ goto out;
+ }
+
+ weight = osc_lock_weight(env, obj, &dlmlock->l_policy_data.l_extent);
+
+out:
+ cl_env_nested_put(&nest, env);
+ return weight;
}
static void osc_lock_build_einfo(const struct lu_env *env,
- const struct cl_lock *clock,
- struct osc_lock *lock,
+ const struct cl_lock *lock,
+ struct osc_object *osc,
struct ldlm_enqueue_info *einfo)
{
- enum cl_lock_mode mode;
-
- mode = clock->cll_descr.cld_mode;
- if (mode == CLM_PHANTOM)
- /*
- * For now, enqueue all glimpse locks in read mode. In the
- * future, client might choose to enqueue LCK_PW lock for
- * glimpse on a file opened for write.
- */
- mode = CLM_READ;
-
einfo->ei_type = LDLM_EXTENT;
- einfo->ei_mode = osc_cl_lock2ldlm(mode);
+ einfo->ei_mode = osc_cl_lock2ldlm(lock->cll_descr.cld_mode);
einfo->ei_cb_bl = osc_ldlm_blocking_ast;
- einfo->ei_cb_cp = osc_ldlm_completion_ast;
+ einfo->ei_cb_cp = ldlm_completion_ast;
einfo->ei_cb_gl = osc_ldlm_glimpse_ast;
- einfo->ei_cbdata = lock; /* value to be put into ->l_ast_data */
+ einfo->ei_cbdata = osc; /* value to be put into ->l_ast_data */
}
/**
@@ -981,113 +796,100 @@
LASSERT(ergo(ols->ols_glimpse, !osc_lock_is_lockless(ols)));
}
-static int osc_lock_compatible(const struct osc_lock *qing,
- const struct osc_lock *qed)
+static bool osc_lock_compatible(const struct osc_lock *qing,
+ const struct osc_lock *qed)
{
- enum cl_lock_mode qing_mode;
- enum cl_lock_mode qed_mode;
+ struct cl_lock_descr *qed_descr = &qed->ols_cl.cls_lock->cll_descr;
+ struct cl_lock_descr *qing_descr = &qing->ols_cl.cls_lock->cll_descr;
- qing_mode = qing->ols_cl.cls_lock->cll_descr.cld_mode;
- if (qed->ols_glimpse &&
- (qed->ols_state >= OLS_UPCALL_RECEIVED || qing_mode == CLM_READ))
- return 1;
+ if (qed->ols_glimpse)
+ return true;
- qed_mode = qed->ols_cl.cls_lock->cll_descr.cld_mode;
- return ((qing_mode == CLM_READ) && (qed_mode == CLM_READ));
+ if (qing_descr->cld_mode == CLM_READ && qed_descr->cld_mode == CLM_READ)
+ return true;
+
+ if (qed->ols_state < OLS_GRANTED)
+ return true;
+
+ if (qed_descr->cld_mode >= qing_descr->cld_mode &&
+ qed_descr->cld_start <= qing_descr->cld_start &&
+ qed_descr->cld_end >= qing_descr->cld_end)
+ return true;
+
+ return false;
}
-/**
- * Cancel all conflicting locks and wait for them to be destroyed.
- *
- * This function is used for two purposes:
- *
- * - early cancel all conflicting locks before starting IO, and
- *
- * - guarantee that pages added to the page cache by lockless IO are never
- * covered by locks other than lockless IO lock, and, hence, are not
- * visible to other threads.
- */
-static int osc_lock_enqueue_wait(const struct lu_env *env,
- const struct osc_lock *olck)
+static void osc_lock_wake_waiters(const struct lu_env *env,
+ struct osc_object *osc,
+ struct osc_lock *oscl)
{
- struct cl_lock *lock = olck->ols_cl.cls_lock;
- struct cl_lock_descr *descr = &lock->cll_descr;
- struct cl_object_header *hdr = cl_object_header(descr->cld_obj);
- struct cl_lock *scan;
- struct cl_lock *conflict = NULL;
- int lockless = osc_lock_is_lockless(olck);
- int rc = 0;
+ spin_lock(&osc->oo_ol_spin);
+ list_del_init(&oscl->ols_nextlock_oscobj);
+ spin_unlock(&osc->oo_ol_spin);
- LASSERT(cl_lock_is_mutexed(lock));
+ spin_lock(&oscl->ols_lock);
+ while (!list_empty(&oscl->ols_waiting_list)) {
+ struct osc_lock *scan;
- /* make it enqueue anyway for glimpse lock, because we actually
- * don't need to cancel any conflicting locks.
- */
- if (olck->ols_glimpse)
- return 0;
+ scan = list_entry(oscl->ols_waiting_list.next, struct osc_lock,
+ ols_wait_entry);
+ list_del_init(&scan->ols_wait_entry);
- spin_lock(&hdr->coh_lock_guard);
- list_for_each_entry(scan, &hdr->coh_locks, cll_linkage) {
- struct cl_lock_descr *cld = &scan->cll_descr;
- const struct osc_lock *scan_ols;
+ cl_sync_io_note(env, scan->ols_owner, 0);
+ }
+ spin_unlock(&oscl->ols_lock);
+}
- if (scan == lock)
+static void osc_lock_enqueue_wait(const struct lu_env *env,
+ struct osc_object *obj,
+ struct osc_lock *oscl)
+{
+ struct osc_lock *tmp_oscl;
+ struct cl_lock_descr *need = &oscl->ols_cl.cls_lock->cll_descr;
+ struct cl_sync_io *waiter = &osc_env_info(env)->oti_anchor;
+
+ spin_lock(&obj->oo_ol_spin);
+ list_add_tail(&oscl->ols_nextlock_oscobj, &obj->oo_ol_list);
+
+restart:
+ list_for_each_entry(tmp_oscl, &obj->oo_ol_list,
+ ols_nextlock_oscobj) {
+ struct cl_lock_descr *descr;
+
+ if (tmp_oscl == oscl)
break;
- if (scan->cll_state < CLS_QUEUING ||
- scan->cll_state == CLS_FREEING ||
- cld->cld_start > descr->cld_end ||
- cld->cld_end < descr->cld_start)
+ descr = &tmp_oscl->ols_cl.cls_lock->cll_descr;
+ if (descr->cld_start > need->cld_end ||
+ descr->cld_end < need->cld_start)
continue;
- /* overlapped and living locks. */
+ /* We're not supposed to give up group lock */
+ if (descr->cld_mode == CLM_GROUP)
+ break;
- /* We're not supposed to give up group lock. */
- if (scan->cll_descr.cld_mode == CLM_GROUP) {
- LASSERT(descr->cld_mode != CLM_GROUP ||
- descr->cld_gid != scan->cll_descr.cld_gid);
- continue;
- }
-
- scan_ols = osc_lock_at(scan);
-
- /* We need to cancel the compatible locks if we're enqueuing
- * a lockless lock, for example:
- * imagine that client has PR lock on [0, 1000], and thread T0
- * is doing lockless IO in [500, 1500] region. Concurrent
- * thread T1 can see lockless data in [500, 1000], which is
- * wrong, because these data are possibly stale.
- */
- if (!lockless && osc_lock_compatible(olck, scan_ols))
+ if (!osc_lock_is_lockless(oscl) &&
+ osc_lock_compatible(oscl, tmp_oscl))
continue;
- cl_lock_get_trust(scan);
- conflict = scan;
- break;
+ /* wait for conflicting lock to be canceled */
+ cl_sync_io_init(waiter, 1, cl_sync_io_end);
+ oscl->ols_owner = waiter;
+
+ spin_lock(&tmp_oscl->ols_lock);
+ /* add oscl into tmp's ols_waiting list */
+ list_add_tail(&oscl->ols_wait_entry,
+ &tmp_oscl->ols_waiting_list);
+ spin_unlock(&tmp_oscl->ols_lock);
+
+ spin_unlock(&obj->oo_ol_spin);
+ (void)cl_sync_io_wait(env, waiter, 0);
+
+ spin_lock(&obj->oo_ol_spin);
+ oscl->ols_owner = NULL;
+ goto restart;
}
- spin_unlock(&hdr->coh_lock_guard);
-
- if (conflict) {
- if (lock->cll_descr.cld_mode == CLM_GROUP) {
- /* we want a group lock but a previous lock request
- * conflicts, we do not wait but return 0 so the
- * request is send to the server
- */
- CDEBUG(D_DLMTRACE, "group lock %p is conflicted with %p, no wait, send to server\n",
- lock, conflict);
- cl_lock_put(env, conflict);
- rc = 0;
- } else {
- CDEBUG(D_DLMTRACE, "lock %p is conflicted with %p, will wait\n",
- lock, conflict);
- LASSERT(!lock->cll_conflict);
- lu_ref_add(&conflict->cll_reference, "cancel-wait",
- lock);
- lock->cll_conflict = conflict;
- rc = CLO_WAIT;
- }
- }
- return rc;
+ spin_unlock(&obj->oo_ol_spin);
}
/**
@@ -1106,188 +908,122 @@
*/
static int osc_lock_enqueue(const struct lu_env *env,
const struct cl_lock_slice *slice,
- struct cl_io *unused, __u32 enqflags)
+ struct cl_io *unused, struct cl_sync_io *anchor)
{
- struct osc_lock *ols = cl2osc_lock(slice);
- struct cl_lock *lock = ols->ols_cl.cls_lock;
+ struct osc_thread_info *info = osc_env_info(env);
+ struct osc_io *oio = osc_env_io(env);
+ struct osc_object *osc = cl2osc(slice->cls_obj);
+ struct osc_lock *oscl = cl2osc_lock(slice);
+ struct cl_lock *lock = slice->cls_lock;
+ struct ldlm_res_id *resname = &info->oti_resname;
+ ldlm_policy_data_t *policy = &info->oti_policy;
+ osc_enqueue_upcall_f upcall = osc_lock_upcall;
+ void *cookie = oscl;
+ bool async = false;
int result;
- LASSERT(cl_lock_is_mutexed(lock));
- LASSERTF(ols->ols_state == OLS_NEW,
- "Impossible state: %d\n", ols->ols_state);
+ LASSERTF(ergo(oscl->ols_glimpse, lock->cll_descr.cld_mode <= CLM_READ),
+ "lock = %p, ols = %p\n", lock, oscl);
- LASSERTF(ergo(ols->ols_glimpse, lock->cll_descr.cld_mode <= CLM_READ),
- "lock = %p, ols = %p\n", lock, ols);
+ if (oscl->ols_state == OLS_GRANTED)
+ return 0;
- result = osc_lock_enqueue_wait(env, ols);
- if (result == 0) {
- if (!osc_lock_is_lockless(ols)) {
- struct osc_object *obj = cl2osc(slice->cls_obj);
- struct osc_thread_info *info = osc_env_info(env);
- struct ldlm_res_id *resname = &info->oti_resname;
- ldlm_policy_data_t *policy = &info->oti_policy;
- struct ldlm_enqueue_info *einfo = &ols->ols_einfo;
+ if (oscl->ols_flags & LDLM_FL_TEST_LOCK)
+ goto enqueue_base;
- /* lock will be passed as upcall cookie,
- * hold ref to prevent to be released.
- */
- cl_lock_hold_add(env, lock, "upcall", lock);
- /* a user for lock also */
- cl_lock_user_add(env, lock);
- ols->ols_state = OLS_ENQUEUED;
+ if (oscl->ols_glimpse) {
+ LASSERT(equi(oscl->ols_agl, !anchor));
+ async = true;
+ goto enqueue_base;
+ }
- /*
- * XXX: this is possible blocking point as
- * ldlm_lock_match(LDLM_FL_LVB_READY) waits for
- * LDLM_CP_CALLBACK.
- */
- ostid_build_res_name(&obj->oo_oinfo->loi_oi, resname);
- osc_lock_build_policy(env, lock, policy);
- result = osc_enqueue_base(osc_export(obj), resname,
- &ols->ols_flags, policy,
- &ols->ols_lvb,
- obj->oo_oinfo->loi_kms_valid,
- osc_lock_upcall,
- ols, einfo, &ols->ols_handle,
- PTLRPCD_SET, 1, ols->ols_agl);
- if (result != 0) {
- cl_lock_user_del(env, lock);
- cl_lock_unhold(env, lock, "upcall", lock);
- if (unlikely(result == -ECANCELED)) {
- ols->ols_state = OLS_NEW;
- result = 0;
- }
- }
- } else {
- ols->ols_state = OLS_GRANTED;
- ols->ols_owner = osc_env_io(env);
+ osc_lock_enqueue_wait(env, osc, oscl);
+
+ /* we can grant lockless lock right after all conflicting locks
+ * are canceled.
+ */
+ if (osc_lock_is_lockless(oscl)) {
+ oscl->ols_state = OLS_GRANTED;
+ oio->oi_lockless = 1;
+ return 0;
+ }
+
+enqueue_base:
+ oscl->ols_state = OLS_ENQUEUED;
+ if (anchor) {
+ atomic_inc(&anchor->csi_sync_nr);
+ oscl->ols_owner = anchor;
+ }
+
+ /**
+ * DLM lock's ast data must be osc_object;
+ * if glimpse or AGL lock, async of osc_enqueue_base() must be true,
+ * DLM's enqueue callback set to osc_lock_upcall() with cookie as
+ * osc_lock.
+ */
+ ostid_build_res_name(&osc->oo_oinfo->loi_oi, resname);
+ osc_lock_build_einfo(env, lock, osc, &oscl->ols_einfo);
+ osc_lock_build_policy(env, lock, policy);
+ if (oscl->ols_agl) {
+ oscl->ols_einfo.ei_cbdata = NULL;
+ /* hold a reference for callback */
+ cl_object_get(osc2cl(osc));
+ upcall = osc_lock_upcall_agl;
+ cookie = osc;
+ }
+ result = osc_enqueue_base(osc_export(osc), resname, &oscl->ols_flags,
+ policy, &oscl->ols_lvb,
+ osc->oo_oinfo->loi_kms_valid,
+ upcall, cookie,
+ &oscl->ols_einfo, PTLRPCD_SET, async,
+ oscl->ols_agl);
+ if (result != 0) {
+ oscl->ols_state = OLS_CANCELLED;
+ osc_lock_wake_waiters(env, osc, oscl);
+
+ /* hide error for AGL lock. */
+ if (oscl->ols_agl) {
+ cl_object_put(env, osc2cl(osc));
+ result = 0;
+ }
+ if (anchor)
+ cl_sync_io_note(env, anchor, result);
+ } else {
+ if (osc_lock_is_lockless(oscl)) {
+ oio->oi_lockless = 1;
+ } else if (!async) {
+ LASSERT(oscl->ols_state == OLS_GRANTED);
+ LASSERT(oscl->ols_hold);
+ LASSERT(oscl->ols_dlmlock);
}
}
- LASSERT(ergo(ols->ols_glimpse, !osc_lock_is_lockless(ols)));
return result;
}
-static int osc_lock_wait(const struct lu_env *env,
- const struct cl_lock_slice *slice)
-{
- struct osc_lock *olck = cl2osc_lock(slice);
- struct cl_lock *lock = olck->ols_cl.cls_lock;
-
- LINVRNT(osc_lock_invariant(olck));
-
- if (olck->ols_glimpse && olck->ols_state >= OLS_UPCALL_RECEIVED) {
- if (olck->ols_flags & LDLM_FL_LVB_READY) {
- return 0;
- } else if (olck->ols_agl) {
- if (lock->cll_flags & CLF_FROM_UPCALL)
- /* It is from enqueue RPC reply upcall for
- * updating state. Do not re-enqueue.
- */
- return -ENAVAIL;
- olck->ols_state = OLS_NEW;
- } else {
- LASSERT(lock->cll_error);
- return lock->cll_error;
- }
- }
-
- if (olck->ols_state == OLS_NEW) {
- int rc;
-
- LASSERT(olck->ols_agl);
- olck->ols_agl = 0;
- olck->ols_flags &= ~LDLM_FL_BLOCK_NOWAIT;
- rc = osc_lock_enqueue(env, slice, NULL, CEF_ASYNC | CEF_MUST);
- if (rc != 0)
- return rc;
- else
- return CLO_REENQUEUED;
- }
-
- LASSERT(equi(olck->ols_state >= OLS_UPCALL_RECEIVED &&
- lock->cll_error == 0, olck->ols_lock));
-
- return lock->cll_error ?: olck->ols_state >= OLS_GRANTED ? 0 : CLO_WAIT;
-}
-
/**
- * An implementation of cl_lock_operations::clo_use() method that pins cached
- * lock.
+ * Breaks a link between osc_lock and dlm_lock.
*/
-static int osc_lock_use(const struct lu_env *env,
- const struct cl_lock_slice *slice)
+static void osc_lock_detach(const struct lu_env *env, struct osc_lock *olck)
{
- struct osc_lock *olck = cl2osc_lock(slice);
- int rc;
+ struct ldlm_lock *dlmlock;
- LASSERT(!olck->ols_hold);
+ dlmlock = olck->ols_dlmlock;
+ if (!dlmlock)
+ return;
- /*
- * Atomically check for LDLM_FL_CBPENDING and addref a lock if this
- * flag is not set. This protects us from a concurrent blocking ast.
- */
- rc = ldlm_lock_addref_try(&olck->ols_handle, olck->ols_einfo.ei_mode);
- if (rc == 0) {
- olck->ols_hold = 1;
- olck->ols_state = OLS_GRANTED;
- } else {
- struct cl_lock *lock;
-
- /*
- * Lock is being cancelled somewhere within
- * ldlm_handle_bl_callback(): LDLM_FL_CBPENDING is already
- * set, but osc_ldlm_blocking_ast() hasn't yet acquired
- * cl_lock mutex.
- */
- lock = slice->cls_lock;
- LASSERT(lock->cll_state == CLS_INTRANSIT);
- LASSERT(lock->cll_users > 0);
- /* set a flag for osc_dlm_blocking_ast0() to signal the
- * lock.
- */
- olck->ols_ast_wait = 1;
- rc = CLO_WAIT;
+ if (olck->ols_hold) {
+ olck->ols_hold = 0;
+ osc_cancel_base(&olck->ols_handle, olck->ols_einfo.ei_mode);
+ olck->ols_handle.cookie = 0ULL;
}
- return rc;
-}
-static int osc_lock_flush(struct osc_lock *ols, int discard)
-{
- struct cl_lock *lock = ols->ols_cl.cls_lock;
- struct cl_env_nest nest;
- struct lu_env *env;
- int result = 0;
+ olck->ols_dlmlock = NULL;
- env = cl_env_nested_get(&nest);
- if (!IS_ERR(env)) {
- struct osc_object *obj = cl2osc(ols->ols_cl.cls_obj);
- struct cl_lock_descr *descr = &lock->cll_descr;
- int rc = 0;
-
- if (descr->cld_mode >= CLM_WRITE) {
- result = osc_cache_writeback_range(env, obj,
- descr->cld_start,
- descr->cld_end,
- 1, discard);
- LDLM_DEBUG(ols->ols_lock,
- "lock %p: %d pages were %s.\n", lock, result,
- discard ? "discarded" : "written");
- if (result > 0)
- result = 0;
- }
-
- rc = cl_lock_discard_pages(env, lock);
- if (result == 0 && rc < 0)
- result = rc;
-
- cl_env_nested_put(&nest, env);
- } else
- result = PTR_ERR(env);
- if (result == 0) {
- ols->ols_flush = 1;
- LINVRNT(!osc_lock_has_pages(ols));
- }
- return result;
+ /* release a reference taken in osc_lock_upcall(). */
+ LASSERT(olck->ols_has_ref);
+ lu_ref_del(&dlmlock->l_reference, "osc_lock", olck);
+ LDLM_LOCK_RELEASE(dlmlock);
+ olck->ols_has_ref = 0;
}
/**
@@ -1307,96 +1043,16 @@
static void osc_lock_cancel(const struct lu_env *env,
const struct cl_lock_slice *slice)
{
- struct cl_lock *lock = slice->cls_lock;
- struct osc_lock *olck = cl2osc_lock(slice);
- struct ldlm_lock *dlmlock = olck->ols_lock;
- int result = 0;
- int discard;
+ struct osc_object *obj = cl2osc(slice->cls_obj);
+ struct osc_lock *oscl = cl2osc_lock(slice);
- LASSERT(cl_lock_is_mutexed(lock));
- LINVRNT(osc_lock_invariant(olck));
+ LINVRNT(osc_lock_invariant(oscl));
- if (dlmlock) {
- int do_cancel;
+ osc_lock_detach(env, oscl);
+ oscl->ols_state = OLS_CANCELLED;
+ oscl->ols_flags &= ~LDLM_FL_LVB_READY;
- discard = !!(dlmlock->l_flags & LDLM_FL_DISCARD_DATA);
- if (olck->ols_state >= OLS_GRANTED)
- result = osc_lock_flush(olck, discard);
- osc_lock_unhold(olck);
-
- lock_res_and_lock(dlmlock);
- /* Now that we're the only user of dlm read/write reference,
- * mostly the ->l_readers + ->l_writers should be zero.
- * However, there is a corner case.
- * See bug 18829 for details.
- */
- do_cancel = (dlmlock->l_readers == 0 &&
- dlmlock->l_writers == 0);
- dlmlock->l_flags |= LDLM_FL_CBPENDING;
- unlock_res_and_lock(dlmlock);
- if (do_cancel)
- result = ldlm_cli_cancel(&olck->ols_handle, LCF_ASYNC);
- if (result < 0)
- CL_LOCK_DEBUG(D_ERROR, env, lock,
- "lock %p cancel failure with error(%d)\n",
- lock, result);
- }
- olck->ols_state = OLS_CANCELLED;
- olck->ols_flags &= ~LDLM_FL_LVB_READY;
- osc_lock_detach(env, olck);
-}
-
-static int osc_lock_has_pages(struct osc_lock *olck)
-{
- return 0;
-}
-
-static void osc_lock_delete(const struct lu_env *env,
- const struct cl_lock_slice *slice)
-{
- struct osc_lock *olck;
-
- olck = cl2osc_lock(slice);
- if (olck->ols_glimpse) {
- LASSERT(!olck->ols_hold);
- LASSERT(!olck->ols_lock);
- return;
- }
-
- LINVRNT(osc_lock_invariant(olck));
- LINVRNT(!osc_lock_has_pages(olck));
-
- osc_lock_unhold(olck);
- osc_lock_detach(env, olck);
-}
-
-/**
- * Implements cl_lock_operations::clo_state() method for osc layer.
- *
- * Maintains osc_lock::ols_owner field.
- *
- * This assumes that lock always enters CLS_HELD (from some other state) in
- * the same IO context as one that requested the lock. This should not be a
- * problem, because context is by definition shared by all activity pertaining
- * to the same high-level IO.
- */
-static void osc_lock_state(const struct lu_env *env,
- const struct cl_lock_slice *slice,
- enum cl_lock_state state)
-{
- struct osc_lock *lock = cl2osc_lock(slice);
-
- /*
- * XXX multiple io contexts can use the lock at the same time.
- */
- LINVRNT(osc_lock_invariant(lock));
- if (state == CLS_HELD && slice->cls_lock->cll_state != CLS_HELD) {
- struct osc_io *oio = osc_env_io(env);
-
- LASSERT(!lock->ols_owner);
- lock->ols_owner = oio;
- } else if (state != CLS_HELD)
- lock->ols_owner = NULL;
+ osc_lock_wake_waiters(env, obj, oscl);
}
static int osc_lock_print(const struct lu_env *env, void *cookie,
@@ -1404,221 +1060,161 @@
{
struct osc_lock *lock = cl2osc_lock(slice);
- /*
- * XXX print ldlm lock and einfo properly.
- */
(*p)(env, cookie, "%p %#16llx %#llx %d %p ",
- lock->ols_lock, lock->ols_flags, lock->ols_handle.cookie,
+ lock->ols_dlmlock, lock->ols_flags, lock->ols_handle.cookie,
lock->ols_state, lock->ols_owner);
osc_lvb_print(env, cookie, p, &lock->ols_lvb);
return 0;
}
-static int osc_lock_fits_into(const struct lu_env *env,
- const struct cl_lock_slice *slice,
- const struct cl_lock_descr *need,
- const struct cl_io *io)
-{
- struct osc_lock *ols = cl2osc_lock(slice);
-
- if (need->cld_enq_flags & CEF_NEVER)
- return 0;
-
- if (ols->ols_state >= OLS_CANCELLED)
- return 0;
-
- if (need->cld_mode == CLM_PHANTOM) {
- if (ols->ols_agl)
- return !(ols->ols_state > OLS_RELEASED);
-
- /*
- * Note: the QUEUED lock can't be matched here, otherwise
- * it might cause the deadlocks.
- * In read_process,
- * P1: enqueued read lock, create sublock1
- * P2: enqueued write lock, create sublock2(conflicted
- * with sublock1).
- * P1: Grant read lock.
- * P1: enqueued glimpse lock(with holding sublock1_read),
- * matched with sublock2, waiting sublock2 to be granted.
- * But sublock2 can not be granted, because P1
- * will not release sublock1. Bang!
- */
- if (ols->ols_state < OLS_GRANTED ||
- ols->ols_state > OLS_RELEASED)
- return 0;
- } else if (need->cld_enq_flags & CEF_MUST) {
- /*
- * If the lock hasn't ever enqueued, it can't be matched
- * because enqueue process brings in many information
- * which can be used to determine things such as lockless,
- * CEF_MUST, etc.
- */
- if (ols->ols_state < OLS_UPCALL_RECEIVED &&
- ols->ols_locklessable)
- return 0;
- }
- return 1;
-}
-
static const struct cl_lock_operations osc_lock_ops = {
.clo_fini = osc_lock_fini,
.clo_enqueue = osc_lock_enqueue,
- .clo_wait = osc_lock_wait,
- .clo_unuse = osc_lock_unuse,
- .clo_use = osc_lock_use,
- .clo_delete = osc_lock_delete,
- .clo_state = osc_lock_state,
.clo_cancel = osc_lock_cancel,
- .clo_weigh = osc_lock_weigh,
.clo_print = osc_lock_print,
- .clo_fits_into = osc_lock_fits_into,
};
-static int osc_lock_lockless_unuse(const struct lu_env *env,
- const struct cl_lock_slice *slice)
-{
- struct osc_lock *ols = cl2osc_lock(slice);
- struct cl_lock *lock = slice->cls_lock;
-
- LASSERT(ols->ols_state == OLS_GRANTED);
- LINVRNT(osc_lock_invariant(ols));
-
- cl_lock_cancel(env, lock);
- cl_lock_delete(env, lock);
- return 0;
-}
-
static void osc_lock_lockless_cancel(const struct lu_env *env,
const struct cl_lock_slice *slice)
{
struct osc_lock *ols = cl2osc_lock(slice);
+ struct osc_object *osc = cl2osc(slice->cls_obj);
+ struct cl_lock_descr *descr = &slice->cls_lock->cll_descr;
int result;
- result = osc_lock_flush(ols, 0);
+ LASSERT(!ols->ols_dlmlock);
+ result = osc_lock_flush(osc, descr->cld_start, descr->cld_end,
+ descr->cld_mode, 0);
if (result)
CERROR("Pages for lockless lock %p were not purged(%d)\n",
ols, result);
- ols->ols_state = OLS_CANCELLED;
-}
-static int osc_lock_lockless_wait(const struct lu_env *env,
- const struct cl_lock_slice *slice)
-{
- struct osc_lock *olck = cl2osc_lock(slice);
- struct cl_lock *lock = olck->ols_cl.cls_lock;
-
- LINVRNT(osc_lock_invariant(olck));
- LASSERT(olck->ols_state >= OLS_UPCALL_RECEIVED);
-
- return lock->cll_error;
-}
-
-static void osc_lock_lockless_state(const struct lu_env *env,
- const struct cl_lock_slice *slice,
- enum cl_lock_state state)
-{
- struct osc_lock *lock = cl2osc_lock(slice);
-
- LINVRNT(osc_lock_invariant(lock));
- if (state == CLS_HELD) {
- struct osc_io *oio = osc_env_io(env);
-
- LASSERT(ergo(lock->ols_owner, lock->ols_owner == oio));
- lock->ols_owner = oio;
-
- /* set the io to be lockless if this lock is for io's
- * host object
- */
- if (cl_object_same(oio->oi_cl.cis_obj, slice->cls_obj))
- oio->oi_lockless = 1;
- }
-}
-
-static int osc_lock_lockless_fits_into(const struct lu_env *env,
- const struct cl_lock_slice *slice,
- const struct cl_lock_descr *need,
- const struct cl_io *io)
-{
- struct osc_lock *lock = cl2osc_lock(slice);
-
- if (!(need->cld_enq_flags & CEF_NEVER))
- return 0;
-
- /* lockless lock should only be used by its owning io. b22147 */
- return (lock->ols_owner == osc_env_io(env));
+ osc_lock_wake_waiters(env, osc, ols);
}
static const struct cl_lock_operations osc_lock_lockless_ops = {
- .clo_fini = osc_lock_fini,
- .clo_enqueue = osc_lock_enqueue,
- .clo_wait = osc_lock_lockless_wait,
- .clo_unuse = osc_lock_lockless_unuse,
- .clo_state = osc_lock_lockless_state,
- .clo_fits_into = osc_lock_lockless_fits_into,
- .clo_cancel = osc_lock_lockless_cancel,
- .clo_print = osc_lock_print
+ .clo_fini = osc_lock_fini,
+ .clo_enqueue = osc_lock_enqueue,
+ .clo_cancel = osc_lock_lockless_cancel,
+ .clo_print = osc_lock_print
};
+static void osc_lock_set_writer(const struct lu_env *env,
+ const struct cl_io *io,
+ struct cl_object *obj, struct osc_lock *oscl)
+{
+ struct cl_lock_descr *descr = &oscl->ols_cl.cls_lock->cll_descr;
+ pgoff_t io_start;
+ pgoff_t io_end;
+
+ if (!cl_object_same(io->ci_obj, obj))
+ return;
+
+ if (likely(io->ci_type == CIT_WRITE)) {
+ io_start = cl_index(obj, io->u.ci_rw.crw_pos);
+ io_end = cl_index(obj, io->u.ci_rw.crw_pos +
+ io->u.ci_rw.crw_count - 1);
+ if (cl_io_is_append(io)) {
+ io_start = 0;
+ io_end = CL_PAGE_EOF;
+ }
+ } else {
+ LASSERT(cl_io_is_mkwrite(io));
+ io_start = io_end = io->u.ci_fault.ft_index;
+ }
+
+ if (descr->cld_mode >= CLM_WRITE &&
+ descr->cld_start <= io_start && descr->cld_end >= io_end) {
+ struct osc_io *oio = osc_env_io(env);
+
+ /* There must be only one lock to match the write region */
+ LASSERT(!oio->oi_write_osclock);
+ oio->oi_write_osclock = oscl;
+ }
+}
+
int osc_lock_init(const struct lu_env *env,
struct cl_object *obj, struct cl_lock *lock,
- const struct cl_io *unused)
+ const struct cl_io *io)
{
- struct osc_lock *clk;
- int result;
+ struct osc_lock *oscl;
+ __u32 enqflags = lock->cll_descr.cld_enq_flags;
- clk = kmem_cache_zalloc(osc_lock_kmem, GFP_NOFS);
- if (clk) {
- __u32 enqflags = lock->cll_descr.cld_enq_flags;
+ oscl = kmem_cache_zalloc(osc_lock_kmem, GFP_NOFS);
+ if (!oscl)
+ return -ENOMEM;
- osc_lock_build_einfo(env, lock, clk, &clk->ols_einfo);
- atomic_set(&clk->ols_pageref, 0);
- clk->ols_state = OLS_NEW;
+ oscl->ols_state = OLS_NEW;
+ spin_lock_init(&oscl->ols_lock);
+ INIT_LIST_HEAD(&oscl->ols_waiting_list);
+ INIT_LIST_HEAD(&oscl->ols_wait_entry);
+ INIT_LIST_HEAD(&oscl->ols_nextlock_oscobj);
- clk->ols_flags = osc_enq2ldlm_flags(enqflags);
- clk->ols_agl = !!(enqflags & CEF_AGL);
- if (clk->ols_agl)
- clk->ols_flags |= LDLM_FL_BLOCK_NOWAIT;
- if (clk->ols_flags & LDLM_FL_HAS_INTENT)
- clk->ols_glimpse = 1;
+ oscl->ols_flags = osc_enq2ldlm_flags(enqflags);
+ oscl->ols_agl = !!(enqflags & CEF_AGL);
+ if (oscl->ols_agl)
+ oscl->ols_flags |= LDLM_FL_BLOCK_NOWAIT;
+ if (oscl->ols_flags & LDLM_FL_HAS_INTENT) {
+ oscl->ols_flags |= LDLM_FL_BLOCK_GRANTED;
+ oscl->ols_glimpse = 1;
+ }
- cl_lock_slice_add(lock, &clk->ols_cl, obj, &osc_lock_ops);
+ cl_lock_slice_add(lock, &oscl->ols_cl, obj, &osc_lock_ops);
- if (!(enqflags & CEF_MUST))
- /* try to convert this lock to a lockless lock */
- osc_lock_to_lockless(env, clk, (enqflags & CEF_NEVER));
- if (clk->ols_locklessable && !(enqflags & CEF_DISCARD_DATA))
- clk->ols_flags |= LDLM_FL_DENY_ON_CONTENTION;
+ if (!(enqflags & CEF_MUST))
+ /* try to convert this lock to a lockless lock */
+ osc_lock_to_lockless(env, oscl, (enqflags & CEF_NEVER));
+ if (oscl->ols_locklessable && !(enqflags & CEF_DISCARD_DATA))
+ oscl->ols_flags |= LDLM_FL_DENY_ON_CONTENTION;
- LDLM_DEBUG_NOLOCK("lock %p, osc lock %p, flags %llx",
- lock, clk, clk->ols_flags);
+ if (io->ci_type == CIT_WRITE || cl_io_is_mkwrite(io))
+ osc_lock_set_writer(env, io, obj, oscl);
- result = 0;
- } else
- result = -ENOMEM;
- return result;
+
+ LDLM_DEBUG_NOLOCK("lock %p, osc lock %p, flags %llx\n",
+ lock, oscl, oscl->ols_flags);
+
+ return 0;
}
-int osc_dlm_lock_pageref(struct ldlm_lock *dlm)
+/**
+ * Finds an existing lock covering given index and optionally different from a
+ * given \a except lock.
+ */
+struct ldlm_lock *osc_dlmlock_at_pgoff(const struct lu_env *env,
+ struct osc_object *obj, pgoff_t index,
+ int pending, int canceling)
{
- struct osc_lock *olock;
- int rc = 0;
+ struct osc_thread_info *info = osc_env_info(env);
+ struct ldlm_res_id *resname = &info->oti_resname;
+ ldlm_policy_data_t *policy = &info->oti_policy;
+ struct lustre_handle lockh;
+ struct ldlm_lock *lock = NULL;
+ enum ldlm_mode mode;
+ __u64 flags;
- spin_lock(&osc_ast_guard);
- olock = dlm->l_ast_data;
+ ostid_build_res_name(&obj->oo_oinfo->loi_oi, resname);
+ osc_index2policy(policy, osc2cl(obj), index, index);
+ policy->l_extent.gid = LDLM_GID_ANY;
+
+ flags = LDLM_FL_BLOCK_GRANTED | LDLM_FL_TEST_LOCK;
+ if (pending)
+ flags |= LDLM_FL_CBPENDING;
/*
- * there's a very rare race with osc_page_addref_lock(), but that
- * doesn't matter because in the worst case we don't cancel a lock
- * which we actually can, that's no harm.
+ * It is fine to match any group lock since there could be only one
+ * with a uniq gid and it conflicts with all other lock modes too
*/
- if (olock &&
- atomic_add_return(_PAGEREF_MAGIC,
- &olock->ols_pageref) != _PAGEREF_MAGIC) {
- atomic_sub(_PAGEREF_MAGIC, &olock->ols_pageref);
- rc = 1;
+again:
+ mode = ldlm_lock_match(osc_export(obj)->exp_obd->obd_namespace,
+ flags, resname, LDLM_EXTENT, policy,
+ LCK_PR | LCK_PW | LCK_GROUP, &lockh, canceling);
+ if (mode != 0) {
+ lock = ldlm_handle2lock(&lockh);
+ /* RACE: the lock is cancelled so let's try again */
+ if (unlikely(!lock))
+ goto again;
}
- spin_unlock(&osc_ast_guard);
- return rc;
+ return lock;
}
/** @} osc */
diff --git a/drivers/staging/lustre/lustre/osc/osc_object.c b/drivers/staging/lustre/lustre/osc/osc_object.c
index 9d474fc..738ab10a 100644
--- a/drivers/staging/lustre/lustre/osc/osc_object.c
+++ b/drivers/staging/lustre/lustre/osc/osc_object.c
@@ -36,6 +36,7 @@
* Implementation of cl_object for OSC layer.
*
* Author: Nikita Danilov <nikita.danilov@sun.com>
+ * Author: Jinshan Xiong <jinshan.xiong@intel.com>
*/
#define DEBUG_SUBSYSTEM S_OSC
@@ -94,6 +95,9 @@
atomic_set(&osc->oo_nr_reads, 0);
atomic_set(&osc->oo_nr_writes, 0);
spin_lock_init(&osc->oo_lock);
+ spin_lock_init(&osc->oo_tree_lock);
+ spin_lock_init(&osc->oo_ol_spin);
+ INIT_LIST_HEAD(&osc->oo_ol_list);
cl_object_page_init(lu2cl(obj), sizeof(struct osc_page));
@@ -120,6 +124,7 @@
LASSERT(list_empty(&osc->oo_reading_exts));
LASSERT(atomic_read(&osc->oo_nr_reads) == 0);
LASSERT(atomic_read(&osc->oo_nr_writes) == 0);
+ LASSERT(list_empty(&osc->oo_ol_list));
lu_object_fini(obj);
kmem_cache_free(osc_object_kmem, osc);
@@ -192,6 +197,32 @@
return 0;
}
+static int osc_object_ast_clear(struct ldlm_lock *lock, void *data)
+{
+ LASSERT(lock->l_granted_mode == lock->l_req_mode);
+ if (lock->l_ast_data == data)
+ lock->l_ast_data = NULL;
+ return LDLM_ITER_CONTINUE;
+}
+
+static int osc_object_prune(const struct lu_env *env, struct cl_object *obj)
+{
+ struct osc_object *osc = cl2osc(obj);
+ struct ldlm_res_id *resname = &osc_env_info(env)->oti_resname;
+
+ LASSERTF(osc->oo_npages == 0,
+ DFID "still have %lu pages, obj: %p, osc: %p\n",
+ PFID(lu_object_fid(&obj->co_lu)), osc->oo_npages, obj, osc);
+
+ /* DLM locks don't hold a reference of osc_object so we have to
+ * clear it before the object is being destroyed.
+ */
+ ostid_build_res_name(&osc->oo_oinfo->loi_oi, resname);
+ ldlm_resource_iterate(osc_export(osc)->exp_obd->obd_namespace, resname,
+ osc_object_ast_clear, osc);
+ return 0;
+}
+
void osc_object_set_contended(struct osc_object *obj)
{
obj->oo_contention_time = cfs_time_current();
@@ -236,12 +267,12 @@
.coo_io_init = osc_io_init,
.coo_attr_get = osc_attr_get,
.coo_attr_set = osc_attr_set,
- .coo_glimpse = osc_object_glimpse
+ .coo_glimpse = osc_object_glimpse,
+ .coo_prune = osc_object_prune
};
static const struct lu_object_operations osc_lu_obj_ops = {
.loo_object_init = osc_object_init,
- .loo_object_delete = NULL,
.loo_object_release = NULL,
.loo_object_free = osc_object_free,
.loo_object_print = osc_object_print,
@@ -261,8 +292,9 @@
lu_object_init(obj, NULL, dev);
osc->oo_cl.co_ops = &osc_ops;
obj->lo_ops = &osc_lu_obj_ops;
- } else
+ } else {
obj = NULL;
+ }
return obj;
}
diff --git a/drivers/staging/lustre/lustre/osc/osc_page.c b/drivers/staging/lustre/lustre/osc/osc_page.c
index ce9ddd5..c29c2ea 100644
--- a/drivers/staging/lustre/lustre/osc/osc_page.c
+++ b/drivers/staging/lustre/lustre/osc/osc_page.c
@@ -36,14 +36,15 @@
* Implementation of cl_page for OSC layer.
*
* Author: Nikita Danilov <nikita.danilov@sun.com>
+ * Author: Jinshan Xiong <jinshan.xiong@intel.com>
*/
#define DEBUG_SUBSYSTEM S_OSC
#include "osc_cl_internal.h"
-static void osc_lru_del(struct client_obd *cli, struct osc_page *opg, bool del);
-static void osc_lru_add(struct client_obd *cli, struct osc_page *opg);
+static void osc_lru_del(struct client_obd *cli, struct osc_page *opg);
+static void osc_lru_use(struct client_obd *cli, struct osc_page *opg);
static int osc_lru_reserve(const struct lu_env *env, struct osc_object *obj,
struct osc_page *opg);
@@ -63,18 +64,9 @@
* Page operations.
*
*/
-static void osc_page_fini(const struct lu_env *env,
- struct cl_page_slice *slice)
-{
- struct osc_page *opg = cl2osc_page(slice);
-
- CDEBUG(D_TRACE, "%p\n", opg);
- LASSERT(!opg->ops_lock);
-}
-
static void osc_page_transfer_get(struct osc_page *opg, const char *label)
{
- struct cl_page *page = cl_page_top(opg->ops_cl.cpl_page);
+ struct cl_page *page = opg->ops_cl.cpl_page;
LASSERT(!opg->ops_transfer_pinned);
cl_page_get(page);
@@ -85,11 +77,11 @@
static void osc_page_transfer_put(const struct lu_env *env,
struct osc_page *opg)
{
- struct cl_page *page = cl_page_top(opg->ops_cl.cpl_page);
+ struct cl_page *page = opg->ops_cl.cpl_page;
if (opg->ops_transfer_pinned) {
- lu_ref_del(&page->cp_reference, "transfer", page);
opg->ops_transfer_pinned = 0;
+ lu_ref_del(&page->cp_reference, "transfer", page);
cl_page_put(env, page);
}
}
@@ -104,10 +96,7 @@
{
struct osc_object *obj = cl2osc(opg->ops_cl.cpl_obj);
- /* ops_lru and ops_inflight share the same field, so take it from LRU
- * first and then use it as inflight.
- */
- osc_lru_del(osc_cli(obj), opg, false);
+ osc_lru_use(osc_cli(obj), opg);
spin_lock(&obj->oo_seatbelt);
list_add(&opg->ops_inflight, &obj->oo_inflight[crt]);
@@ -115,11 +104,9 @@
spin_unlock(&obj->oo_seatbelt);
}
-static int osc_page_cache_add(const struct lu_env *env,
- const struct cl_page_slice *slice,
- struct cl_io *io)
+int osc_page_cache_add(const struct lu_env *env,
+ const struct cl_page_slice *slice, struct cl_io *io)
{
- struct osc_io *oio = osc_env_io(env);
struct osc_page *opg = cl2osc_page(slice);
int result;
@@ -132,17 +119,6 @@
else
osc_page_transfer_add(env, opg, CRT_WRITE);
- /* for sync write, kernel will wait for this page to be flushed before
- * osc_io_end() is called, so release it earlier.
- * for mkwrite(), it's known there is no further pages.
- */
- if (cl_io_is_sync_write(io) || cl_io_is_mkwrite(io)) {
- if (oio->oi_active) {
- osc_extent_release(env, oio->oi_active);
- oio->oi_active = NULL;
- }
- }
-
return result;
}
@@ -154,102 +130,25 @@
policy->l_extent.end = cl_offset(obj, end + 1) - 1;
}
-static int osc_page_addref_lock(const struct lu_env *env,
- struct osc_page *opg,
- struct cl_lock *lock)
-{
- struct osc_lock *olock;
- int rc;
-
- LASSERT(!opg->ops_lock);
-
- olock = osc_lock_at(lock);
- if (atomic_inc_return(&olock->ols_pageref) <= 0) {
- atomic_dec(&olock->ols_pageref);
- rc = -ENODATA;
- } else {
- cl_lock_get(lock);
- opg->ops_lock = lock;
- rc = 0;
- }
- return rc;
-}
-
-static void osc_page_putref_lock(const struct lu_env *env,
- struct osc_page *opg)
-{
- struct cl_lock *lock = opg->ops_lock;
- struct osc_lock *olock;
-
- LASSERT(lock);
- olock = osc_lock_at(lock);
-
- atomic_dec(&olock->ols_pageref);
- opg->ops_lock = NULL;
-
- cl_lock_put(env, lock);
-}
-
static int osc_page_is_under_lock(const struct lu_env *env,
const struct cl_page_slice *slice,
- struct cl_io *unused)
+ struct cl_io *unused, pgoff_t *max_index)
{
- struct cl_lock *lock;
+ struct osc_page *opg = cl2osc_page(slice);
+ struct ldlm_lock *dlmlock;
int result = -ENODATA;
- lock = cl_lock_at_page(env, slice->cpl_obj, slice->cpl_page,
- NULL, 1, 0);
- if (lock) {
- if (osc_page_addref_lock(env, cl2osc_page(slice), lock) == 0)
- result = -EBUSY;
- cl_lock_put(env, lock);
+ dlmlock = osc_dlmlock_at_pgoff(env, cl2osc(slice->cpl_obj),
+ osc_index(opg), 1, 0);
+ if (dlmlock) {
+ *max_index = cl_index(slice->cpl_obj,
+ dlmlock->l_policy_data.l_extent.end);
+ LDLM_LOCK_PUT(dlmlock);
+ result = 0;
}
return result;
}
-static void osc_page_disown(const struct lu_env *env,
- const struct cl_page_slice *slice,
- struct cl_io *io)
-{
- struct osc_page *opg = cl2osc_page(slice);
-
- if (unlikely(opg->ops_lock))
- osc_page_putref_lock(env, opg);
-}
-
-static void osc_page_completion_read(const struct lu_env *env,
- const struct cl_page_slice *slice,
- int ioret)
-{
- struct osc_page *opg = cl2osc_page(slice);
- struct osc_object *obj = cl2osc(opg->ops_cl.cpl_obj);
-
- if (likely(opg->ops_lock))
- osc_page_putref_lock(env, opg);
- osc_lru_add(osc_cli(obj), opg);
-}
-
-static void osc_page_completion_write(const struct lu_env *env,
- const struct cl_page_slice *slice,
- int ioret)
-{
- struct osc_page *opg = cl2osc_page(slice);
- struct osc_object *obj = cl2osc(slice->cpl_obj);
-
- osc_lru_add(osc_cli(obj), opg);
-}
-
-static int osc_page_fail(const struct lu_env *env,
- const struct cl_page_slice *slice,
- struct cl_io *unused)
-{
- /*
- * Cached read?
- */
- LBUG();
- return 0;
-}
-
static const char *osc_list(struct list_head *head)
{
return list_empty(head) ? "-" : "+";
@@ -272,8 +171,8 @@
struct osc_object *obj = cl2osc(slice->cpl_obj);
struct client_obd *cli = &osc_export(obj)->exp_obd->u.cli;
- return (*printer)(env, cookie, LUSTRE_OSC_NAME "-page@%p: 1< %#x %d %u %s %s > 2< %llu %u %u %#x %#x | %p %p %p > 3< %s %p %d %lu %d > 4< %d %d %d %lu %s | %s %s %s %s > 5< %s %s %s %s | %d %s | %d %s %s>\n",
- opg,
+ return (*printer)(env, cookie, LUSTRE_OSC_NAME "-page@%p %lu: 1< %#x %d %u %s %s > 2< %llu %u %u %#x %#x | %p %p %p > 3< %s %p %d %lu %d > 4< %d %d %d %lu %s | %s %s %s %s > 5< %s %s %s %s | %d %s | %d %s %s>\n",
+ opg, osc_index(opg),
/* 1 */
oap->oap_magic, oap->oap_cmd,
oap->oap_interrupted,
@@ -321,7 +220,7 @@
osc_page_transfer_put(env, opg);
rc = osc_teardown_async_page(env, obj, opg);
if (rc) {
- CL_PAGE_DEBUG(D_ERROR, env, cl_page_top(slice->cpl_page),
+ CL_PAGE_DEBUG(D_ERROR, env, slice->cpl_page,
"Trying to teardown failed: %d\n", rc);
LASSERT(0);
}
@@ -334,7 +233,19 @@
}
spin_unlock(&obj->oo_seatbelt);
- osc_lru_del(osc_cli(obj), opg, true);
+ osc_lru_del(osc_cli(obj), opg);
+
+ if (slice->cpl_page->cp_type == CPT_CACHEABLE) {
+ void *value;
+
+ spin_lock(&obj->oo_tree_lock);
+ value = radix_tree_delete(&obj->oo_tree, osc_index(opg));
+ if (value)
+ --obj->oo_npages;
+ spin_unlock(&obj->oo_tree_lock);
+
+ LASSERT(ergo(value, value == opg));
+ }
}
static void osc_page_clip(const struct lu_env *env,
@@ -382,28 +293,16 @@
}
static const struct cl_page_operations osc_page_ops = {
- .cpo_fini = osc_page_fini,
.cpo_print = osc_page_print,
.cpo_delete = osc_page_delete,
.cpo_is_under_lock = osc_page_is_under_lock,
- .cpo_disown = osc_page_disown,
- .io = {
- [CRT_READ] = {
- .cpo_cache_add = osc_page_fail,
- .cpo_completion = osc_page_completion_read
- },
- [CRT_WRITE] = {
- .cpo_cache_add = osc_page_cache_add,
- .cpo_completion = osc_page_completion_write
- }
- },
.cpo_clip = osc_page_clip,
.cpo_cancel = osc_page_cancel,
.cpo_flush = osc_page_flush
};
int osc_page_init(const struct lu_env *env, struct cl_object *obj,
- struct cl_page *page, struct page *vmpage)
+ struct cl_page *page, pgoff_t index)
{
struct osc_object *osc = cl2osc(obj);
struct osc_page *opg = cl_object_page_slice(obj, page);
@@ -412,13 +311,14 @@
opg->ops_from = 0;
opg->ops_to = PAGE_SIZE;
- result = osc_prep_async_page(osc, opg, vmpage,
- cl_offset(obj, page->cp_index));
+ result = osc_prep_async_page(osc, opg, page->cp_vmpage,
+ cl_offset(obj, index));
if (result == 0) {
struct osc_io *oio = osc_env_io(env);
opg->ops_srvlock = osc_io_srvlock(oio);
- cl_page_slice_add(page, &opg->ops_cl, obj, &osc_page_ops);
+ cl_page_slice_add(page, &opg->ops_cl, obj, index,
+ &osc_page_ops);
}
/*
* Cannot assert osc_page_protected() here as read-ahead
@@ -431,12 +331,47 @@
INIT_LIST_HEAD(&opg->ops_lru);
/* reserve an LRU space for this page */
- if (page->cp_type == CPT_CACHEABLE && result == 0)
+ if (page->cp_type == CPT_CACHEABLE && result == 0) {
result = osc_lru_reserve(env, osc, opg);
+ if (result == 0) {
+ spin_lock(&osc->oo_tree_lock);
+ result = radix_tree_insert(&osc->oo_tree, index, opg);
+ if (result == 0)
+ ++osc->oo_npages;
+ spin_unlock(&osc->oo_tree_lock);
+ LASSERT(result == 0);
+ }
+ }
return result;
}
+int osc_over_unstable_soft_limit(struct client_obd *cli)
+{
+ long obd_upages, obd_dpages, osc_upages;
+
+ /* Can't check cli->cl_unstable_count, therefore, no soft limit */
+ if (!cli)
+ return 0;
+
+ obd_upages = atomic_read(&obd_unstable_pages);
+ obd_dpages = atomic_read(&obd_dirty_pages);
+
+ osc_upages = atomic_read(&cli->cl_unstable_count);
+
+ /*
+ * obd_max_dirty_pages is the max number of (dirty + unstable)
+ * pages allowed at any given time. To simulate an unstable page
+ * only limit, we subtract the current number of dirty pages
+ * from this max. This difference is roughly the amount of pages
+ * currently available for unstable pages. Thus, the soft limit
+ * is half of that difference. Check osc_upages to ensure we don't
+ * set SOFT_SYNC for OSCs without any outstanding unstable pages.
+ */
+ return osc_upages &&
+ obd_upages >= (obd_max_dirty_pages - obd_dpages) / 2;
+}
+
/**
* Helper function called by osc_io_submit() for every page in an immediate
* transfer (i.e., transferred synchronously).
@@ -460,6 +395,9 @@
oap->oap_count = opg->ops_to - opg->ops_from;
oap->oap_brw_flags = brw_flags | OBD_BRW_SYNC;
+ if (osc_over_unstable_soft_limit(oap->oap_cli))
+ oap->oap_brw_flags |= OBD_BRW_SOFT_SYNC;
+
if (!client_is_remote(osc_export(obj)) &&
capable(CFS_CAP_SYS_RESOURCE)) {
oap->oap_brw_flags |= OBD_BRW_NOQUOTA;
@@ -483,13 +421,12 @@
*/
static DECLARE_WAIT_QUEUE_HEAD(osc_lru_waitq);
-static atomic_t osc_lru_waiters = ATOMIC_INIT(0);
/* LRU pages are freed in batch mode. OSC should at least free this
* number of pages to avoid running out of LRU budget, and..
*/
static const int lru_shrink_min = 2 << (20 - PAGE_SHIFT); /* 2M */
/* free this number at most otherwise it will take too long time to finish. */
-static const int lru_shrink_max = 32 << (20 - PAGE_SHIFT); /* 32M */
+static const int lru_shrink_max = 8 << (20 - PAGE_SHIFT); /* 8M */
/* Check if we can free LRU slots from this OSC. If there exists LRU waiters,
* we should free slots aggressively. In this way, slots are freed in a steady
@@ -500,65 +437,142 @@
static int osc_cache_too_much(struct client_obd *cli)
{
struct cl_client_cache *cache = cli->cl_cache;
- int pages = atomic_read(&cli->cl_lru_in_list) >> 1;
+ int pages = atomic_read(&cli->cl_lru_in_list);
+ unsigned long budget;
- if (atomic_read(&osc_lru_waiters) > 0 &&
- atomic_read(cli->cl_lru_left) < lru_shrink_max)
- /* drop lru pages aggressively */
- return min(pages, lru_shrink_max);
+ budget = cache->ccc_lru_max / atomic_read(&cache->ccc_users);
/* if it's going to run out LRU slots, we should free some, but not
* too much to maintain fairness among OSCs.
*/
if (atomic_read(cli->cl_lru_left) < cache->ccc_lru_max >> 4) {
- unsigned long tmp;
-
- tmp = cache->ccc_lru_max / atomic_read(&cache->ccc_users);
- if (pages > tmp)
- return min(pages, lru_shrink_max);
-
- return pages > lru_shrink_min ? lru_shrink_min : 0;
+ if (pages >= budget)
+ return lru_shrink_max;
+ else if (pages >= budget / 2)
+ return lru_shrink_min;
+ } else if (pages >= budget * 2) {
+ return lru_shrink_min;
}
+ return 0;
+}
+
+int lru_queue_work(const struct lu_env *env, void *data)
+{
+ struct client_obd *cli = data;
+
+ CDEBUG(D_CACHE, "Run LRU work for client obd %p.\n", cli);
+
+ if (osc_cache_too_much(cli))
+ osc_lru_shrink(env, cli, lru_shrink_max, true);
return 0;
}
-/* Return how many pages are not discarded in @pvec. */
-static int discard_pagevec(const struct lu_env *env, struct cl_io *io,
- struct cl_page **pvec, int max_index)
+void osc_lru_add_batch(struct client_obd *cli, struct list_head *plist)
{
- int count;
+ LIST_HEAD(lru);
+ struct osc_async_page *oap;
+ int npages = 0;
+
+ list_for_each_entry(oap, plist, oap_pending_item) {
+ struct osc_page *opg = oap2osc_page(oap);
+
+ if (!opg->ops_in_lru)
+ continue;
+
+ ++npages;
+ LASSERT(list_empty(&opg->ops_lru));
+ list_add(&opg->ops_lru, &lru);
+ }
+
+ if (npages > 0) {
+ spin_lock(&cli->cl_lru_list_lock);
+ list_splice_tail(&lru, &cli->cl_lru_list);
+ atomic_sub(npages, &cli->cl_lru_busy);
+ atomic_add(npages, &cli->cl_lru_in_list);
+ spin_unlock(&cli->cl_lru_list_lock);
+
+ /* XXX: May set force to be true for better performance */
+ if (osc_cache_too_much(cli))
+ (void)ptlrpcd_queue_work(cli->cl_lru_work);
+ }
+}
+
+static void __osc_lru_del(struct client_obd *cli, struct osc_page *opg)
+{
+ LASSERT(atomic_read(&cli->cl_lru_in_list) > 0);
+ list_del_init(&opg->ops_lru);
+ atomic_dec(&cli->cl_lru_in_list);
+}
+
+/**
+ * Page is being destroyed. The page may be not in LRU list, if the transfer
+ * has never finished(error occurred).
+ */
+static void osc_lru_del(struct client_obd *cli, struct osc_page *opg)
+{
+ if (opg->ops_in_lru) {
+ spin_lock(&cli->cl_lru_list_lock);
+ if (!list_empty(&opg->ops_lru)) {
+ __osc_lru_del(cli, opg);
+ } else {
+ LASSERT(atomic_read(&cli->cl_lru_busy) > 0);
+ atomic_dec(&cli->cl_lru_busy);
+ }
+ spin_unlock(&cli->cl_lru_list_lock);
+
+ atomic_inc(cli->cl_lru_left);
+ /* this is a great place to release more LRU pages if
+ * this osc occupies too many LRU pages and kernel is
+ * stealing one of them.
+ */
+ if (!memory_pressure_get())
+ (void)ptlrpcd_queue_work(cli->cl_lru_work);
+ wake_up(&osc_lru_waitq);
+ } else {
+ LASSERT(list_empty(&opg->ops_lru));
+ }
+}
+
+/**
+ * Delete page from LRUlist for redirty.
+ */
+static void osc_lru_use(struct client_obd *cli, struct osc_page *opg)
+{
+ /* If page is being transferred for the first time,
+ * ops_lru should be empty
+ */
+ if (opg->ops_in_lru && !list_empty(&opg->ops_lru)) {
+ spin_lock(&cli->cl_lru_list_lock);
+ __osc_lru_del(cli, opg);
+ spin_unlock(&cli->cl_lru_list_lock);
+ atomic_inc(&cli->cl_lru_busy);
+ }
+}
+
+static void discard_pagevec(const struct lu_env *env, struct cl_io *io,
+ struct cl_page **pvec, int max_index)
+{
int i;
- for (count = 0, i = 0; i < max_index; i++) {
+ for (i = 0; i < max_index; i++) {
struct cl_page *page = pvec[i];
- if (cl_page_own_try(env, io, page) == 0) {
- /* free LRU page only if nobody is using it.
- * This check is necessary to avoid freeing the pages
- * having already been removed from LRU and pinned
- * for IO.
- */
- if (!cl_page_in_use(page)) {
- cl_page_unmap(env, io, page);
- cl_page_discard(env, io, page);
- ++count;
- }
- cl_page_disown(env, io, page);
- }
+ LASSERT(cl_page_is_owned(page, io));
+ cl_page_discard(env, io, page);
+ cl_page_disown(env, io, page);
cl_page_put(env, page);
+
pvec[i] = NULL;
}
- return max_index - count;
}
/**
* Drop @target of pages from LRU at most.
*/
-int osc_lru_shrink(struct client_obd *cli, int target)
+int osc_lru_shrink(const struct lu_env *env, struct client_obd *cli,
+ int target, bool force)
{
- struct cl_env_nest nest;
- struct lu_env *env;
struct cl_io *io;
struct cl_object *clobj = NULL;
struct cl_page **pvec;
@@ -573,23 +587,31 @@
if (atomic_read(&cli->cl_lru_in_list) == 0 || target <= 0)
return 0;
- env = cl_env_nested_get(&nest);
- if (IS_ERR(env))
- return PTR_ERR(env);
+ if (!force) {
+ if (atomic_read(&cli->cl_lru_shrinkers) > 0)
+ return -EBUSY;
- pvec = osc_env_info(env)->oti_pvec;
+ if (atomic_inc_return(&cli->cl_lru_shrinkers) > 1) {
+ atomic_dec(&cli->cl_lru_shrinkers);
+ return -EBUSY;
+ }
+ } else {
+ atomic_inc(&cli->cl_lru_shrinkers);
+ }
+
+ pvec = (struct cl_page **)osc_env_info(env)->oti_pvec;
io = &osc_env_info(env)->oti_io;
- client_obd_list_lock(&cli->cl_lru_list_lock);
- atomic_inc(&cli->cl_lru_shrinkers);
+ spin_lock(&cli->cl_lru_list_lock);
maxscan = min(target << 1, atomic_read(&cli->cl_lru_in_list));
list_for_each_entry_safe(opg, temp, &cli->cl_lru_list, ops_lru) {
struct cl_page *page;
+ bool will_free = false;
if (--maxscan < 0)
break;
- page = cl_page_top(opg->ops_cl.cpl_page);
+ page = opg->ops_cl.cpl_page;
if (cl_page_in_use_noref(page)) {
list_move_tail(&opg->ops_lru, &cli->cl_lru_list);
continue;
@@ -600,10 +622,10 @@
struct cl_object *tmp = page->cp_obj;
cl_object_get(tmp);
- client_obd_list_unlock(&cli->cl_lru_list_lock);
+ spin_unlock(&cli->cl_lru_list_lock);
if (clobj) {
- count -= discard_pagevec(env, io, pvec, index);
+ discard_pagevec(env, io, pvec, index);
index = 0;
cl_io_fini(env, io);
@@ -616,7 +638,7 @@
io->ci_ignore_layout = 1;
rc = cl_io_init(env, io, CIT_MISC, clobj);
- client_obd_list_lock(&cli->cl_lru_list_lock);
+ spin_lock(&cli->cl_lru_list_lock);
if (rc != 0)
break;
@@ -625,98 +647,54 @@
continue;
}
- /* move this page to the end of list as it will be discarded
- * soon. The page will be finally removed from LRU list in
- * osc_page_delete().
- */
- list_move_tail(&opg->ops_lru, &cli->cl_lru_list);
+ if (cl_page_own_try(env, io, page) == 0) {
+ if (!cl_page_in_use_noref(page)) {
+ /* remove it from lru list earlier to avoid
+ * lock contention
+ */
+ __osc_lru_del(cli, opg);
+ opg->ops_in_lru = 0; /* will be discarded */
- /* it's okay to grab a refcount here w/o holding lock because
- * it has to grab cl_lru_list_lock to delete the page.
- */
- cl_page_get(page);
+ cl_page_get(page);
+ will_free = true;
+ } else {
+ cl_page_disown(env, io, page);
+ }
+ }
+
+ if (!will_free) {
+ list_move_tail(&opg->ops_lru, &cli->cl_lru_list);
+ continue;
+ }
+
+ /* Don't discard and free the page with cl_lru_list held */
pvec[index++] = page;
- if (++count >= target)
- break;
-
if (unlikely(index == OTI_PVEC_SIZE)) {
- client_obd_list_unlock(&cli->cl_lru_list_lock);
- count -= discard_pagevec(env, io, pvec, index);
+ spin_unlock(&cli->cl_lru_list_lock);
+ discard_pagevec(env, io, pvec, index);
index = 0;
- client_obd_list_lock(&cli->cl_lru_list_lock);
+ spin_lock(&cli->cl_lru_list_lock);
}
+
+ if (++count >= target)
+ break;
}
- client_obd_list_unlock(&cli->cl_lru_list_lock);
+ spin_unlock(&cli->cl_lru_list_lock);
if (clobj) {
- count -= discard_pagevec(env, io, pvec, index);
+ discard_pagevec(env, io, pvec, index);
cl_io_fini(env, io);
cl_object_put(env, clobj);
}
- cl_env_nested_put(&nest, env);
atomic_dec(&cli->cl_lru_shrinkers);
- return count > 0 ? count : rc;
-}
-
-static void osc_lru_add(struct client_obd *cli, struct osc_page *opg)
-{
- bool wakeup = false;
-
- if (!opg->ops_in_lru)
- return;
-
- atomic_dec(&cli->cl_lru_busy);
- client_obd_list_lock(&cli->cl_lru_list_lock);
- if (list_empty(&opg->ops_lru)) {
- list_move_tail(&opg->ops_lru, &cli->cl_lru_list);
- atomic_inc_return(&cli->cl_lru_in_list);
- wakeup = atomic_read(&osc_lru_waiters) > 0;
- }
- client_obd_list_unlock(&cli->cl_lru_list_lock);
-
- if (wakeup) {
- osc_lru_shrink(cli, osc_cache_too_much(cli));
+ if (count > 0) {
+ atomic_add(count, cli->cl_lru_left);
wake_up_all(&osc_lru_waitq);
}
-}
-
-/* delete page from LRUlist. The page can be deleted from LRUlist for two
- * reasons: redirtied or deleted from page cache.
- */
-static void osc_lru_del(struct client_obd *cli, struct osc_page *opg, bool del)
-{
- if (opg->ops_in_lru) {
- client_obd_list_lock(&cli->cl_lru_list_lock);
- if (!list_empty(&opg->ops_lru)) {
- LASSERT(atomic_read(&cli->cl_lru_in_list) > 0);
- list_del_init(&opg->ops_lru);
- atomic_dec(&cli->cl_lru_in_list);
- if (!del)
- atomic_inc(&cli->cl_lru_busy);
- } else if (del) {
- LASSERT(atomic_read(&cli->cl_lru_busy) > 0);
- atomic_dec(&cli->cl_lru_busy);
- }
- client_obd_list_unlock(&cli->cl_lru_list_lock);
- if (del) {
- atomic_inc(cli->cl_lru_left);
- /* this is a great place to release more LRU pages if
- * this osc occupies too many LRU pages and kernel is
- * stealing one of them.
- * cl_lru_shrinkers is to avoid recursive call in case
- * we're already in the context of osc_lru_shrink().
- */
- if (atomic_read(&cli->cl_lru_shrinkers) == 0 &&
- !memory_pressure_get())
- osc_lru_shrink(cli, osc_cache_too_much(cli));
- wake_up(&osc_lru_waitq);
- }
- } else {
- LASSERT(list_empty(&opg->ops_lru));
- }
+ return count > 0 ? count : rc;
}
static inline int max_to_shrink(struct client_obd *cli)
@@ -724,19 +702,28 @@
return min(atomic_read(&cli->cl_lru_in_list) >> 1, lru_shrink_max);
}
-static int osc_lru_reclaim(struct client_obd *cli)
+int osc_lru_reclaim(struct client_obd *cli)
{
+ struct cl_env_nest nest;
+ struct lu_env *env;
struct cl_client_cache *cache = cli->cl_cache;
int max_scans;
- int rc;
+ int rc = 0;
LASSERT(cache);
- rc = osc_lru_shrink(cli, lru_shrink_min);
+ env = cl_env_nested_get(&nest);
+ if (IS_ERR(env))
+ return 0;
+
+ rc = osc_lru_shrink(env, cli, osc_cache_too_much(cli), false);
if (rc != 0) {
+ if (rc == -EBUSY)
+ rc = 0;
+
CDEBUG(D_CACHE, "%s: Free %d pages from own LRU: %p.\n",
cli->cl_import->imp_obd->obd_name, rc, cli);
- return rc;
+ goto out;
}
CDEBUG(D_CACHE, "%s: cli %p no free slots, pages: %d, busy: %d.\n",
@@ -764,10 +751,11 @@
atomic_read(&cli->cl_lru_busy));
list_move_tail(&cli->cl_lru_osc, &cache->ccc_lru);
- if (atomic_read(&cli->cl_lru_in_list) > 0) {
+ if (osc_cache_too_much(cli) > 0) {
spin_unlock(&cache->ccc_lru_lock);
- rc = osc_lru_shrink(cli, max_to_shrink(cli));
+ rc = osc_lru_shrink(env, cli, osc_cache_too_much(cli),
+ true);
spin_lock(&cache->ccc_lru_lock);
if (rc != 0)
break;
@@ -775,6 +763,8 @@
}
spin_unlock(&cache->ccc_lru_lock);
+out:
+ cl_env_nested_put(&nest, env);
CDEBUG(D_CACHE, "%s: cli %p freed %d pages.\n",
cli->cl_import->imp_obd->obd_name, cli, rc);
return rc;
@@ -784,16 +774,20 @@
struct osc_page *opg)
{
struct l_wait_info lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL);
+ struct osc_io *oio = osc_env_io(env);
struct client_obd *cli = osc_cli(obj);
int rc = 0;
if (!cli->cl_cache) /* shall not be in LRU */
return 0;
+ if (oio->oi_lru_reserved > 0) {
+ --oio->oi_lru_reserved;
+ goto out;
+ }
+
LASSERT(atomic_read(cli->cl_lru_left) >= 0);
while (!atomic_add_unless(cli->cl_lru_left, -1, 0)) {
- int gen;
-
/* run out of LRU spaces, try to drop some by itself */
rc = osc_lru_reclaim(cli);
if (rc < 0)
@@ -803,23 +797,15 @@
cond_resched();
- /* slowest case, all of caching pages are busy, notifying
- * other OSCs that we're lack of LRU slots.
- */
- atomic_inc(&osc_lru_waiters);
-
- gen = atomic_read(&cli->cl_lru_in_list);
rc = l_wait_event(osc_lru_waitq,
- atomic_read(cli->cl_lru_left) > 0 ||
- (atomic_read(&cli->cl_lru_in_list) > 0 &&
- gen != atomic_read(&cli->cl_lru_in_list)),
+ atomic_read(cli->cl_lru_left) > 0,
&lwi);
- atomic_dec(&osc_lru_waiters);
if (rc < 0)
break;
}
+out:
if (rc >= 0) {
atomic_inc(&cli->cl_lru_busy);
opg->ops_in_lru = 1;
diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c
index 30526eb..47417f8 100644
--- a/drivers/staging/lustre/lustre/osc/osc_request.c
+++ b/drivers/staging/lustre/lustre/osc/osc_request.c
@@ -92,12 +92,13 @@
struct osc_enqueue_args {
struct obd_export *oa_exp;
+ enum ldlm_type oa_type;
+ enum ldlm_mode oa_mode;
__u64 *oa_flags;
- obd_enqueue_update_f oa_upcall;
+ osc_enqueue_upcall_f oa_upcall;
void *oa_cookie;
struct ost_lvb *oa_lvb;
- struct lustre_handle *oa_lockh;
- struct ldlm_enqueue_info *oa_ei;
+ struct lustre_handle oa_lockh;
unsigned int oa_agl:1;
};
@@ -801,21 +802,24 @@
LASSERT(!(oa->o_valid & bits));
oa->o_valid |= bits;
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
oa->o_dirty = cli->cl_dirty;
if (unlikely(cli->cl_dirty - cli->cl_dirty_transit >
cli->cl_dirty_max)) {
CERROR("dirty %lu - %lu > dirty_max %lu\n",
cli->cl_dirty, cli->cl_dirty_transit, cli->cl_dirty_max);
oa->o_undirty = 0;
- } else if (unlikely(atomic_read(&obd_dirty_pages) -
+ } else if (unlikely(atomic_read(&obd_unstable_pages) +
+ atomic_read(&obd_dirty_pages) -
atomic_read(&obd_dirty_transit_pages) >
(long)(obd_max_dirty_pages + 1))) {
/* The atomic_read() allowing the atomic_inc() are
* not covered by a lock thus they may safely race and trip
* this CERROR() unless we add in a small fudge factor (+1).
*/
- CERROR("dirty %d - %d > system dirty_max %d\n",
+ CERROR("%s: dirty %d + %d - %d > system dirty_max %d\n",
+ cli->cl_import->imp_obd->obd_name,
+ atomic_read(&obd_unstable_pages),
atomic_read(&obd_dirty_pages),
atomic_read(&obd_dirty_transit_pages),
obd_max_dirty_pages);
@@ -833,10 +837,9 @@
oa->o_grant = cli->cl_avail_grant + cli->cl_reserved_grant;
oa->o_dropped = cli->cl_lost_grant;
cli->cl_lost_grant = 0;
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
CDEBUG(D_CACHE, "dirty: %llu undirty: %u dropped %u grant: %llu\n",
oa->o_dirty, oa->o_undirty, oa->o_dropped, oa->o_grant);
-
}
void osc_update_next_shrink(struct client_obd *cli)
@@ -849,9 +852,9 @@
static void __osc_update_grant(struct client_obd *cli, u64 grant)
{
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
cli->cl_avail_grant += grant;
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
}
static void osc_update_grant(struct client_obd *cli, struct ost_body *body)
@@ -889,10 +892,10 @@
static void osc_shrink_grant_local(struct client_obd *cli, struct obdo *oa)
{
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
oa->o_grant = cli->cl_avail_grant / 4;
cli->cl_avail_grant -= oa->o_grant;
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
if (!(oa->o_valid & OBD_MD_FLFLAGS)) {
oa->o_valid |= OBD_MD_FLFLAGS;
oa->o_flags = 0;
@@ -911,10 +914,10 @@
__u64 target_bytes = (cli->cl_max_rpcs_in_flight + 1) *
(cli->cl_max_pages_per_rpc << PAGE_SHIFT);
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
if (cli->cl_avail_grant <= target_bytes)
target_bytes = cli->cl_max_pages_per_rpc << PAGE_SHIFT;
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
return osc_shrink_grant_to_target(cli, target_bytes);
}
@@ -924,7 +927,7 @@
int rc = 0;
struct ost_body *body;
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
/* Don't shrink if we are already above or below the desired limit
* We don't want to shrink below a single RPC, as that will negatively
* impact block allocation and long-term performance.
@@ -933,10 +936,10 @@
target_bytes = cli->cl_max_pages_per_rpc << PAGE_SHIFT;
if (target_bytes >= cli->cl_avail_grant) {
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
return 0;
}
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
body = kzalloc(sizeof(*body), GFP_NOFS);
if (!body)
@@ -944,10 +947,10 @@
osc_announce_cached(cli, &body->oa, 0);
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
body->oa.o_grant = cli->cl_avail_grant - target_bytes;
cli->cl_avail_grant = target_bytes;
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
if (!(body->oa.o_valid & OBD_MD_FLFLAGS)) {
body->oa.o_valid |= OBD_MD_FLFLAGS;
body->oa.o_flags = 0;
@@ -1035,7 +1038,7 @@
* race is tolerable here: if we're evicted, but imp_state already
* left EVICTED state, then cl_dirty must be 0 already.
*/
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
if (cli->cl_import->imp_state == LUSTRE_IMP_EVICTED)
cli->cl_avail_grant = ocd->ocd_grant;
else
@@ -1053,7 +1056,7 @@
/* determine the appropriate chunk size used by osc_extent. */
cli->cl_chunkbits = max_t(int, PAGE_SHIFT, ocd->ocd_blocksize);
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
CDEBUG(D_CACHE, "%s, setting cl_avail_grant: %ld cl_lost_grant: %ld chunk bits: %d\n",
cli->cl_import->imp_obd->obd_name,
@@ -1082,7 +1085,7 @@
if (pga[i]->count > nob_read) {
/* EOF inside this page */
ptr = kmap(pga[i]->pg) +
- (pga[i]->off & ~CFS_PAGE_MASK);
+ (pga[i]->off & ~PAGE_MASK);
memset(ptr + nob_read, 0, pga[i]->count - nob_read);
kunmap(pga[i]->pg);
page_count--;
@@ -1097,7 +1100,7 @@
/* zero remaining pages */
while (page_count-- > 0) {
- ptr = kmap(pga[i]->pg) + (pga[i]->off & ~CFS_PAGE_MASK);
+ ptr = kmap(pga[i]->pg) + (pga[i]->off & ~PAGE_MASK);
memset(ptr, 0, pga[i]->count);
kunmap(pga[i]->pg);
i++;
@@ -1144,7 +1147,8 @@
{
if (p1->flag != p2->flag) {
unsigned mask = ~(OBD_BRW_FROM_GRANT | OBD_BRW_NOCACHE |
- OBD_BRW_SYNC | OBD_BRW_ASYNC|OBD_BRW_NOQUOTA);
+ OBD_BRW_SYNC | OBD_BRW_ASYNC |
+ OBD_BRW_NOQUOTA | OBD_BRW_SOFT_SYNC);
/* warn if we try to combine flags that we don't know to be
* safe to combine
@@ -1188,32 +1192,29 @@
if (i == 0 && opc == OST_READ &&
OBD_FAIL_CHECK(OBD_FAIL_OSC_CHECKSUM_RECEIVE)) {
unsigned char *ptr = kmap(pga[i]->pg);
- int off = pga[i]->off & ~CFS_PAGE_MASK;
+ int off = pga[i]->off & ~PAGE_MASK;
memcpy(ptr + off, "bad1", min(4, nob));
kunmap(pga[i]->pg);
}
cfs_crypto_hash_update_page(hdesc, pga[i]->pg,
- pga[i]->off & ~CFS_PAGE_MASK,
+ pga[i]->off & ~PAGE_MASK,
count);
CDEBUG(D_PAGE,
"page %p map %p index %lu flags %lx count %u priv %0lx: off %d\n",
pga[i]->pg, pga[i]->pg->mapping, pga[i]->pg->index,
(long)pga[i]->pg->flags, page_count(pga[i]->pg),
page_private(pga[i]->pg),
- (int)(pga[i]->off & ~CFS_PAGE_MASK));
+ (int)(pga[i]->off & ~PAGE_MASK));
nob -= pga[i]->count;
pg_count--;
i++;
}
- bufsize = 4;
+ bufsize = sizeof(cksum);
err = cfs_crypto_hash_final(hdesc, (unsigned char *)&cksum, &bufsize);
- if (err)
- cfs_crypto_hash_final(hdesc, NULL, NULL);
-
/* For sending we only compute the wrong checksum instead
* of corrupting the data so it is still correct on a redo
*/
@@ -1312,7 +1313,7 @@
pg_prev = pga[0];
for (requested_nob = i = 0; i < page_count; i++, niobuf++) {
struct brw_page *pg = pga[i];
- int poff = pg->off & ~CFS_PAGE_MASK;
+ int poff = pg->off & ~PAGE_MASK;
LASSERT(pg->count > 0);
/* make sure there is no gap in the middle of page array */
@@ -1658,6 +1659,7 @@
aa->aa_resends++;
new_req->rq_interpret_reply = request->rq_interpret_reply;
new_req->rq_async_args = request->rq_async_args;
+ new_req->rq_commit_cb = request->rq_commit_cb;
/* cap resend delay to the current request timeout, this is similar to
* what ptlrpc does (see after_reply())
*/
@@ -1737,7 +1739,6 @@
struct osc_brw_async_args *aa = data;
struct osc_extent *ext;
struct osc_extent *tmp;
- struct cl_object *obj = NULL;
struct client_obd *cli = aa->aa_cli;
rc = osc_brw_fini_request(req, rc);
@@ -1766,24 +1767,17 @@
rc = -EIO;
}
- list_for_each_entry_safe(ext, tmp, &aa->aa_exts, oe_link) {
- if (!obj && rc == 0) {
- obj = osc2cl(ext->oe_obj);
- cl_object_get(obj);
- }
-
- list_del_init(&ext->oe_link);
- osc_extent_finish(env, ext, 1, rc);
- }
- LASSERT(list_empty(&aa->aa_exts));
- LASSERT(list_empty(&aa->aa_oaps));
-
- if (obj) {
+ if (rc == 0) {
struct obdo *oa = aa->aa_oa;
struct cl_attr *attr = &osc_env_info(env)->oti_attr;
unsigned long valid = 0;
+ struct cl_object *obj;
+ struct osc_async_page *last;
- LASSERT(rc == 0);
+ last = brw_page2oap(aa->aa_ppga[aa->aa_page_count - 1]);
+ obj = osc2cl(last->oap_obj);
+
+ cl_object_attr_lock(obj);
if (oa->o_valid & OBD_MD_FLBLOCKS) {
attr->cat_blocks = oa->o_blocks;
valid |= CAT_BLOCKS;
@@ -1800,21 +1794,45 @@
attr->cat_ctime = oa->o_ctime;
valid |= CAT_CTIME;
}
- if (valid != 0) {
- cl_object_attr_lock(obj);
- cl_object_attr_set(env, obj, attr, valid);
- cl_object_attr_unlock(obj);
+
+ if (lustre_msg_get_opc(req->rq_reqmsg) == OST_WRITE) {
+ struct lov_oinfo *loi = cl2osc(obj)->oo_oinfo;
+ loff_t last_off = last->oap_count + last->oap_obj_off;
+
+ /* Change file size if this is an out of quota or
+ * direct IO write and it extends the file size
+ */
+ if (loi->loi_lvb.lvb_size < last_off) {
+ attr->cat_size = last_off;
+ valid |= CAT_SIZE;
+ }
+ /* Extend KMS if it's not a lockless write */
+ if (loi->loi_kms < last_off &&
+ oap2osc_page(last)->ops_srvlock == 0) {
+ attr->cat_kms = last_off;
+ valid |= CAT_KMS;
+ }
}
- cl_object_put(env, obj);
+
+ if (valid != 0)
+ cl_object_attr_set(env, obj, attr, valid);
+ cl_object_attr_unlock(obj);
}
kmem_cache_free(obdo_cachep, aa->aa_oa);
+ list_for_each_entry_safe(ext, tmp, &aa->aa_exts, oe_link) {
+ list_del_init(&ext->oe_link);
+ osc_extent_finish(env, ext, 1, rc);
+ }
+ LASSERT(list_empty(&aa->aa_exts));
+ LASSERT(list_empty(&aa->aa_oaps));
+
cl_req_completion(env, aa->aa_clerq, rc < 0 ? rc :
req->rq_bulk->bd_nob_transferred);
osc_release_ppga(aa->aa_ppga, aa->aa_page_count);
ptlrpc_lprocfs_brw(req, req->rq_bulk->bd_nob_transferred);
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
/* We need to decrement before osc_ap_completion->osc_wake_cache_waiters
* is called so we know whether to go to sync BRWs or wait for more
* RPCs to complete
@@ -1824,12 +1842,31 @@
else
cli->cl_r_in_flight--;
osc_wake_cache_waiters(cli);
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
osc_io_unplug(env, cli, NULL);
return rc;
}
+static void brw_commit(struct ptlrpc_request *req)
+{
+ spin_lock(&req->rq_lock);
+ /*
+ * If osc_inc_unstable_pages (via osc_extent_finish) races with
+ * this called via the rq_commit_cb, I need to ensure
+ * osc_dec_unstable_pages is still called. Otherwise unstable
+ * pages may be leaked.
+ */
+ if (req->rq_unstable) {
+ spin_unlock(&req->rq_lock);
+ osc_dec_unstable_pages(req);
+ spin_lock(&req->rq_lock);
+ } else {
+ req->rq_committed = 1;
+ }
+ spin_unlock(&req->rq_lock);
+}
+
/**
* Build an RPC by the list of extent @ext_list. The caller must ensure
* that the total pages in this list are NOT over max pages per RPC.
@@ -1920,7 +1957,7 @@
pga[i] = &oap->oap_brw_page;
pga[i]->off = oap->oap_obj_off + oap->oap_page_off;
CDEBUG(0, "put page %p index %lu oap %p flg %x to pga\n",
- pga[i]->pg, page_index(oap->oap_page), oap,
+ pga[i]->pg, oap->oap_page->index, oap,
pga[i]->flag);
i++;
cl_req_page_add(env, clerq, page);
@@ -1949,6 +1986,7 @@
goto out;
}
+ req->rq_commit_cb = brw_commit;
req->rq_interpret_reply = brw_interpret;
if (mem_tight != 0)
@@ -1992,7 +2030,7 @@
if (tmp)
tmp->oap_request = ptlrpc_request_addref(req);
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
starting_offset >>= PAGE_SHIFT;
if (cmd == OBD_BRW_READ) {
cli->cl_r_in_flight++;
@@ -2007,7 +2045,7 @@
lprocfs_oh_tally_log2(&cli->cl_write_offset_hist,
starting_offset + 1);
}
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
DEBUG_REQ(D_INODE, req, "%d pages, aa %p. now %dr/%dw in flight",
page_count, aa, cli->cl_r_in_flight,
@@ -2055,14 +2093,12 @@
LASSERT(lock->l_glimpse_ast == einfo->ei_cb_gl);
lock_res_and_lock(lock);
- spin_lock(&osc_ast_guard);
if (!lock->l_ast_data)
lock->l_ast_data = data;
if (lock->l_ast_data == data)
set = 1;
- spin_unlock(&osc_ast_guard);
unlock_res_and_lock(lock);
return set;
@@ -2104,36 +2140,38 @@
return rc;
}
-static int osc_enqueue_fini(struct ptlrpc_request *req, struct ost_lvb *lvb,
- obd_enqueue_update_f upcall, void *cookie,
- __u64 *flags, int agl, int rc)
+static int osc_enqueue_fini(struct ptlrpc_request *req,
+ osc_enqueue_upcall_f upcall, void *cookie,
+ struct lustre_handle *lockh, enum ldlm_mode mode,
+ __u64 *flags, int agl, int errcode)
{
- int intent = *flags & LDLM_FL_HAS_INTENT;
+ bool intent = *flags & LDLM_FL_HAS_INTENT;
+ int rc;
- if (intent) {
- /* The request was created before ldlm_cli_enqueue call. */
- if (rc == ELDLM_LOCK_ABORTED) {
- struct ldlm_reply *rep;
+ /* The request was created before ldlm_cli_enqueue call. */
+ if (intent && errcode == ELDLM_LOCK_ABORTED) {
+ struct ldlm_reply *rep;
- rep = req_capsule_server_get(&req->rq_pill,
- &RMF_DLM_REP);
+ rep = req_capsule_server_get(&req->rq_pill, &RMF_DLM_REP);
- rep->lock_policy_res1 =
- ptlrpc_status_ntoh(rep->lock_policy_res1);
- if (rep->lock_policy_res1)
- rc = rep->lock_policy_res1;
- }
- }
-
- if ((intent != 0 && rc == ELDLM_LOCK_ABORTED && agl == 0) ||
- (rc == 0)) {
+ rep->lock_policy_res1 =
+ ptlrpc_status_ntoh(rep->lock_policy_res1);
+ if (rep->lock_policy_res1)
+ errcode = rep->lock_policy_res1;
+ if (!agl)
+ *flags |= LDLM_FL_LVB_READY;
+ } else if (errcode == ELDLM_OK) {
*flags |= LDLM_FL_LVB_READY;
- CDEBUG(D_INODE, "got kms %llu blocks %llu mtime %llu\n",
- lvb->lvb_size, lvb->lvb_blocks, lvb->lvb_mtime);
}
/* Call the update callback. */
- rc = (*upcall)(cookie, rc);
+ rc = (*upcall)(cookie, lockh, errcode);
+ /* release the reference taken in ldlm_cli_enqueue() */
+ if (errcode == ELDLM_LOCK_MATCHED)
+ errcode = ELDLM_OK;
+ if (errcode == ELDLM_OK && lustre_handle_is_used(lockh))
+ ldlm_lock_decref(lockh, mode);
+
return rc;
}
@@ -2142,62 +2180,50 @@
struct osc_enqueue_args *aa, int rc)
{
struct ldlm_lock *lock;
- struct lustre_handle handle;
- __u32 mode;
- struct ost_lvb *lvb;
- __u32 lvb_len;
- __u64 *flags = aa->oa_flags;
+ struct lustre_handle *lockh = &aa->oa_lockh;
+ enum ldlm_mode mode = aa->oa_mode;
+ struct ost_lvb *lvb = aa->oa_lvb;
+ __u32 lvb_len = sizeof(*lvb);
+ __u64 flags = 0;
- /* Make a local copy of a lock handle and a mode, because aa->oa_*
- * might be freed anytime after lock upcall has been called.
- */
- lustre_handle_copy(&handle, aa->oa_lockh);
- mode = aa->oa_ei->ei_mode;
/* ldlm_cli_enqueue is holding a reference on the lock, so it must
* be valid.
*/
- lock = ldlm_handle2lock(&handle);
+ lock = ldlm_handle2lock(lockh);
+ LASSERTF(lock, "lockh %llx, req %p, aa %p - client evicted?\n",
+ lockh->cookie, req, aa);
/* Take an additional reference so that a blocking AST that
* ldlm_cli_enqueue_fini() might post for a failed lock, is guaranteed
* to arrive after an upcall has been executed by
* osc_enqueue_fini().
*/
- ldlm_lock_addref(&handle, mode);
+ ldlm_lock_addref(lockh, mode);
+
+ /* Let cl_lock_state_wait fail with -ERESTARTSYS to unuse sublocks. */
+ OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_ENQUEUE_HANG, 2);
/* Let CP AST to grant the lock first. */
OBD_FAIL_TIMEOUT(OBD_FAIL_OSC_CP_ENQ_RACE, 1);
- if (aa->oa_agl && rc == ELDLM_LOCK_ABORTED) {
- lvb = NULL;
- lvb_len = 0;
- } else {
- lvb = aa->oa_lvb;
- lvb_len = sizeof(*aa->oa_lvb);
+ if (aa->oa_agl) {
+ LASSERT(!aa->oa_lvb);
+ LASSERT(!aa->oa_flags);
+ aa->oa_flags = &flags;
}
/* Complete obtaining the lock procedure. */
- rc = ldlm_cli_enqueue_fini(aa->oa_exp, req, aa->oa_ei->ei_type, 1,
- mode, flags, lvb, lvb_len, &handle, rc);
+ rc = ldlm_cli_enqueue_fini(aa->oa_exp, req, aa->oa_type, 1,
+ aa->oa_mode, aa->oa_flags, lvb, lvb_len,
+ lockh, rc);
/* Complete osc stuff. */
- rc = osc_enqueue_fini(req, aa->oa_lvb, aa->oa_upcall, aa->oa_cookie,
- flags, aa->oa_agl, rc);
+ rc = osc_enqueue_fini(req, aa->oa_upcall, aa->oa_cookie, lockh, mode,
+ aa->oa_flags, aa->oa_agl, rc);
OBD_FAIL_TIMEOUT(OBD_FAIL_OSC_CP_CANCEL_RACE, 10);
- /* Release the lock for async request. */
- if (lustre_handle_is_used(&handle) && rc == ELDLM_OK)
- /*
- * Releases a reference taken by ldlm_cli_enqueue(), if it is
- * not already released by
- * ldlm_cli_enqueue_fini()->failed_lock_cleanup()
- */
- ldlm_lock_decref(&handle, mode);
-
- LASSERTF(lock, "lockh %p, req %p, aa %p - client evicted?\n",
- aa->oa_lockh, req, aa);
- ldlm_lock_decref(&handle, mode);
+ ldlm_lock_decref(lockh, mode);
LDLM_LOCK_PUT(lock);
return rc;
}
@@ -2209,29 +2235,29 @@
* other synchronous requests, however keeping some locks and trying to obtain
* others may take a considerable amount of time in a case of ost failure; and
* when other sync requests do not get released lock from a client, the client
- * is excluded from the cluster -- such scenarious make the life difficult, so
+ * is evicted from the cluster -- such scenaries make the life difficult, so
* release locks just after they are obtained.
*/
int osc_enqueue_base(struct obd_export *exp, struct ldlm_res_id *res_id,
__u64 *flags, ldlm_policy_data_t *policy,
struct ost_lvb *lvb, int kms_valid,
- obd_enqueue_update_f upcall, void *cookie,
+ osc_enqueue_upcall_f upcall, void *cookie,
struct ldlm_enqueue_info *einfo,
- struct lustre_handle *lockh,
struct ptlrpc_request_set *rqset, int async, int agl)
{
struct obd_device *obd = exp->exp_obd;
+ struct lustre_handle lockh = { 0 };
struct ptlrpc_request *req = NULL;
int intent = *flags & LDLM_FL_HAS_INTENT;
- __u64 match_lvb = (agl != 0 ? 0 : LDLM_FL_LVB_READY);
+ __u64 match_lvb = agl ? 0 : LDLM_FL_LVB_READY;
enum ldlm_mode mode;
int rc;
/* Filesystem lock extents are extended to page boundaries so that
* dealing with the page cache is a little smoother.
*/
- policy->l_extent.start -= policy->l_extent.start & ~CFS_PAGE_MASK;
- policy->l_extent.end |= ~CFS_PAGE_MASK;
+ policy->l_extent.start -= policy->l_extent.start & ~PAGE_MASK;
+ policy->l_extent.end |= ~PAGE_MASK;
/*
* kms is not valid when either object is completely fresh (so that no
@@ -2259,64 +2285,46 @@
if (einfo->ei_mode == LCK_PR)
mode |= LCK_PW;
mode = ldlm_lock_match(obd->obd_namespace, *flags | match_lvb, res_id,
- einfo->ei_type, policy, mode, lockh, 0);
+ einfo->ei_type, policy, mode, &lockh, 0);
if (mode) {
- struct ldlm_lock *matched = ldlm_handle2lock(lockh);
+ struct ldlm_lock *matched;
- if ((agl != 0) && !(matched->l_flags & LDLM_FL_LVB_READY)) {
- /* For AGL, if enqueue RPC is sent but the lock is not
- * granted, then skip to process this strpe.
- * Return -ECANCELED to tell the caller.
+ if (*flags & LDLM_FL_TEST_LOCK)
+ return ELDLM_OK;
+
+ matched = ldlm_handle2lock(&lockh);
+ if (agl) {
+ /* AGL enqueues DLM locks speculatively. Therefore if
+ * it already exists a DLM lock, it wll just inform the
+ * caller to cancel the AGL process for this stripe.
*/
- ldlm_lock_decref(lockh, mode);
+ ldlm_lock_decref(&lockh, mode);
LDLM_LOCK_PUT(matched);
return -ECANCELED;
- }
-
- if (osc_set_lock_data_with_check(matched, einfo)) {
+ } else if (osc_set_lock_data_with_check(matched, einfo)) {
*flags |= LDLM_FL_LVB_READY;
- /* addref the lock only if not async requests and PW
- * lock is matched whereas we asked for PR.
- */
- if (!rqset && einfo->ei_mode != mode)
- ldlm_lock_addref(lockh, LCK_PR);
- if (intent) {
- /* I would like to be able to ASSERT here that
- * rss <= kms, but I can't, for reasons which
- * are explained in lov_enqueue()
- */
- }
+ /* We already have a lock, and it's referenced. */
+ (*upcall)(cookie, &lockh, ELDLM_LOCK_MATCHED);
- /* We already have a lock, and it's referenced.
- *
- * At this point, the cl_lock::cll_state is CLS_QUEUING,
- * AGL upcall may change it to CLS_HELD directly.
- */
- (*upcall)(cookie, ELDLM_OK);
-
- if (einfo->ei_mode != mode)
- ldlm_lock_decref(lockh, LCK_PW);
- else if (rqset)
- /* For async requests, decref the lock. */
- ldlm_lock_decref(lockh, einfo->ei_mode);
+ ldlm_lock_decref(&lockh, mode);
LDLM_LOCK_PUT(matched);
return ELDLM_OK;
+ } else {
+ ldlm_lock_decref(&lockh, mode);
+ LDLM_LOCK_PUT(matched);
}
-
- ldlm_lock_decref(lockh, mode);
- LDLM_LOCK_PUT(matched);
}
- no_match:
+no_match:
+ if (*flags & LDLM_FL_TEST_LOCK)
+ return -ENOLCK;
if (intent) {
- LIST_HEAD(cancels);
-
req = ptlrpc_request_alloc(class_exp2cliimp(exp),
&RQF_LDLM_ENQUEUE_LVB);
if (!req)
return -ENOMEM;
- rc = ldlm_prep_enqueue_req(exp, req, &cancels, 0);
+ rc = ldlm_prep_enqueue_req(exp, req, NULL, 0);
if (rc) {
ptlrpc_request_free(req);
return rc;
@@ -2331,21 +2339,31 @@
*flags &= ~LDLM_FL_BLOCK_GRANTED;
rc = ldlm_cli_enqueue(exp, &req, einfo, res_id, policy, flags, lvb,
- sizeof(*lvb), LVB_T_OST, lockh, async);
- if (rqset) {
+ sizeof(*lvb), LVB_T_OST, &lockh, async);
+ if (async) {
if (!rc) {
struct osc_enqueue_args *aa;
- CLASSERT (sizeof(*aa) <= sizeof(req->rq_async_args));
+ CLASSERT(sizeof(*aa) <= sizeof(req->rq_async_args));
aa = ptlrpc_req_async_args(req);
- aa->oa_ei = einfo;
aa->oa_exp = exp;
- aa->oa_flags = flags;
+ aa->oa_mode = einfo->ei_mode;
+ aa->oa_type = einfo->ei_type;
+ lustre_handle_copy(&aa->oa_lockh, &lockh);
aa->oa_upcall = upcall;
aa->oa_cookie = cookie;
- aa->oa_lvb = lvb;
- aa->oa_lockh = lockh;
aa->oa_agl = !!agl;
+ if (!agl) {
+ aa->oa_flags = flags;
+ aa->oa_lvb = lvb;
+ } else {
+ /* AGL is essentially to enqueue an DLM lock
+ * in advance, so we don't care about the
+ * result of AGL enqueue.
+ */
+ aa->oa_lvb = NULL;
+ aa->oa_flags = NULL;
+ }
req->rq_interpret_reply =
(ptlrpc_interpterer_t)osc_enqueue_interpret;
@@ -2359,7 +2377,8 @@
return rc;
}
- rc = osc_enqueue_fini(req, lvb, upcall, cookie, flags, agl, rc);
+ rc = osc_enqueue_fini(req, upcall, cookie, &lockh, einfo->ei_mode,
+ flags, agl, rc);
if (intent)
ptlrpc_req_finished(req);
@@ -2381,8 +2400,8 @@
/* Filesystem lock extents are extended to page boundaries so that
* dealing with the page cache is a little smoother
*/
- policy->l_extent.start -= policy->l_extent.start & ~CFS_PAGE_MASK;
- policy->l_extent.end |= ~CFS_PAGE_MASK;
+ policy->l_extent.start -= policy->l_extent.start & ~PAGE_MASK;
+ policy->l_extent.end |= ~PAGE_MASK;
/* Next, search for already existing extent locks that will cover us */
/* If we're trying to read, we also search for an existing PW lock. The
@@ -2493,7 +2512,7 @@
}
req->rq_interpret_reply = (ptlrpc_interpterer_t)osc_statfs_interpret;
- CLASSERT (sizeof(*aa) <= sizeof(req->rq_async_args));
+ CLASSERT(sizeof(*aa) <= sizeof(req->rq_async_args));
aa = ptlrpc_req_async_args(req);
aa->aa_oi = oinfo;
@@ -2787,7 +2806,7 @@
goto skip_locking;
policy.l_extent.start = fm_key->fiemap.fm_start &
- CFS_PAGE_MASK;
+ PAGE_MASK;
if (OBD_OBJECT_EOF - fm_key->fiemap.fm_length <=
fm_key->fiemap.fm_start + PAGE_SIZE - 1)
@@ -2795,7 +2814,7 @@
else
policy.l_extent.end = (fm_key->fiemap.fm_start +
fm_key->fiemap.fm_length +
- PAGE_SIZE - 1) & CFS_PAGE_MASK;
+ PAGE_SIZE - 1) & PAGE_MASK;
ostid_build_res_name(&fm_key->oa.o_oi, &res_id);
mode = ldlm_lock_match(exp->exp_obd->obd_namespace,
@@ -2913,7 +2932,7 @@
int nr = atomic_read(&cli->cl_lru_in_list) >> 1;
int target = *(int *)val;
- nr = osc_lru_shrink(cli, min(nr, target));
+ nr = osc_lru_shrink(env, cli, min(nr, target), true);
*(int *)val -= nr;
return 0;
}
@@ -2992,12 +3011,12 @@
if (data && (data->ocd_connect_flags & OBD_CONNECT_GRANT)) {
long lost_grant;
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
data->ocd_grant = (cli->cl_avail_grant + cli->cl_dirty) ?:
2 * cli_brw_size(obd);
lost_grant = cli->cl_lost_grant;
cli->cl_lost_grant = 0;
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
CDEBUG(D_RPCTRACE, "ocd_connect_flags: %#llx ocd_version: %d ocd_grant: %d, lost: %ld.\n",
data->ocd_connect_flags,
@@ -3047,10 +3066,10 @@
switch (event) {
case IMP_EVENT_DISCON: {
cli = &obd->u.cli;
- client_obd_list_lock(&cli->cl_loi_list_lock);
+ spin_lock(&cli->cl_loi_list_lock);
cli->cl_avail_grant = 0;
cli->cl_lost_grant = 0;
- client_obd_list_unlock(&cli->cl_loi_list_lock);
+ spin_unlock(&cli->cl_loi_list_lock);
break;
}
case IMP_EVENT_INACTIVE: {
@@ -3073,8 +3092,9 @@
ldlm_namespace_cleanup(ns, LDLM_FL_LOCAL_ONLY);
cl_env_put(env, &refcheck);
- } else
+ } else {
rc = PTR_ERR(env);
+ }
break;
}
case IMP_EVENT_ACTIVE: {
@@ -3116,20 +3136,14 @@
* \retval zero the lock can't be canceled
* \retval other ok to cancel
*/
-static int osc_cancel_for_recovery(struct ldlm_lock *lock)
+static int osc_cancel_weight(struct ldlm_lock *lock)
{
- check_res_locked(lock->l_resource);
-
/*
- * Cancel all unused extent lock in granted mode LCK_PR or LCK_CR.
- *
- * XXX as a future improvement, we can also cancel unused write lock
- * if it doesn't have dirty data and active mmaps.
+ * Cancel all unused and granted extent lock.
*/
if (lock->l_resource->lr_type == LDLM_EXTENT &&
- (lock->l_granted_mode == LCK_PR ||
- lock->l_granted_mode == LCK_CR) &&
- (osc_dlm_lock_pageref(lock) == 0))
+ lock->l_granted_mode == lock->l_req_mode &&
+ osc_ldlm_weigh_ast(lock) == 0)
return 1;
return 0;
@@ -3170,6 +3184,14 @@
}
cli->cl_writeback_work = handler;
+ handler = ptlrpcd_alloc_work(cli->cl_import, lru_queue_work, cli);
+ if (IS_ERR(handler)) {
+ rc = PTR_ERR(handler);
+ goto out_ptlrpcd_work;
+ }
+
+ cli->cl_lru_work = handler;
+
rc = osc_quota_setup(obd);
if (rc)
goto out_ptlrpcd_work;
@@ -3198,11 +3220,18 @@
}
INIT_LIST_HEAD(&cli->cl_grant_shrink_list);
- ns_register_cancel(obd->obd_namespace, osc_cancel_for_recovery);
+ ns_register_cancel(obd->obd_namespace, osc_cancel_weight);
return rc;
out_ptlrpcd_work:
- ptlrpcd_destroy_work(handler);
+ if (cli->cl_writeback_work) {
+ ptlrpcd_destroy_work(cli->cl_writeback_work);
+ cli->cl_writeback_work = NULL;
+ }
+ if (cli->cl_lru_work) {
+ ptlrpcd_destroy_work(cli->cl_lru_work);
+ cli->cl_lru_work = NULL;
+ }
out_client_setup:
client_obd_cleanup(obd);
out_ptlrpcd:
@@ -3241,6 +3270,10 @@
ptlrpcd_destroy_work(cli->cl_writeback_work);
cli->cl_writeback_work = NULL;
}
+ if (cli->cl_lru_work) {
+ ptlrpcd_destroy_work(cli->cl_lru_work);
+ cli->cl_lru_work = NULL;
+ }
obd_cleanup_client_import(obd);
ptlrpc_lprocfs_unregister_obd(obd);
lprocfs_obd_cleanup(obd);
@@ -3330,7 +3363,6 @@
};
extern struct lu_kmem_descr osc_caches[];
-extern spinlock_t osc_ast_guard;
extern struct lock_class_key osc_ast_guard_class;
static int __init osc_init(void)
@@ -3357,9 +3389,6 @@
if (rc)
goto out_kmem;
- spin_lock_init(&osc_ast_guard);
- lockdep_set_class(&osc_ast_guard, &osc_ast_guard_class);
-
/* This is obviously too much memory, only prevent overflow here */
if (osc_reqpool_mem_max >= 1 << 12 || osc_reqpool_mem_max == 0) {
rc = -EINVAL;
diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c
index cf3ac8e..4b7912a 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/client.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/client.c
@@ -595,9 +595,9 @@
struct obd_import *imp = request->rq_import;
int rc;
- if (unlikely(ctx))
+ if (unlikely(ctx)) {
request->rq_cli_ctx = sptlrpc_cli_ctx_get(ctx);
- else {
+ } else {
rc = sptlrpc_req_get_ctx(request);
if (rc)
goto out_free;
@@ -1082,7 +1082,6 @@
*/
if ((lustre_handle_is_used(&req->rq_import->imp_remote_handle)) &&
(opc == OST_CONNECT || opc == MDS_CONNECT || opc == MGS_CONNECT)) {
-
/* Suppress timed out reconnect requests */
if (req->rq_timedout)
return 0;
@@ -2087,7 +2086,7 @@
CDEBUG(D_RPCTRACE, "set %p going to sleep for %d seconds\n",
set, timeout);
- if (timeout == 0 && !cfs_signal_pending())
+ if (timeout == 0 && !signal_pending(current))
/*
* No requests are in-flight (ether timed out
* or delayed), so we can allow interrupts.
@@ -2114,7 +2113,7 @@
* it being ignored forever
*/
if (rc == -ETIMEDOUT && !lwi.lwi_allow_intr &&
- cfs_signal_pending()) {
+ signal_pending(current)) {
sigset_t blocked_sigs =
cfs_block_sigsinv(LUSTRE_FATAL_SIGS);
@@ -2124,7 +2123,7 @@
* important signals since ptlrpc set is not easily
* reentrant from userspace again
*/
- if (cfs_signal_pending())
+ if (signal_pending(current))
ptlrpc_interrupted_set(set);
cfs_restore_sigs(blocked_sigs);
}
diff --git a/drivers/staging/lustre/lustre/ptlrpc/events.c b/drivers/staging/lustre/lustre/ptlrpc/events.c
index 47be21a..fdcde9b 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/events.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/events.c
@@ -69,7 +69,6 @@
req->rq_req_unlink = 0;
if (ev->type == LNET_EVENT_UNLINK || ev->status != 0) {
-
/* Failed send: make it seem like the reply timed out, just
* like failing sends in client.c does currently...
*/
diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c b/drivers/staging/lustre/lustre/ptlrpc/import.c
index cd94fed..a4f7544 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/import.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/import.c
@@ -1001,6 +1001,7 @@
return 0;
}
} else {
+ static bool warned;
spin_lock(&imp->imp_lock);
list_del(&imp->imp_conn_current->oic_item);
@@ -1021,7 +1022,7 @@
goto out;
}
- if ((ocd->ocd_connect_flags & OBD_CONNECT_VERSION) &&
+ if (!warned && (ocd->ocd_connect_flags & OBD_CONNECT_VERSION) &&
(ocd->ocd_version > LUSTRE_VERSION_CODE +
LUSTRE_VERSION_OFFSET_WARN ||
ocd->ocd_version < LUSTRE_VERSION_CODE -
@@ -1029,10 +1030,8 @@
/* Sigh, some compilers do not like #ifdef in the middle
* of macro arguments
*/
- const char *older = "older. Consider upgrading server or downgrading client"
- ;
- const char *newer = "newer than client version. Consider upgrading client"
- ;
+ const char *older = "older than client. Consider upgrading server";
+ const char *newer = "newer than client. Consider recompiling application";
LCONSOLE_WARN("Server %s version (%d.%d.%d.%d) is much %s (%s)\n",
obd2cli_tgt(imp->imp_obd),
@@ -1042,6 +1041,7 @@
OBD_OCD_VERSION_FIX(ocd->ocd_version),
ocd->ocd_version > LUSTRE_VERSION_CODE ?
newer : older, LUSTRE_VERSION_STRING);
+ warned = true;
}
#if LUSTRE_VERSION_CODE < OBD_OCD_VERSION(3, 2, 50, 0)
@@ -1370,7 +1370,6 @@
if (rc)
goto out;
}
-
}
if (imp->imp_state == LUSTRE_IMP_REPLAY_WAIT) {
@@ -1453,7 +1452,6 @@
back_to_sleep, LWI_ON_SIGNAL_NOOP, NULL);
rc = l_wait_event(imp->imp_recovery_waitq,
!ptlrpc_import_in_recovery(imp), &lwi);
-
}
spin_lock(&imp->imp_lock);
diff --git a/drivers/staging/lustre/lustre/ptlrpc/layout.c b/drivers/staging/lustre/lustre/ptlrpc/layout.c
index 5b06901..c0ecd16 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/layout.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/layout.c
@@ -160,6 +160,16 @@
&RMF_FLD_MDFLD
};
+static const struct req_msg_field *fld_read_client[] = {
+ &RMF_PTLRPC_BODY,
+ &RMF_FLD_MDFLD
+};
+
+static const struct req_msg_field *fld_read_server[] = {
+ &RMF_PTLRPC_BODY,
+ &RMF_GENERIC_DATA
+};
+
static const struct req_msg_field *mds_getattr_name_client[] = {
&RMF_PTLRPC_BODY,
&RMF_MDT_BODY,
@@ -566,7 +576,7 @@
static const struct req_msg_field *ost_get_info_generic_client[] = {
&RMF_PTLRPC_BODY,
- &RMF_SETINFO_KEY
+ &RMF_GETINFO_KEY
};
static const struct req_msg_field *ost_get_last_id_server[] = {
@@ -574,6 +584,12 @@
&RMF_OBD_ID
};
+static const struct req_msg_field *ost_get_last_fid_client[] = {
+ &RMF_PTLRPC_BODY,
+ &RMF_GETINFO_KEY,
+ &RMF_FID,
+};
+
static const struct req_msg_field *ost_get_last_fid_server[] = {
&RMF_PTLRPC_BODY,
&RMF_FID,
@@ -643,6 +659,7 @@
&RQF_MGS_CONFIG_READ,
&RQF_SEQ_QUERY,
&RQF_FLD_QUERY,
+ &RQF_FLD_READ,
&RQF_MDS_CONNECT,
&RQF_MDS_DISCONNECT,
&RQF_MDS_GET_INFO,
@@ -696,7 +713,7 @@
&RQF_OST_BRW_WRITE,
&RQF_OST_STATFS,
&RQF_OST_SET_GRANT_INFO,
- &RQF_OST_GET_INFO_GENERIC,
+ &RQF_OST_GET_INFO,
&RQF_OST_GET_INFO_LAST_ID,
&RQF_OST_GET_INFO_LAST_FID,
&RQF_OST_SET_INFO_LAST_FID,
@@ -1162,6 +1179,10 @@
DEFINE_REQ_FMT0("FLD_QUERY", fld_query_client, fld_query_server);
EXPORT_SYMBOL(RQF_FLD_QUERY);
+struct req_format RQF_FLD_READ =
+ DEFINE_REQ_FMT0("FLD_READ", fld_read_client, fld_read_server);
+EXPORT_SYMBOL(RQF_FLD_READ);
+
struct req_format RQF_LOG_CANCEL =
DEFINE_REQ_FMT0("OBD_LOG_CANCEL", log_cancel_client, empty);
EXPORT_SYMBOL(RQF_LOG_CANCEL);
@@ -1519,10 +1540,10 @@
ost_body_only);
EXPORT_SYMBOL(RQF_OST_SET_GRANT_INFO);
-struct req_format RQF_OST_GET_INFO_GENERIC =
+struct req_format RQF_OST_GET_INFO =
DEFINE_REQ_FMT0("OST_GET_INFO", ost_get_info_generic_client,
ost_get_info_generic_server);
-EXPORT_SYMBOL(RQF_OST_GET_INFO_GENERIC);
+EXPORT_SYMBOL(RQF_OST_GET_INFO);
struct req_format RQF_OST_GET_INFO_LAST_ID =
DEFINE_REQ_FMT0("OST_GET_INFO_LAST_ID", ost_get_info_generic_client,
@@ -1530,7 +1551,7 @@
EXPORT_SYMBOL(RQF_OST_GET_INFO_LAST_ID);
struct req_format RQF_OST_GET_INFO_LAST_FID =
- DEFINE_REQ_FMT0("OST_GET_INFO_LAST_FID", obd_set_info_client,
+ DEFINE_REQ_FMT0("OST_GET_INFO_LAST_FID", ost_get_last_fid_client,
ost_get_last_fid_server);
EXPORT_SYMBOL(RQF_OST_GET_INFO_LAST_FID);
diff --git a/drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c b/drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c
index c95a91c..64c0f1e 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/lproc_ptlrpc.c
@@ -131,6 +131,7 @@
{ SEC_CTX_INIT_CONT, "sec_ctx_init_cont" },
{ SEC_CTX_FINI, "sec_ctx_fini" },
{ FLD_QUERY, "fld_query" },
+ { FLD_READ, "fld_read" },
};
static struct ll_eopcode {
@@ -679,11 +680,11 @@
/**
* The second token is either NULL, or an optional [reg|hp] string
*/
- if (strcmp(cmd, "reg") == 0)
+ if (strcmp(cmd, "reg") == 0) {
queue = PTLRPC_NRS_QUEUE_REG;
- else if (strcmp(cmd, "hp") == 0)
+ } else if (strcmp(cmd, "hp") == 0) {
queue = PTLRPC_NRS_QUEUE_HP;
- else {
+ } else {
rc = -EINVAL;
goto out;
}
@@ -693,8 +694,9 @@
if (queue == PTLRPC_NRS_QUEUE_HP && !nrs_svc_has_hp(svc)) {
rc = -ENODEV;
goto out;
- } else if (queue == PTLRPC_NRS_QUEUE_BOTH && !nrs_svc_has_hp(svc))
+ } else if (queue == PTLRPC_NRS_QUEUE_BOTH && !nrs_svc_has_hp(svc)) {
queue = PTLRPC_NRS_QUEUE_REG;
+ }
/**
* Serialize NRS core lprocfs operations with policy registration/
@@ -1320,6 +1322,5 @@
up_read(&obd->u.cli.cl_sem);
return count;
-
}
EXPORT_SYMBOL(lprocfs_wr_pinger_recov);
diff --git a/drivers/staging/lustre/lustre/ptlrpc/nrs.c b/drivers/staging/lustre/lustre/ptlrpc/nrs.c
index 710fb80..c444f51 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/nrs.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/nrs.c
@@ -975,7 +975,11 @@
LASSERT(mutex_is_locked(&nrs_core.nrs_mutex));
again:
- nrs = nrs_svcpt2nrs(svcpt, hp);
+ /* scp_nrs_hp could be NULL due to short of memory. */
+ nrs = hp ? svcpt->scp_nrs_hp : &svcpt->scp_nrs_reg;
+ /* check the nrs_svcpt to see if nrs is initialized. */
+ if (!nrs || !nrs->nrs_svcpt)
+ return;
nrs->nrs_stopping = 1;
list_for_each_entry_safe(policy, tmp, &nrs->nrs_policy_list, pol_list) {
@@ -1038,7 +1042,6 @@
LASSERT(mutex_is_locked(&ptlrpc_all_services_mutex));
list_for_each_entry(svc, &ptlrpc_all_services, srv_list) {
-
if (!nrs_policy_compatible(svc, desc) ||
unlikely(svc->srv_is_stopping))
continue;
diff --git a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c
index 492d63f..811acf6 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c
@@ -1160,7 +1160,6 @@
if (!pb) {
CERROR("invalid msg %p: no ptlrpc body!\n", msg);
return 0;
-
}
return pb->pb_timeout;
}
@@ -1179,7 +1178,6 @@
if (!pb) {
CERROR("invalid msg %p: no ptlrpc body!\n", msg);
return 0;
-
}
return pb->pb_service_time;
}
@@ -1572,7 +1570,6 @@
CLASSERT(offsetof(typeof(*o), o_padding_4) != 0);
CLASSERT(offsetof(typeof(*o), o_padding_5) != 0);
CLASSERT(offsetof(typeof(*o), o_padding_6) != 0);
-
}
void lustre_swab_obd_statfs(struct obd_statfs *os)
diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c
index db003f5..76a355a 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c
@@ -387,7 +387,8 @@
{
struct ptlrpcd_ctl *pc = arg;
struct ptlrpc_request_set *set;
- struct lu_env env = { .le_ses = NULL };
+ struct lu_context ses = { 0 };
+ struct lu_env env = { .le_ses = &ses };
int rc = 0;
int exit = 0;
@@ -416,6 +417,13 @@
*/
rc = lu_context_init(&env.le_ctx,
LCT_CL_THREAD|LCT_REMEMBER|LCT_NOREF);
+ if (rc == 0) {
+ rc = lu_context_init(env.le_ses,
+ LCT_SESSION | LCT_REMEMBER | LCT_NOREF);
+ if (rc != 0)
+ lu_context_fini(&env.le_ctx);
+ }
+
if (rc != 0)
goto failed;
@@ -436,9 +444,10 @@
ptlrpc_expired_set, set);
lu_context_enter(&env.le_ctx);
- l_wait_event(set->set_waitq,
- ptlrpcd_check(&env, pc), &lwi);
+ lu_context_enter(env.le_ses);
+ l_wait_event(set->set_waitq, ptlrpcd_check(&env, pc), &lwi);
lu_context_exit(&env.le_ctx);
+ lu_context_exit(env.le_ses);
/*
* Abort inflight rpcs for forced stop case.
@@ -461,6 +470,7 @@
if (!list_empty(&set->set_requests))
ptlrpc_set_wait(set);
lu_context_fini(&env.le_ctx);
+ lu_context_fini(env.le_ses);
complete(&pc->pc_finishing);
@@ -899,8 +909,11 @@
int rc = 0;
mutex_lock(&ptlrpcd_mutex);
- if (++ptlrpcd_users == 1)
+ if (++ptlrpcd_users == 1) {
rc = ptlrpcd_init();
+ if (rc < 0)
+ ptlrpcd_users--;
+ }
mutex_unlock(&ptlrpcd_mutex);
return rc;
}
diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c b/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c
index d3872b8..02e6cda 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/sec_bulk.c
@@ -41,7 +41,6 @@
#define DEBUG_SUBSYSTEM S_SEC
#include "../../include/linux/libcfs/libcfs.h"
-#include <linux/crypto.h>
#include "../include/obd.h"
#include "../include/obd_cksum.h"
@@ -511,7 +510,6 @@
{
struct cfs_crypto_hash_desc *hdesc;
int hashsize;
- char hashbuf[64];
unsigned int bufsize;
int i, err;
@@ -529,21 +527,23 @@
for (i = 0; i < desc->bd_iov_count; i++) {
cfs_crypto_hash_update_page(hdesc, desc->bd_iov[i].kiov_page,
- desc->bd_iov[i].kiov_offset & ~CFS_PAGE_MASK,
+ desc->bd_iov[i].kiov_offset & ~PAGE_MASK,
desc->bd_iov[i].kiov_len);
}
+
if (hashsize > buflen) {
+ unsigned char hashbuf[CFS_CRYPTO_HASH_DIGESTSIZE_MAX];
+
bufsize = sizeof(hashbuf);
- err = cfs_crypto_hash_final(hdesc, (unsigned char *)hashbuf,
- &bufsize);
+ LASSERTF(bufsize >= hashsize, "bufsize = %u < hashsize %u\n",
+ bufsize, hashsize);
+ err = cfs_crypto_hash_final(hdesc, hashbuf, &bufsize);
memcpy(buf, hashbuf, buflen);
} else {
bufsize = buflen;
err = cfs_crypto_hash_final(hdesc, buf, &bufsize);
}
- if (err)
- cfs_crypto_hash_final(hdesc, NULL, NULL);
return err;
}
EXPORT_SYMBOL(sptlrpc_get_bulk_checksum);
diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_plain.c b/drivers/staging/lustre/lustre/ptlrpc/sec_plain.c
index 6276bf5..37c9f4c 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/sec_plain.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/sec_plain.c
@@ -162,7 +162,7 @@
continue;
ptr = kmap(desc->bd_iov[i].kiov_page);
- off = desc->bd_iov[i].kiov_offset & ~CFS_PAGE_MASK;
+ off = desc->bd_iov[i].kiov_offset & ~PAGE_MASK;
ptr[off] ^= 0x1;
kunmap(desc->bd_iov[i].kiov_page);
return;
diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c
index 1bbd1d3..17c7b97 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/service.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/service.c
@@ -838,6 +838,11 @@
{
ptlrpc_server_hpreq_fini(req);
+ if (req->rq_session.lc_thread) {
+ lu_context_exit(&req->rq_session);
+ lu_context_fini(&req->rq_session);
+ }
+
ptlrpc_server_drop_request(req);
}
@@ -1579,6 +1584,21 @@
}
req->rq_svc_thread = thread;
+ if (thread) {
+ /* initialize request session, it is needed for request
+ * processing by target
+ */
+ rc = lu_context_init(&req->rq_session,
+ LCT_SERVER_SESSION | LCT_NOREF);
+ if (rc) {
+ CERROR("%s: failure to initialize session: rc = %d\n",
+ thread->t_name, rc);
+ goto err_req;
+ }
+ req->rq_session.lc_thread = thread;
+ lu_context_enter(&req->rq_session);
+ req->rq_svc_thread->t_env->le_ses = &req->rq_session;
+ }
ptlrpc_at_add_timed(req);
@@ -1612,7 +1632,6 @@
struct timespec64 arrived;
unsigned long timediff_usecs;
unsigned long arrived_usecs;
- int rc;
int fail_opc = 0;
request = ptlrpc_server_request_get(svcpt, false);
@@ -1649,21 +1668,6 @@
at_get(&svcpt->scp_at_estimate));
}
- rc = lu_context_init(&request->rq_session, LCT_SESSION | LCT_NOREF);
- if (rc) {
- CERROR("Failure to initialize session: %d\n", rc);
- goto out_req;
- }
- request->rq_session.lc_thread = thread;
- request->rq_session.lc_cookie = 0x5;
- lu_context_enter(&request->rq_session);
-
- CDEBUG(D_NET, "got req %llu\n", request->rq_xid);
-
- request->rq_svc_thread = thread;
- if (thread)
- request->rq_svc_thread->t_env->le_ses = &request->rq_session;
-
if (likely(request->rq_export)) {
if (unlikely(ptlrpc_check_req(request)))
goto put_conn;
@@ -1695,14 +1699,21 @@
if (lustre_msg_get_opc(request->rq_reqmsg) != OBD_PING)
CFS_FAIL_TIMEOUT_MS(OBD_FAIL_PTLRPC_PAUSE_REQ, cfs_fail_val);
- rc = svc->srv_ops.so_req_handler(request);
+ CDEBUG(D_NET, "got req %llu\n", request->rq_xid);
+
+ /* re-assign request and sesson thread to the current one */
+ request->rq_svc_thread = thread;
+ if (thread) {
+ LASSERT(request->rq_session.lc_thread);
+ request->rq_session.lc_thread = thread;
+ request->rq_session.lc_cookie = 0x55;
+ thread->t_env->le_ses = &request->rq_session;
+ }
+ svc->srv_ops.so_req_handler(request);
ptlrpc_rqphase_move(request, RQ_PHASE_COMPLETE);
put_conn:
- lu_context_exit(&request->rq_session);
- lu_context_fini(&request->rq_session);
-
if (unlikely(ktime_get_real_seconds() > request->rq_deadline)) {
DEBUG_REQ(D_WARNING, request,
"Request took longer than estimated (%lld:%llds); "
@@ -1756,7 +1767,6 @@
request->rq_arrival_time.tv_sec);
}
-out_req:
ptlrpc_server_finish_active_request(svcpt, request);
return 1;
diff --git a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
index 3ffd2d9..aacc810 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c
@@ -276,7 +276,9 @@
(long long)FLD_QUERY);
LASSERTF(FLD_FIRST_OPC == 900, "found %lld\n",
(long long)FLD_FIRST_OPC);
- LASSERTF(FLD_LAST_OPC == 901, "found %lld\n",
+ LASSERTF(FLD_READ == 901, "found %lld\n",
+ (long long)FLD_READ);
+ LASSERTF(FLD_LAST_OPC == 902, "found %lld\n",
(long long)FLD_LAST_OPC);
LASSERTF(SEQ_QUERY == 700, "found %lld\n",
(long long)SEQ_QUERY);
@@ -1069,6 +1071,8 @@
OBD_CONNECT_PINGLESS);
LASSERTF(OBD_CONNECT_FLOCK_DEAD == 0x8000000000000ULL,
"found 0x%.16llxULL\n", OBD_CONNECT_FLOCK_DEAD);
+ LASSERTF(OBD_CONNECT_OPEN_BY_FID == 0x20000000000000ULL,
+ "found 0x%.16llxULL\n", OBD_CONNECT_OPEN_BY_FID);
LASSERTF(OBD_CKSUM_CRC32 == 0x00000001UL, "found 0x%.8xUL\n",
(unsigned)OBD_CKSUM_CRC32);
LASSERTF(OBD_CKSUM_ADLER == 0x00000002UL, "found 0x%.8xUL\n",
@@ -1639,6 +1643,12 @@
OBD_BRW_ASYNC);
LASSERTF(OBD_BRW_MEMALLOC == 0x800, "found 0x%.8x\n",
OBD_BRW_MEMALLOC);
+ LASSERTF(OBD_BRW_OVER_USRQUOTA == 0x1000, "found 0x%.8x\n",
+ OBD_BRW_OVER_USRQUOTA);
+ LASSERTF(OBD_BRW_OVER_GRPQUOTA == 0x2000, "found 0x%.8x\n",
+ OBD_BRW_OVER_GRPQUOTA);
+ LASSERTF(OBD_BRW_SOFT_SYNC == 0x4000, "found 0x%.8x\n",
+ OBD_BRW_SOFT_SYNC);
/* Checks for struct ost_body */
LASSERTF((int)sizeof(struct ost_body) == 208, "found %lld\n",
diff --git a/drivers/staging/media/Kconfig b/drivers/staging/media/Kconfig
index 0078b6a..de7e9f5 100644
--- a/drivers/staging/media/Kconfig
+++ b/drivers/staging/media/Kconfig
@@ -37,6 +37,8 @@
source "drivers/staging/media/timb/Kconfig"
+source "drivers/staging/media/tw686x-kh/Kconfig"
+
# Keep LIRC at the end, as it has sub-menus
source "drivers/staging/media/lirc/Kconfig"
diff --git a/drivers/staging/media/Makefile b/drivers/staging/media/Makefile
index 9149588..60a35b3 100644
--- a/drivers/staging/media/Makefile
+++ b/drivers/staging/media/Makefile
@@ -8,3 +8,4 @@
obj-$(CONFIG_VIDEO_OMAP4) += omap4iss/
obj-$(CONFIG_DVB_MN88472) += mn88472/
obj-$(CONFIG_VIDEO_TIMBERDALE) += timb/
+obj-$(CONFIG_VIDEO_TW686X_KH) += tw686x-kh/
diff --git a/drivers/staging/media/bcm2048/radio-bcm2048.c b/drivers/staging/media/bcm2048/radio-bcm2048.c
index abf330f..8dade19 100644
--- a/drivers/staging/media/bcm2048/radio-bcm2048.c
+++ b/drivers/staging/media/bcm2048/radio-bcm2048.c
@@ -308,7 +308,7 @@
MODULE_PARM_DESC(radio_nr,
"Minor number for radio device (-1 ==> auto assign)");
-static struct region_info region_configs[] = {
+static const struct region_info region_configs[] = {
/* USA */
{
.channel_spacing = 20,
diff --git a/drivers/staging/media/davinci_vpfe/vpfe_video.c b/drivers/staging/media/davinci_vpfe/vpfe_video.c
index be72a8e..ea3ddec 100644
--- a/drivers/staging/media/davinci_vpfe/vpfe_video.c
+++ b/drivers/staging/media/davinci_vpfe/vpfe_video.c
@@ -154,7 +154,7 @@
while ((entity = media_entity_graph_walk_next(&graph))) {
if (entity == &video->video_dev.entity)
continue;
- if (!is_media_entity_v4l2_io(entity))
+ if (!is_media_entity_v4l2_video_device(entity))
continue;
far_end = to_vpfe_video(media_entity_to_video_device(entity));
if (far_end->type == V4L2_BUF_TYPE_VIDEO_OUTPUT)
diff --git a/drivers/staging/media/omap1/omap1_camera.c b/drivers/staging/media/omap1/omap1_camera.c
index bd721e3..54b8dd2 100644
--- a/drivers/staging/media/omap1/omap1_camera.c
+++ b/drivers/staging/media/omap1/omap1_camera.c
@@ -1569,27 +1569,21 @@
unsigned int irq;
int err = 0;
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
irq = platform_get_irq(pdev, 0);
- if (!res || (int)irq <= 0) {
+ if ((int)irq <= 0) {
err = -ENODEV;
goto exit;
}
- clk = clk_get(&pdev->dev, "armper_ck");
- if (IS_ERR(clk)) {
- err = PTR_ERR(clk);
- goto exit;
- }
+ clk = devm_clk_get(&pdev->dev, "armper_ck");
+ if (IS_ERR(clk))
+ return PTR_ERR(clk);
- pcdev = kzalloc(sizeof(*pcdev) + resource_size(res), GFP_KERNEL);
- if (!pcdev) {
- dev_err(&pdev->dev, "Could not allocate pcdev\n");
- err = -ENOMEM;
- goto exit_put_clk;
- }
+ pcdev = devm_kzalloc(&pdev->dev, sizeof(*pcdev) + resource_size(res),
+ GFP_KERNEL);
+ if (!pcdev)
+ return -ENOMEM;
- pcdev->res = res;
pcdev->clk = clk;
pcdev->pdata = pdev->dev.platform_data;
@@ -1620,19 +1614,11 @@
INIT_LIST_HEAD(&pcdev->capture);
spin_lock_init(&pcdev->lock);
- /*
- * Request the region.
- */
- if (!request_mem_region(res->start, resource_size(res), DRIVER_NAME)) {
- err = -EBUSY;
- goto exit_kfree;
- }
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(base))
+ return PTR_ERR(base);
- base = ioremap(res->start, resource_size(res));
- if (!base) {
- err = -ENOMEM;
- goto exit_release;
- }
pcdev->irq = irq;
pcdev->base = base;
@@ -1642,8 +1628,7 @@
dma_isr, (void *)pcdev, &pcdev->dma_ch);
if (err < 0) {
dev_err(&pdev->dev, "Can't request DMA for OMAP1 Camera\n");
- err = -EBUSY;
- goto exit_iounmap;
+ return -EBUSY;
}
dev_dbg(&pdev->dev, "got DMA channel %d\n", pcdev->dma_ch);
@@ -1655,7 +1640,8 @@
/* setup DMA autoinitialization */
omap_dma_link_lch(pcdev->dma_ch, pcdev->dma_ch);
- err = request_irq(pcdev->irq, cam_isr, 0, DRIVER_NAME, pcdev);
+ err = devm_request_irq(&pdev->dev, pcdev->irq, cam_isr, 0, DRIVER_NAME,
+ pcdev);
if (err) {
dev_err(&pdev->dev, "Camera interrupt register failed\n");
goto exit_free_dma;
@@ -1669,24 +1655,14 @@
err = soc_camera_host_register(&pcdev->soc_host);
if (err)
- goto exit_free_irq;
+ return err;
dev_info(&pdev->dev, "OMAP1 Camera Interface driver loaded\n");
return 0;
-exit_free_irq:
- free_irq(pcdev->irq, pcdev);
exit_free_dma:
omap_free_dma(pcdev->dma_ch);
-exit_iounmap:
- iounmap(base);
-exit_release:
- release_mem_region(res->start, resource_size(res));
-exit_kfree:
- kfree(pcdev);
-exit_put_clk:
- clk_put(clk);
exit:
return err;
}
@@ -1696,23 +1672,11 @@
struct soc_camera_host *soc_host = to_soc_camera_host(&pdev->dev);
struct omap1_cam_dev *pcdev = container_of(soc_host,
struct omap1_cam_dev, soc_host);
- struct resource *res;
-
- free_irq(pcdev->irq, pcdev);
omap_free_dma(pcdev->dma_ch);
soc_camera_host_unregister(soc_host);
- iounmap(pcdev->base);
-
- res = pcdev->res;
- release_mem_region(res->start, resource_size(res));
-
- clk_put(pcdev->clk);
-
- kfree(pcdev);
-
dev_info(&pdev->dev, "OMAP1 Camera Interface driver unloaded\n");
return 0;
diff --git a/drivers/staging/media/omap4iss/iss.c b/drivers/staging/media/omap4iss/iss.c
index c5a5138..6ceb4eb 100644
--- a/drivers/staging/media/omap4iss/iss.c
+++ b/drivers/staging/media/omap4iss/iss.c
@@ -1065,7 +1065,7 @@
}
ret = media_create_pad_link(&sensor->entity, 0, input, pad,
- flags);
+ flags);
if (ret < 0)
goto done;
}
diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
index f54349b..cf8da23 100644
--- a/drivers/staging/media/omap4iss/iss_video.c
+++ b/drivers/staging/media/omap4iss/iss_video.c
@@ -223,7 +223,7 @@
if (entity == &video->video.entity)
continue;
- if (!is_media_entity_v4l2_io(entity))
+ if (!is_media_entity_v4l2_video_device(entity))
continue;
far_end = to_iss_video(media_entity_to_video_device(entity));
diff --git a/drivers/staging/media/tw686x-kh/Kconfig b/drivers/staging/media/tw686x-kh/Kconfig
new file mode 100644
index 0000000..6264d30
--- /dev/null
+++ b/drivers/staging/media/tw686x-kh/Kconfig
@@ -0,0 +1,17 @@
+config VIDEO_TW686X_KH
+ tristate "Intersil/Techwell TW686x Video For Linux"
+ depends on VIDEO_DEV && PCI && VIDEO_V4L2
+ depends on !(VIDEO_TW686X=y || VIDEO_TW686X=m) || COMPILE_TEST
+ select VIDEOBUF2_DMA_SG
+ help
+ Support for Intersil/Techwell TW686x-based frame grabber cards.
+
+ Currently supported chips:
+ - TW6864 (4 video channels),
+ - TW6865 (4 video channels, not tested, second generation chip),
+ - TW6868 (8 video channels but only 4 first channels using
+ built-in video decoder are supported, not tested),
+ - TW6869 (8 video channels, second generation chip).
+
+ To compile this driver as a module, choose M here: the module
+ will be named tw686x-kh.
diff --git a/drivers/staging/media/tw686x-kh/Makefile b/drivers/staging/media/tw686x-kh/Makefile
new file mode 100644
index 0000000..2a36a38
--- /dev/null
+++ b/drivers/staging/media/tw686x-kh/Makefile
@@ -0,0 +1,3 @@
+tw686x-kh-objs := tw686x-kh-core.o tw686x-kh-video.o
+
+obj-$(CONFIG_VIDEO_TW686X_KH) += tw686x-kh.o
diff --git a/drivers/staging/media/tw686x-kh/TODO b/drivers/staging/media/tw686x-kh/TODO
new file mode 100644
index 0000000..480a495
--- /dev/null
+++ b/drivers/staging/media/tw686x-kh/TODO
@@ -0,0 +1,6 @@
+TODO:
+
+- implement V4L2_FIELD_INTERLACED* mode(s).
+- add audio support
+
+Please Cc: patches to Krzysztof Halasa <khalasa@piap.pl>.
diff --git a/drivers/staging/media/tw686x-kh/tw686x-kh-core.c b/drivers/staging/media/tw686x-kh/tw686x-kh-core.c
new file mode 100644
index 0000000..03b3b62
--- /dev/null
+++ b/drivers/staging/media/tw686x-kh/tw686x-kh-core.c
@@ -0,0 +1,140 @@
+/*
+ * Copyright (C) 2015 Industrial Research Institute for Automation
+ * and Measurements PIAP
+ *
+ * Written by Krzysztof Ha?asa.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of version 2 of the GNU General Public License
+ * as published by the Free Software Foundation.
+ */
+
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include "tw686x-kh.h"
+#include "tw686x-kh-regs.h"
+
+static irqreturn_t tw686x_irq(int irq, void *dev_id)
+{
+ struct tw686x_dev *dev = (struct tw686x_dev *)dev_id;
+ u32 int_status = reg_read(dev, INT_STATUS); /* cleared on read */
+ unsigned long flags;
+ unsigned int handled = 0;
+
+ if (int_status) {
+ spin_lock_irqsave(&dev->irq_lock, flags);
+ dev->dma_requests |= int_status;
+ spin_unlock_irqrestore(&dev->irq_lock, flags);
+
+ if (int_status & 0xFF0000FF)
+ handled = tw686x_kh_video_irq(dev);
+ }
+
+ return IRQ_RETVAL(handled);
+}
+
+static int tw686x_probe(struct pci_dev *pci_dev,
+ const struct pci_device_id *pci_id)
+{
+ struct tw686x_dev *dev;
+ int err;
+
+ dev = devm_kzalloc(&pci_dev->dev, sizeof(*dev) +
+ (pci_id->driver_data & TYPE_MAX_CHANNELS) *
+ sizeof(dev->video_channels[0]), GFP_KERNEL);
+ if (!dev)
+ return -ENOMEM;
+
+ sprintf(dev->name, "TW%04X", pci_dev->device);
+ dev->type = pci_id->driver_data;
+
+ pr_info("%s: PCI %s, IRQ %d, MMIO 0x%lx\n", dev->name,
+ pci_name(pci_dev), pci_dev->irq,
+ (unsigned long)pci_resource_start(pci_dev, 0));
+
+ dev->pci_dev = pci_dev;
+ if (pcim_enable_device(pci_dev))
+ return -EIO;
+
+ pci_set_master(pci_dev);
+
+ if (pci_set_dma_mask(pci_dev, DMA_BIT_MASK(32))) {
+ pr_err("%s: 32-bit PCI DMA not supported\n", dev->name);
+ return -EIO;
+ }
+
+ err = pci_request_regions(pci_dev, dev->name);
+ if (err < 0) {
+ pr_err("%s: Unable to get MMIO region\n", dev->name);
+ return err;
+ }
+
+ dev->mmio = pci_ioremap_bar(pci_dev, 0);
+ if (!dev->mmio) {
+ pr_err("%s: Unable to remap MMIO region\n", dev->name);
+ return -EIO;
+ }
+
+ reg_write(dev, SYS_SOFT_RST, 0x0F); /* Reset all subsystems */
+ mdelay(1);
+
+ reg_write(dev, SRST[0], 0x3F);
+ if (max_channels(dev) > 4)
+ reg_write(dev, SRST[1], 0x3F);
+ reg_write(dev, DMA_CMD, 0);
+ reg_write(dev, DMA_CHANNEL_ENABLE, 0);
+ reg_write(dev, DMA_CHANNEL_TIMEOUT, 0x3EFF0FF0);
+ reg_write(dev, DMA_TIMER_INTERVAL, 0x38000);
+ reg_write(dev, DMA_CONFIG, 0xFFFFFF04);
+
+ spin_lock_init(&dev->irq_lock);
+
+ err = devm_request_irq(&pci_dev->dev, pci_dev->irq, tw686x_irq,
+ IRQF_SHARED, dev->name, dev);
+ if (err < 0) {
+ pr_err("%s: Unable to get IRQ\n", dev->name);
+ return err;
+ }
+
+ err = tw686x_kh_video_init(dev);
+ if (err)
+ return err;
+
+ pci_set_drvdata(pci_dev, dev);
+ return 0;
+}
+
+static void tw686x_remove(struct pci_dev *pci_dev)
+{
+ struct tw686x_dev *dev = pci_get_drvdata(pci_dev);
+
+ tw686x_kh_video_free(dev);
+}
+
+/* driver_data is number of A/V channels */
+static const struct pci_device_id tw686x_pci_tbl[] = {
+ {PCI_DEVICE(0x1797, 0x6864), .driver_data = 4},
+ /* not tested */
+ {PCI_DEVICE(0x1797, 0x6865), .driver_data = 4 | TYPE_SECOND_GEN},
+ /* TW6868 supports 8 A/V channels with an external TW2865 chip -
+ not supported by the driver */
+ {PCI_DEVICE(0x1797, 0x6868), .driver_data = 4}, /* not tested */
+ {PCI_DEVICE(0x1797, 0x6869), .driver_data = 8 | TYPE_SECOND_GEN},
+ {}
+};
+
+static struct pci_driver tw686x_pci_driver = {
+ .name = "tw686x-kh",
+ .id_table = tw686x_pci_tbl,
+ .probe = tw686x_probe,
+ .remove = tw686x_remove,
+};
+
+MODULE_DESCRIPTION("Driver for video frame grabber cards based on Intersil/Techwell TW686[4589]");
+MODULE_AUTHOR("Krzysztof Halasa");
+MODULE_LICENSE("GPL v2");
+MODULE_DEVICE_TABLE(pci, tw686x_pci_tbl);
+module_pci_driver(tw686x_pci_driver);
diff --git a/drivers/staging/media/tw686x-kh/tw686x-kh-regs.h b/drivers/staging/media/tw686x-kh/tw686x-kh-regs.h
new file mode 100644
index 0000000..53e1889
--- /dev/null
+++ b/drivers/staging/media/tw686x-kh/tw686x-kh-regs.h
@@ -0,0 +1,103 @@
+/* DMA controller registers */
+#define REG8_1(a0) ((const u16[8]) {a0, a0 + 1, a0 + 2, a0 + 3, \
+ a0 + 4, a0 + 5, a0 + 6, a0 + 7})
+#define REG8_2(a0) ((const u16[8]) {a0, a0 + 2, a0 + 4, a0 + 6, \
+ a0 + 8, a0 + 0xA, a0 + 0xC, a0 + 0xE})
+#define REG8_8(a0) ((const u16[8]) {a0, a0 + 8, a0 + 0x10, a0 + 0x18, \
+ a0 + 0x20, a0 + 0x28, a0 + 0x30, a0 + 0x38})
+#define INT_STATUS 0x00
+#define PB_STATUS 0x01
+#define DMA_CMD 0x02
+#define VIDEO_FIFO_STATUS 0x03
+#define VIDEO_CHANNEL_ID 0x04
+#define VIDEO_PARSER_STATUS 0x05
+#define SYS_SOFT_RST 0x06
+#define DMA_PAGE_TABLE0_ADDR ((const u16[8]) {0x08, 0xD0, 0xD2, 0xD4, \
+ 0xD6, 0xD8, 0xDA, 0xDC})
+#define DMA_PAGE_TABLE1_ADDR ((const u16[8]) {0x09, 0xD1, 0xD3, 0xD5, \
+ 0xD7, 0xD9, 0xDB, 0xDD})
+#define DMA_CHANNEL_ENABLE 0x0A
+#define DMA_CONFIG 0x0B
+#define DMA_TIMER_INTERVAL 0x0C
+#define DMA_CHANNEL_TIMEOUT 0x0D
+#define VDMA_CHANNEL_CONFIG REG8_1(0x10)
+#define ADMA_P_ADDR REG8_2(0x18)
+#define ADMA_B_ADDR REG8_2(0x19)
+#define DMA10_P_ADDR 0x28 /* ??? */
+#define DMA10_B_ADDR 0x29
+#define VIDEO_CONTROL1 0x2A
+#define VIDEO_CONTROL2 0x2B
+#define AUDIO_CONTROL1 0x2C
+#define AUDIO_CONTROL2 0x2D
+#define PHASE_REF 0x2E
+#define GPIO_REG 0x2F
+#define INTL_HBAR_CTRL REG8_1(0x30)
+#define AUDIO_CONTROL3 0x38
+#define VIDEO_FIELD_CTRL REG8_1(0x39)
+#define HSCALER_CTRL REG8_1(0x42)
+#define VIDEO_SIZE REG8_1(0x4A)
+#define VIDEO_SIZE_F2 REG8_1(0x52)
+#define MD_CONF REG8_1(0x60)
+#define MD_INIT REG8_1(0x68)
+#define MD_MAP0 REG8_1(0x70)
+#define VDMA_P_ADDR REG8_8(0x80) /* not used in DMA SG mode */
+#define VDMA_WHP REG8_8(0x81)
+#define VDMA_B_ADDR REG8_8(0x82)
+#define VDMA_F2_P_ADDR REG8_8(0x84)
+#define VDMA_F2_WHP REG8_8(0x85)
+#define VDMA_F2_B_ADDR REG8_8(0x86)
+#define EP_REG_ADDR 0xFE
+#define EP_REG_DATA 0xFF
+
+/* Video decoder registers */
+#define VDREG8(a0) ((const u16[8]) { \
+ a0 + 0x000, a0 + 0x010, a0 + 0x020, a0 + 0x030, \
+ a0 + 0x100, a0 + 0x110, a0 + 0x120, a0 + 0x130})
+#define VIDSTAT VDREG8(0x100)
+#define BRIGHT VDREG8(0x101)
+#define CONTRAST VDREG8(0x102)
+#define SHARPNESS VDREG8(0x103)
+#define SAT_U VDREG8(0x104)
+#define SAT_V VDREG8(0x105)
+#define HUE VDREG8(0x106)
+#define CROP_HI VDREG8(0x107)
+#define VDELAY_LO VDREG8(0x108)
+#define VACTIVE_LO VDREG8(0x109)
+#define HDELAY_LO VDREG8(0x10A)
+#define HACTIVE_LO VDREG8(0x10B)
+#define MVSN VDREG8(0x10C)
+#define STATUS2 VDREG8(0x10C)
+#define SDT VDREG8(0x10E)
+#define SDT_EN VDREG8(0x10F)
+
+#define VSCALE_LO VDREG8(0x144)
+#define SCALE_HI VDREG8(0x145)
+#define HSCALE_LO VDREG8(0x146)
+#define F2CROP_HI VDREG8(0x147)
+#define F2VDELAY_LO VDREG8(0x148)
+#define F2VACTIVE_LO VDREG8(0x149)
+#define F2HDELAY_LO VDREG8(0x14A)
+#define F2HACTIVE_LO VDREG8(0x14B)
+#define F2VSCALE_LO VDREG8(0x14C)
+#define F2SCALE_HI VDREG8(0x14D)
+#define F2HSCALE_LO VDREG8(0x14E)
+#define F2CNT VDREG8(0x14F)
+
+#define VDREG2(a0) ((const u16[2]) {a0, a0 + 0x100})
+#define SRST VDREG2(0x180)
+#define ACNTL VDREG2(0x181)
+#define ACNTL2 VDREG2(0x182)
+#define CNTRL1 VDREG2(0x183)
+#define CKHY VDREG2(0x184)
+#define SHCOR VDREG2(0x185)
+#define CORING VDREG2(0x186)
+#define CLMPG VDREG2(0x187)
+#define IAGC VDREG2(0x188)
+#define VCTRL1 VDREG2(0x18F)
+#define MISC1 VDREG2(0x194)
+#define LOOP VDREG2(0x195)
+#define MISC2 VDREG2(0x196)
+
+#define CLMD VDREG2(0x197)
+#define AIGAIN ((const u16[8]) {0x1D0, 0x1D1, 0x1D2, 0x1D3, \
+ 0x2D0, 0x2D1, 0x2D2, 0x2D3})
diff --git a/drivers/staging/media/tw686x-kh/tw686x-kh-video.c b/drivers/staging/media/tw686x-kh/tw686x-kh-video.c
new file mode 100644
index 0000000..6ecb504
--- /dev/null
+++ b/drivers/staging/media/tw686x-kh/tw686x-kh-video.c
@@ -0,0 +1,821 @@
+/*
+ * Copyright (C) 2015 Industrial Research Institute for Automation
+ * and Measurements PIAP
+ *
+ * Written by Krzysztof Ha?asa.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of version 2 of the GNU General Public License
+ * as published by the Free Software Foundation.
+ */
+
+#include <linux/init.h>
+#include <linux/list.h>
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+#include <media/v4l2-common.h>
+#include <media/v4l2-event.h>
+#include "tw686x-kh.h"
+#include "tw686x-kh-regs.h"
+
+#define MAX_SG_ENTRY_SIZE (/* 8192 - 128 */ 4096)
+#define MAX_SG_DESC_COUNT 256 /* PAL 704x576 needs up to 198 4-KB pages */
+
+static const struct tw686x_format formats[] = {
+ {
+ .name = "4:2:2 packed, UYVY", /* aka Y422 */
+ .fourcc = V4L2_PIX_FMT_UYVY,
+ .mode = 0,
+ .depth = 16,
+ }, {
+#if 0
+ .name = "4:2:0 packed, YUV",
+ .mode = 1, /* non-standard */
+ .depth = 12,
+ }, {
+ .name = "4:1:1 packed, YUV",
+ .mode = 2, /* non-standard */
+ .depth = 12,
+ }, {
+#endif
+ .name = "4:1:1 packed, YUV",
+ .fourcc = V4L2_PIX_FMT_Y41P,
+ .mode = 3,
+ .depth = 12,
+ }, {
+ .name = "15 bpp RGB",
+ .fourcc = V4L2_PIX_FMT_RGB555,
+ .mode = 4,
+ .depth = 16,
+ }, {
+ .name = "16 bpp RGB",
+ .fourcc = V4L2_PIX_FMT_RGB565,
+ .mode = 5,
+ .depth = 16,
+ }, {
+ .name = "4:2:2 packed, YUYV",
+ .fourcc = V4L2_PIX_FMT_YUYV,
+ .mode = 6,
+ .depth = 16,
+ }
+ /* mode 7 is "reserved" */
+};
+
+static const v4l2_std_id video_standards[7] = {
+ V4L2_STD_NTSC,
+ V4L2_STD_PAL,
+ V4L2_STD_SECAM,
+ V4L2_STD_NTSC_443,
+ V4L2_STD_PAL_M,
+ V4L2_STD_PAL_N,
+ V4L2_STD_PAL_60,
+};
+
+static const struct tw686x_format *format_by_fourcc(unsigned int fourcc)
+{
+ unsigned int cnt;
+
+ for (cnt = 0; cnt < ARRAY_SIZE(formats); cnt++)
+ if (formats[cnt].fourcc == fourcc)
+ return &formats[cnt];
+ return NULL;
+}
+
+static void tw686x_get_format(struct tw686x_video_channel *vc,
+ struct v4l2_format *f)
+{
+ const struct tw686x_format *format;
+ unsigned int width, height, height_div = 1;
+
+ format = format_by_fourcc(f->fmt.pix.pixelformat);
+ if (!format) {
+ format = &formats[0];
+ f->fmt.pix.pixelformat = format->fourcc;
+ }
+
+ width = 704;
+ if (f->fmt.pix.width < width * 3 / 4 /* halfway */)
+ width /= 2;
+
+ height = (vc->video_standard & V4L2_STD_625_50) ? 576 : 480;
+ if (f->fmt.pix.height < height * 3 / 4 /* halfway */)
+ height_div = 2;
+
+ switch (f->fmt.pix.field) {
+ case V4L2_FIELD_TOP:
+ case V4L2_FIELD_BOTTOM:
+ height_div = 2;
+ break;
+ case V4L2_FIELD_SEQ_BT:
+ if (height_div > 1)
+ f->fmt.pix.field = V4L2_FIELD_BOTTOM;
+ break;
+ default:
+ if (height_div > 1)
+ f->fmt.pix.field = V4L2_FIELD_TOP;
+ else
+ f->fmt.pix.field = V4L2_FIELD_SEQ_TB;
+ }
+ height /= height_div;
+
+ f->fmt.pix.width = width;
+ f->fmt.pix.height = height;
+ f->fmt.pix.bytesperline = f->fmt.pix.width * format->depth / 8;
+ f->fmt.pix.sizeimage = f->fmt.pix.height * f->fmt.pix.bytesperline;
+ f->fmt.pix.colorspace = V4L2_COLORSPACE_SMPTE170M;
+}
+
+/* video queue operations */
+
+static int tw686x_queue_setup(struct vb2_queue *vq, unsigned int *nbuffers,
+ unsigned int *nplanes, unsigned int sizes[],
+ void *alloc_ctxs[])
+{
+ struct tw686x_video_channel *vc = vb2_get_drv_priv(vq);
+ unsigned int size = vc->width * vc->height * vc->format->depth / 8;
+
+ alloc_ctxs[0] = vc->alloc_ctx;
+ if (*nbuffers < 2)
+ *nbuffers = 2;
+
+ if (*nplanes)
+ return sizes[0] < size ? -EINVAL : 0;
+
+ sizes[0] = size;
+ *nplanes = 1; /* packed formats only */
+ return 0;
+}
+
+static void tw686x_buf_queue(struct vb2_buffer *vb)
+{
+ struct tw686x_video_channel *vc = vb2_get_drv_priv(vb->vb2_queue);
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct tw686x_vb2_buf *buf;
+
+ buf = container_of(vbuf, struct tw686x_vb2_buf, vb);
+
+ spin_lock(&vc->qlock);
+ list_add_tail(&buf->list, &vc->vidq_queued);
+ spin_unlock(&vc->qlock);
+}
+
+static void setup_descs(struct tw686x_video_channel *vc, unsigned int n)
+{
+loop:
+ while (!list_empty(&vc->vidq_queued)) {
+ struct vdma_desc *descs = vc->sg_descs[n];
+ struct tw686x_vb2_buf *buf;
+ struct sg_table *vbuf;
+ struct scatterlist *sg;
+ unsigned int buf_len, count = 0;
+ int i;
+
+ buf = list_first_entry(&vc->vidq_queued, struct tw686x_vb2_buf,
+ list);
+ list_del(&buf->list);
+
+ buf_len = vc->width * vc->height * vc->format->depth / 8;
+ if (vb2_plane_size(&buf->vb.vb2_buf, 0) < buf_len) {
+ pr_err("Video buffer size too small\n");
+ vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
+ goto loop; /* try another */
+ }
+
+ vbuf = vb2_dma_sg_plane_desc(&buf->vb.vb2_buf, 0);
+ for_each_sg(vbuf->sgl, sg, vbuf->nents, i) {
+ dma_addr_t phys = sg_dma_address(sg);
+ unsigned int len = sg_dma_len(sg);
+
+ while (len && buf_len) {
+ unsigned int entry_len = min_t(unsigned int, len,
+ MAX_SG_ENTRY_SIZE);
+ entry_len = min(entry_len, buf_len);
+ if (count == MAX_SG_DESC_COUNT) {
+ pr_err("Video buffer size too fragmented\n");
+ vb2_buffer_done(&buf->vb.vb2_buf,
+ VB2_BUF_STATE_ERROR);
+ goto loop;
+ }
+ descs[count].phys = cpu_to_le32(phys);
+ descs[count++].flags_length =
+ cpu_to_le32(0x40000000 /* available */ |
+ entry_len);
+ phys += entry_len;
+ len -= entry_len;
+ buf_len -= entry_len;
+ }
+ if (!buf_len)
+ break;
+ }
+
+ /* clear the remaining entries */
+ while (count < MAX_SG_DESC_COUNT) {
+ descs[count].phys = 0;
+ descs[count++].flags_length = 0; /* unavailable */
+ }
+
+ buf->vb.vb2_buf.state = VB2_BUF_STATE_ACTIVE;
+ vc->curr_bufs[n] = buf;
+ return;
+ }
+ vc->curr_bufs[n] = NULL;
+}
+
+/* On TW6864 and TW6868, all channels share the pair of video DMA SG tables,
+ with 10-bit start_idx and end_idx determining start and end of frame buffer
+ for particular channel.
+ TW6868 with all its 8 channels would be problematic (only 127 SG entries per
+ channel) but we support only 4 channels on this chip anyway (the first
+ 4 channels are driven with internal video decoder, the other 4 would require
+ an external TW286x part).
+
+ On TW6865 and TW6869, each channel has its own DMA SG table, with indexes
+ starting with 0. Both chips have complete sets of internal video decoders
+ (respectively 4 or 8-channel).
+
+ All chips have separate SG tables for two video frames. */
+
+static void setup_dma_cfg(struct tw686x_video_channel *vc)
+{
+ unsigned int field_width = 704;
+ unsigned int field_height = (vc->video_standard & V4L2_STD_625_50) ?
+ 288 : 240;
+ unsigned int start_idx = is_second_gen(vc->dev) ? 0 :
+ vc->ch * MAX_SG_DESC_COUNT;
+ unsigned int end_idx = start_idx + MAX_SG_DESC_COUNT - 1;
+ u32 dma_cfg = (0 << 30) /* input selection */ |
+ (1 << 29) /* field2 dropped (if any) */ |
+ ((vc->height < 300) << 28) /* field dropping */ |
+ (1 << 27) /* master */ |
+ (0 << 25) /* master channel (for slave only) */ |
+ (0 << 24) /* (no) vertical (line) decimation */ |
+ ((vc->width < 400) << 23) /* horizontal decimation */ |
+ (vc->format->mode << 20) /* output video format */ |
+ (end_idx << 10) /* DMA end index */ |
+ start_idx /* DMA start index */;
+ u32 reg;
+
+ reg_write(vc->dev, VDMA_CHANNEL_CONFIG[vc->ch], dma_cfg);
+ reg_write(vc->dev, VIDEO_SIZE[vc->ch], (1 << 31) | (field_height << 16)
+ | field_width);
+ reg = reg_read(vc->dev, VIDEO_CONTROL1);
+ if (vc->video_standard & V4L2_STD_625_50)
+ reg |= 1 << (vc->ch + 13);
+ else
+ reg &= ~(1 << (vc->ch + 13));
+ reg_write(vc->dev, VIDEO_CONTROL1, reg);
+}
+
+static int tw686x_start_streaming(struct vb2_queue *vq, unsigned int count)
+{
+ struct tw686x_video_channel *vc = vb2_get_drv_priv(vq);
+ struct tw686x_dev *dev = vc->dev;
+ u32 dma_ch_mask;
+ unsigned int n;
+
+ setup_dma_cfg(vc);
+
+ /* queue video buffers if available */
+ spin_lock(&vc->qlock);
+ for (n = 0; n < 2; n++)
+ setup_descs(vc, n);
+ spin_unlock(&vc->qlock);
+
+ dev->video_active |= 1 << vc->ch;
+ vc->seq = 0;
+ dma_ch_mask = reg_read(dev, DMA_CHANNEL_ENABLE) | (1 << vc->ch);
+ reg_write(dev, DMA_CHANNEL_ENABLE, dma_ch_mask);
+ reg_write(dev, DMA_CMD, (1 << 31) | dma_ch_mask);
+ return 0;
+}
+
+static void tw686x_stop_streaming(struct vb2_queue *vq)
+{
+ struct tw686x_video_channel *vc = vb2_get_drv_priv(vq);
+ struct tw686x_dev *dev = vc->dev;
+ u32 dma_ch_mask = reg_read(dev, DMA_CHANNEL_ENABLE);
+ u32 dma_cmd = reg_read(dev, DMA_CMD);
+ unsigned int n;
+
+ dma_ch_mask &= ~(1 << vc->ch);
+ reg_write(dev, DMA_CHANNEL_ENABLE, dma_ch_mask);
+
+ dev->video_active &= ~(1 << vc->ch);
+
+ dma_cmd &= ~(1 << vc->ch);
+ reg_write(dev, DMA_CMD, dma_cmd);
+
+ if (!dev->video_active) {
+ reg_write(dev, DMA_CMD, 0);
+ reg_write(dev, DMA_CHANNEL_ENABLE, 0);
+ }
+
+ spin_lock(&vc->qlock);
+ while (!list_empty(&vc->vidq_queued)) {
+ struct tw686x_vb2_buf *buf;
+
+ buf = list_entry(vc->vidq_queued.next, struct tw686x_vb2_buf,
+ list);
+ list_del(&buf->list);
+ vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
+ }
+
+ for (n = 0; n < 2; n++)
+ if (vc->curr_bufs[n])
+ vb2_buffer_done(&vc->curr_bufs[n]->vb.vb2_buf,
+ VB2_BUF_STATE_ERROR);
+
+ spin_unlock(&vc->qlock);
+}
+
+static struct vb2_ops tw686x_video_qops = {
+ .queue_setup = tw686x_queue_setup,
+ .buf_queue = tw686x_buf_queue,
+ .start_streaming = tw686x_start_streaming,
+ .stop_streaming = tw686x_stop_streaming,
+ .wait_prepare = vb2_ops_wait_prepare,
+ .wait_finish = vb2_ops_wait_finish,
+};
+
+static int tw686x_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct tw686x_video_channel *vc;
+ struct tw686x_dev *dev;
+ unsigned int ch;
+
+ vc = container_of(ctrl->handler, struct tw686x_video_channel,
+ ctrl_handler);
+ dev = vc->dev;
+ ch = vc->ch;
+
+ switch (ctrl->id) {
+ case V4L2_CID_BRIGHTNESS:
+ reg_write(dev, BRIGHT[ch], ctrl->val & 0xFF);
+ return 0;
+
+ case V4L2_CID_CONTRAST:
+ reg_write(dev, CONTRAST[ch], ctrl->val);
+ return 0;
+
+ case V4L2_CID_SATURATION:
+ reg_write(dev, SAT_U[ch], ctrl->val);
+ reg_write(dev, SAT_V[ch], ctrl->val);
+ return 0;
+
+ case V4L2_CID_HUE:
+ reg_write(dev, HUE[ch], ctrl->val & 0xFF);
+ return 0;
+ }
+
+ return -EINVAL;
+}
+
+static const struct v4l2_ctrl_ops ctrl_ops = {
+ .s_ctrl = tw686x_s_ctrl,
+};
+
+static int tw686x_g_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct tw686x_video_channel *vc = video_drvdata(file);
+
+ f->fmt.pix.width = vc->width;
+ f->fmt.pix.height = vc->height;
+ f->fmt.pix.field = vc->field;
+ f->fmt.pix.pixelformat = vc->format->fourcc;
+ f->fmt.pix.colorspace = V4L2_COLORSPACE_SMPTE170M;
+ f->fmt.pix.bytesperline = f->fmt.pix.width * vc->format->depth / 8;
+ f->fmt.pix.sizeimage = f->fmt.pix.height * f->fmt.pix.bytesperline;
+ return 0;
+}
+
+static int tw686x_try_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ tw686x_get_format(video_drvdata(file), f);
+ return 0;
+}
+
+static int tw686x_s_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct tw686x_video_channel *vc = video_drvdata(file);
+
+ tw686x_get_format(vc, f);
+ vc->format = format_by_fourcc(f->fmt.pix.pixelformat);
+ vc->field = f->fmt.pix.field;
+ vc->width = f->fmt.pix.width;
+ vc->height = f->fmt.pix.height;
+ return 0;
+}
+
+static int tw686x_querycap(struct file *file, void *priv,
+ struct v4l2_capability *cap)
+{
+ struct tw686x_video_channel *vc = video_drvdata(file);
+ struct tw686x_dev *dev = vc->dev;
+
+ strcpy(cap->driver, "tw686x-kh");
+ strcpy(cap->card, dev->name);
+ sprintf(cap->bus_info, "PCI:%s", pci_name(dev->pci_dev));
+ cap->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
+ cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
+ return 0;
+}
+
+static int tw686x_s_std(struct file *file, void *priv, v4l2_std_id id)
+{
+ struct tw686x_video_channel *vc = video_drvdata(file);
+ unsigned int cnt;
+ u32 sdt = 0; /* default */
+
+ for (cnt = 0; cnt < ARRAY_SIZE(video_standards); cnt++)
+ if (id & video_standards[cnt]) {
+ sdt = cnt;
+ break;
+ }
+
+ reg_write(vc->dev, SDT[vc->ch], sdt);
+ vc->video_standard = video_standards[sdt];
+ return 0;
+}
+
+static int tw686x_g_std(struct file *file, void *priv, v4l2_std_id *id)
+{
+ struct tw686x_video_channel *vc = video_drvdata(file);
+
+ *id = vc->video_standard;
+ return 0;
+}
+
+static int tw686x_enum_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_fmtdesc *f)
+{
+ if (f->index >= ARRAY_SIZE(formats))
+ return -EINVAL;
+
+ strlcpy(f->description, formats[f->index].name, sizeof(f->description));
+ f->pixelformat = formats[f->index].fourcc;
+ return 0;
+}
+
+static int tw686x_g_parm(struct file *file, void *priv,
+ struct v4l2_streamparm *sp)
+{
+ struct tw686x_video_channel *vc = video_drvdata(file);
+
+ if (sp->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+ return -EINVAL;
+ memset(&sp->parm.capture, 0, sizeof(sp->parm.capture));
+ sp->parm.capture.capability = V4L2_CAP_TIMEPERFRAME;
+ v4l2_video_std_frame_period(vc->video_standard,
+ &sp->parm.capture.timeperframe);
+
+ return 0;
+}
+
+static int tw686x_enum_input(struct file *file, void *priv,
+ struct v4l2_input *inp)
+{
+ /* the chip has internal multiplexer, support can be added
+ if the actual hw uses it */
+ if (inp->index)
+ return -EINVAL;
+
+ snprintf(inp->name, sizeof(inp->name), "Composite");
+ inp->type = V4L2_INPUT_TYPE_CAMERA;
+ inp->std = V4L2_STD_ALL;
+ inp->capabilities = V4L2_IN_CAP_STD;
+ return 0;
+}
+
+static int tw686x_g_input(struct file *file, void *priv, unsigned int *v)
+{
+ *v = 0;
+ return 0;
+}
+
+static int tw686x_s_input(struct file *file, void *priv, unsigned int v)
+{
+ if (v)
+ return -EINVAL;
+ return 0;
+}
+
+static const struct v4l2_file_operations tw686x_video_fops = {
+ .owner = THIS_MODULE,
+ .open = v4l2_fh_open,
+ .unlocked_ioctl = video_ioctl2,
+ .release = vb2_fop_release,
+ .poll = vb2_fop_poll,
+ .read = vb2_fop_read,
+ .mmap = vb2_fop_mmap,
+};
+
+static const struct v4l2_ioctl_ops tw686x_video_ioctl_ops = {
+ .vidioc_querycap = tw686x_querycap,
+ .vidioc_enum_fmt_vid_cap = tw686x_enum_fmt_vid_cap,
+ .vidioc_g_fmt_vid_cap = tw686x_g_fmt_vid_cap,
+ .vidioc_s_fmt_vid_cap = tw686x_s_fmt_vid_cap,
+ .vidioc_try_fmt_vid_cap = tw686x_try_fmt_vid_cap,
+ .vidioc_reqbufs = vb2_ioctl_reqbufs,
+ .vidioc_querybuf = vb2_ioctl_querybuf,
+ .vidioc_qbuf = vb2_ioctl_qbuf,
+ .vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_create_bufs = vb2_ioctl_create_bufs,
+ .vidioc_streamon = vb2_ioctl_streamon,
+ .vidioc_streamoff = vb2_ioctl_streamoff,
+ .vidioc_g_std = tw686x_g_std,
+ .vidioc_s_std = tw686x_s_std,
+ .vidioc_g_parm = tw686x_g_parm,
+ .vidioc_enum_input = tw686x_enum_input,
+ .vidioc_g_input = tw686x_g_input,
+ .vidioc_s_input = tw686x_s_input,
+ .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static int video_thread(void *arg)
+{
+ struct tw686x_dev *dev = arg;
+ DECLARE_WAITQUEUE(wait, current);
+
+ set_freezable();
+ add_wait_queue(&dev->video_thread_wait, &wait);
+
+ while (1) {
+ long timeout = schedule_timeout_interruptible(HZ);
+ unsigned int ch;
+
+ if (timeout == -ERESTARTSYS || kthread_should_stop())
+ break;
+
+ for (ch = 0; ch < max_channels(dev); ch++) {
+ struct tw686x_video_channel *vc;
+ unsigned long flags;
+ u32 request, n, stat = VB2_BUF_STATE_DONE;
+
+ vc = &dev->video_channels[ch];
+ if (!(dev->video_active & (1 << ch)))
+ continue;
+
+ spin_lock_irq(&dev->irq_lock);
+ request = dev->dma_requests & (0x01000001 << ch);
+ if (request)
+ dev->dma_requests &= ~request;
+ spin_unlock_irq(&dev->irq_lock);
+
+ if (!request)
+ continue;
+
+ request >>= ch;
+
+ /* handle channel events */
+ if ((request & 0x01000000) |
+ (reg_read(dev, VIDEO_FIFO_STATUS) & (0x01010001 << ch)) |
+ (reg_read(dev, VIDEO_PARSER_STATUS) & (0x00000101 << ch))) {
+ /* DMA Errors - reset channel */
+ u32 reg;
+
+ spin_lock_irqsave(&dev->irq_lock, flags);
+ reg = reg_read(dev, DMA_CMD);
+ /* Reset DMA channel */
+ reg_write(dev, DMA_CMD, reg & ~(1 << ch));
+ reg_write(dev, DMA_CMD, reg);
+ spin_unlock_irqrestore(&dev->irq_lock, flags);
+ stat = VB2_BUF_STATE_ERROR;
+ }
+
+ /* handle video stream */
+ mutex_lock(&vc->vb_mutex);
+ spin_lock(&vc->qlock);
+ n = !!(reg_read(dev, PB_STATUS) & (1 << ch));
+ if (vc->curr_bufs[n]) {
+ struct vb2_v4l2_buffer *vb;
+
+ vb = &vc->curr_bufs[n]->vb;
+ vb->vb2_buf.timestamp = ktime_get_ns();
+ vb->field = vc->field;
+ if (V4L2_FIELD_HAS_BOTH(vc->field))
+ vb->sequence = vc->seq++;
+ else
+ vb->sequence = (vc->seq++) / 2;
+ vb2_set_plane_payload(&vb->vb2_buf, 0,
+ vc->width * vc->height * vc->format->depth / 8);
+ vb2_buffer_done(&vb->vb2_buf, stat);
+ }
+ setup_descs(vc, n);
+ spin_unlock(&vc->qlock);
+ mutex_unlock(&vc->vb_mutex);
+ }
+ try_to_freeze();
+ }
+
+ remove_wait_queue(&dev->video_thread_wait, &wait);
+ return 0;
+}
+
+int tw686x_kh_video_irq(struct tw686x_dev *dev)
+{
+ unsigned long flags, handled = 0;
+ u32 requests;
+
+ spin_lock_irqsave(&dev->irq_lock, flags);
+ requests = dev->dma_requests;
+ spin_unlock_irqrestore(&dev->irq_lock, flags);
+
+ if (requests & dev->video_active) {
+ wake_up_interruptible_all(&dev->video_thread_wait);
+ handled = 1;
+ }
+ return handled;
+}
+
+void tw686x_kh_video_free(struct tw686x_dev *dev)
+{
+ unsigned int ch, n;
+
+ if (dev->video_thread)
+ kthread_stop(dev->video_thread);
+
+ for (ch = 0; ch < max_channels(dev); ch++) {
+ struct tw686x_video_channel *vc = &dev->video_channels[ch];
+
+ v4l2_ctrl_handler_free(&vc->ctrl_handler);
+ if (vc->device)
+ video_unregister_device(vc->device);
+ vb2_dma_sg_cleanup_ctx(vc->alloc_ctx);
+ for (n = 0; n < 2; n++) {
+ struct dma_desc *descs = &vc->sg_tables[n];
+
+ if (descs->virt)
+ pci_free_consistent(dev->pci_dev, descs->size,
+ descs->virt, descs->phys);
+ }
+ }
+
+ v4l2_device_unregister(&dev->v4l2_dev);
+}
+
+#define SG_TABLE_SIZE (MAX_SG_DESC_COUNT * sizeof(struct vdma_desc))
+
+int tw686x_kh_video_init(struct tw686x_dev *dev)
+{
+ unsigned int ch, n;
+ int err;
+
+ init_waitqueue_head(&dev->video_thread_wait);
+
+ err = v4l2_device_register(&dev->pci_dev->dev, &dev->v4l2_dev);
+ if (err)
+ return err;
+
+ reg_write(dev, VIDEO_CONTROL1, 0); /* NTSC, disable scaler */
+ reg_write(dev, PHASE_REF, 0x00001518); /* Scatter-gather DMA mode */
+
+ /* setup required SG table sizes */
+ for (n = 0; n < 2; n++)
+ if (is_second_gen(dev)) {
+ /* TW 6865, TW6869 - each channel needs a pair of
+ descriptor tables */
+ for (ch = 0; ch < max_channels(dev); ch++)
+ dev->video_channels[ch].sg_tables[n].size =
+ SG_TABLE_SIZE;
+
+ } else
+ /* TW 6864, TW6868 - we need to allocate a pair of
+ descriptor tables, common for all channels.
+ Each table will be bigger than 4 KB. */
+ dev->video_channels[0].sg_tables[n].size =
+ max_channels(dev) * SG_TABLE_SIZE;
+
+ /* allocate SG tables and initialize video channels */
+ for (ch = 0; ch < max_channels(dev); ch++) {
+ struct tw686x_video_channel *vc = &dev->video_channels[ch];
+ struct video_device *vdev;
+
+ mutex_init(&vc->vb_mutex);
+ spin_lock_init(&vc->qlock);
+ INIT_LIST_HEAD(&vc->vidq_queued);
+
+ vc->dev = dev;
+ vc->ch = ch;
+
+ /* default settings: NTSC */
+ vc->format = &formats[0];
+ vc->video_standard = V4L2_STD_NTSC;
+ reg_write(vc->dev, SDT[vc->ch], 0);
+ vc->field = V4L2_FIELD_SEQ_BT;
+ vc->width = 704;
+ vc->height = 480;
+
+ for (n = 0; n < 2; n++) {
+ void *cpu;
+
+ if (vc->sg_tables[n].size) {
+ unsigned int reg = n ? DMA_PAGE_TABLE1_ADDR[ch] :
+ DMA_PAGE_TABLE0_ADDR[ch];
+
+ cpu = pci_alloc_consistent(dev->pci_dev,
+ vc->sg_tables[n].size,
+ &vc->sg_tables[n].phys);
+ if (!cpu) {
+ pr_err("Error allocating video DMA scatter-gather tables\n");
+ err = -ENOMEM;
+ goto error;
+ }
+ vc->sg_tables[n].virt = cpu;
+ reg_write(dev, reg, vc->sg_tables[n].phys);
+ } else
+ cpu = dev->video_channels[0].sg_tables[n].virt +
+ ch * SG_TABLE_SIZE;
+
+ vc->sg_descs[n] = cpu;
+ }
+
+ reg_write(dev, VCTRL1[0], 0x24);
+ reg_write(dev, LOOP[0], 0xA5);
+ if (max_channels(dev) > 4) {
+ reg_write(dev, VCTRL1[1], 0x24);
+ reg_write(dev, LOOP[1], 0xA5);
+ }
+ reg_write(dev, VIDEO_FIELD_CTRL[ch], 0);
+ reg_write(dev, VDELAY_LO[ch], 0x14);
+
+ vdev = video_device_alloc();
+ if (!vdev) {
+ pr_warn("Unable to allocate video device\n");
+ err = -ENOMEM;
+ goto error;
+ }
+
+ vc->alloc_ctx = vb2_dma_sg_init_ctx(&dev->pci_dev->dev);
+ if (IS_ERR(vc->alloc_ctx)) {
+ pr_warn("Unable to initialize DMA scatter-gather context\n");
+ err = PTR_ERR(vc->alloc_ctx);
+ goto error;
+ }
+
+ vc->vidq.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+ vc->vidq.io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF;
+ vc->vidq.drv_priv = vc;
+ vc->vidq.buf_struct_size = sizeof(struct tw686x_vb2_buf);
+ vc->vidq.ops = &tw686x_video_qops;
+ vc->vidq.mem_ops = &vb2_dma_sg_memops;
+ vc->vidq.timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+ vc->vidq.min_buffers_needed = 2;
+ vc->vidq.lock = &vc->vb_mutex;
+ vc->vidq.gfp_flags = GFP_DMA32;
+
+ err = vb2_queue_init(&vc->vidq);
+ if (err)
+ goto error;
+
+ strcpy(vdev->name, "TW686x-video");
+ snprintf(vdev->name, sizeof(vdev->name), "%s video", dev->name);
+ vdev->fops = &tw686x_video_fops;
+ vdev->ioctl_ops = &tw686x_video_ioctl_ops;
+ vdev->release = video_device_release;
+ vdev->v4l2_dev = &dev->v4l2_dev;
+ vdev->queue = &vc->vidq;
+ vdev->tvnorms = V4L2_STD_ALL;
+ vdev->minor = -1;
+ vdev->lock = &vc->vb_mutex;
+
+ dev->video_channels[ch].device = vdev;
+ video_set_drvdata(vdev, vc);
+ err = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+ if (err < 0)
+ goto error;
+
+ v4l2_ctrl_handler_init(&vc->ctrl_handler,
+ 4 /* number of controls */);
+ vdev->ctrl_handler = &vc->ctrl_handler;
+ v4l2_ctrl_new_std(&vc->ctrl_handler, &ctrl_ops,
+ V4L2_CID_BRIGHTNESS, -128, 127, 1, 0);
+ v4l2_ctrl_new_std(&vc->ctrl_handler, &ctrl_ops,
+ V4L2_CID_CONTRAST, 0, 255, 1, 64);
+ v4l2_ctrl_new_std(&vc->ctrl_handler, &ctrl_ops,
+ V4L2_CID_SATURATION, 0, 255, 1, 128);
+ v4l2_ctrl_new_std(&vc->ctrl_handler, &ctrl_ops, V4L2_CID_HUE,
+ -124, 127, 1, 0);
+ err = vc->ctrl_handler.error;
+ if (err)
+ goto error;
+
+ v4l2_ctrl_handler_setup(&vc->ctrl_handler);
+ }
+
+ dev->video_thread = kthread_run(video_thread, dev, "tw686x_video");
+ if (IS_ERR(dev->video_thread)) {
+ err = PTR_ERR(dev->video_thread);
+ goto error;
+ }
+
+ return 0;
+
+error:
+ tw686x_kh_video_free(dev);
+ return err;
+}
diff --git a/drivers/staging/media/tw686x-kh/tw686x-kh.h b/drivers/staging/media/tw686x-kh/tw686x-kh.h
new file mode 100644
index 0000000..dc25796
--- /dev/null
+++ b/drivers/staging/media/tw686x-kh/tw686x-kh.h
@@ -0,0 +1,118 @@
+/*
+ * Copyright (C) 2015 Industrial Research Institute for Automation
+ * and Measurements PIAP
+ *
+ * Written by Krzysztof Ha?asa.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of version 2 of the GNU General Public License
+ * as published by the Free Software Foundation.
+ */
+
+#include <linux/delay.h>
+#include <linux/freezer.h>
+#include <linux/interrupt.h>
+#include <linux/kthread.h>
+#include <linux/mutex.h>
+#include <linux/pci.h>
+#include <media/videobuf2-dma-sg.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-common.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+
+#define TYPE_MAX_CHANNELS 0x0F
+#define TYPE_SECOND_GEN 0x10
+
+struct tw686x_format {
+ char *name;
+ unsigned int fourcc;
+ unsigned int depth;
+ unsigned int mode;
+};
+
+struct dma_desc {
+ dma_addr_t phys;
+ void *virt;
+ unsigned int size;
+};
+
+struct vdma_desc {
+ __le32 flags_length; /* 3 MSBits for flags, 13 LSBits for length */
+ __le32 phys;
+};
+
+struct tw686x_vb2_buf {
+ struct vb2_v4l2_buffer vb;
+ struct list_head list;
+};
+
+struct tw686x_video_channel {
+ struct tw686x_dev *dev;
+
+ struct vb2_queue vidq;
+ struct list_head vidq_queued;
+ struct video_device *device;
+ struct dma_desc sg_tables[2];
+ struct tw686x_vb2_buf *curr_bufs[2];
+ void *alloc_ctx;
+ struct vdma_desc *sg_descs[2];
+
+ struct v4l2_ctrl_handler ctrl_handler;
+ const struct tw686x_format *format;
+ struct mutex vb_mutex;
+ spinlock_t qlock;
+ v4l2_std_id video_standard;
+ unsigned int width, height;
+ enum v4l2_field field; /* supported TOP, BOTTOM, SEQ_TB and SEQ_BT */
+ unsigned int seq; /* video field or frame counter */
+ unsigned int ch;
+};
+
+/* global device status */
+struct tw686x_dev {
+ spinlock_t irq_lock;
+
+ struct v4l2_device v4l2_dev;
+ struct snd_card *card; /* sound card */
+
+ unsigned int video_active; /* active video channel mask */
+
+ char name[32];
+ unsigned int type;
+ struct pci_dev *pci_dev;
+ __u32 __iomem *mmio;
+
+ struct task_struct *video_thread;
+ wait_queue_head_t video_thread_wait;
+ u32 dma_requests;
+
+ struct tw686x_video_channel video_channels[0];
+};
+
+static inline uint32_t reg_read(struct tw686x_dev *dev, unsigned int reg)
+{
+ return readl(dev->mmio + reg);
+}
+
+static inline void reg_write(struct tw686x_dev *dev, unsigned int reg,
+ uint32_t value)
+{
+ writel(value, dev->mmio + reg);
+}
+
+static inline unsigned int max_channels(struct tw686x_dev *dev)
+{
+ return dev->type & TYPE_MAX_CHANNELS; /* 4 or 8 channels */
+}
+
+static inline unsigned int is_second_gen(struct tw686x_dev *dev)
+{
+ /* each channel has its own DMA SG table */
+ return dev->type & TYPE_SECOND_GEN;
+}
+
+int tw686x_kh_video_irq(struct tw686x_dev *dev);
+int tw686x_kh_video_init(struct tw686x_dev *dev);
+void tw686x_kh_video_free(struct tw686x_dev *dev);
diff --git a/drivers/staging/most/hdm-dim2/dim2_errors.h b/drivers/staging/most/hdm-dim2/dim2_errors.h
index 5a713df..66343ba 100644
--- a/drivers/staging/most/hdm-dim2/dim2_errors.h
+++ b/drivers/staging/most/hdm-dim2/dim2_errors.h
@@ -15,10 +15,6 @@
#ifndef _MOST_DIM_ERRORS_H
#define _MOST_DIM_ERRORS_H
-#ifdef __cplusplus
-extern "C" {
-#endif
-
/**
* MOST DIM errors.
*/
@@ -58,8 +54,4 @@
DIM_ERR_OVERFLOW,
};
-#ifdef __cplusplus
-}
-#endif
-
#endif /* _MOST_DIM_ERRORS_H */
diff --git a/drivers/staging/most/hdm-dim2/dim2_hal.h b/drivers/staging/most/hdm-dim2/dim2_hal.h
index fc73d4f..1c924e8 100644
--- a/drivers/staging/most/hdm-dim2/dim2_hal.h
+++ b/drivers/staging/most/hdm-dim2/dim2_hal.h
@@ -18,10 +18,6 @@
#include <linux/types.h>
#include "dim2_reg.h"
-#ifdef __cplusplus
-extern "C" {
-#endif
-
/*
* The values below are specified in the hardware specification.
* So, they should not be changed until the hardware specification changes.
@@ -42,14 +38,12 @@
u16 done_buffers; /* Number of completed buffers */
};
-typedef int atomic_counter_t;
-
struct int_ch_state {
/* changed only in interrupt context */
- volatile atomic_counter_t request_counter;
+ volatile int request_counter;
/* changed only in task context */
- volatile atomic_counter_t service_counter;
+ volatile int service_counter;
u8 idx1;
u8 idx2;
@@ -110,8 +104,4 @@
void dimcb_on_error(u8 error_id, const char *error_message);
-#ifdef __cplusplus
-}
-#endif
-
#endif /* _DIM2_HAL_H */
diff --git a/drivers/staging/most/hdm-dim2/dim2_reg.h b/drivers/staging/most/hdm-dim2/dim2_reg.h
index bcf6a79..e0837b6 100644
--- a/drivers/staging/most/hdm-dim2/dim2_reg.h
+++ b/drivers/staging/most/hdm-dim2/dim2_reg.h
@@ -17,10 +17,6 @@
#include <linux/types.h>
-#ifdef __cplusplus
-extern "C" {
-#endif
-
struct dim2_regs {
/* 0x00 */ u32 MLBC0;
/* 0x01 */ u32 rsvd0[1];
@@ -166,8 +162,4 @@
CAT_CL_MASK = DIM2_MASK(6)
};
-#ifdef __cplusplus
-}
-#endif
-
#endif /* DIM2_OS62420_H */
diff --git a/drivers/staging/mt29f_spinand/mt29f_spinand.c b/drivers/staging/mt29f_spinand/mt29f_spinand.c
index 163f21a..e389009 100644
--- a/drivers/staging/mt29f_spinand/mt29f_spinand.c
+++ b/drivers/staging/mt29f_spinand/mt29f_spinand.c
@@ -42,23 +42,33 @@
static int enable_hw_ecc;
static int enable_read_hw_ecc;
-static struct nand_ecclayout spinand_oob_64 = {
- .eccbytes = 24,
- .eccpos = {
- 1, 2, 3, 4, 5, 6,
- 17, 18, 19, 20, 21, 22,
- 33, 34, 35, 36, 37, 38,
- 49, 50, 51, 52, 53, 54, },
- .oobfree = {
- {.offset = 8,
- .length = 8},
- {.offset = 24,
- .length = 8},
- {.offset = 40,
- .length = 8},
- {.offset = 56,
- .length = 8},
- }
+static int spinand_ooblayout_64_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section > 3)
+ return -ERANGE;
+
+ oobregion->offset = (section * 16) + 1;
+ oobregion->length = 6;
+
+ return 0;
+}
+
+static int spinand_ooblayout_64_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobregion)
+{
+ if (section > 3)
+ return -ERANGE;
+
+ oobregion->offset = (section * 16) + 8;
+ oobregion->length = 8;
+
+ return 0;
+}
+
+static const struct mtd_ooblayout_ops spinand_oob_64_ops = {
+ .ecc = spinand_ooblayout_64_ecc,
+ .free = spinand_ooblayout_64_free,
};
#endif
@@ -886,11 +896,11 @@
chip->ecc.strength = 1;
chip->ecc.total = chip->ecc.steps * chip->ecc.bytes;
- chip->ecc.layout = &spinand_oob_64;
chip->ecc.read_page = spinand_read_page_hwecc;
chip->ecc.write_page = spinand_write_page_hwecc;
#else
chip->ecc.mode = NAND_ECC_SOFT;
+ chip->ecc.algo = NAND_ECC_HAMMING;
if (spinand_disable_ecc(spi_nand) < 0)
dev_info(&spi_nand->dev, "%s: disable ecc failed!\n",
__func__);
@@ -912,6 +922,9 @@
mtd->dev.parent = &spi_nand->dev;
mtd->oobsize = 64;
+#ifdef CONFIG_MTD_SPINAND_ONDIEECC
+ mtd_set_ooblayout(mtd, &spinand_oob_64_ops);
+#endif
if (nand_scan(mtd, 1))
return -ENXIO;
diff --git a/drivers/staging/netlogic/xlr_net.c b/drivers/staging/netlogic/xlr_net.c
index aa1cdf6..99445d0 100644
--- a/drivers/staging/netlogic/xlr_net.c
+++ b/drivers/staging/netlogic/xlr_net.c
@@ -850,7 +850,7 @@
/* Attach MAC to PHY */
phydev = phy_connect(priv->ndev, phydev_name(phydev),
- &xlr_gmac_link_adjust, priv->nd->phy_interface);
+ xlr_gmac_link_adjust, priv->nd->phy_interface);
if (IS_ERR(phydev)) {
pr_err("could not attach PHY\n");
diff --git a/drivers/staging/nvec/nvec.c b/drivers/staging/nvec/nvec.c
index 9fda136..c1feccf 100644
--- a/drivers/staging/nvec/nvec.c
+++ b/drivers/staging/nvec/nvec.c
@@ -264,7 +264,7 @@
msg = nvec_msg_alloc(nvec, NVEC_MSG_TX);
- if (msg == NULL)
+ if (!msg)
return -ENOMEM;
msg->data[0] = size;
@@ -620,7 +620,7 @@
} else {
nvec->rx = nvec_msg_alloc(nvec, NVEC_MSG_RX);
/* Should not happen in a normal world */
- if (unlikely(nvec->rx == NULL)) {
+ if (unlikely(!nvec->rx)) {
nvec->state = 0;
break;
}
@@ -659,10 +659,11 @@
} else if (nvec->tx && nvec->tx->pos < nvec->tx->size) {
to_send = nvec->tx->data[nvec->tx->pos++];
} else {
- dev_err(nvec->dev, "tx buffer underflow on %p (%u > %u)\n",
+ dev_err(nvec->dev,
+ "tx buffer underflow on %p (%u > %u)\n",
nvec->tx,
- (uint) (nvec->tx ? nvec->tx->pos : 0),
- (uint) (nvec->tx ? nvec->tx->size : 0));
+ (uint)(nvec->tx ? nvec->tx->pos : 0),
+ (uint)(nvec->tx ? nvec->tx->size : 0));
nvec->state = 0;
}
break;
diff --git a/drivers/staging/nvec/nvec_power.c b/drivers/staging/nvec/nvec_power.c
index b4a0545..fcbb0fa 100644
--- a/drivers/staging/nvec/nvec_power.c
+++ b/drivers/staging/nvec/nvec_power.c
@@ -90,7 +90,7 @@
{
struct nvec_power *power =
container_of(nb, struct nvec_power, notifier);
- struct bat_response *res = (struct bat_response *)data;
+ struct bat_response *res = data;
if (event_type != NVEC_SYS)
return NOTIFY_DONE;
@@ -126,7 +126,7 @@
{
struct nvec_power *power =
container_of(nb, struct nvec_power, notifier);
- struct bat_response *res = (struct bat_response *)data;
+ struct bat_response *res = data;
int status_changed = 0;
if (event_type != NVEC_BAT)
diff --git a/drivers/staging/octeon/ethernet-rx.c b/drivers/staging/octeon/ethernet-rx.c
index b6993b0..a10fe3a 100644
--- a/drivers/staging/octeon/ethernet-rx.c
+++ b/drivers/staging/octeon/ethernet-rx.c
@@ -172,12 +172,13 @@
if (OCTEON_IS_MODEL(OCTEON_CN68XX)) {
old_group_mask = cvmx_read_csr(CVMX_SSO_PPX_GRP_MSK(coreid));
cvmx_write_csr(CVMX_SSO_PPX_GRP_MSK(coreid),
- 1ull << pow_receive_group);
+ 1ull << pow_receive_group);
cvmx_read_csr(CVMX_SSO_PPX_GRP_MSK(coreid)); /* Flush */
} else {
old_group_mask = cvmx_read_csr(CVMX_POW_PP_GRP_MSKX(coreid));
cvmx_write_csr(CVMX_POW_PP_GRP_MSKX(coreid),
- (old_group_mask & ~0xFFFFull) | 1 << pow_receive_group);
+ (old_group_mask & ~0xFFFFull) |
+ 1 << pow_receive_group);
}
if (USE_ASYNC_IOBDMA) {
@@ -374,7 +375,7 @@
* doesn't exist.
*/
printk_ratelimited("Port %d not controlled by Linux, packet dropped\n",
- port);
+ port);
dev_kfree_skb_irq(skb);
}
/*
diff --git a/drivers/staging/octeon/ethernet-rx.h b/drivers/staging/octeon/ethernet-rx.h
index a5973fd..315a63d 100644
--- a/drivers/staging/octeon/ethernet-rx.h
+++ b/drivers/staging/octeon/ethernet-rx.h
@@ -30,7 +30,7 @@
number_to_free);
if (num_freed != number_to_free) {
cvmx_fau_atomic_add32(FAU_NUM_PACKET_BUFFERS_TO_FREE,
- number_to_free - num_freed);
+ number_to_free - num_freed);
}
}
}
diff --git a/drivers/staging/octeon/ethernet-tx.c b/drivers/staging/octeon/ethernet-tx.c
index ffe9bd7..6b4c208 100644
--- a/drivers/staging/octeon/ethernet-tx.c
+++ b/drivers/staging/octeon/ethernet-tx.c
@@ -58,9 +58,9 @@
/* Maximum number of SKBs to try to free per xmit packet. */
#define MAX_SKB_TO_FREE (MAX_OUT_QUEUE_DEPTH * 2)
-static inline int32_t cvm_oct_adjust_skb_to_free(int32_t skb_to_free, int fau)
+static inline int cvm_oct_adjust_skb_to_free(int skb_to_free, int fau)
{
- int32_t undo;
+ int undo;
undo = skb_to_free > 0 ? MAX_SKB_TO_FREE : skb_to_free +
MAX_SKB_TO_FREE;
@@ -83,7 +83,7 @@
static void cvm_oct_free_tx_skbs(struct net_device *dev)
{
- int32_t skb_to_free;
+ int skb_to_free;
int qos, queues_per_port;
int total_freed = 0;
int total_remaining = 0;
@@ -148,8 +148,8 @@
enum {QUEUE_CORE, QUEUE_HW, QUEUE_DROP} queue_type;
struct octeon_ethernet *priv = netdev_priv(dev);
struct sk_buff *to_free_list;
- int32_t skb_to_free;
- int32_t buffers_to_free;
+ int skb_to_free;
+ int buffers_to_free;
u32 total_to_clean;
unsigned long flags;
#if REUSE_SKBUFFS_WITHOUT_FREE
@@ -220,7 +220,8 @@
priv->fau + qos * 4, MAX_SKB_TO_FREE);
}
skb_to_free = cvm_oct_adjust_skb_to_free(skb_to_free,
- priv->fau + qos * 4);
+ priv->fau +
+ qos * 4);
spin_lock_irqsave(&priv->tx_free_list[qos].lock, flags);
goto skip_xmit;
}
@@ -402,7 +403,7 @@
}
skb_to_free = cvm_oct_adjust_skb_to_free(skb_to_free,
- priv->fau + qos * 4);
+ priv->fau + qos * 4);
/*
* If we're sending faster than the receive can free them then
diff --git a/drivers/staging/octeon/ethernet.c b/drivers/staging/octeon/ethernet.c
index 271e1b8d..e9cd5f2 100644
--- a/drivers/staging/octeon/ethernet.c
+++ b/drivers/staging/octeon/ethernet.c
@@ -635,7 +635,7 @@
}
static struct device_node *cvm_oct_node_for_port(struct device_node *pip,
- int interface, int port)
+ int interface, int port)
{
struct device_node *ni, *np;
@@ -815,7 +815,7 @@
free_netdev(dev);
} else if (register_netdev(dev) < 0) {
pr_err("Failed to register ethernet device for interface %d, port %d\n",
- interface, priv->port);
+ interface, priv->port);
free_netdev(dev);
} else {
cvm_oct_device[priv->port] = dev;
diff --git a/drivers/staging/rdma/hfi1/affinity.c b/drivers/staging/rdma/hfi1/affinity.c
index 2cb8ca7..6e7050a 100644
--- a/drivers/staging/rdma/hfi1/affinity.c
+++ b/drivers/staging/rdma/hfi1/affinity.c
@@ -53,20 +53,6 @@
#include "sdma.h"
#include "trace.h"
-struct cpu_mask_set {
- struct cpumask mask;
- struct cpumask used;
- uint gen;
-};
-
-struct hfi1_affinity {
- struct cpu_mask_set def_intr;
- struct cpu_mask_set rcv_intr;
- struct cpu_mask_set proc;
- /* spin lock to protect affinity struct */
- spinlock_t lock;
-};
-
/* Name of IRQ types, indexed by enum irq_type */
static const char * const irq_type_names[] = {
"SDMA",
@@ -82,6 +68,48 @@
set->gen = 0;
}
+/* Initialize non-HT cpu cores mask */
+int init_real_cpu_mask(struct hfi1_devdata *dd)
+{
+ struct hfi1_affinity *info;
+ int possible, curr_cpu, i, ht;
+
+ info = kzalloc(sizeof(*info), GFP_KERNEL);
+ if (!info)
+ return -ENOMEM;
+
+ cpumask_clear(&info->real_cpu_mask);
+
+ /* Start with cpu online mask as the real cpu mask */
+ cpumask_copy(&info->real_cpu_mask, cpu_online_mask);
+
+ /*
+ * Remove HT cores from the real cpu mask. Do this in two steps below.
+ */
+ possible = cpumask_weight(&info->real_cpu_mask);
+ ht = cpumask_weight(topology_sibling_cpumask(
+ cpumask_first(&info->real_cpu_mask)));
+ /*
+ * Step 1. Skip over the first N HT siblings and use them as the
+ * "real" cores. Assumes that HT cores are not enumerated in
+ * succession (except in the single core case).
+ */
+ curr_cpu = cpumask_first(&info->real_cpu_mask);
+ for (i = 0; i < possible / ht; i++)
+ curr_cpu = cpumask_next(curr_cpu, &info->real_cpu_mask);
+ /*
+ * Step 2. Remove the remaining HT siblings. Use cpumask_next() to
+ * skip any gaps.
+ */
+ for (; i < possible; i++) {
+ cpumask_clear_cpu(curr_cpu, &info->real_cpu_mask);
+ curr_cpu = cpumask_next(curr_cpu, &info->real_cpu_mask);
+ }
+
+ dd->affinity = info;
+ return 0;
+}
+
/*
* Interrupt affinity.
*
@@ -93,20 +121,17 @@
* to the node relative 1 as necessary.
*
*/
-int hfi1_dev_affinity_init(struct hfi1_devdata *dd)
+void hfi1_dev_affinity_init(struct hfi1_devdata *dd)
{
int node = pcibus_to_node(dd->pcidev->bus);
- struct hfi1_affinity *info;
+ struct hfi1_affinity *info = dd->affinity;
const struct cpumask *local_mask;
- int curr_cpu, possible, i, ht;
+ int curr_cpu, possible, i;
if (node < 0)
node = numa_node_id();
dd->node = node;
- info = kzalloc(sizeof(*info), GFP_KERNEL);
- if (!info)
- return -ENOMEM;
spin_lock_init(&info->lock);
init_cpu_mask_set(&info->def_intr);
@@ -116,30 +141,8 @@
local_mask = cpumask_of_node(dd->node);
if (cpumask_first(local_mask) >= nr_cpu_ids)
local_mask = topology_core_cpumask(0);
- /* use local mask as default */
- cpumask_copy(&info->def_intr.mask, local_mask);
- /*
- * Remove HT cores from the default mask. Do this in two steps below.
- */
- possible = cpumask_weight(&info->def_intr.mask);
- ht = cpumask_weight(topology_sibling_cpumask(
- cpumask_first(&info->def_intr.mask)));
- /*
- * Step 1. Skip over the first N HT siblings and use them as the
- * "real" cores. Assumes that HT cores are not enumerated in
- * succession (except in the single core case).
- */
- curr_cpu = cpumask_first(&info->def_intr.mask);
- for (i = 0; i < possible / ht; i++)
- curr_cpu = cpumask_next(curr_cpu, &info->def_intr.mask);
- /*
- * Step 2. Remove the remaining HT siblings. Use cpumask_next() to
- * skip any gaps.
- */
- for (; i < possible; i++) {
- cpumask_clear_cpu(curr_cpu, &info->def_intr.mask);
- curr_cpu = cpumask_next(curr_cpu, &info->def_intr.mask);
- }
+ /* Use the "real" cpu mask of this node as the default */
+ cpumask_and(&info->def_intr.mask, &info->real_cpu_mask, local_mask);
/* fill in the receive list */
possible = cpumask_weight(&info->def_intr.mask);
@@ -167,8 +170,6 @@
}
cpumask_copy(&info->proc.mask, cpu_online_mask);
- dd->affinity = info;
- return 0;
}
void hfi1_dev_affinity_free(struct hfi1_devdata *dd)
diff --git a/drivers/staging/rdma/hfi1/affinity.h b/drivers/staging/rdma/hfi1/affinity.h
index b287e496..20f52fe 100644
--- a/drivers/staging/rdma/hfi1/affinity.h
+++ b/drivers/staging/rdma/hfi1/affinity.h
@@ -64,10 +64,27 @@
AFF_IRQ_LOCAL
};
+struct cpu_mask_set {
+ struct cpumask mask;
+ struct cpumask used;
+ uint gen;
+};
+
+struct hfi1_affinity {
+ struct cpu_mask_set def_intr;
+ struct cpu_mask_set rcv_intr;
+ struct cpu_mask_set proc;
+ struct cpumask real_cpu_mask;
+ /* spin lock to protect affinity struct */
+ spinlock_t lock;
+};
+
struct hfi1_msix_entry;
+/* Initialize non-HT cpu cores mask */
+int init_real_cpu_mask(struct hfi1_devdata *);
/* Initialize driver affinity data */
-int hfi1_dev_affinity_init(struct hfi1_devdata *);
+void hfi1_dev_affinity_init(struct hfi1_devdata *);
/* Free driver affinity data */
void hfi1_dev_affinity_free(struct hfi1_devdata *);
/*
diff --git a/drivers/staging/rdma/hfi1/chip.c b/drivers/staging/rdma/hfi1/chip.c
index 16eb653..dcae8e7 100644
--- a/drivers/staging/rdma/hfi1/chip.c
+++ b/drivers/staging/rdma/hfi1/chip.c
@@ -123,6 +123,8 @@
#define MIN_KERNEL_KCTXTS 2
#define FIRST_KERNEL_KCTXT 1
+/* sizes for both the QP and RSM map tables */
+#define NUM_MAP_ENTRIES 256
#define NUM_MAP_REGS 32
/* Bit offset into the GUID which carries HFI id information */
@@ -1029,9 +1031,12 @@
static int wait_logical_linkstate(struct hfi1_pportdata *ppd, u32 state,
int msecs);
static void read_planned_down_reason_code(struct hfi1_devdata *dd, u8 *pdrrc);
+static void read_link_down_reason(struct hfi1_devdata *dd, u8 *ldr);
static void handle_temp_err(struct hfi1_devdata *);
static void dc_shutdown(struct hfi1_devdata *);
static void dc_start(struct hfi1_devdata *);
+static int qos_rmt_entries(struct hfi1_devdata *dd, unsigned int *mp,
+ unsigned int *np);
/*
* Error interrupt table entry. This is used as input to the interrupt
@@ -5661,7 +5666,7 @@
sci = &dd->send_contexts[sw_index];
/* there is no information for user (PSM) and ack contexts */
- if (sci->type != SC_KERNEL)
+ if ((sci->type != SC_KERNEL) && (sci->type != SC_VL15))
return -1;
sc = sci->sc;
@@ -6199,18 +6204,13 @@
/*
* Handle host requests from the 8051.
- *
- * This is a work-queue function outside of the interrupt.
*/
-void handle_8051_request(struct work_struct *work)
+static void handle_8051_request(struct hfi1_pportdata *ppd)
{
- struct hfi1_pportdata *ppd = container_of(work, struct hfi1_pportdata,
- dc_host_req_work);
struct hfi1_devdata *dd = ppd->dd;
u64 reg;
u16 data = 0;
- u8 type, i, lanes, *cache = ppd->qsfp_info.cache;
- u8 cdr_ctrl_byte = cache[QSFP_CDR_CTRL_BYTE_OFFS];
+ u8 type;
reg = read_csr(dd, DC_DC8051_CFG_EXT_DEV_1);
if ((reg & DC_DC8051_CFG_EXT_DEV_1_REQ_NEW_SMASK) == 0)
@@ -6231,46 +6231,11 @@
case HREQ_READ_CONFIG:
case HREQ_SET_TX_EQ_ABS:
case HREQ_SET_TX_EQ_REL:
+ case HREQ_ENABLE:
dd_dev_info(dd, "8051 request: request 0x%x not supported\n",
type);
hreq_response(dd, HREQ_NOT_SUPPORTED, 0);
break;
-
- case HREQ_ENABLE:
- lanes = data & 0xF;
- for (i = 0; lanes; lanes >>= 1, i++) {
- if (!(lanes & 1))
- continue;
- if (data & 0x200) {
- /* enable TX CDR */
- if (cache[QSFP_MOD_PWR_OFFS] & 0x8 &&
- cache[QSFP_CDR_INFO_OFFS] & 0x80)
- cdr_ctrl_byte |= (1 << (i + 4));
- } else {
- /* disable TX CDR */
- if (cache[QSFP_MOD_PWR_OFFS] & 0x8 &&
- cache[QSFP_CDR_INFO_OFFS] & 0x80)
- cdr_ctrl_byte &= ~(1 << (i + 4));
- }
-
- if (data & 0x800) {
- /* enable RX CDR */
- if (cache[QSFP_MOD_PWR_OFFS] & 0x4 &&
- cache[QSFP_CDR_INFO_OFFS] & 0x40)
- cdr_ctrl_byte |= (1 << i);
- } else {
- /* disable RX CDR */
- if (cache[QSFP_MOD_PWR_OFFS] & 0x4 &&
- cache[QSFP_CDR_INFO_OFFS] & 0x40)
- cdr_ctrl_byte &= ~(1 << i);
- }
- }
- one_qsfp_write(ppd, dd->hfi1_id, QSFP_CDR_CTRL_BYTE_OFFS,
- &cdr_ctrl_byte, 1);
- hreq_response(dd, HREQ_SUCCESS, data);
- refresh_qsfp_cache(ppd, &ppd->qsfp_info);
- break;
-
case HREQ_CONFIG_DONE:
hreq_response(dd, HREQ_SUCCESS, 0);
break;
@@ -6278,7 +6243,6 @@
case HREQ_INTERFACE_TEST:
hreq_response(dd, HREQ_SUCCESS, data);
break;
-
default:
dd_dev_err(dd, "8051 request: unknown request 0x%x\n", type);
hreq_response(dd, HREQ_NOT_SUPPORTED, 0);
@@ -6849,6 +6813,75 @@
ppd->neighbor_fm_security = 0;
}
+static const char * const link_down_reason_strs[] = {
+ [OPA_LINKDOWN_REASON_NONE] = "None",
+ [OPA_LINKDOWN_REASON_RCV_ERROR_0] = "Recive error 0",
+ [OPA_LINKDOWN_REASON_BAD_PKT_LEN] = "Bad packet length",
+ [OPA_LINKDOWN_REASON_PKT_TOO_LONG] = "Packet too long",
+ [OPA_LINKDOWN_REASON_PKT_TOO_SHORT] = "Packet too short",
+ [OPA_LINKDOWN_REASON_BAD_SLID] = "Bad SLID",
+ [OPA_LINKDOWN_REASON_BAD_DLID] = "Bad DLID",
+ [OPA_LINKDOWN_REASON_BAD_L2] = "Bad L2",
+ [OPA_LINKDOWN_REASON_BAD_SC] = "Bad SC",
+ [OPA_LINKDOWN_REASON_RCV_ERROR_8] = "Receive error 8",
+ [OPA_LINKDOWN_REASON_BAD_MID_TAIL] = "Bad mid tail",
+ [OPA_LINKDOWN_REASON_RCV_ERROR_10] = "Receive error 10",
+ [OPA_LINKDOWN_REASON_PREEMPT_ERROR] = "Preempt error",
+ [OPA_LINKDOWN_REASON_PREEMPT_VL15] = "Preempt vl15",
+ [OPA_LINKDOWN_REASON_BAD_VL_MARKER] = "Bad VL marker",
+ [OPA_LINKDOWN_REASON_RCV_ERROR_14] = "Receive error 14",
+ [OPA_LINKDOWN_REASON_RCV_ERROR_15] = "Receive error 15",
+ [OPA_LINKDOWN_REASON_BAD_HEAD_DIST] = "Bad head distance",
+ [OPA_LINKDOWN_REASON_BAD_TAIL_DIST] = "Bad tail distance",
+ [OPA_LINKDOWN_REASON_BAD_CTRL_DIST] = "Bad control distance",
+ [OPA_LINKDOWN_REASON_BAD_CREDIT_ACK] = "Bad credit ack",
+ [OPA_LINKDOWN_REASON_UNSUPPORTED_VL_MARKER] = "Unsupported VL marker",
+ [OPA_LINKDOWN_REASON_BAD_PREEMPT] = "Bad preempt",
+ [OPA_LINKDOWN_REASON_BAD_CONTROL_FLIT] = "Bad control flit",
+ [OPA_LINKDOWN_REASON_EXCEED_MULTICAST_LIMIT] = "Exceed multicast limit",
+ [OPA_LINKDOWN_REASON_RCV_ERROR_24] = "Receive error 24",
+ [OPA_LINKDOWN_REASON_RCV_ERROR_25] = "Receive error 25",
+ [OPA_LINKDOWN_REASON_RCV_ERROR_26] = "Receive error 26",
+ [OPA_LINKDOWN_REASON_RCV_ERROR_27] = "Receive error 27",
+ [OPA_LINKDOWN_REASON_RCV_ERROR_28] = "Receive error 28",
+ [OPA_LINKDOWN_REASON_RCV_ERROR_29] = "Receive error 29",
+ [OPA_LINKDOWN_REASON_RCV_ERROR_30] = "Receive error 30",
+ [OPA_LINKDOWN_REASON_EXCESSIVE_BUFFER_OVERRUN] =
+ "Excessive buffer overrun",
+ [OPA_LINKDOWN_REASON_UNKNOWN] = "Unknown",
+ [OPA_LINKDOWN_REASON_REBOOT] = "Reboot",
+ [OPA_LINKDOWN_REASON_NEIGHBOR_UNKNOWN] = "Neighbor unknown",
+ [OPA_LINKDOWN_REASON_FM_BOUNCE] = "FM bounce",
+ [OPA_LINKDOWN_REASON_SPEED_POLICY] = "Speed policy",
+ [OPA_LINKDOWN_REASON_WIDTH_POLICY] = "Width policy",
+ [OPA_LINKDOWN_REASON_DISCONNECTED] = "Disconnected",
+ [OPA_LINKDOWN_REASON_LOCAL_MEDIA_NOT_INSTALLED] =
+ "Local media not installed",
+ [OPA_LINKDOWN_REASON_NOT_INSTALLED] = "Not installed",
+ [OPA_LINKDOWN_REASON_CHASSIS_CONFIG] = "Chassis config",
+ [OPA_LINKDOWN_REASON_END_TO_END_NOT_INSTALLED] =
+ "End to end not installed",
+ [OPA_LINKDOWN_REASON_POWER_POLICY] = "Power policy",
+ [OPA_LINKDOWN_REASON_LINKSPEED_POLICY] = "Link speed policy",
+ [OPA_LINKDOWN_REASON_LINKWIDTH_POLICY] = "Link width policy",
+ [OPA_LINKDOWN_REASON_SWITCH_MGMT] = "Switch management",
+ [OPA_LINKDOWN_REASON_SMA_DISABLED] = "SMA disabled",
+ [OPA_LINKDOWN_REASON_TRANSIENT] = "Transient"
+};
+
+/* return the neighbor link down reason string */
+static const char *link_down_reason_str(u8 reason)
+{
+ const char *str = NULL;
+
+ if (reason < ARRAY_SIZE(link_down_reason_strs))
+ str = link_down_reason_strs[reason];
+ if (!str)
+ str = "(invalid)";
+
+ return str;
+}
+
/*
* Handle a link down interrupt from the 8051.
*
@@ -6857,8 +6890,11 @@
void handle_link_down(struct work_struct *work)
{
u8 lcl_reason, neigh_reason = 0;
+ u8 link_down_reason;
struct hfi1_pportdata *ppd = container_of(work, struct hfi1_pportdata,
- link_down_work);
+ link_down_work);
+ int was_up;
+ static const char ldr_str[] = "Link down reason: ";
if ((ppd->host_link_state &
(HLS_DN_POLL | HLS_VERIFY_CAP | HLS_GOING_UP)) &&
@@ -6867,20 +6903,63 @@
HFI1_ODR_MASK(OPA_LINKDOWN_REASON_NOT_INSTALLED);
/* Go offline first, then deal with reading/writing through 8051 */
+ was_up = !!(ppd->host_link_state & HLS_UP);
set_link_state(ppd, HLS_DN_OFFLINE);
- lcl_reason = 0;
- read_planned_down_reason_code(ppd->dd, &neigh_reason);
+ if (was_up) {
+ lcl_reason = 0;
+ /* link down reason is only valid if the link was up */
+ read_link_down_reason(ppd->dd, &link_down_reason);
+ switch (link_down_reason) {
+ case LDR_LINK_TRANSFER_ACTIVE_LOW:
+ /* the link went down, no idle message reason */
+ dd_dev_info(ppd->dd, "%sUnexpected link down\n",
+ ldr_str);
+ break;
+ case LDR_RECEIVED_LINKDOWN_IDLE_MSG:
+ /*
+ * The neighbor reason is only valid if an idle message
+ * was received for it.
+ */
+ read_planned_down_reason_code(ppd->dd, &neigh_reason);
+ dd_dev_info(ppd->dd,
+ "%sNeighbor link down message %d, %s\n",
+ ldr_str, neigh_reason,
+ link_down_reason_str(neigh_reason));
+ break;
+ case LDR_RECEIVED_HOST_OFFLINE_REQ:
+ dd_dev_info(ppd->dd,
+ "%sHost requested link to go offline\n",
+ ldr_str);
+ break;
+ default:
+ dd_dev_info(ppd->dd, "%sUnknown reason 0x%x\n",
+ ldr_str, link_down_reason);
+ break;
+ }
- /*
- * If no reason, assume peer-initiated but missed
- * LinkGoingDown idle flits.
- */
- if (neigh_reason == 0)
- lcl_reason = OPA_LINKDOWN_REASON_NEIGHBOR_UNKNOWN;
+ /*
+ * If no reason, assume peer-initiated but missed
+ * LinkGoingDown idle flits.
+ */
+ if (neigh_reason == 0)
+ lcl_reason = OPA_LINKDOWN_REASON_NEIGHBOR_UNKNOWN;
+ } else {
+ /* went down while polling or going up */
+ lcl_reason = OPA_LINKDOWN_REASON_TRANSIENT;
+ }
set_link_down_reason(ppd, lcl_reason, neigh_reason, 0);
+ /* inform the SMA when the link transitions from up to down */
+ if (was_up && ppd->local_link_down_reason.sma == 0 &&
+ ppd->neigh_link_down_reason.sma == 0) {
+ ppd->local_link_down_reason.sma =
+ ppd->local_link_down_reason.latest;
+ ppd->neigh_link_down_reason.sma =
+ ppd->neigh_link_down_reason.latest;
+ }
+
reset_neighbor_info(ppd);
/* disable the port */
@@ -6890,7 +6969,7 @@
* If there is no cable attached, turn the DC off. Otherwise,
* start the link bring up.
*/
- if (!qsfp_mod_present(ppd)) {
+ if (ppd->port_type == PORT_TYPE_QSFP && !qsfp_mod_present(ppd)) {
dc_shutdown(ppd->dd);
} else {
tune_serdes(ppd);
@@ -7373,7 +7452,11 @@
ppd->link_width_downgrade_rx_active = rx;
}
- if (lwde == 0) {
+ if (ppd->link_width_downgrade_tx_active == 0 ||
+ ppd->link_width_downgrade_rx_active == 0) {
+ /* the 8051 reported a dead link as a downgrade */
+ dd_dev_err(ppd->dd, "Link downgrade is really a link down, ignoring\n");
+ } else if (lwde == 0) {
/* downgrade is disabled */
/* bounce if not at starting active width */
@@ -7534,7 +7617,7 @@
host_msg &= ~(u64)LINKUP_ACHIEVED;
}
if (host_msg & EXT_DEVICE_CFG_REQ) {
- queue_work(ppd->hfi1_wq, &ppd->dc_host_req_work);
+ handle_8051_request(ppd);
host_msg &= ~(u64)EXT_DEVICE_CFG_REQ;
}
if (host_msg & VERIFY_CAP_FRAME) {
@@ -8660,6 +8743,14 @@
*pdrrc = (frame >> DOWN_REMOTE_REASON_SHIFT) & DOWN_REMOTE_REASON_MASK;
}
+static void read_link_down_reason(struct hfi1_devdata *dd, u8 *ldr)
+{
+ u32 frame;
+
+ read_8051_config(dd, LINK_DOWN_REASON, GENERAL_CONFIG, &frame);
+ *ldr = (frame & 0xff);
+}
+
static int read_tx_settings(struct hfi1_devdata *dd,
u8 *enable_lane_tx,
u8 *tx_polarity_inversion,
@@ -9049,9 +9140,9 @@
}
/*
- * Call this to start the link. Schedule a retry if the cable is not
- * present or if unable to start polling. Do not do anything if the
- * link is disabled. Returns 0 if link is disabled or moved to polling
+ * Call this to start the link.
+ * Do not do anything if the link is disabled.
+ * Returns 0 if link is disabled, moved to polling, or the driver is not ready.
*/
int start_link(struct hfi1_pportdata *ppd)
{
@@ -9068,15 +9159,7 @@
return 0;
}
- if (qsfp_mod_present(ppd) || loopback == LOOPBACK_SERDES ||
- loopback == LOOPBACK_LCB ||
- ppd->dd->icode == ICODE_FUNCTIONAL_SIMULATOR)
- return set_link_state(ppd, HLS_DN_POLL);
-
- dd_dev_info(ppd->dd,
- "%s: stopping link start because no cable is present\n",
- __func__);
- return -EAGAIN;
+ return set_link_state(ppd, HLS_DN_POLL);
}
static void wait_for_qsfp_init(struct hfi1_pportdata *ppd)
@@ -9247,7 +9330,7 @@
return 0;
}
-/* This routine will only be scheduled if the QSFP module is present */
+/* This routine will only be scheduled if the QSFP module present is asserted */
void qsfp_event(struct work_struct *work)
{
struct qsfp_data *qd;
@@ -9676,6 +9759,7 @@
& SEND_LEN_CHECK1_LEN_VL15_MASK) <<
SEND_LEN_CHECK1_LEN_VL15_SHIFT;
int i;
+ u32 thres;
for (i = 0; i < ppd->vls_supported; i++) {
if (dd->vld[i].mtu > maxvlmtu)
@@ -9694,16 +9778,17 @@
/* adjust kernel credit return thresholds based on new MTUs */
/* all kernel receive contexts have the same hdrqentsize */
for (i = 0; i < ppd->vls_supported; i++) {
- sc_set_cr_threshold(dd->vld[i].sc,
- sc_mtu_to_threshold(dd->vld[i].sc,
- dd->vld[i].mtu,
- dd->rcd[0]->
- rcvhdrqentsize));
- }
- sc_set_cr_threshold(dd->vld[15].sc,
- sc_mtu_to_threshold(dd->vld[15].sc,
- dd->vld[15].mtu,
+ thres = min(sc_percent_to_threshold(dd->vld[i].sc, 50),
+ sc_mtu_to_threshold(dd->vld[i].sc,
+ dd->vld[i].mtu,
dd->rcd[0]->rcvhdrqentsize));
+ sc_set_cr_threshold(dd->vld[i].sc, thres);
+ }
+ thres = min(sc_percent_to_threshold(dd->vld[15].sc, 50),
+ sc_mtu_to_threshold(dd->vld[15].sc,
+ dd->vld[15].mtu,
+ dd->rcd[0]->rcvhdrqentsize));
+ sc_set_cr_threshold(dd->vld[15].sc, thres);
/* Adjust maximum MTU for the port in DC */
dcmtu = maxvlmtu == 10240 ? DCC_CFG_PORT_MTU_CAP_10240 :
@@ -10030,7 +10115,6 @@
struct hfi1_devdata *dd = ppd->dd;
struct ib_event event = {.device = NULL};
int ret1, ret = 0;
- int was_up, is_down;
int orig_new_state, poll_bounce;
mutex_lock(&ppd->hls_lock);
@@ -10049,8 +10133,6 @@
poll_bounce ? "(bounce) " : "",
link_state_reason_name(ppd, state));
- was_up = !!(ppd->host_link_state & HLS_UP);
-
/*
* If we're going to a (HLS_*) link state that implies the logical
* link state is neither of (IB_PORT_ARMED, IB_PORT_ACTIVE), then
@@ -10261,17 +10343,6 @@
break;
}
- is_down = !!(ppd->host_link_state & (HLS_DN_POLL |
- HLS_DN_DISABLE | HLS_DN_OFFLINE));
-
- if (was_up && is_down && ppd->local_link_down_reason.sma == 0 &&
- ppd->neigh_link_down_reason.sma == 0) {
- ppd->local_link_down_reason.sma =
- ppd->local_link_down_reason.latest;
- ppd->neigh_link_down_reason.sma =
- ppd->neigh_link_down_reason.latest;
- }
-
goto done;
unexpected:
@@ -12673,22 +12744,24 @@
int total_contexts;
int ret;
unsigned ngroups;
+ int qos_rmt_count;
+ int user_rmt_reduced;
/*
- * Kernel contexts: (to be fixed later):
- * - min or 2 or 1 context/numa
+ * Kernel receive contexts:
+ * - min of 2 or 1 context/numa (excluding control context)
* - Context 0 - control context (VL15/multicast/error)
- * - Context 1 - default context
+ * - Context 1 - first kernel context
+ * - Context 2 - second kernel context
+ * ...
*/
if (n_krcvqs)
/*
- * Don't count context 0 in n_krcvqs since
- * is isn't used for normal verbs traffic.
- *
- * krcvqs will reflect number of kernel
- * receive contexts above 0.
+ * n_krcvqs is the sum of module parameter kernel receive
+ * contexts, krcvqs[]. It does not include the control
+ * context, so add that.
*/
- num_kernel_contexts = n_krcvqs + MIN_KERNEL_KCTXTS - 1;
+ num_kernel_contexts = n_krcvqs + 1;
else
num_kernel_contexts = num_online_nodes() + 1;
num_kernel_contexts =
@@ -12705,12 +12778,13 @@
num_kernel_contexts = dd->chip_send_contexts - num_vls - 1;
}
/*
- * User contexts: (to be fixed later)
- * - default to 1 user context per CPU if num_user_contexts is
- * negative
+ * User contexts:
+ * - default to 1 user context per real (non-HT) CPU core if
+ * num_user_contexts is negative
*/
if (num_user_contexts < 0)
- num_user_contexts = num_online_cpus();
+ num_user_contexts =
+ cpumask_weight(&dd->affinity->real_cpu_mask);
total_contexts = num_kernel_contexts + num_user_contexts;
@@ -12727,6 +12801,19 @@
total_contexts = num_kernel_contexts + num_user_contexts;
}
+ /* each user context requires an entry in the RMT */
+ qos_rmt_count = qos_rmt_entries(dd, NULL, NULL);
+ if (qos_rmt_count + num_user_contexts > NUM_MAP_ENTRIES) {
+ user_rmt_reduced = NUM_MAP_ENTRIES - qos_rmt_count;
+ dd_dev_err(dd,
+ "RMT size is reducing the number of user receive contexts from %d to %d\n",
+ (int)num_user_contexts,
+ user_rmt_reduced);
+ /* recalculate */
+ num_user_contexts = user_rmt_reduced;
+ total_contexts = num_kernel_contexts + num_user_contexts;
+ }
+
/* the first N are kernel contexts, the rest are user contexts */
dd->num_rcv_contexts = total_contexts;
dd->n_krcv_queues = num_kernel_contexts;
@@ -12776,12 +12863,13 @@
dd->num_send_contexts = ret;
dd_dev_info(
dd,
- "send contexts: chip %d, used %d (kernel %d, ack %d, user %d)\n",
+ "send contexts: chip %d, used %d (kernel %d, ack %d, user %d, vl15 %d)\n",
dd->chip_send_contexts,
dd->num_send_contexts,
dd->sc_sizes[SC_KERNEL].count,
dd->sc_sizes[SC_ACK].count,
- dd->sc_sizes[SC_USER].count);
+ dd->sc_sizes[SC_USER].count,
+ dd->sc_sizes[SC_VL15].count);
ret = 0; /* success */
}
@@ -13451,122 +13539,224 @@
int i;
u64 ctxt = first_ctxt;
- for (i = 0; i < 256;) {
+ for (i = 0; i < 256; i++) {
reg |= ctxt << (8 * (i % 8));
- i++;
ctxt++;
if (ctxt > last_ctxt)
ctxt = first_ctxt;
- if (i % 8 == 0) {
+ if (i % 8 == 7) {
write_csr(dd, regno, reg);
reg = 0;
regno += 8;
}
}
- if (i % 8)
- write_csr(dd, regno, reg);
add_rcvctrl(dd, RCV_CTRL_RCV_QP_MAP_ENABLE_SMASK
| RCV_CTRL_RCV_BYPASS_ENABLE_SMASK);
}
-/**
- * init_qos - init RX qos
- * @dd - device data
- * @first_context
- *
- * This routine initializes Rule 0 and the
- * RSM map table to implement qos.
- *
- * If all of the limit tests succeed,
- * qos is applied based on the array
- * interpretation of krcvqs where
- * entry 0 is VL0.
- *
- * The number of vl bits (n) and the number of qpn
- * bits (m) are computed to feed both the RSM map table
- * and the single rule.
- *
- */
-static void init_qos(struct hfi1_devdata *dd, u32 first_ctxt)
-{
- u8 max_by_vl = 0;
- unsigned qpns_per_vl, ctxt, i, qpn, n = 1, m;
- u64 *rsmmap;
- u64 reg;
- u8 rxcontext = is_ax(dd) ? 0 : 0xff; /* 0 is default if a0 ver. */
+struct rsm_map_table {
+ u64 map[NUM_MAP_REGS];
+ unsigned int used;
+};
- /* validate */
+struct rsm_rule_data {
+ u8 offset;
+ u8 pkt_type;
+ u32 field1_off;
+ u32 field2_off;
+ u32 index1_off;
+ u32 index1_width;
+ u32 index2_off;
+ u32 index2_width;
+ u32 mask1;
+ u32 value1;
+ u32 mask2;
+ u32 value2;
+};
+
+/*
+ * Return an initialized RMT map table for users to fill in. OK if it
+ * returns NULL, indicating no table.
+ */
+static struct rsm_map_table *alloc_rsm_map_table(struct hfi1_devdata *dd)
+{
+ struct rsm_map_table *rmt;
+ u8 rxcontext = is_ax(dd) ? 0 : 0xff; /* 0 is default if a0 ver. */
+
+ rmt = kmalloc(sizeof(*rmt), GFP_KERNEL);
+ if (rmt) {
+ memset(rmt->map, rxcontext, sizeof(rmt->map));
+ rmt->used = 0;
+ }
+
+ return rmt;
+}
+
+/*
+ * Write the final RMT map table to the chip and free the table. OK if
+ * table is NULL.
+ */
+static void complete_rsm_map_table(struct hfi1_devdata *dd,
+ struct rsm_map_table *rmt)
+{
+ int i;
+
+ if (rmt) {
+ /* write table to chip */
+ for (i = 0; i < NUM_MAP_REGS; i++)
+ write_csr(dd, RCV_RSM_MAP_TABLE + (8 * i), rmt->map[i]);
+
+ /* enable RSM */
+ add_rcvctrl(dd, RCV_CTRL_RCV_RSM_ENABLE_SMASK);
+ }
+}
+
+/*
+ * Add a receive side mapping rule.
+ */
+static void add_rsm_rule(struct hfi1_devdata *dd, u8 rule_index,
+ struct rsm_rule_data *rrd)
+{
+ write_csr(dd, RCV_RSM_CFG + (8 * rule_index),
+ (u64)rrd->offset << RCV_RSM_CFG_OFFSET_SHIFT |
+ 1ull << rule_index | /* enable bit */
+ (u64)rrd->pkt_type << RCV_RSM_CFG_PACKET_TYPE_SHIFT);
+ write_csr(dd, RCV_RSM_SELECT + (8 * rule_index),
+ (u64)rrd->field1_off << RCV_RSM_SELECT_FIELD1_OFFSET_SHIFT |
+ (u64)rrd->field2_off << RCV_RSM_SELECT_FIELD2_OFFSET_SHIFT |
+ (u64)rrd->index1_off << RCV_RSM_SELECT_INDEX1_OFFSET_SHIFT |
+ (u64)rrd->index1_width << RCV_RSM_SELECT_INDEX1_WIDTH_SHIFT |
+ (u64)rrd->index2_off << RCV_RSM_SELECT_INDEX2_OFFSET_SHIFT |
+ (u64)rrd->index2_width << RCV_RSM_SELECT_INDEX2_WIDTH_SHIFT);
+ write_csr(dd, RCV_RSM_MATCH + (8 * rule_index),
+ (u64)rrd->mask1 << RCV_RSM_MATCH_MASK1_SHIFT |
+ (u64)rrd->value1 << RCV_RSM_MATCH_VALUE1_SHIFT |
+ (u64)rrd->mask2 << RCV_RSM_MATCH_MASK2_SHIFT |
+ (u64)rrd->value2 << RCV_RSM_MATCH_VALUE2_SHIFT);
+}
+
+/* return the number of RSM map table entries that will be used for QOS */
+static int qos_rmt_entries(struct hfi1_devdata *dd, unsigned int *mp,
+ unsigned int *np)
+{
+ int i;
+ unsigned int m, n;
+ u8 max_by_vl = 0;
+
+ /* is QOS active at all? */
if (dd->n_krcv_queues <= MIN_KERNEL_KCTXTS ||
num_vls == 1 ||
krcvqsset <= 1)
- goto bail;
- for (i = 0; i < min_t(unsigned, num_vls, krcvqsset); i++)
+ goto no_qos;
+
+ /* determine bits for qpn */
+ for (i = 0; i < min_t(unsigned int, num_vls, krcvqsset); i++)
if (krcvqs[i] > max_by_vl)
max_by_vl = krcvqs[i];
if (max_by_vl > 32)
- goto bail;
- qpns_per_vl = __roundup_pow_of_two(max_by_vl);
- /* determine bits vl */
- n = ilog2(num_vls);
- /* determine bits for qpn */
- m = ilog2(qpns_per_vl);
+ goto no_qos;
+ m = ilog2(__roundup_pow_of_two(max_by_vl));
+
+ /* determine bits for vl */
+ n = ilog2(__roundup_pow_of_two(num_vls));
+
+ /* reject if too much is used */
if ((m + n) > 7)
+ goto no_qos;
+
+ if (mp)
+ *mp = m;
+ if (np)
+ *np = n;
+
+ return 1 << (m + n);
+
+no_qos:
+ if (mp)
+ *mp = 0;
+ if (np)
+ *np = 0;
+ return 0;
+}
+
+/**
+ * init_qos - init RX qos
+ * @dd - device data
+ * @rmt - RSM map table
+ *
+ * This routine initializes Rule 0 and the RSM map table to implement
+ * quality of service (qos).
+ *
+ * If all of the limit tests succeed, qos is applied based on the array
+ * interpretation of krcvqs where entry 0 is VL0.
+ *
+ * The number of vl bits (n) and the number of qpn bits (m) are computed to
+ * feed both the RSM map table and the single rule.
+ */
+static void init_qos(struct hfi1_devdata *dd, struct rsm_map_table *rmt)
+{
+ struct rsm_rule_data rrd;
+ unsigned qpns_per_vl, ctxt, i, qpn, n = 1, m;
+ unsigned int rmt_entries;
+ u64 reg;
+
+ if (!rmt)
goto bail;
- if (num_vls * qpns_per_vl > dd->chip_rcv_contexts)
+ rmt_entries = qos_rmt_entries(dd, &m, &n);
+ if (rmt_entries == 0)
goto bail;
- rsmmap = kmalloc_array(NUM_MAP_REGS, sizeof(u64), GFP_KERNEL);
- if (!rsmmap)
+ qpns_per_vl = 1 << m;
+
+ /* enough room in the map table? */
+ rmt_entries = 1 << (m + n);
+ if (rmt->used + rmt_entries >= NUM_MAP_ENTRIES)
goto bail;
- memset(rsmmap, rxcontext, NUM_MAP_REGS * sizeof(u64));
- /* init the local copy of the table */
- for (i = 0, ctxt = first_ctxt; i < num_vls; i++) {
+
+ /* add qos entries to the the RSM map table */
+ for (i = 0, ctxt = FIRST_KERNEL_KCTXT; i < num_vls; i++) {
unsigned tctxt;
for (qpn = 0, tctxt = ctxt;
krcvqs[i] && qpn < qpns_per_vl; qpn++) {
unsigned idx, regoff, regidx;
- /* generate index <= 128 */
- idx = (qpn << n) ^ i;
+ /* generate the index the hardware will produce */
+ idx = rmt->used + ((qpn << n) ^ i);
regoff = (idx % 8) * 8;
regidx = idx / 8;
- reg = rsmmap[regidx];
- /* replace 0xff with context number */
+ /* replace default with context number */
+ reg = rmt->map[regidx];
reg &= ~(RCV_RSM_MAP_TABLE_RCV_CONTEXT_A_MASK
<< regoff);
reg |= (u64)(tctxt++) << regoff;
- rsmmap[regidx] = reg;
+ rmt->map[regidx] = reg;
if (tctxt == ctxt + krcvqs[i])
tctxt = ctxt;
}
ctxt += krcvqs[i];
}
- /* flush cached copies to chip */
- for (i = 0; i < NUM_MAP_REGS; i++)
- write_csr(dd, RCV_RSM_MAP_TABLE + (8 * i), rsmmap[i]);
- /* add rule0 */
- write_csr(dd, RCV_RSM_CFG /* + (8 * 0) */,
- RCV_RSM_CFG_ENABLE_OR_CHAIN_RSM0_MASK <<
- RCV_RSM_CFG_ENABLE_OR_CHAIN_RSM0_SHIFT |
- 2ull << RCV_RSM_CFG_PACKET_TYPE_SHIFT);
- write_csr(dd, RCV_RSM_SELECT /* + (8 * 0) */,
- LRH_BTH_MATCH_OFFSET << RCV_RSM_SELECT_FIELD1_OFFSET_SHIFT |
- LRH_SC_MATCH_OFFSET << RCV_RSM_SELECT_FIELD2_OFFSET_SHIFT |
- LRH_SC_SELECT_OFFSET << RCV_RSM_SELECT_INDEX1_OFFSET_SHIFT |
- ((u64)n) << RCV_RSM_SELECT_INDEX1_WIDTH_SHIFT |
- QPN_SELECT_OFFSET << RCV_RSM_SELECT_INDEX2_OFFSET_SHIFT |
- ((u64)m + (u64)n) << RCV_RSM_SELECT_INDEX2_WIDTH_SHIFT);
- write_csr(dd, RCV_RSM_MATCH /* + (8 * 0) */,
- LRH_BTH_MASK << RCV_RSM_MATCH_MASK1_SHIFT |
- LRH_BTH_VALUE << RCV_RSM_MATCH_VALUE1_SHIFT |
- LRH_SC_MASK << RCV_RSM_MATCH_MASK2_SHIFT |
- LRH_SC_VALUE << RCV_RSM_MATCH_VALUE2_SHIFT);
- /* Enable RSM */
- add_rcvctrl(dd, RCV_CTRL_RCV_RSM_ENABLE_SMASK);
- kfree(rsmmap);
- /* map everything else to first context */
- init_qpmap_table(dd, FIRST_KERNEL_KCTXT, MIN_KERNEL_KCTXTS - 1);
+
+ rrd.offset = rmt->used;
+ rrd.pkt_type = 2;
+ rrd.field1_off = LRH_BTH_MATCH_OFFSET;
+ rrd.field2_off = LRH_SC_MATCH_OFFSET;
+ rrd.index1_off = LRH_SC_SELECT_OFFSET;
+ rrd.index1_width = n;
+ rrd.index2_off = QPN_SELECT_OFFSET;
+ rrd.index2_width = m + n;
+ rrd.mask1 = LRH_BTH_MASK;
+ rrd.value1 = LRH_BTH_VALUE;
+ rrd.mask2 = LRH_SC_MASK;
+ rrd.value2 = LRH_SC_VALUE;
+
+ /* add rule 0 */
+ add_rsm_rule(dd, 0, &rrd);
+
+ /* mark RSM map entries as used */
+ rmt->used += rmt_entries;
+ /* map everything else to the mcast/err/vl15 context */
+ init_qpmap_table(dd, HFI1_CTRL_CTXT, HFI1_CTRL_CTXT);
dd->qos_shift = n + 1;
return;
bail:
@@ -13574,13 +13764,86 @@
init_qpmap_table(dd, FIRST_KERNEL_KCTXT, dd->n_krcv_queues - 1);
}
+static void init_user_fecn_handling(struct hfi1_devdata *dd,
+ struct rsm_map_table *rmt)
+{
+ struct rsm_rule_data rrd;
+ u64 reg;
+ int i, idx, regoff, regidx;
+ u8 offset;
+
+ /* there needs to be enough room in the map table */
+ if (rmt->used + dd->num_user_contexts >= NUM_MAP_ENTRIES) {
+ dd_dev_err(dd, "User FECN handling disabled - too many user contexts allocated\n");
+ return;
+ }
+
+ /*
+ * RSM will extract the destination context as an index into the
+ * map table. The destination contexts are a sequential block
+ * in the range first_user_ctxt...num_rcv_contexts-1 (inclusive).
+ * Map entries are accessed as offset + extracted value. Adjust
+ * the added offset so this sequence can be placed anywhere in
+ * the table - as long as the entries themselves do not wrap.
+ * There are only enough bits in offset for the table size, so
+ * start with that to allow for a "negative" offset.
+ */
+ offset = (u8)(NUM_MAP_ENTRIES + (int)rmt->used -
+ (int)dd->first_user_ctxt);
+
+ for (i = dd->first_user_ctxt, idx = rmt->used;
+ i < dd->num_rcv_contexts; i++, idx++) {
+ /* replace with identity mapping */
+ regoff = (idx % 8) * 8;
+ regidx = idx / 8;
+ reg = rmt->map[regidx];
+ reg &= ~(RCV_RSM_MAP_TABLE_RCV_CONTEXT_A_MASK << regoff);
+ reg |= (u64)i << regoff;
+ rmt->map[regidx] = reg;
+ }
+
+ /*
+ * For RSM intercept of Expected FECN packets:
+ * o packet type 0 - expected
+ * o match on F (bit 95), using select/match 1, and
+ * o match on SH (bit 133), using select/match 2.
+ *
+ * Use index 1 to extract the 8-bit receive context from DestQP
+ * (start at bit 64). Use that as the RSM map table index.
+ */
+ rrd.offset = offset;
+ rrd.pkt_type = 0;
+ rrd.field1_off = 95;
+ rrd.field2_off = 133;
+ rrd.index1_off = 64;
+ rrd.index1_width = 8;
+ rrd.index2_off = 0;
+ rrd.index2_width = 0;
+ rrd.mask1 = 1;
+ rrd.value1 = 1;
+ rrd.mask2 = 1;
+ rrd.value2 = 1;
+
+ /* add rule 1 */
+ add_rsm_rule(dd, 1, &rrd);
+
+ rmt->used += dd->num_user_contexts;
+}
+
static void init_rxe(struct hfi1_devdata *dd)
{
+ struct rsm_map_table *rmt;
+
/* enable all receive errors */
write_csr(dd, RCV_ERR_MASK, ~0ull);
- /* setup QPN map table - start where VL15 context leaves off */
- init_qos(dd, dd->n_krcv_queues > MIN_KERNEL_KCTXTS ?
- MIN_KERNEL_KCTXTS : 0);
+
+ rmt = alloc_rsm_map_table(dd);
+ /* set up QOS, including the QPN map table */
+ init_qos(dd, rmt);
+ init_user_fecn_handling(dd, rmt);
+ complete_rsm_map_table(dd, rmt);
+ kfree(rmt);
+
/*
* make sure RcvCtrl.RcvWcb <= PCIe Device Control
* Register Max_Payload_Size (PCI_EXP_DEVCTL in Linux PCIe config
@@ -13762,6 +14025,7 @@
write_kctxt_csr(dd, sctxt, SEND_CTXT_CHECK_PARTITION_KEY, reg);
reg = read_kctxt_csr(dd, sctxt, SEND_CTXT_CHECK_ENABLE);
reg |= SEND_CTXT_CHECK_ENABLE_CHECK_PARTITION_KEY_SMASK;
+ reg &= ~SEND_CTXT_CHECK_ENABLE_DISALLOW_KDETH_PACKETS_SMASK;
write_kctxt_csr(dd, sctxt, SEND_CTXT_CHECK_ENABLE, reg);
done:
return ret;
@@ -14148,6 +14412,19 @@
(dd->revision >> CCE_REVISION_SW_SHIFT)
& CCE_REVISION_SW_MASK);
+ /*
+ * The real cpu mask is part of the affinity struct but has to be
+ * initialized earlier than the rest of the affinity struct because it
+ * is needed to calculate the number of user contexts in
+ * set_up_context_variables(). However, hfi1_dev_affinity_init(),
+ * which initializes the rest of the affinity struct members,
+ * depends on set_up_context_variables() for the number of kernel
+ * contexts, so it cannot be called before set_up_context_variables().
+ */
+ ret = init_real_cpu_mask(dd);
+ if (ret)
+ goto bail_cleanup;
+
ret = set_up_context_variables(dd);
if (ret)
goto bail_cleanup;
@@ -14161,9 +14438,7 @@
/* set up KDETH QP prefix in both RX and TX CSRs */
init_kdeth_qp(dd);
- ret = hfi1_dev_affinity_init(dd);
- if (ret)
- goto bail_cleanup;
+ hfi1_dev_affinity_init(dd);
/* send contexts must be set up before receive contexts */
ret = init_send_contexts(dd);
diff --git a/drivers/staging/rdma/hfi1/chip.h b/drivers/staging/rdma/hfi1/chip.h
index 4f3b878..1948706 100644
--- a/drivers/staging/rdma/hfi1/chip.h
+++ b/drivers/staging/rdma/hfi1/chip.h
@@ -389,6 +389,7 @@
#define LAST_REMOTE_STATE_COMPLETE 0x13
#define LINK_QUALITY_INFO 0x14
#define REMOTE_DEVICE_ID 0x15
+#define LINK_DOWN_REASON 0x16
/* 8051 lane specific register field IDs */
#define TX_EQ_SETTINGS 0x00
@@ -497,6 +498,11 @@
#define PWRM_BER_CONTROL 0x1
#define PWRM_BANDWIDTH_CONTROL 0x2
+/* 8051 link down reasons */
+#define LDR_LINK_TRANSFER_ACTIVE_LOW 0xa
+#define LDR_RECEIVED_LINKDOWN_IDLE_MSG 0xb
+#define LDR_RECEIVED_HOST_OFFLINE_REQ 0xc
+
/* verify capability fabric CRC size bits */
enum {
CAP_CRC_14B = (1 << 0), /* 14b CRC */
@@ -691,7 +697,6 @@
void handle_freeze(struct work_struct *work);
void handle_link_up(struct work_struct *work);
void handle_link_down(struct work_struct *work);
-void handle_8051_request(struct work_struct *work);
void handle_link_downgrade(struct work_struct *work);
void handle_link_bounce(struct work_struct *work);
void handle_sma_message(struct work_struct *work);
diff --git a/drivers/staging/rdma/hfi1/chip_registers.h b/drivers/staging/rdma/hfi1/chip_registers.h
index 770f05c..8744de6 100644
--- a/drivers/staging/rdma/hfi1/chip_registers.h
+++ b/drivers/staging/rdma/hfi1/chip_registers.h
@@ -771,6 +771,7 @@
#define RCV_RSM_CFG_ENABLE_OR_CHAIN_RSM0_MASK 0x1ull
#define RCV_RSM_CFG_ENABLE_OR_CHAIN_RSM0_SHIFT 0
#define RCV_RSM_CFG_PACKET_TYPE_SHIFT 60
+#define RCV_RSM_CFG_OFFSET_SHIFT 32
#define RCV_RSM_MAP_TABLE (RXE + 0x000000000900)
#define RCV_RSM_MAP_TABLE_RCV_CONTEXT_A_MASK 0xFFull
#define RCV_RSM_MATCH (RXE + 0x000000000800)
diff --git a/drivers/staging/rdma/hfi1/diag.c b/drivers/staging/rdma/hfi1/diag.c
index c5b520b..bb2409a 100644
--- a/drivers/staging/rdma/hfi1/diag.c
+++ b/drivers/staging/rdma/hfi1/diag.c
@@ -413,7 +413,8 @@
goto bail;
}
/* can only use kernel contexts */
- if (dd->send_contexts[dp->sw_index].type != SC_KERNEL) {
+ if (dd->send_contexts[dp->sw_index].type != SC_KERNEL &&
+ dd->send_contexts[dp->sw_index].type != SC_VL15) {
ret = -EINVAL;
goto bail;
}
diff --git a/drivers/staging/rdma/hfi1/driver.c b/drivers/staging/rdma/hfi1/driver.c
index 34511e5..700c6fa 100644
--- a/drivers/staging/rdma/hfi1/driver.c
+++ b/drivers/staging/rdma/hfi1/driver.c
@@ -75,7 +75,8 @@
unsigned int hfi1_max_mtu = HFI1_DEFAULT_MAX_MTU;
module_param_named(max_mtu, hfi1_max_mtu, uint, S_IRUGO);
-MODULE_PARM_DESC(max_mtu, "Set max MTU bytes, default is 8192");
+MODULE_PARM_DESC(max_mtu, "Set max MTU bytes, default is " __stringify(
+ HFI1_DEFAULT_MAX_MTU));
unsigned int hfi1_cu = 1;
module_param_named(cu, hfi1_cu, uint, S_IRUGO);
diff --git a/drivers/staging/rdma/hfi1/firmware.c b/drivers/staging/rdma/hfi1/firmware.c
index 3040162..ed680fd 100644
--- a/drivers/staging/rdma/hfi1/firmware.c
+++ b/drivers/staging/rdma/hfi1/firmware.c
@@ -1413,8 +1413,15 @@
if (resource & CR_DYN_MASK) {
/* a dynamic resource is in use if either HFI has set the bit */
- all_bits = resource_mask(0, resource) |
+ if (dd->pcidev->device == PCI_DEVICE_ID_INTEL0 &&
+ (resource & (CR_I2C1 | CR_I2C2))) {
+ /* discrete devices must serialize across both chains */
+ all_bits = resource_mask(0, CR_I2C1 | CR_I2C2) |
+ resource_mask(1, CR_I2C1 | CR_I2C2);
+ } else {
+ all_bits = resource_mask(0, resource) |
resource_mask(1, resource);
+ }
my_bit = resource_mask(dd->hfi1_id, resource);
} else {
/* non-dynamic resources are not split between HFIs */
diff --git a/drivers/staging/rdma/hfi1/hfi.h b/drivers/staging/rdma/hfi1/hfi.h
index 16cbdc4..7b78d56 100644
--- a/drivers/staging/rdma/hfi1/hfi.h
+++ b/drivers/staging/rdma/hfi1/hfi.h
@@ -455,9 +455,9 @@
#define HLS_UP (HLS_UP_INIT | HLS_UP_ARMED | HLS_UP_ACTIVE)
/* use this MTU size if none other is given */
-#define HFI1_DEFAULT_ACTIVE_MTU 8192
+#define HFI1_DEFAULT_ACTIVE_MTU 10240
/* use this MTU size as the default maximum */
-#define HFI1_DEFAULT_MAX_MTU 8192
+#define HFI1_DEFAULT_MAX_MTU 10240
/* default partition key */
#define DEFAULT_PKEY 0xffff
@@ -606,7 +606,6 @@
struct work_struct link_vc_work;
struct work_struct link_up_work;
struct work_struct link_down_work;
- struct work_struct dc_host_req_work;
struct work_struct sma_message_work;
struct work_struct freeze_work;
struct work_struct link_downgrade_work;
@@ -1258,7 +1257,7 @@
static inline int hdr2sc(struct hfi1_message_header *hdr, u64 rhf)
{
return ((be16_to_cpu(hdr->lrh[0]) >> 12) & 0xf) |
- ((!!(rhf & RHF_DC_INFO_MASK)) << 4);
+ ((!!(rhf & RHF_DC_INFO_SMASK)) << 4);
}
static inline u16 generate_jkey(kuid_t uid)
@@ -1333,6 +1332,9 @@
void return_cnp(struct hfi1_ibport *ibp, struct rvt_qp *qp, u32 remote_qpn,
u32 pkey, u32 slid, u32 dlid, u8 sc5,
const struct ib_grh *old_grh);
+#define PKEY_CHECK_INVALID -1
+int egress_pkey_check(struct hfi1_pportdata *ppd, __be16 *lrh, __be32 *bth,
+ u8 sc5, int8_t s_pkey_index);
#define PACKET_EGRESS_TIMEOUT 350
static inline void pause_for_credit_return(struct hfi1_devdata *dd)
@@ -1776,6 +1778,7 @@
#define HFI1_PKT_USER_SC_INTEGRITY \
(SEND_CTXT_CHECK_ENABLE_DISALLOW_NON_KDETH_PACKETS_SMASK \
+ | SEND_CTXT_CHECK_ENABLE_DISALLOW_KDETH_PACKETS_SMASK \
| SEND_CTXT_CHECK_ENABLE_DISALLOW_BYPASS_SMASK \
| SEND_CTXT_CHECK_ENABLE_DISALLOW_GRH_SMASK)
diff --git a/drivers/staging/rdma/hfi1/init.c b/drivers/staging/rdma/hfi1/init.c
index cfcdc16..502b7cf 100644
--- a/drivers/staging/rdma/hfi1/init.c
+++ b/drivers/staging/rdma/hfi1/init.c
@@ -422,9 +422,10 @@
struct cca_timer *cca_timer;
struct hfi1_pportdata *ppd;
int sl;
- u16 ccti, ccti_timer, ccti_min;
+ u16 ccti_timer, ccti_min;
struct cc_state *cc_state;
unsigned long flags;
+ enum hrtimer_restart ret = HRTIMER_NORESTART;
cca_timer = container_of(t, struct cca_timer, hrtimer);
ppd = cca_timer->ppd;
@@ -450,24 +451,21 @@
spin_lock_irqsave(&ppd->cca_timer_lock, flags);
- ccti = cca_timer->ccti;
-
- if (ccti > ccti_min) {
+ if (cca_timer->ccti > ccti_min) {
cca_timer->ccti--;
set_link_ipg(ppd);
}
- spin_unlock_irqrestore(&ppd->cca_timer_lock, flags);
-
- rcu_read_unlock();
-
- if (ccti > ccti_min) {
+ if (cca_timer->ccti > ccti_min) {
unsigned long nsec = 1024 * ccti_timer;
/* ccti_timer is in units of 1.024 usec */
hrtimer_forward_now(t, ns_to_ktime(nsec));
- return HRTIMER_RESTART;
+ ret = HRTIMER_RESTART;
}
- return HRTIMER_NORESTART;
+
+ spin_unlock_irqrestore(&ppd->cca_timer_lock, flags);
+ rcu_read_unlock();
+ return ret;
}
/*
@@ -496,7 +494,6 @@
INIT_WORK(&ppd->link_vc_work, handle_verify_cap);
INIT_WORK(&ppd->link_up_work, handle_link_up);
INIT_WORK(&ppd->link_down_work, handle_link_down);
- INIT_WORK(&ppd->dc_host_req_work, handle_8051_request);
INIT_WORK(&ppd->freeze_work, handle_freeze);
INIT_WORK(&ppd->link_downgrade_work, handle_link_downgrade);
INIT_WORK(&ppd->sma_message_work, handle_sma_message);
@@ -1007,7 +1004,7 @@
free_percpu(dd->rcv_limit);
hfi1_dev_affinity_free(dd);
free_percpu(dd->send_schedule);
- ib_dealloc_device(&dd->verbs_dev.rdi.ibdev);
+ rvt_dealloc_device(&dd->verbs_dev.rdi);
}
/*
@@ -1110,7 +1107,7 @@
bail:
if (!list_empty(&dd->list))
list_del_init(&dd->list);
- ib_dealloc_device(&dd->verbs_dev.rdi.ibdev);
+ rvt_dealloc_device(&dd->verbs_dev.rdi);
return ERR_PTR(ret);
}
diff --git a/drivers/staging/rdma/hfi1/mad.c b/drivers/staging/rdma/hfi1/mad.c
index d1e7f4d..ed58cf2 100644
--- a/drivers/staging/rdma/hfi1/mad.c
+++ b/drivers/staging/rdma/hfi1/mad.c
@@ -999,7 +999,21 @@
break;
}
- set_link_state(ppd, link_state);
+ if ((link_state == HLS_DN_POLL ||
+ link_state == HLS_DN_DOWNDEF)) {
+ /*
+ * Going to poll. No matter what the current state,
+ * always move offline first, then tune and start the
+ * link. This correctly handles a FM link bounce and
+ * a link enable. Going offline is a no-op if already
+ * offline.
+ */
+ set_link_state(ppd, HLS_DN_OFFLINE);
+ tune_serdes(ppd);
+ start_link(ppd);
+ } else {
+ set_link_state(ppd, link_state);
+ }
if (link_state == HLS_DN_DISABLE &&
(ppd->offline_disabled_reason >
HFI1_ODR_MASK(OPA_LINKDOWN_REASON_SMA_DISABLED) ||
diff --git a/drivers/staging/rdma/hfi1/mmu_rb.c b/drivers/staging/rdma/hfi1/mmu_rb.c
index b3f0682..2b0e91d 100644
--- a/drivers/staging/rdma/hfi1/mmu_rb.c
+++ b/drivers/staging/rdma/hfi1/mmu_rb.c
@@ -91,7 +91,7 @@
static unsigned long mmu_node_last(struct mmu_rb_node *node)
{
- return PAGE_ALIGN((node->addr & PAGE_MASK) + node->len) - 1;
+ return PAGE_ALIGN(node->addr + node->len) - 1;
}
int hfi1_mmu_rb_register(struct rb_root *root, struct mmu_rb_ops *ops)
@@ -126,10 +126,15 @@
if (!handler)
return;
+ /* Unregister first so we don't get any more notifications. */
+ if (current->mm)
+ mmu_notifier_unregister(&handler->mn, current->mm);
+
spin_lock_irqsave(&mmu_rb_lock, flags);
list_del(&handler->list);
spin_unlock_irqrestore(&mmu_rb_lock, flags);
+ spin_lock_irqsave(&handler->lock, flags);
if (!RB_EMPTY_ROOT(root)) {
struct rb_node *node;
struct mmu_rb_node *rbnode;
@@ -141,9 +146,8 @@
handler->ops->remove(root, rbnode, NULL);
}
}
+ spin_unlock_irqrestore(&handler->lock, flags);
- if (current->mm)
- mmu_notifier_unregister(&handler->mn, current->mm);
kfree(handler);
}
@@ -235,6 +239,25 @@
return node;
}
+struct mmu_rb_node *hfi1_mmu_rb_extract(struct rb_root *root,
+ unsigned long addr, unsigned long len)
+{
+ struct mmu_rb_handler *handler = find_mmu_handler(root);
+ struct mmu_rb_node *node;
+ unsigned long flags;
+
+ if (!handler)
+ return ERR_PTR(-EINVAL);
+
+ spin_lock_irqsave(&handler->lock, flags);
+ node = __mmu_rb_search(handler, addr, len);
+ if (node)
+ __mmu_int_rb_remove(node, handler->root);
+ spin_unlock_irqrestore(&handler->lock, flags);
+
+ return node;
+}
+
void hfi1_mmu_rb_remove(struct rb_root *root, struct mmu_rb_node *node)
{
struct mmu_rb_handler *handler = find_mmu_handler(root);
@@ -293,9 +316,9 @@
hfi1_cdbg(MMU, "Invalidating node addr 0x%llx, len %u",
node->addr, node->len);
if (handler->ops->invalidate(root, node)) {
- spin_unlock_irqrestore(&handler->lock, flags);
- __mmu_rb_remove(handler, node, mm);
- spin_lock_irqsave(&handler->lock, flags);
+ __mmu_int_rb_remove(node, root);
+ if (handler->ops->remove)
+ handler->ops->remove(root, node, mm);
}
}
spin_unlock_irqrestore(&handler->lock, flags);
diff --git a/drivers/staging/rdma/hfi1/mmu_rb.h b/drivers/staging/rdma/hfi1/mmu_rb.h
index 19a306e..7a57b9c4 100644
--- a/drivers/staging/rdma/hfi1/mmu_rb.h
+++ b/drivers/staging/rdma/hfi1/mmu_rb.h
@@ -70,5 +70,7 @@
void hfi1_mmu_rb_remove(struct rb_root *, struct mmu_rb_node *);
struct mmu_rb_node *hfi1_mmu_rb_search(struct rb_root *, unsigned long,
unsigned long);
+struct mmu_rb_node *hfi1_mmu_rb_extract(struct rb_root *, unsigned long,
+ unsigned long);
#endif /* _HFI1_MMU_RB_H */
diff --git a/drivers/staging/rdma/hfi1/pio.c b/drivers/staging/rdma/hfi1/pio.c
index c6849ce..c67b9ad 100644
--- a/drivers/staging/rdma/hfi1/pio.c
+++ b/drivers/staging/rdma/hfi1/pio.c
@@ -139,23 +139,30 @@
/* Send Context Size (SCS) wildcards */
#define SCS_POOL_0 -1
#define SCS_POOL_1 -2
+
/* Send Context Count (SCC) wildcards */
#define SCC_PER_VL -1
#define SCC_PER_CPU -2
-
#define SCC_PER_KRCVQ -3
-#define SCC_ACK_CREDITS 32
+
+/* Send Context Size (SCS) constants */
+#define SCS_ACK_CREDITS 32
+#define SCS_VL15_CREDITS 102 /* 3 pkts of 2048B data + 128B header */
+
+#define PIO_THRESHOLD_CEILING 4096
#define PIO_WAIT_BATCH_SIZE 5
/* default send context sizes */
static struct sc_config_sizes sc_config_sizes[SC_MAX] = {
[SC_KERNEL] = { .size = SCS_POOL_0, /* even divide, pool 0 */
- .count = SCC_PER_VL },/* one per NUMA */
- [SC_ACK] = { .size = SCC_ACK_CREDITS,
+ .count = SCC_PER_VL }, /* one per NUMA */
+ [SC_ACK] = { .size = SCS_ACK_CREDITS,
.count = SCC_PER_KRCVQ },
[SC_USER] = { .size = SCS_POOL_0, /* even divide, pool 0 */
.count = SCC_PER_CPU }, /* one per CPU */
+ [SC_VL15] = { .size = SCS_VL15_CREDITS,
+ .count = 1 },
};
@@ -202,7 +209,8 @@
static const char *sc_type_names[SC_MAX] = {
"kernel",
"ack",
- "user"
+ "user",
+ "vl15"
};
static const char *sc_type_name(int index)
@@ -231,6 +239,22 @@
int i;
/*
+ * When SDMA is enabled, kernel context pio packet size is capped by
+ * "piothreshold". Reduce pio buffer allocation for kernel context by
+ * setting it to a fixed size. The allocation allows 3-deep buffering
+ * of the largest pio packets plus up to 128 bytes header, sufficient
+ * to maintain verbs performance.
+ *
+ * When SDMA is disabled, keep the default pooling allocation.
+ */
+ if (HFI1_CAP_IS_KSET(SDMA)) {
+ u16 max_pkt_size = (piothreshold < PIO_THRESHOLD_CEILING) ?
+ piothreshold : PIO_THRESHOLD_CEILING;
+ sc_config_sizes[SC_KERNEL].size =
+ 3 * (max_pkt_size + 128) / PIO_BLOCK_SIZE;
+ }
+
+ /*
* Step 0:
* - copy the centipercents/absolute sizes from the pool config
* - sanity check these values
@@ -311,7 +335,7 @@
if (i == SC_ACK) {
count = dd->n_krcv_queues;
} else if (i == SC_KERNEL) {
- count = (INIT_SC_PER_VL * num_vls) + 1 /* VL15 */;
+ count = INIT_SC_PER_VL * num_vls;
} else if (count == SCC_PER_CPU) {
count = dd->num_rcv_contexts - dd->n_krcv_queues;
} else if (count < 0) {
@@ -596,7 +620,7 @@
* Return value is what to write into the CSR: trigger return when
* unreturned credits pass this count.
*/
-static u32 sc_percent_to_threshold(struct send_context *sc, u32 percent)
+u32 sc_percent_to_threshold(struct send_context *sc, u32 percent)
{
return (sc->credits * percent) / 100;
}
@@ -790,7 +814,10 @@
* For Ack contexts, set a threshold for half the credits.
* For User contexts use the given percentage. This has been
* sanitized on driver start-up.
- * For Kernel contexts, use the default MTU plus a header.
+ * For Kernel contexts, use the default MTU plus a header
+ * or half the credits, whichever is smaller. This should
+ * work for both the 3-deep buffering allocation and the
+ * pooling allocation.
*/
if (type == SC_ACK) {
thresh = sc_percent_to_threshold(sc, 50);
@@ -798,7 +825,9 @@
thresh = sc_percent_to_threshold(sc,
user_credit_return_threshold);
} else { /* kernel */
- thresh = sc_mtu_to_threshold(sc, hfi1_max_mtu, hdrqentsize);
+ thresh = min(sc_percent_to_threshold(sc, 50),
+ sc_mtu_to_threshold(sc, hfi1_max_mtu,
+ hdrqentsize));
}
reg = thresh << SC(CREDIT_CTRL_THRESHOLD_SHIFT);
/* add in early return */
@@ -1531,7 +1560,8 @@
unsigned long flags;
unsigned i, n = 0;
- if (dd->send_contexts[sc->sw_index].type != SC_KERNEL)
+ if (dd->send_contexts[sc->sw_index].type != SC_KERNEL &&
+ dd->send_contexts[sc->sw_index].type != SC_VL15)
return;
list = &sc->piowait;
/*
@@ -1900,7 +1930,7 @@
u32 ctxt;
struct hfi1_pportdata *ppd = dd->pport;
- dd->vld[15].sc = sc_alloc(dd, SC_KERNEL,
+ dd->vld[15].sc = sc_alloc(dd, SC_VL15,
dd->rcd[0]->rcvhdrqentsize, dd->node);
if (!dd->vld[15].sc)
goto nomem;
diff --git a/drivers/staging/rdma/hfi1/pio.h b/drivers/staging/rdma/hfi1/pio.h
index 0026976..53a08ed 100644
--- a/drivers/staging/rdma/hfi1/pio.h
+++ b/drivers/staging/rdma/hfi1/pio.h
@@ -51,7 +51,8 @@
#define SC_KERNEL 0
#define SC_ACK 1
#define SC_USER 2
-#define SC_MAX 3
+#define SC_VL15 3
+#define SC_MAX 4
/* invalid send context index */
#define INVALID_SCI 0xff
@@ -293,6 +294,7 @@
void sc_add_credit_return_intr(struct send_context *sc);
void sc_del_credit_return_intr(struct send_context *sc);
void sc_set_cr_threshold(struct send_context *sc, u32 new_threshold);
+u32 sc_percent_to_threshold(struct send_context *sc, u32 percent);
u32 sc_mtu_to_threshold(struct send_context *sc, u32 mtu, u32 hdrqentsize);
void hfi1_sc_wantpiobuf_intr(struct send_context *sc, u32 needint);
void sc_wait(struct hfi1_devdata *dd);
diff --git a/drivers/staging/rdma/hfi1/platform.c b/drivers/staging/rdma/hfi1/platform.c
index 0a1d074..8fe8a20 100644
--- a/drivers/staging/rdma/hfi1/platform.c
+++ b/drivers/staging/rdma/hfi1/platform.c
@@ -114,21 +114,11 @@
if (ret)
return ret;
- if (QSFP_HIGH_PWR(cache[QSFP_MOD_PWR_OFFS]) != 4)
- cable_power_class = QSFP_HIGH_PWR(cache[QSFP_MOD_PWR_OFFS]);
- else
- cable_power_class = QSFP_PWR(cache[QSFP_MOD_PWR_OFFS]);
+ cable_power_class = get_qsfp_power_class(cache[QSFP_MOD_PWR_OFFS]);
- if (cable_power_class <= 3 && cable_power_class > (power_class_max - 1))
+ if (cable_power_class > power_class_max)
ppd->offline_disabled_reason =
HFI1_ODR_MASK(OPA_LINKDOWN_REASON_POWER_POLICY);
- else if (cable_power_class > 4 && cable_power_class > (power_class_max))
- ppd->offline_disabled_reason =
- HFI1_ODR_MASK(OPA_LINKDOWN_REASON_POWER_POLICY);
- /*
- * cable_power_class will never have value 4 as this simply
- * means the high power settings are unused
- */
if (ppd->offline_disabled_reason ==
HFI1_ODR_MASK(OPA_LINKDOWN_REASON_POWER_POLICY)) {
@@ -173,12 +163,9 @@
u8 *cache = ppd->qsfp_info.cache;
int ret;
- if (QSFP_HIGH_PWR(cache[QSFP_MOD_PWR_OFFS]) != 4)
- cable_power_class = QSFP_HIGH_PWR(cache[QSFP_MOD_PWR_OFFS]);
- else
- cable_power_class = QSFP_PWR(cache[QSFP_MOD_PWR_OFFS]);
+ cable_power_class = get_qsfp_power_class(cache[QSFP_MOD_PWR_OFFS]);
- if (cable_power_class) {
+ if (cable_power_class > QSFP_POWER_CLASS_1) {
power_ctrl_byte = cache[QSFP_PWR_CTRL_BYTE_OFFS];
power_ctrl_byte |= 1;
@@ -190,8 +177,7 @@
if (ret != 1)
return -EIO;
- if (cable_power_class > 3) {
- /* > power class 4*/
+ if (cable_power_class > QSFP_POWER_CLASS_4) {
power_ctrl_byte |= (1 << 2);
ret = qsfp_write(ppd, ppd->dd->hfi1_id,
QSFP_PWR_CTRL_BYTE_OFFS,
@@ -212,12 +198,21 @@
{
u32 rx_preset;
u8 *cache = ppd->qsfp_info.cache;
+ int cable_power_class;
if (!((cache[QSFP_MOD_PWR_OFFS] & 0x4) &&
(cache[QSFP_CDR_INFO_OFFS] & 0x40)))
return;
- /* rx_preset preset to zero to catch error */
+ /* RX CDR present, bypass supported */
+ cable_power_class = get_qsfp_power_class(cache[QSFP_MOD_PWR_OFFS]);
+
+ if (cable_power_class <= QSFP_POWER_CLASS_3) {
+ /* Power class <= 3, ignore config & turn RX CDR on */
+ *cdr_ctrl_byte |= 0xF;
+ return;
+ }
+
get_platform_config_field(
ppd->dd, PLATFORM_CONFIG_RX_PRESET_TABLE,
rx_preset_index, RX_PRESET_TABLE_QSFP_RX_CDR_APPLY,
@@ -250,15 +245,25 @@
static void apply_tx_cdr(struct hfi1_pportdata *ppd,
u32 tx_preset_index,
- u8 *ctr_ctrl_byte)
+ u8 *cdr_ctrl_byte)
{
u32 tx_preset;
u8 *cache = ppd->qsfp_info.cache;
+ int cable_power_class;
if (!((cache[QSFP_MOD_PWR_OFFS] & 0x8) &&
(cache[QSFP_CDR_INFO_OFFS] & 0x80)))
return;
+ /* TX CDR present, bypass supported */
+ cable_power_class = get_qsfp_power_class(cache[QSFP_MOD_PWR_OFFS]);
+
+ if (cable_power_class <= QSFP_POWER_CLASS_3) {
+ /* Power class <= 3, ignore config & turn TX CDR on */
+ *cdr_ctrl_byte |= 0xF0;
+ return;
+ }
+
get_platform_config_field(
ppd->dd,
PLATFORM_CONFIG_TX_PRESET_TABLE, tx_preset_index,
@@ -282,10 +287,10 @@
(tx_preset << 2) | (tx_preset << 3));
if (tx_preset)
- *ctr_ctrl_byte |= (tx_preset << 4);
+ *cdr_ctrl_byte |= (tx_preset << 4);
else
/* Preserve current/determined RX CDR status */
- *ctr_ctrl_byte &= ((tx_preset << 4) | 0xF);
+ *cdr_ctrl_byte &= ((tx_preset << 4) | 0xF);
}
static void apply_cdr_settings(
@@ -598,6 +603,7 @@
"Applying TX settings");
}
+/* Must be holding the QSFP i2c resource */
static int tune_active_qsfp(struct hfi1_pportdata *ppd, u32 *ptr_tx_preset,
u32 *ptr_rx_preset, u32 *ptr_total_atten)
{
@@ -605,26 +611,19 @@
u16 lss = ppd->link_speed_supported, lse = ppd->link_speed_enabled;
u8 *cache = ppd->qsfp_info.cache;
- ret = acquire_chip_resource(ppd->dd, qsfp_resource(ppd->dd), QSFP_WAIT);
- if (ret) {
- dd_dev_err(ppd->dd, "%s: hfi%d: cannot lock i2c chain\n",
- __func__, (int)ppd->dd->hfi1_id);
- return ret;
- }
-
ppd->qsfp_info.limiting_active = 1;
ret = set_qsfp_tx(ppd, 0);
if (ret)
- goto bail_unlock;
+ return ret;
ret = qual_power(ppd);
if (ret)
- goto bail_unlock;
+ return ret;
ret = qual_bitrate(ppd);
if (ret)
- goto bail_unlock;
+ return ret;
if (ppd->qsfp_info.reset_needed) {
reset_qsfp(ppd);
@@ -636,7 +635,7 @@
ret = set_qsfp_high_power(ppd);
if (ret)
- goto bail_unlock;
+ return ret;
if (cache[QSFP_EQ_INFO_OFFS] & 0x4) {
ret = get_platform_config_field(
@@ -646,7 +645,7 @@
ptr_tx_preset, 4);
if (ret) {
*ptr_tx_preset = OPA_INVALID_INDEX;
- goto bail_unlock;
+ return ret;
}
} else {
ret = get_platform_config_field(
@@ -656,7 +655,7 @@
ptr_tx_preset, 4);
if (ret) {
*ptr_tx_preset = OPA_INVALID_INDEX;
- goto bail_unlock;
+ return ret;
}
}
@@ -665,7 +664,7 @@
PORT_TABLE_RX_PRESET_IDX, ptr_rx_preset, 4);
if (ret) {
*ptr_rx_preset = OPA_INVALID_INDEX;
- goto bail_unlock;
+ return ret;
}
if ((lss & OPA_LINK_SPEED_25G) && (lse & OPA_LINK_SPEED_25G))
@@ -685,8 +684,6 @@
ret = set_qsfp_tx(ppd, 1);
-bail_unlock:
- release_chip_resource(ppd->dd, qsfp_resource(ppd->dd));
return ret;
}
@@ -833,12 +830,22 @@
total_atten = platform_atten + remote_atten;
tuning_method = OPA_PASSIVE_TUNING;
- } else
+ } else {
ppd->offline_disabled_reason =
HFI1_ODR_MASK(OPA_LINKDOWN_REASON_CHASSIS_CONFIG);
+ goto bail;
+ }
break;
case PORT_TYPE_QSFP:
if (qsfp_mod_present(ppd)) {
+ ret = acquire_chip_resource(ppd->dd,
+ qsfp_resource(ppd->dd),
+ QSFP_WAIT);
+ if (ret) {
+ dd_dev_err(ppd->dd, "%s: hfi%d: cannot lock i2c chain\n",
+ __func__, (int)ppd->dd->hfi1_id);
+ goto bail;
+ }
refresh_qsfp_cache(ppd, &ppd->qsfp_info);
if (ppd->qsfp_info.cache_valid) {
@@ -853,21 +860,23 @@
* update the cache to reflect the changes
*/
refresh_qsfp_cache(ppd, &ppd->qsfp_info);
- if (ret)
- goto bail;
-
limiting_active =
ppd->qsfp_info.limiting_active;
} else {
dd_dev_err(dd,
"%s: Reading QSFP memory failed\n",
__func__);
- goto bail;
+ ret = -EINVAL; /* a fail indication */
}
- } else
+ release_chip_resource(ppd->dd, qsfp_resource(ppd->dd));
+ if (ret)
+ goto bail;
+ } else {
ppd->offline_disabled_reason =
HFI1_ODR_MASK(
OPA_LINKDOWN_REASON_LOCAL_MEDIA_NOT_INSTALLED);
+ goto bail;
+ }
break;
default:
dd_dev_info(ppd->dd, "%s: Unknown port type\n", __func__);
diff --git a/drivers/staging/rdma/hfi1/qp.c b/drivers/staging/rdma/hfi1/qp.c
index dc9119e..91eb423 100644
--- a/drivers/staging/rdma/hfi1/qp.c
+++ b/drivers/staging/rdma/hfi1/qp.c
@@ -167,8 +167,12 @@
*/
static inline int verbs_mtu_enum_to_int(struct ib_device *dev, enum ib_mtu mtu)
{
- int val = opa_mtu_enum_to_int((int)mtu);
+ int val;
+ /* Constraining 10KB packets to 8KB packets */
+ if (mtu == (enum ib_mtu)OPA_MTU_10240)
+ mtu = OPA_MTU_8192;
+ val = opa_mtu_enum_to_int((int)mtu);
if (val > 0)
return val;
return ib_mtu_enum_to_int(mtu);
diff --git a/drivers/staging/rdma/hfi1/qsfp.c b/drivers/staging/rdma/hfi1/qsfp.c
index 9ed1963..2441669 100644
--- a/drivers/staging/rdma/hfi1/qsfp.c
+++ b/drivers/staging/rdma/hfi1/qsfp.c
@@ -96,7 +96,7 @@
{
int ret;
- if (!check_chip_resource(ppd->dd, qsfp_resource(ppd->dd), __func__))
+ if (!check_chip_resource(ppd->dd, i2c_target(target), __func__))
return -EACCES;
/* make sure the TWSI bus is in a sane state */
@@ -162,7 +162,7 @@
{
int ret;
- if (!check_chip_resource(ppd->dd, qsfp_resource(ppd->dd), __func__))
+ if (!check_chip_resource(ppd->dd, i2c_target(target), __func__))
return -EACCES;
/* make sure the TWSI bus is in a sane state */
@@ -192,7 +192,7 @@
int ret;
u8 page;
- if (!check_chip_resource(ppd->dd, qsfp_resource(ppd->dd), __func__))
+ if (!check_chip_resource(ppd->dd, i2c_target(target), __func__))
return -EACCES;
/* make sure the TWSI bus is in a sane state */
@@ -276,7 +276,7 @@
int ret;
u8 page;
- if (!check_chip_resource(ppd->dd, qsfp_resource(ppd->dd), __func__))
+ if (!check_chip_resource(ppd->dd, i2c_target(target), __func__))
return -EACCES;
/* make sure the TWSI bus is in a sane state */
@@ -355,6 +355,8 @@
* The calls to qsfp_{read,write} in this function correctly handle the
* address map difference between this mapping and the mapping implemented
* by those functions
+ *
+ * The caller must be holding the QSFP i2c chain resource.
*/
int refresh_qsfp_cache(struct hfi1_pportdata *ppd, struct qsfp_data *cp)
{
@@ -371,13 +373,9 @@
if (!qsfp_mod_present(ppd)) {
ret = -ENODEV;
- goto bail_no_release;
+ goto bail;
}
- ret = acquire_chip_resource(ppd->dd, qsfp_resource(ppd->dd), QSFP_WAIT);
- if (ret)
- goto bail_no_release;
-
ret = qsfp_read(ppd, target, 0, cache, QSFP_PAGESIZE);
if (ret != QSFP_PAGESIZE) {
dd_dev_info(ppd->dd,
@@ -440,8 +438,6 @@
}
}
- release_chip_resource(ppd->dd, qsfp_resource(ppd->dd));
-
spin_lock_irqsave(&ppd->qsfp_info.qsfp_lock, flags);
ppd->qsfp_info.cache_valid = 1;
ppd->qsfp_info.cache_refresh_required = 0;
@@ -450,8 +446,6 @@
return 0;
bail:
- release_chip_resource(ppd->dd, qsfp_resource(ppd->dd));
-bail_no_release:
memset(cache, 0, (QSFP_MAX_NUM_PAGES * 128));
return ret;
}
@@ -466,7 +460,28 @@
#define QSFP_DUMP_CHUNK 16 /* Holds longest string */
#define QSFP_DEFAULT_HDR_CNT 224
-static const char *pwr_codes = "1.5W2.0W2.5W3.5W";
+#define QSFP_PWR(pbyte) (((pbyte) >> 6) & 3)
+#define QSFP_HIGH_PWR(pbyte) ((pbyte) & 3)
+/* For use with QSFP_HIGH_PWR macro */
+#define QSFP_HIGH_PWR_UNUSED 0 /* Bits [1:0] = 00 implies low power module */
+
+/*
+ * Takes power class byte [Page 00 Byte 129] in SFF 8636
+ * Returns power class as integer (1 through 7, per SFF 8636 rev 2.4)
+ */
+int get_qsfp_power_class(u8 power_byte)
+{
+ if (QSFP_HIGH_PWR(power_byte) == QSFP_HIGH_PWR_UNUSED)
+ /* power classes count from 1, their bit encodings from 0 */
+ return (QSFP_PWR(power_byte) + 1);
+ /*
+ * 00 in the high power classes stands for unused, bringing
+ * balance to the off-by-1 offset above, we add 4 here to
+ * account for the difference between the low and high power
+ * groups
+ */
+ return (QSFP_HIGH_PWR(power_byte) + 4);
+}
int qsfp_mod_present(struct hfi1_pportdata *ppd)
{
@@ -537,6 +552,16 @@
return ret;
}
+static const char *pwr_codes[8] = {"N/AW",
+ "1.5W",
+ "2.0W",
+ "2.5W",
+ "3.5W",
+ "4.0W",
+ "4.5W",
+ "5.0W"
+ };
+
int qsfp_dump(struct hfi1_pportdata *ppd, char *buf, int len)
{
u8 *cache = &ppd->qsfp_info.cache[0];
@@ -546,6 +571,7 @@
int bidx = 0;
u8 *atten = &cache[QSFP_ATTEN_OFFS];
u8 *vendor_oui = &cache[QSFP_VOUI_OFFS];
+ u8 power_byte = 0;
sofar = 0;
lenstr[0] = ' ';
@@ -555,9 +581,9 @@
if (QSFP_IS_CU(cache[QSFP_MOD_TECH_OFFS]))
sprintf(lenstr, "%dM ", cache[QSFP_MOD_LEN_OFFS]);
+ power_byte = cache[QSFP_MOD_PWR_OFFS];
sofar += scnprintf(buf + sofar, len - sofar, "PWR:%.3sW\n",
- pwr_codes +
- (QSFP_PWR(cache[QSFP_MOD_PWR_OFFS]) * 4));
+ pwr_codes[get_qsfp_power_class(power_byte)]);
sofar += scnprintf(buf + sofar, len - sofar, "TECH:%s%s\n",
lenstr,
diff --git a/drivers/staging/rdma/hfi1/qsfp.h b/drivers/staging/rdma/hfi1/qsfp.h
index 831fe4c..dadc66c 100644
--- a/drivers/staging/rdma/hfi1/qsfp.h
+++ b/drivers/staging/rdma/hfi1/qsfp.h
@@ -82,8 +82,9 @@
/* Byte 128 is Identifier: must be 0x0c for QSFP, or 0x0d for QSFP+ */
#define QSFP_MOD_ID_OFFS 128
/*
- * Byte 129 is "Extended Identifier". We only care about D7,D6: Power class
- * 0:1.5W, 1:2.0W, 2:2.5W, 3:3.5W
+ * Byte 129 is "Extended Identifier".
+ * For bits [7:6]: 0:1.5W, 1:2.0W, 2:2.5W, 3:3.5W
+ * For bits [1:0]: 0:Unused, 1:4W, 2:4.5W, 3:5W
*/
#define QSFP_MOD_PWR_OFFS 129
/* Byte 130 is Connector type. Not Intel req'd */
@@ -190,6 +191,9 @@
#define QSFP_HIGH_BIAS_WARNING 0x22
#define QSFP_LOW_BIAS_WARNING 0x11
+#define QSFP_ATTEN_SDR(attenarray) (attenarray[0])
+#define QSFP_ATTEN_DDR(attenarray) (attenarray[1])
+
/*
* struct qsfp_data encapsulates state of QSFP device for one port.
* it will be part of port-specific data if a board supports QSFP.
@@ -201,12 +205,6 @@
* and let the qsfp_lock arbitrate access to common resources.
*
*/
-
-#define QSFP_PWR(pbyte) (((pbyte) >> 6) & 3)
-#define QSFP_HIGH_PWR(pbyte) (((pbyte) & 3) | 4)
-#define QSFP_ATTEN_SDR(attenarray) (attenarray[0])
-#define QSFP_ATTEN_DDR(attenarray) (attenarray[1])
-
struct qsfp_data {
/* Helps to find our way */
struct hfi1_pportdata *ppd;
@@ -223,6 +221,7 @@
int refresh_qsfp_cache(struct hfi1_pportdata *ppd,
struct qsfp_data *cp);
+int get_qsfp_power_class(u8 power_byte);
int qsfp_mod_present(struct hfi1_pportdata *ppd);
int get_cable_info(struct hfi1_devdata *dd, u32 port_num, u32 addr,
u32 len, u8 *data);
diff --git a/drivers/staging/rdma/hfi1/rc.c b/drivers/staging/rdma/hfi1/rc.c
index 0d7e101..792f15e 100644
--- a/drivers/staging/rdma/hfi1/rc.c
+++ b/drivers/staging/rdma/hfi1/rc.c
@@ -1497,7 +1497,7 @@
/* Ignore reserved NAK codes. */
goto bail_stop;
}
- return ret;
+ /* cannot be reached */
bail_stop:
hfi1_stop_rc_timers(qp);
return ret;
@@ -2021,8 +2021,6 @@
if (sl >= OPA_MAX_SLS)
return;
- cca_timer = &ppd->cca_timer[sl];
-
cc_state = get_cc_state(ppd);
if (!cc_state)
@@ -2041,6 +2039,7 @@
spin_lock_irqsave(&ppd->cca_timer_lock, flags);
+ cca_timer = &ppd->cca_timer[sl];
if (cca_timer->ccti < ccti_limit) {
if (cca_timer->ccti + ccti_incr <= ccti_limit)
cca_timer->ccti += ccti_incr;
@@ -2049,8 +2048,6 @@
set_link_ipg(ppd);
}
- spin_unlock_irqrestore(&ppd->cca_timer_lock, flags);
-
ccti = cca_timer->ccti;
if (!hrtimer_active(&cca_timer->hrtimer)) {
@@ -2061,6 +2058,8 @@
HRTIMER_MODE_REL);
}
+ spin_unlock_irqrestore(&ppd->cca_timer_lock, flags);
+
if ((trigger_threshold != 0) && (ccti >= trigger_threshold))
log_cca_event(ppd, sl, rlid, lqpn, rqpn, svc_type);
}
diff --git a/drivers/staging/rdma/hfi1/ruc.c b/drivers/staging/rdma/hfi1/ruc.c
index 08813cd..a659aec 100644
--- a/drivers/staging/rdma/hfi1/ruc.c
+++ b/drivers/staging/rdma/hfi1/ruc.c
@@ -831,7 +831,6 @@
struct hfi1_pkt_state ps;
struct hfi1_qp_priv *priv = qp->priv;
int (*make_req)(struct rvt_qp *qp, struct hfi1_pkt_state *ps);
- unsigned long flags;
unsigned long timeout;
unsigned long timeout_int;
int cpu;
@@ -866,11 +865,11 @@
timeout_int = SEND_RESCHED_TIMEOUT;
}
- spin_lock_irqsave(&qp->s_lock, flags);
+ spin_lock_irqsave(&qp->s_lock, ps.flags);
/* Return if we are already busy processing a work request. */
if (!hfi1_send_ok(qp)) {
- spin_unlock_irqrestore(&qp->s_lock, flags);
+ spin_unlock_irqrestore(&qp->s_lock, ps.flags);
return;
}
@@ -884,7 +883,7 @@
do {
/* Check for a constructed packet to be sent. */
if (qp->s_hdrwords != 0) {
- spin_unlock_irqrestore(&qp->s_lock, flags);
+ spin_unlock_irqrestore(&qp->s_lock, ps.flags);
/*
* If the packet cannot be sent now, return and
* the send tasklet will be woken up later.
@@ -897,11 +896,14 @@
if (unlikely(time_after(jiffies, timeout))) {
if (workqueue_congested(cpu,
ps.ppd->hfi1_wq)) {
- spin_lock_irqsave(&qp->s_lock, flags);
+ spin_lock_irqsave(
+ &qp->s_lock,
+ ps.flags);
qp->s_flags &= ~RVT_S_BUSY;
hfi1_schedule_send(qp);
- spin_unlock_irqrestore(&qp->s_lock,
- flags);
+ spin_unlock_irqrestore(
+ &qp->s_lock,
+ ps.flags);
this_cpu_inc(
*ps.ppd->dd->send_schedule);
return;
@@ -913,11 +915,11 @@
}
timeout = jiffies + (timeout_int) / 8;
}
- spin_lock_irqsave(&qp->s_lock, flags);
+ spin_lock_irqsave(&qp->s_lock, ps.flags);
}
} while (make_req(qp, &ps));
- spin_unlock_irqrestore(&qp->s_lock, flags);
+ spin_unlock_irqrestore(&qp->s_lock, ps.flags);
}
/*
diff --git a/drivers/staging/rdma/hfi1/sysfs.c b/drivers/staging/rdma/hfi1/sysfs.c
index c7f1271..8cd6df8 100644
--- a/drivers/staging/rdma/hfi1/sysfs.c
+++ b/drivers/staging/rdma/hfi1/sysfs.c
@@ -84,7 +84,7 @@
rcu_read_unlock();
return -EINVAL;
}
- memcpy(buf, &cc_state->cct, count);
+ memcpy(buf, (void *)&cc_state->cct + pos, count);
rcu_read_unlock();
return count;
@@ -131,7 +131,7 @@
rcu_read_unlock();
return -EINVAL;
}
- memcpy(buf, &cc_state->cong_setting, count);
+ memcpy(buf, (void *)&cc_state->cong_setting + pos, count);
rcu_read_unlock();
return count;
diff --git a/drivers/staging/rdma/hfi1/ud.c b/drivers/staging/rdma/hfi1/ud.c
index ae8a70f..1e503ad 100644
--- a/drivers/staging/rdma/hfi1/ud.c
+++ b/drivers/staging/rdma/hfi1/ud.c
@@ -322,7 +322,7 @@
(lid == ppd->lid ||
(lid == be16_to_cpu(IB_LID_PERMISSIVE) &&
qp->ibqp.qp_type == IB_QPT_GSI)))) {
- unsigned long flags;
+ unsigned long tflags = ps->flags;
/*
* If DMAs are in progress, we can't generate
* a completion for the loopback packet since
@@ -335,10 +335,10 @@
goto bail;
}
qp->s_cur = next_cur;
- local_irq_save(flags);
- spin_unlock_irqrestore(&qp->s_lock, flags);
+ spin_unlock_irqrestore(&qp->s_lock, tflags);
ud_loopback(qp, wqe);
- spin_lock_irqsave(&qp->s_lock, flags);
+ spin_lock_irqsave(&qp->s_lock, tflags);
+ ps->flags = tflags;
hfi1_send_complete(qp, wqe, IB_WC_SUCCESS);
goto done_free_tx;
}
diff --git a/drivers/staging/rdma/hfi1/user_exp_rcv.c b/drivers/staging/rdma/hfi1/user_exp_rcv.c
index 8bd56d5..1b640a3 100644
--- a/drivers/staging/rdma/hfi1/user_exp_rcv.c
+++ b/drivers/staging/rdma/hfi1/user_exp_rcv.c
@@ -399,8 +399,11 @@
* pages, accept the amount pinned so far and program only that.
* User space knows how to deal with partially programmed buffers.
*/
- if (!hfi1_can_pin_pages(dd, fd->tid_n_pinned, npages))
- return -ENOMEM;
+ if (!hfi1_can_pin_pages(dd, fd->tid_n_pinned, npages)) {
+ ret = -ENOMEM;
+ goto bail;
+ }
+
pinned = hfi1_acquire_user_pages(vaddr, npages, true, pages);
if (pinned <= 0) {
ret = pinned;
diff --git a/drivers/staging/rdma/hfi1/user_sdma.c b/drivers/staging/rdma/hfi1/user_sdma.c
index d53a659..0014c9c 100644
--- a/drivers/staging/rdma/hfi1/user_sdma.c
+++ b/drivers/staging/rdma/hfi1/user_sdma.c
@@ -180,6 +180,8 @@
u64 offset;
};
+#define SDMA_CACHE_NODE_EVICT BIT(0)
+
struct sdma_mmu_node {
struct mmu_rb_node rb;
struct list_head list;
@@ -187,6 +189,7 @@
atomic_t refcount;
struct page **pages;
unsigned npages;
+ unsigned long flags;
};
struct user_sdma_request {
@@ -597,6 +600,13 @@
goto free_req;
}
+ /* Checking P_KEY for requests from user-space */
+ if (egress_pkey_check(dd->pport, req->hdr.lrh, req->hdr.bth, sc,
+ PKEY_CHECK_INVALID)) {
+ ret = -EINVAL;
+ goto free_req;
+ }
+
/*
* Also should check the BTH.lnh. If it says the next header is GRH then
* the RXE parsing will be off and will land in the middle of the KDETH
@@ -1030,27 +1040,29 @@
return 1 + ((epage - spage) >> PAGE_SHIFT);
}
-/* Caller must hold pq->evict_lock */
static u32 sdma_cache_evict(struct hfi1_user_sdma_pkt_q *pq, u32 npages)
{
u32 cleared = 0;
struct sdma_mmu_node *node, *ptr;
+ struct list_head to_evict = LIST_HEAD_INIT(to_evict);
+ spin_lock(&pq->evict_lock);
list_for_each_entry_safe_reverse(node, ptr, &pq->evict, list) {
/* Make sure that no one is still using the node. */
if (!atomic_read(&node->refcount)) {
- /*
- * Need to use the page count now as the remove callback
- * will free the node.
- */
+ set_bit(SDMA_CACHE_NODE_EVICT, &node->flags);
+ list_del_init(&node->list);
+ list_add(&node->list, &to_evict);
cleared += node->npages;
- spin_unlock(&pq->evict_lock);
- hfi1_mmu_rb_remove(&pq->sdma_rb_root, &node->rb);
- spin_lock(&pq->evict_lock);
if (cleared >= npages)
break;
}
}
+ spin_unlock(&pq->evict_lock);
+
+ list_for_each_entry_safe(node, ptr, &to_evict, list)
+ hfi1_mmu_rb_remove(&pq->sdma_rb_root, &node->rb);
+
return cleared;
}
@@ -1062,9 +1074,9 @@
struct sdma_mmu_node *node = NULL;
struct mmu_rb_node *rb_node;
- rb_node = hfi1_mmu_rb_search(&pq->sdma_rb_root,
- (unsigned long)iovec->iov.iov_base,
- iovec->iov.iov_len);
+ rb_node = hfi1_mmu_rb_extract(&pq->sdma_rb_root,
+ (unsigned long)iovec->iov.iov_base,
+ iovec->iov.iov_len);
if (rb_node && !IS_ERR(rb_node))
node = container_of(rb_node, struct sdma_mmu_node, rb);
else
@@ -1076,7 +1088,6 @@
return -ENOMEM;
node->rb.addr = (unsigned long)iovec->iov.iov_base;
- node->rb.len = iovec->iov.iov_len;
node->pq = pq;
atomic_set(&node->refcount, 0);
INIT_LIST_HEAD(&node->list);
@@ -1093,11 +1104,25 @@
memcpy(pages, node->pages, node->npages * sizeof(*pages));
npages -= node->npages;
+
+ /*
+ * If rb_node is NULL, it means that this is brand new node
+ * and, therefore not on the eviction list.
+ * If, however, the rb_node is non-NULL, it means that the
+ * node is already in RB tree and, therefore on the eviction
+ * list (nodes are unconditionally inserted in the eviction
+ * list). In that case, we have to remove the node prior to
+ * calling the eviction function in order to prevent it from
+ * freeing this node.
+ */
+ if (rb_node) {
+ spin_lock(&pq->evict_lock);
+ list_del_init(&node->list);
+ spin_unlock(&pq->evict_lock);
+ }
retry:
if (!hfi1_can_pin_pages(pq->dd, pq->n_locked, npages)) {
- spin_lock(&pq->evict_lock);
cleared = sdma_cache_evict(pq, npages);
- spin_unlock(&pq->evict_lock);
if (cleared >= npages)
goto retry;
}
@@ -1117,37 +1142,32 @@
goto bail;
}
kfree(node->pages);
+ node->rb.len = iovec->iov.iov_len;
node->pages = pages;
node->npages += pinned;
npages = node->npages;
spin_lock(&pq->evict_lock);
- if (!rb_node)
- list_add(&node->list, &pq->evict);
- else
- list_move(&node->list, &pq->evict);
+ list_add(&node->list, &pq->evict);
pq->n_locked += pinned;
spin_unlock(&pq->evict_lock);
}
iovec->pages = node->pages;
iovec->npages = npages;
- if (!rb_node) {
- ret = hfi1_mmu_rb_insert(&req->pq->sdma_rb_root, &node->rb);
- if (ret) {
- spin_lock(&pq->evict_lock);
+ ret = hfi1_mmu_rb_insert(&req->pq->sdma_rb_root, &node->rb);
+ if (ret) {
+ spin_lock(&pq->evict_lock);
+ if (!list_empty(&node->list))
list_del(&node->list);
- pq->n_locked -= node->npages;
- spin_unlock(&pq->evict_lock);
- ret = 0;
- goto bail;
- }
- } else {
- atomic_inc(&node->refcount);
+ pq->n_locked -= node->npages;
+ spin_unlock(&pq->evict_lock);
+ goto bail;
}
return 0;
bail:
- if (!rb_node)
- kfree(node);
+ if (rb_node)
+ unpin_vector_pages(current->mm, node->pages, 0, node->npages);
+ kfree(node);
return ret;
}
@@ -1558,7 +1578,20 @@
container_of(mnode, struct sdma_mmu_node, rb);
spin_lock(&node->pq->evict_lock);
- list_del(&node->list);
+ /*
+ * We've been called by the MMU notifier but this node has been
+ * scheduled for eviction. The eviction function will take care
+ * of freeing this node.
+ * We have to take the above lock first because we are racing
+ * against the setting of the bit in the eviction function.
+ */
+ if (mm && test_bit(SDMA_CACHE_NODE_EVICT, &node->flags)) {
+ spin_unlock(&node->pq->evict_lock);
+ return;
+ }
+
+ if (!list_empty(&node->list))
+ list_del(&node->list);
node->pq->n_locked -= node->npages;
spin_unlock(&node->pq->evict_lock);
diff --git a/drivers/staging/rdma/hfi1/verbs.c b/drivers/staging/rdma/hfi1/verbs.c
index 89f2aad..9cdc85f 100644
--- a/drivers/staging/rdma/hfi1/verbs.c
+++ b/drivers/staging/rdma/hfi1/verbs.c
@@ -545,7 +545,7 @@
if (!(ib_rvt_state_ops[packet->qp->state] & RVT_PROCESS_RECV_OK))
goto dropit;
- if (((opcode & OPCODE_QP_MASK) == packet->qp->allowed_ops) ||
+ if (((opcode & RVT_OPCODE_QP_MASK) == packet->qp->allowed_ops) ||
(opcode == IB_OPCODE_CNP))
return 1;
dropit:
@@ -1089,16 +1089,16 @@
/*
* egress_pkey_matches_entry - return 1 if the pkey matches ent (ent
- * being an entry from the ingress partition key table), return 0
+ * being an entry from the partition key table), return 0
* otherwise. Use the matching criteria for egress partition keys
* specified in the OPAv1 spec., section 9.1l.7.
*/
static inline int egress_pkey_matches_entry(u16 pkey, u16 ent)
{
u16 mkey = pkey & PKEY_LOW_15_MASK;
- u16 ment = ent & PKEY_LOW_15_MASK;
+ u16 mentry = ent & PKEY_LOW_15_MASK;
- if (mkey == ment) {
+ if (mkey == mentry) {
/*
* If pkey[15] is set (full partition member),
* is bit 15 in the corresponding table element
@@ -1111,32 +1111,32 @@
return 0;
}
-/*
- * egress_pkey_check - return 0 if hdr's pkey matches according to the
- * criteria in the OPAv1 spec., section 9.11.7.
+/**
+ * egress_pkey_check - check P_KEY of a packet
+ * @ppd: Physical IB port data
+ * @lrh: Local route header
+ * @bth: Base transport header
+ * @sc5: SC for packet
+ * @s_pkey_index: It will be used for look up optimization for kernel contexts
+ * only. If it is negative value, then it means user contexts is calling this
+ * function.
+ *
+ * It checks if hdr's pkey is valid.
+ *
+ * Return: 0 on success, otherwise, 1
*/
-static inline int egress_pkey_check(struct hfi1_pportdata *ppd,
- struct hfi1_ib_header *hdr,
- struct rvt_qp *qp)
+int egress_pkey_check(struct hfi1_pportdata *ppd, __be16 *lrh, __be32 *bth,
+ u8 sc5, int8_t s_pkey_index)
{
- struct hfi1_qp_priv *priv = qp->priv;
- struct hfi1_other_headers *ohdr;
struct hfi1_devdata *dd;
- int i = 0;
+ int i;
u16 pkey;
- u8 lnh, sc5 = priv->s_sc;
+ int is_user_ctxt_mechanism = (s_pkey_index < 0);
if (!(ppd->part_enforce & HFI1_PART_ENFORCE_OUT))
return 0;
- /* locate the pkey within the headers */
- lnh = be16_to_cpu(hdr->lrh[0]) & 3;
- if (lnh == HFI1_LRH_GRH)
- ohdr = &hdr->u.l.oth;
- else
- ohdr = &hdr->u.oth;
-
- pkey = (u16)be32_to_cpu(ohdr->bth[0]);
+ pkey = (u16)be32_to_cpu(bth[0]);
/* If SC15, pkey[0:14] must be 0x7fff */
if ((sc5 == 0xf) && ((pkey & PKEY_LOW_15_MASK) != PKEY_LOW_15_MASK))
@@ -1146,28 +1146,37 @@
if ((pkey & PKEY_LOW_15_MASK) == 0)
goto bad;
- /* The most likely matching pkey has index qp->s_pkey_index */
- if (unlikely(!egress_pkey_matches_entry(pkey,
- ppd->pkeys
- [qp->s_pkey_index]))) {
- /* no match - try the entire table */
- for (; i < MAX_PKEY_VALUES; i++) {
- if (egress_pkey_matches_entry(pkey, ppd->pkeys[i]))
- break;
- }
+ /*
+ * For the kernel contexts only, if a qp is passed into the function,
+ * the most likely matching pkey has index qp->s_pkey_index
+ */
+ if (!is_user_ctxt_mechanism &&
+ egress_pkey_matches_entry(pkey, ppd->pkeys[s_pkey_index])) {
+ return 0;
}
- if (i < MAX_PKEY_VALUES)
- return 0;
+ for (i = 0; i < MAX_PKEY_VALUES; i++) {
+ if (egress_pkey_matches_entry(pkey, ppd->pkeys[i]))
+ return 0;
+ }
bad:
- incr_cntr64(&ppd->port_xmit_constraint_errors);
- dd = ppd->dd;
- if (!(dd->err_info_xmit_constraint.status & OPA_EI_STATUS_SMASK)) {
- u16 slid = be16_to_cpu(hdr->lrh[3]);
+ /*
+ * For the user-context mechanism, the P_KEY check would only happen
+ * once per SDMA request, not once per packet. Therefore, there's no
+ * need to increment the counter for the user-context mechanism.
+ */
+ if (!is_user_ctxt_mechanism) {
+ incr_cntr64(&ppd->port_xmit_constraint_errors);
+ dd = ppd->dd;
+ if (!(dd->err_info_xmit_constraint.status &
+ OPA_EI_STATUS_SMASK)) {
+ u16 slid = be16_to_cpu(lrh[3]);
- dd->err_info_xmit_constraint.status |= OPA_EI_STATUS_SMASK;
- dd->err_info_xmit_constraint.slid = slid;
- dd->err_info_xmit_constraint.pkey = pkey;
+ dd->err_info_xmit_constraint.status |=
+ OPA_EI_STATUS_SMASK;
+ dd->err_info_xmit_constraint.slid = slid;
+ dd->err_info_xmit_constraint.pkey = pkey;
+ }
}
return 1;
}
@@ -1227,11 +1236,26 @@
{
struct hfi1_devdata *dd = dd_from_ibdev(qp->ibqp.device);
struct hfi1_qp_priv *priv = qp->priv;
+ struct hfi1_other_headers *ohdr;
+ struct hfi1_ib_header *hdr;
send_routine sr;
int ret;
+ u8 lnh;
+
+ hdr = &ps->s_txreq->phdr.hdr;
+ /* locate the pkey within the headers */
+ lnh = be16_to_cpu(hdr->lrh[0]) & 3;
+ if (lnh == HFI1_LRH_GRH)
+ ohdr = &hdr->u.l.oth;
+ else
+ ohdr = &hdr->u.oth;
sr = get_send_routine(qp, ps->s_txreq);
- ret = egress_pkey_check(dd->pport, &ps->s_txreq->phdr.hdr, qp);
+ ret = egress_pkey_check(dd->pport,
+ hdr->lrh,
+ ohdr->bth,
+ priv->s_sc,
+ qp->s_pkey_index);
if (unlikely(ret)) {
/*
* The value we are returning here does not get propagated to
diff --git a/drivers/staging/rdma/hfi1/verbs.h b/drivers/staging/rdma/hfi1/verbs.h
index 6c4670f..3ee223983 100644
--- a/drivers/staging/rdma/hfi1/verbs.h
+++ b/drivers/staging/rdma/hfi1/verbs.h
@@ -215,6 +215,7 @@
struct hfi1_ibport *ibp;
struct hfi1_pportdata *ppd;
struct verbs_txreq *s_txreq;
+ unsigned long flags;
};
#define HFI1_PSN_CREDIT 16
@@ -334,9 +335,6 @@
#endif
#define PSN_MODIFY_MASK 0xFFFFFF
-/* Number of bits to pay attention to in the opcode for checking qp type */
-#define OPCODE_QP_MASK 0xE0
-
/*
* Compare the lower 24 bits of the msn values.
* Returns an integer <, ==, or > than zero.
diff --git a/drivers/staging/rtl8188eu/core/rtw_ap.c b/drivers/staging/rtl8188eu/core/rtw_ap.c
index 012860b..a575535 100644
--- a/drivers/staging/rtl8188eu/core/rtw_ap.c
+++ b/drivers/staging/rtl8188eu/core/rtw_ap.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _RTW_AP_C_
diff --git a/drivers/staging/rtl8188eu/core/rtw_cmd.c b/drivers/staging/rtl8188eu/core/rtw_cmd.c
index e5a6b7a..7748523 100644
--- a/drivers/staging/rtl8188eu/core/rtw_cmd.c
+++ b/drivers/staging/rtl8188eu/core/rtw_cmd.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _RTW_CMD_C_
@@ -263,11 +258,11 @@
rtw_lps_ctrl_wk_cmd(padapter, LPS_CTRL_SCAN, 1);
ph2c = kzalloc(sizeof(struct cmd_obj), GFP_ATOMIC);
- if (ph2c == NULL)
+ if (!ph2c)
return _FAIL;
psurveyPara = kzalloc(sizeof(struct sitesurvey_parm), GFP_ATOMIC);
- if (psurveyPara == NULL) {
+ if (!psurveyPara) {
kfree(ph2c);
return _FAIL;
}
@@ -350,7 +345,7 @@
RT_TRACE(_module_rtl871x_cmd_c_, _drv_info_, (" createbss for SSid:%s\n", pmlmepriv->assoc_ssid.Ssid));
pcmd = kzalloc(sizeof(struct cmd_obj), GFP_KERNEL);
- if (pcmd == NULL) {
+ if (!pcmd) {
res = _FAIL;
goto exit;
}
@@ -521,7 +516,7 @@
/* prepare cmd parameter */
param = kzalloc(sizeof(*param), GFP_KERNEL);
- if (param == NULL) {
+ if (!param) {
res = _FAIL;
goto exit;
}
@@ -530,7 +525,7 @@
if (enqueue) {
/* need enqueue, prepare cmd_obj and enqueue */
cmdobj = kzalloc(sizeof(*cmdobj), GFP_KERNEL);
- if (cmdobj == NULL) {
+ if (!cmdobj) {
res = _FAIL;
kfree(param);
goto exit;
@@ -629,20 +624,20 @@
clear_cam_entry(padapter, entry);
} else {
ph2c = kzalloc(sizeof(struct cmd_obj), GFP_ATOMIC);
- if (ph2c == NULL) {
+ if (!ph2c) {
res = _FAIL;
goto exit;
}
psetstakey_para = kzalloc(sizeof(struct set_stakey_parm), GFP_ATOMIC);
- if (psetstakey_para == NULL) {
+ if (!psetstakey_para) {
kfree(ph2c);
res = _FAIL;
goto exit;
}
psetstakey_rsp = kzalloc(sizeof(struct set_stakey_rsp), GFP_ATOMIC);
- if (psetstakey_rsp == NULL) {
+ if (!psetstakey_rsp) {
kfree(ph2c);
kfree(psetstakey_para);
res = _FAIL;
@@ -676,13 +671,13 @@
ph2c = kzalloc(sizeof(struct cmd_obj), GFP_KERNEL);
- if (ph2c == NULL) {
+ if (!ph2c) {
res = _FAIL;
goto exit;
}
paddbareq_parm = kzalloc(sizeof(struct addBaReq_parm), GFP_KERNEL);
- if (paddbareq_parm == NULL) {
+ if (!paddbareq_parm) {
kfree(ph2c);
res = _FAIL;
goto exit;
@@ -713,13 +708,13 @@
ph2c = kzalloc(sizeof(struct cmd_obj), GFP_ATOMIC);
- if (ph2c == NULL) {
+ if (!ph2c) {
res = _FAIL;
goto exit;
}
pdrvextra_cmd_parm = kzalloc(sizeof(struct drvextra_cmd_parm), GFP_ATOMIC);
- if (pdrvextra_cmd_parm == NULL) {
+ if (!pdrvextra_cmd_parm) {
kfree(ph2c);
res = _FAIL;
goto exit;
@@ -757,7 +752,7 @@
/* prepare cmd parameter */
setChannelPlan_param = kzalloc(sizeof(struct SetChannelPlan_param), GFP_KERNEL);
- if (setChannelPlan_param == NULL) {
+ if (!setChannelPlan_param) {
res = _FAIL;
goto exit;
}
@@ -766,7 +761,7 @@
if (enqueue) {
/* need enqueue, prepare cmd_obj and enqueue */
pcmdobj = kzalloc(sizeof(struct cmd_obj), GFP_KERNEL);
- if (pcmdobj == NULL) {
+ if (!pcmdobj) {
kfree(setChannelPlan_param);
res = _FAIL;
goto exit;
@@ -925,13 +920,13 @@
if (enqueue) {
ph2c = kzalloc(sizeof(struct cmd_obj), GFP_ATOMIC);
- if (ph2c == NULL) {
+ if (!ph2c) {
res = _FAIL;
goto exit;
}
pdrvextra_cmd_parm = kzalloc(sizeof(struct drvextra_cmd_parm), GFP_ATOMIC);
- if (pdrvextra_cmd_parm == NULL) {
+ if (!pdrvextra_cmd_parm) {
kfree(ph2c);
res = _FAIL;
goto exit;
@@ -968,13 +963,13 @@
u8 res = _SUCCESS;
ph2c = kzalloc(sizeof(struct cmd_obj), GFP_ATOMIC);
- if (ph2c == NULL) {
+ if (!ph2c) {
res = _FAIL;
goto exit;
}
pdrvextra_cmd_parm = kzalloc(sizeof(struct drvextra_cmd_parm), GFP_ATOMIC);
- if (pdrvextra_cmd_parm == NULL) {
+ if (!pdrvextra_cmd_parm) {
kfree(ph2c);
res = _FAIL;
goto exit;
@@ -1010,13 +1005,13 @@
if (enqueue) {
ph2c = kzalloc(sizeof(struct cmd_obj), GFP_KERNEL);
- if (ph2c == NULL) {
+ if (!ph2c) {
res = _FAIL;
goto exit;
}
pdrvextra_cmd_parm = kzalloc(sizeof(struct drvextra_cmd_parm), GFP_KERNEL);
- if (pdrvextra_cmd_parm == NULL) {
+ if (!pdrvextra_cmd_parm) {
kfree(ph2c);
res = _FAIL;
goto exit;
@@ -1108,13 +1103,13 @@
u8 res = _SUCCESS;
ph2c = kzalloc(sizeof(struct cmd_obj), GFP_KERNEL);
- if (ph2c == NULL) {
+ if (!ph2c) {
res = _FAIL;
goto exit;
}
pdrvextra_cmd_parm = kzalloc(sizeof(struct drvextra_cmd_parm), GFP_KERNEL);
- if (pdrvextra_cmd_parm == NULL) {
+ if (!pdrvextra_cmd_parm) {
kfree(ph2c);
res = _FAIL;
goto exit;
diff --git a/drivers/staging/rtl8188eu/core/rtw_debug.c b/drivers/staging/rtl8188eu/core/rtw_debug.c
index 93e898d..db5c952 100644
--- a/drivers/staging/rtl8188eu/core/rtw_debug.c
+++ b/drivers/staging/rtl8188eu/core/rtw_debug.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _RTW_DEBUG_C_
diff --git a/drivers/staging/rtl8188eu/core/rtw_efuse.c b/drivers/staging/rtl8188eu/core/rtw_efuse.c
index 19f11d0..c17870c 100644
--- a/drivers/staging/rtl8188eu/core/rtw_efuse.c
+++ b/drivers/staging/rtl8188eu/core/rtw_efuse.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _RTW_EFUSE_C_
@@ -107,7 +102,7 @@
if (!efuseTbl)
return;
- eFuseWord = (u16 **)rtw_malloc2d(EFUSE_MAX_SECTION_88E, EFUSE_MAX_WORD_UNIT, sizeof(u16));
+ eFuseWord = (u16 **)rtw_malloc2d(EFUSE_MAX_SECTION_88E, EFUSE_MAX_WORD_UNIT, sizeof(*eFuseWord));
if (!eFuseWord) {
DBG_88E("%s: alloc eFuseWord fail!\n", __func__);
goto eFuseWord_failed;
diff --git a/drivers/staging/rtl8188eu/core/rtw_ieee80211.c b/drivers/staging/rtl8188eu/core/rtw_ieee80211.c
index f4e4baf..0b0d78f 100644
--- a/drivers/staging/rtl8188eu/core/rtw_ieee80211.c
+++ b/drivers/staging/rtl8188eu/core/rtw_ieee80211.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _IEEE80211_C
diff --git a/drivers/staging/rtl8188eu/core/rtw_ioctl_set.c b/drivers/staging/rtl8188eu/core/rtw_ioctl_set.c
index cf60717..f85a6ab 100644
--- a/drivers/staging/rtl8188eu/core/rtw_ioctl_set.c
+++ b/drivers/staging/rtl8188eu/core/rtw_ioctl_set.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _RTW_IOCTL_SET_C_
diff --git a/drivers/staging/rtl8188eu/core/rtw_mlme.c b/drivers/staging/rtl8188eu/core/rtw_mlme.c
index a645a62..1456499 100644
--- a/drivers/staging/rtl8188eu/core/rtw_mlme.c
+++ b/drivers/staging/rtl8188eu/core/rtw_mlme.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _RTW_MLME_C_
@@ -1584,13 +1579,13 @@
int res = _SUCCESS;
pcmd = kzalloc(sizeof(struct cmd_obj), GFP_KERNEL);
- if (pcmd == NULL) {
+ if (!pcmd) {
res = _FAIL; /* try again */
goto exit;
}
psetauthparm = kzalloc(sizeof(struct setauth_parm), GFP_KERNEL);
- if (psetauthparm == NULL) {
+ if (!psetauthparm) {
kfree(pcmd);
res = _FAIL;
goto exit;
@@ -1621,11 +1616,11 @@
int res = _SUCCESS;
pcmd = kzalloc(sizeof(struct cmd_obj), GFP_KERNEL);
- if (pcmd == NULL)
+ if (!pcmd)
return _FAIL; /* try again */
psetkeyparm = kzalloc(sizeof(struct setkey_parm), GFP_KERNEL);
- if (psetkeyparm == NULL) {
+ if (!psetkeyparm) {
res = _FAIL;
goto err_free_cmd;
}
diff --git a/drivers/staging/rtl8188eu/core/rtw_mlme_ext.c b/drivers/staging/rtl8188eu/core/rtw_mlme_ext.c
index 591a912..7f32b39 100644
--- a/drivers/staging/rtl8188eu/core/rtw_mlme_ext.c
+++ b/drivers/staging/rtl8188eu/core/rtw_mlme_ext.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _RTW_MLME_EXT_C_
@@ -606,8 +601,6 @@
pattrib->last_txcmdsz = pattrib->pktlen;
dump_mgntframe(padapter, pmgntframe);
-
- return;
}
static int issue_probereq(struct adapter *padapter, struct ndis_802_11_ssid *pssid, u8 *da, bool wait_ack)
@@ -888,8 +881,6 @@
rtw_wep_encrypt(padapter, (u8 *)pmgntframe);
DBG_88E("%s\n", __func__);
dump_mgntframe(padapter, pmgntframe);
-
- return;
}
@@ -1212,8 +1203,6 @@
rtw_buf_update(&pmlmepriv->assoc_req, &pmlmepriv->assoc_req_len, (u8 *)pwlanhdr, pattrib->pktlen);
else
rtw_buf_free(&pmlmepriv->assoc_req, &pmlmepriv->assoc_req_len);
-
- return;
}
/* when wait_ack is true, this function should be called at process context */
@@ -2105,7 +2094,6 @@
issue_action_BSSCoexistPacket(padapter);
issue_action_BSSCoexistPacket(padapter);
}
- return;
}
/* collect bss info from Beacon and Probe request/response frames. */
@@ -4295,12 +4283,12 @@
pcmd_obj = kzalloc(sizeof(struct cmd_obj), GFP_ATOMIC);
- if (pcmd_obj == NULL)
+ if (!pcmd_obj)
return;
cmdsz = sizeof(struct survey_event) + sizeof(struct C2HEvent_Header);
pevtcmd = kzalloc(cmdsz, GFP_ATOMIC);
- if (pevtcmd == NULL) {
+ if (!pevtcmd) {
kfree(pcmd_obj);
return;
}
@@ -4332,8 +4320,6 @@
rtw_enqueue_cmd(pcmdpriv, pcmd_obj);
pmlmeext->sitesurvey_res.bss_cnt++;
-
- return;
}
void report_surveydone_event(struct adapter *padapter)
@@ -4347,12 +4333,12 @@
struct cmd_priv *pcmdpriv = &padapter->cmdpriv;
pcmd_obj = kzalloc(sizeof(struct cmd_obj), GFP_KERNEL);
- if (pcmd_obj == NULL)
+ if (!pcmd_obj)
return;
cmdsz = sizeof(struct surveydone_event) + sizeof(struct C2HEvent_Header);
pevtcmd = kzalloc(cmdsz, GFP_KERNEL);
- if (pevtcmd == NULL) {
+ if (!pevtcmd) {
kfree(pcmd_obj);
return;
}
@@ -4377,8 +4363,6 @@
DBG_88E("survey done event(%x)\n", psurveydone_evt->bss_cnt);
rtw_enqueue_cmd(pcmdpriv, pcmd_obj);
-
- return;
}
void report_join_res(struct adapter *padapter, int res)
@@ -4393,12 +4377,12 @@
struct cmd_priv *pcmdpriv = &padapter->cmdpriv;
pcmd_obj = kzalloc(sizeof(struct cmd_obj), GFP_ATOMIC);
- if (pcmd_obj == NULL)
+ if (!pcmd_obj)
return;
cmdsz = sizeof(struct joinbss_event) + sizeof(struct C2HEvent_Header);
pevtcmd = kzalloc(cmdsz, GFP_ATOMIC);
- if (pevtcmd == NULL) {
+ if (!pevtcmd) {
kfree(pcmd_obj);
return;
}
@@ -4429,8 +4413,6 @@
rtw_enqueue_cmd(pcmdpriv, pcmd_obj);
-
- return;
}
void report_del_sta_event(struct adapter *padapter, unsigned char *MacAddr, unsigned short reason)
@@ -4446,12 +4428,12 @@
struct cmd_priv *pcmdpriv = &padapter->cmdpriv;
pcmd_obj = kzalloc(sizeof(struct cmd_obj), GFP_KERNEL);
- if (pcmd_obj == NULL)
+ if (!pcmd_obj)
return;
cmdsz = sizeof(struct stadel_event) + sizeof(struct C2HEvent_Header);
pevtcmd = kzalloc(cmdsz, GFP_KERNEL);
- if (pevtcmd == NULL) {
+ if (!pevtcmd) {
kfree(pcmd_obj);
return;
}
@@ -4486,8 +4468,6 @@
DBG_88E("report_del_sta_event: delete STA, mac_id =%d\n", mac_id);
rtw_enqueue_cmd(pcmdpriv, pcmd_obj);
-
- return;
}
void report_add_sta_event(struct adapter *padapter, unsigned char *MacAddr, int cam_idx)
@@ -4501,12 +4481,12 @@
struct cmd_priv *pcmdpriv = &padapter->cmdpriv;
pcmd_obj = kzalloc(sizeof(struct cmd_obj), GFP_KERNEL);
- if (pcmd_obj == NULL)
+ if (!pcmd_obj)
return;
cmdsz = sizeof(struct stassoc_event) + sizeof(struct C2HEvent_Header);
pevtcmd = kzalloc(cmdsz, GFP_KERNEL);
- if (pevtcmd == NULL) {
+ if (!pevtcmd) {
kfree(pcmd_obj);
return;
}
@@ -4532,8 +4512,6 @@
DBG_88E("report_add_sta_event: add STA\n");
rtw_enqueue_cmd(pcmdpriv, pcmd_obj);
-
- return;
}
@@ -4917,11 +4895,11 @@
}
ph2c = kzalloc(sizeof(struct cmd_obj), GFP_ATOMIC);
- if (ph2c == NULL)
+ if (!ph2c)
goto exit_survey_timer_hdl;
psurveyPara = kzalloc(sizeof(struct sitesurvey_parm), GFP_ATOMIC);
- if (psurveyPara == NULL) {
+ if (!psurveyPara) {
kfree(ph2c);
goto exit_survey_timer_hdl;
}
@@ -4969,7 +4947,6 @@
issue_assocreq(padapter);
set_link_timer(pmlmeext, REASSOC_TO);
}
- return;
}
void addba_timer_hdl(unsigned long data)
@@ -5485,7 +5462,7 @@
ph2c = kzalloc(sizeof(struct cmd_obj), GFP_KERNEL);
- if (ph2c == NULL) {
+ if (!ph2c) {
res = _FAIL;
goto exit;
}
diff --git a/drivers/staging/rtl8188eu/core/rtw_pwrctrl.c b/drivers/staging/rtl8188eu/core/rtw_pwrctrl.c
index 5e1ef9f..59c6d8a 100644
--- a/drivers/staging/rtl8188eu/core/rtw_pwrctrl.c
+++ b/drivers/staging/rtl8188eu/core/rtw_pwrctrl.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _RTW_PWRCTRL_C_
diff --git a/drivers/staging/rtl8188eu/core/rtw_recv.c b/drivers/staging/rtl8188eu/core/rtw_recv.c
index 5f53aa1..977bb25 100644
--- a/drivers/staging/rtl8188eu/core/rtw_recv.c
+++ b/drivers/staging/rtl8188eu/core/rtw_recv.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _RTW_RECV_C_
diff --git a/drivers/staging/rtl8188eu/core/rtw_rf.c b/drivers/staging/rtl8188eu/core/rtw_rf.c
index 4ad2d8f..3fc1a8f 100644
--- a/drivers/staging/rtl8188eu/core/rtw_rf.c
+++ b/drivers/staging/rtl8188eu/core/rtw_rf.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _RTW_RF_C_
diff --git a/drivers/staging/rtl8188eu/core/rtw_security.c b/drivers/staging/rtl8188eu/core/rtw_security.c
index b781ccf..442a614 100644
--- a/drivers/staging/rtl8188eu/core/rtw_security.c
+++ b/drivers/staging/rtl8188eu/core/rtw_security.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _RTW_SECURITY_C_
diff --git a/drivers/staging/rtl8188eu/core/rtw_sreset.c b/drivers/staging/rtl8188eu/core/rtw_sreset.c
index e725a47..13a5bf4 100644
--- a/drivers/staging/rtl8188eu/core/rtw_sreset.c
+++ b/drivers/staging/rtl8188eu/core/rtw_sreset.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#include <rtw_sreset.h>
diff --git a/drivers/staging/rtl8188eu/core/rtw_sta_mgt.c b/drivers/staging/rtl8188eu/core/rtw_sta_mgt.c
index 78a9b9b..a71e252 100644
--- a/drivers/staging/rtl8188eu/core/rtw_sta_mgt.c
+++ b/drivers/staging/rtl8188eu/core/rtw_sta_mgt.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _RTW_STA_MGT_C_
diff --git a/drivers/staging/rtl8188eu/core/rtw_wlan_util.c b/drivers/staging/rtl8188eu/core/rtw_wlan_util.c
index 8309669..4410fe8 100644
--- a/drivers/staging/rtl8188eu/core/rtw_wlan_util.c
+++ b/drivers/staging/rtl8188eu/core/rtw_wlan_util.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _RTW_WLAN_UTIL_C_
diff --git a/drivers/staging/rtl8188eu/core/rtw_xmit.c b/drivers/staging/rtl8188eu/core/rtw_xmit.c
index f2dd7a6..e0a5567 100644
--- a/drivers/staging/rtl8188eu/core/rtw_xmit.c
+++ b/drivers/staging/rtl8188eu/core/rtw_xmit.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _RTW_XMIT_C_
diff --git a/drivers/staging/rtl8188eu/hal/Hal8188ERateAdaptive.c b/drivers/staging/rtl8188eu/hal/Hal8188ERateAdaptive.c
index a108e80..201c15b 100644
--- a/drivers/staging/rtl8188eu/hal/Hal8188ERateAdaptive.c
+++ b/drivers/staging/rtl8188eu/hal/Hal8188ERateAdaptive.c
@@ -557,7 +557,7 @@
u8 WirelessMode = 0xFF; /* invalid value */
u8 max_rate_idx = 0x13; /* MCS7 */
- if (dm_odm->pWirelessMode != NULL)
+ if (dm_odm->pWirelessMode)
WirelessMode = *(dm_odm->pWirelessMode);
if (WirelessMode != 0xFF) {
diff --git a/drivers/staging/rtl8188eu/hal/bb_cfg.c b/drivers/staging/rtl8188eu/hal/bb_cfg.c
index c2ad6a3..cce1ea2 100644
--- a/drivers/staging/rtl8188eu/hal/bb_cfg.c
+++ b/drivers/staging/rtl8188eu/hal/bb_cfg.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
-* You should have received a copy of the GNU General Public License along with
-* this program; if not, write to the Free Software Foundation, Inc.,
-* 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
-*
-*
******************************************************************************/
#include "odm_precomp.h"
diff --git a/drivers/staging/rtl8188eu/hal/fw.c b/drivers/staging/rtl8188eu/hal/fw.c
index 656133c..03d091b 100644
--- a/drivers/staging/rtl8188eu/hal/fw.c
+++ b/drivers/staging/rtl8188eu/hal/fw.c
@@ -11,10 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
* The full GNU General Public License is included in this distribution in the
* file called LICENSE.
*
diff --git a/drivers/staging/rtl8188eu/hal/hal_com.c b/drivers/staging/rtl8188eu/hal/hal_com.c
index 3871cda..960cc40 100644
--- a/drivers/staging/rtl8188eu/hal/hal_com.c
+++ b/drivers/staging/rtl8188eu/hal/hal_com.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#include <osdep_service.h>
#include <drv_types.h>
diff --git a/drivers/staging/rtl8188eu/hal/hal_intf.c b/drivers/staging/rtl8188eu/hal/hal_intf.c
index 85c17ef..085f0fb 100644
--- a/drivers/staging/rtl8188eu/hal/hal_intf.c
+++ b/drivers/staging/rtl8188eu/hal/hal_intf.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _HAL_INTF_C_
@@ -186,7 +181,7 @@
s32 rtw_hal_init_xmit_priv(struct adapter *adapt)
{
- if (adapt->HalFunc.init_xmit_priv != NULL)
+ if (adapt->HalFunc.init_xmit_priv)
return adapt->HalFunc.init_xmit_priv(adapt);
return _FAIL;
}
diff --git a/drivers/staging/rtl8188eu/hal/mac_cfg.c b/drivers/staging/rtl8188eu/hal/mac_cfg.c
index 0bc1b21..6ed5e15 100644
--- a/drivers/staging/rtl8188eu/hal/mac_cfg.c
+++ b/drivers/staging/rtl8188eu/hal/mac_cfg.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
-* You should have received a copy of the GNU General Public License along with
-* this program; if not, write to the Free Software Foundation, Inc.,
-* 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
-*
-*
******************************************************************************/
#include "odm_precomp.h"
diff --git a/drivers/staging/rtl8188eu/hal/odm.c b/drivers/staging/rtl8188eu/hal/odm.c
index 8d2316b..57a1275 100644
--- a/drivers/staging/rtl8188eu/hal/odm.c
+++ b/drivers/staging/rtl8188eu/hal/odm.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
/* include files */
diff --git a/drivers/staging/rtl8188eu/hal/odm_HWConfig.c b/drivers/staging/rtl8188eu/hal/odm_HWConfig.c
index 28b9f7f..0555e42 100644
--- a/drivers/staging/rtl8188eu/hal/odm_HWConfig.c
+++ b/drivers/staging/rtl8188eu/hal/odm_HWConfig.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
/* include files */
diff --git a/drivers/staging/rtl8188eu/hal/odm_RTL8188E.c b/drivers/staging/rtl8188eu/hal/odm_RTL8188E.c
index c0242a0..dd9b902 100644
--- a/drivers/staging/rtl8188eu/hal/odm_RTL8188E.c
+++ b/drivers/staging/rtl8188eu/hal/odm_RTL8188E.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#include "odm_precomp.h"
diff --git a/drivers/staging/rtl8188eu/hal/phy.c b/drivers/staging/rtl8188eu/hal/phy.c
index ae42b44..a83bbea9 100644
--- a/drivers/staging/rtl8188eu/hal/phy.c
+++ b/drivers/staging/rtl8188eu/hal/phy.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _RTL8188E_PHYCFG_C_
diff --git a/drivers/staging/rtl8188eu/hal/pwrseq.c b/drivers/staging/rtl8188eu/hal/pwrseq.c
index 20dce42..d92a34e 100644
--- a/drivers/staging/rtl8188eu/hal/pwrseq.c
+++ b/drivers/staging/rtl8188eu/hal/pwrseq.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#include "pwrseq.h"
diff --git a/drivers/staging/rtl8188eu/hal/pwrseqcmd.c b/drivers/staging/rtl8188eu/hal/pwrseqcmd.c
index b76b0f5..2867864 100644
--- a/drivers/staging/rtl8188eu/hal/pwrseqcmd.c
+++ b/drivers/staging/rtl8188eu/hal/pwrseqcmd.c
@@ -11,10 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
******************************************************************************/
#include <pwrseqcmd.h>
diff --git a/drivers/staging/rtl8188eu/hal/rf.c b/drivers/staging/rtl8188eu/hal/rf.c
index 38845d1..1596274 100644
--- a/drivers/staging/rtl8188eu/hal/rf.c
+++ b/drivers/staging/rtl8188eu/hal/rf.c
@@ -11,10 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
******************************************************************************/
#include <osdep_service.h>
diff --git a/drivers/staging/rtl8188eu/hal/rf_cfg.c b/drivers/staging/rtl8188eu/hal/rf_cfg.c
index 4494542..453f9e7 100644
--- a/drivers/staging/rtl8188eu/hal/rf_cfg.c
+++ b/drivers/staging/rtl8188eu/hal/rf_cfg.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
-* You should have received a copy of the GNU General Public License along with
-* this program; if not, write to the Free Software Foundation, Inc.,
-* 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
-*
-*
******************************************************************************/
#include "odm_precomp.h"
diff --git a/drivers/staging/rtl8188eu/hal/rtl8188e_cmd.c b/drivers/staging/rtl8188eu/hal/rtl8188e_cmd.c
index 5808763..2422c02 100644
--- a/drivers/staging/rtl8188eu/hal/rtl8188e_cmd.c
+++ b/drivers/staging/rtl8188eu/hal/rtl8188e_cmd.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _RTL8188E_CMD_C_
diff --git a/drivers/staging/rtl8188eu/hal/rtl8188e_dm.c b/drivers/staging/rtl8188eu/hal/rtl8188e_dm.c
index f9919a9..81f2931 100644
--- a/drivers/staging/rtl8188eu/hal/rtl8188e_dm.c
+++ b/drivers/staging/rtl8188eu/hal/rtl8188e_dm.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
/* */
/* Description: */
diff --git a/drivers/staging/rtl8188eu/hal/rtl8188e_hal_init.c b/drivers/staging/rtl8188eu/hal/rtl8188e_hal_init.c
index 2592bc2..0b444fd 100644
--- a/drivers/staging/rtl8188eu/hal/rtl8188e_hal_init.c
+++ b/drivers/staging/rtl8188eu/hal/rtl8188e_hal_init.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _HAL_INIT_C_
diff --git a/drivers/staging/rtl8188eu/hal/rtl8188e_rxdesc.c b/drivers/staging/rtl8188eu/hal/rtl8188e_rxdesc.c
index 53cf3ba..f110c96 100644
--- a/drivers/staging/rtl8188eu/hal/rtl8188e_rxdesc.c
+++ b/drivers/staging/rtl8188eu/hal/rtl8188e_rxdesc.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _RTL8188E_REDESC_C_
@@ -45,7 +40,7 @@
struct rx_pkt_attrib *pattrib;
struct signal_stat *signal_stat;
- if (prframe == NULL || padapter == NULL)
+ if (!prframe || !padapter)
return;
pattrib = &prframe->attrib;
@@ -64,7 +59,7 @@
void rtl8188e_process_phy_info(struct adapter *padapter, void *prframe)
{
- struct recv_frame *precvframe = (struct recv_frame *)prframe;
+ struct recv_frame *precvframe = prframe;
/* Check RSSI */
process_rssi(padapter, precvframe);
diff --git a/drivers/staging/rtl8188eu/hal/rtl8188e_xmit.c b/drivers/staging/rtl8188eu/hal/rtl8188e_xmit.c
index a6ba53b..460a205 100644
--- a/drivers/staging/rtl8188eu/hal/rtl8188e_xmit.c
+++ b/drivers/staging/rtl8188eu/hal/rtl8188e_xmit.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _RTL8188E_XMIT_C_
diff --git a/drivers/staging/rtl8188eu/hal/rtl8188eu_led.c b/drivers/staging/rtl8188eu/hal/rtl8188eu_led.c
index 564cf53..d9e677e 100644
--- a/drivers/staging/rtl8188eu/hal/rtl8188eu_led.c
+++ b/drivers/staging/rtl8188eu/hal/rtl8188eu_led.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#include <osdep_service.h>
diff --git a/drivers/staging/rtl8188eu/hal/rtl8188eu_recv.c b/drivers/staging/rtl8188eu/hal/rtl8188eu_recv.c
index d6d009a..255d6f2 100644
--- a/drivers/staging/rtl8188eu/hal/rtl8188eu_recv.c
+++ b/drivers/staging/rtl8188eu/hal/rtl8188eu_recv.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _RTL8188EU_RECV_C_
#include <osdep_service.h>
diff --git a/drivers/staging/rtl8188eu/hal/rtl8188eu_xmit.c b/drivers/staging/rtl8188eu/hal/rtl8188eu_xmit.c
index c96d804..ec21d8c 100644
--- a/drivers/staging/rtl8188eu/hal/rtl8188eu_xmit.c
+++ b/drivers/staging/rtl8188eu/hal/rtl8188eu_xmit.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _RTL8188E_XMIT_C_
#include <osdep_service.h>
diff --git a/drivers/staging/rtl8188eu/hal/usb_halinit.c b/drivers/staging/rtl8188eu/hal/usb_halinit.c
index 07a61b8..87ea3b8 100644
--- a/drivers/staging/rtl8188eu/hal/usb_halinit.c
+++ b/drivers/staging/rtl8188eu/hal/usb_halinit.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _HCI_HAL_INIT_C_
@@ -62,8 +57,8 @@
_ConfigNormalChipOutEP_8188E(adapt, NumOutPipe);
/* Normal chip with one IN and one OUT doesn't have interrupt IN EP. */
- if (1 == haldata->OutEpNumber) {
- if (1 != NumInPipe)
+ if (haldata->OutEpNumber == 1) {
+ if (NumInPipe != 1)
return result;
}
@@ -179,7 +174,7 @@
if (haldata->OutEpQueueSel & TX_SELE_LQ)
numLQ = 0x1C;
- /* NOTE: This step shall be proceed before writting REG_RQPN. */
+ /* NOTE: This step shall be proceed before writing REG_RQPN. */
if (haldata->OutEpQueueSel & TX_SELE_NQ)
numNQ = 0x1C;
value8 = (u8)_NPQ(numNQ);
@@ -457,7 +452,8 @@
* When Who Remark
* 12/10/2010 MHC Separate to smaller function.
*
- *---------------------------------------------------------------------------*/
+ *---------------------------------------------------------------------------
+ */
static void usb_AggSettingTxUpdate(struct adapter *Adapter)
{
struct hal_data_8188e *haldata = GET_HAL_DATA(Adapter);
@@ -489,7 +485,8 @@
* When Who Remark
* 12/10/2010 MHC Separate to smaller function.
*
- *---------------------------------------------------------------------------*/
+ *---------------------------------------------------------------------------
+ */
static void
usb_AggSettingRxUpdate(
struct adapter *Adapter
@@ -655,7 +652,8 @@
* Revised History:
* When Who Remark
* 08/23/2010 MHC HW suspend mode switch test..
- *---------------------------------------------------------------------------*/
+ *---------------------------------------------------------------------------
+ */
enum rt_rf_power_state RfOnOffDetect(struct adapter *adapt)
{
u8 val8;
@@ -687,11 +685,9 @@
#define HAL_INIT_PROFILE_TAG(stage) do {} while (0)
-
HAL_INIT_PROFILE_TAG(HAL_INIT_STAGES_BEGIN);
if (Adapter->pwrctrlpriv.bkeepfwalive) {
-
if (haldata->odmpriv.RFCalibrateInfo.bIQKInitialized) {
rtl88eu_phy_iq_calibrate(Adapter, true);
} else {
@@ -715,9 +711,8 @@
/* Save target channel */
haldata->CurrentChannel = 6;/* default set to 6 */
- if (pwrctrlpriv->reg_rfoff) {
+ if (pwrctrlpriv->reg_rfoff)
pwrctrlpriv->rf_pwrstate = rf_off;
- }
/* 2010/08/09 MH We need to check if we need to turnon or off RF after detecting */
/* HW GPIO pin. Before PHY_RFConfig8192C. */
@@ -749,10 +744,9 @@
DBG_88E("%s: Download Firmware failed!!\n", __func__);
Adapter->bFWReady = false;
return status;
- } else {
- RT_TRACE(_module_hci_hal_init_c_, _drv_info_, ("Initializeadapt8192CSdio(): Download Firmware Success!!\n"));
- Adapter->bFWReady = true;
}
+ RT_TRACE(_module_hci_hal_init_c_, _drv_info_, ("Initializeadapt8192CSdio(): Download Firmware Success!!\n"));
+ Adapter->bFWReady = true;
}
rtl8188e_InitializeFirmwareVars(Adapter);
@@ -878,7 +872,7 @@
/* 2010/08/26 MH Merge from 8192CE. */
if (pwrctrlpriv->rf_pwrstate == rf_on) {
if (haldata->odmpriv.RFCalibrateInfo.bIQKInitialized) {
- rtl88eu_phy_iq_calibrate(Adapter, true);
+ rtl88eu_phy_iq_calibrate(Adapter, true);
} else {
rtl88eu_phy_iq_calibrate(Adapter, false);
haldata->odmpriv.RFCalibrateInfo.bIQKInitialized = true;
@@ -905,7 +899,6 @@
DBG_88E("%s in %dms\n", __func__,
jiffies_to_msecs(jiffies - init_start_time));
-
return status;
}
@@ -968,6 +961,7 @@
haldata->bMacPwrCtrlOn = false;
Adapter->bFWReady = false;
}
+
static void rtl8192cu_hw_power_down(struct adapter *adapt)
{
/* 2010/-8/09 MH For power down module, we need to enable register block contrl reg at 0x1c. */
@@ -980,7 +974,6 @@
static u32 rtl8188eu_hal_deinit(struct adapter *Adapter)
{
-
DBG_88E("==> %s\n", __func__);
usb_write32(Adapter, REG_HIMR_88E, IMR_DISABLED_88E);
@@ -999,14 +992,14 @@
}
}
return _SUCCESS;
- }
+}
static unsigned int rtl8188eu_inirp_init(struct adapter *Adapter)
{
u8 i;
struct recv_buf *precvbuf;
uint status;
- struct recv_priv *precvpriv = &(Adapter->recvpriv);
+ struct recv_priv *precvpriv = &Adapter->recvpriv;
status = _SUCCESS;
@@ -1116,7 +1109,6 @@
Hal_ReadAntennaDiversity88E(adapt, eeprom->efuse_eeprom_data, eeprom->bautoload_fail_flag);
Hal_EfuseParseBoardType88E(adapt, eeprom->efuse_eeprom_data, eeprom->bautoload_fail_flag);
Hal_ReadThermalMeter_88E(adapt, eeprom->efuse_eeprom_data, eeprom->bautoload_fail_flag);
-
}
static void _ReadPROMContent(
@@ -1212,7 +1204,7 @@
StopTxBeacon(Adapter);
usb_write8(Adapter, REG_BCN_CTRL, 0x19);/* disable atim wnd */
- } else if ((mode == _HW_STATE_ADHOC_)) {
+ } else if (mode == _HW_STATE_ADHOC_) {
ResumeTxBeacon(Adapter);
usb_write8(Adapter, REG_BCN_CTRL, 0x1a);
} else if (mode == _HW_STATE_AP_) {
@@ -1363,7 +1355,7 @@
{
u64 tsf;
struct mlme_ext_priv *pmlmeext = &Adapter->mlmeextpriv;
- struct mlme_ext_info *pmlmeinfo = &(pmlmeext->mlmext_info);
+ struct mlme_ext_info *pmlmeinfo = &pmlmeext->mlmext_info;
tsf = pmlmeext->TSFValue - rtw_modular64(pmlmeext->TSFValue, (pmlmeinfo->bcn_interval*1024)) - 1024; /* us */
@@ -1420,7 +1412,7 @@
usb_write8(Adapter, REG_BCN_CTRL, usb_read8(Adapter, REG_BCN_CTRL) | BIT(4));
} else { /* sitesurvey done */
struct mlme_ext_priv *pmlmeext = &Adapter->mlmeextpriv;
- struct mlme_ext_info *pmlmeinfo = &(pmlmeext->mlmext_info);
+ struct mlme_ext_info *pmlmeinfo = &pmlmeext->mlmext_info;
if ((is_client_associated_to_ap(Adapter)) ||
((pmlmeinfo->state&0x03) == WIFI_FW_ADHOC_STATE)) {
@@ -1490,7 +1482,7 @@
{
u8 u1bAIFS, aSifsTime;
struct mlme_ext_priv *pmlmeext = &Adapter->mlmeextpriv;
- struct mlme_ext_info *pmlmeinfo = &(pmlmeext->mlmext_info);
+ struct mlme_ext_info *pmlmeinfo = &pmlmeext->mlmext_info;
usb_write8(Adapter, REG_SLOT, val[0]);
@@ -1790,7 +1782,7 @@
}
break;
case HW_VAR_H2C_MEDIA_STATUS_RPT:
- rtl8188e_set_FwMediaStatus_cmd(Adapter , (*(__le16 *)val));
+ rtl8188e_set_FwMediaStatus_cmd(Adapter, (*(__le16 *)val));
break;
case HW_VAR_BCN_VALID:
/* BCN_VALID, BIT16 of REG_TDECTRL = BIT0 of REG_TDECTRL+2, write 1 to clear, Clear by sw */
@@ -1855,7 +1847,6 @@
default:
break;
}
-
}
/* */
@@ -1904,19 +1895,19 @@
case HAL_DEF_RA_DECISION_RATE:
{
u8 MacID = *((u8 *)pValue);
- *((u8 *)pValue) = ODM_RA_GetDecisionRate_8188E(&(haldata->odmpriv), MacID);
+ *((u8 *)pValue) = ODM_RA_GetDecisionRate_8188E(&haldata->odmpriv, MacID);
}
break;
case HAL_DEF_RA_SGI:
{
u8 MacID = *((u8 *)pValue);
- *((u8 *)pValue) = ODM_RA_GetShortGI_8188E(&(haldata->odmpriv), MacID);
+ *((u8 *)pValue) = ODM_RA_GetShortGI_8188E(&haldata->odmpriv, MacID);
}
break;
case HAL_DEF_PT_PWR_STATUS:
{
u8 MacID = *((u8 *)pValue);
- *((u8 *)pValue) = ODM_RA_GetHwPwrStatus_8188E(&(haldata->odmpriv), MacID);
+ *((u8 *)pValue) = ODM_RA_GetHwPwrStatus_8188E(&haldata->odmpriv, MacID);
}
break;
case HW_VAR_MAX_RX_AMPDU_FACTOR:
@@ -1939,7 +1930,7 @@
break;
case HW_DEF_ODM_DBG_FLAG:
{
- struct odm_dm_struct *dm_ocm = &(haldata->odmpriv);
+ struct odm_dm_struct *dm_ocm = &haldata->odmpriv;
pr_info("dm_ocm->DebugComponents = 0x%llx\n", dm_ocm->DebugComponents);
}
break;
@@ -1967,8 +1958,8 @@
struct sta_info *psta;
struct hal_data_8188e *haldata = GET_HAL_DATA(adapt);
struct mlme_ext_priv *pmlmeext = &adapt->mlmeextpriv;
- struct mlme_ext_info *pmlmeinfo = &(pmlmeext->mlmext_info);
- struct wlan_bssid_ex *cur_network = &(pmlmeinfo->network);
+ struct mlme_ext_info *pmlmeinfo = &pmlmeext->mlmext_info;
+ struct wlan_bssid_ex *cur_network = &pmlmeinfo->network;
if (mac_id >= NUM_STA) /* CAM_SIZE */
return;
@@ -1981,8 +1972,8 @@
networkType = judge_network_type(adapt, cur_network->SupportedRates, supportRateNum) & 0xf;
raid = networktype_to_raid(networkType);
mask = update_supported_rate(cur_network->SupportedRates, supportRateNum);
- mask |= (pmlmeinfo->HT_enable) ? update_MSC_rate(&(pmlmeinfo->HT_caps)) : 0;
- if (support_short_GI(adapt, &(pmlmeinfo->HT_caps)))
+ mask |= (pmlmeinfo->HT_enable) ? update_MSC_rate(&pmlmeinfo->HT_caps) : 0;
+ if (support_short_GI(adapt, &pmlmeinfo->HT_caps))
shortGIrate = true;
break;
case 1:/* for broadcast/multicast */
@@ -2023,8 +2014,8 @@
static void SetBeaconRelatedRegisters8188EUsb(struct adapter *adapt)
{
u32 value32;
- struct mlme_ext_priv *pmlmeext = &(adapt->mlmeextpriv);
- struct mlme_ext_info *pmlmeinfo = &(pmlmeext->mlmext_info);
+ struct mlme_ext_priv *pmlmeext = &adapt->mlmeextpriv;
+ struct mlme_ext_info *pmlmeinfo = &pmlmeext->mlmext_info;
u32 bcn_ctrl_reg = REG_BCN_CTRL;
/* reset TSF, enable update TSF, correcting TSF On Beacon */
@@ -2081,9 +2072,8 @@
{
struct hal_ops *halfunc = &adapt->HalFunc;
-
- adapt->HalData = kzalloc(sizeof(struct hal_data_8188e), GFP_KERNEL);
- if (adapt->HalData == NULL)
+ adapt->HalData = kzalloc(sizeof(*adapt->HalData), GFP_KERNEL);
+ if (!adapt->HalData)
DBG_88E("cant not alloc memory for HAL DATA\n");
halfunc->hal_power_on = rtl8188eu_InitPowerOn;
diff --git a/drivers/staging/rtl8188eu/include/Hal8188EPhyCfg.h b/drivers/staging/rtl8188eu/include/Hal8188EPhyCfg.h
index 2670d6b..8990748 100644
--- a/drivers/staging/rtl8188eu/include/Hal8188EPhyCfg.h
+++ b/drivers/staging/rtl8188eu/include/Hal8188EPhyCfg.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __INC_HAL8188EPHYCFG_H__
#define __INC_HAL8188EPHYCFG_H__
diff --git a/drivers/staging/rtl8188eu/include/Hal8188EPhyReg.h b/drivers/staging/rtl8188eu/include/Hal8188EPhyReg.h
index 9f2969b..344c73d 100644
--- a/drivers/staging/rtl8188eu/include/Hal8188EPhyReg.h
+++ b/drivers/staging/rtl8188eu/include/Hal8188EPhyReg.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __INC_HAL8188EPHYREG_H__
#define __INC_HAL8188EPHYREG_H__
diff --git a/drivers/staging/rtl8188eu/include/HalHWImg8188E_FW.h b/drivers/staging/rtl8188eu/include/HalHWImg8188E_FW.h
index 1bf9bc7..dbb5524 100644
--- a/drivers/staging/rtl8188eu/include/HalHWImg8188E_FW.h
+++ b/drivers/staging/rtl8188eu/include/HalHWImg8188E_FW.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
-* You should have received a copy of the GNU General Public License along with
-* this program; if not, write to the Free Software Foundation, Inc.,
-* 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
-*
-*
******************************************************************************/
#ifndef __INC_FW_8188E_HW_IMG_H
diff --git a/drivers/staging/rtl8188eu/include/HalVerDef.h b/drivers/staging/rtl8188eu/include/HalVerDef.h
index 6f2b2a4..d244eff 100644
--- a/drivers/staging/rtl8188eu/include/HalVerDef.h
+++ b/drivers/staging/rtl8188eu/include/HalVerDef.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __HAL_VERSION_DEF_H__
#define __HAL_VERSION_DEF_H__
diff --git a/drivers/staging/rtl8188eu/include/basic_types.h b/drivers/staging/rtl8188eu/include/basic_types.h
index 3fb691d..2c1676d 100644
--- a/drivers/staging/rtl8188eu/include/basic_types.h
+++ b/drivers/staging/rtl8188eu/include/basic_types.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __BASIC_TYPES_H__
#define __BASIC_TYPES_H__
diff --git a/drivers/staging/rtl8188eu/include/drv_types.h b/drivers/staging/rtl8188eu/include/drv_types.h
index dcb032b..55506a7 100644
--- a/drivers/staging/rtl8188eu/include/drv_types.h
+++ b/drivers/staging/rtl8188eu/include/drv_types.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
/*-----------------------------------------------------------------------------
diff --git a/drivers/staging/rtl8188eu/include/fw.h b/drivers/staging/rtl8188eu/include/fw.h
index 7884d8f..b016f32 100644
--- a/drivers/staging/rtl8188eu/include/fw.h
+++ b/drivers/staging/rtl8188eu/include/fw.h
@@ -11,10 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
* The full GNU General Public License is included in this distribution in the
* file called LICENSE.
*
diff --git a/drivers/staging/rtl8188eu/include/hal_com.h b/drivers/staging/rtl8188eu/include/hal_com.h
index 47715d9..aaf4447 100644
--- a/drivers/staging/rtl8188eu/include/hal_com.h
+++ b/drivers/staging/rtl8188eu/include/hal_com.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __HAL_COMMON_H__
#define __HAL_COMMON_H__
diff --git a/drivers/staging/rtl8188eu/include/hal_intf.h b/drivers/staging/rtl8188eu/include/hal_intf.h
index 1b1c102..eaf939b 100644
--- a/drivers/staging/rtl8188eu/include/hal_intf.h
+++ b/drivers/staging/rtl8188eu/include/hal_intf.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __HAL_INTF_H__
#define __HAL_INTF_H__
diff --git a/drivers/staging/rtl8188eu/include/ieee80211.h b/drivers/staging/rtl8188eu/include/ieee80211.h
index f8f5eb6..d8284c8 100644
--- a/drivers/staging/rtl8188eu/include/ieee80211.h
+++ b/drivers/staging/rtl8188eu/include/ieee80211.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __IEEE80211_H
#define __IEEE80211_H
diff --git a/drivers/staging/rtl8188eu/include/mlme_osdep.h b/drivers/staging/rtl8188eu/include/mlme_osdep.h
index ae1722c..5a35b08 100644
--- a/drivers/staging/rtl8188eu/include/mlme_osdep.h
+++ b/drivers/staging/rtl8188eu/include/mlme_osdep.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __MLME_OSDEP_H_
#define __MLME_OSDEP_H_
diff --git a/drivers/staging/rtl8188eu/include/mp_custom_oid.h b/drivers/staging/rtl8188eu/include/mp_custom_oid.h
index 6fa52cf..1a06ee6 100644
--- a/drivers/staging/rtl8188eu/include/mp_custom_oid.h
+++ b/drivers/staging/rtl8188eu/include/mp_custom_oid.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __CUSTOM_OID_H
#define __CUSTOM_OID_H
diff --git a/drivers/staging/rtl8188eu/include/odm.h b/drivers/staging/rtl8188eu/include/odm.h
index af781c7..dbebf17 100644
--- a/drivers/staging/rtl8188eu/include/odm.h
+++ b/drivers/staging/rtl8188eu/include/odm.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
diff --git a/drivers/staging/rtl8188eu/include/odm_HWConfig.h b/drivers/staging/rtl8188eu/include/odm_HWConfig.h
index ef792bf..da7325d 100644
--- a/drivers/staging/rtl8188eu/include/odm_HWConfig.h
+++ b/drivers/staging/rtl8188eu/include/odm_HWConfig.h
@@ -11,10 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
*
******************************************************************************/
diff --git a/drivers/staging/rtl8188eu/include/odm_RTL8188E.h b/drivers/staging/rtl8188eu/include/odm_RTL8188E.h
index 14dce6c..72b4db6 100644
--- a/drivers/staging/rtl8188eu/include/odm_RTL8188E.h
+++ b/drivers/staging/rtl8188eu/include/odm_RTL8188E.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __ODM_RTL8188E_H__
#define __ODM_RTL8188E_H__
diff --git a/drivers/staging/rtl8188eu/include/odm_RegDefine11N.h b/drivers/staging/rtl8188eu/include/odm_RegDefine11N.h
index 5a61f90..c82c090 100644
--- a/drivers/staging/rtl8188eu/include/odm_RegDefine11N.h
+++ b/drivers/staging/rtl8188eu/include/odm_RegDefine11N.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __ODM_REGDEFINE11N_H__
diff --git a/drivers/staging/rtl8188eu/include/odm_debug.h b/drivers/staging/rtl8188eu/include/odm_debug.h
index e939096..52e51f19 100644
--- a/drivers/staging/rtl8188eu/include/odm_debug.h
+++ b/drivers/staging/rtl8188eu/include/odm_debug.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
diff --git a/drivers/staging/rtl8188eu/include/odm_precomp.h b/drivers/staging/rtl8188eu/include/odm_precomp.h
index 0f236da..9e5fe177 100644
--- a/drivers/staging/rtl8188eu/include/odm_precomp.h
+++ b/drivers/staging/rtl8188eu/include/odm_precomp.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __ODM_PRECOMP_H__
diff --git a/drivers/staging/rtl8188eu/include/odm_reg.h b/drivers/staging/rtl8188eu/include/odm_reg.h
index 7f10b69..3405a44 100644
--- a/drivers/staging/rtl8188eu/include/odm_reg.h
+++ b/drivers/staging/rtl8188eu/include/odm_reg.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
/* */
/* File Name: odm_reg.h */
diff --git a/drivers/staging/rtl8188eu/include/odm_types.h b/drivers/staging/rtl8188eu/include/odm_types.h
index c1355b9..3474a9c 100644
--- a/drivers/staging/rtl8188eu/include/odm_types.h
+++ b/drivers/staging/rtl8188eu/include/odm_types.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __ODM_TYPES_H__
#define __ODM_TYPES_H__
diff --git a/drivers/staging/rtl8188eu/include/osdep_intf.h b/drivers/staging/rtl8188eu/include/osdep_intf.h
index 1521744..54fca79 100644
--- a/drivers/staging/rtl8188eu/include/osdep_intf.h
+++ b/drivers/staging/rtl8188eu/include/osdep_intf.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __OSDEP_INTF_H_
diff --git a/drivers/staging/rtl8188eu/include/osdep_service.h b/drivers/staging/rtl8188eu/include/osdep_service.h
index 22de53d..5475956 100644
--- a/drivers/staging/rtl8188eu/include/osdep_service.h
+++ b/drivers/staging/rtl8188eu/include/osdep_service.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __OSDEP_SERVICE_H_
#define __OSDEP_SERVICE_H_
diff --git a/drivers/staging/rtl8188eu/include/pwrseq.h b/drivers/staging/rtl8188eu/include/pwrseq.h
index 9dbf843..afd61cf 100644
--- a/drivers/staging/rtl8188eu/include/pwrseq.h
+++ b/drivers/staging/rtl8188eu/include/pwrseq.h
@@ -12,11 +12,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __HAL8188EPWRSEQ_H__
diff --git a/drivers/staging/rtl8188eu/include/pwrseqcmd.h b/drivers/staging/rtl8188eu/include/pwrseqcmd.h
index 468a3fb..c4a919e 100644
--- a/drivers/staging/rtl8188eu/include/pwrseqcmd.h
+++ b/drivers/staging/rtl8188eu/include/pwrseqcmd.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __HALPWRSEQCMD_H__
#define __HALPWRSEQCMD_H__
diff --git a/drivers/staging/rtl8188eu/include/recv_osdep.h b/drivers/staging/rtl8188eu/include/recv_osdep.h
index fdeb603..cad3158 100644
--- a/drivers/staging/rtl8188eu/include/recv_osdep.h
+++ b/drivers/staging/rtl8188eu/include/recv_osdep.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __RECV_OSDEP_H_
#define __RECV_OSDEP_H_
diff --git a/drivers/staging/rtl8188eu/include/rtl8188e_cmd.h b/drivers/staging/rtl8188eu/include/rtl8188e_cmd.h
index f813ce0..4d7d804 100644
--- a/drivers/staging/rtl8188eu/include/rtl8188e_cmd.h
+++ b/drivers/staging/rtl8188eu/include/rtl8188e_cmd.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __RTL8188E_CMD_H__
#define __RTL8188E_CMD_H__
diff --git a/drivers/staging/rtl8188eu/include/rtl8188e_dm.h b/drivers/staging/rtl8188eu/include/rtl8188e_dm.h
index 5e0ac31..4190112 100644
--- a/drivers/staging/rtl8188eu/include/rtl8188e_dm.h
+++ b/drivers/staging/rtl8188eu/include/rtl8188e_dm.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __RTL8188E_DM_H__
#define __RTL8188E_DM_H__
diff --git a/drivers/staging/rtl8188eu/include/rtl8188e_hal.h b/drivers/staging/rtl8188eu/include/rtl8188e_hal.h
index 9f5050e..9dd5c29 100644
--- a/drivers/staging/rtl8188eu/include/rtl8188e_hal.h
+++ b/drivers/staging/rtl8188eu/include/rtl8188e_hal.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __RTL8188E_HAL_H__
#define __RTL8188E_HAL_H__
diff --git a/drivers/staging/rtl8188eu/include/rtl8188e_led.h b/drivers/staging/rtl8188eu/include/rtl8188e_led.h
index c0147e7..fca6d8c 100644
--- a/drivers/staging/rtl8188eu/include/rtl8188e_led.h
+++ b/drivers/staging/rtl8188eu/include/rtl8188e_led.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __RTL8188E_LED_H__
#define __RTL8188E_LED_H__
diff --git a/drivers/staging/rtl8188eu/include/rtl8188e_recv.h b/drivers/staging/rtl8188eu/include/rtl8188e_recv.h
index 5fed30d..54048bc 100644
--- a/drivers/staging/rtl8188eu/include/rtl8188e_recv.h
+++ b/drivers/staging/rtl8188eu/include/rtl8188e_recv.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __RTL8188E_RECV_H__
#define __RTL8188E_RECV_H__
diff --git a/drivers/staging/rtl8188eu/include/rtl8188e_spec.h b/drivers/staging/rtl8188eu/include/rtl8188e_spec.h
index beeee4a6..fb82f66 100644
--- a/drivers/staging/rtl8188eu/include/rtl8188e_spec.h
+++ b/drivers/staging/rtl8188eu/include/rtl8188e_spec.h
@@ -11,10 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
*******************************************************************************/
#ifndef __RTL8188E_SPEC_H__
#define __RTL8188E_SPEC_H__
diff --git a/drivers/staging/rtl8188eu/include/rtl8188e_xmit.h b/drivers/staging/rtl8188eu/include/rtl8188e_xmit.h
index 0b96d42..65a63df 100644
--- a/drivers/staging/rtl8188eu/include/rtl8188e_xmit.h
+++ b/drivers/staging/rtl8188eu/include/rtl8188e_xmit.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __RTL8188E_XMIT_H__
#define __RTL8188E_XMIT_H__
diff --git a/drivers/staging/rtl8188eu/include/rtw_android.h b/drivers/staging/rtl8188eu/include/rtw_android.h
index e85bf1f..e81ee92 100644
--- a/drivers/staging/rtl8188eu/include/rtw_android.h
+++ b/drivers/staging/rtl8188eu/include/rtw_android.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __RTW_ANDROID_H__
diff --git a/drivers/staging/rtl8188eu/include/rtw_ap.h b/drivers/staging/rtl8188eu/include/rtw_ap.h
index 6128ccc..b820684 100644
--- a/drivers/staging/rtl8188eu/include/rtw_ap.h
+++ b/drivers/staging/rtl8188eu/include/rtw_ap.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __RTW_AP_H_
#define __RTW_AP_H_
diff --git a/drivers/staging/rtl8188eu/include/rtw_cmd.h b/drivers/staging/rtl8188eu/include/rtw_cmd.h
index 9e9f5f4..08ca592 100644
--- a/drivers/staging/rtl8188eu/include/rtw_cmd.h
+++ b/drivers/staging/rtl8188eu/include/rtw_cmd.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __RTW_CMD_H_
#define __RTW_CMD_H_
diff --git a/drivers/staging/rtl8188eu/include/rtw_debug.h b/drivers/staging/rtl8188eu/include/rtw_debug.h
index 971bf45..7ed4cad 100644
--- a/drivers/staging/rtl8188eu/include/rtw_debug.h
+++ b/drivers/staging/rtl8188eu/include/rtw_debug.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __RTW_DEBUG_H__
#define __RTW_DEBUG_H__
diff --git a/drivers/staging/rtl8188eu/include/rtw_eeprom.h b/drivers/staging/rtl8188eu/include/rtw_eeprom.h
index 904fea1..5dd7384 100644
--- a/drivers/staging/rtl8188eu/include/rtw_eeprom.h
+++ b/drivers/staging/rtl8188eu/include/rtw_eeprom.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __RTW_EEPROM_H__
#define __RTW_EEPROM_H__
diff --git a/drivers/staging/rtl8188eu/include/rtw_efuse.h b/drivers/staging/rtl8188eu/include/rtw_efuse.h
index 5660eed..9bfb10c 100644
--- a/drivers/staging/rtl8188eu/include/rtw_efuse.h
+++ b/drivers/staging/rtl8188eu/include/rtw_efuse.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __RTW_EFUSE_H__
#define __RTW_EFUSE_H__
diff --git a/drivers/staging/rtl8188eu/include/rtw_event.h b/drivers/staging/rtl8188eu/include/rtw_event.h
index 52151dc..5c34e56 100644
--- a/drivers/staging/rtl8188eu/include/rtw_event.h
+++ b/drivers/staging/rtl8188eu/include/rtw_event.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef _RTW_EVENT_H_
#define _RTW_EVENT_H_
diff --git a/drivers/staging/rtl8188eu/include/rtw_ht.h b/drivers/staging/rtl8188eu/include/rtw_ht.h
index beb210b..b45483f 100644
--- a/drivers/staging/rtl8188eu/include/rtw_ht.h
+++ b/drivers/staging/rtl8188eu/include/rtw_ht.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef _RTW_HT_H_
#define _RTW_HT_H_
diff --git a/drivers/staging/rtl8188eu/include/rtw_ioctl.h b/drivers/staging/rtl8188eu/include/rtw_ioctl.h
index ee2cb54..3a652df 100644
--- a/drivers/staging/rtl8188eu/include/rtw_ioctl.h
+++ b/drivers/staging/rtl8188eu/include/rtw_ioctl.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef _RTW_IOCTL_H_
#define _RTW_IOCTL_H_
diff --git a/drivers/staging/rtl8188eu/include/rtw_ioctl_rtl.h b/drivers/staging/rtl8188eu/include/rtw_ioctl_rtl.h
index 8fa3858..da4949f 100644
--- a/drivers/staging/rtl8188eu/include/rtw_ioctl_rtl.h
+++ b/drivers/staging/rtl8188eu/include/rtw_ioctl_rtl.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef _RTW_IOCTL_RTL_H_
#define _RTW_IOCTL_RTL_H_
diff --git a/drivers/staging/rtl8188eu/include/rtw_ioctl_set.h b/drivers/staging/rtl8188eu/include/rtw_ioctl_set.h
index fa9d655..b6e14a8 100644
--- a/drivers/staging/rtl8188eu/include/rtw_ioctl_set.h
+++ b/drivers/staging/rtl8188eu/include/rtw_ioctl_set.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __RTW_IOCTL_SET_H_
#define __RTW_IOCTL_SET_H_
diff --git a/drivers/staging/rtl8188eu/include/rtw_iol.h b/drivers/staging/rtl8188eu/include/rtw_iol.h
index 68aae7f..1f324e6 100644
--- a/drivers/staging/rtl8188eu/include/rtw_iol.h
+++ b/drivers/staging/rtl8188eu/include/rtw_iol.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __RTW_IOL_H_
#define __RTW_IOL_H_
diff --git a/drivers/staging/rtl8188eu/include/rtw_mlme.h b/drivers/staging/rtl8188eu/include/rtw_mlme.h
index 4c99257..5d8bce0 100644
--- a/drivers/staging/rtl8188eu/include/rtw_mlme.h
+++ b/drivers/staging/rtl8188eu/include/rtw_mlme.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __RTW_MLME_H_
#define __RTW_MLME_H_
diff --git a/drivers/staging/rtl8188eu/include/rtw_mlme_ext.h b/drivers/staging/rtl8188eu/include/rtw_mlme_ext.h
index 4471133..27382ff 100644
--- a/drivers/staging/rtl8188eu/include/rtw_mlme_ext.h
+++ b/drivers/staging/rtl8188eu/include/rtw_mlme_ext.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __RTW_MLME_EXT_H_
#define __RTW_MLME_EXT_H_
diff --git a/drivers/staging/rtl8188eu/include/rtw_mp_phy_regdef.h b/drivers/staging/rtl8188eu/include/rtw_mp_phy_regdef.h
index 30fd17f..02b3002 100644
--- a/drivers/staging/rtl8188eu/include/rtw_mp_phy_regdef.h
+++ b/drivers/staging/rtl8188eu/include/rtw_mp_phy_regdef.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
/*****************************************************************************
*
diff --git a/drivers/staging/rtl8188eu/include/rtw_pwrctrl.h b/drivers/staging/rtl8188eu/include/rtw_pwrctrl.h
index a493d4c..9680e2e 100644
--- a/drivers/staging/rtl8188eu/include/rtw_pwrctrl.h
+++ b/drivers/staging/rtl8188eu/include/rtw_pwrctrl.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __RTW_PWRCTRL_H_
#define __RTW_PWRCTRL_H_
diff --git a/drivers/staging/rtl8188eu/include/rtw_qos.h b/drivers/staging/rtl8188eu/include/rtw_qos.h
index bbee1dd..45a77f6 100644
--- a/drivers/staging/rtl8188eu/include/rtw_qos.h
+++ b/drivers/staging/rtl8188eu/include/rtw_qos.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef _RTW_QOS_H_
#define _RTW_QOS_H_
diff --git a/drivers/staging/rtl8188eu/include/rtw_recv.h b/drivers/staging/rtl8188eu/include/rtw_recv.h
index eb1ac3d..b0373b6 100644
--- a/drivers/staging/rtl8188eu/include/rtw_recv.h
+++ b/drivers/staging/rtl8188eu/include/rtw_recv.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef _RTW_RECV_H_
#define _RTW_RECV_H_
diff --git a/drivers/staging/rtl8188eu/include/rtw_rf.h b/drivers/staging/rtl8188eu/include/rtw_rf.h
index 35f61be..66896af 100644
--- a/drivers/staging/rtl8188eu/include/rtw_rf.h
+++ b/drivers/staging/rtl8188eu/include/rtw_rf.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __RTW_RF_H_
#define __RTW_RF_H_
diff --git a/drivers/staging/rtl8188eu/include/rtw_security.h b/drivers/staging/rtl8188eu/include/rtw_security.h
index a1aebe6..ca1247b 100644
--- a/drivers/staging/rtl8188eu/include/rtw_security.h
+++ b/drivers/staging/rtl8188eu/include/rtw_security.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __RTW_SECURITY_H_
#define __RTW_SECURITY_H_
diff --git a/drivers/staging/rtl8188eu/include/rtw_sreset.h b/drivers/staging/rtl8188eu/include/rtw_sreset.h
index 3a62ed0..ce027df 100644
--- a/drivers/staging/rtl8188eu/include/rtw_sreset.h
+++ b/drivers/staging/rtl8188eu/include/rtw_sreset.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef _RTW_SRESET_C_
#define _RTW_SRESET_C_
diff --git a/drivers/staging/rtl8188eu/include/rtw_xmit.h b/drivers/staging/rtl8188eu/include/rtw_xmit.h
index b7c2088..a0853ba 100644
--- a/drivers/staging/rtl8188eu/include/rtw_xmit.h
+++ b/drivers/staging/rtl8188eu/include/rtw_xmit.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef _RTW_XMIT_H_
#define _RTW_XMIT_H_
diff --git a/drivers/staging/rtl8188eu/include/sta_info.h b/drivers/staging/rtl8188eu/include/sta_info.h
index d4e7832..42a03512 100644
--- a/drivers/staging/rtl8188eu/include/sta_info.h
+++ b/drivers/staging/rtl8188eu/include/sta_info.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __STA_INFO_H_
#define __STA_INFO_H_
diff --git a/drivers/staging/rtl8188eu/include/usb_hal.h b/drivers/staging/rtl8188eu/include/usb_hal.h
index 8a65995..b1bf07a 100644
--- a/drivers/staging/rtl8188eu/include/usb_hal.h
+++ b/drivers/staging/rtl8188eu/include/usb_hal.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __USB_HAL_H__
#define __USB_HAL_H__
diff --git a/drivers/staging/rtl8188eu/include/usb_ops_linux.h b/drivers/staging/rtl8188eu/include/usb_ops_linux.h
index 4fdc536..2207333 100644
--- a/drivers/staging/rtl8188eu/include/usb_ops_linux.h
+++ b/drivers/staging/rtl8188eu/include/usb_ops_linux.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __USB_OPS_LINUX_H__
#define __USB_OPS_LINUX_H__
diff --git a/drivers/staging/rtl8188eu/include/wifi.h b/drivers/staging/rtl8188eu/include/wifi.h
index 6cb5bec..e7c5121 100644
--- a/drivers/staging/rtl8188eu/include/wifi.h
+++ b/drivers/staging/rtl8188eu/include/wifi.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef _WIFI_H_
#define _WIFI_H_
diff --git a/drivers/staging/rtl8188eu/include/wlan_bssdef.h b/drivers/staging/rtl8188eu/include/wlan_bssdef.h
index 85b99da..560966c 100644
--- a/drivers/staging/rtl8188eu/include/wlan_bssdef.h
+++ b/drivers/staging/rtl8188eu/include/wlan_bssdef.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __WLAN_BSSDEF_H__
#define __WLAN_BSSDEF_H__
diff --git a/drivers/staging/rtl8188eu/include/xmit_osdep.h b/drivers/staging/rtl8188eu/include/xmit_osdep.h
index 13965f2..f96ca6a 100644
--- a/drivers/staging/rtl8188eu/include/xmit_osdep.h
+++ b/drivers/staging/rtl8188eu/include/xmit_osdep.h
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#ifndef __XMIT_OSDEP_H_
#define __XMIT_OSDEP_H_
diff --git a/drivers/staging/rtl8188eu/os_dep/ioctl_linux.c b/drivers/staging/rtl8188eu/os_dep/ioctl_linux.c
index 9119804..5672f01 100644
--- a/drivers/staging/rtl8188eu/os_dep/ioctl_linux.c
+++ b/drivers/staging/rtl8188eu/os_dep/ioctl_linux.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _IOCTL_LINUX_C_
@@ -2120,13 +2115,13 @@
u8 res = _SUCCESS;
ph2c = kzalloc(sizeof(struct cmd_obj), GFP_KERNEL);
- if (ph2c == NULL) {
+ if (!ph2c) {
res = _FAIL;
goto exit;
}
psetstakey_para = kzalloc(sizeof(struct set_stakey_parm), GFP_KERNEL);
- if (psetstakey_para == NULL) {
+ if (!psetstakey_para) {
kfree(ph2c);
res = _FAIL;
goto exit;
@@ -2158,12 +2153,12 @@
DBG_88E("%s\n", __func__);
pcmd = kzalloc(sizeof(struct cmd_obj), GFP_KERNEL);
- if (pcmd == NULL) {
+ if (!pcmd) {
res = _FAIL;
goto exit;
}
psetkeyparm = kzalloc(sizeof(struct setkey_parm), GFP_KERNEL);
- if (psetkeyparm == NULL) {
+ if (!psetkeyparm) {
kfree(pcmd);
res = _FAIL;
goto exit;
diff --git a/drivers/staging/rtl8188eu/os_dep/mlme_linux.c b/drivers/staging/rtl8188eu/os_dep/mlme_linux.c
index 08bfa76..bc75626 100644
--- a/drivers/staging/rtl8188eu/os_dep/mlme_linux.c
+++ b/drivers/staging/rtl8188eu/os_dep/mlme_linux.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
diff --git a/drivers/staging/rtl8188eu/os_dep/os_intfs.c b/drivers/staging/rtl8188eu/os_dep/os_intfs.c
index 7986e67..ae2caff 100644
--- a/drivers/staging/rtl8188eu/os_dep/os_intfs.c
+++ b/drivers/staging/rtl8188eu/os_dep/os_intfs.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _OS_INTFS_C_
diff --git a/drivers/staging/rtl8188eu/os_dep/osdep_service.c b/drivers/staging/rtl8188eu/os_dep/osdep_service.c
index f090bef..764250b 100644
--- a/drivers/staging/rtl8188eu/os_dep/osdep_service.c
+++ b/drivers/staging/rtl8188eu/os_dep/osdep_service.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
diff --git a/drivers/staging/rtl8188eu/os_dep/recv_linux.c b/drivers/staging/rtl8188eu/os_dep/recv_linux.c
index d4734baf..0c44914 100644
--- a/drivers/staging/rtl8188eu/os_dep/recv_linux.c
+++ b/drivers/staging/rtl8188eu/os_dep/recv_linux.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#include <osdep_service.h>
#include <drv_types.h>
diff --git a/drivers/staging/rtl8188eu/os_dep/rtw_android.c b/drivers/staging/rtl8188eu/os_dep/rtw_android.c
index 5f3337c..41e1b1d 100644
--- a/drivers/staging/rtl8188eu/os_dep/rtw_android.c
+++ b/drivers/staging/rtl8188eu/os_dep/rtw_android.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#include <linux/module.h>
diff --git a/drivers/staging/rtl8188eu/os_dep/usb_intf.c b/drivers/staging/rtl8188eu/os_dep/usb_intf.c
index 794cc11..11d51a3 100644
--- a/drivers/staging/rtl8188eu/os_dep/usb_intf.c
+++ b/drivers/staging/rtl8188eu/os_dep/usb_intf.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define pr_fmt(fmt) "R8188EU: " fmt
@@ -65,7 +60,7 @@
struct usb_device *pusbd;
pdvobjpriv = kzalloc(sizeof(*pdvobjpriv), GFP_KERNEL);
- if (pdvobjpriv == NULL)
+ if (!pdvobjpriv)
return NULL;
pdvobjpriv->pusbintf = usb_intf;
diff --git a/drivers/staging/rtl8188eu/os_dep/usb_ops_linux.c b/drivers/staging/rtl8188eu/os_dep/usb_ops_linux.c
index 0fea338..ce1e1a1 100644
--- a/drivers/staging/rtl8188eu/os_dep/usb_ops_linux.c
+++ b/drivers/staging/rtl8188eu/os_dep/usb_ops_linux.c
@@ -11,10 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
******************************************************************************/
#define _USB_OPS_LINUX_C_
diff --git a/drivers/staging/rtl8188eu/os_dep/xmit_linux.c b/drivers/staging/rtl8188eu/os_dep/xmit_linux.c
index 1593e28..221e275 100644
--- a/drivers/staging/rtl8188eu/os_dep/xmit_linux.c
+++ b/drivers/staging/rtl8188eu/os_dep/xmit_linux.c
@@ -11,11 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
- *
******************************************************************************/
#define _XMIT_OSDEP_C_
diff --git a/drivers/staging/rtl8192u/ieee80211/ieee80211_rx.c b/drivers/staging/rtl8192u/ieee80211/ieee80211_rx.c
index f18fc0b..051c2be 100644
--- a/drivers/staging/rtl8192u/ieee80211/ieee80211_rx.c
+++ b/drivers/staging/rtl8192u/ieee80211/ieee80211_rx.c
@@ -746,7 +746,7 @@
// Indicate packets
if(index>REORDER_WIN_SIZE){
- IEEE80211_DEBUG(IEEE80211_DL_ERR, "RxReorderIndicatePacket(): Rx Reorer buffer full!! \n");
+ IEEE80211_DEBUG(IEEE80211_DL_ERR, "RxReorderIndicatePacket(): Rx Reorder buffer full!! \n");
kfree(prxbIndicateArray);
return;
}
diff --git a/drivers/staging/rtl8192u/ieee80211/rtl819x_TSProc.c b/drivers/staging/rtl8192u/ieee80211/rtl819x_TSProc.c
index 148d0d4..6033502 100644
--- a/drivers/staging/rtl8192u/ieee80211/rtl819x_TSProc.c
+++ b/drivers/staging/rtl8192u/ieee80211/rtl819x_TSProc.c
@@ -75,7 +75,7 @@
// Indicate packets
if(index > REORDER_WIN_SIZE){
- IEEE80211_DEBUG(IEEE80211_DL_ERR, "RxReorderIndicatePacket(): Rx Reorer buffer full!! \n");
+ IEEE80211_DEBUG(IEEE80211_DL_ERR, "RxReorderIndicatePacket(): Rx Reorder buffer full!! \n");
spin_unlock_irqrestore(&(ieee->reorder_spinlock), flags);
return;
}
diff --git a/drivers/staging/rtl8192u/r8190_rtl8256.c b/drivers/staging/rtl8192u/r8190_rtl8256.c
index 5c3bb3b..d733fb2 100644
--- a/drivers/staging/rtl8192u/r8190_rtl8256.c
+++ b/drivers/staging/rtl8192u/r8190_rtl8256.c
@@ -194,7 +194,7 @@
break;
}
- /*----Restore RFENV control type----*/;
+ /*----Restore RFENV control type----*/
switch (eRFPath) {
case RF90_PATH_A:
case RF90_PATH_C:
diff --git a/drivers/staging/rtl8192u/r8192U_core.c b/drivers/staging/rtl8192u/r8192U_core.c
index 4af0140..8c1d737 100644
--- a/drivers/staging/rtl8192u/r8192U_core.c
+++ b/drivers/staging/rtl8192u/r8192U_core.c
@@ -235,7 +235,6 @@
*/
ulcommand |= BIT(31) | BIT(30);
write_nic_dword(dev, RWCAM, ulcommand);
-
}
@@ -298,6 +297,7 @@
return 0;
}
+
/* as 92U has extend page from 4 to 16, so modify functions below. */
void write_nic_byte(struct net_device *dev, int indx, u8 data)
{
@@ -319,14 +319,11 @@
if (status < 0)
netdev_err(dev, "write_nic_byte TimeOut! status: %d\n", status);
-
-
}
void write_nic_word(struct net_device *dev, int indx, u16 data)
{
-
int status;
struct r8192_priv *priv = (struct r8192_priv *)ieee80211_priv(dev);
@@ -345,13 +342,11 @@
if (status < 0)
netdev_err(dev, "write_nic_word TimeOut! status: %d\n", status);
-
}
void write_nic_dword(struct net_device *dev, int indx, u32 data)
{
-
int status;
struct r8192_priv *priv = (struct r8192_priv *)ieee80211_priv(dev);
@@ -372,7 +367,6 @@
if (status < 0)
netdev_err(dev, "write_nic_dword TimeOut! status: %d\n",
status);
-
}
@@ -738,7 +732,6 @@
* master (see the create BSS/IBSS func)
*/
if (priv->ieee80211->state == IEEE80211_LINKED) {
-
if (priv->ieee80211->iw_mode == IW_MODE_INFRA)
msr |= (MSR_LINK_MANAGED << MSR_LINK_SHIFT);
else if (priv->ieee80211->iw_mode == IW_MODE_ADHOC)
@@ -773,11 +766,10 @@
static u32 get_rxpacket_shiftbytes_819xusb(struct ieee80211_rx_stats *pstats)
{
-
return (sizeof(rx_desc_819x_usb) + pstats->RxDrvInfoSize
+ pstats->RxBufShift);
-
}
+
static int rtl8192_rx_initiate(struct net_device *dev)
{
struct r8192_priv *priv = (struct r8192_priv *)ieee80211_priv(dev);
@@ -874,6 +866,7 @@
write_nic_dword(dev, RCR, rxconf);
}
+
/* wait to be removed */
void rtl8192_rx_enable(struct net_device *dev)
{
@@ -943,9 +936,9 @@
return 11;
default:
return 3;
-
}
}
+
static u16 rtl_rate[] = {10, 20, 55, 110, 60, 90, 120, 180, 240, 360, 480, 540};
inline u16 rtl8192_rate2rate(short rate)
{
@@ -1050,7 +1043,7 @@
spin_lock_irqsave(&priv->tx_lock, flags);
- memcpy((unsigned char *)(skb->cb), &dev, sizeof(dev));
+ *(struct net_device **)(skb->cb) = dev;
tcb_desc->bTxEnableFwCalcDur = 1;
skb_push(skb, priv->ieee80211->tx_headroom);
ret = rtl8192_tx(dev, skb);
@@ -1100,7 +1093,7 @@
if (!skb)
return;
- dev = (struct net_device *)(skb->cb);
+ dev = *(struct net_device **)(skb->cb);
tcb_desc = (cb_desc *)(skb->cb + MAX_DEV_ADDR_SIZE);
queue_index = tcb_desc->queue_index;
@@ -1149,7 +1142,6 @@
return; /* avoid further processing AMSDU */
}
}
-
}
static void rtl8192_config_rate(struct net_device *dev, u16 *rate_config)
@@ -1272,11 +1264,10 @@
priv->slot_time = slot_time;
write_nic_byte(dev, SLOT_TIME, slot_time);
}
-
}
+
static void rtl8192_net_update(struct net_device *dev)
{
-
struct r8192_priv *priv = ieee80211_priv(dev);
struct ieee80211_network *net;
u16 BcnTimeCfg = 0, BcnCW = 6, BcnIFS = 0xf;
@@ -1303,9 +1294,6 @@
write_nic_word(dev, BCN_TCFG, BcnTimeCfg);
}
-
-
-
}
/* temporary hw beacon is not used any more.
@@ -1315,6 +1303,7 @@
{
}
+
inline u8 rtl8192_IsWirelessBMode(u16 rate)
{
if (((rate <= 110) && (rate != 60) && (rate != 90)) || (rate == 220))
@@ -1737,7 +1726,6 @@
#ifndef JACKSON_NEW_RX
for (i = 0; i < (MAX_RX_URB + 1); i++) {
-
priv->rx_urb[i] = usb_alloc_urb(0, GFP_KERNEL);
priv->rx_urb[i]->transfer_buffer =
@@ -1782,8 +1770,8 @@
netdev_dbg(dev, "End of initendpoints\n");
return 0;
-
}
+
#ifdef THOMAS_BEACON
static void rtl8192_usb_deleteendpoints(struct net_device *dev)
{
@@ -1820,7 +1808,6 @@
}
kfree(priv->rx_urb);
priv->rx_urb = NULL;
-
}
#else
kfree(priv->rx_urb);
@@ -1888,6 +1875,7 @@
net->bssht.bdRT2RTLongSlotTime;
rtl8192_update_cap(dev, net->capability);
}
+
/*
* background support to run QoS activate functionality
*/
@@ -1992,7 +1980,6 @@
rtl8192_qos_handle_probe_response(priv, 1, network);
schedule_delayed_work(&priv->update_beacon_wq, 0);
return 0;
-
}
/*
@@ -2007,7 +1994,7 @@
u32 size = sizeof(struct ieee80211_qos_parameters);
int set_qos_param = 0;
- if ((priv == NULL) || (network == NULL))
+ if (!priv || !network)
return 0;
if (priv->ieee80211->state != IEEE80211_LINKED)
@@ -2182,6 +2169,7 @@
}
return ret;
}
+
static void rtl8192_SetWirelessMode(struct net_device *dev, u8 wireless_mode)
{
struct r8192_priv *priv = ieee80211_priv(dev);
@@ -2223,8 +2211,8 @@
priv->ieee80211->pHTInfo->bEnableHT = 0;
RT_TRACE(COMP_INIT, "Current Wireless Mode is %x\n", wireless_mode);
rtl8192_refresh_supportrate(priv);
-
}
+
/* init priv variables here. only non_zero value should be initialized here. */
static void rtl8192_init_priv_variable(struct net_device *dev)
{
@@ -2432,6 +2420,7 @@
*data = (tmp >> 8) | (tmp << 8);
return *data;
}
+
static void rtl8192_read_eeprom_info(struct net_device *dev)
{
u16 wEPROM_ID = 0;
@@ -2627,7 +2616,6 @@
default:
priv->CustomerID = RT_CID_DEFAULT;
break;
-
}
switch (priv->CustomerID) {
@@ -2642,7 +2630,6 @@
default:
priv->LedStrategy = SW_LED_MODE0;
break;
-
}
@@ -2676,7 +2663,6 @@
static short rtl8192_init(struct net_device *dev)
{
-
struct r8192_priv *priv = ieee80211_priv(dev);
memset(&(priv->stats), 0, sizeof(struct Stats));
@@ -2797,8 +2783,6 @@
/* Set Tx Antenna including Feedback control */
/* Set Auto Rate fallback control */
-
-
}
@@ -3027,7 +3011,6 @@
bMaskByte2);
for (i = 0; i < CCKTxBBGainTableLength; i++) {
-
if (TempCCk == priv->cck_txbbgain_table[i].ccktxbb_valuearray[0]) {
priv->cck_present_attentuation_20Mdefault = (u8)i;
break;
@@ -3037,7 +3020,6 @@
priv->cck_present_attentuation_difference = 0;
priv->cck_present_attentuation =
priv->cck_present_attentuation_20Mdefault;
-
}
}
write_nic_byte(dev, 0x87, 0x0);
@@ -3222,7 +3204,6 @@
} else {
return RESET_TYPE_NORESET;
}
-
}
static void rtl8192_cancel_deferred_work(struct r8192_priv *priv);
@@ -3250,7 +3231,6 @@
if ((priv->ieee80211->pairwise_key_type == KEY_TYPE_WEP40) ||
(priv->ieee80211->pairwise_key_type == KEY_TYPE_WEP104)) {
-
for (EntryId = 0; EntryId < 4; EntryId++) {
MacAddr = CAM_CONST_ADDR[EntryId];
setKey(dev, EntryId, EntryId,
@@ -3259,7 +3239,6 @@
}
} else if (priv->ieee80211->pairwise_key_type == KEY_TYPE_TKIP) {
-
if (priv->ieee80211->iw_mode == IW_MODE_ADHOC)
setKey(dev, 4, 0, priv->ieee80211->pairwise_key_type,
(u8 *)dev->dev_addr, 0, NULL);
@@ -3267,7 +3246,6 @@
setKey(dev, 4, 0, priv->ieee80211->pairwise_key_type,
MacAddr, 0, NULL);
} else if (priv->ieee80211->pairwise_key_type == KEY_TYPE_CCMP) {
-
if (priv->ieee80211->iw_mode == IW_MODE_ADHOC)
setKey(dev, 4, 0, priv->ieee80211->pairwise_key_type,
(u8 *)dev->dev_addr, 0, NULL);
@@ -3301,6 +3279,7 @@
CAM_CONST_ADDR[0], 0, NULL);
}
}
+
/* This function is used to fix Tx/Rx stop bug temporarily.
* This function will do "system reset" to NIC when Tx or Rx is stuck.
* The method checking Tx/Rx stuck of this function is supported by FW,
@@ -3468,7 +3447,6 @@
/* for AP roaming */
if (priv->ieee80211->state == IEEE80211_LINKED &&
priv->ieee80211->iw_mode == IW_MODE_INFRA) {
-
rtl819x_update_rxcounts(priv, &TotalRxBcnNum, &TotalRxDataNum);
if ((TotalRxBcnNum + TotalRxDataNum) == 0) {
#ifdef TODO
@@ -3485,7 +3463,6 @@
priv->ieee80211->link_change(dev);
queue_work(priv->ieee80211->wq,
&priv->ieee80211->associate_procedure_wq);
-
}
}
priv->ieee80211->LinkDetectInfo.NumRecvBcnInPeriod = 0;
@@ -3510,7 +3487,6 @@
priv->bForcedSilentReset = false;
priv->bResetInProgress = false;
RT_TRACE(COMP_TRACE, " <==RtUsbCheckForHangWorkItemCallback()\n");
-
}
static void watch_dog_timer_callback(unsigned long data)
@@ -3521,6 +3497,7 @@
mod_timer(&priv->watch_dog_timer,
jiffies + msecs_to_jiffies(IEEE80211_WATCH_DOG_TIME));
}
+
static int _rtl8192_up(struct net_device *dev)
{
struct r8192_priv *priv = ieee80211_priv(dev);
@@ -3560,7 +3537,6 @@
ret = rtl8192_up(dev);
up(&priv->wx_sem);
return ret;
-
}
@@ -3587,7 +3563,6 @@
up(&priv->wx_sem);
return ret;
-
}
int rtl8192_down(struct net_device *dev)
@@ -3649,7 +3624,6 @@
rtl8192_rtx_disable(dev);
reset_status = _rtl8192_up(dev);
-
}
static void rtl8192_restart(struct work_struct *work)
@@ -4111,7 +4085,6 @@
(((priv->undecorated_smoothed_pwdb) * (Rx_Smooth_Factor - 1)) +
(pprevious_stats->RxPWDBAll)) / (Rx_Smooth_Factor);
}
-
}
/* Check EVM */
@@ -4159,8 +4132,6 @@
}
}
}
-
-
}
/*-----------------------------------------------------------------------------
@@ -4201,6 +4172,7 @@
ret_val = 100;
return ret_val;
}
+
/* We want good-looking for signal strength/quality */
static long rtl819x_signal_scale_mapping(long currsig)
{
@@ -4542,7 +4514,6 @@
bpacket_match_bssid, bpacket_toself,
bPacketBeacon, bToSelfBA);
rtl8192_record_rxdesc_forlateruse(pstats, &previous_stats);
-
}
/**
@@ -4758,7 +4729,6 @@
RT_TRACE(COMP_RXDESC,
"driver_info->FirstAGGR = %d, driver_info->PartAggr = %d\n",
driver_info->FirstAGGR, driver_info->PartAggr);
-
}
skb_pull(skb, sizeof(rx_desc_819x_usb));
@@ -4822,7 +4792,6 @@
netdev_dbg(dev, "actual_length: %d\n", skb->len);
dev_kfree_skb_any(skb);
}
-
}
static void rtl819xusb_process_received_packet(
@@ -4898,7 +4867,6 @@
};
if ((skb->len >= (20 + sizeof(rx_desc_819x_usb))) && (skb->len < RX_URB_SIZE)) {
-
query_rx_cmdpkt_desc_status(skb, &stats);
/* prfd->queue_id = 1; */
@@ -4937,7 +4905,6 @@
info->out_pipe);
dev_kfree_skb(skb);
break;
-
}
}
}
@@ -4971,7 +4938,7 @@
RT_TRACE(COMP_INIT, "Oops: i'm coming\n");
dev = alloc_ieee80211(sizeof(struct r8192_priv));
- if (dev == NULL)
+ if (!dev)
return -ENOMEM;
usb_set_intfdata(intf, dev);
@@ -5034,7 +5001,6 @@
*/
static void rtl8192_cancel_deferred_work(struct r8192_priv *priv)
{
-
cancel_work_sync(&priv->reset_wq);
cancel_delayed_work(&priv->watch_dog_wq);
cancel_delayed_work(&priv->update_beacon_wq);
@@ -5191,13 +5157,12 @@
write_nic_dword(dev, RWCAM, TargetCommand);
} else {
/* Key Material */
- if (KeyContent != NULL) {
+ if (KeyContent) {
write_nic_dword(dev, WCAMI, (u32)(*(KeyContent + i - 2)));
write_nic_dword(dev, RWCAM, TargetCommand);
}
}
}
-
}
/***************************************************************************
diff --git a/drivers/staging/rtl8192u/r8192U_wx.c b/drivers/staging/rtl8192u/r8192U_wx.c
index f828e64..837704d 100644
--- a/drivers/staging/rtl8192u/r8192U_wx.c
+++ b/drivers/staging/rtl8192u/r8192U_wx.c
@@ -30,7 +30,6 @@
static const u32 rtl8180_rates[] = {1000000, 2000000, 5500000, 11000000,
6000000, 9000000, 12000000, 18000000, 24000000, 36000000, 48000000, 54000000};
-
#ifndef ENETDOWN
#define ENETDOWN 1
#endif
@@ -44,7 +43,6 @@
return ieee80211_wx_get_freq(priv->ieee80211, a, wrqu, b);
}
-
static int r8192_wx_get_mode(struct net_device *dev, struct iw_request_info *a,
union iwreq_data *wrqu, char *b)
{
@@ -53,8 +51,6 @@
return ieee80211_wx_get_mode(priv->ieee80211, a, wrqu, b);
}
-
-
static int r8192_wx_get_rate(struct net_device *dev,
struct iw_request_info *info,
union iwreq_data *wrqu, char *extra)
@@ -64,8 +60,6 @@
return ieee80211_wx_get_rate(priv->ieee80211, info, wrqu, extra);
}
-
-
static int r8192_wx_set_rate(struct net_device *dev,
struct iw_request_info *info,
union iwreq_data *wrqu, char *extra)
@@ -82,7 +76,6 @@
return ret;
}
-
static int r8192_wx_set_rts(struct net_device *dev,
struct iw_request_info *info,
union iwreq_data *wrqu, char *extra)
@@ -148,7 +141,6 @@
}
-
static int r8192_wx_set_rawtx(struct net_device *dev,
struct iw_request_info *info,
union iwreq_data *wrqu, char *extra)
@@ -301,7 +293,6 @@
/* range->min_r_time; */ /* Minimal retry lifetime */
/* range->max_r_time; */ /* Maximal retry lifetime */
-
for (i = 0, val = 0; i < 14; i++) {
/* Include only legal frequencies for some countries */
@@ -326,7 +317,6 @@
return 0;
}
-
static int r8192_wx_set_scan(struct net_device *dev, struct iw_request_info *a,
union iwreq_data *wrqu, char *b)
{
@@ -396,9 +386,6 @@
return ret;
}
-
-
-
static int r8192_wx_get_essid(struct net_device *dev,
struct iw_request_info *a,
union iwreq_data *wrqu, char *b)
@@ -415,7 +402,6 @@
return ret;
}
-
static int r8192_wx_set_freq(struct net_device *dev, struct iw_request_info *a,
union iwreq_data *wrqu, char *b)
{
@@ -439,7 +425,6 @@
return ieee80211_wx_get_name(priv->ieee80211, info, wrqu, extra);
}
-
static int r8192_wx_set_frag(struct net_device *dev,
struct iw_request_info *info,
union iwreq_data *wrqu, char *extra)
@@ -493,7 +478,6 @@
}
-
static int r8192_wx_get_wap(struct net_device *dev,
struct iw_request_info *info,
union iwreq_data *wrqu, char *extra)
@@ -503,7 +487,6 @@
return ieee80211_wx_get_wap(priv->ieee80211, info, wrqu, extra);
}
-
static int r8192_wx_get_enc(struct net_device *dev,
struct iw_request_info *info,
union iwreq_data *wrqu, char *key)
@@ -695,7 +678,6 @@
wrqu->retry.value = priv->retry_data;
}
-
return 0;
}
@@ -711,7 +693,6 @@
return 0;
}
-
static int r8192_wx_set_sens(struct net_device *dev,
struct iw_request_info *info,
union iwreq_data *wrqu, char *extra)
@@ -862,7 +843,6 @@
return -1;
}
-
static iw_handler r8192_wx_handlers[] = {
NULL, /* SIOCSIWCOMMIT */
r8192_wx_get_name, /* SIOCGIWNAME */
@@ -949,7 +929,6 @@
};
-
static iw_handler r8192_private_handler[] = {
r8192_wx_set_crcmon,
r8192_wx_set_scan_type,
@@ -985,7 +964,6 @@
return wstats;
}
-
struct iw_handler_def r8192_wx_handlers_def = {
.standard = r8192_wx_handlers,
.num_standard = ARRAY_SIZE(r8192_wx_handlers),
diff --git a/drivers/staging/rtl8712/basic_types.h b/drivers/staging/rtl8712/basic_types.h
index 7561bed..f5c0231 100644
--- a/drivers/staging/rtl8712/basic_types.h
+++ b/drivers/staging/rtl8712/basic_types.h
@@ -11,10 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
* Modifications for inclusion into the Linux staging tree are
* Copyright(c) 2010 Larry Finger. All rights reserved.
*
diff --git a/drivers/staging/rtl8712/drv_types.h b/drivers/staging/rtl8712/drv_types.h
index 29e47e1..ae79047 100644
--- a/drivers/staging/rtl8712/drv_types.h
+++ b/drivers/staging/rtl8712/drv_types.h
@@ -11,10 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
* Modifications for inclusion into the Linux staging tree are
* Copyright(c) 2010 Larry Finger. All rights reserved.
*
diff --git a/drivers/staging/rtl8712/ethernet.h b/drivers/staging/rtl8712/ethernet.h
index fad173f..039da36 100644
--- a/drivers/staging/rtl8712/ethernet.h
+++ b/drivers/staging/rtl8712/ethernet.h
@@ -11,10 +11,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
* Modifications for inclusion into the Linux staging tree are
* Copyright(c) 2010 Larry Finger. All rights reserved.
*
diff --git a/drivers/staging/rtl8712/hal_init.c b/drivers/staging/rtl8712/hal_init.c
index 8008efe..0dd458d 100644
--- a/drivers/staging/rtl8712/hal_init.c
+++ b/drivers/staging/rtl8712/hal_init.c
@@ -13,10 +13,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
* Modifications for inclusion into the Linux staging tree are
* Copyright(c) 2010 Larry Finger. All rights reserved.
*
@@ -201,8 +197,8 @@
0x0000ffff);
memcpy(ppayload, ptr, dump_imem_sz);
r8712_write_mem(padapter, RTL8712_DMA_VOQ,
- dump_imem_sz + TXDESC_SIZE,
- (u8 *)ptx_desc);
+ dump_imem_sz + TXDESC_SIZE,
+ (u8 *)ptx_desc);
ptr += dump_imem_sz;
imem_sz -= dump_imem_sz;
} while (imem_sz > 0);
@@ -230,7 +226,8 @@
0x0000ffff);
memcpy(ppayload, ptr, dump_emem_sz);
r8712_write_mem(padapter, RTL8712_DMA_VOQ,
- dump_emem_sz + TXDESC_SIZE, (u8 *)ptx_desc);
+ dump_emem_sz + TXDESC_SIZE,
+ (u8 *)ptx_desc);
ptr += dump_emem_sz;
emem_sz -= dump_emem_sz;
} while (emem_sz > 0);
@@ -282,7 +279,7 @@
ptx_desc->txdw0 |= cpu_to_le32(BIT(28));
memcpy(ppayload, &fwhdr.fwpriv, fwhdr.fw_priv_sz);
r8712_write_mem(padapter, RTL8712_DMA_VOQ,
- fwhdr.fw_priv_sz + TXDESC_SIZE, (u8 *)ptx_desc);
+ fwhdr.fw_priv_sz + TXDESC_SIZE, (u8 *)ptx_desc);
/* polling dmem code done */
i = 100;
@@ -297,7 +294,8 @@
tmp8 = r8712_read8(padapter, 0x1025000A);
if (tmp8 & BIT(4)) /* When boot from EEPROM,
- & FW need more time to read EEPROM */
+ * & FW need more time to read EEPROM
+ */
i = 60;
else /* boot from EFUSE */
i = 30;
@@ -332,7 +330,8 @@
r8712_read32(padapter, RCR));
val32 = r8712_read32(padapter, RCR);
r8712_write32(padapter, RCR, (val32 | BIT(26))); /* Enable RX TCP
- Checksum offload */
+ * Checksum offload
+ */
netdev_info(padapter->pnetdev, "2 RCR=0x%x\n",
r8712_read32(padapter, RCR));
val32 = r8712_read32(padapter, RCR);
@@ -346,7 +345,8 @@
r8712_write8(padapter, 0x102500BD, r8712_read8(padapter, 0x102500BD) |
BIT(7)); /* enable usb rx aggregation */
r8712_write8(padapter, 0x102500D9, 1); /* TH=1 => means that invalidate
- * usb rx aggregation */
+ * usb rx aggregation
+ */
r8712_write8(padapter, 0x1025FE5B, 0x04); /* 1.7ms/4 */
/* Fix the RX FIFO issue(USB error) */
r8712_write8(padapter, 0x1025fe5C, r8712_read8(padapter, 0x1025fe5C)
@@ -367,7 +367,8 @@
r8712_write8(padapter, SYS_FUNC_EN + 1, 0x70);
r8712_write8(padapter, PMC_FSM, 0x06); /* Enable Loader Data Keep */
r8712_write8(padapter, SYS_ISO_CTRL, 0xF9); /* Isolation signals from
- * CORE, PLL */
+ * CORE, PLL
+ */
r8712_write8(padapter, SYS_ISO_CTRL + 1, 0xe8); /* Enable EFUSE 1.2V */
r8712_write8(padapter, AFE_PLL_CTRL, 0x00); /* Disable AFE PLL. */
r8712_write8(padapter, LDOA15_CTRL, 0x54); /* Disable A15V */
diff --git a/drivers/staging/rtl8712/ieee80211.c b/drivers/staging/rtl8712/ieee80211.c
index d13b4d5..8918654 100644
--- a/drivers/staging/rtl8712/ieee80211.c
+++ b/drivers/staging/rtl8712/ieee80211.c
@@ -13,10 +13,6 @@
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
- * You should have received a copy of the GNU General Public License along with
- * this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110, USA
- *
* Modifications for inclusion into the Linux staging tree are
* Copyright(c) 2010 Larry Finger. All rights reserved.
*
diff --git a/drivers/staging/rtl8712/mlme_linux.c b/drivers/staging/rtl8712/mlme_linux.c
index e4e4bde..af7c4a4 100644
--- a/drivers/staging/rtl8712/mlme_linux.c
+++ b/drivers/staging/rtl8712/mlme_linux.c
@@ -153,7 +153,7 @@
buff = NULL;
if (authmode == _WPA_IE_ID_) {
buff = kzalloc(IW_CUSTOM_MAX, GFP_ATOMIC);
- if (buff == NULL)
+ if (!buff)
return;
p = buff;
p += sprintf(p, "ASSOCINFO(ReqIEs=");
diff --git a/drivers/staging/rtl8712/os_intfs.c b/drivers/staging/rtl8712/os_intfs.c
index ab19112..57211f7 100644
--- a/drivers/staging/rtl8712/os_intfs.c
+++ b/drivers/staging/rtl8712/os_intfs.c
@@ -389,7 +389,7 @@
padapter->bup = true;
if (rtl871x_hal_init(padapter) != _SUCCESS)
goto netdev_open_error;
- if (r8712_initmac == NULL)
+ if (!r8712_initmac)
/* Use the mac address stored in the Efuse */
memcpy(pnetdev->dev_addr,
padapter->eeprompriv.mac_addr, ETH_ALEN);
@@ -413,7 +413,7 @@
}
if (start_drv_threads(padapter) != _SUCCESS)
goto netdev_open_error;
- if (padapter->dvobjpriv.inirp_init == NULL)
+ if (!padapter->dvobjpriv.inirp_init)
goto netdev_open_error;
else
padapter->dvobjpriv.inirp_init(padapter);
diff --git a/drivers/staging/rtl8712/osdep_service.h b/drivers/staging/rtl8712/osdep_service.h
index 076d508..ad041c9 100644
--- a/drivers/staging/rtl8712/osdep_service.h
+++ b/drivers/staging/rtl8712/osdep_service.h
@@ -57,9 +57,6 @@
spin_lock_init(&((pqueue)->lock)); \
} while (0)
-#define LIST_CONTAINOR(ptr, type, member) \
- ((type *)((char *)(ptr)-(SIZE_T)(&((type *)0)->member)))
-
static inline u32 _down_sema(struct semaphore *sema)
{
if (down_interruptible(sema))
diff --git a/drivers/staging/rtl8712/rtl8712_cmd.c b/drivers/staging/rtl8712/rtl8712_cmd.c
index 50f4002..13c0183 100644
--- a/drivers/staging/rtl8712/rtl8712_cmd.c
+++ b/drivers/staging/rtl8712/rtl8712_cmd.c
@@ -135,7 +135,7 @@
/* invoke cmd->callback function */
pcmd_callback = cmd_callback[pcmd->cmdcode].callback;
- if (pcmd_callback == NULL)
+ if (!pcmd_callback)
r8712_free_cmd_obj(pcmd);
else
pcmd_callback(padapter, pcmd);
@@ -149,7 +149,7 @@
/* invoke cmd->callback function */
pcmd_callback = cmd_callback[pcmd->cmdcode].callback;
- if (pcmd_callback == NULL)
+ if (!pcmd_callback)
r8712_free_cmd_obj(pcmd);
else
pcmd_callback(padapter, pcmd);
@@ -165,7 +165,7 @@
if (pcmd->rsp && pcmd->rspsz > 0)
memcpy(pcmd->rsp, (u8 *)&val, pcmd->rspsz);
pcmd_callback = cmd_callback[pcmd->cmdcode].callback;
- if (pcmd_callback == NULL)
+ if (!pcmd_callback)
r8712_free_cmd_obj(pcmd);
else
pcmd_callback(padapter, pcmd);
@@ -178,7 +178,7 @@
struct cmd_obj *pcmd = (struct cmd_obj *)pbuf;
pcmd_callback = cmd_callback[pcmd->cmdcode].callback;
- if (pcmd_callback == NULL)
+ if (!pcmd_callback)
r8712_free_cmd_obj(pcmd);
else
pcmd_callback(padapter, pcmd);
@@ -194,7 +194,7 @@
if (pcmd->rsp && pcmd->rspsz > 0)
memcpy(pcmd->rsp, (u8 *)&val, pcmd->rspsz);
pcmd_callback = cmd_callback[pcmd->cmdcode].callback;
- if (pcmd_callback == NULL)
+ if (!pcmd_callback)
r8712_free_cmd_obj(pcmd);
else
pcmd_callback(padapter, pcmd);
@@ -207,7 +207,7 @@
struct cmd_obj *pcmd = (struct cmd_obj *)pbuf;
pcmd_callback = cmd_callback[pcmd->cmdcode].callback;
- if (pcmd_callback == NULL)
+ if (!pcmd_callback)
r8712_free_cmd_obj(pcmd);
else
pcmd_callback(padapter, pcmd);
@@ -227,7 +227,7 @@
{
struct cmd_obj *pcmd_r;
- if (pcmd == NULL)
+ if (!pcmd)
return pcmd;
pcmd_r = NULL;
@@ -416,7 +416,7 @@
/* free all cmd_obj resources */
do {
pcmd = r8712_dequeue_cmd(&(pcmdpriv->cmd_queue));
- if (pcmd == NULL)
+ if (!pcmd)
break;
r8712_free_cmd_obj(pcmd);
} while (1);
@@ -431,7 +431,7 @@
void (*event_callback)(struct _adapter *dev, u8 *pbuf);
struct evt_priv *pevt_priv = &(padapter->evtpriv);
- if (peventbuf == NULL)
+ if (!peventbuf)
goto _abort_event_;
evt_sz = (u16)(le32_to_cpu(*peventbuf) & 0xffff);
evt_seq = (u8)((le32_to_cpu(*peventbuf) >> 24) & 0x7f);
diff --git a/drivers/staging/rtl8712/rtl8712_recv.c b/drivers/staging/rtl8712/rtl8712_recv.c
index d187508..f25b34c 100644
--- a/drivers/staging/rtl8712/rtl8712_recv.c
+++ b/drivers/staging/rtl8712/rtl8712_recv.c
@@ -204,7 +204,7 @@
pfree_recv_queue = &adapter->recvpriv.free_recv_queue;
phead = &defrag_q->queue;
plist = phead->next;
- prframe = LIST_CONTAINOR(plist, union recv_frame, u);
+ prframe = container_of(plist, union recv_frame, u.list);
list_del_init(&prframe->u.list);
pfhdr = &prframe->u.hdr;
curfragnum = 0;
@@ -219,7 +219,7 @@
plist = &defrag_q->queue;
plist = plist->next;
while (!end_of_queue_search(phead, plist)) {
- pnextrframe = LIST_CONTAINOR(plist, union recv_frame, u);
+ pnextrframe = container_of(plist, union recv_frame, u.list);
pnfhdr = &pnextrframe->u.hdr;
/*check the fragment sequence (2nd ~n fragment frame) */
if (curfragnum != pnfhdr->attrib.frag_num) {
@@ -492,7 +492,7 @@
phead = &ppending_recvframe_queue->queue;
plist = phead->next;
while (!end_of_queue_search(phead, plist)) {
- pnextrframe = LIST_CONTAINOR(plist, union recv_frame, u);
+ pnextrframe = container_of(plist, union recv_frame, u.list);
pnextattrib = &pnextrframe->u.hdr.attrib;
if (SN_LESS(pnextattrib->seq_num, pattrib->seq_num))
plist = plist->next;
@@ -525,14 +525,14 @@
if (list_empty(phead))
return true;
- prframe = LIST_CONTAINOR(plist, union recv_frame, u);
+ prframe = container_of(plist, union recv_frame, u.list);
pattrib = &prframe->u.hdr.attrib;
preorder_ctrl->indicate_seq = pattrib->seq_num;
}
/* Prepare indication list and indication.
* Check if there is any packet need indicate. */
while (!list_empty(phead)) {
- prframe = LIST_CONTAINOR(plist, union recv_frame, u);
+ prframe = container_of(plist, union recv_frame, u.list);
pattrib = &prframe->u.hdr.attrib;
if (!SN_LESS(preorder_ctrl->indicate_seq, pattrib->seq_num)) {
plist = plist->next;
diff --git a/drivers/staging/rtl8712/rtl8712_xmit.c b/drivers/staging/rtl8712/rtl8712_xmit.c
index b21a60e..7e0b945 100644
--- a/drivers/staging/rtl8712/rtl8712_xmit.c
+++ b/drivers/staging/rtl8712/rtl8712_xmit.c
@@ -169,8 +169,8 @@
xmitframe_phead = &pframe_queue->queue;
xmitframe_plist = xmitframe_phead->next;
if (!end_of_queue_search(xmitframe_phead, xmitframe_plist)) {
- pxmitframe = LIST_CONTAINOR(xmitframe_plist,
- struct xmit_frame, list);
+ pxmitframe = container_of(xmitframe_plist,
+ struct xmit_frame, list);
list_del_init(&pxmitframe->list);
ptxservq->qcnt--;
phwxmit->txcmdcnt++;
@@ -209,8 +209,8 @@
sta_phead = &phwxmit->sta_queue->queue;
sta_plist = sta_phead->next;
while (!end_of_queue_search(sta_phead, sta_plist)) {
- ptxservq = LIST_CONTAINOR(sta_plist, struct tx_servq,
- tx_pending);
+ ptxservq = container_of(sta_plist, struct tx_servq,
+ tx_pending);
pframe_queue = &ptxservq->sta_pending;
pxmitframe = dequeue_one_xmitframe(pxmitpriv, phwxmit,
ptxservq, pframe_queue);
diff --git a/drivers/staging/rtl8712/rtl871x_cmd.c b/drivers/staging/rtl8712/rtl871x_cmd.c
index 86136cc..aed03cf 100644
--- a/drivers/staging/rtl8712/rtl871x_cmd.c
+++ b/drivers/staging/rtl8712/rtl871x_cmd.c
@@ -225,10 +225,10 @@
struct mlme_priv *pmlmepriv = &padapter->mlmepriv;
ph2c = kmalloc(sizeof(*ph2c), GFP_ATOMIC);
- if (ph2c == NULL)
+ if (!ph2c)
return _FAIL;
psurveyPara = kmalloc(sizeof(*psurveyPara), GFP_ATOMIC);
- if (psurveyPara == NULL) {
+ if (!psurveyPara) {
kfree(ph2c);
return _FAIL;
}
@@ -258,10 +258,10 @@
struct cmd_priv *pcmdpriv = &padapter->cmdpriv;
ph2c = kmalloc(sizeof(*ph2c), GFP_ATOMIC);
- if (ph2c == NULL)
+ if (!ph2c)
return _FAIL;
pbsetdataratepara = kmalloc(sizeof(*pbsetdataratepara), GFP_ATOMIC);
- if (pbsetdataratepara == NULL) {
+ if (!pbsetdataratepara) {
kfree(ph2c);
return _FAIL;
}
@@ -280,10 +280,10 @@
struct cmd_priv *pcmdpriv = &padapter->cmdpriv;
ph2c = kmalloc(sizeof(*ph2c), GFP_ATOMIC);
- if (ph2c == NULL)
+ if (!ph2c)
return _FAIL;
psetchplanpara = kmalloc(sizeof(*psetchplanpara), GFP_ATOMIC);
- if (psetchplanpara == NULL) {
+ if (!psetchplanpara) {
kfree(ph2c);
return _FAIL;
}
@@ -301,10 +301,10 @@
struct cmd_priv *pcmdpriv = &padapter->cmdpriv;
ph2c = kmalloc(sizeof(*ph2c), GFP_ATOMIC);
- if (ph2c == NULL)
+ if (!ph2c)
return _FAIL;
pssetbasicratepara = kmalloc(sizeof(*pssetbasicratepara), GFP_ATOMIC);
- if (pssetbasicratepara == NULL) {
+ if (!pssetbasicratepara) {
kfree(ph2c);
return _FAIL;
}
@@ -322,10 +322,10 @@
struct cmd_priv *pcmdpriv = &padapter->cmdpriv;
ph2c = kmalloc(sizeof(*ph2c), GFP_ATOMIC);
- if (ph2c == NULL)
+ if (!ph2c)
return _FAIL;
pwriteptmparm = kmalloc(sizeof(*pwriteptmparm), GFP_ATOMIC);
- if (pwriteptmparm == NULL) {
+ if (!pwriteptmparm) {
kfree(ph2c);
return _FAIL;
}
@@ -342,10 +342,10 @@
struct cmd_priv *pcmdpriv = &padapter->cmdpriv;
ph2c = kmalloc(sizeof(*ph2c), GFP_ATOMIC);
- if (ph2c == NULL)
+ if (!ph2c)
return _FAIL;
pwriteptmparm = kmalloc(sizeof(*pwriteptmparm), GFP_ATOMIC);
- if (pwriteptmparm == NULL) {
+ if (!pwriteptmparm) {
kfree(ph2c);
return _FAIL;
}
@@ -362,10 +362,10 @@
struct cmd_priv *pcmdpriv = &padapter->cmdpriv;
ph2c = kmalloc(sizeof(*ph2c), GFP_ATOMIC);
- if (ph2c == NULL)
+ if (!ph2c)
return _FAIL;
pwriterfparm = kmalloc(sizeof(*pwriterfparm), GFP_ATOMIC);
- if (pwriterfparm == NULL) {
+ if (!pwriterfparm) {
kfree(ph2c);
return _FAIL;
}
@@ -383,10 +383,10 @@
struct cmd_priv *pcmdpriv = &padapter->cmdpriv;
ph2c = kmalloc(sizeof(*ph2c), GFP_ATOMIC);
- if (ph2c == NULL)
+ if (!ph2c)
return _FAIL;
prdrfparm = kmalloc(sizeof(*prdrfparm), GFP_ATOMIC);
- if (prdrfparm == NULL) {
+ if (!prdrfparm) {
kfree(ph2c);
return _FAIL;
}
@@ -427,7 +427,7 @@
padapter->ledpriv.LedControlHandler(padapter, LED_CTL_START_TO_LINK);
pcmd = kmalloc(sizeof(*pcmd), GFP_ATOMIC);
- if (pcmd == NULL)
+ if (!pcmd)
return _FAIL;
INIT_LIST_HEAD(&pcmd->list);
pcmd->cmdcode = _CreateBss_CMD_;
@@ -457,7 +457,7 @@
padapter->ledpriv.LedControlHandler(padapter, LED_CTL_START_TO_LINK);
pcmd = kmalloc(sizeof(*pcmd), GFP_ATOMIC);
- if (pcmd == NULL)
+ if (!pcmd)
return _FAIL;
/* for hidden ap to set fw_state here */
@@ -587,10 +587,10 @@
struct cmd_priv *pcmdpriv = &padapter->cmdpriv;
pdisconnect_cmd = kmalloc(sizeof(*pdisconnect_cmd), GFP_ATOMIC);
- if (pdisconnect_cmd == NULL)
+ if (!pdisconnect_cmd)
return _FAIL;
pdisconnect = kmalloc(sizeof(*pdisconnect), GFP_ATOMIC);
- if (pdisconnect == NULL) {
+ if (!pdisconnect) {
kfree(pdisconnect_cmd);
return _FAIL;
}
@@ -609,10 +609,10 @@
struct cmd_priv *pcmdpriv = &padapter->cmdpriv;
ph2c = kmalloc(sizeof(*ph2c), GFP_ATOMIC);
- if (ph2c == NULL)
+ if (!ph2c)
return _FAIL;
psetop = kmalloc(sizeof(*psetop), GFP_ATOMIC);
- if (psetop == NULL) {
+ if (!psetop) {
kfree(ph2c);
return _FAIL;
}
@@ -633,15 +633,15 @@
struct sta_info *sta = (struct sta_info *)psta;
ph2c = kmalloc(sizeof(*ph2c), GFP_ATOMIC);
- if (ph2c == NULL)
+ if (!ph2c)
return _FAIL;
psetstakey_para = kmalloc(sizeof(*psetstakey_para), GFP_ATOMIC);
- if (psetstakey_para == NULL) {
+ if (!psetstakey_para) {
kfree(ph2c);
return _FAIL;
}
psetstakey_rsp = kmalloc(sizeof(*psetstakey_rsp), GFP_ATOMIC);
- if (psetstakey_rsp == NULL) {
+ if (!psetstakey_rsp) {
kfree(ph2c);
kfree(psetstakey_para);
return _FAIL;
@@ -673,10 +673,10 @@
struct cmd_priv *pcmdpriv = &padapter->cmdpriv;
ph2c = kmalloc(sizeof(*ph2c), GFP_ATOMIC);
- if (ph2c == NULL)
+ if (!ph2c)
return _FAIL;
psetrfintfsparm = kmalloc(sizeof(*psetrfintfsparm), GFP_ATOMIC);
- if (psetrfintfsparm == NULL) {
+ if (!psetrfintfsparm) {
kfree(ph2c);
return _FAIL;
}
@@ -695,10 +695,10 @@
struct cmd_priv *pcmdpriv = &padapter->cmdpriv;
ph2c = kmalloc(sizeof(*ph2c), GFP_ATOMIC);
- if (ph2c == NULL)
+ if (!ph2c)
return _FAIL;
psetrttblparm = kmalloc(sizeof(*psetrttblparm), GFP_ATOMIC);
- if (psetrttblparm == NULL) {
+ if (!psetrttblparm) {
kfree(ph2c);
return _FAIL;
}
@@ -716,10 +716,10 @@
struct SetMacAddr_param *psetMacAddr_para;
ph2c = kmalloc(sizeof(*ph2c), GFP_ATOMIC);
- if (ph2c == NULL)
+ if (!ph2c)
return _FAIL;
psetMacAddr_para = kmalloc(sizeof(*psetMacAddr_para), GFP_ATOMIC);
- if (psetMacAddr_para == NULL) {
+ if (!psetMacAddr_para) {
kfree(ph2c);
return _FAIL;
}
@@ -738,15 +738,15 @@
struct set_assocsta_rsp *psetassocsta_rsp = NULL;
ph2c = kmalloc(sizeof(*ph2c), GFP_ATOMIC);
- if (ph2c == NULL)
+ if (!ph2c)
return _FAIL;
psetassocsta_para = kmalloc(sizeof(*psetassocsta_para), GFP_ATOMIC);
- if (psetassocsta_para == NULL) {
+ if (!psetassocsta_para) {
kfree(ph2c);
return _FAIL;
}
psetassocsta_rsp = kmalloc(sizeof(*psetassocsta_rsp), GFP_ATOMIC);
- if (psetassocsta_rsp == NULL) {
+ if (!psetassocsta_rsp) {
kfree(ph2c);
kfree(psetassocsta_para);
return _FAIL;
@@ -766,10 +766,10 @@
struct addBaReq_parm *paddbareq_parm;
ph2c = kmalloc(sizeof(*ph2c), GFP_ATOMIC);
- if (ph2c == NULL)
+ if (!ph2c)
return _FAIL;
paddbareq_parm = kmalloc(sizeof(*paddbareq_parm), GFP_ATOMIC);
- if (paddbareq_parm == NULL) {
+ if (!paddbareq_parm) {
kfree(ph2c);
return _FAIL;
}
@@ -787,10 +787,10 @@
struct cmd_priv *pcmdpriv = &padapter->cmdpriv;
ph2c = kmalloc(sizeof(*ph2c), GFP_ATOMIC);
- if (ph2c == NULL)
+ if (!ph2c)
return _FAIL;
pdrvintcmd_param = kmalloc(sizeof(*pdrvintcmd_param), GFP_ATOMIC);
- if (pdrvintcmd_param == NULL) {
+ if (!pdrvintcmd_param) {
kfree(ph2c);
return _FAIL;
}
@@ -961,10 +961,10 @@
struct cmd_priv *pcmdpriv = &adapter->cmdpriv;
ph2c = kmalloc(sizeof(*ph2c), GFP_ATOMIC);
- if (ph2c == NULL)
+ if (!ph2c)
return _FAIL;
param = kzalloc(sizeof(*param), GFP_ATOMIC);
- if (param == NULL) {
+ if (!param) {
kfree(ph2c);
return _FAIL;
}
diff --git a/drivers/staging/rtl8712/rtl871x_ioctl_linux.c b/drivers/staging/rtl8712/rtl871x_ioctl_linux.c
index 1b9e249..e205adf 100644
--- a/drivers/staging/rtl8712/rtl871x_ioctl_linux.c
+++ b/drivers/staging/rtl8712/rtl871x_ioctl_linux.c
@@ -399,7 +399,7 @@
if (wep_key_len > 0) {
wep_key_len = wep_key_len <= 5 ? 5 : 13;
pwep = kzalloc(sizeof(*pwep), GFP_ATOMIC);
- if (pwep == NULL)
+ if (!pwep)
return -ENOMEM;
pwep->KeyLength = wep_key_len;
pwep->Length = wep_key_len +
@@ -1060,8 +1060,8 @@
while (1) {
if (end_of_queue_search(phead, pmlmepriv->pscanned))
break;
- pnetwork = LIST_CONTAINOR(pmlmepriv->pscanned,
- struct wlan_network, list);
+ pnetwork = container_of(pmlmepriv->pscanned,
+ struct wlan_network, list);
pmlmepriv->pscanned = pmlmepriv->pscanned->next;
dst_bssid = pnetwork->network.MacAddress;
if (!memcmp(dst_bssid, temp->sa_data, ETH_ALEN)) {
@@ -1216,7 +1216,7 @@
ret = -E2BIG;
break;
}
- pnetwork = LIST_CONTAINOR(plist, struct wlan_network, list);
+ pnetwork = container_of(plist, struct wlan_network, list);
ev = translate_scan(padapter, a, pnetwork, ev, stop);
plist = plist->next;
}
@@ -1271,8 +1271,8 @@
while (1) {
if (end_of_queue_search(phead, pmlmepriv->pscanned))
break;
- pnetwork = LIST_CONTAINOR(pmlmepriv->pscanned,
- struct wlan_network, list);
+ pnetwork = container_of(pmlmepriv->pscanned,
+ struct wlan_network, list);
pmlmepriv->pscanned = pmlmepriv->pscanned->next;
dst_ssid = pnetwork->network.Ssid.Ssid;
if ((!memcmp(dst_ssid, src_ssid, ndis_ssid.SsidLength))
@@ -1793,7 +1793,7 @@
param_len = sizeof(struct ieee_param) + pext->key_len;
param = kzalloc(param_len, GFP_ATOMIC);
- if (param == NULL)
+ if (!param)
return -ENOMEM;
param->cmd = IEEE_CMD_SET_ENCRYPTION;
eth_broadcast_addr(param->sta_addr);
@@ -1986,7 +1986,7 @@
while (1) {
if (end_of_queue_search(phead, plist))
break;
- pnetwork = LIST_CONTAINOR(plist, struct wlan_network, list);
+ pnetwork = container_of(plist, struct wlan_network, list);
if (!mac_pton(data, bssid)) {
netdev_info(dev, "r8712u: Invalid BSSID '%s'.\n",
(u8 *)data);
diff --git a/drivers/staging/rtl8712/rtl871x_ioctl_set.c b/drivers/staging/rtl8712/rtl871x_ioctl_set.c
index f772675..56760cd 100644
--- a/drivers/staging/rtl8712/rtl871x_ioctl_set.c
+++ b/drivers/staging/rtl8712/rtl871x_ioctl_set.c
@@ -34,12 +34,6 @@
#include "usb_osintf.h"
#include "usb_ops.h"
-#define IS_MAC_ADDRESS_BROADCAST(addr) \
-( \
- ((addr[0] == 0xff) && (addr[1] == 0xff) && \
- (addr[2] == 0xff) && (addr[3] == 0xff) && \
- (addr[4] == 0xff) && (addr[5] == 0xff)) ? true : false \
-)
static u8 validate_ssid(struct ndis_802_11_ssid *ssid)
{
diff --git a/drivers/staging/rtl8712/rtl871x_mlme.c b/drivers/staging/rtl8712/rtl871x_mlme.c
index 62d4ae8..772bf9f 100644
--- a/drivers/staging/rtl8712/rtl871x_mlme.c
+++ b/drivers/staging/rtl8712/rtl871x_mlme.c
@@ -155,7 +155,7 @@
phead = &scanned_queue->queue;
plist = phead->next;
while (plist != phead) {
- pnetwork = LIST_CONTAINOR(plist, struct wlan_network, list);
+ pnetwork = container_of(plist, struct wlan_network, list);
plist = plist->next;
if (!memcmp(addr, pnetwork->network.MacAddress, ETH_ALEN))
break;
@@ -176,7 +176,7 @@
phead = &scanned_queue->queue;
plist = phead->next;
while (!end_of_queue_search(phead, plist)) {
- pnetwork = LIST_CONTAINOR(plist, struct wlan_network, list);
+ pnetwork = container_of(plist, struct wlan_network, list);
plist = plist->next;
_free_network(pmlmepriv, pnetwork);
}
@@ -304,7 +304,7 @@
while (1) {
if (end_of_queue_search(phead, plist) == true)
break;
- pwlan = LIST_CONTAINOR(plist, struct wlan_network, list);
+ pwlan = container_of(plist, struct wlan_network, list);
if (pwlan->fixed != true) {
if (oldest == NULL ||
time_after((unsigned long)oldest->last_scanned,
@@ -390,7 +390,7 @@
if (end_of_queue_search(phead, plist))
break;
- pnetwork = LIST_CONTAINOR(plist, struct wlan_network, list);
+ pnetwork = container_of(plist, struct wlan_network, list);
if (is_same_network(&pnetwork->network, target))
break;
if ((oldest == ((struct wlan_network *)0)) ||
@@ -1135,8 +1135,8 @@
}
return _FAIL;
}
- pnetwork = LIST_CONTAINOR(pmlmepriv->pscanned,
- struct wlan_network, list);
+ pnetwork = container_of(pmlmepriv->pscanned,
+ struct wlan_network, list);
if (pnetwork == NULL)
return _FAIL;
pmlmepriv->pscanned = pmlmepriv->pscanned->next;
@@ -1205,7 +1205,7 @@
return _FAIL;
psetauthparm = kzalloc(sizeof(*psetauthparm), GFP_ATOMIC);
- if (psetauthparm == NULL) {
+ if (!psetauthparm) {
kfree(pcmd);
return _FAIL;
}
@@ -1234,7 +1234,7 @@
if (!pcmd)
return _FAIL;
psetkeyparm = kzalloc(sizeof(*psetkeyparm), GFP_ATOMIC);
- if (psetkeyparm == NULL) {
+ if (!psetkeyparm) {
ret = _FAIL;
goto err_free_cmd;
}
diff --git a/drivers/staging/rtl8712/rtl871x_recv.c b/drivers/staging/rtl8712/rtl871x_recv.c
index 616ca39..23c1438 100644
--- a/drivers/staging/rtl8712/rtl871x_recv.c
+++ b/drivers/staging/rtl8712/rtl871x_recv.c
@@ -142,7 +142,7 @@
phead = &pframequeue->queue;
plist = phead->next;
while (!end_of_queue_search(phead, plist)) {
- precvframe = LIST_CONTAINOR(plist, union recv_frame, u);
+ precvframe = container_of(plist, union recv_frame, u.list);
plist = plist->next;
r8712_free_recvframe(precvframe, pfree_recv_queue);
}
diff --git a/drivers/staging/rtl8712/rtl871x_sta_mgt.c b/drivers/staging/rtl8712/rtl871x_sta_mgt.c
index e90c00d..e11ce28 100644
--- a/drivers/staging/rtl8712/rtl871x_sta_mgt.c
+++ b/drivers/staging/rtl8712/rtl871x_sta_mgt.c
@@ -216,8 +216,8 @@
phead = &(pstapriv->sta_hash[index]);
plist = phead->next;
while (!end_of_queue_search(phead, plist)) {
- psta = LIST_CONTAINOR(plist,
- struct sta_info, hash_list);
+ psta = container_of(plist,
+ struct sta_info, hash_list);
plist = plist->next;
if (pbcmc_stainfo != psta)
r8712_free_stainfo(padapter, psta);
@@ -241,7 +241,7 @@
phead = &(pstapriv->sta_hash[index]);
plist = phead->next;
while (!end_of_queue_search(phead, plist)) {
- psta = LIST_CONTAINOR(plist, struct sta_info, hash_list);
+ psta = container_of(plist, struct sta_info, hash_list);
if ((!memcmp(psta->hwaddr, hwaddr, ETH_ALEN))) {
/* if found the matched address */
break;
diff --git a/drivers/staging/rtl8712/rtl871x_xmit.c b/drivers/staging/rtl8712/rtl871x_xmit.c
index c6d952f..99256ba 100644
--- a/drivers/staging/rtl8712/rtl871x_xmit.c
+++ b/drivers/staging/rtl8712/rtl871x_xmit.c
@@ -848,7 +848,7 @@
phead = &pframequeue->queue;
plist = phead->next;
while (!end_of_queue_search(phead, plist)) {
- pxmitframe = LIST_CONTAINOR(plist, struct xmit_frame, list);
+ pxmitframe = container_of(plist, struct xmit_frame, list);
plist = plist->next;
r8712_free_xmitframe(pxmitpriv, pxmitframe);
}
diff --git a/drivers/staging/rtl8712/usb_ops_linux.c b/drivers/staging/rtl8712/usb_ops_linux.c
index 454cdf6..6f12345 100644
--- a/drivers/staging/rtl8712/usb_ops_linux.c
+++ b/drivers/staging/rtl8712/usb_ops_linux.c
@@ -504,7 +504,7 @@
u8 *palloc_buf, *pIo_buf;
palloc_buf = kmalloc((u32)len + 16, GFP_ATOMIC);
- if (palloc_buf == NULL)
+ if (!palloc_buf)
return -ENOMEM;
pIo_buf = palloc_buf + 16 - ((addr_t)(palloc_buf) & 0x0f);
if (requesttype == 0x01) {
diff --git a/drivers/staging/rtl8723au/Kconfig b/drivers/staging/rtl8723au/Kconfig
index 435f359..277c1ab 100644
--- a/drivers/staging/rtl8723au/Kconfig
+++ b/drivers/staging/rtl8723au/Kconfig
@@ -1,5 +1,5 @@
config R8723AU
- tristate "Realtek RTL8723AU Wireless LAN NIC driver"
+ tristate "Realtek RTL8723AU Wireless LAN NIC driver (deprecated)"
depends on USB && WLAN && RFKILL
select WIRELESS_EXT
select WEXT_PRIV
@@ -7,7 +7,10 @@
default n
---help---
This option adds the Realtek RTL8723AU USB device such as found in
- the Lenovo Yogi 13 tablet. If built as a module, it will be called r8723au.
+ the Lenovo Yoga 13 tablet. If built as a module, it will be called r8723au.
+
+ Note: This driver is deprecated and scheduled to be removed in a
+ future kernel release. Please use rtl8xxxu instead.
if R8723AU
diff --git a/drivers/staging/rtl8723au/core/rtw_ap.c b/drivers/staging/rtl8723au/core/rtw_ap.c
index f68e277..aad686d 100644
--- a/drivers/staging/rtl8723au/core/rtw_ap.c
+++ b/drivers/staging/rtl8723au/core/rtw_ap.c
@@ -1719,7 +1719,8 @@
}
spin_unlock_bh(&pacl_node_q->lock);
- DBG_8723A("%s, free acl_node_queue, num =%d\n", __func__, pacl_list->num);
+ DBG_8723A("%s, free acl_node_queue, num =%d\n",
+ __func__, pacl_list->num);
rtw_sta_flush23a(padapter);
diff --git a/drivers/staging/rtl8723au/core/rtw_recv.c b/drivers/staging/rtl8723au/core/rtw_recv.c
index 989ed07..150dabc 100644
--- a/drivers/staging/rtl8723au/core/rtw_recv.c
+++ b/drivers/staging/rtl8723au/core/rtw_recv.c
@@ -211,31 +211,6 @@
return cnt;
}
-int rtw_enqueue_recvbuf23a_to_head(struct recv_buf *precvbuf, struct rtw_queue *queue)
-{
- spin_lock_bh(&queue->lock);
-
- list_del_init(&precvbuf->list);
- list_add(&precvbuf->list, get_list_head(queue));
-
- spin_unlock_bh(&queue->lock);
-
- return _SUCCESS;
-}
-
-int rtw_enqueue_recvbuf23a(struct recv_buf *precvbuf, struct rtw_queue *queue)
-{
- unsigned long irqL;
-
- spin_lock_irqsave(&queue->lock, irqL);
-
- list_del_init(&precvbuf->list);
-
- list_add_tail(&precvbuf->list, get_list_head(queue));
- spin_unlock_irqrestore(&queue->lock, irqL);
- return _SUCCESS;
-}
-
struct recv_buf *rtw_dequeue_recvbuf23a (struct rtw_queue *queue)
{
unsigned long irqL;
diff --git a/drivers/staging/rtl8723au/core/rtw_wlan_util.c b/drivers/staging/rtl8723au/core/rtw_wlan_util.c
index cc2b84b..694cf17 100644
--- a/drivers/staging/rtl8723au/core/rtw_wlan_util.c
+++ b/drivers/staging/rtl8723au/core/rtw_wlan_util.c
@@ -304,21 +304,11 @@
adapter_to_dvobj(adapter)->oper_channel = ch;
}
-inline u8 rtw_get_oper_bw23a(struct rtw_adapter *adapter)
-{
- return adapter_to_dvobj(adapter)->oper_bwmode;
-}
-
inline void rtw_set_oper_bw23a(struct rtw_adapter *adapter, u8 bw)
{
adapter_to_dvobj(adapter)->oper_bwmode = bw;
}
-inline u8 rtw_get_oper_ch23aoffset(struct rtw_adapter *adapter)
-{
- return adapter_to_dvobj(adapter)->oper_ch_offset;
-}
-
inline void rtw_set_oper_ch23aoffset23a(struct rtw_adapter *adapter, u8 offset)
{
adapter_to_dvobj(adapter)->oper_ch_offset = offset;
diff --git a/drivers/staging/rtl8723au/hal/rtl8723a_hal_init.c b/drivers/staging/rtl8723au/hal/rtl8723a_hal_init.c
index e81301f..1ea0af4 100644
--- a/drivers/staging/rtl8723au/hal/rtl8723a_hal_init.c
+++ b/drivers/staging/rtl8723au/hal/rtl8723a_hal_init.c
@@ -1175,8 +1175,6 @@
/* Let last entry point to the start entry of ring buffer */
status = _LLTWrite(padapter, Last_Entry_Of_TxPktBuf, txpktbuf_bndy);
- if (status != _SUCCESS)
- return status;
return status;
}
diff --git a/drivers/staging/rtl8723au/hal/rtl8723a_rf6052.c b/drivers/staging/rtl8723au/hal/rtl8723a_rf6052.c
index ce0d8d8..24c0ff3 100644
--- a/drivers/staging/rtl8723au/hal/rtl8723a_rf6052.c
+++ b/drivers/staging/rtl8723au/hal/rtl8723a_rf6052.c
@@ -465,7 +465,7 @@
break;
}
- /*----Restore RFENV control type----*/;
+ /*----Restore RFENV control type----*/
switch (eRFPath) {
case RF_PATH_A:
PHY_SetBBReg(Adapter, pPhyReg->rfintfs,
diff --git a/drivers/staging/rtl8723au/include/rtw_mlme_ext.h b/drivers/staging/rtl8723au/include/rtw_mlme_ext.h
index ea2a6c91..0e7d3da9 100644
--- a/drivers/staging/rtl8723au/include/rtw_mlme_ext.h
+++ b/drivers/staging/rtl8723au/include/rtw_mlme_ext.h
@@ -461,9 +461,7 @@
u8 rtw_get_oper_ch23a(struct rtw_adapter *adapter);
void rtw_set_oper_ch23a(struct rtw_adapter *adapter, u8 ch);
-u8 rtw_get_oper_bw23a(struct rtw_adapter *adapter);
void rtw_set_oper_bw23a(struct rtw_adapter *adapter, u8 bw);
-u8 rtw_get_oper_ch23aoffset(struct rtw_adapter *adapter);
void rtw_set_oper_ch23aoffset23a(struct rtw_adapter *adapter, u8 offset);
void set_channel_bwmode23a(struct rtw_adapter *padapter, unsigned char channel,
diff --git a/drivers/staging/rtl8723au/include/rtw_recv.h b/drivers/staging/rtl8723au/include/rtw_recv.h
index dc784be..85a5edb 100644
--- a/drivers/staging/rtl8723au/include/rtw_recv.h
+++ b/drivers/staging/rtl8723au/include/rtw_recv.h
@@ -279,8 +279,6 @@
u32 rtw_free_uc_swdec_pending_queue23a(struct rtw_adapter *adapter);
-int rtw_enqueue_recvbuf23a_to_head(struct recv_buf *precvbuf, struct rtw_queue *queue);
-int rtw_enqueue_recvbuf23a(struct recv_buf *precvbuf, struct rtw_queue *queue);
struct recv_buf *rtw_dequeue_recvbuf23a(struct rtw_queue *queue);
void rtw_reordering_ctrl_timeout_handler23a(unsigned long pcontext);
diff --git a/drivers/staging/rtl8723au/os_dep/usb_intf.c b/drivers/staging/rtl8723au/os_dep/usb_intf.c
index 27b3a5b..cf83eff 100644
--- a/drivers/staging/rtl8723au/os_dep/usb_intf.c
+++ b/drivers/staging/rtl8723au/os_dep/usb_intf.c
@@ -532,6 +532,7 @@
{
struct rtw_adapter *if1 = NULL;
struct dvobj_priv *dvobj;
+ struct usb_device *udev;
int status = _FAIL;
RT_TRACE(_module_hci_intfs_c_, _drv_err_, "+rtw_drv_init\n");
@@ -544,6 +545,10 @@
goto exit;
}
+ udev = dvobj->pusbdev;
+ dev_warn(&udev->dev, "WARNING: The rtl8723au driver is deprecated!");
+ dev_warn(&udev->dev, "Please use the rtl8xxxu driver for this device!");
+
if1 = rtw_usb_if1_init(dvobj, pusb_intf, pdid);
if (!if1) {
DBG_8723A("rtw_init_primary_adapter Failed!\n");
diff --git a/drivers/staging/rts5208/ms.c b/drivers/staging/rts5208/ms.c
index a780185..0f0cd4a 100644
--- a/drivers/staging/rts5208/ms.c
+++ b/drivers/staging/rts5208/ms.c
@@ -49,7 +49,7 @@
}
static int ms_transfer_tpc(struct rtsx_chip *chip, u8 trans_mode,
- u8 tpc, u8 cnt, u8 cfg)
+ u8 tpc, u8 cnt, u8 cfg)
{
struct ms_info *ms_card = &chip->ms_card;
int retval;
@@ -2691,7 +2691,7 @@
}
if ((log_blk < ms_start_idx[seg_no]) ||
- (log_blk >= ms_start_idx[seg_no+1])) {
+ (log_blk >= ms_start_idx[seg_no + 1])) {
if (!(chip->card_wp & MS_CARD)) {
retval = ms_erase_block(chip, phy_blk);
if (retval != STATUS_SUCCESS)
@@ -3836,7 +3836,7 @@
start_page = (u8)(start_sector & ms_card->page_off);
for (seg_no = 0; seg_no < ARRAY_SIZE(ms_start_idx) - 1; seg_no++) {
- if (log_blk < ms_start_idx[seg_no+1])
+ if (log_blk < ms_start_idx[seg_no + 1])
break;
}
@@ -4264,7 +4264,7 @@
memset(buf1, 0, 32);
rtsx_stor_get_xfer_buf(buf2, min_t(int, 12, scsi_bufflen(srb)), srb);
for (i = 0; i < 8; i++)
- buf1[8+i] = buf2[4+i];
+ buf1[8 + i] = buf2[4 + i];
retval = ms_write_bytes(chip, PRO_WRITE_SHORT_DATA, 32, WAIT_INT,
buf1, 32);
@@ -4399,10 +4399,10 @@
rtsx_stor_get_xfer_buf(buf, bufflen, srb);
for (i = 0; i < 8; i++)
- buf[i] = buf[4+i];
+ buf[i] = buf[4 + i];
for (i = 0; i < 24; i++)
- buf[8+i] = 0;
+ buf[8 + i] = 0;
retval = ms_write_bytes(chip, PRO_WRITE_SHORT_DATA,
32, WAIT_INT, buf, 32);
@@ -4511,10 +4511,10 @@
rtsx_stor_get_xfer_buf(buf, bufflen, srb);
for (i = 0; i < 8; i++)
- buf[i] = buf[4+i];
+ buf[i] = buf[4 + i];
for (i = 0; i < 24; i++)
- buf[8+i] = 0;
+ buf[8 + i] = 0;
retval = ms_write_bytes(chip, PRO_WRITE_SHORT_DATA, 32, WAIT_INT,
buf, 32);
diff --git a/drivers/staging/rts5208/rtsx_card.c b/drivers/staging/rts5208/rtsx_card.c
index 437436f..231833a 100644
--- a/drivers/staging/rts5208/rtsx_card.c
+++ b/drivers/staging/rts5208/rtsx_card.c
@@ -628,11 +628,6 @@
}
}
-static inline u8 double_depth(u8 depth)
-{
- return (depth > 1) ? (depth - 1) : depth;
-}
-
int switch_ssc_clock(struct rtsx_chip *chip, int clk)
{
int retval;
@@ -1184,22 +1179,6 @@
return 0;
}
-int check_card_fail(struct rtsx_chip *chip, unsigned int lun)
-{
- if (chip->card_fail & chip->lun2card[lun])
- return 1;
-
- return 0;
-}
-
-int check_card_ejected(struct rtsx_chip *chip, unsigned int lun)
-{
- if (chip->card_ejected & chip->lun2card[lun])
- return 1;
-
- return 0;
-}
-
u8 get_lun_card(struct rtsx_chip *chip, unsigned int lun)
{
if ((chip->card_ready & chip->lun2card[lun]) == XD_CARD)
diff --git a/drivers/staging/rts5208/rtsx_card.h b/drivers/staging/rts5208/rtsx_card.h
index 8f2cf9a4e..56df9a4 100644
--- a/drivers/staging/rts5208/rtsx_card.h
+++ b/drivers/staging/rts5208/rtsx_card.h
@@ -1024,8 +1024,6 @@
int check_card_exist(struct rtsx_chip *chip, unsigned int lun);
int check_card_ready(struct rtsx_chip *chip, unsigned int lun);
int check_card_wp(struct rtsx_chip *chip, unsigned int lun);
-int check_card_fail(struct rtsx_chip *chip, unsigned int lun);
-int check_card_ejected(struct rtsx_chip *chip, unsigned int lun);
void eject_card(struct rtsx_chip *chip, unsigned int lun);
u8 get_lun_card(struct rtsx_chip *chip, unsigned int lun);
diff --git a/drivers/staging/rts5208/rtsx_chip.c b/drivers/staging/rts5208/rtsx_chip.c
index c0ce659..bcc4b66 100644
--- a/drivers/staging/rts5208/rtsx_chip.c
+++ b/drivers/staging/rts5208/rtsx_chip.c
@@ -43,14 +43,6 @@
rtsx_write_phy_register(chip, 0x00, 0x0288);
}
-void rtsx_disable_card_int(struct rtsx_chip *chip)
-{
- u32 reg = rtsx_readl(chip, RTSX_BIER);
-
- reg &= ~(XD_INT_EN | SD_INT_EN | MS_INT_EN);
- rtsx_writel(chip, RTSX_BIER, reg);
-}
-
void rtsx_enable_card_int(struct rtsx_chip *chip)
{
u32 reg = rtsx_readl(chip, RTSX_BIER);
@@ -1447,12 +1439,6 @@
rtsx_delink_stage(chip);
}
-void rtsx_undo_delink(struct rtsx_chip *chip)
-{
- chip->auto_delink_allowed = 0;
- rtsx_write_register(chip, CHANGE_LINK_STATE, 0x0A, 0x00);
-}
-
/**
* rtsx_stop_cmd - stop command transfer and DMA transfer
* @chip: Realtek's card reader chip
@@ -2000,27 +1986,6 @@
return STATUS_SUCCESS;
}
-int rtsx_check_link_ready(struct rtsx_chip *chip)
-{
- int retval;
- u8 val;
-
- retval = rtsx_read_register(chip, IRQSTAT0, &val);
- if (retval) {
- rtsx_trace(chip);
- return retval;
- }
-
- dev_dbg(rtsx_dev(chip), "IRQSTAT0: 0x%x\n", val);
- if (val & LINK_RDY_INT) {
- dev_dbg(rtsx_dev(chip), "Delinked!\n");
- rtsx_write_register(chip, IRQSTAT0, LINK_RDY_INT, LINK_RDY_INT);
- return STATUS_FAIL;
- }
-
- return STATUS_SUCCESS;
-}
-
static void rtsx_handle_pm_dstate(struct rtsx_chip *chip, u8 dstate)
{
u32 ultmp;
diff --git a/drivers/staging/rts5208/rtsx_chip.h b/drivers/staging/rts5208/rtsx_chip.h
index c295b1e..c08164f 100644
--- a/drivers/staging/rts5208/rtsx_chip.h
+++ b/drivers/staging/rts5208/rtsx_chip.h
@@ -950,7 +950,6 @@
int rtsx_force_power_on(struct rtsx_chip *chip, u8 ctl);
int rtsx_force_power_down(struct rtsx_chip *chip, u8 ctl);
-void rtsx_disable_card_int(struct rtsx_chip *chip);
void rtsx_enable_card_int(struct rtsx_chip *chip);
void rtsx_enable_bus_int(struct rtsx_chip *chip);
void rtsx_disable_bus_int(struct rtsx_chip *chip);
@@ -958,7 +957,6 @@
int rtsx_init_chip(struct rtsx_chip *chip);
void rtsx_release_chip(struct rtsx_chip *chip);
void rtsx_polling_func(struct rtsx_chip *chip);
-void rtsx_undo_delink(struct rtsx_chip *chip);
void rtsx_stop_cmd(struct rtsx_chip *chip, int card);
int rtsx_write_register(struct rtsx_chip *chip, u16 addr, u8 mask, u8 data);
int rtsx_read_register(struct rtsx_chip *chip, u16 addr, u8 *data);
@@ -975,7 +973,6 @@
int rtsx_write_efuse(struct rtsx_chip *chip, u8 addr, u8 val);
int rtsx_clr_phy_reg_bit(struct rtsx_chip *chip, u8 reg, u8 bit);
int rtsx_set_phy_reg_bit(struct rtsx_chip *chip, u8 reg, u8 bit);
-int rtsx_check_link_ready(struct rtsx_chip *chip);
void rtsx_enter_ss(struct rtsx_chip *chip);
void rtsx_exit_ss(struct rtsx_chip *chip);
int rtsx_pre_handle_interrupt(struct rtsx_chip *chip);
diff --git a/drivers/staging/rts5208/sd.c b/drivers/staging/rts5208/sd.c
index 87d6976..6219e04 100644
--- a/drivers/staging/rts5208/sd.c
+++ b/drivers/staging/rts5208/sd.c
@@ -1928,9 +1928,9 @@
tuning_cmd = sd_sdr_tuning_rx_cmd;
} else {
- if (CHK_MMC_DDR52(sd_card))
+ if (CHK_MMC_DDR52(sd_card)) {
tuning_cmd = mmc_ddr_tunning_rx_cmd;
- else {
+ } else {
rtsx_trace(chip);
return STATUS_FAIL;
}
@@ -2054,9 +2054,9 @@
tuning_cmd = sd_sdr_tuning_tx_cmd;
} else {
- if (CHK_MMC_DDR52(sd_card))
+ if (CHK_MMC_DDR52(sd_card)) {
tuning_cmd = sd_ddr_tuning_tx_cmd;
- else {
+ } else {
rtsx_trace(chip);
return STATUS_FAIL;
}
@@ -2678,9 +2678,9 @@
}
j++;
- if (j < 3)
+ if (j < 3) {
goto RTY_SD_RST;
- else {
+ } else {
rtsx_trace(chip);
return STATUS_FAIL;
}
@@ -2690,9 +2690,9 @@
SD_RSP_TYPE_R3, rsp, 5);
if (retval != STATUS_SUCCESS) {
k++;
- if (k < 3)
+ if (k < 3) {
goto RTY_SD_RST;
- else {
+ } else {
rtsx_trace(chip);
return STATUS_FAIL;
}
diff --git a/drivers/staging/skein/skein_api.c b/drivers/staging/skein/skein_api.c
index 36f849f..cab26e7 100644
--- a/drivers/staging/skein/skein_api.c
+++ b/drivers/staging/skein/skein_api.c
@@ -165,7 +165,6 @@
break;
}
return ret;
-
}
int skein_update_bits(struct skein_ctx *ctx, const u8 *msg,
@@ -210,7 +209,7 @@
/* internal sanity check: there IS a partial byte in the buffer! */
skein_assert(length != 0);
/* partial byte bit mask */
- mask = (u8) (1u << (7 - (msg_bit_cnt & 7)));
+ mask = (u8)(1u << (7 - (msg_bit_cnt & 7)));
/* apply bit padding on final byte (in the buffer) */
up[length - 1] = (u8)((up[length - 1] & (0 - mask)) | mask);
diff --git a/drivers/staging/skein/skein_base.c b/drivers/staging/skein/skein_base.c
index 25a01ca..c24a573 100644
--- a/drivers/staging/skein/skein_base.c
+++ b/drivers/staging/skein/skein_base.c
@@ -58,7 +58,7 @@
cfg.w[1] = skein_swap64(hash_bit_len);
cfg.w[2] = skein_swap64(SKEIN_CFG_TREE_INFO_SEQUENTIAL);
/* zero pad config block */
- memset(&cfg.w[3], 0, sizeof(cfg) - 3*sizeof(cfg.w[0]));
+ memset(&cfg.w[3], 0, sizeof(cfg) - 3 * sizeof(cfg.w[0]));
/* compute the initial chaining values from config block */
/* zero the chaining variables */
@@ -98,7 +98,7 @@
skein_assert(sizeof(cfg.b) >= sizeof(ctx->x));
/* do a mini-Init right here */
/* set output hash bit count = state size */
- ctx->h.hash_bit_len = 8*sizeof(ctx->x);
+ ctx->h.hash_bit_len = 8 * sizeof(ctx->x);
/* set tweaks: T0 = 0; T1 = KEY type */
skein_start_new_type(ctx, KEY);
/* zero the initial chaining variables */
@@ -171,7 +171,7 @@
*/
if (msg_byte_cnt > SKEIN_256_BLOCK_BYTES) {
/* number of full blocks to process */
- n = (msg_byte_cnt-1) / SKEIN_256_BLOCK_BYTES;
+ n = (msg_byte_cnt - 1) / SKEIN_256_BLOCK_BYTES;
skein_256_process_block(ctx, msg, n,
SKEIN_256_BLOCK_BYTES);
msg_byte_cnt -= n * SKEIN_256_BLOCK_BYTES;
@@ -205,7 +205,7 @@
/* zero pad b[] if necessary */
if (ctx->h.b_cnt < SKEIN_256_BLOCK_BYTES)
memset(&ctx->b[ctx->h.b_cnt], 0,
- SKEIN_256_BLOCK_BYTES - ctx->h.b_cnt);
+ SKEIN_256_BLOCK_BYTES - ctx->h.b_cnt);
/* process the final block */
skein_256_process_block(ctx, ctx->b, 1, ctx->h.b_cnt);
@@ -219,19 +219,19 @@
memset(ctx->b, 0, sizeof(ctx->b));
/* keep a local copy of counter mode "key" */
memcpy(x, ctx->x, sizeof(x));
- for (i = 0; i*SKEIN_256_BLOCK_BYTES < byte_cnt; i++) {
+ for (i = 0; i * SKEIN_256_BLOCK_BYTES < byte_cnt; i++) {
/* build the counter block */
- ((u64 *)ctx->b)[0] = skein_swap64((u64) i);
+ ((u64 *)ctx->b)[0] = skein_swap64((u64)i);
skein_start_new_type(ctx, OUT_FINAL);
/* run "counter mode" */
skein_256_process_block(ctx, ctx->b, 1, sizeof(u64));
/* number of output bytes left to go */
- n = byte_cnt - i*SKEIN_256_BLOCK_BYTES;
+ n = byte_cnt - i * SKEIN_256_BLOCK_BYTES;
if (n >= SKEIN_256_BLOCK_BYTES)
n = SKEIN_256_BLOCK_BYTES;
/* "output" the ctr mode bytes */
- skein_put64_lsb_first(hash_val+i*SKEIN_256_BLOCK_BYTES, ctx->x,
- n);
+ skein_put64_lsb_first(hash_val + (i * SKEIN_256_BLOCK_BYTES),
+ ctx->x, n);
/* restore the counter mode key for next time */
memcpy(ctx->x, x, sizeof(x));
}
@@ -282,7 +282,7 @@
cfg.w[1] = skein_swap64(hash_bit_len);
cfg.w[2] = skein_swap64(SKEIN_CFG_TREE_INFO_SEQUENTIAL);
/* zero pad config block */
- memset(&cfg.w[3], 0, sizeof(cfg) - 3*sizeof(cfg.w[0]));
+ memset(&cfg.w[3], 0, sizeof(cfg) - 3 * sizeof(cfg.w[0]));
/* compute the initial chaining values from config block */
/* zero the chaining variables */
@@ -326,7 +326,7 @@
skein_assert(sizeof(cfg.b) >= sizeof(ctx->x));
/* do a mini-Init right here */
/* set output hash bit count = state size */
- ctx->h.hash_bit_len = 8*sizeof(ctx->x);
+ ctx->h.hash_bit_len = 8 * sizeof(ctx->x);
/* set tweaks: T0 = 0; T1 = KEY type */
skein_start_new_type(ctx, KEY);
/* zero the initial chaining variables */
@@ -398,7 +398,7 @@
*/
if (msg_byte_cnt > SKEIN_512_BLOCK_BYTES) {
/* number of full blocks to process */
- n = (msg_byte_cnt-1) / SKEIN_512_BLOCK_BYTES;
+ n = (msg_byte_cnt - 1) / SKEIN_512_BLOCK_BYTES;
skein_512_process_block(ctx, msg, n,
SKEIN_512_BLOCK_BYTES);
msg_byte_cnt -= n * SKEIN_512_BLOCK_BYTES;
@@ -432,7 +432,7 @@
/* zero pad b[] if necessary */
if (ctx->h.b_cnt < SKEIN_512_BLOCK_BYTES)
memset(&ctx->b[ctx->h.b_cnt], 0,
- SKEIN_512_BLOCK_BYTES - ctx->h.b_cnt);
+ SKEIN_512_BLOCK_BYTES - ctx->h.b_cnt);
/* process the final block */
skein_512_process_block(ctx, ctx->b, 1, ctx->h.b_cnt);
@@ -446,19 +446,19 @@
memset(ctx->b, 0, sizeof(ctx->b));
/* keep a local copy of counter mode "key" */
memcpy(x, ctx->x, sizeof(x));
- for (i = 0; i*SKEIN_512_BLOCK_BYTES < byte_cnt; i++) {
+ for (i = 0; i * SKEIN_512_BLOCK_BYTES < byte_cnt; i++) {
/* build the counter block */
- ((u64 *)ctx->b)[0] = skein_swap64((u64) i);
+ ((u64 *)ctx->b)[0] = skein_swap64((u64)i);
skein_start_new_type(ctx, OUT_FINAL);
/* run "counter mode" */
skein_512_process_block(ctx, ctx->b, 1, sizeof(u64));
/* number of output bytes left to go */
- n = byte_cnt - i*SKEIN_512_BLOCK_BYTES;
+ n = byte_cnt - i * SKEIN_512_BLOCK_BYTES;
if (n >= SKEIN_512_BLOCK_BYTES)
n = SKEIN_512_BLOCK_BYTES;
/* "output" the ctr mode bytes */
- skein_put64_lsb_first(hash_val+i*SKEIN_512_BLOCK_BYTES, ctx->x,
- n);
+ skein_put64_lsb_first(hash_val + (i * SKEIN_512_BLOCK_BYTES),
+ ctx->x, n);
/* restore the counter mode key for next time */
memcpy(ctx->x, x, sizeof(x));
}
@@ -506,7 +506,7 @@
cfg.w[1] = skein_swap64(hash_bit_len);
cfg.w[2] = skein_swap64(SKEIN_CFG_TREE_INFO_SEQUENTIAL);
/* zero pad config block */
- memset(&cfg.w[3], 0, sizeof(cfg) - 3*sizeof(cfg.w[0]));
+ memset(&cfg.w[3], 0, sizeof(cfg) - 3 * sizeof(cfg.w[0]));
/* compute the initial chaining values from config block */
/* zero the chaining variables */
@@ -547,7 +547,7 @@
skein_assert(sizeof(cfg.b) >= sizeof(ctx->x));
/* do a mini-Init right here */
/* set output hash bit count = state size */
- ctx->h.hash_bit_len = 8*sizeof(ctx->x);
+ ctx->h.hash_bit_len = 8 * sizeof(ctx->x);
/* set tweaks: T0 = 0; T1 = KEY type */
skein_start_new_type(ctx, KEY);
/* zero the initial chaining variables */
@@ -620,7 +620,7 @@
*/
if (msg_byte_cnt > SKEIN_1024_BLOCK_BYTES) {
/* number of full blocks to process */
- n = (msg_byte_cnt-1) / SKEIN_1024_BLOCK_BYTES;
+ n = (msg_byte_cnt - 1) / SKEIN_1024_BLOCK_BYTES;
skein_1024_process_block(ctx, msg, n,
SKEIN_1024_BLOCK_BYTES);
msg_byte_cnt -= n * SKEIN_1024_BLOCK_BYTES;
@@ -654,7 +654,7 @@
/* zero pad b[] if necessary */
if (ctx->h.b_cnt < SKEIN_1024_BLOCK_BYTES)
memset(&ctx->b[ctx->h.b_cnt], 0,
- SKEIN_1024_BLOCK_BYTES - ctx->h.b_cnt);
+ SKEIN_1024_BLOCK_BYTES - ctx->h.b_cnt);
/* process the final block */
skein_1024_process_block(ctx, ctx->b, 1, ctx->h.b_cnt);
@@ -668,19 +668,19 @@
memset(ctx->b, 0, sizeof(ctx->b));
/* keep a local copy of counter mode "key" */
memcpy(x, ctx->x, sizeof(x));
- for (i = 0; i*SKEIN_1024_BLOCK_BYTES < byte_cnt; i++) {
+ for (i = 0; i * SKEIN_1024_BLOCK_BYTES < byte_cnt; i++) {
/* build the counter block */
- ((u64 *)ctx->b)[0] = skein_swap64((u64) i);
+ ((u64 *)ctx->b)[0] = skein_swap64((u64)i);
skein_start_new_type(ctx, OUT_FINAL);
/* run "counter mode" */
skein_1024_process_block(ctx, ctx->b, 1, sizeof(u64));
/* number of output bytes left to go */
- n = byte_cnt - i*SKEIN_1024_BLOCK_BYTES;
+ n = byte_cnt - i * SKEIN_1024_BLOCK_BYTES;
if (n >= SKEIN_1024_BLOCK_BYTES)
n = SKEIN_1024_BLOCK_BYTES;
/* "output" the ctr mode bytes */
- skein_put64_lsb_first(hash_val+i*SKEIN_1024_BLOCK_BYTES, ctx->x,
- n);
+ skein_put64_lsb_first(hash_val + (i * SKEIN_1024_BLOCK_BYTES),
+ ctx->x, n);
/* restore the counter mode key for next time */
memcpy(ctx->x, x, sizeof(x));
}
@@ -702,7 +702,7 @@
/* zero pad b[] if necessary */
if (ctx->h.b_cnt < SKEIN_256_BLOCK_BYTES)
memset(&ctx->b[ctx->h.b_cnt], 0,
- SKEIN_256_BLOCK_BYTES - ctx->h.b_cnt);
+ SKEIN_256_BLOCK_BYTES - ctx->h.b_cnt);
/* process the final block */
skein_256_process_block(ctx, ctx->b, 1, ctx->h.b_cnt);
@@ -724,7 +724,7 @@
/* zero pad b[] if necessary */
if (ctx->h.b_cnt < SKEIN_512_BLOCK_BYTES)
memset(&ctx->b[ctx->h.b_cnt], 0,
- SKEIN_512_BLOCK_BYTES - ctx->h.b_cnt);
+ SKEIN_512_BLOCK_BYTES - ctx->h.b_cnt);
/* process the final block */
skein_512_process_block(ctx, ctx->b, 1, ctx->h.b_cnt);
@@ -746,7 +746,7 @@
/* zero pad b[] if necessary */
if (ctx->h.b_cnt < SKEIN_1024_BLOCK_BYTES)
memset(&ctx->b[ctx->h.b_cnt], 0,
- SKEIN_1024_BLOCK_BYTES - ctx->h.b_cnt);
+ SKEIN_1024_BLOCK_BYTES - ctx->h.b_cnt);
/* process the final block */
skein_1024_process_block(ctx, ctx->b, 1, ctx->h.b_cnt);
@@ -775,19 +775,19 @@
memset(ctx->b, 0, sizeof(ctx->b));
/* keep a local copy of counter mode "key" */
memcpy(x, ctx->x, sizeof(x));
- for (i = 0; i*SKEIN_256_BLOCK_BYTES < byte_cnt; i++) {
+ for (i = 0; i * SKEIN_256_BLOCK_BYTES < byte_cnt; i++) {
/* build the counter block */
- ((u64 *)ctx->b)[0] = skein_swap64((u64) i);
+ ((u64 *)ctx->b)[0] = skein_swap64((u64)i);
skein_start_new_type(ctx, OUT_FINAL);
/* run "counter mode" */
skein_256_process_block(ctx, ctx->b, 1, sizeof(u64));
/* number of output bytes left to go */
- n = byte_cnt - i*SKEIN_256_BLOCK_BYTES;
+ n = byte_cnt - i * SKEIN_256_BLOCK_BYTES;
if (n >= SKEIN_256_BLOCK_BYTES)
n = SKEIN_256_BLOCK_BYTES;
/* "output" the ctr mode bytes */
- skein_put64_lsb_first(hash_val+i*SKEIN_256_BLOCK_BYTES, ctx->x,
- n);
+ skein_put64_lsb_first(hash_val + (i * SKEIN_256_BLOCK_BYTES),
+ ctx->x, n);
/* restore the counter mode key for next time */
memcpy(ctx->x, x, sizeof(x));
}
@@ -812,19 +812,19 @@
memset(ctx->b, 0, sizeof(ctx->b));
/* keep a local copy of counter mode "key" */
memcpy(x, ctx->x, sizeof(x));
- for (i = 0; i*SKEIN_512_BLOCK_BYTES < byte_cnt; i++) {
+ for (i = 0; i * SKEIN_512_BLOCK_BYTES < byte_cnt; i++) {
/* build the counter block */
- ((u64 *)ctx->b)[0] = skein_swap64((u64) i);
+ ((u64 *)ctx->b)[0] = skein_swap64((u64)i);
skein_start_new_type(ctx, OUT_FINAL);
/* run "counter mode" */
skein_512_process_block(ctx, ctx->b, 1, sizeof(u64));
/* number of output bytes left to go */
- n = byte_cnt - i*SKEIN_512_BLOCK_BYTES;
+ n = byte_cnt - i * SKEIN_512_BLOCK_BYTES;
if (n >= SKEIN_512_BLOCK_BYTES)
n = SKEIN_512_BLOCK_BYTES;
/* "output" the ctr mode bytes */
- skein_put64_lsb_first(hash_val+i*SKEIN_512_BLOCK_BYTES, ctx->x,
- n);
+ skein_put64_lsb_first(hash_val + (i * SKEIN_512_BLOCK_BYTES),
+ ctx->x, n);
/* restore the counter mode key for next time */
memcpy(ctx->x, x, sizeof(x));
}
@@ -849,19 +849,19 @@
memset(ctx->b, 0, sizeof(ctx->b));
/* keep a local copy of counter mode "key" */
memcpy(x, ctx->x, sizeof(x));
- for (i = 0; i*SKEIN_1024_BLOCK_BYTES < byte_cnt; i++) {
+ for (i = 0; i * SKEIN_1024_BLOCK_BYTES < byte_cnt; i++) {
/* build the counter block */
- ((u64 *)ctx->b)[0] = skein_swap64((u64) i);
+ ((u64 *)ctx->b)[0] = skein_swap64((u64)i);
skein_start_new_type(ctx, OUT_FINAL);
/* run "counter mode" */
skein_1024_process_block(ctx, ctx->b, 1, sizeof(u64));
/* number of output bytes left to go */
- n = byte_cnt - i*SKEIN_1024_BLOCK_BYTES;
+ n = byte_cnt - i * SKEIN_1024_BLOCK_BYTES;
if (n >= SKEIN_1024_BLOCK_BYTES)
n = SKEIN_1024_BLOCK_BYTES;
/* "output" the ctr mode bytes */
- skein_put64_lsb_first(hash_val+i*SKEIN_1024_BLOCK_BYTES, ctx->x,
- n);
+ skein_put64_lsb_first(hash_val + (i * SKEIN_1024_BLOCK_BYTES),
+ ctx->x, n);
/* restore the counter mode key for next time */
memcpy(ctx->x, x, sizeof(x));
}
diff --git a/drivers/staging/skein/skein_base.h b/drivers/staging/skein/skein_base.h
index 3c7f8ad..dc464f3 100644
--- a/drivers/staging/skein/skein_base.h
+++ b/drivers/staging/skein/skein_base.h
@@ -32,7 +32,7 @@
/* below two prototype assume we are handed aligned data */
#define skein_put64_lsb_first(dst08, src64, b_cnt) memcpy(dst08, src64, b_cnt)
#define skein_get64_lsb_first(dst64, src08, w_cnt) \
- memcpy(dst64, src08, 8*(w_cnt))
+ memcpy(dst64, src08, 8 * (w_cnt))
#define skein_swap64(w64) (w64)
enum {
@@ -48,17 +48,17 @@
#define SKEIN_1024_STATE_WORDS 16
#define SKEIN_MAX_STATE_WORDS 16
-#define SKEIN_256_STATE_BYTES (8*SKEIN_256_STATE_WORDS)
-#define SKEIN_512_STATE_BYTES (8*SKEIN_512_STATE_WORDS)
-#define SKEIN_1024_STATE_BYTES (8*SKEIN_1024_STATE_WORDS)
+#define SKEIN_256_STATE_BYTES (8 * SKEIN_256_STATE_WORDS)
+#define SKEIN_512_STATE_BYTES (8 * SKEIN_512_STATE_WORDS)
+#define SKEIN_1024_STATE_BYTES (8 * SKEIN_1024_STATE_WORDS)
-#define SKEIN_256_STATE_BITS (64*SKEIN_256_STATE_WORDS)
-#define SKEIN_512_STATE_BITS (64*SKEIN_512_STATE_WORDS)
-#define SKEIN_1024_STATE_BITS (64*SKEIN_1024_STATE_WORDS)
+#define SKEIN_256_STATE_BITS (64 * SKEIN_256_STATE_WORDS)
+#define SKEIN_512_STATE_BITS (64 * SKEIN_512_STATE_WORDS)
+#define SKEIN_1024_STATE_BITS (64 * SKEIN_1024_STATE_WORDS)
-#define SKEIN_256_BLOCK_BYTES (8*SKEIN_256_STATE_WORDS)
-#define SKEIN_512_BLOCK_BYTES (8*SKEIN_512_STATE_WORDS)
-#define SKEIN_1024_BLOCK_BYTES (8*SKEIN_1024_STATE_WORDS)
+#define SKEIN_256_BLOCK_BYTES (8 * SKEIN_256_STATE_WORDS)
+#define SKEIN_512_BLOCK_BYTES (8 * SKEIN_512_STATE_WORDS)
+#define SKEIN_1024_BLOCK_BYTES (8 * SKEIN_1024_STATE_WORDS)
struct skein_ctx_hdr {
size_t hash_bit_len; /* size of hash result, in bits */
@@ -84,11 +84,6 @@
u8 b[SKEIN_1024_BLOCK_BYTES]; /* partial block buf (8-byte aligned) */
};
-static inline u64 rotl_64(u64 x, u8 N)
-{
- return (x << N) | (x >> (64 - N));
-}
-
/* Skein APIs for (incremental) "straight hashing" */
int skein_256_init(struct skein_256_ctx *ctx, size_t hash_bit_len);
int skein_512_init(struct skein_512_ctx *ctx, size_t hash_bit_len);
@@ -162,13 +157,13 @@
#define SKEIN_T1_POS_FINAL SKEIN_T1_BIT(127) /* 127 final blk flag */
/* tweak word tweak[1]: flag bit definition(s) */
-#define SKEIN_T1_FLAG_FIRST (((u64) 1) << SKEIN_T1_POS_FIRST)
-#define SKEIN_T1_FLAG_FINAL (((u64) 1) << SKEIN_T1_POS_FINAL)
-#define SKEIN_T1_FLAG_BIT_PAD (((u64) 1) << SKEIN_T1_POS_BIT_PAD)
+#define SKEIN_T1_FLAG_FIRST (((u64)1) << SKEIN_T1_POS_FIRST)
+#define SKEIN_T1_FLAG_FINAL (((u64)1) << SKEIN_T1_POS_FINAL)
+#define SKEIN_T1_FLAG_BIT_PAD (((u64)1) << SKEIN_T1_POS_BIT_PAD)
/* tweak word tweak[1]: tree level bit field mask */
#define SKEIN_T1_TREE_LVL_MASK (((u64)0x7F) << SKEIN_T1_POS_TREE_LVL)
-#define SKEIN_T1_TREE_LEVEL(n) (((u64) (n)) << SKEIN_T1_POS_TREE_LVL)
+#define SKEIN_T1_TREE_LEVEL(n) (((u64)(n)) << SKEIN_T1_POS_TREE_LVL)
/* tweak word tweak[1]: block type field */
#define SKEIN_BLK_TYPE_KEY (0) /* key, for MAC and KDF */
@@ -181,7 +176,7 @@
#define SKEIN_BLK_TYPE_OUT (63) /* output stage */
#define SKEIN_BLK_TYPE_MASK (63) /* bit field mask */
-#define SKEIN_T1_BLK_TYPE(T) (((u64) (SKEIN_BLK_TYPE_##T)) << \
+#define SKEIN_T1_BLK_TYPE(T) (((u64)(SKEIN_BLK_TYPE_##T)) << \
SKEIN_T1_POS_BLK_TYPE)
#define SKEIN_T1_BLK_TYPE_KEY SKEIN_T1_BLK_TYPE(KEY) /* for MAC and KDF */
#define SKEIN_T1_BLK_TYPE_CFG SKEIN_T1_BLK_TYPE(CFG) /* config block */
@@ -204,11 +199,11 @@
#define SKEIN_ID_STRING_LE (0x33414853) /* "SHA3" (little-endian)*/
#endif
-#define SKEIN_MK_64(hi32, lo32) ((lo32) + (((u64) (hi32)) << 32))
+#define SKEIN_MK_64(hi32, lo32) ((lo32) + (((u64)(hi32)) << 32))
#define SKEIN_SCHEMA_VER SKEIN_MK_64(SKEIN_VERSION, SKEIN_ID_STRING_LE)
#define SKEIN_KS_PARITY SKEIN_MK_64(0x1BD11BDA, 0xA9FC1A22)
-#define SKEIN_CFG_STR_LEN (4*8)
+#define SKEIN_CFG_STR_LEN (4 * 8)
/* bit field definitions in config block tree_info word */
#define SKEIN_CFG_TREE_LEAF_SIZE_POS (0)
@@ -327,9 +322,9 @@
#define SKEIN_512_ROUNDS_TOTAL (72)
#define SKEIN_1024_ROUNDS_TOTAL (80)
#else /* allow command-line define in range 8*(5..14) */
-#define SKEIN_256_ROUNDS_TOTAL (8*((((SKEIN_ROUNDS/100) + 5) % 10) + 5))
-#define SKEIN_512_ROUNDS_TOTAL (8*((((SKEIN_ROUNDS/10) + 5) % 10) + 5))
-#define SKEIN_1024_ROUNDS_TOTAL (8*((((SKEIN_ROUNDS) + 5) % 10) + 5))
+#define SKEIN_256_ROUNDS_TOTAL (8 * ((((SKEIN_ROUNDS / 100) + 5) % 10) + 5))
+#define SKEIN_512_ROUNDS_TOTAL (8 * ((((SKEIN_ROUNDS / 10) + 5) % 10) + 5))
+#define SKEIN_1024_ROUNDS_TOTAL (8 * ((((SKEIN_ROUNDS) + 5) % 10) + 5))
#endif
#endif /* ifndef _SKEIN_H_ */
diff --git a/drivers/staging/skein/skein_block.c b/drivers/staging/skein/skein_block.c
index 45b4732..59a0a8a8 100644
--- a/drivers/staging/skein/skein_block.c
+++ b/drivers/staging/skein/skein_block.c
@@ -15,6 +15,7 @@
************************************************************************/
#include <linux/string.h>
+#include <linux/bitops.h>
#include "skein_base.h"
#include "skein_block.h"
@@ -59,10 +60,10 @@
#define ROUND256(p0, p1, p2, p3, ROT, r_num) \
do { \
X##p0 += X##p1; \
- X##p1 = rotl_64(X##p1, ROT##_0); \
+ X##p1 = rol64(X##p1, ROT##_0); \
X##p1 ^= X##p0; \
X##p2 += X##p3; \
- X##p3 = rotl_64(X##p3, ROT##_1); \
+ X##p3 = rol64(X##p3, ROT##_1); \
X##p3 ^= X##p2; \
} while (0)
@@ -120,10 +121,10 @@
#if !(SKEIN_USE_ASM & 512)
#undef RCNT
-#define RCNT (SKEIN_512_ROUNDS_TOTAL/8)
+#define RCNT (SKEIN_512_ROUNDS_TOTAL / 8)
#ifdef SKEIN_LOOP /* configure how much to unroll the loop */
-#define SKEIN_UNROLL_512 (((SKEIN_LOOP)/10)%10)
+#define SKEIN_UNROLL_512 (((SKEIN_LOOP) / 10) % 10)
#else
#define SKEIN_UNROLL_512 (0)
#endif
@@ -136,15 +137,16 @@
#define ROUND512(p0, p1, p2, p3, p4, p5, p6, p7, ROT, r_num) \
do { \
X##p0 += X##p1; \
- X##p1 = rotl_64(X##p1, ROT##_0); \
+ X##p1 = rol64(X##p1, ROT##_0); \
X##p1 ^= X##p0; \
X##p2 += X##p3; \
- X##p3 = rotl_64(X##p3, ROT##_1); \
+ X##p3 = rol64(X##p3, ROT##_1); \
X##p3 ^= X##p2; \
X##p4 += X##p5; \
- X##p5 = rotl_64(X##p5, ROT##_2); \
+ X##p5 = rol64(X##p5, ROT##_2); \
X##p5 ^= X##p4; \
- X##p6 += X##p7; X##p7 = rotl_64(X##p7, ROT##_3);\
+ X##p6 += X##p7; \
+ X##p7 = rol64(X##p7, ROT##_3); \
X##p7 ^= X##p6; \
} while (0)
@@ -200,7 +202,7 @@
} while (0)
#define R512_UNROLL_R(NN) \
((SKEIN_UNROLL_512 == 0 && \
- SKEIN_512_ROUNDS_TOTAL/8 > (NN)) || \
+ SKEIN_512_ROUNDS_TOTAL / 8 > (NN)) || \
(SKEIN_UNROLL_512 > (NN)))
#if (SKEIN_UNROLL_512 > 14)
@@ -210,7 +212,7 @@
#if !(SKEIN_USE_ASM & 1024)
#undef RCNT
-#define RCNT (SKEIN_1024_ROUNDS_TOTAL/8)
+#define RCNT (SKEIN_1024_ROUNDS_TOTAL / 8)
#ifdef SKEIN_LOOP /* configure how much to unroll the loop */
#define SKEIN_UNROLL_1024 ((SKEIN_LOOP) % 10)
#else
@@ -226,28 +228,28 @@
pF, ROT, r_num) \
do { \
X##p0 += X##p1; \
- X##p1 = rotl_64(X##p1, ROT##_0); \
+ X##p1 = rol64(X##p1, ROT##_0); \
X##p1 ^= X##p0; \
X##p2 += X##p3; \
- X##p3 = rotl_64(X##p3, ROT##_1); \
+ X##p3 = rol64(X##p3, ROT##_1); \
X##p3 ^= X##p2; \
X##p4 += X##p5; \
- X##p5 = rotl_64(X##p5, ROT##_2); \
+ X##p5 = rol64(X##p5, ROT##_2); \
X##p5 ^= X##p4; \
X##p6 += X##p7; \
- X##p7 = rotl_64(X##p7, ROT##_3); \
+ X##p7 = rol64(X##p7, ROT##_3); \
X##p7 ^= X##p6; \
X##p8 += X##p9; \
- X##p9 = rotl_64(X##p9, ROT##_4); \
+ X##p9 = rol64(X##p9, ROT##_4); \
X##p9 ^= X##p8; \
X##pA += X##pB; \
- X##pB = rotl_64(X##pB, ROT##_5); \
+ X##pB = rol64(X##pB, ROT##_5); \
X##pB ^= X##pA; \
X##pC += X##pD; \
- X##pD = rotl_64(X##pD, ROT##_6); \
+ X##pD = rol64(X##pD, ROT##_6); \
X##pD ^= X##pC; \
X##pE += X##pF; \
- X##pF = rotl_64(X##pF, ROT##_7); \
+ X##pF = rol64(X##pF, ROT##_7); \
X##pF ^= X##pE; \
} while (0)
@@ -311,28 +313,28 @@
#define R1024_8_ROUNDS(R) \
do { \
R1024(00, 01, 02, 03, 04, 05, 06, 07, 08, 09, 10, 11, 12, \
- 13, 14, 15, R1024_0, 8*(R) + 1); \
+ 13, 14, 15, R1024_0, 8 * (R) + 1); \
R1024(00, 09, 02, 13, 06, 11, 04, 15, 10, 07, 12, 03, 14, \
- 05, 08, 01, R1024_1, 8*(R) + 2); \
+ 05, 08, 01, R1024_1, 8 * (R) + 2); \
R1024(00, 07, 02, 05, 04, 03, 06, 01, 12, 15, 14, 13, 08, \
- 11, 10, 09, R1024_2, 8*(R) + 3); \
+ 11, 10, 09, R1024_2, 8 * (R) + 3); \
R1024(00, 15, 02, 11, 06, 13, 04, 09, 14, 01, 08, 05, 10, \
- 03, 12, 07, R1024_3, 8*(R) + 4); \
- I1024(2*(R)); \
+ 03, 12, 07, R1024_3, 8 * (R) + 4); \
+ I1024(2 * (R)); \
R1024(00, 01, 02, 03, 04, 05, 06, 07, 08, 09, 10, 11, 12, \
- 13, 14, 15, R1024_4, 8*(R) + 5); \
+ 13, 14, 15, R1024_4, 8 * (R) + 5); \
R1024(00, 09, 02, 13, 06, 11, 04, 15, 10, 07, 12, 03, 14, \
- 05, 08, 01, R1024_5, 8*(R) + 6); \
+ 05, 08, 01, R1024_5, 8 * (R) + 6); \
R1024(00, 07, 02, 05, 04, 03, 06, 01, 12, 15, 14, 13, 08, \
- 11, 10, 09, R1024_6, 8*(R) + 7); \
+ 11, 10, 09, R1024_6, 8 * (R) + 7); \
R1024(00, 15, 02, 11, 06, 13, 04, 09, 14, 01, 08, 05, 10, \
- 03, 12, 07, R1024_7, 8*(R) + 8); \
- I1024(2*(R)+1); \
+ 03, 12, 07, R1024_7, 8 * (R) + 8); \
+ I1024(2 * (R) + 1); \
} while (0)
#define R1024_UNROLL_R(NN) \
((SKEIN_UNROLL_1024 == 0 && \
- SKEIN_1024_ROUNDS_TOTAL/8 > (NN)) || \
+ SKEIN_1024_ROUNDS_TOTAL / 8 > (NN)) || \
(SKEIN_UNROLL_1024 > (NN)))
#if (SKEIN_UNROLL_1024 > 14)
@@ -351,10 +353,10 @@
size_t r;
#if SKEIN_UNROLL_256
/* key schedule: chaining vars + tweak + "rot"*/
- u64 kw[WCNT+4+RCNT*2];
+ u64 kw[WCNT + 4 + (RCNT * 2)];
#else
/* key schedule words : chaining vars + tweak */
- u64 kw[WCNT+4];
+ u64 kw[WCNT + 4];
#endif
u64 X0, X1, X2, X3; /* local copy of context vars, for speed */
u64 w[WCNT]; /* local copy of input block */
@@ -460,9 +462,10 @@
#if defined(SKEIN_CODE_SIZE) || defined(SKEIN_PERF)
size_t skein_256_process_block_code_size(void)
{
- return ((u8 *) skein_256_process_block_code_size) -
- ((u8 *) skein_256_process_block);
+ return ((u8 *)skein_256_process_block_code_size) -
+ ((u8 *)skein_256_process_block);
}
+
unsigned int skein_256_unroll_cnt(void)
{
return SKEIN_UNROLL_256;
@@ -480,9 +483,11 @@
};
size_t r;
#if SKEIN_UNROLL_512
- u64 kw[WCNT+4+RCNT*2]; /* key sched: chaining vars + tweak + "rot"*/
+ /* key sched: chaining vars + tweak + "rot"*/
+ u64 kw[WCNT + 4 + RCNT * 2];
#else
- u64 kw[WCNT+4]; /* key schedule words : chaining vars + tweak */
+ /* key schedule words : chaining vars + tweak */
+ u64 kw[WCNT + 4];
#endif
u64 X0, X1, X2, X3, X4, X5, X6, X7; /* local copies, for speed */
u64 w[WCNT]; /* local copy of input block */
@@ -543,7 +548,6 @@
for (r = 1;
r < (SKEIN_UNROLL_512 ? 2 * RCNT : 2);
r += (SKEIN_UNROLL_512 ? 2 * SKEIN_UNROLL_512 : 1)) {
-
R512_8_ROUNDS(0);
#if R512_UNROLL_R(1)
@@ -609,9 +613,10 @@
#if defined(SKEIN_CODE_SIZE) || defined(SKEIN_PERF)
size_t skein_512_process_block_code_size(void)
{
- return ((u8 *) skein_512_process_block_code_size) -
- ((u8 *) skein_512_process_block);
+ return ((u8 *)skein_512_process_block_code_size) -
+ ((u8 *)skein_512_process_block);
}
+
unsigned int skein_512_unroll_cnt(void)
{
return SKEIN_UNROLL_512;
@@ -629,9 +634,11 @@
};
size_t r;
#if (SKEIN_UNROLL_1024 != 0)
- u64 kw[WCNT+4+RCNT*2]; /* key sched: chaining vars + tweak + "rot" */
+ /* key sched: chaining vars + tweak + "rot" */
+ u64 kw[WCNT + 4 + (RCNT * 2)];
#else
- u64 kw[WCNT+4]; /* key schedule words : chaining vars + tweak */
+ /* key schedule words : chaining vars + tweak */
+ u64 kw[WCNT + 4];
#endif
/* local copy of vars, for speed */
@@ -771,9 +778,10 @@
#if defined(SKEIN_CODE_SIZE) || defined(SKEIN_PERF)
size_t skein_1024_process_block_code_size(void)
{
- return ((u8 *) skein_1024_process_block_code_size) -
- ((u8 *) skein_1024_process_block);
+ return ((u8 *)skein_1024_process_block_code_size) -
+ ((u8 *)skein_1024_process_block);
}
+
unsigned int skein_1024_unroll_cnt(void)
{
return SKEIN_UNROLL_1024;
diff --git a/drivers/staging/skein/skein_generic.c b/drivers/staging/skein/skein_generic.c
index e29b9ab..11f5e53 100644
--- a/drivers/staging/skein/skein_generic.c
+++ b/drivers/staging/skein/skein_generic.c
@@ -27,7 +27,7 @@
}
static int skein256_update(struct shash_desc *desc, const u8 *data,
- unsigned int len)
+ unsigned int len)
{
return skein_256_update((struct skein_256_ctx *)shash_desc_ctx(desc),
data, len);
@@ -62,7 +62,7 @@
}
static int skein512_update(struct shash_desc *desc, const u8 *data,
- unsigned int len)
+ unsigned int len)
{
return skein_512_update((struct skein_512_ctx *)shash_desc_ctx(desc),
data, len);
@@ -97,7 +97,7 @@
}
static int skein1024_update(struct shash_desc *desc, const u8 *data,
- unsigned int len)
+ unsigned int len)
{
return skein_1024_update((struct skein_1024_ctx *)shash_desc_ctx(desc),
data, len);
diff --git a/drivers/staging/skein/threefish_api.h b/drivers/staging/skein/threefish_api.h
index 8e0a0b7..615e467 100644
--- a/drivers/staging/skein/threefish_api.h
+++ b/drivers/staging/skein/threefish_api.h
@@ -52,7 +52,7 @@
*/
struct threefish_key {
u64 state_size;
- u64 key[SKEIN_MAX_STATE_WORDS+1]; /* max number of key words*/
+ u64 key[SKEIN_MAX_STATE_WORDS + 1]; /* max number of key words*/
u64 tweak[3];
};
diff --git a/drivers/staging/skein/threefish_block.c b/drivers/staging/skein/threefish_block.c
index e19ac43..a95563f 100644
--- a/drivers/staging/skein/threefish_block.c
+++ b/drivers/staging/skein/threefish_block.c
@@ -512,622 +512,622 @@
b2 -= k0 + t1;
b3 -= k1 + 18;
tmp = b3 ^ b0;
- b3 = (tmp >> 32) | (tmp << (64 - 32));
+ b3 = ror64(tmp, 32);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 32) | (tmp << (64 - 32));
+ b1 = ror64(tmp, 32);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 58) | (tmp << (64 - 58));
+ b1 = ror64(tmp, 58);
b0 -= b1;
tmp = b3 ^ b2;
- b3 = (tmp >> 22) | (tmp << (64 - 22));
+ b3 = ror64(tmp, 22);
b2 -= b3;
tmp = b3 ^ b0;
- b3 = (tmp >> 46) | (tmp << (64 - 46));
+ b3 = ror64(tmp, 46);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 12) | (tmp << (64 - 12));
+ b1 = ror64(tmp, 12);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 25) | (tmp << (64 - 25));
+ b1 = ror64(tmp, 25);
b0 -= b1 + k2;
b1 -= k3 + t2;
tmp = b3 ^ b2;
- b3 = (tmp >> 33) | (tmp << (64 - 33));
+ b3 = ror64(tmp, 33);
b2 -= b3 + k4 + t0;
b3 -= k0 + 17;
tmp = b3 ^ b0;
- b3 = (tmp >> 5) | (tmp << (64 - 5));
+ b3 = ror64(tmp, 5);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 37) | (tmp << (64 - 37));
+ b1 = ror64(tmp, 37);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 23) | (tmp << (64 - 23));
+ b1 = ror64(tmp, 23);
b0 -= b1;
tmp = b3 ^ b2;
- b3 = (tmp >> 40) | (tmp << (64 - 40));
+ b3 = ror64(tmp, 40);
b2 -= b3;
tmp = b3 ^ b0;
- b3 = (tmp >> 52) | (tmp << (64 - 52));
+ b3 = ror64(tmp, 52);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 57) | (tmp << (64 - 57));
+ b1 = ror64(tmp, 57);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 14) | (tmp << (64 - 14));
+ b1 = ror64(tmp, 14);
b0 -= b1 + k1;
b1 -= k2 + t1;
tmp = b3 ^ b2;
- b3 = (tmp >> 16) | (tmp << (64 - 16));
+ b3 = ror64(tmp, 16);
b2 -= b3 + k3 + t2;
b3 -= k4 + 16;
tmp = b3 ^ b0;
- b3 = (tmp >> 32) | (tmp << (64 - 32));
+ b3 = ror64(tmp, 32);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 32) | (tmp << (64 - 32));
+ b1 = ror64(tmp, 32);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 58) | (tmp << (64 - 58));
+ b1 = ror64(tmp, 58);
b0 -= b1;
tmp = b3 ^ b2;
- b3 = (tmp >> 22) | (tmp << (64 - 22));
+ b3 = ror64(tmp, 22);
b2 -= b3;
tmp = b3 ^ b0;
- b3 = (tmp >> 46) | (tmp << (64 - 46));
+ b3 = ror64(tmp, 46);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 12) | (tmp << (64 - 12));
+ b1 = ror64(tmp, 12);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 25) | (tmp << (64 - 25));
+ b1 = ror64(tmp, 25);
b0 -= b1 + k0;
b1 -= k1 + t0;
tmp = b3 ^ b2;
- b3 = (tmp >> 33) | (tmp << (64 - 33));
+ b3 = ror64(tmp, 33);
b2 -= b3 + k2 + t1;
b3 -= k3 + 15;
tmp = b3 ^ b0;
- b3 = (tmp >> 5) | (tmp << (64 - 5));
+ b3 = ror64(tmp, 5);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 37) | (tmp << (64 - 37));
+ b1 = ror64(tmp, 37);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 23) | (tmp << (64 - 23));
+ b1 = ror64(tmp, 23);
b0 -= b1;
tmp = b3 ^ b2;
- b3 = (tmp >> 40) | (tmp << (64 - 40));
+ b3 = ror64(tmp, 40);
b2 -= b3;
tmp = b3 ^ b0;
- b3 = (tmp >> 52) | (tmp << (64 - 52));
+ b3 = ror64(tmp, 52);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 57) | (tmp << (64 - 57));
+ b1 = ror64(tmp, 57);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 14) | (tmp << (64 - 14));
+ b1 = ror64(tmp, 14);
b0 -= b1 + k4;
b1 -= k0 + t2;
tmp = b3 ^ b2;
- b3 = (tmp >> 16) | (tmp << (64 - 16));
+ b3 = ror64(tmp, 16);
b2 -= b3 + k1 + t0;
b3 -= k2 + 14;
tmp = b3 ^ b0;
- b3 = (tmp >> 32) | (tmp << (64 - 32));
+ b3 = ror64(tmp, 32);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 32) | (tmp << (64 - 32));
+ b1 = ror64(tmp, 32);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 58) | (tmp << (64 - 58));
+ b1 = ror64(tmp, 58);
b0 -= b1;
tmp = b3 ^ b2;
- b3 = (tmp >> 22) | (tmp << (64 - 22));
+ b3 = ror64(tmp, 22);
b2 -= b3;
tmp = b3 ^ b0;
- b3 = (tmp >> 46) | (tmp << (64 - 46));
+ b3 = ror64(tmp, 46);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 12) | (tmp << (64 - 12));
+ b1 = ror64(tmp, 12);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 25) | (tmp << (64 - 25));
+ b1 = ror64(tmp, 25);
b0 -= b1 + k3;
b1 -= k4 + t1;
tmp = b3 ^ b2;
- b3 = (tmp >> 33) | (tmp << (64 - 33));
+ b3 = ror64(tmp, 33);
b2 -= b3 + k0 + t2;
b3 -= k1 + 13;
tmp = b3 ^ b0;
- b3 = (tmp >> 5) | (tmp << (64 - 5));
+ b3 = ror64(tmp, 5);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 37) | (tmp << (64 - 37));
+ b1 = ror64(tmp, 37);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 23) | (tmp << (64 - 23));
+ b1 = ror64(tmp, 23);
b0 -= b1;
tmp = b3 ^ b2;
- b3 = (tmp >> 40) | (tmp << (64 - 40));
+ b3 = ror64(tmp, 40);
b2 -= b3;
tmp = b3 ^ b0;
- b3 = (tmp >> 52) | (tmp << (64 - 52));
+ b3 = ror64(tmp, 52);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 57) | (tmp << (64 - 57));
+ b1 = ror64(tmp, 57);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 14) | (tmp << (64 - 14));
+ b1 = ror64(tmp, 14);
b0 -= b1 + k2;
b1 -= k3 + t0;
tmp = b3 ^ b2;
- b3 = (tmp >> 16) | (tmp << (64 - 16));
+ b3 = ror64(tmp, 16);
b2 -= b3 + k4 + t1;
b3 -= k0 + 12;
tmp = b3 ^ b0;
- b3 = (tmp >> 32) | (tmp << (64 - 32));
+ b3 = ror64(tmp, 32);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 32) | (tmp << (64 - 32));
+ b1 = ror64(tmp, 32);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 58) | (tmp << (64 - 58));
+ b1 = ror64(tmp, 58);
b0 -= b1;
tmp = b3 ^ b2;
- b3 = (tmp >> 22) | (tmp << (64 - 22));
+ b3 = ror64(tmp, 22);
b2 -= b3;
tmp = b3 ^ b0;
- b3 = (tmp >> 46) | (tmp << (64 - 46));
+ b3 = ror64(tmp, 46);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 12) | (tmp << (64 - 12));
+ b1 = ror64(tmp, 12);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 25) | (tmp << (64 - 25));
+ b1 = ror64(tmp, 25);
b0 -= b1 + k1;
b1 -= k2 + t2;
tmp = b3 ^ b2;
- b3 = (tmp >> 33) | (tmp << (64 - 33));
+ b3 = ror64(tmp, 33);
b2 -= b3 + k3 + t0;
b3 -= k4 + 11;
tmp = b3 ^ b0;
- b3 = (tmp >> 5) | (tmp << (64 - 5));
+ b3 = ror64(tmp, 5);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 37) | (tmp << (64 - 37));
+ b1 = ror64(tmp, 37);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 23) | (tmp << (64 - 23));
+ b1 = ror64(tmp, 23);
b0 -= b1;
tmp = b3 ^ b2;
- b3 = (tmp >> 40) | (tmp << (64 - 40));
+ b3 = ror64(tmp, 40);
b2 -= b3;
tmp = b3 ^ b0;
- b3 = (tmp >> 52) | (tmp << (64 - 52));
+ b3 = ror64(tmp, 52);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 57) | (tmp << (64 - 57));
+ b1 = ror64(tmp, 57);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 14) | (tmp << (64 - 14));
+ b1 = ror64(tmp, 14);
b0 -= b1 + k0;
b1 -= k1 + t1;
tmp = b3 ^ b2;
- b3 = (tmp >> 16) | (tmp << (64 - 16));
+ b3 = ror64(tmp, 16);
b2 -= b3 + k2 + t2;
b3 -= k3 + 10;
tmp = b3 ^ b0;
- b3 = (tmp >> 32) | (tmp << (64 - 32));
+ b3 = ror64(tmp, 32);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 32) | (tmp << (64 - 32));
+ b1 = ror64(tmp, 32);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 58) | (tmp << (64 - 58));
+ b1 = ror64(tmp, 58);
b0 -= b1;
tmp = b3 ^ b2;
- b3 = (tmp >> 22) | (tmp << (64 - 22));
+ b3 = ror64(tmp, 22);
b2 -= b3;
tmp = b3 ^ b0;
- b3 = (tmp >> 46) | (tmp << (64 - 46));
+ b3 = ror64(tmp, 46);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 12) | (tmp << (64 - 12));
+ b1 = ror64(tmp, 12);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 25) | (tmp << (64 - 25));
+ b1 = ror64(tmp, 25);
b0 -= b1 + k4;
b1 -= k0 + t0;
tmp = b3 ^ b2;
- b3 = (tmp >> 33) | (tmp << (64 - 33));
+ b3 = ror64(tmp, 33);
b2 -= b3 + k1 + t1;
b3 -= k2 + 9;
tmp = b3 ^ b0;
- b3 = (tmp >> 5) | (tmp << (64 - 5));
+ b3 = ror64(tmp, 5);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 37) | (tmp << (64 - 37));
+ b1 = ror64(tmp, 37);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 23) | (tmp << (64 - 23));
+ b1 = ror64(tmp, 23);
b0 -= b1;
tmp = b3 ^ b2;
- b3 = (tmp >> 40) | (tmp << (64 - 40));
+ b3 = ror64(tmp, 40);
b2 -= b3;
tmp = b3 ^ b0;
- b3 = (tmp >> 52) | (tmp << (64 - 52));
+ b3 = ror64(tmp, 52);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 57) | (tmp << (64 - 57));
+ b1 = ror64(tmp, 57);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 14) | (tmp << (64 - 14));
+ b1 = ror64(tmp, 14);
b0 -= b1 + k3;
b1 -= k4 + t2;
tmp = b3 ^ b2;
- b3 = (tmp >> 16) | (tmp << (64 - 16));
+ b3 = ror64(tmp, 16);
b2 -= b3 + k0 + t0;
b3 -= k1 + 8;
tmp = b3 ^ b0;
- b3 = (tmp >> 32) | (tmp << (64 - 32));
+ b3 = ror64(tmp, 32);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 32) | (tmp << (64 - 32));
+ b1 = ror64(tmp, 32);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 58) | (tmp << (64 - 58));
+ b1 = ror64(tmp, 58);
b0 -= b1;
tmp = b3 ^ b2;
- b3 = (tmp >> 22) | (tmp << (64 - 22));
+ b3 = ror64(tmp, 22);
b2 -= b3;
tmp = b3 ^ b0;
- b3 = (tmp >> 46) | (tmp << (64 - 46));
+ b3 = ror64(tmp, 46);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 12) | (tmp << (64 - 12));
+ b1 = ror64(tmp, 12);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 25) | (tmp << (64 - 25));
+ b1 = ror64(tmp, 25);
b0 -= b1 + k2;
b1 -= k3 + t1;
tmp = b3 ^ b2;
- b3 = (tmp >> 33) | (tmp << (64 - 33));
+ b3 = ror64(tmp, 33);
b2 -= b3 + k4 + t2;
b3 -= k0 + 7;
tmp = b3 ^ b0;
- b3 = (tmp >> 5) | (tmp << (64 - 5));
+ b3 = ror64(tmp, 5);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 37) | (tmp << (64 - 37));
+ b1 = ror64(tmp, 37);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 23) | (tmp << (64 - 23));
+ b1 = ror64(tmp, 23);
b0 -= b1;
tmp = b3 ^ b2;
- b3 = (tmp >> 40) | (tmp << (64 - 40));
+ b3 = ror64(tmp, 40);
b2 -= b3;
tmp = b3 ^ b0;
- b3 = (tmp >> 52) | (tmp << (64 - 52));
+ b3 = ror64(tmp, 52);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 57) | (tmp << (64 - 57));
+ b1 = ror64(tmp, 57);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 14) | (tmp << (64 - 14));
+ b1 = ror64(tmp, 14);
b0 -= b1 + k1;
b1 -= k2 + t0;
tmp = b3 ^ b2;
- b3 = (tmp >> 16) | (tmp << (64 - 16));
+ b3 = ror64(tmp, 16);
b2 -= b3 + k3 + t1;
b3 -= k4 + 6;
tmp = b3 ^ b0;
- b3 = (tmp >> 32) | (tmp << (64 - 32));
+ b3 = ror64(tmp, 32);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 32) | (tmp << (64 - 32));
+ b1 = ror64(tmp, 32);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 58) | (tmp << (64 - 58));
+ b1 = ror64(tmp, 58);
b0 -= b1;
tmp = b3 ^ b2;
- b3 = (tmp >> 22) | (tmp << (64 - 22));
+ b3 = ror64(tmp, 22);
b2 -= b3;
tmp = b3 ^ b0;
- b3 = (tmp >> 46) | (tmp << (64 - 46));
+ b3 = ror64(tmp, 46);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 12) | (tmp << (64 - 12));
+ b1 = ror64(tmp, 12);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 25) | (tmp << (64 - 25));
+ b1 = ror64(tmp, 25);
b0 -= b1 + k0;
b1 -= k1 + t2;
tmp = b3 ^ b2;
- b3 = (tmp >> 33) | (tmp << (64 - 33));
+ b3 = ror64(tmp, 33);
b2 -= b3 + k2 + t0;
b3 -= k3 + 5;
tmp = b3 ^ b0;
- b3 = (tmp >> 5) | (tmp << (64 - 5));
+ b3 = ror64(tmp, 5);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 37) | (tmp << (64 - 37));
+ b1 = ror64(tmp, 37);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 23) | (tmp << (64 - 23));
+ b1 = ror64(tmp, 23);
b0 -= b1;
tmp = b3 ^ b2;
- b3 = (tmp >> 40) | (tmp << (64 - 40));
+ b3 = ror64(tmp, 40);
b2 -= b3;
tmp = b3 ^ b0;
- b3 = (tmp >> 52) | (tmp << (64 - 52));
+ b3 = ror64(tmp, 52);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 57) | (tmp << (64 - 57));
+ b1 = ror64(tmp, 57);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 14) | (tmp << (64 - 14));
+ b1 = ror64(tmp, 14);
b0 -= b1 + k4;
b1 -= k0 + t1;
tmp = b3 ^ b2;
- b3 = (tmp >> 16) | (tmp << (64 - 16));
+ b3 = ror64(tmp, 16);
b2 -= b3 + k1 + t2;
b3 -= k2 + 4;
tmp = b3 ^ b0;
- b3 = (tmp >> 32) | (tmp << (64 - 32));
+ b3 = ror64(tmp, 32);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 32) | (tmp << (64 - 32));
+ b1 = ror64(tmp, 32);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 58) | (tmp << (64 - 58));
+ b1 = ror64(tmp, 58);
b0 -= b1;
tmp = b3 ^ b2;
- b3 = (tmp >> 22) | (tmp << (64 - 22));
+ b3 = ror64(tmp, 22);
b2 -= b3;
tmp = b3 ^ b0;
- b3 = (tmp >> 46) | (tmp << (64 - 46));
+ b3 = ror64(tmp, 46);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 12) | (tmp << (64 - 12));
+ b1 = ror64(tmp, 12);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 25) | (tmp << (64 - 25));
+ b1 = ror64(tmp, 25);
b0 -= b1 + k3;
b1 -= k4 + t0;
tmp = b3 ^ b2;
- b3 = (tmp >> 33) | (tmp << (64 - 33));
+ b3 = ror64(tmp, 33);
b2 -= b3 + k0 + t1;
b3 -= k1 + 3;
tmp = b3 ^ b0;
- b3 = (tmp >> 5) | (tmp << (64 - 5));
+ b3 = ror64(tmp, 5);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 37) | (tmp << (64 - 37));
+ b1 = ror64(tmp, 37);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 23) | (tmp << (64 - 23));
+ b1 = ror64(tmp, 23);
b0 -= b1;
tmp = b3 ^ b2;
- b3 = (tmp >> 40) | (tmp << (64 - 40));
+ b3 = ror64(tmp, 40);
b2 -= b3;
tmp = b3 ^ b0;
- b3 = (tmp >> 52) | (tmp << (64 - 52));
+ b3 = ror64(tmp, 52);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 57) | (tmp << (64 - 57));
+ b1 = ror64(tmp, 57);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 14) | (tmp << (64 - 14));
+ b1 = ror64(tmp, 14);
b0 -= b1 + k2;
b1 -= k3 + t2;
tmp = b3 ^ b2;
- b3 = (tmp >> 16) | (tmp << (64 - 16));
+ b3 = ror64(tmp, 16);
b2 -= b3 + k4 + t0;
b3 -= k0 + 2;
tmp = b3 ^ b0;
- b3 = (tmp >> 32) | (tmp << (64 - 32));
+ b3 = ror64(tmp, 32);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 32) | (tmp << (64 - 32));
+ b1 = ror64(tmp, 32);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 58) | (tmp << (64 - 58));
+ b1 = ror64(tmp, 58);
b0 -= b1;
tmp = b3 ^ b2;
- b3 = (tmp >> 22) | (tmp << (64 - 22));
+ b3 = ror64(tmp, 22);
b2 -= b3;
tmp = b3 ^ b0;
- b3 = (tmp >> 46) | (tmp << (64 - 46));
+ b3 = ror64(tmp, 46);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 12) | (tmp << (64 - 12));
+ b1 = ror64(tmp, 12);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 25) | (tmp << (64 - 25));
+ b1 = ror64(tmp, 25);
b0 -= b1 + k1;
b1 -= k2 + t1;
tmp = b3 ^ b2;
- b3 = (tmp >> 33) | (tmp << (64 - 33));
+ b3 = ror64(tmp, 33);
b2 -= b3 + k3 + t2;
b3 -= k4 + 1;
tmp = b3 ^ b0;
- b3 = (tmp >> 5) | (tmp << (64 - 5));
+ b3 = ror64(tmp, 5);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 37) | (tmp << (64 - 37));
+ b1 = ror64(tmp, 37);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 23) | (tmp << (64 - 23));
+ b1 = ror64(tmp, 23);
b0 -= b1;
tmp = b3 ^ b2;
- b3 = (tmp >> 40) | (tmp << (64 - 40));
+ b3 = ror64(tmp, 40);
b2 -= b3;
tmp = b3 ^ b0;
- b3 = (tmp >> 52) | (tmp << (64 - 52));
+ b3 = ror64(tmp, 52);
b0 -= b3;
tmp = b1 ^ b2;
- b1 = (tmp >> 57) | (tmp << (64 - 57));
+ b1 = ror64(tmp, 57);
b2 -= b1;
tmp = b1 ^ b0;
- b1 = (tmp >> 14) | (tmp << (64 - 14));
+ b1 = ror64(tmp, 14);
b0 -= b1 + k0;
b1 -= k1 + t0;
tmp = b3 ^ b2;
- b3 = (tmp >> 16) | (tmp << (64 - 16));
+ b3 = ror64(tmp, 16);
b2 -= b3 + k2 + t1;
b3 -= k3;
@@ -2125,1226 +2125,1226 @@
b7 -= k7 + 18;
tmp = b3 ^ b4;
- b3 = (tmp >> 22) | (tmp << (64 - 22));
+ b3 = ror64(tmp, 22);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 56) | (tmp << (64 - 56));
+ b5 = ror64(tmp, 56);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 35) | (tmp << (64 - 35));
+ b7 = ror64(tmp, 35);
b0 -= b7;
tmp = b1 ^ b6;
- b1 = (tmp >> 8) | (tmp << (64 - 8));
+ b1 = ror64(tmp, 8);
b6 -= b1;
tmp = b7 ^ b2;
- b7 = (tmp >> 43) | (tmp << (64 - 43));
+ b7 = ror64(tmp, 43);
b2 -= b7;
tmp = b5 ^ b0;
- b5 = (tmp >> 39) | (tmp << (64 - 39));
+ b5 = ror64(tmp, 39);
b0 -= b5;
tmp = b3 ^ b6;
- b3 = (tmp >> 29) | (tmp << (64 - 29));
+ b3 = ror64(tmp, 29);
b6 -= b3;
tmp = b1 ^ b4;
- b1 = (tmp >> 25) | (tmp << (64 - 25));
+ b1 = ror64(tmp, 25);
b4 -= b1;
tmp = b3 ^ b0;
- b3 = (tmp >> 17) | (tmp << (64 - 17));
+ b3 = ror64(tmp, 17);
b0 -= b3;
tmp = b5 ^ b6;
- b5 = (tmp >> 10) | (tmp << (64 - 10));
+ b5 = ror64(tmp, 10);
b6 -= b5;
tmp = b7 ^ b4;
- b7 = (tmp >> 50) | (tmp << (64 - 50));
+ b7 = ror64(tmp, 50);
b4 -= b7;
tmp = b1 ^ b2;
- b1 = (tmp >> 13) | (tmp << (64 - 13));
+ b1 = ror64(tmp, 13);
b2 -= b1;
tmp = b7 ^ b6;
- b7 = (tmp >> 24) | (tmp << (64 - 24));
+ b7 = ror64(tmp, 24);
b6 -= b7 + k5 + t0;
b7 -= k6 + 17;
tmp = b5 ^ b4;
- b5 = (tmp >> 34) | (tmp << (64 - 34));
+ b5 = ror64(tmp, 34);
b4 -= b5 + k3;
b5 -= k4 + t2;
tmp = b3 ^ b2;
- b3 = (tmp >> 30) | (tmp << (64 - 30));
+ b3 = ror64(tmp, 30);
b2 -= b3 + k1;
b3 -= k2;
tmp = b1 ^ b0;
- b1 = (tmp >> 39) | (tmp << (64 - 39));
+ b1 = ror64(tmp, 39);
b0 -= b1 + k8;
b1 -= k0;
tmp = b3 ^ b4;
- b3 = (tmp >> 56) | (tmp << (64 - 56));
+ b3 = ror64(tmp, 56);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 54) | (tmp << (64 - 54));
+ b5 = ror64(tmp, 54);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 9) | (tmp << (64 - 9));
+ b7 = ror64(tmp, 9);
b0 -= b7;
tmp = b1 ^ b6;
- b1 = (tmp >> 44) | (tmp << (64 - 44));
+ b1 = ror64(tmp, 44);
b6 -= b1;
tmp = b7 ^ b2;
- b7 = (tmp >> 39) | (tmp << (64 - 39));
+ b7 = ror64(tmp, 39);
b2 -= b7;
tmp = b5 ^ b0;
- b5 = (tmp >> 36) | (tmp << (64 - 36));
+ b5 = ror64(tmp, 36);
b0 -= b5;
tmp = b3 ^ b6;
- b3 = (tmp >> 49) | (tmp << (64 - 49));
+ b3 = ror64(tmp, 49);
b6 -= b3;
tmp = b1 ^ b4;
- b1 = (tmp >> 17) | (tmp << (64 - 17));
+ b1 = ror64(tmp, 17);
b4 -= b1;
tmp = b3 ^ b0;
- b3 = (tmp >> 42) | (tmp << (64 - 42));
+ b3 = ror64(tmp, 42);
b0 -= b3;
tmp = b5 ^ b6;
- b5 = (tmp >> 14) | (tmp << (64 - 14));
+ b5 = ror64(tmp, 14);
b6 -= b5;
tmp = b7 ^ b4;
- b7 = (tmp >> 27) | (tmp << (64 - 27));
+ b7 = ror64(tmp, 27);
b4 -= b7;
tmp = b1 ^ b2;
- b1 = (tmp >> 33) | (tmp << (64 - 33));
+ b1 = ror64(tmp, 33);
b2 -= b1;
tmp = b7 ^ b6;
- b7 = (tmp >> 37) | (tmp << (64 - 37));
+ b7 = ror64(tmp, 37);
b6 -= b7 + k4 + t2;
b7 -= k5 + 16;
tmp = b5 ^ b4;
- b5 = (tmp >> 19) | (tmp << (64 - 19));
+ b5 = ror64(tmp, 19);
b4 -= b5 + k2;
b5 -= k3 + t1;
tmp = b3 ^ b2;
- b3 = (tmp >> 36) | (tmp << (64 - 36));
+ b3 = ror64(tmp, 36);
b2 -= b3 + k0;
b3 -= k1;
tmp = b1 ^ b0;
- b1 = (tmp >> 46) | (tmp << (64 - 46));
+ b1 = ror64(tmp, 46);
b0 -= b1 + k7;
b1 -= k8;
tmp = b3 ^ b4;
- b3 = (tmp >> 22) | (tmp << (64 - 22));
+ b3 = ror64(tmp, 22);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 56) | (tmp << (64 - 56));
+ b5 = ror64(tmp, 56);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 35) | (tmp << (64 - 35));
+ b7 = ror64(tmp, 35);
b0 -= b7;
tmp = b1 ^ b6;
- b1 = (tmp >> 8) | (tmp << (64 - 8));
+ b1 = ror64(tmp, 8);
b6 -= b1;
tmp = b7 ^ b2;
- b7 = (tmp >> 43) | (tmp << (64 - 43));
+ b7 = ror64(tmp, 43);
b2 -= b7;
tmp = b5 ^ b0;
- b5 = (tmp >> 39) | (tmp << (64 - 39));
+ b5 = ror64(tmp, 39);
b0 -= b5;
tmp = b3 ^ b6;
- b3 = (tmp >> 29) | (tmp << (64 - 29));
+ b3 = ror64(tmp, 29);
b6 -= b3;
tmp = b1 ^ b4;
- b1 = (tmp >> 25) | (tmp << (64 - 25));
+ b1 = ror64(tmp, 25);
b4 -= b1;
tmp = b3 ^ b0;
- b3 = (tmp >> 17) | (tmp << (64 - 17));
+ b3 = ror64(tmp, 17);
b0 -= b3;
tmp = b5 ^ b6;
- b5 = (tmp >> 10) | (tmp << (64 - 10));
+ b5 = ror64(tmp, 10);
b6 -= b5;
tmp = b7 ^ b4;
- b7 = (tmp >> 50) | (tmp << (64 - 50));
+ b7 = ror64(tmp, 50);
b4 -= b7;
tmp = b1 ^ b2;
- b1 = (tmp >> 13) | (tmp << (64 - 13));
+ b1 = ror64(tmp, 13);
b2 -= b1;
tmp = b7 ^ b6;
- b7 = (tmp >> 24) | (tmp << (64 - 24));
+ b7 = ror64(tmp, 24);
b6 -= b7 + k3 + t1;
b7 -= k4 + 15;
tmp = b5 ^ b4;
- b5 = (tmp >> 34) | (tmp << (64 - 34));
+ b5 = ror64(tmp, 34);
b4 -= b5 + k1;
b5 -= k2 + t0;
tmp = b3 ^ b2;
- b3 = (tmp >> 30) | (tmp << (64 - 30));
+ b3 = ror64(tmp, 30);
b2 -= b3 + k8;
b3 -= k0;
tmp = b1 ^ b0;
- b1 = (tmp >> 39) | (tmp << (64 - 39));
+ b1 = ror64(tmp, 39);
b0 -= b1 + k6;
b1 -= k7;
tmp = b3 ^ b4;
- b3 = (tmp >> 56) | (tmp << (64 - 56));
+ b3 = ror64(tmp, 56);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 54) | (tmp << (64 - 54));
+ b5 = ror64(tmp, 54);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 9) | (tmp << (64 - 9));
+ b7 = ror64(tmp, 9);
b0 -= b7;
tmp = b1 ^ b6;
- b1 = (tmp >> 44) | (tmp << (64 - 44));
+ b1 = ror64(tmp, 44);
b6 -= b1;
tmp = b7 ^ b2;
- b7 = (tmp >> 39) | (tmp << (64 - 39));
+ b7 = ror64(tmp, 39);
b2 -= b7;
tmp = b5 ^ b0;
- b5 = (tmp >> 36) | (tmp << (64 - 36));
+ b5 = ror64(tmp, 36);
b0 -= b5;
tmp = b3 ^ b6;
- b3 = (tmp >> 49) | (tmp << (64 - 49));
+ b3 = ror64(tmp, 49);
b6 -= b3;
tmp = b1 ^ b4;
- b1 = (tmp >> 17) | (tmp << (64 - 17));
+ b1 = ror64(tmp, 17);
b4 -= b1;
tmp = b3 ^ b0;
- b3 = (tmp >> 42) | (tmp << (64 - 42));
+ b3 = ror64(tmp, 42);
b0 -= b3;
tmp = b5 ^ b6;
- b5 = (tmp >> 14) | (tmp << (64 - 14));
+ b5 = ror64(tmp, 14);
b6 -= b5;
tmp = b7 ^ b4;
- b7 = (tmp >> 27) | (tmp << (64 - 27));
+ b7 = ror64(tmp, 27);
b4 -= b7;
tmp = b1 ^ b2;
- b1 = (tmp >> 33) | (tmp << (64 - 33));
+ b1 = ror64(tmp, 33);
b2 -= b1;
tmp = b7 ^ b6;
- b7 = (tmp >> 37) | (tmp << (64 - 37));
+ b7 = ror64(tmp, 37);
b6 -= b7 + k2 + t0;
b7 -= k3 + 14;
tmp = b5 ^ b4;
- b5 = (tmp >> 19) | (tmp << (64 - 19));
+ b5 = ror64(tmp, 19);
b4 -= b5 + k0;
b5 -= k1 + t2;
tmp = b3 ^ b2;
- b3 = (tmp >> 36) | (tmp << (64 - 36));
+ b3 = ror64(tmp, 36);
b2 -= b3 + k7;
b3 -= k8;
tmp = b1 ^ b0;
- b1 = (tmp >> 46) | (tmp << (64 - 46));
+ b1 = ror64(tmp, 46);
b0 -= b1 + k5;
b1 -= k6;
tmp = b3 ^ b4;
- b3 = (tmp >> 22) | (tmp << (64 - 22));
+ b3 = ror64(tmp, 22);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 56) | (tmp << (64 - 56));
+ b5 = ror64(tmp, 56);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 35) | (tmp << (64 - 35));
+ b7 = ror64(tmp, 35);
b0 -= b7;
tmp = b1 ^ b6;
- b1 = (tmp >> 8) | (tmp << (64 - 8));
+ b1 = ror64(tmp, 8);
b6 -= b1;
tmp = b7 ^ b2;
- b7 = (tmp >> 43) | (tmp << (64 - 43));
+ b7 = ror64(tmp, 43);
b2 -= b7;
tmp = b5 ^ b0;
- b5 = (tmp >> 39) | (tmp << (64 - 39));
+ b5 = ror64(tmp, 39);
b0 -= b5;
tmp = b3 ^ b6;
- b3 = (tmp >> 29) | (tmp << (64 - 29));
+ b3 = ror64(tmp, 29);
b6 -= b3;
tmp = b1 ^ b4;
- b1 = (tmp >> 25) | (tmp << (64 - 25));
+ b1 = ror64(tmp, 25);
b4 -= b1;
tmp = b3 ^ b0;
- b3 = (tmp >> 17) | (tmp << (64 - 17));
+ b3 = ror64(tmp, 17);
b0 -= b3;
tmp = b5 ^ b6;
- b5 = (tmp >> 10) | (tmp << (64 - 10));
+ b5 = ror64(tmp, 10);
b6 -= b5;
tmp = b7 ^ b4;
- b7 = (tmp >> 50) | (tmp << (64 - 50));
+ b7 = ror64(tmp, 50);
b4 -= b7;
tmp = b1 ^ b2;
- b1 = (tmp >> 13) | (tmp << (64 - 13));
+ b1 = ror64(tmp, 13);
b2 -= b1;
tmp = b7 ^ b6;
- b7 = (tmp >> 24) | (tmp << (64 - 24));
+ b7 = ror64(tmp, 24);
b6 -= b7 + k1 + t2;
b7 -= k2 + 13;
tmp = b5 ^ b4;
- b5 = (tmp >> 34) | (tmp << (64 - 34));
+ b5 = ror64(tmp, 34);
b4 -= b5 + k8;
b5 -= k0 + t1;
tmp = b3 ^ b2;
- b3 = (tmp >> 30) | (tmp << (64 - 30));
+ b3 = ror64(tmp, 30);
b2 -= b3 + k6;
b3 -= k7;
tmp = b1 ^ b0;
- b1 = (tmp >> 39) | (tmp << (64 - 39));
+ b1 = ror64(tmp, 39);
b0 -= b1 + k4;
b1 -= k5;
tmp = b3 ^ b4;
- b3 = (tmp >> 56) | (tmp << (64 - 56));
+ b3 = ror64(tmp, 56);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 54) | (tmp << (64 - 54));
+ b5 = ror64(tmp, 54);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 9) | (tmp << (64 - 9));
+ b7 = ror64(tmp, 9);
b0 -= b7;
tmp = b1 ^ b6;
- b1 = (tmp >> 44) | (tmp << (64 - 44));
+ b1 = ror64(tmp, 44);
b6 -= b1;
tmp = b7 ^ b2;
- b7 = (tmp >> 39) | (tmp << (64 - 39));
+ b7 = ror64(tmp, 39);
b2 -= b7;
tmp = b5 ^ b0;
- b5 = (tmp >> 36) | (tmp << (64 - 36));
+ b5 = ror64(tmp, 36);
b0 -= b5;
tmp = b3 ^ b6;
- b3 = (tmp >> 49) | (tmp << (64 - 49));
+ b3 = ror64(tmp, 49);
b6 -= b3;
tmp = b1 ^ b4;
- b1 = (tmp >> 17) | (tmp << (64 - 17));
+ b1 = ror64(tmp, 17);
b4 -= b1;
tmp = b3 ^ b0;
- b3 = (tmp >> 42) | (tmp << (64 - 42));
+ b3 = ror64(tmp, 42);
b0 -= b3;
tmp = b5 ^ b6;
- b5 = (tmp >> 14) | (tmp << (64 - 14));
+ b5 = ror64(tmp, 14);
b6 -= b5;
tmp = b7 ^ b4;
- b7 = (tmp >> 27) | (tmp << (64 - 27));
+ b7 = ror64(tmp, 27);
b4 -= b7;
tmp = b1 ^ b2;
- b1 = (tmp >> 33) | (tmp << (64 - 33));
+ b1 = ror64(tmp, 33);
b2 -= b1;
tmp = b7 ^ b6;
- b7 = (tmp >> 37) | (tmp << (64 - 37));
+ b7 = ror64(tmp, 37);
b6 -= b7 + k0 + t1;
b7 -= k1 + 12;
tmp = b5 ^ b4;
- b5 = (tmp >> 19) | (tmp << (64 - 19));
+ b5 = ror64(tmp, 19);
b4 -= b5 + k7;
b5 -= k8 + t0;
tmp = b3 ^ b2;
- b3 = (tmp >> 36) | (tmp << (64 - 36));
+ b3 = ror64(tmp, 36);
b2 -= b3 + k5;
b3 -= k6;
tmp = b1 ^ b0;
- b1 = (tmp >> 46) | (tmp << (64 - 46));
+ b1 = ror64(tmp, 46);
b0 -= b1 + k3;
b1 -= k4;
tmp = b3 ^ b4;
- b3 = (tmp >> 22) | (tmp << (64 - 22));
+ b3 = ror64(tmp, 22);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 56) | (tmp << (64 - 56));
+ b5 = ror64(tmp, 56);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 35) | (tmp << (64 - 35));
+ b7 = ror64(tmp, 35);
b0 -= b7;
tmp = b1 ^ b6;
- b1 = (tmp >> 8) | (tmp << (64 - 8));
+ b1 = ror64(tmp, 8);
b6 -= b1;
tmp = b7 ^ b2;
- b7 = (tmp >> 43) | (tmp << (64 - 43));
+ b7 = ror64(tmp, 43);
b2 -= b7;
tmp = b5 ^ b0;
- b5 = (tmp >> 39) | (tmp << (64 - 39));
+ b5 = ror64(tmp, 39);
b0 -= b5;
tmp = b3 ^ b6;
- b3 = (tmp >> 29) | (tmp << (64 - 29));
+ b3 = ror64(tmp, 29);
b6 -= b3;
tmp = b1 ^ b4;
- b1 = (tmp >> 25) | (tmp << (64 - 25));
+ b1 = ror64(tmp, 25);
b4 -= b1;
tmp = b3 ^ b0;
- b3 = (tmp >> 17) | (tmp << (64 - 17));
+ b3 = ror64(tmp, 17);
b0 -= b3;
tmp = b5 ^ b6;
- b5 = (tmp >> 10) | (tmp << (64 - 10));
+ b5 = ror64(tmp, 10);
b6 -= b5;
tmp = b7 ^ b4;
- b7 = (tmp >> 50) | (tmp << (64 - 50));
+ b7 = ror64(tmp, 50);
b4 -= b7;
tmp = b1 ^ b2;
- b1 = (tmp >> 13) | (tmp << (64 - 13));
+ b1 = ror64(tmp, 13);
b2 -= b1;
tmp = b7 ^ b6;
- b7 = (tmp >> 24) | (tmp << (64 - 24));
+ b7 = ror64(tmp, 24);
b6 -= b7 + k8 + t0;
b7 -= k0 + 11;
tmp = b5 ^ b4;
- b5 = (tmp >> 34) | (tmp << (64 - 34));
+ b5 = ror64(tmp, 34);
b4 -= b5 + k6;
b5 -= k7 + t2;
tmp = b3 ^ b2;
- b3 = (tmp >> 30) | (tmp << (64 - 30));
+ b3 = ror64(tmp, 30);
b2 -= b3 + k4;
b3 -= k5;
tmp = b1 ^ b0;
- b1 = (tmp >> 39) | (tmp << (64 - 39));
+ b1 = ror64(tmp, 39);
b0 -= b1 + k2;
b1 -= k3;
tmp = b3 ^ b4;
- b3 = (tmp >> 56) | (tmp << (64 - 56));
+ b3 = ror64(tmp, 56);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 54) | (tmp << (64 - 54));
+ b5 = ror64(tmp, 54);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 9) | (tmp << (64 - 9));
+ b7 = ror64(tmp, 9);
b0 -= b7;
tmp = b1 ^ b6;
- b1 = (tmp >> 44) | (tmp << (64 - 44));
+ b1 = ror64(tmp, 44);
b6 -= b1;
tmp = b7 ^ b2;
- b7 = (tmp >> 39) | (tmp << (64 - 39));
+ b7 = ror64(tmp, 39);
b2 -= b7;
tmp = b5 ^ b0;
- b5 = (tmp >> 36) | (tmp << (64 - 36));
+ b5 = ror64(tmp, 36);
b0 -= b5;
tmp = b3 ^ b6;
- b3 = (tmp >> 49) | (tmp << (64 - 49));
+ b3 = ror64(tmp, 49);
b6 -= b3;
tmp = b1 ^ b4;
- b1 = (tmp >> 17) | (tmp << (64 - 17));
+ b1 = ror64(tmp, 17);
b4 -= b1;
tmp = b3 ^ b0;
- b3 = (tmp >> 42) | (tmp << (64 - 42));
+ b3 = ror64(tmp, 42);
b0 -= b3;
tmp = b5 ^ b6;
- b5 = (tmp >> 14) | (tmp << (64 - 14));
+ b5 = ror64(tmp, 14);
b6 -= b5;
tmp = b7 ^ b4;
- b7 = (tmp >> 27) | (tmp << (64 - 27));
+ b7 = ror64(tmp, 27);
b4 -= b7;
tmp = b1 ^ b2;
- b1 = (tmp >> 33) | (tmp << (64 - 33));
+ b1 = ror64(tmp, 33);
b2 -= b1;
tmp = b7 ^ b6;
- b7 = (tmp >> 37) | (tmp << (64 - 37));
+ b7 = ror64(tmp, 37);
b6 -= b7 + k7 + t2;
b7 -= k8 + 10;
tmp = b5 ^ b4;
- b5 = (tmp >> 19) | (tmp << (64 - 19));
+ b5 = ror64(tmp, 19);
b4 -= b5 + k5;
b5 -= k6 + t1;
tmp = b3 ^ b2;
- b3 = (tmp >> 36) | (tmp << (64 - 36));
+ b3 = ror64(tmp, 36);
b2 -= b3 + k3;
b3 -= k4;
tmp = b1 ^ b0;
- b1 = (tmp >> 46) | (tmp << (64 - 46));
+ b1 = ror64(tmp, 46);
b0 -= b1 + k1;
b1 -= k2;
tmp = b3 ^ b4;
- b3 = (tmp >> 22) | (tmp << (64 - 22));
+ b3 = ror64(tmp, 22);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 56) | (tmp << (64 - 56));
+ b5 = ror64(tmp, 56);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 35) | (tmp << (64 - 35));
+ b7 = ror64(tmp, 35);
b0 -= b7;
tmp = b1 ^ b6;
- b1 = (tmp >> 8) | (tmp << (64 - 8));
+ b1 = ror64(tmp, 8);
b6 -= b1;
tmp = b7 ^ b2;
- b7 = (tmp >> 43) | (tmp << (64 - 43));
+ b7 = ror64(tmp, 43);
b2 -= b7;
tmp = b5 ^ b0;
- b5 = (tmp >> 39) | (tmp << (64 - 39));
+ b5 = ror64(tmp, 39);
b0 -= b5;
tmp = b3 ^ b6;
- b3 = (tmp >> 29) | (tmp << (64 - 29));
+ b3 = ror64(tmp, 29);
b6 -= b3;
tmp = b1 ^ b4;
- b1 = (tmp >> 25) | (tmp << (64 - 25));
+ b1 = ror64(tmp, 25);
b4 -= b1;
tmp = b3 ^ b0;
- b3 = (tmp >> 17) | (tmp << (64 - 17));
+ b3 = ror64(tmp, 17);
b0 -= b3;
tmp = b5 ^ b6;
- b5 = (tmp >> 10) | (tmp << (64 - 10));
+ b5 = ror64(tmp, 10);
b6 -= b5;
tmp = b7 ^ b4;
- b7 = (tmp >> 50) | (tmp << (64 - 50));
+ b7 = ror64(tmp, 50);
b4 -= b7;
tmp = b1 ^ b2;
- b1 = (tmp >> 13) | (tmp << (64 - 13));
+ b1 = ror64(tmp, 13);
b2 -= b1;
tmp = b7 ^ b6;
- b7 = (tmp >> 24) | (tmp << (64 - 24));
+ b7 = ror64(tmp, 24);
b6 -= b7 + k6 + t1;
b7 -= k7 + 9;
tmp = b5 ^ b4;
- b5 = (tmp >> 34) | (tmp << (64 - 34));
+ b5 = ror64(tmp, 34);
b4 -= b5 + k4;
b5 -= k5 + t0;
tmp = b3 ^ b2;
- b3 = (tmp >> 30) | (tmp << (64 - 30));
+ b3 = ror64(tmp, 30);
b2 -= b3 + k2;
b3 -= k3;
tmp = b1 ^ b0;
- b1 = (tmp >> 39) | (tmp << (64 - 39));
+ b1 = ror64(tmp, 39);
b0 -= b1 + k0;
b1 -= k1;
tmp = b3 ^ b4;
- b3 = (tmp >> 56) | (tmp << (64 - 56));
+ b3 = ror64(tmp, 56);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 54) | (tmp << (64 - 54));
+ b5 = ror64(tmp, 54);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 9) | (tmp << (64 - 9));
+ b7 = ror64(tmp, 9);
b0 -= b7;
tmp = b1 ^ b6;
- b1 = (tmp >> 44) | (tmp << (64 - 44));
+ b1 = ror64(tmp, 44);
b6 -= b1;
tmp = b7 ^ b2;
- b7 = (tmp >> 39) | (tmp << (64 - 39));
+ b7 = ror64(tmp, 39);
b2 -= b7;
tmp = b5 ^ b0;
- b5 = (tmp >> 36) | (tmp << (64 - 36));
+ b5 = ror64(tmp, 36);
b0 -= b5;
tmp = b3 ^ b6;
- b3 = (tmp >> 49) | (tmp << (64 - 49));
+ b3 = ror64(tmp, 49);
b6 -= b3;
tmp = b1 ^ b4;
- b1 = (tmp >> 17) | (tmp << (64 - 17));
+ b1 = ror64(tmp, 17);
b4 -= b1;
tmp = b3 ^ b0;
- b3 = (tmp >> 42) | (tmp << (64 - 42));
+ b3 = ror64(tmp, 42);
b0 -= b3;
tmp = b5 ^ b6;
- b5 = (tmp >> 14) | (tmp << (64 - 14));
+ b5 = ror64(tmp, 14);
b6 -= b5;
tmp = b7 ^ b4;
- b7 = (tmp >> 27) | (tmp << (64 - 27));
+ b7 = ror64(tmp, 27);
b4 -= b7;
tmp = b1 ^ b2;
- b1 = (tmp >> 33) | (tmp << (64 - 33));
+ b1 = ror64(tmp, 33);
b2 -= b1;
tmp = b7 ^ b6;
- b7 = (tmp >> 37) | (tmp << (64 - 37));
+ b7 = ror64(tmp, 37);
b6 -= b7 + k5 + t0;
b7 -= k6 + 8;
tmp = b5 ^ b4;
- b5 = (tmp >> 19) | (tmp << (64 - 19));
+ b5 = ror64(tmp, 19);
b4 -= b5 + k3;
b5 -= k4 + t2;
tmp = b3 ^ b2;
- b3 = (tmp >> 36) | (tmp << (64 - 36));
+ b3 = ror64(tmp, 36);
b2 -= b3 + k1;
b3 -= k2;
tmp = b1 ^ b0;
- b1 = (tmp >> 46) | (tmp << (64 - 46));
+ b1 = ror64(tmp, 46);
b0 -= b1 + k8;
b1 -= k0;
tmp = b3 ^ b4;
- b3 = (tmp >> 22) | (tmp << (64 - 22));
+ b3 = ror64(tmp, 22);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 56) | (tmp << (64 - 56));
+ b5 = ror64(tmp, 56);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 35) | (tmp << (64 - 35));
+ b7 = ror64(tmp, 35);
b0 -= b7;
tmp = b1 ^ b6;
- b1 = (tmp >> 8) | (tmp << (64 - 8));
+ b1 = ror64(tmp, 8);
b6 -= b1;
tmp = b7 ^ b2;
- b7 = (tmp >> 43) | (tmp << (64 - 43));
+ b7 = ror64(tmp, 43);
b2 -= b7;
tmp = b5 ^ b0;
- b5 = (tmp >> 39) | (tmp << (64 - 39));
+ b5 = ror64(tmp, 39);
b0 -= b5;
tmp = b3 ^ b6;
- b3 = (tmp >> 29) | (tmp << (64 - 29));
+ b3 = ror64(tmp, 29);
b6 -= b3;
tmp = b1 ^ b4;
- b1 = (tmp >> 25) | (tmp << (64 - 25));
+ b1 = ror64(tmp, 25);
b4 -= b1;
tmp = b3 ^ b0;
- b3 = (tmp >> 17) | (tmp << (64 - 17));
+ b3 = ror64(tmp, 17);
b0 -= b3;
tmp = b5 ^ b6;
- b5 = (tmp >> 10) | (tmp << (64 - 10));
+ b5 = ror64(tmp, 10);
b6 -= b5;
tmp = b7 ^ b4;
- b7 = (tmp >> 50) | (tmp << (64 - 50));
+ b7 = ror64(tmp, 50);
b4 -= b7;
tmp = b1 ^ b2;
- b1 = (tmp >> 13) | (tmp << (64 - 13));
+ b1 = ror64(tmp, 13);
b2 -= b1;
tmp = b7 ^ b6;
- b7 = (tmp >> 24) | (tmp << (64 - 24));
+ b7 = ror64(tmp, 24);
b6 -= b7 + k4 + t2;
b7 -= k5 + 7;
tmp = b5 ^ b4;
- b5 = (tmp >> 34) | (tmp << (64 - 34));
+ b5 = ror64(tmp, 34);
b4 -= b5 + k2;
b5 -= k3 + t1;
tmp = b3 ^ b2;
- b3 = (tmp >> 30) | (tmp << (64 - 30));
+ b3 = ror64(tmp, 30);
b2 -= b3 + k0;
b3 -= k1;
tmp = b1 ^ b0;
- b1 = (tmp >> 39) | (tmp << (64 - 39));
+ b1 = ror64(tmp, 39);
b0 -= b1 + k7;
b1 -= k8;
tmp = b3 ^ b4;
- b3 = (tmp >> 56) | (tmp << (64 - 56));
+ b3 = ror64(tmp, 56);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 54) | (tmp << (64 - 54));
+ b5 = ror64(tmp, 54);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 9) | (tmp << (64 - 9));
+ b7 = ror64(tmp, 9);
b0 -= b7;
tmp = b1 ^ b6;
- b1 = (tmp >> 44) | (tmp << (64 - 44));
+ b1 = ror64(tmp, 44);
b6 -= b1;
tmp = b7 ^ b2;
- b7 = (tmp >> 39) | (tmp << (64 - 39));
+ b7 = ror64(tmp, 39);
b2 -= b7;
tmp = b5 ^ b0;
- b5 = (tmp >> 36) | (tmp << (64 - 36));
+ b5 = ror64(tmp, 36);
b0 -= b5;
tmp = b3 ^ b6;
- b3 = (tmp >> 49) | (tmp << (64 - 49));
+ b3 = ror64(tmp, 49);
b6 -= b3;
tmp = b1 ^ b4;
- b1 = (tmp >> 17) | (tmp << (64 - 17));
+ b1 = ror64(tmp, 17);
b4 -= b1;
tmp = b3 ^ b0;
- b3 = (tmp >> 42) | (tmp << (64 - 42));
+ b3 = ror64(tmp, 42);
b0 -= b3;
tmp = b5 ^ b6;
- b5 = (tmp >> 14) | (tmp << (64 - 14));
+ b5 = ror64(tmp, 14);
b6 -= b5;
tmp = b7 ^ b4;
- b7 = (tmp >> 27) | (tmp << (64 - 27));
+ b7 = ror64(tmp, 27);
b4 -= b7;
tmp = b1 ^ b2;
- b1 = (tmp >> 33) | (tmp << (64 - 33));
+ b1 = ror64(tmp, 33);
b2 -= b1;
tmp = b7 ^ b6;
- b7 = (tmp >> 37) | (tmp << (64 - 37));
+ b7 = ror64(tmp, 37);
b6 -= b7 + k3 + t1;
b7 -= k4 + 6;
tmp = b5 ^ b4;
- b5 = (tmp >> 19) | (tmp << (64 - 19));
+ b5 = ror64(tmp, 19);
b4 -= b5 + k1;
b5 -= k2 + t0;
tmp = b3 ^ b2;
- b3 = (tmp >> 36) | (tmp << (64 - 36));
+ b3 = ror64(tmp, 36);
b2 -= b3 + k8;
b3 -= k0;
tmp = b1 ^ b0;
- b1 = (tmp >> 46) | (tmp << (64 - 46));
+ b1 = ror64(tmp, 46);
b0 -= b1 + k6;
b1 -= k7;
tmp = b3 ^ b4;
- b3 = (tmp >> 22) | (tmp << (64 - 22));
+ b3 = ror64(tmp, 22);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 56) | (tmp << (64 - 56));
+ b5 = ror64(tmp, 56);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 35) | (tmp << (64 - 35));
+ b7 = ror64(tmp, 35);
b0 -= b7;
tmp = b1 ^ b6;
- b1 = (tmp >> 8) | (tmp << (64 - 8));
+ b1 = ror64(tmp, 8);
b6 -= b1;
tmp = b7 ^ b2;
- b7 = (tmp >> 43) | (tmp << (64 - 43));
+ b7 = ror64(tmp, 43);
b2 -= b7;
tmp = b5 ^ b0;
- b5 = (tmp >> 39) | (tmp << (64 - 39));
+ b5 = ror64(tmp, 39);
b0 -= b5;
tmp = b3 ^ b6;
- b3 = (tmp >> 29) | (tmp << (64 - 29));
+ b3 = ror64(tmp, 29);
b6 -= b3;
tmp = b1 ^ b4;
- b1 = (tmp >> 25) | (tmp << (64 - 25));
+ b1 = ror64(tmp, 25);
b4 -= b1;
tmp = b3 ^ b0;
- b3 = (tmp >> 17) | (tmp << (64 - 17));
+ b3 = ror64(tmp, 17);
b0 -= b3;
tmp = b5 ^ b6;
- b5 = (tmp >> 10) | (tmp << (64 - 10));
+ b5 = ror64(tmp, 10);
b6 -= b5;
tmp = b7 ^ b4;
- b7 = (tmp >> 50) | (tmp << (64 - 50));
+ b7 = ror64(tmp, 50);
b4 -= b7;
tmp = b1 ^ b2;
- b1 = (tmp >> 13) | (tmp << (64 - 13));
+ b1 = ror64(tmp, 13);
b2 -= b1;
tmp = b7 ^ b6;
- b7 = (tmp >> 24) | (tmp << (64 - 24));
+ b7 = ror64(tmp, 24);
b6 -= b7 + k2 + t0;
b7 -= k3 + 5;
tmp = b5 ^ b4;
- b5 = (tmp >> 34) | (tmp << (64 - 34));
+ b5 = ror64(tmp, 34);
b4 -= b5 + k0;
b5 -= k1 + t2;
tmp = b3 ^ b2;
- b3 = (tmp >> 30) | (tmp << (64 - 30));
+ b3 = ror64(tmp, 30);
b2 -= b3 + k7;
b3 -= k8;
tmp = b1 ^ b0;
- b1 = (tmp >> 39) | (tmp << (64 - 39));
+ b1 = ror64(tmp, 39);
b0 -= b1 + k5;
b1 -= k6;
tmp = b3 ^ b4;
- b3 = (tmp >> 56) | (tmp << (64 - 56));
+ b3 = ror64(tmp, 56);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 54) | (tmp << (64 - 54));
+ b5 = ror64(tmp, 54);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 9) | (tmp << (64 - 9));
+ b7 = ror64(tmp, 9);
b0 -= b7;
tmp = b1 ^ b6;
- b1 = (tmp >> 44) | (tmp << (64 - 44));
+ b1 = ror64(tmp, 44);
b6 -= b1;
tmp = b7 ^ b2;
- b7 = (tmp >> 39) | (tmp << (64 - 39));
+ b7 = ror64(tmp, 39);
b2 -= b7;
tmp = b5 ^ b0;
- b5 = (tmp >> 36) | (tmp << (64 - 36));
+ b5 = ror64(tmp, 36);
b0 -= b5;
tmp = b3 ^ b6;
- b3 = (tmp >> 49) | (tmp << (64 - 49));
+ b3 = ror64(tmp, 49);
b6 -= b3;
tmp = b1 ^ b4;
- b1 = (tmp >> 17) | (tmp << (64 - 17));
+ b1 = ror64(tmp, 17);
b4 -= b1;
tmp = b3 ^ b0;
- b3 = (tmp >> 42) | (tmp << (64 - 42));
+ b3 = ror64(tmp, 42);
b0 -= b3;
tmp = b5 ^ b6;
- b5 = (tmp >> 14) | (tmp << (64 - 14));
+ b5 = ror64(tmp, 14);
b6 -= b5;
tmp = b7 ^ b4;
- b7 = (tmp >> 27) | (tmp << (64 - 27));
+ b7 = ror64(tmp, 27);
b4 -= b7;
tmp = b1 ^ b2;
- b1 = (tmp >> 33) | (tmp << (64 - 33));
+ b1 = ror64(tmp, 33);
b2 -= b1;
tmp = b7 ^ b6;
- b7 = (tmp >> 37) | (tmp << (64 - 37));
+ b7 = ror64(tmp, 37);
b6 -= b7 + k1 + t2;
b7 -= k2 + 4;
tmp = b5 ^ b4;
- b5 = (tmp >> 19) | (tmp << (64 - 19));
+ b5 = ror64(tmp, 19);
b4 -= b5 + k8;
b5 -= k0 + t1;
tmp = b3 ^ b2;
- b3 = (tmp >> 36) | (tmp << (64 - 36));
+ b3 = ror64(tmp, 36);
b2 -= b3 + k6;
b3 -= k7;
tmp = b1 ^ b0;
- b1 = (tmp >> 46) | (tmp << (64 - 46));
+ b1 = ror64(tmp, 46);
b0 -= b1 + k4;
b1 -= k5;
tmp = b3 ^ b4;
- b3 = (tmp >> 22) | (tmp << (64 - 22));
+ b3 = ror64(tmp, 22);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 56) | (tmp << (64 - 56));
+ b5 = ror64(tmp, 56);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 35) | (tmp << (64 - 35));
+ b7 = ror64(tmp, 35);
b0 -= b7;
tmp = b1 ^ b6;
- b1 = (tmp >> 8) | (tmp << (64 - 8));
+ b1 = ror64(tmp, 8);
b6 -= b1;
tmp = b7 ^ b2;
- b7 = (tmp >> 43) | (tmp << (64 - 43));
+ b7 = ror64(tmp, 43);
b2 -= b7;
tmp = b5 ^ b0;
- b5 = (tmp >> 39) | (tmp << (64 - 39));
+ b5 = ror64(tmp, 39);
b0 -= b5;
tmp = b3 ^ b6;
- b3 = (tmp >> 29) | (tmp << (64 - 29));
+ b3 = ror64(tmp, 29);
b6 -= b3;
tmp = b1 ^ b4;
- b1 = (tmp >> 25) | (tmp << (64 - 25));
+ b1 = ror64(tmp, 25);
b4 -= b1;
tmp = b3 ^ b0;
- b3 = (tmp >> 17) | (tmp << (64 - 17));
+ b3 = ror64(tmp, 17);
b0 -= b3;
tmp = b5 ^ b6;
- b5 = (tmp >> 10) | (tmp << (64 - 10));
+ b5 = ror64(tmp, 10);
b6 -= b5;
tmp = b7 ^ b4;
- b7 = (tmp >> 50) | (tmp << (64 - 50));
+ b7 = ror64(tmp, 50);
b4 -= b7;
tmp = b1 ^ b2;
- b1 = (tmp >> 13) | (tmp << (64 - 13));
+ b1 = ror64(tmp, 13);
b2 -= b1;
tmp = b7 ^ b6;
- b7 = (tmp >> 24) | (tmp << (64 - 24));
+ b7 = ror64(tmp, 24);
b6 -= b7 + k0 + t1;
b7 -= k1 + 3;
tmp = b5 ^ b4;
- b5 = (tmp >> 34) | (tmp << (64 - 34));
+ b5 = ror64(tmp, 34);
b4 -= b5 + k7;
b5 -= k8 + t0;
tmp = b3 ^ b2;
- b3 = (tmp >> 30) | (tmp << (64 - 30));
+ b3 = ror64(tmp, 30);
b2 -= b3 + k5;
b3 -= k6;
tmp = b1 ^ b0;
- b1 = (tmp >> 39) | (tmp << (64 - 39));
+ b1 = ror64(tmp, 39);
b0 -= b1 + k3;
b1 -= k4;
tmp = b3 ^ b4;
- b3 = (tmp >> 56) | (tmp << (64 - 56));
+ b3 = ror64(tmp, 56);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 54) | (tmp << (64 - 54));
+ b5 = ror64(tmp, 54);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 9) | (tmp << (64 - 9));
+ b7 = ror64(tmp, 9);
b0 -= b7;
tmp = b1 ^ b6;
- b1 = (tmp >> 44) | (tmp << (64 - 44));
+ b1 = ror64(tmp, 44);
b6 -= b1;
tmp = b7 ^ b2;
- b7 = (tmp >> 39) | (tmp << (64 - 39));
+ b7 = ror64(tmp, 39);
b2 -= b7;
tmp = b5 ^ b0;
- b5 = (tmp >> 36) | (tmp << (64 - 36));
+ b5 = ror64(tmp, 36);
b0 -= b5;
tmp = b3 ^ b6;
- b3 = (tmp >> 49) | (tmp << (64 - 49));
+ b3 = ror64(tmp, 49);
b6 -= b3;
tmp = b1 ^ b4;
- b1 = (tmp >> 17) | (tmp << (64 - 17));
+ b1 = ror64(tmp, 17);
b4 -= b1;
tmp = b3 ^ b0;
- b3 = (tmp >> 42) | (tmp << (64 - 42));
+ b3 = ror64(tmp, 42);
b0 -= b3;
tmp = b5 ^ b6;
- b5 = (tmp >> 14) | (tmp << (64 - 14));
+ b5 = ror64(tmp, 14);
b6 -= b5;
tmp = b7 ^ b4;
- b7 = (tmp >> 27) | (tmp << (64 - 27));
+ b7 = ror64(tmp, 27);
b4 -= b7;
tmp = b1 ^ b2;
- b1 = (tmp >> 33) | (tmp << (64 - 33));
+ b1 = ror64(tmp, 33);
b2 -= b1;
tmp = b7 ^ b6;
- b7 = (tmp >> 37) | (tmp << (64 - 37));
+ b7 = ror64(tmp, 37);
b6 -= b7 + k8 + t0;
b7 -= k0 + 2;
tmp = b5 ^ b4;
- b5 = (tmp >> 19) | (tmp << (64 - 19));
+ b5 = ror64(tmp, 19);
b4 -= b5 + k6;
b5 -= k7 + t2;
tmp = b3 ^ b2;
- b3 = (tmp >> 36) | (tmp << (64 - 36));
+ b3 = ror64(tmp, 36);
b2 -= b3 + k4;
b3 -= k5;
tmp = b1 ^ b0;
- b1 = (tmp >> 46) | (tmp << (64 - 46));
+ b1 = ror64(tmp, 46);
b0 -= b1 + k2;
b1 -= k3;
tmp = b3 ^ b4;
- b3 = (tmp >> 22) | (tmp << (64 - 22));
+ b3 = ror64(tmp, 22);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 56) | (tmp << (64 - 56));
+ b5 = ror64(tmp, 56);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 35) | (tmp << (64 - 35));
+ b7 = ror64(tmp, 35);
b0 -= b7;
tmp = b1 ^ b6;
- b1 = (tmp >> 8) | (tmp << (64 - 8));
+ b1 = ror64(tmp, 8);
b6 -= b1;
tmp = b7 ^ b2;
- b7 = (tmp >> 43) | (tmp << (64 - 43));
+ b7 = ror64(tmp, 43);
b2 -= b7;
tmp = b5 ^ b0;
- b5 = (tmp >> 39) | (tmp << (64 - 39));
+ b5 = ror64(tmp, 39);
b0 -= b5;
tmp = b3 ^ b6;
- b3 = (tmp >> 29) | (tmp << (64 - 29));
+ b3 = ror64(tmp, 29);
b6 -= b3;
tmp = b1 ^ b4;
- b1 = (tmp >> 25) | (tmp << (64 - 25));
+ b1 = ror64(tmp, 25);
b4 -= b1;
tmp = b3 ^ b0;
- b3 = (tmp >> 17) | (tmp << (64 - 17));
+ b3 = ror64(tmp, 17);
b0 -= b3;
tmp = b5 ^ b6;
- b5 = (tmp >> 10) | (tmp << (64 - 10));
+ b5 = ror64(tmp, 10);
b6 -= b5;
tmp = b7 ^ b4;
- b7 = (tmp >> 50) | (tmp << (64 - 50));
+ b7 = ror64(tmp, 50);
b4 -= b7;
tmp = b1 ^ b2;
- b1 = (tmp >> 13) | (tmp << (64 - 13));
+ b1 = ror64(tmp, 13);
b2 -= b1;
tmp = b7 ^ b6;
- b7 = (tmp >> 24) | (tmp << (64 - 24));
+ b7 = ror64(tmp, 24);
b6 -= b7 + k7 + t2;
b7 -= k8 + 1;
tmp = b5 ^ b4;
- b5 = (tmp >> 34) | (tmp << (64 - 34));
+ b5 = ror64(tmp, 34);
b4 -= b5 + k5;
b5 -= k6 + t1;
tmp = b3 ^ b2;
- b3 = (tmp >> 30) | (tmp << (64 - 30));
+ b3 = ror64(tmp, 30);
b2 -= b3 + k3;
b3 -= k4;
tmp = b1 ^ b0;
- b1 = (tmp >> 39) | (tmp << (64 - 39));
+ b1 = ror64(tmp, 39);
b0 -= b1 + k1;
b1 -= k2;
tmp = b3 ^ b4;
- b3 = (tmp >> 56) | (tmp << (64 - 56));
+ b3 = ror64(tmp, 56);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 54) | (tmp << (64 - 54));
+ b5 = ror64(tmp, 54);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 9) | (tmp << (64 - 9));
+ b7 = ror64(tmp, 9);
b0 -= b7;
tmp = b1 ^ b6;
- b1 = (tmp >> 44) | (tmp << (64 - 44));
+ b1 = ror64(tmp, 44);
b6 -= b1;
tmp = b7 ^ b2;
- b7 = (tmp >> 39) | (tmp << (64 - 39));
+ b7 = ror64(tmp, 39);
b2 -= b7;
tmp = b5 ^ b0;
- b5 = (tmp >> 36) | (tmp << (64 - 36));
+ b5 = ror64(tmp, 36);
b0 -= b5;
tmp = b3 ^ b6;
- b3 = (tmp >> 49) | (tmp << (64 - 49));
+ b3 = ror64(tmp, 49);
b6 -= b3;
tmp = b1 ^ b4;
- b1 = (tmp >> 17) | (tmp << (64 - 17));
+ b1 = ror64(tmp, 17);
b4 -= b1;
tmp = b3 ^ b0;
- b3 = (tmp >> 42) | (tmp << (64 - 42));
+ b3 = ror64(tmp, 42);
b0 -= b3;
tmp = b5 ^ b6;
- b5 = (tmp >> 14) | (tmp << (64 - 14));
+ b5 = ror64(tmp, 14);
b6 -= b5;
tmp = b7 ^ b4;
- b7 = (tmp >> 27) | (tmp << (64 - 27));
+ b7 = ror64(tmp, 27);
b4 -= b7;
tmp = b1 ^ b2;
- b1 = (tmp >> 33) | (tmp << (64 - 33));
+ b1 = ror64(tmp, 33);
b2 -= b1;
tmp = b7 ^ b6;
- b7 = (tmp >> 37) | (tmp << (64 - 37));
+ b7 = ror64(tmp, 37);
b6 -= b7 + k6 + t1;
b7 -= k7;
tmp = b5 ^ b4;
- b5 = (tmp >> 19) | (tmp << (64 - 19));
+ b5 = ror64(tmp, 19);
b4 -= b5 + k4;
b5 -= k5 + t0;
tmp = b3 ^ b2;
- b3 = (tmp >> 36) | (tmp << (64 - 36));
+ b3 = ror64(tmp, 36);
b2 -= b3 + k2;
b3 -= k3;
tmp = b1 ^ b0;
- b1 = (tmp >> 46) | (tmp << (64 - 46));
+ b1 = ror64(tmp, 46);
b0 -= b1 + k0;
b1 -= k1;
@@ -5521,2722 +5521,2722 @@
b14 -= k0 + t0;
b15 -= k1 + 20;
tmp = b7 ^ b12;
- b7 = (tmp >> 20) | (tmp << (64 - 20));
+ b7 = ror64(tmp, 20);
b12 -= b7;
tmp = b3 ^ b10;
- b3 = (tmp >> 37) | (tmp << (64 - 37));
+ b3 = ror64(tmp, 37);
b10 -= b3;
tmp = b5 ^ b8;
- b5 = (tmp >> 31) | (tmp << (64 - 31));
+ b5 = ror64(tmp, 31);
b8 -= b5;
tmp = b1 ^ b14;
- b1 = (tmp >> 23) | (tmp << (64 - 23));
+ b1 = ror64(tmp, 23);
b14 -= b1;
tmp = b9 ^ b4;
- b9 = (tmp >> 52) | (tmp << (64 - 52));
+ b9 = ror64(tmp, 52);
b4 -= b9;
tmp = b13 ^ b6;
- b13 = (tmp >> 35) | (tmp << (64 - 35));
+ b13 = ror64(tmp, 35);
b6 -= b13;
tmp = b11 ^ b2;
- b11 = (tmp >> 48) | (tmp << (64 - 48));
+ b11 = ror64(tmp, 48);
b2 -= b11;
tmp = b15 ^ b0;
- b15 = (tmp >> 9) | (tmp << (64 - 9));
+ b15 = ror64(tmp, 9);
b0 -= b15;
tmp = b9 ^ b10;
- b9 = (tmp >> 25) | (tmp << (64 - 25));
+ b9 = ror64(tmp, 25);
b10 -= b9;
tmp = b11 ^ b8;
- b11 = (tmp >> 44) | (tmp << (64 - 44));
+ b11 = ror64(tmp, 44);
b8 -= b11;
tmp = b13 ^ b14;
- b13 = (tmp >> 42) | (tmp << (64 - 42));
+ b13 = ror64(tmp, 42);
b14 -= b13;
tmp = b15 ^ b12;
- b15 = (tmp >> 19) | (tmp << (64 - 19));
+ b15 = ror64(tmp, 19);
b12 -= b15;
tmp = b1 ^ b6;
- b1 = (tmp >> 46) | (tmp << (64 - 46));
+ b1 = ror64(tmp, 46);
b6 -= b1;
tmp = b3 ^ b4;
- b3 = (tmp >> 47) | (tmp << (64 - 47));
+ b3 = ror64(tmp, 47);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 44) | (tmp << (64 - 44));
+ b5 = ror64(tmp, 44);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 31) | (tmp << (64 - 31));
+ b7 = ror64(tmp, 31);
b0 -= b7;
tmp = b1 ^ b8;
- b1 = (tmp >> 41) | (tmp << (64 - 41));
+ b1 = ror64(tmp, 41);
b8 -= b1;
tmp = b5 ^ b14;
- b5 = (tmp >> 42) | (tmp << (64 - 42));
+ b5 = ror64(tmp, 42);
b14 -= b5;
tmp = b3 ^ b12;
- b3 = (tmp >> 53) | (tmp << (64 - 53));
+ b3 = ror64(tmp, 53);
b12 -= b3;
tmp = b7 ^ b10;
- b7 = (tmp >> 4) | (tmp << (64 - 4));
+ b7 = ror64(tmp, 4);
b10 -= b7;
tmp = b15 ^ b4;
- b15 = (tmp >> 51) | (tmp << (64 - 51));
+ b15 = ror64(tmp, 51);
b4 -= b15;
tmp = b11 ^ b6;
- b11 = (tmp >> 56) | (tmp << (64 - 56));
+ b11 = ror64(tmp, 56);
b6 -= b11;
tmp = b13 ^ b2;
- b13 = (tmp >> 34) | (tmp << (64 - 34));
+ b13 = ror64(tmp, 34);
b2 -= b13;
tmp = b9 ^ b0;
- b9 = (tmp >> 16) | (tmp << (64 - 16));
+ b9 = ror64(tmp, 16);
b0 -= b9;
tmp = b15 ^ b14;
- b15 = (tmp >> 30) | (tmp << (64 - 30));
+ b15 = ror64(tmp, 30);
b14 -= b15 + k16 + t2;
b15 -= k0 + 19;
tmp = b13 ^ b12;
- b13 = (tmp >> 44) | (tmp << (64 - 44));
+ b13 = ror64(tmp, 44);
b12 -= b13 + k14;
b13 -= k15 + t1;
tmp = b11 ^ b10;
- b11 = (tmp >> 47) | (tmp << (64 - 47));
+ b11 = ror64(tmp, 47);
b10 -= b11 + k12;
b11 -= k13;
tmp = b9 ^ b8;
- b9 = (tmp >> 12) | (tmp << (64 - 12));
+ b9 = ror64(tmp, 12);
b8 -= b9 + k10;
b9 -= k11;
tmp = b7 ^ b6;
- b7 = (tmp >> 31) | (tmp << (64 - 31));
+ b7 = ror64(tmp, 31);
b6 -= b7 + k8;
b7 -= k9;
tmp = b5 ^ b4;
- b5 = (tmp >> 37) | (tmp << (64 - 37));
+ b5 = ror64(tmp, 37);
b4 -= b5 + k6;
b5 -= k7;
tmp = b3 ^ b2;
- b3 = (tmp >> 9) | (tmp << (64 - 9));
+ b3 = ror64(tmp, 9);
b2 -= b3 + k4;
b3 -= k5;
tmp = b1 ^ b0;
- b1 = (tmp >> 41) | (tmp << (64 - 41));
+ b1 = ror64(tmp, 41);
b0 -= b1 + k2;
b1 -= k3;
tmp = b7 ^ b12;
- b7 = (tmp >> 25) | (tmp << (64 - 25));
+ b7 = ror64(tmp, 25);
b12 -= b7;
tmp = b3 ^ b10;
- b3 = (tmp >> 16) | (tmp << (64 - 16));
+ b3 = ror64(tmp, 16);
b10 -= b3;
tmp = b5 ^ b8;
- b5 = (tmp >> 28) | (tmp << (64 - 28));
+ b5 = ror64(tmp, 28);
b8 -= b5;
tmp = b1 ^ b14;
- b1 = (tmp >> 47) | (tmp << (64 - 47));
+ b1 = ror64(tmp, 47);
b14 -= b1;
tmp = b9 ^ b4;
- b9 = (tmp >> 41) | (tmp << (64 - 41));
+ b9 = ror64(tmp, 41);
b4 -= b9;
tmp = b13 ^ b6;
- b13 = (tmp >> 48) | (tmp << (64 - 48));
+ b13 = ror64(tmp, 48);
b6 -= b13;
tmp = b11 ^ b2;
- b11 = (tmp >> 20) | (tmp << (64 - 20));
+ b11 = ror64(tmp, 20);
b2 -= b11;
tmp = b15 ^ b0;
- b15 = (tmp >> 5) | (tmp << (64 - 5));
+ b15 = ror64(tmp, 5);
b0 -= b15;
tmp = b9 ^ b10;
- b9 = (tmp >> 17) | (tmp << (64 - 17));
+ b9 = ror64(tmp, 17);
b10 -= b9;
tmp = b11 ^ b8;
- b11 = (tmp >> 59) | (tmp << (64 - 59));
+ b11 = ror64(tmp, 59);
b8 -= b11;
tmp = b13 ^ b14;
- b13 = (tmp >> 41) | (tmp << (64 - 41));
+ b13 = ror64(tmp, 41);
b14 -= b13;
tmp = b15 ^ b12;
- b15 = (tmp >> 34) | (tmp << (64 - 34));
+ b15 = ror64(tmp, 34);
b12 -= b15;
tmp = b1 ^ b6;
- b1 = (tmp >> 13) | (tmp << (64 - 13));
+ b1 = ror64(tmp, 13);
b6 -= b1;
tmp = b3 ^ b4;
- b3 = (tmp >> 51) | (tmp << (64 - 51));
+ b3 = ror64(tmp, 51);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 4) | (tmp << (64 - 4));
+ b5 = ror64(tmp, 4);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 33) | (tmp << (64 - 33));
+ b7 = ror64(tmp, 33);
b0 -= b7;
tmp = b1 ^ b8;
- b1 = (tmp >> 52) | (tmp << (64 - 52));
+ b1 = ror64(tmp, 52);
b8 -= b1;
tmp = b5 ^ b14;
- b5 = (tmp >> 23) | (tmp << (64 - 23));
+ b5 = ror64(tmp, 23);
b14 -= b5;
tmp = b3 ^ b12;
- b3 = (tmp >> 18) | (tmp << (64 - 18));
+ b3 = ror64(tmp, 18);
b12 -= b3;
tmp = b7 ^ b10;
- b7 = (tmp >> 49) | (tmp << (64 - 49));
+ b7 = ror64(tmp, 49);
b10 -= b7;
tmp = b15 ^ b4;
- b15 = (tmp >> 55) | (tmp << (64 - 55));
+ b15 = ror64(tmp, 55);
b4 -= b15;
tmp = b11 ^ b6;
- b11 = (tmp >> 10) | (tmp << (64 - 10));
+ b11 = ror64(tmp, 10);
b6 -= b11;
tmp = b13 ^ b2;
- b13 = (tmp >> 19) | (tmp << (64 - 19));
+ b13 = ror64(tmp, 19);
b2 -= b13;
tmp = b9 ^ b0;
- b9 = (tmp >> 38) | (tmp << (64 - 38));
+ b9 = ror64(tmp, 38);
b0 -= b9;
tmp = b15 ^ b14;
- b15 = (tmp >> 37) | (tmp << (64 - 37));
+ b15 = ror64(tmp, 37);
b14 -= b15 + k15 + t1;
b15 -= k16 + 18;
tmp = b13 ^ b12;
- b13 = (tmp >> 22) | (tmp << (64 - 22));
+ b13 = ror64(tmp, 22);
b12 -= b13 + k13;
b13 -= k14 + t0;
tmp = b11 ^ b10;
- b11 = (tmp >> 17) | (tmp << (64 - 17));
+ b11 = ror64(tmp, 17);
b10 -= b11 + k11;
b11 -= k12;
tmp = b9 ^ b8;
- b9 = (tmp >> 8) | (tmp << (64 - 8));
+ b9 = ror64(tmp, 8);
b8 -= b9 + k9;
b9 -= k10;
tmp = b7 ^ b6;
- b7 = (tmp >> 47) | (tmp << (64 - 47));
+ b7 = ror64(tmp, 47);
b6 -= b7 + k7;
b7 -= k8;
tmp = b5 ^ b4;
- b5 = (tmp >> 8) | (tmp << (64 - 8));
+ b5 = ror64(tmp, 8);
b4 -= b5 + k5;
b5 -= k6;
tmp = b3 ^ b2;
- b3 = (tmp >> 13) | (tmp << (64 - 13));
+ b3 = ror64(tmp, 13);
b2 -= b3 + k3;
b3 -= k4;
tmp = b1 ^ b0;
- b1 = (tmp >> 24) | (tmp << (64 - 24));
+ b1 = ror64(tmp, 24);
b0 -= b1 + k1;
b1 -= k2;
tmp = b7 ^ b12;
- b7 = (tmp >> 20) | (tmp << (64 - 20));
+ b7 = ror64(tmp, 20);
b12 -= b7;
tmp = b3 ^ b10;
- b3 = (tmp >> 37) | (tmp << (64 - 37));
+ b3 = ror64(tmp, 37);
b10 -= b3;
tmp = b5 ^ b8;
- b5 = (tmp >> 31) | (tmp << (64 - 31));
+ b5 = ror64(tmp, 31);
b8 -= b5;
tmp = b1 ^ b14;
- b1 = (tmp >> 23) | (tmp << (64 - 23));
+ b1 = ror64(tmp, 23);
b14 -= b1;
tmp = b9 ^ b4;
- b9 = (tmp >> 52) | (tmp << (64 - 52));
+ b9 = ror64(tmp, 52);
b4 -= b9;
tmp = b13 ^ b6;
- b13 = (tmp >> 35) | (tmp << (64 - 35));
+ b13 = ror64(tmp, 35);
b6 -= b13;
tmp = b11 ^ b2;
- b11 = (tmp >> 48) | (tmp << (64 - 48));
+ b11 = ror64(tmp, 48);
b2 -= b11;
tmp = b15 ^ b0;
- b15 = (tmp >> 9) | (tmp << (64 - 9));
+ b15 = ror64(tmp, 9);
b0 -= b15;
tmp = b9 ^ b10;
- b9 = (tmp >> 25) | (tmp << (64 - 25));
+ b9 = ror64(tmp, 25);
b10 -= b9;
tmp = b11 ^ b8;
- b11 = (tmp >> 44) | (tmp << (64 - 44));
+ b11 = ror64(tmp, 44);
b8 -= b11;
tmp = b13 ^ b14;
- b13 = (tmp >> 42) | (tmp << (64 - 42));
+ b13 = ror64(tmp, 42);
b14 -= b13;
tmp = b15 ^ b12;
- b15 = (tmp >> 19) | (tmp << (64 - 19));
+ b15 = ror64(tmp, 19);
b12 -= b15;
tmp = b1 ^ b6;
- b1 = (tmp >> 46) | (tmp << (64 - 46));
+ b1 = ror64(tmp, 46);
b6 -= b1;
tmp = b3 ^ b4;
- b3 = (tmp >> 47) | (tmp << (64 - 47));
+ b3 = ror64(tmp, 47);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 44) | (tmp << (64 - 44));
+ b5 = ror64(tmp, 44);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 31) | (tmp << (64 - 31));
+ b7 = ror64(tmp, 31);
b0 -= b7;
tmp = b1 ^ b8;
- b1 = (tmp >> 41) | (tmp << (64 - 41));
+ b1 = ror64(tmp, 41);
b8 -= b1;
tmp = b5 ^ b14;
- b5 = (tmp >> 42) | (tmp << (64 - 42));
+ b5 = ror64(tmp, 42);
b14 -= b5;
tmp = b3 ^ b12;
- b3 = (tmp >> 53) | (tmp << (64 - 53));
+ b3 = ror64(tmp, 53);
b12 -= b3;
tmp = b7 ^ b10;
- b7 = (tmp >> 4) | (tmp << (64 - 4));
+ b7 = ror64(tmp, 4);
b10 -= b7;
tmp = b15 ^ b4;
- b15 = (tmp >> 51) | (tmp << (64 - 51));
+ b15 = ror64(tmp, 51);
b4 -= b15;
tmp = b11 ^ b6;
- b11 = (tmp >> 56) | (tmp << (64 - 56));
+ b11 = ror64(tmp, 56);
b6 -= b11;
tmp = b13 ^ b2;
- b13 = (tmp >> 34) | (tmp << (64 - 34));
+ b13 = ror64(tmp, 34);
b2 -= b13;
tmp = b9 ^ b0;
- b9 = (tmp >> 16) | (tmp << (64 - 16));
+ b9 = ror64(tmp, 16);
b0 -= b9;
tmp = b15 ^ b14;
- b15 = (tmp >> 30) | (tmp << (64 - 30));
+ b15 = ror64(tmp, 30);
b14 -= b15 + k14 + t0;
b15 -= k15 + 17;
tmp = b13 ^ b12;
- b13 = (tmp >> 44) | (tmp << (64 - 44));
+ b13 = ror64(tmp, 44);
b12 -= b13 + k12;
b13 -= k13 + t2;
tmp = b11 ^ b10;
- b11 = (tmp >> 47) | (tmp << (64 - 47));
+ b11 = ror64(tmp, 47);
b10 -= b11 + k10;
b11 -= k11;
tmp = b9 ^ b8;
- b9 = (tmp >> 12) | (tmp << (64 - 12));
+ b9 = ror64(tmp, 12);
b8 -= b9 + k8;
b9 -= k9;
tmp = b7 ^ b6;
- b7 = (tmp >> 31) | (tmp << (64 - 31));
+ b7 = ror64(tmp, 31);
b6 -= b7 + k6;
b7 -= k7;
tmp = b5 ^ b4;
- b5 = (tmp >> 37) | (tmp << (64 - 37));
+ b5 = ror64(tmp, 37);
b4 -= b5 + k4;
b5 -= k5;
tmp = b3 ^ b2;
- b3 = (tmp >> 9) | (tmp << (64 - 9));
+ b3 = ror64(tmp, 9);
b2 -= b3 + k2;
b3 -= k3;
tmp = b1 ^ b0;
- b1 = (tmp >> 41) | (tmp << (64 - 41));
+ b1 = ror64(tmp, 41);
b0 -= b1 + k0;
b1 -= k1;
tmp = b7 ^ b12;
- b7 = (tmp >> 25) | (tmp << (64 - 25));
+ b7 = ror64(tmp, 25);
b12 -= b7;
tmp = b3 ^ b10;
- b3 = (tmp >> 16) | (tmp << (64 - 16));
+ b3 = ror64(tmp, 16);
b10 -= b3;
tmp = b5 ^ b8;
- b5 = (tmp >> 28) | (tmp << (64 - 28));
+ b5 = ror64(tmp, 28);
b8 -= b5;
tmp = b1 ^ b14;
- b1 = (tmp >> 47) | (tmp << (64 - 47));
+ b1 = ror64(tmp, 47);
b14 -= b1;
tmp = b9 ^ b4;
- b9 = (tmp >> 41) | (tmp << (64 - 41));
+ b9 = ror64(tmp, 41);
b4 -= b9;
tmp = b13 ^ b6;
- b13 = (tmp >> 48) | (tmp << (64 - 48));
+ b13 = ror64(tmp, 48);
b6 -= b13;
tmp = b11 ^ b2;
- b11 = (tmp >> 20) | (tmp << (64 - 20));
+ b11 = ror64(tmp, 20);
b2 -= b11;
tmp = b15 ^ b0;
- b15 = (tmp >> 5) | (tmp << (64 - 5));
+ b15 = ror64(tmp, 5);
b0 -= b15;
tmp = b9 ^ b10;
- b9 = (tmp >> 17) | (tmp << (64 - 17));
+ b9 = ror64(tmp, 17);
b10 -= b9;
tmp = b11 ^ b8;
- b11 = (tmp >> 59) | (tmp << (64 - 59));
+ b11 = ror64(tmp, 59);
b8 -= b11;
tmp = b13 ^ b14;
- b13 = (tmp >> 41) | (tmp << (64 - 41));
+ b13 = ror64(tmp, 41);
b14 -= b13;
tmp = b15 ^ b12;
- b15 = (tmp >> 34) | (tmp << (64 - 34));
+ b15 = ror64(tmp, 34);
b12 -= b15;
tmp = b1 ^ b6;
- b1 = (tmp >> 13) | (tmp << (64 - 13));
+ b1 = ror64(tmp, 13);
b6 -= b1;
tmp = b3 ^ b4;
- b3 = (tmp >> 51) | (tmp << (64 - 51));
+ b3 = ror64(tmp, 51);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 4) | (tmp << (64 - 4));
+ b5 = ror64(tmp, 4);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 33) | (tmp << (64 - 33));
+ b7 = ror64(tmp, 33);
b0 -= b7;
tmp = b1 ^ b8;
- b1 = (tmp >> 52) | (tmp << (64 - 52));
+ b1 = ror64(tmp, 52);
b8 -= b1;
tmp = b5 ^ b14;
- b5 = (tmp >> 23) | (tmp << (64 - 23));
+ b5 = ror64(tmp, 23);
b14 -= b5;
tmp = b3 ^ b12;
- b3 = (tmp >> 18) | (tmp << (64 - 18));
+ b3 = ror64(tmp, 18);
b12 -= b3;
tmp = b7 ^ b10;
- b7 = (tmp >> 49) | (tmp << (64 - 49));
+ b7 = ror64(tmp, 49);
b10 -= b7;
tmp = b15 ^ b4;
- b15 = (tmp >> 55) | (tmp << (64 - 55));
+ b15 = ror64(tmp, 55);
b4 -= b15;
tmp = b11 ^ b6;
- b11 = (tmp >> 10) | (tmp << (64 - 10));
+ b11 = ror64(tmp, 10);
b6 -= b11;
tmp = b13 ^ b2;
- b13 = (tmp >> 19) | (tmp << (64 - 19));
+ b13 = ror64(tmp, 19);
b2 -= b13;
tmp = b9 ^ b0;
- b9 = (tmp >> 38) | (tmp << (64 - 38));
+ b9 = ror64(tmp, 38);
b0 -= b9;
tmp = b15 ^ b14;
- b15 = (tmp >> 37) | (tmp << (64 - 37));
+ b15 = ror64(tmp, 37);
b14 -= b15 + k13 + t2;
b15 -= k14 + 16;
tmp = b13 ^ b12;
- b13 = (tmp >> 22) | (tmp << (64 - 22));
+ b13 = ror64(tmp, 22);
b12 -= b13 + k11;
b13 -= k12 + t1;
tmp = b11 ^ b10;
- b11 = (tmp >> 17) | (tmp << (64 - 17));
+ b11 = ror64(tmp, 17);
b10 -= b11 + k9;
b11 -= k10;
tmp = b9 ^ b8;
- b9 = (tmp >> 8) | (tmp << (64 - 8));
+ b9 = ror64(tmp, 8);
b8 -= b9 + k7;
b9 -= k8;
tmp = b7 ^ b6;
- b7 = (tmp >> 47) | (tmp << (64 - 47));
+ b7 = ror64(tmp, 47);
b6 -= b7 + k5;
b7 -= k6;
tmp = b5 ^ b4;
- b5 = (tmp >> 8) | (tmp << (64 - 8));
+ b5 = ror64(tmp, 8);
b4 -= b5 + k3;
b5 -= k4;
tmp = b3 ^ b2;
- b3 = (tmp >> 13) | (tmp << (64 - 13));
+ b3 = ror64(tmp, 13);
b2 -= b3 + k1;
b3 -= k2;
tmp = b1 ^ b0;
- b1 = (tmp >> 24) | (tmp << (64 - 24));
+ b1 = ror64(tmp, 24);
b0 -= b1 + k16;
b1 -= k0;
tmp = b7 ^ b12;
- b7 = (tmp >> 20) | (tmp << (64 - 20));
+ b7 = ror64(tmp, 20);
b12 -= b7;
tmp = b3 ^ b10;
- b3 = (tmp >> 37) | (tmp << (64 - 37));
+ b3 = ror64(tmp, 37);
b10 -= b3;
tmp = b5 ^ b8;
- b5 = (tmp >> 31) | (tmp << (64 - 31));
+ b5 = ror64(tmp, 31);
b8 -= b5;
tmp = b1 ^ b14;
- b1 = (tmp >> 23) | (tmp << (64 - 23));
+ b1 = ror64(tmp, 23);
b14 -= b1;
tmp = b9 ^ b4;
- b9 = (tmp >> 52) | (tmp << (64 - 52));
+ b9 = ror64(tmp, 52);
b4 -= b9;
tmp = b13 ^ b6;
- b13 = (tmp >> 35) | (tmp << (64 - 35));
+ b13 = ror64(tmp, 35);
b6 -= b13;
tmp = b11 ^ b2;
- b11 = (tmp >> 48) | (tmp << (64 - 48));
+ b11 = ror64(tmp, 48);
b2 -= b11;
tmp = b15 ^ b0;
- b15 = (tmp >> 9) | (tmp << (64 - 9));
+ b15 = ror64(tmp, 9);
b0 -= b15;
tmp = b9 ^ b10;
- b9 = (tmp >> 25) | (tmp << (64 - 25));
+ b9 = ror64(tmp, 25);
b10 -= b9;
tmp = b11 ^ b8;
- b11 = (tmp >> 44) | (tmp << (64 - 44));
+ b11 = ror64(tmp, 44);
b8 -= b11;
tmp = b13 ^ b14;
- b13 = (tmp >> 42) | (tmp << (64 - 42));
+ b13 = ror64(tmp, 42);
b14 -= b13;
tmp = b15 ^ b12;
- b15 = (tmp >> 19) | (tmp << (64 - 19));
+ b15 = ror64(tmp, 19);
b12 -= b15;
tmp = b1 ^ b6;
- b1 = (tmp >> 46) | (tmp << (64 - 46));
+ b1 = ror64(tmp, 46);
b6 -= b1;
tmp = b3 ^ b4;
- b3 = (tmp >> 47) | (tmp << (64 - 47));
+ b3 = ror64(tmp, 47);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 44) | (tmp << (64 - 44));
+ b5 = ror64(tmp, 44);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 31) | (tmp << (64 - 31));
+ b7 = ror64(tmp, 31);
b0 -= b7;
tmp = b1 ^ b8;
- b1 = (tmp >> 41) | (tmp << (64 - 41));
+ b1 = ror64(tmp, 41);
b8 -= b1;
tmp = b5 ^ b14;
- b5 = (tmp >> 42) | (tmp << (64 - 42));
+ b5 = ror64(tmp, 42);
b14 -= b5;
tmp = b3 ^ b12;
- b3 = (tmp >> 53) | (tmp << (64 - 53));
+ b3 = ror64(tmp, 53);
b12 -= b3;
tmp = b7 ^ b10;
- b7 = (tmp >> 4) | (tmp << (64 - 4));
+ b7 = ror64(tmp, 4);
b10 -= b7;
tmp = b15 ^ b4;
- b15 = (tmp >> 51) | (tmp << (64 - 51));
+ b15 = ror64(tmp, 51);
b4 -= b15;
tmp = b11 ^ b6;
- b11 = (tmp >> 56) | (tmp << (64 - 56));
+ b11 = ror64(tmp, 56);
b6 -= b11;
tmp = b13 ^ b2;
- b13 = (tmp >> 34) | (tmp << (64 - 34));
+ b13 = ror64(tmp, 34);
b2 -= b13;
tmp = b9 ^ b0;
- b9 = (tmp >> 16) | (tmp << (64 - 16));
+ b9 = ror64(tmp, 16);
b0 -= b9;
tmp = b15 ^ b14;
- b15 = (tmp >> 30) | (tmp << (64 - 30));
+ b15 = ror64(tmp, 30);
b14 -= b15 + k12 + t1;
b15 -= k13 + 15;
tmp = b13 ^ b12;
- b13 = (tmp >> 44) | (tmp << (64 - 44));
+ b13 = ror64(tmp, 44);
b12 -= b13 + k10;
b13 -= k11 + t0;
tmp = b11 ^ b10;
- b11 = (tmp >> 47) | (tmp << (64 - 47));
+ b11 = ror64(tmp, 47);
b10 -= b11 + k8;
b11 -= k9;
tmp = b9 ^ b8;
- b9 = (tmp >> 12) | (tmp << (64 - 12));
+ b9 = ror64(tmp, 12);
b8 -= b9 + k6;
b9 -= k7;
tmp = b7 ^ b6;
- b7 = (tmp >> 31) | (tmp << (64 - 31));
+ b7 = ror64(tmp, 31);
b6 -= b7 + k4;
b7 -= k5;
tmp = b5 ^ b4;
- b5 = (tmp >> 37) | (tmp << (64 - 37));
+ b5 = ror64(tmp, 37);
b4 -= b5 + k2;
b5 -= k3;
tmp = b3 ^ b2;
- b3 = (tmp >> 9) | (tmp << (64 - 9));
+ b3 = ror64(tmp, 9);
b2 -= b3 + k0;
b3 -= k1;
tmp = b1 ^ b0;
- b1 = (tmp >> 41) | (tmp << (64 - 41));
+ b1 = ror64(tmp, 41);
b0 -= b1 + k15;
b1 -= k16;
tmp = b7 ^ b12;
- b7 = (tmp >> 25) | (tmp << (64 - 25));
+ b7 = ror64(tmp, 25);
b12 -= b7;
tmp = b3 ^ b10;
- b3 = (tmp >> 16) | (tmp << (64 - 16));
+ b3 = ror64(tmp, 16);
b10 -= b3;
tmp = b5 ^ b8;
- b5 = (tmp >> 28) | (tmp << (64 - 28));
+ b5 = ror64(tmp, 28);
b8 -= b5;
tmp = b1 ^ b14;
- b1 = (tmp >> 47) | (tmp << (64 - 47));
+ b1 = ror64(tmp, 47);
b14 -= b1;
tmp = b9 ^ b4;
- b9 = (tmp >> 41) | (tmp << (64 - 41));
+ b9 = ror64(tmp, 41);
b4 -= b9;
tmp = b13 ^ b6;
- b13 = (tmp >> 48) | (tmp << (64 - 48));
+ b13 = ror64(tmp, 48);
b6 -= b13;
tmp = b11 ^ b2;
- b11 = (tmp >> 20) | (tmp << (64 - 20));
+ b11 = ror64(tmp, 20);
b2 -= b11;
tmp = b15 ^ b0;
- b15 = (tmp >> 5) | (tmp << (64 - 5));
+ b15 = ror64(tmp, 5);
b0 -= b15;
tmp = b9 ^ b10;
- b9 = (tmp >> 17) | (tmp << (64 - 17));
+ b9 = ror64(tmp, 17);
b10 -= b9;
tmp = b11 ^ b8;
- b11 = (tmp >> 59) | (tmp << (64 - 59));
+ b11 = ror64(tmp, 59);
b8 -= b11;
tmp = b13 ^ b14;
- b13 = (tmp >> 41) | (tmp << (64 - 41));
+ b13 = ror64(tmp, 41);
b14 -= b13;
tmp = b15 ^ b12;
- b15 = (tmp >> 34) | (tmp << (64 - 34));
+ b15 = ror64(tmp, 34);
b12 -= b15;
tmp = b1 ^ b6;
- b1 = (tmp >> 13) | (tmp << (64 - 13));
+ b1 = ror64(tmp, 13);
b6 -= b1;
tmp = b3 ^ b4;
- b3 = (tmp >> 51) | (tmp << (64 - 51));
+ b3 = ror64(tmp, 51);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 4) | (tmp << (64 - 4));
+ b5 = ror64(tmp, 4);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 33) | (tmp << (64 - 33));
+ b7 = ror64(tmp, 33);
b0 -= b7;
tmp = b1 ^ b8;
- b1 = (tmp >> 52) | (tmp << (64 - 52));
+ b1 = ror64(tmp, 52);
b8 -= b1;
tmp = b5 ^ b14;
- b5 = (tmp >> 23) | (tmp << (64 - 23));
+ b5 = ror64(tmp, 23);
b14 -= b5;
tmp = b3 ^ b12;
- b3 = (tmp >> 18) | (tmp << (64 - 18));
+ b3 = ror64(tmp, 18);
b12 -= b3;
tmp = b7 ^ b10;
- b7 = (tmp >> 49) | (tmp << (64 - 49));
+ b7 = ror64(tmp, 49);
b10 -= b7;
tmp = b15 ^ b4;
- b15 = (tmp >> 55) | (tmp << (64 - 55));
+ b15 = ror64(tmp, 55);
b4 -= b15;
tmp = b11 ^ b6;
- b11 = (tmp >> 10) | (tmp << (64 - 10));
+ b11 = ror64(tmp, 10);
b6 -= b11;
tmp = b13 ^ b2;
- b13 = (tmp >> 19) | (tmp << (64 - 19));
+ b13 = ror64(tmp, 19);
b2 -= b13;
tmp = b9 ^ b0;
- b9 = (tmp >> 38) | (tmp << (64 - 38));
+ b9 = ror64(tmp, 38);
b0 -= b9;
tmp = b15 ^ b14;
- b15 = (tmp >> 37) | (tmp << (64 - 37));
+ b15 = ror64(tmp, 37);
b14 -= b15 + k11 + t0;
b15 -= k12 + 14;
tmp = b13 ^ b12;
- b13 = (tmp >> 22) | (tmp << (64 - 22));
+ b13 = ror64(tmp, 22);
b12 -= b13 + k9;
b13 -= k10 + t2;
tmp = b11 ^ b10;
- b11 = (tmp >> 17) | (tmp << (64 - 17));
+ b11 = ror64(tmp, 17);
b10 -= b11 + k7;
b11 -= k8;
tmp = b9 ^ b8;
- b9 = (tmp >> 8) | (tmp << (64 - 8));
+ b9 = ror64(tmp, 8);
b8 -= b9 + k5;
b9 -= k6;
tmp = b7 ^ b6;
- b7 = (tmp >> 47) | (tmp << (64 - 47));
+ b7 = ror64(tmp, 47);
b6 -= b7 + k3;
b7 -= k4;
tmp = b5 ^ b4;
- b5 = (tmp >> 8) | (tmp << (64 - 8));
+ b5 = ror64(tmp, 8);
b4 -= b5 + k1;
b5 -= k2;
tmp = b3 ^ b2;
- b3 = (tmp >> 13) | (tmp << (64 - 13));
+ b3 = ror64(tmp, 13);
b2 -= b3 + k16;
b3 -= k0;
tmp = b1 ^ b0;
- b1 = (tmp >> 24) | (tmp << (64 - 24));
+ b1 = ror64(tmp, 24);
b0 -= b1 + k14;
b1 -= k15;
tmp = b7 ^ b12;
- b7 = (tmp >> 20) | (tmp << (64 - 20));
+ b7 = ror64(tmp, 20);
b12 -= b7;
tmp = b3 ^ b10;
- b3 = (tmp >> 37) | (tmp << (64 - 37));
+ b3 = ror64(tmp, 37);
b10 -= b3;
tmp = b5 ^ b8;
- b5 = (tmp >> 31) | (tmp << (64 - 31));
+ b5 = ror64(tmp, 31);
b8 -= b5;
tmp = b1 ^ b14;
- b1 = (tmp >> 23) | (tmp << (64 - 23));
+ b1 = ror64(tmp, 23);
b14 -= b1;
tmp = b9 ^ b4;
- b9 = (tmp >> 52) | (tmp << (64 - 52));
+ b9 = ror64(tmp, 52);
b4 -= b9;
tmp = b13 ^ b6;
- b13 = (tmp >> 35) | (tmp << (64 - 35));
+ b13 = ror64(tmp, 35);
b6 -= b13;
tmp = b11 ^ b2;
- b11 = (tmp >> 48) | (tmp << (64 - 48));
+ b11 = ror64(tmp, 48);
b2 -= b11;
tmp = b15 ^ b0;
- b15 = (tmp >> 9) | (tmp << (64 - 9));
+ b15 = ror64(tmp, 9);
b0 -= b15;
tmp = b9 ^ b10;
- b9 = (tmp >> 25) | (tmp << (64 - 25));
+ b9 = ror64(tmp, 25);
b10 -= b9;
tmp = b11 ^ b8;
- b11 = (tmp >> 44) | (tmp << (64 - 44));
+ b11 = ror64(tmp, 44);
b8 -= b11;
tmp = b13 ^ b14;
- b13 = (tmp >> 42) | (tmp << (64 - 42));
+ b13 = ror64(tmp, 42);
b14 -= b13;
tmp = b15 ^ b12;
- b15 = (tmp >> 19) | (tmp << (64 - 19));
+ b15 = ror64(tmp, 19);
b12 -= b15;
tmp = b1 ^ b6;
- b1 = (tmp >> 46) | (tmp << (64 - 46));
+ b1 = ror64(tmp, 46);
b6 -= b1;
tmp = b3 ^ b4;
- b3 = (tmp >> 47) | (tmp << (64 - 47));
+ b3 = ror64(tmp, 47);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 44) | (tmp << (64 - 44));
+ b5 = ror64(tmp, 44);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 31) | (tmp << (64 - 31));
+ b7 = ror64(tmp, 31);
b0 -= b7;
tmp = b1 ^ b8;
- b1 = (tmp >> 41) | (tmp << (64 - 41));
+ b1 = ror64(tmp, 41);
b8 -= b1;
tmp = b5 ^ b14;
- b5 = (tmp >> 42) | (tmp << (64 - 42));
+ b5 = ror64(tmp, 42);
b14 -= b5;
tmp = b3 ^ b12;
- b3 = (tmp >> 53) | (tmp << (64 - 53));
+ b3 = ror64(tmp, 53);
b12 -= b3;
tmp = b7 ^ b10;
- b7 = (tmp >> 4) | (tmp << (64 - 4));
+ b7 = ror64(tmp, 4);
b10 -= b7;
tmp = b15 ^ b4;
- b15 = (tmp >> 51) | (tmp << (64 - 51));
+ b15 = ror64(tmp, 51);
b4 -= b15;
tmp = b11 ^ b6;
- b11 = (tmp >> 56) | (tmp << (64 - 56));
+ b11 = ror64(tmp, 56);
b6 -= b11;
tmp = b13 ^ b2;
- b13 = (tmp >> 34) | (tmp << (64 - 34));
+ b13 = ror64(tmp, 34);
b2 -= b13;
tmp = b9 ^ b0;
- b9 = (tmp >> 16) | (tmp << (64 - 16));
+ b9 = ror64(tmp, 16);
b0 -= b9;
tmp = b15 ^ b14;
- b15 = (tmp >> 30) | (tmp << (64 - 30));
+ b15 = ror64(tmp, 30);
b14 -= b15 + k10 + t2;
b15 -= k11 + 13;
tmp = b13 ^ b12;
- b13 = (tmp >> 44) | (tmp << (64 - 44));
+ b13 = ror64(tmp, 44);
b12 -= b13 + k8;
b13 -= k9 + t1;
tmp = b11 ^ b10;
- b11 = (tmp >> 47) | (tmp << (64 - 47));
+ b11 = ror64(tmp, 47);
b10 -= b11 + k6;
b11 -= k7;
tmp = b9 ^ b8;
- b9 = (tmp >> 12) | (tmp << (64 - 12));
+ b9 = ror64(tmp, 12);
b8 -= b9 + k4;
b9 -= k5;
tmp = b7 ^ b6;
- b7 = (tmp >> 31) | (tmp << (64 - 31));
+ b7 = ror64(tmp, 31);
b6 -= b7 + k2;
b7 -= k3;
tmp = b5 ^ b4;
- b5 = (tmp >> 37) | (tmp << (64 - 37));
+ b5 = ror64(tmp, 37);
b4 -= b5 + k0;
b5 -= k1;
tmp = b3 ^ b2;
- b3 = (tmp >> 9) | (tmp << (64 - 9));
+ b3 = ror64(tmp, 9);
b2 -= b3 + k15;
b3 -= k16;
tmp = b1 ^ b0;
- b1 = (tmp >> 41) | (tmp << (64 - 41));
+ b1 = ror64(tmp, 41);
b0 -= b1 + k13;
b1 -= k14;
tmp = b7 ^ b12;
- b7 = (tmp >> 25) | (tmp << (64 - 25));
+ b7 = ror64(tmp, 25);
b12 -= b7;
tmp = b3 ^ b10;
- b3 = (tmp >> 16) | (tmp << (64 - 16));
+ b3 = ror64(tmp, 16);
b10 -= b3;
tmp = b5 ^ b8;
- b5 = (tmp >> 28) | (tmp << (64 - 28));
+ b5 = ror64(tmp, 28);
b8 -= b5;
tmp = b1 ^ b14;
- b1 = (tmp >> 47) | (tmp << (64 - 47));
+ b1 = ror64(tmp, 47);
b14 -= b1;
tmp = b9 ^ b4;
- b9 = (tmp >> 41) | (tmp << (64 - 41));
+ b9 = ror64(tmp, 41);
b4 -= b9;
tmp = b13 ^ b6;
- b13 = (tmp >> 48) | (tmp << (64 - 48));
+ b13 = ror64(tmp, 48);
b6 -= b13;
tmp = b11 ^ b2;
- b11 = (tmp >> 20) | (tmp << (64 - 20));
+ b11 = ror64(tmp, 20);
b2 -= b11;
tmp = b15 ^ b0;
- b15 = (tmp >> 5) | (tmp << (64 - 5));
+ b15 = ror64(tmp, 5);
b0 -= b15;
tmp = b9 ^ b10;
- b9 = (tmp >> 17) | (tmp << (64 - 17));
+ b9 = ror64(tmp, 17);
b10 -= b9;
tmp = b11 ^ b8;
- b11 = (tmp >> 59) | (tmp << (64 - 59));
+ b11 = ror64(tmp, 59);
b8 -= b11;
tmp = b13 ^ b14;
- b13 = (tmp >> 41) | (tmp << (64 - 41));
+ b13 = ror64(tmp, 41);
b14 -= b13;
tmp = b15 ^ b12;
- b15 = (tmp >> 34) | (tmp << (64 - 34));
+ b15 = ror64(tmp, 34);
b12 -= b15;
tmp = b1 ^ b6;
- b1 = (tmp >> 13) | (tmp << (64 - 13));
+ b1 = ror64(tmp, 13);
b6 -= b1;
tmp = b3 ^ b4;
- b3 = (tmp >> 51) | (tmp << (64 - 51));
+ b3 = ror64(tmp, 51);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 4) | (tmp << (64 - 4));
+ b5 = ror64(tmp, 4);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 33) | (tmp << (64 - 33));
+ b7 = ror64(tmp, 33);
b0 -= b7;
tmp = b1 ^ b8;
- b1 = (tmp >> 52) | (tmp << (64 - 52));
+ b1 = ror64(tmp, 52);
b8 -= b1;
tmp = b5 ^ b14;
- b5 = (tmp >> 23) | (tmp << (64 - 23));
+ b5 = ror64(tmp, 23);
b14 -= b5;
tmp = b3 ^ b12;
- b3 = (tmp >> 18) | (tmp << (64 - 18));
+ b3 = ror64(tmp, 18);
b12 -= b3;
tmp = b7 ^ b10;
- b7 = (tmp >> 49) | (tmp << (64 - 49));
+ b7 = ror64(tmp, 49);
b10 -= b7;
tmp = b15 ^ b4;
- b15 = (tmp >> 55) | (tmp << (64 - 55));
+ b15 = ror64(tmp, 55);
b4 -= b15;
tmp = b11 ^ b6;
- b11 = (tmp >> 10) | (tmp << (64 - 10));
+ b11 = ror64(tmp, 10);
b6 -= b11;
tmp = b13 ^ b2;
- b13 = (tmp >> 19) | (tmp << (64 - 19));
+ b13 = ror64(tmp, 19);
b2 -= b13;
tmp = b9 ^ b0;
- b9 = (tmp >> 38) | (tmp << (64 - 38));
+ b9 = ror64(tmp, 38);
b0 -= b9;
tmp = b15 ^ b14;
- b15 = (tmp >> 37) | (tmp << (64 - 37));
+ b15 = ror64(tmp, 37);
b14 -= b15 + k9 + t1;
b15 -= k10 + 12;
tmp = b13 ^ b12;
- b13 = (tmp >> 22) | (tmp << (64 - 22));
+ b13 = ror64(tmp, 22);
b12 -= b13 + k7;
b13 -= k8 + t0;
tmp = b11 ^ b10;
- b11 = (tmp >> 17) | (tmp << (64 - 17));
+ b11 = ror64(tmp, 17);
b10 -= b11 + k5;
b11 -= k6;
tmp = b9 ^ b8;
- b9 = (tmp >> 8) | (tmp << (64 - 8));
+ b9 = ror64(tmp, 8);
b8 -= b9 + k3;
b9 -= k4;
tmp = b7 ^ b6;
- b7 = (tmp >> 47) | (tmp << (64 - 47));
+ b7 = ror64(tmp, 47);
b6 -= b7 + k1;
b7 -= k2;
tmp = b5 ^ b4;
- b5 = (tmp >> 8) | (tmp << (64 - 8));
+ b5 = ror64(tmp, 8);
b4 -= b5 + k16;
b5 -= k0;
tmp = b3 ^ b2;
- b3 = (tmp >> 13) | (tmp << (64 - 13));
+ b3 = ror64(tmp, 13);
b2 -= b3 + k14;
b3 -= k15;
tmp = b1 ^ b0;
- b1 = (tmp >> 24) | (tmp << (64 - 24));
+ b1 = ror64(tmp, 24);
b0 -= b1 + k12;
b1 -= k13;
tmp = b7 ^ b12;
- b7 = (tmp >> 20) | (tmp << (64 - 20));
+ b7 = ror64(tmp, 20);
b12 -= b7;
tmp = b3 ^ b10;
- b3 = (tmp >> 37) | (tmp << (64 - 37));
+ b3 = ror64(tmp, 37);
b10 -= b3;
tmp = b5 ^ b8;
- b5 = (tmp >> 31) | (tmp << (64 - 31));
+ b5 = ror64(tmp, 31);
b8 -= b5;
tmp = b1 ^ b14;
- b1 = (tmp >> 23) | (tmp << (64 - 23));
+ b1 = ror64(tmp, 23);
b14 -= b1;
tmp = b9 ^ b4;
- b9 = (tmp >> 52) | (tmp << (64 - 52));
+ b9 = ror64(tmp, 52);
b4 -= b9;
tmp = b13 ^ b6;
- b13 = (tmp >> 35) | (tmp << (64 - 35));
+ b13 = ror64(tmp, 35);
b6 -= b13;
tmp = b11 ^ b2;
- b11 = (tmp >> 48) | (tmp << (64 - 48));
+ b11 = ror64(tmp, 48);
b2 -= b11;
tmp = b15 ^ b0;
- b15 = (tmp >> 9) | (tmp << (64 - 9));
+ b15 = ror64(tmp, 9);
b0 -= b15;
tmp = b9 ^ b10;
- b9 = (tmp >> 25) | (tmp << (64 - 25));
+ b9 = ror64(tmp, 25);
b10 -= b9;
tmp = b11 ^ b8;
- b11 = (tmp >> 44) | (tmp << (64 - 44));
+ b11 = ror64(tmp, 44);
b8 -= b11;
tmp = b13 ^ b14;
- b13 = (tmp >> 42) | (tmp << (64 - 42));
+ b13 = ror64(tmp, 42);
b14 -= b13;
tmp = b15 ^ b12;
- b15 = (tmp >> 19) | (tmp << (64 - 19));
+ b15 = ror64(tmp, 19);
b12 -= b15;
tmp = b1 ^ b6;
- b1 = (tmp >> 46) | (tmp << (64 - 46));
+ b1 = ror64(tmp, 46);
b6 -= b1;
tmp = b3 ^ b4;
- b3 = (tmp >> 47) | (tmp << (64 - 47));
+ b3 = ror64(tmp, 47);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 44) | (tmp << (64 - 44));
+ b5 = ror64(tmp, 44);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 31) | (tmp << (64 - 31));
+ b7 = ror64(tmp, 31);
b0 -= b7;
tmp = b1 ^ b8;
- b1 = (tmp >> 41) | (tmp << (64 - 41));
+ b1 = ror64(tmp, 41);
b8 -= b1;
tmp = b5 ^ b14;
- b5 = (tmp >> 42) | (tmp << (64 - 42));
+ b5 = ror64(tmp, 42);
b14 -= b5;
tmp = b3 ^ b12;
- b3 = (tmp >> 53) | (tmp << (64 - 53));
+ b3 = ror64(tmp, 53);
b12 -= b3;
tmp = b7 ^ b10;
- b7 = (tmp >> 4) | (tmp << (64 - 4));
+ b7 = ror64(tmp, 4);
b10 -= b7;
tmp = b15 ^ b4;
- b15 = (tmp >> 51) | (tmp << (64 - 51));
+ b15 = ror64(tmp, 51);
b4 -= b15;
tmp = b11 ^ b6;
- b11 = (tmp >> 56) | (tmp << (64 - 56));
+ b11 = ror64(tmp, 56);
b6 -= b11;
tmp = b13 ^ b2;
- b13 = (tmp >> 34) | (tmp << (64 - 34));
+ b13 = ror64(tmp, 34);
b2 -= b13;
tmp = b9 ^ b0;
- b9 = (tmp >> 16) | (tmp << (64 - 16));
+ b9 = ror64(tmp, 16);
b0 -= b9;
tmp = b15 ^ b14;
- b15 = (tmp >> 30) | (tmp << (64 - 30));
+ b15 = ror64(tmp, 30);
b14 -= b15 + k8 + t0;
b15 -= k9 + 11;
tmp = b13 ^ b12;
- b13 = (tmp >> 44) | (tmp << (64 - 44));
+ b13 = ror64(tmp, 44);
b12 -= b13 + k6;
b13 -= k7 + t2;
tmp = b11 ^ b10;
- b11 = (tmp >> 47) | (tmp << (64 - 47));
+ b11 = ror64(tmp, 47);
b10 -= b11 + k4;
b11 -= k5;
tmp = b9 ^ b8;
- b9 = (tmp >> 12) | (tmp << (64 - 12));
+ b9 = ror64(tmp, 12);
b8 -= b9 + k2;
b9 -= k3;
tmp = b7 ^ b6;
- b7 = (tmp >> 31) | (tmp << (64 - 31));
+ b7 = ror64(tmp, 31);
b6 -= b7 + k0;
b7 -= k1;
tmp = b5 ^ b4;
- b5 = (tmp >> 37) | (tmp << (64 - 37));
+ b5 = ror64(tmp, 37);
b4 -= b5 + k15;
b5 -= k16;
tmp = b3 ^ b2;
- b3 = (tmp >> 9) | (tmp << (64 - 9));
+ b3 = ror64(tmp, 9);
b2 -= b3 + k13;
b3 -= k14;
tmp = b1 ^ b0;
- b1 = (tmp >> 41) | (tmp << (64 - 41));
+ b1 = ror64(tmp, 41);
b0 -= b1 + k11;
b1 -= k12;
tmp = b7 ^ b12;
- b7 = (tmp >> 25) | (tmp << (64 - 25));
+ b7 = ror64(tmp, 25);
b12 -= b7;
tmp = b3 ^ b10;
- b3 = (tmp >> 16) | (tmp << (64 - 16));
+ b3 = ror64(tmp, 16);
b10 -= b3;
tmp = b5 ^ b8;
- b5 = (tmp >> 28) | (tmp << (64 - 28));
+ b5 = ror64(tmp, 28);
b8 -= b5;
tmp = b1 ^ b14;
- b1 = (tmp >> 47) | (tmp << (64 - 47));
+ b1 = ror64(tmp, 47);
b14 -= b1;
tmp = b9 ^ b4;
- b9 = (tmp >> 41) | (tmp << (64 - 41));
+ b9 = ror64(tmp, 41);
b4 -= b9;
tmp = b13 ^ b6;
- b13 = (tmp >> 48) | (tmp << (64 - 48));
+ b13 = ror64(tmp, 48);
b6 -= b13;
tmp = b11 ^ b2;
- b11 = (tmp >> 20) | (tmp << (64 - 20));
+ b11 = ror64(tmp, 20);
b2 -= b11;
tmp = b15 ^ b0;
- b15 = (tmp >> 5) | (tmp << (64 - 5));
+ b15 = ror64(tmp, 5);
b0 -= b15;
tmp = b9 ^ b10;
- b9 = (tmp >> 17) | (tmp << (64 - 17));
+ b9 = ror64(tmp, 17);
b10 -= b9;
tmp = b11 ^ b8;
- b11 = (tmp >> 59) | (tmp << (64 - 59));
+ b11 = ror64(tmp, 59);
b8 -= b11;
tmp = b13 ^ b14;
- b13 = (tmp >> 41) | (tmp << (64 - 41));
+ b13 = ror64(tmp, 41);
b14 -= b13;
tmp = b15 ^ b12;
- b15 = (tmp >> 34) | (tmp << (64 - 34));
+ b15 = ror64(tmp, 34);
b12 -= b15;
tmp = b1 ^ b6;
- b1 = (tmp >> 13) | (tmp << (64 - 13));
+ b1 = ror64(tmp, 13);
b6 -= b1;
tmp = b3 ^ b4;
- b3 = (tmp >> 51) | (tmp << (64 - 51));
+ b3 = ror64(tmp, 51);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 4) | (tmp << (64 - 4));
+ b5 = ror64(tmp, 4);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 33) | (tmp << (64 - 33));
+ b7 = ror64(tmp, 33);
b0 -= b7;
tmp = b1 ^ b8;
- b1 = (tmp >> 52) | (tmp << (64 - 52));
+ b1 = ror64(tmp, 52);
b8 -= b1;
tmp = b5 ^ b14;
- b5 = (tmp >> 23) | (tmp << (64 - 23));
+ b5 = ror64(tmp, 23);
b14 -= b5;
tmp = b3 ^ b12;
- b3 = (tmp >> 18) | (tmp << (64 - 18));
+ b3 = ror64(tmp, 18);
b12 -= b3;
tmp = b7 ^ b10;
- b7 = (tmp >> 49) | (tmp << (64 - 49));
+ b7 = ror64(tmp, 49);
b10 -= b7;
tmp = b15 ^ b4;
- b15 = (tmp >> 55) | (tmp << (64 - 55));
+ b15 = ror64(tmp, 55);
b4 -= b15;
tmp = b11 ^ b6;
- b11 = (tmp >> 10) | (tmp << (64 - 10));
+ b11 = ror64(tmp, 10);
b6 -= b11;
tmp = b13 ^ b2;
- b13 = (tmp >> 19) | (tmp << (64 - 19));
+ b13 = ror64(tmp, 19);
b2 -= b13;
tmp = b9 ^ b0;
- b9 = (tmp >> 38) | (tmp << (64 - 38));
+ b9 = ror64(tmp, 38);
b0 -= b9;
tmp = b15 ^ b14;
- b15 = (tmp >> 37) | (tmp << (64 - 37));
+ b15 = ror64(tmp, 37);
b14 -= b15 + k7 + t2;
b15 -= k8 + 10;
tmp = b13 ^ b12;
- b13 = (tmp >> 22) | (tmp << (64 - 22));
+ b13 = ror64(tmp, 22);
b12 -= b13 + k5;
b13 -= k6 + t1;
tmp = b11 ^ b10;
- b11 = (tmp >> 17) | (tmp << (64 - 17));
+ b11 = ror64(tmp, 17);
b10 -= b11 + k3;
b11 -= k4;
tmp = b9 ^ b8;
- b9 = (tmp >> 8) | (tmp << (64 - 8));
+ b9 = ror64(tmp, 8);
b8 -= b9 + k1;
b9 -= k2;
tmp = b7 ^ b6;
- b7 = (tmp >> 47) | (tmp << (64 - 47));
+ b7 = ror64(tmp, 47);
b6 -= b7 + k16;
b7 -= k0;
tmp = b5 ^ b4;
- b5 = (tmp >> 8) | (tmp << (64 - 8));
+ b5 = ror64(tmp, 8);
b4 -= b5 + k14;
b5 -= k15;
tmp = b3 ^ b2;
- b3 = (tmp >> 13) | (tmp << (64 - 13));
+ b3 = ror64(tmp, 13);
b2 -= b3 + k12;
b3 -= k13;
tmp = b1 ^ b0;
- b1 = (tmp >> 24) | (tmp << (64 - 24));
+ b1 = ror64(tmp, 24);
b0 -= b1 + k10;
b1 -= k11;
tmp = b7 ^ b12;
- b7 = (tmp >> 20) | (tmp << (64 - 20));
+ b7 = ror64(tmp, 20);
b12 -= b7;
tmp = b3 ^ b10;
- b3 = (tmp >> 37) | (tmp << (64 - 37));
+ b3 = ror64(tmp, 37);
b10 -= b3;
tmp = b5 ^ b8;
- b5 = (tmp >> 31) | (tmp << (64 - 31));
+ b5 = ror64(tmp, 31);
b8 -= b5;
tmp = b1 ^ b14;
- b1 = (tmp >> 23) | (tmp << (64 - 23));
+ b1 = ror64(tmp, 23);
b14 -= b1;
tmp = b9 ^ b4;
- b9 = (tmp >> 52) | (tmp << (64 - 52));
+ b9 = ror64(tmp, 52);
b4 -= b9;
tmp = b13 ^ b6;
- b13 = (tmp >> 35) | (tmp << (64 - 35));
+ b13 = ror64(tmp, 35);
b6 -= b13;
tmp = b11 ^ b2;
- b11 = (tmp >> 48) | (tmp << (64 - 48));
+ b11 = ror64(tmp, 48);
b2 -= b11;
tmp = b15 ^ b0;
- b15 = (tmp >> 9) | (tmp << (64 - 9));
+ b15 = ror64(tmp, 9);
b0 -= b15;
tmp = b9 ^ b10;
- b9 = (tmp >> 25) | (tmp << (64 - 25));
+ b9 = ror64(tmp, 25);
b10 -= b9;
tmp = b11 ^ b8;
- b11 = (tmp >> 44) | (tmp << (64 - 44));
+ b11 = ror64(tmp, 44);
b8 -= b11;
tmp = b13 ^ b14;
- b13 = (tmp >> 42) | (tmp << (64 - 42));
+ b13 = ror64(tmp, 42);
b14 -= b13;
tmp = b15 ^ b12;
- b15 = (tmp >> 19) | (tmp << (64 - 19));
+ b15 = ror64(tmp, 19);
b12 -= b15;
tmp = b1 ^ b6;
- b1 = (tmp >> 46) | (tmp << (64 - 46));
+ b1 = ror64(tmp, 46);
b6 -= b1;
tmp = b3 ^ b4;
- b3 = (tmp >> 47) | (tmp << (64 - 47));
+ b3 = ror64(tmp, 47);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 44) | (tmp << (64 - 44));
+ b5 = ror64(tmp, 44);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 31) | (tmp << (64 - 31));
+ b7 = ror64(tmp, 31);
b0 -= b7;
tmp = b1 ^ b8;
- b1 = (tmp >> 41) | (tmp << (64 - 41));
+ b1 = ror64(tmp, 41);
b8 -= b1;
tmp = b5 ^ b14;
- b5 = (tmp >> 42) | (tmp << (64 - 42));
+ b5 = ror64(tmp, 42);
b14 -= b5;
tmp = b3 ^ b12;
- b3 = (tmp >> 53) | (tmp << (64 - 53));
+ b3 = ror64(tmp, 53);
b12 -= b3;
tmp = b7 ^ b10;
- b7 = (tmp >> 4) | (tmp << (64 - 4));
+ b7 = ror64(tmp, 4);
b10 -= b7;
tmp = b15 ^ b4;
- b15 = (tmp >> 51) | (tmp << (64 - 51));
+ b15 = ror64(tmp, 51);
b4 -= b15;
tmp = b11 ^ b6;
- b11 = (tmp >> 56) | (tmp << (64 - 56));
+ b11 = ror64(tmp, 56);
b6 -= b11;
tmp = b13 ^ b2;
- b13 = (tmp >> 34) | (tmp << (64 - 34));
+ b13 = ror64(tmp, 34);
b2 -= b13;
tmp = b9 ^ b0;
- b9 = (tmp >> 16) | (tmp << (64 - 16));
+ b9 = ror64(tmp, 16);
b0 -= b9;
tmp = b15 ^ b14;
- b15 = (tmp >> 30) | (tmp << (64 - 30));
+ b15 = ror64(tmp, 30);
b14 -= b15 + k6 + t1;
b15 -= k7 + 9;
tmp = b13 ^ b12;
- b13 = (tmp >> 44) | (tmp << (64 - 44));
+ b13 = ror64(tmp, 44);
b12 -= b13 + k4;
b13 -= k5 + t0;
tmp = b11 ^ b10;
- b11 = (tmp >> 47) | (tmp << (64 - 47));
+ b11 = ror64(tmp, 47);
b10 -= b11 + k2;
b11 -= k3;
tmp = b9 ^ b8;
- b9 = (tmp >> 12) | (tmp << (64 - 12));
+ b9 = ror64(tmp, 12);
b8 -= b9 + k0;
b9 -= k1;
tmp = b7 ^ b6;
- b7 = (tmp >> 31) | (tmp << (64 - 31));
+ b7 = ror64(tmp, 31);
b6 -= b7 + k15;
b7 -= k16;
tmp = b5 ^ b4;
- b5 = (tmp >> 37) | (tmp << (64 - 37));
+ b5 = ror64(tmp, 37);
b4 -= b5 + k13;
b5 -= k14;
tmp = b3 ^ b2;
- b3 = (tmp >> 9) | (tmp << (64 - 9));
+ b3 = ror64(tmp, 9);
b2 -= b3 + k11;
b3 -= k12;
tmp = b1 ^ b0;
- b1 = (tmp >> 41) | (tmp << (64 - 41));
+ b1 = ror64(tmp, 41);
b0 -= b1 + k9;
b1 -= k10;
tmp = b7 ^ b12;
- b7 = (tmp >> 25) | (tmp << (64 - 25));
+ b7 = ror64(tmp, 25);
b12 -= b7;
tmp = b3 ^ b10;
- b3 = (tmp >> 16) | (tmp << (64 - 16));
+ b3 = ror64(tmp, 16);
b10 -= b3;
tmp = b5 ^ b8;
- b5 = (tmp >> 28) | (tmp << (64 - 28));
+ b5 = ror64(tmp, 28);
b8 -= b5;
tmp = b1 ^ b14;
- b1 = (tmp >> 47) | (tmp << (64 - 47));
+ b1 = ror64(tmp, 47);
b14 -= b1;
tmp = b9 ^ b4;
- b9 = (tmp >> 41) | (tmp << (64 - 41));
+ b9 = ror64(tmp, 41);
b4 -= b9;
tmp = b13 ^ b6;
- b13 = (tmp >> 48) | (tmp << (64 - 48));
+ b13 = ror64(tmp, 48);
b6 -= b13;
tmp = b11 ^ b2;
- b11 = (tmp >> 20) | (tmp << (64 - 20));
+ b11 = ror64(tmp, 20);
b2 -= b11;
tmp = b15 ^ b0;
- b15 = (tmp >> 5) | (tmp << (64 - 5));
+ b15 = ror64(tmp, 5);
b0 -= b15;
tmp = b9 ^ b10;
- b9 = (tmp >> 17) | (tmp << (64 - 17));
+ b9 = ror64(tmp, 17);
b10 -= b9;
tmp = b11 ^ b8;
- b11 = (tmp >> 59) | (tmp << (64 - 59));
+ b11 = ror64(tmp, 59);
b8 -= b11;
tmp = b13 ^ b14;
- b13 = (tmp >> 41) | (tmp << (64 - 41));
+ b13 = ror64(tmp, 41);
b14 -= b13;
tmp = b15 ^ b12;
- b15 = (tmp >> 34) | (tmp << (64 - 34));
+ b15 = ror64(tmp, 34);
b12 -= b15;
tmp = b1 ^ b6;
- b1 = (tmp >> 13) | (tmp << (64 - 13));
+ b1 = ror64(tmp, 13);
b6 -= b1;
tmp = b3 ^ b4;
- b3 = (tmp >> 51) | (tmp << (64 - 51));
+ b3 = ror64(tmp, 51);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 4) | (tmp << (64 - 4));
+ b5 = ror64(tmp, 4);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 33) | (tmp << (64 - 33));
+ b7 = ror64(tmp, 33);
b0 -= b7;
tmp = b1 ^ b8;
- b1 = (tmp >> 52) | (tmp << (64 - 52));
+ b1 = ror64(tmp, 52);
b8 -= b1;
tmp = b5 ^ b14;
- b5 = (tmp >> 23) | (tmp << (64 - 23));
+ b5 = ror64(tmp, 23);
b14 -= b5;
tmp = b3 ^ b12;
- b3 = (tmp >> 18) | (tmp << (64 - 18));
+ b3 = ror64(tmp, 18);
b12 -= b3;
tmp = b7 ^ b10;
- b7 = (tmp >> 49) | (tmp << (64 - 49));
+ b7 = ror64(tmp, 49);
b10 -= b7;
tmp = b15 ^ b4;
- b15 = (tmp >> 55) | (tmp << (64 - 55));
+ b15 = ror64(tmp, 55);
b4 -= b15;
tmp = b11 ^ b6;
- b11 = (tmp >> 10) | (tmp << (64 - 10));
+ b11 = ror64(tmp, 10);
b6 -= b11;
tmp = b13 ^ b2;
- b13 = (tmp >> 19) | (tmp << (64 - 19));
+ b13 = ror64(tmp, 19);
b2 -= b13;
tmp = b9 ^ b0;
- b9 = (tmp >> 38) | (tmp << (64 - 38));
+ b9 = ror64(tmp, 38);
b0 -= b9;
tmp = b15 ^ b14;
- b15 = (tmp >> 37) | (tmp << (64 - 37));
+ b15 = ror64(tmp, 37);
b14 -= b15 + k5 + t0;
b15 -= k6 + 8;
tmp = b13 ^ b12;
- b13 = (tmp >> 22) | (tmp << (64 - 22));
+ b13 = ror64(tmp, 22);
b12 -= b13 + k3;
b13 -= k4 + t2;
tmp = b11 ^ b10;
- b11 = (tmp >> 17) | (tmp << (64 - 17));
+ b11 = ror64(tmp, 17);
b10 -= b11 + k1;
b11 -= k2;
tmp = b9 ^ b8;
- b9 = (tmp >> 8) | (tmp << (64 - 8));
+ b9 = ror64(tmp, 8);
b8 -= b9 + k16;
b9 -= k0;
tmp = b7 ^ b6;
- b7 = (tmp >> 47) | (tmp << (64 - 47));
+ b7 = ror64(tmp, 47);
b6 -= b7 + k14;
b7 -= k15;
tmp = b5 ^ b4;
- b5 = (tmp >> 8) | (tmp << (64 - 8));
+ b5 = ror64(tmp, 8);
b4 -= b5 + k12;
b5 -= k13;
tmp = b3 ^ b2;
- b3 = (tmp >> 13) | (tmp << (64 - 13));
+ b3 = ror64(tmp, 13);
b2 -= b3 + k10;
b3 -= k11;
tmp = b1 ^ b0;
- b1 = (tmp >> 24) | (tmp << (64 - 24));
+ b1 = ror64(tmp, 24);
b0 -= b1 + k8;
b1 -= k9;
tmp = b7 ^ b12;
- b7 = (tmp >> 20) | (tmp << (64 - 20));
+ b7 = ror64(tmp, 20);
b12 -= b7;
tmp = b3 ^ b10;
- b3 = (tmp >> 37) | (tmp << (64 - 37));
+ b3 = ror64(tmp, 37);
b10 -= b3;
tmp = b5 ^ b8;
- b5 = (tmp >> 31) | (tmp << (64 - 31));
+ b5 = ror64(tmp, 31);
b8 -= b5;
tmp = b1 ^ b14;
- b1 = (tmp >> 23) | (tmp << (64 - 23));
+ b1 = ror64(tmp, 23);
b14 -= b1;
tmp = b9 ^ b4;
- b9 = (tmp >> 52) | (tmp << (64 - 52));
+ b9 = ror64(tmp, 52);
b4 -= b9;
tmp = b13 ^ b6;
- b13 = (tmp >> 35) | (tmp << (64 - 35));
+ b13 = ror64(tmp, 35);
b6 -= b13;
tmp = b11 ^ b2;
- b11 = (tmp >> 48) | (tmp << (64 - 48));
+ b11 = ror64(tmp, 48);
b2 -= b11;
tmp = b15 ^ b0;
- b15 = (tmp >> 9) | (tmp << (64 - 9));
+ b15 = ror64(tmp, 9);
b0 -= b15;
tmp = b9 ^ b10;
- b9 = (tmp >> 25) | (tmp << (64 - 25));
+ b9 = ror64(tmp, 25);
b10 -= b9;
tmp = b11 ^ b8;
- b11 = (tmp >> 44) | (tmp << (64 - 44));
+ b11 = ror64(tmp, 44);
b8 -= b11;
tmp = b13 ^ b14;
- b13 = (tmp >> 42) | (tmp << (64 - 42));
+ b13 = ror64(tmp, 42);
b14 -= b13;
tmp = b15 ^ b12;
- b15 = (tmp >> 19) | (tmp << (64 - 19));
+ b15 = ror64(tmp, 19);
b12 -= b15;
tmp = b1 ^ b6;
- b1 = (tmp >> 46) | (tmp << (64 - 46));
+ b1 = ror64(tmp, 46);
b6 -= b1;
tmp = b3 ^ b4;
- b3 = (tmp >> 47) | (tmp << (64 - 47));
+ b3 = ror64(tmp, 47);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 44) | (tmp << (64 - 44));
+ b5 = ror64(tmp, 44);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 31) | (tmp << (64 - 31));
+ b7 = ror64(tmp, 31);
b0 -= b7;
tmp = b1 ^ b8;
- b1 = (tmp >> 41) | (tmp << (64 - 41));
+ b1 = ror64(tmp, 41);
b8 -= b1;
tmp = b5 ^ b14;
- b5 = (tmp >> 42) | (tmp << (64 - 42));
+ b5 = ror64(tmp, 42);
b14 -= b5;
tmp = b3 ^ b12;
- b3 = (tmp >> 53) | (tmp << (64 - 53));
+ b3 = ror64(tmp, 53);
b12 -= b3;
tmp = b7 ^ b10;
- b7 = (tmp >> 4) | (tmp << (64 - 4));
+ b7 = ror64(tmp, 4);
b10 -= b7;
tmp = b15 ^ b4;
- b15 = (tmp >> 51) | (tmp << (64 - 51));
+ b15 = ror64(tmp, 51);
b4 -= b15;
tmp = b11 ^ b6;
- b11 = (tmp >> 56) | (tmp << (64 - 56));
+ b11 = ror64(tmp, 56);
b6 -= b11;
tmp = b13 ^ b2;
- b13 = (tmp >> 34) | (tmp << (64 - 34));
+ b13 = ror64(tmp, 34);
b2 -= b13;
tmp = b9 ^ b0;
- b9 = (tmp >> 16) | (tmp << (64 - 16));
+ b9 = ror64(tmp, 16);
b0 -= b9;
tmp = b15 ^ b14;
- b15 = (tmp >> 30) | (tmp << (64 - 30));
+ b15 = ror64(tmp, 30);
b14 -= b15 + k4 + t2;
b15 -= k5 + 7;
tmp = b13 ^ b12;
- b13 = (tmp >> 44) | (tmp << (64 - 44));
+ b13 = ror64(tmp, 44);
b12 -= b13 + k2;
b13 -= k3 + t1;
tmp = b11 ^ b10;
- b11 = (tmp >> 47) | (tmp << (64 - 47));
+ b11 = ror64(tmp, 47);
b10 -= b11 + k0;
b11 -= k1;
tmp = b9 ^ b8;
- b9 = (tmp >> 12) | (tmp << (64 - 12));
+ b9 = ror64(tmp, 12);
b8 -= b9 + k15;
b9 -= k16;
tmp = b7 ^ b6;
- b7 = (tmp >> 31) | (tmp << (64 - 31));
+ b7 = ror64(tmp, 31);
b6 -= b7 + k13;
b7 -= k14;
tmp = b5 ^ b4;
- b5 = (tmp >> 37) | (tmp << (64 - 37));
+ b5 = ror64(tmp, 37);
b4 -= b5 + k11;
b5 -= k12;
tmp = b3 ^ b2;
- b3 = (tmp >> 9) | (tmp << (64 - 9));
+ b3 = ror64(tmp, 9);
b2 -= b3 + k9;
b3 -= k10;
tmp = b1 ^ b0;
- b1 = (tmp >> 41) | (tmp << (64 - 41));
+ b1 = ror64(tmp, 41);
b0 -= b1 + k7;
b1 -= k8;
tmp = b7 ^ b12;
- b7 = (tmp >> 25) | (tmp << (64 - 25));
+ b7 = ror64(tmp, 25);
b12 -= b7;
tmp = b3 ^ b10;
- b3 = (tmp >> 16) | (tmp << (64 - 16));
+ b3 = ror64(tmp, 16);
b10 -= b3;
tmp = b5 ^ b8;
- b5 = (tmp >> 28) | (tmp << (64 - 28));
+ b5 = ror64(tmp, 28);
b8 -= b5;
tmp = b1 ^ b14;
- b1 = (tmp >> 47) | (tmp << (64 - 47));
+ b1 = ror64(tmp, 47);
b14 -= b1;
tmp = b9 ^ b4;
- b9 = (tmp >> 41) | (tmp << (64 - 41));
+ b9 = ror64(tmp, 41);
b4 -= b9;
tmp = b13 ^ b6;
- b13 = (tmp >> 48) | (tmp << (64 - 48));
+ b13 = ror64(tmp, 48);
b6 -= b13;
tmp = b11 ^ b2;
- b11 = (tmp >> 20) | (tmp << (64 - 20));
+ b11 = ror64(tmp, 20);
b2 -= b11;
tmp = b15 ^ b0;
- b15 = (tmp >> 5) | (tmp << (64 - 5));
+ b15 = ror64(tmp, 5);
b0 -= b15;
tmp = b9 ^ b10;
- b9 = (tmp >> 17) | (tmp << (64 - 17));
+ b9 = ror64(tmp, 17);
b10 -= b9;
tmp = b11 ^ b8;
- b11 = (tmp >> 59) | (tmp << (64 - 59));
+ b11 = ror64(tmp, 59);
b8 -= b11;
tmp = b13 ^ b14;
- b13 = (tmp >> 41) | (tmp << (64 - 41));
+ b13 = ror64(tmp, 41);
b14 -= b13;
tmp = b15 ^ b12;
- b15 = (tmp >> 34) | (tmp << (64 - 34));
+ b15 = ror64(tmp, 34);
b12 -= b15;
tmp = b1 ^ b6;
- b1 = (tmp >> 13) | (tmp << (64 - 13));
+ b1 = ror64(tmp, 13);
b6 -= b1;
tmp = b3 ^ b4;
- b3 = (tmp >> 51) | (tmp << (64 - 51));
+ b3 = ror64(tmp, 51);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 4) | (tmp << (64 - 4));
+ b5 = ror64(tmp, 4);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 33) | (tmp << (64 - 33));
+ b7 = ror64(tmp, 33);
b0 -= b7;
tmp = b1 ^ b8;
- b1 = (tmp >> 52) | (tmp << (64 - 52));
+ b1 = ror64(tmp, 52);
b8 -= b1;
tmp = b5 ^ b14;
- b5 = (tmp >> 23) | (tmp << (64 - 23));
+ b5 = ror64(tmp, 23);
b14 -= b5;
tmp = b3 ^ b12;
- b3 = (tmp >> 18) | (tmp << (64 - 18));
+ b3 = ror64(tmp, 18);
b12 -= b3;
tmp = b7 ^ b10;
- b7 = (tmp >> 49) | (tmp << (64 - 49));
+ b7 = ror64(tmp, 49);
b10 -= b7;
tmp = b15 ^ b4;
- b15 = (tmp >> 55) | (tmp << (64 - 55));
+ b15 = ror64(tmp, 55);
b4 -= b15;
tmp = b11 ^ b6;
- b11 = (tmp >> 10) | (tmp << (64 - 10));
+ b11 = ror64(tmp, 10);
b6 -= b11;
tmp = b13 ^ b2;
- b13 = (tmp >> 19) | (tmp << (64 - 19));
+ b13 = ror64(tmp, 19);
b2 -= b13;
tmp = b9 ^ b0;
- b9 = (tmp >> 38) | (tmp << (64 - 38));
+ b9 = ror64(tmp, 38);
b0 -= b9;
tmp = b15 ^ b14;
- b15 = (tmp >> 37) | (tmp << (64 - 37));
+ b15 = ror64(tmp, 37);
b14 -= b15 + k3 + t1;
b15 -= k4 + 6;
tmp = b13 ^ b12;
- b13 = (tmp >> 22) | (tmp << (64 - 22));
+ b13 = ror64(tmp, 22);
b12 -= b13 + k1;
b13 -= k2 + t0;
tmp = b11 ^ b10;
- b11 = (tmp >> 17) | (tmp << (64 - 17));
+ b11 = ror64(tmp, 17);
b10 -= b11 + k16;
b11 -= k0;
tmp = b9 ^ b8;
- b9 = (tmp >> 8) | (tmp << (64 - 8));
+ b9 = ror64(tmp, 8);
b8 -= b9 + k14;
b9 -= k15;
tmp = b7 ^ b6;
- b7 = (tmp >> 47) | (tmp << (64 - 47));
+ b7 = ror64(tmp, 47);
b6 -= b7 + k12;
b7 -= k13;
tmp = b5 ^ b4;
- b5 = (tmp >> 8) | (tmp << (64 - 8));
+ b5 = ror64(tmp, 8);
b4 -= b5 + k10;
b5 -= k11;
tmp = b3 ^ b2;
- b3 = (tmp >> 13) | (tmp << (64 - 13));
+ b3 = ror64(tmp, 13);
b2 -= b3 + k8;
b3 -= k9;
tmp = b1 ^ b0;
- b1 = (tmp >> 24) | (tmp << (64 - 24));
+ b1 = ror64(tmp, 24);
b0 -= b1 + k6;
b1 -= k7;
tmp = b7 ^ b12;
- b7 = (tmp >> 20) | (tmp << (64 - 20));
+ b7 = ror64(tmp, 20);
b12 -= b7;
tmp = b3 ^ b10;
- b3 = (tmp >> 37) | (tmp << (64 - 37));
+ b3 = ror64(tmp, 37);
b10 -= b3;
tmp = b5 ^ b8;
- b5 = (tmp >> 31) | (tmp << (64 - 31));
+ b5 = ror64(tmp, 31);
b8 -= b5;
tmp = b1 ^ b14;
- b1 = (tmp >> 23) | (tmp << (64 - 23));
+ b1 = ror64(tmp, 23);
b14 -= b1;
tmp = b9 ^ b4;
- b9 = (tmp >> 52) | (tmp << (64 - 52));
+ b9 = ror64(tmp, 52);
b4 -= b9;
tmp = b13 ^ b6;
- b13 = (tmp >> 35) | (tmp << (64 - 35));
+ b13 = ror64(tmp, 35);
b6 -= b13;
tmp = b11 ^ b2;
- b11 = (tmp >> 48) | (tmp << (64 - 48));
+ b11 = ror64(tmp, 48);
b2 -= b11;
tmp = b15 ^ b0;
- b15 = (tmp >> 9) | (tmp << (64 - 9));
+ b15 = ror64(tmp, 9);
b0 -= b15;
tmp = b9 ^ b10;
- b9 = (tmp >> 25) | (tmp << (64 - 25));
+ b9 = ror64(tmp, 25);
b10 -= b9;
tmp = b11 ^ b8;
- b11 = (tmp >> 44) | (tmp << (64 - 44));
+ b11 = ror64(tmp, 44);
b8 -= b11;
tmp = b13 ^ b14;
- b13 = (tmp >> 42) | (tmp << (64 - 42));
+ b13 = ror64(tmp, 42);
b14 -= b13;
tmp = b15 ^ b12;
- b15 = (tmp >> 19) | (tmp << (64 - 19));
+ b15 = ror64(tmp, 19);
b12 -= b15;
tmp = b1 ^ b6;
- b1 = (tmp >> 46) | (tmp << (64 - 46));
+ b1 = ror64(tmp, 46);
b6 -= b1;
tmp = b3 ^ b4;
- b3 = (tmp >> 47) | (tmp << (64 - 47));
+ b3 = ror64(tmp, 47);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 44) | (tmp << (64 - 44));
+ b5 = ror64(tmp, 44);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 31) | (tmp << (64 - 31));
+ b7 = ror64(tmp, 31);
b0 -= b7;
tmp = b1 ^ b8;
- b1 = (tmp >> 41) | (tmp << (64 - 41));
+ b1 = ror64(tmp, 41);
b8 -= b1;
tmp = b5 ^ b14;
- b5 = (tmp >> 42) | (tmp << (64 - 42));
+ b5 = ror64(tmp, 42);
b14 -= b5;
tmp = b3 ^ b12;
- b3 = (tmp >> 53) | (tmp << (64 - 53));
+ b3 = ror64(tmp, 53);
b12 -= b3;
tmp = b7 ^ b10;
- b7 = (tmp >> 4) | (tmp << (64 - 4));
+ b7 = ror64(tmp, 4);
b10 -= b7;
tmp = b15 ^ b4;
- b15 = (tmp >> 51) | (tmp << (64 - 51));
+ b15 = ror64(tmp, 51);
b4 -= b15;
tmp = b11 ^ b6;
- b11 = (tmp >> 56) | (tmp << (64 - 56));
+ b11 = ror64(tmp, 56);
b6 -= b11;
tmp = b13 ^ b2;
- b13 = (tmp >> 34) | (tmp << (64 - 34));
+ b13 = ror64(tmp, 34);
b2 -= b13;
tmp = b9 ^ b0;
- b9 = (tmp >> 16) | (tmp << (64 - 16));
+ b9 = ror64(tmp, 16);
b0 -= b9;
tmp = b15 ^ b14;
- b15 = (tmp >> 30) | (tmp << (64 - 30));
+ b15 = ror64(tmp, 30);
b14 -= b15 + k2 + t0;
b15 -= k3 + 5;
tmp = b13 ^ b12;
- b13 = (tmp >> 44) | (tmp << (64 - 44));
+ b13 = ror64(tmp, 44);
b12 -= b13 + k0;
b13 -= k1 + t2;
tmp = b11 ^ b10;
- b11 = (tmp >> 47) | (tmp << (64 - 47));
+ b11 = ror64(tmp, 47);
b10 -= b11 + k15;
b11 -= k16;
tmp = b9 ^ b8;
- b9 = (tmp >> 12) | (tmp << (64 - 12));
+ b9 = ror64(tmp, 12);
b8 -= b9 + k13;
b9 -= k14;
tmp = b7 ^ b6;
- b7 = (tmp >> 31) | (tmp << (64 - 31));
+ b7 = ror64(tmp, 31);
b6 -= b7 + k11;
b7 -= k12;
tmp = b5 ^ b4;
- b5 = (tmp >> 37) | (tmp << (64 - 37));
+ b5 = ror64(tmp, 37);
b4 -= b5 + k9;
b5 -= k10;
tmp = b3 ^ b2;
- b3 = (tmp >> 9) | (tmp << (64 - 9));
+ b3 = ror64(tmp, 9);
b2 -= b3 + k7;
b3 -= k8;
tmp = b1 ^ b0;
- b1 = (tmp >> 41) | (tmp << (64 - 41));
+ b1 = ror64(tmp, 41);
b0 -= b1 + k5;
b1 -= k6;
tmp = b7 ^ b12;
- b7 = (tmp >> 25) | (tmp << (64 - 25));
+ b7 = ror64(tmp, 25);
b12 -= b7;
tmp = b3 ^ b10;
- b3 = (tmp >> 16) | (tmp << (64 - 16));
+ b3 = ror64(tmp, 16);
b10 -= b3;
tmp = b5 ^ b8;
- b5 = (tmp >> 28) | (tmp << (64 - 28));
+ b5 = ror64(tmp, 28);
b8 -= b5;
tmp = b1 ^ b14;
- b1 = (tmp >> 47) | (tmp << (64 - 47));
+ b1 = ror64(tmp, 47);
b14 -= b1;
tmp = b9 ^ b4;
- b9 = (tmp >> 41) | (tmp << (64 - 41));
+ b9 = ror64(tmp, 41);
b4 -= b9;
tmp = b13 ^ b6;
- b13 = (tmp >> 48) | (tmp << (64 - 48));
+ b13 = ror64(tmp, 48);
b6 -= b13;
tmp = b11 ^ b2;
- b11 = (tmp >> 20) | (tmp << (64 - 20));
+ b11 = ror64(tmp, 20);
b2 -= b11;
tmp = b15 ^ b0;
- b15 = (tmp >> 5) | (tmp << (64 - 5));
+ b15 = ror64(tmp, 5);
b0 -= b15;
tmp = b9 ^ b10;
- b9 = (tmp >> 17) | (tmp << (64 - 17));
+ b9 = ror64(tmp, 17);
b10 -= b9;
tmp = b11 ^ b8;
- b11 = (tmp >> 59) | (tmp << (64 - 59));
+ b11 = ror64(tmp, 59);
b8 -= b11;
tmp = b13 ^ b14;
- b13 = (tmp >> 41) | (tmp << (64 - 41));
+ b13 = ror64(tmp, 41);
b14 -= b13;
tmp = b15 ^ b12;
- b15 = (tmp >> 34) | (tmp << (64 - 34));
+ b15 = ror64(tmp, 34);
b12 -= b15;
tmp = b1 ^ b6;
- b1 = (tmp >> 13) | (tmp << (64 - 13));
+ b1 = ror64(tmp, 13);
b6 -= b1;
tmp = b3 ^ b4;
- b3 = (tmp >> 51) | (tmp << (64 - 51));
+ b3 = ror64(tmp, 51);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 4) | (tmp << (64 - 4));
+ b5 = ror64(tmp, 4);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 33) | (tmp << (64 - 33));
+ b7 = ror64(tmp, 33);
b0 -= b7;
tmp = b1 ^ b8;
- b1 = (tmp >> 52) | (tmp << (64 - 52));
+ b1 = ror64(tmp, 52);
b8 -= b1;
tmp = b5 ^ b14;
- b5 = (tmp >> 23) | (tmp << (64 - 23));
+ b5 = ror64(tmp, 23);
b14 -= b5;
tmp = b3 ^ b12;
- b3 = (tmp >> 18) | (tmp << (64 - 18));
+ b3 = ror64(tmp, 18);
b12 -= b3;
tmp = b7 ^ b10;
- b7 = (tmp >> 49) | (tmp << (64 - 49));
+ b7 = ror64(tmp, 49);
b10 -= b7;
tmp = b15 ^ b4;
- b15 = (tmp >> 55) | (tmp << (64 - 55));
+ b15 = ror64(tmp, 55);
b4 -= b15;
tmp = b11 ^ b6;
- b11 = (tmp >> 10) | (tmp << (64 - 10));
+ b11 = ror64(tmp, 10);
b6 -= b11;
tmp = b13 ^ b2;
- b13 = (tmp >> 19) | (tmp << (64 - 19));
+ b13 = ror64(tmp, 19);
b2 -= b13;
tmp = b9 ^ b0;
- b9 = (tmp >> 38) | (tmp << (64 - 38));
+ b9 = ror64(tmp, 38);
b0 -= b9;
tmp = b15 ^ b14;
- b15 = (tmp >> 37) | (tmp << (64 - 37));
+ b15 = ror64(tmp, 37);
b14 -= b15 + k1 + t2;
b15 -= k2 + 4;
tmp = b13 ^ b12;
- b13 = (tmp >> 22) | (tmp << (64 - 22));
+ b13 = ror64(tmp, 22);
b12 -= b13 + k16;
b13 -= k0 + t1;
tmp = b11 ^ b10;
- b11 = (tmp >> 17) | (tmp << (64 - 17));
+ b11 = ror64(tmp, 17);
b10 -= b11 + k14;
b11 -= k15;
tmp = b9 ^ b8;
- b9 = (tmp >> 8) | (tmp << (64 - 8));
+ b9 = ror64(tmp, 8);
b8 -= b9 + k12;
b9 -= k13;
tmp = b7 ^ b6;
- b7 = (tmp >> 47) | (tmp << (64 - 47));
+ b7 = ror64(tmp, 47);
b6 -= b7 + k10;
b7 -= k11;
tmp = b5 ^ b4;
- b5 = (tmp >> 8) | (tmp << (64 - 8));
+ b5 = ror64(tmp, 8);
b4 -= b5 + k8;
b5 -= k9;
tmp = b3 ^ b2;
- b3 = (tmp >> 13) | (tmp << (64 - 13));
+ b3 = ror64(tmp, 13);
b2 -= b3 + k6;
b3 -= k7;
tmp = b1 ^ b0;
- b1 = (tmp >> 24) | (tmp << (64 - 24));
+ b1 = ror64(tmp, 24);
b0 -= b1 + k4;
b1 -= k5;
tmp = b7 ^ b12;
- b7 = (tmp >> 20) | (tmp << (64 - 20));
+ b7 = ror64(tmp, 20);
b12 -= b7;
tmp = b3 ^ b10;
- b3 = (tmp >> 37) | (tmp << (64 - 37));
+ b3 = ror64(tmp, 37);
b10 -= b3;
tmp = b5 ^ b8;
- b5 = (tmp >> 31) | (tmp << (64 - 31));
+ b5 = ror64(tmp, 31);
b8 -= b5;
tmp = b1 ^ b14;
- b1 = (tmp >> 23) | (tmp << (64 - 23));
+ b1 = ror64(tmp, 23);
b14 -= b1;
tmp = b9 ^ b4;
- b9 = (tmp >> 52) | (tmp << (64 - 52));
+ b9 = ror64(tmp, 52);
b4 -= b9;
tmp = b13 ^ b6;
- b13 = (tmp >> 35) | (tmp << (64 - 35));
+ b13 = ror64(tmp, 35);
b6 -= b13;
tmp = b11 ^ b2;
- b11 = (tmp >> 48) | (tmp << (64 - 48));
+ b11 = ror64(tmp, 48);
b2 -= b11;
tmp = b15 ^ b0;
- b15 = (tmp >> 9) | (tmp << (64 - 9));
+ b15 = ror64(tmp, 9);
b0 -= b15;
tmp = b9 ^ b10;
- b9 = (tmp >> 25) | (tmp << (64 - 25));
+ b9 = ror64(tmp, 25);
b10 -= b9;
tmp = b11 ^ b8;
- b11 = (tmp >> 44) | (tmp << (64 - 44));
+ b11 = ror64(tmp, 44);
b8 -= b11;
tmp = b13 ^ b14;
- b13 = (tmp >> 42) | (tmp << (64 - 42));
+ b13 = ror64(tmp, 42);
b14 -= b13;
tmp = b15 ^ b12;
- b15 = (tmp >> 19) | (tmp << (64 - 19));
+ b15 = ror64(tmp, 19);
b12 -= b15;
tmp = b1 ^ b6;
- b1 = (tmp >> 46) | (tmp << (64 - 46));
+ b1 = ror64(tmp, 46);
b6 -= b1;
tmp = b3 ^ b4;
- b3 = (tmp >> 47) | (tmp << (64 - 47));
+ b3 = ror64(tmp, 47);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 44) | (tmp << (64 - 44));
+ b5 = ror64(tmp, 44);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 31) | (tmp << (64 - 31));
+ b7 = ror64(tmp, 31);
b0 -= b7;
tmp = b1 ^ b8;
- b1 = (tmp >> 41) | (tmp << (64 - 41));
+ b1 = ror64(tmp, 41);
b8 -= b1;
tmp = b5 ^ b14;
- b5 = (tmp >> 42) | (tmp << (64 - 42));
+ b5 = ror64(tmp, 42);
b14 -= b5;
tmp = b3 ^ b12;
- b3 = (tmp >> 53) | (tmp << (64 - 53));
+ b3 = ror64(tmp, 53);
b12 -= b3;
tmp = b7 ^ b10;
- b7 = (tmp >> 4) | (tmp << (64 - 4));
+ b7 = ror64(tmp, 4);
b10 -= b7;
tmp = b15 ^ b4;
- b15 = (tmp >> 51) | (tmp << (64 - 51));
+ b15 = ror64(tmp, 51);
b4 -= b15;
tmp = b11 ^ b6;
- b11 = (tmp >> 56) | (tmp << (64 - 56));
+ b11 = ror64(tmp, 56);
b6 -= b11;
tmp = b13 ^ b2;
- b13 = (tmp >> 34) | (tmp << (64 - 34));
+ b13 = ror64(tmp, 34);
b2 -= b13;
tmp = b9 ^ b0;
- b9 = (tmp >> 16) | (tmp << (64 - 16));
+ b9 = ror64(tmp, 16);
b0 -= b9;
tmp = b15 ^ b14;
- b15 = (tmp >> 30) | (tmp << (64 - 30));
+ b15 = ror64(tmp, 30);
b14 -= b15 + k0 + t1;
b15 -= k1 + 3;
tmp = b13 ^ b12;
- b13 = (tmp >> 44) | (tmp << (64 - 44));
+ b13 = ror64(tmp, 44);
b12 -= b13 + k15;
b13 -= k16 + t0;
tmp = b11 ^ b10;
- b11 = (tmp >> 47) | (tmp << (64 - 47));
+ b11 = ror64(tmp, 47);
b10 -= b11 + k13;
b11 -= k14;
tmp = b9 ^ b8;
- b9 = (tmp >> 12) | (tmp << (64 - 12));
+ b9 = ror64(tmp, 12);
b8 -= b9 + k11;
b9 -= k12;
tmp = b7 ^ b6;
- b7 = (tmp >> 31) | (tmp << (64 - 31));
+ b7 = ror64(tmp, 31);
b6 -= b7 + k9;
b7 -= k10;
tmp = b5 ^ b4;
- b5 = (tmp >> 37) | (tmp << (64 - 37));
+ b5 = ror64(tmp, 37);
b4 -= b5 + k7;
b5 -= k8;
tmp = b3 ^ b2;
- b3 = (tmp >> 9) | (tmp << (64 - 9));
+ b3 = ror64(tmp, 9);
b2 -= b3 + k5;
b3 -= k6;
tmp = b1 ^ b0;
- b1 = (tmp >> 41) | (tmp << (64 - 41));
+ b1 = ror64(tmp, 41);
b0 -= b1 + k3;
b1 -= k4;
tmp = b7 ^ b12;
- b7 = (tmp >> 25) | (tmp << (64 - 25));
+ b7 = ror64(tmp, 25);
b12 -= b7;
tmp = b3 ^ b10;
- b3 = (tmp >> 16) | (tmp << (64 - 16));
+ b3 = ror64(tmp, 16);
b10 -= b3;
tmp = b5 ^ b8;
- b5 = (tmp >> 28) | (tmp << (64 - 28));
+ b5 = ror64(tmp, 28);
b8 -= b5;
tmp = b1 ^ b14;
- b1 = (tmp >> 47) | (tmp << (64 - 47));
+ b1 = ror64(tmp, 47);
b14 -= b1;
tmp = b9 ^ b4;
- b9 = (tmp >> 41) | (tmp << (64 - 41));
+ b9 = ror64(tmp, 41);
b4 -= b9;
tmp = b13 ^ b6;
- b13 = (tmp >> 48) | (tmp << (64 - 48));
+ b13 = ror64(tmp, 48);
b6 -= b13;
tmp = b11 ^ b2;
- b11 = (tmp >> 20) | (tmp << (64 - 20));
+ b11 = ror64(tmp, 20);
b2 -= b11;
tmp = b15 ^ b0;
- b15 = (tmp >> 5) | (tmp << (64 - 5));
+ b15 = ror64(tmp, 5);
b0 -= b15;
tmp = b9 ^ b10;
- b9 = (tmp >> 17) | (tmp << (64 - 17));
+ b9 = ror64(tmp, 17);
b10 -= b9;
tmp = b11 ^ b8;
- b11 = (tmp >> 59) | (tmp << (64 - 59));
+ b11 = ror64(tmp, 59);
b8 -= b11;
tmp = b13 ^ b14;
- b13 = (tmp >> 41) | (tmp << (64 - 41));
+ b13 = ror64(tmp, 41);
b14 -= b13;
tmp = b15 ^ b12;
- b15 = (tmp >> 34) | (tmp << (64 - 34));
+ b15 = ror64(tmp, 34);
b12 -= b15;
tmp = b1 ^ b6;
- b1 = (tmp >> 13) | (tmp << (64 - 13));
+ b1 = ror64(tmp, 13);
b6 -= b1;
tmp = b3 ^ b4;
- b3 = (tmp >> 51) | (tmp << (64 - 51));
+ b3 = ror64(tmp, 51);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 4) | (tmp << (64 - 4));
+ b5 = ror64(tmp, 4);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 33) | (tmp << (64 - 33));
+ b7 = ror64(tmp, 33);
b0 -= b7;
tmp = b1 ^ b8;
- b1 = (tmp >> 52) | (tmp << (64 - 52));
+ b1 = ror64(tmp, 52);
b8 -= b1;
tmp = b5 ^ b14;
- b5 = (tmp >> 23) | (tmp << (64 - 23));
+ b5 = ror64(tmp, 23);
b14 -= b5;
tmp = b3 ^ b12;
- b3 = (tmp >> 18) | (tmp << (64 - 18));
+ b3 = ror64(tmp, 18);
b12 -= b3;
tmp = b7 ^ b10;
- b7 = (tmp >> 49) | (tmp << (64 - 49));
+ b7 = ror64(tmp, 49);
b10 -= b7;
tmp = b15 ^ b4;
- b15 = (tmp >> 55) | (tmp << (64 - 55));
+ b15 = ror64(tmp, 55);
b4 -= b15;
tmp = b11 ^ b6;
- b11 = (tmp >> 10) | (tmp << (64 - 10));
+ b11 = ror64(tmp, 10);
b6 -= b11;
tmp = b13 ^ b2;
- b13 = (tmp >> 19) | (tmp << (64 - 19));
+ b13 = ror64(tmp, 19);
b2 -= b13;
tmp = b9 ^ b0;
- b9 = (tmp >> 38) | (tmp << (64 - 38));
+ b9 = ror64(tmp, 38);
b0 -= b9;
tmp = b15 ^ b14;
- b15 = (tmp >> 37) | (tmp << (64 - 37));
+ b15 = ror64(tmp, 37);
b14 -= b15 + k16 + t0;
b15 -= k0 + 2;
tmp = b13 ^ b12;
- b13 = (tmp >> 22) | (tmp << (64 - 22));
+ b13 = ror64(tmp, 22);
b12 -= b13 + k14;
b13 -= k15 + t2;
tmp = b11 ^ b10;
- b11 = (tmp >> 17) | (tmp << (64 - 17));
+ b11 = ror64(tmp, 17);
b10 -= b11 + k12;
b11 -= k13;
tmp = b9 ^ b8;
- b9 = (tmp >> 8) | (tmp << (64 - 8));
+ b9 = ror64(tmp, 8);
b8 -= b9 + k10;
b9 -= k11;
tmp = b7 ^ b6;
- b7 = (tmp >> 47) | (tmp << (64 - 47));
+ b7 = ror64(tmp, 47);
b6 -= b7 + k8;
b7 -= k9;
tmp = b5 ^ b4;
- b5 = (tmp >> 8) | (tmp << (64 - 8));
+ b5 = ror64(tmp, 8);
b4 -= b5 + k6;
b5 -= k7;
tmp = b3 ^ b2;
- b3 = (tmp >> 13) | (tmp << (64 - 13));
+ b3 = ror64(tmp, 13);
b2 -= b3 + k4;
b3 -= k5;
tmp = b1 ^ b0;
- b1 = (tmp >> 24) | (tmp << (64 - 24));
+ b1 = ror64(tmp, 24);
b0 -= b1 + k2;
b1 -= k3;
tmp = b7 ^ b12;
- b7 = (tmp >> 20) | (tmp << (64 - 20));
+ b7 = ror64(tmp, 20);
b12 -= b7;
tmp = b3 ^ b10;
- b3 = (tmp >> 37) | (tmp << (64 - 37));
+ b3 = ror64(tmp, 37);
b10 -= b3;
tmp = b5 ^ b8;
- b5 = (tmp >> 31) | (tmp << (64 - 31));
+ b5 = ror64(tmp, 31);
b8 -= b5;
tmp = b1 ^ b14;
- b1 = (tmp >> 23) | (tmp << (64 - 23));
+ b1 = ror64(tmp, 23);
b14 -= b1;
tmp = b9 ^ b4;
- b9 = (tmp >> 52) | (tmp << (64 - 52));
+ b9 = ror64(tmp, 52);
b4 -= b9;
tmp = b13 ^ b6;
- b13 = (tmp >> 35) | (tmp << (64 - 35));
+ b13 = ror64(tmp, 35);
b6 -= b13;
tmp = b11 ^ b2;
- b11 = (tmp >> 48) | (tmp << (64 - 48));
+ b11 = ror64(tmp, 48);
b2 -= b11;
tmp = b15 ^ b0;
- b15 = (tmp >> 9) | (tmp << (64 - 9));
+ b15 = ror64(tmp, 9);
b0 -= b15;
tmp = b9 ^ b10;
- b9 = (tmp >> 25) | (tmp << (64 - 25));
+ b9 = ror64(tmp, 25);
b10 -= b9;
tmp = b11 ^ b8;
- b11 = (tmp >> 44) | (tmp << (64 - 44));
+ b11 = ror64(tmp, 44);
b8 -= b11;
tmp = b13 ^ b14;
- b13 = (tmp >> 42) | (tmp << (64 - 42));
+ b13 = ror64(tmp, 42);
b14 -= b13;
tmp = b15 ^ b12;
- b15 = (tmp >> 19) | (tmp << (64 - 19));
+ b15 = ror64(tmp, 19);
b12 -= b15;
tmp = b1 ^ b6;
- b1 = (tmp >> 46) | (tmp << (64 - 46));
+ b1 = ror64(tmp, 46);
b6 -= b1;
tmp = b3 ^ b4;
- b3 = (tmp >> 47) | (tmp << (64 - 47));
+ b3 = ror64(tmp, 47);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 44) | (tmp << (64 - 44));
+ b5 = ror64(tmp, 44);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 31) | (tmp << (64 - 31));
+ b7 = ror64(tmp, 31);
b0 -= b7;
tmp = b1 ^ b8;
- b1 = (tmp >> 41) | (tmp << (64 - 41));
+ b1 = ror64(tmp, 41);
b8 -= b1;
tmp = b5 ^ b14;
- b5 = (tmp >> 42) | (tmp << (64 - 42));
+ b5 = ror64(tmp, 42);
b14 -= b5;
tmp = b3 ^ b12;
- b3 = (tmp >> 53) | (tmp << (64 - 53));
+ b3 = ror64(tmp, 53);
b12 -= b3;
tmp = b7 ^ b10;
- b7 = (tmp >> 4) | (tmp << (64 - 4));
+ b7 = ror64(tmp, 4);
b10 -= b7;
tmp = b15 ^ b4;
- b15 = (tmp >> 51) | (tmp << (64 - 51));
+ b15 = ror64(tmp, 51);
b4 -= b15;
tmp = b11 ^ b6;
- b11 = (tmp >> 56) | (tmp << (64 - 56));
+ b11 = ror64(tmp, 56);
b6 -= b11;
tmp = b13 ^ b2;
- b13 = (tmp >> 34) | (tmp << (64 - 34));
+ b13 = ror64(tmp, 34);
b2 -= b13;
tmp = b9 ^ b0;
- b9 = (tmp >> 16) | (tmp << (64 - 16));
+ b9 = ror64(tmp, 16);
b0 -= b9;
tmp = b15 ^ b14;
- b15 = (tmp >> 30) | (tmp << (64 - 30));
+ b15 = ror64(tmp, 30);
b14 -= b15 + k15 + t2;
b15 -= k16 + 1;
tmp = b13 ^ b12;
- b13 = (tmp >> 44) | (tmp << (64 - 44));
+ b13 = ror64(tmp, 44);
b12 -= b13 + k13;
b13 -= k14 + t1;
tmp = b11 ^ b10;
- b11 = (tmp >> 47) | (tmp << (64 - 47));
+ b11 = ror64(tmp, 47);
b10 -= b11 + k11;
b11 -= k12;
tmp = b9 ^ b8;
- b9 = (tmp >> 12) | (tmp << (64 - 12));
+ b9 = ror64(tmp, 12);
b8 -= b9 + k9;
b9 -= k10;
tmp = b7 ^ b6;
- b7 = (tmp >> 31) | (tmp << (64 - 31));
+ b7 = ror64(tmp, 31);
b6 -= b7 + k7;
b7 -= k8;
tmp = b5 ^ b4;
- b5 = (tmp >> 37) | (tmp << (64 - 37));
+ b5 = ror64(tmp, 37);
b4 -= b5 + k5;
b5 -= k6;
tmp = b3 ^ b2;
- b3 = (tmp >> 9) | (tmp << (64 - 9));
+ b3 = ror64(tmp, 9);
b2 -= b3 + k3;
b3 -= k4;
tmp = b1 ^ b0;
- b1 = (tmp >> 41) | (tmp << (64 - 41));
+ b1 = ror64(tmp, 41);
b0 -= b1 + k1;
b1 -= k2;
tmp = b7 ^ b12;
- b7 = (tmp >> 25) | (tmp << (64 - 25));
+ b7 = ror64(tmp, 25);
b12 -= b7;
tmp = b3 ^ b10;
- b3 = (tmp >> 16) | (tmp << (64 - 16));
+ b3 = ror64(tmp, 16);
b10 -= b3;
tmp = b5 ^ b8;
- b5 = (tmp >> 28) | (tmp << (64 - 28));
+ b5 = ror64(tmp, 28);
b8 -= b5;
tmp = b1 ^ b14;
- b1 = (tmp >> 47) | (tmp << (64 - 47));
+ b1 = ror64(tmp, 47);
b14 -= b1;
tmp = b9 ^ b4;
- b9 = (tmp >> 41) | (tmp << (64 - 41));
+ b9 = ror64(tmp, 41);
b4 -= b9;
tmp = b13 ^ b6;
- b13 = (tmp >> 48) | (tmp << (64 - 48));
+ b13 = ror64(tmp, 48);
b6 -= b13;
tmp = b11 ^ b2;
- b11 = (tmp >> 20) | (tmp << (64 - 20));
+ b11 = ror64(tmp, 20);
b2 -= b11;
tmp = b15 ^ b0;
- b15 = (tmp >> 5) | (tmp << (64 - 5));
+ b15 = ror64(tmp, 5);
b0 -= b15;
tmp = b9 ^ b10;
- b9 = (tmp >> 17) | (tmp << (64 - 17));
+ b9 = ror64(tmp, 17);
b10 -= b9;
tmp = b11 ^ b8;
- b11 = (tmp >> 59) | (tmp << (64 - 59));
+ b11 = ror64(tmp, 59);
b8 -= b11;
tmp = b13 ^ b14;
- b13 = (tmp >> 41) | (tmp << (64 - 41));
+ b13 = ror64(tmp, 41);
b14 -= b13;
tmp = b15 ^ b12;
- b15 = (tmp >> 34) | (tmp << (64 - 34));
+ b15 = ror64(tmp, 34);
b12 -= b15;
tmp = b1 ^ b6;
- b1 = (tmp >> 13) | (tmp << (64 - 13));
+ b1 = ror64(tmp, 13);
b6 -= b1;
tmp = b3 ^ b4;
- b3 = (tmp >> 51) | (tmp << (64 - 51));
+ b3 = ror64(tmp, 51);
b4 -= b3;
tmp = b5 ^ b2;
- b5 = (tmp >> 4) | (tmp << (64 - 4));
+ b5 = ror64(tmp, 4);
b2 -= b5;
tmp = b7 ^ b0;
- b7 = (tmp >> 33) | (tmp << (64 - 33));
+ b7 = ror64(tmp, 33);
b0 -= b7;
tmp = b1 ^ b8;
- b1 = (tmp >> 52) | (tmp << (64 - 52));
+ b1 = ror64(tmp, 52);
b8 -= b1;
tmp = b5 ^ b14;
- b5 = (tmp >> 23) | (tmp << (64 - 23));
+ b5 = ror64(tmp, 23);
b14 -= b5;
tmp = b3 ^ b12;
- b3 = (tmp >> 18) | (tmp << (64 - 18));
+ b3 = ror64(tmp, 18);
b12 -= b3;
tmp = b7 ^ b10;
- b7 = (tmp >> 49) | (tmp << (64 - 49));
+ b7 = ror64(tmp, 49);
b10 -= b7;
tmp = b15 ^ b4;
- b15 = (tmp >> 55) | (tmp << (64 - 55));
+ b15 = ror64(tmp, 55);
b4 -= b15;
tmp = b11 ^ b6;
- b11 = (tmp >> 10) | (tmp << (64 - 10));
+ b11 = ror64(tmp, 10);
b6 -= b11;
tmp = b13 ^ b2;
- b13 = (tmp >> 19) | (tmp << (64 - 19));
+ b13 = ror64(tmp, 19);
b2 -= b13;
tmp = b9 ^ b0;
- b9 = (tmp >> 38) | (tmp << (64 - 38));
+ b9 = ror64(tmp, 38);
b0 -= b9;
tmp = b15 ^ b14;
- b15 = (tmp >> 37) | (tmp << (64 - 37));
+ b15 = ror64(tmp, 37);
b14 -= b15 + k14 + t1;
b15 -= k15;
tmp = b13 ^ b12;
- b13 = (tmp >> 22) | (tmp << (64 - 22));
+ b13 = ror64(tmp, 22);
b12 -= b13 + k12;
b13 -= k13 + t0;
tmp = b11 ^ b10;
- b11 = (tmp >> 17) | (tmp << (64 - 17));
+ b11 = ror64(tmp, 17);
b10 -= b11 + k10;
b11 -= k11;
tmp = b9 ^ b8;
- b9 = (tmp >> 8) | (tmp << (64 - 8));
+ b9 = ror64(tmp, 8);
b8 -= b9 + k8;
b9 -= k9;
tmp = b7 ^ b6;
- b7 = (tmp >> 47) | (tmp << (64 - 47));
+ b7 = ror64(tmp, 47);
b6 -= b7 + k6;
b7 -= k7;
tmp = b5 ^ b4;
- b5 = (tmp >> 8) | (tmp << (64 - 8));
+ b5 = ror64(tmp, 8);
b4 -= b5 + k4;
b5 -= k5;
tmp = b3 ^ b2;
- b3 = (tmp >> 13) | (tmp << (64 - 13));
+ b3 = ror64(tmp, 13);
b2 -= b3 + k2;
b3 -= k3;
tmp = b1 ^ b0;
- b1 = (tmp >> 24) | (tmp << (64 - 24));
+ b1 = ror64(tmp, 24);
b0 -= b1 + k0;
b1 -= k1;
diff --git a/drivers/staging/slicoss/slicoss.c b/drivers/staging/slicoss/slicoss.c
index 6d50fc4..ac126d4 100644
--- a/drivers/staging/slicoss/slicoss.c
+++ b/drivers/staging/slicoss/slicoss.c
@@ -898,6 +898,7 @@
{
struct slic_upr *upr;
__iomem struct slic_regs *slic_regs = adapter->slic_regs;
+
upr = adapter->upr_list;
if (!upr)
return;
@@ -1144,7 +1145,7 @@
/*
* Compute a checksum of the EEPROM according to RFC 1071.
*/
-static u16 slic_eeprom_cksum(void *eeprom, unsigned len)
+static u16 slic_eeprom_cksum(void *eeprom, unsigned int len)
{
u16 *wp = eeprom;
u32 checksum = 0;
@@ -1855,6 +1856,11 @@
ihcmd->u.slic_buffers.totlen = skb->len;
phys_addr = pci_map_single(adapter->pcidev, skb->data, skb->len,
PCI_DMA_TODEVICE);
+ if (pci_dma_mapping_error(adapter->pcidev, phys_addr)) {
+ kfree_skb(skb);
+ dev_err(&adapter->pcidev->dev, "DMA mapping error\n");
+ return;
+ }
ihcmd->u.slic_buffers.bufs[0].paddrl = SLIC_GET_ADDR_LOW(phys_addr);
ihcmd->u.slic_buffers.bufs[0].paddrh = SLIC_GET_ADDR_HIGH(phys_addr);
ihcmd->u.slic_buffers.bufs[0].length = skb->len;
diff --git a/drivers/staging/sm750fb/ddk750_chip.c b/drivers/staging/sm750fb/ddk750_chip.c
index 95f7cae..f80ee77 100644
--- a/drivers/staging/sm750fb/ddk750_chip.c
+++ b/drivers/staging/sm750fb/ddk750_chip.c
@@ -306,7 +306,7 @@
unsigned int input, request;
unsigned int tmpClock, ret;
const int max_OD = 3;
- int max_d;
+ int max_d = 6;
if (getChipType() == SM750LE) {
/* SM750LE don't have prgrammable PLL and M/N values to work on.
diff --git a/drivers/staging/speakup/main.c b/drivers/staging/speakup/main.c
index a22fb075..97ca4ec 100644
--- a/drivers/staging/speakup/main.c
+++ b/drivers/staging/speakup/main.c
@@ -263,7 +263,7 @@
static unsigned char get_attributes(struct vc_data *vc, u16 *pos)
{
pos = screen_pos(vc, pos - (u16 *)vc->vc_origin, 1);
- return (u_char) (scr_readw(pos) >> 8);
+ return (scr_readw(pos) & ~vc->vc_hi_font_mask) >> 8;
}
static void speakup_date(struct vc_data *vc)
@@ -473,8 +473,10 @@
w = scr_readw(pos);
c = w & 0xff;
- if (w & vc->vc_hi_font_mask)
+ if (w & vc->vc_hi_font_mask) {
+ w &= ~vc->vc_hi_font_mask;
c |= 0x100;
+ }
ch = inverse_translate(vc, c, 0);
*attribs = (w & 0xff00) >> 8;
diff --git a/drivers/staging/speakup/selection.c b/drivers/staging/speakup/selection.c
index 41ef099..0149edc 100644
--- a/drivers/staging/speakup/selection.c
+++ b/drivers/staging/speakup/selection.c
@@ -150,7 +150,7 @@
add_wait_queue(&vc->paste_wait, &wait);
while (sel_buffer && sel_buffer_lth > pasted) {
set_current_state(TASK_INTERRUPTIBLE);
- if (test_bit(TTY_THROTTLED, &tty->flags)) {
+ if (tty_throttled(tty)) {
schedule();
continue;
}
diff --git a/drivers/staging/speakup/serialio.h b/drivers/staging/speakup/serialio.h
index 1b39921..3ad7ff0 100644
--- a/drivers/staging/speakup/serialio.h
+++ b/drivers/staging/speakup/serialio.h
@@ -6,6 +6,7 @@
#ifndef __sparc__
#include <linux/serial.h>
#endif
+#include <linux/serial_core.h>
/*
* this is cut&paste from 8250.h. Get rid of the structure, the definitions
@@ -16,7 +17,7 @@
unsigned int baud_base;
unsigned int port;
unsigned int irq;
- unsigned int flags; /* unused */
+ upf_t flags; /* unused */
};
/* countdown values for serial timeouts in us */
diff --git a/drivers/staging/unisys/Documentation/ABI/sysfs-platform-visorchipset b/drivers/staging/unisys/Documentation/ABI/sysfs-platform-visorchipset
index b0498ff..c2359de 100644
--- a/drivers/staging/unisys/Documentation/ABI/sysfs-platform-visorchipset
+++ b/drivers/staging/unisys/Documentation/ABI/sysfs-platform-visorchipset
@@ -50,20 +50,6 @@
commission the guest.
Users: sparmaintainer@unisys.com
-What: guest/chipsetready
-Date: 7/18/2014
-KernelVersion: TBD
-Contact: sparmaintainer@unisys.com
-Description: This entry is used by Unisys application software on the guest
- to acknowledge completion of specific events for integration
- purposes, but these acknowledgements are not required for the
- guest to operate correctly. The interface accepts one of two
- strings: MODULES_LOADED to indicate that the s-Par driver
- modules have been loaded successfully, or CALLHOMEDISK_MOUNTED,
- which indicates that the disk used to support call home services
- has been successfully mounted.
-Users: sparmaintainer@unisys.com
-
What: parahotplug/deviceenabled
Date: 7/18/2014
KernelVersion: TBD
diff --git a/drivers/staging/unisys/Documentation/overview.txt b/drivers/staging/unisys/Documentation/overview.txt
index c2d8dd4..1146c1c 100644
--- a/drivers/staging/unisys/Documentation/overview.txt
+++ b/drivers/staging/unisys/Documentation/overview.txt
@@ -137,12 +137,6 @@
* The resume() function is the "book-end" to pause(), and is described above.
-If/when a function driver creates a Linux device (that needs to be accessed
-from usermode), it calls visorbus_registerdevnode(), passing the major and
-minor number of the device. (Of course not all function drivers will need
-to do this.) This simply creates the appropriate "devmajorminor" sysfs entry
-described below, so that a hotplug script can use it to create a device node.
-
2.1.3. sysfs Advertised Information
-----------------------------------
@@ -197,19 +191,6 @@
if the appropriate function driver has not
been loaded yet.
- devmajorminor
-
- <devname> if applicable, each file here identifies (via
- ... its file contents) the
- "<major>:<minor>" needed for a device node to
- enable access from usermode. There is exactly
- one file here for each different device node
- that can be accessed (from usermode). Note
- that this info is provided by a particular
- function driver, so these will not exist
- until AFTER the appropriate function driver
- controlling this device class is loaded.
-
channel properties of the device channel (all in
ascii text format)
diff --git a/drivers/staging/unisys/Documentation/proc-entries.txt b/drivers/staging/unisys/Documentation/proc-entries.txt
deleted file mode 100644
index 426f92b..0000000
--- a/drivers/staging/unisys/Documentation/proc-entries.txt
+++ /dev/null
@@ -1,93 +0,0 @@
- s-Par Proc Entries
-This document describes the proc entries created by the Unisys s-Par modules.
-
-Support Module Entries
-These entries are provided primarily for debugging.
-
-/proc/uislib/info: This entry contains debugging information for the
-uislib module, including bus information and memory usage.
-
-/proc/visorchipset/controlvm: This directory contains debugging
-entries for the controlvm channel used by visorchipset.
-
-/proc/uislib/platform: This entry is used to display the platform
-number this node is in the system. For some guests, this may be
-invalid.
-
-/proc/visorchipset/chipsetready: This entry is written to by scripts
-to signify that any user level activity has been completed before the
-guest can be considered running and is shown as running in the s-Par
-UI.
-
-Device Entries
-These entries provide status of the devices shared by a service partition.
-
-/proc/uislib/vbus: this is a directory containing entries for each
-virtual bus. Each numbered sub-directory contains an info entry, which
-describes the devices that appear on that bus.
-
-/proc/uislib/cycles_before_wait: This entry is used to tune
-performance, by setting the number of cycles we wait before going idle
-when in polling mode. A longer time will reduce message latency but
-spend more processing time polling.
-
-/proc/uislib/smart_wakeup: This entry is used to tune performance, by
-enabling or disabling smart wakeup.
-
-/proc/virthba/info: This entry contains debugging information for the
-virthba module, including interrupt information and memory usage.
-
-/proc/virthba/enable_ints: This entry controls interrupt use by the
-virthba module. Writing a 0 to this entry will disable interrupts.
-
-/proc/virtnic/info: This entry contains debugging information for the
-virtnic module, including interrupt information, send and receive
-counts, and other device information.
-
-/proc/virtnic/ethX: This is a directory containing entries for each
-virtual NIC. Each named subdirectory contains two entries,
-clientstring and zone.
-
-/proc/virtpci/info: This entry contains debugging information for the
-virtpci module, including virtual PCI bus information and device
-locations.
-
-/proc/virtnic/enable_ints: This entry controls interrupt use by the
-virtnic module. Writing a 0 to this entry will disable interrupts.
-
-Visorconinclient, visordiag, visornoop, visorserialclient, and
-visorvideoclient Entries
-
-The entries in proc for these modules all follow the same
-pattern. Each module has its own proc directory with the same name,
-e.g. visordiag presents a /proc/visordiag directory. Inside of the
-module's directory are a device directory, which contains one numbered
-directory for each device provided by that module. Each device has a
-diag entry that presents the device number and visorbus name for that
-device. The module directory also has a driver/diag entry, which
-reports the corresponding s-Par version number of the driver.
-
-Automated Installation Entries
-
-These entries are used to pass information between the s-Par platform
-and the Linux-based installation and recovery tool. These values are
-read/write, however, the guest can only reset them to 0, or report an
-error status through the installer entry. The values are only set via
-s-Par's firmware interface, to help prevent accidentally booting into
-the tool.
-
-/proc/visorchipset/boottotool: This entry instructs s-Par that the
-next reboot will launch the installation and recovery tool. If set to
-0, the next boot will happen according to the UEFI boot manager
-settings.
-
-/proc/visorchipset/toolaction: This entry indicates the installation
-and recovery tool mode requested for the next boot.
-
-/proc/visorchipset/installer: this entry is used by the installation
-and recovery tool to pass status and result information back to the
-s-Par firmware.
-
-/proc/visorchipset/partition: This directory contains the guest
-partition configuration data for each virtual bus, for use during
-installation and at runtime for s-Par service partitions.
diff --git a/drivers/staging/unisys/MAINTAINERS b/drivers/staging/unisys/MAINTAINERS
index cc46e37..1f0425b 100644
--- a/drivers/staging/unisys/MAINTAINERS
+++ b/drivers/staging/unisys/MAINTAINERS
@@ -2,5 +2,4 @@
M: David Kershner <sparmaintainer@unisys.com>
S: Maintained
F: Documentation/s-Par/overview.txt
-F: Documentation/s-Par/proc-entries.txt
F: drivers/staging/unisys/
diff --git a/drivers/staging/unisys/include/channel.h b/drivers/staging/unisys/include/channel.h
index 5af59a5..db4e6b2 100644
--- a/drivers/staging/unisys/include/channel.h
+++ b/drivers/staging/unisys/include/channel.h
@@ -76,9 +76,9 @@
};
static inline const u8 *
-ULTRA_CHANNELCLI_STRING(u32 v)
+ULTRA_CHANNELCLI_STRING(u32 state)
{
- switch (v) {
+ switch (state) {
case CHANNELCLI_DETACHED:
return (const u8 *)("DETACHED");
case CHANNELCLI_DISABLED:
@@ -411,7 +411,7 @@
mb(); /* required for channel synch */
}
if (readl(&hdr->cli_state_os) == CHANNELCLI_OWNED) {
- if (readb(&hdr->cli_error_os) != 0) {
+ if (readb(&hdr->cli_error_os)) {
/* we are in an error msg throttling state;
* come out of it
*/
@@ -459,7 +459,7 @@
mb(); /* required for channel synch */
return 0;
}
- if (readb(&hdr->cli_error_os) != 0) {
+ if (readb(&hdr->cli_error_os)) {
/* we are in an error msg throttling state; come out of it */
pr_info("%s Channel OS client acquire now successful\n", id);
writeb(0, &hdr->cli_error_os);
@@ -472,7 +472,7 @@
{
struct channel_header __iomem *hdr = ch;
- if (readb(&hdr->cli_error_os) != 0) {
+ if (readb(&hdr->cli_error_os)) {
/* we are in an error msg throttling state; come out of it */
pr_info("%s Channel OS client error state cleared\n", id);
writeb(0, &hdr->cli_error_os);
diff --git a/drivers/staging/unisys/include/iochannel.h b/drivers/staging/unisys/include/iochannel.h
index 880d9f0..5ccf814 100644
--- a/drivers/staging/unisys/include/iochannel.h
+++ b/drivers/staging/unisys/include/iochannel.h
@@ -253,48 +253,6 @@
/* SCSI device version for no disk inquiry result */
#define SCSI_SPC2_VER 4 /* indicates SCSI SPC2 (SPC3 is 5) */
-/* Windows and Linux want different things for a non-existent lun. So, we'll let
- * caller pass in the peripheral qualifier and type.
- * NOTE:[4] SCSI returns (n-4); so we return length-1-4 or length-5.
- */
-
-#define SET_NO_DISK_INQUIRY_RESULT(buf, len, lun, lun0notpresent, notpresent) \
- do { \
- memset(buf, 0, \
- MINNUM(len, \
- (unsigned int)NO_DISK_INQUIRY_RESULT_LEN)); \
- buf[2] = (u8)SCSI_SPC2_VER; \
- if (lun == 0) { \
- buf[0] = (u8)lun0notpresent; \
- buf[3] = (u8)DEV_HISUPPORT; \
- } else \
- buf[0] = (u8)notpresent; \
- buf[4] = (u8)( \
- MINNUM(len, \
- (unsigned int)NO_DISK_INQUIRY_RESULT_LEN) - 5);\
- if (len >= NO_DISK_INQUIRY_RESULT_LEN) { \
- buf[8] = 'D'; \
- buf[9] = 'E'; \
- buf[10] = 'L'; \
- buf[11] = 'L'; \
- buf[16] = 'P'; \
- buf[17] = 'S'; \
- buf[18] = 'E'; \
- buf[19] = 'U'; \
- buf[20] = 'D'; \
- buf[21] = 'O'; \
- buf[22] = ' '; \
- buf[23] = 'D'; \
- buf[24] = 'E'; \
- buf[25] = 'V'; \
- buf[26] = 'I'; \
- buf[27] = 'C'; \
- buf[28] = 'E'; \
- buf[30] = ' '; \
- buf[31] = '.'; \
- } \
- } while (0)
-
/* Struct & Defines to support sense information. */
/* The following struct is returned in sensebuf field in uiscmdrsp_scsi. It is
diff --git a/drivers/staging/unisys/include/visorbus.h b/drivers/staging/unisys/include/visorbus.h
index 2a64a9c..9baf1ec 100644
--- a/drivers/staging/unisys/include/visorbus.h
+++ b/drivers/staging/unisys/include/visorbus.h
@@ -61,54 +61,55 @@
const char *name;
};
-/** Information provided by each visor driver when it registers with the
- * visorbus driver.
+/**
+ * struct visor_driver - Information provided by each visor driver when it
+ * registers with the visorbus driver.
+ * @name: Name of the visor driver.
+ * @version: The numbered version of the driver (x.x.xxx).
+ * @vertag: A human readable version string.
+ * @owner: The module owner.
+ * @channel_types: Types of channels handled by this driver, ending with
+ * a zero GUID. Our specialized BUS.match() method knows
+ * about this list, and uses it to determine whether this
+ * driver will in fact handle a new device that it has
+ * detected.
+ * @probe: Called when a new device comes online, by our probe()
+ * function specified by driver.probe() (triggered
+ * ultimately by some call to driver_register(),
+ * bus_add_driver(), or driver_attach()).
+ * @remove: Called when a new device is removed, by our remove()
+ * function specified by driver.remove() (triggered
+ * ultimately by some call to device_release_driver()).
+ * @channel_interrupt: Called periodically, whenever there is a possiblity
+ * that "something interesting" may have happened to the
+ * channel.
+ * @pause: Called to initiate a change of the device's state. If
+ * the return valu`e is < 0, there was an error and the
+ * state transition will NOT occur. If the return value
+ * is >= 0, then the state transition was INITIATED
+ * successfully, and complete_func() will be called (or
+ * was just called) with the final status when either the
+ * state transition fails or completes successfully.
+ * @resume: Behaves similar to pause.
+ * @driver: Private reference to the device driver. For use by bus
+ * driver only.
+ * @version_attr: Private version field. For use by bus driver only.
*/
struct visor_driver {
const char *name;
const char *version;
const char *vertag;
- const char *build_date;
- const char *build_time;
struct module *owner;
-
- /** Types of channels handled by this driver, ending with 0 GUID.
- * Our specialized BUS.match() method knows about this list, and
- * uses it to determine whether this driver will in fact handle a
- * new device that it has detected.
- */
struct visor_channeltype_descriptor *channel_types;
-
- /** Called when a new device comes online, by our probe() function
- * specified by driver.probe() (triggered ultimately by some call
- * to driver_register() / bus_add_driver() / driver_attach()).
- */
int (*probe)(struct visor_device *dev);
-
- /** Called when a new device is removed, by our remove() function
- * specified by driver.remove() (triggered ultimately by some call
- * to device_release_driver()).
- */
void (*remove)(struct visor_device *dev);
-
- /** Called periodically, whenever there is a possibility that
- * "something interesting" may have happened to the channel state.
- */
void (*channel_interrupt)(struct visor_device *dev);
-
- /** Called to initiate a change of the device's state. If the return
- * valu`e is < 0, there was an error and the state transition will NOT
- * occur. If the return value is >= 0, then the state transition was
- * INITIATED successfully, and complete_func() will be called (or was
- * just called) with the final status when either the state transition
- * fails or completes successfully.
- */
int (*pause)(struct visor_device *dev,
visorbus_state_complete_func complete_func);
int (*resume)(struct visor_device *dev,
visorbus_state_complete_func complete_func);
- /** These fields are for private use by the bus driver only. */
+ /* These fields are for private use by the bus driver only. */
struct device_driver driver;
struct driver_attribute version_attr;
};
@@ -116,48 +117,58 @@
#define to_visor_driver(x) ((x) ? \
(container_of(x, struct visor_driver, driver)) : (NULL))
-/** A device type for things "plugged" into the visorbus bus */
+/**
+ * struct visor_device - A device type for things "plugged" into the visorbus
+ * bus
+ * visorchannel: Points to the channel that the device is
+ * associated with.
+ * channel_type_guid: Identifies the channel type to the bus driver.
+ * device: Device struct meant for use by the bus driver
+ * only.
+ * list_all: Used by the bus driver to enumerate devices.
+ * periodic_work: Device work queue. Private use by bus driver
+ * only.
+ * being_removed: Indicates that the device is being removed from
+ * the bus. Private bus driver use only.
+ * visordriver_callback_lock: Used by the bus driver to lock when handling
+ * channel events.
+ * pausing: Indicates that a change towards a paused state.
+ * is in progress. Only modified by the bus driver.
+ * resuming: Indicates that a change towards a running state
+ * is in progress. Only modified by the bus driver.
+ * chipset_bus_no: Private field used by the bus driver.
+ * chipset_dev_no: Private field used the bus driver.
+ * state: Used to indicate the current state of the
+ * device.
+ * inst: Unique GUID for this instance of the device.
+ * name: Name of the device.
+ * pending_msg_hdr: For private use by bus driver to respond to
+ * hypervisor requests.
+ * vbus_hdr_info: A pointer to header info. Private use by bus
+ * driver.
+ * partition_uuid: Indicates client partion id. This should be the
+ * same across all visor_devices in the current
+ * guest. Private use by bus driver only.
+ */
struct visor_device {
- /** visor driver can use the visorchannel member with the functions
- * defined in visorchannel.h to access the channel
- */
struct visorchannel *visorchannel;
uuid_le channel_type_guid;
- u64 channel_bytes;
-
- /** These fields are for private use by the bus driver only.
- * A notable exception is that the visor driver can use
- * visor_get_drvdata() and visor_set_drvdata() to retrieve or stash
- * private visor driver specific data within the device member.
- */
+ /* These fields are for private use by the bus driver only. */
struct device device;
struct list_head list_all;
struct periodic_work *periodic_work;
bool being_removed;
- bool responded_to_device_create;
- struct kobject kobjdevmajorminor; /* visorbus<x>/dev<y>/devmajorminor/*/
- struct {
- int major, minor;
- void *attr; /* private use by devmajorminor_attr.c you can
- * change this constant to whatever you want
- */
- } devnodes[5];
- /* the code will detect and behave appropriately) */
struct semaphore visordriver_callback_lock;
bool pausing;
bool resuming;
u32 chipset_bus_no;
u32 chipset_dev_no;
struct visorchipset_state state;
- uuid_le type;
uuid_le inst;
u8 *name;
- u8 *description;
struct controlvm_message_header *pending_msg_hdr;
void *vbus_hdr_info;
- u32 switch_no;
- u32 internal_port_no;
uuid_le partition_uuid;
};
@@ -174,8 +185,6 @@
unsigned long nbytes);
int visorbus_clear_channel(struct visor_device *dev,
unsigned long offset, u8 ch, unsigned long nbytes);
-int visorbus_registerdevnode(struct visor_device *dev,
- const char *name, int major, int minor);
void visorbus_enable_channel_interrupts(struct visor_device *dev);
void visorbus_disable_channel_interrupts(struct visor_device *dev);
#endif
diff --git a/drivers/staging/unisys/visorbus/visorbus_main.c b/drivers/staging/unisys/visorbus/visorbus_main.c
index 533bb5b..3a147db 100644
--- a/drivers/staging/unisys/visorbus/visorbus_main.c
+++ b/drivers/staging/unisys/visorbus/visorbus_main.c
@@ -33,6 +33,9 @@
static int visorbus_debugref;
#define SERIALLOOPBACKCHANADDR (100 * 1024 * 1024)
+/* Display string that is guaranteed to be no longer the 99 characters*/
+#define LINESIZE 99
+
#define CURRENT_FILE_PC VISOR_BUS_PC_visorbus_main_c
#define POLLJIFFIES_TESTWORK 100
#define POLLJIFFIES_NORMALCHANNEL 10
@@ -182,7 +185,6 @@
visorbus_match(struct device *xdev, struct device_driver *xdrv)
{
uuid_le channel_type;
- int rc = 0;
int i;
struct visor_device *dev;
struct visor_driver *drv;
@@ -190,26 +192,23 @@
dev = to_visor_device(xdev);
drv = to_visor_driver(xdrv);
channel_type = visorchannel_get_uuid(dev->visorchannel);
- if (visorbus_forcematch) {
- rc = 1;
- goto away;
- }
- if (visorbus_forcenomatch)
- goto away;
+ if (visorbus_forcematch)
+ return 1;
+ if (visorbus_forcenomatch)
+ return 0;
if (!drv->channel_types)
- goto away;
+ return 0;
+
for (i = 0;
(uuid_le_cmp(drv->channel_types[i].guid, NULL_UUID_LE) != 0) ||
(drv->channel_types[i].name);
i++)
if (uuid_le_cmp(drv->channel_types[i].guid,
- channel_type) == 0) {
- rc = i + 1;
- goto away;
- }
-away:
- return rc;
+ channel_type) == 0)
+ return i + 1;
+
+ return 0;
}
/** This is called when device_unregister() is called for the bus device
@@ -243,180 +242,6 @@
kfree(dev);
}
-/* Implement publishing of device node attributes under:
- *
- * /sys/bus/visorbus<x>/dev<y>/devmajorminor
- *
- */
-
-#define to_devmajorminor_attr(_attr) \
- container_of(_attr, struct devmajorminor_attribute, attr)
-#define to_visor_device_from_kobjdevmajorminor(obj) \
- container_of(obj, struct visor_device, kobjdevmajorminor)
-
-struct devmajorminor_attribute {
- struct attribute attr;
- int slot;
- ssize_t (*show)(struct visor_device *, int slot, char *buf);
- ssize_t (*store)(struct visor_device *, int slot, const char *buf,
- size_t count);
-};
-
-static ssize_t DEVMAJORMINOR_ATTR(struct visor_device *dev, int slot, char *buf)
-{
- int maxdevnodes = ARRAY_SIZE(dev->devnodes) / sizeof(dev->devnodes[0]);
-
- if (slot < 0 || slot >= maxdevnodes)
- return 0;
- return snprintf(buf, PAGE_SIZE, "%d:%d\n",
- dev->devnodes[slot].major, dev->devnodes[slot].minor);
-}
-
-static ssize_t
-devmajorminor_attr_show(struct kobject *kobj, struct attribute *attr, char *buf)
-{
- struct devmajorminor_attribute *devmajorminor_attr =
- to_devmajorminor_attr(attr);
- struct visor_device *dev = to_visor_device_from_kobjdevmajorminor(kobj);
- ssize_t ret = 0;
-
- if (devmajorminor_attr->show)
- ret = devmajorminor_attr->show(dev,
- devmajorminor_attr->slot, buf);
- return ret;
-}
-
-static ssize_t
-devmajorminor_attr_store(struct kobject *kobj,
- struct attribute *attr, const char *buf, size_t count)
-{
- struct devmajorminor_attribute *devmajorminor_attr =
- to_devmajorminor_attr(attr);
- struct visor_device *dev = to_visor_device_from_kobjdevmajorminor(kobj);
- ssize_t ret = 0;
-
- if (devmajorminor_attr->store)
- ret = devmajorminor_attr->store(dev,
- devmajorminor_attr->slot,
- buf, count);
- return ret;
-}
-
-static int register_devmajorminor_attributes(struct visor_device *dev);
-
-static int
-devmajorminor_create_file(struct visor_device *dev, const char *name,
- int major, int minor)
-{
- int maxdevnodes = ARRAY_SIZE(dev->devnodes) / sizeof(dev->devnodes[0]);
- struct devmajorminor_attribute *myattr = NULL;
- int x = -1, rc = 0, slot = -1;
-
- register_devmajorminor_attributes(dev);
- for (slot = 0; slot < maxdevnodes; slot++)
- if (!dev->devnodes[slot].attr)
- break;
- if (slot == maxdevnodes) {
- rc = -ENOMEM;
- goto away;
- }
- myattr = kzalloc(sizeof(*myattr), GFP_KERNEL);
- if (!myattr) {
- rc = -ENOMEM;
- goto away;
- }
- myattr->show = DEVMAJORMINOR_ATTR;
- myattr->store = NULL;
- myattr->slot = slot;
- myattr->attr.name = name;
- myattr->attr.mode = S_IRUGO;
- dev->devnodes[slot].attr = myattr;
- dev->devnodes[slot].major = major;
- dev->devnodes[slot].minor = minor;
- x = sysfs_create_file(&dev->kobjdevmajorminor, &myattr->attr);
- if (x < 0) {
- rc = x;
- goto away;
- }
- kobject_uevent(&dev->device.kobj, KOBJ_ONLINE);
-away:
- if (rc < 0) {
- kfree(myattr);
- myattr = NULL;
- dev->devnodes[slot].attr = NULL;
- }
- return rc;
-}
-
-static void
-devmajorminor_remove_file(struct visor_device *dev, int slot)
-{
- int maxdevnodes = ARRAY_SIZE(dev->devnodes) / sizeof(dev->devnodes[0]);
- struct devmajorminor_attribute *myattr = NULL;
-
- if (slot < 0 || slot >= maxdevnodes)
- return;
- myattr = (struct devmajorminor_attribute *)(dev->devnodes[slot].attr);
- if (!myattr)
- return;
- sysfs_remove_file(&dev->kobjdevmajorminor, &myattr->attr);
- kobject_uevent(&dev->device.kobj, KOBJ_OFFLINE);
- dev->devnodes[slot].attr = NULL;
- kfree(myattr);
-}
-
-static void
-devmajorminor_remove_all_files(struct visor_device *dev)
-{
- int i = 0;
- int maxdevnodes = ARRAY_SIZE(dev->devnodes) / sizeof(dev->devnodes[0]);
-
- for (i = 0; i < maxdevnodes; i++)
- devmajorminor_remove_file(dev, i);
-}
-
-static const struct sysfs_ops devmajorminor_sysfs_ops = {
- .show = devmajorminor_attr_show,
- .store = devmajorminor_attr_store,
-};
-
-static struct kobj_type devmajorminor_kobj_type = {
- .sysfs_ops = &devmajorminor_sysfs_ops
-};
-
-static int
-register_devmajorminor_attributes(struct visor_device *dev)
-{
- int rc = 0, x = 0;
-
- if (dev->kobjdevmajorminor.parent)
- goto away; /* already registered */
- x = kobject_init_and_add(&dev->kobjdevmajorminor,
- &devmajorminor_kobj_type, &dev->device.kobj,
- "devmajorminor");
- if (x < 0) {
- rc = x;
- goto away;
- }
-
- kobject_uevent(&dev->kobjdevmajorminor, KOBJ_ADD);
-
-away:
- return rc;
-}
-
-static void
-unregister_devmajorminor_attributes(struct visor_device *dev)
-{
- if (!dev->kobjdevmajorminor.parent)
- return; /* already unregistered */
- devmajorminor_remove_all_files(dev);
-
- kobject_del(&dev->kobjdevmajorminor);
- kobject_put(&dev->kobjdevmajorminor);
- dev->kobjdevmajorminor.parent = NULL;
-}
-
/* begin implementation of specific channel attributes to appear under
* /sys/bus/visorbus<x>/dev<y>/channel
*/
@@ -427,7 +252,7 @@
if (!vdev->visorchannel)
return 0;
- return snprintf(buf, PAGE_SIZE, "0x%Lx\n",
+ return snprintf(buf, PAGE_SIZE, "0x%llx\n",
visorchannel_get_physaddr(vdev->visorchannel));
}
@@ -449,7 +274,7 @@
if (!vdev->visorchannel)
return 0;
- return snprintf(buf, PAGE_SIZE, "0x%Lx\n",
+ return snprintf(buf, PAGE_SIZE, "0x%llx\n",
visorchannel_get_clientpartition(vdev->visorchannel));
}
@@ -457,24 +282,24 @@
char *buf)
{
struct visor_device *vdev = to_visor_device(dev);
- char s[99];
+ char typeid[LINESIZE];
if (!vdev->visorchannel)
return 0;
return snprintf(buf, PAGE_SIZE, "%s\n",
- visorchannel_id(vdev->visorchannel, s));
+ visorchannel_id(vdev->visorchannel, typeid));
}
static ssize_t zoneguid_show(struct device *dev, struct device_attribute *attr,
char *buf)
{
struct visor_device *vdev = to_visor_device(dev);
- char s[99];
+ char zoneid[LINESIZE];
if (!vdev->visorchannel)
return 0;
return snprintf(buf, PAGE_SIZE, "%s\n",
- visorchannel_zoneid(vdev->visorchannel, s));
+ visorchannel_zoneid(vdev->visorchannel, zoneid));
}
static ssize_t typename_show(struct device *dev, struct device_attribute *attr,
@@ -541,7 +366,7 @@
struct visor_device *vdev = to_visor_device(dev);
u64 handle = visorchannel_get_clientpartition(vdev->visorchannel);
- return snprintf(buf, PAGE_SIZE, "0x%Lx\n", handle);
+ return snprintf(buf, PAGE_SIZE, "0x%llx\n", handle);
}
static ssize_t partition_guid_show(struct device *dev,
@@ -566,7 +391,7 @@
struct visor_device *vdev = to_visor_device(dev);
u64 addr = visorchannel_get_physaddr(vdev->visorchannel);
- return snprintf(buf, PAGE_SIZE, "0x%Lx\n", addr);
+ return snprintf(buf, PAGE_SIZE, "0x%llx\n", addr);
}
static ssize_t channel_bytes_show(struct device *dev,
@@ -575,7 +400,7 @@
struct visor_device *vdev = to_visor_device(dev);
u64 nbytes = visorchannel_get_nbytes(vdev->visorchannel);
- return snprintf(buf, PAGE_SIZE, "0x%Lx\n", nbytes);
+ return snprintf(buf, PAGE_SIZE, "0x%llx\n", nbytes);
}
static ssize_t channel_id_show(struct device *dev,
@@ -598,9 +423,9 @@
struct visor_device *vdev = to_visor_device(dev);
struct visorchannel *channel = vdev->visorchannel;
- int i, x, remain = PAGE_SIZE;
+ int i, shift, remain = PAGE_SIZE;
unsigned long off;
- char *p = buf;
+ char *pos = buf;
u8 *partition_name;
struct ultra_vbus_deviceinfo dev_info;
@@ -608,44 +433,45 @@
if (channel) {
if (vdev->name)
partition_name = vdev->name;
- x = snprintf(p, remain,
- "Client device / client driver info for %s partition (vbus #%d):\n",
- partition_name, vdev->chipset_dev_no);
- p += x;
- remain -= x;
- x = visorchannel_read(channel,
- offsetof(struct
- spar_vbus_channel_protocol,
- chp_info),
- &dev_info, sizeof(dev_info));
- if (x >= 0) {
- x = vbuschannel_devinfo_to_string(&dev_info, p,
- remain, -1);
- p += x;
- remain -= x;
+ shift = snprintf(pos, remain,
+ "Client device / client driver info for %s eartition (vbus #%d):\n",
+ partition_name, vdev->chipset_dev_no);
+ pos += shift;
+ remain -= shift;
+ shift = visorchannel_read(channel,
+ offsetof(struct
+ spar_vbus_channel_protocol,
+ chp_info),
+ &dev_info, sizeof(dev_info));
+ if (shift >= 0) {
+ shift = vbuschannel_devinfo_to_string(&dev_info, pos,
+ remain, -1);
+ pos += shift;
+ remain -= shift;
}
- x = visorchannel_read(channel,
- offsetof(struct
- spar_vbus_channel_protocol,
- bus_info),
- &dev_info, sizeof(dev_info));
- if (x >= 0) {
- x = vbuschannel_devinfo_to_string(&dev_info, p,
- remain, -1);
- p += x;
- remain -= x;
+ shift = visorchannel_read(channel,
+ offsetof(struct
+ spar_vbus_channel_protocol,
+ bus_info),
+ &dev_info, sizeof(dev_info));
+ if (shift >= 0) {
+ shift = vbuschannel_devinfo_to_string(&dev_info, pos,
+ remain, -1);
+ pos += shift;
+ remain -= shift;
}
off = offsetof(struct spar_vbus_channel_protocol, dev_info);
i = 0;
while (off + sizeof(dev_info) <=
visorchannel_get_nbytes(channel)) {
- x = visorchannel_read(channel,
- off, &dev_info, sizeof(dev_info));
- if (x >= 0) {
- x = vbuschannel_devinfo_to_string
- (&dev_info, p, remain, i);
- p += x;
- remain -= x;
+ shift = visorchannel_read(channel,
+ off, &dev_info,
+ sizeof(dev_info));
+ if (shift >= 0) {
+ shift = vbuschannel_devinfo_to_string
+ (&dev_info, pos, remain, i);
+ pos += shift;
+ remain -= shift;
}
off += sizeof(dev_info);
i++;
@@ -752,36 +578,28 @@
static int
visordriver_probe_device(struct device *xdev)
{
- int rc;
+ int res;
struct visor_driver *drv;
struct visor_device *dev;
drv = to_visor_driver(xdev->driver);
dev = to_visor_device(xdev);
+
+ if (!drv->probe)
+ return -ENODEV;
+
down(&dev->visordriver_callback_lock);
dev->being_removed = false;
- /*
- * ensure that the dev->being_removed flag is cleared before
- * we start the probe
- */
- wmb();
- get_device(&dev->device);
- if (!drv->probe) {
- up(&dev->visordriver_callback_lock);
- rc = -ENODEV;
- goto away;
- }
- rc = drv->probe(dev);
- if (rc < 0)
- goto away;
- fix_vbus_dev_info(dev);
+ res = drv->probe(dev);
+ if (res >= 0) {
+ /* success: reference kept via unmatched get_device() */
+ get_device(&dev->device);
+ fix_vbus_dev_info(dev);
+ }
+
up(&dev->visordriver_callback_lock);
- rc = 0;
-away:
- if (rc != 0)
- put_device(&dev->device);
- return rc;
+ return res;
}
/** This is called when device_unregister() is called for each child device
@@ -798,21 +616,12 @@
drv = to_visor_driver(xdev->driver);
down(&dev->visordriver_callback_lock);
dev->being_removed = true;
- /*
- * ensure that the dev->being_removed flag is set before we start the
- * actual removal
- */
- wmb();
- if (drv) {
- if (drv->remove)
- drv->remove(dev);
- }
+ if (drv->remove)
+ drv->remove(dev);
up(&dev->visordriver_callback_lock);
dev_stop_periodic_work(dev);
- devmajorminor_remove_all_files(dev);
put_device(&dev->device);
-
return 0;
}
@@ -928,14 +737,6 @@
}
EXPORT_SYMBOL_GPL(visorbus_clear_channel);
-int
-visorbus_registerdevnode(struct visor_device *dev,
- const char *name, int major, int minor)
-{
- return devmajorminor_create_file(dev, name, major, minor);
-}
-EXPORT_SYMBOL_GPL(visorbus_registerdevnode);
-
/** We don't really have a real interrupt, so for now we just call the
* interrupt function periodically...
*/
@@ -970,7 +771,7 @@
static int
create_visor_device(struct visor_device *dev)
{
- int rc;
+ int err;
u32 chipset_bus_no = dev->chipset_bus_no;
u32 chipset_dev_no = dev->chipset_dev_no;
@@ -992,8 +793,8 @@
if (!dev->periodic_work) {
POSTCODE_LINUX_3(DEVICE_CREATE_FAILURE_PC, chipset_dev_no,
DIAG_SEVERITY_ERR);
- rc = -EINVAL;
- goto away;
+ err = -EINVAL;
+ goto err_put;
}
/* bus_id must be a unique name with respect to this bus TYPE
@@ -1019,36 +820,25 @@
* claim the device. The device will be linked onto
* bus_type.klist_devices regardless (use bus_for_each_dev).
*/
- rc = device_add(&dev->device);
- if (rc < 0) {
+ err = device_add(&dev->device);
+ if (err < 0) {
POSTCODE_LINUX_3(DEVICE_ADD_PC, chipset_bus_no,
DIAG_SEVERITY_ERR);
- goto away;
- }
-
- rc = register_devmajorminor_attributes(dev);
- if (rc < 0) {
- POSTCODE_LINUX_3(DEVICE_REGISTER_FAILURE_PC, chipset_dev_no,
- DIAG_SEVERITY_ERR);
- goto away_unregister;
+ goto err_put;
}
list_add_tail(&dev->list_all, &list_all_device_instances);
- return 0;
+ return 0; /* success: reference kept via unmatched get_device() */
-away_unregister:
- device_unregister(&dev->device);
-
-away:
+err_put:
put_device(&dev->device);
- return rc;
+ return err;
}
static void
remove_visor_device(struct visor_device *dev)
{
list_del(&dev->list_all);
- unregister_devmajorminor_attributes(dev);
put_device(&dev->device);
device_unregister(&dev->device);
}
@@ -1477,24 +1267,24 @@
int
visorbus_init(void)
{
- int rc = 0;
+ int err;
- POSTCODE_LINUX_3(DRIVER_ENTRY_PC, rc, POSTCODE_SEVERITY_INFO);
+ POSTCODE_LINUX_3(DRIVER_ENTRY_PC, 0, POSTCODE_SEVERITY_INFO);
bus_device_info_init(&clientbus_driverinfo,
"clientbus", "visorbus",
VERSION, NULL);
- rc = create_bus_type();
- if (rc < 0) {
+ err = create_bus_type();
+ if (err < 0) {
POSTCODE_LINUX_2(BUS_CREATE_ENTRY_PC, DIAG_SEVERITY_ERR);
- goto away;
+ goto error;
}
periodic_dev_workqueue = create_singlethread_workqueue("visorbus_dev");
if (!periodic_dev_workqueue) {
POSTCODE_LINUX_2(CREATE_WORKQUEUE_PC, DIAG_SEVERITY_ERR);
- rc = -ENOMEM;
- goto away;
+ err = -ENOMEM;
+ goto error;
}
/* This enables us to receive notifications when devices appear for
@@ -1504,13 +1294,11 @@
&chipset_responders,
&chipset_driverinfo);
- rc = 0;
+ return 0;
-away:
- if (rc)
- POSTCODE_LINUX_3(CHIPSET_INIT_FAILURE_PC, rc,
- POSTCODE_SEVERITY_ERR);
- return rc;
+error:
+ POSTCODE_LINUX_3(CHIPSET_INIT_FAILURE_PC, err, POSTCODE_SEVERITY_ERR);
+ return err;
}
void
diff --git a/drivers/staging/unisys/visorbus/visorchannel.c b/drivers/staging/unisys/visorbus/visorchannel.c
index b68a904..4337358 100644
--- a/drivers/staging/unisys/visorbus/visorchannel.c
+++ b/drivers/staging/unisys/visorbus/visorchannel.c
@@ -40,7 +40,6 @@
bool requested;
struct channel_header chan_hdr;
uuid_le guid;
- ulong size;
bool needs_lock; /* channel creator knows if more than one */
/* thread will be inserting or removing */
spinlock_t insert_lock; /* protect head writes in chan_hdr */
@@ -134,8 +133,6 @@
}
channel->nbytes = channel_bytes;
-
- channel->size = channel_bytes;
channel->guid = guid;
return channel;
@@ -186,7 +183,7 @@
ulong
visorchannel_get_nbytes(struct visorchannel *channel)
{
- return channel->size;
+ return channel->nbytes;
}
EXPORT_SYMBOL_GPL(visorchannel_get_nbytes);
diff --git a/drivers/staging/unisys/visorbus/visorchipset.c b/drivers/staging/unisys/visorbus/visorchipset.c
index 9cf4f84..5ba5936 100644
--- a/drivers/staging/unisys/visorbus/visorchipset.c
+++ b/drivers/staging/unisys/visorbus/visorchipset.c
@@ -59,14 +59,13 @@
*/
static int visorchipset_major;
static int visorchipset_visorbusregwait = 1; /* default is on */
-static int visorchipset_holdchipsetready;
static unsigned long controlvm_payload_bytes_buffered;
static u32 dump_vhba_bus;
static int
visorchipset_open(struct inode *inode, struct file *file)
{
- unsigned minor_number = iminor(inode);
+ unsigned int minor_number = iminor(inode);
if (minor_number)
return -ENODEV;
@@ -90,9 +89,6 @@
static unsigned long most_recent_message_jiffies;
static int visorbusregistered;
-#define MAX_CHIPSET_EVENTS 2
-static u8 chipset_events[MAX_CHIPSET_EVENTS] = { 0, 0 };
-
struct parser_context {
unsigned long allocbytes;
unsigned long param_bytes;
@@ -107,7 +103,6 @@
static struct cdev file_cdev;
static struct visorchannel **file_controlvm_channel;
-static struct controlvm_message_header g_chipset_msg_hdr;
static struct controlvm_message_packet g_devicechangestate_packet;
static LIST_HEAD(bus_info_list);
@@ -156,8 +151,6 @@
/* a payload from a controlvm message, containing a file data buffer */
struct parser_context *parser_ctx;
/* points within data area of parser_ctx to next byte of data */
- u8 *pnext;
- /* # bytes left from <pnext> to the end of this data buffer */
size_t bytes_remaining;
};
@@ -171,14 +164,10 @@
/* header from original TransmitFile request */
struct controlvm_message_header controlvm_header;
- u64 file_request_number; /* from original TransmitFile request */
/* link to next struct putfile_request */
struct list_head next_putfile_request;
- /* most-recent sequence number supplied via a controlvm message */
- u64 data_sequence_number;
-
/* head of putfile_buffer_entry list, which describes the data to be
* supplied as putfile data;
* - this list is added to when controlvm messages come in that supply
@@ -274,11 +263,6 @@
const char *buf, size_t count);
static DEVICE_ATTR_RW(remaining_steps);
-static ssize_t chipsetready_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t count);
-static DEVICE_ATTR_WO(chipsetready);
-
static ssize_t devicedisabled_store(struct device *dev,
struct device_attribute *attr,
const char *buf, size_t count);
@@ -303,16 +287,6 @@
.attrs = visorchipset_install_attrs
};
-static struct attribute *visorchipset_guest_attrs[] = {
- &dev_attr_chipsetready.attr,
- NULL
-};
-
-static struct attribute_group visorchipset_guest_group = {
- .name = "guest",
- .attrs = visorchipset_guest_attrs
-};
-
static struct attribute *visorchipset_parahotplug_attrs[] = {
&dev_attr_devicedisabled.attr,
&dev_attr_deviceenabled.attr,
@@ -326,7 +300,6 @@
static const struct attribute_group *visorchipset_dev_groups[] = {
&visorchipset_install_group,
- &visorchipset_guest_group,
&visorchipset_parahotplug_group,
NULL
};
@@ -359,8 +332,7 @@
parser_init_byte_stream(u64 addr, u32 bytes, bool local, bool *retry)
{
int allocbytes = sizeof(struct parser_context) + bytes;
- struct parser_context *rc = NULL;
- struct parser_context *ctx = NULL;
+ struct parser_context *ctx;
if (retry)
*retry = false;
@@ -374,15 +346,13 @@
> MAX_CONTROLVM_PAYLOAD_BYTES) {
if (retry)
*retry = true;
- rc = NULL;
- goto cleanup;
+ return NULL;
}
ctx = kzalloc(allocbytes, GFP_KERNEL | __GFP_NORETRY);
if (!ctx) {
if (retry)
*retry = true;
- rc = NULL;
- goto cleanup;
+ return NULL;
}
ctx->allocbytes = allocbytes;
@@ -393,35 +363,27 @@
if (local) {
void *p;
- if (addr > virt_to_phys(high_memory - 1)) {
- rc = NULL;
- goto cleanup;
- }
+ if (addr > virt_to_phys(high_memory - 1))
+ goto err_finish_ctx;
p = __va((unsigned long)(addr));
memcpy(ctx->data, p, bytes);
} else {
void *mapping = memremap(addr, bytes, MEMREMAP_WB);
- if (!mapping) {
- rc = NULL;
- goto cleanup;
- }
+ if (!mapping)
+ goto err_finish_ctx;
memcpy(ctx->data, mapping, bytes);
memunmap(mapping);
}
ctx->byte_stream = true;
- rc = ctx;
-cleanup:
- if (rc) {
- controlvm_payload_bytes_buffered += ctx->param_bytes;
- } else {
- if (ctx) {
- parser_done(ctx);
- ctx = NULL;
- }
- }
- return rc;
+ controlvm_payload_bytes_buffered += ctx->param_bytes;
+
+ return ctx;
+
+err_finish_ctx:
+ parser_done(ctx);
+ return NULL;
}
static uuid_le
@@ -523,7 +485,7 @@
struct device_attribute *attr,
char *buf)
{
- u8 tool_action;
+ u8 tool_action = 0;
visorchannel_read(controlvm_channel,
offsetof(struct spar_controlvm_channel_protocol,
@@ -541,10 +503,11 @@
if (kstrtou8(buf, 10, &tool_action))
return -EINVAL;
- ret = visorchannel_write(controlvm_channel,
- offsetof(struct spar_controlvm_channel_protocol,
- tool_action),
- &tool_action, sizeof(u8));
+ ret = visorchannel_write
+ (controlvm_channel,
+ offsetof(struct spar_controlvm_channel_protocol,
+ tool_action),
+ &tool_action, sizeof(u8));
if (ret)
return ret;
@@ -576,10 +539,11 @@
return -EINVAL;
efi_spar_indication.boot_to_tool = val;
- ret = visorchannel_write(controlvm_channel,
- offsetof(struct spar_controlvm_channel_protocol,
- efi_spar_ind), &(efi_spar_indication),
- sizeof(struct efi_spar_indication));
+ ret = visorchannel_write
+ (controlvm_channel,
+ offsetof(struct spar_controlvm_channel_protocol,
+ efi_spar_ind), &(efi_spar_indication),
+ sizeof(struct efi_spar_indication));
if (ret)
return ret;
@@ -589,7 +553,7 @@
static ssize_t error_show(struct device *dev, struct device_attribute *attr,
char *buf)
{
- u32 error;
+ u32 error = 0;
visorchannel_read(controlvm_channel,
offsetof(struct spar_controlvm_channel_protocol,
@@ -607,10 +571,11 @@
if (kstrtou32(buf, 10, &error))
return -EINVAL;
- ret = visorchannel_write(controlvm_channel,
- offsetof(struct spar_controlvm_channel_protocol,
- installation_error),
- &error, sizeof(u32));
+ ret = visorchannel_write
+ (controlvm_channel,
+ offsetof(struct spar_controlvm_channel_protocol,
+ installation_error),
+ &error, sizeof(u32));
if (ret)
return ret;
return count;
@@ -619,12 +584,13 @@
static ssize_t textid_show(struct device *dev, struct device_attribute *attr,
char *buf)
{
- u32 text_id;
+ u32 text_id = 0;
- visorchannel_read(controlvm_channel,
- offsetof(struct spar_controlvm_channel_protocol,
- installation_text_id),
- &text_id, sizeof(u32));
+ visorchannel_read
+ (controlvm_channel,
+ offsetof(struct spar_controlvm_channel_protocol,
+ installation_text_id),
+ &text_id, sizeof(u32));
return scnprintf(buf, PAGE_SIZE, "%i\n", text_id);
}
@@ -637,10 +603,11 @@
if (kstrtou32(buf, 10, &text_id))
return -EINVAL;
- ret = visorchannel_write(controlvm_channel,
- offsetof(struct spar_controlvm_channel_protocol,
- installation_text_id),
- &text_id, sizeof(u32));
+ ret = visorchannel_write
+ (controlvm_channel,
+ offsetof(struct spar_controlvm_channel_protocol,
+ installation_text_id),
+ &text_id, sizeof(u32));
if (ret)
return ret;
return count;
@@ -649,7 +616,7 @@
static ssize_t remaining_steps_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
- u16 remaining_steps;
+ u16 remaining_steps = 0;
visorchannel_read(controlvm_channel,
offsetof(struct spar_controlvm_channel_protocol,
@@ -668,10 +635,11 @@
if (kstrtou16(buf, 10, &remaining_steps))
return -EINVAL;
- ret = visorchannel_write(controlvm_channel,
- offsetof(struct spar_controlvm_channel_protocol,
- installation_remaining_steps),
- &remaining_steps, sizeof(u16));
+ ret = visorchannel_write
+ (controlvm_channel,
+ offsetof(struct spar_controlvm_channel_protocol,
+ installation_remaining_steps),
+ &remaining_steps, sizeof(u16));
if (ret)
return ret;
return count;
@@ -717,26 +685,6 @@
}
EXPORT_SYMBOL(visorbus_get_device_by_id);
-static u8
-check_chipset_events(void)
-{
- int i;
- u8 send_msg = 1;
- /* Check events to determine if response should be sent */
- for (i = 0; i < MAX_CHIPSET_EVENTS; i++)
- send_msg &= chipset_events[i];
- return send_msg;
-}
-
-static void
-clear_chipset_events(void)
-{
- int i;
- /* Clear chipset_events */
- for (i = 0; i < MAX_CHIPSET_EVENTS; i++)
- chipset_events[i] = 0;
-}
-
void
visorchipset_register_busdev(
struct visorchipset_busdev_notifiers *notifiers,
@@ -772,7 +720,7 @@
POSTCODE_LINUX_2(CHIPSET_INIT_ENTRY_PC, POSTCODE_SEVERITY_INFO);
if (chipset_inited) {
rc = -CONTROLVM_RESP_ERROR_ALREADY_DONE;
- goto cleanup;
+ goto out_respond;
}
chipset_inited = 1;
POSTCODE_LINUX_2(CHIPSET_INIT_EXIT_PC, POSTCODE_SEVERITY_INFO);
@@ -789,7 +737,7 @@
*/
features |= ULTRA_CHIPSET_FEATURE_REPLY;
-cleanup:
+out_respond:
if (inmsg->hdr.flags.response_expected)
controlvm_respond_chipset_init(&inmsg->hdr, rc, features);
}
@@ -970,28 +918,31 @@
u32 cmd, struct controlvm_message_header *msg_hdr,
int response, bool need_response)
{
- bool notified = false;
struct controlvm_message_header *pmsg_hdr = NULL;
+ down(¬ifier_lock);
+
if (!bus_info) {
/* relying on a valid passed in response code */
/* be lazy and re-use msg_hdr for this failure, is this ok?? */
pmsg_hdr = msg_hdr;
- goto away;
+ goto out_respond_and_unlock;
}
if (bus_info->pending_msg_hdr) {
/* only non-NULL if dev is still waiting on a response */
response = -CONTROLVM_RESP_ERROR_MESSAGE_ID_INVALID_FOR_CLIENT;
pmsg_hdr = bus_info->pending_msg_hdr;
- goto away;
+ goto out_respond_and_unlock;
}
if (need_response) {
pmsg_hdr = kzalloc(sizeof(*pmsg_hdr), GFP_KERNEL);
if (!pmsg_hdr) {
- response = -CONTROLVM_RESP_ERROR_KMALLOC_FAILED;
- goto away;
+ POSTCODE_LINUX_4(MALLOC_FAILURE_PC, cmd,
+ bus_info->chipset_bus_no,
+ POSTCODE_SEVERITY_ERR);
+ goto out_unlock;
}
memcpy(pmsg_hdr, msg_hdr,
@@ -999,37 +950,27 @@
bus_info->pending_msg_hdr = pmsg_hdr;
}
- down(¬ifier_lock);
if (response == CONTROLVM_RESP_SUCCESS) {
switch (cmd) {
case CONTROLVM_BUS_CREATE:
if (busdev_notifiers.bus_create) {
(*busdev_notifiers.bus_create) (bus_info);
- notified = true;
+ goto out_unlock;
}
break;
case CONTROLVM_BUS_DESTROY:
if (busdev_notifiers.bus_destroy) {
(*busdev_notifiers.bus_destroy) (bus_info);
- notified = true;
+ goto out_unlock;
}
break;
}
}
-away:
- if (notified)
- /* The callback function just called above is responsible
- * for calling the appropriate visorchipset_busdev_responders
- * function, which will call bus_responder()
- */
- ;
- else
- /*
- * Do not kfree(pmsg_hdr) as this is the failure path.
- * The success path ('notified') will call the responder
- * directly and kfree() there.
- */
- bus_responder(cmd, pmsg_hdr, response);
+
+out_respond_and_unlock:
+ bus_responder(cmd, pmsg_hdr, response);
+
+out_unlock:
up(¬ifier_lock);
}
@@ -1040,30 +981,30 @@
bool need_response, bool for_visorbus)
{
struct visorchipset_busdev_notifiers *notifiers;
- bool notified = false;
struct controlvm_message_header *pmsg_hdr = NULL;
notifiers = &busdev_notifiers;
+ down(¬ifier_lock);
if (!dev_info) {
/* relying on a valid passed in response code */
/* be lazy and re-use msg_hdr for this failure, is this ok?? */
pmsg_hdr = msg_hdr;
- goto away;
+ goto out_respond_and_unlock;
}
if (dev_info->pending_msg_hdr) {
/* only non-NULL if dev is still waiting on a response */
response = -CONTROLVM_RESP_ERROR_MESSAGE_ID_INVALID_FOR_CLIENT;
pmsg_hdr = dev_info->pending_msg_hdr;
- goto away;
+ goto out_respond_and_unlock;
}
if (need_response) {
pmsg_hdr = kzalloc(sizeof(*pmsg_hdr), GFP_KERNEL);
if (!pmsg_hdr) {
response = -CONTROLVM_RESP_ERROR_KMALLOC_FAILED;
- goto away;
+ goto out_respond_and_unlock;
}
memcpy(pmsg_hdr, msg_hdr,
@@ -1071,13 +1012,12 @@
dev_info->pending_msg_hdr = pmsg_hdr;
}
- down(¬ifier_lock);
if (response >= 0) {
switch (cmd) {
case CONTROLVM_DEVICE_CREATE:
if (notifiers->device_create) {
(*notifiers->device_create) (dev_info);
- notified = true;
+ goto out_unlock;
}
break;
case CONTROLVM_DEVICE_CHANGESTATE:
@@ -1087,7 +1027,7 @@
segment_state_running.operating) {
if (notifiers->device_resume) {
(*notifiers->device_resume) (dev_info);
- notified = true;
+ goto out_unlock;
}
}
/* ServerNotReady / ServerLost / SegmentStateStandby */
@@ -1099,32 +1039,23 @@
*/
if (notifiers->device_pause) {
(*notifiers->device_pause) (dev_info);
- notified = true;
+ goto out_unlock;
}
}
break;
case CONTROLVM_DEVICE_DESTROY:
if (notifiers->device_destroy) {
(*notifiers->device_destroy) (dev_info);
- notified = true;
+ goto out_unlock;
}
break;
}
}
-away:
- if (notified)
- /* The callback function just called above is responsible
- * for calling the appropriate visorchipset_busdev_responders
- * function, which will call device_responder()
- */
- ;
- else
- /*
- * Do not kfree(pmsg_hdr) as this is the failure path.
- * The success path ('notified') will call the responder
- * directly and kfree() there.
- */
- device_responder(cmd, pmsg_hdr, response);
+
+out_respond_and_unlock:
+ device_responder(cmd, pmsg_hdr, response);
+
+out_unlock:
up(¬ifier_lock);
}
@@ -1142,14 +1073,14 @@
POSTCODE_LINUX_3(BUS_CREATE_FAILURE_PC, bus_no,
POSTCODE_SEVERITY_ERR);
rc = -CONTROLVM_RESP_ERROR_ALREADY_DONE;
- goto cleanup;
+ goto out_bus_epilog;
}
bus_info = kzalloc(sizeof(*bus_info), GFP_KERNEL);
if (!bus_info) {
POSTCODE_LINUX_3(BUS_CREATE_FAILURE_PC, bus_no,
POSTCODE_SEVERITY_ERR);
rc = -CONTROLVM_RESP_ERROR_KMALLOC_FAILED;
- goto cleanup;
+ goto out_bus_epilog;
}
INIT_LIST_HEAD(&bus_info->list_all);
@@ -1169,7 +1100,7 @@
rc = -CONTROLVM_RESP_ERROR_KMALLOC_FAILED;
kfree(bus_info);
bus_info = NULL;
- goto cleanup;
+ goto out_bus_epilog;
}
bus_info->visorchannel = visorchannel;
if (uuid_le_cmp(cmd->create_bus.bus_inst_uuid, spar_siovm_uuid) == 0) {
@@ -1179,7 +1110,7 @@
POSTCODE_LINUX_3(BUS_CREATE_EXIT_PC, bus_no, POSTCODE_SEVERITY_INFO);
-cleanup:
+out_bus_epilog:
bus_epilog(bus_info, CONTROLVM_BUS_CREATE, &inmsg->hdr,
rc, inmsg->hdr.flags.response_expected == 1);
}
@@ -1231,8 +1162,9 @@
POSTCODE_SEVERITY_ERR);
rc = -CONTROLVM_RESP_ERROR_MESSAGE_ID_INVALID_FOR_CLIENT;
} else {
- visorchannel_set_clientpartition(bus_info->visorchannel,
- cmd->configure_bus.guest_handle);
+ visorchannel_set_clientpartition
+ (bus_info->visorchannel,
+ cmd->configure_bus.guest_handle);
bus_info->partition_uuid = parser_id_get(parser_ctx);
parser_param_start(parser_ctx, PARSERSTRING_NAME);
bus_info->name = parser_string_get(parser_ctx);
@@ -1260,14 +1192,14 @@
POSTCODE_LINUX_4(DEVICE_CREATE_FAILURE_PC, dev_no, bus_no,
POSTCODE_SEVERITY_ERR);
rc = -CONTROLVM_RESP_ERROR_BUS_INVALID;
- goto cleanup;
+ goto out_respond;
}
if (bus_info->state.created == 0) {
POSTCODE_LINUX_4(DEVICE_CREATE_FAILURE_PC, dev_no, bus_no,
POSTCODE_SEVERITY_ERR);
rc = -CONTROLVM_RESP_ERROR_BUS_INVALID;
- goto cleanup;
+ goto out_respond;
}
dev_info = visorbus_get_device_by_id(bus_no, dev_no, NULL);
@@ -1275,7 +1207,7 @@
POSTCODE_LINUX_4(DEVICE_CREATE_FAILURE_PC, dev_no, bus_no,
POSTCODE_SEVERITY_ERR);
rc = -CONTROLVM_RESP_ERROR_ALREADY_DONE;
- goto cleanup;
+ goto out_respond;
}
dev_info = kzalloc(sizeof(*dev_info), GFP_KERNEL);
@@ -1283,7 +1215,7 @@
POSTCODE_LINUX_4(DEVICE_CREATE_FAILURE_PC, dev_no, bus_no,
POSTCODE_SEVERITY_ERR);
rc = -CONTROLVM_RESP_ERROR_KMALLOC_FAILED;
- goto cleanup;
+ goto out_respond;
}
dev_info->chipset_bus_no = bus_no;
@@ -1308,7 +1240,7 @@
rc = -CONTROLVM_RESP_ERROR_KMALLOC_FAILED;
kfree(dev_info);
dev_info = NULL;
- goto cleanup;
+ goto out_respond;
}
dev_info->visorchannel = visorchannel;
dev_info->channel_type_guid = cmd->create_device.data_type_uuid;
@@ -1318,7 +1250,7 @@
POSTCODE_LINUX_4(DEVICE_CREATE_EXIT_PC, dev_no, bus_no,
POSTCODE_SEVERITY_INFO);
-cleanup:
+out_respond:
device_epilog(dev_info, segment_state_running,
CONTROLVM_DEVICE_CREATE, &inmsg->hdr, rc,
inmsg->hdr.flags.response_expected == 1, 1);
@@ -1382,35 +1314,23 @@
struct visor_controlvm_payload_info *info)
{
u8 *payload = NULL;
- int rc = CONTROLVM_RESP_SUCCESS;
- if (!info) {
- rc = -CONTROLVM_RESP_ERROR_PAYLOAD_INVALID;
- goto cleanup;
- }
+ if (!info)
+ return -CONTROLVM_RESP_ERROR_PAYLOAD_INVALID;
+
memset(info, 0, sizeof(struct visor_controlvm_payload_info));
- if ((offset == 0) || (bytes == 0)) {
- rc = -CONTROLVM_RESP_ERROR_PAYLOAD_INVALID;
- goto cleanup;
- }
+ if ((offset == 0) || (bytes == 0))
+ return -CONTROLVM_RESP_ERROR_PAYLOAD_INVALID;
+
payload = memremap(phys_addr + offset, bytes, MEMREMAP_WB);
- if (!payload) {
- rc = -CONTROLVM_RESP_ERROR_IOREMAP_FAILED;
- goto cleanup;
- }
+ if (!payload)
+ return -CONTROLVM_RESP_ERROR_IOREMAP_FAILED;
info->offset = offset;
info->bytes = bytes;
info->ptr = payload;
-cleanup:
- if (rc < 0) {
- if (payload) {
- memunmap(payload);
- payload = NULL;
- }
- }
- return rc;
+ return CONTROLVM_RESP_SUCCESS;
}
static void
@@ -1490,14 +1410,8 @@
if (rc != CONTROLVM_RESP_SUCCESS)
rc = -rc;
- if (msg_hdr->flags.response_expected && !visorchipset_holdchipsetready)
+ if (msg_hdr->flags.response_expected)
controlvm_respond(msg_hdr, rc);
- if (msg_hdr->flags.response_expected && visorchipset_holdchipsetready) {
- /* Send CHIPSET_READY response when all modules have been loaded
- * and disks mounted for the partition
- */
- g_chipset_msg_hdr = *msg_hdr;
- }
}
static void
@@ -1726,9 +1640,10 @@
* initialization.
*/
parahotplug_request_kickoff(req);
- controlvm_respond_physdev_changestate(&inmsg->hdr,
- CONTROLVM_RESP_SUCCESS,
- inmsg->cmd.device_change_state.state);
+ controlvm_respond_physdev_changestate
+ (&inmsg->hdr,
+ CONTROLVM_RESP_SUCCESS,
+ inmsg->cmd.device_change_state.state);
parahotplug_request_destroy(req);
} else {
/* For disable messages, add the request to the
@@ -1840,8 +1755,9 @@
break;
default:
if (inmsg.hdr.flags.response_expected)
- controlvm_respond(&inmsg.hdr,
- -CONTROLVM_RESP_ERROR_MESSAGE_ID_UNKNOWN);
+ controlvm_respond
+ (&inmsg.hdr,
+ -CONTROLVM_RESP_ERROR_MESSAGE_ID_UNKNOWN);
break;
}
@@ -1885,31 +1801,11 @@
struct controlvm_message inmsg;
bool got_command = false;
bool handle_command_failed = false;
- static u64 poll_count;
/* make sure visorbus server is registered for controlvm callbacks */
if (visorchipset_visorbusregwait && !visorbusregistered)
goto cleanup;
- poll_count++;
- if (poll_count >= 250)
- ; /* keep going */
- else
- goto cleanup;
-
- /* Check events to determine if response to CHIPSET_READY
- * should be sent
- */
- if (visorchipset_holdchipsetready &&
- (g_chipset_msg_hdr.id != CONTROLVM_INVALID)) {
- if (check_chipset_events() == 1) {
- controlvm_respond(&g_chipset_msg_hdr, 0);
- clear_chipset_events();
- memset(&g_chipset_msg_hdr, 0,
- sizeof(struct controlvm_message_header));
- }
- }
-
while (visorchannel_signalremove(controlvm_channel,
CONTROLVM_QUEUE_RESPONSE,
&inmsg))
@@ -1979,8 +1875,11 @@
u16 local_crash_msg_count;
/* make sure visorbus is registered for controlvm callbacks */
- if (visorchipset_visorbusregwait && !visorbusregistered)
- goto cleanup;
+ if (visorchipset_visorbusregwait && !visorbusregistered) {
+ poll_jiffies = POLLJIFFIES_CONTROLVMCHANNEL_SLOW;
+ schedule_delayed_work(&periodic_controlvm_work, poll_jiffies);
+ return;
+ }
POSTCODE_LINUX_2(CRASH_DEV_ENTRY_PC, POSTCODE_SEVERITY_INFO);
@@ -2057,13 +1956,6 @@
return;
}
POSTCODE_LINUX_2(CRASH_DEV_EXIT_PC, POSTCODE_SEVERITY_INFO);
- return;
-
-cleanup:
-
- poll_jiffies = POLLJIFFIES_CONTROLVMCHANNEL_SLOW;
-
- schedule_delayed_work(&periodic_controlvm_work, poll_jiffies);
}
static void
@@ -2135,25 +2027,6 @@
dev_info->pending_msg_hdr = NULL;
}
-static ssize_t chipsetready_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t count)
-{
- char msgtype[64];
-
- if (sscanf(buf, "%63s", msgtype) != 1)
- return -EINVAL;
-
- if (!strcmp(msgtype, "CALLHOMEDISK_MOUNTED")) {
- chipset_events[0] = 1;
- return count;
- } else if (!strcmp(msgtype, "MODULES_LOADED")) {
- chipset_events[1] = 1;
- return count;
- }
- return -EINVAL;
-}
-
/* The parahotplug/devicedisabled interface gets called by our support script
* when an SR-IOV device has been shut down. The ID is passed to the script
* and then passed back when the device has been removed.
@@ -2205,10 +2078,11 @@
if (!*file_controlvm_channel)
return -ENXIO;
- visorchannel_read(*file_controlvm_channel,
- offsetof(struct spar_controlvm_channel_protocol,
- gp_control_channel),
- &addr, sizeof(addr));
+ visorchannel_read
+ (*file_controlvm_channel,
+ offsetof(struct spar_controlvm_channel_protocol,
+ gp_control_channel),
+ &addr, sizeof(addr));
if (!addr)
return -ENXIO;
@@ -2308,16 +2182,25 @@
return 0;
}
+static void
+visorchipset_file_cleanup(dev_t major_dev)
+{
+ if (file_cdev.ops)
+ cdev_del(&file_cdev);
+ file_cdev.ops = NULL;
+ unregister_chrdev_region(major_dev, 1);
+}
+
static int
visorchipset_init(struct acpi_device *acpi_device)
{
- int rc = 0;
+ int err = -ENODEV;
u64 addr;
uuid_le uuid = SPAR_CONTROLVM_CHANNEL_PROTOCOL_UUID;
addr = controlvm_get_channel_address();
if (!addr)
- return -ENODEV;
+ goto error;
memset(&busdev_notifiers, 0, sizeof(busdev_notifiers));
memset(&controlvm_payload_info, 0, sizeof(controlvm_payload_info));
@@ -2325,24 +2208,19 @@
controlvm_channel = visorchannel_create_with_lock(addr, 0,
GFP_KERNEL, uuid);
if (!controlvm_channel)
- return -ENODEV;
+ goto error;
+
if (SPAR_CONTROLVM_CHANNEL_OK_CLIENT(
visorchannel_get_header(controlvm_channel))) {
initialize_controlvm_payload();
} else {
- visorchannel_destroy(controlvm_channel);
- controlvm_channel = NULL;
- return -ENODEV;
+ goto error_destroy_channel;
}
major_dev = MKDEV(visorchipset_major, 0);
- rc = visorchipset_file_init(major_dev, &controlvm_channel);
- if (rc < 0) {
- POSTCODE_LINUX_2(CHIPSET_INIT_FAILURE_PC, DIAG_SEVERITY_ERR);
- goto cleanup;
- }
-
- memset(&g_chipset_msg_hdr, 0, sizeof(struct controlvm_message_header));
+ err = visorchipset_file_init(major_dev, &controlvm_channel);
+ if (err < 0)
+ goto error_destroy_payload;
/* if booting in a crash kernel */
if (is_kdump_kernel())
@@ -2359,27 +2237,33 @@
visorchipset_platform_device.dev.devt = major_dev;
if (platform_device_register(&visorchipset_platform_device) < 0) {
POSTCODE_LINUX_2(DEVICE_REGISTER_FAILURE_PC, DIAG_SEVERITY_ERR);
- rc = -ENODEV;
- goto cleanup;
+ err = -ENODEV;
+ goto error_cancel_work;
}
POSTCODE_LINUX_2(CHIPSET_INIT_SUCCESS_PC, POSTCODE_SEVERITY_INFO);
- rc = visorbus_init();
-cleanup:
- if (rc) {
- POSTCODE_LINUX_3(CHIPSET_INIT_FAILURE_PC, rc,
- POSTCODE_SEVERITY_ERR);
- }
- return rc;
-}
+ err = visorbus_init();
+ if (err < 0)
+ goto error_unregister;
-static void
-visorchipset_file_cleanup(dev_t major_dev)
-{
- if (file_cdev.ops)
- cdev_del(&file_cdev);
- file_cdev.ops = NULL;
- unregister_chrdev_region(major_dev, 1);
+ return 0;
+
+error_unregister:
+ platform_device_unregister(&visorchipset_platform_device);
+
+error_cancel_work:
+ cancel_delayed_work_sync(&periodic_controlvm_work);
+ visorchipset_file_cleanup(major_dev);
+
+error_destroy_payload:
+ destroy_controlvm_payload_info(&controlvm_payload_info);
+
+error_destroy_channel:
+ visorchannel_destroy(controlvm_channel);
+
+error:
+ POSTCODE_LINUX_3(CHIPSET_INIT_FAILURE_PC, err, POSTCODE_SEVERITY_ERR);
+ return err;
}
static int
@@ -2392,8 +2276,6 @@
cancel_delayed_work_sync(&periodic_controlvm_work);
destroy_controlvm_payload_info(&controlvm_payload_info);
- memset(&g_chipset_msg_hdr, 0, sizeof(struct controlvm_message_header));
-
visorchannel_destroy(controlvm_channel);
visorchipset_file_cleanup(visorchipset_platform_device.dev.devt);
@@ -2460,12 +2342,8 @@
MODULE_PARM_DESC(visorchipset_major,
"major device number to use for the device node");
module_param_named(visorbusregwait, visorchipset_visorbusregwait, int, S_IRUGO);
-MODULE_PARM_DESC(visorchipset_visorbusreqwait,
+MODULE_PARM_DESC(visorchipset_visorbusregwait,
"1 to have the module wait for the visor bus to register");
-module_param_named(holdchipsetready, visorchipset_holdchipsetready,
- int, S_IRUGO);
-MODULE_PARM_DESC(visorchipset_holdchipsetready,
- "1 to hold response to CHIPSET_READY");
module_init(init_unisys);
module_exit(exit_unisys);
diff --git a/drivers/staging/unisys/visorhba/visorhba_main.c b/drivers/staging/unisys/visorhba/visorhba_main.c
index e93bb1d..6a4570d 100644
--- a/drivers/staging/unisys/visorhba/visorhba_main.c
+++ b/drivers/staging/unisys/visorhba/visorhba_main.c
@@ -52,6 +52,8 @@
static ssize_t info_debugfs_read(struct file *file, char __user *buf,
size_t len, loff_t *offset);
+static int set_no_disk_inquiry_result(unsigned char *buf,
+ size_t len, bool is_lun0);
static struct dentry *visorhba_debugfs_dir;
static const struct file_operations debugfs_info_fops = {
.read = info_debugfs_read,
@@ -83,12 +85,6 @@
MODULE_DEVICE_TABLE(visorbus, visorhba_channel_types);
MODULE_ALIAS("visorbus:" SPAR_VHBA_CHANNEL_PROTOCOL_UUID_STR);
-struct visor_thread_info {
- struct task_struct *task;
- struct completion has_stopped;
- int id;
-};
-
struct visordisk_info {
u32 valid;
u32 channel, id, lun; /* Disk Path */
@@ -135,7 +131,7 @@
struct visordisk_info head;
unsigned int max_buff_len;
int devnum;
- struct visor_thread_info threadinfo;
+ struct task_struct *thread;
int thread_wait_ms;
};
@@ -152,28 +148,36 @@
(iter->lun == match->lun))
/**
* visor_thread_start - starts a thread for the device
- * @thrinfo: The thread to start
* @threadfn: Function the thread starts
* @thrcontext: Context to pass to the thread, i.e. devdata
* @name: string describing name of thread
*
* Starts a thread for the device.
*
- * Return 0 on success;
+ * Return the task_struct * denoting the thread on success,
+ * or NULL on failure
*/
-static int visor_thread_start(struct visor_thread_info *thrinfo,
- int (*threadfn)(void *),
- void *thrcontext, char *name)
+static struct task_struct *visor_thread_start
+(int (*threadfn)(void *), void *thrcontext, char *name)
{
- /* used to stop the thread */
- init_completion(&thrinfo->has_stopped);
- thrinfo->task = kthread_run(threadfn, thrcontext, "%s", name);
- if (IS_ERR(thrinfo->task)) {
- thrinfo->id = 0;
- return PTR_ERR(thrinfo->task);
+ struct task_struct *task;
+
+ task = kthread_run(threadfn, thrcontext, "%s", name);
+ if (IS_ERR(task)) {
+ pr_err("visorbus failed to start thread\n");
+ return NULL;
}
- thrinfo->id = thrinfo->task->pid;
- return 0;
+ return task;
+}
+
+/**
+ * visor_thread_stop - stops the thread if it is running
+ */
+static void visor_thread_stop(struct task_struct *task)
+{
+ if (!task)
+ return; /* no thread running */
+ kthread_stop(task);
}
/**
@@ -231,16 +235,17 @@
int del)
{
unsigned long flags;
- void *sent = NULL;
+ void *sent;
- if (del < MAX_PENDING_REQUESTS) {
- spin_lock_irqsave(&devdata->privlock, flags);
- sent = devdata->pending[del].sent;
+ if (del >= MAX_PENDING_REQUESTS)
+ return NULL;
- devdata->pending[del].cmdtype = 0;
- devdata->pending[del].sent = NULL;
- spin_unlock_irqrestore(&devdata->privlock, flags);
- }
+ spin_lock_irqsave(&devdata->privlock, flags);
+ sent = devdata->pending[del].sent;
+
+ devdata->pending[del].cmdtype = 0;
+ devdata->pending[del].sent = NULL;
+ spin_unlock_irqrestore(&devdata->privlock, flags);
return sent;
}
@@ -681,7 +686,7 @@
/* Stop using the IOVM response queue (queue should be drained
* by the end)
*/
- kthread_stop(devdata->threadinfo.task);
+ visor_thread_stop(devdata->thread);
/* Fail commands that weren't completed */
spin_lock_irqsave(&devdata->privlock, flags);
@@ -772,6 +777,24 @@
}
}
+static int set_no_disk_inquiry_result(unsigned char *buf,
+ size_t len, bool is_lun0)
+{
+ if (!buf || len < NO_DISK_INQUIRY_RESULT_LEN)
+ return -EINVAL;
+ memset(buf, 0, NO_DISK_INQUIRY_RESULT_LEN);
+ buf[2] = SCSI_SPC2_VER;
+ if (is_lun0) {
+ buf[0] = DEV_DISK_CAPABLE_NOT_PRESENT;
+ buf[3] = DEV_HISUPPORT;
+ } else {
+ buf[0] = DEV_NOT_CAPABLE;
+ }
+ buf[4] = NO_DISK_INQUIRY_RESULT_LEN - 5;
+ strncpy(buf + 8, "DELLPSEUDO DEVICE .", NO_DISK_INQUIRY_RESULT_LEN - 8);
+ return 0;
+}
+
/**
* do_scsi_nolinuxstat - scsi command didn't have linuxstat
* @cmdrsp: response from IOVM
@@ -804,10 +827,8 @@
* a disk there so we'll present a processor
* there.
*/
- SET_NO_DISK_INQUIRY_RESULT(buf, cmdrsp->scsi.bufflen,
- scsidev->lun,
- DEV_DISK_CAPABLE_NOT_PRESENT,
- DEV_NOT_CAPABLE);
+ set_no_disk_inquiry_result(buf, (size_t)cmdrsp->scsi.bufflen,
+ scsidev->lun == 0);
if (scsi_sg_count(scsicmd) == 0) {
memcpy(scsi_sglist(scsicmd), buf,
@@ -929,14 +950,15 @@
struct diskaddremove *dar;
dar = kzalloc(sizeof(*dar), GFP_ATOMIC);
- if (dar) {
- dar->add = cmdrsp->disknotify.add;
- dar->shost = shost;
- dar->channel = cmdrsp->disknotify.channel;
- dar->id = cmdrsp->disknotify.id;
- dar->lun = cmdrsp->disknotify.lun;
- queue_disk_add_remove(dar);
- }
+ if (!dar)
+ return;
+
+ dar->add = cmdrsp->disknotify.add;
+ dar->shost = shost;
+ dar->channel = cmdrsp->disknotify.channel;
+ dar->id = cmdrsp->disknotify.id;
+ dar->lun = cmdrsp->disknotify.lun;
+ queue_disk_add_remove(dar);
}
/**
@@ -1064,8 +1086,8 @@
if (devdata->serverdown && !devdata->serverchangingstate)
devdata->serverchangingstate = true;
- visor_thread_start(&devdata->threadinfo, process_incoming_rsps,
- devdata, "vhba_incming");
+ devdata->thread = visor_thread_start(process_incoming_rsps, devdata,
+ "vhba_incming");
devdata->serverdown = false;
devdata->serverchangingstate = false;
@@ -1141,8 +1163,8 @@
goto err_scsi_remove_host;
devdata->thread_wait_ms = 2;
- visor_thread_start(&devdata->threadinfo, process_incoming_rsps,
- devdata, "vhba_incoming");
+ devdata->thread = visor_thread_start(process_incoming_rsps, devdata,
+ "vhba_incoming");
scsi_scan_host(scsihost);
@@ -1172,7 +1194,7 @@
return;
scsihost = devdata->scsihost;
- kthread_stop(devdata->threadinfo.task);
+ visor_thread_stop(devdata->thread);
scsi_remove_host(scsihost);
scsi_host_put(scsihost);
diff --git a/drivers/staging/unisys/visorinput/visorinput.c b/drivers/staging/unisys/visorinput/visorinput.c
index 13c0316..12a3570 100644
--- a/drivers/staging/unisys/visorinput/visorinput.c
+++ b/drivers/staging/unisys/visorinput/visorinput.c
@@ -123,9 +123,9 @@
[38] = KEY_L,
[39] = KEY_SEMICOLON,
[40] = KEY_APOSTROPHE,
- [41] = KEY_GRAVE, /* FIXME, '#' */
+ [41] = KEY_GRAVE,
[42] = KEY_LEFTSHIFT,
- [43] = KEY_BACKSLASH, /* FIXME, '~' */
+ [43] = KEY_BACKSLASH,
[44] = KEY_Z,
[45] = KEY_X,
[46] = KEY_C,
@@ -173,7 +173,7 @@
[88] = KEY_F12,
[90] = KEY_KPLEFTPAREN,
[91] = KEY_KPRIGHTPAREN,
- [92] = KEY_KPASTERISK, /* FIXME */
+ [92] = KEY_KPASTERISK,
[93] = KEY_KPASTERISK,
[94] = KEY_KPPLUS,
[95] = KEY_HELP,
@@ -467,18 +467,14 @@
break;
default:
led = -1;
- break;
+ return;
}
- if (led >= 0) {
- int old_state = (test_bit(led, visorinput_dev->led) != 0);
-
- if (old_state != desired_state) {
- input_report_key(visorinput_dev, keycode, 1);
- input_sync(visorinput_dev);
- input_report_key(visorinput_dev, keycode, 0);
- input_sync(visorinput_dev);
- __change_bit(led, visorinput_dev->led);
- }
+ if (test_bit(led, visorinput_dev->led) != desired_state) {
+ input_report_key(visorinput_dev, keycode, 1);
+ input_sync(visorinput_dev);
+ input_report_key(visorinput_dev, keycode, 0);
+ input_sync(visorinput_dev);
+ __change_bit(led, visorinput_dev->led);
}
}
diff --git a/drivers/staging/unisys/visornic/visornic_main.c b/drivers/staging/unisys/visornic/visornic_main.c
index be0d057..fd7c9a6 100644
--- a/drivers/staging/unisys/visornic/visornic_main.c
+++ b/drivers/staging/unisys/visornic/visornic_main.c
@@ -109,51 +109,46 @@
};
struct visornic_devdata {
- unsigned short enabled; /* 0 disabled 1 enabled to receive */
- unsigned short enab_dis_acked; /* NET_RCV_ENABLE/DISABLE acked by
- * IOPART
- */
+ /* 0 disabled 1 enabled to receive */
+ unsigned short enabled;
+ /* NET_RCV_ENABLE/DISABLE acked by IOPART */
+ unsigned short enab_dis_acked;
+
struct visor_device *dev;
struct net_device *netdev;
struct net_device_stats net_stats;
atomic_t interrupt_rcvd;
wait_queue_head_t rsp_queue;
struct sk_buff **rcvbuf;
- u64 incarnation_id; /* lets IOPART know about re-birth */
- unsigned short old_flags; /* flags as they were prior to
- * set_multicast_list
- */
- atomic_t usage; /* count of users */
- int num_rcv_bufs; /* indicates how many rcv buffers
- * the vnic will post
- */
+ /* incarnation_id lets IOPART know about re-birth */
+ u64 incarnation_id;
+ /* flags as they were prior to set_multicast_list */
+ unsigned short old_flags;
+ atomic_t usage; /* count of users */
+
+ /* number of rcv buffers the vnic will post */
+ int num_rcv_bufs;
int num_rcv_bufs_could_not_alloc;
atomic_t num_rcvbuf_in_iovm;
unsigned long alloc_failed_in_if_needed_cnt;
unsigned long alloc_failed_in_repost_rtn_cnt;
- unsigned long max_outstanding_net_xmits; /* absolute max number of
- * outstanding xmits - should
- * never hit this
- */
- unsigned long upper_threshold_net_xmits; /* high water mark for
- * calling netif_stop_queue()
- */
- unsigned long lower_threshold_net_xmits; /* high water mark for calling
- * netif_wake_queue()
- */
- struct sk_buff_head xmitbufhead; /* xmitbufhead is the head of the
- * xmit buffer list that have been
- * sent to the IOPART end
- */
+
+ /* absolute max number of outstanding xmits - should never hit this */
+ unsigned long max_outstanding_net_xmits;
+ /* high water mark for calling netif_stop_queue() */
+ unsigned long upper_threshold_net_xmits;
+ /* high water mark for calling netif_wake_queue() */
+ unsigned long lower_threshold_net_xmits;
+ /* xmitbufhead - head of the xmit buffer list sent to the IOPART end */
+ struct sk_buff_head xmitbufhead;
+
visorbus_state_complete_func server_down_complete_func;
struct work_struct timeout_reset;
- struct uiscmdrsp *cmdrsp_rcv; /* cmdrsp_rcv is used for
- * posting/unposting rcv buffers
- */
- struct uiscmdrsp *xmit_cmdrsp; /* used to issue NET_XMIT - there is
- * never more that one xmit in
- * progress at a time
- */
+ /* cmdrsp_rcv is used for posting/unposting rcv buffers */
+ struct uiscmdrsp *cmdrsp_rcv;
+ /* xmit_cmdrsp - issues NET_XMIT - only one active xmit at a time */
+ struct uiscmdrsp *xmit_cmdrsp;
+
bool server_down; /* IOPART is down */
bool server_change_state; /* Processing SERVER_CHANGESTATE msg */
bool going_away; /* device is being torn down */
@@ -173,18 +168,10 @@
unsigned long n_rcv1; /* # rcvs of 1 buffers */
unsigned long n_rcv2; /* # rcvs of 2 buffers */
unsigned long n_rcvx; /* # rcvs of >2 buffers */
- unsigned long found_repost_rcvbuf_cnt; /* # times we called
- * repost_rcvbuf_cnt
- */
- unsigned long repost_found_skb_cnt; /* # times found the skb */
- unsigned long n_repost_deficit; /* # times we couldn't find
- * all of the rcv buffers
- */
- unsigned long bad_rcv_buf; /* # times we negleted to
- * free the rcv skb because
- * we didn't know where it
- * came from
- */
+ unsigned long found_repost_rcvbuf_cnt; /* # repost_rcvbuf_cnt */
+ unsigned long repost_found_skb_cnt; /* # of found the skb */
+ unsigned long n_repost_deficit; /* # of lost rcv buffers */
+ unsigned long bad_rcv_buf; /* # of unknown rcv skb not freed */
unsigned long n_rcv_packets_not_accepted;/* # bogs rcv packets */
int queuefullmsg_logged;
@@ -209,18 +196,17 @@
* Return value indicates number of entries filled in frags
* Negative values indicate an error.
*/
-static unsigned int
+static int
visor_copy_fragsinfo_from_skb(struct sk_buff *skb, unsigned int firstfraglen,
unsigned int frags_max,
struct phys_info frags[])
{
- unsigned int count = 0, ii, size, offset = 0, numfrags;
+ unsigned int count = 0, frag, size, offset = 0, numfrags;
unsigned int total_count;
numfrags = skb_shinfo(skb)->nr_frags;
- /*
- * Compute the number of fragments this skb has, and if its more than
+ /* Compute the number of fragments this skb has, and if its more than
* frag array can hold, linearize the skb
*/
total_count = numfrags + (firstfraglen / PI_PAGE_SIZE);
@@ -257,23 +243,20 @@
if ((count + numfrags) > frags_max)
return -EINVAL;
- for (ii = 0; ii < numfrags; ii++) {
+ for (frag = 0; frag < numfrags; frag++) {
count = add_physinfo_entries(page_to_pfn(
- skb_frag_page(&skb_shinfo(skb)->frags[ii])),
- skb_shinfo(skb)->frags[ii].
+ skb_frag_page(&skb_shinfo(skb)->frags[frag])),
+ skb_shinfo(skb)->frags[frag].
page_offset,
- skb_shinfo(skb)->frags[ii].
+ skb_shinfo(skb)->frags[frag].
size, count, frags_max, frags);
- /*
- * add_physinfo_entries only returns
+ /* add_physinfo_entries only returns
* zero if the frags array is out of room
* That should never happen because we
* fail above, if count+numfrags > frags_max.
- * Given that theres no recovery mechanism from putting
- * half a packet in the I/O channel, panic here as this
- * should never happen
*/
- BUG_ON(!count);
+ if (!count)
+ return -EINVAL;
}
}
if (skb_shinfo(skb)->frag_list) {
@@ -299,8 +282,7 @@
const char __user *buffer,
size_t count, loff_t *ppos)
{
- /*
- * Don't want to break ABI here by having a debugfs
+ /* Don't want to break ABI here by having a debugfs
* file that no longer exists or is writable, so
* lets just make this a vestigual function
*/
@@ -308,8 +290,7 @@
}
/**
- * visornic_serverdown_complete - IOPART went down, need to pause
- * device
+ * visornic_serverdown_complete - IOPART went down, pause device
* @work: Work queue it was scheduled on
*
* The IO partition has gone down and we need to do some cleanup
@@ -344,7 +325,7 @@
}
/**
- * visornic_serverdown - Command has notified us that IOPARt is down
+ * visornic_serverdown - Command has notified us that IOPART is down
* @devdata: device that is being managed by IOPART
*
* Schedule the work needed to handle the server down request. Make
@@ -356,28 +337,38 @@
visorbus_state_complete_func complete_func)
{
unsigned long flags;
+ int err;
spin_lock_irqsave(&devdata->priv_lock, flags);
- if (!devdata->server_down && !devdata->server_change_state) {
- if (devdata->going_away) {
- spin_unlock_irqrestore(&devdata->priv_lock, flags);
- dev_dbg(&devdata->dev->device,
- "%s aborting because device removal pending\n",
- __func__);
- return -ENODEV;
- }
- devdata->server_change_state = true;
- devdata->server_down_complete_func = complete_func;
- spin_unlock_irqrestore(&devdata->priv_lock, flags);
- visornic_serverdown_complete(devdata);
- } else if (devdata->server_change_state) {
+ if (devdata->server_change_state) {
dev_dbg(&devdata->dev->device, "%s changing state\n",
__func__);
- spin_unlock_irqrestore(&devdata->priv_lock, flags);
- return -EINVAL;
+ err = -EINVAL;
+ goto err_unlock;
}
+ if (devdata->server_down) {
+ dev_dbg(&devdata->dev->device, "%s already down\n",
+ __func__);
+ err = -EINVAL;
+ goto err_unlock;
+ }
+ if (devdata->going_away) {
+ dev_dbg(&devdata->dev->device,
+ "%s aborting because device removal pending\n",
+ __func__);
+ err = -ENODEV;
+ goto err_unlock;
+ }
+ devdata->server_change_state = true;
+ devdata->server_down_complete_func = complete_func;
spin_unlock_irqrestore(&devdata->priv_lock, flags);
+
+ visornic_serverdown_complete(devdata);
return 0;
+
+err_unlock:
+ spin_unlock_irqrestore(&devdata->priv_lock, flags);
+ return err;
}
/**
@@ -395,20 +386,19 @@
/* NOTE: the first fragment in each rcv buffer is pointed to by
* rcvskb->data. For now all rcv buffers will be RCVPOST_BUF_SIZE
- * in length, so the firstfrag is large enough to hold 1514.
+ * in length, so the first frag is large enough to hold 1514.
*/
skb = alloc_skb(RCVPOST_BUF_SIZE, GFP_ATOMIC);
if (!skb)
return NULL;
skb->dev = netdev;
- skb->len = RCVPOST_BUF_SIZE;
/* current value of mtu doesn't come into play here; large
* packets will just end up using multiple rcv buffers all of
- * same size
+ * same size.
*/
- skb->data_len = 0; /* dev_alloc_skb already zeroes it out
- * for clarification.
- */
+ skb->len = RCVPOST_BUF_SIZE;
+ /* alloc_skb already zeroes it out for clarification. */
+ skb->data_len = 0;
return skb;
}
@@ -436,8 +426,8 @@
cmdrsp->net.type = NET_RCV_POST;
cmdrsp->cmdtype = CMD_NET_TYPE;
if (visorchannel_signalinsert(devdata->dev->visorchannel,
- IOCHAN_TO_IOPART,
- cmdrsp)) {
+ IOCHAN_TO_IOPART,
+ cmdrsp)) {
atomic_inc(&devdata->num_rcvbuf_in_iovm);
devdata->chstat.sent_post++;
} else {
@@ -465,8 +455,8 @@
devdata->cmdrsp_rcv->net.type = NET_RCV_ENBDIS;
devdata->cmdrsp_rcv->cmdtype = CMD_NET_TYPE;
if (visorchannel_signalinsert(devdata->dev->visorchannel,
- IOCHAN_TO_IOPART,
- devdata->cmdrsp_rcv))
+ IOCHAN_TO_IOPART,
+ devdata->cmdrsp_rcv))
devdata->chstat.sent_enbdis++;
}
@@ -872,8 +862,7 @@
if (vnic_hit_high_watermark(devdata,
devdata->max_outstanding_net_xmits)) {
- /* too many NET_XMITs queued over to IOVM - need to wait
- */
+ /* extra NET_XMITs queued over to IOVM - need to wait */
devdata->chstat.reject_count++;
if (!devdata->queuefullmsg_logged &&
((devdata->chstat.reject_count & 0x3ff) == 1))
@@ -950,16 +939,12 @@
devdata->net_stats.tx_bytes += skb->len;
devdata->chstat.sent_xmit++;
- /* check to see if we have hit the high watermark for
- * netif_stop_queue()
- */
+ /* check if we have hit the high watermark for netif_stop_queue() */
if (vnic_hit_high_watermark(devdata,
devdata->upper_threshold_net_xmits)) {
- /* too many NET_XMITs queued over to IOVM - need to wait */
- netif_stop_queue(netdev); /* calling stop queue - call
- * netif_wake_queue() after lower
- * threshold
- */
+ /* extra NET_XMITs queued over to IOVM - need to wait */
+ /* stop queue - call netif_wake_queue() after lower threshold */
+ netif_stop_queue(netdev);
dev_dbg(&netdev->dev,
"%s busy - invoking iovm flow control\n",
__func__);
@@ -1312,16 +1297,13 @@
break;
}
}
+ /* accept pkt, dest matches a multicast addr */
if (found_mc)
- break; /* accept packet, dest
- * matches a multicast
- * address
- */
+ break;
}
+ /* accept packet, h_dest must match vnic mac address */
} else if (skb->pkt_type == PACKET_HOST) {
- break; /* accept packet, h_dest must match vnic
- * mac address
- */
+ break;
} else if (skb->pkt_type == PACKET_OTHERHOST) {
/* something is not right */
dev_err(&devdata->netdev->dev,
@@ -1409,14 +1391,10 @@
if (!vbuf)
return -ENOMEM;
- /* for each vnic channel
- * dump out channel specific data
- */
+ /* for each vnic channel dump out channel specific data */
rcu_read_lock();
for_each_netdev_rcu(current->nsproxy->net_ns, dev) {
- /*
- * Only consider netdevs that are visornic, and are open
- */
+ /* Only consider netdevs that are visornic, and are open */
if ((dev->netdev_ops != &visornic_dev_ops) ||
(!netif_queue_stopped(dev)))
continue;
@@ -1643,12 +1621,12 @@
/* ASSERT netdev == vnicinfo->netdev; */
if ((netdev == devdata->netdev) &&
netif_queue_stopped(netdev)) {
- /* check to see if we have crossed
- * the lower watermark for
- * netif_wake_queue()
+ /* check if we have crossed the lower watermark
+ * for netif_wake_queue()
*/
- if (vnic_hit_low_watermark(devdata,
- devdata->lower_threshold_net_xmits)) {
+ if (vnic_hit_low_watermark
+ (devdata,
+ devdata->lower_threshold_net_xmits)) {
/* enough NET_XMITs completed
* so can restart netif queue
*/
@@ -1712,10 +1690,7 @@
send_rcv_posts_if_needed(devdata);
service_resp_queue(devdata->cmdrsp, devdata, &rx_count, budget);
- /*
- * If there aren't any more packets to receive
- * stop the poll
- */
+ /* If there aren't any more packets to receive stop the poll */
if (rx_count < budget)
napi_complete(napi);
@@ -1867,8 +1842,7 @@
setup_timer(&devdata->irq_poll_timer, poll_for_irq,
(unsigned long)devdata);
- /*
- * Note: This time has to start running before the while
+ /* Note: This time has to start running before the while
* loop below because the napi routine is responsible for
* setting enab_dis_acked
*/
@@ -1897,8 +1871,7 @@
/* Let's start our threads to get responses */
netif_napi_add(netdev, &devdata->napi, visornic_poll, NAPI_WEIGHT);
- /*
- * Note: Interupts have to be enable before the while
+ /* Note: Interupts have to be enable before the while
* loop below because the napi routine is responsible for
* setting enab_dis_acked
*/
diff --git a/drivers/staging/vme/devices/vme_pio2_gpio.c b/drivers/staging/vme/devices/vme_pio2_gpio.c
index 6d361201..ba9fe3b 100644
--- a/drivers/staging/vme/devices/vme_pio2_gpio.c
+++ b/drivers/staging/vme/devices/vme_pio2_gpio.c
@@ -92,7 +92,7 @@
}
/* Directionality configured at board build - send appropriate response */
-static int pio2_gpio_dir_in(struct gpio_chip *chip, unsigned offset)
+static int pio2_gpio_dir_in(struct gpio_chip *chip, unsigned int offset)
{
int data;
struct pio2_card *card = gpiochip_get_data(chip);
@@ -111,7 +111,8 @@
}
/* Directionality configured at board build - send appropriate response */
-static int pio2_gpio_dir_out(struct gpio_chip *chip, unsigned offset, int value)
+static int pio2_gpio_dir_out(struct gpio_chip *chip,
+ unsigned int offset, int value)
{
int data;
struct pio2_card *card = gpiochip_get_data(chip);
diff --git a/drivers/staging/vt6655/baseband.c b/drivers/staging/vt6655/baseband.c
index 1e6c0c4..654d072 100644
--- a/drivers/staging/vt6655/baseband.c
+++ b/drivers/staging/vt6655/baseband.c
@@ -36,8 +36,10 @@
* Revision History:
* 06-10-2003 Bryan YC Fan: Re-write codes to support VT3253 spec.
* 08-07-2003 Bryan YC Fan: Add MAXIM2827/2825 and RFMD2959 support.
- * 08-26-2003 Kyle Hsu : Modify BBuGetFrameTime() and BBvCalculateParameter().
- * cancel the setting of MAC_REG_SOFTPWRCTL on BBbVT3253Init().
+ * 08-26-2003 Kyle Hsu : Modify BBuGetFrameTime() and
+ * BBvCalculateParameter().
+ * cancel the setting of MAC_REG_SOFTPWRCTL on
+ * BBbVT3253Init().
* Add the comments.
* 09-01-2003 Bryan YC Fan: RF & BB tables updated.
* Modified BBvLoopbackOn & BBvLoopbackOff().
@@ -66,7 +68,7 @@
/*--------------------- Static Variables --------------------------*/
#define CB_VT3253_INIT_FOR_RFMD 446
-static unsigned char byVT3253InitTab_RFMD[CB_VT3253_INIT_FOR_RFMD][2] = {
+static const unsigned char byVT3253InitTab_RFMD[CB_VT3253_INIT_FOR_RFMD][2] = {
{0x00, 0x30},
{0x01, 0x00},
{0x02, 0x00},
@@ -516,7 +518,7 @@
};
#define CB_VT3253B0_INIT_FOR_RFMD 256
-static unsigned char byVT3253B0_RFMD[CB_VT3253B0_INIT_FOR_RFMD][2] = {
+static const unsigned char byVT3253B0_RFMD[CB_VT3253B0_INIT_FOR_RFMD][2] = {
{0x00, 0x31},
{0x01, 0x00},
{0x02, 0x00},
@@ -777,7 +779,8 @@
#define CB_VT3253B0_AGC_FOR_RFMD2959 195
/* For RFMD2959 */
-static unsigned char byVT3253B0_AGC4_RFMD2959[CB_VT3253B0_AGC_FOR_RFMD2959][2] = {
+static
+unsigned char byVT3253B0_AGC4_RFMD2959[CB_VT3253B0_AGC_FOR_RFMD2959][2] = {
{0xF0, 0x00},
{0xF1, 0x3E},
{0xF0, 0x80},
@@ -977,7 +980,8 @@
#define CB_VT3253B0_INIT_FOR_AIROHA2230 256
/* For AIROHA */
-static unsigned char byVT3253B0_AIROHA2230[CB_VT3253B0_INIT_FOR_AIROHA2230][2] = {
+static
+unsigned char byVT3253B0_AIROHA2230[CB_VT3253B0_INIT_FOR_AIROHA2230][2] = {
{0x00, 0x31},
{0x01, 0x00},
{0x02, 0x00},
@@ -2160,9 +2164,13 @@
/* {{ RobertYu:20050223, request by JerryChung */
- /* Init ANT B select,TX Config CR09 = 0x61->0x45, 0x45->0x41(VC1/VC2 define, make the ANT_A, ANT_B inverted) */
+ /* Init ANT B select,TX Config CR09 = 0x61->0x45,
+ * 0x45->0x41(VC1/VC2 define, make the ANT_A, ANT_B inverted)
+ */
/*bResult &= BBbWriteEmbedded(dwIoBase,0x09,0x41);*/
- /* Init ANT B select,RX Config CR10 = 0x28->0x2A, 0x2A->0x28(VC1/VC2 define, make the ANT_A, ANT_B inverted) */
+ /* Init ANT B select,RX Config CR10 = 0x28->0x2A,
+ * 0x2A->0x28(VC1/VC2 define, make the ANT_A, ANT_B inverted)
+ */
/*bResult &= BBbWriteEmbedded(dwIoBase,0x0a,0x28);*/
/* Select VC1/VC2, CR215 = 0x02->0x06 */
bResult &= BBbWriteEmbedded(priv, 0xd7, 0x06);
diff --git a/drivers/staging/vt6655/baseband.h b/drivers/staging/vt6655/baseband.h
index 43a4fb1..b4e8c43 100644
--- a/drivers/staging/vt6655/baseband.h
+++ b/drivers/staging/vt6655/baseband.h
@@ -77,8 +77,10 @@
void vnt_get_phy_field(struct vnt_private *, u32 frame_length,
u16 tx_rate, u8 pkt_type, struct vnt_phy_field *);
-bool BBbReadEmbedded(struct vnt_private *, unsigned char byBBAddr, unsigned char *pbyData);
-bool BBbWriteEmbedded(struct vnt_private *, unsigned char byBBAddr, unsigned char byData);
+bool BBbReadEmbedded(struct vnt_private *, unsigned char byBBAddr,
+ unsigned char *pbyData);
+bool BBbWriteEmbedded(struct vnt_private *, unsigned char byBBAddr,
+ unsigned char byData);
void BBvSetShortSlotTime(struct vnt_private *);
void BBvSetVGAGainOffset(struct vnt_private *, unsigned char byData);
diff --git a/drivers/staging/vt6655/card.c b/drivers/staging/vt6655/card.c
index 3d33812..afb1e8b 100644
--- a/drivers/staging/vt6655/card.c
+++ b/drivers/staging/vt6655/card.c
@@ -336,7 +336,8 @@
}
if (priv->byCWMaxMin != byCWMaxMin) {
priv->byCWMaxMin = byCWMaxMin;
- VNSvOutPortB(priv->PortOffset + MAC_REG_CWMAXMIN0, priv->byCWMaxMin);
+ VNSvOutPortB(priv->PortOffset + MAC_REG_CWMAXMIN0,
+ priv->byCWMaxMin);
}
priv->byPacketType = CARDbyGetPktType(priv);
@@ -373,9 +374,12 @@
qwTSFOffset = CARDqGetTSFOffset(byRxRate, qwBSSTimestamp,
local_tsf);
/* adjust TSF, HW's TSF add TSF Offset reg */
- VNSvOutPortD(priv->PortOffset + MAC_REG_TSFOFST, (u32)qwTSFOffset);
- VNSvOutPortD(priv->PortOffset + MAC_REG_TSFOFST + 4, (u32)(qwTSFOffset >> 32));
- MACvRegBitsOn(priv->PortOffset, MAC_REG_TFTCTL, TFTCTL_TSFSYNCEN);
+ VNSvOutPortD(priv->PortOffset + MAC_REG_TSFOFST,
+ (u32)qwTSFOffset);
+ VNSvOutPortD(priv->PortOffset + MAC_REG_TSFOFST + 4,
+ (u32)(qwTSFOffset >> 32));
+ MACvRegBitsOn(priv->PortOffset, MAC_REG_TFTCTL,
+ TFTCTL_TSFSYNCEN);
}
return true;
}
@@ -407,7 +411,8 @@
priv->wBeaconInterval = wBeaconInterval;
/* Set NextTBTT */
VNSvOutPortD(priv->PortOffset + MAC_REG_NEXTTBTT, (u32)qwNextTBTT);
- VNSvOutPortD(priv->PortOffset + MAC_REG_NEXTTBTT + 4, (u32)(qwNextTBTT >> 32));
+ VNSvOutPortD(priv->PortOffset + MAC_REG_NEXTTBTT + 4,
+ (u32)(qwNextTBTT >> 32));
MACvRegBitsOn(priv->PortOffset, MAC_REG_TFTCTL, TFTCTL_TBTTSYNCEN);
return true;
@@ -433,15 +438,19 @@
switch (priv->byRFType) {
case RF_RFMD2959:
- MACvWordRegBitsOff(priv->PortOffset, MAC_REG_SOFTPWRCTL, SOFTPWRCTL_TXPEINV);
- MACvWordRegBitsOn(priv->PortOffset, MAC_REG_SOFTPWRCTL, SOFTPWRCTL_SWPE1);
+ MACvWordRegBitsOff(priv->PortOffset, MAC_REG_SOFTPWRCTL,
+ SOFTPWRCTL_TXPEINV);
+ MACvWordRegBitsOn(priv->PortOffset, MAC_REG_SOFTPWRCTL,
+ SOFTPWRCTL_SWPE1);
break;
case RF_AIROHA:
case RF_AL2230S:
case RF_AIROHA7230:
- MACvWordRegBitsOff(priv->PortOffset, MAC_REG_SOFTPWRCTL, SOFTPWRCTL_SWPE2);
- MACvWordRegBitsOff(priv->PortOffset, MAC_REG_SOFTPWRCTL, SOFTPWRCTL_SWPE3);
+ MACvWordRegBitsOff(priv->PortOffset, MAC_REG_SOFTPWRCTL,
+ SOFTPWRCTL_SWPE2);
+ MACvWordRegBitsOff(priv->PortOffset, MAC_REG_SOFTPWRCTL,
+ SOFTPWRCTL_SWPE3);
break;
}
@@ -451,7 +460,8 @@
priv->bRadioOff = true;
pr_debug("chester power off\n");
- MACvRegBitsOn(priv->PortOffset, MAC_REG_GPIOCTL0, LED_ACTSET); /* LED issue */
+ MACvRegBitsOn(priv->PortOffset, MAC_REG_GPIOCTL0,
+ LED_ACTSET); /* LED issue */
return bResult;
}
@@ -488,21 +498,24 @@
switch (priv->byRFType) {
case RF_RFMD2959:
- MACvWordRegBitsOn(priv->PortOffset, MAC_REG_SOFTPWRCTL, SOFTPWRCTL_TXPEINV);
- MACvWordRegBitsOff(priv->PortOffset, MAC_REG_SOFTPWRCTL, SOFTPWRCTL_SWPE1);
+ MACvWordRegBitsOn(priv->PortOffset, MAC_REG_SOFTPWRCTL,
+ SOFTPWRCTL_TXPEINV);
+ MACvWordRegBitsOff(priv->PortOffset, MAC_REG_SOFTPWRCTL,
+ SOFTPWRCTL_SWPE1);
break;
case RF_AIROHA:
case RF_AL2230S:
case RF_AIROHA7230:
- MACvWordRegBitsOn(priv->PortOffset, MAC_REG_SOFTPWRCTL, (SOFTPWRCTL_SWPE2 |
- SOFTPWRCTL_SWPE3));
+ MACvWordRegBitsOn(priv->PortOffset, MAC_REG_SOFTPWRCTL,
+ (SOFTPWRCTL_SWPE2 | SOFTPWRCTL_SWPE3));
break;
}
priv->bRadioOff = false;
pr_debug("chester power on\n");
- MACvRegBitsOff(priv->PortOffset, MAC_REG_GPIOCTL0, LED_ACTSET); /* LED issue */
+ MACvRegBitsOff(priv->PortOffset, MAC_REG_GPIOCTL0,
+ LED_ACTSET); /* LED issue */
return bResult;
}
@@ -717,55 +730,72 @@
bb_type,
&byTxRate,
&byRsvTime);
- VNSvOutPortW(priv->PortOffset + MAC_REG_RSPINF_A_6, MAKEWORD(byTxRate, byRsvTime));
+ VNSvOutPortW(priv->PortOffset + MAC_REG_RSPINF_A_6,
+ MAKEWORD(byTxRate, byRsvTime));
/* RSPINF_a_9 */
s_vCalculateOFDMRParameter(RATE_9M,
bb_type,
&byTxRate,
&byRsvTime);
- VNSvOutPortW(priv->PortOffset + MAC_REG_RSPINF_A_9, MAKEWORD(byTxRate, byRsvTime));
+ VNSvOutPortW(priv->PortOffset + MAC_REG_RSPINF_A_9,
+ MAKEWORD(byTxRate, byRsvTime));
/* RSPINF_a_12 */
s_vCalculateOFDMRParameter(RATE_12M,
bb_type,
&byTxRate,
&byRsvTime);
- VNSvOutPortW(priv->PortOffset + MAC_REG_RSPINF_A_12, MAKEWORD(byTxRate, byRsvTime));
+ VNSvOutPortW(priv->PortOffset + MAC_REG_RSPINF_A_12,
+ MAKEWORD(byTxRate, byRsvTime));
/* RSPINF_a_18 */
s_vCalculateOFDMRParameter(RATE_18M,
bb_type,
&byTxRate,
&byRsvTime);
- VNSvOutPortW(priv->PortOffset + MAC_REG_RSPINF_A_18, MAKEWORD(byTxRate, byRsvTime));
+ VNSvOutPortW(priv->PortOffset + MAC_REG_RSPINF_A_18,
+ MAKEWORD(byTxRate, byRsvTime));
/* RSPINF_a_24 */
s_vCalculateOFDMRParameter(RATE_24M,
bb_type,
&byTxRate,
&byRsvTime);
- VNSvOutPortW(priv->PortOffset + MAC_REG_RSPINF_A_24, MAKEWORD(byTxRate, byRsvTime));
+ VNSvOutPortW(priv->PortOffset + MAC_REG_RSPINF_A_24,
+ MAKEWORD(byTxRate, byRsvTime));
/* RSPINF_a_36 */
- s_vCalculateOFDMRParameter(CARDwGetOFDMControlRate((void *)priv, RATE_36M),
+ s_vCalculateOFDMRParameter(CARDwGetOFDMControlRate(
+ (void *)priv,
+ RATE_36M),
bb_type,
&byTxRate,
&byRsvTime);
- VNSvOutPortW(priv->PortOffset + MAC_REG_RSPINF_A_36, MAKEWORD(byTxRate, byRsvTime));
+ VNSvOutPortW(priv->PortOffset + MAC_REG_RSPINF_A_36,
+ MAKEWORD(byTxRate, byRsvTime));
/* RSPINF_a_48 */
- s_vCalculateOFDMRParameter(CARDwGetOFDMControlRate((void *)priv, RATE_48M),
+ s_vCalculateOFDMRParameter(CARDwGetOFDMControlRate(
+ (void *)priv,
+ RATE_48M),
bb_type,
&byTxRate,
&byRsvTime);
- VNSvOutPortW(priv->PortOffset + MAC_REG_RSPINF_A_48, MAKEWORD(byTxRate, byRsvTime));
+ VNSvOutPortW(priv->PortOffset + MAC_REG_RSPINF_A_48,
+ MAKEWORD(byTxRate, byRsvTime));
/* RSPINF_a_54 */
- s_vCalculateOFDMRParameter(CARDwGetOFDMControlRate((void *)priv, RATE_54M),
+ s_vCalculateOFDMRParameter(CARDwGetOFDMControlRate(
+ (void *)priv,
+ RATE_54M),
bb_type,
&byTxRate,
&byRsvTime);
- VNSvOutPortW(priv->PortOffset + MAC_REG_RSPINF_A_54, MAKEWORD(byTxRate, byRsvTime));
+ VNSvOutPortW(priv->PortOffset + MAC_REG_RSPINF_A_54,
+ MAKEWORD(byTxRate, byRsvTime));
/* RSPINF_a_72 */
- s_vCalculateOFDMRParameter(CARDwGetOFDMControlRate((void *)priv, RATE_54M),
+ s_vCalculateOFDMRParameter(CARDwGetOFDMControlRate(
+ (void *)priv,
+ RATE_54M),
bb_type,
&byTxRate,
&byRsvTime);
- VNSvOutPortW(priv->PortOffset + MAC_REG_RSPINF_A_72, MAKEWORD(byTxRate, byRsvTime));
+ VNSvOutPortW(priv->PortOffset + MAC_REG_RSPINF_A_72,
+ MAKEWORD(byTxRate, byRsvTime));
/* Set to Page0 */
MACvSelectPage0(priv->PortOffset);
@@ -830,7 +860,8 @@
*
* Return Value: none
*/
-void CARDvSetLoopbackMode(struct vnt_private *priv, unsigned short wLoopbackMode)
+void CARDvSetLoopbackMode(struct vnt_private *priv,
+ unsigned short wLoopbackMode)
{
switch (wLoopbackMode) {
case CARD_LB_NONE:
@@ -965,7 +996,8 @@
*
* Return Value: none
*/
-void CARDvSetFirstNextTBTT(struct vnt_private *priv, unsigned short wBeaconInterval)
+void CARDvSetFirstNextTBTT(struct vnt_private *priv,
+ unsigned short wBeaconInterval)
{
void __iomem *dwIoBase = priv->PortOffset;
u64 qwNextTBTT = 0;
@@ -993,7 +1025,8 @@
*
* Return Value: none
*/
-void CARDvUpdateNextTBTT(struct vnt_private *priv, u64 qwTSF, unsigned short wBeaconInterval)
+void CARDvUpdateNextTBTT(struct vnt_private *priv, u64 qwTSF,
+ unsigned short wBeaconInterval)
{
void __iomem *dwIoBase = priv->PortOffset;
diff --git a/drivers/staging/vt6655/card.h b/drivers/staging/vt6655/card.h
index 16cca49..0203c7f 100644
--- a/drivers/staging/vt6655/card.h
+++ b/drivers/staging/vt6655/card.h
@@ -38,7 +38,8 @@
* LOBYTE is MAC LB mode, HIBYTE is MII LB mode
*/
#define CARD_LB_NONE MAKEWORD(MAC_LB_NONE, 0)
-#define CARD_LB_MAC MAKEWORD(MAC_LB_INTERNAL, 0) /* PHY must ISO, avoid MAC loopback packet go out */
+/* PHY must ISO, avoid MAC loopback packet go out */
+#define CARD_LB_MAC MAKEWORD(MAC_LB_INTERNAL, 0)
#define CARD_LB_PHY MAKEWORD(MAC_LB_EXT, 0)
#define DEFAULT_MSDU_LIFETIME 512 /* ms */
@@ -71,8 +72,10 @@
bool CARDbIsOFDMinBasicRate(struct vnt_private *);
void CARDvSetLoopbackMode(struct vnt_private *, unsigned short wLoopbackMode);
bool CARDbSoftwareReset(struct vnt_private *);
-void CARDvSetFirstNextTBTT(struct vnt_private *, unsigned short wBeaconInterval);
-void CARDvUpdateNextTBTT(struct vnt_private *, u64 qwTSF, unsigned short wBeaconInterval);
+void CARDvSetFirstNextTBTT(struct vnt_private *,
+ unsigned short wBeaconInterval);
+void CARDvUpdateNextTBTT(struct vnt_private *, u64 qwTSF,
+ unsigned short wBeaconInterval);
bool CARDbGetCurrentTSF(struct vnt_private *, u64 *pqwCurrTSF);
u64 CARDqGetNextTBTT(u64 qwTSF, unsigned short wBeaconInterval);
u64 CARDqGetTSFOffset(unsigned char byRxRate, u64 qwTSF1, u64 qwTSF2);
diff --git a/drivers/staging/vt6655/desc.h b/drivers/staging/vt6655/desc.h
index 9fbc717..2d7f6ae 100644
--- a/drivers/staging/vt6655/desc.h
+++ b/drivers/staging/vt6655/desc.h
@@ -157,7 +157,8 @@
/* TD_INFO flags control bit */
#define TD_FLAGS_NETIF_SKB 0x01 /* check if need release skb */
-#define TD_FLAGS_PRIV_SKB 0x02 /* check if called from private skb (hostap) */
+/* check if called from private skb (hostap) */
+#define TD_FLAGS_PRIV_SKB 0x02
#define TD_FLAGS_PS_RETRY 0x04 /* check if PS STA frame re-transmit */
/*
diff --git a/drivers/staging/vt6655/mac.c b/drivers/staging/vt6655/mac.c
index 45196c6..8e13f7f 100644
--- a/drivers/staging/vt6655/mac.c
+++ b/drivers/staging/vt6655/mac.c
@@ -47,7 +47,8 @@
*
* Revision History:
* 08-22-2003 Kyle Hsu : Porting MAC functions from sim53
- * 09-03-2003 Bryan YC Fan : Add MACvClearBusSusInd()& MACvEnableBusSusEn()
+ * 09-03-2003 Bryan YC Fan : Add MACvClearBusSusInd()&
+ * MACvEnableBusSusEn()
* 09-18-2003 Jerry Chen : Add MACvSetKeyEntry & MACvDisableKeyEntry
*
*/
@@ -138,7 +139,8 @@
* Return Value: none
*
*/
-void MACvSetShortRetryLimit(struct vnt_private *priv, unsigned char byRetryLimit)
+void MACvSetShortRetryLimit(struct vnt_private *priv,
+ unsigned char byRetryLimit)
{
void __iomem *io_base = priv->PortOffset;
/* set SRT */
@@ -160,7 +162,8 @@
* Return Value: none
*
*/
-void MACvSetLongRetryLimit(struct vnt_private *priv, unsigned char byRetryLimit)
+void MACvSetLongRetryLimit(struct vnt_private *priv,
+ unsigned char byRetryLimit)
{
void __iomem *io_base = priv->PortOffset;
/* set LRT */
@@ -304,7 +307,8 @@
/*
* Description:
- * save some important register's value, then do reset, then restore register's value
+ * save some important register's value, then do reset, then restore
+ * register's value
*
* Parameters:
* In:
@@ -738,7 +742,8 @@
* Return Value: none
*
*/
-void MACvOneShotTimer1MicroSec(struct vnt_private *priv, unsigned int uDelayTime)
+void MACvOneShotTimer1MicroSec(struct vnt_private *priv,
+ unsigned int uDelayTime)
{
void __iomem *io_base = priv->PortOffset;
diff --git a/drivers/staging/vt6655/srom.c b/drivers/staging/vt6655/srom.c
index 9ec49e6..ee99277 100644
--- a/drivers/staging/vt6655/srom.c
+++ b/drivers/staging/vt6655/srom.c
@@ -72,7 +72,8 @@
* Return Value: data read
*
*/
-unsigned char SROMbyReadEmbedded(void __iomem *dwIoBase, unsigned char byContntOffset)
+unsigned char SROMbyReadEmbedded(void __iomem *dwIoBase,
+ unsigned char byContntOffset)
{
unsigned short wDelay, wNoACK;
unsigned char byWait;
@@ -124,7 +125,8 @@
/* ii = Rom Address */
for (ii = 0; ii < EEP_MAX_CONTEXT_SIZE; ii++) {
- *pbyEepromRegs = SROMbyReadEmbedded(dwIoBase, (unsigned char)ii);
+ *pbyEepromRegs = SROMbyReadEmbedded(dwIoBase,
+ (unsigned char)ii);
pbyEepromRegs++;
}
}
@@ -141,7 +143,8 @@
* Return Value: none
*
*/
-void SROMvReadEtherAddress(void __iomem *dwIoBase, unsigned char *pbyEtherAddress)
+void SROMvReadEtherAddress(void __iomem *dwIoBase,
+ unsigned char *pbyEtherAddress)
{
unsigned char ii;
diff --git a/drivers/staging/vt6656/baseband.c b/drivers/staging/vt6656/baseband.c
index 9417c93..882fe54 100644
--- a/drivers/staging/vt6656/baseband.c
+++ b/drivers/staging/vt6656/baseband.c
@@ -138,7 +138,7 @@
*
*/
unsigned int vnt_get_frame_time(u8 preamble_type, u8 pkt_type,
- unsigned int frame_length, u16 tx_rate)
+ unsigned int frame_length, u16 tx_rate)
{
unsigned int frame_time;
unsigned int preamble;
@@ -195,7 +195,7 @@
*
*/
void vnt_get_phy_field(struct vnt_private *priv, u32 frame_length,
- u16 tx_rate, u8 pkt_type, struct vnt_phy_field *phy)
+ u16 tx_rate, u8 pkt_type, struct vnt_phy_field *phy)
{
u32 bit_count;
u32 count = 0;
@@ -355,7 +355,7 @@
}
vnt_control_out(priv, MESSAGE_TYPE_SET_ANTMD,
- (u16)antenna_mode, 0, 0, NULL);
+ (u16)antenna_mode, 0, 0, NULL);
}
/*
@@ -383,7 +383,7 @@
u8 data;
status = vnt_control_in(priv, MESSAGE_TYPE_READ, 0,
- MESSAGE_REQUEST_EEPROM, EEP_MAX_CONTEXT_SIZE,
+ MESSAGE_REQUEST_EEPROM, EEP_MAX_CONTEXT_SIZE,
priv->eeprom);
if (status != STATUS_SUCCESS)
return false;
@@ -393,7 +393,7 @@
dev_dbg(&priv->usb->dev, "RF Type %d\n", priv->rf_type);
if ((priv->rf_type == RF_AL2230) ||
- (priv->rf_type == RF_AL2230S)) {
+ (priv->rf_type == RF_AL2230S)) {
priv->bb_rx_conf = vnt_vt3184_al2230[10];
length = sizeof(vnt_vt3184_al2230);
addr = vnt_vt3184_al2230;
@@ -457,21 +457,21 @@
memcpy(array, addr, length);
vnt_control_out(priv, MESSAGE_TYPE_WRITE, 0,
- MESSAGE_REQUEST_BBREG, length, array);
+ MESSAGE_REQUEST_BBREG, length, array);
memcpy(array, agc, length_agc);
vnt_control_out(priv, MESSAGE_TYPE_WRITE, 0,
- MESSAGE_REQUEST_BBAGC, length_agc, array);
+ MESSAGE_REQUEST_BBAGC, length_agc, array);
if ((priv->rf_type == RF_VT3226) ||
- (priv->rf_type == RF_VT3342A0)) {
+ (priv->rf_type == RF_VT3342A0)) {
vnt_control_out_u8(priv, MESSAGE_REQUEST_MACREG,
- MAC_REG_ITRTMSET, 0x23);
+ MAC_REG_ITRTMSET, 0x23);
vnt_mac_reg_bits_on(priv, MAC_REG_PAPEDELAY, 0x01);
} else if (priv->rf_type == RF_VT3226D0) {
vnt_control_out_u8(priv, MESSAGE_REQUEST_MACREG,
- MAC_REG_ITRTMSET, 0x11);
+ MAC_REG_ITRTMSET, 0x11);
vnt_mac_reg_bits_on(priv, MAC_REG_PAPEDELAY, 0x01);
}
@@ -482,12 +482,12 @@
/* Fix for TX USB resets from vendors driver */
vnt_control_in(priv, MESSAGE_TYPE_READ, USB_REG4,
- MESSAGE_REQUEST_MEM, sizeof(data), &data);
+ MESSAGE_REQUEST_MEM, sizeof(data), &data);
data |= 0x2;
vnt_control_out(priv, MESSAGE_TYPE_WRITE, USB_REG4,
- MESSAGE_REQUEST_MEM, sizeof(data), &data);
+ MESSAGE_REQUEST_MEM, sizeof(data), &data);
return true;
}
@@ -814,7 +814,7 @@
priv->bb_pre_ed_index = ed_inx;
dev_dbg(&priv->usb->dev, "%s bb_pre_ed_rssi %d\n",
- __func__, priv->bb_pre_ed_rssi);
+ __func__, priv->bb_pre_ed_rssi);
if (!cr_201 && !cr_206)
return;
diff --git a/drivers/staging/vt6656/main_usb.c b/drivers/staging/vt6656/main_usb.c
index fc5fe4e..ac4fecb 100644
--- a/drivers/staging/vt6656/main_usb.c
+++ b/drivers/staging/vt6656/main_usb.c
@@ -238,7 +238,7 @@
priv->tx_antenna_mode = ANT_B;
priv->rx_antenna_sel = 1;
- if (priv->tx_rx_ant_inv == true)
+ if (priv->tx_rx_ant_inv)
priv->rx_antenna_mode = ANT_A;
else
priv->rx_antenna_mode = ANT_B;
@@ -248,14 +248,14 @@
if (antenna & EEP_ANTENNA_AUX) {
priv->tx_antenna_mode = ANT_A;
- if (priv->tx_rx_ant_inv == true)
+ if (priv->tx_rx_ant_inv)
priv->rx_antenna_mode = ANT_B;
else
priv->rx_antenna_mode = ANT_A;
} else {
priv->tx_antenna_mode = ANT_B;
- if (priv->tx_rx_ant_inv == true)
+ if (priv->tx_rx_ant_inv)
priv->rx_antenna_mode = ANT_A;
else
priv->rx_antenna_mode = ANT_B;
diff --git a/drivers/staging/vt6656/wcmd.c b/drivers/staging/vt6656/wcmd.c
index 4846a89..95faaeb 100644
--- a/drivers/staging/vt6656/wcmd.c
+++ b/drivers/staging/vt6656/wcmd.c
@@ -97,7 +97,7 @@
if (test_bit(DEVICE_FLAGS_DISCONNECTED, &priv->flags))
return;
- if (priv->cmd_running != true)
+ if (!priv->cmd_running)
return;
switch (priv->command_state) {
@@ -143,13 +143,13 @@
if (priv->rx_antenna_sel == 0) {
priv->rx_antenna_sel = 1;
- if (priv->tx_rx_ant_inv == true)
+ if (priv->tx_rx_ant_inv)
vnt_set_antenna_mode(priv, ANT_RXA);
else
vnt_set_antenna_mode(priv, ANT_RXB);
} else {
priv->rx_antenna_sel = 0;
- if (priv->tx_rx_ant_inv == true)
+ if (priv->tx_rx_ant_inv)
vnt_set_antenna_mode(priv, ANT_RXB);
else
vnt_set_antenna_mode(priv, ANT_RXA);
@@ -174,7 +174,7 @@
ADD_ONE_WITH_WRAP_AROUND(priv->cmd_enqueue_idx, CMD_Q_SIZE);
priv->free_cmd_queue--;
- if (priv->cmd_running == false)
+ if (!priv->cmd_running)
vnt_cmd_complete(priv);
return true;
diff --git a/drivers/staging/wilc1000/Kconfig b/drivers/staging/wilc1000/Kconfig
index dce9cee..73f7fef 100644
--- a/drivers/staging/wilc1000/Kconfig
+++ b/drivers/staging/wilc1000/Kconfig
@@ -1,6 +1,5 @@
config WILC1000
tristate
- select WIRELESS_EXT
---help---
This module only support IEEE 802.11n WiFi.
diff --git a/drivers/staging/wilc1000/host_interface.c b/drivers/staging/wilc1000/host_interface.c
index 0a922c7..9535842 100644
--- a/drivers/staging/wilc1000/host_interface.c
+++ b/drivers/staging/wilc1000/host_interface.c
@@ -2,6 +2,7 @@
#include <linux/time.h>
#include <linux/kthread.h>
#include <linux/delay.h>
+#include <linux/completion.h>
#include "host_interface.h"
#include "coreconfigurator.h"
#include "wilc_wlan.h"
@@ -230,10 +231,10 @@
static u8 P2P_LISTEN_STATE;
static struct task_struct *hif_thread_handler;
static struct message_queue hif_msg_q;
-static struct semaphore hif_sema_thread;
-static struct semaphore hif_sema_driver;
-static struct semaphore hif_sema_wait_response;
-static struct semaphore hif_sema_deinit;
+static struct completion hif_thread_comp;
+static struct completion hif_driver_comp;
+static struct completion hif_wait_response;
+static struct mutex hif_deinit_lock;
static struct timer_list periodic_rssi;
u8 wilc_multicast_mac_addr_list[WILC_MULTICAST_TABLE_SIZE][ETH_ALEN];
@@ -262,6 +263,7 @@
static void *host_int_ParseJoinBssParam(struct network_info *ptstrNetworkInfo);
static int host_int_get_ipaddress(struct wilc_vif *vif, u8 *ip_addr, u8 idx);
+static s32 Handle_ScanDone(struct wilc_vif *vif, enum scan_event enuEvent);
/* The u8IfIdx starts from 0 to NUM_CONCURRENT_IFC -1, but 0 index used as
* special purpose in wilc device, so we add 1 to the index to starts from 1.
@@ -305,10 +307,10 @@
netdev_err(vif->ndev, "Failed to set channel\n");
}
-static s32 handle_set_wfi_drv_handler(struct wilc_vif *vif,
- struct drv_handler *hif_drv_handler)
+static void handle_set_wfi_drv_handler(struct wilc_vif *vif,
+ struct drv_handler *hif_drv_handler)
{
- s32 result = 0;
+ int ret = 0;
struct wid wid;
wid.id = (u16)WID_SET_DRV_HANDLER;
@@ -316,24 +318,20 @@
wid.val = (s8 *)hif_drv_handler;
wid.size = sizeof(*hif_drv_handler);
- result = wilc_send_config_pkt(vif, SET_CFG, &wid, 1,
- hif_drv_handler->handler);
+ ret = wilc_send_config_pkt(vif, SET_CFG, &wid, 1,
+ hif_drv_handler->handler);
if (!hif_drv_handler->handler)
- up(&hif_sema_driver);
+ complete(&hif_driver_comp);
- if (result) {
+ if (ret)
netdev_err(vif->ndev, "Failed to set driver handler\n");
- return -EINVAL;
- }
-
- return result;
}
-static s32 handle_set_operation_mode(struct wilc_vif *vif,
- struct op_mode *hif_op_mode)
+static void handle_set_operation_mode(struct wilc_vif *vif,
+ struct op_mode *hif_op_mode)
{
- s32 result = 0;
+ int ret = 0;
struct wid wid;
wid.id = (u16)WID_SET_OPERATION_MODE;
@@ -341,23 +339,19 @@
wid.val = (s8 *)&hif_op_mode->mode;
wid.size = sizeof(u32);
- result = wilc_send_config_pkt(vif, SET_CFG, &wid, 1,
- wilc_get_vif_idx(vif));
+ ret = wilc_send_config_pkt(vif, SET_CFG, &wid, 1,
+ wilc_get_vif_idx(vif));
if ((hif_op_mode->mode) == IDLE_MODE)
- up(&hif_sema_driver);
+ complete(&hif_driver_comp);
- if (result) {
+ if (ret)
netdev_err(vif->ndev, "Failed to set driver handler\n");
- return -EINVAL;
- }
-
- return result;
}
-static s32 handle_set_ip_address(struct wilc_vif *vif, u8 *ip_addr, u8 idx)
+static void handle_set_ip_address(struct wilc_vif *vif, u8 *ip_addr, u8 idx)
{
- s32 result = 0;
+ int ret = 0;
struct wid wid;
char firmware_ip_addr[4] = {0};
@@ -371,22 +365,18 @@
wid.val = (u8 *)ip_addr;
wid.size = IP_ALEN;
- result = wilc_send_config_pkt(vif, SET_CFG, &wid, 1,
- wilc_get_vif_idx(vif));
+ ret = wilc_send_config_pkt(vif, SET_CFG, &wid, 1,
+ wilc_get_vif_idx(vif));
host_int_get_ipaddress(vif, firmware_ip_addr, idx);
- if (result) {
+ if (ret)
netdev_err(vif->ndev, "Failed to set IP address\n");
- return -EINVAL;
- }
-
- return result;
}
-static s32 handle_get_ip_address(struct wilc_vif *vif, u8 idx)
+static void handle_get_ip_address(struct wilc_vif *vif, u8 idx)
{
- s32 result = 0;
+ int ret = 0;
struct wid wid;
wid.id = (u16)WID_IP_ADDRESS;
@@ -394,8 +384,8 @@
wid.val = kmalloc(IP_ALEN, GFP_KERNEL);
wid.size = IP_ALEN;
- result = wilc_send_config_pkt(vif, GET_CFG, &wid, 1,
- wilc_get_vif_idx(vif));
+ ret = wilc_send_config_pkt(vif, GET_CFG, &wid, 1,
+ wilc_get_vif_idx(vif));
memcpy(get_ip[idx], wid.val, IP_ALEN);
@@ -404,18 +394,14 @@
if (memcmp(get_ip[idx], set_ip[idx], IP_ALEN) != 0)
wilc_setup_ipaddress(vif, set_ip[idx], idx);
- if (result != 0) {
+ if (ret)
netdev_err(vif->ndev, "Failed to get IP address\n");
- return -EINVAL;
- }
-
- return result;
}
-static s32 handle_get_mac_address(struct wilc_vif *vif,
- struct get_mac_addr *get_mac_addr)
+static void handle_get_mac_address(struct wilc_vif *vif,
+ struct get_mac_addr *get_mac_addr)
{
- s32 result = 0;
+ int ret = 0;
struct wid wid;
wid.id = (u16)WID_MAC_ADDR;
@@ -423,16 +409,12 @@
wid.val = get_mac_addr->mac_addr;
wid.size = ETH_ALEN;
- result = wilc_send_config_pkt(vif, GET_CFG, &wid, 1,
- wilc_get_vif_idx(vif));
+ ret = wilc_send_config_pkt(vif, GET_CFG, &wid, 1,
+ wilc_get_vif_idx(vif));
- if (result) {
+ if (ret)
netdev_err(vif->ndev, "Failed to get mac address\n");
- result = -EFAULT;
- }
- up(&hif_sema_wait_response);
-
- return result;
+ complete(&hif_wait_response);
}
static s32 handle_cfg_param(struct wilc_vif *vif,
@@ -455,7 +437,7 @@
} else {
netdev_err(vif->ndev, "check value 6 over\n");
result = -EINVAL;
- goto ERRORHANDLER;
+ goto unlock;
}
i++;
}
@@ -471,7 +453,7 @@
} else {
netdev_err(vif->ndev, "Impossible value\n");
result = -EINVAL;
- goto ERRORHANDLER;
+ goto unlock;
}
i++;
}
@@ -486,7 +468,7 @@
} else {
netdev_err(vif->ndev, "Range(1 ~ 65535) over\n");
result = -EINVAL;
- goto ERRORHANDLER;
+ goto unlock;
}
i++;
}
@@ -500,7 +482,7 @@
} else {
netdev_err(vif->ndev, "Invalid power mode\n");
result = -EINVAL;
- goto ERRORHANDLER;
+ goto unlock;
}
i++;
}
@@ -515,7 +497,7 @@
} else {
netdev_err(vif->ndev, "Range(1~256) over\n");
result = -EINVAL;
- goto ERRORHANDLER;
+ goto unlock;
}
i++;
}
@@ -530,7 +512,7 @@
} else {
netdev_err(vif->ndev, "Range(1~256) over\n");
result = -EINVAL;
- goto ERRORHANDLER;
+ goto unlock;
}
i++;
}
@@ -545,7 +527,7 @@
} else {
netdev_err(vif->ndev, "Threshold Range fail\n");
result = -EINVAL;
- goto ERRORHANDLER;
+ goto unlock;
}
i++;
}
@@ -560,7 +542,7 @@
} else {
netdev_err(vif->ndev, "Threshold Range fail\n");
result = -EINVAL;
- goto ERRORHANDLER;
+ goto unlock;
}
i++;
}
@@ -574,7 +556,7 @@
} else {
netdev_err(vif->ndev, "Preamle Range(0~2) over\n");
result = -EINVAL;
- goto ERRORHANDLER;
+ goto unlock;
}
i++;
}
@@ -588,7 +570,7 @@
} else {
netdev_err(vif->ndev, "Short slot(2) over\n");
result = -EINVAL;
- goto ERRORHANDLER;
+ goto unlock;
}
i++;
}
@@ -602,7 +584,7 @@
} else {
netdev_err(vif->ndev, "TXOP prot disable\n");
result = -EINVAL;
- goto ERRORHANDLER;
+ goto unlock;
}
i++;
}
@@ -617,7 +599,7 @@
} else {
netdev_err(vif->ndev, "Beacon interval(1~65535)fail\n");
result = -EINVAL;
- goto ERRORHANDLER;
+ goto unlock;
}
i++;
}
@@ -632,7 +614,7 @@
} else {
netdev_err(vif->ndev, "DTIM range(1~255) fail\n");
result = -EINVAL;
- goto ERRORHANDLER;
+ goto unlock;
}
i++;
}
@@ -646,7 +628,7 @@
} else {
netdev_err(vif->ndev, "Site survey disable\n");
result = -EINVAL;
- goto ERRORHANDLER;
+ goto unlock;
}
i++;
}
@@ -661,7 +643,7 @@
} else {
netdev_err(vif->ndev, "Site scan time(1~65535) over\n");
result = -EINVAL;
- goto ERRORHANDLER;
+ goto unlock;
}
i++;
}
@@ -676,7 +658,7 @@
} else {
netdev_err(vif->ndev, "Active time(1~65535) over\n");
result = -EINVAL;
- goto ERRORHANDLER;
+ goto unlock;
}
i++;
}
@@ -691,7 +673,7 @@
} else {
netdev_err(vif->ndev, "Passive time(1~65535) over\n");
result = -EINVAL;
- goto ERRORHANDLER;
+ goto unlock;
}
i++;
}
@@ -713,7 +695,7 @@
} else {
netdev_err(vif->ndev, "out of TX rate\n");
result = -EINVAL;
- goto ERRORHANDLER;
+ goto unlock;
}
i++;
}
@@ -724,28 +706,24 @@
if (result)
netdev_err(vif->ndev, "Error in setting CFG params\n");
-ERRORHANDLER:
+unlock:
mutex_unlock(&hif_drv->cfg_values_lock);
return result;
}
-static s32 Handle_ScanDone(struct wilc_vif *vif,
- enum scan_event enuEvent);
-
-static s32 Handle_Scan(struct wilc_vif *vif,
- struct scan_attr *pstrHostIFscanAttr)
+static s32 handle_scan(struct wilc_vif *vif, struct scan_attr *scan_info)
{
s32 result = 0;
- struct wid strWIDList[5];
- u32 u32WidsCount = 0;
+ struct wid wid_list[5];
+ u32 index = 0;
u32 i;
- u8 *pu8Buffer;
+ u8 *buffer;
u8 valuesize = 0;
u8 *pu8HdnNtwrksWidVal = NULL;
struct host_if_drv *hif_drv = vif->hif_drv;
- hif_drv->usr_scan_req.scan_result = pstrHostIFscanAttr->result;
- hif_drv->usr_scan_req.arg = pstrHostIFscanAttr->arg;
+ hif_drv->usr_scan_req.scan_result = scan_info->result;
+ hif_drv->usr_scan_req.arg = scan_info->arg;
if ((hif_drv->hif_state >= HOST_IF_SCANNING) &&
(hif_drv->hif_state < HOST_IF_CONNECTED)) {
@@ -762,72 +740,70 @@
hif_drv->usr_scan_req.rcvd_ch_cnt = 0;
- strWIDList[u32WidsCount].id = (u16)WID_SSID_PROBE_REQ;
- strWIDList[u32WidsCount].type = WID_STR;
+ wid_list[index].id = (u16)WID_SSID_PROBE_REQ;
+ wid_list[index].type = WID_STR;
- for (i = 0; i < pstrHostIFscanAttr->hidden_network.n_ssids; i++)
- valuesize += ((pstrHostIFscanAttr->hidden_network.net_info[i].ssid_len) + 1);
+ for (i = 0; i < scan_info->hidden_network.n_ssids; i++)
+ valuesize += ((scan_info->hidden_network.net_info[i].ssid_len) + 1);
pu8HdnNtwrksWidVal = kmalloc(valuesize + 1, GFP_KERNEL);
- strWIDList[u32WidsCount].val = pu8HdnNtwrksWidVal;
- if (strWIDList[u32WidsCount].val) {
- pu8Buffer = strWIDList[u32WidsCount].val;
+ wid_list[index].val = pu8HdnNtwrksWidVal;
+ if (wid_list[index].val) {
+ buffer = wid_list[index].val;
- *pu8Buffer++ = pstrHostIFscanAttr->hidden_network.n_ssids;
+ *buffer++ = scan_info->hidden_network.n_ssids;
- for (i = 0; i < pstrHostIFscanAttr->hidden_network.n_ssids; i++) {
- *pu8Buffer++ = pstrHostIFscanAttr->hidden_network.net_info[i].ssid_len;
- memcpy(pu8Buffer, pstrHostIFscanAttr->hidden_network.net_info[i].ssid, pstrHostIFscanAttr->hidden_network.net_info[i].ssid_len);
- pu8Buffer += pstrHostIFscanAttr->hidden_network.net_info[i].ssid_len;
+ for (i = 0; i < scan_info->hidden_network.n_ssids; i++) {
+ *buffer++ = scan_info->hidden_network.net_info[i].ssid_len;
+ memcpy(buffer, scan_info->hidden_network.net_info[i].ssid, scan_info->hidden_network.net_info[i].ssid_len);
+ buffer += scan_info->hidden_network.net_info[i].ssid_len;
}
- strWIDList[u32WidsCount].size = (s32)(valuesize + 1);
- u32WidsCount++;
+ wid_list[index].size = (s32)(valuesize + 1);
+ index++;
}
- {
- strWIDList[u32WidsCount].id = WID_INFO_ELEMENT_PROBE;
- strWIDList[u32WidsCount].type = WID_BIN_DATA;
- strWIDList[u32WidsCount].val = pstrHostIFscanAttr->ies;
- strWIDList[u32WidsCount].size = pstrHostIFscanAttr->ies_len;
- u32WidsCount++;
- }
+ wid_list[index].id = WID_INFO_ELEMENT_PROBE;
+ wid_list[index].type = WID_BIN_DATA;
+ wid_list[index].val = scan_info->ies;
+ wid_list[index].size = scan_info->ies_len;
+ index++;
- strWIDList[u32WidsCount].id = WID_SCAN_TYPE;
- strWIDList[u32WidsCount].type = WID_CHAR;
- strWIDList[u32WidsCount].size = sizeof(char);
- strWIDList[u32WidsCount].val = (s8 *)&pstrHostIFscanAttr->type;
- u32WidsCount++;
+ wid_list[index].id = WID_SCAN_TYPE;
+ wid_list[index].type = WID_CHAR;
+ wid_list[index].size = sizeof(char);
+ wid_list[index].val = (s8 *)&scan_info->type;
+ index++;
- strWIDList[u32WidsCount].id = WID_SCAN_CHANNEL_LIST;
- strWIDList[u32WidsCount].type = WID_BIN_DATA;
+ wid_list[index].id = WID_SCAN_CHANNEL_LIST;
+ wid_list[index].type = WID_BIN_DATA;
- if (pstrHostIFscanAttr->ch_freq_list &&
- pstrHostIFscanAttr->ch_list_len > 0) {
+ if (scan_info->ch_freq_list &&
+ scan_info->ch_list_len > 0) {
int i;
- for (i = 0; i < pstrHostIFscanAttr->ch_list_len; i++) {
- if (pstrHostIFscanAttr->ch_freq_list[i] > 0)
- pstrHostIFscanAttr->ch_freq_list[i] = pstrHostIFscanAttr->ch_freq_list[i] - 1;
+ for (i = 0; i < scan_info->ch_list_len; i++) {
+ if (scan_info->ch_freq_list[i] > 0)
+ scan_info->ch_freq_list[i] = scan_info->ch_freq_list[i] - 1;
}
}
- strWIDList[u32WidsCount].val = pstrHostIFscanAttr->ch_freq_list;
- strWIDList[u32WidsCount].size = pstrHostIFscanAttr->ch_list_len;
- u32WidsCount++;
+ wid_list[index].val = scan_info->ch_freq_list;
+ wid_list[index].size = scan_info->ch_list_len;
+ index++;
- strWIDList[u32WidsCount].id = WID_START_SCAN_REQ;
- strWIDList[u32WidsCount].type = WID_CHAR;
- strWIDList[u32WidsCount].size = sizeof(char);
- strWIDList[u32WidsCount].val = (s8 *)&pstrHostIFscanAttr->src;
- u32WidsCount++;
+ wid_list[index].id = WID_START_SCAN_REQ;
+ wid_list[index].type = WID_CHAR;
+ wid_list[index].size = sizeof(char);
+ wid_list[index].val = (s8 *)&scan_info->src;
+ index++;
if (hif_drv->hif_state == HOST_IF_CONNECTED)
scan_while_connected = true;
else if (hif_drv->hif_state == HOST_IF_IDLE)
scan_while_connected = false;
- result = wilc_send_config_pkt(vif, SET_CFG, strWIDList,
- u32WidsCount,
+ result = wilc_send_config_pkt(vif, SET_CFG, wid_list,
+ index,
wilc_get_vif_idx(vif));
if (result)
@@ -839,13 +815,13 @@
Handle_ScanDone(vif, SCAN_EVENT_ABORTED);
}
- kfree(pstrHostIFscanAttr->ch_freq_list);
- pstrHostIFscanAttr->ch_freq_list = NULL;
+ kfree(scan_info->ch_freq_list);
+ scan_info->ch_freq_list = NULL;
- kfree(pstrHostIFscanAttr->ies);
- pstrHostIFscanAttr->ies = NULL;
- kfree(pstrHostIFscanAttr->hidden_network.net_info);
- pstrHostIFscanAttr->hidden_network.net_info = NULL;
+ kfree(scan_info->ies);
+ scan_info->ies = NULL;
+ kfree(scan_info->hidden_network.net_info);
+ scan_info->hidden_network.net_info = NULL;
kfree(pu8HdnNtwrksWidVal);
@@ -1610,7 +1586,7 @@
&wid, 1,
wilc_get_vif_idx(vif));
}
- up(&hif_drv->sem_test_key_block);
+ complete(&hif_drv->comp_test_key_block);
break;
case WPA_RX_GTK:
@@ -1644,10 +1620,10 @@
wilc_get_vif_idx(vif));
kfree(pu8keybuf);
- up(&hif_drv->sem_test_key_block);
+ complete(&hif_drv->comp_test_key_block);
} else if (pstrHostIFkeyAttr->action & ADDKEY) {
pu8keybuf = kzalloc(RX_MIC_KEY_MSG_LEN, GFP_KERNEL);
- if (pu8keybuf == NULL) {
+ if (!pu8keybuf) {
ret = -ENOMEM;
goto _WPARxGtk_end_case_;
}
@@ -1673,7 +1649,7 @@
wilc_get_vif_idx(vif));
kfree(pu8keybuf);
- up(&hif_drv->sem_test_key_block);
+ complete(&hif_drv->comp_test_key_block);
}
_WPARxGtk_end_case_:
kfree(pstrHostIFkeyAttr->attr.wpa.key);
@@ -1711,7 +1687,7 @@
strWIDList, 2,
wilc_get_vif_idx(vif));
kfree(pu8keybuf);
- up(&hif_drv->sem_test_key_block);
+ complete(&hif_drv->comp_test_key_block);
} else if (pstrHostIFkeyAttr->action & ADDKEY) {
pu8keybuf = kmalloc(PTK_KEY_MSG_LEN, GFP_KERNEL);
if (!pu8keybuf) {
@@ -1734,7 +1710,7 @@
&wid, 1,
wilc_get_vif_idx(vif));
kfree(pu8keybuf);
- up(&hif_drv->sem_test_key_block);
+ complete(&hif_drv->comp_test_key_block);
}
_WPAPtk_end_case_:
@@ -1856,7 +1832,7 @@
}
}
- up(&hif_drv->sem_test_disconn_block);
+ complete(&hif_drv->comp_test_disconn_block);
}
void wilc_resolve_disconnect_aberration(struct wilc_vif *vif)
@@ -1885,7 +1861,7 @@
result = -EFAULT;
}
- up(&vif->hif_drv->sem_get_rssi);
+ complete(&vif->hif_drv->comp_get_rssi);
}
static s32 Handle_GetStatistics(struct wilc_vif *vif,
@@ -1938,7 +1914,7 @@
wilc_enable_tcp_ack_filter(false);
if (pstrStatistics != &vif->wilc->dummy_statistics)
- up(&hif_sema_wait_response);
+ complete(&hif_wait_response);
return 0;
}
@@ -1979,7 +1955,7 @@
return -EFAULT;
}
- up(&hif_drv->sem_inactive_time);
+ complete(&hif_drv->comp_inactive_time);
return result;
}
@@ -2172,7 +2148,7 @@
ERRORHANDLER:
kfree(wid.val);
- up(&hif_sema_wait_response);
+ complete(&hif_wait_response);
}
static void Handle_DelStation(struct wilc_vif *vif,
@@ -2472,7 +2448,7 @@
static void handle_get_tx_pwr(struct wilc_vif *vif, u8 *tx_pwr)
{
- s32 ret = 0;
+ int ret = 0;
struct wid wid;
wid.id = (u16)WID_TX_POWER;
@@ -2485,7 +2461,7 @@
if (ret)
netdev_err(vif->ndev, "Failed to get TX PWR\n");
- up(&hif_sema_wait_response);
+ complete(&hif_wait_response);
}
static int hostIFthread(void *pvArg)
@@ -2518,7 +2494,7 @@
switch (msg.id) {
case HOST_IF_MSG_SCAN:
- Handle_Scan(msg.vif, &msg.body.scan_info);
+ handle_scan(msg.vif, &msg.body.scan_info);
break;
case HOST_IF_MSG_CONNECT:
@@ -2667,7 +2643,7 @@
}
}
- up(&hif_sema_thread);
+ complete(&hif_thread_comp);
return 0;
}
@@ -2730,7 +2706,8 @@
result = wilc_mq_send(&hif_msg_q, &msg, sizeof(struct host_if_msg));
if (result)
netdev_err(vif->ndev, "Request to remove WEP key\n");
- down(&hif_drv->sem_test_key_block);
+ else
+ wait_for_completion(&hif_drv->comp_test_key_block);
return result;
}
@@ -2758,7 +2735,8 @@
result = wilc_mq_send(&hif_msg_q, &msg, sizeof(struct host_if_msg));
if (result)
netdev_err(vif->ndev, "Default key index\n");
- down(&hif_drv->sem_test_key_block);
+ else
+ wait_for_completion(&hif_drv->comp_test_key_block);
return result;
}
@@ -2791,7 +2769,7 @@
result = wilc_mq_send(&hif_msg_q, &msg, sizeof(struct host_if_msg));
if (result)
netdev_err(vif->ndev, "STA - WEP Key\n");
- down(&hif_drv->sem_test_key_block);
+ wait_for_completion(&hif_drv->comp_test_key_block);
return result;
}
@@ -2827,7 +2805,8 @@
if (result)
netdev_err(vif->ndev, "AP - WEP Key\n");
- down(&hif_drv->sem_test_key_block);
+ else
+ wait_for_completion(&hif_drv->comp_test_key_block);
return result;
}
@@ -2882,8 +2861,8 @@
if (result)
netdev_err(vif->ndev, "PTK Key\n");
-
- down(&hif_drv->sem_test_key_block);
+ else
+ wait_for_completion(&hif_drv->comp_test_key_block);
return result;
}
@@ -2950,8 +2929,8 @@
result = wilc_mq_send(&hif_msg_q, &msg, sizeof(struct host_if_msg));
if (result)
netdev_err(vif->ndev, "RX GTK\n");
-
- down(&hif_drv->sem_test_key_block);
+ else
+ wait_for_completion(&hif_drv->comp_test_key_block);
return result;
}
@@ -2961,14 +2940,8 @@
{
int result = 0;
struct host_if_msg msg;
- struct host_if_drv *hif_drv = vif->hif_drv;
int i;
- if (!hif_drv) {
- netdev_err(vif->ndev, "driver is null\n");
- return -EFAULT;
- }
-
memset(&msg, 0, sizeof(struct host_if_msg));
msg.id = HOST_IF_MSG_KEY;
@@ -3007,7 +2980,7 @@
return -EFAULT;
}
- down(&hif_sema_wait_response);
+ wait_for_completion(&hif_wait_response);
return result;
}
@@ -3097,8 +3070,8 @@
result = wilc_mq_send(&hif_msg_q, &msg, sizeof(struct host_if_msg));
if (result)
netdev_err(vif->ndev, "Failed to send message: disconnect\n");
-
- down(&hif_drv->sem_test_disconn_block);
+ else
+ wait_for_completion(&hif_drv->comp_test_disconn_block);
return result;
}
@@ -3110,12 +3083,6 @@
{
s32 result = 0;
struct wid wid;
- struct host_if_drv *hif_drv = vif->hif_drv;
-
- if (!hif_drv) {
- netdev_err(vif->ndev, "Driver is null\n");
- return -EFAULT;
- }
wid.id = (u16)WID_ASSOC_RES_INFO;
wid.type = WID_STR;
@@ -3138,12 +3105,6 @@
{
int result;
struct host_if_msg msg;
- struct host_if_drv *hif_drv = vif->hif_drv;
-
- if (!hif_drv) {
- netdev_err(vif->ndev, "driver is null\n");
- return -EFAULT;
- }
memset(&msg, 0, sizeof(struct host_if_msg));
msg.id = HOST_IF_MSG_SET_CHANNEL;
@@ -3219,8 +3180,8 @@
result = wilc_mq_send(&hif_msg_q, &msg, sizeof(struct host_if_msg));
if (result)
netdev_err(vif->ndev, "Failed to send get host ch param\n");
-
- down(&hif_drv->sem_inactive_time);
+ else
+ wait_for_completion(&hif_drv->comp_inactive_time);
*pu32InactiveTime = inactive_time;
@@ -3243,7 +3204,7 @@
return -EFAULT;
}
- down(&hif_drv->sem_get_rssi);
+ wait_for_completion(&hif_drv->comp_get_rssi);
if (!rssi_level) {
netdev_err(vif->ndev, "RSS pointer value is null\n");
@@ -3272,7 +3233,7 @@
}
if (stats != &vif->wilc->dummy_statistics)
- down(&hif_sema_wait_response);
+ wait_for_completion(&hif_wait_response);
return result;
}
@@ -3382,7 +3343,7 @@
scan_while_connected = false;
- sema_init(&hif_sema_wait_response, 0);
+ init_completion(&hif_wait_response);
hif_drv = kzalloc(sizeof(struct host_if_drv), GFP_KERNEL);
if (!hif_drv) {
@@ -3399,15 +3360,15 @@
wilc_optaining_ip = false;
if (clients_count == 0) {
- sema_init(&hif_sema_thread, 0);
- sema_init(&hif_sema_driver, 0);
- sema_init(&hif_sema_deinit, 1);
+ init_completion(&hif_thread_comp);
+ init_completion(&hif_driver_comp);
+ mutex_init(&hif_deinit_lock);
}
- sema_init(&hif_drv->sem_test_key_block, 0);
- sema_init(&hif_drv->sem_test_disconn_block, 0);
- sema_init(&hif_drv->sem_get_rssi, 0);
- sema_init(&hif_drv->sem_inactive_time, 0);
+ init_completion(&hif_drv->comp_test_key_block);
+ init_completion(&hif_drv->comp_test_disconn_block);
+ init_completion(&hif_drv->comp_get_rssi);
+ init_completion(&hif_drv->comp_inactive_time);
if (clients_count == 0) {
result = wilc_mq_create(&hif_msg_q);
@@ -3469,7 +3430,7 @@
return -EFAULT;
}
- down(&hif_sema_deinit);
+ mutex_lock(&hif_deinit_lock);
terminated_handle = hif_drv;
@@ -3479,7 +3440,7 @@
del_timer_sync(&hif_drv->remain_on_ch_timer);
wilc_set_wfi_drv_handler(vif, 0, 0);
- down(&hif_sema_driver);
+ wait_for_completion(&hif_driver_comp);
if (hif_drv->usr_scan_req.scan_result) {
hif_drv->usr_scan_req.scan_result(SCAN_EVENT_ABORTED, NULL,
@@ -3494,15 +3455,14 @@
memset(&msg, 0, sizeof(struct host_if_msg));
if (clients_count == 1) {
- del_timer_sync(&periodic_rssi);
msg.id = HOST_IF_MSG_EXIT;
msg.vif = vif;
result = wilc_mq_send(&hif_msg_q, &msg, sizeof(struct host_if_msg));
if (result != 0)
netdev_err(vif->ndev, "deinit : Error(%d)\n", result);
-
- down(&hif_sema_thread);
+ else
+ wait_for_completion(&hif_thread_comp);
wilc_mq_destroy(&hif_msg_q);
}
@@ -3511,7 +3471,7 @@
clients_count--;
terminated_handle = NULL;
- up(&hif_sema_deinit);
+ mutex_unlock(&hif_deinit_lock);
return result;
}
@@ -3558,25 +3518,25 @@
struct host_if_drv *hif_drv = NULL;
struct wilc_vif *vif;
- down(&hif_sema_deinit);
+ mutex_lock(&hif_deinit_lock);
id = ((pu8Buffer[u32Length - 4]) | (pu8Buffer[u32Length - 3] << 8) | (pu8Buffer[u32Length - 2] << 16) | (pu8Buffer[u32Length - 1] << 24));
vif = wilc_get_vif_from_idx(wilc, id);
if (!vif) {
- up(&hif_sema_deinit);
+ mutex_unlock(&hif_deinit_lock);
return;
}
hif_drv = vif->hif_drv;
if (!hif_drv || hif_drv == terminated_handle) {
- up(&hif_sema_deinit);
+ mutex_unlock(&hif_deinit_lock);
return;
}
if (!hif_drv->usr_conn_req.conn_result) {
netdev_err(vif->ndev, "there is no current Connect Request\n");
- up(&hif_sema_deinit);
+ mutex_unlock(&hif_deinit_lock);
return;
}
@@ -3593,7 +3553,7 @@
if (result)
netdev_err(vif->ndev, "synchronous info (%d)\n", result);
- up(&hif_sema_deinit);
+ mutex_unlock(&hif_deinit_lock);
}
void wilc_scan_complete_received(struct wilc *wilc, u8 *pu8Buffer,
@@ -3634,12 +3594,6 @@
{
int result = 0;
struct host_if_msg msg;
- struct host_if_drv *hif_drv = vif->hif_drv;
-
- if (!hif_drv) {
- netdev_err(vif->ndev, "driver is null\n");
- return -EFAULT;
- }
memset(&msg, 0, sizeof(struct host_if_msg));
@@ -3688,12 +3642,6 @@
{
int result = 0;
struct host_if_msg msg;
- struct host_if_drv *hif_drv = vif->hif_drv;
-
- if (!hif_drv) {
- netdev_err(vif->ndev, "driver is null\n");
- return -EFAULT;
- }
memset(&msg, 0, sizeof(struct host_if_msg));
@@ -3727,12 +3675,6 @@
int result = 0;
struct host_if_msg msg;
struct beacon_attr *beacon_info = &msg.body.beacon_info;
- struct host_if_drv *hif_drv = vif->hif_drv;
-
- if (!hif_drv) {
- netdev_err(vif->ndev, "driver is null\n");
- return -EFAULT;
- }
memset(&msg, 0, sizeof(struct host_if_msg));
@@ -3776,12 +3718,6 @@
{
int result = 0;
struct host_if_msg msg;
- struct host_if_drv *hif_drv = vif->hif_drv;
-
- if (!hif_drv) {
- netdev_err(vif->ndev, "driver is null\n");
- return -EFAULT;
- }
msg.id = HOST_IF_MSG_DEL_BEACON;
msg.vif = vif;
@@ -3798,12 +3734,6 @@
int result = 0;
struct host_if_msg msg;
struct add_sta_param *add_sta_info = &msg.body.add_sta_info;
- struct host_if_drv *hif_drv = vif->hif_drv;
-
- if (!hif_drv) {
- netdev_err(vif->ndev, "driver is null\n");
- return -EFAULT;
- }
memset(&msg, 0, sizeof(struct host_if_msg));
@@ -3830,12 +3760,6 @@
int result = 0;
struct host_if_msg msg;
struct del_sta *del_sta_info = &msg.body.del_sta_info;
- struct host_if_drv *hif_drv = vif->hif_drv;
-
- if (!hif_drv) {
- netdev_err(vif->ndev, "driver is null\n");
- return -EFAULT;
- }
memset(&msg, 0, sizeof(struct host_if_msg));
@@ -3858,16 +3782,10 @@
int result = 0;
struct host_if_msg msg;
struct del_all_sta *del_all_sta_info = &msg.body.del_all_sta_info;
- struct host_if_drv *hif_drv = vif->hif_drv;
u8 zero_addr[ETH_ALEN] = {0};
int i;
u8 assoc_sta = 0;
- if (!hif_drv) {
- netdev_err(vif->ndev, "driver is null\n");
- return -EFAULT;
- }
-
memset(&msg, 0, sizeof(struct host_if_msg));
msg.id = HOST_IF_MSG_DEL_ALL_STA;
@@ -3887,8 +3805,8 @@
if (result)
netdev_err(vif->ndev, "wilc_mq_send fail\n");
-
- down(&hif_sema_wait_response);
+ else
+ wait_for_completion(&hif_wait_response);
return result;
}
@@ -3899,12 +3817,6 @@
int result = 0;
struct host_if_msg msg;
struct add_sta_param *add_sta_info = &msg.body.add_sta_info;
- struct host_if_drv *hif_drv = vif->hif_drv;
-
- if (!hif_drv) {
- netdev_err(vif->ndev, "driver is null\n");
- return -EFAULT;
- }
memset(&msg, 0, sizeof(struct host_if_msg));
@@ -3932,12 +3844,6 @@
int result = 0;
struct host_if_msg msg;
struct power_mgmt_param *pwr_mgmt_info = &msg.body.pwr_mgmt_info;
- struct host_if_drv *hif_drv = vif->hif_drv;
-
- if (!hif_drv) {
- netdev_err(vif->ndev, "driver is null\n");
- return -EFAULT;
- }
if (wilc_wlan_get_num_conn_ifcs(vif->wilc) == 2 && enabled)
return 0;
@@ -3962,12 +3868,6 @@
int result = 0;
struct host_if_msg msg;
struct set_multicast *multicast_filter_param = &msg.body.multicast_info;
- struct host_if_drv *hif_drv = vif->hif_drv;
-
- if (!hif_drv) {
- netdev_err(vif->ndev, "driver is null\n");
- return -EFAULT;
- }
memset(&msg, 0, sizeof(struct host_if_msg));
@@ -4141,12 +4041,6 @@
{
int result = 0;
struct host_if_msg msg;
- struct host_if_drv *hif_drv = vif->hif_drv;
-
- if (!hif_drv) {
- netdev_err(vif->ndev, "driver is null\n");
- return -EFAULT;
- }
memset(&msg, 0, sizeof(struct host_if_msg));
@@ -4167,12 +4061,6 @@
{
int result = 0;
struct host_if_msg msg;
- struct host_if_drv *hif_drv = vif->hif_drv;
-
- if (!hif_drv) {
- netdev_err(vif->ndev, "driver is null\n");
- return -EFAULT;
- }
memset(&msg, 0, sizeof(struct host_if_msg));
@@ -4221,7 +4109,7 @@
if (ret)
netdev_err(vif->ndev, "Failed to get TX PWR\n");
- down(&hif_sema_wait_response);
+ wait_for_completion(&hif_wait_response);
*tx_power = msg.body.tx_power.tx_pwr;
return ret;
diff --git a/drivers/staging/wilc1000/host_interface.h b/drivers/staging/wilc1000/host_interface.h
index 01f3222..8d2dd0d 100644
--- a/drivers/staging/wilc1000/host_interface.h
+++ b/drivers/staging/wilc1000/host_interface.h
@@ -275,10 +275,10 @@
struct cfg_param_attr cfg_values;
struct mutex cfg_values_lock;
- struct semaphore sem_test_key_block;
- struct semaphore sem_test_disconn_block;
- struct semaphore sem_get_rssi;
- struct semaphore sem_inactive_time;
+ struct completion comp_test_key_block;
+ struct completion comp_test_disconn_block;
+ struct completion comp_get_rssi;
+ struct completion comp_inactive_time;
struct timer_list scan_timer;
struct timer_list connect_timer;
diff --git a/drivers/staging/wilc1000/linux_mon.c b/drivers/staging/wilc1000/linux_mon.c
index 7d9e5de..242f82f 100644
--- a/drivers/staging/wilc1000/linux_mon.c
+++ b/drivers/staging/wilc1000/linux_mon.c
@@ -24,7 +24,7 @@
static struct net_device *wilc_wfi_mon; /* global monitor netdev */
-static u8 srcAdd[6];
+static u8 srcadd[6];
static u8 bssid[6];
static u8 broadcast[] = {0xff, 0xff, 0xff, 0xff, 0xff, 0xff};
/**
@@ -59,9 +59,10 @@
/* Get WILC header */
memcpy(&header, (buff - HOST_HDR_OFFSET), HOST_HDR_OFFSET);
-
- /* The packet offset field conain info about what type of managment frame */
- /* we are dealing with and ack status */
+ /*
+ * The packet offset field contain info about what type of management
+ * the frame we are dealing with and ack status
+ */
pkt_offset = GET_PKT_OFFSET(header);
if (pkt_offset & IS_MANAGMEMENT_CALLBACK) {
@@ -105,7 +106,7 @@
hdr->hdr.it_version = 0; /* PKTHDR_RADIOTAP_VERSION; */
hdr->hdr.it_len = cpu_to_le16(sizeof(struct wilc_wfi_radiotap_hdr));
hdr->hdr.it_present = cpu_to_le32
- (1 << IEEE80211_RADIOTAP_RATE); /* | */
+ (1 << IEEE80211_RADIOTAP_RATE); /* | */
hdr->rate = 5; /* txrate->bitrate / 5; */
}
@@ -127,8 +128,10 @@
static void mgmt_tx_complete(void *priv, int status)
{
struct tx_complete_mon_data *pv_data = priv;
-
- /* incase of fully hosting mode, the freeing will be done in response to the cfg packet */
+ /*
+ * in case of fully hosting mode, the freeing will be done
+ * in response to the cfg packet
+ */
kfree(pv_data->buff);
kfree(pv_data);
@@ -225,11 +228,11 @@
skb->dev = mon_priv->real_ndev;
/* Identify if Ethernet or MAC header (data or mgmt) */
- memcpy(srcAdd, &skb->data[10], 6);
+ memcpy(srcadd, &skb->data[10], 6);
memcpy(bssid, &skb->data[16], 6);
/* if source address and bssid fields are equal>>Mac header */
/*send it to mgmt frames handler */
- if (!(memcmp(srcAdd, bssid, 6))) {
+ if (!(memcmp(srcadd, bssid, 6))) {
ret = mon_mgmt_tx(mon_priv->real_ndev, skb->data, skb->len);
if (ret)
netdev_err(dev, "fail to mgmt tx\n");
@@ -255,7 +258,8 @@
* @date 12 JUL 2012
* @version 1.0
*/
-struct net_device *WILC_WFI_init_mon_interface(const char *name, struct net_device *real_dev)
+struct net_device *WILC_WFI_init_mon_interface(const char *name,
+ struct net_device *real_dev)
{
u32 ret = 0;
struct WILC_WFI_mon_priv *priv;
diff --git a/drivers/staging/wilc1000/linux_wlan.c b/drivers/staging/wilc1000/linux_wlan.c
index bfa754b..4f93c11e 100644
--- a/drivers/staging/wilc1000/linux_wlan.c
+++ b/drivers/staging/wilc1000/linux_wlan.c
@@ -22,6 +22,7 @@
#include <linux/skbuff.h>
#include <linux/semaphore.h>
+#include <linux/completion.h>
static int dev_state_ev_handler(struct notifier_block *this,
unsigned long event, void *ptr);
@@ -30,8 +31,6 @@
.notifier_call = dev_state_ev_handler
};
-#define IRQ_WAIT 1
-#define IRQ_NO_WAIT 0
static struct semaphore close_exit_sync;
static int wlan_deinit_locks(struct net_device *dev);
@@ -259,10 +258,12 @@
for (i = 0; i < wilc->vif_num; i++) {
if (wilc->vif[i]->mode == STATION_MODE)
- if (!memcmp(bssid, wilc->vif[i]->bssid, ETH_ALEN))
+ if (ether_addr_equal_unaligned(bssid,
+ wilc->vif[i]->bssid))
return wilc->vif[i]->ndev;
if (wilc->vif[i]->mode == AP_MODE)
- if (!memcmp(bssid1, wilc->vif[i]->bssid, ETH_ALEN))
+ if (ether_addr_equal_unaligned(bssid1,
+ wilc->vif[i]->bssid))
return wilc->vif[i]->ndev;
}
@@ -303,40 +304,27 @@
return ret_val;
}
-#define USE_TX_BACKOFF_DELAY_IF_NO_BUFFERS
-
static int linux_wlan_txq_task(void *vp)
{
int ret, txq_count;
struct wilc_vif *vif;
struct wilc *wl;
struct net_device *dev = vp;
-#if defined USE_TX_BACKOFF_DELAY_IF_NO_BUFFERS
-#define TX_BACKOFF_WEIGHT_INCR_STEP (1)
-#define TX_BACKOFF_WEIGHT_DECR_STEP (1)
-#define TX_BACKOFF_WEIGHT_MAX (7)
-#define TX_BACKOFF_WEIGHT_MIN (0)
-#define TX_BACKOFF_WEIGHT_UNIT_MS (10)
- int backoff_weight = TX_BACKOFF_WEIGHT_MIN;
-#endif
vif = netdev_priv(dev);
wl = vif->wilc;
- up(&wl->txq_thread_started);
+ complete(&wl->txq_thread_started);
while (1) {
down(&wl->txq_event);
if (wl->close) {
- up(&wl->txq_thread_started);
+ complete(&wl->txq_thread_started);
while (!kthread_should_stop())
schedule();
break;
}
-#if !defined USE_TX_BACKOFF_DELAY_IF_NO_BUFFERS
- ret = wilc_wlan_handle_txq(dev, &txq_count);
-#else
do {
ret = wilc_wlan_handle_txq(dev, &txq_count);
if (txq_count < FLOW_CONTROL_LOWER_THRESHOLD) {
@@ -345,20 +333,7 @@
if (netif_queue_stopped(wl->vif[1]->ndev))
netif_wake_queue(wl->vif[1]->ndev);
}
-
- if (ret == WILC_TX_ERR_NO_BUF) {
- backoff_weight += TX_BACKOFF_WEIGHT_INCR_STEP;
- if (backoff_weight > TX_BACKOFF_WEIGHT_MAX)
- backoff_weight = TX_BACKOFF_WEIGHT_MAX;
- } else {
- if (backoff_weight > TX_BACKOFF_WEIGHT_MIN) {
- backoff_weight -= TX_BACKOFF_WEIGHT_DECR_STEP;
- if (backoff_weight < TX_BACKOFF_WEIGHT_MIN)
- backoff_weight = TX_BACKOFF_WEIGHT_MIN;
- }
- }
} while (ret == WILC_TX_ERR_NO_BUF && !wl->close);
-#endif
}
return 0;
}
@@ -449,7 +424,6 @@
struct wilc_vif *vif)
{
unsigned char c_val[64];
- unsigned char mac_add[] = {0x00, 0x80, 0xC2, 0x5E, 0xa2, 0xff};
struct wilc *wilc = vif->wilc;
struct wilc_priv *priv;
struct host_if_drv *hif_drv;
@@ -458,9 +432,6 @@
priv = wiphy_priv(dev->ieee80211_ptr->wiphy);
hif_drv = (struct host_if_drv *)priv->hif_drv;
netdev_dbg(dev, "Host = %p\n", hif_drv);
- wilc_get_mac_address(vif, mac_add);
-
- netdev_dbg(dev, "MAC address is : %pM\n", mac_add);
wilc_get_chipid(wilc, false);
*(int *)c_val = 1;
@@ -622,11 +593,6 @@
0))
goto _fail_;
- memcpy(c_val, mac_add, 6);
-
- if (!wilc_wlan_cfg_set(vif, 0, WID_MAC_ADDR, c_val, 6, 0, 0))
- goto _fail_;
-
c_val[0] = DETECT_PROTECT_REPORT;
if (!wilc_wlan_cfg_set(vif, 0, WID_11N_OBSS_NONHT_DETECTION, c_val, 1,
0, 0))
@@ -691,14 +657,6 @@
wilc_wlan_stop(wl);
wilc_wlan_cleanup(dev);
-#if defined(PLAT_ALLWINNER_A20) || defined(PLAT_ALLWINNER_A23) || defined(PLAT_ALLWINNER_A31)
- if (!wl->dev_irq_num &&
- wl->hif_func->disable_interrupt) {
- mutex_lock(&wl->hif_cs);
- wl->hif_func->disable_interrupt(wl);
- mutex_unlock(&wl->hif_cs);
- }
-#endif
wlan_deinit_locks(dev);
wl->initialized = false;
@@ -727,8 +685,7 @@
sema_init(&wl->cfg_event, 0);
sema_init(&wl->sync_event, 0);
-
- sema_init(&wl->txq_thread_started, 0);
+ init_completion(&wl->txq_thread_started);
return 0;
}
@@ -765,7 +722,7 @@
wilc->close = 0;
return -ENOBUFS;
}
- down(&wilc->txq_thread_started);
+ wait_for_completion(&wilc->txq_thread_started);
return 0;
}
@@ -896,25 +853,20 @@
int wilc_mac_open(struct net_device *ndev)
{
struct wilc_vif *vif;
- struct wilc *wilc;
unsigned char mac_add[ETH_ALEN] = {0};
int ret = 0;
int i = 0;
- struct wilc_priv *priv;
struct wilc *wl;
vif = netdev_priv(ndev);
wl = vif->wilc;
if (!wl || !wl->dev) {
- netdev_err(ndev, "wilc1000: SPI device not ready\n");
+ netdev_err(ndev, "device not ready\n");
return -ENODEV;
}
- vif = netdev_priv(ndev);
- wilc = vif->wilc;
- priv = wiphy_priv(vif->ndev->ieee80211_ptr->wiphy);
netdev_dbg(ndev, "MAC OPEN[%p]\n", ndev);
ret = wilc_init_host_int(ndev);
@@ -933,13 +885,13 @@
wilc_set_wfi_drv_handler(vif,
wilc_get_vif_idx(vif),
0);
- } else if (!wilc_wlan_get_num_conn_ifcs(wilc)) {
+ } else if (!wilc_wlan_get_num_conn_ifcs(wl)) {
wilc_set_wfi_drv_handler(vif,
wilc_get_vif_idx(vif),
- wilc->open_ifcs);
+ wl->open_ifcs);
} else {
- if (memcmp(wilc->vif[i ^ 1]->bssid,
- wilc->vif[i ^ 1]->src_addr, 6))
+ if (memcmp(wl->vif[i ^ 1]->bssid,
+ wl->vif[i ^ 1]->src_addr, 6))
wilc_set_wfi_drv_handler(vif,
wilc_get_vif_idx(vif),
0);
@@ -969,12 +921,12 @@
wilc_mgmt_frame_register(vif->ndev->ieee80211_ptr->wiphy,
vif->ndev->ieee80211_ptr,
- vif->g_struct_frame_reg[0].frame_type,
- vif->g_struct_frame_reg[0].reg);
+ vif->frame_reg[0].type,
+ vif->frame_reg[0].reg);
wilc_mgmt_frame_register(vif->ndev->ieee80211_ptr->wiphy,
vif->ndev->ieee80211_ptr,
- vif->g_struct_frame_reg[1].frame_type,
- vif->g_struct_frame_reg[1].reg);
+ vif->frame_reg[1].type,
+ vif->frame_reg[1].reg);
netif_wake_queue(ndev);
wl->open_ifcs++;
vif->mac_opened = 1;
@@ -991,14 +943,10 @@
static void wilc_set_multicast_list(struct net_device *dev)
{
struct netdev_hw_addr *ha;
- struct wilc_priv *priv;
- struct host_if_drv *hif_drv;
struct wilc_vif *vif;
int i = 0;
- priv = wiphy_priv(dev->ieee80211_ptr->wiphy);
vif = netdev_priv(dev);
- hif_drv = (struct host_if_drv *)priv->hif_drv;
if (dev->flags & IFF_PROMISC)
return;
@@ -1152,7 +1100,6 @@
s8 rssi;
u32 size = 0, length = 0;
struct wilc_vif *vif;
- struct wilc_priv *priv;
s32 ret = 0;
struct wilc *wilc;
@@ -1176,7 +1123,6 @@
return PTR_ERR(buff);
if (strncasecmp(buff, "RSSI", length) == 0) {
- priv = wiphy_priv(vif->ndev->ieee80211_ptr->wiphy);
ret = wilc_get_rssi(vif, &rssi);
netdev_info(ndev, "RSSI :%d\n", rssi);
@@ -1263,8 +1209,8 @@
}
vif = netdev_priv(wilc->vif[1]->ndev);
- if ((buff[0] == vif->g_struct_frame_reg[0].frame_type && vif->g_struct_frame_reg[0].reg) ||
- (buff[0] == vif->g_struct_frame_reg[1].frame_type && vif->g_struct_frame_reg[1].reg))
+ if ((buff[0] == vif->frame_reg[0].type && vif->frame_reg[0].reg) ||
+ (buff[0] == vif->frame_reg[1].type && vif->frame_reg[1].reg))
WILC_WFI_p2p_rx(wilc->vif[1]->ndev, buff, size);
}
@@ -1280,8 +1226,10 @@
vif[i] = netdev_priv(wilc->vif[i]->ndev);
}
- if (wilc && wilc->firmware)
+ if (wilc && wilc->firmware) {
release_firmware(wilc->firmware);
+ wilc->firmware = NULL;
+ }
if (wilc && (wilc->vif[0]->ndev || wilc->vif[1]->ndev)) {
wilc_lock_timeout(wilc, &close_exit_sync, 5 * 1000);
diff --git a/drivers/staging/wilc1000/wilc_spi.c b/drivers/staging/wilc1000/wilc_spi.c
index d41b8b6..4268e2f 100644
--- a/drivers/staging/wilc1000/wilc_spi.c
+++ b/drivers/staging/wilc1000/wilc_spi.c
@@ -196,9 +196,6 @@
dev_err(&spi->dev,
"can't write data with the following length: %d\n",
len);
- dev_err(&spi->dev,
- "FAILED due to NULL buffer or ZERO length check the following length: %d\n",
- len);
ret = -EINVAL;
}
diff --git a/drivers/staging/wilc1000/wilc_wfi_cfgoperations.c b/drivers/staging/wilc1000/wilc_wfi_cfgoperations.c
index 544917d..51aff4f 100644
--- a/drivers/staging/wilc1000/wilc_wfi_cfgoperations.c
+++ b/drivers/staging/wilc1000/wilc_wfi_cfgoperations.c
@@ -451,7 +451,7 @@
} else if (scan_event == SCAN_EVENT_DONE) {
refresh_scan(priv, 1, false);
- down(&(priv->hSemScanReq));
+ mutex_lock(&priv->scan_req_lock);
if (priv->pstrScanReq) {
cfg80211_scan_done(priv->pstrScanReq, false);
@@ -459,9 +459,9 @@
priv->bCfgScanning = false;
priv->pstrScanReq = NULL;
}
- up(&(priv->hSemScanReq));
+ mutex_unlock(&priv->scan_req_lock);
} else if (scan_event == SCAN_EVENT_ABORTED) {
- down(&(priv->hSemScanReq));
+ mutex_lock(&priv->scan_req_lock);
if (priv->pstrScanReq) {
update_scan_time();
@@ -471,7 +471,7 @@
priv->bCfgScanning = false;
priv->pstrScanReq = NULL;
}
- up(&(priv->hSemScanReq));
+ mutex_unlock(&priv->scan_req_lock);
}
}
}
@@ -558,11 +558,11 @@
if (!pstrWFIDrv->p2p_connect)
wlan_channel = INVALID_CHANNEL;
- if ((pstrWFIDrv->IFC_UP) && (dev == wl->vif[1]->ndev)) {
+ if ((pstrWFIDrv->IFC_UP) && (dev == wl->vif[1]->ndev))
pstrDisconnectNotifInfo->reason = 3;
- } else if ((!pstrWFIDrv->IFC_UP) && (dev == wl->vif[1]->ndev)) {
+ else if ((!pstrWFIDrv->IFC_UP) && (dev == wl->vif[1]->ndev))
pstrDisconnectNotifInfo->reason = 1;
- }
+
cfg80211_disconnected(dev, pstrDisconnectNotifInfo->reason, pstrDisconnectNotifInfo->ie,
pstrDisconnectNotifInfo->ie_len, false,
GFP_KERNEL);
@@ -739,18 +739,15 @@
wilc_add_wep_key_bss_sta(vif, sme->key, sme->key_len,
sme->key_idx);
} else if (sme->crypto.wpa_versions & NL80211_WPA_VERSION_2) {
- if (sme->crypto.cipher_group == WLAN_CIPHER_SUITE_TKIP) {
+ if (sme->crypto.cipher_group == WLAN_CIPHER_SUITE_TKIP)
u8security = ENCRYPT_ENABLED | WPA2 | TKIP;
- } else {
+ else
u8security = ENCRYPT_ENABLED | WPA2 | AES;
- }
} else if (sme->crypto.wpa_versions & NL80211_WPA_VERSION_1) {
- if (sme->crypto.cipher_group == WLAN_CIPHER_SUITE_TKIP) {
+ if (sme->crypto.cipher_group == WLAN_CIPHER_SUITE_TKIP)
u8security = ENCRYPT_ENABLED | WPA | TKIP;
- } else {
+ else
u8security = ENCRYPT_ENABLED | WPA | AES;
- }
-
} else {
s32Error = -ENOTSUPP;
netdev_err(dev, "Not supported cipher\n");
@@ -762,11 +759,10 @@
if ((sme->crypto.wpa_versions & NL80211_WPA_VERSION_1)
|| (sme->crypto.wpa_versions & NL80211_WPA_VERSION_2)) {
for (i = 0; i < sme->crypto.n_ciphers_pairwise; i++) {
- if (sme->crypto.ciphers_pairwise[i] == WLAN_CIPHER_SUITE_TKIP) {
+ if (sme->crypto.ciphers_pairwise[i] == WLAN_CIPHER_SUITE_TKIP)
u8security = u8security | TKIP;
- } else {
+ else
u8security = u8security | AES;
- }
}
}
@@ -823,11 +819,22 @@
struct wilc_priv *priv;
struct host_if_drv *pstrWFIDrv;
struct wilc_vif *vif;
+ struct wilc *wilc;
u8 NullBssid[ETH_ALEN] = {0};
wilc_connecting = 0;
priv = wiphy_priv(wiphy);
vif = netdev_priv(priv->dev);
+ wilc = vif->wilc;
+
+ if (!wilc)
+ return -EIO;
+
+ if (wilc->close) {
+ /* already disconnected done */
+ cfg80211_disconnected(dev, 0, NULL, 0, true, GFP_KERNEL);
+ return 0;
+ }
pstrWFIDrv = (struct host_if_drv *)priv->hif_drv;
if (!pstrWFIDrv->p2p_connect)
@@ -1115,9 +1122,12 @@
}
if (key_index >= 0 && key_index <= 3) {
- memset(priv->WILC_WFI_wep_key[key_index], 0, priv->WILC_WFI_wep_key_len[key_index]);
- priv->WILC_WFI_wep_key_len[key_index] = 0;
- wilc_remove_wep_key(vif, key_index);
+ if (priv->WILC_WFI_wep_key_len[key_index]) {
+ memset(priv->WILC_WFI_wep_key[key_index], 0,
+ priv->WILC_WFI_wep_key_len[key_index]);
+ priv->WILC_WFI_wep_key_len[key_index] = 0;
+ wilc_remove_wep_key(vif, key_index);
+ }
} else {
wilc_remove_key(priv->hif_drv, mac_addr);
}
@@ -1355,9 +1365,8 @@
u8 channel_list_attr_index = 0;
while (index < len) {
- if (buf[index] == GO_INTENT_ATTR_ID) {
+ if (buf[index] == GO_INTENT_ATTR_ID)
buf[index + 3] = (buf[index + 3] & 0x01) | (0x00 << 1);
- }
if (buf[index] == CHANLIST_ATTR_ID)
channel_list_attr_index = index;
@@ -1369,9 +1378,8 @@
if (channel_list_attr_index) {
for (i = channel_list_attr_index + 3; i < ((channel_list_attr_index + 3) + buf[channel_list_attr_index + 1]); i++) {
if (buf[i] == 0x51) {
- for (j = i + 2; j < ((i + 2) + buf[i + 1]); j++) {
+ for (j = i + 2; j < ((i + 2) + buf[i + 1]); j++)
buf[j] = wlan_channel;
- }
break;
}
}
@@ -1409,9 +1417,8 @@
if (channel_list_attr_index) {
for (i = channel_list_attr_index + 3; i < ((channel_list_attr_index + 3) + buf[channel_list_attr_index + 1]); i++) {
if (buf[i] == 0x51) {
- for (j = i + 2; j < ((i + 2) + buf[i + 1]); j++) {
+ for (j = i + 2; j < ((i + 2) + buf[i + 1]); j++)
buf[j] = wlan_channel;
- }
break;
}
}
@@ -1752,15 +1759,15 @@
switch (frame_type) {
case PROBE_REQ:
{
- vif->g_struct_frame_reg[0].frame_type = frame_type;
- vif->g_struct_frame_reg[0].reg = reg;
+ vif->frame_reg[0].type = frame_type;
+ vif->frame_reg[0].reg = reg;
}
break;
case ACTION:
{
- vif->g_struct_frame_reg[1].frame_type = frame_type;
- vif->g_struct_frame_reg[1].reg = reg;
+ vif->frame_reg[1].type = frame_type;
+ vif->frame_reg[1].reg = reg;
}
break;
@@ -1797,6 +1804,7 @@
wilc_get_rssi(vif, &sinfo->signal);
+ memcpy(mac, priv->au8AssociatedBss, ETH_ALEN);
return 0;
}
@@ -2269,7 +2277,6 @@
}
priv = wdev_priv(wdev);
- sema_init(&(priv->SemHandleUpdateStats), 1);
priv->wdev = wdev;
wdev->wiphy->max_scan_ssids = MAX_NUM_PROBED_SSID;
#ifdef CONFIG_PM
@@ -2315,7 +2322,7 @@
priv->bInP2PlistenState = false;
- sema_init(&(priv->hSemScanReq), 1);
+ mutex_init(&priv->scan_req_lock);
s32Error = wilc_init(net, &priv->hif_drv);
if (s32Error)
netdev_err(net, "Error while initializing hostinterface\n");
diff --git a/drivers/staging/wilc1000/wilc_wfi_netdevice.h b/drivers/staging/wilc1000/wilc_wfi_netdevice.h
index 4123cff..3a561df6 100644
--- a/drivers/staging/wilc1000/wilc_wfi_netdevice.h
+++ b/drivers/staging/wilc1000/wilc_wfi_netdevice.h
@@ -130,8 +130,7 @@
struct wilc_wfi_key *wilc_ptk[MAX_NUM_STA];
u8 wilc_groupkey;
/* semaphores */
- struct semaphore SemHandleUpdateStats;
- struct semaphore hSemScanReq;
+ struct mutex scan_req_lock;
/* */
bool gbAutoRateAdjusted;
@@ -139,18 +138,17 @@
};
-typedef struct {
- u16 frame_type;
+struct frame_reg {
+ u16 type;
bool reg;
-
-} struct_frame_reg;
+};
struct wilc_vif {
u8 idx;
u8 iftype;
int monitor_flag;
int mac_opened;
- struct_frame_reg g_struct_frame_reg[num_reg_frame];
+ struct frame_reg frame_reg[num_reg_frame];
struct net_device_stats netstats;
struct wilc *wilc;
u8 src_addr[ETH_ALEN];
@@ -181,8 +179,7 @@
struct semaphore cfg_event;
struct semaphore sync_event;
struct semaphore txq_event;
-
- struct semaphore txq_thread_started;
+ struct completion txq_thread_started;
struct task_struct *txq_thread;
diff --git a/drivers/staging/wilc1000/wilc_wlan.c b/drivers/staging/wilc1000/wilc_wlan.c
index fd938fb..11e16d5 100644
--- a/drivers/staging/wilc1000/wilc_wlan.c
+++ b/drivers/staging/wilc1000/wilc_wlan.c
@@ -150,11 +150,6 @@
static u32 tcp_session;
static u32 pending_acks;
-static inline int init_tcp_tracking(void)
-{
- return 0;
-}
-
static inline int add_tcp_session(u32 src_prt, u32 dst_prt, u32 seq)
{
if (tcp_session < 2 * MAX_TCP_SESSION) {
@@ -330,8 +325,11 @@
tqe->priv = NULL;
tqe->tcp_pending_ack_idx = NOT_TCP_ACK;
- if (wilc_wlan_txq_add_to_head(vif, tqe))
+ if (wilc_wlan_txq_add_to_head(vif, tqe)) {
+ kfree(tqe);
return 0;
+ }
+
return 1;
}
@@ -626,13 +624,12 @@
if ((reg & 0x1) == 0) {
break;
- } else {
- counter++;
- if (counter > 200) {
- counter = 0;
- ret = wilc->hif_func->hif_write_reg(wilc, WILC_HOST_TX_CTRL, 0);
- break;
- }
+ }
+ counter++;
+ if (counter > 200) {
+ counter = 0;
+ ret = wilc->hif_func->hif_write_reg(wilc, WILC_HOST_TX_CTRL, 0);
+ break;
}
} while (!wilc->quit);
@@ -658,9 +655,8 @@
if ((reg >> 2) & 0x1) {
entries = ((reg >> 3) & 0x3f);
break;
- } else {
- release_bus(wilc, RELEASE_ALLOW_SLEEP);
}
+ release_bus(wilc, RELEASE_ALLOW_SLEEP);
} while (--timeout);
if (timeout <= 0) {
ret = wilc->hif_func->hif_write_reg(wilc, WILC_HOST_VMM_CTL, 0x0);
@@ -679,9 +675,8 @@
if (!ret)
break;
break;
- } else {
- break;
}
+ break;
} while (1);
if (!ret)
@@ -900,8 +895,6 @@
DATA_INT_CLR | ENABLE_RX_VMM);
ret = wilc->hif_func->hif_block_rx_ext(wilc, 0, buffer, size);
- if (!ret)
- goto _end_;
_end_:
if (ret) {
offset += size;
@@ -951,10 +944,8 @@
blksz = BIT(12);
dma_buffer = kmalloc(blksz, GFP_KERNEL);
- if (!dma_buffer) {
- ret = -EIO;
- goto _fail_1;
- }
+ if (!dma_buffer)
+ return -EIO;
offset = 0;
do {
@@ -992,8 +983,6 @@
kfree(dma_buffer);
-_fail_1:
-
return (ret < 0) ? ret : 0;
}
@@ -1211,7 +1200,7 @@
return 0;
}
-int wilc_wlan_cfg_set(struct wilc_vif *vif, int start, u32 wid, u8 *buffer,
+int wilc_wlan_cfg_set(struct wilc_vif *vif, int start, u16 wid, u8 *buffer,
u32 buffer_size, int commit, u32 drv_handler)
{
u32 offset;
@@ -1226,7 +1215,7 @@
offset = wilc->cfg_frame_offset;
ret_size = wilc_wlan_cfg_set_wid(wilc->cfg_frame.frame, offset,
- (u16)wid, buffer, buffer_size);
+ wid, buffer, buffer_size);
offset += ret_size;
wilc->cfg_frame_offset = offset;
@@ -1253,7 +1242,7 @@
return ret_size;
}
-int wilc_wlan_cfg_get(struct wilc_vif *vif, int start, u32 wid, int commit,
+int wilc_wlan_cfg_get(struct wilc_vif *vif, int start, u16 wid, int commit,
u32 drv_handler)
{
u32 offset;
@@ -1267,8 +1256,7 @@
wilc->cfg_frame_offset = 0;
offset = wilc->cfg_frame_offset;
- ret_size = wilc_wlan_cfg_get_wid(wilc->cfg_frame.frame, offset,
- (u16)wid);
+ ret_size = wilc_wlan_cfg_get_wid(wilc->cfg_frame.frame, offset, wid);
offset += ret_size;
wilc->cfg_frame_offset = offset;
@@ -1291,9 +1279,9 @@
return ret_size;
}
-int wilc_wlan_cfg_get_val(u32 wid, u8 *buffer, u32 buffer_size)
+int wilc_wlan_cfg_get_val(u16 wid, u8 *buffer, u32 buffer_size)
{
- return wilc_wlan_cfg_get_wid_value((u16)wid, buffer, buffer_size);
+ return wilc_wlan_cfg_get_wid_value(wid, buffer, buffer_size);
}
int wilc_send_config_pkt(struct wilc_vif *vif, u8 mode, struct wid *wids,
@@ -1440,7 +1428,6 @@
ret = -EIO;
goto _fail_;
}
- init_tcp_tracking();
return 1;
diff --git a/drivers/staging/wilc1000/wilc_wlan.h b/drivers/staging/wilc1000/wilc_wlan.h
index bcd4bfa..30e5312 100644
--- a/drivers/staging/wilc1000/wilc_wlan.h
+++ b/drivers/staging/wilc1000/wilc_wlan.h
@@ -284,11 +284,11 @@
int wilc_wlan_handle_txq(struct net_device *dev, u32 *txq_count);
void wilc_handle_isr(struct wilc *wilc);
void wilc_wlan_cleanup(struct net_device *dev);
-int wilc_wlan_cfg_set(struct wilc_vif *vif, int start, u32 wid, u8 *buffer,
+int wilc_wlan_cfg_set(struct wilc_vif *vif, int start, u16 wid, u8 *buffer,
u32 buffer_size, int commit, u32 drv_handler);
-int wilc_wlan_cfg_get(struct wilc_vif *vif, int start, u32 wid, int commit,
+int wilc_wlan_cfg_get(struct wilc_vif *vif, int start, u16 wid, int commit,
u32 drv_handler);
-int wilc_wlan_cfg_get_val(u32 wid, u8 *buffer, u32 buffer_size);
+int wilc_wlan_cfg_get_val(u16 wid, u8 *buffer, u32 buffer_size);
int wilc_wlan_txq_add_mgmt_pkt(struct net_device *dev, void *priv, u8 *buffer,
u32 buffer_size, wilc_tx_complete_func_t func);
void wilc_chip_sleep_manually(struct wilc *wilc);
diff --git a/drivers/staging/wilc1000/wilc_wlan_cfg.c b/drivers/staging/wilc1000/wilc_wlan_cfg.c
index b3425b9..926fc16 100644
--- a/drivers/staging/wilc1000/wilc_wlan_cfg.c
+++ b/drivers/staging/wilc1000/wilc_wlan_cfg.c
@@ -230,7 +230,7 @@
buf[1] = (u8)(id >> 8);
buf[2] = (u8)size;
- if ((str != NULL) && (size != 0))
+ if ((str) && (size != 0))
memcpy(&buf[3], str, size);
return (size + 3);
@@ -251,11 +251,10 @@
buf[2] = (u8)size;
buf[3] = (u8)(size >> 8);
- if ((b != NULL) && (size != 0)) {
+ if ((b) && (size != 0)) {
memcpy(&buf[4], b, size);
- for (i = 0; i < size; i++) {
+ for (i = 0; i < size; i++)
checksum += buf[i + 4];
- }
}
buf[size + 4] = checksum;
diff --git a/drivers/staging/wilc1000/wilc_wlan_if.h b/drivers/staging/wilc1000/wilc_wlan_if.h
index 83cf84d..410bfc0 100644
--- a/drivers/staging/wilc1000/wilc_wlan_if.h
+++ b/drivers/staging/wilc1000/wilc_wlan_if.h
@@ -15,18 +15,6 @@
/********************************************
*
- * Debug Flags
- *
- ********************************************/
-
-#define N_INIT 0x00000001
-#define N_ERR 0x00000002
-#define N_TXQ 0x00000004
-#define N_INTR 0x00000008
-#define N_RXQ 0x00000010
-
-/********************************************
- *
* Host Interface Defines
*
********************************************/
@@ -37,15 +25,6 @@
/********************************************
*
- * Tx/Rx Buffer Size Defines
- *
- ********************************************/
-
-#define CE_TX_BUFFER_SIZE (64 * 1024)
-#define CE_RX_BUFFER_SIZE (384 * 1024)
-
-/********************************************
- *
* Wlan Interface Defines
*
********************************************/
diff --git a/drivers/staging/wlan-ng/cfg80211.c b/drivers/staging/wlan-ng/cfg80211.c
index 2438cf7..a6e6fb9 100644
--- a/drivers/staging/wlan-ng/cfg80211.c
+++ b/drivers/staging/wlan-ng/cfg80211.c
@@ -771,8 +771,10 @@
wiphy->n_cipher_suites = PRISM2_NUM_CIPHER_SUITES;
wiphy->cipher_suites = prism2_cipher_suites;
- if (wiphy_register(wiphy) < 0)
+ if (wiphy_register(wiphy) < 0) {
+ wiphy_free(wiphy);
return NULL;
+ }
return wiphy;
}
diff --git a/drivers/staging/wlan-ng/hfa384x_usb.c b/drivers/staging/wlan-ng/hfa384x_usb.c
index 21a92df..3378107 100644
--- a/drivers/staging/wlan-ng/hfa384x_usb.c
+++ b/drivers/staging/wlan-ng/hfa384x_usb.c
@@ -614,7 +614,7 @@
ctlx = kzalloc(sizeof(*ctlx),
in_interrupt() ? GFP_ATOMIC : GFP_KERNEL);
- if (ctlx != NULL)
+ if (ctlx)
init_completion(&ctlx->done);
return ctlx;
@@ -797,7 +797,7 @@
----------------------------------------------------------------*/
static void hfa384x_cb_status(hfa384x_t *hw, const hfa384x_usbctlx_t *ctlx)
{
- if (ctlx->usercb != NULL) {
+ if (ctlx->usercb) {
hfa384x_cmdresult_t cmdresult;
if (ctlx->state != CTLX_COMPLETE) {
@@ -2738,7 +2738,7 @@
/* Call the completion function that this
* command was assigned, assuming it has one.
*/
- if (ctlx->cmdcb != NULL) {
+ if (ctlx->cmdcb) {
spin_unlock_irqrestore(&hw->ctlxq.lock, flags);
ctlx->cmdcb(hw, ctlx);
spin_lock_irqsave(&hw->ctlxq.lock, flags);
@@ -3629,7 +3629,7 @@
dbprint_urb(urb);
#endif
if ((urb->status == -ESHUTDOWN) ||
- (urb->status == -ENODEV) || (hw == NULL))
+ (urb->status == -ENODEV) || !hw)
return;
retry:
diff --git a/drivers/staging/wlan-ng/p80211conv.c b/drivers/staging/wlan-ng/p80211conv.c
index 0a8f396..6354036 100644
--- a/drivers/staging/wlan-ng/p80211conv.c
+++ b/drivers/staging/wlan-ng/p80211conv.c
@@ -75,8 +75,8 @@
#include "p80211ioctl.h"
#include "p80211req.h"
-static u8 oui_rfc1042[] = { 0x00, 0x00, 0x00 };
-static u8 oui_8021h[] = { 0x00, 0x00, 0xf8 };
+static const u8 oui_rfc1042[] = { 0x00, 0x00, 0x00 };
+static const u8 oui_8021h[] = { 0x00, 0x00, 0xf8 };
/*----------------------------------------------------------------
* p80211pb_ether_to_80211
@@ -243,7 +243,6 @@
for (i = 0; i < wlandev->spy_number; i++) {
if (!memcmp(wlandev->spy_address[i], mac, ETH_ALEN)) {
- memcpy(wlandev->spy_address[i], mac, ETH_ALEN);
wlandev->spy_stat[i].level = rxmeta->signal;
wlandev->spy_stat[i].noise = rxmeta->noise;
wlandev->spy_stat[i].qual =
diff --git a/drivers/staging/wlan-ng/p80211netdev.c b/drivers/staging/wlan-ng/p80211netdev.c
index 1f9dfba..90cc8cd 100644
--- a/drivers/staging/wlan-ng/p80211netdev.c
+++ b/drivers/staging/wlan-ng/p80211netdev.c
@@ -156,7 +156,7 @@
return -ENODEV;
/* Tell the MSD to open */
- if (wlandev->open != NULL) {
+ if (wlandev->open) {
result = wlandev->open(wlandev);
if (result == 0) {
netif_start_queue(wlandev->netdev);
@@ -186,7 +186,7 @@
int result = 0;
wlandevice_t *wlandev = netdev->ml_priv;
- if (wlandev->close != NULL)
+ if (wlandev->close)
result = wlandev->close(wlandev);
netif_stop_queue(wlandev->netdev);
diff --git a/drivers/staging/wlan-ng/p80211netdev.h b/drivers/staging/wlan-ng/p80211netdev.h
index 810ee68..820a0e2 100644
--- a/drivers/staging/wlan-ng/p80211netdev.h
+++ b/drivers/staging/wlan-ng/p80211netdev.h
@@ -158,7 +158,6 @@
/* WLAN device type */
typedef struct wlandevice {
- struct wlandevice *next; /* link for list of devices */
void *priv; /* private data for MSD */
/* Subsystem State */
diff --git a/drivers/staging/wlan-ng/prism2fw.c b/drivers/staging/wlan-ng/prism2fw.c
index 8564d9e..56bffd9 100644
--- a/drivers/staging/wlan-ng/prism2fw.c
+++ b/drivers/staging/wlan-ng/prism2fw.c
@@ -278,7 +278,8 @@
/* Build the PDA we're going to use. */
if (read_cardpda(&pda, wlandev)) {
netdev_err(wlandev->netdev, "load_cardpda failed, exiting.\n");
- return 1;
+ result = 1;
+ goto out;
}
/* read the card's PRI-SUP */
@@ -315,55 +316,58 @@
if (result) {
netdev_err(wlandev->netdev,
"Failed to read the data exiting.\n");
- return 1;
+ goto out;
}
result = validate_identity();
-
if (result) {
netdev_err(wlandev->netdev, "Incompatible firmware image.\n");
- return 1;
+ goto out;
}
if (startaddr == 0x00000000) {
netdev_err(wlandev->netdev,
"Can't RAM download a Flash image!\n");
- return 1;
+ result = 1;
+ goto out;
}
/* Make the image chunks */
result = mkimage(fchunk, &nfchunks);
if (result) {
netdev_err(wlandev->netdev, "Failed to make image chunk.\n");
- return 1;
+ goto free_chunks;
}
/* Do any plugging */
result = plugimage(fchunk, nfchunks, s3plug, ns3plug, &pda);
if (result) {
netdev_err(wlandev->netdev, "Failed to plug data.\n");
- return 1;
+ goto free_chunks;
}
/* Insert any CRCs */
- if (crcimage(fchunk, nfchunks, s3crc, ns3crc)) {
+ result = crcimage(fchunk, nfchunks, s3crc, ns3crc);
+ if (result) {
netdev_err(wlandev->netdev, "Failed to insert all CRCs\n");
- return 1;
+ goto free_chunks;
}
/* Write the image */
result = writeimage(wlandev, fchunk, nfchunks);
if (result) {
netdev_err(wlandev->netdev, "Failed to ramwrite image data.\n");
- return 1;
+ goto free_chunks;
}
+ netdev_info(wlandev->netdev, "prism2_usb: firmware loading finished.\n");
+
+free_chunks:
/* clear any allocated memory */
free_chunks(fchunk, &nfchunks);
free_srecs();
- netdev_info(wlandev->netdev, "prism2_usb: firmware loading finished.\n");
-
+out:
return result;
}
diff --git a/drivers/staging/wlan-ng/prism2usb.c b/drivers/staging/wlan-ng/prism2usb.c
index 41358bb..b26d09f 100644
--- a/drivers/staging/wlan-ng/prism2usb.c
+++ b/drivers/staging/wlan-ng/prism2usb.c
@@ -8,7 +8,7 @@
{ USB_DEVICE(vid, pid), \
.driver_info = (unsigned long)name }
-static struct usb_device_id usb_prism_tbl[] = {
+static const struct usb_device_id usb_prism_tbl[] = {
PRISM_DEV(0x04bb, 0x0922, "IOData AirPort WN-B11/USBS"),
PRISM_DEV(0x07aa, 0x0012, "Corega Wireless LAN USB Stick-11"),
PRISM_DEV(0x09aa, 0x3642, "Prism2.x 11Mbps WLAN USB Adapter"),
diff --git a/drivers/staging/xgifb/XGI_main_26.c b/drivers/staging/xgifb/XGI_main_26.c
index 7eadf92..d56ef14 100644
--- a/drivers/staging/xgifb/XGI_main_26.c
+++ b/drivers/staging/xgifb/XGI_main_26.c
@@ -1130,8 +1130,9 @@
return (var->bits_per_pixel == 8) ? 256 : 16;
}
-static int XGIfb_setcolreg(unsigned regno, unsigned red, unsigned green,
- unsigned blue, unsigned transp, struct fb_info *info)
+static int XGIfb_setcolreg(unsigned int regno, unsigned int red,
+ unsigned int green, unsigned int blue,
+ unsigned int transp, struct fb_info *info)
{
struct xgifb_video_info *xgifb_info = info->par;
diff --git a/drivers/staging/xgifb/vb_init.c b/drivers/staging/xgifb/vb_init.c
index 26b539b..062ece2 100644
--- a/drivers/staging/xgifb/vb_init.c
+++ b/drivers/staging/xgifb/vb_init.c
@@ -355,7 +355,8 @@
unsigned long P3d4 = Port, P3c4 = Port - 0x10;
/* keep following setting sequence, each setting in
- * the same reg insert idle */
+ * the same reg insert idle
+ */
xgifb_reg_set(P3d4, 0x82, 0x77);
xgifb_reg_set(P3d4, 0x86, 0x00);
xgifb_reg_get(P3d4, 0x86); /* Insert read command for delay */
@@ -551,7 +552,8 @@
writel(Position, fbaddr + Position);
}
- usleep_range(500, 1500); /* Fix #1759 Memory Size error in Multi-Adapter. */
+ /* Fix #1759 Memory Size error in Multi-Adapter. */
+ usleep_range(500, 1500);
Position = 0;
@@ -699,11 +701,11 @@
break;
case XG42:
/*
- XG42 SR14 D[3] Reserve
- D[2] = 1, Dual Channel
- = 0, Single Channel
-
- It's Different from Other XG40 Series.
+ * XG42 SR14 D[3] Reserve
+ * D[2] = 1, Dual Channel
+ * = 0, Single Channel
+ *
+ * It's Different from Other XG40 Series.
*/
if (XGINew_CheckFrequence(pVBInfo) == 1) { /* DDRII, DDR2x */
pVBInfo->ram_bus = 32; /* 32 bits */
diff --git a/drivers/staging/xgifb/vb_setmode.c b/drivers/staging/xgifb/vb_setmode.c
index f97c77d..50c8ea4 100644
--- a/drivers/staging/xgifb/vb_setmode.c
+++ b/drivers/staging/xgifb/vb_setmode.c
@@ -108,9 +108,9 @@
if (pVBInfo->VBInfo & XGI_SetCRT2ToLCDA) {
ARdata = 0;
} else if ((pVBInfo->VBInfo &
- (SetCRT2ToTV | SetCRT2ToLCD)) &&
- (pVBInfo->VBInfo & SetInSlaveMode)) {
- ARdata = 0;
+ (SetCRT2ToTV | SetCRT2ToLCD)) &&
+ (pVBInfo->VBInfo & SetInSlaveMode)) {
+ ARdata = 0;
}
}
@@ -1992,7 +1992,8 @@
}
/* LCD+TV can't support in slave mode
- * (Force LCDA+TV->LCDB) */
+ * (Force LCDA+TV->LCDB)
+ */
if ((tempbx & SetInSlaveMode) && (tempbx & XGI_SetCRT2ToLCDA)) {
tempbx ^= (SetCRT2ToLCD | XGI_SetCRT2ToLCDA |
SetCRT2ToDualEdge);
@@ -2983,7 +2984,7 @@
if ((pVBInfo->VBInfo & SetCRT2ToHiVision) &&
!(pVBInfo->VBType & VB_SIS301LV) && (resinfo == 7))
- temp -= 2;
+ temp -= 2;
}
/* 0x05 Horizontal Display Start */
@@ -3450,8 +3451,9 @@
if (!(pVBInfo->TVInfo &
(TVSetYPbPr525p | TVSetYPbPr750p)))
tempbx >>= 1;
- } else
+ } else {
tempbx >>= 1;
+ }
}
tempbx -= 2;
@@ -3839,9 +3841,9 @@
if (pVBInfo->VGAVDE == 525) {
if (pVBInfo->VBType & (VB_SIS301B | VB_SIS302B
| VB_SIS301LV | VB_SIS302LV
- | VB_XGI301C)) {
+ | VB_XGI301C))
temp = 0xC6;
- } else
+ else
temp = 0xC4;
xgifb_reg_set(pVBInfo->Part2Port, 0x2f, temp);
@@ -3851,9 +3853,9 @@
if (pVBInfo->VGAVDE == 420) {
if (pVBInfo->VBType & (VB_SIS301B | VB_SIS302B
| VB_SIS301LV | VB_SIS302LV
- | VB_XGI301C)) {
+ | VB_XGI301C))
temp = 0x4F;
- } else
+ else
temp = 0x4E;
xgifb_reg_set(pVBInfo->Part2Port, 0x2f, temp);
}
diff --git a/drivers/staging/xgifb/vb_table.h b/drivers/staging/xgifb/vb_table.h
index 45f2c99..c801deb 100644
--- a/drivers/staging/xgifb/vb_table.h
+++ b/drivers/staging/xgifb/vb_table.h
@@ -58,8 +58,9 @@
{0xC4, 0x40, 0x84}, /* 1 CR8A */
{0xC4, 0x40, 0x84}, /* 2 CR8B */
{0xB3, 0x13, 0xa4}, /* 3 CR40[7],
- CR99[2:0],
- CR45[3:0]*/
+ * CR99[2:0],
+ * CR45[3:0]
+ */
{0xf0, 0xf5, 0xf0}, /* 4 CR59 */
{0x90, 0x90, 0x24}, /* 5 CR68 */
{0x77, 0x67, 0x44}, /* 6 CR69 */
@@ -101,9 +102,11 @@
{0x38, 0x0a1b, 0x0508, 0x08, 0x00, 0x16},
{0x3a, 0x0e3b, 0x0609, 0x09, 0x00, 0x1e},
{0x3c, 0x0e3b, 0x070a, 0x0a, 0x00, 0x22}, /* mode 1600x1200
- add CRT2MODE [2003/10/07] */
+ * add CRT2MODE [2003/10/07]
+ */
{0x3d, 0x0e7d, 0x070a, 0x0a, 0x00, 0x22}, /* mode 1600x1200
- add CRT2MODE */
+ * add CRT2MODE
+ */
{0x40, 0x9a1c, 0x0000, 0x00, 0x04, 0x00},
{0x41, 0x9a1d, 0x0000, 0x00, 0x04, 0x00},
{0x43, 0x0a1c, 0x0306, 0x06, 0x05, 0x06},
@@ -129,7 +132,8 @@
{0x64, 0x0a7f, 0x0508, 0x08, 0x00, 0x16},
{0x65, 0x0eff, 0x0609, 0x09, 0x00, 0x1e},
{0x66, 0x0eff, 0x070a, 0x0a, 0x00, 0x22}, /* mode 1600x1200
- add CRT2MODE */
+ * add CRT2MODE
+ */
{0x68, 0x067b, 0x080b, 0x0b, 0x00, 0x29},
{0x69, 0x06fd, 0x080b, 0x0b, 0x00, 0x29},
{0x6b, 0x07ff, 0x080b, 0x0b, 0x00, 0x29},
@@ -223,38 +227,38 @@
0x0D, 0x3E, 0xE0, 0x83, 0xDF, 0x0E, 0x90} }, /* 0xb */
{ {0x65, 0x4F, 0x89, 0x57, 0x9F, 0x00, 0x01, 0x00,
0xFB, 0x1F, 0xE6, 0x8A, 0xDF, 0xFC, 0x10} }, /* 0xc */
- { {0x7B, 0x63, 0x9F, 0x6A, 0x93, 0x00, 0x05, 0x00, /* ;
- 0D (800x600,56Hz) */
- 0x6F, 0xF0, 0x58, 0x8A, 0x57, 0x70, 0xA0} }, /* ;
- (VCLK 36.0MHz) */
- { {0x7F, 0x63, 0x83, 0x6C, 0x1C, 0x00, 0x06, 0x00, /* ;
- 0E (800x600,60Hz) */
- 0x72, 0xF0, 0x58, 0x8C, 0x57, 0x73, 0xA0} }, /* ;
- (VCLK 40.0MHz) */
- { {0x7D, 0x63, 0x81, 0x6E, 0x1D, 0x00, 0x06, 0x00, /* ;
- 0F (800x600,72Hz) */
- 0x98, 0xF0, 0x7C, 0x82, 0x57, 0x99, 0x80} }, /* ;
- (VCLK 50.0MHz) */
- { {0x7F, 0x63, 0x83, 0x69, 0x13, 0x00, 0x06, 0x00, /* ;
- 10 (800x600,75Hz) */
- 0x6F, 0xF0, 0x58, 0x8B, 0x57, 0x70, 0xA0} }, /* ;
- (VCLK 49.5MHz) */
- { {0x7E, 0x63, 0x82, 0x6B, 0x13, 0x00, 0x06, 0x00, /* ;
- 11 (800x600,85Hz) */
- 0x75, 0xF0, 0x58, 0x8B, 0x57, 0x76, 0xA0} }, /* ;
- (VCLK 56.25MHz) */
- { {0x81, 0x63, 0x85, 0x6D, 0x18, 0x00, 0x06, 0x60, /* ;
- 12 (800x600,100Hz) */
- 0x7A, 0xF0, 0x58, 0x8B, 0x57, 0x7B, 0xA0} }, /* ;
- (VCLK 75.8MHz) */
- { {0x83, 0x63, 0x87, 0x6E, 0x19, 0x00, 0x06, 0x60, /* ;
- 13 (800x600,120Hz) */
- 0x81, 0xF0, 0x58, 0x8B, 0x57, 0x82, 0xA0} }, /* ;
- (VCLK 79.411MHz) */
- { {0x85, 0x63, 0x89, 0x6F, 0x1A, 0x00, 0x06, 0x60, /* ;
- 14 (800x600,160Hz) */
- 0x91, 0xF0, 0x58, 0x8B, 0x57, 0x92, 0xA0} }, /* ;
- (VCLK 105.822MHz) */
+ /* 0D (800x600,56Hz) */
+ { {0x7B, 0x63, 0x9F, 0x6A, 0x93, 0x00, 0x05, 0x00,
+ /* (VCLK 36.0MHz) */
+ 0x6F, 0xF0, 0x58, 0x8A, 0x57, 0x70, 0xA0} },
+ /* 0E (800x600,60Hz) */
+ { {0x7F, 0x63, 0x83, 0x6C, 0x1C, 0x00, 0x06, 0x00,
+ /* (VCLK 40.0MHz) */
+ 0x72, 0xF0, 0x58, 0x8C, 0x57, 0x73, 0xA0} },
+ /* 0F (800x600,72Hz) */
+ { {0x7D, 0x63, 0x81, 0x6E, 0x1D, 0x00, 0x06, 0x00,
+ /* (VCLK 50.0MHz) */
+ 0x98, 0xF0, 0x7C, 0x82, 0x57, 0x99, 0x80} },
+ /* 10 (800x600,75Hz) */
+ { {0x7F, 0x63, 0x83, 0x69, 0x13, 0x00, 0x06, 0x00,
+ /* (VCLK 49.5MHz) */
+ 0x6F, 0xF0, 0x58, 0x8B, 0x57, 0x70, 0xA0} },
+ /* 11 (800x600,85Hz) */
+ { {0x7E, 0x63, 0x82, 0x6B, 0x13, 0x00, 0x06, 0x00,
+ /* (VCLK 56.25MHz) */
+ 0x75, 0xF0, 0x58, 0x8B, 0x57, 0x76, 0xA0} },
+ /* 12 (800x600,100Hz) */
+ { {0x81, 0x63, 0x85, 0x6D, 0x18, 0x00, 0x06, 0x60,
+ /* (VCLK 75.8MHz) */
+ 0x7A, 0xF0, 0x58, 0x8B, 0x57, 0x7B, 0xA0} },
+ /* 13 (800x600,120Hz) */
+ { {0x83, 0x63, 0x87, 0x6E, 0x19, 0x00, 0x06, 0x60,
+ /* (VCLK 79.411MHz) */
+ 0x81, 0xF0, 0x58, 0x8B, 0x57, 0x82, 0xA0} },
+ /* 14 (800x600,160Hz) */
+ { {0x85, 0x63, 0x89, 0x6F, 0x1A, 0x00, 0x06, 0x60,
+ /* (VCLK 105.822MHz) */
+ 0x91, 0xF0, 0x58, 0x8B, 0x57, 0x92, 0xA0} },
{ {0x99, 0x7F, 0x9D, 0x84, 0x1A, 0x00, 0x02, 0x00,
0x96, 0x1F, 0x7F, 0x83, 0x7F, 0x97, 0x10} }, /* 0x15 */
{ {0xA3, 0x7F, 0x87, 0x86, 0x97, 0x00, 0x02, 0x00,
@@ -388,7 +392,8 @@
static const struct SiS_LCDData XGI_CetLCD1024x768Data[] = {
{1, 1, 1344, 806, 1344, 806}, /* ; 00 (320x200,320x400,
- 640x200,640x400) */
+ * 640x200,640x400)
+ */
{1, 1, 1344, 806, 1344, 806}, /* 01 (320x350,640x350) */
{1, 1, 1344, 806, 1344, 806}, /* 02 (360x400,720x400) */
{1, 1, 1344, 806, 1344, 806}, /* 03 (720x350) */
@@ -421,7 +426,8 @@
static const struct SiS_LCDData XGI_CetLCD1280x1024Data[] = {
{1, 1, 1688, 1066, 1688, 1066}, /* 00 (320x200,320x400,
- 640x200,640x400) */
+ * 640x200,640x400)
+ */
{1, 1, 1688, 1066, 1688, 1066}, /* 01 (320x350,640x350) */
{1, 1, 1688, 1066, 1688, 1066}, /* 02 (360x400,720x400) */
{1, 1, 1688, 1066, 1688, 1066}, /* 03 (720x350) */
@@ -434,7 +440,8 @@
static const struct SiS_LCDData xgifb_lcd_1400x1050[] = {
{211, 100, 2100, 408, 1688, 1066}, /* 00 (320x200,320x400,
- 640x200,640x400) */
+ * 640x200,640x400)
+ */
{211, 64, 1536, 358, 1688, 1066}, /* 01 (320x350,640x350) */
{211, 100, 2100, 408, 1688, 1066}, /* 02 (360x400,720x400) */
{211, 64, 1536, 358, 1688, 1066}, /* 03 (720x350) */
@@ -442,13 +449,15 @@
{211, 72, 1008, 609, 1688, 1066}, /* 05 (800x600x60Hz) */
{211, 128, 1400, 776, 1688, 1066}, /* 06 (1024x768x60Hz) */
{1, 1, 1688, 1066, 1688, 1066}, /* 07 (1280x1024x60Hz
- w/o Scaling) */
+ * w/o Scaling)
+ */
{1, 1, 1688, 1066, 1688, 1066} /* 08 (1400x1050x60Hz) */
};
static const struct SiS_LCDData XGI_ExtLCD1600x1200Data[] = {
{4, 1, 1620, 420, 2160, 1250}, /* 00 (320x200,320x400,
- 640x200,640x400)*/
+ * 640x200,640x400)
+ */
{27, 7, 1920, 375, 2160, 1250}, /* 01 (320x350,640x350) */
{4, 1, 1620, 420, 2160, 1250}, /* 02 (360x400,720x400)*/
{27, 7, 1920, 375, 2160, 1250}, /* 03 (720x350) */
@@ -462,7 +471,8 @@
static const struct SiS_LCDData XGI_StLCD1600x1200Data[] = {
{27, 4, 800, 500, 2160, 1250}, /* 00 (320x200,320x400,
- 640x200,640x400) */
+ * 640x200,640x400)
+ */
{27, 4, 800, 500, 2160, 1250}, /* 01 (320x350,640x350) */
{27, 4, 800, 500, 2160, 1250}, /* 02 (360x400,720x400) */
{27, 4, 800, 500, 2160, 1250}, /* 03 (720x350) */
@@ -489,7 +499,8 @@
static const struct SiS_LCDData XGI_ExtLCD1024x768x75Data[] = {
{42, 25, 1536, 419, 1344, 806}, /* ; 00 (320x200,320x400,
- 640x200,640x400) */
+ * 640x200,640x400)
+ */
{48, 25, 1536, 369, 1344, 806}, /* ; 01 (320x350,640x350) */
{42, 25, 1536, 419, 1344, 806}, /* ; 02 (360x400,720x400) */
{48, 25, 1536, 369, 1344, 806}, /* ; 03 (720x350) */
@@ -500,7 +511,8 @@
static const struct SiS_LCDData XGI_CetLCD1024x768x75Data[] = {
{1, 1, 1312, 800, 1312, 800}, /* ; 00 (320x200,320x400,
- 640x200,640x400) */
+ * 640x200,640x400)
+ */
{1, 1, 1312, 800, 1312, 800}, /* ; 01 (320x350,640x350) */
{1, 1, 1312, 800, 1312, 800}, /* ; 02 (360x400,720x400) */
{1, 1, 1312, 800, 1312, 800}, /* ; 03 (720x350) */
@@ -511,7 +523,8 @@
static const struct SiS_LCDData xgifb_lcd_1280x1024x75[] = {
{211, 60, 1024, 501, 1688, 1066}, /* ; 00 (320x200,320x400,
- 640x200,640x400) */
+ * 640x200,640x400)
+ */
{211, 60, 1024, 508, 1688, 1066}, /* ; 01 (320x350,640x350) */
{211, 60, 1024, 501, 1688, 1066}, /* ; 02 (360x400,720x400) */
{211, 60, 1024, 508, 1688, 1066}, /* ; 03 (720x350) */
@@ -525,7 +538,8 @@
static const struct SiS_LCDData XGI_NoScalingDatax75[] = {
{1, 1, 800, 449, 800, 449}, /* ; 00 (320x200, 320x400,
- 640x200, 640x400) */
+ * 640x200, 640x400)
+ */
{1, 1, 800, 449, 800, 449}, /* ; 01 (320x350, 640x350) */
{1, 1, 900, 449, 900, 449}, /* ; 02 (360x400, 720x400) */
{1, 1, 900, 449, 900, 449}, /* ; 03 (720x350) */
@@ -732,7 +746,8 @@
static const struct XGI330_LCDDataDesStruct2 XGI_NoScalingDesData[] = {
{9, 657, 448, 405, 96, 2}, /* 00 (320x200,320x400,
- 640x200,640x400) */
+ * 640x200,640x400)
+ */
{9, 657, 448, 355, 96, 2}, /* 01 (320x350,640x350) */
{9, 657, 448, 405, 96, 2}, /* 02 (360x400,720x400) */
{9, 657, 448, 355, 96, 2}, /* 03 (720x350) */
@@ -818,7 +833,8 @@
/* Scaling LCD 75Hz */
static const struct XGI330_LCDDataDesStruct2 XGI_NoScalingDesDatax75[] = {
{9, 657, 448, 405, 96, 2}, /* ; 00 (320x200,320x400,
- 640x200,640x400) */
+ * 640x200,640x400)
+ */
{9, 657, 448, 355, 96, 2}, /* ; 01 (320x350,640x350) */
{9, 738, 448, 405, 108, 2}, /* ; 02 (360x400,720x400) */
{9, 738, 448, 355, 108, 2}, /* ; 03 (720x350) */
@@ -873,7 +889,8 @@
static const struct SiS_TVData XGI_St1HiTVData[] = {
{1, 1, 892, 563, 690, 800, 0, 0, 0}, /* 00 (320x200,320x400,
- 640x200,640x400) */
+ * 640x200,640x400)
+ */
{1, 1, 892, 563, 690, 700, 0, 0, 0}, /* 01 (320x350,640x350) */
{1, 1, 1000, 563, 785, 800, 0, 0, 0}, /* 02 (360x400,720x400) */
{1, 1, 1000, 563, 785, 700, 0, 0, 0}, /* 03 (720x350) */
@@ -883,7 +900,8 @@
static const struct SiS_TVData XGI_St2HiTVData[] = {
{3, 1, 840, 483, 1648, 960, 0x032, 0, 0}, /* 00 (320x200,320x400,
- 640x200,640x400) */
+ * 640x200,640x400)
+ */
{1, 1, 892, 563, 690, 700, 0, 0, 0}, /* 01 (320x350,640x350) */
{3, 1, 840, 483, 1648, 960, 0x032, 0, 0}, /* 02 (360x400,720x400) */
{1, 1, 1000, 563, 785, 700, 0, 0, 0}, /* 03 (720x350) */
@@ -893,7 +911,8 @@
static const struct SiS_TVData XGI_ExtHiTVData[] = {
{6, 1, 840, 563, 1632, 960, 0, 0, 0}, /* 00 (320x200,320x400,
- 640x200,640x400) */
+ * 640x200,640x400)
+ */
{3, 1, 960, 563, 1632, 960, 0, 0, 0}, /* 01 (320x350,640x350) */
{3, 1, 840, 483, 1632, 960, 0, 0, 0}, /* 02 (360x400,720x400) */
{3, 1, 960, 563, 1632, 960, 0, 0, 0}, /* 03 (720x350) */
@@ -948,7 +967,8 @@
static const struct SiS_TVData XGI_ExtYPbPr750pData[] = {
{ 3, 1, 935, 470, 1130, 680, 50, 0, 0}, /* 00 (320x200,320x400,
- 640x200,640x400) */
+ * 640x200,640x400)
+ */
{24, 7, 935, 420, 1130, 680, 50, 0, 0}, /* 01 (320x350,640x350) */
{ 3, 1, 935, 470, 1130, 680, 50, 0, 0}, /* 02 (360x400,720x400) */
{24, 7, 935, 420, 1130, 680, 50, 0, 0}, /* 03 (720x350) */
@@ -1269,7 +1289,8 @@
{1312, 800, 1312, 800}, /* ; 06 (1024x768x75Hz) */
{1688, 1066, 1688, 1066}, /* ; 07 (1280x1024x75Hz) */
{1688, 1066, 1688, 1066}, /* ; 08 (1400x1050x75Hz)
- ;;[ycchen] 12/19/02 */
+ * ;;[ycchen] 12/19/02
+ */
{2160, 1250, 2160, 1250}, /* ; 09 (1600x1200x75Hz) */
{1688, 806, 1688, 806}, /* ; 0A (1280x768x75Hz) */
};
@@ -1364,7 +1385,8 @@
static const struct XGI330_LCDDataDesStruct2 XGI_LVDSNoScalingDesData[] = {
{0, 648, 448, 405, 96, 2}, /* 00 (320x200,320x400,
- 640x200,640x400) */
+ * 640x200,640x400)
+ */
{0, 648, 448, 355, 96, 2}, /* 01 (320x350,640x350) */
{0, 648, 448, 405, 96, 2}, /* 02 (360x400,720x400) */
{0, 648, 448, 355, 96, 2}, /* 03 (720x350) */
@@ -1435,7 +1457,8 @@
/* Scaling LCD 75Hz */
static const struct XGI330_LCDDataDesStruct2 XGI_LVDSNoScalingDesDatax75[] = {
{0, 648, 448, 405, 96, 2}, /* ; 00 (320x200,320x400,
- 640x200,640x400) */
+ * 640x200,640x400)
+ */
{0, 648, 448, 355, 96, 2}, /* ; 01 (320x350,640x350) */
{0, 729, 448, 405, 108, 2}, /* ; 02 (360x400,720x400) */
{0, 729, 448, 355, 108, 2}, /* ; 03 (720x350) */
diff --git a/drivers/staging/xgifb/vb_util.h b/drivers/staging/xgifb/vb_util.h
index f613f54..08db58b 100644
--- a/drivers/staging/xgifb/vb_util.h
+++ b/drivers/staging/xgifb/vb_util.h
@@ -13,7 +13,7 @@
}
static inline void xgifb_reg_and_or(unsigned long port, u8 index,
- unsigned data_and, unsigned data_or)
+ unsigned int data_and, unsigned int data_or)
{
u8 temp;
@@ -22,7 +22,8 @@
xgifb_reg_set(port, index, temp);
}
-static inline void xgifb_reg_and(unsigned long port, u8 index, unsigned data_and)
+static inline void xgifb_reg_and(unsigned long port, u8 index,
+ unsigned int data_and)
{
u8 temp;
@@ -31,7 +32,8 @@
xgifb_reg_set(port, index, temp);
}
-static inline void xgifb_reg_or(unsigned long port, u8 index, unsigned data_or)
+static inline void xgifb_reg_or(unsigned long port, u8 index,
+ unsigned int data_or)
{
u8 temp;
diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c
index ab2bf12..590384a 100644
--- a/drivers/target/target_core_transport.c
+++ b/drivers/target/target_core_transport.c
@@ -2195,7 +2195,7 @@
transport_handle_queue_full(cmd, cmd->se_dev);
}
-static inline void transport_free_sgl(struct scatterlist *sgl, int nents)
+void target_free_sgl(struct scatterlist *sgl, int nents)
{
struct scatterlist *sg;
int count;
@@ -2205,6 +2205,7 @@
kfree(sgl);
}
+EXPORT_SYMBOL(target_free_sgl);
static inline void transport_reset_sgl_orig(struct se_cmd *cmd)
{
@@ -2225,7 +2226,7 @@
static inline void transport_free_pages(struct se_cmd *cmd)
{
if (!(cmd->se_cmd_flags & SCF_PASSTHROUGH_PROT_SG_TO_MEM_NOALLOC)) {
- transport_free_sgl(cmd->t_prot_sg, cmd->t_prot_nents);
+ target_free_sgl(cmd->t_prot_sg, cmd->t_prot_nents);
cmd->t_prot_sg = NULL;
cmd->t_prot_nents = 0;
}
@@ -2236,7 +2237,7 @@
* SG_TO_MEM_NOALLOC to function with COMPARE_AND_WRITE
*/
if (cmd->se_cmd_flags & SCF_COMPARE_AND_WRITE) {
- transport_free_sgl(cmd->t_bidi_data_sg,
+ target_free_sgl(cmd->t_bidi_data_sg,
cmd->t_bidi_data_nents);
cmd->t_bidi_data_sg = NULL;
cmd->t_bidi_data_nents = 0;
@@ -2246,11 +2247,11 @@
}
transport_reset_sgl_orig(cmd);
- transport_free_sgl(cmd->t_data_sg, cmd->t_data_nents);
+ target_free_sgl(cmd->t_data_sg, cmd->t_data_nents);
cmd->t_data_sg = NULL;
cmd->t_data_nents = 0;
- transport_free_sgl(cmd->t_bidi_data_sg, cmd->t_bidi_data_nents);
+ target_free_sgl(cmd->t_bidi_data_sg, cmd->t_bidi_data_nents);
cmd->t_bidi_data_sg = NULL;
cmd->t_bidi_data_nents = 0;
}
@@ -2324,20 +2325,22 @@
int
target_alloc_sgl(struct scatterlist **sgl, unsigned int *nents, u32 length,
- bool zero_page)
+ bool zero_page, bool chainable)
{
struct scatterlist *sg;
struct page *page;
gfp_t zero_flag = (zero_page) ? __GFP_ZERO : 0;
- unsigned int nent;
+ unsigned int nalloc, nent;
int i = 0;
- nent = DIV_ROUND_UP(length, PAGE_SIZE);
- sg = kmalloc(sizeof(struct scatterlist) * nent, GFP_KERNEL);
+ nalloc = nent = DIV_ROUND_UP(length, PAGE_SIZE);
+ if (chainable)
+ nalloc++;
+ sg = kmalloc_array(nalloc, sizeof(struct scatterlist), GFP_KERNEL);
if (!sg)
return -ENOMEM;
- sg_init_table(sg, nent);
+ sg_init_table(sg, nalloc);
while (length) {
u32 page_len = min_t(u32, length, PAGE_SIZE);
@@ -2361,6 +2364,7 @@
kfree(sg);
return -ENOMEM;
}
+EXPORT_SYMBOL(target_alloc_sgl);
/*
* Allocate any required resources to execute the command. For writes we
@@ -2376,7 +2380,7 @@
if (cmd->prot_op != TARGET_PROT_NORMAL &&
!(cmd->se_cmd_flags & SCF_PASSTHROUGH_PROT_SG_TO_MEM_NOALLOC)) {
ret = target_alloc_sgl(&cmd->t_prot_sg, &cmd->t_prot_nents,
- cmd->prot_length, true);
+ cmd->prot_length, true, false);
if (ret < 0)
return TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE;
}
@@ -2401,13 +2405,13 @@
ret = target_alloc_sgl(&cmd->t_bidi_data_sg,
&cmd->t_bidi_data_nents,
- bidi_length, zero_flag);
+ bidi_length, zero_flag, false);
if (ret < 0)
return TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE;
}
ret = target_alloc_sgl(&cmd->t_data_sg, &cmd->t_data_nents,
- cmd->data_length, zero_flag);
+ cmd->data_length, zero_flag, false);
if (ret < 0)
return TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE;
} else if ((cmd->se_cmd_flags & SCF_COMPARE_AND_WRITE) &&
@@ -2421,7 +2425,7 @@
ret = target_alloc_sgl(&cmd->t_bidi_data_sg,
&cmd->t_bidi_data_nents,
- caw_length, zero_flag);
+ caw_length, zero_flag, false);
if (ret < 0)
return TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE;
}
diff --git a/drivers/target/target_core_xcopy.c b/drivers/target/target_core_xcopy.c
index 47fe94e..75cd854 100644
--- a/drivers/target/target_core_xcopy.c
+++ b/drivers/target/target_core_xcopy.c
@@ -563,7 +563,7 @@
if (alloc_mem) {
rc = target_alloc_sgl(&cmd->t_data_sg, &cmd->t_data_nents,
- cmd->data_length, false);
+ cmd->data_length, false, false);
if (rc < 0) {
ret = rc;
goto out;
diff --git a/drivers/thunderbolt/ctl.c b/drivers/thunderbolt/ctl.c
index 799634b..1146ff4 100644
--- a/drivers/thunderbolt/ctl.c
+++ b/drivers/thunderbolt/ctl.c
@@ -249,7 +249,7 @@
* cfg_read/cfg_write.
*/
tb_ctl_WARN(ctl,
- "CFG_ERROR(%llx:%x): Invalid config space of offset\n",
+ "CFG_ERROR(%llx:%x): Invalid config space or offset\n",
res->response_route, res->response_port);
return;
case TB_CFG_ERROR_NO_SUCH_PORT:
diff --git a/drivers/thunderbolt/eeprom.c b/drivers/thunderbolt/eeprom.c
index 0dde34e..2b9602c 100644
--- a/drivers/thunderbolt/eeprom.c
+++ b/drivers/thunderbolt/eeprom.c
@@ -221,7 +221,7 @@
u8 micro1:4;
u8 micro3;
- /* BYTES 5-6, TODO: verify (find hardware that has these set) */
+ /* BYTES 6-7, TODO: verify (find hardware that has these set) */
u8 peer_port_rid:4;
u8 unknown3:3;
bool has_peer_port:1;
@@ -388,6 +388,11 @@
sw->ports[4].link_nr = 1;
sw->ports[3].dual_link_port = &sw->ports[4];
sw->ports[4].dual_link_port = &sw->ports[3];
+
+ /* Port 5 is inaccessible on this gen 1 controller */
+ if (sw->config.device_id == PCI_DEVICE_ID_INTEL_LIGHT_RIDGE)
+ sw->ports[5].disabled = true;
+
return 0;
}
@@ -444,6 +449,7 @@
return tb_drom_parse_entries(sw);
err:
kfree(sw->drom);
+ sw->drom = NULL;
return -EIO;
}
diff --git a/drivers/thunderbolt/nhi.c b/drivers/thunderbolt/nhi.c
index 20a41f7..9c15344 100644
--- a/drivers/thunderbolt/nhi.c
+++ b/drivers/thunderbolt/nhi.c
@@ -37,7 +37,8 @@
*/
static void ring_interrupt_active(struct tb_ring *ring, bool active)
{
- int reg = REG_RING_INTERRUPT_BASE + ring_interrupt_index(ring) / 32;
+ int reg = REG_RING_INTERRUPT_BASE +
+ ring_interrupt_index(ring) / 32 * 4;
int bit = ring_interrupt_index(ring) & 31;
int mask = 1 << bit;
u32 old, new;
@@ -564,7 +565,7 @@
/* cannot fail - table is allocated bin pcim_iomap_regions */
nhi->iobase = pcim_iomap_table(pdev)[0];
nhi->hop_count = ioread32(nhi->iobase + REG_HOP_COUNT) & 0x3ff;
- if (nhi->hop_count != 12)
+ if (nhi->hop_count != 12 && nhi->hop_count != 32)
dev_warn(&pdev->dev, "unexpected hop count: %d\n",
nhi->hop_count);
INIT_WORK(&nhi->interrupt_work, nhi_interrupt_work);
@@ -633,16 +634,24 @@
static struct pci_device_id nhi_ids[] = {
/*
* We have to specify class, the TB bridges use the same device and
- * vendor (sub)id.
+ * vendor (sub)id on gen 1 and gen 2 controllers.
*/
{
.class = PCI_CLASS_SYSTEM_OTHER << 8, .class_mask = ~0,
- .vendor = PCI_VENDOR_ID_INTEL, .device = 0x1547,
+ .vendor = PCI_VENDOR_ID_INTEL,
+ .device = PCI_DEVICE_ID_INTEL_LIGHT_RIDGE,
.subvendor = 0x2222, .subdevice = 0x1111,
},
{
.class = PCI_CLASS_SYSTEM_OTHER << 8, .class_mask = ~0,
- .vendor = PCI_VENDOR_ID_INTEL, .device = 0x156c,
+ .vendor = PCI_VENDOR_ID_INTEL,
+ .device = PCI_DEVICE_ID_INTEL_CACTUS_RIDGE_4C,
+ .subvendor = 0x2222, .subdevice = 0x1111,
+ },
+ {
+ .class = PCI_CLASS_SYSTEM_OTHER << 8, .class_mask = ~0,
+ .vendor = PCI_VENDOR_ID_INTEL,
+ .device = PCI_DEVICE_ID_INTEL_FALCON_RIDGE_4C_NHI,
.subvendor = PCI_ANY_ID, .subdevice = PCI_ANY_ID,
},
{ 0,}
diff --git a/drivers/thunderbolt/switch.c b/drivers/thunderbolt/switch.c
index aeb9829..1e116f5 100644
--- a/drivers/thunderbolt/switch.c
+++ b/drivers/thunderbolt/switch.c
@@ -293,9 +293,9 @@
if (active) {
data = data & 0xFFFFFF83;
switch (sw->config.device_id) {
- case 0x1513:
- case 0x151a:
- case 0x1549:
+ case PCI_DEVICE_ID_INTEL_LIGHT_RIDGE:
+ case PCI_DEVICE_ID_INTEL_EAGLE_RIDGE:
+ case PCI_DEVICE_ID_INTEL_PORT_RIDGE:
break;
default:
data |= 4;
@@ -350,7 +350,7 @@
return NULL;
sw->tb = tb;
- if (tb_cfg_read(tb->ctl, &sw->config, route, 0, 2, 0, 5))
+ if (tb_cfg_read(tb->ctl, &sw->config, route, 0, TB_CFG_SWITCH, 0, 5))
goto err;
tb_info(tb,
"initializing Switch at %#llx (depth: %d, up port: %d)\n",
@@ -370,7 +370,9 @@
tb_sw_warn(sw, "unknown switch vendor id %#x\n",
sw->config.vendor_id);
- if (sw->config.device_id != 0x1547 && sw->config.device_id != 0x1549)
+ if (sw->config.device_id != PCI_DEVICE_ID_INTEL_LIGHT_RIDGE &&
+ sw->config.device_id != PCI_DEVICE_ID_INTEL_CACTUS_RIDGE_4C &&
+ sw->config.device_id != PCI_DEVICE_ID_INTEL_PORT_RIDGE)
tb_sw_warn(sw, "unsupported switch device id %#x\n",
sw->config.device_id);
@@ -425,9 +427,9 @@
}
/**
- * tb_sw_set_unpplugged() - set is_unplugged on switch and downstream switches
+ * tb_sw_set_unplugged() - set is_unplugged on switch and downstream switches
*/
-void tb_sw_set_unpplugged(struct tb_switch *sw)
+void tb_sw_set_unplugged(struct tb_switch *sw)
{
int i;
if (sw == sw->tb->root_switch) {
@@ -441,7 +443,7 @@
sw->is_unplugged = true;
for (i = 0; i <= sw->config.max_port_number; i++) {
if (!tb_is_upstream_port(&sw->ports[i]) && sw->ports[i].remote)
- tb_sw_set_unpplugged(sw->ports[i].remote->sw);
+ tb_sw_set_unplugged(sw->ports[i].remote->sw);
}
}
@@ -483,7 +485,7 @@
|| tb_switch_resume(port->remote->sw)) {
tb_port_warn(port,
"lost during suspend, disconnecting\n");
- tb_sw_set_unpplugged(port->remote->sw);
+ tb_sw_set_unplugged(port->remote->sw);
}
}
return 0;
diff --git a/drivers/thunderbolt/tb.c b/drivers/thunderbolt/tb.c
index d2c3fe3..24b6d30 100644
--- a/drivers/thunderbolt/tb.c
+++ b/drivers/thunderbolt/tb.c
@@ -246,7 +246,7 @@
if (ev->unplug) {
if (port->remote) {
tb_port_info(port, "unplugged\n");
- tb_sw_set_unpplugged(port->remote->sw);
+ tb_sw_set_unplugged(port->remote->sw);
tb_free_invalid_tunnels(tb);
tb_switch_free(port->remote->sw);
port->remote = NULL;
diff --git a/drivers/thunderbolt/tb.h b/drivers/thunderbolt/tb.h
index 8b0d7cf..61d57ba 100644
--- a/drivers/thunderbolt/tb.h
+++ b/drivers/thunderbolt/tb.h
@@ -226,7 +226,7 @@
void tb_switch_suspend(struct tb_switch *sw);
int tb_switch_resume(struct tb_switch *sw);
int tb_switch_reset(struct tb *tb, u64 route);
-void tb_sw_set_unpplugged(struct tb_switch *sw);
+void tb_sw_set_unplugged(struct tb_switch *sw);
struct tb_switch *get_switch_at_route(struct tb_switch *sw, u64 route);
int tb_wait_for_port(struct tb_port *port, bool wait_if_unplugged);
diff --git a/drivers/thunderbolt/tb_regs.h b/drivers/thunderbolt/tb_regs.h
index 6577af7..1e2a4a8 100644
--- a/drivers/thunderbolt/tb_regs.h
+++ b/drivers/thunderbolt/tb_regs.h
@@ -30,7 +30,7 @@
TB_CAP_I2C = 0x0005,
TB_CAP_PLUG_EVENTS = 0x0105, /* also EEPROM */
TB_CAP_TIME2 = 0x0305,
- TB_CAL_IECS = 0x0405,
+ TB_CAP_IECS = 0x0405,
TB_CAP_LINK_CONTROLLER = 0x0605, /* also IECS */
};
diff --git a/drivers/tty/amiserial.c b/drivers/tty/amiserial.c
index eacf4c9..208f573 100644
--- a/drivers/tty/amiserial.c
+++ b/drivers/tty/amiserial.c
@@ -398,7 +398,7 @@
wake_up_interruptible(&port->delta_msr_wait);
}
- if ((port->flags & ASYNC_CHECK_CD) && (dstatus & SER_DCD)) {
+ if (tty_port_check_carrier(port) && (dstatus & SER_DCD)) {
#if (defined(SERIAL_DEBUG_OPEN) || defined(SERIAL_DEBUG_INTR))
printk("ttyS%d CD now %s...", info->line,
(!(status & SER_DCD)) ? "on" : "off");
@@ -525,7 +525,7 @@
local_irq_save(flags);
- if (port->flags & ASYNC_INITIALIZED) {
+ if (tty_port_initialized(port)) {
free_page(page);
goto errout;
}
@@ -586,7 +586,7 @@
*/
change_speed(tty, info, NULL);
- port->flags |= ASYNC_INITIALIZED;
+ tty_port_set_initialized(port, 1);
local_irq_restore(flags);
return 0;
@@ -604,7 +604,7 @@
unsigned long flags;
struct serial_state *state;
- if (!(info->tport.flags & ASYNC_INITIALIZED))
+ if (!tty_port_initialized(&info->tport))
return;
state = info;
@@ -645,7 +645,7 @@
set_bit(TTY_IO_ERROR, &tty->flags);
- info->tport.flags &= ~ASYNC_INITIALIZED;
+ tty_port_set_initialized(&info->tport, 0);
local_irq_restore(flags);
}
@@ -727,17 +727,12 @@
info->IER &= ~UART_IER_MSI;
if (port->flags & ASYNC_HARDPPS_CD)
info->IER |= UART_IER_MSI;
- if (cflag & CRTSCTS) {
- port->flags |= ASYNC_CTS_FLOW;
+ tty_port_set_cts_flow(port, cflag & CRTSCTS);
+ if (cflag & CRTSCTS)
info->IER |= UART_IER_MSI;
- } else
- port->flags &= ~ASYNC_CTS_FLOW;
- if (cflag & CLOCAL)
- port->flags &= ~ASYNC_CHECK_CD;
- else {
- port->flags |= ASYNC_CHECK_CD;
+ tty_port_set_check_carrier(port, ~cflag & CLOCAL);
+ if (~cflag & CLOCAL)
info->IER |= UART_IER_MSI;
- }
/* TBD:
* Does clearing IER_MSI imply that we should disable the VBL interrupt ?
*/
@@ -1089,7 +1084,7 @@
port->low_latency = (port->flags & ASYNC_LOW_LATENCY) ? 1 : 0;
check_and_exit:
- if (port->flags & ASYNC_INITIALIZED) {
+ if (tty_port_initialized(port)) {
if (change_spd) {
if ((port->flags & ASYNC_SPD_MASK) == ASYNC_SPD_HI)
tty->alt_speed = 57600;
@@ -1143,7 +1138,7 @@
if (serial_paranoia_check(info, tty->name, "rs_ioctl"))
return -ENODEV;
- if (tty->flags & (1 << TTY_IO_ERROR))
+ if (tty_io_error(tty))
return -EIO;
control = info->MCR;
@@ -1165,7 +1160,7 @@
if (serial_paranoia_check(info, tty->name, "rs_ioctl"))
return -ENODEV;
- if (tty->flags & (1 << TTY_IO_ERROR))
+ if (tty_io_error(tty))
return -EIO;
local_irq_save(flags);
@@ -1250,7 +1245,7 @@
if ((cmd != TIOCGSERIAL) && (cmd != TIOCSSERIAL) &&
(cmd != TIOCSERCONFIG) && (cmd != TIOCSERGSTRUCT) &&
(cmd != TIOCMIWAIT) && (cmd != TIOCGICOUNT)) {
- if (tty->flags & (1 << TTY_IO_ERROR))
+ if (tty_io_error(tty))
return -EIO;
}
@@ -1342,7 +1337,7 @@
/* Handle transition away from B0 status */
if (!(old_termios->c_cflag & CBAUD) && (cflag & CBAUD)) {
info->MCR |= SER_DTR;
- if (!C_CRTSCTS(tty) || !test_bit(TTY_THROTTLED, &tty->flags))
+ if (!C_CRTSCTS(tty) || !tty_throttled(tty))
info->MCR |= SER_RTS;
local_irq_save(flags);
rtsdtr_ctrl(info->MCR);
@@ -1395,7 +1390,7 @@
* line status register.
*/
state->read_status_mask &= ~UART_LSR_DR;
- if (port->flags & ASYNC_INITIALIZED) {
+ if (tty_port_initialized(port)) {
/* disable receive interrupts */
custom.intena = IF_RBF;
mb();
@@ -1495,7 +1490,7 @@
rs_flush_buffer(tty);
shutdown(tty, info);
info->tport.count = 0;
- info->tport.flags &= ~ASYNC_NORMAL_ACTIVE;
+ tty_port_set_active(&info->tport, 0);
info->tport.tty = NULL;
wake_up_interruptible(&info->tport.open_wait);
}
@@ -1543,7 +1538,7 @@
local_irq_save(flags);
status = ciab.pra;
- control = (state->tport.flags & ASYNC_INITIALIZED) ? state->MCR : status;
+ control = tty_port_initialized(&state->tport) ? state->MCR : status;
local_irq_restore(flags);
stat_buf[0] = 0;
diff --git a/drivers/tty/cyclades.c b/drivers/tty/cyclades.c
index d67e542..3840d6b 100644
--- a/drivers/tty/cyclades.c
+++ b/drivers/tty/cyclades.c
@@ -714,7 +714,7 @@
wake_up_interruptible(&info->port.delta_msr_wait);
}
- if ((mdm_change & CyDCD) && (info->port.flags & ASYNC_CHECK_CD)) {
+ if ((mdm_change & CyDCD) && tty_port_check_carrier(&info->port)) {
if (mdm_status & CyDCD)
wake_up_interruptible(&info->port.open_wait);
else
@@ -1119,7 +1119,7 @@
case C_CM_MDCD:
info->icount.dcd++;
delta_count++;
- if (info->port.flags & ASYNC_CHECK_CD) {
+ if (tty_port_check_carrier(&info->port)) {
u32 dcd = fw_ver > 241 ? param :
readl(&info->u.cyz.ch_ctrl->rs_status);
if (dcd & C_RS_DCD)
@@ -1279,7 +1279,7 @@
spin_lock_irqsave(&card->card_lock, flags);
- if (info->port.flags & ASYNC_INITIALIZED)
+ if (tty_port_initialized(&info->port))
goto errout;
if (!info->type) {
@@ -1364,7 +1364,7 @@
/* enable send, recv, modem !!! */
}
- info->port.flags |= ASYNC_INITIALIZED;
+ tty_port_set_initialized(&info->port, 1);
clear_bit(TTY_IO_ERROR, &tty->flags);
info->xmit_cnt = info->xmit_head = info->xmit_tail = 0;
@@ -1424,7 +1424,7 @@
struct cyclades_card *card;
unsigned long flags;
- if (!(info->port.flags & ASYNC_INITIALIZED))
+ if (!tty_port_initialized(&info->port))
return;
card = info->card;
@@ -1448,7 +1448,7 @@
some later date (after testing)!!! */
set_bit(TTY_IO_ERROR, &tty->flags);
- info->port.flags &= ~ASYNC_INITIALIZED;
+ tty_port_set_initialized(&info->port, 0);
spin_unlock_irqrestore(&card->card_lock, flags);
} else {
#ifdef CY_DEBUG_OPEN
@@ -1473,7 +1473,7 @@
tty_port_lower_dtr_rts(&info->port);
set_bit(TTY_IO_ERROR, &tty->flags);
- info->port.flags &= ~ASYNC_INITIALIZED;
+ tty_port_set_initialized(&info->port, 0);
spin_unlock_irqrestore(&card->card_lock, flags);
}
@@ -1711,7 +1711,7 @@
/* Stop accepting input */
cyy_writeb(info, CyCAR, channel & 0x03);
cyy_writeb(info, CySRER, cyy_readb(info, CySRER) & ~CyRxData);
- if (info->port.flags & ASYNC_INITIALIZED) {
+ if (tty_port_initialized(&info->port)) {
/* Waiting for on-board buffers to be empty before
closing the port */
spin_unlock_irqrestore(&card->card_lock, flags);
@@ -2083,17 +2083,12 @@
info->cor1 |= CyPARITY_NONE;
/* CTS flow control flag */
- if (cflag & CRTSCTS) {
- info->port.flags |= ASYNC_CTS_FLOW;
+ tty_port_set_cts_flow(&info->port, cflag & CRTSCTS);
+ if (cflag & CRTSCTS)
info->cor2 |= CyCtsAE;
- } else {
- info->port.flags &= ~ASYNC_CTS_FLOW;
- info->cor2 &= ~CyCtsAE;
- }
- if (cflag & CLOCAL)
- info->port.flags &= ~ASYNC_CHECK_CD;
else
- info->port.flags |= ASYNC_CHECK_CD;
+ info->cor2 &= ~CyCtsAE;
+ tty_port_set_check_carrier(&info->port, ~cflag & CLOCAL);
/***********************************************
The hardware option, CyRtsAO, presents RTS when
@@ -2234,7 +2229,7 @@
}
/* As the HW flow control is done in firmware, the driver
doesn't need to care about it */
- info->port.flags &= ~ASYNC_CTS_FLOW;
+ tty_port_set_cts_flow(&info->port, 0);
/* XON/XOFF/XANY flow control flags */
sw_flow = 0;
@@ -2252,10 +2247,7 @@
}
/* CD sensitivity */
- if (cflag & CLOCAL)
- info->port.flags &= ~ASYNC_CHECK_CD;
- else
- info->port.flags |= ASYNC_CHECK_CD;
+ tty_port_set_check_carrier(&info->port, ~cflag & CLOCAL);
if (baud == 0) { /* baud rate is zero, turn off line */
cy_writel(&ch_ctrl->rs_control,
@@ -2342,7 +2334,7 @@
info->port.closing_wait = new_serial.closing_wait * HZ / 100;
check_and_exit:
- if (info->port.flags & ASYNC_INITIALIZED) {
+ if (tty_port_initialized(&info->port)) {
cy_set_line_char(info, tty);
ret = 0;
} else {
diff --git a/drivers/tty/hvc/hvc_console.c b/drivers/tty/hvc/hvc_console.c
index e46d628..ce86487 100644
--- a/drivers/tty/hvc/hvc_console.c
+++ b/drivers/tty/hvc/hvc_console.c
@@ -632,7 +632,7 @@
goto bail;
/* Now check if we can get data (are we throttled ?) */
- if (test_bit(TTY_THROTTLED, &tty->flags))
+ if (tty_throttled(tty))
goto throttled;
/* If we aren't notifier driven and aren't throttled, we always
@@ -814,7 +814,7 @@
n = hp->ops->get_chars(hp->vtermno, &ch, 1);
- if (n == 0)
+ if (n <= 0)
return NO_POLL_CHAR;
return ch;
diff --git a/drivers/tty/hvc/hvcs.c b/drivers/tty/hvc/hvcs.c
index 5997b17..3c4d7c2 100644
--- a/drivers/tty/hvc/hvcs.c
+++ b/drivers/tty/hvc/hvcs.c
@@ -600,7 +600,7 @@
hvcs_try_write(hvcsd);
- if (!tty || test_bit(TTY_THROTTLED, &tty->flags)) {
+ if (!tty || tty_throttled(tty)) {
hvcsd->todo_mask &= ~(HVCS_READ_MASK);
goto bail;
} else if (!(hvcsd->todo_mask & (HVCS_READ_MASK)))
diff --git a/drivers/tty/hvc/hvsi.c b/drivers/tty/hvc/hvsi.c
index a75146f..96ce6bd 100644
--- a/drivers/tty/hvc/hvsi.c
+++ b/drivers/tty/hvc/hvsi.c
@@ -509,7 +509,7 @@
}
spin_lock_irqsave(&hp->lock, flags);
- if (tty && hp->n_throttle && !test_bit(TTY_THROTTLED, &tty->flags)) {
+ if (tty && hp->n_throttle && !tty_throttled(tty)) {
/* we weren't hung up and we weren't throttled, so we can
* deliver the rest now */
hvsi_send_overflow(hp);
diff --git a/drivers/tty/ipwireless/hardware.c b/drivers/tty/ipwireless/hardware.c
index ad7031a..df0204b 100644
--- a/drivers/tty/ipwireless/hardware.c
+++ b/drivers/tty/ipwireless/hardware.c
@@ -1572,6 +1572,11 @@
sizeof(struct ipw_setup_reboot_msg_ack),
ADDR_SETUP_PROT, TL_PROTOCOLID_SETUP,
TL_SETUP_SIGNO_REBOOT_MSG_ACK);
+ if (!packet) {
+ pr_err(IPWIRELESS_PCCARD_NAME
+ ": Not enough memory to send reboot packet");
+ break;
+ }
packet->header.length =
sizeof(struct TlSetupRebootMsgAck);
send_packet(hw, PRIO_SETUP, &packet->header);
diff --git a/drivers/tty/isicom.c b/drivers/tty/isicom.c
index 8bf6763..b70187b 100644
--- a/drivers/tty/isicom.c
+++ b/drivers/tty/isicom.c
@@ -438,8 +438,8 @@
for (; count > 0; count--, port++) {
/* port not active or tx disabled to force flow control */
- if (!(port->port.flags & ASYNC_INITIALIZED) ||
- !(port->status & ISI_TXOK))
+ if (!tty_port_initialized(&port->port) ||
+ !(port->status & ISI_TXOK))
continue;
txcount = min_t(short, TX_SIZE, port->xmit_cnt);
@@ -553,7 +553,7 @@
return IRQ_HANDLED;
}
port = card->ports + channel;
- if (!(port->port.flags & ASYNC_INITIALIZED)) {
+ if (!tty_port_initialized(&port->port)) {
outw(0x0000, base+0x04); /* enable interrupts */
spin_unlock(&card->card_lock);
return IRQ_HANDLED;
@@ -577,7 +577,7 @@
header = inw(base);
switch (header & 0xff) {
case 0: /* Change in EIA signals */
- if (port->port.flags & ASYNC_CHECK_CD) {
+ if (tty_port_check_carrier(&port->port)) {
if (port->status & ISI_DCD) {
if (!(header & ISI_DCD)) {
/* Carrier has been lost */
@@ -758,18 +758,13 @@
outw(channel_setup, base);
InterruptTheCard(base);
}
- if (C_CLOCAL(tty))
- port->port.flags &= ~ASYNC_CHECK_CD;
- else
- port->port.flags |= ASYNC_CHECK_CD;
+ tty_port_set_check_carrier(&port->port, !C_CLOCAL(tty));
/* flow control settings ...*/
flow_ctrl = 0;
- port->port.flags &= ~ASYNC_CTS_FLOW;
- if (C_CRTSCTS(tty)) {
- port->port.flags |= ASYNC_CTS_FLOW;
+ tty_port_set_cts_flow(&port->port, C_CRTSCTS(tty));
+ if (C_CRTSCTS(tty))
flow_ctrl |= ISICOM_CTSRTS;
- }
if (I_IXON(tty))
flow_ctrl |= ISICOM_RESPOND_XONXOFF;
if (I_IXOFF(tty))
diff --git a/drivers/tty/moxa.c b/drivers/tty/moxa.c
index 92982d7..60d37b2 100644
--- a/drivers/tty/moxa.c
+++ b/drivers/tty/moxa.c
@@ -912,7 +912,7 @@
/* pci hot-un-plug support */
for (a = 0; a < brd->numPorts; a++)
- if (brd->ports[a].port.flags & ASYNC_INITIALIZED)
+ if (tty_port_initialized(&brd->ports[a].port))
tty_port_tty_hangup(&brd->ports[a].port, false);
for (a = 0; a < MAX_PORTS_PER_BOARD; a++)
@@ -921,7 +921,7 @@
while (1) {
opened = 0;
for (a = 0; a < brd->numPorts; a++)
- if (brd->ports[a].port.flags & ASYNC_INITIALIZED)
+ if (tty_port_initialized(&brd->ports[a].port))
opened++;
mutex_unlock(&moxa_openlock);
if (!opened)
@@ -1192,13 +1192,13 @@
tty->driver_data = ch;
tty_port_tty_set(&ch->port, tty);
mutex_lock(&ch->port.mutex);
- if (!(ch->port.flags & ASYNC_INITIALIZED)) {
+ if (!tty_port_initialized(&ch->port)) {
ch->statusflags = 0;
moxa_set_tty_param(tty, &tty->termios);
MoxaPortLineCtrl(ch, 1, 1);
MoxaPortEnable(ch);
MoxaSetFifo(ch, ch->type == PORT_16550A);
- ch->port.flags |= ASYNC_INITIALIZED;
+ tty_port_set_initialized(&ch->port, 1);
}
mutex_unlock(&ch->port.mutex);
mutex_unlock(&moxa_openlock);
@@ -1379,7 +1379,7 @@
{
struct tty_struct *tty = tty_port_tty_get(&p->port);
void __iomem *ofsAddr;
- unsigned int inited = p->port.flags & ASYNC_INITIALIZED;
+ unsigned int inited = tty_port_initialized(&p->port);
u16 intr;
if (tty) {
@@ -1394,7 +1394,7 @@
tty_wakeup(tty);
}
- if (inited && !test_bit(TTY_THROTTLED, &tty->flags) &&
+ if (inited && !tty_throttled(tty) &&
MoxaPortRxQueue(p) > 0) { /* RX */
MoxaPortReadData(p);
tty_schedule_flip(&p->port);
diff --git a/drivers/tty/mxser.c b/drivers/tty/mxser.c
index 2f12bb9..98d2bd1 100644
--- a/drivers/tty/mxser.c
+++ b/drivers/tty/mxser.c
@@ -711,8 +711,8 @@
/* CTS flow control flag and modem status interrupts */
info->IER &= ~UART_IER_MSI;
info->MCR &= ~UART_MCR_AFE;
+ tty_port_set_cts_flow(&info->port, cflag & CRTSCTS);
if (cflag & CRTSCTS) {
- info->port.flags |= ASYNC_CTS_FLOW;
info->IER |= UART_IER_MSI;
if ((info->type == PORT_16550A) || (info->board->chip_flag)) {
info->MCR |= UART_MCR_AFE;
@@ -744,16 +744,11 @@
}
}
}
- } else {
- info->port.flags &= ~ASYNC_CTS_FLOW;
}
outb(info->MCR, info->ioaddr + UART_MCR);
- if (cflag & CLOCAL) {
- info->port.flags &= ~ASYNC_CHECK_CD;
- } else {
- info->port.flags |= ASYNC_CHECK_CD;
+ tty_port_set_check_carrier(&info->port, ~cflag & CLOCAL);
+ if (~cflag & CLOCAL)
info->IER |= UART_IER_MSI;
- }
outb(info->IER, info->ioaddr + UART_IER);
/*
@@ -826,7 +821,7 @@
port->mon_data.modem_status = status;
wake_up_interruptible(&port->port.delta_msr_wait);
- if ((port->port.flags & ASYNC_CHECK_CD) && (status & UART_MSR_DDCD)) {
+ if (tty_port_check_carrier(&port->port) && (status & UART_MSR_DDCD)) {
if (status & UART_MSR_DCD)
wake_up_interruptible(&port->port.open_wait);
}
@@ -1086,12 +1081,10 @@
mutex_lock(&port->mutex);
mxser_close_port(port);
mxser_flush_buffer(tty);
- if (test_bit(ASYNCB_INITIALIZED, &port->flags)) {
- if (C_HUPCL(tty))
- tty_port_lower_dtr_rts(port);
- }
+ if (tty_port_initialized(port) && C_HUPCL(tty))
+ tty_port_lower_dtr_rts(port);
mxser_shutdown_port(port);
- clear_bit(ASYNCB_INITIALIZED, &port->flags);
+ tty_port_set_initialized(port, 0);
mutex_unlock(&port->mutex);
info->closing = 0;
/* Right now the tty_port set is done outside of the close_end helper
@@ -1287,7 +1280,7 @@
process_txrx_fifo(info);
- if (test_bit(ASYNCB_INITIALIZED, &port->flags)) {
+ if (tty_port_initialized(port)) {
if (flags != (port->flags & ASYNC_SPD_MASK)) {
spin_lock_irqsave(&info->slock, sl_flags);
mxser_change_speed(tty, NULL);
@@ -1296,7 +1289,7 @@
} else {
retval = mxser_activate(port, tty);
if (retval == 0)
- set_bit(ASYNCB_INITIALIZED, &port->flags);
+ tty_port_set_initialized(port, 1);
}
return retval;
}
@@ -1334,7 +1327,7 @@
if (tty->index == MXSER_PORTS)
return -ENOIOCTLCMD;
- if (test_bit(TTY_IO_ERROR, &tty->flags))
+ if (tty_io_error(tty))
return -EIO;
control = info->MCR;
@@ -1361,7 +1354,7 @@
if (tty->index == MXSER_PORTS)
return -ENOIOCTLCMD;
- if (test_bit(TTY_IO_ERROR, &tty->flags))
+ if (tty_io_error(tty))
return -EIO;
spin_lock_irqsave(&info->slock, flags);
@@ -1715,8 +1708,7 @@
return 0;
}
- if (cmd != TIOCGSERIAL && cmd != TIOCMIWAIT &&
- test_bit(TTY_IO_ERROR, &tty->flags))
+ if (cmd != TIOCGSERIAL && cmd != TIOCMIWAIT && tty_io_error(tty))
return -EIO;
switch (cmd) {
@@ -2257,7 +2249,7 @@
iir &= MOXA_MUST_IIR_MASK;
tty = tty_port_tty_get(&port->port);
if (!tty || port->closing ||
- !(port->port.flags & ASYNC_INITIALIZED)) {
+ !tty_port_initialized(&port->port)) {
status = inb(port->ioaddr + UART_LSR);
outb(0x27, port->ioaddr + UART_FCR);
inb(port->ioaddr + UART_MSR);
@@ -2400,7 +2392,6 @@
if (brd->chip_flag != MOXA_OTHER_UART)
mxser_enable_must_enchance_mode(info->ioaddr);
- info->port.flags = ASYNC_SHARE_IRQ;
info->type = brd->uart_type;
process_txrx_fifo(info);
diff --git a/drivers/tty/n_gsm.c b/drivers/tty/n_gsm.c
index 0c27a00..54cab59 100644
--- a/drivers/tty/n_gsm.c
+++ b/drivers/tty/n_gsm.c
@@ -2045,7 +2045,9 @@
}
}
spin_unlock(&gsm_mux_lock);
- WARN_ON(i == MAX_MUX);
+ /* open failed before registering => nothing to do */
+ if (i == MAX_MUX)
+ return;
/* In theory disconnecting DLCI 0 is sufficient but for some
modems this is apparently not the case. */
@@ -2947,7 +2949,7 @@
dlci->modem_rx = 0;
/* We could in theory open and close before we wait - eg if we get
a DM straight back. This is ok as that will have caused a hangup */
- set_bit(ASYNCB_INITIALIZED, &port->flags);
+ tty_port_set_initialized(port, 1);
/* Start sending off SABM messages */
gsm_dlci_begin_open(dlci);
/* And wait for virtual carrier */
@@ -2970,10 +2972,8 @@
if (tty_port_close_start(&dlci->port, tty, filp) == 0)
return;
gsm_dlci_begin_close(dlci);
- if (test_bit(ASYNCB_INITIALIZED, &dlci->port.flags)) {
- if (C_HUPCL(tty))
- tty_port_lower_dtr_rts(&dlci->port);
- }
+ if (tty_port_initialized(&dlci->port) && C_HUPCL(tty))
+ tty_port_lower_dtr_rts(&dlci->port);
tty_port_close_end(&dlci->port, tty);
tty_port_tty_set(&dlci->port, NULL);
return;
diff --git a/drivers/tty/n_hdlc.c b/drivers/tty/n_hdlc.c
index bcaba17..a7fa016 100644
--- a/drivers/tty/n_hdlc.c
+++ b/drivers/tty/n_hdlc.c
@@ -599,7 +599,7 @@
add_wait_queue(&tty->read_wait, &wait);
for (;;) {
- if (test_bit(TTY_OTHER_DONE, &tty->flags)) {
+ if (test_bit(TTY_OTHER_CLOSED, &tty->flags)) {
ret = -EIO;
break;
}
@@ -827,7 +827,7 @@
/* set bits for operations that won't block */
if (n_hdlc->rx_buf_list.head)
mask |= POLLIN | POLLRDNORM; /* readable */
- if (test_bit(TTY_OTHER_DONE, &tty->flags))
+ if (test_bit(TTY_OTHER_CLOSED, &tty->flags))
mask |= POLLHUP;
if (tty_hung_up_p(filp))
mask |= POLLHUP;
diff --git a/drivers/tty/n_tty.c b/drivers/tty/n_tty.c
index fb76a7d..bdf0e6e 100644
--- a/drivers/tty/n_tty.c
+++ b/drivers/tty/n_tty.c
@@ -1917,18 +1917,6 @@
return ldata->commit_head - ldata->read_tail >= amt;
}
-static inline int check_other_done(struct tty_struct *tty)
-{
- int done = test_bit(TTY_OTHER_DONE, &tty->flags);
- if (done) {
- /* paired with cmpxchg() in check_other_closed(); ensures
- * read buffer head index is not stale
- */
- smp_mb__after_atomic();
- }
- return done;
-}
-
/**
* copy_from_read_buf - copy read data directly
* @tty: terminal device
@@ -2124,7 +2112,7 @@
struct n_tty_data *ldata = tty->disc_data;
unsigned char __user *b = buf;
DEFINE_WAIT_FUNC(wait, woken_wake_function);
- int c, done;
+ int c;
int minimum, time;
ssize_t retval = 0;
long timeout;
@@ -2183,32 +2171,35 @@
break;
}
- done = check_other_done(tty);
-
if (!input_available_p(tty, 0)) {
- if (done) {
- retval = -EIO;
- break;
- }
- if (tty_hung_up_p(file))
- break;
- if (!timeout)
- break;
- if (file->f_flags & O_NONBLOCK) {
- retval = -EAGAIN;
- break;
- }
- if (signal_pending(current)) {
- retval = -ERESTARTSYS;
- break;
- }
up_read(&tty->termios_rwsem);
-
- timeout = wait_woken(&wait, TASK_INTERRUPTIBLE,
- timeout);
-
+ tty_buffer_flush_work(tty->port);
down_read(&tty->termios_rwsem);
- continue;
+ if (!input_available_p(tty, 0)) {
+ if (test_bit(TTY_OTHER_CLOSED, &tty->flags)) {
+ retval = -EIO;
+ break;
+ }
+ if (tty_hung_up_p(file))
+ break;
+ if (!timeout)
+ break;
+ if (file->f_flags & O_NONBLOCK) {
+ retval = -EAGAIN;
+ break;
+ }
+ if (signal_pending(current)) {
+ retval = -ERESTARTSYS;
+ break;
+ }
+ up_read(&tty->termios_rwsem);
+
+ timeout = wait_woken(&wait, TASK_INTERRUPTIBLE,
+ timeout);
+
+ down_read(&tty->termios_rwsem);
+ continue;
+ }
}
if (ldata->icanon && !L_EXTPROC(tty)) {
@@ -2386,12 +2377,17 @@
poll_wait(file, &tty->read_wait, wait);
poll_wait(file, &tty->write_wait, wait);
- if (check_other_done(tty))
- mask |= POLLHUP;
if (input_available_p(tty, 1))
mask |= POLLIN | POLLRDNORM;
+ else {
+ tty_buffer_flush_work(tty->port);
+ if (input_available_p(tty, 1))
+ mask |= POLLIN | POLLRDNORM;
+ }
if (tty->packet && tty->link->ctrl_status)
mask |= POLLPRI | POLLIN | POLLRDNORM;
+ if (test_bit(TTY_OTHER_CLOSED, &tty->flags))
+ mask |= POLLHUP;
if (tty_hung_up_p(file))
mask |= POLLHUP;
if (tty->ops->write && !tty_is_writelocked(tty) &&
diff --git a/drivers/tty/nozomi.c b/drivers/tty/nozomi.c
index 5cc80b8..d6fd0e8 100644
--- a/drivers/tty/nozomi.c
+++ b/drivers/tty/nozomi.c
@@ -826,7 +826,7 @@
size = __le32_to_cpu(readl(addr));
/* DBG1( "%d bytes port: %d", size, index); */
- if (tty && test_bit(TTY_THROTTLED, &tty->flags)) {
+ if (tty && tty_throttled(tty)) {
DBG1("No room in tty, don't read data, don't ack interrupt, "
"disable interrupt");
diff --git a/drivers/tty/pty.c b/drivers/tty/pty.c
index cf0dc51..dd4b841 100644
--- a/drivers/tty/pty.c
+++ b/drivers/tty/pty.c
@@ -44,7 +44,7 @@
if (tty->driver->subtype == PTY_TYPE_MASTER)
WARN_ON(tty->count > 1);
else {
- if (test_bit(TTY_IO_ERROR, &tty->flags))
+ if (tty_io_error(tty))
return;
if (tty->count > 2)
return;
@@ -59,7 +59,7 @@
if (!tty->link)
return;
set_bit(TTY_OTHER_CLOSED, &tty->link->flags);
- tty_flip_buffer_push(tty->link->port);
+ wake_up_interruptible(&tty->link->read_wait);
wake_up_interruptible(&tty->link->write_wait);
if (tty->driver->subtype == PTY_TYPE_MASTER) {
set_bit(TTY_OTHER_CLOSED, &tty->flags);
@@ -247,9 +247,7 @@
goto out;
clear_bit(TTY_IO_ERROR, &tty->flags);
- /* TTY_OTHER_CLOSED must be cleared before TTY_OTHER_DONE */
clear_bit(TTY_OTHER_CLOSED, &tty->link->flags);
- clear_bit(TTY_OTHER_DONE, &tty->link->flags);
set_bit(TTY_THROTTLED, &tty->flags);
return 0;
diff --git a/drivers/tty/rocket.c b/drivers/tty/rocket.c
index 0b802cd..b0cc47c 100644
--- a/drivers/tty/rocket.c
+++ b/drivers/tty/rocket.c
@@ -495,7 +495,7 @@
if (!info)
return;
- if ((info->port.flags & ASYNC_INITIALIZED) == 0) {
+ if (!tty_port_initialized(&info->port)) {
printk(KERN_WARNING "rp: WARNING: rp_handle_port called with "
"info->flags & NOT_INIT\n");
return;
@@ -615,7 +615,8 @@
* the board.
* Inputs: board, aiop, chan numbers
*/
-static void init_r_port(int board, int aiop, int chan, struct pci_dev *pci_dev)
+static void __init
+init_r_port(int board, int aiop, int chan, struct pci_dev *pci_dev)
{
unsigned rocketMode;
struct r_port *info;
@@ -920,7 +921,7 @@
/*
* Info->count is now 1; so it's safe to sleep now.
*/
- if (!test_bit(ASYNCB_INITIALIZED, &port->flags)) {
+ if (!tty_port_initialized(port)) {
cp = &info->channel;
sSetRxTrigger(cp, TRIG_1);
if (sGetChanStatus(cp) & CD_ACT)
@@ -944,7 +945,7 @@
sEnRxFIFO(cp);
sEnTransmit(cp);
- set_bit(ASYNCB_INITIALIZED, &info->port.flags);
+ tty_port_set_initialized(&info->port, 1);
/*
* Set up the tty->alt_speed kludge
@@ -1042,9 +1043,10 @@
}
}
spin_lock_irq(&port->lock);
- info->port.flags &= ~(ASYNC_INITIALIZED | ASYNC_NORMAL_ACTIVE);
tty->closing = 0;
spin_unlock_irq(&port->lock);
+ tty_port_set_initialized(port, 0);
+ tty_port_set_active(port, 0);
mutex_unlock(&port->mutex);
tty_port_tty_set(port, NULL);
@@ -1512,7 +1514,7 @@
sDisCTSFlowCtl(cp);
sDisTxSoftFlowCtl(cp);
sClrTxXOFF(cp);
- clear_bit(ASYNCB_INITIALIZED, &info->port.flags);
+ tty_port_set_initialized(&info->port, 0);
wake_up_interruptible(&info->port.open_wait);
}
@@ -1624,7 +1626,7 @@
/* Write remaining data into the port's xmit_buf */
while (1) {
/* Hung up ? */
- if (!test_bit(ASYNCB_NORMAL_ACTIVE, &info->port.flags))
+ if (!tty_port_active(&info->port))
goto end;
c = min(count, XMIT_BUF_SIZE - info->xmit_cnt - 1);
c = min(c, XMIT_BUF_SIZE - info->xmit_head);
diff --git a/drivers/tty/serial/8250/8250.h b/drivers/tty/serial/8250/8250.h
index 047a7ba..215a992 100644
--- a/drivers/tty/serial/8250/8250.h
+++ b/drivers/tty/serial/8250/8250.h
@@ -17,7 +17,7 @@
struct uart_8250_dma {
int (*tx_dma)(struct uart_8250_port *p);
- int (*rx_dma)(struct uart_8250_port *p, unsigned int iir);
+ int (*rx_dma)(struct uart_8250_port *p);
/* Filter function */
dma_filter_fn fn;
@@ -84,7 +84,6 @@
#define UART_BUG_THRE (1 << 3) /* UART has buggy THRE reassertion */
#define UART_BUG_PARITY (1 << 4) /* UART mishandles parity if FIFO enabled */
-#define HIGH_BITS_OFFSET ((sizeof(long)-sizeof(int))*8)
#ifdef CONFIG_SERIAL_8250_SHARE_IRQ
#define SERIAL8250_SHARE_IRQS 1
@@ -151,6 +150,12 @@
static inline void serial8250_pnp_exit(void) { }
#endif
+#ifdef CONFIG_SERIAL_8250_FINTEK
+int fintek_8250_probe(struct uart_8250_port *uart);
+#else
+static inline int fintek_8250_probe(struct uart_8250_port *uart) { return 0; }
+#endif
+
#ifdef CONFIG_ARCH_OMAP1
static inline int is_omap1_8250(struct uart_8250_port *pt)
{
@@ -190,7 +195,8 @@
#ifdef CONFIG_SERIAL_8250_DMA
extern int serial8250_tx_dma(struct uart_8250_port *);
-extern int serial8250_rx_dma(struct uart_8250_port *, unsigned int iir);
+extern int serial8250_rx_dma(struct uart_8250_port *);
+extern void serial8250_rx_dma_flush(struct uart_8250_port *);
extern int serial8250_request_dma(struct uart_8250_port *);
extern void serial8250_release_dma(struct uart_8250_port *);
#else
@@ -198,10 +204,11 @@
{
return -1;
}
-static inline int serial8250_rx_dma(struct uart_8250_port *p, unsigned int iir)
+static inline int serial8250_rx_dma(struct uart_8250_port *p)
{
return -1;
}
+static inline void serial8250_rx_dma_flush(struct uart_8250_port *p) { }
static inline int serial8250_request_dma(struct uart_8250_port *p)
{
return -1;
diff --git a/drivers/tty/serial/8250/8250_core.c b/drivers/tty/serial/8250/8250_core.c
index 2f4f5ee..0fbd7c0 100644
--- a/drivers/tty/serial/8250/8250_core.c
+++ b/drivers/tty/serial/8250/8250_core.c
@@ -830,6 +830,7 @@
uart.port.handle_irq = p->handle_irq;
uart.port.handle_break = p->handle_break;
uart.port.set_termios = p->set_termios;
+ uart.port.get_mctrl = p->get_mctrl;
uart.port.pm = p->pm;
uart.port.dev = &dev->dev;
uart.port.irqflags |= irqflag;
@@ -1022,6 +1023,8 @@
/* Possibly override set_termios call */
if (up->port.set_termios)
uart->port.set_termios = up->port.set_termios;
+ if (up->port.get_mctrl)
+ uart->port.get_mctrl = up->port.get_mctrl;
if (up->port.set_mctrl)
uart->port.set_mctrl = up->port.set_mctrl;
if (up->port.startup)
diff --git a/drivers/tty/serial/8250/8250_dma.c b/drivers/tty/serial/8250/8250_dma.c
index 78259d3..7f33d1c 100644
--- a/drivers/tty/serial/8250/8250_dma.c
+++ b/drivers/tty/serial/8250/8250_dma.c
@@ -110,30 +110,11 @@
return ret;
}
-int serial8250_rx_dma(struct uart_8250_port *p, unsigned int iir)
+int serial8250_rx_dma(struct uart_8250_port *p)
{
struct uart_8250_dma *dma = p->dma;
struct dma_async_tx_descriptor *desc;
- switch (iir & 0x3f) {
- case UART_IIR_RLSI:
- /* 8250_core handles errors and break interrupts */
- return -EIO;
- case UART_IIR_RX_TIMEOUT:
- /*
- * If RCVR FIFO trigger level was not reached, complete the
- * transfer and let 8250_core copy the remaining data.
- */
- if (dma->rx_running) {
- dmaengine_pause(dma->rxchan);
- __dma_rx_complete(p);
- dmaengine_terminate_all(dma->rxchan);
- }
- return -ETIMEDOUT;
- default:
- break;
- }
-
if (dma->rx_running)
return 0;
@@ -154,10 +135,23 @@
return 0;
}
+void serial8250_rx_dma_flush(struct uart_8250_port *p)
+{
+ struct uart_8250_dma *dma = p->dma;
+
+ if (dma->rx_running) {
+ dmaengine_pause(dma->rxchan);
+ __dma_rx_complete(p);
+ dmaengine_terminate_all(dma->rxchan);
+ }
+}
+
int serial8250_request_dma(struct uart_8250_port *p)
{
struct uart_8250_dma *dma = p->dma;
dma_cap_mask_t mask;
+ struct dma_slave_caps caps;
+ int ret;
/* Default slave configuration parameters */
dma->rxconf.direction = DMA_DEV_TO_MEM;
@@ -178,6 +172,16 @@
if (!dma->rxchan)
return -ENODEV;
+ /* 8250 rx dma requires dmaengine driver to support pause/terminate */
+ ret = dma_get_slave_caps(dma->rxchan, &caps);
+ if (ret)
+ goto release_rx;
+ if (!caps.cmd_pause || !caps.cmd_terminate ||
+ caps.residue_granularity == DMA_RESIDUE_GRANULARITY_DESCRIPTOR) {
+ ret = -EINVAL;
+ goto release_rx;
+ }
+
dmaengine_slave_config(dma->rxchan, &dma->rxconf);
/* Get a channel for TX */
@@ -185,8 +189,17 @@
dma->fn, dma->tx_param,
p->port.dev, "tx");
if (!dma->txchan) {
- dma_release_channel(dma->rxchan);
- return -ENODEV;
+ ret = -ENODEV;
+ goto release_rx;
+ }
+
+ /* 8250 tx dma requires dmaengine driver to support terminate */
+ ret = dma_get_slave_caps(dma->txchan, &caps);
+ if (ret)
+ goto err;
+ if (!caps.cmd_terminate) {
+ ret = -EINVAL;
+ goto err;
}
dmaengine_slave_config(dma->txchan, &dma->txconf);
@@ -197,8 +210,10 @@
dma->rx_buf = dma_alloc_coherent(dma->rxchan->device->dev, dma->rx_size,
&dma->rx_addr, GFP_KERNEL);
- if (!dma->rx_buf)
+ if (!dma->rx_buf) {
+ ret = -ENOMEM;
goto err;
+ }
/* TX buffer */
dma->tx_addr = dma_map_single(dma->txchan->device->dev,
@@ -208,6 +223,7 @@
if (dma_mapping_error(dma->txchan->device->dev, dma->tx_addr)) {
dma_free_coherent(dma->rxchan->device->dev, dma->rx_size,
dma->rx_buf, dma->rx_addr);
+ ret = -ENOMEM;
goto err;
}
@@ -215,10 +231,10 @@
return 0;
err:
- dma_release_channel(dma->rxchan);
dma_release_channel(dma->txchan);
-
- return -ENOMEM;
+release_rx:
+ dma_release_channel(dma->rxchan);
+ return ret;
}
EXPORT_SYMBOL_GPL(serial8250_request_dma);
diff --git a/drivers/tty/serial/8250/8250_dw.c b/drivers/tty/serial/8250/8250_dw.c
index a3fb95d..e199696 100644
--- a/drivers/tty/serial/8250/8250_dw.c
+++ b/drivers/tty/serial/8250/8250_dw.c
@@ -104,15 +104,16 @@
dw8250_force_idle(p);
#ifdef CONFIG_64BIT
- __raw_writeq(value & 0xff, offset);
-#else
+ if (p->type == PORT_OCTEON)
+ __raw_writeq(value & 0xff, offset);
+ else
+#endif
if (p->iotype == UPIO_MEM32)
writel(value, offset);
else if (p->iotype == UPIO_MEM32BE)
iowrite32be(value, offset);
else
writeb(value, offset);
-#endif
}
/*
* FIXME: this deadlocks if port->lock is already held
@@ -617,6 +618,7 @@
{ "8086228A", 0 },
{ "APMC0D08", 0},
{ "AMD0020", 0 },
+ { "AMDI0020", 0 },
{ },
};
MODULE_DEVICE_TABLE(acpi, dw8250_acpi_match);
diff --git a/drivers/tty/serial/8250/8250_fintek.c b/drivers/tty/serial/8250/8250_fintek.c
index 8947439..870981d 100644
--- a/drivers/tty/serial/8250/8250_fintek.c
+++ b/drivers/tty/serial/8250/8250_fintek.c
@@ -1,9 +1,7 @@
/*
* Probe for F81216A LPC to 4 UART
*
- * Based on drivers/tty/serial/8250_pnp.c, by Russell King, et al
- *
- * Copyright (C) 2014 Ricardo Ribalda, Qtechnology A/S
+ * Copyright (C) 2014-2016 Ricardo Ribalda, Qtechnology A/S
*
*
* This program is free software; you can redistribute it and/or modify
@@ -38,19 +36,15 @@
#define RXW4C_IRA BIT(3)
#define TXW4C_IRA BIT(2)
-#define DRIVER_NAME "8250_fintek"
-
struct fintek_8250 {
u16 base_port;
u8 index;
u8 key;
- long line;
};
static int fintek_8250_enter_key(u16 base_port, u8 key)
{
-
- if (!request_muxed_region(base_port, 2, DRIVER_NAME))
+ if (!request_muxed_region(base_port, 2, "8250_fintek"))
return -EBUSY;
outb(key, base_port + ADDR_PORT);
@@ -138,7 +132,7 @@
return 0;
}
-static int fintek_8250_base_port(u16 io_address, u8 *key, u8 *index)
+static int find_base_port(struct fintek_8250 *pdata, u16 io_address)
{
static const u16 addr[] = {0x4e, 0x2e};
static const u8 keys[] = {0x77, 0xa0, 0x87, 0x67};
@@ -168,10 +162,13 @@
continue;
fintek_8250_exit_key(addr[i]);
- *key = keys[j];
- *index = k;
- return addr[i];
+ pdata->key = keys[j];
+ pdata->base_port = addr[i];
+ pdata->index = k;
+
+ return 0;
}
+
fintek_8250_exit_key(addr[i]);
}
}
@@ -179,104 +176,21 @@
return -ENODEV;
}
-static int
-fintek_8250_probe(struct pnp_dev *dev, const struct pnp_device_id *dev_id)
+int fintek_8250_probe(struct uart_8250_port *uart)
{
- struct uart_8250_port uart;
struct fintek_8250 *pdata;
- int base_port;
- u8 key;
- u8 index;
+ struct fintek_8250 probe_data;
- if (!pnp_port_valid(dev, 0))
+ if (find_base_port(&probe_data, uart->port.iobase))
return -ENODEV;
- base_port = fintek_8250_base_port(pnp_port_start(dev, 0), &key, &index);
- if (base_port < 0)
- return -ENODEV;
-
- memset(&uart, 0, sizeof(uart));
-
- pdata = devm_kzalloc(&dev->dev, sizeof(*pdata), GFP_KERNEL);
+ pdata = devm_kzalloc(uart->port.dev, sizeof(*pdata), GFP_KERNEL);
if (!pdata)
return -ENOMEM;
- uart.port.private_data = pdata;
- if (!pnp_irq_valid(dev, 0))
- return -ENODEV;
- uart.port.irq = pnp_irq(dev, 0);
- uart.port.iobase = pnp_port_start(dev, 0);
- uart.port.iotype = UPIO_PORT;
- uart.port.rs485_config = fintek_8250_rs485_config;
+ memcpy(pdata, &probe_data, sizeof(probe_data));
+ uart->port.rs485_config = fintek_8250_rs485_config;
+ uart->port.private_data = pdata;
- uart.port.flags |= UPF_SKIP_TEST | UPF_BOOT_AUTOCONF;
- if (pnp_irq_flags(dev, 0) & IORESOURCE_IRQ_SHAREABLE)
- uart.port.flags |= UPF_SHARE_IRQ;
- uart.port.uartclk = 1843200;
- uart.port.dev = &dev->dev;
-
- pdata->key = key;
- pdata->base_port = base_port;
- pdata->index = index;
- pdata->line = serial8250_register_8250_port(&uart);
- if (pdata->line < 0)
- return -ENODEV;
-
- pnp_set_drvdata(dev, pdata);
return 0;
}
-
-static void fintek_8250_remove(struct pnp_dev *dev)
-{
- struct fintek_8250 *pdata = pnp_get_drvdata(dev);
-
- if (pdata)
- serial8250_unregister_port(pdata->line);
-}
-
-#ifdef CONFIG_PM
-static int fintek_8250_suspend(struct pnp_dev *dev, pm_message_t state)
-{
- struct fintek_8250 *pdata = pnp_get_drvdata(dev);
-
- if (!pdata)
- return -ENODEV;
- serial8250_suspend_port(pdata->line);
- return 0;
-}
-
-static int fintek_8250_resume(struct pnp_dev *dev)
-{
- struct fintek_8250 *pdata = pnp_get_drvdata(dev);
-
- if (!pdata)
- return -ENODEV;
- serial8250_resume_port(pdata->line);
- return 0;
-}
-#else
-#define fintek_8250_suspend NULL
-#define fintek_8250_resume NULL
-#endif /* CONFIG_PM */
-
-static const struct pnp_device_id fintek_dev_table[] = {
- /* Qtechnology Panel PC / IO1000 */
- { "PNP0501"},
- {}
-};
-
-MODULE_DEVICE_TABLE(pnp, fintek_dev_table);
-
-static struct pnp_driver fintek_8250_driver = {
- .name = DRIVER_NAME,
- .probe = fintek_8250_probe,
- .remove = fintek_8250_remove,
- .suspend = fintek_8250_suspend,
- .resume = fintek_8250_resume,
- .id_table = fintek_dev_table,
-};
-
-module_pnp_driver(fintek_8250_driver);
-MODULE_DESCRIPTION("Fintek F812164 module");
-MODULE_AUTHOR("Ricardo Ribalda <ricardo.ribalda@gmail.com>");
-MODULE_LICENSE("GPL");
diff --git a/drivers/tty/serial/8250/8250_mid.c b/drivers/tty/serial/8250/8250_mid.c
index 88531a3..86379a7 100644
--- a/drivers/tty/serial/8250/8250_mid.c
+++ b/drivers/tty/serial/8250/8250_mid.c
@@ -9,11 +9,13 @@
* published by the Free Software Foundation.
*/
-#include <linux/rational.h>
+#include <linux/bitops.h>
#include <linux/module.h>
#include <linux/pci.h>
+#include <linux/rational.h>
#include <linux/dma/hsu.h>
+#include <linux/8250_pci.h>
#include "8250.h"
@@ -24,6 +26,7 @@
#define PCI_DEVICE_ID_INTEL_DNV_UART 0x19d8
/* Intel MID Specific registers */
+#define INTEL_MID_UART_DNV_FISR 0x08
#define INTEL_MID_UART_PS 0x30
#define INTEL_MID_UART_MUL 0x34
#define INTEL_MID_UART_DIV 0x38
@@ -31,6 +34,7 @@
struct mid8250;
struct mid8250_board {
+ unsigned int flags;
unsigned long freq;
unsigned int base_baud;
int (*setup)(struct mid8250 *, struct uart_port *p);
@@ -76,7 +80,11 @@
struct pci_dev *pdev = to_pci_dev(p->dev);
int index = PCI_FUNC(pdev->devfn);
- /* Currently no support for HSU port0 */
+ /*
+ * Device 0000:00:04.0 is not a real HSU port. It provides a global
+ * register set for all HSU ports, although it has the same PCI ID.
+ * Skip it here.
+ */
if (index-- == 0)
return -ENODEV;
@@ -88,16 +96,16 @@
static int dnv_handle_irq(struct uart_port *p)
{
struct mid8250 *mid = p->private_data;
- int ret;
+ unsigned int fisr = serial_port_in(p, INTEL_MID_UART_DNV_FISR);
+ int ret = IRQ_NONE;
- ret = hsu_dma_irq(&mid->dma_chip, 0);
- ret |= hsu_dma_irq(&mid->dma_chip, 1);
-
- /* For now, letting the HW generate separate interrupt for the UART */
- if (ret)
- return ret;
-
- return serial8250_handle_irq(p, serial_port_in(p, UART_IIR));
+ if (fisr & BIT(2))
+ ret |= hsu_dma_irq(&mid->dma_chip, 1);
+ if (fisr & BIT(1))
+ ret |= hsu_dma_irq(&mid->dma_chip, 0);
+ if (fisr & BIT(0))
+ ret |= serial8250_handle_irq(p, serial_port_in(p, UART_IIR));
+ return ret;
}
#define DNV_DMA_CHAN_OFFSET 0x80
@@ -106,12 +114,13 @@
{
struct hsu_dma_chip *chip = &mid->dma_chip;
struct pci_dev *pdev = to_pci_dev(p->dev);
+ unsigned int bar = FL_GET_BASE(mid->board->flags);
int ret;
chip->dev = &pdev->dev;
chip->irq = pdev->irq;
chip->regs = p->membase;
- chip->length = pci_resource_len(pdev, 0);
+ chip->length = pci_resource_len(pdev, bar);
chip->offset = DNV_DMA_CHAN_OFFSET;
/* Falling back to PIO mode if DMA probing fails */
@@ -217,6 +226,7 @@
{
struct uart_8250_port uart;
struct mid8250 *mid;
+ unsigned int bar;
int ret;
ret = pcim_enable_device(pdev);
@@ -230,6 +240,7 @@
return -ENOMEM;
mid->board = (struct mid8250_board *)id->driver_data;
+ bar = FL_GET_BASE(mid->board->flags);
memset(&uart, 0, sizeof(struct uart_8250_port));
@@ -242,8 +253,8 @@
uart.port.flags = UPF_SHARE_IRQ | UPF_FIXED_PORT | UPF_FIXED_TYPE;
uart.port.set_termios = mid8250_set_termios;
- uart.port.mapbase = pci_resource_start(pdev, 0);
- uart.port.membase = pcim_iomap(pdev, 0, 0);
+ uart.port.mapbase = pci_resource_start(pdev, bar);
+ uart.port.membase = pcim_iomap(pdev, bar, 0);
if (!uart.port.membase)
return -ENOMEM;
@@ -282,18 +293,21 @@
}
static const struct mid8250_board pnw_board = {
+ .flags = FL_BASE0,
.freq = 50000000,
.base_baud = 115200,
.setup = pnw_setup,
};
static const struct mid8250_board tng_board = {
+ .flags = FL_BASE0,
.freq = 38400000,
.base_baud = 1843200,
.setup = tng_setup,
};
static const struct mid8250_board dnv_board = {
+ .flags = FL_BASE1,
.freq = 133333333,
.base_baud = 115200,
.setup = dnv_setup,
diff --git a/drivers/tty/serial/8250/8250_of.c b/drivers/tty/serial/8250/8250_of.c
index c7ed3d2b..38963d7 100644
--- a/drivers/tty/serial/8250/8250_of.c
+++ b/drivers/tty/serial/8250/8250_of.c
@@ -29,7 +29,7 @@
};
#ifdef CONFIG_ARCH_TEGRA
-void tegra_serial_handle_break(struct uart_port *p)
+static void tegra_serial_handle_break(struct uart_port *p)
{
unsigned int status, tmout = 10000;
diff --git a/drivers/tty/serial/8250/8250_omap.c b/drivers/tty/serial/8250/8250_omap.c
index 6f76051..2c44c79 100644
--- a/drivers/tty/serial/8250/8250_omap.c
+++ b/drivers/tty/serial/8250/8250_omap.c
@@ -115,6 +115,12 @@
bool rx_dma_broken;
};
+#ifdef CONFIG_SERIAL_8250_DMA
+static void omap_8250_rx_dma_flush(struct uart_8250_port *p);
+#else
+static inline void omap_8250_rx_dma_flush(struct uart_8250_port *p) { }
+#endif
+
static u32 uart_read(struct uart_8250_port *up, u32 reg)
{
return readl(up->port.membase + (reg << up->port.regshift));
@@ -635,7 +641,7 @@
serial_out(up, UART_OMAP_WER, priv->wer);
if (up->dma)
- up->dma->rx_dma(up, 0);
+ up->dma->rx_dma(up);
pm_runtime_mark_last_busy(port->dev);
pm_runtime_put_autosuspend(port->dev);
@@ -654,7 +660,7 @@
flush_work(&priv->qos_work);
if (up->dma)
- up->dma->rx_dma(up, UART_IIR_RX_TIMEOUT);
+ omap_8250_rx_dma_flush(up);
pm_runtime_get_sync(port->dev);
@@ -742,9 +748,9 @@
}
#ifdef CONFIG_SERIAL_8250_DMA
-static int omap_8250_rx_dma(struct uart_8250_port *p, unsigned int iir);
+static int omap_8250_rx_dma(struct uart_8250_port *p);
-static void __dma_rx_do_complete(struct uart_8250_port *p, bool error)
+static void __dma_rx_do_complete(struct uart_8250_port *p)
{
struct omap8250_priv *priv = p->port.private_data;
struct uart_8250_dma *dma = p->dma;
@@ -754,9 +760,6 @@
unsigned long flags;
int ret;
- dma_sync_single_for_cpu(dma->rxchan->device->dev, dma->rx_addr,
- dma->rx_size, DMA_FROM_DEVICE);
-
spin_lock_irqsave(&priv->rx_dma_lock, flags);
if (!dma->rx_running)
@@ -764,7 +767,6 @@
dma->rx_running = 0;
dmaengine_tx_status(dma->rxchan, dma->rx_cookie, &state);
- dmaengine_terminate_all(dma->rxchan);
count = dma->rx_size - state.residue;
@@ -775,15 +777,13 @@
unlock:
spin_unlock_irqrestore(&priv->rx_dma_lock, flags);
- if (!error)
- omap_8250_rx_dma(p, 0);
-
tty_flip_buffer_push(tty_port);
}
static void __dma_rx_complete(void *param)
{
- __dma_rx_do_complete(param, false);
+ __dma_rx_do_complete(param);
+ omap_8250_rx_dma(param);
}
static void omap_8250_rx_dma_flush(struct uart_8250_port *p)
@@ -806,10 +806,11 @@
spin_unlock_irqrestore(&priv->rx_dma_lock, flags);
- __dma_rx_do_complete(p, true);
+ __dma_rx_do_complete(p);
+ dmaengine_terminate_all(dma->rxchan);
}
-static int omap_8250_rx_dma(struct uart_8250_port *p, unsigned int iir)
+static int omap_8250_rx_dma(struct uart_8250_port *p)
{
struct omap8250_priv *priv = p->port.private_data;
struct uart_8250_dma *dma = p->dma;
@@ -817,35 +818,6 @@
struct dma_async_tx_descriptor *desc;
unsigned long flags;
- switch (iir & 0x3f) {
- case UART_IIR_RLSI:
- /* 8250_core handles errors and break interrupts */
- omap_8250_rx_dma_flush(p);
- return -EIO;
- case UART_IIR_RX_TIMEOUT:
- /*
- * If RCVR FIFO trigger level was not reached, complete the
- * transfer and let 8250_core copy the remaining data.
- */
- omap_8250_rx_dma_flush(p);
- return -ETIMEDOUT;
- case UART_IIR_RDI:
- /*
- * The OMAP UART is a special BEAST. If we receive RDI we _have_
- * a DMA transfer programmed but it didn't work. One reason is
- * that we were too slow and there were too many bytes in the
- * FIFO, the UART counted wrong and never kicked the DMA engine
- * to do anything. That means once we receive RDI on OMAP then
- * the DMA won't do anything soon so we have to cancel the DMA
- * transfer and purge the FIFO manually.
- */
- omap_8250_rx_dma_flush(p);
- return -ETIMEDOUT;
-
- default:
- break;
- }
-
if (priv->rx_dma_broken)
return -EINVAL;
@@ -868,9 +840,6 @@
dma->rx_cookie = dmaengine_submit(desc);
- dma_sync_single_for_device(dma->rxchan->device->dev, dma->rx_addr,
- dma->rx_size, DMA_FROM_DEVICE);
-
dma_async_issue_pending(dma->rxchan);
out:
spin_unlock_irqrestore(&priv->rx_dma_lock, flags);
@@ -1022,6 +991,18 @@
return ret;
}
+static bool handle_rx_dma(struct uart_8250_port *up, unsigned int iir)
+{
+ switch (iir & 0x3f) {
+ case UART_IIR_RLSI:
+ case UART_IIR_RX_TIMEOUT:
+ case UART_IIR_RDI:
+ omap_8250_rx_dma_flush(up);
+ return true;
+ }
+ return omap_8250_rx_dma(up);
+}
+
/*
* This is mostly serial8250_handle_irq(). We have a slightly different DMA
* hoook for RX/TX and need different logic for them in the ISR. Therefore we
@@ -1033,7 +1014,6 @@
unsigned char status;
unsigned long flags;
u8 iir;
- int dma_err = 0;
serial8250_rpm_get(up);
@@ -1048,11 +1028,9 @@
status = serial_port_in(port, UART_LSR);
if (status & (UART_LSR_DR | UART_LSR_BI)) {
-
- dma_err = omap_8250_rx_dma(up, iir);
- if (dma_err) {
+ if (handle_rx_dma(up, iir)) {
status = serial8250_rx_chars(up, status);
- omap_8250_rx_dma(up, 0);
+ omap_8250_rx_dma(up);
}
}
serial8250_modem_status(up);
@@ -1066,8 +1044,7 @@
* try again due to an earlier failer which
* might have been resolved by now.
*/
- dma_err = omap_8250_tx_dma(up);
- if (dma_err)
+ if (omap_8250_tx_dma(up))
serial8250_tx_chars(up);
}
}
@@ -1084,7 +1061,7 @@
#else
-static inline int omap_8250_rx_dma(struct uart_8250_port *p, unsigned int iir)
+static inline int omap_8250_rx_dma(struct uart_8250_port *p)
{
return -EINVAL;
}
@@ -1395,7 +1372,7 @@
}
if (up->dma && up->dma->rxchan)
- omap_8250_rx_dma(up, UART_IIR_RX_TIMEOUT);
+ omap_8250_rx_dma_flush(up);
priv->latency = PM_QOS_CPU_DMA_LAT_DEFAULT_VALUE;
schedule_work(&priv->qos_work);
@@ -1407,20 +1384,18 @@
{
struct omap8250_priv *priv = dev_get_drvdata(dev);
struct uart_8250_port *up;
- int loss_cntx;
/* In case runtime-pm tries this before we are setup */
if (!priv)
return 0;
up = serial8250_get_port(priv->line);
- loss_cntx = omap8250_lost_context(up);
- if (loss_cntx)
+ if (omap8250_lost_context(up))
omap8250_restore_regs(up);
if (up->dma && up->dma->rxchan)
- omap_8250_rx_dma(up, 0);
+ omap_8250_rx_dma(up);
priv->latency = priv->calc_latency;
schedule_work(&priv->qos_work);
diff --git a/drivers/tty/serial/8250/8250_pci.c b/drivers/tty/serial/8250/8250_pci.c
index 98862aa..8dd250f 100644
--- a/drivers/tty/serial/8250/8250_pci.c
+++ b/drivers/tty/serial/8250/8250_pci.c
@@ -1377,6 +1377,9 @@
unsigned long m, n;
u32 reg;
+ /* Gracefully handle the B0 case: fall back to B9600 */
+ fuart = fuart ? fuart : 9600 * 16;
+
/* Get Fuart closer to Fref */
fuart *= rounddown_pow_of_two(fref / fuart);
@@ -1413,6 +1416,17 @@
return true;
}
+static unsigned int
+byt_get_mctrl(struct uart_port *port)
+{
+ unsigned int ret = serial8250_do_get_mctrl(port);
+
+ /* Force DCD and DSR signals to permanently be reported as active. */
+ ret |= TIOCM_CAR | TIOCM_DSR;
+
+ return ret;
+}
+
static int
byt_serial_setup(struct serial_private *priv,
const struct pciserial_board *board,
@@ -1454,13 +1468,13 @@
return -EINVAL;
}
- rx_param->src_master = 1;
- rx_param->dst_master = 0;
+ rx_param->m_master = 0;
+ rx_param->p_master = 1;
dma->rxconf.src_maxburst = 16;
- tx_param->src_master = 1;
- tx_param->dst_master = 0;
+ tx_param->m_master = 0;
+ tx_param->p_master = 1;
dma->txconf.dst_maxburst = 16;
@@ -1477,6 +1491,7 @@
port->port.type = PORT_16550A;
port->port.flags = (port->port.flags | UPF_FIXED_PORT | UPF_FIXED_TYPE);
port->port.set_termios = byt_set_termios;
+ port->port.get_mctrl = byt_get_mctrl;
port->port.fifosize = 64;
port->tx_loadsz = 64;
port->dma = dma;
diff --git a/drivers/tty/serial/8250/8250_port.c b/drivers/tty/serial/8250/8250_port.c
index 00ad263..d4036038 100644
--- a/drivers/tty/serial/8250/8250_port.c
+++ b/drivers/tty/serial/8250/8250_port.c
@@ -1315,6 +1315,13 @@
out_lock:
spin_unlock_irqrestore(&port->lock, flags);
+
+ /*
+ * Check if the device is a Fintek F81216A
+ */
+ if (port->type == PORT_16550A)
+ fintek_8250_probe(up);
+
if (up->capabilities != old_capabilities) {
pr_warn("ttyS%d: detected caps %08x should be %08x\n",
serial_index(port), old_capabilities,
@@ -1788,6 +1795,18 @@
}
EXPORT_SYMBOL_GPL(serial8250_modem_status);
+static bool handle_rx_dma(struct uart_8250_port *up, unsigned int iir)
+{
+ switch (iir & 0x3f) {
+ case UART_IIR_RX_TIMEOUT:
+ serial8250_rx_dma_flush(up);
+ /* fall-through */
+ case UART_IIR_RLSI:
+ return true;
+ }
+ return up->dma->rx_dma(up);
+}
+
/*
* This handles the interrupt from one port.
*/
@@ -1796,7 +1815,6 @@
unsigned char status;
unsigned long flags;
struct uart_8250_port *up = up_to_u8250p(port);
- int dma_err = 0;
if (iir & UART_IIR_NO_INT)
return 0;
@@ -1808,15 +1826,11 @@
DEBUG_INTR("status = %x...", status);
if (status & (UART_LSR_DR | UART_LSR_BI)) {
- if (up->dma)
- dma_err = up->dma->rx_dma(up, iir);
-
- if (!up->dma || dma_err)
+ if (!up->dma || handle_rx_dma(up, iir))
status = serial8250_rx_chars(up, status);
}
serial8250_modem_status(up);
- if ((!up->dma || (up->dma && up->dma->tx_err)) &&
- (status & UART_LSR_THRE))
+ if ((!up->dma || up->dma->tx_err) && (status & UART_LSR_THRE))
serial8250_tx_chars(up);
spin_unlock_irqrestore(&port->lock, flags);
@@ -1882,7 +1896,7 @@
return (lsr & BOTH_EMPTY) == BOTH_EMPTY ? TIOCSER_TEMT : 0;
}
-static unsigned int serial8250_get_mctrl(struct uart_port *port)
+unsigned int serial8250_do_get_mctrl(struct uart_port *port)
{
struct uart_8250_port *up = up_to_u8250p(port);
unsigned int status;
@@ -1903,6 +1917,14 @@
ret |= TIOCM_CTS;
return ret;
}
+EXPORT_SYMBOL_GPL(serial8250_do_get_mctrl);
+
+static unsigned int serial8250_get_mctrl(struct uart_port *port)
+{
+ if (port->get_mctrl)
+ return port->get_mctrl(port);
+ return serial8250_do_get_mctrl(port);
+}
void serial8250_do_set_mctrl(struct uart_port *port, unsigned int mctrl)
{
diff --git a/drivers/tty/serial/8250/8250_uniphier.c b/drivers/tty/serial/8250/8250_uniphier.c
index 1b7bd26..efd1f9c 100644
--- a/drivers/tty/serial/8250/8250_uniphier.c
+++ b/drivers/tty/serial/8250/8250_uniphier.c
@@ -209,7 +209,7 @@
irq = platform_get_irq(pdev, 0);
if (irq < 0) {
- dev_err(dev, "failed to get IRQ number");
+ dev_err(dev, "failed to get IRQ number\n");
return irq;
}
diff --git a/drivers/tty/serial/8250/Kconfig b/drivers/tty/serial/8250/Kconfig
index 4d7cb9c..e46761d 100644
--- a/drivers/tty/serial/8250/Kconfig
+++ b/drivers/tty/serial/8250/Kconfig
@@ -57,6 +57,18 @@
This builds standard PNP serial support. You may be able to
disable this feature if you only need legacy serial support.
+config SERIAL_8250_FINTEK
+ bool "Support for Fintek F81216A LPC to 4 UART RS485 API"
+ depends on SERIAL_8250
+ ---help---
+ Selecting this option will add support for the RS485 capabilities
+ of the Fintek F81216A LPC to 4 UART.
+
+ If this option is not selected the device will be configured as a
+ standard 16550A serial port.
+
+ If unsure, say N.
+
config SERIAL_8250_CONSOLE
bool "Console on 8250/16550 and compatible serial port"
depends on SERIAL_8250=y
@@ -358,14 +370,6 @@
not booting kernel because the serial console remains silent in case
they forgot to update the command line.
-config SERIAL_8250_FINTEK
- tristate "Support for Fintek F81216A LPC to 4 UART"
- depends on SERIAL_8250 && PNP
- help
- Selecting this option will add support for the Fintek F81216A
- LPC to 4 UART. This device has some RS485 functionality not available
- through the PNP driver. If unsure, say N.
-
config SERIAL_8250_LPC18XX
tristate "NXP LPC18xx/43xx serial port support"
depends on SERIAL_8250 && OF && (ARCH_LPC18XX || COMPILE_TEST)
@@ -398,8 +402,10 @@
its UARTs, say Y to this option. If unsure, say N.
config SERIAL_8250_MID
- tristate "Support for serial ports on Intel MID platforms"
+ tristate "Support for serial ports on Intel MID platforms" if EXPERT
+ default SERIAL_8250
depends on SERIAL_8250 && PCI
+ depends on X86 || COMPILE_TEST
select HSU_DMA if SERIAL_8250_DMA
select HSU_DMA_PCI if (HSU_DMA && X86_INTEL_MID)
select RATIONAL
diff --git a/drivers/tty/serial/8250/Makefile b/drivers/tty/serial/8250/Makefile
index c9a2d6e..367d403 100644
--- a/drivers/tty/serial/8250/Makefile
+++ b/drivers/tty/serial/8250/Makefile
@@ -7,6 +7,7 @@
8250-$(CONFIG_SERIAL_8250_PNP) += 8250_pnp.o
8250_base-y := 8250_port.o
8250_base-$(CONFIG_SERIAL_8250_DMA) += 8250_dma.o
+8250_base-$(CONFIG_SERIAL_8250_FINTEK) += 8250_fintek.o
obj-$(CONFIG_SERIAL_8250_GSC) += 8250_gsc.o
obj-$(CONFIG_SERIAL_8250_PCI) += 8250_pci.o
obj-$(CONFIG_SERIAL_8250_HP300) += 8250_hp300.o
@@ -23,7 +24,6 @@
obj-$(CONFIG_SERIAL_8250_DW) += 8250_dw.o
obj-$(CONFIG_SERIAL_8250_EM) += 8250_em.o
obj-$(CONFIG_SERIAL_8250_OMAP) += 8250_omap.o
-obj-$(CONFIG_SERIAL_8250_FINTEK) += 8250_fintek.o
obj-$(CONFIG_SERIAL_8250_LPC18XX) += 8250_lpc18xx.o
obj-$(CONFIG_SERIAL_8250_MT6577) += 8250_mtk.o
obj-$(CONFIG_SERIAL_8250_UNIPHIER) += 8250_uniphier.o
diff --git a/drivers/tty/serial/Kconfig b/drivers/tty/serial/Kconfig
index 13d4ed6..7e3a58c 100644
--- a/drivers/tty/serial/Kconfig
+++ b/drivers/tty/serial/Kconfig
@@ -213,6 +213,7 @@
bool "Support for console on meson"
depends on SERIAL_MESON=y
select SERIAL_CORE_CONSOLE
+ select SERIAL_EARLYCON
help
Say Y here if you wish to use a Amlogic MesonX UART as the
system console (the system console is the device which
@@ -900,6 +901,27 @@
controller serial port as your console (you want this!),
say Y. Otherwise, say N.
+config SERIAL_PIC32
+ tristate "Microchip PIC32 serial support"
+ depends on MACH_PIC32
+ select SERIAL_CORE
+ help
+ If you have a PIC32, this driver supports the serial ports.
+
+ Say Y or M to use PIC32 serial ports, otherwise say N. Note that
+ to use a serial port as a console, this must be included in kernel and
+ not as a module.
+
+config SERIAL_PIC32_CONSOLE
+ bool "PIC32 serial console support"
+ depends on SERIAL_PIC32
+ select SERIAL_CORE_CONSOLE
+ help
+ If you have a PIC32, this driver supports the putting a console on one
+ of the serial ports.
+
+ Say Y to use the PIC32 console, otherwise say N.
+
config SERIAL_MPC52xx
tristate "Freescale MPC52xx/MPC512x family PSC serial support"
depends on PPC_MPC52xx || PPC_MPC512x
@@ -1383,11 +1405,12 @@
config SERIAL_MXS_AUART
tristate "MXS AUART support"
depends on HAS_DMA
- depends on ARCH_MXS || COMPILE_TEST
+ depends on ARCH_MXS || MACH_ASM9260 || COMPILE_TEST
select SERIAL_CORE
select SERIAL_MCTRL_GPIO if GPIOLIB
help
- This driver supports the MXS Application UART (AUART) port.
+ This driver supports the MXS and Alphascale ASM9260 Application
+ UART (AUART) port.
config SERIAL_MXS_AUART_CONSOLE
bool "MXS AUART console support"
@@ -1446,6 +1469,19 @@
This driver support the USART and UART ports on
Energy Micro's efm32 SoCs.
+config SERIAL_MPS2_UART_CONSOLE
+ bool "MPS2 UART console support"
+ depends on SERIAL_MPS2_UART
+ select SERIAL_CORE_CONSOLE
+ select SERIAL_EARLYCON
+
+config SERIAL_MPS2_UART
+ bool "MPS2 UART port"
+ depends on ARM || COMPILE_TEST
+ select SERIAL_CORE
+ help
+ This driver support the UART ports on ARM MPS2.
+
config SERIAL_EFM32_UART_CONSOLE
bool "EFM32 UART/USART console support"
depends on SERIAL_EFM32_UART=y
@@ -1604,6 +1640,7 @@
config SERIAL_MVEBU_UART
bool "Marvell EBU serial port support"
+ depends on ARCH_MVEBU || COMPILE_TEST
select SERIAL_CORE
help
This driver is for Marvell EBU SoC's UART. If you have a machine
diff --git a/drivers/tty/serial/Makefile b/drivers/tty/serial/Makefile
index 8c261ad..1278d37 100644
--- a/drivers/tty/serial/Makefile
+++ b/drivers/tty/serial/Makefile
@@ -91,6 +91,8 @@
obj-$(CONFIG_SERIAL_SPRD) += sprd_serial.o
obj-$(CONFIG_SERIAL_STM32) += stm32-usart.o
obj-$(CONFIG_SERIAL_MVEBU_UART) += mvebu-uart.o
+obj-$(CONFIG_SERIAL_PIC32) += pic32_uart.o
+obj-$(CONFIG_SERIAL_MPS2_UART) += mps2-uart.o
# GPIOLIB helpers for modem control lines
obj-$(CONFIG_SERIAL_MCTRL_GPIO) += serial_mctrl_gpio.o
diff --git a/drivers/tty/serial/amba-pl011.c b/drivers/tty/serial/amba-pl011.c
index 7c198e0..a2aa655 100644
--- a/drivers/tty/serial/amba-pl011.c
+++ b/drivers/tty/serial/amba-pl011.c
@@ -121,6 +121,7 @@
static struct vendor_data vendor_sbsa = {
.reg_offset = pl011_std_offsets,
+ .access_32b = true,
.oversampling = false,
.dma_threshold = false,
.cts_event_workaround = false,
diff --git a/drivers/tty/serial/atmel_serial.c b/drivers/tty/serial/atmel_serial.c
index d9439e6..954941d 100644
--- a/drivers/tty/serial/atmel_serial.c
+++ b/drivers/tty/serial/atmel_serial.c
@@ -274,6 +274,13 @@
return atmel_port->use_dma_rx;
}
+static bool atmel_use_fifo(struct uart_port *port)
+{
+ struct atmel_uart_port *atmel_port = to_atmel_uart_port(port);
+
+ return atmel_port->fifo_size;
+}
+
static unsigned int atmel_get_lines_status(struct uart_port *port)
{
struct atmel_uart_port *atmel_port = to_atmel_uart_port(port);
@@ -2090,7 +2097,12 @@
mode |= ATMEL_US_USMODE_RS485;
} else if (termios->c_cflag & CRTSCTS) {
/* RS232 with hardware handshake (RTS/CTS) */
- mode |= ATMEL_US_USMODE_HWHS;
+ if (atmel_use_dma_rx(port) && !atmel_use_fifo(port)) {
+ dev_info(port->dev, "not enabling hardware flow control because DMA is used");
+ termios->c_cflag &= ~CRTSCTS;
+ } else {
+ mode |= ATMEL_US_USMODE_HWHS;
+ }
} else {
/* RS232 without hadware handshake */
mode |= ATMEL_US_USMODE_NORMAL;
diff --git a/drivers/tty/serial/crisv10.c b/drivers/tty/serial/crisv10.c
index c0172bf54..315c849 100644
--- a/drivers/tty/serial/crisv10.c
+++ b/drivers/tty/serial/crisv10.c
@@ -2599,7 +2599,7 @@
/* if it was already initialized, skip this */
- if (info->port.flags & ASYNC_INITIALIZED) {
+ if (tty_port_initialized(&info->port)) {
local_irq_restore(flags);
free_page(xmit_page);
return 0;
@@ -2703,7 +2703,7 @@
e100_rts(info, 1);
e100_dtr(info, 1);
- info->port.flags |= ASYNC_INITIALIZED;
+ tty_port_set_initialized(&info->port, 1);
local_irq_restore(flags);
return 0;
@@ -2745,7 +2745,7 @@
info->tr_running = 0;
}
- if (!(info->port.flags & ASYNC_INITIALIZED))
+ if (!tty_port_initialized(&info->port))
return;
#ifdef SERIAL_DEBUG_OPEN
@@ -2776,7 +2776,7 @@
if (info->port.tty)
set_bit(TTY_IO_ERROR, &info->port.tty->flags);
- info->port.flags &= ~ASYNC_INITIALIZED;
+ tty_port_set_initialized(&info->port, 0);
local_irq_restore(flags);
}
@@ -3273,9 +3273,9 @@
info->port.low_latency = (info->port.flags & ASYNC_LOW_LATENCY) ? 1 : 0;
check_and_exit:
- if (info->port.flags & ASYNC_INITIALIZED) {
+ if (tty_port_initialized(&info->port))
change_speed(info);
- } else
+ else
retval = startup(info);
return retval;
}
@@ -3445,7 +3445,7 @@
if ((cmd != TIOCGSERIAL) && (cmd != TIOCSSERIAL) &&
(cmd != TIOCSERCONFIG) && (cmd != TIOCSERGWILD) &&
(cmd != TIOCSERSWILD) && (cmd != TIOCSERGSTRUCT)) {
- if (tty->flags & (1 << TTY_IO_ERROR))
+ if (tty_io_error(tty))
return -EIO;
}
@@ -3628,7 +3628,7 @@
e100_disable_rx(info);
e100_disable_rx_irq(info);
- if (info->port.flags & ASYNC_INITIALIZED) {
+ if (tty_port_initialized(&info->port)) {
/*
* Before we drop DTR, make sure the UART transmitter
* has completely drained; this is especially
@@ -3648,8 +3648,8 @@
schedule_timeout_interruptible(info->port.close_delay);
wake_up_interruptible(&info->port.open_wait);
}
- info->port.flags &= ~ASYNC_NORMAL_ACTIVE;
local_irq_restore(flags);
+ tty_port_set_active(&info->port, 0);
/* port closed */
@@ -3732,7 +3732,7 @@
shutdown(info);
info->event = 0;
info->port.count = 0;
- info->port.flags &= ~ASYNC_NORMAL_ACTIVE;
+ tty_port_set_active(&info->port, 0);
info->port.tty = NULL;
wake_up_interruptible(&info->port.open_wait);
}
@@ -3755,9 +3755,8 @@
* If non-blocking mode is set, or the port is not enabled,
* then make the check up front and then exit.
*/
- if ((filp->f_flags & O_NONBLOCK) ||
- (tty->flags & (1 << TTY_IO_ERROR))) {
- info->port.flags |= ASYNC_NORMAL_ACTIVE;
+ if ((filp->f_flags & O_NONBLOCK) || tty_io_error(tty)) {
+ tty_port_set_active(&info->port, 1);
return 0;
}
@@ -3788,8 +3787,7 @@
e100_dtr(info, 1);
local_irq_restore(flags);
set_current_state(TASK_INTERRUPTIBLE);
- if (tty_hung_up_p(filp) ||
- !(info->port.flags & ASYNC_INITIALIZED)) {
+ if (tty_hung_up_p(filp) || !tty_port_initialized(&info->port)) {
#ifdef SERIAL_DO_RESTART
if (info->port.flags & ASYNC_HUP_NOTIFY)
retval = -EAGAIN;
@@ -3826,7 +3824,7 @@
#endif
if (retval)
return retval;
- info->port.flags |= ASYNC_NORMAL_ACTIVE;
+ tty_port_set_active(&info->port, 1);
return 0;
}
diff --git a/drivers/tty/serial/ifx6x60.c b/drivers/tty/serial/ifx6x60.c
index 2085a6c..d386346 100644
--- a/drivers/tty/serial/ifx6x60.c
+++ b/drivers/tty/serial/ifx6x60.c
@@ -651,7 +651,7 @@
struct ifx_spi_device *ifx_dev = ctx;
int length;
int actual_length;
- unsigned char more;
+ unsigned char more = 0;
unsigned char cts;
int local_write_pending = 0;
int queue_length;
diff --git a/drivers/tty/serial/imx.c b/drivers/tty/serial/imx.c
index 231e7d5..0df2b1c 100644
--- a/drivers/tty/serial/imx.c
+++ b/drivers/tty/serial/imx.c
@@ -114,6 +114,7 @@
#define UCR3_RXDSEN (1<<6) /* Receive status interrupt enable */
#define UCR3_AIRINTEN (1<<5) /* Async IR wake interrupt enable */
#define UCR3_AWAKEN (1<<4) /* Async wake interrupt enable */
+#define UCR3_DTRDEN (1<<3) /* Data Terminal Ready Delta Enable. */
#define IMX21_UCR3_RXDMUXSEL (1<<2) /* RXD Muxed Input Select */
#define UCR3_INVT (1<<1) /* Inverted Infrared transmission */
#define UCR3_BPEN (1<<0) /* Preset registers enable */
@@ -142,7 +143,7 @@
#define USR1_FRAMERR (1<<10) /* Frame error interrupt flag */
#define USR1_RRDY (1<<9) /* Receiver ready interrupt/dma flag */
#define USR1_AGTIM (1<<8) /* Ageing timer interrupt flag */
-#define USR1_TIMEOUT (1<<7) /* Receive timeout interrupt status */
+#define USR1_DTRD (1<<7) /* DTR Delta */
#define USR1_RXDS (1<<6) /* Receiver idle interrupt flag */
#define USR1_AIRINT (1<<5) /* Async IR wake interrupt flag */
#define USR1_AWAKE (1<<4) /* Aysnc wake interrupt flag */
@@ -361,6 +362,7 @@
imx_port_rts_inactive(sport, &temp);
else
imx_port_rts_active(sport, &temp);
+ temp |= UCR2_RXEN;
writel(temp, port->membase + UCR2);
temp = readl(port->membase + UCR4);
@@ -568,6 +570,8 @@
imx_port_rts_inactive(sport, &temp);
else
imx_port_rts_active(sport, &temp);
+ if (!(port->rs485.flags & SER_RS485_RX_DURING_TX))
+ temp &= ~UCR2_RXEN;
writel(temp, port->membase + UCR2);
/* enable transmitter and shifter empty irq */
@@ -729,11 +733,61 @@
spin_unlock_irqrestore(&sport->port.lock, flags);
}
+/*
+ * We have a modem side uart, so the meanings of RTS and CTS are inverted.
+ */
+static unsigned int imx_get_hwmctrl(struct imx_port *sport)
+{
+ unsigned int tmp = TIOCM_DSR;
+ unsigned usr1 = readl(sport->port.membase + USR1);
+
+ if (usr1 & USR1_RTSS)
+ tmp |= TIOCM_CTS;
+
+ /* in DCE mode DCDIN is always 0 */
+ if (!(usr1 & USR2_DCDIN))
+ tmp |= TIOCM_CAR;
+
+ if (sport->dte_mode)
+ if (!(readl(sport->port.membase + USR2) & USR2_RIIN))
+ tmp |= TIOCM_RI;
+
+ return tmp;
+}
+
+/*
+ * Handle any change of modem status signal since we were last called.
+ */
+static void imx_mctrl_check(struct imx_port *sport)
+{
+ unsigned int status, changed;
+
+ status = imx_get_hwmctrl(sport);
+ changed = status ^ sport->old_status;
+
+ if (changed == 0)
+ return;
+
+ sport->old_status = status;
+
+ if (changed & TIOCM_RI && status & TIOCM_RI)
+ sport->port.icount.rng++;
+ if (changed & TIOCM_DSR)
+ sport->port.icount.dsr++;
+ if (changed & TIOCM_CAR)
+ uart_handle_dcd_change(&sport->port, status & TIOCM_CAR);
+ if (changed & TIOCM_CTS)
+ uart_handle_cts_change(&sport->port, status & TIOCM_CTS);
+
+ wake_up_interruptible(&sport->port.state->port.delta_msr_wait);
+}
+
static irqreturn_t imx_int(int irq, void *dev_id)
{
struct imx_port *sport = dev_id;
unsigned int sts;
unsigned int sts2;
+ irqreturn_t ret = IRQ_NONE;
sts = readl(sport->port.membase + USR1);
sts2 = readl(sport->port.membase + USR2);
@@ -743,26 +797,47 @@
imx_dma_rxint(sport);
else
imx_rxint(irq, dev_id);
+ ret = IRQ_HANDLED;
}
if ((sts & USR1_TRDY &&
readl(sport->port.membase + UCR1) & UCR1_TXMPTYEN) ||
(sts2 & USR2_TXDC &&
- readl(sport->port.membase + UCR4) & UCR4_TCEN))
+ readl(sport->port.membase + UCR4) & UCR4_TCEN)) {
imx_txint(irq, dev_id);
+ ret = IRQ_HANDLED;
+ }
- if (sts & USR1_RTSD)
+ if (sts & USR1_DTRD) {
+ unsigned long flags;
+
+ if (sts & USR1_DTRD)
+ writel(USR1_DTRD, sport->port.membase + USR1);
+
+ spin_lock_irqsave(&sport->port.lock, flags);
+ imx_mctrl_check(sport);
+ spin_unlock_irqrestore(&sport->port.lock, flags);
+
+ ret = IRQ_HANDLED;
+ }
+
+ if (sts & USR1_RTSD) {
imx_rtsint(irq, dev_id);
+ ret = IRQ_HANDLED;
+ }
- if (sts & USR1_AWAKE)
+ if (sts & USR1_AWAKE) {
writel(USR1_AWAKE, sport->port.membase + USR1);
+ ret = IRQ_HANDLED;
+ }
if (sts2 & USR2_ORE) {
sport->port.icount.overrun++;
writel(USR2_ORE, sport->port.membase + USR2);
+ ret = IRQ_HANDLED;
}
- return IRQ_HANDLED;
+ return ret;
}
/*
@@ -782,28 +857,6 @@
return ret;
}
-/*
- * We have a modem side uart, so the meanings of RTS and CTS are inverted.
- */
-static unsigned int imx_get_hwmctrl(struct imx_port *sport)
-{
- unsigned int tmp = TIOCM_DSR;
- unsigned usr1 = readl(sport->port.membase + USR1);
-
- if (usr1 & USR1_RTSS)
- tmp |= TIOCM_CTS;
-
- /* in DCE mode DCDIN is always 0 */
- if (!(usr1 & USR2_DCDIN))
- tmp |= TIOCM_CAR;
-
- /* in DCE mode RIIN is always 0 */
- if (readl(sport->port.membase + USR2) & USR2_RIIN)
- tmp |= TIOCM_RI;
-
- return tmp;
-}
-
static unsigned int imx_get_mctrl(struct uart_port *port)
{
struct imx_port *sport = (struct imx_port *)port;
@@ -861,33 +914,6 @@
}
/*
- * Handle any change of modem status signal since we were last called.
- */
-static void imx_mctrl_check(struct imx_port *sport)
-{
- unsigned int status, changed;
-
- status = imx_get_hwmctrl(sport);
- changed = status ^ sport->old_status;
-
- if (changed == 0)
- return;
-
- sport->old_status = status;
-
- if (changed & TIOCM_RI)
- sport->port.icount.rng++;
- if (changed & TIOCM_DSR)
- sport->port.icount.dsr++;
- if (changed & TIOCM_CAR)
- uart_handle_dcd_change(&sport->port, status & TIOCM_CAR);
- if (changed & TIOCM_CTS)
- uart_handle_cts_change(&sport->port, status & TIOCM_CTS);
-
- wake_up_interruptible(&sport->port.state->port.delta_msr_wait);
-}
-
-/*
* This is our per-port timeout handler, for checking the
* modem status signals.
*/
@@ -1193,7 +1219,7 @@
/*
* Finally, clear and enable interrupts
*/
- writel(USR1_RTSD, sport->port.membase + USR1);
+ writel(USR1_RTSD | USR1_DTRD, sport->port.membase + USR1);
writel(USR2_ORE, sport->port.membase + USR2);
if (sport->dma_is_inited && !sport->dma_is_enabled)
@@ -1212,11 +1238,32 @@
temp |= (UCR2_RXEN | UCR2_TXEN);
if (!sport->have_rtscts)
temp |= UCR2_IRTS;
+ /*
+ * make sure the edge sensitive RTS-irq is disabled,
+ * we're using RTSD instead.
+ */
+ if (!is_imx1_uart(sport))
+ temp &= ~UCR2_RTSEN;
writel(temp, sport->port.membase + UCR2);
if (!is_imx1_uart(sport)) {
temp = readl(sport->port.membase + UCR3);
- temp |= IMX21_UCR3_RXDMUXSEL | UCR3_ADNIMP;
+
+ /*
+ * The effect of RI and DCD differs depending on the UFCR_DCEDTE
+ * bit. In DCE mode they control the outputs, in DTE mode they
+ * enable the respective irqs. At least the DCD irq cannot be
+ * cleared on i.MX25 at least, so it's not usable and must be
+ * disabled. I don't have test hardware to check if RI has the
+ * same problem but I consider this likely so it's disabled for
+ * now, too.
+ */
+ temp |= IMX21_UCR3_RXDMUXSEL | UCR3_ADNIMP |
+ UCR3_DTRDEN | UCR3_RI | UCR3_DCD;
+
+ if (sport->dte_mode)
+ temp &= ~(UCR3_RI | UCR3_DCD);
+
writel(temp, sport->port.membase + UCR3);
}
@@ -1610,19 +1657,17 @@
struct serial_rs485 *rs485conf)
{
struct imx_port *sport = (struct imx_port *)port;
+ unsigned long temp;
/* unimplemented */
rs485conf->delay_rts_before_send = 0;
rs485conf->delay_rts_after_send = 0;
- rs485conf->flags |= SER_RS485_RX_DURING_TX;
/* RTS is required to control the transmitter */
if (!sport->have_rtscts)
rs485conf->flags &= ~SER_RS485_ENABLED;
if (rs485conf->flags & SER_RS485_ENABLED) {
- unsigned long temp;
-
/* disable transmitter */
temp = readl(sport->port.membase + UCR2);
if (rs485conf->flags & SER_RS485_RTS_AFTER_SEND)
@@ -1632,6 +1677,14 @@
writel(temp, sport->port.membase + UCR2);
}
+ /* Make sure Rx is enabled in case Tx is active with Rx disabled */
+ if (!(rs485conf->flags & SER_RS485_ENABLED) ||
+ rs485conf->flags & SER_RS485_RX_DURING_TX) {
+ temp = readl(sport->port.membase + UCR2);
+ temp |= UCR2_RXEN;
+ writel(temp, sport->port.membase + UCR2);
+ }
+
port->rs485 = *rs485conf;
return 0;
@@ -1927,7 +1980,8 @@
}
sport->port.line = ret;
- if (of_get_property(np, "fsl,uart-has-rtscts", NULL))
+ if (of_get_property(np, "uart-has-rtscts", NULL) ||
+ of_get_property(np, "fsl,uart-has-rtscts", NULL) /* deprecated */)
sport->have_rtscts = 1;
if (of_get_property(np, "fsl,dte-mode", NULL))
diff --git a/drivers/tty/serial/meson_uart.c b/drivers/tty/serial/meson_uart.c
index 024445a..6aea0f4 100644
--- a/drivers/tty/serial/meson_uart.c
+++ b/drivers/tty/serial/meson_uart.c
@@ -481,18 +481,13 @@
writel(ch, port->membase + AML_UART_WFIFO);
}
-static void meson_serial_console_write(struct console *co, const char *s,
- u_int count)
+static void meson_serial_port_write(struct uart_port *port, const char *s,
+ u_int count)
{
- struct uart_port *port;
unsigned long flags;
int locked;
u32 val, tmp;
- port = meson_ports[co->index];
- if (!port)
- return;
-
local_irq_save(flags);
if (port->sysrq) {
locked = 0;
@@ -516,6 +511,18 @@
local_irq_restore(flags);
}
+static void meson_serial_console_write(struct console *co, const char *s,
+ u_int count)
+{
+ struct uart_port *port;
+
+ port = meson_ports[co->index];
+ if (!port)
+ return;
+
+ meson_serial_port_write(port, s, count);
+}
+
static int meson_serial_console_setup(struct console *co, char *options)
{
struct uart_port *port;
@@ -554,6 +561,27 @@
}
console_initcall(meson_serial_console_init);
+static void meson_serial_early_console_write(struct console *co,
+ const char *s,
+ u_int count)
+{
+ struct earlycon_device *dev = co->data;
+
+ meson_serial_port_write(&dev->port, s, count);
+}
+
+static int __init
+meson_serial_early_console_setup(struct earlycon_device *device, const char *opt)
+{
+ if (!device->port.membase)
+ return -ENODEV;
+
+ device->con->write = meson_serial_early_console_write;
+ return 0;
+}
+OF_EARLYCON_DECLARE(meson, "amlogic,meson-uart",
+ meson_serial_early_console_setup);
+
#define MESON_SERIAL_CONSOLE (&meson_serial_console)
#else
#define MESON_SERIAL_CONSOLE NULL
diff --git a/drivers/tty/serial/mps2-uart.c b/drivers/tty/serial/mps2-uart.c
new file mode 100644
index 0000000..da9e27d
--- /dev/null
+++ b/drivers/tty/serial/mps2-uart.c
@@ -0,0 +1,625 @@
+/*
+ * Copyright (C) 2015 ARM Limited
+ *
+ * Author: Vladimir Murzin <vladimir.murzin@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * TODO: support for SysRq
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/bitops.h>
+#include <linux/clk.h>
+#include <linux/console.h>
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/serial_core.h>
+#include <linux/tty_flip.h>
+#include <linux/types.h>
+
+#define SERIAL_NAME "ttyMPS"
+#define DRIVER_NAME "mps2-uart"
+#define MAKE_NAME(x) (DRIVER_NAME # x)
+
+#define UARTn_DATA 0x00
+
+#define UARTn_STATE 0x04
+#define UARTn_STATE_TX_FULL BIT(0)
+#define UARTn_STATE_RX_FULL BIT(1)
+#define UARTn_STATE_TX_OVERRUN BIT(2)
+#define UARTn_STATE_RX_OVERRUN BIT(3)
+
+#define UARTn_CTRL 0x08
+#define UARTn_CTRL_TX_ENABLE BIT(0)
+#define UARTn_CTRL_RX_ENABLE BIT(1)
+#define UARTn_CTRL_TX_INT_ENABLE BIT(2)
+#define UARTn_CTRL_RX_INT_ENABLE BIT(3)
+#define UARTn_CTRL_TX_OVERRUN_INT_ENABLE BIT(4)
+#define UARTn_CTRL_RX_OVERRUN_INT_ENABLE BIT(5)
+
+#define UARTn_INT 0x0c
+#define UARTn_INT_TX BIT(0)
+#define UARTn_INT_RX BIT(1)
+#define UARTn_INT_TX_OVERRUN BIT(2)
+#define UARTn_INT_RX_OVERRUN BIT(3)
+
+#define UARTn_BAUDDIV 0x10
+#define UARTn_BAUDDIV_MASK GENMASK(20, 0)
+
+/*
+ * Helpers to make typical enable/disable operations more readable.
+ */
+#define UARTn_CTRL_TX_GRP (UARTn_CTRL_TX_ENABLE |\
+ UARTn_CTRL_TX_INT_ENABLE |\
+ UARTn_CTRL_TX_OVERRUN_INT_ENABLE)
+
+#define UARTn_CTRL_RX_GRP (UARTn_CTRL_RX_ENABLE |\
+ UARTn_CTRL_RX_INT_ENABLE |\
+ UARTn_CTRL_RX_OVERRUN_INT_ENABLE)
+
+#define MPS2_MAX_PORTS 3
+
+struct mps2_uart_port {
+ struct uart_port port;
+ struct clk *clk;
+ unsigned int tx_irq;
+ unsigned int rx_irq;
+};
+
+static inline struct mps2_uart_port *to_mps2_port(struct uart_port *port)
+{
+ return container_of(port, struct mps2_uart_port, port);
+}
+
+static void mps2_uart_write8(struct uart_port *port, u8 val, unsigned int off)
+{
+ struct mps2_uart_port *mps_port = to_mps2_port(port);
+
+ writeb(val, mps_port->port.membase + off);
+}
+
+static u8 mps2_uart_read8(struct uart_port *port, unsigned int off)
+{
+ struct mps2_uart_port *mps_port = to_mps2_port(port);
+
+ return readb(mps_port->port.membase + off);
+}
+
+static void mps2_uart_write32(struct uart_port *port, u32 val, unsigned int off)
+{
+ struct mps2_uart_port *mps_port = to_mps2_port(port);
+
+ writel_relaxed(val, mps_port->port.membase + off);
+}
+
+static unsigned int mps2_uart_tx_empty(struct uart_port *port)
+{
+ u8 status = mps2_uart_read8(port, UARTn_STATE);
+
+ return (status & UARTn_STATE_TX_FULL) ? 0 : TIOCSER_TEMT;
+}
+
+static void mps2_uart_set_mctrl(struct uart_port *port, unsigned int mctrl)
+{
+}
+
+static unsigned int mps2_uart_get_mctrl(struct uart_port *port)
+{
+ return TIOCM_CAR | TIOCM_CTS | TIOCM_DSR;
+}
+
+static void mps2_uart_stop_tx(struct uart_port *port)
+{
+ u8 control = mps2_uart_read8(port, UARTn_CTRL);
+
+ control &= ~UARTn_CTRL_TX_INT_ENABLE;
+
+ mps2_uart_write8(port, control, UARTn_CTRL);
+}
+
+static void mps2_uart_tx_chars(struct uart_port *port)
+{
+ struct circ_buf *xmit = &port->state->xmit;
+
+ while (!(mps2_uart_read8(port, UARTn_STATE) & UARTn_STATE_TX_FULL)) {
+ if (port->x_char) {
+ mps2_uart_write8(port, port->x_char, UARTn_DATA);
+ port->x_char = 0;
+ port->icount.tx++;
+ continue;
+ }
+
+ if (uart_circ_empty(xmit) || uart_tx_stopped(port))
+ break;
+
+ mps2_uart_write8(port, xmit->buf[xmit->tail], UARTn_DATA);
+ xmit->tail = (xmit->tail + 1) % UART_XMIT_SIZE;
+ port->icount.tx++;
+ }
+
+ if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
+ uart_write_wakeup(port);
+
+ if (uart_circ_empty(xmit))
+ mps2_uart_stop_tx(port);
+}
+
+static void mps2_uart_start_tx(struct uart_port *port)
+{
+ u8 control = mps2_uart_read8(port, UARTn_CTRL);
+
+ control |= UARTn_CTRL_TX_INT_ENABLE;
+
+ mps2_uart_write8(port, control, UARTn_CTRL);
+
+ /*
+ * We've just unmasked the TX IRQ and now slow-starting via
+ * polling; if there is enough data to fill up the internal
+ * write buffer in one go, the TX IRQ should assert, at which
+ * point we switch to fully interrupt-driven TX.
+ */
+
+ mps2_uart_tx_chars(port);
+}
+
+static void mps2_uart_stop_rx(struct uart_port *port)
+{
+ u8 control = mps2_uart_read8(port, UARTn_CTRL);
+
+ control &= ~UARTn_CTRL_RX_GRP;
+
+ mps2_uart_write8(port, control, UARTn_CTRL);
+}
+
+static void mps2_uart_break_ctl(struct uart_port *port, int ctl)
+{
+}
+
+static void mps2_uart_rx_chars(struct uart_port *port)
+{
+ struct tty_port *tport = &port->state->port;
+
+ while (mps2_uart_read8(port, UARTn_STATE) & UARTn_STATE_RX_FULL) {
+ u8 rxdata = mps2_uart_read8(port, UARTn_DATA);
+
+ port->icount.rx++;
+ tty_insert_flip_char(&port->state->port, rxdata, TTY_NORMAL);
+ }
+
+ tty_flip_buffer_push(tport);
+}
+
+static irqreturn_t mps2_uart_rxirq(int irq, void *data)
+{
+ struct uart_port *port = data;
+ u8 irqflag = mps2_uart_read8(port, UARTn_INT);
+
+ if (unlikely(!(irqflag & UARTn_INT_RX)))
+ return IRQ_NONE;
+
+ spin_lock(&port->lock);
+
+ mps2_uart_write8(port, UARTn_INT_RX, UARTn_INT);
+ mps2_uart_rx_chars(port);
+
+ spin_unlock(&port->lock);
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t mps2_uart_txirq(int irq, void *data)
+{
+ struct uart_port *port = data;
+ u8 irqflag = mps2_uart_read8(port, UARTn_INT);
+
+ if (unlikely(!(irqflag & UARTn_INT_TX)))
+ return IRQ_NONE;
+
+ spin_lock(&port->lock);
+
+ mps2_uart_write8(port, UARTn_INT_TX, UARTn_INT);
+ mps2_uart_tx_chars(port);
+
+ spin_unlock(&port->lock);
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t mps2_uart_oerrirq(int irq, void *data)
+{
+ irqreturn_t handled = IRQ_NONE;
+ struct uart_port *port = data;
+ u8 irqflag = mps2_uart_read8(port, UARTn_INT);
+
+ spin_lock(&port->lock);
+
+ if (irqflag & UARTn_INT_RX_OVERRUN) {
+ struct tty_port *tport = &port->state->port;
+
+ mps2_uart_write8(port, UARTn_INT_RX_OVERRUN, UARTn_INT);
+ port->icount.overrun++;
+ tty_insert_flip_char(tport, 0, TTY_OVERRUN);
+ tty_flip_buffer_push(tport);
+ handled = IRQ_HANDLED;
+ }
+
+ /*
+ * It's never been seen in practice and it never *should* happen since
+ * we check if there is enough room in TX buffer before sending data.
+ * So we keep this check in case something suspicious has happened.
+ */
+ if (irqflag & UARTn_INT_TX_OVERRUN) {
+ mps2_uart_write8(port, UARTn_INT_TX_OVERRUN, UARTn_INT);
+ handled = IRQ_HANDLED;
+ }
+
+ spin_unlock(&port->lock);
+
+ return handled;
+}
+
+static int mps2_uart_startup(struct uart_port *port)
+{
+ struct mps2_uart_port *mps_port = to_mps2_port(port);
+ u8 control = mps2_uart_read8(port, UARTn_CTRL);
+ int ret;
+
+ control &= ~(UARTn_CTRL_RX_GRP | UARTn_CTRL_TX_GRP);
+
+ mps2_uart_write8(port, control, UARTn_CTRL);
+
+ ret = request_irq(mps_port->rx_irq, mps2_uart_rxirq, 0,
+ MAKE_NAME(-rx), mps_port);
+ if (ret) {
+ dev_err(port->dev, "failed to register rxirq (%d)\n", ret);
+ return ret;
+ }
+
+ ret = request_irq(mps_port->tx_irq, mps2_uart_txirq, 0,
+ MAKE_NAME(-tx), mps_port);
+ if (ret) {
+ dev_err(port->dev, "failed to register txirq (%d)\n", ret);
+ goto err_free_rxirq;
+ }
+
+ ret = request_irq(port->irq, mps2_uart_oerrirq, IRQF_SHARED,
+ MAKE_NAME(-overrun), mps_port);
+
+ if (ret) {
+ dev_err(port->dev, "failed to register oerrirq (%d)\n", ret);
+ goto err_free_txirq;
+ }
+
+ control |= UARTn_CTRL_RX_GRP | UARTn_CTRL_TX_GRP;
+
+ mps2_uart_write8(port, control, UARTn_CTRL);
+
+ return 0;
+
+err_free_txirq:
+ free_irq(mps_port->tx_irq, mps_port);
+err_free_rxirq:
+ free_irq(mps_port->rx_irq, mps_port);
+
+ return ret;
+}
+
+static void mps2_uart_shutdown(struct uart_port *port)
+{
+ struct mps2_uart_port *mps_port = to_mps2_port(port);
+ u8 control = mps2_uart_read8(port, UARTn_CTRL);
+
+ control &= ~(UARTn_CTRL_RX_GRP | UARTn_CTRL_TX_GRP);
+
+ mps2_uart_write8(port, control, UARTn_CTRL);
+
+ free_irq(mps_port->rx_irq, mps_port);
+ free_irq(mps_port->tx_irq, mps_port);
+ free_irq(port->irq, mps_port);
+}
+
+static void
+mps2_uart_set_termios(struct uart_port *port, struct ktermios *termios,
+ struct ktermios *old)
+{
+ unsigned long flags;
+ unsigned int baud, bauddiv;
+
+ termios->c_cflag &= ~(CRTSCTS | CMSPAR);
+ termios->c_cflag &= ~CSIZE;
+ termios->c_cflag |= CS8;
+ termios->c_cflag &= ~PARENB;
+ termios->c_cflag &= ~CSTOPB;
+
+ baud = uart_get_baud_rate(port, termios, old,
+ DIV_ROUND_CLOSEST(port->uartclk, UARTn_BAUDDIV_MASK),
+ DIV_ROUND_CLOSEST(port->uartclk, 16));
+
+ bauddiv = DIV_ROUND_CLOSEST(port->uartclk, baud);
+
+ spin_lock_irqsave(&port->lock, flags);
+
+ uart_update_timeout(port, termios->c_cflag, baud);
+ mps2_uart_write32(port, bauddiv, UARTn_BAUDDIV);
+
+ spin_unlock_irqrestore(&port->lock, flags);
+
+ if (tty_termios_baud_rate(termios))
+ tty_termios_encode_baud_rate(termios, baud, baud);
+}
+
+static const char *mps2_uart_type(struct uart_port *port)
+{
+ return (port->type == PORT_MPS2UART) ? DRIVER_NAME : NULL;
+}
+
+static void mps2_uart_release_port(struct uart_port *port)
+{
+}
+
+static int mps2_uart_request_port(struct uart_port *port)
+{
+ return 0;
+}
+
+static void mps2_uart_config_port(struct uart_port *port, int type)
+{
+ if (type & UART_CONFIG_TYPE && !mps2_uart_request_port(port))
+ port->type = PORT_MPS2UART;
+}
+
+static int mps2_uart_verify_port(struct uart_port *port, struct serial_struct *serinfo)
+{
+ return -EINVAL;
+}
+
+static const struct uart_ops mps2_uart_pops = {
+ .tx_empty = mps2_uart_tx_empty,
+ .set_mctrl = mps2_uart_set_mctrl,
+ .get_mctrl = mps2_uart_get_mctrl,
+ .stop_tx = mps2_uart_stop_tx,
+ .start_tx = mps2_uart_start_tx,
+ .stop_rx = mps2_uart_stop_rx,
+ .break_ctl = mps2_uart_break_ctl,
+ .startup = mps2_uart_startup,
+ .shutdown = mps2_uart_shutdown,
+ .set_termios = mps2_uart_set_termios,
+ .type = mps2_uart_type,
+ .release_port = mps2_uart_release_port,
+ .request_port = mps2_uart_request_port,
+ .config_port = mps2_uart_config_port,
+ .verify_port = mps2_uart_verify_port,
+};
+
+static struct mps2_uart_port mps2_uart_ports[MPS2_MAX_PORTS];
+
+#ifdef CONFIG_SERIAL_MPS2_UART_CONSOLE
+static void mps2_uart_console_putchar(struct uart_port *port, int ch)
+{
+ while (mps2_uart_read8(port, UARTn_STATE) & UARTn_STATE_TX_FULL)
+ cpu_relax();
+
+ mps2_uart_write8(port, ch, UARTn_DATA);
+}
+
+static void mps2_uart_console_write(struct console *co, const char *s, unsigned int cnt)
+{
+ struct uart_port *port = &mps2_uart_ports[co->index].port;
+
+ uart_console_write(port, s, cnt, mps2_uart_console_putchar);
+}
+
+static int mps2_uart_console_setup(struct console *co, char *options)
+{
+ struct mps2_uart_port *mps_port;
+ int baud = 9600;
+ int bits = 8;
+ int parity = 'n';
+ int flow = 'n';
+
+ if (co->index < 0 || co->index >= MPS2_MAX_PORTS)
+ return -ENODEV;
+
+ mps_port = &mps2_uart_ports[co->index];
+
+ if (options)
+ uart_parse_options(options, &baud, &parity, &bits, &flow);
+
+ return uart_set_options(&mps_port->port, co, baud, parity, bits, flow);
+}
+
+static struct uart_driver mps2_uart_driver;
+
+static struct console mps2_uart_console = {
+ .name = SERIAL_NAME,
+ .device = uart_console_device,
+ .write = mps2_uart_console_write,
+ .setup = mps2_uart_console_setup,
+ .flags = CON_PRINTBUFFER,
+ .index = -1,
+ .data = &mps2_uart_driver,
+};
+
+#define MPS2_SERIAL_CONSOLE (&mps2_uart_console)
+
+static void mps2_early_putchar(struct uart_port *port, int ch)
+{
+ while (readb(port->membase + UARTn_STATE) & UARTn_STATE_TX_FULL)
+ cpu_relax();
+
+ writeb((unsigned char)ch, port->membase + UARTn_DATA);
+}
+
+static void mps2_early_write(struct console *con, const char *s, unsigned int n)
+{
+ struct earlycon_device *dev = con->data;
+
+ uart_console_write(&dev->port, s, n, mps2_early_putchar);
+}
+
+static int __init mps2_early_console_setup(struct earlycon_device *device,
+ const char *opt)
+{
+ if (!device->port.membase)
+ return -ENODEV;
+
+ device->con->write = mps2_early_write;
+
+ return 0;
+}
+
+OF_EARLYCON_DECLARE(mps2, "arm,mps2-uart", mps2_early_console_setup);
+
+#else
+#define MPS2_SERIAL_CONSOLE NULL
+#endif
+
+static struct uart_driver mps2_uart_driver = {
+ .driver_name = DRIVER_NAME,
+ .dev_name = SERIAL_NAME,
+ .nr = MPS2_MAX_PORTS,
+ .cons = MPS2_SERIAL_CONSOLE,
+};
+
+static struct mps2_uart_port *mps2_of_get_port(struct platform_device *pdev)
+{
+ struct device_node *np = pdev->dev.of_node;
+ int id;
+
+ if (!np)
+ return NULL;
+
+ id = of_alias_get_id(np, "serial");
+ if (id < 0)
+ id = 0;
+
+ if (WARN_ON(id >= MPS2_MAX_PORTS))
+ return NULL;
+
+ mps2_uart_ports[id].port.line = id;
+ return &mps2_uart_ports[id];
+}
+
+static int mps2_init_port(struct mps2_uart_port *mps_port,
+ struct platform_device *pdev)
+{
+ struct resource *res;
+ int ret;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ mps_port->port.membase = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(mps_port->port.membase))
+ return PTR_ERR(mps_port->port.membase);
+
+ mps_port->port.mapbase = res->start;
+ mps_port->port.mapsize = resource_size(res);
+
+ mps_port->rx_irq = platform_get_irq(pdev, 0);
+ mps_port->tx_irq = platform_get_irq(pdev, 1);
+ mps_port->port.irq = platform_get_irq(pdev, 2);
+
+ mps_port->port.iotype = UPIO_MEM;
+ mps_port->port.flags = UPF_BOOT_AUTOCONF;
+ mps_port->port.fifosize = 1;
+ mps_port->port.ops = &mps2_uart_pops;
+ mps_port->port.dev = &pdev->dev;
+
+ mps_port->clk = devm_clk_get(&pdev->dev, NULL);
+ if (IS_ERR(mps_port->clk))
+ return PTR_ERR(mps_port->clk);
+
+ ret = clk_prepare_enable(mps_port->clk);
+ if (ret)
+ return ret;
+
+ mps_port->port.uartclk = clk_get_rate(mps_port->clk);
+
+ clk_disable_unprepare(mps_port->clk);
+
+ return ret;
+}
+
+static int mps2_serial_probe(struct platform_device *pdev)
+{
+ struct mps2_uart_port *mps_port;
+ int ret;
+
+ mps_port = mps2_of_get_port(pdev);
+ if (!mps_port)
+ return -ENODEV;
+
+ ret = mps2_init_port(mps_port, pdev);
+ if (ret)
+ return ret;
+
+ ret = uart_add_one_port(&mps2_uart_driver, &mps_port->port);
+ if (ret)
+ return ret;
+
+ platform_set_drvdata(pdev, mps_port);
+
+ return 0;
+}
+
+static int mps2_serial_remove(struct platform_device *pdev)
+{
+ struct mps2_uart_port *mps_port = platform_get_drvdata(pdev);
+
+ uart_remove_one_port(&mps2_uart_driver, &mps_port->port);
+
+ return 0;
+}
+
+#ifdef CONFIG_OF
+static const struct of_device_id mps2_match[] = {
+ { .compatible = "arm,mps2-uart", },
+ {},
+};
+MODULE_DEVICE_TABLE(of, mps2_match);
+#endif
+
+static struct platform_driver mps2_serial_driver = {
+ .probe = mps2_serial_probe,
+ .remove = mps2_serial_remove,
+
+ .driver = {
+ .name = DRIVER_NAME,
+ .of_match_table = of_match_ptr(mps2_match),
+ },
+};
+
+static int __init mps2_uart_init(void)
+{
+ int ret;
+
+ ret = uart_register_driver(&mps2_uart_driver);
+ if (ret)
+ return ret;
+
+ ret = platform_driver_register(&mps2_serial_driver);
+ if (ret)
+ uart_unregister_driver(&mps2_uart_driver);
+
+ return ret;
+}
+module_init(mps2_uart_init);
+
+static void __exit mps2_uart_exit(void)
+{
+ platform_driver_unregister(&mps2_serial_driver);
+ uart_unregister_driver(&mps2_uart_driver);
+}
+module_exit(mps2_uart_exit);
+
+MODULE_AUTHOR("Vladimir Murzin <vladimir.murzin@arm.com>");
+MODULE_DESCRIPTION("MPS2 UART driver");
+MODULE_LICENSE("GPL v2");
+MODULE_ALIAS("platform:" DRIVER_NAME);
diff --git a/drivers/tty/serial/msm_serial.c b/drivers/tty/serial/msm_serial.c
index 96d3ce8..b7d80bd 100644
--- a/drivers/tty/serial/msm_serial.c
+++ b/drivers/tty/serial/msm_serial.c
@@ -861,37 +861,72 @@
};
static const struct msm_baud_map *
-msm_find_best_baud(struct uart_port *port, unsigned int baud)
+msm_find_best_baud(struct uart_port *port, unsigned int baud,
+ unsigned long *rate)
{
- unsigned int i, divisor;
- const struct msm_baud_map *entry;
+ struct msm_port *msm_port = UART_TO_MSM(port);
+ unsigned int divisor, result;
+ unsigned long target, old, best_rate = 0, diff, best_diff = ULONG_MAX;
+ const struct msm_baud_map *entry, *end, *best;
static const struct msm_baud_map table[] = {
- { 1536, 0x00, 1 },
- { 768, 0x11, 1 },
- { 384, 0x22, 1 },
- { 192, 0x33, 1 },
- { 96, 0x44, 1 },
- { 48, 0x55, 1 },
- { 32, 0x66, 1 },
- { 24, 0x77, 1 },
- { 16, 0x88, 1 },
- { 12, 0x99, 6 },
- { 8, 0xaa, 6 },
- { 6, 0xbb, 6 },
- { 4, 0xcc, 6 },
- { 3, 0xdd, 8 },
- { 2, 0xee, 16 },
{ 1, 0xff, 31 },
- { 0, 0xff, 31 },
+ { 2, 0xee, 16 },
+ { 3, 0xdd, 8 },
+ { 4, 0xcc, 6 },
+ { 6, 0xbb, 6 },
+ { 8, 0xaa, 6 },
+ { 12, 0x99, 6 },
+ { 16, 0x88, 1 },
+ { 24, 0x77, 1 },
+ { 32, 0x66, 1 },
+ { 48, 0x55, 1 },
+ { 96, 0x44, 1 },
+ { 192, 0x33, 1 },
+ { 384, 0x22, 1 },
+ { 768, 0x11, 1 },
+ { 1536, 0x00, 1 },
};
- divisor = uart_get_divisor(port, baud);
+ best = table; /* Default to smallest divider */
+ target = clk_round_rate(msm_port->clk, 16 * baud);
+ divisor = DIV_ROUND_CLOSEST(target, 16 * baud);
- for (i = 0, entry = table; i < ARRAY_SIZE(table); i++, entry++)
- if (entry->divisor <= divisor)
- break;
+ end = table + ARRAY_SIZE(table);
+ entry = table;
+ while (entry < end) {
+ if (entry->divisor <= divisor) {
+ result = target / entry->divisor / 16;
+ diff = abs(result - baud);
- return entry; /* Default to smallest divider */
+ /* Keep track of best entry */
+ if (diff < best_diff) {
+ best_diff = diff;
+ best = entry;
+ best_rate = target;
+ }
+
+ if (result == baud)
+ break;
+ } else if (entry->divisor > divisor) {
+ old = target;
+ target = clk_round_rate(msm_port->clk, old + 1);
+ /*
+ * The rate didn't get any faster so we can't do
+ * better at dividing it down
+ */
+ if (target == old)
+ break;
+
+ /* Start the divisor search over at this new rate */
+ entry = table;
+ divisor = DIV_ROUND_CLOSEST(target, 16 * baud);
+ continue;
+ }
+ entry++;
+ }
+
+ *rate = best_rate;
+ return best;
}
static int msm_set_baud_rate(struct uart_port *port, unsigned int baud,
@@ -900,22 +935,20 @@
unsigned int rxstale, watermark, mask;
struct msm_port *msm_port = UART_TO_MSM(port);
const struct msm_baud_map *entry;
- unsigned long flags;
-
- entry = msm_find_best_baud(port, baud);
-
- msm_write(port, entry->code, UART_CSR);
-
- if (baud > 460800)
- port->uartclk = baud * 16;
+ unsigned long flags, rate;
flags = *saved_flags;
spin_unlock_irqrestore(&port->lock, flags);
- clk_set_rate(msm_port->clk, port->uartclk);
+ entry = msm_find_best_baud(port, baud, &rate);
+ clk_set_rate(msm_port->clk, rate);
+ baud = rate / 16 / entry->divisor;
spin_lock_irqsave(&port->lock, flags);
*saved_flags = flags;
+ port->uartclk = rate;
+
+ msm_write(port, entry->code, UART_CSR);
/* RX stale watermark */
rxstale = entry->rxstale;
@@ -1577,8 +1610,6 @@
msm_port->pclk = devm_clk_get(&pdev->dev, "iface");
if (IS_ERR(msm_port->pclk))
return PTR_ERR(msm_port->pclk);
-
- clk_set_rate(msm_port->clk, 1843200);
}
port->uartclk = clk_get_rate(msm_port->clk);
diff --git a/drivers/tty/serial/mvebu-uart.c b/drivers/tty/serial/mvebu-uart.c
index 0ff2781..ce362bd 100644
--- a/drivers/tty/serial/mvebu-uart.c
+++ b/drivers/tty/serial/mvebu-uart.c
@@ -1,5 +1,7 @@
/*
* ***************************************************************************
+* Marvell Armada-3700 Serial Driver
+* Author: Wilson Ding <dingwei@marvell.com>
* Copyright (C) 2015 Marvell International Ltd.
* ***************************************************************************
* This program is free software: you can redistribute it and/or modify it
@@ -23,7 +25,6 @@
#include <linux/init.h>
#include <linux/io.h>
#include <linux/iopoll.h>
-#include <linux/module.h>
#include <linux/of.h>
#include <linux/of_address.h>
#include <linux/of_device.h>
@@ -594,30 +595,18 @@
return 0;
}
-static int mvebu_uart_remove(struct platform_device *pdev)
-{
- struct mvebu_uart_data *data = platform_get_drvdata(pdev);
-
- uart_remove_one_port(&mvebu_uart_driver, data->port);
- data->port->private_data = NULL;
- data->port->mapbase = 0;
- return 0;
-}
-
/* Match table for of_platform binding */
static const struct of_device_id mvebu_uart_of_match[] = {
{ .compatible = "marvell,armada-3700-uart", },
{}
};
-MODULE_DEVICE_TABLE(of, mvebu_uart_of_match);
static struct platform_driver mvebu_uart_platform_driver = {
.probe = mvebu_uart_probe,
- .remove = mvebu_uart_remove,
.driver = {
- .owner = THIS_MODULE,
.name = "mvebu-uart",
.of_match_table = of_match_ptr(mvebu_uart_of_match),
+ .suppress_bind_attrs = true,
},
};
@@ -635,16 +624,4 @@
return ret;
}
-
-static void __exit mvebu_uart_exit(void)
-{
- platform_driver_unregister(&mvebu_uart_platform_driver);
- uart_unregister_driver(&mvebu_uart_driver);
-}
-
arch_initcall(mvebu_uart_init);
-module_exit(mvebu_uart_exit);
-
-MODULE_AUTHOR("Wilson Ding <dingwei@marvell.com>");
-MODULE_DESCRIPTION("Marvell Armada-3700 Serial Driver");
-MODULE_LICENSE("GPL");
diff --git a/drivers/tty/serial/mxs-auart.c b/drivers/tty/serial/mxs-auart.c
index cd0414b..eb54e5c 100644
--- a/drivers/tty/serial/mxs-auart.c
+++ b/drivers/tty/serial/mxs-auart.c
@@ -1,17 +1,18 @@
/*
- * Freescale STMP37XX/STMP378X Application UART driver
+ * Application UART driver for:
+ * Freescale STMP37XX/STMP378X
+ * Alphascale ASM9260
*
* Author: dmitry pervushin <dimka@embeddedalley.com>
*
+ * Copyright 2014 Oleksij Rempel <linux@rempel-privat.de>
+ * Provide Alphascale ASM9260 support.
* Copyright 2008-2010 Freescale Semiconductor, Inc.
* Copyright 2008 Embedded Alley Solutions, Inc All Rights Reserved.
*
* The code contained herein is licensed under the GNU General Public
* License. You may obtain a copy of the GNU General Public License
* Version 2 or later at the following locations:
- *
- * http://www.opensource.org/licenses/gpl-license.html
- * http://www.gnu.org/copyleft/gpl.html
*/
#if defined(CONFIG_SERIAL_MXS_AUART_CONSOLE) && defined(CONFIG_MAGIC_SYSRQ)
@@ -51,30 +52,16 @@
#define MXS_AUART_PORTS 5
#define MXS_AUART_FIFO_SIZE 16
+#define SET_REG 0x4
+#define CLR_REG 0x8
+#define TOG_REG 0xc
+
#define AUART_CTRL0 0x00000000
-#define AUART_CTRL0_SET 0x00000004
-#define AUART_CTRL0_CLR 0x00000008
-#define AUART_CTRL0_TOG 0x0000000c
#define AUART_CTRL1 0x00000010
-#define AUART_CTRL1_SET 0x00000014
-#define AUART_CTRL1_CLR 0x00000018
-#define AUART_CTRL1_TOG 0x0000001c
#define AUART_CTRL2 0x00000020
-#define AUART_CTRL2_SET 0x00000024
-#define AUART_CTRL2_CLR 0x00000028
-#define AUART_CTRL2_TOG 0x0000002c
#define AUART_LINECTRL 0x00000030
-#define AUART_LINECTRL_SET 0x00000034
-#define AUART_LINECTRL_CLR 0x00000038
-#define AUART_LINECTRL_TOG 0x0000003c
#define AUART_LINECTRL2 0x00000040
-#define AUART_LINECTRL2_SET 0x00000044
-#define AUART_LINECTRL2_CLR 0x00000048
-#define AUART_LINECTRL2_TOG 0x0000004c
#define AUART_INTR 0x00000050
-#define AUART_INTR_SET 0x00000054
-#define AUART_INTR_CLR 0x00000058
-#define AUART_INTR_TOG 0x0000005c
#define AUART_DATA 0x00000060
#define AUART_STAT 0x00000070
#define AUART_DEBUG 0x00000080
@@ -136,11 +123,301 @@
#define AUART_STAT_FERR (1 << 16)
#define AUART_STAT_RXCOUNT_MASK 0xffff
+/*
+ * Start of Alphascale asm9260 defines
+ * This list contains only differences of existing bits
+ * between imx2x and asm9260
+ */
+#define ASM9260_HW_CTRL0 0x0000
+/*
+ * RW. Tell the UART to execute the RX DMA Command. The
+ * UART will clear this bit at the end of receive execution.
+ */
+#define ASM9260_BM_CTRL0_RXDMA_RUN BIT(28)
+/* RW. 0 use FIFO for status register; 1 use DMA */
+#define ASM9260_BM_CTRL0_RXTO_SOURCE_STATUS BIT(25)
+/*
+ * RW. RX TIMEOUT Enable. Valid for FIFO and DMA.
+ * Warning: If this bit is set to 0, the RX timeout will not affect receive DMA
+ * operation. If this bit is set to 1, a receive timeout will cause the receive
+ * DMA logic to terminate by filling the remaining DMA bytes with garbage data.
+ */
+#define ASM9260_BM_CTRL0_RXTO_ENABLE BIT(24)
+/*
+ * RW. Receive Timeout Counter Value: number of 8-bit-time to wait before
+ * asserting timeout on the RX input. If the RXFIFO is not empty and the RX
+ * input is idle, then the watchdog counter will decrement each bit-time. Note
+ * 7-bit-time is added to the programmed value, so a value of zero will set
+ * the counter to 7-bit-time, a value of 0x1 gives 15-bit-time and so on. Also
+ * note that the counter is reloaded at the end of each frame, so if the frame
+ * is 10 bits long and the timeout counter value is zero, then timeout will
+ * occur (when FIFO is not empty) even if the RX input is not idle. The default
+ * value is 0x3 (31 bit-time).
+ */
+#define ASM9260_BM_CTRL0_RXTO_MASK (0xff << 16)
+/* TIMEOUT = (100*7+1)*(1/BAUD) */
+#define ASM9260_BM_CTRL0_DEFAULT_RXTIMEOUT (20 << 16)
+
+/* TX ctrl register */
+#define ASM9260_HW_CTRL1 0x0010
+/*
+ * RW. Tell the UART to execute the TX DMA Command. The
+ * UART will clear this bit at the end of transmit execution.
+ */
+#define ASM9260_BM_CTRL1_TXDMA_RUN BIT(28)
+
+#define ASM9260_HW_CTRL2 0x0020
+/*
+ * RW. Receive Interrupt FIFO Level Select.
+ * The trigger points for the receive interrupt are as follows:
+ * ONE_EIGHTHS = 0x0 Trigger on FIFO full to at least 2 of 16 entries.
+ * ONE_QUARTER = 0x1 Trigger on FIFO full to at least 4 of 16 entries.
+ * ONE_HALF = 0x2 Trigger on FIFO full to at least 8 of 16 entries.
+ * THREE_QUARTERS = 0x3 Trigger on FIFO full to at least 12 of 16 entries.
+ * SEVEN_EIGHTHS = 0x4 Trigger on FIFO full to at least 14 of 16 entries.
+ */
+#define ASM9260_BM_CTRL2_RXIFLSEL (7 << 20)
+#define ASM9260_BM_CTRL2_DEFAULT_RXIFLSEL (3 << 20)
+/* RW. Same as RXIFLSEL */
+#define ASM9260_BM_CTRL2_TXIFLSEL (7 << 16)
+#define ASM9260_BM_CTRL2_DEFAULT_TXIFLSEL (2 << 16)
+/* RW. Set DTR. When this bit is 1, the output is 0. */
+#define ASM9260_BM_CTRL2_DTR BIT(10)
+/* RW. Loop Back Enable */
+#define ASM9260_BM_CTRL2_LBE BIT(7)
+#define ASM9260_BM_CTRL2_PORT_ENABLE BIT(0)
+
+#define ASM9260_HW_LINECTRL 0x0030
+/*
+ * RW. Stick Parity Select. When bits 1, 2, and 7 of this register are set, the
+ * parity bit is transmitted and checked as a 0. When bits 1 and 7 are set,
+ * and bit 2 is 0, the parity bit is transmitted and checked as a 1. When this
+ * bit is cleared stick parity is disabled.
+ */
+#define ASM9260_BM_LCTRL_SPS BIT(7)
+/* RW. Word length */
+#define ASM9260_BM_LCTRL_WLEN (3 << 5)
+#define ASM9260_BM_LCTRL_CHRL_5 (0 << 5)
+#define ASM9260_BM_LCTRL_CHRL_6 (1 << 5)
+#define ASM9260_BM_LCTRL_CHRL_7 (2 << 5)
+#define ASM9260_BM_LCTRL_CHRL_8 (3 << 5)
+
+/*
+ * Interrupt register.
+ * contains the interrupt enables and the interrupt status bits
+ */
+#define ASM9260_HW_INTR 0x0040
+/* Tx FIFO EMPTY Raw Interrupt enable */
+#define ASM9260_BM_INTR_TFEIEN BIT(27)
+/* Overrun Error Interrupt Enable. */
+#define ASM9260_BM_INTR_OEIEN BIT(26)
+/* Break Error Interrupt Enable. */
+#define ASM9260_BM_INTR_BEIEN BIT(25)
+/* Parity Error Interrupt Enable. */
+#define ASM9260_BM_INTR_PEIEN BIT(24)
+/* Framing Error Interrupt Enable. */
+#define ASM9260_BM_INTR_FEIEN BIT(23)
+
+/* nUARTDSR Modem Interrupt Enable. */
+#define ASM9260_BM_INTR_DSRMIEN BIT(19)
+/* nUARTDCD Modem Interrupt Enable. */
+#define ASM9260_BM_INTR_DCDMIEN BIT(18)
+/* nUARTRI Modem Interrupt Enable. */
+#define ASM9260_BM_INTR_RIMIEN BIT(16)
+/* Auto-Boud Timeout */
+#define ASM9260_BM_INTR_ABTO BIT(13)
+#define ASM9260_BM_INTR_ABEO BIT(12)
+/* Tx FIFO EMPTY Raw Interrupt state */
+#define ASM9260_BM_INTR_TFEIS BIT(11)
+/* Overrun Error */
+#define ASM9260_BM_INTR_OEIS BIT(10)
+/* Break Error */
+#define ASM9260_BM_INTR_BEIS BIT(9)
+/* Parity Error */
+#define ASM9260_BM_INTR_PEIS BIT(8)
+/* Framing Error */
+#define ASM9260_BM_INTR_FEIS BIT(7)
+#define ASM9260_BM_INTR_DSRMIS BIT(3)
+#define ASM9260_BM_INTR_DCDMIS BIT(2)
+#define ASM9260_BM_INTR_RIMIS BIT(0)
+
+/*
+ * RW. In DMA mode, up to 4 Received/Transmit characters can be accessed at a
+ * time. In PIO mode, only one character can be accessed at a time. The status
+ * register contains the receive data flags and valid bits.
+ */
+#define ASM9260_HW_DATA 0x0050
+
+#define ASM9260_HW_STAT 0x0060
+/* RO. If 1, UARTAPP is present in this product. */
+#define ASM9260_BM_STAT_PRESENT BIT(31)
+/* RO. If 1, HISPEED is present in this product. */
+#define ASM9260_BM_STAT_HISPEED BIT(30)
+/* RO. Receive FIFO Full. */
+#define ASM9260_BM_STAT_RXFULL BIT(26)
+
+/* RO. The UART Debug Register contains the state of the DMA signals. */
+#define ASM9260_HW_DEBUG 0x0070
+/* DMA Command Run Status */
+#define ASM9260_BM_DEBUG_TXDMARUN BIT(5)
+#define ASM9260_BM_DEBUG_RXDMARUN BIT(4)
+/* DMA Command End Status */
+#define ASM9260_BM_DEBUG_TXCMDEND BIT(3)
+#define ASM9260_BM_DEBUG_RXCMDEND BIT(2)
+/* DMA Request Status */
+#define ASM9260_BM_DEBUG_TXDMARQ BIT(1)
+#define ASM9260_BM_DEBUG_RXDMARQ BIT(0)
+
+#define ASM9260_HW_ILPR 0x0080
+
+#define ASM9260_HW_RS485CTRL 0x0090
+/*
+ * RW. This bit reverses the polarity of the direction control signal on the RTS
+ * (or DTR) pin.
+ * If 0, The direction control pin will be driven to logic ‘0’ when the
+ * transmitter has data to be sent. It will be driven to logic ‘1’ after the
+ * last bit of data has been transmitted.
+ */
+#define ASM9260_BM_RS485CTRL_ONIV BIT(5)
+/* RW. Enable Auto Direction Control. */
+#define ASM9260_BM_RS485CTRL_DIR_CTRL BIT(4)
+/*
+ * RW. If 0 and DIR_CTRL = 1, pin RTS is used for direction control.
+ * If 1 and DIR_CTRL = 1, pin DTR is used for direction control.
+ */
+#define ASM9260_BM_RS485CTRL_PINSEL BIT(3)
+/* RW. Enable Auto Address Detect (AAD). */
+#define ASM9260_BM_RS485CTRL_AADEN BIT(2)
+/* RW. Disable receiver. */
+#define ASM9260_BM_RS485CTRL_RXDIS BIT(1)
+/* RW. Enable RS-485/EIA-485 Normal Multidrop Mode (NMM) */
+#define ASM9260_BM_RS485CTRL_RS485EN BIT(0)
+
+#define ASM9260_HW_RS485ADRMATCH 0x00a0
+/* Contains the address match value. */
+#define ASM9260_BM_RS485ADRMATCH_MASK (0xff << 0)
+
+#define ASM9260_HW_RS485DLY 0x00b0
+/*
+ * RW. Contains the direction control (RTS or DTR) delay value. This delay time
+ * is in periods of the baud clock.
+ */
+#define ASM9260_BM_RS485DLY_MASK (0xff << 0)
+
+#define ASM9260_HW_AUTOBAUD 0x00c0
+/* WO. Auto-baud time-out interrupt clear bit. */
+#define ASM9260_BM_AUTOBAUD_TO_INT_CLR BIT(9)
+/* WO. End of auto-baud interrupt clear bit. */
+#define ASM9260_BM_AUTOBAUD_EO_INT_CLR BIT(8)
+/* Restart in case of timeout (counter restarts at next UART Rx falling edge) */
+#define ASM9260_BM_AUTOBAUD_AUTORESTART BIT(2)
+/* Auto-baud mode select bit. 0 - Mode 0, 1 - Mode 1. */
+#define ASM9260_BM_AUTOBAUD_MODE BIT(1)
+/*
+ * Auto-baud start (auto-baud is running). Auto-baud run bit. This bit is
+ * automatically cleared after auto-baud completion.
+ */
+#define ASM9260_BM_AUTOBAUD_START BIT(0)
+
+#define ASM9260_HW_CTRL3 0x00d0
+#define ASM9260_BM_CTRL3_OUTCLK_DIV_MASK (0xffff << 16)
+/*
+ * RW. Provide clk over OUTCLK pin. In case of asm9260 it can be configured on
+ * pins 137 and 144.
+ */
+#define ASM9260_BM_CTRL3_MASTERMODE BIT(6)
+/* RW. Baud Rate Mode: 1 - Enable sync mode. 0 - async mode. */
+#define ASM9260_BM_CTRL3_SYNCMODE BIT(4)
+/* RW. 1 - MSB bit send frist; 0 - LSB bit frist. */
+#define ASM9260_BM_CTRL3_MSBF BIT(2)
+/* RW. 1 - sample rate = 8 x Baudrate; 0 - sample rate = 16 x Baudrate. */
+#define ASM9260_BM_CTRL3_BAUD8 BIT(1)
+/* RW. 1 - Set word length to 9bit. 0 - use ASM9260_BM_LCTRL_WLEN */
+#define ASM9260_BM_CTRL3_9BIT BIT(0)
+
+#define ASM9260_HW_ISO7816_CTRL 0x00e0
+/* RW. Enable High Speed mode. */
+#define ASM9260_BM_ISO7816CTRL_HS BIT(12)
+/* Disable Successive Receive NACK */
+#define ASM9260_BM_ISO7816CTRL_DS_NACK BIT(8)
+#define ASM9260_BM_ISO7816CTRL_MAX_ITER_MASK (0xff << 4)
+/* Receive NACK Inhibit */
+#define ASM9260_BM_ISO7816CTRL_INACK BIT(3)
+#define ASM9260_BM_ISO7816CTRL_NEG_DATA BIT(2)
+/* RW. 1 - ISO7816 mode; 0 - USART mode */
+#define ASM9260_BM_ISO7816CTRL_ENABLE BIT(0)
+
+#define ASM9260_HW_ISO7816_ERRCNT 0x00f0
+/* Parity error counter. Will be cleared after reading */
+#define ASM9260_BM_ISO7816_NB_ERRORS_MASK (0xff << 0)
+
+#define ASM9260_HW_ISO7816_STATUS 0x0100
+/* Max number of Repetitions Reached */
+#define ASM9260_BM_ISO7816_STAT_ITERATION BIT(0)
+
+/* End of Alphascale asm9260 defines */
+
static struct uart_driver auart_driver;
enum mxs_auart_type {
IMX23_AUART,
IMX28_AUART,
+ ASM9260_AUART,
+};
+
+struct vendor_data {
+ const u16 *reg_offset;
+};
+
+enum {
+ REG_CTRL0,
+ REG_CTRL1,
+ REG_CTRL2,
+ REG_LINECTRL,
+ REG_LINECTRL2,
+ REG_INTR,
+ REG_DATA,
+ REG_STAT,
+ REG_DEBUG,
+ REG_VERSION,
+ REG_AUTOBAUD,
+
+ /* The size of the array - must be last */
+ REG_ARRAY_SIZE,
+};
+
+static const u16 mxs_asm9260_offsets[REG_ARRAY_SIZE] = {
+ [REG_CTRL0] = ASM9260_HW_CTRL0,
+ [REG_CTRL1] = ASM9260_HW_CTRL1,
+ [REG_CTRL2] = ASM9260_HW_CTRL2,
+ [REG_LINECTRL] = ASM9260_HW_LINECTRL,
+ [REG_INTR] = ASM9260_HW_INTR,
+ [REG_DATA] = ASM9260_HW_DATA,
+ [REG_STAT] = ASM9260_HW_STAT,
+ [REG_DEBUG] = ASM9260_HW_DEBUG,
+ [REG_AUTOBAUD] = ASM9260_HW_AUTOBAUD,
+};
+
+static const u16 mxs_stmp37xx_offsets[REG_ARRAY_SIZE] = {
+ [REG_CTRL0] = AUART_CTRL0,
+ [REG_CTRL1] = AUART_CTRL1,
+ [REG_CTRL2] = AUART_CTRL2,
+ [REG_LINECTRL] = AUART_LINECTRL,
+ [REG_LINECTRL2] = AUART_LINECTRL2,
+ [REG_INTR] = AUART_INTR,
+ [REG_DATA] = AUART_DATA,
+ [REG_STAT] = AUART_STAT,
+ [REG_DEBUG] = AUART_DEBUG,
+ [REG_VERSION] = AUART_VERSION,
+ [REG_AUTOBAUD] = AUART_AUTOBAUD,
+};
+
+static const struct vendor_data vendor_alphascale_asm9260 = {
+ .reg_offset = mxs_asm9260_offsets,
+};
+
+static const struct vendor_data vendor_freescale_stmp37xx = {
+ .reg_offset = mxs_stmp37xx_offsets,
};
struct mxs_auart_port {
@@ -153,8 +430,10 @@
unsigned long flags;
unsigned int mctrl_prev;
enum mxs_auart_type devtype;
+ const struct vendor_data *vendor;
struct clk *clk;
+ struct clk *clk_ahb;
struct device *dev;
/* for DMA */
@@ -174,6 +453,7 @@
static const struct platform_device_id mxs_auart_devtype[] = {
{ .name = "mxs-auart-imx23", .driver_data = IMX23_AUART },
{ .name = "mxs-auart-imx28", .driver_data = IMX28_AUART },
+ { .name = "as-auart-asm9260", .driver_data = ASM9260_AUART },
{ /* sentinel */ }
};
MODULE_DEVICE_TABLE(platform, mxs_auart_devtype);
@@ -185,6 +465,9 @@
}, {
.compatible = "fsl,imx23-auart",
.data = &mxs_auart_devtype[IMX23_AUART]
+ }, {
+ .compatible = "alphascale,asm9260-auart",
+ .data = &mxs_auart_devtype[ASM9260_AUART]
}, { /* sentinel */ }
};
MODULE_DEVICE_TABLE(of, mxs_auart_dt_ids);
@@ -194,11 +477,54 @@
return s->devtype == IMX28_AUART;
}
+static inline int is_asm9260_auart(struct mxs_auart_port *s)
+{
+ return s->devtype == ASM9260_AUART;
+}
+
static inline bool auart_dma_enabled(struct mxs_auart_port *s)
{
return s->flags & MXS_AUART_DMA_ENABLED;
}
+static unsigned int mxs_reg_to_offset(const struct mxs_auart_port *uap,
+ unsigned int reg)
+{
+ return uap->vendor->reg_offset[reg];
+}
+
+static unsigned int mxs_read(const struct mxs_auart_port *uap,
+ unsigned int reg)
+{
+ void __iomem *addr = uap->port.membase + mxs_reg_to_offset(uap, reg);
+
+ return readl_relaxed(addr);
+}
+
+static void mxs_write(unsigned int val, struct mxs_auart_port *uap,
+ unsigned int reg)
+{
+ void __iomem *addr = uap->port.membase + mxs_reg_to_offset(uap, reg);
+
+ writel_relaxed(val, addr);
+}
+
+static void mxs_set(unsigned int val, struct mxs_auart_port *uap,
+ unsigned int reg)
+{
+ void __iomem *addr = uap->port.membase + mxs_reg_to_offset(uap, reg);
+
+ writel_relaxed(val, addr + SET_REG);
+}
+
+static void mxs_clr(unsigned int val, struct mxs_auart_port *uap,
+ unsigned int reg)
+{
+ void __iomem *addr = uap->port.membase + mxs_reg_to_offset(uap, reg);
+
+ writel_relaxed(val, addr + CLR_REG);
+}
+
static void mxs_auart_stop_tx(struct uart_port *u);
#define to_auart_port(u) container_of(u, struct mxs_auart_port, port)
@@ -295,19 +621,16 @@
}
- while (!(readl(s->port.membase + AUART_STAT) &
- AUART_STAT_TXFF)) {
+ while (!(mxs_read(s, REG_STAT) & AUART_STAT_TXFF)) {
if (s->port.x_char) {
s->port.icount.tx++;
- writel(s->port.x_char,
- s->port.membase + AUART_DATA);
+ mxs_write(s->port.x_char, s, REG_DATA);
s->port.x_char = 0;
continue;
}
if (!uart_circ_empty(xmit) && !uart_tx_stopped(&s->port)) {
s->port.icount.tx++;
- writel(xmit->buf[xmit->tail],
- s->port.membase + AUART_DATA);
+ mxs_write(xmit->buf[xmit->tail], s, REG_DATA);
xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
} else
break;
@@ -316,11 +639,9 @@
uart_write_wakeup(&s->port);
if (uart_circ_empty(&(s->port.state->xmit)))
- writel(AUART_INTR_TXIEN,
- s->port.membase + AUART_INTR_CLR);
+ mxs_clr(AUART_INTR_TXIEN, s, REG_INTR);
else
- writel(AUART_INTR_TXIEN,
- s->port.membase + AUART_INTR_SET);
+ mxs_set(AUART_INTR_TXIEN, s, REG_INTR);
if (uart_tx_stopped(&s->port))
mxs_auart_stop_tx(&s->port);
@@ -332,8 +653,8 @@
u32 stat;
u8 c;
- c = readl(s->port.membase + AUART_DATA);
- stat = readl(s->port.membase + AUART_STAT);
+ c = mxs_read(s, REG_DATA);
+ stat = mxs_read(s, REG_STAT);
flag = TTY_NORMAL;
s->port.icount.rx++;
@@ -368,7 +689,7 @@
uart_insert_char(&s->port, stat, AUART_STAT_OERR, c, flag);
out:
- writel(stat, s->port.membase + AUART_STAT);
+ mxs_write(stat, s, REG_STAT);
}
static void mxs_auart_rx_chars(struct mxs_auart_port *s)
@@ -376,13 +697,13 @@
u32 stat = 0;
for (;;) {
- stat = readl(s->port.membase + AUART_STAT);
+ stat = mxs_read(s, REG_STAT);
if (stat & AUART_STAT_RXFE)
break;
mxs_auart_rx_char(s);
}
- writel(stat, s->port.membase + AUART_STAT);
+ mxs_write(stat, s, REG_STAT);
tty_flip_buffer_push(&s->port.state->port);
}
@@ -418,7 +739,7 @@
{
struct mxs_auart_port *s = to_auart_port(u);
- u32 ctrl = readl(u->membase + AUART_CTRL2);
+ u32 ctrl = mxs_read(s, REG_CTRL2);
ctrl &= ~(AUART_CTRL2_RTSEN | AUART_CTRL2_RTS);
if (mctrl & TIOCM_RTS) {
@@ -428,7 +749,7 @@
ctrl |= AUART_CTRL2_RTS;
}
- writel(ctrl, u->membase + AUART_CTRL2);
+ mxs_write(ctrl, s, REG_CTRL2);
mctrl_gpio_set(s->gpios, mctrl);
}
@@ -459,7 +780,7 @@
static u32 mxs_auart_get_mctrl(struct uart_port *u)
{
struct mxs_auart_port *s = to_auart_port(u);
- u32 stat = readl(u->membase + AUART_STAT);
+ u32 stat = mxs_read(s, REG_STAT);
u32 mctrl = 0;
if (stat & AUART_STAT_CTS)
@@ -536,14 +857,14 @@
dma_unmap_sg(s->dev, &s->rx_sgl, 1, DMA_FROM_DEVICE);
- stat = readl(s->port.membase + AUART_STAT);
+ stat = mxs_read(s, REG_STAT);
stat &= ~(AUART_STAT_OERR | AUART_STAT_BERR |
AUART_STAT_PERR | AUART_STAT_FERR);
count = stat & AUART_STAT_RXCOUNT_MASK;
tty_insert_flip_string(port, s->rx_dma_buf, count);
- writel(stat, s->port.membase + AUART_STAT);
+ mxs_write(stat, s, REG_STAT);
tty_flip_buffer_push(port);
/* start the next DMA for RX. */
@@ -606,8 +927,8 @@
static void mxs_auart_dma_exit(struct mxs_auart_port *s)
{
- writel(AUART_CTRL2_TXDMAE | AUART_CTRL2_RXDMAE | AUART_CTRL2_DMAONERR,
- s->port.membase + AUART_CTRL2_CLR);
+ mxs_clr(AUART_CTRL2_TXDMAE | AUART_CTRL2_RXDMAE | AUART_CTRL2_DMAONERR,
+ s, REG_CTRL2);
mxs_auart_dma_exit_channel(s);
s->flags &= ~MXS_AUART_DMA_ENABLED;
@@ -666,7 +987,7 @@
cflag = termios->c_cflag;
ctrl = AUART_LINECTRL_FEN;
- ctrl2 = readl(u->membase + AUART_CTRL2);
+ ctrl2 = mxs_read(s, REG_CTRL2);
/* byte size */
switch (cflag & CSIZE) {
@@ -754,15 +1075,24 @@
}
/* set baud rate */
- baud_min = DIV_ROUND_UP(u->uartclk * 32, AUART_LINECTRL_BAUD_DIV_MAX);
- baud_max = u->uartclk * 32 / AUART_LINECTRL_BAUD_DIV_MIN;
- baud = uart_get_baud_rate(u, termios, old, baud_min, baud_max);
- div = u->uartclk * 32 / baud;
+ if (is_asm9260_auart(s)) {
+ baud = uart_get_baud_rate(u, termios, old,
+ u->uartclk * 4 / 0x3FFFFF,
+ u->uartclk / 16);
+ div = u->uartclk * 4 / baud;
+ } else {
+ baud_min = DIV_ROUND_UP(u->uartclk * 32,
+ AUART_LINECTRL_BAUD_DIV_MAX);
+ baud_max = u->uartclk * 32 / AUART_LINECTRL_BAUD_DIV_MIN;
+ baud = uart_get_baud_rate(u, termios, old, baud_min, baud_max);
+ div = u->uartclk * 32 / baud;
+ }
+
ctrl |= AUART_LINECTRL_BAUD_DIVFRAC(div & 0x3F);
ctrl |= AUART_LINECTRL_BAUD_DIVINT(div >> 6);
+ mxs_write(ctrl, s, REG_LINECTRL);
- writel(ctrl, u->membase + AUART_LINECTRL);
- writel(ctrl2, u->membase + AUART_CTRL2);
+ mxs_write(ctrl2, s, REG_CTRL2);
uart_update_timeout(u, termios->c_cflag, baud);
@@ -771,8 +1101,8 @@
!test_and_set_bit(MXS_AUART_DMA_RX_READY, &s->flags)) {
if (!mxs_auart_dma_prep_rx(s)) {
/* Disable the normal RX interrupt. */
- writel(AUART_INTR_RXIEN | AUART_INTR_RTIEN,
- u->membase + AUART_INTR_CLR);
+ mxs_clr(AUART_INTR_RXIEN | AUART_INTR_RTIEN,
+ s, REG_INTR);
} else {
mxs_auart_dma_exit(s);
dev_err(s->dev, "We can not start up the DMA.\n");
@@ -802,16 +1132,13 @@
u32 istat;
struct mxs_auart_port *s = context;
u32 mctrl_temp = s->mctrl_prev;
- u32 stat = readl(s->port.membase + AUART_STAT);
+ u32 stat = mxs_read(s, REG_STAT);
- istat = readl(s->port.membase + AUART_INTR);
+ istat = mxs_read(s, REG_INTR);
/* ack irq */
- writel(istat & (AUART_INTR_RTIS
- | AUART_INTR_TXIS
- | AUART_INTR_RXIS
- | AUART_INTR_CTSMIS),
- s->port.membase + AUART_INTR_CLR);
+ mxs_clr(istat & (AUART_INTR_RTIS | AUART_INTR_TXIS | AUART_INTR_RXIS
+ | AUART_INTR_CTSMIS), s, REG_INTR);
/*
* Dealing with GPIO interrupt
@@ -827,8 +1154,7 @@
if (CTS_AT_AUART() && s->ms_irq_enabled)
uart_handle_cts_change(&s->port,
stat & AUART_STAT_CTS);
- writel(AUART_INTR_CTSMIS,
- s->port.membase + AUART_INTR_CLR);
+ mxs_clr(AUART_INTR_CTSMIS, s, REG_INTR);
istat &= ~AUART_INTR_CTSMIS;
}
@@ -846,44 +1172,44 @@
return IRQ_HANDLED;
}
-static void mxs_auart_reset_deassert(struct uart_port *u)
+static void mxs_auart_reset_deassert(struct mxs_auart_port *s)
{
int i;
unsigned int reg;
- writel(AUART_CTRL0_SFTRST, u->membase + AUART_CTRL0_CLR);
+ mxs_clr(AUART_CTRL0_SFTRST, s, REG_CTRL0);
for (i = 0; i < 10000; i++) {
- reg = readl(u->membase + AUART_CTRL0);
+ reg = mxs_read(s, REG_CTRL0);
if (!(reg & AUART_CTRL0_SFTRST))
break;
udelay(3);
}
- writel(AUART_CTRL0_CLKGATE, u->membase + AUART_CTRL0_CLR);
+ mxs_clr(AUART_CTRL0_CLKGATE, s, REG_CTRL0);
}
-static void mxs_auart_reset_assert(struct uart_port *u)
+static void mxs_auart_reset_assert(struct mxs_auart_port *s)
{
int i;
u32 reg;
- reg = readl(u->membase + AUART_CTRL0);
+ reg = mxs_read(s, REG_CTRL0);
/* if already in reset state, keep it untouched */
if (reg & AUART_CTRL0_SFTRST)
return;
- writel(AUART_CTRL0_CLKGATE, u->membase + AUART_CTRL0_CLR);
- writel(AUART_CTRL0_SFTRST, u->membase + AUART_CTRL0_SET);
+ mxs_clr(AUART_CTRL0_CLKGATE, s, REG_CTRL0);
+ mxs_set(AUART_CTRL0_SFTRST, s, REG_CTRL0);
for (i = 0; i < 1000; i++) {
- reg = readl(u->membase + AUART_CTRL0);
+ reg = mxs_read(s, REG_CTRL0);
/* reset is finished when the clock is gated */
if (reg & AUART_CTRL0_CLKGATE)
return;
udelay(10);
}
- dev_err(u->dev, "Failed to reset the unit.");
+ dev_err(s->dev, "Failed to reset the unit.");
}
static int mxs_auart_startup(struct uart_port *u)
@@ -896,17 +1222,17 @@
return ret;
if (uart_console(u)) {
- writel(AUART_CTRL0_CLKGATE, u->membase + AUART_CTRL0_CLR);
+ mxs_clr(AUART_CTRL0_CLKGATE, s, REG_CTRL0);
} else {
/* reset the unit to a well known state */
- mxs_auart_reset_assert(u);
- mxs_auart_reset_deassert(u);
+ mxs_auart_reset_assert(s);
+ mxs_auart_reset_deassert(s);
}
- writel(AUART_CTRL2_UARTEN, u->membase + AUART_CTRL2_SET);
+ mxs_set(AUART_CTRL2_UARTEN, s, REG_CTRL2);
- writel(AUART_INTR_RXIEN | AUART_INTR_RTIEN | AUART_INTR_CTSMIEN,
- u->membase + AUART_INTR);
+ mxs_write(AUART_INTR_RXIEN | AUART_INTR_RTIEN | AUART_INTR_CTSMIEN,
+ s, REG_INTR);
/* Reset FIFO size (it could have changed if DMA was enabled) */
u->fifosize = MXS_AUART_FIFO_SIZE;
@@ -915,7 +1241,7 @@
* Enable fifo so all four bytes of a DMA word are written to
* output (otherwise, only the LSB is written, ie. 1 in 4 bytes)
*/
- writel(AUART_LINECTRL_FEN, u->membase + AUART_LINECTRL_SET);
+ mxs_set(AUART_LINECTRL_FEN, s, REG_LINECTRL);
/* get initial status of modem lines */
mctrl_gpio_get(s->gpios, &s->mctrl_prev);
@@ -934,12 +1260,13 @@
mxs_auart_dma_exit(s);
if (uart_console(u)) {
- writel(AUART_CTRL2_UARTEN, u->membase + AUART_CTRL2_CLR);
- writel(AUART_INTR_RXIEN | AUART_INTR_RTIEN | AUART_INTR_CTSMIEN,
- u->membase + AUART_INTR_CLR);
- writel(AUART_CTRL0_CLKGATE, u->membase + AUART_CTRL0_SET);
+ mxs_clr(AUART_CTRL2_UARTEN, s, REG_CTRL2);
+
+ mxs_clr(AUART_INTR_RXIEN | AUART_INTR_RTIEN |
+ AUART_INTR_CTSMIEN, s, REG_INTR);
+ mxs_set(AUART_CTRL0_CLKGATE, s, REG_CTRL0);
} else {
- mxs_auart_reset_assert(u);
+ mxs_auart_reset_assert(s);
}
clk_disable_unprepare(s->clk);
@@ -947,7 +1274,9 @@
static unsigned int mxs_auart_tx_empty(struct uart_port *u)
{
- if ((readl(u->membase + AUART_STAT) &
+ struct mxs_auart_port *s = to_auart_port(u);
+
+ if ((mxs_read(s, REG_STAT) &
(AUART_STAT_TXFE | AUART_STAT_BUSY)) == AUART_STAT_TXFE)
return TIOCSER_TEMT;
@@ -959,29 +1288,33 @@
struct mxs_auart_port *s = to_auart_port(u);
/* enable transmitter */
- writel(AUART_CTRL2_TXE, u->membase + AUART_CTRL2_SET);
+ mxs_set(AUART_CTRL2_TXE, s, REG_CTRL2);
mxs_auart_tx_chars(s);
}
static void mxs_auart_stop_tx(struct uart_port *u)
{
- writel(AUART_CTRL2_TXE, u->membase + AUART_CTRL2_CLR);
+ struct mxs_auart_port *s = to_auart_port(u);
+
+ mxs_clr(AUART_CTRL2_TXE, s, REG_CTRL2);
}
static void mxs_auart_stop_rx(struct uart_port *u)
{
- writel(AUART_CTRL2_RXE, u->membase + AUART_CTRL2_CLR);
+ struct mxs_auart_port *s = to_auart_port(u);
+
+ mxs_clr(AUART_CTRL2_RXE, s, REG_CTRL2);
}
static void mxs_auart_break_ctl(struct uart_port *u, int ctl)
{
+ struct mxs_auart_port *s = to_auart_port(u);
+
if (ctl)
- writel(AUART_LINECTRL_BRK,
- u->membase + AUART_LINECTRL_SET);
+ mxs_set(AUART_LINECTRL_BRK, s, REG_LINECTRL);
else
- writel(AUART_LINECTRL_BRK,
- u->membase + AUART_LINECTRL_CLR);
+ mxs_clr(AUART_LINECTRL_BRK, s, REG_LINECTRL);
}
static struct uart_ops mxs_auart_ops = {
@@ -1009,15 +1342,16 @@
#ifdef CONFIG_SERIAL_MXS_AUART_CONSOLE
static void mxs_auart_console_putchar(struct uart_port *port, int ch)
{
+ struct mxs_auart_port *s = to_auart_port(port);
unsigned int to = 1000;
- while (readl(port->membase + AUART_STAT) & AUART_STAT_TXFF) {
+ while (mxs_read(s, REG_STAT) & AUART_STAT_TXFF) {
if (!to--)
break;
udelay(1);
}
- writel(ch, port->membase + AUART_DATA);
+ mxs_write(ch, s, REG_DATA);
}
static void
@@ -1037,18 +1371,16 @@
clk_enable(s->clk);
/* First save the CR then disable the interrupts */
- old_ctrl2 = readl(port->membase + AUART_CTRL2);
- old_ctrl0 = readl(port->membase + AUART_CTRL0);
+ old_ctrl2 = mxs_read(s, REG_CTRL2);
+ old_ctrl0 = mxs_read(s, REG_CTRL0);
- writel(AUART_CTRL0_CLKGATE,
- port->membase + AUART_CTRL0_CLR);
- writel(AUART_CTRL2_UARTEN | AUART_CTRL2_TXE,
- port->membase + AUART_CTRL2_SET);
+ mxs_clr(AUART_CTRL0_CLKGATE, s, REG_CTRL0);
+ mxs_set(AUART_CTRL2_UARTEN | AUART_CTRL2_TXE, s, REG_CTRL2);
uart_console_write(port, str, count, mxs_auart_console_putchar);
/* Finally, wait for transmitter to become empty ... */
- while (readl(port->membase + AUART_STAT) & AUART_STAT_BUSY) {
+ while (mxs_read(s, REG_STAT) & AUART_STAT_BUSY) {
udelay(1);
if (!to--)
break;
@@ -1060,24 +1392,25 @@
* unused, but that is better than to disable it while it is still
* transmitting.
*/
- if (!(readl(port->membase + AUART_STAT) & AUART_STAT_BUSY)) {
- writel(old_ctrl0, port->membase + AUART_CTRL0);
- writel(old_ctrl2, port->membase + AUART_CTRL2);
+ if (!(mxs_read(s, REG_STAT) & AUART_STAT_BUSY)) {
+ mxs_write(old_ctrl0, s, REG_CTRL0);
+ mxs_write(old_ctrl2, s, REG_CTRL2);
}
clk_disable(s->clk);
}
static void __init
-auart_console_get_options(struct uart_port *port, int *baud,
+auart_console_get_options(struct mxs_auart_port *s, int *baud,
int *parity, int *bits)
{
+ struct uart_port *port = &s->port;
unsigned int lcr_h, quot;
- if (!(readl(port->membase + AUART_CTRL2) & AUART_CTRL2_UARTEN))
+ if (!(mxs_read(s, REG_CTRL2) & AUART_CTRL2_UARTEN))
return;
- lcr_h = readl(port->membase + AUART_LINECTRL);
+ lcr_h = mxs_read(s, REG_LINECTRL);
*parity = 'n';
if (lcr_h & AUART_LINECTRL_PEN) {
@@ -1092,12 +1425,10 @@
else
*bits = 8;
- quot = ((readl(port->membase + AUART_LINECTRL)
- & AUART_LINECTRL_BAUD_DIVINT_MASK))
- >> (AUART_LINECTRL_BAUD_DIVINT_SHIFT - 6);
- quot |= ((readl(port->membase + AUART_LINECTRL)
- & AUART_LINECTRL_BAUD_DIVFRAC_MASK))
- >> AUART_LINECTRL_BAUD_DIVFRAC_SHIFT;
+ quot = ((mxs_read(s, REG_LINECTRL) & AUART_LINECTRL_BAUD_DIVINT_MASK))
+ >> (AUART_LINECTRL_BAUD_DIVINT_SHIFT - 6);
+ quot |= ((mxs_read(s, REG_LINECTRL) & AUART_LINECTRL_BAUD_DIVFRAC_MASK))
+ >> AUART_LINECTRL_BAUD_DIVFRAC_SHIFT;
if (quot == 0)
quot = 1;
@@ -1132,7 +1463,7 @@
if (options)
uart_parse_options(options, &baud, &parity, &bits, &flow);
else
- auart_console_get_options(&s->port, &baud, &parity, &bits);
+ auart_console_get_options(s, &baud, &parity, &bits);
ret = uart_set_options(&s->port, co, baud, parity, bits, flow);
@@ -1164,6 +1495,60 @@
#endif
};
+static void mxs_init_regs(struct mxs_auart_port *s)
+{
+ if (is_asm9260_auart(s))
+ s->vendor = &vendor_alphascale_asm9260;
+ else
+ s->vendor = &vendor_freescale_stmp37xx;
+}
+
+static int mxs_get_clks(struct mxs_auart_port *s,
+ struct platform_device *pdev)
+{
+ int err;
+
+ if (!is_asm9260_auart(s)) {
+ s->clk = devm_clk_get(&pdev->dev, NULL);
+ if (IS_ERR(s->clk))
+ return PTR_ERR(s->clk);
+
+ return 0;
+ }
+
+ s->clk = devm_clk_get(s->dev, "mod");
+ if (IS_ERR(s->clk)) {
+ dev_err(s->dev, "Failed to get \"mod\" clk\n");
+ return PTR_ERR(s->clk);
+ }
+
+ s->clk_ahb = devm_clk_get(s->dev, "ahb");
+ if (IS_ERR(s->clk_ahb)) {
+ dev_err(s->dev, "Failed to get \"ahb\" clk\n");
+ return PTR_ERR(s->clk_ahb);
+ }
+
+ err = clk_prepare_enable(s->clk_ahb);
+ if (err) {
+ dev_err(s->dev, "Failed to enable ahb_clk!\n");
+ return err;
+ }
+
+ err = clk_set_rate(s->clk, clk_get_rate(s->clk_ahb));
+ if (err) {
+ dev_err(s->dev, "Failed to set rate!\n");
+ return err;
+ }
+
+ err = clk_prepare_enable(s->clk);
+ if (err) {
+ dev_err(s->dev, "Failed to enable clk!\n");
+ return err;
+ }
+
+ return 0;
+}
+
/*
* This function returns 1 if pdev isn't a device instatiated by dt, 0 if it
* could successfully get all information from dt or a negative errno.
@@ -1185,7 +1570,8 @@
}
s->port.line = ret;
- if (of_get_property(np, "fsl,uart-has-rtscts", NULL))
+ if (of_get_property(np, "uart-has-rtscts", NULL) ||
+ of_get_property(np, "fsl,uart-has-rtscts", NULL) /* deprecated */)
set_bit(MXS_AUART_RTSCTS, &s->flags);
return 0;
@@ -1269,6 +1655,9 @@
if (!s)
return -ENOMEM;
+ s->port.dev = &pdev->dev;
+ s->dev = &pdev->dev;
+
ret = serial_mxs_probe_dt(s, pdev);
if (ret > 0)
s->port.line = pdev->id < 0 ? 0 : pdev->id;
@@ -1280,15 +1669,14 @@
s->devtype = pdev->id_entry->driver_data;
}
- s->clk = devm_clk_get(&pdev->dev, NULL);
- if (IS_ERR(s->clk))
- return PTR_ERR(s->clk);
+ ret = mxs_get_clks(s, pdev);
+ if (ret)
+ return ret;
r = platform_get_resource(pdev, IORESOURCE_MEM, 0);
if (!r)
return -ENXIO;
-
s->port.mapbase = r->start;
s->port.membase = ioremap(r->start, resource_size(r));
s->port.ops = &mxs_auart_ops;
@@ -1296,7 +1684,8 @@
s->port.fifosize = MXS_AUART_FIFO_SIZE;
s->port.uartclk = clk_get_rate(s->clk);
s->port.type = PORT_IMX;
- s->port.dev = s->dev = &pdev->dev;
+
+ mxs_init_regs(s);
s->mctrl_prev = 0;
@@ -1327,16 +1716,21 @@
auart_port[s->port.line] = s;
- mxs_auart_reset_deassert(&s->port);
+ mxs_auart_reset_deassert(s);
ret = uart_add_one_port(&auart_driver, &s->port);
if (ret)
goto out_free_gpio_irq;
- version = readl(s->port.membase + AUART_VERSION);
- dev_info(&pdev->dev, "Found APPUART %d.%d.%d\n",
- (version >> 24) & 0xff,
- (version >> 16) & 0xff, version & 0xffff);
+ /* ASM9260 don't have version reg */
+ if (is_asm9260_auart(s)) {
+ dev_info(&pdev->dev, "Found APPUART ASM9260\n");
+ } else {
+ version = mxs_read(s, REG_VERSION);
+ dev_info(&pdev->dev, "Found APPUART %d.%d.%d\n",
+ (version >> 24) & 0xff,
+ (version >> 16) & 0xff, version & 0xffff);
+ }
return 0;
diff --git a/drivers/tty/serial/pic32_uart.c b/drivers/tty/serial/pic32_uart.c
new file mode 100644
index 0000000..62a43bf
--- /dev/null
+++ b/drivers/tty/serial/pic32_uart.c
@@ -0,0 +1,960 @@
+/*
+ * PIC32 Integrated Serial Driver.
+ *
+ * Copyright (C) 2015 Microchip Technology, Inc.
+ *
+ * Authors:
+ * Sorin-Andrei Pistirica <andrei.pistirica@microchip.com>
+ *
+ * Licensed under GPLv2 or later.
+ */
+
+#include <linux/kernel.h>
+#include <linux/platform_device.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/of_irq.h>
+#include <linux/of_gpio.h>
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/console.h>
+#include <linux/clk.h>
+#include <linux/tty.h>
+#include <linux/tty_flip.h>
+#include <linux/serial_core.h>
+#include <linux/delay.h>
+
+#include <asm/mach-pic32/pic32.h>
+#include "pic32_uart.h"
+
+/* UART name and device definitions */
+#define PIC32_DEV_NAME "pic32-uart"
+#define PIC32_MAX_UARTS 6
+#define PIC32_SDEV_NAME "ttyPIC"
+
+/* pic32_sport pointer for console use */
+static struct pic32_sport *pic32_sports[PIC32_MAX_UARTS];
+
+static inline void pic32_wait_deplete_txbuf(struct pic32_sport *sport)
+{
+ /* wait for tx empty, otherwise chars will be lost or corrupted */
+ while (!(pic32_uart_readl(sport, PIC32_UART_STA) & PIC32_UART_STA_TRMT))
+ udelay(1);
+}
+
+static inline int pic32_enable_clock(struct pic32_sport *sport)
+{
+ int ret = clk_prepare_enable(sport->clk);
+
+ if (ret)
+ return ret;
+
+ sport->ref_clk++;
+ return 0;
+}
+
+static inline void pic32_disable_clock(struct pic32_sport *sport)
+{
+ sport->ref_clk--;
+ clk_disable_unprepare(sport->clk);
+}
+
+/* serial core request to check if uart tx buffer is empty */
+static unsigned int pic32_uart_tx_empty(struct uart_port *port)
+{
+ struct pic32_sport *sport = to_pic32_sport(port);
+ u32 val = pic32_uart_readl(sport, PIC32_UART_STA);
+
+ return (val & PIC32_UART_STA_TRMT) ? 1 : 0;
+}
+
+/* serial core request to set UART outputs */
+static void pic32_uart_set_mctrl(struct uart_port *port, unsigned int mctrl)
+{
+ struct pic32_sport *sport = to_pic32_sport(port);
+
+ /* set loopback mode */
+ if (mctrl & TIOCM_LOOP)
+ pic32_uart_writel(sport, PIC32_SET(PIC32_UART_MODE),
+ PIC32_UART_MODE_LPBK);
+ else
+ pic32_uart_writel(sport, PIC32_CLR(PIC32_UART_MODE),
+ PIC32_UART_MODE_LPBK);
+}
+
+/* get the state of CTS input pin for this port */
+static unsigned int get_cts_state(struct pic32_sport *sport)
+{
+ /* read and invert UxCTS */
+ if (gpio_is_valid(sport->cts_gpio))
+ return !gpio_get_value(sport->cts_gpio);
+
+ return 1;
+}
+
+/* serial core request to return the state of misc UART input pins */
+static unsigned int pic32_uart_get_mctrl(struct uart_port *port)
+{
+ struct pic32_sport *sport = to_pic32_sport(port);
+ unsigned int mctrl = 0;
+
+ if (!sport->hw_flow_ctrl)
+ mctrl |= TIOCM_CTS;
+ else if (get_cts_state(sport))
+ mctrl |= TIOCM_CTS;
+
+ /* DSR and CD are not supported in PIC32, so return 1
+ * RI is not supported in PIC32, so return 0
+ */
+ mctrl |= TIOCM_CD;
+ mctrl |= TIOCM_DSR;
+
+ return mctrl;
+}
+
+/* stop tx and start tx are not called in pairs, therefore a flag indicates
+ * the status of irq to control the irq-depth.
+ */
+static inline void pic32_uart_irqtxen(struct pic32_sport *sport, u8 en)
+{
+ if (en && !tx_irq_enabled(sport)) {
+ enable_irq(sport->irq_tx);
+ tx_irq_enabled(sport) = 1;
+ } else if (!en && tx_irq_enabled(sport)) {
+ /* use disable_irq_nosync() and not disable_irq() to avoid self
+ * imposed deadlock by not waiting for irq handler to end,
+ * since this callback is called from interrupt context.
+ */
+ disable_irq_nosync(sport->irq_tx);
+ tx_irq_enabled(sport) = 0;
+ }
+}
+
+/* serial core request to disable tx ASAP (used for flow control) */
+static void pic32_uart_stop_tx(struct uart_port *port)
+{
+ struct pic32_sport *sport = to_pic32_sport(port);
+
+ if (!(pic32_uart_readl(sport, PIC32_UART_MODE) & PIC32_UART_MODE_ON))
+ return;
+
+ if (!(pic32_uart_readl(sport, PIC32_UART_STA) & PIC32_UART_STA_UTXEN))
+ return;
+
+ /* wait for tx empty */
+ pic32_wait_deplete_txbuf(sport);
+
+ pic32_uart_writel(sport, PIC32_CLR(PIC32_UART_STA),
+ PIC32_UART_STA_UTXEN);
+ pic32_uart_irqtxen(sport, 0);
+}
+
+/* serial core request to (re)enable tx */
+static void pic32_uart_start_tx(struct uart_port *port)
+{
+ struct pic32_sport *sport = to_pic32_sport(port);
+
+ pic32_uart_irqtxen(sport, 1);
+ pic32_uart_writel(sport, PIC32_SET(PIC32_UART_STA),
+ PIC32_UART_STA_UTXEN);
+}
+
+/* serial core request to stop rx, called before port shutdown */
+static void pic32_uart_stop_rx(struct uart_port *port)
+{
+ struct pic32_sport *sport = to_pic32_sport(port);
+
+ /* disable rx interrupts */
+ disable_irq(sport->irq_rx);
+
+ /* receiver Enable bit OFF */
+ pic32_uart_writel(sport, PIC32_CLR(PIC32_UART_STA),
+ PIC32_UART_STA_URXEN);
+}
+
+/* serial core request to start/stop emitting break char */
+static void pic32_uart_break_ctl(struct uart_port *port, int ctl)
+{
+ struct pic32_sport *sport = to_pic32_sport(port);
+ unsigned long flags;
+
+ spin_lock_irqsave(&port->lock, flags);
+
+ if (ctl)
+ pic32_uart_writel(sport, PIC32_SET(PIC32_UART_STA),
+ PIC32_UART_STA_UTXBRK);
+ else
+ pic32_uart_writel(sport, PIC32_CLR(PIC32_UART_STA),
+ PIC32_UART_STA_UTXBRK);
+
+ spin_unlock_irqrestore(&port->lock, flags);
+}
+
+/* get port type in string format */
+static const char *pic32_uart_type(struct uart_port *port)
+{
+ return (port->type == PORT_PIC32) ? PIC32_DEV_NAME : NULL;
+}
+
+/* read all chars in rx fifo and send them to core */
+static void pic32_uart_do_rx(struct uart_port *port)
+{
+ struct pic32_sport *sport = to_pic32_sport(port);
+ struct tty_port *tty;
+ unsigned int max_count;
+
+ /* limit number of char read in interrupt, should not be
+ * higher than fifo size anyway since we're much faster than
+ * serial port
+ */
+ max_count = PIC32_UART_RX_FIFO_DEPTH;
+
+ spin_lock(&port->lock);
+
+ tty = &port->state->port;
+
+ do {
+ u32 sta_reg, c;
+ char flag;
+
+ /* get overrun/fifo empty information from status register */
+ sta_reg = pic32_uart_readl(sport, PIC32_UART_STA);
+ if (unlikely(sta_reg & PIC32_UART_STA_OERR)) {
+
+ /* fifo reset is required to clear interrupt */
+ pic32_uart_writel(sport, PIC32_CLR(PIC32_UART_STA),
+ PIC32_UART_STA_OERR);
+
+ port->icount.overrun++;
+ tty_insert_flip_char(tty, 0, TTY_OVERRUN);
+ }
+
+ /* Can at least one more character can be read? */
+ if (!(sta_reg & PIC32_UART_STA_URXDA))
+ break;
+
+ /* read the character and increment the rx counter */
+ c = pic32_uart_readl(sport, PIC32_UART_RX);
+
+ port->icount.rx++;
+ flag = TTY_NORMAL;
+ c &= 0xff;
+
+ if (unlikely((sta_reg & PIC32_UART_STA_PERR) ||
+ (sta_reg & PIC32_UART_STA_FERR))) {
+
+ /* do stats first */
+ if (sta_reg & PIC32_UART_STA_PERR)
+ port->icount.parity++;
+ if (sta_reg & PIC32_UART_STA_FERR)
+ port->icount.frame++;
+
+ /* update flag wrt read_status_mask */
+ sta_reg &= port->read_status_mask;
+
+ if (sta_reg & PIC32_UART_STA_FERR)
+ flag = TTY_FRAME;
+ if (sta_reg & PIC32_UART_STA_PERR)
+ flag = TTY_PARITY;
+ }
+
+ if (uart_handle_sysrq_char(port, c))
+ continue;
+
+ if ((sta_reg & port->ignore_status_mask) == 0)
+ tty_insert_flip_char(tty, c, flag);
+
+ } while (--max_count);
+
+ spin_unlock(&port->lock);
+
+ tty_flip_buffer_push(tty);
+}
+
+/* fill tx fifo with chars to send, stop when fifo is about to be full
+ * or when all chars have been sent.
+ */
+static void pic32_uart_do_tx(struct uart_port *port)
+{
+ struct pic32_sport *sport = to_pic32_sport(port);
+ struct circ_buf *xmit = &port->state->xmit;
+ unsigned int max_count = PIC32_UART_TX_FIFO_DEPTH;
+
+ if (port->x_char) {
+ pic32_uart_writel(sport, PIC32_UART_TX, port->x_char);
+ port->icount.tx++;
+ port->x_char = 0;
+ return;
+ }
+
+ if (uart_tx_stopped(port)) {
+ pic32_uart_stop_tx(port);
+ return;
+ }
+
+ if (uart_circ_empty(xmit))
+ goto txq_empty;
+
+ /* keep stuffing chars into uart tx buffer
+ * 1) until uart fifo is full
+ * or
+ * 2) until the circ buffer is empty
+ * (all chars have been sent)
+ * or
+ * 3) until the max count is reached
+ * (prevents lingering here for too long in certain cases)
+ */
+ while (!(PIC32_UART_STA_UTXBF &
+ pic32_uart_readl(sport, PIC32_UART_STA))) {
+ unsigned int c = xmit->buf[xmit->tail];
+
+ pic32_uart_writel(sport, PIC32_UART_TX, c);
+
+ xmit->tail = (xmit->tail + 1) & (UART_XMIT_SIZE - 1);
+ port->icount.tx++;
+ if (uart_circ_empty(xmit))
+ break;
+ if (--max_count == 0)
+ break;
+ }
+
+ if (uart_circ_chars_pending(xmit) < WAKEUP_CHARS)
+ uart_write_wakeup(port);
+
+ if (uart_circ_empty(xmit))
+ goto txq_empty;
+
+ return;
+
+txq_empty:
+ pic32_uart_irqtxen(sport, 0);
+}
+
+/* RX interrupt handler */
+static irqreturn_t pic32_uart_rx_interrupt(int irq, void *dev_id)
+{
+ struct uart_port *port = dev_id;
+
+ pic32_uart_do_rx(port);
+
+ return IRQ_HANDLED;
+}
+
+/* TX interrupt handler */
+static irqreturn_t pic32_uart_tx_interrupt(int irq, void *dev_id)
+{
+ struct uart_port *port = dev_id;
+ unsigned long flags;
+
+ spin_lock_irqsave(&port->lock, flags);
+ pic32_uart_do_tx(port);
+ spin_unlock_irqrestore(&port->lock, flags);
+
+ return IRQ_HANDLED;
+}
+
+/* FAULT interrupt handler */
+static irqreturn_t pic32_uart_fault_interrupt(int irq, void *dev_id)
+{
+ /* do nothing: pic32_uart_do_rx() handles faults. */
+ return IRQ_HANDLED;
+}
+
+/* enable rx & tx operation on uart */
+static void pic32_uart_en_and_unmask(struct uart_port *port)
+{
+ struct pic32_sport *sport = to_pic32_sport(port);
+
+ pic32_uart_writel(sport, PIC32_SET(PIC32_UART_STA),
+ PIC32_UART_STA_UTXEN | PIC32_UART_STA_URXEN);
+ pic32_uart_writel(sport, PIC32_SET(PIC32_UART_MODE),
+ PIC32_UART_MODE_ON);
+}
+
+/* disable rx & tx operation on uart */
+static void pic32_uart_dsbl_and_mask(struct uart_port *port)
+{
+ struct pic32_sport *sport = to_pic32_sport(port);
+
+ /* wait for tx empty, otherwise chars will be lost or corrupted */
+ pic32_wait_deplete_txbuf(sport);
+
+ pic32_uart_writel(sport, PIC32_CLR(PIC32_UART_STA),
+ PIC32_UART_STA_UTXEN | PIC32_UART_STA_URXEN);
+ pic32_uart_writel(sport, PIC32_CLR(PIC32_UART_MODE),
+ PIC32_UART_MODE_ON);
+}
+
+/* serial core request to initialize uart and start rx operation */
+static int pic32_uart_startup(struct uart_port *port)
+{
+ struct pic32_sport *sport = to_pic32_sport(port);
+ u32 dflt_baud = (port->uartclk / PIC32_UART_DFLT_BRATE / 16) - 1;
+ unsigned long flags;
+ int ret;
+
+ local_irq_save(flags);
+
+ ret = pic32_enable_clock(sport);
+ if (ret) {
+ local_irq_restore(flags);
+ goto out_done;
+ }
+
+ /* clear status and mode registers */
+ pic32_uart_writel(sport, PIC32_UART_MODE, 0);
+ pic32_uart_writel(sport, PIC32_UART_STA, 0);
+
+ /* disable uart and mask all interrupts */
+ pic32_uart_dsbl_and_mask(port);
+
+ /* set default baud */
+ pic32_uart_writel(sport, PIC32_UART_BRG, dflt_baud);
+
+ local_irq_restore(flags);
+
+ /* Each UART of a PIC32 has three interrupts therefore,
+ * we setup driver to register the 3 irqs for the device.
+ *
+ * For each irq request_irq() is called with interrupt disabled.
+ * And the irq is enabled as soon as we are ready to handle them.
+ */
+ tx_irq_enabled(sport) = 0;
+
+ sport->irq_fault_name = kasprintf(GFP_KERNEL, "%s%d-fault",
+ pic32_uart_type(port),
+ sport->idx);
+ if (!sport->irq_fault_name) {
+ dev_err(port->dev, "%s: kasprintf err!", __func__);
+ ret = -ENOMEM;
+ goto out_done;
+ }
+ irq_set_status_flags(sport->irq_fault, IRQ_NOAUTOEN);
+ ret = request_irq(sport->irq_fault, pic32_uart_fault_interrupt,
+ sport->irqflags_fault, sport->irq_fault_name, port);
+ if (ret) {
+ dev_err(port->dev, "%s: request irq(%d) err! ret:%d name:%s\n",
+ __func__, sport->irq_fault, ret,
+ pic32_uart_type(port));
+ goto out_f;
+ }
+
+ sport->irq_rx_name = kasprintf(GFP_KERNEL, "%s%d-rx",
+ pic32_uart_type(port),
+ sport->idx);
+ if (!sport->irq_rx_name) {
+ dev_err(port->dev, "%s: kasprintf err!", __func__);
+ kfree(sport->irq_fault_name);
+ ret = -ENOMEM;
+ goto out_f;
+ }
+ irq_set_status_flags(sport->irq_rx, IRQ_NOAUTOEN);
+ ret = request_irq(sport->irq_rx, pic32_uart_rx_interrupt,
+ sport->irqflags_rx, sport->irq_rx_name, port);
+ if (ret) {
+ dev_err(port->dev, "%s: request irq(%d) err! ret:%d name:%s\n",
+ __func__, sport->irq_rx, ret,
+ pic32_uart_type(port));
+ goto out_r;
+ }
+
+ sport->irq_tx_name = kasprintf(GFP_KERNEL, "%s%d-tx",
+ pic32_uart_type(port),
+ sport->idx);
+ if (!sport->irq_tx_name) {
+ dev_err(port->dev, "%s: kasprintf err!", __func__);
+ ret = -ENOMEM;
+ goto out_r;
+ }
+ irq_set_status_flags(sport->irq_tx, IRQ_NOAUTOEN);
+ ret = request_irq(sport->irq_tx, pic32_uart_tx_interrupt,
+ sport->irqflags_tx, sport->irq_tx_name, port);
+ if (ret) {
+ dev_err(port->dev, "%s: request irq(%d) err! ret:%d name:%s\n",
+ __func__, sport->irq_tx, ret,
+ pic32_uart_type(port));
+ goto out_t;
+ }
+
+ local_irq_save(flags);
+
+ /* set rx interrupt on first receive */
+ pic32_uart_writel(sport, PIC32_CLR(PIC32_UART_STA),
+ PIC32_UART_STA_URXISEL1 | PIC32_UART_STA_URXISEL0);
+
+ /* set interrupt on empty */
+ pic32_uart_writel(sport, PIC32_CLR(PIC32_UART_STA),
+ PIC32_UART_STA_UTXISEL1);
+
+ /* enable all interrupts and eanable uart */
+ pic32_uart_en_and_unmask(port);
+
+ enable_irq(sport->irq_rx);
+
+ return 0;
+
+out_t:
+ kfree(sport->irq_tx_name);
+ free_irq(sport->irq_tx, sport);
+out_r:
+ kfree(sport->irq_rx_name);
+ free_irq(sport->irq_rx, sport);
+out_f:
+ kfree(sport->irq_fault_name);
+ free_irq(sport->irq_fault, sport);
+out_done:
+ return ret;
+}
+
+/* serial core request to flush & disable uart */
+static void pic32_uart_shutdown(struct uart_port *port)
+{
+ struct pic32_sport *sport = to_pic32_sport(port);
+ unsigned long flags;
+
+ /* disable uart */
+ spin_lock_irqsave(&port->lock, flags);
+ pic32_uart_dsbl_and_mask(port);
+ spin_unlock_irqrestore(&port->lock, flags);
+ pic32_disable_clock(sport);
+
+ /* free all 3 interrupts for this UART */
+ free_irq(sport->irq_fault, port);
+ free_irq(sport->irq_tx, port);
+ free_irq(sport->irq_rx, port);
+}
+
+/* serial core request to change current uart setting */
+static void pic32_uart_set_termios(struct uart_port *port,
+ struct ktermios *new,
+ struct ktermios *old)
+{
+ struct pic32_sport *sport = to_pic32_sport(port);
+ unsigned int baud;
+ unsigned int quot;
+ unsigned long flags;
+
+ spin_lock_irqsave(&port->lock, flags);
+
+ /* disable uart and mask all interrupts while changing speed */
+ pic32_uart_dsbl_and_mask(port);
+
+ /* stop bit options */
+ if (new->c_cflag & CSTOPB)
+ pic32_uart_writel(sport, PIC32_SET(PIC32_UART_MODE),
+ PIC32_UART_MODE_STSEL);
+ else
+ pic32_uart_writel(sport, PIC32_CLR(PIC32_UART_MODE),
+ PIC32_UART_MODE_STSEL);
+
+ /* parity options */
+ if (new->c_cflag & PARENB) {
+ if (new->c_cflag & PARODD) {
+ pic32_uart_writel(sport, PIC32_SET(PIC32_UART_MODE),
+ PIC32_UART_MODE_PDSEL1);
+ pic32_uart_writel(sport, PIC32_CLR(PIC32_UART_MODE),
+ PIC32_UART_MODE_PDSEL0);
+ } else {
+ pic32_uart_writel(sport, PIC32_SET(PIC32_UART_MODE),
+ PIC32_UART_MODE_PDSEL0);
+ pic32_uart_writel(sport, PIC32_CLR(PIC32_UART_MODE),
+ PIC32_UART_MODE_PDSEL1);
+ }
+ } else {
+ pic32_uart_writel(sport, PIC32_CLR(PIC32_UART_MODE),
+ PIC32_UART_MODE_PDSEL1 |
+ PIC32_UART_MODE_PDSEL0);
+ }
+ /* if hw flow ctrl, then the pins must be specified in device tree */
+ if ((new->c_cflag & CRTSCTS) && sport->hw_flow_ctrl) {
+ /* enable hardware flow control */
+ pic32_uart_writel(sport, PIC32_SET(PIC32_UART_MODE),
+ PIC32_UART_MODE_UEN1);
+ pic32_uart_writel(sport, PIC32_CLR(PIC32_UART_MODE),
+ PIC32_UART_MODE_UEN0);
+ pic32_uart_writel(sport, PIC32_CLR(PIC32_UART_MODE),
+ PIC32_UART_MODE_RTSMD);
+ } else {
+ /* disable hardware flow control */
+ pic32_uart_writel(sport, PIC32_CLR(PIC32_UART_MODE),
+ PIC32_UART_MODE_UEN1);
+ pic32_uart_writel(sport, PIC32_CLR(PIC32_UART_MODE),
+ PIC32_UART_MODE_UEN0);
+ pic32_uart_writel(sport, PIC32_CLR(PIC32_UART_MODE),
+ PIC32_UART_MODE_RTSMD);
+ }
+
+ /* Always 8-bit */
+ new->c_cflag |= CS8;
+
+ /* Mark/Space parity is not supported */
+ new->c_cflag &= ~CMSPAR;
+
+ /* update baud */
+ baud = uart_get_baud_rate(port, new, old, 0, port->uartclk / 16);
+ quot = uart_get_divisor(port, baud) - 1;
+ pic32_uart_writel(sport, PIC32_UART_BRG, quot);
+ uart_update_timeout(port, new->c_cflag, baud);
+
+ if (tty_termios_baud_rate(new))
+ tty_termios_encode_baud_rate(new, baud, baud);
+
+ /* enable uart */
+ pic32_uart_en_and_unmask(port);
+
+ spin_unlock_irqrestore(&port->lock, flags);
+}
+
+/* serial core request to claim uart iomem */
+static int pic32_uart_request_port(struct uart_port *port)
+{
+ struct platform_device *pdev = to_platform_device(port->dev);
+ struct resource *res_mem;
+
+ res_mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (unlikely(!res_mem))
+ return -EINVAL;
+
+ if (!request_mem_region(port->mapbase, resource_size(res_mem),
+ "pic32_uart_mem"))
+ return -EBUSY;
+
+ port->membase = devm_ioremap_nocache(port->dev, port->mapbase,
+ resource_size(res_mem));
+ if (!port->membase) {
+ dev_err(port->dev, "Unable to map registers\n");
+ release_mem_region(port->mapbase, resource_size(res_mem));
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+/* serial core request to release uart iomem */
+static void pic32_uart_release_port(struct uart_port *port)
+{
+ struct platform_device *pdev = to_platform_device(port->dev);
+ struct resource *res_mem;
+ unsigned int res_size;
+
+ res_mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (unlikely(!res_mem))
+ return;
+ res_size = resource_size(res_mem);
+
+ release_mem_region(port->mapbase, res_size);
+}
+
+/* serial core request to do any port required auto-configuration */
+static void pic32_uart_config_port(struct uart_port *port, int flags)
+{
+ if (flags & UART_CONFIG_TYPE) {
+ if (pic32_uart_request_port(port))
+ return;
+ port->type = PORT_PIC32;
+ }
+}
+
+/* serial core request to check that port information in serinfo are suitable */
+static int pic32_uart_verify_port(struct uart_port *port,
+ struct serial_struct *serinfo)
+{
+ if (port->type != PORT_PIC32)
+ return -EINVAL;
+ if (port->irq != serinfo->irq)
+ return -EINVAL;
+ if (port->iotype != serinfo->io_type)
+ return -EINVAL;
+ if (port->mapbase != (unsigned long)serinfo->iomem_base)
+ return -EINVAL;
+
+ return 0;
+}
+
+/* serial core callbacks */
+static const struct uart_ops pic32_uart_ops = {
+ .tx_empty = pic32_uart_tx_empty,
+ .get_mctrl = pic32_uart_get_mctrl,
+ .set_mctrl = pic32_uart_set_mctrl,
+ .start_tx = pic32_uart_start_tx,
+ .stop_tx = pic32_uart_stop_tx,
+ .stop_rx = pic32_uart_stop_rx,
+ .break_ctl = pic32_uart_break_ctl,
+ .startup = pic32_uart_startup,
+ .shutdown = pic32_uart_shutdown,
+ .set_termios = pic32_uart_set_termios,
+ .type = pic32_uart_type,
+ .release_port = pic32_uart_release_port,
+ .request_port = pic32_uart_request_port,
+ .config_port = pic32_uart_config_port,
+ .verify_port = pic32_uart_verify_port,
+};
+
+#ifdef CONFIG_SERIAL_PIC32_CONSOLE
+/* output given char */
+static void pic32_console_putchar(struct uart_port *port, int ch)
+{
+ struct pic32_sport *sport = to_pic32_sport(port);
+
+ if (!(pic32_uart_readl(sport, PIC32_UART_MODE) & PIC32_UART_MODE_ON))
+ return;
+
+ if (!(pic32_uart_readl(sport, PIC32_UART_STA) & PIC32_UART_STA_UTXEN))
+ return;
+
+ /* wait for tx empty */
+ pic32_wait_deplete_txbuf(sport);
+
+ pic32_uart_writel(sport, PIC32_UART_TX, ch & 0xff);
+}
+
+/* console core request to output given string */
+static void pic32_console_write(struct console *co, const char *s,
+ unsigned int count)
+{
+ struct pic32_sport *sport = pic32_sports[co->index];
+ struct uart_port *port = pic32_get_port(sport);
+
+ /* call uart helper to deal with \r\n */
+ uart_console_write(port, s, count, pic32_console_putchar);
+}
+
+/* console core request to setup given console, find matching uart
+ * port and setup it.
+ */
+static int pic32_console_setup(struct console *co, char *options)
+{
+ struct pic32_sport *sport;
+ struct uart_port *port = NULL;
+ int baud = 115200;
+ int bits = 8;
+ int parity = 'n';
+ int flow = 'n';
+ int ret = 0;
+
+ if (unlikely(co->index < 0 || co->index >= PIC32_MAX_UARTS))
+ return -ENODEV;
+
+ sport = pic32_sports[co->index];
+ if (!sport)
+ return -ENODEV;
+ port = pic32_get_port(sport);
+
+ ret = pic32_enable_clock(sport);
+ if (ret)
+ return ret;
+
+ if (options)
+ uart_parse_options(options, &baud, &parity, &bits, &flow);
+
+ return uart_set_options(port, co, baud, parity, bits, flow);
+}
+
+static struct uart_driver pic32_uart_driver;
+static struct console pic32_console = {
+ .name = PIC32_SDEV_NAME,
+ .write = pic32_console_write,
+ .device = uart_console_device,
+ .setup = pic32_console_setup,
+ .flags = CON_PRINTBUFFER,
+ .index = -1,
+ .data = &pic32_uart_driver,
+};
+#define PIC32_SCONSOLE (&pic32_console)
+
+static int __init pic32_console_init(void)
+{
+ register_console(&pic32_console);
+ return 0;
+}
+console_initcall(pic32_console_init);
+
+static inline bool is_pic32_console_port(struct uart_port *port)
+{
+ return port->cons && port->cons->index == port->line;
+}
+
+/*
+ * Late console initialization.
+ */
+static int __init pic32_late_console_init(void)
+{
+ if (!(pic32_console.flags & CON_ENABLED))
+ register_console(&pic32_console);
+
+ return 0;
+}
+
+core_initcall(pic32_late_console_init);
+
+#else
+#define PIC32_SCONSOLE NULL
+#endif
+
+static struct uart_driver pic32_uart_driver = {
+ .owner = THIS_MODULE,
+ .driver_name = PIC32_DEV_NAME,
+ .dev_name = PIC32_SDEV_NAME,
+ .nr = PIC32_MAX_UARTS,
+ .cons = PIC32_SCONSOLE,
+};
+
+static int pic32_uart_probe(struct platform_device *pdev)
+{
+ struct device_node *np = pdev->dev.of_node;
+ struct pic32_sport *sport;
+ int uart_idx = 0;
+ struct resource *res_mem;
+ struct uart_port *port;
+ int ret;
+
+ uart_idx = of_alias_get_id(np, "serial");
+ if (uart_idx < 0 || uart_idx >= PIC32_MAX_UARTS)
+ return -EINVAL;
+
+ res_mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (!res_mem)
+ return -EINVAL;
+
+ sport = devm_kzalloc(&pdev->dev, sizeof(*sport), GFP_KERNEL);
+ if (!sport)
+ return -ENOMEM;
+
+ sport->idx = uart_idx;
+ sport->irq_fault = irq_of_parse_and_map(np, 0);
+ sport->irqflags_fault = IRQF_NO_THREAD;
+ sport->irq_rx = irq_of_parse_and_map(np, 1);
+ sport->irqflags_rx = IRQF_NO_THREAD;
+ sport->irq_tx = irq_of_parse_and_map(np, 2);
+ sport->irqflags_tx = IRQF_NO_THREAD;
+ sport->clk = devm_clk_get(&pdev->dev, NULL);
+ sport->cts_gpio = -EINVAL;
+ sport->dev = &pdev->dev;
+
+ /* Hardware flow control: gpios
+ * !Note: Basically, CTS is needed for reading the status.
+ */
+ sport->hw_flow_ctrl = false;
+ sport->cts_gpio = of_get_named_gpio(np, "cts-gpios", 0);
+ if (gpio_is_valid(sport->cts_gpio)) {
+ sport->hw_flow_ctrl = true;
+
+ ret = devm_gpio_request(sport->dev,
+ sport->cts_gpio, "CTS");
+ if (ret) {
+ dev_err(&pdev->dev,
+ "error requesting CTS GPIO\n");
+ goto err;
+ }
+
+ ret = gpio_direction_input(sport->cts_gpio);
+ if (ret) {
+ dev_err(&pdev->dev, "error setting CTS GPIO\n");
+ goto err;
+ }
+ }
+
+ pic32_sports[uart_idx] = sport;
+ port = &sport->port;
+ memset(port, 0, sizeof(*port));
+ port->iotype = UPIO_MEM;
+ port->mapbase = res_mem->start;
+ port->ops = &pic32_uart_ops;
+ port->flags = UPF_BOOT_AUTOCONF;
+ port->dev = &pdev->dev;
+ port->fifosize = PIC32_UART_TX_FIFO_DEPTH;
+ port->uartclk = clk_get_rate(sport->clk);
+ port->line = uart_idx;
+
+ ret = uart_add_one_port(&pic32_uart_driver, port);
+ if (ret) {
+ port->membase = NULL;
+ dev_err(port->dev, "%s: uart add port error!\n", __func__);
+ goto err;
+ }
+
+#ifdef CONFIG_SERIAL_PIC32_CONSOLE
+ if (is_pic32_console_port(port) &&
+ (pic32_console.flags & CON_ENABLED)) {
+ /* The peripheral clock has been enabled by console_setup,
+ * so disable it till the port is used.
+ */
+ pic32_disable_clock(sport);
+ }
+#endif
+
+ platform_set_drvdata(pdev, port);
+
+ dev_info(&pdev->dev, "%s: uart(%d) driver initialized.\n",
+ __func__, uart_idx);
+
+ return 0;
+err:
+ /* automatic unroll of sport and gpios */
+ return ret;
+}
+
+static int pic32_uart_remove(struct platform_device *pdev)
+{
+ struct uart_port *port = platform_get_drvdata(pdev);
+ struct pic32_sport *sport = to_pic32_sport(port);
+
+ uart_remove_one_port(&pic32_uart_driver, port);
+ pic32_disable_clock(sport);
+ platform_set_drvdata(pdev, NULL);
+ pic32_sports[sport->idx] = NULL;
+
+ /* automatic unroll of sport and gpios */
+ return 0;
+}
+
+static const struct of_device_id pic32_serial_dt_ids[] = {
+ { .compatible = "microchip,pic32mzda-uart" },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, pic32_serial_dt_ids);
+
+static struct platform_driver pic32_uart_platform_driver = {
+ .probe = pic32_uart_probe,
+ .remove = pic32_uart_remove,
+ .driver = {
+ .name = PIC32_DEV_NAME,
+ .of_match_table = of_match_ptr(pic32_serial_dt_ids),
+ },
+};
+
+static int __init pic32_uart_init(void)
+{
+ int ret;
+
+ ret = uart_register_driver(&pic32_uart_driver);
+ if (ret) {
+ pr_err("failed to register %s:%d\n",
+ pic32_uart_driver.driver_name, ret);
+ return ret;
+ }
+
+ ret = platform_driver_register(&pic32_uart_platform_driver);
+ if (ret) {
+ pr_err("fail to register pic32 uart\n");
+ uart_unregister_driver(&pic32_uart_driver);
+ }
+
+ return ret;
+}
+arch_initcall(pic32_uart_init);
+
+static void __exit pic32_uart_exit(void)
+{
+#ifdef CONFIG_SERIAL_PIC32_CONSOLE
+ unregister_console(&pic32_console);
+#endif
+ platform_driver_unregister(&pic32_uart_platform_driver);
+ uart_unregister_driver(&pic32_uart_driver);
+}
+module_exit(pic32_uart_exit);
+
+MODULE_AUTHOR("Sorin-Andrei Pistirica <andrei.pistirica@microchip.com>");
+MODULE_DESCRIPTION("Microchip PIC32 integrated serial port driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/tty/serial/pic32_uart.h b/drivers/tty/serial/pic32_uart.h
new file mode 100644
index 0000000..ec379da
--- /dev/null
+++ b/drivers/tty/serial/pic32_uart.h
@@ -0,0 +1,126 @@
+/*
+ * PIC32 Integrated Serial Driver.
+ *
+ * Copyright (C) 2015 Microchip Technology, Inc.
+ *
+ * Authors:
+ * Sorin-Andrei Pistirica <andrei.pistirica@microchip.com>
+ *
+ * Licensed under GPLv2 or later.
+ */
+#ifndef __DT_PIC32_UART_H__
+#define __DT_PIC32_UART_H__
+
+#define PIC32_UART_DFLT_BRATE (9600)
+#define PIC32_UART_TX_FIFO_DEPTH (8)
+#define PIC32_UART_RX_FIFO_DEPTH (8)
+
+#define PIC32_UART_MODE 0x00
+#define PIC32_UART_STA 0x10
+#define PIC32_UART_TX 0x20
+#define PIC32_UART_RX 0x30
+#define PIC32_UART_BRG 0x40
+
+struct pic32_console_opt {
+ int baud;
+ int parity;
+ int bits;
+ int flow;
+};
+
+/* struct pic32_sport - pic32 serial port descriptor
+ * @port: uart port descriptor
+ * @idx: port index
+ * @irq_fault: virtual fault interrupt number
+ * @irqflags_fault: flags related to fault irq
+ * @irq_fault_name: irq fault name
+ * @irq_rx: virtual rx interrupt number
+ * @irqflags_rx: flags related to rx irq
+ * @irq_rx_name: irq rx name
+ * @irq_tx: virtual tx interrupt number
+ * @irqflags_tx: : flags related to tx irq
+ * @irq_tx_name: irq tx name
+ * @cts_gpio: clear to send gpio
+ * @dev: device descriptor
+ **/
+struct pic32_sport {
+ struct uart_port port;
+ struct pic32_console_opt opt;
+ int idx;
+
+ int irq_fault;
+ int irqflags_fault;
+ const char *irq_fault_name;
+ int irq_rx;
+ int irqflags_rx;
+ const char *irq_rx_name;
+ int irq_tx;
+ int irqflags_tx;
+ const char *irq_tx_name;
+ u8 enable_tx_irq;
+
+ bool hw_flow_ctrl;
+ int cts_gpio;
+
+ int ref_clk;
+ struct clk *clk;
+
+ struct device *dev;
+};
+#define to_pic32_sport(c) container_of(c, struct pic32_sport, port)
+#define pic32_get_port(sport) (&sport->port)
+#define pic32_get_opt(sport) (&sport->opt)
+#define tx_irq_enabled(sport) (sport->enable_tx_irq)
+
+static inline void pic32_uart_writel(struct pic32_sport *sport,
+ u32 reg, u32 val)
+{
+ struct uart_port *port = pic32_get_port(sport);
+
+ __raw_writel(val, port->membase + reg);
+}
+
+static inline u32 pic32_uart_readl(struct pic32_sport *sport, u32 reg)
+{
+ struct uart_port *port = pic32_get_port(sport);
+
+ return __raw_readl(port->membase + reg);
+}
+
+/* pic32 uart mode register bits */
+#define PIC32_UART_MODE_ON BIT(15)
+#define PIC32_UART_MODE_FRZ BIT(14)
+#define PIC32_UART_MODE_SIDL BIT(13)
+#define PIC32_UART_MODE_IREN BIT(12)
+#define PIC32_UART_MODE_RTSMD BIT(11)
+#define PIC32_UART_MODE_RESV1 BIT(10)
+#define PIC32_UART_MODE_UEN1 BIT(9)
+#define PIC32_UART_MODE_UEN0 BIT(8)
+#define PIC32_UART_MODE_WAKE BIT(7)
+#define PIC32_UART_MODE_LPBK BIT(6)
+#define PIC32_UART_MODE_ABAUD BIT(5)
+#define PIC32_UART_MODE_RXINV BIT(4)
+#define PIC32_UART_MODE_BRGH BIT(3)
+#define PIC32_UART_MODE_PDSEL1 BIT(2)
+#define PIC32_UART_MODE_PDSEL0 BIT(1)
+#define PIC32_UART_MODE_STSEL BIT(0)
+
+/* pic32 uart status register bits */
+#define PIC32_UART_STA_UTXISEL1 BIT(15)
+#define PIC32_UART_STA_UTXISEL0 BIT(14)
+#define PIC32_UART_STA_UTXINV BIT(13)
+#define PIC32_UART_STA_URXEN BIT(12)
+#define PIC32_UART_STA_UTXBRK BIT(11)
+#define PIC32_UART_STA_UTXEN BIT(10)
+#define PIC32_UART_STA_UTXBF BIT(9)
+#define PIC32_UART_STA_TRMT BIT(8)
+#define PIC32_UART_STA_URXISEL1 BIT(7)
+#define PIC32_UART_STA_URXISEL0 BIT(6)
+#define PIC32_UART_STA_ADDEN BIT(5)
+#define PIC32_UART_STA_RIDLE BIT(4)
+#define PIC32_UART_STA_PERR BIT(3)
+#define PIC32_UART_STA_FERR BIT(2)
+#define PIC32_UART_STA_OERR BIT(1)
+#define PIC32_UART_STA_URXDA BIT(0)
+
+#endif /* __DT_PIC32_UART_H__ */
diff --git a/drivers/tty/serial/samsung.c b/drivers/tty/serial/samsung.c
index ac7f8df..99bb231 100644
--- a/drivers/tty/serial/samsung.c
+++ b/drivers/tty/serial/samsung.c
@@ -1271,6 +1271,8 @@
/* check to see if we need to change clock source */
if (ourport->baudclk != clk) {
+ clk_prepare_enable(clk);
+
s3c24xx_serial_setsource(port, clk_sel);
if (!IS_ERR(ourport->baudclk)) {
@@ -1278,8 +1280,6 @@
ourport->baudclk = ERR_PTR(-EINVAL);
}
- clk_prepare_enable(clk);
-
ourport->baudclk = clk;
ourport->baudclk_rate = clk ? clk_get_rate(clk) : 0;
}
diff --git a/drivers/tty/serial/sc16is7xx.c b/drivers/tty/serial/sc16is7xx.c
index e639361..f36e6df 100644
--- a/drivers/tty/serial/sc16is7xx.c
+++ b/drivers/tty/serial/sc16is7xx.c
@@ -666,7 +666,7 @@
struct uart_port *port = &s->p[portno].port;
do {
- unsigned int iir, msr, rxlen;
+ unsigned int iir, rxlen;
iir = sc16is7xx_port_read(port, SC16IS7XX_IIR_REG);
if (iir & SC16IS7XX_IIR_NO_INT_BIT)
@@ -683,12 +683,6 @@
if (rxlen)
sc16is7xx_handle_rx(port, rxlen, iir);
break;
-
- case SC16IS7XX_IIR_CTSRTS_SRC:
- msr = sc16is7xx_port_read(port, SC16IS7XX_MSR_REG);
- uart_handle_cts_change(port,
- !!(msr & SC16IS7XX_MSR_DCTS_BIT));
- break;
case SC16IS7XX_IIR_THRI_SRC:
sc16is7xx_handle_tx(port);
break;
@@ -1014,9 +1008,8 @@
SC16IS7XX_EFCR_TXDISABLE_BIT,
0);
- /* Enable RX, TX, CTS change interrupts */
- val = SC16IS7XX_IER_RDI_BIT | SC16IS7XX_IER_THRI_BIT |
- SC16IS7XX_IER_CTSI_BIT;
+ /* Enable RX, TX interrupts */
+ val = SC16IS7XX_IER_RDI_BIT | SC16IS7XX_IER_THRI_BIT;
sc16is7xx_port_write(port, SC16IS7XX_IER_REG, val);
return 0;
diff --git a/drivers/tty/serial/serial-tegra.c b/drivers/tty/serial/serial-tegra.c
index 1d6fc60..1dba671 100644
--- a/drivers/tty/serial/serial-tegra.c
+++ b/drivers/tty/serial/serial-tegra.c
@@ -206,10 +206,8 @@
static void tegra_uart_set_mctrl(struct uart_port *u, unsigned int mctrl)
{
struct tegra_uart_port *tup = to_tegra_uport(u);
- unsigned long mcr;
int dtr_enable;
- mcr = tup->mcr_shadow;
tup->rts_active = !!(mctrl & TIOCM_RTS);
set_rts(tup, tup->rts_active);
diff --git a/drivers/tty/serial/serial_core.c b/drivers/tty/serial/serial_core.c
index a126a60..a333c59 100644
--- a/drivers/tty/serial/serial_core.c
+++ b/drivers/tty/serial/serial_core.c
@@ -64,6 +64,41 @@
return !!(uport->status & UPSTAT_DCD_ENABLE);
}
+static inline struct uart_port *uart_port_ref(struct uart_state *state)
+{
+ if (atomic_add_unless(&state->refcount, 1, 0))
+ return state->uart_port;
+ return NULL;
+}
+
+static inline void uart_port_deref(struct uart_port *uport)
+{
+ if (uport && atomic_dec_and_test(&uport->state->refcount))
+ wake_up(&uport->state->remove_wait);
+}
+
+#define uart_port_lock(state, flags) \
+ ({ \
+ struct uart_port *__uport = uart_port_ref(state); \
+ if (__uport) \
+ spin_lock_irqsave(&__uport->lock, flags); \
+ __uport; \
+ })
+
+#define uart_port_unlock(uport, flags) \
+ ({ \
+ struct uart_port *__uport = uport; \
+ if (__uport) \
+ spin_unlock_irqrestore(&__uport->lock, flags); \
+ uart_port_deref(__uport); \
+ })
+
+static inline struct uart_port *uart_port_check(struct uart_state *state)
+{
+ lockdep_assert_held(&state->port.mutex);
+ return state->uart_port;
+}
+
/*
* This routine is used by the interrupt handler to schedule processing in
* the software interrupt portion of the driver.
@@ -82,12 +117,13 @@
static void uart_stop(struct tty_struct *tty)
{
struct uart_state *state = tty->driver_data;
- struct uart_port *port = state->uart_port;
+ struct uart_port *port;
unsigned long flags;
- spin_lock_irqsave(&port->lock, flags);
- port->ops->stop_tx(port);
- spin_unlock_irqrestore(&port->lock, flags);
+ port = uart_port_lock(state, flags);
+ if (port)
+ port->ops->stop_tx(port);
+ uart_port_unlock(port, flags);
}
static void __uart_start(struct tty_struct *tty)
@@ -95,19 +131,19 @@
struct uart_state *state = tty->driver_data;
struct uart_port *port = state->uart_port;
- if (!uart_tx_stopped(port))
+ if (port && !uart_tx_stopped(port))
port->ops->start_tx(port);
}
static void uart_start(struct tty_struct *tty)
{
struct uart_state *state = tty->driver_data;
- struct uart_port *port = state->uart_port;
+ struct uart_port *port;
unsigned long flags;
- spin_lock_irqsave(&port->lock, flags);
+ port = uart_port_lock(state, flags);
__uart_start(tty);
- spin_unlock_irqrestore(&port->lock, flags);
+ uart_port_unlock(port, flags);
}
static void
@@ -134,7 +170,7 @@
static int uart_port_startup(struct tty_struct *tty, struct uart_state *state,
int init_hw)
{
- struct uart_port *uport = state->uart_port;
+ struct uart_port *uport = uart_port_check(state);
unsigned long page;
int retval = 0;
@@ -196,7 +232,7 @@
struct tty_port *port = &state->port;
int retval;
- if (port->flags & ASYNC_INITIALIZED)
+ if (tty_port_initialized(port))
return 0;
/*
@@ -207,7 +243,7 @@
retval = uart_port_startup(tty, state, init_hw);
if (!retval) {
- set_bit(ASYNCB_INITIALIZED, &port->flags);
+ tty_port_set_initialized(port, 1);
clear_bit(TTY_IO_ERROR, &tty->flags);
} else if (retval > 0)
retval = 0;
@@ -219,10 +255,12 @@
* This routine will shutdown a serial port; interrupts are disabled, and
* DTR is dropped if the hangup on close termio flag is on. Calls to
* uart_shutdown are serialised by the per-port semaphore.
+ *
+ * uport == NULL if uart_port has already been removed
*/
static void uart_shutdown(struct tty_struct *tty, struct uart_state *state)
{
- struct uart_port *uport = state->uart_port;
+ struct uart_port *uport = uart_port_check(state);
struct tty_port *port = &state->port;
/*
@@ -231,11 +269,13 @@
if (tty)
set_bit(TTY_IO_ERROR, &tty->flags);
- if (test_and_clear_bit(ASYNCB_INITIALIZED, &port->flags)) {
+ if (tty_port_initialized(port)) {
+ tty_port_set_initialized(port, 0);
+
/*
* Turn off DTR and RTS early.
*/
- if (uart_console(uport) && tty)
+ if (uport && uart_console(uport) && tty)
uport->cons->cflag = tty->termios.c_cflag;
if (!tty || C_HUPCL(tty))
@@ -249,7 +289,7 @@
* a DCD drop (hangup) at just the right time. Clear suspended bit so
* we don't try to resume a port that has been shutdown.
*/
- clear_bit(ASYNCB_SUSPENDED, &port->flags);
+ tty_port_set_suspended(port, 0);
/*
* Free the transmit buffer page.
@@ -441,7 +481,7 @@
static void uart_change_speed(struct tty_struct *tty, struct uart_state *state,
struct ktermios *old_termios)
{
- struct uart_port *uport = state->uart_port;
+ struct uart_port *uport = uart_port_check(state);
struct ktermios *termios;
int hw_stopped;
@@ -486,7 +526,7 @@
static int uart_put_char(struct tty_struct *tty, unsigned char c)
{
struct uart_state *state = tty->driver_data;
- struct uart_port *port = state->uart_port;
+ struct uart_port *port;
struct circ_buf *circ;
unsigned long flags;
int ret = 0;
@@ -495,13 +535,13 @@
if (!circ->buf)
return 0;
- spin_lock_irqsave(&port->lock, flags);
- if (uart_circ_chars_free(circ) != 0) {
+ port = uart_port_lock(state, flags);
+ if (port && uart_circ_chars_free(circ) != 0) {
circ->buf[circ->head] = c;
circ->head = (circ->head + 1) & (UART_XMIT_SIZE - 1);
ret = 1;
}
- spin_unlock_irqrestore(&port->lock, flags);
+ uart_port_unlock(port, flags);
return ret;
}
@@ -528,14 +568,12 @@
return -EL3HLT;
}
- port = state->uart_port;
circ = &state->xmit;
-
if (!circ->buf)
return 0;
- spin_lock_irqsave(&port->lock, flags);
- while (1) {
+ port = uart_port_lock(state, flags);
+ while (port) {
c = CIRC_SPACE_TO_END(circ->head, circ->tail, UART_XMIT_SIZE);
if (count < c)
c = count;
@@ -549,32 +587,33 @@
}
__uart_start(tty);
- spin_unlock_irqrestore(&port->lock, flags);
-
+ uart_port_unlock(port, flags);
return ret;
}
static int uart_write_room(struct tty_struct *tty)
{
struct uart_state *state = tty->driver_data;
+ struct uart_port *port;
unsigned long flags;
int ret;
- spin_lock_irqsave(&state->uart_port->lock, flags);
+ port = uart_port_lock(state, flags);
ret = uart_circ_chars_free(&state->xmit);
- spin_unlock_irqrestore(&state->uart_port->lock, flags);
+ uart_port_unlock(port, flags);
return ret;
}
static int uart_chars_in_buffer(struct tty_struct *tty)
{
struct uart_state *state = tty->driver_data;
+ struct uart_port *port;
unsigned long flags;
int ret;
- spin_lock_irqsave(&state->uart_port->lock, flags);
+ port = uart_port_lock(state, flags);
ret = uart_circ_chars_pending(&state->xmit);
- spin_unlock_irqrestore(&state->uart_port->lock, flags);
+ uart_port_unlock(port, flags);
return ret;
}
@@ -593,14 +632,15 @@
return;
}
- port = state->uart_port;
pr_debug("uart_flush_buffer(%d) called\n", tty->index);
- spin_lock_irqsave(&port->lock, flags);
+ port = uart_port_lock(state, flags);
+ if (!port)
+ return;
uart_circ_clear(&state->xmit);
if (port->ops->flush_buffer)
port->ops->flush_buffer(port);
- spin_unlock_irqrestore(&port->lock, flags);
+ uart_port_unlock(port, flags);
tty_wakeup(tty);
}
@@ -611,9 +651,13 @@
static void uart_send_xchar(struct tty_struct *tty, char ch)
{
struct uart_state *state = tty->driver_data;
- struct uart_port *port = state->uart_port;
+ struct uart_port *port;
unsigned long flags;
+ port = uart_port_ref(state);
+ if (!port)
+ return;
+
if (port->ops->send_xchar)
port->ops->send_xchar(port, ch);
else {
@@ -623,14 +667,19 @@
port->ops->start_tx(port);
spin_unlock_irqrestore(&port->lock, flags);
}
+ uart_port_deref(port);
}
static void uart_throttle(struct tty_struct *tty)
{
struct uart_state *state = tty->driver_data;
- struct uart_port *port = state->uart_port;
+ struct uart_port *port;
upstat_t mask = 0;
+ port = uart_port_ref(state);
+ if (!port)
+ return;
+
if (I_IXOFF(tty))
mask |= UPSTAT_AUTOXOFF;
if (C_CRTSCTS(tty))
@@ -646,14 +695,20 @@
if (mask & UPSTAT_AUTOXOFF)
uart_send_xchar(tty, STOP_CHAR(tty));
+
+ uart_port_deref(port);
}
static void uart_unthrottle(struct tty_struct *tty)
{
struct uart_state *state = tty->driver_data;
- struct uart_port *port = state->uart_port;
+ struct uart_port *port;
upstat_t mask = 0;
+ port = uart_port_ref(state);
+ if (!port)
+ return;
+
if (I_IXOFF(tty))
mask |= UPSTAT_AUTOXOFF;
if (C_CRTSCTS(tty))
@@ -669,12 +724,15 @@
if (mask & UPSTAT_AUTOXOFF)
uart_send_xchar(tty, START_CHAR(tty));
+
+ uart_port_deref(port);
}
-static void uart_get_info(struct tty_port *port, struct serial_struct *retinfo)
+static int uart_get_info(struct tty_port *port, struct serial_struct *retinfo)
{
struct uart_state *state = container_of(port, struct uart_state, port);
- struct uart_port *uport = state->uart_port;
+ struct uart_port *uport;
+ int ret = -ENODEV;
memset(retinfo, 0, sizeof(*retinfo));
@@ -683,6 +741,10 @@
* occur as we go
*/
mutex_lock(&port->mutex);
+ uport = uart_port_check(state);
+ if (!uport)
+ goto out;
+
retinfo->type = uport->type;
retinfo->line = uport->line;
retinfo->port = uport->iobase;
@@ -701,7 +763,11 @@
retinfo->io_type = uport->iotype;
retinfo->iomem_reg_shift = uport->regshift;
retinfo->iomem_base = (void *)(unsigned long)uport->mapbase;
+
+ ret = 0;
+out:
mutex_unlock(&port->mutex);
+ return ret;
}
static int uart_get_info_user(struct tty_port *port,
@@ -709,7 +775,8 @@
{
struct serial_struct tmp;
- uart_get_info(port, &tmp);
+ if (uart_get_info(port, &tmp) < 0)
+ return -EIO;
if (copy_to_user(retinfo, &tmp, sizeof(*retinfo)))
return -EFAULT;
@@ -720,13 +787,16 @@
struct uart_state *state,
struct serial_struct *new_info)
{
- struct uart_port *uport = state->uart_port;
+ struct uart_port *uport = uart_port_check(state);
unsigned long new_port;
unsigned int change_irq, change_port, closing_wait;
unsigned int old_custom_divisor, close_delay;
upf_t old_flags, new_flags;
int retval = 0;
+ if (!uport)
+ return -EIO;
+
new_port = new_info->port;
if (HIGH_BITS_OFFSET)
new_port += (unsigned long) new_info->port_high << HIGH_BITS_OFFSET;
@@ -886,7 +956,7 @@
retval = 0;
if (uport->type == PORT_UNKNOWN)
goto exit;
- if (port->flags & ASYNC_INITIALIZED) {
+ if (tty_port_initialized(port)) {
if (((old_flags ^ uport->flags) & UPF_SPD_MASK) ||
old_custom_divisor != uport->custom_divisor) {
/*
@@ -936,13 +1006,11 @@
* @tty: tty associated with the UART
* @state: UART being queried
* @value: returned modem value
- *
- * Note: uart_ioctl protects us against hangups.
*/
static int uart_get_lsr_info(struct tty_struct *tty,
struct uart_state *state, unsigned int __user *value)
{
- struct uart_port *uport = state->uart_port;
+ struct uart_port *uport = uart_port_check(state);
unsigned int result;
result = uport->ops->tx_empty(uport);
@@ -965,18 +1033,22 @@
{
struct uart_state *state = tty->driver_data;
struct tty_port *port = &state->port;
- struct uart_port *uport = state->uart_port;
+ struct uart_port *uport;
int result = -EIO;
mutex_lock(&port->mutex);
- if (!(tty->flags & (1 << TTY_IO_ERROR))) {
+ uport = uart_port_check(state);
+ if (!uport)
+ goto out;
+
+ if (!tty_io_error(tty)) {
result = uport->mctrl;
spin_lock_irq(&uport->lock);
result |= uport->ops->get_mctrl(uport);
spin_unlock_irq(&uport->lock);
}
+out:
mutex_unlock(&port->mutex);
-
return result;
}
@@ -984,15 +1056,20 @@
uart_tiocmset(struct tty_struct *tty, unsigned int set, unsigned int clear)
{
struct uart_state *state = tty->driver_data;
- struct uart_port *uport = state->uart_port;
struct tty_port *port = &state->port;
+ struct uart_port *uport;
int ret = -EIO;
mutex_lock(&port->mutex);
- if (!(tty->flags & (1 << TTY_IO_ERROR))) {
+ uport = uart_port_check(state);
+ if (!uport)
+ goto out;
+
+ if (!tty_io_error(tty)) {
uart_update_mctrl(uport, set, clear);
ret = 0;
}
+out:
mutex_unlock(&port->mutex);
return ret;
}
@@ -1001,21 +1078,26 @@
{
struct uart_state *state = tty->driver_data;
struct tty_port *port = &state->port;
- struct uart_port *uport = state->uart_port;
+ struct uart_port *uport;
+ int ret = -EIO;
mutex_lock(&port->mutex);
+ uport = uart_port_check(state);
+ if (!uport)
+ goto out;
if (uport->type != PORT_UNKNOWN)
uport->ops->break_ctl(uport, break_state);
-
+ ret = 0;
+out:
mutex_unlock(&port->mutex);
- return 0;
+ return ret;
}
static int uart_do_autoconfig(struct tty_struct *tty,struct uart_state *state)
{
- struct uart_port *uport = state->uart_port;
struct tty_port *port = &state->port;
+ struct uart_port *uport;
int flags, ret;
if (!capable(CAP_SYS_ADMIN))
@@ -1029,6 +1111,12 @@
if (mutex_lock_interruptible(&port->mutex))
return -ERESTARTSYS;
+ uport = uart_port_check(state);
+ if (!uport) {
+ ret = -EIO;
+ goto out;
+ }
+
ret = -EBUSY;
if (tty_port_users(port) == 1) {
uart_shutdown(tty, state);
@@ -1052,6 +1140,7 @@
ret = uart_startup(tty, state, 1);
}
+out:
mutex_unlock(&port->mutex);
return ret;
}
@@ -1074,10 +1163,9 @@
* FIXME: This wants extracting into a common all driver implementation
* of TIOCMWAIT using tty_port.
*/
-static int
-uart_wait_modem_status(struct uart_state *state, unsigned long arg)
+static int uart_wait_modem_status(struct uart_state *state, unsigned long arg)
{
- struct uart_port *uport = state->uart_port;
+ struct uart_port *uport;
struct tty_port *port = &state->port;
DECLARE_WAITQUEUE(wait, current);
struct uart_icount cprev, cnow;
@@ -1086,6 +1174,9 @@
/*
* note the counters on entry
*/
+ uport = uart_port_ref(state);
+ if (!uport)
+ return -EIO;
spin_lock_irq(&uport->lock);
memcpy(&cprev, &uport->icount, sizeof(struct uart_icount));
uart_enable_ms(uport);
@@ -1119,6 +1210,7 @@
}
__set_current_state(TASK_RUNNING);
remove_wait_queue(&port->delta_msr_wait, &wait);
+ uart_port_deref(uport);
return ret;
}
@@ -1134,11 +1226,15 @@
{
struct uart_state *state = tty->driver_data;
struct uart_icount cnow;
- struct uart_port *uport = state->uart_port;
+ struct uart_port *uport;
+ uport = uart_port_ref(state);
+ if (!uport)
+ return -EIO;
spin_lock_irq(&uport->lock);
memcpy(&cnow, &uport->icount, sizeof(struct uart_icount));
spin_unlock_irq(&uport->lock);
+ uart_port_deref(uport);
icount->cts = cnow.cts;
icount->dsr = cnow.dsr;
@@ -1200,11 +1296,11 @@
* Called via sys_ioctl. We can use spin_lock_irq() here.
*/
static int
-uart_ioctl(struct tty_struct *tty, unsigned int cmd,
- unsigned long arg)
+uart_ioctl(struct tty_struct *tty, unsigned int cmd, unsigned long arg)
{
struct uart_state *state = tty->driver_data;
struct tty_port *port = &state->port;
+ struct uart_port *uport;
void __user *uarg = (void __user *)arg;
int ret = -ENOIOCTLCMD;
@@ -1238,7 +1334,7 @@
if (ret != -ENOIOCTLCMD)
goto out;
- if (tty->flags & (1 << TTY_IO_ERROR)) {
+ if (tty_io_error(tty)) {
ret = -EIO;
goto out;
}
@@ -1256,8 +1352,9 @@
goto out;
mutex_lock(&port->mutex);
+ uport = uart_port_check(state);
- if (tty->flags & (1 << TTY_IO_ERROR)) {
+ if (!uport || tty_io_error(tty)) {
ret = -EIO;
goto out_up;
}
@@ -1273,19 +1370,17 @@
break;
case TIOCGRS485:
- ret = uart_get_rs485_config(state->uart_port, uarg);
+ ret = uart_get_rs485_config(uport, uarg);
break;
case TIOCSRS485:
- ret = uart_set_rs485_config(state->uart_port, uarg);
+ ret = uart_set_rs485_config(uport, uarg);
break;
- default: {
- struct uart_port *uport = state->uart_port;
+ default:
if (uport->ops->ioctl)
ret = uport->ops->ioctl(uport, cmd, arg);
break;
}
- }
out_up:
mutex_unlock(&port->mutex);
out:
@@ -1295,24 +1390,29 @@
static void uart_set_ldisc(struct tty_struct *tty)
{
struct uart_state *state = tty->driver_data;
- struct uart_port *uport = state->uart_port;
+ struct uart_port *uport;
- if (uport->ops->set_ldisc) {
- mutex_lock(&state->port.mutex);
+ mutex_lock(&state->port.mutex);
+ uport = uart_port_check(state);
+ if (uport && uport->ops->set_ldisc)
uport->ops->set_ldisc(uport, &tty->termios);
- mutex_unlock(&state->port.mutex);
- }
+ mutex_unlock(&state->port.mutex);
}
static void uart_set_termios(struct tty_struct *tty,
struct ktermios *old_termios)
{
struct uart_state *state = tty->driver_data;
- struct uart_port *uport = state->uart_port;
+ struct uart_port *uport;
unsigned int cflag = tty->termios.c_cflag;
unsigned int iflag_mask = IGNBRK|BRKINT|IGNPAR|PARMRK|INPCK;
bool sw_changed = false;
+ mutex_lock(&state->port.mutex);
+ uport = uart_port_check(state);
+ if (!uport)
+ goto out;
+
/*
* Drivers doing software flow control also need to know
* about changes to these input settings.
@@ -1335,12 +1435,10 @@
tty->termios.c_ispeed == old_termios->c_ispeed &&
((tty->termios.c_iflag ^ old_termios->c_iflag) & iflag_mask) == 0 &&
!sw_changed) {
- return;
+ goto out;
}
- mutex_lock(&state->port.mutex);
uart_change_speed(tty, state, old_termios);
- mutex_unlock(&state->port.mutex);
/* reload cflag from termios; port driver may have overriden flags */
cflag = tty->termios.c_cflag;
@@ -1350,17 +1448,18 @@
/* Handle transition away from B0 status */
else if (!(old_termios->c_cflag & CBAUD) && (cflag & CBAUD)) {
unsigned int mask = TIOCM_DTR;
- if (!(cflag & CRTSCTS) || !test_bit(TTY_THROTTLED, &tty->flags))
+ if (!(cflag & CRTSCTS) || !tty_throttled(tty))
mask |= TIOCM_RTS;
uart_set_mctrl(uport, mask);
}
+out:
+ mutex_unlock(&state->port.mutex);
}
/*
* Calls to uart_close() are serialised via the tty_lock in
* drivers/tty/tty_io.c:tty_release()
* drivers/tty/tty_io.c:do_tty_hangup()
- * This runs from a workqueue and can sleep for a _short_ time only.
*/
static void uart_close(struct tty_struct *tty, struct file *filp)
{
@@ -1379,18 +1478,21 @@
return;
}
- uport = state->uart_port;
port = &state->port;
pr_debug("uart_close(%d) called\n", tty->index);
- if (!port->count || tty_port_close_start(port, tty, filp) == 0)
+ if (tty_port_close_start(port, tty, filp) == 0)
return;
+ mutex_lock(&port->mutex);
+ uport = uart_port_check(state);
+
/*
* At this point, we stop accepting input. To do this, we
* disable the receive line status interrupts.
*/
- if (port->flags & ASYNC_INITIALIZED) {
+ if (tty_port_initialized(port) &&
+ !WARN(!uport, "detached port still initialized!\n")) {
spin_lock_irq(&uport->lock);
uport->ops->stop_rx(uport);
spin_unlock_irq(&uport->lock);
@@ -1402,7 +1504,6 @@
uart_wait_until_sent(tty, uport->timeout);
}
- mutex_lock(&port->mutex);
uart_shutdown(tty, state);
tty_port_tty_set(port, NULL);
@@ -1413,17 +1514,17 @@
if (port->close_delay)
msleep_interruptible(jiffies_to_msecs(port->close_delay));
spin_lock_irq(&port->lock);
- } else if (!uart_console(uport)) {
+ } else if (uport && !uart_console(uport)) {
spin_unlock_irq(&port->lock);
uart_change_pm(state, UART_PM_STATE_OFF);
spin_lock_irq(&port->lock);
}
+ spin_unlock_irq(&port->lock);
+ tty_port_set_active(port, 0);
/*
* Wake up anyone trying to open this port.
*/
- clear_bit(ASYNCB_NORMAL_ACTIVE, &port->flags);
- spin_unlock_irq(&port->lock);
wake_up_interruptible(&port->open_wait);
mutex_unlock(&port->mutex);
@@ -1435,11 +1536,14 @@
static void uart_wait_until_sent(struct tty_struct *tty, int timeout)
{
struct uart_state *state = tty->driver_data;
- struct uart_port *port = state->uart_port;
+ struct uart_port *port;
unsigned long char_time, expire;
- if (port->type == PORT_UNKNOWN || port->fifosize == 0)
+ port = uart_port_ref(state);
+ if (!port || port->type == PORT_UNKNOWN || port->fifosize == 0) {
+ uart_port_deref(port);
return;
+ }
/*
* Set the check interval to be 1/5 of the estimated time to
@@ -1485,6 +1589,7 @@
if (time_after(jiffies, expire))
break;
}
+ uart_port_deref(port);
}
/*
@@ -1496,20 +1601,24 @@
{
struct uart_state *state = tty->driver_data;
struct tty_port *port = &state->port;
+ struct uart_port *uport;
unsigned long flags;
pr_debug("uart_hangup(%d)\n", tty->index);
mutex_lock(&port->mutex);
- if (port->flags & ASYNC_NORMAL_ACTIVE) {
+ uport = uart_port_check(state);
+ WARN(!uport, "hangup of detached port!\n");
+
+ if (tty_port_active(port)) {
uart_flush_buffer(tty);
uart_shutdown(tty, state);
spin_lock_irqsave(&port->lock, flags);
port->count = 0;
- clear_bit(ASYNCB_NORMAL_ACTIVE, &port->flags);
spin_unlock_irqrestore(&port->lock, flags);
+ tty_port_set_active(port, 0);
tty_port_tty_set(port, NULL);
- if (!uart_console(state->uart_port))
+ if (uport && !uart_console(uport))
uart_change_pm(state, UART_PM_STATE_OFF);
wake_up_interruptible(&port->open_wait);
wake_up_interruptible(&port->delta_msr_wait);
@@ -1517,10 +1626,11 @@
mutex_unlock(&port->mutex);
}
+/* uport == NULL if uart_port has already been removed */
static void uart_port_shutdown(struct tty_port *port)
{
struct uart_state *state = container_of(port, struct uart_state, port);
- struct uart_port *uport = state->uart_port;
+ struct uart_port *uport = uart_port_check(state);
/*
* clear delta_msr_wait queue to avoid mem leaks: we may free
@@ -1534,23 +1644,36 @@
/*
* Free the IRQ and disable the port.
*/
- uport->ops->shutdown(uport);
+ if (uport)
+ uport->ops->shutdown(uport);
/*
* Ensure that the IRQ handler isn't running on another CPU.
*/
- synchronize_irq(uport->irq);
+ if (uport)
+ synchronize_irq(uport->irq);
}
static int uart_carrier_raised(struct tty_port *port)
{
struct uart_state *state = container_of(port, struct uart_state, port);
- struct uart_port *uport = state->uart_port;
+ struct uart_port *uport;
int mctrl;
+
+ uport = uart_port_ref(state);
+ /*
+ * Should never observe uport == NULL since checks for hangup should
+ * abort the tty_port_block_til_ready() loop before checking for carrier
+ * raised -- but report carrier raised if it does anyway so open will
+ * continue and not sleep
+ */
+ if (WARN_ON(!uport))
+ return 1;
spin_lock_irq(&uport->lock);
uart_enable_ms(uport);
mctrl = uport->ops->get_mctrl(uport);
spin_unlock_irq(&uport->lock);
+ uart_port_deref(uport);
if (mctrl & TIOCM_CAR)
return 1;
return 0;
@@ -1559,12 +1682,18 @@
static void uart_dtr_rts(struct tty_port *port, int onoff)
{
struct uart_state *state = container_of(port, struct uart_state, port);
- struct uart_port *uport = state->uart_port;
+ struct uart_port *uport;
+
+ uport = uart_port_ref(state);
+ if (!uport)
+ return;
if (onoff)
uart_set_mctrl(uport, TIOCM_DTR | TIOCM_RTS);
else
uart_clear_mctrl(uport, TIOCM_DTR | TIOCM_RTS);
+
+ uart_port_deref(uport);
}
/*
@@ -1583,6 +1712,7 @@
int retval, line = tty->index;
struct uart_state *state = drv->state + line;
struct tty_port *port = &state->port;
+ struct uart_port *uport;
pr_debug("uart_open(%d) called\n", line);
@@ -1602,15 +1732,15 @@
goto end;
}
- if (!state->uart_port || state->uart_port->flags & UPF_DEAD) {
+ uport = uart_port_check(state);
+ if (!uport || uport->flags & UPF_DEAD) {
retval = -ENXIO;
goto err_unlock;
}
tty->driver_data = state;
- state->uart_port->state = state;
- state->port.low_latency =
- (state->uart_port->flags & UPF_LOW_LATENCY) ? 1 : 0;
+ uport->state = state;
+ port->low_latency = (uport->flags & UPF_LOW_LATENCY) ? 1 : 0;
tty_port_tty_set(port, tty);
/*
@@ -1649,13 +1779,15 @@
struct uart_state *state = drv->state + i;
struct tty_port *port = &state->port;
enum uart_pm_state pm_state;
- struct uart_port *uport = state->uart_port;
+ struct uart_port *uport;
char stat_buf[32];
unsigned int status;
int mmio;
+ mutex_lock(&port->mutex);
+ uport = uart_port_check(state);
if (!uport)
- return;
+ goto out;
mmio = uport->iotype >= UPIO_MEM;
seq_printf(m, "%d: uart:%s %s%08llX irq:%d",
@@ -1667,11 +1799,10 @@
if (uport->type == PORT_UNKNOWN) {
seq_putc(m, '\n');
- return;
+ goto out;
}
if (capable(CAP_SYS_ADMIN)) {
- mutex_lock(&port->mutex);
pm_state = state->pm_state;
if (pm_state != UART_PM_STATE_ON)
uart_change_pm(state, UART_PM_STATE_ON);
@@ -1680,7 +1811,6 @@
spin_unlock_irq(&uport->lock);
if (pm_state != UART_PM_STATE_ON)
uart_change_pm(state, pm_state);
- mutex_unlock(&port->mutex);
seq_printf(m, " tx:%d rx:%d",
uport->icount.tx, uport->icount.rx);
@@ -1718,6 +1848,8 @@
seq_putc(m, '\n');
#undef STATBIT
#undef INFOBIT
+out:
+ mutex_unlock(&port->mutex);
}
static int uart_proc_show(struct seq_file *m, void *v)
@@ -1954,10 +2086,10 @@
static void uart_change_pm(struct uart_state *state,
enum uart_pm_state pm_state)
{
- struct uart_port *port = state->uart_port;
+ struct uart_port *port = uart_port_check(state);
if (state->pm_state != pm_state) {
- if (port->ops->pm)
+ if (port && port->ops->pm)
port->ops->pm(port, pm_state, state->pm_state);
state->pm_state = pm_state;
}
@@ -2003,12 +2135,12 @@
uport->suspended = 1;
- if (port->flags & ASYNC_INITIALIZED) {
+ if (tty_port_initialized(port)) {
const struct uart_ops *ops = uport->ops;
int tries;
- set_bit(ASYNCB_SUSPENDED, &port->flags);
- clear_bit(ASYNCB_INITIALIZED, &port->flags);
+ tty_port_set_suspended(port, 1);
+ tty_port_set_initialized(port, 0);
spin_lock_irq(&uport->lock);
ops->stop_tx(uport);
@@ -2088,7 +2220,7 @@
console_start(uport->cons);
}
- if (port->flags & ASYNC_SUSPENDED) {
+ if (tty_port_suspended(port)) {
const struct uart_ops *ops = uport->ops;
int ret;
@@ -2107,7 +2239,7 @@
ops->set_mctrl(uport, uport->mctrl);
ops->start_tx(uport);
spin_unlock_irq(&uport->lock);
- set_bit(ASYNCB_INITIALIZED, &port->flags);
+ tty_port_set_initialized(port, 1);
} else {
/*
* Failed to resume - maybe hardware went away?
@@ -2118,7 +2250,7 @@
}
}
- clear_bit(ASYNCB_SUSPENDED, &port->flags);
+ tty_port_set_suspended(port, 0);
}
mutex_unlock(&port->mutex);
@@ -2228,42 +2360,42 @@
{
struct uart_driver *drv = driver->driver_state;
struct uart_state *state = drv->state + line;
+ struct tty_port *tport;
struct uart_port *port;
int baud = 9600;
int bits = 8;
int parity = 'n';
int flow = 'n';
- int ret;
+ int ret = 0;
- if (!state || !state->uart_port)
+ if (!state)
return -1;
- port = state->uart_port;
- if (!(port->ops->poll_get_char && port->ops->poll_put_char))
- return -1;
+ tport = &state->port;
+ mutex_lock(&tport->mutex);
+
+ port = uart_port_check(state);
+ if (!port || !(port->ops->poll_get_char && port->ops->poll_put_char)) {
+ ret = -1;
+ goto out;
+ }
if (port->ops->poll_init) {
- struct tty_port *tport = &state->port;
-
- ret = 0;
- mutex_lock(&tport->mutex);
/*
- * We don't set ASYNCB_INITIALIZED as we only initialized the
- * hw, e.g. state->xmit is still uninitialized.
+ * We don't set initialized as we only initialized the hw,
+ * e.g. state->xmit is still uninitialized.
*/
- if (!test_bit(ASYNCB_INITIALIZED, &tport->flags))
+ if (!tty_port_initialized(tport))
ret = port->ops->poll_init(port);
- mutex_unlock(&tport->mutex);
- if (ret)
- return ret;
}
- if (options) {
+ if (!ret && options) {
uart_parse_options(options, &baud, &parity, &bits, &flow);
- return uart_set_options(port, NULL, baud, parity, bits, flow);
+ ret = uart_set_options(port, NULL, baud, parity, bits, flow);
}
-
- return 0;
+out:
+ mutex_unlock(&tport->mutex);
+ return ret;
}
static int uart_poll_get_char(struct tty_driver *driver, int line)
@@ -2271,12 +2403,15 @@
struct uart_driver *drv = driver->driver_state;
struct uart_state *state = drv->state + line;
struct uart_port *port;
+ int ret = -1;
- if (!state || !state->uart_port)
- return -1;
-
- port = state->uart_port;
- return port->ops->poll_get_char(port);
+ if (state) {
+ port = uart_port_ref(state);
+ if (port)
+ ret = port->ops->poll_get_char(port);
+ uart_port_deref(port);
+ }
+ return ret;
}
static void uart_poll_put_char(struct tty_driver *driver, int line, char ch)
@@ -2285,14 +2420,17 @@
struct uart_state *state = drv->state + line;
struct uart_port *port;
- if (!state || !state->uart_port)
+ if (!state)
return;
- port = state->uart_port;
+ port = uart_port_ref(state);
+ if (!port)
+ return;
if (ch == '\n')
port->ops->poll_put_char(port, '\r');
port->ops->poll_put_char(port, ch);
+ uart_port_deref(port);
}
#endif
@@ -2639,6 +2777,8 @@
}
/* Link the port to the driver state table and vice versa */
+ atomic_set(&state->refcount, 1);
+ init_waitqueue_head(&state->remove_wait);
state->uart_port = uport;
uport->state = state;
@@ -2711,15 +2851,12 @@
{
struct uart_state *state = drv->state + uport->line;
struct tty_port *port = &state->port;
+ struct uart_port *uart_port;
struct tty_struct *tty;
int ret = 0;
BUG_ON(in_interrupt());
- if (state->uart_port != uport)
- dev_alert(uport->dev, "Removing wrong port: %p != %p\n",
- state->uart_port, uport);
-
mutex_lock(&port_mutex);
/*
@@ -2727,7 +2864,12 @@
* succeeding while we shut down the port.
*/
mutex_lock(&port->mutex);
- if (!state->uart_port) {
+ uart_port = uart_port_check(state);
+ if (uart_port != uport)
+ dev_alert(uport->dev, "Removing wrong port: %p != %p\n",
+ uart_port, uport);
+
+ if (!uart_port) {
mutex_unlock(&port->mutex);
ret = -EINVAL;
goto out;
@@ -2764,7 +2906,11 @@
*/
uport->type = PORT_UNKNOWN;
+ mutex_lock(&port->mutex);
+ WARN_ON(atomic_dec_return(&state->refcount) < 0);
+ wait_event(state->remove_wait, !atomic_read(&state->refcount));
state->uart_port = NULL;
+ mutex_unlock(&port->mutex);
out:
mutex_unlock(&port_mutex);
diff --git a/drivers/tty/serial/serial_mctrl_gpio.c b/drivers/tty/serial/serial_mctrl_gpio.c
index 0214736..e8dd509 100644
--- a/drivers/tty/serial/serial_mctrl_gpio.c
+++ b/drivers/tty/serial/serial_mctrl_gpio.c
@@ -43,8 +43,6 @@
{ "rng", TIOCM_RNG, false, },
{ "rts", TIOCM_RTS, true, },
{ "dtr", TIOCM_DTR, true, },
- { "out1", TIOCM_OUT1, true, },
- { "out2", TIOCM_OUT2, true, },
};
void mctrl_gpio_set(struct mctrl_gpios *gpios, unsigned int mctrl)
@@ -125,9 +123,12 @@
struct uart_port *port = gpios->port;
u32 mctrl = gpios->mctrl_prev;
u32 mctrl_diff;
+ unsigned long flags;
mctrl_gpio_get(gpios, &mctrl);
+ spin_lock_irqsave(&port->lock, flags);
+
mctrl_diff = mctrl ^ gpios->mctrl_prev;
gpios->mctrl_prev = mctrl;
@@ -147,6 +148,8 @@
wake_up_interruptible(&port->state->port.delta_msr_wait);
}
+ spin_unlock_irqrestore(&port->lock, flags);
+
return IRQ_HANDLED;
}
diff --git a/drivers/tty/serial/serial_mctrl_gpio.h b/drivers/tty/serial/serial_mctrl_gpio.h
index bcfad5d..332a33a 100644
--- a/drivers/tty/serial/serial_mctrl_gpio.h
+++ b/drivers/tty/serial/serial_mctrl_gpio.h
@@ -32,8 +32,6 @@
UART_GPIO_RI = UART_GPIO_RNG,
UART_GPIO_RTS,
UART_GPIO_DTR,
- UART_GPIO_OUT1,
- UART_GPIO_OUT2,
UART_GPIO_MAX,
};
diff --git a/drivers/tty/serial/sirfsoc_uart.c b/drivers/tty/serial/sirfsoc_uart.c
index c6657de..b186c9c 100644
--- a/drivers/tty/serial/sirfsoc_uart.c
+++ b/drivers/tty/serial/sirfsoc_uart.c
@@ -1264,6 +1264,7 @@
static int sirfsoc_uart_probe(struct platform_device *pdev)
{
+ struct device_node *np = pdev->dev.of_node;
struct sirfsoc_uart_port *sirfport;
struct uart_port *port;
struct resource *res;
@@ -1276,13 +1277,13 @@
};
const struct of_device_id *match;
- match = of_match_node(sirfsoc_uart_ids, pdev->dev.of_node);
+ match = of_match_node(sirfsoc_uart_ids, np);
sirfport = devm_kzalloc(&pdev->dev, sizeof(*sirfport), GFP_KERNEL);
if (!sirfport) {
ret = -ENOMEM;
goto err;
}
- sirfport->port.line = of_alias_get_id(pdev->dev.of_node, "serial");
+ sirfport->port.line = of_alias_get_id(np, "serial");
sirf_ports[sirfport->port.line] = sirfport;
sirfport->port.iotype = UPIO_MEM;
sirfport->port.flags = UPF_BOOT_AUTOCONF;
@@ -1291,25 +1292,25 @@
port->private_data = sirfport;
sirfport->uart_reg = (struct sirfsoc_uart_register *)match->data;
- sirfport->hw_flow_ctrl = of_property_read_bool(pdev->dev.of_node,
- "sirf,uart-has-rtscts");
- if (of_device_is_compatible(pdev->dev.of_node, "sirf,prima2-uart") ||
- of_device_is_compatible(pdev->dev.of_node, "sirf,atlas7-uart"))
+ sirfport->hw_flow_ctrl =
+ of_property_read_bool(np, "uart-has-rtscts") ||
+ of_property_read_bool(np, "sirf,uart-has-rtscts") /* deprecated */;
+ if (of_device_is_compatible(np, "sirf,prima2-uart") ||
+ of_device_is_compatible(np, "sirf,atlas7-uart"))
sirfport->uart_reg->uart_type = SIRF_REAL_UART;
- if (of_device_is_compatible(pdev->dev.of_node,
- "sirf,prima2-usp-uart") || of_device_is_compatible(
- pdev->dev.of_node, "sirf,atlas7-usp-uart")) {
+ if (of_device_is_compatible(np, "sirf,prima2-usp-uart") ||
+ of_device_is_compatible(np, "sirf,atlas7-usp-uart")) {
sirfport->uart_reg->uart_type = SIRF_USP_UART;
if (!sirfport->hw_flow_ctrl)
goto usp_no_flow_control;
- if (of_find_property(pdev->dev.of_node, "cts-gpios", NULL))
- sirfport->cts_gpio = of_get_named_gpio(
- pdev->dev.of_node, "cts-gpios", 0);
+ if (of_find_property(np, "cts-gpios", NULL))
+ sirfport->cts_gpio =
+ of_get_named_gpio(np, "cts-gpios", 0);
else
sirfport->cts_gpio = -1;
- if (of_find_property(pdev->dev.of_node, "rts-gpios", NULL))
- sirfport->rts_gpio = of_get_named_gpio(
- pdev->dev.of_node, "rts-gpios", 0);
+ if (of_find_property(np, "rts-gpios", NULL))
+ sirfport->rts_gpio =
+ of_get_named_gpio(np, "rts-gpios", 0);
else
sirfport->rts_gpio = -1;
@@ -1336,13 +1337,11 @@
gpio_direction_output(sirfport->rts_gpio, 1);
}
usp_no_flow_control:
- if (of_device_is_compatible(pdev->dev.of_node, "sirf,atlas7-uart") ||
- of_device_is_compatible(pdev->dev.of_node, "sirf,atlas7-usp-uart"))
+ if (of_device_is_compatible(np, "sirf,atlas7-uart") ||
+ of_device_is_compatible(np, "sirf,atlas7-usp-uart"))
sirfport->is_atlas7 = true;
- if (of_property_read_u32(pdev->dev.of_node,
- "fifosize",
- &port->fifosize)) {
+ if (of_property_read_u32(np, "fifosize", &port->fifosize)) {
dev_err(&pdev->dev,
"Unable to find fifosize in uart node.\n");
ret = -EFAULT;
diff --git a/drivers/tty/serial/uartlite.c b/drivers/tty/serial/uartlite.c
index d08baa6..05089b6 100644
--- a/drivers/tty/serial/uartlite.c
+++ b/drivers/tty/serial/uartlite.c
@@ -72,7 +72,7 @@
iowrite32be(val, addr);
}
-static struct uartlite_reg_ops uartlite_be = {
+static const struct uartlite_reg_ops uartlite_be = {
.in = uartlite_inbe32,
.out = uartlite_outbe32,
};
@@ -87,21 +87,21 @@
iowrite32(val, addr);
}
-static struct uartlite_reg_ops uartlite_le = {
+static const struct uartlite_reg_ops uartlite_le = {
.in = uartlite_inle32,
.out = uartlite_outle32,
};
static inline u32 uart_in32(u32 offset, struct uart_port *port)
{
- struct uartlite_reg_ops *reg_ops = port->private_data;
+ const struct uartlite_reg_ops *reg_ops = port->private_data;
return reg_ops->in(port->membase + offset);
}
static inline void uart_out32(u32 val, u32 offset, struct uart_port *port)
{
- struct uartlite_reg_ops *reg_ops = port->private_data;
+ const struct uartlite_reg_ops *reg_ops = port->private_data;
reg_ops->out(val, port->membase + offset);
}
@@ -345,13 +345,13 @@
return -EBUSY;
}
- port->private_data = &uartlite_be;
+ port->private_data = (void *)&uartlite_be;
ret = uart_in32(ULITE_CONTROL, port);
uart_out32(ULITE_CONTROL_RST_TX, ULITE_CONTROL, port);
ret = uart_in32(ULITE_STATUS, port);
/* Endianess detection */
if ((ret & ULITE_STATUS_TXEMPTY) != ULITE_STATUS_TXEMPTY)
- port->private_data = &uartlite_le;
+ port->private_data = (void *)&uartlite_le;
return 0;
}
diff --git a/drivers/tty/serial/ucc_uart.c b/drivers/tty/serial/ucc_uart.c
index 1a7dc3c..481eb29 100644
--- a/drivers/tty/serial/ucc_uart.c
+++ b/drivers/tty/serial/ucc_uart.c
@@ -1478,6 +1478,9 @@
.type = "serial",
.compatible = "ucc_uart",
},
+ {
+ .compatible = "fsl,t1040-ucc-uart",
+ },
{},
};
MODULE_DEVICE_TABLE(of, ucc_uart_match);
diff --git a/drivers/tty/synclink.c b/drivers/tty/synclink.c
index c8c7601..c13e27e 100644
--- a/drivers/tty/synclink.c
+++ b/drivers/tty/synclink.c
@@ -1340,7 +1340,7 @@
wake_up_interruptible(&info->status_event_wait_q);
wake_up_interruptible(&info->event_wait_q);
- if ( (info->port.flags & ASYNC_CHECK_CD) &&
+ if (tty_port_check_carrier(&info->port) &&
(status & MISCSTATUS_DCD_LATCHED) ) {
if ( debug_level >= DEBUG_LEVEL_ISR )
printk("%s CD now %s...", info->device_name,
@@ -1361,8 +1361,7 @@
if (status & MISCSTATUS_CTS) {
if ( debug_level >= DEBUG_LEVEL_ISR )
printk("CTS tx start...");
- if (info->port.tty)
- info->port.tty->hw_stopped = 0;
+ info->port.tty->hw_stopped = 0;
usc_start_transmitter(info);
info->pending_bh |= BH_TRANSMIT;
return;
@@ -1749,13 +1748,13 @@
static int startup(struct mgsl_struct * info)
{
int retval = 0;
-
+
if ( debug_level >= DEBUG_LEVEL_INFO )
printk("%s(%d):mgsl_startup(%s)\n",__FILE__,__LINE__,info->device_name);
-
- if (info->port.flags & ASYNC_INITIALIZED)
+
+ if (tty_port_initialized(&info->port))
return 0;
-
+
if (!info->xmit_buf) {
/* allocate a page of memory for a transmit buffer */
info->xmit_buf = (unsigned char *)get_zeroed_page(GFP_KERNEL);
@@ -1788,14 +1787,13 @@
/* program hardware for current parameters */
mgsl_change_params(info);
-
+
if (info->port.tty)
clear_bit(TTY_IO_ERROR, &info->port.tty->flags);
- info->port.flags |= ASYNC_INITIALIZED;
-
+ tty_port_set_initialized(&info->port, 1);
+
return 0;
-
} /* end of startup() */
/* shutdown()
@@ -1808,8 +1806,8 @@
static void shutdown(struct mgsl_struct * info)
{
unsigned long flags;
-
- if (!(info->port.flags & ASYNC_INITIALIZED))
+
+ if (!tty_port_initialized(&info->port))
return;
if (debug_level >= DEBUG_LEVEL_INFO)
@@ -1853,13 +1851,12 @@
spin_unlock_irqrestore(&info->irq_spinlock,flags);
- mgsl_release_resources(info);
-
+ mgsl_release_resources(info);
+
if (info->port.tty)
set_bit(TTY_IO_ERROR, &info->port.tty->flags);
- info->port.flags &= ~ASYNC_INITIALIZED;
-
+ tty_port_set_initialized(&info->port, 0);
} /* end of shutdown() */
static void mgsl_program_hw(struct mgsl_struct *info)
@@ -1966,15 +1963,8 @@
}
info->timeout += HZ/50; /* Add .02 seconds of slop */
- if (cflag & CRTSCTS)
- info->port.flags |= ASYNC_CTS_FLOW;
- else
- info->port.flags &= ~ASYNC_CTS_FLOW;
-
- if (cflag & CLOCAL)
- info->port.flags &= ~ASYNC_CHECK_CD;
- else
- info->port.flags |= ASYNC_CHECK_CD;
+ tty_port_set_cts_flow(&info->port, cflag & CRTSCTS);
+ tty_port_set_check_carrier(&info->port, ~cflag & CLOCAL);
/* process tty input control flags */
@@ -2972,7 +2962,7 @@
if ((cmd != TIOCGSERIAL) && (cmd != TIOCSSERIAL) &&
(cmd != TIOCMIWAIT)) {
- if (tty->flags & (1 << TTY_IO_ERROR))
+ if (tty_io_error(tty))
return -EIO;
}
@@ -3049,7 +3039,7 @@
/* Handle transition away from B0 status */
if (!(old_termios->c_cflag & CBAUD) && C_BAUD(tty)) {
info->serial_signals |= SerialSignal_DTR;
- if (!C_CRTSCTS(tty) || !test_bit(TTY_THROTTLED, &tty->flags))
+ if (!C_CRTSCTS(tty) || !tty_throttled(tty))
info->serial_signals |= SerialSignal_RTS;
spin_lock_irqsave(&info->irq_spinlock,flags);
usc_set_serial_signals(info);
@@ -3091,7 +3081,7 @@
goto cleanup;
mutex_lock(&info->port.mutex);
- if (info->port.flags & ASYNC_INITIALIZED)
+ if (tty_port_initialized(&info->port))
mgsl_wait_until_sent(tty, info->timeout);
mgsl_flush_buffer(tty);
tty_ldisc_flush(tty);
@@ -3129,15 +3119,15 @@
if (debug_level >= DEBUG_LEVEL_INFO)
printk("%s(%d):mgsl_wait_until_sent(%s) entry\n",
__FILE__,__LINE__, info->device_name );
-
+
if (mgsl_paranoia_check(info, tty->name, "mgsl_wait_until_sent"))
return;
- if (!(info->port.flags & ASYNC_INITIALIZED))
+ if (!tty_port_initialized(&info->port))
goto exit;
-
+
orig_jiffies = jiffies;
-
+
/* Set check interval to 1/5 of estimated time to
* send a character, and make it at least 1. The check
* interval should also be less than the timeout.
@@ -3204,7 +3194,7 @@
shutdown(info);
info->port.count = 0;
- info->port.flags &= ~ASYNC_NORMAL_ACTIVE;
+ tty_port_set_active(&info->port, 0);
info->port.tty = NULL;
wake_up_interruptible(&info->port.open_wait);
@@ -3270,9 +3260,9 @@
printk("%s(%d):block_til_ready on %s\n",
__FILE__,__LINE__, tty->driver->name );
- if (filp->f_flags & O_NONBLOCK || tty->flags & (1 << TTY_IO_ERROR)){
+ if (filp->f_flags & O_NONBLOCK || tty_io_error(tty)) {
/* nonblock mode is set or port is not enabled */
- port->flags |= ASYNC_NORMAL_ACTIVE;
+ tty_port_set_active(port, 1);
return 0;
}
@@ -3297,14 +3287,14 @@
port->count--;
spin_unlock_irqrestore(&info->irq_spinlock, flags);
port->blocked_open++;
-
+
while (1) {
- if (C_BAUD(tty) && test_bit(ASYNCB_INITIALIZED, &port->flags))
+ if (C_BAUD(tty) && tty_port_initialized(port))
tty_port_raise_dtr_rts(port);
-
+
set_current_state(TASK_INTERRUPTIBLE);
-
- if (tty_hung_up_p(filp) || !(port->flags & ASYNC_INITIALIZED)){
+
+ if (tty_hung_up_p(filp) || !tty_port_initialized(port)) {
retval = (port->flags & ASYNC_HUP_NOTIFY) ?
-EAGAIN : -ERESTARTSYS;
break;
@@ -3341,7 +3331,7 @@
__FILE__,__LINE__, tty->driver->name, port->count );
if (!retval)
- port->flags |= ASYNC_NORMAL_ACTIVE;
+ tty_port_set_active(port, 1);
return retval;
diff --git a/drivers/tty/synclink_gt.c b/drivers/tty/synclink_gt.c
index d5b6471..7aca2d4 100644
--- a/drivers/tty/synclink_gt.c
+++ b/drivers/tty/synclink_gt.c
@@ -726,7 +726,7 @@
goto cleanup;
mutex_lock(&info->port.mutex);
- if (info->port.flags & ASYNC_INITIALIZED)
+ if (tty_port_initialized(&info->port))
wait_until_sent(tty, info->timeout);
flush_buffer(tty);
tty_ldisc_flush(tty);
@@ -756,9 +756,9 @@
spin_lock_irqsave(&info->port.lock, flags);
info->port.count = 0;
- info->port.flags &= ~ASYNC_NORMAL_ACTIVE;
info->port.tty = NULL;
spin_unlock_irqrestore(&info->port.lock, flags);
+ tty_port_set_active(&info->port, 0);
mutex_unlock(&info->port.mutex);
wake_up_interruptible(&info->port.open_wait);
@@ -784,7 +784,7 @@
/* Handle transition away from B0 status */
if (!(old_termios->c_cflag & CBAUD) && C_BAUD(tty)) {
info->signals |= SerialSignal_DTR;
- if (!C_CRTSCTS(tty) || !test_bit(TTY_THROTTLED, &tty->flags))
+ if (!C_CRTSCTS(tty) || !tty_throttled(tty))
info->signals |= SerialSignal_RTS;
spin_lock_irqsave(&info->lock,flags);
set_signals(info);
@@ -893,7 +893,7 @@
if (sanity_check(info, tty->name, "wait_until_sent"))
return;
DBGINFO(("%s wait_until_sent entry\n", info->device_name));
- if (!(info->port.flags & ASYNC_INITIALIZED))
+ if (!tty_port_initialized(&info->port))
goto exit;
orig_jiffies = jiffies;
@@ -1032,7 +1032,7 @@
if ((cmd != TIOCGSERIAL) && (cmd != TIOCSSERIAL) &&
(cmd != TIOCMIWAIT)) {
- if (tty->flags & (1 << TTY_IO_ERROR))
+ if (tty_io_error(tty))
return -EIO;
}
@@ -2080,7 +2080,7 @@
wake_up_interruptible(&info->event_wait_q);
info->pending_bh |= BH_STATUS;
- if (info->port.flags & ASYNC_CHECK_CD) {
+ if (tty_port_check_carrier(&info->port)) {
if (info->signals & SerialSignal_DCD)
wake_up_interruptible(&info->port.open_wait);
else {
@@ -2421,7 +2421,7 @@
{
DBGINFO(("%s startup\n", info->device_name));
- if (info->port.flags & ASYNC_INITIALIZED)
+ if (tty_port_initialized(&info->port))
return 0;
if (!info->tx_buf) {
@@ -2442,7 +2442,7 @@
if (info->port.tty)
clear_bit(TTY_IO_ERROR, &info->port.tty->flags);
- info->port.flags |= ASYNC_INITIALIZED;
+ tty_port_set_initialized(&info->port, 1);
return 0;
}
@@ -2454,7 +2454,7 @@
{
unsigned long flags;
- if (!(info->port.flags & ASYNC_INITIALIZED))
+ if (!tty_port_initialized(&info->port))
return;
DBGINFO(("%s shutdown\n", info->device_name));
@@ -2489,7 +2489,7 @@
if (info->port.tty)
set_bit(TTY_IO_ERROR, &info->port.tty->flags);
- info->port.flags &= ~ASYNC_INITIALIZED;
+ tty_port_set_initialized(&info->port, 0);
}
static void program_hw(struct slgt_info *info)
@@ -2576,15 +2576,8 @@
}
info->timeout += HZ/50; /* Add .02 seconds of slop */
- if (cflag & CRTSCTS)
- info->port.flags |= ASYNC_CTS_FLOW;
- else
- info->port.flags &= ~ASYNC_CTS_FLOW;
-
- if (cflag & CLOCAL)
- info->port.flags &= ~ASYNC_CHECK_CD;
- else
- info->port.flags |= ASYNC_CHECK_CD;
+ tty_port_set_cts_flow(&info->port, cflag & CRTSCTS);
+ tty_port_set_check_carrier(&info->port, ~cflag & CLOCAL);
/* process tty input control flags */
@@ -3269,9 +3262,9 @@
DBGINFO(("%s block_til_ready\n", tty->driver->name));
- if (filp->f_flags & O_NONBLOCK || tty->flags & (1 << TTY_IO_ERROR)){
+ if (filp->f_flags & O_NONBLOCK || tty_io_error(tty)) {
/* nonblock mode is set or port is not enabled */
- port->flags |= ASYNC_NORMAL_ACTIVE;
+ tty_port_set_active(port, 1);
return 0;
}
@@ -3294,12 +3287,12 @@
port->blocked_open++;
while (1) {
- if (C_BAUD(tty) && test_bit(ASYNCB_INITIALIZED, &port->flags))
+ if (C_BAUD(tty) && tty_port_initialized(port))
tty_port_raise_dtr_rts(port);
set_current_state(TASK_INTERRUPTIBLE);
- if (tty_hung_up_p(filp) || !(port->flags & ASYNC_INITIALIZED)){
+ if (tty_hung_up_p(filp) || !tty_port_initialized(port)) {
retval = (port->flags & ASYNC_HUP_NOTIFY) ?
-EAGAIN : -ERESTARTSYS;
break;
@@ -3328,7 +3321,7 @@
port->blocked_open--;
if (!retval)
- port->flags |= ASYNC_NORMAL_ACTIVE;
+ tty_port_set_active(port, 1);
DBGINFO(("%s block_til_ready ready, rc=%d\n", tty->driver->name, retval));
return retval;
diff --git a/drivers/tty/synclinkmp.c b/drivers/tty/synclinkmp.c
index 3f89685..dec1565 100644
--- a/drivers/tty/synclinkmp.c
+++ b/drivers/tty/synclinkmp.c
@@ -812,7 +812,7 @@
goto cleanup;
mutex_lock(&info->port.mutex);
- if (info->port.flags & ASYNC_INITIALIZED)
+ if (tty_port_initialized(&info->port))
wait_until_sent(tty, info->timeout);
flush_buffer(tty);
@@ -849,9 +849,9 @@
spin_lock_irqsave(&info->port.lock, flags);
info->port.count = 0;
- info->port.flags &= ~ASYNC_NORMAL_ACTIVE;
info->port.tty = NULL;
spin_unlock_irqrestore(&info->port.lock, flags);
+ tty_port_set_active(&info->port, 1);
mutex_unlock(&info->port.mutex);
wake_up_interruptible(&info->port.open_wait);
@@ -881,7 +881,7 @@
/* Handle transition away from B0 status */
if (!(old_termios->c_cflag & CBAUD) && C_BAUD(tty)) {
info->serial_signals |= SerialSignal_DTR;
- if (!C_CRTSCTS(tty) || !test_bit(TTY_THROTTLED, &tty->flags))
+ if (!C_CRTSCTS(tty) || !tty_throttled(tty))
info->serial_signals |= SerialSignal_RTS;
spin_lock_irqsave(&info->lock,flags);
set_signals(info);
@@ -1061,7 +1061,7 @@
if (sanity_check(info, tty->name, "wait_until_sent"))
return;
- if (!test_bit(ASYNCB_INITIALIZED, &info->port.flags))
+ if (!tty_port_initialized(&info->port))
goto exit;
orig_jiffies = jiffies;
@@ -1261,7 +1261,7 @@
if ((cmd != TIOCGSERIAL) && (cmd != TIOCSSERIAL) &&
(cmd != TIOCMIWAIT)) {
- if (tty->flags & (1 << TTY_IO_ERROR))
+ if (tty_io_error(tty))
return -EIO;
}
@@ -2463,7 +2463,7 @@
wake_up_interruptible(&info->status_event_wait_q);
wake_up_interruptible(&info->event_wait_q);
- if ( (info->port.flags & ASYNC_CHECK_CD) &&
+ if (tty_port_check_carrier(&info->port) &&
(status & MISCSTATUS_DCD_LATCHED) ) {
if ( debug_level >= DEBUG_LEVEL_ISR )
printk("%s CD now %s...", info->device_name,
@@ -2636,7 +2636,7 @@
if ( debug_level >= DEBUG_LEVEL_INFO )
printk("%s(%d):%s tx_releaseup()\n",__FILE__,__LINE__,info->device_name);
- if (info->port.flags & ASYNC_INITIALIZED)
+ if (tty_port_initialized(&info->port))
return 0;
if (!info->tx_buf) {
@@ -2662,7 +2662,7 @@
if (info->port.tty)
clear_bit(TTY_IO_ERROR, &info->port.tty->flags);
- info->port.flags |= ASYNC_INITIALIZED;
+ tty_port_set_initialized(&info->port, 1);
return 0;
}
@@ -2673,7 +2673,7 @@
{
unsigned long flags;
- if (!(info->port.flags & ASYNC_INITIALIZED))
+ if (!tty_port_initialized(&info->port))
return;
if (debug_level >= DEBUG_LEVEL_INFO)
@@ -2705,7 +2705,7 @@
if (info->port.tty)
set_bit(TTY_IO_ERROR, &info->port.tty->flags);
- info->port.flags &= ~ASYNC_INITIALIZED;
+ tty_port_set_initialized(&info->port, 0);
}
static void program_hw(SLMP_INFO *info)
@@ -2813,15 +2813,8 @@
}
info->timeout += HZ/50; /* Add .02 seconds of slop */
- if (cflag & CRTSCTS)
- info->port.flags |= ASYNC_CTS_FLOW;
- else
- info->port.flags &= ~ASYNC_CTS_FLOW;
-
- if (cflag & CLOCAL)
- info->port.flags &= ~ASYNC_CHECK_CD;
- else
- info->port.flags |= ASYNC_CHECK_CD;
+ tty_port_set_cts_flow(&info->port, cflag & CRTSCTS);
+ tty_port_set_check_carrier(&info->port, ~cflag & CLOCAL);
/* process tty input control flags */
@@ -3285,10 +3278,10 @@
printk("%s(%d):%s block_til_ready()\n",
__FILE__,__LINE__, tty->driver->name );
- if (filp->f_flags & O_NONBLOCK || tty->flags & (1 << TTY_IO_ERROR)){
+ if (filp->f_flags & O_NONBLOCK || tty_io_error(tty)) {
/* nonblock mode is set or port is not enabled */
/* just verify that callout device is not active */
- port->flags |= ASYNC_NORMAL_ACTIVE;
+ tty_port_set_active(port, 1);
return 0;
}
@@ -3315,12 +3308,12 @@
port->blocked_open++;
while (1) {
- if (C_BAUD(tty) && test_bit(ASYNCB_INITIALIZED, &port->flags))
+ if (C_BAUD(tty) && tty_port_initialized(port))
tty_port_raise_dtr_rts(port);
set_current_state(TASK_INTERRUPTIBLE);
- if (tty_hung_up_p(filp) || !(port->flags & ASYNC_INITIALIZED)){
+ if (tty_hung_up_p(filp) || !tty_port_initialized(port)) {
retval = (port->flags & ASYNC_HUP_NOTIFY) ?
-EAGAIN : -ERESTARTSYS;
break;
@@ -3355,7 +3348,7 @@
__FILE__,__LINE__, tty->driver->name, port->count );
if (!retval)
- port->flags |= ASYNC_NORMAL_ACTIVE;
+ tty_port_set_active(port, 1);
return retval;
}
diff --git a/drivers/tty/tty_buffer.c b/drivers/tty/tty_buffer.c
index a946e49..aa80dc9 100644
--- a/drivers/tty/tty_buffer.c
+++ b/drivers/tty/tty_buffer.c
@@ -37,29 +37,6 @@
#define TTY_BUFFER_PAGE (((PAGE_SIZE - sizeof(struct tty_buffer)) / 2) & ~0xFF)
-/*
- * If all tty flip buffers have been processed by flush_to_ldisc() or
- * dropped by tty_buffer_flush(), check if the linked pty has been closed.
- * If so, wake the reader/poll to process
- */
-static inline void check_other_closed(struct tty_struct *tty)
-{
- unsigned long flags, old;
-
- /* transition from TTY_OTHER_CLOSED => TTY_OTHER_DONE must be atomic */
- for (flags = ACCESS_ONCE(tty->flags);
- test_bit(TTY_OTHER_CLOSED, &flags);
- ) {
- old = flags;
- __set_bit(TTY_OTHER_DONE, &flags);
- flags = cmpxchg(&tty->flags, old, flags);
- if (old == flags) {
- wake_up_interruptible(&tty->read_wait);
- break;
- }
- }
-}
-
/**
* tty_buffer_lock_exclusive - gain exclusive access to buffer
* tty_buffer_unlock_exclusive - release exclusive access
@@ -254,8 +231,6 @@
if (ld && ld->ops->flush_buffer)
ld->ops->flush_buffer(tty);
- check_other_closed(tty);
-
atomic_dec(&buf->priority);
mutex_unlock(&buf->lock);
}
@@ -522,10 +497,8 @@
*/
count = smp_load_acquire(&head->commit) - head->read;
if (!count) {
- if (next == NULL) {
- check_other_closed(tty);
+ if (next == NULL)
break;
- }
buf->head = next;
tty_buffer_free(port, head);
continue;
@@ -614,3 +587,8 @@
{
return cancel_work_sync(&port->buf.work);
}
+
+void tty_buffer_flush_work(struct tty_port *port)
+{
+ flush_work(&port->buf.work);
+}
diff --git a/drivers/tty/tty_io.c b/drivers/tty/tty_io.c
index 24d5491..734a635 100644
--- a/drivers/tty/tty_io.c
+++ b/drivers/tty/tty_io.c
@@ -230,9 +230,6 @@
tty_free_file(file);
}
-
-#define TTY_NUMBER(tty) ((tty)->index + (tty)->driver->name_base)
-
/**
* tty_name - return tty naming
* @tty: tty structure
@@ -1070,7 +1067,7 @@
if (tty_paranoia_check(tty, inode, "tty_read"))
return -EIO;
- if (!tty || (test_bit(TTY_IO_ERROR, &tty->flags)))
+ if (!tty || tty_io_error(tty))
return -EIO;
/* We want to wait for the line discipline to sort out in this
@@ -1245,8 +1242,7 @@
if (tty_paranoia_check(tty, file_inode(file), "tty_write"))
return -EIO;
- if (!tty || !tty->ops->write ||
- (test_bit(TTY_IO_ERROR, &tty->flags)))
+ if (!tty || !tty->ops->write || tty_io_error(tty))
return -EIO;
/* Short term debug to catch buggy drivers */
if (tty->ops->write_room == NULL)
@@ -1964,7 +1960,6 @@
* tty_lookup_driver - lookup a tty driver for a given device file
* @device: device number
* @filp: file pointer to tty
- * @noctty: set if the device should not become a controlling tty
* @index: index for the device in the @return driver
* @return: driver for this inode (with increased refcount)
*
diff --git a/drivers/tty/tty_ioctl.c b/drivers/tty/tty_ioctl.c
index 23bf5bb..bf36ac9 100644
--- a/drivers/tty/tty_ioctl.c
+++ b/drivers/tty/tty_ioctl.c
@@ -158,7 +158,7 @@
int ret = 0;
mutex_lock(&tty->throttle_mutex);
- if (!test_bit(TTY_THROTTLED, &tty->flags)) {
+ if (!tty_throttled(tty)) {
if (tty->flow_change != TTY_THROTTLE_SAFE)
ret = 1;
else {
@@ -189,7 +189,7 @@
int ret = 0;
mutex_lock(&tty->throttle_mutex);
- if (test_bit(TTY_THROTTLED, &tty->flags)) {
+ if (tty_throttled(tty)) {
if (tty->flow_change != TTY_UNTHROTTLE_SAFE)
ret = 1;
else {
diff --git a/drivers/tty/tty_port.c b/drivers/tty/tty_port.c
index dbcca30..c3f9d93 100644
--- a/drivers/tty/tty_port.c
+++ b/drivers/tty/tty_port.c
@@ -204,7 +204,8 @@
if (port->console)
goto out;
- if (test_and_clear_bit(ASYNCB_INITIALIZED, &port->flags)) {
+ if (tty_port_initialized(port)) {
+ tty_port_set_initialized(port, 0);
/*
* Drop DTR/RTS if HUPCL is set. This causes any attached
* modem to hang up the line.
@@ -236,12 +237,12 @@
spin_lock_irqsave(&port->lock, flags);
port->count = 0;
- port->flags &= ~ASYNC_NORMAL_ACTIVE;
tty = port->tty;
if (tty)
set_bit(TTY_IO_ERROR, &tty->flags);
port->tty = NULL;
spin_unlock_irqrestore(&port->lock, flags);
+ tty_port_set_active(port, 0);
tty_port_shutdown(port, tty);
tty_kref_put(tty);
wake_up_interruptible(&port->open_wait);
@@ -364,15 +365,15 @@
/* if non-blocking mode is set we can pass directly to open unless
the port has just hung up or is in another error state */
- if (tty->flags & (1 << TTY_IO_ERROR)) {
- port->flags |= ASYNC_NORMAL_ACTIVE;
+ if (tty_io_error(tty)) {
+ tty_port_set_active(port, 1);
return 0;
}
if (filp->f_flags & O_NONBLOCK) {
/* Indicate we are open */
if (C_BAUD(tty))
tty_port_raise_dtr_rts(port);
- port->flags |= ASYNC_NORMAL_ACTIVE;
+ tty_port_set_active(port, 1);
return 0;
}
@@ -393,13 +394,13 @@
while (1) {
/* Indicate we are open */
- if (C_BAUD(tty) && test_bit(ASYNCB_INITIALIZED, &port->flags))
+ if (C_BAUD(tty) && tty_port_initialized(port))
tty_port_raise_dtr_rts(port);
prepare_to_wait(&port->open_wait, &wait, TASK_INTERRUPTIBLE);
/* Check for a hangup or uninitialised port.
Return accordingly */
- if (tty_hung_up_p(filp) || !(port->flags & ASYNC_INITIALIZED)) {
+ if (tty_hung_up_p(filp) || !tty_port_initialized(port)) {
if (port->flags & ASYNC_HUP_NOTIFY)
retval = -EAGAIN;
else
@@ -430,9 +431,9 @@
if (!tty_hung_up_p(filp))
port->count++;
port->blocked_open--;
- if (retval == 0)
- port->flags |= ASYNC_NORMAL_ACTIVE;
spin_unlock_irqrestore(&port->lock, flags);
+ if (retval == 0)
+ tty_port_set_active(port, 1);
return retval;
}
EXPORT_SYMBOL(tty_port_block_til_ready);
@@ -480,7 +481,7 @@
tty->closing = 1;
- if (test_bit(ASYNCB_INITIALIZED, &port->flags)) {
+ if (tty_port_initialized(port)) {
/* Don't block on a stalled port, just pull the chain */
if (tty->flow_stopped)
tty_driver_flush_buffer(tty);
@@ -514,8 +515,8 @@
spin_lock_irqsave(&port->lock, flags);
wake_up_interruptible(&port->open_wait);
}
- port->flags &= ~ASYNC_NORMAL_ACTIVE;
spin_unlock_irqrestore(&port->lock, flags);
+ tty_port_set_active(port, 0);
}
EXPORT_SYMBOL(tty_port_close_end);
@@ -578,7 +579,7 @@
mutex_lock(&port->mutex);
- if (!test_bit(ASYNCB_INITIALIZED, &port->flags)) {
+ if (!tty_port_initialized(port)) {
clear_bit(TTY_IO_ERROR, &tty->flags);
if (port->ops->activate) {
int retval = port->ops->activate(port, tty);
@@ -587,7 +588,7 @@
return retval;
}
}
- set_bit(ASYNCB_INITIALIZED, &port->flags);
+ tty_port_set_initialized(port, 1);
}
mutex_unlock(&port->mutex);
return tty_port_block_til_ready(port, tty, filp);
diff --git a/drivers/tty/vt/selection.c b/drivers/tty/vt/selection.c
index 4dd9dd2..368ce18 100644
--- a/drivers/tty/vt/selection.c
+++ b/drivers/tty/vt/selection.c
@@ -354,7 +354,7 @@
add_wait_queue(&vc->paste_wait, &wait);
while (sel_buffer && sel_buffer_lth > pasted) {
set_current_state(TASK_INTERRUPTIBLE);
- if (test_bit(TTY_THROTTLED, &tty->flags)) {
+ if (tty_throttled(tty)) {
schedule();
continue;
}
diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 3e3c757..dc12532 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -760,50 +760,54 @@
int vc_allocate(unsigned int currcons) /* return 0 on success */
{
+ struct vt_notifier_param param;
+ struct vc_data *vc;
+
WARN_CONSOLE_UNLOCKED();
if (currcons >= MAX_NR_CONSOLES)
return -ENXIO;
- if (!vc_cons[currcons].d) {
- struct vc_data *vc;
- struct vt_notifier_param param;
- /* prevent users from taking too much memory */
- if (currcons >= MAX_NR_USER_CONSOLES && !capable(CAP_SYS_RESOURCE))
- return -EPERM;
+ if (vc_cons[currcons].d)
+ return 0;
- /* due to the granularity of kmalloc, we waste some memory here */
- /* the alloc is done in two steps, to optimize the common situation
- of a 25x80 console (structsize=216, screenbuf_size=4000) */
- /* although the numbers above are not valid since long ago, the
- point is still up-to-date and the comment still has its value
- even if only as a historical artifact. --mj, July 1998 */
- param.vc = vc = kzalloc(sizeof(struct vc_data), GFP_KERNEL);
- if (!vc)
+ /* due to the granularity of kmalloc, we waste some memory here */
+ /* the alloc is done in two steps, to optimize the common situation
+ of a 25x80 console (structsize=216, screenbuf_size=4000) */
+ /* although the numbers above are not valid since long ago, the
+ point is still up-to-date and the comment still has its value
+ even if only as a historical artifact. --mj, July 1998 */
+ param.vc = vc = kzalloc(sizeof(struct vc_data), GFP_KERNEL);
+ if (!vc)
return -ENOMEM;
- vc_cons[currcons].d = vc;
- tty_port_init(&vc->port);
- INIT_WORK(&vc_cons[currcons].SAK_work, vc_SAK);
- visual_init(vc, currcons, 1);
- if (!*vc->vc_uni_pagedir_loc)
+
+ vc_cons[currcons].d = vc;
+ tty_port_init(&vc->port);
+ INIT_WORK(&vc_cons[currcons].SAK_work, vc_SAK);
+
+ visual_init(vc, currcons, 1);
+
+ if (!*vc->vc_uni_pagedir_loc)
con_set_default_unimap(vc);
- vc->vc_screenbuf = kmalloc(vc->vc_screenbuf_size, GFP_KERNEL);
- if (!vc->vc_screenbuf) {
- kfree(vc);
- vc_cons[currcons].d = NULL;
- return -ENOMEM;
- }
- /* If no drivers have overridden us and the user didn't pass a
- boot option, default to displaying the cursor */
- if (global_cursor_default == -1)
- global_cursor_default = 1;
+ vc->vc_screenbuf = kmalloc(vc->vc_screenbuf_size, GFP_KERNEL);
+ if (!vc->vc_screenbuf)
+ goto err_free;
- vc_init(vc, vc->vc_rows, vc->vc_cols, 1);
- vcs_make_sysfs(currcons);
- atomic_notifier_call_chain(&vt_notifier_list, VT_ALLOCATE, ¶m);
- }
+ /* If no drivers have overridden us and the user didn't pass a
+ boot option, default to displaying the cursor */
+ if (global_cursor_default == -1)
+ global_cursor_default = 1;
+
+ vc_init(vc, vc->vc_rows, vc->vc_cols, 1);
+ vcs_make_sysfs(currcons);
+ atomic_notifier_call_chain(&vt_notifier_list, VT_ALLOCATE, ¶m);
+
return 0;
+err_free:
+ kfree(vc);
+ vc_cons[currcons].d = NULL;
+ return -ENOMEM;
}
static inline int resize_screen(struct vc_data *vc, int width, int height,
@@ -1035,20 +1039,27 @@
#define VT100ID "\033[?1;2c"
#define VT102ID "\033[?6c"
-unsigned char color_table[] = { 0, 4, 2, 6, 1, 5, 3, 7,
+const unsigned char color_table[] = { 0, 4, 2, 6, 1, 5, 3, 7,
8,12,10,14, 9,13,11,15 };
/* the default colour table, for VGA+ colour systems */
-int default_red[] = {0x00,0xaa,0x00,0xaa,0x00,0xaa,0x00,0xaa,
- 0x55,0xff,0x55,0xff,0x55,0xff,0x55,0xff};
-int default_grn[] = {0x00,0x00,0xaa,0x55,0x00,0x00,0xaa,0xaa,
- 0x55,0x55,0xff,0xff,0x55,0x55,0xff,0xff};
-int default_blu[] = {0x00,0x00,0x00,0x00,0xaa,0xaa,0xaa,0xaa,
- 0x55,0x55,0x55,0x55,0xff,0xff,0xff,0xff};
+unsigned char default_red[] = {
+ 0x00, 0xaa, 0x00, 0xaa, 0x00, 0xaa, 0x00, 0xaa,
+ 0x55, 0xff, 0x55, 0xff, 0x55, 0xff, 0x55, 0xff
+};
+module_param_array(default_red, byte, NULL, S_IRUGO | S_IWUSR);
-module_param_array(default_red, int, NULL, S_IRUGO | S_IWUSR);
-module_param_array(default_grn, int, NULL, S_IRUGO | S_IWUSR);
-module_param_array(default_blu, int, NULL, S_IRUGO | S_IWUSR);
+unsigned char default_grn[] = {
+ 0x00, 0x00, 0xaa, 0x55, 0x00, 0x00, 0xaa, 0xaa,
+ 0x55, 0x55, 0xff, 0xff, 0x55, 0x55, 0xff, 0xff
+};
+module_param_array(default_grn, byte, NULL, S_IRUGO | S_IWUSR);
+
+unsigned char default_blu[] = {
+ 0x00, 0x00, 0x00, 0x00, 0xaa, 0xaa, 0xaa, 0xaa,
+ 0x55, 0x55, 0x55, 0x55, 0xff, 0xff, 0xff, 0xff
+};
+module_param_array(default_blu, byte, NULL, S_IRUGO | S_IWUSR);
/*
* gotoxy() must verify all boundaries, because the arguments
@@ -3564,7 +3575,7 @@
struct module *owner = csw->owner;
struct con_driver *con_driver;
const char *desc;
- int i, retval = 0;
+ int i, retval;
WARN_CONSOLE_UNLOCKED();
@@ -3575,17 +3586,17 @@
con_driver = ®istered_con_driver[i];
/* already registered */
- if (con_driver->con == csw)
+ if (con_driver->con == csw) {
retval = -EBUSY;
+ goto err;
+ }
}
- if (retval)
- goto err;
-
desc = csw->con_startup();
-
- if (!desc)
+ if (!desc) {
+ retval = -ENODEV;
goto err;
+ }
retval = -EINVAL;
diff --git a/drivers/uio/uio.c b/drivers/uio/uio.c
index bcc1fc0..fba021f 100644
--- a/drivers/uio/uio.c
+++ b/drivers/uio/uio.c
@@ -271,12 +271,16 @@
map_found = 1;
idev->map_dir = kobject_create_and_add("maps",
&idev->dev->kobj);
- if (!idev->map_dir)
+ if (!idev->map_dir) {
+ ret = -ENOMEM;
goto err_map;
+ }
}
map = kzalloc(sizeof(*map), GFP_KERNEL);
- if (!map)
+ if (!map) {
+ ret = -ENOMEM;
goto err_map_kobj;
+ }
kobject_init(&map->kobj, &map_attr_type);
map->mem = mem;
mem->map = map;
@@ -296,12 +300,16 @@
portio_found = 1;
idev->portio_dir = kobject_create_and_add("portio",
&idev->dev->kobj);
- if (!idev->portio_dir)
+ if (!idev->portio_dir) {
+ ret = -ENOMEM;
goto err_portio;
+ }
}
portio = kzalloc(sizeof(*portio), GFP_KERNEL);
- if (!portio)
+ if (!portio) {
+ ret = -ENOMEM;
goto err_portio_kobj;
+ }
kobject_init(&portio->kobj, &portio_attr_type);
portio->port = port;
port->portio = portio;
diff --git a/drivers/usb/Kconfig b/drivers/usb/Kconfig
index 8ed451d..8689dcb 100644
--- a/drivers/usb/Kconfig
+++ b/drivers/usb/Kconfig
@@ -31,8 +31,6 @@
config USB_COMMON
tristate
- default y
- depends on USB || USB_GADGET
config USB_ARCH_HAS_HCD
def_bool y
@@ -41,6 +39,7 @@
config USB
tristate "Support for Host-side USB"
depends on USB_ARCH_HAS_HCD
+ select USB_COMMON
select NLS # for UTF-8 strings
---help---
Universal Serial Bus (USB) is a specification for a serial bus
diff --git a/drivers/usb/atm/ueagle-atm.c b/drivers/usb/atm/ueagle-atm.c
index a2ae88d..4333dc5 100644
--- a/drivers/usb/atm/ueagle-atm.c
+++ b/drivers/usb/atm/ueagle-atm.c
@@ -173,10 +173,10 @@
const struct firmware *dsp_firm;
struct urb *urb_int;
- void (*dispatch_cmv) (struct uea_softc *, struct intr_pkt *);
- void (*schedule_load_page) (struct uea_softc *, struct intr_pkt *);
- int (*stat) (struct uea_softc *);
- int (*send_cmvs) (struct uea_softc *);
+ void (*dispatch_cmv)(struct uea_softc *, struct intr_pkt *);
+ void (*schedule_load_page)(struct uea_softc *, struct intr_pkt *);
+ int (*stat)(struct uea_softc *);
+ int (*send_cmvs)(struct uea_softc *);
/* keep in sync with eaglectl */
struct uea_stats {
@@ -2454,7 +2454,7 @@
/* Retrieve the device End System Identifier (MAC) */
-static int uea_getesi(struct uea_softc *sc, u_char * esi)
+static int uea_getesi(struct uea_softc *sc, u_char *esi)
{
unsigned char mac_str[2 * ETH_ALEN + 1];
int i;
diff --git a/drivers/usb/chipidea/ci_hdrc_imx.c b/drivers/usb/chipidea/ci_hdrc_imx.c
index 9ce8c9f..dedc33e5 100644
--- a/drivers/usb/chipidea/ci_hdrc_imx.c
+++ b/drivers/usb/chipidea/ci_hdrc_imx.c
@@ -292,10 +292,6 @@
if (pdata.flags & CI_HDRC_SUPPORTS_RUNTIME_PM)
data->supports_runtime_pm = true;
- ret = dma_coerce_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
- if (ret)
- goto err_clk;
-
ret = imx_usbmisc_init(data->usbmisc_data);
if (ret) {
dev_err(&pdev->dev, "usbmisc init failed, ret=%d\n", ret);
diff --git a/drivers/usb/class/cdc-acm.c b/drivers/usb/class/cdc-acm.c
index a6c4a1b..94a14f5 100644
--- a/drivers/usb/class/cdc-acm.c
+++ b/drivers/usb/class/cdc-acm.c
@@ -1680,7 +1680,7 @@
if (--acm->susp_count)
goto out;
- if (test_bit(ASYNCB_INITIALIZED, &acm->port.flags)) {
+ if (tty_port_initialized(&acm->port)) {
rv = usb_submit_urb(acm->ctrlurb, GFP_ATOMIC);
for (;;) {
@@ -1710,7 +1710,7 @@
{
struct acm *acm = usb_get_intfdata(intf);
- if (test_bit(ASYNCB_INITIALIZED, &acm->port.flags))
+ if (tty_port_initialized(&acm->port))
tty_port_tty_hangup(&acm->port, false);
return acm_resume(intf);
diff --git a/drivers/usb/common/usb-otg-fsm.c b/drivers/usb/common/usb-otg-fsm.c
index 504708f..9059b7d 100644
--- a/drivers/usb/common/usb-otg-fsm.c
+++ b/drivers/usb/common/usb-otg-fsm.c
@@ -61,8 +61,6 @@
return 0;
}
-static int state_changed;
-
/* Called when leaving a state. Do state clean up jobs here */
static void otg_leave_state(struct otg_fsm *fsm, enum usb_otg_state old_state)
{
@@ -208,7 +206,6 @@
/* Called when entering a state */
static int otg_set_state(struct otg_fsm *fsm, enum usb_otg_state new_state)
{
- state_changed = 1;
if (fsm->otg->state == new_state)
return 0;
VDBG("Set state: %s\n", usb_otg_state_string(new_state));
@@ -324,6 +321,7 @@
}
fsm->otg->state = new_state;
+ fsm->state_changed = 1;
return 0;
}
@@ -335,7 +333,7 @@
mutex_lock(&fsm->lock);
state = fsm->otg->state;
- state_changed = 0;
+ fsm->state_changed = 0;
/* State machine state change judgement */
switch (state) {
@@ -448,7 +446,7 @@
}
mutex_unlock(&fsm->lock);
- VDBG("quit statemachine, changed = %d\n", state_changed);
- return state_changed;
+ VDBG("quit statemachine, changed = %d\n", fsm->state_changed);
+ return fsm->state_changed;
}
EXPORT_SYMBOL_GPL(otg_statemachine);
diff --git a/drivers/usb/core/buffer.c b/drivers/usb/core/buffer.c
index 2741566..98e39f9 100644
--- a/drivers/usb/core/buffer.c
+++ b/drivers/usb/core/buffer.c
@@ -122,6 +122,9 @@
struct usb_hcd *hcd = bus_to_hcd(bus);
int i;
+ if (size == 0)
+ return NULL;
+
/* some USB hosts just use PIO */
if (!IS_ENABLED(CONFIG_HAS_DMA) ||
(!bus->controller->dma_mask &&
diff --git a/drivers/usb/core/devio.c b/drivers/usb/core/devio.c
index 52c4461..e9f5043 100644
--- a/drivers/usb/core/devio.c
+++ b/drivers/usb/core/devio.c
@@ -216,7 +216,7 @@
dec_usb_memory_use_count(usbm, &usbm->vma_use_count);
}
-struct vm_operations_struct usbdev_vm_ops = {
+static struct vm_operations_struct usbdev_vm_ops = {
.open = usbdev_vm_open,
.close = usbdev_vm_close
};
@@ -1316,10 +1316,11 @@
static int proc_connectinfo(struct usb_dev_state *ps, void __user *arg)
{
- struct usbdevfs_connectinfo ci = {
- .devnum = ps->dev->devnum,
- .slow = ps->dev->speed == USB_SPEED_LOW
- };
+ struct usbdevfs_connectinfo ci;
+
+ memset(&ci, 0, sizeof(ci));
+ ci.devnum = ps->dev->devnum;
+ ci.slow = ps->dev->speed == USB_SPEED_LOW;
if (copy_to_user(arg, &ci, sizeof(ci)))
return -EFAULT;
diff --git a/drivers/usb/core/driver.c b/drivers/usb/core/driver.c
index 2057d91..dadd1e8d 100644
--- a/drivers/usb/core/driver.c
+++ b/drivers/usb/core/driver.c
@@ -284,7 +284,7 @@
struct usb_device *udev = interface_to_usbdev(intf);
const struct usb_device_id *id;
int error = -ENODEV;
- int lpm_disable_error;
+ int lpm_disable_error = -ENODEV;
dev_dbg(dev, "%s\n", __func__);
@@ -336,12 +336,14 @@
* setting during probe, that should also be fine. usb_set_interface()
* will attempt to disable LPM, and fail if it can't disable it.
*/
- lpm_disable_error = usb_unlocked_disable_lpm(udev);
- if (lpm_disable_error && driver->disable_hub_initiated_lpm) {
- dev_err(&intf->dev, "%s Failed to disable LPM for driver %s\n.",
- __func__, driver->name);
- error = lpm_disable_error;
- goto err;
+ if (driver->disable_hub_initiated_lpm) {
+ lpm_disable_error = usb_unlocked_disable_lpm(udev);
+ if (lpm_disable_error) {
+ dev_err(&intf->dev, "%s Failed to disable LPM for driver %s\n.",
+ __func__, driver->name);
+ error = lpm_disable_error;
+ goto err;
+ }
}
/* Carry out a deferred switch to altsetting 0 */
@@ -391,7 +393,8 @@
struct usb_interface *intf = to_usb_interface(dev);
struct usb_host_endpoint *ep, **eps = NULL;
struct usb_device *udev;
- int i, j, error, r, lpm_disable_error;
+ int i, j, error, r;
+ int lpm_disable_error = -ENODEV;
intf->condition = USB_INTERFACE_UNBINDING;
@@ -399,12 +402,13 @@
udev = interface_to_usbdev(intf);
error = usb_autoresume_device(udev);
- /* Hub-initiated LPM policy may change, so attempt to disable LPM until
+ /* If hub-initiated LPM policy may change, attempt to disable LPM until
* the driver is unbound. If LPM isn't disabled, that's fine because it
* wouldn't be enabled unless all the bound interfaces supported
* hub-initiated LPM.
*/
- lpm_disable_error = usb_unlocked_disable_lpm(udev);
+ if (driver->disable_hub_initiated_lpm)
+ lpm_disable_error = usb_unlocked_disable_lpm(udev);
/*
* Terminate all URBs for this interface unless the driver
@@ -505,7 +509,7 @@
struct device *dev;
struct usb_device *udev;
int retval = 0;
- int lpm_disable_error;
+ int lpm_disable_error = -ENODEV;
if (!iface)
return -ENODEV;
@@ -526,12 +530,14 @@
iface->condition = USB_INTERFACE_BOUND;
- /* Disable LPM until this driver is bound. */
- lpm_disable_error = usb_unlocked_disable_lpm(udev);
- if (lpm_disable_error && driver->disable_hub_initiated_lpm) {
- dev_err(&iface->dev, "%s Failed to disable LPM for driver %s\n.",
- __func__, driver->name);
- return -ENOMEM;
+ /* See the comment about disabling LPM in usb_probe_interface(). */
+ if (driver->disable_hub_initiated_lpm) {
+ lpm_disable_error = usb_unlocked_disable_lpm(udev);
+ if (lpm_disable_error) {
+ dev_err(&iface->dev, "%s Failed to disable LPM for driver %s\n.",
+ __func__, driver->name);
+ return -ENOMEM;
+ }
}
/* Claimed interfaces are initially inactive (suspended) and
diff --git a/drivers/usb/core/hcd.c b/drivers/usb/core/hcd.c
index 2ca2cef..34b837a 100644
--- a/drivers/usb/core/hcd.c
+++ b/drivers/usb/core/hcd.c
@@ -994,7 +994,7 @@
bus->bandwidth_allocated = 0;
bus->bandwidth_int_reqs = 0;
bus->bandwidth_isoc_reqs = 0;
- mutex_init(&bus->usb_address0_mutex);
+ mutex_init(&bus->devnum_next_mutex);
}
/*-------------------------------------------------------------------------*/
@@ -1118,6 +1118,7 @@
/* Did the HC die before the root hub was registered? */
if (HCD_DEAD(hcd))
usb_hc_died (hcd); /* This time clean up */
+ usb_dev->dev.of_node = parent_dev->of_node;
}
mutex_unlock(&usb_bus_idr_lock);
@@ -2521,6 +2522,14 @@
return NULL;
}
if (primary_hcd == NULL) {
+ hcd->address0_mutex = kmalloc(sizeof(*hcd->address0_mutex),
+ GFP_KERNEL);
+ if (!hcd->address0_mutex) {
+ kfree(hcd);
+ dev_dbg(dev, "hcd address0 mutex alloc failed\n");
+ return NULL;
+ }
+ mutex_init(hcd->address0_mutex);
hcd->bandwidth_mutex = kmalloc(sizeof(*hcd->bandwidth_mutex),
GFP_KERNEL);
if (!hcd->bandwidth_mutex) {
@@ -2532,6 +2541,7 @@
dev_set_drvdata(dev, hcd);
} else {
mutex_lock(&usb_port_peer_mutex);
+ hcd->address0_mutex = primary_hcd->address0_mutex;
hcd->bandwidth_mutex = primary_hcd->bandwidth_mutex;
hcd->primary_hcd = primary_hcd;
primary_hcd->primary_hcd = primary_hcd;
@@ -2598,8 +2608,10 @@
struct usb_hcd *hcd = container_of (kref, struct usb_hcd, kref);
mutex_lock(&usb_port_peer_mutex);
- if (usb_hcd_is_primary_hcd(hcd))
+ if (usb_hcd_is_primary_hcd(hcd)) {
+ kfree(hcd->address0_mutex);
kfree(hcd->bandwidth_mutex);
+ }
if (hcd->shared_hcd) {
struct usb_hcd *peer = hcd->shared_hcd;
diff --git a/drivers/usb/core/hub.c b/drivers/usb/core/hub.c
index 38cc4ba..bee1351 100644
--- a/drivers/usb/core/hub.c
+++ b/drivers/usb/core/hub.c
@@ -104,6 +104,8 @@
static inline char *portspeed(struct usb_hub *hub, int portstatus)
{
+ if (hub_is_superspeedplus(hub->hdev))
+ return "10.0 Gb/s";
if (hub_is_superspeed(hub->hdev))
return "5.0 Gb/s";
if (portstatus & USB_PORT_STAT_HIGH_SPEED)
@@ -2080,7 +2082,7 @@
struct usb_bus *bus = udev->bus;
/* be safe when more hub events are proceed in parallel */
- mutex_lock(&bus->usb_address0_mutex);
+ mutex_lock(&bus->devnum_next_mutex);
if (udev->wusb) {
devnum = udev->portnum + 1;
BUG_ON(test_bit(devnum, bus->devmap.devicemap));
@@ -2098,7 +2100,7 @@
set_bit(devnum, bus->devmap.devicemap);
udev->devnum = devnum;
}
- mutex_unlock(&bus->usb_address0_mutex);
+ mutex_unlock(&bus->devnum_next_mutex);
}
static void release_devnum(struct usb_device *udev)
@@ -4364,7 +4366,7 @@
if (oldspeed == USB_SPEED_LOW)
delay = HUB_LONG_RESET_TIME;
- mutex_lock(&hdev->bus->usb_address0_mutex);
+ mutex_lock(hcd->address0_mutex);
/* Reset the device; full speed may morph to high speed */
/* FIXME a USB 2.0 device may morph into SuperSpeed on reset. */
@@ -4650,7 +4652,7 @@
hub_port_disable(hub, port1, 0);
update_devnum(udev, devnum); /* for disconnect processing */
}
- mutex_unlock(&hdev->bus->usb_address0_mutex);
+ mutex_unlock(hcd->address0_mutex);
return retval;
}
diff --git a/drivers/usb/core/message.c b/drivers/usb/core/message.c
index 8e641b5..ea681f1 100644
--- a/drivers/usb/core/message.c
+++ b/drivers/usb/core/message.c
@@ -302,9 +302,10 @@
*/
spin_unlock(&io->lock);
for (i = 0, found = 0; i < io->entries; i++) {
- if (!io->urbs[i] || !io->urbs[i]->dev)
+ if (!io->urbs[i])
continue;
if (found) {
+ usb_block_urb(io->urbs[i]);
retval = usb_unlink_urb(io->urbs[i]);
if (retval != -EINPROGRESS &&
retval != -ENODEV &&
@@ -515,12 +516,10 @@
int retval;
io->urbs[i]->dev = io->dev;
- retval = usb_submit_urb(io->urbs[i], GFP_ATOMIC);
-
- /* after we submit, let completions or cancellations fire;
- * we handshake using io->status.
- */
spin_unlock_irq(&io->lock);
+
+ retval = usb_submit_urb(io->urbs[i], GFP_NOIO);
+
switch (retval) {
/* maybe we retrying will recover */
case -ENXIO: /* hc didn't queue this one */
@@ -578,31 +577,28 @@
void usb_sg_cancel(struct usb_sg_request *io)
{
unsigned long flags;
+ int i, retval;
spin_lock_irqsave(&io->lock, flags);
-
- /* shut everything down, if it didn't already */
- if (!io->status) {
- int i;
-
- io->status = -ECONNRESET;
- spin_unlock(&io->lock);
- for (i = 0; i < io->entries; i++) {
- int retval;
-
- if (!io->urbs[i]->dev)
- continue;
- retval = usb_unlink_urb(io->urbs[i]);
- if (retval != -EINPROGRESS
- && retval != -ENODEV
- && retval != -EBUSY
- && retval != -EIDRM)
- dev_warn(&io->dev->dev, "%s, unlink --> %d\n",
- __func__, retval);
- }
- spin_lock(&io->lock);
+ if (io->status) {
+ spin_unlock_irqrestore(&io->lock, flags);
+ return;
}
+ /* shut everything down */
+ io->status = -ECONNRESET;
spin_unlock_irqrestore(&io->lock, flags);
+
+ for (i = io->entries - 1; i >= 0; --i) {
+ usb_block_urb(io->urbs[i]);
+
+ retval = usb_unlink_urb(io->urbs[i]);
+ if (retval != -EINPROGRESS
+ && retval != -ENODEV
+ && retval != -EBUSY
+ && retval != -EIDRM)
+ dev_warn(&io->dev->dev, "%s, unlink --> %d\n",
+ __func__, retval);
+ }
}
EXPORT_SYMBOL_GPL(usb_sg_cancel);
diff --git a/drivers/usb/core/usb.c b/drivers/usb/core/usb.c
index 479187c3..5e80697 100644
--- a/drivers/usb/core/usb.c
+++ b/drivers/usb/core/usb.c
@@ -466,7 +466,6 @@
dev->route = 0;
dev->dev.parent = bus->controller;
- dev->dev.of_node = bus->controller->of_node;
dev_set_name(&dev->dev, "usb%d", bus->busnum);
root_hub = 1;
} else {
diff --git a/drivers/usb/dwc2/gadget.c b/drivers/usb/dwc2/gadget.c
index 818f158..4c5e300 100644
--- a/drivers/usb/dwc2/gadget.c
+++ b/drivers/usb/dwc2/gadget.c
@@ -2425,6 +2425,9 @@
u32 gintsts;
u32 gintmsk;
+ if (!dwc2_is_device_mode(hsotg))
+ return IRQ_NONE;
+
spin_lock(&hsotg->lock);
irq_retry:
gintsts = dwc2_readl(hsotg->regs + GINTSTS);
@@ -2631,7 +2634,10 @@
desc->wMaxPacketSize, desc->bInterval);
/* not to be called for EP0 */
- WARN_ON(index == 0);
+ if (index == 0) {
+ dev_err(hsotg->dev, "%s: called for EP 0\n", __func__);
+ return -EINVAL;
+ }
dir_in = (desc->bEndpointAddress & USB_ENDPOINT_DIR_MASK) ? 1 : 0;
if (dir_in != hs_ep->dir_in) {
diff --git a/drivers/usb/dwc2/hcd.c b/drivers/usb/dwc2/hcd.c
index 1f62551..2df3d04 100644
--- a/drivers/usb/dwc2/hcd.c
+++ b/drivers/usb/dwc2/hcd.c
@@ -4703,6 +4703,7 @@
spin_unlock_irqrestore(&hsotg->lock, flags);
urb->hcpriv = NULL;
kfree(qtd);
+ qtd = NULL;
fail1:
if (qh_allocated) {
struct dwc2_qtd *qtd2, *qtd2_tmp;
diff --git a/drivers/usb/dwc2/hcd.h b/drivers/usb/dwc2/hcd.h
index 89fa26c..7758bfb 100644
--- a/drivers/usb/dwc2/hcd.h
+++ b/drivers/usb/dwc2/hcd.h
@@ -552,6 +552,7 @@
{
list_del(&qtd->qtd_list_entry);
kfree(qtd);
+ qtd = NULL;
}
/* Descriptor DMA support functions */
diff --git a/drivers/usb/dwc2/hcd_queue.c b/drivers/usb/dwc2/hcd_queue.c
index 7f634fd..b5c7793 100644
--- a/drivers/usb/dwc2/hcd_queue.c
+++ b/drivers/usb/dwc2/hcd_queue.c
@@ -1709,7 +1709,8 @@
dwc2_deschedule_periodic(hsotg, qh);
hsotg->periodic_qh_count--;
- if (!hsotg->periodic_qh_count) {
+ if (!hsotg->periodic_qh_count &&
+ hsotg->core_params->dma_desc_enable <= 0) {
intr_mask = dwc2_readl(hsotg->regs + GINTMSK);
intr_mask &= ~GINTSTS_SOF;
dwc2_writel(intr_mask, hsotg->regs + GINTMSK);
diff --git a/drivers/usb/dwc2/platform.c b/drivers/usb/dwc2/platform.c
index 88629be..fc6f525 100644
--- a/drivers/usb/dwc2/platform.c
+++ b/drivers/usb/dwc2/platform.c
@@ -562,7 +562,7 @@
retval = dwc2_get_dr_mode(hsotg);
if (retval)
- return retval;
+ goto error;
/*
* Reset before dwc2_get_hwparams() then it could get power-on real
diff --git a/drivers/usb/dwc3/core.c b/drivers/usb/dwc3/core.c
index 34277ce..a590cd2 100644
--- a/drivers/usb/dwc3/core.c
+++ b/drivers/usb/dwc3/core.c
@@ -60,6 +60,20 @@
dwc3_writel(dwc->regs, DWC3_GCTL, reg);
}
+u32 dwc3_core_fifo_space(struct dwc3_ep *dep, u8 type)
+{
+ struct dwc3 *dwc = dep->dwc;
+ u32 reg;
+
+ dwc3_writel(dwc->regs, DWC3_GDBGFIFOSPACE,
+ DWC3_GDBGFIFOSPACE_NUM(dep->number) |
+ DWC3_GDBGFIFOSPACE_TYPE(type));
+
+ reg = dwc3_readl(dwc->regs, DWC3_GDBGFIFOSPACE);
+
+ return DWC3_GDBGFIFOSPACE_SPACE_AVAILABLE(reg);
+}
+
/**
* dwc3_core_soft_reset - Issues core soft reset and PHY reset
* @dwc: pointer to our context structure
@@ -203,13 +217,10 @@
static void dwc3_free_event_buffers(struct dwc3 *dwc)
{
struct dwc3_event_buffer *evt;
- int i;
- for (i = 0; i < dwc->num_event_buffers; i++) {
- evt = dwc->ev_buffs[i];
- if (evt)
- dwc3_free_one_event_buffer(dwc, evt);
- }
+ evt = dwc->ev_buf;
+ if (evt)
+ dwc3_free_one_event_buffer(dwc, evt);
}
/**
@@ -222,27 +233,14 @@
*/
static int dwc3_alloc_event_buffers(struct dwc3 *dwc, unsigned length)
{
- int num;
- int i;
+ struct dwc3_event_buffer *evt;
- num = DWC3_NUM_INT(dwc->hwparams.hwparams1);
- dwc->num_event_buffers = num;
-
- dwc->ev_buffs = devm_kzalloc(dwc->dev, sizeof(*dwc->ev_buffs) * num,
- GFP_KERNEL);
- if (!dwc->ev_buffs)
- return -ENOMEM;
-
- for (i = 0; i < num; i++) {
- struct dwc3_event_buffer *evt;
-
- evt = dwc3_alloc_one_event_buffer(dwc, length);
- if (IS_ERR(evt)) {
- dev_err(dwc->dev, "can't allocate event buffer\n");
- return PTR_ERR(evt);
- }
- dwc->ev_buffs[i] = evt;
+ evt = dwc3_alloc_one_event_buffer(dwc, length);
+ if (IS_ERR(evt)) {
+ dev_err(dwc->dev, "can't allocate event buffer\n");
+ return PTR_ERR(evt);
}
+ dwc->ev_buf = evt;
return 0;
}
@@ -256,25 +254,22 @@
static int dwc3_event_buffers_setup(struct dwc3 *dwc)
{
struct dwc3_event_buffer *evt;
- int n;
- for (n = 0; n < dwc->num_event_buffers; n++) {
- evt = dwc->ev_buffs[n];
- dwc3_trace(trace_dwc3_core,
- "Event buf %p dma %08llx length %d\n",
- evt->buf, (unsigned long long) evt->dma,
- evt->length);
+ evt = dwc->ev_buf;
+ dwc3_trace(trace_dwc3_core,
+ "Event buf %p dma %08llx length %d\n",
+ evt->buf, (unsigned long long) evt->dma,
+ evt->length);
- evt->lpos = 0;
+ evt->lpos = 0;
- dwc3_writel(dwc->regs, DWC3_GEVNTADRLO(n),
- lower_32_bits(evt->dma));
- dwc3_writel(dwc->regs, DWC3_GEVNTADRHI(n),
- upper_32_bits(evt->dma));
- dwc3_writel(dwc->regs, DWC3_GEVNTSIZ(n),
- DWC3_GEVNTSIZ_SIZE(evt->length));
- dwc3_writel(dwc->regs, DWC3_GEVNTCOUNT(n), 0);
- }
+ dwc3_writel(dwc->regs, DWC3_GEVNTADRLO(0),
+ lower_32_bits(evt->dma));
+ dwc3_writel(dwc->regs, DWC3_GEVNTADRHI(0),
+ upper_32_bits(evt->dma));
+ dwc3_writel(dwc->regs, DWC3_GEVNTSIZ(0),
+ DWC3_GEVNTSIZ_SIZE(evt->length));
+ dwc3_writel(dwc->regs, DWC3_GEVNTCOUNT(0), 0);
return 0;
}
@@ -282,19 +277,16 @@
static void dwc3_event_buffers_cleanup(struct dwc3 *dwc)
{
struct dwc3_event_buffer *evt;
- int n;
- for (n = 0; n < dwc->num_event_buffers; n++) {
- evt = dwc->ev_buffs[n];
+ evt = dwc->ev_buf;
- evt->lpos = 0;
+ evt->lpos = 0;
- dwc3_writel(dwc->regs, DWC3_GEVNTADRLO(n), 0);
- dwc3_writel(dwc->regs, DWC3_GEVNTADRHI(n), 0);
- dwc3_writel(dwc->regs, DWC3_GEVNTSIZ(n), DWC3_GEVNTSIZ_INTMASK
- | DWC3_GEVNTSIZ_SIZE(0));
- dwc3_writel(dwc->regs, DWC3_GEVNTCOUNT(n), 0);
- }
+ dwc3_writel(dwc->regs, DWC3_GEVNTADRLO(0), 0);
+ dwc3_writel(dwc->regs, DWC3_GEVNTADRHI(0), 0);
+ dwc3_writel(dwc->regs, DWC3_GEVNTSIZ(0), DWC3_GEVNTSIZ_INTMASK
+ | DWC3_GEVNTSIZ_SIZE(0));
+ dwc3_writel(dwc->regs, DWC3_GEVNTCOUNT(0), 0);
}
static int dwc3_alloc_scratch_buffers(struct dwc3 *dwc)
@@ -434,6 +426,9 @@
if (dwc->u2ss_inp3_quirk)
reg |= DWC3_GUSB3PIPECTL_U2SSINP3OK;
+ if (dwc->dis_rxdet_inp3_quirk)
+ reg |= DWC3_GUSB3PIPECTL_DISRXDETINP3;
+
if (dwc->req_p1p2p3_quirk)
reg |= DWC3_GUSB3PIPECTL_REQP1P2P3;
@@ -882,9 +877,6 @@
dwc->usb3_lpm_capable = device_property_read_bool(dev,
"snps,usb3_lpm_capable");
- dwc->needs_fifo_resize = device_property_read_bool(dev,
- "tx-fifo-resize");
-
dwc->disable_scramble_quirk = device_property_read_bool(dev,
"snps,disable_scramble_quirk");
dwc->u2exit_lfps_quirk = device_property_read_bool(dev,
@@ -907,6 +899,8 @@
"snps,dis_u2_susphy_quirk");
dwc->dis_enblslpm_quirk = device_property_read_bool(dev,
"snps,dis_enblslpm_quirk");
+ dwc->dis_rxdet_inp3_quirk = device_property_read_bool(dev,
+ "snps,dis_rxdet_inp3_quirk");
dwc->tx_de_emphasis_quirk = device_property_read_bool(dev,
"snps,tx_de_emphasis_quirk");
@@ -926,7 +920,6 @@
if (pdata->hird_threshold)
hird_threshold = pdata->hird_threshold;
- dwc->needs_fifo_resize = pdata->tx_fifo_resize;
dwc->usb3_lpm_capable = pdata->usb3_lpm_capable;
dwc->dr_mode = pdata->dr_mode;
@@ -941,6 +934,7 @@
dwc->dis_u3_susphy_quirk = pdata->dis_u3_susphy_quirk;
dwc->dis_u2_susphy_quirk = pdata->dis_u2_susphy_quirk;
dwc->dis_enblslpm_quirk = pdata->dis_enblslpm_quirk;
+ dwc->dis_rxdet_inp3_quirk = pdata->dis_rxdet_inp3_quirk;
dwc->tx_de_emphasis_quirk = pdata->tx_de_emphasis_quirk;
if (pdata->tx_de_emphasis)
@@ -1050,19 +1044,11 @@
if (ret)
goto err5;
- ret = dwc3_debugfs_init(dwc);
- if (ret) {
- dev_err(dev, "failed to initialize debugfs\n");
- goto err6;
- }
-
+ dwc3_debugfs_init(dwc);
pm_runtime_allow(dev);
return 0;
-err6:
- dwc3_core_exit_mode(dwc);
-
err5:
dwc3_event_buffers_cleanup(dwc);
diff --git a/drivers/usb/dwc3/core.h b/drivers/usb/dwc3/core.h
index 6254b2f..7ddf944 100644
--- a/drivers/usb/dwc3/core.h
+++ b/drivers/usb/dwc3/core.h
@@ -152,6 +152,24 @@
/* Bit fields */
+/* Global Debug Queue/FIFO Space Available Register */
+#define DWC3_GDBGFIFOSPACE_NUM(n) ((n) & 0x1f)
+#define DWC3_GDBGFIFOSPACE_TYPE(n) (((n) << 5) & 0x1e0)
+#define DWC3_GDBGFIFOSPACE_SPACE_AVAILABLE(n) (((n) >> 16) & 0xffff)
+
+#define DWC3_TXFIFOQ 1
+#define DWC3_RXFIFOQ 3
+#define DWC3_TXREQQ 5
+#define DWC3_RXREQQ 7
+#define DWC3_RXINFOQ 9
+#define DWC3_DESCFETCHQ 13
+#define DWC3_EVENTQ 15
+
+/* Global RX Threshold Configuration Register */
+#define DWC3_GRXTHRCFG_MAXRXBURSTSIZE(n) (((n) & 0x1f) << 19)
+#define DWC3_GRXTHRCFG_RXPKTCNT(n) (((n) & 0xf) << 24)
+#define DWC3_GRXTHRCFG_PKTCNTSEL (1 << 29)
+
/* Global Configuration Register */
#define DWC3_GCTL_PWRDNSCALE(n) ((n) << 19)
#define DWC3_GCTL_U2RSTECN (1 << 16)
@@ -193,6 +211,7 @@
/* Global USB3 PIPE Control Register */
#define DWC3_GUSB3PIPECTL_PHYSOFTRST (1 << 31)
#define DWC3_GUSB3PIPECTL_U2SSINP3OK (1 << 29)
+#define DWC3_GUSB3PIPECTL_DISRXDETINP3 (1 << 28)
#define DWC3_GUSB3PIPECTL_REQP1P2P3 (1 << 24)
#define DWC3_GUSB3PIPECTL_DEP1P2P3(n) ((n) << 19)
#define DWC3_GUSB3PIPECTL_DEP1P2P3_MASK DWC3_GUSB3PIPECTL_DEP1P2P3(7)
@@ -257,6 +276,9 @@
#define DWC3_DCFG_LOWSPEED (2 << 0)
#define DWC3_DCFG_FULLSPEED1 (3 << 0)
+#define DWC3_DCFG_NUMP_SHIFT 17
+#define DWC3_DCFG_NUMP(n) (((n) >> DWC3_DCFG_NUMP_SHIFT) & 0x1f)
+#define DWC3_DCFG_NUMP_MASK (0x1f << DWC3_DCFG_NUMP_SHIFT)
#define DWC3_DCFG_LPM_CAP (1 << 22)
/* Device Control Register */
@@ -438,18 +460,17 @@
#define DWC3_EP_DIRECTION_TX true
#define DWC3_EP_DIRECTION_RX false
-#define DWC3_TRB_NUM 32
-#define DWC3_TRB_MASK (DWC3_TRB_NUM - 1)
+#define DWC3_TRB_NUM 256
/**
* struct dwc3_ep - device side endpoint representation
* @endpoint: usb endpoint
- * @request_list: list of requests for this endpoint
- * @req_queued: list of requests on this ep which have TRBs setup
+ * @pending_list: list of pending requests for this endpoint
+ * @started_list: list of started requests on this endpoint
* @trb_pool: array of transaction buffers
* @trb_pool_dma: dma address of @trb_pool
- * @free_slot: next slot which is going to be used
- * @busy_slot: first slot which is owned by HW
+ * @trb_enqueue: enqueue 'pointer' into TRB array
+ * @trb_dequeue: dequeue 'pointer' into TRB array
* @desc: usb_endpoint_descriptor pointer
* @dwc: pointer to DWC controller
* @saved_state: ep state saved during hibernation
@@ -464,13 +485,11 @@
*/
struct dwc3_ep {
struct usb_ep endpoint;
- struct list_head request_list;
- struct list_head req_queued;
+ struct list_head pending_list;
+ struct list_head started_list;
struct dwc3_trb *trb_pool;
dma_addr_t trb_pool_dma;
- u32 free_slot;
- u32 busy_slot;
const struct usb_ss_ep_comp_descriptor *comp_desc;
struct dwc3 *dwc;
@@ -486,6 +505,18 @@
/* This last one is specific to EP0 */
#define DWC3_EP0_DIR_IN (1 << 31)
+ /*
+ * IMPORTANT: we *know* we have 256 TRBs in our @trb_pool, so we will
+ * use a u8 type here. If anybody decides to increase number of TRBs to
+ * anything larger than 256 - I can't see why people would want to do
+ * this though - then this type needs to be changed.
+ *
+ * By using u8 types we ensure that our % operator when incrementing
+ * enqueue and dequeue get optimized away by the compiler.
+ */
+ u8 trb_enqueue;
+ u8 trb_dequeue;
+
u8 number;
u8 type;
u8 resource_index;
@@ -557,6 +588,7 @@
#define DWC3_TRB_CTRL_IOC (1 << 11)
#define DWC3_TRB_CTRL_SID_SOFN(n) (((n) & 0xffff) << 14)
+#define DWC3_TRBCTL_TYPE(n) ((n) & (0x3f << 4))
#define DWC3_TRBCTL_NORMAL DWC3_TRB_CTRL_TRBCTL(1)
#define DWC3_TRBCTL_CONTROL_SETUP DWC3_TRB_CTRL_TRBCTL(2)
#define DWC3_TRBCTL_CONTROL_STATUS2 DWC3_TRB_CTRL_TRBCTL(3)
@@ -623,19 +655,32 @@
/* HWPARAMS7 */
#define DWC3_RAM1_DEPTH(n) ((n) & 0xffff)
+/**
+ * struct dwc3_request - representation of a transfer request
+ * @request: struct usb_request to be transferred
+ * @list: a list_head used for request queueing
+ * @dep: struct dwc3_ep owning this request
+ * @first_trb_index: index to first trb used by this request
+ * @epnum: endpoint number to which this request refers
+ * @trb: pointer to struct dwc3_trb
+ * @trb_dma: DMA address of @trb
+ * @direction: IN or OUT direction flag
+ * @mapped: true when request has been dma-mapped
+ * @queued: true when request has been queued to HW
+ */
struct dwc3_request {
struct usb_request request;
struct list_head list;
struct dwc3_ep *dep;
- u32 start_slot;
+ u8 first_trb_index;
u8 epnum;
struct dwc3_trb *trb;
dma_addr_t trb_dma;
unsigned direction:1;
unsigned mapped:1;
- unsigned queued:1;
+ unsigned started:1;
};
/*
@@ -667,7 +712,6 @@
* @regs: base address for our registers
* @regs_size: address space size
* @nr_scratch: number of scratch buffers
- * @num_event_buffers: calculated number of event buffers
* @u1u2: only used on revisions <1.83a for workaround
* @maximum_speed: maximum speed requested (mainly for testing purposes)
* @revision: revision register contents
@@ -709,9 +753,7 @@
* 0 - utmi_sleep_n
* 1 - utmi_l1_suspend_n
* @is_fpga: true when we are using the FPGA board
- * @needs_fifo_resize: not all users might want fifo resizing, flag it
* @pullups_connected: true when Run/Stop bit is set
- * @resize_fifos: tells us it's ok to reconfigure our TxFIFO sizes.
* @setup_packet_pending: true when there's a Setup Packet in FIFO. Workaround
* @start_config_issued: true when StartConfig command has been issued
* @three_stage_setup: set if we perform a three phase setup
@@ -756,7 +798,7 @@
struct platform_device *xhci;
struct resource xhci_resources[DWC3_XHCI_RESOURCES_NUM];
- struct dwc3_event_buffer **ev_buffs;
+ struct dwc3_event_buffer *ev_buf;
struct dwc3_ep *eps[DWC3_ENDPOINTS_NUM];
struct usb_gadget gadget;
@@ -780,7 +822,6 @@
u32 gctl;
u32 nr_scratch;
- u32 num_event_buffers;
u32 u1u2;
u32 maximum_speed;
@@ -855,9 +896,7 @@
unsigned has_lpm_erratum:1;
unsigned is_utmi_l1_suspend:1;
unsigned is_fpga:1;
- unsigned needs_fifo_resize:1;
unsigned pullups_connected:1;
- unsigned resize_fifos:1;
unsigned setup_packet_pending:1;
unsigned three_stage_setup:1;
unsigned usb3_lpm_capable:1;
@@ -873,6 +912,7 @@
unsigned dis_u3_susphy_quirk:1;
unsigned dis_u2_susphy_quirk:1;
unsigned dis_enblslpm_quirk:1;
+ unsigned dis_rxdet_inp3_quirk:1;
unsigned tx_de_emphasis_quirk:1;
unsigned tx_de_emphasis:2;
@@ -938,6 +978,10 @@
#define DEPEVT_STATUS_CONTROL_DATA 1
#define DEPEVT_STATUS_CONTROL_STATUS 2
+/* In response to Start Transfer */
+#define DEPEVT_TRANSFER_NO_RESOURCE 1
+#define DEPEVT_TRANSFER_BUS_EXPIRY 2
+
u32 parameters:16;
} __packed;
@@ -1025,7 +1069,7 @@
/* prototypes */
void dwc3_set_mode(struct dwc3 *dwc, u32 mode);
-int dwc3_gadget_resize_tx_fifos(struct dwc3 *dwc);
+u32 dwc3_core_fifo_space(struct dwc3_ep *dep, u8 type);
/* check whether we are on the DWC_usb31 core */
static inline bool dwc3_is_usb31(struct dwc3 *dwc)
diff --git a/drivers/usb/dwc3/debug.h b/drivers/usb/dwc3/debug.h
index 07fbc2d..71e3180 100644
--- a/drivers/usb/dwc3/debug.h
+++ b/drivers/usb/dwc3/debug.h
@@ -217,11 +217,11 @@
void dwc3_trace(void (*trace)(struct va_format *), const char *fmt, ...);
#ifdef CONFIG_DEBUG_FS
-extern int dwc3_debugfs_init(struct dwc3 *);
+extern void dwc3_debugfs_init(struct dwc3 *);
extern void dwc3_debugfs_exit(struct dwc3 *);
#else
-static inline int dwc3_debugfs_init(struct dwc3 *d)
-{ return 0; }
+static inline void dwc3_debugfs_init(struct dwc3 *d)
+{ }
static inline void dwc3_debugfs_exit(struct dwc3 *d)
{ }
#endif
diff --git a/drivers/usb/dwc3/debugfs.c b/drivers/usb/dwc3/debugfs.c
index cebf9e3..b1dd3c6 100644
--- a/drivers/usb/dwc3/debugfs.c
+++ b/drivers/usb/dwc3/debugfs.c
@@ -618,24 +618,323 @@
.release = single_release,
};
-int dwc3_debugfs_init(struct dwc3 *dwc)
-{
- struct dentry *root;
- struct dentry *file;
- int ret;
+struct dwc3_ep_file_map {
+ char name[25];
+ int (*show)(struct seq_file *s, void *unused);
+};
- root = debugfs_create_dir(dev_name(dwc->dev), NULL);
- if (!root) {
- ret = -ENOMEM;
- goto err0;
+static int dwc3_tx_fifo_queue_show(struct seq_file *s, void *unused)
+{
+ struct dwc3_ep *dep = s->private;
+ struct dwc3 *dwc = dep->dwc;
+ unsigned long flags;
+ u32 val;
+
+ spin_lock_irqsave(&dwc->lock, flags);
+ val = dwc3_core_fifo_space(dep, DWC3_TXFIFOQ);
+ seq_printf(s, "%u\n", val);
+ spin_unlock_irqrestore(&dwc->lock, flags);
+
+ return 0;
+}
+
+static int dwc3_rx_fifo_queue_show(struct seq_file *s, void *unused)
+{
+ struct dwc3_ep *dep = s->private;
+ struct dwc3 *dwc = dep->dwc;
+ unsigned long flags;
+ u32 val;
+
+ spin_lock_irqsave(&dwc->lock, flags);
+ val = dwc3_core_fifo_space(dep, DWC3_RXFIFOQ);
+ seq_printf(s, "%u\n", val);
+ spin_unlock_irqrestore(&dwc->lock, flags);
+
+ return 0;
+}
+
+static int dwc3_tx_request_queue_show(struct seq_file *s, void *unused)
+{
+ struct dwc3_ep *dep = s->private;
+ struct dwc3 *dwc = dep->dwc;
+ unsigned long flags;
+ u32 val;
+
+ spin_lock_irqsave(&dwc->lock, flags);
+ val = dwc3_core_fifo_space(dep, DWC3_TXREQQ);
+ seq_printf(s, "%u\n", val);
+ spin_unlock_irqrestore(&dwc->lock, flags);
+
+ return 0;
+}
+
+static int dwc3_rx_request_queue_show(struct seq_file *s, void *unused)
+{
+ struct dwc3_ep *dep = s->private;
+ struct dwc3 *dwc = dep->dwc;
+ unsigned long flags;
+ u32 val;
+
+ spin_lock_irqsave(&dwc->lock, flags);
+ val = dwc3_core_fifo_space(dep, DWC3_RXREQQ);
+ seq_printf(s, "%u\n", val);
+ spin_unlock_irqrestore(&dwc->lock, flags);
+
+ return 0;
+}
+
+static int dwc3_rx_info_queue_show(struct seq_file *s, void *unused)
+{
+ struct dwc3_ep *dep = s->private;
+ struct dwc3 *dwc = dep->dwc;
+ unsigned long flags;
+ u32 val;
+
+ spin_lock_irqsave(&dwc->lock, flags);
+ val = dwc3_core_fifo_space(dep, DWC3_RXINFOQ);
+ seq_printf(s, "%u\n", val);
+ spin_unlock_irqrestore(&dwc->lock, flags);
+
+ return 0;
+}
+
+static int dwc3_descriptor_fetch_queue_show(struct seq_file *s, void *unused)
+{
+ struct dwc3_ep *dep = s->private;
+ struct dwc3 *dwc = dep->dwc;
+ unsigned long flags;
+ u32 val;
+
+ spin_lock_irqsave(&dwc->lock, flags);
+ val = dwc3_core_fifo_space(dep, DWC3_DESCFETCHQ);
+ seq_printf(s, "%u\n", val);
+ spin_unlock_irqrestore(&dwc->lock, flags);
+
+ return 0;
+}
+
+static int dwc3_event_queue_show(struct seq_file *s, void *unused)
+{
+ struct dwc3_ep *dep = s->private;
+ struct dwc3 *dwc = dep->dwc;
+ unsigned long flags;
+ u32 val;
+
+ spin_lock_irqsave(&dwc->lock, flags);
+ val = dwc3_core_fifo_space(dep, DWC3_EVENTQ);
+ seq_printf(s, "%u\n", val);
+ spin_unlock_irqrestore(&dwc->lock, flags);
+
+ return 0;
+}
+
+static int dwc3_ep_transfer_type_show(struct seq_file *s, void *unused)
+{
+ struct dwc3_ep *dep = s->private;
+ struct dwc3 *dwc = dep->dwc;
+ unsigned long flags;
+
+ spin_lock_irqsave(&dwc->lock, flags);
+ if (!(dep->flags & DWC3_EP_ENABLED) ||
+ !dep->endpoint.desc) {
+ seq_printf(s, "--\n");
+ goto out;
}
+ switch (usb_endpoint_type(dep->endpoint.desc)) {
+ case USB_ENDPOINT_XFER_CONTROL:
+ seq_printf(s, "control\n");
+ break;
+ case USB_ENDPOINT_XFER_ISOC:
+ seq_printf(s, "isochronous\n");
+ break;
+ case USB_ENDPOINT_XFER_BULK:
+ seq_printf(s, "bulk\n");
+ break;
+ case USB_ENDPOINT_XFER_INT:
+ seq_printf(s, "interrupt\n");
+ break;
+ default:
+ seq_printf(s, "--\n");
+ }
+
+out:
+ spin_unlock_irqrestore(&dwc->lock, flags);
+
+ return 0;
+}
+
+static inline const char *dwc3_trb_type_string(struct dwc3_trb *trb)
+{
+ switch (DWC3_TRBCTL_TYPE(trb->ctrl)) {
+ case DWC3_TRBCTL_NORMAL:
+ return "normal";
+ case DWC3_TRBCTL_CONTROL_SETUP:
+ return "control-setup";
+ case DWC3_TRBCTL_CONTROL_STATUS2:
+ return "control-status2";
+ case DWC3_TRBCTL_CONTROL_STATUS3:
+ return "control-status3";
+ case DWC3_TRBCTL_CONTROL_DATA:
+ return "control-data";
+ case DWC3_TRBCTL_ISOCHRONOUS_FIRST:
+ return "isoc-first";
+ case DWC3_TRBCTL_ISOCHRONOUS:
+ return "isoc";
+ case DWC3_TRBCTL_LINK_TRB:
+ return "link";
+ default:
+ return "UNKNOWN";
+ }
+}
+
+static int dwc3_ep_trb_ring_show(struct seq_file *s, void *unused)
+{
+ struct dwc3_ep *dep = s->private;
+ struct dwc3 *dwc = dep->dwc;
+ unsigned long flags;
+ int i;
+
+ spin_lock_irqsave(&dwc->lock, flags);
+ if (dep->number <= 1) {
+ seq_printf(s, "--\n");
+ goto out;
+ }
+
+ seq_printf(s, "enqueue pointer %d\n", dep->trb_enqueue);
+ seq_printf(s, "dequeue pointer %d\n", dep->trb_dequeue);
+ seq_printf(s, "\n--------------------------------------------------\n\n");
+ seq_printf(s, "buffer_addr,size,type,ioc,isp_imi,csp,chn,lst,hwo\n");
+
+ for (i = 0; i < DWC3_TRB_NUM; i++) {
+ struct dwc3_trb *trb = &dep->trb_pool[i];
+
+ seq_printf(s, "%08x%08x,%d,%s,%d,%d,%d,%d,%d,%d\n",
+ trb->bph, trb->bpl, trb->size,
+ dwc3_trb_type_string(trb),
+ !!(trb->ctrl & DWC3_TRB_CTRL_IOC),
+ !!(trb->ctrl & DWC3_TRB_CTRL_ISP_IMI),
+ !!(trb->ctrl & DWC3_TRB_CTRL_CSP),
+ !!(trb->ctrl & DWC3_TRB_CTRL_CHN),
+ !!(trb->ctrl & DWC3_TRB_CTRL_LST),
+ !!(trb->ctrl & DWC3_TRB_CTRL_HWO));
+ }
+
+out:
+ spin_unlock_irqrestore(&dwc->lock, flags);
+
+ return 0;
+}
+
+static struct dwc3_ep_file_map map[] = {
+ { "tx_fifo_queue", dwc3_tx_fifo_queue_show, },
+ { "rx_fifo_queue", dwc3_rx_fifo_queue_show, },
+ { "tx_request_queue", dwc3_tx_request_queue_show, },
+ { "rx_request_queue", dwc3_rx_request_queue_show, },
+ { "rx_info_queue", dwc3_rx_info_queue_show, },
+ { "descriptor_fetch_queue", dwc3_descriptor_fetch_queue_show, },
+ { "event_queue", dwc3_event_queue_show, },
+ { "transfer_type", dwc3_ep_transfer_type_show, },
+ { "trb_ring", dwc3_ep_trb_ring_show, },
+};
+
+static int dwc3_endpoint_open(struct inode *inode, struct file *file)
+{
+ const char *file_name = file_dentry(file)->d_iname;
+ struct dwc3_ep_file_map *f_map;
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(map); i++) {
+ f_map = &map[i];
+
+ if (strcmp(f_map->name, file_name) == 0)
+ break;
+ }
+
+ return single_open(file, f_map->show, inode->i_private);
+}
+
+static const struct file_operations dwc3_endpoint_fops = {
+ .open = dwc3_endpoint_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
+static void dwc3_debugfs_create_endpoint_file(struct dwc3_ep *dep,
+ struct dentry *parent, int type)
+{
+ struct dentry *file;
+ struct dwc3_ep_file_map *ep_file = &map[type];
+
+ file = debugfs_create_file(ep_file->name, S_IRUGO, parent, dep,
+ &dwc3_endpoint_fops);
+}
+
+static void dwc3_debugfs_create_endpoint_files(struct dwc3_ep *dep,
+ struct dentry *parent)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(map); i++)
+ dwc3_debugfs_create_endpoint_file(dep, parent, i);
+}
+
+static void dwc3_debugfs_create_endpoint_dir(struct dwc3_ep *dep,
+ struct dentry *parent)
+{
+ struct dentry *dir;
+
+ dir = debugfs_create_dir(dep->name, parent);
+ if (IS_ERR_OR_NULL(dir))
+ return;
+
+ dwc3_debugfs_create_endpoint_files(dep, dir);
+}
+
+static void dwc3_debugfs_create_endpoint_dirs(struct dwc3 *dwc,
+ struct dentry *parent)
+{
+ int i;
+
+ for (i = 0; i < dwc->num_in_eps; i++) {
+ u8 epnum = (i << 1) | 1;
+ struct dwc3_ep *dep = dwc->eps[epnum];
+
+ if (!dep)
+ continue;
+
+ dwc3_debugfs_create_endpoint_dir(dep, parent);
+ }
+
+ for (i = 0; i < dwc->num_out_eps; i++) {
+ u8 epnum = (i << 1);
+ struct dwc3_ep *dep = dwc->eps[epnum];
+
+ if (!dep)
+ continue;
+
+ dwc3_debugfs_create_endpoint_dir(dep, parent);
+ }
+}
+
+void dwc3_debugfs_init(struct dwc3 *dwc)
+{
+ struct dentry *root;
+ struct dentry *file;
+
+ root = debugfs_create_dir(dev_name(dwc->dev), NULL);
+ if (IS_ERR_OR_NULL(root)) {
+ if (!root)
+ dev_err(dwc->dev, "Can't create debugfs root\n");
+ return;
+ }
dwc->root = root;
dwc->regset = kzalloc(sizeof(*dwc->regset), GFP_KERNEL);
if (!dwc->regset) {
- ret = -ENOMEM;
- goto err1;
+ debugfs_remove_recursive(root);
+ return;
}
dwc->regset->regs = dwc3_regs;
@@ -643,47 +942,30 @@
dwc->regset->base = dwc->regs;
file = debugfs_create_regset32("regdump", S_IRUGO, root, dwc->regset);
- if (!file) {
- ret = -ENOMEM;
- goto err2;
- }
+ if (!file)
+ dev_dbg(dwc->dev, "Can't create debugfs regdump\n");
if (IS_ENABLED(CONFIG_USB_DWC3_DUAL_ROLE)) {
file = debugfs_create_file("mode", S_IRUGO | S_IWUSR, root,
dwc, &dwc3_mode_fops);
- if (!file) {
- ret = -ENOMEM;
- goto err2;
- }
+ if (!file)
+ dev_dbg(dwc->dev, "Can't create debugfs mode\n");
}
if (IS_ENABLED(CONFIG_USB_DWC3_DUAL_ROLE) ||
IS_ENABLED(CONFIG_USB_DWC3_GADGET)) {
file = debugfs_create_file("testmode", S_IRUGO | S_IWUSR, root,
dwc, &dwc3_testmode_fops);
- if (!file) {
- ret = -ENOMEM;
- goto err2;
- }
+ if (!file)
+ dev_dbg(dwc->dev, "Can't create debugfs testmode\n");
- file = debugfs_create_file("link_state", S_IRUGO | S_IWUSR, root,
- dwc, &dwc3_link_state_fops);
- if (!file) {
- ret = -ENOMEM;
- goto err2;
- }
+ file = debugfs_create_file("link_state", S_IRUGO | S_IWUSR,
+ root, dwc, &dwc3_link_state_fops);
+ if (!file)
+ dev_dbg(dwc->dev, "Can't create debugfs link_state\n");
+
+ dwc3_debugfs_create_endpoint_dirs(dwc, root);
}
-
- return 0;
-
-err2:
- kfree(dwc->regset);
-
-err1:
- debugfs_remove_recursive(root);
-
-err0:
- return ret;
}
void dwc3_debugfs_exit(struct dwc3 *dwc)
diff --git a/drivers/usb/dwc3/dwc3-omap.c b/drivers/usb/dwc3/dwc3-omap.c
index 55da2c7..af26449 100644
--- a/drivers/usb/dwc3/dwc3-omap.c
+++ b/drivers/usb/dwc3/dwc3-omap.c
@@ -126,8 +126,6 @@
u32 debug_offset;
u32 irq0_offset;
- u32 dma_status:1;
-
struct extcon_dev *edev;
struct notifier_block vbus_nb;
struct notifier_block id_nb;
@@ -277,9 +275,6 @@
reg = dwc3_omap_read_irqmisc_status(omap);
- if (reg & USBOTGSS_IRQMISC_DMADISABLECLR)
- omap->dma_status = false;
-
dwc3_omap_write_irqmisc_status(omap, reg);
reg = dwc3_omap_read_irq0_status(omap);
@@ -331,8 +326,6 @@
dwc3_omap_write_irqmisc_clr(omap, reg);
}
-static u64 dwc3_omap_dma_mask = DMA_BIT_MASK(32);
-
static int dwc3_omap_id_notifier(struct notifier_block *nb,
unsigned long event, void *ptr)
{
@@ -490,7 +483,6 @@
omap->irq = irq;
omap->base = base;
omap->vbus_reg = vbus_reg;
- dev->dma_mask = &dwc3_omap_dma_mask;
pm_runtime_enable(dev);
ret = pm_runtime_get_sync(dev);
@@ -504,7 +496,6 @@
/* check the DMA Status */
reg = dwc3_omap_readl(omap->base, USBOTGSS_SYSCONFIG);
- omap->dma_status = !!(reg & USBOTGSS_SYSCONFIG_DMADISABLE);
ret = devm_request_irq(dev, omap->irq, dwc3_omap_interrupt, 0,
"dwc3-omap", omap);
diff --git a/drivers/usb/dwc3/dwc3-pci.c b/drivers/usb/dwc3/dwc3-pci.c
index adc1e8a..14196cd 100644
--- a/drivers/usb/dwc3/dwc3-pci.c
+++ b/drivers/usb/dwc3/dwc3-pci.c
@@ -47,7 +47,7 @@
{ },
};
-static int dwc3_pci_quirks(struct pci_dev *pdev)
+static int dwc3_pci_quirks(struct pci_dev *pdev, struct platform_device *dwc3)
{
if (pdev->vendor == PCI_VENDOR_ID_AMD &&
pdev->device == PCI_DEVICE_ID_AMD_NL_USB) {
@@ -77,8 +77,7 @@
pdata.dis_u3_susphy_quirk = true;
pdata.dis_u2_susphy_quirk = true;
- return platform_device_add_data(pci_get_drvdata(pdev), &pdata,
- sizeof(pdata));
+ return platform_device_add_data(dwc3, &pdata, sizeof(pdata));
}
if (pdev->vendor == PCI_VENDOR_ID_INTEL &&
@@ -123,8 +122,7 @@
pdata.has_lpm_erratum = true;
pdata.dis_enblslpm_quirk = true;
- return platform_device_add_data(pci_get_drvdata(pdev), &pdata,
- sizeof(pdata));
+ return platform_device_add_data(dwc3, &pdata, sizeof(pdata));
}
return 0;
@@ -169,20 +167,20 @@
return ret;
}
- pci_set_drvdata(pci, dwc3);
- ret = dwc3_pci_quirks(pci);
- if (ret)
- goto err;
-
dwc3->dev.parent = dev;
ACPI_COMPANION_SET(&dwc3->dev, ACPI_COMPANION(dev));
+ ret = dwc3_pci_quirks(pci, dwc3);
+ if (ret)
+ goto err;
+
ret = platform_device_add(dwc3);
if (ret) {
dev_err(dev, "failed to register dwc3 device\n");
goto err;
}
+ pci_set_drvdata(pci, dwc3);
return 0;
err:
platform_device_put(dwc3);
diff --git a/drivers/usb/dwc3/ep0.c b/drivers/usb/dwc3/ep0.c
index eca2e6d..51b52a7 100644
--- a/drivers/usb/dwc3/ep0.c
+++ b/drivers/usb/dwc3/ep0.c
@@ -70,10 +70,10 @@
return 0;
}
- trb = &dwc->ep0_trb[dep->free_slot];
+ trb = &dwc->ep0_trb[dep->trb_enqueue];
if (chain)
- dep->free_slot++;
+ dep->trb_enqueue++;
trb->bpl = lower_32_bits(buf_dma);
trb->bph = upper_32_bits(buf_dma);
@@ -124,7 +124,7 @@
req->request.status = -EINPROGRESS;
req->epnum = dep->number;
- list_add_tail(&req->list, &dep->request_list);
+ list_add_tail(&req->list, &dep->pending_list);
/*
* Gadget driver might not be quick enough to queue a request
@@ -240,7 +240,7 @@
}
/* we share one TRB for ep0/1 */
- if (!list_empty(&dep->request_list)) {
+ if (!list_empty(&dep->pending_list)) {
ret = -EBUSY;
goto out;
}
@@ -272,10 +272,10 @@
dep->flags = DWC3_EP_ENABLED;
dwc->delayed_status = false;
- if (!list_empty(&dep->request_list)) {
+ if (!list_empty(&dep->pending_list)) {
struct dwc3_request *req;
- req = next_request(&dep->request_list);
+ req = next_request(&dep->pending_list);
dwc3_gadget_giveback(dep, req, -ECONNRESET);
}
@@ -463,8 +463,18 @@
if (!set)
return -EINVAL;
- dwc->test_mode_nr = wIndex >> 8;
- dwc->test_mode = true;
+ switch (wIndex >> 8) {
+ case TEST_J:
+ case TEST_K:
+ case TEST_SE0_NAK:
+ case TEST_PACKET:
+ case TEST_FORCE_EN:
+ dwc->test_mode_nr = wIndex >> 8;
+ dwc->test_mode = true;
+ break;
+ default:
+ return -EINVAL;
+ }
break;
default:
return -EINVAL;
@@ -586,9 +596,6 @@
reg = dwc3_readl(dwc->regs, DWC3_DCTL);
reg |= (DWC3_DCTL_ACCEPTU1ENA | DWC3_DCTL_ACCEPTU2ENA);
dwc3_writel(dwc->regs, DWC3_DCTL, reg);
-
- dwc->resize_fifos = true;
- dwc3_trace(trace_dwc3_ep0, "resize FIFOs flag SET");
}
break;
@@ -809,7 +816,7 @@
trace_dwc3_complete_trb(ep0, trb);
- r = next_request(&ep0->request_list);
+ r = next_request(&ep0->pending_list);
if (!r)
return;
@@ -848,7 +855,7 @@
trb++;
length = trb->size & DWC3_TRB_SIZE_MASK;
- ep0->free_slot = 0;
+ ep0->trb_enqueue = 0;
}
transfer_size = roundup((ur->length - transfer_size),
@@ -897,8 +904,8 @@
trace_dwc3_complete_trb(dep, trb);
- if (!list_empty(&dep->request_list)) {
- r = next_request(&dep->request_list);
+ if (!list_empty(&dep->pending_list)) {
+ r = next_request(&dep->pending_list);
dwc3_gadget_giveback(dep, r, 0);
}
@@ -1027,12 +1034,6 @@
static void __dwc3_ep0_do_control_status(struct dwc3 *dwc, struct dwc3_ep *dep)
{
- if (dwc->resize_fifos) {
- dwc3_trace(trace_dwc3_ep0, "Resizing FIFOs");
- dwc3_gadget_resize_tx_fifos(dwc);
- dwc->resize_fifos = 0;
- }
-
WARN_ON(dwc3_ep0_start_control_status(dep));
}
diff --git a/drivers/usb/dwc3/gadget.c b/drivers/usb/dwc3/gadget.c
index 8e4a1b1..9a7d0bd 100644
--- a/drivers/usb/dwc3/gadget.c
+++ b/drivers/usb/dwc3/gadget.c
@@ -145,90 +145,21 @@
return -ETIMEDOUT;
}
-/**
- * dwc3_gadget_resize_tx_fifos - reallocate fifo spaces for current use-case
- * @dwc: pointer to our context structure
- *
- * This function will a best effort FIFO allocation in order
- * to improve FIFO usage and throughput, while still allowing
- * us to enable as many endpoints as possible.
- *
- * Keep in mind that this operation will be highly dependent
- * on the configured size for RAM1 - which contains TxFifo -,
- * the amount of endpoints enabled on coreConsultant tool, and
- * the width of the Master Bus.
- *
- * In the ideal world, we would always be able to satisfy the
- * following equation:
- *
- * ((512 + 2 * MDWIDTH-Bytes) + (Number of IN Endpoints - 1) * \
- * (3 * (1024 + MDWIDTH-Bytes) + MDWIDTH-Bytes)) / MDWIDTH-Bytes
- *
- * Unfortunately, due to many variables that's not always the case.
- */
-int dwc3_gadget_resize_tx_fifos(struct dwc3 *dwc)
+static void dwc3_ep_inc_enq(struct dwc3_ep *dep)
{
- int last_fifo_depth = 0;
- int ram1_depth;
- int fifo_size;
- int mdwidth;
- int num;
+ dep->trb_enqueue++;
+ dep->trb_enqueue %= DWC3_TRB_NUM;
+}
- if (!dwc->needs_fifo_resize)
- return 0;
+static void dwc3_ep_inc_deq(struct dwc3_ep *dep)
+{
+ dep->trb_dequeue++;
+ dep->trb_dequeue %= DWC3_TRB_NUM;
+}
- ram1_depth = DWC3_RAM1_DEPTH(dwc->hwparams.hwparams7);
- mdwidth = DWC3_MDWIDTH(dwc->hwparams.hwparams0);
-
- /* MDWIDTH is represented in bits, we need it in bytes */
- mdwidth >>= 3;
-
- /*
- * FIXME For now we will only allocate 1 wMaxPacketSize space
- * for each enabled endpoint, later patches will come to
- * improve this algorithm so that we better use the internal
- * FIFO space
- */
- for (num = 0; num < dwc->num_in_eps; num++) {
- /* bit0 indicates direction; 1 means IN ep */
- struct dwc3_ep *dep = dwc->eps[(num << 1) | 1];
- int mult = 1;
- int tmp;
-
- if (!(dep->flags & DWC3_EP_ENABLED))
- continue;
-
- if (usb_endpoint_xfer_bulk(dep->endpoint.desc)
- || usb_endpoint_xfer_isoc(dep->endpoint.desc))
- mult = 3;
-
- /*
- * REVISIT: the following assumes we will always have enough
- * space available on the FIFO RAM for all possible use cases.
- * Make sure that's true somehow and change FIFO allocation
- * accordingly.
- *
- * If we have Bulk or Isochronous endpoints, we want
- * them to be able to be very, very fast. So we're giving
- * those endpoints a fifo_size which is enough for 3 full
- * packets
- */
- tmp = mult * (dep->endpoint.maxpacket + mdwidth);
- tmp += mdwidth;
-
- fifo_size = DIV_ROUND_UP(tmp, mdwidth);
-
- fifo_size |= (last_fifo_depth << 16);
-
- dwc3_trace(trace_dwc3_gadget, "%s: Fifo Addr %04x Size %d",
- dep->name, last_fifo_depth, fifo_size & 0xffff);
-
- dwc3_writel(dwc->regs, DWC3_GTXFIFOSIZ(num), fifo_size);
-
- last_fifo_depth += (fifo_size & 0xffff);
- }
-
- return 0;
+static int dwc3_ep_is_last_trb(unsigned int index)
+{
+ return index == DWC3_TRB_NUM - 1;
}
void dwc3_gadget_giveback(struct dwc3_ep *dep, struct dwc3_request *req,
@@ -237,21 +168,19 @@
struct dwc3 *dwc = dep->dwc;
int i;
- if (req->queued) {
+ if (req->started) {
i = 0;
do {
- dep->busy_slot++;
+ dwc3_ep_inc_deq(dep);
/*
* Skip LINK TRB. We can't use req->trb and check for
* DWC3_TRBCTL_LINK_TRB because it points the TRB we
* just completed (not the LINK TRB).
*/
- if (((dep->busy_slot & DWC3_TRB_MASK) ==
- DWC3_TRB_NUM- 1) &&
- usb_endpoint_xfer_isoc(dep->endpoint.desc))
- dep->busy_slot++;
+ if (dwc3_ep_is_last_trb(dep->trb_dequeue))
+ dwc3_ep_inc_deq(dep);
} while(++i < req->request.num_mapped_sgs);
- req->queued = false;
+ req->started = false;
}
list_del(&req->list);
req->trb = NULL;
@@ -307,6 +236,8 @@
} while (1);
}
+static int __dwc3_gadget_wakeup(struct dwc3 *dwc);
+
int dwc3_send_gadget_ep_cmd(struct dwc3 *dwc, unsigned ep,
unsigned cmd, struct dwc3_gadget_ep_cmd_params *params)
{
@@ -314,8 +245,40 @@
u32 timeout = 500;
u32 reg;
+ int susphy = false;
+ int ret = -EINVAL;
+
trace_dwc3_gadget_ep_cmd(dep, cmd, params);
+ /*
+ * Synopsys Databook 2.60a states, on section 6.3.2.5.[1-8], that if
+ * we're issuing an endpoint command, we must check if
+ * GUSB2PHYCFG.SUSPHY bit is set. If it is, then we need to clear it.
+ *
+ * We will also set SUSPHY bit to what it was before returning as stated
+ * by the same section on Synopsys databook.
+ */
+ reg = dwc3_readl(dwc->regs, DWC3_GUSB2PHYCFG(0));
+ if (unlikely(reg & DWC3_GUSB2PHYCFG_SUSPHY)) {
+ susphy = true;
+ reg &= ~DWC3_GUSB2PHYCFG_SUSPHY;
+ dwc3_writel(dwc->regs, DWC3_GUSB2PHYCFG(0), reg);
+ }
+
+ if (cmd == DWC3_DEPCMD_STARTTRANSFER) {
+ int needs_wakeup;
+
+ needs_wakeup = (dwc->link_state == DWC3_LINK_STATE_U1 ||
+ dwc->link_state == DWC3_LINK_STATE_U2 ||
+ dwc->link_state == DWC3_LINK_STATE_U3);
+
+ if (unlikely(needs_wakeup)) {
+ ret = __dwc3_gadget_wakeup(dwc);
+ dev_WARN_ONCE(dwc->dev, ret, "wakeup failed --> %d\n",
+ ret);
+ }
+ }
+
dwc3_writel(dwc->regs, DWC3_DEPCMDPAR0(ep), params->param0);
dwc3_writel(dwc->regs, DWC3_DEPCMDPAR1(ep), params->param1);
dwc3_writel(dwc->regs, DWC3_DEPCMDPAR2(ep), params->param2);
@@ -324,12 +287,40 @@
do {
reg = dwc3_readl(dwc->regs, DWC3_DEPCMD(ep));
if (!(reg & DWC3_DEPCMD_CMDACT)) {
+ int cmd_status = DWC3_DEPCMD_STATUS(reg);
+
dwc3_trace(trace_dwc3_gadget,
"Command Complete --> %d",
- DWC3_DEPCMD_STATUS(reg));
- if (DWC3_DEPCMD_STATUS(reg))
- return -EINVAL;
- return 0;
+ cmd_status);
+
+ switch (cmd_status) {
+ case 0:
+ ret = 0;
+ break;
+ case DEPEVT_TRANSFER_NO_RESOURCE:
+ dwc3_trace(trace_dwc3_gadget, "%s: no resource available");
+ ret = -EINVAL;
+ break;
+ case DEPEVT_TRANSFER_BUS_EXPIRY:
+ /*
+ * SW issues START TRANSFER command to
+ * isochronous ep with future frame interval. If
+ * future interval time has already passed when
+ * core receives the command, it will respond
+ * with an error status of 'Bus Expiry'.
+ *
+ * Instead of always returning -EINVAL, let's
+ * give a hint to the gadget driver that this is
+ * the case by returning -EAGAIN.
+ */
+ dwc3_trace(trace_dwc3_gadget, "%s: bus expiry");
+ ret = -EAGAIN;
+ break;
+ default:
+ dev_WARN(dwc->dev, "UNKNOWN cmd status\n");
+ }
+
+ break;
}
/*
@@ -340,11 +331,20 @@
if (!timeout) {
dwc3_trace(trace_dwc3_gadget,
"Command Timed Out");
- return -ETIMEDOUT;
+ ret = -ETIMEDOUT;
+ break;
}
udelay(1);
} while (1);
+
+ if (unlikely(susphy)) {
+ reg = dwc3_readl(dwc->regs, DWC3_GUSB2PHYCFG(0));
+ reg |= DWC3_GUSB2PHYCFG_SUSPHY;
+ dwc3_writel(dwc->regs, DWC3_GUSB2PHYCFG(0), reg);
+ }
+
+ return ret;
}
static dma_addr_t dwc3_trb_dma_offset(struct dwc3_ep *dep,
@@ -464,9 +464,19 @@
/* Burst size is only needed in SuperSpeed mode */
if (dwc->gadget.speed >= USB_SPEED_SUPER) {
- u32 burst = dep->endpoint.maxburst - 1;
+ u32 burst = dep->endpoint.maxburst;
+ u32 nump;
+ u32 reg;
- params.param0 |= DWC3_DEPCFG_BURST_SIZE(burst);
+ /* update NumP */
+ reg = dwc3_readl(dwc->regs, DWC3_DCFG);
+ nump = DWC3_DCFG_NUMP(reg);
+ nump = max(nump, burst);
+ reg &= ~DWC3_DCFG_NUMP_MASK;
+ reg |= nump << DWC3_DCFG_NUMP_SHIFT;
+ dwc3_writel(dwc->regs, DWC3_DCFG, reg);
+
+ params.param0 |= DWC3_DEPCFG_BURST_SIZE(burst - 1);
}
if (ignore)
@@ -567,10 +577,10 @@
reg |= DWC3_DALEPENA_EP(dep->number);
dwc3_writel(dwc->regs, DWC3_DALEPENA, reg);
- if (!usb_endpoint_xfer_isoc(desc))
+ if (usb_endpoint_xfer_control(desc))
goto out;
- /* Link TRB for ISOC. The HWO bit is never reset */
+ /* Link TRB. The HWO bit is never reset */
trb_st_hw = &dep->trb_pool[0];
trb_link = &dep->trb_pool[DWC3_TRB_NUM - 1];
@@ -608,19 +618,19 @@
{
struct dwc3_request *req;
- if (!list_empty(&dep->req_queued)) {
+ if (!list_empty(&dep->started_list)) {
dwc3_stop_active_transfer(dwc, dep->number, true);
/* - giveback all requests to gadget driver */
- while (!list_empty(&dep->req_queued)) {
- req = next_request(&dep->req_queued);
+ while (!list_empty(&dep->started_list)) {
+ req = next_request(&dep->started_list);
dwc3_gadget_giveback(dep, req, -ESHUTDOWN);
}
}
- while (!list_empty(&dep->request_list)) {
- req = next_request(&dep->request_list);
+ while (!list_empty(&dep->pending_list)) {
+ req = next_request(&dep->pending_list);
dwc3_gadget_giveback(dep, req, -ESHUTDOWN);
}
@@ -783,20 +793,19 @@
chain ? " chain" : "");
- trb = &dep->trb_pool[dep->free_slot & DWC3_TRB_MASK];
+ trb = &dep->trb_pool[dep->trb_enqueue];
if (!req->trb) {
- dwc3_gadget_move_request_queued(req);
+ dwc3_gadget_move_started_request(req);
req->trb = trb;
req->trb_dma = dwc3_trb_dma_offset(dep, trb);
- req->start_slot = dep->free_slot & DWC3_TRB_MASK;
+ req->first_trb_index = dep->trb_enqueue;
}
- dep->free_slot++;
- /* Skip the LINK-TRB on ISOC */
- if (((dep->free_slot & DWC3_TRB_MASK) == DWC3_TRB_NUM - 1) &&
- usb_endpoint_xfer_isoc(dep->endpoint.desc))
- dep->free_slot++;
+ dwc3_ep_inc_enq(dep);
+ /* Skip the LINK-TRB */
+ if (dwc3_ep_is_last_trb(dep->trb_enqueue))
+ dwc3_ep_inc_enq(dep);
trb->size = DWC3_TRB_SIZE_LENGTH(length);
trb->bpl = lower_32_bits(dma);
@@ -812,6 +821,9 @@
trb->ctrl = DWC3_TRBCTL_ISOCHRONOUS_FIRST;
else
trb->ctrl = DWC3_TRBCTL_ISOCHRONOUS;
+
+ /* always enable Interrupt on Missed ISOC */
+ trb->ctrl |= DWC3_TRB_CTRL_ISP_IMI;
break;
case USB_ENDPOINT_XFER_BULK:
@@ -826,15 +838,14 @@
BUG();
}
- if (!req->request.no_interrupt && !chain)
- trb->ctrl |= DWC3_TRB_CTRL_IOC;
+ /* always enable Continue on Short Packet */
+ trb->ctrl |= DWC3_TRB_CTRL_CSP;
- if (usb_endpoint_xfer_isoc(dep->endpoint.desc)) {
- trb->ctrl |= DWC3_TRB_CTRL_ISP_IMI;
- trb->ctrl |= DWC3_TRB_CTRL_CSP;
- } else if (last) {
+ if (!req->request.no_interrupt && !chain)
+ trb->ctrl |= DWC3_TRB_CTRL_IOC | DWC3_TRB_CTRL_ISP_IMI;
+
+ if (last)
trb->ctrl |= DWC3_TRB_CTRL_LST;
- }
if (chain)
trb->ctrl |= DWC3_TRB_CTRL_CHN;
@@ -860,55 +871,29 @@
{
struct dwc3_request *req, *n;
u32 trbs_left;
- u32 max;
unsigned int last_one = 0;
BUILD_BUG_ON_NOT_POWER_OF_2(DWC3_TRB_NUM);
- /* the first request must not be queued */
- trbs_left = (dep->busy_slot - dep->free_slot) & DWC3_TRB_MASK;
-
- /* Can't wrap around on a non-isoc EP since there's no link TRB */
- if (!usb_endpoint_xfer_isoc(dep->endpoint.desc)) {
- max = DWC3_TRB_NUM - (dep->free_slot & DWC3_TRB_MASK);
- if (trbs_left > max)
- trbs_left = max;
- }
+ trbs_left = dep->trb_dequeue - dep->trb_enqueue;
/*
- * If busy & slot are equal than it is either full or empty. If we are
- * starting to process requests then we are empty. Otherwise we are
+ * If enqueue & dequeue are equal than it is either full or empty. If we
+ * are starting to process requests then we are empty. Otherwise we are
* full and don't do anything
*/
if (!trbs_left) {
if (!starting)
return;
+
trbs_left = DWC3_TRB_NUM;
- /*
- * In case we start from scratch, we queue the ISOC requests
- * starting from slot 1. This is done because we use ring
- * buffer and have no LST bit to stop us. Instead, we place
- * IOC bit every TRB_NUM/4. We try to avoid having an interrupt
- * after the first request so we start at slot 1 and have
- * 7 requests proceed before we hit the first IOC.
- * Other transfer types don't use the ring buffer and are
- * processed from the first TRB until the last one. Since we
- * don't wrap around we have to start at the beginning.
- */
- if (usb_endpoint_xfer_isoc(dep->endpoint.desc)) {
- dep->busy_slot = 1;
- dep->free_slot = 1;
- } else {
- dep->busy_slot = 0;
- dep->free_slot = 0;
- }
}
/* The last TRB is a link TRB, not used for xfer */
- if ((trbs_left <= 1) && usb_endpoint_xfer_isoc(dep->endpoint.desc))
+ if (trbs_left <= 1)
return;
- list_for_each_entry_safe(req, n, &dep->request_list, list) {
+ list_for_each_entry_safe(req, n, &dep->pending_list, list) {
unsigned length;
dma_addr_t dma;
last_one = false;
@@ -927,7 +912,7 @@
if (i == (request->num_mapped_sgs - 1) ||
sg_is_last(s)) {
- if (list_empty(&dep->request_list))
+ if (list_empty(&dep->pending_list))
last_one = true;
chain = false;
}
@@ -957,7 +942,7 @@
last_one = 1;
/* Is this the last request? */
- if (list_is_last(&req->list, &dep->request_list))
+ if (list_is_last(&req->list, &dep->pending_list))
last_one = 1;
dwc3_prepare_one_trb(dep, req, dma, length,
@@ -988,18 +973,18 @@
* new requests as we try to set the IOC bit only on the last request.
*/
if (start_new) {
- if (list_empty(&dep->req_queued))
+ if (list_empty(&dep->started_list))
dwc3_prepare_trbs(dep, start_new);
/* req points to the first request which will be sent */
- req = next_request(&dep->req_queued);
+ req = next_request(&dep->started_list);
} else {
dwc3_prepare_trbs(dep, start_new);
/*
* req points to the first request where HWO changed from 0 to 1
*/
- req = next_request(&dep->req_queued);
+ req = next_request(&dep->started_list);
}
if (!req) {
dep->flags |= DWC3_EP_PENDING_REQUEST;
@@ -1046,7 +1031,7 @@
{
u32 uf;
- if (list_empty(&dep->request_list)) {
+ if (list_empty(&dep->pending_list)) {
dwc3_trace(trace_dwc3_gadget,
"ISOC ep %s run out for requests",
dep->name);
@@ -1114,7 +1099,7 @@
if (ret)
return ret;
- list_add_tail(&req->list, &dep->request_list);
+ list_add_tail(&req->list, &dep->pending_list);
/*
* If there are no pending requests and the endpoint isn't already
@@ -1149,7 +1134,7 @@
* notion of current microframe.
*/
if (usb_endpoint_xfer_isoc(dep->endpoint.desc)) {
- if (list_empty(&dep->req_queued)) {
+ if (list_empty(&dep->started_list)) {
dwc3_stop_active_transfer(dwc, dep->number, true);
dep->flags = DWC3_EP_ENABLED;
}
@@ -1267,13 +1252,13 @@
spin_lock_irqsave(&dwc->lock, flags);
- list_for_each_entry(r, &dep->request_list, list) {
+ list_for_each_entry(r, &dep->pending_list, list) {
if (r == req)
break;
}
if (r != req) {
- list_for_each_entry(r, &dep->req_queued, list) {
+ list_for_each_entry(r, &dep->started_list, list) {
if (r == req)
break;
}
@@ -1313,10 +1298,10 @@
if (value) {
if (!protocol && ((dep->direction && dep->flags & DWC3_EP_BUSY) ||
- (!list_empty(&dep->req_queued) ||
- !list_empty(&dep->request_list)))) {
+ (!list_empty(&dep->started_list) ||
+ !list_empty(&dep->pending_list)))) {
dwc3_trace(trace_dwc3_gadget,
- "%s: pending request, cannot halt\n",
+ "%s: pending request, cannot halt",
dep->name);
return -EAGAIN;
}
@@ -1417,22 +1402,16 @@
return DWC3_DSTS_SOFFN(reg);
}
-static int dwc3_gadget_wakeup(struct usb_gadget *g)
+static int __dwc3_gadget_wakeup(struct dwc3 *dwc)
{
- struct dwc3 *dwc = gadget_to_dwc(g);
-
unsigned long timeout;
- unsigned long flags;
+ int ret;
u32 reg;
- int ret = 0;
-
u8 link_state;
u8 speed;
- spin_lock_irqsave(&dwc->lock, flags);
-
/*
* According to the Databook Remote wakeup request should
* be issued only when the device is in early suspend state.
@@ -1445,8 +1424,7 @@
if ((speed == DWC3_DSTS_SUPERSPEED) ||
(speed == DWC3_DSTS_SUPERSPEED_PLUS)) {
dwc3_trace(trace_dwc3_gadget, "no wakeup on SuperSpeed\n");
- ret = -EINVAL;
- goto out;
+ return -EINVAL;
}
link_state = DWC3_DSTS_USBLNKST(reg);
@@ -1459,14 +1437,13 @@
dwc3_trace(trace_dwc3_gadget,
"can't wakeup from '%s'\n",
dwc3_gadget_link_string(link_state));
- ret = -EINVAL;
- goto out;
+ return -EINVAL;
}
ret = dwc3_gadget_set_link_state(dwc, DWC3_LINK_STATE_RECOV);
if (ret < 0) {
dev_err(dwc->dev, "failed to put link in Recovery\n");
- goto out;
+ return ret;
}
/* Recent versions do this automatically */
@@ -1490,10 +1467,20 @@
if (DWC3_DSTS_USBLNKST(reg) != DWC3_LINK_STATE_U0) {
dev_err(dwc->dev, "failed to send remote wakeup\n");
- ret = -EINVAL;
+ return -EINVAL;
}
-out:
+ return 0;
+}
+
+static int dwc3_gadget_wakeup(struct usb_gadget *g)
+{
+ struct dwc3 *dwc = gadget_to_dwc(g);
+ unsigned long flags;
+ int ret;
+
+ spin_lock_irqsave(&dwc->lock, flags);
+ ret = __dwc3_gadget_wakeup(dwc);
spin_unlock_irqrestore(&dwc->lock, flags);
return ret;
@@ -1620,7 +1607,7 @@
irq = platform_get_irq(to_platform_device(dwc->dev), 0);
ret = request_threaded_irq(irq, dwc3_interrupt, dwc3_thread_interrupt,
- IRQF_SHARED, "dwc3", dwc);
+ IRQF_SHARED, "dwc3", dwc->ev_buf);
if (ret) {
dev_err(dwc->dev, "failed to request irq #%d --> %d\n",
irq, ret);
@@ -1682,6 +1669,17 @@
}
dwc3_writel(dwc->regs, DWC3_DCFG, reg);
+ /*
+ * We are telling dwc3 that we want to use DCFG.NUMP as ACK TP's NUMP
+ * field instead of letting dwc3 itself calculate that automatically.
+ *
+ * This way, we maximize the chances that we'll be able to get several
+ * bursts of data without going through any sort of endpoint throttling.
+ */
+ reg = dwc3_readl(dwc->regs, DWC3_GRXTHRCFG);
+ reg &= ~DWC3_GRXTHRCFG_PKTCNTSEL;
+ dwc3_writel(dwc->regs, DWC3_GRXTHRCFG, reg);
+
/* Start with SuperSpeed Default */
dwc3_gadget_ep0_desc.wMaxPacketSize = cpu_to_le16(512);
@@ -1720,7 +1718,7 @@
err1:
spin_unlock_irqrestore(&dwc->lock, flags);
- free_irq(irq, dwc);
+ free_irq(irq, dwc->ev_buf);
err0:
return ret;
@@ -1743,7 +1741,7 @@
spin_unlock_irqrestore(&dwc->lock, flags);
irq = platform_get_irq(to_platform_device(dwc->dev), 0);
- free_irq(irq, dwc);
+ free_irq(irq, dwc->ev_buf);
return 0;
}
@@ -1815,8 +1813,8 @@
dep->endpoint.caps.dir_in = !!direction;
dep->endpoint.caps.dir_out = !direction;
- INIT_LIST_HEAD(&dep->request_list);
- INIT_LIST_HEAD(&dep->req_queued);
+ INIT_LIST_HEAD(&dep->pending_list);
+ INIT_LIST_HEAD(&dep->started_list);
}
return 0;
@@ -1913,11 +1911,11 @@
* If there are still queued request
* then wait, do not issue either END
* or UPDATE TRANSFER, just attach next
- * request in request_list during
+ * request in pending_list during
* giveback.If any future queued request
* is successfully transferred then we
* will issue UPDATE TRANSFER for all
- * request in the request_list.
+ * request in the pending_list.
*/
dep->flags |= DWC3_EP_MISSED_ISOC;
} else {
@@ -1963,15 +1961,14 @@
int ret;
do {
- req = next_request(&dep->req_queued);
+ req = next_request(&dep->started_list);
if (WARN_ON_ONCE(!req))
return 1;
i = 0;
do {
- slot = req->start_slot + i;
- if ((slot == DWC3_TRB_NUM - 1) &&
- usb_endpoint_xfer_isoc(dep->endpoint.desc))
+ slot = req->first_trb_index + i;
+ if (slot == DWC3_TRB_NUM - 1)
slot++;
slot %= DWC3_TRB_NUM;
trb = &dep->trb_pool[slot];
@@ -1989,8 +1986,8 @@
} while (1);
if (usb_endpoint_xfer_isoc(dep->endpoint.desc) &&
- list_empty(&dep->req_queued)) {
- if (list_empty(&dep->request_list)) {
+ list_empty(&dep->started_list)) {
+ if (list_empty(&dep->pending_list)) {
/*
* If there is no entry in request list then do
* not issue END TRANSFER now. Just set PENDING
@@ -2039,7 +2036,7 @@
if (!(dep->flags & DWC3_EP_ENABLED))
continue;
- if (!list_empty(&dep->req_queued))
+ if (!list_empty(&dep->started_list))
return;
}
@@ -2686,14 +2683,13 @@
}
}
-static irqreturn_t dwc3_process_event_buf(struct dwc3 *dwc, u32 buf)
+static irqreturn_t dwc3_process_event_buf(struct dwc3_event_buffer *evt)
{
- struct dwc3_event_buffer *evt;
+ struct dwc3 *dwc = evt->dwc;
irqreturn_t ret = IRQ_NONE;
int left;
u32 reg;
- evt = dwc->ev_buffs[buf];
left = evt->count;
if (!(evt->flags & DWC3_EVENT_PENDING))
@@ -2718,7 +2714,7 @@
evt->lpos = (evt->lpos + 4) % DWC3_EVENT_BUFFERS_SIZE;
left -= 4;
- dwc3_writel(dwc->regs, DWC3_GEVNTCOUNT(buf), 4);
+ dwc3_writel(dwc->regs, DWC3_GEVNTCOUNT(0), 4);
}
evt->count = 0;
@@ -2726,39 +2722,34 @@
ret = IRQ_HANDLED;
/* Unmask interrupt */
- reg = dwc3_readl(dwc->regs, DWC3_GEVNTSIZ(buf));
+ reg = dwc3_readl(dwc->regs, DWC3_GEVNTSIZ(0));
reg &= ~DWC3_GEVNTSIZ_INTMASK;
- dwc3_writel(dwc->regs, DWC3_GEVNTSIZ(buf), reg);
+ dwc3_writel(dwc->regs, DWC3_GEVNTSIZ(0), reg);
return ret;
}
-static irqreturn_t dwc3_thread_interrupt(int irq, void *_dwc)
+static irqreturn_t dwc3_thread_interrupt(int irq, void *_evt)
{
- struct dwc3 *dwc = _dwc;
+ struct dwc3_event_buffer *evt = _evt;
+ struct dwc3 *dwc = evt->dwc;
unsigned long flags;
irqreturn_t ret = IRQ_NONE;
- int i;
spin_lock_irqsave(&dwc->lock, flags);
-
- for (i = 0; i < dwc->num_event_buffers; i++)
- ret |= dwc3_process_event_buf(dwc, i);
-
+ ret = dwc3_process_event_buf(evt);
spin_unlock_irqrestore(&dwc->lock, flags);
return ret;
}
-static irqreturn_t dwc3_check_event_buf(struct dwc3 *dwc, u32 buf)
+static irqreturn_t dwc3_check_event_buf(struct dwc3_event_buffer *evt)
{
- struct dwc3_event_buffer *evt;
+ struct dwc3 *dwc = evt->dwc;
u32 count;
u32 reg;
- evt = dwc->ev_buffs[buf];
-
- count = dwc3_readl(dwc->regs, DWC3_GEVNTCOUNT(buf));
+ count = dwc3_readl(dwc->regs, DWC3_GEVNTCOUNT(0));
count &= DWC3_GEVNTCOUNT_MASK;
if (!count)
return IRQ_NONE;
@@ -2767,28 +2758,18 @@
evt->flags |= DWC3_EVENT_PENDING;
/* Mask interrupt */
- reg = dwc3_readl(dwc->regs, DWC3_GEVNTSIZ(buf));
+ reg = dwc3_readl(dwc->regs, DWC3_GEVNTSIZ(0));
reg |= DWC3_GEVNTSIZ_INTMASK;
- dwc3_writel(dwc->regs, DWC3_GEVNTSIZ(buf), reg);
+ dwc3_writel(dwc->regs, DWC3_GEVNTSIZ(0), reg);
return IRQ_WAKE_THREAD;
}
-static irqreturn_t dwc3_interrupt(int irq, void *_dwc)
+static irqreturn_t dwc3_interrupt(int irq, void *_evt)
{
- struct dwc3 *dwc = _dwc;
- int i;
- irqreturn_t ret = IRQ_NONE;
+ struct dwc3_event_buffer *evt = _evt;
- for (i = 0; i < dwc->num_event_buffers; i++) {
- irqreturn_t status;
-
- status = dwc3_check_event_buf(dwc, i);
- if (status == IRQ_WAKE_THREAD)
- ret = status;
- }
-
- return ret;
+ return dwc3_check_event_buf(evt);
}
/**
diff --git a/drivers/usb/dwc3/gadget.h b/drivers/usb/dwc3/gadget.h
index 18ae3ea..f21c0fc 100644
--- a/drivers/usb/dwc3/gadget.h
+++ b/drivers/usb/dwc3/gadget.h
@@ -68,12 +68,12 @@
return list_first_entry(list, struct dwc3_request, list);
}
-static inline void dwc3_gadget_move_request_queued(struct dwc3_request *req)
+static inline void dwc3_gadget_move_started_request(struct dwc3_request *req)
{
struct dwc3_ep *dep = req->dep;
- req->queued = true;
- list_move_tail(&req->list, &dep->req_queued);
+ req->started = true;
+ list_move_tail(&req->list, &dep->started_list);
}
void dwc3_gadget_giveback(struct dwc3_ep *dep, struct dwc3_request *req,
diff --git a/drivers/usb/dwc3/platform_data.h b/drivers/usb/dwc3/platform_data.h
index 2bb4d3a..8826cca 100644
--- a/drivers/usb/dwc3/platform_data.h
+++ b/drivers/usb/dwc3/platform_data.h
@@ -23,7 +23,6 @@
struct dwc3_platform_data {
enum usb_device_speed maximum_speed;
enum usb_dr_mode dr_mode;
- bool tx_fifo_resize;
bool usb3_lpm_capable;
unsigned is_utmi_l1_suspend:1;
@@ -43,6 +42,7 @@
unsigned dis_u3_susphy_quirk:1;
unsigned dis_u2_susphy_quirk:1;
unsigned dis_enblslpm_quirk:1;
+ unsigned dis_rxdet_inp3_quirk:1;
unsigned tx_de_emphasis_quirk:1;
unsigned tx_de_emphasis:2;
diff --git a/drivers/usb/gadget/Kconfig b/drivers/usb/gadget/Kconfig
index af5d922..2057add 100644
--- a/drivers/usb/gadget/Kconfig
+++ b/drivers/usb/gadget/Kconfig
@@ -15,6 +15,7 @@
menuconfig USB_GADGET
tristate "USB Gadget Support"
+ select USB_COMMON
select NLS
help
USB is a master/slave protocol, organized with one master
diff --git a/drivers/usb/gadget/composite.c b/drivers/usb/gadget/composite.c
index 524e233..d67de0d 100644
--- a/drivers/usb/gadget/composite.c
+++ b/drivers/usb/gadget/composite.c
@@ -66,20 +66,36 @@
{
struct usb_descriptor_header **descriptors;
+ /*
+ * NOTE: we try to help gadget drivers which might not be setting
+ * max_speed appropriately.
+ */
+
switch (speed) {
case USB_SPEED_SUPER_PLUS:
descriptors = f->ssp_descriptors;
- break;
+ if (descriptors)
+ break;
+ /* FALLTHROUGH */
case USB_SPEED_SUPER:
descriptors = f->ss_descriptors;
- break;
+ if (descriptors)
+ break;
+ /* FALLTHROUGH */
case USB_SPEED_HIGH:
descriptors = f->hs_descriptors;
- break;
+ if (descriptors)
+ break;
+ /* FALLTHROUGH */
default:
descriptors = f->fs_descriptors;
}
+ /*
+ * if we can't find any descriptors at all, then this gadget deserves to
+ * Oops with a NULL pointer dereference
+ */
+
return descriptors;
}
diff --git a/drivers/usb/gadget/function/f_fs.c b/drivers/usb/gadget/function/f_fs.c
index 15b648c..73515d5 100644
--- a/drivers/usb/gadget/function/f_fs.c
+++ b/drivers/usb/gadget/function/f_fs.c
@@ -651,7 +651,7 @@
if (io_data->read && ret > 0) {
use_mm(io_data->mm);
ret = copy_to_iter(io_data->buf, ret, &io_data->data);
- if (iov_iter_count(&io_data->data))
+ if (ret != io_data->req->actual && iov_iter_count(&io_data->data))
ret = -EFAULT;
unuse_mm(io_data->mm);
}
diff --git a/drivers/usb/gadget/function/f_mass_storage.c b/drivers/usb/gadget/function/f_mass_storage.c
index acf210f..5c6d4d7 100644
--- a/drivers/usb/gadget/function/f_mass_storage.c
+++ b/drivers/usb/gadget/function/f_mass_storage.c
@@ -2977,25 +2977,6 @@
}
EXPORT_SYMBOL_GPL(fsg_common_set_inquiry_string);
-int fsg_common_run_thread(struct fsg_common *common)
-{
- common->state = FSG_STATE_IDLE;
- /* Tell the thread to start working */
- common->thread_task =
- kthread_create(fsg_main_thread, common, "file-storage");
- if (IS_ERR(common->thread_task)) {
- common->state = FSG_STATE_TERMINATED;
- return PTR_ERR(common->thread_task);
- }
-
- DBG(common, "I/O thread pid: %d\n", task_pid_nr(common->thread_task));
-
- wake_up_process(common->thread_task);
-
- return 0;
-}
-EXPORT_SYMBOL_GPL(fsg_common_run_thread);
-
static void fsg_common_release(struct kref *ref)
{
struct fsg_common *common = container_of(ref, struct fsg_common, ref);
@@ -3005,6 +2986,7 @@
if (common->state != FSG_STATE_TERMINATED) {
raise_exception(common, FSG_STATE_EXIT);
wait_for_completion(&common->thread_notifier);
+ common->thread_task = NULL;
}
for (i = 0; i < ARRAY_SIZE(common->luns); ++i) {
@@ -3050,9 +3032,21 @@
if (ret)
return ret;
fsg_common_set_inquiry_string(fsg->common, NULL, NULL);
- ret = fsg_common_run_thread(fsg->common);
- if (ret)
+ }
+
+ if (!common->thread_task) {
+ common->state = FSG_STATE_IDLE;
+ common->thread_task =
+ kthread_create(fsg_main_thread, common, "file-storage");
+ if (IS_ERR(common->thread_task)) {
+ int ret = PTR_ERR(common->thread_task);
+ common->thread_task = NULL;
+ common->state = FSG_STATE_TERMINATED;
return ret;
+ }
+ DBG(common, "I/O thread pid: %d\n",
+ task_pid_nr(common->thread_task));
+ wake_up_process(common->thread_task);
}
fsg->gadget = gadget;
diff --git a/drivers/usb/gadget/function/f_mass_storage.h b/drivers/usb/gadget/function/f_mass_storage.h
index 445df67..b6a9918 100644
--- a/drivers/usb/gadget/function/f_mass_storage.h
+++ b/drivers/usb/gadget/function/f_mass_storage.h
@@ -153,8 +153,6 @@
void fsg_common_set_inquiry_string(struct fsg_common *common, const char *vn,
const char *pn);
-int fsg_common_run_thread(struct fsg_common *common);
-
void fsg_config_from_params(struct fsg_config *cfg,
const struct fsg_module_parameters *params,
unsigned int fsg_num_buffers);
diff --git a/drivers/usb/gadget/function/u_serial.c b/drivers/usb/gadget/function/u_serial.c
index 6af145f..3580f19 100644
--- a/drivers/usb/gadget/function/u_serial.c
+++ b/drivers/usb/gadget/function/u_serial.c
@@ -512,7 +512,7 @@
req = list_first_entry(queue, struct usb_request, list);
/* leave data queued if tty was rx throttled */
- if (tty && test_bit(TTY_THROTTLED, &tty->flags))
+ if (tty && tty_throttled(tty))
break;
switch (req->status) {
@@ -579,7 +579,7 @@
* from starving ... but it's not clear that case ever happens.
*/
if (!list_empty(queue) && tty) {
- if (!test_bit(TTY_THROTTLED, &tty->flags)) {
+ if (!tty_throttled(tty)) {
if (do_push)
tasklet_schedule(&port->push);
else
diff --git a/drivers/usb/gadget/legacy/acm_ms.c b/drivers/usb/gadget/legacy/acm_ms.c
index c16089e..c39de65 100644
--- a/drivers/usb/gadget/legacy/acm_ms.c
+++ b/drivers/usb/gadget/legacy/acm_ms.c
@@ -133,10 +133,6 @@
if (status < 0)
goto put_msg;
- status = fsg_common_run_thread(opts->common);
- if (status)
- goto remove_acm;
-
status = usb_add_function(c, f_msg);
if (status)
goto remove_acm;
diff --git a/drivers/usb/gadget/legacy/mass_storage.c b/drivers/usb/gadget/legacy/mass_storage.c
index e61af53..125974f 100644
--- a/drivers/usb/gadget/legacy/mass_storage.c
+++ b/drivers/usb/gadget/legacy/mass_storage.c
@@ -132,10 +132,6 @@
if (IS_ERR(f_msg))
return PTR_ERR(f_msg);
- ret = fsg_common_run_thread(opts->common);
- if (ret)
- goto put_func;
-
ret = usb_add_function(c, f_msg);
if (ret)
goto put_func;
diff --git a/drivers/usb/gadget/legacy/multi.c b/drivers/usb/gadget/legacy/multi.c
index 229d704..a70a406 100644
--- a/drivers/usb/gadget/legacy/multi.c
+++ b/drivers/usb/gadget/legacy/multi.c
@@ -137,7 +137,6 @@
static int rndis_do_config(struct usb_configuration *c)
{
- struct fsg_opts *fsg_opts;
int ret;
if (gadget_is_otg(c->cdev->gadget)) {
@@ -169,11 +168,6 @@
goto err_fsg;
}
- fsg_opts = fsg_opts_from_func_inst(fi_msg);
- ret = fsg_common_run_thread(fsg_opts->common);
- if (ret)
- goto err_run;
-
ret = usb_add_function(c, f_msg_rndis);
if (ret)
goto err_run;
@@ -225,7 +219,6 @@
static int cdc_do_config(struct usb_configuration *c)
{
- struct fsg_opts *fsg_opts;
int ret;
if (gadget_is_otg(c->cdev->gadget)) {
@@ -258,11 +251,6 @@
goto err_fsg;
}
- fsg_opts = fsg_opts_from_func_inst(fi_msg);
- ret = fsg_common_run_thread(fsg_opts->common);
- if (ret)
- goto err_run;
-
ret = usb_add_function(c, f_msg_multi);
if (ret)
goto err_run;
diff --git a/drivers/usb/gadget/legacy/nokia.c b/drivers/usb/gadget/legacy/nokia.c
index 0997504..b1e535f 100644
--- a/drivers/usb/gadget/legacy/nokia.c
+++ b/drivers/usb/gadget/legacy/nokia.c
@@ -152,7 +152,6 @@
struct usb_function *f_ecm;
struct usb_function *f_obex2 = NULL;
struct usb_function *f_msg;
- struct fsg_opts *fsg_opts;
int status = 0;
int obex1_stat = -1;
int obex2_stat = -1;
@@ -222,12 +221,6 @@
goto err_ecm;
}
- fsg_opts = fsg_opts_from_func_inst(fi_msg);
-
- status = fsg_common_run_thread(fsg_opts->common);
- if (status)
- goto err_msg;
-
status = usb_add_function(c, f_msg);
if (status)
goto err_msg;
diff --git a/drivers/usb/gadget/udc/at91_udc.c b/drivers/usb/gadget/udc/at91_udc.c
index d0d1894..8bc7841 100644
--- a/drivers/usb/gadget/udc/at91_udc.c
+++ b/drivers/usb/gadget/udc/at91_udc.c
@@ -1726,10 +1726,7 @@
udc->matrix = syscon_regmap_lookup_by_phandle(udc->pdev->dev.of_node,
"atmel,matrix");
- if (IS_ERR(udc->matrix))
- return PTR_ERR(udc->matrix);
-
- return 0;
+ return PTR_ERR_OR_ZERO(udc->matrix);
}
static void at91sam9261_udc_pullup(struct at91_udc *udc, int is_on)
diff --git a/drivers/usb/gadget/udc/pch_udc.c b/drivers/usb/gadget/udc/pch_udc.c
index 9571ef5..ebc51ec 100644
--- a/drivers/usb/gadget/udc/pch_udc.c
+++ b/drivers/usb/gadget/udc/pch_udc.c
@@ -325,11 +325,8 @@
* @pdev: reference to the PCI device
* @ep: array of endpoints
* @lock: protects all state
- * @active: enabled the PCI device
* @stall: stall requested
* @prot_stall: protcol stall requested
- * @irq_registered: irq registered with system
- * @mem_region: device memory mapped
* @registered: driver registered with system
* @suspended: driver in suspended state
* @connected: gadget driver associated
@@ -339,12 +336,8 @@
* @data_requests: DMA pool for data requests
* @stp_requests: DMA pool for setup requests
* @dma_addr: DMA pool for received
- * @ep0out_buf: Buffer for DMA
* @setup_data: Received setup data
- * @phys_addr: of device memory
* @base_addr: for mapped device memory
- * @bar: Indicates which PCI BAR for USB regs
- * @irq: IRQ line for the device
* @cfg_data: current cfg, intf, and alt in use
* @vbus_gpio: GPIO informaton for detecting VBUS
*/
@@ -354,11 +347,9 @@
struct pci_dev *pdev;
struct pch_udc_ep ep[PCH_UDC_EP_NUM];
spinlock_t lock; /* protects all state */
- unsigned active:1,
+ unsigned
stall:1,
prot_stall:1,
- irq_registered:1,
- mem_region:1,
suspended:1,
connected:1,
vbus_session:1,
@@ -367,12 +358,8 @@
struct pci_pool *data_requests;
struct pci_pool *stp_requests;
dma_addr_t dma_addr;
- void *ep0out_buf;
struct usb_ctrlrequest setup_data;
- unsigned long phys_addr;
void __iomem *base_addr;
- unsigned bar;
- unsigned irq;
struct pch_udc_cfg_data cfg_data;
struct pch_vbus_gpio_data vbus_gpio;
};
@@ -380,8 +367,10 @@
#define PCH_UDC_PCI_BAR_QUARK_X1000 0
#define PCH_UDC_PCI_BAR 1
-#define PCI_DEVICE_ID_INTEL_EG20T_UDC 0x8808
+
#define PCI_DEVICE_ID_INTEL_QUARK_X1000_UDC 0x0939
+#define PCI_DEVICE_ID_INTEL_EG20T_UDC 0x8808
+
#define PCI_VENDOR_ID_ROHM 0x10DB
#define PCI_DEVICE_ID_ML7213_IOH_UDC 0x801D
#define PCI_DEVICE_ID_ML7831_IOH_UDC 0x8808
@@ -1732,14 +1721,12 @@
static int pch_udc_pcd_ep_disable(struct usb_ep *usbep)
{
struct pch_udc_ep *ep;
- struct pch_udc_dev *dev;
unsigned long iflags;
if (!usbep)
return -EINVAL;
ep = container_of(usbep, struct pch_udc_ep, ep);
- dev = ep->dev;
if ((usbep->name == ep0_string) || !ep->ep.desc)
return -EINVAL;
@@ -1770,12 +1757,10 @@
struct pch_udc_request *req;
struct pch_udc_ep *ep;
struct pch_udc_data_dma_desc *dma_desc;
- struct pch_udc_dev *dev;
if (!usbep)
return NULL;
ep = container_of(usbep, struct pch_udc_ep, ep);
- dev = ep->dev;
req = kzalloc(sizeof *req, gfp);
if (!req)
return NULL;
@@ -1948,12 +1933,10 @@
{
struct pch_udc_ep *ep;
struct pch_udc_request *req;
- struct pch_udc_dev *dev;
unsigned long flags;
int ret = -EINVAL;
ep = container_of(usbep, struct pch_udc_ep, ep);
- dev = ep->dev;
if (!usbep || !usbreq || (!ep->ep.desc && ep->num))
return ret;
req = container_of(usbreq, struct pch_udc_request, req);
@@ -1985,14 +1968,12 @@
static int pch_udc_pcd_set_halt(struct usb_ep *usbep, int halt)
{
struct pch_udc_ep *ep;
- struct pch_udc_dev *dev;
unsigned long iflags;
int ret;
if (!usbep)
return -EINVAL;
ep = container_of(usbep, struct pch_udc_ep, ep);
- dev = ep->dev;
if (!ep->ep.desc && !ep->num)
return -EINVAL;
if (!ep->dev->driver || (ep->dev->gadget.speed == USB_SPEED_UNKNOWN))
@@ -2030,14 +2011,12 @@
static int pch_udc_pcd_set_wedge(struct usb_ep *usbep)
{
struct pch_udc_ep *ep;
- struct pch_udc_dev *dev;
unsigned long iflags;
int ret;
if (!usbep)
return -EINVAL;
ep = container_of(usbep, struct pch_udc_ep, ep);
- dev = ep->dev;
if (!ep->ep.desc && !ep->num)
return -EINVAL;
if (!ep->dev->driver || (ep->dev->gadget.speed == USB_SPEED_UNKNOWN))
@@ -2647,7 +2626,7 @@
static void pch_udc_svc_intf_interrupt(struct pch_udc_dev *dev)
{
u32 reg, dev_stat = 0;
- int i, ret;
+ int i;
dev_stat = pch_udc_read_device_status(dev);
dev->cfg_data.cur_intf = (dev_stat & UDC_DEVSTS_INTF_MASK) >>
@@ -2676,7 +2655,7 @@
}
dev->stall = 0;
spin_lock(&dev->lock);
- ret = dev->driver->setup(&dev->gadget, &dev->setup_data);
+ dev->driver->setup(&dev->gadget, &dev->setup_data);
spin_unlock(&dev->lock);
}
@@ -2687,7 +2666,7 @@
*/
static void pch_udc_svc_cfg_interrupt(struct pch_udc_dev *dev)
{
- int i, ret;
+ int i;
u32 reg, dev_stat = 0;
dev_stat = pch_udc_read_device_status(dev);
@@ -2713,7 +2692,7 @@
/* call gadget zero with setup data received */
spin_lock(&dev->lock);
- ret = dev->driver->setup(&dev->gadget, &dev->setup_data);
+ dev->driver->setup(&dev->gadget, &dev->setup_data);
spin_unlock(&dev->lock);
}
@@ -2856,17 +2835,6 @@
}
/**
- * gadget_release() - Free the gadget driver private data
- * @pdev reference to struct pci_dev
- */
-static void gadget_release(struct device *pdev)
-{
- struct pch_udc_dev *dev = dev_get_drvdata(pdev);
-
- kfree(dev);
-}
-
-/**
* pch_udc_pcd_reinit() - This API initializes the endpoint structures
* @dev: Reference to the driver structure
*/
@@ -2949,6 +2917,7 @@
{
struct pch_udc_stp_dma_desc *td_stp;
struct pch_udc_data_dma_desc *td_data;
+ void *ep0out_buf;
/* DMA setup */
dev->data_requests = pci_pool_create("data_requests", dev->pdev,
@@ -2991,10 +2960,11 @@
dev->ep[UDC_EP0IN_IDX].td_data = NULL;
dev->ep[UDC_EP0IN_IDX].td_data_phys = 0;
- dev->ep0out_buf = kzalloc(UDC_EP0OUT_BUFF_SIZE * 4, GFP_KERNEL);
- if (!dev->ep0out_buf)
+ ep0out_buf = devm_kzalloc(&dev->pdev->dev, UDC_EP0OUT_BUFF_SIZE * 4,
+ GFP_KERNEL);
+ if (!ep0out_buf)
return -ENOMEM;
- dev->dma_addr = dma_map_single(&dev->pdev->dev, dev->ep0out_buf,
+ dev->dma_addr = dma_map_single(&dev->pdev->dev, ep0out_buf,
UDC_EP0OUT_BUFF_SIZE * 4,
DMA_FROM_DEVICE);
return 0;
@@ -3078,129 +3048,80 @@
if (dev->dma_addr)
dma_unmap_single(&dev->pdev->dev, dev->dma_addr,
UDC_EP0OUT_BUFF_SIZE * 4, DMA_FROM_DEVICE);
- kfree(dev->ep0out_buf);
pch_vbus_gpio_free(dev);
pch_udc_exit(dev);
-
- if (dev->irq_registered)
- free_irq(pdev->irq, dev);
- if (dev->base_addr)
- iounmap(dev->base_addr);
- if (dev->mem_region)
- release_mem_region(dev->phys_addr,
- pci_resource_len(pdev, dev->bar));
- if (dev->active)
- pci_disable_device(pdev);
- kfree(dev);
}
-#ifdef CONFIG_PM
-static int pch_udc_suspend(struct pci_dev *pdev, pm_message_t state)
+#ifdef CONFIG_PM_SLEEP
+static int pch_udc_suspend(struct device *d)
{
+ struct pci_dev *pdev = to_pci_dev(d);
struct pch_udc_dev *dev = pci_get_drvdata(pdev);
pch_udc_disable_interrupts(dev, UDC_DEVINT_MSK);
pch_udc_disable_ep_interrupts(dev, UDC_EPINT_MSK_DISABLE_ALL);
- pci_disable_device(pdev);
- pci_enable_wake(pdev, PCI_D3hot, 0);
-
- if (pci_save_state(pdev)) {
- dev_err(&pdev->dev,
- "%s: could not save PCI config state\n", __func__);
- return -ENOMEM;
- }
- pci_set_power_state(pdev, pci_choose_state(pdev, state));
return 0;
}
-static int pch_udc_resume(struct pci_dev *pdev)
+static int pch_udc_resume(struct device *d)
{
- int ret;
-
- pci_set_power_state(pdev, PCI_D0);
- pci_restore_state(pdev);
- ret = pci_enable_device(pdev);
- if (ret) {
- dev_err(&pdev->dev, "%s: pci_enable_device failed\n", __func__);
- return ret;
- }
- pci_enable_wake(pdev, PCI_D3hot, 0);
return 0;
}
+
+static SIMPLE_DEV_PM_OPS(pch_udc_pm, pch_udc_suspend, pch_udc_resume);
+#define PCH_UDC_PM_OPS (&pch_udc_pm)
#else
-#define pch_udc_suspend NULL
-#define pch_udc_resume NULL
-#endif /* CONFIG_PM */
+#define PCH_UDC_PM_OPS NULL
+#endif /* CONFIG_PM_SLEEP */
static int pch_udc_probe(struct pci_dev *pdev,
const struct pci_device_id *id)
{
- unsigned long resource;
- unsigned long len;
+ int bar;
int retval;
struct pch_udc_dev *dev;
/* init */
- dev = kzalloc(sizeof *dev, GFP_KERNEL);
- if (!dev) {
- pr_err("%s: no memory for device structure\n", __func__);
+ dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
+ if (!dev)
return -ENOMEM;
- }
+
/* pci setup */
- if (pci_enable_device(pdev) < 0) {
- kfree(dev);
- pr_err("%s: pci_enable_device failed\n", __func__);
- return -ENODEV;
- }
- dev->active = 1;
+ retval = pcim_enable_device(pdev);
+ if (retval)
+ return retval;
+
pci_set_drvdata(pdev, dev);
/* Determine BAR based on PCI ID */
if (id->device == PCI_DEVICE_ID_INTEL_QUARK_X1000_UDC)
- dev->bar = PCH_UDC_PCI_BAR_QUARK_X1000;
+ bar = PCH_UDC_PCI_BAR_QUARK_X1000;
else
- dev->bar = PCH_UDC_PCI_BAR;
+ bar = PCH_UDC_PCI_BAR;
/* PCI resource allocation */
- resource = pci_resource_start(pdev, dev->bar);
- len = pci_resource_len(pdev, dev->bar);
+ retval = pcim_iomap_regions(pdev, 1 << bar, pci_name(pdev));
+ if (retval)
+ return retval;
- if (!request_mem_region(resource, len, KBUILD_MODNAME)) {
- dev_err(&pdev->dev, "%s: pci device used already\n", __func__);
- retval = -EBUSY;
- goto finished;
- }
- dev->phys_addr = resource;
- dev->mem_region = 1;
+ dev->base_addr = pcim_iomap_table(pdev)[bar];
- dev->base_addr = ioremap_nocache(resource, len);
- if (!dev->base_addr) {
- pr_err("%s: device memory cannot be mapped\n", __func__);
- retval = -ENOMEM;
- goto finished;
- }
- if (!pdev->irq) {
- dev_err(&pdev->dev, "%s: irq not set\n", __func__);
- retval = -ENODEV;
- goto finished;
- }
/* initialize the hardware */
- if (pch_udc_pcd_init(dev)) {
- retval = -ENODEV;
- goto finished;
- }
- if (request_irq(pdev->irq, pch_udc_isr, IRQF_SHARED, KBUILD_MODNAME,
- dev)) {
+ if (pch_udc_pcd_init(dev))
+ return -ENODEV;
+
+ pci_enable_msi(pdev);
+
+ retval = devm_request_irq(&pdev->dev, pdev->irq, pch_udc_isr,
+ IRQF_SHARED, KBUILD_MODNAME, dev);
+ if (retval) {
dev_err(&pdev->dev, "%s: request_irq(%d) fail\n", __func__,
pdev->irq);
- retval = -ENODEV;
goto finished;
}
- dev->irq = pdev->irq;
- dev->irq_registered = 1;
pci_set_master(pdev);
pci_try_set_mwi(pdev);
@@ -3219,8 +3140,7 @@
/* Put the device in disconnected state till a driver is bound */
pch_udc_set_disconnect(dev);
- retval = usb_add_gadget_udc_release(&pdev->dev, &dev->gadget,
- gadget_release);
+ retval = usb_add_gadget_udc(&pdev->dev, &dev->gadget);
if (retval)
goto finished;
return 0;
@@ -3262,9 +3182,10 @@
.id_table = pch_udc_pcidev_id,
.probe = pch_udc_probe,
.remove = pch_udc_remove,
- .suspend = pch_udc_suspend,
- .resume = pch_udc_resume,
.shutdown = pch_udc_shutdown,
+ .driver = {
+ .pm = PCH_UDC_PM_OPS,
+ },
};
module_pci_driver(pch_udc_driver);
diff --git a/drivers/usb/gadget/udc/r8a66597-udc.c b/drivers/usb/gadget/udc/r8a66597-udc.c
index baa0609..8b300e6 100644
--- a/drivers/usb/gadget/udc/r8a66597-udc.c
+++ b/drivers/usb/gadget/udc/r8a66597-udc.c
@@ -296,7 +296,7 @@
} while ((tmp & mask) != loop);
}
-static inline void pipe_change(struct r8a66597 *r8a66597, u16 pipenum)
+static void pipe_change(struct r8a66597 *r8a66597, u16 pipenum)
{
struct r8a66597_ep *ep = r8a66597->pipenum2ep[pipenum];
diff --git a/drivers/usb/gadget/udc/udc-core.c b/drivers/usb/gadget/udc/udc-core.c
index e4e70e1..6e8300d 100644
--- a/drivers/usb/gadget/udc/udc-core.c
+++ b/drivers/usb/gadget/udc/udc-core.c
@@ -61,11 +61,9 @@
#ifdef CONFIG_HAS_DMA
-int usb_gadget_map_request(struct usb_gadget *gadget,
+int usb_gadget_map_request_by_dev(struct device *dev,
struct usb_request *req, int is_in)
{
- struct device *dev = gadget->dev.parent;
-
if (req->length == 0)
return 0;
@@ -75,7 +73,7 @@
mapped = dma_map_sg(dev, req->sg, req->num_sgs,
is_in ? DMA_TO_DEVICE : DMA_FROM_DEVICE);
if (mapped == 0) {
- dev_err(&gadget->dev, "failed to map SGs\n");
+ dev_err(dev, "failed to map SGs\n");
return -EFAULT;
}
@@ -92,24 +90,38 @@
return 0;
}
+EXPORT_SYMBOL_GPL(usb_gadget_map_request_by_dev);
+
+int usb_gadget_map_request(struct usb_gadget *gadget,
+ struct usb_request *req, int is_in)
+{
+ return usb_gadget_map_request_by_dev(gadget->dev.parent, req, is_in);
+}
EXPORT_SYMBOL_GPL(usb_gadget_map_request);
-void usb_gadget_unmap_request(struct usb_gadget *gadget,
+void usb_gadget_unmap_request_by_dev(struct device *dev,
struct usb_request *req, int is_in)
{
if (req->length == 0)
return;
if (req->num_mapped_sgs) {
- dma_unmap_sg(gadget->dev.parent, req->sg, req->num_mapped_sgs,
+ dma_unmap_sg(dev, req->sg, req->num_mapped_sgs,
is_in ? DMA_TO_DEVICE : DMA_FROM_DEVICE);
req->num_mapped_sgs = 0;
} else {
- dma_unmap_single(gadget->dev.parent, req->dma, req->length,
+ dma_unmap_single(dev, req->dma, req->length,
is_in ? DMA_TO_DEVICE : DMA_FROM_DEVICE);
}
}
+EXPORT_SYMBOL_GPL(usb_gadget_unmap_request_by_dev);
+
+void usb_gadget_unmap_request(struct usb_gadget *gadget,
+ struct usb_request *req, int is_in)
+{
+ usb_gadget_unmap_request_by_dev(gadget->dev.parent, req, is_in);
+}
EXPORT_SYMBOL_GPL(usb_gadget_unmap_request);
#endif /* CONFIG_HAS_DMA */
diff --git a/drivers/usb/host/Kconfig b/drivers/usb/host/Kconfig
index 6acac62..d8f5674 100644
--- a/drivers/usb/host/Kconfig
+++ b/drivers/usb/host/Kconfig
@@ -35,6 +35,7 @@
config USB_XHCI_PLATFORM
tristate "Generic xHCI driver for a platform device"
+ select USB_XHCI_RCAR if ARCH_RENESAS
---help---
Adds an xHCI host driver for a generic platform device, which
provides a memory space and an irq.
@@ -63,7 +64,7 @@
config USB_XHCI_RCAR
tristate "xHCI support for Renesas R-Car SoCs"
- select USB_XHCI_PLATFORM
+ depends on USB_XHCI_PLATFORM
depends on ARCH_RENESAS || COMPILE_TEST
---help---
Say 'Y' to enable the support for the xHCI host controller
diff --git a/drivers/usb/host/bcma-hcd.c b/drivers/usb/host/bcma-hcd.c
index 963e2d0..172ef17 100644
--- a/drivers/usb/host/bcma-hcd.c
+++ b/drivers/usb/host/bcma-hcd.c
@@ -352,10 +352,8 @@
usb_dev->core = core;
if (core->dev.of_node)
- usb_dev->gpio_desc = devm_get_gpiod_from_child(&core->dev, "vcc",
- &core->dev.of_node->fwnode);
- if (!IS_ERR_OR_NULL(usb_dev->gpio_desc))
- gpiod_direction_output(usb_dev->gpio_desc, 1);
+ usb_dev->gpio_desc = devm_gpiod_get(&core->dev, "vcc",
+ GPIOD_OUT_HIGH);
switch (core->id.id) {
case BCMA_CORE_USB20_HOST:
diff --git a/drivers/usb/host/ehci-dbg.c b/drivers/usb/host/ehci-dbg.c
index 79d12b2..1a2614a 100644
--- a/drivers/usb/host/ehci-dbg.c
+++ b/drivers/usb/host/ehci-dbg.c
@@ -52,13 +52,6 @@
ehci_dbg(ehci, "%s portroute %s\n", label, buf);
}
}
-#else
-
-static inline void dbg_hcs_params(struct ehci_hcd *ehci, char *label) {}
-
-#endif
-
-#ifdef CONFIG_DYNAMIC_DEBUG
/*
* check the values in the HCCPARAMS register
@@ -92,13 +85,6 @@
" 32 periodic list" : "");
}
}
-#else
-
-static inline void dbg_hcc_params(struct ehci_hcd *ehci, char *label) {}
-
-#endif
-
-#ifdef CONFIG_DYNAMIC_DEBUG
static void __maybe_unused
dbg_qtd(const char *label, struct ehci_hcd *ehci, struct ehci_qtd *qtd)
@@ -281,37 +267,6 @@
(status & PORT_CONNECT) ? " CONNECT" : "");
}
-#else
-static inline void __maybe_unused
-dbg_qh(char *label, struct ehci_hcd *ehci, struct ehci_qh *qh)
-{}
-
-static inline int __maybe_unused
-dbg_status_buf(char *buf, unsigned len, const char *label, u32 status)
-{
- return 0;
-}
-
-static inline int __maybe_unused
-dbg_command_buf(char *buf, unsigned len, const char *label, u32 command)
-{
- return 0;
-}
-
-static inline int __maybe_unused
-dbg_intr_buf(char *buf, unsigned len, const char *label, u32 enable)
-{
- return 0;
-}
-
-static inline int __maybe_unused
-dbg_port_buf(char *buf, unsigned len, const char *label, int port, u32 status)
-{
- return 0;
-}
-
-#endif /* CONFIG_DYNAMIC_DEBUG */
-
static inline void
dbg_status(struct ehci_hcd *ehci, const char *label, u32 status)
{
@@ -341,13 +296,6 @@
/*-------------------------------------------------------------------------*/
-#ifndef CONFIG_DYNAMIC_DEBUG
-
-static inline void create_debug_files(struct ehci_hcd *bus) { }
-static inline void remove_debug_files(struct ehci_hcd *bus) { }
-
-#else
-
/* troubleshooting help: expose state in debugfs */
static int debug_async_open(struct inode *, struct file *);
@@ -1120,4 +1068,38 @@
debugfs_remove_recursive(ehci->debug_dir);
}
+#else /* CONFIG_DYNAMIC_DEBUG */
+
+static inline void dbg_hcs_params(struct ehci_hcd *ehci, char *label) { }
+static inline void dbg_hcc_params(struct ehci_hcd *ehci, char *label) { }
+
+static inline void __maybe_unused dbg_qh(const char *label,
+ struct ehci_hcd *ehci, struct ehci_qh *qh) { }
+
+static inline int __maybe_unused dbg_status_buf(const char *buf,
+ unsigned int len, const char *label, u32 status)
+{ return 0; }
+
+static inline int __maybe_unused dbg_command_buf(const char *buf,
+ unsigned int len, const char *label, u32 command)
+{ return 0; }
+
+static inline int __maybe_unused dbg_intr_buf(const char *buf,
+ unsigned int len, const char *label, u32 enable)
+{ return 0; }
+
+static inline int __maybe_unused dbg_port_buf(char *buf,
+ unsigned int len, const char *label, int port, u32 status)
+{ return 0; }
+
+static inline void dbg_status(struct ehci_hcd *ehci, const char *label,
+ u32 status) { }
+static inline void dbg_cmd(struct ehci_hcd *ehci, const char *label,
+ u32 command) { }
+static inline void dbg_port(struct ehci_hcd *ehci, const char *label,
+ int port, u32 status) { }
+
+static inline void create_debug_files(struct ehci_hcd *bus) { }
+static inline void remove_debug_files(struct ehci_hcd *bus) { }
+
#endif /* CONFIG_DYNAMIC_DEBUG */
diff --git a/drivers/usb/host/ehci-exynos.c b/drivers/usb/host/ehci-exynos.c
index df538fd..42e5b66 100644
--- a/drivers/usb/host/ehci-exynos.c
+++ b/drivers/usb/host/ehci-exynos.c
@@ -321,7 +321,7 @@
.of_match_table = of_match_ptr(exynos_ehci_match),
}
};
-static const struct ehci_driver_overrides exynos_overrides __initdata = {
+static const struct ehci_driver_overrides exynos_overrides __initconst = {
.extra_priv_size = sizeof(struct exynos_ehci_hcd),
};
diff --git a/drivers/usb/host/ehci-msm.c b/drivers/usb/host/ehci-msm.c
index 3e226ef..d3afc89 100644
--- a/drivers/usb/host/ehci-msm.c
+++ b/drivers/usb/host/ehci-msm.c
@@ -229,7 +229,7 @@
},
};
-static const struct ehci_driver_overrides msm_overrides __initdata = {
+static const struct ehci_driver_overrides msm_overrides __initconst = {
.reset = ehci_msm_reset,
};
diff --git a/drivers/usb/host/ehci-omap.c b/drivers/usb/host/ehci-omap.c
index a24720b..94ea9ff 100644
--- a/drivers/usb/host/ehci-omap.c
+++ b/drivers/usb/host/ehci-omap.c
@@ -86,7 +86,7 @@
static struct hc_driver __read_mostly ehci_omap_hc_driver;
-static const struct ehci_driver_overrides ehci_omap_overrides __initdata = {
+static const struct ehci_driver_overrides ehci_omap_overrides __initconst = {
.extra_priv_size = sizeof(struct omap_hcd),
};
diff --git a/drivers/usb/host/ehci-spear.c b/drivers/usb/host/ehci-spear.c
index 3c4e525..1f25c79 100644
--- a/drivers/usb/host/ehci-spear.c
+++ b/drivers/usb/host/ehci-spear.c
@@ -163,7 +163,7 @@
}
};
-static const struct ehci_driver_overrides spear_overrides __initdata = {
+static const struct ehci_driver_overrides spear_overrides __initconst = {
.extra_priv_size = sizeof(struct spear_ehci),
};
diff --git a/drivers/usb/host/fhci-sched.c b/drivers/usb/host/fhci-sched.c
index a9609a3..2f162fa 100644
--- a/drivers/usb/host/fhci-sched.c
+++ b/drivers/usb/host/fhci-sched.c
@@ -288,7 +288,7 @@
list_for_each_entry(ed, list, node) {
td = ed->td_head;
- if (!td || (td && td->status == USB_TD_INPROGRESS))
+ if (!td || td->status == USB_TD_INPROGRESS)
continue;
if (ed->state != FHCI_ED_OPER) {
diff --git a/drivers/usb/host/fotg210-hcd.c b/drivers/usb/host/fotg210-hcd.c
index 360a5e9..66efa9a 100644
--- a/drivers/usb/host/fotg210-hcd.c
+++ b/drivers/usb/host/fotg210-hcd.c
@@ -4795,14 +4795,8 @@
static inline int create_sysfs_files(struct fotg210_hcd *fotg210)
{
struct device *controller = fotg210_to_hcd(fotg210)->self.controller;
- int i = 0;
- if (i)
- goto out;
-
- i = device_create_file(controller, &dev_attr_uframe_periodic_max);
-out:
- return i;
+ return device_create_file(controller, &dev_attr_uframe_periodic_max);
}
static inline void remove_sysfs_files(struct fotg210_hcd *fotg210)
diff --git a/drivers/usb/host/ohci-hcd.c b/drivers/usb/host/ohci-hcd.c
index 04dcedf..0449235 100644
--- a/drivers/usb/host/ohci-hcd.c
+++ b/drivers/usb/host/ohci-hcd.c
@@ -1245,11 +1245,6 @@
#define TMIO_OHCI_DRIVER ohci_hcd_tmio_driver
#endif
-#ifdef CONFIG_MACH_JZ4740
-#include "ohci-jz4740.c"
-#define PLATFORM_DRIVER ohci_hcd_jz4740_driver
-#endif
-
#ifdef CONFIG_TILE_USB
#include "ohci-tilegx.c"
#define PLATFORM_DRIVER ohci_hcd_tilegx_driver
diff --git a/drivers/usb/host/ohci-jz4740.c b/drivers/usb/host/ohci-jz4740.c
deleted file mode 100644
index 4db78f1..0000000
--- a/drivers/usb/host/ohci-jz4740.c
+++ /dev/null
@@ -1,245 +0,0 @@
-/*
- * Copyright (C) 2010, Lars-Peter Clausen <lars@metafoo.de>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 675 Mass Ave, Cambridge, MA 02139, USA.
- *
- */
-
-#include <linux/platform_device.h>
-#include <linux/clk.h>
-#include <linux/regulator/consumer.h>
-
-struct jz4740_ohci_hcd {
- struct ohci_hcd ohci_hcd;
-
- struct regulator *vbus;
- bool vbus_enabled;
- struct clk *clk;
-};
-
-static inline struct jz4740_ohci_hcd *hcd_to_jz4740_hcd(struct usb_hcd *hcd)
-{
- return (struct jz4740_ohci_hcd *)(hcd->hcd_priv);
-}
-
-static inline struct usb_hcd *jz4740_hcd_to_hcd(struct jz4740_ohci_hcd *jz4740_ohci)
-{
- return container_of((void *)jz4740_ohci, struct usb_hcd, hcd_priv);
-}
-
-static int ohci_jz4740_start(struct usb_hcd *hcd)
-{
- struct ohci_hcd *ohci = hcd_to_ohci(hcd);
- int ret;
-
- ret = ohci_init(ohci);
- if (ret < 0)
- return ret;
-
- ohci->num_ports = 1;
-
- ret = ohci_run(ohci);
- if (ret < 0) {
- dev_err(hcd->self.controller, "Can not start %s",
- hcd->self.bus_name);
- ohci_stop(hcd);
- return ret;
- }
- return 0;
-}
-
-static int ohci_jz4740_set_vbus_power(struct jz4740_ohci_hcd *jz4740_ohci,
- bool enabled)
-{
- int ret = 0;
-
- if (!jz4740_ohci->vbus)
- return 0;
-
- if (enabled && !jz4740_ohci->vbus_enabled) {
- ret = regulator_enable(jz4740_ohci->vbus);
- if (ret)
- dev_err(jz4740_hcd_to_hcd(jz4740_ohci)->self.controller,
- "Could not power vbus\n");
- } else if (!enabled && jz4740_ohci->vbus_enabled) {
- ret = regulator_disable(jz4740_ohci->vbus);
- }
-
- if (ret == 0)
- jz4740_ohci->vbus_enabled = enabled;
-
- return ret;
-}
-
-static int ohci_jz4740_hub_control(struct usb_hcd *hcd, u16 typeReq, u16 wValue,
- u16 wIndex, char *buf, u16 wLength)
-{
- struct jz4740_ohci_hcd *jz4740_ohci = hcd_to_jz4740_hcd(hcd);
- int ret = 0;
-
- switch (typeReq) {
- case SetPortFeature:
- if (wValue == USB_PORT_FEAT_POWER)
- ret = ohci_jz4740_set_vbus_power(jz4740_ohci, true);
- break;
- case ClearPortFeature:
- if (wValue == USB_PORT_FEAT_POWER)
- ret = ohci_jz4740_set_vbus_power(jz4740_ohci, false);
- break;
- }
-
- if (ret)
- return ret;
-
- return ohci_hub_control(hcd, typeReq, wValue, wIndex, buf, wLength);
-}
-
-
-static const struct hc_driver ohci_jz4740_hc_driver = {
- .description = hcd_name,
- .product_desc = "JZ4740 OHCI",
- .hcd_priv_size = sizeof(struct jz4740_ohci_hcd),
-
- /*
- * generic hardware linkage
- */
- .irq = ohci_irq,
- .flags = HCD_USB11 | HCD_MEMORY,
-
- /*
- * basic lifecycle operations
- */
- .start = ohci_jz4740_start,
- .stop = ohci_stop,
- .shutdown = ohci_shutdown,
-
- /*
- * managing i/o requests and associated device resources
- */
- .urb_enqueue = ohci_urb_enqueue,
- .urb_dequeue = ohci_urb_dequeue,
- .endpoint_disable = ohci_endpoint_disable,
-
- /*
- * scheduling support
- */
- .get_frame_number = ohci_get_frame,
-
- /*
- * root hub support
- */
- .hub_status_data = ohci_hub_status_data,
- .hub_control = ohci_jz4740_hub_control,
-#ifdef CONFIG_PM
- .bus_suspend = ohci_bus_suspend,
- .bus_resume = ohci_bus_resume,
-#endif
- .start_port_reset = ohci_start_port_reset,
-};
-
-
-static int jz4740_ohci_probe(struct platform_device *pdev)
-{
- int ret;
- struct usb_hcd *hcd;
- struct jz4740_ohci_hcd *jz4740_ohci;
- struct resource *res;
- int irq;
-
- irq = platform_get_irq(pdev, 0);
- if (irq < 0) {
- dev_err(&pdev->dev, "Failed to get platform irq\n");
- return irq;
- }
-
- hcd = usb_create_hcd(&ohci_jz4740_hc_driver, &pdev->dev, "jz4740");
- if (!hcd) {
- dev_err(&pdev->dev, "Failed to create hcd.\n");
- return -ENOMEM;
- }
-
- jz4740_ohci = hcd_to_jz4740_hcd(hcd);
-
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- hcd->regs = devm_ioremap_resource(&pdev->dev, res);
- if (IS_ERR(hcd->regs)) {
- ret = PTR_ERR(hcd->regs);
- goto err_free;
- }
- hcd->rsrc_start = res->start;
- hcd->rsrc_len = resource_size(res);
-
- jz4740_ohci->clk = devm_clk_get(&pdev->dev, "uhc");
- if (IS_ERR(jz4740_ohci->clk)) {
- ret = PTR_ERR(jz4740_ohci->clk);
- dev_err(&pdev->dev, "Failed to get clock: %d\n", ret);
- goto err_free;
- }
-
- jz4740_ohci->vbus = devm_regulator_get(&pdev->dev, "vbus");
- if (IS_ERR(jz4740_ohci->vbus))
- jz4740_ohci->vbus = NULL;
-
-
- clk_set_rate(jz4740_ohci->clk, 48000000);
- clk_enable(jz4740_ohci->clk);
- if (jz4740_ohci->vbus)
- ohci_jz4740_set_vbus_power(jz4740_ohci, true);
-
- platform_set_drvdata(pdev, hcd);
-
- ohci_hcd_init(hcd_to_ohci(hcd));
-
- ret = usb_add_hcd(hcd, irq, 0);
- if (ret) {
- dev_err(&pdev->dev, "Failed to add hcd: %d\n", ret);
- goto err_disable;
- }
- device_wakeup_enable(hcd->self.controller);
-
- return 0;
-
-err_disable:
- if (jz4740_ohci->vbus)
- regulator_disable(jz4740_ohci->vbus);
- clk_disable(jz4740_ohci->clk);
-
-err_free:
- usb_put_hcd(hcd);
-
- return ret;
-}
-
-static int jz4740_ohci_remove(struct platform_device *pdev)
-{
- struct usb_hcd *hcd = platform_get_drvdata(pdev);
- struct jz4740_ohci_hcd *jz4740_ohci = hcd_to_jz4740_hcd(hcd);
-
- usb_remove_hcd(hcd);
-
- if (jz4740_ohci->vbus)
- regulator_disable(jz4740_ohci->vbus);
-
- clk_disable(jz4740_ohci->clk);
-
- usb_put_hcd(hcd);
-
- return 0;
-}
-
-static struct platform_driver ohci_hcd_jz4740_driver = {
- .probe = jz4740_ohci_probe,
- .remove = jz4740_ohci_remove,
- .driver = {
- .name = "jz4740-ohci",
- },
-};
-
-MODULE_ALIAS("platform:jz4740-ohci");
diff --git a/drivers/usb/host/whci/hcd.c b/drivers/usb/host/whci/hcd.c
index 43626c4..5b3603c 100644
--- a/drivers/usb/host/whci/hcd.c
+++ b/drivers/usb/host/whci/hcd.c
@@ -257,14 +257,14 @@
ret = whc_init(whc);
if (ret)
- goto error;
+ goto error_whc_init;
wusbhc->dev = dev;
wusbhc->uwb_rc = uwb_rc_get_by_grandpa(umc->dev.parent);
if (!wusbhc->uwb_rc) {
ret = -ENODEV;
dev_err(dev, "cannot get radio controller\n");
- goto error;
+ goto error_uwb_rc;
}
if (whc->n_devices > USB_MAXCHILDREN) {
@@ -311,8 +311,9 @@
wusbhc_destroy(wusbhc);
error_wusbhc_create:
uwb_rc_put(wusbhc->uwb_rc);
-error:
+error_uwb_rc:
whc_clean_up(whc);
+error_whc_init:
usb_put_hcd(usb_hcd);
return ret;
}
diff --git a/drivers/usb/host/whci/qset.c b/drivers/usb/host/whci/qset.c
index 1a8e960..c0e6812 100644
--- a/drivers/usb/host/whci/qset.c
+++ b/drivers/usb/host/whci/qset.c
@@ -314,7 +314,7 @@
kfree(std->bounce_buf);
}
if (std->pl_virt) {
- if (std->dma_addr)
+ if (!dma_mapping_error(whc->wusbhc.dev, std->dma_addr))
dma_unmap_single(whc->wusbhc.dev, std->dma_addr,
std->num_pointers * sizeof(struct whc_page_list_entry),
DMA_TO_DEVICE);
@@ -535,9 +535,11 @@
list_for_each_entry(std, &qset->stds, list_node) {
if (std->ntds_remaining == -1) {
pl_len = std->num_pointers * sizeof(struct whc_page_list_entry);
- std->ntds_remaining = ntds--;
std->dma_addr = dma_map_single(whc->wusbhc.dev, std->pl_virt,
pl_len, DMA_TO_DEVICE);
+ if (dma_mapping_error(whc->wusbhc.dev, std->dma_addr))
+ return -EFAULT;
+ std->ntds_remaining = ntds--;
}
}
return 0;
@@ -618,6 +620,8 @@
std->dma_addr = dma_map_single(&whc->umc->dev, std->bounce_buf, std->len,
is_out ? DMA_TO_DEVICE : DMA_FROM_DEVICE);
+ if (dma_mapping_error(&whc->umc->dev, std->dma_addr))
+ return -EFAULT;
if (qset_fill_page_list(whc, std, mem_flags) < 0)
return -ENOMEM;
diff --git a/drivers/usb/host/xhci-mvebu.c b/drivers/usb/host/xhci-mvebu.c
index 1eefc98..85908a3 100644
--- a/drivers/usb/host/xhci-mvebu.c
+++ b/drivers/usb/host/xhci-mvebu.c
@@ -12,6 +12,9 @@
#include <linux/of.h>
#include <linux/platform_device.h>
+#include <linux/usb.h>
+#include <linux/usb/hcd.h>
+
#include "xhci-mvebu.h"
#define USB3_MAX_WINDOWS 4
@@ -41,8 +44,10 @@
}
}
-int xhci_mvebu_mbus_init_quirk(struct platform_device *pdev)
+int xhci_mvebu_mbus_init_quirk(struct usb_hcd *hcd)
{
+ struct device *dev = hcd->self.controller;
+ struct platform_device *pdev = to_platform_device(dev);
struct resource *res;
void __iomem *base;
const struct mbus_dram_target_info *dram;
diff --git a/drivers/usb/host/xhci-mvebu.h b/drivers/usb/host/xhci-mvebu.h
index 7ede92a..301fc98 100644
--- a/drivers/usb/host/xhci-mvebu.h
+++ b/drivers/usb/host/xhci-mvebu.h
@@ -10,10 +10,13 @@
#ifndef __LINUX_XHCI_MVEBU_H
#define __LINUX_XHCI_MVEBU_H
+
+struct usb_hcd;
+
#if IS_ENABLED(CONFIG_USB_XHCI_MVEBU)
-int xhci_mvebu_mbus_init_quirk(struct platform_device *pdev);
+int xhci_mvebu_mbus_init_quirk(struct usb_hcd *hcd);
#else
-static inline int xhci_mvebu_mbus_init_quirk(struct platform_device *pdev)
+static inline int xhci_mvebu_mbus_init_quirk(struct usb_hcd *hcd)
{
return 0;
}
diff --git a/drivers/usb/host/xhci-plat.c b/drivers/usb/host/xhci-plat.c
index 474b5fa..676ea45 100644
--- a/drivers/usb/host/xhci-plat.c
+++ b/drivers/usb/host/xhci-plat.c
@@ -37,27 +37,32 @@
.start = xhci_plat_start,
};
+static void xhci_priv_plat_start(struct usb_hcd *hcd)
+{
+ struct xhci_plat_priv *priv = hcd_to_xhci_priv(hcd);
+
+ if (priv->plat_start)
+ priv->plat_start(hcd);
+}
+
+static int xhci_priv_init_quirk(struct usb_hcd *hcd)
+{
+ struct xhci_plat_priv *priv = hcd_to_xhci_priv(hcd);
+
+ if (!priv->init_quirk)
+ return 0;
+
+ return priv->init_quirk(hcd);
+}
+
static void xhci_plat_quirks(struct device *dev, struct xhci_hcd *xhci)
{
- struct usb_hcd *hcd = xhci_to_hcd(xhci);
-
/*
* As of now platform drivers don't provide MSI support so we ensure
* here that the generic code does not try to make a pci_dev from our
* dev struct in order to setup MSI
*/
xhci->quirks |= XHCI_PLAT;
-
- /*
- * On R-Car Gen2 and Gen3, the AC64 bit (bit 0) of HCCPARAMS1 is set
- * to 1. However, these SoCs don't support 64-bit address memory
- * pointers. So, this driver clears the AC64 bit of xhci->hcc_params
- * to call dma_set_coherent_mask(dev, DMA_BIT_MASK(32)) in
- * xhci_gen_setup().
- */
- if (xhci_plat_type_is(hcd, XHCI_PLAT_TYPE_RENESAS_RCAR_GEN2) ||
- xhci_plat_type_is(hcd, XHCI_PLAT_TYPE_RENESAS_RCAR_GEN3))
- xhci->quirks |= XHCI_NO_64BIT_SUPPORT;
}
/* called during probe() after chip reset completes */
@@ -65,38 +70,35 @@
{
int ret;
- if (xhci_plat_type_is(hcd, XHCI_PLAT_TYPE_RENESAS_RCAR_GEN2) ||
- xhci_plat_type_is(hcd, XHCI_PLAT_TYPE_RENESAS_RCAR_GEN3)) {
- ret = xhci_rcar_init_quirk(hcd);
- if (ret)
- return ret;
- }
+
+ ret = xhci_priv_init_quirk(hcd);
+ if (ret)
+ return ret;
return xhci_gen_setup(hcd, xhci_plat_quirks);
}
static int xhci_plat_start(struct usb_hcd *hcd)
{
- if (xhci_plat_type_is(hcd, XHCI_PLAT_TYPE_RENESAS_RCAR_GEN2) ||
- xhci_plat_type_is(hcd, XHCI_PLAT_TYPE_RENESAS_RCAR_GEN3))
- xhci_rcar_start(hcd);
-
+ xhci_priv_plat_start(hcd);
return xhci_run(hcd);
}
#ifdef CONFIG_OF
static const struct xhci_plat_priv xhci_plat_marvell_armada = {
- .type = XHCI_PLAT_TYPE_MARVELL_ARMADA,
+ .init_quirk = xhci_mvebu_mbus_init_quirk,
};
static const struct xhci_plat_priv xhci_plat_renesas_rcar_gen2 = {
- .type = XHCI_PLAT_TYPE_RENESAS_RCAR_GEN2,
.firmware_name = XHCI_RCAR_FIRMWARE_NAME_V1,
+ .init_quirk = xhci_rcar_init_quirk,
+ .plat_start = xhci_rcar_start,
};
static const struct xhci_plat_priv xhci_plat_renesas_rcar_gen3 = {
- .type = XHCI_PLAT_TYPE_RENESAS_RCAR_GEN3,
.firmware_name = XHCI_RCAR_FIRMWARE_NAME_V2,
+ .init_quirk = xhci_rcar_init_quirk,
+ .plat_start = xhci_rcar_start,
};
static const struct of_device_id usb_xhci_of_match[] = {
@@ -207,12 +209,6 @@
*priv = *priv_match;
}
- if (xhci_plat_type_is(hcd, XHCI_PLAT_TYPE_MARVELL_ARMADA)) {
- ret = xhci_mvebu_mbus_init_quirk(pdev);
- if (ret)
- goto disable_clk;
- }
-
device_wakeup_enable(hcd->self.controller);
xhci->clk = clk;
diff --git a/drivers/usb/host/xhci-plat.h b/drivers/usb/host/xhci-plat.h
index 529c3c4..9af0cb4 100644
--- a/drivers/usb/host/xhci-plat.h
+++ b/drivers/usb/host/xhci-plat.h
@@ -13,27 +13,11 @@
#include "xhci.h" /* for hcd_to_xhci() */
-enum xhci_plat_type {
- XHCI_PLAT_TYPE_MARVELL_ARMADA = 1,
- XHCI_PLAT_TYPE_RENESAS_RCAR_GEN2,
- XHCI_PLAT_TYPE_RENESAS_RCAR_GEN3,
-};
-
struct xhci_plat_priv {
- enum xhci_plat_type type;
const char *firmware_name;
+ void (*plat_start)(struct usb_hcd *);
+ int (*init_quirk)(struct usb_hcd *);
};
#define hcd_to_xhci_priv(h) ((struct xhci_plat_priv *)hcd_to_xhci(h)->priv)
-
-static inline bool xhci_plat_type_is(struct usb_hcd *hcd,
- enum xhci_plat_type type)
-{
- struct xhci_plat_priv *priv = hcd_to_xhci_priv(hcd);
-
- if (priv && priv->type == type)
- return true;
- else
- return false;
-}
#endif /* _XHCI_PLAT_H */
diff --git a/drivers/usb/host/xhci-rcar.c b/drivers/usb/host/xhci-rcar.c
index 623100e..0e4535e 100644
--- a/drivers/usb/host/xhci-rcar.c
+++ b/drivers/usb/host/xhci-rcar.c
@@ -11,6 +11,7 @@
#include <linux/firmware.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/of.h>
#include <linux/usb/phy.h>
#include "xhci.h"
@@ -76,6 +77,24 @@
writel(RCAR_USB3_TX_POL_VAL, hcd->regs + RCAR_USB3_TX_POL);
}
+static int xhci_rcar_is_gen2(struct device *dev)
+{
+ struct device_node *node = dev->of_node;
+
+ return of_device_is_compatible(node, "renesas,xhci-r8a7790") ||
+ of_device_is_compatible(node, "renesas,xhci-r8a7791") ||
+ of_device_is_compatible(node, "renesas,xhci-r8a7793") ||
+ of_device_is_compatible(node, "renensas,rcar-gen2-xhci");
+}
+
+static int xhci_rcar_is_gen3(struct device *dev)
+{
+ struct device_node *node = dev->of_node;
+
+ return of_device_is_compatible(node, "renesas,xhci-r8a7795") ||
+ of_device_is_compatible(node, "renesas,rcar-gen3-xhci");
+}
+
void xhci_rcar_start(struct usb_hcd *hcd)
{
u32 temp;
@@ -85,7 +104,7 @@
temp = readl(hcd->regs + RCAR_USB3_INT_ENA);
temp |= RCAR_USB3_INT_ENA_VAL;
writel(temp, hcd->regs + RCAR_USB3_INT_ENA);
- if (xhci_plat_type_is(hcd, XHCI_PLAT_TYPE_RENESAS_RCAR_GEN2))
+ if (xhci_rcar_is_gen2(hcd->self.controller))
xhci_rcar_start_gen2(hcd);
}
}
@@ -156,9 +175,22 @@
/* This function needs to initialize a "phy" of usb before */
int xhci_rcar_init_quirk(struct usb_hcd *hcd)
{
+ struct xhci_hcd *xhci = hcd_to_xhci(hcd);
+
/* If hcd->regs is NULL, we don't just call the following function */
if (!hcd->regs)
return 0;
+ /*
+ * On R-Car Gen2 and Gen3, the AC64 bit (bit 0) of HCCPARAMS1 is set
+ * to 1. However, these SoCs don't support 64-bit address memory
+ * pointers. So, this driver clears the AC64 bit of xhci->hcc_params
+ * to call dma_set_coherent_mask(dev, DMA_BIT_MASK(32)) in
+ * xhci_gen_setup().
+ */
+ if (xhci_rcar_is_gen2(hcd->self.controller) ||
+ xhci_rcar_is_gen3(hcd->self.controller))
+ xhci->quirks |= XHCI_NO_64BIT_SUPPORT;
+
return xhci_rcar_download_firmware(hcd);
}
diff --git a/drivers/usb/host/xhci-ring.c b/drivers/usb/host/xhci-ring.c
index 99b4ff4..52deae4 100644
--- a/drivers/usb/host/xhci-ring.c
+++ b/drivers/usb/host/xhci-ring.c
@@ -373,7 +373,11 @@
}
}
-static struct xhci_ring *xhci_triad_to_transfer_ring(struct xhci_hcd *xhci,
+/* Get the right ring for the given slot_id, ep_index and stream_id.
+ * If the endpoint supports streams, boundary check the URB's stream ID.
+ * If the endpoint doesn't support streams, return the singular endpoint ring.
+ */
+struct xhci_ring *xhci_triad_to_transfer_ring(struct xhci_hcd *xhci,
unsigned int slot_id, unsigned int ep_index,
unsigned int stream_id)
{
@@ -405,17 +409,6 @@
return NULL;
}
-/* Get the right ring for the given URB.
- * If the endpoint supports streams, boundary check the URB's stream ID.
- * If the endpoint doesn't support streams, return the singular endpoint ring.
- */
-static struct xhci_ring *xhci_urb_to_transfer_ring(struct xhci_hcd *xhci,
- struct urb *urb)
-{
- return xhci_triad_to_transfer_ring(xhci, urb->dev->slot_id,
- xhci_get_endpoint_index(&urb->ep->desc), urb->stream_id);
-}
-
/*
* Move the xHC's endpoint ring dequeue pointer past cur_td.
* Record the new state of the xHC's endpoint ring dequeue segment,
@@ -1768,7 +1761,7 @@
if (trb_comp_code == COMP_TX_ERR ||
trb_comp_code == COMP_BABBLE ||
trb_comp_code == COMP_SPLIT_ERR)
- /* The 0.96 spec says a babbling control endpoint
+ /* The 0.95 spec says a babbling control endpoint
* is not halted. The 0.96 spec says it is. Some HW
* claims to be 0.95 compliant, but it halts the control
* endpoint anyway. Check if a babble halted the
@@ -2938,46 +2931,55 @@
return 0;
}
-static unsigned int count_sg_trbs_needed(struct xhci_hcd *xhci, struct urb *urb)
+static unsigned int count_trbs(u64 addr, u64 len)
{
- int num_sgs, num_trbs, running_total, temp, i;
- struct scatterlist *sg;
+ unsigned int num_trbs;
- sg = NULL;
- num_sgs = urb->num_mapped_sgs;
- temp = urb->transfer_buffer_length;
+ num_trbs = DIV_ROUND_UP(len + (addr & (TRB_MAX_BUFF_SIZE - 1)),
+ TRB_MAX_BUFF_SIZE);
+ if (num_trbs == 0)
+ num_trbs++;
- num_trbs = 0;
- for_each_sg(urb->sg, sg, num_sgs, i) {
- unsigned int len = sg_dma_len(sg);
-
- /* Scatter gather list entries may cross 64KB boundaries */
- running_total = TRB_MAX_BUFF_SIZE -
- (sg_dma_address(sg) & (TRB_MAX_BUFF_SIZE - 1));
- running_total &= TRB_MAX_BUFF_SIZE - 1;
- if (running_total != 0)
- num_trbs++;
-
- /* How many more 64KB chunks to transfer, how many more TRBs? */
- while (running_total < sg_dma_len(sg) && running_total < temp) {
- num_trbs++;
- running_total += TRB_MAX_BUFF_SIZE;
- }
- len = min_t(int, len, temp);
- temp -= len;
- if (temp == 0)
- break;
- }
return num_trbs;
}
-static void check_trb_math(struct urb *urb, int num_trbs, int running_total)
+static inline unsigned int count_trbs_needed(struct urb *urb)
{
- if (num_trbs != 0)
- dev_err(&urb->dev->dev, "%s - ep %#x - Miscalculated number of "
- "TRBs, %d left\n", __func__,
- urb->ep->desc.bEndpointAddress, num_trbs);
- if (running_total != urb->transfer_buffer_length)
+ return count_trbs(urb->transfer_dma, urb->transfer_buffer_length);
+}
+
+static unsigned int count_sg_trbs_needed(struct urb *urb)
+{
+ struct scatterlist *sg;
+ unsigned int i, len, full_len, num_trbs = 0;
+
+ full_len = urb->transfer_buffer_length;
+
+ for_each_sg(urb->sg, sg, urb->num_mapped_sgs, i) {
+ len = sg_dma_len(sg);
+ num_trbs += count_trbs(sg_dma_address(sg), len);
+ len = min_t(unsigned int, len, full_len);
+ full_len -= len;
+ if (full_len == 0)
+ break;
+ }
+
+ return num_trbs;
+}
+
+static unsigned int count_isoc_trbs_needed(struct urb *urb, int i)
+{
+ u64 addr, len;
+
+ addr = (u64) (urb->transfer_dma + urb->iso_frame_desc[i].offset);
+ len = urb->iso_frame_desc[i].length;
+
+ return count_trbs(addr, len);
+}
+
+static void check_trb_math(struct urb *urb, int running_total)
+{
+ if (unlikely(running_total != urb->transfer_buffer_length))
dev_err(&urb->dev->dev, "%s - ep %#x - Miscalculated tx length, "
"queued %#x (%d), asked for %#x (%d)\n",
__func__,
@@ -3003,26 +3005,20 @@
xhci_ring_ep_doorbell(xhci, slot_id, ep_index, stream_id);
}
-/*
- * xHCI uses normal TRBs for both bulk and interrupt. When the interrupt
- * endpoint is to be serviced, the xHC will consume (at most) one TD. A TD
- * (comprised of sg list entries) can take several service intervals to
- * transmit.
- */
-int xhci_queue_intr_tx(struct xhci_hcd *xhci, gfp_t mem_flags,
- struct urb *urb, int slot_id, unsigned int ep_index)
+static void check_interval(struct xhci_hcd *xhci, struct urb *urb,
+ struct xhci_ep_ctx *ep_ctx)
{
- struct xhci_ep_ctx *ep_ctx = xhci_get_ep_ctx(xhci,
- xhci->devs[slot_id]->out_ctx, ep_index);
int xhci_interval;
int ep_interval;
xhci_interval = EP_INTERVAL_TO_UFRAMES(le32_to_cpu(ep_ctx->ep_info));
ep_interval = urb->interval;
+
/* Convert to microframes */
if (urb->dev->speed == USB_SPEED_LOW ||
urb->dev->speed == USB_SPEED_FULL)
ep_interval *= 8;
+
/* FIXME change this to a warning and a suggestion to use the new API
* to set the polling interval (once the API is added).
*/
@@ -3037,6 +3033,22 @@
urb->dev->speed == USB_SPEED_FULL)
urb->interval /= 8;
}
+}
+
+/*
+ * xHCI uses normal TRBs for both bulk and interrupt. When the interrupt
+ * endpoint is to be serviced, the xHC will consume (at most) one TD. A TD
+ * (comprised of sg list entries) can take several service intervals to
+ * transmit.
+ */
+int xhci_queue_intr_tx(struct xhci_hcd *xhci, gfp_t mem_flags,
+ struct urb *urb, int slot_id, unsigned int ep_index)
+{
+ struct xhci_ep_ctx *ep_ctx;
+
+ ep_ctx = xhci_get_ep_ctx(xhci, xhci->devs[slot_id]->out_ctx, ep_index);
+ check_interval(xhci, urb, ep_ctx);
+
return xhci_queue_bulk_tx(xhci, mem_flags, urb, slot_id, ep_index);
}
@@ -3086,177 +3098,6 @@
return (total_packet_count - ((transferred + trb_buff_len) / maxp));
}
-
-static int queue_bulk_sg_tx(struct xhci_hcd *xhci, gfp_t mem_flags,
- struct urb *urb, int slot_id, unsigned int ep_index)
-{
- struct xhci_ring *ep_ring;
- unsigned int num_trbs;
- struct urb_priv *urb_priv;
- struct xhci_td *td;
- struct scatterlist *sg;
- int num_sgs;
- int trb_buff_len, this_sg_len, running_total, ret;
- unsigned int total_packet_count;
- bool zero_length_needed;
- bool first_trb;
- int last_trb_num;
- u64 addr;
- bool more_trbs_coming;
-
- struct xhci_generic_trb *start_trb;
- int start_cycle;
-
- ep_ring = xhci_urb_to_transfer_ring(xhci, urb);
- if (!ep_ring)
- return -EINVAL;
-
- num_trbs = count_sg_trbs_needed(xhci, urb);
- num_sgs = urb->num_mapped_sgs;
- total_packet_count = DIV_ROUND_UP(urb->transfer_buffer_length,
- usb_endpoint_maxp(&urb->ep->desc));
-
- ret = prepare_transfer(xhci, xhci->devs[slot_id],
- ep_index, urb->stream_id,
- num_trbs, urb, 0, mem_flags);
- if (ret < 0)
- return ret;
-
- urb_priv = urb->hcpriv;
-
- /* Deal with URB_ZERO_PACKET - need one more td/trb */
- zero_length_needed = urb->transfer_flags & URB_ZERO_PACKET &&
- urb_priv->length == 2;
- if (zero_length_needed) {
- num_trbs++;
- xhci_dbg(xhci, "Creating zero length td.\n");
- ret = prepare_transfer(xhci, xhci->devs[slot_id],
- ep_index, urb->stream_id,
- 1, urb, 1, mem_flags);
- if (ret < 0)
- return ret;
- }
-
- td = urb_priv->td[0];
-
- /*
- * Don't give the first TRB to the hardware (by toggling the cycle bit)
- * until we've finished creating all the other TRBs. The ring's cycle
- * state may change as we enqueue the other TRBs, so save it too.
- */
- start_trb = &ep_ring->enqueue->generic;
- start_cycle = ep_ring->cycle_state;
-
- running_total = 0;
- /*
- * How much data is in the first TRB?
- *
- * There are three forces at work for TRB buffer pointers and lengths:
- * 1. We don't want to walk off the end of this sg-list entry buffer.
- * 2. The transfer length that the driver requested may be smaller than
- * the amount of memory allocated for this scatter-gather list.
- * 3. TRBs buffers can't cross 64KB boundaries.
- */
- sg = urb->sg;
- addr = (u64) sg_dma_address(sg);
- this_sg_len = sg_dma_len(sg);
- trb_buff_len = TRB_MAX_BUFF_SIZE - (addr & (TRB_MAX_BUFF_SIZE - 1));
- trb_buff_len = min_t(int, trb_buff_len, this_sg_len);
- if (trb_buff_len > urb->transfer_buffer_length)
- trb_buff_len = urb->transfer_buffer_length;
-
- first_trb = true;
- last_trb_num = zero_length_needed ? 2 : 1;
- /* Queue the first TRB, even if it's zero-length */
- do {
- u32 field = 0;
- u32 length_field = 0;
- u32 remainder = 0;
-
- /* Don't change the cycle bit of the first TRB until later */
- if (first_trb) {
- first_trb = false;
- if (start_cycle == 0)
- field |= 0x1;
- } else
- field |= ep_ring->cycle_state;
-
- /* Chain all the TRBs together; clear the chain bit in the last
- * TRB to indicate it's the last TRB in the chain.
- */
- if (num_trbs > last_trb_num) {
- field |= TRB_CHAIN;
- } else if (num_trbs == last_trb_num) {
- td->last_trb = ep_ring->enqueue;
- field |= TRB_IOC;
- } else if (zero_length_needed && num_trbs == 1) {
- trb_buff_len = 0;
- urb_priv->td[1]->last_trb = ep_ring->enqueue;
- field |= TRB_IOC;
- }
-
- /* Only set interrupt on short packet for IN endpoints */
- if (usb_urb_dir_in(urb))
- field |= TRB_ISP;
-
- if (TRB_MAX_BUFF_SIZE -
- (addr & (TRB_MAX_BUFF_SIZE - 1)) < trb_buff_len) {
- xhci_warn(xhci, "WARN: sg dma xfer crosses 64KB boundaries!\n");
- xhci_dbg(xhci, "Next boundary at %#x, end dma = %#x\n",
- (unsigned int) (addr + TRB_MAX_BUFF_SIZE) & ~(TRB_MAX_BUFF_SIZE - 1),
- (unsigned int) addr + trb_buff_len);
- }
-
- /* Set the TRB length, TD size, and interrupter fields. */
- remainder = xhci_td_remainder(xhci, running_total, trb_buff_len,
- urb->transfer_buffer_length,
- urb, num_trbs - 1);
-
- length_field = TRB_LEN(trb_buff_len) |
- TRB_TD_SIZE(remainder) |
- TRB_INTR_TARGET(0);
-
- if (num_trbs > 1)
- more_trbs_coming = true;
- else
- more_trbs_coming = false;
- queue_trb(xhci, ep_ring, more_trbs_coming,
- lower_32_bits(addr),
- upper_32_bits(addr),
- length_field,
- field | TRB_TYPE(TRB_NORMAL));
- --num_trbs;
- running_total += trb_buff_len;
-
- /* Calculate length for next transfer --
- * Are we done queueing all the TRBs for this sg entry?
- */
- this_sg_len -= trb_buff_len;
- if (this_sg_len == 0) {
- --num_sgs;
- if (num_sgs == 0)
- break;
- sg = sg_next(sg);
- addr = (u64) sg_dma_address(sg);
- this_sg_len = sg_dma_len(sg);
- } else {
- addr += trb_buff_len;
- }
-
- trb_buff_len = TRB_MAX_BUFF_SIZE -
- (addr & (TRB_MAX_BUFF_SIZE - 1));
- trb_buff_len = min_t(int, trb_buff_len, this_sg_len);
- if (running_total + trb_buff_len > urb->transfer_buffer_length)
- trb_buff_len =
- urb->transfer_buffer_length - running_total;
- } while (num_trbs > 0);
-
- check_trb_math(urb, num_trbs, running_total);
- giveback_first_trb(xhci, slot_id, ep_index, urb->stream_id,
- start_cycle, start_trb);
- return 0;
-}
-
/* This is very similar to what ehci-q.c qtd_fill() does */
int xhci_queue_bulk_tx(struct xhci_hcd *xhci, gfp_t mem_flags,
struct urb *urb, int slot_id, unsigned int ep_index)
@@ -3264,51 +3105,40 @@
struct xhci_ring *ep_ring;
struct urb_priv *urb_priv;
struct xhci_td *td;
- int num_trbs;
struct xhci_generic_trb *start_trb;
- bool first_trb;
- int last_trb_num;
+ struct scatterlist *sg = NULL;
bool more_trbs_coming;
bool zero_length_needed;
- int start_cycle;
- u32 field, length_field;
-
- int running_total, trb_buff_len, ret;
- unsigned int total_packet_count;
+ unsigned int num_trbs, last_trb_num, i;
+ unsigned int start_cycle, num_sgs = 0;
+ unsigned int running_total, block_len, trb_buff_len;
+ unsigned int full_len;
+ int ret;
+ u32 field, length_field, remainder;
u64 addr;
- if (urb->num_sgs)
- return queue_bulk_sg_tx(xhci, mem_flags, urb, slot_id, ep_index);
-
ep_ring = xhci_urb_to_transfer_ring(xhci, urb);
if (!ep_ring)
return -EINVAL;
- num_trbs = 0;
- /* How much data is (potentially) left before the 64KB boundary? */
- running_total = TRB_MAX_BUFF_SIZE -
- (urb->transfer_dma & (TRB_MAX_BUFF_SIZE - 1));
- running_total &= TRB_MAX_BUFF_SIZE - 1;
-
- /* If there's some data on this 64KB chunk, or we have to send a
- * zero-length transfer, we need at least one TRB
- */
- if (running_total != 0 || urb->transfer_buffer_length == 0)
- num_trbs++;
- /* How many more 64KB chunks to transfer, how many more TRBs? */
- while (running_total < urb->transfer_buffer_length) {
- num_trbs++;
- running_total += TRB_MAX_BUFF_SIZE;
- }
+ /* If we have scatter/gather list, we use it. */
+ if (urb->num_sgs) {
+ num_sgs = urb->num_mapped_sgs;
+ sg = urb->sg;
+ num_trbs = count_sg_trbs_needed(urb);
+ } else
+ num_trbs = count_trbs_needed(urb);
ret = prepare_transfer(xhci, xhci->devs[slot_id],
ep_index, urb->stream_id,
num_trbs, urb, 0, mem_flags);
- if (ret < 0)
+ if (unlikely(ret < 0))
return ret;
urb_priv = urb->hcpriv;
+ last_trb_num = num_trbs - 1;
+
/* Deal with URB_ZERO_PACKET - need one more td/trb */
zero_length_needed = urb->transfer_flags & URB_ZERO_PACKET &&
urb_priv->length == 2;
@@ -3318,7 +3148,7 @@
ret = prepare_transfer(xhci, xhci->devs[slot_id],
ep_index, urb->stream_id,
1, urb, 1, mem_flags);
- if (ret < 0)
+ if (unlikely(ret < 0))
return ret;
}
@@ -3332,43 +3162,58 @@
start_trb = &ep_ring->enqueue->generic;
start_cycle = ep_ring->cycle_state;
+ full_len = urb->transfer_buffer_length;
running_total = 0;
- total_packet_count = DIV_ROUND_UP(urb->transfer_buffer_length,
- usb_endpoint_maxp(&urb->ep->desc));
- /* How much data is in the first TRB? */
- addr = (u64) urb->transfer_dma;
- trb_buff_len = TRB_MAX_BUFF_SIZE -
- (urb->transfer_dma & (TRB_MAX_BUFF_SIZE - 1));
- if (trb_buff_len > urb->transfer_buffer_length)
- trb_buff_len = urb->transfer_buffer_length;
+ block_len = 0;
- first_trb = true;
- last_trb_num = zero_length_needed ? 2 : 1;
- /* Queue the first TRB, even if it's zero-length */
- do {
- u32 remainder = 0;
- field = 0;
+ /* Queue the TRBs, even if they are zero-length */
+ for (i = 0; i < num_trbs; i++) {
+ field = TRB_TYPE(TRB_NORMAL);
+
+ if (block_len == 0) {
+ /* A new contiguous block. */
+ if (sg) {
+ addr = (u64) sg_dma_address(sg);
+ block_len = sg_dma_len(sg);
+ } else {
+ addr = (u64) urb->transfer_dma;
+ block_len = full_len;
+ }
+ /* TRB buffer should not cross 64KB boundaries */
+ trb_buff_len = TRB_BUFF_LEN_UP_TO_BOUNDARY(addr);
+ trb_buff_len = min_t(unsigned int,
+ trb_buff_len,
+ block_len);
+ } else {
+ /* Further through the contiguous block. */
+ trb_buff_len = block_len;
+ if (trb_buff_len > TRB_MAX_BUFF_SIZE)
+ trb_buff_len = TRB_MAX_BUFF_SIZE;
+ }
+
+ if (running_total + trb_buff_len > full_len)
+ trb_buff_len = full_len - running_total;
/* Don't change the cycle bit of the first TRB until later */
- if (first_trb) {
- first_trb = false;
+ if (i == 0) {
if (start_cycle == 0)
- field |= 0x1;
+ field |= TRB_CYCLE;
} else
field |= ep_ring->cycle_state;
/* Chain all the TRBs together; clear the chain bit in the last
* TRB to indicate it's the last TRB in the chain.
*/
- if (num_trbs > last_trb_num) {
+ if (i < last_trb_num) {
field |= TRB_CHAIN;
- } else if (num_trbs == last_trb_num) {
- td->last_trb = ep_ring->enqueue;
+ } else {
field |= TRB_IOC;
- } else if (zero_length_needed && num_trbs == 1) {
- trb_buff_len = 0;
- urb_priv->td[1]->last_trb = ep_ring->enqueue;
- field |= TRB_IOC;
+ if (i == last_trb_num)
+ td->last_trb = ep_ring->enqueue;
+ else if (zero_length_needed) {
+ trb_buff_len = 0;
+ urb_priv->td[1]->last_trb = ep_ring->enqueue;
+ }
}
/* Only set interrupt on short packet for IN endpoints */
@@ -3376,15 +3221,15 @@
field |= TRB_ISP;
/* Set the TRB length, TD size, and interrupter fields. */
- remainder = xhci_td_remainder(xhci, running_total, trb_buff_len,
- urb->transfer_buffer_length,
- urb, num_trbs - 1);
+ remainder = xhci_td_remainder(xhci, running_total,
+ trb_buff_len, full_len,
+ urb, num_trbs - i - 1);
length_field = TRB_LEN(trb_buff_len) |
TRB_TD_SIZE(remainder) |
TRB_INTR_TARGET(0);
- if (num_trbs > 1)
+ if (i < num_trbs - 1)
more_trbs_coming = true;
else
more_trbs_coming = false;
@@ -3392,18 +3237,24 @@
lower_32_bits(addr),
upper_32_bits(addr),
length_field,
- field | TRB_TYPE(TRB_NORMAL));
- --num_trbs;
+ field);
+
running_total += trb_buff_len;
-
- /* Calculate length for next transfer */
addr += trb_buff_len;
- trb_buff_len = urb->transfer_buffer_length - running_total;
- if (trb_buff_len > TRB_MAX_BUFF_SIZE)
- trb_buff_len = TRB_MAX_BUFF_SIZE;
- } while (num_trbs > 0);
+ block_len -= trb_buff_len;
- check_trb_math(urb, num_trbs, running_total);
+ if (sg) {
+ if (block_len == 0) {
+ /* New sg entry */
+ --num_sgs;
+ if (num_sgs == 0)
+ break;
+ sg = sg_next(sg);
+ }
+ }
+ }
+
+ check_trb_math(urb, running_total);
giveback_first_trb(xhci, slot_id, ep_index, urb->stream_id,
start_cycle, start_trb);
return 0;
@@ -3532,23 +3383,6 @@
return 0;
}
-static int count_isoc_trbs_needed(struct xhci_hcd *xhci,
- struct urb *urb, int i)
-{
- int num_trbs = 0;
- u64 addr, td_len;
-
- addr = (u64) (urb->transfer_dma + urb->iso_frame_desc[i].offset);
- td_len = urb->iso_frame_desc[i].length;
-
- num_trbs = DIV_ROUND_UP(td_len + (addr & (TRB_MAX_BUFF_SIZE - 1)),
- TRB_MAX_BUFF_SIZE);
- if (num_trbs == 0)
- num_trbs++;
-
- return num_trbs;
-}
-
/*
* The transfer burst count field of the isochronous TRB defines the number of
* bursts that are required to move all packets in this TD. Only SuperSpeed
@@ -3746,7 +3580,7 @@
last_burst_pkt_count = xhci_get_last_burst_packet_count(xhci,
urb, total_pkt_count);
- trbs_per_td = count_isoc_trbs_needed(xhci, urb, i);
+ trbs_per_td = count_isoc_trbs_needed(urb, i);
ret = prepare_transfer(xhci, xhci->devs[slot_id], ep_index,
urb->stream_id, trbs_per_td, urb, i, mem_flags);
@@ -3807,8 +3641,7 @@
field |= TRB_BEI;
}
/* Calculate TRB length */
- trb_buff_len = TRB_MAX_BUFF_SIZE -
- (addr & ((1 << TRB_MAX_BUFF_SHIFT) - 1));
+ trb_buff_len = TRB_BUFF_LEN_UP_TO_BOUNDARY(addr);
if (trb_buff_len > td_remain_len)
trb_buff_len = td_remain_len;
@@ -3897,8 +3730,6 @@
struct xhci_ring *ep_ring;
struct xhci_ep_ctx *ep_ctx;
int start_frame;
- int xhci_interval;
- int ep_interval;
int num_tds, num_trbs, i;
int ret;
struct xhci_virt_ep *xep;
@@ -3912,7 +3743,7 @@
num_trbs = 0;
num_tds = urb->number_of_packets;
for (i = 0; i < num_tds; i++)
- num_trbs += count_isoc_trbs_needed(xhci, urb, i);
+ num_trbs += count_isoc_trbs_needed(urb, i);
/* Check the ring to guarantee there is enough room for the whole urb.
* Do not insert any td of the urb to the ring if the check failed.
@@ -3926,26 +3757,7 @@
* Check interval value. This should be done before we start to
* calculate the start frame value.
*/
- xhci_interval = EP_INTERVAL_TO_UFRAMES(le32_to_cpu(ep_ctx->ep_info));
- ep_interval = urb->interval;
- /* Convert to microframes */
- if (urb->dev->speed == USB_SPEED_LOW ||
- urb->dev->speed == USB_SPEED_FULL)
- ep_interval *= 8;
- /* FIXME change this to a warning and a suggestion to use the new API
- * to set the polling interval (once the API is added).
- */
- if (xhci_interval != ep_interval) {
- dev_dbg_ratelimited(&urb->dev->dev,
- "Driver uses different interval (%d microframe%s) than xHCI (%d microframe%s)\n",
- ep_interval, ep_interval == 1 ? "" : "s",
- xhci_interval, xhci_interval == 1 ? "" : "s");
- urb->interval = xhci_interval;
- /* Convert back to frames for LS/FS devices */
- if (urb->dev->speed == USB_SPEED_LOW ||
- urb->dev->speed == USB_SPEED_FULL)
- urb->interval /= 8;
- }
+ check_interval(xhci, urb, ep_ctx);
/* Calculate the start frame and put it in urb->start_frame. */
if (HCC_CFC(xhci->hcc_params) && !list_empty(&ep_ring->td_list)) {
diff --git a/drivers/usb/host/xhci.c b/drivers/usb/host/xhci.c
index 9e71c96..fa7e1ef 100644
--- a/drivers/usb/host/xhci.c
+++ b/drivers/usb/host/xhci.c
@@ -1459,47 +1459,6 @@
return ret;
}
-/* Get the right ring for the given URB.
- * If the endpoint supports streams, boundary check the URB's stream ID.
- * If the endpoint doesn't support streams, return the singular endpoint ring.
- */
-static struct xhci_ring *xhci_urb_to_transfer_ring(struct xhci_hcd *xhci,
- struct urb *urb)
-{
- unsigned int slot_id;
- unsigned int ep_index;
- unsigned int stream_id;
- struct xhci_virt_ep *ep;
-
- slot_id = urb->dev->slot_id;
- ep_index = xhci_get_endpoint_index(&urb->ep->desc);
- stream_id = urb->stream_id;
- ep = &xhci->devs[slot_id]->eps[ep_index];
- /* Common case: no streams */
- if (!(ep->ep_state & EP_HAS_STREAMS))
- return ep->ring;
-
- if (stream_id == 0) {
- xhci_warn(xhci,
- "WARN: Slot ID %u, ep index %u has streams, "
- "but URB has no stream ID.\n",
- slot_id, ep_index);
- return NULL;
- }
-
- if (stream_id < ep->stream_info->num_streams)
- return ep->stream_info->stream_rings[stream_id];
-
- xhci_warn(xhci,
- "WARN: Slot ID %u, ep index %u has "
- "stream IDs 1 to %u allocated, "
- "but stream ID %u is requested.\n",
- slot_id, ep_index,
- ep->stream_info->num_streams - 1,
- stream_id);
- return NULL;
-}
-
/*
* Remove the URB's TD from the endpoint ring. This may cause the HC to stop
* USB transfers, potentially stopping in the middle of a TRB buffer. The HC
diff --git a/drivers/usb/host/xhci.h b/drivers/usb/host/xhci.h
index 6c629c9..b0b8d0f 100644
--- a/drivers/usb/host/xhci.h
+++ b/drivers/usb/host/xhci.h
@@ -1338,6 +1338,9 @@
/* TRB buffer pointers can't cross 64KB boundaries */
#define TRB_MAX_BUFF_SHIFT 16
#define TRB_MAX_BUFF_SIZE (1 << TRB_MAX_BUFF_SHIFT)
+/* How much data is left before the 64KB boundary? */
+#define TRB_BUFF_LEN_UP_TO_BOUNDARY(addr) (TRB_MAX_BUFF_SIZE - \
+ (addr & (TRB_MAX_BUFF_SIZE - 1)))
struct xhci_segment {
union xhci_trb *trbs;
@@ -1965,4 +1968,15 @@
struct xhci_slot_ctx *xhci_get_slot_ctx(struct xhci_hcd *xhci, struct xhci_container_ctx *ctx);
struct xhci_ep_ctx *xhci_get_ep_ctx(struct xhci_hcd *xhci, struct xhci_container_ctx *ctx, unsigned int ep_index);
+struct xhci_ring *xhci_triad_to_transfer_ring(struct xhci_hcd *xhci,
+ unsigned int slot_id, unsigned int ep_index,
+ unsigned int stream_id);
+static inline struct xhci_ring *xhci_urb_to_transfer_ring(struct xhci_hcd *xhci,
+ struct urb *urb)
+{
+ return xhci_triad_to_transfer_ring(xhci, urb->dev->slot_id,
+ xhci_get_endpoint_index(&urb->ep->desc),
+ urb->stream_id);
+}
+
#endif /* __LINUX_XHCI_HCD_H */
diff --git a/drivers/usb/isp1760/isp1760-if.c b/drivers/usb/isp1760/isp1760-if.c
index 264be4d..9535b28 100644
--- a/drivers/usb/isp1760/isp1760-if.c
+++ b/drivers/usb/isp1760/isp1760-if.c
@@ -163,7 +163,7 @@
printk(KERN_ERR "ips1761_pci_shutdown\n");
}
-static const struct pci_device_id isp1760_plx [] = {
+static const struct pci_device_id isp1760_plx[] = {
{
.class = PCI_CLASS_BRIDGE_OTHER << 8,
.class_mask = ~0,
diff --git a/drivers/usb/misc/Kconfig b/drivers/usb/misc/Kconfig
index f7a7fc2..e9e5ae5 100644
--- a/drivers/usb/misc/Kconfig
+++ b/drivers/usb/misc/Kconfig
@@ -268,3 +268,29 @@
To compile this driver as a module, choose M here: the
module will be called chaoskey.
+
+config UCSI
+ tristate "USB Type-C Connector System Software Interface driver"
+ depends on ACPI
+ help
+ UCSI driver is meant to be used as a convenience tool for desktop and
+ server systems that are not equipped to handle USB in device mode. It
+ will always select USB host role for the USB Type-C ports on systems
+ that provide UCSI interface.
+
+ USB Type-C Connector System Software Interface (UCSI) is a
+ specification for an interface that allows the Operating System to
+ control the USB Type-C ports on a system. Things the need controlling
+ include the USB Data Role (host or device), and when USB Power
+ Delivery is supported, the Power Role (source or sink). With USB
+ Type-C connectors, when two dual role capable devices are attached
+ together, the data role is selected randomly. Therefore it is
+ important to give the OS a way to select the role. Otherwise the user
+ would have to unplug and replug in order in order to attempt to swap
+ the data and power roles.
+
+ The UCSI specification can be downloaded from:
+ http://www.intel.com/content/www/us/en/io/universal-serial-bus/usb-type-c-ucsi-spec.html
+
+ To compile the driver as a module, choose M here: the module will be
+ called ucsi.
diff --git a/drivers/usb/misc/Makefile b/drivers/usb/misc/Makefile
index 45fd4ac..2769cf6 100644
--- a/drivers/usb/misc/Makefile
+++ b/drivers/usb/misc/Makefile
@@ -26,6 +26,7 @@
obj-$(CONFIG_USB_YUREX) += yurex.o
obj-$(CONFIG_USB_HSIC_USB3503) += usb3503.o
obj-$(CONFIG_USB_CHAOSKEY) += chaoskey.o
+obj-$(CONFIG_UCSI) += ucsi.o
obj-$(CONFIG_USB_SISUSBVGA) += sisusbvga/
obj-$(CONFIG_USB_LINK_LAYER_TEST) += lvstest.o
diff --git a/drivers/usb/misc/sisusbvga/sisusb.c b/drivers/usb/misc/sisusbvga/sisusb.c
index a22de52..15666ad 100644
--- a/drivers/usb/misc/sisusbvga/sisusb.c
+++ b/drivers/usb/misc/sisusbvga/sisusb.c
@@ -2420,7 +2420,7 @@
if (!sisusb->devinit) {
if (sisusb->sisusb_dev->speed == USB_SPEED_HIGH ||
- sisusb->sisusb_dev->speed == USB_SPEED_SUPER) {
+ sisusb->sisusb_dev->speed >= USB_SPEED_SUPER) {
if (sisusb_init_gfxdevice(sisusb, 0)) {
mutex_unlock(&sisusb->lock);
dev_err(&sisusb->sisusb_dev->dev,
@@ -3127,7 +3127,7 @@
sisusb->present = 1;
- if (dev->speed == USB_SPEED_HIGH || dev->speed == USB_SPEED_SUPER) {
+ if (dev->speed == USB_SPEED_HIGH || dev->speed >= USB_SPEED_SUPER) {
int initscreen = 1;
#ifdef INCL_SISUSB_CON
if (sisusb_first_vc > 0 && sisusb_last_vc > 0 &&
diff --git a/drivers/usb/misc/sisusbvga/sisusb_con.c b/drivers/usb/misc/sisusbvga/sisusb_con.c
index ace3430..afa8532 100644
--- a/drivers/usb/misc/sisusbvga/sisusb_con.c
+++ b/drivers/usb/misc/sisusbvga/sisusb_con.c
@@ -601,7 +601,7 @@
/* interface routine */
static int
-sisusbcon_set_palette(struct vc_data *c, unsigned char *table)
+sisusbcon_set_palette(struct vc_data *c, const unsigned char *table)
{
struct sisusb_usb_data *sisusb;
int i, j;
diff --git a/drivers/usb/misc/ucsi.c b/drivers/usb/misc/ucsi.c
new file mode 100644
index 0000000..07397bd
--- /dev/null
+++ b/drivers/usb/misc/ucsi.c
@@ -0,0 +1,478 @@
+/*
+ * USB Type-C Connector System Software Interface driver
+ *
+ * Copyright (C) 2016, Intel Corporation
+ * Author: Heikki Krogerus <heikki.krogerus@linux.intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/platform_device.h>
+#include <linux/module.h>
+#include <linux/delay.h>
+#include <linux/acpi.h>
+
+#include "ucsi.h"
+
+/* Double the time defined by MIN_TIME_TO_RESPOND_WITH_BUSY */
+#define UCSI_TIMEOUT_MS 20
+
+enum ucsi_status {
+ UCSI_IDLE = 0,
+ UCSI_BUSY,
+ UCSI_ERROR,
+};
+
+struct ucsi_connector {
+ int num;
+ struct ucsi *ucsi;
+ struct work_struct work;
+ struct ucsi_connector_capability cap;
+};
+
+struct ucsi {
+ struct device *dev;
+ struct ucsi_data __iomem *data;
+
+ enum ucsi_status status;
+ struct completion complete;
+ struct ucsi_capability cap;
+ struct ucsi_connector *connector;
+
+ /* device lock */
+ spinlock_t dev_lock;
+
+ /* PPM Communication lock */
+ struct mutex ppm_lock;
+
+ /* PPM communication flags */
+ unsigned long flags;
+#define EVENT_PENDING 0
+#define COMMAND_PENDING 1
+};
+
+static int ucsi_acpi_cmd(struct ucsi *ucsi, struct ucsi_control *ctrl)
+{
+ uuid_le uuid = UUID_LE(0x6f8398c2, 0x7ca4, 0x11e4,
+ 0xad, 0x36, 0x63, 0x10, 0x42, 0xb5, 0x00, 0x8f);
+ union acpi_object *obj;
+
+ ucsi->data->ctrl.raw_cmd = ctrl->raw_cmd;
+
+ obj = acpi_evaluate_dsm(ACPI_HANDLE(ucsi->dev), uuid.b, 1, 1, NULL);
+ if (!obj) {
+ dev_err(ucsi->dev, "%s: failed to evaluate _DSM\n", __func__);
+ return -EIO;
+ }
+
+ ACPI_FREE(obj);
+ return 0;
+}
+
+static void ucsi_acpi_notify(acpi_handle handle, u32 event, void *data)
+{
+ struct ucsi *ucsi = data;
+ struct ucsi_cci *cci;
+
+ spin_lock(&ucsi->dev_lock);
+
+ ucsi->status = UCSI_IDLE;
+ cci = &ucsi->data->cci;
+
+ /*
+ * REVISIT: This is not documented behavior, but all known PPMs ACK
+ * asynchronous events by sending notification with cleared CCI.
+ */
+ if (!ucsi->data->raw_cci) {
+ if (test_bit(EVENT_PENDING, &ucsi->flags))
+ complete(&ucsi->complete);
+ else
+ dev_WARN(ucsi->dev, "spurious notification\n");
+ goto out_unlock;
+ }
+
+ if (test_bit(COMMAND_PENDING, &ucsi->flags)) {
+ if (cci->busy) {
+ ucsi->status = UCSI_BUSY;
+ complete(&ucsi->complete);
+
+ goto out_unlock;
+ } else if (cci->ack_complete || cci->cmd_complete) {
+ /* Error Indication is only valid with commands */
+ if (cci->error && cci->cmd_complete)
+ ucsi->status = UCSI_ERROR;
+
+ ucsi->data->ctrl.raw_cmd = 0;
+ complete(&ucsi->complete);
+ }
+ }
+
+ if (cci->connector_change) {
+ struct ucsi_connector *con;
+
+ /*
+ * This is workaround for buggy PPMs that create asynchronous
+ * event notifications before OPM has enabled them.
+ */
+ if (!ucsi->connector)
+ goto out_unlock;
+
+ con = ucsi->connector + (cci->connector_change - 1);
+
+ /*
+ * PPM will not clear the connector specific bit in Connector
+ * Change Indication field of CCI until the driver has ACK it,
+ * and the driver can not ACK it before it has been processed.
+ * The PPM will not generate new events before the first has
+ * been acknowledged, even if they are for an other connector.
+ * So only one event at a time.
+ */
+ if (!test_and_set_bit(EVENT_PENDING, &ucsi->flags))
+ schedule_work(&con->work);
+ }
+out_unlock:
+ spin_unlock(&ucsi->dev_lock);
+}
+
+static int ucsi_ack(struct ucsi *ucsi, u8 cmd)
+{
+ struct ucsi_control ctrl;
+ int ret;
+
+ ctrl.cmd.cmd = UCSI_ACK_CC_CI;
+ ctrl.cmd.length = 0;
+ ctrl.cmd.data = cmd;
+ ret = ucsi_acpi_cmd(ucsi, &ctrl);
+ if (ret)
+ return ret;
+
+ /* Waiting for ACK also with ACK CMD for now */
+ ret = wait_for_completion_timeout(&ucsi->complete,
+ msecs_to_jiffies(UCSI_TIMEOUT_MS));
+ if (!ret)
+ return -ETIMEDOUT;
+ return 0;
+}
+
+static int ucsi_run_cmd(struct ucsi *ucsi, struct ucsi_control *ctrl,
+ void *data, size_t size)
+{
+ u16 err_value = 0;
+ int ret;
+
+ set_bit(COMMAND_PENDING, &ucsi->flags);
+
+ ret = ucsi_acpi_cmd(ucsi, ctrl);
+ if (ret)
+ goto err_clear_flag;
+
+ ret = wait_for_completion_timeout(&ucsi->complete,
+ msecs_to_jiffies(UCSI_TIMEOUT_MS));
+ if (!ret) {
+ ret = -ETIMEDOUT;
+ goto err_clear_flag;
+ }
+
+ switch (ucsi->status) {
+ case UCSI_IDLE:
+ if (data)
+ memcpy(data, ucsi->data->message_in, size);
+
+ ret = ucsi_ack(ucsi, UCSI_ACK_CMD);
+ break;
+ case UCSI_BUSY:
+ /* The caller decides whether to cancel or not */
+ ret = -EBUSY;
+ goto err_clear_flag;
+ case UCSI_ERROR:
+ ret = ucsi_ack(ucsi, UCSI_ACK_CMD);
+ if (ret)
+ goto err_clear_flag;
+
+ ctrl->cmd.cmd = UCSI_GET_ERROR_STATUS;
+ ctrl->cmd.length = 0;
+ ctrl->cmd.data = 0;
+ ret = ucsi_acpi_cmd(ucsi, ctrl);
+ if (ret)
+ goto err_clear_flag;
+
+ ret = wait_for_completion_timeout(&ucsi->complete,
+ msecs_to_jiffies(UCSI_TIMEOUT_MS));
+ if (!ret) {
+ ret = -ETIMEDOUT;
+ goto err_clear_flag;
+ }
+
+ memcpy(&err_value, ucsi->data->message_in, sizeof(err_value));
+
+ /* Something has really gone wrong */
+ if (WARN_ON(ucsi->status == UCSI_ERROR)) {
+ ret = -ENODEV;
+ goto err_clear_flag;
+ }
+
+ ret = ucsi_ack(ucsi, UCSI_ACK_CMD);
+ if (ret)
+ goto err_clear_flag;
+
+ switch (err_value) {
+ case UCSI_ERROR_INCOMPATIBLE_PARTNER:
+ ret = -EOPNOTSUPP;
+ break;
+ case UCSI_ERROR_CC_COMMUNICATION_ERR:
+ ret = -ECOMM;
+ break;
+ case UCSI_ERROR_CONTRACT_NEGOTIATION_FAIL:
+ ret = -EIO;
+ break;
+ case UCSI_ERROR_DEAD_BATTERY:
+ dev_warn(ucsi->dev, "Dead battery condition!\n");
+ ret = -EPERM;
+ break;
+ /* The following mean a bug in this driver */
+ case UCSI_ERROR_INVALID_CON_NUM:
+ case UCSI_ERROR_UNREGONIZED_CMD:
+ case UCSI_ERROR_INVALID_CMD_ARGUMENT:
+ default:
+ dev_warn(ucsi->dev,
+ "%s: possible UCSI driver bug - error %hu\n",
+ __func__, err_value);
+ ret = -EINVAL;
+ break;
+ }
+ break;
+ }
+ ctrl->raw_cmd = 0;
+err_clear_flag:
+ clear_bit(COMMAND_PENDING, &ucsi->flags);
+ return ret;
+}
+
+static void ucsi_connector_change(struct work_struct *work)
+{
+ struct ucsi_connector *con = container_of(work, struct ucsi_connector,
+ work);
+ struct ucsi_connector_status constat;
+ struct ucsi *ucsi = con->ucsi;
+ struct ucsi_control ctrl;
+ int ret;
+
+ mutex_lock(&ucsi->ppm_lock);
+
+ ctrl.cmd.cmd = UCSI_GET_CONNECTOR_STATUS;
+ ctrl.cmd.length = 0;
+ ctrl.cmd.data = con->num;
+ ret = ucsi_run_cmd(con->ucsi, &ctrl, &constat, sizeof(constat));
+ if (ret) {
+ dev_err(ucsi->dev, "%s: failed to read connector status (%d)\n",
+ __func__, ret);
+ goto out_ack_event;
+ }
+
+ /* Ignoring disconnections and Alternate Modes */
+ if (!constat.connected || !(constat.change &
+ (UCSI_CONSTAT_PARTNER_CHANGE | UCSI_CONSTAT_CONNECT_CHANGE)) ||
+ constat.partner_flags & UCSI_CONSTAT_PARTNER_FLAG_ALT_MODE)
+ goto out_ack_event;
+
+ /* If the partner got USB Host role, attempting swap */
+ if (constat.partner_type & UCSI_CONSTAT_PARTNER_TYPE_DFP) {
+ ctrl.uor.cmd = UCSI_SET_UOR;
+ ctrl.uor.con_num = con->num;
+ ctrl.uor.role = UCSI_UOR_ROLE_DFP;
+
+ ret = ucsi_run_cmd(con->ucsi, &ctrl, NULL, 0);
+ if (ret)
+ dev_err(ucsi->dev, "%s: failed to swap role (%d)\n",
+ __func__, ret);
+ }
+out_ack_event:
+ ucsi_ack(ucsi, UCSI_ACK_EVENT);
+ clear_bit(EVENT_PENDING, &ucsi->flags);
+ mutex_unlock(&ucsi->ppm_lock);
+}
+
+static int ucsi_reset_ppm(struct ucsi *ucsi)
+{
+ int timeout = UCSI_TIMEOUT_MS;
+ struct ucsi_control ctrl;
+ int ret;
+
+ memset(&ctrl, 0, sizeof(ctrl));
+ ctrl.cmd.cmd = UCSI_PPM_RESET;
+ ret = ucsi_acpi_cmd(ucsi, &ctrl);
+ if (ret)
+ return ret;
+
+ /* There is no quarantee the PPM will ever set the RESET_COMPLETE bit */
+ while (!ucsi->data->cci.reset_complete && timeout--)
+ usleep_range(1000, 2000);
+ return 0;
+}
+
+static int ucsi_init(struct ucsi *ucsi)
+{
+ struct ucsi_connector *con;
+ struct ucsi_control ctrl;
+ int ret;
+ int i;
+
+ init_completion(&ucsi->complete);
+ spin_lock_init(&ucsi->dev_lock);
+ mutex_init(&ucsi->ppm_lock);
+
+ /* Reset the PPM */
+ ret = ucsi_reset_ppm(ucsi);
+ if (ret)
+ return ret;
+
+ /*
+ * REVISIT: Executing second reset to WA an issue seen on some of the
+ * Broxton based platforms, where the first reset puts the PPM into a
+ * state where it's unable to recognise some of the commands.
+ */
+ ret = ucsi_reset_ppm(ucsi);
+ if (ret)
+ return ret;
+
+ mutex_lock(&ucsi->ppm_lock);
+
+ /* Enable basic notifications */
+ ctrl.cmd.cmd = UCSI_SET_NOTIFICATION_ENABLE;
+ ctrl.cmd.length = 0;
+ ctrl.cmd.data = UCSI_ENABLE_NTFY_CMD_COMPLETE | UCSI_ENABLE_NTFY_ERROR;
+ ret = ucsi_run_cmd(ucsi, &ctrl, NULL, 0);
+ if (ret)
+ goto err_reset;
+
+ /* Get PPM capabilities */
+ ctrl.cmd.cmd = UCSI_GET_CAPABILITY;
+ ret = ucsi_run_cmd(ucsi, &ctrl, &ucsi->cap, sizeof(ucsi->cap));
+ if (ret)
+ goto err_reset;
+
+ if (!ucsi->cap.num_connectors) {
+ ret = -ENODEV;
+ goto err_reset;
+ }
+
+ ucsi->connector = devm_kcalloc(ucsi->dev, ucsi->cap.num_connectors,
+ sizeof(*ucsi->connector), GFP_KERNEL);
+ if (!ucsi->connector) {
+ ret = -ENOMEM;
+ goto err_reset;
+ }
+
+ for (i = 1, con = ucsi->connector; i < ucsi->cap.num_connectors + 1;
+ i++, con++) {
+ /* Get connector capability */
+ ctrl.cmd.cmd = UCSI_GET_CONNECTOR_CAPABILITY;
+ ctrl.cmd.data = i;
+ ret = ucsi_run_cmd(ucsi, &ctrl, &con->cap, sizeof(con->cap));
+ if (ret)
+ goto err_reset;
+
+ con->num = i;
+ con->ucsi = ucsi;
+ INIT_WORK(&con->work, ucsi_connector_change);
+ }
+
+ /* Enable all notifications */
+ ctrl.cmd.cmd = UCSI_SET_NOTIFICATION_ENABLE;
+ ctrl.cmd.data = UCSI_ENABLE_NTFY_ALL;
+ ret = ucsi_run_cmd(ucsi, &ctrl, NULL, 0);
+ if (ret < 0)
+ goto err_reset;
+
+ mutex_unlock(&ucsi->ppm_lock);
+ return 0;
+err_reset:
+ ucsi_reset_ppm(ucsi);
+ mutex_unlock(&ucsi->ppm_lock);
+ return ret;
+}
+
+static int ucsi_acpi_probe(struct platform_device *pdev)
+{
+ struct resource *res;
+ acpi_status status;
+ struct ucsi *ucsi;
+ int ret;
+
+ ucsi = devm_kzalloc(&pdev->dev, sizeof(*ucsi), GFP_KERNEL);
+ if (!ucsi)
+ return -ENOMEM;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (!res) {
+ dev_err(&pdev->dev, "missing memory resource\n");
+ return -ENODEV;
+ }
+
+ /*
+ * NOTE: ACPI has claimed the memory region as it's also an Operation
+ * Region. It's not possible to request it in the driver.
+ */
+ ucsi->data = devm_ioremap(&pdev->dev, res->start, resource_size(res));
+ if (!ucsi->data)
+ return -ENOMEM;
+
+ ucsi->dev = &pdev->dev;
+
+ status = acpi_install_notify_handler(ACPI_HANDLE(&pdev->dev),
+ ACPI_ALL_NOTIFY,
+ ucsi_acpi_notify, ucsi);
+ if (ACPI_FAILURE(status))
+ return -ENODEV;
+
+ ret = ucsi_init(ucsi);
+ if (ret) {
+ acpi_remove_notify_handler(ACPI_HANDLE(&pdev->dev),
+ ACPI_ALL_NOTIFY,
+ ucsi_acpi_notify);
+ return ret;
+ }
+
+ platform_set_drvdata(pdev, ucsi);
+ return 0;
+}
+
+static int ucsi_acpi_remove(struct platform_device *pdev)
+{
+ struct ucsi *ucsi = platform_get_drvdata(pdev);
+
+ acpi_remove_notify_handler(ACPI_HANDLE(&pdev->dev),
+ ACPI_ALL_NOTIFY, ucsi_acpi_notify);
+
+ /* Make sure there are no events in the middle of being processed */
+ if (wait_on_bit_timeout(&ucsi->flags, EVENT_PENDING,
+ TASK_UNINTERRUPTIBLE,
+ msecs_to_jiffies(UCSI_TIMEOUT_MS)))
+ dev_WARN(ucsi->dev, "%s: Events still pending\n", __func__);
+
+ ucsi_reset_ppm(ucsi);
+ return 0;
+}
+
+static const struct acpi_device_id ucsi_acpi_match[] = {
+ { "PNP0CA0", 0 },
+ { },
+};
+MODULE_DEVICE_TABLE(acpi, ucsi_acpi_match);
+
+static struct platform_driver ucsi_acpi_platform_driver = {
+ .driver = {
+ .name = "ucsi_acpi",
+ .acpi_match_table = ACPI_PTR(ucsi_acpi_match),
+ },
+ .probe = ucsi_acpi_probe,
+ .remove = ucsi_acpi_remove,
+};
+
+module_platform_driver(ucsi_acpi_platform_driver);
+
+MODULE_AUTHOR("Heikki Krogerus <heikki.krogerus@linux.intel.com>");
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("USB Type-C System Software Interface (UCSI) driver");
diff --git a/drivers/usb/misc/ucsi.h b/drivers/usb/misc/ucsi.h
new file mode 100644
index 0000000..6dd11d1
--- /dev/null
+++ b/drivers/usb/misc/ucsi.h
@@ -0,0 +1,215 @@
+
+#include <linux/types.h>
+
+/* -------------------------------------------------------------------------- */
+
+/* Command Status and Connector Change Indication (CCI) data structure */
+struct ucsi_cci {
+ unsigned int RESERVED1:1;
+ unsigned int connector_change:7;
+ u8 data_length;
+ unsigned int RESERVED9:9;
+ unsigned int not_supported:1;
+ unsigned int cancel_complete:1;
+ unsigned int reset_complete:1;
+ unsigned int busy:1;
+ unsigned int ack_complete:1;
+ unsigned int error:1;
+ unsigned int cmd_complete:1;
+} __packed;
+
+/* Default fields in CONTROL data structure */
+struct ucsi_command {
+ u8 cmd;
+ u8 length;
+ u64 data:48;
+} __packed;
+
+/* Set USB Operation Mode Command structure */
+struct ucsi_uor_cmd {
+ u8 cmd;
+ u8 length;
+ u64 con_num:7;
+ u64 role:3;
+#define UCSI_UOR_ROLE_DFP BIT(0)
+#define UCSI_UOR_ROLE_UFP BIT(1)
+#define UCSI_UOR_ROLE_DRP BIT(2)
+ u64 data:38;
+} __packed;
+
+struct ucsi_control {
+ union {
+ u64 raw_cmd;
+ struct ucsi_command cmd;
+ struct ucsi_uor_cmd uor;
+ };
+};
+
+struct ucsi_data {
+ u16 version;
+ u16 RESERVED;
+ union {
+ u32 raw_cci;
+ struct ucsi_cci cci;
+ };
+ struct ucsi_control ctrl;
+ u32 message_in[4];
+ u32 message_out[4];
+} __packed;
+
+/* Commands */
+#define UCSI_PPM_RESET 0x01
+#define UCSI_CANCEL 0x02
+#define UCSI_CONNECTOR_RESET 0x03
+#define UCSI_ACK_CC_CI 0x04
+#define UCSI_SET_NOTIFICATION_ENABLE 0x05
+#define UCSI_GET_CAPABILITY 0x06
+#define UCSI_GET_CONNECTOR_CAPABILITY 0x07
+#define UCSI_SET_UOM 0x08
+#define UCSI_SET_UOR 0x09
+#define UCSI_SET_PDM 0x0A
+#define UCSI_SET_PDR 0x0B
+#define UCSI_GET_ALTERNATE_MODES 0x0C
+#define UCSI_GET_CAM_SUPPORTED 0x0D
+#define UCSI_GET_CURRENT_CAM 0x0E
+#define UCSI_SET_NEW_CAM 0x0F
+#define UCSI_GET_PDOS 0x10
+#define UCSI_GET_CABLE_PROPERTY 0x11
+#define UCSI_GET_CONNECTOR_STATUS 0x12
+#define UCSI_GET_ERROR_STATUS 0x13
+
+/* ACK_CC_CI commands */
+#define UCSI_ACK_EVENT 1
+#define UCSI_ACK_CMD 2
+
+/* Bits for SET_NOTIFICATION_ENABLE command */
+#define UCSI_ENABLE_NTFY_CMD_COMPLETE BIT(0)
+#define UCSI_ENABLE_NTFY_EXT_PWR_SRC_CHANGE BIT(1)
+#define UCSI_ENABLE_NTFY_PWR_OPMODE_CHANGE BIT(2)
+#define UCSI_ENABLE_NTFY_CAP_CHANGE BIT(5)
+#define UCSI_ENABLE_NTFY_PWR_LEVEL_CHANGE BIT(6)
+#define UCSI_ENABLE_NTFY_PD_RESET_COMPLETE BIT(7)
+#define UCSI_ENABLE_NTFY_CAM_CHANGE BIT(8)
+#define UCSI_ENABLE_NTFY_BAT_STATUS_CHANGE BIT(9)
+#define UCSI_ENABLE_NTFY_PARTNER_CHANGE BIT(11)
+#define UCSI_ENABLE_NTFY_PWR_DIR_CHANGE BIT(12)
+#define UCSI_ENABLE_NTFY_CONNECTOR_CHANGE BIT(14)
+#define UCSI_ENABLE_NTFY_ERROR BIT(15)
+#define UCSI_ENABLE_NTFY_ALL 0xdbe7
+
+/* Error information returned by PPM in response to GET_ERROR_STATUS command. */
+#define UCSI_ERROR_UNREGONIZED_CMD BIT(0)
+#define UCSI_ERROR_INVALID_CON_NUM BIT(1)
+#define UCSI_ERROR_INVALID_CMD_ARGUMENT BIT(2)
+#define UCSI_ERROR_INCOMPATIBLE_PARTNER BIT(3)
+#define UCSI_ERROR_CC_COMMUNICATION_ERR BIT(4)
+#define UCSI_ERROR_DEAD_BATTERY BIT(5)
+#define UCSI_ERROR_CONTRACT_NEGOTIATION_FAIL BIT(6)
+
+/* Data structure filled by PPM in response to GET_CAPABILITY command. */
+struct ucsi_capability {
+ u32 attributes;
+#define UCSI_CAP_ATTR_DISABLE_STATE BIT(0)
+#define UCSI_CAP_ATTR_BATTERY_CHARGING BIT(1)
+#define UCSI_CAP_ATTR_USB_PD BIT(2)
+#define UCSI_CAP_ATTR_TYPEC_CURRENT BIT(6)
+#define UCSI_CAP_ATTR_POWER_AC_SUPPLY BIT(8)
+#define UCSI_CAP_ATTR_POWER_OTHER BIT(10)
+#define UCSI_CAP_ATTR_POWER_VBUS BIT(14)
+ u8 num_connectors;
+ u32 features:24;
+#define UCSI_CAP_SET_UOM BIT(0)
+#define UCSI_CAP_SET_PDM BIT(1)
+#define UCSI_CAP_ALT_MODE_DETAILS BIT(2)
+#define UCSI_CAP_ALT_MODE_OVERRIDE BIT(3)
+#define UCSI_CAP_PDO_DETAILS BIT(4)
+#define UCSI_CAP_CABLE_DETAILS BIT(5)
+#define UCSI_CAP_EXT_SUPPLY_NOTIFICATIONS BIT(6)
+#define UCSI_CAP_PD_RESET BIT(7)
+ u8 num_alt_modes;
+ u8 RESERVED;
+ u16 bc_version;
+ u16 pd_version;
+ u16 typec_version;
+} __packed;
+
+/* Data structure filled by PPM in response to GET_CONNECTOR_CAPABILITY cmd. */
+struct ucsi_connector_capability {
+ u8 op_mode;
+#define UCSI_CONCAP_OPMODE_DFP BIT(0)
+#define UCSI_CONCAP_OPMODE_UFP BIT(1)
+#define UCSI_CONCAP_OPMODE_DRP BIT(2)
+#define UCSI_CONCAP_OPMODE_AUDIO_ACCESSORY BIT(3)
+#define UCSI_CONCAP_OPMODE_DEBUG_ACCESSORY BIT(4)
+#define UCSI_CONCAP_OPMODE_USB2 BIT(5)
+#define UCSI_CONCAP_OPMODE_USB3 BIT(6)
+#define UCSI_CONCAP_OPMODE_ALT_MODE BIT(7)
+ u8 provider:1;
+ u8 consumer:1;
+} __packed;
+
+/* Data structure filled by PPM in response to GET_CABLE_PROPERTY command. */
+struct ucsi_cable_property {
+ u16 speed_supported;
+ u8 current_capability;
+ u8 vbus_in_cable:1;
+ u8 active_cable:1;
+ u8 directionality:1;
+ u8 plug_type:2;
+#define UCSI_CABLE_PROPERTY_PLUG_TYPE_A 0
+#define UCSI_CABLE_PROPERTY_PLUG_TYPE_B 1
+#define UCSI_CABLE_PROPERTY_PLUG_TYPE_C 2
+#define UCSI_CABLE_PROPERTY_PLUG_OTHER 3
+ u8 mode_support:1;
+ u8 RESERVED_2:2;
+ u8 latency:4;
+ u8 RESERVED_4:4;
+} __packed;
+
+/* Data structure filled by PPM in response to GET_CONNECTOR_STATUS command. */
+struct ucsi_connector_status {
+ u16 change;
+#define UCSI_CONSTAT_EXT_SUPPLY_CHANGE BIT(1)
+#define UCSI_CONSTAT_POWER_OPMODE_CHANGE BIT(2)
+#define UCSI_CONSTAT_PDOS_CHANGE BIT(5)
+#define UCSI_CONSTAT_POWER_LEVEL_CHANGE BIT(6)
+#define UCSI_CONSTAT_PD_RESET_COMPLETE BIT(7)
+#define UCSI_CONSTAT_CAM_CHANGE BIT(8)
+#define UCSI_CONSTAT_BC_CHANGE BIT(9)
+#define UCSI_CONSTAT_PARTNER_CHANGE BIT(11)
+#define UCSI_CONSTAT_POWER_DIR_CHANGE BIT(12)
+#define UCSI_CONSTAT_CONNECT_CHANGE BIT(14)
+#define UCSI_CONSTAT_ERROR BIT(15)
+ u16 pwr_op_mode:3;
+#define UCSI_CONSTAT_PWR_OPMODE_NONE 0
+#define UCSI_CONSTAT_PWR_OPMODE_DEFAULT 1
+#define UCSI_CONSTAT_PWR_OPMODE_BC 2
+#define UCSI_CONSTAT_PWR_OPMODE_PD 3
+#define UCSI_CONSTAT_PWR_OPMODE_TYPEC1_3 4
+#define UCSI_CONSTAT_PWR_OPMODE_TYPEC3_0 5
+ u16 connected:1;
+ u16 pwr_dir:1;
+ u16 partner_flags:8;
+#define UCSI_CONSTAT_PARTNER_FLAG_USB BIT(0)
+#define UCSI_CONSTAT_PARTNER_FLAG_ALT_MODE BIT(1)
+ u16 partner_type:3;
+#define UCSI_CONSTAT_PARTNER_TYPE_DFP 1
+#define UCSI_CONSTAT_PARTNER_TYPE_UFP 2
+#define UCSI_CONSTAT_PARTNER_TYPE_CABLE_NO_UFP 3 /* Powered Cable */
+#define UCSI_CONSTAT_PARTNER_TYPE_CABLE_AND_UFP 4 /* Powered Cable */
+#define UCSI_CONSTAT_PARTNER_TYPE_DEBUG 5
+#define UCSI_CONSTAT_PARTNER_TYPE_AUDIO 6
+ u32 request_data_obj;
+ u8 bc_status:2;
+#define UCSI_CONSTAT_BC_NOT_CHARGING 0
+#define UCSI_CONSTAT_BC_NOMINAL_CHARGING 1
+#define UCSI_CONSTAT_BC_SLOW_CHARGING 2
+#define UCSI_CONSTAT_BC_TRICKLE_CHARGING 3
+ u8 provider_cap_limit_reason:4;
+#define UCSI_CONSTAT_CAP_PWR_LOWERED 0
+#define UCSI_CONSTAT_CAP_PWR_BUDGET_LIMIT 1
+ u8 RESERVED:2;
+} __packed;
+
+/* -------------------------------------------------------------------------- */
+
diff --git a/drivers/usb/misc/usbtest.c b/drivers/usb/misc/usbtest.c
index 92fdb6e..6b978f0 100644
--- a/drivers/usb/misc/usbtest.c
+++ b/drivers/usb/misc/usbtest.c
@@ -287,6 +287,9 @@
if (usb_pipein(pipe))
urb->transfer_flags |= URB_SHORT_NOT_OK;
+ if ((bytes + offset) == 0)
+ return urb;
+
if (urb->transfer_flags & URB_NO_TRANSFER_DMA_MAP)
urb->transfer_buffer = usb_alloc_coherent(udev, bytes + offset,
GFP_KERNEL, &urb->transfer_dma);
@@ -529,6 +532,7 @@
alloc_sglist(int nents, int max, int vary, struct usbtest_dev *dev, int pipe)
{
struct scatterlist *sg;
+ unsigned int n_size = 0;
unsigned i;
unsigned size = max;
unsigned maxpacket =
@@ -561,7 +565,8 @@
break;
case 1:
for (j = 0; j < size; j++)
- *buf++ = (u8) ((j % maxpacket) % 63);
+ *buf++ = (u8) (((j + n_size) % maxpacket) % 63);
+ n_size += size;
break;
}
diff --git a/drivers/usb/phy/phy-qcom-8x16-usb.c b/drivers/usb/phy/phy-qcom-8x16-usb.c
index 3d7af85..d8593ad 100644
--- a/drivers/usb/phy/phy-qcom-8x16-usb.c
+++ b/drivers/usb/phy/phy-qcom-8x16-usb.c
@@ -240,10 +240,7 @@
qphy->switch_gpio = devm_gpiod_get_optional(dev, "switch",
GPIOD_OUT_LOW);
- if (IS_ERR(qphy->switch_gpio))
- return PTR_ERR(qphy->switch_gpio);
-
- return 0;
+ return PTR_ERR_OR_ZERO(qphy->switch_gpio);
}
static int phy_8x16_reboot_notify(struct notifier_block *this,
diff --git a/drivers/usb/phy/phy-twl6030-usb.c b/drivers/usb/phy/phy-twl6030-usb.c
index 014dbbd7..24e2b3c 100644
--- a/drivers/usb/phy/phy-twl6030-usb.c
+++ b/drivers/usb/phy/phy-twl6030-usb.c
@@ -155,13 +155,13 @@
static int twl6030_usb_ldo_init(struct twl6030_usb *twl)
{
/* Set to OTG_REV 1.3 and turn on the ID_WAKEUP_COMP */
- twl6030_writeb(twl, TWL6030_MODULE_ID0 , 0x1, TWL6030_BACKUP_REG);
+ twl6030_writeb(twl, TWL6030_MODULE_ID0, 0x1, TWL6030_BACKUP_REG);
/* Program CFG_LDO_PD2 register and set VUSB bit */
- twl6030_writeb(twl, TWL6030_MODULE_ID0 , 0x1, TWL6030_CFG_LDO_PD2);
+ twl6030_writeb(twl, TWL6030_MODULE_ID0, 0x1, TWL6030_CFG_LDO_PD2);
/* Program MISC2 register and set bit VUSB_IN_VBAT */
- twl6030_writeb(twl, TWL6030_MODULE_ID0 , 0x10, TWL6030_MISC2);
+ twl6030_writeb(twl, TWL6030_MODULE_ID0, 0x10, TWL6030_MISC2);
twl->usb3v3 = regulator_get(twl->dev, twl->regulator);
if (IS_ERR(twl->usb3v3))
@@ -301,10 +301,10 @@
*/
if (twl->vbus_enable)
- twl6030_writeb(twl, TWL_MODULE_MAIN_CHARGE , 0x40,
+ twl6030_writeb(twl, TWL_MODULE_MAIN_CHARGE, 0x40,
CHARGERUSB_CTRL1);
else
- twl6030_writeb(twl, TWL_MODULE_MAIN_CHARGE , 0x00,
+ twl6030_writeb(twl, TWL_MODULE_MAIN_CHARGE, 0x00,
CHARGERUSB_CTRL1);
}
diff --git a/drivers/usb/renesas_usbhs/fifo.c b/drivers/usb/renesas_usbhs/fifo.c
index 000f9750..7be4e7d 100644
--- a/drivers/usb/renesas_usbhs/fifo.c
+++ b/drivers/usb/renesas_usbhs/fifo.c
@@ -799,8 +799,10 @@
struct usbhs_pipe *pipe = pkt->pipe;
struct usbhs_priv *priv = usbhs_pipe_to_priv(pipe);
struct usbhs_pipe_info *info = usbhs_priv_to_pipeinfo(priv);
+ struct usbhs_fifo *fifo = usbhs_pipe_to_fifo(pipe);
+ struct dma_chan *chan = usbhsf_dma_chan_get(fifo, pkt);
- return info->dma_map_ctrl(pkt, map);
+ return info->dma_map_ctrl(chan->device->dev, pkt, map);
}
static void usbhsf_dma_complete(void *arg);
@@ -881,12 +883,12 @@
if (!fifo)
goto usbhsf_pio_prepare_push;
- if (usbhsf_dma_map(pkt) < 0)
- goto usbhsf_pio_prepare_push;
-
ret = usbhsf_fifo_select(pipe, fifo, 0);
if (ret < 0)
- goto usbhsf_pio_prepare_push_unmap;
+ goto usbhsf_pio_prepare_push;
+
+ if (usbhsf_dma_map(pkt) < 0)
+ goto usbhsf_pio_prepare_push_unselect;
pkt->trans = len;
@@ -896,8 +898,8 @@
return 0;
-usbhsf_pio_prepare_push_unmap:
- usbhsf_dma_unmap(pkt);
+usbhsf_pio_prepare_push_unselect:
+ usbhsf_fifo_unselect(pipe, fifo);
usbhsf_pio_prepare_push:
/*
* change handler to PIO
diff --git a/drivers/usb/renesas_usbhs/mod_gadget.c b/drivers/usb/renesas_usbhs/mod_gadget.c
index 53d104b..30345c2 100644
--- a/drivers/usb/renesas_usbhs/mod_gadget.c
+++ b/drivers/usb/renesas_usbhs/mod_gadget.c
@@ -191,13 +191,12 @@
/*
* dma map/unmap
*/
-static int usbhsg_dma_map_ctrl(struct usbhs_pkt *pkt, int map)
+static int usbhsg_dma_map_ctrl(struct device *dma_dev, struct usbhs_pkt *pkt,
+ int map)
{
struct usbhsg_request *ureq = usbhsg_pkt_to_ureq(pkt);
struct usb_request *req = &ureq->req;
struct usbhs_pipe *pipe = pkt->pipe;
- struct usbhsg_uep *uep = usbhsg_pipe_to_uep(pipe);
- struct usbhsg_gpriv *gpriv = usbhsg_uep_to_gpriv(uep);
enum dma_data_direction dir;
int ret = 0;
@@ -207,13 +206,13 @@
/* it can not use scatter/gather */
WARN_ON(req->num_sgs);
- ret = usb_gadget_map_request(&gpriv->gadget, req, dir);
+ ret = usb_gadget_map_request_by_dev(dma_dev, req, dir);
if (ret < 0)
return ret;
pkt->dma = req->dma;
} else {
- usb_gadget_unmap_request(&gpriv->gadget, req, dir);
+ usb_gadget_unmap_request_by_dev(dma_dev, req, dir);
}
return ret;
diff --git a/drivers/usb/renesas_usbhs/mod_host.c b/drivers/usb/renesas_usbhs/mod_host.c
index 1a8e4c4..3bf0b72 100644
--- a/drivers/usb/renesas_usbhs/mod_host.c
+++ b/drivers/usb/renesas_usbhs/mod_host.c
@@ -929,7 +929,8 @@
/*
* dma map functions
*/
-static int usbhsh_dma_map_ctrl(struct usbhs_pkt *pkt, int map)
+static int usbhsh_dma_map_ctrl(struct device *dma_dev, struct usbhs_pkt *pkt,
+ int map)
{
if (map) {
struct usbhsh_request *ureq = usbhsh_pkt_to_ureq(pkt);
diff --git a/drivers/usb/renesas_usbhs/pipe.c b/drivers/usb/renesas_usbhs/pipe.c
index 78e9dba..c238772 100644
--- a/drivers/usb/renesas_usbhs/pipe.c
+++ b/drivers/usb/renesas_usbhs/pipe.c
@@ -391,9 +391,8 @@
/*
* pipe setup
*/
-static u16 usbhsp_setup_pipecfg(struct usbhs_pipe *pipe,
- int is_host,
- int dir_in)
+static int usbhsp_setup_pipecfg(struct usbhs_pipe *pipe, int is_host,
+ int dir_in, u16 *pipecfg)
{
u16 type = 0;
u16 bfre = 0;
@@ -451,14 +450,14 @@
/* EPNUM */
epnum = 0; /* see usbhs_pipe_config_update() */
-
- return type |
- bfre |
- dblb |
- cntmd |
- dir |
- shtnak |
- epnum;
+ *pipecfg = type |
+ bfre |
+ dblb |
+ cntmd |
+ dir |
+ shtnak |
+ epnum;
+ return 0;
}
static u16 usbhsp_setup_pipebuff(struct usbhs_pipe *pipe)
@@ -655,7 +654,8 @@
}
void usbhs_pipe_init(struct usbhs_priv *priv,
- int (*dma_map_ctrl)(struct usbhs_pkt *pkt, int map))
+ int (*dma_map_ctrl)(struct device *dma_dev,
+ struct usbhs_pkt *pkt, int map))
{
struct usbhs_pipe_info *info = usbhs_priv_to_pipeinfo(priv);
struct usbhs_pipe *pipe;
@@ -702,7 +702,11 @@
return NULL;
}
- pipecfg = usbhsp_setup_pipecfg(pipe, is_host, dir_in);
+ if (usbhsp_setup_pipecfg(pipe, is_host, dir_in, &pipecfg)) {
+ dev_err(dev, "can't setup pipe\n");
+ return NULL;
+ }
+
pipebuf = usbhsp_setup_pipebuff(pipe);
usbhsp_pipe_select(pipe);
diff --git a/drivers/usb/renesas_usbhs/pipe.h b/drivers/usb/renesas_usbhs/pipe.h
index 7835747..95185fd 100644
--- a/drivers/usb/renesas_usbhs/pipe.h
+++ b/drivers/usb/renesas_usbhs/pipe.h
@@ -47,7 +47,8 @@
struct usbhs_pipe *pipe;
int size; /* array size of "pipe" */
- int (*dma_map_ctrl)(struct usbhs_pkt *pkt, int map);
+ int (*dma_map_ctrl)(struct device *dma_dev, struct usbhs_pkt *pkt,
+ int map);
};
/*
@@ -84,7 +85,8 @@
void usbhs_pipe_running(struct usbhs_pipe *pipe, int running);
void usbhs_pipe_init(struct usbhs_priv *priv,
- int (*dma_map_ctrl)(struct usbhs_pkt *pkt, int map));
+ int (*dma_map_ctrl)(struct device *dma_dev,
+ struct usbhs_pkt *pkt, int map));
int usbhs_pipe_get_maxpacket(struct usbhs_pipe *pipe);
void usbhs_pipe_clear(struct usbhs_pipe *pipe);
int usbhs_pipe_is_accessible(struct usbhs_pipe *pipe);
diff --git a/drivers/usb/serial/console.c b/drivers/usb/serial/console.c
index a66b01b..8967715 100644
--- a/drivers/usb/serial/console.c
+++ b/drivers/usb/serial/console.c
@@ -127,7 +127,7 @@
info->port = port;
++port->port.count;
- if (!test_bit(ASYNCB_INITIALIZED, &port->port.flags)) {
+ if (!tty_port_initialized(&port->port)) {
if (serial->type->set_termios) {
/*
* allocate a fake tty so the driver can initialize
@@ -168,7 +168,7 @@
tty_port_tty_set(&port->port, NULL);
tty_kref_put(tty);
}
- set_bit(ASYNCB_INITIALIZED, &port->port.flags);
+ tty_port_set_initialized(&port->port, 1);
}
/* Now that any required fake tty operations are completed restore
* the tty port count */
diff --git a/drivers/usb/serial/cp210x.c b/drivers/usb/serial/cp210x.c
index 7c9f25e..96a7078 100644
--- a/drivers/usb/serial/cp210x.c
+++ b/drivers/usb/serial/cp210x.c
@@ -331,6 +331,42 @@
*/
#define PURGE_ALL 0x000f
+/* CP210X_GET_FLOW/CP210X_SET_FLOW read/write these 0x10 bytes */
+struct cp210x_flow_ctl {
+ __le32 ulControlHandshake;
+ __le32 ulFlowReplace;
+ __le32 ulXonLimit;
+ __le32 ulXoffLimit;
+} __packed;
+
+/* cp210x_flow_ctl::ulControlHandshake */
+#define CP210X_SERIAL_DTR_MASK GENMASK(1, 0)
+#define CP210X_SERIAL_DTR_SHIFT(_mode) (_mode)
+#define CP210X_SERIAL_CTS_HANDSHAKE BIT(3)
+#define CP210X_SERIAL_DSR_HANDSHAKE BIT(4)
+#define CP210X_SERIAL_DCD_HANDSHAKE BIT(5)
+#define CP210X_SERIAL_DSR_SENSITIVITY BIT(6)
+
+/* values for cp210x_flow_ctl::ulControlHandshake::CP210X_SERIAL_DTR_MASK */
+#define CP210X_SERIAL_DTR_INACTIVE 0
+#define CP210X_SERIAL_DTR_ACTIVE 1
+#define CP210X_SERIAL_DTR_FLOW_CTL 2
+
+/* cp210x_flow_ctl::ulFlowReplace */
+#define CP210X_SERIAL_AUTO_TRANSMIT BIT(0)
+#define CP210X_SERIAL_AUTO_RECEIVE BIT(1)
+#define CP210X_SERIAL_ERROR_CHAR BIT(2)
+#define CP210X_SERIAL_NULL_STRIPPING BIT(3)
+#define CP210X_SERIAL_BREAK_CHAR BIT(4)
+#define CP210X_SERIAL_RTS_MASK GENMASK(7, 6)
+#define CP210X_SERIAL_RTS_SHIFT(_mode) (_mode << 6)
+#define CP210X_SERIAL_XOFF_CONTINUE BIT(31)
+
+/* values for cp210x_flow_ctl::ulFlowReplace::CP210X_SERIAL_RTS_MASK */
+#define CP210X_SERIAL_RTS_INACTIVE 0
+#define CP210X_SERIAL_RTS_ACTIVE 1
+#define CP210X_SERIAL_RTS_FLOW_CTL 2
+
/*
* Reads a variable-sized block of CP210X_ registers, identified by req.
* Returns data into buf in native USB byte order.
@@ -698,9 +734,10 @@
{
struct device *dev = &port->dev;
unsigned int cflag;
- u8 modem_ctl[16];
+ struct cp210x_flow_ctl flow_ctl;
u32 baud;
u16 bits;
+ u32 ctl_hs;
cp210x_read_u32_reg(port, CP210X_GET_BAUDRATE, &baud);
@@ -796,9 +833,10 @@
break;
}
- cp210x_read_reg_block(port, CP210X_GET_FLOW, modem_ctl,
- sizeof(modem_ctl));
- if (modem_ctl[0] & 0x08) {
+ cp210x_read_reg_block(port, CP210X_GET_FLOW, &flow_ctl,
+ sizeof(flow_ctl));
+ ctl_hs = le32_to_cpu(flow_ctl.ulControlHandshake);
+ if (ctl_hs & CP210X_SERIAL_CTS_HANDSHAKE) {
dev_dbg(dev, "%s - flow control = CRTSCTS\n", __func__);
cflag |= CRTSCTS;
} else {
@@ -867,7 +905,6 @@
struct device *dev = &port->dev;
unsigned int cflag, old_cflag;
u16 bits;
- u8 modem_ctl[16];
cflag = tty->termios.c_cflag;
old_cflag = old_termios->c_cflag;
@@ -951,35 +988,44 @@
}
if ((cflag & CRTSCTS) != (old_cflag & CRTSCTS)) {
+ struct cp210x_flow_ctl flow_ctl;
+ u32 ctl_hs;
+ u32 flow_repl;
- /* Only bytes 0, 4 and 7 out of first 8 have functional bits */
+ cp210x_read_reg_block(port, CP210X_GET_FLOW, &flow_ctl,
+ sizeof(flow_ctl));
+ ctl_hs = le32_to_cpu(flow_ctl.ulControlHandshake);
+ flow_repl = le32_to_cpu(flow_ctl.ulFlowReplace);
+ dev_dbg(dev, "%s - read ulControlHandshake=0x%08x, ulFlowReplace=0x%08x\n",
+ __func__, ctl_hs, flow_repl);
- cp210x_read_reg_block(port, CP210X_GET_FLOW, modem_ctl,
- sizeof(modem_ctl));
- dev_dbg(dev, "%s - read modem controls = %02x .. .. .. %02x .. .. %02x\n",
- __func__, modem_ctl[0], modem_ctl[4], modem_ctl[7]);
-
+ ctl_hs &= ~CP210X_SERIAL_DSR_HANDSHAKE;
+ ctl_hs &= ~CP210X_SERIAL_DCD_HANDSHAKE;
+ ctl_hs &= ~CP210X_SERIAL_DSR_SENSITIVITY;
+ ctl_hs &= ~CP210X_SERIAL_DTR_MASK;
+ ctl_hs |= CP210X_SERIAL_DTR_SHIFT(CP210X_SERIAL_DTR_ACTIVE);
if (cflag & CRTSCTS) {
- modem_ctl[0] &= ~0x7B;
- modem_ctl[0] |= 0x09;
- modem_ctl[4] = 0x80;
- /* FIXME - why clear reserved bits just read? */
- modem_ctl[5] = 0;
- modem_ctl[6] = 0;
- modem_ctl[7] = 0;
+ ctl_hs |= CP210X_SERIAL_CTS_HANDSHAKE;
+
+ flow_repl &= ~CP210X_SERIAL_RTS_MASK;
+ flow_repl |= CP210X_SERIAL_RTS_SHIFT(
+ CP210X_SERIAL_RTS_FLOW_CTL);
dev_dbg(dev, "%s - flow control = CRTSCTS\n", __func__);
} else {
- modem_ctl[0] &= ~0x7B;
- modem_ctl[0] |= 0x01;
- /* FIXME - OR here instead of assignment looks wrong */
- modem_ctl[4] |= 0x40;
+ ctl_hs &= ~CP210X_SERIAL_CTS_HANDSHAKE;
+
+ flow_repl &= ~CP210X_SERIAL_RTS_MASK;
+ flow_repl |= CP210X_SERIAL_RTS_SHIFT(
+ CP210X_SERIAL_RTS_ACTIVE);
dev_dbg(dev, "%s - flow control = NONE\n", __func__);
}
- dev_dbg(dev, "%s - write modem controls = %02x .. .. .. %02x .. .. %02x\n",
- __func__, modem_ctl[0], modem_ctl[4], modem_ctl[7]);
- cp210x_write_reg_block(port, CP210X_SET_FLOW, modem_ctl,
- sizeof(modem_ctl));
+ dev_dbg(dev, "%s - write ulControlHandshake=0x%08x, ulFlowReplace=0x%08x\n",
+ __func__, ctl_hs, flow_repl);
+ flow_ctl.ulControlHandshake = cpu_to_le32(ctl_hs);
+ flow_ctl.ulFlowReplace = cpu_to_le32(flow_repl);
+ cp210x_write_reg_block(port, CP210X_SET_FLOW, &flow_ctl,
+ sizeof(flow_ctl));
}
}
diff --git a/drivers/usb/serial/digi_acceleport.c b/drivers/usb/serial/digi_acceleport.c
index 16e8e37..6a1df9e 100644
--- a/drivers/usb/serial/digi_acceleport.c
+++ b/drivers/usb/serial/digi_acceleport.c
@@ -699,8 +699,7 @@
/* don't set RTS if using hardware flow control */
/* and throttling input */
modem_signals = TIOCM_DTR;
- if (!C_CRTSCTS(tty) ||
- !test_bit(TTY_THROTTLED, &tty->flags))
+ if (!C_CRTSCTS(tty) || !tty_throttled(tty))
modem_signals |= TIOCM_RTS;
digi_set_modem_signals(port, modem_signals, 1);
}
diff --git a/drivers/usb/serial/ftdi_sio.c b/drivers/usb/serial/ftdi_sio.c
index 3a814e8..0082080 100644
--- a/drivers/usb/serial/ftdi_sio.c
+++ b/drivers/usb/serial/ftdi_sio.c
@@ -93,27 +93,27 @@
static void ftdi_USB_UIRT_setup(struct ftdi_private *priv);
static void ftdi_HE_TIRA1_setup(struct ftdi_private *priv);
-static struct ftdi_sio_quirk ftdi_jtag_quirk = {
+static const struct ftdi_sio_quirk ftdi_jtag_quirk = {
.probe = ftdi_jtag_probe,
};
-static struct ftdi_sio_quirk ftdi_NDI_device_quirk = {
+static const struct ftdi_sio_quirk ftdi_NDI_device_quirk = {
.probe = ftdi_NDI_device_setup,
};
-static struct ftdi_sio_quirk ftdi_USB_UIRT_quirk = {
+static const struct ftdi_sio_quirk ftdi_USB_UIRT_quirk = {
.port_probe = ftdi_USB_UIRT_setup,
};
-static struct ftdi_sio_quirk ftdi_HE_TIRA1_quirk = {
+static const struct ftdi_sio_quirk ftdi_HE_TIRA1_quirk = {
.port_probe = ftdi_HE_TIRA1_setup,
};
-static struct ftdi_sio_quirk ftdi_stmclite_quirk = {
+static const struct ftdi_sio_quirk ftdi_stmclite_quirk = {
.probe = ftdi_stmclite_probe,
};
-static struct ftdi_sio_quirk ftdi_8u2232c_quirk = {
+static const struct ftdi_sio_quirk ftdi_8u2232c_quirk = {
.probe = ftdi_8u2232c_probe,
};
@@ -1775,7 +1775,7 @@
static int ftdi_sio_probe(struct usb_serial *serial,
const struct usb_device_id *id)
{
- struct ftdi_sio_quirk *quirk =
+ const struct ftdi_sio_quirk *quirk =
(struct ftdi_sio_quirk *)id->driver_info;
if (quirk && quirk->probe) {
@@ -1792,7 +1792,7 @@
static int ftdi_sio_port_probe(struct usb_serial_port *port)
{
struct ftdi_private *priv;
- struct ftdi_sio_quirk *quirk = usb_get_serial_data(port->serial);
+ const struct ftdi_sio_quirk *quirk = usb_get_serial_data(port->serial);
priv = kzalloc(sizeof(struct ftdi_private), GFP_KERNEL);
diff --git a/drivers/usb/serial/generic.c b/drivers/usb/serial/generic.c
index 54e170d..ae8c036 100644
--- a/drivers/usb/serial/generic.c
+++ b/drivers/usb/serial/generic.c
@@ -473,7 +473,7 @@
* Use tty-port initialised flag to detect all hangups including the
* one generated at USB-device disconnect.
*/
- if (!test_bit(ASYNCB_INITIALIZED, &port->port.flags))
+ if (!tty_port_initialized(&port->port))
return true;
spin_lock_irqsave(&port->lock, flags);
@@ -503,7 +503,7 @@
ret = wait_event_interruptible(port->port.delta_msr_wait,
usb_serial_generic_msr_changed(tty, arg, &cnow));
- if (!ret && !test_bit(ASYNCB_INITIALIZED, &port->port.flags))
+ if (!ret && !tty_port_initialized(&port->port))
ret = -EIO;
return ret;
@@ -606,7 +606,7 @@
for (i = 0; i < serial->num_ports; i++) {
port = serial->port[i];
- if (!test_bit(ASYNCB_INITIALIZED, &port->port.flags))
+ if (!tty_port_initialized(&port->port))
continue;
if (port->bulk_in_size) {
diff --git a/drivers/usb/serial/io_edgeport.c b/drivers/usb/serial/io_edgeport.c
index f3007ec..11c05ce 100644
--- a/drivers/usb/serial/io_edgeport.c
+++ b/drivers/usb/serial/io_edgeport.c
@@ -2849,14 +2849,16 @@
/* not set up yet, so do it now */
edge_serial->interrupt_read_urb =
usb_alloc_urb(0, GFP_KERNEL);
- if (!edge_serial->interrupt_read_urb)
- return -ENOMEM;
+ if (!edge_serial->interrupt_read_urb) {
+ response = -ENOMEM;
+ break;
+ }
edge_serial->interrupt_in_buffer =
kmalloc(buffer_size, GFP_KERNEL);
if (!edge_serial->interrupt_in_buffer) {
- usb_free_urb(edge_serial->interrupt_read_urb);
- return -ENOMEM;
+ response = -ENOMEM;
+ break;
}
edge_serial->interrupt_in_endpoint =
endpoint->bEndpointAddress;
@@ -2884,14 +2886,16 @@
/* not set up yet, so do it now */
edge_serial->read_urb =
usb_alloc_urb(0, GFP_KERNEL);
- if (!edge_serial->read_urb)
- return -ENOMEM;
+ if (!edge_serial->read_urb) {
+ response = -ENOMEM;
+ break;
+ }
edge_serial->bulk_in_buffer =
kmalloc(buffer_size, GFP_KERNEL);
if (!edge_serial->bulk_in_buffer) {
- usb_free_urb(edge_serial->read_urb);
- return -ENOMEM;
+ response = -ENOMEM;
+ break;
}
edge_serial->bulk_in_endpoint =
endpoint->bEndpointAddress;
@@ -2917,9 +2921,22 @@
}
}
- if (!interrupt_in_found || !bulk_in_found || !bulk_out_found) {
- dev_err(ddev, "Error - the proper endpoints were not found!\n");
- return -ENODEV;
+ if (response || !interrupt_in_found || !bulk_in_found ||
+ !bulk_out_found) {
+ if (!response) {
+ dev_err(ddev, "expected endpoints not found\n");
+ response = -ENODEV;
+ }
+
+ usb_free_urb(edge_serial->interrupt_read_urb);
+ kfree(edge_serial->interrupt_in_buffer);
+
+ usb_free_urb(edge_serial->read_urb);
+ kfree(edge_serial->bulk_in_buffer);
+
+ kfree(edge_serial);
+
+ return response;
}
/* start interrupt read for this edgeport this interrupt will
@@ -2942,16 +2959,9 @@
{
struct edgeport_serial *edge_serial = usb_get_serial_data(serial);
- /* stop reads and writes on all ports */
- /* free up our endpoint stuff */
if (edge_serial->is_epic) {
usb_kill_urb(edge_serial->interrupt_read_urb);
- usb_free_urb(edge_serial->interrupt_read_urb);
- kfree(edge_serial->interrupt_in_buffer);
-
usb_kill_urb(edge_serial->read_urb);
- usb_free_urb(edge_serial->read_urb);
- kfree(edge_serial->bulk_in_buffer);
}
}
@@ -2964,6 +2974,16 @@
{
struct edgeport_serial *edge_serial = usb_get_serial_data(serial);
+ if (edge_serial->is_epic) {
+ usb_kill_urb(edge_serial->interrupt_read_urb);
+ usb_free_urb(edge_serial->interrupt_read_urb);
+ kfree(edge_serial->interrupt_in_buffer);
+
+ usb_kill_urb(edge_serial->read_urb);
+ usb_free_urb(edge_serial->read_urb);
+ kfree(edge_serial->bulk_in_buffer);
+ }
+
kfree(edge_serial);
}
diff --git a/drivers/usb/serial/keyspan.c b/drivers/usb/serial/keyspan.c
index b6bd8e4..1f9414b 100644
--- a/drivers/usb/serial/keyspan.c
+++ b/drivers/usb/serial/keyspan.c
@@ -255,7 +255,7 @@
return count;
}
- dev_dbg(&port->dev, "%s - endpoint %d flip %d\n",
+ dev_dbg(&port->dev, "%s - endpoint %x flip %d\n",
__func__, usb_pipeendpoint(this_urb->pipe), flip);
if (this_urb->status == -EINPROGRESS) {
@@ -300,7 +300,7 @@
endpoint = usb_pipeendpoint(urb->pipe);
if (status) {
- dev_dbg(&urb->dev->dev, "%s - nonzero status: %x on endpoint %d.\n",
+ dev_dbg(&urb->dev->dev, "%s - nonzero status %d on endpoint %x\n",
__func__, status, endpoint);
return;
}
@@ -393,7 +393,8 @@
serial = urb->context;
if (status) {
- dev_dbg(&urb->dev->dev, "%s - nonzero status: %x\n", __func__, status);
+ dev_dbg(&urb->dev->dev, "%s - nonzero status: %d\n",
+ __func__, status);
return;
}
if (urb->actual_length != 9) {
@@ -452,7 +453,7 @@
do {
if (status) {
- dev_dbg(&urb->dev->dev, "%s - nonzero status: %x on endpoint %d.\n",
+ dev_dbg(&urb->dev->dev, "%s - nonzero status %d on endpoint %x\n",
__func__, status, usb_pipeendpoint(urb->pipe));
return;
}
@@ -511,7 +512,8 @@
serial = urb->context;
if (status) {
- dev_dbg(&urb->dev->dev, "%s - nonzero status: %x\n", __func__, status);
+ dev_dbg(&urb->dev->dev, "%s - nonzero status: %d\n",
+ __func__, status);
return;
}
@@ -591,7 +593,8 @@
serial = urb->context;
if (status) {
- dev_dbg(&urb->dev->dev, "%s - nonzero status: %x\n", __func__, status);
+ dev_dbg(&urb->dev->dev, "%s - nonzero status: %d\n",
+ __func__, status);
return;
}
@@ -646,7 +649,7 @@
endpoint = usb_pipeendpoint(urb->pipe);
if (status) {
- dev_dbg(&urb->dev->dev, "%s - nonzero status: %x on endpoint %d.\n",
+ dev_dbg(&urb->dev->dev, "%s - nonzero status %d on endpoint %x\n",
__func__, status, endpoint);
return;
}
@@ -698,7 +701,8 @@
serial = urb->context;
if (status) {
- dev_dbg(&urb->dev->dev, "%s - nonzero status: %x\n", __func__, status);
+ dev_dbg(&urb->dev->dev, "%s - nonzero status: %d\n",
+ __func__, status);
return;
}
@@ -774,8 +778,8 @@
endpoint = usb_pipeendpoint(urb->pipe);
if (status) {
- dev_dbg(&urb->dev->dev, "%s - nonzero status: %x on endpoint %d.\n",
- __func__, status, endpoint);
+ dev_dbg(&urb->dev->dev, "%s - nonzero status %d on endpoint %x\n",
+ __func__, status, endpoint);
return;
}
@@ -847,7 +851,8 @@
serial = urb->context;
if (status) {
- dev_dbg(&urb->dev->dev, "%s - nonzero status: %x\n", __func__, status);
+ dev_dbg(&urb->dev->dev, "%s - nonzero status: %d\n",
+ __func__, status);
return;
}
if (urb->actual_length < 14) {
@@ -912,7 +917,8 @@
serial = urb->context;
if (status) {
- dev_dbg(&urb->dev->dev, "%s - nonzero status: %x\n", __func__, status);
+ dev_dbg(&urb->dev->dev, "%s - nonzero status: %d\n",
+ __func__, status);
return;
}
@@ -1082,12 +1088,6 @@
return 0;
}
-static inline void stop_urb(struct urb *urb)
-{
- if (urb && urb->status == -EINPROGRESS)
- usb_kill_urb(urb);
-}
-
static void keyspan_dtr_rts(struct usb_serial_port *port, int on)
{
struct keyspan_port_private *p_priv = usb_get_serial_port_data(port);
@@ -1114,10 +1114,10 @@
p_priv->out_flip = 0;
p_priv->in_flip = 0;
- stop_urb(p_priv->inack_urb);
+ usb_kill_urb(p_priv->inack_urb);
for (i = 0; i < 2; i++) {
- stop_urb(p_priv->in_urbs[i]);
- stop_urb(p_priv->out_urbs[i]);
+ usb_kill_urb(p_priv->in_urbs[i]);
+ usb_kill_urb(p_priv->out_urbs[i]);
}
}
@@ -1221,8 +1221,8 @@
if (ep->bEndpointAddress == endpoint)
return ep;
}
- dev_warn(&serial->interface->dev, "found no endpoint descriptor for "
- "endpoint %x\n", endpoint);
+ dev_warn(&serial->interface->dev, "found no endpoint descriptor for endpoint %x\n",
+ endpoint);
return NULL;
}
@@ -1237,7 +1237,8 @@
if (endpoint == -1)
return NULL; /* endpoint not needed */
- dev_dbg(&serial->interface->dev, "%s - alloc for endpoint %d.\n", __func__, endpoint);
+ dev_dbg(&serial->interface->dev, "%s - alloc for endpoint %x\n",
+ __func__, endpoint);
urb = usb_alloc_urb(0, GFP_KERNEL); /* No ISO */
if (!urb)
return NULL;
@@ -1572,7 +1573,8 @@
return -1;
}
- dev_dbg(&port->dev, "%s - endpoint %d\n", __func__, usb_pipeendpoint(this_urb->pipe));
+ dev_dbg(&port->dev, "%s - endpoint %x\n",
+ __func__, usb_pipeendpoint(this_urb->pipe));
/* Save reset port val for resend.
Don't overwrite resend for open/close condition. */
@@ -1838,7 +1840,7 @@
return -1;
}
- dev_dbg(&port->dev, "%s - endpoint %d (%d)\n",
+ dev_dbg(&port->dev, "%s - endpoint %x (%d)\n",
__func__, usb_pipeendpoint(this_urb->pipe), device_port);
/* Save reset port val for resend.
@@ -2365,9 +2367,9 @@
s_priv = usb_get_serial_data(serial);
- stop_urb(s_priv->instat_urb);
- stop_urb(s_priv->glocont_urb);
- stop_urb(s_priv->indat_urb);
+ usb_kill_urb(s_priv->instat_urb);
+ usb_kill_urb(s_priv->glocont_urb);
+ usb_kill_urb(s_priv->indat_urb);
}
static void keyspan_release(struct usb_serial *serial)
@@ -2376,6 +2378,10 @@
s_priv = usb_get_serial_data(serial);
+ /* Make sure to unlink the URBs submitted in attach. */
+ usb_kill_urb(s_priv->instat_urb);
+ usb_kill_urb(s_priv->indat_urb);
+
usb_free_urb(s_priv->instat_urb);
usb_free_urb(s_priv->indat_urb);
usb_free_urb(s_priv->glocont_urb);
@@ -2491,11 +2497,11 @@
p_priv = usb_get_serial_port_data(port);
- stop_urb(p_priv->inack_urb);
- stop_urb(p_priv->outcont_urb);
+ usb_kill_urb(p_priv->inack_urb);
+ usb_kill_urb(p_priv->outcont_urb);
for (i = 0; i < 2; i++) {
- stop_urb(p_priv->in_urbs[i]);
- stop_urb(p_priv->out_urbs[i]);
+ usb_kill_urb(p_priv->in_urbs[i]);
+ usb_kill_urb(p_priv->out_urbs[i]);
}
usb_free_urb(p_priv->inack_urb);
diff --git a/drivers/usb/serial/mxuport.c b/drivers/usb/serial/mxuport.c
index 31a8b47..c88215a 100644
--- a/drivers/usb/serial/mxuport.c
+++ b/drivers/usb/serial/mxuport.c
@@ -503,7 +503,7 @@
return;
}
- if (test_bit(ASYNCB_INITIALIZED, &demux_port->port.flags)) {
+ if (tty_port_initialized(&demux_port->port)) {
ch = data + HEADER_SIZE;
mxuport_process_read_urb_data(demux_port, ch, rcv_len);
} else {
@@ -544,7 +544,7 @@
}
demux_port = serial->port[rcv_port];
- if (test_bit(ASYNCB_INITIALIZED, &demux_port->port.flags)) {
+ if (tty_port_initialized(&demux_port->port)) {
ch = data + HEADER_SIZE;
rcv_event = get_unaligned_be16(data + 2);
mxuport_process_read_urb_event(demux_port, ch,
@@ -1259,6 +1259,15 @@
return 0;
}
+static void mxuport_release(struct usb_serial *serial)
+{
+ struct usb_serial_port *port0 = serial->port[0];
+ struct usb_serial_port *port1 = serial->port[1];
+
+ usb_serial_generic_close(port1);
+ usb_serial_generic_close(port0);
+}
+
static int mxuport_open(struct tty_struct *tty, struct usb_serial_port *port)
{
struct mxuport_port *mxport = usb_get_serial_port_data(port);
@@ -1339,7 +1348,7 @@
for (i = 0; i < serial->num_ports; i++) {
port = serial->port[i];
- if (!test_bit(ASYNCB_INITIALIZED, &port->port.flags))
+ if (!tty_port_initialized(&port->port))
continue;
r = usb_serial_generic_write_start(port, GFP_NOIO);
@@ -1361,6 +1370,7 @@
.probe = mxuport_probe,
.port_probe = mxuport_port_probe,
.attach = mxuport_attach,
+ .release = mxuport_release,
.calc_num_ports = mxuport_calc_num_ports,
.open = mxuport_open,
.close = mxuport_close,
diff --git a/drivers/usb/serial/option.c b/drivers/usb/serial/option.c
index c6f497f..d96d423 100644
--- a/drivers/usb/serial/option.c
+++ b/drivers/usb/serial/option.c
@@ -375,18 +375,22 @@
#define HAIER_PRODUCT_CE81B 0x10f8
#define HAIER_PRODUCT_CE100 0x2009
-/* Cinterion (formerly Siemens) products */
-#define SIEMENS_VENDOR_ID 0x0681
-#define CINTERION_VENDOR_ID 0x1e2d
+/* Gemalto's Cinterion products (formerly Siemens) */
+#define SIEMENS_VENDOR_ID 0x0681
+#define CINTERION_VENDOR_ID 0x1e2d
+#define CINTERION_PRODUCT_HC25_MDMNET 0x0040
#define CINTERION_PRODUCT_HC25_MDM 0x0047
-#define CINTERION_PRODUCT_HC25_MDMNET 0x0040
+#define CINTERION_PRODUCT_HC28_MDMNET 0x004A /* same for HC28J */
#define CINTERION_PRODUCT_HC28_MDM 0x004C
-#define CINTERION_PRODUCT_HC28_MDMNET 0x004A /* same for HC28J */
#define CINTERION_PRODUCT_EU3_E 0x0051
#define CINTERION_PRODUCT_EU3_P 0x0052
#define CINTERION_PRODUCT_PH8 0x0053
#define CINTERION_PRODUCT_AHXX 0x0055
#define CINTERION_PRODUCT_PLXX 0x0060
+#define CINTERION_PRODUCT_PH8_2RMNET 0x0082
+#define CINTERION_PRODUCT_PH8_AUDIO 0x0083
+#define CINTERION_PRODUCT_AHXX_2RMNET 0x0084
+#define CINTERION_PRODUCT_AHXX_AUDIO 0x0085
/* Olivetti products */
#define OLIVETTI_VENDOR_ID 0x0b3c
@@ -633,6 +637,10 @@
.reserved = BIT(1) | BIT(2) | BIT(3),
};
+static const struct option_blacklist_info cinterion_rmnet2_blacklist = {
+ .reserved = BIT(4) | BIT(5),
+};
+
static const struct usb_device_id option_ids[] = {
{ USB_DEVICE(OPTION_VENDOR_ID, OPTION_PRODUCT_COLT) },
{ USB_DEVICE(OPTION_VENDOR_ID, OPTION_PRODUCT_RICOLA) },
@@ -1602,7 +1610,79 @@
.driver_info = (kernel_ulong_t)&net_intf3_blacklist },
{ USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0x0178, 0xff, 0xff, 0xff),
.driver_info = (kernel_ulong_t)&net_intf3_blacklist },
- { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffe9, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff42, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff43, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff44, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff45, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff46, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff47, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff48, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff49, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff4a, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff4b, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff4c, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff4d, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff4e, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff4f, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff50, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff51, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff52, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff53, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff54, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff55, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff56, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff57, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff58, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff59, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff5a, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff5b, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff5c, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff5d, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff5e, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff5f, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff60, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff61, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff62, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff63, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff64, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff65, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff66, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff67, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff68, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff69, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff6a, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff6b, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff6c, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff6d, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff6e, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff6f, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff70, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff71, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff72, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff73, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff74, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff75, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff76, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff77, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff78, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff79, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff7a, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff7b, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff7c, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff7d, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff7e, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff7f, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff80, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff81, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff82, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff83, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff84, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff85, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff86, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff87, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff88, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff89, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff8a, 0xff, 0xff, 0xff) },
{ USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff8b, 0xff, 0xff, 0xff) },
{ USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff8c, 0xff, 0xff, 0xff) },
{ USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff8d, 0xff, 0xff, 0xff) },
@@ -1613,6 +1693,61 @@
{ USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff92, 0xff, 0xff, 0xff) },
{ USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff93, 0xff, 0xff, 0xff) },
{ USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff94, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff9f, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffa0, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffa1, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffa2, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffa3, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffa4, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffa5, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffa6, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffa7, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffa8, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffa9, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffaa, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffab, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffac, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffae, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffaf, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffb0, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffb1, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffb2, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffb3, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffb4, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffb5, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffb6, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffb7, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffb8, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffb9, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffba, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffbb, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffbc, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffbd, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffbe, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffbf, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffc0, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffc1, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffc2, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffc3, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffc4, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffc5, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffc6, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffc7, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffc8, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffc9, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffca, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffcb, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffcc, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffcd, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffce, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffcf, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffd0, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffd1, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffd2, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffd3, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffd4, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffd5, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffe9, 0xff, 0xff, 0xff) },
{ USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffec, 0xff, 0xff, 0xff) },
{ USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xffee, 0xff, 0xff, 0xff) },
{ USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xfff6, 0xff, 0xff, 0xff) },
@@ -1712,7 +1847,13 @@
{ USB_DEVICE_INTERFACE_CLASS(CINTERION_VENDOR_ID, CINTERION_PRODUCT_AHXX, 0xff) },
{ USB_DEVICE(CINTERION_VENDOR_ID, CINTERION_PRODUCT_PLXX),
.driver_info = (kernel_ulong_t)&net_intf4_blacklist },
- { USB_DEVICE(CINTERION_VENDOR_ID, CINTERION_PRODUCT_HC28_MDM) },
+ { USB_DEVICE_INTERFACE_CLASS(CINTERION_VENDOR_ID, CINTERION_PRODUCT_PH8_2RMNET, 0xff),
+ .driver_info = (kernel_ulong_t)&cinterion_rmnet2_blacklist },
+ { USB_DEVICE_INTERFACE_CLASS(CINTERION_VENDOR_ID, CINTERION_PRODUCT_PH8_AUDIO, 0xff),
+ .driver_info = (kernel_ulong_t)&net_intf4_blacklist },
+ { USB_DEVICE_INTERFACE_CLASS(CINTERION_VENDOR_ID, CINTERION_PRODUCT_AHXX_2RMNET, 0xff) },
+ { USB_DEVICE_INTERFACE_CLASS(CINTERION_VENDOR_ID, CINTERION_PRODUCT_AHXX_AUDIO, 0xff) },
+ { USB_DEVICE(CINTERION_VENDOR_ID, CINTERION_PRODUCT_HC28_MDM) },
{ USB_DEVICE(CINTERION_VENDOR_ID, CINTERION_PRODUCT_HC28_MDMNET) },
{ USB_DEVICE(SIEMENS_VENDOR_ID, CINTERION_PRODUCT_HC25_MDM) },
{ USB_DEVICE(SIEMENS_VENDOR_ID, CINTERION_PRODUCT_HC25_MDMNET) },
diff --git a/drivers/usb/serial/quatech2.c b/drivers/usb/serial/quatech2.c
index 2df8ad5..85acb50 100644
--- a/drivers/usb/serial/quatech2.c
+++ b/drivers/usb/serial/quatech2.c
@@ -141,6 +141,7 @@
serial_priv = usb_get_serial_data(serial);
+ usb_kill_urb(serial_priv->read_urb);
usb_free_urb(serial_priv->read_urb);
kfree(serial_priv->read_buffer);
kfree(serial_priv);
diff --git a/drivers/usb/serial/sierra.c b/drivers/usb/serial/sierra.c
index 07d1ecd..e1994e2 100644
--- a/drivers/usb/serial/sierra.c
+++ b/drivers/usb/serial/sierra.c
@@ -776,7 +776,7 @@
/*
* Need to take susp_lock to make sure port is not already being
- * resumed, but no need to hold it due to ASYNC_INITIALIZED.
+ * resumed, but no need to hold it due to initialized
*/
spin_lock_irq(&intfdata->susp_lock);
if (--intfdata->open_ports == 0)
@@ -1039,7 +1039,7 @@
for (i = 0; i < serial->num_ports; i++) {
port = serial->port[i];
- if (!test_bit(ASYNCB_INITIALIZED, &port->port.flags))
+ if (!tty_port_initialized(&port->port))
continue;
err = sierra_submit_delayed_urbs(port);
diff --git a/drivers/usb/serial/ti_usb_3410_5052.c b/drivers/usb/serial/ti_usb_3410_5052.c
index 2694df2..e7dbbef2 100644
--- a/drivers/usb/serial/ti_usb_3410_5052.c
+++ b/drivers/usb/serial/ti_usb_3410_5052.c
@@ -80,6 +80,7 @@
int td_open_port_count;
struct usb_serial *td_serial;
int td_is_3410;
+ bool td_rs485_only;
int td_urb_error;
};
@@ -160,6 +161,11 @@
{ USB_DEVICE(ABBOTT_VENDOR_ID, ABBOTT_STRIP_PORT_ID) },
{ USB_DEVICE(TI_VENDOR_ID, FRI2_PRODUCT_ID) },
{ USB_DEVICE(HONEYWELL_VENDOR_ID, HONEYWELL_HGI80_PRODUCT_ID) },
+ { USB_DEVICE(MXU1_VENDOR_ID, MXU1_1110_PRODUCT_ID) },
+ { USB_DEVICE(MXU1_VENDOR_ID, MXU1_1130_PRODUCT_ID) },
+ { USB_DEVICE(MXU1_VENDOR_ID, MXU1_1131_PRODUCT_ID) },
+ { USB_DEVICE(MXU1_VENDOR_ID, MXU1_1150_PRODUCT_ID) },
+ { USB_DEVICE(MXU1_VENDOR_ID, MXU1_1151_PRODUCT_ID) },
{ } /* terminator */
};
@@ -193,6 +199,11 @@
{ USB_DEVICE(ABBOTT_VENDOR_ID, ABBOTT_STRIP_PORT_ID) },
{ USB_DEVICE(TI_VENDOR_ID, FRI2_PRODUCT_ID) },
{ USB_DEVICE(HONEYWELL_VENDOR_ID, HONEYWELL_HGI80_PRODUCT_ID) },
+ { USB_DEVICE(MXU1_VENDOR_ID, MXU1_1110_PRODUCT_ID) },
+ { USB_DEVICE(MXU1_VENDOR_ID, MXU1_1130_PRODUCT_ID) },
+ { USB_DEVICE(MXU1_VENDOR_ID, MXU1_1131_PRODUCT_ID) },
+ { USB_DEVICE(MXU1_VENDOR_ID, MXU1_1150_PRODUCT_ID) },
+ { USB_DEVICE(MXU1_VENDOR_ID, MXU1_1151_PRODUCT_ID) },
{ } /* terminator */
};
@@ -277,6 +288,11 @@
MODULE_FIRMWARE("mts_edge.fw");
MODULE_FIRMWARE("mts_mt9234mu.fw");
MODULE_FIRMWARE("mts_mt9234zba.fw");
+MODULE_FIRMWARE("moxa/moxa-1110.fw");
+MODULE_FIRMWARE("moxa/moxa-1130.fw");
+MODULE_FIRMWARE("moxa/moxa-1131.fw");
+MODULE_FIRMWARE("moxa/moxa-1150.fw");
+MODULE_FIRMWARE("moxa/moxa-1151.fw");
module_param(closing_wait, int, S_IRUGO | S_IWUSR);
MODULE_PARM_DESC(closing_wait,
@@ -292,6 +308,9 @@
{
struct ti_device *tdev;
struct usb_device *dev = serial->dev;
+ struct usb_host_interface *cur_altsetting;
+ int num_endpoints;
+ u16 vid, pid;
int status;
dev_dbg(&dev->dev,
@@ -315,8 +334,22 @@
dev_dbg(&dev->dev, "%s - device type is %s\n", __func__,
tdev->td_is_3410 ? "3410" : "5052");
- /* if we have only 1 configuration, download firmware */
- if (dev->descriptor.bNumConfigurations == 1) {
+ vid = le16_to_cpu(dev->descriptor.idVendor);
+ pid = le16_to_cpu(dev->descriptor.idProduct);
+ if (vid == MXU1_VENDOR_ID) {
+ switch (pid) {
+ case MXU1_1130_PRODUCT_ID:
+ case MXU1_1131_PRODUCT_ID:
+ tdev->td_rs485_only = true;
+ break;
+ }
+ }
+
+ cur_altsetting = serial->interface->cur_altsetting;
+ num_endpoints = cur_altsetting->desc.bNumEndpoints;
+
+ /* if we have only 1 configuration and 1 endpoint, download firmware */
+ if (dev->descriptor.bNumConfigurations == 1 && num_endpoints == 1) {
status = ti_download_firmware(tdev);
if (status != 0)
@@ -371,7 +404,11 @@
port->port.closing_wait = msecs_to_jiffies(10 * closing_wait);
tport->tp_port = port;
tport->tp_tdev = usb_get_serial_data(port->serial);
- tport->tp_uart_mode = 0; /* default is RS232 */
+
+ if (tport->tp_tdev->td_rs485_only)
+ tport->tp_uart_mode = TI_UART_485_RECEIVER_DISABLED;
+ else
+ tport->tp_uart_mode = TI_UART_232;
usb_set_serial_port_data(port, tport);
@@ -1450,6 +1487,16 @@
const struct firmware *fw_p;
char buf[32];
+ if (le16_to_cpu(dev->descriptor.idVendor) == MXU1_VENDOR_ID) {
+ snprintf(buf,
+ sizeof(buf),
+ "moxa/moxa-%04x.fw",
+ le16_to_cpu(dev->descriptor.idProduct));
+
+ status = request_firmware(&fw_p, buf, &dev->dev);
+ goto check_firmware;
+ }
+
/* try ID specific firmware first, then try generic firmware */
sprintf(buf, "ti_usb-v%04x-p%04x.fw",
le16_to_cpu(dev->descriptor.idVendor),
@@ -1487,6 +1534,8 @@
}
status = request_firmware(&fw_p, buf, &dev->dev);
}
+
+check_firmware:
if (status) {
dev_err(&dev->dev, "%s - firmware not found\n", __func__);
return -ENOENT;
diff --git a/drivers/usb/serial/ti_usb_3410_5052.h b/drivers/usb/serial/ti_usb_3410_5052.h
index 98f35c6..bbfd3a1 100644
--- a/drivers/usb/serial/ti_usb_3410_5052.h
+++ b/drivers/usb/serial/ti_usb_3410_5052.h
@@ -60,6 +60,14 @@
#define HONEYWELL_VENDOR_ID 0x10ac
#define HONEYWELL_HGI80_PRODUCT_ID 0x0102 /* Honeywell HGI80 */
+/* Moxa UPORT 11x0 vendor and product IDs */
+#define MXU1_VENDOR_ID 0x110a
+#define MXU1_1110_PRODUCT_ID 0x1110
+#define MXU1_1130_PRODUCT_ID 0x1130
+#define MXU1_1131_PRODUCT_ID 0x1131
+#define MXU1_1150_PRODUCT_ID 0x1150
+#define MXU1_1151_PRODUCT_ID 0x1151
+
/* Commands */
#define TI_GET_VERSION 0x01
#define TI_GET_PORT_STATUS 0x02
diff --git a/drivers/usb/serial/usb-serial.c b/drivers/usb/serial/usb-serial.c
index 46f1f13..b1b9bac 100644
--- a/drivers/usb/serial/usb-serial.c
+++ b/drivers/usb/serial/usb-serial.c
@@ -96,7 +96,8 @@
mutex_lock(&table_lock);
for (i = 0; i < num_ports; ++i) {
port = serial->port[i];
- minor = idr_alloc(&serial_minors, port, 0, 0, GFP_KERNEL);
+ minor = idr_alloc(&serial_minors, port, 0,
+ USB_SERIAL_TTY_MINORS, GFP_KERNEL);
if (minor < 0)
goto error;
port->minor = minor;
@@ -254,7 +255,7 @@
*
* Shut down a USB serial port. Serialized against activate by the
* tport mutex and kept to matching open/close pairs
- * of calls by the ASYNCB_INITIALIZED flag.
+ * of calls by the initialized flag.
*
* Not called if tty is console.
*/
@@ -815,7 +816,7 @@
}
}
-#if defined(CONFIG_USB_SERIAL_PL2303) || defined(CONFIG_USB_SERIAL_PL2303_MODULE)
+#if IS_ENABLED(CONFIG_USB_SERIAL_PL2303)
/* BEGIN HORRIBLE HACK FOR PL2303 */
/* this is needed due to the looney way its endpoints are set up */
if (((le16_to_cpu(dev->descriptor.idVendor) == PL2303_VENDOR_ID) &&
diff --git a/drivers/usb/serial/usb_wwan.c b/drivers/usb/serial/usb_wwan.c
index be9cb61..3dfdfc8 100644
--- a/drivers/usb/serial/usb_wwan.c
+++ b/drivers/usb/serial/usb_wwan.c
@@ -464,7 +464,7 @@
/*
* Need to take susp_lock to make sure port is not already being
- * resumed, but no need to hold it due to ASYNC_INITIALIZED.
+ * resumed, but no need to hold it due to initialized
*/
spin_lock_irq(&intfdata->susp_lock);
if (--intfdata->open_ports == 0)
@@ -682,7 +682,7 @@
for (i = 0; i < serial->num_ports; i++) {
port = serial->port[i];
- if (!test_bit(ASYNCB_INITIALIZED, &port->port.flags))
+ if (!tty_port_initialized(&port->port))
continue;
portdata = usb_get_serial_port_data(port);
diff --git a/drivers/usb/storage/alauda.c b/drivers/usb/storage/alauda.c
index 171fa7d..1d8b03c 100644
--- a/drivers/usb/storage/alauda.c
+++ b/drivers/usb/storage/alauda.c
@@ -829,8 +829,10 @@
pba = MEDIA_INFO(us).lba_to_pba[zone][lba_offset];
if (pba == 1) {
- /* Maybe it is impossible to write to PBA 1.
- Fake success, but don't do anything. */
+ /*
+ * Maybe it is impossible to write to PBA 1.
+ * Fake success, but don't do anything.
+ */
printk(KERN_WARNING
"alauda_write_lba: avoid writing to pba 1\n");
return USB_STOR_TRANSPORT_GOOD;
@@ -977,10 +979,12 @@
usb_stor_dbg(us, "Read %d zero pages (LBA %d) page %d\n",
pages, lba, page);
- /* This is not really an error. It just means
- that the block has never been written.
- Instead of returning USB_STOR_TRANSPORT_ERROR
- it is better to return all zero data. */
+ /*
+ * This is not really an error. It just means
+ * that the block has never been written.
+ * Instead of returning USB_STOR_TRANSPORT_ERROR
+ * it is better to return all zero data.
+ */
memset(buffer, 0, len);
} else {
@@ -1222,8 +1226,10 @@
}
if (srb->cmnd[0] == ALLOW_MEDIUM_REMOVAL) {
- /* sure. whatever. not like we can stop the user from popping
- the media out of the device (no locking doors, etc) */
+ /*
+ * sure. whatever. not like we can stop the user from popping
+ * the media out of the device (no locking doors, etc)
+ */
return USB_STOR_TRANSPORT_GOOD;
}
diff --git a/drivers/usb/storage/cypress_atacb.c b/drivers/usb/storage/cypress_atacb.c
index c80d3de..5e4af44 100644
--- a/drivers/usb/storage/cypress_atacb.c
+++ b/drivers/usb/storage/cypress_atacb.c
@@ -110,13 +110,17 @@
/* first build the ATACB command */
srb->cmd_len = 16;
- srb->cmnd[0] = 0x24; /* bVSCBSignature : vendor-specific command
- this value can change, but most(all ?) manufacturers
- keep the cypress default : 0x24 */
+ srb->cmnd[0] = 0x24; /*
+ * bVSCBSignature : vendor-specific command
+ * this value can change, but most(all ?) manufacturers
+ * keep the cypress default : 0x24
+ */
srb->cmnd[1] = 0x24; /* bVSCBSubCommand : 0x24 for ATACB */
- srb->cmnd[3] = 0xff - 1; /* features, sector count, lba low, lba med
- lba high, device, command are valid */
+ srb->cmnd[3] = 0xff - 1; /*
+ * features, sector count, lba low, lba med
+ * lba high, device, command are valid
+ */
srb->cmnd[4] = 1; /* TransferBlockCount : 512 */
if (save_cmnd[0] == ATA_16) {
@@ -155,8 +159,7 @@
usb_stor_transparent_scsi_command(srb, us);
- /* if the device doesn't support ATACB
- */
+ /* if the device doesn't support ATACB */
if (srb->result == SAM_STAT_CHECK_CONDITION &&
memcmp(srb->sense_buffer, usb_stor_sense_invalidCDB,
sizeof(usb_stor_sense_invalidCDB)) == 0) {
@@ -164,7 +167,8 @@
goto end;
}
- /* if ck_cond flags is set, and there wasn't critical error,
+ /*
+ * if ck_cond flags is set, and there wasn't critical error,
* build the special sense
*/
if ((srb->result != (DID_ERROR << 16) &&
@@ -176,11 +180,11 @@
unsigned char *desc = sb + 8;
int tmp_result;
- /* build the command for
- * reading the ATA registers */
+ /* build the command for reading the ATA registers */
scsi_eh_prep_cmnd(srb, &ses, NULL, 0, sizeof(regs));
- /* we use the same command as before, but we set
+ /*
+ * we use the same command as before, but we set
* the read taskfile bit, for not executing atacb command,
* but reading register selected in srb->cmnd[4]
*/
@@ -204,10 +208,11 @@
sb[2] = 0; /* ATA PASS THROUGH INFORMATION AVAILABLE */
sb[3] = 0x1D;
- /* XXX we should generate sk, asc, ascq from status and error
+ /*
+ * XXX we should generate sk, asc, ascq from status and error
* regs
* (see 11.1 Error translation ATA device error to SCSI error
- * map, and ata_to_sense_error from libata.)
+ * map, and ata_to_sense_error from libata.)
*/
/* Sense data is current and format is descriptor. */
@@ -258,7 +263,8 @@
if (result)
return result;
- /* Among CY7C68300 chips, the A revision does not support Cypress ATACB
+ /*
+ * Among CY7C68300 chips, the A revision does not support Cypress ATACB
* Filter out this revision from EEPROM default descriptor values
*/
device = interface_to_usbdev(intf);
diff --git a/drivers/usb/storage/datafab.c b/drivers/usb/storage/datafab.c
index aa4f519..723197a 100644
--- a/drivers/usb/storage/datafab.c
+++ b/drivers/usb/storage/datafab.c
@@ -1,4 +1,5 @@
-/* Driver for Datafab USB Compact Flash reader
+/*
+ * Driver for Datafab USB Compact Flash reader
*
* datafab driver v0.1:
*
@@ -693,18 +694,23 @@
}
if (srb->cmnd[0] == ALLOW_MEDIUM_REMOVAL) {
- // sure. whatever. not like we can stop the user from
- // popping the media out of the device (no locking doors, etc)
- //
+ /*
+ * sure. whatever. not like we can stop the user from
+ * popping the media out of the device (no locking doors, etc)
+ */
return USB_STOR_TRANSPORT_GOOD;
}
if (srb->cmnd[0] == START_STOP) {
- /* this is used by sd.c'check_scsidisk_media_change to detect
- media change */
+ /*
+ * this is used by sd.c'check_scsidisk_media_change to detect
+ * media change
+ */
usb_stor_dbg(us, "START_STOP\n");
- /* the first datafab_id_device after a media change returns
- an error (determined experimentally) */
+ /*
+ * the first datafab_id_device after a media change returns
+ * an error (determined experimentally)
+ */
rc = datafab_id_device(us, info);
if (rc == USB_STOR_TRANSPORT_GOOD) {
info->sense_key = NO_SENSE;
diff --git a/drivers/usb/storage/debug.c b/drivers/usb/storage/debug.c
index 5a12c03..8d20804 100644
--- a/drivers/usb/storage/debug.c
+++ b/drivers/usb/storage/debug.c
@@ -1,4 +1,5 @@
-/* Driver for USB Mass Storage compliant devices
+/*
+ * Driver for USB Mass Storage compliant devices
* Debugging Functions Source Code File
*
* Current development and maintenance by:
diff --git a/drivers/usb/storage/debug.h b/drivers/usb/storage/debug.h
index 6b365ce..8ab7329 100644
--- a/drivers/usb/storage/debug.h
+++ b/drivers/usb/storage/debug.h
@@ -1,4 +1,5 @@
-/* Driver for USB Mass Storage compliant devices
+/*
+ * Driver for USB Mass Storage compliant devices
* Debugging Functions Header File
*
* Current development and maintenance by:
diff --git a/drivers/usb/storage/ene_ub6250.c b/drivers/usb/storage/ene_ub6250.c
index d3a17c6..02bdaa9 100644
--- a/drivers/usb/storage/ene_ub6250.c
+++ b/drivers/usb/storage/ene_ub6250.c
@@ -560,8 +560,10 @@
/* check bulk status */
residue = le32_to_cpu(bcs->Residue);
- /* try to compute the actual residue, based on how much data
- * was really transferred and what the device tells us */
+ /*
+ * try to compute the actual residue, based on how much data
+ * was really transferred and what the device tells us
+ */
if (residue && !(us->fflags & US_FL_IGNORE_RESIDUE)) {
residue = min(residue, transfer_length);
if (us->srb != NULL)
@@ -862,9 +864,6 @@
u8 ExtBuf[4];
u32 bn = PhyBlockAddr * 0x20 + PageNum;
- /* printk(KERN_INFO "MS --- MS_ReaderReadPage,
- PhyBlockAddr = %x, PageNum = %x\n", PhyBlockAddr, PageNum); */
-
result = ene_load_bincode(us, MS_RW_PATTERN);
if (result != USB_STOR_XFER_GOOD)
return USB_STOR_TRANSPORT_ERROR;
@@ -1141,8 +1140,6 @@
struct bulk_cb_wrap *bcb = (struct bulk_cb_wrap *) us->iobuf;
int result;
- /* printk(KERN_INFO "MS_ReaderCopyBlock --- PhyBlockAddr = %x,
- PageNum = %x\n", PhyBlockAddr, PageNum); */
result = ene_load_bincode(us, MS_RW_PATTERN);
if (result != USB_STOR_XFER_GOOD)
return USB_STOR_TRANSPORT_ERROR;
@@ -1176,8 +1173,6 @@
int result;
u32 bn = PhyBlockAddr;
- /* printk(KERN_INFO "MS --- ms_read_eraseblock,
- PhyBlockAddr = %x\n", PhyBlockAddr); */
result = ene_load_bincode(us, MS_RW_PATTERN);
if (result != USB_STOR_XFER_GOOD)
return USB_STOR_TRANSPORT_ERROR;
@@ -1255,8 +1250,6 @@
struct bulk_cb_wrap *bcb = (struct bulk_cb_wrap *) us->iobuf;
int result;
- /* printk("MS --- MS_LibOverwriteExtra,
- PhyBlockAddr = %x, PageNum = %x\n", PhyBlockAddr, PageNum); */
result = ene_load_bincode(us, MS_RW_PATTERN);
if (result != USB_STOR_XFER_GOOD)
return USB_STOR_TRANSPORT_ERROR;
@@ -1342,7 +1335,6 @@
int result;
u8 ExtBuf[4];
- /* printk("MS_LibReadExtra --- PhyBlock = %x, PageNum = %x\n", PhyBlock, PageNum); */
memset(bcb, 0, sizeof(struct bulk_cb_wrap));
bcb->Signature = cpu_to_le32(US_BULK_CB_SIGN);
bcb->DataTransferLength = 0x4;
@@ -1541,9 +1533,6 @@
struct bulk_cb_wrap *bcb = (struct bulk_cb_wrap *) us->iobuf;
int result;
- /* printk("MS_LibReadExtraBlock --- PhyBlock = %x,
- PageNum = %x, blen = %x\n", PhyBlock, PageNum, blen); */
-
/* Read Extra Data */
memset(bcb, 0, sizeof(struct bulk_cb_wrap));
bcb->Signature = cpu_to_le32(US_BULK_CB_SIGN);
@@ -2390,8 +2379,10 @@
/* Report the reset to the SCSI core */
usb_stor_reset_resume(iface);
- /* FIXME: Notify the subdrivers that they need to reinitialize
- * the device */
+ /*
+ * FIXME: Notify the subdrivers that they need to reinitialize
+ * the device
+ */
info->Power_IsResum = true;
/*info->SD_Status.Ready = 0; */
info->SD_Status = *(struct SD_STATUS *)&tmp;
diff --git a/drivers/usb/storage/freecom.c b/drivers/usb/storage/freecom.c
index 3f2b089..c0a5d95 100644
--- a/drivers/usb/storage/freecom.c
+++ b/drivers/usb/storage/freecom.c
@@ -1,4 +1,5 @@
-/* Driver for Freecom USB/IDE adaptor
+/*
+ * Driver for Freecom USB/IDE adaptor
*
* Freecom v0.1:
*
@@ -84,25 +85,33 @@
u8 Pad[60];
};
-/* Freecom stuffs the interrupt status in the INDEX_STAT bit of the ide
- * register. */
+/*
+ * Freecom stuffs the interrupt status in the INDEX_STAT bit of the ide
+ * register.
+ */
#define FCM_INT_STATUS 0x02 /* INDEX_STAT */
#define FCM_STATUS_BUSY 0x80
-/* These are the packet types. The low bit indicates that this command
- * should wait for an interrupt. */
+/*
+ * These are the packet types. The low bit indicates that this command
+ * should wait for an interrupt.
+ */
#define FCM_PACKET_ATAPI 0x21
#define FCM_PACKET_STATUS 0x20
-/* Receive data from the IDE interface. The ATAPI packet has already
- * waited, so the data should be immediately available. */
+/*
+ * Receive data from the IDE interface. The ATAPI packet has already
+ * waited, so the data should be immediately available.
+ */
#define FCM_PACKET_INPUT 0x81
/* Send data to the IDE interface. */
#define FCM_PACKET_OUTPUT 0x01
-/* Write a value to an ide register. Or the ide register to write after
- * munging the address a bit. */
+/*
+ * Write a value to an ide register. Or the ide register to write after
+ * munging the address a bit.
+ */
#define FCM_PACKET_IDE_WRITE 0x40
#define FCM_PACKET_IDE_READ 0xC0
@@ -251,16 +260,20 @@
result = usb_stor_bulk_transfer_buf (us, opipe, fcb,
FCM_PACKET_LENGTH, NULL);
- /* The Freecom device will only fail if there is something wrong in
+ /*
+ * The Freecom device will only fail if there is something wrong in
* USB land. It returns the status in its own registers, which
- * come back in the bulk pipe. */
+ * come back in the bulk pipe.
+ */
if (result != USB_STOR_XFER_GOOD) {
usb_stor_dbg(us, "freecom transport error\n");
return USB_STOR_TRANSPORT_ERROR;
}
- /* There are times we can optimize out this status read, but it
- * doesn't hurt us to always do it now. */
+ /*
+ * There are times we can optimize out this status read, but it
+ * doesn't hurt us to always do it now.
+ */
result = usb_stor_bulk_transfer_buf (us, ipipe, fst,
FCM_STATUS_PACKET_LENGTH, &partial);
usb_stor_dbg(us, "foo Status result %d %u\n", result, partial);
@@ -269,7 +282,8 @@
US_DEBUG(pdump(us, (void *)fst, partial));
- /* The firmware will time-out commands after 20 seconds. Some commands
+ /*
+ * The firmware will time-out commands after 20 seconds. Some commands
* can legitimately take longer than this, so we use a different
* command that only waits for the interrupt and then sends status,
* without having to send a new ATAPI command to the device.
@@ -291,7 +305,8 @@
result = usb_stor_bulk_transfer_buf (us, opipe, fcb,
FCM_PACKET_LENGTH, NULL);
- /* The Freecom device will only fail if there is something
+ /*
+ * The Freecom device will only fail if there is something
* wrong in USB land. It returns the status in its own
* registers, which come back in the bulk pipe.
*/
@@ -318,9 +333,11 @@
return USB_STOR_TRANSPORT_FAILED;
}
- /* The device might not have as much data available as we
+ /*
+ * The device might not have as much data available as we
* requested. If you ask for more than the device has, this reads
- * and such will hang. */
+ * and such will hang.
+ */
usb_stor_dbg(us, "Device indicates that it has %d bytes available\n",
le16_to_cpu(fst->Count));
usb_stor_dbg(us, "SCSI requested %d\n", scsi_bufflen(srb));
@@ -344,16 +361,20 @@
length);
}
- /* What we do now depends on what direction the data is supposed to
- * move in. */
+ /*
+ * What we do now depends on what direction the data is supposed to
+ * move in.
+ */
switch (us->srb->sc_data_direction) {
case DMA_FROM_DEVICE:
/* catch bogus "read 0 length" case */
if (!length)
break;
- /* Make sure that the status indicates that the device
- * wants data as well. */
+ /*
+ * Make sure that the status indicates that the device
+ * wants data as well.
+ */
if ((fst->Status & DRQ_STAT) == 0 || (fst->Reason & 3) != 2) {
usb_stor_dbg(us, "SCSI wants data, drive doesn't have any\n");
return USB_STOR_TRANSPORT_FAILED;
@@ -384,8 +405,10 @@
/* catch bogus "write 0 length" case */
if (!length)
break;
- /* Make sure the status indicates that the device wants to
- * send us data. */
+ /*
+ * Make sure the status indicates that the device wants to
+ * send us data.
+ */
/* !!IMPLEMENT!! */
result = freecom_writedata (srb, us, ipipe, opipe, length);
if (result != USB_STOR_TRANSPORT_GOOD)
@@ -431,7 +454,8 @@
int result;
char *buffer = us->iobuf;
- /* The DMA-mapped I/O buffer is 64 bytes long, just right for
+ /*
+ * The DMA-mapped I/O buffer is 64 bytes long, just right for
* all our packets. No need to allocate any extra buffer space.
*/
@@ -440,7 +464,8 @@
buffer[32] = '\0';
usb_stor_dbg(us, "String returned from FC init is: %s\n", buffer);
- /* Special thanks to the people at Freecom for providing me with
+ /*
+ * Special thanks to the people at Freecom for providing me with
* this "magic sequence", which they use in their Windows and MacOS
* drivers to make sure that all the attached perhiperals are
* properly reset.
diff --git a/drivers/usb/storage/initializers.c b/drivers/usb/storage/initializers.c
index 31fa2e9..d9d8c17 100644
--- a/drivers/usb/storage/initializers.c
+++ b/drivers/usb/storage/initializers.c
@@ -1,4 +1,5 @@
-/* Special Initializers for certain USB Mass Storage devices
+/*
+ * Special Initializers for certain USB Mass Storage devices
*
* Current development and maintenance by:
* (c) 1999, 2000 Matthew Dharm (mdharm-usb@one-eyed-alien.net)
@@ -42,8 +43,10 @@
#include "debug.h"
#include "transport.h"
-/* This places the Shuttle/SCM USB<->SCSI bridge devices in multi-target
- * mode */
+/*
+ * This places the Shuttle/SCM USB<->SCSI bridge devices in multi-target
+ * mode
+ */
int usb_stor_euscsi_init(struct us_data *us)
{
int result;
@@ -57,8 +60,10 @@
return 0;
}
-/* This function is required to activate all four slots on the UCR-61S2B
- * flash reader */
+/*
+ * This function is required to activate all four slots on the UCR-61S2B
+ * flash reader
+ */
int usb_stor_ucr61s2b_init(struct us_data *us)
{
struct bulk_cb_wrap *bcb = (struct bulk_cb_wrap*) us->iobuf;
diff --git a/drivers/usb/storage/initializers.h b/drivers/usb/storage/initializers.h
index 529327f..039abf4 100644
--- a/drivers/usb/storage/initializers.h
+++ b/drivers/usb/storage/initializers.h
@@ -1,4 +1,5 @@
-/* Header file for Special Initializers for certain USB Mass Storage devices
+/*
+ * Header file for Special Initializers for certain USB Mass Storage devices
*
* Current development and maintenance by:
* (c) 1999, 2000 Matthew Dharm (mdharm-usb@one-eyed-alien.net)
@@ -38,12 +39,16 @@
#include "usb.h"
#include "transport.h"
-/* This places the Shuttle/SCM USB<->SCSI bridge devices in multi-target
- * mode */
+/*
+ * This places the Shuttle/SCM USB<->SCSI bridge devices in multi-target
+ * mode
+ */
int usb_stor_euscsi_init(struct us_data *us);
-/* This function is required to activate all four slots on the UCR-61S2B
- * flash reader */
+/*
+ * This function is required to activate all four slots on the UCR-61S2B
+ * flash reader
+ */
int usb_stor_ucr61s2b_init(struct us_data *us);
/* This places the HUAWEI E220 devices in multi-port mode */
diff --git a/drivers/usb/storage/isd200.c b/drivers/usb/storage/isd200.c
index 39afd70..fba4005 100644
--- a/drivers/usb/storage/isd200.c
+++ b/drivers/usb/storage/isd200.c
@@ -1,4 +1,5 @@
-/* Transport & Protocol Driver for In-System Design, Inc. ISD200 ASIC
+/*
+ * Transport & Protocol Driver for In-System Design, Inc. ISD200 ASIC
*
* Current development and maintenance:
* (C) 2001-2002 Björn Stenberg (bjorn@haxx.se)
@@ -628,7 +629,8 @@
srb->cmd_len = sizeof(ataCdb->generic);
transferStatus = usb_stor_Bulk_transport(srb, us);
- /* if the command gets aborted by the higher layers, we need to
+ /*
+ * if the command gets aborted by the higher layers, we need to
* short-circuit all other processing
*/
if (test_bit(US_FLIDX_TIMED_OUT, &us->dflags)) {
@@ -695,15 +697,18 @@
}
}
- /* Regardless of auto-sense, if we _know_ we have an error
+ /*
+ * Regardless of auto-sense, if we _know_ we have an error
* condition, show that in the result code
*/
if (transferStatus == USB_STOR_TRANSPORT_FAILED)
srb->result = SAM_STAT_CHECK_CONDITION;
return;
- /* abort processing: the bulk-only transport requires a reset
- * following an abort */
+ /*
+ * abort processing: the bulk-only transport requires a reset
+ * following an abort
+ */
Handle_Abort:
srb->result = DID_ABORT << 16;
@@ -965,20 +970,22 @@
info->DeviceHead = master_slave;
break;
}
- /* check Cylinder High/Low to
- determine if it is an ATAPI device
- */
+ /*
+ * check Cylinder High/Low to
+ * determine if it is an ATAPI device
+ */
else if (regs[ATA_REG_HCYL_OFFSET] == 0xEB &&
regs[ATA_REG_LCYL_OFFSET] == 0x14) {
- /* It seems that the RICOH
- MP6200A CD/RW drive will
- report itself okay as a
- slave when it is really a
- master. So this check again
- as a master device just to
- make sure it doesn't report
- itself okay as a master also
- */
+ /*
+ * It seems that the RICOH
+ * MP6200A CD/RW drive will
+ * report itself okay as a
+ * slave when it is really a
+ * master. So this check again
+ * as a master device just to
+ * make sure it doesn't report
+ * itself okay as a master also
+ */
if ((master_slave & ATA_ADDRESS_DEVHEAD_SLAVE) &&
!recheckAsMaster) {
usb_stor_dbg(us, " Identified ATAPI device as slave. Rechecking again as master\n");
@@ -1176,9 +1183,11 @@
if (id[ATA_ID_COMMAND_SET_2] & COMMANDSET_MEDIA_STATUS) {
usb_stor_dbg(us, " Device supports Media Status Notification\n");
- /* Indicate that it is enabled, even though it is not
- * This allows the lock/unlock of the media to work
- * correctly.
+ /*
+ * Indicate that it is enabled, even
+ * though it is not.
+ * This allows the lock/unlock of the
+ * media to work correctly.
*/
info->DeviceFlags |= DF_MEDIA_STATUS_ENABLED;
}
@@ -1197,7 +1206,7 @@
usb_stor_dbg(us, "Protocol changed to: %s\n",
us->protocol_name);
- /* Free driver structure */
+ /* Free driver structure */
us->extra_destructor(info);
kfree(info);
us->extra = NULL;
diff --git a/drivers/usb/storage/jumpshot.c b/drivers/usb/storage/jumpshot.c
index ee613e2..011e527 100644
--- a/drivers/usb/storage/jumpshot.c
+++ b/drivers/usb/storage/jumpshot.c
@@ -1,4 +1,5 @@
-/* Driver for Lexar "Jumpshot" Compact Flash reader
+/*
+ * Driver for Lexar "Jumpshot" Compact Flash reader
*
* jumpshot driver v0.1:
*
@@ -618,18 +619,23 @@
}
if (srb->cmnd[0] == ALLOW_MEDIUM_REMOVAL) {
- // sure. whatever. not like we can stop the user from popping
- // the media out of the device (no locking doors, etc)
- //
+ /*
+ * sure. whatever. not like we can stop the user from popping
+ * the media out of the device (no locking doors, etc)
+ */
return USB_STOR_TRANSPORT_GOOD;
}
if (srb->cmnd[0] == START_STOP) {
- /* this is used by sd.c'check_scsidisk_media_change to detect
- media change */
+ /*
+ * this is used by sd.c'check_scsidisk_media_change to detect
+ * media change
+ */
usb_stor_dbg(us, "START_STOP\n");
- /* the first jumpshot_id_device after a media change returns
- an error (determined experimentally) */
+ /*
+ * the first jumpshot_id_device after a media change returns
+ * an error (determined experimentally)
+ */
rc = jumpshot_id_device(us, info);
if (rc == USB_STOR_TRANSPORT_GOOD) {
info->sense_key = NO_SENSE;
diff --git a/drivers/usb/storage/karma.c b/drivers/usb/storage/karma.c
index ae201e6..f9d407f 100644
--- a/drivers/usb/storage/karma.c
+++ b/drivers/usb/storage/karma.c
@@ -1,4 +1,5 @@
-/* Driver for Rio Karma
+/*
+ * Driver for Rio Karma
*
* (c) 2006 Bob Copeland <me@bobcopeland.com>
* (c) 2006 Keith Bennett <keith@mcs.st-and.ac.uk>
diff --git a/drivers/usb/storage/option_ms.c b/drivers/usb/storage/option_ms.c
index b2b35b1..57282f1 100644
--- a/drivers/usb/storage/option_ms.c
+++ b/drivers/usb/storage/option_ms.c
@@ -65,7 +65,8 @@
goto out;
}
- /* Some of the devices need to be asked for a response, but we don't
+ /*
+ * Some of the devices need to be asked for a response, but we don't
* care what that response is.
*/
usb_stor_bulk_transfer_buf(us,
@@ -140,7 +141,8 @@
usb_stor_dbg(us, "Option MS: %s\n", "option_ms_init called");
- /* Additional test for vendor information via INQUIRY,
+ /*
+ * Additional test for vendor information via INQUIRY,
* because some vendor/product IDs are ambiguous
*/
result = option_inquiry(us);
diff --git a/drivers/usb/storage/protocol.c b/drivers/usb/storage/protocol.c
index 12e3c2f..74c3887 100644
--- a/drivers/usb/storage/protocol.c
+++ b/drivers/usb/storage/protocol.c
@@ -1,4 +1,5 @@
-/* Driver for USB Mass Storage compliant devices
+/*
+ * Driver for USB Mass Storage compliant devices
*
* Current development and maintenance by:
* (c) 1999-2002 Matthew Dharm (mdharm-usb@one-eyed-alien.net)
@@ -75,7 +76,8 @@
void usb_stor_ufi_command(struct scsi_cmnd *srb, struct us_data *us)
{
- /* fix some commands -- this is a form of mode translation
+ /*
+ * fix some commands -- this is a form of mode translation
* UFI devices only accept 12 byte long commands
*
* NOTE: This only works because a scsi_cmnd struct field contains
@@ -127,7 +129,8 @@
* Scatter-gather transfer buffer access routines
***********************************************************************/
-/* Copy a buffer of length buflen to/from the srb's transfer buffer.
+/*
+ * Copy a buffer of length buflen to/from the srb's transfer buffer.
* Update the **sgptr and *offset variables so that the next copy will
* pick up from where this one left off.
*/
@@ -175,7 +178,8 @@
}
EXPORT_SYMBOL_GPL(usb_stor_access_xfer_buf);
-/* Store the contents of buffer into srb's transfer buffer and set the
+/*
+ * Store the contents of buffer into srb's transfer buffer and set the
* SCSI residue.
*/
void usb_stor_set_xfer_buf(unsigned char *buffer,
diff --git a/drivers/usb/storage/protocol.h b/drivers/usb/storage/protocol.h
index ffc3e2a..a556668 100644
--- a/drivers/usb/storage/protocol.h
+++ b/drivers/usb/storage/protocol.h
@@ -1,4 +1,5 @@
-/* Driver for USB Mass Storage compliant devices
+/*
+ * Driver for USB Mass Storage compliant devices
* Protocol Functions Header File
*
* Current development and maintenance by:
diff --git a/drivers/usb/storage/realtek_cr.c b/drivers/usb/storage/realtek_cr.c
index 2043356..4176d1a 100644
--- a/drivers/usb/storage/realtek_cr.c
+++ b/drivers/usb/storage/realtek_cr.c
@@ -1,4 +1,5 @@
-/* Driver for Realtek RTS51xx USB card reader
+/*
+ * Driver for Realtek RTS51xx USB card reader
*
* Copyright(c) 2009 Realtek Semiconductor Corp. All rights reserved.
*
@@ -267,8 +268,10 @@
if (bcs->Tag != us->tag)
return USB_STOR_TRANSPORT_ERROR;
- /* try to compute the actual residue, based on how much data
- * was really transferred and what the device tells us */
+ /*
+ * try to compute the actual residue, based on how much data
+ * was really transferred and what the device tells us
+ */
if (residue)
residue = residue < buf_len ? residue : buf_len;
@@ -286,7 +289,8 @@
return USB_STOR_TRANSPORT_FAILED;
case US_BULK_STAT_PHASE:
- /* phase error -- note that a transport reset will be
+ /*
+ * phase error -- note that a transport reset will be
* invoked by the invoke_transport() function
*/
return USB_STOR_TRANSPORT_ERROR;
diff --git a/drivers/usb/storage/scsiglue.c b/drivers/usb/storage/scsiglue.c
index 9090186..33eb923 100644
--- a/drivers/usb/storage/scsiglue.c
+++ b/drivers/usb/storage/scsiglue.c
@@ -1,4 +1,5 @@
-/* Driver for USB Mass Storage compliant devices
+/*
+ * Driver for USB Mass Storage compliant devices
* SCSI layer glue code
*
* Current development and maintenance by:
@@ -58,7 +59,8 @@
#include "transport.h"
#include "protocol.h"
-/* Vendor IDs for companies that seem to include the READ CAPACITY bug
+/*
+ * Vendor IDs for companies that seem to include the READ CAPACITY bug
* in all their devices
*/
#define VENDOR_ID_NOKIA 0x0421
@@ -87,7 +89,8 @@
*/
sdev->inquiry_len = 36;
- /* USB has unusual DMA-alignment requirements: Although the
+ /*
+ * USB has unusual DMA-alignment requirements: Although the
* starting address of each scatter-gather element doesn't matter,
* the length of each element except the last must be divisible
* by the Bulk maxpacket value. There's currently no way to
@@ -115,7 +118,8 @@
{
struct us_data *us = host_to_us(sdev->host);
- /* Many devices have trouble transferring more than 32KB at a time,
+ /*
+ * Many devices have trouble transferring more than 32KB at a time,
* while others have trouble with more than 64K. At this time we
* are limiting both to 32K (64 sectores).
*/
@@ -128,14 +132,22 @@
blk_queue_max_hw_sectors(sdev->request_queue,
max_sectors);
} else if (sdev->type == TYPE_TAPE) {
- /* Tapes need much higher max_sector limits, so just
+ /*
+ * Tapes need much higher max_sector limits, so just
* raise it to the maximum possible (4 GB / 512) and
* let the queue segment size sort out the real limit.
*/
blk_queue_max_hw_sectors(sdev->request_queue, 0x7FFFFF);
+ } else if (us->pusb_dev->speed >= USB_SPEED_SUPER) {
+ /*
+ * USB3 devices will be limited to 2048 sectors. This gives us
+ * better throughput on most devices.
+ */
+ blk_queue_max_hw_sectors(sdev->request_queue, 2048);
}
- /* Some USB host controllers can't do DMA; they have to use PIO.
+ /*
+ * Some USB host controllers can't do DMA; they have to use PIO.
* They indicate this by setting their dma_mask to NULL. For
* such controllers we need to make sure the block layer sets
* up bounce buffers in addressable memory.
@@ -143,17 +155,21 @@
if (!us->pusb_dev->bus->controller->dma_mask)
blk_queue_bounce_limit(sdev->request_queue, BLK_BOUNCE_HIGH);
- /* We can't put these settings in slave_alloc() because that gets
+ /*
+ * We can't put these settings in slave_alloc() because that gets
* called before the device type is known. Consequently these
- * settings can't be overridden via the scsi devinfo mechanism. */
+ * settings can't be overridden via the scsi devinfo mechanism.
+ */
if (sdev->type == TYPE_DISK) {
- /* Some vendors seem to put the READ CAPACITY bug into
+ /*
+ * Some vendors seem to put the READ CAPACITY bug into
* all their devices -- primarily makers of cell phones
* and digital cameras. Since these devices always use
* flash media and can be expected to have an even number
* of sectors, we will always enable the CAPACITY_HEURISTICS
- * flag unless told otherwise. */
+ * flag unless told otherwise.
+ */
switch (le16_to_cpu(us->pusb_dev->descriptor.idVendor)) {
case VENDOR_ID_NOKIA:
case VENDOR_ID_NIKON:
@@ -165,28 +181,36 @@
break;
}
- /* Disk-type devices use MODE SENSE(6) if the protocol
+ /*
+ * Disk-type devices use MODE SENSE(6) if the protocol
* (SubClass) is Transparent SCSI, otherwise they use
- * MODE SENSE(10). */
+ * MODE SENSE(10).
+ */
if (us->subclass != USB_SC_SCSI && us->subclass != USB_SC_CYP_ATACB)
sdev->use_10_for_ms = 1;
- /* Many disks only accept MODE SENSE transfer lengths of
- * 192 bytes (that's what Windows uses). */
+ /*
+ *Many disks only accept MODE SENSE transfer lengths of
+ * 192 bytes (that's what Windows uses).
+ */
sdev->use_192_bytes_for_3f = 1;
- /* Some devices don't like MODE SENSE with page=0x3f,
+ /*
+ * Some devices don't like MODE SENSE with page=0x3f,
* which is the command used for checking if a device
* is write-protected. Now that we tell the sd driver
* to do a 192-byte transfer with this command the
* majority of devices work fine, but a few still can't
* handle it. The sd driver will simply assume those
- * devices are write-enabled. */
+ * devices are write-enabled.
+ */
if (us->fflags & US_FL_NO_WP_DETECT)
sdev->skip_ms_page_3f = 1;
- /* A number of devices have problems with MODE SENSE for
- * page x08, so we will skip it. */
+ /*
+ * A number of devices have problems with MODE SENSE for
+ * page x08, so we will skip it.
+ */
sdev->skip_ms_page_8 = 1;
/* Some devices don't handle VPD pages correctly */
@@ -198,15 +222,19 @@
/* Do not attempt to use WRITE SAME */
sdev->no_write_same = 1;
- /* Some disks return the total number of blocks in response
+ /*
+ * Some disks return the total number of blocks in response
* to READ CAPACITY rather than the highest block number.
- * If this device makes that mistake, tell the sd driver. */
+ * If this device makes that mistake, tell the sd driver.
+ */
if (us->fflags & US_FL_FIX_CAPACITY)
sdev->fix_capacity = 1;
- /* A few disks have two indistinguishable version, one of
+ /*
+ * A few disks have two indistinguishable version, one of
* which reports the correct capacity and the other does not.
- * The sd driver has to guess which is the case. */
+ * The sd driver has to guess which is the case.
+ */
if (us->fflags & US_FL_CAPACITY_HEURISTICS)
sdev->guess_capacity = 1;
@@ -227,26 +255,34 @@
if (sdev->scsi_level > SCSI_SPC_2)
us->fflags |= US_FL_SANE_SENSE;
- /* USB-IDE bridges tend to report SK = 0x04 (Non-recoverable
+ /*
+ * USB-IDE bridges tend to report SK = 0x04 (Non-recoverable
* Hardware Error) when any low-level error occurs,
* recoverable or not. Setting this flag tells the SCSI
* midlayer to retry such commands, which frequently will
* succeed and fix the error. The worst this can lead to
- * is an occasional series of retries that will all fail. */
+ * is an occasional series of retries that will all fail.
+ */
sdev->retry_hwerror = 1;
- /* USB disks should allow restart. Some drives spin down
- * automatically, requiring a START-STOP UNIT command. */
+ /*
+ * USB disks should allow restart. Some drives spin down
+ * automatically, requiring a START-STOP UNIT command.
+ */
sdev->allow_restart = 1;
- /* Some USB cardreaders have trouble reading an sdcard's last
+ /*
+ * Some USB cardreaders have trouble reading an sdcard's last
* sector in a larger then 1 sector read, since the performance
- * impact is negligible we set this flag for all USB disks */
+ * impact is negligible we set this flag for all USB disks
+ */
sdev->last_sector_bug = 1;
- /* Enable last-sector hacks for single-target devices using
+ /*
+ * Enable last-sector hacks for single-target devices using
* the Bulk-only transport, unless we already know the
- * capacity will be decremented or is correct. */
+ * capacity will be decremented or is correct.
+ */
if (!(us->fflags & (US_FL_FIX_CAPACITY | US_FL_CAPACITY_OK |
US_FL_SCM_MULT_TARG)) &&
us->protocol == USB_PR_BULK)
@@ -262,9 +298,11 @@
} else {
- /* Non-disk-type devices don't need to blacklist any pages
+ /*
+ * Non-disk-type devices don't need to blacklist any pages
* or to force 192-byte transfer lengths for MODE SENSE.
- * But they do need to use MODE SENSE(10). */
+ * But they do need to use MODE SENSE(10).
+ */
sdev->use_10_for_ms = 1;
/* Some (fake) usb cdrom devices don't like READ_DISC_INFO */
@@ -272,7 +310,8 @@
sdev->no_read_disc_info = 1;
}
- /* The CB and CBI transports have no way to pass LUN values
+ /*
+ * The CB and CBI transports have no way to pass LUN values
* other than the bits in the second byte of a CDB. But those
* bits don't get set to the LUN value if the device reports
* scsi_level == 0 (UNKNOWN). Hence such devices must necessarily
@@ -282,13 +321,17 @@
sdev->scsi_level == SCSI_UNKNOWN)
us->max_lun = 0;
- /* Some devices choke when they receive a PREVENT-ALLOW MEDIUM
- * REMOVAL command, so suppress those commands. */
+ /*
+ * Some devices choke when they receive a PREVENT-ALLOW MEDIUM
+ * REMOVAL command, so suppress those commands.
+ */
if (us->fflags & US_FL_NOT_LOCKABLE)
sdev->lockable = 0;
- /* this is to satisfy the compiler, tho I don't think the
- * return code is ever checked anywhere. */
+ /*
+ * this is to satisfy the compiler, tho I don't think the
+ * return code is ever checked anywhere.
+ */
return 0;
}
@@ -362,8 +405,10 @@
usb_stor_dbg(us, "%s called\n", __func__);
- /* us->srb together with the TIMED_OUT, RESETTING, and ABORTING
- * bits are protected by the host lock. */
+ /*
+ * us->srb together with the TIMED_OUT, RESETTING, and ABORTING
+ * bits are protected by the host lock.
+ */
scsi_lock(us_to_host(us));
/* Is this command still active? */
@@ -373,11 +418,13 @@
return FAILED;
}
- /* Set the TIMED_OUT bit. Also set the ABORTING bit, but only if
+ /*
+ * Set the TIMED_OUT bit. Also set the ABORTING bit, but only if
* a device reset isn't already in progress (to avoid interfering
* with the reset). Note that we must retain the host lock while
* calling usb_stor_stop_transport(); otherwise it might interfere
- * with an auto-reset that begins as soon as we release the lock. */
+ * with an auto-reset that begins as soon as we release the lock.
+ */
set_bit(US_FLIDX_TIMED_OUT, &us->dflags);
if (!test_bit(US_FLIDX_RESETTING, &us->dflags)) {
set_bit(US_FLIDX_ABORTING, &us->dflags);
@@ -390,8 +437,10 @@
return SUCCESS;
}
-/* This invokes the transport reset mechanism to reset the state of the
- * device */
+/*
+ * This invokes the transport reset mechanism to reset the state of the
+ * device
+ */
static int device_reset(struct scsi_cmnd *srb)
{
struct us_data *us = host_to_us(srb->device->host);
@@ -419,9 +468,11 @@
return result < 0 ? FAILED : SUCCESS;
}
-/* Report a driver-initiated device reset to the SCSI layer.
+/*
+ * Report a driver-initiated device reset to the SCSI layer.
* Calling this for a SCSI-initiated reset is unnecessary but harmless.
- * The caller must own the SCSI host lock. */
+ * The caller must own the SCSI host lock.
+ */
void usb_stor_report_device_reset(struct us_data *us)
{
int i;
@@ -434,9 +485,11 @@
}
}
-/* Report a driver-initiated bus reset to the SCSI layer.
+/*
+ * Report a driver-initiated bus reset to the SCSI layer.
* Calling this for a SCSI-initiated reset is unnecessary but harmless.
- * The caller must not own the SCSI host lock. */
+ * The caller must not own the SCSI host lock.
+ */
void usb_stor_report_bus_reset(struct us_data *us)
{
struct Scsi_Host *host = us_to_host(us);
@@ -563,12 +616,30 @@
.target_alloc = target_alloc,
/* lots of sg segments can be handled */
- .sg_tablesize = SCSI_MAX_SG_CHAIN_SEGMENTS,
+ .sg_tablesize = SG_MAX_SEGMENTS,
- /* limit the total size of a transfer to 120 KB */
+
+ /*
+ * Limit the total size of a transfer to 120 KB.
+ *
+ * Some devices are known to choke with anything larger. It seems like
+ * the problem stems from the fact that original IDE controllers had
+ * only an 8-bit register to hold the number of sectors in one transfer
+ * and even those couldn't handle a full 256 sectors.
+ *
+ * Because we want to make sure we interoperate with as many devices as
+ * possible, we will maintain a 240 sector transfer size limit for USB
+ * Mass Storage devices.
+ *
+ * Tests show that other operating have similar limits with Microsoft
+ * Windows 7 limiting transfers to 128 sectors for both USB2 and USB3
+ * and Apple Mac OS X 10.11 limiting transfers to 256 sectors for USB2
+ * and 2048 for USB3 devices.
+ */
.max_sectors = 240,
- /* merge commands... this seems to help performance, but
+ /*
+ * merge commands... this seems to help performance, but
* periodically someone should test to see which setting is more
* optimal.
*/
diff --git a/drivers/usb/storage/scsiglue.h b/drivers/usb/storage/scsiglue.h
index 5494d87..d0a331d 100644
--- a/drivers/usb/storage/scsiglue.h
+++ b/drivers/usb/storage/scsiglue.h
@@ -1,4 +1,5 @@
-/* Driver for USB Mass Storage compliant devices
+/*
+ * Driver for USB Mass Storage compliant devices
* SCSI Connecting Glue Header File
*
* Current development and maintenance by:
diff --git a/drivers/usb/storage/sddr09.c b/drivers/usb/storage/sddr09.c
index 79224fc..c5797fa 100644
--- a/drivers/usb/storage/sddr09.c
+++ b/drivers/usb/storage/sddr09.c
@@ -1,4 +1,5 @@
-/* Driver for SanDisk SDDR-09 SmartMedia reader
+/*
+ * Driver for SanDisk SDDR-09 SmartMedia reader
*
* (c) 2000, 2001 Robert Baruch (autophile@starband.net)
* (c) 2002 Andries Brouwer (aeb@cwi.nl)
@@ -799,10 +800,12 @@
usb_stor_dbg(us, "Read %d zero pages (LBA %d) page %d\n",
pages, lba, page);
- /* This is not really an error. It just means
- that the block has never been written.
- Instead of returning an error
- it is better to return all zero data. */
+ /*
+ * This is not really an error. It just means
+ * that the block has never been written.
+ * Instead of returning an error
+ * it is better to return all zero data.
+ */
memset(buffer, 0, len);
@@ -890,8 +893,10 @@
}
if (pba == 1) {
- /* Maybe it is impossible to write to PBA 1.
- Fake success, but don't do anything. */
+ /*
+ * Maybe it is impossible to write to PBA 1.
+ * Fake success, but don't do anything.
+ */
printk(KERN_WARNING "sddr09: avoid writing to pba 1\n");
return 0;
}
@@ -979,18 +984,22 @@
struct scatterlist *sg;
int result;
- // Figure out the initial LBA and page
+ /* Figure out the initial LBA and page */
lba = address >> info->blockshift;
page = (address & info->blockmask);
maxlba = info->capacity >> (info->pageshift + info->blockshift);
if (lba >= maxlba)
return -EIO;
- // blockbuffer is used for reading in the old data, overwriting
- // with the new data, and performing ECC calculations
+ /*
+ * blockbuffer is used for reading in the old data, overwriting
+ * with the new data, and performing ECC calculations
+ */
- /* TODO: instead of doing kmalloc/kfree for each write,
- add a bufferpointer to the info structure */
+ /*
+ * TODO: instead of doing kmalloc/kfree for each write,
+ * add a bufferpointer to the info structure
+ */
pagelen = (1 << info->pageshift) + (1 << CONTROL_SHIFT);
blocklen = (pagelen << info->blockshift);
@@ -1000,9 +1009,11 @@
return -ENOMEM;
}
- // Since we don't write the user data directly to the device,
- // we have to create a bounce buffer and move the data a piece
- // at a time between the bounce buffer and the actual transfer buffer.
+ /*
+ * Since we don't write the user data directly to the device,
+ * we have to create a bounce buffer and move the data a piece
+ * at a time between the bounce buffer and the actual transfer buffer.
+ */
len = min(sectors, (unsigned int) info->blocksize) * info->pagesize;
buffer = kmalloc(len, GFP_NOIO);
@@ -1018,7 +1029,7 @@
while (sectors > 0) {
- // Write as many sectors as possible in this block
+ /* Write as many sectors as possible in this block */
pages = min(sectors, info->blocksize - page);
len = (pages << info->pageshift);
@@ -1031,7 +1042,7 @@
break;
}
- // Get the data from the transfer buffer
+ /* Get the data from the transfer buffer */
usb_stor_access_xfer_buf(buffer, len, us->srb,
&sg, &offset, FROM_XFER_BUF);
@@ -1168,9 +1179,11 @@
/* Byte 1 is the device type */
cardinfo = nand_find_id(deviceID[1]);
if (cardinfo) {
- /* MB or MiB? It is neither. A 16 MB card has
- 17301504 raw bytes, of which 16384000 are
- usable for user data. */
+ /*
+ * MB or MiB? It is neither. A 16 MB card has
+ * 17301504 raw bytes, of which 16384000 are
+ * usable for user data.
+ */
sprintf(blurbtxt + strlen(blurbtxt),
", %d MB", 1<<(cardinfo->chipshift - 20));
} else {
@@ -1211,14 +1224,18 @@
if (!info->capacity)
return -1;
- // size of a block is 1 << (blockshift + pageshift) bytes
- // divide into the total capacity to get the number of blocks
+ /*
+ * size of a block is 1 << (blockshift + pageshift) bytes
+ * divide into the total capacity to get the number of blocks
+ */
numblocks = info->capacity >> (info->blockshift + info->pageshift);
- // read 64 bytes for every block (actually 1 << CONTROL_SHIFT)
- // but only use a 64 KB buffer
- // buffer size used must be a multiple of (1 << CONTROL_SHIFT)
+ /*
+ * read 64 bytes for every block (actually 1 << CONTROL_SHIFT)
+ * but only use a 64 KB buffer
+ * buffer size used must be a multiple of (1 << CONTROL_SHIFT)
+ */
#define SDDR09_READ_MAP_BUFSZ 65536
alloc_blocks = min(numblocks, SDDR09_READ_MAP_BUFSZ >> CONTROL_SHIFT);
@@ -1575,8 +1592,10 @@
havefakesense = 1;
- /* Dummy up a response for INQUIRY since SDDR09 doesn't
- respond to INQUIRY commands */
+ /*
+ * Dummy up a response for INQUIRY since SDDR09 doesn't
+ * respond to INQUIRY commands
+ */
if (srb->cmnd[0] == INQUIRY) {
memcpy(ptr, inquiry_response, 8);
@@ -1628,8 +1647,10 @@
if (srb->cmnd[0] == MODE_SENSE_10) {
int modepage = (srb->cmnd[2] & 0x3F);
- /* They ask for the Read/Write error recovery page,
- or for all pages. */
+ /*
+ * They ask for the Read/Write error recovery page,
+ * or for all pages.
+ */
/* %% We should check DBD %% */
if (modepage == 0x01 || modepage == 0x3F) {
usb_stor_dbg(us, "Dummy up request for mode page 0x%x\n",
@@ -1682,7 +1703,8 @@
USB_STOR_TRANSPORT_ERROR);
}
- /* catch-all for all other commands, except
+ /*
+ * catch-all for all other commands, except
* pass TEST_UNIT_READY and REQUEST_SENSE through
*/
if (srb->cmnd[0] != TEST_UNIT_READY &&
diff --git a/drivers/usb/storage/sddr55.c b/drivers/usb/storage/sddr55.c
index e5e0a25..147c50b 100644
--- a/drivers/usb/storage/sddr55.c
+++ b/drivers/usb/storage/sddr55.c
@@ -1,4 +1,5 @@
-/* Driver for SanDisk SDDR-55 SmartMedia reader
+/*
+ * Driver for SanDisk SDDR-55 SmartMedia reader
*
* SDDR55 driver v0.1:
*
@@ -130,7 +131,8 @@
return usb_stor_bulk_transfer_buf(us, pipe, data, len, NULL);
}
-/* check if card inserted, if there is, update read_only status
+/*
+ * check if card inserted, if there is, update read_only status
* return non zero if no card
*/
@@ -714,15 +716,18 @@
if (max_lba > 1000)
max_lba = 1000;
- // Each block is 64 bytes of control data, so block i is located in
- // scatterlist block i*64/128k = i*(2^6)*(2^-17) = i*(2^-11)
+ /*
+ * Each block is 64 bytes of control data, so block i is located in
+ * scatterlist block i*64/128k = i*(2^6)*(2^-17) = i*(2^-11)
+ */
for (i=0; i<numblocks; i++) {
int zone = i / 1024;
lba = short_pack(buffer[i * 2], buffer[i * 2 + 1]);
- /* Every 1024 physical blocks ("zone"), the LBA numbers
+ /*
+ * Every 1024 physical blocks ("zone"), the LBA numbers
* go back to zero, but are within a higher
* block of LBA's. Also, there is a maximum of
* 1000 LBA's per zone. In other words, in PBA
@@ -733,7 +738,8 @@
* are 24 spare blocks to use when blocks do go bad.
*/
- /* SDDR55 returns 0xffff for a bad block, and 0x400 for the
+ /*
+ * SDDR55 returns 0xffff for a bad block, and 0x400 for the
* CIS block. (Is this true for cards 8MB or less??)
* Record these in the physical to logical map
*/
@@ -824,8 +830,10 @@
memset (info->sense_data, 0, sizeof info->sense_data);
- /* Dummy up a response for INQUIRY since SDDR55 doesn't
- respond to INQUIRY commands */
+ /*
+ * Dummy up a response for INQUIRY since SDDR55 doesn't
+ * respond to INQUIRY commands
+ */
if (srb->cmnd[0] == INQUIRY) {
memcpy(ptr, inquiry_response, 8);
@@ -833,7 +841,8 @@
return USB_STOR_TRANSPORT_GOOD;
}
- /* only check card status if the map isn't allocated, ie no card seen yet
+ /*
+ * only check card status if the map isn't allocated, ie no card seen yet
* or if it's been over half a second since we last accessed it
*/
if (info->lba_to_pba == NULL || time_after(jiffies, info->last_access + HZ/2)) {
@@ -849,8 +858,10 @@
}
}
- /* if we detected a problem with the map when writing,
- don't allow any more access */
+ /*
+ * if we detected a problem with the map when writing,
+ * don't allow any more access
+ */
if (info->fatal_error) {
set_sense_info (3, 0x31, 0);
@@ -868,12 +879,16 @@
info->capacity = capacity;
- /* figure out the maximum logical block number, allowing for
- * the fact that only 250 out of every 256 are used */
+ /*
+ * figure out the maximum logical block number, allowing for
+ * the fact that only 250 out of every 256 are used
+ */
info->max_log_blks = ((info->capacity >> (info->pageshift + info->blockshift)) / 256) * 250;
- /* Last page in the card, adjust as we only use 250 out of
- * every 256 pages */
+ /*
+ * Last page in the card, adjust as we only use 250 out of
+ * every 256 pages
+ */
capacity = (capacity / 256) * 250;
capacity /= PAGESIZE;
diff --git a/drivers/usb/storage/shuttle_usbat.c b/drivers/usb/storage/shuttle_usbat.c
index a3ec86b..3b0294e4 100644
--- a/drivers/usb/storage/shuttle_usbat.c
+++ b/drivers/usb/storage/shuttle_usbat.c
@@ -1,4 +1,5 @@
-/* Driver for SCM Microsystems (a.k.a. Shuttle) USB-ATAPI cable
+/*
+ * Driver for SCM Microsystems (a.k.a. Shuttle) USB-ATAPI cable
*
* Current development and maintenance by:
* (c) 2000, 2001 Robert Baruch (autophile@starband.net)
@@ -408,7 +409,8 @@
int result;
unsigned char *status = us->iobuf;
- /* Synchronizing cache on a CDR could take a heck of a long time,
+ /*
+ * Synchronizing cache on a CDR could take a heck of a long time,
* but probably not more than 10 minutes or so. On the other hand,
* doing a full blank on a CDRW at speed 1 will take about 75
* minutes!
@@ -1570,9 +1572,10 @@
len = scsi_bufflen(srb);
- /* Send A0 (ATA PACKET COMMAND).
- Note: I guess we're never going to get any of the ATA
- commands... just ATA Packet Commands.
+ /*
+ * Send A0 (ATA PACKET COMMAND).
+ * Note: I guess we're never going to get any of the ATA
+ * commands... just ATA Packet Commands.
*/
registers[0] = USBAT_ATA_FEATURES;
@@ -1851,7 +1854,8 @@
if (result)
return result;
- /* The actual transport will be determined later by the
+ /*
+ * The actual transport will be determined later by the
* initialization routine; this is just a placeholder.
*/
us->transport_name = "Shuttle USBAT";
diff --git a/drivers/usb/storage/sierra_ms.c b/drivers/usb/storage/sierra_ms.c
index 2ea657b..9a51019 100644
--- a/drivers/usb/storage/sierra_ms.c
+++ b/drivers/usb/storage/sierra_ms.c
@@ -177,7 +177,8 @@
debug_swoc(&us->pusb_dev->dev, swocInfo);
- /* If there is not Linux software on the TRU-Install device
+ /*
+ * If there is not Linux software on the TRU-Install device
* then switch to modem mode
*/
if (!containsFullLinuxPackage(swocInfo)) {
diff --git a/drivers/usb/storage/transport.c b/drivers/usb/storage/transport.c
index 5e67f63..ffd0867 100644
--- a/drivers/usb/storage/transport.c
+++ b/drivers/usb/storage/transport.c
@@ -1,4 +1,5 @@
-/* Driver for USB Mass Storage compliant devices
+/*
+ * Driver for USB Mass Storage compliant devices
*
* Current development and maintenance by:
* (c) 1999-2002 Matthew Dharm (mdharm-usb@one-eyed-alien.net)
@@ -109,7 +110,8 @@
* called more than once or from being called during usb_submit_urb().
*/
-/* This is the completion handler which will wake us up when an URB
+/*
+ * This is the completion handler which will wake us up when an URB
* completes.
*/
static void usb_stor_blocking_completion(struct urb *urb)
@@ -119,7 +121,8 @@
complete(urb_done_ptr);
}
-/* This is the common part of the URB message submission code
+/*
+ * This is the common part of the URB message submission code
*
* All URBs from the usb-storage driver involved in handling a queued scsi
* command _must_ pass through this function (or something like it) for the
@@ -142,10 +145,12 @@
us->current_urb->context = &urb_done;
us->current_urb->transfer_flags = 0;
- /* we assume that if transfer_buffer isn't us->iobuf then it
+ /*
+ * we assume that if transfer_buffer isn't us->iobuf then it
* hasn't been mapped for DMA. Yes, this is clunky, but it's
* easier than always having the caller tell us whether the
- * transfer buffer has already been mapped. */
+ * transfer buffer has already been mapped.
+ */
if (us->current_urb->transfer_buffer == us->iobuf)
us->current_urb->transfer_flags |= URB_NO_TRANSFER_DMA_MAP;
us->current_urb->transfer_dma = us->iobuf_dma;
@@ -157,8 +162,10 @@
return status;
}
- /* since the URB has been submitted successfully, it's now okay
- * to cancel it */
+ /*
+ * since the URB has been submitted successfully, it's now okay
+ * to cancel it
+ */
set_bit(US_FLIDX_URB_ACTIVE, &us->dflags);
/* did an abort occur during the submission? */
@@ -220,7 +227,8 @@
}
EXPORT_SYMBOL_GPL(usb_stor_control_msg);
-/* This is a version of usb_clear_halt() that allows early termination and
+/*
+ * This is a version of usb_clear_halt() that allows early termination and
* doesn't read the status from the device -- this is because some devices
* crash their internal firmware when the status is requested after a halt.
*
@@ -280,8 +288,10 @@
/* stalled */
case -EPIPE:
- /* for control endpoints, (used by CB[I]) a stall indicates
- * a failed command */
+ /*
+ * for control endpoints, (used by CB[I]) a stall indicates
+ * a failed command
+ */
if (usb_pipecontrol(pipe)) {
usb_stor_dbg(us, "-- stall on control pipe\n");
return USB_STOR_XFER_STALLED;
@@ -433,8 +443,10 @@
return USB_STOR_XFER_ERROR;
}
- /* since the block has been initialized successfully, it's now
- * okay to cancel it */
+ /*
+ * since the block has been initialized successfully, it's now
+ * okay to cancel it
+ */
set_bit(US_FLIDX_SG_ACTIVE, &us->dflags);
/* did an abort occur during the submission? */
@@ -515,7 +527,8 @@
* Transport routines
***********************************************************************/
-/* There are so many devices that report the capacity incorrectly,
+/*
+ * There are so many devices that report the capacity incorrectly,
* this routine was written to counteract some of the resulting
* problems.
*/
@@ -533,7 +546,8 @@
[12] = 0x14 /* Record Not Found */
};
- /* If last-sector problems can't occur, whether because the
+ /*
+ * If last-sector problems can't occur, whether because the
* capacity was already decremented or because the device is
* known to report the correct capacity, then we don't need
* to do anything.
@@ -559,13 +573,15 @@
if (srb->result == SAM_STAT_GOOD && scsi_get_resid(srb) == 0) {
- /* The command succeeded. We know this device doesn't
+ /*
+ * The command succeeded. We know this device doesn't
* have the last-sector bug, so stop checking it.
*/
us->use_last_sector_hacks = 0;
} else {
- /* The command failed. Allow up to 3 retries in case this
+ /*
+ * The command failed. Allow up to 3 retries in case this
* is some normal sort of failure. After that, assume the
* capacity is wrong and we're trying to access the sector
* beyond the end. Replace the result code and sense data
@@ -581,7 +597,8 @@
}
done:
- /* Don't reset the retry counter for TEST UNIT READY commands,
+ /*
+ * Don't reset the retry counter for TEST UNIT READY commands,
* because they get issued after device resets which might be
* caused by a failed last-sector access.
*/
@@ -589,7 +606,8 @@
us->last_sector_retries = 0;
}
-/* Invoke the transport and basic error-handling/recovery methods
+/*
+ * Invoke the transport and basic error-handling/recovery methods
*
* This is used by the protocol layers to actually send the message to
* the device and receive the response.
@@ -603,7 +621,8 @@
scsi_set_resid(srb, 0);
result = us->transport(srb, us);
- /* if the command gets aborted by the higher layers, we need to
+ /*
+ * if the command gets aborted by the higher layers, we need to
* short-circuit all other processing
*/
if (test_bit(US_FLIDX_TIMED_OUT, &us->dflags)) {
@@ -628,7 +647,8 @@
srb->result = SAM_STAT_GOOD;
- /* Determine if we need to auto-sense
+ /*
+ * Determine if we need to auto-sense
*
* I normally don't use a flag like this, but it's almost impossible
* to understand what's going on here if I don't.
@@ -728,7 +748,8 @@
goto Handle_Errors;
}
- /* Some devices claim to support larger sense but fail when
+ /*
+ * Some devices claim to support larger sense but fail when
* trying to request it. When a transport failure happens
* using US_FS_SANE_SENSE, we always retry with a standard
* (small) sense request. This fixes some USB GSM modems
@@ -746,7 +767,8 @@
if (temp_result != USB_STOR_TRANSPORT_GOOD) {
usb_stor_dbg(us, "-- auto-sense failure\n");
- /* we skip the reset if this happens to be a
+ /*
+ * we skip the reset if this happens to be a
* multi-target device, since failure of an
* auto-sense is perfectly valid
*/
@@ -756,7 +778,8 @@
return;
}
- /* If the sense data returned is larger than 18-bytes then we
+ /*
+ * If the sense data returned is larger than 18-bytes then we
* assume this device supports requesting more in the future.
* The response code must be 70h through 73h inclusive.
*/
@@ -767,7 +790,8 @@
usb_stor_dbg(us, "-- SANE_SENSE support enabled\n");
us->fflags |= US_FL_SANE_SENSE;
- /* Indicate to the user that we truncated their sense
+ /*
+ * Indicate to the user that we truncated their sense
* because we didn't know it supported larger sense.
*/
usb_stor_dbg(us, "-- Sense data truncated to %i from %i\n",
@@ -795,13 +819,15 @@
SCSI_SENSE_BUFFERSIZE, 4);
fm_ili = (scdd ? scdd[3] : srb->sense_buffer[2]) & 0xA0;
- /* We often get empty sense data. This could indicate that
+ /*
+ * We often get empty sense data. This could indicate that
* everything worked or that there was an unspecified
* problem. We have to decide which.
*/
if (sshdr.sense_key == 0 && sshdr.asc == 0 && sshdr.ascq == 0 &&
fm_ili == 0) {
- /* If things are really okay, then let's show that.
+ /*
+ * If things are really okay, then let's show that.
* Zero out the sense buffer so the higher layers
* won't realize we did an unsolicited auto-sense.
*/
@@ -809,7 +835,8 @@
srb->result = SAM_STAT_GOOD;
srb->sense_buffer[0] = 0x0;
- /* If there was a problem, report an unspecified
+ /*
+ * If there was a problem, report an unspecified
* hardware error to prevent the higher layers from
* entering an infinite retry loop.
*/
@@ -860,20 +887,26 @@
last_sector_hacks(us, srb);
return;
- /* Error and abort processing: try to resynchronize with the device
+ /*
+ * Error and abort processing: try to resynchronize with the device
* by issuing a port reset. If that fails, try a class-specific
- * device reset. */
+ * device reset.
+ */
Handle_Errors:
- /* Set the RESETTING bit, and clear the ABORTING bit so that
- * the reset may proceed. */
+ /*
+ * Set the RESETTING bit, and clear the ABORTING bit so that
+ * the reset may proceed.
+ */
scsi_lock(us_to_host(us));
set_bit(US_FLIDX_RESETTING, &us->dflags);
clear_bit(US_FLIDX_ABORTING, &us->dflags);
scsi_unlock(us_to_host(us));
- /* We must release the device lock because the pre_reset routine
- * will want to acquire it. */
+ /*
+ * We must release the device lock because the pre_reset routine
+ * will want to acquire it.
+ */
mutex_unlock(&us->dev_mutex);
result = usb_stor_port_reset(us);
mutex_lock(&us->dev_mutex);
@@ -891,10 +924,12 @@
/* Stop the current URB transfer */
void usb_stor_stop_transport(struct us_data *us)
{
- /* If the state machine is blocked waiting for an URB,
+ /*
+ * If the state machine is blocked waiting for an URB,
* let's wake it up. The test_and_clear_bit() call
* guarantees that if a URB has just been submitted,
- * it won't be cancelled more than once. */
+ * it won't be cancelled more than once.
+ */
if (test_and_clear_bit(US_FLIDX_URB_ACTIVE, &us->dflags)) {
usb_stor_dbg(us, "-- cancelling URB\n");
usb_unlink_urb(us->current_urb);
@@ -955,7 +990,8 @@
/* STATUS STAGE */
- /* NOTE: CB does not have a status stage. Silly, I know. So
+ /*
+ * NOTE: CB does not have a status stage. Silly, I know. So
* we have to catch this at a higher level.
*/
if (us->protocol != USB_PR_CBI)
@@ -967,7 +1003,8 @@
if (result != USB_STOR_XFER_GOOD)
return USB_STOR_TRANSPORT_ERROR;
- /* UFI gives us ASC and ASCQ, like a request sense
+ /*
+ * UFI gives us ASC and ASCQ, like a request sense
*
* REQUEST_SENSE and INQUIRY don't affect the sense data on UFI
* devices, so we ignore the information for those commands. Note
@@ -983,7 +1020,8 @@
return USB_STOR_TRANSPORT_GOOD;
}
- /* If not UFI, we interpret the data as a result code
+ /*
+ * If not UFI, we interpret the data as a result code
* The first byte should always be a 0x0.
*
* Some bogus devices don't follow that rule. They stuff the ASC
@@ -1005,7 +1043,8 @@
}
return USB_STOR_TRANSPORT_ERROR;
- /* the CBI spec requires that the bulk pipe must be cleared
+ /*
+ * the CBI spec requires that the bulk pipe must be cleared
* following any data-in/out command failure (section 2.4.3.1.3)
*/
Failed:
@@ -1107,9 +1146,11 @@
/* DATA STAGE */
/* send/receive data payload, if there is any */
- /* Some USB-IDE converter chips need a 100us delay between the
+ /*
+ * Some USB-IDE converter chips need a 100us delay between the
* command phase and the data phase. Some devices need a little
- * more than that, probably because of clock rate inaccuracies. */
+ * more than that, probably because of clock rate inaccuracies.
+ */
if (unlikely(us->fflags & US_FL_GO_SLOW))
usleep_range(125, 150);
@@ -1121,7 +1162,8 @@
if (result == USB_STOR_XFER_ERROR)
return USB_STOR_TRANSPORT_ERROR;
- /* If the device tried to send back more data than the
+ /*
+ * If the device tried to send back more data than the
* amount requested, the spec requires us to transfer
* the CSW anyway. Since there's no point retrying the
* the command, we'll return fake sense data indicating
@@ -1156,7 +1198,8 @@
}
}
- /* See flow chart on pg 15 of the Bulk Only Transport spec for
+ /*
+ * See flow chart on pg 15 of the Bulk Only Transport spec for
* an explanation of how this code works.
*/
@@ -1165,7 +1208,8 @@
result = usb_stor_bulk_transfer_buf(us, us->recv_bulk_pipe,
bcs, US_BULK_CS_WRAP_LEN, &cswlen);
- /* Some broken devices add unnecessary zero-length packets to the
+ /*
+ * Some broken devices add unnecessary zero-length packets to the
* end of their data transfers. Such packets show up as 0-length
* CSWs. If we encounter such a thing, try to read the CSW again.
*/
@@ -1201,7 +1245,8 @@
return USB_STOR_TRANSPORT_ERROR;
}
- /* Some broken devices report odd signatures, so we do not check them
+ /*
+ * Some broken devices report odd signatures, so we do not check them
* for validity against the spec. We store the first one we see,
* and check subsequent transfers for validity against this signature.
*/
@@ -1217,11 +1262,14 @@
return USB_STOR_TRANSPORT_ERROR;
}
- /* try to compute the actual residue, based on how much data
- * was really transferred and what the device tells us */
+ /*
+ * try to compute the actual residue, based on how much data
+ * was really transferred and what the device tells us
+ */
if (residue && !(us->fflags & US_FL_IGNORE_RESIDUE)) {
- /* Heuristically detect devices that generate bogus residues
+ /*
+ * Heuristically detect devices that generate bogus residues
* by seeing what happens with INQUIRY and READ CAPACITY
* commands.
*/
@@ -1259,7 +1307,8 @@
return USB_STOR_TRANSPORT_FAILED;
case US_BULK_STAT_PHASE:
- /* phase error -- note that a transport reset will be
+ /*
+ * phase error -- note that a transport reset will be
* invoked by the invoke_transport() function
*/
return USB_STOR_TRANSPORT_ERROR;
@@ -1274,7 +1323,8 @@
* Reset routines
***********************************************************************/
-/* This is the common part of the device reset code.
+/*
+ * This is the common part of the device reset code.
*
* It's handy that every transport mechanism uses the control endpoint for
* resets.
@@ -1302,8 +1352,10 @@
return result;
}
- /* Give the device some time to recover from the reset,
- * but don't delay disconnect processing. */
+ /*
+ * Give the device some time to recover from the reset,
+ * but don't delay disconnect processing.
+ */
wait_event_interruptible_timeout(us->delay_wait,
test_bit(US_FLIDX_DISCONNECTING, &us->dflags),
HZ*6);
@@ -1328,8 +1380,7 @@
return result;
}
-/* This issues a CB[I] Reset to the device in question
- */
+/* This issues a CB[I] Reset to the device in question */
#define CB_RESET_CMD_SIZE 12
int usb_stor_CB_reset(struct us_data *us)
@@ -1343,7 +1394,8 @@
}
EXPORT_SYMBOL_GPL(usb_stor_CB_reset);
-/* This issues a Bulk-only Reset to the device in question, including
+/*
+ * This issues a Bulk-only Reset to the device in question, including
* clearing the subsequent endpoint halts that may occur.
*/
int usb_stor_Bulk_reset(struct us_data *us)
@@ -1354,7 +1406,8 @@
}
EXPORT_SYMBOL_GPL(usb_stor_Bulk_reset);
-/* Issue a USB port reset to the device. The caller must not hold
+/*
+ * Issue a USB port reset to the device. The caller must not hold
* us->dev_mutex.
*/
int usb_stor_port_reset(struct us_data *us)
diff --git a/drivers/usb/storage/transport.h b/drivers/usb/storage/transport.h
index 9369d75..dae3ecd 100644
--- a/drivers/usb/storage/transport.h
+++ b/drivers/usb/storage/transport.h
@@ -1,4 +1,5 @@
-/* Driver for USB Mass Storage compliant devices
+/*
+ * Driver for USB Mass Storage compliant devices
* Transport Functions Header File
*
* Current development and maintenance by:
diff --git a/drivers/usb/storage/uas.c b/drivers/usb/storage/uas.c
index 16bc679..4d49fce 100644
--- a/drivers/usb/storage/uas.c
+++ b/drivers/usb/storage/uas.c
@@ -799,7 +799,8 @@
sdev->hostdata = devinfo;
- /* USB has unusual DMA-alignment requirements: Although the
+ /*
+ * USB has unusual DMA-alignment requirements: Although the
* starting address of each scatter-gather element doesn't matter,
* the length of each element except the last must be divisible
* by the Bulk maxpacket value. There's currently no way to
diff --git a/drivers/usb/storage/unusual_alauda.h b/drivers/usb/storage/unusual_alauda.h
index fa3e9ed..763bc03 100644
--- a/drivers/usb/storage/unusual_alauda.h
+++ b/drivers/usb/storage/unusual_alauda.h
@@ -1,4 +1,5 @@
-/* Unusual Devices File for the Alauda-based card readers
+/*
+ * Unusual Devices File for the Alauda-based card readers
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
diff --git a/drivers/usb/storage/unusual_cypress.h b/drivers/usb/storage/unusual_cypress.h
index 82e8ed0..e9a2eb8 100644
--- a/drivers/usb/storage/unusual_cypress.h
+++ b/drivers/usb/storage/unusual_cypress.h
@@ -1,4 +1,5 @@
-/* Unusual Devices File for devices based on the Cypress USB/ATA bridge
+/*
+ * Unusual Devices File for devices based on the Cypress USB/ATA bridge
* with support for ATACB
*
* This program is free software; you can redistribute it and/or modify it
diff --git a/drivers/usb/storage/unusual_datafab.h b/drivers/usb/storage/unusual_datafab.h
index 582a603..5049b6b 100644
--- a/drivers/usb/storage/unusual_datafab.h
+++ b/drivers/usb/storage/unusual_datafab.h
@@ -1,4 +1,5 @@
-/* Unusual Devices File for the Datafab USB Compact Flash reader
+/*
+ * Unusual Devices File for the Datafab USB Compact Flash reader
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
@@ -79,7 +80,8 @@
USB_SC_SCSI, USB_PR_DATAFAB, NULL,
0),
-/* Reported by Felix Moeller <felix@derklecks.de>
+/*
+ * Reported by Felix Moeller <felix@derklecks.de>
* in Germany this is sold by Hama with the productnumber 46952
* as "DualSlot CompactFlash(TM) & MStick Drive USB"
*/
diff --git a/drivers/usb/storage/unusual_devs.h b/drivers/usb/storage/unusual_devs.h
index 7ffe420..aa35392 100644
--- a/drivers/usb/storage/unusual_devs.h
+++ b/drivers/usb/storage/unusual_devs.h
@@ -1,4 +1,5 @@
-/* Driver for USB Mass Storage compliant devices
+/*
+ * Driver for USB Mass Storage compliant devices
* Unusual Devices File
*
* Current development and maintenance by:
@@ -25,13 +26,15 @@
* 675 Mass Ave, Cambridge, MA 02139, USA.
*/
-/* IMPORTANT NOTE: This file must be included in another file which does
+/*
+ * IMPORTANT NOTE: This file must be included in another file which does
* the following thing for it to work:
* The UNUSUAL_DEV, COMPLIANT_DEV, and USUAL_DEV macros must be defined
* before this file is included.
*/
-/* If you edit this file, please try to keep it sorted first by VendorID,
+/*
+ * If you edit this file, please try to keep it sorted first by VendorID,
* then by ProductID.
*
* If you want to add an entry for this file, be sure to include the
@@ -47,13 +50,15 @@
* <usb-storage@lists.one-eyed-alien.net>
*/
-/* Note: If you add an entry only in order to set the CAPACITY_OK flag,
+/*
+ * Note: If you add an entry only in order to set the CAPACITY_OK flag,
* use the COMPLIANT_DEV macro instead of UNUSUAL_DEV. This is
* because such entries mark devices which actually work correctly,
* as opposed to devices that do something strangely or wrongly.
*/
-/* In-kernel mode switching is deprecated. Do not add new devices to
+/*
+ * In-kernel mode switching is deprecated. Do not add new devices to
* this list for the sole purpose of switching them to a different
* mode. Existing userspace solutions are superior.
*
@@ -66,8 +71,7 @@
#define NO_SDDR09
#endif
-/* patch submitted by Vivian Bregier <Vivian.Bregier@imag.fr>
- */
+/* patch submitted by Vivian Bregier <Vivian.Bregier@imag.fr> */
UNUSUAL_DEV( 0x03eb, 0x2002, 0x0100, 0x0100,
"ATMEL",
"SND1 Storage",
@@ -93,7 +97,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_SANE_SENSE ),
-/* Reported by Grant Grundler <grundler@parisc-linux.org>
+/*
+ * Reported by Grant Grundler <grundler@parisc-linux.org>
* HP r707 camera in "Disk" mode with 2.00.23 or 2.00.24 firmware.
*/
UNUSUAL_DEV( 0x03f0, 0x4002, 0x0001, 0x0001,
@@ -107,7 +112,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, usb_stor_euscsi_init,
US_FL_SCM_MULT_TARG ),
-/* Reported by Sebastian Kapfer <sebastian_kapfer@gmx.net>
+/*
+ * Reported by Sebastian Kapfer <sebastian_kapfer@gmx.net>
* and Olaf Hering <olh@suse.de> (different bcd's, same vendor/product)
* for USB floppies that need the SINGLE_LUN enforcement.
*/
@@ -124,7 +130,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_IGNORE_RESIDUE ),
-/* Deduced by Jonathan Woithe <jwoithe@just42.net>
+/*
+ * Deduced by Jonathan Woithe <jwoithe@just42.net>
* Entry needed for flags: US_FL_FIX_INQUIRY because initial inquiry message
* always fails and confuses drive.
*/
@@ -167,8 +174,10 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_IGNORE_RESIDUE ),
-/* Reported by Andrew Nayenko <relan@bk.ru>
- * Updated for new firmware by Phillip Potter <phillipinda@hotmail.com> */
+/*
+ * Reported by Andrew Nayenko <relan@bk.ru>
+ * Updated for new firmware by Phillip Potter <phillipinda@hotmail.com>
+ */
UNUSUAL_DEV( 0x0421, 0x0019, 0x0592, 0x0610,
"Nokia",
"Nokia 6288",
@@ -196,16 +205,20 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_FIX_CAPACITY | US_FL_IGNORE_RESIDUE ),
-/* Reported by Sumedha Swamy <sumedhaswamy@gmail.com> and
- * Einar Th. Einarsson <einarthered@gmail.com> */
+/*
+ * Reported by Sumedha Swamy <sumedhaswamy@gmail.com> and
+ * Einar Th. Einarsson <einarthered@gmail.com>
+ */
UNUSUAL_DEV( 0x0421, 0x0444, 0x0100, 0x0100,
"Nokia",
"N91",
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_IGNORE_RESIDUE | US_FL_FIX_CAPACITY ),
-/* Reported by Jiri Slaby <jirislaby@gmail.com> and
- * Rene C. Castberg <Rene@Castberg.org> */
+/*
+ * Reported by Jiri Slaby <jirislaby@gmail.com> and
+ * Rene C. Castberg <Rene@Castberg.org>
+ */
UNUSUAL_DEV( 0x0421, 0x0446, 0x0100, 0x0100,
"Nokia",
"N80",
@@ -269,8 +282,10 @@
US_FL_SINGLE_LUN ),
#endif
-/* Patch submitted by Daniel Drake <dsd@gentoo.org>
- * Device reports nonsense bInterfaceProtocol 6 when connected over USB2 */
+/*
+ * Patch submitted by Daniel Drake <dsd@gentoo.org>
+ * Device reports nonsense bInterfaceProtocol 6 when connected over USB2
+ */
UNUSUAL_DEV( 0x0451, 0x5416, 0x0100, 0x0100,
"Neuros Audio",
"USB 2.0 HD 2.5",
@@ -288,17 +303,18 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL, US_FL_NOT_LOCKABLE ),
/*
-* Bohdan Linda <bohdan.linda@gmail.com>
-* 1GB USB sticks MyFlash High Speed. I have restricted
-* the revision to my model only
-*/
+ * Bohdan Linda <bohdan.linda@gmail.com>
+ * 1GB USB sticks MyFlash High Speed. I have restricted
+ * the revision to my model only
+ */
UNUSUAL_DEV( 0x0457, 0x0151, 0x0100, 0x0100,
"USB 2.0",
"Flash Disk",
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_NOT_LOCKABLE ),
-/* Reported by Tamas Kerecsen <kerecsen@bigfoot.com>
+/*
+ * Reported by Tamas Kerecsen <kerecsen@bigfoot.com>
* Obviously the PROM has not been customized by the VAR;
* the Vendor and Product string descriptors are:
* Generic Mass Storage (PROTOTYPE--Remember to change idVendor)
@@ -347,24 +363,30 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_FIX_CAPACITY | US_FL_NOT_LOCKABLE),
-/* Reported by Paul Stewart <stewart@wetlogic.net>
- * This entry is needed because the device reports Sub=ff */
+/*
+ * Reported by Paul Stewart <stewart@wetlogic.net>
+ * This entry is needed because the device reports Sub=ff
+ */
UNUSUAL_DEV( 0x04a4, 0x0004, 0x0001, 0x0001,
"Hitachi",
"DVD-CAM DZ-MV100A Camcorder",
USB_SC_SCSI, USB_PR_CB, NULL, US_FL_SINGLE_LUN),
-/* BENQ DC5330
+/*
+ * BENQ DC5330
* Reported by Manuel Fombuena <mfombuena@ya.com> and
- * Frank Copeland <fjc@thingy.apana.org.au> */
+ * Frank Copeland <fjc@thingy.apana.org.au>
+ */
UNUSUAL_DEV( 0x04a5, 0x3010, 0x0100, 0x0100,
"Tekom Technologies, Inc",
"300_CAMERA",
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_IGNORE_RESIDUE ),
-/* Patch for Nikon coolpix 2000
- * Submitted by Fabien Cosse <fabien.cosse@wanadoo.fr>*/
+/*
+ * Patch for Nikon coolpix 2000
+ * Submitted by Fabien Cosse <fabien.cosse@wanadoo.fr>
+ */
UNUSUAL_DEV( 0x04b0, 0x0301, 0x0010, 0x0010,
"NIKON",
"NIKON DSC E2000",
@@ -378,21 +400,26 @@
USB_SC_DEVICE, USB_PR_CB, NULL,
US_FL_MAX_SECTORS_MIN),
-/* Reported by Simon Levitt <simon@whattf.com>
- * This entry needs Sub and Proto fields */
+/*
+ * Reported by Simon Levitt <simon@whattf.com>
+ * This entry needs Sub and Proto fields
+ */
UNUSUAL_DEV( 0x04b8, 0x0601, 0x0100, 0x0100,
"Epson",
"875DC Storage",
USB_SC_SCSI, USB_PR_CB, NULL, US_FL_FIX_INQUIRY),
-/* Reported by Khalid Aziz <khalid@gonehiking.org>
- * This entry is needed because the device reports Sub=ff */
+/*
+ * Reported by Khalid Aziz <khalid@gonehiking.org>
+ * This entry is needed because the device reports Sub=ff
+ */
UNUSUAL_DEV( 0x04b8, 0x0602, 0x0110, 0x0110,
"Epson",
"785EPX Storage",
USB_SC_SCSI, USB_PR_BULK, NULL, US_FL_SINGLE_LUN),
-/* Not sure who reported this originally but
+/*
+ * Not sure who reported this originally but
* Pavel Machek <pavel@ucw.cz> reported that the extra US_FL_SINGLE_LUN
* flag be added */
UNUSUAL_DEV( 0x04cb, 0x0100, 0x0000, 0x2210,
@@ -400,7 +427,8 @@
"FinePix 1400Zoom",
USB_SC_UFI, USB_PR_DEVICE, NULL, US_FL_FIX_INQUIRY | US_FL_SINGLE_LUN),
-/* Reported by Ondrej Zary <linux@rainbow-software.org>
+/*
+ * Reported by Ondrej Zary <linux@rainbow-software.org>
* The device reports one sector more and breaks when that sector is accessed
*/
UNUSUAL_DEV( 0x04ce, 0x0002, 0x026c, 0x026c,
@@ -409,7 +437,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_FIX_CAPACITY),
-/* Reported by Kriston Fincher <kriston@airmail.net>
+/*
+ * Reported by Kriston Fincher <kriston@airmail.net>
* Patch submitted by Sean Millichamp <sean@bruenor.org>
* This is to support the Panasonic PalmCam PV-SD4090
* This entry is needed because the device reports Sub=ff
@@ -419,8 +448,10 @@
"LS-120 Camera",
USB_SC_UFI, USB_PR_DEVICE, NULL, 0),
-/* From Yukihiro Nakai, via zaitcev@yahoo.com.
- * This is needed for CB instead of CBI */
+/*
+ * From Yukihiro Nakai, via zaitcev@yahoo.com.
+ * This is needed for CB instead of CBI
+ */
UNUSUAL_DEV( 0x04da, 0x0d05, 0x0000, 0x0000,
"Sharp CE-CW05",
"CD-R/RW Drive",
@@ -440,7 +471,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_FIX_CAPACITY | US_FL_NOT_LOCKABLE ),
-/* Most of the following entries were developed with the help of
+/*
+ * Most of the following entries were developed with the help of
* Shuttle/SCM directly.
*/
UNUSUAL_DEV( 0x04e6, 0x0001, 0x0200, 0x0200,
@@ -536,7 +568,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_MAX_SECTORS_64),
-/* Entry and supporting patch by Theodore Kilgore <kilgota@auburn.edu>.
+/*
+ * Entry and supporting patch by Theodore Kilgore <kilgota@auburn.edu>.
* Device uses standards-violating 32-byte Bulk Command Block Wrappers and
* reports itself as "Proprietary SCSI Bulk." Cf. device entry 0x084d:0x0011.
*/
@@ -553,7 +586,8 @@
USB_SC_SCSI, USB_PR_BULK, usb_stor_euscsi_init,
US_FL_SCM_MULT_TARG ),
-/* Iomega Clik! Drive
+/*
+ * Iomega Clik! Drive
* Reported by David Chatenay <dchatenay@hotmail.com>
* The reason this is needed is not fully known.
*/
@@ -570,7 +604,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_CAPACITY_OK ),
-/* Yakumo Mega Image 37
+/*
+ * Yakumo Mega Image 37
* Submitted by Stephan Fuhrmann <atomenergie@t-online.de> */
UNUSUAL_DEV( 0x052b, 0x1801, 0x0100, 0x0100,
"Tekom Technologies, Inc",
@@ -578,8 +613,10 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_IGNORE_RESIDUE ),
-/* Another Yakumo camera.
- * Reported by Michele Alzetta <michele.alzetta@aliceposta.it> */
+/*
+ * Another Yakumo camera.
+ * Reported by Michele Alzetta <michele.alzetta@aliceposta.it>
+ */
UNUSUAL_DEV( 0x052b, 0x1804, 0x0100, 0x0100,
"Tekom Technologies, Inc",
"300_CAMERA",
@@ -593,16 +630,20 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_IGNORE_RESIDUE ),
-/* Yakumo Mega Image 47
- * Reported by Bjoern Paetzel <kolrabi@kolrabi.de> */
+/*
+ * Yakumo Mega Image 47
+ * Reported by Bjoern Paetzel <kolrabi@kolrabi.de>
+ */
UNUSUAL_DEV( 0x052b, 0x1905, 0x0100, 0x0100,
"Tekom Technologies, Inc",
"400_CAMERA",
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_IGNORE_RESIDUE ),
-/* Reported by Paul Ortyl <ortylp@3miasto.net>
- * Note that it's similar to the device above, only different prodID */
+/*
+ * Reported by Paul Ortyl <ortylp@3miasto.net>
+ * Note that it's similar to the device above, only different prodID
+ */
UNUSUAL_DEV( 0x052b, 0x1911, 0x0100, 0x0100,
"Tekom Technologies, Inc",
"400_CAMERA",
@@ -615,8 +656,10 @@
USB_SC_SCSI, USB_PR_DEVICE, NULL,
US_FL_SINGLE_LUN | US_FL_NOT_LOCKABLE | US_FL_NO_WP_DETECT ),
-/* Submitted by Lars Jacob <jacob.lars@googlemail.com>
- * This entry is needed because the device reports Sub=ff */
+/*
+ * Submitted by Lars Jacob <jacob.lars@googlemail.com>
+ * This entry is needed because the device reports Sub=ff
+ */
UNUSUAL_DEV( 0x054c, 0x0010, 0x0500, 0x0610,
"Sony",
"DSC-T1/T5/H5",
@@ -719,7 +762,8 @@
USB_SC_DEVICE, USB_PR_CB, NULL,
US_FL_SINGLE_LUN),
-/* Reported by Johann Cardon <johann.cardon@free.fr>
+/*
+ * Reported by Johann Cardon <johann.cardon@free.fr>
* This entry is needed only because the device reports
* bInterfaceClass = 0xff (vendor-specific)
*/
@@ -741,7 +785,8 @@
"Digital Camera EX-20 DSC",
USB_SC_8070, USB_PR_DEVICE, NULL, 0 ),
-/* Reported by Andre Welter <a.r.welter@gmx.de>
+/*
+ * Reported by Andre Welter <a.r.welter@gmx.de>
* This antique device predates the release of the Bulk-only Transport
* spec, and if it gets a Get-Max-LUN then it requires the host to do a
* Clear-Halt on the bulk endpoints. The SINGLE_LUN flag will prevent
@@ -773,7 +818,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_NO_WP_DETECT ),
-/* Submitted by Joel Bourquard <numlock@freesurf.ch>
+/*
+ * Submitted by Joel Bourquard <numlock@freesurf.ch>
* Some versions of this device need the SubClass and Protocol overrides
* while others don't.
*/
@@ -783,7 +829,8 @@
USB_SC_SCSI, USB_PR_BULK, NULL,
US_FL_NEED_OVERRIDE ),
-/* Submitted by Sven Anderson <sven-linux@anderson.de>
+/*
+ * Submitted by Sven Anderson <sven-linux@anderson.de>
* There are at least four ProductIDs used for iPods, so I added 0x1202 and
* 0x1204. They just need the US_FL_FIX_CAPACITY. As the bcdDevice appears
* to change with firmware updates, I changed the range to maximum for all
@@ -824,7 +871,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_FIX_CAPACITY ),
-/* Reported by Dan Williams <dcbw@redhat.com>
+/*
+ * Reported by Dan Williams <dcbw@redhat.com>
* Option N.V. mobile broadband modems
* Ignore driver CD mode and force into modem mode by default.
*/
@@ -843,7 +891,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_FIX_INQUIRY ),
-/* The following two entries are for a Genesys USB to IDE
+/*
+ * The following two entries are for a Genesys USB to IDE
* converter chip, but it changes its ProductId depending
* on whether or not a disk or an optical device is enclosed
* They were originally reported by Alexander Oltu
@@ -873,8 +922,10 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_SANE_SENSE ),
-/* Reported by Hanno Boeck <hanno@gmx.de>
- * Taken from the Lycoris Kernel */
+/*
+ * Reported by Hanno Boeck <hanno@gmx.de>
+ * Taken from the Lycoris Kernel
+ */
UNUSUAL_DEV( 0x0636, 0x0003, 0x0000, 0x9999,
"Vivitar",
"Vivicam 35Xx",
@@ -908,8 +959,10 @@
US_FL_NOT_LOCKABLE ),
/* Reported by Richard -=[]=- <micro_flyer@hotmail.com> */
-/* Change to bcdDeviceMin (0x0100 to 0x0001) reported by
- * Thomas Bartosik <tbartdev@gmx-topmail.de> */
+/*
+ * Change to bcdDeviceMin (0x0100 to 0x0001) reported by
+ * Thomas Bartosik <tbartdev@gmx-topmail.de>
+ */
UNUSUAL_DEV( 0x067b, 0x2507, 0x0001, 0x0100,
"Prolific Technology Inc.",
"Mass Storage Device",
@@ -961,7 +1014,8 @@
US_FL_NO_WP_DETECT | US_FL_MAX_SECTORS_64 |
US_FL_NO_READ_CAPACITY_16),
-/* Reported by Jean-Baptiste Onofre <jb@nanthrax.net>
+/*
+ * Reported by Jean-Baptiste Onofre <jb@nanthrax.net>
* Support the following product :
* "Dane-Elec MediaTouch"
*/
@@ -971,7 +1025,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_NO_WP_DETECT | US_FL_MAX_SECTORS_64),
-/* Reported by Massimiliano Ghilardi <massimiliano.ghilardi@gmail.com>
+/*
+ * Reported by Massimiliano Ghilardi <massimiliano.ghilardi@gmail.com>
* This USB MP3/AVI player device fails and disconnects if more than 128
* sectors (64kB) are read/written in a single command, and may be present
* at least in the following products:
@@ -1040,7 +1095,8 @@
US_FL_SINGLE_LUN ),
#endif
-/* Datafab KECF-USB / Sagatek DCS-CF / Simpletech Flashlink UCF-100
+/*
+ * Datafab KECF-USB / Sagatek DCS-CF / Simpletech Flashlink UCF-100
* Only revision 1.13 tested (same for all of the above devices,
* based on the Datafab DF-UG-07 chip). Needed for US_FL_FIX_INQUIRY.
* Submitted by Marek Michalkiewicz <marekm@amelek.gda.pl>.
@@ -1052,7 +1108,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_FIX_INQUIRY | US_FL_FIX_CAPACITY ),
-/* Reported by Rauch Wolke <rauchwolke@gmx.net>
+/*
+ * Reported by Rauch Wolke <rauchwolke@gmx.net>
* and augmented by binbin <binbinsh@gmail.com> (Bugzilla #12882)
*/
UNUSUAL_DEV( 0x07c4, 0xa4a5, 0x0000, 0xffff,
@@ -1061,7 +1118,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_IGNORE_RESIDUE | US_FL_MAX_SECTORS_64 ),
-/* Casio QV 2x00/3x00/4000/8000 digital still cameras are not conformant
+/*
+ * Casio QV 2x00/3x00/4000/8000 digital still cameras are not conformant
* to the USB storage specification in two ways:
* - They tell us they are using transport protocol CBI. In reality they
* are using transport protocol CB.
@@ -1119,7 +1177,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, usb_stor_euscsi_init,
US_FL_SCM_MULT_TARG ),
-/* Entry and supporting patch by Theodore Kilgore <kilgota@auburn.edu>.
+/*
+ * Entry and supporting patch by Theodore Kilgore <kilgota@auburn.edu>.
* Flag will support Bulk devices which use a standards-violating 32-byte
* Command Block Wrapper. Here, the "DC2MEGA" cameras (several brands) with
* Grandtech GT892x chip, which request "Proprietary SCSI Bulk" support.
@@ -1131,7 +1190,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_BULK32),
-/* Reported by <ttkspam@free.fr>
+/*
+ * Reported by <ttkspam@free.fr>
* The device reports a vendor-specific device class, requiring an
* explicit vendor/product match.
*/
@@ -1140,11 +1200,12 @@
"FW_Omega2",
USB_SC_DEVICE, USB_PR_DEVICE, NULL, 0),
-/* Andrew Lunn <andrew@lunn.ch>
+/*
+ * Andrew Lunn <andrew@lunn.ch>
* PanDigital Digital Picture Frame. Does not like ALLOW_MEDIUM_REMOVAL
* on LUN 4.
* Note: Vend:Prod clash with "Ltd Maxell WS30 Slim Digital Camera"
-*/
+ */
UNUSUAL_DEV( 0x0851, 0x1543, 0x0200, 0x0200,
"PanDigital",
"Photo Frame",
@@ -1170,7 +1231,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_SINGLE_LUN),
-/* Submitted by Dylan Taft <d13f00l@gmail.com>
+/*
+ * Submitted by Dylan Taft <d13f00l@gmail.com>
* US_FL_IGNORE_RESIDUE Needed
*/
UNUSUAL_DEV( 0x08ca, 0x3103, 0x0100, 0x0100,
@@ -1179,7 +1241,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_IGNORE_RESIDUE),
-/* Entry needed for flags. Moreover, all devices with this ID use
+/*
+ * Entry needed for flags. Moreover, all devices with this ID use
* bulk-only transport, but _some_ falsely report Control/Bulk instead.
* One example is "Trumpion Digital Research MYMP3".
* Submitted by Bjoern Brill <brill(at)fs.math.uni-frankfurt.de>
@@ -1190,7 +1253,8 @@
USB_SC_DEVICE, USB_PR_BULK, NULL,
US_FL_NEED_OVERRIDE ),
-/* Reported by Filippo Bardelli <filibard@libero.it>
+/*
+ * Reported by Filippo Bardelli <filibard@libero.it>
* The device reports a subclass of RBC, which is wrong.
*/
UNUSUAL_DEV( 0x090a, 0x1050, 0x0100, 0x0100,
@@ -1213,7 +1277,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_FIX_CAPACITY ),
-/* Reported by Paul Hartman <paul.hartman+linux@gmail.com>
+/*
+ * Reported by Paul Hartman <paul.hartman+linux@gmail.com>
* This card reader returns "Illegal Request, Logical Block Address
* Out of Range" for the first READ(10) after a new card is inserted.
*/
@@ -1223,7 +1288,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_INITIAL_READ10 ),
-/* This Pentax still camera is not conformant
+/*
+ * This Pentax still camera is not conformant
* to the USB storage specification: -
* - It does not like the INQUIRY command. So we must handle this command
* of the SCSI layer ourselves.
@@ -1236,8 +1302,10 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_FIX_INQUIRY ),
-/* These are virtual windows driver CDs, which the zd1211rw driver
- * automatically converts into WLAN devices. */
+/*
+ * These are virtual windows driver CDs, which the zd1211rw driver
+ * automatically converts into WLAN devices.
+ */
UNUSUAL_DEV( 0x0ace, 0x2011, 0x0101, 0x0101,
"ZyXEL",
"G-220F USB-WLAN Install",
@@ -1250,7 +1318,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_IGNORE_DEVICE ),
-/* Reported by Dan Williams <dcbw@redhat.com>
+/*
+ * Reported by Dan Williams <dcbw@redhat.com>
* Option N.V. mobile broadband modems
* Ignore driver CD mode and force into modem mode by default.
*/
@@ -1262,20 +1331,24 @@
USB_SC_DEVICE, USB_PR_DEVICE, option_ms_init,
0),
-/* Reported by F. Aben <f.aben@option.com>
+/*
+ * Reported by F. Aben <f.aben@option.com>
* This device (wrongly) has a vendor-specific device descriptor.
* The entry is needed so usb-storage can bind to it's mass-storage
- * interface as an interface driver */
+ * interface as an interface driver
+ */
UNUSUAL_DEV( 0x0af0, 0x7401, 0x0000, 0x0000,
"Option",
"GI 0401 SD-Card",
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
0 ),
-/* Reported by Jan Dumon <j.dumon@option.com>
+/*
+ * Reported by Jan Dumon <j.dumon@option.com>
* These devices (wrongly) have a vendor-specific device descriptor.
* These entries are needed so usb-storage can bind to their mass-storage
- * interface as an interface driver */
+ * interface as an interface driver
+ */
UNUSUAL_DEV( 0x0af0, 0x7501, 0x0000, 0x0000,
"Option",
"GI 0431 SD-Card",
@@ -1419,7 +1492,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_FIX_CAPACITY),
-/* Reported by Lubomir Blaha <tritol@trilogic.cz>
+/*
+ * Reported by Lubomir Blaha <tritol@trilogic.cz>
* I _REALLY_ don't know what 3rd, 4th number and all defines mean, but this
* works for me. Can anybody correct these values? (I able to test corrected
* version.)
@@ -1430,8 +1504,10 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_FIX_INQUIRY ),
-/* Reported by Edward Chapman (taken from linux-usb mailing list)
- Netac OnlyDisk Mini U2CV2 512MB USB 2.0 Flash Drive */
+/*
+ * Reported by Edward Chapman (taken from linux-usb mailing list)
+ * Netac OnlyDisk Mini U2CV2 512MB USB 2.0 Flash Drive
+ */
UNUSUAL_DEV( 0x0dd8, 0xd202, 0x0000, 0x9999,
"Netac",
"USB Flash Disk",
@@ -1439,8 +1515,10 @@
US_FL_IGNORE_RESIDUE ),
-/* Patch by Stephan Walter <stephan.walter@epfl.ch>
- * I don't know why, but it works... */
+/*
+ * Patch by Stephan Walter <stephan.walter@epfl.ch>
+ * I don't know why, but it works...
+ */
UNUSUAL_DEV( 0x0dda, 0x0001, 0x0012, 0x0012,
"WINWARD",
"Music Disk",
@@ -1468,8 +1546,10 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_FIX_INQUIRY ),
-/* Submitted by Daniel Drake <dsd@gentoo.org>
- * Reported by dayul on the Gentoo Forums */
+/*
+ * Submitted by Daniel Drake <dsd@gentoo.org>
+ * Reported by dayul on the Gentoo Forums
+ */
UNUSUAL_DEV( 0x0ea0, 0x2168, 0x0110, 0x0110,
"Ours Technology",
"Flash Disk",
@@ -1483,15 +1563,18 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_IGNORE_RESIDUE ),
-/* Reported by Benjamin Schiller <sbenni@gmx.de>
- * It is also sold by Easylite as DJ 20 */
+/*
+ * Reported by Benjamin Schiller <sbenni@gmx.de>
+ * It is also sold by Easylite as DJ 20
+ */
UNUSUAL_DEV( 0x0ed1, 0x7636, 0x0103, 0x0103,
"Typhoon",
"My DJ 1820",
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_IGNORE_RESIDUE | US_FL_GO_SLOW | US_FL_MAX_SECTORS_64),
-/* Patch by Leonid Petrov mail at lpetrov.net
+/*
+ * Patch by Leonid Petrov mail at lpetrov.net
* Reported by Robert Spitzenpfeil <robert@spitzenpfeil.org>
* http://www.qbik.ch/usb/devices/showdev.php?id=1705
* Updated to 103 device by MJ Ray mjr at phonecoop.coop
@@ -1502,7 +1585,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_IGNORE_RESIDUE ),
-/* David Kuehling <dvdkhlng@gmx.de>:
+/*
+ * David Kuehling <dvdkhlng@gmx.de>:
* for MP3-Player AVOX WSX-300ER (bought in Japan). Reports lots of SCSI
* errors when trying to write.
*/
@@ -1540,8 +1624,10 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_IGNORE_DEVICE),
-/* Reported by Jan Mate <mate@fiit.stuba.sk>
- * and by Soeren Sonnenburg <kernel@nn7.de> */
+/*
+ * Reported by Jan Mate <mate@fiit.stuba.sk>
+ * and by Soeren Sonnenburg <kernel@nn7.de>
+ */
UNUSUAL_DEV( 0x0fce, 0xe030, 0x0000, 0x0000,
"Sony Ericsson",
"P990i",
@@ -1562,7 +1648,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_IGNORE_RESIDUE ),
-/* Reported by Kevin Cernekee <kpc-usbdev@gelato.uiuc.edu>
+/*
+ * Reported by Kevin Cernekee <kpc-usbdev@gelato.uiuc.edu>
* Tested on hardware version 1.10.
* Entry is needed only for the initializer function override.
* Devices with bcd > 110 seem to not need it while those
@@ -1586,7 +1673,8 @@
"My Passport HDD",
USB_SC_DEVICE, USB_PR_DEVICE, NULL, US_FL_WRITE_CACHE),
-/* Reported by Fabio Venturi <f.venturi@tdnet.it>
+/*
+ * Reported by Fabio Venturi <f.venturi@tdnet.it>
* The device reports a vendor-specific bDeviceClass.
*/
UNUSUAL_DEV( 0x10d6, 0x2200, 0x0100, 0x0100,
@@ -1595,7 +1683,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
0),
-/* Reported by Pascal Terjan <pterjan@mandriva.com>
+/*
+ * Reported by Pascal Terjan <pterjan@mandriva.com>
* Ignore driver CD mode and force into modem mode by default.
*/
UNUSUAL_DEV( 0x1186, 0x3e04, 0x0000, 0x0000,
@@ -1603,7 +1692,8 @@
"USB Mass Storage",
USB_SC_DEVICE, USB_PR_DEVICE, option_ms_init, US_FL_IGNORE_DEVICE),
-/* Reported by Kevin Lloyd <linux@sierrawireless.com>
+/*
+ * Reported by Kevin Lloyd <linux@sierrawireless.com>
* Entry is needed for the initializer function override,
* which instructs the device to load as a modem
* device.
@@ -1614,7 +1704,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, sierra_ms_init,
0),
-/* Reported by Jaco Kroon <jaco@kroon.co.za>
+/*
+ * Reported by Jaco Kroon <jaco@kroon.co.za>
* The usb-storage module found on the Digitech GNX4 (and supposedly other
* devices) misbehaves and causes a bunch of invalid I/O errors.
*/
@@ -1624,7 +1715,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_IGNORE_RESIDUE ),
-/* Reported by fangxiaozhi <huananhu@huawei.com>
+/*
+ * Reported by fangxiaozhi <huananhu@huawei.com>
* This brings the HUAWEI data card devices into multi-port mode
*/
UNUSUAL_DEV( 0x12d1, 0x1001, 0x0000, 0x0000,
@@ -1993,7 +2085,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_BROKEN_FUA ),
-/* Reported by Alexandre Oliva <oliva@lsd.ic.unicamp.br>
+/*
+ * Reported by Alexandre Oliva <oliva@lsd.ic.unicamp.br>
* JMicron responds to USN and several other SCSI ioctls with a
* residue that causes subsequent I/O requests to fail. */
UNUSUAL_DEV( 0x152d, 0x2329, 0x0100, 0x0100,
@@ -2009,7 +2102,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_BROKEN_FUA ),
-/* Entrega Technologies U1-SC25 (later Xircom PortGear PGSCSI)
+/*
+ * Entrega Technologies U1-SC25 (later Xircom PortGear PGSCSI)
* and Mac USB Dock USB-SCSI */
UNUSUAL_DEV( 0x1645, 0x0007, 0x0100, 0x0133,
"Entrega Technologies",
@@ -2017,8 +2111,10 @@
USB_SC_DEVICE, USB_PR_DEVICE, usb_stor_euscsi_init,
US_FL_SCM_MULT_TARG ),
-/* Reported by Robert Schedel <r.schedel@yahoo.de>
- * Note: this is a 'super top' device like the above 14cd/6600 device */
+/*
+ * Reported by Robert Schedel <r.schedel@yahoo.de>
+ * Note: this is a 'super top' device like the above 14cd/6600 device
+ */
UNUSUAL_DEV( 0x1652, 0x6600, 0x0201, 0x0201,
"Teac",
"HD-35PUK-B",
@@ -2045,10 +2141,12 @@
USB_SC_DEVICE, USB_PR_DEVICE, usb_stor_euscsi_init,
US_FL_SCM_MULT_TARG ),
-/* Reported by Hans de Goede <hdegoede@redhat.com>
+/*
+ * Reported by Hans de Goede <hdegoede@redhat.com>
* These Appotech controllers are found in Picture Frames, they provide a
* (buggy) emulation of a cdrom drive which contains the windows software
- * Uploading of pictures happens over the corresponding /dev/sg device. */
+ * Uploading of pictures happens over the corresponding /dev/sg device.
+ */
UNUSUAL_DEV( 0x1908, 0x1315, 0x0000, 0x0000,
"BUILDWIN",
"Photo Frame",
@@ -2065,19 +2163,22 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_NO_READ_DISC_INFO ),
-/* Reported by Oliver Neukum <oneukum@suse.com>
+/*
+ * Reported by Oliver Neukum <oneukum@suse.com>
* This device morphes spontaneously into another device if the access
* pattern of Windows isn't followed. Thus writable media would be dirty
* if the initial instance is used. So the device is limited to its
* virtual CD.
- * And yes, the concept that BCD goes up to 9 is not heeded */
+ * And yes, the concept that BCD goes up to 9 is not heeded
+ */
UNUSUAL_DEV( 0x19d2, 0x1225, 0x0000, 0xffff,
"ZTE,Incorporated",
"ZTE WCDMA Technologies MSM",
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_SINGLE_LUN ),
-/* Reported by Sven Geggus <sven-usbst@geggus.net>
+/*
+ * Reported by Sven Geggus <sven-usbst@geggus.net>
* This encrypted pen drive returns bogus data for the initial READ(10).
*/
UNUSUAL_DEV( 0x1b1c, 0x1ab5, 0x0200, 0x0200,
@@ -2086,7 +2187,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_INITIAL_READ10 ),
-/* Reported by Hans de Goede <hdegoede@redhat.com>
+/*
+ * Reported by Hans de Goede <hdegoede@redhat.com>
* These are mini projectors using USB for both power and video data transport
* The usb-storage interface is a virtual windows driver CD, which the gm12u320
* driver automatically converts into framebuffer & kms dri device nodes.
@@ -2097,9 +2199,11 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_IGNORE_DEVICE ),
-/* Patch by Richard Schütz <r.schtz@t-online.de>
+/*
+ * Patch by Richard Schütz <r.schtz@t-online.de>
* This external hard drive enclosure uses a JMicron chip which
- * needs the US_FL_IGNORE_RESIDUE flag to work properly. */
+ * needs the US_FL_IGNORE_RESIDUE flag to work properly.
+ */
UNUSUAL_DEV( 0x1e68, 0x001b, 0x0000, 0x0000,
"TrekStor GmbH & Co. KG",
"DataStation maxi g.u",
@@ -2126,7 +2230,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_FIX_CAPACITY),
-/* patch submitted by Davide Perini <perini.davide@dpsoftware.org>
+/*
+ * patch submitted by Davide Perini <perini.davide@dpsoftware.org>
* and Renato Perini <rperini@email.it>
*/
UNUSUAL_DEV( 0x22b8, 0x3010, 0x0001, 0x0001,
@@ -2153,7 +2258,8 @@
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_GO_SLOW ),
-/* Reported by Frederic Marchal <frederic.marchal@wowcompany.com>
+/*
+ * Reported by Frederic Marchal <frederic.marchal@wowcompany.com>
* Mio Moov 330
*/
UNUSUAL_DEV( 0x3340, 0xffff, 0x0000, 0x0000,
diff --git a/drivers/usb/storage/unusual_freecom.h b/drivers/usb/storage/unusual_freecom.h
index 59a2611..1f5aab4 100644
--- a/drivers/usb/storage/unusual_freecom.h
+++ b/drivers/usb/storage/unusual_freecom.h
@@ -1,4 +1,5 @@
-/* Unusual Devices File for the Freecom USB/IDE adaptor
+/*
+ * Unusual Devices File for the Freecom USB/IDE adaptor
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
diff --git a/drivers/usb/storage/unusual_isd200.h b/drivers/usb/storage/unusual_isd200.h
index 14cca0c..9b6862e 100644
--- a/drivers/usb/storage/unusual_isd200.h
+++ b/drivers/usb/storage/unusual_isd200.h
@@ -1,4 +1,5 @@
-/* Unusual Devices File for In-System Design, Inc. ISD200 ASIC
+/*
+ * Unusual Devices File for In-System Design, Inc. ISD200 ASIC
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
diff --git a/drivers/usb/storage/unusual_jumpshot.h b/drivers/usb/storage/unusual_jumpshot.h
index 54be78b..413e64f 100644
--- a/drivers/usb/storage/unusual_jumpshot.h
+++ b/drivers/usb/storage/unusual_jumpshot.h
@@ -1,4 +1,5 @@
-/* Unusual Devices File for the Lexar "Jumpshot" Compact Flash reader
+/*
+ * Unusual Devices File for the Lexar "Jumpshot" Compact Flash reader
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
diff --git a/drivers/usb/storage/unusual_karma.h b/drivers/usb/storage/unusual_karma.h
index 6df0397..e6fad3a 100644
--- a/drivers/usb/storage/unusual_karma.h
+++ b/drivers/usb/storage/unusual_karma.h
@@ -1,4 +1,5 @@
-/* Unusual Devices File for the Rio Karma
+/*
+ * Unusual Devices File for the Rio Karma
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
diff --git a/drivers/usb/storage/unusual_onetouch.h b/drivers/usb/storage/unusual_onetouch.h
index 0abb819..425dc22 100644
--- a/drivers/usb/storage/unusual_onetouch.h
+++ b/drivers/usb/storage/unusual_onetouch.h
@@ -1,4 +1,5 @@
-/* Unusual Devices File for the Maxtor OneTouch USB hard drive's button
+/*
+ * Unusual Devices File for the Maxtor OneTouch USB hard drive's button
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
@@ -18,7 +19,8 @@
#if defined(CONFIG_USB_STORAGE_ONETOUCH) || \
defined(CONFIG_USB_STORAGE_ONETOUCH_MODULE)
-/* Submitted by: Nick Sillik <n.sillik@temple.edu>
+/*
+ * Submitted by: Nick Sillik <n.sillik@temple.edu>
* Needed for OneTouch extension to usb-storage
*/
UNUSUAL_DEV( 0x0d49, 0x7000, 0x0000, 0x9999,
diff --git a/drivers/usb/storage/unusual_realtek.h b/drivers/usb/storage/unusual_realtek.h
index e41f50c..8fe624a 100644
--- a/drivers/usb/storage/unusual_realtek.h
+++ b/drivers/usb/storage/unusual_realtek.h
@@ -1,4 +1,5 @@
-/* Driver for Realtek RTS51xx USB card reader
+/*
+ * Driver for Realtek RTS51xx USB card reader
*
* Copyright(c) 2009 Realtek Semiconductor Corp. All rights reserved.
*
diff --git a/drivers/usb/storage/unusual_sddr09.h b/drivers/usb/storage/unusual_sddr09.h
index 59a7e37..d9d38ac 100644
--- a/drivers/usb/storage/unusual_sddr09.h
+++ b/drivers/usb/storage/unusual_sddr09.h
@@ -1,4 +1,5 @@
-/* Unusual Devices File for SanDisk SDDR-09 SmartMedia reader
+/*
+ * Unusual Devices File for SanDisk SDDR-09 SmartMedia reader
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
diff --git a/drivers/usb/storage/unusual_sddr55.h b/drivers/usb/storage/unusual_sddr55.h
index fcb7e12..ebb1d1c 100644
--- a/drivers/usb/storage/unusual_sddr55.h
+++ b/drivers/usb/storage/unusual_sddr55.h
@@ -1,4 +1,5 @@
-/* Unusual Devices File for SanDisk SDDR-55 SmartMedia reader
+/*
+ * Unusual Devices File for SanDisk SDDR-55 SmartMedia reader
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
diff --git a/drivers/usb/storage/unusual_uas.h b/drivers/usb/storage/unusual_uas.h
index 53341a7..cbea9f3 100644
--- a/drivers/usb/storage/unusual_uas.h
+++ b/drivers/usb/storage/unusual_uas.h
@@ -1,4 +1,5 @@
-/* Driver for USB Attached SCSI devices - Unusual Devices File
+/*
+ * Driver for USB Attached SCSI devices - Unusual Devices File
*
* (c) 2013 Hans de Goede <hdegoede@redhat.com>
*
diff --git a/drivers/usb/storage/unusual_usbat.h b/drivers/usb/storage/unusual_usbat.h
index 38e79c4..2044ad5 100644
--- a/drivers/usb/storage/unusual_usbat.h
+++ b/drivers/usb/storage/unusual_usbat.h
@@ -1,4 +1,5 @@
-/* Unusual Devices File for SCM Microsystems (a.k.a. Shuttle) USB-ATAPI cable
+/*
+ * Unusual Devices File for SCM Microsystems (a.k.a. Shuttle) USB-ATAPI cable
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
diff --git a/drivers/usb/storage/usb.c b/drivers/usb/storage/usb.c
index 9de988a..ef2d8cd 100644
--- a/drivers/usb/storage/usb.c
+++ b/drivers/usb/storage/usb.c
@@ -1,4 +1,5 @@
-/* Driver for USB Mass Storage compliant devices
+/*
+ * Driver for USB Mass Storage compliant devices
*
* Current development and maintenance by:
* (c) 1999-2003 Matthew Dharm (mdharm-usb@one-eyed-alien.net)
@@ -97,7 +98,8 @@
* with the entries in usb_storage_usb_ids[], defined in usual-tables.c.
*/
-/* The vendor name should be kept at eight characters or less, and
+/*
+ *The vendor name should be kept at eight characters or less, and
* the product name should be kept at 16 characters or less. If a device
* has the US_FL_FIX_INQUIRY flag, then the vendor and product names
* normally generated by a device through the INQUIRY response will be
@@ -191,8 +193,10 @@
if (us->suspend_resume_hook)
(us->suspend_resume_hook)(us, US_SUSPEND);
- /* When runtime PM is working, we'll set a flag to indicate
- * whether we should autoresume when a SCSI request arrives. */
+ /*
+ * When runtime PM is working, we'll set a flag to indicate
+ * whether we should autoresume when a SCSI request arrives.
+ */
mutex_unlock(&us->dev_mutex);
return 0;
@@ -220,8 +224,10 @@
/* Report the reset to the SCSI core */
usb_stor_report_bus_reset(us);
- /* FIXME: Notify the subdrivers that they need to reinitialize
- * the device */
+ /*
+ * FIXME: Notify the subdrivers that they need to reinitialize
+ * the device
+ */
return 0;
}
EXPORT_SYMBOL_GPL(usb_stor_reset_resume);
@@ -250,8 +256,10 @@
/* Report the reset to the SCSI core */
usb_stor_report_bus_reset(us);
- /* FIXME: Notify the subdrivers that they need to reinitialize
- * the device */
+ /*
+ * FIXME: Notify the subdrivers that they need to reinitialize
+ * the device
+ */
mutex_unlock(&us->dev_mutex);
return 0;
@@ -274,15 +282,17 @@
return;
memset(data+8, ' ', 28);
- if (data[0]&0x20) { /* USB device currently not connected. Return
- peripheral qualifier 001b ("...however, the
- physical device is not currently connected
- to this logical unit") and leave vendor and
- product identification empty. ("If the target
- does store some of the INQUIRY data on the
- device, it may return zeros or ASCII spaces
- (20h) in those fields until the data is
- available from the device."). */
+ if (data[0]&0x20) { /*
+ * USB device currently not connected. Return
+ * peripheral qualifier 001b ("...however, the
+ * physical device is not currently connected
+ * to this logical unit") and leave vendor and
+ * product identification empty. ("If the target
+ * does store some of the INQUIRY data on the
+ * device, it may return zeros or ASCII spaces
+ * (20h) in those fields until the data is
+ * available from the device.").
+ */
} else {
u16 bcdDevice = le16_to_cpu(us->pusb_dev->descriptor.bcdDevice);
int n;
@@ -336,7 +346,8 @@
scsi_unlock(host);
- /* reject the command if the direction indicator
+ /*
+ * reject the command if the direction indicator
* is UNKNOWN
*/
if (us->srb->sc_data_direction == DMA_BIDIRECTIONAL) {
@@ -344,7 +355,8 @@
us->srb->result = DID_ERROR << 16;
}
- /* reject if target != 0 or if LUN is higher than
+ /*
+ * reject if target != 0 or if LUN is higher than
* the maximum known LUN
*/
else if (us->srb->device->id &&
@@ -362,8 +374,10 @@
us->srb->result = DID_BAD_TARGET << 16;
}
- /* Handle those devices which need us to fake
- * their inquiry data */
+ /*
+ * Handle those devices which need us to fake
+ * their inquiry data
+ */
else if ((us->srb->cmnd[0] == INQUIRY) &&
(us->fflags & US_FL_FIX_INQUIRY)) {
unsigned char data_ptr[36] = {
@@ -395,11 +409,13 @@
usb_stor_dbg(us, "scsi command aborted\n");
}
- /* If an abort request was received we need to signal that
+ /*
+ * If an abort request was received we need to signal that
* the abort has finished. The proper test for this is
* the TIMED_OUT flag, not srb->result == DID_ABORT, because
* the timeout might have occurred after the command had
- * already completed with a different result code. */
+ * already completed with a different result code.
+ */
if (test_bit(US_FLIDX_TIMED_OUT, &us->dflags)) {
complete(&(us->notify));
@@ -610,7 +626,8 @@
le16_to_cpu(dev->descriptor.idProduct),
us->fflags);
- /* Log a message if a non-generic unusual_dev entry contains an
+ /*
+ * Log a message if a non-generic unusual_dev entry contains an
* unnecessary subclass or protocol override. This may stimulate
* reports from users that will help us remove unneeded entries
* from the unusual_devs.h table.
@@ -782,8 +799,10 @@
return -ENOMEM;
}
- /* Just before we start our control thread, initialize
- * the device if it needs initialization */
+ /*
+ * Just before we start our control thread, initialize
+ * the device if it needs initialization
+ */
if (us->unusual_dev->initFunction) {
p = us->unusual_dev->initFunction(us);
if (p)
@@ -805,7 +824,8 @@
/* Release all our dynamic resources */
static void usb_stor_release_resources(struct us_data *us)
{
- /* Tell the control thread to exit. The SCSI host must
+ /*
+ * Tell the control thread to exit. The SCSI host must
* already have been removed and the DISCONNECTING flag set
* so that we won't accept any more commands.
*/
@@ -836,7 +856,8 @@
usb_set_intfdata(us->pusb_intf, NULL);
}
-/* First stage of disconnect processing: stop SCSI scanning,
+/*
+ * First stage of disconnect processing: stop SCSI scanning,
* remove the host, and stop accepting new commands
*/
static void quiesce_and_remove_host(struct us_data *us)
@@ -849,7 +870,8 @@
wake_up(&us->delay_wait);
}
- /* Prevent SCSI scanning (if it hasn't started yet)
+ /*
+ * Prevent SCSI scanning (if it hasn't started yet)
* or wait for the SCSI-scanning routine to stop.
*/
cancel_delayed_work_sync(&us->scan_dwork);
@@ -858,12 +880,14 @@
if (test_bit(US_FLIDX_SCAN_PENDING, &us->dflags))
usb_autopm_put_interface_no_suspend(us->pusb_intf);
- /* Removing the host will perform an orderly shutdown: caches
+ /*
+ * Removing the host will perform an orderly shutdown: caches
* synchronized, disks spun down, etc.
*/
scsi_remove_host(host);
- /* Prevent any new commands from being accepted and cut short
+ /*
+ * Prevent any new commands from being accepted and cut short
* reset delays.
*/
scsi_lock(host);
@@ -878,8 +902,10 @@
usb_stor_release_resources(us);
dissociate_dev(us);
- /* Drop our reference to the host; the SCSI core will free it
- * (and "us" along with it) when the refcount becomes 0. */
+ /*
+ * Drop our reference to the host; the SCSI core will free it
+ * (and "us" along with it) when the refcount becomes 0.
+ */
scsi_host_put(us_to_host(us));
}
@@ -900,7 +926,8 @@
us->max_lun = usb_stor_Bulk_max_lun(us);
/*
* Allow proper scanning of devices that present more than 8 LUNs
- * While not affecting other devices that may need the previous behavior
+ * While not affecting other devices that may need the previous
+ * behavior
*/
if (us->max_lun >= 8)
us_to_host(us)->max_lun = us->max_lun+1;
@@ -975,7 +1002,8 @@
get_transport(us);
get_protocol(us);
- /* Give the caller a chance to fill in specialized transport
+ /*
+ * Give the caller a chance to fill in specialized transport
* or protocol settings.
*/
return 0;
diff --git a/drivers/usb/storage/usb.h b/drivers/usb/storage/usb.h
index da0ad32..8fae28b 100644
--- a/drivers/usb/storage/usb.h
+++ b/drivers/usb/storage/usb.h
@@ -1,4 +1,5 @@
-/* Driver for USB Mass Storage compliant devices
+/*
+ * Driver for USB Mass Storage compliant devices
* Main Header File
*
* Current development and maintenance by:
@@ -100,7 +101,8 @@
/* we allocate one of these for every device that we remember */
struct us_data {
- /* The device we're working with
+ /*
+ * The device we're working with
* It's important to note:
* (o) you must hold dev_mutex to change pusb_dev
*/
@@ -125,7 +127,7 @@
u8 max_lun;
u8 ifnum; /* interface number */
- u8 ep_bInterval; /* interrupt interval */
+ u8 ep_bInterval; /* interrupt interval */
/* function pointers for this device */
trans_cmnd transport; /* transport function */
@@ -175,8 +177,10 @@
extern void fill_inquiry_response(struct us_data *us,
unsigned char *data, unsigned int data_len);
-/* The scsi_lock() and scsi_unlock() macros protect the sm_state and the
- * single queue element srb for write access */
+/*
+ * The scsi_lock() and scsi_unlock() macros protect the sm_state and the
+ * single queue element srb for write access
+ */
#define scsi_unlock(host) spin_unlock_irq(host->host_lock)
#define scsi_lock(host) spin_lock_irq(host->host_lock)
diff --git a/drivers/usb/storage/usual-tables.c b/drivers/usb/storage/usual-tables.c
index 5ef8ce7..499669b 100644
--- a/drivers/usb/storage/usual-tables.c
+++ b/drivers/usb/storage/usual-tables.c
@@ -1,4 +1,5 @@
-/* Driver for USB Mass Storage devices
+/*
+ * Driver for USB Mass Storage devices
* Usual Tables File for usb-storage and libusual
*
* Copyright (C) 2009 Alan Stern (stern@rowland.harvard.edu)
diff --git a/drivers/usb/usbip/Kconfig b/drivers/usb/usbip/Kconfig
index bd99e9e..17646b2 100644
--- a/drivers/usb/usbip/Kconfig
+++ b/drivers/usb/usbip/Kconfig
@@ -1,6 +1,6 @@
config USBIP_CORE
tristate "USB/IP support"
- depends on USB && NET
+ depends on USB_COMMON && NET
---help---
This enables pushing USB packets over IP to allow remote
machines direct access to USB devices. It provides the
@@ -16,7 +16,7 @@
config USBIP_VHCI_HCD
tristate "VHCI hcd"
- depends on USBIP_CORE
+ depends on USBIP_CORE && USB
---help---
This enables the USB/IP virtual host controller driver,
which is run on the remote machine.
@@ -26,7 +26,7 @@
config USBIP_HOST
tristate "Host driver"
- depends on USBIP_CORE
+ depends on USBIP_CORE && USB
---help---
This enables the USB/IP host driver, which is run on the
machine that is sharing the USB devices.
@@ -34,6 +34,17 @@
To compile this driver as a module, choose M here: the
module will be called usbip-host.
+config USBIP_VUDC
+ tristate "VUDC driver"
+ depends on USBIP_CORE && USB_GADGET
+ ---help---
+ This enables the USB/IP virtual USB device controller
+ driver, which is run on the host machine, allowing the
+ machine itself to act as a device.
+
+ To compile this driver as a module, choose M here: the
+ module will be called usbip-vudc.
+
config USBIP_DEBUG
bool "Debug messages for USB/IP"
depends on USBIP_CORE
diff --git a/drivers/usb/usbip/Makefile b/drivers/usb/usbip/Makefile
index 9ecd615..d843a9e 100644
--- a/drivers/usb/usbip/Makefile
+++ b/drivers/usb/usbip/Makefile
@@ -8,3 +8,6 @@
obj-$(CONFIG_USBIP_HOST) += usbip-host.o
usbip-host-y := stub_dev.o stub_main.o stub_rx.o stub_tx.o
+
+obj-$(CONFIG_USBIP_VUDC) += usbip-vudc.o
+usbip-vudc-y := vudc_dev.o vudc_sysfs.o vudc_tx.o vudc_rx.o vudc_transfer.o vudc_main.o
diff --git a/drivers/usb/usbip/stub.h b/drivers/usb/usbip/stub.h
index 266e2b0c..910f027 100644
--- a/drivers/usb/usbip/stub.h
+++ b/drivers/usb/usbip/stub.h
@@ -33,7 +33,6 @@
#define STUB_BUSID_ALLOC 3
struct stub_device {
- struct usb_interface *interface;
struct usb_device *udev;
struct usbip_device ud;
diff --git a/drivers/usb/usbip/stub_dev.c b/drivers/usb/usbip/stub_dev.c
index a3ec49b..c653ce5 100644
--- a/drivers/usb/usbip/stub_dev.c
+++ b/drivers/usb/usbip/stub_dev.c
@@ -219,7 +219,7 @@
dev_dbg(&udev->dev, "device reset");
- ret = usb_lock_device_for_reset(udev, sdev->interface);
+ ret = usb_lock_device_for_reset(udev, NULL);
if (ret < 0) {
dev_err(&udev->dev, "lock for reset\n");
spin_lock_irq(&ud->lock);
@@ -252,7 +252,7 @@
/**
* stub_device_alloc - allocate a new stub_device struct
- * @interface: usb_interface of a new device
+ * @udev: usb_device of a new device
*
* Allocates and initializes a new stub_device struct.
*/
@@ -388,7 +388,6 @@
err_port:
dev_set_drvdata(&udev->dev, NULL);
usb_put_dev(udev);
- kthread_stop_put(sdev->ud.eh);
busid_priv->sdev = NULL;
stub_device_free(sdev);
@@ -449,7 +448,7 @@
}
/* If usb reset is called from event handler */
- if (busid_priv->sdev->ud.eh == current)
+ if (usbip_in_eh(current))
return;
/* shutdown the current connection */
diff --git a/drivers/usb/usbip/stub_rx.c b/drivers/usb/usbip/stub_rx.c
index 00e475c..2df63e3 100644
--- a/drivers/usb/usbip/stub_rx.c
+++ b/drivers/usb/usbip/stub_rx.c
@@ -165,12 +165,7 @@
dev_info(&urb->dev->dev, "usb_queue_reset_device\n");
- /*
- * With the implementation of pre_reset and post_reset the driver no
- * longer unbinds. This allows the use of synchronous reset.
- */
-
- if (usb_lock_device_for_reset(sdev->udev, sdev->interface) < 0) {
+ if (usb_lock_device_for_reset(sdev->udev, NULL) < 0) {
dev_err(&urb->dev->dev, "could not obtain lock to reset device\n");
return 0;
}
@@ -321,7 +316,7 @@
priv = kmem_cache_zalloc(stub_priv_cache, GFP_ATOMIC);
if (!priv) {
- dev_err(&sdev->interface->dev, "alloc stub_priv\n");
+ dev_err(&sdev->udev->dev, "alloc stub_priv\n");
spin_unlock_irqrestore(&sdev->priv_lock, flags);
usbip_event_add(ud, SDEV_EVENT_ERROR_MALLOC);
return NULL;
@@ -352,7 +347,7 @@
else
ep = udev->ep_out[epnum & 0x7f];
if (!ep) {
- dev_err(&sdev->interface->dev, "no such endpoint?, %d\n",
+ dev_err(&sdev->udev->dev, "no such endpoint?, %d\n",
epnum);
BUG();
}
@@ -387,7 +382,7 @@
}
/* NOT REACHED */
- dev_err(&sdev->interface->dev, "get pipe, epnum %d\n", epnum);
+ dev_err(&sdev->udev->dev, "get pipe, epnum %d\n", epnum);
return 0;
}
@@ -466,7 +461,7 @@
priv->urb = usb_alloc_urb(0, GFP_KERNEL);
if (!priv->urb) {
- dev_err(&sdev->interface->dev, "malloc urb\n");
+ dev_err(&udev->dev, "malloc urb\n");
usbip_event_add(ud, SDEV_EVENT_ERROR_MALLOC);
return;
}
@@ -486,7 +481,7 @@
priv->urb->setup_packet = kmemdup(&pdu->u.cmd_submit.setup, 8,
GFP_KERNEL);
if (!priv->urb->setup_packet) {
- dev_err(&sdev->interface->dev, "allocate setup_packet\n");
+ dev_err(&udev->dev, "allocate setup_packet\n");
usbip_event_add(ud, SDEV_EVENT_ERROR_MALLOC);
return;
}
@@ -517,7 +512,7 @@
usbip_dbg_stub_rx("submit urb ok, seqnum %u\n",
pdu->base.seqnum);
else {
- dev_err(&sdev->interface->dev, "submit_urb error, %d\n", ret);
+ dev_err(&udev->dev, "submit_urb error, %d\n", ret);
usbip_dump_header(pdu);
usbip_dump_urb(priv->urb);
diff --git a/drivers/usb/usbip/stub_tx.c b/drivers/usb/usbip/stub_tx.c
index dbcabc9..6b1e8c3 100644
--- a/drivers/usb/usbip/stub_tx.c
+++ b/drivers/usb/usbip/stub_tx.c
@@ -97,7 +97,10 @@
/* link a urb to the queue of tx. */
spin_lock_irqsave(&sdev->priv_lock, flags);
- if (priv->unlinking) {
+ if (sdev->ud.tcp_socket == NULL) {
+ usbip_dbg_stub_tx("ignore urb for closed connection %p", urb);
+ /* It will be freed in stub_device_cleanup_urbs(). */
+ } else if (priv->unlinking) {
stub_enqueue_ret_unlink(sdev, priv->seqnum, urb->status);
stub_free_priv_and_urb(priv);
} else {
@@ -229,7 +232,7 @@
}
if (txsize != sizeof(pdu_header) + urb->actual_length) {
- dev_err(&sdev->interface->dev,
+ dev_err(&sdev->udev->dev,
"actual length of urb %d does not match iso packet sizes %zu\n",
urb->actual_length,
txsize-sizeof(pdu_header));
@@ -261,7 +264,7 @@
ret = kernel_sendmsg(sdev->ud.tcp_socket, &msg,
iov, iovnum, txsize);
if (ret != txsize) {
- dev_err(&sdev->interface->dev,
+ dev_err(&sdev->udev->dev,
"sendmsg failed!, retval %d for %zd\n",
ret, txsize);
kfree(iov);
@@ -336,7 +339,7 @@
ret = kernel_sendmsg(sdev->ud.tcp_socket, &msg, iov,
1, txsize);
if (ret != txsize) {
- dev_err(&sdev->interface->dev,
+ dev_err(&sdev->udev->dev,
"sendmsg failed!, retval %d for %zd\n",
ret, txsize);
usbip_event_add(&sdev->ud, SDEV_EVENT_ERROR_TCP);
diff --git a/drivers/usb/usbip/usbip_common.c b/drivers/usb/usbip/usbip_common.c
index e40da77..8b23229 100644
--- a/drivers/usb/usbip/usbip_common.c
+++ b/drivers/usb/usbip/usbip_common.c
@@ -1,5 +1,7 @@
/*
* Copyright (C) 2003-2008 Takahiro Hirofuchi
+ * Copyright (C) 2015-2016 Samsung Electronics
+ * Krzysztof Opasiak <k.opasiak@samsung.com>
*
* This is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -643,7 +645,7 @@
ret);
kfree(buff);
- if (ud->side == USBIP_STUB)
+ if (ud->side == USBIP_STUB || ud->side == USBIP_VUDC)
usbip_event_add(ud, SDEV_EVENT_ERROR_TCP);
else
usbip_event_add(ud, VDEV_EVENT_ERROR_TCP);
@@ -665,7 +667,7 @@
"total length of iso packets %d not equal to actual length of buffer %d\n",
total_length, urb->actual_length);
- if (ud->side == USBIP_STUB)
+ if (ud->side == USBIP_STUB || ud->side == USBIP_VUDC)
usbip_event_add(ud, SDEV_EVENT_ERROR_TCP);
else
usbip_event_add(ud, VDEV_EVENT_ERROR_TCP);
@@ -723,7 +725,7 @@
int ret;
int size;
- if (ud->side == USBIP_STUB) {
+ if (ud->side == USBIP_STUB || ud->side == USBIP_VUDC) {
/* the direction of urb must be OUT. */
if (usb_pipein(urb->pipe))
return 0;
@@ -755,7 +757,7 @@
ret = usbip_recv(ud->tcp_socket, urb->transfer_buffer, size);
if (ret != size) {
dev_err(&urb->dev->dev, "recv xbuf, %d\n", ret);
- if (ud->side == USBIP_STUB) {
+ if (ud->side == USBIP_STUB || ud->side == USBIP_VUDC) {
usbip_event_add(ud, SDEV_EVENT_ERROR_TCP);
} else {
usbip_event_add(ud, VDEV_EVENT_ERROR_TCP);
@@ -769,12 +771,19 @@
static int __init usbip_core_init(void)
{
+ int ret;
+
pr_info(DRIVER_DESC " v" USBIP_VERSION "\n");
+ ret = usbip_init_eh();
+ if (ret)
+ return ret;
+
return 0;
}
static void __exit usbip_core_exit(void)
{
+ usbip_finish_eh();
return;
}
diff --git a/drivers/usb/usbip/usbip_common.h b/drivers/usb/usbip/usbip_common.h
index 86b0847..c7508cb 100644
--- a/drivers/usb/usbip/usbip_common.h
+++ b/drivers/usb/usbip/usbip_common.h
@@ -1,5 +1,7 @@
/*
* Copyright (C) 2003-2008 Takahiro Hirofuchi
+ * Copyright (C) 2015-2016 Samsung Electronics
+ * Krzysztof Opasiak <k.opasiak@samsung.com>
*
* This is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -234,6 +236,7 @@
enum usbip_side {
USBIP_VHCI,
USBIP_STUB,
+ USBIP_VUDC,
};
/* event handler */
@@ -248,6 +251,13 @@
#define SDEV_EVENT_ERROR_SUBMIT (USBIP_EH_SHUTDOWN | USBIP_EH_RESET)
#define SDEV_EVENT_ERROR_MALLOC (USBIP_EH_SHUTDOWN | USBIP_EH_UNUSABLE)
+#define VUDC_EVENT_REMOVED (USBIP_EH_SHUTDOWN | USBIP_EH_RESET | USBIP_EH_BYE)
+#define VUDC_EVENT_DOWN (USBIP_EH_SHUTDOWN | USBIP_EH_RESET)
+#define VUDC_EVENT_ERROR_TCP (USBIP_EH_SHUTDOWN | USBIP_EH_RESET)
+/* catastrophic emulated usb error */
+#define VUDC_EVENT_ERROR_USB (USBIP_EH_SHUTDOWN | USBIP_EH_UNUSABLE)
+#define VUDC_EVENT_ERROR_MALLOC (USBIP_EH_SHUTDOWN | USBIP_EH_UNUSABLE)
+
#define VDEV_EVENT_REMOVED (USBIP_EH_SHUTDOWN | USBIP_EH_BYE)
#define VDEV_EVENT_DOWN (USBIP_EH_SHUTDOWN | USBIP_EH_RESET)
#define VDEV_EVENT_ERROR_TCP (USBIP_EH_SHUTDOWN | USBIP_EH_RESET)
@@ -267,7 +277,6 @@
struct task_struct *tcp_tx;
unsigned long event;
- struct task_struct *eh;
wait_queue_head_t eh_waitq;
struct eh_ops {
@@ -313,10 +322,13 @@
int usbip_recv_xbuff(struct usbip_device *ud, struct urb *urb);
/* usbip_event.c */
+int usbip_init_eh(void);
+void usbip_finish_eh(void);
int usbip_start_eh(struct usbip_device *ud);
void usbip_stop_eh(struct usbip_device *ud);
void usbip_event_add(struct usbip_device *ud, unsigned long event);
int usbip_event_happened(struct usbip_device *ud);
+int usbip_in_eh(struct task_struct *task);
static inline int interface_to_busnum(struct usb_interface *interface)
{
diff --git a/drivers/usb/usbip/usbip_event.c b/drivers/usb/usbip/usbip_event.c
index 2580a32..f163566 100644
--- a/drivers/usb/usbip/usbip_event.c
+++ b/drivers/usb/usbip/usbip_event.c
@@ -1,5 +1,6 @@
/*
* Copyright (C) 2003-2008 Takahiro Hirofuchi
+ * Copyright (C) 2015 Nobuo Iwata
*
* This is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -19,17 +20,68 @@
#include <linux/kthread.h>
#include <linux/export.h>
+#include <linux/slab.h>
+#include <linux/workqueue.h>
#include "usbip_common.h"
-static int event_handler(struct usbip_device *ud)
-{
- usbip_dbg_eh("enter\n");
+struct usbip_event {
+ struct list_head node;
+ struct usbip_device *ud;
+};
- /*
- * Events are handled by only this thread.
- */
- while (usbip_event_happened(ud)) {
+static DEFINE_SPINLOCK(event_lock);
+static LIST_HEAD(event_list);
+
+static void set_event(struct usbip_device *ud, unsigned long event)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&ud->lock, flags);
+ ud->event |= event;
+ spin_unlock_irqrestore(&ud->lock, flags);
+}
+
+static void unset_event(struct usbip_device *ud, unsigned long event)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&ud->lock, flags);
+ ud->event &= ~event;
+ spin_unlock_irqrestore(&ud->lock, flags);
+}
+
+static struct usbip_device *get_event(void)
+{
+ struct usbip_event *ue = NULL;
+ struct usbip_device *ud = NULL;
+ unsigned long flags;
+
+ spin_lock_irqsave(&event_lock, flags);
+ if (!list_empty(&event_list)) {
+ ue = list_first_entry(&event_list, struct usbip_event, node);
+ list_del(&ue->node);
+ }
+ spin_unlock_irqrestore(&event_lock, flags);
+
+ if (ue) {
+ ud = ue->ud;
+ kfree(ue);
+ }
+ return ud;
+}
+
+static struct task_struct *worker_context;
+
+static void event_handler(struct work_struct *work)
+{
+ struct usbip_device *ud;
+
+ if (worker_context == NULL) {
+ worker_context = current;
+ }
+
+ while ((ud = get_event()) != NULL) {
usbip_dbg_eh("pending event %lx\n", ud->event);
/*
@@ -38,79 +90,102 @@
*/
if (ud->event & USBIP_EH_SHUTDOWN) {
ud->eh_ops.shutdown(ud);
- ud->event &= ~USBIP_EH_SHUTDOWN;
+ unset_event(ud, USBIP_EH_SHUTDOWN);
}
/* Reset the device. */
if (ud->event & USBIP_EH_RESET) {
ud->eh_ops.reset(ud);
- ud->event &= ~USBIP_EH_RESET;
+ unset_event(ud, USBIP_EH_RESET);
}
/* Mark the device as unusable. */
if (ud->event & USBIP_EH_UNUSABLE) {
ud->eh_ops.unusable(ud);
- ud->event &= ~USBIP_EH_UNUSABLE;
+ unset_event(ud, USBIP_EH_UNUSABLE);
}
/* Stop the error handler. */
if (ud->event & USBIP_EH_BYE)
- return -1;
+ usbip_dbg_eh("removed %p\n", ud);
+
+ wake_up(&ud->eh_waitq);
}
-
- return 0;
-}
-
-static int event_handler_loop(void *data)
-{
- struct usbip_device *ud = data;
-
- while (!kthread_should_stop()) {
- wait_event_interruptible(ud->eh_waitq,
- usbip_event_happened(ud) ||
- kthread_should_stop());
- usbip_dbg_eh("wakeup\n");
-
- if (event_handler(ud) < 0)
- break;
- }
-
- return 0;
}
int usbip_start_eh(struct usbip_device *ud)
{
init_waitqueue_head(&ud->eh_waitq);
ud->event = 0;
-
- ud->eh = kthread_run(event_handler_loop, ud, "usbip_eh");
- if (IS_ERR(ud->eh)) {
- pr_warn("Unable to start control thread\n");
- return PTR_ERR(ud->eh);
- }
-
return 0;
}
EXPORT_SYMBOL_GPL(usbip_start_eh);
void usbip_stop_eh(struct usbip_device *ud)
{
- if (ud->eh == current)
- return; /* do not wait for myself */
+ unsigned long pending = ud->event & ~USBIP_EH_BYE;
- kthread_stop(ud->eh);
- usbip_dbg_eh("usbip_eh has finished\n");
+ if (!(ud->event & USBIP_EH_BYE))
+ usbip_dbg_eh("usbip_eh stopping but not removed\n");
+
+ if (pending)
+ usbip_dbg_eh("usbip_eh waiting completion %lx\n", pending);
+
+ wait_event_interruptible(ud->eh_waitq, !(ud->event & ~USBIP_EH_BYE));
+ usbip_dbg_eh("usbip_eh has stopped\n");
}
EXPORT_SYMBOL_GPL(usbip_stop_eh);
+#define WORK_QUEUE_NAME "usbip_event"
+
+static struct workqueue_struct *usbip_queue;
+static DECLARE_WORK(usbip_work, event_handler);
+
+int usbip_init_eh(void)
+{
+ usbip_queue = create_singlethread_workqueue(WORK_QUEUE_NAME);
+ if (usbip_queue == NULL) {
+ pr_err("failed to create usbip_event\n");
+ return -ENOMEM;
+ }
+ return 0;
+}
+
+void usbip_finish_eh(void)
+{
+ flush_workqueue(usbip_queue);
+ destroy_workqueue(usbip_queue);
+ usbip_queue = NULL;
+}
+
void usbip_event_add(struct usbip_device *ud, unsigned long event)
{
+ struct usbip_event *ue;
unsigned long flags;
- spin_lock_irqsave(&ud->lock, flags);
- ud->event |= event;
- wake_up(&ud->eh_waitq);
- spin_unlock_irqrestore(&ud->lock, flags);
+ if (ud->event & USBIP_EH_BYE)
+ return;
+
+ set_event(ud, event);
+
+ spin_lock_irqsave(&event_lock, flags);
+
+ list_for_each_entry_reverse(ue, &event_list, node) {
+ if (ue->ud == ud)
+ goto out;
+ }
+
+ ue = kmalloc(sizeof(struct usbip_event), GFP_ATOMIC);
+ if (ue == NULL)
+ goto out;
+
+ ue->ud = ud;
+
+ list_add_tail(&ue->node, &event_list);
+ queue_work(usbip_queue, &usbip_work);
+
+out:
+ spin_unlock_irqrestore(&event_lock, flags);
}
EXPORT_SYMBOL_GPL(usbip_event_add);
@@ -127,3 +202,12 @@
return happened;
}
EXPORT_SYMBOL_GPL(usbip_event_happened);
+
+int usbip_in_eh(struct task_struct *task)
+{
+ if (task == worker_context)
+ return 1;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(usbip_in_eh);
diff --git a/drivers/usb/usbip/vudc.h b/drivers/usb/usbip/vudc.h
new file mode 100644
index 0000000..25e01b0
--- /dev/null
+++ b/drivers/usb/usbip/vudc.h
@@ -0,0 +1,190 @@
+/*
+ * Copyright (C) 2015 Karol Kosik <karo9@interia.eu>
+ * Copyright (C) 2015-2016 Samsung Electronics
+ * Igor Kotrasinski <i.kotrasinsk@samsung.com>
+ * Krzysztof Opasiak <k.opasiak@samsung.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __USBIP_VUDC_H
+#define __USBIP_VUDC_H
+
+#include <linux/platform_device.h>
+#include <linux/usb.h>
+#include <linux/usb/gadget.h>
+#include <linux/usb/ch9.h>
+#include <linux/list.h>
+#include <linux/timer.h>
+#include <linux/time.h>
+#include <linux/sysfs.h>
+
+#include "usbip_common.h"
+
+#define GADGET_NAME "usbip-vudc"
+
+struct vep {
+ struct usb_ep ep;
+ unsigned type:2; /* type, as USB_ENDPOINT_XFER_* */
+ char name[8]; /* space for ep name */
+
+ const struct usb_endpoint_descriptor *desc;
+ struct usb_gadget *gadget;
+ struct list_head req_queue; /* Request queue */
+ unsigned halted:1;
+ unsigned wedged:1;
+ unsigned already_seen:1;
+ unsigned setup_stage:1;
+};
+
+struct vrequest {
+ struct usb_request req;
+ struct vudc *udc;
+ struct list_head req_entry; /* Request queue */
+};
+
+struct urbp {
+ struct urb *urb;
+ struct vep *ep;
+ struct list_head urb_entry; /* urb queue */
+ unsigned long seqnum;
+ unsigned type:2; /* for tx, since ep type can change after */
+ unsigned new:1;
+};
+
+struct v_unlink {
+ unsigned long seqnum;
+ __u32 status;
+};
+
+enum tx_type {
+ TX_UNLINK,
+ TX_SUBMIT,
+};
+
+struct tx_item {
+ struct list_head tx_entry;
+ enum tx_type type;
+ union {
+ struct urbp *s;
+ struct v_unlink *u;
+ };
+};
+
+enum tr_state {
+ VUDC_TR_RUNNING,
+ VUDC_TR_IDLE,
+ VUDC_TR_STOPPED,
+};
+
+struct transfer_timer {
+ struct timer_list timer;
+ enum tr_state state;
+ unsigned long frame_start;
+ int frame_limit;
+};
+
+struct vudc {
+ struct usb_gadget gadget;
+ struct usb_gadget_driver *driver;
+ struct platform_device *pdev;
+
+ struct usb_device_descriptor dev_desc;
+
+ struct usbip_device ud;
+ struct transfer_timer tr_timer;
+ struct timeval start_time;
+
+ struct list_head urb_queue;
+
+ spinlock_t lock_tx;
+ struct list_head tx_queue;
+ wait_queue_head_t tx_waitq;
+
+ spinlock_t lock;
+ struct vep *ep;
+ int address;
+ u16 devstatus;
+
+ unsigned pullup:1;
+ unsigned connected:1;
+ unsigned desc_cached:1;
+};
+
+struct vudc_device {
+ struct platform_device *pdev;
+ struct list_head dev_entry;
+};
+
+extern const struct attribute_group vudc_attr_group;
+
+/* visible everywhere */
+
+static inline struct vep *to_vep(struct usb_ep *_ep)
+{
+ return container_of(_ep, struct vep, ep);
+}
+
+static inline struct vrequest *to_vrequest(
+ struct usb_request *_req)
+{
+ return container_of(_req, struct vrequest, req);
+}
+
+static inline struct vudc *usb_gadget_to_vudc(
+ struct usb_gadget *_gadget)
+{
+ return container_of(_gadget, struct vudc, gadget);
+}
+
+static inline struct vudc *ep_to_vudc(struct vep *ep)
+{
+ return container_of(ep->gadget, struct vudc, gadget);
+}
+
+/* vudc_sysfs.c */
+
+int get_gadget_descs(struct vudc *udc);
+
+/* vudc_tx.c */
+
+int v_tx_loop(void *data);
+void v_enqueue_ret_unlink(struct vudc *udc, __u32 seqnum, __u32 status);
+void v_enqueue_ret_submit(struct vudc *udc, struct urbp *urb_p);
+
+/* vudc_rx.c */
+
+int v_rx_loop(void *data);
+
+/* vudc_transfer.c */
+
+void v_init_timer(struct vudc *udc);
+void v_start_timer(struct vudc *udc);
+void v_kick_timer(struct vudc *udc, unsigned long time);
+void v_stop_timer(struct vudc *udc);
+
+/* vudc_dev.c */
+
+struct urbp *alloc_urbp(void);
+void free_urbp_and_urb(struct urbp *urb_p);
+
+struct vep *vudc_find_endpoint(struct vudc *udc, u8 address);
+
+struct vudc_device *alloc_vudc_device(int devid);
+void put_vudc_device(struct vudc_device *udc_dev);
+
+int vudc_probe(struct platform_device *pdev);
+int vudc_remove(struct platform_device *pdev);
+
+#endif /* __USBIP_VUDC_H */
diff --git a/drivers/usb/usbip/vudc_dev.c b/drivers/usb/usbip/vudc_dev.c
new file mode 100644
index 0000000..8994a13
--- /dev/null
+++ b/drivers/usb/usbip/vudc_dev.c
@@ -0,0 +1,661 @@
+/*
+ * Copyright (C) 2015 Karol Kosik <karo9@interia.eu>
+ * Copyright (C) 2015-2016 Samsung Electronics
+ * Igor Kotrasinski <i.kotrasinsk@samsung.com>
+ * Krzysztof Opasiak <k.opasiak@samsung.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/device.h>
+#include <linux/kernel.h>
+#include <linux/list.h>
+#include <linux/platform_device.h>
+#include <linux/usb.h>
+#include <linux/usb/gadget.h>
+#include <linux/usb/hcd.h>
+#include <linux/kthread.h>
+#include <linux/file.h>
+#include <linux/byteorder/generic.h>
+
+#include "usbip_common.h"
+#include "vudc.h"
+
+#define VIRTUAL_ENDPOINTS (1 /* ep0 */ + 15 /* in eps */ + 15 /* out eps */)
+
+/* urb-related structures alloc / free */
+
+
+static void free_urb(struct urb *urb)
+{
+ if (!urb)
+ return;
+
+ kfree(urb->setup_packet);
+ urb->setup_packet = NULL;
+
+ kfree(urb->transfer_buffer);
+ urb->transfer_buffer = NULL;
+
+ usb_free_urb(urb);
+}
+
+struct urbp *alloc_urbp(void)
+{
+ struct urbp *urb_p;
+
+ urb_p = kzalloc(sizeof(*urb_p), GFP_KERNEL);
+ if (!urb_p)
+ return urb_p;
+
+ urb_p->urb = NULL;
+ urb_p->ep = NULL;
+ INIT_LIST_HEAD(&urb_p->urb_entry);
+ return urb_p;
+}
+
+static void free_urbp(struct urbp *urb_p)
+{
+ kfree(urb_p);
+}
+
+void free_urbp_and_urb(struct urbp *urb_p)
+{
+ if (!urb_p)
+ return;
+ free_urb(urb_p->urb);
+ free_urbp(urb_p);
+}
+
+
+/* utilities ; almost verbatim from dummy_hcd.c */
+
+/* called with spinlock held */
+static void nuke(struct vudc *udc, struct vep *ep)
+{
+ struct vrequest *req;
+
+ while (!list_empty(&ep->req_queue)) {
+ req = list_first_entry(&ep->req_queue, struct vrequest,
+ req_entry);
+ list_del_init(&req->req_entry);
+ req->req.status = -ESHUTDOWN;
+
+ spin_unlock(&udc->lock);
+ usb_gadget_giveback_request(&ep->ep, &req->req);
+ spin_lock(&udc->lock);
+ }
+}
+
+/* caller must hold lock */
+static void stop_activity(struct vudc *udc)
+{
+ int i;
+ struct urbp *urb_p, *tmp;
+
+ udc->address = 0;
+
+ for (i = 0; i < VIRTUAL_ENDPOINTS; i++)
+ nuke(udc, &udc->ep[i]);
+
+ list_for_each_entry_safe(urb_p, tmp, &udc->urb_queue, urb_entry) {
+ list_del(&urb_p->urb_entry);
+ free_urbp_and_urb(urb_p);
+ }
+}
+
+struct vep *vudc_find_endpoint(struct vudc *udc, u8 address)
+{
+ int i;
+
+ if ((address & ~USB_DIR_IN) == 0)
+ return &udc->ep[0];
+
+ for (i = 1; i < VIRTUAL_ENDPOINTS; i++) {
+ struct vep *ep = &udc->ep[i];
+
+ if (!ep->desc)
+ continue;
+ if (ep->desc->bEndpointAddress == address)
+ return ep;
+ }
+ return NULL;
+}
+
+/* gadget ops */
+
+/* FIXME - this will probably misbehave when suspend/resume is added */
+static int vgadget_get_frame(struct usb_gadget *_gadget)
+{
+ struct timeval now;
+ struct vudc *udc = usb_gadget_to_vudc(_gadget);
+
+ do_gettimeofday(&now);
+ return ((now.tv_sec - udc->start_time.tv_sec) * 1000 +
+ (now.tv_usec - udc->start_time.tv_usec) / 1000)
+ % 0x7FF;
+}
+
+static int vgadget_set_selfpowered(struct usb_gadget *_gadget, int value)
+{
+ struct vudc *udc = usb_gadget_to_vudc(_gadget);
+
+ if (value)
+ udc->devstatus |= (1 << USB_DEVICE_SELF_POWERED);
+ else
+ udc->devstatus &= ~(1 << USB_DEVICE_SELF_POWERED);
+ return 0;
+}
+
+static int vgadget_pullup(struct usb_gadget *_gadget, int value)
+{
+ struct vudc *udc = usb_gadget_to_vudc(_gadget);
+ unsigned long flags;
+ int ret;
+
+
+ spin_lock_irqsave(&udc->lock, flags);
+ value = !!value;
+ if (value == udc->pullup)
+ goto unlock;
+
+ udc->pullup = value;
+ if (value) {
+ udc->gadget.speed = min_t(u8, USB_SPEED_HIGH,
+ udc->driver->max_speed);
+ udc->ep[0].ep.maxpacket = 64;
+ /*
+ * This is the first place where we can ask our
+ * gadget driver for descriptors.
+ */
+ ret = get_gadget_descs(udc);
+ if (ret) {
+ dev_err(&udc->gadget.dev, "Unable go get desc: %d", ret);
+ goto unlock;
+ }
+
+ spin_unlock_irqrestore(&udc->lock, flags);
+ usbip_start_eh(&udc->ud);
+ } else {
+ /* Invalidate descriptors */
+ udc->desc_cached = 0;
+
+ spin_unlock_irqrestore(&udc->lock, flags);
+ usbip_event_add(&udc->ud, VUDC_EVENT_REMOVED);
+ usbip_stop_eh(&udc->ud); /* Wait for eh completion */
+ }
+
+ return 0;
+
+unlock:
+ spin_unlock_irqrestore(&udc->lock, flags);
+ return 0;
+}
+
+static int vgadget_udc_start(struct usb_gadget *g,
+ struct usb_gadget_driver *driver)
+{
+ struct vudc *udc = usb_gadget_to_vudc(g);
+ unsigned long flags;
+
+ spin_lock_irqsave(&udc->lock, flags);
+ udc->driver = driver;
+ udc->pullup = udc->connected = udc->desc_cached = 0;
+ spin_unlock_irqrestore(&udc->lock, flags);
+
+ return 0;
+}
+
+static int vgadget_udc_stop(struct usb_gadget *g)
+{
+ struct vudc *udc = usb_gadget_to_vudc(g);
+ unsigned long flags;
+
+ spin_lock_irqsave(&udc->lock, flags);
+ udc->driver = NULL;
+ spin_unlock_irqrestore(&udc->lock, flags);
+ return 0;
+}
+
+static const struct usb_gadget_ops vgadget_ops = {
+ .get_frame = vgadget_get_frame,
+ .set_selfpowered = vgadget_set_selfpowered,
+ .pullup = vgadget_pullup,
+ .udc_start = vgadget_udc_start,
+ .udc_stop = vgadget_udc_stop,
+};
+
+
+/* endpoint ops */
+
+static int vep_enable(struct usb_ep *_ep,
+ const struct usb_endpoint_descriptor *desc)
+{
+ struct vep *ep;
+ struct vudc *udc;
+ unsigned maxp;
+ unsigned long flags;
+
+ ep = to_vep(_ep);
+ udc = ep_to_vudc(ep);
+
+ if (!_ep || !desc || ep->desc || _ep->caps.type_control
+ || desc->bDescriptorType != USB_DT_ENDPOINT)
+ return -EINVAL;
+
+ if (!udc->driver)
+ return -ESHUTDOWN;
+
+ spin_lock_irqsave(&udc->lock, flags);
+
+ maxp = usb_endpoint_maxp(desc) & 0x7ff;
+ _ep->maxpacket = maxp;
+ ep->desc = desc;
+ ep->type = usb_endpoint_type(desc);
+ ep->halted = ep->wedged = 0;
+
+ spin_unlock_irqrestore(&udc->lock, flags);
+
+ return 0;
+}
+
+static int vep_disable(struct usb_ep *_ep)
+{
+ struct vep *ep;
+ struct vudc *udc;
+ unsigned long flags;
+
+ ep = to_vep(_ep);
+ udc = ep_to_vudc(ep);
+ if (!_ep || !ep->desc || _ep->caps.type_control)
+ return -EINVAL;
+
+ spin_lock_irqsave(&udc->lock, flags);
+ ep->desc = NULL;
+ nuke(udc, ep);
+ spin_unlock_irqrestore(&udc->lock, flags);
+
+ return 0;
+}
+
+static struct usb_request *vep_alloc_request(struct usb_ep *_ep,
+ gfp_t mem_flags)
+{
+ struct vep *ep;
+ struct vrequest *req;
+
+ if (!_ep)
+ return NULL;
+ ep = to_vep(_ep);
+
+ req = kzalloc(sizeof(*req), mem_flags);
+ if (!req)
+ return NULL;
+
+ INIT_LIST_HEAD(&req->req_entry);
+
+ return &req->req;
+}
+
+static void vep_free_request(struct usb_ep *_ep, struct usb_request *_req)
+{
+ struct vrequest *req;
+
+ if (WARN_ON(!_ep || !_req))
+ return;
+
+ req = to_vrequest(_req);
+ kfree(req);
+}
+
+static int vep_queue(struct usb_ep *_ep, struct usb_request *_req,
+ gfp_t mem_flags)
+{
+ struct vep *ep;
+ struct vrequest *req;
+ struct vudc *udc;
+ unsigned long flags;
+
+ if (!_ep || !_req)
+ return -EINVAL;
+
+ ep = to_vep(_ep);
+ req = to_vrequest(_req);
+ udc = ep_to_vudc(ep);
+
+ spin_lock_irqsave(&udc->lock, flags);
+ _req->actual = 0;
+ _req->status = -EINPROGRESS;
+
+ list_add_tail(&req->req_entry, &ep->req_queue);
+ spin_unlock_irqrestore(&udc->lock, flags);
+
+ return 0;
+}
+
+static int vep_dequeue(struct usb_ep *_ep, struct usb_request *_req)
+{
+ struct vep *ep;
+ struct vrequest *req;
+ struct vudc *udc;
+ struct vrequest *lst;
+ unsigned long flags;
+ int ret = -EINVAL;
+
+ if (!_ep || !_req)
+ return ret;
+
+ ep = to_vep(_ep);
+ req = to_vrequest(_req);
+ udc = req->udc;
+
+ if (!udc->driver)
+ return -ESHUTDOWN;
+
+ spin_lock_irqsave(&udc->lock, flags);
+ list_for_each_entry(lst, &ep->req_queue, req_entry) {
+ if (&lst->req == _req) {
+ list_del_init(&lst->req_entry);
+ _req->status = -ECONNRESET;
+ ret = 0;
+ break;
+ }
+ }
+ spin_unlock_irqrestore(&udc->lock, flags);
+
+ if (ret == 0)
+ usb_gadget_giveback_request(_ep, _req);
+
+ return ret;
+}
+
+static int
+vep_set_halt_and_wedge(struct usb_ep *_ep, int value, int wedged)
+{
+ struct vep *ep;
+ struct vudc *udc;
+ unsigned long flags;
+ int ret = 0;
+
+ ep = to_vep(_ep);
+ if (!_ep)
+ return -EINVAL;
+
+ udc = ep_to_vudc(ep);
+ if (!udc->driver)
+ return -ESHUTDOWN;
+
+ spin_lock_irqsave(&udc->lock, flags);
+ if (!value)
+ ep->halted = ep->wedged = 0;
+ else if (ep->desc && (ep->desc->bEndpointAddress & USB_DIR_IN) &&
+ !list_empty(&ep->req_queue))
+ ret = -EAGAIN;
+ else {
+ ep->halted = 1;
+ if (wedged)
+ ep->wedged = 1;
+ }
+
+ spin_unlock_irqrestore(&udc->lock, flags);
+ return ret;
+}
+
+static int
+vep_set_halt(struct usb_ep *_ep, int value)
+{
+ return vep_set_halt_and_wedge(_ep, value, 0);
+}
+
+static int vep_set_wedge(struct usb_ep *_ep)
+{
+ return vep_set_halt_and_wedge(_ep, 1, 1);
+}
+
+static const struct usb_ep_ops vep_ops = {
+ .enable = vep_enable,
+ .disable = vep_disable,
+
+ .alloc_request = vep_alloc_request,
+ .free_request = vep_free_request,
+
+ .queue = vep_queue,
+ .dequeue = vep_dequeue,
+
+ .set_halt = vep_set_halt,
+ .set_wedge = vep_set_wedge,
+};
+
+
+/* shutdown / reset / error handlers */
+
+static void vudc_shutdown(struct usbip_device *ud)
+{
+ struct vudc *udc = container_of(ud, struct vudc, ud);
+ int call_disconnect = 0;
+ unsigned long flags;
+
+ dev_dbg(&udc->pdev->dev, "device shutdown");
+ if (ud->tcp_socket)
+ kernel_sock_shutdown(ud->tcp_socket, SHUT_RDWR);
+
+ if (ud->tcp_tx) {
+ kthread_stop_put(ud->tcp_rx);
+ ud->tcp_rx = NULL;
+ }
+ if (ud->tcp_tx) {
+ kthread_stop_put(ud->tcp_tx);
+ ud->tcp_tx = NULL;
+ }
+
+ if (ud->tcp_socket) {
+ sockfd_put(ud->tcp_socket);
+ ud->tcp_socket = NULL;
+ }
+
+ spin_lock_irqsave(&udc->lock, flags);
+ stop_activity(udc);
+ if (udc->connected && udc->driver->disconnect)
+ call_disconnect = 1;
+ udc->connected = 0;
+ spin_unlock_irqrestore(&udc->lock, flags);
+ if (call_disconnect)
+ udc->driver->disconnect(&udc->gadget);
+}
+
+static void vudc_device_reset(struct usbip_device *ud)
+{
+ struct vudc *udc = container_of(ud, struct vudc, ud);
+ unsigned long flags;
+
+ dev_dbg(&udc->pdev->dev, "device reset");
+ spin_lock_irqsave(&udc->lock, flags);
+ stop_activity(udc);
+ spin_unlock_irqrestore(&udc->lock, flags);
+ if (udc->driver)
+ usb_gadget_udc_reset(&udc->gadget, udc->driver);
+ spin_lock_irqsave(&ud->lock, flags);
+ ud->status = SDEV_ST_AVAILABLE;
+ spin_unlock_irqrestore(&ud->lock, flags);
+}
+
+static void vudc_device_unusable(struct usbip_device *ud)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&ud->lock, flags);
+ ud->status = SDEV_ST_ERROR;
+ spin_unlock_irqrestore(&ud->lock, flags);
+}
+
+/* device setup / cleanup */
+
+struct vudc_device *alloc_vudc_device(int devid)
+{
+ struct vudc_device *udc_dev = NULL;
+
+ udc_dev = kzalloc(sizeof(*udc_dev), GFP_KERNEL);
+ if (!udc_dev)
+ goto out;
+
+ INIT_LIST_HEAD(&udc_dev->dev_entry);
+
+ udc_dev->pdev = platform_device_alloc(GADGET_NAME, devid);
+ if (!udc_dev->pdev) {
+ kfree(udc_dev);
+ udc_dev = NULL;
+ }
+
+out:
+ return udc_dev;
+}
+
+void put_vudc_device(struct vudc_device *udc_dev)
+{
+ platform_device_put(udc_dev->pdev);
+ kfree(udc_dev);
+}
+
+static int init_vudc_hw(struct vudc *udc)
+{
+ int i;
+ struct usbip_device *ud = &udc->ud;
+ struct vep *ep;
+
+ udc->ep = kcalloc(VIRTUAL_ENDPOINTS, sizeof(*udc->ep), GFP_KERNEL);
+ if (!udc->ep)
+ goto nomem_ep;
+
+ INIT_LIST_HEAD(&udc->gadget.ep_list);
+
+ /* create ep0 and 15 in, 15 out general purpose eps */
+ for (i = 0; i < VIRTUAL_ENDPOINTS; ++i) {
+ int is_out = i % 2;
+ int num = (i + 1) / 2;
+
+ ep = &udc->ep[i];
+
+ sprintf(ep->name, "ep%d%s", num,
+ i ? (is_out ? "out" : "in") : "");
+ ep->ep.name = ep->name;
+ if (i == 0) {
+ ep->ep.caps.type_control = true;
+ ep->ep.caps.dir_out = true;
+ ep->ep.caps.dir_in = true;
+ } else {
+ ep->ep.caps.type_iso = true;
+ ep->ep.caps.type_int = true;
+ ep->ep.caps.type_bulk = true;
+ }
+
+ if (is_out)
+ ep->ep.caps.dir_out = true;
+ else
+ ep->ep.caps.dir_in = true;
+
+ ep->ep.ops = &vep_ops;
+ list_add_tail(&ep->ep.ep_list, &udc->gadget.ep_list);
+ ep->halted = ep->wedged = ep->already_seen =
+ ep->setup_stage = 0;
+ usb_ep_set_maxpacket_limit(&ep->ep, ~0);
+ ep->ep.max_streams = 16;
+ ep->gadget = &udc->gadget;
+ ep->desc = NULL;
+ INIT_LIST_HEAD(&ep->req_queue);
+ }
+
+ spin_lock_init(&udc->lock);
+ spin_lock_init(&udc->lock_tx);
+ INIT_LIST_HEAD(&udc->urb_queue);
+ INIT_LIST_HEAD(&udc->tx_queue);
+ init_waitqueue_head(&udc->tx_waitq);
+
+ spin_lock_init(&ud->lock);
+ ud->status = SDEV_ST_AVAILABLE;
+ ud->side = USBIP_VUDC;
+
+ ud->eh_ops.shutdown = vudc_shutdown;
+ ud->eh_ops.reset = vudc_device_reset;
+ ud->eh_ops.unusable = vudc_device_unusable;
+
+ udc->gadget.ep0 = &udc->ep[0].ep;
+ list_del_init(&udc->ep[0].ep.ep_list);
+
+ v_init_timer(udc);
+ return 0;
+
+nomem_ep:
+ return -ENOMEM;
+}
+
+static void cleanup_vudc_hw(struct vudc *udc)
+{
+ kfree(udc->ep);
+}
+
+/* platform driver ops */
+
+int vudc_probe(struct platform_device *pdev)
+{
+ struct vudc *udc;
+ int ret = -ENOMEM;
+
+ udc = kzalloc(sizeof(*udc), GFP_KERNEL);
+ if (!udc)
+ goto out;
+
+ udc->gadget.name = GADGET_NAME;
+ udc->gadget.ops = &vgadget_ops;
+ udc->gadget.max_speed = USB_SPEED_HIGH;
+ udc->gadget.dev.parent = &pdev->dev;
+ udc->pdev = pdev;
+
+ ret = init_vudc_hw(udc);
+ if (ret)
+ goto err_init_vudc_hw;
+
+ ret = usb_add_gadget_udc(&pdev->dev, &udc->gadget);
+ if (ret < 0)
+ goto err_add_udc;
+
+ ret = sysfs_create_group(&pdev->dev.kobj, &vudc_attr_group);
+ if (ret) {
+ dev_err(&udc->pdev->dev, "create sysfs files\n");
+ goto err_sysfs;
+ }
+
+ platform_set_drvdata(pdev, udc);
+
+ return ret;
+
+err_sysfs:
+ usb_del_gadget_udc(&udc->gadget);
+err_add_udc:
+ cleanup_vudc_hw(udc);
+err_init_vudc_hw:
+ kfree(udc);
+out:
+ return ret;
+}
+
+int vudc_remove(struct platform_device *pdev)
+{
+ struct vudc *udc = platform_get_drvdata(pdev);
+
+ sysfs_remove_group(&pdev->dev.kobj, &vudc_attr_group);
+ usb_del_gadget_udc(&udc->gadget);
+ cleanup_vudc_hw(udc);
+ kfree(udc);
+ return 0;
+}
diff --git a/drivers/usb/usbip/vudc_main.c b/drivers/usb/usbip/vudc_main.c
new file mode 100644
index 0000000..9e65571
--- /dev/null
+++ b/drivers/usb/usbip/vudc_main.c
@@ -0,0 +1,113 @@
+/*
+ * Copyright (C) 2015 Karol Kosik <karo9@interia.eu>
+ * Copyright (C) 2015-2016 Samsung Electronics
+ * Igor Kotrasinski <i.kotrasinsk@samsung.com>
+ * Krzysztof Opasiak <k.opasiak@samsung.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/device.h>
+#include <linux/list.h>
+#include <linux/module.h>
+
+#include "vudc.h"
+
+static unsigned int vudc_number = 1;
+
+module_param_named(num, vudc_number, uint, S_IRUGO);
+MODULE_PARM_DESC(num, "number of emulated controllers");
+
+static struct platform_driver vudc_driver = {
+ .probe = vudc_probe,
+ .remove = vudc_remove,
+ .driver = {
+ .name = GADGET_NAME,
+ },
+};
+
+static struct list_head vudc_devices = LIST_HEAD_INIT(vudc_devices);
+
+static int __init init(void)
+{
+ int retval = -ENOMEM;
+ int i;
+ struct vudc_device *udc_dev = NULL, *udc_dev2 = NULL;
+
+ if (usb_disabled())
+ return -ENODEV;
+
+ if (vudc_number < 1) {
+ pr_err("Number of emulated UDC must be no less than 1");
+ return -EINVAL;
+ }
+
+ retval = platform_driver_register(&vudc_driver);
+ if (retval < 0)
+ goto out;
+
+ for (i = 0; i < vudc_number; i++) {
+ udc_dev = alloc_vudc_device(i);
+ if (!udc_dev) {
+ retval = -ENOMEM;
+ goto cleanup;
+ }
+
+ retval = platform_device_add(udc_dev->pdev);
+ if (retval < 0) {
+ put_vudc_device(udc_dev);
+ goto cleanup;
+ }
+
+ list_add_tail(&udc_dev->dev_entry, &vudc_devices);
+ if (!platform_get_drvdata(udc_dev->pdev)) {
+ /*
+ * The udc was added successfully but its probe
+ * function failed for some reason.
+ */
+ retval = -EINVAL;
+ goto cleanup;
+ }
+ }
+ goto out;
+
+cleanup:
+ list_for_each_entry_safe(udc_dev, udc_dev2, &vudc_devices, dev_entry) {
+ list_del(&udc_dev->dev_entry);
+ platform_device_del(udc_dev->pdev);
+ put_vudc_device(udc_dev);
+ }
+
+ platform_driver_unregister(&vudc_driver);
+out:
+ return retval;
+}
+module_init(init);
+
+static void __exit cleanup(void)
+{
+ struct vudc_device *udc_dev = NULL, *udc_dev2 = NULL;
+
+ list_for_each_entry_safe(udc_dev, udc_dev2, &vudc_devices, dev_entry) {
+ list_del(&udc_dev->dev_entry);
+ platform_device_unregister(udc_dev->pdev);
+ put_vudc_device(udc_dev);
+ }
+ platform_driver_unregister(&vudc_driver);
+}
+module_exit(cleanup);
+
+MODULE_DESCRIPTION("USB over IP Device Controller");
+MODULE_AUTHOR("Krzysztof Opasiak, Karol Kosik, Igor Kotrasinski");
+MODULE_LICENSE("GPL");
diff --git a/drivers/usb/usbip/vudc_rx.c b/drivers/usb/usbip/vudc_rx.c
new file mode 100644
index 0000000..344bd94
--- /dev/null
+++ b/drivers/usb/usbip/vudc_rx.c
@@ -0,0 +1,234 @@
+/*
+ * Copyright (C) 2015 Karol Kosik <karo9@interia.eu>
+ * Copyright (C) 2015-2016 Samsung Electronics
+ * Igor Kotrasinski <i.kotrasinsk@samsung.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <net/sock.h>
+#include <linux/list.h>
+#include <linux/kthread.h>
+
+#include "usbip_common.h"
+#include "vudc.h"
+
+static int alloc_urb_from_cmd(struct urb **urbp,
+ struct usbip_header *pdu, u8 type)
+{
+ struct urb *urb;
+
+ if (type == USB_ENDPOINT_XFER_ISOC)
+ urb = usb_alloc_urb(pdu->u.cmd_submit.number_of_packets,
+ GFP_KERNEL);
+ else
+ urb = usb_alloc_urb(0, GFP_KERNEL);
+
+ if (!urb)
+ goto err;
+
+ usbip_pack_pdu(pdu, urb, USBIP_CMD_SUBMIT, 0);
+
+ if (urb->transfer_buffer_length > 0) {
+ urb->transfer_buffer = kzalloc(urb->transfer_buffer_length,
+ GFP_KERNEL);
+ if (!urb->transfer_buffer)
+ goto free_urb;
+ }
+
+ urb->setup_packet = kmemdup(&pdu->u.cmd_submit.setup, 8,
+ GFP_KERNEL);
+ if (!urb->setup_packet)
+ goto free_buffer;
+
+ /*
+ * FIXME - we only setup pipe enough for usbip functions
+ * to behave nicely
+ */
+ urb->pipe |= pdu->base.direction == USBIP_DIR_IN ?
+ USB_DIR_IN : USB_DIR_OUT;
+
+ *urbp = urb;
+ return 0;
+
+free_buffer:
+ kfree(urb->transfer_buffer);
+ urb->transfer_buffer = NULL;
+free_urb:
+ usb_free_urb(urb);
+err:
+ return -ENOMEM;
+}
+
+static int v_recv_cmd_unlink(struct vudc *udc,
+ struct usbip_header *pdu)
+{
+ unsigned long flags;
+ struct urbp *urb_p;
+
+ spin_lock_irqsave(&udc->lock, flags);
+ list_for_each_entry(urb_p, &udc->urb_queue, urb_entry) {
+ if (urb_p->seqnum != pdu->u.cmd_unlink.seqnum)
+ continue;
+ urb_p->urb->unlinked = -ECONNRESET;
+ urb_p->seqnum = pdu->base.seqnum;
+ v_kick_timer(udc, jiffies);
+ spin_unlock_irqrestore(&udc->lock, flags);
+ return 0;
+ }
+ /* Not found, completed / not queued */
+ spin_lock(&udc->lock_tx);
+ v_enqueue_ret_unlink(udc, pdu->base.seqnum, 0);
+ wake_up(&udc->tx_waitq);
+ spin_unlock(&udc->lock_tx);
+ spin_unlock_irqrestore(&udc->lock, flags);
+
+ return 0;
+}
+
+static int v_recv_cmd_submit(struct vudc *udc,
+ struct usbip_header *pdu)
+{
+ int ret = 0;
+ struct urbp *urb_p;
+ u8 address;
+ unsigned long flags;
+
+ urb_p = alloc_urbp();
+ if (!urb_p) {
+ usbip_event_add(&udc->ud, VUDC_EVENT_ERROR_MALLOC);
+ return -ENOMEM;
+ }
+
+ /* base.ep is pipeendpoint(pipe) */
+ address = pdu->base.ep;
+ if (pdu->base.direction == USBIP_DIR_IN)
+ address |= USB_DIR_IN;
+
+ spin_lock_irq(&udc->lock);
+ urb_p->ep = vudc_find_endpoint(udc, address);
+ if (!urb_p->ep) {
+ /* we don't know the type, there may be isoc data! */
+ dev_err(&udc->pdev->dev, "request to nonexistent endpoint");
+ spin_unlock_irq(&udc->lock);
+ usbip_event_add(&udc->ud, VUDC_EVENT_ERROR_TCP);
+ ret = -EPIPE;
+ goto free_urbp;
+ }
+ urb_p->type = urb_p->ep->type;
+ spin_unlock_irq(&udc->lock);
+
+ urb_p->new = 1;
+ urb_p->seqnum = pdu->base.seqnum;
+
+ ret = alloc_urb_from_cmd(&urb_p->urb, pdu, urb_p->ep->type);
+ if (ret) {
+ usbip_event_add(&udc->ud, VUDC_EVENT_ERROR_MALLOC);
+ ret = -ENOMEM;
+ goto free_urbp;
+ }
+
+ urb_p->urb->status = -EINPROGRESS;
+
+ /* FIXME: more pipe setup to please usbip_common */
+ urb_p->urb->pipe &= ~(11 << 30);
+ switch (urb_p->ep->type) {
+ case USB_ENDPOINT_XFER_BULK:
+ urb_p->urb->pipe |= (PIPE_BULK << 30);
+ break;
+ case USB_ENDPOINT_XFER_INT:
+ urb_p->urb->pipe |= (PIPE_INTERRUPT << 30);
+ break;
+ case USB_ENDPOINT_XFER_CONTROL:
+ urb_p->urb->pipe |= (PIPE_CONTROL << 30);
+ break;
+ case USB_ENDPOINT_XFER_ISOC:
+ urb_p->urb->pipe |= (PIPE_ISOCHRONOUS << 30);
+ break;
+ }
+ ret = usbip_recv_xbuff(&udc->ud, urb_p->urb);
+ if (ret < 0)
+ goto free_urbp;
+
+ ret = usbip_recv_iso(&udc->ud, urb_p->urb);
+ if (ret < 0)
+ goto free_urbp;
+
+ spin_lock_irqsave(&udc->lock, flags);
+ v_kick_timer(udc, jiffies);
+ list_add_tail(&urb_p->urb_entry, &udc->urb_queue);
+ spin_unlock_irqrestore(&udc->lock, flags);
+
+ return 0;
+
+free_urbp:
+ free_urbp_and_urb(urb_p);
+ return ret;
+}
+
+static int v_rx_pdu(struct usbip_device *ud)
+{
+ int ret;
+ struct usbip_header pdu;
+ struct vudc *udc = container_of(ud, struct vudc, ud);
+
+ memset(&pdu, 0, sizeof(pdu));
+ ret = usbip_recv(ud->tcp_socket, &pdu, sizeof(pdu));
+ if (ret != sizeof(pdu)) {
+ usbip_event_add(ud, VUDC_EVENT_ERROR_TCP);
+ if (ret >= 0)
+ return -EPIPE;
+ return ret;
+ }
+ usbip_header_correct_endian(&pdu, 0);
+
+ spin_lock_irq(&ud->lock);
+ ret = (ud->status == SDEV_ST_USED);
+ spin_unlock_irq(&ud->lock);
+ if (!ret) {
+ usbip_event_add(ud, VUDC_EVENT_ERROR_TCP);
+ return -EBUSY;
+ }
+
+ switch (pdu.base.command) {
+ case USBIP_CMD_UNLINK:
+ ret = v_recv_cmd_unlink(udc, &pdu);
+ break;
+ case USBIP_CMD_SUBMIT:
+ ret = v_recv_cmd_submit(udc, &pdu);
+ break;
+ default:
+ ret = -EPIPE;
+ pr_err("rx: unknown command");
+ break;
+ }
+ return ret;
+}
+
+int v_rx_loop(void *data)
+{
+ struct usbip_device *ud = data;
+ int ret = 0;
+
+ while (!kthread_should_stop()) {
+ if (usbip_event_happened(ud))
+ break;
+ ret = v_rx_pdu(ud);
+ if (ret < 0) {
+ pr_warn("v_rx exit with error %d", ret);
+ break;
+ }
+ }
+ return ret;
+}
diff --git a/drivers/usb/usbip/vudc_sysfs.c b/drivers/usb/usbip/vudc_sysfs.c
new file mode 100644
index 0000000..99397fa
--- /dev/null
+++ b/drivers/usb/usbip/vudc_sysfs.c
@@ -0,0 +1,229 @@
+/*
+ * Copyright (C) 2015 Karol Kosik <karo9@interia.eu>
+ * Copyright (C) 2015-2016 Samsung Electronics
+ * Igor Kotrasinski <i.kotrasinsk@samsung.com>
+ * Krzysztof Opasiak <k.opasiak@samsung.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/device.h>
+#include <linux/list.h>
+#include <linux/usb/gadget.h>
+#include <linux/usb/ch9.h>
+#include <linux/sysfs.h>
+#include <linux/kthread.h>
+#include <linux/byteorder/generic.h>
+
+#include "usbip_common.h"
+#include "vudc.h"
+
+#include <net/sock.h>
+
+/* called with udc->lock held */
+int get_gadget_descs(struct vudc *udc)
+{
+ struct vrequest *usb_req;
+ struct vep *ep0 = to_vep(udc->gadget.ep0);
+ struct usb_device_descriptor *ddesc = &udc->dev_desc;
+ struct usb_ctrlrequest req;
+ int ret;
+
+ if (!udc || !udc->driver || !udc->pullup)
+ return -EINVAL;
+
+ req.bRequestType = USB_DIR_IN | USB_TYPE_STANDARD | USB_RECIP_DEVICE;
+ req.bRequest = USB_REQ_GET_DESCRIPTOR;
+ req.wValue = cpu_to_le16(USB_DT_DEVICE << 8);
+ req.wIndex = cpu_to_le16(0);
+ req.wLength = cpu_to_le16(sizeof(*ddesc));
+
+ spin_unlock(&udc->lock);
+ ret = udc->driver->setup(&(udc->gadget), &req);
+ spin_lock(&udc->lock);
+ if (ret < 0)
+ goto out;
+
+ /* assuming request queue is empty; request is now on top */
+ usb_req = list_last_entry(&ep0->req_queue, struct vrequest, req_entry);
+ list_del(&usb_req->req_entry);
+
+ if (usb_req->req.length > sizeof(*ddesc)) {
+ ret = -EOVERFLOW;
+ goto giveback_req;
+ }
+
+ memcpy(ddesc, usb_req->req.buf, sizeof(*ddesc));
+ udc->desc_cached = 1;
+ ret = 0;
+giveback_req:
+ usb_req->req.status = 0;
+ usb_req->req.actual = usb_req->req.length;
+ usb_gadget_giveback_request(&(ep0->ep), &(usb_req->req));
+out:
+ return ret;
+}
+
+/*
+ * Exposes device descriptor from the gadget driver.
+ */
+static ssize_t dev_desc_read(struct file *file, struct kobject *kobj,
+ struct bin_attribute *attr, char *out,
+ loff_t off, size_t count)
+{
+ struct device *dev = kobj_to_dev(kobj);
+ struct vudc *udc = (struct vudc *)dev_get_drvdata(dev);
+ char *desc_ptr = (char *) &udc->dev_desc;
+ unsigned long flags;
+ int ret;
+
+ spin_lock_irqsave(&udc->lock, flags);
+ if (!udc->desc_cached) {
+ ret = -ENODEV;
+ goto unlock;
+ }
+
+ memcpy(out, desc_ptr + off, count);
+ ret = count;
+unlock:
+ spin_unlock_irqrestore(&udc->lock, flags);
+ return ret;
+}
+static BIN_ATTR_RO(dev_desc, sizeof(struct usb_device_descriptor));
+
+static ssize_t store_sockfd(struct device *dev, struct device_attribute *attr,
+ const char *in, size_t count)
+{
+ struct vudc *udc = (struct vudc *) dev_get_drvdata(dev);
+ int rv;
+ int sockfd = 0;
+ int err;
+ struct socket *socket;
+ unsigned long flags;
+ int ret;
+
+ rv = kstrtoint(in, 0, &sockfd);
+ if (rv != 0)
+ return -EINVAL;
+
+ spin_lock_irqsave(&udc->lock, flags);
+ /* Don't export what we don't have */
+ if (!udc || !udc->driver || !udc->pullup) {
+ dev_err(dev, "no device or gadget not bound");
+ ret = -ENODEV;
+ goto unlock;
+ }
+
+ if (sockfd != -1) {
+ if (udc->connected) {
+ dev_err(dev, "Device already connected");
+ ret = -EBUSY;
+ goto unlock;
+ }
+
+ spin_lock_irq(&udc->ud.lock);
+
+ if (udc->ud.status != SDEV_ST_AVAILABLE) {
+ ret = -EINVAL;
+ goto unlock_ud;
+ }
+
+ socket = sockfd_lookup(sockfd, &err);
+ if (!socket) {
+ dev_err(dev, "failed to lookup sock");
+ ret = -EINVAL;
+ goto unlock_ud;
+ }
+
+ udc->ud.tcp_socket = socket;
+
+ spin_unlock_irq(&udc->ud.lock);
+ spin_unlock_irqrestore(&udc->lock, flags);
+
+ udc->ud.tcp_rx = kthread_get_run(&v_rx_loop,
+ &udc->ud, "vudc_rx");
+ udc->ud.tcp_tx = kthread_get_run(&v_tx_loop,
+ &udc->ud, "vudc_tx");
+
+ spin_lock_irqsave(&udc->lock, flags);
+ spin_lock_irq(&udc->ud.lock);
+ udc->ud.status = SDEV_ST_USED;
+ spin_unlock_irq(&udc->ud.lock);
+
+ do_gettimeofday(&udc->start_time);
+ v_start_timer(udc);
+ udc->connected = 1;
+ } else {
+ if (!udc->connected) {
+ dev_err(dev, "Device not connected");
+ ret = -EINVAL;
+ goto unlock;
+ }
+
+ spin_lock_irq(&udc->ud.lock);
+ if (udc->ud.status != SDEV_ST_USED) {
+ ret = -EINVAL;
+ goto unlock_ud;
+ }
+ spin_unlock_irq(&udc->ud.lock);
+
+ usbip_event_add(&udc->ud, VUDC_EVENT_DOWN);
+ }
+
+ spin_unlock_irqrestore(&udc->lock, flags);
+
+ return count;
+
+unlock_ud:
+ spin_unlock_irq(&udc->ud.lock);
+unlock:
+ spin_unlock_irqrestore(&udc->lock, flags);
+
+ return ret;
+}
+static DEVICE_ATTR(usbip_sockfd, S_IWUSR, NULL, store_sockfd);
+
+static ssize_t usbip_status_show(struct device *dev,
+ struct device_attribute *attr, char *out)
+{
+ struct vudc *udc = (struct vudc *) dev_get_drvdata(dev);
+ int status;
+
+ if (!udc) {
+ dev_err(dev, "no device");
+ return -ENODEV;
+ }
+ spin_lock_irq(&udc->ud.lock);
+ status = udc->ud.status;
+ spin_unlock_irq(&udc->ud.lock);
+
+ return snprintf(out, PAGE_SIZE, "%d\n", status);
+}
+static DEVICE_ATTR_RO(usbip_status);
+
+static struct attribute *dev_attrs[] = {
+ &dev_attr_usbip_sockfd.attr,
+ &dev_attr_usbip_status.attr,
+ NULL,
+};
+
+static struct bin_attribute *dev_bin_attrs[] = {
+ &bin_attr_dev_desc,
+ NULL,
+};
+
+const struct attribute_group vudc_attr_group = {
+ .attrs = dev_attrs,
+ .bin_attrs = dev_bin_attrs,
+};
diff --git a/drivers/usb/usbip/vudc_transfer.c b/drivers/usb/usbip/vudc_transfer.c
new file mode 100644
index 0000000..aba6bd4
--- /dev/null
+++ b/drivers/usb/usbip/vudc_transfer.c
@@ -0,0 +1,506 @@
+/*
+ * Copyright (C) 2015 Karol Kosik <karo9@interia.eu>
+ * Copyright (C) 2015-2016 Samsung Electronics
+ * Igor Kotrasinski <i.kotrasinsk@samsung.com>
+ *
+ * Based on dummy_hcd.c, which is:
+ * Copyright (C) 2003 David Brownell
+ * Copyright (C) 2003-2005 Alan Stern
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/usb.h>
+#include <linux/timer.h>
+#include <linux/usb/ch9.h>
+
+#include "vudc.h"
+
+#define DEV_REQUEST (USB_TYPE_STANDARD | USB_RECIP_DEVICE)
+#define DEV_INREQUEST (DEV_REQUEST | USB_DIR_IN)
+#define INTF_REQUEST (USB_TYPE_STANDARD | USB_RECIP_INTERFACE)
+#define INTF_INREQUEST (INTF_REQUEST | USB_DIR_IN)
+#define EP_REQUEST (USB_TYPE_STANDARD | USB_RECIP_ENDPOINT)
+#define EP_INREQUEST (EP_REQUEST | USB_DIR_IN)
+
+static int get_frame_limit(enum usb_device_speed speed)
+{
+ switch (speed) {
+ case USB_SPEED_LOW:
+ return 8 /*bytes*/ * 12 /*packets*/;
+ case USB_SPEED_FULL:
+ return 64 /*bytes*/ * 19 /*packets*/;
+ case USB_SPEED_HIGH:
+ return 512 /*bytes*/ * 13 /*packets*/ * 8 /*uframes*/;
+ case USB_SPEED_SUPER:
+ /* Bus speed is 500000 bytes/ms, so use a little less */
+ return 490000;
+ default:
+ /* error */
+ return -1;
+ }
+
+}
+
+/*
+ * handle_control_request() - handles all control transfers
+ * @udc: pointer to vudc
+ * @urb: the urb request to handle
+ * @setup: pointer to the setup data for a USB device control
+ * request
+ * @status: pointer to request handling status
+ *
+ * Return 0 - if the request was handled
+ * 1 - if the request wasn't handles
+ * error code on error
+ *
+ * Adapted from drivers/usb/gadget/udc/dummy_hcd.c
+ */
+static int handle_control_request(struct vudc *udc, struct urb *urb,
+ struct usb_ctrlrequest *setup,
+ int *status)
+{
+ struct vep *ep2;
+ int ret_val = 1;
+ unsigned w_index;
+ unsigned w_value;
+
+ w_index = le16_to_cpu(setup->wIndex);
+ w_value = le16_to_cpu(setup->wValue);
+ switch (setup->bRequest) {
+ case USB_REQ_SET_ADDRESS:
+ if (setup->bRequestType != DEV_REQUEST)
+ break;
+ udc->address = w_value;
+ ret_val = 0;
+ *status = 0;
+ break;
+ case USB_REQ_SET_FEATURE:
+ if (setup->bRequestType == DEV_REQUEST) {
+ ret_val = 0;
+ switch (w_value) {
+ case USB_DEVICE_REMOTE_WAKEUP:
+ break;
+ case USB_DEVICE_B_HNP_ENABLE:
+ udc->gadget.b_hnp_enable = 1;
+ break;
+ case USB_DEVICE_A_HNP_SUPPORT:
+ udc->gadget.a_hnp_support = 1;
+ break;
+ case USB_DEVICE_A_ALT_HNP_SUPPORT:
+ udc->gadget.a_alt_hnp_support = 1;
+ break;
+ default:
+ ret_val = -EOPNOTSUPP;
+ }
+ if (ret_val == 0) {
+ udc->devstatus |= (1 << w_value);
+ *status = 0;
+ }
+ } else if (setup->bRequestType == EP_REQUEST) {
+ /* endpoint halt */
+ ep2 = vudc_find_endpoint(udc, w_index);
+ if (!ep2 || ep2->ep.name == udc->ep[0].ep.name) {
+ ret_val = -EOPNOTSUPP;
+ break;
+ }
+ ep2->halted = 1;
+ ret_val = 0;
+ *status = 0;
+ }
+ break;
+ case USB_REQ_CLEAR_FEATURE:
+ if (setup->bRequestType == DEV_REQUEST) {
+ ret_val = 0;
+ switch (w_value) {
+ case USB_DEVICE_REMOTE_WAKEUP:
+ w_value = USB_DEVICE_REMOTE_WAKEUP;
+ break;
+
+ case USB_DEVICE_U1_ENABLE:
+ case USB_DEVICE_U2_ENABLE:
+ case USB_DEVICE_LTM_ENABLE:
+ ret_val = -EOPNOTSUPP;
+ break;
+ default:
+ ret_val = -EOPNOTSUPP;
+ break;
+ }
+ if (ret_val == 0) {
+ udc->devstatus &= ~(1 << w_value);
+ *status = 0;
+ }
+ } else if (setup->bRequestType == EP_REQUEST) {
+ /* endpoint halt */
+ ep2 = vudc_find_endpoint(udc, w_index);
+ if (!ep2) {
+ ret_val = -EOPNOTSUPP;
+ break;
+ }
+ if (!ep2->wedged)
+ ep2->halted = 0;
+ ret_val = 0;
+ *status = 0;
+ }
+ break;
+ case USB_REQ_GET_STATUS:
+ if (setup->bRequestType == DEV_INREQUEST
+ || setup->bRequestType == INTF_INREQUEST
+ || setup->bRequestType == EP_INREQUEST) {
+ char *buf;
+ /*
+ * device: remote wakeup, selfpowered
+ * interface: nothing
+ * endpoint: halt
+ */
+ buf = (char *)urb->transfer_buffer;
+ if (urb->transfer_buffer_length > 0) {
+ if (setup->bRequestType == EP_INREQUEST) {
+ ep2 = vudc_find_endpoint(udc, w_index);
+ if (!ep2) {
+ ret_val = -EOPNOTSUPP;
+ break;
+ }
+ buf[0] = ep2->halted;
+ } else if (setup->bRequestType ==
+ DEV_INREQUEST) {
+ buf[0] = (u8)udc->devstatus;
+ } else
+ buf[0] = 0;
+ }
+ if (urb->transfer_buffer_length > 1)
+ buf[1] = 0;
+ urb->actual_length = min_t(u32, 2,
+ urb->transfer_buffer_length);
+ ret_val = 0;
+ *status = 0;
+ }
+ break;
+ }
+ return ret_val;
+}
+
+/* Adapted from dummy_hcd.c ; caller must hold lock */
+static int transfer(struct vudc *udc,
+ struct urb *urb, struct vep *ep, int limit)
+{
+ struct vrequest *req;
+ int sent = 0;
+top:
+ /* if there's no request queued, the device is NAKing; return */
+ list_for_each_entry(req, &ep->req_queue, req_entry) {
+ unsigned host_len, dev_len, len;
+ void *ubuf_pos, *rbuf_pos;
+ int is_short, to_host;
+ int rescan = 0;
+
+ /*
+ * 1..N packets of ep->ep.maxpacket each ... the last one
+ * may be short (including zero length).
+ *
+ * writer can send a zlp explicitly (length 0) or implicitly
+ * (length mod maxpacket zero, and 'zero' flag); they always
+ * terminate reads.
+ */
+ host_len = urb->transfer_buffer_length - urb->actual_length;
+ dev_len = req->req.length - req->req.actual;
+ len = min(host_len, dev_len);
+
+ to_host = usb_pipein(urb->pipe);
+ if (unlikely(len == 0))
+ is_short = 1;
+ else {
+ /* send multiple of maxpacket first, then remainder */
+ if (len >= ep->ep.maxpacket) {
+ is_short = 0;
+ if (len % ep->ep.maxpacket > 0)
+ rescan = 1;
+ len -= len % ep->ep.maxpacket;
+ } else {
+ is_short = 1;
+ }
+
+ ubuf_pos = urb->transfer_buffer + urb->actual_length;
+ rbuf_pos = req->req.buf + req->req.actual;
+
+ if (urb->pipe & USB_DIR_IN)
+ memcpy(ubuf_pos, rbuf_pos, len);
+ else
+ memcpy(rbuf_pos, ubuf_pos, len);
+
+ urb->actual_length += len;
+ req->req.actual += len;
+ sent += len;
+ }
+
+ /*
+ * short packets terminate, maybe with overflow/underflow.
+ * it's only really an error to write too much.
+ *
+ * partially filling a buffer optionally blocks queue advances
+ * (so completion handlers can clean up the queue) but we don't
+ * need to emulate such data-in-flight.
+ */
+ if (is_short) {
+ if (host_len == dev_len) {
+ req->req.status = 0;
+ urb->status = 0;
+ } else if (to_host) {
+ req->req.status = 0;
+ if (dev_len > host_len)
+ urb->status = -EOVERFLOW;
+ else
+ urb->status = 0;
+ } else {
+ urb->status = 0;
+ if (host_len > dev_len)
+ req->req.status = -EOVERFLOW;
+ else
+ req->req.status = 0;
+ }
+
+ /* many requests terminate without a short packet */
+ /* also check if we need to send zlp */
+ } else {
+ if (req->req.length == req->req.actual) {
+ if (req->req.zero && to_host)
+ rescan = 1;
+ else
+ req->req.status = 0;
+ }
+ if (urb->transfer_buffer_length == urb->actual_length) {
+ if (urb->transfer_flags & URB_ZERO_PACKET &&
+ !to_host)
+ rescan = 1;
+ else
+ urb->status = 0;
+ }
+ }
+
+ /* device side completion --> continuable */
+ if (req->req.status != -EINPROGRESS) {
+
+ list_del_init(&req->req_entry);
+ spin_unlock(&udc->lock);
+ usb_gadget_giveback_request(&ep->ep, &req->req);
+ spin_lock(&udc->lock);
+
+ /* requests might have been unlinked... */
+ rescan = 1;
+ }
+
+ /* host side completion --> terminate */
+ if (urb->status != -EINPROGRESS)
+ break;
+
+ /* rescan to continue with any other queued i/o */
+ if (rescan)
+ goto top;
+ }
+ return sent;
+}
+
+static void v_timer(unsigned long _vudc)
+{
+ struct vudc *udc = (struct vudc *) _vudc;
+ struct transfer_timer *timer = &udc->tr_timer;
+ struct urbp *urb_p, *tmp;
+ unsigned long flags;
+ struct usb_ep *_ep;
+ struct vep *ep;
+ int ret = 0;
+ int total, limit;
+
+ spin_lock_irqsave(&udc->lock, flags);
+
+ total = get_frame_limit(udc->gadget.speed);
+ if (total < 0) { /* unknown speed, or not set yet */
+ timer->state = VUDC_TR_IDLE;
+ spin_unlock_irqrestore(&udc->lock, flags);
+ return;
+ }
+ /* is it next frame now? */
+ if (time_after(jiffies, timer->frame_start + msecs_to_jiffies(1))) {
+ timer->frame_limit = total;
+ /* FIXME: how to make it accurate? */
+ timer->frame_start = jiffies;
+ } else {
+ total = timer->frame_limit;
+ }
+
+ list_for_each_entry(_ep, &udc->gadget.ep_list, ep_list) {
+ ep = to_vep(_ep);
+ ep->already_seen = 0;
+ }
+
+restart:
+ list_for_each_entry_safe(urb_p, tmp, &udc->urb_queue, urb_entry) {
+ struct urb *urb = urb_p->urb;
+
+ ep = urb_p->ep;
+ if (urb->unlinked)
+ goto return_urb;
+ if (timer->state != VUDC_TR_RUNNING)
+ continue;
+
+ if (!ep) {
+ urb->status = -EPROTO;
+ goto return_urb;
+ }
+
+ /* Used up bandwidth? */
+ if (total <= 0 && ep->type == USB_ENDPOINT_XFER_BULK)
+ continue;
+
+ if (ep->already_seen)
+ continue;
+ ep->already_seen = 1;
+ if (ep == &udc->ep[0] && urb_p->new) {
+ ep->setup_stage = 1;
+ urb_p->new = 0;
+ }
+ if (ep->halted && !ep->setup_stage) {
+ urb->status = -EPIPE;
+ goto return_urb;
+ }
+
+ if (ep == &udc->ep[0] && ep->setup_stage) {
+ /* TODO - flush any stale requests */
+ ep->setup_stage = 0;
+ ep->halted = 0;
+
+ ret = handle_control_request(udc, urb,
+ (struct usb_ctrlrequest *) urb->setup_packet,
+ (&urb->status));
+ if (ret > 0) {
+ spin_unlock(&udc->lock);
+ ret = udc->driver->setup(&udc->gadget,
+ (struct usb_ctrlrequest *)
+ urb->setup_packet);
+ spin_lock(&udc->lock);
+ }
+ if (ret >= 0) {
+ /* no delays (max 64kb data stage) */
+ limit = 64 * 1024;
+ goto treat_control_like_bulk;
+ } else {
+ urb->status = -EPIPE;
+ urb->actual_length = 0;
+ goto return_urb;
+ }
+ }
+
+ limit = total;
+ switch (ep->type) {
+ case USB_ENDPOINT_XFER_ISOC:
+ /* TODO: support */
+ urb->status = -EXDEV;
+ break;
+
+ case USB_ENDPOINT_XFER_INT:
+ /*
+ * TODO: figure out bandwidth guarantees
+ * for now, give unlimited bandwidth
+ */
+ limit += urb->transfer_buffer_length;
+ /* fallthrough */
+ default:
+treat_control_like_bulk:
+ total -= transfer(udc, urb, ep, limit);
+ }
+ if (urb->status == -EINPROGRESS)
+ continue;
+
+return_urb:
+ if (ep)
+ ep->already_seen = ep->setup_stage = 0;
+
+ spin_lock(&udc->lock_tx);
+ list_del(&urb_p->urb_entry);
+ if (!urb->unlinked) {
+ v_enqueue_ret_submit(udc, urb_p);
+ } else {
+ v_enqueue_ret_unlink(udc, urb_p->seqnum,
+ urb->unlinked);
+ free_urbp_and_urb(urb_p);
+ }
+ wake_up(&udc->tx_waitq);
+ spin_unlock(&udc->lock_tx);
+
+ goto restart;
+ }
+
+ /* TODO - also wait on empty usb_request queues? */
+ if (list_empty(&udc->urb_queue))
+ timer->state = VUDC_TR_IDLE;
+ else
+ mod_timer(&timer->timer,
+ timer->frame_start + msecs_to_jiffies(1));
+
+ spin_unlock_irqrestore(&udc->lock, flags);
+}
+
+/* All timer functions are run with udc->lock held */
+
+void v_init_timer(struct vudc *udc)
+{
+ struct transfer_timer *t = &udc->tr_timer;
+
+ setup_timer(&t->timer, v_timer, (unsigned long) udc);
+ t->state = VUDC_TR_STOPPED;
+}
+
+void v_start_timer(struct vudc *udc)
+{
+ struct transfer_timer *t = &udc->tr_timer;
+
+ dev_dbg(&udc->pdev->dev, "timer start");
+ switch (t->state) {
+ case VUDC_TR_RUNNING:
+ return;
+ case VUDC_TR_IDLE:
+ return v_kick_timer(udc, jiffies);
+ case VUDC_TR_STOPPED:
+ t->state = VUDC_TR_IDLE;
+ t->frame_start = jiffies;
+ t->frame_limit = get_frame_limit(udc->gadget.speed);
+ return v_kick_timer(udc, jiffies);
+ }
+}
+
+void v_kick_timer(struct vudc *udc, unsigned long time)
+{
+ struct transfer_timer *t = &udc->tr_timer;
+
+ dev_dbg(&udc->pdev->dev, "timer kick");
+ switch (t->state) {
+ case VUDC_TR_RUNNING:
+ return;
+ case VUDC_TR_IDLE:
+ t->state = VUDC_TR_RUNNING;
+ /* fallthrough */
+ case VUDC_TR_STOPPED:
+ /* we may want to kick timer to unqueue urbs */
+ mod_timer(&t->timer, time);
+ }
+}
+
+void v_stop_timer(struct vudc *udc)
+{
+ struct transfer_timer *t = &udc->tr_timer;
+
+ /* timer itself will take care of stopping */
+ dev_dbg(&udc->pdev->dev, "timer stop");
+ t->state = VUDC_TR_STOPPED;
+}
diff --git a/drivers/usb/usbip/vudc_tx.c b/drivers/usb/usbip/vudc_tx.c
new file mode 100644
index 0000000..2346617
--- /dev/null
+++ b/drivers/usb/usbip/vudc_tx.c
@@ -0,0 +1,289 @@
+/*
+ * Copyright (C) 2015 Karol Kosik <karo9@interia.eu>
+ * Copyright (C) 2015-2016 Samsung Electronics
+ * Igor Kotrasinski <i.kotrasinsk@samsung.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <net/sock.h>
+#include <linux/list.h>
+#include <linux/kthread.h>
+
+#include "usbip_common.h"
+#include "vudc.h"
+
+static inline void setup_base_pdu(struct usbip_header_basic *base,
+ __u32 command, __u32 seqnum)
+{
+ base->command = command;
+ base->seqnum = seqnum;
+ base->devid = 0;
+ base->ep = 0;
+ base->direction = 0;
+}
+
+static void setup_ret_submit_pdu(struct usbip_header *rpdu, struct urbp *urb_p)
+{
+ setup_base_pdu(&rpdu->base, USBIP_RET_SUBMIT, urb_p->seqnum);
+ usbip_pack_pdu(rpdu, urb_p->urb, USBIP_RET_SUBMIT, 1);
+}
+
+static void setup_ret_unlink_pdu(struct usbip_header *rpdu,
+ struct v_unlink *unlink)
+{
+ setup_base_pdu(&rpdu->base, USBIP_RET_UNLINK, unlink->seqnum);
+ rpdu->u.ret_unlink.status = unlink->status;
+}
+
+static int v_send_ret_unlink(struct vudc *udc, struct v_unlink *unlink)
+{
+ struct msghdr msg;
+ struct kvec iov[1];
+ size_t txsize;
+
+ int ret;
+ struct usbip_header pdu_header;
+
+ txsize = 0;
+ memset(&pdu_header, 0, sizeof(pdu_header));
+ memset(&msg, 0, sizeof(msg));
+ memset(&iov, 0, sizeof(iov));
+
+ /* 1. setup usbip_header */
+ setup_ret_unlink_pdu(&pdu_header, unlink);
+ usbip_header_correct_endian(&pdu_header, 1);
+
+ iov[0].iov_base = &pdu_header;
+ iov[0].iov_len = sizeof(pdu_header);
+ txsize += sizeof(pdu_header);
+
+ ret = kernel_sendmsg(udc->ud.tcp_socket, &msg, iov,
+ 1, txsize);
+ if (ret != txsize) {
+ usbip_event_add(&udc->ud, VUDC_EVENT_ERROR_TCP);
+ if (ret >= 0)
+ return -EPIPE;
+ return ret;
+ }
+ kfree(unlink);
+
+ return txsize;
+}
+
+static int v_send_ret_submit(struct vudc *udc, struct urbp *urb_p)
+{
+ struct urb *urb = urb_p->urb;
+ struct usbip_header pdu_header;
+ struct usbip_iso_packet_descriptor *iso_buffer = NULL;
+ struct kvec *iov = NULL;
+ int iovnum = 0;
+ int ret = 0;
+ size_t txsize;
+ struct msghdr msg;
+
+ txsize = 0;
+ memset(&pdu_header, 0, sizeof(pdu_header));
+ memset(&msg, 0, sizeof(msg));
+
+ if (urb_p->type == USB_ENDPOINT_XFER_ISOC)
+ iovnum = 2 + urb->number_of_packets;
+ else
+ iovnum = 2;
+
+ iov = kcalloc(iovnum, sizeof(*iov), GFP_KERNEL);
+ if (!iov) {
+ usbip_event_add(&udc->ud, VUDC_EVENT_ERROR_MALLOC);
+ ret = -ENOMEM;
+ goto out;
+ }
+ iovnum = 0;
+
+ /* 1. setup usbip_header */
+ setup_ret_submit_pdu(&pdu_header, urb_p);
+ usbip_dbg_stub_tx("setup txdata seqnum: %d urb: %p\n",
+ pdu_header.base.seqnum, urb);
+ usbip_header_correct_endian(&pdu_header, 1);
+
+ iov[iovnum].iov_base = &pdu_header;
+ iov[iovnum].iov_len = sizeof(pdu_header);
+ iovnum++;
+ txsize += sizeof(pdu_header);
+
+ /* 2. setup transfer buffer */
+ if (urb_p->type != USB_ENDPOINT_XFER_ISOC &&
+ usb_pipein(urb->pipe) && urb->actual_length > 0) {
+ iov[iovnum].iov_base = urb->transfer_buffer;
+ iov[iovnum].iov_len = urb->actual_length;
+ iovnum++;
+ txsize += urb->actual_length;
+ } else if (urb_p->type == USB_ENDPOINT_XFER_ISOC &&
+ usb_pipein(urb->pipe)) {
+ /* FIXME - copypasted from stub_tx, refactor */
+ int i;
+
+ for (i = 0; i < urb->number_of_packets; i++) {
+ iov[iovnum].iov_base = urb->transfer_buffer +
+ urb->iso_frame_desc[i].offset;
+ iov[iovnum].iov_len =
+ urb->iso_frame_desc[i].actual_length;
+ iovnum++;
+ txsize += urb->iso_frame_desc[i].actual_length;
+ }
+
+ if (txsize != sizeof(pdu_header) + urb->actual_length) {
+ usbip_event_add(&udc->ud, VUDC_EVENT_ERROR_TCP);
+ ret = -EPIPE;
+ goto out;
+ }
+ }
+ /* else - no buffer to send */
+
+ /* 3. setup iso_packet_descriptor */
+ if (urb_p->type == USB_ENDPOINT_XFER_ISOC) {
+ ssize_t len = 0;
+
+ iso_buffer = usbip_alloc_iso_desc_pdu(urb, &len);
+ if (!iso_buffer) {
+ usbip_event_add(&udc->ud,
+ VUDC_EVENT_ERROR_MALLOC);
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ iov[iovnum].iov_base = iso_buffer;
+ iov[iovnum].iov_len = len;
+ txsize += len;
+ iovnum++;
+ }
+
+ ret = kernel_sendmsg(udc->ud.tcp_socket, &msg,
+ iov, iovnum, txsize);
+ if (ret != txsize) {
+ usbip_event_add(&udc->ud, VUDC_EVENT_ERROR_TCP);
+ if (ret >= 0)
+ ret = -EPIPE;
+ goto out;
+ }
+
+out:
+ kfree(iov);
+ kfree(iso_buffer);
+ free_urbp_and_urb(urb_p);
+ if (ret < 0)
+ return ret;
+ return txsize;
+}
+
+static int v_send_ret(struct vudc *udc)
+{
+ unsigned long flags;
+ struct tx_item *txi;
+ size_t total_size = 0;
+ int ret = 0;
+
+ spin_lock_irqsave(&udc->lock_tx, flags);
+ while (!list_empty(&udc->tx_queue)) {
+ txi = list_first_entry(&udc->tx_queue, struct tx_item,
+ tx_entry);
+ list_del(&txi->tx_entry);
+ spin_unlock_irqrestore(&udc->lock_tx, flags);
+
+ switch (txi->type) {
+ case TX_SUBMIT:
+ ret = v_send_ret_submit(udc, txi->s);
+ break;
+ case TX_UNLINK:
+ ret = v_send_ret_unlink(udc, txi->u);
+ break;
+ }
+ kfree(txi);
+
+ if (ret < 0)
+ return ret;
+
+ total_size += ret;
+
+ spin_lock_irqsave(&udc->lock_tx, flags);
+ }
+
+ spin_unlock_irqrestore(&udc->lock_tx, flags);
+ return total_size;
+}
+
+
+int v_tx_loop(void *data)
+{
+ struct usbip_device *ud = (struct usbip_device *) data;
+ struct vudc *udc = container_of(ud, struct vudc, ud);
+ int ret;
+
+ while (!kthread_should_stop()) {
+ if (usbip_event_happened(&udc->ud))
+ break;
+ ret = v_send_ret(udc);
+ if (ret < 0) {
+ pr_warn("v_tx exit with error %d", ret);
+ break;
+ }
+ wait_event_interruptible(udc->tx_waitq,
+ (!list_empty(&udc->tx_queue) ||
+ kthread_should_stop()));
+ }
+
+ return 0;
+}
+
+/* called with spinlocks held */
+void v_enqueue_ret_unlink(struct vudc *udc, __u32 seqnum, __u32 status)
+{
+ struct tx_item *txi;
+ struct v_unlink *unlink;
+
+ txi = kzalloc(sizeof(*txi), GFP_ATOMIC);
+ if (!txi) {
+ usbip_event_add(&udc->ud, VDEV_EVENT_ERROR_MALLOC);
+ return;
+ }
+ unlink = kzalloc(sizeof(*unlink), GFP_ATOMIC);
+ if (!unlink) {
+ kfree(txi);
+ usbip_event_add(&udc->ud, VDEV_EVENT_ERROR_MALLOC);
+ return;
+ }
+
+ unlink->seqnum = seqnum;
+ unlink->status = status;
+ txi->type = TX_UNLINK;
+ txi->u = unlink;
+
+ list_add_tail(&txi->tx_entry, &udc->tx_queue);
+}
+
+/* called with spinlocks held */
+void v_enqueue_ret_submit(struct vudc *udc, struct urbp *urb_p)
+{
+ struct tx_item *txi;
+
+ txi = kzalloc(sizeof(*txi), GFP_ATOMIC);
+ if (!txi) {
+ usbip_event_add(&udc->ud, VDEV_EVENT_ERROR_MALLOC);
+ return;
+ }
+
+ txi->type = TX_SUBMIT;
+ txi->s = urb_p;
+
+ list_add_tail(&txi->tx_entry, &udc->tx_queue);
+}
diff --git a/drivers/usb/wusbcore/crypto.c b/drivers/usb/wusbcore/crypto.c
index 8ed8e34..33acd15 100644
--- a/drivers/usb/wusbcore/crypto.c
+++ b/drivers/usb/wusbcore/crypto.c
@@ -54,7 +54,7 @@
#include <linux/usb/wusb.h>
#include <linux/scatterlist.h>
-static int debug_crypto_verify = 0;
+static int debug_crypto_verify;
module_param(debug_crypto_verify, int, 0);
MODULE_PARM_DESC(debug_crypto_verify, "verify the key generation algorithms");
@@ -390,7 +390,7 @@
0x26, 0x27, 0x28, 0x29, 0x2a, 0x2b,
0x2c, 0x2d, 0x2e, 0x2f },
.MIC = { 0x75, 0x6a, 0x97, 0x51, 0x0c, 0x8c,
- 0x14, 0x7b } ,
+ 0x14, 0x7b },
};
size_t hs_size;
@@ -480,7 +480,7 @@
printk(KERN_ERR "E: keydvt in: key\n");
wusb_key_dump(stv_key_a1, sizeof(stv_key_a1));
printk(KERN_ERR "E: keydvt in: nonce\n");
- wusb_key_dump( &stv_keydvt_n_a1, sizeof(stv_keydvt_n_a1));
+ wusb_key_dump(&stv_keydvt_n_a1, sizeof(stv_keydvt_n_a1));
printk(KERN_ERR "E: keydvt in: hnonce & dnonce\n");
wusb_key_dump(&stv_keydvt_in_a1, sizeof(stv_keydvt_in_a1));
printk(KERN_ERR "E: keydvt out: KCK\n");
diff --git a/drivers/usb/wusbcore/devconnect.c b/drivers/usb/wusbcore/devconnect.c
index 3f4f5fb..bf95517 100644
--- a/drivers/usb/wusbcore/devconnect.c
+++ b/drivers/usb/wusbcore/devconnect.c
@@ -893,7 +893,6 @@
error_nodev:
return;
- wusb_dev_sysfs_rm(wusb_dev);
error_add_sysfs:
wusb_dev_bos_rm(wusb_dev);
error_bos_add:
diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c
index 0582b72..3054e3f 100644
--- a/drivers/vfio/vfio_iommu_spapr_tce.c
+++ b/drivers/vfio/vfio_iommu_spapr_tce.c
@@ -1188,7 +1188,8 @@
goto unlock_exit;
}
table_group_tmp = iommu_group_get_iommudata(tcegrp->grp);
- if (table_group_tmp->ops != table_group->ops) {
+ if (table_group_tmp->ops->create_table !=
+ table_group->ops->create_table) {
pr_warn("tce_vfio: Group %d is incompatible with group %d\n",
iommu_group_id(iommu_group),
iommu_group_id(tcegrp->grp));
diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
index 75b24e9..15a6582 100644
--- a/drivers/vfio/vfio_iommu_type1.c
+++ b/drivers/vfio/vfio_iommu_type1.c
@@ -407,7 +407,7 @@
mutex_lock(&iommu->lock);
list_for_each_entry(domain, &iommu->domain_list, next)
- bitmap &= domain->domain->ops->pgsize_bitmap;
+ bitmap &= domain->domain->pgsize_bitmap;
mutex_unlock(&iommu->lock);
/*
diff --git a/drivers/video/Kconfig b/drivers/video/Kconfig
index e0606c0..3c20af9 100644
--- a/drivers/video/Kconfig
+++ b/drivers/video/Kconfig
@@ -8,10 +8,6 @@
config HAVE_FB_ATMEL
bool
-config SH_MIPI_DSI
- tristate
- depends on (SUPERH || ARCH_SHMOBILE) && HAVE_CLK
-
config SH_LCD_MIPI_DSI
bool
diff --git a/drivers/video/console/fbcon.c b/drivers/video/console/fbcon.c
index 6e92917..afd3301 100644
--- a/drivers/video/console/fbcon.c
+++ b/drivers/video/console/fbcon.c
@@ -170,7 +170,7 @@
int height, int width);
static int fbcon_switch(struct vc_data *vc);
static int fbcon_blank(struct vc_data *vc, int blank, int mode_switch);
-static int fbcon_set_palette(struct vc_data *vc, unsigned char *table);
+static int fbcon_set_palette(struct vc_data *vc, const unsigned char *table);
static int fbcon_scrolldelta(struct vc_data *vc, int lines);
/*
@@ -2652,7 +2652,7 @@
0, 16, palette_red, palette_green, palette_blue, NULL
};
-static int fbcon_set_palette(struct vc_data *vc, unsigned char *table)
+static int fbcon_set_palette(struct vc_data *vc, const unsigned char *table)
{
struct fb_info *info = registered_fb[con2fb_map[vc->vc_num]];
int i, j, k, depth;
diff --git a/drivers/video/console/mdacon.c b/drivers/video/console/mdacon.c
index 296e945..8edc062 100644
--- a/drivers/video/console/mdacon.c
+++ b/drivers/video/console/mdacon.c
@@ -481,7 +481,7 @@
return 1; /* redrawing needed */
}
-static int mdacon_set_palette(struct vc_data *c, unsigned char *table)
+static int mdacon_set_palette(struct vc_data *c, const unsigned char *table)
{
return -EINVAL;
}
diff --git a/drivers/video/console/newport_con.c b/drivers/video/console/newport_con.c
index bb4e962..0553dfe 100644
--- a/drivers/video/console/newport_con.c
+++ b/drivers/video/console/newport_con.c
@@ -574,7 +574,7 @@
return newport_set_font(vc->vc_num, font);
}
-static int newport_set_palette(struct vc_data *vc, unsigned char *table)
+static int newport_set_palette(struct vc_data *vc, const unsigned char *table)
{
return -EINVAL;
}
diff --git a/drivers/video/console/sticon.c b/drivers/video/console/sticon.c
index 026fd12..e440c2d 100644
--- a/drivers/video/console/sticon.c
+++ b/drivers/video/console/sticon.c
@@ -79,7 +79,7 @@
return "STI console";
}
-static int sticon_set_palette(struct vc_data *c, unsigned char *table)
+static int sticon_set_palette(struct vc_data *c, const unsigned char *table)
{
return -EINVAL;
}
diff --git a/drivers/video/console/vgacon.c b/drivers/video/console/vgacon.c
index 517f565..8bf9110 100644
--- a/drivers/video/console/vgacon.c
+++ b/drivers/video/console/vgacon.c
@@ -80,7 +80,6 @@
static void vgacon_cursor(struct vc_data *c, int mode);
static int vgacon_switch(struct vc_data *c);
static int vgacon_blank(struct vc_data *c, int blank, int mode_switch);
-static int vgacon_set_palette(struct vc_data *vc, unsigned char *table);
static int vgacon_scrolldelta(struct vc_data *c, int lines);
static int vgacon_set_origin(struct vc_data *c);
static void vgacon_save_screen(struct vc_data *c);
@@ -847,7 +846,7 @@
return 0; /* Redrawing not needed */
}
-static void vga_set_palette(struct vc_data *vc, unsigned char *table)
+static void vga_set_palette(struct vc_data *vc, const unsigned char *table)
{
int i, j;
@@ -860,7 +859,7 @@
}
}
-static int vgacon_set_palette(struct vc_data *vc, unsigned char *table)
+static int vgacon_set_palette(struct vc_data *vc, const unsigned char *table)
{
#ifdef CAN_LOAD_PALETTE
if (vga_video_type != VIDEO_TYPE_VGAC || vga_palette_blanked
diff --git a/drivers/video/fbdev/Kconfig b/drivers/video/fbdev/Kconfig
index e5a391a..88b008f 100644
--- a/drivers/video/fbdev/Kconfig
+++ b/drivers/video/fbdev/Kconfig
@@ -1993,7 +1993,6 @@
select FB_SYS_FOPS
select FB_DEFERRED_IO
select FB_BACKLIGHT
- select SH_MIPI_DSI if SH_LCD_MIPI_DSI
---help---
Frame buffer driver for the on-chip SH-Mobile LCD controller.
diff --git a/drivers/video/fbdev/Makefile b/drivers/video/fbdev/Makefile
index 65fb150..f673186 100644
--- a/drivers/video/fbdev/Makefile
+++ b/drivers/video/fbdev/Makefile
@@ -117,7 +117,6 @@
obj-$(CONFIG_FB_UDL) += udlfb.o
obj-$(CONFIG_FB_SMSCUFX) += smscufx.o
obj-$(CONFIG_FB_XILINX) += xilinxfb.o
-obj-$(CONFIG_SH_MIPI_DSI) += sh_mipi_dsi.o
obj-$(CONFIG_FB_SH_MOBILE_MERAM) += sh_mobile_meram.o
obj-$(CONFIG_FB_SH_MOBILE_LCDC) += sh_mobile_lcdcfb.o
obj-$(CONFIG_FB_OMAP) += omap/
diff --git a/drivers/video/fbdev/amba-clcd.c b/drivers/video/fbdev/amba-clcd.c
index 93e66a9..9b15886 100644
--- a/drivers/video/fbdev/amba-clcd.c
+++ b/drivers/video/fbdev/amba-clcd.c
@@ -34,8 +34,6 @@
#include <video/of_display_timing.h>
#include <video/videomode.h>
-#include <asm/sizes.h>
-
#define to_clcd(info) container_of(info, struct clcd_fb, fb)
/* This is limited to 16 characters when displayed by X startup */
diff --git a/drivers/video/fbdev/core/fb_defio.c b/drivers/video/fbdev/core/fb_defio.c
index 57721c7..74b5bca 100644
--- a/drivers/video/fbdev/core/fb_defio.c
+++ b/drivers/video/fbdev/core/fb_defio.c
@@ -164,7 +164,7 @@
.set_page_dirty = fb_deferred_io_set_page_dirty,
};
-static int fb_deferred_io_mmap(struct fb_info *info, struct vm_area_struct *vma)
+int fb_deferred_io_mmap(struct fb_info *info, struct vm_area_struct *vma)
{
vma->vm_ops = &fb_deferred_io_vm_ops;
vma->vm_flags |= VM_DONTEXPAND | VM_DONTDUMP;
@@ -173,6 +173,7 @@
vma->vm_private_data = info;
return 0;
}
+EXPORT_SYMBOL(fb_deferred_io_mmap);
/* workqueue callback */
static void fb_deferred_io_work(struct work_struct *work)
diff --git a/drivers/video/fbdev/core/fbmem.c b/drivers/video/fbdev/core/fbmem.c
index 4e73b6f..76c1ad9 100644
--- a/drivers/video/fbdev/core/fbmem.c
+++ b/drivers/video/fbdev/core/fbmem.c
@@ -1854,17 +1854,31 @@
static int __init
fbmem_init(void)
{
- proc_create("fb", 0, NULL, &fb_proc_fops);
+ int ret;
- if (register_chrdev(FB_MAJOR,"fb",&fb_fops))
+ if (!proc_create("fb", 0, NULL, &fb_proc_fops))
+ return -ENOMEM;
+
+ ret = register_chrdev(FB_MAJOR, "fb", &fb_fops);
+ if (ret) {
printk("unable to get major %d for fb devs\n", FB_MAJOR);
+ goto err_chrdev;
+ }
fb_class = class_create(THIS_MODULE, "graphics");
if (IS_ERR(fb_class)) {
- printk(KERN_WARNING "Unable to create fb class; errno = %ld\n", PTR_ERR(fb_class));
+ ret = PTR_ERR(fb_class);
+ pr_warn("Unable to create fb class; errno = %d\n", ret);
fb_class = NULL;
+ goto err_class;
}
return 0;
+
+err_class:
+ unregister_chrdev(FB_MAJOR, "fb");
+err_chrdev:
+ remove_proc_entry("fb", NULL);
+ return ret;
}
#ifdef MODULE
diff --git a/drivers/video/fbdev/efifb.c b/drivers/video/fbdev/efifb.c
index f4c045c..924bad4 100644
--- a/drivers/video/fbdev/efifb.c
+++ b/drivers/video/fbdev/efifb.c
@@ -237,10 +237,8 @@
goto err_release_fb;
}
- printk(KERN_INFO "efifb: framebuffer at 0x%lx, mapped to 0x%p, "
- "using %dk, total %dk\n",
- efifb_fix.smem_start, info->screen_base,
- size_remap/1024, size_total/1024);
+ printk(KERN_INFO "efifb: framebuffer at 0x%lx, using %dk, total %dk\n",
+ efifb_fix.smem_start, size_remap/1024, size_total/1024);
printk(KERN_INFO "efifb: mode is %dx%dx%d, linelength=%d, pages=%d\n",
efifb_defined.xres, efifb_defined.yres,
efifb_defined.bits_per_pixel, efifb_fix.line_length,
diff --git a/drivers/video/fbdev/hyperv_fb.c b/drivers/video/fbdev/hyperv_fb.c
index e2451bd..2fd49b2 100644
--- a/drivers/video/fbdev/hyperv_fb.c
+++ b/drivers/video/fbdev/hyperv_fb.c
@@ -743,7 +743,7 @@
err3:
iounmap(fb_virt);
err2:
- release_mem_region(par->mem->start, screen_fb_size);
+ vmbus_free_mmio(par->mem->start, screen_fb_size);
par->mem = NULL;
err1:
if (!gen2vm)
@@ -758,7 +758,7 @@
struct hvfb_par *par = info->par;
iounmap(info->screen_base);
- release_mem_region(par->mem->start, screen_fb_size);
+ vmbus_free_mmio(par->mem->start, screen_fb_size);
par->mem = NULL;
}
diff --git a/drivers/video/fbdev/imxfb.c b/drivers/video/fbdev/imxfb.c
index 76b6a77..fe0c4ee 100644
--- a/drivers/video/fbdev/imxfb.c
+++ b/drivers/video/fbdev/imxfb.c
@@ -473,11 +473,12 @@
return 0;
}
-static void imxfb_enable_controller(struct imxfb_info *fbi)
+static int imxfb_enable_controller(struct imxfb_info *fbi)
{
+ int ret;
if (fbi->enabled)
- return;
+ return 0;
pr_debug("Enabling LCD controller\n");
@@ -496,10 +497,29 @@
*/
writel(RMCR_LCDC_EN_MX1, fbi->regs + LCDC_RMCR);
- clk_prepare_enable(fbi->clk_ipg);
- clk_prepare_enable(fbi->clk_ahb);
- clk_prepare_enable(fbi->clk_per);
+ ret = clk_prepare_enable(fbi->clk_ipg);
+ if (ret)
+ goto err_enable_ipg;
+
+ ret = clk_prepare_enable(fbi->clk_ahb);
+ if (ret)
+ goto err_enable_ahb;
+
+ ret = clk_prepare_enable(fbi->clk_per);
+ if (ret)
+ goto err_enable_per;
+
fbi->enabled = true;
+ return 0;
+
+err_enable_per:
+ clk_disable_unprepare(fbi->clk_ahb);
+err_enable_ahb:
+ clk_disable_unprepare(fbi->clk_ipg);
+err_enable_ipg:
+ writel(0, fbi->regs + LCDC_RMCR);
+
+ return ret;
}
static void imxfb_disable_controller(struct imxfb_info *fbi)
@@ -510,8 +530,8 @@
pr_debug("Disabling LCD controller\n");
clk_disable_unprepare(fbi->clk_per);
- clk_disable_unprepare(fbi->clk_ipg);
clk_disable_unprepare(fbi->clk_ahb);
+ clk_disable_unprepare(fbi->clk_ipg);
fbi->enabled = false;
writel(0, fbi->regs + LCDC_RMCR);
@@ -532,8 +552,7 @@
break;
case FB_BLANK_UNBLANK:
- imxfb_enable_controller(fbi);
- break;
+ return imxfb_enable_controller(fbi);
}
return 0;
}
@@ -758,10 +777,11 @@
{
struct imxfb_info *fbi = dev_get_drvdata(&lcddev->dev);
- if (!IS_ERR(fbi->lcd_pwr))
- return regulator_is_enabled(fbi->lcd_pwr);
+ if (!IS_ERR(fbi->lcd_pwr) &&
+ !regulator_is_enabled(fbi->lcd_pwr))
+ return FB_BLANK_POWERDOWN;
- return 1;
+ return FB_BLANK_UNBLANK;
}
static int imxfb_lcd_set_power(struct lcd_device *lcddev, int power)
@@ -769,7 +789,7 @@
struct imxfb_info *fbi = dev_get_drvdata(&lcddev->dev);
if (!IS_ERR(fbi->lcd_pwr)) {
- if (power)
+ if (power == FB_BLANK_UNBLANK)
return regulator_enable(fbi->lcd_pwr);
else
return regulator_disable(fbi->lcd_pwr);
diff --git a/drivers/video/fbdev/omap2/omapfb/dss/dsi.c b/drivers/video/fbdev/omap2/omapfb/dss/dsi.c
index 0eec073..d63e598 100644
--- a/drivers/video/fbdev/omap2/omapfb/dss/dsi.c
+++ b/drivers/video/fbdev/omap2/omapfb/dss/dsi.c
@@ -1180,13 +1180,11 @@
return PTR_ERR(vdds_dsi);
}
- if (regulator_can_change_voltage(vdds_dsi)) {
- r = regulator_set_voltage(vdds_dsi, 1800000, 1800000);
- if (r) {
- devm_regulator_put(vdds_dsi);
- DSSERR("can't set the DSI regulator voltage\n");
- return r;
- }
+ r = regulator_set_voltage(vdds_dsi, 1800000, 1800000);
+ if (r) {
+ devm_regulator_put(vdds_dsi);
+ DSSERR("can't set the DSI regulator voltage\n");
+ return r;
}
dsi->vdds_dsi_reg = vdds_dsi;
diff --git a/drivers/video/fbdev/omap2/omapfb/dss/hdmi4.c b/drivers/video/fbdev/omap2/omapfb/dss/hdmi4.c
index 7103c65..2e71aec8 100644
--- a/drivers/video/fbdev/omap2/omapfb/dss/hdmi4.c
+++ b/drivers/video/fbdev/omap2/omapfb/dss/hdmi4.c
@@ -114,13 +114,11 @@
return PTR_ERR(reg);
}
- if (regulator_can_change_voltage(reg)) {
- r = regulator_set_voltage(reg, 1800000, 1800000);
- if (r) {
- devm_regulator_put(reg);
- DSSWARN("can't set the regulator voltage\n");
- return r;
- }
+ r = regulator_set_voltage(reg, 1800000, 1800000);
+ if (r) {
+ devm_regulator_put(reg);
+ DSSWARN("can't set the regulator voltage\n");
+ return r;
}
hdmi.vdda_reg = reg;
diff --git a/drivers/video/fbdev/omap2/omapfb/dss/hdmi5.c b/drivers/video/fbdev/omap2/omapfb/dss/hdmi5.c
index a955a2c..aade6d99 100644
--- a/drivers/video/fbdev/omap2/omapfb/dss/hdmi5.c
+++ b/drivers/video/fbdev/omap2/omapfb/dss/hdmi5.c
@@ -131,13 +131,11 @@
return PTR_ERR(reg);
}
- if (regulator_can_change_voltage(reg)) {
- r = regulator_set_voltage(reg, 1800000, 1800000);
- if (r) {
- devm_regulator_put(reg);
- DSSWARN("can't set the regulator voltage\n");
- return r;
- }
+ r = regulator_set_voltage(reg, 1800000, 1800000);
+ if (r) {
+ devm_regulator_put(reg);
+ DSSWARN("can't set the regulator voltage\n");
+ return r;
}
hdmi.vdda_reg = reg;
diff --git a/drivers/video/fbdev/sh_mipi_dsi.c b/drivers/video/fbdev/sh_mipi_dsi.c
deleted file mode 100644
index 8f6e8ff..0000000
--- a/drivers/video/fbdev/sh_mipi_dsi.c
+++ /dev/null
@@ -1,587 +0,0 @@
-/*
- * Renesas SH-mobile MIPI DSI support
- *
- * Copyright (C) 2010 Guennadi Liakhovetski <g.liakhovetski@gmx.de>
- *
- * This is free software; you can redistribute it and/or modify
- * it under the terms of version 2 of the GNU General Public License as
- * published by the Free Software Foundation.
- */
-
-#include <linux/bitmap.h>
-#include <linux/clk.h>
-#include <linux/delay.h>
-#include <linux/init.h>
-#include <linux/io.h>
-#include <linux/platform_device.h>
-#include <linux/pm_runtime.h>
-#include <linux/slab.h>
-#include <linux/string.h>
-#include <linux/types.h>
-#include <linux/module.h>
-
-#include <video/mipi_display.h>
-#include <video/sh_mipi_dsi.h>
-#include <video/sh_mobile_lcdc.h>
-
-#include "sh_mobile_lcdcfb.h"
-
-#define SYSCTRL 0x0000
-#define SYSCONF 0x0004
-#define TIMSET 0x0008
-#define RESREQSET0 0x0018
-#define RESREQSET1 0x001c
-#define HSTTOVSET 0x0020
-#define LPRTOVSET 0x0024
-#define TATOVSET 0x0028
-#define PRTOVSET 0x002c
-#define DSICTRL 0x0030
-#define DSIINTE 0x0060
-#define PHYCTRL 0x0070
-
-/* relative to linkbase */
-#define DTCTR 0x0000
-#define VMCTR1 0x0020
-#define VMCTR2 0x0024
-#define VMLEN1 0x0028
-#define VMLEN2 0x002c
-#define CMTSRTREQ 0x0070
-#define CMTSRTCTR 0x00d0
-
-/* E.g., sh7372 has 2 MIPI-DSIs - one for each LCDC */
-#define MAX_SH_MIPI_DSI 2
-
-struct sh_mipi {
- struct sh_mobile_lcdc_entity entity;
-
- void __iomem *base;
- void __iomem *linkbase;
- struct clk *dsit_clk;
- struct platform_device *pdev;
-};
-
-#define to_sh_mipi(e) container_of(e, struct sh_mipi, entity)
-
-static struct sh_mipi *mipi_dsi[MAX_SH_MIPI_DSI];
-
-/* Protect the above array */
-static DEFINE_MUTEX(array_lock);
-
-static struct sh_mipi *sh_mipi_by_handle(int handle)
-{
- if (handle >= ARRAY_SIZE(mipi_dsi) || handle < 0)
- return NULL;
-
- return mipi_dsi[handle];
-}
-
-static int sh_mipi_send_short(struct sh_mipi *mipi, u8 dsi_cmd,
- u8 cmd, u8 param)
-{
- u32 data = (dsi_cmd << 24) | (cmd << 16) | (param << 8);
- int cnt = 100;
-
- /* transmit a short packet to LCD panel */
- iowrite32(1 | data, mipi->linkbase + CMTSRTCTR);
- iowrite32(1, mipi->linkbase + CMTSRTREQ);
-
- while ((ioread32(mipi->linkbase + CMTSRTREQ) & 1) && --cnt)
- udelay(1);
-
- return cnt ? 0 : -ETIMEDOUT;
-}
-
-#define LCD_CHAN2MIPI(c) ((c) < LCDC_CHAN_MAINLCD || (c) > LCDC_CHAN_SUBLCD ? \
- -EINVAL : (c) - 1)
-
-static int sh_mipi_dcs(int handle, u8 cmd)
-{
- struct sh_mipi *mipi = sh_mipi_by_handle(LCD_CHAN2MIPI(handle));
- if (!mipi)
- return -ENODEV;
- return sh_mipi_send_short(mipi, MIPI_DSI_DCS_SHORT_WRITE, cmd, 0);
-}
-
-static int sh_mipi_dcs_param(int handle, u8 cmd, u8 param)
-{
- struct sh_mipi *mipi = sh_mipi_by_handle(LCD_CHAN2MIPI(handle));
- if (!mipi)
- return -ENODEV;
- return sh_mipi_send_short(mipi, MIPI_DSI_DCS_SHORT_WRITE_PARAM, cmd,
- param);
-}
-
-static void sh_mipi_dsi_enable(struct sh_mipi *mipi, bool enable)
-{
- /*
- * enable LCDC data tx, transition to LPS after completion of each HS
- * packet
- */
- iowrite32(0x00000002 | enable, mipi->linkbase + DTCTR);
-}
-
-static void sh_mipi_shutdown(struct platform_device *pdev)
-{
- struct sh_mipi *mipi = to_sh_mipi(platform_get_drvdata(pdev));
-
- sh_mipi_dsi_enable(mipi, false);
-}
-
-static int sh_mipi_setup(struct sh_mipi *mipi, const struct fb_videomode *mode)
-{
- void __iomem *base = mipi->base;
- struct sh_mipi_dsi_info *pdata = mipi->pdev->dev.platform_data;
- u32 pctype, datatype, pixfmt, linelength, vmctr2;
- u32 tmp, top, bottom, delay, div;
- int bpp;
-
- /*
- * Select data format. MIPI DSI is not hot-pluggable, so, we just use
- * the default videomode. If this ever becomes a problem, We'll have to
- * move this to mipi_display_on() above and use info->var.xres
- */
- switch (pdata->data_format) {
- case MIPI_RGB888:
- pctype = 0;
- datatype = MIPI_DSI_PACKED_PIXEL_STREAM_24;
- pixfmt = MIPI_DCS_PIXEL_FMT_24BIT;
- linelength = mode->xres * 3;
- break;
- case MIPI_RGB565:
- pctype = 1;
- datatype = MIPI_DSI_PACKED_PIXEL_STREAM_16;
- pixfmt = MIPI_DCS_PIXEL_FMT_16BIT;
- linelength = mode->xres * 2;
- break;
- case MIPI_RGB666_LP:
- pctype = 2;
- datatype = MIPI_DSI_PIXEL_STREAM_3BYTE_18;
- pixfmt = MIPI_DCS_PIXEL_FMT_24BIT;
- linelength = mode->xres * 3;
- break;
- case MIPI_RGB666:
- pctype = 3;
- datatype = MIPI_DSI_PACKED_PIXEL_STREAM_18;
- pixfmt = MIPI_DCS_PIXEL_FMT_18BIT;
- linelength = (mode->xres * 18 + 7) / 8;
- break;
- case MIPI_BGR888:
- pctype = 8;
- datatype = MIPI_DSI_PACKED_PIXEL_STREAM_24;
- pixfmt = MIPI_DCS_PIXEL_FMT_24BIT;
- linelength = mode->xres * 3;
- break;
- case MIPI_BGR565:
- pctype = 9;
- datatype = MIPI_DSI_PACKED_PIXEL_STREAM_16;
- pixfmt = MIPI_DCS_PIXEL_FMT_16BIT;
- linelength = mode->xres * 2;
- break;
- case MIPI_BGR666_LP:
- pctype = 0xa;
- datatype = MIPI_DSI_PIXEL_STREAM_3BYTE_18;
- pixfmt = MIPI_DCS_PIXEL_FMT_24BIT;
- linelength = mode->xres * 3;
- break;
- case MIPI_BGR666:
- pctype = 0xb;
- datatype = MIPI_DSI_PACKED_PIXEL_STREAM_18;
- pixfmt = MIPI_DCS_PIXEL_FMT_18BIT;
- linelength = (mode->xres * 18 + 7) / 8;
- break;
- case MIPI_YUYV:
- pctype = 4;
- datatype = MIPI_DSI_PACKED_PIXEL_STREAM_YCBCR16;
- pixfmt = MIPI_DCS_PIXEL_FMT_16BIT;
- linelength = mode->xres * 2;
- break;
- case MIPI_UYVY:
- pctype = 5;
- datatype = MIPI_DSI_PACKED_PIXEL_STREAM_YCBCR16;
- pixfmt = MIPI_DCS_PIXEL_FMT_16BIT;
- linelength = mode->xres * 2;
- break;
- case MIPI_YUV420_L:
- pctype = 6;
- datatype = MIPI_DSI_PACKED_PIXEL_STREAM_YCBCR12;
- pixfmt = MIPI_DCS_PIXEL_FMT_12BIT;
- linelength = (mode->xres * 12 + 7) / 8;
- break;
- case MIPI_YUV420:
- pctype = 7;
- datatype = MIPI_DSI_PACKED_PIXEL_STREAM_YCBCR12;
- pixfmt = MIPI_DCS_PIXEL_FMT_12BIT;
- /* Length of U/V line */
- linelength = (mode->xres + 1) / 2;
- break;
- default:
- return -EINVAL;
- }
-
- if (!pdata->lane)
- return -EINVAL;
-
- /* reset DSI link */
- iowrite32(0x00000001, base + SYSCTRL);
- /* Hold reset for 100 cycles of the slowest of bus, HS byte and LP clock */
- udelay(50);
- iowrite32(0x00000000, base + SYSCTRL);
-
- /* setup DSI link */
-
- /*
- * T_wakeup = 0x7000
- * T_hs-trail = 3
- * T_hs-prepare = 3
- * T_clk-trail = 3
- * T_clk-prepare = 2
- */
- iowrite32(0x70003332, base + TIMSET);
- /* no responses requested */
- iowrite32(0x00000000, base + RESREQSET0);
- /* request response to packets of type 0x28 */
- iowrite32(0x00000100, base + RESREQSET1);
- /* High-speed transmission timeout, default 0xffffffff */
- iowrite32(0x0fffffff, base + HSTTOVSET);
- /* LP reception timeout, default 0xffffffff */
- iowrite32(0x0fffffff, base + LPRTOVSET);
- /* Turn-around timeout, default 0xffffffff */
- iowrite32(0x0fffffff, base + TATOVSET);
- /* Peripheral reset timeout, default 0xffffffff */
- iowrite32(0x0fffffff, base + PRTOVSET);
- /* Interrupts not used, disable all */
- iowrite32(0, base + DSIINTE);
- /* DSI-Tx bias on */
- iowrite32(0x00000001, base + PHYCTRL);
- udelay(200);
- /* Deassert resets, power on */
- iowrite32(0x03070001 | pdata->phyctrl, base + PHYCTRL);
-
- /*
- * Default = ULPS enable |
- * Contention detection enabled |
- * EoT packet transmission enable |
- * CRC check enable |
- * ECC check enable
- */
- bitmap_fill((unsigned long *)&tmp, pdata->lane);
- tmp |= 0x00003700;
- iowrite32(tmp, base + SYSCONF);
-
- /* setup l-bridge */
-
- /*
- * Enable transmission of all packets,
- * transmit LPS after each HS packet completion
- */
- iowrite32(0x00000006, mipi->linkbase + DTCTR);
- /* VSYNC width = 2 (<< 17) */
- iowrite32((mode->vsync_len << pdata->vsynw_offset) |
- (pdata->clksrc << 16) | (pctype << 12) | datatype,
- mipi->linkbase + VMCTR1);
-
- /*
- * Non-burst mode with sync pulses: VSE and HSE are output,
- * HSA period allowed, no commands in LP
- */
- vmctr2 = 0;
- if (pdata->flags & SH_MIPI_DSI_VSEE)
- vmctr2 |= 1 << 23;
- if (pdata->flags & SH_MIPI_DSI_HSEE)
- vmctr2 |= 1 << 22;
- if (pdata->flags & SH_MIPI_DSI_HSAE)
- vmctr2 |= 1 << 21;
- if (pdata->flags & SH_MIPI_DSI_BL2E)
- vmctr2 |= 1 << 17;
- if (pdata->flags & SH_MIPI_DSI_HSABM)
- vmctr2 |= 1 << 5;
- if (pdata->flags & SH_MIPI_DSI_HBPBM)
- vmctr2 |= 1 << 4;
- if (pdata->flags & SH_MIPI_DSI_HFPBM)
- vmctr2 |= 1 << 3;
- iowrite32(vmctr2, mipi->linkbase + VMCTR2);
-
- /*
- * VMLEN1 = RGBLEN | HSALEN
- *
- * see
- * Video mode - Blanking Packet setting
- */
- top = linelength << 16; /* RGBLEN */
- bottom = 0x00000001;
- if (pdata->flags & SH_MIPI_DSI_HSABM) /* HSALEN */
- bottom = (pdata->lane * mode->hsync_len) - 10;
- iowrite32(top | bottom , mipi->linkbase + VMLEN1);
-
- /*
- * VMLEN2 = HBPLEN | HFPLEN
- *
- * see
- * Video mode - Blanking Packet setting
- */
- top = 0x00010000;
- bottom = 0x00000001;
- delay = 0;
-
- div = 1; /* HSbyteCLK is calculation base
- * HS4divCLK = HSbyteCLK/2
- * HS6divCLK is not supported for now */
- if (pdata->flags & SH_MIPI_DSI_HS4divCLK)
- div = 2;
-
- if (pdata->flags & SH_MIPI_DSI_HFPBM) { /* HBPLEN */
- top = mode->hsync_len + mode->left_margin;
- top = ((pdata->lane * top / div) - 10) << 16;
- }
- if (pdata->flags & SH_MIPI_DSI_HBPBM) { /* HFPLEN */
- bottom = mode->right_margin;
- bottom = (pdata->lane * bottom / div) - 12;
- }
-
- bpp = linelength / mode->xres; /* byte / pixel */
- if ((pdata->lane / div) > bpp) {
- tmp = mode->xres / bpp; /* output cycle */
- tmp = mode->xres - tmp; /* (input - output) cycle */
- delay = (pdata->lane * tmp);
- }
-
- iowrite32(top | (bottom + delay) , mipi->linkbase + VMLEN2);
-
- msleep(5);
-
- /* setup LCD panel */
-
- /* cf. drivers/video/omap/lcd_mipid.c */
- sh_mipi_dcs(pdata->channel, MIPI_DCS_EXIT_SLEEP_MODE);
- msleep(120);
- /*
- * [7] - Page Address Mode
- * [6] - Column Address Mode
- * [5] - Page / Column Address Mode
- * [4] - Display Device Line Refresh Order
- * [3] - RGB/BGR Order
- * [2] - Display Data Latch Data Order
- * [1] - Flip Horizontal
- * [0] - Flip Vertical
- */
- sh_mipi_dcs_param(pdata->channel, MIPI_DCS_SET_ADDRESS_MODE, 0x00);
- /* cf. set_data_lines() */
- sh_mipi_dcs_param(pdata->channel, MIPI_DCS_SET_PIXEL_FORMAT,
- pixfmt << 4);
- sh_mipi_dcs(pdata->channel, MIPI_DCS_SET_DISPLAY_ON);
-
- /* Enable timeout counters */
- iowrite32(0x00000f00, base + DSICTRL);
-
- return 0;
-}
-
-static int mipi_display_on(struct sh_mobile_lcdc_entity *entity)
-{
- struct sh_mipi *mipi = to_sh_mipi(entity);
- struct sh_mipi_dsi_info *pdata = mipi->pdev->dev.platform_data;
- int ret;
-
- pm_runtime_get_sync(&mipi->pdev->dev);
-
- ret = pdata->set_dot_clock(mipi->pdev, mipi->base, 1);
- if (ret < 0)
- goto mipi_display_on_fail1;
-
- ret = sh_mipi_setup(mipi, &entity->def_mode);
- if (ret < 0)
- goto mipi_display_on_fail2;
-
- sh_mipi_dsi_enable(mipi, true);
-
- return SH_MOBILE_LCDC_DISPLAY_CONNECTED;
-
-mipi_display_on_fail1:
- pm_runtime_put_sync(&mipi->pdev->dev);
-mipi_display_on_fail2:
- pdata->set_dot_clock(mipi->pdev, mipi->base, 0);
-
- return ret;
-}
-
-static void mipi_display_off(struct sh_mobile_lcdc_entity *entity)
-{
- struct sh_mipi *mipi = to_sh_mipi(entity);
- struct sh_mipi_dsi_info *pdata = mipi->pdev->dev.platform_data;
-
- sh_mipi_dsi_enable(mipi, false);
-
- pdata->set_dot_clock(mipi->pdev, mipi->base, 0);
-
- pm_runtime_put_sync(&mipi->pdev->dev);
-}
-
-static const struct sh_mobile_lcdc_entity_ops mipi_ops = {
- .display_on = mipi_display_on,
- .display_off = mipi_display_off,
-};
-
-static int __init sh_mipi_probe(struct platform_device *pdev)
-{
- struct sh_mipi *mipi;
- struct sh_mipi_dsi_info *pdata = pdev->dev.platform_data;
- struct resource *res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- struct resource *res2 = platform_get_resource(pdev, IORESOURCE_MEM, 1);
- unsigned long rate, f_current;
- int idx = pdev->id, ret;
-
- if (!res || !res2 || idx >= ARRAY_SIZE(mipi_dsi) || !pdata)
- return -ENODEV;
-
- if (!pdata->set_dot_clock)
- return -EINVAL;
-
- mutex_lock(&array_lock);
- if (idx < 0)
- for (idx = 0; idx < ARRAY_SIZE(mipi_dsi) && mipi_dsi[idx]; idx++)
- ;
-
- if (idx == ARRAY_SIZE(mipi_dsi)) {
- ret = -EBUSY;
- goto efindslot;
- }
-
- mipi = kzalloc(sizeof(*mipi), GFP_KERNEL);
- if (!mipi) {
- ret = -ENOMEM;
- goto ealloc;
- }
-
- mipi->entity.owner = THIS_MODULE;
- mipi->entity.ops = &mipi_ops;
-
- if (!request_mem_region(res->start, resource_size(res), pdev->name)) {
- dev_err(&pdev->dev, "MIPI register region already claimed\n");
- ret = -EBUSY;
- goto ereqreg;
- }
-
- mipi->base = ioremap(res->start, resource_size(res));
- if (!mipi->base) {
- ret = -ENOMEM;
- goto emap;
- }
-
- if (!request_mem_region(res2->start, resource_size(res2), pdev->name)) {
- dev_err(&pdev->dev, "MIPI register region 2 already claimed\n");
- ret = -EBUSY;
- goto ereqreg2;
- }
-
- mipi->linkbase = ioremap(res2->start, resource_size(res2));
- if (!mipi->linkbase) {
- ret = -ENOMEM;
- goto emap2;
- }
-
- mipi->pdev = pdev;
-
- mipi->dsit_clk = clk_get(&pdev->dev, "dsit_clk");
- if (IS_ERR(mipi->dsit_clk)) {
- ret = PTR_ERR(mipi->dsit_clk);
- goto eclktget;
- }
-
- f_current = clk_get_rate(mipi->dsit_clk);
- /* 80MHz required by the datasheet */
- rate = clk_round_rate(mipi->dsit_clk, 80000000);
- if (rate > 0 && rate != f_current)
- ret = clk_set_rate(mipi->dsit_clk, rate);
- else
- ret = rate;
- if (ret < 0)
- goto esettrate;
-
- dev_dbg(&pdev->dev, "DSI-T clk %lu -> %lu\n", f_current, rate);
-
- ret = clk_enable(mipi->dsit_clk);
- if (ret < 0)
- goto eclkton;
-
- mipi_dsi[idx] = mipi;
-
- pm_runtime_enable(&pdev->dev);
- pm_runtime_resume(&pdev->dev);
-
- mutex_unlock(&array_lock);
- platform_set_drvdata(pdev, &mipi->entity);
-
- return 0;
-
-eclkton:
-esettrate:
- clk_put(mipi->dsit_clk);
-eclktget:
- iounmap(mipi->linkbase);
-emap2:
- release_mem_region(res2->start, resource_size(res2));
-ereqreg2:
- iounmap(mipi->base);
-emap:
- release_mem_region(res->start, resource_size(res));
-ereqreg:
- kfree(mipi);
-ealloc:
-efindslot:
- mutex_unlock(&array_lock);
-
- return ret;
-}
-
-static int sh_mipi_remove(struct platform_device *pdev)
-{
- struct resource *res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- struct resource *res2 = platform_get_resource(pdev, IORESOURCE_MEM, 1);
- struct sh_mipi *mipi = to_sh_mipi(platform_get_drvdata(pdev));
- int i, ret;
-
- mutex_lock(&array_lock);
-
- for (i = 0; i < ARRAY_SIZE(mipi_dsi) && mipi_dsi[i] != mipi; i++)
- ;
-
- if (i == ARRAY_SIZE(mipi_dsi)) {
- ret = -EINVAL;
- } else {
- ret = 0;
- mipi_dsi[i] = NULL;
- }
-
- mutex_unlock(&array_lock);
-
- if (ret < 0)
- return ret;
-
- pm_runtime_disable(&pdev->dev);
- clk_disable(mipi->dsit_clk);
- clk_put(mipi->dsit_clk);
-
- iounmap(mipi->linkbase);
- if (res2)
- release_mem_region(res2->start, resource_size(res2));
- iounmap(mipi->base);
- if (res)
- release_mem_region(res->start, resource_size(res));
- kfree(mipi);
-
- return 0;
-}
-
-static struct platform_driver sh_mipi_driver = {
- .remove = sh_mipi_remove,
- .shutdown = sh_mipi_shutdown,
- .driver = {
- .name = "sh-mipi-dsi",
- },
-};
-
-module_platform_driver_probe(sh_mipi_driver, sh_mipi_probe);
-
-MODULE_AUTHOR("Guennadi Liakhovetski <g.liakhovetski@gmx.de>");
-MODULE_DESCRIPTION("SuperH / ARM-shmobile MIPI DSI driver");
-MODULE_LICENSE("GPL v2");
diff --git a/drivers/video/fbdev/ssd1307fb.c b/drivers/video/fbdev/ssd1307fb.c
index fa34808..21dafe5 100644
--- a/drivers/video/fbdev/ssd1307fb.c
+++ b/drivers/video/fbdev/ssd1307fb.c
@@ -389,7 +389,7 @@
return ret;
ret = ssd1307fb_write_cmd(par->client,
- (par->device_info->need_chargepump & 0x1 << 2) & 0x14);
+ BIT(4) | (par->device_info->need_chargepump ? BIT(2) : 0));
if (ret < 0)
return ret;
diff --git a/drivers/video/fbdev/via/accel.c b/drivers/video/fbdev/via/accel.c
index 4b67b8e..eb3615c 100644
--- a/drivers/video/fbdev/via/accel.c
+++ b/drivers/video/fbdev/via/accel.c
@@ -358,7 +358,7 @@
viapar->shared->vq_vram_addr = viapar->fbmem_free;
viapar->fbmem_used += VQ_SIZE;
-#if defined(CONFIG_VIDEO_VIA_CAMERA) || defined(CONFIG_VIDEO_VIA_CAMERA_MODULE)
+#if IS_ENABLED(CONFIG_VIDEO_VIA_CAMERA)
/*
* Set aside a chunk of framebuffer memory for the camera
* driver. Someday this driver probably needs a proper allocator
diff --git a/drivers/video/fbdev/via/via-core.c b/drivers/video/fbdev/via/via-core.c
index 6e27482..1d28e16 100644
--- a/drivers/video/fbdev/via/via-core.c
+++ b/drivers/video/fbdev/via/via-core.c
@@ -116,7 +116,7 @@
* most viafb systems will not need to have this extra code for a while.
* As soon as another user comes long, the ifdef can be removed.
*/
-#if defined(CONFIG_VIDEO_VIA_CAMERA) || defined(CONFIG_VIDEO_VIA_CAMERA_MODULE)
+#if IS_ENABLED(CONFIG_VIDEO_VIA_CAMERA)
/*
* Access to the DMA engine. This currently provides what the camera
* driver needs (i.e. outgoing only) but is easily expandable if need
@@ -542,7 +542,7 @@
{
.name = "viafb-i2c",
},
-#if defined(CONFIG_VIDEO_VIA_CAMERA) || defined(CONFIG_VIDEO_VIA_CAMERA_MODULE)
+#if IS_ENABLED(CONFIG_VIDEO_VIA_CAMERA)
{
.name = "viafb-camera",
},
diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
index 7b6d74f..476c0e3 100644
--- a/drivers/virtio/virtio_balloon.c
+++ b/drivers/virtio/virtio_balloon.c
@@ -75,7 +75,7 @@
/* The array of pfns we tell the Host about. */
unsigned int num_pfns;
- u32 pfns[VIRTIO_BALLOON_ARRAY_PFNS_MAX];
+ __virtio32 pfns[VIRTIO_BALLOON_ARRAY_PFNS_MAX];
/* Memory statistics */
struct virtio_balloon_stat stats[VIRTIO_BALLOON_S_NR];
@@ -127,14 +127,16 @@
}
-static void set_page_pfns(u32 pfns[], struct page *page)
+static void set_page_pfns(struct virtio_balloon *vb,
+ __virtio32 pfns[], struct page *page)
{
unsigned int i;
/* Set balloon pfns pointing at this page.
* Note that the first pfn points at start of the page. */
for (i = 0; i < VIRTIO_BALLOON_PAGES_PER_PAGE; i++)
- pfns[i] = page_to_balloon_pfn(page) + i;
+ pfns[i] = cpu_to_virtio32(vb->vdev,
+ page_to_balloon_pfn(page) + i);
}
static unsigned fill_balloon(struct virtio_balloon *vb, size_t num)
@@ -158,7 +160,7 @@
msleep(200);
break;
}
- set_page_pfns(vb->pfns + vb->num_pfns, page);
+ set_page_pfns(vb, vb->pfns + vb->num_pfns, page);
vb->num_pages += VIRTIO_BALLOON_PAGES_PER_PAGE;
if (!virtio_has_feature(vb->vdev,
VIRTIO_BALLOON_F_DEFLATE_ON_OOM))
@@ -177,10 +179,12 @@
static void release_pages_balloon(struct virtio_balloon *vb)
{
unsigned int i;
+ struct page *page;
/* Find pfns pointing at start of each page, get pages and free them. */
for (i = 0; i < vb->num_pfns; i += VIRTIO_BALLOON_PAGES_PER_PAGE) {
- struct page *page = balloon_pfn_to_page(vb->pfns[i]);
+ page = balloon_pfn_to_page(virtio32_to_cpu(vb->vdev,
+ vb->pfns[i]));
if (!virtio_has_feature(vb->vdev,
VIRTIO_BALLOON_F_DEFLATE_ON_OOM))
adjust_managed_page_count(page, 1);
@@ -203,7 +207,7 @@
page = balloon_page_dequeue(vb_dev_info);
if (!page)
break;
- set_page_pfns(vb->pfns + vb->num_pfns, page);
+ set_page_pfns(vb, vb->pfns + vb->num_pfns, page);
vb->num_pages -= VIRTIO_BALLOON_PAGES_PER_PAGE;
}
@@ -471,13 +475,13 @@
__count_vm_event(BALLOON_MIGRATE);
spin_unlock_irqrestore(&vb_dev_info->pages_lock, flags);
vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
- set_page_pfns(vb->pfns, newpage);
+ set_page_pfns(vb, vb->pfns, newpage);
tell_host(vb, vb->inflate_vq);
/* balloon's page migration 2nd step -- deflate "page" */
balloon_page_delete(page);
vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
- set_page_pfns(vb->pfns, page);
+ set_page_pfns(vb, vb->pfns, page);
tell_host(vb, vb->deflate_vq);
mutex_unlock(&vb->balloon_lock);
diff --git a/drivers/vme/bridges/vme_ca91cx42.c b/drivers/vme/bridges/vme_ca91cx42.c
index 5fbeab3..9f2c834 100644
--- a/drivers/vme/bridges/vme_ca91cx42.c
+++ b/drivers/vme/bridges/vme_ca91cx42.c
@@ -204,10 +204,6 @@
/* Need pdev */
pdev = to_pci_dev(ca91cx42_bridge->parent);
- INIT_LIST_HEAD(&ca91cx42_bridge->vme_error_handlers);
-
- mutex_init(&ca91cx42_bridge->irq_mtx);
-
/* Disable interrupts from PCI to VME */
iowrite32(0, bridge->base + VINT_EN);
@@ -1626,6 +1622,7 @@
retval = -ENOMEM;
goto err_struct;
}
+ vme_init_bridge(ca91cx42_bridge);
ca91cx42_device = kzalloc(sizeof(struct ca91cx42_driver), GFP_KERNEL);
@@ -1686,7 +1683,6 @@
}
/* Add master windows to list */
- INIT_LIST_HEAD(&ca91cx42_bridge->master_resources);
for (i = 0; i < CA91C142_MAX_MASTER; i++) {
master_image = kmalloc(sizeof(struct vme_master_resource),
GFP_KERNEL);
@@ -1713,7 +1709,6 @@
}
/* Add slave windows to list */
- INIT_LIST_HEAD(&ca91cx42_bridge->slave_resources);
for (i = 0; i < CA91C142_MAX_SLAVE; i++) {
slave_image = kmalloc(sizeof(struct vme_slave_resource),
GFP_KERNEL);
@@ -1741,7 +1736,6 @@
}
/* Add dma engines to list */
- INIT_LIST_HEAD(&ca91cx42_bridge->dma_resources);
for (i = 0; i < CA91C142_MAX_DMA; i++) {
dma_ctrlr = kmalloc(sizeof(struct vme_dma_resource),
GFP_KERNEL);
@@ -1764,7 +1758,6 @@
}
/* Add location monitor to list */
- INIT_LIST_HEAD(&ca91cx42_bridge->lm_resources);
lm = kmalloc(sizeof(struct vme_lm_resource), GFP_KERNEL);
if (lm == NULL) {
dev_err(&pdev->dev, "Failed to allocate memory for "
diff --git a/drivers/vme/bridges/vme_tsi148.c b/drivers/vme/bridges/vme_tsi148.c
index 6052483..4bc5d45 100644
--- a/drivers/vme/bridges/vme_tsi148.c
+++ b/drivers/vme/bridges/vme_tsi148.c
@@ -314,10 +314,6 @@
bridge = tsi148_bridge->driver_priv;
- INIT_LIST_HEAD(&tsi148_bridge->vme_error_handlers);
-
- mutex_init(&tsi148_bridge->irq_mtx);
-
result = request_irq(pdev->irq,
tsi148_irqhandler,
IRQF_SHARED,
@@ -2301,6 +2297,7 @@
retval = -ENOMEM;
goto err_struct;
}
+ vme_init_bridge(tsi148_bridge);
tsi148_device = kzalloc(sizeof(struct tsi148_driver), GFP_KERNEL);
if (tsi148_device == NULL) {
@@ -2387,7 +2384,6 @@
}
/* Add master windows to list */
- INIT_LIST_HEAD(&tsi148_bridge->master_resources);
for (i = 0; i < master_num; i++) {
master_image = kmalloc(sizeof(struct vme_master_resource),
GFP_KERNEL);
@@ -2417,7 +2413,6 @@
}
/* Add slave windows to list */
- INIT_LIST_HEAD(&tsi148_bridge->slave_resources);
for (i = 0; i < TSI148_MAX_SLAVE; i++) {
slave_image = kmalloc(sizeof(struct vme_slave_resource),
GFP_KERNEL);
@@ -2442,7 +2437,6 @@
}
/* Add dma engines to list */
- INIT_LIST_HEAD(&tsi148_bridge->dma_resources);
for (i = 0; i < TSI148_MAX_DMA; i++) {
dma_ctrlr = kmalloc(sizeof(struct vme_dma_resource),
GFP_KERNEL);
@@ -2467,7 +2461,6 @@
}
/* Add location monitor to list */
- INIT_LIST_HEAD(&tsi148_bridge->lm_resources);
lm = kmalloc(sizeof(struct vme_lm_resource), GFP_KERNEL);
if (lm == NULL) {
dev_err(&pdev->dev, "Failed to allocate memory for "
diff --git a/drivers/vme/vme.c b/drivers/vme/vme.c
index 72924b0..37ac0a5 100644
--- a/drivers/vme/vme.c
+++ b/drivers/vme/vme.c
@@ -782,7 +782,7 @@
dma_list = kmalloc(sizeof(struct vme_dma_list), GFP_KERNEL);
if (dma_list == NULL) {
- printk(KERN_ERR "Unable to allocate memory for new dma list\n");
+ printk(KERN_ERR "Unable to allocate memory for new DMA list\n");
return NULL;
}
INIT_LIST_HEAD(&dma_list->entries);
@@ -846,7 +846,7 @@
pci_attr = kmalloc(sizeof(struct vme_dma_pci), GFP_KERNEL);
if (pci_attr == NULL) {
- printk(KERN_ERR "Unable to allocate memory for pci attributes\n");
+ printk(KERN_ERR "Unable to allocate memory for PCI attributes\n");
goto err_pci;
}
@@ -884,7 +884,7 @@
vme_attr = kmalloc(sizeof(struct vme_dma_vme), GFP_KERNEL);
if (vme_attr == NULL) {
- printk(KERN_ERR "Unable to allocate memory for vme attributes\n");
+ printk(KERN_ERR "Unable to allocate memory for VME attributes\n");
goto err_vme;
}
@@ -975,8 +975,8 @@
}
/*
- * Empty out all of the entries from the dma list. We need to go to the
- * low level driver as dma entries are driver specific.
+ * Empty out all of the entries from the DMA list. We need to go to the
+ * low level driver as DMA entries are driver specific.
*/
retval = bridge->dma_list_empty(list);
if (retval) {
@@ -1091,7 +1091,7 @@
if (call != NULL)
call(level, statid, priv_data);
else
- printk(KERN_WARNING "Spurilous VME interrupt, level:%x, vector:%x\n",
+ printk(KERN_WARNING "Spurious VME interrupt, level:%x, vector:%x\n",
level, statid);
}
EXPORT_SYMBOL(vme_irq_handler);
@@ -1429,6 +1429,20 @@
kfree(dev_to_vme_dev(dev));
}
+/* Common bridge initialization */
+struct vme_bridge *vme_init_bridge(struct vme_bridge *bridge)
+{
+ INIT_LIST_HEAD(&bridge->vme_error_handlers);
+ INIT_LIST_HEAD(&bridge->master_resources);
+ INIT_LIST_HEAD(&bridge->slave_resources);
+ INIT_LIST_HEAD(&bridge->dma_resources);
+ INIT_LIST_HEAD(&bridge->lm_resources);
+ mutex_init(&bridge->irq_mtx);
+
+ return bridge;
+}
+EXPORT_SYMBOL(vme_init_bridge);
+
int vme_register_bridge(struct vme_bridge *bridge)
{
int i;
diff --git a/drivers/vme/vme_bridge.h b/drivers/vme/vme_bridge.h
index b59cbee..cb8246f 100644
--- a/drivers/vme/vme_bridge.h
+++ b/drivers/vme/vme_bridge.h
@@ -177,6 +177,7 @@
unsigned long long address, int am);
void vme_irq_handler(struct vme_bridge *, int, int);
+struct vme_bridge *vme_init_bridge(struct vme_bridge *);
int vme_register_bridge(struct vme_bridge *);
void vme_unregister_bridge(struct vme_bridge *);
struct vme_error_handler *vme_register_error_handler(
diff --git a/drivers/w1/masters/ds2482.c b/drivers/w1/masters/ds2482.c
index b05e8fe..2e30db1 100644
--- a/drivers/w1/masters/ds2482.c
+++ b/drivers/w1/masters/ds2482.c
@@ -24,6 +24,19 @@
#include "../w1_int.h"
/**
+ * Allow the active pullup to be disabled, default is enabled.
+ *
+ * Note from the DS2482 datasheet:
+ * The APU bit controls whether an active pullup (controlled slew-rate
+ * transistor) or a passive pullup (Rwpu resistor) will be used to drive
+ * a 1-Wire line from low to high. When APU = 0, active pullup is disabled
+ * (resistor mode). Active Pullup should always be selected unless there is
+ * only a single slave on the 1-Wire line.
+ */
+static int ds2482_active_pullup = 1;
+module_param_named(active_pullup, ds2482_active_pullup, int, 0644);
+
+/**
* The DS2482 registers - there are 3 registers that are addressed by a read
* pointer. The read pointer is set by the last command executed.
*
@@ -138,6 +151,9 @@
*/
static inline u8 ds2482_calculate_config(u8 conf)
{
+ if (ds2482_active_pullup)
+ conf |= DS2482_REG_CFG_APU;
+
return conf | ((~conf & 0x0f) << 4);
}
@@ -546,6 +562,8 @@
module_i2c_driver(ds2482_driver);
+MODULE_PARM_DESC(active_pullup, "Active pullup (apply to all buses): " \
+ "0-disable, 1-enable (default)");
MODULE_AUTHOR("Ben Gardner <bgardner@wabtec.com>");
MODULE_DESCRIPTION("DS2482 driver");
MODULE_LICENSE("GPL");
diff --git a/drivers/w1/slaves/w1_therm.c b/drivers/w1/slaves/w1_therm.c
index 2f029e8..581a300 100644
--- a/drivers/w1/slaves/w1_therm.c
+++ b/drivers/w1/slaves/w1_therm.c
@@ -92,10 +92,13 @@
static ssize_t w1_slave_show(struct device *device,
struct device_attribute *attr, char *buf);
+static ssize_t w1_slave_store(struct device *device,
+ struct device_attribute *attr, const char *buf, size_t size);
+
static ssize_t w1_seq_show(struct device *device,
struct device_attribute *attr, char *buf);
-static DEVICE_ATTR_RO(w1_slave);
+static DEVICE_ATTR_RW(w1_slave);
static DEVICE_ATTR_RO(w1_seq);
static struct attribute *w1_therm_attrs[] = {
@@ -154,8 +157,17 @@
u16 reserved;
struct w1_family *f;
int (*convert)(u8 rom[9]);
+ int (*precision)(struct device *device, int val);
+ int (*eeprom)(struct device *device);
};
+/* write configuration to eeprom */
+static inline int w1_therm_eeprom(struct device *device);
+
+/* Set precision for conversion */
+static inline int w1_DS18B20_precision(struct device *device, int val);
+static inline int w1_DS18S20_precision(struct device *device, int val);
+
/* The return value is millidegrees Centigrade. */
static inline int w1_DS18B20_convert_temp(u8 rom[9]);
static inline int w1_DS18S20_convert_temp(u8 rom[9]);
@@ -163,26 +175,194 @@
static struct w1_therm_family_converter w1_therm_families[] = {
{
.f = &w1_therm_family_DS18S20,
- .convert = w1_DS18S20_convert_temp
+ .convert = w1_DS18S20_convert_temp,
+ .precision = w1_DS18S20_precision,
+ .eeprom = w1_therm_eeprom
},
{
.f = &w1_therm_family_DS1822,
- .convert = w1_DS18B20_convert_temp
+ .convert = w1_DS18B20_convert_temp,
+ .precision = w1_DS18S20_precision,
+ .eeprom = w1_therm_eeprom
},
{
.f = &w1_therm_family_DS18B20,
- .convert = w1_DS18B20_convert_temp
+ .convert = w1_DS18B20_convert_temp,
+ .precision = w1_DS18B20_precision,
+ .eeprom = w1_therm_eeprom
},
{
.f = &w1_therm_family_DS28EA00,
- .convert = w1_DS18B20_convert_temp
+ .convert = w1_DS18B20_convert_temp,
+ .precision = w1_DS18S20_precision,
+ .eeprom = w1_therm_eeprom
},
{
.f = &w1_therm_family_DS1825,
- .convert = w1_DS18B20_convert_temp
+ .convert = w1_DS18B20_convert_temp,
+ .precision = w1_DS18S20_precision,
+ .eeprom = w1_therm_eeprom
}
};
+static inline int w1_therm_eeprom(struct device *device)
+{
+ struct w1_slave *sl = dev_to_w1_slave(device);
+ struct w1_master *dev = sl->master;
+ u8 rom[9], external_power;
+ int ret, max_trying = 10;
+ u8 *family_data = sl->family_data;
+
+ ret = mutex_lock_interruptible(&dev->bus_mutex);
+ if (ret != 0)
+ goto post_unlock;
+
+ if (!sl->family_data) {
+ ret = -ENODEV;
+ goto pre_unlock;
+ }
+
+ /* prevent the slave from going away in sleep */
+ atomic_inc(THERM_REFCNT(family_data));
+ memset(rom, 0, sizeof(rom));
+
+ while (max_trying--) {
+ if (!w1_reset_select_slave(sl)) {
+ unsigned int tm = 10;
+ unsigned long sleep_rem;
+
+ /* check if in parasite mode */
+ w1_write_8(dev, W1_READ_PSUPPLY);
+ external_power = w1_read_8(dev);
+
+ if (w1_reset_select_slave(sl))
+ continue;
+
+ /* 10ms strong pullup/delay after the copy command */
+ if (w1_strong_pullup == 2 ||
+ (!external_power && w1_strong_pullup))
+ w1_next_pullup(dev, tm);
+
+ w1_write_8(dev, W1_COPY_SCRATCHPAD);
+
+ if (external_power) {
+ mutex_unlock(&dev->bus_mutex);
+
+ sleep_rem = msleep_interruptible(tm);
+ if (sleep_rem != 0) {
+ ret = -EINTR;
+ goto post_unlock;
+ }
+
+ ret = mutex_lock_interruptible(&dev->bus_mutex);
+ if (ret != 0)
+ goto post_unlock;
+ } else if (!w1_strong_pullup) {
+ sleep_rem = msleep_interruptible(tm);
+ if (sleep_rem != 0) {
+ ret = -EINTR;
+ goto pre_unlock;
+ }
+ }
+
+ break;
+ }
+ }
+
+pre_unlock:
+ mutex_unlock(&dev->bus_mutex);
+
+post_unlock:
+ atomic_dec(THERM_REFCNT(family_data));
+ return ret;
+}
+
+/* DS18S20 does not feature configuration register */
+static inline int w1_DS18S20_precision(struct device *device, int val)
+{
+ return 0;
+}
+
+static inline int w1_DS18B20_precision(struct device *device, int val)
+{
+ struct w1_slave *sl = dev_to_w1_slave(device);
+ struct w1_master *dev = sl->master;
+ u8 rom[9], crc;
+ int ret, max_trying = 10;
+ u8 *family_data = sl->family_data;
+ uint8_t precision_bits;
+ uint8_t mask = 0x60;
+
+ if(val > 12 || val < 9) {
+ pr_warn("Unsupported precision\n");
+ return -1;
+ }
+
+ ret = mutex_lock_interruptible(&dev->bus_mutex);
+ if (ret != 0)
+ goto post_unlock;
+
+ if (!sl->family_data) {
+ ret = -ENODEV;
+ goto pre_unlock;
+ }
+
+ /* prevent the slave from going away in sleep */
+ atomic_inc(THERM_REFCNT(family_data));
+ memset(rom, 0, sizeof(rom));
+
+ /* translate precision to bitmask (see datasheet page 9) */
+ switch (val) {
+ case 9:
+ precision_bits = 0x00;
+ break;
+ case 10:
+ precision_bits = 0x20;
+ break;
+ case 11:
+ precision_bits = 0x40;
+ break;
+ case 12:
+ default:
+ precision_bits = 0x60;
+ break;
+ }
+
+ while (max_trying--) {
+ crc = 0;
+
+ if (!w1_reset_select_slave(sl)) {
+ int count = 0;
+
+ /* read values to only alter precision bits */
+ w1_write_8(dev, W1_READ_SCRATCHPAD);
+ if ((count = w1_read_block(dev, rom, 9)) != 9)
+ dev_warn(device, "w1_read_block() returned %u instead of 9.\n", count);
+
+ crc = w1_calc_crc8(rom, 8);
+ if (rom[8] == crc) {
+ rom[4] = (rom[4] & ~mask) | (precision_bits & mask);
+
+ if (!w1_reset_select_slave(sl)) {
+ w1_write_8(dev, W1_WRITE_SCRATCHPAD);
+ w1_write_8(dev, rom[2]);
+ w1_write_8(dev, rom[3]);
+ w1_write_8(dev, rom[4]);
+
+ break;
+ }
+ }
+ }
+ }
+
+pre_unlock:
+ mutex_unlock(&dev->bus_mutex);
+
+post_unlock:
+ atomic_dec(THERM_REFCNT(family_data));
+ return ret;
+}
+
static inline int w1_DS18B20_convert_temp(u8 rom[9])
{
s16 t = le16_to_cpup((__le16 *)rom);
@@ -220,6 +400,30 @@
return 0;
}
+static ssize_t w1_slave_store(struct device *device,
+ struct device_attribute *attr, const char *buf,
+ size_t size)
+{
+ int val, ret;
+ struct w1_slave *sl = dev_to_w1_slave(device);
+ int i;
+
+ ret = kstrtoint(buf, 0, &val);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < ARRAY_SIZE(w1_therm_families); ++i) {
+ if (w1_therm_families[i].f->fid == sl->family->fid) {
+ /* zero value indicates to write current configuration to eeprom */
+ if (0 == val)
+ ret = w1_therm_families[i].eeprom(device);
+ else
+ ret = w1_therm_families[i].precision(device, val);
+ break;
+ }
+ }
+ return ret ? : size;
+}
static ssize_t w1_slave_show(struct device *device,
struct device_attribute *attr, char *buf)
@@ -311,7 +515,7 @@
for (i = 0; i < 9; ++i)
c -= snprintf(buf + PAGE_SIZE - c, c, "%02x ", rom[i]);
c -= snprintf(buf + PAGE_SIZE - c, c, ": crc=%02x %s\n",
- crc, (verdict) ? "YES" : "NO");
+ crc, (verdict) ? "YES" : "NO");
if (verdict)
memcpy(family_data, rom, sizeof(rom));
else
diff --git a/drivers/w1/w1.c b/drivers/w1/w1.c
index 89a7847..bb34362 100644
--- a/drivers/w1/w1.c
+++ b/drivers/w1/w1.c
@@ -335,7 +335,7 @@
int tmp;
struct w1_master *md = dev_to_w1_master(dev);
- if (kstrtoint(buf, 0, &tmp) == -EINVAL || tmp < 1)
+ if (kstrtoint(buf, 0, &tmp) || tmp < 1)
return -EINVAL;
mutex_lock(&md->mutex);
diff --git a/drivers/w1/w1.h b/drivers/w1/w1.h
index 56a49ba..129895f 100644
--- a/drivers/w1/w1.h
+++ b/drivers/w1/w1.h
@@ -58,6 +58,8 @@
#define W1_ALARM_SEARCH 0xEC
#define W1_CONVERT_TEMP 0x44
#define W1_SKIP_ROM 0xCC
+#define W1_COPY_SCRATCHPAD 0x48
+#define W1_WRITE_SCRATCHPAD 0x4E
#define W1_READ_SCRATCHPAD 0xBE
#define W1_READ_ROM 0x33
#define W1_READ_PSUPPLY 0xB4
diff --git a/drivers/watchdog/Kconfig b/drivers/watchdog/Kconfig
index fb94765..5b45e27 100644
--- a/drivers/watchdog/Kconfig
+++ b/drivers/watchdog/Kconfig
@@ -738,7 +738,7 @@
config EBC_C384_WDT
tristate "WinSystems EBC-C384 Watchdog Timer"
- depends on X86
+ depends on X86 && ISA
select WATCHDOG_CORE
help
Enables watchdog timer support for the watchdog timer on the
@@ -1475,6 +1475,32 @@
help
Hardware driver for the Mediatek/Ralink MT7621/8 SoC Watchdog Timer.
+config PIC32_WDT
+ tristate "Microchip PIC32 hardware watchdog"
+ select WATCHDOG_CORE
+ depends on MACH_PIC32
+ help
+ Watchdog driver for the built in watchdog hardware in a PIC32.
+
+ Configuration bits must be set appropriately for the watchdog to be
+ controlled by this driver.
+
+ To compile this driver as a loadable module, choose M here.
+ The module will be called pic32-wdt.
+
+config PIC32_DMT
+ tristate "Microchip PIC32 Deadman Timer"
+ select WATCHDOG_CORE
+ depends on MACH_PIC32
+ help
+ Watchdog driver for PIC32 instruction fetch counting timer. This specific
+ timer is typically be used in misson critical and safety critical
+ applications, where any single failure of the software functionality
+ and sequencing must be detected.
+
+ To compile this driver as a loadable module, choose M here.
+ The module will be called pic32-dmt.
+
# PARISC Architecture
# POWERPC Architecture
diff --git a/drivers/watchdog/Makefile b/drivers/watchdog/Makefile
index feb6270..9bde095 100644
--- a/drivers/watchdog/Makefile
+++ b/drivers/watchdog/Makefile
@@ -157,6 +157,8 @@
obj-$(CONFIG_RALINK_WDT) += rt2880_wdt.o
obj-$(CONFIG_IMGPDC_WDT) += imgpdc_wdt.o
obj-$(CONFIG_MT7621_WDT) += mt7621_wdt.o
+obj-$(CONFIG_PIC32_WDT) += pic32-wdt.o
+obj-$(CONFIG_PIC32_DMT) += pic32-dmt.o
# PARISC Architecture
diff --git a/drivers/watchdog/ebc-c384_wdt.c b/drivers/watchdog/ebc-c384_wdt.c
index 77fda0b..4b849b8 100644
--- a/drivers/watchdog/ebc-c384_wdt.c
+++ b/drivers/watchdog/ebc-c384_wdt.c
@@ -16,10 +16,10 @@
#include <linux/errno.h>
#include <linux/io.h>
#include <linux/ioport.h>
+#include <linux/isa.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
-#include <linux/platform_device.h>
#include <linux/types.h>
#include <linux/watchdog.h>
@@ -95,9 +95,8 @@
.identity = MODULE_NAME
};
-static int __init ebc_c384_wdt_probe(struct platform_device *pdev)
+static int ebc_c384_wdt_probe(struct device *dev, unsigned int id)
{
- struct device *dev = &pdev->dev;
struct watchdog_device *wdd;
if (!devm_request_region(dev, BASE_ADDR, ADDR_EXTENT, dev_name(dev))) {
@@ -122,61 +121,39 @@
dev_warn(dev, "Invalid timeout (%u seconds), using default (%u seconds)\n",
timeout, WATCHDOG_TIMEOUT);
- platform_set_drvdata(pdev, wdd);
+ dev_set_drvdata(dev, wdd);
return watchdog_register_device(wdd);
}
-static int ebc_c384_wdt_remove(struct platform_device *pdev)
+static int ebc_c384_wdt_remove(struct device *dev, unsigned int id)
{
- struct watchdog_device *wdd = platform_get_drvdata(pdev);
+ struct watchdog_device *wdd = dev_get_drvdata(dev);
watchdog_unregister_device(wdd);
return 0;
}
-static struct platform_driver ebc_c384_wdt_driver = {
+static struct isa_driver ebc_c384_wdt_driver = {
+ .probe = ebc_c384_wdt_probe,
.driver = {
.name = MODULE_NAME
},
.remove = ebc_c384_wdt_remove
};
-static struct platform_device *ebc_c384_wdt_device;
-
static int __init ebc_c384_wdt_init(void)
{
- int err;
-
if (!dmi_match(DMI_BOARD_NAME, "EBC-C384 SBC"))
return -ENODEV;
- ebc_c384_wdt_device = platform_device_alloc(MODULE_NAME, -1);
- if (!ebc_c384_wdt_device)
- return -ENOMEM;
-
- err = platform_device_add(ebc_c384_wdt_device);
- if (err)
- goto err_platform_device;
-
- err = platform_driver_probe(&ebc_c384_wdt_driver, ebc_c384_wdt_probe);
- if (err)
- goto err_platform_driver;
-
- return 0;
-
-err_platform_driver:
- platform_device_del(ebc_c384_wdt_device);
-err_platform_device:
- platform_device_put(ebc_c384_wdt_device);
- return err;
+ return isa_register_driver(&ebc_c384_wdt_driver, 1);
}
static void __exit ebc_c384_wdt_exit(void)
{
- platform_device_unregister(ebc_c384_wdt_device);
- platform_driver_unregister(&ebc_c384_wdt_driver);
+ isa_unregister_driver(&ebc_c384_wdt_driver);
}
module_init(ebc_c384_wdt_init);
@@ -185,4 +162,4 @@
MODULE_AUTHOR("William Breathitt Gray <vilhelm.gray@gmail.com>");
MODULE_DESCRIPTION("WinSystems EBC-C384 watchdog timer driver");
MODULE_LICENSE("GPL v2");
-MODULE_ALIAS("platform:" MODULE_NAME);
+MODULE_ALIAS("isa:" MODULE_NAME);
diff --git a/drivers/watchdog/pic32-dmt.c b/drivers/watchdog/pic32-dmt.c
new file mode 100644
index 0000000..962f58c
--- /dev/null
+++ b/drivers/watchdog/pic32-dmt.c
@@ -0,0 +1,257 @@
+/*
+ * PIC32 deadman timer driver
+ *
+ * Purna Chandra Mandal <purna.mandal@microchip.com>
+ * Copyright (c) 2016, Microchip Technology Inc.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+#include <linux/clk.h>
+#include <linux/device.h>
+#include <linux/err.h>
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+#include <linux/pm.h>
+#include <linux/watchdog.h>
+
+#include <asm/mach-pic32/pic32.h>
+
+/* Deadman Timer Regs */
+#define DMTCON_REG 0x00
+#define DMTPRECLR_REG 0x10
+#define DMTCLR_REG 0x20
+#define DMTSTAT_REG 0x30
+#define DMTCNT_REG 0x40
+#define DMTPSCNT_REG 0x60
+#define DMTPSINTV_REG 0x70
+
+/* Deadman Timer Regs fields */
+#define DMT_ON BIT(15)
+#define DMT_STEP1_KEY BIT(6)
+#define DMT_STEP2_KEY BIT(3)
+#define DMTSTAT_WINOPN BIT(0)
+#define DMTSTAT_EVENT BIT(5)
+#define DMTSTAT_BAD2 BIT(6)
+#define DMTSTAT_BAD1 BIT(7)
+
+/* Reset Control Register fields for watchdog */
+#define RESETCON_DMT_TIMEOUT BIT(5)
+
+struct pic32_dmt {
+ void __iomem *regs;
+ struct clk *clk;
+};
+
+static inline void dmt_enable(struct pic32_dmt *dmt)
+{
+ writel(DMT_ON, PIC32_SET(dmt->regs + DMTCON_REG));
+}
+
+static inline void dmt_disable(struct pic32_dmt *dmt)
+{
+ writel(DMT_ON, PIC32_CLR(dmt->regs + DMTCON_REG));
+ /*
+ * Cannot touch registers in the CPU cycle following clearing the
+ * ON bit.
+ */
+ nop();
+}
+
+static inline int dmt_bad_status(struct pic32_dmt *dmt)
+{
+ u32 val;
+
+ val = readl(dmt->regs + DMTSTAT_REG);
+ val &= (DMTSTAT_BAD1 | DMTSTAT_BAD2 | DMTSTAT_EVENT);
+ if (val)
+ return -EAGAIN;
+
+ return 0;
+}
+
+static inline int dmt_keepalive(struct pic32_dmt *dmt)
+{
+ u32 v;
+ u32 timeout = 500;
+
+ /* set pre-clear key */
+ writel(DMT_STEP1_KEY << 8, dmt->regs + DMTPRECLR_REG);
+
+ /* wait for DMT window to open */
+ while (--timeout) {
+ v = readl(dmt->regs + DMTSTAT_REG) & DMTSTAT_WINOPN;
+ if (v == DMTSTAT_WINOPN)
+ break;
+ }
+
+ /* apply key2 */
+ writel(DMT_STEP2_KEY, dmt->regs + DMTCLR_REG);
+
+ /* check whether keys are latched correctly */
+ return dmt_bad_status(dmt);
+}
+
+static inline u32 pic32_dmt_get_timeout_secs(struct pic32_dmt *dmt)
+{
+ unsigned long rate;
+
+ rate = clk_get_rate(dmt->clk);
+ if (rate)
+ return readl(dmt->regs + DMTPSCNT_REG) / rate;
+
+ return 0;
+}
+
+static inline u32 pic32_dmt_bootstatus(struct pic32_dmt *dmt)
+{
+ u32 v;
+ void __iomem *rst_base;
+
+ rst_base = ioremap(PIC32_BASE_RESET, 0x10);
+ if (!rst_base)
+ return 0;
+
+ v = readl(rst_base);
+
+ writel(RESETCON_DMT_TIMEOUT, PIC32_CLR(rst_base));
+
+ iounmap(rst_base);
+ return v & RESETCON_DMT_TIMEOUT;
+}
+
+static int pic32_dmt_start(struct watchdog_device *wdd)
+{
+ struct pic32_dmt *dmt = watchdog_get_drvdata(wdd);
+
+ dmt_enable(dmt);
+ return dmt_keepalive(dmt);
+}
+
+static int pic32_dmt_stop(struct watchdog_device *wdd)
+{
+ struct pic32_dmt *dmt = watchdog_get_drvdata(wdd);
+
+ dmt_disable(dmt);
+
+ return 0;
+}
+
+static int pic32_dmt_ping(struct watchdog_device *wdd)
+{
+ struct pic32_dmt *dmt = watchdog_get_drvdata(wdd);
+
+ return dmt_keepalive(dmt);
+}
+
+static const struct watchdog_ops pic32_dmt_fops = {
+ .owner = THIS_MODULE,
+ .start = pic32_dmt_start,
+ .stop = pic32_dmt_stop,
+ .ping = pic32_dmt_ping,
+};
+
+static const struct watchdog_info pic32_dmt_ident = {
+ .options = WDIOF_KEEPALIVEPING |
+ WDIOF_MAGICCLOSE,
+ .identity = "PIC32 Deadman Timer",
+};
+
+static struct watchdog_device pic32_dmt_wdd = {
+ .info = &pic32_dmt_ident,
+ .ops = &pic32_dmt_fops,
+};
+
+static int pic32_dmt_probe(struct platform_device *pdev)
+{
+ int ret;
+ struct pic32_dmt *dmt;
+ struct resource *mem;
+ struct watchdog_device *wdd = &pic32_dmt_wdd;
+
+ dmt = devm_kzalloc(&pdev->dev, sizeof(*dmt), GFP_KERNEL);
+ if (IS_ERR(dmt))
+ return PTR_ERR(dmt);
+
+ mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ dmt->regs = devm_ioremap_resource(&pdev->dev, mem);
+ if (IS_ERR(dmt->regs))
+ return PTR_ERR(dmt->regs);
+
+ dmt->clk = devm_clk_get(&pdev->dev, NULL);
+ if (IS_ERR(dmt->clk)) {
+ dev_err(&pdev->dev, "clk not found\n");
+ return PTR_ERR(dmt->clk);
+ }
+
+ ret = clk_prepare_enable(dmt->clk);
+ if (ret)
+ return ret;
+
+ wdd->timeout = pic32_dmt_get_timeout_secs(dmt);
+ if (!wdd->timeout) {
+ dev_err(&pdev->dev,
+ "failed to read watchdog register timeout\n");
+ ret = -EINVAL;
+ goto out_disable_clk;
+ }
+
+ dev_info(&pdev->dev, "timeout %d\n", wdd->timeout);
+
+ wdd->bootstatus = pic32_dmt_bootstatus(dmt) ? WDIOF_CARDRESET : 0;
+
+ watchdog_set_nowayout(wdd, WATCHDOG_NOWAYOUT);
+ watchdog_set_drvdata(wdd, dmt);
+
+ ret = watchdog_register_device(wdd);
+ if (ret) {
+ dev_err(&pdev->dev, "watchdog register failed, err %d\n", ret);
+ goto out_disable_clk;
+ }
+
+ platform_set_drvdata(pdev, wdd);
+ return 0;
+
+out_disable_clk:
+ clk_disable_unprepare(dmt->clk);
+ return ret;
+}
+
+static int pic32_dmt_remove(struct platform_device *pdev)
+{
+ struct watchdog_device *wdd = platform_get_drvdata(pdev);
+ struct pic32_dmt *dmt = watchdog_get_drvdata(wdd);
+
+ watchdog_unregister_device(wdd);
+ clk_disable_unprepare(dmt->clk);
+
+ return 0;
+}
+
+static const struct of_device_id pic32_dmt_of_ids[] = {
+ { .compatible = "microchip,pic32mzda-dmt",},
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, pic32_dmt_of_ids);
+
+static struct platform_driver pic32_dmt_driver = {
+ .probe = pic32_dmt_probe,
+ .remove = pic32_dmt_remove,
+ .driver = {
+ .name = "pic32-dmt",
+ .owner = THIS_MODULE,
+ .of_match_table = of_match_ptr(pic32_dmt_of_ids),
+ }
+};
+
+module_platform_driver(pic32_dmt_driver);
+
+MODULE_AUTHOR("Purna Chandra Mandal <purna.mandal@microchip.com>");
+MODULE_DESCRIPTION("Microchip PIC32 DMT Driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/watchdog/pic32-wdt.c b/drivers/watchdog/pic32-wdt.c
new file mode 100644
index 0000000..6047aa8
--- /dev/null
+++ b/drivers/watchdog/pic32-wdt.c
@@ -0,0 +1,263 @@
+/*
+ * PIC32 watchdog driver
+ *
+ * Joshua Henderson <joshua.henderson@microchip.com>
+ * Copyright (c) 2016, Microchip Technology Inc.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+#include <linux/clk.h>
+#include <linux/device.h>
+#include <linux/err.h>
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+#include <linux/pm.h>
+#include <linux/watchdog.h>
+
+#include <asm/mach-pic32/pic32.h>
+
+/* Watchdog Timer Registers */
+#define WDTCON_REG 0x00
+
+/* Watchdog Timer Control Register fields */
+#define WDTCON_WIN_EN BIT(0)
+#define WDTCON_RMCS_MASK 0x0003
+#define WDTCON_RMCS_SHIFT 0x0006
+#define WDTCON_RMPS_MASK 0x001F
+#define WDTCON_RMPS_SHIFT 0x0008
+#define WDTCON_ON BIT(15)
+#define WDTCON_CLR_KEY 0x5743
+
+/* Reset Control Register fields for watchdog */
+#define RESETCON_TIMEOUT_IDLE BIT(2)
+#define RESETCON_TIMEOUT_SLEEP BIT(3)
+#define RESETCON_WDT_TIMEOUT BIT(4)
+
+struct pic32_wdt {
+ void __iomem *regs;
+ void __iomem *rst_base;
+ struct clk *clk;
+};
+
+static inline bool pic32_wdt_is_win_enabled(struct pic32_wdt *wdt)
+{
+ return !!(readl(wdt->regs + WDTCON_REG) & WDTCON_WIN_EN);
+}
+
+static inline u32 pic32_wdt_get_post_scaler(struct pic32_wdt *wdt)
+{
+ u32 v = readl(wdt->regs + WDTCON_REG);
+
+ return (v >> WDTCON_RMPS_SHIFT) & WDTCON_RMPS_MASK;
+}
+
+static inline u32 pic32_wdt_get_clk_id(struct pic32_wdt *wdt)
+{
+ u32 v = readl(wdt->regs + WDTCON_REG);
+
+ return (v >> WDTCON_RMCS_SHIFT) & WDTCON_RMCS_MASK;
+}
+
+static int pic32_wdt_bootstatus(struct pic32_wdt *wdt)
+{
+ u32 v = readl(wdt->rst_base);
+
+ writel(RESETCON_WDT_TIMEOUT, PIC32_CLR(wdt->rst_base));
+
+ return v & RESETCON_WDT_TIMEOUT;
+}
+
+static u32 pic32_wdt_get_timeout_secs(struct pic32_wdt *wdt, struct device *dev)
+{
+ unsigned long rate;
+ u32 period, ps, terminal;
+
+ rate = clk_get_rate(wdt->clk);
+
+ dev_dbg(dev, "wdt: clk_id %d, clk_rate %lu (prescale)\n",
+ pic32_wdt_get_clk_id(wdt), rate);
+
+ /* default, prescaler of 32 (i.e. div-by-32) is implicit. */
+ rate >>= 5;
+ if (!rate)
+ return 0;
+
+ /* calculate terminal count from postscaler. */
+ ps = pic32_wdt_get_post_scaler(wdt);
+ terminal = BIT(ps);
+
+ /* find time taken (in secs) to reach terminal count */
+ period = terminal / rate;
+ dev_dbg(dev,
+ "wdt: clk_rate %lu (postscale) / terminal %d, timeout %dsec\n",
+ rate, terminal, period);
+
+ return period;
+}
+
+static void pic32_wdt_keepalive(struct pic32_wdt *wdt)
+{
+ /* write key through single half-word */
+ writew(WDTCON_CLR_KEY, wdt->regs + WDTCON_REG + 2);
+}
+
+static int pic32_wdt_start(struct watchdog_device *wdd)
+{
+ struct pic32_wdt *wdt = watchdog_get_drvdata(wdd);
+
+ writel(WDTCON_ON, PIC32_SET(wdt->regs + WDTCON_REG));
+ pic32_wdt_keepalive(wdt);
+
+ return 0;
+}
+
+static int pic32_wdt_stop(struct watchdog_device *wdd)
+{
+ struct pic32_wdt *wdt = watchdog_get_drvdata(wdd);
+
+ writel(WDTCON_ON, PIC32_CLR(wdt->regs + WDTCON_REG));
+
+ /*
+ * Cannot touch registers in the CPU cycle following clearing the
+ * ON bit.
+ */
+ nop();
+
+ return 0;
+}
+
+static int pic32_wdt_ping(struct watchdog_device *wdd)
+{
+ struct pic32_wdt *wdt = watchdog_get_drvdata(wdd);
+
+ pic32_wdt_keepalive(wdt);
+
+ return 0;
+}
+
+static const struct watchdog_ops pic32_wdt_fops = {
+ .owner = THIS_MODULE,
+ .start = pic32_wdt_start,
+ .stop = pic32_wdt_stop,
+ .ping = pic32_wdt_ping,
+};
+
+static const struct watchdog_info pic32_wdt_ident = {
+ .options = WDIOF_KEEPALIVEPING |
+ WDIOF_MAGICCLOSE | WDIOF_CARDRESET,
+ .identity = "PIC32 Watchdog",
+};
+
+static struct watchdog_device pic32_wdd = {
+ .info = &pic32_wdt_ident,
+ .ops = &pic32_wdt_fops,
+};
+
+static const struct of_device_id pic32_wdt_dt_ids[] = {
+ { .compatible = "microchip,pic32mzda-wdt", },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, pic32_wdt_dt_ids);
+
+static int pic32_wdt_drv_probe(struct platform_device *pdev)
+{
+ int ret;
+ struct watchdog_device *wdd = &pic32_wdd;
+ struct pic32_wdt *wdt;
+ struct resource *mem;
+
+ wdt = devm_kzalloc(&pdev->dev, sizeof(*wdt), GFP_KERNEL);
+ if (IS_ERR(wdt))
+ return PTR_ERR(wdt);
+
+ mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ wdt->regs = devm_ioremap_resource(&pdev->dev, mem);
+ if (IS_ERR(wdt->regs))
+ return PTR_ERR(wdt->regs);
+
+ wdt->rst_base = devm_ioremap(&pdev->dev, PIC32_BASE_RESET, 0x10);
+ if (IS_ERR(wdt->rst_base))
+ return PTR_ERR(wdt->rst_base);
+
+ wdt->clk = devm_clk_get(&pdev->dev, NULL);
+ if (IS_ERR(wdt->clk)) {
+ dev_err(&pdev->dev, "clk not found\n");
+ return PTR_ERR(wdt->clk);
+ }
+
+ ret = clk_prepare_enable(wdt->clk);
+ if (ret) {
+ dev_err(&pdev->dev, "clk enable failed\n");
+ return ret;
+ }
+
+ if (pic32_wdt_is_win_enabled(wdt)) {
+ dev_err(&pdev->dev, "windowed-clear mode is not supported.\n");
+ ret = -ENODEV;
+ goto out_disable_clk;
+ }
+
+ wdd->timeout = pic32_wdt_get_timeout_secs(wdt, &pdev->dev);
+ if (!wdd->timeout) {
+ dev_err(&pdev->dev,
+ "failed to read watchdog register timeout\n");
+ ret = -EINVAL;
+ goto out_disable_clk;
+ }
+
+ dev_info(&pdev->dev, "timeout %d\n", wdd->timeout);
+
+ wdd->bootstatus = pic32_wdt_bootstatus(wdt) ? WDIOF_CARDRESET : 0;
+
+ watchdog_set_nowayout(wdd, WATCHDOG_NOWAYOUT);
+ watchdog_set_drvdata(wdd, wdt);
+
+ ret = watchdog_register_device(wdd);
+ if (ret) {
+ dev_err(&pdev->dev, "watchdog register failed, err %d\n", ret);
+ goto out_disable_clk;
+ }
+
+ platform_set_drvdata(pdev, wdd);
+
+ return 0;
+
+out_disable_clk:
+ clk_disable_unprepare(wdt->clk);
+
+ return ret;
+}
+
+static int pic32_wdt_drv_remove(struct platform_device *pdev)
+{
+ struct watchdog_device *wdd = platform_get_drvdata(pdev);
+ struct pic32_wdt *wdt = watchdog_get_drvdata(wdd);
+
+ watchdog_unregister_device(wdd);
+ clk_disable_unprepare(wdt->clk);
+
+ return 0;
+}
+
+static struct platform_driver pic32_wdt_driver = {
+ .probe = pic32_wdt_drv_probe,
+ .remove = pic32_wdt_drv_remove,
+ .driver = {
+ .name = "pic32-wdt",
+ .owner = THIS_MODULE,
+ .of_match_table = of_match_ptr(pic32_wdt_dt_ids),
+ }
+};
+
+module_platform_driver(pic32_wdt_driver);
+
+MODULE_AUTHOR("Joshua Henderson <joshua.henderson@microchip.com>");
+MODULE_DESCRIPTION("Microchip PIC32 Watchdog Timer");
+MODULE_LICENSE("GPL");
diff --git a/drivers/xen/Makefile b/drivers/xen/Makefile
index 9b7a35c..030e91b 100644
--- a/drivers/xen/Makefile
+++ b/drivers/xen/Makefile
@@ -8,6 +8,7 @@
CFLAGS_features.o := $(nostackp)
CFLAGS_efi.o += -fshort-wchar
+LDFLAGS += $(call ld-option, --no-wchar-size-warning)
dom0-$(CONFIG_PCI) += pci.o
dom0-$(CONFIG_USB_SUPPORT) += dbgp.o
diff --git a/drivers/xen/events/events_base.c b/drivers/xen/events/events_base.c
index cb7138c..71d49a9 100644
--- a/drivers/xen/events/events_base.c
+++ b/drivers/xen/events/events_base.c
@@ -487,7 +487,8 @@
if (!VALID_EVTCHN(evtchn))
return;
- if (unlikely(irqd_is_setaffinity_pending(data))) {
+ if (unlikely(irqd_is_setaffinity_pending(data)) &&
+ likely(!irqd_irq_disabled(data))) {
int masked = test_and_set_mask(evtchn);
clear_evtchn(evtchn);
@@ -1370,7 +1371,8 @@
if (!VALID_EVTCHN(evtchn))
return;
- if (unlikely(irqd_is_setaffinity_pending(data))) {
+ if (unlikely(irqd_is_setaffinity_pending(data)) &&
+ likely(!irqd_irq_disabled(data))) {
int masked = test_and_set_mask(evtchn);
clear_evtchn(evtchn);
diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c
index dc49538..6793957 100644
--- a/drivers/xen/gntdev.c
+++ b/drivers/xen/gntdev.c
@@ -748,7 +748,7 @@
return rc;
}
-#define GNTDEV_COPY_BATCH 24
+#define GNTDEV_COPY_BATCH 16
struct gntdev_copy_batch {
struct gnttab_copy ops[GNTDEV_COPY_BATCH];
diff --git a/fs/Kconfig.binfmt b/fs/Kconfig.binfmt
index 2d0cbbd..72c0335 100644
--- a/fs/Kconfig.binfmt
+++ b/fs/Kconfig.binfmt
@@ -1,6 +1,7 @@
config BINFMT_ELF
bool "Kernel support for ELF binaries"
depends on MMU && (BROKEN || !FRV)
+ select ELFCORE
default y
---help---
ELF (Executable and Linkable Format) is a format for libraries and
@@ -26,6 +27,7 @@
config COMPAT_BINFMT_ELF
bool
depends on COMPAT && BINFMT_ELF
+ select ELFCORE
config ARCH_BINFMT_ELF_STATE
bool
@@ -34,6 +36,7 @@
bool "Kernel support for FDPIC ELF binaries"
default y
depends on (FRV || BLACKFIN || (SUPERH32 && !MMU) || C6X)
+ select ELFCORE
help
ELF FDPIC binaries are based on ELF, but allow the individual load
segments of a binary to be located in memory independently of each
@@ -43,6 +46,11 @@
It is also possible to run FDPIC ELF binaries on MMU linux also.
+config ELFCORE
+ bool
+ help
+ This option enables kernel/elfcore.o.
+
config CORE_DUMP_DEFAULT_ELF_HEADERS
bool "Write ELF core dumps with partial segments"
default y
diff --git a/fs/aio.c b/fs/aio.c
index a6deaa7..fb8e45b 100644
--- a/fs/aio.c
+++ b/fs/aio.c
@@ -496,7 +496,12 @@
ctx->mmap_size = nr_pages * PAGE_SIZE;
pr_debug("attempting mmap of %lu bytes\n", ctx->mmap_size);
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem)) {
+ ctx->mmap_size = 0;
+ aio_free_ring(ctx);
+ return -EINTR;
+ }
+
ctx->mmap_base = do_mmap_pgoff(ctx->aio_ring_file, 0, ctx->mmap_size,
PROT_READ | PROT_WRITE,
MAP_SHARED, 0, &unused);
diff --git a/fs/befs/datastream.c b/fs/befs/datastream.c
index dde0b79..af1bc19 100644
--- a/fs/befs/datastream.c
+++ b/fs/befs/datastream.c
@@ -48,7 +48,7 @@
befs_read_datastream(struct super_block *sb, const befs_data_stream *ds,
befs_off_t pos, uint * off)
{
- struct buffer_head *bh = NULL;
+ struct buffer_head *bh;
befs_block_run run;
befs_blocknr_t block; /* block coresponding to pos */
@@ -127,7 +127,7 @@
{
befs_off_t bytes_read = 0; /* bytes readed */
u16 plen;
- struct buffer_head *bh = NULL;
+ struct buffer_head *bh;
befs_debug(sb, "---> %s length: %llu", __func__, len);
while (bytes_read < len) {
@@ -429,7 +429,7 @@
struct buffer_head *dbl_indir_block;
struct buffer_head *indir_block;
befs_block_run indir_run;
- befs_disk_inode_addr *iaddr_array = NULL;
+ befs_disk_inode_addr *iaddr_array;
struct befs_sb_info *befs_sb = BEFS_SB(sb);
befs_blocknr_t indir_start_blk =
@@ -488,7 +488,6 @@
iaddr_array = (befs_disk_inode_addr *) dbl_indir_block->b_data;
indir_run = fsrun_to_cpu(sb, iaddr_array[dbl_block_indx]);
brelse(dbl_indir_block);
- iaddr_array = NULL;
/* Read indirect block */
which_block = indir_indx / befs_iaddrs_per_block(sb);
@@ -513,7 +512,6 @@
iaddr_array = (befs_disk_inode_addr *) indir_block->b_data;
*run = fsrun_to_cpu(sb, iaddr_array[block_indx]);
brelse(indir_block);
- iaddr_array = NULL;
blockno_at_run_start = indir_start_blk;
blockno_at_run_start += diblklen * dblindir_indx;
diff --git a/fs/befs/io.c b/fs/befs/io.c
index 7a5b4ec..523c8af 100644
--- a/fs/befs/io.c
+++ b/fs/befs/io.c
@@ -26,7 +26,7 @@
struct buffer_head *
befs_bread_iaddr(struct super_block *sb, befs_inode_addr iaddr)
{
- struct buffer_head *bh = NULL;
+ struct buffer_head *bh;
befs_blocknr_t block = 0;
struct befs_sb_info *befs_sb = BEFS_SB(sb);
@@ -63,7 +63,7 @@
struct buffer_head *
befs_bread(struct super_block *sb, befs_blocknr_t block)
{
- struct buffer_head *bh = NULL;
+ struct buffer_head *bh;
befs_debug(sb, "---> Enter %s %lu", __func__, (unsigned long)block);
diff --git a/fs/befs/linuxvfs.c b/fs/befs/linuxvfs.c
index 71112aa..7da05b1 100644
--- a/fs/befs/linuxvfs.c
+++ b/fs/befs/linuxvfs.c
@@ -155,7 +155,7 @@
static struct dentry *
befs_lookup(struct inode *dir, struct dentry *dentry, unsigned int flags)
{
- struct inode *inode = NULL;
+ struct inode *inode;
struct super_block *sb = dir->i_sb;
const befs_data_stream *ds = &BEFS_I(dir)->i_data.ds;
befs_off_t offset;
@@ -294,10 +294,10 @@
static struct inode *befs_iget(struct super_block *sb, unsigned long ino)
{
- struct buffer_head *bh = NULL;
- befs_inode *raw_inode = NULL;
+ struct buffer_head *bh;
+ befs_inode *raw_inode;
struct befs_sb_info *befs_sb = BEFS_SB(sb);
- struct befs_inode_info *befs_ino = NULL;
+ struct befs_inode_info *befs_ino;
struct inode *inode;
long ret = -EIO;
diff --git a/fs/binfmt_aout.c b/fs/binfmt_aout.c
index 4c55668..2fab9f1 100644
--- a/fs/binfmt_aout.c
+++ b/fs/binfmt_aout.c
@@ -297,7 +297,10 @@
}
if (!bprm->file->f_op->mmap||((fd_offset & ~PAGE_MASK) != 0)) {
- vm_brk(N_TXTADDR(ex), ex.a_text+ex.a_data);
+ error = vm_brk(N_TXTADDR(ex), ex.a_text+ex.a_data);
+ if (IS_ERR_VALUE(error))
+ return error;
+
read_code(bprm->file, N_TXTADDR(ex), fd_offset,
ex.a_text + ex.a_data);
goto beyond_if;
@@ -378,8 +381,10 @@
"N_TXTOFF is not page aligned. Please convert library: %pD\n",
file);
}
- vm_brk(start_addr, ex.a_text + ex.a_data + ex.a_bss);
-
+ retval = vm_brk(start_addr, ex.a_text + ex.a_data + ex.a_bss);
+ if (IS_ERR_VALUE(retval))
+ goto out;
+
read_code(file, start_addr, N_TXTOFF(ex),
ex.a_text + ex.a_data);
retval = 0;
diff --git a/fs/binfmt_elf.c b/fs/binfmt_elf.c
index 56224ff..938fc4e 100644
--- a/fs/binfmt_elf.c
+++ b/fs/binfmt_elf.c
@@ -1176,8 +1176,11 @@
len = ELF_PAGESTART(eppnt->p_filesz + eppnt->p_vaddr +
ELF_MIN_ALIGN - 1);
bss = eppnt->p_memsz + eppnt->p_vaddr;
- if (bss > len)
- vm_brk(len, bss - len);
+ if (bss > len) {
+ error = vm_brk(len, bss - len);
+ if (BAD_ADDR(error))
+ goto out_free_ph;
+ }
error = 0;
out_free_ph:
diff --git a/fs/block_dev.c b/fs/block_dev.c
index a063d4d..1089dbf 100644
--- a/fs/block_dev.c
+++ b/fs/block_dev.c
@@ -29,6 +29,7 @@
#include <linux/log2.h>
#include <linux/cleancache.h>
#include <linux/dax.h>
+#include <linux/badblocks.h>
#include <asm/uaccess.h>
#include "internal.h"
@@ -1159,6 +1160,33 @@
}
EXPORT_SYMBOL(bd_set_size);
+static bool blkdev_dax_capable(struct block_device *bdev)
+{
+ struct gendisk *disk = bdev->bd_disk;
+
+ if (!disk->fops->direct_access || !IS_ENABLED(CONFIG_FS_DAX))
+ return false;
+
+ /*
+ * If the partition is not aligned on a page boundary, we can't
+ * do dax I/O to it.
+ */
+ if ((bdev->bd_part->start_sect % (PAGE_SIZE / 512))
+ || (bdev->bd_part->nr_sects % (PAGE_SIZE / 512)))
+ return false;
+
+ /*
+ * If the device has known bad blocks, force all I/O through the
+ * driver / page cache.
+ *
+ * TODO: support finer grained dax error handling
+ */
+ if (disk->bb && disk->bb->count)
+ return false;
+
+ return true;
+}
+
static void __blkdev_put(struct block_device *bdev, fmode_t mode, int for_part);
/*
@@ -1720,79 +1748,13 @@
.is_dirty_writeback = buffer_check_dirty_writeback,
};
-#ifdef CONFIG_FS_DAX
-/*
- * In the raw block case we do not need to contend with truncation nor
- * unwritten file extents. Without those concerns there is no need for
- * additional locking beyond the mmap_sem context that these routines
- * are already executing under.
- *
- * Note, there is no protection if the block device is dynamically
- * resized (partition grow/shrink) during a fault. A stable block device
- * size is already not enforced in the blkdev_direct_IO path.
- *
- * For DAX, it is the responsibility of the block device driver to
- * ensure the whole-disk device size is stable while requests are in
- * flight.
- *
- * Finally, unlike the filemap_page_mkwrite() case there is no
- * filesystem superblock to sync against freezing. We still include a
- * pfn_mkwrite callback for dax drivers to receive write fault
- * notifications.
- */
-static int blkdev_dax_fault(struct vm_area_struct *vma, struct vm_fault *vmf)
-{
- return __dax_fault(vma, vmf, blkdev_get_block, NULL);
-}
-
-static int blkdev_dax_pfn_mkwrite(struct vm_area_struct *vma,
- struct vm_fault *vmf)
-{
- return dax_pfn_mkwrite(vma, vmf);
-}
-
-static int blkdev_dax_pmd_fault(struct vm_area_struct *vma, unsigned long addr,
- pmd_t *pmd, unsigned int flags)
-{
- return __dax_pmd_fault(vma, addr, pmd, flags, blkdev_get_block, NULL);
-}
-
-static const struct vm_operations_struct blkdev_dax_vm_ops = {
- .fault = blkdev_dax_fault,
- .pmd_fault = blkdev_dax_pmd_fault,
- .pfn_mkwrite = blkdev_dax_pfn_mkwrite,
-};
-
-static const struct vm_operations_struct blkdev_default_vm_ops = {
- .fault = filemap_fault,
- .map_pages = filemap_map_pages,
-};
-
-static int blkdev_mmap(struct file *file, struct vm_area_struct *vma)
-{
- struct inode *bd_inode = bdev_file_inode(file);
-
- file_accessed(file);
- if (IS_DAX(bd_inode)) {
- vma->vm_ops = &blkdev_dax_vm_ops;
- vma->vm_flags |= VM_MIXEDMAP | VM_HUGEPAGE;
- } else {
- vma->vm_ops = &blkdev_default_vm_ops;
- }
-
- return 0;
-}
-#else
-#define blkdev_mmap generic_file_mmap
-#endif
-
const struct file_operations def_blk_fops = {
.open = blkdev_open,
.release = blkdev_close,
.llseek = block_llseek,
.read_iter = blkdev_read_iter,
.write_iter = blkdev_write_iter,
- .mmap = blkdev_mmap,
+ .mmap = generic_file_mmap,
.fsync = blkdev_fsync,
.unlocked_ioctl = block_ioctl,
#ifdef CONFIG_COMPAT
diff --git a/fs/btrfs/backref.c b/fs/btrfs/backref.c
index 80e8472..d309018 100644
--- a/fs/btrfs/backref.c
+++ b/fs/btrfs/backref.c
@@ -1991,7 +1991,7 @@
ifp = kmalloc(sizeof(*ifp), GFP_NOFS);
if (!ifp) {
- kfree(fspath);
+ vfree(fspath);
return ERR_PTR(-ENOMEM);
}
diff --git a/fs/btrfs/btrfs_inode.h b/fs/btrfs/btrfs_inode.h
index 61205e3..1da5753 100644
--- a/fs/btrfs/btrfs_inode.h
+++ b/fs/btrfs/btrfs_inode.h
@@ -196,6 +196,16 @@
struct list_head delayed_iput;
long delayed_iput_count;
+ /*
+ * To avoid races between lockless (i_mutex not held) direct IO writes
+ * and concurrent fsync requests. Direct IO writes must acquire read
+ * access on this semaphore for creating an extent map and its
+ * corresponding ordered extent. The fast fsync path must acquire write
+ * access on this semaphore before it collects ordered extents and
+ * extent maps.
+ */
+ struct rw_semaphore dio_sem;
+
struct inode vfs_inode;
};
diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c
index ff61a41..658c39b 100644
--- a/fs/btrfs/compression.c
+++ b/fs/btrfs/compression.c
@@ -743,8 +743,11 @@
static struct {
struct list_head idle_ws;
spinlock_t ws_lock;
- int num_ws;
- atomic_t alloc_ws;
+ /* Number of free workspaces */
+ int free_ws;
+ /* Total number of allocated workspaces */
+ atomic_t total_ws;
+ /* Waiters for a free workspace */
wait_queue_head_t ws_wait;
} btrfs_comp_ws[BTRFS_COMPRESS_TYPES];
@@ -758,16 +761,34 @@
int i;
for (i = 0; i < BTRFS_COMPRESS_TYPES; i++) {
+ struct list_head *workspace;
+
INIT_LIST_HEAD(&btrfs_comp_ws[i].idle_ws);
spin_lock_init(&btrfs_comp_ws[i].ws_lock);
- atomic_set(&btrfs_comp_ws[i].alloc_ws, 0);
+ atomic_set(&btrfs_comp_ws[i].total_ws, 0);
init_waitqueue_head(&btrfs_comp_ws[i].ws_wait);
+
+ /*
+ * Preallocate one workspace for each compression type so
+ * we can guarantee forward progress in the worst case
+ */
+ workspace = btrfs_compress_op[i]->alloc_workspace();
+ if (IS_ERR(workspace)) {
+ printk(KERN_WARNING
+ "BTRFS: cannot preallocate compression workspace, will try later");
+ } else {
+ atomic_set(&btrfs_comp_ws[i].total_ws, 1);
+ btrfs_comp_ws[i].free_ws = 1;
+ list_add(workspace, &btrfs_comp_ws[i].idle_ws);
+ }
}
}
/*
- * this finds an available workspace or allocates a new one
- * ERR_PTR is returned if things go bad.
+ * This finds an available workspace or allocates a new one.
+ * If it's not possible to allocate a new one, waits until there's one.
+ * Preallocation makes a forward progress guarantees and we do not return
+ * errors.
*/
static struct list_head *find_workspace(int type)
{
@@ -777,36 +798,58 @@
struct list_head *idle_ws = &btrfs_comp_ws[idx].idle_ws;
spinlock_t *ws_lock = &btrfs_comp_ws[idx].ws_lock;
- atomic_t *alloc_ws = &btrfs_comp_ws[idx].alloc_ws;
+ atomic_t *total_ws = &btrfs_comp_ws[idx].total_ws;
wait_queue_head_t *ws_wait = &btrfs_comp_ws[idx].ws_wait;
- int *num_ws = &btrfs_comp_ws[idx].num_ws;
+ int *free_ws = &btrfs_comp_ws[idx].free_ws;
again:
spin_lock(ws_lock);
if (!list_empty(idle_ws)) {
workspace = idle_ws->next;
list_del(workspace);
- (*num_ws)--;
+ (*free_ws)--;
spin_unlock(ws_lock);
return workspace;
}
- if (atomic_read(alloc_ws) > cpus) {
+ if (atomic_read(total_ws) > cpus) {
DEFINE_WAIT(wait);
spin_unlock(ws_lock);
prepare_to_wait(ws_wait, &wait, TASK_UNINTERRUPTIBLE);
- if (atomic_read(alloc_ws) > cpus && !*num_ws)
+ if (atomic_read(total_ws) > cpus && !*free_ws)
schedule();
finish_wait(ws_wait, &wait);
goto again;
}
- atomic_inc(alloc_ws);
+ atomic_inc(total_ws);
spin_unlock(ws_lock);
workspace = btrfs_compress_op[idx]->alloc_workspace();
if (IS_ERR(workspace)) {
- atomic_dec(alloc_ws);
+ atomic_dec(total_ws);
wake_up(ws_wait);
+
+ /*
+ * Do not return the error but go back to waiting. There's a
+ * workspace preallocated for each type and the compression
+ * time is bounded so we get to a workspace eventually. This
+ * makes our caller's life easier.
+ *
+ * To prevent silent and low-probability deadlocks (when the
+ * initial preallocation fails), check if there are any
+ * workspaces at all.
+ */
+ if (atomic_read(total_ws) == 0) {
+ static DEFINE_RATELIMIT_STATE(_rs,
+ /* once per minute */ 60 * HZ,
+ /* no burst */ 1);
+
+ if (__ratelimit(&_rs)) {
+ printk(KERN_WARNING
+ "no compression workspaces, low memory, retrying");
+ }
+ }
+ goto again;
}
return workspace;
}
@@ -820,21 +863,21 @@
int idx = type - 1;
struct list_head *idle_ws = &btrfs_comp_ws[idx].idle_ws;
spinlock_t *ws_lock = &btrfs_comp_ws[idx].ws_lock;
- atomic_t *alloc_ws = &btrfs_comp_ws[idx].alloc_ws;
+ atomic_t *total_ws = &btrfs_comp_ws[idx].total_ws;
wait_queue_head_t *ws_wait = &btrfs_comp_ws[idx].ws_wait;
- int *num_ws = &btrfs_comp_ws[idx].num_ws;
+ int *free_ws = &btrfs_comp_ws[idx].free_ws;
spin_lock(ws_lock);
- if (*num_ws < num_online_cpus()) {
+ if (*free_ws < num_online_cpus()) {
list_add(workspace, idle_ws);
- (*num_ws)++;
+ (*free_ws)++;
spin_unlock(ws_lock);
goto wake;
}
spin_unlock(ws_lock);
btrfs_compress_op[idx]->free_workspace(workspace);
- atomic_dec(alloc_ws);
+ atomic_dec(total_ws);
wake:
/*
* Make sure counter is updated before we wake up waiters.
@@ -857,7 +900,7 @@
workspace = btrfs_comp_ws[i].idle_ws.next;
list_del(workspace);
btrfs_compress_op[i]->free_workspace(workspace);
- atomic_dec(&btrfs_comp_ws[i].alloc_ws);
+ atomic_dec(&btrfs_comp_ws[i].total_ws);
}
}
}
@@ -894,8 +937,6 @@
int ret;
workspace = find_workspace(type);
- if (IS_ERR(workspace))
- return PTR_ERR(workspace);
ret = btrfs_compress_op[type-1]->compress_pages(workspace, mapping,
start, len, pages,
@@ -930,8 +971,6 @@
int ret;
workspace = find_workspace(type);
- if (IS_ERR(workspace))
- return PTR_ERR(workspace);
ret = btrfs_compress_op[type-1]->decompress_biovec(workspace, pages_in,
disk_start,
@@ -952,8 +991,6 @@
int ret;
workspace = find_workspace(type);
- if (IS_ERR(workspace))
- return PTR_ERR(workspace);
ret = btrfs_compress_op[type-1]->decompress(workspace, data_in,
dest_page, start_byte,
diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c
index ec7928a..decd0a3 100644
--- a/fs/btrfs/ctree.c
+++ b/fs/btrfs/ctree.c
@@ -1011,7 +1011,7 @@
return ret;
if (refs == 0) {
ret = -EROFS;
- btrfs_std_error(root->fs_info, ret, NULL);
+ btrfs_handle_fs_error(root->fs_info, ret, NULL);
return ret;
}
} else {
@@ -1928,7 +1928,7 @@
child = read_node_slot(root, mid, 0);
if (!child) {
ret = -EROFS;
- btrfs_std_error(root->fs_info, ret, NULL);
+ btrfs_handle_fs_error(root->fs_info, ret, NULL);
goto enospc;
}
@@ -2031,7 +2031,7 @@
*/
if (!left) {
ret = -EROFS;
- btrfs_std_error(root->fs_info, ret, NULL);
+ btrfs_handle_fs_error(root->fs_info, ret, NULL);
goto enospc;
}
wret = balance_node_right(trans, root, mid, left);
diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 84a6a5b..ddcc58f 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -33,6 +33,7 @@
#include <asm/kmap_types.h>
#include <linux/pagemap.h>
#include <linux/btrfs.h>
+#include <linux/btrfs_tree.h>
#include <linux/workqueue.h>
#include <linux/security.h>
#include <linux/sizes.h>
@@ -64,98 +65,6 @@
#define BTRFS_COMPAT_EXTENT_TREE_V0
-/* holds pointers to all of the tree roots */
-#define BTRFS_ROOT_TREE_OBJECTID 1ULL
-
-/* stores information about which extents are in use, and reference counts */
-#define BTRFS_EXTENT_TREE_OBJECTID 2ULL
-
-/*
- * chunk tree stores translations from logical -> physical block numbering
- * the super block points to the chunk tree
- */
-#define BTRFS_CHUNK_TREE_OBJECTID 3ULL
-
-/*
- * stores information about which areas of a given device are in use.
- * one per device. The tree of tree roots points to the device tree
- */
-#define BTRFS_DEV_TREE_OBJECTID 4ULL
-
-/* one per subvolume, storing files and directories */
-#define BTRFS_FS_TREE_OBJECTID 5ULL
-
-/* directory objectid inside the root tree */
-#define BTRFS_ROOT_TREE_DIR_OBJECTID 6ULL
-
-/* holds checksums of all the data extents */
-#define BTRFS_CSUM_TREE_OBJECTID 7ULL
-
-/* holds quota configuration and tracking */
-#define BTRFS_QUOTA_TREE_OBJECTID 8ULL
-
-/* for storing items that use the BTRFS_UUID_KEY* types */
-#define BTRFS_UUID_TREE_OBJECTID 9ULL
-
-/* tracks free space in block groups. */
-#define BTRFS_FREE_SPACE_TREE_OBJECTID 10ULL
-
-/* device stats in the device tree */
-#define BTRFS_DEV_STATS_OBJECTID 0ULL
-
-/* for storing balance parameters in the root tree */
-#define BTRFS_BALANCE_OBJECTID -4ULL
-
-/* orhpan objectid for tracking unlinked/truncated files */
-#define BTRFS_ORPHAN_OBJECTID -5ULL
-
-/* does write ahead logging to speed up fsyncs */
-#define BTRFS_TREE_LOG_OBJECTID -6ULL
-#define BTRFS_TREE_LOG_FIXUP_OBJECTID -7ULL
-
-/* for space balancing */
-#define BTRFS_TREE_RELOC_OBJECTID -8ULL
-#define BTRFS_DATA_RELOC_TREE_OBJECTID -9ULL
-
-/*
- * extent checksums all have this objectid
- * this allows them to share the logging tree
- * for fsyncs
- */
-#define BTRFS_EXTENT_CSUM_OBJECTID -10ULL
-
-/* For storing free space cache */
-#define BTRFS_FREE_SPACE_OBJECTID -11ULL
-
-/*
- * The inode number assigned to the special inode for storing
- * free ino cache
- */
-#define BTRFS_FREE_INO_OBJECTID -12ULL
-
-/* dummy objectid represents multiple objectids */
-#define BTRFS_MULTIPLE_OBJECTIDS -255ULL
-
-/*
- * All files have objectids in this range.
- */
-#define BTRFS_FIRST_FREE_OBJECTID 256ULL
-#define BTRFS_LAST_FREE_OBJECTID -256ULL
-#define BTRFS_FIRST_CHUNK_TREE_OBJECTID 256ULL
-
-
-/*
- * the device items go into the chunk tree. The key is in the form
- * [ 1 BTRFS_DEV_ITEM_KEY device_id ]
- */
-#define BTRFS_DEV_ITEMS_OBJECTID 1ULL
-
-#define BTRFS_BTREE_INODE_OBJECTID 1
-
-#define BTRFS_EMPTY_SUBVOL_DIR_OBJECTID 2
-
-#define BTRFS_DEV_REPLACE_DEVID 0ULL
-
/*
* the max metadata block size. This limit is somewhat artificial,
* but the memmove costs go through the roof for larger blocks.
@@ -175,12 +84,6 @@
*/
#define BTRFS_LINK_MAX 65535U
-/* 32 bytes in various csum fields */
-#define BTRFS_CSUM_SIZE 32
-
-/* csum types */
-#define BTRFS_CSUM_TYPE_CRC32 0
-
static const int btrfs_csum_sizes[] = { 4 };
/* four bytes for CRC32 */
@@ -189,17 +92,6 @@
/* spefic to btrfs_map_block(), therefore not in include/linux/blk_types.h */
#define REQ_GET_READ_MIRRORS (1 << 30)
-#define BTRFS_FT_UNKNOWN 0
-#define BTRFS_FT_REG_FILE 1
-#define BTRFS_FT_DIR 2
-#define BTRFS_FT_CHRDEV 3
-#define BTRFS_FT_BLKDEV 4
-#define BTRFS_FT_FIFO 5
-#define BTRFS_FT_SOCK 6
-#define BTRFS_FT_SYMLINK 7
-#define BTRFS_FT_XATTR 8
-#define BTRFS_FT_MAX 9
-
/* ioprio of readahead is set to idle */
#define BTRFS_IOPRIO_READA (IOPRIO_PRIO_VALUE(IOPRIO_CLASS_IDLE, 0))
@@ -207,138 +99,10 @@
#define BTRFS_MAX_EXTENT_SIZE SZ_128M
-/*
- * The key defines the order in the tree, and so it also defines (optimal)
- * block layout.
- *
- * objectid corresponds to the inode number.
- *
- * type tells us things about the object, and is a kind of stream selector.
- * so for a given inode, keys with type of 1 might refer to the inode data,
- * type of 2 may point to file data in the btree and type == 3 may point to
- * extents.
- *
- * offset is the starting byte offset for this key in the stream.
- *
- * btrfs_disk_key is in disk byte order. struct btrfs_key is always
- * in cpu native order. Otherwise they are identical and their sizes
- * should be the same (ie both packed)
- */
-struct btrfs_disk_key {
- __le64 objectid;
- u8 type;
- __le64 offset;
-} __attribute__ ((__packed__));
-
-struct btrfs_key {
- u64 objectid;
- u8 type;
- u64 offset;
-} __attribute__ ((__packed__));
-
struct btrfs_mapping_tree {
struct extent_map_tree map_tree;
};
-struct btrfs_dev_item {
- /* the internal btrfs device id */
- __le64 devid;
-
- /* size of the device */
- __le64 total_bytes;
-
- /* bytes used */
- __le64 bytes_used;
-
- /* optimal io alignment for this device */
- __le32 io_align;
-
- /* optimal io width for this device */
- __le32 io_width;
-
- /* minimal io size for this device */
- __le32 sector_size;
-
- /* type and info about this device */
- __le64 type;
-
- /* expected generation for this device */
- __le64 generation;
-
- /*
- * starting byte of this partition on the device,
- * to allow for stripe alignment in the future
- */
- __le64 start_offset;
-
- /* grouping information for allocation decisions */
- __le32 dev_group;
-
- /* seek speed 0-100 where 100 is fastest */
- u8 seek_speed;
-
- /* bandwidth 0-100 where 100 is fastest */
- u8 bandwidth;
-
- /* btrfs generated uuid for this device */
- u8 uuid[BTRFS_UUID_SIZE];
-
- /* uuid of FS who owns this device */
- u8 fsid[BTRFS_UUID_SIZE];
-} __attribute__ ((__packed__));
-
-struct btrfs_stripe {
- __le64 devid;
- __le64 offset;
- u8 dev_uuid[BTRFS_UUID_SIZE];
-} __attribute__ ((__packed__));
-
-struct btrfs_chunk {
- /* size of this chunk in bytes */
- __le64 length;
-
- /* objectid of the root referencing this chunk */
- __le64 owner;
-
- __le64 stripe_len;
- __le64 type;
-
- /* optimal io alignment for this chunk */
- __le32 io_align;
-
- /* optimal io width for this chunk */
- __le32 io_width;
-
- /* minimal io size for this chunk */
- __le32 sector_size;
-
- /* 2^16 stripes is quite a lot, a second limit is the size of a single
- * item in the btree
- */
- __le16 num_stripes;
-
- /* sub stripes only matter for raid10 */
- __le16 sub_stripes;
- struct btrfs_stripe stripe;
- /* additional stripes go here */
-} __attribute__ ((__packed__));
-
-#define BTRFS_FREE_SPACE_EXTENT 1
-#define BTRFS_FREE_SPACE_BITMAP 2
-
-struct btrfs_free_space_entry {
- __le64 offset;
- __le64 bytes;
- u8 type;
-} __attribute__ ((__packed__));
-
-struct btrfs_free_space_header {
- struct btrfs_disk_key location;
- __le64 generation;
- __le64 num_entries;
- __le64 num_bitmaps;
-} __attribute__ ((__packed__));
-
static inline unsigned long btrfs_chunk_item_size(int num_stripes)
{
BUG_ON(num_stripes == 0);
@@ -346,9 +110,6 @@
sizeof(struct btrfs_stripe) * (num_stripes - 1);
}
-#define BTRFS_HEADER_FLAG_WRITTEN (1ULL << 0)
-#define BTRFS_HEADER_FLAG_RELOC (1ULL << 1)
-
/*
* File system states
*/
@@ -357,13 +118,6 @@
#define BTRFS_FS_STATE_TRANS_ABORTED 2
#define BTRFS_FS_STATE_DEV_REPLACING 3
-/* Super block flags */
-/* Errors detected */
-#define BTRFS_SUPER_FLAG_ERROR (1ULL << 2)
-
-#define BTRFS_SUPER_FLAG_SEEDING (1ULL << 32)
-#define BTRFS_SUPER_FLAG_METADUMP (1ULL << 33)
-
#define BTRFS_BACKREF_REV_MAX 256
#define BTRFS_BACKREF_REV_SHIFT 56
#define BTRFS_BACKREF_REV_MASK (((u64)BTRFS_BACKREF_REV_MAX - 1) << \
@@ -410,7 +164,6 @@
* room to translate 14 chunks with 3 stripes each.
*/
#define BTRFS_SYSTEM_CHUNK_ARRAY_SIZE 2048
-#define BTRFS_LABEL_SIZE 256
/*
* just in case we somehow lose the roots and are not able to mount,
@@ -507,31 +260,6 @@
* Compat flags that we support. If any incompat flags are set other than the
* ones specified below then we will fail to mount
*/
-#define BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE (1ULL << 0)
-
-#define BTRFS_FEATURE_INCOMPAT_MIXED_BACKREF (1ULL << 0)
-#define BTRFS_FEATURE_INCOMPAT_DEFAULT_SUBVOL (1ULL << 1)
-#define BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS (1ULL << 2)
-#define BTRFS_FEATURE_INCOMPAT_COMPRESS_LZO (1ULL << 3)
-/*
- * some patches floated around with a second compression method
- * lets save that incompat here for when they do get in
- * Note we don't actually support it, we're just reserving the
- * number
- */
-#define BTRFS_FEATURE_INCOMPAT_COMPRESS_LZOv2 (1ULL << 4)
-
-/*
- * older kernels tried to do bigger metadata blocks, but the
- * code was pretty buggy. Lets not let them try anymore.
- */
-#define BTRFS_FEATURE_INCOMPAT_BIG_METADATA (1ULL << 5)
-
-#define BTRFS_FEATURE_INCOMPAT_EXTENDED_IREF (1ULL << 6)
-#define BTRFS_FEATURE_INCOMPAT_RAID56 (1ULL << 7)
-#define BTRFS_FEATURE_INCOMPAT_SKINNY_METADATA (1ULL << 8)
-#define BTRFS_FEATURE_INCOMPAT_NO_HOLES (1ULL << 9)
-
#define BTRFS_FEATURE_COMPAT_SUPP 0ULL
#define BTRFS_FEATURE_COMPAT_SAFE_SET 0ULL
#define BTRFS_FEATURE_COMPAT_SAFE_CLEAR 0ULL
@@ -624,357 +352,8 @@
unsigned int need_commit_sem:1;
unsigned int skip_release_on_error:1;
};
-
-/*
- * items in the extent btree are used to record the objectid of the
- * owner of the block and the number of references
- */
-
-struct btrfs_extent_item {
- __le64 refs;
- __le64 generation;
- __le64 flags;
-} __attribute__ ((__packed__));
-
-struct btrfs_extent_item_v0 {
- __le32 refs;
-} __attribute__ ((__packed__));
-
#define BTRFS_MAX_EXTENT_ITEM_SIZE(r) ((BTRFS_LEAF_DATA_SIZE(r) >> 4) - \
sizeof(struct btrfs_item))
-
-#define BTRFS_EXTENT_FLAG_DATA (1ULL << 0)
-#define BTRFS_EXTENT_FLAG_TREE_BLOCK (1ULL << 1)
-
-/* following flags only apply to tree blocks */
-
-/* use full backrefs for extent pointers in the block */
-#define BTRFS_BLOCK_FLAG_FULL_BACKREF (1ULL << 8)
-
-/*
- * this flag is only used internally by scrub and may be changed at any time
- * it is only declared here to avoid collisions
- */
-#define BTRFS_EXTENT_FLAG_SUPER (1ULL << 48)
-
-struct btrfs_tree_block_info {
- struct btrfs_disk_key key;
- u8 level;
-} __attribute__ ((__packed__));
-
-struct btrfs_extent_data_ref {
- __le64 root;
- __le64 objectid;
- __le64 offset;
- __le32 count;
-} __attribute__ ((__packed__));
-
-struct btrfs_shared_data_ref {
- __le32 count;
-} __attribute__ ((__packed__));
-
-struct btrfs_extent_inline_ref {
- u8 type;
- __le64 offset;
-} __attribute__ ((__packed__));
-
-/* old style backrefs item */
-struct btrfs_extent_ref_v0 {
- __le64 root;
- __le64 generation;
- __le64 objectid;
- __le32 count;
-} __attribute__ ((__packed__));
-
-
-/* dev extents record free space on individual devices. The owner
- * field points back to the chunk allocation mapping tree that allocated
- * the extent. The chunk tree uuid field is a way to double check the owner
- */
-struct btrfs_dev_extent {
- __le64 chunk_tree;
- __le64 chunk_objectid;
- __le64 chunk_offset;
- __le64 length;
- u8 chunk_tree_uuid[BTRFS_UUID_SIZE];
-} __attribute__ ((__packed__));
-
-struct btrfs_inode_ref {
- __le64 index;
- __le16 name_len;
- /* name goes here */
-} __attribute__ ((__packed__));
-
-struct btrfs_inode_extref {
- __le64 parent_objectid;
- __le64 index;
- __le16 name_len;
- __u8 name[0];
- /* name goes here */
-} __attribute__ ((__packed__));
-
-struct btrfs_timespec {
- __le64 sec;
- __le32 nsec;
-} __attribute__ ((__packed__));
-
-struct btrfs_inode_item {
- /* nfs style generation number */
- __le64 generation;
- /* transid that last touched this inode */
- __le64 transid;
- __le64 size;
- __le64 nbytes;
- __le64 block_group;
- __le32 nlink;
- __le32 uid;
- __le32 gid;
- __le32 mode;
- __le64 rdev;
- __le64 flags;
-
- /* modification sequence number for NFS */
- __le64 sequence;
-
- /*
- * a little future expansion, for more than this we can
- * just grow the inode item and version it
- */
- __le64 reserved[4];
- struct btrfs_timespec atime;
- struct btrfs_timespec ctime;
- struct btrfs_timespec mtime;
- struct btrfs_timespec otime;
-} __attribute__ ((__packed__));
-
-struct btrfs_dir_log_item {
- __le64 end;
-} __attribute__ ((__packed__));
-
-struct btrfs_dir_item {
- struct btrfs_disk_key location;
- __le64 transid;
- __le16 data_len;
- __le16 name_len;
- u8 type;
-} __attribute__ ((__packed__));
-
-#define BTRFS_ROOT_SUBVOL_RDONLY (1ULL << 0)
-
-/*
- * Internal in-memory flag that a subvolume has been marked for deletion but
- * still visible as a directory
- */
-#define BTRFS_ROOT_SUBVOL_DEAD (1ULL << 48)
-
-struct btrfs_root_item {
- struct btrfs_inode_item inode;
- __le64 generation;
- __le64 root_dirid;
- __le64 bytenr;
- __le64 byte_limit;
- __le64 bytes_used;
- __le64 last_snapshot;
- __le64 flags;
- __le32 refs;
- struct btrfs_disk_key drop_progress;
- u8 drop_level;
- u8 level;
-
- /*
- * The following fields appear after subvol_uuids+subvol_times
- * were introduced.
- */
-
- /*
- * This generation number is used to test if the new fields are valid
- * and up to date while reading the root item. Every time the root item
- * is written out, the "generation" field is copied into this field. If
- * anyone ever mounted the fs with an older kernel, we will have
- * mismatching generation values here and thus must invalidate the
- * new fields. See btrfs_update_root and btrfs_find_last_root for
- * details.
- * the offset of generation_v2 is also used as the start for the memset
- * when invalidating the fields.
- */
- __le64 generation_v2;
- u8 uuid[BTRFS_UUID_SIZE];
- u8 parent_uuid[BTRFS_UUID_SIZE];
- u8 received_uuid[BTRFS_UUID_SIZE];
- __le64 ctransid; /* updated when an inode changes */
- __le64 otransid; /* trans when created */
- __le64 stransid; /* trans when sent. non-zero for received subvol */
- __le64 rtransid; /* trans when received. non-zero for received subvol */
- struct btrfs_timespec ctime;
- struct btrfs_timespec otime;
- struct btrfs_timespec stime;
- struct btrfs_timespec rtime;
- __le64 reserved[8]; /* for future */
-} __attribute__ ((__packed__));
-
-/*
- * this is used for both forward and backward root refs
- */
-struct btrfs_root_ref {
- __le64 dirid;
- __le64 sequence;
- __le16 name_len;
-} __attribute__ ((__packed__));
-
-struct btrfs_disk_balance_args {
- /*
- * profiles to operate on, single is denoted by
- * BTRFS_AVAIL_ALLOC_BIT_SINGLE
- */
- __le64 profiles;
-
- /*
- * usage filter
- * BTRFS_BALANCE_ARGS_USAGE with a single value means '0..N'
- * BTRFS_BALANCE_ARGS_USAGE_RANGE - range syntax, min..max
- */
- union {
- __le64 usage;
- struct {
- __le32 usage_min;
- __le32 usage_max;
- };
- };
-
- /* devid filter */
- __le64 devid;
-
- /* devid subset filter [pstart..pend) */
- __le64 pstart;
- __le64 pend;
-
- /* btrfs virtual address space subset filter [vstart..vend) */
- __le64 vstart;
- __le64 vend;
-
- /*
- * profile to convert to, single is denoted by
- * BTRFS_AVAIL_ALLOC_BIT_SINGLE
- */
- __le64 target;
-
- /* BTRFS_BALANCE_ARGS_* */
- __le64 flags;
-
- /*
- * BTRFS_BALANCE_ARGS_LIMIT with value 'limit'
- * BTRFS_BALANCE_ARGS_LIMIT_RANGE - the extend version can use minimum
- * and maximum
- */
- union {
- __le64 limit;
- struct {
- __le32 limit_min;
- __le32 limit_max;
- };
- };
-
- /*
- * Process chunks that cross stripes_min..stripes_max devices,
- * BTRFS_BALANCE_ARGS_STRIPES_RANGE
- */
- __le32 stripes_min;
- __le32 stripes_max;
-
- __le64 unused[6];
-} __attribute__ ((__packed__));
-
-/*
- * store balance parameters to disk so that balance can be properly
- * resumed after crash or unmount
- */
-struct btrfs_balance_item {
- /* BTRFS_BALANCE_* */
- __le64 flags;
-
- struct btrfs_disk_balance_args data;
- struct btrfs_disk_balance_args meta;
- struct btrfs_disk_balance_args sys;
-
- __le64 unused[4];
-} __attribute__ ((__packed__));
-
-#define BTRFS_FILE_EXTENT_INLINE 0
-#define BTRFS_FILE_EXTENT_REG 1
-#define BTRFS_FILE_EXTENT_PREALLOC 2
-
-struct btrfs_file_extent_item {
- /*
- * transaction id that created this extent
- */
- __le64 generation;
- /*
- * max number of bytes to hold this extent in ram
- * when we split a compressed extent we can't know how big
- * each of the resulting pieces will be. So, this is
- * an upper limit on the size of the extent in ram instead of
- * an exact limit.
- */
- __le64 ram_bytes;
-
- /*
- * 32 bits for the various ways we might encode the data,
- * including compression and encryption. If any of these
- * are set to something a given disk format doesn't understand
- * it is treated like an incompat flag for reading and writing,
- * but not for stat.
- */
- u8 compression;
- u8 encryption;
- __le16 other_encoding; /* spare for later use */
-
- /* are we inline data or a real extent? */
- u8 type;
-
- /*
- * disk space consumed by the extent, checksum blocks are included
- * in these numbers
- *
- * At this offset in the structure, the inline extent data start.
- */
- __le64 disk_bytenr;
- __le64 disk_num_bytes;
- /*
- * the logical offset in file blocks (no csums)
- * this extent record is for. This allows a file extent to point
- * into the middle of an existing extent on disk, sharing it
- * between two snapshots (useful if some bytes in the middle of the
- * extent have changed
- */
- __le64 offset;
- /*
- * the logical number of file blocks (no csums included). This
- * always reflects the size uncompressed and without encoding.
- */
- __le64 num_bytes;
-
-} __attribute__ ((__packed__));
-
-struct btrfs_csum_item {
- u8 csum;
-} __attribute__ ((__packed__));
-
-struct btrfs_dev_stats_item {
- /*
- * grow this item struct at the end for future enhancements and keep
- * the existing values unchanged
- */
- __le64 values[BTRFS_DEV_STAT_VALUES_MAX];
-} __attribute__ ((__packed__));
-
-#define BTRFS_DEV_REPLACE_ITEM_CONT_READING_FROM_SRCDEV_MODE_ALWAYS 0
-#define BTRFS_DEV_REPLACE_ITEM_CONT_READING_FROM_SRCDEV_MODE_AVOID 1
-#define BTRFS_DEV_REPLACE_ITEM_STATE_NEVER_STARTED 0
-#define BTRFS_DEV_REPLACE_ITEM_STATE_STARTED 1
-#define BTRFS_DEV_REPLACE_ITEM_STATE_SUSPENDED 2
-#define BTRFS_DEV_REPLACE_ITEM_STATE_FINISHED 3
-#define BTRFS_DEV_REPLACE_ITEM_STATE_CANCELED 4
-
struct btrfs_dev_replace {
u64 replace_state; /* see #define above */
u64 time_started; /* seconds since 1-Jan-1970 */
@@ -1005,175 +384,6 @@
struct btrfs_scrub_progress scrub_progress;
};
-struct btrfs_dev_replace_item {
- /*
- * grow this item struct at the end for future enhancements and keep
- * the existing values unchanged
- */
- __le64 src_devid;
- __le64 cursor_left;
- __le64 cursor_right;
- __le64 cont_reading_from_srcdev_mode;
-
- __le64 replace_state;
- __le64 time_started;
- __le64 time_stopped;
- __le64 num_write_errors;
- __le64 num_uncorrectable_read_errors;
-} __attribute__ ((__packed__));
-
-/* different types of block groups (and chunks) */
-#define BTRFS_BLOCK_GROUP_DATA (1ULL << 0)
-#define BTRFS_BLOCK_GROUP_SYSTEM (1ULL << 1)
-#define BTRFS_BLOCK_GROUP_METADATA (1ULL << 2)
-#define BTRFS_BLOCK_GROUP_RAID0 (1ULL << 3)
-#define BTRFS_BLOCK_GROUP_RAID1 (1ULL << 4)
-#define BTRFS_BLOCK_GROUP_DUP (1ULL << 5)
-#define BTRFS_BLOCK_GROUP_RAID10 (1ULL << 6)
-#define BTRFS_BLOCK_GROUP_RAID5 (1ULL << 7)
-#define BTRFS_BLOCK_GROUP_RAID6 (1ULL << 8)
-#define BTRFS_BLOCK_GROUP_RESERVED (BTRFS_AVAIL_ALLOC_BIT_SINGLE | \
- BTRFS_SPACE_INFO_GLOBAL_RSV)
-
-enum btrfs_raid_types {
- BTRFS_RAID_RAID10,
- BTRFS_RAID_RAID1,
- BTRFS_RAID_DUP,
- BTRFS_RAID_RAID0,
- BTRFS_RAID_SINGLE,
- BTRFS_RAID_RAID5,
- BTRFS_RAID_RAID6,
- BTRFS_NR_RAID_TYPES
-};
-
-#define BTRFS_BLOCK_GROUP_TYPE_MASK (BTRFS_BLOCK_GROUP_DATA | \
- BTRFS_BLOCK_GROUP_SYSTEM | \
- BTRFS_BLOCK_GROUP_METADATA)
-
-#define BTRFS_BLOCK_GROUP_PROFILE_MASK (BTRFS_BLOCK_GROUP_RAID0 | \
- BTRFS_BLOCK_GROUP_RAID1 | \
- BTRFS_BLOCK_GROUP_RAID5 | \
- BTRFS_BLOCK_GROUP_RAID6 | \
- BTRFS_BLOCK_GROUP_DUP | \
- BTRFS_BLOCK_GROUP_RAID10)
-#define BTRFS_BLOCK_GROUP_RAID56_MASK (BTRFS_BLOCK_GROUP_RAID5 | \
- BTRFS_BLOCK_GROUP_RAID6)
-
-/*
- * We need a bit for restriper to be able to tell when chunks of type
- * SINGLE are available. This "extended" profile format is used in
- * fs_info->avail_*_alloc_bits (in-memory) and balance item fields
- * (on-disk). The corresponding on-disk bit in chunk.type is reserved
- * to avoid remappings between two formats in future.
- */
-#define BTRFS_AVAIL_ALLOC_BIT_SINGLE (1ULL << 48)
-
-/*
- * A fake block group type that is used to communicate global block reserve
- * size to userspace via the SPACE_INFO ioctl.
- */
-#define BTRFS_SPACE_INFO_GLOBAL_RSV (1ULL << 49)
-
-#define BTRFS_EXTENDED_PROFILE_MASK (BTRFS_BLOCK_GROUP_PROFILE_MASK | \
- BTRFS_AVAIL_ALLOC_BIT_SINGLE)
-
-static inline u64 chunk_to_extended(u64 flags)
-{
- if ((flags & BTRFS_BLOCK_GROUP_PROFILE_MASK) == 0)
- flags |= BTRFS_AVAIL_ALLOC_BIT_SINGLE;
-
- return flags;
-}
-static inline u64 extended_to_chunk(u64 flags)
-{
- return flags & ~BTRFS_AVAIL_ALLOC_BIT_SINGLE;
-}
-
-struct btrfs_block_group_item {
- __le64 used;
- __le64 chunk_objectid;
- __le64 flags;
-} __attribute__ ((__packed__));
-
-struct btrfs_free_space_info {
- __le32 extent_count;
- __le32 flags;
-} __attribute__ ((__packed__));
-
-#define BTRFS_FREE_SPACE_USING_BITMAPS (1ULL << 0)
-
-#define BTRFS_QGROUP_LEVEL_SHIFT 48
-static inline u64 btrfs_qgroup_level(u64 qgroupid)
-{
- return qgroupid >> BTRFS_QGROUP_LEVEL_SHIFT;
-}
-
-/*
- * is subvolume quota turned on?
- */
-#define BTRFS_QGROUP_STATUS_FLAG_ON (1ULL << 0)
-/*
- * RESCAN is set during the initialization phase
- */
-#define BTRFS_QGROUP_STATUS_FLAG_RESCAN (1ULL << 1)
-/*
- * Some qgroup entries are known to be out of date,
- * either because the configuration has changed in a way that
- * makes a rescan necessary, or because the fs has been mounted
- * with a non-qgroup-aware version.
- * Turning qouta off and on again makes it inconsistent, too.
- */
-#define BTRFS_QGROUP_STATUS_FLAG_INCONSISTENT (1ULL << 2)
-
-#define BTRFS_QGROUP_STATUS_VERSION 1
-
-struct btrfs_qgroup_status_item {
- __le64 version;
- /*
- * the generation is updated during every commit. As older
- * versions of btrfs are not aware of qgroups, it will be
- * possible to detect inconsistencies by checking the
- * generation on mount time
- */
- __le64 generation;
-
- /* flag definitions see above */
- __le64 flags;
-
- /*
- * only used during scanning to record the progress
- * of the scan. It contains a logical address
- */
- __le64 rescan;
-} __attribute__ ((__packed__));
-
-struct btrfs_qgroup_info_item {
- __le64 generation;
- __le64 rfer;
- __le64 rfer_cmpr;
- __le64 excl;
- __le64 excl_cmpr;
-} __attribute__ ((__packed__));
-
-/* flags definition for qgroup limits */
-#define BTRFS_QGROUP_LIMIT_MAX_RFER (1ULL << 0)
-#define BTRFS_QGROUP_LIMIT_MAX_EXCL (1ULL << 1)
-#define BTRFS_QGROUP_LIMIT_RSV_RFER (1ULL << 2)
-#define BTRFS_QGROUP_LIMIT_RSV_EXCL (1ULL << 3)
-#define BTRFS_QGROUP_LIMIT_RFER_CMPR (1ULL << 4)
-#define BTRFS_QGROUP_LIMIT_EXCL_CMPR (1ULL << 5)
-
-struct btrfs_qgroup_limit_item {
- /*
- * only updated when any of the other values change
- */
- __le64 flags;
- __le64 max_rfer;
- __le64 max_excl;
- __le64 rsv_rfer;
- __le64 rsv_excl;
-} __attribute__ ((__packed__));
-
/* For raid type sysfs entries */
struct raid_kobject {
int raid_type;
@@ -1408,6 +618,27 @@
struct btrfs_io_ctl io_ctl;
+ /*
+ * Incremented when doing extent allocations and holding a read lock
+ * on the space_info's groups_sem semaphore.
+ * Decremented when an ordered extent that represents an IO against this
+ * block group's range is created (after it's added to its inode's
+ * root's list of ordered extents) or immediately after the allocation
+ * if it's a metadata extent or fallocate extent (for these cases we
+ * don't create ordered extents).
+ */
+ atomic_t reservations;
+
+ /*
+ * Incremented while holding the spinlock *lock* by a task checking if
+ * it can perform a nocow write (incremented if the value for the *ro*
+ * field is 0). Decremented by such tasks once they create an ordered
+ * extent or before that if some error happens before reaching that step.
+ * This is to prevent races between block group relocation and nocow
+ * writes through direct IO.
+ */
+ atomic_t nocow_writers;
+
/* Lock for free space tree operations. */
struct mutex free_space_lock;
@@ -2026,228 +1257,6 @@
atomic_t qgroup_meta_rsv;
};
-struct btrfs_ioctl_defrag_range_args {
- /* start of the defrag operation */
- __u64 start;
-
- /* number of bytes to defrag, use (u64)-1 to say all */
- __u64 len;
-
- /*
- * flags for the operation, which can include turning
- * on compression for this one defrag
- */
- __u64 flags;
-
- /*
- * any extent bigger than this will be considered
- * already defragged. Use 0 to take the kernel default
- * Use 1 to say every single extent must be rewritten
- */
- __u32 extent_thresh;
-
- /*
- * which compression method to use if turning on compression
- * for this defrag operation. If unspecified, zlib will
- * be used
- */
- __u32 compress_type;
-
- /* spare for later */
- __u32 unused[4];
-};
-
-
-/*
- * inode items have the data typically returned from stat and store other
- * info about object characteristics. There is one for every file and dir in
- * the FS
- */
-#define BTRFS_INODE_ITEM_KEY 1
-#define BTRFS_INODE_REF_KEY 12
-#define BTRFS_INODE_EXTREF_KEY 13
-#define BTRFS_XATTR_ITEM_KEY 24
-#define BTRFS_ORPHAN_ITEM_KEY 48
-/* reserve 2-15 close to the inode for later flexibility */
-
-/*
- * dir items are the name -> inode pointers in a directory. There is one
- * for every name in a directory.
- */
-#define BTRFS_DIR_LOG_ITEM_KEY 60
-#define BTRFS_DIR_LOG_INDEX_KEY 72
-#define BTRFS_DIR_ITEM_KEY 84
-#define BTRFS_DIR_INDEX_KEY 96
-/*
- * extent data is for file data
- */
-#define BTRFS_EXTENT_DATA_KEY 108
-
-/*
- * extent csums are stored in a separate tree and hold csums for
- * an entire extent on disk.
- */
-#define BTRFS_EXTENT_CSUM_KEY 128
-
-/*
- * root items point to tree roots. They are typically in the root
- * tree used by the super block to find all the other trees
- */
-#define BTRFS_ROOT_ITEM_KEY 132
-
-/*
- * root backrefs tie subvols and snapshots to the directory entries that
- * reference them
- */
-#define BTRFS_ROOT_BACKREF_KEY 144
-
-/*
- * root refs make a fast index for listing all of the snapshots and
- * subvolumes referenced by a given root. They point directly to the
- * directory item in the root that references the subvol
- */
-#define BTRFS_ROOT_REF_KEY 156
-
-/*
- * extent items are in the extent map tree. These record which blocks
- * are used, and how many references there are to each block
- */
-#define BTRFS_EXTENT_ITEM_KEY 168
-
-/*
- * The same as the BTRFS_EXTENT_ITEM_KEY, except it's metadata we already know
- * the length, so we save the level in key->offset instead of the length.
- */
-#define BTRFS_METADATA_ITEM_KEY 169
-
-#define BTRFS_TREE_BLOCK_REF_KEY 176
-
-#define BTRFS_EXTENT_DATA_REF_KEY 178
-
-#define BTRFS_EXTENT_REF_V0_KEY 180
-
-#define BTRFS_SHARED_BLOCK_REF_KEY 182
-
-#define BTRFS_SHARED_DATA_REF_KEY 184
-
-/*
- * block groups give us hints into the extent allocation trees. Which
- * blocks are free etc etc
- */
-#define BTRFS_BLOCK_GROUP_ITEM_KEY 192
-
-/*
- * Every block group is represented in the free space tree by a free space info
- * item, which stores some accounting information. It is keyed on
- * (block_group_start, FREE_SPACE_INFO, block_group_length).
- */
-#define BTRFS_FREE_SPACE_INFO_KEY 198
-
-/*
- * A free space extent tracks an extent of space that is free in a block group.
- * It is keyed on (start, FREE_SPACE_EXTENT, length).
- */
-#define BTRFS_FREE_SPACE_EXTENT_KEY 199
-
-/*
- * When a block group becomes very fragmented, we convert it to use bitmaps
- * instead of extents. A free space bitmap is keyed on
- * (start, FREE_SPACE_BITMAP, length); the corresponding item is a bitmap with
- * (length / sectorsize) bits.
- */
-#define BTRFS_FREE_SPACE_BITMAP_KEY 200
-
-#define BTRFS_DEV_EXTENT_KEY 204
-#define BTRFS_DEV_ITEM_KEY 216
-#define BTRFS_CHUNK_ITEM_KEY 228
-
-/*
- * Records the overall state of the qgroups.
- * There's only one instance of this key present,
- * (0, BTRFS_QGROUP_STATUS_KEY, 0)
- */
-#define BTRFS_QGROUP_STATUS_KEY 240
-/*
- * Records the currently used space of the qgroup.
- * One key per qgroup, (0, BTRFS_QGROUP_INFO_KEY, qgroupid).
- */
-#define BTRFS_QGROUP_INFO_KEY 242
-/*
- * Contains the user configured limits for the qgroup.
- * One key per qgroup, (0, BTRFS_QGROUP_LIMIT_KEY, qgroupid).
- */
-#define BTRFS_QGROUP_LIMIT_KEY 244
-/*
- * Records the child-parent relationship of qgroups. For
- * each relation, 2 keys are present:
- * (childid, BTRFS_QGROUP_RELATION_KEY, parentid)
- * (parentid, BTRFS_QGROUP_RELATION_KEY, childid)
- */
-#define BTRFS_QGROUP_RELATION_KEY 246
-
-/*
- * Obsolete name, see BTRFS_TEMPORARY_ITEM_KEY.
- */
-#define BTRFS_BALANCE_ITEM_KEY 248
-
-/*
- * The key type for tree items that are stored persistently, but do not need to
- * exist for extended period of time. The items can exist in any tree.
- *
- * [subtype, BTRFS_TEMPORARY_ITEM_KEY, data]
- *
- * Existing items:
- *
- * - balance status item
- * (BTRFS_BALANCE_OBJECTID, BTRFS_TEMPORARY_ITEM_KEY, 0)
- */
-#define BTRFS_TEMPORARY_ITEM_KEY 248
-
-/*
- * Obsolete name, see BTRFS_PERSISTENT_ITEM_KEY
- */
-#define BTRFS_DEV_STATS_KEY 249
-
-/*
- * The key type for tree items that are stored persistently and usually exist
- * for a long period, eg. filesystem lifetime. The item kinds can be status
- * information, stats or preference values. The item can exist in any tree.
- *
- * [subtype, BTRFS_PERSISTENT_ITEM_KEY, data]
- *
- * Existing items:
- *
- * - device statistics, store IO stats in the device tree, one key for all
- * stats
- * (BTRFS_DEV_STATS_OBJECTID, BTRFS_DEV_STATS_KEY, 0)
- */
-#define BTRFS_PERSISTENT_ITEM_KEY 249
-
-/*
- * Persistantly stores the device replace state in the device tree.
- * The key is built like this: (0, BTRFS_DEV_REPLACE_KEY, 0).
- */
-#define BTRFS_DEV_REPLACE_KEY 250
-
-/*
- * Stores items that allow to quickly map UUIDs to something else.
- * These items are part of the filesystem UUID tree.
- * The key is built like this:
- * (UUID_upper_64_bits, BTRFS_UUID_KEY*, UUID_lower_64_bits).
- */
-#if BTRFS_UUID_SIZE != 16
-#error "UUID items require BTRFS_UUID_SIZE == 16!"
-#endif
-#define BTRFS_UUID_KEY_SUBVOL 251 /* for UUIDs assigned to subvols */
-#define BTRFS_UUID_KEY_RECEIVED_SUBVOL 252 /* for UUIDs assigned to
- * received subvols */
-
-/*
- * string items are for debugging. They just store a short string of
- * data in the FS
- */
-#define BTRFS_STRING_ITEM_KEY 253
-
/*
* Flags for mount options.
*
@@ -3499,6 +2508,12 @@
struct btrfs_root *root);
int btrfs_check_space_for_delayed_refs(struct btrfs_trans_handle *trans,
struct btrfs_root *root);
+void btrfs_dec_block_group_reservations(struct btrfs_fs_info *fs_info,
+ const u64 start);
+void btrfs_wait_block_group_reservations(struct btrfs_block_group_cache *bg);
+bool btrfs_inc_nocow_writers(struct btrfs_fs_info *fs_info, u64 bytenr);
+void btrfs_dec_nocow_writers(struct btrfs_fs_info *fs_info, u64 bytenr);
+void btrfs_wait_nocow_writers(struct btrfs_block_group_cache *bg);
void btrfs_put_block_group(struct btrfs_block_group_cache *cache);
int btrfs_run_delayed_refs(struct btrfs_trans_handle *trans,
struct btrfs_root *root, unsigned long count);
@@ -4122,6 +3137,7 @@
/* ioctl.c */
long btrfs_ioctl(struct file *file, unsigned int cmd, unsigned long arg);
+long btrfs_compat_ioctl(struct file *file, unsigned int cmd, unsigned long arg);
int btrfs_ioctl_get_supported_features(void __user *arg);
void btrfs_update_iflags(struct inode *inode);
void btrfs_inherit_iflags(struct inode *inode, struct inode *dir);
@@ -4326,10 +3342,9 @@
#define ASSERT(expr) ((void)0)
#endif
-#define btrfs_assert()
__printf(5, 6)
__cold
-void __btrfs_std_error(struct btrfs_fs_info *fs_info, const char *function,
+void __btrfs_handle_fs_error(struct btrfs_fs_info *fs_info, const char *function,
unsigned int line, int errno, const char *fmt, ...);
const char *btrfs_decode_error(int errno);
@@ -4339,6 +3354,46 @@
struct btrfs_root *root, const char *function,
unsigned int line, int errno);
+/*
+ * Call btrfs_abort_transaction as early as possible when an error condition is
+ * detected, that way the exact line number is reported.
+ */
+#define btrfs_abort_transaction(trans, root, errno) \
+do { \
+ /* Report first abort since mount */ \
+ if (!test_and_set_bit(BTRFS_FS_STATE_TRANS_ABORTED, \
+ &((root)->fs_info->fs_state))) { \
+ WARN(1, KERN_DEBUG \
+ "BTRFS: Transaction aborted (error %d)\n", \
+ (errno)); \
+ } \
+ __btrfs_abort_transaction((trans), (root), __func__, \
+ __LINE__, (errno)); \
+} while (0)
+
+#define btrfs_handle_fs_error(fs_info, errno, fmt, args...) \
+do { \
+ __btrfs_handle_fs_error((fs_info), __func__, __LINE__, \
+ (errno), fmt, ##args); \
+} while (0)
+
+__printf(5, 6)
+__cold
+void __btrfs_panic(struct btrfs_fs_info *fs_info, const char *function,
+ unsigned int line, int errno, const char *fmt, ...);
+/*
+ * If BTRFS_MOUNT_PANIC_ON_FATAL_ERROR is in mount_opt, __btrfs_panic
+ * will panic(). Otherwise we BUG() here.
+ */
+#define btrfs_panic(fs_info, errno, fmt, args...) \
+do { \
+ __btrfs_panic(fs_info, __func__, __LINE__, errno, fmt, ##args); \
+ BUG(); \
+} while (0)
+
+
+/* compatibility and incompatibility defines */
+
#define btrfs_set_fs_incompat(__fs_info, opt) \
__btrfs_set_fs_incompat((__fs_info), BTRFS_FEATURE_INCOMPAT_##opt)
@@ -4455,44 +3510,6 @@
return !!(btrfs_super_compat_ro_flags(disk_super) & flag);
}
-/*
- * Call btrfs_abort_transaction as early as possible when an error condition is
- * detected, that way the exact line number is reported.
- */
-#define btrfs_abort_transaction(trans, root, errno) \
-do { \
- /* Report first abort since mount */ \
- if (!test_and_set_bit(BTRFS_FS_STATE_TRANS_ABORTED, \
- &((root)->fs_info->fs_state))) { \
- WARN(1, KERN_DEBUG \
- "BTRFS: Transaction aborted (error %d)\n", \
- (errno)); \
- } \
- __btrfs_abort_transaction((trans), (root), __func__, \
- __LINE__, (errno)); \
-} while (0)
-
-#define btrfs_std_error(fs_info, errno, fmt, args...) \
-do { \
- __btrfs_std_error((fs_info), __func__, __LINE__, \
- (errno), fmt, ##args); \
-} while (0)
-
-__printf(5, 6)
-__cold
-void __btrfs_panic(struct btrfs_fs_info *fs_info, const char *function,
- unsigned int line, int errno, const char *fmt, ...);
-
-/*
- * If BTRFS_MOUNT_PANIC_ON_FATAL_ERROR is in mount_opt, __btrfs_panic
- * will panic(). Otherwise we BUG() here.
- */
-#define btrfs_panic(fs_info, errno, fmt, args...) \
-do { \
- __btrfs_panic(fs_info, __func__, __LINE__, errno, fmt, ##args); \
- BUG(); \
-} while (0)
-
/* acl.c */
#ifdef CONFIG_BTRFS_FS_POSIX_ACL
struct posix_acl *btrfs_get_acl(struct inode *inode, int type);
diff --git a/fs/btrfs/delayed-inode.c b/fs/btrfs/delayed-inode.c
index 6cef006..61561c2 100644
--- a/fs/btrfs/delayed-inode.c
+++ b/fs/btrfs/delayed-inode.c
@@ -134,7 +134,7 @@
/* cached in the btrfs inode and can be accessed */
atomic_add(2, &node->refs);
- ret = radix_tree_preload(GFP_NOFS & ~__GFP_HIGHMEM);
+ ret = radix_tree_preload(GFP_NOFS);
if (ret) {
kmem_cache_free(delayed_node_cache, node);
return ERR_PTR(ret);
diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c
index 26bcb48..85f12e6 100644
--- a/fs/btrfs/dev-replace.c
+++ b/fs/btrfs/dev-replace.c
@@ -44,9 +44,6 @@
struct btrfs_fs_info *fs_info,
struct btrfs_device *srcdev,
struct btrfs_device *tgtdev);
-static int btrfs_dev_replace_find_srcdev(struct btrfs_root *root, u64 srcdevid,
- char *srcdev_name,
- struct btrfs_device **device);
static u64 __btrfs_dev_replace_cancel(struct btrfs_fs_info *fs_info);
static int btrfs_dev_replace_kthread(void *data);
static int btrfs_dev_replace_continue_on_mount(struct btrfs_fs_info *fs_info);
@@ -305,8 +302,8 @@
dev_replace->cursor_left_last_write_of_item;
}
-int btrfs_dev_replace_start(struct btrfs_root *root,
- struct btrfs_ioctl_dev_replace_args *args)
+int btrfs_dev_replace_start(struct btrfs_root *root, char *tgtdev_name,
+ u64 srcdevid, char *srcdev_name, int read_src)
{
struct btrfs_trans_handle *trans;
struct btrfs_fs_info *fs_info = root->fs_info;
@@ -315,29 +312,16 @@
struct btrfs_device *tgt_device = NULL;
struct btrfs_device *src_device = NULL;
- switch (args->start.cont_reading_from_srcdev_mode) {
- case BTRFS_IOCTL_DEV_REPLACE_CONT_READING_FROM_SRCDEV_MODE_ALWAYS:
- case BTRFS_IOCTL_DEV_REPLACE_CONT_READING_FROM_SRCDEV_MODE_AVOID:
- break;
- default:
- return -EINVAL;
- }
-
- if ((args->start.srcdevid == 0 && args->start.srcdev_name[0] == '\0') ||
- args->start.tgtdev_name[0] == '\0')
- return -EINVAL;
-
/* the disk copy procedure reuses the scrub code */
mutex_lock(&fs_info->volume_mutex);
- ret = btrfs_dev_replace_find_srcdev(root, args->start.srcdevid,
- args->start.srcdev_name,
- &src_device);
+ ret = btrfs_find_device_by_devspec(root, srcdevid,
+ srcdev_name, &src_device);
if (ret) {
mutex_unlock(&fs_info->volume_mutex);
return ret;
}
- ret = btrfs_init_dev_replace_tgtdev(root, args->start.tgtdev_name,
+ ret = btrfs_init_dev_replace_tgtdev(root, tgtdev_name,
src_device, &tgt_device);
mutex_unlock(&fs_info->volume_mutex);
if (ret)
@@ -364,18 +348,17 @@
break;
case BTRFS_IOCTL_DEV_REPLACE_STATE_STARTED:
case BTRFS_IOCTL_DEV_REPLACE_STATE_SUSPENDED:
- args->result = BTRFS_IOCTL_DEV_REPLACE_RESULT_ALREADY_STARTED;
+ ret = BTRFS_IOCTL_DEV_REPLACE_RESULT_ALREADY_STARTED;
goto leave;
}
- dev_replace->cont_reading_from_srcdev_mode =
- args->start.cont_reading_from_srcdev_mode;
+ dev_replace->cont_reading_from_srcdev_mode = read_src;
WARN_ON(!src_device);
dev_replace->srcdev = src_device;
WARN_ON(!tgt_device);
dev_replace->tgtdev = tgt_device;
- btrfs_info_in_rcu(root->fs_info,
+ btrfs_info_in_rcu(fs_info,
"dev_replace from %s (devid %llu) to %s started",
src_device->missing ? "<missing disk>" :
rcu_str_deref(src_device->name),
@@ -396,14 +379,13 @@
dev_replace->item_needs_writeback = 1;
atomic64_set(&dev_replace->num_write_errors, 0);
atomic64_set(&dev_replace->num_uncorrectable_read_errors, 0);
- args->result = BTRFS_IOCTL_DEV_REPLACE_RESULT_NO_ERROR;
btrfs_dev_replace_unlock(dev_replace, 1);
ret = btrfs_sysfs_add_device_link(tgt_device->fs_devices, tgt_device);
if (ret)
- btrfs_err(root->fs_info, "kobj add dev failed %d\n", ret);
+ btrfs_err(fs_info, "kobj add dev failed %d\n", ret);
- btrfs_wait_ordered_roots(root->fs_info, -1);
+ btrfs_wait_ordered_roots(root->fs_info, -1, 0, (u64)-1);
/* force writing the updated state information to disk */
trans = btrfs_start_transaction(root, 0);
@@ -421,11 +403,9 @@
btrfs_device_get_total_bytes(src_device),
&dev_replace->scrub_progress, 0, 1);
- ret = btrfs_dev_replace_finishing(root->fs_info, ret);
- /* don't warn if EINPROGRESS, someone else might be running scrub */
+ ret = btrfs_dev_replace_finishing(fs_info, ret);
if (ret == -EINPROGRESS) {
- args->result = BTRFS_IOCTL_DEV_REPLACE_RESULT_SCRUB_INPROGRESS;
- ret = 0;
+ ret = BTRFS_IOCTL_DEV_REPLACE_RESULT_SCRUB_INPROGRESS;
} else {
WARN_ON(ret);
}
@@ -440,6 +420,35 @@
return ret;
}
+int btrfs_dev_replace_by_ioctl(struct btrfs_root *root,
+ struct btrfs_ioctl_dev_replace_args *args)
+{
+ int ret;
+
+ switch (args->start.cont_reading_from_srcdev_mode) {
+ case BTRFS_IOCTL_DEV_REPLACE_CONT_READING_FROM_SRCDEV_MODE_ALWAYS:
+ case BTRFS_IOCTL_DEV_REPLACE_CONT_READING_FROM_SRCDEV_MODE_AVOID:
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ if ((args->start.srcdevid == 0 && args->start.srcdev_name[0] == '\0') ||
+ args->start.tgtdev_name[0] == '\0')
+ return -EINVAL;
+
+ ret = btrfs_dev_replace_start(root, args->start.tgtdev_name,
+ args->start.srcdevid,
+ args->start.srcdev_name,
+ args->start.cont_reading_from_srcdev_mode);
+ args->result = ret;
+ /* don't warn if EINPROGRESS, someone else might be running scrub */
+ if (ret == BTRFS_IOCTL_DEV_REPLACE_RESULT_SCRUB_INPROGRESS)
+ ret = 0;
+
+ return ret;
+}
+
/*
* blocked until all flighting bios are finished.
*/
@@ -495,7 +504,7 @@
mutex_unlock(&dev_replace->lock_finishing_cancel_unmount);
return ret;
}
- btrfs_wait_ordered_roots(root->fs_info, -1);
+ btrfs_wait_ordered_roots(root->fs_info, -1, 0, (u64)-1);
trans = btrfs_start_transaction(root, 0);
if (IS_ERR(trans)) {
@@ -560,10 +569,9 @@
ASSERT(list_empty(&src_device->resized_list));
tgt_device->commit_total_bytes = src_device->commit_total_bytes;
tgt_device->commit_bytes_used = src_device->bytes_used;
- if (fs_info->sb->s_bdev == src_device->bdev)
- fs_info->sb->s_bdev = tgt_device->bdev;
- if (fs_info->fs_devices->latest_bdev == src_device->bdev)
- fs_info->fs_devices->latest_bdev = tgt_device->bdev;
+
+ btrfs_assign_next_active_device(fs_info, src_device, tgt_device);
+
list_add(&tgt_device->dev_alloc_list, &fs_info->fs_devices->alloc_list);
fs_info->fs_devices->rw_devices++;
@@ -626,25 +634,6 @@
write_unlock(&em_tree->lock);
}
-static int btrfs_dev_replace_find_srcdev(struct btrfs_root *root, u64 srcdevid,
- char *srcdev_name,
- struct btrfs_device **device)
-{
- int ret;
-
- if (srcdevid) {
- ret = 0;
- *device = btrfs_find_device(root->fs_info, srcdevid, NULL,
- NULL);
- if (!*device)
- ret = -ENOENT;
- } else {
- ret = btrfs_find_device_missing_or_by_path(root, srcdev_name,
- device);
- }
- return ret;
-}
-
void btrfs_dev_replace_status(struct btrfs_fs_info *fs_info,
struct btrfs_ioctl_dev_replace_args *args)
{
diff --git a/fs/btrfs/dev-replace.h b/fs/btrfs/dev-replace.h
index 29e3ef5f..e922b42 100644
--- a/fs/btrfs/dev-replace.h
+++ b/fs/btrfs/dev-replace.h
@@ -25,8 +25,10 @@
int btrfs_run_dev_replace(struct btrfs_trans_handle *trans,
struct btrfs_fs_info *fs_info);
void btrfs_after_dev_replace_commit(struct btrfs_fs_info *fs_info);
-int btrfs_dev_replace_start(struct btrfs_root *root,
+int btrfs_dev_replace_by_ioctl(struct btrfs_root *root,
struct btrfs_ioctl_dev_replace_args *args);
+int btrfs_dev_replace_start(struct btrfs_root *root, char *tgtdev_name,
+ u64 srcdevid, char *srcdev_name, int read_src);
void btrfs_dev_replace_status(struct btrfs_fs_info *fs_info,
struct btrfs_ioctl_dev_replace_args *args);
int btrfs_dev_replace_cancel(struct btrfs_fs_info *fs_info,
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 4e47849..91d1239 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -1640,7 +1640,7 @@
{
int ret;
- ret = radix_tree_preload(GFP_NOFS & ~__GFP_HIGHMEM);
+ ret = radix_tree_preload(GFP_NOFS);
if (ret)
return ret;
@@ -2417,7 +2417,7 @@
/* returns with log_tree_root freed on success */
ret = btrfs_recover_log_trees(log_tree_root);
if (ret) {
- btrfs_std_error(tree_root->fs_info, ret,
+ btrfs_handle_fs_error(tree_root->fs_info, ret,
"Failed to recover log tree");
free_extent_buffer(log_tree_root->node);
kfree(log_tree_root);
@@ -2517,6 +2517,7 @@
int num_backups_tried = 0;
int backup_index = 0;
int max_active;
+ bool cleaner_mutex_locked = false;
tree_root = fs_info->tree_root = btrfs_alloc_root(fs_info, GFP_KERNEL);
chunk_root = fs_info->chunk_root = btrfs_alloc_root(fs_info, GFP_KERNEL);
@@ -2713,7 +2714,7 @@
* Pass the whole disk block of size BTRFS_SUPER_INFO_SIZE (4k).
*/
if (btrfs_check_super_csum(bh->b_data)) {
- printk(KERN_ERR "BTRFS: superblock checksum mismatch\n");
+ btrfs_err(fs_info, "superblock checksum mismatch");
err = -EINVAL;
brelse(bh);
goto fail_alloc;
@@ -2733,7 +2734,7 @@
ret = btrfs_check_super_valid(fs_info, sb->s_flags & MS_RDONLY);
if (ret) {
- printk(KERN_ERR "BTRFS: superblock contains fatal errors\n");
+ btrfs_err(fs_info, "superblock contains fatal errors");
err = -EINVAL;
goto fail_alloc;
}
@@ -2768,9 +2769,9 @@
features = btrfs_super_incompat_flags(disk_super) &
~BTRFS_FEATURE_INCOMPAT_SUPP;
if (features) {
- printk(KERN_ERR "BTRFS: couldn't mount because of "
- "unsupported optional features (%Lx).\n",
- features);
+ btrfs_err(fs_info,
+ "cannot mount because of unsupported optional features (%llx)",
+ features);
err = -EINVAL;
goto fail_alloc;
}
@@ -2781,7 +2782,7 @@
features |= BTRFS_FEATURE_INCOMPAT_COMPRESS_LZO;
if (features & BTRFS_FEATURE_INCOMPAT_SKINNY_METADATA)
- printk(KERN_INFO "BTRFS: has skinny extents\n");
+ btrfs_info(fs_info, "has skinny extents");
/*
* flag our filesystem as having big metadata blocks if
@@ -2789,7 +2790,8 @@
*/
if (btrfs_super_nodesize(disk_super) > PAGE_SIZE) {
if (!(features & BTRFS_FEATURE_INCOMPAT_BIG_METADATA))
- printk(KERN_INFO "BTRFS: flagging fs with big metadata feature\n");
+ btrfs_info(fs_info,
+ "flagging fs with big metadata feature");
features |= BTRFS_FEATURE_INCOMPAT_BIG_METADATA;
}
@@ -2805,9 +2807,9 @@
*/
if ((features & BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS) &&
(sectorsize != nodesize)) {
- printk(KERN_ERR "BTRFS: unequal leaf/node/sector sizes "
- "are not allowed for mixed block groups on %s\n",
- sb->s_id);
+ btrfs_err(fs_info,
+"unequal nodesize/sectorsize (%u != %u) are not allowed for mixed block groups",
+ nodesize, sectorsize);
goto fail_alloc;
}
@@ -2820,8 +2822,8 @@
features = btrfs_super_compat_ro_flags(disk_super) &
~BTRFS_FEATURE_COMPAT_RO_SUPP;
if (!(sb->s_flags & MS_RDONLY) && features) {
- printk(KERN_ERR "BTRFS: couldn't mount RDWR because of "
- "unsupported option features (%Lx).\n",
+ btrfs_err(fs_info,
+ "cannot mount read-write because of unsupported optional features (%llx)",
features);
err = -EINVAL;
goto fail_alloc;
@@ -2850,8 +2852,7 @@
ret = btrfs_read_sys_array(tree_root);
mutex_unlock(&fs_info->chunk_mutex);
if (ret) {
- printk(KERN_ERR "BTRFS: failed to read the system "
- "array on %s\n", sb->s_id);
+ btrfs_err(fs_info, "failed to read the system array: %d", ret);
goto fail_sb_buffer;
}
@@ -2865,8 +2866,7 @@
generation);
if (IS_ERR(chunk_root->node) ||
!extent_buffer_uptodate(chunk_root->node)) {
- printk(KERN_ERR "BTRFS: failed to read chunk root on %s\n",
- sb->s_id);
+ btrfs_err(fs_info, "failed to read chunk root");
if (!IS_ERR(chunk_root->node))
free_extent_buffer(chunk_root->node);
chunk_root->node = NULL;
@@ -2880,8 +2880,7 @@
ret = btrfs_read_chunk_tree(chunk_root);
if (ret) {
- printk(KERN_ERR "BTRFS: failed to read chunk tree on %s\n",
- sb->s_id);
+ btrfs_err(fs_info, "failed to read chunk tree: %d", ret);
goto fail_tree_roots;
}
@@ -2892,8 +2891,7 @@
btrfs_close_extra_devices(fs_devices, 0);
if (!fs_devices->latest_bdev) {
- printk(KERN_ERR "BTRFS: failed to read devices on %s\n",
- sb->s_id);
+ btrfs_err(fs_info, "failed to read devices");
goto fail_tree_roots;
}
@@ -2905,8 +2903,7 @@
generation);
if (IS_ERR(tree_root->node) ||
!extent_buffer_uptodate(tree_root->node)) {
- printk(KERN_WARNING "BTRFS: failed to read tree root on %s\n",
- sb->s_id);
+ btrfs_warn(fs_info, "failed to read tree root");
if (!IS_ERR(tree_root->node))
free_extent_buffer(tree_root->node);
tree_root->node = NULL;
@@ -2938,20 +2935,19 @@
ret = btrfs_recover_balance(fs_info);
if (ret) {
- printk(KERN_ERR "BTRFS: failed to recover balance\n");
+ btrfs_err(fs_info, "failed to recover balance: %d", ret);
goto fail_block_groups;
}
ret = btrfs_init_dev_stats(fs_info);
if (ret) {
- printk(KERN_ERR "BTRFS: failed to init dev_stats: %d\n",
- ret);
+ btrfs_err(fs_info, "failed to init dev_stats: %d", ret);
goto fail_block_groups;
}
ret = btrfs_init_dev_replace(fs_info);
if (ret) {
- pr_err("BTRFS: failed to init dev_replace: %d\n", ret);
+ btrfs_err(fs_info, "failed to init dev_replace: %d", ret);
goto fail_block_groups;
}
@@ -2959,31 +2955,33 @@
ret = btrfs_sysfs_add_fsid(fs_devices, NULL);
if (ret) {
- pr_err("BTRFS: failed to init sysfs fsid interface: %d\n", ret);
+ btrfs_err(fs_info, "failed to init sysfs fsid interface: %d",
+ ret);
goto fail_block_groups;
}
ret = btrfs_sysfs_add_device(fs_devices);
if (ret) {
- pr_err("BTRFS: failed to init sysfs device interface: %d\n", ret);
+ btrfs_err(fs_info, "failed to init sysfs device interface: %d",
+ ret);
goto fail_fsdev_sysfs;
}
ret = btrfs_sysfs_add_mounted(fs_info);
if (ret) {
- pr_err("BTRFS: failed to init sysfs interface: %d\n", ret);
+ btrfs_err(fs_info, "failed to init sysfs interface: %d", ret);
goto fail_fsdev_sysfs;
}
ret = btrfs_init_space_info(fs_info);
if (ret) {
- printk(KERN_ERR "BTRFS: Failed to initial space info: %d\n", ret);
+ btrfs_err(fs_info, "failed to initialize space info: %d", ret);
goto fail_sysfs;
}
ret = btrfs_read_block_groups(fs_info->extent_root);
if (ret) {
- printk(KERN_ERR "BTRFS: Failed to read block groups: %d\n", ret);
+ btrfs_err(fs_info, "failed to read block groups: %d", ret);
goto fail_sysfs;
}
fs_info->num_tolerated_disk_barrier_failures =
@@ -2991,12 +2989,20 @@
if (fs_info->fs_devices->missing_devices >
fs_info->num_tolerated_disk_barrier_failures &&
!(sb->s_flags & MS_RDONLY)) {
- pr_warn("BTRFS: missing devices(%llu) exceeds the limit(%d), writeable mount is not allowed\n",
+ btrfs_warn(fs_info,
+"missing devices (%llu) exceeds the limit (%d), writeable mount is not allowed",
fs_info->fs_devices->missing_devices,
fs_info->num_tolerated_disk_barrier_failures);
goto fail_sysfs;
}
+ /*
+ * Hold the cleaner_mutex thread here so that we don't block
+ * for a long time on btrfs_recover_relocation. cleaner_kthread
+ * will wait for us to finish mounting the filesystem.
+ */
+ mutex_lock(&fs_info->cleaner_mutex);
+ cleaner_mutex_locked = true;
fs_info->cleaner_kthread = kthread_run(cleaner_kthread, tree_root,
"btrfs-cleaner");
if (IS_ERR(fs_info->cleaner_kthread))
@@ -3011,8 +3017,7 @@
if (!btrfs_test_opt(tree_root, SSD) &&
!btrfs_test_opt(tree_root, NOSSD) &&
!fs_info->fs_devices->rotating) {
- printk(KERN_INFO "BTRFS: detected SSD devices, enabling SSD "
- "mode\n");
+ btrfs_info(fs_info, "detected SSD devices, enabling SSD mode");
btrfs_set_opt(fs_info->mount_opt, SSD);
}
@@ -3030,8 +3035,9 @@
1 : 0,
fs_info->check_integrity_print_mask);
if (ret)
- printk(KERN_WARNING "BTRFS: failed to initialize"
- " integrity check module %s\n", sb->s_id);
+ btrfs_warn(fs_info,
+ "failed to initialize integrity check module: %d",
+ ret);
}
#endif
ret = btrfs_read_qgroup_config(fs_info);
@@ -3056,17 +3062,17 @@
ret = btrfs_cleanup_fs_roots(fs_info);
if (ret)
goto fail_qgroup;
-
- mutex_lock(&fs_info->cleaner_mutex);
+ /* We locked cleaner_mutex before creating cleaner_kthread. */
ret = btrfs_recover_relocation(tree_root);
- mutex_unlock(&fs_info->cleaner_mutex);
if (ret < 0) {
- printk(KERN_WARNING
- "BTRFS: failed to recover relocation\n");
+ btrfs_warn(fs_info, "failed to recover relocation: %d",
+ ret);
err = -EINVAL;
goto fail_qgroup;
}
}
+ mutex_unlock(&fs_info->cleaner_mutex);
+ cleaner_mutex_locked = false;
location.objectid = BTRFS_FS_TREE_OBJECTID;
location.type = BTRFS_ROOT_ITEM_KEY;
@@ -3083,11 +3089,11 @@
if (btrfs_test_opt(tree_root, FREE_SPACE_TREE) &&
!btrfs_fs_compat_ro(fs_info, FREE_SPACE_TREE)) {
- pr_info("BTRFS: creating free space tree\n");
+ btrfs_info(fs_info, "creating free space tree");
ret = btrfs_create_free_space_tree(fs_info);
if (ret) {
- pr_warn("BTRFS: failed to create free space tree %d\n",
- ret);
+ btrfs_warn(fs_info,
+ "failed to create free space tree: %d", ret);
close_ctree(tree_root);
return ret;
}
@@ -3104,14 +3110,14 @@
ret = btrfs_resume_balance_async(fs_info);
if (ret) {
- printk(KERN_WARNING "BTRFS: failed to resume balance\n");
+ btrfs_warn(fs_info, "failed to resume balance: %d", ret);
close_ctree(tree_root);
return ret;
}
ret = btrfs_resume_dev_replace_async(fs_info);
if (ret) {
- pr_warn("BTRFS: failed to resume dev_replace\n");
+ btrfs_warn(fs_info, "failed to resume device replace: %d", ret);
close_ctree(tree_root);
return ret;
}
@@ -3120,33 +3126,33 @@
if (btrfs_test_opt(tree_root, CLEAR_CACHE) &&
btrfs_fs_compat_ro(fs_info, FREE_SPACE_TREE)) {
- pr_info("BTRFS: clearing free space tree\n");
+ btrfs_info(fs_info, "clearing free space tree");
ret = btrfs_clear_free_space_tree(fs_info);
if (ret) {
- pr_warn("BTRFS: failed to clear free space tree %d\n",
- ret);
+ btrfs_warn(fs_info,
+ "failed to clear free space tree: %d", ret);
close_ctree(tree_root);
return ret;
}
}
if (!fs_info->uuid_root) {
- pr_info("BTRFS: creating UUID tree\n");
+ btrfs_info(fs_info, "creating UUID tree");
ret = btrfs_create_uuid_tree(fs_info);
if (ret) {
- pr_warn("BTRFS: failed to create the UUID tree %d\n",
- ret);
+ btrfs_warn(fs_info,
+ "failed to create the UUID tree: %d", ret);
close_ctree(tree_root);
return ret;
}
} else if (btrfs_test_opt(tree_root, RESCAN_UUID_TREE) ||
fs_info->generation !=
btrfs_super_uuid_tree_generation(disk_super)) {
- pr_info("BTRFS: checking UUID tree\n");
+ btrfs_info(fs_info, "checking UUID tree");
ret = btrfs_check_uuid_tree(fs_info);
if (ret) {
- pr_warn("BTRFS: failed to check the UUID tree %d\n",
- ret);
+ btrfs_warn(fs_info,
+ "failed to check the UUID tree: %d", ret);
close_ctree(tree_root);
return ret;
}
@@ -3180,6 +3186,10 @@
filemap_write_and_wait(fs_info->btree_inode->i_mapping);
fail_sysfs:
+ if (cleaner_mutex_locked) {
+ mutex_unlock(&fs_info->cleaner_mutex);
+ cleaner_mutex_locked = false;
+ }
btrfs_sysfs_remove_mounted(fs_info);
fail_fsdev_sysfs:
@@ -3646,7 +3656,7 @@
if (ret) {
mutex_unlock(
&root->fs_info->fs_devices->device_list_mutex);
- btrfs_std_error(root->fs_info, ret,
+ btrfs_handle_fs_error(root->fs_info, ret,
"errors while submitting device barriers.");
return ret;
}
@@ -3686,7 +3696,7 @@
mutex_unlock(&root->fs_info->fs_devices->device_list_mutex);
/* FUA is masked off if unsupported and can't be the reason */
- btrfs_std_error(root->fs_info, -EIO,
+ btrfs_handle_fs_error(root->fs_info, -EIO,
"%d errors while writing supers", total_errors);
return -EIO;
}
@@ -3704,7 +3714,7 @@
}
mutex_unlock(&root->fs_info->fs_devices->device_list_mutex);
if (total_errors > max_errors) {
- btrfs_std_error(root->fs_info, -EIO,
+ btrfs_handle_fs_error(root->fs_info, -EIO,
"%d errors while writing supers", total_errors);
return -EIO;
}
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 84e060e..9424864 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -3824,6 +3824,59 @@
return readonly;
}
+bool btrfs_inc_nocow_writers(struct btrfs_fs_info *fs_info, u64 bytenr)
+{
+ struct btrfs_block_group_cache *bg;
+ bool ret = true;
+
+ bg = btrfs_lookup_block_group(fs_info, bytenr);
+ if (!bg)
+ return false;
+
+ spin_lock(&bg->lock);
+ if (bg->ro)
+ ret = false;
+ else
+ atomic_inc(&bg->nocow_writers);
+ spin_unlock(&bg->lock);
+
+ /* no put on block group, done by btrfs_dec_nocow_writers */
+ if (!ret)
+ btrfs_put_block_group(bg);
+
+ return ret;
+
+}
+
+void btrfs_dec_nocow_writers(struct btrfs_fs_info *fs_info, u64 bytenr)
+{
+ struct btrfs_block_group_cache *bg;
+
+ bg = btrfs_lookup_block_group(fs_info, bytenr);
+ ASSERT(bg);
+ if (atomic_dec_and_test(&bg->nocow_writers))
+ wake_up_atomic_t(&bg->nocow_writers);
+ /*
+ * Once for our lookup and once for the lookup done by a previous call
+ * to btrfs_inc_nocow_writers()
+ */
+ btrfs_put_block_group(bg);
+ btrfs_put_block_group(bg);
+}
+
+static int btrfs_wait_nocow_writers_atomic_t(atomic_t *a)
+{
+ schedule();
+ return 0;
+}
+
+void btrfs_wait_nocow_writers(struct btrfs_block_group_cache *bg)
+{
+ wait_on_atomic_t(&bg->nocow_writers,
+ btrfs_wait_nocow_writers_atomic_t,
+ TASK_UNINTERRUPTIBLE);
+}
+
static const char *alloc_name(u64 flags)
{
switch (flags) {
@@ -4141,7 +4194,7 @@
if (need_commit > 0) {
btrfs_start_delalloc_roots(fs_info, 0, -1);
- btrfs_wait_ordered_roots(fs_info, -1);
+ btrfs_wait_ordered_roots(fs_info, -1, 0, (u64)-1);
}
trans = btrfs_join_transaction(root);
@@ -4583,7 +4636,8 @@
*/
btrfs_start_delalloc_roots(root->fs_info, 0, nr_items);
if (!current->journal_info)
- btrfs_wait_ordered_roots(root->fs_info, nr_items);
+ btrfs_wait_ordered_roots(root->fs_info, nr_items,
+ 0, (u64)-1);
}
}
@@ -4620,7 +4674,7 @@
/* Calc the number of the pages we need flush for space reservation */
items = calc_reclaim_items_nr(root, to_reclaim);
- to_reclaim = items * EXTENT_SIZE_PER_ITEM;
+ to_reclaim = (u64)items * EXTENT_SIZE_PER_ITEM;
trans = (struct btrfs_trans_handle *)current->journal_info;
block_rsv = &root->fs_info->delalloc_block_rsv;
@@ -4632,7 +4686,8 @@
if (trans)
return;
if (wait_ordered)
- btrfs_wait_ordered_roots(root->fs_info, items);
+ btrfs_wait_ordered_roots(root->fs_info, items,
+ 0, (u64)-1);
return;
}
@@ -4671,7 +4726,8 @@
loops++;
if (wait_ordered && !trans) {
- btrfs_wait_ordered_roots(root->fs_info, items);
+ btrfs_wait_ordered_roots(root->fs_info, items,
+ 0, (u64)-1);
} else {
time_left = schedule_timeout_killable(1);
if (time_left)
@@ -6172,6 +6228,57 @@
return 0;
}
+static void
+btrfs_inc_block_group_reservations(struct btrfs_block_group_cache *bg)
+{
+ atomic_inc(&bg->reservations);
+}
+
+void btrfs_dec_block_group_reservations(struct btrfs_fs_info *fs_info,
+ const u64 start)
+{
+ struct btrfs_block_group_cache *bg;
+
+ bg = btrfs_lookup_block_group(fs_info, start);
+ ASSERT(bg);
+ if (atomic_dec_and_test(&bg->reservations))
+ wake_up_atomic_t(&bg->reservations);
+ btrfs_put_block_group(bg);
+}
+
+static int btrfs_wait_bg_reservations_atomic_t(atomic_t *a)
+{
+ schedule();
+ return 0;
+}
+
+void btrfs_wait_block_group_reservations(struct btrfs_block_group_cache *bg)
+{
+ struct btrfs_space_info *space_info = bg->space_info;
+
+ ASSERT(bg->ro);
+
+ if (!(bg->flags & BTRFS_BLOCK_GROUP_DATA))
+ return;
+
+ /*
+ * Our block group is read only but before we set it to read only,
+ * some task might have had allocated an extent from it already, but it
+ * has not yet created a respective ordered extent (and added it to a
+ * root's list of ordered extents).
+ * Therefore wait for any task currently allocating extents, since the
+ * block group's reservations counter is incremented while a read lock
+ * on the groups' semaphore is held and decremented after releasing
+ * the read access on that semaphore and creating the ordered extent.
+ */
+ down_write(&space_info->groups_sem);
+ up_write(&space_info->groups_sem);
+
+ wait_on_atomic_t(&bg->reservations,
+ btrfs_wait_bg_reservations_atomic_t,
+ TASK_UNINTERRUPTIBLE);
+}
+
/**
* btrfs_update_reserved_bytes - update the block_group and space info counters
* @cache: The cache we are manipulating
@@ -7025,36 +7132,35 @@
int delalloc)
{
struct btrfs_block_group_cache *used_bg = NULL;
- bool locked = false;
-again:
+
spin_lock(&cluster->refill_lock);
- if (locked) {
+ while (1) {
+ used_bg = cluster->block_group;
+ if (!used_bg)
+ return NULL;
+
+ if (used_bg == block_group)
+ return used_bg;
+
+ btrfs_get_block_group(used_bg);
+
+ if (!delalloc)
+ return used_bg;
+
+ if (down_read_trylock(&used_bg->data_rwsem))
+ return used_bg;
+
+ spin_unlock(&cluster->refill_lock);
+
+ down_read(&used_bg->data_rwsem);
+
+ spin_lock(&cluster->refill_lock);
if (used_bg == cluster->block_group)
return used_bg;
up_read(&used_bg->data_rwsem);
btrfs_put_block_group(used_bg);
}
-
- used_bg = cluster->block_group;
- if (!used_bg)
- return NULL;
-
- if (used_bg == block_group)
- return used_bg;
-
- btrfs_get_block_group(used_bg);
-
- if (!delalloc)
- return used_bg;
-
- if (down_read_trylock(&used_bg->data_rwsem))
- return used_bg;
-
- spin_unlock(&cluster->refill_lock);
- down_read(&used_bg->data_rwsem);
- locked = true;
- goto again;
}
static inline void
@@ -7431,6 +7537,7 @@
btrfs_add_free_space(block_group, offset, num_bytes);
goto loop;
}
+ btrfs_inc_block_group_reservations(block_group);
/* we are all good, lets return */
ins->objectid = search_start;
@@ -7612,8 +7719,10 @@
WARN_ON(num_bytes < root->sectorsize);
ret = find_free_extent(root, num_bytes, empty_size, hint_byte, ins,
flags, delalloc);
-
- if (ret == -ENOSPC) {
+ if (!ret && !is_data) {
+ btrfs_dec_block_group_reservations(root->fs_info,
+ ins->objectid);
+ } else if (ret == -ENOSPC) {
if (!final_tried && ins->offset) {
num_bytes = min(num_bytes >> 1, ins->offset);
num_bytes = round_down(num_bytes, root->sectorsize);
@@ -9058,7 +9167,7 @@
if (!for_reloc && root_dropped == false)
btrfs_add_dead_root(root);
if (err && err != -EAGAIN)
- btrfs_std_error(root->fs_info, err, NULL);
+ btrfs_handle_fs_error(root->fs_info, err, NULL);
return err;
}
diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index d247fc0..2f83448 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -3200,14 +3200,10 @@
return ret;
}
-static noinline void update_nr_written(struct page *page,
- struct writeback_control *wbc,
- unsigned long nr_written)
+static void update_nr_written(struct page *page, struct writeback_control *wbc,
+ unsigned long nr_written)
{
wbc->nr_to_write -= nr_written;
- if (wbc->range_cyclic || (wbc->nr_to_write > 0 &&
- wbc->range_start == 0 && wbc->range_end == LLONG_MAX))
- page->mapping->writeback_index = page->index + nr_written;
}
/*
@@ -3368,6 +3364,8 @@
while (cur <= end) {
u64 em_end;
+ unsigned long max_nr;
+
if (cur >= i_size) {
if (tree->ops && tree->ops->writepage_end_io_hook)
tree->ops->writepage_end_io_hook(page, cur,
@@ -3423,32 +3421,23 @@
continue;
}
- if (tree->ops && tree->ops->writepage_io_hook) {
- ret = tree->ops->writepage_io_hook(page, cur,
- cur + iosize - 1);
- } else {
- ret = 0;
+ max_nr = (i_size >> PAGE_SHIFT) + 1;
+
+ set_range_writeback(tree, cur, cur + iosize - 1);
+ if (!PageWriteback(page)) {
+ btrfs_err(BTRFS_I(inode)->root->fs_info,
+ "page %lu not writeback, cur %llu end %llu",
+ page->index, cur, end);
}
- if (ret) {
+
+ ret = submit_extent_page(write_flags, tree, wbc, page,
+ sector, iosize, pg_offset,
+ bdev, &epd->bio, max_nr,
+ end_bio_extent_writepage,
+ 0, 0, 0, false);
+ if (ret)
SetPageError(page);
- } else {
- unsigned long max_nr = (i_size >> PAGE_SHIFT) + 1;
- set_range_writeback(tree, cur, cur + iosize - 1);
- if (!PageWriteback(page)) {
- btrfs_err(BTRFS_I(inode)->root->fs_info,
- "page %lu not writeback, cur %llu end %llu",
- page->index, cur, end);
- }
-
- ret = submit_extent_page(write_flags, tree, wbc, page,
- sector, iosize, pg_offset,
- bdev, &epd->bio, max_nr,
- end_bio_extent_writepage,
- 0, 0, 0, false);
- if (ret)
- SetPageError(page);
- }
cur = cur + iosize;
pg_offset += iosize;
nr++;
@@ -3920,12 +3909,13 @@
struct inode *inode = mapping->host;
int ret = 0;
int done = 0;
- int err = 0;
int nr_to_write_done = 0;
struct pagevec pvec;
int nr_pages;
pgoff_t index;
pgoff_t end; /* Inclusive */
+ pgoff_t done_index;
+ int range_whole = 0;
int scanned = 0;
int tag;
@@ -3948,6 +3938,8 @@
} else {
index = wbc->range_start >> PAGE_SHIFT;
end = wbc->range_end >> PAGE_SHIFT;
+ if (wbc->range_start == 0 && wbc->range_end == LLONG_MAX)
+ range_whole = 1;
scanned = 1;
}
if (wbc->sync_mode == WB_SYNC_ALL)
@@ -3957,6 +3949,7 @@
retry:
if (wbc->sync_mode == WB_SYNC_ALL)
tag_pages_for_writeback(mapping, index, end);
+ done_index = index;
while (!done && !nr_to_write_done && (index <= end) &&
(nr_pages = pagevec_lookup_tag(&pvec, mapping, &index, tag,
min(end - index, (pgoff_t)PAGEVEC_SIZE-1) + 1))) {
@@ -3966,6 +3959,7 @@
for (i = 0; i < nr_pages; i++) {
struct page *page = pvec.pages[i];
+ done_index = page->index;
/*
* At this point we hold neither mapping->tree_lock nor
* lock on the page itself: the page may be truncated or
@@ -4007,8 +4001,20 @@
unlock_page(page);
ret = 0;
}
- if (!err && ret < 0)
- err = ret;
+ if (ret < 0) {
+ /*
+ * done_index is set past this page,
+ * so media errors will not choke
+ * background writeout for the entire
+ * file. This has consequences for
+ * range_cyclic semantics (ie. it may
+ * not be suitable for data integrity
+ * writeout).
+ */
+ done_index = page->index + 1;
+ done = 1;
+ break;
+ }
/*
* the filesystem may choose to bump up nr_to_write.
@@ -4020,7 +4026,7 @@
pagevec_release(&pvec);
cond_resched();
}
- if (!scanned && !done && !err) {
+ if (!scanned && !done) {
/*
* We hit the last page and there is more work to be done: wrap
* back to the start of the file
@@ -4029,8 +4035,12 @@
index = 0;
goto retry;
}
+
+ if (wbc->range_cyclic || (wbc->nr_to_write > 0 && range_whole))
+ mapping->writeback_index = done_index;
+
btrfs_add_delayed_iput(inode);
- return err;
+ return ret;
}
static void flush_epd_write_bio(struct extent_page_data *epd)
@@ -4822,7 +4832,7 @@
return NULL;
eb->fs_info = fs_info;
again:
- ret = radix_tree_preload(GFP_NOFS & ~__GFP_HIGHMEM);
+ ret = radix_tree_preload(GFP_NOFS);
if (ret)
goto free_eb;
spin_lock(&fs_info->buffer_lock);
@@ -4923,7 +4933,7 @@
if (uptodate)
set_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags);
again:
- ret = radix_tree_preload(GFP_NOFS & ~__GFP_HIGHMEM);
+ ret = radix_tree_preload(GFP_NOFS);
if (ret)
goto free_eb;
diff --git a/fs/btrfs/extent_io.h b/fs/btrfs/extent_io.h
index b5e0ade..981f402 100644
--- a/fs/btrfs/extent_io.h
+++ b/fs/btrfs/extent_io.h
@@ -71,7 +71,6 @@
u64 start, u64 end, int *page_started,
unsigned long *nr_written);
int (*writepage_start_hook)(struct page *page, u64 start, u64 end);
- int (*writepage_io_hook)(struct page *page, u64 start, u64 end);
extent_submit_bio_hook_t *submit_bio_hook;
int (*merge_bio_hook)(int rw, struct page *page, unsigned long offset,
size_t size, struct bio *bio,
diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c
index ea9f10b..c98805c 100644
--- a/fs/btrfs/file.c
+++ b/fs/btrfs/file.c
@@ -1696,7 +1696,9 @@
btrfs_end_write_no_snapshoting(root);
btrfs_delalloc_release_metadata(inode, release_bytes);
} else {
- btrfs_delalloc_release_space(inode, pos, release_bytes);
+ btrfs_delalloc_release_space(inode,
+ round_down(pos, root->sectorsize),
+ release_bytes);
}
}
@@ -2952,7 +2954,7 @@
.fallocate = btrfs_fallocate,
.unlocked_ioctl = btrfs_ioctl,
#ifdef CONFIG_COMPAT
- .compat_ioctl = btrfs_ioctl,
+ .compat_ioctl = btrfs_compat_ioctl,
#endif
.copy_file_range = btrfs_copy_file_range,
.clone_file_range = btrfs_clone_file_range,
diff --git a/fs/btrfs/inode-item.c b/fs/btrfs/inode-item.c
index be4d22a..b8acc07 100644
--- a/fs/btrfs/inode-item.c
+++ b/fs/btrfs/inode-item.c
@@ -157,7 +157,7 @@
*/
if (!btrfs_find_name_in_ext_backref(path, ref_objectid,
name, name_len, &extref)) {
- btrfs_std_error(root->fs_info, -ENOENT, NULL);
+ btrfs_handle_fs_error(root->fs_info, -ENOENT, NULL);
ret = -EROFS;
goto out;
}
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 6b7fe29..91419ef 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -824,6 +824,7 @@
async_extent->ram_size - 1, 0);
goto out_free_reserve;
}
+ btrfs_dec_block_group_reservations(root->fs_info, ins.objectid);
/*
* clear dirty, set writeback and unlock the pages.
@@ -861,6 +862,7 @@
}
return;
out_free_reserve:
+ btrfs_dec_block_group_reservations(root->fs_info, ins.objectid);
btrfs_free_reserved_extent(root, ins.objectid, ins.offset, 1);
out_free:
extent_clear_unlock_delalloc(inode, async_extent->start,
@@ -1038,6 +1040,8 @@
goto out_drop_extent_cache;
}
+ btrfs_dec_block_group_reservations(root->fs_info, ins.objectid);
+
if (disk_num_bytes < cur_alloc_size)
break;
@@ -1066,6 +1070,7 @@
out_drop_extent_cache:
btrfs_drop_extent_cache(inode, start, start + ram_size - 1, 0);
out_reserve:
+ btrfs_dec_block_group_reservations(root->fs_info, ins.objectid);
btrfs_free_reserved_extent(root, ins.objectid, ins.offset, 1);
out_unlock:
extent_clear_unlock_delalloc(inode, start, end, locked_page,
@@ -1377,6 +1382,9 @@
*/
if (csum_exist_in_range(root, disk_bytenr, num_bytes))
goto out_check;
+ if (!btrfs_inc_nocow_writers(root->fs_info,
+ disk_bytenr))
+ goto out_check;
nocow = 1;
} else if (extent_type == BTRFS_FILE_EXTENT_INLINE) {
extent_end = found_key.offset +
@@ -1391,6 +1399,9 @@
path->slots[0]++;
if (!nolock && nocow)
btrfs_end_write_no_snapshoting(root);
+ if (nocow)
+ btrfs_dec_nocow_writers(root->fs_info,
+ disk_bytenr);
goto next_slot;
}
if (!nocow) {
@@ -1411,6 +1422,9 @@
if (ret) {
if (!nolock && nocow)
btrfs_end_write_no_snapshoting(root);
+ if (nocow)
+ btrfs_dec_nocow_writers(root->fs_info,
+ disk_bytenr);
goto error;
}
cow_start = (u64)-1;
@@ -1453,6 +1467,8 @@
ret = btrfs_add_ordered_extent(inode, cur_offset, disk_bytenr,
num_bytes, num_bytes, type);
+ if (nocow)
+ btrfs_dec_nocow_writers(root->fs_info, disk_bytenr);
BUG_ON(ret); /* -ENOMEM */
if (root->root_key.objectid ==
@@ -7129,6 +7145,43 @@
return em;
}
+static struct extent_map *btrfs_create_dio_extent(struct inode *inode,
+ const u64 start,
+ const u64 len,
+ const u64 orig_start,
+ const u64 block_start,
+ const u64 block_len,
+ const u64 orig_block_len,
+ const u64 ram_bytes,
+ const int type)
+{
+ struct extent_map *em = NULL;
+ int ret;
+
+ down_read(&BTRFS_I(inode)->dio_sem);
+ if (type != BTRFS_ORDERED_NOCOW) {
+ em = create_pinned_em(inode, start, len, orig_start,
+ block_start, block_len, orig_block_len,
+ ram_bytes, type);
+ if (IS_ERR(em))
+ goto out;
+ }
+ ret = btrfs_add_ordered_extent_dio(inode, start, block_start,
+ len, block_len, type);
+ if (ret) {
+ if (em) {
+ free_extent_map(em);
+ btrfs_drop_extent_cache(inode, start,
+ start + len - 1, 0);
+ }
+ em = ERR_PTR(ret);
+ }
+ out:
+ up_read(&BTRFS_I(inode)->dio_sem);
+
+ return em;
+}
+
static struct extent_map *btrfs_new_extent_direct(struct inode *inode,
u64 start, u64 len)
{
@@ -7144,41 +7197,13 @@
if (ret)
return ERR_PTR(ret);
- /*
- * Create the ordered extent before the extent map. This is to avoid
- * races with the fast fsync path that would lead to it logging file
- * extent items that point to disk extents that were not yet written to.
- * The fast fsync path collects ordered extents into a local list and
- * then collects all the new extent maps, so we must create the ordered
- * extent first and make sure the fast fsync path collects any new
- * ordered extents after collecting new extent maps as well.
- * The fsync path simply can not rely on inode_dio_wait() because it
- * causes deadlock with AIO.
- */
- ret = btrfs_add_ordered_extent_dio(inode, start, ins.objectid,
- ins.offset, ins.offset, 0);
- if (ret) {
+ em = btrfs_create_dio_extent(inode, start, ins.offset, start,
+ ins.objectid, ins.offset, ins.offset,
+ ins.offset, 0);
+ btrfs_dec_block_group_reservations(root->fs_info, ins.objectid);
+ if (IS_ERR(em))
btrfs_free_reserved_extent(root, ins.objectid, ins.offset, 1);
- return ERR_PTR(ret);
- }
- em = create_pinned_em(inode, start, ins.offset, start, ins.objectid,
- ins.offset, ins.offset, ins.offset, 0);
- if (IS_ERR(em)) {
- struct btrfs_ordered_extent *oe;
-
- btrfs_free_reserved_extent(root, ins.objectid, ins.offset, 1);
- oe = btrfs_lookup_ordered_extent(inode, start);
- ASSERT(oe);
- if (WARN_ON(!oe))
- return em;
- set_bit(BTRFS_ORDERED_IOERR, &oe->flags);
- set_bit(BTRFS_ORDERED_IO_DONE, &oe->flags);
- btrfs_remove_ordered_extent(inode, oe);
- /* Once for our lookup and once for the ordered extents tree. */
- btrfs_put_ordered_extent(oe);
- btrfs_put_ordered_extent(oe);
- }
return em;
}
@@ -7650,24 +7675,21 @@
block_start = em->block_start + (start - em->start);
if (can_nocow_extent(inode, start, &len, &orig_start,
- &orig_block_len, &ram_bytes) == 1) {
+ &orig_block_len, &ram_bytes) == 1 &&
+ btrfs_inc_nocow_writers(root->fs_info, block_start)) {
+ struct extent_map *em2;
+
+ em2 = btrfs_create_dio_extent(inode, start, len,
+ orig_start, block_start,
+ len, orig_block_len,
+ ram_bytes, type);
+ btrfs_dec_nocow_writers(root->fs_info, block_start);
if (type == BTRFS_ORDERED_PREALLOC) {
free_extent_map(em);
- em = create_pinned_em(inode, start, len,
- orig_start,
- block_start, len,
- orig_block_len,
- ram_bytes, type);
- if (IS_ERR(em)) {
- ret = PTR_ERR(em);
- goto unlock_err;
- }
+ em = em2;
}
-
- ret = btrfs_add_ordered_extent_dio(inode, start,
- block_start, len, len, type);
- if (ret) {
- free_extent_map(em);
+ if (em2 && IS_ERR(em2)) {
+ ret = PTR_ERR(em2);
goto unlock_err;
}
goto unlock;
@@ -9230,6 +9252,7 @@
INIT_LIST_HEAD(&ei->delalloc_inodes);
INIT_LIST_HEAD(&ei->delayed_iput);
RB_CLEAR_NODE(&ei->rb_node);
+ init_rwsem(&ei->dio_sem);
return inode;
}
@@ -9387,18 +9410,290 @@
return 0;
}
-static int btrfs_rename(struct inode *old_dir, struct dentry *old_dentry,
- struct inode *new_dir, struct dentry *new_dentry)
+static int btrfs_rename_exchange(struct inode *old_dir,
+ struct dentry *old_dentry,
+ struct inode *new_dir,
+ struct dentry *new_dentry)
{
struct btrfs_trans_handle *trans;
struct btrfs_root *root = BTRFS_I(old_dir)->root;
struct btrfs_root *dest = BTRFS_I(new_dir)->root;
+ struct inode *new_inode = new_dentry->d_inode;
+ struct inode *old_inode = old_dentry->d_inode;
+ struct timespec ctime = CURRENT_TIME;
+ struct dentry *parent;
+ u64 old_ino = btrfs_ino(old_inode);
+ u64 new_ino = btrfs_ino(new_inode);
+ u64 old_idx = 0;
+ u64 new_idx = 0;
+ u64 root_objectid;
+ int ret;
+ bool root_log_pinned = false;
+ bool dest_log_pinned = false;
+
+ /* we only allow rename subvolume link between subvolumes */
+ if (old_ino != BTRFS_FIRST_FREE_OBJECTID && root != dest)
+ return -EXDEV;
+
+ /* close the race window with snapshot create/destroy ioctl */
+ if (old_ino == BTRFS_FIRST_FREE_OBJECTID)
+ down_read(&root->fs_info->subvol_sem);
+ if (new_ino == BTRFS_FIRST_FREE_OBJECTID)
+ down_read(&dest->fs_info->subvol_sem);
+
+ /*
+ * We want to reserve the absolute worst case amount of items. So if
+ * both inodes are subvols and we need to unlink them then that would
+ * require 4 item modifications, but if they are both normal inodes it
+ * would require 5 item modifications, so we'll assume their normal
+ * inodes. So 5 * 2 is 10, plus 2 for the new links, so 12 total items
+ * should cover the worst case number of items we'll modify.
+ */
+ trans = btrfs_start_transaction(root, 12);
+ if (IS_ERR(trans)) {
+ ret = PTR_ERR(trans);
+ goto out_notrans;
+ }
+
+ /*
+ * We need to find a free sequence number both in the source and
+ * in the destination directory for the exchange.
+ */
+ ret = btrfs_set_inode_index(new_dir, &old_idx);
+ if (ret)
+ goto out_fail;
+ ret = btrfs_set_inode_index(old_dir, &new_idx);
+ if (ret)
+ goto out_fail;
+
+ BTRFS_I(old_inode)->dir_index = 0ULL;
+ BTRFS_I(new_inode)->dir_index = 0ULL;
+
+ /* Reference for the source. */
+ if (old_ino == BTRFS_FIRST_FREE_OBJECTID) {
+ /* force full log commit if subvolume involved. */
+ btrfs_set_log_full_commit(root->fs_info, trans);
+ } else {
+ btrfs_pin_log_trans(root);
+ root_log_pinned = true;
+ ret = btrfs_insert_inode_ref(trans, dest,
+ new_dentry->d_name.name,
+ new_dentry->d_name.len,
+ old_ino,
+ btrfs_ino(new_dir), old_idx);
+ if (ret)
+ goto out_fail;
+ }
+
+ /* And now for the dest. */
+ if (new_ino == BTRFS_FIRST_FREE_OBJECTID) {
+ /* force full log commit if subvolume involved. */
+ btrfs_set_log_full_commit(dest->fs_info, trans);
+ } else {
+ btrfs_pin_log_trans(dest);
+ dest_log_pinned = true;
+ ret = btrfs_insert_inode_ref(trans, root,
+ old_dentry->d_name.name,
+ old_dentry->d_name.len,
+ new_ino,
+ btrfs_ino(old_dir), new_idx);
+ if (ret)
+ goto out_fail;
+ }
+
+ /* Update inode version and ctime/mtime. */
+ inode_inc_iversion(old_dir);
+ inode_inc_iversion(new_dir);
+ inode_inc_iversion(old_inode);
+ inode_inc_iversion(new_inode);
+ old_dir->i_ctime = old_dir->i_mtime = ctime;
+ new_dir->i_ctime = new_dir->i_mtime = ctime;
+ old_inode->i_ctime = ctime;
+ new_inode->i_ctime = ctime;
+
+ if (old_dentry->d_parent != new_dentry->d_parent) {
+ btrfs_record_unlink_dir(trans, old_dir, old_inode, 1);
+ btrfs_record_unlink_dir(trans, new_dir, new_inode, 1);
+ }
+
+ /* src is a subvolume */
+ if (old_ino == BTRFS_FIRST_FREE_OBJECTID) {
+ root_objectid = BTRFS_I(old_inode)->root->root_key.objectid;
+ ret = btrfs_unlink_subvol(trans, root, old_dir,
+ root_objectid,
+ old_dentry->d_name.name,
+ old_dentry->d_name.len);
+ } else { /* src is an inode */
+ ret = __btrfs_unlink_inode(trans, root, old_dir,
+ old_dentry->d_inode,
+ old_dentry->d_name.name,
+ old_dentry->d_name.len);
+ if (!ret)
+ ret = btrfs_update_inode(trans, root, old_inode);
+ }
+ if (ret) {
+ btrfs_abort_transaction(trans, root, ret);
+ goto out_fail;
+ }
+
+ /* dest is a subvolume */
+ if (new_ino == BTRFS_FIRST_FREE_OBJECTID) {
+ root_objectid = BTRFS_I(new_inode)->root->root_key.objectid;
+ ret = btrfs_unlink_subvol(trans, dest, new_dir,
+ root_objectid,
+ new_dentry->d_name.name,
+ new_dentry->d_name.len);
+ } else { /* dest is an inode */
+ ret = __btrfs_unlink_inode(trans, dest, new_dir,
+ new_dentry->d_inode,
+ new_dentry->d_name.name,
+ new_dentry->d_name.len);
+ if (!ret)
+ ret = btrfs_update_inode(trans, dest, new_inode);
+ }
+ if (ret) {
+ btrfs_abort_transaction(trans, root, ret);
+ goto out_fail;
+ }
+
+ ret = btrfs_add_link(trans, new_dir, old_inode,
+ new_dentry->d_name.name,
+ new_dentry->d_name.len, 0, old_idx);
+ if (ret) {
+ btrfs_abort_transaction(trans, root, ret);
+ goto out_fail;
+ }
+
+ ret = btrfs_add_link(trans, old_dir, new_inode,
+ old_dentry->d_name.name,
+ old_dentry->d_name.len, 0, new_idx);
+ if (ret) {
+ btrfs_abort_transaction(trans, root, ret);
+ goto out_fail;
+ }
+
+ if (old_inode->i_nlink == 1)
+ BTRFS_I(old_inode)->dir_index = old_idx;
+ if (new_inode->i_nlink == 1)
+ BTRFS_I(new_inode)->dir_index = new_idx;
+
+ if (root_log_pinned) {
+ parent = new_dentry->d_parent;
+ btrfs_log_new_name(trans, old_inode, old_dir, parent);
+ btrfs_end_log_trans(root);
+ root_log_pinned = false;
+ }
+ if (dest_log_pinned) {
+ parent = old_dentry->d_parent;
+ btrfs_log_new_name(trans, new_inode, new_dir, parent);
+ btrfs_end_log_trans(dest);
+ dest_log_pinned = false;
+ }
+out_fail:
+ /*
+ * If we have pinned a log and an error happened, we unpin tasks
+ * trying to sync the log and force them to fallback to a transaction
+ * commit if the log currently contains any of the inodes involved in
+ * this rename operation (to ensure we do not persist a log with an
+ * inconsistent state for any of these inodes or leading to any
+ * inconsistencies when replayed). If the transaction was aborted, the
+ * abortion reason is propagated to userspace when attempting to commit
+ * the transaction. If the log does not contain any of these inodes, we
+ * allow the tasks to sync it.
+ */
+ if (ret && (root_log_pinned || dest_log_pinned)) {
+ if (btrfs_inode_in_log(old_dir, root->fs_info->generation) ||
+ btrfs_inode_in_log(new_dir, root->fs_info->generation) ||
+ btrfs_inode_in_log(old_inode, root->fs_info->generation) ||
+ (new_inode &&
+ btrfs_inode_in_log(new_inode, root->fs_info->generation)))
+ btrfs_set_log_full_commit(root->fs_info, trans);
+
+ if (root_log_pinned) {
+ btrfs_end_log_trans(root);
+ root_log_pinned = false;
+ }
+ if (dest_log_pinned) {
+ btrfs_end_log_trans(dest);
+ dest_log_pinned = false;
+ }
+ }
+ ret = btrfs_end_transaction(trans, root);
+out_notrans:
+ if (new_ino == BTRFS_FIRST_FREE_OBJECTID)
+ up_read(&dest->fs_info->subvol_sem);
+ if (old_ino == BTRFS_FIRST_FREE_OBJECTID)
+ up_read(&root->fs_info->subvol_sem);
+
+ return ret;
+}
+
+static int btrfs_whiteout_for_rename(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ struct inode *dir,
+ struct dentry *dentry)
+{
+ int ret;
+ struct inode *inode;
+ u64 objectid;
+ u64 index;
+
+ ret = btrfs_find_free_ino(root, &objectid);
+ if (ret)
+ return ret;
+
+ inode = btrfs_new_inode(trans, root, dir,
+ dentry->d_name.name,
+ dentry->d_name.len,
+ btrfs_ino(dir),
+ objectid,
+ S_IFCHR | WHITEOUT_MODE,
+ &index);
+
+ if (IS_ERR(inode)) {
+ ret = PTR_ERR(inode);
+ return ret;
+ }
+
+ inode->i_op = &btrfs_special_inode_operations;
+ init_special_inode(inode, inode->i_mode,
+ WHITEOUT_DEV);
+
+ ret = btrfs_init_inode_security(trans, inode, dir,
+ &dentry->d_name);
+ if (ret)
+ goto out;
+
+ ret = btrfs_add_nondir(trans, dir, dentry,
+ inode, 0, index);
+ if (ret)
+ goto out;
+
+ ret = btrfs_update_inode(trans, root, inode);
+out:
+ unlock_new_inode(inode);
+ if (ret)
+ inode_dec_link_count(inode);
+ iput(inode);
+
+ return ret;
+}
+
+static int btrfs_rename(struct inode *old_dir, struct dentry *old_dentry,
+ struct inode *new_dir, struct dentry *new_dentry,
+ unsigned int flags)
+{
+ struct btrfs_trans_handle *trans;
+ unsigned int trans_num_items;
+ struct btrfs_root *root = BTRFS_I(old_dir)->root;
+ struct btrfs_root *dest = BTRFS_I(new_dir)->root;
struct inode *new_inode = d_inode(new_dentry);
struct inode *old_inode = d_inode(old_dentry);
u64 index = 0;
u64 root_objectid;
int ret;
u64 old_ino = btrfs_ino(old_inode);
+ bool log_pinned = false;
if (btrfs_ino(new_dir) == BTRFS_EMPTY_SUBVOL_DIR_OBJECTID)
return -EPERM;
@@ -9449,15 +9744,21 @@
* We want to reserve the absolute worst case amount of items. So if
* both inodes are subvols and we need to unlink them then that would
* require 4 item modifications, but if they are both normal inodes it
- * would require 5 item modifications, so we'll assume their normal
+ * would require 5 item modifications, so we'll assume they are normal
* inodes. So 5 * 2 is 10, plus 1 for the new link, so 11 total items
* should cover the worst case number of items we'll modify.
+ * If our rename has the whiteout flag, we need more 5 units for the
+ * new inode (1 inode item, 1 inode ref, 2 dir items and 1 xattr item
+ * when selinux is enabled).
*/
- trans = btrfs_start_transaction(root, 11);
+ trans_num_items = 11;
+ if (flags & RENAME_WHITEOUT)
+ trans_num_items += 5;
+ trans = btrfs_start_transaction(root, trans_num_items);
if (IS_ERR(trans)) {
- ret = PTR_ERR(trans);
- goto out_notrans;
- }
+ ret = PTR_ERR(trans);
+ goto out_notrans;
+ }
if (dest != root)
btrfs_record_root_in_trans(trans, dest);
@@ -9471,6 +9772,8 @@
/* force full log commit if subvolume involved. */
btrfs_set_log_full_commit(root->fs_info, trans);
} else {
+ btrfs_pin_log_trans(root);
+ log_pinned = true;
ret = btrfs_insert_inode_ref(trans, dest,
new_dentry->d_name.name,
new_dentry->d_name.len,
@@ -9478,14 +9781,6 @@
btrfs_ino(new_dir), index);
if (ret)
goto out_fail;
- /*
- * this is an ugly little race, but the rename is required
- * to make sure that if we crash, the inode is either at the
- * old name or the new one. pinning the log transaction lets
- * us make sure we don't allow a log commit to come in after
- * we unlink the name but before we add the new name back in.
- */
- btrfs_pin_log_trans(root);
}
inode_inc_iversion(old_dir);
@@ -9552,12 +9847,46 @@
if (old_inode->i_nlink == 1)
BTRFS_I(old_inode)->dir_index = index;
- if (old_ino != BTRFS_FIRST_FREE_OBJECTID) {
+ if (log_pinned) {
struct dentry *parent = new_dentry->d_parent;
+
btrfs_log_new_name(trans, old_inode, old_dir, parent);
btrfs_end_log_trans(root);
+ log_pinned = false;
+ }
+
+ if (flags & RENAME_WHITEOUT) {
+ ret = btrfs_whiteout_for_rename(trans, root, old_dir,
+ old_dentry);
+
+ if (ret) {
+ btrfs_abort_transaction(trans, root, ret);
+ goto out_fail;
+ }
}
out_fail:
+ /*
+ * If we have pinned the log and an error happened, we unpin tasks
+ * trying to sync the log and force them to fallback to a transaction
+ * commit if the log currently contains any of the inodes involved in
+ * this rename operation (to ensure we do not persist a log with an
+ * inconsistent state for any of these inodes or leading to any
+ * inconsistencies when replayed). If the transaction was aborted, the
+ * abortion reason is propagated to userspace when attempting to commit
+ * the transaction. If the log does not contain any of these inodes, we
+ * allow the tasks to sync it.
+ */
+ if (ret && log_pinned) {
+ if (btrfs_inode_in_log(old_dir, root->fs_info->generation) ||
+ btrfs_inode_in_log(new_dir, root->fs_info->generation) ||
+ btrfs_inode_in_log(old_inode, root->fs_info->generation) ||
+ (new_inode &&
+ btrfs_inode_in_log(new_inode, root->fs_info->generation)))
+ btrfs_set_log_full_commit(root->fs_info, trans);
+
+ btrfs_end_log_trans(root);
+ log_pinned = false;
+ }
btrfs_end_transaction(trans, root);
out_notrans:
if (old_ino == BTRFS_FIRST_FREE_OBJECTID)
@@ -9570,10 +9899,14 @@
struct inode *new_dir, struct dentry *new_dentry,
unsigned int flags)
{
- if (flags & ~RENAME_NOREPLACE)
+ if (flags & ~(RENAME_NOREPLACE | RENAME_EXCHANGE | RENAME_WHITEOUT))
return -EINVAL;
- return btrfs_rename(old_dir, old_dentry, new_dir, new_dentry);
+ if (flags & RENAME_EXCHANGE)
+ return btrfs_rename_exchange(old_dir, old_dentry, new_dir,
+ new_dentry);
+
+ return btrfs_rename(old_dir, old_dentry, new_dir, new_dentry, flags);
}
static void btrfs_run_delalloc_work(struct btrfs_work *work)
@@ -9942,6 +10275,7 @@
btrfs_end_transaction(trans, root);
break;
}
+ btrfs_dec_block_group_reservations(root->fs_info, ins.objectid);
last_alloc = ins.offset;
ret = insert_reserved_file_extent(trans, inode,
@@ -10184,7 +10518,7 @@
.iterate = btrfs_real_readdir,
.unlocked_ioctl = btrfs_ioctl,
#ifdef CONFIG_COMPAT
- .compat_ioctl = btrfs_ioctl,
+ .compat_ioctl = btrfs_compat_ioctl,
#endif
.release = btrfs_release_file,
.fsync = btrfs_sync_file,
diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
index 0b8ba71..4e70069 100644
--- a/fs/btrfs/ioctl.c
+++ b/fs/btrfs/ioctl.c
@@ -125,10 +125,10 @@
if (flags & BTRFS_INODE_NODATACOW)
iflags |= FS_NOCOW_FL;
- if ((flags & BTRFS_INODE_COMPRESS) && !(flags & BTRFS_INODE_NOCOMPRESS))
- iflags |= FS_COMPR_FL;
- else if (flags & BTRFS_INODE_NOCOMPRESS)
+ if (flags & BTRFS_INODE_NOCOMPRESS)
iflags |= FS_NOCOMP_FL;
+ else if (flags & BTRFS_INODE_COMPRESS)
+ iflags |= FS_COMPR_FL;
return iflags;
}
@@ -439,7 +439,7 @@
{
struct btrfs_trans_handle *trans;
struct btrfs_key key;
- struct btrfs_root_item root_item;
+ struct btrfs_root_item *root_item;
struct btrfs_inode_item *inode_item;
struct extent_buffer *leaf;
struct btrfs_root *root = BTRFS_I(dir)->root;
@@ -455,16 +455,22 @@
u64 qgroup_reserved;
uuid_le new_uuid;
+ root_item = kzalloc(sizeof(*root_item), GFP_KERNEL);
+ if (!root_item)
+ return -ENOMEM;
+
ret = btrfs_find_free_objectid(root->fs_info->tree_root, &objectid);
if (ret)
- return ret;
+ goto fail_free;
/*
* Don't create subvolume whose level is not zero. Or qgroup will be
* screwed up since it assume subvolme qgroup's level to be 0.
*/
- if (btrfs_qgroup_level(objectid))
- return -ENOSPC;
+ if (btrfs_qgroup_level(objectid)) {
+ ret = -ENOSPC;
+ goto fail_free;
+ }
btrfs_init_block_rsv(&block_rsv, BTRFS_BLOCK_RSV_TEMP);
/*
@@ -474,14 +480,14 @@
ret = btrfs_subvolume_reserve_metadata(root, &block_rsv,
8, &qgroup_reserved, false);
if (ret)
- return ret;
+ goto fail_free;
trans = btrfs_start_transaction(root, 0);
if (IS_ERR(trans)) {
ret = PTR_ERR(trans);
btrfs_subvolume_release_metadata(root, &block_rsv,
qgroup_reserved);
- return ret;
+ goto fail_free;
}
trans->block_rsv = &block_rsv;
trans->bytes_reserved = block_rsv.size;
@@ -509,47 +515,45 @@
BTRFS_UUID_SIZE);
btrfs_mark_buffer_dirty(leaf);
- memset(&root_item, 0, sizeof(root_item));
-
- inode_item = &root_item.inode;
+ inode_item = &root_item->inode;
btrfs_set_stack_inode_generation(inode_item, 1);
btrfs_set_stack_inode_size(inode_item, 3);
btrfs_set_stack_inode_nlink(inode_item, 1);
btrfs_set_stack_inode_nbytes(inode_item, root->nodesize);
btrfs_set_stack_inode_mode(inode_item, S_IFDIR | 0755);
- btrfs_set_root_flags(&root_item, 0);
- btrfs_set_root_limit(&root_item, 0);
+ btrfs_set_root_flags(root_item, 0);
+ btrfs_set_root_limit(root_item, 0);
btrfs_set_stack_inode_flags(inode_item, BTRFS_INODE_ROOT_ITEM_INIT);
- btrfs_set_root_bytenr(&root_item, leaf->start);
- btrfs_set_root_generation(&root_item, trans->transid);
- btrfs_set_root_level(&root_item, 0);
- btrfs_set_root_refs(&root_item, 1);
- btrfs_set_root_used(&root_item, leaf->len);
- btrfs_set_root_last_snapshot(&root_item, 0);
+ btrfs_set_root_bytenr(root_item, leaf->start);
+ btrfs_set_root_generation(root_item, trans->transid);
+ btrfs_set_root_level(root_item, 0);
+ btrfs_set_root_refs(root_item, 1);
+ btrfs_set_root_used(root_item, leaf->len);
+ btrfs_set_root_last_snapshot(root_item, 0);
- btrfs_set_root_generation_v2(&root_item,
- btrfs_root_generation(&root_item));
+ btrfs_set_root_generation_v2(root_item,
+ btrfs_root_generation(root_item));
uuid_le_gen(&new_uuid);
- memcpy(root_item.uuid, new_uuid.b, BTRFS_UUID_SIZE);
- btrfs_set_stack_timespec_sec(&root_item.otime, cur_time.tv_sec);
- btrfs_set_stack_timespec_nsec(&root_item.otime, cur_time.tv_nsec);
- root_item.ctime = root_item.otime;
- btrfs_set_root_ctransid(&root_item, trans->transid);
- btrfs_set_root_otransid(&root_item, trans->transid);
+ memcpy(root_item->uuid, new_uuid.b, BTRFS_UUID_SIZE);
+ btrfs_set_stack_timespec_sec(&root_item->otime, cur_time.tv_sec);
+ btrfs_set_stack_timespec_nsec(&root_item->otime, cur_time.tv_nsec);
+ root_item->ctime = root_item->otime;
+ btrfs_set_root_ctransid(root_item, trans->transid);
+ btrfs_set_root_otransid(root_item, trans->transid);
btrfs_tree_unlock(leaf);
free_extent_buffer(leaf);
leaf = NULL;
- btrfs_set_root_dirid(&root_item, new_dirid);
+ btrfs_set_root_dirid(root_item, new_dirid);
key.objectid = objectid;
key.offset = 0;
key.type = BTRFS_ROOT_ITEM_KEY;
ret = btrfs_insert_root(trans, root->fs_info->tree_root, &key,
- &root_item);
+ root_item);
if (ret)
goto fail;
@@ -601,12 +605,13 @@
BUG_ON(ret);
ret = btrfs_uuid_tree_add(trans, root->fs_info->uuid_root,
- root_item.uuid, BTRFS_UUID_KEY_SUBVOL,
+ root_item->uuid, BTRFS_UUID_KEY_SUBVOL,
objectid);
if (ret)
btrfs_abort_transaction(trans, root, ret);
fail:
+ kfree(root_item);
trans->block_rsv = NULL;
trans->bytes_reserved = 0;
btrfs_subvolume_release_metadata(root, &block_rsv, qgroup_reserved);
@@ -629,6 +634,10 @@
d_instantiate(dentry, inode);
}
return ret;
+
+fail_free:
+ kfree(root_item);
+ return ret;
}
static void btrfs_wait_for_no_snapshoting_writes(struct btrfs_root *root)
@@ -681,7 +690,7 @@
if (ret)
goto dec_and_free;
- btrfs_wait_ordered_extents(root, -1);
+ btrfs_wait_ordered_extents(root, -1, 0, (u64)-1);
btrfs_init_block_rsv(&pending_snapshot->block_rsv,
BTRFS_BLOCK_RSV_TEMP);
@@ -2671,10 +2680,10 @@
return ret;
}
-static long btrfs_ioctl_rm_dev(struct file *file, void __user *arg)
+static long btrfs_ioctl_rm_dev_v2(struct file *file, void __user *arg)
{
struct btrfs_root *root = BTRFS_I(file_inode(file))->root;
- struct btrfs_ioctl_vol_args *vol_args;
+ struct btrfs_ioctl_vol_args_v2 *vol_args;
int ret;
if (!capable(CAP_SYS_ADMIN))
@@ -2690,7 +2699,9 @@
goto err_drop;
}
- vol_args->name[BTRFS_PATH_NAME_MAX] = '\0';
+ /* Check for compatibility reject unknown flags */
+ if (vol_args->flags & ~BTRFS_VOL_ARG_V2_FLAGS_SUPPORTED)
+ return -EOPNOTSUPP;
if (atomic_xchg(&root->fs_info->mutually_exclusive_operation_running,
1)) {
@@ -2699,13 +2710,23 @@
}
mutex_lock(&root->fs_info->volume_mutex);
- ret = btrfs_rm_device(root, vol_args->name);
+ if (vol_args->flags & BTRFS_DEVICE_SPEC_BY_ID) {
+ ret = btrfs_rm_device(root, NULL, vol_args->devid);
+ } else {
+ vol_args->name[BTRFS_SUBVOL_NAME_MAX] = '\0';
+ ret = btrfs_rm_device(root, vol_args->name, 0);
+ }
mutex_unlock(&root->fs_info->volume_mutex);
atomic_set(&root->fs_info->mutually_exclusive_operation_running, 0);
- if (!ret)
- btrfs_info(root->fs_info, "disk deleted %s",vol_args->name);
-
+ if (!ret) {
+ if (vol_args->flags & BTRFS_DEVICE_SPEC_BY_ID)
+ btrfs_info(root->fs_info, "device deleted: id %llu",
+ vol_args->devid);
+ else
+ btrfs_info(root->fs_info, "device deleted: %s",
+ vol_args->name);
+ }
out:
kfree(vol_args);
err_drop:
@@ -2713,6 +2734,47 @@
return ret;
}
+static long btrfs_ioctl_rm_dev(struct file *file, void __user *arg)
+{
+ struct btrfs_root *root = BTRFS_I(file_inode(file))->root;
+ struct btrfs_ioctl_vol_args *vol_args;
+ int ret;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ ret = mnt_want_write_file(file);
+ if (ret)
+ return ret;
+
+ if (atomic_xchg(&root->fs_info->mutually_exclusive_operation_running,
+ 1)) {
+ ret = BTRFS_ERROR_DEV_EXCL_RUN_IN_PROGRESS;
+ goto out_drop_write;
+ }
+
+ vol_args = memdup_user(arg, sizeof(*vol_args));
+ if (IS_ERR(vol_args)) {
+ ret = PTR_ERR(vol_args);
+ goto out;
+ }
+
+ vol_args->name[BTRFS_PATH_NAME_MAX] = '\0';
+ mutex_lock(&root->fs_info->volume_mutex);
+ ret = btrfs_rm_device(root, vol_args->name, 0);
+ mutex_unlock(&root->fs_info->volume_mutex);
+
+ if (!ret)
+ btrfs_info(root->fs_info, "disk deleted %s",vol_args->name);
+ kfree(vol_args);
+out:
+ atomic_set(&root->fs_info->mutually_exclusive_operation_running, 0);
+out_drop_write:
+ mnt_drop_write_file(file);
+
+ return ret;
+}
+
static long btrfs_ioctl_fs_info(struct btrfs_root *root, void __user *arg)
{
struct btrfs_ioctl_fs_info_args *fi_args;
@@ -3472,13 +3534,16 @@
u64 last_dest_end = destoff;
ret = -ENOMEM;
- buf = vmalloc(root->nodesize);
- if (!buf)
- return ret;
+ buf = kmalloc(root->nodesize, GFP_KERNEL | __GFP_NOWARN);
+ if (!buf) {
+ buf = vmalloc(root->nodesize);
+ if (!buf)
+ return ret;
+ }
path = btrfs_alloc_path();
if (!path) {
- vfree(buf);
+ kvfree(buf);
return ret;
}
@@ -3779,7 +3844,7 @@
out:
btrfs_free_path(path);
- vfree(buf);
+ kvfree(buf);
return ret;
}
@@ -4380,7 +4445,7 @@
1)) {
ret = BTRFS_ERROR_DEV_EXCL_RUN_IN_PROGRESS;
} else {
- ret = btrfs_dev_replace_start(root, p);
+ ret = btrfs_dev_replace_by_ioctl(root, p);
atomic_set(
&root->fs_info->mutually_exclusive_operation_running,
0);
@@ -4851,8 +4916,8 @@
/* update qgroup status and info */
err = btrfs_run_qgroups(trans, root->fs_info);
if (err < 0)
- btrfs_std_error(root->fs_info, ret,
- "failed to update qgroup status and info\n");
+ btrfs_handle_fs_error(root->fs_info, err,
+ "failed to update qgroup status and info");
err = btrfs_end_transaction(trans, root);
if (err && !ret)
ret = err;
@@ -5398,9 +5463,15 @@
if (ret)
return ret;
+ ret = mnt_want_write_file(file);
+ if (ret)
+ return ret;
+
trans = btrfs_start_transaction(root, 0);
- if (IS_ERR(trans))
- return PTR_ERR(trans);
+ if (IS_ERR(trans)) {
+ ret = PTR_ERR(trans);
+ goto out_drop_write;
+ }
spin_lock(&root->fs_info->super_lock);
newflags = btrfs_super_compat_flags(super_block);
@@ -5419,7 +5490,11 @@
btrfs_set_super_incompat_flags(super_block, newflags);
spin_unlock(&root->fs_info->super_lock);
- return btrfs_commit_transaction(trans, root);
+ ret = btrfs_commit_transaction(trans, root);
+out_drop_write:
+ mnt_drop_write_file(file);
+
+ return ret;
}
long btrfs_ioctl(struct file *file, unsigned int
@@ -5463,6 +5538,8 @@
return btrfs_ioctl_add_dev(root, argp);
case BTRFS_IOC_RM_DEV:
return btrfs_ioctl_rm_dev(file, argp);
+ case BTRFS_IOC_RM_DEV_V2:
+ return btrfs_ioctl_rm_dev_v2(file, argp);
case BTRFS_IOC_FS_INFO:
return btrfs_ioctl_fs_info(root, argp);
case BTRFS_IOC_DEV_INFO:
@@ -5556,3 +5633,24 @@
return -ENOTTY;
}
+
+#ifdef CONFIG_COMPAT
+long btrfs_compat_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
+{
+ switch (cmd) {
+ case FS_IOC32_GETFLAGS:
+ cmd = FS_IOC_GETFLAGS;
+ break;
+ case FS_IOC32_SETFLAGS:
+ cmd = FS_IOC_SETFLAGS;
+ break;
+ case FS_IOC32_GETVERSION:
+ cmd = FS_IOC_GETVERSION;
+ break;
+ default:
+ return -ENOIOCTLCMD;
+ }
+
+ return btrfs_ioctl(file, cmd, (unsigned long) compat_ptr(arg));
+}
+#endif
diff --git a/fs/btrfs/ordered-data.c b/fs/btrfs/ordered-data.c
index 0de7da5..5591704 100644
--- a/fs/btrfs/ordered-data.c
+++ b/fs/btrfs/ordered-data.c
@@ -661,14 +661,15 @@
* wait for all the ordered extents in a root. This is done when balancing
* space between drives.
*/
-int btrfs_wait_ordered_extents(struct btrfs_root *root, int nr)
+int btrfs_wait_ordered_extents(struct btrfs_root *root, int nr,
+ const u64 range_start, const u64 range_len)
{
- struct list_head splice, works;
+ LIST_HEAD(splice);
+ LIST_HEAD(skipped);
+ LIST_HEAD(works);
struct btrfs_ordered_extent *ordered, *next;
int count = 0;
-
- INIT_LIST_HEAD(&splice);
- INIT_LIST_HEAD(&works);
+ const u64 range_end = range_start + range_len;
mutex_lock(&root->ordered_extent_mutex);
spin_lock(&root->ordered_extent_lock);
@@ -676,6 +677,14 @@
while (!list_empty(&splice) && nr) {
ordered = list_first_entry(&splice, struct btrfs_ordered_extent,
root_extent_list);
+
+ if (range_end <= ordered->start ||
+ ordered->start + ordered->disk_len <= range_start) {
+ list_move_tail(&ordered->root_extent_list, &skipped);
+ cond_resched_lock(&root->ordered_extent_lock);
+ continue;
+ }
+
list_move_tail(&ordered->root_extent_list,
&root->ordered_extents);
atomic_inc(&ordered->refs);
@@ -694,6 +703,7 @@
nr--;
count++;
}
+ list_splice_tail(&skipped, &root->ordered_extents);
list_splice_tail(&splice, &root->ordered_extents);
spin_unlock(&root->ordered_extent_lock);
@@ -708,7 +718,8 @@
return count;
}
-void btrfs_wait_ordered_roots(struct btrfs_fs_info *fs_info, int nr)
+void btrfs_wait_ordered_roots(struct btrfs_fs_info *fs_info, int nr,
+ const u64 range_start, const u64 range_len)
{
struct btrfs_root *root;
struct list_head splice;
@@ -728,7 +739,8 @@
&fs_info->ordered_roots);
spin_unlock(&fs_info->ordered_root_lock);
- done = btrfs_wait_ordered_extents(root, nr);
+ done = btrfs_wait_ordered_extents(root, nr,
+ range_start, range_len);
btrfs_put_fs_root(root);
spin_lock(&fs_info->ordered_root_lock);
diff --git a/fs/btrfs/ordered-data.h b/fs/btrfs/ordered-data.h
index 23c9605..8ef1262 100644
--- a/fs/btrfs/ordered-data.h
+++ b/fs/btrfs/ordered-data.h
@@ -197,8 +197,10 @@
struct btrfs_ordered_extent *ordered);
int btrfs_find_ordered_sum(struct inode *inode, u64 offset, u64 disk_bytenr,
u32 *sum, int len);
-int btrfs_wait_ordered_extents(struct btrfs_root *root, int nr);
-void btrfs_wait_ordered_roots(struct btrfs_fs_info *fs_info, int nr);
+int btrfs_wait_ordered_extents(struct btrfs_root *root, int nr,
+ const u64 range_start, const u64 range_len);
+void btrfs_wait_ordered_roots(struct btrfs_fs_info *fs_info, int nr,
+ const u64 range_start, const u64 range_len);
void btrfs_get_logged_extents(struct inode *inode,
struct list_head *logged_list,
const loff_t start,
diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
index 08ef890..1cfd35c 100644
--- a/fs/btrfs/relocation.c
+++ b/fs/btrfs/relocation.c
@@ -2418,7 +2418,7 @@
}
out:
if (ret) {
- btrfs_std_error(root->fs_info, ret, NULL);
+ btrfs_handle_fs_error(root->fs_info, ret, NULL);
if (!list_empty(&reloc_roots))
free_reloc_roots(&reloc_roots);
@@ -4254,12 +4254,11 @@
btrfs_info(extent_root->fs_info, "relocating block group %llu flags %llu",
rc->block_group->key.objectid, rc->block_group->flags);
- ret = btrfs_start_delalloc_roots(fs_info, 0, -1);
- if (ret < 0) {
- err = ret;
- goto out;
- }
- btrfs_wait_ordered_roots(fs_info, -1);
+ btrfs_wait_block_group_reservations(rc->block_group);
+ btrfs_wait_nocow_writers(rc->block_group);
+ btrfs_wait_ordered_roots(fs_info, -1,
+ rc->block_group->key.objectid,
+ rc->block_group->key.offset);
while (1) {
mutex_lock(&fs_info->cleaner_mutex);
diff --git a/fs/btrfs/root-tree.c b/fs/btrfs/root-tree.c
index 9fcd6df..b2b14e7 100644
--- a/fs/btrfs/root-tree.c
+++ b/fs/btrfs/root-tree.c
@@ -284,7 +284,7 @@
trans = btrfs_join_transaction(tree_root);
if (IS_ERR(trans)) {
err = PTR_ERR(trans);
- btrfs_std_error(tree_root->fs_info, err,
+ btrfs_handle_fs_error(tree_root->fs_info, err,
"Failed to start trans to delete "
"orphan item");
break;
@@ -293,7 +293,7 @@
root_key.objectid);
btrfs_end_transaction(trans, tree_root);
if (err) {
- btrfs_std_error(tree_root->fs_info, err,
+ btrfs_handle_fs_error(tree_root->fs_info, err,
"Failed to delete root orphan "
"item");
break;
diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c
index 4678f03..fa35cdc 100644
--- a/fs/btrfs/scrub.c
+++ b/fs/btrfs/scrub.c
@@ -1350,7 +1350,7 @@
recover->bbio = bbio;
recover->map_length = mapped_length;
- BUG_ON(page_index >= SCRUB_PAGES_PER_RD_BIO);
+ BUG_ON(page_index >= SCRUB_MAX_PAGES_PER_BLOCK);
nmirrors = min(scrub_nr_raid_mirrors(bbio), BTRFS_MAX_MIRRORS);
@@ -2127,6 +2127,8 @@
if (bio->bi_error)
sblock->no_io_error_seen = 0;
+ bio_put(bio);
+
btrfs_queue_work(fs_info->scrub_workers, &sblock->work);
}
@@ -2860,7 +2862,7 @@
int extent_mirror_num;
int stop_loop = 0;
- nsectors = map->stripe_len / root->sectorsize;
+ nsectors = div_u64(map->stripe_len, root->sectorsize);
bitmap_len = scrub_calc_parity_bitmap_len(nsectors);
sparity = kzalloc(sizeof(struct scrub_parity) + 2 * bitmap_len,
GFP_NOFS);
@@ -3070,7 +3072,6 @@
int slot;
u64 nstripes;
struct extent_buffer *l;
- struct btrfs_key key;
u64 physical;
u64 logical;
u64 logic_end;
@@ -3079,7 +3080,7 @@
int mirror_num;
struct reada_control *reada1;
struct reada_control *reada2;
- struct btrfs_key key_start;
+ struct btrfs_key key;
struct btrfs_key key_end;
u64 increment = map->stripe_len;
u64 offset;
@@ -3158,21 +3159,21 @@
scrub_blocked_if_needed(fs_info);
/* FIXME it might be better to start readahead at commit root */
- key_start.objectid = logical;
- key_start.type = BTRFS_EXTENT_ITEM_KEY;
- key_start.offset = (u64)0;
+ key.objectid = logical;
+ key.type = BTRFS_EXTENT_ITEM_KEY;
+ key.offset = (u64)0;
key_end.objectid = logic_end;
key_end.type = BTRFS_METADATA_ITEM_KEY;
key_end.offset = (u64)-1;
- reada1 = btrfs_reada_add(root, &key_start, &key_end);
+ reada1 = btrfs_reada_add(root, &key, &key_end);
- key_start.objectid = BTRFS_EXTENT_CSUM_OBJECTID;
- key_start.type = BTRFS_EXTENT_CSUM_KEY;
- key_start.offset = logical;
+ key.objectid = BTRFS_EXTENT_CSUM_OBJECTID;
+ key.type = BTRFS_EXTENT_CSUM_KEY;
+ key.offset = logical;
key_end.objectid = BTRFS_EXTENT_CSUM_OBJECTID;
key_end.type = BTRFS_EXTENT_CSUM_KEY;
key_end.offset = logic_end;
- reada2 = btrfs_reada_add(csum_root, &key_start, &key_end);
+ reada2 = btrfs_reada_add(csum_root, &key, &key_end);
if (!IS_ERR(reada1))
btrfs_reada_wait(reada1);
diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
index 8d358c5..6a8c860 100644
--- a/fs/btrfs/send.c
+++ b/fs/btrfs/send.c
@@ -5939,6 +5939,7 @@
u32 i;
u64 *clone_sources_tmp = NULL;
int clone_sources_to_rollback = 0;
+ unsigned alloc_size;
int sort_clone_roots = 0;
int index;
@@ -5978,6 +5979,12 @@
goto out;
}
+ if (arg->clone_sources_count >
+ ULLONG_MAX / sizeof(*arg->clone_sources)) {
+ ret = -EINVAL;
+ goto out;
+ }
+
if (!access_ok(VERIFY_READ, arg->clone_sources,
sizeof(*arg->clone_sources) *
arg->clone_sources_count)) {
@@ -6022,40 +6029,53 @@
sctx->clone_roots_cnt = arg->clone_sources_count;
sctx->send_max_size = BTRFS_SEND_BUF_SIZE;
- sctx->send_buf = vmalloc(sctx->send_max_size);
+ sctx->send_buf = kmalloc(sctx->send_max_size, GFP_KERNEL | __GFP_NOWARN);
if (!sctx->send_buf) {
- ret = -ENOMEM;
- goto out;
+ sctx->send_buf = vmalloc(sctx->send_max_size);
+ if (!sctx->send_buf) {
+ ret = -ENOMEM;
+ goto out;
+ }
}
- sctx->read_buf = vmalloc(BTRFS_SEND_READ_SIZE);
+ sctx->read_buf = kmalloc(BTRFS_SEND_READ_SIZE, GFP_KERNEL | __GFP_NOWARN);
if (!sctx->read_buf) {
- ret = -ENOMEM;
- goto out;
+ sctx->read_buf = vmalloc(BTRFS_SEND_READ_SIZE);
+ if (!sctx->read_buf) {
+ ret = -ENOMEM;
+ goto out;
+ }
}
sctx->pending_dir_moves = RB_ROOT;
sctx->waiting_dir_moves = RB_ROOT;
sctx->orphan_dirs = RB_ROOT;
- sctx->clone_roots = vzalloc(sizeof(struct clone_root) *
- (arg->clone_sources_count + 1));
- if (!sctx->clone_roots) {
- ret = -ENOMEM;
- goto out;
- }
+ alloc_size = sizeof(struct clone_root) * (arg->clone_sources_count + 1);
- if (arg->clone_sources_count) {
- clone_sources_tmp = vmalloc(arg->clone_sources_count *
- sizeof(*arg->clone_sources));
- if (!clone_sources_tmp) {
+ sctx->clone_roots = kzalloc(alloc_size, GFP_KERNEL | __GFP_NOWARN);
+ if (!sctx->clone_roots) {
+ sctx->clone_roots = vzalloc(alloc_size);
+ if (!sctx->clone_roots) {
ret = -ENOMEM;
goto out;
}
+ }
+
+ alloc_size = arg->clone_sources_count * sizeof(*arg->clone_sources);
+
+ if (arg->clone_sources_count) {
+ clone_sources_tmp = kmalloc(alloc_size, GFP_KERNEL | __GFP_NOWARN);
+ if (!clone_sources_tmp) {
+ clone_sources_tmp = vmalloc(alloc_size);
+ if (!clone_sources_tmp) {
+ ret = -ENOMEM;
+ goto out;
+ }
+ }
ret = copy_from_user(clone_sources_tmp, arg->clone_sources,
- arg->clone_sources_count *
- sizeof(*arg->clone_sources));
+ alloc_size);
if (ret) {
ret = -EFAULT;
goto out;
@@ -6089,7 +6109,7 @@
sctx->clone_roots[i].root = clone_root;
clone_sources_to_rollback = i + 1;
}
- vfree(clone_sources_tmp);
+ kvfree(clone_sources_tmp);
clone_sources_tmp = NULL;
}
@@ -6207,15 +6227,15 @@
btrfs_root_dec_send_in_progress(sctx->parent_root);
kfree(arg);
- vfree(clone_sources_tmp);
+ kvfree(clone_sources_tmp);
if (sctx) {
if (sctx->send_filp)
fput(sctx->send_filp);
- vfree(sctx->clone_roots);
- vfree(sctx->send_buf);
- vfree(sctx->read_buf);
+ kvfree(sctx->clone_roots);
+ kvfree(sctx->send_buf);
+ kvfree(sctx->read_buf);
name_cache_free(sctx);
diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index 00b8f37..bf71071 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -97,15 +97,6 @@
return errstr;
}
-static void save_error_info(struct btrfs_fs_info *fs_info)
-{
- /*
- * today we only save the error info into ram. Long term we'll
- * also send it down to the disk
- */
- set_bit(BTRFS_FS_STATE_ERROR, &fs_info->fs_state);
-}
-
/* btrfs handle error by forcing the filesystem readonly */
static void btrfs_handle_error(struct btrfs_fs_info *fs_info)
{
@@ -131,11 +122,11 @@
}
/*
- * __btrfs_std_error decodes expected errors from the caller and
+ * __btrfs_handle_fs_error decodes expected errors from the caller and
* invokes the approciate error response.
*/
__cold
-void __btrfs_std_error(struct btrfs_fs_info *fs_info, const char *function,
+void __btrfs_handle_fs_error(struct btrfs_fs_info *fs_info, const char *function,
unsigned int line, int errno, const char *fmt, ...)
{
struct super_block *sb = fs_info->sb;
@@ -170,8 +161,13 @@
}
#endif
+ /*
+ * Today we only save the error info to memory. Long term we'll
+ * also send it down to the disk
+ */
+ set_bit(BTRFS_FS_STATE_ERROR, &fs_info->fs_state);
+
/* Don't go through full error handling during mount */
- save_error_info(fs_info);
if (sb->s_flags & MS_BORN)
btrfs_handle_error(fs_info);
}
@@ -252,7 +248,7 @@
/* Wake up anybody who may be waiting on this transaction */
wake_up(&root->fs_info->transaction_wait);
wake_up(&root->fs_info->transaction_blocked_wait);
- __btrfs_std_error(root->fs_info, function, line, errno, NULL);
+ __btrfs_handle_fs_error(root->fs_info, function, line, errno, NULL);
}
/*
* __btrfs_panic decodes unexpected, fatal errors from the caller,
@@ -1160,7 +1156,7 @@
return 0;
}
- btrfs_wait_ordered_roots(fs_info, -1);
+ btrfs_wait_ordered_roots(fs_info, -1, 0, (u64)-1);
trans = btrfs_attach_transaction_barrier(root);
if (IS_ERR(trans)) {
@@ -1488,10 +1484,10 @@
memcpy(&fs_info->security_opts, sec_opts, sizeof(*sec_opts));
} else {
/*
- * Since SELinux(the only one supports security_mnt_opts) does
- * NOT support changing context during remount/mount same sb,
- * This must be the same or part of the same security options,
- * just free it.
+ * Since SELinux (the only one supporting security_mnt_opts)
+ * does NOT support changing context during remount/mount of
+ * the same sb, this must be the same or part of the same
+ * security options, just free it.
*/
security_free_mnt_opts(sec_opts);
}
@@ -1669,8 +1665,8 @@
unsigned long old_opts)
{
/*
- * We need cleanup all defragable inodes if the autodefragment is
- * close or the fs is R/O.
+ * We need to cleanup all defragable inodes if the autodefragment is
+ * close or the filesystem is read only.
*/
if (btrfs_raw_test_opt(old_opts, AUTO_DEFRAG) &&
(!btrfs_raw_test_opt(fs_info->mount_opt, AUTO_DEFRAG) ||
@@ -2051,9 +2047,10 @@
struct btrfs_block_rsv *block_rsv = &fs_info->global_block_rsv;
int ret;
u64 thresh = 0;
+ int mixed = 0;
/*
- * holding chunk_muext to avoid allocating new chunks, holding
+ * holding chunk_mutex to avoid allocating new chunks, holding
* device_list_mutex to avoid the device being removed
*/
rcu_read_lock();
@@ -2076,8 +2073,17 @@
}
}
}
- if (found->flags & BTRFS_BLOCK_GROUP_METADATA)
- total_free_meta += found->disk_total - found->disk_used;
+
+ /*
+ * Metadata in mixed block goup profiles are accounted in data
+ */
+ if (!mixed && found->flags & BTRFS_BLOCK_GROUP_METADATA) {
+ if (found->flags & BTRFS_BLOCK_GROUP_DATA)
+ mixed = 1;
+ else
+ total_free_meta += found->disk_total -
+ found->disk_used;
+ }
total_used += found->disk_used;
}
@@ -2090,7 +2096,11 @@
/* Account global block reserve as used, it's in logical size already */
spin_lock(&block_rsv->lock);
- buf->f_bfree -= block_rsv->size >> bits;
+ /* Mixed block groups accounting is not byte-accurate, avoid overflow */
+ if (buf->f_bfree >= block_rsv->size >> bits)
+ buf->f_bfree -= block_rsv->size >> bits;
+ else
+ buf->f_bfree = 0;
spin_unlock(&block_rsv->lock);
buf->f_bavail = div_u64(total_free_data, factor);
@@ -2115,7 +2125,7 @@
*/
thresh = 4 * 1024 * 1024;
- if (total_free_meta - thresh < block_rsv->size)
+ if (!mixed && total_free_meta - thresh < block_rsv->size)
buf->f_bavail = 0;
buf->f_type = BTRFS_SUPER_MAGIC;
diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index 539e7b5..4879656 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -120,6 +120,9 @@
if (!fs_info)
return -EPERM;
+ if (fs_info->sb->s_flags & MS_RDONLY)
+ return -EROFS;
+
ret = kstrtoul(skip_spaces(buf), 0, &val);
if (ret)
return ret;
@@ -364,7 +367,13 @@
{
struct btrfs_fs_info *fs_info = to_fs_info(kobj);
char *label = fs_info->super_copy->label;
- return snprintf(buf, PAGE_SIZE, label[0] ? "%s\n" : "%s", label);
+ ssize_t ret;
+
+ spin_lock(&fs_info->super_lock);
+ ret = snprintf(buf, PAGE_SIZE, label[0] ? "%s\n" : "%s", label);
+ spin_unlock(&fs_info->super_lock);
+
+ return ret;
}
static ssize_t btrfs_label_store(struct kobject *kobj,
@@ -374,6 +383,9 @@
struct btrfs_fs_info *fs_info = to_fs_info(kobj);
size_t p_len;
+ if (!fs_info)
+ return -EPERM;
+
if (fs_info->sb->s_flags & MS_RDONLY)
return -EROFS;
diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index 43885e5..5b0b758 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -311,10 +311,11 @@
* when the transaction commits
*/
static int record_root_in_trans(struct btrfs_trans_handle *trans,
- struct btrfs_root *root)
+ struct btrfs_root *root,
+ int force)
{
- if (test_bit(BTRFS_ROOT_REF_COWS, &root->state) &&
- root->last_trans < trans->transid) {
+ if ((test_bit(BTRFS_ROOT_REF_COWS, &root->state) &&
+ root->last_trans < trans->transid) || force) {
WARN_ON(root == root->fs_info->extent_root);
WARN_ON(root->commit_root != root->node);
@@ -331,7 +332,7 @@
smp_wmb();
spin_lock(&root->fs_info->fs_roots_radix_lock);
- if (root->last_trans == trans->transid) {
+ if (root->last_trans == trans->transid && !force) {
spin_unlock(&root->fs_info->fs_roots_radix_lock);
return 0;
}
@@ -402,7 +403,7 @@
return 0;
mutex_lock(&root->fs_info->reloc_mutex);
- record_root_in_trans(trans, root);
+ record_root_in_trans(trans, root, 0);
mutex_unlock(&root->fs_info->reloc_mutex);
return 0;
@@ -1310,6 +1311,97 @@
return ret;
}
+/* Bisesctability fixup, remove in 4.8 */
+#ifndef btrfs_std_error
+#define btrfs_std_error btrfs_handle_fs_error
+#endif
+
+/*
+ * Do all special snapshot related qgroup dirty hack.
+ *
+ * Will do all needed qgroup inherit and dirty hack like switch commit
+ * roots inside one transaction and write all btree into disk, to make
+ * qgroup works.
+ */
+static int qgroup_account_snapshot(struct btrfs_trans_handle *trans,
+ struct btrfs_root *src,
+ struct btrfs_root *parent,
+ struct btrfs_qgroup_inherit *inherit,
+ u64 dst_objectid)
+{
+ struct btrfs_fs_info *fs_info = src->fs_info;
+ int ret;
+
+ /*
+ * Save some performance in the case that qgroups are not
+ * enabled. If this check races with the ioctl, rescan will
+ * kick in anyway.
+ */
+ mutex_lock(&fs_info->qgroup_ioctl_lock);
+ if (!fs_info->quota_enabled) {
+ mutex_unlock(&fs_info->qgroup_ioctl_lock);
+ return 0;
+ }
+ mutex_unlock(&fs_info->qgroup_ioctl_lock);
+
+ /*
+ * We are going to commit transaction, see btrfs_commit_transaction()
+ * comment for reason locking tree_log_mutex
+ */
+ mutex_lock(&fs_info->tree_log_mutex);
+
+ ret = commit_fs_roots(trans, src);
+ if (ret)
+ goto out;
+ ret = btrfs_qgroup_prepare_account_extents(trans, fs_info);
+ if (ret < 0)
+ goto out;
+ ret = btrfs_qgroup_account_extents(trans, fs_info);
+ if (ret < 0)
+ goto out;
+
+ /* Now qgroup are all updated, we can inherit it to new qgroups */
+ ret = btrfs_qgroup_inherit(trans, fs_info,
+ src->root_key.objectid, dst_objectid,
+ inherit);
+ if (ret < 0)
+ goto out;
+
+ /*
+ * Now we do a simplified commit transaction, which will:
+ * 1) commit all subvolume and extent tree
+ * To ensure all subvolume and extent tree have a valid
+ * commit_root to accounting later insert_dir_item()
+ * 2) write all btree blocks onto disk
+ * This is to make sure later btree modification will be cowed
+ * Or commit_root can be populated and cause wrong qgroup numbers
+ * In this simplified commit, we don't really care about other trees
+ * like chunk and root tree, as they won't affect qgroup.
+ * And we don't write super to avoid half committed status.
+ */
+ ret = commit_cowonly_roots(trans, src);
+ if (ret)
+ goto out;
+ switch_commit_roots(trans->transaction, fs_info);
+ ret = btrfs_write_and_wait_transaction(trans, src);
+ if (ret)
+ btrfs_std_error(fs_info, ret,
+ "Error while writing out transaction for qgroup");
+
+out:
+ mutex_unlock(&fs_info->tree_log_mutex);
+
+ /*
+ * Force parent root to be updated, as we recorded it before so its
+ * last_trans == cur_transid.
+ * Or it won't be committed again onto disk after later
+ * insert_dir_item()
+ */
+ if (!ret)
+ record_root_in_trans(trans, parent, 1);
+ return ret;
+}
+
/*
* new snapshots need to be created at a very specific time in the
* transaction commit. This does the actual creation.
@@ -1383,7 +1475,7 @@
dentry = pending->dentry;
parent_inode = pending->dir;
parent_root = BTRFS_I(parent_inode)->root;
- record_root_in_trans(trans, parent_root);
+ record_root_in_trans(trans, parent_root, 0);
cur_time = current_fs_time(parent_inode->i_sb);
@@ -1420,7 +1512,7 @@
goto fail;
}
- record_root_in_trans(trans, root);
+ record_root_in_trans(trans, root, 0);
btrfs_set_root_last_snapshot(&root->root_item, trans->transid);
memcpy(new_root_item, &root->root_item, sizeof(*new_root_item));
btrfs_check_and_init_root_item(new_root_item);
@@ -1516,6 +1608,17 @@
goto fail;
}
+ /*
+ * Do special qgroup accounting for snapshot, as we do some qgroup
+ * snapshot hack to do fast snapshot.
+ * To co-operate with that hack, we do hack again.
+ * Or snapshot will be greatly slowed down by a subtree qgroup rescan
+ */
+ ret = qgroup_account_snapshot(trans, root, parent_root,
+ pending->inherit, objectid);
+ if (ret < 0)
+ goto fail;
+
ret = btrfs_insert_dir_item(trans, parent_root,
dentry->d_name.name, dentry->d_name.len,
parent_inode, &key,
@@ -1559,23 +1662,6 @@
goto fail;
}
- /*
- * account qgroup counters before qgroup_inherit()
- */
- ret = btrfs_qgroup_prepare_account_extents(trans, fs_info);
- if (ret)
- goto fail;
- ret = btrfs_qgroup_account_extents(trans, fs_info);
- if (ret)
- goto fail;
- ret = btrfs_qgroup_inherit(trans, fs_info,
- root->root_key.objectid,
- objectid, pending->inherit);
- if (ret) {
- btrfs_abort_transaction(trans, root, ret);
- goto fail;
- }
-
fail:
pending->error = ret;
dir_item_existed:
@@ -1821,7 +1907,7 @@
static inline void btrfs_wait_delalloc_flush(struct btrfs_fs_info *fs_info)
{
if (btrfs_test_opt(fs_info->tree_root, FLUSHONCOMMIT))
- btrfs_wait_ordered_roots(fs_info, -1);
+ btrfs_wait_ordered_roots(fs_info, -1, 0, (u64)-1);
}
static inline void
@@ -2145,7 +2231,7 @@
ret = btrfs_write_and_wait_transaction(trans, root);
if (ret) {
- btrfs_std_error(root->fs_info, ret,
+ btrfs_handle_fs_error(root->fs_info, ret,
"Error while writing out transaction");
mutex_unlock(&root->fs_info->tree_log_mutex);
goto scrub_continue;
diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c
index e692eea..8aaca5c 100644
--- a/fs/btrfs/tree-log.c
+++ b/fs/btrfs/tree-log.c
@@ -4141,6 +4141,7 @@
INIT_LIST_HEAD(&extents);
+ down_write(&BTRFS_I(inode)->dio_sem);
write_lock(&tree->lock);
test_gen = root->fs_info->last_trans_committed;
@@ -4169,13 +4170,20 @@
}
list_sort(NULL, &extents, extent_cmp);
- /*
- * Collect any new ordered extents within the range. This is to
- * prevent logging file extent items without waiting for the disk
- * location they point to being written. We do this only to deal
- * with races against concurrent lockless direct IO writes.
- */
btrfs_get_logged_extents(inode, logged_list, start, end);
+ /*
+ * Some ordered extents started by fsync might have completed
+ * before we could collect them into the list logged_list, which
+ * means they're gone, not in our logged_list nor in the inode's
+ * ordered tree. We want the application/user space to know an
+ * error happened while attempting to persist file data so that
+ * it can take proper action. If such error happened, we leave
+ * without writing to the log tree and the fsync must report the
+ * file data write error and not commit the current transaction.
+ */
+ ret = btrfs_inode_check_errors(inode);
+ if (ret)
+ ctx->io_err = ret;
process:
while (!list_empty(&extents)) {
em = list_entry(extents.next, struct extent_map, list);
@@ -4202,6 +4210,7 @@
}
WARN_ON(!list_empty(&extents));
write_unlock(&tree->lock);
+ up_write(&BTRFS_I(inode)->dio_sem);
btrfs_release_path(path);
return ret;
@@ -4623,23 +4632,6 @@
mutex_lock(&BTRFS_I(inode)->log_mutex);
/*
- * Collect ordered extents only if we are logging data. This is to
- * ensure a subsequent request to log this inode in LOG_INODE_ALL mode
- * will process the ordered extents if they still exists at the time,
- * because when we collect them we test and set for the flag
- * BTRFS_ORDERED_LOGGED to prevent multiple log requests to process the
- * same ordered extents. The consequence for the LOG_INODE_ALL log mode
- * not processing the ordered extents is that we end up logging the
- * corresponding file extent items, based on the extent maps in the
- * inode's extent_map_tree's modified_list, without logging the
- * respective checksums (since the may still be only attached to the
- * ordered extents and have not been inserted in the csum tree by
- * btrfs_finish_ordered_io() yet).
- */
- if (inode_only == LOG_INODE_ALL)
- btrfs_get_logged_extents(inode, &logged_list, start, end);
-
- /*
* a brute force approach to making sure we get the most uptodate
* copies of everything.
*/
@@ -4846,21 +4838,6 @@
goto out_unlock;
}
if (fast_search) {
- /*
- * Some ordered extents started by fsync might have completed
- * before we collected the ordered extents in logged_list, which
- * means they're gone, not in our logged_list nor in the inode's
- * ordered tree. We want the application/user space to know an
- * error happened while attempting to persist file data so that
- * it can take proper action. If such error happened, we leave
- * without writing to the log tree and the fsync must report the
- * file data write error and not commit the current transaction.
- */
- err = btrfs_inode_check_errors(inode);
- if (err) {
- ctx->io_err = err;
- goto out_unlock;
- }
ret = btrfs_log_changed_extents(trans, root, inode, dst_path,
&logged_list, ctx, start, end);
if (ret) {
@@ -5158,7 +5135,7 @@
}
ctx->log_new_dentries = false;
- if (type == BTRFS_FT_DIR)
+ if (type == BTRFS_FT_DIR || type == BTRFS_FT_SYMLINK)
log_mode = LOG_INODE_ALL;
btrfs_release_path(path);
ret = btrfs_log_inode(trans, root, di_inode,
@@ -5278,11 +5255,16 @@
if (IS_ERR(dir_inode))
continue;
+ if (ctx)
+ ctx->log_new_dentries = false;
ret = btrfs_log_inode(trans, root, dir_inode,
LOG_INODE_ALL, 0, LLONG_MAX, ctx);
if (!ret &&
btrfs_must_commit_transaction(trans, dir_inode))
ret = 1;
+ if (!ret && ctx && ctx->log_new_dentries)
+ ret = log_new_dir_dentries(trans, root,
+ dir_inode, ctx);
iput(dir_inode);
if (ret)
goto out;
@@ -5519,7 +5501,7 @@
ret = walk_log_tree(trans, log_root_tree, &wc);
if (ret) {
- btrfs_std_error(fs_info, ret, "Failed to pin buffers while "
+ btrfs_handle_fs_error(fs_info, ret, "Failed to pin buffers while "
"recovering log root tree.");
goto error;
}
@@ -5533,7 +5515,7 @@
ret = btrfs_search_slot(NULL, log_root_tree, &key, path, 0, 0);
if (ret < 0) {
- btrfs_std_error(fs_info, ret,
+ btrfs_handle_fs_error(fs_info, ret,
"Couldn't find tree log root.");
goto error;
}
@@ -5551,7 +5533,7 @@
log = btrfs_read_fs_root(log_root_tree, &found_key);
if (IS_ERR(log)) {
ret = PTR_ERR(log);
- btrfs_std_error(fs_info, ret,
+ btrfs_handle_fs_error(fs_info, ret,
"Couldn't read tree log root.");
goto error;
}
@@ -5566,7 +5548,7 @@
free_extent_buffer(log->node);
free_extent_buffer(log->commit_root);
kfree(log);
- btrfs_std_error(fs_info, ret, "Couldn't read target root "
+ btrfs_handle_fs_error(fs_info, ret, "Couldn't read target root "
"for tree log recovery.");
goto error;
}
@@ -5652,11 +5634,9 @@
* into the file. When the file is logged we check it and
* don't log the parents if the file is fully on disk.
*/
- if (S_ISREG(inode->i_mode)) {
- mutex_lock(&BTRFS_I(inode)->log_mutex);
- BTRFS_I(inode)->last_unlink_trans = trans->transid;
- mutex_unlock(&BTRFS_I(inode)->log_mutex);
- }
+ mutex_lock(&BTRFS_I(inode)->log_mutex);
+ BTRFS_I(inode)->last_unlink_trans = trans->transid;
+ mutex_unlock(&BTRFS_I(inode)->log_mutex);
/*
* if this directory was already logged any new
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index bd0f45f..2b88127 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -20,13 +20,13 @@
#include <linux/slab.h>
#include <linux/buffer_head.h>
#include <linux/blkdev.h>
-#include <linux/random.h>
#include <linux/iocontext.h>
#include <linux/capability.h>
#include <linux/ratelimit.h>
#include <linux/kthread.h>
#include <linux/raid/pq.h>
#include <linux/semaphore.h>
+#include <linux/uuid.h>
#include <asm/div64.h>
#include "ctree.h"
#include "extent_map.h"
@@ -118,6 +118,21 @@
[BTRFS_RAID_RAID6] = BTRFS_BLOCK_GROUP_RAID6,
};
+/*
+ * Table to convert BTRFS_RAID_* to the error code if minimum number of devices
+ * condition is not met. Zero means there's no corresponding
+ * BTRFS_ERROR_DEV_*_NOT_MET value.
+ */
+const int btrfs_raid_mindev_error[BTRFS_NR_RAID_TYPES] = {
+ [BTRFS_RAID_RAID10] = BTRFS_ERROR_DEV_RAID10_MIN_NOT_MET,
+ [BTRFS_RAID_RAID1] = BTRFS_ERROR_DEV_RAID1_MIN_NOT_MET,
+ [BTRFS_RAID_DUP] = 0,
+ [BTRFS_RAID_RAID0] = 0,
+ [BTRFS_RAID_SINGLE] = 0,
+ [BTRFS_RAID_RAID5] = BTRFS_ERROR_DEV_RAID5_MIN_NOT_MET,
+ [BTRFS_RAID_RAID6] = BTRFS_ERROR_DEV_RAID6_MIN_NOT_MET,
+};
+
static int init_first_rw_device(struct btrfs_trans_handle *trans,
struct btrfs_root *root,
struct btrfs_device *device);
@@ -699,7 +714,8 @@
* if there is new btrfs on an already registered device,
* then remove the stale device entry.
*/
- btrfs_free_stale_device(device);
+ if (ret > 0)
+ btrfs_free_stale_device(device);
*fs_devices_ret = fs_devices;
@@ -988,6 +1004,56 @@
return ret;
}
+void btrfs_release_disk_super(struct page *page)
+{
+ kunmap(page);
+ put_page(page);
+}
+
+int btrfs_read_disk_super(struct block_device *bdev, u64 bytenr,
+ struct page **page, struct btrfs_super_block **disk_super)
+{
+ void *p;
+ pgoff_t index;
+
+ /* make sure our super fits in the device */
+ if (bytenr + PAGE_SIZE >= i_size_read(bdev->bd_inode))
+ return 1;
+
+ /* make sure our super fits in the page */
+ if (sizeof(**disk_super) > PAGE_SIZE)
+ return 1;
+
+ /* make sure our super doesn't straddle pages on disk */
+ index = bytenr >> PAGE_SHIFT;
+ if ((bytenr + sizeof(**disk_super) - 1) >> PAGE_SHIFT != index)
+ return 1;
+
+ /* pull in the page with our super */
+ *page = read_cache_page_gfp(bdev->bd_inode->i_mapping,
+ index, GFP_KERNEL);
+
+ if (IS_ERR_OR_NULL(*page))
+ return 1;
+
+ p = kmap(*page);
+
+ /* align our pointer to the offset of the super block */
+ *disk_super = p + (bytenr & ~PAGE_MASK);
+
+ if (btrfs_super_bytenr(*disk_super) != bytenr ||
+ btrfs_super_magic(*disk_super) != BTRFS_MAGIC) {
+ btrfs_release_disk_super(*page);
+ return 1;
+ }
+
+ if ((*disk_super)->label[0] &&
+ (*disk_super)->label[BTRFS_LABEL_SIZE - 1])
+ (*disk_super)->label[BTRFS_LABEL_SIZE - 1] = '\0';
+
+ return 0;
+}
+
/*
* Look for a btrfs signature on a device. This may be called out of the mount path
* and we are not allowed to call set_blocksize during the scan. The superblock
@@ -999,13 +1065,11 @@
struct btrfs_super_block *disk_super;
struct block_device *bdev;
struct page *page;
- void *p;
int ret = -EINVAL;
u64 devid;
u64 transid;
u64 total_devices;
u64 bytenr;
- pgoff_t index;
/*
* we would like to check all the supers, but that would make
@@ -1018,41 +1082,14 @@
mutex_lock(&uuid_mutex);
bdev = blkdev_get_by_path(path, flags, holder);
-
if (IS_ERR(bdev)) {
ret = PTR_ERR(bdev);
goto error;
}
- /* make sure our super fits in the device */
- if (bytenr + PAGE_SIZE >= i_size_read(bdev->bd_inode))
+ if (btrfs_read_disk_super(bdev, bytenr, &page, &disk_super))
goto error_bdev_put;
- /* make sure our super fits in the page */
- if (sizeof(*disk_super) > PAGE_SIZE)
- goto error_bdev_put;
-
- /* make sure our super doesn't straddle pages on disk */
- index = bytenr >> PAGE_SHIFT;
- if ((bytenr + sizeof(*disk_super) - 1) >> PAGE_SHIFT != index)
- goto error_bdev_put;
-
- /* pull in the page with our super */
- page = read_cache_page_gfp(bdev->bd_inode->i_mapping,
- index, GFP_NOFS);
-
- if (IS_ERR_OR_NULL(page))
- goto error_bdev_put;
-
- p = kmap(page);
-
- /* align our pointer to the offset of the super block */
- disk_super = p + (bytenr & ~PAGE_MASK);
-
- if (btrfs_super_bytenr(disk_super) != bytenr ||
- btrfs_super_magic(disk_super) != BTRFS_MAGIC)
- goto error_unmap;
-
devid = btrfs_stack_device_id(&disk_super->dev_item);
transid = btrfs_super_generation(disk_super);
total_devices = btrfs_super_num_devices(disk_super);
@@ -1060,8 +1097,6 @@
ret = device_list_add(path, disk_super, devid, fs_devices_ret);
if (ret > 0) {
if (disk_super->label[0]) {
- if (disk_super->label[BTRFS_LABEL_SIZE - 1])
- disk_super->label[BTRFS_LABEL_SIZE - 1] = '\0';
printk(KERN_INFO "BTRFS: device label %s ", disk_super->label);
} else {
printk(KERN_INFO "BTRFS: device fsid %pU ", disk_super->fsid);
@@ -1073,9 +1108,7 @@
if (!ret && fs_devices_ret)
(*fs_devices_ret)->total_devices = total_devices;
-error_unmap:
- kunmap(page);
- put_page(page);
+ btrfs_release_disk_super(page);
error_bdev_put:
blkdev_put(bdev, flags);
@@ -1454,7 +1487,7 @@
extent = btrfs_item_ptr(leaf, path->slots[0],
struct btrfs_dev_extent);
} else {
- btrfs_std_error(root->fs_info, ret, "Slot search failed");
+ btrfs_handle_fs_error(root->fs_info, ret, "Slot search failed");
goto out;
}
@@ -1462,7 +1495,7 @@
ret = btrfs_del_item(trans, root, path);
if (ret) {
- btrfs_std_error(root->fs_info, ret,
+ btrfs_handle_fs_error(root->fs_info, ret,
"Failed to remove dev extent item");
} else {
set_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags);
@@ -1688,31 +1721,91 @@
return ret;
}
-int btrfs_rm_device(struct btrfs_root *root, char *device_path)
+/*
+ * Verify that @num_devices satisfies the RAID profile constraints in the whole
+ * filesystem. It's up to the caller to adjust that number regarding eg. device
+ * replace.
+ */
+static int btrfs_check_raid_min_devices(struct btrfs_fs_info *fs_info,
+ u64 num_devices)
{
- struct btrfs_device *device;
- struct btrfs_device *next_device;
- struct block_device *bdev;
- struct buffer_head *bh = NULL;
- struct btrfs_super_block *disk_super;
- struct btrfs_fs_devices *cur_devices;
u64 all_avail;
- u64 devid;
- u64 num_devices;
- u8 *dev_uuid;
unsigned seq;
- int ret = 0;
- bool clear_super = false;
-
- mutex_lock(&uuid_mutex);
+ int i;
do {
- seq = read_seqbegin(&root->fs_info->profiles_lock);
+ seq = read_seqbegin(&fs_info->profiles_lock);
- all_avail = root->fs_info->avail_data_alloc_bits |
- root->fs_info->avail_system_alloc_bits |
- root->fs_info->avail_metadata_alloc_bits;
- } while (read_seqretry(&root->fs_info->profiles_lock, seq));
+ all_avail = fs_info->avail_data_alloc_bits |
+ fs_info->avail_system_alloc_bits |
+ fs_info->avail_metadata_alloc_bits;
+ } while (read_seqretry(&fs_info->profiles_lock, seq));
+
+ for (i = 0; i < BTRFS_NR_RAID_TYPES; i++) {
+ if (!(all_avail & btrfs_raid_group[i]))
+ continue;
+
+ if (num_devices < btrfs_raid_array[i].devs_min) {
+ int ret = btrfs_raid_mindev_error[i];
+
+ if (ret)
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+struct btrfs_device *btrfs_find_next_active_device(struct btrfs_fs_devices *fs_devs,
+ struct btrfs_device *device)
+{
+ struct btrfs_device *next_device;
+
+ list_for_each_entry(next_device, &fs_devs->devices, dev_list) {
+ if (next_device != device &&
+ !next_device->missing && next_device->bdev)
+ return next_device;
+ }
+
+ return NULL;
+}
+
+/*
+ * Helper function to check if the given device is part of s_bdev / latest_bdev
+ * and replace it with the provided or the next active device, in the context
+ * where this function called, there should be always be another device (or
+ * this_dev) which is active.
+ */
+void btrfs_assign_next_active_device(struct btrfs_fs_info *fs_info,
+ struct btrfs_device *device, struct btrfs_device *this_dev)
+{
+ struct btrfs_device *next_device;
+
+ if (this_dev)
+ next_device = this_dev;
+ else
+ next_device = btrfs_find_next_active_device(fs_info->fs_devices,
+ device);
+ ASSERT(next_device);
+
+ if (fs_info->sb->s_bdev &&
+ (fs_info->sb->s_bdev == device->bdev))
+ fs_info->sb->s_bdev = next_device->bdev;
+
+ if (fs_info->fs_devices->latest_bdev == device->bdev)
+ fs_info->fs_devices->latest_bdev = next_device->bdev;
+}
+
+int btrfs_rm_device(struct btrfs_root *root, char *device_path, u64 devid)
+{
+ struct btrfs_device *device;
+ struct btrfs_fs_devices *cur_devices;
+ u64 num_devices;
+ int ret = 0;
+ bool clear_super = false;
+ char *dev_name = NULL;
+
+ mutex_lock(&uuid_mutex);
num_devices = root->fs_info->fs_devices->num_devices;
btrfs_dev_replace_lock(&root->fs_info->dev_replace, 0);
@@ -1722,78 +1815,23 @@
}
btrfs_dev_replace_unlock(&root->fs_info->dev_replace, 0);
- if ((all_avail & BTRFS_BLOCK_GROUP_RAID10) && num_devices <= 4) {
- ret = BTRFS_ERROR_DEV_RAID10_MIN_NOT_MET;
+ ret = btrfs_check_raid_min_devices(root->fs_info, num_devices - 1);
+ if (ret)
goto out;
- }
- if ((all_avail & BTRFS_BLOCK_GROUP_RAID1) && num_devices <= 2) {
- ret = BTRFS_ERROR_DEV_RAID1_MIN_NOT_MET;
+ ret = btrfs_find_device_by_devspec(root, devid, device_path,
+ &device);
+ if (ret)
goto out;
- }
-
- if ((all_avail & BTRFS_BLOCK_GROUP_RAID5) &&
- root->fs_info->fs_devices->rw_devices <= 2) {
- ret = BTRFS_ERROR_DEV_RAID5_MIN_NOT_MET;
- goto out;
- }
- if ((all_avail & BTRFS_BLOCK_GROUP_RAID6) &&
- root->fs_info->fs_devices->rw_devices <= 3) {
- ret = BTRFS_ERROR_DEV_RAID6_MIN_NOT_MET;
- goto out;
- }
-
- if (strcmp(device_path, "missing") == 0) {
- struct list_head *devices;
- struct btrfs_device *tmp;
-
- device = NULL;
- devices = &root->fs_info->fs_devices->devices;
- /*
- * It is safe to read the devices since the volume_mutex
- * is held.
- */
- list_for_each_entry(tmp, devices, dev_list) {
- if (tmp->in_fs_metadata &&
- !tmp->is_tgtdev_for_dev_replace &&
- !tmp->bdev) {
- device = tmp;
- break;
- }
- }
- bdev = NULL;
- bh = NULL;
- disk_super = NULL;
- if (!device) {
- ret = BTRFS_ERROR_DEV_MISSING_NOT_FOUND;
- goto out;
- }
- } else {
- ret = btrfs_get_bdev_and_sb(device_path,
- FMODE_WRITE | FMODE_EXCL,
- root->fs_info->bdev_holder, 0,
- &bdev, &bh);
- if (ret)
- goto out;
- disk_super = (struct btrfs_super_block *)bh->b_data;
- devid = btrfs_stack_device_id(&disk_super->dev_item);
- dev_uuid = disk_super->dev_item.uuid;
- device = btrfs_find_device(root->fs_info, devid, dev_uuid,
- disk_super->fsid);
- if (!device) {
- ret = -ENOENT;
- goto error_brelse;
- }
- }
if (device->is_tgtdev_for_dev_replace) {
ret = BTRFS_ERROR_DEV_TGT_REPLACE;
- goto error_brelse;
+ goto out;
}
if (device->writeable && root->fs_info->fs_devices->rw_devices == 1) {
ret = BTRFS_ERROR_DEV_ONLY_WRITABLE;
- goto error_brelse;
+ goto out;
}
if (device->writeable) {
@@ -1801,6 +1839,11 @@
list_del_init(&device->dev_alloc_list);
device->fs_devices->rw_devices--;
unlock_chunks(root);
+ dev_name = kstrdup(device->name->str, GFP_KERNEL);
+ if (!dev_name) {
+ ret = -ENOMEM;
+ goto error_undo;
+ }
clear_super = true;
}
@@ -1842,12 +1885,7 @@
if (device->missing)
device->fs_devices->missing_devices--;
- next_device = list_entry(root->fs_info->fs_devices->devices.next,
- struct btrfs_device, dev_list);
- if (device->bdev == root->fs_info->sb->s_bdev)
- root->fs_info->sb->s_bdev = next_device->bdev;
- if (device->bdev == root->fs_info->fs_devices->latest_bdev)
- root->fs_info->fs_devices->latest_bdev = next_device->bdev;
+ btrfs_assign_next_active_device(root->fs_info, device, NULL);
if (device->bdev) {
device->fs_devices->open_devices--;
@@ -1883,63 +1921,23 @@
* at this point, the device is zero sized. We want to
* remove it from the devices list and zero out the old super
*/
- if (clear_super && disk_super) {
- u64 bytenr;
- int i;
+ if (clear_super) {
+ struct block_device *bdev;
- /* make sure this device isn't detected as part of
- * the FS anymore
- */
- memset(&disk_super->magic, 0, sizeof(disk_super->magic));
- set_buffer_dirty(bh);
- sync_dirty_buffer(bh);
-
- /* clear the mirror copies of super block on the disk
- * being removed, 0th copy is been taken care above and
- * the below would take of the rest
- */
- for (i = 1; i < BTRFS_SUPER_MIRROR_MAX; i++) {
- bytenr = btrfs_sb_offset(i);
- if (bytenr + BTRFS_SUPER_INFO_SIZE >=
- i_size_read(bdev->bd_inode))
- break;
-
- brelse(bh);
- bh = __bread(bdev, bytenr / 4096,
- BTRFS_SUPER_INFO_SIZE);
- if (!bh)
- continue;
-
- disk_super = (struct btrfs_super_block *)bh->b_data;
-
- if (btrfs_super_bytenr(disk_super) != bytenr ||
- btrfs_super_magic(disk_super) != BTRFS_MAGIC) {
- continue;
- }
- memset(&disk_super->magic, 0,
- sizeof(disk_super->magic));
- set_buffer_dirty(bh);
- sync_dirty_buffer(bh);
+ bdev = blkdev_get_by_path(dev_name, FMODE_READ | FMODE_EXCL,
+ root->fs_info->bdev_holder);
+ if (!IS_ERR(bdev)) {
+ btrfs_scratch_superblocks(bdev, dev_name);
+ blkdev_put(bdev, FMODE_READ | FMODE_EXCL);
}
}
- ret = 0;
-
- if (bdev) {
- /* Notify udev that device has changed */
- btrfs_kobject_uevent(bdev, KOBJ_CHANGE);
-
- /* Update ctime/mtime for device path for libblkid */
- update_dev_time(device_path);
- }
-
-error_brelse:
- brelse(bh);
- if (bdev)
- blkdev_put(bdev, FMODE_READ | FMODE_EXCL);
out:
+ kfree(dev_name);
+
mutex_unlock(&uuid_mutex);
return ret;
+
error_undo:
if (device->writeable) {
lock_chunks(root);
@@ -1948,7 +1946,7 @@
device->fs_devices->rw_devices++;
unlock_chunks(root);
}
- goto error_brelse;
+ goto out;
}
void btrfs_rm_dev_replace_remove_srcdev(struct btrfs_fs_info *fs_info,
@@ -1972,11 +1970,8 @@
if (srcdev->missing)
fs_devices->missing_devices--;
- if (srcdev->writeable) {
+ if (srcdev->writeable)
fs_devices->rw_devices--;
- /* zero out the old super if it is writable */
- btrfs_scratch_superblocks(srcdev->bdev, srcdev->name->str);
- }
if (srcdev->bdev)
fs_devices->open_devices--;
@@ -1987,6 +1982,10 @@
{
struct btrfs_fs_devices *fs_devices = srcdev->fs_devices;
+ if (srcdev->writeable) {
+ /* zero out the old super if it is writable */
+ btrfs_scratch_superblocks(srcdev->bdev, srcdev->name->str);
+ }
call_rcu(&srcdev->rcu, free_device);
/*
@@ -2016,32 +2015,33 @@
void btrfs_destroy_dev_replace_tgtdev(struct btrfs_fs_info *fs_info,
struct btrfs_device *tgtdev)
{
- struct btrfs_device *next_device;
-
mutex_lock(&uuid_mutex);
WARN_ON(!tgtdev);
mutex_lock(&fs_info->fs_devices->device_list_mutex);
btrfs_sysfs_rm_device_link(fs_info->fs_devices, tgtdev);
- if (tgtdev->bdev) {
- btrfs_scratch_superblocks(tgtdev->bdev, tgtdev->name->str);
+ if (tgtdev->bdev)
fs_info->fs_devices->open_devices--;
- }
+
fs_info->fs_devices->num_devices--;
- next_device = list_entry(fs_info->fs_devices->devices.next,
- struct btrfs_device, dev_list);
- if (tgtdev->bdev == fs_info->sb->s_bdev)
- fs_info->sb->s_bdev = next_device->bdev;
- if (tgtdev->bdev == fs_info->fs_devices->latest_bdev)
- fs_info->fs_devices->latest_bdev = next_device->bdev;
- list_del_rcu(&tgtdev->dev_list);
+ btrfs_assign_next_active_device(fs_info, tgtdev, NULL);
- call_rcu(&tgtdev->rcu, free_device);
+ list_del_rcu(&tgtdev->dev_list);
mutex_unlock(&fs_info->fs_devices->device_list_mutex);
mutex_unlock(&uuid_mutex);
+
+ /*
+ * The update_dev_time() with in btrfs_scratch_superblocks()
+ * may lead to a call to btrfs_show_devname() which will try
+ * to hold device_list_mutex. And here this device
+ * is already out of device list, so we don't have to hold
+ * the device_list_mutex lock.
+ */
+ btrfs_scratch_superblocks(tgtdev->bdev, tgtdev->name->str);
+ call_rcu(&tgtdev->rcu, free_device);
}
static int btrfs_find_device_by_path(struct btrfs_root *root, char *device_path,
@@ -2102,6 +2102,31 @@
}
/*
+ * Lookup a device given by device id, or the path if the id is 0.
+ */
+int btrfs_find_device_by_devspec(struct btrfs_root *root, u64 devid,
+ char *devpath,
+ struct btrfs_device **device)
+{
+ int ret;
+
+ if (devid) {
+ ret = 0;
+ *device = btrfs_find_device(root->fs_info, devid, NULL,
+ NULL);
+ if (!*device)
+ ret = -ENOENT;
+ } else {
+ if (!devpath || !devpath[0])
+ return -EINVAL;
+
+ ret = btrfs_find_device_missing_or_by_path(root, devpath,
+ device);
+ }
+ return ret;
+}
+
+/*
* does all the dirty work required for changing file system's UUID.
*/
static int btrfs_prepare_sprout(struct btrfs_root *root)
@@ -2418,7 +2443,7 @@
ret = btrfs_relocate_sys_chunks(root);
if (ret < 0)
- btrfs_std_error(root->fs_info, ret,
+ btrfs_handle_fs_error(root->fs_info, ret,
"Failed to relocate sys chunks after "
"device initialization. This can be fixed "
"using the \"btrfs balance\" command.");
@@ -2663,7 +2688,7 @@
if (ret < 0)
goto out;
else if (ret > 0) { /* Logic error or corruption */
- btrfs_std_error(root->fs_info, -ENOENT,
+ btrfs_handle_fs_error(root->fs_info, -ENOENT,
"Failed lookup while freeing chunk.");
ret = -ENOENT;
goto out;
@@ -2671,7 +2696,7 @@
ret = btrfs_del_item(trans, root, path);
if (ret < 0)
- btrfs_std_error(root->fs_info, ret,
+ btrfs_handle_fs_error(root->fs_info, ret,
"Failed to delete chunk item.");
out:
btrfs_free_path(path);
@@ -2857,7 +2882,7 @@
chunk_offset);
if (IS_ERR(trans)) {
ret = PTR_ERR(trans);
- btrfs_std_error(root->fs_info, ret, NULL);
+ btrfs_handle_fs_error(root->fs_info, ret, NULL);
return ret;
}
@@ -3402,6 +3427,7 @@
u32 count_meta = 0;
u32 count_sys = 0;
int chunk_reserved = 0;
+ u64 bytes_used = 0;
/* step one make some room on all the devices */
devices = &fs_info->fs_devices->devices;
@@ -3540,7 +3566,13 @@
goto loop;
}
- if ((chunk_type & BTRFS_BLOCK_GROUP_DATA) && !chunk_reserved) {
+ ASSERT(fs_info->data_sinfo);
+ spin_lock(&fs_info->data_sinfo->lock);
+ bytes_used = fs_info->data_sinfo->bytes_used;
+ spin_unlock(&fs_info->data_sinfo->lock);
+
+ if ((chunk_type & BTRFS_BLOCK_GROUP_DATA) &&
+ !chunk_reserved && !bytes_used) {
trans = btrfs_start_transaction(chunk_root, 0);
if (IS_ERR(trans)) {
mutex_unlock(&fs_info->delete_unused_bgs_mutex);
@@ -3632,7 +3664,7 @@
unset_balance_control(fs_info);
ret = del_balance_item(fs_info->tree_root);
if (ret)
- btrfs_std_error(fs_info, ret, NULL);
+ btrfs_handle_fs_error(fs_info, ret, NULL);
atomic_set(&fs_info->mutually_exclusive_operation_running, 0);
}
@@ -3693,10 +3725,8 @@
num_devices--;
}
btrfs_dev_replace_unlock(&fs_info->dev_replace, 0);
- allowed = BTRFS_AVAIL_ALLOC_BIT_SINGLE;
- if (num_devices == 1)
- allowed |= BTRFS_BLOCK_GROUP_DUP;
- else if (num_devices > 1)
+ allowed = BTRFS_AVAIL_ALLOC_BIT_SINGLE | BTRFS_BLOCK_GROUP_DUP;
+ if (num_devices > 1)
allowed |= (BTRFS_BLOCK_GROUP_RAID0 | BTRFS_BLOCK_GROUP_RAID1);
if (num_devices > 2)
allowed |= BTRFS_BLOCK_GROUP_RAID5;
@@ -5278,7 +5308,15 @@
stripe_nr = div64_u64(stripe_nr, stripe_len);
stripe_offset = stripe_nr * stripe_len;
- BUG_ON(offset < stripe_offset);
+ if (offset < stripe_offset) {
+ btrfs_crit(fs_info, "stripe math has gone wrong, "
+ "stripe_offset=%llu, offset=%llu, start=%llu, "
+ "logical=%llu, stripe_len=%llu",
+ stripe_offset, offset, em->start, logical,
+ stripe_len);
+ free_extent_map(em);
+ return -EINVAL;
+ }
/* stripe_offset is the offset of this block in its stripe*/
stripe_offset = offset - stripe_offset;
@@ -5519,7 +5557,13 @@
&stripe_index);
mirror_num = stripe_index + 1;
}
- BUG_ON(stripe_index >= map->num_stripes);
+ if (stripe_index >= map->num_stripes) {
+ btrfs_crit(fs_info, "stripe index math went horribly wrong, "
+ "got stripe_index=%u, num_stripes=%u",
+ stripe_index, map->num_stripes);
+ ret = -EINVAL;
+ goto out;
+ }
num_alloc_stripes = num_stripes;
if (dev_replace_is_ongoing) {
@@ -6242,7 +6286,7 @@
"invalid chunk length %llu", length);
return -EIO;
}
- if (!is_power_of_2(stripe_len)) {
+ if (!is_power_of_2(stripe_len) || stripe_len != BTRFS_STRIPE_LEN) {
btrfs_err(root->fs_info, "invalid chunk stripe length: %llu",
stripe_len);
return -EIO;
diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h
index 1939ebd..0ac90f8 100644
--- a/fs/btrfs/volumes.h
+++ b/fs/btrfs/volumes.h
@@ -340,14 +340,14 @@
};
extern const struct btrfs_raid_attr btrfs_raid_array[BTRFS_NR_RAID_TYPES];
-
+extern const int btrfs_raid_mindev_error[BTRFS_NR_RAID_TYPES];
extern const u64 btrfs_raid_group[BTRFS_NR_RAID_TYPES];
struct map_lookup {
u64 type;
int io_align;
int io_width;
- int stripe_len;
+ u64 stripe_len;
int sector_size;
int num_stripes;
int sub_stripes;
@@ -357,52 +357,6 @@
#define map_lookup_size(n) (sizeof(struct map_lookup) + \
(sizeof(struct btrfs_bio_stripe) * (n)))
-/*
- * Restriper's general type filter
- */
-#define BTRFS_BALANCE_DATA (1ULL << 0)
-#define BTRFS_BALANCE_SYSTEM (1ULL << 1)
-#define BTRFS_BALANCE_METADATA (1ULL << 2)
-
-#define BTRFS_BALANCE_TYPE_MASK (BTRFS_BALANCE_DATA | \
- BTRFS_BALANCE_SYSTEM | \
- BTRFS_BALANCE_METADATA)
-
-#define BTRFS_BALANCE_FORCE (1ULL << 3)
-#define BTRFS_BALANCE_RESUME (1ULL << 4)
-
-/*
- * Balance filters
- */
-#define BTRFS_BALANCE_ARGS_PROFILES (1ULL << 0)
-#define BTRFS_BALANCE_ARGS_USAGE (1ULL << 1)
-#define BTRFS_BALANCE_ARGS_DEVID (1ULL << 2)
-#define BTRFS_BALANCE_ARGS_DRANGE (1ULL << 3)
-#define BTRFS_BALANCE_ARGS_VRANGE (1ULL << 4)
-#define BTRFS_BALANCE_ARGS_LIMIT (1ULL << 5)
-#define BTRFS_BALANCE_ARGS_LIMIT_RANGE (1ULL << 6)
-#define BTRFS_BALANCE_ARGS_STRIPES_RANGE (1ULL << 7)
-#define BTRFS_BALANCE_ARGS_USAGE_RANGE (1ULL << 10)
-
-#define BTRFS_BALANCE_ARGS_MASK \
- (BTRFS_BALANCE_ARGS_PROFILES | \
- BTRFS_BALANCE_ARGS_USAGE | \
- BTRFS_BALANCE_ARGS_DEVID | \
- BTRFS_BALANCE_ARGS_DRANGE | \
- BTRFS_BALANCE_ARGS_VRANGE | \
- BTRFS_BALANCE_ARGS_LIMIT | \
- BTRFS_BALANCE_ARGS_LIMIT_RANGE | \
- BTRFS_BALANCE_ARGS_STRIPES_RANGE | \
- BTRFS_BALANCE_ARGS_USAGE_RANGE)
-
-/*
- * Profile changing flags. When SOFT is set we won't relocate chunk if
- * it already has the target profile (even though it may be
- * half-filled).
- */
-#define BTRFS_BALANCE_ARGS_CONVERT (1ULL << 8)
-#define BTRFS_BALANCE_ARGS_SOFT (1ULL << 9)
-
struct btrfs_balance_args;
struct btrfs_balance_progress;
struct btrfs_balance_control {
@@ -445,13 +399,18 @@
struct btrfs_fs_devices **fs_devices_ret);
int btrfs_close_devices(struct btrfs_fs_devices *fs_devices);
void btrfs_close_extra_devices(struct btrfs_fs_devices *fs_devices, int step);
+void btrfs_assign_next_active_device(struct btrfs_fs_info *fs_info,
+ struct btrfs_device *device, struct btrfs_device *this_dev);
int btrfs_find_device_missing_or_by_path(struct btrfs_root *root,
char *device_path,
struct btrfs_device **device);
+int btrfs_find_device_by_devspec(struct btrfs_root *root, u64 devid,
+ char *devpath,
+ struct btrfs_device **device);
struct btrfs_device *btrfs_alloc_device(struct btrfs_fs_info *fs_info,
const u64 *devid,
const u8 *uuid);
-int btrfs_rm_device(struct btrfs_root *root, char *device_path);
+int btrfs_rm_device(struct btrfs_root *root, char *device_path, u64 devid);
void btrfs_cleanup_fs_uuids(void);
int btrfs_num_copies(struct btrfs_fs_info *fs_info, u64 logical, u64 len);
int btrfs_grow_device(struct btrfs_trans_handle *trans,
diff --git a/fs/buffer.c b/fs/buffer.c
index af0d9a8..754813a 100644
--- a/fs/buffer.c
+++ b/fs/buffer.c
@@ -255,17 +255,17 @@
*/
static void free_more_memory(void)
{
- struct zone *zone;
+ struct zoneref *z;
int nid;
wakeup_flusher_threads(1024, WB_REASON_FREE_MORE_MEM);
yield();
for_each_online_node(nid) {
- (void)first_zones_zonelist(node_zonelist(nid, GFP_NOFS),
- gfp_zone(GFP_NOFS), NULL,
- &zone);
- if (zone)
+
+ z = first_zones_zonelist(node_zonelist(nid, GFP_NOFS),
+ gfp_zone(GFP_NOFS), NULL);
+ if (z->zone)
try_to_free_pages(node_zonelist(nid, GFP_NOFS), 0,
GFP_NOFS, NULL);
}
diff --git a/fs/char_dev.c b/fs/char_dev.c
index 24b1425..687471d 100644
--- a/fs/char_dev.c
+++ b/fs/char_dev.c
@@ -91,6 +91,10 @@
break;
}
+ if (i < CHRDEV_MAJOR_DYN_END)
+ pr_warn("CHRDEV \"%s\" major number %d goes below the dynamic allocation range",
+ name, i);
+
if (i == 0) {
ret = -EBUSY;
goto out;
diff --git a/fs/cifs/cifs_spnego.c b/fs/cifs/cifs_spnego.c
index 6908080..b611fc2 100644
--- a/fs/cifs/cifs_spnego.c
+++ b/fs/cifs/cifs_spnego.c
@@ -24,10 +24,13 @@
#include <linux/string.h>
#include <keys/user-type.h>
#include <linux/key-type.h>
+#include <linux/keyctl.h>
#include <linux/inet.h>
#include "cifsglob.h"
#include "cifs_spnego.h"
#include "cifs_debug.h"
+#include "cifsproto.h"
+static const struct cred *spnego_cred;
/* create a new cifs key */
static int
@@ -102,6 +105,7 @@
size_t desc_len;
struct key *spnego_key;
const char *hostname = server->hostname;
+ const struct cred *saved_cred;
/* length of fields (with semicolons): ver=0xyz ip4=ipaddress
host=hostname sec=mechanism uid=0xFF user=username */
@@ -163,7 +167,9 @@
sprintf(dp, ";pid=0x%x", current->pid);
cifs_dbg(FYI, "key description = %s\n", description);
+ saved_cred = override_creds(spnego_cred);
spnego_key = request_key(&cifs_spnego_key_type, description, "");
+ revert_creds(saved_cred);
#ifdef CONFIG_CIFS_DEBUG2
if (cifsFYI && !IS_ERR(spnego_key)) {
@@ -177,3 +183,64 @@
kfree(description);
return spnego_key;
}
+
+int
+init_cifs_spnego(void)
+{
+ struct cred *cred;
+ struct key *keyring;
+ int ret;
+
+ cifs_dbg(FYI, "Registering the %s key type\n",
+ cifs_spnego_key_type.name);
+
+ /*
+ * Create an override credential set with special thread keyring for
+ * spnego upcalls.
+ */
+
+ cred = prepare_kernel_cred(NULL);
+ if (!cred)
+ return -ENOMEM;
+
+ keyring = keyring_alloc(".cifs_spnego",
+ GLOBAL_ROOT_UID, GLOBAL_ROOT_GID, cred,
+ (KEY_POS_ALL & ~KEY_POS_SETATTR) |
+ KEY_USR_VIEW | KEY_USR_READ,
+ KEY_ALLOC_NOT_IN_QUOTA, NULL, NULL);
+ if (IS_ERR(keyring)) {
+ ret = PTR_ERR(keyring);
+ goto failed_put_cred;
+ }
+
+ ret = register_key_type(&cifs_spnego_key_type);
+ if (ret < 0)
+ goto failed_put_key;
+
+ /*
+ * instruct request_key() to use this special keyring as a cache for
+ * the results it looks up
+ */
+ set_bit(KEY_FLAG_ROOT_CAN_CLEAR, &keyring->flags);
+ cred->thread_keyring = keyring;
+ cred->jit_keyring = KEY_REQKEY_DEFL_THREAD_KEYRING;
+ spnego_cred = cred;
+
+ cifs_dbg(FYI, "cifs spnego keyring: %d\n", key_serial(keyring));
+ return 0;
+
+failed_put_key:
+ key_put(keyring);
+failed_put_cred:
+ put_cred(cred);
+ return ret;
+}
+
+void
+exit_cifs_spnego(void)
+{
+ key_revoke(spnego_cred->thread_keyring);
+ unregister_key_type(&cifs_spnego_key_type);
+ put_cred(spnego_cred);
+ cifs_dbg(FYI, "Unregistered %s key type\n", cifs_spnego_key_type.name);
+}
diff --git a/fs/cifs/cifsacl.c b/fs/cifs/cifsacl.c
index 3f93125..71e8a56 100644
--- a/fs/cifs/cifsacl.c
+++ b/fs/cifs/cifsacl.c
@@ -360,7 +360,7 @@
GLOBAL_ROOT_UID, GLOBAL_ROOT_GID, cred,
(KEY_POS_ALL & ~KEY_POS_SETATTR) |
KEY_USR_VIEW | KEY_USR_READ,
- KEY_ALLOC_NOT_IN_QUOTA, NULL);
+ KEY_ALLOC_NOT_IN_QUOTA, NULL, NULL);
if (IS_ERR(keyring)) {
ret = PTR_ERR(keyring);
goto failed_put_cred;
diff --git a/fs/cifs/cifsfs.c b/fs/cifs/cifsfs.c
index 08fa36e..5d8b7ed 100644
--- a/fs/cifs/cifsfs.c
+++ b/fs/cifs/cifsfs.c
@@ -890,7 +890,6 @@
.rmdir = cifs_rmdir,
.rename2 = cifs_rename2,
.permission = cifs_permission,
-/* revalidate:cifs_revalidate, */
.setattr = cifs_setattr,
.symlink = cifs_symlink,
.mknod = cifs_mknod,
@@ -901,9 +900,8 @@
};
const struct inode_operations cifs_file_inode_ops = {
-/* revalidate:cifs_revalidate, */
.setattr = cifs_setattr,
- .getattr = cifs_getattr, /* do we need this anymore? */
+ .getattr = cifs_getattr,
.permission = cifs_permission,
.setxattr = generic_setxattr,
.getxattr = generic_getxattr,
@@ -915,9 +913,6 @@
.readlink = generic_readlink,
.get_link = cifs_get_link,
.permission = cifs_permission,
- /* BB add the following two eventually */
- /* revalidate: cifs_revalidate,
- setattr: cifs_notify_change, *//* BB do we need notify change */
.setxattr = generic_setxattr,
.getxattr = generic_getxattr,
.listxattr = cifs_listxattr,
@@ -1303,7 +1298,7 @@
goto out_destroy_mids;
#ifdef CONFIG_CIFS_UPCALL
- rc = register_key_type(&cifs_spnego_key_type);
+ rc = init_cifs_spnego();
if (rc)
goto out_destroy_request_bufs;
#endif /* CONFIG_CIFS_UPCALL */
@@ -1326,7 +1321,7 @@
out_register_key_type:
#endif
#ifdef CONFIG_CIFS_UPCALL
- unregister_key_type(&cifs_spnego_key_type);
+ exit_cifs_spnego();
out_destroy_request_bufs:
#endif
cifs_destroy_request_bufs();
diff --git a/fs/cifs/cifsproto.h b/fs/cifs/cifsproto.h
index 0f9a6bc..1243bd3 100644
--- a/fs/cifs/cifsproto.h
+++ b/fs/cifs/cifsproto.h
@@ -58,6 +58,8 @@
} while (0)
extern int init_cifs_idmap(void);
extern void exit_cifs_idmap(void);
+extern int init_cifs_spnego(void);
+extern void exit_cifs_spnego(void);
extern char *build_path_from_dentry(struct dentry *);
extern char *cifs_build_path_to_root(struct smb_vol *vol,
struct cifs_sb_info *cifs_sb,
diff --git a/fs/compat.c b/fs/compat.c
index 8754e9a..be6e48b 100644
--- a/fs/compat.c
+++ b/fs/compat.c
@@ -936,6 +936,8 @@
}
dirent = buf->previous;
if (dirent) {
+ if (signal_pending(current))
+ return -EINTR;
if (__put_user(offset, &dirent->d_off))
goto efault;
}
@@ -1020,6 +1022,8 @@
dirent = buf->previous;
if (dirent) {
+ if (signal_pending(current))
+ return -EINTR;
if (__put_user_unaligned(offset, &dirent->d_off))
goto efault;
}
diff --git a/fs/coredump.c b/fs/coredump.c
index 492c2db..38a7ab8 100644
--- a/fs/coredump.c
+++ b/fs/coredump.c
@@ -413,7 +413,9 @@
core_state->dumper.task = tsk;
core_state->dumper.next = NULL;
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
+
if (!mm->core_state)
core_waiters = zap_threads(tsk, mm, core_state, exit_code);
up_write(&mm->mmap_sem);
diff --git a/fs/crypto/keyinfo.c b/fs/crypto/keyinfo.c
index 06f5aa4..1ac263e 100644
--- a/fs/crypto/keyinfo.c
+++ b/fs/crypto/keyinfo.c
@@ -78,6 +78,67 @@
return res;
}
+static int validate_user_key(struct fscrypt_info *crypt_info,
+ struct fscrypt_context *ctx, u8 *raw_key,
+ u8 *prefix, int prefix_size)
+{
+ u8 *full_key_descriptor;
+ struct key *keyring_key;
+ struct fscrypt_key *master_key;
+ const struct user_key_payload *ukp;
+ int full_key_len = prefix_size + (FS_KEY_DESCRIPTOR_SIZE * 2) + 1;
+ int res;
+
+ full_key_descriptor = kmalloc(full_key_len, GFP_NOFS);
+ if (!full_key_descriptor)
+ return -ENOMEM;
+
+ memcpy(full_key_descriptor, prefix, prefix_size);
+ sprintf(full_key_descriptor + prefix_size,
+ "%*phN", FS_KEY_DESCRIPTOR_SIZE,
+ ctx->master_key_descriptor);
+ full_key_descriptor[full_key_len - 1] = '\0';
+ keyring_key = request_key(&key_type_logon, full_key_descriptor, NULL);
+ kfree(full_key_descriptor);
+ if (IS_ERR(keyring_key))
+ return PTR_ERR(keyring_key);
+
+ if (keyring_key->type != &key_type_logon) {
+ printk_once(KERN_WARNING
+ "%s: key type must be logon\n", __func__);
+ res = -ENOKEY;
+ goto out;
+ }
+ down_read(&keyring_key->sem);
+ ukp = user_key_payload(keyring_key);
+ if (ukp->datalen != sizeof(struct fscrypt_key)) {
+ res = -EINVAL;
+ up_read(&keyring_key->sem);
+ goto out;
+ }
+ master_key = (struct fscrypt_key *)ukp->data;
+ BUILD_BUG_ON(FS_AES_128_ECB_KEY_SIZE != FS_KEY_DERIVATION_NONCE_SIZE);
+
+ if (master_key->size != FS_AES_256_XTS_KEY_SIZE) {
+ printk_once(KERN_WARNING
+ "%s: key size incorrect: %d\n",
+ __func__, master_key->size);
+ res = -ENOKEY;
+ up_read(&keyring_key->sem);
+ goto out;
+ }
+ res = derive_key_aes(ctx->nonce, master_key->raw, raw_key);
+ up_read(&keyring_key->sem);
+ if (res)
+ goto out;
+
+ crypt_info->ci_keyring_key = keyring_key;
+ return 0;
+out:
+ key_put(keyring_key);
+ return res;
+}
+
static void put_crypt_info(struct fscrypt_info *ci)
{
if (!ci)
@@ -91,12 +152,7 @@
int get_crypt_info(struct inode *inode)
{
struct fscrypt_info *crypt_info;
- u8 full_key_descriptor[FS_KEY_DESC_PREFIX_SIZE +
- (FS_KEY_DESCRIPTOR_SIZE * 2) + 1];
- struct key *keyring_key = NULL;
- struct fscrypt_key *master_key;
struct fscrypt_context ctx;
- const struct user_key_payload *ukp;
struct crypto_skcipher *ctfm;
const char *cipher_str;
u8 raw_key[FS_MAX_KEY_SIZE];
@@ -167,48 +223,24 @@
memset(raw_key, 0x42, FS_AES_256_XTS_KEY_SIZE);
goto got_key;
}
- memcpy(full_key_descriptor, FS_KEY_DESC_PREFIX,
- FS_KEY_DESC_PREFIX_SIZE);
- sprintf(full_key_descriptor + FS_KEY_DESC_PREFIX_SIZE,
- "%*phN", FS_KEY_DESCRIPTOR_SIZE,
- ctx.master_key_descriptor);
- full_key_descriptor[FS_KEY_DESC_PREFIX_SIZE +
- (2 * FS_KEY_DESCRIPTOR_SIZE)] = '\0';
- keyring_key = request_key(&key_type_logon, full_key_descriptor, NULL);
- if (IS_ERR(keyring_key)) {
- res = PTR_ERR(keyring_key);
- keyring_key = NULL;
- goto out;
- }
- crypt_info->ci_keyring_key = keyring_key;
- if (keyring_key->type != &key_type_logon) {
- printk_once(KERN_WARNING
- "%s: key type must be logon\n", __func__);
- res = -ENOKEY;
- goto out;
- }
- down_read(&keyring_key->sem);
- ukp = user_key_payload(keyring_key);
- if (ukp->datalen != sizeof(struct fscrypt_key)) {
- res = -EINVAL;
- up_read(&keyring_key->sem);
- goto out;
- }
- master_key = (struct fscrypt_key *)ukp->data;
- BUILD_BUG_ON(FS_AES_128_ECB_KEY_SIZE != FS_KEY_DERIVATION_NONCE_SIZE);
- if (master_key->size != FS_AES_256_XTS_KEY_SIZE) {
- printk_once(KERN_WARNING
- "%s: key size incorrect: %d\n",
- __func__, master_key->size);
- res = -ENOKEY;
- up_read(&keyring_key->sem);
+ res = validate_user_key(crypt_info, &ctx, raw_key,
+ FS_KEY_DESC_PREFIX, FS_KEY_DESC_PREFIX_SIZE);
+ if (res && inode->i_sb->s_cop->key_prefix) {
+ u8 *prefix = NULL;
+ int prefix_size, res2;
+
+ prefix_size = inode->i_sb->s_cop->key_prefix(inode, &prefix);
+ res2 = validate_user_key(crypt_info, &ctx, raw_key,
+ prefix, prefix_size);
+ if (res2) {
+ if (res2 == -ENOKEY)
+ res = -ENOKEY;
+ goto out;
+ }
+ } else if (res) {
goto out;
}
- res = derive_key_aes(ctx.nonce, master_key->raw, raw_key);
- up_read(&keyring_key->sem);
- if (res)
- goto out;
got_key:
ctfm = crypto_alloc_skcipher(cipher_str, 0, 0);
if (!ctfm || IS_ERR(ctfm)) {
diff --git a/fs/dax.c b/fs/dax.c
index 0dbe4e0..7d9df93 100644
--- a/fs/dax.c
+++ b/fs/dax.c
@@ -32,6 +32,15 @@
#include <linux/pfn_t.h>
#include <linux/sizes.h>
+#define RADIX_DAX_MASK 0xf
+#define RADIX_DAX_SHIFT 4
+#define RADIX_DAX_PTE (0x4 | RADIX_TREE_EXCEPTIONAL_ENTRY)
+#define RADIX_DAX_PMD (0x8 | RADIX_TREE_EXCEPTIONAL_ENTRY)
+#define RADIX_DAX_TYPE(entry) ((unsigned long)entry & RADIX_DAX_MASK)
+#define RADIX_DAX_SECTOR(entry) (((unsigned long)entry >> RADIX_DAX_SHIFT))
+#define RADIX_DAX_ENTRY(sector, pmd) ((void *)((unsigned long)sector << \
+ RADIX_DAX_SHIFT | (pmd ? RADIX_DAX_PMD : RADIX_DAX_PTE)))
+
static long dax_map_atomic(struct block_device *bdev, struct blk_dax_ctl *dax)
{
struct request_queue *q = bdev->bd_queue;
@@ -667,7 +676,7 @@
if (error)
goto unlock_page;
- if (!buffer_mapped(&bh) && !buffer_unwritten(&bh) && !vmf->cow_page) {
+ if (!buffer_mapped(&bh) && !vmf->cow_page) {
if (vmf->flags & FAULT_FLAG_WRITE) {
error = get_block(inode, block, &bh, 1);
count_vm_event(PGMAJFAULT);
diff --git a/fs/debugfs/file.c b/fs/debugfs/file.c
index d2ba12e..9c1c9a0 100644
--- a/fs/debugfs/file.c
+++ b/fs/debugfs/file.c
@@ -22,6 +22,12 @@
#include <linux/slab.h>
#include <linux/atomic.h>
#include <linux/device.h>
+#include <linux/srcu.h>
+#include <asm/poll.h>
+
+#include "internal.h"
+
+struct poll_table_struct;
static ssize_t default_read_file(struct file *file, char __user *buf,
size_t count, loff_t *ppos)
@@ -35,27 +41,293 @@
return count;
}
-const struct file_operations debugfs_file_operations = {
+const struct file_operations debugfs_noop_file_operations = {
.read = default_read_file,
.write = default_write_file,
.open = simple_open,
.llseek = noop_llseek,
};
-static struct dentry *debugfs_create_mode(const char *name, umode_t mode,
- struct dentry *parent, void *value,
- const struct file_operations *fops,
- const struct file_operations *fops_ro,
- const struct file_operations *fops_wo)
+/**
+ * debugfs_use_file_start - mark the beginning of file data access
+ * @dentry: the dentry object whose data is being accessed.
+ * @srcu_idx: a pointer to some memory to store a SRCU index in.
+ *
+ * Up to a matching call to debugfs_use_file_finish(), any
+ * successive call into the file removing functions debugfs_remove()
+ * and debugfs_remove_recursive() will block. Since associated private
+ * file data may only get freed after a successful return of any of
+ * the removal functions, you may safely access it after a successful
+ * call to debugfs_use_file_start() without worrying about
+ * lifetime issues.
+ *
+ * If -%EIO is returned, the file has already been removed and thus,
+ * it is not safe to access any of its data. If, on the other hand,
+ * it is allowed to access the file data, zero is returned.
+ *
+ * Regardless of the return code, any call to
+ * debugfs_use_file_start() must be followed by a matching call
+ * to debugfs_use_file_finish().
+ */
+int debugfs_use_file_start(const struct dentry *dentry, int *srcu_idx)
+ __acquires(&debugfs_srcu)
+{
+ *srcu_idx = srcu_read_lock(&debugfs_srcu);
+ barrier();
+ if (d_unlinked(dentry))
+ return -EIO;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(debugfs_use_file_start);
+
+/**
+ * debugfs_use_file_finish - mark the end of file data access
+ * @srcu_idx: the SRCU index "created" by a former call to
+ * debugfs_use_file_start().
+ *
+ * Allow any ongoing concurrent call into debugfs_remove() or
+ * debugfs_remove_recursive() blocked by a former call to
+ * debugfs_use_file_start() to proceed and return to its caller.
+ */
+void debugfs_use_file_finish(int srcu_idx) __releases(&debugfs_srcu)
+{
+ srcu_read_unlock(&debugfs_srcu, srcu_idx);
+}
+EXPORT_SYMBOL_GPL(debugfs_use_file_finish);
+
+#define F_DENTRY(filp) ((filp)->f_path.dentry)
+
+#define REAL_FOPS_DEREF(dentry) \
+ ((const struct file_operations *)(dentry)->d_fsdata)
+
+static int open_proxy_open(struct inode *inode, struct file *filp)
+{
+ const struct dentry *dentry = F_DENTRY(filp);
+ const struct file_operations *real_fops = NULL;
+ int srcu_idx, r;
+
+ r = debugfs_use_file_start(dentry, &srcu_idx);
+ if (r) {
+ r = -ENOENT;
+ goto out;
+ }
+
+ real_fops = REAL_FOPS_DEREF(dentry);
+ real_fops = fops_get(real_fops);
+ if (!real_fops) {
+ /* Huh? Module did not clean up after itself at exit? */
+ WARN(1, "debugfs file owner did not clean up at exit: %pd",
+ dentry);
+ r = -ENXIO;
+ goto out;
+ }
+ replace_fops(filp, real_fops);
+
+ if (real_fops->open)
+ r = real_fops->open(inode, filp);
+
+out:
+ fops_put(real_fops);
+ debugfs_use_file_finish(srcu_idx);
+ return r;
+}
+
+const struct file_operations debugfs_open_proxy_file_operations = {
+ .open = open_proxy_open,
+};
+
+#define PROTO(args...) args
+#define ARGS(args...) args
+
+#define FULL_PROXY_FUNC(name, ret_type, filp, proto, args) \
+static ret_type full_proxy_ ## name(proto) \
+{ \
+ const struct dentry *dentry = F_DENTRY(filp); \
+ const struct file_operations *real_fops = \
+ REAL_FOPS_DEREF(dentry); \
+ int srcu_idx; \
+ ret_type r; \
+ \
+ r = debugfs_use_file_start(dentry, &srcu_idx); \
+ if (likely(!r)) \
+ r = real_fops->name(args); \
+ debugfs_use_file_finish(srcu_idx); \
+ return r; \
+}
+
+FULL_PROXY_FUNC(llseek, loff_t, filp,
+ PROTO(struct file *filp, loff_t offset, int whence),
+ ARGS(filp, offset, whence));
+
+FULL_PROXY_FUNC(read, ssize_t, filp,
+ PROTO(struct file *filp, char __user *buf, size_t size,
+ loff_t *ppos),
+ ARGS(filp, buf, size, ppos));
+
+FULL_PROXY_FUNC(write, ssize_t, filp,
+ PROTO(struct file *filp, const char __user *buf, size_t size,
+ loff_t *ppos),
+ ARGS(filp, buf, size, ppos));
+
+FULL_PROXY_FUNC(unlocked_ioctl, long, filp,
+ PROTO(struct file *filp, unsigned int cmd, unsigned long arg),
+ ARGS(filp, cmd, arg));
+
+static unsigned int full_proxy_poll(struct file *filp,
+ struct poll_table_struct *wait)
+{
+ const struct dentry *dentry = F_DENTRY(filp);
+ const struct file_operations *real_fops = REAL_FOPS_DEREF(dentry);
+ int srcu_idx;
+ unsigned int r = 0;
+
+ if (debugfs_use_file_start(dentry, &srcu_idx)) {
+ debugfs_use_file_finish(srcu_idx);
+ return POLLHUP;
+ }
+
+ r = real_fops->poll(filp, wait);
+ debugfs_use_file_finish(srcu_idx);
+ return r;
+}
+
+static int full_proxy_release(struct inode *inode, struct file *filp)
+{
+ const struct dentry *dentry = F_DENTRY(filp);
+ const struct file_operations *real_fops = REAL_FOPS_DEREF(dentry);
+ const struct file_operations *proxy_fops = filp->f_op;
+ int r = 0;
+
+ /*
+ * We must not protect this against removal races here: the
+ * original releaser should be called unconditionally in order
+ * not to leak any resources. Releasers must not assume that
+ * ->i_private is still being meaningful here.
+ */
+ if (real_fops->release)
+ r = real_fops->release(inode, filp);
+
+ replace_fops(filp, d_inode(dentry)->i_fop);
+ kfree((void *)proxy_fops);
+ fops_put(real_fops);
+ return 0;
+}
+
+static void __full_proxy_fops_init(struct file_operations *proxy_fops,
+ const struct file_operations *real_fops)
+{
+ proxy_fops->release = full_proxy_release;
+ if (real_fops->llseek)
+ proxy_fops->llseek = full_proxy_llseek;
+ if (real_fops->read)
+ proxy_fops->read = full_proxy_read;
+ if (real_fops->write)
+ proxy_fops->write = full_proxy_write;
+ if (real_fops->poll)
+ proxy_fops->poll = full_proxy_poll;
+ if (real_fops->unlocked_ioctl)
+ proxy_fops->unlocked_ioctl = full_proxy_unlocked_ioctl;
+}
+
+static int full_proxy_open(struct inode *inode, struct file *filp)
+{
+ const struct dentry *dentry = F_DENTRY(filp);
+ const struct file_operations *real_fops = NULL;
+ struct file_operations *proxy_fops = NULL;
+ int srcu_idx, r;
+
+ r = debugfs_use_file_start(dentry, &srcu_idx);
+ if (r) {
+ r = -ENOENT;
+ goto out;
+ }
+
+ real_fops = REAL_FOPS_DEREF(dentry);
+ real_fops = fops_get(real_fops);
+ if (!real_fops) {
+ /* Huh? Module did not cleanup after itself at exit? */
+ WARN(1, "debugfs file owner did not clean up at exit: %pd",
+ dentry);
+ r = -ENXIO;
+ goto out;
+ }
+
+ proxy_fops = kzalloc(sizeof(*proxy_fops), GFP_KERNEL);
+ if (!proxy_fops) {
+ r = -ENOMEM;
+ goto free_proxy;
+ }
+ __full_proxy_fops_init(proxy_fops, real_fops);
+ replace_fops(filp, proxy_fops);
+
+ if (real_fops->open) {
+ r = real_fops->open(inode, filp);
+
+ if (filp->f_op != proxy_fops) {
+ /* No protection against file removal anymore. */
+ WARN(1, "debugfs file owner replaced proxy fops: %pd",
+ dentry);
+ goto free_proxy;
+ }
+ }
+
+ goto out;
+free_proxy:
+ kfree(proxy_fops);
+ fops_put(real_fops);
+out:
+ debugfs_use_file_finish(srcu_idx);
+ return r;
+}
+
+const struct file_operations debugfs_full_proxy_file_operations = {
+ .open = full_proxy_open,
+};
+
+ssize_t debugfs_attr_read(struct file *file, char __user *buf,
+ size_t len, loff_t *ppos)
+{
+ ssize_t ret;
+ int srcu_idx;
+
+ ret = debugfs_use_file_start(F_DENTRY(file), &srcu_idx);
+ if (likely(!ret))
+ ret = simple_attr_read(file, buf, len, ppos);
+ debugfs_use_file_finish(srcu_idx);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(debugfs_attr_read);
+
+ssize_t debugfs_attr_write(struct file *file, const char __user *buf,
+ size_t len, loff_t *ppos)
+{
+ ssize_t ret;
+ int srcu_idx;
+
+ ret = debugfs_use_file_start(F_DENTRY(file), &srcu_idx);
+ if (likely(!ret))
+ ret = simple_attr_write(file, buf, len, ppos);
+ debugfs_use_file_finish(srcu_idx);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(debugfs_attr_write);
+
+static struct dentry *debugfs_create_mode_unsafe(const char *name, umode_t mode,
+ struct dentry *parent, void *value,
+ const struct file_operations *fops,
+ const struct file_operations *fops_ro,
+ const struct file_operations *fops_wo)
{
/* if there are no write bits set, make read only */
if (!(mode & S_IWUGO))
- return debugfs_create_file(name, mode, parent, value, fops_ro);
+ return debugfs_create_file_unsafe(name, mode, parent, value,
+ fops_ro);
/* if there are no read bits set, make write only */
if (!(mode & S_IRUGO))
- return debugfs_create_file(name, mode, parent, value, fops_wo);
+ return debugfs_create_file_unsafe(name, mode, parent, value,
+ fops_wo);
- return debugfs_create_file(name, mode, parent, value, fops);
+ return debugfs_create_file_unsafe(name, mode, parent, value, fops);
}
static int debugfs_u8_set(void *data, u64 val)
@@ -68,9 +340,9 @@
*val = *(u8 *)data;
return 0;
}
-DEFINE_SIMPLE_ATTRIBUTE(fops_u8, debugfs_u8_get, debugfs_u8_set, "%llu\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_u8_ro, debugfs_u8_get, NULL, "%llu\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_u8_wo, NULL, debugfs_u8_set, "%llu\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_u8, debugfs_u8_get, debugfs_u8_set, "%llu\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_u8_ro, debugfs_u8_get, NULL, "%llu\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_u8_wo, NULL, debugfs_u8_set, "%llu\n");
/**
* debugfs_create_u8 - create a debugfs file that is used to read and write an unsigned 8-bit value
@@ -99,7 +371,7 @@
struct dentry *debugfs_create_u8(const char *name, umode_t mode,
struct dentry *parent, u8 *value)
{
- return debugfs_create_mode(name, mode, parent, value, &fops_u8,
+ return debugfs_create_mode_unsafe(name, mode, parent, value, &fops_u8,
&fops_u8_ro, &fops_u8_wo);
}
EXPORT_SYMBOL_GPL(debugfs_create_u8);
@@ -114,9 +386,9 @@
*val = *(u16 *)data;
return 0;
}
-DEFINE_SIMPLE_ATTRIBUTE(fops_u16, debugfs_u16_get, debugfs_u16_set, "%llu\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_u16_ro, debugfs_u16_get, NULL, "%llu\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_u16_wo, NULL, debugfs_u16_set, "%llu\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_u16, debugfs_u16_get, debugfs_u16_set, "%llu\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_u16_ro, debugfs_u16_get, NULL, "%llu\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_u16_wo, NULL, debugfs_u16_set, "%llu\n");
/**
* debugfs_create_u16 - create a debugfs file that is used to read and write an unsigned 16-bit value
@@ -145,7 +417,7 @@
struct dentry *debugfs_create_u16(const char *name, umode_t mode,
struct dentry *parent, u16 *value)
{
- return debugfs_create_mode(name, mode, parent, value, &fops_u16,
+ return debugfs_create_mode_unsafe(name, mode, parent, value, &fops_u16,
&fops_u16_ro, &fops_u16_wo);
}
EXPORT_SYMBOL_GPL(debugfs_create_u16);
@@ -160,9 +432,9 @@
*val = *(u32 *)data;
return 0;
}
-DEFINE_SIMPLE_ATTRIBUTE(fops_u32, debugfs_u32_get, debugfs_u32_set, "%llu\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_u32_ro, debugfs_u32_get, NULL, "%llu\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_u32_wo, NULL, debugfs_u32_set, "%llu\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_u32, debugfs_u32_get, debugfs_u32_set, "%llu\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_u32_ro, debugfs_u32_get, NULL, "%llu\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_u32_wo, NULL, debugfs_u32_set, "%llu\n");
/**
* debugfs_create_u32 - create a debugfs file that is used to read and write an unsigned 32-bit value
@@ -191,7 +463,7 @@
struct dentry *debugfs_create_u32(const char *name, umode_t mode,
struct dentry *parent, u32 *value)
{
- return debugfs_create_mode(name, mode, parent, value, &fops_u32,
+ return debugfs_create_mode_unsafe(name, mode, parent, value, &fops_u32,
&fops_u32_ro, &fops_u32_wo);
}
EXPORT_SYMBOL_GPL(debugfs_create_u32);
@@ -207,9 +479,9 @@
*val = *(u64 *)data;
return 0;
}
-DEFINE_SIMPLE_ATTRIBUTE(fops_u64, debugfs_u64_get, debugfs_u64_set, "%llu\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_u64_ro, debugfs_u64_get, NULL, "%llu\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_u64_wo, NULL, debugfs_u64_set, "%llu\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_u64, debugfs_u64_get, debugfs_u64_set, "%llu\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_u64_ro, debugfs_u64_get, NULL, "%llu\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_u64_wo, NULL, debugfs_u64_set, "%llu\n");
/**
* debugfs_create_u64 - create a debugfs file that is used to read and write an unsigned 64-bit value
@@ -238,7 +510,7 @@
struct dentry *debugfs_create_u64(const char *name, umode_t mode,
struct dentry *parent, u64 *value)
{
- return debugfs_create_mode(name, mode, parent, value, &fops_u64,
+ return debugfs_create_mode_unsafe(name, mode, parent, value, &fops_u64,
&fops_u64_ro, &fops_u64_wo);
}
EXPORT_SYMBOL_GPL(debugfs_create_u64);
@@ -254,9 +526,10 @@
*val = *(unsigned long *)data;
return 0;
}
-DEFINE_SIMPLE_ATTRIBUTE(fops_ulong, debugfs_ulong_get, debugfs_ulong_set, "%llu\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_ulong_ro, debugfs_ulong_get, NULL, "%llu\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_ulong_wo, NULL, debugfs_ulong_set, "%llu\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_ulong, debugfs_ulong_get, debugfs_ulong_set,
+ "%llu\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_ulong_ro, debugfs_ulong_get, NULL, "%llu\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_ulong_wo, NULL, debugfs_ulong_set, "%llu\n");
/**
* debugfs_create_ulong - create a debugfs file that is used to read and write
@@ -286,26 +559,30 @@
struct dentry *debugfs_create_ulong(const char *name, umode_t mode,
struct dentry *parent, unsigned long *value)
{
- return debugfs_create_mode(name, mode, parent, value, &fops_ulong,
- &fops_ulong_ro, &fops_ulong_wo);
+ return debugfs_create_mode_unsafe(name, mode, parent, value,
+ &fops_ulong, &fops_ulong_ro,
+ &fops_ulong_wo);
}
EXPORT_SYMBOL_GPL(debugfs_create_ulong);
-DEFINE_SIMPLE_ATTRIBUTE(fops_x8, debugfs_u8_get, debugfs_u8_set, "0x%02llx\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_x8_ro, debugfs_u8_get, NULL, "0x%02llx\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_x8_wo, NULL, debugfs_u8_set, "0x%02llx\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_x8, debugfs_u8_get, debugfs_u8_set, "0x%02llx\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_x8_ro, debugfs_u8_get, NULL, "0x%02llx\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_x8_wo, NULL, debugfs_u8_set, "0x%02llx\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_x16, debugfs_u16_get, debugfs_u16_set, "0x%04llx\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_x16_ro, debugfs_u16_get, NULL, "0x%04llx\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_x16_wo, NULL, debugfs_u16_set, "0x%04llx\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_x16, debugfs_u16_get, debugfs_u16_set,
+ "0x%04llx\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_x16_ro, debugfs_u16_get, NULL, "0x%04llx\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_x16_wo, NULL, debugfs_u16_set, "0x%04llx\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_x32, debugfs_u32_get, debugfs_u32_set, "0x%08llx\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_x32_ro, debugfs_u32_get, NULL, "0x%08llx\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_x32_wo, NULL, debugfs_u32_set, "0x%08llx\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_x32, debugfs_u32_get, debugfs_u32_set,
+ "0x%08llx\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_x32_ro, debugfs_u32_get, NULL, "0x%08llx\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_x32_wo, NULL, debugfs_u32_set, "0x%08llx\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_x64, debugfs_u64_get, debugfs_u64_set, "0x%016llx\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_x64_ro, debugfs_u64_get, NULL, "0x%016llx\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_x64_wo, NULL, debugfs_u64_set, "0x%016llx\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_x64, debugfs_u64_get, debugfs_u64_set,
+ "0x%016llx\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_x64_ro, debugfs_u64_get, NULL, "0x%016llx\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_x64_wo, NULL, debugfs_u64_set, "0x%016llx\n");
/*
* debugfs_create_x{8,16,32,64} - create a debugfs file that is used to read and write an unsigned {8,16,32,64}-bit value
@@ -328,7 +605,7 @@
struct dentry *debugfs_create_x8(const char *name, umode_t mode,
struct dentry *parent, u8 *value)
{
- return debugfs_create_mode(name, mode, parent, value, &fops_x8,
+ return debugfs_create_mode_unsafe(name, mode, parent, value, &fops_x8,
&fops_x8_ro, &fops_x8_wo);
}
EXPORT_SYMBOL_GPL(debugfs_create_x8);
@@ -346,7 +623,7 @@
struct dentry *debugfs_create_x16(const char *name, umode_t mode,
struct dentry *parent, u16 *value)
{
- return debugfs_create_mode(name, mode, parent, value, &fops_x16,
+ return debugfs_create_mode_unsafe(name, mode, parent, value, &fops_x16,
&fops_x16_ro, &fops_x16_wo);
}
EXPORT_SYMBOL_GPL(debugfs_create_x16);
@@ -364,7 +641,7 @@
struct dentry *debugfs_create_x32(const char *name, umode_t mode,
struct dentry *parent, u32 *value)
{
- return debugfs_create_mode(name, mode, parent, value, &fops_x32,
+ return debugfs_create_mode_unsafe(name, mode, parent, value, &fops_x32,
&fops_x32_ro, &fops_x32_wo);
}
EXPORT_SYMBOL_GPL(debugfs_create_x32);
@@ -382,7 +659,7 @@
struct dentry *debugfs_create_x64(const char *name, umode_t mode,
struct dentry *parent, u64 *value)
{
- return debugfs_create_mode(name, mode, parent, value, &fops_x64,
+ return debugfs_create_mode_unsafe(name, mode, parent, value, &fops_x64,
&fops_x64_ro, &fops_x64_wo);
}
EXPORT_SYMBOL_GPL(debugfs_create_x64);
@@ -398,10 +675,10 @@
*val = *(size_t *)data;
return 0;
}
-DEFINE_SIMPLE_ATTRIBUTE(fops_size_t, debugfs_size_t_get, debugfs_size_t_set,
- "%llu\n"); /* %llu and %zu are more or less the same */
-DEFINE_SIMPLE_ATTRIBUTE(fops_size_t_ro, debugfs_size_t_get, NULL, "%llu\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_size_t_wo, NULL, debugfs_size_t_set, "%llu\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_size_t, debugfs_size_t_get, debugfs_size_t_set,
+ "%llu\n"); /* %llu and %zu are more or less the same */
+DEFINE_DEBUGFS_ATTRIBUTE(fops_size_t_ro, debugfs_size_t_get, NULL, "%llu\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_size_t_wo, NULL, debugfs_size_t_set, "%llu\n");
/**
* debugfs_create_size_t - create a debugfs file that is used to read and write an size_t value
@@ -416,8 +693,9 @@
struct dentry *debugfs_create_size_t(const char *name, umode_t mode,
struct dentry *parent, size_t *value)
{
- return debugfs_create_mode(name, mode, parent, value, &fops_size_t,
- &fops_size_t_ro, &fops_size_t_wo);
+ return debugfs_create_mode_unsafe(name, mode, parent, value,
+ &fops_size_t, &fops_size_t_ro,
+ &fops_size_t_wo);
}
EXPORT_SYMBOL_GPL(debugfs_create_size_t);
@@ -431,10 +709,12 @@
*val = atomic_read((atomic_t *)data);
return 0;
}
-DEFINE_SIMPLE_ATTRIBUTE(fops_atomic_t, debugfs_atomic_t_get,
+DEFINE_DEBUGFS_ATTRIBUTE(fops_atomic_t, debugfs_atomic_t_get,
debugfs_atomic_t_set, "%lld\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_atomic_t_ro, debugfs_atomic_t_get, NULL, "%lld\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_atomic_t_wo, NULL, debugfs_atomic_t_set, "%lld\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_atomic_t_ro, debugfs_atomic_t_get, NULL,
+ "%lld\n");
+DEFINE_DEBUGFS_ATTRIBUTE(fops_atomic_t_wo, NULL, debugfs_atomic_t_set,
+ "%lld\n");
/**
* debugfs_create_atomic_t - create a debugfs file that is used to read and
@@ -450,8 +730,9 @@
struct dentry *debugfs_create_atomic_t(const char *name, umode_t mode,
struct dentry *parent, atomic_t *value)
{
- return debugfs_create_mode(name, mode, parent, value, &fops_atomic_t,
- &fops_atomic_t_ro, &fops_atomic_t_wo);
+ return debugfs_create_mode_unsafe(name, mode, parent, value,
+ &fops_atomic_t, &fops_atomic_t_ro,
+ &fops_atomic_t_wo);
}
EXPORT_SYMBOL_GPL(debugfs_create_atomic_t);
@@ -459,9 +740,17 @@
size_t count, loff_t *ppos)
{
char buf[3];
- bool *val = file->private_data;
+ bool val;
+ int r, srcu_idx;
- if (*val)
+ r = debugfs_use_file_start(F_DENTRY(file), &srcu_idx);
+ if (likely(!r))
+ val = *(bool *)file->private_data;
+ debugfs_use_file_finish(srcu_idx);
+ if (r)
+ return r;
+
+ if (val)
buf[0] = 'Y';
else
buf[0] = 'N';
@@ -477,6 +766,7 @@
char buf[32];
size_t buf_size;
bool bv;
+ int r, srcu_idx;
bool *val = file->private_data;
buf_size = min(count, (sizeof(buf)-1));
@@ -484,8 +774,14 @@
return -EFAULT;
buf[buf_size] = '\0';
- if (strtobool(buf, &bv) == 0)
- *val = bv;
+ if (strtobool(buf, &bv) == 0) {
+ r = debugfs_use_file_start(F_DENTRY(file), &srcu_idx);
+ if (likely(!r))
+ *val = bv;
+ debugfs_use_file_finish(srcu_idx);
+ if (r)
+ return r;
+ }
return count;
}
@@ -537,7 +833,7 @@
struct dentry *debugfs_create_bool(const char *name, umode_t mode,
struct dentry *parent, bool *value)
{
- return debugfs_create_mode(name, mode, parent, value, &fops_bool,
+ return debugfs_create_mode_unsafe(name, mode, parent, value, &fops_bool,
&fops_bool_ro, &fops_bool_wo);
}
EXPORT_SYMBOL_GPL(debugfs_create_bool);
@@ -546,8 +842,15 @@
size_t count, loff_t *ppos)
{
struct debugfs_blob_wrapper *blob = file->private_data;
- return simple_read_from_buffer(user_buf, count, ppos, blob->data,
- blob->size);
+ ssize_t r;
+ int srcu_idx;
+
+ r = debugfs_use_file_start(F_DENTRY(file), &srcu_idx);
+ if (likely(!r))
+ r = simple_read_from_buffer(user_buf, count, ppos, blob->data,
+ blob->size);
+ debugfs_use_file_finish(srcu_idx);
+ return r;
}
static const struct file_operations fops_blob = {
@@ -584,7 +887,7 @@
struct dentry *parent,
struct debugfs_blob_wrapper *blob)
{
- return debugfs_create_file(name, mode, parent, blob, &fops_blob);
+ return debugfs_create_file_unsafe(name, mode, parent, blob, &fops_blob);
}
EXPORT_SYMBOL_GPL(debugfs_create_blob);
@@ -689,7 +992,8 @@
data->array = array;
data->elements = elements;
- return debugfs_create_file(name, mode, parent, data, &u32_array_fops);
+ return debugfs_create_file_unsafe(name, mode, parent, data,
+ &u32_array_fops);
}
EXPORT_SYMBOL_GPL(debugfs_create_u32_array);
diff --git a/fs/debugfs/inode.c b/fs/debugfs/inode.c
index 8580831..4bc1f68 100644
--- a/fs/debugfs/inode.c
+++ b/fs/debugfs/inode.c
@@ -27,9 +27,14 @@
#include <linux/parser.h>
#include <linux/magic.h>
#include <linux/slab.h>
+#include <linux/srcu.h>
+
+#include "internal.h"
#define DEBUGFS_DEFAULT_MODE 0700
+DEFINE_SRCU(debugfs_srcu);
+
static struct vfsmount *debugfs_mount;
static int debugfs_mount_count;
static bool debugfs_registered;
@@ -39,7 +44,8 @@
struct inode *inode = new_inode(sb);
if (inode) {
inode->i_ino = get_next_ino();
- inode->i_atime = inode->i_mtime = inode->i_ctime = CURRENT_TIME;
+ inode->i_atime = inode->i_mtime =
+ inode->i_ctime = current_fs_time(sb);
}
return inode;
}
@@ -294,6 +300,37 @@
return dentry;
}
+static struct dentry *__debugfs_create_file(const char *name, umode_t mode,
+ struct dentry *parent, void *data,
+ const struct file_operations *proxy_fops,
+ const struct file_operations *real_fops)
+{
+ struct dentry *dentry;
+ struct inode *inode;
+
+ if (!(mode & S_IFMT))
+ mode |= S_IFREG;
+ BUG_ON(!S_ISREG(mode));
+ dentry = start_creating(name, parent);
+
+ if (IS_ERR(dentry))
+ return NULL;
+
+ inode = debugfs_get_inode(dentry->d_sb);
+ if (unlikely(!inode))
+ return failed_creating(dentry);
+
+ inode->i_mode = mode;
+ inode->i_private = data;
+
+ inode->i_fop = proxy_fops;
+ dentry->d_fsdata = (void *)real_fops;
+
+ d_instantiate(dentry, inode);
+ fsnotify_create(d_inode(dentry->d_parent), dentry);
+ return end_creating(dentry);
+}
+
/**
* debugfs_create_file - create a file in the debugfs filesystem
* @name: a pointer to a string containing the name of the file to create.
@@ -324,31 +361,54 @@
struct dentry *parent, void *data,
const struct file_operations *fops)
{
- struct dentry *dentry;
- struct inode *inode;
- if (!(mode & S_IFMT))
- mode |= S_IFREG;
- BUG_ON(!S_ISREG(mode));
- dentry = start_creating(name, parent);
-
- if (IS_ERR(dentry))
- return NULL;
-
- inode = debugfs_get_inode(dentry->d_sb);
- if (unlikely(!inode))
- return failed_creating(dentry);
-
- inode->i_mode = mode;
- inode->i_fop = fops ? fops : &debugfs_file_operations;
- inode->i_private = data;
- d_instantiate(dentry, inode);
- fsnotify_create(d_inode(dentry->d_parent), dentry);
- return end_creating(dentry);
+ return __debugfs_create_file(name, mode, parent, data,
+ fops ? &debugfs_full_proxy_file_operations :
+ &debugfs_noop_file_operations,
+ fops);
}
EXPORT_SYMBOL_GPL(debugfs_create_file);
/**
+ * debugfs_create_file_unsafe - create a file in the debugfs filesystem
+ * @name: a pointer to a string containing the name of the file to create.
+ * @mode: the permission that the file should have.
+ * @parent: a pointer to the parent dentry for this file. This should be a
+ * directory dentry if set. If this parameter is NULL, then the
+ * file will be created in the root of the debugfs filesystem.
+ * @data: a pointer to something that the caller will want to get to later
+ * on. The inode.i_private pointer will point to this value on
+ * the open() call.
+ * @fops: a pointer to a struct file_operations that should be used for
+ * this file.
+ *
+ * debugfs_create_file_unsafe() is completely analogous to
+ * debugfs_create_file(), the only difference being that the fops
+ * handed it will not get protected against file removals by the
+ * debugfs core.
+ *
+ * It is your responsibility to protect your struct file_operation
+ * methods against file removals by means of debugfs_use_file_start()
+ * and debugfs_use_file_finish(). ->open() is still protected by
+ * debugfs though.
+ *
+ * Any struct file_operations defined by means of
+ * DEFINE_DEBUGFS_ATTRIBUTE() is protected against file removals and
+ * thus, may be used here.
+ */
+struct dentry *debugfs_create_file_unsafe(const char *name, umode_t mode,
+ struct dentry *parent, void *data,
+ const struct file_operations *fops)
+{
+
+ return __debugfs_create_file(name, mode, parent, data,
+ fops ? &debugfs_open_proxy_file_operations :
+ &debugfs_noop_file_operations,
+ fops);
+}
+EXPORT_SYMBOL_GPL(debugfs_create_file_unsafe);
+
+/**
* debugfs_create_file_size - create a file in the debugfs filesystem
* @name: a pointer to a string containing the name of the file to create.
* @mode: the permission that the file should have.
@@ -461,7 +521,11 @@
inode->i_flags |= S_AUTOMOUNT;
inode->i_private = data;
dentry->d_fsdata = (void *)f;
+ /* directory inodes start off with i_nlink == 2 (for "." entry) */
+ inc_nlink(inode);
d_instantiate(dentry, inode);
+ inc_nlink(d_inode(dentry->d_parent));
+ fsnotify_mkdir(d_inode(dentry->d_parent), dentry);
return end_creating(dentry);
}
EXPORT_SYMBOL(debugfs_create_automount);
@@ -565,6 +629,8 @@
inode_unlock(d_inode(parent));
if (!ret)
simple_release_fs(&debugfs_mount, &debugfs_mount_count);
+
+ synchronize_srcu(&debugfs_srcu);
}
EXPORT_SYMBOL_GPL(debugfs_remove);
@@ -642,6 +708,8 @@
if (!__debugfs_remove(child, parent))
simple_release_fs(&debugfs_mount, &debugfs_mount_count);
inode_unlock(d_inode(parent));
+
+ synchronize_srcu(&debugfs_srcu);
}
EXPORT_SYMBOL_GPL(debugfs_remove_recursive);
diff --git a/fs/debugfs/internal.h b/fs/debugfs/internal.h
new file mode 100644
index 0000000..bba5263
--- /dev/null
+++ b/fs/debugfs/internal.h
@@ -0,0 +1,26 @@
+/*
+ * internal.h - declarations internal to debugfs
+ *
+ * Copyright (C) 2016 Nicolai Stange <nicstange@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ */
+
+#ifndef _DEBUGFS_INTERNAL_H_
+#define _DEBUGFS_INTERNAL_H_
+
+struct file_operations;
+
+/* declared over in file.c */
+extern const struct file_operations debugfs_noop_file_operations;
+extern const struct file_operations debugfs_open_proxy_file_operations;
+extern const struct file_operations debugfs_full_proxy_file_operations;
+
+struct dentry *debugfs_create_file_unsafe(const char *name, umode_t mode,
+ struct dentry *parent, void *data,
+ const struct file_operations *fops);
+
+#endif /* _DEBUGFS_INTERNAL_H_ */
diff --git a/fs/efivarfs/inode.c b/fs/efivarfs/inode.c
index e2ab6d0..1d73fc6 100644
--- a/fs/efivarfs/inode.c
+++ b/fs/efivarfs/inode.c
@@ -11,6 +11,7 @@
#include <linux/fs.h>
#include <linux/ctype.h>
#include <linux/slab.h>
+#include <linux/uuid.h>
#include "internal.h"
@@ -46,11 +47,7 @@
*/
bool efivarfs_valid_name(const char *str, int len)
{
- static const char dashes[EFI_VARIABLE_GUID_LEN] = {
- [8] = 1, [13] = 1, [18] = 1, [23] = 1
- };
const char *s = str + len - EFI_VARIABLE_GUID_LEN;
- int i;
/*
* We need a GUID, plus at least one letter for the variable name,
@@ -68,37 +65,7 @@
*
* 12345678-1234-1234-1234-123456789abc
*/
- for (i = 0; i < EFI_VARIABLE_GUID_LEN; i++) {
- if (dashes[i]) {
- if (*s++ != '-')
- return false;
- } else {
- if (!isxdigit(*s++))
- return false;
- }
- }
-
- return true;
-}
-
-static void efivarfs_hex_to_guid(const char *str, efi_guid_t *guid)
-{
- guid->b[0] = hex_to_bin(str[6]) << 4 | hex_to_bin(str[7]);
- guid->b[1] = hex_to_bin(str[4]) << 4 | hex_to_bin(str[5]);
- guid->b[2] = hex_to_bin(str[2]) << 4 | hex_to_bin(str[3]);
- guid->b[3] = hex_to_bin(str[0]) << 4 | hex_to_bin(str[1]);
- guid->b[4] = hex_to_bin(str[11]) << 4 | hex_to_bin(str[12]);
- guid->b[5] = hex_to_bin(str[9]) << 4 | hex_to_bin(str[10]);
- guid->b[6] = hex_to_bin(str[16]) << 4 | hex_to_bin(str[17]);
- guid->b[7] = hex_to_bin(str[14]) << 4 | hex_to_bin(str[15]);
- guid->b[8] = hex_to_bin(str[19]) << 4 | hex_to_bin(str[20]);
- guid->b[9] = hex_to_bin(str[21]) << 4 | hex_to_bin(str[22]);
- guid->b[10] = hex_to_bin(str[24]) << 4 | hex_to_bin(str[25]);
- guid->b[11] = hex_to_bin(str[26]) << 4 | hex_to_bin(str[27]);
- guid->b[12] = hex_to_bin(str[28]) << 4 | hex_to_bin(str[29]);
- guid->b[13] = hex_to_bin(str[30]) << 4 | hex_to_bin(str[31]);
- guid->b[14] = hex_to_bin(str[32]) << 4 | hex_to_bin(str[33]);
- guid->b[15] = hex_to_bin(str[34]) << 4 | hex_to_bin(str[35]);
+ return uuid_is_valid(s);
}
static int efivarfs_create(struct inode *dir, struct dentry *dentry,
@@ -119,8 +86,7 @@
/* length of the variable name itself: remove GUID and separator */
namelen = dentry->d_name.len - EFI_VARIABLE_GUID_LEN - 1;
- efivarfs_hex_to_guid(dentry->d_name.name + namelen + 1,
- &var->var.VendorGuid);
+ uuid_le_to_bin(dentry->d_name.name + namelen + 1, &var->var.VendorGuid);
if (efivar_variable_is_removable(var->var.VendorGuid,
dentry->d_name.name, namelen))
diff --git a/fs/efs/super.c b/fs/efs/super.c
index cb68dac..368f7dd 100644
--- a/fs/efs/super.c
+++ b/fs/efs/super.c
@@ -275,7 +275,7 @@
if (!bh) {
pr_err("cannot read volume header\n");
- return -EINVAL;
+ return -EIO;
}
/*
@@ -293,7 +293,7 @@
bh = sb_bread(s, sb->fs_start + EFS_SUPER);
if (!bh) {
pr_err("cannot read superblock\n");
- return -EINVAL;
+ return -EIO;
}
if (efs_validate_super(sb, (struct efs_super *) bh->b_data)) {
diff --git a/fs/eventpoll.c b/fs/eventpoll.c
index 8a74a2a..10db912 100644
--- a/fs/eventpoll.c
+++ b/fs/eventpoll.c
@@ -1583,15 +1583,15 @@
return ep_scan_ready_list(ep, ep_send_events_proc, &esed, 0, false);
}
-static inline struct timespec ep_set_mstimeout(long ms)
+static inline struct timespec64 ep_set_mstimeout(long ms)
{
- struct timespec now, ts = {
+ struct timespec64 now, ts = {
.tv_sec = ms / MSEC_PER_SEC,
.tv_nsec = NSEC_PER_MSEC * (ms % MSEC_PER_SEC),
};
- ktime_get_ts(&now);
- return timespec_add_safe(now, ts);
+ ktime_get_ts64(&now);
+ return timespec64_add_safe(now, ts);
}
/**
@@ -1621,11 +1621,11 @@
ktime_t expires, *to = NULL;
if (timeout > 0) {
- struct timespec end_time = ep_set_mstimeout(timeout);
+ struct timespec64 end_time = ep_set_mstimeout(timeout);
slack = select_estimate_accuracy(&end_time);
to = &expires;
- *to = timespec_to_ktime(end_time);
+ *to = timespec64_to_ktime(end_time);
} else if (timeout == 0) {
/*
* Avoid the unnecessary trip to the wait queue loop, if the
diff --git a/fs/exec.c b/fs/exec.c
index a98b21d4..887c1c9 100644
--- a/fs/exec.c
+++ b/fs/exec.c
@@ -243,10 +243,6 @@
put_page(page);
}
-static void free_arg_page(struct linux_binprm *bprm, int i)
-{
-}
-
static void free_arg_pages(struct linux_binprm *bprm)
{
}
@@ -267,7 +263,10 @@
if (!vma)
return -ENOMEM;
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem)) {
+ err = -EINTR;
+ goto err_free;
+ }
vma->vm_mm = mm;
/*
@@ -294,6 +293,7 @@
return 0;
err:
up_write(&mm->mmap_sem);
+err_free:
bprm->vma = NULL;
kmem_cache_free(vm_area_cachep, vma);
return err;
@@ -700,7 +700,9 @@
bprm->loader -= stack_shift;
bprm->exec -= stack_shift;
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
+
vm_flags = VM_STACK_FLAGS;
/*
@@ -850,15 +852,25 @@
if (ret)
return ret;
+ ret = deny_write_access(file);
+ if (ret)
+ return ret;
+
i_size = i_size_read(file_inode(file));
- if (max_size > 0 && i_size > max_size)
- return -EFBIG;
- if (i_size <= 0)
- return -EINVAL;
+ if (max_size > 0 && i_size > max_size) {
+ ret = -EFBIG;
+ goto out;
+ }
+ if (i_size <= 0) {
+ ret = -EINVAL;
+ goto out;
+ }
*buf = vmalloc(i_size);
- if (!*buf)
- return -ENOMEM;
+ if (!*buf) {
+ ret = -ENOMEM;
+ goto out;
+ }
pos = 0;
while (pos < i_size) {
@@ -876,18 +888,21 @@
if (pos != i_size) {
ret = -EIO;
- goto out;
+ goto out_free;
}
ret = security_kernel_post_read_file(file, *buf, i_size, id);
if (!ret)
*size = pos;
-out:
+out_free:
if (ret < 0) {
vfree(*buf);
*buf = NULL;
}
+
+out:
+ allow_write_access(file);
return ret;
}
EXPORT_SYMBOL_GPL(kernel_read_file);
@@ -1486,9 +1501,6 @@
kunmap_atomic(kaddr);
put_arg_page(page);
-
- if (offset == PAGE_SIZE)
- free_arg_page(bprm, (bprm->p >> PAGE_SHIFT) - 1);
} while (offset == PAGE_SIZE);
bprm->p++;
diff --git a/fs/ext4/balloc.c b/fs/ext4/balloc.c
index fe1f50f..3020fd7 100644
--- a/fs/ext4/balloc.c
+++ b/fs/ext4/balloc.c
@@ -610,7 +610,8 @@
jbd_debug(1, "%s: retrying operation after ENOSPC\n", sb->s_id);
- return jbd2_journal_force_commit_nested(EXT4_SB(sb)->s_journal);
+ jbd2_journal_force_commit_nested(EXT4_SB(sb)->s_journal);
+ return 1;
}
/*
diff --git a/fs/ext4/dir.c b/fs/ext4/dir.c
index 5d00bf0..68323e3 100644
--- a/fs/ext4/dir.c
+++ b/fs/ext4/dir.c
@@ -150,6 +150,11 @@
while (ctx->pos < inode->i_size) {
struct ext4_map_blocks map;
+ if (fatal_signal_pending(current)) {
+ err = -ERESTARTSYS;
+ goto errout;
+ }
+ cond_resched();
map.m_lblk = ctx->pos >> EXT4_BLOCK_SIZE_BITS(sb);
map.m_len = 1;
err = ext4_map_blocks(NULL, inode, &map, 0);
diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
index 72f4c9e..b84aa1c 100644
--- a/fs/ext4/ext4.h
+++ b/fs/ext4/ext4.h
@@ -33,6 +33,7 @@
#include <linux/ratelimit.h>
#include <crypto/hash.h>
#include <linux/falloc.h>
+#include <linux/percpu-rwsem.h>
#ifdef __KERNEL__
#include <linux/compat.h>
#endif
@@ -581,6 +582,9 @@
#define EXT4_GET_BLOCKS_ZERO 0x0200
#define EXT4_GET_BLOCKS_CREATE_ZERO (EXT4_GET_BLOCKS_CREATE |\
EXT4_GET_BLOCKS_ZERO)
+ /* Caller will submit data before dropping transaction handle. This
+ * allows jbd2 to avoid submitting data before commit. */
+#define EXT4_GET_BLOCKS_IO_SUBMIT 0x0400
/*
* The bit position of these flags must not overlap with any of the
@@ -1505,6 +1509,9 @@
struct ratelimit_state s_err_ratelimit_state;
struct ratelimit_state s_warning_ratelimit_state;
struct ratelimit_state s_msg_ratelimit_state;
+
+ /* Barrier between changing inodes' journal flags and writepages ops. */
+ struct percpu_rw_semaphore s_journal_flag_rwsem;
};
static inline struct ext4_sb_info *EXT4_SB(struct super_block *sb)
@@ -1549,7 +1556,6 @@
EXT4_STATE_DIOREAD_LOCK, /* Disable support for dio read
nolocking */
EXT4_STATE_MAY_INLINE_DATA, /* may have in-inode data */
- EXT4_STATE_ORDERED_MODE, /* data=ordered mode */
EXT4_STATE_EXT_PRECACHED, /* extents have been precached */
};
@@ -2521,8 +2527,8 @@
struct buffer_head *ext4_bread(handle_t *, struct inode *, ext4_lblk_t, int);
int ext4_get_block_unwritten(struct inode *inode, sector_t iblock,
struct buffer_head *bh_result, int create);
-int ext4_dax_mmap_get_block(struct inode *inode, sector_t iblock,
- struct buffer_head *bh_result, int create);
+int ext4_dax_get_block(struct inode *inode, sector_t iblock,
+ struct buffer_head *bh_result, int create);
int ext4_get_block(struct inode *inode, sector_t iblock,
struct buffer_head *bh_result, int create);
int ext4_dio_get_block(struct inode *inode, sector_t iblock,
@@ -2581,7 +2587,6 @@
/* indirect.c */
extern int ext4_ind_map_blocks(handle_t *handle, struct inode *inode,
struct ext4_map_blocks *map, int flags);
-extern ssize_t ext4_ind_direct_IO(struct kiocb *iocb, struct iov_iter *iter);
extern int ext4_ind_calc_metadata_amount(struct inode *inode, sector_t lblock);
extern int ext4_ind_trans_blocks(struct inode *inode, int nrblocks);
extern void ext4_ind_truncate(handle_t *, struct inode *inode);
@@ -3329,6 +3334,13 @@
}
}
+static inline bool ext4_aligned_io(struct inode *inode, loff_t off, loff_t len)
+{
+ int blksize = 1 << inode->i_blkbits;
+
+ return IS_ALIGNED(off, blksize) && IS_ALIGNED(len, blksize);
+}
+
#endif /* __KERNEL__ */
#define EFSBADCRC EBADMSG /* Bad CRC detected */
diff --git a/fs/ext4/ext4_jbd2.h b/fs/ext4/ext4_jbd2.h
index 5f58462..09c1ef3 100644
--- a/fs/ext4/ext4_jbd2.h
+++ b/fs/ext4/ext4_jbd2.h
@@ -359,10 +359,21 @@
return 0;
}
-static inline int ext4_jbd2_file_inode(handle_t *handle, struct inode *inode)
+static inline int ext4_jbd2_inode_add_write(handle_t *handle,
+ struct inode *inode)
{
if (ext4_handle_valid(handle))
- return jbd2_journal_file_inode(handle, EXT4_I(inode)->jinode);
+ return jbd2_journal_inode_add_write(handle,
+ EXT4_I(inode)->jinode);
+ return 0;
+}
+
+static inline int ext4_jbd2_inode_add_wait(handle_t *handle,
+ struct inode *inode)
+{
+ if (ext4_handle_valid(handle))
+ return jbd2_journal_inode_add_wait(handle,
+ EXT4_I(inode)->jinode);
return 0;
}
diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index 95bf467..2a2eef9 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -120,9 +120,14 @@
if (!ext4_handle_valid(handle))
return 0;
- if (handle->h_buffer_credits > needed)
+ if (handle->h_buffer_credits >= needed)
return 0;
- err = ext4_journal_extend(handle, needed);
+ /*
+ * If we need to extend the journal get a few extra blocks
+ * while we're at it for efficiency's sake.
+ */
+ needed += 3;
+ err = ext4_journal_extend(handle, needed - handle->h_buffer_credits);
if (err <= 0)
return err;
err = ext4_truncate_restart_trans(handle, inode, needed);
@@ -907,13 +912,6 @@
eh = ext_block_hdr(bh);
ppos++;
- if (unlikely(ppos > depth)) {
- put_bh(bh);
- EXT4_ERROR_INODE(inode,
- "ppos %d > depth %d", ppos, depth);
- ret = -EFSCORRUPTED;
- goto err;
- }
path[ppos].p_bh = bh;
path[ppos].p_hdr = eh;
}
@@ -2583,7 +2581,7 @@
}
} else
ext4_error(sbi->s_sb, "strange request: removal(2) "
- "%u-%u from %u:%u\n",
+ "%u-%u from %u:%u",
from, to, le32_to_cpu(ex->ee_block), ee_len);
return 0;
}
@@ -3738,7 +3736,7 @@
if (ee_block != map->m_lblk || ee_len > map->m_len) {
#ifdef EXT4_DEBUG
ext4_warning("Inode (%ld) finished: extent logical block %llu,"
- " len %u; IO logical block %llu, len %u\n",
+ " len %u; IO logical block %llu, len %u",
inode->i_ino, (unsigned long long)ee_block, ee_len,
(unsigned long long)map->m_lblk, map->m_len);
#endif
diff --git a/fs/ext4/extents_status.c b/fs/ext4/extents_status.c
index e38b987a..37e0592 100644
--- a/fs/ext4/extents_status.c
+++ b/fs/ext4/extents_status.c
@@ -707,7 +707,7 @@
(status & EXTENT_STATUS_WRITTEN)) {
ext4_warning(inode->i_sb, "Inserting extent [%u/%u] as "
" delayed and written which can potentially "
- " cause data loss.\n", lblk, len);
+ " cause data loss.", lblk, len);
WARN_ON(1);
}
diff --git a/fs/ext4/file.c b/fs/ext4/file.c
index 00ff691..d478110 100644
--- a/fs/ext4/file.c
+++ b/fs/ext4/file.c
@@ -202,7 +202,7 @@
if (IS_ERR(handle))
result = VM_FAULT_SIGBUS;
else
- result = __dax_fault(vma, vmf, ext4_dax_mmap_get_block, NULL);
+ result = __dax_fault(vma, vmf, ext4_dax_get_block, NULL);
if (write) {
if (!IS_ERR(handle))
@@ -238,7 +238,7 @@
result = VM_FAULT_SIGBUS;
else
result = __dax_pmd_fault(vma, addr, pmd, flags,
- ext4_dax_mmap_get_block, NULL);
+ ext4_dax_get_block, NULL);
if (write) {
if (!IS_ERR(handle))
@@ -373,7 +373,7 @@
if (ext4_encrypted_inode(d_inode(dir)) &&
!ext4_is_child_context_consistent_with_parent(d_inode(dir), inode)) {
ext4_warning(inode->i_sb,
- "Inconsistent encryption contexts: %lu/%lu\n",
+ "Inconsistent encryption contexts: %lu/%lu",
(unsigned long) d_inode(dir)->i_ino,
(unsigned long) inode->i_ino);
dput(dir);
diff --git a/fs/ext4/ialloc.c b/fs/ext4/ialloc.c
index 237b877..3da4cf8 100644
--- a/fs/ext4/ialloc.c
+++ b/fs/ext4/ialloc.c
@@ -1150,25 +1150,20 @@
unsigned long max_ino = le32_to_cpu(EXT4_SB(sb)->s_es->s_inodes_count);
ext4_group_t block_group;
int bit;
- struct buffer_head *bitmap_bh;
+ struct buffer_head *bitmap_bh = NULL;
struct inode *inode = NULL;
- long err = -EIO;
+ int err = -EFSCORRUPTED;
- /* Error cases - e2fsck has already cleaned up for us */
- if (ino > max_ino) {
- ext4_warning(sb, "bad orphan ino %lu! e2fsck was run?", ino);
- err = -EFSCORRUPTED;
- goto error;
- }
+ if (ino < EXT4_FIRST_INO(sb) || ino > max_ino)
+ goto bad_orphan;
block_group = (ino - 1) / EXT4_INODES_PER_GROUP(sb);
bit = (ino - 1) % EXT4_INODES_PER_GROUP(sb);
bitmap_bh = ext4_read_inode_bitmap(sb, block_group);
if (IS_ERR(bitmap_bh)) {
- err = PTR_ERR(bitmap_bh);
- ext4_warning(sb, "inode bitmap error %ld for orphan %lu",
- ino, err);
- goto error;
+ ext4_error(sb, "inode bitmap error %ld for orphan %lu",
+ ino, PTR_ERR(bitmap_bh));
+ return (struct inode *) bitmap_bh;
}
/* Having the inode bit set should be a 100% indicator that this
@@ -1179,15 +1174,21 @@
goto bad_orphan;
inode = ext4_iget(sb, ino);
- if (IS_ERR(inode))
- goto iget_failed;
+ if (IS_ERR(inode)) {
+ err = PTR_ERR(inode);
+ ext4_error(sb, "couldn't read orphan inode %lu (err %d)",
+ ino, err);
+ return inode;
+ }
/*
- * If the orphans has i_nlinks > 0 then it should be able to be
- * truncated, otherwise it won't be removed from the orphan list
- * during processing and an infinite loop will result.
+ * If the orphans has i_nlinks > 0 then it should be able to
+ * be truncated, otherwise it won't be removed from the orphan
+ * list during processing and an infinite loop will result.
+ * Similarly, it must not be a bad inode.
*/
- if (inode->i_nlink && !ext4_can_truncate(inode))
+ if ((inode->i_nlink && !ext4_can_truncate(inode)) ||
+ is_bad_inode(inode))
goto bad_orphan;
if (NEXT_ORPHAN(inode) > max_ino)
@@ -1195,29 +1196,25 @@
brelse(bitmap_bh);
return inode;
-iget_failed:
- err = PTR_ERR(inode);
- inode = NULL;
bad_orphan:
- ext4_warning(sb, "bad orphan inode %lu! e2fsck was run?", ino);
- printk(KERN_WARNING "ext4_test_bit(bit=%d, block=%llu) = %d\n",
- bit, (unsigned long long)bitmap_bh->b_blocknr,
- ext4_test_bit(bit, bitmap_bh->b_data));
- printk(KERN_WARNING "inode=%p\n", inode);
+ ext4_error(sb, "bad orphan inode %lu", ino);
+ if (bitmap_bh)
+ printk(KERN_ERR "ext4_test_bit(bit=%d, block=%llu) = %d\n",
+ bit, (unsigned long long)bitmap_bh->b_blocknr,
+ ext4_test_bit(bit, bitmap_bh->b_data));
if (inode) {
- printk(KERN_WARNING "is_bad_inode(inode)=%d\n",
+ printk(KERN_ERR "is_bad_inode(inode)=%d\n",
is_bad_inode(inode));
- printk(KERN_WARNING "NEXT_ORPHAN(inode)=%u\n",
+ printk(KERN_ERR "NEXT_ORPHAN(inode)=%u\n",
NEXT_ORPHAN(inode));
- printk(KERN_WARNING "max_ino=%lu\n", max_ino);
- printk(KERN_WARNING "i_nlink=%u\n", inode->i_nlink);
+ printk(KERN_ERR "max_ino=%lu\n", max_ino);
+ printk(KERN_ERR "i_nlink=%u\n", inode->i_nlink);
/* Avoid freeing blocks if we got a bad deleted inode */
if (inode->i_nlink == 0)
inode->i_blocks = 0;
iput(inode);
}
brelse(bitmap_bh);
-error:
return ERR_PTR(err);
}
diff --git a/fs/ext4/indirect.c b/fs/ext4/indirect.c
index 627b7e8..bc15c2c 100644
--- a/fs/ext4/indirect.c
+++ b/fs/ext4/indirect.c
@@ -649,133 +649,6 @@
}
/*
- * O_DIRECT for ext3 (or indirect map) based files
- *
- * If the O_DIRECT write will extend the file then add this inode to the
- * orphan list. So recovery will truncate it back to the original size
- * if the machine crashes during the write.
- *
- * If the O_DIRECT write is intantiating holes inside i_size and the machine
- * crashes then stale disk data _may_ be exposed inside the file. But current
- * VFS code falls back into buffered path in that case so we are safe.
- */
-ssize_t ext4_ind_direct_IO(struct kiocb *iocb, struct iov_iter *iter)
-{
- struct file *file = iocb->ki_filp;
- struct inode *inode = file->f_mapping->host;
- struct ext4_inode_info *ei = EXT4_I(inode);
- loff_t offset = iocb->ki_pos;
- handle_t *handle;
- ssize_t ret;
- int orphan = 0;
- size_t count = iov_iter_count(iter);
- int retries = 0;
-
- if (iov_iter_rw(iter) == WRITE) {
- loff_t final_size = offset + count;
-
- if (final_size > inode->i_size) {
- /* Credits for sb + inode write */
- handle = ext4_journal_start(inode, EXT4_HT_INODE, 2);
- if (IS_ERR(handle)) {
- ret = PTR_ERR(handle);
- goto out;
- }
- ret = ext4_orphan_add(handle, inode);
- if (ret) {
- ext4_journal_stop(handle);
- goto out;
- }
- orphan = 1;
- ei->i_disksize = inode->i_size;
- ext4_journal_stop(handle);
- }
- }
-
-retry:
- if (iov_iter_rw(iter) == READ && ext4_should_dioread_nolock(inode)) {
- /*
- * Nolock dioread optimization may be dynamically disabled
- * via ext4_inode_block_unlocked_dio(). Check inode's state
- * while holding extra i_dio_count ref.
- */
- inode_dio_begin(inode);
- smp_mb();
- if (unlikely(ext4_test_inode_state(inode,
- EXT4_STATE_DIOREAD_LOCK))) {
- inode_dio_end(inode);
- goto locked;
- }
- if (IS_DAX(inode))
- ret = dax_do_io(iocb, inode, iter,
- ext4_dio_get_block, NULL, 0);
- else
- ret = __blockdev_direct_IO(iocb, inode,
- inode->i_sb->s_bdev, iter,
- ext4_dio_get_block,
- NULL, NULL, 0);
- inode_dio_end(inode);
- } else {
-locked:
- if (IS_DAX(inode))
- ret = dax_do_io(iocb, inode, iter,
- ext4_dio_get_block, NULL, DIO_LOCKING);
- else
- ret = blockdev_direct_IO(iocb, inode, iter,
- ext4_dio_get_block);
-
- if (unlikely(iov_iter_rw(iter) == WRITE && ret < 0)) {
- loff_t isize = i_size_read(inode);
- loff_t end = offset + count;
-
- if (end > isize)
- ext4_truncate_failed_write(inode);
- }
- }
- if (ret == -ENOSPC && ext4_should_retry_alloc(inode->i_sb, &retries))
- goto retry;
-
- if (orphan) {
- int err;
-
- /* Credits for sb + inode write */
- handle = ext4_journal_start(inode, EXT4_HT_INODE, 2);
- if (IS_ERR(handle)) {
- /* This is really bad luck. We've written the data
- * but cannot extend i_size. Bail out and pretend
- * the write failed... */
- ret = PTR_ERR(handle);
- if (inode->i_nlink)
- ext4_orphan_del(NULL, inode);
-
- goto out;
- }
- if (inode->i_nlink)
- ext4_orphan_del(handle, inode);
- if (ret > 0) {
- loff_t end = offset + ret;
- if (end > inode->i_size) {
- ei->i_disksize = end;
- i_size_write(inode, end);
- /*
- * We're going to return a positive `ret'
- * here due to non-zero-length I/O, so there's
- * no way of reporting error returns from
- * ext4_mark_inode_dirty() to userspace. So
- * ignore it.
- */
- ext4_mark_inode_dirty(handle, inode);
- }
- }
- err = ext4_journal_stop(handle);
- if (ret == 0)
- ret = err;
- }
-out:
- return ret;
-}
-
-/*
* Calculate the number of metadata blocks need to reserve
* to allocate a new block at @lblocks for non extent file based file
*/
diff --git a/fs/ext4/inline.c b/fs/ext4/inline.c
index 7bc6c85..ff7538c 100644
--- a/fs/ext4/inline.c
+++ b/fs/ext4/inline.c
@@ -1780,7 +1780,7 @@
ext4_warning(dir->i_sb,
"bad inline directory (dir #%lu) - "
"inode %u, rec_len %u, name_len %d"
- "inline size %d\n",
+ "inline size %d",
dir->i_ino, le32_to_cpu(de->inode),
le16_to_cpu(de->rec_len), de->name_len,
inline_size);
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 79b298d..f7140ca6 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -684,6 +684,24 @@
ret = check_block_validity(inode, map);
if (ret != 0)
return ret;
+
+ /*
+ * Inodes with freshly allocated blocks where contents will be
+ * visible after transaction commit must be on transaction's
+ * ordered data list.
+ */
+ if (map->m_flags & EXT4_MAP_NEW &&
+ !(map->m_flags & EXT4_MAP_UNWRITTEN) &&
+ !(flags & EXT4_GET_BLOCKS_ZERO) &&
+ !IS_NOQUOTA(inode) &&
+ ext4_should_order_data(inode)) {
+ if (flags & EXT4_GET_BLOCKS_IO_SUBMIT)
+ ret = ext4_jbd2_inode_add_wait(handle, inode);
+ else
+ ret = ext4_jbd2_inode_add_write(handle, inode);
+ if (ret)
+ return ret;
+ }
}
return retval;
}
@@ -1289,15 +1307,6 @@
int i_size_changed = 0;
trace_ext4_write_end(inode, pos, len, copied);
- if (ext4_test_inode_state(inode, EXT4_STATE_ORDERED_MODE)) {
- ret = ext4_jbd2_file_inode(handle, inode);
- if (ret) {
- unlock_page(page);
- put_page(page);
- goto errout;
- }
- }
-
if (ext4_has_inline_data(inode)) {
ret = ext4_write_inline_data_end(inode, pos, len,
copied, page);
@@ -2313,7 +2322,8 @@
* the data was copied into the page cache.
*/
get_blocks_flags = EXT4_GET_BLOCKS_CREATE |
- EXT4_GET_BLOCKS_METADATA_NOFAIL;
+ EXT4_GET_BLOCKS_METADATA_NOFAIL |
+ EXT4_GET_BLOCKS_IO_SUBMIT;
dioread_nolock = ext4_should_dioread_nolock(inode);
if (dioread_nolock)
get_blocks_flags |= EXT4_GET_BLOCKS_IO_CREATE_EXT;
@@ -2602,11 +2612,14 @@
struct blk_plug plug;
bool give_up_on_write = false;
+ percpu_down_read(&sbi->s_journal_flag_rwsem);
trace_ext4_writepages(inode, wbc);
- if (dax_mapping(mapping))
- return dax_writeback_mapping_range(mapping, inode->i_sb->s_bdev,
- wbc);
+ if (dax_mapping(mapping)) {
+ ret = dax_writeback_mapping_range(mapping, inode->i_sb->s_bdev,
+ wbc);
+ goto out_writepages;
+ }
/*
* No pages to write? This is mainly a kludge to avoid starting
@@ -2776,6 +2789,7 @@
out_writepages:
trace_ext4_writepages_result(inode, wbc, ret,
nr_to_write - wbc->nr_to_write);
+ percpu_up_read(&sbi->s_journal_flag_rwsem);
return ret;
}
@@ -3215,75 +3229,52 @@
}
#ifdef CONFIG_FS_DAX
-int ext4_dax_mmap_get_block(struct inode *inode, sector_t iblock,
- struct buffer_head *bh_result, int create)
+/*
+ * Get block function for DAX IO and mmap faults. It takes care of converting
+ * unwritten extents to written ones and initializes new / converted blocks
+ * to zeros.
+ */
+int ext4_dax_get_block(struct inode *inode, sector_t iblock,
+ struct buffer_head *bh_result, int create)
{
- int ret, err;
- int credits;
- struct ext4_map_blocks map;
- handle_t *handle = NULL;
- int flags = 0;
+ int ret;
- ext4_debug("ext4_dax_mmap_get_block: inode %lu, create flag %d\n",
- inode->i_ino, create);
- map.m_lblk = iblock;
- map.m_len = bh_result->b_size >> inode->i_blkbits;
- credits = ext4_chunk_trans_blocks(inode, map.m_len);
- if (create) {
- flags |= EXT4_GET_BLOCKS_PRE_IO | EXT4_GET_BLOCKS_CREATE_ZERO;
- handle = ext4_journal_start(inode, EXT4_HT_MAP_BLOCKS, credits);
- if (IS_ERR(handle)) {
- ret = PTR_ERR(handle);
+ ext4_debug("inode %lu, create flag %d\n", inode->i_ino, create);
+ if (!create)
+ return _ext4_get_block(inode, iblock, bh_result, 0);
+
+ ret = ext4_get_block_trans(inode, iblock, bh_result,
+ EXT4_GET_BLOCKS_PRE_IO |
+ EXT4_GET_BLOCKS_CREATE_ZERO);
+ if (ret < 0)
+ return ret;
+
+ if (buffer_unwritten(bh_result)) {
+ /*
+ * We are protected by i_mmap_sem or i_mutex so we know block
+ * cannot go away from under us even though we dropped
+ * i_data_sem. Convert extent to written and write zeros there.
+ */
+ ret = ext4_get_block_trans(inode, iblock, bh_result,
+ EXT4_GET_BLOCKS_CONVERT |
+ EXT4_GET_BLOCKS_CREATE_ZERO);
+ if (ret < 0)
return ret;
- }
}
-
- ret = ext4_map_blocks(handle, inode, &map, flags);
- if (create) {
- err = ext4_journal_stop(handle);
- if (ret >= 0 && err < 0)
- ret = err;
- }
- if (ret <= 0)
- goto out;
- if (map.m_flags & EXT4_MAP_UNWRITTEN) {
- int err2;
-
- /*
- * We are protected by i_mmap_sem so we know block cannot go
- * away from under us even though we dropped i_data_sem.
- * Convert extent to written and write zeros there.
- *
- * Note: We may get here even when create == 0.
- */
- handle = ext4_journal_start(inode, EXT4_HT_MAP_BLOCKS, credits);
- if (IS_ERR(handle)) {
- ret = PTR_ERR(handle);
- goto out;
- }
-
- err = ext4_map_blocks(handle, inode, &map,
- EXT4_GET_BLOCKS_CONVERT | EXT4_GET_BLOCKS_CREATE_ZERO);
- if (err < 0)
- ret = err;
- err2 = ext4_journal_stop(handle);
- if (err2 < 0 && ret > 0)
- ret = err2;
- }
-out:
- WARN_ON_ONCE(ret == 0 && create);
- if (ret > 0) {
- map_bh(bh_result, inode->i_sb, map.m_pblk);
- /*
- * At least for now we have to clear BH_New so that DAX code
- * doesn't attempt to zero blocks again in a racy way.
- */
- map.m_flags &= ~EXT4_MAP_NEW;
- ext4_update_bh_state(bh_result, map.m_flags);
- bh_result->b_size = map.m_len << inode->i_blkbits;
- ret = 0;
- }
- return ret;
+ /*
+ * At least for now we have to clear BH_New so that DAX code
+ * doesn't attempt to zero blocks again in a racy way.
+ */
+ clear_buffer_new(bh_result);
+ return 0;
+}
+#else
+/* Just define empty function, it will never get called. */
+int ext4_dax_get_block(struct inode *inode, sector_t iblock,
+ struct buffer_head *bh_result, int create)
+{
+ BUG();
+ return 0;
}
#endif
@@ -3316,7 +3307,9 @@
}
/*
- * For ext4 extent files, ext4 will do direct-io write to holes,
+ * Handling of direct IO writes.
+ *
+ * For ext4 extent files, ext4 will do direct-io write even to holes,
* preallocated extents, and those write extend the file, no need to
* fall back to buffered IO.
*
@@ -3334,10 +3327,11 @@
* if the machine crashes during the write.
*
*/
-static ssize_t ext4_ext_direct_IO(struct kiocb *iocb, struct iov_iter *iter)
+static ssize_t ext4_direct_IO_write(struct kiocb *iocb, struct iov_iter *iter)
{
struct file *file = iocb->ki_filp;
struct inode *inode = file->f_mapping->host;
+ struct ext4_inode_info *ei = EXT4_I(inode);
ssize_t ret;
loff_t offset = iocb->ki_pos;
size_t count = iov_iter_count(iter);
@@ -3345,10 +3339,25 @@
get_block_t *get_block_func = NULL;
int dio_flags = 0;
loff_t final_size = offset + count;
+ int orphan = 0;
+ handle_t *handle;
- /* Use the old path for reads and writes beyond i_size. */
- if (iov_iter_rw(iter) != WRITE || final_size > inode->i_size)
- return ext4_ind_direct_IO(iocb, iter);
+ if (final_size > inode->i_size) {
+ /* Credits for sb + inode write */
+ handle = ext4_journal_start(inode, EXT4_HT_INODE, 2);
+ if (IS_ERR(handle)) {
+ ret = PTR_ERR(handle);
+ goto out;
+ }
+ ret = ext4_orphan_add(handle, inode);
+ if (ret) {
+ ext4_journal_stop(handle);
+ goto out;
+ }
+ orphan = 1;
+ ei->i_disksize = inode->i_size;
+ ext4_journal_stop(handle);
+ }
BUG_ON(iocb->private == NULL);
@@ -3357,8 +3366,7 @@
* conversion. This also disallows race between truncate() and
* overwrite DIO as i_dio_count needs to be incremented under i_mutex.
*/
- if (iov_iter_rw(iter) == WRITE)
- inode_dio_begin(inode);
+ inode_dio_begin(inode);
/* If we do a overwrite dio, i_mutex locking can be released */
overwrite = *((int *)iocb->private);
@@ -3367,7 +3375,7 @@
inode_unlock(inode);
/*
- * We could direct write to holes and fallocate.
+ * For extent mapped files we could direct write to holes and fallocate.
*
* Allocated blocks to fill the hole are marked as unwritten to prevent
* parallel buffered read to expose the stale data before DIO complete
@@ -3389,7 +3397,23 @@
iocb->private = NULL;
if (overwrite)
get_block_func = ext4_dio_get_block_overwrite;
- else if (is_sync_kiocb(iocb)) {
+ else if (IS_DAX(inode)) {
+ /*
+ * We can avoid zeroing for aligned DAX writes beyond EOF. Other
+ * writes need zeroing either because they can race with page
+ * faults or because they use partial blocks.
+ */
+ if (round_down(offset, 1<<inode->i_blkbits) >= inode->i_size &&
+ ext4_aligned_io(inode, offset, count))
+ get_block_func = ext4_dio_get_block;
+ else
+ get_block_func = ext4_dax_get_block;
+ dio_flags = DIO_LOCKING;
+ } else if (!ext4_test_inode_flag(inode, EXT4_INODE_EXTENTS) ||
+ round_down(offset, 1 << inode->i_blkbits) >= inode->i_size) {
+ get_block_func = ext4_dio_get_block;
+ dio_flags = DIO_LOCKING | DIO_SKIP_HOLES;
+ } else if (is_sync_kiocb(iocb)) {
get_block_func = ext4_dio_get_block_unwritten_sync;
dio_flags = DIO_LOCKING;
} else {
@@ -3399,10 +3423,10 @@
#ifdef CONFIG_EXT4_FS_ENCRYPTION
BUG_ON(ext4_encrypted_inode(inode) && S_ISREG(inode->i_mode));
#endif
- if (IS_DAX(inode))
+ if (IS_DAX(inode)) {
ret = dax_do_io(iocb, inode, iter, get_block_func,
ext4_end_io_dio, dio_flags);
- else
+ } else
ret = __blockdev_direct_IO(iocb, inode,
inode->i_sb->s_bdev, iter,
get_block_func,
@@ -3422,12 +3446,86 @@
ext4_clear_inode_state(inode, EXT4_STATE_DIO_UNWRITTEN);
}
- if (iov_iter_rw(iter) == WRITE)
- inode_dio_end(inode);
+ inode_dio_end(inode);
/* take i_mutex locking again if we do a ovewrite dio */
if (overwrite)
inode_lock(inode);
+ if (ret < 0 && final_size > inode->i_size)
+ ext4_truncate_failed_write(inode);
+
+ /* Handle extending of i_size after direct IO write */
+ if (orphan) {
+ int err;
+
+ /* Credits for sb + inode write */
+ handle = ext4_journal_start(inode, EXT4_HT_INODE, 2);
+ if (IS_ERR(handle)) {
+ /* This is really bad luck. We've written the data
+ * but cannot extend i_size. Bail out and pretend
+ * the write failed... */
+ ret = PTR_ERR(handle);
+ if (inode->i_nlink)
+ ext4_orphan_del(NULL, inode);
+
+ goto out;
+ }
+ if (inode->i_nlink)
+ ext4_orphan_del(handle, inode);
+ if (ret > 0) {
+ loff_t end = offset + ret;
+ if (end > inode->i_size) {
+ ei->i_disksize = end;
+ i_size_write(inode, end);
+ /*
+ * We're going to return a positive `ret'
+ * here due to non-zero-length I/O, so there's
+ * no way of reporting error returns from
+ * ext4_mark_inode_dirty() to userspace. So
+ * ignore it.
+ */
+ ext4_mark_inode_dirty(handle, inode);
+ }
+ }
+ err = ext4_journal_stop(handle);
+ if (ret == 0)
+ ret = err;
+ }
+out:
+ return ret;
+}
+
+static ssize_t ext4_direct_IO_read(struct kiocb *iocb, struct iov_iter *iter)
+{
+ int unlocked = 0;
+ struct inode *inode = iocb->ki_filp->f_mapping->host;
+ ssize_t ret;
+
+ if (ext4_should_dioread_nolock(inode)) {
+ /*
+ * Nolock dioread optimization may be dynamically disabled
+ * via ext4_inode_block_unlocked_dio(). Check inode's state
+ * while holding extra i_dio_count ref.
+ */
+ inode_dio_begin(inode);
+ smp_mb();
+ if (unlikely(ext4_test_inode_state(inode,
+ EXT4_STATE_DIOREAD_LOCK)))
+ inode_dio_end(inode);
+ else
+ unlocked = 1;
+ }
+ if (IS_DAX(inode)) {
+ ret = dax_do_io(iocb, inode, iter, ext4_dio_get_block,
+ NULL, unlocked ? 0 : DIO_LOCKING);
+ } else {
+ ret = __blockdev_direct_IO(iocb, inode, inode->i_sb->s_bdev,
+ iter, ext4_dio_get_block,
+ NULL, NULL,
+ unlocked ? 0 : DIO_LOCKING);
+ }
+ if (unlocked)
+ inode_dio_end(inode);
return ret;
}
@@ -3455,10 +3553,10 @@
return 0;
trace_ext4_direct_IO_enter(inode, offset, count, iov_iter_rw(iter));
- if (ext4_test_inode_flag(inode, EXT4_INODE_EXTENTS))
- ret = ext4_ext_direct_IO(iocb, iter);
+ if (iov_iter_rw(iter) == READ)
+ ret = ext4_direct_IO_read(iocb, iter);
else
- ret = ext4_ind_direct_IO(iocb, iter);
+ ret = ext4_direct_IO_write(iocb, iter);
trace_ext4_direct_IO_exit(inode, offset, count, iov_iter_rw(iter), ret);
return ret;
}
@@ -3534,10 +3632,7 @@
{
switch (ext4_inode_journal_mode(inode)) {
case EXT4_INODE_ORDERED_DATA_MODE:
- ext4_set_inode_state(inode, EXT4_STATE_ORDERED_MODE);
- break;
case EXT4_INODE_WRITEBACK_DATA_MODE:
- ext4_clear_inode_state(inode, EXT4_STATE_ORDERED_MODE);
break;
case EXT4_INODE_JOURNAL_DATA_MODE:
inode->i_mapping->a_ops = &ext4_journalled_aops;
@@ -3630,8 +3725,8 @@
} else {
err = 0;
mark_buffer_dirty(bh);
- if (ext4_test_inode_state(inode, EXT4_STATE_ORDERED_MODE))
- err = ext4_jbd2_file_inode(handle, inode);
+ if (ext4_should_order_data(inode))
+ err = ext4_jbd2_inode_add_write(handle, inode);
}
unlock:
@@ -5429,6 +5524,7 @@
journal_t *journal;
handle_t *handle;
int err;
+ struct ext4_sb_info *sbi = EXT4_SB(inode->i_sb);
/*
* We have to be very careful here: changing a data block's
@@ -5445,22 +5541,30 @@
return 0;
if (is_journal_aborted(journal))
return -EROFS;
- /* We have to allocate physical blocks for delalloc blocks
- * before flushing journal. otherwise delalloc blocks can not
- * be allocated any more. even more truncate on delalloc blocks
- * could trigger BUG by flushing delalloc blocks in journal.
- * There is no delalloc block in non-journal data mode.
- */
- if (val && test_opt(inode->i_sb, DELALLOC)) {
- err = ext4_alloc_da_blocks(inode);
- if (err < 0)
- return err;
- }
/* Wait for all existing dio workers */
ext4_inode_block_unlocked_dio(inode);
inode_dio_wait(inode);
+ /*
+ * Before flushing the journal and switching inode's aops, we have
+ * to flush all dirty data the inode has. There can be outstanding
+ * delayed allocations, there can be unwritten extents created by
+ * fallocate or buffered writes in dioread_nolock mode covered by
+ * dirty data which can be converted only after flushing the dirty
+ * data (and journalled aops don't know how to handle these cases).
+ */
+ if (val) {
+ down_write(&EXT4_I(inode)->i_mmap_sem);
+ err = filemap_write_and_wait(inode->i_mapping);
+ if (err < 0) {
+ up_write(&EXT4_I(inode)->i_mmap_sem);
+ ext4_inode_resume_unlocked_dio(inode);
+ return err;
+ }
+ }
+
+ percpu_down_write(&sbi->s_journal_flag_rwsem);
jbd2_journal_lock_updates(journal);
/*
@@ -5477,6 +5581,7 @@
err = jbd2_journal_flush(journal);
if (err < 0) {
jbd2_journal_unlock_updates(journal);
+ percpu_up_write(&sbi->s_journal_flag_rwsem);
ext4_inode_resume_unlocked_dio(inode);
return err;
}
@@ -5485,6 +5590,10 @@
ext4_set_aops(inode);
jbd2_journal_unlock_updates(journal);
+ percpu_up_write(&sbi->s_journal_flag_rwsem);
+
+ if (val)
+ up_write(&EXT4_I(inode)->i_mmap_sem);
ext4_inode_resume_unlocked_dio(inode);
/* Finally we can mark the inode as dirty. */
diff --git a/fs/ext4/ioctl.c b/fs/ext4/ioctl.c
index eae5917..28cc412 100644
--- a/fs/ext4/ioctl.c
+++ b/fs/ext4/ioctl.c
@@ -13,8 +13,8 @@
#include <linux/compat.h>
#include <linux/mount.h>
#include <linux/file.h>
-#include <linux/random.h>
#include <linux/quotaops.h>
+#include <linux/uuid.h>
#include <asm/uaccess.h>
#include "ext4_jbd2.h"
#include "ext4.h"
@@ -365,7 +365,7 @@
struct dquot *transfer_to[MAXQUOTAS] = { };
transfer_to[PRJQUOTA] = dqget(sb, make_kqid_projid(kprojid));
- if (transfer_to[PRJQUOTA]) {
+ if (!IS_ERR(transfer_to[PRJQUOTA])) {
err = __dquot_transfer(inode, transfer_to);
dqput(transfer_to[PRJQUOTA]);
if (err)
diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c
index eeeade7..c1ab3ec 100644
--- a/fs/ext4/mballoc.c
+++ b/fs/ext4/mballoc.c
@@ -1266,6 +1266,7 @@
static int mb_find_order_for_block(struct ext4_buddy *e4b, int block)
{
int order = 1;
+ int bb_incr = 1 << (e4b->bd_blkbits - 1);
void *bb;
BUG_ON(e4b->bd_bitmap == e4b->bd_buddy);
@@ -1278,7 +1279,8 @@
/* this block is part of buddy of order 'order' */
return order;
}
- bb += 1 << (e4b->bd_blkbits - order);
+ bb += bb_incr;
+ bb_incr >>= 1;
order++;
}
return 0;
@@ -2583,7 +2585,7 @@
{
struct ext4_sb_info *sbi = EXT4_SB(sb);
unsigned i, j;
- unsigned offset;
+ unsigned offset, offset_incr;
unsigned max;
int ret;
@@ -2612,11 +2614,13 @@
i = 1;
offset = 0;
+ offset_incr = 1 << (sb->s_blocksize_bits - 1);
max = sb->s_blocksize << 2;
do {
sbi->s_mb_offsets[i] = offset;
sbi->s_mb_maxs[i] = max;
- offset += 1 << (sb->s_blocksize_bits - i);
+ offset += offset_incr;
+ offset_incr = offset_incr >> 1;
max = max >> 1;
i++;
} while (i <= sb->s_blocksize_bits + 1);
@@ -4935,7 +4939,7 @@
* boundary.
*/
if (bit + count > EXT4_BLOCKS_PER_GROUP(sb)) {
- ext4_warning(sb, "too much blocks added to group %u\n",
+ ext4_warning(sb, "too much blocks added to group %u",
block_group);
err = -EINVAL;
goto error_return;
diff --git a/fs/ext4/mmp.c b/fs/ext4/mmp.c
index 2444527..23d436d 100644
--- a/fs/ext4/mmp.c
+++ b/fs/ext4/mmp.c
@@ -121,7 +121,7 @@
__ext4_warning(sb, function, line, "%s", msg);
__ext4_warning(sb, function, line,
"MMP failure info: last update time: %llu, last update "
- "node: %s, last update device: %s\n",
+ "node: %s, last update device: %s",
(long long unsigned int) le64_to_cpu(mmp->mmp_time),
mmp->mmp_nodename, mmp->mmp_bdevname);
}
@@ -353,7 +353,7 @@
* wait for MMP interval and check mmp_seq.
*/
if (schedule_timeout_interruptible(HZ * wait_time) != 0) {
- ext4_warning(sb, "MMP startup interrupted, failing mount\n");
+ ext4_warning(sb, "MMP startup interrupted, failing mount");
goto failed;
}
diff --git a/fs/ext4/move_extent.c b/fs/ext4/move_extent.c
index 325cef4..a920c5d 100644
--- a/fs/ext4/move_extent.c
+++ b/fs/ext4/move_extent.c
@@ -400,7 +400,7 @@
/* Even in case of data=writeback it is reasonable to pin
* inode to transaction, to prevent unexpected data loss */
- *err = ext4_jbd2_file_inode(handle, orig_inode);
+ *err = ext4_jbd2_inode_add_write(handle, orig_inode);
unlock_pages:
unlock_page(pagep[0]);
diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c
index 5611ec9..ec4c399 100644
--- a/fs/ext4/namei.c
+++ b/fs/ext4/namei.c
@@ -1107,6 +1107,11 @@
}
while (1) {
+ if (fatal_signal_pending(current)) {
+ err = -ERESTARTSYS;
+ goto errout;
+ }
+ cond_resched();
block = dx_get_block(frame->at);
ret = htree_dirblock_to_tree(dir_file, dir, block, &hinfo,
start_hash, start_minor_hash);
@@ -1613,7 +1618,7 @@
if (nokey)
return ERR_PTR(-ENOKEY);
ext4_warning(inode->i_sb,
- "Inconsistent encryption contexts: %lu/%lu\n",
+ "Inconsistent encryption contexts: %lu/%lu",
(unsigned long) dir->i_ino,
(unsigned long) inode->i_ino);
return ERR_PTR(-EPERM);
@@ -2828,7 +2833,7 @@
* list entries can cause panics at unmount time.
*/
mutex_lock(&sbi->s_orphan_lock);
- list_del(&EXT4_I(inode)->i_orphan);
+ list_del_init(&EXT4_I(inode)->i_orphan);
mutex_unlock(&sbi->s_orphan_lock);
}
}
diff --git a/fs/ext4/page-io.c b/fs/ext4/page-io.c
index e4fc8ea..2a01df9 100644
--- a/fs/ext4/page-io.c
+++ b/fs/ext4/page-io.c
@@ -342,9 +342,7 @@
if (bio) {
int io_op = io->io_wbc->sync_mode == WB_SYNC_ALL ?
WRITE_SYNC : WRITE;
- bio_get(io->io_bio);
submit_bio(io_op, io->io_bio);
- bio_put(io->io_bio);
}
io->io_bio = NULL;
}
diff --git a/fs/ext4/resize.c b/fs/ext4/resize.c
index 34038e3..cf68100 100644
--- a/fs/ext4/resize.c
+++ b/fs/ext4/resize.c
@@ -41,7 +41,7 @@
*/
if (EXT4_SB(sb)->s_mount_state & EXT4_ERROR_FS) {
ext4_warning(sb, "There are errors in the filesystem, "
- "so online resizing is not allowed\n");
+ "so online resizing is not allowed");
return -EPERM;
}
diff --git a/fs/ext4/super.c b/fs/ext4/super.c
index 304c712..20c5d52 100644
--- a/fs/ext4/super.c
+++ b/fs/ext4/super.c
@@ -859,6 +859,7 @@
percpu_counter_destroy(&sbi->s_freeinodes_counter);
percpu_counter_destroy(&sbi->s_dirs_counter);
percpu_counter_destroy(&sbi->s_dirtyclusters_counter);
+ percpu_free_rwsem(&sbi->s_journal_flag_rwsem);
brelse(sbi->s_sbh);
#ifdef CONFIG_QUOTA
for (i = 0; i < EXT4_MAXQUOTAS; i++)
@@ -3930,6 +3931,9 @@
if (!err)
err = percpu_counter_init(&sbi->s_dirtyclusters_counter, 0,
GFP_KERNEL);
+ if (!err)
+ err = percpu_init_rwsem(&sbi->s_journal_flag_rwsem);
+
if (err) {
ext4_msg(sb, KERN_ERR, "insufficient memory");
goto failed_mount6;
diff --git a/fs/f2fs/Kconfig b/fs/f2fs/Kconfig
index 1f8982a..378c221 100644
--- a/fs/f2fs/Kconfig
+++ b/fs/f2fs/Kconfig
@@ -94,3 +94,11 @@
information and block IO patterns in the filesystem level.
If unsure, say N.
+
+config F2FS_FAULT_INJECTION
+ bool "F2FS fault injection facility"
+ depends on F2FS_FS
+ help
+ Test F2FS to inject faults such as ENOMEM, ENOSPC, and so on.
+
+ If unsure, say N.
diff --git a/fs/f2fs/acl.c b/fs/f2fs/acl.c
index 6f1fdda..a31c7e8 100644
--- a/fs/f2fs/acl.c
+++ b/fs/f2fs/acl.c
@@ -115,7 +115,7 @@
struct f2fs_acl_entry *entry;
int i;
- f2fs_acl = kmalloc(sizeof(struct f2fs_acl_header) + acl->a_count *
+ f2fs_acl = f2fs_kmalloc(sizeof(struct f2fs_acl_header) + acl->a_count *
sizeof(struct f2fs_acl_entry), GFP_NOFS);
if (!f2fs_acl)
return ERR_PTR(-ENOMEM);
@@ -175,7 +175,7 @@
retval = f2fs_getxattr(inode, name_index, "", NULL, 0, dpage);
if (retval > 0) {
- value = kmalloc(retval, GFP_F2FS_ZERO);
+ value = f2fs_kmalloc(retval, GFP_F2FS_ZERO);
if (!value)
return ERR_PTR(-ENOMEM);
retval = f2fs_getxattr(inode, name_index, "", value,
diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c
index 0955312..3891600 100644
--- a/fs/f2fs/checkpoint.c
+++ b/fs/f2fs/checkpoint.c
@@ -26,6 +26,14 @@
static struct kmem_cache *ino_entry_slab;
struct kmem_cache *inode_entry_slab;
+void f2fs_stop_checkpoint(struct f2fs_sb_info *sbi, bool end_io)
+{
+ set_ckpt_flags(sbi->ckpt, CP_ERROR_FLAG);
+ sbi->sb->s_flags |= MS_RDONLY;
+ if (!end_io)
+ f2fs_flush_merged_bios(sbi);
+}
+
/*
* We guarantee no failure on the returned page.
*/
@@ -34,7 +42,7 @@
struct address_space *mapping = META_MAPPING(sbi);
struct page *page = NULL;
repeat:
- page = grab_cache_page(mapping, index);
+ page = f2fs_grab_cache_page(mapping, index, false);
if (!page) {
cond_resched();
goto repeat;
@@ -64,7 +72,7 @@
if (unlikely(!is_meta))
fio.rw &= ~REQ_META;
repeat:
- page = grab_cache_page(mapping, index);
+ page = f2fs_grab_cache_page(mapping, index, false);
if (!page) {
cond_resched();
goto repeat;
@@ -91,7 +99,7 @@
* meta page.
*/
if (unlikely(!PageUptodate(page)))
- f2fs_stop_checkpoint(sbi);
+ f2fs_stop_checkpoint(sbi, false);
out:
return page;
}
@@ -186,7 +194,8 @@
BUG();
}
- page = grab_cache_page(META_MAPPING(sbi), fio.new_blkaddr);
+ page = f2fs_grab_cache_page(META_MAPPING(sbi),
+ fio.new_blkaddr, false);
if (!page)
continue;
if (PageUptodate(page)) {
@@ -211,7 +220,7 @@
bool readahead = false;
page = find_get_page(META_MAPPING(sbi), index);
- if (!page || (page && !PageUptodate(page)))
+ if (!page || !PageUptodate(page))
readahead = true;
f2fs_put_page(page, 0);
@@ -448,12 +457,12 @@
return e ? true : false;
}
-void release_ino_entry(struct f2fs_sb_info *sbi)
+void release_ino_entry(struct f2fs_sb_info *sbi, bool all)
{
struct ino_entry *e, *tmp;
int i;
- for (i = APPEND_INO; i <= UPDATE_INO; i++) {
+ for (i = all ? ORPHAN_INO: APPEND_INO; i <= UPDATE_INO; i++) {
struct inode_management *im = &sbi->im[i];
spin_lock(&im->ino_lock);
@@ -473,6 +482,13 @@
int err = 0;
spin_lock(&im->ino_lock);
+
+#ifdef CONFIG_F2FS_FAULT_INJECTION
+ if (time_to_inject(FAULT_ORPHAN)) {
+ spin_unlock(&im->ino_lock);
+ return -ENOSPC;
+ }
+#endif
if (unlikely(im->ino_num >= sbi->max_orphans))
err = -ENOSPC;
else
@@ -777,43 +793,32 @@
!S_ISLNK(inode->i_mode))
return;
- spin_lock(&sbi->inode_lock[type]);
- __add_dirty_inode(inode, type);
- inode_inc_dirty_pages(inode);
- spin_unlock(&sbi->inode_lock[type]);
+ if (type != FILE_INODE || test_opt(sbi, DATA_FLUSH)) {
+ spin_lock(&sbi->inode_lock[type]);
+ __add_dirty_inode(inode, type);
+ spin_unlock(&sbi->inode_lock[type]);
+ }
+ inode_inc_dirty_pages(inode);
SetPagePrivate(page);
f2fs_trace_pid(page);
}
-void add_dirty_dir_inode(struct inode *inode)
-{
- struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
-
- spin_lock(&sbi->inode_lock[DIR_INODE]);
- __add_dirty_inode(inode, DIR_INODE);
- spin_unlock(&sbi->inode_lock[DIR_INODE]);
-}
-
void remove_dirty_inode(struct inode *inode)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
- struct f2fs_inode_info *fi = F2FS_I(inode);
enum inode_type type = S_ISDIR(inode->i_mode) ? DIR_INODE : FILE_INODE;
if (!S_ISDIR(inode->i_mode) && !S_ISREG(inode->i_mode) &&
!S_ISLNK(inode->i_mode))
return;
+ if (type == FILE_INODE && !test_opt(sbi, DATA_FLUSH))
+ return;
+
spin_lock(&sbi->inode_lock[type]);
__remove_dirty_inode(inode, type);
spin_unlock(&sbi->inode_lock[type]);
-
- /* Only from the recovery routine */
- if (is_inode_flag_set(fi, FI_DELAY_IPUT)) {
- clear_inode_flag(fi, FI_DELAY_IPUT);
- iput(inode);
- }
}
int sync_dirty_inodes(struct f2fs_sb_info *sbi, enum inode_type type)
@@ -892,7 +897,7 @@
if (get_pages(sbi, F2FS_DIRTY_NODES)) {
up_write(&sbi->node_write);
- err = sync_node_pages(sbi, 0, &wbc);
+ err = sync_node_pages(sbi, &wbc);
if (err) {
f2fs_unlock_all(sbi);
goto out;
@@ -917,7 +922,7 @@
for (;;) {
prepare_to_wait(&sbi->cp_wait, &wait, TASK_UNINTERRUPTIBLE);
- if (!get_pages(sbi, F2FS_WRITEBACK))
+ if (!atomic_read(&sbi->nr_wb_bios))
break;
io_schedule_timeout(5*HZ);
@@ -1082,7 +1087,7 @@
/* update user_block_counts */
sbi->last_valid_block_count = sbi->total_valid_block_count;
- sbi->alloc_valid_block_count = 0;
+ percpu_counter_set(&sbi->alloc_valid_block_count, 0);
/* Here, we only have one bio having CP pack */
sync_meta_pages(sbi, META_FLUSH, LONG_MAX);
@@ -1098,7 +1103,7 @@
invalidate_mapping_pages(META_MAPPING(sbi), discard_blk,
discard_blk);
- release_ino_entry(sbi);
+ release_ino_entry(sbi, false);
if (unlikely(f2fs_cp_error(sbi)))
return -EIO;
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index bb376c3..9a8bbc1 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -68,13 +68,12 @@
if (unlikely(bio->bi_error)) {
set_bit(AS_EIO, &page->mapping->flags);
- f2fs_stop_checkpoint(sbi);
+ f2fs_stop_checkpoint(sbi, true);
}
end_page_writeback(page);
- dec_page_count(sbi, F2FS_WRITEBACK);
}
-
- if (!get_pages(sbi, F2FS_WRITEBACK) && wq_has_sleeper(&sbi->cp_wait))
+ if (atomic_dec_and_test(&sbi->nr_wb_bios) &&
+ wq_has_sleeper(&sbi->cp_wait))
wake_up(&sbi->cp_wait);
bio_put(bio);
@@ -98,6 +97,14 @@
return bio;
}
+static inline void __submit_bio(struct f2fs_sb_info *sbi, int rw,
+ struct bio *bio)
+{
+ if (!is_read_io(rw))
+ atomic_inc(&sbi->nr_wb_bios);
+ submit_bio(rw, bio);
+}
+
static void __submit_merged_bio(struct f2fs_bio_info *io)
{
struct f2fs_io_info *fio = &io->fio;
@@ -110,7 +117,7 @@
else
trace_f2fs_submit_write_bio(io->sbi->sb, fio, io->bio);
- submit_bio(fio->rw, io->bio);
+ __submit_bio(io->sbi, fio->rw, io->bio);
io->bio = NULL;
}
@@ -228,7 +235,7 @@
return -EFAULT;
}
- submit_bio(fio->rw, bio);
+ __submit_bio(fio->sbi, fio->rw, bio);
return 0;
}
@@ -248,9 +255,6 @@
down_write(&io->io_rwsem);
- if (!is_read)
- inc_page_count(sbi, F2FS_WRITEBACK);
-
if (io->bio && (io->last_block_in_bio != fio->new_blkaddr - 1 ||
io->fio.rw != fio->rw))
__submit_merged_bio(io);
@@ -278,6 +282,16 @@
trace_f2fs_submit_page_mbio(fio->page, fio);
}
+static void __set_data_blkaddr(struct dnode_of_data *dn)
+{
+ struct f2fs_node *rn = F2FS_NODE(dn->node_page);
+ __le32 *addr_array;
+
+ /* Get physical address of data block */
+ addr_array = blkaddr_in_node(rn);
+ addr_array[dn->ofs_in_node] = cpu_to_le32(dn->data_blkaddr);
+}
+
/*
* Lock ordering for the change of data block address:
* ->data_page
@@ -286,19 +300,9 @@
*/
void set_data_blkaddr(struct dnode_of_data *dn)
{
- struct f2fs_node *rn;
- __le32 *addr_array;
- struct page *node_page = dn->node_page;
- unsigned int ofs_in_node = dn->ofs_in_node;
-
- f2fs_wait_on_page_writeback(node_page, NODE, true);
-
- rn = F2FS_NODE(node_page);
-
- /* Get physical address of data block */
- addr_array = blkaddr_in_node(rn);
- addr_array[ofs_in_node] = cpu_to_le32(dn->data_blkaddr);
- if (set_page_dirty(node_page))
+ f2fs_wait_on_page_writeback(dn->node_page, NODE, true);
+ __set_data_blkaddr(dn);
+ if (set_page_dirty(dn->node_page))
dn->node_changed = true;
}
@@ -309,24 +313,53 @@
f2fs_update_extent_cache(dn);
}
-int reserve_new_block(struct dnode_of_data *dn)
+/* dn->ofs_in_node will be returned with up-to-date last block pointer */
+int reserve_new_blocks(struct dnode_of_data *dn, blkcnt_t count)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(dn->inode);
+ if (!count)
+ return 0;
+
if (unlikely(is_inode_flag_set(F2FS_I(dn->inode), FI_NO_ALLOC)))
return -EPERM;
- if (unlikely(!inc_valid_block_count(sbi, dn->inode, 1)))
+ if (unlikely(!inc_valid_block_count(sbi, dn->inode, &count)))
return -ENOSPC;
- trace_f2fs_reserve_new_block(dn->inode, dn->nid, dn->ofs_in_node);
+ trace_f2fs_reserve_new_blocks(dn->inode, dn->nid,
+ dn->ofs_in_node, count);
- dn->data_blkaddr = NEW_ADDR;
- set_data_blkaddr(dn);
+ f2fs_wait_on_page_writeback(dn->node_page, NODE, true);
+
+ for (; count > 0; dn->ofs_in_node++) {
+ block_t blkaddr =
+ datablock_addr(dn->node_page, dn->ofs_in_node);
+ if (blkaddr == NULL_ADDR) {
+ dn->data_blkaddr = NEW_ADDR;
+ __set_data_blkaddr(dn);
+ count--;
+ }
+ }
+
+ if (set_page_dirty(dn->node_page))
+ dn->node_changed = true;
+
mark_inode_dirty(dn->inode);
sync_inode_page(dn);
return 0;
}
+/* Should keep dn->ofs_in_node unchanged */
+int reserve_new_block(struct dnode_of_data *dn)
+{
+ unsigned int ofs_in_node = dn->ofs_in_node;
+ int ret;
+
+ ret = reserve_new_blocks(dn, 1);
+ dn->ofs_in_node = ofs_in_node;
+ return ret;
+}
+
int f2fs_reserve_block(struct dnode_of_data *dn, pgoff_t index)
{
bool need_put = dn->inode_page ? false : true;
@@ -545,6 +578,7 @@
struct node_info ni;
int seg = CURSEG_WARM_DATA;
pgoff_t fofs;
+ blkcnt_t count = 1;
if (unlikely(is_inode_flag_set(F2FS_I(dn->inode), FI_NO_ALLOC)))
return -EPERM;
@@ -553,7 +587,7 @@
if (dn->data_blkaddr == NEW_ADDR)
goto alloc;
- if (unlikely(!inc_valid_block_count(sbi, dn->inode, 1)))
+ if (unlikely(!inc_valid_block_count(sbi, dn->inode, &count)))
return -ENOSPC;
alloc:
@@ -582,8 +616,8 @@
struct f2fs_map_blocks map;
ssize_t ret = 0;
- map.m_lblk = F2FS_BYTES_TO_BLK(iocb->ki_pos);
- map.m_len = F2FS_BLK_ALIGN(iov_iter_count(from));
+ map.m_lblk = F2FS_BLK_ALIGN(iocb->ki_pos);
+ map.m_len = F2FS_BYTES_TO_BLK(iov_iter_count(from));
map.m_next_pgofs = NULL;
if (f2fs_encrypted_inode(inode))
@@ -621,8 +655,10 @@
struct dnode_of_data dn;
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
int mode = create ? ALLOC_NODE : LOOKUP_NODE_RA;
- pgoff_t pgofs, end_offset;
+ pgoff_t pgofs, end_offset, end;
int err = 0, ofs = 1;
+ unsigned int ofs_in_node, last_ofs_in_node;
+ blkcnt_t prealloc;
struct extent_info ei;
bool allocated = false;
block_t blkaddr;
@@ -632,6 +668,7 @@
/* it only supports block size == page size */
pgofs = (pgoff_t)map->m_lblk;
+ end = pgofs + maxblocks;
if (!create && f2fs_lookup_extent_cache(inode, pgofs, &ei)) {
map->m_pblk = ei.blk + pgofs - ei.fofs;
@@ -648,6 +685,8 @@
set_new_dnode(&dn, inode, NULL, NULL, 0);
err = get_dnode_of_data(&dn, pgofs, mode);
if (err) {
+ if (flag == F2FS_GET_BLOCK_BMAP)
+ map->m_pblk = 0;
if (err == -ENOENT) {
err = 0;
if (map->m_next_pgofs)
@@ -657,6 +696,8 @@
goto unlock_out;
}
+ prealloc = 0;
+ ofs_in_node = dn.ofs_in_node;
end_offset = ADDRS_PER_PAGE(dn.node_page, inode);
next_block:
@@ -669,31 +710,41 @@
goto sync_out;
}
if (flag == F2FS_GET_BLOCK_PRE_AIO) {
- if (blkaddr == NULL_ADDR)
- err = reserve_new_block(&dn);
+ if (blkaddr == NULL_ADDR) {
+ prealloc++;
+ last_ofs_in_node = dn.ofs_in_node;
+ }
} else {
err = __allocate_data_block(&dn);
+ if (!err) {
+ set_inode_flag(F2FS_I(inode),
+ FI_APPEND_WRITE);
+ allocated = true;
+ }
}
if (err)
goto sync_out;
- allocated = true;
map->m_flags = F2FS_MAP_NEW;
blkaddr = dn.data_blkaddr;
} else {
+ if (flag == F2FS_GET_BLOCK_BMAP) {
+ map->m_pblk = 0;
+ goto sync_out;
+ }
if (flag == F2FS_GET_BLOCK_FIEMAP &&
blkaddr == NULL_ADDR) {
if (map->m_next_pgofs)
*map->m_next_pgofs = pgofs + 1;
}
if (flag != F2FS_GET_BLOCK_FIEMAP ||
- blkaddr != NEW_ADDR) {
- if (flag == F2FS_GET_BLOCK_BMAP)
- err = -ENOENT;
+ blkaddr != NEW_ADDR)
goto sync_out;
- }
}
}
+ if (flag == F2FS_GET_BLOCK_PRE_AIO)
+ goto skip;
+
if (map->m_len == 0) {
/* preallocated unwritten block should be mapped for fiemap. */
if (blkaddr == NEW_ADDR)
@@ -705,33 +756,50 @@
} else if ((map->m_pblk != NEW_ADDR &&
blkaddr == (map->m_pblk + ofs)) ||
(map->m_pblk == NEW_ADDR && blkaddr == NEW_ADDR) ||
- flag == F2FS_GET_BLOCK_PRE_DIO ||
- flag == F2FS_GET_BLOCK_PRE_AIO) {
+ flag == F2FS_GET_BLOCK_PRE_DIO) {
ofs++;
map->m_len++;
} else {
goto sync_out;
}
+skip:
dn.ofs_in_node++;
pgofs++;
- if (map->m_len < maxblocks) {
- if (dn.ofs_in_node < end_offset)
- goto next_block;
+ /* preallocate blocks in batch for one dnode page */
+ if (flag == F2FS_GET_BLOCK_PRE_AIO &&
+ (pgofs == end || dn.ofs_in_node == end_offset)) {
- if (allocated)
- sync_inode_page(&dn);
- f2fs_put_dnode(&dn);
+ dn.ofs_in_node = ofs_in_node;
+ err = reserve_new_blocks(&dn, prealloc);
+ if (err)
+ goto sync_out;
- if (create) {
- f2fs_unlock_op(sbi);
- f2fs_balance_fs(sbi, allocated);
+ map->m_len += dn.ofs_in_node - ofs_in_node;
+ if (prealloc && dn.ofs_in_node != last_ofs_in_node + 1) {
+ err = -ENOSPC;
+ goto sync_out;
}
- allocated = false;
- goto next_dnode;
+ dn.ofs_in_node = end_offset;
}
+ if (pgofs >= end)
+ goto sync_out;
+ else if (dn.ofs_in_node < end_offset)
+ goto next_block;
+
+ if (allocated)
+ sync_inode_page(&dn);
+ f2fs_put_dnode(&dn);
+
+ if (create) {
+ f2fs_unlock_op(sbi);
+ f2fs_balance_fs(sbi, allocated);
+ }
+ allocated = false;
+ goto next_dnode;
+
sync_out:
if (allocated)
sync_inode_page(&dn);
@@ -983,7 +1051,7 @@
*/
if (bio && (last_block_in_bio != block_nr - 1)) {
submit_and_realloc:
- submit_bio(READ, bio);
+ __submit_bio(F2FS_I_SB(inode), READ, bio);
bio = NULL;
}
if (bio == NULL) {
@@ -1026,7 +1094,7 @@
goto next_page;
confused:
if (bio) {
- submit_bio(READ, bio);
+ __submit_bio(F2FS_I_SB(inode), READ, bio);
bio = NULL;
}
unlock_page(page);
@@ -1036,7 +1104,7 @@
}
BUG_ON(pages && !list_empty(pages));
if (bio)
- submit_bio(READ, bio);
+ __submit_bio(F2FS_I_SB(inode), READ, bio);
return 0;
}
@@ -1177,8 +1245,10 @@
goto redirty_out;
if (f2fs_is_drop_cache(inode))
goto out;
- if (f2fs_is_volatile_file(inode) && !wbc->for_reclaim &&
- available_free_memory(sbi, BASE_CHECK))
+ /* we should not write 0'th page having journal header */
+ if (f2fs_is_volatile_file(inode) && (!page->index ||
+ (!wbc->for_reclaim &&
+ available_free_memory(sbi, BASE_CHECK))))
goto redirty_out;
/* Dentry blocks are controlled by checkpoint */
@@ -1480,7 +1550,8 @@
if (pos + len <= MAX_INLINE_DATA) {
read_inline_data(page, ipage);
set_inode_flag(F2FS_I(inode), FI_DATA_EXIST);
- set_inline_node(ipage);
+ if (inode->i_nlink)
+ set_inline_node(ipage);
} else {
err = f2fs_convert_inline_page(&dn, page);
if (err)
@@ -1496,7 +1567,7 @@
} else {
/* hole case */
err = get_dnode_of_data(&dn, index, LOOKUP_NODE);
- if (err || (!err && dn.data_blkaddr == NULL_ADDR)) {
+ if (err || dn.data_blkaddr == NULL_ADDR) {
f2fs_put_dnode(&dn);
f2fs_lock_op(sbi);
locked = true;
@@ -1683,8 +1754,12 @@
trace_f2fs_direct_IO_enter(inode, offset, count, iov_iter_rw(iter));
err = blockdev_direct_IO(iocb, inode, iter, get_data_block_dio);
- if (err < 0 && iov_iter_rw(iter) == WRITE)
- f2fs_write_failed(mapping, offset + count);
+ if (iov_iter_rw(iter) == WRITE) {
+ if (err > 0)
+ set_inode_flag(F2FS_I(inode), FI_UPDATE_WRITE);
+ else if (err < 0)
+ f2fs_write_failed(mapping, offset + count);
+ }
trace_f2fs_direct_IO_exit(inode, offset, count, iov_iter_rw(iter), err);
@@ -1714,6 +1789,7 @@
if (IS_ATOMIC_WRITTEN_PAGE(page))
return;
+ set_page_private(page, 0);
ClearPagePrivate(page);
}
@@ -1727,6 +1803,7 @@
if (IS_ATOMIC_WRITTEN_PAGE(page))
return 0;
+ set_page_private(page, 0);
ClearPagePrivate(page);
return 1;
}
diff --git a/fs/f2fs/debug.c b/fs/f2fs/debug.c
index f4a61a5..d89a425 100644
--- a/fs/f2fs/debug.c
+++ b/fs/f2fs/debug.c
@@ -48,7 +48,7 @@
si->ndirty_dirs = sbi->ndirty_inode[DIR_INODE];
si->ndirty_files = sbi->ndirty_inode[FILE_INODE];
si->inmem_pages = get_pages(sbi, F2FS_INMEM_PAGES);
- si->wb_pages = get_pages(sbi, F2FS_WRITEBACK);
+ si->wb_bios = atomic_read(&sbi->nr_wb_bios);
si->total_count = (int)sbi->user_block_count / sbi->blocks_per_seg;
si->rsvd_segs = reserved_segments(sbi);
si->overp_segs = overprovision_segments(sbi);
@@ -58,6 +58,7 @@
si->inline_xattr = atomic_read(&sbi->inline_xattr);
si->inline_inode = atomic_read(&sbi->inline_inode);
si->inline_dir = atomic_read(&sbi->inline_dir);
+ si->orphans = sbi->im[ORPHAN_INO].ino_num;
si->utilization = utilization(sbi);
si->free_segs = free_segments(sbi);
@@ -143,6 +144,7 @@
si->base_mem = sizeof(struct f2fs_sb_info) + sbi->sb->s_blocksize;
si->base_mem += 2 * sizeof(struct f2fs_inode_info);
si->base_mem += sizeof(*sbi->ckpt);
+ si->base_mem += sizeof(struct percpu_counter) * NR_COUNT_TYPE;
/* build sm */
si->base_mem += sizeof(struct f2fs_sm_info);
@@ -192,7 +194,7 @@
si->cache_mem += NM_I(sbi)->dirty_nat_cnt *
sizeof(struct nat_entry_set);
si->cache_mem += si->inmem_pages * sizeof(struct inmem_pages);
- for (i = 0; i <= UPDATE_INO; i++)
+ for (i = 0; i <= ORPHAN_INO; i++)
si->cache_mem += sbi->im[i].ino_num * sizeof(struct ino_entry);
si->cache_mem += atomic_read(&sbi->total_ext_tree) *
sizeof(struct extent_tree);
@@ -216,8 +218,9 @@
list_for_each_entry(si, &f2fs_stat_list, stat_list) {
update_general_status(si->sbi);
- seq_printf(s, "\n=====[ partition info(%pg). #%d ]=====\n",
- si->sbi->sb->s_bdev, i++);
+ seq_printf(s, "\n=====[ partition info(%pg). #%d, %s]=====\n",
+ si->sbi->sb->s_bdev, i++,
+ f2fs_readonly(si->sbi->sb) ? "RO": "RW");
seq_printf(s, "[SB: 1] [CP: 2] [SIT: %d] [NAT: %d] ",
si->sit_area_segs, si->nat_area_segs);
seq_printf(s, "[SSA: %d] [MAIN: %d",
@@ -237,6 +240,8 @@
si->inline_inode);
seq_printf(s, " - Inline_dentry Inode: %u\n",
si->inline_dir);
+ seq_printf(s, " - Orphan Inode: %u\n",
+ si->orphans);
seq_printf(s, "\nMain area: %d segs, %d secs %d zones\n",
si->main_area_segs, si->main_area_sections,
si->main_area_zones);
@@ -295,15 +300,15 @@
seq_printf(s, " - Inner Struct Count: tree: %d(%d), node: %d\n",
si->ext_tree, si->zombie_tree, si->ext_node);
seq_puts(s, "\nBalancing F2FS Async:\n");
- seq_printf(s, " - inmem: %4d, wb: %4d\n",
- si->inmem_pages, si->wb_pages);
- seq_printf(s, " - nodes: %4d in %4d\n",
+ seq_printf(s, " - inmem: %4lld, wb_bios: %4d\n",
+ si->inmem_pages, si->wb_bios);
+ seq_printf(s, " - nodes: %4lld in %4d\n",
si->ndirty_node, si->node_pages);
- seq_printf(s, " - dents: %4d in dirs:%4d\n",
+ seq_printf(s, " - dents: %4lld in dirs:%4d\n",
si->ndirty_dent, si->ndirty_dirs);
- seq_printf(s, " - datas: %4d in files:%4d\n",
+ seq_printf(s, " - datas: %4lld in files:%4d\n",
si->ndirty_data, si->ndirty_files);
- seq_printf(s, " - meta: %4d in %4d\n",
+ seq_printf(s, " - meta: %4lld in %4d\n",
si->ndirty_meta, si->meta_pages);
seq_printf(s, " - NATs: %9d/%9d\n - SITs: %9d/%9d\n",
si->dirty_nats, si->nats, si->dirty_sits, si->sits);
diff --git a/fs/f2fs/dir.c b/fs/f2fs/dir.c
index 9e46151..f9313f6 100644
--- a/fs/f2fs/dir.c
+++ b/fs/f2fs/dir.c
@@ -48,7 +48,6 @@
[F2FS_FT_SYMLINK] = DT_LNK,
};
-#define S_SHIFT 12
static unsigned char f2fs_type_by_mode[S_IFMT >> S_SHIFT] = {
[S_IFREG >> S_SHIFT] = F2FS_FT_REG_FILE,
[S_IFDIR >> S_SHIFT] = F2FS_FT_DIR,
@@ -64,6 +63,13 @@
de->file_type = f2fs_type_by_mode[(mode & S_IFMT) >> S_SHIFT];
}
+unsigned char get_de_type(struct f2fs_dir_entry *de)
+{
+ if (de->file_type < F2FS_FT_MAX)
+ return f2fs_filetype_table[de->file_type];
+ return DT_UNKNOWN;
+}
+
static unsigned long dir_block_index(unsigned int level,
int dir_level, unsigned int idx)
{
@@ -95,11 +101,6 @@
else
kunmap(dentry_page);
- /*
- * For the most part, it should be a bug when name_len is zero.
- * We stop here for figuring out where the bugs has occurred.
- */
- f2fs_bug_on(F2FS_P_SB(dentry_page), d.max < 0);
return de;
}
@@ -124,6 +125,11 @@
de = &d->dentry[bit_pos];
+ if (unlikely(!de->name_len)) {
+ bit_pos++;
+ continue;
+ }
+
/* encrypted case */
de_name.name = d->filename[bit_pos];
de_name.len = le16_to_cpu(de->name_len);
@@ -141,10 +147,6 @@
*max_slots = max_len;
max_len = 0;
- /* remain bug on condition */
- if (unlikely(!de->name_len))
- d->max = -1;
-
bit_pos += GET_DENTRY_SLOTS(le16_to_cpu(de->name_len));
}
@@ -389,9 +391,14 @@
return page;
if (S_ISDIR(inode->i_mode)) {
+ /* in order to handle error case */
+ get_page(page);
err = make_empty_dir(inode, dir, page);
- if (err)
- goto error;
+ if (err) {
+ lock_page(page);
+ goto put_error;
+ }
+ put_page(page);
}
err = f2fs_init_acl(inode, dir, page, dpage);
@@ -435,13 +442,12 @@
return page;
put_error:
- f2fs_put_page(page, 1);
-error:
- /* once the failed inode becomes a bad inode, i_mode is S_IFREG */
+ /* truncate empty dir pages */
truncate_inode_pages(&inode->i_data, 0);
- truncate_blocks(inode, 0, false);
- remove_dirty_inode(inode);
- remove_inode_page(inode);
+
+ clear_nlink(inode);
+ update_inode(inode, page);
+ f2fs_put_page(page, 1);
return ERR_PTR(err);
}
@@ -509,11 +515,7 @@
}
}
-/*
- * Caller should grab and release a rwsem by calling f2fs_lock_op() and
- * f2fs_unlock_op().
- */
-int __f2fs_add_link(struct inode *dir, const struct qstr *name,
+int f2fs_add_regular_entry(struct inode *dir, const struct qstr *new_name,
struct inode *inode, nid_t ino, umode_t mode)
{
unsigned int bit_pos;
@@ -526,28 +528,11 @@
struct f2fs_dentry_block *dentry_blk = NULL;
struct f2fs_dentry_ptr d;
struct page *page = NULL;
- struct fscrypt_name fname;
- struct qstr new_name;
- int slots, err;
-
- err = fscrypt_setup_filename(dir, name, 0, &fname);
- if (err)
- return err;
-
- new_name.name = fname_name(&fname);
- new_name.len = fname_len(&fname);
-
- if (f2fs_has_inline_dentry(dir)) {
- err = f2fs_add_inline_entry(dir, &new_name, inode, ino, mode);
- if (!err || err != -EAGAIN)
- goto out;
- else
- err = 0;
- }
+ int slots, err = 0;
level = 0;
- slots = GET_DENTRY_SLOTS(new_name.len);
- dentry_hash = f2fs_dentry_hash(&new_name);
+ slots = GET_DENTRY_SLOTS(new_name->len);
+ dentry_hash = f2fs_dentry_hash(new_name);
current_depth = F2FS_I(dir)->i_current_depth;
if (F2FS_I(dir)->chash == dentry_hash) {
@@ -556,10 +541,12 @@
}
start:
- if (unlikely(current_depth == MAX_DIR_HASH_DEPTH)) {
- err = -ENOSPC;
- goto out;
- }
+#ifdef CONFIG_F2FS_FAULT_INJECTION
+ if (time_to_inject(FAULT_DIR_DEPTH))
+ return -ENOSPC;
+#endif
+ if (unlikely(current_depth == MAX_DIR_HASH_DEPTH))
+ return -ENOSPC;
/* Increase the depth, if required */
if (level == current_depth)
@@ -573,10 +560,8 @@
for (block = bidx; block <= (bidx + nblock - 1); block++) {
dentry_page = get_new_data_page(dir, NULL, block, true);
- if (IS_ERR(dentry_page)) {
- err = PTR_ERR(dentry_page);
- goto out;
- }
+ if (IS_ERR(dentry_page))
+ return PTR_ERR(dentry_page);
dentry_blk = kmap(dentry_page);
bit_pos = room_for_filename(&dentry_blk->dentry_bitmap,
@@ -596,7 +581,7 @@
if (inode) {
down_write(&F2FS_I(inode)->i_sem);
- page = init_inode_metadata(inode, dir, &new_name, NULL);
+ page = init_inode_metadata(inode, dir, new_name, NULL);
if (IS_ERR(page)) {
err = PTR_ERR(page);
goto fail;
@@ -606,7 +591,7 @@
}
make_dentry_ptr(NULL, &d, (void *)dentry_blk, 1);
- f2fs_update_dentry(ino, mode, &d, &new_name, dentry_hash, bit_pos);
+ f2fs_update_dentry(ino, mode, &d, new_name, dentry_hash, bit_pos);
set_page_dirty(dentry_page);
@@ -628,7 +613,34 @@
}
kunmap(dentry_page);
f2fs_put_page(dentry_page, 1);
-out:
+
+ return err;
+}
+
+/*
+ * Caller should grab and release a rwsem by calling f2fs_lock_op() and
+ * f2fs_unlock_op().
+ */
+int __f2fs_add_link(struct inode *dir, const struct qstr *name,
+ struct inode *inode, nid_t ino, umode_t mode)
+{
+ struct fscrypt_name fname;
+ struct qstr new_name;
+ int err;
+
+ err = fscrypt_setup_filename(dir, name, 0, &fname);
+ if (err)
+ return err;
+
+ new_name.name = fname_name(&fname);
+ new_name.len = fname_len(&fname);
+
+ err = -EAGAIN;
+ if (f2fs_has_inline_dentry(dir))
+ err = f2fs_add_inline_entry(dir, &new_name, inode, ino, mode);
+ if (err == -EAGAIN)
+ err = f2fs_add_regular_entry(dir, &new_name, inode, ino, mode);
+
fscrypt_free_filename(&fname);
f2fs_update_time(F2FS_I_SB(dir), REQ_TIME);
return err;
@@ -792,10 +804,7 @@
continue;
}
- if (de->file_type < F2FS_FT_MAX)
- d_type = f2fs_filetype_table[de->file_type];
- else
- d_type = DT_UNKNOWN;
+ d_type = get_de_type(de);
de_name.name = d->filename[bit_pos];
de_name.len = le16_to_cpu(de->name_len);
@@ -804,7 +813,7 @@
int save_len = fstr->len;
int ret;
- de_name.name = kmalloc(de_name.len, GFP_NOFS);
+ de_name.name = f2fs_kmalloc(de_name.len, GFP_NOFS);
if (!de_name.name)
return false;
@@ -887,6 +896,7 @@
kunmap(dentry_page);
f2fs_put_page(dentry_page, 1);
}
+ err = 0;
out:
fscrypt_fname_free_buffer(&fstr);
return err;
diff --git a/fs/f2fs/extent_cache.c b/fs/f2fs/extent_cache.c
index c859bb0..5bfcdb9 100644
--- a/fs/f2fs/extent_cache.c
+++ b/fs/f2fs/extent_cache.c
@@ -196,8 +196,7 @@
if (!i_ext || !i_ext->len)
return false;
- set_extent_info(&ei, le32_to_cpu(i_ext->fofs),
- le32_to_cpu(i_ext->blk), le32_to_cpu(i_ext->len));
+ get_extent_info(&ei, i_ext);
write_lock(&et->lock);
if (atomic_read(&et->node_cnt))
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 7a4558d..916e7c2 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -37,6 +37,57 @@
} while (0)
#endif
+#ifdef CONFIG_F2FS_FAULT_INJECTION
+enum {
+ FAULT_KMALLOC,
+ FAULT_PAGE_ALLOC,
+ FAULT_ALLOC_NID,
+ FAULT_ORPHAN,
+ FAULT_BLOCK,
+ FAULT_DIR_DEPTH,
+ FAULT_MAX,
+};
+
+struct f2fs_fault_info {
+ atomic_t inject_ops;
+ unsigned int inject_rate;
+ unsigned int inject_type;
+};
+
+extern struct f2fs_fault_info f2fs_fault;
+extern char *fault_name[FAULT_MAX];
+#define IS_FAULT_SET(type) (f2fs_fault.inject_type & (1 << (type)))
+
+static inline bool time_to_inject(int type)
+{
+ if (!f2fs_fault.inject_rate)
+ return false;
+ if (type == FAULT_KMALLOC && !IS_FAULT_SET(type))
+ return false;
+ else if (type == FAULT_PAGE_ALLOC && !IS_FAULT_SET(type))
+ return false;
+ else if (type == FAULT_ALLOC_NID && !IS_FAULT_SET(type))
+ return false;
+ else if (type == FAULT_ORPHAN && !IS_FAULT_SET(type))
+ return false;
+ else if (type == FAULT_BLOCK && !IS_FAULT_SET(type))
+ return false;
+ else if (type == FAULT_DIR_DEPTH && !IS_FAULT_SET(type))
+ return false;
+
+ atomic_inc(&f2fs_fault.inject_ops);
+ if (atomic_read(&f2fs_fault.inject_ops) >= f2fs_fault.inject_rate) {
+ atomic_set(&f2fs_fault.inject_ops, 0);
+ printk("%sF2FS-fs : inject %s in %pF\n",
+ KERN_INFO,
+ fault_name[type],
+ __builtin_return_address(0));
+ return true;
+ }
+ return false;
+}
+#endif
+
/*
* For mount options
*/
@@ -56,6 +107,7 @@
#define F2FS_MOUNT_EXTENT_CACHE 0x00002000
#define F2FS_MOUNT_FORCE_FG_GC 0x00004000
#define F2FS_MOUNT_DATA_FLUSH 0x00008000
+#define F2FS_MOUNT_FAULT_INJECTION 0x00010000
#define clear_opt(sbi, option) (sbi->mount_opt.opt &= ~F2FS_MOUNT_##option)
#define set_opt(sbi, option) (sbi->mount_opt.opt |= F2FS_MOUNT_##option)
@@ -159,7 +211,6 @@
struct inode *inode; /* vfs inode pointer */
block_t blkaddr; /* block address locating the last fsync */
block_t last_dentry; /* block address locating the last dentry */
- block_t last_inode; /* block address locating the last inode */
};
#define nats_in_cursum(jnl) (le16_to_cpu(jnl->n_nats))
@@ -385,7 +436,7 @@
/* Use below internally in f2fs*/
unsigned long flags; /* use to pass per-file flags */
struct rw_semaphore i_sem; /* protect fi info */
- atomic_t dirty_pages; /* # of dirty pages */
+ struct percpu_counter dirty_pages; /* # of dirty pages */
f2fs_hash_t chash; /* hash value of given file name */
unsigned int clevel; /* maximum level of given file name */
nid_t i_xattr_nid; /* node id that contains xattrs */
@@ -398,11 +449,11 @@
};
static inline void get_extent_info(struct extent_info *ext,
- struct f2fs_extent i_ext)
+ struct f2fs_extent *i_ext)
{
- ext->fofs = le32_to_cpu(i_ext.fofs);
- ext->blk = le32_to_cpu(i_ext.blk);
- ext->len = le32_to_cpu(i_ext.len);
+ ext->fofs = le32_to_cpu(i_ext->fofs);
+ ext->blk = le32_to_cpu(i_ext->blk);
+ ext->len = le32_to_cpu(i_ext->len);
}
static inline void set_raw_extent(struct extent_info *ext,
@@ -599,7 +650,6 @@
* dirty dentry blocks, dirty node blocks, and dirty meta blocks.
*/
enum count_type {
- F2FS_WRITEBACK,
F2FS_DIRTY_DENTS,
F2FS_DIRTY_DATA,
F2FS_DIRTY_NODES,
@@ -672,6 +722,7 @@
SBI_IS_CLOSE, /* specify unmounting */
SBI_NEED_FSCK, /* need fsck.f2fs to fix */
SBI_POR_DOING, /* recovery is doing or not */
+ SBI_NEED_SB_WRITE, /* need to recover superblock */
};
enum {
@@ -680,6 +731,10 @@
MAX_TIME,
};
+#ifdef CONFIG_F2FS_FS_ENCRYPTION
+#define F2FS_KEY_DESC_PREFIX "f2fs:"
+#define F2FS_KEY_DESC_PREFIX_SIZE 5
+#endif
struct f2fs_sb_info {
struct super_block *sb; /* pointer to VFS super block */
struct proc_dir_entry *s_proc; /* proc entry */
@@ -687,6 +742,10 @@
int valid_super_block; /* valid super block no */
int s_flag; /* flags for sbi */
+#ifdef CONFIG_F2FS_FS_ENCRYPTION
+ u8 key_prefix[F2FS_KEY_DESC_PREFIX_SIZE];
+ u8 key_prefix_size;
+#endif
/* for node-related operations */
struct f2fs_nm_info *nm_info; /* node manager */
struct inode *node_inode; /* cache node blocks */
@@ -742,18 +801,24 @@
unsigned int total_sections; /* total section count */
unsigned int total_node_count; /* total node block count */
unsigned int total_valid_node_count; /* valid node block count */
- unsigned int total_valid_inode_count; /* valid inode count */
loff_t max_file_blocks; /* max block index of file */
int active_logs; /* # of active logs */
int dir_level; /* directory level */
block_t user_block_count; /* # of user blocks */
block_t total_valid_block_count; /* # of valid blocks */
- block_t alloc_valid_block_count; /* # of allocated blocks */
block_t discard_blks; /* discard command candidats */
block_t last_valid_block_count; /* for recovery */
u32 s_next_generation; /* for NFS support */
- atomic_t nr_pages[NR_COUNT_TYPE]; /* # of pages, see count_type */
+ atomic_t nr_wb_bios; /* # of writeback bios */
+
+ /* # of pages, see count_type */
+ struct percpu_counter nr_pages[NR_COUNT_TYPE];
+ /* # of allocated blocks */
+ struct percpu_counter alloc_valid_block_count;
+
+ /* valid inode count */
+ struct percpu_counter total_valid_inode_count;
struct f2fs_mount_info mount_opt; /* mount options */
@@ -1055,21 +1120,33 @@
}
static inline bool inc_valid_block_count(struct f2fs_sb_info *sbi,
- struct inode *inode, blkcnt_t count)
+ struct inode *inode, blkcnt_t *count)
{
block_t valid_block_count;
spin_lock(&sbi->stat_lock);
- valid_block_count =
- sbi->total_valid_block_count + (block_t)count;
- if (unlikely(valid_block_count > sbi->user_block_count)) {
+#ifdef CONFIG_F2FS_FAULT_INJECTION
+ if (time_to_inject(FAULT_BLOCK)) {
spin_unlock(&sbi->stat_lock);
return false;
}
- inode->i_blocks += count;
- sbi->total_valid_block_count = valid_block_count;
- sbi->alloc_valid_block_count += (block_t)count;
+#endif
+ valid_block_count =
+ sbi->total_valid_block_count + (block_t)(*count);
+ if (unlikely(valid_block_count > sbi->user_block_count)) {
+ *count = sbi->user_block_count - sbi->total_valid_block_count;
+ if (!*count) {
+ spin_unlock(&sbi->stat_lock);
+ return false;
+ }
+ }
+ /* *count can be recalculated */
+ inode->i_blocks += *count;
+ sbi->total_valid_block_count =
+ sbi->total_valid_block_count + (block_t)(*count);
spin_unlock(&sbi->stat_lock);
+
+ percpu_counter_add(&sbi->alloc_valid_block_count, (*count));
return true;
}
@@ -1087,20 +1164,20 @@
static inline void inc_page_count(struct f2fs_sb_info *sbi, int count_type)
{
- atomic_inc(&sbi->nr_pages[count_type]);
+ percpu_counter_inc(&sbi->nr_pages[count_type]);
set_sbi_flag(sbi, SBI_IS_DIRTY);
}
static inline void inode_inc_dirty_pages(struct inode *inode)
{
- atomic_inc(&F2FS_I(inode)->dirty_pages);
+ percpu_counter_inc(&F2FS_I(inode)->dirty_pages);
inc_page_count(F2FS_I_SB(inode), S_ISDIR(inode->i_mode) ?
F2FS_DIRTY_DENTS : F2FS_DIRTY_DATA);
}
static inline void dec_page_count(struct f2fs_sb_info *sbi, int count_type)
{
- atomic_dec(&sbi->nr_pages[count_type]);
+ percpu_counter_dec(&sbi->nr_pages[count_type]);
}
static inline void inode_dec_dirty_pages(struct inode *inode)
@@ -1109,26 +1186,28 @@
!S_ISLNK(inode->i_mode))
return;
- atomic_dec(&F2FS_I(inode)->dirty_pages);
+ percpu_counter_dec(&F2FS_I(inode)->dirty_pages);
dec_page_count(F2FS_I_SB(inode), S_ISDIR(inode->i_mode) ?
F2FS_DIRTY_DENTS : F2FS_DIRTY_DATA);
}
-static inline int get_pages(struct f2fs_sb_info *sbi, int count_type)
+static inline s64 get_pages(struct f2fs_sb_info *sbi, int count_type)
{
- return atomic_read(&sbi->nr_pages[count_type]);
+ return percpu_counter_sum_positive(&sbi->nr_pages[count_type]);
}
-static inline int get_dirty_pages(struct inode *inode)
+static inline s64 get_dirty_pages(struct inode *inode)
{
- return atomic_read(&F2FS_I(inode)->dirty_pages);
+ return percpu_counter_sum_positive(&F2FS_I(inode)->dirty_pages);
}
static inline int get_blocktype_secs(struct f2fs_sb_info *sbi, int block_type)
{
unsigned int pages_per_sec = sbi->segs_per_sec * sbi->blocks_per_seg;
- return ((get_pages(sbi, block_type) + pages_per_sec - 1)
- >> sbi->log_blocks_per_seg) / sbi->segs_per_sec;
+ unsigned int segs = (get_pages(sbi, block_type) + pages_per_sec - 1) >>
+ sbi->log_blocks_per_seg;
+
+ return segs / sbi->segs_per_sec;
}
static inline block_t valid_user_blocks(struct f2fs_sb_info *sbi)
@@ -1217,11 +1296,11 @@
if (inode)
inode->i_blocks++;
- sbi->alloc_valid_block_count++;
sbi->total_valid_node_count++;
sbi->total_valid_block_count++;
spin_unlock(&sbi->stat_lock);
+ percpu_counter_inc(&sbi->alloc_valid_block_count);
return true;
}
@@ -1248,28 +1327,30 @@
static inline void inc_valid_inode_count(struct f2fs_sb_info *sbi)
{
- spin_lock(&sbi->stat_lock);
- f2fs_bug_on(sbi, sbi->total_valid_inode_count == sbi->total_node_count);
- sbi->total_valid_inode_count++;
- spin_unlock(&sbi->stat_lock);
+ percpu_counter_inc(&sbi->total_valid_inode_count);
}
static inline void dec_valid_inode_count(struct f2fs_sb_info *sbi)
{
- spin_lock(&sbi->stat_lock);
- f2fs_bug_on(sbi, !sbi->total_valid_inode_count);
- sbi->total_valid_inode_count--;
- spin_unlock(&sbi->stat_lock);
+ percpu_counter_dec(&sbi->total_valid_inode_count);
}
-static inline unsigned int valid_inode_count(struct f2fs_sb_info *sbi)
+static inline s64 valid_inode_count(struct f2fs_sb_info *sbi)
{
- return sbi->total_valid_inode_count;
+ return percpu_counter_sum_positive(&sbi->total_valid_inode_count);
}
static inline struct page *f2fs_grab_cache_page(struct address_space *mapping,
pgoff_t index, bool for_write)
{
+#ifdef CONFIG_F2FS_FAULT_INJECTION
+ struct page *page = find_lock_page(mapping, index);
+ if (page)
+ return page;
+
+ if (time_to_inject(FAULT_PAGE_ALLOC))
+ return NULL;
+#endif
if (!for_write)
return grab_cache_page(mapping, index);
return grab_cache_page_write_begin(mapping, index, AOP_FLAG_NOFS);
@@ -1435,7 +1516,6 @@
FI_NO_ALLOC, /* should not allocate any blocks */
FI_FREE_NID, /* free allocated nide */
FI_UPDATE_DIR, /* should update inode block for consistency */
- FI_DELAY_IPUT, /* used for the recovery */
FI_NO_EXTENT, /* not to use the extent cache */
FI_INLINE_XATTR, /* used for inline xattr */
FI_INLINE_DATA, /* used for inline data*/
@@ -1618,12 +1698,6 @@
return is_set_ckpt_flags(sbi->ckpt, CP_ERROR_FLAG);
}
-static inline void f2fs_stop_checkpoint(struct f2fs_sb_info *sbi)
-{
- set_ckpt_flags(sbi->ckpt, CP_ERROR_FLAG);
- sbi->sb->s_flags |= MS_RDONLY;
-}
-
static inline bool is_dot_dotdot(const struct qstr *str)
{
if (str->len == 1 && str->name[0] == '.')
@@ -1644,6 +1718,15 @@
return S_ISREG(inode->i_mode);
}
+static inline void *f2fs_kmalloc(size_t size, gfp_t flags)
+{
+#ifdef CONFIG_F2FS_FAULT_INJECTION
+ if (time_to_inject(FAULT_KMALLOC))
+ return NULL;
+#endif
+ return kmalloc(size, flags);
+}
+
static inline void *f2fs_kvmalloc(size_t size, gfp_t flags)
{
void *ret;
@@ -1710,7 +1793,7 @@
*/
extern unsigned char f2fs_filetype_table[F2FS_FT_MAX];
void set_de_type(struct f2fs_dir_entry *, umode_t);
-
+unsigned char get_de_type(struct f2fs_dir_entry *);
struct f2fs_dir_entry *find_target_dentry(struct fscrypt_name *,
f2fs_hash_t, int *, struct f2fs_dentry_ptr *);
bool f2fs_fill_dentries(struct dir_context *, struct f2fs_dentry_ptr *,
@@ -1731,6 +1814,8 @@
int update_dent_inode(struct inode *, struct inode *, const struct qstr *);
void f2fs_update_dentry(nid_t ino, umode_t mode, struct f2fs_dentry_ptr *,
const struct qstr *, f2fs_hash_t , unsigned int);
+int f2fs_add_regular_entry(struct inode *, const struct qstr *,
+ struct inode *, nid_t, umode_t);
int __f2fs_add_link(struct inode *, const struct qstr *, struct inode *, nid_t,
umode_t);
void f2fs_delete_entry(struct f2fs_dir_entry *, struct page *, struct inode *,
@@ -1781,7 +1866,10 @@
struct page *get_node_page(struct f2fs_sb_info *, pgoff_t);
struct page *get_node_page_ra(struct page *, int);
void sync_inode_page(struct dnode_of_data *);
-int sync_node_pages(struct f2fs_sb_info *, nid_t, struct writeback_control *);
+void move_node_page(struct page *, int);
+int fsync_node_pages(struct f2fs_sb_info *, nid_t, struct writeback_control *,
+ bool);
+int sync_node_pages(struct f2fs_sb_info *, struct writeback_control *);
bool alloc_nid(struct f2fs_sb_info *, nid_t *);
void alloc_nid_done(struct f2fs_sb_info *, nid_t);
void alloc_nid_failed(struct f2fs_sb_info *, nid_t);
@@ -1843,6 +1931,7 @@
/*
* checkpoint.c
*/
+void f2fs_stop_checkpoint(struct f2fs_sb_info *, bool);
struct page *grab_meta_page(struct f2fs_sb_info *, pgoff_t);
struct page *get_meta_page(struct f2fs_sb_info *, pgoff_t);
struct page *get_tmp_page(struct f2fs_sb_info *, pgoff_t);
@@ -1852,7 +1941,7 @@
long sync_meta_pages(struct f2fs_sb_info *, enum page_type, long);
void add_ino_entry(struct f2fs_sb_info *, nid_t, int type);
void remove_ino_entry(struct f2fs_sb_info *, nid_t, int type);
-void release_ino_entry(struct f2fs_sb_info *);
+void release_ino_entry(struct f2fs_sb_info *, bool);
bool exist_written_data(struct f2fs_sb_info *, nid_t, int);
int acquire_orphan_inode(struct f2fs_sb_info *);
void release_orphan_inode(struct f2fs_sb_info *);
@@ -1861,7 +1950,6 @@
int recover_orphan_inodes(struct f2fs_sb_info *);
int get_valid_checkpoint(struct f2fs_sb_info *);
void update_dirty_page(struct inode *, struct page *);
-void add_dirty_dir_inode(struct inode *);
void remove_dirty_inode(struct inode *);
int sync_dirty_inodes(struct f2fs_sb_info *, enum inode_type);
int write_checkpoint(struct f2fs_sb_info *, struct cp_control *);
@@ -1880,6 +1968,7 @@
void f2fs_submit_page_mbio(struct f2fs_io_info *);
void set_data_blkaddr(struct dnode_of_data *);
void f2fs_update_data_blkaddr(struct dnode_of_data *, block_t);
+int reserve_new_blocks(struct dnode_of_data *, blkcnt_t);
int reserve_new_block(struct dnode_of_data *);
int f2fs_get_block(struct dnode_of_data *, pgoff_t);
ssize_t f2fs_preallocate_blocks(struct kiocb *, struct iov_iter *);
@@ -1906,7 +1995,7 @@
/*
* recovery.c
*/
-int recover_fsync_data(struct f2fs_sb_info *);
+int recover_fsync_data(struct f2fs_sb_info *, bool);
bool space_for_roll_forward(struct f2fs_sb_info *);
/*
@@ -1921,12 +2010,12 @@
unsigned long long hit_largest, hit_cached, hit_rbtree;
unsigned long long hit_total, total_ext;
int ext_tree, zombie_tree, ext_node;
- int ndirty_node, ndirty_meta;
- int ndirty_dent, ndirty_dirs, ndirty_data, ndirty_files;
+ s64 ndirty_node, ndirty_dent, ndirty_meta, ndirty_data, inmem_pages;
+ unsigned int ndirty_dirs, ndirty_files;
int nats, dirty_nats, sits, dirty_sits, fnids;
int total_count, utilization;
- int bg_gc, inmem_pages, wb_pages;
- int inline_xattr, inline_inode, inline_dir;
+ int bg_gc, wb_bios;
+ int inline_xattr, inline_inode, inline_dir, orphans;
unsigned int valid_count, valid_node_count, valid_inode_count;
unsigned int bimodal, avg_vblocks;
int util_free, util_valid, util_invalid;
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index eb9d027..f4c0086 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -20,7 +20,7 @@
#include <linux/uaccess.h>
#include <linux/mount.h>
#include <linux/pagevec.h>
-#include <linux/random.h>
+#include <linux/uuid.h>
#include "f2fs.h"
#include "node.h"
@@ -182,7 +182,8 @@
}
}
-int f2fs_sync_file(struct file *file, loff_t start, loff_t end, int datasync)
+static int f2fs_do_sync_file(struct file *file, loff_t start, loff_t end,
+ int datasync, bool atomic)
{
struct inode *inode = file->f_mapping->host;
struct f2fs_inode_info *fi = F2FS_I(inode);
@@ -256,7 +257,9 @@
goto out;
}
sync_nodes:
- sync_node_pages(sbi, ino, &wbc);
+ ret = fsync_node_pages(sbi, ino, &wbc, atomic);
+ if (ret)
+ goto out;
/* if cp_error was enabled, we should avoid infinite loop */
if (unlikely(f2fs_cp_error(sbi))) {
@@ -288,6 +291,11 @@
return ret;
}
+int f2fs_sync_file(struct file *file, loff_t start, loff_t end, int datasync)
+{
+ return f2fs_do_sync_file(file, start, end, datasync, false);
+}
+
static pgoff_t __get_first_dirty_index(struct address_space *mapping,
pgoff_t pgofs, int whence)
{
@@ -555,6 +563,9 @@
free_from = (pgoff_t)F2FS_BYTES_TO_BLK(from + blocksize - 1);
+ if (free_from >= sbi->max_file_blocks)
+ goto free_partial;
+
if (lock)
f2fs_lock_op(sbi);
@@ -573,7 +584,7 @@
}
set_new_dnode(&dn, inode, ipage, NULL, 0);
- err = get_dnode_of_data(&dn, free_from, LOOKUP_NODE);
+ err = get_dnode_of_data(&dn, free_from, LOOKUP_NODE_RA);
if (err) {
if (err == -ENOENT)
goto free_next;
@@ -596,7 +607,7 @@
out:
if (lock)
f2fs_unlock_op(sbi);
-
+free_partial:
/* lastly zero out the first data page */
if (!err)
err = truncate_partial_data_page(inode, from, truncate_page);
@@ -986,6 +997,49 @@
return ret;
}
+static int f2fs_do_zero_range(struct dnode_of_data *dn, pgoff_t start,
+ pgoff_t end)
+{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(dn->inode);
+ pgoff_t index = start;
+ unsigned int ofs_in_node = dn->ofs_in_node;
+ blkcnt_t count = 0;
+ int ret;
+
+ for (; index < end; index++, dn->ofs_in_node++) {
+ if (datablock_addr(dn->node_page, dn->ofs_in_node) == NULL_ADDR)
+ count++;
+ }
+
+ dn->ofs_in_node = ofs_in_node;
+ ret = reserve_new_blocks(dn, count);
+ if (ret)
+ return ret;
+
+ dn->ofs_in_node = ofs_in_node;
+ for (index = start; index < end; index++, dn->ofs_in_node++) {
+ dn->data_blkaddr =
+ datablock_addr(dn->node_page, dn->ofs_in_node);
+ /*
+ * reserve_new_blocks will not guarantee entire block
+ * allocation.
+ */
+ if (dn->data_blkaddr == NULL_ADDR) {
+ ret = -ENOSPC;
+ break;
+ }
+ if (dn->data_blkaddr != NEW_ADDR) {
+ invalidate_blocks(sbi, dn->data_blkaddr);
+ dn->data_blkaddr = NEW_ADDR;
+ set_data_blkaddr(dn);
+ }
+ }
+
+ f2fs_update_extent_cache_range(dn, start, 0, index - start);
+
+ return ret;
+}
+
static int f2fs_zero_range(struct inode *inode, loff_t offset, loff_t len,
int mode)
{
@@ -1036,35 +1090,32 @@
(loff_t)pg_start << PAGE_SHIFT);
}
- for (index = pg_start; index < pg_end; index++) {
+ for (index = pg_start; index < pg_end;) {
struct dnode_of_data dn;
- struct page *ipage;
+ unsigned int end_offset;
+ pgoff_t end;
f2fs_lock_op(sbi);
- ipage = get_node_page(sbi, inode->i_ino);
- if (IS_ERR(ipage)) {
- ret = PTR_ERR(ipage);
- f2fs_unlock_op(sbi);
- goto out;
- }
-
- set_new_dnode(&dn, inode, ipage, NULL, 0);
- ret = f2fs_reserve_block(&dn, index);
+ set_new_dnode(&dn, inode, NULL, NULL, 0);
+ ret = get_dnode_of_data(&dn, index, ALLOC_NODE);
if (ret) {
f2fs_unlock_op(sbi);
goto out;
}
- if (dn.data_blkaddr != NEW_ADDR) {
- invalidate_blocks(sbi, dn.data_blkaddr);
- f2fs_update_data_blkaddr(&dn, NEW_ADDR);
- }
+ end_offset = ADDRS_PER_PAGE(dn.node_page, inode);
+ end = min(pg_end, end_offset - dn.ofs_in_node + index);
+
+ ret = f2fs_do_zero_range(&dn, index, end);
f2fs_put_dnode(&dn);
f2fs_unlock_op(sbi);
+ if (ret)
+ goto out;
+ index = end;
new_size = max_t(loff_t, new_size,
- (loff_t)(index + 1) << PAGE_SHIFT);
+ (loff_t)index << PAGE_SHIFT);
}
if (off_end) {
@@ -1147,10 +1198,11 @@
loff_t len, int mode)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
- pgoff_t index, pg_start, pg_end;
+ struct f2fs_map_blocks map = { .m_next_pgofs = NULL };
+ pgoff_t pg_end;
loff_t new_size = i_size_read(inode);
- loff_t off_start, off_end;
- int ret = 0;
+ loff_t off_end;
+ int ret;
ret = inode_newsize_ok(inode, (len + offset));
if (ret)
@@ -1162,43 +1214,35 @@
f2fs_balance_fs(sbi, true);
- pg_start = ((unsigned long long) offset) >> PAGE_SHIFT;
- pg_end = ((unsigned long long) offset + len) >> PAGE_SHIFT;
-
- off_start = offset & (PAGE_SIZE - 1);
+ pg_end = ((unsigned long long)offset + len) >> PAGE_SHIFT;
off_end = (offset + len) & (PAGE_SIZE - 1);
- f2fs_lock_op(sbi);
+ map.m_lblk = ((unsigned long long)offset) >> PAGE_SHIFT;
+ map.m_len = pg_end - map.m_lblk;
+ if (off_end)
+ map.m_len++;
- for (index = pg_start; index <= pg_end; index++) {
- struct dnode_of_data dn;
+ ret = f2fs_map_blocks(inode, &map, 1, F2FS_GET_BLOCK_PRE_AIO);
+ if (ret) {
+ pgoff_t last_off;
- if (index == pg_end && !off_end)
- goto noalloc;
+ if (!map.m_len)
+ return ret;
- set_new_dnode(&dn, inode, NULL, NULL, 0);
- ret = f2fs_reserve_block(&dn, index);
- if (ret)
- break;
-noalloc:
- if (pg_start == pg_end)
- new_size = offset + len;
- else if (index == pg_start && off_start)
- new_size = (loff_t)(index + 1) << PAGE_SHIFT;
- else if (index == pg_end)
- new_size = ((loff_t)index << PAGE_SHIFT) +
- off_end;
- else
- new_size += PAGE_SIZE;
+ last_off = map.m_lblk + map.m_len - 1;
+
+ /* update new size to the failed position */
+ new_size = (last_off == pg_end) ? offset + len:
+ (loff_t)(last_off + 1) << PAGE_SHIFT;
+ } else {
+ new_size = ((loff_t)pg_end << PAGE_SHIFT) + off_end;
}
- if (!(mode & FALLOC_FL_KEEP_SIZE) &&
- i_size_read(inode) < new_size) {
+ if (!(mode & FALLOC_FL_KEEP_SIZE) && i_size_read(inode) < new_size) {
i_size_write(inode, new_size);
mark_inode_dirty(inode);
update_inode_page(inode);
}
- f2fs_unlock_op(sbi);
return ret;
}
@@ -1254,10 +1298,19 @@
static int f2fs_release_file(struct inode *inode, struct file *filp)
{
+ /*
+ * f2fs_relase_file is called at every close calls. So we should
+ * not drop any inmemory pages by close called by other process.
+ */
+ if (!(filp->f_mode & FMODE_WRITE) ||
+ atomic_read(&inode->i_writecount) != 1)
+ return 0;
+
/* some remained atomic pages should discarded */
if (f2fs_is_atomic_file(inode))
drop_inmem_pages(inode);
if (f2fs_is_volatile_file(inode)) {
+ clear_inode_flag(F2FS_I(inode), FI_VOLATILE_FILE);
set_inode_flag(F2FS_I(inode), FI_DROP_CACHE);
filemap_fdatawrite(inode->i_mapping);
clear_inode_flag(F2FS_I(inode), FI_DROP_CACHE);
@@ -1294,20 +1347,16 @@
unsigned int oldflags;
int ret;
+ if (!inode_owner_or_capable(inode))
+ return -EACCES;
+
+ if (get_user(flags, (int __user *)arg))
+ return -EFAULT;
+
ret = mnt_want_write_file(filp);
if (ret)
return ret;
- if (!inode_owner_or_capable(inode)) {
- ret = -EACCES;
- goto out;
- }
-
- if (get_user(flags, (int __user *)arg)) {
- ret = -EFAULT;
- goto out;
- }
-
flags = f2fs_mask_flags(inode->i_mode, flags);
inode_lock(inode);
@@ -1350,17 +1399,35 @@
if (!inode_owner_or_capable(inode))
return -EACCES;
+ ret = mnt_want_write_file(filp);
+ if (ret)
+ return ret;
+
+ inode_lock(inode);
+
if (f2fs_is_atomic_file(inode))
- return 0;
+ goto out;
ret = f2fs_convert_inline_inode(inode);
if (ret)
- return ret;
+ goto out;
set_inode_flag(F2FS_I(inode), FI_ATOMIC_FILE);
f2fs_update_time(F2FS_I_SB(inode), REQ_TIME);
- return 0;
+ if (!get_dirty_pages(inode))
+ goto out;
+
+ f2fs_msg(F2FS_I_SB(inode)->sb, KERN_WARNING,
+ "Unexpected flush for atomic writes: ino=%lu, npages=%lld",
+ inode->i_ino, get_dirty_pages(inode));
+ ret = filemap_write_and_wait_range(inode->i_mapping, 0, LLONG_MAX);
+ if (ret)
+ clear_inode_flag(F2FS_I(inode), FI_ATOMIC_FILE);
+out:
+ inode_unlock(inode);
+ mnt_drop_write_file(filp);
+ return ret;
}
static int f2fs_ioc_commit_atomic_write(struct file *filp)
@@ -1371,13 +1438,15 @@
if (!inode_owner_or_capable(inode))
return -EACCES;
- if (f2fs_is_volatile_file(inode))
- return 0;
-
ret = mnt_want_write_file(filp);
if (ret)
return ret;
+ inode_lock(inode);
+
+ if (f2fs_is_volatile_file(inode))
+ goto err_out;
+
if (f2fs_is_atomic_file(inode)) {
clear_inode_flag(F2FS_I(inode), FI_ATOMIC_FILE);
ret = commit_inmem_pages(inode);
@@ -1387,8 +1456,9 @@
}
}
- ret = f2fs_sync_file(filp, 0, LLONG_MAX, 0);
+ ret = f2fs_do_sync_file(filp, 0, LLONG_MAX, 0, true);
err_out:
+ inode_unlock(inode);
mnt_drop_write_file(filp);
return ret;
}
@@ -1401,32 +1471,54 @@
if (!inode_owner_or_capable(inode))
return -EACCES;
- if (f2fs_is_volatile_file(inode))
- return 0;
-
- ret = f2fs_convert_inline_inode(inode);
+ ret = mnt_want_write_file(filp);
if (ret)
return ret;
+ inode_lock(inode);
+
+ if (f2fs_is_volatile_file(inode))
+ goto out;
+
+ ret = f2fs_convert_inline_inode(inode);
+ if (ret)
+ goto out;
+
set_inode_flag(F2FS_I(inode), FI_VOLATILE_FILE);
f2fs_update_time(F2FS_I_SB(inode), REQ_TIME);
- return 0;
+out:
+ inode_unlock(inode);
+ mnt_drop_write_file(filp);
+ return ret;
}
static int f2fs_ioc_release_volatile_write(struct file *filp)
{
struct inode *inode = file_inode(filp);
+ int ret;
if (!inode_owner_or_capable(inode))
return -EACCES;
+ ret = mnt_want_write_file(filp);
+ if (ret)
+ return ret;
+
+ inode_lock(inode);
+
if (!f2fs_is_volatile_file(inode))
- return 0;
+ goto out;
- if (!f2fs_is_first_block_written(inode))
- return truncate_partial_data_page(inode, 0, true);
+ if (!f2fs_is_first_block_written(inode)) {
+ ret = truncate_partial_data_page(inode, 0, true);
+ goto out;
+ }
- return punch_hole(inode, 0, F2FS_BLKSIZE);
+ ret = punch_hole(inode, 0, F2FS_BLKSIZE);
+out:
+ inode_unlock(inode);
+ mnt_drop_write_file(filp);
+ return ret;
}
static int f2fs_ioc_abort_volatile_write(struct file *filp)
@@ -1441,15 +1533,17 @@
if (ret)
return ret;
- if (f2fs_is_atomic_file(inode)) {
- clear_inode_flag(F2FS_I(inode), FI_ATOMIC_FILE);
+ inode_lock(inode);
+
+ if (f2fs_is_atomic_file(inode))
drop_inmem_pages(inode);
- }
if (f2fs_is_volatile_file(inode)) {
clear_inode_flag(F2FS_I(inode), FI_VOLATILE_FILE);
- ret = f2fs_sync_file(filp, 0, LLONG_MAX, 0);
+ ret = f2fs_do_sync_file(filp, 0, LLONG_MAX, 0, true);
}
+ inode_unlock(inode);
+
mnt_drop_write_file(filp);
f2fs_update_time(F2FS_I_SB(inode), REQ_TIME);
return ret;
@@ -1461,6 +1555,7 @@
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct super_block *sb = sbi->sb;
__u32 in;
+ int ret;
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
@@ -1468,31 +1563,38 @@
if (get_user(in, (__u32 __user *)arg))
return -EFAULT;
+ ret = mnt_want_write_file(filp);
+ if (ret)
+ return ret;
+
switch (in) {
case F2FS_GOING_DOWN_FULLSYNC:
sb = freeze_bdev(sb->s_bdev);
if (sb && !IS_ERR(sb)) {
- f2fs_stop_checkpoint(sbi);
+ f2fs_stop_checkpoint(sbi, false);
thaw_bdev(sb->s_bdev, sb);
}
break;
case F2FS_GOING_DOWN_METASYNC:
/* do checkpoint only */
f2fs_sync_fs(sb, 1);
- f2fs_stop_checkpoint(sbi);
+ f2fs_stop_checkpoint(sbi, false);
break;
case F2FS_GOING_DOWN_NOSYNC:
- f2fs_stop_checkpoint(sbi);
+ f2fs_stop_checkpoint(sbi, false);
break;
case F2FS_GOING_DOWN_METAFLUSH:
sync_meta_pages(sbi, META, LONG_MAX);
- f2fs_stop_checkpoint(sbi);
+ f2fs_stop_checkpoint(sbi, false);
break;
default:
- return -EINVAL;
+ ret = -EINVAL;
+ goto out;
}
f2fs_update_time(sbi, REQ_TIME);
- return 0;
+out:
+ mnt_drop_write_file(filp);
+ return ret;
}
static int f2fs_ioc_fitrim(struct file *filp, unsigned long arg)
@@ -1513,9 +1615,14 @@
sizeof(range)))
return -EFAULT;
+ ret = mnt_want_write_file(filp);
+ if (ret)
+ return ret;
+
range.minlen = max((unsigned int)range.minlen,
q->limits.discard_granularity);
ret = f2fs_trim_fs(F2FS_SB(sb), &range);
+ mnt_drop_write_file(filp);
if (ret < 0)
return ret;
@@ -1540,13 +1647,21 @@
{
struct fscrypt_policy policy;
struct inode *inode = file_inode(filp);
+ int ret;
if (copy_from_user(&policy, (struct fscrypt_policy __user *)arg,
sizeof(policy)))
return -EFAULT;
+ ret = mnt_want_write_file(filp);
+ if (ret)
+ return ret;
+
f2fs_update_time(F2FS_I_SB(inode), REQ_TIME);
- return fscrypt_process_policy(inode, &policy);
+ ret = fscrypt_process_policy(inode, &policy);
+
+ mnt_drop_write_file(filp);
+ return ret;
}
static int f2fs_ioc_get_encryption_policy(struct file *filp, unsigned long arg)
@@ -1603,6 +1718,7 @@
struct inode *inode = file_inode(filp);
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
__u32 sync;
+ int ret;
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
@@ -1613,20 +1729,30 @@
if (f2fs_readonly(sbi->sb))
return -EROFS;
+ ret = mnt_want_write_file(filp);
+ if (ret)
+ return ret;
+
if (!sync) {
- if (!mutex_trylock(&sbi->gc_mutex))
- return -EBUSY;
+ if (!mutex_trylock(&sbi->gc_mutex)) {
+ ret = -EBUSY;
+ goto out;
+ }
} else {
mutex_lock(&sbi->gc_mutex);
}
- return f2fs_gc(sbi, sync);
+ ret = f2fs_gc(sbi, sync);
+out:
+ mnt_drop_write_file(filp);
+ return ret;
}
static int f2fs_ioc_write_checkpoint(struct file *filp, unsigned long arg)
{
struct inode *inode = file_inode(filp);
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
+ int ret;
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
@@ -1634,7 +1760,14 @@
if (f2fs_readonly(sbi->sb))
return -EROFS;
- return f2fs_sync_fs(sbi->sb, 1);
+ ret = mnt_want_write_file(filp);
+ if (ret)
+ return ret;
+
+ ret = f2fs_sync_fs(sbi->sb, 1);
+
+ mnt_drop_write_file(filp);
+ return ret;
}
static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c
index b0051a9..38d56f6 100644
--- a/fs/f2fs/gc.c
+++ b/fs/f2fs/gc.c
@@ -96,7 +96,7 @@
dev_t dev = sbi->sb->s_bdev->bd_dev;
int err = 0;
- gc_th = kmalloc(sizeof(struct f2fs_gc_kthread), GFP_KERNEL);
+ gc_th = f2fs_kmalloc(sizeof(struct f2fs_gc_kthread), GFP_KERNEL);
if (!gc_th) {
err = -ENOMEM;
goto out;
@@ -465,15 +465,7 @@
continue;
}
- /* set page dirty and write it */
- if (gc_type == FG_GC) {
- f2fs_wait_on_page_writeback(node_page, NODE, true);
- set_page_dirty(node_page);
- } else {
- if (!PageWriteback(node_page))
- set_page_dirty(node_page);
- }
- f2fs_put_page(node_page, 1);
+ move_node_page(node_page, gc_type);
stat_inc_node_blk_count(sbi, 1, gc_type);
}
@@ -834,18 +826,9 @@
f2fs_put_page(sum_page, 0);
}
- if (gc_type == FG_GC) {
- if (type == SUM_TYPE_NODE) {
- struct writeback_control wbc = {
- .sync_mode = WB_SYNC_ALL,
- .nr_to_write = LONG_MAX,
- .for_reclaim = 0,
- };
- sync_node_pages(sbi, 0, &wbc);
- } else {
- f2fs_submit_merged_bio(sbi, DATA, WRITE);
- }
- }
+ if (gc_type == FG_GC)
+ f2fs_submit_merged_bio(sbi,
+ (type == SUM_TYPE_NODE) ? NODE : DATA, WRITE);
blk_finish_plug(&plug);
diff --git a/fs/f2fs/inline.c b/fs/f2fs/inline.c
index a2fbe6f..a4bb155 100644
--- a/fs/f2fs/inline.c
+++ b/fs/f2fs/inline.c
@@ -161,7 +161,7 @@
if (!f2fs_has_inline_data(inode))
return 0;
- page = grab_cache_page(inode->i_mapping, 0);
+ page = f2fs_grab_cache_page(inode->i_mapping, 0, false);
if (!page)
return -ENOMEM;
@@ -303,11 +303,6 @@
else
f2fs_put_page(ipage, 0);
- /*
- * For the most part, it should be a bug when name_len is zero.
- * We stop here for figuring out where the bugs has occurred.
- */
- f2fs_bug_on(sbi, d.max < 0);
return de;
}
@@ -355,7 +350,7 @@
* NOTE: ipage is grabbed by caller, but if any error occurs, we should
* release ipage in this function.
*/
-static int f2fs_convert_inline_dir(struct inode *dir, struct page *ipage,
+static int f2fs_move_inline_dirents(struct inode *dir, struct page *ipage,
struct f2fs_inline_dentry *inline_dentry)
{
struct page *page;
@@ -363,7 +358,7 @@
struct f2fs_dentry_block *dentry_blk;
int err;
- page = grab_cache_page(dir->i_mapping, 0);
+ page = f2fs_grab_cache_page(dir->i_mapping, 0, false);
if (!page) {
f2fs_put_page(ipage, 1);
return -ENOMEM;
@@ -405,6 +400,7 @@
stat_dec_inline_dir(dir);
clear_inode_flag(F2FS_I(dir), FI_INLINE_DENTRY);
+ F2FS_I(dir)->i_current_depth = 1;
if (i_size_read(dir) < PAGE_SIZE) {
i_size_write(dir, PAGE_SIZE);
set_inode_flag(F2FS_I(dir), FI_UPDATE_DIR);
@@ -416,6 +412,105 @@
return err;
}
+static int f2fs_add_inline_entries(struct inode *dir,
+ struct f2fs_inline_dentry *inline_dentry)
+{
+ struct f2fs_dentry_ptr d;
+ unsigned long bit_pos = 0;
+ int err = 0;
+
+ make_dentry_ptr(NULL, &d, (void *)inline_dentry, 2);
+
+ while (bit_pos < d.max) {
+ struct f2fs_dir_entry *de;
+ struct qstr new_name;
+ nid_t ino;
+ umode_t fake_mode;
+
+ if (!test_bit_le(bit_pos, d.bitmap)) {
+ bit_pos++;
+ continue;
+ }
+
+ de = &d.dentry[bit_pos];
+
+ if (unlikely(!de->name_len)) {
+ bit_pos++;
+ continue;
+ }
+
+ new_name.name = d.filename[bit_pos];
+ new_name.len = de->name_len;
+
+ ino = le32_to_cpu(de->ino);
+ fake_mode = get_de_type(de) << S_SHIFT;
+
+ err = f2fs_add_regular_entry(dir, &new_name, NULL,
+ ino, fake_mode);
+ if (err)
+ goto punch_dentry_pages;
+
+ bit_pos += GET_DENTRY_SLOTS(le16_to_cpu(de->name_len));
+ }
+ return 0;
+punch_dentry_pages:
+ truncate_inode_pages(&dir->i_data, 0);
+ truncate_blocks(dir, 0, false);
+ remove_dirty_inode(dir);
+ return err;
+}
+
+static int f2fs_move_rehashed_dirents(struct inode *dir, struct page *ipage,
+ struct f2fs_inline_dentry *inline_dentry)
+{
+ struct f2fs_inline_dentry *backup_dentry;
+ struct f2fs_inode_info *fi = F2FS_I(dir);
+ int err;
+
+ backup_dentry = f2fs_kmalloc(sizeof(struct f2fs_inline_dentry),
+ GFP_F2FS_ZERO);
+ if (!backup_dentry) {
+ f2fs_put_page(ipage, 1);
+ return -ENOMEM;
+ }
+
+ memcpy(backup_dentry, inline_dentry, MAX_INLINE_DATA);
+ truncate_inline_inode(ipage, 0);
+
+ unlock_page(ipage);
+
+ err = f2fs_add_inline_entries(dir, backup_dentry);
+ if (err)
+ goto recover;
+
+ lock_page(ipage);
+
+ stat_dec_inline_dir(dir);
+ clear_inode_flag(fi, FI_INLINE_DENTRY);
+ update_inode(dir, ipage);
+ kfree(backup_dentry);
+ return 0;
+recover:
+ lock_page(ipage);
+ memcpy(inline_dentry, backup_dentry, MAX_INLINE_DATA);
+ fi->i_current_depth = 0;
+ i_size_write(dir, MAX_INLINE_DATA);
+ update_inode(dir, ipage);
+ f2fs_put_page(ipage, 1);
+
+ kfree(backup_dentry);
+ return err;
+}
+
+static int f2fs_convert_inline_dir(struct inode *dir, struct page *ipage,
+ struct f2fs_inline_dentry *inline_dentry)
+{
+ if (!F2FS_I(dir)->i_dir_level)
+ return f2fs_move_inline_dirents(dir, ipage, inline_dentry);
+ else
+ return f2fs_move_rehashed_dirents(dir, ipage, inline_dentry);
+}
+
int f2fs_add_inline_entry(struct inode *dir, const struct qstr *name,
struct inode *inode, nid_t ino, umode_t mode)
{
diff --git a/fs/f2fs/inode.c b/fs/f2fs/inode.c
index cb269c4..2e68ada 100644
--- a/fs/f2fs/inode.c
+++ b/fs/f2fs/inode.c
@@ -283,7 +283,7 @@
cond_resched();
goto retry;
} else if (err != -ENOENT) {
- f2fs_stop_checkpoint(sbi);
+ f2fs_stop_checkpoint(sbi, false);
}
return 0;
}
@@ -344,7 +344,7 @@
sb_start_intwrite(inode->i_sb);
set_inode_flag(fi, FI_NO_ALLOC);
i_size_write(inode, 0);
-
+retry:
if (F2FS_HAS_BLOCKS(inode))
err = f2fs_truncate(inode, true);
@@ -354,6 +354,12 @@
f2fs_unlock_op(sbi);
}
+ /* give more chances, if ENOMEM case */
+ if (err == -ENOMEM) {
+ err = 0;
+ goto retry;
+ }
+
sb_end_intwrite(inode->i_sb);
no_delete:
stat_dec_inline_xattr(inode);
@@ -368,26 +374,11 @@
if (is_inode_flag_set(fi, FI_UPDATE_WRITE))
add_ino_entry(sbi, inode->i_ino, UPDATE_INO);
if (is_inode_flag_set(fi, FI_FREE_NID)) {
- if (err && err != -ENOENT)
- alloc_nid_done(sbi, inode->i_ino);
- else
- alloc_nid_failed(sbi, inode->i_ino);
+ alloc_nid_failed(sbi, inode->i_ino);
clear_inode_flag(fi, FI_FREE_NID);
}
-
- if (err && err != -ENOENT) {
- if (!exist_written_data(sbi, inode->i_ino, ORPHAN_INO)) {
- /*
- * get here because we failed to release resource
- * of inode previously, reminder our user to run fsck
- * for fixing.
- */
- set_sbi_flag(sbi, SBI_NEED_FSCK);
- f2fs_msg(sbi->sb, KERN_WARNING,
- "inode (ino:%lu) resource leak, run fsck "
- "to fix this issue!", inode->i_ino);
- }
- }
+ f2fs_bug_on(sbi, err &&
+ !exist_written_data(sbi, inode->i_ino, ORPHAN_INO));
out_clear:
fscrypt_put_encryption_info(inode, NULL);
clear_inode(inode);
@@ -397,37 +388,32 @@
void handle_failed_inode(struct inode *inode)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
- int err = 0;
+ struct node_info ni;
- clear_nlink(inode);
- make_bad_inode(inode);
+ /* don't make bad inode, since it becomes a regular file. */
unlock_new_inode(inode);
- i_size_write(inode, 0);
- if (F2FS_HAS_BLOCKS(inode))
- err = f2fs_truncate(inode, false);
-
- if (!err)
- err = remove_inode_page(inode);
-
/*
- * if we skip truncate_node in remove_inode_page bacause we failed
- * before, it's better to find another way to release resource of
- * this inode (e.g. valid block count, node block or nid). Here we
- * choose to add this inode to orphan list, so that we can call iput
- * for releasing in orphan recovery flow.
- *
* Note: we should add inode to orphan list before f2fs_unlock_op()
* so we can prevent losing this orphan when encoutering checkpoint
* and following suddenly power-off.
*/
- if (err && err != -ENOENT) {
- err = acquire_orphan_inode(sbi);
- if (!err)
+ get_node_info(sbi, inode->i_ino, &ni);
+
+ if (ni.blk_addr != NULL_ADDR) {
+ int err = acquire_orphan_inode(sbi);
+ if (err) {
+ set_sbi_flag(sbi, SBI_NEED_FSCK);
+ f2fs_msg(sbi->sb, KERN_WARNING,
+ "Too many orphan inodes, run fsck to fix.");
+ } else {
add_orphan_inode(sbi, inode->i_ino);
+ }
+ alloc_nid_done(sbi, inode->i_ino);
+ } else {
+ set_inode_flag(F2FS_I(inode), FI_FREE_NID);
}
- set_inode_flag(F2FS_I(inode), FI_FREE_NID);
f2fs_unlock_op(sbi);
/* iput will drop the inode object */
diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c
index 1a33de9..1f21aae 100644
--- a/fs/f2fs/node.c
+++ b/fs/f2fs/node.c
@@ -407,6 +407,29 @@
up_write(&nm_i->nat_tree_lock);
}
+/*
+ * readahead MAX_RA_NODE number of node pages.
+ */
+static void ra_node_pages(struct page *parent, int start, int n)
+{
+ struct f2fs_sb_info *sbi = F2FS_P_SB(parent);
+ struct blk_plug plug;
+ int i, end;
+ nid_t nid;
+
+ blk_start_plug(&plug);
+
+ /* Then, try readahead for siblings of the desired node */
+ end = start + n;
+ end = min(end, NIDS_PER_BLOCK);
+ for (i = start; i < end; i++) {
+ nid = get_nid(parent, i, false);
+ ra_node_page(sbi, nid);
+ }
+
+ blk_finish_plug(&plug);
+}
+
pgoff_t get_next_page_offset(struct dnode_of_data *dn, pgoff_t pgofs)
{
const long direct_index = ADDRS_PER_INODE(dn->inode);
@@ -707,6 +730,8 @@
return PTR_ERR(page);
}
+ ra_node_pages(page, ofs, NIDS_PER_BLOCK);
+
rn = F2FS_NODE(page);
if (depth < 3) {
for (i = ofs; i < NIDS_PER_BLOCK; i++, freed++) {
@@ -784,6 +809,8 @@
nid[i + 1] = get_nid(pages[i], offset[i + 1], false);
}
+ ra_node_pages(pages[idx], offset[idx + 1], NIDS_PER_BLOCK);
+
/* free direct nodes linked to a partial indirect node */
for (i = offset[idx + 1]; i < NIDS_PER_BLOCK; i++) {
child_nid = get_nid(pages[idx], i, false);
@@ -832,7 +859,7 @@
trace_f2fs_truncate_inode_blocks_enter(inode, from);
level = get_node_path(inode, from, offset, noffset);
-restart:
+
page = get_node_page(sbi, inode->i_ino);
if (IS_ERR(page)) {
trace_f2fs_truncate_inode_blocks_exit(inode, PTR_ERR(page));
@@ -896,10 +923,7 @@
if (offset[1] == 0 &&
ri->i_nid[offset[0] - NODE_DIR1_BLOCK]) {
lock_page(page);
- if (unlikely(page->mapping != NODE_MAPPING(sbi))) {
- f2fs_put_page(page, 1);
- goto restart;
- }
+ BUG_ON(page->mapping != NODE_MAPPING(sbi));
f2fs_wait_on_page_writeback(page, NODE, true);
ri->i_nid[offset[0] - NODE_DIR1_BLOCK] = 0;
set_page_dirty(page);
@@ -998,7 +1022,7 @@
if (unlikely(is_inode_flag_set(F2FS_I(dn->inode), FI_NO_ALLOC)))
return ERR_PTR(-EPERM);
- page = grab_cache_page(NODE_MAPPING(sbi), dn->nid);
+ page = f2fs_grab_cache_page(NODE_MAPPING(sbi), dn->nid, false);
if (!page)
return ERR_PTR(-ENOMEM);
@@ -1090,7 +1114,7 @@
if (apage)
return;
- apage = grab_cache_page(NODE_MAPPING(sbi), nid);
+ apage = f2fs_grab_cache_page(NODE_MAPPING(sbi), nid, false);
if (!apage)
return;
@@ -1098,29 +1122,6 @@
f2fs_put_page(apage, err ? 1 : 0);
}
-/*
- * readahead MAX_RA_NODE number of node pages.
- */
-static void ra_node_pages(struct page *parent, int start)
-{
- struct f2fs_sb_info *sbi = F2FS_P_SB(parent);
- struct blk_plug plug;
- int i, end;
- nid_t nid;
-
- blk_start_plug(&plug);
-
- /* Then, try readahead for siblings of the desired node */
- end = start + MAX_RA_NODE;
- end = min(end, NIDS_PER_BLOCK);
- for (i = start; i < end; i++) {
- nid = get_nid(parent, i, false);
- ra_node_page(sbi, nid);
- }
-
- blk_finish_plug(&plug);
-}
-
static struct page *__get_node_page(struct f2fs_sb_info *sbi, pgoff_t nid,
struct page *parent, int start)
{
@@ -1131,7 +1132,7 @@
return ERR_PTR(-ENOENT);
f2fs_bug_on(sbi, check_nid_range(sbi, nid));
repeat:
- page = grab_cache_page(NODE_MAPPING(sbi), nid);
+ page = f2fs_grab_cache_page(NODE_MAPPING(sbi), nid, false);
if (!page)
return ERR_PTR(-ENOMEM);
@@ -1144,7 +1145,7 @@
}
if (parent)
- ra_node_pages(parent, start + 1);
+ ra_node_pages(parent, start + 1, MAX_RA_NODE);
lock_page(page);
@@ -1196,19 +1197,17 @@
{
struct inode *inode;
struct page *page;
+ int ret;
/* should flush inline_data before evict_inode */
inode = ilookup(sbi->sb, ino);
if (!inode)
return;
- page = pagecache_get_page(inode->i_mapping, 0, FGP_NOWAIT, 0);
+ page = pagecache_get_page(inode->i_mapping, 0, FGP_LOCK|FGP_NOWAIT, 0);
if (!page)
goto iput_out;
- if (!trylock_page(page))
- goto release_out;
-
if (!PageUptodate(page))
goto page_out;
@@ -1218,24 +1217,214 @@
if (!clear_page_dirty_for_io(page))
goto page_out;
- if (!f2fs_write_inline_data(inode, page))
- inode_dec_dirty_pages(inode);
- else
+ ret = f2fs_write_inline_data(inode, page);
+ inode_dec_dirty_pages(inode);
+ if (ret)
set_page_dirty(page);
page_out:
- unlock_page(page);
-release_out:
- f2fs_put_page(page, 0);
+ f2fs_put_page(page, 1);
iput_out:
iput(inode);
}
-int sync_node_pages(struct f2fs_sb_info *sbi, nid_t ino,
- struct writeback_control *wbc)
+void move_node_page(struct page *node_page, int gc_type)
+{
+ if (gc_type == FG_GC) {
+ struct f2fs_sb_info *sbi = F2FS_P_SB(node_page);
+ struct writeback_control wbc = {
+ .sync_mode = WB_SYNC_ALL,
+ .nr_to_write = 1,
+ .for_reclaim = 0,
+ };
+
+ set_page_dirty(node_page);
+ f2fs_wait_on_page_writeback(node_page, NODE, true);
+
+ f2fs_bug_on(sbi, PageWriteback(node_page));
+ if (!clear_page_dirty_for_io(node_page))
+ goto out_page;
+
+ if (NODE_MAPPING(sbi)->a_ops->writepage(node_page, &wbc))
+ unlock_page(node_page);
+ goto release_page;
+ } else {
+ /* set page dirty and write it */
+ if (!PageWriteback(node_page))
+ set_page_dirty(node_page);
+ }
+out_page:
+ unlock_page(node_page);
+release_page:
+ f2fs_put_page(node_page, 0);
+}
+
+static struct page *last_fsync_dnode(struct f2fs_sb_info *sbi, nid_t ino)
{
pgoff_t index, end;
struct pagevec pvec;
- int step = ino ? 2 : 0;
+ struct page *last_page = NULL;
+
+ pagevec_init(&pvec, 0);
+ index = 0;
+ end = ULONG_MAX;
+
+ while (index <= end) {
+ int i, nr_pages;
+ nr_pages = pagevec_lookup_tag(&pvec, NODE_MAPPING(sbi), &index,
+ PAGECACHE_TAG_DIRTY,
+ min(end - index, (pgoff_t)PAGEVEC_SIZE-1) + 1);
+ if (nr_pages == 0)
+ break;
+
+ for (i = 0; i < nr_pages; i++) {
+ struct page *page = pvec.pages[i];
+
+ if (unlikely(f2fs_cp_error(sbi))) {
+ f2fs_put_page(last_page, 0);
+ pagevec_release(&pvec);
+ return ERR_PTR(-EIO);
+ }
+
+ if (!IS_DNODE(page) || !is_cold_node(page))
+ continue;
+ if (ino_of_node(page) != ino)
+ continue;
+
+ lock_page(page);
+
+ if (unlikely(page->mapping != NODE_MAPPING(sbi))) {
+continue_unlock:
+ unlock_page(page);
+ continue;
+ }
+ if (ino_of_node(page) != ino)
+ goto continue_unlock;
+
+ if (!PageDirty(page)) {
+ /* someone wrote it for us */
+ goto continue_unlock;
+ }
+
+ if (last_page)
+ f2fs_put_page(last_page, 0);
+
+ get_page(page);
+ last_page = page;
+ unlock_page(page);
+ }
+ pagevec_release(&pvec);
+ cond_resched();
+ }
+ return last_page;
+}
+
+int fsync_node_pages(struct f2fs_sb_info *sbi, nid_t ino,
+ struct writeback_control *wbc, bool atomic)
+{
+ pgoff_t index, end;
+ struct pagevec pvec;
+ int ret = 0;
+ struct page *last_page = NULL;
+ bool marked = false;
+
+ if (atomic) {
+ last_page = last_fsync_dnode(sbi, ino);
+ if (IS_ERR_OR_NULL(last_page))
+ return PTR_ERR_OR_ZERO(last_page);
+ }
+retry:
+ pagevec_init(&pvec, 0);
+ index = 0;
+ end = ULONG_MAX;
+
+ while (index <= end) {
+ int i, nr_pages;
+ nr_pages = pagevec_lookup_tag(&pvec, NODE_MAPPING(sbi), &index,
+ PAGECACHE_TAG_DIRTY,
+ min(end - index, (pgoff_t)PAGEVEC_SIZE-1) + 1);
+ if (nr_pages == 0)
+ break;
+
+ for (i = 0; i < nr_pages; i++) {
+ struct page *page = pvec.pages[i];
+
+ if (unlikely(f2fs_cp_error(sbi))) {
+ f2fs_put_page(last_page, 0);
+ pagevec_release(&pvec);
+ return -EIO;
+ }
+
+ if (!IS_DNODE(page) || !is_cold_node(page))
+ continue;
+ if (ino_of_node(page) != ino)
+ continue;
+
+ lock_page(page);
+
+ if (unlikely(page->mapping != NODE_MAPPING(sbi))) {
+continue_unlock:
+ unlock_page(page);
+ continue;
+ }
+ if (ino_of_node(page) != ino)
+ goto continue_unlock;
+
+ if (!PageDirty(page) && page != last_page) {
+ /* someone wrote it for us */
+ goto continue_unlock;
+ }
+
+ f2fs_wait_on_page_writeback(page, NODE, true);
+ BUG_ON(PageWriteback(page));
+
+ if (!atomic || page == last_page) {
+ set_fsync_mark(page, 1);
+ if (IS_INODE(page))
+ set_dentry_mark(page,
+ need_dentry_mark(sbi, ino));
+ /* may be written by other thread */
+ if (!PageDirty(page))
+ set_page_dirty(page);
+ }
+
+ if (!clear_page_dirty_for_io(page))
+ goto continue_unlock;
+
+ ret = NODE_MAPPING(sbi)->a_ops->writepage(page, wbc);
+ if (ret) {
+ unlock_page(page);
+ f2fs_put_page(last_page, 0);
+ break;
+ }
+ if (page == last_page) {
+ f2fs_put_page(page, 0);
+ marked = true;
+ break;
+ }
+ }
+ pagevec_release(&pvec);
+ cond_resched();
+
+ if (ret || marked)
+ break;
+ }
+ if (!ret && atomic && !marked) {
+ f2fs_msg(sbi->sb, KERN_DEBUG,
+ "Retry to write fsync mark: ino=%u, idx=%lx",
+ ino, last_page->index);
+ lock_page(last_page);
+ set_page_dirty(last_page);
+ unlock_page(last_page);
+ goto retry;
+ }
+ return ret ? -EIO: 0;
+}
+
+int sync_node_pages(struct f2fs_sb_info *sbi, struct writeback_control *wbc)
+{
+ pgoff_t index, end;
+ struct pagevec pvec;
+ int step = 0;
int nwritten = 0;
pagevec_init(&pvec, 0);
@@ -1274,15 +1463,8 @@
if (step == 2 && (!IS_DNODE(page) ||
!is_cold_node(page)))
continue;
-
- /*
- * If an fsync mode,
- * we should not skip writing node pages.
- */
lock_node:
- if (ino && ino_of_node(page) == ino)
- lock_page(page);
- else if (!trylock_page(page))
+ if (!trylock_page(page))
continue;
if (unlikely(page->mapping != NODE_MAPPING(sbi))) {
@@ -1290,8 +1472,6 @@
unlock_page(page);
continue;
}
- if (ino && ino_of_node(page) != ino)
- goto continue_unlock;
if (!PageDirty(page)) {
/* someone wrote it for us */
@@ -1299,7 +1479,7 @@
}
/* flush inline_data */
- if (!ino && is_inline_node(page)) {
+ if (is_inline_node(page)) {
clear_inline_node(page);
unlock_page(page);
flush_inline_data(sbi, ino_of_node(page));
@@ -1312,17 +1492,8 @@
if (!clear_page_dirty_for_io(page))
goto continue_unlock;
- /* called by fsync() */
- if (ino && IS_DNODE(page)) {
- set_fsync_mark(page, 1);
- if (IS_INODE(page))
- set_dentry_mark(page,
- need_dentry_mark(sbi, ino));
- nwritten++;
- } else {
- set_fsync_mark(page, 0);
- set_dentry_mark(page, 0);
- }
+ set_fsync_mark(page, 0);
+ set_dentry_mark(page, 0);
if (NODE_MAPPING(sbi)->a_ops->writepage(page, wbc))
unlock_page(page);
@@ -1470,7 +1641,7 @@
diff = nr_pages_to_write(sbi, NODE, wbc);
wbc->sync_mode = WB_SYNC_NONE;
- sync_node_pages(sbi, 0, wbc);
+ sync_node_pages(sbi, wbc);
wbc->nr_to_write = max((long)0, wbc->nr_to_write - diff);
return 0;
@@ -1524,7 +1695,6 @@
struct f2fs_nm_info *nm_i = NM_I(sbi);
struct free_nid *i;
struct nat_entry *ne;
- bool allocated = false;
if (!available_free_memory(sbi, FREE_NIDS))
return -1;
@@ -1538,8 +1708,6 @@
ne = __lookup_nat_cache(nm_i, nid);
if (ne && (!get_nat_flag(ne, IS_CHECKPOINTED) ||
nat_get_blkaddr(ne) != NULL_ADDR))
- allocated = true;
- if (allocated)
return 0;
}
@@ -1672,6 +1840,10 @@
struct f2fs_nm_info *nm_i = NM_I(sbi);
struct free_nid *i = NULL;
retry:
+#ifdef CONFIG_F2FS_FAULT_INJECTION
+ if (time_to_inject(FAULT_ALLOC_NID))
+ return false;
+#endif
if (unlikely(sbi->total_valid_node_count + 1 > nm_i->available_nids))
return false;
@@ -1846,7 +2018,7 @@
if (unlikely(old_ni.blk_addr != NULL_ADDR))
return -EINVAL;
- ipage = grab_cache_page(NODE_MAPPING(sbi), ino);
+ ipage = f2fs_grab_cache_page(NODE_MAPPING(sbi), ino, false);
if (!ipage)
return -ENOMEM;
diff --git a/fs/f2fs/recovery.c b/fs/f2fs/recovery.c
index 011942f..3d7216d 100644
--- a/fs/f2fs/recovery.c
+++ b/fs/f2fs/recovery.c
@@ -49,8 +49,9 @@
bool space_for_roll_forward(struct f2fs_sb_info *sbi)
{
- if (sbi->last_valid_block_count + sbi->alloc_valid_block_count
- > sbi->user_block_count)
+ s64 nalloc = percpu_counter_sum_positive(&sbi->alloc_valid_block_count);
+
+ if (sbi->last_valid_block_count + nalloc > sbi->user_block_count)
return false;
return true;
}
@@ -67,7 +68,30 @@
return NULL;
}
-static int recover_dentry(struct inode *inode, struct page *ipage)
+static struct fsync_inode_entry *add_fsync_inode(struct list_head *head,
+ struct inode *inode)
+{
+ struct fsync_inode_entry *entry;
+
+ entry = kmem_cache_alloc(fsync_entry_slab, GFP_F2FS_ZERO);
+ if (!entry)
+ return NULL;
+
+ entry->inode = inode;
+ list_add_tail(&entry->list, head);
+
+ return entry;
+}
+
+static void del_fsync_inode(struct fsync_inode_entry *entry)
+{
+ iput(entry->inode);
+ list_del(&entry->list);
+ kmem_cache_free(fsync_entry_slab, entry);
+}
+
+static int recover_dentry(struct inode *inode, struct page *ipage,
+ struct list_head *dir_list)
{
struct f2fs_inode *raw_inode = F2FS_INODE(ipage);
nid_t pino = le32_to_cpu(raw_inode->i_pino);
@@ -75,18 +99,29 @@
struct qstr name;
struct page *page;
struct inode *dir, *einode;
+ struct fsync_inode_entry *entry;
int err = 0;
- dir = f2fs_iget(inode->i_sb, pino);
- if (IS_ERR(dir)) {
- err = PTR_ERR(dir);
- goto out;
+ entry = get_fsync_inode(dir_list, pino);
+ if (!entry) {
+ dir = f2fs_iget(inode->i_sb, pino);
+ if (IS_ERR(dir)) {
+ err = PTR_ERR(dir);
+ goto out;
+ }
+
+ entry = add_fsync_inode(dir_list, dir);
+ if (!entry) {
+ err = -ENOMEM;
+ iput(dir);
+ goto out;
+ }
}
- if (file_enc_name(inode)) {
- iput(dir);
+ dir = entry->inode;
+
+ if (file_enc_name(inode))
return 0;
- }
name.len = le32_to_cpu(raw_inode->i_namelen);
name.name = raw_inode->i_name;
@@ -94,7 +129,7 @@
if (unlikely(name.len > F2FS_NAME_LEN)) {
WARN_ON(1);
err = -ENAMETOOLONG;
- goto out_err;
+ goto out;
}
retry:
de = f2fs_find_entry(dir, &name, &page);
@@ -120,23 +155,12 @@
goto retry;
}
err = __f2fs_add_link(dir, &name, inode, inode->i_ino, inode->i_mode);
- if (err)
- goto out_err;
-
- if (is_inode_flag_set(F2FS_I(dir), FI_DELAY_IPUT)) {
- iput(dir);
- } else {
- add_dirty_dir_inode(dir);
- set_inode_flag(F2FS_I(dir), FI_DELAY_IPUT);
- }
goto out;
out_unmap_put:
f2fs_dentry_kunmap(dir, page);
f2fs_put_page(page, 0);
-out_err:
- iput(dir);
out:
f2fs_msg(inode->i_sb, KERN_NOTICE,
"%s: ino = %x, name = %s, dir = %lx, err = %d",
@@ -198,6 +222,7 @@
{
unsigned long long cp_ver = cur_cp_version(F2FS_CKPT(sbi));
struct curseg_info *curseg;
+ struct inode *inode;
struct page *page = NULL;
block_t blkaddr;
int err = 0;
@@ -206,8 +231,6 @@
curseg = CURSEG_I(sbi, CURSEG_WARM_NODE);
blkaddr = NEXT_FREE_BLKADDR(sbi, curseg);
- ra_meta_pages(sbi, blkaddr, 1, META_POR, true);
-
while (1) {
struct fsync_inode_entry *entry;
@@ -233,35 +256,32 @@
break;
}
- /* add this fsync inode to the list */
- entry = kmem_cache_alloc(fsync_entry_slab, GFP_F2FS_ZERO);
- if (!entry) {
- err = -ENOMEM;
- break;
- }
/*
* CP | dnode(F) | inode(DF)
* For this case, we should not give up now.
*/
- entry->inode = f2fs_iget(sbi->sb, ino_of_node(page));
- if (IS_ERR(entry->inode)) {
- err = PTR_ERR(entry->inode);
- kmem_cache_free(fsync_entry_slab, entry);
+ inode = f2fs_iget(sbi->sb, ino_of_node(page));
+ if (IS_ERR(inode)) {
+ err = PTR_ERR(inode);
if (err == -ENOENT) {
err = 0;
goto next;
}
break;
}
- list_add_tail(&entry->list, head);
+
+ /* add this fsync inode to the list */
+ entry = add_fsync_inode(head, inode);
+ if (!entry) {
+ err = -ENOMEM;
+ iput(inode);
+ break;
+ }
}
entry->blkaddr = blkaddr;
- if (IS_INODE(page)) {
- entry->last_inode = blkaddr;
- if (is_dent_dnode(page))
- entry->last_dentry = blkaddr;
- }
+ if (IS_INODE(page) && is_dent_dnode(page))
+ entry->last_dentry = blkaddr;
next:
/* check next segment */
blkaddr = next_blkaddr_of_node(page);
@@ -277,11 +297,8 @@
{
struct fsync_inode_entry *entry, *tmp;
- list_for_each_entry_safe(entry, tmp, head, list) {
- iput(entry->inode);
- list_del(&entry->list);
- kmem_cache_free(fsync_entry_slab, entry);
- }
+ list_for_each_entry_safe(entry, tmp, head, list)
+ del_fsync_inode(entry);
}
static int check_index_in_prev_nodes(struct f2fs_sb_info *sbi,
@@ -444,8 +461,7 @@
*/
if (dest == NEW_ADDR) {
truncate_data_blocks_range(&dn, 1);
- err = reserve_new_block(&dn);
- f2fs_bug_on(sbi, err);
+ reserve_new_block(&dn);
continue;
}
@@ -454,6 +470,10 @@
if (src == NULL_ADDR) {
err = reserve_new_block(&dn);
+#ifdef CONFIG_F2FS_FAULT_INJECTION
+ while (err)
+ err = reserve_new_block(&dn);
+#endif
/* We should not get -ENOSPC */
f2fs_bug_on(sbi, err);
}
@@ -486,7 +506,8 @@
return err;
}
-static int recover_data(struct f2fs_sb_info *sbi, struct list_head *head)
+static int recover_data(struct f2fs_sb_info *sbi, struct list_head *inode_list,
+ struct list_head *dir_list)
{
unsigned long long cp_ver = cur_cp_version(F2FS_CKPT(sbi));
struct curseg_info *curseg;
@@ -513,7 +534,7 @@
break;
}
- entry = get_fsync_inode(head, ino_of_node(page));
+ entry = get_fsync_inode(inode_list, ino_of_node(page));
if (!entry)
goto next;
/*
@@ -521,10 +542,10 @@
* In this case, we can lose the latest inode(x).
* So, call recover_inode for the inode update.
*/
- if (entry->last_inode == blkaddr)
+ if (IS_INODE(page))
recover_inode(entry->inode, page);
if (entry->last_dentry == blkaddr) {
- err = recover_dentry(entry->inode, page);
+ err = recover_dentry(entry->inode, page, dir_list);
if (err) {
f2fs_put_page(page, 1);
break;
@@ -536,11 +557,8 @@
break;
}
- if (entry->blkaddr == blkaddr) {
- iput(entry->inode);
- list_del(&entry->list);
- kmem_cache_free(fsync_entry_slab, entry);
- }
+ if (entry->blkaddr == blkaddr)
+ del_fsync_inode(entry);
next:
/* check next segment */
blkaddr = next_blkaddr_of_node(page);
@@ -551,12 +569,14 @@
return err;
}
-int recover_fsync_data(struct f2fs_sb_info *sbi)
+int recover_fsync_data(struct f2fs_sb_info *sbi, bool check_only)
{
struct curseg_info *curseg = CURSEG_I(sbi, CURSEG_WARM_NODE);
struct list_head inode_list;
+ struct list_head dir_list;
block_t blkaddr;
int err;
+ int ret = 0;
bool need_writecp = false;
fsync_entry_slab = f2fs_kmem_cache_create("f2fs_fsync_inode_entry",
@@ -565,6 +585,7 @@
return -ENOMEM;
INIT_LIST_HEAD(&inode_list);
+ INIT_LIST_HEAD(&dir_list);
/* prevent checkpoint */
mutex_lock(&sbi->cp_mutex);
@@ -573,21 +594,22 @@
/* step #1: find fsynced inode numbers */
err = find_fsync_dnodes(sbi, &inode_list);
- if (err)
+ if (err || list_empty(&inode_list))
goto out;
- if (list_empty(&inode_list))
+ if (check_only) {
+ ret = 1;
goto out;
+ }
need_writecp = true;
/* step #2: recover data */
- err = recover_data(sbi, &inode_list);
+ err = recover_data(sbi, &inode_list, &dir_list);
if (!err)
f2fs_bug_on(sbi, !list_empty(&inode_list));
out:
destroy_fsync_dnodes(&inode_list);
- kmem_cache_destroy(fsync_entry_slab);
/* truncate meta pages to be used by the recovery */
truncate_inode_pages_range(META_MAPPING(sbi),
@@ -625,5 +647,8 @@
} else {
mutex_unlock(&sbi->cp_mutex);
}
- return err;
+
+ destroy_fsync_dnodes(&dir_list);
+ kmem_cache_destroy(fsync_entry_slab);
+ return ret ? ret: err;
}
diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
index 540669d..2e6f537 100644
--- a/fs/f2fs/segment.c
+++ b/fs/f2fs/segment.c
@@ -223,9 +223,11 @@
f2fs_put_dnode(&dn);
}
next:
- ClearPageUptodate(page);
+ /* we don't need to invalidate this in the sccessful status */
+ if (drop || recover)
+ ClearPageUptodate(page);
set_page_private(page, 0);
- ClearPageUptodate(page);
+ ClearPagePrivate(page);
f2fs_put_page(page, 1);
list_del(&cur->list);
@@ -239,6 +241,8 @@
{
struct f2fs_inode_info *fi = F2FS_I(inode);
+ clear_inode_flag(F2FS_I(inode), FI_ATOMIC_FILE);
+
mutex_lock(&fi->inmem_lock);
__revoke_inmem_pages(inode, &fi->inmem_pages, true, false);
mutex_unlock(&fi->inmem_lock);
diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h
index 975c33d..7a756ff 100644
--- a/fs/f2fs/segment.h
+++ b/fs/f2fs/segment.h
@@ -158,16 +158,17 @@
};
struct seg_entry {
- unsigned short valid_blocks; /* # of valid blocks */
+ unsigned int type:6; /* segment type like CURSEG_XXX_TYPE */
+ unsigned int valid_blocks:10; /* # of valid blocks */
+ unsigned int ckpt_valid_blocks:10; /* # of valid blocks last cp */
+ unsigned int padding:6; /* padding */
unsigned char *cur_valid_map; /* validity bitmap of blocks */
/*
* # of valid blocks and the validity bitmap stored in the the last
* checkpoint pack. This information is used by the SSR mode.
*/
- unsigned short ckpt_valid_blocks;
- unsigned char *ckpt_valid_map;
+ unsigned char *ckpt_valid_map; /* validity bitmap of blocks last cp */
unsigned char *discard_map;
- unsigned char type; /* segment type like CURSEG_XXX_TYPE */
unsigned long long mtime; /* modification time of the segment */
};
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index 006f87d..74cc852 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -39,6 +39,30 @@
static struct kmem_cache *f2fs_inode_cachep;
static struct kset *f2fs_kset;
+#ifdef CONFIG_F2FS_FAULT_INJECTION
+struct f2fs_fault_info f2fs_fault;
+
+char *fault_name[FAULT_MAX] = {
+ [FAULT_KMALLOC] = "kmalloc",
+ [FAULT_PAGE_ALLOC] = "page alloc",
+ [FAULT_ALLOC_NID] = "alloc nid",
+ [FAULT_ORPHAN] = "orphan",
+ [FAULT_BLOCK] = "no more block",
+ [FAULT_DIR_DEPTH] = "too big dir depth",
+};
+
+static void f2fs_build_fault_attr(unsigned int rate)
+{
+ if (rate) {
+ atomic_set(&f2fs_fault.inject_ops, 0);
+ f2fs_fault.inject_rate = rate;
+ f2fs_fault.inject_type = (1 << FAULT_MAX) - 1;
+ } else {
+ memset(&f2fs_fault, 0, sizeof(struct f2fs_fault_info));
+ }
+}
+#endif
+
/* f2fs-wide shrinker description */
static struct shrinker f2fs_shrinker_info = {
.scan_objects = f2fs_shrink_scan,
@@ -68,6 +92,7 @@
Opt_noextent_cache,
Opt_noinline_data,
Opt_data_flush,
+ Opt_fault_injection,
Opt_err,
};
@@ -93,6 +118,7 @@
{Opt_noextent_cache, "noextent_cache"},
{Opt_noinline_data, "noinline_data"},
{Opt_data_flush, "data_flush"},
+ {Opt_fault_injection, "fault_injection=%u"},
{Opt_err, NULL},
};
@@ -102,6 +128,10 @@
SM_INFO, /* struct f2fs_sm_info */
NM_INFO, /* struct f2fs_nm_info */
F2FS_SBI, /* struct f2fs_sb_info */
+#ifdef CONFIG_F2FS_FAULT_INJECTION
+ FAULT_INFO_RATE, /* struct f2fs_fault_info */
+ FAULT_INFO_TYPE, /* struct f2fs_fault_info */
+#endif
};
struct f2fs_attr {
@@ -123,6 +153,11 @@
return (unsigned char *)NM_I(sbi);
else if (struct_type == F2FS_SBI)
return (unsigned char *)sbi;
+#ifdef CONFIG_F2FS_FAULT_INJECTION
+ else if (struct_type == FAULT_INFO_RATE ||
+ struct_type == FAULT_INFO_TYPE)
+ return (unsigned char *)&f2fs_fault;
+#endif
return NULL;
}
@@ -172,6 +207,10 @@
ret = kstrtoul(skip_spaces(buf), 0, &t);
if (ret < 0)
return ret;
+#ifdef CONFIG_F2FS_FAULT_INJECTION
+ if (a->struct_type == FAULT_INFO_TYPE && t >= (1 << FAULT_MAX))
+ return -EINVAL;
+#endif
*ui = t;
return count;
}
@@ -237,6 +276,10 @@
F2FS_RW_ATTR(F2FS_SBI, f2fs_sb_info, dir_level, dir_level);
F2FS_RW_ATTR(F2FS_SBI, f2fs_sb_info, cp_interval, interval_time[CP_TIME]);
F2FS_RW_ATTR(F2FS_SBI, f2fs_sb_info, idle_interval, interval_time[REQ_TIME]);
+#ifdef CONFIG_F2FS_FAULT_INJECTION
+F2FS_RW_ATTR(FAULT_INFO_RATE, f2fs_fault_info, inject_rate, inject_rate);
+F2FS_RW_ATTR(FAULT_INFO_TYPE, f2fs_fault_info, inject_type, inject_type);
+#endif
F2FS_GENERAL_RO_ATTR(lifetime_write_kbytes);
#define ATTR_LIST(name) (&f2fs_attr_##name.attr)
@@ -273,6 +316,22 @@
.release = f2fs_sb_release,
};
+#ifdef CONFIG_F2FS_FAULT_INJECTION
+/* sysfs for f2fs fault injection */
+static struct kobject f2fs_fault_inject;
+
+static struct attribute *f2fs_fault_attrs[] = {
+ ATTR_LIST(inject_rate),
+ ATTR_LIST(inject_type),
+ NULL
+};
+
+static struct kobj_type f2fs_fault_ktype = {
+ .default_attrs = f2fs_fault_attrs,
+ .sysfs_ops = &f2fs_attr_ops,
+};
+#endif
+
void f2fs_msg(struct super_block *sb, const char *level, const char *fmt, ...)
{
struct va_format vaf;
@@ -300,6 +359,10 @@
char *p, *name;
int arg = 0;
+#ifdef CONFIG_F2FS_FAULT_INJECTION
+ f2fs_build_fault_attr(0);
+#endif
+
if (!options)
return 0;
@@ -433,6 +496,16 @@
case Opt_data_flush:
set_opt(sbi, DATA_FLUSH);
break;
+ case Opt_fault_injection:
+ if (args->from && match_int(args, &arg))
+ return -EINVAL;
+#ifdef CONFIG_F2FS_FAULT_INJECTION
+ f2fs_build_fault_attr(arg);
+#else
+ f2fs_msg(sb, KERN_INFO,
+ "FAULT_INJECTION was not selected");
+#endif
+ break;
default:
f2fs_msg(sb, KERN_ERR,
"Unrecognized mount option \"%s\" or missing value",
@@ -453,9 +526,13 @@
init_once((void *) fi);
+ if (percpu_counter_init(&fi->dirty_pages, 0, GFP_NOFS)) {
+ kmem_cache_free(f2fs_inode_cachep, fi);
+ return NULL;
+ }
+
/* Initialize f2fs-specific inode info */
fi->vfs_inode.i_version = 1;
- atomic_set(&fi->dirty_pages, 0);
fi->i_current_depth = 1;
fi->i_advise = 0;
init_rwsem(&fi->i_sem);
@@ -530,15 +607,27 @@
static void f2fs_destroy_inode(struct inode *inode)
{
+ percpu_counter_destroy(&F2FS_I(inode)->dirty_pages);
call_rcu(&inode->i_rcu, f2fs_i_callback);
}
+static void destroy_percpu_info(struct f2fs_sb_info *sbi)
+{
+ int i;
+
+ for (i = 0; i < NR_COUNT_TYPE; i++)
+ percpu_counter_destroy(&sbi->nr_pages[i]);
+ percpu_counter_destroy(&sbi->alloc_valid_block_count);
+ percpu_counter_destroy(&sbi->total_valid_inode_count);
+}
+
static void f2fs_put_super(struct super_block *sb)
{
struct f2fs_sb_info *sbi = F2FS_SB(sb);
if (sbi->s_proc) {
remove_proc_entry("segment_info", sbi->s_proc);
+ remove_proc_entry("segment_bits", sbi->s_proc);
remove_proc_entry(sb->s_id, f2fs_proc_root);
}
kobject_del(&sbi->s_kobj);
@@ -568,15 +657,14 @@
* normally superblock is clean, so we need to release this.
* In addition, EIO will skip do checkpoint, we need this as well.
*/
- release_ino_entry(sbi);
+ release_ino_entry(sbi, true);
release_discard_addrs(sbi);
f2fs_leave_shrinker(sbi);
mutex_unlock(&sbi->umount_mutex);
/* our cp_error case, we can wait for any writeback page */
- if (get_pages(sbi, F2FS_WRITEBACK))
- f2fs_flush_merged_bios(sbi);
+ f2fs_flush_merged_bios(sbi);
iput(sbi->node_inode);
iput(sbi->meta_inode);
@@ -593,6 +681,8 @@
if (sbi->s_chksum_driver)
crypto_free_shash(sbi->s_chksum_driver);
kfree(sbi->raw_super);
+
+ destroy_percpu_info(sbi);
kfree(sbi);
}
@@ -745,19 +835,47 @@
return 0;
}
-static int segment_info_open_fs(struct inode *inode, struct file *file)
+static int segment_bits_seq_show(struct seq_file *seq, void *offset)
{
- return single_open(file, segment_info_seq_show, PDE_DATA(inode));
+ struct super_block *sb = seq->private;
+ struct f2fs_sb_info *sbi = F2FS_SB(sb);
+ unsigned int total_segs =
+ le32_to_cpu(sbi->raw_super->segment_count_main);
+ int i, j;
+
+ seq_puts(seq, "format: segment_type|valid_blocks|bitmaps\n"
+ "segment_type(0:HD, 1:WD, 2:CD, 3:HN, 4:WN, 5:CN)\n");
+
+ for (i = 0; i < total_segs; i++) {
+ struct seg_entry *se = get_seg_entry(sbi, i);
+
+ seq_printf(seq, "%-10d", i);
+ seq_printf(seq, "%d|%-3u|", se->type,
+ get_valid_blocks(sbi, i, 1));
+ for (j = 0; j < SIT_VBLOCK_MAP_SIZE; j++)
+ seq_printf(seq, "%x ", se->cur_valid_map[j]);
+ seq_putc(seq, '\n');
+ }
+ return 0;
}
-static const struct file_operations f2fs_seq_segment_info_fops = {
- .owner = THIS_MODULE,
- .open = segment_info_open_fs,
- .read = seq_read,
- .llseek = seq_lseek,
- .release = single_release,
+#define F2FS_PROC_FILE_DEF(_name) \
+static int _name##_open_fs(struct inode *inode, struct file *file) \
+{ \
+ return single_open(file, _name##_seq_show, PDE_DATA(inode)); \
+} \
+ \
+static const struct file_operations f2fs_seq_##_name##_fops = { \
+ .owner = THIS_MODULE, \
+ .open = _name##_open_fs, \
+ .read = seq_read, \
+ .llseek = seq_lseek, \
+ .release = single_release, \
};
+F2FS_PROC_FILE_DEF(segment_info);
+F2FS_PROC_FILE_DEF(segment_bits);
+
static void default_options(struct f2fs_sb_info *sbi)
{
/* init some FS parameters */
@@ -791,13 +909,15 @@
org_mount_opt = sbi->mount_opt;
active_logs = sbi->active_logs;
- if (*flags & MS_RDONLY) {
- set_opt(sbi, FASTBOOT);
- set_sbi_flag(sbi, SBI_IS_DIRTY);
+ /* recover superblocks we couldn't write due to previous RO mount */
+ if (!(*flags & MS_RDONLY) && is_sbi_flag_set(sbi, SBI_NEED_SB_WRITE)) {
+ err = f2fs_commit_super(sbi, false);
+ f2fs_msg(sb, KERN_INFO,
+ "Try to recover all the superblocks, ret: %d", err);
+ if (!err)
+ clear_sbi_flag(sbi, SBI_NEED_SB_WRITE);
}
- sync_filesystem(sb);
-
sbi->mount_opt.opt = 0;
default_options(sbi);
@@ -829,7 +949,6 @@
if ((*flags & MS_RDONLY) || !test_opt(sbi, BG_GC)) {
if (sbi->gc_thread) {
stop_gc_thread(sbi);
- f2fs_sync_fs(sb, 1);
need_restart_gc = true;
}
} else if (!sbi->gc_thread) {
@@ -839,6 +958,16 @@
need_stop_gc = true;
}
+ if (*flags & MS_RDONLY) {
+ writeback_inodes_sb(sb, WB_REASON_SYNC);
+ sync_inodes_sb(sb);
+
+ set_sbi_flag(sbi, SBI_IS_DIRTY);
+ set_sbi_flag(sbi, SBI_IS_CLOSE);
+ f2fs_sync_fs(sb, 1);
+ clear_sbi_flag(sbi, SBI_IS_CLOSE);
+ }
+
/*
* We stop issue flush thread if FS is mounted as RO
* or if flush_merge is not passed in mount option.
@@ -852,8 +981,9 @@
}
skip:
/* Update the POSIXACL Flag */
- sb->s_flags = (sb->s_flags & ~MS_POSIXACL) |
+ sb->s_flags = (sb->s_flags & ~MS_POSIXACL) |
(test_opt(sbi, POSIX_ACL) ? MS_POSIXACL : 0);
+
return 0;
restore_gc:
if (need_restart_gc) {
@@ -893,6 +1023,12 @@
ctx, len, NULL);
}
+static int f2fs_key_prefix(struct inode *inode, u8 **key)
+{
+ *key = F2FS_I_SB(inode)->key_prefix;
+ return F2FS_I_SB(inode)->key_prefix_size;
+}
+
static int f2fs_set_context(struct inode *inode, const void *ctx, size_t len,
void *fs_data)
{
@@ -909,6 +1045,7 @@
static struct fscrypt_operations f2fs_cryptops = {
.get_context = f2fs_get_context,
+ .key_prefix = f2fs_key_prefix,
.set_context = f2fs_set_context,
.is_encrypted = f2fs_encrypted_inode,
.empty_dir = f2fs_empty_dir,
@@ -998,11 +1135,12 @@
return __sync_dirty_buffer(bh, WRITE_FLUSH_FUA);
}
-static inline bool sanity_check_area_boundary(struct super_block *sb,
+static inline bool sanity_check_area_boundary(struct f2fs_sb_info *sbi,
struct buffer_head *bh)
{
struct f2fs_super_block *raw_super = (struct f2fs_super_block *)
(bh->b_data + F2FS_SUPER_OFFSET);
+ struct super_block *sb = sbi->sb;
u32 segment0_blkaddr = le32_to_cpu(raw_super->segment0_blkaddr);
u32 cp_blkaddr = le32_to_cpu(raw_super->cp_blkaddr);
u32 sit_blkaddr = le32_to_cpu(raw_super->sit_blkaddr);
@@ -1081,6 +1219,7 @@
segment0_blkaddr) >> log_blocks_per_seg);
if (f2fs_readonly(sb) || bdev_read_only(sb->s_bdev)) {
+ set_sbi_flag(sbi, SBI_NEED_SB_WRITE);
res = "internally";
} else {
err = __f2fs_commit_super(bh, NULL);
@@ -1098,11 +1237,12 @@
return false;
}
-static int sanity_check_raw_super(struct super_block *sb,
+static int sanity_check_raw_super(struct f2fs_sb_info *sbi,
struct buffer_head *bh)
{
struct f2fs_super_block *raw_super = (struct f2fs_super_block *)
(bh->b_data + F2FS_SUPER_OFFSET);
+ struct super_block *sb = sbi->sb;
unsigned int blocksize;
if (F2FS_SUPER_MAGIC != le32_to_cpu(raw_super->magic)) {
@@ -1169,7 +1309,7 @@
}
/* check CP/SIT/NAT/SSA/MAIN_AREA area boundary */
- if (sanity_check_area_boundary(sb, bh))
+ if (sanity_check_area_boundary(sbi, bh))
return 1;
return 0;
@@ -1201,7 +1341,6 @@
static void init_sb_info(struct f2fs_sb_info *sbi)
{
struct f2fs_super_block *raw_super = sbi->raw_super;
- int i;
sbi->log_sectors_per_block =
le32_to_cpu(raw_super->log_sectors_per_block);
@@ -1221,9 +1360,6 @@
sbi->cur_victim_sec = NULL_SECNO;
sbi->max_victim_search = DEF_MAX_VICTIM_SEARCH;
- for (i = 0; i < NR_COUNT_TYPE; i++)
- atomic_set(&sbi->nr_pages[i], 0);
-
sbi->dir_level = DEF_DIR_LEVEL;
sbi->interval_time[CP_TIME] = DEF_CP_INTERVAL;
sbi->interval_time[REQ_TIME] = DEF_IDLE_INTERVAL;
@@ -1231,6 +1367,30 @@
INIT_LIST_HEAD(&sbi->s_list);
mutex_init(&sbi->umount_mutex);
+
+#ifdef CONFIG_F2FS_FS_ENCRYPTION
+ memcpy(sbi->key_prefix, F2FS_KEY_DESC_PREFIX,
+ F2FS_KEY_DESC_PREFIX_SIZE);
+ sbi->key_prefix_size = F2FS_KEY_DESC_PREFIX_SIZE;
+#endif
+}
+
+static int init_percpu_info(struct f2fs_sb_info *sbi)
+{
+ int i, err;
+
+ for (i = 0; i < NR_COUNT_TYPE; i++) {
+ err = percpu_counter_init(&sbi->nr_pages[i], 0, GFP_KERNEL);
+ if (err)
+ return err;
+ }
+
+ err = percpu_counter_init(&sbi->alloc_valid_block_count, 0, GFP_KERNEL);
+ if (err)
+ return err;
+
+ return percpu_counter_init(&sbi->total_valid_inode_count, 0,
+ GFP_KERNEL);
}
/*
@@ -1239,10 +1399,11 @@
* to get the first valid one. If any one of them is broken, we pass
* them recovery flag back to the caller.
*/
-static int read_raw_super_block(struct super_block *sb,
+static int read_raw_super_block(struct f2fs_sb_info *sbi,
struct f2fs_super_block **raw_super,
int *valid_super_block, int *recovery)
{
+ struct super_block *sb = sbi->sb;
int block;
struct buffer_head *bh;
struct f2fs_super_block *super;
@@ -1262,7 +1423,7 @@
}
/* sanity checking of raw super */
- if (sanity_check_raw_super(sb, bh)) {
+ if (sanity_check_raw_super(sbi, bh)) {
f2fs_msg(sb, KERN_ERR,
"Can't find valid F2FS filesystem in %dth superblock",
block + 1);
@@ -1298,6 +1459,12 @@
struct buffer_head *bh;
int err;
+ if ((recover && f2fs_readonly(sbi->sb)) ||
+ bdev_read_only(sbi->sb->s_bdev)) {
+ set_sbi_flag(sbi, SBI_NEED_SB_WRITE);
+ return -EROFS;
+ }
+
/* write back-up superblock first */
bh = sb_getblk(sbi->sb, sbi->valid_super_block ? 0: 1);
if (!bh)
@@ -1323,7 +1490,7 @@
struct f2fs_sb_info *sbi;
struct f2fs_super_block *raw_super;
struct inode *root;
- long err;
+ int err;
bool retry = true, need_fsck = false;
char *options = NULL;
int recovery, i, valid_super_block;
@@ -1340,6 +1507,8 @@
if (!sbi)
return -ENOMEM;
+ sbi->sb = sb;
+
/* Load the checksum driver */
sbi->s_chksum_driver = crypto_alloc_shash("crc32", 0, 0);
if (IS_ERR(sbi->s_chksum_driver)) {
@@ -1355,7 +1524,7 @@
goto free_sbi;
}
- err = read_raw_super_block(sb, &raw_super, &valid_super_block,
+ err = read_raw_super_block(sbi, &raw_super, &valid_super_block,
&recovery);
if (err)
goto free_sbi;
@@ -1390,7 +1559,6 @@
memcpy(sb->s_uuid, raw_super->uuid, sizeof(raw_super->uuid));
/* init f2fs-specific super block info */
- sbi->sb = sb;
sbi->raw_super = raw_super;
sbi->valid_super_block = valid_super_block;
mutex_init(&sbi->gc_mutex);
@@ -1415,6 +1583,10 @@
init_waitqueue_head(&sbi->cp_wait);
init_sb_info(sbi);
+ err = init_percpu_info(sbi);
+ if (err)
+ goto free_options;
+
/* get an inode for meta space */
sbi->meta_inode = f2fs_iget(sb, F2FS_META_INO(sbi));
if (IS_ERR(sbi->meta_inode)) {
@@ -1431,13 +1603,13 @@
sbi->total_valid_node_count =
le32_to_cpu(sbi->ckpt->valid_node_count);
- sbi->total_valid_inode_count =
- le32_to_cpu(sbi->ckpt->valid_inode_count);
+ percpu_counter_set(&sbi->total_valid_inode_count,
+ le32_to_cpu(sbi->ckpt->valid_inode_count));
sbi->user_block_count = le64_to_cpu(sbi->ckpt->user_block_count);
sbi->total_valid_block_count =
le64_to_cpu(sbi->ckpt->valid_block_count);
sbi->last_valid_block_count = sbi->total_valid_block_count;
- sbi->alloc_valid_block_count = 0;
+
for (i = 0; i < NR_INODE_TYPE; i++) {
INIT_LIST_HEAD(&sbi->inode_list[i]);
spin_lock_init(&sbi->inode_lock[i]);
@@ -1515,9 +1687,12 @@
if (f2fs_proc_root)
sbi->s_proc = proc_mkdir(sb->s_id, f2fs_proc_root);
- if (sbi->s_proc)
+ if (sbi->s_proc) {
proc_create_data("segment_info", S_IRUGO, sbi->s_proc,
&f2fs_seq_segment_info_fops, sb);
+ proc_create_data("segment_bits", S_IRUGO, sbi->s_proc,
+ &f2fs_seq_segment_bits_fops, sb);
+ }
sbi->s_kobj.kset = f2fs_kset;
init_completion(&sbi->s_kobj_unregister);
@@ -1541,14 +1716,24 @@
if (need_fsck)
set_sbi_flag(sbi, SBI_NEED_FSCK);
- err = recover_fsync_data(sbi);
- if (err) {
+ err = recover_fsync_data(sbi, false);
+ if (err < 0) {
need_fsck = true;
f2fs_msg(sb, KERN_ERR,
- "Cannot recover all fsync data errno=%ld", err);
+ "Cannot recover all fsync data errno=%d", err);
+ goto free_kobj;
+ }
+ } else {
+ err = recover_fsync_data(sbi, true);
+
+ if (!f2fs_readonly(sb) && err > 0) {
+ err = -EINVAL;
+ f2fs_msg(sb, KERN_ERR,
+ "Need to recover fsync data");
goto free_kobj;
}
}
+
/* recover_fsync_data() cleared this already */
clear_sbi_flag(sbi, SBI_POR_DOING);
@@ -1565,10 +1750,10 @@
kfree(options);
/* recover broken superblock */
- if (recovery && !f2fs_readonly(sb) && !bdev_read_only(sb->s_bdev)) {
+ if (recovery) {
err = f2fs_commit_super(sbi, true);
f2fs_msg(sb, KERN_INFO,
- "Try to recover %dth superblock, ret: %ld",
+ "Try to recover %dth superblock, ret: %d",
sbi->valid_super_block ? 1 : 2, err);
}
@@ -1583,6 +1768,7 @@
free_proc:
if (sbi->s_proc) {
remove_proc_entry("segment_info", sbi->s_proc);
+ remove_proc_entry("segment_bits", sbi->s_proc);
remove_proc_entry(sb->s_id, f2fs_proc_root);
}
f2fs_destroy_stats(sbi);
@@ -1603,6 +1789,7 @@
make_bad_inode(sbi->meta_inode);
iput(sbi->meta_inode);
free_options:
+ destroy_percpu_info(sbi);
kfree(options);
free_sb_buf:
kfree(raw_super);
@@ -1688,6 +1875,16 @@
err = -ENOMEM;
goto free_extent_cache;
}
+#ifdef CONFIG_F2FS_FAULT_INJECTION
+ f2fs_fault_inject.kset = f2fs_kset;
+ f2fs_build_fault_attr(0);
+ err = kobject_init_and_add(&f2fs_fault_inject, &f2fs_fault_ktype,
+ NULL, "fault_injection");
+ if (err) {
+ f2fs_fault_inject.kset = NULL;
+ goto free_kset;
+ }
+#endif
err = register_shrinker(&f2fs_shrinker_info);
if (err)
goto free_kset;
@@ -1706,6 +1903,10 @@
free_shrinker:
unregister_shrinker(&f2fs_shrinker_info);
free_kset:
+#ifdef CONFIG_F2FS_FAULT_INJECTION
+ if (f2fs_fault_inject.kset)
+ kobject_put(&f2fs_fault_inject);
+#endif
kset_unregister(f2fs_kset);
free_extent_cache:
destroy_extent_cache();
@@ -1725,14 +1926,17 @@
{
remove_proc_entry("fs/f2fs", NULL);
f2fs_destroy_root_stats();
- unregister_shrinker(&f2fs_shrinker_info);
unregister_filesystem(&f2fs_fs_type);
+ unregister_shrinker(&f2fs_shrinker_info);
+#ifdef CONFIG_F2FS_FAULT_INJECTION
+ kobject_put(&f2fs_fault_inject);
+#endif
+ kset_unregister(f2fs_kset);
destroy_extent_cache();
destroy_checkpoint_caches();
destroy_segment_manager_caches();
destroy_node_manager_caches();
destroy_inodecache();
- kset_unregister(f2fs_kset);
f2fs_destroy_trace_ios();
}
diff --git a/fs/f2fs/xattr.c b/fs/f2fs/xattr.c
index 17fd2b1..00ea567 100644
--- a/fs/f2fs/xattr.c
+++ b/fs/f2fs/xattr.c
@@ -498,7 +498,7 @@
free = free + ENTRY_SIZE(here);
if (unlikely(free < newsize)) {
- error = -ENOSPC;
+ error = -E2BIG;
goto exit;
}
}
@@ -526,7 +526,6 @@
* Before we come here, old entry is removed.
* We just write new entry.
*/
- memset(last, 0, newsize);
last->e_name_index = index;
last->e_name_len = len;
memcpy(last->e_name, name, len);
diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
index 592cea5..989a2ce 100644
--- a/fs/fs-writeback.c
+++ b/fs/fs-writeback.c
@@ -931,7 +931,8 @@
* This is WB_SYNC_NONE writeback, so if allocation fails just
* wakeup the thread for old dirty data writeback
*/
- work = kzalloc(sizeof(*work), GFP_ATOMIC);
+ work = kzalloc(sizeof(*work),
+ GFP_NOWAIT | __GFP_NOMEMALLOC | __GFP_NOWARN);
if (!work) {
trace_writeback_nowork(wb);
wb_wakeup(wb);
diff --git a/fs/gfs2/aops.c b/fs/gfs2/aops.c
index 8524c0e..37b7bc1 100644
--- a/fs/gfs2/aops.c
+++ b/fs/gfs2/aops.c
@@ -977,7 +977,7 @@
if (!list_empty(&bd->bd_list) && !buffer_pinned(bh))
list_del_init(&bd->bd_list);
else
- gfs2_remove_from_journal(bh, current->journal_info, 0);
+ gfs2_remove_from_journal(bh, REMOVE_JDATA);
}
bh->b_bdev = NULL;
clear_buffer_mapped(bh);
@@ -1063,7 +1063,7 @@
gfs2_holder_init(ip->i_gl, LM_ST_DEFERRED, 0, &gh);
rv = gfs2_glock_nq(&gh);
if (rv)
- return rv;
+ goto out_uninit;
rv = gfs2_ok_for_dio(ip, offset);
if (rv != 1)
goto out; /* dio not valid, fall back to buffered i/o */
@@ -1102,6 +1102,7 @@
gfs2_get_block_direct, NULL, NULL, 0);
out:
gfs2_glock_dq(&gh);
+out_uninit:
gfs2_holder_uninit(&gh);
return rv;
}
diff --git a/fs/gfs2/file.c b/fs/gfs2/file.c
index e53b723..e0f98e4 100644
--- a/fs/gfs2/file.c
+++ b/fs/gfs2/file.c
@@ -160,7 +160,7 @@
gfs2_holder_init(ip->i_gl, LM_ST_SHARED, 0, &gh);
error = gfs2_glock_nq(&gh);
if (error)
- return error;
+ goto out_uninit;
fsflags = fsflags_cvt(gfs2_to_fsflags, ip->i_diskflags);
if (!S_ISDIR(inode->i_mode) && ip->i_diskflags & GFS2_DIF_JDATA)
@@ -169,6 +169,7 @@
error = -EFAULT;
gfs2_glock_dq(&gh);
+out_uninit:
gfs2_holder_uninit(&gh);
return error;
}
@@ -953,6 +954,30 @@
return ret;
}
+static ssize_t gfs2_file_splice_read(struct file *in, loff_t *ppos,
+ struct pipe_inode_info *pipe, size_t len,
+ unsigned int flags)
+{
+ struct inode *inode = in->f_mapping->host;
+ struct gfs2_inode *ip = GFS2_I(inode);
+ struct gfs2_holder gh;
+ int ret;
+
+ inode_lock(inode);
+
+ ret = gfs2_glock_nq_init(ip->i_gl, LM_ST_SHARED, 0, &gh);
+ if (ret) {
+ inode_unlock(inode);
+ return ret;
+ }
+
+ gfs2_glock_dq_uninit(&gh);
+ inode_unlock(inode);
+
+ return generic_file_splice_read(in, ppos, pipe, len, flags);
+}
+
+
static ssize_t gfs2_file_splice_write(struct pipe_inode_info *pipe,
struct file *out, loff_t *ppos,
size_t len, unsigned int flags)
@@ -1115,7 +1140,7 @@
.fsync = gfs2_fsync,
.lock = gfs2_lock,
.flock = gfs2_flock,
- .splice_read = generic_file_splice_read,
+ .splice_read = gfs2_file_splice_read,
.splice_write = gfs2_file_splice_write,
.setlease = simple_nosetlease,
.fallocate = gfs2_fallocate,
@@ -1143,7 +1168,7 @@
.open = gfs2_open,
.release = gfs2_release,
.fsync = gfs2_fsync,
- .splice_read = generic_file_splice_read,
+ .splice_read = gfs2_file_splice_read,
.splice_write = gfs2_file_splice_write,
.setlease = generic_setlease,
.fallocate = gfs2_fallocate,
diff --git a/fs/gfs2/glock.c b/fs/gfs2/glock.c
index 4b73bd1..706fd93 100644
--- a/fs/gfs2/glock.c
+++ b/fs/gfs2/glock.c
@@ -218,7 +218,7 @@
*
*/
-static inline void do_error(struct gfs2_glock *gl, const int ret)
+static void do_error(struct gfs2_glock *gl, const int ret)
{
struct gfs2_holder *gh, *tmp;
@@ -475,7 +475,14 @@
if (sdp->sd_lockstruct.ls_ops->lm_lock) {
/* lock_dlm */
ret = sdp->sd_lockstruct.ls_ops->lm_lock(gl, target, lck_flags);
- if (ret) {
+ if (ret == -EINVAL && gl->gl_target == LM_ST_UNLOCKED &&
+ target == LM_ST_UNLOCKED &&
+ test_bit(SDF_SKIP_DLM_UNLOCK, &sdp->sd_flags)) {
+ finish_xmote(gl, target);
+ if (queue_delayed_work(glock_workqueue, &gl->gl_work, 0) == 0)
+ gfs2_glock_put(gl);
+ }
+ else if (ret) {
pr_err("lm_lock ret %d\n", ret);
GLOCK_BUG_ON(gl, 1);
}
diff --git a/fs/gfs2/glops.c b/fs/gfs2/glops.c
index 437fd73..5db59d4 100644
--- a/fs/gfs2/glops.c
+++ b/fs/gfs2/glops.c
@@ -286,17 +286,10 @@
static int inode_go_demote_ok(const struct gfs2_glock *gl)
{
struct gfs2_sbd *sdp = gl->gl_name.ln_sbd;
- struct gfs2_holder *gh;
if (sdp->sd_jindex == gl->gl_object || sdp->sd_rindex == gl->gl_object)
return 0;
- if (!list_empty(&gl->gl_holders)) {
- gh = list_entry(gl->gl_holders.next, struct gfs2_holder, gh_list);
- if (gh->gh_list.next != &gl->gl_holders)
- return 0;
- }
-
return 1;
}
diff --git a/fs/gfs2/inode.c b/fs/gfs2/inode.c
index 72e9c64..21dc784 100644
--- a/fs/gfs2/inode.c
+++ b/fs/gfs2/inode.c
@@ -93,12 +93,12 @@
int error;
inode = iget_locked(sb, (unsigned long)no_addr);
- ip = GFS2_I(inode);
- ip->i_no_addr = no_addr;
-
if (!inode)
return ERR_PTR(-ENOMEM);
+ ip = GFS2_I(inode);
+ ip->i_no_addr = no_addr;
+
if (inode->i_state & I_NEW) {
struct gfs2_sbd *sdp = GFS2_SB(inode);
ip->i_no_formal_ino = no_formal_ino;
diff --git a/fs/gfs2/meta_io.c b/fs/gfs2/meta_io.c
index 0448524..8eaadab 100644
--- a/fs/gfs2/meta_io.c
+++ b/fs/gfs2/meta_io.c
@@ -325,18 +325,19 @@
return 0;
}
-void gfs2_remove_from_journal(struct buffer_head *bh, struct gfs2_trans *tr, int meta)
+void gfs2_remove_from_journal(struct buffer_head *bh, int meta)
{
struct address_space *mapping = bh->b_page->mapping;
struct gfs2_sbd *sdp = gfs2_mapping2sbd(mapping);
struct gfs2_bufdata *bd = bh->b_private;
+ struct gfs2_trans *tr = current->journal_info;
int was_pinned = 0;
if (test_clear_buffer_pinned(bh)) {
trace_gfs2_pin(bd, 0);
atomic_dec(&sdp->sd_log_pinned);
list_del_init(&bd->bd_list);
- if (meta)
+ if (meta == REMOVE_META)
tr->tr_num_buf_rm++;
else
tr->tr_num_databuf_rm++;
@@ -376,7 +377,7 @@
if (bh) {
lock_buffer(bh);
gfs2_log_lock(sdp);
- gfs2_remove_from_journal(bh, current->journal_info, 1);
+ gfs2_remove_from_journal(bh, REMOVE_META);
gfs2_log_unlock(sdp);
unlock_buffer(bh);
brelse(bh);
diff --git a/fs/gfs2/meta_io.h b/fs/gfs2/meta_io.h
index c5086c8..ffdf6aa 100644
--- a/fs/gfs2/meta_io.h
+++ b/fs/gfs2/meta_io.h
@@ -57,8 +57,12 @@
extern int gfs2_meta_wait(struct gfs2_sbd *sdp, struct buffer_head *bh);
extern struct buffer_head *gfs2_getbuf(struct gfs2_glock *gl, u64 blkno,
int create);
-extern void gfs2_remove_from_journal(struct buffer_head *bh,
- struct gfs2_trans *tr, int meta);
+enum {
+ REMOVE_JDATA = 0,
+ REMOVE_META = 1,
+};
+
+extern void gfs2_remove_from_journal(struct buffer_head *bh, int meta);
extern void gfs2_meta_wipe(struct gfs2_inode *ip, u64 bstart, u32 blen);
extern int gfs2_meta_indirect_buffer(struct gfs2_inode *ip, int height, u64 num,
struct buffer_head **bhp);
diff --git a/fs/gfs2/rgrp.c b/fs/gfs2/rgrp.c
index 99a0bda..5bd2169 100644
--- a/fs/gfs2/rgrp.c
+++ b/fs/gfs2/rgrp.c
@@ -73,8 +73,7 @@
};
static int gfs2_rbm_find(struct gfs2_rbm *rbm, u8 state, u32 *minext,
- const struct gfs2_inode *ip, bool nowrap,
- const struct gfs2_alloc_parms *ap);
+ const struct gfs2_inode *ip, bool nowrap);
/**
@@ -1511,7 +1510,7 @@
if (WARN_ON(gfs2_rbm_from_block(&rbm, goal)))
return;
- ret = gfs2_rbm_find(&rbm, GFS2_BLKST_FREE, &extlen, ip, true, ap);
+ ret = gfs2_rbm_find(&rbm, GFS2_BLKST_FREE, &extlen, ip, true);
if (ret == 0) {
rs->rs_rbm = rbm;
rs->rs_free = extlen;
@@ -1638,7 +1637,6 @@
* @ip: If set, check for reservations
* @nowrap: Stop looking at the end of the rgrp, rather than wrapping
* around until we've reached the starting point.
- * @ap: the allocation parameters
*
* Side effects:
* - If looking for free blocks, we set GBF_FULL on each bitmap which
@@ -1650,8 +1648,7 @@
*/
static int gfs2_rbm_find(struct gfs2_rbm *rbm, u8 state, u32 *minext,
- const struct gfs2_inode *ip, bool nowrap,
- const struct gfs2_alloc_parms *ap)
+ const struct gfs2_inode *ip, bool nowrap)
{
struct buffer_head *bh;
int initial_bii;
@@ -1772,7 +1769,7 @@
while (1) {
down_write(&sdp->sd_log_flush_lock);
error = gfs2_rbm_find(&rbm, GFS2_BLKST_UNLINKED, NULL, NULL,
- true, NULL);
+ true);
up_write(&sdp->sd_log_flush_lock);
if (error == -ENOSPC)
break;
@@ -2329,12 +2326,11 @@
int error;
gfs2_set_alloc_start(&rbm, ip, dinode);
- error = gfs2_rbm_find(&rbm, GFS2_BLKST_FREE, NULL, ip, false, NULL);
+ error = gfs2_rbm_find(&rbm, GFS2_BLKST_FREE, NULL, ip, false);
if (error == -ENOSPC) {
gfs2_set_alloc_start(&rbm, ip, dinode);
- error = gfs2_rbm_find(&rbm, GFS2_BLKST_FREE, NULL, NULL, false,
- NULL);
+ error = gfs2_rbm_find(&rbm, GFS2_BLKST_FREE, NULL, NULL, false);
}
/* Since all blocks are reserved in advance, this shouldn't happen */
diff --git a/fs/gfs2/util.c b/fs/gfs2/util.c
index cf64583..aee4485 100644
--- a/fs/gfs2/util.c
+++ b/fs/gfs2/util.c
@@ -68,6 +68,7 @@
fs_err(sdp, "telling LM to unmount\n");
lm->lm_unmount(sdp);
}
+ set_bit(SDF_SKIP_DLM_UNLOCK, &sdp->sd_flags);
fs_err(sdp, "withdrawn\n");
dump_stack();
}
diff --git a/fs/jbd2/commit.c b/fs/jbd2/commit.c
index 2ad98d6..7007809 100644
--- a/fs/jbd2/commit.c
+++ b/fs/jbd2/commit.c
@@ -219,6 +219,8 @@
spin_lock(&journal->j_list_lock);
list_for_each_entry(jinode, &commit_transaction->t_inode_list, i_list) {
+ if (!(jinode->i_flags & JI_WRITE_DATA))
+ continue;
mapping = jinode->i_vfs_inode->i_mapping;
jinode->i_flags |= JI_COMMIT_RUNNING;
spin_unlock(&journal->j_list_lock);
@@ -256,6 +258,8 @@
/* For locking, see the comment in journal_submit_data_buffers() */
spin_lock(&journal->j_list_lock);
list_for_each_entry(jinode, &commit_transaction->t_inode_list, i_list) {
+ if (!(jinode->i_flags & JI_WAIT_DATA))
+ continue;
jinode->i_flags |= JI_COMMIT_RUNNING;
spin_unlock(&journal->j_list_lock);
err = filemap_fdatawait(jinode->i_vfs_inode->i_mapping);
diff --git a/fs/jbd2/journal.c b/fs/jbd2/journal.c
index 435f0b2..b31852f 100644
--- a/fs/jbd2/journal.c
+++ b/fs/jbd2/journal.c
@@ -94,7 +94,8 @@
EXPORT_SYMBOL(jbd2_journal_invalidatepage);
EXPORT_SYMBOL(jbd2_journal_try_to_free_buffers);
EXPORT_SYMBOL(jbd2_journal_force_commit);
-EXPORT_SYMBOL(jbd2_journal_file_inode);
+EXPORT_SYMBOL(jbd2_journal_inode_add_write);
+EXPORT_SYMBOL(jbd2_journal_inode_add_wait);
EXPORT_SYMBOL(jbd2_journal_init_jbd_inode);
EXPORT_SYMBOL(jbd2_journal_release_jbd_inode);
EXPORT_SYMBOL(jbd2_journal_begin_ordered_truncate);
diff --git a/fs/jbd2/transaction.c b/fs/jbd2/transaction.c
index 2c56c3e..1749519 100644
--- a/fs/jbd2/transaction.c
+++ b/fs/jbd2/transaction.c
@@ -2462,7 +2462,8 @@
/*
* File inode in the inode list of the handle's transaction
*/
-int jbd2_journal_file_inode(handle_t *handle, struct jbd2_inode *jinode)
+static int jbd2_journal_file_inode(handle_t *handle, struct jbd2_inode *jinode,
+ unsigned long flags)
{
transaction_t *transaction = handle->h_transaction;
journal_t *journal;
@@ -2487,12 +2488,14 @@
* and if jinode->i_next_transaction == transaction, commit code
* will only file the inode where we want it.
*/
- if (jinode->i_transaction == transaction ||
- jinode->i_next_transaction == transaction)
+ if ((jinode->i_transaction == transaction ||
+ jinode->i_next_transaction == transaction) &&
+ (jinode->i_flags & flags) == flags)
return 0;
spin_lock(&journal->j_list_lock);
-
+ jinode->i_flags |= flags;
+ /* Is inode already attached where we need it? */
if (jinode->i_transaction == transaction ||
jinode->i_next_transaction == transaction)
goto done;
@@ -2523,6 +2526,17 @@
return 0;
}
+int jbd2_journal_inode_add_write(handle_t *handle, struct jbd2_inode *jinode)
+{
+ return jbd2_journal_file_inode(handle, jinode,
+ JI_WRITE_DATA | JI_WAIT_DATA);
+}
+
+int jbd2_journal_inode_add_wait(handle_t *handle, struct jbd2_inode *jinode)
+{
+ return jbd2_journal_file_inode(handle, jinode, JI_WAIT_DATA);
+}
+
/*
* File truncate and transaction commit interact with each other in a
* non-trivial way. If a transaction writing data block A is
diff --git a/fs/kernfs/dir.c b/fs/kernfs/dir.c
index 68a4431..8a65240 100644
--- a/fs/kernfs/dir.c
+++ b/fs/kernfs/dir.c
@@ -753,7 +753,8 @@
ps_iattr = parent->iattr;
if (ps_iattr) {
struct iattr *ps_iattrs = &ps_iattr->ia_iattr;
- ps_iattrs->ia_ctime = ps_iattrs->ia_mtime = CURRENT_TIME;
+ ktime_get_real_ts(&ps_iattrs->ia_ctime);
+ ps_iattrs->ia_mtime = ps_iattrs->ia_ctime;
}
mutex_unlock(&kernfs_mutex);
@@ -1279,8 +1280,9 @@
/* update timestamps on the parent */
if (ps_iattr) {
- ps_iattr->ia_iattr.ia_ctime = CURRENT_TIME;
- ps_iattr->ia_iattr.ia_mtime = CURRENT_TIME;
+ ktime_get_real_ts(&ps_iattr->ia_iattr.ia_ctime);
+ ps_iattr->ia_iattr.ia_mtime =
+ ps_iattr->ia_iattr.ia_ctime;
}
kernfs_put(pos);
diff --git a/fs/kernfs/file.c b/fs/kernfs/file.c
index 7247252..e157400 100644
--- a/fs/kernfs/file.c
+++ b/fs/kernfs/file.c
@@ -190,15 +190,16 @@
char *buf;
buf = of->prealloc_buf;
- if (!buf)
+ if (buf)
+ mutex_lock(&of->prealloc_mutex);
+ else
buf = kmalloc(len, GFP_KERNEL);
if (!buf)
return -ENOMEM;
/*
* @of->mutex nests outside active ref and is used both to ensure that
- * the ops aren't called concurrently for the same open file, and
- * to provide exclusive access to ->prealloc_buf (when that exists).
+ * the ops aren't called concurrently for the same open file.
*/
mutex_lock(&of->mutex);
if (!kernfs_get_active(of->kn)) {
@@ -214,21 +215,23 @@
else
len = -EINVAL;
+ kernfs_put_active(of->kn);
+ mutex_unlock(&of->mutex);
+
if (len < 0)
- goto out_unlock;
+ goto out_free;
if (copy_to_user(user_buf, buf, len)) {
len = -EFAULT;
- goto out_unlock;
+ goto out_free;
}
*ppos += len;
- out_unlock:
- kernfs_put_active(of->kn);
- mutex_unlock(&of->mutex);
out_free:
- if (buf != of->prealloc_buf)
+ if (buf == of->prealloc_buf)
+ mutex_unlock(&of->prealloc_mutex);
+ else
kfree(buf);
return len;
}
@@ -284,15 +287,22 @@
}
buf = of->prealloc_buf;
- if (!buf)
+ if (buf)
+ mutex_lock(&of->prealloc_mutex);
+ else
buf = kmalloc(len + 1, GFP_KERNEL);
if (!buf)
return -ENOMEM;
+ if (copy_from_user(buf, user_buf, len)) {
+ len = -EFAULT;
+ goto out_free;
+ }
+ buf[len] = '\0'; /* guarantee string termination */
+
/*
* @of->mutex nests outside active ref and is used both to ensure that
- * the ops aren't called concurrently for the same open file, and
- * to provide exclusive access to ->prealloc_buf (when that exists).
+ * the ops aren't called concurrently for the same open file.
*/
mutex_lock(&of->mutex);
if (!kernfs_get_active(of->kn)) {
@@ -301,26 +311,22 @@
goto out_free;
}
- if (copy_from_user(buf, user_buf, len)) {
- len = -EFAULT;
- goto out_unlock;
- }
- buf[len] = '\0'; /* guarantee string termination */
-
ops = kernfs_ops(of->kn);
if (ops->write)
len = ops->write(of, buf, len, *ppos);
else
len = -EINVAL;
+ kernfs_put_active(of->kn);
+ mutex_unlock(&of->mutex);
+
if (len > 0)
*ppos += len;
-out_unlock:
- kernfs_put_active(of->kn);
- mutex_unlock(&of->mutex);
out_free:
- if (buf != of->prealloc_buf)
+ if (buf == of->prealloc_buf)
+ mutex_unlock(&of->prealloc_mutex);
+ else
kfree(buf);
return len;
}
@@ -687,6 +693,7 @@
error = -ENOMEM;
if (!of->prealloc_buf)
goto err_free;
+ mutex_init(&of->prealloc_mutex);
}
/*
diff --git a/fs/kernfs/inode.c b/fs/kernfs/inode.c
index b524722..1719649 100644
--- a/fs/kernfs/inode.c
+++ b/fs/kernfs/inode.c
@@ -54,7 +54,10 @@
iattrs->ia_mode = kn->mode;
iattrs->ia_uid = GLOBAL_ROOT_UID;
iattrs->ia_gid = GLOBAL_ROOT_GID;
- iattrs->ia_atime = iattrs->ia_mtime = iattrs->ia_ctime = CURRENT_TIME;
+
+ ktime_get_real_ts(&iattrs->ia_atime);
+ iattrs->ia_mtime = iattrs->ia_atime;
+ iattrs->ia_ctime = iattrs->ia_atime;
simple_xattrs_init(&kn->iattr->xattrs);
out_unlock:
@@ -236,16 +239,18 @@
static inline void set_default_inode_attr(struct inode *inode, umode_t mode)
{
inode->i_mode = mode;
- inode->i_atime = inode->i_mtime = inode->i_ctime = CURRENT_TIME;
+ inode->i_atime = inode->i_mtime =
+ inode->i_ctime = current_fs_time(inode->i_sb);
}
static inline void set_inode_attr(struct inode *inode, struct iattr *iattr)
{
+ struct super_block *sb = inode->i_sb;
inode->i_uid = iattr->ia_uid;
inode->i_gid = iattr->ia_gid;
- inode->i_atime = iattr->ia_atime;
- inode->i_mtime = iattr->ia_mtime;
- inode->i_ctime = iattr->ia_ctime;
+ inode->i_atime = timespec_trunc(iattr->ia_atime, sb->s_time_gran);
+ inode->i_mtime = timespec_trunc(iattr->ia_mtime, sb->s_time_gran);
+ inode->i_ctime = timespec_trunc(iattr->ia_ctime, sb->s_time_gran);
}
static void kernfs_refresh_inode(struct kernfs_node *kn, struct inode *inode)
diff --git a/fs/namei.c b/fs/namei.c
index 9d193d3..5375571 100644
--- a/fs/namei.c
+++ b/fs/namei.c
@@ -3627,6 +3627,8 @@
switch (mode & S_IFMT) {
case 0: case S_IFREG:
error = vfs_create(path.dentry->d_inode,dentry,mode,true);
+ if (!error)
+ ima_post_path_mknod(dentry);
break;
case S_IFCHR: case S_IFBLK:
error = vfs_mknod(path.dentry->d_inode,dentry,mode,
diff --git a/fs/nfs/nfs4idmap.c b/fs/nfs/nfs4idmap.c
index 5ba22c6..c444285 100644
--- a/fs/nfs/nfs4idmap.c
+++ b/fs/nfs/nfs4idmap.c
@@ -201,7 +201,7 @@
GLOBAL_ROOT_UID, GLOBAL_ROOT_GID, cred,
(KEY_POS_ALL & ~KEY_POS_SETATTR) |
KEY_USR_VIEW | KEY_USR_READ,
- KEY_ALLOC_NOT_IN_QUOTA, NULL);
+ KEY_ALLOC_NOT_IN_QUOTA, NULL, NULL);
if (IS_ERR(keyring)) {
ret = PTR_ERR(keyring);
goto failed_put_cred;
diff --git a/fs/nfsd/nfs3xdr.c b/fs/nfsd/nfs3xdr.c
index 93d5853..dba2ff8 100644
--- a/fs/nfsd/nfs3xdr.c
+++ b/fs/nfsd/nfs3xdr.c
@@ -379,7 +379,7 @@
*/
hdr = (void*)p - rqstp->rq_arg.head[0].iov_base;
dlen = rqstp->rq_arg.head[0].iov_len + rqstp->rq_arg.page_len
- - hdr;
+ + rqstp->rq_arg.tail[0].iov_len - hdr;
/*
* Round the length of the data which was specified up to
* the next multiple of XDR units and then compare that
diff --git a/fs/nfsd/nfs4layouts.c b/fs/nfsd/nfs4layouts.c
index 825c7bc..953c075 100644
--- a/fs/nfsd/nfs4layouts.c
+++ b/fs/nfsd/nfs4layouts.c
@@ -289,7 +289,7 @@
status = nfserr_bad_stateid;
mutex_lock(&ls->ls_mutex);
- if (stateid->si_generation > stid->sc_stateid.si_generation)
+ if (nfsd4_stateid_generation_after(stateid, &stid->sc_stateid))
goto out_unlock_stid;
if (layout_type != ls->ls_layout_type)
goto out_unlock_stid;
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 0462eed..f5f82e1 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -4651,12 +4651,6 @@
return opens_in_grace(net) && mandatory_lock(inode);
}
-/* Returns true iff a is later than b: */
-static bool stateid_generation_after(stateid_t *a, stateid_t *b)
-{
- return (s32)(a->si_generation - b->si_generation) > 0;
-}
-
static __be32 check_stateid_generation(stateid_t *in, stateid_t *ref, bool has_session)
{
/*
@@ -4670,7 +4664,7 @@
return nfs_ok;
/* If the client sends us a stateid from the future, it's buggy: */
- if (stateid_generation_after(in, ref))
+ if (nfsd4_stateid_generation_after(in, ref))
return nfserr_bad_stateid;
/*
* However, we could see a stateid from the past, even from a
diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h
index c050c53..986e51e 100644
--- a/fs/nfsd/state.h
+++ b/fs/nfsd/state.h
@@ -573,6 +573,11 @@
NFSPROC4_CLNT_CB_SEQUENCE,
};
+/* Returns true iff a is later than b: */
+static inline bool nfsd4_stateid_generation_after(stateid_t *a, stateid_t *b)
+{
+ return (s32)(a->si_generation - b->si_generation) > 0;
+}
struct nfsd4_compound_state;
struct nfsd_net;
diff --git a/fs/nilfs2/alloc.c b/fs/nilfs2/alloc.c
index 2ccbf55..1a85d94 100644
--- a/fs/nilfs2/alloc.c
+++ b/fs/nilfs2/alloc.c
@@ -13,13 +13,8 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Original code was written by Koji Sato <koji@osrg.net>.
- * Two allocators were unified by Ryusuke Konishi <ryusuke@osrg.net>,
- * Amagai Yoshiji <amagai@osrg.net>.
+ * Originally written by Koji Sato.
+ * Two allocators were unified by Ryusuke Konishi and Amagai Yoshiji.
*/
#include <linux/types.h>
@@ -58,7 +53,7 @@
* @inode: inode of metadata file using this allocator
* @entry_size: size of the persistent object
*/
-int nilfs_palloc_init_blockgroup(struct inode *inode, unsigned entry_size)
+int nilfs_palloc_init_blockgroup(struct inode *inode, unsigned int entry_size)
{
struct nilfs_mdt_info *mi = NILFS_MDT(inode);
@@ -73,13 +68,17 @@
mi->mi_blocks_per_group =
DIV_ROUND_UP(nilfs_palloc_entries_per_group(inode),
mi->mi_entries_per_block) + 1;
- /* Number of blocks in a group including entry blocks and
- a bitmap block */
+ /*
+ * Number of blocks in a group including entry blocks
+ * and a bitmap block
+ */
mi->mi_blocks_per_desc_block =
nilfs_palloc_groups_per_desc_block(inode) *
mi->mi_blocks_per_group + 1;
- /* Number of blocks per descriptor including the
- descriptor block */
+ /*
+ * Number of blocks per descriptor including the
+ * descriptor block
+ */
return 0;
}
@@ -389,7 +388,7 @@
*/
static int nilfs_palloc_find_available_slot(unsigned char *bitmap,
unsigned long target,
- unsigned bsize,
+ unsigned int bsize,
spinlock_t *lock)
{
int pos, end = bsize;
@@ -624,7 +623,7 @@
if (!nilfs_clear_bit_atomic(lock, group_offset, bitmap))
nilfs_warning(inode->i_sb, __func__,
- "entry number %llu already freed: ino=%lu\n",
+ "entry number %llu already freed: ino=%lu",
(unsigned long long)req->pr_entry_nr,
(unsigned long)inode->i_ino);
else
@@ -665,7 +664,7 @@
if (!nilfs_clear_bit_atomic(lock, group_offset, bitmap))
nilfs_warning(inode->i_sb, __func__,
- "entry number %llu already freed: ino=%lu\n",
+ "entry number %llu already freed: ino=%lu",
(unsigned long long)req->pr_entry_nr,
(unsigned long)inode->i_ino);
else
@@ -740,8 +739,8 @@
unsigned long group, group_offset;
__u64 group_min_nr, last_nrs[8];
const unsigned long epg = nilfs_palloc_entries_per_group(inode);
- const unsigned epb = NILFS_MDT(inode)->mi_entries_per_block;
- unsigned entry_start, end, pos;
+ const unsigned int epb = NILFS_MDT(inode)->mi_entries_per_block;
+ unsigned int entry_start, end, pos;
spinlock_t *lock;
int i, j, k, ret;
u32 nfree;
@@ -774,7 +773,7 @@
if (!nilfs_clear_bit_atomic(lock, group_offset,
bitmap)) {
nilfs_warning(inode->i_sb, __func__,
- "entry number %llu already freed: ino=%lu\n",
+ "entry number %llu already freed: ino=%lu",
(unsigned long long)entry_nrs[j],
(unsigned long)inode->i_ino);
} else {
@@ -819,7 +818,7 @@
last_nrs[k]);
if (ret && ret != -ENOENT) {
nilfs_warning(inode->i_sb, __func__,
- "failed to delete block of entry %llu: ino=%lu, err=%d\n",
+ "failed to delete block of entry %llu: ino=%lu, err=%d",
(unsigned long long)last_nrs[k],
(unsigned long)inode->i_ino, ret);
}
@@ -838,7 +837,7 @@
ret = nilfs_palloc_delete_bitmap_block(inode, group);
if (ret && ret != -ENOENT) {
nilfs_warning(inode->i_sb, __func__,
- "failed to delete bitmap block of group %lu: ino=%lu, err=%d\n",
+ "failed to delete bitmap block of group %lu: ino=%lu, err=%d",
group,
(unsigned long)inode->i_ino, ret);
}
diff --git a/fs/nilfs2/alloc.h b/fs/nilfs2/alloc.h
index 6e6f49a..05149e6 100644
--- a/fs/nilfs2/alloc.h
+++ b/fs/nilfs2/alloc.h
@@ -13,13 +13,8 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Original code was written by Koji Sato <koji@osrg.net>.
- * Two allocators were unified by Ryusuke Konishi <ryusuke@osrg.net>,
- * Amagai Yoshiji <amagai@osrg.net>.
+ * Originally written by Koji Sato.
+ * Two allocators were unified by Ryusuke Konishi and Amagai Yoshiji.
*/
#ifndef _NILFS_ALLOC_H
@@ -42,7 +37,7 @@
return 1UL << (inode->i_blkbits + 3 /* log2(8 = CHAR_BITS) */);
}
-int nilfs_palloc_init_blockgroup(struct inode *, unsigned);
+int nilfs_palloc_init_blockgroup(struct inode *, unsigned int);
int nilfs_palloc_get_entry_block(struct inode *, __u64, int,
struct buffer_head **);
void *nilfs_palloc_block_get_entry(const struct inode *, __u64,
diff --git a/fs/nilfs2/bmap.c b/fs/nilfs2/bmap.c
index a9fb363..f2a7877 100644
--- a/fs/nilfs2/bmap.c
+++ b/fs/nilfs2/bmap.c
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Koji Sato <koji@osrg.net>.
+ * Written by Koji Sato.
*/
#include <linux/fs.h>
@@ -46,7 +42,7 @@
if (err == -EINVAL) {
nilfs_error(inode->i_sb, fname,
- "broken bmap (inode number=%lu)\n", inode->i_ino);
+ "broken bmap (inode number=%lu)", inode->i_ino);
err = -EIO;
}
return err;
@@ -97,7 +93,7 @@
}
int nilfs_bmap_lookup_contig(struct nilfs_bmap *bmap, __u64 key, __u64 *ptrp,
- unsigned maxblocks)
+ unsigned int maxblocks)
{
int ret;
diff --git a/fs/nilfs2/bmap.h b/fs/nilfs2/bmap.h
index bfa817c..b6a4c8f9 100644
--- a/fs/nilfs2/bmap.h
+++ b/fs/nilfs2/bmap.h
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Koji Sato <koji@osrg.net>.
+ * Written by Koji Sato.
*/
#ifndef _NILFS_BMAP_H
@@ -61,7 +57,7 @@
struct nilfs_bmap_operations {
int (*bop_lookup)(const struct nilfs_bmap *, __u64, int, __u64 *);
int (*bop_lookup_contig)(const struct nilfs_bmap *, __u64, __u64 *,
- unsigned);
+ unsigned int);
int (*bop_insert)(struct nilfs_bmap *, __u64, __u64);
int (*bop_delete)(struct nilfs_bmap *, __u64);
void (*bop_clear)(struct nilfs_bmap *);
@@ -126,10 +122,14 @@
/* pointer type */
#define NILFS_BMAP_PTR_P 0 /* physical block number (i.e. LBN) */
-#define NILFS_BMAP_PTR_VS 1 /* virtual block number (single
- version) */
-#define NILFS_BMAP_PTR_VM 2 /* virtual block number (has multiple
- versions) */
+#define NILFS_BMAP_PTR_VS 1 /*
+ * virtual block number (single
+ * version)
+ */
+#define NILFS_BMAP_PTR_VM 2 /*
+ * virtual block number (has multiple
+ * versions)
+ */
#define NILFS_BMAP_PTR_U (-1) /* never perform pointer operations */
#define NILFS_BMAP_USE_VBN(bmap) ((bmap)->b_ptr_type > 0)
@@ -154,7 +154,7 @@
int nilfs_bmap_test_and_clear_dirty(struct nilfs_bmap *);
int nilfs_bmap_read(struct nilfs_bmap *, struct nilfs_inode *);
void nilfs_bmap_write(struct nilfs_bmap *, struct nilfs_inode *);
-int nilfs_bmap_lookup_contig(struct nilfs_bmap *, __u64, __u64 *, unsigned);
+int nilfs_bmap_lookup_contig(struct nilfs_bmap *, __u64, __u64 *, unsigned int);
int nilfs_bmap_insert(struct nilfs_bmap *bmap, __u64 key, unsigned long rec);
int nilfs_bmap_delete(struct nilfs_bmap *bmap, __u64 key);
int nilfs_bmap_seek_key(struct nilfs_bmap *bmap, __u64 start, __u64 *keyp);
diff --git a/fs/nilfs2/btnode.c b/fs/nilfs2/btnode.c
index e0c9daf..0576033 100644
--- a/fs/nilfs2/btnode.c
+++ b/fs/nilfs2/btnode.c
@@ -13,13 +13,8 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * This file was originally written by Seiji Kihara <kihara@osrg.net>
- * and fully revised by Ryusuke Konishi <ryusuke@osrg.net> for
- * stabilization and simplification.
+ * Originally written by Seiji Kihara.
+ * Fully revised by Ryusuke Konishi for stabilization and simplification.
*
*/
diff --git a/fs/nilfs2/btnode.h b/fs/nilfs2/btnode.h
index d876b56..2cc1b80 100644
--- a/fs/nilfs2/btnode.h
+++ b/fs/nilfs2/btnode.h
@@ -13,12 +13,8 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Seiji Kihara <kihara@osrg.net>
- * Revised by Ryusuke Konishi <ryusuke@osrg.net>
+ * Written by Seiji Kihara.
+ * Revised by Ryusuke Konishi.
*/
#ifndef _NILFS_BTNODE_H
diff --git a/fs/nilfs2/btree.c b/fs/nilfs2/btree.c
index 3a3821b..eccb1c8 100644
--- a/fs/nilfs2/btree.c
+++ b/fs/nilfs2/btree.c
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Koji Sato <koji@osrg.net>.
+ * Written by Koji Sato.
*/
#include <linux/slab.h>
@@ -689,7 +685,8 @@
}
static int nilfs_btree_lookup_contig(const struct nilfs_bmap *btree,
- __u64 key, __u64 *ptrp, unsigned maxblocks)
+ __u64 key, __u64 *ptrp,
+ unsigned int maxblocks)
{
struct nilfs_btree_path *path;
struct nilfs_btree_node *node;
@@ -1032,12 +1029,12 @@
if (ptr != NILFS_BMAP_INVALID_PTR)
/* sequential access */
return ptr;
- else {
- ptr = nilfs_btree_find_near(btree, path);
- if (ptr != NILFS_BMAP_INVALID_PTR)
- /* near */
- return ptr;
- }
+
+ ptr = nilfs_btree_find_near(btree, path);
+ if (ptr != NILFS_BMAP_INVALID_PTR)
+ /* near */
+ return ptr;
+
/* block group */
return nilfs_bmap_find_target_in_group(btree);
}
diff --git a/fs/nilfs2/btree.h b/fs/nilfs2/btree.h
index 22c02e3..df1a25f 100644
--- a/fs/nilfs2/btree.h
+++ b/fs/nilfs2/btree.h
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Koji Sato <koji@osrg.net>.
+ * Written by Koji Sato.
*/
#ifndef _NILFS_BTREE_H
diff --git a/fs/nilfs2/cpfile.c b/fs/nilfs2/cpfile.c
index b6596ca..8a3d3b6 100644
--- a/fs/nilfs2/cpfile.c
+++ b/fs/nilfs2/cpfile.c
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Koji Sato <koji@osrg.net>.
+ * Written by Koji Sato.
*/
#include <linux/kernel.h>
@@ -41,6 +37,7 @@
nilfs_cpfile_get_blkoff(const struct inode *cpfile, __u64 cno)
{
__u64 tcno = cno + NILFS_MDT(cpfile)->mi_first_entry_offset - 1;
+
do_div(tcno, nilfs_cpfile_checkpoints_per_block(cpfile));
return (unsigned long)tcno;
}
@@ -50,6 +47,7 @@
nilfs_cpfile_get_offset(const struct inode *cpfile, __u64 cno)
{
__u64 tcno = cno + NILFS_MDT(cpfile)->mi_first_entry_offset - 1;
+
return do_div(tcno, nilfs_cpfile_checkpoints_per_block(cpfile));
}
@@ -433,7 +431,8 @@
}
static ssize_t nilfs_cpfile_do_get_cpinfo(struct inode *cpfile, __u64 *cnop,
- void *buf, unsigned cisz, size_t nci)
+ void *buf, unsigned int cisz,
+ size_t nci)
{
struct nilfs_checkpoint *cp;
struct nilfs_cpinfo *ci = buf;
@@ -484,7 +483,8 @@
}
static ssize_t nilfs_cpfile_do_get_ssinfo(struct inode *cpfile, __u64 *cnop,
- void *buf, unsigned cisz, size_t nci)
+ void *buf, unsigned int cisz,
+ size_t nci)
{
struct buffer_head *bh;
struct nilfs_cpfile_header *header;
@@ -570,7 +570,7 @@
*/
ssize_t nilfs_cpfile_get_cpinfo(struct inode *cpfile, __u64 *cnop, int mode,
- void *buf, unsigned cisz, size_t nci)
+ void *buf, unsigned int cisz, size_t nci)
{
switch (mode) {
case NILFS_CHECKPOINT:
@@ -870,8 +870,10 @@
void *kaddr;
int ret;
- /* CP number is invalid if it's zero or larger than the
- largest exist one.*/
+ /*
+ * CP number is invalid if it's zero or larger than the
+ * largest existing one.
+ */
if (cno == 0 || cno >= nilfs_mdt_cno(cpfile))
return -ENOENT;
down_read(&NILFS_MDT(cpfile)->mi_sem);
diff --git a/fs/nilfs2/cpfile.h b/fs/nilfs2/cpfile.h
index a242b9a..0249744 100644
--- a/fs/nilfs2/cpfile.h
+++ b/fs/nilfs2/cpfile.h
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Koji Sato <koji@osrg.net>.
+ * Written by Koji Sato.
*/
#ifndef _NILFS_CPFILE_H
@@ -37,8 +33,8 @@
int nilfs_cpfile_change_cpmode(struct inode *, __u64, int);
int nilfs_cpfile_is_snapshot(struct inode *, __u64);
int nilfs_cpfile_get_stat(struct inode *, struct nilfs_cpstat *);
-ssize_t nilfs_cpfile_get_cpinfo(struct inode *, __u64 *, int, void *, unsigned,
- size_t);
+ssize_t nilfs_cpfile_get_cpinfo(struct inode *, __u64 *, int, void *,
+ unsigned int, size_t);
int nilfs_cpfile_read(struct super_block *sb, size_t cpsize,
struct nilfs_inode *raw_inode, struct inode **inodep);
diff --git a/fs/nilfs2/dat.c b/fs/nilfs2/dat.c
index 7dc23f1..7367610 100644
--- a/fs/nilfs2/dat.c
+++ b/fs/nilfs2/dat.c
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Koji Sato <koji@osrg.net>.
+ * Written by Koji Sato.
*/
#include <linux/types.h>
@@ -428,7 +424,7 @@
return ret;
}
-ssize_t nilfs_dat_get_vinfo(struct inode *dat, void *buf, unsigned visz,
+ssize_t nilfs_dat_get_vinfo(struct inode *dat, void *buf, unsigned int visz,
size_t nvi)
{
struct buffer_head *entry_bh;
diff --git a/fs/nilfs2/dat.h b/fs/nilfs2/dat.h
index cbd8e97..abbfdab 100644
--- a/fs/nilfs2/dat.h
+++ b/fs/nilfs2/dat.h
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Koji Sato <koji@osrg.net>.
+ * Written by Koji Sato.
*/
#ifndef _NILFS_DAT_H
@@ -51,7 +47,7 @@
int nilfs_dat_mark_dirty(struct inode *, __u64);
int nilfs_dat_freev(struct inode *, __u64 *, size_t);
int nilfs_dat_move(struct inode *, __u64, sector_t);
-ssize_t nilfs_dat_get_vinfo(struct inode *, void *, unsigned, size_t);
+ssize_t nilfs_dat_get_vinfo(struct inode *, void *, unsigned int, size_t);
int nilfs_dat_read(struct super_block *sb, size_t entry_size,
struct nilfs_inode *raw_inode, struct inode **inodep);
diff --git a/fs/nilfs2/dir.c b/fs/nilfs2/dir.c
index 6723d45..e506f4f 100644
--- a/fs/nilfs2/dir.c
+++ b/fs/nilfs2/dir.c
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Modified for NILFS by Amagai Yoshiji <amagai@osrg.net>
+ * Modified for NILFS by Amagai Yoshiji.
*/
/*
* linux/fs/ext2/dir.c
@@ -50,7 +46,7 @@
* nilfs uses block-sized chunks. Arguably, sector-sized ones would be
* more robust, but we have what we have
*/
-static inline unsigned nilfs_chunk_size(struct inode *inode)
+static inline unsigned int nilfs_chunk_size(struct inode *inode)
{
return inode->i_sb->s_blocksize;
}
@@ -65,9 +61,9 @@
* Return the offset into page `page_nr' of the last valid
* byte in that page, plus one.
*/
-static unsigned nilfs_last_byte(struct inode *inode, unsigned long page_nr)
+static unsigned int nilfs_last_byte(struct inode *inode, unsigned long page_nr)
{
- unsigned last_byte = inode->i_size;
+ unsigned int last_byte = inode->i_size;
last_byte -= page_nr << PAGE_SHIFT;
if (last_byte > PAGE_SIZE)
@@ -75,20 +71,22 @@
return last_byte;
}
-static int nilfs_prepare_chunk(struct page *page, unsigned from, unsigned to)
+static int nilfs_prepare_chunk(struct page *page, unsigned int from,
+ unsigned int to)
{
loff_t pos = page_offset(page) + from;
+
return __block_write_begin(page, pos, to - from, nilfs_get_block);
}
static void nilfs_commit_chunk(struct page *page,
struct address_space *mapping,
- unsigned from, unsigned to)
+ unsigned int from, unsigned int to)
{
struct inode *dir = mapping->host;
loff_t pos = page_offset(page) + from;
- unsigned len = to - from;
- unsigned nr_dirty, copied;
+ unsigned int len = to - from;
+ unsigned int nr_dirty, copied;
int err;
nr_dirty = nilfs_page_count_clean_buffers(page, from, to);
@@ -106,10 +104,10 @@
{
struct inode *dir = page->mapping->host;
struct super_block *sb = dir->i_sb;
- unsigned chunk_size = nilfs_chunk_size(dir);
+ unsigned int chunk_size = nilfs_chunk_size(dir);
char *kaddr = page_address(page);
- unsigned offs, rec_len;
- unsigned limit = PAGE_SIZE;
+ unsigned int offs, rec_len;
+ unsigned int limit = PAGE_SIZE;
struct nilfs_dir_entry *p;
char *error;
@@ -259,7 +257,6 @@
unsigned int offset = pos & ~PAGE_MASK;
unsigned long n = pos >> PAGE_SHIFT;
unsigned long npages = dir_pages(inode);
-/* unsigned chunk_mask = ~(nilfs_chunk_size(inode)-1); */
if (pos > inode->i_size - NILFS_DIR_REC_LEN(1))
return 0;
@@ -321,7 +318,7 @@
{
const unsigned char *name = qstr->name;
int namelen = qstr->len;
- unsigned reclen = NILFS_DIR_REC_LEN(namelen);
+ unsigned int reclen = NILFS_DIR_REC_LEN(namelen);
unsigned long start, n;
unsigned long npages = dir_pages(dir);
struct page *page = NULL;
@@ -340,6 +337,7 @@
n = start;
do {
char *kaddr;
+
page = nilfs_get_page(dir, n);
if (!IS_ERR(page)) {
kaddr = page_address(page);
@@ -410,8 +408,8 @@
void nilfs_set_link(struct inode *dir, struct nilfs_dir_entry *de,
struct page *page, struct inode *inode)
{
- unsigned from = (char *) de - (char *) page_address(page);
- unsigned to = from + nilfs_rec_len_from_disk(de->rec_len);
+ unsigned int from = (char *)de - (char *)page_address(page);
+ unsigned int to = from + nilfs_rec_len_from_disk(de->rec_len);
struct address_space *mapping = page->mapping;
int err;
@@ -433,15 +431,15 @@
struct inode *dir = d_inode(dentry->d_parent);
const unsigned char *name = dentry->d_name.name;
int namelen = dentry->d_name.len;
- unsigned chunk_size = nilfs_chunk_size(dir);
- unsigned reclen = NILFS_DIR_REC_LEN(namelen);
+ unsigned int chunk_size = nilfs_chunk_size(dir);
+ unsigned int reclen = NILFS_DIR_REC_LEN(namelen);
unsigned short rec_len, name_len;
struct page *page = NULL;
struct nilfs_dir_entry *de;
unsigned long npages = dir_pages(dir);
unsigned long n;
char *kaddr;
- unsigned from, to;
+ unsigned int from, to;
int err;
/*
@@ -533,13 +531,14 @@
struct address_space *mapping = page->mapping;
struct inode *inode = mapping->host;
char *kaddr = page_address(page);
- unsigned from = ((char *)dir - kaddr) & ~(nilfs_chunk_size(inode) - 1);
- unsigned to = ((char *)dir - kaddr) +
- nilfs_rec_len_from_disk(dir->rec_len);
- struct nilfs_dir_entry *pde = NULL;
- struct nilfs_dir_entry *de = (struct nilfs_dir_entry *)(kaddr + from);
+ unsigned int from, to;
+ struct nilfs_dir_entry *de, *pde = NULL;
int err;
+ from = ((char *)dir - kaddr) & ~(nilfs_chunk_size(inode) - 1);
+ to = ((char *)dir - kaddr) + nilfs_rec_len_from_disk(dir->rec_len);
+ de = (struct nilfs_dir_entry *)(kaddr + from);
+
while ((char *)de < (char *)dir) {
if (de->rec_len == 0) {
nilfs_error(inode->i_sb, __func__,
@@ -572,7 +571,7 @@
{
struct address_space *mapping = inode->i_mapping;
struct page *page = grab_cache_page(mapping, 0);
- unsigned chunk_size = nilfs_chunk_size(inode);
+ unsigned int chunk_size = nilfs_chunk_size(inode);
struct nilfs_dir_entry *de;
int err;
void *kaddr;
@@ -630,8 +629,8 @@
while ((char *)de <= kaddr) {
if (de->rec_len == 0) {
nilfs_error(inode->i_sb, __func__,
- "zero-length directory entry "
- "(kaddr=%p, de=%p)\n", kaddr, de);
+ "zero-length directory entry (kaddr=%p, de=%p)",
+ kaddr, de);
goto not_empty;
}
if (de->inode != 0) {
diff --git a/fs/nilfs2/direct.c b/fs/nilfs2/direct.c
index ebf89fd..251a4492 100644
--- a/fs/nilfs2/direct.c
+++ b/fs/nilfs2/direct.c
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Koji Sato <koji@osrg.net>.
+ * Written by Koji Sato.
*/
#include <linux/errno.h>
@@ -62,7 +58,7 @@
static int nilfs_direct_lookup_contig(const struct nilfs_bmap *direct,
__u64 key, __u64 *ptrp,
- unsigned maxblocks)
+ unsigned int maxblocks)
{
struct inode *dat = NULL;
__u64 ptr, ptr2;
@@ -83,7 +79,8 @@
ptr = blocknr;
}
- maxblocks = min_t(unsigned, maxblocks, NILFS_DIRECT_KEY_MAX - key + 1);
+ maxblocks = min_t(unsigned int, maxblocks,
+ NILFS_DIRECT_KEY_MAX - key + 1);
for (cnt = 1; cnt < maxblocks &&
(ptr2 = nilfs_direct_get_ptr(direct, key + cnt)) !=
NILFS_BMAP_INVALID_PTR;
@@ -110,9 +107,9 @@
if (ptr != NILFS_BMAP_INVALID_PTR)
/* sequential access */
return ptr;
- else
- /* block group */
- return nilfs_bmap_find_target_in_group(direct);
+
+ /* block group */
+ return nilfs_bmap_find_target_in_group(direct);
}
static int nilfs_direct_insert(struct nilfs_bmap *bmap, __u64 key, __u64 ptr)
diff --git a/fs/nilfs2/direct.h b/fs/nilfs2/direct.h
index dc643de..3015a6e 100644
--- a/fs/nilfs2/direct.h
+++ b/fs/nilfs2/direct.h
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Koji Sato <koji@osrg.net>.
+ * Written by Koji Sato.
*/
#ifndef _NILFS_DIRECT_H
diff --git a/fs/nilfs2/export.h b/fs/nilfs2/export.h
index 19ccbf9..00107fd 100644
--- a/fs/nilfs2/export.h
+++ b/fs/nilfs2/export.h
@@ -20,6 +20,6 @@
u32 parent_gen;
u64 parent_ino;
-} __attribute__ ((packed));
+} __packed;
#endif
diff --git a/fs/nilfs2/file.c b/fs/nilfs2/file.c
index 088ba00..547381f 100644
--- a/fs/nilfs2/file.c
+++ b/fs/nilfs2/file.c
@@ -13,12 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Amagai Yoshiji <amagai@osrg.net>,
- * Ryusuke Konishi <ryusuke@osrg.net>
+ * Written by Amagai Yoshiji and Ryusuke Konishi.
*/
#include <linux/fs.h>
diff --git a/fs/nilfs2/gcinode.c b/fs/nilfs2/gcinode.c
index 0224b78..693aded 100644
--- a/fs/nilfs2/gcinode.c
+++ b/fs/nilfs2/gcinode.c
@@ -13,13 +13,8 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Seiji Kihara <kihara@osrg.net>, Amagai Yoshiji <amagai@osrg.net>,
- * and Ryusuke Konishi <ryusuke@osrg.net>.
- * Revised by Ryusuke Konishi <ryusuke@osrg.net>.
+ * Written by Seiji Kihara, Amagai Yoshiji, and Ryusuke Konishi.
+ * Revised by Ryusuke Konishi.
*
*/
/*
diff --git a/fs/nilfs2/ifile.c b/fs/nilfs2/ifile.c
index 6548c78..1d2b180 100644
--- a/fs/nilfs2/ifile.c
+++ b/fs/nilfs2/ifile.c
@@ -13,12 +13,8 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Amagai Yoshiji <amagai@osrg.net>.
- * Revised by Ryusuke Konishi <ryusuke@osrg.net>.
+ * Written by Amagai Yoshiji.
+ * Revised by Ryusuke Konishi.
*
*/
@@ -68,8 +64,10 @@
struct nilfs_palloc_req req;
int ret;
- req.pr_entry_nr = 0; /* 0 says find free inode from beginning of
- a group. dull code!! */
+ req.pr_entry_nr = 0; /*
+ * 0 says find free inode from beginning
+ * of a group. dull code!!
+ */
req.pr_entry_bh = NULL;
ret = nilfs_palloc_prepare_alloc_entry(ifile, &req);
diff --git a/fs/nilfs2/ifile.h b/fs/nilfs2/ifile.h
index 679674d..23ad2f0 100644
--- a/fs/nilfs2/ifile.h
+++ b/fs/nilfs2/ifile.h
@@ -13,12 +13,8 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Amagai Yoshiji <amagai@osrg.net>
- * Revised by Ryusuke Konishi <ryusuke@osrg.net>
+ * Written by Amagai Yoshiji.
+ * Revised by Ryusuke Konishi.
*
*/
@@ -36,6 +32,7 @@
nilfs_ifile_map_inode(struct inode *ifile, ino_t ino, struct buffer_head *ibh)
{
void *kaddr = kmap(ibh->b_page);
+
return nilfs_palloc_block_get_entry(ifile, ino, ibh, kaddr);
}
diff --git a/fs/nilfs2/inode.c b/fs/nilfs2/inode.c
index cfebcd2..a0ebdb1 100644
--- a/fs/nilfs2/inode.c
+++ b/fs/nilfs2/inode.c
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Ryusuke Konishi <ryusuke@osrg.net>
+ * Written by Ryusuke Konishi.
*
*/
@@ -87,7 +83,7 @@
struct the_nilfs *nilfs = inode->i_sb->s_fs_info;
__u64 blknum = 0;
int err = 0, ret;
- unsigned maxblocks = bh_result->b_size >> inode->i_blkbits;
+ unsigned int maxblocks = bh_result->b_size >> inode->i_blkbits;
down_read(&NILFS_MDT(nilfs->ns_dat)->mi_sem);
ret = nilfs_bmap_lookup_contig(ii->i_bmap, blkoff, &blknum, maxblocks);
@@ -133,11 +129,14 @@
/* Error handling should be detailed */
set_buffer_new(bh_result);
set_buffer_delay(bh_result);
- map_bh(bh_result, inode->i_sb, 0); /* dbn must be changed
- to proper value */
+ map_bh(bh_result, inode->i_sb, 0);
+ /* Disk block number must be changed to proper value */
+
} else if (ret == -ENOENT) {
- /* not found is not error (e.g. hole); must return without
- the mapped state flag. */
+ /*
+ * not found is not error (e.g. hole); must return without
+ * the mapped state flag.
+ */
;
} else {
err = ret;
@@ -167,7 +166,7 @@
* @nr_pages - number of pages to be read
*/
static int nilfs_readpages(struct file *file, struct address_space *mapping,
- struct list_head *pages, unsigned nr_pages)
+ struct list_head *pages, unsigned int nr_pages)
{
return mpage_readpages(mapping, pages, nr_pages, nilfs_get_block);
}
@@ -226,7 +225,7 @@
int ret = __set_page_dirty_nobuffers(page);
if (page_has_buffers(page)) {
- unsigned nr_dirty = 0;
+ unsigned int nr_dirty = 0;
struct buffer_head *bh, *head;
/*
@@ -249,7 +248,7 @@
if (nr_dirty)
nilfs_set_file_dirty(inode, nr_dirty);
} else if (ret) {
- unsigned nr_dirty = 1 << (PAGE_SHIFT - inode->i_blkbits);
+ unsigned int nr_dirty = 1 << (PAGE_SHIFT - inode->i_blkbits);
nilfs_set_file_dirty(inode, nr_dirty);
}
@@ -291,8 +290,8 @@
struct page *page, void *fsdata)
{
struct inode *inode = mapping->host;
- unsigned start = pos & (PAGE_SIZE - 1);
- unsigned nr_dirty;
+ unsigned int start = pos & (PAGE_SIZE - 1);
+ unsigned int nr_dirty;
int err;
nr_dirty = nilfs_page_count_clean_buffers(page, start,
@@ -399,23 +398,26 @@
err = nilfs_init_acl(inode, dir);
if (unlikely(err))
- goto failed_after_creation; /* never occur. When supporting
- nilfs_init_acl(), proper cancellation of
- above jobs should be considered */
+ /*
+ * Never occur. When supporting nilfs_init_acl(),
+ * proper cancellation of above jobs should be considered.
+ */
+ goto failed_after_creation;
return inode;
failed_after_creation:
clear_nlink(inode);
unlock_new_inode(inode);
- iput(inode); /* raw_inode will be deleted through
- nilfs_evict_inode() */
+ iput(inode); /*
+ * raw_inode will be deleted through
+ * nilfs_evict_inode().
+ */
goto failed;
failed_ifile_create_inode:
make_bad_inode(inode);
- iput(inode); /* if i_nlink == 1, generic_forget_inode() will be
- called */
+ iput(inode);
failed:
return ERR_PTR(err);
}
@@ -666,8 +668,10 @@
else if (S_ISCHR(inode->i_mode) || S_ISBLK(inode->i_mode))
raw_inode->i_device_code =
cpu_to_le64(huge_encode_dev(inode->i_rdev));
- /* When extending inode, nilfs->ns_inode_size should be checked
- for substitutions of appended fields */
+ /*
+ * When extending inode, nilfs->ns_inode_size should be checked
+ * for substitutions of appended fields.
+ */
}
void nilfs_update_inode(struct inode *inode, struct buffer_head *ibh, int flags)
@@ -685,9 +689,12 @@
set_bit(NILFS_I_INODE_SYNC, &ii->i_state);
nilfs_write_inode_common(inode, raw_inode, 0);
- /* XXX: call with has_bmap = 0 is a workaround to avoid
- deadlock of bmap. This delays update of i_bmap to just
- before writing */
+ /*
+ * XXX: call with has_bmap = 0 is a workaround to avoid
+ * deadlock of bmap. This delays update of i_bmap to just
+ * before writing.
+ */
+
nilfs_ifile_unmap_inode(ifile, ino, ibh);
}
@@ -752,14 +759,15 @@
nilfs_mark_inode_dirty(inode);
nilfs_set_file_dirty(inode, 0);
nilfs_transaction_commit(sb);
- /* May construct a logical segment and may fail in sync mode.
- But truncate has no return value. */
+ /*
+ * May construct a logical segment and may fail in sync mode.
+ * But truncate has no return value.
+ */
}
static void nilfs_clear_inode(struct inode *inode)
{
struct nilfs_inode_info *ii = NILFS_I(inode);
- struct nilfs_mdt_info *mdi = NILFS_MDT(inode);
/*
* Free resources allocated in nilfs_read_inode(), here.
@@ -768,8 +776,8 @@
brelse(ii->i_bh);
ii->i_bh = NULL;
- if (mdi && mdi->mi_palloc_cache)
- nilfs_palloc_destroy_cache(inode);
+ if (nilfs_is_metadata_file_inode(inode))
+ nilfs_mdt_clear(inode);
if (test_bit(NILFS_I_BMAP, &ii->i_state))
nilfs_bmap_clear(ii->i_bmap);
@@ -811,8 +819,10 @@
if (IS_SYNC(inode))
nilfs_set_transaction_flag(NILFS_TI_SYNC);
nilfs_transaction_commit(sb);
- /* May construct a logical segment and may fail in sync mode.
- But delete_inode has no return value. */
+ /*
+ * May construct a logical segment and may fail in sync mode.
+ * But delete_inode has no return value.
+ */
}
int nilfs_setattr(struct dentry *dentry, struct iattr *iattr)
@@ -856,6 +866,7 @@
int nilfs_permission(struct inode *inode, int mask)
{
struct nilfs_root *root = NILFS_I(inode)->i_root;
+
if ((mask & MAY_WRITE) && root &&
root->cno != NILFS_CPTREE_CURRENT_CNO)
return -EROFS; /* snapshot is not writable */
@@ -906,7 +917,7 @@
return ret;
}
-int nilfs_set_file_dirty(struct inode *inode, unsigned nr_dirty)
+int nilfs_set_file_dirty(struct inode *inode, unsigned int nr_dirty)
{
struct nilfs_inode_info *ii = NILFS_I(inode);
struct the_nilfs *nilfs = inode->i_sb->s_fs_info;
@@ -919,17 +930,23 @@
spin_lock(&nilfs->ns_inode_lock);
if (!test_bit(NILFS_I_QUEUED, &ii->i_state) &&
!test_bit(NILFS_I_BUSY, &ii->i_state)) {
- /* Because this routine may race with nilfs_dispose_list(),
- we have to check NILFS_I_QUEUED here, too. */
+ /*
+ * Because this routine may race with nilfs_dispose_list(),
+ * we have to check NILFS_I_QUEUED here, too.
+ */
if (list_empty(&ii->i_dirty) && igrab(inode) == NULL) {
- /* This will happen when somebody is freeing
- this inode. */
+ /*
+ * This will happen when somebody is freeing
+ * this inode.
+ */
nilfs_warning(inode->i_sb, __func__,
- "cannot get inode (ino=%lu)\n",
+ "cannot get inode (ino=%lu)",
inode->i_ino);
spin_unlock(&nilfs->ns_inode_lock);
- return -EINVAL; /* NILFS_I_DIRTY may remain for
- freeing inode */
+ return -EINVAL; /*
+ * NILFS_I_DIRTY may remain for
+ * freeing inode.
+ */
}
list_move_tail(&ii->i_dirty, &nilfs->ns_dirty_files);
set_bit(NILFS_I_QUEUED, &ii->i_state);
@@ -946,7 +963,7 @@
err = nilfs_load_inode_block(inode, &ibh);
if (unlikely(err)) {
nilfs_warning(inode->i_sb, __func__,
- "failed to reget inode block.\n");
+ "failed to reget inode block.");
return err;
}
nilfs_update_inode(inode, ibh, flags);
@@ -973,7 +990,7 @@
if (is_bad_inode(inode)) {
nilfs_warning(inode->i_sb, __func__,
- "tried to mark bad_inode dirty. ignored.\n");
+ "tried to mark bad_inode dirty. ignored.");
dump_stack();
return;
}
diff --git a/fs/nilfs2/ioctl.c b/fs/nilfs2/ioctl.c
index e8fe248..358b57e 100644
--- a/fs/nilfs2/ioctl.c
+++ b/fs/nilfs2/ioctl.c
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Koji Sato <koji@osrg.net>.
+ * Written by Koji Sato.
*/
#include <linux/fs.h>
@@ -783,6 +779,7 @@
size_t nmembs = argv->v_nmembs;
struct nilfs_bmap *bmap = NILFS_I(nilfs->ns_dat)->i_bmap;
struct nilfs_bdesc *bdescs = buf;
+ struct buffer_head *bh;
int ret, i;
for (i = 0; i < nmembs; i++) {
@@ -800,12 +797,16 @@
/* skip dead block */
continue;
if (bdescs[i].bd_level == 0) {
- ret = nilfs_mdt_mark_block_dirty(nilfs->ns_dat,
- bdescs[i].bd_offset);
- if (ret < 0) {
+ ret = nilfs_mdt_get_block(nilfs->ns_dat,
+ bdescs[i].bd_offset,
+ false, NULL, &bh);
+ if (unlikely(ret)) {
WARN_ON(ret == -ENOENT);
return ret;
}
+ mark_buffer_dirty(bh);
+ nilfs_mdt_mark_dirty(nilfs->ns_dat);
+ put_bh(bh);
} else {
ret = nilfs_bmap_mark(bmap, bdescs[i].bd_offset,
bdescs[i].bd_level);
diff --git a/fs/nilfs2/mdt.c b/fs/nilfs2/mdt.c
index f6982b9..3417d85 100644
--- a/fs/nilfs2/mdt.c
+++ b/fs/nilfs2/mdt.c
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Ryusuke Konishi <ryusuke@osrg.net>
+ * Written by Ryusuke Konishi.
*/
#include <linux/buffer_head.h>
@@ -32,6 +28,7 @@
#include "segment.h"
#include "page.h"
#include "mdt.h"
+#include "alloc.h" /* nilfs_palloc_destroy_cache() */
#include <trace/events/nilfs2.h>
@@ -393,34 +390,6 @@
return ret;
}
-/**
- * nilfs_mdt_mark_block_dirty - mark a block on the meta data file dirty.
- * @inode: inode of the meta data file
- * @block: block offset
- *
- * Return Value: On success, it returns 0. On error, the following negative
- * error code is returned.
- *
- * %-ENOMEM - Insufficient memory available.
- *
- * %-EIO - I/O error
- *
- * %-ENOENT - the specified block does not exist (hole block)
- */
-int nilfs_mdt_mark_block_dirty(struct inode *inode, unsigned long block)
-{
- struct buffer_head *bh;
- int err;
-
- err = nilfs_mdt_read_block(inode, block, 0, &bh);
- if (unlikely(err))
- return err;
- mark_buffer_dirty(bh);
- nilfs_mdt_mark_dirty(inode);
- brelse(bh);
- return 0;
-}
-
int nilfs_mdt_fetch_dirty(struct inode *inode)
{
struct nilfs_inode_info *ii = NILFS_I(inode);
@@ -497,8 +466,32 @@
return 0;
}
-void nilfs_mdt_set_entry_size(struct inode *inode, unsigned entry_size,
- unsigned header_size)
+/**
+ * nilfs_mdt_clear - do cleanup for the metadata file
+ * @inode: inode of the metadata file
+ */
+void nilfs_mdt_clear(struct inode *inode)
+{
+ struct nilfs_mdt_info *mdi = NILFS_MDT(inode);
+
+ if (mdi->mi_palloc_cache)
+ nilfs_palloc_destroy_cache(inode);
+}
+
+/**
+ * nilfs_mdt_destroy - release resources used by the metadata file
+ * @inode: inode of the metadata file
+ */
+void nilfs_mdt_destroy(struct inode *inode)
+{
+ struct nilfs_mdt_info *mdi = NILFS_MDT(inode);
+
+ kfree(mdi->mi_bgl); /* kfree(NULL) is safe */
+ kfree(mdi);
+}
+
+void nilfs_mdt_set_entry_size(struct inode *inode, unsigned int entry_size,
+ unsigned int header_size)
{
struct nilfs_mdt_info *mi = NILFS_MDT(inode);
diff --git a/fs/nilfs2/mdt.h b/fs/nilfs2/mdt.h
index 03246ca..3f67f39 100644
--- a/fs/nilfs2/mdt.h
+++ b/fs/nilfs2/mdt.h
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Ryusuke Konishi <ryusuke@osrg.net>
+ * Written by Ryusuke Konishi.
*/
#ifndef _NILFS_MDT_H
@@ -57,8 +53,8 @@
struct nilfs_mdt_info {
struct rw_semaphore mi_sem;
struct blockgroup_lock *mi_bgl;
- unsigned mi_entry_size;
- unsigned mi_first_entry_offset;
+ unsigned int mi_entry_size;
+ unsigned int mi_first_entry_offset;
unsigned long mi_entries_per_block;
struct nilfs_palloc_cache *mi_palloc_cache;
struct nilfs_shadow_map *mi_shadow;
@@ -71,6 +67,11 @@
return inode->i_private;
}
+static inline int nilfs_is_metadata_file_inode(const struct inode *inode)
+{
+ return inode->i_private != NULL;
+}
+
/* Default GFP flags using highmem */
#define NILFS_MDT_GFP (__GFP_RECLAIM | __GFP_IO | __GFP_HIGHMEM)
@@ -83,11 +84,13 @@
struct buffer_head **out_bh);
int nilfs_mdt_delete_block(struct inode *, unsigned long);
int nilfs_mdt_forget_block(struct inode *, unsigned long);
-int nilfs_mdt_mark_block_dirty(struct inode *, unsigned long);
int nilfs_mdt_fetch_dirty(struct inode *);
int nilfs_mdt_init(struct inode *inode, gfp_t gfp_mask, size_t objsz);
-void nilfs_mdt_set_entry_size(struct inode *, unsigned, unsigned);
+void nilfs_mdt_clear(struct inode *inode);
+void nilfs_mdt_destroy(struct inode *inode);
+
+void nilfs_mdt_set_entry_size(struct inode *, unsigned int, unsigned int);
int nilfs_mdt_setup_shadow_map(struct inode *inode,
struct nilfs_shadow_map *shadow);
diff --git a/fs/nilfs2/namei.c b/fs/nilfs2/namei.c
index 3b2af05..1ec8ae5 100644
--- a/fs/nilfs2/namei.c
+++ b/fs/nilfs2/namei.c
@@ -13,12 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Modified for NILFS by Amagai Yoshiji <amagai@osrg.net>,
- * Ryusuke Konishi <ryusuke@osrg.net>
+ * Modified for NILFS by Amagai Yoshiji and Ryusuke Konishi.
*/
/*
* linux/fs/ext2/namei.c
@@ -49,6 +44,7 @@
static inline int nilfs_add_nondir(struct dentry *dentry, struct inode *inode)
{
int err = nilfs_add_link(dentry, inode);
+
if (!err) {
d_instantiate(dentry, inode);
unlock_new_inode(inode);
@@ -143,7 +139,7 @@
{
struct nilfs_transaction_info ti;
struct super_block *sb = dir->i_sb;
- unsigned l = strlen(symname)+1;
+ unsigned int l = strlen(symname) + 1;
struct inode *inode;
int err;
@@ -288,7 +284,7 @@
if (!inode->i_nlink) {
nilfs_warning(inode->i_sb, __func__,
- "deleting nonexistent file (%lu), %d\n",
+ "deleting nonexistent file (%lu), %d",
inode->i_ino, inode->i_nlink);
set_nlink(inode, 1);
}
diff --git a/fs/nilfs2/nilfs.h b/fs/nilfs2/nilfs.h
index 3857040..b1d48bc 100644
--- a/fs/nilfs2/nilfs.h
+++ b/fs/nilfs2/nilfs.h
@@ -13,12 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Koji Sato <koji@osrg.net>
- * Ryusuke Konishi <ryusuke@osrg.net>
+ * Written by Koji Sato and Ryusuke Konishi.
*/
#ifndef _NILFS_H
@@ -69,8 +64,10 @@
*/
struct rw_semaphore xattr_sem;
#endif
- struct buffer_head *i_bh; /* i_bh contains a new or dirty
- disk inode */
+ struct buffer_head *i_bh; /*
+ * i_bh contains a new or dirty
+ * disk inode.
+ */
struct nilfs_root *i_root;
struct inode vfs_inode;
};
@@ -100,8 +97,10 @@
NILFS_I_NEW = 0, /* Inode is newly created */
NILFS_I_DIRTY, /* The file is dirty */
NILFS_I_QUEUED, /* inode is in dirty_files list */
- NILFS_I_BUSY, /* inode is grabbed by a segment
- constructor */
+ NILFS_I_BUSY, /*
+ * Inode is grabbed by a segment
+ * constructor
+ */
NILFS_I_COLLECTED, /* All dirty blocks are collected */
NILFS_I_UPDATED, /* The file has been written back */
NILFS_I_INODE_SYNC, /* dsync is not allowed for inode */
@@ -145,8 +144,10 @@
struct nilfs_transaction_info {
u32 ti_magic;
void *ti_save;
- /* This should never used. If this happens,
- one of other filesystems has a bug. */
+ /*
+ * This should never be used. If it happens,
+ * one of other filesystems has a bug.
+ */
unsigned short ti_flags;
unsigned short ti_count;
};
@@ -156,8 +157,10 @@
/* ti_flags */
#define NILFS_TI_DYNAMIC_ALLOC 0x0001 /* Allocated from slab */
-#define NILFS_TI_SYNC 0x0002 /* Force to construct segment at the
- end of transaction. */
+#define NILFS_TI_SYNC 0x0002 /*
+ * Force to construct segment at the
+ * end of transaction.
+ */
#define NILFS_TI_GC 0x0004 /* GC context */
#define NILFS_TI_COMMIT 0x0008 /* Change happened or not */
#define NILFS_TI_WRITER 0x0010 /* Constructor context */
@@ -279,7 +282,7 @@
int nilfs_permission(struct inode *inode, int mask);
int nilfs_load_inode_block(struct inode *inode, struct buffer_head **pbh);
extern int nilfs_inode_dirty(struct inode *);
-int nilfs_set_file_dirty(struct inode *inode, unsigned nr_dirty);
+int nilfs_set_file_dirty(struct inode *inode, unsigned int nr_dirty);
extern int __nilfs_mark_inode_dirty(struct inode *, int);
extern void nilfs_dirty_inode(struct inode *, int flags);
int nilfs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
diff --git a/fs/nilfs2/page.c b/fs/nilfs2/page.c
index 4893915..d97ba5f 100644
--- a/fs/nilfs2/page.c
+++ b/fs/nilfs2/page.c
@@ -13,12 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Ryusuke Konishi <ryusuke@osrg.net>,
- * Seiji Kihara <kihara@osrg.net>.
+ * Written by Ryusuke Konishi and Seiji Kihara.
*/
#include <linux/pagemap.h>
@@ -440,12 +435,12 @@
__nilfs_clear_page_dirty(page);
}
-unsigned nilfs_page_count_clean_buffers(struct page *page,
- unsigned from, unsigned to)
+unsigned int nilfs_page_count_clean_buffers(struct page *page,
+ unsigned int from, unsigned int to)
{
- unsigned block_start, block_end;
+ unsigned int block_start, block_end;
struct buffer_head *bh, *head;
- unsigned nc = 0;
+ unsigned int nc = 0;
for (bh = head = page_buffers(page), block_start = 0;
bh != head || !block_start;
diff --git a/fs/nilfs2/page.h b/fs/nilfs2/page.h
index a43b8287..f3687c9 100644
--- a/fs/nilfs2/page.h
+++ b/fs/nilfs2/page.h
@@ -13,12 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Ryusuke Konishi <ryusuke@osrg.net>,
- * Seiji Kihara <kihara@osrg.net>.
+ * Written by Ryusuke Konishi and Seiji Kihara.
*/
#ifndef _NILFS_PAGE_H
@@ -58,7 +53,8 @@
void nilfs_clear_dirty_page(struct page *, bool);
void nilfs_clear_dirty_pages(struct address_space *, bool);
void nilfs_mapping_init(struct address_space *mapping, struct inode *inode);
-unsigned nilfs_page_count_clean_buffers(struct page *, unsigned, unsigned);
+unsigned int nilfs_page_count_clean_buffers(struct page *, unsigned int,
+ unsigned int);
unsigned long nilfs_find_uncommitted_extent(struct inode *inode,
sector_t start_blk,
sector_t *blkoff);
diff --git a/fs/nilfs2/recovery.c b/fs/nilfs2/recovery.c
index 5afa77f..d893dc9 100644
--- a/fs/nilfs2/recovery.c
+++ b/fs/nilfs2/recovery.c
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Ryusuke Konishi <ryusuke@osrg.net>
+ * Written by Ryusuke Konishi.
*/
#include <linux/buffer_head.h>
@@ -47,8 +43,10 @@
/* work structure for recovery */
struct nilfs_recovery_block {
- ino_t ino; /* Inode number of the file that this block
- belongs to */
+ ino_t ino; /*
+ * Inode number of the file that this block
+ * belongs to
+ */
sector_t blocknr; /* block number */
__u64 vblocknr; /* virtual block number */
unsigned long blkoff; /* File offset of the data block (per block) */
@@ -156,7 +154,7 @@
sr = (struct nilfs_super_root *)bh_sr->b_data;
if (check) {
- unsigned bytes = le16_to_cpu(sr->sr_bytes);
+ unsigned int bytes = le16_to_cpu(sr->sr_bytes);
if (bytes == 0 || bytes > nilfs->ns_blocksize) {
ret = NILFS_SEG_FAIL_CHECKSUM_SUPER_ROOT;
@@ -508,7 +506,7 @@
{
struct inode *inode;
struct nilfs_recovery_block *rb, *n;
- unsigned blocksize = nilfs->ns_blocksize;
+ unsigned int blocksize = nilfs->ns_blocksize;
struct page *page;
loff_t pos;
int err = 0, err2 = 0;
@@ -526,6 +524,7 @@
0, &page, nilfs_get_block);
if (unlikely(err)) {
loff_t isize = inode->i_size;
+
if (pos + blocksize > isize)
nilfs_write_failed(inode->i_mapping,
pos + blocksize);
@@ -872,9 +871,11 @@
flags = le16_to_cpu(sum->ss_flags);
if (!(flags & NILFS_SS_SR) && !scan_newer) {
- /* This will never happen because a superblock
- (last_segment) always points to a pseg
- having a super root. */
+ /*
+ * This will never happen because a superblock
+ * (last_segment) always points to a pseg with
+ * a super root.
+ */
ret = NILFS_SEG_FAIL_CONSISTENCY;
goto failed;
}
diff --git a/fs/nilfs2/segbuf.c b/fs/nilfs2/segbuf.c
index f63620c..bf36df1 100644
--- a/fs/nilfs2/segbuf.c
+++ b/fs/nilfs2/segbuf.c
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Ryusuke Konishi <ryusuke@osrg.net>
+ * Written by Ryusuke Konishi.
*
*/
@@ -133,7 +129,7 @@
return 0;
}
-int nilfs_segbuf_reset(struct nilfs_segment_buffer *segbuf, unsigned flags,
+int nilfs_segbuf_reset(struct nilfs_segment_buffer *segbuf, unsigned int flags,
time_t ctime, __u64 cno)
{
int err;
@@ -240,7 +236,7 @@
{
struct nilfs_super_root *raw_sr;
struct the_nilfs *nilfs = segbuf->sb_super->s_fs_info;
- unsigned srsize;
+ unsigned int srsize;
u32 crc;
raw_sr = (struct nilfs_super_root *)segbuf->sb_super_root->b_data;
diff --git a/fs/nilfs2/segbuf.h b/fs/nilfs2/segbuf.h
index b04f08c..7bbccc0 100644
--- a/fs/nilfs2/segbuf.h
+++ b/fs/nilfs2/segbuf.h
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Ryusuke Konishi <ryusuke@osrg.net>
+ * Written by Ryusuke Konishi.
*
*/
#ifndef _NILFS_SEGBUF_H
@@ -82,7 +78,7 @@
__u64 sb_nextnum;
sector_t sb_fseg_start, sb_fseg_end;
sector_t sb_pseg_start;
- unsigned sb_rest_blocks;
+ unsigned int sb_rest_blocks;
/* Buffers */
struct list_head sb_segsum_buffers;
@@ -124,7 +120,8 @@
struct nilfs_segment_buffer *prev);
void nilfs_segbuf_set_next_segnum(struct nilfs_segment_buffer *, __u64,
struct the_nilfs *);
-int nilfs_segbuf_reset(struct nilfs_segment_buffer *, unsigned, time_t, __u64);
+int nilfs_segbuf_reset(struct nilfs_segment_buffer *, unsigned int, time_t,
+ __u64);
int nilfs_segbuf_extend_segsum(struct nilfs_segment_buffer *);
int nilfs_segbuf_extend_payload(struct nilfs_segment_buffer *,
struct buffer_head **);
diff --git a/fs/nilfs2/segment.c b/fs/nilfs2/segment.c
index 4317f72..e78b68a8 100644
--- a/fs/nilfs2/segment.c
+++ b/fs/nilfs2/segment.c
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Ryusuke Konishi <ryusuke@osrg.net>
+ * Written by Ryusuke Konishi.
*
*/
@@ -49,18 +45,26 @@
*/
#define SC_N_INODEVEC 16 /* Size of locally allocated inode vector */
-#define SC_MAX_SEGDELTA 64 /* Upper limit of the number of segments
- appended in collection retry loop */
+#define SC_MAX_SEGDELTA 64 /*
+ * Upper limit of the number of segments
+ * appended in collection retry loop
+ */
/* Construction mode */
enum {
SC_LSEG_SR = 1, /* Make a logical segment having a super root */
- SC_LSEG_DSYNC, /* Flush data blocks of a given file and make
- a logical segment without a super root */
- SC_FLUSH_FILE, /* Flush data files, leads to segment writes without
- creating a checkpoint */
- SC_FLUSH_DAT, /* Flush DAT file. This also creates segments without
- a checkpoint */
+ SC_LSEG_DSYNC, /*
+ * Flush data blocks of a given file and make
+ * a logical segment without a super root.
+ */
+ SC_FLUSH_FILE, /*
+ * Flush data files, leads to segment writes without
+ * creating a checkpoint.
+ */
+ SC_FLUSH_DAT, /*
+ * Flush DAT file. This also creates segments
+ * without a checkpoint.
+ */
};
/* Stage numbers of dirty block collection */
@@ -154,17 +158,15 @@
if (cur_ti) {
if (cur_ti->ti_magic == NILFS_TI_MAGIC)
return ++cur_ti->ti_count;
- else {
- /*
- * If journal_info field is occupied by other FS,
- * it is saved and will be restored on
- * nilfs_transaction_commit().
- */
- printk(KERN_WARNING
- "NILFS warning: journal info from a different "
- "FS\n");
- save = current->journal_info;
- }
+
+ /*
+ * If journal_info field is occupied by other FS,
+ * it is saved and will be restored on
+ * nilfs_transaction_commit().
+ */
+ printk(KERN_WARNING
+ "NILFS warning: journal info from a different FS\n");
+ save = current->journal_info;
}
if (!ti) {
ti = kmem_cache_alloc(nilfs_transaction_cachep, GFP_NOFS);
@@ -397,10 +399,10 @@
static void *nilfs_segctor_map_segsum_entry(struct nilfs_sc_info *sci,
struct nilfs_segsum_pointer *ssp,
- unsigned bytes)
+ unsigned int bytes)
{
struct nilfs_segment_buffer *segbuf = sci->sc_curseg;
- unsigned blocksize = sci->sc_super->s_blocksize;
+ unsigned int blocksize = sci->sc_super->s_blocksize;
void *p;
if (unlikely(ssp->offset + bytes > blocksize)) {
@@ -422,8 +424,8 @@
{
struct nilfs_segment_buffer *segbuf = sci->sc_curseg;
struct buffer_head *sumbh;
- unsigned sumbytes;
- unsigned flags = 0;
+ unsigned int sumbytes;
+ unsigned int flags = 0;
int err;
if (nilfs_doing_gc())
@@ -444,8 +446,10 @@
{
sci->sc_nblk_this_inc += sci->sc_curseg->sb_sum.nblocks;
if (NILFS_SEGBUF_IS_LAST(sci->sc_curseg, &sci->sc_segbufs))
- return -E2BIG; /* The current segment is filled up
- (internal code) */
+ return -E2BIG; /*
+ * The current segment is filled up
+ * (internal code)
+ */
sci->sc_curseg = NILFS_NEXT_SEGBUF(sci->sc_curseg);
return nilfs_segctor_reset_segment_buffer(sci);
}
@@ -472,9 +476,9 @@
*/
static int nilfs_segctor_segsum_block_required(
struct nilfs_sc_info *sci, const struct nilfs_segsum_pointer *ssp,
- unsigned binfo_size)
+ unsigned int binfo_size)
{
- unsigned blocksize = sci->sc_super->s_blocksize;
+ unsigned int blocksize = sci->sc_super->s_blocksize;
/* Size of finfo and binfo is enough small against blocksize */
return ssp->offset + binfo_size +
@@ -533,7 +537,7 @@
static int nilfs_segctor_add_file_block(struct nilfs_sc_info *sci,
struct buffer_head *bh,
struct inode *inode,
- unsigned binfo_size)
+ unsigned int binfo_size)
{
struct nilfs_segment_buffer *segbuf;
int required, err = 0;
@@ -617,7 +621,7 @@
*vblocknr = binfo->bi_v.bi_vblocknr;
}
-static struct nilfs_sc_operations nilfs_sc_file_ops = {
+static const struct nilfs_sc_operations nilfs_sc_file_ops = {
.collect_data = nilfs_collect_file_data,
.collect_node = nilfs_collect_file_node,
.collect_bmap = nilfs_collect_file_bmap,
@@ -666,7 +670,7 @@
*binfo_dat = binfo->bi_dat;
}
-static struct nilfs_sc_operations nilfs_sc_dat_ops = {
+static const struct nilfs_sc_operations nilfs_sc_dat_ops = {
.collect_data = nilfs_collect_dat_data,
.collect_node = nilfs_collect_file_node,
.collect_bmap = nilfs_collect_dat_bmap,
@@ -674,7 +678,7 @@
.write_node_binfo = nilfs_write_dat_node_binfo,
};
-static struct nilfs_sc_operations nilfs_sc_dsync_ops = {
+static const struct nilfs_sc_operations nilfs_sc_dsync_ops = {
.collect_data = nilfs_collect_file_data,
.collect_node = NULL,
.collect_bmap = NULL,
@@ -777,7 +781,7 @@
{
struct nilfs_inode_info *ii, *n;
struct nilfs_inode_info *ivec[SC_N_INODEVEC], **pii;
- unsigned nv = 0;
+ unsigned int nv = 0;
while (!list_empty(head)) {
spin_lock(&nilfs->ns_inode_lock);
@@ -875,9 +879,11 @@
err = nilfs_cpfile_get_checkpoint(nilfs->ns_cpfile, nilfs->ns_cno, 1,
&raw_cp, &bh_cp);
if (likely(!err)) {
- /* The following code is duplicated with cpfile. But, it is
- needed to collect the checkpoint even if it was not newly
- created */
+ /*
+ * The following code is duplicated with cpfile. But, it is
+ * needed to collect the checkpoint even if it was not newly
+ * created.
+ */
mark_buffer_dirty(bh_cp);
nilfs_mdt_mark_dirty(nilfs->ns_cpfile);
nilfs_cpfile_put_checkpoint(
@@ -958,7 +964,7 @@
{
struct buffer_head *bh_sr;
struct nilfs_super_root *raw_sr;
- unsigned isz, srsz;
+ unsigned int isz, srsz;
bh_sr = NILFS_LAST_SEGBUF(&sci->sc_segbufs)->sb_super_root;
raw_sr = (struct nilfs_super_root *)bh_sr->b_data;
@@ -1043,7 +1049,7 @@
static int nilfs_segctor_scan_file(struct nilfs_sc_info *sci,
struct inode *inode,
- struct nilfs_sc_operations *sc_ops)
+ const struct nilfs_sc_operations *sc_ops)
{
LIST_HEAD(data_buffers);
LIST_HEAD(node_buffers);
@@ -1406,8 +1412,10 @@
if (atomic_read(&segbuf->sb_err)) {
/* Case 1: The first segment failed */
if (segbuf->sb_pseg_start != segbuf->sb_fseg_start)
- /* Case 1a: Partial segment appended into an existing
- segment */
+ /*
+ * Case 1a: Partial segment appended into an existing
+ * segment
+ */
nilfs_terminate_segment(nilfs, segbuf->sb_fseg_start,
segbuf->sb_fseg_end);
else /* Case 1b: New full segment */
@@ -1550,7 +1558,7 @@
sector_t blocknr;
unsigned long nfinfo = segbuf->sb_sum.nfinfo;
unsigned long nblocks = 0, ndatablk = 0;
- struct nilfs_sc_operations *sc_op = NULL;
+ const struct nilfs_sc_operations *sc_op = NULL;
struct nilfs_segsum_pointer ssp;
struct nilfs_finfo *finfo = NULL;
union nilfs_binfo binfo;
@@ -1631,8 +1639,10 @@
static void nilfs_begin_page_io(struct page *page)
{
if (!page || PageWriteback(page))
- /* For split b-tree node pages, this function may be called
- twice. We ignore the 2nd or later calls by this check. */
+ /*
+ * For split b-tree node pages, this function may be called
+ * twice. We ignore the 2nd or later calls by this check.
+ */
return;
lock_page(page);
@@ -1942,7 +1952,7 @@
ifile, ii->vfs_inode.i_ino, &ibh);
if (unlikely(err)) {
nilfs_warning(sci->sc_super, __func__,
- "failed to get inode block.\n");
+ "failed to get inode block.");
return err;
}
mark_buffer_dirty(ibh);
@@ -2395,6 +2405,7 @@
static void nilfs_construction_timeout(unsigned long data)
{
struct task_struct *p = (struct task_struct *)data;
+
wake_up_process(p);
}
@@ -2555,10 +2566,10 @@
if (timeout || sci->sc_seq_request != sci->sc_seq_done)
mode = SC_LSEG_SR;
- else if (!sci->sc_flush_request)
- break;
- else
+ else if (sci->sc_flush_request)
mode = nilfs_segctor_flush_mode(sci);
+ else
+ break;
spin_unlock(&sci->sc_state_lock);
nilfs_segctor_thread_construct(sci, mode);
@@ -2684,8 +2695,10 @@
{
int ret, retrycount = NILFS_SC_CLEANUP_RETRY;
- /* The segctord thread was stopped and its timer was removed.
- But some tasks remain. */
+ /*
+ * The segctord thread was stopped and its timer was removed.
+ * But some tasks remain.
+ */
do {
struct nilfs_transaction_info ti;
@@ -2727,13 +2740,13 @@
if (!list_empty(&sci->sc_dirty_files)) {
nilfs_warning(sci->sc_super, __func__,
- "dirty file(s) after the final construction\n");
+ "dirty file(s) after the final construction");
nilfs_dispose_list(nilfs, &sci->sc_dirty_files, 1);
}
if (!list_empty(&sci->sc_iput_queue)) {
nilfs_warning(sci->sc_super, __func__,
- "iput queue is not empty\n");
+ "iput queue is not empty");
nilfs_dispose_list(nilfs, &sci->sc_iput_queue, 1);
}
@@ -2810,7 +2823,7 @@
if (!list_empty(&nilfs->ns_dirty_files)) {
list_splice_init(&nilfs->ns_dirty_files, &garbage_list);
nilfs_warning(sb, __func__,
- "Hit dirty file after stopped log writer\n");
+ "Hit dirty file after stopped log writer");
}
spin_unlock(&nilfs->ns_inode_lock);
up_write(&nilfs->ns_segctor_sem);
diff --git a/fs/nilfs2/segment.h b/fs/nilfs2/segment.h
index 0408b9b..6565c10 100644
--- a/fs/nilfs2/segment.h
+++ b/fs/nilfs2/segment.h
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Ryusuke Konishi <ryusuke@osrg.net>
+ * Written by Ryusuke Konishi.
*
*/
#ifndef _NILFS_SEGMENT_H
@@ -75,7 +71,7 @@
*/
struct nilfs_cstage {
int scnt;
- unsigned flags;
+ unsigned int flags;
struct nilfs_inode_info *dirty_file_ptr;
struct nilfs_inode_info *gc_inode_ptr;
};
@@ -84,7 +80,7 @@
struct nilfs_segsum_pointer {
struct buffer_head *bh;
- unsigned offset; /* offset in bytes */
+ unsigned int offset; /* offset in bytes */
};
/**
@@ -193,11 +189,15 @@
NILFS_SC_DIRTY, /* One or more dirty meta-data blocks exist */
NILFS_SC_UNCLOSED, /* Logical segment is not closed */
NILFS_SC_SUPER_ROOT, /* The latest segment has a super root */
- NILFS_SC_PRIOR_FLUSH, /* Requesting immediate flush without making a
- checkpoint */
- NILFS_SC_HAVE_DELTA, /* Next checkpoint will have update of files
- other than DAT, cpfile, sufile, or files
- moved by GC */
+ NILFS_SC_PRIOR_FLUSH, /*
+ * Requesting immediate flush without making a
+ * checkpoint
+ */
+ NILFS_SC_HAVE_DELTA, /*
+ * Next checkpoint will have update of files
+ * other than DAT, cpfile, sufile, or files
+ * moved by GC.
+ */
};
/* sc_state */
@@ -207,17 +207,23 @@
/*
* Constant parameters
*/
-#define NILFS_SC_CLEANUP_RETRY 3 /* Retry count of construction when
- destroying segctord */
+#define NILFS_SC_CLEANUP_RETRY 3 /*
+ * Retry count of construction when
+ * destroying segctord
+ */
/*
* Default values of timeout, in seconds.
*/
-#define NILFS_SC_DEFAULT_TIMEOUT 5 /* Timeout value of dirty blocks.
- It triggers construction of a
- logical segment with a super root */
-#define NILFS_SC_DEFAULT_SR_FREQ 30 /* Maximum frequency of super root
- creation */
+#define NILFS_SC_DEFAULT_TIMEOUT 5 /*
+ * Timeout value of dirty blocks.
+ * It triggers construction of a
+ * logical segment with a super root.
+ */
+#define NILFS_SC_DEFAULT_SR_FREQ 30 /*
+ * Maximum frequency of super root
+ * creation
+ */
/*
* The default threshold amount of data, in block counts.
diff --git a/fs/nilfs2/sufile.c b/fs/nilfs2/sufile.c
index 52821ff..1963595 100644
--- a/fs/nilfs2/sufile.c
+++ b/fs/nilfs2/sufile.c
@@ -13,12 +13,8 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Koji Sato <koji@osrg.net>.
- * Revised by Ryusuke Konishi <ryusuke@osrg.net>.
+ * Written by Koji Sato.
+ * Revised by Ryusuke Konishi.
*/
#include <linux/kernel.h>
@@ -61,6 +57,7 @@
nilfs_sufile_get_blkoff(const struct inode *sufile, __u64 segnum)
{
__u64 t = segnum + NILFS_MDT(sufile)->mi_first_entry_offset;
+
do_div(t, nilfs_sufile_segment_usages_per_block(sufile));
return (unsigned long)t;
}
@@ -69,6 +66,7 @@
nilfs_sufile_get_offset(const struct inode *sufile, __u64 segnum)
{
__u64 t = segnum + NILFS_MDT(sufile)->mi_first_entry_offset;
+
return do_div(t, nilfs_sufile_segment_usages_per_block(sufile));
}
@@ -819,7 +817,7 @@
* %-ENOMEM - Insufficient amount of memory available.
*/
ssize_t nilfs_sufile_get_suinfo(struct inode *sufile, __u64 segnum, void *buf,
- unsigned sisz, size_t nsi)
+ unsigned int sisz, size_t nsi)
{
struct buffer_head *su_bh;
struct nilfs_segment_usage *su;
@@ -897,7 +895,7 @@
* %-EINVAL - Invalid values in input (segment number, flags or nblocks)
*/
ssize_t nilfs_sufile_set_suinfo(struct inode *sufile, void *buf,
- unsigned supsz, size_t nsup)
+ unsigned int supsz, size_t nsup)
{
struct the_nilfs *nilfs = sufile->i_sb->s_fs_info;
struct buffer_head *header_bh, *bh;
diff --git a/fs/nilfs2/sufile.h b/fs/nilfs2/sufile.h
index b8afd72..46e8987 100644
--- a/fs/nilfs2/sufile.h
+++ b/fs/nilfs2/sufile.h
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Koji Sato <koji@osrg.net>.
+ * Written by Koji Sato.
*/
#ifndef _NILFS_SUFILE_H
@@ -42,9 +38,9 @@
int nilfs_sufile_set_segment_usage(struct inode *sufile, __u64 segnum,
unsigned long nblocks, time_t modtime);
int nilfs_sufile_get_stat(struct inode *, struct nilfs_sustat *);
-ssize_t nilfs_sufile_get_suinfo(struct inode *, __u64, void *, unsigned,
+ssize_t nilfs_sufile_get_suinfo(struct inode *, __u64, void *, unsigned int,
size_t);
-ssize_t nilfs_sufile_set_suinfo(struct inode *, void *, unsigned , size_t);
+ssize_t nilfs_sufile_set_suinfo(struct inode *, void *, unsigned int, size_t);
int nilfs_sufile_updatev(struct inode *, __u64 *, size_t, int, size_t *,
void (*dofunc)(struct inode *, __u64,
diff --git a/fs/nilfs2/super.c b/fs/nilfs2/super.c
index 7f5d3d9..666107a 100644
--- a/fs/nilfs2/super.c
+++ b/fs/nilfs2/super.c
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Ryusuke Konishi <ryusuke@osrg.net>
+ * Written by Ryusuke Konishi.
*/
/*
* linux/fs/ext2/super.c
@@ -173,12 +169,10 @@
static void nilfs_i_callback(struct rcu_head *head)
{
struct inode *inode = container_of(head, struct inode, i_rcu);
- struct nilfs_mdt_info *mdi = NILFS_MDT(inode);
- if (mdi) {
- kfree(mdi->mi_bgl); /* kfree(NULL) is safe */
- kfree(mdi);
- }
+ if (nilfs_is_metadata_file_inode(inode))
+ nilfs_mdt_destroy(inode);
+
kmem_cache_free(nilfs_inode_cachep, NILFS_I(inode));
}
@@ -279,7 +273,7 @@
}
} else if (sbp[1] &&
sbp[1]->s_magic != cpu_to_le16(NILFS_SUPER_MAGIC)) {
- memcpy(sbp[1], sbp[0], nilfs->ns_sbsize);
+ memcpy(sbp[1], sbp[0], nilfs->ns_sbsize);
}
if (flip && sbp[1])
@@ -749,6 +743,7 @@
while ((p = strsep(&options, ",")) != NULL) {
int token;
+
if (!*p)
continue;
@@ -891,7 +886,7 @@
nilfs->ns_interval = le32_to_cpu(sbp->s_c_interval);
nilfs->ns_watermark = le32_to_cpu(sbp->s_c_block_max);
- return !parse_options(data, sb, 0) ? -EINVAL : 0 ;
+ return !parse_options(data, sb, 0) ? -EINVAL : 0;
}
int nilfs_check_feature_compatibility(struct super_block *sb,
@@ -1316,7 +1311,7 @@
}
if (!s->s_root) {
- s_new = true;
+ s_new = true;
/* New superblock instance created */
s->s_mode = mode;
diff --git a/fs/nilfs2/sysfs.c b/fs/nilfs2/sysfs.c
index bbb0dcc..8ffa42b 100644
--- a/fs/nilfs2/sysfs.c
+++ b/fs/nilfs2/sysfs.c
@@ -68,7 +68,7 @@
static const struct sysfs_ops nilfs_##name##_attr_ops = { \
.show = nilfs_##name##_attr_show, \
.store = nilfs_##name##_attr_store, \
-};
+}
#define NILFS_DEV_INT_GROUP_TYPE(name, parent_name) \
static void nilfs_##name##_attr_release(struct kobject *kobj) \
@@ -84,7 +84,7 @@
.default_attrs = nilfs_##name##_attrs, \
.sysfs_ops = &nilfs_##name##_attr_ops, \
.release = nilfs_##name##_attr_release, \
-};
+}
#define NILFS_DEV_INT_GROUP_FNS(name, parent_name) \
static int nilfs_sysfs_create_##name##_group(struct the_nilfs *nilfs) \
@@ -756,7 +756,7 @@
struct the_nilfs *nilfs,
char *buf)
{
- unsigned sbwcount;
+ unsigned int sbwcount;
down_read(&nilfs->ns_sem);
sbwcount = nilfs->ns_sbwcount;
@@ -770,7 +770,7 @@
struct the_nilfs *nilfs,
char *buf)
{
- unsigned sb_update_freq;
+ unsigned int sb_update_freq;
down_read(&nilfs->ns_sem);
sb_update_freq = nilfs->ns_sb_update_freq;
@@ -784,7 +784,7 @@
struct the_nilfs *nilfs,
const char *buf, size_t count)
{
- unsigned val;
+ unsigned int val;
int err;
err = kstrtouint(skip_spaces(buf), 0, &val);
diff --git a/fs/nilfs2/sysfs.h b/fs/nilfs2/sysfs.h
index 677e3a1..648cedf 100644
--- a/fs/nilfs2/sysfs.h
+++ b/fs/nilfs2/sysfs.h
@@ -66,7 +66,7 @@
char *); \
ssize_t (*store)(struct kobject *, struct attribute *, \
const char *, size_t); \
-};
+}
NILFS_COMMON_ATTR_STRUCT(feature);
@@ -77,7 +77,7 @@
char *); \
ssize_t (*store)(struct nilfs_##name##_attr *, struct the_nilfs *, \
const char *, size_t); \
-};
+}
NILFS_DEV_ATTR_STRUCT(dev);
NILFS_DEV_ATTR_STRUCT(segments);
@@ -93,7 +93,7 @@
char *); \
ssize_t (*store)(struct nilfs_##name##_attr *, struct nilfs_root *, \
const char *, size_t); \
-};
+}
NILFS_CP_ATTR_STRUCT(snapshot);
diff --git a/fs/nilfs2/the_nilfs.c b/fs/nilfs2/the_nilfs.c
index 69bd801..809bd2d 100644
--- a/fs/nilfs2/the_nilfs.c
+++ b/fs/nilfs2/the_nilfs.c
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Ryusuke Konishi <ryusuke@osrg.net>
+ * Written by Ryusuke Konishi.
*
*/
@@ -112,8 +108,8 @@
struct nilfs_super_root *raw_sr;
struct nilfs_super_block **sbp = nilfs->ns_sbp;
struct nilfs_inode *rawi;
- unsigned dat_entry_size, segment_usage_size, checkpoint_size;
- unsigned inode_size;
+ unsigned int dat_entry_size, segment_usage_size, checkpoint_size;
+ unsigned int inode_size;
int err;
err = nilfs_read_super_root_block(nilfs, sr_block, &bh_sr, 1);
@@ -621,8 +617,10 @@
err = nilfs_load_super_block(nilfs, sb, blocksize, &sbp);
if (err)
goto out;
- /* not failed_sbh; sbh is released automatically
- when reloading fails. */
+ /*
+ * Not to failed_sbh; sbh is released automatically
+ * when reloading fails.
+ */
}
nilfs->ns_blocksize_bits = sb->s_blocksize_bits;
nilfs->ns_blocksize = blocksize;
diff --git a/fs/nilfs2/the_nilfs.h b/fs/nilfs2/the_nilfs.h
index 23778d3..79369fd 100644
--- a/fs/nilfs2/the_nilfs.h
+++ b/fs/nilfs2/the_nilfs.h
@@ -13,11 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Ryusuke Konishi <ryusuke@osrg.net>
+ * Written by Ryusuke Konishi.
*
*/
@@ -118,10 +114,10 @@
struct buffer_head *ns_sbh[2];
struct nilfs_super_block *ns_sbp[2];
time_t ns_sbwtime;
- unsigned ns_sbwcount;
- unsigned ns_sbsize;
- unsigned ns_mount_state;
- unsigned ns_sb_update_freq;
+ unsigned int ns_sbwcount;
+ unsigned int ns_sbsize;
+ unsigned int ns_mount_state;
+ unsigned int ns_sb_update_freq;
/*
* Following fields are dedicated to a writable FS-instance.
@@ -226,15 +222,14 @@
* Mount option operations
*/
#define nilfs_clear_opt(nilfs, opt) \
- do { (nilfs)->ns_mount_opt &= ~NILFS_MOUNT_##opt; } while (0)
+ ((nilfs)->ns_mount_opt &= ~NILFS_MOUNT_##opt)
#define nilfs_set_opt(nilfs, opt) \
- do { (nilfs)->ns_mount_opt |= NILFS_MOUNT_##opt; } while (0)
+ ((nilfs)->ns_mount_opt |= NILFS_MOUNT_##opt)
#define nilfs_test_opt(nilfs, opt) ((nilfs)->ns_mount_opt & NILFS_MOUNT_##opt)
#define nilfs_write_opt(nilfs, mask, opt) \
- do { (nilfs)->ns_mount_opt = \
+ ((nilfs)->ns_mount_opt = \
(((nilfs)->ns_mount_opt & ~NILFS_MOUNT_##mask) | \
- NILFS_MOUNT_##opt); \
- } while (0)
+ NILFS_MOUNT_##opt)) \
/**
* struct nilfs_root - nilfs root object
@@ -273,6 +268,7 @@
static inline int nilfs_sb_need_update(struct the_nilfs *nilfs)
{
u64 t = get_seconds();
+
return t < nilfs->ns_sbwtime ||
t > nilfs->ns_sbwtime + nilfs->ns_sb_update_freq;
}
@@ -280,6 +276,7 @@
static inline int nilfs_sb_will_flip(struct the_nilfs *nilfs)
{
int flip_bits = nilfs->ns_sbwcount & 0x0FL;
+
return (flip_bits != 0x08 && flip_bits != 0x0F);
}
@@ -308,7 +305,7 @@
static inline int nilfs_valid_fs(struct the_nilfs *nilfs)
{
- unsigned valid_fs;
+ unsigned int valid_fs;
down_read(&nilfs->ns_sem);
valid_fs = (nilfs->ns_mount_state & NILFS_VALID_FS);
diff --git a/fs/notify/fsnotify.h b/fs/notify/fsnotify.h
index b44c68a..0a3bc2c 100644
--- a/fs/notify/fsnotify.h
+++ b/fs/notify/fsnotify.h
@@ -56,6 +56,13 @@
fsnotify_destroy_marks(&real_mount(mnt)->mnt_fsnotify_marks,
&mnt->mnt_root->d_lock);
}
+/* prepare for freeing all marks associated with given group */
+extern void fsnotify_detach_group_marks(struct fsnotify_group *group);
+/*
+ * wait for fsnotify_mark_srcu period to end and free all marks in destroy_list
+ */
+extern void fsnotify_mark_destroy_list(void);
+
/*
* update the dentry->d_flags of all of inode's children to indicate if inode cares
* about events that happen to its children.
diff --git a/fs/notify/group.c b/fs/notify/group.c
index d16b62c..3e2dd85 100644
--- a/fs/notify/group.c
+++ b/fs/notify/group.c
@@ -47,12 +47,21 @@
*/
void fsnotify_destroy_group(struct fsnotify_group *group)
{
- /* clear all inode marks for this group */
- fsnotify_clear_marks_by_group(group);
+ /* clear all inode marks for this group, attach them to destroy_list */
+ fsnotify_detach_group_marks(group);
- synchronize_srcu(&fsnotify_mark_srcu);
+ /*
+ * Wait for fsnotify_mark_srcu period to end and free all marks in
+ * destroy_list
+ */
+ fsnotify_mark_destroy_list();
- /* clear the notification queue of all events */
+ /*
+ * Since we have waited for fsnotify_mark_srcu in
+ * fsnotify_mark_destroy_list() there can be no outstanding event
+ * notification against this group. So clearing the notification queue
+ * of all events is reliable now.
+ */
fsnotify_flush_notify(group);
/*
diff --git a/fs/notify/mark.c b/fs/notify/mark.c
index 7115c5d..d3fea0b 100644
--- a/fs/notify/mark.c
+++ b/fs/notify/mark.c
@@ -97,8 +97,8 @@
static DEFINE_SPINLOCK(destroy_lock);
static LIST_HEAD(destroy_list);
-static void fsnotify_mark_destroy(struct work_struct *work);
-static DECLARE_DELAYED_WORK(reaper_work, fsnotify_mark_destroy);
+static void fsnotify_mark_destroy_workfn(struct work_struct *work);
+static DECLARE_DELAYED_WORK(reaper_work, fsnotify_mark_destroy_workfn);
void fsnotify_get_mark(struct fsnotify_mark *mark)
{
@@ -173,11 +173,15 @@
}
/*
- * Free fsnotify mark. The freeing is actually happening from a kthread which
- * first waits for srcu period end. Caller must have a reference to the mark
- * or be protected by fsnotify_mark_srcu.
+ * Prepare mark for freeing and add it to the list of marks prepared for
+ * freeing. The actual freeing must happen after SRCU period ends and the
+ * caller is responsible for this.
+ *
+ * The function returns true if the mark was added to the list of marks for
+ * freeing. The function returns false if someone else has already called
+ * __fsnotify_free_mark() for the mark.
*/
-void fsnotify_free_mark(struct fsnotify_mark *mark)
+static bool __fsnotify_free_mark(struct fsnotify_mark *mark)
{
struct fsnotify_group *group = mark->group;
@@ -185,17 +189,11 @@
/* something else already called this function on this mark */
if (!(mark->flags & FSNOTIFY_MARK_FLAG_ALIVE)) {
spin_unlock(&mark->lock);
- return;
+ return false;
}
mark->flags &= ~FSNOTIFY_MARK_FLAG_ALIVE;
spin_unlock(&mark->lock);
- spin_lock(&destroy_lock);
- list_add(&mark->g_list, &destroy_list);
- spin_unlock(&destroy_lock);
- queue_delayed_work(system_unbound_wq, &reaper_work,
- FSNOTIFY_REAPER_DELAY);
-
/*
* Some groups like to know that marks are being freed. This is a
* callback to the group function to let it know that this mark
@@ -203,6 +201,25 @@
*/
if (group->ops->freeing_mark)
group->ops->freeing_mark(mark, group);
+
+ spin_lock(&destroy_lock);
+ list_add(&mark->g_list, &destroy_list);
+ spin_unlock(&destroy_lock);
+
+ return true;
+}
+
+/*
+ * Free fsnotify mark. The freeing is actually happening from a workqueue which
+ * first waits for srcu period end. Caller must have a reference to the mark
+ * or be protected by fsnotify_mark_srcu.
+ */
+void fsnotify_free_mark(struct fsnotify_mark *mark)
+{
+ if (__fsnotify_free_mark(mark)) {
+ queue_delayed_work(system_unbound_wq, &reaper_work,
+ FSNOTIFY_REAPER_DELAY);
+ }
}
void fsnotify_destroy_mark(struct fsnotify_mark *mark,
@@ -468,11 +485,29 @@
}
/*
- * Given a group, destroy all of the marks associated with that group.
+ * Given a group, prepare for freeing all the marks associated with that group.
+ * The marks are attached to the list of marks prepared for destruction, the
+ * caller is responsible for freeing marks in that list after SRCU period has
+ * ended.
*/
-void fsnotify_clear_marks_by_group(struct fsnotify_group *group)
+void fsnotify_detach_group_marks(struct fsnotify_group *group)
{
- fsnotify_clear_marks_by_group_flags(group, (unsigned int)-1);
+ struct fsnotify_mark *mark;
+
+ while (1) {
+ mutex_lock_nested(&group->mark_mutex, SINGLE_DEPTH_NESTING);
+ if (list_empty(&group->marks_list)) {
+ mutex_unlock(&group->mark_mutex);
+ break;
+ }
+ mark = list_first_entry(&group->marks_list,
+ struct fsnotify_mark, g_list);
+ fsnotify_get_mark(mark);
+ fsnotify_detach_mark(mark);
+ mutex_unlock(&group->mark_mutex);
+ __fsnotify_free_mark(mark);
+ fsnotify_put_mark(mark);
+ }
}
void fsnotify_duplicate_mark(struct fsnotify_mark *new, struct fsnotify_mark *old)
@@ -499,7 +534,11 @@
mark->free_mark = free_mark;
}
-static void fsnotify_mark_destroy(struct work_struct *work)
+/*
+ * Destroy all marks in destroy_list, waits for SRCU period to finish before
+ * actually freeing marks.
+ */
+void fsnotify_mark_destroy_list(void)
{
struct fsnotify_mark *mark, *next;
struct list_head private_destroy_list;
@@ -516,3 +555,8 @@
fsnotify_put_mark(mark);
}
}
+
+static void fsnotify_mark_destroy_workfn(struct work_struct *work)
+{
+ fsnotify_mark_destroy_list();
+}
diff --git a/fs/ocfs2/alloc.c b/fs/ocfs2/alloc.c
index e361d1a..460c0ce 100644
--- a/fs/ocfs2/alloc.c
+++ b/fs/ocfs2/alloc.c
@@ -5351,7 +5351,7 @@
{
int ret;
u32 left_cpos, rec_range, trunc_range;
- int wants_rotate = 0, is_rightmost_tree_rec = 0;
+ int is_rightmost_tree_rec = 0;
struct super_block *sb = ocfs2_metadata_cache_get_super(et->et_ci);
struct ocfs2_path *left_path = NULL;
struct ocfs2_extent_list *el = path_leaf_el(path);
@@ -5457,7 +5457,6 @@
memset(rec, 0, sizeof(*rec));
ocfs2_cleanup_merge(el, index);
- wants_rotate = 1;
next_free = le16_to_cpu(el->l_next_free_rec);
if (is_rightmost_tree_rec && next_free > 1) {
diff --git a/fs/ocfs2/cluster/heartbeat.c b/fs/ocfs2/cluster/heartbeat.c
index 1934abb..a8d15be 100644
--- a/fs/ocfs2/cluster/heartbeat.c
+++ b/fs/ocfs2/cluster/heartbeat.c
@@ -1456,7 +1456,6 @@
static int o2hb_read_block_input(struct o2hb_region *reg,
const char *page,
- size_t count,
unsigned long *ret_bytes,
unsigned int *ret_bits)
{
@@ -1499,8 +1498,8 @@
if (reg->hr_bdev)
return -EINVAL;
- status = o2hb_read_block_input(reg, page, count,
- &block_bytes, &block_bits);
+ status = o2hb_read_block_input(reg, page, &block_bytes,
+ &block_bits);
if (status)
return status;
diff --git a/fs/ocfs2/cluster/tcp.c b/fs/ocfs2/cluster/tcp.c
index 2d0acd6..4238eb2 100644
--- a/fs/ocfs2/cluster/tcp.c
+++ b/fs/ocfs2/cluster/tcp.c
@@ -600,10 +600,11 @@
static void o2net_data_ready(struct sock *sk)
{
void (*ready)(struct sock *sk);
+ struct o2net_sock_container *sc;
- read_lock(&sk->sk_callback_lock);
- if (sk->sk_user_data) {
- struct o2net_sock_container *sc = sk->sk_user_data;
+ read_lock_bh(&sk->sk_callback_lock);
+ sc = sk->sk_user_data;
+ if (sc) {
sclog(sc, "data_ready hit\n");
o2net_set_data_ready_time(sc);
o2net_sc_queue_work(sc, &sc->sc_rx_work);
@@ -611,7 +612,7 @@
} else {
ready = sk->sk_data_ready;
}
- read_unlock(&sk->sk_callback_lock);
+ read_unlock_bh(&sk->sk_callback_lock);
ready(sk);
}
@@ -622,7 +623,7 @@
void (*state_change)(struct sock *sk);
struct o2net_sock_container *sc;
- read_lock(&sk->sk_callback_lock);
+ read_lock_bh(&sk->sk_callback_lock);
sc = sk->sk_user_data;
if (sc == NULL) {
state_change = sk->sk_state_change;
@@ -649,7 +650,7 @@
break;
}
out:
- read_unlock(&sk->sk_callback_lock);
+ read_unlock_bh(&sk->sk_callback_lock);
state_change(sk);
}
@@ -2012,7 +2013,7 @@
{
void (*ready)(struct sock *sk);
- read_lock(&sk->sk_callback_lock);
+ read_lock_bh(&sk->sk_callback_lock);
ready = sk->sk_user_data;
if (ready == NULL) { /* check for teardown race */
ready = sk->sk_data_ready;
@@ -2039,7 +2040,7 @@
}
out:
- read_unlock(&sk->sk_callback_lock);
+ read_unlock_bh(&sk->sk_callback_lock);
if (ready != NULL)
ready(sk);
}
diff --git a/fs/ocfs2/journal.h b/fs/ocfs2/journal.h
index f4cd3c3..497a4171 100644
--- a/fs/ocfs2/journal.h
+++ b/fs/ocfs2/journal.h
@@ -619,7 +619,7 @@
static inline int ocfs2_jbd2_file_inode(handle_t *handle, struct inode *inode)
{
- return jbd2_journal_file_inode(handle, &OCFS2_I(inode)->ip_jinode);
+ return jbd2_journal_inode_add_write(handle, &OCFS2_I(inode)->ip_jinode);
}
static inline int ocfs2_begin_ordered_truncate(struct inode *inode,
diff --git a/fs/ocfs2/ocfs2_fs.h b/fs/ocfs2/ocfs2_fs.h
index 540ab5b..44d178b 100644
--- a/fs/ocfs2/ocfs2_fs.h
+++ b/fs/ocfs2/ocfs2_fs.h
@@ -580,7 +580,7 @@
/*00*/ __u8 es_valid;
__u8 es_reserved1[3];
__le32 es_node_num;
-/*10*/
+/*08*/
};
/*
diff --git a/fs/ocfs2/slot_map.c b/fs/ocfs2/slot_map.c
index 1e09592..d740799 100644
--- a/fs/ocfs2/slot_map.c
+++ b/fs/ocfs2/slot_map.c
@@ -535,12 +535,8 @@
spin_unlock(&osb->osb_lock);
status = ocfs2_update_disk_slot(osb, si, slot_num);
- if (status < 0) {
+ if (status < 0)
mlog_errno(status);
- goto bail;
- }
-bail:
ocfs2_free_slot_info(osb);
}
-
diff --git a/fs/proc/array.c b/fs/proc/array.c
index b6c00ce..88c7de1 100644
--- a/fs/proc/array.c
+++ b/fs/proc/array.c
@@ -83,6 +83,7 @@
#include <linux/tracehook.h>
#include <linux/string_helpers.h>
#include <linux/user_namespace.h>
+#include <linux/fs_struct.h>
#include <asm/pgtable.h>
#include <asm/processor.h>
@@ -139,12 +140,25 @@
return task_state_array[fls(state)];
}
+static inline int get_task_umask(struct task_struct *tsk)
+{
+ struct fs_struct *fs;
+ int umask = -ENOENT;
+
+ task_lock(tsk);
+ fs = tsk->fs;
+ if (fs)
+ umask = fs->umask;
+ task_unlock(tsk);
+ return umask;
+}
+
static inline void task_state(struct seq_file *m, struct pid_namespace *ns,
struct pid *pid, struct task_struct *p)
{
struct user_namespace *user_ns = seq_user_ns(m);
struct group_info *group_info;
- int g;
+ int g, umask;
struct task_struct *tracer;
const struct cred *cred;
pid_t ppid, tpid = 0, tgid, ngid;
@@ -162,6 +176,10 @@
ngid = task_numa_group_id(p);
cred = get_task_cred(p);
+ umask = get_task_umask(p);
+ if (umask >= 0)
+ seq_printf(m, "Umask:\t%#04o\n", umask);
+
task_lock(p);
if (p->files)
max_fds = files_fdtable(p->files)->max_fds;
diff --git a/fs/proc/base.c b/fs/proc/base.c
index ff4527d..a11eb71 100644
--- a/fs/proc/base.c
+++ b/fs/proc/base.c
@@ -3163,6 +3163,44 @@
}
/*
+ * proc_tid_comm_permission is a special permission function exclusively
+ * used for the node /proc/<pid>/task/<tid>/comm.
+ * It bypasses generic permission checks in the case where a task of the same
+ * task group attempts to access the node.
+ * The rationale behind this is that glibc and bionic access this node for
+ * cross thread naming (pthread_set/getname_np(!self)). However, if
+ * PR_SET_DUMPABLE gets set to 0 this node among others becomes uid=0 gid=0,
+ * which locks out the cross thread naming implementation.
+ * This function makes sure that the node is always accessible for members of
+ * same thread group.
+ */
+static int proc_tid_comm_permission(struct inode *inode, int mask)
+{
+ bool is_same_tgroup;
+ struct task_struct *task;
+
+ task = get_proc_task(inode);
+ if (!task)
+ return -ESRCH;
+ is_same_tgroup = same_thread_group(current, task);
+ put_task_struct(task);
+
+ if (likely(is_same_tgroup && !(mask & MAY_EXEC))) {
+ /* This file (/proc/<pid>/task/<tid>/comm) can always be
+ * read or written by the members of the corresponding
+ * thread group.
+ */
+ return 0;
+ }
+
+ return generic_permission(inode, mask);
+}
+
+static const struct inode_operations proc_tid_comm_inode_operations = {
+ .permission = proc_tid_comm_permission,
+};
+
+/*
* Tasks
*/
static const struct pid_entry tid_base_stuff[] = {
@@ -3180,7 +3218,9 @@
#ifdef CONFIG_SCHED_DEBUG
REG("sched", S_IRUGO|S_IWUSR, proc_pid_sched_operations),
#endif
- REG("comm", S_IRUGO|S_IWUSR, proc_pid_set_comm_operations),
+ NOD("comm", S_IFREG|S_IRUGO|S_IWUSR,
+ &proc_tid_comm_inode_operations,
+ &proc_pid_set_comm_operations, {}),
#ifdef CONFIG_HAVE_ARCH_TRACEHOOK
ONE("syscall", S_IRUSR, proc_pid_syscall),
#endif
diff --git a/fs/proc/page.c b/fs/proc/page.c
index 712f1b9..3ecd445 100644
--- a/fs/proc/page.c
+++ b/fs/proc/page.c
@@ -142,7 +142,7 @@
/*
- * Caveats on high order pages: page->_count will only be set
+ * Caveats on high order pages: page->_refcount will only be set
* -1 on the head page; SLUB/SLQB do the same for PG_slab;
* SLOB won't set PG_slab at all on compound pages.
*/
diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index 5415835..4648c7f 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -1027,11 +1027,15 @@
};
if (type == CLEAR_REFS_MM_HIWATER_RSS) {
+ if (down_write_killable(&mm->mmap_sem)) {
+ count = -EINTR;
+ goto out_mm;
+ }
+
/*
* Writing 5 to /proc/pid/clear_refs resets the peak
* resident set size to this mm's current rss value.
*/
- down_write(&mm->mmap_sem);
reset_mm_hiwater_rss(mm);
up_write(&mm->mmap_sem);
goto out_mm;
@@ -1043,7 +1047,10 @@
if (!(vma->vm_flags & VM_SOFTDIRTY))
continue;
up_read(&mm->mmap_sem);
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem)) {
+ count = -EINTR;
+ goto out_mm;
+ }
for (vma = mm->mmap; vma; vma = vma->vm_next) {
vma->vm_flags &= ~VM_SOFTDIRTY;
vma_set_page_prot(vma);
diff --git a/fs/proc/vmcore.c b/fs/proc/vmcore.c
index 8afe10c..8ab782d 100644
--- a/fs/proc/vmcore.c
+++ b/fs/proc/vmcore.c
@@ -1071,7 +1071,7 @@
/* Do some basic Verification. */
if (memcmp(ehdr.e_ident, ELFMAG, SELFMAG) != 0 ||
(ehdr.e_type != ET_CORE) ||
- !elf_check_arch(&ehdr) ||
+ !vmcore_elf32_check_arch(&ehdr) ||
ehdr.e_ident[EI_CLASS] != ELFCLASS32||
ehdr.e_ident[EI_VERSION] != EV_CURRENT ||
ehdr.e_version != EV_CURRENT ||
diff --git a/fs/ramfs/file-nommu.c b/fs/ramfs/file-nommu.c
index a586467..be3ddd1 100644
--- a/fs/ramfs/file-nommu.c
+++ b/fs/ramfs/file-nommu.c
@@ -211,14 +211,11 @@
struct page **pages = NULL, **ptr, *page;
loff_t isize;
- if (!(flags & MAP_SHARED))
- return addr;
-
/* the mapping mustn't extend beyond the EOF */
lpages = (len + PAGE_SIZE - 1) >> PAGE_SHIFT;
isize = i_size_read(inode);
- ret = -EINVAL;
+ ret = -ENOSYS;
maxpages = (isize + PAGE_SIZE - 1) >> PAGE_SHIFT;
if (pgoff >= maxpages)
goto out;
@@ -227,7 +224,6 @@
goto out;
/* gang-find the pages */
- ret = -ENOMEM;
pages = kcalloc(lpages, sizeof(struct page *), GFP_KERNEL);
if (!pages)
goto out_free;
@@ -263,7 +259,7 @@
*/
static int ramfs_nommu_mmap(struct file *file, struct vm_area_struct *vma)
{
- if (!(vma->vm_flags & VM_SHARED))
+ if (!(vma->vm_flags & (VM_SHARED | VM_MAYSHARE)))
return -ENOSYS;
file_accessed(file);
diff --git a/fs/readdir.c b/fs/readdir.c
index a86c6c0..68ef06e 100644
--- a/fs/readdir.c
+++ b/fs/readdir.c
@@ -182,6 +182,8 @@
}
dirent = buf->previous;
if (dirent) {
+ if (signal_pending(current))
+ return -EINTR;
if (__put_user(offset, &dirent->d_off))
goto efault;
}
@@ -261,6 +263,8 @@
return -EINVAL;
dirent = buf->previous;
if (dirent) {
+ if (signal_pending(current))
+ return -EINTR;
if (__put_user(offset, &dirent->d_off))
goto efault;
}
diff --git a/fs/reiserfs/objectid.c b/fs/reiserfs/objectid.c
index 99a5d5d..415d66c 100644
--- a/fs/reiserfs/objectid.c
+++ b/fs/reiserfs/objectid.c
@@ -3,8 +3,8 @@
*/
#include <linux/string.h>
-#include <linux/random.h>
#include <linux/time.h>
+#include <linux/uuid.h>
#include "reiserfs.h"
/* find where objectid map starts */
diff --git a/fs/select.c b/fs/select.c
index 8692939..8ed9da5 100644
--- a/fs/select.c
+++ b/fs/select.c
@@ -47,7 +47,7 @@
#define MAX_SLACK (100 * NSEC_PER_MSEC)
-static long __estimate_accuracy(struct timespec *tv)
+static long __estimate_accuracy(struct timespec64 *tv)
{
long slack;
int divfactor = 1000;
@@ -70,10 +70,10 @@
return slack;
}
-u64 select_estimate_accuracy(struct timespec *tv)
+u64 select_estimate_accuracy(struct timespec64 *tv)
{
u64 ret;
- struct timespec now;
+ struct timespec64 now;
/*
* Realtime tasks get a slack of 0 for obvious reasons.
@@ -82,8 +82,8 @@
if (rt_task(current))
return 0;
- ktime_get_ts(&now);
- now = timespec_sub(*tv, now);
+ ktime_get_ts64(&now);
+ now = timespec64_sub(*tv, now);
ret = __estimate_accuracy(&now);
if (ret < current->timer_slack_ns)
return current->timer_slack_ns;
@@ -260,7 +260,7 @@
/**
* poll_select_set_timeout - helper function to setup the timeout value
- * @to: pointer to timespec variable for the final timeout
+ * @to: pointer to timespec64 variable for the final timeout
* @sec: seconds (from user space)
* @nsec: nanoseconds (from user space)
*
@@ -269,26 +269,28 @@
*
* Returns -EINVAL if sec/nsec are not normalized. Otherwise 0.
*/
-int poll_select_set_timeout(struct timespec *to, long sec, long nsec)
+int poll_select_set_timeout(struct timespec64 *to, time64_t sec, long nsec)
{
- struct timespec ts = {.tv_sec = sec, .tv_nsec = nsec};
+ struct timespec64 ts = {.tv_sec = sec, .tv_nsec = nsec};
- if (!timespec_valid(&ts))
+ if (!timespec64_valid(&ts))
return -EINVAL;
/* Optimize for the zero timeout value here */
if (!sec && !nsec) {
to->tv_sec = to->tv_nsec = 0;
} else {
- ktime_get_ts(to);
- *to = timespec_add_safe(*to, ts);
+ ktime_get_ts64(to);
+ *to = timespec64_add_safe(*to, ts);
}
return 0;
}
-static int poll_select_copy_remaining(struct timespec *end_time, void __user *p,
+static int poll_select_copy_remaining(struct timespec64 *end_time,
+ void __user *p,
int timeval, int ret)
{
+ struct timespec64 rts64;
struct timespec rts;
struct timeval rtv;
@@ -302,16 +304,18 @@
if (!end_time->tv_sec && !end_time->tv_nsec)
return ret;
- ktime_get_ts(&rts);
- rts = timespec_sub(*end_time, rts);
- if (rts.tv_sec < 0)
- rts.tv_sec = rts.tv_nsec = 0;
+ ktime_get_ts64(&rts64);
+ rts64 = timespec64_sub(*end_time, rts64);
+ if (rts64.tv_sec < 0)
+ rts64.tv_sec = rts64.tv_nsec = 0;
+
+ rts = timespec64_to_timespec(rts64);
if (timeval) {
if (sizeof(rtv) > sizeof(rtv.tv_sec) + sizeof(rtv.tv_usec))
memset(&rtv, 0, sizeof(rtv));
- rtv.tv_sec = rts.tv_sec;
- rtv.tv_usec = rts.tv_nsec / NSEC_PER_USEC;
+ rtv.tv_sec = rts64.tv_sec;
+ rtv.tv_usec = rts64.tv_nsec / NSEC_PER_USEC;
if (!copy_to_user(p, &rtv, sizeof(rtv)))
return ret;
@@ -396,7 +400,7 @@
wait->_key |= POLLOUT_SET;
}
-int do_select(int n, fd_set_bits *fds, struct timespec *end_time)
+int do_select(int n, fd_set_bits *fds, struct timespec64 *end_time)
{
ktime_t expire, *to = NULL;
struct poll_wqueues table;
@@ -522,7 +526,7 @@
* pointer to the expiry value.
*/
if (end_time && !to) {
- expire = timespec_to_ktime(*end_time);
+ expire = timespec64_to_ktime(*end_time);
to = &expire;
}
@@ -545,7 +549,7 @@
* I'm trying ERESTARTNOHAND which restart only when you want to.
*/
int core_sys_select(int n, fd_set __user *inp, fd_set __user *outp,
- fd_set __user *exp, struct timespec *end_time)
+ fd_set __user *exp, struct timespec64 *end_time)
{
fd_set_bits fds;
void *bits;
@@ -622,7 +626,7 @@
SYSCALL_DEFINE5(select, int, n, fd_set __user *, inp, fd_set __user *, outp,
fd_set __user *, exp, struct timeval __user *, tvp)
{
- struct timespec end_time, *to = NULL;
+ struct timespec64 end_time, *to = NULL;
struct timeval tv;
int ret;
@@ -648,15 +652,17 @@
const sigset_t __user *sigmask, size_t sigsetsize)
{
sigset_t ksigmask, sigsaved;
- struct timespec ts, end_time, *to = NULL;
+ struct timespec ts;
+ struct timespec64 ts64, end_time, *to = NULL;
int ret;
if (tsp) {
if (copy_from_user(&ts, tsp, sizeof(ts)))
return -EFAULT;
+ ts64 = timespec_to_timespec64(ts);
to = &end_time;
- if (poll_select_set_timeout(to, ts.tv_sec, ts.tv_nsec))
+ if (poll_select_set_timeout(to, ts64.tv_sec, ts64.tv_nsec))
return -EINVAL;
}
@@ -779,7 +785,7 @@
}
static int do_poll(struct poll_list *list, struct poll_wqueues *wait,
- struct timespec *end_time)
+ struct timespec64 *end_time)
{
poll_table* pt = &wait->pt;
ktime_t expire, *to = NULL;
@@ -854,7 +860,7 @@
* pointer to the expiry value.
*/
if (end_time && !to) {
- expire = timespec_to_ktime(*end_time);
+ expire = timespec64_to_ktime(*end_time);
to = &expire;
}
@@ -868,7 +874,7 @@
sizeof(struct pollfd))
int do_sys_poll(struct pollfd __user *ufds, unsigned int nfds,
- struct timespec *end_time)
+ struct timespec64 *end_time)
{
struct poll_wqueues table;
int err = -EFAULT, fdcount, len, size;
@@ -936,7 +942,7 @@
{
struct pollfd __user *ufds = restart_block->poll.ufds;
int nfds = restart_block->poll.nfds;
- struct timespec *to = NULL, end_time;
+ struct timespec64 *to = NULL, end_time;
int ret;
if (restart_block->poll.has_timeout) {
@@ -957,7 +963,7 @@
SYSCALL_DEFINE3(poll, struct pollfd __user *, ufds, unsigned int, nfds,
int, timeout_msecs)
{
- struct timespec end_time, *to = NULL;
+ struct timespec64 end_time, *to = NULL;
int ret;
if (timeout_msecs >= 0) {
@@ -993,7 +999,8 @@
size_t, sigsetsize)
{
sigset_t ksigmask, sigsaved;
- struct timespec ts, end_time, *to = NULL;
+ struct timespec ts;
+ struct timespec64 end_time, *to = NULL;
int ret;
if (tsp) {
diff --git a/fs/ubifs/sb.c b/fs/ubifs/sb.c
index f4fbc7b..3cbb904 100644
--- a/fs/ubifs/sb.c
+++ b/fs/ubifs/sb.c
@@ -28,8 +28,8 @@
#include "ubifs.h"
#include <linux/slab.h>
-#include <linux/random.h>
#include <linux/math64.h>
+#include <linux/uuid.h>
/*
* Default journal size in logical eraseblocks as a percent of total
diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
index 66cdb44..2d97952 100644
--- a/fs/userfaultfd.c
+++ b/fs/userfaultfd.c
@@ -137,7 +137,7 @@
VM_BUG_ON(waitqueue_active(&ctx->fault_wqh));
VM_BUG_ON(spin_is_locked(&ctx->fd_wqh.lock));
VM_BUG_ON(waitqueue_active(&ctx->fd_wqh));
- mmput(ctx->mm);
+ mmdrop(ctx->mm);
kmem_cache_free(userfaultfd_ctx_cachep, ctx);
}
}
@@ -434,6 +434,9 @@
ACCESS_ONCE(ctx->released) = true;
+ if (!mmget_not_zero(mm))
+ goto wakeup;
+
/*
* Flush page faults out of all CPUs. NOTE: all page faults
* must be retried without returning VM_FAULT_SIGBUS if
@@ -466,7 +469,8 @@
vma->vm_userfaultfd_ctx = NULL_VM_UFFD_CTX;
}
up_write(&mm->mmap_sem);
-
+ mmput(mm);
+wakeup:
/*
* After no new page faults can wait on this fault_*wqh, flush
* the last page faults that may have been already waiting on
@@ -760,10 +764,12 @@
start = uffdio_register.range.start;
end = start + uffdio_register.range.len;
+ ret = -ENOMEM;
+ if (!mmget_not_zero(mm))
+ goto out;
+
down_write(&mm->mmap_sem);
vma = find_vma_prev(mm, start, &prev);
-
- ret = -ENOMEM;
if (!vma)
goto out_unlock;
@@ -864,6 +870,7 @@
} while (vma && vma->vm_start < end);
out_unlock:
up_write(&mm->mmap_sem);
+ mmput(mm);
if (!ret) {
/*
* Now that we scanned all vmas we can already tell
@@ -902,10 +909,12 @@
start = uffdio_unregister.start;
end = start + uffdio_unregister.len;
+ ret = -ENOMEM;
+ if (!mmget_not_zero(mm))
+ goto out;
+
down_write(&mm->mmap_sem);
vma = find_vma_prev(mm, start, &prev);
-
- ret = -ENOMEM;
if (!vma)
goto out_unlock;
@@ -998,6 +1007,7 @@
} while (vma && vma->vm_start < end);
out_unlock:
up_write(&mm->mmap_sem);
+ mmput(mm);
out:
return ret;
}
@@ -1067,9 +1077,11 @@
goto out;
if (uffdio_copy.mode & ~UFFDIO_COPY_MODE_DONTWAKE)
goto out;
-
- ret = mcopy_atomic(ctx->mm, uffdio_copy.dst, uffdio_copy.src,
- uffdio_copy.len);
+ if (mmget_not_zero(ctx->mm)) {
+ ret = mcopy_atomic(ctx->mm, uffdio_copy.dst, uffdio_copy.src,
+ uffdio_copy.len);
+ mmput(ctx->mm);
+ }
if (unlikely(put_user(ret, &user_uffdio_copy->copy)))
return -EFAULT;
if (ret < 0)
@@ -1110,8 +1122,11 @@
if (uffdio_zeropage.mode & ~UFFDIO_ZEROPAGE_MODE_DONTWAKE)
goto out;
- ret = mfill_zeropage(ctx->mm, uffdio_zeropage.range.start,
- uffdio_zeropage.range.len);
+ if (mmget_not_zero(ctx->mm)) {
+ ret = mfill_zeropage(ctx->mm, uffdio_zeropage.range.start,
+ uffdio_zeropage.range.len);
+ mmput(ctx->mm);
+ }
if (unlikely(put_user(ret, &user_uffdio_zeropage->zeropage)))
return -EFAULT;
if (ret < 0)
@@ -1289,12 +1304,12 @@
ctx->released = false;
ctx->mm = current->mm;
/* prevent the mm struct to be freed */
- atomic_inc(&ctx->mm->mm_users);
+ atomic_inc(&ctx->mm->mm_count);
file = anon_inode_getfile("[userfaultfd]", &userfaultfd_fops, ctx,
O_RDWR | (flags & UFFD_SHARED_FCNTL_FLAGS));
if (IS_ERR(file)) {
- mmput(ctx->mm);
+ mmdrop(ctx->mm);
kmem_cache_free(userfaultfd_ctx_cachep, ctx);
}
out:
diff --git a/include/acpi/acpi_bus.h b/include/acpi/acpi_bus.h
index 4d40e9b..788c6c3 100644
--- a/include/acpi/acpi_bus.h
+++ b/include/acpi/acpi_bus.h
@@ -61,12 +61,12 @@
bool acpi_bay_match(acpi_handle handle);
bool acpi_dock_match(acpi_handle handle);
-bool acpi_check_dsm(acpi_handle handle, const u8 *uuid, int rev, u64 funcs);
+bool acpi_check_dsm(acpi_handle handle, const u8 *uuid, u64 rev, u64 funcs);
union acpi_object *acpi_evaluate_dsm(acpi_handle handle, const u8 *uuid,
- int rev, int func, union acpi_object *argv4);
+ u64 rev, u64 func, union acpi_object *argv4);
static inline union acpi_object *
-acpi_evaluate_dsm_typed(acpi_handle handle, const u8 *uuid, int rev, int func,
+acpi_evaluate_dsm_typed(acpi_handle handle, const u8 *uuid, u64 rev, u64 func,
union acpi_object *argv4, acpi_object_type type)
{
union acpi_object *obj;
diff --git a/include/asm-generic/io.h b/include/asm-generic/io.h
index eed3bbe..002b81f 100644
--- a/include/asm-generic/io.h
+++ b/include/asm-generic/io.h
@@ -191,7 +191,7 @@
#define readl_relaxed readl
#endif
-#ifndef readq_relaxed
+#if defined(readq) && !defined(readq_relaxed)
#define readq_relaxed readq
#endif
@@ -207,7 +207,7 @@
#define writel_relaxed writel
#endif
-#ifndef writeq_relaxed
+#if defined(writeq) && !defined(writeq_relaxed)
#define writeq_relaxed writeq
#endif
diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h
index 9401f48..d4458b6 100644
--- a/include/asm-generic/pgtable.h
+++ b/include/asm-generic/pgtable.h
@@ -806,4 +806,12 @@
#define io_remap_pfn_range remap_pfn_range
#endif
+#ifndef has_transparent_hugepage
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+#define has_transparent_hugepage() 1
+#else
+#define has_transparent_hugepage() 0
+#endif
+#endif
+
#endif /* _ASM_GENERIC_PGTABLE_H */
diff --git a/include/asm-generic/qspinlock.h b/include/asm-generic/qspinlock.h
index 35a52a8..6bd0570 100644
--- a/include/asm-generic/qspinlock.h
+++ b/include/asm-generic/qspinlock.h
@@ -28,7 +28,30 @@
*/
static __always_inline int queued_spin_is_locked(struct qspinlock *lock)
{
- return atomic_read(&lock->val);
+ /*
+ * queued_spin_lock_slowpath() can ACQUIRE the lock before
+ * issuing the unordered store that sets _Q_LOCKED_VAL.
+ *
+ * See both smp_cond_acquire() sites for more detail.
+ *
+ * This however means that in code like:
+ *
+ * spin_lock(A) spin_lock(B)
+ * spin_unlock_wait(B) spin_is_locked(A)
+ * do_something() do_something()
+ *
+ * Both CPUs can end up running do_something() because the store
+ * setting _Q_LOCKED_VAL will pass through the loads in
+ * spin_unlock_wait() and/or spin_is_locked().
+ *
+ * Avoid this by issuing a full memory barrier between the spin_lock()
+ * and the loads in spin_unlock_wait() and spin_is_locked().
+ *
+ * Note that regular mutual exclusion doesn't care about this
+ * delayed store.
+ */
+ smp_mb();
+ return atomic_read(&lock->val) & _Q_LOCKED_MASK;
}
/**
@@ -108,6 +131,8 @@
*/
static inline void queued_spin_unlock_wait(struct qspinlock *lock)
{
+ /* See queued_spin_is_locked() */
+ smp_mb();
while (atomic_read(&lock->val) & _Q_LOCKED_MASK)
cpu_relax();
}
diff --git a/include/asm-generic/seccomp.h b/include/asm-generic/seccomp.h
index c9ccafa..e74072d 100644
--- a/include/asm-generic/seccomp.h
+++ b/include/asm-generic/seccomp.h
@@ -29,4 +29,18 @@
#define __NR_seccomp_sigreturn __NR_rt_sigreturn
#endif
+#ifdef CONFIG_COMPAT
+#ifndef get_compat_mode1_syscalls
+static inline const int *get_compat_mode1_syscalls(void)
+{
+ static const int mode1_syscalls_32[] = {
+ __NR_seccomp_read_32, __NR_seccomp_write_32,
+ __NR_seccomp_exit_32, __NR_seccomp_sigreturn_32,
+ 0, /* null terminated */
+ };
+ return mode1_syscalls_32;
+}
+#endif
+#endif /* CONFIG_COMPAT */
+
#endif /* _ASM_GENERIC_SECCOMP_H */
diff --git a/include/asm-generic/siginfo.h b/include/asm-generic/siginfo.h
index 3d1a3af..a2508a8 100644
--- a/include/asm-generic/siginfo.h
+++ b/include/asm-generic/siginfo.h
@@ -17,21 +17,6 @@
struct siginfo;
void do_schedule_next_timer(struct siginfo *info);
-#ifndef HAVE_ARCH_COPY_SIGINFO
-
-#include <linux/string.h>
-
-static inline void copy_siginfo(struct siginfo *to, struct siginfo *from)
-{
- if (from->si_code < 0)
- memcpy(to, from, sizeof(*to));
- else
- /* _sigchld is currently the largest know union member */
- memcpy(to, from, __ARCH_SI_PREAMBLE_SIZE + sizeof(from->_sifields._sigchld));
-}
-
-#endif
-
extern int copy_siginfo_to_user(struct siginfo __user *to, const struct siginfo *from);
#endif
diff --git a/include/clocksource/arm_arch_timer.h b/include/clocksource/arm_arch_timer.h
index 25d0914..caedb74 100644
--- a/include/clocksource/arm_arch_timer.h
+++ b/include/clocksource/arm_arch_timer.h
@@ -49,11 +49,16 @@
#define ARCH_TIMER_EVT_STREAM_FREQ 10000 /* 100us */
+struct arch_timer_kvm_info {
+ struct timecounter timecounter;
+ int virtual_irq;
+};
+
#ifdef CONFIG_ARM_ARCH_TIMER
extern u32 arch_timer_get_rate(void);
extern u64 (*arch_timer_read_counter)(void);
-extern struct timecounter *arch_timer_get_timecounter(void);
+extern struct arch_timer_kvm_info *arch_timer_get_kvm_info(void);
#else
@@ -67,11 +72,6 @@
return 0;
}
-static inline struct timecounter *arch_timer_get_timecounter(void)
-{
- return NULL;
-}
-
#endif
#endif
diff --git a/include/crypto/pkcs7.h b/include/crypto/pkcs7.h
index 441aff9..583f199 100644
--- a/include/crypto/pkcs7.h
+++ b/include/crypto/pkcs7.h
@@ -12,6 +12,7 @@
#ifndef _CRYPTO_PKCS7_H
#define _CRYPTO_PKCS7_H
+#include <linux/verification.h>
#include <crypto/public_key.h>
struct key;
@@ -26,14 +27,13 @@
extern int pkcs7_get_content_data(const struct pkcs7_message *pkcs7,
const void **_data, size_t *_datalen,
- bool want_wrapper);
+ size_t *_headerlen);
/*
* pkcs7_trust.c
*/
extern int pkcs7_validate_trust(struct pkcs7_message *pkcs7,
- struct key *trust_keyring,
- bool *_trusted);
+ struct key *trust_keyring);
/*
* pkcs7_verify.c
diff --git a/include/crypto/public_key.h b/include/crypto/public_key.h
index aa730ea..882ca0e1 100644
--- a/include/crypto/public_key.h
+++ b/include/crypto/public_key.h
@@ -15,20 +15,6 @@
#define _LINUX_PUBLIC_KEY_H
/*
- * The use to which an asymmetric key is being put.
- */
-enum key_being_used_for {
- VERIFYING_MODULE_SIGNATURE,
- VERIFYING_FIRMWARE_SIGNATURE,
- VERIFYING_KEXEC_PE_SIGNATURE,
- VERIFYING_KEY_SIGNATURE,
- VERIFYING_KEY_SELF_SIGNATURE,
- VERIFYING_UNSPECIFIED_SIGNATURE,
- NR__KEY_BEING_USED_FOR
-};
-extern const char *const key_being_used_for[NR__KEY_BEING_USED_FOR];
-
-/*
* Cryptographic data for the public-key subtype of the asymmetric key type.
*
* Note that this may include private part of the key as well as the public
@@ -41,12 +27,13 @@
const char *pkey_algo;
};
-extern void public_key_destroy(void *payload);
+extern void public_key_free(struct public_key *key);
/*
* Public key cryptography signature data
*/
struct public_key_signature {
+ struct asymmetric_key_id *auth_ids[2];
u8 *s; /* Signature */
u32 s_size; /* Number of bytes in signature */
u8 *digest;
@@ -55,17 +42,21 @@
const char *hash_algo;
};
+extern void public_key_signature_free(struct public_key_signature *sig);
+
extern struct asymmetric_key_subtype public_key_subtype;
+
struct key;
+struct key_type;
+union key_payload;
+
+extern int restrict_link_by_signature(struct key *trust_keyring,
+ const struct key_type *type,
+ const union key_payload *payload);
+
extern int verify_signature(const struct key *key,
const struct public_key_signature *sig);
-struct asymmetric_key_id;
-extern struct key *x509_request_asymmetric_key(struct key *keyring,
- const struct asymmetric_key_id *id,
- const struct asymmetric_key_id *skid,
- bool partial);
-
int public_key_verify_signature(const struct public_key *pkey,
const struct public_key_signature *sig);
diff --git a/include/drm/bridge/analogix_dp.h b/include/drm/bridge/analogix_dp.h
new file mode 100644
index 0000000..25afb31
--- /dev/null
+++ b/include/drm/bridge/analogix_dp.h
@@ -0,0 +1,41 @@
+/*
+ * Analogix DP (Display Port) Core interface driver.
+ *
+ * Copyright (C) 2015 Rockchip Electronics Co., Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+#ifndef _ANALOGIX_DP_H_
+#define _ANALOGIX_DP_H_
+
+#include <drm/drm_crtc.h>
+
+enum analogix_dp_devtype {
+ EXYNOS_DP,
+ RK3288_DP,
+};
+
+struct analogix_dp_plat_data {
+ enum analogix_dp_devtype dev_type;
+ struct drm_panel *panel;
+ struct drm_encoder *encoder;
+ struct drm_connector *connector;
+
+ int (*power_on)(struct analogix_dp_plat_data *);
+ int (*power_off)(struct analogix_dp_plat_data *);
+ int (*attach)(struct analogix_dp_plat_data *, struct drm_bridge *,
+ struct drm_connector *);
+ int (*get_modes)(struct analogix_dp_plat_data *);
+};
+
+int analogix_dp_resume(struct device *dev);
+int analogix_dp_suspend(struct device *dev);
+
+int analogix_dp_bind(struct device *dev, struct drm_device *drm_dev,
+ struct analogix_dp_plat_data *plat_data);
+void analogix_dp_unbind(struct device *dev, struct device *master, void *data);
+
+#endif /* _ANALOGIX_DP_H_ */
diff --git a/include/drm/drmP.h b/include/drm/drmP.h
index 3c8422c..84f1a8e 100644
--- a/include/drm/drmP.h
+++ b/include/drm/drmP.h
@@ -90,7 +90,7 @@
struct dma_buf_attachment;
/*
- * 4 debug categories are defined:
+ * The following categories are defined:
*
* CORE: Used in the generic drm code: drm_ioctl.c, drm_mm.c, drm_memory.c, ...
* This is the category used by the DRM_DEBUG() macro.
@@ -580,12 +580,21 @@
void (*debugfs_cleanup)(struct drm_minor *minor);
/**
- * Driver-specific constructor for drm_gem_objects, to set up
- * obj->driver_private.
+ * @gem_free_object: deconstructor for drm_gem_objects
*
- * Returns 0 on success.
+ * This is deprecated and should not be used by new drivers. Use
+ * @gem_free_object_unlocked instead.
*/
void (*gem_free_object) (struct drm_gem_object *obj);
+
+ /**
+ * @gem_free_object_unlocked: deconstructor for drm_gem_objects
+ *
+ * This is for drivers which are not encumbered with dev->struct_mutex
+ * legacy locking schemes. Use this hook instead of @gem_free_object.
+ */
+ void (*gem_free_object_unlocked) (struct drm_gem_object *obj);
+
int (*gem_open_object) (struct drm_gem_object *, struct drm_file *);
void (*gem_close_object) (struct drm_gem_object *, struct drm_file *);
@@ -769,6 +778,7 @@
atomic_t buf_alloc; /**< Buffer allocation in progress */
/*@} */
+ struct mutex filelist_mutex;
struct list_head filelist;
/** \name Memory management */
@@ -805,14 +815,6 @@
int irq;
/*
- * At load time, disabling the vblank interrupt won't be allowed since
- * old clients may not call the modeset ioctl and therefore misbehave.
- * Once the modeset ioctl *has* been called though, we can safely
- * disable them when unused.
- */
- bool vblank_disable_allowed;
-
- /*
* If true, vblank interrupt will be disabled immediately when the
* refcount drops to zero, as opposed to via the vblank disable
* timer.
diff --git a/include/drm/drm_agpsupport.h b/include/drm/drm_agpsupport.h
index 193ef19..b2d9126 100644
--- a/include/drm/drm_agpsupport.h
+++ b/include/drm/drm_agpsupport.h
@@ -37,7 +37,7 @@
uint32_t type);
struct drm_agp_head *drm_agp_init(struct drm_device *dev);
-void drm_agp_clear(struct drm_device *dev);
+void drm_legacy_agp_clear(struct drm_device *dev);
int drm_agp_acquire(struct drm_device *dev);
int drm_agp_acquire_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv);
@@ -93,7 +93,7 @@
return NULL;
}
-static inline void drm_agp_clear(struct drm_device *dev)
+static inline void drm_legacy_agp_clear(struct drm_device *dev)
{
}
diff --git a/include/drm/drm_atomic.h b/include/drm/drm_atomic.h
index d3eaa5d..92c84e9 100644
--- a/include/drm/drm_atomic.h
+++ b/include/drm/drm_atomic.h
@@ -137,7 +137,7 @@
int __must_check drm_atomic_check_only(struct drm_atomic_state *state);
int __must_check drm_atomic_commit(struct drm_atomic_state *state);
-int __must_check drm_atomic_async_commit(struct drm_atomic_state *state);
+int __must_check drm_atomic_nonblocking_commit(struct drm_atomic_state *state);
#define for_each_connector_in_state(state, connector, connector_state, __i) \
for ((__i) = 0; \
diff --git a/include/drm/drm_atomic_helper.h b/include/drm/drm_atomic_helper.h
index 9054598c..d473dcc 100644
--- a/include/drm/drm_atomic_helper.h
+++ b/include/drm/drm_atomic_helper.h
@@ -40,8 +40,10 @@
struct drm_atomic_state *state);
int drm_atomic_helper_commit(struct drm_device *dev,
struct drm_atomic_state *state,
- bool async);
+ bool nonblock);
+void drm_atomic_helper_wait_for_fences(struct drm_device *dev,
+ struct drm_atomic_state *state);
bool drm_atomic_helper_framebuffer_changed(struct drm_device *dev,
struct drm_atomic_state *old_state,
struct drm_crtc *crtc);
@@ -108,6 +110,8 @@
uint32_t flags);
int drm_atomic_helper_connector_dpms(struct drm_connector *connector,
int mode);
+struct drm_encoder *
+drm_atomic_helper_best_encoder(struct drm_connector *connector);
/* default implementations for state handling */
void drm_atomic_helper_crtc_reset(struct drm_crtc *crtc);
@@ -115,8 +119,7 @@
struct drm_crtc_state *state);
struct drm_crtc_state *
drm_atomic_helper_crtc_duplicate_state(struct drm_crtc *crtc);
-void __drm_atomic_helper_crtc_destroy_state(struct drm_crtc *crtc,
- struct drm_crtc_state *state);
+void __drm_atomic_helper_crtc_destroy_state(struct drm_crtc_state *state);
void drm_atomic_helper_crtc_destroy_state(struct drm_crtc *crtc,
struct drm_crtc_state *state);
@@ -125,8 +128,7 @@
struct drm_plane_state *state);
struct drm_plane_state *
drm_atomic_helper_plane_duplicate_state(struct drm_plane *plane);
-void __drm_atomic_helper_plane_destroy_state(struct drm_plane *plane,
- struct drm_plane_state *state);
+void __drm_atomic_helper_plane_destroy_state(struct drm_plane_state *state);
void drm_atomic_helper_plane_destroy_state(struct drm_plane *plane,
struct drm_plane_state *state);
@@ -142,8 +144,7 @@
drm_atomic_helper_duplicate_state(struct drm_device *dev,
struct drm_modeset_acquire_ctx *ctx);
void
-__drm_atomic_helper_connector_destroy_state(struct drm_connector *connector,
- struct drm_connector_state *state);
+__drm_atomic_helper_connector_destroy_state(struct drm_connector_state *state);
void drm_atomic_helper_connector_destroy_state(struct drm_connector *connector,
struct drm_connector_state *state);
void drm_atomic_helper_legacy_gamma_set(struct drm_crtc *crtc,
diff --git a/include/drm/drm_crtc.h b/include/drm/drm_crtc.h
index e0170bf..d1559cd 100644
--- a/include/drm/drm_crtc.h
+++ b/include/drm/drm_crtc.h
@@ -45,20 +45,12 @@
struct device_node;
struct fence;
-#define DRM_MODE_OBJECT_CRTC 0xcccccccc
-#define DRM_MODE_OBJECT_CONNECTOR 0xc0c0c0c0
-#define DRM_MODE_OBJECT_ENCODER 0xe0e0e0e0
-#define DRM_MODE_OBJECT_MODE 0xdededede
-#define DRM_MODE_OBJECT_PROPERTY 0xb0b0b0b0
-#define DRM_MODE_OBJECT_FB 0xfbfbfbfb
-#define DRM_MODE_OBJECT_BLOB 0xbbbbbbbb
-#define DRM_MODE_OBJECT_PLANE 0xeeeeeeee
-#define DRM_MODE_OBJECT_ANY 0
-
struct drm_mode_object {
uint32_t id;
uint32_t type;
struct drm_object_properties *properties;
+ struct kref refcount;
+ void (*free_cb)(struct kref *kref);
};
#define DRM_OBJECT_MAX_PROPERTY 24
@@ -126,6 +118,14 @@
#define DRM_COLOR_FORMAT_RGB444 (1<<0)
#define DRM_COLOR_FORMAT_YCRCB444 (1<<1)
#define DRM_COLOR_FORMAT_YCRCB422 (1<<2)
+
+#define DRM_BUS_FLAG_DE_LOW (1<<0)
+#define DRM_BUS_FLAG_DE_HIGH (1<<1)
+/* drive data on pos. edge */
+#define DRM_BUS_FLAG_PIXDATA_POSEDGE (1<<2)
+/* drive data on neg. edge */
+#define DRM_BUS_FLAG_PIXDATA_NEGEDGE (1<<3)
+
/*
* Describes a given display (e.g. CRT or flat panel) and its limitations.
*/
@@ -147,6 +147,7 @@
const u32 *bus_formats;
unsigned int num_bus_formats;
+ u32 bus_flags;
/* Mask of supported hdmi deep color modes */
u8 edid_hdmi_dc_modes;
@@ -233,8 +234,8 @@
* should be deferred. In cases like this, the driver would like to
* hold a ref to the fb even though it has already been removed from
* userspace perspective.
+ * The refcount is stored inside the mode object.
*/
- struct kref refcount;
/*
* Place on the dev->mode_config.fb_list, access protected by
* dev->mode_config.fb_lock.
@@ -258,7 +259,6 @@
struct drm_property_blob {
struct drm_mode_object base;
struct drm_device *dev;
- struct kref refcount;
struct list_head head_global;
struct list_head head_file;
size_t length;
@@ -1895,7 +1895,7 @@
* drm_atomic_helper_commit(), or one of the exported sub-functions of
* it.
*
- * Asynchronous commits (as indicated with the async parameter) must
+ * Nonblocking commits (as indicated with the nonblock parameter) must
* do any preparatory work which might result in an unsuccessful commit
* in the context of this callback. The only exceptions are hardware
* errors resulting in -EIO. But even in that case the driver must
@@ -1908,7 +1908,7 @@
* The driver must wait for any pending rendering to the new
* framebuffers to complete before executing the flip. It should also
* wait for any pending rendering from other drivers if the underlying
- * buffer is a shared dma-buf. Asynchronous commits must not wait for
+ * buffer is a shared dma-buf. Nonblocking commits must not wait for
* rendering in the context of this callback.
*
* An application can request to be notified when the atomic commit has
@@ -1939,7 +1939,7 @@
*
* 0 on success or one of the below negative error codes:
*
- * - -EBUSY, if an asynchronous updated is requested and there is
+ * - -EBUSY, if a nonblocking updated is requested and there is
* an earlier updated pending. Drivers are allowed to support a queue
* of outstanding updates, but currently no driver supports that.
* Note that drivers must wait for preceding updates to complete if a
@@ -1969,7 +1969,7 @@
*/
int (*atomic_commit)(struct drm_device *dev,
struct drm_atomic_state *state,
- bool async);
+ bool nonblock);
/**
* @atomic_state_alloc:
@@ -2259,8 +2259,9 @@
return connector->connector_id;
}
-/* helper to unplug all connectors from sysfs for device */
-extern void drm_connector_unplug_all(struct drm_device *dev);
+/* helpers to {un}register all connectors from sysfs for device */
+extern int drm_connector_register_all(struct drm_device *dev);
+extern void drm_connector_unregister_all(struct drm_device *dev);
extern int drm_bridge_add(struct drm_bridge *bridge);
extern void drm_bridge_remove(struct drm_bridge *bridge);
@@ -2386,8 +2387,6 @@
const struct drm_framebuffer_funcs *funcs);
extern struct drm_framebuffer *drm_framebuffer_lookup(struct drm_device *dev,
uint32_t id);
-extern void drm_framebuffer_unreference(struct drm_framebuffer *fb);
-extern void drm_framebuffer_reference(struct drm_framebuffer *fb);
extern void drm_framebuffer_remove(struct drm_framebuffer *fb);
extern void drm_framebuffer_cleanup(struct drm_framebuffer *fb);
extern void drm_framebuffer_unregister_private(struct drm_framebuffer *fb);
@@ -2445,6 +2444,8 @@
int gamma_size);
extern struct drm_mode_object *drm_mode_object_find(struct drm_device *dev,
uint32_t id, uint32_t type);
+void drm_mode_object_reference(struct drm_mode_object *obj);
+void drm_mode_object_unreference(struct drm_mode_object *obj);
/* IOCTLs */
extern int drm_mode_getresources(struct drm_device *dev,
@@ -2510,6 +2511,8 @@
extern bool drm_edid_block_valid(u8 *raw_edid, int block, bool print_bad_edid,
bool *edid_corrupt);
extern bool drm_edid_is_valid(struct edid *edid);
+extern void drm_edid_get_monitor_name(struct edid *edid, char *name,
+ int buflen);
extern struct drm_tile_group *drm_mode_create_tile_group(struct drm_device *dev,
char topology[8]);
@@ -2577,7 +2580,15 @@
return mo ? obj_to_encoder(mo) : NULL;
}
-static inline struct drm_connector *drm_connector_find(struct drm_device *dev,
+/**
+ * drm_connector_lookup - lookup connector object
+ * @dev: DRM device
+ * @id: connector object id
+ *
+ * This function looks up the connector object specified by id
+ * add takes a reference to it.
+ */
+static inline struct drm_connector *drm_connector_lookup(struct drm_device *dev,
uint32_t id)
{
struct drm_mode_object *mo;
@@ -2600,14 +2611,73 @@
static inline uint32_t drm_color_lut_extract(uint32_t user_input,
uint32_t bit_precision)
{
- uint32_t val = user_input + (1 << (16 - bit_precision - 1));
+ uint32_t val = user_input;
uint32_t max = 0xffff >> (16 - bit_precision);
- val >>= 16 - bit_precision;
+ /* Round only if we're not using full precision. */
+ if (bit_precision < 16) {
+ val += 1UL << (16 - bit_precision - 1);
+ val >>= 16 - bit_precision;
+ }
return clamp_val(val, 0, max);
}
+/**
+ * drm_framebuffer_reference - incr the fb refcnt
+ * @fb: framebuffer
+ *
+ * This functions increments the fb's refcount.
+ */
+static inline void drm_framebuffer_reference(struct drm_framebuffer *fb)
+{
+ drm_mode_object_reference(&fb->base);
+}
+
+/**
+ * drm_framebuffer_unreference - unref a framebuffer
+ * @fb: framebuffer to unref
+ *
+ * This functions decrements the fb's refcount and frees it if it drops to zero.
+ */
+static inline void drm_framebuffer_unreference(struct drm_framebuffer *fb)
+{
+ drm_mode_object_unreference(&fb->base);
+}
+
+/**
+ * drm_framebuffer_read_refcount - read the framebuffer reference count.
+ * @fb: framebuffer
+ *
+ * This functions returns the framebuffer's reference count.
+ */
+static inline uint32_t drm_framebuffer_read_refcount(struct drm_framebuffer *fb)
+{
+ return atomic_read(&fb->base.refcount.refcount);
+}
+
+/**
+ * drm_connector_reference - incr the connector refcnt
+ * @connector: connector
+ *
+ * This function increments the connector's refcount.
+ */
+static inline void drm_connector_reference(struct drm_connector *connector)
+{
+ drm_mode_object_reference(&connector->base);
+}
+
+/**
+ * drm_connector_unreference - unref a connector
+ * @connector: connector to unref
+ *
+ * This function decrements the connector's refcount and frees it if it drops to zero.
+ */
+static inline void drm_connector_unreference(struct drm_connector *connector)
+{
+ drm_mode_object_unreference(&connector->base);
+}
+
/* Plane list iterator for legacy (overlay only) planes. */
#define drm_for_each_legacy_plane(plane, dev) \
list_for_each_entry(plane, &(dev)->mode_config.plane_list, head) \
diff --git a/include/drm/drm_displayid.h b/include/drm/drm_displayid.h
index 623b4e9..c0d4df6 100644
--- a/include/drm/drm_displayid.h
+++ b/include/drm/drm_displayid.h
@@ -73,4 +73,21 @@
u8 topology_id[8];
} __packed;
+struct displayid_detailed_timings_1 {
+ u8 pixel_clock[3];
+ u8 flags;
+ u8 hactive[2];
+ u8 hblank[2];
+ u8 hsync[2];
+ u8 hsw[2];
+ u8 vactive[2];
+ u8 vblank[2];
+ u8 vsync[2];
+ u8 vsw[2];
+} __packed;
+
+struct displayid_detailed_timing_block {
+ struct displayid_block base;
+ struct displayid_detailed_timings_1 timings[0];
+};
#endif
diff --git a/include/drm/drm_dp_helper.h b/include/drm/drm_dp_helper.h
index 1252108..9d03f16 100644
--- a/include/drm/drm_dp_helper.h
+++ b/include/drm/drm_dp_helper.h
@@ -73,6 +73,7 @@
# define DP_ENHANCED_FRAME_CAP (1 << 7)
#define DP_MAX_DOWNSPREAD 0x003
+# define DP_MAX_DOWNSPREAD_0_5 (1 << 0)
# define DP_NO_AUX_HANDSHAKE_LINK_TRAINING (1 << 6)
#define DP_NORP 0x004
diff --git a/include/drm/drm_edid.h b/include/drm/drm_edid.h
index dec6221..919933d 100644
--- a/include/drm/drm_edid.h
+++ b/include/drm/drm_edid.h
@@ -328,7 +328,15 @@
int drm_av_sync_delay(struct drm_connector *connector,
const struct drm_display_mode *mode);
struct drm_connector *drm_select_eld(struct drm_encoder *encoder);
+
+#ifdef CONFIG_DRM_LOAD_EDID_FIRMWARE
int drm_load_edid_firmware(struct drm_connector *connector);
+#else
+static inline int drm_load_edid_firmware(struct drm_connector *connector)
+{
+ return 0;
+}
+#endif
int
drm_hdmi_avi_infoframe_from_display_mode(struct hdmi_avi_infoframe *frame,
diff --git a/include/drm/drm_fb_cma_helper.h b/include/drm/drm_fb_cma_helper.h
index be62bd3..fd0dde9 100644
--- a/include/drm/drm_fb_cma_helper.h
+++ b/include/drm/drm_fb_cma_helper.h
@@ -4,11 +4,18 @@
struct drm_fbdev_cma;
struct drm_gem_cma_object;
+struct drm_fb_helper_surface_size;
+struct drm_framebuffer_funcs;
+struct drm_fb_helper_funcs;
struct drm_framebuffer;
+struct drm_fb_helper;
struct drm_device;
struct drm_file;
struct drm_mode_fb_cmd2;
+struct drm_fbdev_cma *drm_fbdev_cma_init_with_funcs(struct drm_device *dev,
+ unsigned int preferred_bpp, unsigned int num_crtc,
+ unsigned int max_conn_count, const struct drm_fb_helper_funcs *funcs);
struct drm_fbdev_cma *drm_fbdev_cma_init(struct drm_device *dev,
unsigned int preferred_bpp, unsigned int num_crtc,
unsigned int max_conn_count);
@@ -16,7 +23,17 @@
void drm_fbdev_cma_restore_mode(struct drm_fbdev_cma *fbdev_cma);
void drm_fbdev_cma_hotplug_event(struct drm_fbdev_cma *fbdev_cma);
+int drm_fbdev_cma_create_with_funcs(struct drm_fb_helper *helper,
+ struct drm_fb_helper_surface_size *sizes,
+ const struct drm_framebuffer_funcs *funcs);
+void drm_fb_cma_destroy(struct drm_framebuffer *fb);
+int drm_fb_cma_create_handle(struct drm_framebuffer *fb,
+ struct drm_file *file_priv, unsigned int *handle);
+
+struct drm_framebuffer *drm_fb_cma_create_with_funcs(struct drm_device *dev,
+ struct drm_file *file_priv, const struct drm_mode_fb_cmd2 *mode_cmd,
+ const struct drm_framebuffer_funcs *funcs);
struct drm_framebuffer *drm_fb_cma_create(struct drm_device *dev,
struct drm_file *file_priv, const struct drm_mode_fb_cmd2 *mode_cmd);
@@ -24,6 +41,8 @@
unsigned int plane);
#ifdef CONFIG_DEBUG_FS
+struct seq_file;
+
int drm_fb_cma_debugfs_show(struct seq_file *m, void *arg);
#endif
diff --git a/include/drm/drm_fb_helper.h b/include/drm/drm_fb_helper.h
index 062723b..5b4aa35 100644
--- a/include/drm/drm_fb_helper.h
+++ b/include/drm/drm_fb_helper.h
@@ -172,6 +172,10 @@
* @funcs: driver callbacks for fb helper
* @fbdev: emulated fbdev device info struct
* @pseudo_palette: fake palette of 16 colors
+ * @dirty_clip: clip rectangle used with deferred_io to accumulate damage to
+ * the screen buffer
+ * @dirty_lock: spinlock protecting @dirty_clip
+ * @dirty_work: worker used to flush the framebuffer
*
* This is the main structure used by the fbdev helpers. Drivers supporting
* fbdev emulation should embedded this into their overall driver structure.
@@ -189,6 +193,9 @@
const struct drm_fb_helper_funcs *funcs;
struct fb_info *fbdev;
u32 pseudo_palette[17];
+ struct drm_clip_rect dirty_clip;
+ spinlock_t dirty_lock;
+ struct work_struct dirty_work;
/**
* @kernel_fb_list:
@@ -245,6 +252,9 @@
void drm_fb_helper_unlink_fbi(struct drm_fb_helper *fb_helper);
+void drm_fb_helper_deferred_io(struct fb_info *info,
+ struct list_head *pagelist);
+
ssize_t drm_fb_helper_sys_read(struct fb_info *info, char __user *buf,
size_t count, loff_t *ppos);
ssize_t drm_fb_helper_sys_write(struct fb_info *info, const char __user *buf,
@@ -368,6 +378,11 @@
{
}
+static inline void drm_fb_helper_deferred_io(struct fb_info *info,
+ struct list_head *pagelist)
+{
+}
+
static inline ssize_t drm_fb_helper_sys_read(struct fb_info *info,
char __user *buf, size_t count,
loff_t *ppos)
diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h
index 0b3e11a..fca1cd1 100644
--- a/include/drm/drm_gem.h
+++ b/include/drm/drm_gem.h
@@ -200,47 +200,29 @@
}
/**
- * drm_gem_object_unreference - release a GEM BO reference
+ * __drm_gem_object_unreference - raw function to release a GEM BO reference
* @obj: GEM buffer object
*
- * This releases a reference to @obj. Callers must hold the dev->struct_mutex
- * lock when calling this function, even when the driver doesn't use
- * dev->struct_mutex for anything.
+ * This function is meant to be used by drivers which are not encumbered with
+ * dev->struct_mutex legacy locking and which are using the
+ * gem_free_object_unlocked callback. It avoids all the locking checks and
+ * locking overhead of drm_gem_object_unreference() and
+ * drm_gem_object_unreference_unlocked().
*
- * For drivers not encumbered with legacy locking use
- * drm_gem_object_unreference_unlocked() instead.
+ * Drivers should never call this directly in their code. Instead they should
+ * wrap it up into a driver_gem_object_unreference(struct driver_gem_object
+ * *obj) wrapper function, and use that. Shared code should never call this, to
+ * avoid breaking drivers by accident which still depend upon dev->struct_mutex
+ * locking.
*/
static inline void
-drm_gem_object_unreference(struct drm_gem_object *obj)
+__drm_gem_object_unreference(struct drm_gem_object *obj)
{
- if (obj != NULL) {
- WARN_ON(!mutex_is_locked(&obj->dev->struct_mutex));
-
- kref_put(&obj->refcount, drm_gem_object_free);
- }
+ kref_put(&obj->refcount, drm_gem_object_free);
}
-/**
- * drm_gem_object_unreference_unlocked - release a GEM BO reference
- * @obj: GEM buffer object
- *
- * This releases a reference to @obj. Callers must not hold the
- * dev->struct_mutex lock when calling this function.
- */
-static inline void
-drm_gem_object_unreference_unlocked(struct drm_gem_object *obj)
-{
- struct drm_device *dev;
-
- if (!obj)
- return;
-
- dev = obj->dev;
- if (kref_put_mutex(&obj->refcount, drm_gem_object_free, &dev->struct_mutex))
- mutex_unlock(&dev->struct_mutex);
- else
- might_lock(&dev->struct_mutex);
-}
+void drm_gem_object_unreference_unlocked(struct drm_gem_object *obj);
+void drm_gem_object_unreference(struct drm_gem_object *obj);
int drm_gem_handle_create(struct drm_file *file_priv,
struct drm_gem_object *obj,
@@ -256,9 +238,7 @@
void drm_gem_put_pages(struct drm_gem_object *obj, struct page **pages,
bool dirty, bool accessed);
-struct drm_gem_object *drm_gem_object_lookup(struct drm_device *dev,
- struct drm_file *filp,
- u32 handle);
+struct drm_gem_object *drm_gem_object_lookup(struct drm_file *filp, u32 handle);
int drm_gem_dumb_destroy(struct drm_file *file,
struct drm_device *dev,
uint32_t handle);
diff --git a/include/drm/drm_legacy.h b/include/drm/drm_legacy.h
index 3e69803..a5ef2c7 100644
--- a/include/drm/drm_legacy.h
+++ b/include/drm/drm_legacy.h
@@ -154,8 +154,10 @@
int drm_legacy_addmap(struct drm_device *d, resource_size_t offset,
unsigned int size, enum drm_map_type type,
enum drm_map_flags flags, struct drm_local_map **map_p);
-int drm_legacy_rmmap(struct drm_device *d, struct drm_local_map *map);
+void drm_legacy_rmmap(struct drm_device *d, struct drm_local_map *map);
int drm_legacy_rmmap_locked(struct drm_device *d, struct drm_local_map *map);
+void drm_legacy_master_rmmaps(struct drm_device *dev,
+ struct drm_master *master);
struct drm_local_map *drm_legacy_getsarea(struct drm_device *dev);
int drm_legacy_mmap(struct file *filp, struct vm_area_struct *vma);
diff --git a/include/drm/drm_mem_util.h b/include/drm/drm_mem_util.h
index e42495a..70d4e22 100644
--- a/include/drm/drm_mem_util.h
+++ b/include/drm/drm_mem_util.h
@@ -54,6 +54,25 @@
GFP_KERNEL | __GFP_HIGHMEM, PAGE_KERNEL);
}
+static __inline__ void *drm_malloc_gfp(size_t nmemb, size_t size, gfp_t gfp)
+{
+ if (size != 0 && nmemb > SIZE_MAX / size)
+ return NULL;
+
+ if (size * nmemb <= PAGE_SIZE)
+ return kmalloc(nmemb * size, gfp);
+
+ if (gfp & __GFP_RECLAIMABLE) {
+ void *ptr = kmalloc(nmemb * size,
+ gfp | __GFP_NOWARN | __GFP_NORETRY);
+ if (ptr)
+ return ptr;
+ }
+
+ return __vmalloc(size * nmemb,
+ gfp | __GFP_HIGHMEM, PAGE_KERNEL);
+}
+
static __inline void drm_free_large(void *ptr)
{
kvfree(ptr);
diff --git a/include/drm/drm_modeset_helper_vtables.h b/include/drm/drm_modeset_helper_vtables.h
index b61c2d4..d4619dc 100644
--- a/include/drm/drm_modeset_helper_vtables.h
+++ b/include/drm/drm_modeset_helper_vtables.h
@@ -672,7 +672,7 @@
* fixed panel can also manually add specific modes using
* drm_mode_probed_add(). Drivers which manually add modes should also
* make sure that the @display_info, @width_mm and @height_mm fields of the
- * struct #drm_connector are filled in.
+ * struct &drm_connector are filled in.
*
* Virtual drivers that just want some standard VESA mode with a given
* resolution can call drm_add_modes_noedid(), and mark the preferred
diff --git a/include/drm/drm_panel.h b/include/drm/drm_panel.h
index 13ff44b..220d1e2b 100644
--- a/include/drm/drm_panel.h
+++ b/include/drm/drm_panel.h
@@ -75,6 +75,14 @@
struct display_timing *timings);
};
+/**
+ * struct drm_panel - DRM panel object
+ * @drm: DRM device owning the panel
+ * @connector: DRM connector that the panel is attached to
+ * @dev: parent device of the panel
+ * @funcs: operations that can be performed on the panel
+ * @list: panel entry in registry
+ */
struct drm_panel {
struct drm_device *drm;
struct drm_connector *connector;
@@ -85,6 +93,17 @@
struct list_head list;
};
+/**
+ * drm_disable_unprepare - power off a panel
+ * @panel: DRM panel
+ *
+ * Calling this function will completely power off a panel (assert the panel's
+ * reset, turn off power supplies, ...). After this function has completed, it
+ * is usually no longer possible to communicate with the panel until another
+ * call to drm_panel_prepare().
+ *
+ * Return: 0 on success or a negative error code on failure.
+ */
static inline int drm_panel_unprepare(struct drm_panel *panel)
{
if (panel && panel->funcs && panel->funcs->unprepare)
@@ -93,6 +112,16 @@
return panel ? -ENOSYS : -EINVAL;
}
+/**
+ * drm_panel_disable - disable a panel
+ * @panel: DRM panel
+ *
+ * This will typically turn off the panel's backlight or disable the display
+ * drivers. For smart panels it should still be possible to communicate with
+ * the integrated circuitry via any command bus after this call.
+ *
+ * Return: 0 on success or a negative error code on failure.
+ */
static inline int drm_panel_disable(struct drm_panel *panel)
{
if (panel && panel->funcs && panel->funcs->disable)
@@ -101,6 +130,16 @@
return panel ? -ENOSYS : -EINVAL;
}
+/**
+ * drm_panel_prepare - power on a panel
+ * @panel: DRM panel
+ *
+ * Calling this function will enable power and deassert any reset signals to
+ * the panel. After this has completed it is possible to communicate with any
+ * integrated circuitry via a command bus.
+ *
+ * Return: 0 on success or a negative error code on failure.
+ */
static inline int drm_panel_prepare(struct drm_panel *panel)
{
if (panel && panel->funcs && panel->funcs->prepare)
@@ -109,6 +148,16 @@
return panel ? -ENOSYS : -EINVAL;
}
+/**
+ * drm_panel_enable - enable a panel
+ * @panel: DRM panel
+ *
+ * Calling this function will cause the panel display drivers to be turned on
+ * and the backlight to be enabled. Content will be visible on screen after
+ * this call completes.
+ *
+ * Return: 0 on success or a negative error code on failure.
+ */
static inline int drm_panel_enable(struct drm_panel *panel)
{
if (panel && panel->funcs && panel->funcs->enable)
@@ -117,6 +166,16 @@
return panel ? -ENOSYS : -EINVAL;
}
+/**
+ * drm_panel_get_modes - probe the available display modes of a panel
+ * @panel: DRM panel
+ *
+ * The modes probed from the panel are automatically added to the connector
+ * that the panel is attached to.
+ *
+ * Return: The number of modes available from the panel on success or a
+ * negative error code on failure.
+ */
static inline int drm_panel_get_modes(struct drm_panel *panel)
{
if (panel && panel->funcs && panel->funcs->get_modes)
diff --git a/include/drm/drm_vma_manager.h b/include/drm/drm_vma_manager.h
index 2f63dd5..06ea8e07 100644
--- a/include/drm/drm_vma_manager.h
+++ b/include/drm/drm_vma_manager.h
@@ -176,19 +176,6 @@
}
/**
- * drm_vma_node_has_offset() - Check whether node is added to offset manager
- * @node: Node to be checked
- *
- * RETURNS:
- * true iff the node was previously allocated an offset and added to
- * an vma offset manager.
- */
-static inline bool drm_vma_node_has_offset(struct drm_vma_offset_node *node)
-{
- return drm_mm_node_allocated(&node->vm_node);
-}
-
-/**
* drm_vma_node_offset_addr() - Return sanitized offset for user-space mmaps
* @node: Linked offset node
*
@@ -220,7 +207,7 @@
static inline void drm_vma_node_unmap(struct drm_vma_offset_node *node,
struct address_space *file_mapping)
{
- if (drm_vma_node_has_offset(node))
+ if (drm_mm_node_allocated(&node->vm_node))
unmap_mapping_range(file_mapping,
drm_vma_node_offset_addr(node),
drm_vma_node_size(node) << PAGE_SHIFT, 1);
diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h
index 055a08d..c801d90 100644
--- a/include/drm/ttm/ttm_bo_api.h
+++ b/include/drm/ttm/ttm_bo_api.h
@@ -314,7 +314,7 @@
* Returns -EBUSY if no_wait is true and the buffer is busy.
* Returns -ERESTARTSYS if interrupted by a signal.
*/
-extern int ttm_bo_wait(struct ttm_buffer_object *bo, bool lazy,
+extern int ttm_bo_wait(struct ttm_buffer_object *bo,
bool interruptible, bool no_wait);
/**
* ttm_bo_validate
diff --git a/include/drm/ttm/ttm_bo_driver.h b/include/drm/ttm/ttm_bo_driver.h
index 3d4bf08..513f7f9 100644
--- a/include/drm/ttm/ttm_bo_driver.h
+++ b/include/drm/ttm/ttm_bo_driver.h
@@ -434,6 +434,18 @@
*/
int (*io_mem_reserve)(struct ttm_bo_device *bdev, struct ttm_mem_reg *mem);
void (*io_mem_free)(struct ttm_bo_device *bdev, struct ttm_mem_reg *mem);
+
+ /**
+ * Optional driver callback for when BO is removed from the LRU.
+ * Called with LRU lock held immediately before the removal.
+ */
+ void (*lru_removal)(struct ttm_buffer_object *bo);
+
+ /**
+ * Return the list_head after which a BO should be inserted in the LRU.
+ */
+ struct list_head *(*lru_tail)(struct ttm_buffer_object *bo);
+ struct list_head *(*swap_lru_tail)(struct ttm_buffer_object *bo);
};
/**
@@ -502,7 +514,6 @@
* @vma_manager: Address space manager
* lru_lock: Spinlock that protects the buffer+device lru lists and
* ddestroy lists.
- * @val_seq: Current validation sequence.
* @dev_mapping: A pointer to the struct address_space representing the
* device address space.
* @wq: Work queue structure for the delayed delete workqueue.
@@ -528,7 +539,6 @@
* Protected by the global:lru lock.
*/
struct list_head ddestroy;
- uint32_t val_seq;
/*
* Protected by load / firstopen / lastclose /unload sync.
@@ -753,14 +763,16 @@
extern void ttm_bo_del_sub_from_lru(struct ttm_buffer_object *bo);
extern void ttm_bo_add_to_lru(struct ttm_buffer_object *bo);
+struct list_head *ttm_bo_default_lru_tail(struct ttm_buffer_object *bo);
+struct list_head *ttm_bo_default_swap_lru_tail(struct ttm_buffer_object *bo);
+
/**
* __ttm_bo_reserve:
*
* @bo: A pointer to a struct ttm_buffer_object.
* @interruptible: Sleep interruptible if waiting.
* @no_wait: Don't sleep while trying to reserve, rather return -EBUSY.
- * @use_ticket: If @bo is already reserved, Only sleep waiting for
- * it to become unreserved if @ticket->stamp is older.
+ * @ticket: ticket used to acquire the ww_mutex.
*
* Will not remove reserved buffers from the lru lists.
* Otherwise identical to ttm_bo_reserve.
@@ -776,8 +788,7 @@
* be returned if @use_ticket is set to true.
*/
static inline int __ttm_bo_reserve(struct ttm_buffer_object *bo,
- bool interruptible,
- bool no_wait, bool use_ticket,
+ bool interruptible, bool no_wait,
struct ww_acquire_ctx *ticket)
{
int ret = 0;
@@ -806,8 +817,7 @@
* @bo: A pointer to a struct ttm_buffer_object.
* @interruptible: Sleep interruptible if waiting.
* @no_wait: Don't sleep while trying to reserve, rather return -EBUSY.
- * @use_ticket: If @bo is already reserved, Only sleep waiting for
- * it to become unreserved if @ticket->stamp is older.
+ * @ticket: ticket used to acquire the ww_mutex.
*
* Locks a buffer object for validation. (Or prevents other processes from
* locking it for validation) and removes it from lru lists, while taking
@@ -846,15 +856,14 @@
* be returned if @use_ticket is set to true.
*/
static inline int ttm_bo_reserve(struct ttm_buffer_object *bo,
- bool interruptible,
- bool no_wait, bool use_ticket,
+ bool interruptible, bool no_wait,
struct ww_acquire_ctx *ticket)
{
int ret;
WARN_ON(!atomic_read(&bo->kref.refcount));
- ret = __ttm_bo_reserve(bo, interruptible, no_wait, use_ticket, ticket);
+ ret = __ttm_bo_reserve(bo, interruptible, no_wait, ticket);
if (likely(ret == 0))
ttm_bo_del_sub_from_lru(bo);
@@ -1030,8 +1039,7 @@
extern const struct ttm_mem_type_manager_func ttm_bo_manager_func;
-#if (defined(CONFIG_AGP) || (defined(CONFIG_AGP_MODULE) && defined(MODULE)))
-#define TTM_HAS_AGP
+#if IS_ENABLED(CONFIG_AGP)
#include <linux/agp_backend.h>
/**
diff --git a/include/dt-bindings/clock/ath79-clk.h b/include/dt-bindings/clock/ath79-clk.h
new file mode 100644
index 0000000..27359ad
--- /dev/null
+++ b/include/dt-bindings/clock/ath79-clk.h
@@ -0,0 +1,19 @@
+/*
+ * Copyright (C) 2014, 2016 Antony Pavlov <antonynpavlov@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#ifndef __DT_BINDINGS_ATH79_CLK_H
+#define __DT_BINDINGS_ATH79_CLK_H
+
+#define ATH79_CLK_CPU 0
+#define ATH79_CLK_DDR 1
+#define ATH79_CLK_AHB 2
+
+#define ATH79_CLK_END 3
+
+#endif /* __DT_BINDINGS_ATH79_CLK_H */
diff --git a/include/dt-bindings/clock/axis,artpec6-clkctrl.h b/include/dt-bindings/clock/axis,artpec6-clkctrl.h
new file mode 100644
index 0000000..f9f04dc
--- /dev/null
+++ b/include/dt-bindings/clock/axis,artpec6-clkctrl.h
@@ -0,0 +1,38 @@
+/*
+ * ARTPEC-6 clock controller indexes
+ *
+ * Copyright 2016 Axis Comunications AB.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef DT_BINDINGS_CLK_ARTPEC6_CLKCTRL_H
+#define DT_BINDINGS_CLK_ARTPEC6_CLKCTRL_H
+
+#define ARTPEC6_CLK_CPU 0
+#define ARTPEC6_CLK_CPU_PERIPH 1
+#define ARTPEC6_CLK_NAND_CLKA 2
+#define ARTPEC6_CLK_NAND_CLKB 3
+#define ARTPEC6_CLK_ETH_ACLK 4
+#define ARTPEC6_CLK_DMA_ACLK 5
+#define ARTPEC6_CLK_PTP_REF 6
+#define ARTPEC6_CLK_SD_PCLK 7
+#define ARTPEC6_CLK_SD_IMCLK 8
+#define ARTPEC6_CLK_I2S_HST 9
+#define ARTPEC6_CLK_I2S0_CLK 10
+#define ARTPEC6_CLK_I2S1_CLK 11
+#define ARTPEC6_CLK_UART_PCLK 12
+#define ARTPEC6_CLK_UART_REFCLK 13
+#define ARTPEC6_CLK_I2C 14
+#define ARTPEC6_CLK_SPI_PCLK 15
+#define ARTPEC6_CLK_SPI_SSPCLK 16
+#define ARTPEC6_CLK_SYS_TIMER 17
+#define ARTPEC6_CLK_FRACDIV_IN 18
+#define ARTPEC6_CLK_DBG_PCLK 19
+
+/* This must be the highest clock index plus one. */
+#define ARTPEC6_CLK_NUMCLOCKS 20
+
+#endif
diff --git a/include/dt-bindings/clock/bcm2835.h b/include/dt-bindings/clock/bcm2835.h
index 61f1d20..360e00c 100644
--- a/include/dt-bindings/clock/bcm2835.h
+++ b/include/dt-bindings/clock/bcm2835.h
@@ -44,5 +44,23 @@
#define BCM2835_CLOCK_EMMC 28
#define BCM2835_CLOCK_PERI_IMAGE 29
#define BCM2835_CLOCK_PWM 30
+#define BCM2835_CLOCK_PCM 31
-#define BCM2835_CLOCK_COUNT 31
+#define BCM2835_PLLA_DSI0 32
+#define BCM2835_PLLA_CCP2 33
+#define BCM2835_PLLD_DSI0 34
+#define BCM2835_PLLD_DSI1 35
+
+#define BCM2835_CLOCK_AVEO 36
+#define BCM2835_CLOCK_DFT 37
+#define BCM2835_CLOCK_GP0 38
+#define BCM2835_CLOCK_GP1 39
+#define BCM2835_CLOCK_GP2 40
+#define BCM2835_CLOCK_SLIM 41
+#define BCM2835_CLOCK_SMI 42
+#define BCM2835_CLOCK_TEC 43
+#define BCM2835_CLOCK_DPI 44
+#define BCM2835_CLOCK_CAM0 45
+#define BCM2835_CLOCK_CAM1 46
+#define BCM2835_CLOCK_DSI0E 47
+#define BCM2835_CLOCK_DSI1E 48
diff --git a/include/dt-bindings/clock/exynos5420.h b/include/dt-bindings/clock/exynos5420.h
index 7699ee9..17ab839 100644
--- a/include/dt-bindings/clock/exynos5420.h
+++ b/include/dt-bindings/clock/exynos5420.h
@@ -217,8 +217,30 @@
/* divider clocks */
#define CLK_DOUT_PIXEL 768
+#define CLK_DOUT_ACLK400_WCORE 769
+#define CLK_DOUT_ACLK400_ISP 770
+#define CLK_DOUT_ACLK400_MSCL 771
+#define CLK_DOUT_ACLK200 772
+#define CLK_DOUT_ACLK200_FSYS2 773
+#define CLK_DOUT_ACLK100_NOC 774
+#define CLK_DOUT_PCLK200_FSYS 775
+#define CLK_DOUT_ACLK200_FSYS 776
+#define CLK_DOUT_ACLK333_432_GSCL 777
+#define CLK_DOUT_ACLK333_432_ISP 778
+#define CLK_DOUT_ACLK66 779
+#define CLK_DOUT_ACLK333_432_ISP0 780
+#define CLK_DOUT_ACLK266 781
+#define CLK_DOUT_ACLK166 782
+#define CLK_DOUT_ACLK333 783
+#define CLK_DOUT_ACLK333_G2D 784
+#define CLK_DOUT_ACLK266_G2D 785
+#define CLK_DOUT_ACLK_G3D 786
+#define CLK_DOUT_ACLK300_JPEG 787
+#define CLK_DOUT_ACLK300_DISP1 788
+#define CLK_DOUT_ACLK300_GSCL 789
+#define CLK_DOUT_ACLK400_DISP1 790
/* must be greater than maximal clock id */
-#define CLK_NR_CLKS 769
+#define CLK_NR_CLKS 791
#endif /* _DT_BINDINGS_CLOCK_EXYNOS_5420_H */
diff --git a/include/dt-bindings/clock/hi3519-clock.h b/include/dt-bindings/clock/hi3519-clock.h
new file mode 100644
index 0000000..14f4d21
--- /dev/null
+++ b/include/dt-bindings/clock/hi3519-clock.h
@@ -0,0 +1,40 @@
+/*
+ * Copyright (c) 2015 HiSilicon Technologies Co., Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __DTS_HI3519_CLOCK_H
+#define __DTS_HI3519_CLOCK_H
+
+#define HI3519_FMC_CLK 1
+#define HI3519_SPI0_CLK 2
+#define HI3519_SPI1_CLK 3
+#define HI3519_SPI2_CLK 4
+#define HI3519_UART0_CLK 5
+#define HI3519_UART1_CLK 6
+#define HI3519_UART2_CLK 7
+#define HI3519_UART3_CLK 8
+#define HI3519_UART4_CLK 9
+#define HI3519_PWM_CLK 10
+#define HI3519_DMA_CLK 11
+#define HI3519_IR_CLK 12
+#define HI3519_ETH_PHY_CLK 13
+#define HI3519_ETH_MAC_CLK 14
+#define HI3519_ETH_MACIF_CLK 15
+#define HI3519_USB2_BUS_CLK 16
+#define HI3519_USB2_PORT_CLK 17
+#define HI3519_USB3_CLK 18
+
+#endif /* __DTS_HI3519_CLOCK_H */
diff --git a/include/dt-bindings/clock/imx7d-clock.h b/include/dt-bindings/clock/imx7d-clock.h
index edca8985c..1183347 100644
--- a/include/dt-bindings/clock/imx7d-clock.h
+++ b/include/dt-bindings/clock/imx7d-clock.h
@@ -448,5 +448,6 @@
#define IMX7D_PLL_DRAM_TEST_DIV 435
#define IMX7D_ADC_ROOT_CLK 436
#define IMX7D_CLK_ARM 437
-#define IMX7D_CLK_END 438
+#define IMX7D_CKIL 438
+#define IMX7D_CLK_END 439
#endif /* __DT_BINDINGS_CLOCK_IMX7D_H */
diff --git a/include/dt-bindings/clock/microchip,pic32-clock.h b/include/dt-bindings/clock/microchip,pic32-clock.h
new file mode 100644
index 0000000..184647a6
--- /dev/null
+++ b/include/dt-bindings/clock/microchip,pic32-clock.h
@@ -0,0 +1,42 @@
+/*
+ * Purna Chandra Mandal,<purna.mandal@microchip.com>
+ * Copyright (C) 2015 Microchip Technology Inc. All rights reserved.
+ *
+ * This program is free software; you can distribute it and/or modify it
+ * under the terms of the GNU General Public License (Version 2) as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+ * for more details.
+ */
+
+#ifndef _DT_BINDINGS_CLK_MICROCHIP_PIC32_H_
+#define _DT_BINDINGS_CLK_MICROCHIP_PIC32_H_
+
+/* clock output indices */
+#define POSCCLK 0
+#define FRCCLK 1
+#define BFRCCLK 2
+#define LPRCCLK 3
+#define SOSCCLK 4
+#define FRCDIVCLK 5
+#define PLLCLK 6
+#define SCLK 7
+#define PB1CLK 8
+#define PB2CLK 9
+#define PB3CLK 10
+#define PB4CLK 11
+#define PB5CLK 12
+#define PB6CLK 13
+#define PB7CLK 14
+#define REF1CLK 15
+#define REF2CLK 16
+#define REF3CLK 17
+#define REF4CLK 18
+#define REF5CLK 19
+#define UPLLCLK 20
+#define MAXCLKS 21
+
+#endif /* _DT_BINDINGS_CLK_MICROCHIP_PIC32_H_ */
diff --git a/include/dt-bindings/clock/mt8173-clk.h b/include/dt-bindings/clock/mt8173-clk.h
index 7956ba1..6094bf7 100644
--- a/include/dt-bindings/clock/mt8173-clk.h
+++ b/include/dt-bindings/clock/mt8173-clk.h
@@ -176,7 +176,8 @@
#define CLK_APMIXED_LVDSPLL 13
#define CLK_APMIXED_MSDCPLL2 14
#define CLK_APMIXED_REF2USB_TX 15
-#define CLK_APMIXED_NR_CLK 16
+#define CLK_APMIXED_HDMI_REF 16
+#define CLK_APMIXED_NR_CLK 17
/* INFRA_SYS */
diff --git a/include/dt-bindings/clock/tegra210-car.h b/include/dt-bindings/clock/tegra210-car.h
index 0a05b0d..bd3530e 100644
--- a/include/dt-bindings/clock/tegra210-car.h
+++ b/include/dt-bindings/clock/tegra210-car.h
@@ -346,7 +346,7 @@
#define TEGRA210_CLK_PLL_P_OUT_HSIO 316
#define TEGRA210_CLK_PLL_P_OUT_XUSB 317
#define TEGRA210_CLK_XUSB_SSP_SRC 318
-/* 319 */
+#define TEGRA210_CLK_PLL_RE_OUT1 319
/* 320 */
/* 321 */
/* 322 */
diff --git a/include/dt-bindings/clock/vf610-clock.h b/include/dt-bindings/clock/vf610-clock.h
index 56c16aa..45997750 100644
--- a/include/dt-bindings/clock/vf610-clock.h
+++ b/include/dt-bindings/clock/vf610-clock.h
@@ -194,7 +194,11 @@
#define VF610_PLL7_BYPASS 181
#define VF610_CLK_SNVS 182
#define VF610_CLK_DAP 183
-#define VF610_CLK_OCOTP 184
-#define VF610_CLK_END 185
+#define VF610_CLK_OCOTP 184
+#define VF610_CLK_DDRMC 185
+#define VF610_CLK_WKPU 186
+#define VF610_CLK_TCON0 187
+#define VF610_CLK_TCON1 188
+#define VF610_CLK_END 189
#endif /* __DT_BINDINGS_CLOCK_VF610_H */
diff --git a/include/dt-bindings/gpio/meson-gxbb-gpio.h b/include/dt-bindings/gpio/meson-gxbb-gpio.h
new file mode 100644
index 0000000..58654fd
--- /dev/null
+++ b/include/dt-bindings/gpio/meson-gxbb-gpio.h
@@ -0,0 +1,154 @@
+/*
+ * GPIO definitions for Amlogic Meson GXBB SoCs
+ *
+ * Copyright (C) 2016 Endless Mobile, Inc.
+ * Author: Carlo Caione <carlo@endlessm.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * version 2 as published by the Free Software Foundation.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef _DT_BINDINGS_MESON_GXBB_GPIO_H
+#define _DT_BINDINGS_MESON_GXBB_GPIO_H
+
+#define GPIOAO_0 0
+#define GPIOAO_1 1
+#define GPIOAO_2 2
+#define GPIOAO_3 3
+#define GPIOAO_4 4
+#define GPIOAO_5 5
+#define GPIOAO_6 6
+#define GPIOAO_7 7
+#define GPIOAO_8 8
+#define GPIOAO_9 9
+#define GPIOAO_10 10
+#define GPIOAO_11 11
+#define GPIOAO_12 12
+#define GPIOAO_13 13
+
+#define GPIOZ_0 0
+#define GPIOZ_1 1
+#define GPIOZ_2 2
+#define GPIOZ_3 3
+#define GPIOZ_4 4
+#define GPIOZ_5 5
+#define GPIOZ_6 6
+#define GPIOZ_7 7
+#define GPIOZ_8 8
+#define GPIOZ_9 9
+#define GPIOZ_10 10
+#define GPIOZ_11 11
+#define GPIOZ_12 12
+#define GPIOZ_13 13
+#define GPIOZ_14 14
+#define GPIOZ_15 15
+#define GPIOH_0 16
+#define GPIOH_1 17
+#define GPIOH_2 18
+#define GPIOH_3 19
+#define BOOT_0 20
+#define BOOT_1 21
+#define BOOT_2 22
+#define BOOT_3 23
+#define BOOT_4 24
+#define BOOT_5 25
+#define BOOT_6 26
+#define BOOT_7 27
+#define BOOT_8 28
+#define BOOT_9 29
+#define BOOT_10 30
+#define BOOT_11 31
+#define BOOT_12 32
+#define BOOT_13 33
+#define BOOT_14 34
+#define BOOT_15 35
+#define BOOT_16 36
+#define BOOT_17 37
+#define CARD_0 38
+#define CARD_1 39
+#define CARD_2 40
+#define CARD_3 41
+#define CARD_4 42
+#define CARD_5 43
+#define CARD_6 44
+#define GPIODV_0 45
+#define GPIODV_1 46
+#define GPIODV_2 47
+#define GPIODV_3 48
+#define GPIODV_4 49
+#define GPIODV_5 50
+#define GPIODV_6 51
+#define GPIODV_7 52
+#define GPIODV_8 53
+#define GPIODV_9 54
+#define GPIODV_10 55
+#define GPIODV_11 56
+#define GPIODV_12 57
+#define GPIODV_13 58
+#define GPIODV_14 59
+#define GPIODV_15 60
+#define GPIODV_16 61
+#define GPIODV_17 62
+#define GPIODV_18 63
+#define GPIODV_19 64
+#define GPIODV_20 65
+#define GPIODV_21 66
+#define GPIODV_22 67
+#define GPIODV_23 68
+#define GPIODV_24 69
+#define GPIODV_25 70
+#define GPIODV_26 71
+#define GPIODV_27 72
+#define GPIODV_28 73
+#define GPIODV_29 74
+#define GPIOY_0 75
+#define GPIOY_1 76
+#define GPIOY_2 77
+#define GPIOY_3 78
+#define GPIOY_4 79
+#define GPIOY_5 80
+#define GPIOY_6 81
+#define GPIOY_7 82
+#define GPIOY_8 83
+#define GPIOY_9 84
+#define GPIOY_10 85
+#define GPIOY_11 86
+#define GPIOY_12 87
+#define GPIOY_13 88
+#define GPIOY_14 89
+#define GPIOY_15 90
+#define GPIOY_16 91
+#define GPIOX_0 92
+#define GPIOX_1 93
+#define GPIOX_2 94
+#define GPIOX_3 95
+#define GPIOX_4 96
+#define GPIOX_5 97
+#define GPIOX_6 98
+#define GPIOX_7 99
+#define GPIOX_8 100
+#define GPIOX_9 101
+#define GPIOX_10 102
+#define GPIOX_11 103
+#define GPIOX_12 104
+#define GPIOX_13 105
+#define GPIOX_14 106
+#define GPIOX_15 107
+#define GPIOX_16 108
+#define GPIOX_17 109
+#define GPIOX_18 110
+#define GPIOX_19 111
+#define GPIOX_20 112
+#define GPIOX_21 113
+#define GPIOX_22 114
+#define GPIOCLK_0 115
+#define GPIOCLK_1 116
+#define GPIOCLK_2 117
+#define GPIOCLK_3 118
+#define GPIO_TEST_N 119
+
+#endif
diff --git a/include/dt-bindings/iio/adi,ad5592r.h b/include/dt-bindings/iio/adi,ad5592r.h
new file mode 100644
index 0000000..c48aca1
--- /dev/null
+++ b/include/dt-bindings/iio/adi,ad5592r.h
@@ -0,0 +1,16 @@
+
+#ifndef _DT_BINDINGS_ADI_AD5592R_H
+#define _DT_BINDINGS_ADI_AD5592R_H
+
+#define CH_MODE_UNUSED 0
+#define CH_MODE_ADC 1
+#define CH_MODE_DAC 2
+#define CH_MODE_DAC_AND_ADC 3
+#define CH_MODE_GPIO 8
+
+#define CH_OFFSTATE_PULLDOWN 0
+#define CH_OFFSTATE_OUT_LOW 1
+#define CH_OFFSTATE_OUT_HIGH 2
+#define CH_OFFSTATE_OUT_TRISTATE 3
+
+#endif /* _DT_BINDINGS_ADI_AD5592R_H */
diff --git a/include/dt-bindings/mfd/arizona.h b/include/dt-bindings/mfd/arizona.h
index c40f665..dedf46f 100644
--- a/include/dt-bindings/mfd/arizona.h
+++ b/include/dt-bindings/mfd/arizona.h
@@ -110,4 +110,9 @@
#define ARIZONA_ACCDET_MODE_HPM 4
#define ARIZONA_ACCDET_MODE_ADC 7
+#define ARIZONA_GPSW_OPEN 0
+#define ARIZONA_GPSW_CLOSED 1
+#define ARIZONA_GPSW_CLAMP_ENABLED 2
+#define ARIZONA_GPSW_CLAMP_DISABLED 3
+
#endif
diff --git a/include/dt-bindings/mfd/max77620.h b/include/dt-bindings/mfd/max77620.h
new file mode 100644
index 0000000..b911a07
--- /dev/null
+++ b/include/dt-bindings/mfd/max77620.h
@@ -0,0 +1,39 @@
+/*
+ * This header provides macros for MAXIM MAX77620 device bindings.
+ *
+ * Copyright (c) 2016, NVIDIA Corporation.
+ * Author: Laxman Dewangan <ldewangan@nvidia.com>
+ */
+
+#ifndef _DT_BINDINGS_MFD_MAX77620_H
+#define _DT_BINDINGS_MFD_MAX77620_H
+
+/* MAX77620 interrupts */
+#define MAX77620_IRQ_TOP_GLBL 0 /* Low-Battery */
+#define MAX77620_IRQ_TOP_SD 1 /* SD power fail */
+#define MAX77620_IRQ_TOP_LDO 2 /* LDO power fail */
+#define MAX77620_IRQ_TOP_GPIO 3 /* GPIO internal int to MAX77620 */
+#define MAX77620_IRQ_TOP_RTC 4 /* RTC */
+#define MAX77620_IRQ_TOP_32K 5 /* 32kHz oscillator */
+#define MAX77620_IRQ_TOP_ONOFF 6 /* ON/OFF oscillator */
+#define MAX77620_IRQ_LBT_MBATLOW 7 /* Thermal alarm status, > 120C */
+#define MAX77620_IRQ_LBT_TJALRM1 8 /* Thermal alarm status, > 120C */
+#define MAX77620_IRQ_LBT_TJALRM2 9 /* Thermal alarm status, > 140C */
+
+/* FPS event source */
+#define MAX77620_FPS_EVENT_SRC_EN0 0
+#define MAX77620_FPS_EVENT_SRC_EN1 1
+#define MAX77620_FPS_EVENT_SRC_SW 2
+
+/* Device state when FPS event LOW */
+#define MAX77620_FPS_INACTIVE_STATE_SLEEP 0
+#define MAX77620_FPS_INACTIVE_STATE_LOW_POWER 1
+
+/* FPS source */
+#define MAX77620_FPS_SRC_0 0
+#define MAX77620_FPS_SRC_1 1
+#define MAX77620_FPS_SRC_2 2
+#define MAX77620_FPS_SRC_NONE 3
+#define MAX77620_FPS_SRC_DEF 4
+
+#endif
diff --git a/include/keys/asymmetric-subtype.h b/include/keys/asymmetric-subtype.h
index 4915d40..2480469 100644
--- a/include/keys/asymmetric-subtype.h
+++ b/include/keys/asymmetric-subtype.h
@@ -32,7 +32,7 @@
void (*describe)(const struct key *key, struct seq_file *m);
/* Destroy a key of this subtype */
- void (*destroy)(void *payload);
+ void (*destroy)(void *payload_crypto, void *payload_auth);
/* Verify the signature on a key of this subtype (optional) */
int (*verify_signature)(const struct key *key,
diff --git a/include/keys/asymmetric-type.h b/include/keys/asymmetric-type.h
index 59c1df9..b382407 100644
--- a/include/keys/asymmetric-type.h
+++ b/include/keys/asymmetric-type.h
@@ -15,6 +15,7 @@
#define _KEYS_ASYMMETRIC_TYPE_H
#include <linux/key-type.h>
+#include <linux/verification.h>
extern struct key_type key_type_asymmetric;
@@ -23,9 +24,10 @@
* follows:
*/
enum asymmetric_payload_bits {
- asym_crypto,
- asym_subtype,
- asym_key_ids,
+ asym_crypto, /* The data representing the key */
+ asym_subtype, /* Pointer to an asymmetric_key_subtype struct */
+ asym_key_ids, /* Pointer to an asymmetric_key_ids struct */
+ asym_auth /* The key's authorisation (signature, parent key ID) */
};
/*
@@ -74,6 +76,11 @@
return key->payload.data[asym_key_ids];
}
+extern struct key *find_asymmetric_key(struct key *keyring,
+ const struct asymmetric_key_id *id_0,
+ const struct asymmetric_key_id *id_1,
+ bool partial);
+
/*
* The payload is at the discretion of the subtype.
*/
diff --git a/include/keys/system_keyring.h b/include/keys/system_keyring.h
index 39fd38c..fbd4647 100644
--- a/include/keys/system_keyring.h
+++ b/include/keys/system_keyring.h
@@ -12,51 +12,40 @@
#ifndef _KEYS_SYSTEM_KEYRING_H
#define _KEYS_SYSTEM_KEYRING_H
+#include <linux/key.h>
+
#ifdef CONFIG_SYSTEM_TRUSTED_KEYRING
-#include <linux/key.h>
-#include <crypto/public_key.h>
+extern int restrict_link_by_builtin_trusted(struct key *keyring,
+ const struct key_type *type,
+ const union key_payload *payload);
-extern struct key *system_trusted_keyring;
-static inline struct key *get_system_trusted_keyring(void)
-{
- return system_trusted_keyring;
-}
#else
-static inline struct key *get_system_trusted_keyring(void)
-{
- return NULL;
-}
+#define restrict_link_by_builtin_trusted restrict_link_reject
#endif
-#ifdef CONFIG_SYSTEM_DATA_VERIFICATION
-extern int system_verify_data(const void *data, unsigned long len,
- const void *raw_pkcs7, size_t pkcs7_len,
- enum key_being_used_for usage);
+#ifdef CONFIG_SECONDARY_TRUSTED_KEYRING
+extern int restrict_link_by_builtin_and_secondary_trusted(
+ struct key *keyring,
+ const struct key_type *type,
+ const union key_payload *payload);
+#else
+#define restrict_link_by_builtin_and_secondary_trusted restrict_link_by_builtin_trusted
#endif
-#ifdef CONFIG_IMA_MOK_KEYRING
-extern struct key *ima_mok_keyring;
+#ifdef CONFIG_IMA_BLACKLIST_KEYRING
extern struct key *ima_blacklist_keyring;
-static inline struct key *get_ima_mok_keyring(void)
-{
- return ima_mok_keyring;
-}
static inline struct key *get_ima_blacklist_keyring(void)
{
return ima_blacklist_keyring;
}
#else
-static inline struct key *get_ima_mok_keyring(void)
-{
- return NULL;
-}
static inline struct key *get_ima_blacklist_keyring(void)
{
return NULL;
}
-#endif /* CONFIG_IMA_MOK_KEYRING */
+#endif /* CONFIG_IMA_BLACKLIST_KEYRING */
#endif /* _KEYS_SYSTEM_KEYRING_H */
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 281caf8..be6037a 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -25,6 +25,7 @@
#include <linux/spinlock.h>
#include <linux/types.h>
#include <kvm/iodev.h>
+#include <linux/irqchip/arm-gic-common.h>
#define VGIC_NR_IRQS_LEGACY 256
#define VGIC_NR_SGIS 16
@@ -353,15 +354,15 @@
#define vgic_initialized(k) (!!((k)->arch.vgic.nr_cpus))
#define vgic_ready(k) ((k)->arch.vgic.ready)
-int vgic_v2_probe(struct device_node *vgic_node,
+int vgic_v2_probe(const struct gic_kvm_info *gic_kvm_info,
const struct vgic_ops **ops,
const struct vgic_params **params);
#ifdef CONFIG_KVM_ARM_VGIC_V3
-int vgic_v3_probe(struct device_node *vgic_node,
+int vgic_v3_probe(const struct gic_kvm_info *gic_kvm_info,
const struct vgic_ops **ops,
const struct vgic_params **params);
#else
-static inline int vgic_v3_probe(struct device_node *vgic_node,
+static inline int vgic_v3_probe(const struct gic_kvm_info *gic_kvm_info,
const struct vgic_ops **ops,
const struct vgic_params **params)
{
diff --git a/include/linux/amba/pl08x.h b/include/linux/amba/pl08x.h
index 10fe2a2..27e9ec8 100644
--- a/include/linux/amba/pl08x.h
+++ b/include/linux/amba/pl08x.h
@@ -86,7 +86,7 @@
* @mem_buses: buses which memory can be accessed from: PL08X_AHB1 | PL08X_AHB2
*/
struct pl08x_platform_data {
- const struct pl08x_channel_data *slave_channels;
+ struct pl08x_channel_data *slave_channels;
unsigned int num_slave_channels;
struct pl08x_channel_data memcpy_channel;
int (*get_xfer_signal)(const struct pl08x_channel_data *);
diff --git a/include/linux/ata.h b/include/linux/ata.h
index f310ec0..99346be 100644
--- a/include/linux/ata.h
+++ b/include/linux/ata.h
@@ -243,6 +243,7 @@
ATA_CMD_WRITE_QUEUED_FUA_EXT = 0x3E,
ATA_CMD_FPDMA_READ = 0x60,
ATA_CMD_FPDMA_WRITE = 0x61,
+ ATA_CMD_NCQ_NON_DATA = 0x63,
ATA_CMD_FPDMA_SEND = 0x64,
ATA_CMD_FPDMA_RECV = 0x65,
ATA_CMD_PIO_READ = 0x20,
@@ -301,19 +302,43 @@
ATA_CMD_CFA_WRITE_MULT_NE = 0xCD,
ATA_CMD_REQ_SENSE_DATA = 0x0B,
ATA_CMD_SANITIZE_DEVICE = 0xB4,
+ ATA_CMD_ZAC_MGMT_IN = 0x4A,
+ ATA_CMD_ZAC_MGMT_OUT = 0x9F,
/* marked obsolete in the ATA/ATAPI-7 spec */
ATA_CMD_RESTORE = 0x10,
+ /* Subcmds for ATA_CMD_FPDMA_RECV */
+ ATA_SUBCMD_FPDMA_RECV_RD_LOG_DMA_EXT = 0x01,
+ ATA_SUBCMD_FPDMA_RECV_ZAC_MGMT_IN = 0x02,
+
/* Subcmds for ATA_CMD_FPDMA_SEND */
ATA_SUBCMD_FPDMA_SEND_DSM = 0x00,
ATA_SUBCMD_FPDMA_SEND_WR_LOG_DMA_EXT = 0x02,
+ /* Subcmds for ATA_CMD_NCQ_NON_DATA */
+ ATA_SUBCMD_NCQ_NON_DATA_ABORT_QUEUE = 0x00,
+ ATA_SUBCMD_NCQ_NON_DATA_SET_FEATURES = 0x05,
+ ATA_SUBCMD_NCQ_NON_DATA_ZERO_EXT = 0x06,
+ ATA_SUBCMD_NCQ_NON_DATA_ZAC_MGMT_OUT = 0x07,
+
+ /* Subcmds for ATA_CMD_ZAC_MGMT_IN */
+ ATA_SUBCMD_ZAC_MGMT_IN_REPORT_ZONES = 0x00,
+
+ /* Subcmds for ATA_CMD_ZAC_MGMT_OUT */
+ ATA_SUBCMD_ZAC_MGMT_OUT_CLOSE_ZONE = 0x01,
+ ATA_SUBCMD_ZAC_MGMT_OUT_FINISH_ZONE = 0x02,
+ ATA_SUBCMD_ZAC_MGMT_OUT_OPEN_ZONE = 0x03,
+ ATA_SUBCMD_ZAC_MGMT_OUT_RESET_WRITE_POINTER = 0x04,
+
/* READ_LOG_EXT pages */
+ ATA_LOG_DIRECTORY = 0x0,
ATA_LOG_SATA_NCQ = 0x10,
+ ATA_LOG_NCQ_NON_DATA = 0x12,
ATA_LOG_NCQ_SEND_RECV = 0x13,
ATA_LOG_SATA_ID_DEV_DATA = 0x30,
ATA_LOG_SATA_SETTINGS = 0x08,
+ ATA_LOG_ZONED_INFORMATION = 0x09,
ATA_LOG_DEVSLP_OFFSET = 0x30,
ATA_LOG_DEVSLP_SIZE = 0x08,
ATA_LOG_DEVSLP_MDAT = 0x00,
@@ -328,8 +353,25 @@
ATA_LOG_NCQ_SEND_RECV_DSM_OFFSET = 0x04,
ATA_LOG_NCQ_SEND_RECV_DSM_TRIM = (1 << 0),
ATA_LOG_NCQ_SEND_RECV_RD_LOG_OFFSET = 0x08,
+ ATA_LOG_NCQ_SEND_RECV_RD_LOG_SUPPORTED = (1 << 0),
ATA_LOG_NCQ_SEND_RECV_WR_LOG_OFFSET = 0x0C,
- ATA_LOG_NCQ_SEND_RECV_SIZE = 0x10,
+ ATA_LOG_NCQ_SEND_RECV_WR_LOG_SUPPORTED = (1 << 0),
+ ATA_LOG_NCQ_SEND_RECV_ZAC_MGMT_OFFSET = 0x10,
+ ATA_LOG_NCQ_SEND_RECV_ZAC_MGMT_OUT_SUPPORTED = (1 << 0),
+ ATA_LOG_NCQ_SEND_RECV_ZAC_MGMT_IN_SUPPORTED = (1 << 1),
+ ATA_LOG_NCQ_SEND_RECV_SIZE = 0x14,
+
+ /* NCQ Non-Data log */
+ ATA_LOG_NCQ_NON_DATA_SUBCMDS_OFFSET = 0x00,
+ ATA_LOG_NCQ_NON_DATA_ABORT_OFFSET = 0x00,
+ ATA_LOG_NCQ_NON_DATA_ABORT_NCQ = (1 << 0),
+ ATA_LOG_NCQ_NON_DATA_ABORT_ALL = (1 << 1),
+ ATA_LOG_NCQ_NON_DATA_ABORT_STREAMING = (1 << 2),
+ ATA_LOG_NCQ_NON_DATA_ABORT_NON_STREAMING = (1 << 3),
+ ATA_LOG_NCQ_NON_DATA_ABORT_SELECTED = (1 << 4),
+ ATA_LOG_NCQ_NON_DATA_ZAC_MGMT_OFFSET = 0x1C,
+ ATA_LOG_NCQ_NON_DATA_ZAC_MGMT_OUT = (1 << 0),
+ ATA_LOG_NCQ_NON_DATA_SIZE = 0x40,
/* READ/WRITE LONG (obsolete) */
ATA_CMD_READ_LONG = 0x22,
@@ -386,6 +428,8 @@
SATA_SSP = 0x06, /* Software Settings Preservation */
SATA_DEVSLP = 0x09, /* Device Sleep */
+ SETFEATURE_SENSE_DATA = 0xC3, /* Sense Data Reporting feature */
+
/* feature values for SET_MAX */
ATA_SET_MAX_ADDR = 0x00,
ATA_SET_MAX_PASSWD = 0x01,
@@ -529,6 +573,8 @@
#define ata_id_cdb_intr(id) (((id)[ATA_ID_CONFIG] & 0x60) == 0x20)
#define ata_id_has_da(id) ((id)[ATA_ID_SATA_CAPABILITY_2] & (1 << 4))
#define ata_id_has_devslp(id) ((id)[ATA_ID_FEATURE_SUPP] & (1 << 8))
+#define ata_id_has_ncq_autosense(id) \
+ ((id)[ATA_ID_FEATURE_SUPP] & (1 << 7))
static inline bool ata_id_has_hipm(const u16 *id)
{
@@ -717,6 +763,20 @@
return false;
}
+static inline bool ata_id_has_sense_reporting(const u16 *id)
+{
+ if (!(id[ATA_ID_CFS_ENABLE_2] & (1 << 15)))
+ return false;
+ return id[ATA_ID_COMMAND_SET_3] & (1 << 6);
+}
+
+static inline bool ata_id_sense_reporting_enabled(const u16 *id)
+{
+ if (!(id[ATA_ID_CFS_ENABLE_2] & (1 << 15)))
+ return false;
+ return id[ATA_ID_COMMAND_SET_4] & (1 << 6);
+}
+
/**
* ata_id_major_version - get ATA level of drive
* @id: Identify data
@@ -821,6 +881,11 @@
return id[ATA_ID_SATA_CAPABILITY_2] & BIT(6);
}
+static inline bool ata_id_has_ncq_non_data(const u16 *id)
+{
+ return id[ATA_ID_SATA_CAPABILITY_2] & BIT(5);
+}
+
static inline bool ata_id_has_trim(const u16 *id)
{
if (ata_id_major_version(id) >= 7 &&
@@ -872,6 +937,11 @@
return id[ATA_ID_ROT_SPEED] == 0x01;
}
+static inline u8 ata_id_zoned_cap(const u16 *id)
+{
+ return (id[ATA_ID_ADDITIONAL_SUPP] & 0x3);
+}
+
static inline bool ata_id_pio_need_iordy(const u16 *id, const u8 pio)
{
/* CF spec. r4.1 Table 22 says no IORDY on PIO5 and PIO6. */
diff --git a/include/linux/audit.h b/include/linux/audit.h
index e38e3fc..961a417 100644
--- a/include/linux/audit.h
+++ b/include/linux/audit.h
@@ -26,6 +26,7 @@
#include <linux/sched.h>
#include <linux/ptrace.h>
#include <uapi/linux/audit.h>
+#include <linux/tty.h>
#define AUDIT_INO_UNSET ((unsigned long)-1)
#define AUDIT_DEV_UNSET ((dev_t)-1)
@@ -347,6 +348,23 @@
return tsk->sessionid;
}
+static inline struct tty_struct *audit_get_tty(struct task_struct *tsk)
+{
+ struct tty_struct *tty = NULL;
+ unsigned long flags;
+
+ spin_lock_irqsave(&tsk->sighand->siglock, flags);
+ if (tsk->signal)
+ tty = tty_kref_get(tsk->signal->tty);
+ spin_unlock_irqrestore(&tsk->sighand->siglock, flags);
+ return tty;
+}
+
+static inline void audit_put_tty(struct tty_struct *tty)
+{
+ tty_kref_put(tty);
+}
+
extern void __audit_ipc_obj(struct kern_ipc_perm *ipcp);
extern void __audit_ipc_set_perm(unsigned long qbytes, uid_t uid, gid_t gid, umode_t mode);
extern void __audit_bprm(struct linux_binprm *bprm);
@@ -504,6 +522,12 @@
{
return -1;
}
+static inline struct tty_struct *audit_get_tty(struct task_struct *tsk)
+{
+ return NULL;
+}
+static inline void audit_put_tty(struct tty_struct *tty)
+{ }
static inline void audit_ipc_obj(struct kern_ipc_perm *ipcp)
{ }
static inline void audit_ipc_set_perm(unsigned long qbytes, uid_t uid,
diff --git a/include/linux/bcm47xx_sprom.h b/include/linux/bcm47xx_sprom.h
new file mode 100644
index 0000000..c06b47c
--- /dev/null
+++ b/include/linux/bcm47xx_sprom.h
@@ -0,0 +1,24 @@
+/*
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#ifndef __BCM47XX_SPROM_H
+#define __BCM47XX_SPROM_H
+
+#include <linux/types.h>
+#include <linux/kernel.h>
+#include <linux/vmalloc.h>
+
+#ifdef CONFIG_BCM47XX_SPROM
+int bcm47xx_sprom_register_fallbacks(void);
+#else
+static inline int bcm47xx_sprom_register_fallbacks(void)
+{
+ return -ENOTSUPP;
+};
+#endif
+
+#endif /* __BCM47XX_SPROM_H */
diff --git a/include/linux/bcma/bcma.h b/include/linux/bcma/bcma.h
index 0367c63..e6b41f4 100644
--- a/include/linux/bcma/bcma.h
+++ b/include/linux/bcma/bcma.h
@@ -4,6 +4,7 @@
#include <linux/pci.h>
#include <linux/mod_devicetable.h>
+#include <linux/bcma/bcma_driver_arm_c9.h>
#include <linux/bcma/bcma_driver_chipcommon.h>
#include <linux/bcma/bcma_driver_pci.h>
#include <linux/bcma/bcma_driver_pcie2.h>
diff --git a/include/linux/bcma/bcma_driver_arm_c9.h b/include/linux/bcma/bcma_driver_arm_c9.h
new file mode 100644
index 0000000..93bd73d
--- /dev/null
+++ b/include/linux/bcma/bcma_driver_arm_c9.h
@@ -0,0 +1,15 @@
+#ifndef LINUX_BCMA_DRIVER_ARM_C9_H_
+#define LINUX_BCMA_DRIVER_ARM_C9_H_
+
+/* DMU (Device Management Unit) */
+#define BCMA_DMU_CRU_USB2_CONTROL 0x0164
+#define BCMA_DMU_CRU_USB2_CONTROL_USB_PLL_NDIV_MASK 0x00000FFC
+#define BCMA_DMU_CRU_USB2_CONTROL_USB_PLL_NDIV_SHIFT 2
+#define BCMA_DMU_CRU_USB2_CONTROL_USB_PLL_PDIV_MASK 0x00007000
+#define BCMA_DMU_CRU_USB2_CONTROL_USB_PLL_PDIV_SHIFT 12
+#define BCMA_DMU_CRU_CLKSET_KEY 0x0180
+#define BCMA_DMU_CRU_STRAPS_CTRL 0x02A0
+#define BCMA_DMU_CRU_STRAPS_CTRL_USB3 0x00000010
+#define BCMA_DMU_CRU_STRAPS_CTRL_4BYTE 0x00008000
+
+#endif /* LINUX_BCMA_DRIVER_ARM_C9_H_ */
diff --git a/include/linux/bcma/bcma_driver_chipcommon.h b/include/linux/bcma/bcma_driver_chipcommon.h
index 846513c..a5ac2ca 100644
--- a/include/linux/bcma/bcma_driver_chipcommon.h
+++ b/include/linux/bcma/bcma_driver_chipcommon.h
@@ -587,7 +587,6 @@
struct bcma_sflash {
bool present;
- u32 window;
u32 blocksize;
u16 numblocks;
u32 size;
diff --git a/include/linux/bitops.h b/include/linux/bitops.h
index defeaac..299e76b 100644
--- a/include/linux/bitops.h
+++ b/include/linux/bitops.h
@@ -227,6 +227,22 @@
})
#endif
+#ifndef bit_clear_unless
+#define bit_clear_unless(ptr, _clear, _test) \
+({ \
+ const typeof(*ptr) clear = (_clear), test = (_test); \
+ typeof(*ptr) old, new; \
+ \
+ do { \
+ old = ACCESS_ONCE(*ptr); \
+ new = old & ~clear; \
+ } while (!(old & test) && \
+ cmpxchg(ptr, old, new) != old); \
+ \
+ !(old & test); \
+})
+#endif
+
#ifndef find_last_bit
/**
* find_last_bit - find the last set bit in a memory region
diff --git a/include/linux/bootmem.h b/include/linux/bootmem.h
index 35b22f9..f9be326 100644
--- a/include/linux/bootmem.h
+++ b/include/linux/bootmem.h
@@ -83,34 +83,34 @@
unsigned long goal);
extern void *__alloc_bootmem_nopanic(unsigned long size,
unsigned long align,
- unsigned long goal);
+ unsigned long goal) __malloc;
extern void *__alloc_bootmem_node(pg_data_t *pgdat,
unsigned long size,
unsigned long align,
- unsigned long goal);
+ unsigned long goal) __malloc;
void *__alloc_bootmem_node_high(pg_data_t *pgdat,
unsigned long size,
unsigned long align,
- unsigned long goal);
+ unsigned long goal) __malloc;
extern void *__alloc_bootmem_node_nopanic(pg_data_t *pgdat,
unsigned long size,
unsigned long align,
- unsigned long goal);
+ unsigned long goal) __malloc;
void *___alloc_bootmem_node_nopanic(pg_data_t *pgdat,
unsigned long size,
unsigned long align,
unsigned long goal,
- unsigned long limit);
+ unsigned long limit) __malloc;
extern void *__alloc_bootmem_low(unsigned long size,
unsigned long align,
- unsigned long goal);
+ unsigned long goal) __malloc;
void *__alloc_bootmem_low_nopanic(unsigned long size,
unsigned long align,
- unsigned long goal);
+ unsigned long goal) __malloc;
extern void *__alloc_bootmem_low_node(pg_data_t *pgdat,
unsigned long size,
unsigned long align,
- unsigned long goal);
+ unsigned long goal) __malloc;
#ifdef CONFIG_NO_BOOTMEM
/* We are using top down, so it is safe to use 0 here */
diff --git a/include/linux/clk-provider.h b/include/linux/clk-provider.h
index da95258..0c72204 100644
--- a/include/linux/clk-provider.h
+++ b/include/linux/clk-provider.h
@@ -32,6 +32,7 @@
#define CLK_GET_ACCURACY_NOCACHE BIT(8) /* do not use the cached clk accuracy */
#define CLK_RECALC_NEW_RATES BIT(9) /* recalc rates after notifications */
#define CLK_SET_RATE_UNGATE BIT(10) /* clock needs to run to set rate */
+#define CLK_IS_CRITICAL BIT(11) /* do not gate, ever */
struct clk;
struct clk_hw;
@@ -282,10 +283,17 @@
struct clk *clk_register_fixed_rate(struct device *dev, const char *name,
const char *parent_name, unsigned long flags,
unsigned long fixed_rate);
+struct clk_hw *clk_hw_register_fixed_rate(struct device *dev, const char *name,
+ const char *parent_name, unsigned long flags,
+ unsigned long fixed_rate);
struct clk *clk_register_fixed_rate_with_accuracy(struct device *dev,
const char *name, const char *parent_name, unsigned long flags,
unsigned long fixed_rate, unsigned long fixed_accuracy);
void clk_unregister_fixed_rate(struct clk *clk);
+struct clk_hw *clk_hw_register_fixed_rate_with_accuracy(struct device *dev,
+ const char *name, const char *parent_name, unsigned long flags,
+ unsigned long fixed_rate, unsigned long fixed_accuracy);
+
void of_fixed_clk_setup(struct device_node *np);
/**
@@ -326,7 +334,12 @@
const char *parent_name, unsigned long flags,
void __iomem *reg, u8 bit_idx,
u8 clk_gate_flags, spinlock_t *lock);
+struct clk_hw *clk_hw_register_gate(struct device *dev, const char *name,
+ const char *parent_name, unsigned long flags,
+ void __iomem *reg, u8 bit_idx,
+ u8 clk_gate_flags, spinlock_t *lock);
void clk_unregister_gate(struct clk *clk);
+void clk_hw_unregister_gate(struct clk_hw *hw);
struct clk_div_table {
unsigned int val;
@@ -407,12 +420,22 @@
const char *parent_name, unsigned long flags,
void __iomem *reg, u8 shift, u8 width,
u8 clk_divider_flags, spinlock_t *lock);
+struct clk_hw *clk_hw_register_divider(struct device *dev, const char *name,
+ const char *parent_name, unsigned long flags,
+ void __iomem *reg, u8 shift, u8 width,
+ u8 clk_divider_flags, spinlock_t *lock);
struct clk *clk_register_divider_table(struct device *dev, const char *name,
const char *parent_name, unsigned long flags,
void __iomem *reg, u8 shift, u8 width,
u8 clk_divider_flags, const struct clk_div_table *table,
spinlock_t *lock);
+struct clk_hw *clk_hw_register_divider_table(struct device *dev,
+ const char *name, const char *parent_name, unsigned long flags,
+ void __iomem *reg, u8 shift, u8 width,
+ u8 clk_divider_flags, const struct clk_div_table *table,
+ spinlock_t *lock);
void clk_unregister_divider(struct clk *clk);
+void clk_hw_unregister_divider(struct clk_hw *hw);
/**
* struct clk_mux - multiplexer clock
@@ -463,14 +486,25 @@
unsigned long flags,
void __iomem *reg, u8 shift, u8 width,
u8 clk_mux_flags, spinlock_t *lock);
+struct clk_hw *clk_hw_register_mux(struct device *dev, const char *name,
+ const char * const *parent_names, u8 num_parents,
+ unsigned long flags,
+ void __iomem *reg, u8 shift, u8 width,
+ u8 clk_mux_flags, spinlock_t *lock);
struct clk *clk_register_mux_table(struct device *dev, const char *name,
const char * const *parent_names, u8 num_parents,
unsigned long flags,
void __iomem *reg, u8 shift, u32 mask,
u8 clk_mux_flags, u32 *table, spinlock_t *lock);
+struct clk_hw *clk_hw_register_mux_table(struct device *dev, const char *name,
+ const char * const *parent_names, u8 num_parents,
+ unsigned long flags,
+ void __iomem *reg, u8 shift, u32 mask,
+ u8 clk_mux_flags, u32 *table, spinlock_t *lock);
void clk_unregister_mux(struct clk *clk);
+void clk_hw_unregister_mux(struct clk_hw *hw);
void of_fixed_factor_clk_setup(struct device_node *node);
@@ -499,6 +533,10 @@
const char *parent_name, unsigned long flags,
unsigned int mult, unsigned int div);
void clk_unregister_fixed_factor(struct clk *clk);
+struct clk_hw *clk_hw_register_fixed_factor(struct device *dev,
+ const char *name, const char *parent_name, unsigned long flags,
+ unsigned int mult, unsigned int div);
+void clk_hw_unregister_fixed_factor(struct clk_hw *hw);
/**
* struct clk_fractional_divider - adjustable fractional divider clock
@@ -533,6 +571,11 @@
const char *name, const char *parent_name, unsigned long flags,
void __iomem *reg, u8 mshift, u8 mwidth, u8 nshift, u8 nwidth,
u8 clk_divider_flags, spinlock_t *lock);
+struct clk_hw *clk_hw_register_fractional_divider(struct device *dev,
+ const char *name, const char *parent_name, unsigned long flags,
+ void __iomem *reg, u8 mshift, u8 mwidth, u8 nshift, u8 nwidth,
+ u8 clk_divider_flags, spinlock_t *lock);
+void clk_hw_unregister_fractional_divider(struct clk_hw *hw);
/**
* struct clk_multiplier - adjustable multiplier clock
@@ -603,6 +646,14 @@
struct clk_hw *rate_hw, const struct clk_ops *rate_ops,
struct clk_hw *gate_hw, const struct clk_ops *gate_ops,
unsigned long flags);
+void clk_unregister_composite(struct clk *clk);
+struct clk_hw *clk_hw_register_composite(struct device *dev, const char *name,
+ const char * const *parent_names, int num_parents,
+ struct clk_hw *mux_hw, const struct clk_ops *mux_ops,
+ struct clk_hw *rate_hw, const struct clk_ops *rate_ops,
+ struct clk_hw *gate_hw, const struct clk_ops *gate_ops,
+ unsigned long flags);
+void clk_hw_unregister_composite(struct clk_hw *hw);
/***
* struct clk_gpio_gate - gpio gated clock
@@ -625,6 +676,10 @@
struct clk *clk_register_gpio_gate(struct device *dev, const char *name,
const char *parent_name, unsigned gpio, bool active_low,
unsigned long flags);
+struct clk_hw *clk_hw_register_gpio_gate(struct device *dev, const char *name,
+ const char *parent_name, unsigned gpio, bool active_low,
+ unsigned long flags);
+void clk_hw_unregister_gpio_gate(struct clk_hw *hw);
/**
* struct clk_gpio_mux - gpio controlled clock multiplexer
@@ -640,6 +695,10 @@
struct clk *clk_register_gpio_mux(struct device *dev, const char *name,
const char * const *parent_names, u8 num_parents, unsigned gpio,
bool active_low, unsigned long flags);
+struct clk_hw *clk_hw_register_gpio_mux(struct device *dev, const char *name,
+ const char * const *parent_names, u8 num_parents, unsigned gpio,
+ bool active_low, unsigned long flags);
+void clk_hw_unregister_gpio_mux(struct clk_hw *hw);
/**
* clk_register - allocate a new clock, register it and return an opaque cookie
@@ -655,9 +714,15 @@
struct clk *clk_register(struct device *dev, struct clk_hw *hw);
struct clk *devm_clk_register(struct device *dev, struct clk_hw *hw);
+int __must_check clk_hw_register(struct device *dev, struct clk_hw *hw);
+int __must_check devm_clk_hw_register(struct device *dev, struct clk_hw *hw);
+
void clk_unregister(struct clk *clk);
void devm_clk_unregister(struct device *dev, struct clk *clk);
+void clk_hw_unregister(struct clk_hw *hw);
+void devm_clk_hw_unregister(struct device *dev, struct clk_hw *hw);
+
/* helper functions */
const char *__clk_get_name(const struct clk *clk);
const char *clk_hw_get_name(const struct clk_hw *hw);
@@ -703,6 +768,11 @@
unsigned int clk_num;
};
+struct clk_hw_onecell_data {
+ size_t num;
+ struct clk_hw *hws[];
+};
+
extern struct of_device_id __clk_of_table;
#define CLK_OF_DECLARE(name, compat, fn) OF_DECLARE_1(clk, name, compat, fn)
@@ -712,15 +782,24 @@
struct clk *(*clk_src_get)(struct of_phandle_args *args,
void *data),
void *data);
+int of_clk_add_hw_provider(struct device_node *np,
+ struct clk_hw *(*get)(struct of_phandle_args *clkspec,
+ void *data),
+ void *data);
void of_clk_del_provider(struct device_node *np);
struct clk *of_clk_src_simple_get(struct of_phandle_args *clkspec,
void *data);
+struct clk_hw *of_clk_hw_simple_get(struct of_phandle_args *clkspec,
+ void *data);
struct clk *of_clk_src_onecell_get(struct of_phandle_args *clkspec, void *data);
+struct clk_hw *of_clk_hw_onecell_get(struct of_phandle_args *clkspec,
+ void *data);
unsigned int of_clk_get_parent_count(struct device_node *np);
int of_clk_parent_fill(struct device_node *np, const char **parents,
unsigned int size);
const char *of_clk_get_parent_name(struct device_node *np, int index);
-
+int of_clk_detect_critical(struct device_node *np, int index,
+ unsigned long *flags);
void of_clk_init(const struct of_device_id *matches);
#else /* !CONFIG_OF */
@@ -732,17 +811,34 @@
{
return 0;
}
+static inline int of_clk_add_hw_provider(struct device_node *np,
+ struct clk_hw *(*get)(struct of_phandle_args *clkspec,
+ void *data),
+ void *data)
+{
+ return 0;
+}
static inline void of_clk_del_provider(struct device_node *np) {}
static inline struct clk *of_clk_src_simple_get(
struct of_phandle_args *clkspec, void *data)
{
return ERR_PTR(-ENOENT);
}
+static inline struct clk_hw *
+of_clk_hw_simple_get(struct of_phandle_args *clkspec, void *data)
+{
+ return ERR_PTR(-ENOENT);
+}
static inline struct clk *of_clk_src_onecell_get(
struct of_phandle_args *clkspec, void *data)
{
return ERR_PTR(-ENOENT);
}
+static inline struct clk_hw *
+of_clk_hw_onecell_get(struct of_phandle_args *clkspec, void *data)
+{
+ return ERR_PTR(-ENOENT);
+}
static inline int of_clk_get_parent_count(struct device_node *np)
{
return 0;
@@ -757,6 +853,11 @@
{
return NULL;
}
+static inline int of_clk_detect_critical(struct device_node *np, int index,
+ unsigned long *flags)
+{
+ return 0;
+}
static inline void of_clk_init(const struct of_device_id *matches) {}
#endif /* CONFIG_OF */
diff --git a/include/linux/clk/ti.h b/include/linux/clk/ti.h
index dc5164a..6110fe0 100644
--- a/include/linux/clk/ti.h
+++ b/include/linux/clk/ti.h
@@ -37,6 +37,7 @@
* @last_rounded_n: cache of the last N result of omap2_dpll_round_rate()
* @min_divider: minimum valid non-bypass divider value (actual)
* @max_divider: maximum valid non-bypass divider value (actual)
+ * @max_rate: maximum clock rate for the DPLL
* @modes: possible values of @enable_mask
* @autoidle_reg: register containing the DPLL autoidle mode bitfield
* @idlest_reg: register containing the DPLL idle status bitfield
@@ -81,6 +82,7 @@
u8 last_rounded_n;
u8 min_divider;
u16 max_divider;
+ unsigned long max_rate;
u8 modes;
void __iomem *autoidle_reg;
void __iomem *idlest_reg;
diff --git a/include/linux/clkdev.h b/include/linux/clkdev.h
index c2c04f7..2eabc86 100644
--- a/include/linux/clkdev.h
+++ b/include/linux/clkdev.h
@@ -15,6 +15,7 @@
#include <asm/clkdev.h>
struct clk;
+struct clk_hw;
struct device;
struct clk_lookup {
@@ -34,18 +35,22 @@
struct clk_lookup *clkdev_alloc(struct clk *clk, const char *con_id,
const char *dev_fmt, ...) __printf(3, 4);
+struct clk_lookup *clkdev_hw_alloc(struct clk_hw *hw, const char *con_id,
+ const char *dev_fmt, ...) __printf(3, 4);
void clkdev_add(struct clk_lookup *cl);
void clkdev_drop(struct clk_lookup *cl);
struct clk_lookup *clkdev_create(struct clk *clk, const char *con_id,
const char *dev_fmt, ...) __printf(3, 4);
+struct clk_lookup *clkdev_hw_create(struct clk_hw *hw, const char *con_id,
+ const char *dev_fmt, ...) __printf(3, 4);
void clkdev_add_table(struct clk_lookup *, size_t);
int clk_add_alias(const char *, const char *, const char *, struct device *);
int clk_register_clkdev(struct clk *, const char *, const char *);
-int clk_register_clkdevs(struct clk *, struct clk_lookup *, size_t);
+int clk_hw_register_clkdev(struct clk_hw *, const char *, const char *);
#ifdef CONFIG_COMMON_CLK
int __clk_get(struct clk *clk);
diff --git a/include/linux/compaction.h b/include/linux/compaction.h
index d7c8de5..a58c852 100644
--- a/include/linux/compaction.h
+++ b/include/linux/compaction.h
@@ -2,21 +2,46 @@
#define _LINUX_COMPACTION_H
/* Return values for compact_zone() and try_to_compact_pages() */
-/* compaction didn't start as it was deferred due to past failures */
-#define COMPACT_DEFERRED 0
-/* compaction didn't start as it was not possible or direct reclaim was more suitable */
-#define COMPACT_SKIPPED 1
-/* compaction should continue to another pageblock */
-#define COMPACT_CONTINUE 2
-/* direct compaction partially compacted a zone and there are suitable pages */
-#define COMPACT_PARTIAL 3
-/* The full zone was compacted */
-#define COMPACT_COMPLETE 4
-/* For more detailed tracepoint output */
-#define COMPACT_NO_SUITABLE_PAGE 5
-#define COMPACT_NOT_SUITABLE_ZONE 6
-#define COMPACT_CONTENDED 7
/* When adding new states, please adjust include/trace/events/compaction.h */
+enum compact_result {
+ /* For more detailed tracepoint output - internal to compaction */
+ COMPACT_NOT_SUITABLE_ZONE,
+ /*
+ * compaction didn't start as it was not possible or direct reclaim
+ * was more suitable
+ */
+ COMPACT_SKIPPED,
+ /* compaction didn't start as it was deferred due to past failures */
+ COMPACT_DEFERRED,
+
+ /* compaction not active last round */
+ COMPACT_INACTIVE = COMPACT_DEFERRED,
+
+ /* For more detailed tracepoint output - internal to compaction */
+ COMPACT_NO_SUITABLE_PAGE,
+ /* compaction should continue to another pageblock */
+ COMPACT_CONTINUE,
+
+ /*
+ * The full zone was compacted scanned but wasn't successfull to compact
+ * suitable pages.
+ */
+ COMPACT_COMPLETE,
+ /*
+ * direct compaction has scanned part of the zone but wasn't successfull
+ * to compact suitable pages.
+ */
+ COMPACT_PARTIAL_SKIPPED,
+
+ /* compaction terminated prematurely due to lock contentions */
+ COMPACT_CONTENDED,
+
+ /*
+ * direct compaction partially compacted a zone and there might be
+ * suitable pages
+ */
+ COMPACT_PARTIAL,
+};
/* Used to signal whether compaction detected need_sched() or lock contention */
/* No contention detected */
@@ -38,13 +63,14 @@
extern int sysctl_compact_unevictable_allowed;
extern int fragmentation_index(struct zone *zone, unsigned int order);
-extern unsigned long try_to_compact_pages(gfp_t gfp_mask, unsigned int order,
- int alloc_flags, const struct alloc_context *ac,
- enum migrate_mode mode, int *contended);
+extern enum compact_result try_to_compact_pages(gfp_t gfp_mask,
+ unsigned int order,
+ unsigned int alloc_flags, const struct alloc_context *ac,
+ enum migrate_mode mode, int *contended);
extern void compact_pgdat(pg_data_t *pgdat, int order);
extern void reset_isolation_suitable(pg_data_t *pgdat);
-extern unsigned long compaction_suitable(struct zone *zone, int order,
- int alloc_flags, int classzone_idx);
+extern enum compact_result compaction_suitable(struct zone *zone, int order,
+ unsigned int alloc_flags, int classzone_idx);
extern void defer_compaction(struct zone *zone, int order);
extern bool compaction_deferred(struct zone *zone, int order);
@@ -52,12 +78,80 @@
bool alloc_success);
extern bool compaction_restarting(struct zone *zone, int order);
+/* Compaction has made some progress and retrying makes sense */
+static inline bool compaction_made_progress(enum compact_result result)
+{
+ /*
+ * Even though this might sound confusing this in fact tells us
+ * that the compaction successfully isolated and migrated some
+ * pageblocks.
+ */
+ if (result == COMPACT_PARTIAL)
+ return true;
+
+ return false;
+}
+
+/* Compaction has failed and it doesn't make much sense to keep retrying. */
+static inline bool compaction_failed(enum compact_result result)
+{
+ /* All zones were scanned completely and still not result. */
+ if (result == COMPACT_COMPLETE)
+ return true;
+
+ return false;
+}
+
+/*
+ * Compaction has backed off for some reason. It might be throttling or
+ * lock contention. Retrying is still worthwhile.
+ */
+static inline bool compaction_withdrawn(enum compact_result result)
+{
+ /*
+ * Compaction backed off due to watermark checks for order-0
+ * so the regular reclaim has to try harder and reclaim something.
+ */
+ if (result == COMPACT_SKIPPED)
+ return true;
+
+ /*
+ * If compaction is deferred for high-order allocations, it is
+ * because sync compaction recently failed. If this is the case
+ * and the caller requested a THP allocation, we do not want
+ * to heavily disrupt the system, so we fail the allocation
+ * instead of entering direct reclaim.
+ */
+ if (result == COMPACT_DEFERRED)
+ return true;
+
+ /*
+ * If compaction in async mode encounters contention or blocks higher
+ * priority task we back off early rather than cause stalls.
+ */
+ if (result == COMPACT_CONTENDED)
+ return true;
+
+ /*
+ * Page scanners have met but we haven't scanned full zones so this
+ * is a back off in fact.
+ */
+ if (result == COMPACT_PARTIAL_SKIPPED)
+ return true;
+
+ return false;
+}
+
+
+bool compaction_zonelist_suitable(struct alloc_context *ac, int order,
+ int alloc_flags);
+
extern int kcompactd_run(int nid);
extern void kcompactd_stop(int nid);
extern void wakeup_kcompactd(pg_data_t *pgdat, int order, int classzone_idx);
#else
-static inline unsigned long try_to_compact_pages(gfp_t gfp_mask,
+static inline enum compact_result try_to_compact_pages(gfp_t gfp_mask,
unsigned int order, int alloc_flags,
const struct alloc_context *ac,
enum migrate_mode mode, int *contended)
@@ -73,7 +167,7 @@
{
}
-static inline unsigned long compaction_suitable(struct zone *zone, int order,
+static inline enum compact_result compaction_suitable(struct zone *zone, int order,
int alloc_flags, int classzone_idx)
{
return COMPACT_SKIPPED;
@@ -88,6 +182,21 @@
return true;
}
+static inline bool compaction_made_progress(enum compact_result result)
+{
+ return false;
+}
+
+static inline bool compaction_failed(enum compact_result result)
+{
+ return false;
+}
+
+static inline bool compaction_withdrawn(enum compact_result result)
+{
+ return true;
+}
+
static inline int kcompactd_run(int nid)
{
return 0;
diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h
index 3d5202e..e294939 100644
--- a/include/linux/compiler-gcc.h
+++ b/include/linux/compiler-gcc.h
@@ -142,6 +142,7 @@
#if GCC_VERSION >= 30400
#define __must_check __attribute__((warn_unused_result))
+#define __malloc __attribute__((__malloc__))
#endif
#if GCC_VERSION >= 40000
diff --git a/include/linux/compiler.h b/include/linux/compiler.h
index b5ff988..793c082 100644
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -357,6 +357,10 @@
#define __deprecated_for_modules
#endif
+#ifndef __malloc
+#define __malloc
+#endif
+
/*
* Allow us to avoid 'defined but not used' warnings on functions and data,
* as well as force them to be emitted to the assembly file.
diff --git a/include/linux/console.h b/include/linux/console.h
index ea731af..98c8615 100644
--- a/include/linux/console.h
+++ b/include/linux/console.h
@@ -47,7 +47,7 @@
int (*con_font_copy)(struct vc_data *, int);
int (*con_resize)(struct vc_data *, unsigned int, unsigned int,
unsigned int);
- int (*con_set_palette)(struct vc_data *, unsigned char *);
+ int (*con_set_palette)(struct vc_data *, const unsigned char *);
int (*con_scrolldelta)(struct vc_data *, int);
int (*con_set_origin)(struct vc_data *);
void (*con_save_screen)(struct vc_data *);
@@ -191,6 +191,8 @@
#ifdef CONFIG_VGA_CONSOLE
extern bool vgacon_text_force(void);
+#else
+static inline bool vgacon_text_force(void) { return false; }
#endif
#endif /* _LINUX_CONSOLE_H */
diff --git a/include/linux/coresight-stm.h b/include/linux/coresight-stm.h
new file mode 100644
index 0000000..a978bb8
--- /dev/null
+++ b/include/linux/coresight-stm.h
@@ -0,0 +1,6 @@
+#ifndef __LINUX_CORESIGHT_STM_H_
+#define __LINUX_CORESIGHT_STM_H_
+
+#include <uapi/linux/coresight-stm.h>
+
+#endif
diff --git a/include/linux/cpuset.h b/include/linux/cpuset.h
index 85a868c..bfc204e 100644
--- a/include/linux/cpuset.h
+++ b/include/linux/cpuset.h
@@ -16,26 +16,26 @@
#ifdef CONFIG_CPUSETS
-extern struct static_key cpusets_enabled_key;
+extern struct static_key_false cpusets_enabled_key;
static inline bool cpusets_enabled(void)
{
- return static_key_false(&cpusets_enabled_key);
+ return static_branch_unlikely(&cpusets_enabled_key);
}
static inline int nr_cpusets(void)
{
/* jump label reference count + the top-level cpuset */
- return static_key_count(&cpusets_enabled_key) + 1;
+ return static_key_count(&cpusets_enabled_key.key) + 1;
}
static inline void cpuset_inc(void)
{
- static_key_slow_inc(&cpusets_enabled_key);
+ static_branch_inc(&cpusets_enabled_key);
}
static inline void cpuset_dec(void)
{
- static_key_slow_dec(&cpusets_enabled_key);
+ static_branch_dec(&cpusets_enabled_key);
}
extern int cpuset_init(void);
@@ -48,16 +48,25 @@
void cpuset_init_current_mems_allowed(void);
int cpuset_nodemask_valid_mems_allowed(nodemask_t *nodemask);
-extern int __cpuset_node_allowed(int node, gfp_t gfp_mask);
+extern bool __cpuset_node_allowed(int node, gfp_t gfp_mask);
-static inline int cpuset_node_allowed(int node, gfp_t gfp_mask)
+static inline bool cpuset_node_allowed(int node, gfp_t gfp_mask)
{
- return nr_cpusets() <= 1 || __cpuset_node_allowed(node, gfp_mask);
+ if (cpusets_enabled())
+ return __cpuset_node_allowed(node, gfp_mask);
+ return true;
}
-static inline int cpuset_zone_allowed(struct zone *z, gfp_t gfp_mask)
+static inline bool __cpuset_zone_allowed(struct zone *z, gfp_t gfp_mask)
{
- return cpuset_node_allowed(zone_to_nid(z), gfp_mask);
+ return __cpuset_node_allowed(zone_to_nid(z), gfp_mask);
+}
+
+static inline bool cpuset_zone_allowed(struct zone *z, gfp_t gfp_mask)
+{
+ if (cpusets_enabled())
+ return __cpuset_zone_allowed(z, gfp_mask);
+ return true;
}
extern int cpuset_mems_allowed_intersects(const struct task_struct *tsk1,
@@ -172,14 +181,19 @@
return 1;
}
-static inline int cpuset_node_allowed(int node, gfp_t gfp_mask)
+static inline bool cpuset_node_allowed(int node, gfp_t gfp_mask)
{
- return 1;
+ return true;
}
-static inline int cpuset_zone_allowed(struct zone *z, gfp_t gfp_mask)
+static inline bool __cpuset_zone_allowed(struct zone *z, gfp_t gfp_mask)
{
- return 1;
+ return true;
+}
+
+static inline bool cpuset_zone_allowed(struct zone *z, gfp_t gfp_mask)
+{
+ return true;
}
static inline int cpuset_mems_allowed_intersects(const struct task_struct *tsk1,
diff --git a/include/linux/crash_dump.h b/include/linux/crash_dump.h
index 3849fce..3873697 100644
--- a/include/linux/crash_dump.h
+++ b/include/linux/crash_dump.h
@@ -34,9 +34,13 @@
/*
* Architecture code can redefine this if there are any special checks
- * needed for 64-bit ELF vmcores. In case of 32-bit only architecture,
- * this can be set to zero.
+ * needed for 32-bit ELF or 64-bit ELF vmcores. In case of 32-bit
+ * only architecture, vmcore_elf64_check_arch can be set to zero.
*/
+#ifndef vmcore_elf32_check_arch
+#define vmcore_elf32_check_arch(x) elf_check_arch(x)
+#endif
+
#ifndef vmcore_elf64_check_arch
#define vmcore_elf64_check_arch(x) (elf_check_arch(x) || vmcore_elf_check_arch_cross(x))
#endif
diff --git a/include/linux/debugfs.h b/include/linux/debugfs.h
index 981e53a..1438e23 100644
--- a/include/linux/debugfs.h
+++ b/include/linux/debugfs.h
@@ -19,9 +19,11 @@
#include <linux/seq_file.h>
#include <linux/types.h>
+#include <linux/compiler.h>
struct device;
struct file_operations;
+struct srcu_struct;
struct debugfs_blob_wrapper {
void *data;
@@ -41,14 +43,16 @@
extern struct dentry *arch_debugfs_dir;
+extern struct srcu_struct debugfs_srcu;
+
#if defined(CONFIG_DEBUG_FS)
-/* declared over in file.c */
-extern const struct file_operations debugfs_file_operations;
-
struct dentry *debugfs_create_file(const char *name, umode_t mode,
struct dentry *parent, void *data,
const struct file_operations *fops);
+struct dentry *debugfs_create_file_unsafe(const char *name, umode_t mode,
+ struct dentry *parent, void *data,
+ const struct file_operations *fops);
struct dentry *debugfs_create_file_size(const char *name, umode_t mode,
struct dentry *parent, void *data,
@@ -68,6 +72,31 @@
void debugfs_remove(struct dentry *dentry);
void debugfs_remove_recursive(struct dentry *dentry);
+int debugfs_use_file_start(const struct dentry *dentry, int *srcu_idx)
+ __acquires(&debugfs_srcu);
+
+void debugfs_use_file_finish(int srcu_idx) __releases(&debugfs_srcu);
+
+ssize_t debugfs_attr_read(struct file *file, char __user *buf,
+ size_t len, loff_t *ppos);
+ssize_t debugfs_attr_write(struct file *file, const char __user *buf,
+ size_t len, loff_t *ppos);
+
+#define DEFINE_DEBUGFS_ATTRIBUTE(__fops, __get, __set, __fmt) \
+static int __fops ## _open(struct inode *inode, struct file *file) \
+{ \
+ __simple_attr_check_format(__fmt, 0ull); \
+ return simple_attr_open(inode, file, __get, __set, __fmt); \
+} \
+static const struct file_operations __fops = { \
+ .owner = THIS_MODULE, \
+ .open = __fops ## _open, \
+ .release = simple_attr_release, \
+ .read = debugfs_attr_read, \
+ .write = debugfs_attr_write, \
+ .llseek = generic_file_llseek, \
+}
+
struct dentry *debugfs_rename(struct dentry *old_dir, struct dentry *old_dentry,
struct dentry *new_dir, const char *new_name);
@@ -176,6 +205,20 @@
static inline void debugfs_remove_recursive(struct dentry *dentry)
{ }
+static inline int debugfs_use_file_start(const struct dentry *dentry,
+ int *srcu_idx)
+ __acquires(&debugfs_srcu)
+{
+ return 0;
+}
+
+static inline void debugfs_use_file_finish(int srcu_idx)
+ __releases(&debugfs_srcu)
+{ }
+
+#define DEFINE_DEBUGFS_ATTRIBUTE(__fops, __get, __set, __fmt) \
+ static const struct file_operations __fops = { 0 }
+
static inline struct dentry *debugfs_rename(struct dentry *old_dir, struct dentry *old_dentry,
struct dentry *new_dir, char *new_name)
{
diff --git a/include/linux/debugobjects.h b/include/linux/debugobjects.h
index 98ffcbd..46056cb 100644
--- a/include/linux/debugobjects.h
+++ b/include/linux/debugobjects.h
@@ -38,8 +38,10 @@
* @name: name of the object typee
* @debug_hint: function returning address, which have associated
* kernel symbol, to allow identify the object
+ * @is_static_object return true if the obj is static, otherwise return false
* @fixup_init: fixup function, which is called when the init check
- * fails
+ * fails. All fixup functions must return true if fixup
+ * was successful, otherwise return false
* @fixup_activate: fixup function, which is called when the activate check
* fails
* @fixup_destroy: fixup function, which is called when the destroy check
@@ -51,12 +53,13 @@
*/
struct debug_obj_descr {
const char *name;
- void *(*debug_hint) (void *addr);
- int (*fixup_init) (void *addr, enum debug_obj_state state);
- int (*fixup_activate) (void *addr, enum debug_obj_state state);
- int (*fixup_destroy) (void *addr, enum debug_obj_state state);
- int (*fixup_free) (void *addr, enum debug_obj_state state);
- int (*fixup_assert_init)(void *addr, enum debug_obj_state state);
+ void *(*debug_hint)(void *addr);
+ bool (*is_static_object)(void *addr);
+ bool (*fixup_init)(void *addr, enum debug_obj_state state);
+ bool (*fixup_activate)(void *addr, enum debug_obj_state state);
+ bool (*fixup_destroy)(void *addr, enum debug_obj_state state);
+ bool (*fixup_free)(void *addr, enum debug_obj_state state);
+ bool (*fixup_assert_init)(void *addr, enum debug_obj_state state);
};
#ifdef CONFIG_DEBUG_OBJECTS
diff --git a/include/linux/devcoredump.h b/include/linux/devcoredump.h
index c0a360e..269521f 100644
--- a/include/linux/devcoredump.h
+++ b/include/linux/devcoredump.h
@@ -1,3 +1,22 @@
+/*
+ * This file is provided under the GPLv2 license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright(c) 2015 Intel Deutschland GmbH
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * The full GNU General Public License is included in this distribution
+ * in the file called COPYING.
+ */
#ifndef __DEVCOREDUMP_H
#define __DEVCOREDUMP_H
@@ -5,17 +24,62 @@
#include <linux/module.h>
#include <linux/vmalloc.h>
+#include <linux/scatterlist.h>
+#include <linux/slab.h>
+
+/*
+ * _devcd_free_sgtable - free all the memory of the given scatterlist table
+ * (i.e. both pages and scatterlist instances)
+ * NOTE: if two tables allocated and chained using the sg_chain function then
+ * this function should be called only once on the first table
+ * @table: pointer to sg_table to free
+ */
+static inline void _devcd_free_sgtable(struct scatterlist *table)
+{
+ int i;
+ struct page *page;
+ struct scatterlist *iter;
+ struct scatterlist *delete_iter;
+
+ /* free pages */
+ iter = table;
+ for_each_sg(table, iter, sg_nents(table), i) {
+ page = sg_page(iter);
+ if (page)
+ __free_page(page);
+ }
+
+ /* then free all chained tables */
+ iter = table;
+ delete_iter = table; /* always points on a head of a table */
+ while (!sg_is_last(iter)) {
+ iter++;
+ if (sg_is_chain(iter)) {
+ iter = sg_chain_ptr(iter);
+ kfree(delete_iter);
+ delete_iter = iter;
+ }
+ }
+
+ /* free the last table */
+ kfree(delete_iter);
+}
+
+
#ifdef CONFIG_DEV_COREDUMP
-void dev_coredumpv(struct device *dev, const void *data, size_t datalen,
+void dev_coredumpv(struct device *dev, void *data, size_t datalen,
gfp_t gfp);
void dev_coredumpm(struct device *dev, struct module *owner,
- const void *data, size_t datalen, gfp_t gfp,
+ void *data, size_t datalen, gfp_t gfp,
ssize_t (*read)(char *buffer, loff_t offset, size_t count,
- const void *data, size_t datalen),
- void (*free)(const void *data));
+ void *data, size_t datalen),
+ void (*free)(void *data));
+
+void dev_coredumpsg(struct device *dev, struct scatterlist *table,
+ size_t datalen, gfp_t gfp);
#else
-static inline void dev_coredumpv(struct device *dev, const void *data,
+static inline void dev_coredumpv(struct device *dev, void *data,
size_t datalen, gfp_t gfp)
{
vfree(data);
@@ -23,13 +87,19 @@
static inline void
dev_coredumpm(struct device *dev, struct module *owner,
- const void *data, size_t datalen, gfp_t gfp,
+ void *data, size_t datalen, gfp_t gfp,
ssize_t (*read)(char *buffer, loff_t offset, size_t count,
- const void *data, size_t datalen),
- void (*free)(const void *data))
+ void *data, size_t datalen),
+ void (*free)(void *data))
{
free(data);
}
+
+static inline void dev_coredumpsg(struct device *dev, struct scatterlist *table,
+ size_t datalen, gfp_t gfp)
+{
+ _devcd_free_sgtable(table);
+}
#endif /* CONFIG_DEV_COREDUMP */
#endif /* __DEVCOREDUMP_H */
diff --git a/include/linux/device.h b/include/linux/device.h
index b130304..38f0281 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -609,14 +609,14 @@
#ifdef CONFIG_DEBUG_DEVRES
extern void *__devres_alloc_node(dr_release_t release, size_t size, gfp_t gfp,
- int nid, const char *name);
+ int nid, const char *name) __malloc;
#define devres_alloc(release, size, gfp) \
__devres_alloc_node(release, size, gfp, NUMA_NO_NODE, #release)
#define devres_alloc_node(release, size, gfp, nid) \
__devres_alloc_node(release, size, gfp, nid, #release)
#else
extern void *devres_alloc_node(dr_release_t release, size_t size, gfp_t gfp,
- int nid);
+ int nid) __malloc;
static inline void *devres_alloc(dr_release_t release, size_t size, gfp_t gfp)
{
return devres_alloc_node(release, size, gfp, NUMA_NO_NODE);
@@ -648,12 +648,12 @@
extern int devres_release_group(struct device *dev, void *id);
/* managed devm_k.alloc/kfree for device drivers */
-extern void *devm_kmalloc(struct device *dev, size_t size, gfp_t gfp);
+extern void *devm_kmalloc(struct device *dev, size_t size, gfp_t gfp) __malloc;
extern __printf(3, 0)
char *devm_kvasprintf(struct device *dev, gfp_t gfp, const char *fmt,
- va_list ap);
+ va_list ap) __malloc;
extern __printf(3, 4)
-char *devm_kasprintf(struct device *dev, gfp_t gfp, const char *fmt, ...);
+char *devm_kasprintf(struct device *dev, gfp_t gfp, const char *fmt, ...) __malloc;
static inline void *devm_kzalloc(struct device *dev, size_t size, gfp_t gfp)
{
return devm_kmalloc(dev, size, gfp | __GFP_ZERO);
@@ -671,7 +671,7 @@
return devm_kmalloc_array(dev, n, size, flags | __GFP_ZERO);
}
extern void devm_kfree(struct device *dev, void *p);
-extern char *devm_kstrdup(struct device *dev, const char *s, gfp_t gfp);
+extern char *devm_kstrdup(struct device *dev, const char *s, gfp_t gfp) __malloc;
extern void *devm_kmemdup(struct device *dev, const void *src, size_t len,
gfp_t gfp);
@@ -1288,8 +1288,11 @@
dev_printk(KERN_DEBUG, dev, fmt, ##__VA_ARGS__); \
} while (0)
#else
-#define dev_dbg_ratelimited(dev, fmt, ...) \
- no_printk(KERN_DEBUG pr_fmt(fmt), ##__VA_ARGS__)
+#define dev_dbg_ratelimited(dev, fmt, ...) \
+do { \
+ if (0) \
+ dev_printk(KERN_DEBUG, dev, fmt, ##__VA_ARGS__); \
+} while (0)
#endif
#ifdef VERBOSE_DEBUG
diff --git a/include/linux/dma-iommu.h b/include/linux/dma-iommu.h
index fc48103..8443bbb 100644
--- a/include/linux/dma-iommu.h
+++ b/include/linux/dma-iommu.h
@@ -38,8 +38,8 @@
* These implement the bulk of the relevant DMA mapping callbacks, but require
* the arch code to take care of attributes and cache maintenance
*/
-struct page **iommu_dma_alloc(struct device *dev, size_t size,
- gfp_t gfp, int prot, dma_addr_t *handle,
+struct page **iommu_dma_alloc(struct device *dev, size_t size, gfp_t gfp,
+ struct dma_attrs *attrs, int prot, dma_addr_t *handle,
void (*flush_page)(struct device *, const void *, phys_addr_t));
void iommu_dma_free(struct device *dev, struct page **pages, size_t size,
dma_addr_t *handle);
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 9ea9aba..71c1b215 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -514,7 +514,7 @@
#ifndef arch_setup_dma_ops
static inline void arch_setup_dma_ops(struct device *dev, u64 dma_base,
- u64 size, struct iommu_ops *iommu,
+ u64 size, const struct iommu_ops *iommu,
bool coherent) { }
#endif
diff --git a/include/linux/dma/dw.h b/include/linux/dma/dw.h
index 7145644..f2e538a 100644
--- a/include/linux/dma/dw.h
+++ b/include/linux/dma/dw.h
@@ -27,6 +27,7 @@
* @regs: memory mapped I/O space
* @clk: hclk clock
* @dw: struct dw_dma that is filed by dw_dma_probe()
+ * @pdata: pointer to platform data
*/
struct dw_dma_chip {
struct device *dev;
@@ -34,10 +35,12 @@
void __iomem *regs;
struct clk *clk;
struct dw_dma *dw;
+
+ const struct dw_dma_platform_data *pdata;
};
/* Export to the platform drivers */
-int dw_dma_probe(struct dw_dma_chip *chip, struct dw_dma_platform_data *pdata);
+int dw_dma_probe(struct dw_dma_chip *chip);
int dw_dma_remove(struct dw_dma_chip *chip);
/* DMA API extensions */
diff --git a/include/linux/dma/xilinx_dma.h b/include/linux/dma/xilinx_dma.h
index 34b98f2..3ae3000 100644
--- a/include/linux/dma/xilinx_dma.h
+++ b/include/linux/dma/xilinx_dma.h
@@ -41,6 +41,20 @@
int ext_fsync;
};
+/**
+ * enum xdma_ip_type: DMA IP type.
+ *
+ * XDMA_TYPE_AXIDMA: Axi dma ip.
+ * XDMA_TYPE_CDMA: Axi cdma ip.
+ * XDMA_TYPE_VDMA: Axi vdma ip.
+ *
+ */
+enum xdma_ip_type {
+ XDMA_TYPE_AXIDMA = 0,
+ XDMA_TYPE_CDMA,
+ XDMA_TYPE_VDMA,
+};
+
int xilinx_vdma_channel_set_config(struct dma_chan *dchan,
struct xilinx_vdma_config *cfg);
diff --git a/include/linux/dmaengine.h b/include/linux/dmaengine.h
index 0174337..30de019 100644
--- a/include/linux/dmaengine.h
+++ b/include/linux/dmaengine.h
@@ -804,6 +804,9 @@
sg_dma_address(&sg) = buf;
sg_dma_len(&sg) = len;
+ if (!chan || !chan->device || !chan->device->device_prep_slave_sg)
+ return NULL;
+
return chan->device->device_prep_slave_sg(chan, &sg, 1,
dir, flags, NULL);
}
@@ -812,6 +815,9 @@
struct dma_chan *chan, struct scatterlist *sgl, unsigned int sg_len,
enum dma_transfer_direction dir, unsigned long flags)
{
+ if (!chan || !chan->device || !chan->device->device_prep_slave_sg)
+ return NULL;
+
return chan->device->device_prep_slave_sg(chan, sgl, sg_len,
dir, flags, NULL);
}
@@ -823,6 +829,9 @@
enum dma_transfer_direction dir, unsigned long flags,
struct rio_dma_ext *rio_ext)
{
+ if (!chan || !chan->device || !chan->device->device_prep_slave_sg)
+ return NULL;
+
return chan->device->device_prep_slave_sg(chan, sgl, sg_len,
dir, flags, rio_ext);
}
@@ -833,6 +842,9 @@
size_t period_len, enum dma_transfer_direction dir,
unsigned long flags)
{
+ if (!chan || !chan->device || !chan->device->device_prep_dma_cyclic)
+ return NULL;
+
return chan->device->device_prep_dma_cyclic(chan, buf_addr, buf_len,
period_len, dir, flags);
}
@@ -841,6 +853,9 @@
struct dma_chan *chan, struct dma_interleaved_template *xt,
unsigned long flags)
{
+ if (!chan || !chan->device || !chan->device->device_prep_interleaved_dma)
+ return NULL;
+
return chan->device->device_prep_interleaved_dma(chan, xt, flags);
}
@@ -848,7 +863,7 @@
struct dma_chan *chan, dma_addr_t dest, int value, size_t len,
unsigned long flags)
{
- if (!chan || !chan->device)
+ if (!chan || !chan->device || !chan->device->device_prep_dma_memset)
return NULL;
return chan->device->device_prep_dma_memset(chan, dest, value,
@@ -861,6 +876,9 @@
struct scatterlist *src_sg, unsigned int src_nents,
unsigned long flags)
{
+ if (!chan || !chan->device || !chan->device->device_prep_dma_sg)
+ return NULL;
+
return chan->device->device_prep_dma_sg(chan, dst_sg, dst_nents,
src_sg, src_nents, flags);
}
diff --git a/include/linux/efi.h b/include/linux/efi.h
index df7acb5..c2db3ca 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -21,6 +21,7 @@
#include <linux/pfn.h>
#include <linux/pstore.h>
#include <linux/reboot.h>
+#include <linux/uuid.h>
#include <linux/screen_info.h>
#include <asm/page.h>
@@ -44,17 +45,10 @@
typedef u64 efi_physical_addr_t;
typedef void *efi_handle_t;
-
-typedef struct {
- u8 b[16];
-} efi_guid_t;
+typedef uuid_le efi_guid_t;
#define EFI_GUID(a,b,c,d0,d1,d2,d3,d4,d5,d6,d7) \
-((efi_guid_t) \
-{{ (a) & 0xff, ((a) >> 8) & 0xff, ((a) >> 16) & 0xff, ((a) >> 24) & 0xff, \
- (b) & 0xff, ((b) >> 8) & 0xff, \
- (c) & 0xff, ((c) >> 8) & 0xff, \
- (d0), (d1), (d2), (d3), (d4), (d5), (d6), (d7) }})
+ UUID_LE(a, b, c, d0, d1, d2, d3, d4, d5, d6, d7)
/*
* Generic EFI table header
@@ -1117,7 +1111,7 @@
* Length of a GUID string (strlen("aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"))
* not including trailing NUL
*/
-#define EFI_VARIABLE_GUID_LEN 36
+#define EFI_VARIABLE_GUID_LEN UUID_STRING_LEN
/*
* The type of search to perform when calling boottime->locate_handle
diff --git a/include/linux/f2fs_fs.h b/include/linux/f2fs_fs.h
index b90e9bd..4c02c65 100644
--- a/include/linux/f2fs_fs.h
+++ b/include/linux/f2fs_fs.h
@@ -508,4 +508,6 @@
F2FS_FT_MAX
};
+#define S_SHIFT 12
+
#endif /* _LINUX_F2FS_FS_H */
diff --git a/include/linux/fb.h b/include/linux/fb.h
index dfe8835..a964d07 100644
--- a/include/linux/fb.h
+++ b/include/linux/fb.h
@@ -673,6 +673,7 @@
}
/* drivers/video/fb_defio.c */
+int fb_deferred_io_mmap(struct fb_info *info, struct vm_area_struct *vma);
extern void fb_deferred_io_init(struct fb_info *info);
extern void fb_deferred_io_open(struct fb_info *info,
struct inode *inode,
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 851390c..5f61431 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -2352,14 +2352,6 @@
extern void emergency_thaw_all(void);
extern int thaw_bdev(struct block_device *bdev, struct super_block *sb);
extern int fsync_bdev(struct block_device *);
-#ifdef CONFIG_FS_DAX
-extern bool blkdev_dax_capable(struct block_device *bdev);
-#else
-static inline bool blkdev_dax_capable(struct block_device *bdev)
-{
- return false;
-}
-#endif
extern struct super_block *blockdev_superblock;
@@ -2427,6 +2419,8 @@
/* fs/char_dev.c */
#define CHRDEV_MAJOR_HASH_SIZE 255
+/* Marks the bottom of the first segment of free char majors */
+#define CHRDEV_MAJOR_DYN_END 234
extern int alloc_chrdev_region(dev_t *, unsigned, unsigned, const char *);
extern int register_chrdev_region(dev_t, unsigned, const char *);
extern int __register_chrdev(unsigned int major, unsigned int baseminor,
@@ -2634,15 +2628,34 @@
#endif
extern int do_pipe_flags(int *, int);
+#define __kernel_read_file_id(id) \
+ id(UNKNOWN, unknown) \
+ id(FIRMWARE, firmware) \
+ id(MODULE, kernel-module) \
+ id(KEXEC_IMAGE, kexec-image) \
+ id(KEXEC_INITRAMFS, kexec-initramfs) \
+ id(POLICY, security-policy) \
+ id(MAX_ID, )
+
+#define __fid_enumify(ENUM, dummy) READING_ ## ENUM,
+#define __fid_stringify(dummy, str) #str,
+
enum kernel_read_file_id {
- READING_FIRMWARE = 1,
- READING_MODULE,
- READING_KEXEC_IMAGE,
- READING_KEXEC_INITRAMFS,
- READING_POLICY,
- READING_MAX_ID
+ __kernel_read_file_id(__fid_enumify)
};
+static const char * const kernel_read_file_str[] = {
+ __kernel_read_file_id(__fid_stringify)
+};
+
+static inline const char *kernel_read_file_id_str(enum kernel_read_file_id id)
+{
+ if (id < 0 || id >= READING_MAX_ID)
+ return kernel_read_file_str[READING_UNKNOWN];
+
+ return kernel_read_file_str[id];
+}
+
extern int kernel_read(struct file *, loff_t, char *, unsigned long);
extern int kernel_read_file(struct file *, void **, loff_t *, loff_t,
enum kernel_read_file_id);
diff --git a/include/linux/fscrypto.h b/include/linux/fscrypto.h
index 6027f6b..cfa6cde 100644
--- a/include/linux/fscrypto.h
+++ b/include/linux/fscrypto.h
@@ -175,6 +175,7 @@
*/
struct fscrypt_operations {
int (*get_context)(struct inode *, void *, size_t);
+ int (*key_prefix)(struct inode *, u8 **);
int (*prepare_context)(struct inode *);
int (*set_context)(struct inode *, const void *, size_t, void *);
int (*dummy_context)(struct inode *);
diff --git a/include/linux/fsl_ifc.h b/include/linux/fsl_ifc.h
index 0023088..3f9778c 100644
--- a/include/linux/fsl_ifc.h
+++ b/include/linux/fsl_ifc.h
@@ -39,6 +39,10 @@
#define FSL_IFC_VERSION_MASK 0x0F0F0000
#define FSL_IFC_VERSION_1_0_0 0x01000000
#define FSL_IFC_VERSION_1_1_0 0x01010000
+#define FSL_IFC_VERSION_2_0_0 0x02000000
+
+#define PGOFFSET_64K (64*1024)
+#define PGOFFSET_4K (4*1024)
/*
* CSPR - Chip Select Property Register
@@ -723,20 +727,26 @@
__be32 nand_evter_en;
u32 res17[0x2];
__be32 nand_evter_intr_en;
- u32 res18[0x2];
+ __be32 nand_vol_addr_stat;
+ u32 res18;
__be32 nand_erattr0;
__be32 nand_erattr1;
u32 res19[0x10];
__be32 nand_fsr;
- u32 res20;
- __be32 nand_eccstat[4];
- u32 res21[0x20];
+ u32 res20[0x3];
+ __be32 nand_eccstat[6];
+ u32 res21[0x1c];
__be32 nanndcr;
u32 res22[0x2];
__be32 nand_autoboot_trgr;
u32 res23;
__be32 nand_mdr;
- u32 res24[0x5C];
+ u32 res24[0x1C];
+ __be32 nand_dll_lowcfg0;
+ __be32 nand_dll_lowcfg1;
+ u32 res25;
+ __be32 nand_dll_lowstat;
+ u32 res26[0x3c];
};
/*
@@ -771,13 +781,12 @@
__be32 gpcm_erattr1;
__be32 gpcm_erattr2;
__be32 gpcm_stat;
- u32 res4[0x1F3];
};
/*
* IFC Controller Registers
*/
-struct fsl_ifc_regs {
+struct fsl_ifc_global {
__be32 ifc_rev;
u32 res1[0x2];
struct {
@@ -803,21 +812,26 @@
} ftim_cs[FSL_IFC_BANK_COUNT];
u32 res9[0x30];
__be32 rb_stat;
- u32 res10[0x2];
+ __be32 rb_map;
+ __be32 wb_map;
__be32 ifc_gcr;
- u32 res11[0x2];
+ u32 res10[0x2];
__be32 cm_evter_stat;
- u32 res12[0x2];
+ u32 res11[0x2];
__be32 cm_evter_en;
- u32 res13[0x2];
+ u32 res12[0x2];
__be32 cm_evter_intr_en;
- u32 res14[0x2];
+ u32 res13[0x2];
__be32 cm_erattr0;
__be32 cm_erattr1;
- u32 res15[0x2];
+ u32 res14[0x2];
__be32 ifc_ccr;
__be32 ifc_csr;
- u32 res16[0x2EB];
+ __be32 ddr_ccr_low;
+};
+
+
+struct fsl_ifc_runtime {
struct fsl_ifc_nand ifc_nand;
struct fsl_ifc_nor ifc_nor;
struct fsl_ifc_gpcm ifc_gpcm;
@@ -831,7 +845,8 @@
struct fsl_ifc_ctrl {
/* device info */
struct device *dev;
- struct fsl_ifc_regs __iomem *regs;
+ struct fsl_ifc_global __iomem *gregs;
+ struct fsl_ifc_runtime __iomem *rregs;
int irq;
int nand_irq;
spinlock_t lock;
diff --git a/include/linux/fsnotify_backend.h b/include/linux/fsnotify_backend.h
index 1259e53..29f9175 100644
--- a/include/linux/fsnotify_backend.h
+++ b/include/linux/fsnotify_backend.h
@@ -359,8 +359,6 @@
extern void fsnotify_clear_inode_marks_by_group(struct fsnotify_group *group);
/* run all the marks in a group, and clear all of the marks where mark->flags & flags is true*/
extern void fsnotify_clear_marks_by_group_flags(struct fsnotify_group *group, unsigned int flags);
-/* run all the marks in a group, and flag them to be freed */
-extern void fsnotify_clear_marks_by_group(struct fsnotify_group *group);
extern void fsnotify_get_mark(struct fsnotify_mark *mark);
extern void fsnotify_put_mark(struct fsnotify_mark *mark);
extern void fsnotify_unmount_inodes(struct super_block *sb);
diff --git a/include/linux/genhd.h b/include/linux/genhd.h
index 5c70676..359a8e4 100644
--- a/include/linux/genhd.h
+++ b/include/linux/genhd.h
@@ -14,6 +14,7 @@
#include <linux/rcupdate.h>
#include <linux/slab.h>
#include <linux/percpu-refcount.h>
+#include <linux/uuid.h>
#ifdef CONFIG_BLOCK
@@ -93,7 +94,7 @@
* Enough for the string representation of any kind of UUID plus NULL.
* EFI UUID is 36 characters. MSDOS UUID is 11 characters.
*/
-#define PARTITION_META_INFO_UUIDLTH 37
+#define PARTITION_META_INFO_UUIDLTH (UUID_STRING_LEN + 1)
struct partition_meta_info {
char uuid[PARTITION_META_INFO_UUIDLTH];
@@ -228,27 +229,9 @@
return NULL;
}
-static inline void part_pack_uuid(const u8 *uuid_str, u8 *to)
-{
- int i;
- for (i = 0; i < 16; ++i) {
- *to++ = (hex_to_bin(*uuid_str) << 4) |
- (hex_to_bin(*(uuid_str + 1)));
- uuid_str += 2;
- switch (i) {
- case 3:
- case 5:
- case 7:
- case 9:
- uuid_str++;
- continue;
- }
- }
-}
-
static inline int blk_part_pack_uuid(const u8 *uuid_str, u8 *to)
{
- part_pack_uuid(uuid_str, to);
+ uuid_be_to_bin(uuid_str, (uuid_be *)to);
return 0;
}
diff --git a/include/linux/hardirq.h b/include/linux/hardirq.h
index dfd59d6..c683996 100644
--- a/include/linux/hardirq.h
+++ b/include/linux/hardirq.h
@@ -61,6 +61,7 @@
#define nmi_enter() \
do { \
+ printk_nmi_enter(); \
lockdep_off(); \
ftrace_nmi_enter(); \
BUG_ON(in_nmi()); \
@@ -77,6 +78,7 @@
preempt_count_sub(NMI_OFFSET + HARDIRQ_OFFSET); \
ftrace_nmi_exit(); \
lockdep_on(); \
+ printk_nmi_exit(); \
} while (0)
#endif /* LINUX_HARDIRQ_H */
diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
index d7b9e53..419fb9e 100644
--- a/include/linux/huge_mm.h
+++ b/include/linux/huge_mm.h
@@ -28,9 +28,7 @@
extern int mincore_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd,
unsigned long addr, unsigned long end,
unsigned char *vec);
-extern bool move_huge_pmd(struct vm_area_struct *vma,
- struct vm_area_struct *new_vma,
- unsigned long old_addr,
+extern bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr,
unsigned long new_addr, unsigned long old_end,
pmd_t *old_pmd, pmd_t *new_pmd);
extern int change_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd,
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index 7d953c2..c26d463 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -338,6 +338,7 @@
/* arch callback */
int __init alloc_bootmem_huge_page(struct hstate *h);
+void __init hugetlb_bad_size(void);
void __init hugetlb_add_hstate(unsigned order);
struct hstate *size_to_hstate(unsigned long size);
@@ -352,9 +353,7 @@
static inline struct hstate *hstate_inode(struct inode *i)
{
- struct hugetlbfs_sb_info *hsb;
- hsb = HUGETLBFS_SB(i->i_sb);
- return hsb->hstate;
+ return HUGETLBFS_SB(i->i_sb)->hstate;
}
static inline struct hstate *hstate_file(struct file *f)
@@ -453,12 +452,12 @@
extern void dissolve_free_huge_pages(unsigned long start_pfn,
unsigned long end_pfn);
-static inline int hugepage_migration_supported(struct hstate *h)
+static inline bool hugepage_migration_supported(struct hstate *h)
{
#ifdef CONFIG_ARCH_ENABLE_HUGEPAGE_MIGRATION
return huge_page_shift(h) == PMD_SHIFT;
#else
- return 0;
+ return false;
#endif
}
@@ -520,7 +519,7 @@
return page->index;
}
#define dissolve_free_huge_pages(s, e) do {} while (0)
-#define hugepage_migration_supported(h) 0
+#define hugepage_migration_supported(h) false
static inline spinlock_t *huge_pte_lockptr(struct hstate *h,
struct mm_struct *mm, pte_t *pte)
diff --git a/include/linux/hugetlb_cgroup.h b/include/linux/hugetlb_cgroup.h
index 24154c2..063962f 100644
--- a/include/linux/hugetlb_cgroup.h
+++ b/include/linux/hugetlb_cgroup.h
@@ -93,20 +93,17 @@
struct hugetlb_cgroup *h_cg,
struct page *page)
{
- return;
}
static inline void
hugetlb_cgroup_uncharge_page(int idx, unsigned long nr_pages, struct page *page)
{
- return;
}
static inline void
hugetlb_cgroup_uncharge_cgroup(int idx, unsigned long nr_pages,
struct hugetlb_cgroup *h_cg)
{
- return;
}
static inline void hugetlb_cgroup_file_init(void)
@@ -116,7 +113,6 @@
static inline void hugetlb_cgroup_migrate(struct page *oldhpage,
struct page *newhpage)
{
- return;
}
#endif /* CONFIG_MEM_RES_CTLR_HUGETLB */
diff --git a/include/linux/hugetlb_inline.h b/include/linux/hugetlb_inline.h
index 2bb681f..a4e7ca0 100644
--- a/include/linux/hugetlb_inline.h
+++ b/include/linux/hugetlb_inline.h
@@ -5,16 +5,16 @@
#include <linux/mm.h>
-static inline int is_vm_hugetlb_page(struct vm_area_struct *vma)
+static inline bool is_vm_hugetlb_page(struct vm_area_struct *vma)
{
return !!(vma->vm_flags & VM_HUGETLB);
}
#else
-static inline int is_vm_hugetlb_page(struct vm_area_struct *vma)
+static inline bool is_vm_hugetlb_page(struct vm_area_struct *vma)
{
- return 0;
+ return false;
}
#endif
diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
index aa0fadc..b10954a 100644
--- a/include/linux/hyperv.h
+++ b/include/linux/hyperv.h
@@ -126,6 +126,8 @@
u32 ring_datasize; /* < ring_size */
u32 ring_data_startoffset;
+ u32 priv_write_index;
+ u32 priv_read_index;
};
/*
@@ -151,6 +153,33 @@
*read = dsize - *write;
}
+static inline u32 hv_get_bytes_to_read(struct hv_ring_buffer_info *rbi)
+{
+ u32 read_loc, write_loc, dsize, read;
+
+ dsize = rbi->ring_datasize;
+ read_loc = rbi->ring_buffer->read_index;
+ write_loc = READ_ONCE(rbi->ring_buffer->write_index);
+
+ read = write_loc >= read_loc ? (write_loc - read_loc) :
+ (dsize - read_loc) + write_loc;
+
+ return read;
+}
+
+static inline u32 hv_get_bytes_to_write(struct hv_ring_buffer_info *rbi)
+{
+ u32 read_loc, write_loc, dsize, write;
+
+ dsize = rbi->ring_datasize;
+ read_loc = READ_ONCE(rbi->ring_buffer->read_index);
+ write_loc = rbi->ring_buffer->write_index;
+
+ write = write_loc >= read_loc ? dsize - (write_loc - read_loc) :
+ read_loc - write_loc;
+ return write;
+}
+
/*
* VMBUS version is 32 bit entity broken up into
* two 16 bit quantities: major_number. minor_number.
@@ -1091,7 +1120,7 @@
resource_size_t min, resource_size_t max,
resource_size_t size, resource_size_t align,
bool fb_overlap_ok);
-
+void vmbus_free_mmio(resource_size_t start, resource_size_t size);
int vmbus_cpu_number_to_vp_number(int cpu_number);
u64 hv_do_hypercall(u64 control, void *input, void *output);
@@ -1338,4 +1367,143 @@
int vmbus_send_tl_connect_request(const uuid_le *shv_guest_servie_id,
const uuid_le *shv_host_servie_id);
+void vmbus_set_event(struct vmbus_channel *channel);
+
+/* Get the start of the ring buffer. */
+static inline void *
+hv_get_ring_buffer(struct hv_ring_buffer_info *ring_info)
+{
+ return (void *)ring_info->ring_buffer->buffer;
+}
+
+/*
+ * To optimize the flow management on the send-side,
+ * when the sender is blocked because of lack of
+ * sufficient space in the ring buffer, potential the
+ * consumer of the ring buffer can signal the producer.
+ * This is controlled by the following parameters:
+ *
+ * 1. pending_send_sz: This is the size in bytes that the
+ * producer is trying to send.
+ * 2. The feature bit feat_pending_send_sz set to indicate if
+ * the consumer of the ring will signal when the ring
+ * state transitions from being full to a state where
+ * there is room for the producer to send the pending packet.
+ */
+
+static inline bool hv_need_to_signal_on_read(struct hv_ring_buffer_info *rbi)
+{
+ u32 cur_write_sz;
+ u32 pending_sz;
+
+ /*
+ * Issue a full memory barrier before making the signaling decision.
+ * Here is the reason for having this barrier:
+ * If the reading of the pend_sz (in this function)
+ * were to be reordered and read before we commit the new read
+ * index (in the calling function) we could
+ * have a problem. If the host were to set the pending_sz after we
+ * have sampled pending_sz and go to sleep before we commit the
+ * read index, we could miss sending the interrupt. Issue a full
+ * memory barrier to address this.
+ */
+ virt_mb();
+
+ pending_sz = READ_ONCE(rbi->ring_buffer->pending_send_sz);
+ /* If the other end is not blocked on write don't bother. */
+ if (pending_sz == 0)
+ return false;
+
+ cur_write_sz = hv_get_bytes_to_write(rbi);
+
+ if (cur_write_sz >= pending_sz)
+ return true;
+
+ return false;
+}
+
+/*
+ * An API to support in-place processing of incoming VMBUS packets.
+ */
+#define VMBUS_PKT_TRAILER 8
+
+static inline struct vmpacket_descriptor *
+get_next_pkt_raw(struct vmbus_channel *channel)
+{
+ struct hv_ring_buffer_info *ring_info = &channel->inbound;
+ u32 read_loc = ring_info->priv_read_index;
+ void *ring_buffer = hv_get_ring_buffer(ring_info);
+ struct vmpacket_descriptor *cur_desc;
+ u32 packetlen;
+ u32 dsize = ring_info->ring_datasize;
+ u32 delta = read_loc - ring_info->ring_buffer->read_index;
+ u32 bytes_avail_toread = (hv_get_bytes_to_read(ring_info) - delta);
+
+ if (bytes_avail_toread < sizeof(struct vmpacket_descriptor))
+ return NULL;
+
+ if ((read_loc + sizeof(*cur_desc)) > dsize)
+ return NULL;
+
+ cur_desc = ring_buffer + read_loc;
+ packetlen = cur_desc->len8 << 3;
+
+ /*
+ * If the packet under consideration is wrapping around,
+ * return failure.
+ */
+ if ((read_loc + packetlen + VMBUS_PKT_TRAILER) > (dsize - 1))
+ return NULL;
+
+ return cur_desc;
+}
+
+/*
+ * A helper function to step through packets "in-place"
+ * This API is to be called after each successful call
+ * get_next_pkt_raw().
+ */
+static inline void put_pkt_raw(struct vmbus_channel *channel,
+ struct vmpacket_descriptor *desc)
+{
+ struct hv_ring_buffer_info *ring_info = &channel->inbound;
+ u32 read_loc = ring_info->priv_read_index;
+ u32 packetlen = desc->len8 << 3;
+ u32 dsize = ring_info->ring_datasize;
+
+ if ((read_loc + packetlen + VMBUS_PKT_TRAILER) > dsize)
+ BUG();
+ /*
+ * Include the packet trailer.
+ */
+ ring_info->priv_read_index += packetlen + VMBUS_PKT_TRAILER;
+}
+
+/*
+ * This call commits the read index and potentially signals the host.
+ * Here is the pattern for using the "in-place" consumption APIs:
+ *
+ * while (get_next_pkt_raw() {
+ * process the packet "in-place";
+ * put_pkt_raw();
+ * }
+ * if (packets processed in place)
+ * commit_rd_index();
+ */
+static inline void commit_rd_index(struct vmbus_channel *channel)
+{
+ struct hv_ring_buffer_info *ring_info = &channel->inbound;
+ /*
+ * Make sure all reads are done before we update the read index since
+ * the writer may start writing to the read area once the read index
+ * is updated.
+ */
+ virt_rmb();
+ ring_info->ring_buffer->read_index = ring_info->priv_read_index;
+
+ if (hv_need_to_signal_on_read(ring_info))
+ vmbus_set_event(channel);
+}
+
+
#endif /* _HYPERV_H */
diff --git a/include/linux/i2c-mux.h b/include/linux/i2c-mux.h
index b5f9a00..d4c1d12 100644
--- a/include/linux/i2c-mux.h
+++ b/include/linux/i2c-mux.h
@@ -27,22 +27,49 @@
#ifdef __KERNEL__
-/*
- * Called to create a i2c bus on a multiplexed bus segment.
- * The mux_dev and chan_id parameters are passed to the select
- * and deselect callback functions to perform hardware-specific
- * mux control.
- */
-struct i2c_adapter *i2c_add_mux_adapter(struct i2c_adapter *parent,
- struct device *mux_dev,
- void *mux_priv, u32 force_nr, u32 chan_id,
- unsigned int class,
- int (*select) (struct i2c_adapter *,
- void *mux_dev, u32 chan_id),
- int (*deselect) (struct i2c_adapter *,
- void *mux_dev, u32 chan_id));
+#include <linux/bitops.h>
-void i2c_del_mux_adapter(struct i2c_adapter *adap);
+struct i2c_mux_core {
+ struct i2c_adapter *parent;
+ struct device *dev;
+ bool mux_locked;
+
+ void *priv;
+
+ int (*select)(struct i2c_mux_core *, u32 chan_id);
+ int (*deselect)(struct i2c_mux_core *, u32 chan_id);
+
+ int num_adapters;
+ int max_adapters;
+ struct i2c_adapter *adapter[0];
+};
+
+struct i2c_mux_core *i2c_mux_alloc(struct i2c_adapter *parent,
+ struct device *dev, int max_adapters,
+ int sizeof_priv, u32 flags,
+ int (*select)(struct i2c_mux_core *, u32),
+ int (*deselect)(struct i2c_mux_core *, u32));
+
+/* flags for i2c_mux_alloc */
+#define I2C_MUX_LOCKED BIT(0)
+
+static inline void *i2c_mux_priv(struct i2c_mux_core *muxc)
+{
+ return muxc->priv;
+}
+
+struct i2c_adapter *i2c_root_adapter(struct device *dev);
+
+/*
+ * Called to create an i2c bus on a multiplexed bus segment.
+ * The chan_id parameter is passed to the select and deselect
+ * callback functions to perform hardware-specific mux control.
+ */
+int i2c_mux_add_adapter(struct i2c_mux_core *muxc,
+ u32 force_nr, u32 chan_id,
+ unsigned int class);
+
+void i2c_mux_del_adapters(struct i2c_mux_core *muxc);
#endif /* __KERNEL__ */
diff --git a/include/linux/i2c.h b/include/linux/i2c.h
index 200cf13b..96a25ae 100644
--- a/include/linux/i2c.h
+++ b/include/linux/i2c.h
@@ -524,6 +524,7 @@
/* data fields that are valid for all devices */
struct rt_mutex bus_lock;
+ struct rt_mutex mux_lock;
int timeout; /* in jiffies */
int retries;
@@ -538,6 +539,10 @@
struct i2c_bus_recovery_info *bus_recovery_info;
const struct i2c_adapter_quirks *quirks;
+
+ void (*lock_bus)(struct i2c_adapter *, unsigned int flags);
+ int (*trylock_bus)(struct i2c_adapter *, unsigned int flags);
+ void (*unlock_bus)(struct i2c_adapter *, unsigned int flags);
};
#define to_i2c_adapter(d) container_of(d, struct i2c_adapter, dev)
@@ -567,8 +572,44 @@
int i2c_for_each_dev(void *data, int (*fn)(struct device *, void *));
/* Adapter locking functions, exported for shared pin cases */
-void i2c_lock_adapter(struct i2c_adapter *);
-void i2c_unlock_adapter(struct i2c_adapter *);
+#define I2C_LOCK_ROOT_ADAPTER BIT(0)
+#define I2C_LOCK_SEGMENT BIT(1)
+
+/**
+ * i2c_lock_bus - Get exclusive access to an I2C bus segment
+ * @adapter: Target I2C bus segment
+ * @flags: I2C_LOCK_ROOT_ADAPTER locks the root i2c adapter, I2C_LOCK_SEGMENT
+ * locks only this branch in the adapter tree
+ */
+static inline void
+i2c_lock_bus(struct i2c_adapter *adapter, unsigned int flags)
+{
+ adapter->lock_bus(adapter, flags);
+}
+
+/**
+ * i2c_unlock_bus - Release exclusive access to an I2C bus segment
+ * @adapter: Target I2C bus segment
+ * @flags: I2C_LOCK_ROOT_ADAPTER unlocks the root i2c adapter, I2C_LOCK_SEGMENT
+ * unlocks only this branch in the adapter tree
+ */
+static inline void
+i2c_unlock_bus(struct i2c_adapter *adapter, unsigned int flags)
+{
+ adapter->unlock_bus(adapter, flags);
+}
+
+static inline void
+i2c_lock_adapter(struct i2c_adapter *adapter)
+{
+ i2c_lock_bus(adapter, I2C_LOCK_ROOT_ADAPTER);
+}
+
+static inline void
+i2c_unlock_adapter(struct i2c_adapter *adapter)
+{
+ i2c_unlock_bus(adapter, I2C_LOCK_ROOT_ADAPTER);
+}
/*flags for the client struct: */
#define I2C_CLIENT_PEC 0x04 /* Use Packet Error Checking */
@@ -654,6 +695,11 @@
return adap->nr;
}
+static inline u8 i2c_8bit_addr_from_msg(const struct i2c_msg *msg)
+{
+ return (msg->addr << 1) | (msg->flags & I2C_M_RD ? 1 : 0);
+}
+
/**
* module_i2c_driver() - Helper macro for registering a modular I2C driver
* @__i2c_driver: i2c_driver struct
diff --git a/include/linux/iio/buffer.h b/include/linux/iio/buffer.h
index 2ec3ad5..70a5164 100644
--- a/include/linux/iio/buffer.h
+++ b/include/linux/iio/buffer.h
@@ -83,10 +83,12 @@
* @access: [DRIVER] buffer access functions associated with the
* implementation.
* @scan_el_dev_attr_list:[INTERN] list of scan element related attributes.
+ * @buffer_group: [INTERN] attributes of the buffer group
* @scan_el_group: [DRIVER] attribute group for those attributes not
* created from the iio_chan_info array.
* @pollq: [INTERN] wait queue to allow for polling on the buffer.
* @stufftoread: [INTERN] flag to indicate new data.
+ * @attrs: [INTERN] standard attributes of the buffer
* @demux_list: [INTERN] list of operations required to demux the scan.
* @demux_bounce: [INTERN] buffer for doing gather from incoming scan.
* @buffer_list: [INTERN] entry in the devices list of current buffers.
diff --git a/include/linux/iio/common/st_sensors.h b/include/linux/iio/common/st_sensors.h
index 6670c3d..d029ffa 100644
--- a/include/linux/iio/common/st_sensors.h
+++ b/include/linux/iio/common/st_sensors.h
@@ -37,6 +37,7 @@
#define ST_SENSORS_DEFAULT_AXIS_ADDR 0x20
#define ST_SENSORS_DEFAULT_AXIS_MASK 0x07
#define ST_SENSORS_DEFAULT_AXIS_N_BIT 3
+#define ST_SENSORS_DEFAULT_STAT_ADDR 0x27
#define ST_SENSORS_MAX_NAME 17
#define ST_SENSORS_MAX_4WAI 7
@@ -121,6 +122,9 @@
* @mask_int2: mask to enable/disable IRQ on INT2 pin.
* @addr_ihl: address to enable/disable active low on the INT lines.
* @mask_ihl: mask to enable/disable active low on the INT lines.
+ * @addr_od: address to enable/disable Open Drain on the INT lines.
+ * @mask_od: mask to enable/disable Open Drain on the INT lines.
+ * @addr_stat_drdy: address to read status of DRDY (data ready) interrupt
* struct ig1 - represents the Interrupt Generator 1 of sensors.
* @en_addr: address of the enable ig1 register.
* @en_mask: mask to write the on/off value for enable.
@@ -131,6 +135,9 @@
u8 mask_int2;
u8 addr_ihl;
u8 mask_ihl;
+ u8 addr_od;
+ u8 mask_od;
+ u8 addr_stat_drdy;
struct {
u8 en_addr;
u8 en_mask;
@@ -212,6 +219,7 @@
* @odr: Output data rate of the sensor [Hz].
* num_data_channels: Number of data channels used in buffer.
* @drdy_int_pin: Redirect DRDY on pin 1 (1) or pin 2 (2).
+ * @int_pin_open_drain: Set the interrupt/DRDY to open drain.
* @get_irq_data_ready: Function to get the IRQ used for data ready signal.
* @tf: Transfer function structure used by I/O operations.
* @tb: Transfer buffers and mutex used by I/O operations.
@@ -233,6 +241,7 @@
unsigned int num_data_channels;
u8 drdy_int_pin;
+ bool int_pin_open_drain;
unsigned int (*get_irq_data_ready) (struct iio_dev *indio_dev);
diff --git a/include/linux/iio/consumer.h b/include/linux/iio/consumer.h
index fad5867..3d672f7 100644
--- a/include/linux/iio/consumer.h
+++ b/include/linux/iio/consumer.h
@@ -49,6 +49,33 @@
void iio_channel_release(struct iio_channel *chan);
/**
+ * devm_iio_channel_get() - Resource managed version of iio_channel_get().
+ * @dev: Pointer to consumer device. Device name must match
+ * the name of the device as provided in the iio_map
+ * with which the desired provider to consumer mapping
+ * was registered.
+ * @consumer_channel: Unique name to identify the channel on the consumer
+ * side. This typically describes the channels use within
+ * the consumer. E.g. 'battery_voltage'
+ *
+ * Returns a pointer to negative errno if it is not able to get the iio channel
+ * otherwise returns valid pointer for iio channel.
+ *
+ * The allocated iio channel is automatically released when the device is
+ * unbound.
+ */
+struct iio_channel *devm_iio_channel_get(struct device *dev,
+ const char *consumer_channel);
+/**
+ * devm_iio_channel_release() - Resource managed version of
+ * iio_channel_release().
+ * @dev: Pointer to consumer device for which resource
+ * is allocared.
+ * @chan: The channel to be released.
+ */
+void devm_iio_channel_release(struct device *dev, struct iio_channel *chan);
+
+/**
* iio_channel_get_all() - get all channels associated with a client
* @dev: Pointer to consumer device.
*
@@ -65,6 +92,32 @@
*/
void iio_channel_release_all(struct iio_channel *chan);
+/**
+ * devm_iio_channel_get_all() - Resource managed version of
+ * iio_channel_get_all().
+ * @dev: Pointer to consumer device.
+ *
+ * Returns a pointer to negative errno if it is not able to get the iio channel
+ * otherwise returns an array of iio_channel structures terminated with one with
+ * null iio_dev pointer.
+ *
+ * This function is used by fairly generic consumers to get all the
+ * channels registered as having this consumer.
+ *
+ * The allocated iio channels are automatically released when the device is
+ * unbounded.
+ */
+struct iio_channel *devm_iio_channel_get_all(struct device *dev);
+
+/**
+ * devm_iio_channel_release_all() - Resource managed version of
+ * iio_channel_release_all().
+ * @dev: Pointer to consumer device for which resource
+ * is allocared.
+ * @chan: Array channel to be released.
+ */
+void devm_iio_channel_release_all(struct device *dev, struct iio_channel *chan);
+
struct iio_cb_buffer;
/**
* iio_channel_get_all_cb() - register callback for triggered capture
diff --git a/include/linux/iio/iio.h b/include/linux/iio/iio.h
index b2b1677..7c29cb0 100644
--- a/include/linux/iio/iio.h
+++ b/include/linux/iio/iio.h
@@ -148,6 +148,37 @@
}
/**
+ * struct iio_mount_matrix - iio mounting matrix
+ * @rotation: 3 dimensional space rotation matrix defining sensor alignment with
+ * main hardware
+ */
+struct iio_mount_matrix {
+ const char *rotation[9];
+};
+
+ssize_t iio_show_mount_matrix(struct iio_dev *indio_dev, uintptr_t priv,
+ const struct iio_chan_spec *chan, char *buf);
+int of_iio_read_mount_matrix(const struct device *dev, const char *propname,
+ struct iio_mount_matrix *matrix);
+
+typedef const struct iio_mount_matrix *
+ (iio_get_mount_matrix_t)(const struct iio_dev *indio_dev,
+ const struct iio_chan_spec *chan);
+
+/**
+ * IIO_MOUNT_MATRIX() - Initialize mount matrix extended channel attribute
+ * @_shared: Whether the attribute is shared between all channels
+ * @_get: Pointer to an iio_get_mount_matrix_t accessor
+ */
+#define IIO_MOUNT_MATRIX(_shared, _get) \
+{ \
+ .name = "mount_matrix", \
+ .shared = (_shared), \
+ .read = iio_show_mount_matrix, \
+ .private = (uintptr_t)(_get), \
+}
+
+/**
* struct iio_event_spec - specification for a channel event
* @type: Type of the event
* @dir: Direction of the event
@@ -527,6 +558,8 @@
int devm_iio_device_register(struct device *dev, struct iio_dev *indio_dev);
void devm_iio_device_unregister(struct device *dev, struct iio_dev *indio_dev);
int iio_push_event(struct iio_dev *indio_dev, u64 ev_code, s64 timestamp);
+int iio_device_claim_direct_mode(struct iio_dev *indio_dev);
+void iio_device_release_direct_mode(struct iio_dev *indio_dev);
extern struct bus_type iio_bus_type;
diff --git a/include/linux/iio/imu/adis.h b/include/linux/iio/imu/adis.h
index fa2d01e..360da7d 100644
--- a/include/linux/iio/imu/adis.h
+++ b/include/linux/iio/imu/adis.h
@@ -41,6 +41,7 @@
unsigned int diag_stat_reg;
unsigned int self_test_mask;
+ bool self_test_no_autoclear;
unsigned int startup_delay;
const char * const *status_error_msgs;
diff --git a/include/linux/iio/magnetometer/ak8975.h b/include/linux/iio/magnetometer/ak8975.h
new file mode 100644
index 0000000..c840095
--- /dev/null
+++ b/include/linux/iio/magnetometer/ak8975.h
@@ -0,0 +1,16 @@
+#ifndef __IIO_MAGNETOMETER_AK8975_H__
+#define __IIO_MAGNETOMETER_AK8975_H__
+
+#include <linux/iio/iio.h>
+
+/**
+ * struct ak8975_platform_data - AK8975 magnetometer driver platform data
+ * @eoc_gpio: data ready event gpio
+ * @orientation: mounting matrix relative to main hardware
+ */
+struct ak8975_platform_data {
+ int eoc_gpio;
+ struct iio_mount_matrix orientation;
+};
+
+#endif
diff --git a/include/linux/ima.h b/include/linux/ima.h
index e6516cb..0eb7c2e 100644
--- a/include/linux/ima.h
+++ b/include/linux/ima.h
@@ -21,6 +21,7 @@
extern int ima_read_file(struct file *file, enum kernel_read_file_id id);
extern int ima_post_read_file(struct file *file, void *buf, loff_t size,
enum kernel_read_file_id id);
+extern void ima_post_path_mknod(struct dentry *dentry);
#else
static inline int ima_bprm_check(struct linux_binprm *bprm)
@@ -54,6 +55,11 @@
return 0;
}
+static inline void ima_post_path_mknod(struct dentry *dentry)
+{
+ return;
+}
+
#endif /* CONFIG_IMA */
#ifdef CONFIG_IMA_APPRAISE
diff --git a/include/linux/io-64-nonatomic-hi-lo.h b/include/linux/io-64-nonatomic-hi-lo.h
index 11d7e84..defcc46 100644
--- a/include/linux/io-64-nonatomic-hi-lo.h
+++ b/include/linux/io-64-nonatomic-hi-lo.h
@@ -21,6 +21,23 @@
writel(val, addr);
}
+static inline __u64 hi_lo_readq_relaxed(const volatile void __iomem *addr)
+{
+ const volatile u32 __iomem *p = addr;
+ u32 low, high;
+
+ high = readl_relaxed(p + 1);
+ low = readl_relaxed(p);
+
+ return low + ((u64)high << 32);
+}
+
+static inline void hi_lo_writeq_relaxed(__u64 val, volatile void __iomem *addr)
+{
+ writel_relaxed(val >> 32, addr + 4);
+ writel_relaxed(val, addr);
+}
+
#ifndef readq
#define readq hi_lo_readq
#endif
@@ -29,4 +46,12 @@
#define writeq hi_lo_writeq
#endif
+#ifndef readq_relaxed
+#define readq_relaxed hi_lo_readq_relaxed
+#endif
+
+#ifndef writeq_relaxed
+#define writeq_relaxed hi_lo_writeq_relaxed
+#endif
+
#endif /* _LINUX_IO_64_NONATOMIC_HI_LO_H_ */
diff --git a/include/linux/io-64-nonatomic-lo-hi.h b/include/linux/io-64-nonatomic-lo-hi.h
index 1a4315f..084461a 100644
--- a/include/linux/io-64-nonatomic-lo-hi.h
+++ b/include/linux/io-64-nonatomic-lo-hi.h
@@ -21,6 +21,23 @@
writel(val >> 32, addr + 4);
}
+static inline __u64 lo_hi_readq_relaxed(const volatile void __iomem *addr)
+{
+ const volatile u32 __iomem *p = addr;
+ u32 low, high;
+
+ low = readl_relaxed(p);
+ high = readl_relaxed(p + 1);
+
+ return low + ((u64)high << 32);
+}
+
+static inline void lo_hi_writeq_relaxed(__u64 val, volatile void __iomem *addr)
+{
+ writel_relaxed(val, addr);
+ writel_relaxed(val >> 32, addr + 4);
+}
+
#ifndef readq
#define readq lo_hi_readq
#endif
@@ -29,4 +46,12 @@
#define writeq lo_hi_writeq
#endif
+#ifndef readq_relaxed
+#define readq_relaxed lo_hi_readq_relaxed
+#endif
+
+#ifndef writeq_relaxed
+#define writeq_relaxed lo_hi_writeq_relaxed
+#endif
+
#endif /* _LINUX_IO_64_NONATOMIC_LO_HI_H_ */
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index ef7a6ec..664683a 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -30,6 +30,7 @@
#define IOMMU_WRITE (1 << 1)
#define IOMMU_CACHE (1 << 2) /* DMA cache coherency */
#define IOMMU_NOEXEC (1 << 3)
+#define IOMMU_MMIO (1 << 4) /* e.g. things like MSI doorbells */
struct iommu_ops;
struct iommu_group;
@@ -78,6 +79,7 @@
struct iommu_domain {
unsigned type;
const struct iommu_ops *ops;
+ unsigned long pgsize_bitmap; /* Bitmap of page sizes in use */
iommu_fault_handler_t handler;
void *handler_token;
struct iommu_domain_geometry geometry;
@@ -155,8 +157,7 @@
* @domain_set_windows: Set the number of windows for a domain
* @domain_get_windows: Return the number of windows for a domain
* @of_xlate: add OF master IDs to iommu grouping
- * @pgsize_bitmap: bitmap of supported page sizes
- * @priv: per-instance data private to the iommu driver
+ * @pgsize_bitmap: bitmap of all possible supported page sizes
*/
struct iommu_ops {
bool (*capable)(enum iommu_cap);
@@ -198,7 +199,6 @@
int (*of_xlate)(struct device *dev, struct of_phandle_args *args);
unsigned long pgsize_bitmap;
- void *priv;
};
#define IOMMU_GROUP_NOTIFY_ADD_DEVICE 1 /* Device added */
diff --git a/include/linux/ioport.h b/include/linux/ioport.h
index 0b65543..6230064 100644
--- a/include/linux/ioport.h
+++ b/include/linux/ioport.h
@@ -26,6 +26,9 @@
/*
* IO resources have these defined flags.
+ *
+ * PCI devices expose these flags to userspace in the "resource" sysfs file,
+ * so don't move them.
*/
#define IORESOURCE_BITS 0x000000ff /* Bus-specific bits */
@@ -110,6 +113,7 @@
/* PCI control bits. Shares IORESOURCE_BITS with above PCI ROM. */
#define IORESOURCE_PCI_FIXED (1<<4) /* Do not move resource */
+#define IORESOURCE_PCI_EA_BEI (1<<5) /* BAR Equivalent Indicator */
/*
* I/O Resource Descriptors
diff --git a/include/linux/irqbypass.h b/include/linux/irqbypass.h
index 1551b5b..f0f5d26 100644
--- a/include/linux/irqbypass.h
+++ b/include/linux/irqbypass.h
@@ -34,7 +34,7 @@
/**
* struct irq_bypass_producer - IRQ bypass producer definition
* @node: IRQ bypass manager private list management
- * @token: opaque token to match between producer and consumer
+ * @token: opaque token to match between producer and consumer (non-NULL)
* @irq: Linux IRQ number for the producer device
* @add_consumer: Connect the IRQ producer to an IRQ consumer (optional)
* @del_consumer: Disconnect the IRQ producer from an IRQ consumer (optional)
@@ -60,7 +60,7 @@
/**
* struct irq_bypass_consumer - IRQ bypass consumer definition
* @node: IRQ bypass manager private list management
- * @token: opaque token to match between producer and consumer
+ * @token: opaque token to match between producer and consumer (non-NULL)
* @add_producer: Connect the IRQ consumer to an IRQ producer
* @del_producer: Disconnect the IRQ consumer from an IRQ producer
* @stop: Perform any quiesce operations necessary prior to add/del (optional)
diff --git a/include/linux/irqchip/arm-gic-common.h b/include/linux/irqchip/arm-gic-common.h
new file mode 100644
index 0000000..c647b05
--- /dev/null
+++ b/include/linux/irqchip/arm-gic-common.h
@@ -0,0 +1,34 @@
+/*
+ * include/linux/irqchip/arm-gic-common.h
+ *
+ * Copyright (C) 2016 ARM Limited, All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+#ifndef __LINUX_IRQCHIP_ARM_GIC_COMMON_H
+#define __LINUX_IRQCHIP_ARM_GIC_COMMON_H
+
+#include <linux/types.h>
+#include <linux/ioport.h>
+
+enum gic_type {
+ GIC_V2,
+ GIC_V3,
+};
+
+struct gic_kvm_info {
+ /* GIC type */
+ enum gic_type type;
+ /* Virtual CPU interface */
+ struct resource vcpu;
+ /* Interrupt number */
+ unsigned int maint_irq;
+ /* Virtual control interface */
+ struct resource vctrl;
+};
+
+const struct gic_kvm_info *gic_get_kvm_info(void);
+
+#endif /* __LINUX_IRQCHIP_ARM_GIC_COMMON_H */
diff --git a/include/linux/irqchip/mips-gic.h b/include/linux/irqchip/mips-gic.h
index 80f89e4..81f930b 100644
--- a/include/linux/irqchip/mips-gic.h
+++ b/include/linux/irqchip/mips-gic.h
@@ -103,6 +103,7 @@
#define GIC_VPE_SWINT0_MAP_OFS 0x0054
#define GIC_VPE_SWINT1_MAP_OFS 0x0058
#define GIC_VPE_OTHER_ADDR_OFS 0x0080
+#define GIC_VP_IDENT_OFS 0x0088
#define GIC_VPE_WD_CONFIG0_OFS 0x0090
#define GIC_VPE_WD_COUNT0_OFS 0x0094
#define GIC_VPE_WD_INITIAL0_OFS 0x0098
@@ -211,6 +212,10 @@
#define GIC_VPE_SMASK_FDC_SHF 6
#define GIC_VPE_SMASK_FDC_MSK (MSK(1) << GIC_VPE_SMASK_FDC_SHF)
+/* GIC_VP_IDENT fields */
+#define GIC_VP_IDENT_VCNUM_SHF 0
+#define GIC_VP_IDENT_VCNUM_MSK (MSK(6) << GIC_VP_IDENT_VCNUM_SHF)
+
/* GIC nomenclature for Core Interrupt Pins. */
#define GIC_CPU_INT0 0 /* Core Interrupt 2 */
#define GIC_CPU_INT1 1 /* . */
@@ -278,4 +283,16 @@
#endif /* CONFIG_MIPS_GIC */
+/**
+ * gic_read_local_vp_id() - read the local VPs VCNUM
+ *
+ * Read the VCNUM of the local VP from the GIC_VP_IDENT register and
+ * return it to the caller. This ID should be used to refer to the VP
+ * via the GICs VP-other region, or when calculating an offset to a
+ * bit representing the VP in interrupt masks.
+ *
+ * Return: The VCNUM value for the local VP.
+ */
+extern unsigned gic_read_local_vp_id(void);
+
#endif /* __LINUX_IRQCHIP_MIPS_GIC_H */
diff --git a/include/linux/isa.h b/include/linux/isa.h
index b0270e3..5ab8528 100644
--- a/include/linux/isa.h
+++ b/include/linux/isa.h
@@ -36,4 +36,36 @@
}
#endif
+/**
+ * module_isa_driver() - Helper macro for registering a ISA driver
+ * @__isa_driver: isa_driver struct
+ * @__num_isa_dev: number of devices to register
+ *
+ * Helper macro for ISA drivers which do not do anything special in module
+ * init/exit. This eliminates a lot of boilerplate code. Each module may only
+ * use this macro once, and calling it replaces module_init and module_exit.
+ */
+#define module_isa_driver(__isa_driver, __num_isa_dev) \
+static int __init __isa_driver##_init(void) \
+{ \
+ return isa_register_driver(&(__isa_driver), __num_isa_dev); \
+} \
+module_init(__isa_driver##_init); \
+static void __exit __isa_driver##_exit(void) \
+{ \
+ isa_unregister_driver(&(__isa_driver)); \
+} \
+module_exit(__isa_driver##_exit);
+
+/**
+ * max_num_isa_dev() - Maximum possible number registered of an ISA device
+ * @__ida_dev_ext: ISA device address extent
+ *
+ * The highest base address possible for an ISA device is 0x3FF; this results in
+ * 1024 possible base addresses. Dividing the number of possible base addresses
+ * by the address extent taken by each device results in the maximum number of
+ * devices on a system.
+ */
+#define max_num_isa_dev(__isa_dev_ext) (1024 / __isa_dev_ext)
+
#endif /* __LINUX_ISA_H */
diff --git a/include/linux/iscsi_boot_sysfs.h b/include/linux/iscsi_boot_sysfs.h
index 548d553..10923d7 100644
--- a/include/linux/iscsi_boot_sysfs.h
+++ b/include/linux/iscsi_boot_sysfs.h
@@ -64,6 +64,12 @@
ISCSI_BOOT_INI_END_MARKER,
};
+enum iscsi_boot_acpitbl_properties_enum {
+ ISCSI_BOOT_ACPITBL_SIGNATURE,
+ ISCSI_BOOT_ACPITBL_OEM_ID,
+ ISCSI_BOOT_ACPITBL_OEM_TABLE_ID,
+};
+
struct attribute_group;
struct iscsi_boot_kobj {
@@ -127,6 +133,13 @@
umode_t (*is_visible) (void *data, int type),
void (*release) (void *data));
+struct iscsi_boot_kobj *
+iscsi_boot_create_acpitbl(struct iscsi_boot_kset *boot_kset, int index,
+ void *data,
+ ssize_t (*show)(void *data, int type, char *buf),
+ umode_t (*is_visible)(void *data, int type),
+ void (*release)(void *data));
+
struct iscsi_boot_kset *iscsi_boot_create_kset(const char *set_name);
struct iscsi_boot_kset *iscsi_boot_create_host_kset(unsigned int hostno);
void iscsi_boot_destroy_kset(struct iscsi_boot_kset *boot_kset);
diff --git a/include/linux/jbd2.h b/include/linux/jbd2.h
index fd1083c..efb232c 100644
--- a/include/linux/jbd2.h
+++ b/include/linux/jbd2.h
@@ -403,11 +403,19 @@
/* Flags in jbd_inode->i_flags */
#define __JI_COMMIT_RUNNING 0
-/* Commit of the inode data in progress. We use this flag to protect us from
+#define __JI_WRITE_DATA 1
+#define __JI_WAIT_DATA 2
+
+/*
+ * Commit of the inode data in progress. We use this flag to protect us from
* concurrent deletion of inode. We cannot use reference to inode for this
* since we cannot afford doing last iput() on behalf of kjournald
*/
#define JI_COMMIT_RUNNING (1 << __JI_COMMIT_RUNNING)
+/* Write allocated dirty buffers in this inode before commit */
+#define JI_WRITE_DATA (1 << __JI_WRITE_DATA)
+/* Wait for outstanding data writes for this inode before commit */
+#define JI_WAIT_DATA (1 << __JI_WAIT_DATA)
/**
* struct jbd_inode is the structure linking inodes in ordered mode
@@ -781,9 +789,6 @@
* @j_wbufsize: maximum number of buffer_heads allowed in j_wbuf, the
* number that will fit in j_blocksize
* @j_last_sync_writer: most recent pid which did a synchronous write
- * @j_history: Buffer storing the transactions statistics history
- * @j_history_max: Maximum number of transactions in the statistics history
- * @j_history_cur: Current number of transactions in the statistics history
* @j_history_lock: Protect the transactions statistics history
* @j_proc_entry: procfs entry for the jbd statistics directory
* @j_stats: Overall statistics
@@ -1270,7 +1275,8 @@
extern int jbd2_journal_bmap(journal_t *, unsigned long, unsigned long long *);
extern int jbd2_journal_force_commit(journal_t *);
extern int jbd2_journal_force_commit_nested(journal_t *);
-extern int jbd2_journal_file_inode(handle_t *handle, struct jbd2_inode *inode);
+extern int jbd2_journal_inode_add_write(handle_t *handle, struct jbd2_inode *inode);
+extern int jbd2_journal_inode_add_wait(handle_t *handle, struct jbd2_inode *inode);
extern int jbd2_journal_begin_ordered_truncate(journal_t *journal,
struct jbd2_inode *inode, loff_t new_size);
extern void jbd2_journal_init_jbd_inode(struct jbd2_inode *jinode, struct inode *inode);
diff --git a/include/linux/kasan-checks.h b/include/linux/kasan-checks.h
new file mode 100644
index 0000000..b7f8ace
--- /dev/null
+++ b/include/linux/kasan-checks.h
@@ -0,0 +1,12 @@
+#ifndef _LINUX_KASAN_CHECKS_H
+#define _LINUX_KASAN_CHECKS_H
+
+#ifdef CONFIG_KASAN
+void kasan_check_read(const void *p, unsigned int size);
+void kasan_check_write(const void *p, unsigned int size);
+#else
+static inline void kasan_check_read(const void *p, unsigned int size) { }
+static inline void kasan_check_write(const void *p, unsigned int size) { }
+#endif
+
+#endif
diff --git a/include/linux/kasan.h b/include/linux/kasan.h
index 737371b..611927f 100644
--- a/include/linux/kasan.h
+++ b/include/linux/kasan.h
@@ -50,6 +50,8 @@
void kasan_cache_create(struct kmem_cache *cache, size_t *size,
unsigned long *flags);
+void kasan_cache_shrink(struct kmem_cache *cache);
+void kasan_cache_destroy(struct kmem_cache *cache);
void kasan_poison_slab(struct page *page);
void kasan_unpoison_object_data(struct kmem_cache *cache, void *object);
@@ -63,7 +65,8 @@
void kasan_krealloc(const void *object, size_t new_size, gfp_t flags);
void kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags);
-void kasan_slab_free(struct kmem_cache *s, void *object);
+bool kasan_slab_free(struct kmem_cache *s, void *object);
+void kasan_poison_slab_free(struct kmem_cache *s, void *object);
struct kasan_cache {
int alloc_meta_offset;
@@ -88,6 +91,8 @@
static inline void kasan_cache_create(struct kmem_cache *cache,
size_t *size,
unsigned long *flags) {}
+static inline void kasan_cache_shrink(struct kmem_cache *cache) {}
+static inline void kasan_cache_destroy(struct kmem_cache *cache) {}
static inline void kasan_poison_slab(struct page *page) {}
static inline void kasan_unpoison_object_data(struct kmem_cache *cache,
@@ -105,7 +110,11 @@
static inline void kasan_slab_alloc(struct kmem_cache *s, void *object,
gfp_t flags) {}
-static inline void kasan_slab_free(struct kmem_cache *s, void *object) {}
+static inline bool kasan_slab_free(struct kmem_cache *s, void *object)
+{
+ return false;
+}
+static inline void kasan_poison_slab_free(struct kmem_cache *s, void *object) {}
static inline int kasan_module_alloc(void *addr, size_t size) { return 0; }
static inline void kasan_free_shadow(const struct vm_struct *vm) {}
diff --git a/include/linux/kernel.h b/include/linux/kernel.h
index 2f7775e..94aa10f 100644
--- a/include/linux/kernel.h
+++ b/include/linux/kernel.h
@@ -53,6 +53,13 @@
#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]) + __must_be_array(arr))
+#define u64_to_user_ptr(x) ( \
+{ \
+ typecheck(u64, x); \
+ (void __user *)(uintptr_t)x; \
+} \
+)
+
/*
* This looks more complex than it should be. But we need to
* get the type for the ~ right in round_down (it needs to be
@@ -412,9 +419,9 @@
int scnprintf(char *buf, size_t size, const char *fmt, ...);
extern __printf(3, 0)
int vscnprintf(char *buf, size_t size, const char *fmt, va_list args);
-extern __printf(2, 3)
+extern __printf(2, 3) __malloc
char *kasprintf(gfp_t gfp, const char *fmt, ...);
-extern __printf(2, 0)
+extern __printf(2, 0) __malloc
char *kvasprintf(gfp_t gfp, const char *fmt, va_list args);
extern __printf(2, 0)
const char *kvasprintf_const(gfp_t gfp, const char *fmt, va_list args);
diff --git a/include/linux/kernfs.h b/include/linux/kernfs.h
index 30f089e..96356ef 100644
--- a/include/linux/kernfs.h
+++ b/include/linux/kernfs.h
@@ -179,6 +179,7 @@
/* private fields, do not use outside kernfs proper */
struct mutex mutex;
+ struct mutex prealloc_mutex;
int event;
struct list_head list;
char *prealloc_buf;
diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index 2cc643c..e8acb2b 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -230,8 +230,6 @@
int kexec_should_crash(struct task_struct *);
void crash_save_cpu(struct pt_regs *regs, int cpu);
void crash_save_vmcoreinfo(void);
-void crash_map_reserved_pages(void);
-void crash_unmap_reserved_pages(void);
void arch_crash_save_vmcoreinfo(void);
__printf(1, 2)
void vmcoreinfo_append_str(const char *fmt, ...);
@@ -317,6 +315,8 @@
Elf_Shdr *sechdrs, unsigned int relsec);
int __weak arch_kexec_apply_relocations(const Elf_Ehdr *ehdr, Elf_Shdr *sechdrs,
unsigned int relsec);
+void arch_kexec_protect_crashkres(void);
+void arch_kexec_unprotect_crashkres(void);
#else /* !CONFIG_KEXEC_CORE */
struct pt_regs;
diff --git a/include/linux/key-type.h b/include/linux/key-type.h
index 7463355..eaee981 100644
--- a/include/linux/key-type.h
+++ b/include/linux/key-type.h
@@ -45,7 +45,6 @@
size_t datalen; /* Raw datalen */
size_t quotalen; /* Quota length for proposed payload */
time_t expiry; /* Expiry time of key */
- bool trusted; /* True if key is trusted */
};
typedef int (*request_key_actor_t)(struct key_construction *key,
diff --git a/include/linux/key.h b/include/linux/key.h
index 5f5b112..7229147 100644
--- a/include/linux/key.h
+++ b/include/linux/key.h
@@ -173,11 +173,9 @@
#define KEY_FLAG_NEGATIVE 5 /* set if key is negative */
#define KEY_FLAG_ROOT_CAN_CLEAR 6 /* set if key can be cleared by root without permission */
#define KEY_FLAG_INVALIDATED 7 /* set if key has been invalidated */
-#define KEY_FLAG_TRUSTED 8 /* set if key is trusted */
-#define KEY_FLAG_TRUSTED_ONLY 9 /* set if keyring only accepts links to trusted keys */
-#define KEY_FLAG_BUILTIN 10 /* set if key is builtin */
-#define KEY_FLAG_ROOT_CAN_INVAL 11 /* set if key can be invalidated by root without permission */
-#define KEY_FLAG_KEEP 12 /* set if key should not be removed */
+#define KEY_FLAG_BUILTIN 8 /* set if key is built in to the kernel */
+#define KEY_FLAG_ROOT_CAN_INVAL 9 /* set if key can be invalidated by root without permission */
+#define KEY_FLAG_KEEP 10 /* set if key should not be removed */
/* the key type and key description string
* - the desc is used to match a key against search criteria
@@ -205,6 +203,20 @@
};
int reject_error;
};
+
+ /* This is set on a keyring to restrict the addition of a link to a key
+ * to it. If this method isn't provided then it is assumed that the
+ * keyring is open to any addition. It is ignored for non-keyring
+ * keys.
+ *
+ * This is intended for use with rings of trusted keys whereby addition
+ * to the keyring needs to be controlled. KEY_ALLOC_BYPASS_RESTRICTION
+ * overrides this, allowing the kernel to add extra keys without
+ * restriction.
+ */
+ int (*restrict_link)(struct key *keyring,
+ const struct key_type *type,
+ const union key_payload *payload);
};
extern struct key *key_alloc(struct key_type *type,
@@ -212,14 +224,17 @@
kuid_t uid, kgid_t gid,
const struct cred *cred,
key_perm_t perm,
- unsigned long flags);
+ unsigned long flags,
+ int (*restrict_link)(struct key *,
+ const struct key_type *,
+ const union key_payload *));
-#define KEY_ALLOC_IN_QUOTA 0x0000 /* add to quota, reject if would overrun */
-#define KEY_ALLOC_QUOTA_OVERRUN 0x0001 /* add to quota, permit even if overrun */
-#define KEY_ALLOC_NOT_IN_QUOTA 0x0002 /* not in quota */
-#define KEY_ALLOC_TRUSTED 0x0004 /* Key should be flagged as trusted */
-#define KEY_ALLOC_BUILT_IN 0x0008 /* Key is built into kernel */
+#define KEY_ALLOC_IN_QUOTA 0x0000 /* add to quota, reject if would overrun */
+#define KEY_ALLOC_QUOTA_OVERRUN 0x0001 /* add to quota, permit even if overrun */
+#define KEY_ALLOC_NOT_IN_QUOTA 0x0002 /* not in quota */
+#define KEY_ALLOC_BUILT_IN 0x0004 /* Key is built into kernel */
+#define KEY_ALLOC_BYPASS_RESTRICTION 0x0008 /* Override the check on restricted keyrings */
extern void key_revoke(struct key *key);
extern void key_invalidate(struct key *key);
@@ -288,8 +303,15 @@
const struct cred *cred,
key_perm_t perm,
unsigned long flags,
+ int (*restrict_link)(struct key *,
+ const struct key_type *,
+ const union key_payload *),
struct key *dest);
+extern int restrict_link_reject(struct key *keyring,
+ const struct key_type *type,
+ const union key_payload *payload);
+
extern int keyring_clear(struct key *keyring);
extern key_ref_t keyring_search(key_ref_t keyring,
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 5276fe0..b1fa8f1 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -35,6 +35,10 @@
#include <asm/kvm_host.h>
+#ifndef KVM_MAX_VCPU_ID
+#define KVM_MAX_VCPU_ID KVM_MAX_VCPUS
+#endif
+
/*
* The bit 16 ~ bit 31 of kvm_memory_region::flags are internally used
* in kvm, other bits are visible for userspace which are defined in
@@ -225,6 +229,7 @@
sigset_t sigset;
struct kvm_vcpu_stat stat;
unsigned int halt_poll_ns;
+ bool valid_wakeup;
#ifdef CONFIG_HAS_IOMEM
int mmio_needed;
@@ -447,12 +452,13 @@
static inline struct kvm_vcpu *kvm_get_vcpu_by_id(struct kvm *kvm, int id)
{
- struct kvm_vcpu *vcpu;
+ struct kvm_vcpu *vcpu = NULL;
int i;
- if (id < 0 || id >= KVM_MAX_VCPUS)
+ if (id < 0)
return NULL;
- vcpu = kvm_get_vcpu(kvm, id);
+ if (id < KVM_MAX_VCPUS)
+ vcpu = kvm_get_vcpu(kvm, id);
if (vcpu && vcpu->vcpu_id == id)
return vcpu;
kvm_for_each_vcpu(i, vcpu, kvm)
@@ -651,6 +657,7 @@
void kvm_vcpu_block(struct kvm_vcpu *vcpu);
void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu);
void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu);
+void kvm_vcpu_wake_up(struct kvm_vcpu *vcpu);
void kvm_vcpu_kick(struct kvm_vcpu *vcpu);
int kvm_vcpu_yield_to(struct kvm_vcpu *target);
void kvm_vcpu_on_spin(struct kvm_vcpu *vcpu);
@@ -1091,6 +1098,11 @@
static inline void kvm_make_request(int req, struct kvm_vcpu *vcpu)
{
+ /*
+ * Ensure the rest of the request is published to kvm_check_request's
+ * caller. Paired with the smp_mb__after_atomic in kvm_check_request.
+ */
+ smp_wmb();
set_bit(req, &vcpu->requests);
}
@@ -1098,6 +1110,12 @@
{
if (test_bit(req, &vcpu->requests)) {
clear_bit(req, &vcpu->requests);
+
+ /*
+ * Ensure the rest of the request is visible to kvm_check_request's
+ * caller. Paired with the smp_wmb in kvm_make_request.
+ */
+ smp_mb__after_atomic();
return true;
} else {
return false;
@@ -1169,6 +1187,7 @@
#endif /* CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT */
#ifdef CONFIG_HAVE_KVM_IRQ_BYPASS
+bool kvm_arch_has_irq_bypass(void);
int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *,
struct irq_bypass_producer *);
void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *,
@@ -1179,4 +1198,18 @@
uint32_t guest_irq, bool set);
#endif /* CONFIG_HAVE_KVM_IRQ_BYPASS */
+#ifdef CONFIG_HAVE_KVM_INVALID_WAKEUPS
+/* If we wakeup during the poll time, was it a sucessful poll? */
+static inline bool vcpu_valid_wakeup(struct kvm_vcpu *vcpu)
+{
+ return vcpu->valid_wakeup;
+}
+
+#else
+static inline bool vcpu_valid_wakeup(struct kvm_vcpu *vcpu)
+{
+ return true;
+}
+#endif /* CONFIG_HAVE_KVM_INVALID_WAKEUPS */
+
#endif
diff --git a/include/linux/libata.h b/include/linux/libata.h
index 2c4ebef..d15c19e 100644
--- a/include/linux/libata.h
+++ b/include/linux/libata.h
@@ -180,6 +180,8 @@
ATA_DFLAG_DA = (1 << 26), /* device supports Device Attention */
ATA_DFLAG_DEVSLP = (1 << 27), /* device supports Device Sleep */
ATA_DFLAG_ACPI_DISABLED = (1 << 28), /* ACPI for the device is disabled */
+ ATA_DFLAG_D_SENSE = (1 << 29), /* Descriptor sense requested */
+ ATA_DFLAG_ZAC = (1 << 30), /* ZAC device */
ATA_DEV_UNKNOWN = 0, /* unknown device */
ATA_DEV_ATA = 1, /* ATA device */
@@ -191,7 +193,8 @@
ATA_DEV_SEMB = 7, /* SEMB */
ATA_DEV_SEMB_UNSUP = 8, /* SEMB (unsupported) */
ATA_DEV_ZAC = 9, /* ZAC device */
- ATA_DEV_NONE = 10, /* no device */
+ ATA_DEV_ZAC_UNSUP = 10, /* ZAC device (unsupported) */
+ ATA_DEV_NONE = 11, /* no device */
/* struct ata_link flags */
ATA_LFLAG_NO_HRST = (1 << 1), /* avoid hardreset */
@@ -727,6 +730,13 @@
/* NCQ send and receive log subcommand support */
u8 ncq_send_recv_cmds[ATA_LOG_NCQ_SEND_RECV_SIZE];
+ u8 ncq_non_data_cmds[ATA_LOG_NCQ_NON_DATA_SIZE];
+
+ /* ZAC zone configuration */
+ u32 zac_zoned_cap;
+ u32 zac_zones_optimal_open;
+ u32 zac_zones_optimal_nonseq;
+ u32 zac_zones_max_open;
/* error history */
int spdn_cnt;
@@ -1523,7 +1533,8 @@
static inline unsigned int ata_class_disabled(unsigned int class)
{
return class == ATA_DEV_ATA_UNSUP || class == ATA_DEV_ATAPI_UNSUP ||
- class == ATA_DEV_PMP_UNSUP || class == ATA_DEV_SEMB_UNSUP;
+ class == ATA_DEV_PMP_UNSUP || class == ATA_DEV_SEMB_UNSUP ||
+ class == ATA_DEV_ZAC_UNSUP;
}
static inline unsigned int ata_class_absent(unsigned int class)
@@ -1641,6 +1652,26 @@
ATA_LOG_NCQ_SEND_RECV_DSM_TRIM);
}
+static inline bool ata_fpdma_read_log_supported(struct ata_device *dev)
+{
+ return (dev->flags & ATA_DFLAG_NCQ_SEND_RECV) &&
+ (dev->ncq_send_recv_cmds[ATA_LOG_NCQ_SEND_RECV_RD_LOG_OFFSET] &
+ ATA_LOG_NCQ_SEND_RECV_RD_LOG_SUPPORTED);
+}
+
+static inline bool ata_fpdma_zac_mgmt_in_supported(struct ata_device *dev)
+{
+ return (dev->flags & ATA_DFLAG_NCQ_SEND_RECV) &&
+ (dev->ncq_send_recv_cmds[ATA_LOG_NCQ_SEND_RECV_ZAC_MGMT_OFFSET] &
+ ATA_LOG_NCQ_SEND_RECV_ZAC_MGMT_IN_SUPPORTED);
+}
+
+static inline bool ata_fpdma_zac_mgmt_out_supported(struct ata_device *dev)
+{
+ return (dev->ncq_non_data_cmds[ATA_LOG_NCQ_NON_DATA_ZAC_MGMT_OFFSET] &
+ ATA_LOG_NCQ_NON_DATA_ZAC_MGMT_OUT);
+}
+
static inline void ata_qc_set_polling(struct ata_queued_cmd *qc)
{
qc->tf.ctl |= ATA_NIEN;
diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h
index 833867b..0c3c30c 100644
--- a/include/linux/libnvdimm.h
+++ b/include/linux/libnvdimm.h
@@ -27,7 +27,7 @@
/* need to set a limit somewhere, but yes, this is likely overkill */
ND_IOCTL_MAX_BUFLEN = SZ_4M,
ND_CMD_MAX_ELEM = 5,
- ND_CMD_MAX_ENVELOPE = 16,
+ ND_CMD_MAX_ENVELOPE = 256,
ND_MAX_MAPPINGS = 32,
/* region flag indicating to direct-map persistent memory by default */
@@ -68,7 +68,7 @@
struct nvdimm_bus_descriptor {
const struct attribute_group **attr_groups;
- unsigned long dsm_mask;
+ unsigned long cmd_mask;
char *provider_name;
ndctl_fn ndctl;
int (*flush_probe)(struct nvdimm_bus_descriptor *nd_desc);
@@ -130,10 +130,11 @@
struct nd_blk_region *to_nd_blk_region(struct device *dev);
struct nvdimm_bus_descriptor *to_nd_desc(struct nvdimm_bus *nvdimm_bus);
const char *nvdimm_name(struct nvdimm *nvdimm);
+unsigned long nvdimm_cmd_mask(struct nvdimm *nvdimm);
void *nvdimm_provider_data(struct nvdimm *nvdimm);
struct nvdimm *nvdimm_create(struct nvdimm_bus *nvdimm_bus, void *provider_data,
const struct attribute_group **groups, unsigned long flags,
- unsigned long *dsm_mask);
+ unsigned long cmd_mask);
const struct nd_cmd_desc *nd_cmd_dimm_desc(int cmd);
const struct nd_cmd_desc *nd_cmd_bus_desc(int cmd);
u32 nd_cmd_in_size(struct nvdimm *nvdimm, int cmd,
diff --git a/include/linux/lsm_hooks.h b/include/linux/lsm_hooks.h
index 512fd00..7ae3976 100644
--- a/include/linux/lsm_hooks.h
+++ b/include/linux/lsm_hooks.h
@@ -1805,7 +1805,6 @@
struct list_head tun_dev_attach_queue;
struct list_head tun_dev_attach;
struct list_head tun_dev_open;
- struct list_head skb_owned_by;
#endif /* CONFIG_SECURITY_NETWORK */
#ifdef CONFIG_SECURITY_NETWORK_XFRM
struct list_head xfrm_policy_alloc_security;
@@ -1894,5 +1893,10 @@
#else
static inline void __init yama_add_hooks(void) { }
#endif
+#ifdef CONFIG_SECURITY_LOADPIN
+void __init loadpin_add_hooks(void);
+#else
+static inline void loadpin_add_hooks(void) { };
+#endif
#endif /* ! __LINUX_LSM_HOOKS_H */
diff --git a/include/linux/mcb.h b/include/linux/mcb.h
index ed06e15..ead13d2 100644
--- a/include/linux/mcb.h
+++ b/include/linux/mcb.h
@@ -15,22 +15,30 @@
#include <linux/device.h>
#include <linux/irqreturn.h>
+#define CHAMELEON_FILENAME_LEN 12
+
struct mcb_driver;
struct mcb_device;
/**
* struct mcb_bus - MEN Chameleon Bus
*
- * @dev: pointer to carrier device
- * @children: the child busses
+ * @dev: bus device
+ * @carrier: pointer to carrier device
* @bus_nr: mcb bus number
* @get_irq: callback to get IRQ number
+ * @revision: the FPGA's revision number
+ * @model: the FPGA's model number
+ * @filename: the FPGA's name
*/
struct mcb_bus {
- struct list_head children;
struct device dev;
struct device *carrier;
int bus_nr;
+ u8 revision;
+ char model;
+ u8 minor;
+ char name[CHAMELEON_FILENAME_LEN + 1];
int (*get_irq)(struct mcb_device *dev);
};
#define to_mcb_bus(b) container_of((b), struct mcb_bus, dev)
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 1191d79..a805474 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -415,25 +415,6 @@
return mz->lru_size[lru];
}
-static inline bool mem_cgroup_inactive_anon_is_low(struct lruvec *lruvec)
-{
- unsigned long inactive_ratio;
- unsigned long inactive;
- unsigned long active;
- unsigned long gb;
-
- inactive = mem_cgroup_get_lru_size(lruvec, LRU_INACTIVE_ANON);
- active = mem_cgroup_get_lru_size(lruvec, LRU_ACTIVE_ANON);
-
- gb = (inactive + active) >> (30 - PAGE_SHIFT);
- if (gb)
- inactive_ratio = int_sqrt(10 * gb);
- else
- inactive_ratio = 1;
-
- return inactive * inactive_ratio < active;
-}
-
void mem_cgroup_handle_over_high(void);
void mem_cgroup_print_oom_info(struct mem_cgroup *memcg,
@@ -646,24 +627,12 @@
return true;
}
-static inline bool
-mem_cgroup_inactive_anon_is_low(struct lruvec *lruvec)
-{
- return true;
-}
-
static inline unsigned long
mem_cgroup_get_lru_size(struct lruvec *lruvec, enum lru_list lru)
{
return 0;
}
-static inline void
-mem_cgroup_update_lru_size(struct lruvec *lruvec, enum lru_list lru,
- int increment)
-{
-}
-
static inline unsigned long
mem_cgroup_node_nr_lru_pages(struct mem_cgroup *memcg,
int nid, unsigned int lru_mask)
diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h
index adbef58..20d8a5d 100644
--- a/include/linux/memory_hotplug.h
+++ b/include/linux/memory_hotplug.h
@@ -247,16 +247,16 @@
#ifdef CONFIG_MEMORY_HOTREMOVE
-extern int is_mem_section_removable(unsigned long pfn, unsigned long nr_pages);
+extern bool is_mem_section_removable(unsigned long pfn, unsigned long nr_pages);
extern void try_offline_node(int nid);
extern int offline_pages(unsigned long start_pfn, unsigned long nr_pages);
extern void remove_memory(int nid, u64 start, u64 size);
#else
-static inline int is_mem_section_removable(unsigned long pfn,
+static inline bool is_mem_section_removable(unsigned long pfn,
unsigned long nr_pages)
{
- return 0;
+ return false;
}
static inline void try_offline_node(int nid) {}
diff --git a/include/linux/mempolicy.h b/include/linux/mempolicy.h
index 2696c1f..4429d25 100644
--- a/include/linux/mempolicy.h
+++ b/include/linux/mempolicy.h
@@ -172,14 +172,14 @@
extern void mpol_to_str(char *buffer, int maxlen, struct mempolicy *pol);
/* Check if a vma is migratable */
-static inline int vma_migratable(struct vm_area_struct *vma)
+static inline bool vma_migratable(struct vm_area_struct *vma)
{
if (vma->vm_flags & (VM_IO | VM_PFNMAP))
- return 0;
+ return false;
#ifndef CONFIG_ARCH_ENABLE_HUGEPAGE_MIGRATION
if (vma->vm_flags & VM_HUGETLB)
- return 0;
+ return false;
#endif
/*
@@ -190,8 +190,8 @@
if (vma->vm_file &&
gfp_zone(mapping_gfp_mask(vma->vm_file->f_mapping))
< policy_zone)
- return 0;
- return 1;
+ return false;
+ return true;
}
extern int mpol_misplaced(struct page *, struct vm_area_struct *, unsigned long);
@@ -228,6 +228,12 @@
{
}
+static inline struct mempolicy *
+mpol_shared_policy_lookup(struct shared_policy *sp, unsigned long idx)
+{
+ return NULL;
+}
+
#define vma_policy(vma) NULL
static inline int
diff --git a/include/linux/mempool.h b/include/linux/mempool.h
index 69b6951..b1086c9 100644
--- a/include/linux/mempool.h
+++ b/include/linux/mempool.h
@@ -5,6 +5,7 @@
#define _LINUX_MEMPOOL_H
#include <linux/wait.h>
+#include <linux/compiler.h>
struct kmem_cache;
@@ -31,7 +32,7 @@
extern int mempool_resize(mempool_t *pool, int new_min_nr);
extern void mempool_destroy(mempool_t *pool);
-extern void * mempool_alloc(mempool_t *pool, gfp_t gfp_mask);
+extern void *mempool_alloc(mempool_t *pool, gfp_t gfp_mask) __malloc;
extern void mempool_free(void *element, mempool_t *pool);
/*
diff --git a/include/linux/mfd/axp20x.h b/include/linux/mfd/axp20x.h
index d82e7d5..0be4982 100644
--- a/include/linux/mfd/axp20x.h
+++ b/include/linux/mfd/axp20x.h
@@ -20,6 +20,7 @@
AXP221_ID,
AXP223_ID,
AXP288_ID,
+ AXP809_ID,
NR_AXP20X_VARIANTS,
};
@@ -264,6 +265,29 @@
AXP22X_REG_ID_MAX,
};
+enum {
+ AXP809_DCDC1 = 0,
+ AXP809_DCDC2,
+ AXP809_DCDC3,
+ AXP809_DCDC4,
+ AXP809_DCDC5,
+ AXP809_DC1SW,
+ AXP809_DC5LDO,
+ AXP809_ALDO1,
+ AXP809_ALDO2,
+ AXP809_ALDO3,
+ AXP809_ELDO1,
+ AXP809_ELDO2,
+ AXP809_ELDO3,
+ AXP809_DLDO1,
+ AXP809_DLDO2,
+ AXP809_RTC_LDO,
+ AXP809_LDO_IO0,
+ AXP809_LDO_IO1,
+ AXP809_SW,
+ AXP809_REG_ID_MAX,
+};
+
/* IRQs */
enum {
AXP152_IRQ_LDO0IN_CONNECT = 1,
@@ -390,6 +414,41 @@
AXP288_IRQ_BC_USB_CHNG,
};
+enum axp809_irqs {
+ AXP809_IRQ_ACIN_OVER_V = 1,
+ AXP809_IRQ_ACIN_PLUGIN,
+ AXP809_IRQ_ACIN_REMOVAL,
+ AXP809_IRQ_VBUS_OVER_V,
+ AXP809_IRQ_VBUS_PLUGIN,
+ AXP809_IRQ_VBUS_REMOVAL,
+ AXP809_IRQ_VBUS_V_LOW,
+ AXP809_IRQ_BATT_PLUGIN,
+ AXP809_IRQ_BATT_REMOVAL,
+ AXP809_IRQ_BATT_ENT_ACT_MODE,
+ AXP809_IRQ_BATT_EXIT_ACT_MODE,
+ AXP809_IRQ_CHARG,
+ AXP809_IRQ_CHARG_DONE,
+ AXP809_IRQ_BATT_CHG_TEMP_HIGH,
+ AXP809_IRQ_BATT_CHG_TEMP_HIGH_END,
+ AXP809_IRQ_BATT_CHG_TEMP_LOW,
+ AXP809_IRQ_BATT_CHG_TEMP_LOW_END,
+ AXP809_IRQ_BATT_ACT_TEMP_HIGH,
+ AXP809_IRQ_BATT_ACT_TEMP_HIGH_END,
+ AXP809_IRQ_BATT_ACT_TEMP_LOW,
+ AXP809_IRQ_BATT_ACT_TEMP_LOW_END,
+ AXP809_IRQ_DIE_TEMP_HIGH,
+ AXP809_IRQ_LOW_PWR_LVL1,
+ AXP809_IRQ_LOW_PWR_LVL2,
+ AXP809_IRQ_TIMER,
+ AXP809_IRQ_PEK_RIS_EDGE,
+ AXP809_IRQ_PEK_FAL_EDGE,
+ AXP809_IRQ_PEK_SHORT,
+ AXP809_IRQ_PEK_LONG,
+ AXP809_IRQ_PEK_OVER_OFF,
+ AXP809_IRQ_GPIO1_INPUT,
+ AXP809_IRQ_GPIO0_INPUT,
+};
+
#define AXP288_TS_ADC_H 0x58
#define AXP288_TS_ADC_L 0x59
#define AXP288_GP_ADC_H 0x5a
diff --git a/include/linux/mfd/core.h b/include/linux/mfd/core.h
index 9837f1e..99c0395 100644
--- a/include/linux/mfd/core.h
+++ b/include/linux/mfd/core.h
@@ -131,4 +131,8 @@
extern void mfd_remove_devices(struct device *parent);
+extern int devm_mfd_add_devices(struct device *dev, int id,
+ const struct mfd_cell *cells, int n_devs,
+ struct resource *mem_base,
+ int irq_base, struct irq_domain *irq_domain);
#endif
diff --git a/include/linux/mfd/hi655x-pmic.h b/include/linux/mfd/hi655x-pmic.h
new file mode 100644
index 0000000..dbbe9a6
--- /dev/null
+++ b/include/linux/mfd/hi655x-pmic.h
@@ -0,0 +1,55 @@
+/*
+ * Device driver for regulators in hi655x IC
+ *
+ * Copyright (c) 2016 Hisilicon.
+ *
+ * Authors:
+ * Chen Feng <puck.chen@hisilicon.com>
+ * Fei Wang <w.f@huawei.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef __HI655X_PMIC_H
+#define __HI655X_PMIC_H
+
+/* Hi655x registers are mapped to memory bus in 4 bytes stride */
+#define HI655X_STRIDE 4
+#define HI655X_BUS_ADDR(x) ((x) << 2)
+
+#define HI655X_BITS 8
+
+#define HI655X_NR_IRQ 32
+
+#define HI655X_IRQ_STAT_BASE (0x003 << 2)
+#define HI655X_IRQ_MASK_BASE (0x007 << 2)
+#define HI655X_ANA_IRQM_BASE (0x1b5 << 2)
+#define HI655X_IRQ_ARRAY 4
+#define HI655X_IRQ_MASK 0xFF
+#define HI655X_IRQ_CLR 0xFF
+#define HI655X_VER_REG 0x00
+
+#define PMU_VER_START 0x10
+#define PMU_VER_END 0x38
+
+#define RESERVE_INT BIT(7)
+#define PWRON_D20R_INT BIT(6)
+#define PWRON_D20F_INT BIT(5)
+#define PWRON_D4SR_INT BIT(4)
+#define VSYS_6P0_D200UR_INT BIT(3)
+#define VSYS_UV_D3R_INT BIT(2)
+#define VSYS_2P5_R_INT BIT(1)
+#define OTMP_D1R_INT BIT(0)
+
+struct hi655x_pmic {
+ struct resource *res;
+ struct device *dev;
+ struct regmap *regmap;
+ int gpio;
+ unsigned int ver;
+ struct regmap_irq_chip_data *irq_data;
+};
+
+#endif
diff --git a/include/linux/mfd/max77620.h b/include/linux/mfd/max77620.h
new file mode 100644
index 0000000..3ca0af07
--- /dev/null
+++ b/include/linux/mfd/max77620.h
@@ -0,0 +1,346 @@
+/*
+ * Defining registers address and its bit definitions of MAX77620 and MAX20024
+ *
+ * Copyright (C) 2016 NVIDIA CORPORATION. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ */
+
+#ifndef _MFD_MAX77620_H_
+#define _MFD_MAX77620_H_
+
+#include <linux/types.h>
+
+/* GLOBAL, PMIC, GPIO, FPS, ONOFFC, CID Registers */
+#define MAX77620_REG_CNFGGLBL1 0x00
+#define MAX77620_REG_CNFGGLBL2 0x01
+#define MAX77620_REG_CNFGGLBL3 0x02
+#define MAX77620_REG_CNFG1_32K 0x03
+#define MAX77620_REG_CNFGBBC 0x04
+#define MAX77620_REG_IRQTOP 0x05
+#define MAX77620_REG_INTLBT 0x06
+#define MAX77620_REG_IRQSD 0x07
+#define MAX77620_REG_IRQ_LVL2_L0_7 0x08
+#define MAX77620_REG_IRQ_LVL2_L8 0x09
+#define MAX77620_REG_IRQ_LVL2_GPIO 0x0A
+#define MAX77620_REG_ONOFFIRQ 0x0B
+#define MAX77620_REG_NVERC 0x0C
+#define MAX77620_REG_IRQTOPM 0x0D
+#define MAX77620_REG_INTENLBT 0x0E
+#define MAX77620_REG_IRQMASKSD 0x0F
+#define MAX77620_REG_IRQ_MSK_L0_7 0x10
+#define MAX77620_REG_IRQ_MSK_L8 0x11
+#define MAX77620_REG_ONOFFIRQM 0x12
+#define MAX77620_REG_STATLBT 0x13
+#define MAX77620_REG_STATSD 0x14
+#define MAX77620_REG_ONOFFSTAT 0x15
+
+/* SD and LDO Registers */
+#define MAX77620_REG_SD0 0x16
+#define MAX77620_REG_SD1 0x17
+#define MAX77620_REG_SD2 0x18
+#define MAX77620_REG_SD3 0x19
+#define MAX77620_REG_SD4 0x1A
+#define MAX77620_REG_DVSSD0 0x1B
+#define MAX77620_REG_DVSSD1 0x1C
+#define MAX77620_REG_SD0_CFG 0x1D
+#define MAX77620_REG_SD1_CFG 0x1E
+#define MAX77620_REG_SD2_CFG 0x1F
+#define MAX77620_REG_SD3_CFG 0x20
+#define MAX77620_REG_SD4_CFG 0x21
+#define MAX77620_REG_SD_CFG2 0x22
+#define MAX77620_REG_LDO0_CFG 0x23
+#define MAX77620_REG_LDO0_CFG2 0x24
+#define MAX77620_REG_LDO1_CFG 0x25
+#define MAX77620_REG_LDO1_CFG2 0x26
+#define MAX77620_REG_LDO2_CFG 0x27
+#define MAX77620_REG_LDO2_CFG2 0x28
+#define MAX77620_REG_LDO3_CFG 0x29
+#define MAX77620_REG_LDO3_CFG2 0x2A
+#define MAX77620_REG_LDO4_CFG 0x2B
+#define MAX77620_REG_LDO4_CFG2 0x2C
+#define MAX77620_REG_LDO5_CFG 0x2D
+#define MAX77620_REG_LDO5_CFG2 0x2E
+#define MAX77620_REG_LDO6_CFG 0x2F
+#define MAX77620_REG_LDO6_CFG2 0x30
+#define MAX77620_REG_LDO7_CFG 0x31
+#define MAX77620_REG_LDO7_CFG2 0x32
+#define MAX77620_REG_LDO8_CFG 0x33
+#define MAX77620_REG_LDO8_CFG2 0x34
+#define MAX77620_REG_LDO_CFG3 0x35
+
+#define MAX77620_LDO_SLEW_RATE_MASK 0x1
+
+/* LDO Configuration 3 */
+#define MAX77620_TRACK4_MASK BIT(5)
+#define MAX77620_TRACK4_SHIFT 5
+
+/* Voltage */
+#define MAX77620_SDX_VOLT_MASK 0xFF
+#define MAX77620_SD0_VOLT_MASK 0x3F
+#define MAX77620_SD1_VOLT_MASK 0x7F
+#define MAX77620_LDO_VOLT_MASK 0x3F
+
+#define MAX77620_REG_GPIO0 0x36
+#define MAX77620_REG_GPIO1 0x37
+#define MAX77620_REG_GPIO2 0x38
+#define MAX77620_REG_GPIO3 0x39
+#define MAX77620_REG_GPIO4 0x3A
+#define MAX77620_REG_GPIO5 0x3B
+#define MAX77620_REG_GPIO6 0x3C
+#define MAX77620_REG_GPIO7 0x3D
+#define MAX77620_REG_PUE_GPIO 0x3E
+#define MAX77620_REG_PDE_GPIO 0x3F
+#define MAX77620_REG_AME_GPIO 0x40
+#define MAX77620_REG_ONOFFCNFG1 0x41
+#define MAX77620_REG_ONOFFCNFG2 0x42
+
+/* FPS Registers */
+#define MAX77620_REG_FPS_CFG0 0x43
+#define MAX77620_REG_FPS_CFG1 0x44
+#define MAX77620_REG_FPS_CFG2 0x45
+#define MAX77620_REG_FPS_LDO0 0x46
+#define MAX77620_REG_FPS_LDO1 0x47
+#define MAX77620_REG_FPS_LDO2 0x48
+#define MAX77620_REG_FPS_LDO3 0x49
+#define MAX77620_REG_FPS_LDO4 0x4A
+#define MAX77620_REG_FPS_LDO5 0x4B
+#define MAX77620_REG_FPS_LDO6 0x4C
+#define MAX77620_REG_FPS_LDO7 0x4D
+#define MAX77620_REG_FPS_LDO8 0x4E
+#define MAX77620_REG_FPS_SD0 0x4F
+#define MAX77620_REG_FPS_SD1 0x50
+#define MAX77620_REG_FPS_SD2 0x51
+#define MAX77620_REG_FPS_SD3 0x52
+#define MAX77620_REG_FPS_SD4 0x53
+#define MAX77620_REG_FPS_NONE 0
+
+#define MAX77620_FPS_SRC_MASK 0xC0
+#define MAX77620_FPS_SRC_SHIFT 6
+#define MAX77620_FPS_PU_PERIOD_MASK 0x38
+#define MAX77620_FPS_PU_PERIOD_SHIFT 3
+#define MAX77620_FPS_PD_PERIOD_MASK 0x07
+#define MAX77620_FPS_PD_PERIOD_SHIFT 0
+#define MAX77620_FPS_TIME_PERIOD_MASK 0x38
+#define MAX77620_FPS_TIME_PERIOD_SHIFT 3
+#define MAX77620_FPS_EN_SRC_MASK 0x06
+#define MAX77620_FPS_EN_SRC_SHIFT 1
+#define MAX77620_FPS_ENFPS_SW_MASK 0x01
+#define MAX77620_FPS_ENFPS_SW 0x01
+
+/* Minimum and maximum FPS period time (in microseconds) are
+ * different for MAX77620 and Max20024.
+ */
+#define MAX77620_FPS_PERIOD_MIN_US 40
+#define MAX20024_FPS_PERIOD_MIN_US 20
+
+#define MAX77620_FPS_PERIOD_MAX_US 2560
+#define MAX20024_FPS_PERIOD_MAX_US 5120
+
+#define MAX77620_REG_FPS_GPIO1 0x54
+#define MAX77620_REG_FPS_GPIO2 0x55
+#define MAX77620_REG_FPS_GPIO3 0x56
+#define MAX77620_REG_FPS_RSO 0x57
+#define MAX77620_REG_CID0 0x58
+#define MAX77620_REG_CID1 0x59
+#define MAX77620_REG_CID2 0x5A
+#define MAX77620_REG_CID3 0x5B
+#define MAX77620_REG_CID4 0x5C
+#define MAX77620_REG_CID5 0x5D
+
+#define MAX77620_REG_DVSSD4 0x5E
+#define MAX20024_REG_MAX_ADD 0x70
+
+#define MAX77620_CID_DIDM_MASK 0xF0
+#define MAX77620_CID_DIDM_SHIFT 4
+
+/* CNCG2SD */
+#define MAX77620_SD_CNF2_ROVS_EN_SD1 BIT(1)
+#define MAX77620_SD_CNF2_ROVS_EN_SD0 BIT(2)
+
+/* Device Identification Metal */
+#define MAX77620_CID5_DIDM(n) (((n) >> 4) & 0xF)
+/* Device Indentification OTP */
+#define MAX77620_CID5_DIDO(n) ((n) & 0xF)
+
+/* SD CNFG1 */
+#define MAX77620_SD_SR_MASK 0xC0
+#define MAX77620_SD_SR_SHIFT 6
+#define MAX77620_SD_POWER_MODE_MASK 0x30
+#define MAX77620_SD_POWER_MODE_SHIFT 4
+#define MAX77620_SD_CFG1_ADE_MASK BIT(3)
+#define MAX77620_SD_CFG1_ADE_DISABLE 0
+#define MAX77620_SD_CFG1_ADE_ENABLE BIT(3)
+#define MAX77620_SD_FPWM_MASK 0x04
+#define MAX77620_SD_FPWM_SHIFT 2
+#define MAX77620_SD_FSRADE_MASK 0x01
+#define MAX77620_SD_FSRADE_SHIFT 0
+#define MAX77620_SD_CFG1_FPWM_SD_MASK BIT(2)
+#define MAX77620_SD_CFG1_FPWM_SD_SKIP 0
+#define MAX77620_SD_CFG1_FPWM_SD_FPWM BIT(2)
+#define MAX77620_SD_CFG1_FSRADE_SD_MASK BIT(0)
+#define MAX77620_SD_CFG1_FSRADE_SD_DISABLE 0
+#define MAX77620_SD_CFG1_FSRADE_SD_ENABLE BIT(0)
+
+/* LDO_CNFG2 */
+#define MAX77620_LDO_POWER_MODE_MASK 0xC0
+#define MAX77620_LDO_POWER_MODE_SHIFT 6
+#define MAX77620_LDO_CFG2_ADE_MASK BIT(1)
+#define MAX77620_LDO_CFG2_ADE_DISABLE 0
+#define MAX77620_LDO_CFG2_ADE_ENABLE BIT(1)
+#define MAX77620_LDO_CFG2_SS_MASK BIT(0)
+#define MAX77620_LDO_CFG2_SS_FAST BIT(0)
+#define MAX77620_LDO_CFG2_SS_SLOW 0
+
+#define MAX77620_IRQ_TOP_GLBL_MASK BIT(7)
+#define MAX77620_IRQ_TOP_SD_MASK BIT(6)
+#define MAX77620_IRQ_TOP_LDO_MASK BIT(5)
+#define MAX77620_IRQ_TOP_GPIO_MASK BIT(4)
+#define MAX77620_IRQ_TOP_RTC_MASK BIT(3)
+#define MAX77620_IRQ_TOP_32K_MASK BIT(2)
+#define MAX77620_IRQ_TOP_ONOFF_MASK BIT(1)
+
+#define MAX77620_IRQ_LBM_MASK BIT(3)
+#define MAX77620_IRQ_TJALRM1_MASK BIT(2)
+#define MAX77620_IRQ_TJALRM2_MASK BIT(1)
+
+#define MAX77620_PWR_I2C_ADDR 0x3c
+#define MAX77620_RTC_I2C_ADDR 0x68
+
+#define MAX77620_CNFG_GPIO_DRV_MASK BIT(0)
+#define MAX77620_CNFG_GPIO_DRV_PUSHPULL BIT(0)
+#define MAX77620_CNFG_GPIO_DRV_OPENDRAIN 0
+#define MAX77620_CNFG_GPIO_DIR_MASK BIT(1)
+#define MAX77620_CNFG_GPIO_DIR_INPUT BIT(1)
+#define MAX77620_CNFG_GPIO_DIR_OUTPUT 0
+#define MAX77620_CNFG_GPIO_INPUT_VAL_MASK BIT(2)
+#define MAX77620_CNFG_GPIO_OUTPUT_VAL_MASK BIT(3)
+#define MAX77620_CNFG_GPIO_OUTPUT_VAL_HIGH BIT(3)
+#define MAX77620_CNFG_GPIO_OUTPUT_VAL_LOW 0
+#define MAX77620_CNFG_GPIO_INT_MASK (0x3 << 4)
+#define MAX77620_CNFG_GPIO_INT_FALLING BIT(4)
+#define MAX77620_CNFG_GPIO_INT_RISING BIT(5)
+#define MAX77620_CNFG_GPIO_DBNC_MASK (0x3 << 6)
+#define MAX77620_CNFG_GPIO_DBNC_None (0x0 << 6)
+#define MAX77620_CNFG_GPIO_DBNC_8ms (0x1 << 6)
+#define MAX77620_CNFG_GPIO_DBNC_16ms (0x2 << 6)
+#define MAX77620_CNFG_GPIO_DBNC_32ms (0x3 << 6)
+
+#define MAX77620_IRQ_LVL2_GPIO_EDGE0 BIT(0)
+#define MAX77620_IRQ_LVL2_GPIO_EDGE1 BIT(1)
+#define MAX77620_IRQ_LVL2_GPIO_EDGE2 BIT(2)
+#define MAX77620_IRQ_LVL2_GPIO_EDGE3 BIT(3)
+#define MAX77620_IRQ_LVL2_GPIO_EDGE4 BIT(4)
+#define MAX77620_IRQ_LVL2_GPIO_EDGE5 BIT(5)
+#define MAX77620_IRQ_LVL2_GPIO_EDGE6 BIT(6)
+#define MAX77620_IRQ_LVL2_GPIO_EDGE7 BIT(7)
+
+#define MAX77620_CNFG1_32K_OUT0_EN BIT(2)
+
+#define MAX77620_ONOFFCNFG1_SFT_RST BIT(7)
+#define MAX77620_ONOFFCNFG1_MRT_MASK 0x38
+#define MAX77620_ONOFFCNFG1_MRT_SHIFT 0x3
+#define MAX77620_ONOFFCNFG1_SLPEN BIT(2)
+#define MAX77620_ONOFFCNFG1_PWR_OFF BIT(1)
+#define MAX20024_ONOFFCNFG1_CLRSE 0x18
+
+#define MAX77620_ONOFFCNFG2_SFT_RST_WK BIT(7)
+#define MAX77620_ONOFFCNFG2_WD_RST_WK BIT(6)
+#define MAX77620_ONOFFCNFG2_SLP_LPM_MSK BIT(5)
+#define MAX77620_ONOFFCNFG2_WK_ALARM1 BIT(2)
+#define MAX77620_ONOFFCNFG2_WK_EN0 BIT(0)
+
+#define MAX77620_GLBLM_MASK BIT(0)
+
+#define MAX77620_WDTC_MASK 0x3
+#define MAX77620_WDTOFFC BIT(4)
+#define MAX77620_WDTSLPC BIT(3)
+#define MAX77620_WDTEN BIT(2)
+
+#define MAX77620_TWD_MASK 0x3
+#define MAX77620_TWD_2s 0x0
+#define MAX77620_TWD_16s 0x1
+#define MAX77620_TWD_64s 0x2
+#define MAX77620_TWD_128s 0x3
+
+#define MAX77620_CNFGGLBL1_LBDAC_EN BIT(7)
+#define MAX77620_CNFGGLBL1_MPPLD BIT(6)
+#define MAX77620_CNFGGLBL1_LBHYST (BIT(5) | BIT(4))
+#define MAX77620_CNFGGLBL1_LBDAC 0x0E
+#define MAX77620_CNFGGLBL1_LBRSTEN BIT(0)
+
+/* CNFG BBC registers */
+#define MAX77620_CNFGBBC_ENABLE BIT(0)
+#define MAX77620_CNFGBBC_CURRENT_MASK 0x06
+#define MAX77620_CNFGBBC_CURRENT_SHIFT 1
+#define MAX77620_CNFGBBC_VOLTAGE_MASK 0x18
+#define MAX77620_CNFGBBC_VOLTAGE_SHIFT 3
+#define MAX77620_CNFGBBC_LOW_CURRENT_DISABLE BIT(5)
+#define MAX77620_CNFGBBC_RESISTOR_MASK 0xC0
+#define MAX77620_CNFGBBC_RESISTOR_SHIFT 6
+
+#define MAX77620_FPS_COUNT 3
+
+/* Interrupts */
+enum {
+ MAX77620_IRQ_TOP_GLBL, /* Low-Battery */
+ MAX77620_IRQ_TOP_SD, /* SD power fail */
+ MAX77620_IRQ_TOP_LDO, /* LDO power fail */
+ MAX77620_IRQ_TOP_GPIO, /* TOP GPIO internal int to MAX77620 */
+ MAX77620_IRQ_TOP_RTC, /* RTC */
+ MAX77620_IRQ_TOP_32K, /* 32kHz oscillator */
+ MAX77620_IRQ_TOP_ONOFF, /* ON/OFF oscillator */
+ MAX77620_IRQ_LBT_MBATLOW, /* Thermal alarm status, > 120C */
+ MAX77620_IRQ_LBT_TJALRM1, /* Thermal alarm status, > 120C */
+ MAX77620_IRQ_LBT_TJALRM2, /* Thermal alarm status, > 140C */
+};
+
+/* GPIOs */
+enum {
+ MAX77620_GPIO0,
+ MAX77620_GPIO1,
+ MAX77620_GPIO2,
+ MAX77620_GPIO3,
+ MAX77620_GPIO4,
+ MAX77620_GPIO5,
+ MAX77620_GPIO6,
+ MAX77620_GPIO7,
+ MAX77620_GPIO_NR,
+};
+
+/* FPS Source */
+enum max77620_fps_src {
+ MAX77620_FPS_SRC_0,
+ MAX77620_FPS_SRC_1,
+ MAX77620_FPS_SRC_2,
+ MAX77620_FPS_SRC_NONE,
+ MAX77620_FPS_SRC_DEF,
+};
+
+enum max77620_chip_id {
+ MAX77620,
+ MAX20024,
+};
+
+struct max77620_chip {
+ struct device *dev;
+ struct regmap *rmap;
+
+ int chip_irq;
+ int irq_base;
+
+ /* chip id */
+ enum max77620_chip_id chip_id;
+
+ bool sleep_enable;
+ bool enable_global_lpm;
+ int shutdown_fps_period[MAX77620_FPS_COUNT];
+ int suspend_fps_period[MAX77620_FPS_COUNT];
+
+ struct regmap_irq_chip_data *top_irq_data;
+ struct regmap_irq_chip_data *gpio_irq_data;
+};
+
+#endif /* _MFD_MAX77620_H_ */
diff --git a/include/linux/mfd/syscon.h b/include/linux/mfd/syscon.h
index 1088149..40a76b9 100644
--- a/include/linux/mfd/syscon.h
+++ b/include/linux/mfd/syscon.h
@@ -16,6 +16,7 @@
#define __LINUX_MFD_SYSCON_H__
#include <linux/err.h>
+#include <linux/errno.h>
struct device_node;
diff --git a/include/linux/mfd/syscon/exynos5-pmu.h b/include/linux/mfd/syscon/exynos5-pmu.h
index 9352adc..76f30f9 100644
--- a/include/linux/mfd/syscon/exynos5-pmu.h
+++ b/include/linux/mfd/syscon/exynos5-pmu.h
@@ -38,6 +38,9 @@
/* Exynos5433 specific register definitions */
#define EXYNOS5433_USBHOST30_PHY_CONTROL (0x728)
+#define EXYNOS5433_MIPI_PHY0_CONTROL (0x710)
+#define EXYNOS5433_MIPI_PHY1_CONTROL (0x714)
+#define EXYNOS5433_MIPI_PHY2_CONTROL (0x718)
#define EXYNOS5_PHY_ENABLE BIT(0)
diff --git a/include/linux/mfd/syscon/imx6q-iomuxc-gpr.h b/include/linux/mfd/syscon/imx6q-iomuxc-gpr.h
index 238c8db..c8e0164 100644
--- a/include/linux/mfd/syscon/imx6q-iomuxc-gpr.h
+++ b/include/linux/mfd/syscon/imx6q-iomuxc-gpr.h
@@ -95,6 +95,7 @@
#define IMX6Q_GPR0_DMAREQ_MUX_SEL0_IOMUX BIT(0)
#define IMX6Q_GPR1_PCIE_REQ_MASK (0x3 << 30)
+#define IMX6Q_GPR1_PCIE_SW_RST BIT(29)
#define IMX6Q_GPR1_PCIE_EXIT_L1 BIT(28)
#define IMX6Q_GPR1_PCIE_RDY_L23 BIT(27)
#define IMX6Q_GPR1_PCIE_ENTER_L1 BIT(26)
@@ -447,5 +448,11 @@
#define IMX6UL_GPR1_ENET2_CLK_OUTPUT (0x1 << 18)
#define IMX6UL_GPR1_ENET_CLK_DIR (0x3 << 17)
#define IMX6UL_GPR1_ENET_CLK_OUTPUT (0x3 << 17)
+#define IMX6UL_GPR1_SAI1_MCLK_DIR (0x1 << 19)
+#define IMX6UL_GPR1_SAI2_MCLK_DIR (0x1 << 20)
+#define IMX6UL_GPR1_SAI3_MCLK_DIR (0x1 << 21)
+#define IMX6UL_GPR1_SAI_MCLK_MASK (0x7 << 19)
+#define MCLK_DIR(x) (x == 1 ? IMX6UL_GPR1_SAI1_MCLK_DIR : x == 2 ? \
+ IMX6UL_GPR1_SAI2_MCLK_DIR : IMX6UL_GPR1_SAI3_MCLK_DIR)
#endif /* __LINUX_IMX6Q_IOMUXC_GPR_H */
diff --git a/include/linux/mfd/wm8400-private.h b/include/linux/mfd/wm8400-private.h
index 2de565b..4ee908f 100644
--- a/include/linux/mfd/wm8400-private.h
+++ b/include/linux/mfd/wm8400-private.h
@@ -923,7 +923,6 @@
#define WM8400_LINE_CMP_VTHD_SHIFT 0 /* LINE_CMP_VTHD - [3:0] */
#define WM8400_LINE_CMP_VTHD_WIDTH 4 /* LINE_CMP_VTHD - [3:0] */
-u16 wm8400_reg_read(struct wm8400 *wm8400, u8 reg);
int wm8400_block_read(struct wm8400 *wm8400, u8 reg, int count, u16 *data);
static inline int wm8400_set_bits(struct wm8400 *wm8400, u8 reg,
diff --git a/include/linux/mlx5/cq.h b/include/linux/mlx5/cq.h
index b2c9fad..2be976d 100644
--- a/include/linux/mlx5/cq.h
+++ b/include/linux/mlx5/cq.h
@@ -53,6 +53,11 @@
unsigned arm_sn;
struct mlx5_rsc_debug *dbg;
int pid;
+ struct {
+ struct list_head list;
+ void (*comp)(struct mlx5_core_cq *);
+ void *priv;
+ } tasklet_ctx;
};
diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h
index 07b504f..80776d0 100644
--- a/include/linux/mlx5/driver.h
+++ b/include/linux/mlx5/driver.h
@@ -42,6 +42,7 @@
#include <linux/vmalloc.h>
#include <linux/radix-tree.h>
#include <linux/workqueue.h>
+#include <linux/interrupt.h>
#include <linux/mlx5/device.h>
#include <linux/mlx5/doorbell.h>
@@ -312,6 +313,14 @@
u8 page_shift;
};
+struct mlx5_eq_tasklet {
+ struct list_head list;
+ struct list_head process_list;
+ struct tasklet_struct task;
+ /* lock on completion tasklet list */
+ spinlock_t lock;
+};
+
struct mlx5_eq {
struct mlx5_core_dev *dev;
__be32 __iomem *doorbell;
@@ -325,6 +334,7 @@
struct list_head list;
int index;
struct mlx5_rsc_debug *dbg;
+ struct mlx5_eq_tasklet tasklet_ctx;
};
struct mlx5_core_psv {
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 727f7997..2835d59 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -447,14 +447,14 @@
* On nommu, vmalloc/vfree wrap through kmalloc/kfree directly, so there
* is no special casing required.
*/
-static inline int is_vmalloc_addr(const void *x)
+static inline bool is_vmalloc_addr(const void *x)
{
#ifdef CONFIG_MMU
unsigned long addr = (unsigned long)x;
return addr >= VMALLOC_START && addr < VMALLOC_END;
#else
- return 0;
+ return false;
#endif
}
#ifdef CONFIG_MMU
@@ -475,8 +475,7 @@
static inline int compound_mapcount(struct page *page)
{
- if (!PageCompound(page))
- return 0;
+ VM_BUG_ON_PAGE(!PageCompound(page), page);
page = compound_head(page);
return atomic_read(compound_mapcount_ptr(page)) + 1;
}
@@ -597,7 +596,7 @@
}
void do_set_pte(struct vm_area_struct *vma, unsigned long address,
- struct page *page, pte_t *pte, bool write, bool anon);
+ struct page *page, pte_t *pte, bool write, bool anon, bool old);
#endif
/*
@@ -734,7 +733,7 @@
page = compound_head(page);
/*
* Getting a normal page or the head of a compound page
- * requires to already have an elevated page->_count.
+ * requires to already have an elevated page->_refcount.
*/
VM_BUG_ON_PAGE(page_ref_count(page) <= 0, page);
page_ref_inc(page);
@@ -850,10 +849,7 @@
static inline void page_cpupid_reset_last(struct page *page)
{
- int cpupid = (1 << LAST_CPUPID_SHIFT) - 1;
-
- page->flags &= ~(LAST_CPUPID_MASK << LAST_CPUPID_PGSHIFT);
- page->flags |= (cpupid & LAST_CPUPID_MASK) << LAST_CPUPID_PGSHIFT;
+ page->flags |= LAST_CPUPID_MASK << LAST_CPUPID_PGSHIFT;
}
#endif /* LAST_CPUPID_NOT_IN_PAGE_FLAGS */
#else /* !CONFIG_NUMA_BALANCING */
@@ -1032,26 +1028,7 @@
return page->index;
}
-/*
- * Return true if this page is mapped into pagetables.
- * For compound page it returns true if any subpage of compound page is mapped.
- */
-static inline bool page_mapped(struct page *page)
-{
- int i;
- if (likely(!PageCompound(page)))
- return atomic_read(&page->_mapcount) >= 0;
- page = compound_head(page);
- if (atomic_read(compound_mapcount_ptr(page)) >= 0)
- return true;
- if (PageHuge(page))
- return false;
- for (i = 0; i < hpage_nr_pages(page); i++) {
- if (atomic_read(&page[i]._mapcount) >= 0)
- return true;
- }
- return false;
-}
+bool page_mapped(struct page *page);
/*
* Return true only if the page has been allocated with
@@ -1786,7 +1763,7 @@
extern void adjust_managed_page_count(struct page *page, long count);
extern void mem_init_print_info(const char *str);
-extern void reserve_bootmem_region(unsigned long start, unsigned long end);
+extern void reserve_bootmem_region(phys_addr_t start, phys_addr_t end);
/* Free the reserved page into the buddy system, so it gets managed. */
static inline void __free_reserved_page(struct page *page)
@@ -2034,9 +2011,9 @@
#endif
/* These take the mm semaphore themselves */
-extern unsigned long vm_brk(unsigned long, unsigned long);
+extern unsigned long __must_check vm_brk(unsigned long, unsigned long);
extern int vm_munmap(unsigned long, size_t);
-extern unsigned long vm_mmap(struct file *, unsigned long,
+extern unsigned long __must_check vm_mmap(struct file *, unsigned long,
unsigned long, unsigned long,
unsigned long, unsigned long);
@@ -2409,6 +2386,9 @@
return false;
page_ext = lookup_page_ext(page);
+ if (unlikely(!page_ext))
+ return false;
+
return test_bit(PAGE_EXT_DEBUG_GUARD, &page_ext->flags);
}
#else
diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h
index 712e8c3..5bd29ba 100644
--- a/include/linux/mm_inline.h
+++ b/include/linux/mm_inline.h
@@ -22,22 +22,34 @@
return !PageSwapBacked(page);
}
+static __always_inline void __update_lru_size(struct lruvec *lruvec,
+ enum lru_list lru, int nr_pages)
+{
+ __mod_zone_page_state(lruvec_zone(lruvec), NR_LRU_BASE + lru, nr_pages);
+}
+
+static __always_inline void update_lru_size(struct lruvec *lruvec,
+ enum lru_list lru, int nr_pages)
+{
+#ifdef CONFIG_MEMCG
+ mem_cgroup_update_lru_size(lruvec, lru, nr_pages);
+#else
+ __update_lru_size(lruvec, lru, nr_pages);
+#endif
+}
+
static __always_inline void add_page_to_lru_list(struct page *page,
struct lruvec *lruvec, enum lru_list lru)
{
- int nr_pages = hpage_nr_pages(page);
- mem_cgroup_update_lru_size(lruvec, lru, nr_pages);
+ update_lru_size(lruvec, lru, hpage_nr_pages(page));
list_add(&page->lru, &lruvec->lists[lru]);
- __mod_zone_page_state(lruvec_zone(lruvec), NR_LRU_BASE + lru, nr_pages);
}
static __always_inline void del_page_from_lru_list(struct page *page,
struct lruvec *lruvec, enum lru_list lru)
{
- int nr_pages = hpage_nr_pages(page);
- mem_cgroup_update_lru_size(lruvec, lru, -nr_pages);
list_del(&page->lru);
- __mod_zone_page_state(lruvec_zone(lruvec), NR_LRU_BASE + lru, -nr_pages);
+ update_lru_size(lruvec, lru, -hpage_nr_pages(page));
}
/**
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index c2d75b4..d553855 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -12,6 +12,7 @@
#include <linux/cpumask.h>
#include <linux/uprobes.h>
#include <linux/page-flags-layout.h>
+#include <linux/workqueue.h>
#include <asm/page.h>
#include <asm/mmu.h>
@@ -73,9 +74,9 @@
unsigned long counters;
#else
/*
- * Keep _count separate from slub cmpxchg_double data.
- * As the rest of the double word is protected by
- * slab_lock but _count is not.
+ * Keep _refcount separate from slub cmpxchg_double
+ * data. As the rest of the double word is protected by
+ * slab_lock but _refcount is not.
*/
unsigned counters;
#endif
@@ -97,7 +98,11 @@
};
int units; /* SLOB */
};
- atomic_t _count; /* Usage count, see below. */
+ /*
+ * Usage count, *USE WRAPPER FUNCTION*
+ * when manual accounting. See page_ref.h
+ */
+ atomic_t _refcount;
};
unsigned int active; /* SLAB */
};
@@ -248,7 +253,7 @@
__u32 offset;
#endif
/* we maintain a pagecount bias, so that we dont dirty cache line
- * containing page->_count every time we allocate a fragment.
+ * containing page->_refcount every time we allocate a fragment.
*/
unsigned int pagecnt_bias;
bool pfmemalloc;
@@ -509,6 +514,7 @@
#ifdef CONFIG_HUGETLB_PAGE
atomic_long_t hugetlb_usage;
#endif
+ struct work_struct async_put_work;
};
static inline void mm_init_cpumask(struct mm_struct *mm)
diff --git a/include/linux/mmc/sdio_ids.h b/include/linux/mmc/sdio_ids.h
index 83430f2..0d126ae 100644
--- a/include/linux/mmc/sdio_ids.h
+++ b/include/linux/mmc/sdio_ids.h
@@ -36,6 +36,7 @@
#define SDIO_DEVICE_ID_BROADCOM_43430 0xa9a6
#define SDIO_DEVICE_ID_BROADCOM_4345 0x4345
#define SDIO_DEVICE_ID_BROADCOM_4354 0x4354
+#define SDIO_DEVICE_ID_BROADCOM_4356 0x4356
#define SDIO_VENDOR_ID_INTEL 0x0089
#define SDIO_DEVICE_ID_INTEL_IWMC3200WIMAX 0x1402
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index c60df92..02069c2 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -85,13 +85,6 @@
get_pfnblock_flags_mask(page, page_to_pfn(page), \
PB_migrate_end, MIGRATETYPE_MASK)
-static inline int get_pfnblock_migratetype(struct page *page, unsigned long pfn)
-{
- BUILD_BUG_ON(PB_migrate_end - PB_migrate != 2);
- return get_pfnblock_flags_mask(page, pfn, PB_migrate_end,
- MIGRATETYPE_MASK);
-}
-
struct free_area {
struct list_head free_list[MIGRATE_TYPES];
unsigned long nr_free;
@@ -746,8 +739,12 @@
extern struct mutex zonelists_mutex;
void build_all_zonelists(pg_data_t *pgdat, struct zone *zone);
void wakeup_kswapd(struct zone *zone, int order, enum zone_type classzone_idx);
+bool __zone_watermark_ok(struct zone *z, unsigned int order, unsigned long mark,
+ int classzone_idx, unsigned int alloc_flags,
+ long free_pages);
bool zone_watermark_ok(struct zone *z, unsigned int order,
- unsigned long mark, int classzone_idx, int alloc_flags);
+ unsigned long mark, int classzone_idx,
+ unsigned int alloc_flags);
bool zone_watermark_ok_safe(struct zone *z, unsigned int order,
unsigned long mark, int classzone_idx);
enum memmap_context {
@@ -828,10 +825,7 @@
static inline int is_highmem(struct zone *zone)
{
#ifdef CONFIG_HIGHMEM
- int zone_off = (char *)zone - (char *)zone->zone_pgdat->node_zones;
- return zone_off == ZONE_HIGHMEM * sizeof(*zone) ||
- (zone_off == ZONE_MOVABLE * sizeof(*zone) &&
- zone_movable_is_highmem());
+ return is_highmem_idx(zone_idx(zone));
#else
return 0;
#endif
@@ -922,6 +916,10 @@
#endif /* CONFIG_NUMA */
}
+struct zoneref *__next_zones_zonelist(struct zoneref *z,
+ enum zone_type highest_zoneidx,
+ nodemask_t *nodes);
+
/**
* next_zones_zonelist - Returns the next zone at or below highest_zoneidx within the allowed nodemask using a cursor within a zonelist as a starting point
* @z - The cursor used as a starting point for the search
@@ -934,9 +932,14 @@
* being examined. It should be advanced by one before calling
* next_zones_zonelist again.
*/
-struct zoneref *next_zones_zonelist(struct zoneref *z,
+static __always_inline struct zoneref *next_zones_zonelist(struct zoneref *z,
enum zone_type highest_zoneidx,
- nodemask_t *nodes);
+ nodemask_t *nodes)
+{
+ if (likely(!nodes && zonelist_zone_idx(z) <= highest_zoneidx))
+ return z;
+ return __next_zones_zonelist(z, highest_zoneidx, nodes);
+}
/**
* first_zones_zonelist - Returns the first zone at or below highest_zoneidx within the allowed nodemask in a zonelist
@@ -952,13 +955,10 @@
*/
static inline struct zoneref *first_zones_zonelist(struct zonelist *zonelist,
enum zone_type highest_zoneidx,
- nodemask_t *nodes,
- struct zone **zone)
+ nodemask_t *nodes)
{
- struct zoneref *z = next_zones_zonelist(zonelist->_zonerefs,
+ return next_zones_zonelist(zonelist->_zonerefs,
highest_zoneidx, nodes);
- *zone = zonelist_zone(z);
- return z;
}
/**
@@ -973,10 +973,17 @@
* within a given nodemask
*/
#define for_each_zone_zonelist_nodemask(zone, z, zlist, highidx, nodemask) \
- for (z = first_zones_zonelist(zlist, highidx, nodemask, &zone); \
+ for (z = first_zones_zonelist(zlist, highidx, nodemask), zone = zonelist_zone(z); \
zone; \
z = next_zones_zonelist(++z, highidx, nodemask), \
- zone = zonelist_zone(z)) \
+ zone = zonelist_zone(z))
+
+#define for_next_zone_zonelist_nodemask(zone, z, zlist, highidx, nodemask) \
+ for (zone = z->zone; \
+ zone; \
+ z = next_zones_zonelist(++z, highidx, nodemask), \
+ zone = zonelist_zone(z))
+
/**
* for_each_zone_zonelist - helper macro to iterate over valid zones in a zonelist at or below a given zone index
@@ -1056,7 +1063,7 @@
unsigned long *pageblock_flags;
#ifdef CONFIG_PAGE_EXTENSION
/*
- * If !SPARSEMEM, pgdat doesn't have page_ext pointer. We use
+ * If SPARSEMEM, pgdat doesn't have page_ext pointer. We use
* section. (see page_ext.h about this.)
*/
struct page_ext *page_ext;
diff --git a/include/linux/mtd/fsmc.h b/include/linux/mtd/fsmc.h
index c8be32e..ad3c348 100644
--- a/include/linux/mtd/fsmc.h
+++ b/include/linux/mtd/fsmc.h
@@ -103,24 +103,6 @@
#define FSMC_BUSY_WAIT_TIMEOUT (1 * HZ)
-/*
- * There are 13 bytes of ecc for every 512 byte block in FSMC version 8
- * and it has to be read consecutively and immediately after the 512
- * byte data block for hardware to generate the error bit offsets
- * Managing the ecc bytes in the following way is easier. This way is
- * similar to oobfree structure maintained already in u-boot nand driver
- */
-#define MAX_ECCPLACE_ENTRIES 32
-
-struct fsmc_nand_eccplace {
- uint8_t offset;
- uint8_t length;
-};
-
-struct fsmc_eccplace {
- struct fsmc_nand_eccplace eccplace[MAX_ECCPLACE_ENTRIES];
-};
-
struct fsmc_nand_timings {
uint8_t tclr;
uint8_t tar;
diff --git a/include/linux/mtd/map.h b/include/linux/mtd/map.h
index 5e0eb7c..3aa56e3 100644
--- a/include/linux/mtd/map.h
+++ b/include/linux/mtd/map.h
@@ -122,18 +122,13 @@
#endif
#ifdef CONFIG_MTD_MAP_BANK_WIDTH_32
-# ifdef map_bankwidth
-# undef map_bankwidth
-# define map_bankwidth(map) ((map)->bankwidth)
-# undef map_bankwidth_is_large
-# define map_bankwidth_is_large(map) (map_bankwidth(map) > BITS_PER_LONG/8)
-# undef map_words
-# define map_words(map) map_calc_words(map)
-# else
-# define map_bankwidth(map) 32
-# define map_bankwidth_is_large(map) (1)
-# define map_words(map) map_calc_words(map)
-# endif
+/* always use indirect access for 256-bit to preserve kernel stack */
+# undef map_bankwidth
+# define map_bankwidth(map) ((map)->bankwidth)
+# undef map_bankwidth_is_large
+# define map_bankwidth_is_large(map) (map_bankwidth(map) > BITS_PER_LONG/8)
+# undef map_words
+# define map_words(map) map_calc_words(map)
#define map_bankwidth_is_32(map) (map_bankwidth(map) == 32)
#undef MAX_MAP_BANKWIDTH
#define MAX_MAP_BANKWIDTH 32
diff --git a/include/linux/mtd/mtd.h b/include/linux/mtd/mtd.h
index ef9fea4..29a1706 100644
--- a/include/linux/mtd/mtd.h
+++ b/include/linux/mtd/mtd.h
@@ -96,16 +96,35 @@
#define MTD_MAX_OOBFREE_ENTRIES_LARGE 32
#define MTD_MAX_ECCPOS_ENTRIES_LARGE 640
-/*
- * Internal ECC layout control structure. For historical reasons, there is a
- * similar, smaller struct nand_ecclayout_user (in mtd-abi.h) that is retained
- * for export to user-space via the ECCGETLAYOUT ioctl.
- * nand_ecclayout should be expandable in the future simply by the above macros.
+/**
+ * struct mtd_oob_region - oob region definition
+ * @offset: region offset
+ * @length: region length
+ *
+ * This structure describes a region of the OOB area, and is used
+ * to retrieve ECC or free bytes sections.
+ * Each section is defined by an offset within the OOB area and a
+ * length.
*/
-struct nand_ecclayout {
- __u32 eccbytes;
- __u32 eccpos[MTD_MAX_ECCPOS_ENTRIES_LARGE];
- struct nand_oobfree oobfree[MTD_MAX_OOBFREE_ENTRIES_LARGE];
+struct mtd_oob_region {
+ u32 offset;
+ u32 length;
+};
+
+/*
+ * struct mtd_ooblayout_ops - NAND OOB layout operations
+ * @ecc: function returning an ECC region in the OOB area.
+ * Should return -ERANGE if %section exceeds the total number of
+ * ECC sections.
+ * @free: function returning a free region in the OOB area.
+ * Should return -ERANGE if %section exceeds the total number of
+ * free sections.
+ */
+struct mtd_ooblayout_ops {
+ int (*ecc)(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobecc);
+ int (*free)(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobfree);
};
struct module; /* only needed for owner field in mtd_info */
@@ -166,8 +185,8 @@
const char *name;
int index;
- /* ECC layout structure pointer - read only! */
- struct nand_ecclayout *ecclayout;
+ /* OOB layout description */
+ const struct mtd_ooblayout_ops *ooblayout;
/* the ecc step size. */
unsigned int ecc_step_size;
@@ -253,6 +272,30 @@
int usecount;
};
+int mtd_ooblayout_ecc(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobecc);
+int mtd_ooblayout_find_eccregion(struct mtd_info *mtd, int eccbyte,
+ int *section,
+ struct mtd_oob_region *oobregion);
+int mtd_ooblayout_get_eccbytes(struct mtd_info *mtd, u8 *eccbuf,
+ const u8 *oobbuf, int start, int nbytes);
+int mtd_ooblayout_set_eccbytes(struct mtd_info *mtd, const u8 *eccbuf,
+ u8 *oobbuf, int start, int nbytes);
+int mtd_ooblayout_free(struct mtd_info *mtd, int section,
+ struct mtd_oob_region *oobfree);
+int mtd_ooblayout_get_databytes(struct mtd_info *mtd, u8 *databuf,
+ const u8 *oobbuf, int start, int nbytes);
+int mtd_ooblayout_set_databytes(struct mtd_info *mtd, const u8 *databuf,
+ u8 *oobbuf, int start, int nbytes);
+int mtd_ooblayout_count_freebytes(struct mtd_info *mtd);
+int mtd_ooblayout_count_eccbytes(struct mtd_info *mtd);
+
+static inline void mtd_set_ooblayout(struct mtd_info *mtd,
+ const struct mtd_ooblayout_ops *ooblayout)
+{
+ mtd->ooblayout = ooblayout;
+}
+
static inline void mtd_set_of_node(struct mtd_info *mtd,
struct device_node *np)
{
diff --git a/include/linux/mtd/nand.h b/include/linux/mtd/nand.h
index 56574ba..fbe8e16 100644
--- a/include/linux/mtd/nand.h
+++ b/include/linux/mtd/nand.h
@@ -116,9 +116,14 @@
NAND_ECC_HW,
NAND_ECC_HW_SYNDROME,
NAND_ECC_HW_OOB_FIRST,
- NAND_ECC_SOFT_BCH,
} nand_ecc_modes_t;
+enum nand_ecc_algo {
+ NAND_ECC_UNKNOWN,
+ NAND_ECC_HAMMING,
+ NAND_ECC_BCH,
+};
+
/*
* Constants for Hardware ECC
*/
@@ -458,6 +463,7 @@
/**
* struct nand_ecc_ctrl - Control structure for ECC
* @mode: ECC mode
+ * @algo: ECC algorithm
* @steps: number of ECC steps per page
* @size: data bytes per ECC step
* @bytes: ECC bytes per step
@@ -466,7 +472,6 @@
* @prepad: padding information for syndrome based ECC generators
* @postpad: padding information for syndrome based ECC generators
* @options: ECC specific options (see NAND_ECC_XXX flags defined above)
- * @layout: ECC layout control struct pointer
* @priv: pointer to private ECC control data
* @hwctl: function to control hardware ECC generator. Must only
* be provided if an hardware ECC is available
@@ -508,6 +513,7 @@
*/
struct nand_ecc_ctrl {
nand_ecc_modes_t mode;
+ enum nand_ecc_algo algo;
int steps;
int size;
int bytes;
@@ -516,7 +522,6 @@
int prepad;
int postpad;
unsigned int options;
- struct nand_ecclayout *layout;
void *priv;
void (*hwctl)(struct mtd_info *mtd, int mode);
int (*calculate)(struct mtd_info *mtd, const uint8_t *dat,
@@ -740,6 +745,9 @@
void *priv;
};
+extern const struct mtd_ooblayout_ops nand_ooblayout_sp_ops;
+extern const struct mtd_ooblayout_ops nand_ooblayout_lp_ops;
+
static inline void nand_set_flash_node(struct nand_chip *chip,
struct device_node *np)
{
@@ -1070,4 +1078,18 @@
void *ecc, int ecclen,
void *extraoob, int extraooblen,
int threshold);
+
+/* Default write_oob implementation */
+int nand_write_oob_std(struct mtd_info *mtd, struct nand_chip *chip, int page);
+
+/* Default write_oob syndrome implementation */
+int nand_write_oob_syndrome(struct mtd_info *mtd, struct nand_chip *chip,
+ int page);
+
+/* Default read_oob implementation */
+int nand_read_oob_std(struct mtd_info *mtd, struct nand_chip *chip, int page);
+
+/* Default read_oob syndrome implementation */
+int nand_read_oob_syndrome(struct mtd_info *mtd, struct nand_chip *chip,
+ int page);
#endif /* __LINUX_MTD_NAND_H */
diff --git a/include/linux/mtd/onenand.h b/include/linux/mtd/onenand.h
index 4596503..0aaa98b 100644
--- a/include/linux/mtd/onenand.h
+++ b/include/linux/mtd/onenand.h
@@ -80,7 +80,6 @@
* @page_buf: [INTERN] page main data buffer
* @oob_buf: [INTERN] page oob data buffer
* @subpagesize: [INTERN] holds the subpagesize
- * @ecclayout: [REPLACEABLE] the default ecc placement scheme
* @bbm: [REPLACEABLE] pointer to Bad Block Management
* @priv: [OPTIONAL] pointer to private chip date
*/
@@ -134,7 +133,6 @@
#endif
int subpagesize;
- struct nand_ecclayout *ecclayout;
void *bbm;
diff --git a/include/linux/mtd/sharpsl.h b/include/linux/mtd/sharpsl.h
index 25f4d2a..65e91d0 100644
--- a/include/linux/mtd/sharpsl.h
+++ b/include/linux/mtd/sharpsl.h
@@ -14,7 +14,7 @@
struct sharpsl_nand_platform_data {
struct nand_bbt_descr *badblock_pattern;
- struct nand_ecclayout *ecc_layout;
+ const struct mtd_ooblayout_ops *ecc_layout;
struct mtd_partition *partitions;
unsigned int nr_partitions;
};
diff --git a/include/linux/mtd/spi-nor.h b/include/linux/mtd/spi-nor.h
index 3c36113..7f041bd 100644
--- a/include/linux/mtd/spi-nor.h
+++ b/include/linux/mtd/spi-nor.h
@@ -21,6 +21,7 @@
* Sometimes these are the same as CFI IDs, but sometimes they aren't.
*/
#define SNOR_MFR_ATMEL CFI_MFR_ATMEL
+#define SNOR_MFR_GIGADEVICE 0xc8
#define SNOR_MFR_INTEL CFI_MFR_INTEL
#define SNOR_MFR_MICRON CFI_MFR_ST /* ST Micro <--> Micron */
#define SNOR_MFR_MACRONIX CFI_MFR_MACRONIX
diff --git a/include/linux/nd.h b/include/linux/nd.h
index 5489ab7..aee2761 100644
--- a/include/linux/nd.h
+++ b/include/linux/nd.h
@@ -15,6 +15,7 @@
#include <linux/fs.h>
#include <linux/ndctl.h>
#include <linux/device.h>
+#include <linux/badblocks.h>
enum nvdimm_event {
NVDIMM_REVALIDATE_POISON,
@@ -55,13 +56,19 @@
}
/**
- * struct nd_namespace_io - infrastructure for loading an nd_pmem instance
+ * struct nd_namespace_io - device representation of a persistent memory range
* @dev: namespace device created by the nd region driver
* @res: struct resource conversion of a NFIT SPA table
+ * @size: cached resource_size(@res) for fast path size checks
+ * @addr: virtual address to access the namespace range
+ * @bb: badblocks list for the namespace range
*/
struct nd_namespace_io {
struct nd_namespace_common common;
struct resource res;
+ resource_size_t size;
+ void __pmem *addr;
+ struct badblocks bb;
};
/**
@@ -82,6 +89,7 @@
* @uuid: namespace name supplied in the dimm label
* @id: ida allocated id
* @lbasize: blk namespaces have a native sector size when btt not present
+ * @size: sum of all the resource ranges allocated to this namespace
* @num_resources: number of dpa extents to claim
* @res: discontiguous dpa extents for given dimm
*/
@@ -91,6 +99,7 @@
u8 *uuid;
int id;
unsigned long lbasize;
+ resource_size_t size;
int num_resources;
struct resource **res;
};
diff --git a/include/linux/netdev_features.h b/include/linux/netdev_features.h
index bc87362..aa7b240 100644
--- a/include/linux/netdev_features.h
+++ b/include/linux/netdev_features.h
@@ -44,8 +44,8 @@
NETIF_F_FSO_BIT, /* ... FCoE segmentation */
NETIF_F_GSO_GRE_BIT, /* ... GRE with TSO */
NETIF_F_GSO_GRE_CSUM_BIT, /* ... GRE with csum with TSO */
- NETIF_F_GSO_IPIP_BIT, /* ... IPIP tunnel with TSO */
- NETIF_F_GSO_SIT_BIT, /* ... SIT tunnel with TSO */
+ NETIF_F_GSO_IPXIP4_BIT, /* ... IP4 or IP6 over IP4 with TSO */
+ NETIF_F_GSO_IPXIP6_BIT, /* ... IP4 or IP6 over IP6 with TSO */
NETIF_F_GSO_UDP_TUNNEL_BIT, /* ... UDP TUNNEL with TSO */
NETIF_F_GSO_UDP_TUNNEL_CSUM_BIT,/* ... UDP TUNNEL with TSO & CSUM */
NETIF_F_GSO_PARTIAL_BIT, /* ... Only segment inner-most L4
@@ -121,8 +121,8 @@
#define NETIF_F_RXALL __NETIF_F(RXALL)
#define NETIF_F_GSO_GRE __NETIF_F(GSO_GRE)
#define NETIF_F_GSO_GRE_CSUM __NETIF_F(GSO_GRE_CSUM)
-#define NETIF_F_GSO_IPIP __NETIF_F(GSO_IPIP)
-#define NETIF_F_GSO_SIT __NETIF_F(GSO_SIT)
+#define NETIF_F_GSO_IPXIP4 __NETIF_F(GSO_IPXIP4)
+#define NETIF_F_GSO_IPXIP6 __NETIF_F(GSO_IPXIP6)
#define NETIF_F_GSO_UDP_TUNNEL __NETIF_F(GSO_UDP_TUNNEL)
#define NETIF_F_GSO_UDP_TUNNEL_CSUM __NETIF_F(GSO_UDP_TUNNEL_CSUM)
#define NETIF_F_TSO_MANGLEID __NETIF_F(TSO_MANGLEID)
@@ -200,8 +200,8 @@
#define NETIF_F_GSO_ENCAP_ALL (NETIF_F_GSO_GRE | \
NETIF_F_GSO_GRE_CSUM | \
- NETIF_F_GSO_IPIP | \
- NETIF_F_GSO_SIT | \
+ NETIF_F_GSO_IPXIP4 | \
+ NETIF_F_GSO_IPXIP6 | \
NETIF_F_GSO_UDP_TUNNEL | \
NETIF_F_GSO_UDP_TUNNEL_CSUM)
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index c148edf..f45929c 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -4006,8 +4006,8 @@
BUILD_BUG_ON(SKB_GSO_FCOE != (NETIF_F_FSO >> NETIF_F_GSO_SHIFT));
BUILD_BUG_ON(SKB_GSO_GRE != (NETIF_F_GSO_GRE >> NETIF_F_GSO_SHIFT));
BUILD_BUG_ON(SKB_GSO_GRE_CSUM != (NETIF_F_GSO_GRE_CSUM >> NETIF_F_GSO_SHIFT));
- BUILD_BUG_ON(SKB_GSO_IPIP != (NETIF_F_GSO_IPIP >> NETIF_F_GSO_SHIFT));
- BUILD_BUG_ON(SKB_GSO_SIT != (NETIF_F_GSO_SIT >> NETIF_F_GSO_SHIFT));
+ BUILD_BUG_ON(SKB_GSO_IPXIP4 != (NETIF_F_GSO_IPXIP4 >> NETIF_F_GSO_SHIFT));
+ BUILD_BUG_ON(SKB_GSO_IPXIP6 != (NETIF_F_GSO_IPXIP6 >> NETIF_F_GSO_SHIFT));
BUILD_BUG_ON(SKB_GSO_UDP_TUNNEL != (NETIF_F_GSO_UDP_TUNNEL >> NETIF_F_GSO_SHIFT));
BUILD_BUG_ON(SKB_GSO_UDP_TUNNEL_CSUM != (NETIF_F_GSO_UDP_TUNNEL_CSUM >> NETIF_F_GSO_SHIFT));
BUILD_BUG_ON(SKB_GSO_PARTIAL != (NETIF_F_GSO_PARTIAL >> NETIF_F_GSO_SHIFT));
diff --git a/include/linux/nilfs2_fs.h b/include/linux/nilfs2_fs.h
index e9fcf90..5988dd5 100644
--- a/include/linux/nilfs2_fs.h
+++ b/include/linux/nilfs2_fs.h
@@ -13,12 +13,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
- * You should have received a copy of the GNU Lesser General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- *
- * Written by Koji Sato <koji@osrg.net>
- * Ryusuke Konishi <ryusuke@osrg.net>
+ * Written by Koji Sato and Ryusuke Konishi.
*/
/*
* linux/include/linux/ext2_fs.h
@@ -132,10 +127,14 @@
#define NILFS_MOUNT_ERRORS_RO 0x0020 /* Remount fs ro on errors */
#define NILFS_MOUNT_ERRORS_PANIC 0x0040 /* Panic on errors */
#define NILFS_MOUNT_BARRIER 0x1000 /* Use block barriers */
-#define NILFS_MOUNT_STRICT_ORDER 0x2000 /* Apply strict in-order
- semantics also for data */
-#define NILFS_MOUNT_NORECOVERY 0x4000 /* Disable write access during
- mount-time recovery */
+#define NILFS_MOUNT_STRICT_ORDER 0x2000 /*
+ * Apply strict in-order
+ * semantics also for data
+ */
+#define NILFS_MOUNT_NORECOVERY 0x4000 /*
+ * Disable write access during
+ * mount-time recovery
+ */
#define NILFS_MOUNT_DISCARD 0x8000 /* Issue DISCARD requests */
@@ -147,16 +146,20 @@
__le16 s_minor_rev_level; /* minor revision level */
__le16 s_magic; /* Magic signature */
- __le16 s_bytes; /* Bytes count of CRC calculation
- for this structure. s_reserved
- is excluded. */
+ __le16 s_bytes; /*
+ * Bytes count of CRC calculation
+ * for this structure. s_reserved
+ * is excluded.
+ */
__le16 s_flags; /* flags */
__le32 s_crc_seed; /* Seed value of CRC calculation */
/*10*/ __le32 s_sum; /* Check sum of super block */
- __le32 s_log_block_size; /* Block size represented as follows
- blocksize =
- 1 << (s_log_block_size + 10) */
+ __le32 s_log_block_size; /*
+ * Block size represented as follows
+ * blocksize =
+ * 1 << (s_log_block_size + 10)
+ */
__le64 s_nsegments; /* Number of segments in filesystem */
/*20*/ __le64 s_dev_size; /* block device size in bytes */
__le64 s_first_data_block; /* 1st seg disk block number */
@@ -168,8 +171,10 @@
__le64 s_last_seq; /* seq. number of seg written last */
/*50*/ __le64 s_free_blocks_count; /* Free blocks count */
- __le64 s_ctime; /* Creation time (execution time of
- newfs) */
+ __le64 s_ctime; /*
+ * Creation time (execution time of
+ * newfs)
+ */
/*60*/ __le64 s_mtime; /* Mount time */
__le64 s_wtime; /* Write time */
/*70*/ __le16 s_mnt_count; /* Mount count */
@@ -193,8 +198,10 @@
/*A8*/ char s_volume_name[80]; /* volume name */
/*F8*/ __le32 s_c_interval; /* Commit interval of segment */
- __le32 s_c_block_max; /* Threshold of data amount for
- the segment construction */
+ __le32 s_c_block_max; /*
+ * Threshold of data amount for
+ * the segment construction
+ */
/*100*/ __le64 s_feature_compat; /* Compatible feature set */
__le64 s_feature_compat_ro; /* Read-only compatible feature set */
__le64 s_feature_incompat; /* Incompatible feature set */
@@ -247,12 +254,18 @@
#define NILFS_SB_OFFSET_BYTES 1024 /* byte offset of nilfs superblock */
-#define NILFS_SEG_MIN_BLOCKS 16 /* Minimum number of blocks in
- a full segment */
-#define NILFS_PSEG_MIN_BLOCKS 2 /* Minimum number of blocks in
- a partial segment */
-#define NILFS_MIN_NRSVSEGS 8 /* Minimum number of reserved
- segments */
+#define NILFS_SEG_MIN_BLOCKS 16 /*
+ * Minimum number of blocks in
+ * a full segment
+ */
+#define NILFS_PSEG_MIN_BLOCKS 2 /*
+ * Minimum number of blocks in
+ * a partial segment
+ */
+#define NILFS_MIN_NRSVSEGS 8 /*
+ * Minimum number of reserved
+ * segments
+ */
/*
* We call DAT, cpfile, and sufile root metadata files. Inodes of
@@ -327,9 +340,9 @@
~NILFS_DIR_ROUND)
#define NILFS_MAX_REC_LEN ((1<<16)-1)
-static inline unsigned nilfs_rec_len_from_disk(__le16 dlen)
+static inline unsigned int nilfs_rec_len_from_disk(__le16 dlen)
{
- unsigned len = le16_to_cpu(dlen);
+ unsigned int len = le16_to_cpu(dlen);
#if !defined(__KERNEL__) || (PAGE_SIZE >= 65536)
if (len == NILFS_MAX_REC_LEN)
@@ -338,7 +351,7 @@
return len;
}
-static inline __le16 nilfs_rec_len_to_disk(unsigned len)
+static inline __le16 nilfs_rec_len_to_disk(unsigned int len)
{
#if !defined(__KERNEL__) || (PAGE_SIZE >= 65536)
if (len == (1 << 16))
@@ -518,9 +531,11 @@
__le64 cp_inodes_count;
__le64 cp_blocks_count;
- /* Do not change the byte offset of ifile inode.
- To keep the compatibility of the disk format,
- additional fields should be added behind cp_ifile_inode. */
+ /*
+ * Do not change the byte offset of ifile inode.
+ * To keep the compatibility of the disk format,
+ * additional fields should be added behind cp_ifile_inode.
+ */
struct nilfs_inode cp_ifile_inode;
};
diff --git a/include/linux/nodemask.h b/include/linux/nodemask.h
index 6e85889..f746e44 100644
--- a/include/linux/nodemask.h
+++ b/include/linux/nodemask.h
@@ -43,8 +43,10 @@
*
* int first_node(mask) Number lowest set bit, or MAX_NUMNODES
* int next_node(node, mask) Next node past 'node', or MAX_NUMNODES
+ * int next_node_in(node, mask) Next node past 'node', or wrap to first,
+ * or MAX_NUMNODES
* int first_unset_node(mask) First node not set in mask, or
- * MAX_NUMNODES.
+ * MAX_NUMNODES
*
* nodemask_t nodemask_of_node(node) Return nodemask with bit 'node' set
* NODE_MASK_ALL Initializer - all bits set
@@ -259,6 +261,13 @@
return min_t(int,MAX_NUMNODES,find_next_bit(srcp->bits, MAX_NUMNODES, n+1));
}
+/*
+ * Find the next present node in src, starting after node n, wrapping around to
+ * the first node in src if needed. Returns MAX_NUMNODES if src is empty.
+ */
+#define next_node_in(n, src) __next_node_in((n), &(src))
+int __next_node_in(int node, const nodemask_t *srcp);
+
static inline void init_nodemask_of_node(nodemask_t *mask, int node)
{
nodes_clear(*mask);
diff --git a/include/linux/nvmem-provider.h b/include/linux/nvmem-provider.h
index a4fcc90..cd93416 100644
--- a/include/linux/nvmem-provider.h
+++ b/include/linux/nvmem-provider.h
@@ -14,6 +14,10 @@
struct nvmem_device;
struct nvmem_cell_info;
+typedef int (*nvmem_reg_read_t)(void *priv, unsigned int offset,
+ void *val, size_t bytes);
+typedef int (*nvmem_reg_write_t)(void *priv, unsigned int offset,
+ void *val, size_t bytes);
struct nvmem_config {
struct device *dev;
@@ -24,6 +28,12 @@
int ncells;
bool read_only;
bool root_only;
+ nvmem_reg_read_t reg_read;
+ nvmem_reg_write_t reg_write;
+ int size;
+ int word_size;
+ int stride;
+ void *priv;
/* To be only used by old driver/misc/eeprom drivers */
bool compat;
struct device *base_dev;
diff --git a/include/linux/of.h b/include/linux/of.h
index 77ddace..c7292e8 100644
--- a/include/linux/of.h
+++ b/include/linux/of.h
@@ -75,6 +75,23 @@
uint32_t args[MAX_PHANDLE_ARGS];
};
+struct of_phandle_iterator {
+ /* Common iterator information */
+ const char *cells_name;
+ int cell_count;
+ const struct device_node *parent;
+
+ /* List size information */
+ const __be32 *list_end;
+ const __be32 *phandle_end;
+
+ /* Current position state */
+ const __be32 *cur;
+ uint32_t cur_count;
+ phandle phandle;
+ struct device_node *node;
+};
+
struct of_reconfig_data {
struct device_node *dn;
struct property *prop;
@@ -334,6 +351,18 @@
extern int of_count_phandle_with_args(const struct device_node *np,
const char *list_name, const char *cells_name);
+/* phandle iterator functions */
+extern int of_phandle_iterator_init(struct of_phandle_iterator *it,
+ const struct device_node *np,
+ const char *list_name,
+ const char *cells_name,
+ int cell_count);
+
+extern int of_phandle_iterator_next(struct of_phandle_iterator *it);
+extern int of_phandle_iterator_args(struct of_phandle_iterator *it,
+ uint32_t *args,
+ int size);
+
extern void of_alias_scan(void * (*dt_alloc)(u64 size, u64 align));
extern int of_alias_get_id(struct device_node *np, const char *stem);
extern int of_alias_get_highest_id(const char *stem);
@@ -608,6 +637,27 @@
return -ENOSYS;
}
+static inline int of_phandle_iterator_init(struct of_phandle_iterator *it,
+ const struct device_node *np,
+ const char *list_name,
+ const char *cells_name,
+ int cell_count)
+{
+ return -ENOSYS;
+}
+
+static inline int of_phandle_iterator_next(struct of_phandle_iterator *it)
+{
+ return -ENOSYS;
+}
+
+static inline int of_phandle_iterator_args(struct of_phandle_iterator *it,
+ uint32_t *args,
+ int size)
+{
+ return 0;
+}
+
static inline int of_alias_get_id(struct device_node *np, const char *stem)
{
return -ENOSYS;
@@ -877,6 +927,12 @@
return of_property_read_u32(np, propname, (u32*) out_value);
}
+#define of_for_each_phandle(it, err, np, ln, cn, cc) \
+ for (of_phandle_iterator_init((it), (np), (ln), (cn), (cc)), \
+ err = of_phandle_iterator_next(it); \
+ err == 0; \
+ err = of_phandle_iterator_next(it))
+
#define of_property_for_each_u32(np, propname, prop, p, u) \
for (prop = of_find_property(np, propname, NULL), \
p = of_prop_next_u32(prop, NULL, &u); \
diff --git a/include/linux/of_address.h b/include/linux/of_address.h
index 01c0a55..3786473 100644
--- a/include/linux/of_address.h
+++ b/include/linux/of_address.h
@@ -47,10 +47,6 @@
extern const __be32 *of_get_address(struct device_node *dev, int index,
u64 *size, unsigned int *flags);
-extern int pci_register_io_range(phys_addr_t addr, resource_size_t size);
-extern unsigned long pci_address_to_pio(phys_addr_t addr);
-extern phys_addr_t pci_pio_to_address(unsigned long pio);
-
extern int of_pci_range_parser_init(struct of_pci_range_parser *parser,
struct device_node *node);
extern struct of_pci_range *of_pci_range_parser_one(
@@ -86,11 +82,6 @@
return NULL;
}
-static inline phys_addr_t pci_pio_to_address(unsigned long pio)
-{
- return 0;
-}
-
static inline int of_pci_range_parser_init(struct of_pci_range_parser *parser,
struct device_node *node)
{
diff --git a/include/linux/of_fdt.h b/include/linux/of_fdt.h
index 2fbe868..901ec01 100644
--- a/include/linux/of_fdt.h
+++ b/include/linux/of_fdt.h
@@ -37,8 +37,9 @@
unsigned long node);
extern int of_fdt_match(const void *blob, unsigned long node,
const char *const *compat);
-extern void of_fdt_unflatten_tree(const unsigned long *blob,
- struct device_node **mynodes);
+extern void *of_fdt_unflatten_tree(const unsigned long *blob,
+ struct device_node *dad,
+ struct device_node **mynodes);
/* TBD: Temporary export of fdt globals - remove when code fully merged */
extern int __initdata dt_root_addr_cells;
diff --git a/include/linux/of_graph.h b/include/linux/of_graph.h
index f8bcd0e..bb3a5a2 100644
--- a/include/linux/of_graph.h
+++ b/include/linux/of_graph.h
@@ -15,6 +15,7 @@
#define __LINUX_OF_GRAPH_H
#include <linux/types.h>
+#include <linux/errno.h>
/**
* struct of_endpoint - the OF graph endpoint data structure
diff --git a/include/linux/of_iommu.h b/include/linux/of_iommu.h
index ffbe470..bd02b44 100644
--- a/include/linux/of_iommu.h
+++ b/include/linux/of_iommu.h
@@ -12,7 +12,7 @@
size_t *size);
extern void of_iommu_init(void);
-extern struct iommu_ops *of_iommu_configure(struct device *dev,
+extern const struct iommu_ops *of_iommu_configure(struct device *dev,
struct device_node *master_np);
#else
@@ -25,7 +25,7 @@
}
static inline void of_iommu_init(void) { }
-static inline struct iommu_ops *of_iommu_configure(struct device *dev,
+static inline const struct iommu_ops *of_iommu_configure(struct device *dev,
struct device_node *master_np)
{
return NULL;
@@ -33,8 +33,8 @@
#endif /* CONFIG_OF_IOMMU */
-void of_iommu_set_ops(struct device_node *np, struct iommu_ops *ops);
-struct iommu_ops *of_iommu_get_ops(struct device_node *np);
+void of_iommu_set_ops(struct device_node *np, const struct iommu_ops *ops);
+const struct iommu_ops *of_iommu_get_ops(struct device_node *np);
extern struct of_device_id __iommu_of_table;
diff --git a/include/linux/of_mtd.h b/include/linux/of_mtd.h
deleted file mode 100644
index e266caa..0000000
--- a/include/linux/of_mtd.h
+++ /dev/null
@@ -1,50 +0,0 @@
-/*
- * Copyright 2012 Jean-Christophe PLAGNIOL-VILLARD <plagnioj@jcrosoft.com>
- *
- * OF helpers for mtd.
- *
- * This file is released under the GPLv2
- */
-
-#ifndef __LINUX_OF_MTD_H
-#define __LINUX_OF_MTD_H
-
-#ifdef CONFIG_OF_MTD
-
-#include <linux/of.h>
-int of_get_nand_ecc_mode(struct device_node *np);
-int of_get_nand_ecc_step_size(struct device_node *np);
-int of_get_nand_ecc_strength(struct device_node *np);
-int of_get_nand_bus_width(struct device_node *np);
-bool of_get_nand_on_flash_bbt(struct device_node *np);
-
-#else /* CONFIG_OF_MTD */
-
-static inline int of_get_nand_ecc_mode(struct device_node *np)
-{
- return -ENOSYS;
-}
-
-static inline int of_get_nand_ecc_step_size(struct device_node *np)
-{
- return -ENOSYS;
-}
-
-static inline int of_get_nand_ecc_strength(struct device_node *np)
-{
- return -ENOSYS;
-}
-
-static inline int of_get_nand_bus_width(struct device_node *np)
-{
- return -ENOSYS;
-}
-
-static inline bool of_get_nand_on_flash_bbt(struct device_node *np)
-{
- return false;
-}
-
-#endif /* CONFIG_OF_MTD */
-
-#endif /* __LINUX_OF_MTD_H */
diff --git a/include/linux/omap-gpmc.h b/include/linux/omap-gpmc.h
index d833eb4..9e9d79e 100644
--- a/include/linux/omap-gpmc.h
+++ b/include/linux/omap-gpmc.h
@@ -7,161 +7,53 @@
* option) any later version.
*/
-/* Maximum Number of Chip Selects */
-#define GPMC_CS_NUM 8
+#include <linux/platform_data/gpmc-omap.h>
#define GPMC_CONFIG_WP 0x00000005
-#define GPMC_IRQ_FIFOEVENTENABLE 0x01
-#define GPMC_IRQ_COUNT_EVENT 0x02
+/* IRQ numbers in GPMC IRQ domain for legacy boot use */
+#define GPMC_IRQ_FIFOEVENTENABLE 0
+#define GPMC_IRQ_COUNT_EVENT 1
-#define GPMC_BURST_4 4 /* 4 word burst */
-#define GPMC_BURST_8 8 /* 8 word burst */
-#define GPMC_BURST_16 16 /* 16 word burst */
-#define GPMC_DEVWIDTH_8BIT 1 /* 8-bit device width */
-#define GPMC_DEVWIDTH_16BIT 2 /* 16-bit device width */
-#define GPMC_MUX_AAD 1 /* Addr-Addr-Data multiplex */
-#define GPMC_MUX_AD 2 /* Addr-Data multiplex */
-
-/* bool type time settings */
-struct gpmc_bool_timings {
- bool cycle2cyclediffcsen;
- bool cycle2cyclesamecsen;
- bool we_extra_delay;
- bool oe_extra_delay;
- bool adv_extra_delay;
- bool cs_extra_delay;
- bool time_para_granularity;
-};
-
-/*
- * Note that all values in this struct are in nanoseconds except sync_clk
- * (which is in picoseconds), while the register values are in gpmc_fck cycles.
+/**
+ * gpmc_nand_ops - Interface between NAND and GPMC
+ * @nand_write_buffer_empty: get the NAND write buffer empty status.
*/
-struct gpmc_timings {
- /* Minimum clock period for synchronous mode (in picoseconds) */
- u32 sync_clk;
-
- /* Chip-select signal timings corresponding to GPMC_CS_CONFIG2 */
- u32 cs_on; /* Assertion time */
- u32 cs_rd_off; /* Read deassertion time */
- u32 cs_wr_off; /* Write deassertion time */
-
- /* ADV signal timings corresponding to GPMC_CONFIG3 */
- u32 adv_on; /* Assertion time */
- u32 adv_rd_off; /* Read deassertion time */
- u32 adv_wr_off; /* Write deassertion time */
- u32 adv_aad_mux_on; /* ADV assertion time for AAD */
- u32 adv_aad_mux_rd_off; /* ADV read deassertion time for AAD */
- u32 adv_aad_mux_wr_off; /* ADV write deassertion time for AAD */
-
- /* WE signals timings corresponding to GPMC_CONFIG4 */
- u32 we_on; /* WE assertion time */
- u32 we_off; /* WE deassertion time */
-
- /* OE signals timings corresponding to GPMC_CONFIG4 */
- u32 oe_on; /* OE assertion time */
- u32 oe_off; /* OE deassertion time */
- u32 oe_aad_mux_on; /* OE assertion time for AAD */
- u32 oe_aad_mux_off; /* OE deassertion time for AAD */
-
- /* Access time and cycle time timings corresponding to GPMC_CONFIG5 */
- u32 page_burst_access; /* Multiple access word delay */
- u32 access; /* Start-cycle to first data valid delay */
- u32 rd_cycle; /* Total read cycle time */
- u32 wr_cycle; /* Total write cycle time */
-
- u32 bus_turnaround;
- u32 cycle2cycle_delay;
-
- u32 wait_monitoring;
- u32 clk_activation;
-
- /* The following are only on OMAP3430 */
- u32 wr_access; /* WRACCESSTIME */
- u32 wr_data_mux_bus; /* WRDATAONADMUXBUS */
-
- struct gpmc_bool_timings bool_timings;
+struct gpmc_nand_ops {
+ bool (*nand_writebuffer_empty)(void);
};
-/* Device timings in picoseconds */
-struct gpmc_device_timings {
- u32 t_ceasu; /* address setup to CS valid */
- u32 t_avdasu; /* address setup to ADV valid */
- /* XXX: try to combine t_avdp_r & t_avdp_w. Issue is
- * of tusb using these timings even for sync whilst
- * ideally for adv_rd/(wr)_off it should have considered
- * t_avdh instead. This indirectly necessitates r/w
- * variations of t_avdp as it is possible to have one
- * sync & other async
- */
- u32 t_avdp_r; /* ADV low time (what about t_cer ?) */
- u32 t_avdp_w;
- u32 t_aavdh; /* address hold time */
- u32 t_oeasu; /* address setup to OE valid */
- u32 t_aa; /* access time from ADV assertion */
- u32 t_iaa; /* initial access time */
- u32 t_oe; /* access time from OE assertion */
- u32 t_ce; /* access time from CS asertion */
- u32 t_rd_cycle; /* read cycle time */
- u32 t_cez_r; /* read CS deassertion to high Z */
- u32 t_cez_w; /* write CS deassertion to high Z */
- u32 t_oez; /* OE deassertion to high Z */
- u32 t_weasu; /* address setup to WE valid */
- u32 t_wpl; /* write assertion time */
- u32 t_wph; /* write deassertion time */
- u32 t_wr_cycle; /* write cycle time */
+struct gpmc_nand_regs;
- u32 clk;
- u32 t_bacc; /* burst access valid clock to output delay */
- u32 t_ces; /* CS setup time to clk */
- u32 t_avds; /* ADV setup time to clk */
- u32 t_avdh; /* ADV hold time from clk */
- u32 t_ach; /* address hold time from clk */
- u32 t_rdyo; /* clk to ready valid */
+#if IS_ENABLED(CONFIG_OMAP_GPMC)
+struct gpmc_nand_ops *gpmc_omap_get_nand_ops(struct gpmc_nand_regs *regs,
+ int cs);
+#else
+static inline gpmc_nand_ops *gpmc_omap_get_nand_ops(struct gpmc_nand_regs *regs,
+ int cs)
+{
+ return NULL;
+}
+#endif /* CONFIG_OMAP_GPMC */
- u32 t_ce_rdyz; /* XXX: description ?, or use t_cez instead */
- u32 t_ce_avd; /* CS on to ADV on delay */
+/*--------------------------------*/
- /* XXX: check the possibility of combining
- * cyc_aavhd_oe & cyc_aavdh_we
- */
- u8 cyc_aavdh_oe;/* read address hold time in cycles */
- u8 cyc_aavdh_we;/* write address hold time in cycles */
- u8 cyc_oe; /* access time from OE assertion in cycles */
- u8 cyc_wpl; /* write deassertion time in cycles */
- u32 cyc_iaa; /* initial access time in cycles */
-
- /* extra delays */
- bool ce_xdelay;
- bool avd_xdelay;
- bool oe_xdelay;
- bool we_xdelay;
-};
-
-struct gpmc_settings {
- bool burst_wrap; /* enables wrap bursting */
- bool burst_read; /* enables read page/burst mode */
- bool burst_write; /* enables write page/burst mode */
- bool device_nand; /* device is NAND */
- bool sync_read; /* enables synchronous reads */
- bool sync_write; /* enables synchronous writes */
- bool wait_on_read; /* monitor wait on reads */
- bool wait_on_write; /* monitor wait on writes */
- u32 burst_len; /* page/burst length */
- u32 device_width; /* device bus width (8 or 16 bit) */
- u32 mux_add_data; /* multiplex address & data */
- u32 wait_pin; /* wait-pin to be used */
-};
+/* deprecated APIs */
+#if IS_ENABLED(CONFIG_OMAP_GPMC)
+void gpmc_update_nand_reg(struct gpmc_nand_regs *reg, int cs);
+#else
+static inline void gpmc_update_nand_reg(struct gpmc_nand_regs *reg, int cs)
+{
+}
+#endif /* CONFIG_OMAP_GPMC */
+/*--------------------------------*/
extern int gpmc_calc_timings(struct gpmc_timings *gpmc_t,
struct gpmc_settings *gpmc_s,
struct gpmc_device_timings *dev_t);
-struct gpmc_nand_regs;
struct device_node;
-extern void gpmc_update_nand_reg(struct gpmc_nand_regs *reg, int cs);
extern int gpmc_get_client_irq(unsigned irq_config);
extern unsigned int gpmc_ticks_to_ns(unsigned int ticks);
diff --git a/include/linux/omap-mailbox.h b/include/linux/omap-mailbox.h
index 587bbdd..c726bd8 100644
--- a/include/linux/omap-mailbox.h
+++ b/include/linux/omap-mailbox.h
@@ -21,8 +21,6 @@
struct mbox_chan *omap_mbox_request_channel(struct mbox_client *cl,
const char *chan_name);
-void omap_mbox_save_ctx(struct mbox_chan *chan);
-void omap_mbox_restore_ctx(struct mbox_chan *chan);
void omap_mbox_enable_irq(struct mbox_chan *chan, omap_mbox_irq_t irq);
void omap_mbox_disable_irq(struct mbox_chan *chan, omap_mbox_irq_t irq);
diff --git a/include/linux/oom.h b/include/linux/oom.h
index 628a432..8346952 100644
--- a/include/linux/oom.h
+++ b/include/linux/oom.h
@@ -50,28 +50,33 @@
OOM_SCAN_SELECT, /* always select this thread first */
};
-/* Thread is the potential origin of an oom condition; kill first on oom */
-#define OOM_FLAG_ORIGIN ((__force oom_flags_t)0x1)
-
extern struct mutex oom_lock;
static inline void set_current_oom_origin(void)
{
- current->signal->oom_flags |= OOM_FLAG_ORIGIN;
+ current->signal->oom_flag_origin = true;
}
static inline void clear_current_oom_origin(void)
{
- current->signal->oom_flags &= ~OOM_FLAG_ORIGIN;
+ current->signal->oom_flag_origin = false;
}
static inline bool oom_task_origin(const struct task_struct *p)
{
- return !!(p->signal->oom_flags & OOM_FLAG_ORIGIN);
+ return p->signal->oom_flag_origin;
}
extern void mark_oom_victim(struct task_struct *tsk);
+#ifdef CONFIG_MMU
+extern void try_oom_reaper(struct task_struct *tsk);
+#else
+static inline void try_oom_reaper(struct task_struct *tsk)
+{
+}
+#endif
+
extern unsigned long oom_badness(struct task_struct *p,
struct mem_cgroup *memcg, const nodemask_t *nodemask,
unsigned long totalpages);
@@ -102,13 +107,24 @@
static inline bool task_will_free_mem(struct task_struct *task)
{
+ struct signal_struct *sig = task->signal;
+
/*
* A coredumping process may sleep for an extended period in exit_mm(),
* so the oom killer cannot assume that the process will promptly exit
* and release memory.
*/
- return (task->flags & PF_EXITING) &&
- !(task->signal->flags & SIGNAL_GROUP_COREDUMP);
+ if (sig->flags & SIGNAL_GROUP_COREDUMP)
+ return false;
+
+ if (!(task->flags & PF_EXITING))
+ return false;
+
+ /* Make sure that the whole thread group is going down */
+ if (!thread_group_empty(task) && !(sig->flags & SIGNAL_GROUP_EXIT))
+ return false;
+
+ return true;
}
/* sysctls */
diff --git a/include/linux/padata.h b/include/linux/padata.h
index 4386946..113ee62 100644
--- a/include/linux/padata.h
+++ b/include/linux/padata.h
@@ -175,11 +175,6 @@
extern void padata_do_serial(struct padata_priv *padata);
extern int padata_set_cpumask(struct padata_instance *pinst, int cpumask_type,
cpumask_var_t cpumask);
-extern int padata_set_cpumasks(struct padata_instance *pinst,
- cpumask_var_t pcpumask,
- cpumask_var_t cbcpumask);
-extern int padata_add_cpu(struct padata_instance *pinst, int cpu, int mask);
-extern int padata_remove_cpu(struct padata_instance *pinst, int cpu, int mask);
extern int padata_start(struct padata_instance *pinst);
extern void padata_stop(struct padata_instance *pinst);
extern int padata_register_cpumask_notifier(struct padata_instance *pinst,
diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
index 6b052aa..e5a3244 100644
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -371,10 +371,15 @@
#define PAGE_MAPPING_KSM 2
#define PAGE_MAPPING_FLAGS (PAGE_MAPPING_ANON | PAGE_MAPPING_KSM)
+static __always_inline int PageAnonHead(struct page *page)
+{
+ return ((unsigned long)page->mapping & PAGE_MAPPING_ANON) != 0;
+}
+
static __always_inline int PageAnon(struct page *page)
{
page = compound_head(page);
- return ((unsigned long)page->mapping & PAGE_MAPPING_ANON) != 0;
+ return PageAnonHead(page);
}
#ifdef CONFIG_KSM
@@ -474,7 +479,7 @@
}
#endif
-#define PG_head_mask ((1L << PG_head))
+#define PG_head_mask ((1UL << PG_head))
#ifdef CONFIG_HUGETLB_PAGE
int PageHuge(struct page *page);
@@ -665,7 +670,7 @@
}
#ifdef CONFIG_MMU
-#define __PG_MLOCKED (1 << PG_mlocked)
+#define __PG_MLOCKED (1UL << PG_mlocked)
#else
#define __PG_MLOCKED 0
#endif
@@ -675,11 +680,11 @@
* these flags set. It they are, there is a problem.
*/
#define PAGE_FLAGS_CHECK_AT_FREE \
- (1 << PG_lru | 1 << PG_locked | \
- 1 << PG_private | 1 << PG_private_2 | \
- 1 << PG_writeback | 1 << PG_reserved | \
- 1 << PG_slab | 1 << PG_swapcache | 1 << PG_active | \
- 1 << PG_unevictable | __PG_MLOCKED)
+ (1UL << PG_lru | 1UL << PG_locked | \
+ 1UL << PG_private | 1UL << PG_private_2 | \
+ 1UL << PG_writeback | 1UL << PG_reserved | \
+ 1UL << PG_slab | 1UL << PG_swapcache | 1UL << PG_active | \
+ 1UL << PG_unevictable | __PG_MLOCKED)
/*
* Flags checked when a page is prepped for return by the page allocator.
@@ -690,10 +695,10 @@
* alloc-free cycle to prevent from reusing the page.
*/
#define PAGE_FLAGS_CHECK_AT_PREP \
- (((1 << NR_PAGEFLAGS) - 1) & ~__PG_HWPOISON)
+ (((1UL << NR_PAGEFLAGS) - 1) & ~__PG_HWPOISON)
#define PAGE_FLAGS_PRIVATE \
- (1 << PG_private | 1 << PG_private_2)
+ (1UL << PG_private | 1UL << PG_private_2)
/**
* page_has_private - Determine if page has private stuff
* @page: The page to be checked
diff --git a/include/linux/page_ref.h b/include/linux/page_ref.h
index e596d5d9..8b5e0a9 100644
--- a/include/linux/page_ref.h
+++ b/include/linux/page_ref.h
@@ -63,17 +63,17 @@
static inline int page_ref_count(struct page *page)
{
- return atomic_read(&page->_count);
+ return atomic_read(&page->_refcount);
}
static inline int page_count(struct page *page)
{
- return atomic_read(&compound_head(page)->_count);
+ return atomic_read(&compound_head(page)->_refcount);
}
static inline void set_page_count(struct page *page, int v)
{
- atomic_set(&page->_count, v);
+ atomic_set(&page->_refcount, v);
if (page_ref_tracepoint_active(__tracepoint_page_ref_set))
__page_ref_set(page, v);
}
@@ -89,35 +89,35 @@
static inline void page_ref_add(struct page *page, int nr)
{
- atomic_add(nr, &page->_count);
+ atomic_add(nr, &page->_refcount);
if (page_ref_tracepoint_active(__tracepoint_page_ref_mod))
__page_ref_mod(page, nr);
}
static inline void page_ref_sub(struct page *page, int nr)
{
- atomic_sub(nr, &page->_count);
+ atomic_sub(nr, &page->_refcount);
if (page_ref_tracepoint_active(__tracepoint_page_ref_mod))
__page_ref_mod(page, -nr);
}
static inline void page_ref_inc(struct page *page)
{
- atomic_inc(&page->_count);
+ atomic_inc(&page->_refcount);
if (page_ref_tracepoint_active(__tracepoint_page_ref_mod))
__page_ref_mod(page, 1);
}
static inline void page_ref_dec(struct page *page)
{
- atomic_dec(&page->_count);
+ atomic_dec(&page->_refcount);
if (page_ref_tracepoint_active(__tracepoint_page_ref_mod))
__page_ref_mod(page, -1);
}
static inline int page_ref_sub_and_test(struct page *page, int nr)
{
- int ret = atomic_sub_and_test(nr, &page->_count);
+ int ret = atomic_sub_and_test(nr, &page->_refcount);
if (page_ref_tracepoint_active(__tracepoint_page_ref_mod_and_test))
__page_ref_mod_and_test(page, -nr, ret);
@@ -126,7 +126,7 @@
static inline int page_ref_dec_and_test(struct page *page)
{
- int ret = atomic_dec_and_test(&page->_count);
+ int ret = atomic_dec_and_test(&page->_refcount);
if (page_ref_tracepoint_active(__tracepoint_page_ref_mod_and_test))
__page_ref_mod_and_test(page, -1, ret);
@@ -135,7 +135,7 @@
static inline int page_ref_dec_return(struct page *page)
{
- int ret = atomic_dec_return(&page->_count);
+ int ret = atomic_dec_return(&page->_refcount);
if (page_ref_tracepoint_active(__tracepoint_page_ref_mod_and_return))
__page_ref_mod_and_return(page, -1, ret);
@@ -144,7 +144,7 @@
static inline int page_ref_add_unless(struct page *page, int nr, int u)
{
- int ret = atomic_add_unless(&page->_count, nr, u);
+ int ret = atomic_add_unless(&page->_refcount, nr, u);
if (page_ref_tracepoint_active(__tracepoint_page_ref_mod_unless))
__page_ref_mod_unless(page, nr, ret);
@@ -153,7 +153,7 @@
static inline int page_ref_freeze(struct page *page, int count)
{
- int ret = likely(atomic_cmpxchg(&page->_count, count, 0) == count);
+ int ret = likely(atomic_cmpxchg(&page->_refcount, count, 0) == count);
if (page_ref_tracepoint_active(__tracepoint_page_ref_freeze))
__page_ref_freeze(page, count, ret);
@@ -165,7 +165,7 @@
VM_BUG_ON_PAGE(page_count(page) != 0, page);
VM_BUG_ON(count == 0);
- atomic_set(&page->_count, count);
+ atomic_set(&page->_refcount, count);
if (page_ref_tracepoint_active(__tracepoint_page_ref_unfreeze))
__page_ref_unfreeze(page, count);
}
diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index 7e1ab15..9735410 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -90,12 +90,12 @@
/*
* speculatively take a reference to a page.
- * If the page is free (_count == 0), then _count is untouched, and 0
- * is returned. Otherwise, _count is incremented by 1 and 1 is returned.
+ * If the page is free (_refcount == 0), then _refcount is untouched, and 0
+ * is returned. Otherwise, _refcount is incremented by 1 and 1 is returned.
*
* This function must be called inside the same rcu_read_lock() section as has
* been used to lookup the page in the pagecache radix-tree (or page table):
- * this allows allocators to use a synchronize_rcu() to stabilize _count.
+ * this allows allocators to use a synchronize_rcu() to stabilize _refcount.
*
* Unless an RCU grace period has passed, the count of all pages coming out
* of the allocator must be considered unstable. page_count may return higher
@@ -111,7 +111,7 @@
* 2. conditionally increment refcount
* 3. check the page is still in pagecache (if no, goto 1)
*
- * Remove-side that cares about stability of _count (eg. reclaim) has the
+ * Remove-side that cares about stability of _refcount (eg. reclaim) has the
* following (with tree_lock held for write):
* A. atomically check refcount is correct and set it to 0 (atomic_cmpxchg)
* B. remove page from pagecache
@@ -518,33 +518,27 @@
extern void add_page_wait_queue(struct page *page, wait_queue_t *waiter);
/*
- * Fault a userspace page into pagetables. Return non-zero on a fault.
- *
- * This assumes that two userspace pages are always sufficient.
+ * Fault one or two userspace pages into pagetables.
+ * Return -EINVAL if more than two pages would be needed.
+ * Return non-zero on a fault.
*/
static inline int fault_in_pages_writeable(char __user *uaddr, int size)
{
- int ret;
+ int span, ret;
if (unlikely(size == 0))
return 0;
+ span = offset_in_page(uaddr) + size;
+ if (span > 2 * PAGE_SIZE)
+ return -EINVAL;
/*
* Writing zeroes into userspace here is OK, because we know that if
* the zero gets there, we'll be overwriting it.
*/
ret = __put_user(0, uaddr);
- if (ret == 0) {
- char __user *end = uaddr + size - 1;
-
- /*
- * If the page was already mapped, this will get a cache miss
- * for sure, so try to avoid doing it.
- */
- if (((unsigned long)uaddr & PAGE_MASK) !=
- ((unsigned long)end & PAGE_MASK))
- ret = __put_user(0, end);
- }
+ if (ret == 0 && span > PAGE_SIZE)
+ ret = __put_user(0, uaddr + size - 1);
return ret;
}
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 932ec74..b67e4df 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -166,8 +166,6 @@
PCI_DEV_FLAGS_ASSIGNED = (__force pci_dev_flags_t) (1 << 2),
/* Flag for quirk use to store if quirk-specific ACS is enabled */
PCI_DEV_FLAGS_ACS_ENABLED_QUIRK = (__force pci_dev_flags_t) (1 << 3),
- /* Flag to indicate the device uses dma_alias_devfn */
- PCI_DEV_FLAGS_DMA_ALIAS_DEVFN = (__force pci_dev_flags_t) (1 << 4),
/* Use a PCIe-to-PCI bridge alias even if !pci_is_pcie */
PCI_DEV_FLAG_PCIE_BRIDGE_ALIAS = (__force pci_dev_flags_t) (1 << 5),
/* Do not use bus resets for device */
@@ -273,7 +271,7 @@
u8 rom_base_reg; /* which config register controls the ROM */
u8 pin; /* which interrupt pin this device uses */
u16 pcie_flags_reg; /* cached PCIe Capabilities Register */
- u8 dma_alias_devfn;/* devfn of DMA alias, if any */
+ unsigned long *dma_alias_mask;/* mask of enabled devfn aliases */
struct pci_driver *driver; /* which driver has allocated this device */
u64 dma_mask; /* Mask of the bits of bus address this
@@ -1165,6 +1163,9 @@
void *alignf_data);
+int pci_register_io_range(phys_addr_t addr, resource_size_t size);
+unsigned long pci_address_to_pio(phys_addr_t addr);
+phys_addr_t pci_pio_to_address(unsigned long pio);
int pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr);
static inline pci_bus_addr_t pci_bus_address(struct pci_dev *pdev, int bar)
@@ -1481,6 +1482,8 @@
{ return -EIO; }
static inline void pci_release_regions(struct pci_dev *dev) { }
+static inline unsigned long pci_address_to_pio(phys_addr_t addr) { return -1; }
+
static inline void pci_block_cfg_access(struct pci_dev *dev) { }
static inline int pci_block_cfg_access_in_atomic(struct pci_dev *dev)
{ return 0; }
@@ -1664,7 +1667,7 @@
#ifdef CONFIG_PCI_QUIRKS
void pci_fixup_device(enum pci_fixup_pass pass, struct pci_dev *dev);
int pci_dev_specific_acs_enabled(struct pci_dev *dev, u16 acs_flags);
-void pci_dev_specific_enable_acs(struct pci_dev *dev);
+int pci_dev_specific_enable_acs(struct pci_dev *dev);
#else
static inline void pci_fixup_device(enum pci_fixup_pass pass,
struct pci_dev *dev) { }
@@ -1673,7 +1676,10 @@
{
return -ENOTTY;
}
-static inline void pci_dev_specific_enable_acs(struct pci_dev *dev) { }
+static inline int pci_dev_specific_enable_acs(struct pci_dev *dev)
+{
+ return -ENOTTY;
+}
#endif
void __iomem *pcim_iomap(struct pci_dev *pdev, int bar, unsigned long maxlen);
@@ -1989,6 +1995,8 @@
}
#endif
+void pci_add_dma_alias(struct pci_dev *dev, u8 devfn);
+bool pci_devs_are_dma_aliases(struct pci_dev *dev1, struct pci_dev *dev2);
int pci_for_each_dma_alias(struct pci_dev *pdev,
int (*fn)(struct pci_dev *pdev,
u16 alias, void *data), void *data);
diff --git a/include/linux/pci_ids.h b/include/linux/pci_ids.h
index 247da8c..c58752f 100644
--- a/include/linux/pci_ids.h
+++ b/include/linux/pci_ids.h
@@ -2604,6 +2604,24 @@
#define PCI_DEVICE_ID_INTEL_82441 0x1237
#define PCI_DEVICE_ID_INTEL_82380FB 0x124b
#define PCI_DEVICE_ID_INTEL_82439 0x1250
+#define PCI_DEVICE_ID_INTEL_LIGHT_RIDGE 0x1513 /* Tbt 1 Gen 1 */
+#define PCI_DEVICE_ID_INTEL_EAGLE_RIDGE 0x151a
+#define PCI_DEVICE_ID_INTEL_LIGHT_PEAK 0x151b
+#define PCI_DEVICE_ID_INTEL_CACTUS_RIDGE_4C 0x1547 /* Tbt 1 Gen 2 */
+#define PCI_DEVICE_ID_INTEL_CACTUS_RIDGE_2C 0x1548
+#define PCI_DEVICE_ID_INTEL_PORT_RIDGE 0x1549
+#define PCI_DEVICE_ID_INTEL_REDWOOD_RIDGE_2C_NHI 0x1566 /* Tbt 1 Gen 3 */
+#define PCI_DEVICE_ID_INTEL_REDWOOD_RIDGE_2C_BRIDGE 0x1567
+#define PCI_DEVICE_ID_INTEL_REDWOOD_RIDGE_4C_NHI 0x1568
+#define PCI_DEVICE_ID_INTEL_REDWOOD_RIDGE_4C_BRIDGE 0x1569
+#define PCI_DEVICE_ID_INTEL_FALCON_RIDGE_2C_NHI 0x156a /* Thunderbolt 2 */
+#define PCI_DEVICE_ID_INTEL_FALCON_RIDGE_2C_BRIDGE 0x156b
+#define PCI_DEVICE_ID_INTEL_FALCON_RIDGE_4C_NHI 0x156c
+#define PCI_DEVICE_ID_INTEL_FALCON_RIDGE_4C_BRIDGE 0x156d
+#define PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_2C_NHI 0x1575 /* Thunderbolt 3 */
+#define PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_2C_BRIDGE 0x1576
+#define PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_4C_NHI 0x1577
+#define PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_4C_BRIDGE 0x1578
#define PCI_DEVICE_ID_INTEL_80960_RP 0x1960
#define PCI_DEVICE_ID_INTEL_82840_HB 0x1a21
#define PCI_DEVICE_ID_INTEL_82845_HB 0x1a30
diff --git a/include/linux/pcieport_if.h b/include/linux/pcieport_if.h
index 4f1089f..afcd130 100644
--- a/include/linux/pcieport_if.h
+++ b/include/linux/pcieport_if.h
@@ -21,6 +21,8 @@
#define PCIE_PORT_SERVICE_HP (1 << PCIE_PORT_SERVICE_HP_SHIFT)
#define PCIE_PORT_SERVICE_VC_SHIFT 3 /* Virtual Channel */
#define PCIE_PORT_SERVICE_VC (1 << PCIE_PORT_SERVICE_VC_SHIFT)
+#define PCIE_PORT_SERVICE_DPC_SHIFT 4 /* Downstream Port Containment */
+#define PCIE_PORT_SERVICE_DPC (1 << PCIE_PORT_SERVICE_DPC_SHIFT)
struct pcie_device {
int irq; /* Service IRQ/MSI/MSI-X Vector */
diff --git a/include/linux/percpu.h b/include/linux/percpu.h
index 4bc6daf..56939d3 100644
--- a/include/linux/percpu.h
+++ b/include/linux/percpu.h
@@ -129,7 +129,4 @@
(typeof(type) __percpu *)__alloc_percpu(sizeof(type), \
__alignof__(type))
-/* To avoid include hell, as printk can not declare this, we declare it here */
-DECLARE_PER_CPU(printk_func_t, printk_func);
-
#endif /* __LINUX_PERCPU_H */
diff --git a/include/linux/pinctrl/pinctrl.h b/include/linux/pinctrl/pinctrl.h
index 9ba59fc..a42e57d 100644
--- a/include/linux/pinctrl/pinctrl.h
+++ b/include/linux/pinctrl/pinctrl.h
@@ -144,6 +144,12 @@
extern struct pinctrl_dev *pinctrl_register(struct pinctrl_desc *pctldesc,
struct device *dev, void *driver_data);
extern void pinctrl_unregister(struct pinctrl_dev *pctldev);
+extern struct pinctrl_dev *devm_pinctrl_register(struct device *dev,
+ struct pinctrl_desc *pctldesc,
+ void *driver_data);
+extern void devm_pinctrl_unregister(struct device *dev,
+ struct pinctrl_dev *pctldev);
+
extern bool pin_is_valid(struct pinctrl_dev *pctldev, int pin);
extern void pinctrl_add_gpio_range(struct pinctrl_dev *pctldev,
struct pinctrl_gpio_range *range);
diff --git a/include/linux/platform_data/dma-dw.h b/include/linux/platform_data/dma-dw.h
index 03b6095..d15d8ba 100644
--- a/include/linux/platform_data/dma-dw.h
+++ b/include/linux/platform_data/dma-dw.h
@@ -21,15 +21,15 @@
* @dma_dev: required DMA master device
* @src_id: src request line
* @dst_id: dst request line
- * @src_master: src master for transfers on allocated channel.
- * @dst_master: dest master for transfers on allocated channel.
+ * @m_master: memory master for transfers on allocated channel
+ * @p_master: peripheral master for transfers on allocated channel
*/
struct dw_dma_slave {
struct device *dma_dev;
u8 src_id;
u8 dst_id;
- u8 src_master;
- u8 dst_master;
+ u8 m_master;
+ u8 p_master;
};
/**
@@ -43,7 +43,7 @@
* @block_size: Maximum block size supported by the controller
* @nr_masters: Number of AHB masters supported by the controller
* @data_width: Maximum data width supported by hardware per AHB master
- * (0 - 8bits, 1 - 16bits, ..., 5 - 256bits)
+ * (in bytes, power of 2)
*/
struct dw_dma_platform_data {
unsigned int nr_channels;
@@ -55,7 +55,7 @@
#define CHAN_PRIORITY_ASCENDING 0 /* chan0 highest */
#define CHAN_PRIORITY_DESCENDING 1 /* chan7 highest */
unsigned char chan_priority;
- unsigned short block_size;
+ unsigned int block_size;
unsigned char nr_masters;
unsigned char data_width[DW_DMA_MAX_NR_MASTERS];
};
diff --git a/include/linux/platform_data/gpmc-omap.h b/include/linux/platform_data/gpmc-omap.h
new file mode 100644
index 0000000..67ccdb0
--- /dev/null
+++ b/include/linux/platform_data/gpmc-omap.h
@@ -0,0 +1,172 @@
+/*
+ * OMAP GPMC Platform data
+ *
+ * Copyright (C) 2014 Texas Instruments, Inc. - http://www.ti.com
+ * Roger Quadros <rogerq@ti.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ */
+
+#ifndef _GPMC_OMAP_H_
+#define _GPMC_OMAP_H_
+
+/* Maximum Number of Chip Selects */
+#define GPMC_CS_NUM 8
+
+/* bool type time settings */
+struct gpmc_bool_timings {
+ bool cycle2cyclediffcsen;
+ bool cycle2cyclesamecsen;
+ bool we_extra_delay;
+ bool oe_extra_delay;
+ bool adv_extra_delay;
+ bool cs_extra_delay;
+ bool time_para_granularity;
+};
+
+/*
+ * Note that all values in this struct are in nanoseconds except sync_clk
+ * (which is in picoseconds), while the register values are in gpmc_fck cycles.
+ */
+struct gpmc_timings {
+ /* Minimum clock period for synchronous mode (in picoseconds) */
+ u32 sync_clk;
+
+ /* Chip-select signal timings corresponding to GPMC_CS_CONFIG2 */
+ u32 cs_on; /* Assertion time */
+ u32 cs_rd_off; /* Read deassertion time */
+ u32 cs_wr_off; /* Write deassertion time */
+
+ /* ADV signal timings corresponding to GPMC_CONFIG3 */
+ u32 adv_on; /* Assertion time */
+ u32 adv_rd_off; /* Read deassertion time */
+ u32 adv_wr_off; /* Write deassertion time */
+ u32 adv_aad_mux_on; /* ADV assertion time for AAD */
+ u32 adv_aad_mux_rd_off; /* ADV read deassertion time for AAD */
+ u32 adv_aad_mux_wr_off; /* ADV write deassertion time for AAD */
+
+ /* WE signals timings corresponding to GPMC_CONFIG4 */
+ u32 we_on; /* WE assertion time */
+ u32 we_off; /* WE deassertion time */
+
+ /* OE signals timings corresponding to GPMC_CONFIG4 */
+ u32 oe_on; /* OE assertion time */
+ u32 oe_off; /* OE deassertion time */
+ u32 oe_aad_mux_on; /* OE assertion time for AAD */
+ u32 oe_aad_mux_off; /* OE deassertion time for AAD */
+
+ /* Access time and cycle time timings corresponding to GPMC_CONFIG5 */
+ u32 page_burst_access; /* Multiple access word delay */
+ u32 access; /* Start-cycle to first data valid delay */
+ u32 rd_cycle; /* Total read cycle time */
+ u32 wr_cycle; /* Total write cycle time */
+
+ u32 bus_turnaround;
+ u32 cycle2cycle_delay;
+
+ u32 wait_monitoring;
+ u32 clk_activation;
+
+ /* The following are only on OMAP3430 */
+ u32 wr_access; /* WRACCESSTIME */
+ u32 wr_data_mux_bus; /* WRDATAONADMUXBUS */
+
+ struct gpmc_bool_timings bool_timings;
+};
+
+/* Device timings in picoseconds */
+struct gpmc_device_timings {
+ u32 t_ceasu; /* address setup to CS valid */
+ u32 t_avdasu; /* address setup to ADV valid */
+ /* XXX: try to combine t_avdp_r & t_avdp_w. Issue is
+ * of tusb using these timings even for sync whilst
+ * ideally for adv_rd/(wr)_off it should have considered
+ * t_avdh instead. This indirectly necessitates r/w
+ * variations of t_avdp as it is possible to have one
+ * sync & other async
+ */
+ u32 t_avdp_r; /* ADV low time (what about t_cer ?) */
+ u32 t_avdp_w;
+ u32 t_aavdh; /* address hold time */
+ u32 t_oeasu; /* address setup to OE valid */
+ u32 t_aa; /* access time from ADV assertion */
+ u32 t_iaa; /* initial access time */
+ u32 t_oe; /* access time from OE assertion */
+ u32 t_ce; /* access time from CS asertion */
+ u32 t_rd_cycle; /* read cycle time */
+ u32 t_cez_r; /* read CS deassertion to high Z */
+ u32 t_cez_w; /* write CS deassertion to high Z */
+ u32 t_oez; /* OE deassertion to high Z */
+ u32 t_weasu; /* address setup to WE valid */
+ u32 t_wpl; /* write assertion time */
+ u32 t_wph; /* write deassertion time */
+ u32 t_wr_cycle; /* write cycle time */
+
+ u32 clk;
+ u32 t_bacc; /* burst access valid clock to output delay */
+ u32 t_ces; /* CS setup time to clk */
+ u32 t_avds; /* ADV setup time to clk */
+ u32 t_avdh; /* ADV hold time from clk */
+ u32 t_ach; /* address hold time from clk */
+ u32 t_rdyo; /* clk to ready valid */
+
+ u32 t_ce_rdyz; /* XXX: description ?, or use t_cez instead */
+ u32 t_ce_avd; /* CS on to ADV on delay */
+
+ /* XXX: check the possibility of combining
+ * cyc_aavhd_oe & cyc_aavdh_we
+ */
+ u8 cyc_aavdh_oe;/* read address hold time in cycles */
+ u8 cyc_aavdh_we;/* write address hold time in cycles */
+ u8 cyc_oe; /* access time from OE assertion in cycles */
+ u8 cyc_wpl; /* write deassertion time in cycles */
+ u32 cyc_iaa; /* initial access time in cycles */
+
+ /* extra delays */
+ bool ce_xdelay;
+ bool avd_xdelay;
+ bool oe_xdelay;
+ bool we_xdelay;
+};
+
+#define GPMC_BURST_4 4 /* 4 word burst */
+#define GPMC_BURST_8 8 /* 8 word burst */
+#define GPMC_BURST_16 16 /* 16 word burst */
+#define GPMC_DEVWIDTH_8BIT 1 /* 8-bit device width */
+#define GPMC_DEVWIDTH_16BIT 2 /* 16-bit device width */
+#define GPMC_MUX_AAD 1 /* Addr-Addr-Data multiplex */
+#define GPMC_MUX_AD 2 /* Addr-Data multiplex */
+
+struct gpmc_settings {
+ bool burst_wrap; /* enables wrap bursting */
+ bool burst_read; /* enables read page/burst mode */
+ bool burst_write; /* enables write page/burst mode */
+ bool device_nand; /* device is NAND */
+ bool sync_read; /* enables synchronous reads */
+ bool sync_write; /* enables synchronous writes */
+ bool wait_on_read; /* monitor wait on reads */
+ bool wait_on_write; /* monitor wait on writes */
+ u32 burst_len; /* page/burst length */
+ u32 device_width; /* device bus width (8 or 16 bit) */
+ u32 mux_add_data; /* multiplex address & data */
+ u32 wait_pin; /* wait-pin to be used */
+};
+
+/* Data for each chip select */
+struct gpmc_omap_cs_data {
+ bool valid; /* data is valid */
+ bool is_nand; /* device within this CS is NAND */
+ struct gpmc_settings *settings;
+ struct gpmc_device_timings *device_timings;
+ struct gpmc_timings *gpmc_timings;
+ struct platform_device *pdev; /* device within this CS region */
+ unsigned int pdata_size;
+};
+
+struct gpmc_omap_platform_data {
+ struct gpmc_omap_cs_data cs[GPMC_CS_NUM];
+};
+
+#endif /* _GPMC_OMAP_H */
diff --git a/include/linux/platform_data/invensense_mpu6050.h b/include/linux/platform_data/invensense_mpu6050.h
index ad3aa7b..554b598 100644
--- a/include/linux/platform_data/invensense_mpu6050.h
+++ b/include/linux/platform_data/invensense_mpu6050.h
@@ -16,13 +16,16 @@
/**
* struct inv_mpu6050_platform_data - Platform data for the mpu driver
- * @orientation: Orientation matrix of the chip
+ * @orientation: Orientation matrix of the chip (deprecated in favor of
+ * mounting matrix retrieved from device-tree)
*
* Contains platform specific information on how to configure the MPU6050 to
* work on this platform. The orientation matricies are 3x3 rotation matricies
* that are applied to the data to rotate from the mounting orientation to the
* platform orientation. The values must be one of 0, 1, or -1 and each row and
* column should have exactly 1 non-zero value.
+ *
+ * Deprecated in favor of mounting matrix retrieved from device-tree.
*/
struct inv_mpu6050_platform_data {
__s8 orientation[9];
diff --git a/include/linux/platform_data/mailbox-omap.h b/include/linux/platform_data/mailbox-omap.h
deleted file mode 100644
index 4631dbb..0000000
--- a/include/linux/platform_data/mailbox-omap.h
+++ /dev/null
@@ -1,58 +0,0 @@
-/*
- * mailbox-omap.h
- *
- * Copyright (C) 2013 Texas Instruments, Inc.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * version 2 as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- */
-
-#ifndef _PLAT_MAILBOX_H
-#define _PLAT_MAILBOX_H
-
-/* Interrupt register configuration types */
-#define MBOX_INTR_CFG_TYPE1 (0)
-#define MBOX_INTR_CFG_TYPE2 (1)
-
-/**
- * struct omap_mbox_dev_info - OMAP mailbox device attribute info
- * @name: name of the mailbox device
- * @tx_id: mailbox queue id used for transmitting messages
- * @rx_id: mailbox queue id on which messages are received
- * @irq_id: irq identifier number to use from the hwmod data
- * @usr_id: mailbox user id for identifying the interrupt into
- * the MPU interrupt controller.
- */
-struct omap_mbox_dev_info {
- const char *name;
- u32 tx_id;
- u32 rx_id;
- u32 irq_id;
- u32 usr_id;
-};
-
-/**
- * struct omap_mbox_pdata - OMAP mailbox platform data
- * @intr_type: type of interrupt configuration registers used
- while programming mailbox queue interrupts
- * @num_users: number of users (processor devices) that the mailbox
- * h/w block can interrupt
- * @num_fifos: number of h/w fifos within the mailbox h/w block
- * @info_cnt: number of mailbox devices for the platform
- * @info: array of mailbox device attributes
- */
-struct omap_mbox_pdata {
- u32 intr_type;
- u32 num_users;
- u32 num_fifos;
- u32 info_cnt;
- struct omap_mbox_dev_info *info;
-};
-
-#endif /* _PLAT_MAILBOX_H */
diff --git a/include/linux/platform_data/mtd-nand-omap2.h b/include/linux/platform_data/mtd-nand-omap2.h
index 090bbab..17d57a1 100644
--- a/include/linux/platform_data/mtd-nand-omap2.h
+++ b/include/linux/platform_data/mtd-nand-omap2.h
@@ -45,7 +45,6 @@
};
struct gpmc_nand_regs {
- void __iomem *gpmc_status;
void __iomem *gpmc_nand_command;
void __iomem *gpmc_nand_address;
void __iomem *gpmc_nand_data;
@@ -64,21 +63,24 @@
void __iomem *gpmc_bch_result4[GPMC_BCH_NUM_REMAINDER];
void __iomem *gpmc_bch_result5[GPMC_BCH_NUM_REMAINDER];
void __iomem *gpmc_bch_result6[GPMC_BCH_NUM_REMAINDER];
+ /* Deprecated. Do not use */
+ void __iomem *gpmc_status;
};
struct omap_nand_platform_data {
int cs;
struct mtd_partition *parts;
int nr_parts;
- bool dev_ready;
bool flash_bbt;
enum nand_io xfer_type;
int devsize;
enum omap_ecc ecc_opt;
- struct gpmc_nand_regs reg;
- /* for passing the partitions */
- struct device_node *of_node;
struct device_node *elm_of_node;
+
+ /* deprecated */
+ struct gpmc_nand_regs reg;
+ struct device_node *of_node;
+ bool dev_ready;
};
#endif
diff --git a/include/linux/platform_data/st_sensors_pdata.h b/include/linux/platform_data/st_sensors_pdata.h
index 7538391..79b0e4c 100644
--- a/include/linux/platform_data/st_sensors_pdata.h
+++ b/include/linux/platform_data/st_sensors_pdata.h
@@ -16,9 +16,11 @@
* @drdy_int_pin: Redirect DRDY on pin 1 (1) or pin 2 (2).
* Available only for accelerometer and pressure sensors.
* Accelerometer DRDY on LSM330 available only on pin 1 (see datasheet).
+ * @open_drain: set the interrupt line to be open drain if possible.
*/
struct st_sensors_platform_data {
u8 drdy_int_pin;
+ bool open_drain;
};
#endif /* ST_SENSORS_PDATA_H */
diff --git a/include/linux/poll.h b/include/linux/poll.h
index 9fb4f40..37b057b 100644
--- a/include/linux/poll.h
+++ b/include/linux/poll.h
@@ -96,7 +96,7 @@
extern void poll_freewait(struct poll_wqueues *pwq);
extern int poll_schedule_timeout(struct poll_wqueues *pwq, int state,
ktime_t *expires, unsigned long slack);
-extern u64 select_estimate_accuracy(struct timespec *tv);
+extern u64 select_estimate_accuracy(struct timespec64 *tv);
static inline int poll_schedule(struct poll_wqueues *pwq, int state)
@@ -153,12 +153,13 @@
#define MAX_INT64_SECONDS (((s64)(~((u64)0)>>1)/HZ)-1)
-extern int do_select(int n, fd_set_bits *fds, struct timespec *end_time);
+extern int do_select(int n, fd_set_bits *fds, struct timespec64 *end_time);
extern int do_sys_poll(struct pollfd __user * ufds, unsigned int nfds,
- struct timespec *end_time);
+ struct timespec64 *end_time);
extern int core_sys_select(int n, fd_set __user *inp, fd_set __user *outp,
- fd_set __user *exp, struct timespec *end_time);
+ fd_set __user *exp, struct timespec64 *end_time);
-extern int poll_select_set_timeout(struct timespec *to, long sec, long nsec);
+extern int poll_select_set_timeout(struct timespec64 *to, time64_t sec,
+ long nsec);
#endif /* _LINUX_POLL_H */
diff --git a/include/linux/printk.h b/include/linux/printk.h
index 9ccbdf2..f4da695 100644
--- a/include/linux/printk.h
+++ b/include/linux/printk.h
@@ -122,7 +122,19 @@
void early_printk(const char *s, ...) { }
#endif
-typedef __printf(1, 0) int (*printk_func_t)(const char *fmt, va_list args);
+#ifdef CONFIG_PRINTK_NMI
+extern void printk_nmi_init(void);
+extern void printk_nmi_enter(void);
+extern void printk_nmi_exit(void);
+extern void printk_nmi_flush(void);
+extern void printk_nmi_flush_on_panic(void);
+#else
+static inline void printk_nmi_init(void) { }
+static inline void printk_nmi_enter(void) { }
+static inline void printk_nmi_exit(void) { }
+static inline void printk_nmi_flush(void) { }
+static inline void printk_nmi_flush_on_panic(void) { }
+#endif /* PRINTK_NMI */
#ifdef CONFIG_PRINTK
asmlinkage __printf(5, 0)
diff --git a/include/linux/radix-tree.h b/include/linux/radix-tree.h
index 51a97ac..cb4b7e8 100644
--- a/include/linux/radix-tree.h
+++ b/include/linux/radix-tree.h
@@ -29,51 +29,45 @@
#include <linux/rcupdate.h>
/*
- * An indirect pointer (root->rnode pointing to a radix_tree_node, rather
- * than a data item) is signalled by the low bit set in the root->rnode
- * pointer.
+ * The bottom two bits of the slot determine how the remaining bits in the
+ * slot are interpreted:
*
- * In this case root->height is > 0, but the indirect pointer tests are
- * needed for RCU lookups (because root->height is unreliable). The only
- * time callers need worry about this is when doing a lookup_slot under
- * RCU.
+ * 00 - data pointer
+ * 01 - internal entry
+ * 10 - exceptional entry
+ * 11 - locked exceptional entry
*
- * Indirect pointer in fact is also used to tag the last pointer of a node
- * when it is shrunk, before we rcu free the node. See shrink code for
- * details.
+ * The internal entry may be a pointer to the next level in the tree, a
+ * sibling entry, or an indicator that the entry in this slot has been moved
+ * to another location in the tree and the lookup should be restarted. While
+ * NULL fits the 'data pointer' pattern, it means that there is no entry in
+ * the tree for this index (no matter what level of the tree it is found at).
+ * This means that you cannot store NULL in the tree as a value for the index.
*/
-#define RADIX_TREE_INDIRECT_PTR 1
+#define RADIX_TREE_ENTRY_MASK 3UL
+#define RADIX_TREE_INTERNAL_NODE 1UL
+
/*
- * A common use of the radix tree is to store pointers to struct pages;
- * but shmem/tmpfs needs also to store swap entries in the same tree:
- * those are marked as exceptional entries to distinguish them.
+ * Most users of the radix tree store pointers but shmem/tmpfs stores swap
+ * entries in the same tree. They are marked as exceptional entries to
+ * distinguish them from pointers to struct page.
* EXCEPTIONAL_ENTRY tests the bit, EXCEPTIONAL_SHIFT shifts content past it.
*/
#define RADIX_TREE_EXCEPTIONAL_ENTRY 2
#define RADIX_TREE_EXCEPTIONAL_SHIFT 2
-#define RADIX_DAX_MASK 0xf
-#define RADIX_DAX_SHIFT 4
-#define RADIX_DAX_PTE (0x4 | RADIX_TREE_EXCEPTIONAL_ENTRY)
-#define RADIX_DAX_PMD (0x8 | RADIX_TREE_EXCEPTIONAL_ENTRY)
-#define RADIX_DAX_TYPE(entry) ((unsigned long)entry & RADIX_DAX_MASK)
-#define RADIX_DAX_SECTOR(entry) (((unsigned long)entry >> RADIX_DAX_SHIFT))
-#define RADIX_DAX_ENTRY(sector, pmd) ((void *)((unsigned long)sector << \
- RADIX_DAX_SHIFT | (pmd ? RADIX_DAX_PMD : RADIX_DAX_PTE)))
-
-static inline int radix_tree_is_indirect_ptr(void *ptr)
+static inline bool radix_tree_is_internal_node(void *ptr)
{
- return (int)((unsigned long)ptr & RADIX_TREE_INDIRECT_PTR);
+ return ((unsigned long)ptr & RADIX_TREE_ENTRY_MASK) ==
+ RADIX_TREE_INTERNAL_NODE;
}
/*** radix-tree API starts here ***/
#define RADIX_TREE_MAX_TAGS 3
-#ifdef __KERNEL__
+#ifndef RADIX_TREE_MAP_SHIFT
#define RADIX_TREE_MAP_SHIFT (CONFIG_BASE_SMALL ? 4 : 6)
-#else
-#define RADIX_TREE_MAP_SHIFT 3 /* For more stressful testing */
#endif
#define RADIX_TREE_MAP_SIZE (1UL << RADIX_TREE_MAP_SHIFT)
@@ -86,16 +80,13 @@
#define RADIX_TREE_MAX_PATH (DIV_ROUND_UP(RADIX_TREE_INDEX_BITS, \
RADIX_TREE_MAP_SHIFT))
-/* Height component in node->path */
-#define RADIX_TREE_HEIGHT_SHIFT (RADIX_TREE_MAX_PATH + 1)
-#define RADIX_TREE_HEIGHT_MASK ((1UL << RADIX_TREE_HEIGHT_SHIFT) - 1)
-
/* Internally used bits of node->count */
#define RADIX_TREE_COUNT_SHIFT (RADIX_TREE_MAP_SHIFT + 1)
#define RADIX_TREE_COUNT_MASK ((1UL << RADIX_TREE_COUNT_SHIFT) - 1)
struct radix_tree_node {
- unsigned int path; /* Offset in parent & height from the bottom */
+ unsigned char shift; /* Bits remaining in each slot */
+ unsigned char offset; /* Slot offset in parent */
unsigned int count;
union {
struct {
@@ -115,13 +106,11 @@
/* root tags are stored in gfp_mask, shifted by __GFP_BITS_SHIFT */
struct radix_tree_root {
- unsigned int height;
gfp_t gfp_mask;
struct radix_tree_node __rcu *rnode;
};
#define RADIX_TREE_INIT(mask) { \
- .height = 0, \
.gfp_mask = (mask), \
.rnode = NULL, \
}
@@ -131,11 +120,15 @@
#define INIT_RADIX_TREE(root, mask) \
do { \
- (root)->height = 0; \
(root)->gfp_mask = (mask); \
(root)->rnode = NULL; \
} while (0)
+static inline bool radix_tree_empty(struct radix_tree_root *root)
+{
+ return root->rnode == NULL;
+}
+
/**
* Radix-tree synchronization
*
@@ -231,7 +224,7 @@
*/
static inline int radix_tree_deref_retry(void *arg)
{
- return unlikely((unsigned long)arg & RADIX_TREE_INDIRECT_PTR);
+ return unlikely(radix_tree_is_internal_node(arg));
}
/**
@@ -252,8 +245,7 @@
*/
static inline int radix_tree_exception(void *arg)
{
- return unlikely((unsigned long)arg &
- (RADIX_TREE_INDIRECT_PTR | RADIX_TREE_EXCEPTIONAL_ENTRY));
+ return unlikely((unsigned long)arg & RADIX_TREE_ENTRY_MASK);
}
/**
@@ -266,7 +258,7 @@
*/
static inline void radix_tree_replace_slot(void **pslot, void *item)
{
- BUG_ON(radix_tree_is_indirect_ptr(item));
+ BUG_ON(radix_tree_is_internal_node(item));
rcu_assign_pointer(*pslot, item);
}
@@ -288,9 +280,12 @@
struct radix_tree_node *node);
void *radix_tree_delete_item(struct radix_tree_root *, unsigned long, void *);
void *radix_tree_delete(struct radix_tree_root *, unsigned long);
-unsigned int
-radix_tree_gang_lookup(struct radix_tree_root *root, void **results,
- unsigned long first_index, unsigned int max_items);
+struct radix_tree_node *radix_tree_replace_clear_tags(
+ struct radix_tree_root *root,
+ unsigned long index, void *entry);
+unsigned int radix_tree_gang_lookup(struct radix_tree_root *root,
+ void **results, unsigned long first_index,
+ unsigned int max_items);
unsigned int radix_tree_gang_lookup_slot(struct radix_tree_root *root,
void ***results, unsigned long *indices,
unsigned long first_index, unsigned int max_items);
@@ -327,8 +322,9 @@
* struct radix_tree_iter - radix tree iterator state
*
* @index: index of current slot
- * @next_index: next-to-last index for this chunk
+ * @next_index: one beyond the last index for this chunk
* @tags: bit-mask for tag-iterating
+ * @shift: shift for the node that holds our slots
*
* This radix tree iterator works in terms of "chunks" of slots. A chunk is a
* subinterval of slots contained within one radix tree leaf node. It is
@@ -341,8 +337,20 @@
unsigned long index;
unsigned long next_index;
unsigned long tags;
+#ifdef CONFIG_RADIX_TREE_MULTIORDER
+ unsigned int shift;
+#endif
};
+static inline unsigned int iter_shift(struct radix_tree_iter *iter)
+{
+#ifdef CONFIG_RADIX_TREE_MULTIORDER
+ return iter->shift;
+#else
+ return 0;
+#endif
+}
+
#define RADIX_TREE_ITER_TAG_MASK 0x00FF /* tag index in lower byte */
#define RADIX_TREE_ITER_TAGGED 0x0100 /* lookup tagged slots */
#define RADIX_TREE_ITER_CONTIG 0x0200 /* stop at first hole */
@@ -402,6 +410,12 @@
return NULL;
}
+static inline unsigned long
+__radix_tree_iter_add(struct radix_tree_iter *iter, unsigned long slots)
+{
+ return iter->index + (slots << iter_shift(iter));
+}
+
/**
* radix_tree_iter_next - resume iterating when the chunk may be invalid
* @iter: iterator state
@@ -413,7 +427,7 @@
static inline __must_check
void **radix_tree_iter_next(struct radix_tree_iter *iter)
{
- iter->next_index = iter->index + 1;
+ iter->next_index = __radix_tree_iter_add(iter, 1);
iter->tags = 0;
return NULL;
}
@@ -427,7 +441,12 @@
static __always_inline long
radix_tree_chunk_size(struct radix_tree_iter *iter)
{
- return iter->next_index - iter->index;
+ return (iter->next_index - iter->index) >> iter_shift(iter);
+}
+
+static inline struct radix_tree_node *entry_to_node(void *ptr)
+{
+ return (void *)((unsigned long)ptr & ~RADIX_TREE_INTERNAL_NODE);
}
/**
@@ -445,24 +464,49 @@
radix_tree_next_slot(void **slot, struct radix_tree_iter *iter, unsigned flags)
{
if (flags & RADIX_TREE_ITER_TAGGED) {
+ void *canon = slot;
+
iter->tags >>= 1;
+ if (unlikely(!iter->tags))
+ return NULL;
+ while (IS_ENABLED(CONFIG_RADIX_TREE_MULTIORDER) &&
+ radix_tree_is_internal_node(slot[1])) {
+ if (entry_to_node(slot[1]) == canon) {
+ iter->tags >>= 1;
+ iter->index = __radix_tree_iter_add(iter, 1);
+ slot++;
+ continue;
+ }
+ iter->next_index = __radix_tree_iter_add(iter, 1);
+ return NULL;
+ }
if (likely(iter->tags & 1ul)) {
- iter->index++;
+ iter->index = __radix_tree_iter_add(iter, 1);
return slot + 1;
}
- if (!(flags & RADIX_TREE_ITER_CONTIG) && likely(iter->tags)) {
+ if (!(flags & RADIX_TREE_ITER_CONTIG)) {
unsigned offset = __ffs(iter->tags);
iter->tags >>= offset;
- iter->index += offset + 1;
+ iter->index = __radix_tree_iter_add(iter, offset + 1);
return slot + offset + 1;
}
} else {
- long size = radix_tree_chunk_size(iter);
+ long count = radix_tree_chunk_size(iter);
+ void *canon = slot;
- while (--size > 0) {
+ while (--count > 0) {
slot++;
- iter->index++;
+ iter->index = __radix_tree_iter_add(iter, 1);
+
+ if (IS_ENABLED(CONFIG_RADIX_TREE_MULTIORDER) &&
+ radix_tree_is_internal_node(*slot)) {
+ if (entry_to_node(*slot) == canon)
+ continue;
+ iter->next_index = iter->index;
+ break;
+ }
+
if (likely(*slot))
return slot;
if (flags & RADIX_TREE_ITER_CONTIG) {
@@ -476,34 +520,6 @@
}
/**
- * radix_tree_for_each_chunk - iterate over chunks
- *
- * @slot: the void** variable for pointer to chunk first slot
- * @root: the struct radix_tree_root pointer
- * @iter: the struct radix_tree_iter pointer
- * @start: iteration starting index
- * @flags: RADIX_TREE_ITER_* and tag index
- *
- * Locks can be released and reacquired between iterations.
- */
-#define radix_tree_for_each_chunk(slot, root, iter, start, flags) \
- for (slot = radix_tree_iter_init(iter, start) ; \
- (slot = radix_tree_next_chunk(root, iter, flags)) ;)
-
-/**
- * radix_tree_for_each_chunk_slot - iterate over slots in one chunk
- *
- * @slot: the void** variable, at the beginning points to chunk first slot
- * @iter: the struct radix_tree_iter pointer
- * @flags: RADIX_TREE_ITER_*, should be constant
- *
- * This macro is designed to be nested inside radix_tree_for_each_chunk().
- * @slot points to the radix tree slot, @iter->index contains its index.
- */
-#define radix_tree_for_each_chunk_slot(slot, iter, flags) \
- for (; slot ; slot = radix_tree_next_slot(slot, iter, flags))
-
-/**
* radix_tree_for_each_slot - iterate over non-empty slots
*
* @slot: the void** variable for pointer to slot
diff --git a/include/linux/random.h b/include/linux/random.h
index 9c29122..e47e533 100644
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -26,7 +26,6 @@
extern int add_random_ready_callback(struct random_ready_callback *rdy);
extern void del_random_ready_callback(struct random_ready_callback *rdy);
extern void get_random_bytes_arch(void *buf, int nbytes);
-void generate_random_uuid(unsigned char uuid_out[16]);
extern int random_int_secret_init(void);
#ifndef MODULE
diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
index 9c4e138..1c457a8 100644
--- a/include/linux/remoteproc.h
+++ b/include/linux/remoteproc.h
@@ -365,6 +365,8 @@
/**
* enum rproc_crash_type - remote processor crash types
* @RPROC_MMUFAULT: iommu fault
+ * @RPROC_WATCHDOG: watchdog bite
+ * @RPROC_FATAL_ERROR fatal error
*
* Each element of the enum is used as an array index. So that, the value of
* the elements should be always something sane.
@@ -373,6 +375,8 @@
*/
enum rproc_crash_type {
RPROC_MMUFAULT,
+ RPROC_WATCHDOG,
+ RPROC_FATAL_ERROR,
};
/**
diff --git a/include/linux/reservation.h b/include/linux/reservation.h
index 5a0b64c..49d0576 100644
--- a/include/linux/reservation.h
+++ b/include/linux/reservation.h
@@ -120,6 +120,24 @@
reservation_object_held(obj));
}
+static inline struct fence *
+reservation_object_get_excl_rcu(struct reservation_object *obj)
+{
+ struct fence *fence;
+ unsigned seq;
+retry:
+ seq = read_seqcount_begin(&obj->seq);
+ rcu_read_lock();
+ fence = rcu_dereference(obj->fence_excl);
+ if (read_seqcount_retry(&obj->seq, seq)) {
+ rcu_read_unlock();
+ goto retry;
+ }
+ fence = fence_get(fence);
+ rcu_read_unlock();
+ return fence;
+}
+
int reservation_object_reserve_shared(struct reservation_object *obj);
void reservation_object_add_shared_fence(struct reservation_object *obj,
struct fence *fence);
diff --git a/include/linux/rpmsg.h b/include/linux/rpmsg.h
index 82a6739..ada50ff 100644
--- a/include/linux/rpmsg.h
+++ b/include/linux/rpmsg.h
@@ -169,7 +169,7 @@
int register_rpmsg_device(struct rpmsg_channel *dev);
void unregister_rpmsg_device(struct rpmsg_channel *dev);
-int register_rpmsg_driver(struct rpmsg_driver *drv);
+int __register_rpmsg_driver(struct rpmsg_driver *drv, struct module *owner);
void unregister_rpmsg_driver(struct rpmsg_driver *drv);
void rpmsg_destroy_ept(struct rpmsg_endpoint *);
struct rpmsg_endpoint *rpmsg_create_ept(struct rpmsg_channel *,
@@ -177,6 +177,22 @@
int
rpmsg_send_offchannel_raw(struct rpmsg_channel *, u32, u32, void *, int, bool);
+/* use a macro to avoid include chaining to get THIS_MODULE */
+#define register_rpmsg_driver(drv) \
+ __register_rpmsg_driver(drv, THIS_MODULE)
+
+/**
+ * module_rpmsg_driver() - Helper macro for registering an rpmsg driver
+ * @__rpmsg_driver: rpmsg_driver struct
+ *
+ * Helper macro for rpmsg drivers which do not do anything special in module
+ * init/exit. This eliminates a lot of boilerplate. Each module may only
+ * use this macro once, and calling it replaces module_init() and module_exit()
+ */
+#define module_rpmsg_driver(__rpmsg_driver) \
+ module_driver(__rpmsg_driver, register_rpmsg_driver, \
+ unregister_rpmsg_driver)
+
/**
* rpmsg_send() - send a message across to the remote processor
* @rpdev: the rpmsg channel
diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h
index 556ec1e..cb3c8fe 100644
--- a/include/linux/scatterlist.h
+++ b/include/linux/scatterlist.h
@@ -286,6 +286,31 @@
#define SG_MAX_SINGLE_ALLOC (PAGE_SIZE / sizeof(struct scatterlist))
/*
+ * The maximum number of SG segments that we will put inside a
+ * scatterlist (unless chaining is used). Should ideally fit inside a
+ * single page, to avoid a higher order allocation. We could define this
+ * to SG_MAX_SINGLE_ALLOC to pack correctly at the highest order. The
+ * minimum value is 32
+ */
+#define SG_CHUNK_SIZE 128
+
+/*
+ * Like SG_CHUNK_SIZE, but for archs that have sg chaining. This limit
+ * is totally arbitrary, a setting of 2048 will get you at least 8mb ios.
+ */
+#ifdef CONFIG_ARCH_HAS_SG_CHAIN
+#define SG_MAX_SEGMENTS 2048
+#else
+#define SG_MAX_SEGMENTS SG_CHUNK_SIZE
+#endif
+
+#ifdef CONFIG_SG_POOL
+void sg_free_table_chained(struct sg_table *table, bool first_chunk);
+int sg_alloc_table_chained(struct sg_table *table, int nents,
+ struct scatterlist *first_chunk);
+#endif
+
+/*
* sg page iterator
*
* Iterates over sg entries page-by-page. On each successful iteration,
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 31bd0d9..21c26e7 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -521,6 +521,7 @@
#define MMF_HAS_UPROBES 19 /* has uprobes */
#define MMF_RECALC_UPROBES 20 /* MMF_HAS_UPROBES can be wrong */
+#define MMF_OOM_REAPED 21 /* mm has been already reaped */
#define MMF_INIT_MASK (MMF_DUMPABLE_MASK | MMF_DUMP_FILTER_MASK)
@@ -668,6 +669,7 @@
atomic_t sigcnt;
atomic_t live;
int nr_threads;
+ atomic_t oom_victims; /* # of TIF_MEDIE threads in this thread group */
struct list_head thread_head;
wait_queue_head_t wait_chldexit; /* for wait4() */
@@ -792,7 +794,11 @@
struct tty_audit_buf *tty_audit_buf;
#endif
- oom_flags_t oom_flags;
+ /*
+ * Thread is the potential origin of an oom condition; kill first on
+ * oom
+ */
+ bool oom_flag_origin;
short oom_score_adj; /* OOM kill score adjustment */
short oom_score_adj_min; /* OOM kill score adjustment min value.
* Only settable by CAP_SYS_RESOURCE. */
@@ -2248,6 +2254,7 @@
#define PFA_NO_NEW_PRIVS 0 /* May not gain new privileges. */
#define PFA_SPREAD_PAGE 1 /* Spread page cache over cpuset */
#define PFA_SPREAD_SLAB 2 /* Spread some slab caches over cpuset */
+#define PFA_LMK_WAITING 3 /* Lowmemorykiller is waiting */
#define TASK_PFA_TEST(name, func) \
@@ -2271,6 +2278,9 @@
TASK_PFA_SET(SPREAD_SLAB, spread_slab)
TASK_PFA_CLEAR(SPREAD_SLAB, spread_slab)
+TASK_PFA_TEST(LMK_WAITING, lmk_waiting)
+TASK_PFA_SET(LMK_WAITING, lmk_waiting)
+
/*
* task->jobctl flags
*/
@@ -2721,14 +2731,24 @@
/* mmdrop drops the mm and the page tables */
extern void __mmdrop(struct mm_struct *);
-static inline void mmdrop(struct mm_struct * mm)
+static inline void mmdrop(struct mm_struct *mm)
{
if (unlikely(atomic_dec_and_test(&mm->mm_count)))
__mmdrop(mm);
}
+static inline bool mmget_not_zero(struct mm_struct *mm)
+{
+ return atomic_inc_not_zero(&mm->mm_users);
+}
+
/* mmput gets rid of the mappings and all user-space */
extern void mmput(struct mm_struct *);
+/* same as above but performs the slow path from the async kontext. Can
+ * be called from the atomic context as well
+ */
+extern void mmput_async(struct mm_struct *);
+
/* Grab a reference to a task's mm, if it is not already going away */
extern struct mm_struct *get_task_mm(struct task_struct *task);
/*
@@ -2757,7 +2777,14 @@
}
#endif
extern void flush_thread(void);
-extern void exit_thread(void);
+
+#ifdef CONFIG_HAVE_EXIT_THREAD
+extern void exit_thread(struct task_struct *tsk);
+#else
+static inline void exit_thread(struct task_struct *tsk)
+{
+}
+#endif
extern void exit_files(struct task_struct *);
extern void __cleanup_sighand(struct sighand_struct *);
diff --git a/include/linux/selection.h b/include/linux/selection.h
index 85193aa..8e4624e 100644
--- a/include/linux/selection.h
+++ b/include/linux/selection.h
@@ -24,10 +24,10 @@
extern int console_blanked;
-extern unsigned char color_table[];
-extern int default_red[];
-extern int default_grn[];
-extern int default_blu[];
+extern const unsigned char color_table[];
+extern unsigned char default_red[];
+extern unsigned char default_grn[];
+extern unsigned char default_blu[];
extern unsigned short *screen_pos(struct vc_data *vc, int w_offset, int viewed);
extern u16 screen_glyph(struct vc_data *vc, int offset);
diff --git a/include/linux/serial_8250.h b/include/linux/serial_8250.h
index 4348797..48ec765 100644
--- a/include/linux/serial_8250.h
+++ b/include/linux/serial_8250.h
@@ -36,6 +36,7 @@
void (*set_termios)(struct uart_port *,
struct ktermios *new,
struct ktermios *old);
+ unsigned int (*get_mctrl)(struct uart_port *);
int (*handle_irq)(struct uart_port *);
void (*pm)(struct uart_port *, unsigned int state,
unsigned old);
@@ -148,6 +149,7 @@
const char *options);
extern void serial8250_do_set_termios(struct uart_port *port,
struct ktermios *termios, struct ktermios *old);
+extern unsigned int serial8250_do_get_mctrl(struct uart_port *port);
extern int serial8250_do_startup(struct uart_port *port);
extern void serial8250_do_shutdown(struct uart_port *port);
extern void serial8250_do_pm(struct uart_port *port, unsigned int state,
diff --git a/include/linux/serial_core.h b/include/linux/serial_core.h
index cbfcf38..a3d7c0d 100644
--- a/include/linux/serial_core.h
+++ b/include/linux/serial_core.h
@@ -123,6 +123,7 @@
void (*set_termios)(struct uart_port *,
struct ktermios *new,
struct ktermios *old);
+ unsigned int (*get_mctrl)(struct uart_port *);
void (*set_mctrl)(struct uart_port *, unsigned int);
int (*startup)(struct uart_port *port);
void (*shutdown)(struct uart_port *port);
@@ -281,6 +282,8 @@
enum uart_pm_state pm_state;
struct circ_buf xmit;
+ atomic_t refcount;
+ wait_queue_head_t remove_wait;
struct uart_port *uart_port;
};
diff --git a/include/linux/signal.h b/include/linux/signal.h
index 3fbe814..b63f63e 100644
--- a/include/linux/signal.h
+++ b/include/linux/signal.h
@@ -28,6 +28,21 @@
sigset_t signal;
};
+#ifndef HAVE_ARCH_COPY_SIGINFO
+
+#include <linux/string.h>
+
+static inline void copy_siginfo(struct siginfo *to, struct siginfo *from)
+{
+ if (from->si_code < 0)
+ memcpy(to, from, sizeof(*to));
+ else
+ /* _sigchld is currently the largest know union member */
+ memcpy(to, from, __ARCH_SI_PREAMBLE_SIZE + sizeof(from->_sifields._sigchld));
+}
+
+#endif
+
/*
* Define some primitives to manipulate sigset_t.
*/
@@ -385,7 +400,9 @@
#else
#define rt_sigmask(sig) sigmask(sig)
#endif
-#define siginmask(sig, mask) (rt_sigmask(sig) & (mask))
+
+#define siginmask(sig, mask) \
+ ((sig) < SIGRTMIN && (rt_sigmask(sig) & (mask)))
#define SIG_KERNEL_ONLY_MASK (\
rt_sigmask(SIGKILL) | rt_sigmask(SIGSTOP))
@@ -406,14 +423,10 @@
rt_sigmask(SIGCONT) | rt_sigmask(SIGCHLD) | \
rt_sigmask(SIGWINCH) | rt_sigmask(SIGURG) )
-#define sig_kernel_only(sig) \
- (((sig) < SIGRTMIN) && siginmask(sig, SIG_KERNEL_ONLY_MASK))
-#define sig_kernel_coredump(sig) \
- (((sig) < SIGRTMIN) && siginmask(sig, SIG_KERNEL_COREDUMP_MASK))
-#define sig_kernel_ignore(sig) \
- (((sig) < SIGRTMIN) && siginmask(sig, SIG_KERNEL_IGNORE_MASK))
-#define sig_kernel_stop(sig) \
- (((sig) < SIGRTMIN) && siginmask(sig, SIG_KERNEL_STOP_MASK))
+#define sig_kernel_only(sig) siginmask(sig, SIG_KERNEL_ONLY_MASK)
+#define sig_kernel_coredump(sig) siginmask(sig, SIG_KERNEL_COREDUMP_MASK)
+#define sig_kernel_ignore(sig) siginmask(sig, SIG_KERNEL_IGNORE_MASK)
+#define sig_kernel_stop(sig) siginmask(sig, SIG_KERNEL_STOP_MASK)
#define sig_user_defined(t, signr) \
(((t)->sighand->action[(signr)-1].sa.sa_handler != SIG_DFL) && \
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index c413c58..ee38a41 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -471,9 +471,9 @@
SKB_GSO_GRE_CSUM = 1 << 8,
- SKB_GSO_IPIP = 1 << 9,
+ SKB_GSO_IPXIP4 = 1 << 9,
- SKB_GSO_SIT = 1 << 10,
+ SKB_GSO_IPXIP6 = 1 << 10,
SKB_GSO_UDP_TUNNEL = 1 << 11,
@@ -2467,7 +2467,7 @@
static inline struct page *dev_alloc_pages(unsigned int order)
{
- return __dev_alloc_pages(GFP_ATOMIC, order);
+ return __dev_alloc_pages(GFP_ATOMIC | __GFP_NOWARN, order);
}
/**
@@ -2485,7 +2485,7 @@
static inline struct page *dev_alloc_page(void)
{
- return __dev_alloc_page(GFP_ATOMIC);
+ return dev_alloc_pages(0);
}
/**
diff --git a/include/linux/slab.h b/include/linux/slab.h
index 508bd82..aeb3e6d 100644
--- a/include/linux/slab.h
+++ b/include/linux/slab.h
@@ -315,8 +315,8 @@
}
#endif /* !CONFIG_SLOB */
-void *__kmalloc(size_t size, gfp_t flags) __assume_kmalloc_alignment;
-void *kmem_cache_alloc(struct kmem_cache *, gfp_t flags) __assume_slab_alignment;
+void *__kmalloc(size_t size, gfp_t flags) __assume_kmalloc_alignment __malloc;
+void *kmem_cache_alloc(struct kmem_cache *, gfp_t flags) __assume_slab_alignment __malloc;
void kmem_cache_free(struct kmem_cache *, void *);
/*
@@ -339,8 +339,8 @@
}
#ifdef CONFIG_NUMA
-void *__kmalloc_node(size_t size, gfp_t flags, int node) __assume_kmalloc_alignment;
-void *kmem_cache_alloc_node(struct kmem_cache *, gfp_t flags, int node) __assume_slab_alignment;
+void *__kmalloc_node(size_t size, gfp_t flags, int node) __assume_kmalloc_alignment __malloc;
+void *kmem_cache_alloc_node(struct kmem_cache *, gfp_t flags, int node) __assume_slab_alignment __malloc;
#else
static __always_inline void *__kmalloc_node(size_t size, gfp_t flags, int node)
{
@@ -354,12 +354,12 @@
#endif
#ifdef CONFIG_TRACING
-extern void *kmem_cache_alloc_trace(struct kmem_cache *, gfp_t, size_t) __assume_slab_alignment;
+extern void *kmem_cache_alloc_trace(struct kmem_cache *, gfp_t, size_t) __assume_slab_alignment __malloc;
#ifdef CONFIG_NUMA
extern void *kmem_cache_alloc_node_trace(struct kmem_cache *s,
gfp_t gfpflags,
- int node, size_t size) __assume_slab_alignment;
+ int node, size_t size) __assume_slab_alignment __malloc;
#else
static __always_inline void *
kmem_cache_alloc_node_trace(struct kmem_cache *s,
@@ -392,10 +392,10 @@
}
#endif /* CONFIG_TRACING */
-extern void *kmalloc_order(size_t size, gfp_t flags, unsigned int order) __assume_page_alignment;
+extern void *kmalloc_order(size_t size, gfp_t flags, unsigned int order) __assume_page_alignment __malloc;
#ifdef CONFIG_TRACING
-extern void *kmalloc_order_trace(size_t size, gfp_t flags, unsigned int order) __assume_page_alignment;
+extern void *kmalloc_order_trace(size_t size, gfp_t flags, unsigned int order) __assume_page_alignment __malloc;
#else
static __always_inline void *
kmalloc_order_trace(size_t size, gfp_t flags, unsigned int order)
diff --git a/include/linux/slab_def.h b/include/linux/slab_def.h
index 9edbbf3..8694f7a 100644
--- a/include/linux/slab_def.h
+++ b/include/linux/slab_def.h
@@ -80,6 +80,10 @@
struct kasan_cache kasan_info;
#endif
+#ifdef CONFIG_SLAB_FREELIST_RANDOM
+ void *random_seq;
+#endif
+
struct kmem_cache_node *node[MAX_NUMNODES];
};
diff --git a/include/linux/spi/spi.h b/include/linux/spi/spi.h
index 857a9a1..1f03483 100644
--- a/include/linux/spi/spi.h
+++ b/include/linux/spi/spi.h
@@ -372,6 +372,7 @@
* @unprepare_message: undo any work done by prepare_message().
* @spi_flash_read: to support spi-controller hardwares that provide
* accelerated interface to read from flash devices.
+ * @flash_read_supported: spi device supports flash read
* @cs_gpios: Array of GPIOs to use as chip select lines; one per CS
* number. Any individual value may be -ENOENT for CS lines that
* are not GPIOs (driven by the SPI controller itself).
@@ -529,6 +530,7 @@
struct spi_message *message);
int (*spi_flash_read)(struct spi_device *spi,
struct spi_flash_read_message *msg);
+ bool (*flash_read_supported)(struct spi_device *spi);
/*
* These hooks are for drivers that use a generic implementation
@@ -1158,7 +1160,9 @@
/* SPI core interface for flash read support */
static inline bool spi_flash_read_supported(struct spi_device *spi)
{
- return spi->master->spi_flash_read ? true : false;
+ return spi->master->spi_flash_read &&
+ (!spi->master->flash_read_supported ||
+ spi->master->flash_read_supported(spi));
}
int spi_flash_read(struct spi_device *spi,
diff --git a/include/linux/stm.h b/include/linux/stm.h
index 1a79ed8..8369d8a 100644
--- a/include/linux/stm.h
+++ b/include/linux/stm.h
@@ -50,6 +50,8 @@
* @sw_end: last STP master available to software
* @sw_nchannels: number of STP channels per master
* @sw_mmiosz: size of one channel's IO space, for mmap, optional
+ * @hw_override: masters in the STP stream will not match the ones
+ * assigned by software, but are up to the STM hardware
* @packet: callback that sends an STP packet
* @mmio_addr: mmap callback, optional
* @link: called when a new stm_source gets linked to us, optional
@@ -85,6 +87,7 @@
unsigned int sw_end;
unsigned int sw_nchannels;
unsigned int sw_mmiosz;
+ unsigned int hw_override;
ssize_t (*packet)(struct stm_data *, unsigned int,
unsigned int, unsigned int,
unsigned int, unsigned int,
diff --git a/include/linux/string.h b/include/linux/string.h
index d3993a7..26b6f6a 100644
--- a/include/linux/string.h
+++ b/include/linux/string.h
@@ -119,7 +119,7 @@
extern void kfree_const(const void *x);
-extern char *kstrdup(const char *s, gfp_t gfp);
+extern char *kstrdup(const char *s, gfp_t gfp) __malloc;
extern const char *kstrdup_const(const char *s, gfp_t gfp);
extern char *kstrndup(const char *s, size_t len, gfp_t gfp);
extern void *kmemdup(const void *src, size_t len, gfp_t gfp);
diff --git a/include/linux/string_helpers.h b/include/linux/string_helpers.h
index dabe643..5ce9538 100644
--- a/include/linux/string_helpers.h
+++ b/include/linux/string_helpers.h
@@ -3,6 +3,8 @@
#include <linux/types.h>
+struct file;
+
/* Descriptions of the types of units to
* print in */
enum string_size_units {
@@ -68,4 +70,8 @@
return string_escape_str(src, dst, sz, ESCAPE_ANY_NP, only);
}
+char *kstrdup_quotable(const char *src, gfp_t gfp);
+char *kstrdup_quotable_cmdline(struct task_struct *task, gfp_t gfp);
+char *kstrdup_quotable_file(struct file *file, gfp_t gfp);
+
#endif
diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h
index 3081339..d6917b8 100644
--- a/include/linux/sunrpc/svc_rdma.h
+++ b/include/linux/sunrpc/svc_rdma.h
@@ -199,7 +199,7 @@
struct xdr_buf *rcvbuf);
/* svc_rdma_marshal.c */
-extern int svc_rdma_xdr_decode_req(struct rpcrdma_msg *, struct svc_rqst *);
+extern int svc_rdma_xdr_decode_req(struct xdr_buf *);
extern int svc_rdma_xdr_encode_error(struct svcxprt_rdma *,
struct rpcrdma_msg *,
enum rpcrdma_errcode, __be32 *);
diff --git a/include/linux/swap.h b/include/linux/swap.h
index ad22035..0af2bb2 100644
--- a/include/linux/swap.h
+++ b/include/linux/swap.h
@@ -316,6 +316,7 @@
struct vm_area_struct *vma);
/* linux/mm/vmscan.c */
+extern unsigned long zone_reclaimable_pages(struct zone *zone);
extern unsigned long try_to_free_pages(struct zonelist *zonelist, int order,
gfp_t gfp_mask, nodemask_t *mask);
extern int __isolate_lru_page(struct page *page, isolate_mode_t mode);
diff --git a/include/linux/sync_file.h b/include/linux/sync_file.h
new file mode 100644
index 0000000..c6ffe8b
--- /dev/null
+++ b/include/linux/sync_file.h
@@ -0,0 +1,57 @@
+/*
+ * include/linux/sync_file.h
+ *
+ * Copyright (C) 2012 Google, Inc.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#ifndef _LINUX_SYNC_FILE_H
+#define _LINUX_SYNC_FILE_H
+
+#include <linux/types.h>
+#include <linux/kref.h>
+#include <linux/ktime.h>
+#include <linux/list.h>
+#include <linux/spinlock.h>
+#include <linux/fence.h>
+
+struct sync_file_cb {
+ struct fence_cb cb;
+ struct fence *fence;
+ struct sync_file *sync_file;
+};
+
+/**
+ * struct sync_file - sync file to export to the userspace
+ * @file: file representing this fence
+ * @kref: reference count on fence.
+ * @name: name of sync_file. Useful for debugging
+ * @sync_file_list: membership in global file list
+ * @num_fences: number of sync_pts in the fence
+ * @wq: wait queue for fence signaling
+ * @status: 0: signaled, >0:active, <0: error
+ * @cbs: sync_pts callback information
+ */
+struct sync_file {
+ struct file *file;
+ struct kref kref;
+ char name[32];
+#ifdef CONFIG_DEBUG_FS
+ struct list_head sync_file_list;
+#endif
+ int num_fences;
+
+ wait_queue_head_t wq;
+ atomic_t status;
+
+ struct sync_file_cb cbs[];
+};
+
+struct sync_file *sync_file_create(struct fence *fence);
+
+#endif /* _LINUX_SYNC_H */
diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h
index d795472..d022390 100644
--- a/include/linux/syscalls.h
+++ b/include/linux/syscalls.h
@@ -371,10 +371,10 @@
size_t sigsetsize);
asmlinkage long sys_rt_tgsigqueueinfo(pid_t tgid, pid_t pid, int sig,
siginfo_t __user *uinfo);
-asmlinkage long sys_kill(int pid, int sig);
-asmlinkage long sys_tgkill(int tgid, int pid, int sig);
-asmlinkage long sys_tkill(int pid, int sig);
-asmlinkage long sys_rt_sigqueueinfo(int pid, int sig, siginfo_t __user *uinfo);
+asmlinkage long sys_kill(pid_t pid, int sig);
+asmlinkage long sys_tgkill(pid_t tgid, pid_t pid, int sig);
+asmlinkage long sys_tkill(pid_t pid, int sig);
+asmlinkage long sys_rt_sigqueueinfo(pid_t pid, int sig, siginfo_t __user *uinfo);
asmlinkage long sys_sgetmask(void);
asmlinkage long sys_ssetmask(int newmask);
asmlinkage long sys_signal(int sig, __sighandler_t handler);
diff --git a/include/linux/time64.h b/include/linux/time64.h
index 367d5af..7e5d2fa 100644
--- a/include/linux/time64.h
+++ b/include/linux/time64.h
@@ -65,7 +65,6 @@
# define timespec64_equal timespec_equal
# define timespec64_compare timespec_compare
# define set_normalized_timespec64 set_normalized_timespec
-# define timespec64_add_safe timespec_add_safe
# define timespec64_add timespec_add
# define timespec64_sub timespec_sub
# define timespec64_valid timespec_valid
@@ -134,15 +133,6 @@
extern void set_normalized_timespec64(struct timespec64 *ts, time64_t sec, s64 nsec);
-/*
- * timespec64_add_safe assumes both values are positive and checks for
- * overflow. It will return TIME_T_MAX if the returned value would be
- * smaller then either of the arguments.
- */
-extern struct timespec64 timespec64_add_safe(const struct timespec64 lhs,
- const struct timespec64 rhs);
-
-
static inline struct timespec64 timespec64_add(struct timespec64 lhs,
struct timespec64 rhs)
{
@@ -224,4 +214,11 @@
#endif
+/*
+ * timespec64_add_safe assumes both values are positive and checks for
+ * overflow. It will return TIME64_MAX in case of overflow.
+ */
+extern struct timespec64 timespec64_add_safe(const struct timespec64 lhs,
+ const struct timespec64 rhs);
+
#endif /* _LINUX_TIME64_H */
diff --git a/include/linux/timer.h b/include/linux/timer.h
index 61aa61d..20ac746 100644
--- a/include/linux/timer.h
+++ b/include/linux/timer.h
@@ -145,6 +145,8 @@
#define setup_timer(timer, fn, data) \
__setup_timer((timer), (fn), (data), 0)
+#define setup_deferrable_timer(timer, fn, data) \
+ __setup_timer((timer), (fn), (data), TIMER_DEFERRABLE)
#define setup_timer_on_stack(timer, fn, data) \
__setup_timer_on_stack((timer), (fn), (data), 0)
#define setup_deferrable_timer_on_stack(timer, fn, data) \
diff --git a/include/linux/trace_events.h b/include/linux/trace_events.h
index 222f6aa..be00761 100644
--- a/include/linux/trace_events.h
+++ b/include/linux/trace_events.h
@@ -154,21 +154,6 @@
struct trace_event_file *trace_file,
int type, unsigned long len,
unsigned long flags, int pc);
-struct ring_buffer_event *
-trace_current_buffer_lock_reserve(struct ring_buffer **current_buffer,
- int type, unsigned long len,
- unsigned long flags, int pc);
-void trace_buffer_unlock_commit(struct trace_array *tr,
- struct ring_buffer *buffer,
- struct ring_buffer_event *event,
- unsigned long flags, int pc);
-void trace_buffer_unlock_commit_regs(struct trace_array *tr,
- struct ring_buffer *buffer,
- struct ring_buffer_event *event,
- unsigned long flags, int pc,
- struct pt_regs *regs);
-void trace_current_buffer_discard_commit(struct ring_buffer *buffer,
- struct ring_buffer_event *event);
void tracing_record_cmdline(struct task_struct *tsk);
@@ -229,7 +214,6 @@
TRACE_EVENT_FL_NO_SET_FILTER_BIT,
TRACE_EVENT_FL_IGNORE_ENABLE_BIT,
TRACE_EVENT_FL_WAS_ENABLED_BIT,
- TRACE_EVENT_FL_USE_CALL_FILTER_BIT,
TRACE_EVENT_FL_TRACEPOINT_BIT,
TRACE_EVENT_FL_KPROBE_BIT,
TRACE_EVENT_FL_UPROBE_BIT,
@@ -244,7 +228,6 @@
* WAS_ENABLED - Set and stays set when an event was ever enabled
* (used for module unloading, if a module event is enabled,
* it is best to clear the buffers that used it).
- * USE_CALL_FILTER - For trace internal events, don't use file filter
* TRACEPOINT - Event is a tracepoint
* KPROBE - Event is a kprobe
* UPROBE - Event is a uprobe
@@ -255,7 +238,6 @@
TRACE_EVENT_FL_NO_SET_FILTER = (1 << TRACE_EVENT_FL_NO_SET_FILTER_BIT),
TRACE_EVENT_FL_IGNORE_ENABLE = (1 << TRACE_EVENT_FL_IGNORE_ENABLE_BIT),
TRACE_EVENT_FL_WAS_ENABLED = (1 << TRACE_EVENT_FL_WAS_ENABLED_BIT),
- TRACE_EVENT_FL_USE_CALL_FILTER = (1 << TRACE_EVENT_FL_USE_CALL_FILTER_BIT),
TRACE_EVENT_FL_TRACEPOINT = (1 << TRACE_EVENT_FL_TRACEPOINT_BIT),
TRACE_EVENT_FL_KPROBE = (1 << TRACE_EVENT_FL_KPROBE_BIT),
TRACE_EVENT_FL_UPROBE = (1 << TRACE_EVENT_FL_UPROBE_BIT),
@@ -407,16 +389,12 @@
ETT_SNAPSHOT = (1 << 1),
ETT_STACKTRACE = (1 << 2),
ETT_EVENT_ENABLE = (1 << 3),
+ ETT_EVENT_HIST = (1 << 4),
+ ETT_HIST_ENABLE = (1 << 5),
};
extern int filter_match_preds(struct event_filter *filter, void *rec);
-extern int filter_check_discard(struct trace_event_file *file, void *rec,
- struct ring_buffer *buffer,
- struct ring_buffer_event *event);
-extern int call_filter_check_discard(struct trace_event_call *call, void *rec,
- struct ring_buffer *buffer,
- struct ring_buffer_event *event);
extern enum event_trigger_type event_triggers_call(struct trace_event_file *file,
void *rec);
extern void event_triggers_post_call(struct trace_event_file *file,
@@ -450,100 +428,6 @@
return false;
}
-/*
- * Helper function for event_trigger_unlock_commit{_regs}().
- * If there are event triggers attached to this event that requires
- * filtering against its fields, then they wil be called as the
- * entry already holds the field information of the current event.
- *
- * It also checks if the event should be discarded or not.
- * It is to be discarded if the event is soft disabled and the
- * event was only recorded to process triggers, or if the event
- * filter is active and this event did not match the filters.
- *
- * Returns true if the event is discarded, false otherwise.
- */
-static inline bool
-__event_trigger_test_discard(struct trace_event_file *file,
- struct ring_buffer *buffer,
- struct ring_buffer_event *event,
- void *entry,
- enum event_trigger_type *tt)
-{
- unsigned long eflags = file->flags;
-
- if (eflags & EVENT_FILE_FL_TRIGGER_COND)
- *tt = event_triggers_call(file, entry);
-
- if (test_bit(EVENT_FILE_FL_SOFT_DISABLED_BIT, &file->flags))
- ring_buffer_discard_commit(buffer, event);
- else if (!filter_check_discard(file, entry, buffer, event))
- return false;
-
- return true;
-}
-
-/**
- * event_trigger_unlock_commit - handle triggers and finish event commit
- * @file: The file pointer assoctiated to the event
- * @buffer: The ring buffer that the event is being written to
- * @event: The event meta data in the ring buffer
- * @entry: The event itself
- * @irq_flags: The state of the interrupts at the start of the event
- * @pc: The state of the preempt count at the start of the event.
- *
- * This is a helper function to handle triggers that require data
- * from the event itself. It also tests the event against filters and
- * if the event is soft disabled and should be discarded.
- */
-static inline void
-event_trigger_unlock_commit(struct trace_event_file *file,
- struct ring_buffer *buffer,
- struct ring_buffer_event *event,
- void *entry, unsigned long irq_flags, int pc)
-{
- enum event_trigger_type tt = ETT_NONE;
-
- if (!__event_trigger_test_discard(file, buffer, event, entry, &tt))
- trace_buffer_unlock_commit(file->tr, buffer, event, irq_flags, pc);
-
- if (tt)
- event_triggers_post_call(file, tt, entry);
-}
-
-/**
- * event_trigger_unlock_commit_regs - handle triggers and finish event commit
- * @file: The file pointer assoctiated to the event
- * @buffer: The ring buffer that the event is being written to
- * @event: The event meta data in the ring buffer
- * @entry: The event itself
- * @irq_flags: The state of the interrupts at the start of the event
- * @pc: The state of the preempt count at the start of the event.
- *
- * This is a helper function to handle triggers that require data
- * from the event itself. It also tests the event against filters and
- * if the event is soft disabled and should be discarded.
- *
- * Same as event_trigger_unlock_commit() but calls
- * trace_buffer_unlock_commit_regs() instead of trace_buffer_unlock_commit().
- */
-static inline void
-event_trigger_unlock_commit_regs(struct trace_event_file *file,
- struct ring_buffer *buffer,
- struct ring_buffer_event *event,
- void *entry, unsigned long irq_flags, int pc,
- struct pt_regs *regs)
-{
- enum event_trigger_type tt = ETT_NONE;
-
- if (!__event_trigger_test_discard(file, buffer, event, entry, &tt))
- trace_buffer_unlock_commit_regs(file->tr, buffer, event,
- irq_flags, pc, regs);
-
- if (tt)
- event_triggers_post_call(file, tt, entry);
-}
-
#ifdef CONFIG_BPF_EVENTS
unsigned int trace_call_bpf(struct bpf_prog *prog, void *ctx);
#else
diff --git a/include/linux/tty.h b/include/linux/tty.h
index 3b09f23..40144f3 100644
--- a/include/linux/tty.h
+++ b/include/linux/tty.h
@@ -228,7 +228,8 @@
int count; /* Usage count */
wait_queue_head_t open_wait; /* Open waiters */
wait_queue_head_t delta_msr_wait; /* Modem status change */
- unsigned long flags; /* TTY flags ASY_*/
+ unsigned long flags; /* User TTY flags ASYNC_ */
+ unsigned long iflags; /* Internal flags TTY_PORT_ */
unsigned char console:1, /* port is a console */
low_latency:1; /* optional: tune for latency */
struct mutex mutex; /* Locking */
@@ -242,6 +243,18 @@
struct kref kref; /* Ref counter */
};
+/* tty_port::iflags bits -- use atomic bit ops */
+#define TTY_PORT_INITIALIZED 0 /* device is initialized */
+#define TTY_PORT_SUSPENDED 1 /* device is suspended */
+#define TTY_PORT_ACTIVE 2 /* device is open */
+
+/*
+ * uart drivers: use the uart_port::status field and the UPSTAT_* defines
+ * for s/w-based flow control steering and carrier detection status
+ */
+#define TTY_PORT_CTS_FLOW 3 /* h/w flow control enabled */
+#define TTY_PORT_CHECK_CD 4 /* carrier detect enabled */
+
/*
* Where all of the state associated with a tty is kept while the tty
* is open. Since the termios state should be kept even if the tty
@@ -338,7 +351,6 @@
#define TTY_OTHER_CLOSED 2 /* Other side (if any) has closed */
#define TTY_EXCLUSIVE 3 /* Exclusive open mode */
#define TTY_DO_WRITE_WAKEUP 5 /* Call write_wakeup after queuing new */
-#define TTY_OTHER_DONE 6 /* Closed pty has completed input processing */
#define TTY_LDISC_OPEN 11 /* Line discipline is open */
#define TTY_PTY_LOCK 16 /* pty private */
#define TTY_NO_WRITE_SPLIT 17 /* Preserve write boundaries to driver */
@@ -360,6 +372,16 @@
smp_mb();
}
+static inline bool tty_io_error(struct tty_struct *tty)
+{
+ return test_bit(TTY_IO_ERROR, &tty->flags);
+}
+
+static inline bool tty_throttled(struct tty_struct *tty)
+{
+ return test_bit(TTY_THROTTLED, &tty->flags);
+}
+
#ifdef CONFIG_TTY
extern void console_init(void);
extern void tty_kref_put(struct tty_struct *tty);
@@ -371,6 +393,7 @@
extern struct tty_struct *get_current_tty(void);
/* tty_io.c */
extern int __init tty_init(void);
+extern const char *tty_name(const struct tty_struct *tty);
#else
static inline void console_init(void)
{ }
@@ -391,6 +414,8 @@
/* tty_io.c */
static inline int __init tty_init(void)
{ return 0; }
+static inline const char *tty_name(const struct tty_struct *tty)
+{ return "(none)"; }
#endif
extern struct ktermios tty_std_termios;
@@ -415,7 +440,6 @@
return tty;
}
-extern const char *tty_name(const struct tty_struct *tty);
extern const char *tty_driver_name(const struct tty_struct *tty);
extern void tty_wait_until_sent(struct tty_struct *tty, long timeout);
extern int __tty_check_change(struct tty_struct *tty, int sig);
@@ -457,6 +481,7 @@
extern void tty_buffer_set_lock_subclass(struct tty_port *port);
extern bool tty_buffer_restart_work(struct tty_port *port);
extern bool tty_buffer_cancel_work(struct tty_port *port);
+extern void tty_buffer_flush_work(struct tty_port *port);
extern speed_t tty_termios_baud_rate(struct ktermios *termios);
extern speed_t tty_termios_input_baud_rate(struct ktermios *termios);
extern void tty_termios_encode_baud_rate(struct ktermios *termios,
@@ -537,7 +562,67 @@
/* If the cts flow control is enabled, return true. */
static inline bool tty_port_cts_enabled(struct tty_port *port)
{
- return port->flags & ASYNC_CTS_FLOW;
+ return test_bit(TTY_PORT_CTS_FLOW, &port->iflags);
+}
+
+static inline void tty_port_set_cts_flow(struct tty_port *port, bool val)
+{
+ if (val)
+ set_bit(TTY_PORT_CTS_FLOW, &port->iflags);
+ else
+ clear_bit(TTY_PORT_CTS_FLOW, &port->iflags);
+}
+
+static inline bool tty_port_active(struct tty_port *port)
+{
+ return test_bit(TTY_PORT_ACTIVE, &port->iflags);
+}
+
+static inline void tty_port_set_active(struct tty_port *port, bool val)
+{
+ if (val)
+ set_bit(TTY_PORT_ACTIVE, &port->iflags);
+ else
+ clear_bit(TTY_PORT_ACTIVE, &port->iflags);
+}
+
+static inline bool tty_port_check_carrier(struct tty_port *port)
+{
+ return test_bit(TTY_PORT_CHECK_CD, &port->iflags);
+}
+
+static inline void tty_port_set_check_carrier(struct tty_port *port, bool val)
+{
+ if (val)
+ set_bit(TTY_PORT_CHECK_CD, &port->iflags);
+ else
+ clear_bit(TTY_PORT_CHECK_CD, &port->iflags);
+}
+
+static inline bool tty_port_suspended(struct tty_port *port)
+{
+ return test_bit(TTY_PORT_SUSPENDED, &port->iflags);
+}
+
+static inline void tty_port_set_suspended(struct tty_port *port, bool val)
+{
+ if (val)
+ set_bit(TTY_PORT_SUSPENDED, &port->iflags);
+ else
+ clear_bit(TTY_PORT_SUSPENDED, &port->iflags);
+}
+
+static inline bool tty_port_initialized(struct tty_port *port)
+{
+ return test_bit(TTY_PORT_INITIALIZED, &port->iflags);
+}
+
+static inline void tty_port_set_initialized(struct tty_port *port, bool val)
+{
+ if (val)
+ set_bit(TTY_PORT_INITIALIZED, &port->iflags);
+ else
+ clear_bit(TTY_PORT_INITIALIZED, &port->iflags);
}
extern struct tty_struct *tty_port_tty_get(struct tty_port *port);
diff --git a/include/linux/types.h b/include/linux/types.h
index 70dd3df..baf7183 100644
--- a/include/linux/types.h
+++ b/include/linux/types.h
@@ -156,7 +156,6 @@
typedef unsigned __bitwise__ gfp_t;
typedef unsigned __bitwise__ fmode_t;
-typedef unsigned __bitwise__ oom_flags_t;
#ifdef CONFIG_PHYS_ADDR_T_64BIT
typedef u64 phys_addr_t;
diff --git a/include/linux/usb.h b/include/linux/usb.h
index 6a9a0c2..eba1f10 100644
--- a/include/linux/usb.h
+++ b/include/linux/usb.h
@@ -374,13 +374,12 @@
int devnum_next; /* Next open device number in
* round-robin allocation */
+ struct mutex devnum_next_mutex; /* devnum_next mutex */
struct usb_devmap devmap; /* device address allocation map */
struct usb_device *root_hub; /* Root hub */
struct usb_bus *hs_companion; /* Companion EHCI bus, if any */
- struct mutex usb_address0_mutex; /* unaddressed device mutex */
-
int bandwidth_allocated; /* on this bus: how much of the time
* reserved for periodic (intr/iso)
* requests is used, on average?
@@ -720,7 +719,7 @@
static inline bool usb_device_supports_ltm(struct usb_device *udev)
{
- if (udev->speed != USB_SPEED_SUPER || !udev->bos || !udev->bos->ss_cap)
+ if (udev->speed < USB_SPEED_SUPER || !udev->bos || !udev->bos->ss_cap)
return false;
return udev->bos->ss_cap->bmAttributes & USB_LTM_SUPPORT;
}
@@ -1069,7 +1068,7 @@
* for interfaces bound to this driver.
* @soft_unbind: if set to 1, the USB core will not kill URBs and disable
* endpoints before calling the driver's disconnect method.
- * @disable_hub_initiated_lpm: if set to 0, the USB core will not allow hubs
+ * @disable_hub_initiated_lpm: if set to 1, the USB core will not allow hubs
* to initiate lower power link state transitions when an idle timeout
* occurs. Device-initiated USB 3.0 link PM will still be allowed.
*
@@ -1569,7 +1568,7 @@
* Initializes a interrupt urb with the proper information needed to submit
* it to a device.
*
- * Note that High Speed and SuperSpeed interrupt endpoints use a logarithmic
+ * Note that High Speed and SuperSpeed(+) interrupt endpoints use a logarithmic
* encoding of the endpoint interval, and express polling intervals in
* microframes (eight per millisecond) rather than in frames (one per
* millisecond).
@@ -1595,7 +1594,7 @@
urb->complete = complete_fn;
urb->context = context;
- if (dev->speed == USB_SPEED_HIGH || dev->speed == USB_SPEED_SUPER) {
+ if (dev->speed == USB_SPEED_HIGH || dev->speed >= USB_SPEED_SUPER) {
/* make sure interval is within allowed range */
interval = clamp(interval, 1, 16);
diff --git a/include/linux/usb/gadget.h b/include/linux/usb/gadget.h
index 5d4e151..457651b 100644
--- a/include/linux/usb/gadget.h
+++ b/include/linux/usb/gadget.h
@@ -1223,9 +1223,13 @@
/* utility to simplify map/unmap of usb_requests to/from DMA */
+extern int usb_gadget_map_request_by_dev(struct device *dev,
+ struct usb_request *req, int is_in);
extern int usb_gadget_map_request(struct usb_gadget *gadget,
struct usb_request *req, int is_in);
+extern void usb_gadget_unmap_request_by_dev(struct device *dev,
+ struct usb_request *req, int is_in);
extern void usb_gadget_unmap_request(struct usb_gadget *gadget,
struct usb_request *req, int is_in);
diff --git a/include/linux/usb/hcd.h b/include/linux/usb/hcd.h
index b98f831..66fc137 100644
--- a/include/linux/usb/hcd.h
+++ b/include/linux/usb/hcd.h
@@ -181,6 +181,7 @@
* bandwidth_mutex should be dropped after a successful control message
* to the device, or resetting the bandwidth after a failed attempt.
*/
+ struct mutex *address0_mutex;
struct mutex *bandwidth_mutex;
struct usb_hcd *shared_hcd;
struct usb_hcd *primary_hcd;
diff --git a/include/linux/usb/otg-fsm.h b/include/linux/usb/otg-fsm.h
index 24198e1..7a03505 100644
--- a/include/linux/usb/otg-fsm.h
+++ b/include/linux/usb/otg-fsm.h
@@ -72,37 +72,113 @@
NUM_OTG_FSM_TIMERS,
};
-/* OTG state machine according to the OTG spec */
+/**
+ * struct otg_fsm - OTG state machine according to the OTG spec
+ *
+ * OTG hardware Inputs
+ *
+ * Common inputs for A and B device
+ * @id: TRUE for B-device, FALSE for A-device.
+ * @adp_change: TRUE when current ADP measurement (n) value, compared to the
+ * ADP measurement taken at n-2, differs by more than CADP_THR
+ * @power_up: TRUE when the OTG device first powers up its USB system and
+ * ADP measurement taken if ADP capable
+ *
+ * A-Device state inputs
+ * @a_srp_det: TRUE if the A-device detects SRP
+ * @a_vbus_vld: TRUE when VBUS voltage is in regulation
+ * @b_conn: TRUE if the A-device detects connection from the B-device
+ * @a_bus_resume: TRUE when the B-device detects that the A-device is signaling
+ * a resume (K state)
+ * B-Device state inputs
+ * @a_bus_suspend: TRUE when the B-device detects that the A-device has put the
+ * bus into suspend
+ * @a_conn: TRUE if the B-device detects a connection from the A-device
+ * @b_se0_srp: TRUE when the line has been at SE0 for more than the minimum
+ * time before generating SRP
+ * @b_ssend_srp: TRUE when the VBUS has been below VOTG_SESS_VLD for more than
+ * the minimum time before generating SRP
+ * @b_sess_vld: TRUE when the B-device detects that the voltage on VBUS is
+ * above VOTG_SESS_VLD
+ * @test_device: TRUE when the B-device switches to B-Host and detects an OTG
+ * test device. This must be set by host/hub driver
+ *
+ * Application inputs (A-Device)
+ * @a_bus_drop: TRUE when A-device application needs to power down the bus
+ * @a_bus_req: TRUE when A-device application wants to use the bus.
+ * FALSE to suspend the bus
+ *
+ * Application inputs (B-Device)
+ * @b_bus_req: TRUE during the time that the Application running on the
+ * B-device wants to use the bus
+ *
+ * Auxilary inputs (OTG v1.3 only. Obsolete now.)
+ * @a_sess_vld: TRUE if the A-device detects that VBUS is above VA_SESS_VLD
+ * @b_bus_suspend: TRUE when the A-device detects that the B-device has put
+ * the bus into suspend
+ * @b_bus_resume: TRUE when the A-device detects that the B-device is signaling
+ * resume on the bus
+ *
+ * OTG Output status. Read only for users. Updated by OTG FSM helpers defined
+ * in this file
+ *
+ * Outputs for Both A and B device
+ * @drv_vbus: TRUE when A-device is driving VBUS
+ * @loc_conn: TRUE when the local device has signaled that it is connected
+ * to the bus
+ * @loc_sof: TRUE when the local device is generating activity on the bus
+ * @adp_prb: TRUE when the local device is in the process of doing
+ * ADP probing
+ *
+ * Outputs for B-device state
+ * @adp_sns: TRUE when the B-device is in the process of carrying out
+ * ADP sensing
+ * @data_pulse: TRUE when the B-device is performing data line pulsing
+ *
+ * Internal Variables
+ *
+ * a_set_b_hnp_en: TRUE when the A-device has successfully set the
+ * b_hnp_enable bit in the B-device.
+ * Unused as OTG fsm uses otg->host->b_hnp_enable instead
+ * b_srp_done: TRUE when the B-device has completed initiating SRP
+ * b_hnp_enable: TRUE when the B-device has accepted the
+ * SetFeature(b_hnp_enable) B-device.
+ * Unused as OTG fsm uses otg->gadget->b_hnp_enable instead
+ * a_clr_err: Asserted (by application ?) to clear a_vbus_err due to an
+ * overcurrent condition and causes the A-device to transition
+ * to a_wait_vfall
+ */
struct otg_fsm {
/* Input */
int id;
int adp_change;
int power_up;
- int test_device;
- int a_bus_drop;
- int a_bus_req;
int a_srp_det;
int a_vbus_vld;
int b_conn;
int a_bus_resume;
int a_bus_suspend;
int a_conn;
- int b_bus_req;
int b_se0_srp;
int b_ssend_srp;
int b_sess_vld;
+ int test_device;
+ int a_bus_drop;
+ int a_bus_req;
+ int b_bus_req;
+
/* Auxilary inputs */
int a_sess_vld;
int b_bus_resume;
int b_bus_suspend;
/* Output */
- int data_pulse;
int drv_vbus;
int loc_conn;
int loc_sof;
int adp_prb;
int adp_sns;
+ int data_pulse;
/* Internal variables */
int a_set_b_hnp_en;
@@ -110,7 +186,7 @@
int b_hnp_enable;
int a_clr_err;
- /* Informative variables */
+ /* Informative variables. All unused as of now */
int a_bus_drop_inf;
int a_bus_req_inf;
int a_clr_err_inf;
@@ -134,6 +210,7 @@
struct mutex lock;
u8 *host_req_flag;
struct delayed_work hnp_polling_work;
+ bool state_changed;
};
struct otg_fsm_ops {
diff --git a/include/linux/uuid.h b/include/linux/uuid.h
index 6df2509..2d095fc 100644
--- a/include/linux/uuid.h
+++ b/include/linux/uuid.h
@@ -1,7 +1,7 @@
/*
* UUID/GUID definition
*
- * Copyright (C) 2010, Intel Corp.
+ * Copyright (C) 2010, 2016 Intel Corp.
* Huang Ying <ying.huang@intel.com>
*
* This program is free software; you can redistribute it and/or
@@ -12,16 +12,17 @@
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#ifndef _LINUX_UUID_H_
#define _LINUX_UUID_H_
#include <uapi/linux/uuid.h>
+/*
+ * The length of a UUID string ("aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee")
+ * not including trailing NUL.
+ */
+#define UUID_STRING_LEN 36
static inline int uuid_le_cmp(const uuid_le u1, const uuid_le u2)
{
@@ -33,7 +34,17 @@
return memcmp(&u1, &u2, sizeof(uuid_be));
}
+void generate_random_uuid(unsigned char uuid[16]);
+
extern void uuid_le_gen(uuid_le *u);
extern void uuid_be_gen(uuid_be *u);
+bool __must_check uuid_is_valid(const char *uuid);
+
+extern const u8 uuid_le_index[16];
+extern const u8 uuid_be_index[16];
+
+int uuid_le_to_bin(const char *uuid, uuid_le *u);
+int uuid_be_to_bin(const char *uuid, uuid_be *u);
+
#endif
diff --git a/include/linux/verification.h b/include/linux/verification.h
new file mode 100644
index 0000000..a10549a
--- /dev/null
+++ b/include/linux/verification.h
@@ -0,0 +1,49 @@
+/* Signature verification
+ *
+ * Copyright (C) 2014 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public Licence
+ * as published by the Free Software Foundation; either version
+ * 2 of the Licence, or (at your option) any later version.
+ */
+
+#ifndef _LINUX_VERIFICATION_H
+#define _LINUX_VERIFICATION_H
+
+/*
+ * The use to which an asymmetric key is being put.
+ */
+enum key_being_used_for {
+ VERIFYING_MODULE_SIGNATURE,
+ VERIFYING_FIRMWARE_SIGNATURE,
+ VERIFYING_KEXEC_PE_SIGNATURE,
+ VERIFYING_KEY_SIGNATURE,
+ VERIFYING_KEY_SELF_SIGNATURE,
+ VERIFYING_UNSPECIFIED_SIGNATURE,
+ NR__KEY_BEING_USED_FOR
+};
+extern const char *const key_being_used_for[NR__KEY_BEING_USED_FOR];
+
+#ifdef CONFIG_SYSTEM_DATA_VERIFICATION
+
+struct key;
+
+extern int verify_pkcs7_signature(const void *data, size_t len,
+ const void *raw_pkcs7, size_t pkcs7_len,
+ struct key *trusted_keys,
+ enum key_being_used_for usage,
+ int (*view_content)(void *ctx,
+ const void *data, size_t len,
+ size_t asn1hdrlen),
+ void *ctx);
+
+#ifdef CONFIG_SIGNED_PE_FILE_VERIFICATION
+extern int verify_pefile_signature(const void *pebuf, unsigned pelen,
+ struct key *trusted_keys,
+ enum key_being_used_for usage);
+#endif
+
+#endif /* CONFIG_SYSTEM_DATA_VERIFICATION */
+#endif /* _LINUX_VERIFY_PEFILE_H */
diff --git a/include/linux/verify_pefile.h b/include/linux/verify_pefile.h
deleted file mode 100644
index da2049b..0000000
--- a/include/linux/verify_pefile.h
+++ /dev/null
@@ -1,22 +0,0 @@
-/* Signed PE file verification
- *
- * Copyright (C) 2014 Red Hat, Inc. All Rights Reserved.
- * Written by David Howells (dhowells@redhat.com)
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public Licence
- * as published by the Free Software Foundation; either version
- * 2 of the Licence, or (at your option) any later version.
- */
-
-#ifndef _LINUX_VERIFY_PEFILE_H
-#define _LINUX_VERIFY_PEFILE_H
-
-#include <crypto/public_key.h>
-
-extern int verify_pefile_signature(const void *pebuf, unsigned pelen,
- struct key *trusted_keyring,
- enum key_being_used_for usage,
- bool *_trusted);
-
-#endif /* _LINUX_VERIFY_PEFILE_H */
diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h
index d1f1d33..3d9d786a 100644
--- a/include/linux/vmalloc.h
+++ b/include/linux/vmalloc.h
@@ -4,10 +4,12 @@
#include <linux/spinlock.h>
#include <linux/init.h>
#include <linux/list.h>
+#include <linux/llist.h>
#include <asm/page.h> /* pgprot_t */
#include <linux/rbtree.h>
struct vm_area_struct; /* vma defining user mapping in mm_types.h */
+struct notifier_block; /* in notifier.h */
/* bits in flags of vmalloc's vm_struct below */
#define VM_IOREMAP 0x00000001 /* ioremap() and friends */
@@ -44,7 +46,7 @@
unsigned long flags;
struct rb_node rb_node; /* address sorted rbtree */
struct list_head list; /* address sorted list */
- struct list_head purge_list; /* "lazy purge" list */
+ struct llist_node purge_list; /* "lazy purge" list */
struct vm_struct *vm;
struct rcu_head rcu_head;
};
@@ -187,4 +189,7 @@
#define VMALLOC_TOTAL 0UL
#endif
+int register_vmap_purge_notifier(struct notifier_block *nb);
+int unregister_vmap_purge_notifier(struct notifier_block *nb);
+
#endif /* _LINUX_VMALLOC_H */
diff --git a/include/linux/vmstat.h b/include/linux/vmstat.h
index 73fae8c..d2da8e0 100644
--- a/include/linux/vmstat.h
+++ b/include/linux/vmstat.h
@@ -163,12 +163,10 @@
#ifdef CONFIG_NUMA
extern unsigned long node_page_state(int node, enum zone_stat_item item);
-extern void zone_statistics(struct zone *, struct zone *, gfp_t gfp);
#else
#define node_page_state(node, item) global_page_state(item)
-#define zone_statistics(_zl, _z, gfp) do { } while (0)
#endif /* CONFIG_NUMA */
@@ -193,6 +191,10 @@
void cpu_vm_stats_fold(int cpu);
void refresh_zone_stat_thresholds(void);
+struct ctl_table;
+int vmstat_refresh(struct ctl_table *, int write,
+ void __user *buffer, size_t *lenp, loff_t *ppos);
+
void drain_zonestat(struct zone *zone, struct per_cpu_pageset *);
int calculate_pressure_threshold(struct zone *zone);
diff --git a/include/linux/zsmalloc.h b/include/linux/zsmalloc.h
index 34eb160..57a8e98 100644
--- a/include/linux/zsmalloc.h
+++ b/include/linux/zsmalloc.h
@@ -41,10 +41,10 @@
struct zs_pool;
-struct zs_pool *zs_create_pool(const char *name, gfp_t flags);
+struct zs_pool *zs_create_pool(const char *name);
void zs_destroy_pool(struct zs_pool *pool);
-unsigned long zs_malloc(struct zs_pool *pool, size_t size);
+unsigned long zs_malloc(struct zs_pool *pool, size_t size, gfp_t flags);
void zs_free(struct zs_pool *pool, unsigned long obj);
void *zs_map_object(struct zs_pool *pool, unsigned long handle,
diff --git a/include/media/media-device.h b/include/media/media-device.h
index df74cfa..a9b33c4 100644
--- a/include/media/media-device.h
+++ b/include/media/media-device.h
@@ -25,7 +25,6 @@
#include <linux/list.h>
#include <linux/mutex.h>
-#include <linux/spinlock.h>
#include <media/media-devnode.h>
#include <media/media-entity.h>
@@ -304,8 +303,7 @@
* @pads: List of registered pads
* @links: List of registered links
* @entity_notify: List of registered entity_notify callbacks
- * @lock: Entities list lock
- * @graph_mutex: Entities graph operation lock
+ * @graph_mutex: Protects access to struct media_device data
* @pm_count_walk: Graph walk for power state walk. Access serialised using
* graph_mutex.
*
@@ -313,7 +311,8 @@
* @enable_source: Enable Source Handler function pointer
* @disable_source: Disable Source Handler function pointer
*
- * @link_notify: Link state change notification callback
+ * @link_notify: Link state change notification callback. This callback is
+ * called with the graph_mutex held.
*
* This structure represents an abstract high-level media device. It allows easy
* access to entities and provides basic media device-level support. The
@@ -357,7 +356,7 @@
u32 hw_revision;
u32 driver_version;
- u32 topology_version;
+ u64 topology_version;
u32 id;
struct ida entity_internal_idx;
@@ -371,8 +370,6 @@
/* notify callback list invoked when a new entity is registered */
struct list_head entity_notify;
- /* Protects the graph objects creation/removal */
- spinlock_t lock;
/* Serializes graph operations. */
struct mutex graph_mutex;
struct media_entity_graph pm_count_walk;
@@ -494,7 +491,7 @@
#define media_device_register(mdev) __media_device_register(mdev, THIS_MODULE)
/**
- * __media_device_unregister() - Unegisters a media device element
+ * media_device_unregister() - Unregisters a media device element
*
* @mdev: pointer to struct &media_device
*
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 6dc9e4e..cbb266f 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -179,6 +179,9 @@
* @link_validate: Return whether a link is valid from the entity point of
* view. The media_entity_pipeline_start() function
* validates all links by calling this operation. Optional.
+ *
+ * Note: Those these callbacks are called with struct media_device.@graph_mutex
+ * mutex held.
*/
struct media_entity_operations {
int (*link_setup)(struct media_entity *entity,
@@ -188,10 +191,38 @@
};
/**
+ * enum media_entity_type - Media entity type
+ *
+ * @MEDIA_ENTITY_TYPE_BASE:
+ * The entity isn't embedded in another subsystem structure.
+ * @MEDIA_ENTITY_TYPE_VIDEO_DEVICE:
+ * The entity is embedded in a struct video_device instance.
+ * @MEDIA_ENTITY_TYPE_V4L2_SUBDEV:
+ * The entity is embedded in a struct v4l2_subdev instance.
+ *
+ * Media entity objects are often not instantiated directly, but the media
+ * entity structure is inherited by (through embedding) other subsystem-specific
+ * structures. The media entity type identifies the type of the subclass
+ * structure that implements a media entity instance.
+ *
+ * This allows runtime type identification of media entities and safe casting to
+ * the correct object type. For instance, a media entity structure instance
+ * embedded in a v4l2_subdev structure instance will have the type
+ * MEDIA_ENTITY_TYPE_V4L2_SUBDEV and can safely be cast to a v4l2_subdev
+ * structure using the container_of() macro.
+ */
+enum media_entity_type {
+ MEDIA_ENTITY_TYPE_BASE,
+ MEDIA_ENTITY_TYPE_VIDEO_DEVICE,
+ MEDIA_ENTITY_TYPE_V4L2_SUBDEV,
+};
+
+/**
* struct media_entity - A media entity graph object.
*
* @graph_obj: Embedded structure containing the media object common data.
* @name: Entity name.
+ * @obj_type: Type of the object that implements the media_entity.
* @function: Entity main function, as defined in uapi/media.h
* (MEDIA_ENT_F_*)
* @flags: Entity flags, as defined in uapi/media.h (MEDIA_ENT_FL_*)
@@ -220,6 +251,7 @@
struct media_entity {
struct media_gobj graph_obj; /* must be first field in struct */
const char *name;
+ enum media_entity_type obj_type;
u32 function;
unsigned long flags;
@@ -329,56 +361,29 @@
}
/**
- * is_media_entity_v4l2_io() - identify if the entity main function
- * is a V4L2 I/O
- *
+ * is_media_entity_v4l2_video_device() - Check if the entity is a video_device
* @entity: pointer to entity
*
- * Return: true if the entity main function is one of the V4L2 I/O types
- * (video, VBI or SDR radio); false otherwise.
+ * Return: true if the entity is an instance of a video_device object and can
+ * safely be cast to a struct video_device using the container_of() macro, or
+ * false otherwise.
*/
-static inline bool is_media_entity_v4l2_io(struct media_entity *entity)
+static inline bool is_media_entity_v4l2_video_device(struct media_entity *entity)
{
- if (!entity)
- return false;
-
- switch (entity->function) {
- case MEDIA_ENT_F_IO_V4L:
- case MEDIA_ENT_F_IO_VBI:
- case MEDIA_ENT_F_IO_SWRADIO:
- return true;
- default:
- return false;
- }
+ return entity && entity->obj_type == MEDIA_ENTITY_TYPE_VIDEO_DEVICE;
}
/**
- * is_media_entity_v4l2_subdev - return true if the entity main function is
- * associated with the V4L2 API subdev usage
- *
+ * is_media_entity_v4l2_subdev() - Check if the entity is a v4l2_subdev
* @entity: pointer to entity
*
- * This is an ancillary function used by subdev-based V4L2 drivers.
- * It checks if the entity function is one of functions used by a V4L2 subdev,
- * e. g. camera-relatef functions, analog TV decoder, TV tuner, V4L2 DSPs.
+ * Return: true if the entity is an instance of a v4l2_subdev object and can
+ * safely be cast to a struct v4l2_subdev using the container_of() macro, or
+ * false otherwise.
*/
static inline bool is_media_entity_v4l2_subdev(struct media_entity *entity)
{
- if (!entity)
- return false;
-
- switch (entity->function) {
- case MEDIA_ENT_F_V4L2_SUBDEV_UNKNOWN:
- case MEDIA_ENT_F_CAM_SENSOR:
- case MEDIA_ENT_F_FLASH:
- case MEDIA_ENT_F_LENS:
- case MEDIA_ENT_F_ATV_DECODER:
- case MEDIA_ENT_F_TUNER:
- return true;
-
- default:
- return false;
- }
+ return entity && entity->obj_type == MEDIA_ENTITY_TYPE_V4L2_SUBDEV;
}
/**
diff --git a/include/media/rc-core.h b/include/media/rc-core.h
index 0f77b3d..b6586a9 100644
--- a/include/media/rc-core.h
+++ b/include/media/rc-core.h
@@ -215,12 +215,9 @@
struct ir_raw_event {
union {
u32 duration;
-
- struct {
- u32 carrier;
- u8 duty_cycle;
- };
+ u32 carrier;
};
+ u8 duty_cycle;
unsigned pulse:1;
unsigned reset:1;
@@ -228,13 +225,7 @@
unsigned carrier_report:1;
};
-#define DEFINE_IR_RAW_EVENT(event) \
- struct ir_raw_event event = { \
- { .duration = 0 } , \
- .pulse = 0, \
- .reset = 0, \
- .timeout = 0, \
- .carrier_report = 0 }
+#define DEFINE_IR_RAW_EVENT(event) struct ir_raw_event event = {}
static inline void init_ir_raw_event(struct ir_raw_event *ev)
{
@@ -256,8 +247,7 @@
static inline void ir_raw_event_reset(struct rc_dev *dev)
{
- DEFINE_IR_RAW_EVENT(ev);
- ev.reset = true;
+ struct ir_raw_event ev = { .reset = true };
ir_raw_event_store(dev, &ev);
ir_raw_event_handle(dev);
diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h
index 76056ab..25a3190 100644
--- a/include/media/v4l2-dev.h
+++ b/include/media/v4l2-dev.h
@@ -92,6 +92,9 @@
/* device ops */
const struct v4l2_file_operations *fops;
+ /* device capabilities as used in v4l2_capabilities */
+ u32 device_caps;
+
/* sysfs */
struct device dev; /* v4l device */
struct cdev *cdev; /* character device */
diff --git a/include/media/v4l2-device.h b/include/media/v4l2-device.h
index 9c58157..d5d45a8 100644
--- a/include/media/v4l2-device.h
+++ b/include/media/v4l2-device.h
@@ -196,11 +196,64 @@
##args); \
})
-#define v4l2_device_has_op(v4l2_dev, o, f) \
+/*
+ * Call the specified callback for all subdevs where grp_id & grpmsk != 0
+ * (if grpmsk == `0, then match them all). Ignore any errors. Note that you
+ * cannot add or delete a subdev while walking the subdevs list.
+ */
+#define v4l2_device_mask_call_all(v4l2_dev, grpmsk, o, f, args...) \
+ do { \
+ struct v4l2_subdev *__sd; \
+ \
+ __v4l2_device_call_subdevs_p(v4l2_dev, __sd, \
+ !(grpmsk) || (__sd->grp_id & (grpmsk)), o, f , \
+ ##args); \
+ } while (0)
+
+/*
+ * Call the specified callback for all subdevs where grp_id & grpmsk != 0
+ * (if grpmsk == `0, then match them all). If the callback returns an error
+ * other than 0 or -ENOIOCTLCMD, then return with that error code. Note that
+ * you cannot add or delete a subdev while walking the subdevs list.
+ */
+#define v4l2_device_mask_call_until_err(v4l2_dev, grpmsk, o, f, args...) \
+({ \
+ struct v4l2_subdev *__sd; \
+ __v4l2_device_call_subdevs_until_err_p(v4l2_dev, __sd, \
+ !(grpmsk) || (__sd->grp_id & (grpmsk)), o, f , \
+ ##args); \
+})
+
+/*
+ * Does any subdev with matching grpid (or all if grpid == 0) has the given
+ * op?
+ */
+#define v4l2_device_has_op(v4l2_dev, grpid, o, f) \
({ \
struct v4l2_subdev *__sd; \
bool __result = false; \
list_for_each_entry(__sd, &(v4l2_dev)->subdevs, list) { \
+ if ((grpid) && __sd->grp_id != (grpid)) \
+ continue; \
+ if (v4l2_subdev_has_op(__sd, o, f)) { \
+ __result = true; \
+ break; \
+ } \
+ } \
+ __result; \
+})
+
+/*
+ * Does any subdev with matching grpmsk (or all if grpmsk == 0) has the given
+ * op?
+ */
+#define v4l2_device_mask_has_op(v4l2_dev, grpmsk, o, f) \
+({ \
+ struct v4l2_subdev *__sd; \
+ bool __result = false; \
+ list_for_each_entry(__sd, &(v4l2_dev)->subdevs, list) { \
+ if ((grpmsk) && !(__sd->grp_id & (grpmsk))) \
+ continue; \
if (v4l2_subdev_has_op(__sd, o, f)) { \
__result = true; \
break; \
diff --git a/include/media/v4l2-rect.h b/include/media/v4l2-rect.h
new file mode 100644
index 0000000..d2125f0
--- /dev/null
+++ b/include/media/v4l2-rect.h
@@ -0,0 +1,173 @@
+/*
+ * v4l2-rect.h - v4l2_rect helper functions
+ *
+ * Copyright 2014 Cisco Systems, Inc. and/or its affiliates. All rights reserved.
+ *
+ * This program is free software; you may redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _V4L2_RECT_H_
+#define _V4L2_RECT_H_
+
+#include <linux/videodev2.h>
+
+/**
+ * v4l2_rect_set_size_to() - copy the width/height values.
+ * @r: rect whose width and height fields will be set
+ * @size: rect containing the width and height fields you need.
+ */
+static inline void v4l2_rect_set_size_to(struct v4l2_rect *r,
+ const struct v4l2_rect *size)
+{
+ r->width = size->width;
+ r->height = size->height;
+}
+
+/**
+ * v4l2_rect_set_min_size() - width and height of r should be >= min_size.
+ * @r: rect whose width and height will be modified
+ * @min_size: rect containing the minimal width and height
+ */
+static inline void v4l2_rect_set_min_size(struct v4l2_rect *r,
+ const struct v4l2_rect *min_size)
+{
+ if (r->width < min_size->width)
+ r->width = min_size->width;
+ if (r->height < min_size->height)
+ r->height = min_size->height;
+}
+
+/**
+ * v4l2_rect_set_max_size() - width and height of r should be <= max_size
+ * @r: rect whose width and height will be modified
+ * @max_size: rect containing the maximum width and height
+ */
+static inline void v4l2_rect_set_max_size(struct v4l2_rect *r,
+ const struct v4l2_rect *max_size)
+{
+ if (r->width > max_size->width)
+ r->width = max_size->width;
+ if (r->height > max_size->height)
+ r->height = max_size->height;
+}
+
+/**
+ * v4l2_rect_map_inside()- r should be inside boundary.
+ * @r: rect that will be modified
+ * @boundary: rect containing the boundary for @r
+ */
+static inline void v4l2_rect_map_inside(struct v4l2_rect *r,
+ const struct v4l2_rect *boundary)
+{
+ v4l2_rect_set_max_size(r, boundary);
+ if (r->left < boundary->left)
+ r->left = boundary->left;
+ if (r->top < boundary->top)
+ r->top = boundary->top;
+ if (r->left + r->width > boundary->width)
+ r->left = boundary->width - r->width;
+ if (r->top + r->height > boundary->height)
+ r->top = boundary->height - r->height;
+}
+
+/**
+ * v4l2_rect_same_size() - return true if r1 has the same size as r2
+ * @r1: rectangle.
+ * @r2: rectangle.
+ *
+ * Return true if both rectangles have the same size.
+ */
+static inline bool v4l2_rect_same_size(const struct v4l2_rect *r1,
+ const struct v4l2_rect *r2)
+{
+ return r1->width == r2->width && r1->height == r2->height;
+}
+
+/**
+ * v4l2_rect_intersect() - calculate the intersection of two rects.
+ * @r: intersection of @r1 and @r2.
+ * @r1: rectangle.
+ * @r2: rectangle.
+ */
+static inline void v4l2_rect_intersect(struct v4l2_rect *r,
+ const struct v4l2_rect *r1,
+ const struct v4l2_rect *r2)
+{
+ int right, bottom;
+
+ r->top = max(r1->top, r2->top);
+ r->left = max(r1->left, r2->left);
+ bottom = min(r1->top + r1->height, r2->top + r2->height);
+ right = min(r1->left + r1->width, r2->left + r2->width);
+ r->height = max(0, bottom - r->top);
+ r->width = max(0, right - r->left);
+}
+
+/**
+ * v4l2_rect_scale() - scale rect r by to/from
+ * @r: rect to be scaled.
+ * @from: from rectangle.
+ * @to: to rectangle.
+ *
+ * This scales rectangle @r horizontally by @to->width / @from->width and
+ * vertically by @to->height / @from->height.
+ *
+ * Typically @r is a rectangle inside @from and you want the rectangle as
+ * it would appear after scaling @from to @to. So the resulting @r will
+ * be the scaled rectangle inside @to.
+ */
+static inline void v4l2_rect_scale(struct v4l2_rect *r,
+ const struct v4l2_rect *from,
+ const struct v4l2_rect *to)
+{
+ if (from->width == 0 || from->height == 0) {
+ r->left = r->top = r->width = r->height = 0;
+ return;
+ }
+ r->left = (((r->left - from->left) * to->width) / from->width) & ~1;
+ r->width = ((r->width * to->width) / from->width) & ~1;
+ r->top = ((r->top - from->top) * to->height) / from->height;
+ r->height = (r->height * to->height) / from->height;
+}
+
+/**
+ * v4l2_rect_overlap() - do r1 and r2 overlap?
+ * @r1: rectangle.
+ * @r2: rectangle.
+ *
+ * Returns true if @r1 and @r2 overlap.
+ */
+static inline bool v4l2_rect_overlap(const struct v4l2_rect *r1,
+ const struct v4l2_rect *r2)
+{
+ /*
+ * IF the left side of r1 is to the right of the right side of r2 OR
+ * the left side of r2 is to the right of the right side of r1 THEN
+ * they do not overlap.
+ */
+ if (r1->left >= r2->left + r2->width ||
+ r2->left >= r1->left + r1->width)
+ return false;
+ /*
+ * IF the top side of r1 is below the bottom of r2 OR
+ * the top side of r2 is below the bottom of r1 THEN
+ * they do not overlap.
+ */
+ if (r1->top >= r2->top + r2->height ||
+ r2->top >= r1->top + r1->height)
+ return false;
+ return true;
+}
+
+#endif
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 11e2dfe..32fc7a4 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -572,6 +572,7 @@
/**
* struct v4l2_subdev_pad_ops - v4l2-subdev pad level operations
*
+ * @init_cfg: initialize the pad config to default values
* @enum_mbus_code: callback for VIDIOC_SUBDEV_ENUM_MBUS_CODE ioctl handler
* code.
* @enum_frame_size: callback for VIDIOC_SUBDEV_ENUM_FRAME_SIZE ioctl handler
@@ -607,6 +608,8 @@
* may be adjusted by the subdev driver to device capabilities.
*/
struct v4l2_subdev_pad_ops {
+ int (*init_cfg)(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg);
int (*enum_mbus_code)(struct v4l2_subdev *sd,
struct v4l2_subdev_pad_config *cfg,
struct v4l2_subdev_mbus_code_enum *code);
@@ -801,7 +804,12 @@
struct v4l2_subdev_format *source_fmt,
struct v4l2_subdev_format *sink_fmt);
int v4l2_subdev_link_validate(struct media_link *link);
+
+struct v4l2_subdev_pad_config *
+v4l2_subdev_alloc_pad_config(struct v4l2_subdev *sd);
+void v4l2_subdev_free_pad_config(struct v4l2_subdev_pad_config *cfg);
#endif /* CONFIG_MEDIA_CONTROLLER */
+
void v4l2_subdev_init(struct v4l2_subdev *sd,
const struct v4l2_subdev_ops *ops);
diff --git a/include/media/v4l2-tpg-colors.h b/include/media/v4l2-tpg-colors.h
new file mode 100644
index 0000000..2a88d1f
--- /dev/null
+++ b/include/media/v4l2-tpg-colors.h
@@ -0,0 +1,68 @@
+/*
+ * v4l2-tpg-colors.h - Color definitions for the test pattern generator
+ *
+ * Copyright 2014 Cisco Systems, Inc. and/or its affiliates. All rights reserved.
+ *
+ * This program is free software; you may redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _V4L2_TPG_COLORS_H_
+#define _V4L2_TPG_COLORS_H_
+
+struct color {
+ unsigned char r, g, b;
+};
+
+struct color16 {
+ int r, g, b;
+};
+
+enum tpg_color {
+ TPG_COLOR_CSC_WHITE,
+ TPG_COLOR_CSC_YELLOW,
+ TPG_COLOR_CSC_CYAN,
+ TPG_COLOR_CSC_GREEN,
+ TPG_COLOR_CSC_MAGENTA,
+ TPG_COLOR_CSC_RED,
+ TPG_COLOR_CSC_BLUE,
+ TPG_COLOR_CSC_BLACK,
+ TPG_COLOR_75_YELLOW,
+ TPG_COLOR_75_CYAN,
+ TPG_COLOR_75_GREEN,
+ TPG_COLOR_75_MAGENTA,
+ TPG_COLOR_75_RED,
+ TPG_COLOR_75_BLUE,
+ TPG_COLOR_100_WHITE,
+ TPG_COLOR_100_YELLOW,
+ TPG_COLOR_100_CYAN,
+ TPG_COLOR_100_GREEN,
+ TPG_COLOR_100_MAGENTA,
+ TPG_COLOR_100_RED,
+ TPG_COLOR_100_BLUE,
+ TPG_COLOR_100_BLACK,
+ TPG_COLOR_TEXTFG,
+ TPG_COLOR_TEXTBG,
+ TPG_COLOR_RANDOM,
+ TPG_COLOR_RAMP,
+ TPG_COLOR_MAX = TPG_COLOR_RAMP + 256
+};
+
+extern const struct color tpg_colors[TPG_COLOR_MAX];
+extern const unsigned short tpg_rec709_to_linear[255 * 16 + 1];
+extern const unsigned short tpg_linear_to_rec709[255 * 16 + 1];
+extern const struct color16 tpg_csc_colors[V4L2_COLORSPACE_DCI_P3 + 1]
+ [V4L2_XFER_FUNC_SMPTE2084 + 1]
+ [TPG_COLOR_CSC_BLACK + 1];
+
+#endif
diff --git a/include/media/v4l2-tpg.h b/include/media/v4l2-tpg.h
new file mode 100644
index 0000000..329bebf
--- /dev/null
+++ b/include/media/v4l2-tpg.h
@@ -0,0 +1,597 @@
+/*
+ * v4l2-tpg.h - Test Pattern Generator
+ *
+ * Copyright 2014 Cisco Systems, Inc. and/or its affiliates. All rights reserved.
+ *
+ * This program is free software; you may redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _V4L2_TPG_H_
+#define _V4L2_TPG_H_
+
+#include <linux/types.h>
+#include <linux/errno.h>
+#include <linux/random.h>
+#include <linux/slab.h>
+#include <linux/vmalloc.h>
+#include <linux/videodev2.h>
+#include <media/v4l2-tpg-colors.h>
+
+enum tpg_pattern {
+ TPG_PAT_75_COLORBAR,
+ TPG_PAT_100_COLORBAR,
+ TPG_PAT_CSC_COLORBAR,
+ TPG_PAT_100_HCOLORBAR,
+ TPG_PAT_100_COLORSQUARES,
+ TPG_PAT_BLACK,
+ TPG_PAT_WHITE,
+ TPG_PAT_RED,
+ TPG_PAT_GREEN,
+ TPG_PAT_BLUE,
+ TPG_PAT_CHECKERS_16X16,
+ TPG_PAT_CHECKERS_2X2,
+ TPG_PAT_CHECKERS_1X1,
+ TPG_PAT_COLOR_CHECKERS_2X2,
+ TPG_PAT_COLOR_CHECKERS_1X1,
+ TPG_PAT_ALTERNATING_HLINES,
+ TPG_PAT_ALTERNATING_VLINES,
+ TPG_PAT_CROSS_1_PIXEL,
+ TPG_PAT_CROSS_2_PIXELS,
+ TPG_PAT_CROSS_10_PIXELS,
+ TPG_PAT_GRAY_RAMP,
+
+ /* Must be the last pattern */
+ TPG_PAT_NOISE,
+};
+
+extern const char * const tpg_pattern_strings[];
+
+enum tpg_quality {
+ TPG_QUAL_COLOR,
+ TPG_QUAL_GRAY,
+ TPG_QUAL_NOISE
+};
+
+enum tpg_video_aspect {
+ TPG_VIDEO_ASPECT_IMAGE,
+ TPG_VIDEO_ASPECT_4X3,
+ TPG_VIDEO_ASPECT_14X9_CENTRE,
+ TPG_VIDEO_ASPECT_16X9_CENTRE,
+ TPG_VIDEO_ASPECT_16X9_ANAMORPHIC,
+};
+
+enum tpg_pixel_aspect {
+ TPG_PIXEL_ASPECT_SQUARE,
+ TPG_PIXEL_ASPECT_NTSC,
+ TPG_PIXEL_ASPECT_PAL,
+};
+
+enum tpg_move_mode {
+ TPG_MOVE_NEG_FAST,
+ TPG_MOVE_NEG,
+ TPG_MOVE_NEG_SLOW,
+ TPG_MOVE_NONE,
+ TPG_MOVE_POS_SLOW,
+ TPG_MOVE_POS,
+ TPG_MOVE_POS_FAST,
+};
+
+extern const char * const tpg_aspect_strings[];
+
+#define TPG_MAX_PLANES 3
+#define TPG_MAX_PAT_LINES 8
+
+struct tpg_data {
+ /* Source frame size */
+ unsigned src_width, src_height;
+ /* Buffer height */
+ unsigned buf_height;
+ /* Scaled output frame size */
+ unsigned scaled_width;
+ u32 field;
+ bool field_alternate;
+ /* crop coordinates are frame-based */
+ struct v4l2_rect crop;
+ /* compose coordinates are format-based */
+ struct v4l2_rect compose;
+ /* border and square coordinates are frame-based */
+ struct v4l2_rect border;
+ struct v4l2_rect square;
+
+ /* Color-related fields */
+ enum tpg_quality qual;
+ unsigned qual_offset;
+ u8 alpha_component;
+ bool alpha_red_only;
+ u8 brightness;
+ u8 contrast;
+ u8 saturation;
+ s16 hue;
+ u32 fourcc;
+ bool is_yuv;
+ u32 colorspace;
+ u32 xfer_func;
+ u32 ycbcr_enc;
+ /*
+ * Stores the actual transfer function, i.e. will never be
+ * V4L2_XFER_FUNC_DEFAULT.
+ */
+ u32 real_xfer_func;
+ /*
+ * Stores the actual Y'CbCr encoding, i.e. will never be
+ * V4L2_YCBCR_ENC_DEFAULT.
+ */
+ u32 real_ycbcr_enc;
+ u32 quantization;
+ /*
+ * Stores the actual quantization, i.e. will never be
+ * V4L2_QUANTIZATION_DEFAULT.
+ */
+ u32 real_quantization;
+ enum tpg_video_aspect vid_aspect;
+ enum tpg_pixel_aspect pix_aspect;
+ unsigned rgb_range;
+ unsigned real_rgb_range;
+ unsigned buffers;
+ unsigned planes;
+ bool interleaved;
+ u8 vdownsampling[TPG_MAX_PLANES];
+ u8 hdownsampling[TPG_MAX_PLANES];
+ /*
+ * horizontal positions must be ANDed with this value to enforce
+ * correct boundaries for packed YUYV values.
+ */
+ unsigned hmask[TPG_MAX_PLANES];
+ /* Used to store the colors in native format, either RGB or YUV */
+ u8 colors[TPG_COLOR_MAX][3];
+ u8 textfg[TPG_MAX_PLANES][8], textbg[TPG_MAX_PLANES][8];
+ /* size in bytes for two pixels in each plane */
+ unsigned twopixelsize[TPG_MAX_PLANES];
+ unsigned bytesperline[TPG_MAX_PLANES];
+
+ /* Configuration */
+ enum tpg_pattern pattern;
+ bool hflip;
+ bool vflip;
+ unsigned perc_fill;
+ bool perc_fill_blank;
+ bool show_border;
+ bool show_square;
+ bool insert_sav;
+ bool insert_eav;
+
+ /* Test pattern movement */
+ enum tpg_move_mode mv_hor_mode;
+ int mv_hor_count;
+ int mv_hor_step;
+ enum tpg_move_mode mv_vert_mode;
+ int mv_vert_count;
+ int mv_vert_step;
+
+ bool recalc_colors;
+ bool recalc_lines;
+ bool recalc_square_border;
+
+ /* Used to store TPG_MAX_PAT_LINES lines, each with up to two planes */
+ unsigned max_line_width;
+ u8 *lines[TPG_MAX_PAT_LINES][TPG_MAX_PLANES];
+ u8 *downsampled_lines[TPG_MAX_PAT_LINES][TPG_MAX_PLANES];
+ u8 *random_line[TPG_MAX_PLANES];
+ u8 *contrast_line[TPG_MAX_PLANES];
+ u8 *black_line[TPG_MAX_PLANES];
+};
+
+void tpg_init(struct tpg_data *tpg, unsigned w, unsigned h);
+int tpg_alloc(struct tpg_data *tpg, unsigned max_w);
+void tpg_free(struct tpg_data *tpg);
+void tpg_reset_source(struct tpg_data *tpg, unsigned width, unsigned height,
+ u32 field);
+void tpg_log_status(struct tpg_data *tpg);
+
+void tpg_set_font(const u8 *f);
+void tpg_gen_text(const struct tpg_data *tpg,
+ u8 *basep[TPG_MAX_PLANES][2], int y, int x, char *text);
+void tpg_calc_text_basep(struct tpg_data *tpg,
+ u8 *basep[TPG_MAX_PLANES][2], unsigned p, u8 *vbuf);
+unsigned tpg_g_interleaved_plane(const struct tpg_data *tpg, unsigned buf_line);
+void tpg_fill_plane_buffer(struct tpg_data *tpg, v4l2_std_id std,
+ unsigned p, u8 *vbuf);
+void tpg_fillbuffer(struct tpg_data *tpg, v4l2_std_id std,
+ unsigned p, u8 *vbuf);
+bool tpg_s_fourcc(struct tpg_data *tpg, u32 fourcc);
+void tpg_s_crop_compose(struct tpg_data *tpg, const struct v4l2_rect *crop,
+ const struct v4l2_rect *compose);
+
+static inline void tpg_s_pattern(struct tpg_data *tpg, enum tpg_pattern pattern)
+{
+ if (tpg->pattern == pattern)
+ return;
+ tpg->pattern = pattern;
+ tpg->recalc_colors = true;
+}
+
+static inline void tpg_s_quality(struct tpg_data *tpg,
+ enum tpg_quality qual, unsigned qual_offset)
+{
+ if (tpg->qual == qual && tpg->qual_offset == qual_offset)
+ return;
+ tpg->qual = qual;
+ tpg->qual_offset = qual_offset;
+ tpg->recalc_colors = true;
+}
+
+static inline enum tpg_quality tpg_g_quality(const struct tpg_data *tpg)
+{
+ return tpg->qual;
+}
+
+static inline void tpg_s_alpha_component(struct tpg_data *tpg,
+ u8 alpha_component)
+{
+ if (tpg->alpha_component == alpha_component)
+ return;
+ tpg->alpha_component = alpha_component;
+ tpg->recalc_colors = true;
+}
+
+static inline void tpg_s_alpha_mode(struct tpg_data *tpg,
+ bool red_only)
+{
+ if (tpg->alpha_red_only == red_only)
+ return;
+ tpg->alpha_red_only = red_only;
+ tpg->recalc_colors = true;
+}
+
+static inline void tpg_s_brightness(struct tpg_data *tpg,
+ u8 brightness)
+{
+ if (tpg->brightness == brightness)
+ return;
+ tpg->brightness = brightness;
+ tpg->recalc_colors = true;
+}
+
+static inline void tpg_s_contrast(struct tpg_data *tpg,
+ u8 contrast)
+{
+ if (tpg->contrast == contrast)
+ return;
+ tpg->contrast = contrast;
+ tpg->recalc_colors = true;
+}
+
+static inline void tpg_s_saturation(struct tpg_data *tpg,
+ u8 saturation)
+{
+ if (tpg->saturation == saturation)
+ return;
+ tpg->saturation = saturation;
+ tpg->recalc_colors = true;
+}
+
+static inline void tpg_s_hue(struct tpg_data *tpg,
+ s16 hue)
+{
+ if (tpg->hue == hue)
+ return;
+ tpg->hue = hue;
+ tpg->recalc_colors = true;
+}
+
+static inline void tpg_s_rgb_range(struct tpg_data *tpg,
+ unsigned rgb_range)
+{
+ if (tpg->rgb_range == rgb_range)
+ return;
+ tpg->rgb_range = rgb_range;
+ tpg->recalc_colors = true;
+}
+
+static inline void tpg_s_real_rgb_range(struct tpg_data *tpg,
+ unsigned rgb_range)
+{
+ if (tpg->real_rgb_range == rgb_range)
+ return;
+ tpg->real_rgb_range = rgb_range;
+ tpg->recalc_colors = true;
+}
+
+static inline void tpg_s_colorspace(struct tpg_data *tpg, u32 colorspace)
+{
+ if (tpg->colorspace == colorspace)
+ return;
+ tpg->colorspace = colorspace;
+ tpg->recalc_colors = true;
+}
+
+static inline u32 tpg_g_colorspace(const struct tpg_data *tpg)
+{
+ return tpg->colorspace;
+}
+
+static inline void tpg_s_ycbcr_enc(struct tpg_data *tpg, u32 ycbcr_enc)
+{
+ if (tpg->ycbcr_enc == ycbcr_enc)
+ return;
+ tpg->ycbcr_enc = ycbcr_enc;
+ tpg->recalc_colors = true;
+}
+
+static inline u32 tpg_g_ycbcr_enc(const struct tpg_data *tpg)
+{
+ return tpg->ycbcr_enc;
+}
+
+static inline void tpg_s_xfer_func(struct tpg_data *tpg, u32 xfer_func)
+{
+ if (tpg->xfer_func == xfer_func)
+ return;
+ tpg->xfer_func = xfer_func;
+ tpg->recalc_colors = true;
+}
+
+static inline u32 tpg_g_xfer_func(const struct tpg_data *tpg)
+{
+ return tpg->xfer_func;
+}
+
+static inline void tpg_s_quantization(struct tpg_data *tpg, u32 quantization)
+{
+ if (tpg->quantization == quantization)
+ return;
+ tpg->quantization = quantization;
+ tpg->recalc_colors = true;
+}
+
+static inline u32 tpg_g_quantization(const struct tpg_data *tpg)
+{
+ return tpg->quantization;
+}
+
+static inline unsigned tpg_g_buffers(const struct tpg_data *tpg)
+{
+ return tpg->buffers;
+}
+
+static inline unsigned tpg_g_planes(const struct tpg_data *tpg)
+{
+ return tpg->interleaved ? 1 : tpg->planes;
+}
+
+static inline bool tpg_g_interleaved(const struct tpg_data *tpg)
+{
+ return tpg->interleaved;
+}
+
+static inline unsigned tpg_g_twopixelsize(const struct tpg_data *tpg, unsigned plane)
+{
+ return tpg->twopixelsize[plane];
+}
+
+static inline unsigned tpg_hdiv(const struct tpg_data *tpg,
+ unsigned plane, unsigned x)
+{
+ return ((x / tpg->hdownsampling[plane]) & tpg->hmask[plane]) *
+ tpg->twopixelsize[plane] / 2;
+}
+
+static inline unsigned tpg_hscale(const struct tpg_data *tpg, unsigned x)
+{
+ return (x * tpg->scaled_width) / tpg->src_width;
+}
+
+static inline unsigned tpg_hscale_div(const struct tpg_data *tpg,
+ unsigned plane, unsigned x)
+{
+ return tpg_hdiv(tpg, plane, tpg_hscale(tpg, x));
+}
+
+static inline unsigned tpg_g_bytesperline(const struct tpg_data *tpg, unsigned plane)
+{
+ return tpg->bytesperline[plane];
+}
+
+static inline void tpg_s_bytesperline(struct tpg_data *tpg, unsigned plane, unsigned bpl)
+{
+ unsigned p;
+
+ if (tpg->buffers > 1) {
+ tpg->bytesperline[plane] = bpl;
+ return;
+ }
+
+ for (p = 0; p < tpg_g_planes(tpg); p++) {
+ unsigned plane_w = bpl * tpg->twopixelsize[p] / tpg->twopixelsize[0];
+
+ tpg->bytesperline[p] = plane_w / tpg->hdownsampling[p];
+ }
+ if (tpg_g_interleaved(tpg))
+ tpg->bytesperline[1] = tpg->bytesperline[0];
+}
+
+
+static inline unsigned tpg_g_line_width(const struct tpg_data *tpg, unsigned plane)
+{
+ unsigned w = 0;
+ unsigned p;
+
+ if (tpg->buffers > 1)
+ return tpg_g_bytesperline(tpg, plane);
+ for (p = 0; p < tpg_g_planes(tpg); p++) {
+ unsigned plane_w = tpg_g_bytesperline(tpg, p);
+
+ w += plane_w / tpg->vdownsampling[p];
+ }
+ return w;
+}
+
+static inline unsigned tpg_calc_line_width(const struct tpg_data *tpg,
+ unsigned plane, unsigned bpl)
+{
+ unsigned w = 0;
+ unsigned p;
+
+ if (tpg->buffers > 1)
+ return bpl;
+ for (p = 0; p < tpg_g_planes(tpg); p++) {
+ unsigned plane_w = bpl * tpg->twopixelsize[p] / tpg->twopixelsize[0];
+
+ plane_w /= tpg->hdownsampling[p];
+ w += plane_w / tpg->vdownsampling[p];
+ }
+ return w;
+}
+
+static inline unsigned tpg_calc_plane_size(const struct tpg_data *tpg, unsigned plane)
+{
+ if (plane >= tpg_g_planes(tpg))
+ return 0;
+
+ return tpg_g_bytesperline(tpg, plane) * tpg->buf_height /
+ tpg->vdownsampling[plane];
+}
+
+static inline void tpg_s_buf_height(struct tpg_data *tpg, unsigned h)
+{
+ tpg->buf_height = h;
+}
+
+static inline void tpg_s_field(struct tpg_data *tpg, unsigned field, bool alternate)
+{
+ tpg->field = field;
+ tpg->field_alternate = alternate;
+}
+
+static inline void tpg_s_perc_fill(struct tpg_data *tpg,
+ unsigned perc_fill)
+{
+ tpg->perc_fill = perc_fill;
+}
+
+static inline unsigned tpg_g_perc_fill(const struct tpg_data *tpg)
+{
+ return tpg->perc_fill;
+}
+
+static inline void tpg_s_perc_fill_blank(struct tpg_data *tpg,
+ bool perc_fill_blank)
+{
+ tpg->perc_fill_blank = perc_fill_blank;
+}
+
+static inline void tpg_s_video_aspect(struct tpg_data *tpg,
+ enum tpg_video_aspect vid_aspect)
+{
+ if (tpg->vid_aspect == vid_aspect)
+ return;
+ tpg->vid_aspect = vid_aspect;
+ tpg->recalc_square_border = true;
+}
+
+static inline enum tpg_video_aspect tpg_g_video_aspect(const struct tpg_data *tpg)
+{
+ return tpg->vid_aspect;
+}
+
+static inline void tpg_s_pixel_aspect(struct tpg_data *tpg,
+ enum tpg_pixel_aspect pix_aspect)
+{
+ if (tpg->pix_aspect == pix_aspect)
+ return;
+ tpg->pix_aspect = pix_aspect;
+ tpg->recalc_square_border = true;
+}
+
+static inline void tpg_s_show_border(struct tpg_data *tpg,
+ bool show_border)
+{
+ tpg->show_border = show_border;
+}
+
+static inline void tpg_s_show_square(struct tpg_data *tpg,
+ bool show_square)
+{
+ tpg->show_square = show_square;
+}
+
+static inline void tpg_s_insert_sav(struct tpg_data *tpg, bool insert_sav)
+{
+ tpg->insert_sav = insert_sav;
+}
+
+static inline void tpg_s_insert_eav(struct tpg_data *tpg, bool insert_eav)
+{
+ tpg->insert_eav = insert_eav;
+}
+
+void tpg_update_mv_step(struct tpg_data *tpg);
+
+static inline void tpg_s_mv_hor_mode(struct tpg_data *tpg,
+ enum tpg_move_mode mv_hor_mode)
+{
+ tpg->mv_hor_mode = mv_hor_mode;
+ tpg_update_mv_step(tpg);
+}
+
+static inline void tpg_s_mv_vert_mode(struct tpg_data *tpg,
+ enum tpg_move_mode mv_vert_mode)
+{
+ tpg->mv_vert_mode = mv_vert_mode;
+ tpg_update_mv_step(tpg);
+}
+
+static inline void tpg_init_mv_count(struct tpg_data *tpg)
+{
+ tpg->mv_hor_count = tpg->mv_vert_count = 0;
+}
+
+static inline void tpg_update_mv_count(struct tpg_data *tpg, bool frame_is_field)
+{
+ tpg->mv_hor_count += tpg->mv_hor_step * (frame_is_field ? 1 : 2);
+ tpg->mv_vert_count += tpg->mv_vert_step * (frame_is_field ? 1 : 2);
+}
+
+static inline void tpg_s_hflip(struct tpg_data *tpg, bool hflip)
+{
+ if (tpg->hflip == hflip)
+ return;
+ tpg->hflip = hflip;
+ tpg_update_mv_step(tpg);
+ tpg->recalc_lines = true;
+}
+
+static inline bool tpg_g_hflip(const struct tpg_data *tpg)
+{
+ return tpg->hflip;
+}
+
+static inline void tpg_s_vflip(struct tpg_data *tpg, bool vflip)
+{
+ tpg->vflip = vflip;
+}
+
+static inline bool tpg_g_vflip(const struct tpg_data *tpg)
+{
+ return tpg->vflip;
+}
+
+static inline bool tpg_pattern_is_static(const struct tpg_data *tpg)
+{
+ return tpg->pattern != TPG_PAT_NOISE &&
+ tpg->mv_hor_mode == TPG_MOVE_NONE &&
+ tpg->mv_vert_mode == TPG_MOVE_NONE;
+}
+
+#endif
diff --git a/include/media/vsp1.h b/include/media/vsp1.h
index cc54175..3e654a0 100644
--- a/include/media/vsp1.h
+++ b/include/media/vsp1.h
@@ -23,11 +23,22 @@
int vsp1_du_setup_lif(struct device *dev, unsigned int width,
unsigned int height);
-int vsp1_du_atomic_begin(struct device *dev);
-int vsp1_du_atomic_update(struct device *dev, unsigned int rpf, u32 pixelformat,
- unsigned int pitch, dma_addr_t mem[2],
- const struct v4l2_rect *src,
- const struct v4l2_rect *dst);
-int vsp1_du_atomic_flush(struct device *dev);
+void vsp1_du_atomic_begin(struct device *dev);
+int vsp1_du_atomic_update_ext(struct device *dev, unsigned int rpf,
+ u32 pixelformat, unsigned int pitch,
+ dma_addr_t mem[2], const struct v4l2_rect *src,
+ const struct v4l2_rect *dst, unsigned int alpha,
+ unsigned int zpos);
+void vsp1_du_atomic_flush(struct device *dev);
+
+static inline int vsp1_du_atomic_update(struct device *dev,
+ unsigned int rpf_index, u32 pixelformat,
+ unsigned int pitch, dma_addr_t mem[2],
+ const struct v4l2_rect *src,
+ const struct v4l2_rect *dst)
+{
+ return vsp1_du_atomic_update_ext(dev, rpf_index, pixelformat, pitch,
+ mem, src, dst, 255, 0);
+}
#endif /* __MEDIA_VSP1_H__ */
diff --git a/include/misc/cxl.h b/include/misc/cxl.h
index 7d5e261..56560c5 100644
--- a/include/misc/cxl.h
+++ b/include/misc/cxl.h
@@ -127,6 +127,14 @@
void cxl_set_master(struct cxl_context *ctx);
/*
+ * Sets the context to use real mode memory accesses to operate with
+ * translation disabled. Note that this only makes sense for kernel contexts
+ * under bare metal, and will not work with virtualisation. May only be
+ * performed on stopped contexts.
+ */
+int cxl_set_translation_mode(struct cxl_context *ctx, bool real_mode);
+
+/*
* Map and unmap the AFU Problem Space area. The amount and location mapped
* depends on if this context is a master or slave.
*/
diff --git a/include/net/act_api.h b/include/net/act_api.h
index 2cd9e9b..9a9a8ed 100644
--- a/include/net/act_api.h
+++ b/include/net/act_api.h
@@ -192,7 +192,7 @@
#else /* CONFIG_NET_CLS_ACT */
#define tc_no_actions(_exts) true
-#define tc_for_each_action(_a, _exts) while (0)
+#define tc_for_each_action(_a, _exts) while ((void)(_a), 0)
#define tcf_action_stats_update(a, bytes, packets, lastuse)
#endif /* CONFIG_NET_CLS_ACT */
diff --git a/include/net/fou.h b/include/net/fou.h
index 19b8a0c..f5cc691 100644
--- a/include/net/fou.h
+++ b/include/net/fou.h
@@ -9,11 +9,11 @@
#include <net/udp.h>
size_t fou_encap_hlen(struct ip_tunnel_encap *e);
-static size_t gue_encap_hlen(struct ip_tunnel_encap *e);
+size_t gue_encap_hlen(struct ip_tunnel_encap *e);
-int fou_build_header(struct sk_buff *skb, struct ip_tunnel_encap *e,
- u8 *protocol, struct flowi4 *fl4);
-int gue_build_header(struct sk_buff *skb, struct ip_tunnel_encap *e,
- u8 *protocol, struct flowi4 *fl4);
+int __fou_build_header(struct sk_buff *skb, struct ip_tunnel_encap *e,
+ u8 *protocol, __be16 *sport, int type);
+int __gue_build_header(struct sk_buff *skb, struct ip_tunnel_encap *e,
+ u8 *protocol, __be16 *sport, int type);
#endif
diff --git a/include/net/inet_common.h b/include/net/inet_common.h
index 109e3ee..5d68342 100644
--- a/include/net/inet_common.h
+++ b/include/net/inet_common.h
@@ -39,6 +39,11 @@
int inet_recv_error(struct sock *sk, struct msghdr *msg, int len,
int *addr_len);
+struct sk_buff **inet_gro_receive(struct sk_buff **head, struct sk_buff *skb);
+int inet_gro_complete(struct sk_buff *skb, int nhoff);
+struct sk_buff *inet_gso_segment(struct sk_buff *skb,
+ netdev_features_t features);
+
static inline void inet_ctl_sock_destroy(struct sock *sk)
{
if (sk)
diff --git a/include/net/ip6_tunnel.h b/include/net/ip6_tunnel.h
index fb9e015..d325c81 100644
--- a/include/net/ip6_tunnel.h
+++ b/include/net/ip6_tunnel.h
@@ -52,10 +52,68 @@
__u32 o_seqno; /* The last output seqno */
int hlen; /* tun_hlen + encap_hlen */
int tun_hlen; /* Precalculated header length */
+ int encap_hlen; /* Encap header length (FOU,GUE) */
+ struct ip_tunnel_encap encap;
int mlink;
-
};
+struct ip6_tnl_encap_ops {
+ size_t (*encap_hlen)(struct ip_tunnel_encap *e);
+ int (*build_header)(struct sk_buff *skb, struct ip_tunnel_encap *e,
+ u8 *protocol, struct flowi6 *fl6);
+};
+
+extern const struct ip6_tnl_encap_ops __rcu *
+ ip6tun_encaps[MAX_IPTUN_ENCAP_OPS];
+
+int ip6_tnl_encap_add_ops(const struct ip6_tnl_encap_ops *ops,
+ unsigned int num);
+int ip6_tnl_encap_del_ops(const struct ip6_tnl_encap_ops *ops,
+ unsigned int num);
+int ip6_tnl_encap_setup(struct ip6_tnl *t,
+ struct ip_tunnel_encap *ipencap);
+
+static inline int ip6_encap_hlen(struct ip_tunnel_encap *e)
+{
+ const struct ip6_tnl_encap_ops *ops;
+ int hlen = -EINVAL;
+
+ if (e->type == TUNNEL_ENCAP_NONE)
+ return 0;
+
+ if (e->type >= MAX_IPTUN_ENCAP_OPS)
+ return -EINVAL;
+
+ rcu_read_lock();
+ ops = rcu_dereference(ip6tun_encaps[e->type]);
+ if (likely(ops && ops->encap_hlen))
+ hlen = ops->encap_hlen(e);
+ rcu_read_unlock();
+
+ return hlen;
+}
+
+static inline int ip6_tnl_encap(struct sk_buff *skb, struct ip6_tnl *t,
+ u8 *protocol, struct flowi6 *fl6)
+{
+ const struct ip6_tnl_encap_ops *ops;
+ int ret = -EINVAL;
+
+ if (t->encap.type == TUNNEL_ENCAP_NONE)
+ return 0;
+
+ if (t->encap.type >= MAX_IPTUN_ENCAP_OPS)
+ return -EINVAL;
+
+ rcu_read_lock();
+ ops = rcu_dereference(ip6tun_encaps[t->encap.type]);
+ if (likely(ops && ops->build_header))
+ ret = ops->build_header(skb, &t->encap, protocol, fl6);
+ rcu_read_unlock();
+
+ return ret;
+}
+
/* Tunnel encapsulation limit destination sub-option */
struct ipv6_tlv_tnl_enc_lim {
diff --git a/include/net/ip_tunnels.h b/include/net/ip_tunnels.h
index d916b43..dbf4444 100644
--- a/include/net/ip_tunnels.h
+++ b/include/net/ip_tunnels.h
@@ -171,22 +171,6 @@
struct ip_tunnel __rcu *collect_md_tun;
};
-struct ip_tunnel_encap_ops {
- size_t (*encap_hlen)(struct ip_tunnel_encap *e);
- int (*build_header)(struct sk_buff *skb, struct ip_tunnel_encap *e,
- u8 *protocol, struct flowi4 *fl4);
-};
-
-#define MAX_IPTUN_ENCAP_OPS 8
-
-extern const struct ip_tunnel_encap_ops __rcu *
- iptun_encaps[MAX_IPTUN_ENCAP_OPS];
-
-int ip_tunnel_encap_add_ops(const struct ip_tunnel_encap_ops *op,
- unsigned int num);
-int ip_tunnel_encap_del_ops(const struct ip_tunnel_encap_ops *op,
- unsigned int num);
-
static inline void ip_tunnel_key_init(struct ip_tunnel_key *key,
__be32 saddr, __be32 daddr,
u8 tos, u8 ttl, __be32 label,
@@ -251,8 +235,6 @@
void ip_tunnel_xmit(struct sk_buff *skb, struct net_device *dev,
const struct iphdr *tnl_params, const u8 protocol);
int ip_tunnel_ioctl(struct net_device *dev, struct ip_tunnel_parm *p, int cmd);
-int ip_tunnel_encap(struct sk_buff *skb, struct ip_tunnel *t,
- u8 *protocol, struct flowi4 *fl4);
int __ip_tunnel_change_mtu(struct net_device *dev, int new_mtu, bool strict);
int ip_tunnel_change_mtu(struct net_device *dev, int new_mtu);
@@ -271,9 +253,67 @@
int ip_tunnel_newlink(struct net_device *dev, struct nlattr *tb[],
struct ip_tunnel_parm *p);
void ip_tunnel_setup(struct net_device *dev, int net_id);
+
+struct ip_tunnel_encap_ops {
+ size_t (*encap_hlen)(struct ip_tunnel_encap *e);
+ int (*build_header)(struct sk_buff *skb, struct ip_tunnel_encap *e,
+ u8 *protocol, struct flowi4 *fl4);
+};
+
+#define MAX_IPTUN_ENCAP_OPS 8
+
+extern const struct ip_tunnel_encap_ops __rcu *
+ iptun_encaps[MAX_IPTUN_ENCAP_OPS];
+
+int ip_tunnel_encap_add_ops(const struct ip_tunnel_encap_ops *op,
+ unsigned int num);
+int ip_tunnel_encap_del_ops(const struct ip_tunnel_encap_ops *op,
+ unsigned int num);
+
int ip_tunnel_encap_setup(struct ip_tunnel *t,
struct ip_tunnel_encap *ipencap);
+static inline int ip_encap_hlen(struct ip_tunnel_encap *e)
+{
+ const struct ip_tunnel_encap_ops *ops;
+ int hlen = -EINVAL;
+
+ if (e->type == TUNNEL_ENCAP_NONE)
+ return 0;
+
+ if (e->type >= MAX_IPTUN_ENCAP_OPS)
+ return -EINVAL;
+
+ rcu_read_lock();
+ ops = rcu_dereference(iptun_encaps[e->type]);
+ if (likely(ops && ops->encap_hlen))
+ hlen = ops->encap_hlen(e);
+ rcu_read_unlock();
+
+ return hlen;
+}
+
+static inline int ip_tunnel_encap(struct sk_buff *skb, struct ip_tunnel *t,
+ u8 *protocol, struct flowi4 *fl4)
+{
+ const struct ip_tunnel_encap_ops *ops;
+ int ret = -EINVAL;
+
+ if (t->encap.type == TUNNEL_ENCAP_NONE)
+ return 0;
+
+ if (t->encap.type >= MAX_IPTUN_ENCAP_OPS)
+ return -EINVAL;
+
+ rcu_read_lock();
+ ops = rcu_dereference(iptun_encaps[t->encap.type]);
+ if (likely(ops && ops->build_header))
+ ret = ops->build_header(skb, &t->encap, protocol, fl4);
+ rcu_read_unlock();
+
+ return ret;
+}
+
/* Extract dsfield from inner protocol */
static inline u8 ip_tunnel_get_dsfield(const struct iphdr *iph,
const struct sk_buff *skb)
diff --git a/include/net/sock.h b/include/net/sock.h
index c9c8b19..649d2a8 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -382,8 +382,13 @@
atomic_t sk_omem_alloc;
int sk_sndbuf;
struct sk_buff_head sk_write_queue;
+
+ /*
+ * Because of non atomicity rules, all
+ * changes are protected by socket lock.
+ */
kmemcheck_bitfield_begin(flags);
- unsigned int sk_shutdown : 2,
+ unsigned int sk_padding : 2,
sk_no_check_tx : 1,
sk_no_check_rx : 1,
sk_userlocks : 4,
@@ -391,6 +396,7 @@
sk_type : 16;
#define SK_PROTOCOL_MAX U8_MAX
kmemcheck_bitfield_end(flags);
+
int sk_wmem_queued;
gfp_t sk_allocation;
u32 sk_pacing_rate; /* bytes per second */
@@ -418,6 +424,7 @@
struct timer_list sk_timer;
ktime_t sk_stamp;
u16 sk_tsflags;
+ u8 sk_shutdown;
u32 sk_tskey;
struct socket *sk_socket;
void *sk_user_data;
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index fb2cef4..fc0320c 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -220,6 +220,7 @@
IB_DEVICE_ON_DEMAND_PAGING = (1 << 31),
IB_DEVICE_SG_GAPS_REG = (1ULL << 32),
IB_DEVICE_VIRTUAL_FUNCTION = ((u64)1 << 33),
+ IB_DEVICE_RAW_SCATTER_FCS = ((u64)1 << 34),
};
enum ib_signature_prot_cap {
@@ -931,6 +932,13 @@
u32 max_send_sge;
u32 max_recv_sge;
u32 max_inline_data;
+
+ /*
+ * Maximum number of rdma_rw_ctx structures in flight at a time.
+ * ib_create_qp() will calculate the right amount of neededed WRs
+ * and MRs based on this.
+ */
+ u32 max_rdma_ctxs;
};
enum ib_sig_type {
@@ -981,6 +989,7 @@
IB_QP_CREATE_NETIF_QP = 1 << 5,
IB_QP_CREATE_SIGNATURE_EN = 1 << 6,
IB_QP_CREATE_USE_GFP_NOIO = 1 << 7,
+ IB_QP_CREATE_SCATTER_FCS = 1 << 8,
/* reserve bits 26-31 for low level drivers' internal use */
IB_QP_CREATE_RESERVED_START = 1 << 26,
IB_QP_CREATE_RESERVED_END = 1 << 31,
@@ -1002,7 +1011,11 @@
enum ib_sig_type sq_sig_type;
enum ib_qp_type qp_type;
enum ib_qp_create_flags create_flags;
- u8 port_num; /* special QP types only */
+
+ /*
+ * Only needed for special QP types, or when using the RW API.
+ */
+ u8 port_num;
};
struct ib_qp_open_attr {
@@ -1421,9 +1434,14 @@
struct ib_pd *pd;
struct ib_cq *send_cq;
struct ib_cq *recv_cq;
+ spinlock_t mr_lock;
+ int mrs_used;
+ struct list_head rdma_mrs;
+ struct list_head sig_mrs;
struct ib_srq *srq;
struct ib_xrcd *xrcd; /* XRC TGT QPs only */
struct list_head xrcd_list;
+
/* count times opened, mcast attaches, flow attaches */
atomic_t usecnt;
struct list_head open_list;
@@ -1438,12 +1456,16 @@
struct ib_mr {
struct ib_device *device;
struct ib_pd *pd;
- struct ib_uobject *uobject;
u32 lkey;
u32 rkey;
u64 iova;
u32 length;
unsigned int page_size;
+ bool need_inval;
+ union {
+ struct ib_uobject *uobject; /* user */
+ struct list_head qp_entry; /* FR */
+ };
};
struct ib_mw {
@@ -1827,7 +1849,8 @@
u32 max_num_sg);
int (*map_mr_sg)(struct ib_mr *mr,
struct scatterlist *sg,
- int sg_nents);
+ int sg_nents,
+ unsigned int *sg_offset);
struct ib_mw * (*alloc_mw)(struct ib_pd *pd,
enum ib_mw_type type,
struct ib_udata *udata);
@@ -2317,6 +2340,18 @@
device->add_gid && device->del_gid;
}
+/*
+ * Check if the device supports READ W/ INVALIDATE.
+ */
+static inline bool rdma_cap_read_inv(struct ib_device *dev, u32 port_num)
+{
+ /*
+ * iWarp drivers must support READ W/ INVALIDATE. No other protocol
+ * has support for it yet.
+ */
+ return rdma_protocol_iwarp(dev, port_num);
+}
+
int ib_query_gid(struct ib_device *device,
u8 port_num, int index, union ib_gid *gid,
struct ib_gid_attr *attr);
@@ -3111,29 +3146,23 @@
u16 pkey, const union ib_gid *gid,
const struct sockaddr *addr);
-int ib_map_mr_sg(struct ib_mr *mr,
- struct scatterlist *sg,
- int sg_nents,
- unsigned int page_size);
+int ib_map_mr_sg(struct ib_mr *mr, struct scatterlist *sg, int sg_nents,
+ unsigned int *sg_offset, unsigned int page_size);
static inline int
-ib_map_mr_sg_zbva(struct ib_mr *mr,
- struct scatterlist *sg,
- int sg_nents,
- unsigned int page_size)
+ib_map_mr_sg_zbva(struct ib_mr *mr, struct scatterlist *sg, int sg_nents,
+ unsigned int *sg_offset, unsigned int page_size)
{
int n;
- n = ib_map_mr_sg(mr, sg, sg_nents, page_size);
+ n = ib_map_mr_sg(mr, sg, sg_nents, sg_offset, page_size);
mr->iova = 0;
return n;
}
-int ib_sg_to_pages(struct ib_mr *mr,
- struct scatterlist *sgl,
- int sg_nents,
- int (*set_page)(struct ib_mr *, u64));
+int ib_sg_to_pages(struct ib_mr *mr, struct scatterlist *sgl, int sg_nents,
+ unsigned int *sg_offset, int (*set_page)(struct ib_mr *, u64));
void ib_drain_rq(struct ib_qp *qp);
void ib_drain_sq(struct ib_qp *qp);
diff --git a/include/rdma/mr_pool.h b/include/rdma/mr_pool.h
new file mode 100644
index 0000000..986010b
--- /dev/null
+++ b/include/rdma/mr_pool.h
@@ -0,0 +1,25 @@
+/*
+ * Copyright (c) 2016 HGST, a Western Digital Company.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ */
+#ifndef _RDMA_MR_POOL_H
+#define _RDMA_MR_POOL_H 1
+
+#include <rdma/ib_verbs.h>
+
+struct ib_mr *ib_mr_pool_get(struct ib_qp *qp, struct list_head *list);
+void ib_mr_pool_put(struct ib_qp *qp, struct list_head *list, struct ib_mr *mr);
+
+int ib_mr_pool_init(struct ib_qp *qp, struct list_head *list, int nr,
+ enum ib_mr_type type, u32 max_num_sg);
+void ib_mr_pool_destroy(struct ib_qp *qp, struct list_head *list);
+
+#endif /* _RDMA_MR_POOL_H */
diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index a869655..d57ceee 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -467,6 +467,7 @@
}
struct rvt_dev_info *rvt_alloc_device(size_t size, int nports);
+void rvt_dealloc_device(struct rvt_dev_info *rdi);
int rvt_register_device(struct rvt_dev_info *rvd);
void rvt_unregister_device(struct rvt_dev_info *rvd);
int rvt_check_ah(struct ib_device *ibdev, struct ib_ah_attr *ah_attr);
diff --git a/include/rdma/rdmavt_qp.h b/include/rdma/rdmavt_qp.h
index 497e590..0e1ff2a 100644
--- a/include/rdma/rdmavt_qp.h
+++ b/include/rdma/rdmavt_qp.h
@@ -117,8 +117,9 @@
/*
* Wait flags that would prevent any packet type from being sent.
*/
-#define RVT_S_ANY_WAIT_IO (RVT_S_WAIT_PIO | RVT_S_WAIT_TX | \
- RVT_S_WAIT_DMA_DESC | RVT_S_WAIT_KMEM)
+#define RVT_S_ANY_WAIT_IO \
+ (RVT_S_WAIT_PIO | RVT_S_WAIT_PIO_DRAIN | RVT_S_WAIT_TX | \
+ RVT_S_WAIT_DMA_DESC | RVT_S_WAIT_KMEM)
/*
* Wait flags that would prevent send work requests from making progress.
diff --git a/include/rdma/rw.h b/include/rdma/rw.h
new file mode 100644
index 0000000..377d865
--- /dev/null
+++ b/include/rdma/rw.h
@@ -0,0 +1,88 @@
+/*
+ * Copyright (c) 2016 HGST, a Western Digital Company.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ */
+#ifndef _RDMA_RW_H
+#define _RDMA_RW_H
+
+#include <linux/dma-mapping.h>
+#include <linux/scatterlist.h>
+#include <rdma/ib_verbs.h>
+#include <rdma/rdma_cm.h>
+#include <rdma/mr_pool.h>
+
+struct rdma_rw_ctx {
+ /* number of RDMA READ/WRITE WRs (not counting MR WRs) */
+ u32 nr_ops;
+
+ /* tag for the union below: */
+ u8 type;
+
+ union {
+ /* for mapping a single SGE: */
+ struct {
+ struct ib_sge sge;
+ struct ib_rdma_wr wr;
+ } single;
+
+ /* for mapping of multiple SGEs: */
+ struct {
+ struct ib_sge *sges;
+ struct ib_rdma_wr *wrs;
+ } map;
+
+ /* for registering multiple WRs: */
+ struct rdma_rw_reg_ctx {
+ struct ib_sge sge;
+ struct ib_rdma_wr wr;
+ struct ib_reg_wr reg_wr;
+ struct ib_send_wr inv_wr;
+ struct ib_mr *mr;
+ } *reg;
+
+ struct {
+ struct rdma_rw_reg_ctx data;
+ struct rdma_rw_reg_ctx prot;
+ struct ib_send_wr sig_inv_wr;
+ struct ib_mr *sig_mr;
+ struct ib_sge sig_sge;
+ struct ib_sig_handover_wr sig_wr;
+ } *sig;
+ };
+};
+
+int rdma_rw_ctx_init(struct rdma_rw_ctx *ctx, struct ib_qp *qp, u8 port_num,
+ struct scatterlist *sg, u32 sg_cnt, u32 sg_offset,
+ u64 remote_addr, u32 rkey, enum dma_data_direction dir);
+void rdma_rw_ctx_destroy(struct rdma_rw_ctx *ctx, struct ib_qp *qp, u8 port_num,
+ struct scatterlist *sg, u32 sg_cnt,
+ enum dma_data_direction dir);
+
+int rdma_rw_ctx_signature_init(struct rdma_rw_ctx *ctx, struct ib_qp *qp,
+ u8 port_num, struct scatterlist *sg, u32 sg_cnt,
+ struct scatterlist *prot_sg, u32 prot_sg_cnt,
+ struct ib_sig_attrs *sig_attrs, u64 remote_addr, u32 rkey,
+ enum dma_data_direction dir);
+void rdma_rw_ctx_destroy_signature(struct rdma_rw_ctx *ctx, struct ib_qp *qp,
+ u8 port_num, struct scatterlist *sg, u32 sg_cnt,
+ struct scatterlist *prot_sg, u32 prot_sg_cnt,
+ enum dma_data_direction dir);
+
+struct ib_send_wr *rdma_rw_ctx_wrs(struct rdma_rw_ctx *ctx, struct ib_qp *qp,
+ u8 port_num, struct ib_cqe *cqe, struct ib_send_wr *chain_wr);
+int rdma_rw_ctx_post(struct rdma_rw_ctx *ctx, struct ib_qp *qp, u8 port_num,
+ struct ib_cqe *cqe, struct ib_send_wr *chain_wr);
+
+void rdma_rw_init_qp(struct ib_device *dev, struct ib_qp_init_attr *attr);
+int rdma_rw_init_mrs(struct ib_qp *qp, struct ib_qp_init_attr *attr);
+void rdma_rw_cleanup_mrs(struct ib_qp *qp);
+
+#endif /* _RDMA_RW_H */
diff --git a/include/scsi/scsi.h b/include/scsi/scsi.h
index e0a3398..8ec7c30 100644
--- a/include/scsi/scsi.h
+++ b/include/scsi/scsi.h
@@ -18,25 +18,6 @@
};
/*
- * The maximum number of SG segments that we will put inside a
- * scatterlist (unless chaining is used). Should ideally fit inside a
- * single page, to avoid a higher order allocation. We could define this
- * to SG_MAX_SINGLE_ALLOC to pack correctly at the highest order. The
- * minimum value is 32
- */
-#define SCSI_MAX_SG_SEGMENTS 128
-
-/*
- * Like SCSI_MAX_SG_SEGMENTS, but for archs that have sg chaining. This limit
- * is totally arbitrary, a setting of 2048 will get you at least 8mb ios.
- */
-#ifdef CONFIG_ARCH_HAS_SG_CHAIN
-#define SCSI_MAX_SG_CHAIN_SEGMENTS 2048
-#else
-#define SCSI_MAX_SG_CHAIN_SEGMENTS SCSI_MAX_SG_SEGMENTS
-#endif
-
-/*
* DIX-capable adapters effectively support infinite chaining for the
* protection information scatterlist
*/
diff --git a/include/scsi/scsi_common.h b/include/scsi/scsi_common.h
index 11571b2..20bf7ea 100644
--- a/include/scsi/scsi_common.h
+++ b/include/scsi/scsi_common.h
@@ -63,6 +63,7 @@
extern void scsi_build_sense_buffer(int desc, u8 *buf, u8 key, u8 asc, u8 ascq);
int scsi_set_sense_information(u8 *buf, int buf_len, u64 info);
+int scsi_set_sense_field_pointer(u8 *buf, int buf_len, u16 fp, u8 bp, bool cd);
extern const u8 * scsi_sense_desc_find(const u8 * sense_buffer, int sb_len,
int desc_type);
diff --git a/include/scsi/scsi_device.h b/include/scsi/scsi_device.h
index 74d79bd..a6c346d 100644
--- a/include/scsi/scsi_device.h
+++ b/include/scsi/scsi_device.h
@@ -50,6 +50,12 @@
SDEV_CREATED_BLOCK, /* same as above but for created devices */
};
+enum scsi_scan_mode {
+ SCSI_SCAN_INITIAL = 0,
+ SCSI_SCAN_RESCAN,
+ SCSI_SCAN_MANUAL,
+};
+
enum scsi_device_event {
SDEV_EVT_MEDIA_CHANGE = 1, /* media has changed */
SDEV_EVT_INQUIRY_CHANGE_REPORTED, /* 3F 03 UA reported */
@@ -242,6 +248,7 @@
enum scsi_target_state {
STARGET_CREATED = 1,
STARGET_RUNNING,
+ STARGET_REMOVE,
STARGET_DEL,
};
@@ -391,7 +398,8 @@
extern void scsi_target_quiesce(struct scsi_target *);
extern void scsi_target_resume(struct scsi_target *);
extern void scsi_scan_target(struct device *parent, unsigned int channel,
- unsigned int id, u64 lun, int rescan);
+ unsigned int id, u64 lun,
+ enum scsi_scan_mode rescan);
extern void scsi_target_reap(struct scsi_target *);
extern void scsi_target_block(struct device *);
extern void scsi_target_unblock(struct device *, enum scsi_device_state);
@@ -534,9 +542,9 @@
/*
* Although VPD inquiries can go to SCSI-2 type devices,
* some USB ones crash on receiving them, and the pages
- * we currently ask for are for SPC-3 and beyond
+ * we currently ask for are mandatory for SPC-2 and beyond
*/
- if (sdev->scsi_level > SCSI_SPC_2 && !sdev->skip_vpd_pages)
+ if (sdev->scsi_level >= SCSI_SPC_2 && !sdev->skip_vpd_pages)
return 1;
return 0;
}
diff --git a/include/scsi/scsi_eh.h b/include/scsi/scsi_eh.h
index dbb8c64..98d366b 100644
--- a/include/scsi/scsi_eh.h
+++ b/include/scsi/scsi_eh.h
@@ -16,6 +16,7 @@
extern int scsi_block_when_processing_errors(struct scsi_device *);
extern bool scsi_command_normalize_sense(const struct scsi_cmnd *cmd,
struct scsi_sense_hdr *sshdr);
+extern int scsi_check_sense(struct scsi_cmnd *);
static inline bool scsi_sense_is_deferred(const struct scsi_sense_hdr *sshdr)
{
diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
index fcfa3d7..76e9d27 100644
--- a/include/scsi/scsi_host.h
+++ b/include/scsi/scsi_host.h
@@ -37,7 +37,7 @@
* used in one scatter-gather request.
*/
#define SG_NONE 0
-#define SG_ALL SCSI_MAX_SG_SEGMENTS
+#define SG_ALL SG_CHUNK_SIZE
#define MODE_UNKNOWN 0x00
#define MODE_INITIATOR 0x01
diff --git a/include/scsi/scsi_proto.h b/include/scsi/scsi_proto.h
index c2ae21c..d1defd1 100644
--- a/include/scsi/scsi_proto.h
+++ b/include/scsi/scsi_proto.h
@@ -115,6 +115,8 @@
#define VERIFY_16 0x8f
#define SYNCHRONIZE_CACHE_16 0x91
#define WRITE_SAME_16 0x93
+#define ZBC_OUT 0x94
+#define ZBC_IN 0x95
#define SERVICE_ACTION_BIDIRECTIONAL 0x9d
#define SERVICE_ACTION_IN_16 0x9e
#define SERVICE_ACTION_OUT_16 0x9f
@@ -143,6 +145,13 @@
#define MO_SET_PRIORITY 0x0e
#define MO_SET_TIMESTAMP 0x0f
#define MO_MANAGEMENT_PROTOCOL_OUT 0x10
+/* values for ZBC_IN */
+#define ZI_REPORT_ZONES 0x00
+/* values for ZBC_OUT */
+#define ZO_CLOSE_ZONE 0x01
+#define ZO_FINISH_ZONE 0x02
+#define ZO_OPEN_ZONE 0x03
+#define ZO_RESET_WRITE_POINTER 0x04
/* values for variable length command */
#define XDREAD_32 0x03
#define XDWRITE_32 0x04
diff --git a/include/soc/nps/common.h b/include/soc/nps/common.h
new file mode 100644
index 0000000..9b1d43d
--- /dev/null
+++ b/include/soc/nps/common.h
@@ -0,0 +1,166 @@
+/*
+ * Copyright (c) 2016, Mellanox Technologies. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef SOC_NPS_COMMON_H
+#define SOC_NPS_COMMON_H
+
+#ifdef CONFIG_SMP
+#define NPS_IPI_IRQ 5
+#endif
+
+#define NPS_HOST_REG_BASE 0xF6000000
+
+#define NPS_MSU_BLKID 0x018
+
+#define CTOP_INST_RSPI_GIC_0_R12 0x3C56117E
+#define CTOP_INST_MOV2B_FLIP_R3_B1_B2_INST 0x5B60
+#define CTOP_INST_MOV2B_FLIP_R3_B1_B2_LIMM 0x00010422
+
+#ifndef __ASSEMBLY__
+
+/* In order to increase compilation test coverage */
+#ifdef CONFIG_ARC
+static inline void nps_ack_gic(void)
+{
+ __asm__ __volatile__ (
+ " .word %0\n"
+ :
+ : "i"(CTOP_INST_RSPI_GIC_0_R12)
+ : "memory");
+}
+#else
+static inline void nps_ack_gic(void) { }
+#define write_aux_reg(r, v)
+#define read_aux_reg(r) 0
+#endif
+
+/* CPU global ID */
+struct global_id {
+ union {
+ struct {
+#ifdef CONFIG_EZNPS_MTM_EXT
+ u32 __reserved:20, cluster:4, core:4, thread:4;
+#else
+ u32 __reserved:24, cluster:4, core:4;
+#endif
+ };
+ u32 value;
+ };
+};
+
+/*
+ * Convert logical to physical CPU IDs
+ *
+ * The conversion swap bits 1 and 2 of cluster id (out of 4 bits)
+ * Now quad of logical clusters id's are adjacent physically,
+ * and not like the id's physically came with each cluster.
+ * Below table is 4x4 mesh of core clusters as it layout on chip.
+ * Cluster ids are in format: logical (physical)
+ *
+ * ----------------- ------------------
+ * 3 | 5 (3) 7 (7) | | 13 (11) 15 (15)|
+ *
+ * 2 | 4 (2) 6 (6) | | 12 (10) 14 (14)|
+ * ----------------- ------------------
+ * 1 | 1 (1) 3 (5) | | 9 (9) 11 (13)|
+ *
+ * 0 | 0 (0) 2 (4) | | 8 (8) 10 (12)|
+ * ----------------- ------------------
+ * 0 1 2 3
+ */
+static inline int nps_cluster_logic_to_phys(int cluster)
+{
+#ifdef __arc__
+ __asm__ __volatile__(
+ " mov r3,%0\n"
+ " .short %1\n"
+ " .word %2\n"
+ " mov %0,r3\n"
+ : "+r"(cluster)
+ : "i"(CTOP_INST_MOV2B_FLIP_R3_B1_B2_INST),
+ "i"(CTOP_INST_MOV2B_FLIP_R3_B1_B2_LIMM)
+ : "r3");
+#endif
+
+ return cluster;
+}
+
+#define NPS_CPU_TO_CLUSTER_NUM(cpu) \
+ ({ struct global_id gid; gid.value = cpu; \
+ nps_cluster_logic_to_phys(gid.cluster); })
+
+struct nps_host_reg_address {
+ union {
+ struct {
+ u32 base:8, cl_x:4, cl_y:4,
+ blkid:6, reg:8, __reserved:2;
+ };
+ u32 value;
+ };
+};
+
+struct nps_host_reg_address_non_cl {
+ union {
+ struct {
+ u32 base:7, blkid:11, reg:12, __reserved:2;
+ };
+ u32 value;
+ };
+};
+
+static inline void *nps_host_reg_non_cl(u32 blkid, u32 reg)
+{
+ struct nps_host_reg_address_non_cl reg_address;
+
+ reg_address.value = NPS_HOST_REG_BASE;
+ reg_address.blkid = blkid;
+ reg_address.reg = reg;
+
+ return (void *)reg_address.value;
+}
+
+static inline void *nps_host_reg(u32 cpu, u32 blkid, u32 reg)
+{
+ struct nps_host_reg_address reg_address;
+ u32 cl = NPS_CPU_TO_CLUSTER_NUM(cpu);
+
+ reg_address.value = NPS_HOST_REG_BASE;
+ reg_address.cl_x = (cl >> 2) & 0x3;
+ reg_address.cl_y = cl & 0x3;
+ reg_address.blkid = blkid;
+ reg_address.reg = reg;
+
+ return (void *)reg_address.value;
+}
+#endif /* __ASSEMBLY__ */
+
+#endif /* SOC_NPS_COMMON_H */
diff --git a/include/sound/dmaengine_pcm.h b/include/sound/dmaengine_pcm.h
index f86ef5e..67be244 100644
--- a/include/sound/dmaengine_pcm.h
+++ b/include/sound/dmaengine_pcm.h
@@ -51,6 +51,16 @@
void *filter_data);
struct dma_chan *snd_dmaengine_pcm_get_chan(struct snd_pcm_substream *substream);
+/*
+ * The DAI supports packed transfers, eg 2 16-bit samples in a 32-bit word.
+ * If this flag is set the dmaengine driver won't put any restriction on
+ * the supported sample formats and set the DMA transfer size to undefined.
+ * The DAI driver is responsible to disable any unsupported formats in it's
+ * configuration and catch corner cases that are not already handled in
+ * the ALSA core.
+ */
+#define SND_DMAENGINE_PCM_DAI_FLAG_PACK BIT(0)
+
/**
* struct snd_dmaengine_dai_dma_data - DAI DMA configuration data
* @addr: Address of the DAI data source or destination register.
@@ -63,6 +73,7 @@
* requesting the DMA channel.
* @chan_name: Custom channel name to use when requesting DMA channel.
* @fifo_size: FIFO size of the DAI controller in bytes
+ * @flags: PCM_DAI flags, only SND_DMAENGINE_PCM_DAI_FLAG_PACK for now
*/
struct snd_dmaengine_dai_dma_data {
dma_addr_t addr;
@@ -72,6 +83,7 @@
void *filter_data;
const char *chan_name;
unsigned int fifo_size;
+ unsigned int flags;
};
void snd_dmaengine_pcm_set_config_from_dai_data(
diff --git a/include/sound/hda_chmap.h b/include/sound/hda_chmap.h
index e20d219..babd445 100644
--- a/include/sound/hda_chmap.h
+++ b/include/sound/hda_chmap.h
@@ -36,6 +36,8 @@
int (*chmap_validate)(struct hdac_chmap *hchmap, int ca,
int channels, unsigned char *chmap);
+ int (*get_spk_alloc)(struct hdac_device *hdac, int pcm_idx);
+
void (*get_chmap)(struct hdac_device *hdac, int pcm_idx,
unsigned char *chmap);
void (*set_chmap)(struct hdac_device *hdac, int pcm_idx,
diff --git a/include/sound/hda_i915.h b/include/sound/hda_i915.h
index f5842bc..796cabf 100644
--- a/include/sound/hda_i915.h
+++ b/include/sound/hda_i915.h
@@ -10,8 +10,8 @@
int snd_hdac_set_codec_wakeup(struct hdac_bus *bus, bool enable);
int snd_hdac_display_power(struct hdac_bus *bus, bool enable);
void snd_hdac_i915_set_bclk(struct hdac_bus *bus);
-int snd_hdac_sync_audio_rate(struct hdac_bus *bus, hda_nid_t nid, int rate);
-int snd_hdac_acomp_get_eld(struct hdac_bus *bus, hda_nid_t nid,
+int snd_hdac_sync_audio_rate(struct hdac_device *codec, hda_nid_t nid, int rate);
+int snd_hdac_acomp_get_eld(struct hdac_device *codec, hda_nid_t nid,
bool *audio_enabled, char *buffer, int max_bytes);
int snd_hdac_i915_init(struct hdac_bus *bus);
int snd_hdac_i915_exit(struct hdac_bus *bus);
@@ -28,12 +28,12 @@
static inline void snd_hdac_i915_set_bclk(struct hdac_bus *bus)
{
}
-static inline int snd_hdac_sync_audio_rate(struct hdac_bus *bus, hda_nid_t nid,
- int rate)
+static inline int snd_hdac_sync_audio_rate(struct hdac_device *codec,
+ hda_nid_t nid, int rate)
{
return 0;
}
-static inline int snd_hdac_acomp_get_eld(struct hdac_bus *bus, hda_nid_t nid,
+static inline int snd_hdac_acomp_get_eld(struct hdac_device *codec, hda_nid_t nid,
bool *audio_enabled, char *buffer,
int max_bytes)
{
diff --git a/include/sound/hdaudio_ext.h b/include/sound/hdaudio_ext.h
index 07fa592..b9593b2 100644
--- a/include/sound/hdaudio_ext.h
+++ b/include/sound/hdaudio_ext.h
@@ -14,6 +14,8 @@
* @gtscap: gts capabilities pointer
* @drsmcap: dma resume capabilities pointer
* @hlink_list: link list of HDA links
+ * @lock: lock for link mgmt
+ * @cmd_dma_state: state of cmd DMAs: CORB and RIRB
*/
struct hdac_ext_bus {
struct hdac_bus bus;
@@ -27,6 +29,9 @@
void __iomem *drsmcap;
struct list_head hlink_list;
+
+ struct mutex lock;
+ bool cmd_dma_state;
};
int snd_hdac_ext_bus_init(struct hdac_ext_bus *sbus, struct device *dev,
@@ -142,6 +147,9 @@
void __iomem *ml_addr; /* link output stream reg pointer */
u32 lcaps; /* link capablities */
u16 lsdiid; /* link sdi identifier */
+
+ int ref_count;
+
struct list_head list;
};
@@ -154,6 +162,11 @@
void snd_hdac_ext_link_clear_stream_id(struct hdac_ext_link *link,
int stream);
+int snd_hdac_ext_bus_link_get(struct hdac_ext_bus *ebus,
+ struct hdac_ext_link *link);
+int snd_hdac_ext_bus_link_put(struct hdac_ext_bus *ebus,
+ struct hdac_ext_link *link);
+
/* update register macro */
#define snd_hdac_updatel(addr, reg, mask, val) \
writel(((readl(addr + reg) & ~(mask)) | (val)), \
diff --git a/include/sound/hdmi-codec.h b/include/sound/hdmi-codec.h
new file mode 100644
index 0000000..fc3a481
--- /dev/null
+++ b/include/sound/hdmi-codec.h
@@ -0,0 +1,100 @@
+/*
+ * hdmi-codec.h - HDMI Codec driver API
+ *
+ * Copyright (C) 2014 Texas Instruments Incorporated - http://www.ti.com
+ *
+ * Author: Jyri Sarha <jsarha@ti.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ */
+
+#ifndef __HDMI_CODEC_H__
+#define __HDMI_CODEC_H__
+
+#include <linux/hdmi.h>
+#include <drm/drm_edid.h>
+#include <sound/asoundef.h>
+#include <uapi/sound/asound.h>
+
+/*
+ * Protocol between ASoC cpu-dai and HDMI-encoder
+ */
+struct hdmi_codec_daifmt {
+ enum {
+ HDMI_I2S,
+ HDMI_RIGHT_J,
+ HDMI_LEFT_J,
+ HDMI_DSP_A,
+ HDMI_DSP_B,
+ HDMI_AC97,
+ HDMI_SPDIF,
+ } fmt;
+ int bit_clk_inv:1;
+ int frame_clk_inv:1;
+ int bit_clk_master:1;
+ int frame_clk_master:1;
+};
+
+/*
+ * HDMI audio parameters
+ */
+struct hdmi_codec_params {
+ struct hdmi_audio_infoframe cea;
+ struct snd_aes_iec958 iec;
+ int sample_rate;
+ int sample_width;
+ int channels;
+};
+
+struct hdmi_codec_ops {
+ /*
+ * Called when ASoC starts an audio stream setup.
+ * Optional
+ */
+ int (*audio_startup)(struct device *dev);
+
+ /*
+ * Configures HDMI-encoder for audio stream.
+ * Mandatory
+ */
+ int (*hw_params)(struct device *dev,
+ struct hdmi_codec_daifmt *fmt,
+ struct hdmi_codec_params *hparms);
+
+ /*
+ * Shuts down the audio stream.
+ * Mandatory
+ */
+ void (*audio_shutdown)(struct device *dev);
+
+ /*
+ * Mute/unmute HDMI audio stream.
+ * Optional
+ */
+ int (*digital_mute)(struct device *dev, bool enable);
+
+ /*
+ * Provides EDID-Like-Data from connected HDMI device.
+ * Optional
+ */
+ int (*get_eld)(struct device *dev, uint8_t *buf, size_t len);
+};
+
+/* HDMI codec initalization data */
+struct hdmi_codec_pdata {
+ const struct hdmi_codec_ops *ops;
+ uint i2s:1;
+ uint spdif:1;
+ int max_i2s_channels;
+};
+
+#define HDMI_CODEC_DRV_NAME "hdmi-audio-codec"
+
+#endif /* __HDMI_CODEC_H__ */
diff --git a/include/sound/pcm_iec958.h b/include/sound/pcm_iec958.h
index 0eed397..36f023a 100644
--- a/include/sound/pcm_iec958.h
+++ b/include/sound/pcm_iec958.h
@@ -6,4 +6,6 @@
int snd_pcm_create_iec958_consumer(struct snd_pcm_runtime *runtime, u8 *cs,
size_t len);
+int snd_pcm_create_iec958_consumer_hw_params(struct snd_pcm_hw_params *params,
+ u8 *cs, size_t len);
#endif
diff --git a/include/sound/soc-dapm.h b/include/sound/soc-dapm.h
index 9706946..3101d53 100644
--- a/include/sound/soc-dapm.h
+++ b/include/sound/soc-dapm.h
@@ -100,6 +100,7 @@
{ .id = snd_soc_dapm_mixer_named_ctl, .name = wname, \
SND_SOC_DAPM_INIT_REG_VAL(wreg, wshift, winvert), \
.kcontrol_news = wcontrols, .num_kcontrols = wncontrols}
+/* DEPRECATED: use SND_SOC_DAPM_SUPPLY */
#define SND_SOC_DAPM_MICBIAS(wname, wreg, wshift, winvert) \
{ .id = snd_soc_dapm_micbias, .name = wname, \
SND_SOC_DAPM_INIT_REG_VAL(wreg, wshift, winvert), \
@@ -473,7 +474,7 @@
snd_soc_dapm_out_drv, /* output driver */
snd_soc_dapm_adc, /* analog to digital converter */
snd_soc_dapm_dac, /* digital to analog converter */
- snd_soc_dapm_micbias, /* microphone bias (power) */
+ snd_soc_dapm_micbias, /* microphone bias (power) - DEPRECATED: use snd_soc_dapm_supply */
snd_soc_dapm_mic, /* microphone */
snd_soc_dapm_hp, /* headphones */
snd_soc_dapm_spk, /* speaker */
diff --git a/include/sound/soc.h b/include/sound/soc.h
index 02b4a21..fd7b58a 100644
--- a/include/sound/soc.h
+++ b/include/sound/soc.h
@@ -1002,7 +1002,7 @@
*/
const char *platform_name;
struct device_node *platform_of_node;
- int be_id; /* optional ID for machine driver BE identification */
+ int id; /* optional ID for machine driver link identification */
const struct snd_soc_pcm_stream *params;
unsigned int num_params;
@@ -1683,6 +1683,9 @@
int snd_soc_register_dai(struct snd_soc_component *component,
struct snd_soc_dai_driver *dai_drv);
+struct snd_soc_dai *snd_soc_find_dai(
+ const struct snd_soc_dai_link_component *dlc);
+
#include <sound/soc-dai.h>
#ifdef CONFIG_DEBUG_FS
diff --git a/include/target/target_core_backend.h b/include/target/target_core_backend.h
index 28ee5c2..d8ab510 100644
--- a/include/target/target_core_backend.h
+++ b/include/target/target_core_backend.h
@@ -85,7 +85,6 @@
void *transport_kmap_data_sg(struct se_cmd *);
void transport_kunmap_data_sg(struct se_cmd *);
/* core helpers also used by xcopy during internal command setup */
-int target_alloc_sgl(struct scatterlist **, unsigned int *, u32, bool);
sense_reason_t transport_generic_map_mem_to_cmd(struct se_cmd *,
struct scatterlist *, u32, struct scatterlist *, u32);
diff --git a/include/target/target_core_fabric.h b/include/target/target_core_fabric.h
index 8ff6d40..78d88f0 100644
--- a/include/target/target_core_fabric.h
+++ b/include/target/target_core_fabric.h
@@ -185,6 +185,10 @@
int core_tpg_register(struct se_wwn *, struct se_portal_group *, int);
int core_tpg_deregister(struct se_portal_group *);
+int target_alloc_sgl(struct scatterlist **sgl, unsigned int *nents,
+ u32 length, bool zero_page, bool chainable);
+void target_free_sgl(struct scatterlist *sgl, int nents);
+
/*
* The LIO target core uses DMA_TO_DEVICE to mean that data is going
* to the target (eg handling a WRITE) and DMA_FROM_DEVICE to mean
diff --git a/include/trace/events/compaction.h b/include/trace/events/compaction.h
index e215bf6..36e2d6f 100644
--- a/include/trace/events/compaction.h
+++ b/include/trace/events/compaction.h
@@ -10,10 +10,11 @@
#include <trace/events/mmflags.h>
#define COMPACTION_STATUS \
- EM( COMPACT_DEFERRED, "deferred") \
EM( COMPACT_SKIPPED, "skipped") \
+ EM( COMPACT_DEFERRED, "deferred") \
EM( COMPACT_CONTINUE, "continue") \
EM( COMPACT_PARTIAL, "partial") \
+ EM( COMPACT_PARTIAL_SKIPPED, "partial_skipped") \
EM( COMPACT_COMPLETE, "complete") \
EM( COMPACT_NO_SUITABLE_PAGE, "no_suitable_page") \
EM( COMPACT_NOT_SUITABLE_ZONE, "not_suitable_zone") \
diff --git a/include/trace/events/f2fs.h b/include/trace/events/f2fs.h
index 0f56584..3a09bb4 100644
--- a/include/trace/events/f2fs.h
+++ b/include/trace/events/f2fs.h
@@ -694,28 +694,32 @@
__entry->ret)
);
-TRACE_EVENT(f2fs_reserve_new_block,
+TRACE_EVENT(f2fs_reserve_new_blocks,
- TP_PROTO(struct inode *inode, nid_t nid, unsigned int ofs_in_node),
+ TP_PROTO(struct inode *inode, nid_t nid, unsigned int ofs_in_node,
+ blkcnt_t count),
- TP_ARGS(inode, nid, ofs_in_node),
+ TP_ARGS(inode, nid, ofs_in_node, count),
TP_STRUCT__entry(
__field(dev_t, dev)
__field(nid_t, nid)
__field(unsigned int, ofs_in_node)
+ __field(blkcnt_t, count)
),
TP_fast_assign(
__entry->dev = inode->i_sb->s_dev;
__entry->nid = nid;
__entry->ofs_in_node = ofs_in_node;
+ __entry->count = count;
),
- TP_printk("dev = (%d,%d), nid = %u, ofs_in_node = %u",
+ TP_printk("dev = (%d,%d), nid = %u, ofs_in_node = %u, count = %llu",
show_dev(__entry),
(unsigned int)__entry->nid,
- __entry->ofs_in_node)
+ __entry->ofs_in_node,
+ (unsigned long long)__entry->count)
);
DECLARE_EVENT_CLASS(f2fs__submit_page_bio,
@@ -1271,14 +1275,14 @@
DECLARE_EVENT_CLASS(f2fs_sync_dirty_inodes,
- TP_PROTO(struct super_block *sb, int type, int count),
+ TP_PROTO(struct super_block *sb, int type, s64 count),
TP_ARGS(sb, type, count),
TP_STRUCT__entry(
__field(dev_t, dev)
__field(int, type)
- __field(int, count)
+ __field(s64, count)
),
TP_fast_assign(
@@ -1287,7 +1291,7 @@
__entry->count = count;
),
- TP_printk("dev = (%d,%d), %s, dirty count = %d",
+ TP_printk("dev = (%d,%d), %s, dirty count = %lld",
show_dev(__entry),
show_file_type(__entry->type),
__entry->count)
@@ -1295,14 +1299,14 @@
DEFINE_EVENT(f2fs_sync_dirty_inodes, f2fs_sync_dirty_inodes_enter,
- TP_PROTO(struct super_block *sb, int type, int count),
+ TP_PROTO(struct super_block *sb, int type, s64 count),
TP_ARGS(sb, type, count)
);
DEFINE_EVENT(f2fs_sync_dirty_inodes, f2fs_sync_dirty_inodes_exit,
- TP_PROTO(struct super_block *sb, int type, int count),
+ TP_PROTO(struct super_block *sb, int type, s64 count),
TP_ARGS(sb, type, count)
);
diff --git a/include/trace/events/kvm.h b/include/trace/events/kvm.h
index aa69253..526fb3d 100644
--- a/include/trace/events/kvm.h
+++ b/include/trace/events/kvm.h
@@ -38,22 +38,25 @@
);
TRACE_EVENT(kvm_vcpu_wakeup,
- TP_PROTO(__u64 ns, bool waited),
- TP_ARGS(ns, waited),
+ TP_PROTO(__u64 ns, bool waited, bool valid),
+ TP_ARGS(ns, waited, valid),
TP_STRUCT__entry(
__field( __u64, ns )
__field( bool, waited )
+ __field( bool, valid )
),
TP_fast_assign(
__entry->ns = ns;
__entry->waited = waited;
+ __entry->valid = valid;
),
- TP_printk("%s time %lld ns",
+ TP_printk("%s time %lld ns, polling %s",
__entry->waited ? "wait" : "poll",
- __entry->ns)
+ __entry->ns,
+ __entry->valid ? "valid" : "invalid")
);
#if defined(CONFIG_HAVE_KVM_IRQFD)
diff --git a/include/trace/events/libata.h b/include/trace/events/libata.h
index 8b0fbd9..75fff86 100644
--- a/include/trace/events/libata.h
+++ b/include/trace/events/libata.h
@@ -39,6 +39,7 @@
ata_opcode_name(ATA_CMD_WRITE_QUEUED_FUA_EXT), \
ata_opcode_name(ATA_CMD_FPDMA_READ), \
ata_opcode_name(ATA_CMD_FPDMA_WRITE), \
+ ata_opcode_name(ATA_CMD_NCQ_NON_DATA), \
ata_opcode_name(ATA_CMD_FPDMA_SEND), \
ata_opcode_name(ATA_CMD_FPDMA_RECV), \
ata_opcode_name(ATA_CMD_PIO_READ), \
@@ -97,6 +98,8 @@
ata_opcode_name(ATA_CMD_CFA_WRITE_MULT_NE), \
ata_opcode_name(ATA_CMD_REQ_SENSE_DATA), \
ata_opcode_name(ATA_CMD_SANITIZE_DEVICE), \
+ ata_opcode_name(ATA_CMD_ZAC_MGMT_IN), \
+ ata_opcode_name(ATA_CMD_ZAC_MGMT_OUT), \
ata_opcode_name(ATA_CMD_RESTORE), \
ata_opcode_name(ATA_CMD_READ_LONG), \
ata_opcode_name(ATA_CMD_READ_LONG_ONCE), \
@@ -139,6 +142,10 @@
const char *libata_trace_parse_qc_flags(struct trace_seq *, unsigned int);
#define __parse_qc_flags(f) libata_trace_parse_qc_flags(p, f)
+const char *libata_trace_parse_subcmd(struct trace_seq *, unsigned char,
+ unsigned char, unsigned char);
+#define __parse_subcmd(c,f,h) libata_trace_parse_subcmd(p, c, f, h)
+
TRACE_EVENT(ata_qc_issue,
TP_PROTO(struct ata_queued_cmd *qc),
@@ -185,11 +192,12 @@
__entry->hob_nsect = qc->tf.hob_nsect;
),
- TP_printk("ata_port=%u ata_dev=%u tag=%d proto=%s cmd=%s " \
+ TP_printk("ata_port=%u ata_dev=%u tag=%d proto=%s cmd=%s%s " \
" tf=(%02x/%02x:%02x:%02x:%02x:%02x/%02x:%02x:%02x:%02x:%02x/%02x)",
__entry->ata_port, __entry->ata_dev, __entry->tag,
show_protocol_name(__entry->proto),
show_opcode_name(__entry->cmd),
+ __parse_subcmd(__entry->cmd, __entry->feature, __entry->hob_nsect),
__entry->cmd, __entry->feature, __entry->nsect,
__entry->lbal, __entry->lbam, __entry->lbah,
__entry->hob_feature, __entry->hob_nsect,
diff --git a/include/trace/events/scsi.h b/include/trace/events/scsi.h
index 079bd10..9a9b3e2 100644
--- a/include/trace/events/scsi.h
+++ b/include/trace/events/scsi.h
@@ -94,11 +94,9 @@
scsi_opcode_name(WRITE_16), \
scsi_opcode_name(VERIFY_16), \
scsi_opcode_name(WRITE_SAME_16), \
+ scsi_opcode_name(ZBC_OUT), \
+ scsi_opcode_name(ZBC_IN), \
scsi_opcode_name(SERVICE_ACTION_IN_16), \
- scsi_opcode_name(SAI_READ_CAPACITY_16), \
- scsi_opcode_name(SAI_GET_LBA_STATUS), \
- scsi_opcode_name(MI_REPORT_TARGET_PGS), \
- scsi_opcode_name(MO_SET_TARGET_PGS), \
scsi_opcode_name(READ_32), \
scsi_opcode_name(WRITE_32), \
scsi_opcode_name(WRITE_SAME_32), \
diff --git a/include/uapi/asm-generic/unistd.h b/include/uapi/asm-generic/unistd.h
index c51afb7..a26415b 100644
--- a/include/uapi/asm-generic/unistd.h
+++ b/include/uapi/asm-generic/unistd.h
@@ -127,8 +127,11 @@
__SYSCALL(__NR_symlinkat, sys_symlinkat)
#define __NR_linkat 37
__SYSCALL(__NR_linkat, sys_linkat)
+#ifdef __ARCH_WANT_RENAMEAT
+/* renameat is superseded with flags by renameat2 */
#define __NR_renameat 38
__SYSCALL(__NR_renameat, sys_renameat)
+#endif /* __ARCH_WANT_RENAMEAT */
/* fs/namespace.c */
#define __NR_umount2 39
diff --git a/include/uapi/drm/amdgpu_drm.h b/include/uapi/drm/amdgpu_drm.h
index 453a76a..cdecf87 100644
--- a/include/uapi/drm/amdgpu_drm.h
+++ b/include/uapi/drm/amdgpu_drm.h
@@ -34,6 +34,10 @@
#include "drm.h"
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
#define DRM_AMDGPU_GEM_CREATE 0x00
#define DRM_AMDGPU_GEM_MMAP 0x01
#define DRM_AMDGPU_CTX 0x02
@@ -642,4 +646,8 @@
#define AMDGPU_FAMILY_VI 130 /* Iceland, Tonga */
#define AMDGPU_FAMILY_CZ 135 /* Carrizo, Stoney */
+#if defined(__cplusplus)
+}
+#endif
+
#endif
diff --git a/include/uapi/drm/armada_drm.h b/include/uapi/drm/armada_drm.h
index 6de7f01..72e326f 100644
--- a/include/uapi/drm/armada_drm.h
+++ b/include/uapi/drm/armada_drm.h
@@ -11,6 +11,10 @@
#include "drm.h"
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
#define DRM_ARMADA_GEM_CREATE 0x00
#define DRM_ARMADA_GEM_MMAP 0x02
#define DRM_ARMADA_GEM_PWRITE 0x03
@@ -44,4 +48,8 @@
#define DRM_IOCTL_ARMADA_GEM_PWRITE \
ARMADA_IOCTL(IOW, GEM_PWRITE, gem_pwrite)
+#if defined(__cplusplus)
+}
+#endif
+
#endif
diff --git a/include/uapi/drm/drm.h b/include/uapi/drm/drm.h
index a0ebfe7..452675f 100644
--- a/include/uapi/drm/drm.h
+++ b/include/uapi/drm/drm.h
@@ -36,7 +36,13 @@
#ifndef _DRM_H_
#define _DRM_H_
-#if defined(__KERNEL__) || defined(__linux__)
+#if defined(__KERNEL__)
+
+#include <linux/types.h>
+#include <asm/ioctl.h>
+typedef unsigned int drm_handle_t;
+
+#elif defined(__linux__)
#include <linux/types.h>
#include <asm/ioctl.h>
@@ -59,6 +65,10 @@
#endif
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
#define DRM_NAME "drm" /**< Name in kernel, /dev, and /proc */
#define DRM_MIN_ORDER 5 /**< At least 2^5 bytes = 32 bytes */
#define DRM_MAX_ORDER 22 /**< Up to 2^22 bytes = 4MB */
@@ -181,7 +191,7 @@
_DRM_SHM = 2, /**< shared, cached */
_DRM_AGP = 3, /**< AGP/GART */
_DRM_SCATTER_GATHER = 4, /**< Scatter/gather memory for PCI DMA */
- _DRM_CONSISTENT = 5, /**< Consistent memory for PCI DMA */
+ _DRM_CONSISTENT = 5 /**< Consistent memory for PCI DMA */
};
/**
@@ -373,7 +383,11 @@
*/
struct drm_buf_map {
int count; /**< Length of the buffer list */
+#ifdef __cplusplus
+ void __user *virt;
+#else
void __user *virtual; /**< Mmap'd area in user-virtual */
+#endif
struct drm_buf_pub __user *list; /**< Buffer information */
};
@@ -431,7 +445,7 @@
* DRM_IOCTL_UPDATE_DRAW ioctl argument type.
*/
typedef enum {
- DRM_DRAWABLE_CLIPRECTS,
+ DRM_DRAWABLE_CLIPRECTS
} drm_drawable_info_type_t;
struct drm_update_draw {
@@ -681,7 +695,15 @@
__s32 fd;
};
-#include <drm/drm_mode.h>
+#if defined(__cplusplus)
+}
+#endif
+
+#include "drm_mode.h"
+
+#if defined(__cplusplus)
+extern "C" {
+#endif
#define DRM_IOCTL_BASE 'd'
#define DRM_IO(nr) _IO(DRM_IOCTL_BASE,nr)
@@ -876,4 +898,8 @@
typedef struct drm_set_version drm_set_version_t;
#endif
+#if defined(__cplusplus)
+}
+#endif
+
#endif
diff --git a/include/uapi/drm/drm_fourcc.h b/include/uapi/drm/drm_fourcc.h
index 4d8da69..a5890bf 100644
--- a/include/uapi/drm/drm_fourcc.h
+++ b/include/uapi/drm/drm_fourcc.h
@@ -26,6 +26,10 @@
#include "drm.h"
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
#define fourcc_code(a, b, c, d) ((__u32)(a) | ((__u32)(b) << 8) | \
((__u32)(c) << 16) | ((__u32)(d) << 24))
@@ -229,4 +233,8 @@
*/
#define DRM_FORMAT_MOD_SAMSUNG_64_32_TILE fourcc_mod_code(SAMSUNG, 1)
+#if defined(__cplusplus)
+}
+#endif
+
#endif /* DRM_FOURCC_H */
diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h
index c021743..49a7265 100644
--- a/include/uapi/drm/drm_mode.h
+++ b/include/uapi/drm/drm_mode.h
@@ -29,6 +29,10 @@
#include "drm.h"
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
#define DRM_DISPLAY_INFO_LEN 32
#define DRM_CONNECTOR_NAME_LEN 32
#define DRM_DISPLAY_MODE_LEN 32
@@ -202,6 +206,7 @@
#define DRM_MODE_ENCODER_VIRTUAL 5
#define DRM_MODE_ENCODER_DSI 6
#define DRM_MODE_ENCODER_DPMST 7
+#define DRM_MODE_ENCODER_DPI 8
struct drm_mode_get_encoder {
__u32 encoder_id;
@@ -241,6 +246,7 @@
#define DRM_MODE_CONNECTOR_eDP 14
#define DRM_MODE_CONNECTOR_VIRTUAL 15
#define DRM_MODE_CONNECTOR_DSI 16
+#define DRM_MODE_CONNECTOR_DPI 17
struct drm_mode_get_connector {
@@ -320,6 +326,16 @@
__u32 connector_id;
};
+#define DRM_MODE_OBJECT_CRTC 0xcccccccc
+#define DRM_MODE_OBJECT_CONNECTOR 0xc0c0c0c0
+#define DRM_MODE_OBJECT_ENCODER 0xe0e0e0e0
+#define DRM_MODE_OBJECT_MODE 0xdededede
+#define DRM_MODE_OBJECT_PROPERTY 0xb0b0b0b0
+#define DRM_MODE_OBJECT_FB 0xfbfbfbfb
+#define DRM_MODE_OBJECT_BLOB 0xbbbbbbbb
+#define DRM_MODE_OBJECT_PLANE 0xeeeeeeee
+#define DRM_MODE_OBJECT_ANY 0
+
struct drm_mode_obj_get_properties {
__u64 props_ptr;
__u64 prop_values_ptr;
@@ -611,4 +627,8 @@
__u32 blob_id;
};
+#if defined(__cplusplus)
+}
+#endif
+
#endif
diff --git a/include/uapi/drm/drm_sarea.h b/include/uapi/drm/drm_sarea.h
index 1d1a858..a951ced 100644
--- a/include/uapi/drm/drm_sarea.h
+++ b/include/uapi/drm/drm_sarea.h
@@ -34,6 +34,10 @@
#include "drm.h"
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
/* SAREA area needs to be at least a page */
#if defined(__alpha__)
#define SAREA_MAX 0x2000U
@@ -83,4 +87,8 @@
typedef struct drm_sarea drm_sarea_t;
#endif
+#if defined(__cplusplus)
+}
+#endif
+
#endif /* _DRM_SAREA_H_ */
diff --git a/include/uapi/drm/etnaviv_drm.h b/include/uapi/drm/etnaviv_drm.h
index f95e1c4..2584c1c 100644
--- a/include/uapi/drm/etnaviv_drm.h
+++ b/include/uapi/drm/etnaviv_drm.h
@@ -19,6 +19,10 @@
#include "drm.h"
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
/* Please note that modifications to all structs defined here are
* subject to backwards-compatibility constraints:
* 1) Do not use pointers, use __u64 instead for 32 bit / 64 bit
@@ -222,4 +226,8 @@
#define DRM_IOCTL_ETNAVIV_GEM_USERPTR DRM_IOWR(DRM_COMMAND_BASE + DRM_ETNAVIV_GEM_USERPTR, struct drm_etnaviv_gem_userptr)
#define DRM_IOCTL_ETNAVIV_GEM_WAIT DRM_IOW(DRM_COMMAND_BASE + DRM_ETNAVIV_GEM_WAIT, struct drm_etnaviv_gem_wait)
+#if defined(__cplusplus)
+}
+#endif
+
#endif /* __ETNAVIV_DRM_H__ */
diff --git a/include/uapi/drm/exynos_drm.h b/include/uapi/drm/exynos_drm.h
index 3947c2e..cb3e9f9 100644
--- a/include/uapi/drm/exynos_drm.h
+++ b/include/uapi/drm/exynos_drm.h
@@ -17,6 +17,10 @@
#include "drm.h"
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
/**
* User-desired buffer creation information structure.
*
@@ -362,4 +366,8 @@
__u32 buf_id[EXYNOS_DRM_OPS_MAX];
};
+#if defined(__cplusplus)
+}
+#endif
+
#endif /* _UAPI_EXYNOS_DRM_H_ */
diff --git a/include/uapi/drm/i810_drm.h b/include/uapi/drm/i810_drm.h
index bdb0287..6e6cf86 100644
--- a/include/uapi/drm/i810_drm.h
+++ b/include/uapi/drm/i810_drm.h
@@ -3,6 +3,10 @@
#include "drm.h"
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
/* WARNING: These defines must be the same as what the Xserver uses.
* if you change them, you must change the defines in the Xserver.
*/
@@ -280,4 +284,8 @@
unsigned int last_render; /* Last Render Request */
} drm_i810_mc_t;
+#if defined(__cplusplus)
+}
+#endif
+
#endif /* _I810_DRM_H_ */
diff --git a/include/uapi/drm/i915_drm.h b/include/uapi/drm/i915_drm.h
index a5524cc..c17d63d 100644
--- a/include/uapi/drm/i915_drm.h
+++ b/include/uapi/drm/i915_drm.h
@@ -29,6 +29,10 @@
#include "drm.h"
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
/* Please note that modifications to all structs defined here are
* subject to backwards-compatibility constraints.
*/
@@ -1170,4 +1174,8 @@
__u64 value;
};
+#if defined(__cplusplus)
+}
+#endif
+
#endif /* _UAPI_I915_DRM_H_ */
diff --git a/include/uapi/drm/mga_drm.h b/include/uapi/drm/mga_drm.h
index fca8170..8c43375 100644
--- a/include/uapi/drm/mga_drm.h
+++ b/include/uapi/drm/mga_drm.h
@@ -37,6 +37,10 @@
#include "drm.h"
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
/* WARNING: If you change any of these defines, make sure to change the
* defines in the Xserver file (mga_sarea.h)
*/
@@ -416,4 +420,8 @@
void __user *value;
} drm_mga_getparam_t;
+#if defined(__cplusplus)
+}
+#endif
+
#endif
diff --git a/include/uapi/drm/msm_drm.h b/include/uapi/drm/msm_drm.h
index 254d3e9..bf19d2c 100644
--- a/include/uapi/drm/msm_drm.h
+++ b/include/uapi/drm/msm_drm.h
@@ -20,6 +20,10 @@
#include "drm.h"
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
/* Please note that modifications to all structs defined here are
* subject to backwards-compatibility constraints:
* 1) Do not use pointers, use __u64 instead for 32 bit / 64 bit
@@ -217,4 +221,8 @@
#define DRM_IOCTL_MSM_GEM_SUBMIT DRM_IOWR(DRM_COMMAND_BASE + DRM_MSM_GEM_SUBMIT, struct drm_msm_gem_submit)
#define DRM_IOCTL_MSM_WAIT_FENCE DRM_IOW (DRM_COMMAND_BASE + DRM_MSM_WAIT_FENCE, struct drm_msm_wait_fence)
+#if defined(__cplusplus)
+}
+#endif
+
#endif /* __MSM_DRM_H__ */
diff --git a/include/uapi/drm/nouveau_drm.h b/include/uapi/drm/nouveau_drm.h
index 500d82a..259588a4b 100644
--- a/include/uapi/drm/nouveau_drm.h
+++ b/include/uapi/drm/nouveau_drm.h
@@ -27,7 +27,11 @@
#define DRM_NOUVEAU_EVENT_NVIF 0x80000000
-#include <drm/drm.h>
+#include "drm.h"
+
+#if defined(__cplusplus)
+extern "C" {
+#endif
#define NOUVEAU_GEM_DOMAIN_CPU (1 << 0)
#define NOUVEAU_GEM_DOMAIN_VRAM (1 << 1)
@@ -141,4 +145,8 @@
#define DRM_IOCTL_NOUVEAU_GEM_CPU_FINI DRM_IOW (DRM_COMMAND_BASE + DRM_NOUVEAU_GEM_CPU_FINI, struct drm_nouveau_gem_cpu_fini)
#define DRM_IOCTL_NOUVEAU_GEM_INFO DRM_IOWR(DRM_COMMAND_BASE + DRM_NOUVEAU_GEM_INFO, struct drm_nouveau_gem_info)
+#if defined(__cplusplus)
+}
+#endif
+
#endif /* __NOUVEAU_DRM_H__ */
diff --git a/include/uapi/drm/omap_drm.h b/include/uapi/drm/omap_drm.h
index 38a3bd8..407cb55 100644
--- a/include/uapi/drm/omap_drm.h
+++ b/include/uapi/drm/omap_drm.h
@@ -22,6 +22,10 @@
#include "drm.h"
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
/* Please note that modifications to all structs defined here are
* subject to backwards-compatibility constraints.
*/
@@ -114,4 +118,8 @@
#define DRM_IOCTL_OMAP_GEM_CPU_FINI DRM_IOW (DRM_COMMAND_BASE + DRM_OMAP_GEM_CPU_FINI, struct drm_omap_gem_cpu_fini)
#define DRM_IOCTL_OMAP_GEM_INFO DRM_IOWR(DRM_COMMAND_BASE + DRM_OMAP_GEM_INFO, struct drm_omap_gem_info)
+#if defined(__cplusplus)
+}
+#endif
+
#endif /* __OMAP_DRM_H__ */
diff --git a/include/uapi/drm/qxl_drm.h b/include/uapi/drm/qxl_drm.h
index 4d1e326..7eef422 100644
--- a/include/uapi/drm/qxl_drm.h
+++ b/include/uapi/drm/qxl_drm.h
@@ -26,6 +26,10 @@
#include "drm.h"
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
/* Please note that modifications to all structs defined here are
* subject to backwards-compatibility constraints.
*
@@ -84,7 +88,6 @@
__u32 pad;
};
-/* XXX: call it drm_qxl_commands? */
struct drm_qxl_execbuffer {
__u32 flags; /* for future use */
__u32 commands_num;
@@ -148,4 +151,8 @@
DRM_IOWR(DRM_COMMAND_BASE + DRM_QXL_ALLOC_SURF,\
struct drm_qxl_alloc_surf)
+#if defined(__cplusplus)
+}
+#endif
+
#endif
diff --git a/include/uapi/drm/r128_drm.h b/include/uapi/drm/r128_drm.h
index 7a44c65..690e9c6 100644
--- a/include/uapi/drm/r128_drm.h
+++ b/include/uapi/drm/r128_drm.h
@@ -35,6 +35,10 @@
#include "drm.h"
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
/* WARNING: If you change any of these defines, make sure to change the
* defines in the X server file (r128_sarea.h)
*/
@@ -325,4 +329,8 @@
void __user *value;
} drm_r128_getparam_t;
+#if defined(__cplusplus)
+}
+#endif
+
#endif
diff --git a/include/uapi/drm/radeon_drm.h b/include/uapi/drm/radeon_drm.h
index ccb9bcd..490a59c 100644
--- a/include/uapi/drm/radeon_drm.h
+++ b/include/uapi/drm/radeon_drm.h
@@ -35,6 +35,10 @@
#include "drm.h"
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
/* WARNING: If you change any of these defines, make sure to change the
* defines in the X server file (radeon_sarea.h)
*/
@@ -1067,4 +1071,8 @@
#define CIK_TILE_MODE_DEPTH_STENCIL_1D 5
+#if defined(__cplusplus)
+}
+#endif
+
#endif
diff --git a/include/uapi/drm/savage_drm.h b/include/uapi/drm/savage_drm.h
index 5741474..0f6edde 100644
--- a/include/uapi/drm/savage_drm.h
+++ b/include/uapi/drm/savage_drm.h
@@ -28,6 +28,10 @@
#include "drm.h"
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
#ifndef __SAVAGE_SAREA_DEFINES__
#define __SAVAGE_SAREA_DEFINES__
@@ -209,4 +213,8 @@
} clear1; /* SAVAGE_CMD_CLEAR data */
};
+#if defined(__cplusplus)
+}
+#endif
+
#endif
diff --git a/include/uapi/drm/sis_drm.h b/include/uapi/drm/sis_drm.h
index 374858c..3e3f7e9 100644
--- a/include/uapi/drm/sis_drm.h
+++ b/include/uapi/drm/sis_drm.h
@@ -27,6 +27,12 @@
#ifndef __SIS_DRM_H__
#define __SIS_DRM_H__
+#include "drm.h"
+
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
/* SiS specific ioctls */
#define NOT_USED_0_3
#define DRM_SIS_FB_ALLOC 0x04
@@ -64,4 +70,8 @@
unsigned long offset, size;
} drm_sis_fb_t;
+#if defined(__cplusplus)
+}
+#endif
+
#endif /* __SIS_DRM_H__ */
diff --git a/include/uapi/drm/tegra_drm.h b/include/uapi/drm/tegra_drm.h
index 27d0b05..d954f8c 100644
--- a/include/uapi/drm/tegra_drm.h
+++ b/include/uapi/drm/tegra_drm.h
@@ -25,6 +25,10 @@
#include "drm.h"
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
#define DRM_TEGRA_GEM_CREATE_TILED (1 << 0)
#define DRM_TEGRA_GEM_CREATE_BOTTOM_UP (1 << 1)
@@ -198,4 +202,8 @@
#define DRM_IOCTL_TEGRA_GEM_SET_FLAGS DRM_IOWR(DRM_COMMAND_BASE + DRM_TEGRA_GEM_SET_FLAGS, struct drm_tegra_gem_set_flags)
#define DRM_IOCTL_TEGRA_GEM_GET_FLAGS DRM_IOWR(DRM_COMMAND_BASE + DRM_TEGRA_GEM_GET_FLAGS, struct drm_tegra_gem_get_flags)
+#if defined(__cplusplus)
+}
+#endif
+
#endif
diff --git a/include/uapi/drm/vc4_drm.h b/include/uapi/drm/vc4_drm.h
index eeb37e3..af12e8a 100644
--- a/include/uapi/drm/vc4_drm.h
+++ b/include/uapi/drm/vc4_drm.h
@@ -26,6 +26,10 @@
#include "drm.h"
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
#define DRM_VC4_SUBMIT_CL 0x00
#define DRM_VC4_WAIT_SEQNO 0x01
#define DRM_VC4_WAIT_BO 0x02
@@ -276,4 +280,8 @@
__u32 pad[16];
};
+#if defined(__cplusplus)
+}
+#endif
+
#endif /* _UAPI_VC4_DRM_H_ */
diff --git a/include/uapi/drm/via_drm.h b/include/uapi/drm/via_drm.h
index fa21ed1..a1e125d 100644
--- a/include/uapi/drm/via_drm.h
+++ b/include/uapi/drm/via_drm.h
@@ -26,6 +26,10 @@
#include "drm.h"
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
/* WARNING: These defines must be the same as what the Xserver uses.
* if you change them, you must change the defines in the Xserver.
*/
@@ -271,4 +275,8 @@
drm_via_blitsync_t sync;
} drm_via_dmablit_t;
+#if defined(__cplusplus)
+}
+#endif
+
#endif /* _VIA_DRM_H_ */
diff --git a/include/uapi/drm/virtgpu_drm.h b/include/uapi/drm/virtgpu_drm.h
index c74f1f9..91a31ff 100644
--- a/include/uapi/drm/virtgpu_drm.h
+++ b/include/uapi/drm/virtgpu_drm.h
@@ -26,6 +26,10 @@
#include "drm.h"
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
/* Please note that modifications to all structs defined here are
* subject to backwards-compatibility constraints.
*
@@ -163,4 +167,8 @@
DRM_IOWR(DRM_COMMAND_BASE + DRM_VIRTGPU_GET_CAPS, \
struct drm_virtgpu_get_caps)
+#if defined(__cplusplus)
+}
+#endif
+
#endif
diff --git a/include/uapi/drm/vmwgfx_drm.h b/include/uapi/drm/vmwgfx_drm.h
index 5b68b4d..d325a41 100644
--- a/include/uapi/drm/vmwgfx_drm.h
+++ b/include/uapi/drm/vmwgfx_drm.h
@@ -30,6 +30,10 @@
#include "drm.h"
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
#define DRM_VMW_MAX_SURFACE_FACES 6
#define DRM_VMW_MAX_MIP_LEVELS 24
@@ -1087,4 +1091,9 @@
enum drm_vmw_extended_context req;
struct drm_vmw_context_arg rep;
};
+
+#if defined(__cplusplus)
+}
+#endif
+
#endif
diff --git a/include/uapi/linux/btrfs.h b/include/uapi/linux/btrfs.h
index dea8931..23c6960 100644
--- a/include/uapi/linux/btrfs.h
+++ b/include/uapi/linux/btrfs.h
@@ -23,6 +23,7 @@
#define BTRFS_IOCTL_MAGIC 0x94
#define BTRFS_VOL_NAME_MAX 255
+#define BTRFS_LABEL_SIZE 256
/* this should be 4k */
#define BTRFS_PATH_NAME_MAX 4087
@@ -33,14 +34,31 @@
#define BTRFS_DEVICE_PATH_NAME_MAX 1024
-#define BTRFS_SUBVOL_CREATE_ASYNC (1ULL << 0)
-#define BTRFS_SUBVOL_RDONLY (1ULL << 1)
-#define BTRFS_SUBVOL_QGROUP_INHERIT (1ULL << 2)
+#define BTRFS_DEVICE_SPEC_BY_ID (1ULL << 3)
+
+#define BTRFS_VOL_ARG_V2_FLAGS_SUPPORTED \
+ (BTRFS_SUBVOL_CREATE_ASYNC | \
+ BTRFS_SUBVOL_RDONLY | \
+ BTRFS_SUBVOL_QGROUP_INHERIT | \
+ BTRFS_DEVICE_SPEC_BY_ID)
+
#define BTRFS_FSID_SIZE 16
#define BTRFS_UUID_SIZE 16
#define BTRFS_UUID_UNPARSED_SIZE 37
-#define BTRFS_QGROUP_INHERIT_SET_LIMITS (1ULL << 0)
+/*
+ * flags definition for qgroup limits
+ *
+ * Used by:
+ * struct btrfs_qgroup_limit.flags
+ * struct btrfs_qgroup_limit_item.flags
+ */
+#define BTRFS_QGROUP_LIMIT_MAX_RFER (1ULL << 0)
+#define BTRFS_QGROUP_LIMIT_MAX_EXCL (1ULL << 1)
+#define BTRFS_QGROUP_LIMIT_RSV_RFER (1ULL << 2)
+#define BTRFS_QGROUP_LIMIT_RSV_EXCL (1ULL << 3)
+#define BTRFS_QGROUP_LIMIT_RFER_CMPR (1ULL << 4)
+#define BTRFS_QGROUP_LIMIT_EXCL_CMPR (1ULL << 5)
struct btrfs_qgroup_limit {
__u64 flags;
@@ -50,6 +68,14 @@
__u64 rsv_excl;
};
+/*
+ * flags definition for qgroup inheritance
+ *
+ * Used by:
+ * struct btrfs_qgroup_inherit.flags
+ */
+#define BTRFS_QGROUP_INHERIT_SET_LIMITS (1ULL << 0)
+
struct btrfs_qgroup_inherit {
__u64 flags;
__u64 num_qgroups;
@@ -64,6 +90,20 @@
struct btrfs_qgroup_limit lim;
};
+/*
+ * flags for subvolumes
+ *
+ * Used by:
+ * struct btrfs_ioctl_vol_args_v2.flags
+ *
+ * BTRFS_SUBVOL_RDONLY is also provided/consumed by the following ioctls:
+ * - BTRFS_IOC_SUBVOL_GETFLAGS
+ * - BTRFS_IOC_SUBVOL_SETFLAGS
+ */
+#define BTRFS_SUBVOL_CREATE_ASYNC (1ULL << 0)
+#define BTRFS_SUBVOL_RDONLY (1ULL << 1)
+#define BTRFS_SUBVOL_QGROUP_INHERIT (1ULL << 2)
+
#define BTRFS_SUBVOL_NAME_MAX 4039
struct btrfs_ioctl_vol_args_v2 {
__s64 fd;
@@ -76,7 +116,10 @@
};
__u64 unused[4];
};
- char name[BTRFS_SUBVOL_NAME_MAX + 1];
+ union {
+ char name[BTRFS_SUBVOL_NAME_MAX + 1];
+ u64 devid;
+ };
};
/*
@@ -190,6 +233,37 @@
__u64 reserved[122]; /* pad to 1k */
};
+/*
+ * feature flags
+ *
+ * Used by:
+ * struct btrfs_ioctl_feature_flags
+ */
+#define BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE (1ULL << 0)
+
+#define BTRFS_FEATURE_INCOMPAT_MIXED_BACKREF (1ULL << 0)
+#define BTRFS_FEATURE_INCOMPAT_DEFAULT_SUBVOL (1ULL << 1)
+#define BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS (1ULL << 2)
+#define BTRFS_FEATURE_INCOMPAT_COMPRESS_LZO (1ULL << 3)
+/*
+ * some patches floated around with a second compression method
+ * lets save that incompat here for when they do get in
+ * Note we don't actually support it, we're just reserving the
+ * number
+ */
+#define BTRFS_FEATURE_INCOMPAT_COMPRESS_LZOv2 (1ULL << 4)
+
+/*
+ * older kernels tried to do bigger metadata blocks, but the
+ * code was pretty buggy. Lets not let them try anymore.
+ */
+#define BTRFS_FEATURE_INCOMPAT_BIG_METADATA (1ULL << 5)
+
+#define BTRFS_FEATURE_INCOMPAT_EXTENDED_IREF (1ULL << 6)
+#define BTRFS_FEATURE_INCOMPAT_RAID56 (1ULL << 7)
+#define BTRFS_FEATURE_INCOMPAT_SKINNY_METADATA (1ULL << 8)
+#define BTRFS_FEATURE_INCOMPAT_NO_HOLES (1ULL << 9)
+
struct btrfs_ioctl_feature_flags {
__u64 compat_flags;
__u64 compat_ro_flags;
@@ -254,6 +328,70 @@
__u64 completed; /* # of chunks relocated so far */
};
+/*
+ * flags definition for balance
+ *
+ * Restriper's general type filter
+ *
+ * Used by:
+ * btrfs_ioctl_balance_args.flags
+ * btrfs_balance_control.flags (internal)
+ */
+#define BTRFS_BALANCE_DATA (1ULL << 0)
+#define BTRFS_BALANCE_SYSTEM (1ULL << 1)
+#define BTRFS_BALANCE_METADATA (1ULL << 2)
+
+#define BTRFS_BALANCE_TYPE_MASK (BTRFS_BALANCE_DATA | \
+ BTRFS_BALANCE_SYSTEM | \
+ BTRFS_BALANCE_METADATA)
+
+#define BTRFS_BALANCE_FORCE (1ULL << 3)
+#define BTRFS_BALANCE_RESUME (1ULL << 4)
+
+/*
+ * flags definitions for per-type balance args
+ *
+ * Balance filters
+ *
+ * Used by:
+ * struct btrfs_balance_args
+ */
+#define BTRFS_BALANCE_ARGS_PROFILES (1ULL << 0)
+#define BTRFS_BALANCE_ARGS_USAGE (1ULL << 1)
+#define BTRFS_BALANCE_ARGS_DEVID (1ULL << 2)
+#define BTRFS_BALANCE_ARGS_DRANGE (1ULL << 3)
+#define BTRFS_BALANCE_ARGS_VRANGE (1ULL << 4)
+#define BTRFS_BALANCE_ARGS_LIMIT (1ULL << 5)
+#define BTRFS_BALANCE_ARGS_LIMIT_RANGE (1ULL << 6)
+#define BTRFS_BALANCE_ARGS_STRIPES_RANGE (1ULL << 7)
+#define BTRFS_BALANCE_ARGS_USAGE_RANGE (1ULL << 10)
+
+#define BTRFS_BALANCE_ARGS_MASK \
+ (BTRFS_BALANCE_ARGS_PROFILES | \
+ BTRFS_BALANCE_ARGS_USAGE | \
+ BTRFS_BALANCE_ARGS_DEVID | \
+ BTRFS_BALANCE_ARGS_DRANGE | \
+ BTRFS_BALANCE_ARGS_VRANGE | \
+ BTRFS_BALANCE_ARGS_LIMIT | \
+ BTRFS_BALANCE_ARGS_LIMIT_RANGE | \
+ BTRFS_BALANCE_ARGS_STRIPES_RANGE | \
+ BTRFS_BALANCE_ARGS_USAGE_RANGE)
+
+/*
+ * Profile changing flags. When SOFT is set we won't relocate chunk if
+ * it already has the target profile (even though it may be
+ * half-filled).
+ */
+#define BTRFS_BALANCE_ARGS_CONVERT (1ULL << 8)
+#define BTRFS_BALANCE_ARGS_SOFT (1ULL << 9)
+
+
+/*
+ * flags definition for balance state
+ *
+ * Used by:
+ * struct btrfs_ioctl_balance_args.state
+ */
#define BTRFS_BALANCE_STATE_RUNNING (1ULL << 0)
#define BTRFS_BALANCE_STATE_PAUSE_REQ (1ULL << 1)
#define BTRFS_BALANCE_STATE_CANCEL_REQ (1ULL << 2)
@@ -347,9 +485,45 @@
__u64 dest_offset;
};
-/* flags for the defrag range ioctl */
+/*
+ * flags definition for the defrag range ioctl
+ *
+ * Used by:
+ * struct btrfs_ioctl_defrag_range_args.flags
+ */
#define BTRFS_DEFRAG_RANGE_COMPRESS 1
#define BTRFS_DEFRAG_RANGE_START_IO 2
+struct btrfs_ioctl_defrag_range_args {
+ /* start of the defrag operation */
+ __u64 start;
+
+ /* number of bytes to defrag, use (u64)-1 to say all */
+ __u64 len;
+
+ /*
+ * flags for the operation, which can include turning
+ * on compression for this one defrag
+ */
+ __u64 flags;
+
+ /*
+ * any extent bigger than this will be considered
+ * already defragged. Use 0 to take the kernel default
+ * Use 1 to say every single extent must be rewritten
+ */
+ __u32 extent_thresh;
+
+ /*
+ * which compression method to use if turning on compression
+ * for this defrag operation. If unspecified, zlib will
+ * be used
+ */
+ __u32 compress_type;
+
+ /* spare for later */
+ __u32 unused[4];
+};
+
#define BTRFS_SAME_DATA_DIFFERS 1
/* For extent-same ioctl */
@@ -659,5 +833,7 @@
struct btrfs_ioctl_feature_flags[2])
#define BTRFS_IOC_GET_SUPPORTED_FEATURES _IOR(BTRFS_IOCTL_MAGIC, 57, \
struct btrfs_ioctl_feature_flags[3])
+#define BTRFS_IOC_RM_DEV_V2 _IOW(BTRFS_IOCTL_MAGIC, 58, \
+ struct btrfs_ioctl_vol_args_v2)
#endif /* _UAPI_LINUX_BTRFS_H */
diff --git a/include/uapi/linux/btrfs_tree.h b/include/uapi/linux/btrfs_tree.h
new file mode 100644
index 0000000..d5ad15a
--- /dev/null
+++ b/include/uapi/linux/btrfs_tree.h
@@ -0,0 +1,966 @@
+#ifndef _BTRFS_CTREE_H_
+#define _BTRFS_CTREE_H_
+
+/*
+ * This header contains the structure definitions and constants used
+ * by file system objects that can be retrieved using
+ * the BTRFS_IOC_SEARCH_TREE ioctl. That means basically anything that
+ * is needed to describe a leaf node's key or item contents.
+ */
+
+/* holds pointers to all of the tree roots */
+#define BTRFS_ROOT_TREE_OBJECTID 1ULL
+
+/* stores information about which extents are in use, and reference counts */
+#define BTRFS_EXTENT_TREE_OBJECTID 2ULL
+
+/*
+ * chunk tree stores translations from logical -> physical block numbering
+ * the super block points to the chunk tree
+ */
+#define BTRFS_CHUNK_TREE_OBJECTID 3ULL
+
+/*
+ * stores information about which areas of a given device are in use.
+ * one per device. The tree of tree roots points to the device tree
+ */
+#define BTRFS_DEV_TREE_OBJECTID 4ULL
+
+/* one per subvolume, storing files and directories */
+#define BTRFS_FS_TREE_OBJECTID 5ULL
+
+/* directory objectid inside the root tree */
+#define BTRFS_ROOT_TREE_DIR_OBJECTID 6ULL
+
+/* holds checksums of all the data extents */
+#define BTRFS_CSUM_TREE_OBJECTID 7ULL
+
+/* holds quota configuration and tracking */
+#define BTRFS_QUOTA_TREE_OBJECTID 8ULL
+
+/* for storing items that use the BTRFS_UUID_KEY* types */
+#define BTRFS_UUID_TREE_OBJECTID 9ULL
+
+/* tracks free space in block groups. */
+#define BTRFS_FREE_SPACE_TREE_OBJECTID 10ULL
+
+/* device stats in the device tree */
+#define BTRFS_DEV_STATS_OBJECTID 0ULL
+
+/* for storing balance parameters in the root tree */
+#define BTRFS_BALANCE_OBJECTID -4ULL
+
+/* orhpan objectid for tracking unlinked/truncated files */
+#define BTRFS_ORPHAN_OBJECTID -5ULL
+
+/* does write ahead logging to speed up fsyncs */
+#define BTRFS_TREE_LOG_OBJECTID -6ULL
+#define BTRFS_TREE_LOG_FIXUP_OBJECTID -7ULL
+
+/* for space balancing */
+#define BTRFS_TREE_RELOC_OBJECTID -8ULL
+#define BTRFS_DATA_RELOC_TREE_OBJECTID -9ULL
+
+/*
+ * extent checksums all have this objectid
+ * this allows them to share the logging tree
+ * for fsyncs
+ */
+#define BTRFS_EXTENT_CSUM_OBJECTID -10ULL
+
+/* For storing free space cache */
+#define BTRFS_FREE_SPACE_OBJECTID -11ULL
+
+/*
+ * The inode number assigned to the special inode for storing
+ * free ino cache
+ */
+#define BTRFS_FREE_INO_OBJECTID -12ULL
+
+/* dummy objectid represents multiple objectids */
+#define BTRFS_MULTIPLE_OBJECTIDS -255ULL
+
+/*
+ * All files have objectids in this range.
+ */
+#define BTRFS_FIRST_FREE_OBJECTID 256ULL
+#define BTRFS_LAST_FREE_OBJECTID -256ULL
+#define BTRFS_FIRST_CHUNK_TREE_OBJECTID 256ULL
+
+
+/*
+ * the device items go into the chunk tree. The key is in the form
+ * [ 1 BTRFS_DEV_ITEM_KEY device_id ]
+ */
+#define BTRFS_DEV_ITEMS_OBJECTID 1ULL
+
+#define BTRFS_BTREE_INODE_OBJECTID 1
+
+#define BTRFS_EMPTY_SUBVOL_DIR_OBJECTID 2
+
+#define BTRFS_DEV_REPLACE_DEVID 0ULL
+
+/*
+ * inode items have the data typically returned from stat and store other
+ * info about object characteristics. There is one for every file and dir in
+ * the FS
+ */
+#define BTRFS_INODE_ITEM_KEY 1
+#define BTRFS_INODE_REF_KEY 12
+#define BTRFS_INODE_EXTREF_KEY 13
+#define BTRFS_XATTR_ITEM_KEY 24
+#define BTRFS_ORPHAN_ITEM_KEY 48
+/* reserve 2-15 close to the inode for later flexibility */
+
+/*
+ * dir items are the name -> inode pointers in a directory. There is one
+ * for every name in a directory.
+ */
+#define BTRFS_DIR_LOG_ITEM_KEY 60
+#define BTRFS_DIR_LOG_INDEX_KEY 72
+#define BTRFS_DIR_ITEM_KEY 84
+#define BTRFS_DIR_INDEX_KEY 96
+/*
+ * extent data is for file data
+ */
+#define BTRFS_EXTENT_DATA_KEY 108
+
+/*
+ * extent csums are stored in a separate tree and hold csums for
+ * an entire extent on disk.
+ */
+#define BTRFS_EXTENT_CSUM_KEY 128
+
+/*
+ * root items point to tree roots. They are typically in the root
+ * tree used by the super block to find all the other trees
+ */
+#define BTRFS_ROOT_ITEM_KEY 132
+
+/*
+ * root backrefs tie subvols and snapshots to the directory entries that
+ * reference them
+ */
+#define BTRFS_ROOT_BACKREF_KEY 144
+
+/*
+ * root refs make a fast index for listing all of the snapshots and
+ * subvolumes referenced by a given root. They point directly to the
+ * directory item in the root that references the subvol
+ */
+#define BTRFS_ROOT_REF_KEY 156
+
+/*
+ * extent items are in the extent map tree. These record which blocks
+ * are used, and how many references there are to each block
+ */
+#define BTRFS_EXTENT_ITEM_KEY 168
+
+/*
+ * The same as the BTRFS_EXTENT_ITEM_KEY, except it's metadata we already know
+ * the length, so we save the level in key->offset instead of the length.
+ */
+#define BTRFS_METADATA_ITEM_KEY 169
+
+#define BTRFS_TREE_BLOCK_REF_KEY 176
+
+#define BTRFS_EXTENT_DATA_REF_KEY 178
+
+#define BTRFS_EXTENT_REF_V0_KEY 180
+
+#define BTRFS_SHARED_BLOCK_REF_KEY 182
+
+#define BTRFS_SHARED_DATA_REF_KEY 184
+
+/*
+ * block groups give us hints into the extent allocation trees. Which
+ * blocks are free etc etc
+ */
+#define BTRFS_BLOCK_GROUP_ITEM_KEY 192
+
+/*
+ * Every block group is represented in the free space tree by a free space info
+ * item, which stores some accounting information. It is keyed on
+ * (block_group_start, FREE_SPACE_INFO, block_group_length).
+ */
+#define BTRFS_FREE_SPACE_INFO_KEY 198
+
+/*
+ * A free space extent tracks an extent of space that is free in a block group.
+ * It is keyed on (start, FREE_SPACE_EXTENT, length).
+ */
+#define BTRFS_FREE_SPACE_EXTENT_KEY 199
+
+/*
+ * When a block group becomes very fragmented, we convert it to use bitmaps
+ * instead of extents. A free space bitmap is keyed on
+ * (start, FREE_SPACE_BITMAP, length); the corresponding item is a bitmap with
+ * (length / sectorsize) bits.
+ */
+#define BTRFS_FREE_SPACE_BITMAP_KEY 200
+
+#define BTRFS_DEV_EXTENT_KEY 204
+#define BTRFS_DEV_ITEM_KEY 216
+#define BTRFS_CHUNK_ITEM_KEY 228
+
+/*
+ * Records the overall state of the qgroups.
+ * There's only one instance of this key present,
+ * (0, BTRFS_QGROUP_STATUS_KEY, 0)
+ */
+#define BTRFS_QGROUP_STATUS_KEY 240
+/*
+ * Records the currently used space of the qgroup.
+ * One key per qgroup, (0, BTRFS_QGROUP_INFO_KEY, qgroupid).
+ */
+#define BTRFS_QGROUP_INFO_KEY 242
+/*
+ * Contains the user configured limits for the qgroup.
+ * One key per qgroup, (0, BTRFS_QGROUP_LIMIT_KEY, qgroupid).
+ */
+#define BTRFS_QGROUP_LIMIT_KEY 244
+/*
+ * Records the child-parent relationship of qgroups. For
+ * each relation, 2 keys are present:
+ * (childid, BTRFS_QGROUP_RELATION_KEY, parentid)
+ * (parentid, BTRFS_QGROUP_RELATION_KEY, childid)
+ */
+#define BTRFS_QGROUP_RELATION_KEY 246
+
+/*
+ * Obsolete name, see BTRFS_TEMPORARY_ITEM_KEY.
+ */
+#define BTRFS_BALANCE_ITEM_KEY 248
+
+/*
+ * The key type for tree items that are stored persistently, but do not need to
+ * exist for extended period of time. The items can exist in any tree.
+ *
+ * [subtype, BTRFS_TEMPORARY_ITEM_KEY, data]
+ *
+ * Existing items:
+ *
+ * - balance status item
+ * (BTRFS_BALANCE_OBJECTID, BTRFS_TEMPORARY_ITEM_KEY, 0)
+ */
+#define BTRFS_TEMPORARY_ITEM_KEY 248
+
+/*
+ * Obsolete name, see BTRFS_PERSISTENT_ITEM_KEY
+ */
+#define BTRFS_DEV_STATS_KEY 249
+
+/*
+ * The key type for tree items that are stored persistently and usually exist
+ * for a long period, eg. filesystem lifetime. The item kinds can be status
+ * information, stats or preference values. The item can exist in any tree.
+ *
+ * [subtype, BTRFS_PERSISTENT_ITEM_KEY, data]
+ *
+ * Existing items:
+ *
+ * - device statistics, store IO stats in the device tree, one key for all
+ * stats
+ * (BTRFS_DEV_STATS_OBJECTID, BTRFS_DEV_STATS_KEY, 0)
+ */
+#define BTRFS_PERSISTENT_ITEM_KEY 249
+
+/*
+ * Persistantly stores the device replace state in the device tree.
+ * The key is built like this: (0, BTRFS_DEV_REPLACE_KEY, 0).
+ */
+#define BTRFS_DEV_REPLACE_KEY 250
+
+/*
+ * Stores items that allow to quickly map UUIDs to something else.
+ * These items are part of the filesystem UUID tree.
+ * The key is built like this:
+ * (UUID_upper_64_bits, BTRFS_UUID_KEY*, UUID_lower_64_bits).
+ */
+#if BTRFS_UUID_SIZE != 16
+#error "UUID items require BTRFS_UUID_SIZE == 16!"
+#endif
+#define BTRFS_UUID_KEY_SUBVOL 251 /* for UUIDs assigned to subvols */
+#define BTRFS_UUID_KEY_RECEIVED_SUBVOL 252 /* for UUIDs assigned to
+ * received subvols */
+
+/*
+ * string items are for debugging. They just store a short string of
+ * data in the FS
+ */
+#define BTRFS_STRING_ITEM_KEY 253
+
+
+
+/* 32 bytes in various csum fields */
+#define BTRFS_CSUM_SIZE 32
+
+/* csum types */
+#define BTRFS_CSUM_TYPE_CRC32 0
+
+/*
+ * flags definitions for directory entry item type
+ *
+ * Used by:
+ * struct btrfs_dir_item.type
+ */
+#define BTRFS_FT_UNKNOWN 0
+#define BTRFS_FT_REG_FILE 1
+#define BTRFS_FT_DIR 2
+#define BTRFS_FT_CHRDEV 3
+#define BTRFS_FT_BLKDEV 4
+#define BTRFS_FT_FIFO 5
+#define BTRFS_FT_SOCK 6
+#define BTRFS_FT_SYMLINK 7
+#define BTRFS_FT_XATTR 8
+#define BTRFS_FT_MAX 9
+
+/*
+ * The key defines the order in the tree, and so it also defines (optimal)
+ * block layout.
+ *
+ * objectid corresponds to the inode number.
+ *
+ * type tells us things about the object, and is a kind of stream selector.
+ * so for a given inode, keys with type of 1 might refer to the inode data,
+ * type of 2 may point to file data in the btree and type == 3 may point to
+ * extents.
+ *
+ * offset is the starting byte offset for this key in the stream.
+ *
+ * btrfs_disk_key is in disk byte order. struct btrfs_key is always
+ * in cpu native order. Otherwise they are identical and their sizes
+ * should be the same (ie both packed)
+ */
+struct btrfs_disk_key {
+ __le64 objectid;
+ __u8 type;
+ __le64 offset;
+} __attribute__ ((__packed__));
+
+struct btrfs_key {
+ __u64 objectid;
+ __u8 type;
+ __u64 offset;
+} __attribute__ ((__packed__));
+
+struct btrfs_dev_item {
+ /* the internal btrfs device id */
+ __le64 devid;
+
+ /* size of the device */
+ __le64 total_bytes;
+
+ /* bytes used */
+ __le64 bytes_used;
+
+ /* optimal io alignment for this device */
+ __le32 io_align;
+
+ /* optimal io width for this device */
+ __le32 io_width;
+
+ /* minimal io size for this device */
+ __le32 sector_size;
+
+ /* type and info about this device */
+ __le64 type;
+
+ /* expected generation for this device */
+ __le64 generation;
+
+ /*
+ * starting byte of this partition on the device,
+ * to allow for stripe alignment in the future
+ */
+ __le64 start_offset;
+
+ /* grouping information for allocation decisions */
+ __le32 dev_group;
+
+ /* seek speed 0-100 where 100 is fastest */
+ __u8 seek_speed;
+
+ /* bandwidth 0-100 where 100 is fastest */
+ __u8 bandwidth;
+
+ /* btrfs generated uuid for this device */
+ __u8 uuid[BTRFS_UUID_SIZE];
+
+ /* uuid of FS who owns this device */
+ __u8 fsid[BTRFS_UUID_SIZE];
+} __attribute__ ((__packed__));
+
+struct btrfs_stripe {
+ __le64 devid;
+ __le64 offset;
+ __u8 dev_uuid[BTRFS_UUID_SIZE];
+} __attribute__ ((__packed__));
+
+struct btrfs_chunk {
+ /* size of this chunk in bytes */
+ __le64 length;
+
+ /* objectid of the root referencing this chunk */
+ __le64 owner;
+
+ __le64 stripe_len;
+ __le64 type;
+
+ /* optimal io alignment for this chunk */
+ __le32 io_align;
+
+ /* optimal io width for this chunk */
+ __le32 io_width;
+
+ /* minimal io size for this chunk */
+ __le32 sector_size;
+
+ /* 2^16 stripes is quite a lot, a second limit is the size of a single
+ * item in the btree
+ */
+ __le16 num_stripes;
+
+ /* sub stripes only matter for raid10 */
+ __le16 sub_stripes;
+ struct btrfs_stripe stripe;
+ /* additional stripes go here */
+} __attribute__ ((__packed__));
+
+#define BTRFS_FREE_SPACE_EXTENT 1
+#define BTRFS_FREE_SPACE_BITMAP 2
+
+struct btrfs_free_space_entry {
+ __le64 offset;
+ __le64 bytes;
+ __u8 type;
+} __attribute__ ((__packed__));
+
+struct btrfs_free_space_header {
+ struct btrfs_disk_key location;
+ __le64 generation;
+ __le64 num_entries;
+ __le64 num_bitmaps;
+} __attribute__ ((__packed__));
+
+#define BTRFS_HEADER_FLAG_WRITTEN (1ULL << 0)
+#define BTRFS_HEADER_FLAG_RELOC (1ULL << 1)
+
+/* Super block flags */
+/* Errors detected */
+#define BTRFS_SUPER_FLAG_ERROR (1ULL << 2)
+
+#define BTRFS_SUPER_FLAG_SEEDING (1ULL << 32)
+#define BTRFS_SUPER_FLAG_METADUMP (1ULL << 33)
+
+
+/*
+ * items in the extent btree are used to record the objectid of the
+ * owner of the block and the number of references
+ */
+
+struct btrfs_extent_item {
+ __le64 refs;
+ __le64 generation;
+ __le64 flags;
+} __attribute__ ((__packed__));
+
+struct btrfs_extent_item_v0 {
+ __le32 refs;
+} __attribute__ ((__packed__));
+
+
+#define BTRFS_EXTENT_FLAG_DATA (1ULL << 0)
+#define BTRFS_EXTENT_FLAG_TREE_BLOCK (1ULL << 1)
+
+/* following flags only apply to tree blocks */
+
+/* use full backrefs for extent pointers in the block */
+#define BTRFS_BLOCK_FLAG_FULL_BACKREF (1ULL << 8)
+
+/*
+ * this flag is only used internally by scrub and may be changed at any time
+ * it is only declared here to avoid collisions
+ */
+#define BTRFS_EXTENT_FLAG_SUPER (1ULL << 48)
+
+struct btrfs_tree_block_info {
+ struct btrfs_disk_key key;
+ __u8 level;
+} __attribute__ ((__packed__));
+
+struct btrfs_extent_data_ref {
+ __le64 root;
+ __le64 objectid;
+ __le64 offset;
+ __le32 count;
+} __attribute__ ((__packed__));
+
+struct btrfs_shared_data_ref {
+ __le32 count;
+} __attribute__ ((__packed__));
+
+struct btrfs_extent_inline_ref {
+ __u8 type;
+ __le64 offset;
+} __attribute__ ((__packed__));
+
+/* old style backrefs item */
+struct btrfs_extent_ref_v0 {
+ __le64 root;
+ __le64 generation;
+ __le64 objectid;
+ __le32 count;
+} __attribute__ ((__packed__));
+
+
+/* dev extents record free space on individual devices. The owner
+ * field points back to the chunk allocation mapping tree that allocated
+ * the extent. The chunk tree uuid field is a way to double check the owner
+ */
+struct btrfs_dev_extent {
+ __le64 chunk_tree;
+ __le64 chunk_objectid;
+ __le64 chunk_offset;
+ __le64 length;
+ __u8 chunk_tree_uuid[BTRFS_UUID_SIZE];
+} __attribute__ ((__packed__));
+
+struct btrfs_inode_ref {
+ __le64 index;
+ __le16 name_len;
+ /* name goes here */
+} __attribute__ ((__packed__));
+
+struct btrfs_inode_extref {
+ __le64 parent_objectid;
+ __le64 index;
+ __le16 name_len;
+ __u8 name[0];
+ /* name goes here */
+} __attribute__ ((__packed__));
+
+struct btrfs_timespec {
+ __le64 sec;
+ __le32 nsec;
+} __attribute__ ((__packed__));
+
+struct btrfs_inode_item {
+ /* nfs style generation number */
+ __le64 generation;
+ /* transid that last touched this inode */
+ __le64 transid;
+ __le64 size;
+ __le64 nbytes;
+ __le64 block_group;
+ __le32 nlink;
+ __le32 uid;
+ __le32 gid;
+ __le32 mode;
+ __le64 rdev;
+ __le64 flags;
+
+ /* modification sequence number for NFS */
+ __le64 sequence;
+
+ /*
+ * a little future expansion, for more than this we can
+ * just grow the inode item and version it
+ */
+ __le64 reserved[4];
+ struct btrfs_timespec atime;
+ struct btrfs_timespec ctime;
+ struct btrfs_timespec mtime;
+ struct btrfs_timespec otime;
+} __attribute__ ((__packed__));
+
+struct btrfs_dir_log_item {
+ __le64 end;
+} __attribute__ ((__packed__));
+
+struct btrfs_dir_item {
+ struct btrfs_disk_key location;
+ __le64 transid;
+ __le16 data_len;
+ __le16 name_len;
+ __u8 type;
+} __attribute__ ((__packed__));
+
+#define BTRFS_ROOT_SUBVOL_RDONLY (1ULL << 0)
+
+/*
+ * Internal in-memory flag that a subvolume has been marked for deletion but
+ * still visible as a directory
+ */
+#define BTRFS_ROOT_SUBVOL_DEAD (1ULL << 48)
+
+struct btrfs_root_item {
+ struct btrfs_inode_item inode;
+ __le64 generation;
+ __le64 root_dirid;
+ __le64 bytenr;
+ __le64 byte_limit;
+ __le64 bytes_used;
+ __le64 last_snapshot;
+ __le64 flags;
+ __le32 refs;
+ struct btrfs_disk_key drop_progress;
+ __u8 drop_level;
+ __u8 level;
+
+ /*
+ * The following fields appear after subvol_uuids+subvol_times
+ * were introduced.
+ */
+
+ /*
+ * This generation number is used to test if the new fields are valid
+ * and up to date while reading the root item. Every time the root item
+ * is written out, the "generation" field is copied into this field. If
+ * anyone ever mounted the fs with an older kernel, we will have
+ * mismatching generation values here and thus must invalidate the
+ * new fields. See btrfs_update_root and btrfs_find_last_root for
+ * details.
+ * the offset of generation_v2 is also used as the start for the memset
+ * when invalidating the fields.
+ */
+ __le64 generation_v2;
+ __u8 uuid[BTRFS_UUID_SIZE];
+ __u8 parent_uuid[BTRFS_UUID_SIZE];
+ __u8 received_uuid[BTRFS_UUID_SIZE];
+ __le64 ctransid; /* updated when an inode changes */
+ __le64 otransid; /* trans when created */
+ __le64 stransid; /* trans when sent. non-zero for received subvol */
+ __le64 rtransid; /* trans when received. non-zero for received subvol */
+ struct btrfs_timespec ctime;
+ struct btrfs_timespec otime;
+ struct btrfs_timespec stime;
+ struct btrfs_timespec rtime;
+ __le64 reserved[8]; /* for future */
+} __attribute__ ((__packed__));
+
+/*
+ * this is used for both forward and backward root refs
+ */
+struct btrfs_root_ref {
+ __le64 dirid;
+ __le64 sequence;
+ __le16 name_len;
+} __attribute__ ((__packed__));
+
+struct btrfs_disk_balance_args {
+ /*
+ * profiles to operate on, single is denoted by
+ * BTRFS_AVAIL_ALLOC_BIT_SINGLE
+ */
+ __le64 profiles;
+
+ /*
+ * usage filter
+ * BTRFS_BALANCE_ARGS_USAGE with a single value means '0..N'
+ * BTRFS_BALANCE_ARGS_USAGE_RANGE - range syntax, min..max
+ */
+ union {
+ __le64 usage;
+ struct {
+ __le32 usage_min;
+ __le32 usage_max;
+ };
+ };
+
+ /* devid filter */
+ __le64 devid;
+
+ /* devid subset filter [pstart..pend) */
+ __le64 pstart;
+ __le64 pend;
+
+ /* btrfs virtual address space subset filter [vstart..vend) */
+ __le64 vstart;
+ __le64 vend;
+
+ /*
+ * profile to convert to, single is denoted by
+ * BTRFS_AVAIL_ALLOC_BIT_SINGLE
+ */
+ __le64 target;
+
+ /* BTRFS_BALANCE_ARGS_* */
+ __le64 flags;
+
+ /*
+ * BTRFS_BALANCE_ARGS_LIMIT with value 'limit'
+ * BTRFS_BALANCE_ARGS_LIMIT_RANGE - the extend version can use minimum
+ * and maximum
+ */
+ union {
+ __le64 limit;
+ struct {
+ __le32 limit_min;
+ __le32 limit_max;
+ };
+ };
+
+ /*
+ * Process chunks that cross stripes_min..stripes_max devices,
+ * BTRFS_BALANCE_ARGS_STRIPES_RANGE
+ */
+ __le32 stripes_min;
+ __le32 stripes_max;
+
+ __le64 unused[6];
+} __attribute__ ((__packed__));
+
+/*
+ * store balance parameters to disk so that balance can be properly
+ * resumed after crash or unmount
+ */
+struct btrfs_balance_item {
+ /* BTRFS_BALANCE_* */
+ __le64 flags;
+
+ struct btrfs_disk_balance_args data;
+ struct btrfs_disk_balance_args meta;
+ struct btrfs_disk_balance_args sys;
+
+ __le64 unused[4];
+} __attribute__ ((__packed__));
+
+#define BTRFS_FILE_EXTENT_INLINE 0
+#define BTRFS_FILE_EXTENT_REG 1
+#define BTRFS_FILE_EXTENT_PREALLOC 2
+
+struct btrfs_file_extent_item {
+ /*
+ * transaction id that created this extent
+ */
+ __le64 generation;
+ /*
+ * max number of bytes to hold this extent in ram
+ * when we split a compressed extent we can't know how big
+ * each of the resulting pieces will be. So, this is
+ * an upper limit on the size of the extent in ram instead of
+ * an exact limit.
+ */
+ __le64 ram_bytes;
+
+ /*
+ * 32 bits for the various ways we might encode the data,
+ * including compression and encryption. If any of these
+ * are set to something a given disk format doesn't understand
+ * it is treated like an incompat flag for reading and writing,
+ * but not for stat.
+ */
+ __u8 compression;
+ __u8 encryption;
+ __le16 other_encoding; /* spare for later use */
+
+ /* are we inline data or a real extent? */
+ __u8 type;
+
+ /*
+ * disk space consumed by the extent, checksum blocks are included
+ * in these numbers
+ *
+ * At this offset in the structure, the inline extent data start.
+ */
+ __le64 disk_bytenr;
+ __le64 disk_num_bytes;
+ /*
+ * the logical offset in file blocks (no csums)
+ * this extent record is for. This allows a file extent to point
+ * into the middle of an existing extent on disk, sharing it
+ * between two snapshots (useful if some bytes in the middle of the
+ * extent have changed
+ */
+ __le64 offset;
+ /*
+ * the logical number of file blocks (no csums included). This
+ * always reflects the size uncompressed and without encoding.
+ */
+ __le64 num_bytes;
+
+} __attribute__ ((__packed__));
+
+struct btrfs_csum_item {
+ __u8 csum;
+} __attribute__ ((__packed__));
+
+struct btrfs_dev_stats_item {
+ /*
+ * grow this item struct at the end for future enhancements and keep
+ * the existing values unchanged
+ */
+ __le64 values[BTRFS_DEV_STAT_VALUES_MAX];
+} __attribute__ ((__packed__));
+
+#define BTRFS_DEV_REPLACE_ITEM_CONT_READING_FROM_SRCDEV_MODE_ALWAYS 0
+#define BTRFS_DEV_REPLACE_ITEM_CONT_READING_FROM_SRCDEV_MODE_AVOID 1
+#define BTRFS_DEV_REPLACE_ITEM_STATE_NEVER_STARTED 0
+#define BTRFS_DEV_REPLACE_ITEM_STATE_STARTED 1
+#define BTRFS_DEV_REPLACE_ITEM_STATE_SUSPENDED 2
+#define BTRFS_DEV_REPLACE_ITEM_STATE_FINISHED 3
+#define BTRFS_DEV_REPLACE_ITEM_STATE_CANCELED 4
+
+struct btrfs_dev_replace_item {
+ /*
+ * grow this item struct at the end for future enhancements and keep
+ * the existing values unchanged
+ */
+ __le64 src_devid;
+ __le64 cursor_left;
+ __le64 cursor_right;
+ __le64 cont_reading_from_srcdev_mode;
+
+ __le64 replace_state;
+ __le64 time_started;
+ __le64 time_stopped;
+ __le64 num_write_errors;
+ __le64 num_uncorrectable_read_errors;
+} __attribute__ ((__packed__));
+
+/* different types of block groups (and chunks) */
+#define BTRFS_BLOCK_GROUP_DATA (1ULL << 0)
+#define BTRFS_BLOCK_GROUP_SYSTEM (1ULL << 1)
+#define BTRFS_BLOCK_GROUP_METADATA (1ULL << 2)
+#define BTRFS_BLOCK_GROUP_RAID0 (1ULL << 3)
+#define BTRFS_BLOCK_GROUP_RAID1 (1ULL << 4)
+#define BTRFS_BLOCK_GROUP_DUP (1ULL << 5)
+#define BTRFS_BLOCK_GROUP_RAID10 (1ULL << 6)
+#define BTRFS_BLOCK_GROUP_RAID5 (1ULL << 7)
+#define BTRFS_BLOCK_GROUP_RAID6 (1ULL << 8)
+#define BTRFS_BLOCK_GROUP_RESERVED (BTRFS_AVAIL_ALLOC_BIT_SINGLE | \
+ BTRFS_SPACE_INFO_GLOBAL_RSV)
+
+enum btrfs_raid_types {
+ BTRFS_RAID_RAID10,
+ BTRFS_RAID_RAID1,
+ BTRFS_RAID_DUP,
+ BTRFS_RAID_RAID0,
+ BTRFS_RAID_SINGLE,
+ BTRFS_RAID_RAID5,
+ BTRFS_RAID_RAID6,
+ BTRFS_NR_RAID_TYPES
+};
+
+#define BTRFS_BLOCK_GROUP_TYPE_MASK (BTRFS_BLOCK_GROUP_DATA | \
+ BTRFS_BLOCK_GROUP_SYSTEM | \
+ BTRFS_BLOCK_GROUP_METADATA)
+
+#define BTRFS_BLOCK_GROUP_PROFILE_MASK (BTRFS_BLOCK_GROUP_RAID0 | \
+ BTRFS_BLOCK_GROUP_RAID1 | \
+ BTRFS_BLOCK_GROUP_RAID5 | \
+ BTRFS_BLOCK_GROUP_RAID6 | \
+ BTRFS_BLOCK_GROUP_DUP | \
+ BTRFS_BLOCK_GROUP_RAID10)
+#define BTRFS_BLOCK_GROUP_RAID56_MASK (BTRFS_BLOCK_GROUP_RAID5 | \
+ BTRFS_BLOCK_GROUP_RAID6)
+
+/*
+ * We need a bit for restriper to be able to tell when chunks of type
+ * SINGLE are available. This "extended" profile format is used in
+ * fs_info->avail_*_alloc_bits (in-memory) and balance item fields
+ * (on-disk). The corresponding on-disk bit in chunk.type is reserved
+ * to avoid remappings between two formats in future.
+ */
+#define BTRFS_AVAIL_ALLOC_BIT_SINGLE (1ULL << 48)
+
+/*
+ * A fake block group type that is used to communicate global block reserve
+ * size to userspace via the SPACE_INFO ioctl.
+ */
+#define BTRFS_SPACE_INFO_GLOBAL_RSV (1ULL << 49)
+
+#define BTRFS_EXTENDED_PROFILE_MASK (BTRFS_BLOCK_GROUP_PROFILE_MASK | \
+ BTRFS_AVAIL_ALLOC_BIT_SINGLE)
+
+static inline __u64 chunk_to_extended(__u64 flags)
+{
+ if ((flags & BTRFS_BLOCK_GROUP_PROFILE_MASK) == 0)
+ flags |= BTRFS_AVAIL_ALLOC_BIT_SINGLE;
+
+ return flags;
+}
+static inline __u64 extended_to_chunk(__u64 flags)
+{
+ return flags & ~BTRFS_AVAIL_ALLOC_BIT_SINGLE;
+}
+
+struct btrfs_block_group_item {
+ __le64 used;
+ __le64 chunk_objectid;
+ __le64 flags;
+} __attribute__ ((__packed__));
+
+struct btrfs_free_space_info {
+ __le32 extent_count;
+ __le32 flags;
+} __attribute__ ((__packed__));
+
+#define BTRFS_FREE_SPACE_USING_BITMAPS (1ULL << 0)
+
+#define BTRFS_QGROUP_LEVEL_SHIFT 48
+static inline __u64 btrfs_qgroup_level(__u64 qgroupid)
+{
+ return qgroupid >> BTRFS_QGROUP_LEVEL_SHIFT;
+}
+
+/*
+ * is subvolume quota turned on?
+ */
+#define BTRFS_QGROUP_STATUS_FLAG_ON (1ULL << 0)
+/*
+ * RESCAN is set during the initialization phase
+ */
+#define BTRFS_QGROUP_STATUS_FLAG_RESCAN (1ULL << 1)
+/*
+ * Some qgroup entries are known to be out of date,
+ * either because the configuration has changed in a way that
+ * makes a rescan necessary, or because the fs has been mounted
+ * with a non-qgroup-aware version.
+ * Turning qouta off and on again makes it inconsistent, too.
+ */
+#define BTRFS_QGROUP_STATUS_FLAG_INCONSISTENT (1ULL << 2)
+
+#define BTRFS_QGROUP_STATUS_VERSION 1
+
+struct btrfs_qgroup_status_item {
+ __le64 version;
+ /*
+ * the generation is updated during every commit. As older
+ * versions of btrfs are not aware of qgroups, it will be
+ * possible to detect inconsistencies by checking the
+ * generation on mount time
+ */
+ __le64 generation;
+
+ /* flag definitions see above */
+ __le64 flags;
+
+ /*
+ * only used during scanning to record the progress
+ * of the scan. It contains a logical address
+ */
+ __le64 rescan;
+} __attribute__ ((__packed__));
+
+struct btrfs_qgroup_info_item {
+ __le64 generation;
+ __le64 rfer;
+ __le64 rfer_cmpr;
+ __le64 excl;
+ __le64 excl_cmpr;
+} __attribute__ ((__packed__));
+
+struct btrfs_qgroup_limit_item {
+ /*
+ * only updated when any of the other values change
+ */
+ __le64 flags;
+ __le64 max_rfer;
+ __le64 max_excl;
+ __le64 rsv_rfer;
+ __le64 rsv_excl;
+} __attribute__ ((__packed__));
+
+#endif /* _BTRFS_CTREE_H_ */
diff --git a/include/uapi/linux/coresight-stm.h b/include/uapi/linux/coresight-stm.h
new file mode 100644
index 0000000..7e4272c
--- /dev/null
+++ b/include/uapi/linux/coresight-stm.h
@@ -0,0 +1,21 @@
+#ifndef __UAPI_CORESIGHT_STM_H_
+#define __UAPI_CORESIGHT_STM_H_
+
+#define STM_FLAG_TIMESTAMPED BIT(3)
+#define STM_FLAG_GUARANTEED BIT(7)
+
+/*
+ * The CoreSight STM supports guaranteed and invariant timing
+ * transactions. Guaranteed transactions are guaranteed to be
+ * traced, this might involve stalling the bus or system to
+ * ensure the transaction is accepted by the STM. While invariant
+ * timing transactions are not guaranteed to be traced, they
+ * will take an invariant amount of time regardless of the
+ * state of the STM.
+ */
+enum {
+ STM_OPTION_GUARANTEED = 0,
+ STM_OPTION_INVARIANT,
+};
+
+#endif
diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h
index e21fe04..3b00f7c 100644
--- a/include/uapi/linux/fs.h
+++ b/include/uapi/linux/fs.h
@@ -222,7 +222,6 @@
#define BLKSECDISCARD _IO(0x12,125)
#define BLKROTATIONAL _IO(0x12,126)
#define BLKZEROOUT _IO(0x12,127)
-#define BLKDAXGET _IO(0x12,129)
#define BMAP_IOCTL 1 /* obsolete - kept for compatibility */
#define FIBMAP _IO(0x00,1) /* bmap access */
diff --git a/include/uapi/linux/i2c.h b/include/uapi/linux/i2c.h
index b0a7dd6..adcbef4 100644
--- a/include/uapi/linux/i2c.h
+++ b/include/uapi/linux/i2c.h
@@ -68,14 +68,15 @@
struct i2c_msg {
__u16 addr; /* slave address */
__u16 flags;
-#define I2C_M_TEN 0x0010 /* this is a ten bit chip address */
#define I2C_M_RD 0x0001 /* read data, from slave to master */
-#define I2C_M_STOP 0x8000 /* if I2C_FUNC_PROTOCOL_MANGLING */
-#define I2C_M_NOSTART 0x4000 /* if I2C_FUNC_NOSTART */
-#define I2C_M_REV_DIR_ADDR 0x2000 /* if I2C_FUNC_PROTOCOL_MANGLING */
-#define I2C_M_IGNORE_NAK 0x1000 /* if I2C_FUNC_PROTOCOL_MANGLING */
-#define I2C_M_NO_RD_ACK 0x0800 /* if I2C_FUNC_PROTOCOL_MANGLING */
+ /* I2C_M_RD is guaranteed to be 0x0001! */
+#define I2C_M_TEN 0x0010 /* this is a ten bit chip address */
#define I2C_M_RECV_LEN 0x0400 /* length will be first received byte */
+#define I2C_M_NO_RD_ACK 0x0800 /* if I2C_FUNC_PROTOCOL_MANGLING */
+#define I2C_M_IGNORE_NAK 0x1000 /* if I2C_FUNC_PROTOCOL_MANGLING */
+#define I2C_M_REV_DIR_ADDR 0x2000 /* if I2C_FUNC_PROTOCOL_MANGLING */
+#define I2C_M_NOSTART 0x4000 /* if I2C_FUNC_NOSTART */
+#define I2C_M_STOP 0x8000 /* if I2C_FUNC_PROTOCOL_MANGLING */
__u16 len; /* msg length */
__u8 *buf; /* pointer to msg data */
};
diff --git a/include/uapi/linux/iio/types.h b/include/uapi/linux/iio/types.h
index c077617..b0916fc 100644
--- a/include/uapi/linux/iio/types.h
+++ b/include/uapi/linux/iio/types.h
@@ -38,6 +38,7 @@
IIO_CONCENTRATION,
IIO_RESISTANCE,
IIO_PH,
+ IIO_UVINDEX,
};
enum iio_modifier {
@@ -77,6 +78,7 @@
IIO_MOD_Q,
IIO_MOD_CO2,
IIO_MOD_VOC,
+ IIO_MOD_LIGHT_UV,
};
enum iio_event_type {
diff --git a/include/uapi/linux/keyctl.h b/include/uapi/linux/keyctl.h
index 840cb99..86eddd6 100644
--- a/include/uapi/linux/keyctl.h
+++ b/include/uapi/linux/keyctl.h
@@ -12,6 +12,8 @@
#ifndef _LINUX_KEYCTL_H
#define _LINUX_KEYCTL_H
+#include <linux/types.h>
+
/* special process keyring shortcut IDs */
#define KEY_SPEC_THREAD_KEYRING -1 /* - key ID for thread-specific keyring */
#define KEY_SPEC_PROCESS_KEYRING -2 /* - key ID for process-specific keyring */
@@ -57,5 +59,13 @@
#define KEYCTL_INSTANTIATE_IOV 20 /* instantiate a partially constructed key */
#define KEYCTL_INVALIDATE 21 /* invalidate a key */
#define KEYCTL_GET_PERSISTENT 22 /* get a user's persistent keyring */
+#define KEYCTL_DH_COMPUTE 23 /* Compute Diffie-Hellman values */
+
+/* keyctl structures */
+struct keyctl_dh_params {
+ __s32 private;
+ __s32 prime;
+ __s32 base;
+};
#endif /* _LINUX_KEYCTL_H */
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index a7f1f80..05ebf47 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -865,6 +865,7 @@
#define KVM_CAP_SPAPR_TCE_64 125
#define KVM_CAP_ARM_PMU_V3 126
#define KVM_CAP_VCPU_ATTRIBUTES 127
+#define KVM_CAP_MAX_VCPU_ID 128
#ifdef KVM_CAP_IRQ_ROUTING
diff --git a/include/uapi/linux/libc-compat.h b/include/uapi/linux/libc-compat.h
index d5e38c7..e4f048e 100644
--- a/include/uapi/linux/libc-compat.h
+++ b/include/uapi/linux/libc-compat.h
@@ -52,7 +52,7 @@
#if defined(__GLIBC__)
/* Coordinate with glibc net/if.h header. */
-#if defined(_NET_IF_H)
+#if defined(_NET_IF_H) && defined(__USE_MISC)
/* GLIBC headers included first so don't define anything
* that would already be defined. */
diff --git a/include/uapi/linux/ndctl.h b/include/uapi/linux/ndctl.h
index 7cc28ab..309915f 100644
--- a/include/uapi/linux/ndctl.h
+++ b/include/uapi/linux/ndctl.h
@@ -1,5 +1,5 @@
/*
- * Copyright (c) 2014-2015, Intel Corporation.
+ * Copyright (c) 2014-2016, Intel Corporation.
*
* This program is free software; you can redistribute it and/or modify it
* under the terms and conditions of the GNU Lesser General Public License,
@@ -20,11 +20,45 @@
__u8 data[128];
} __packed;
+#define ND_SMART_HEALTH_VALID (1 << 0)
+#define ND_SMART_TEMP_VALID (1 << 1)
+#define ND_SMART_SPARES_VALID (1 << 2)
+#define ND_SMART_ALARM_VALID (1 << 3)
+#define ND_SMART_USED_VALID (1 << 4)
+#define ND_SMART_SHUTDOWN_VALID (1 << 5)
+#define ND_SMART_VENDOR_VALID (1 << 6)
+#define ND_SMART_TEMP_TRIP (1 << 0)
+#define ND_SMART_SPARE_TRIP (1 << 1)
+#define ND_SMART_NON_CRITICAL_HEALTH (1 << 0)
+#define ND_SMART_CRITICAL_HEALTH (1 << 1)
+#define ND_SMART_FATAL_HEALTH (1 << 2)
+
+struct nd_smart_payload {
+ __u32 flags;
+ __u8 reserved0[4];
+ __u8 health;
+ __u16 temperature;
+ __u8 spares;
+ __u8 alarm_flags;
+ __u8 life_used;
+ __u8 shutdown_state;
+ __u8 reserved1;
+ __u32 vendor_size;
+ __u8 vendor_data[108];
+} __packed;
+
struct nd_cmd_smart_threshold {
__u32 status;
__u8 data[8];
} __packed;
+struct nd_smart_threshold_payload {
+ __u16 alarm_control;
+ __u16 temperature;
+ __u8 spares;
+ __u8 reserved[3];
+} __packed;
+
struct nd_cmd_dimm_flags {
__u32 status;
__u32 flags;
@@ -125,6 +159,7 @@
ND_CMD_VENDOR_EFFECT_LOG_SIZE = 7,
ND_CMD_VENDOR_EFFECT_LOG = 8,
ND_CMD_VENDOR = 9,
+ ND_CMD_CALL = 10,
};
enum {
@@ -158,6 +193,7 @@
[ND_CMD_VENDOR_EFFECT_LOG_SIZE] = "effect_size",
[ND_CMD_VENDOR_EFFECT_LOG] = "effect_log",
[ND_CMD_VENDOR] = "vendor",
+ [ND_CMD_CALL] = "cmd_call",
};
if (cmd < ARRAY_SIZE(names) && names[cmd])
@@ -206,6 +242,7 @@
#define ND_DEVICE_NAMESPACE_IO 4 /* legacy persistent memory */
#define ND_DEVICE_NAMESPACE_PMEM 5 /* PMEM namespace (may alias with BLK) */
#define ND_DEVICE_NAMESPACE_BLK 6 /* BLK namespace (may alias with PMEM) */
+#define ND_DEVICE_DAX_PMEM 7 /* Device DAX interface to pmem */
enum nd_driver_flags {
ND_DRIVER_DIMM = 1 << ND_DEVICE_DIMM,
@@ -214,6 +251,7 @@
ND_DRIVER_NAMESPACE_IO = 1 << ND_DEVICE_NAMESPACE_IO,
ND_DRIVER_NAMESPACE_PMEM = 1 << ND_DEVICE_NAMESPACE_PMEM,
ND_DRIVER_NAMESPACE_BLK = 1 << ND_DEVICE_NAMESPACE_BLK,
+ ND_DRIVER_DAX_PMEM = 1 << ND_DEVICE_DAX_PMEM,
};
enum {
@@ -224,4 +262,44 @@
ARS_STATUS_MASK = 0x0000FFFF,
ARS_EXT_STATUS_SHIFT = 16,
};
+
+/*
+ * struct nd_cmd_pkg
+ *
+ * is a wrapper to a quasi pass thru interface for invoking firmware
+ * associated with nvdimms.
+ *
+ * INPUT PARAMETERS
+ *
+ * nd_family corresponds to the firmware (e.g. DSM) interface.
+ *
+ * nd_command are the function index advertised by the firmware.
+ *
+ * nd_size_in is the size of the input parameters being passed to firmware
+ *
+ * OUTPUT PARAMETERS
+ *
+ * nd_fw_size is the size of the data firmware wants to return for
+ * the call. If nd_fw_size is greater than size of nd_size_out, only
+ * the first nd_size_out bytes are returned.
+ */
+
+struct nd_cmd_pkg {
+ __u64 nd_family; /* family of commands */
+ __u64 nd_command;
+ __u32 nd_size_in; /* INPUT: size of input args */
+ __u32 nd_size_out; /* INPUT: size of payload */
+ __u32 nd_reserved2[9]; /* reserved must be zero */
+ __u32 nd_fw_size; /* OUTPUT: size fw wants to return */
+ unsigned char nd_payload[]; /* Contents of call */
+};
+
+/* These NVDIMM families represent pre-standardization command sets */
+#define NVDIMM_FAMILY_INTEL 0
+#define NVDIMM_FAMILY_HPE1 1
+#define NVDIMM_FAMILY_HPE2 2
+
+#define ND_IOCTL_CALL _IOWR(ND_IOCTL, ND_CMD_CALL,\
+ struct nd_cmd_pkg)
+
#endif /* __NDCTL_H__ */
diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
index 1becea8..4040951 100644
--- a/include/uapi/linux/pci_regs.h
+++ b/include/uapi/linux/pci_regs.h
@@ -670,7 +670,8 @@
#define PCI_EXT_CAP_ID_SECPCI 0x19 /* Secondary PCIe Capability */
#define PCI_EXT_CAP_ID_PMUX 0x1A /* Protocol Multiplexing */
#define PCI_EXT_CAP_ID_PASID 0x1B /* Process Address Space ID */
-#define PCI_EXT_CAP_ID_MAX PCI_EXT_CAP_ID_PASID
+#define PCI_EXT_CAP_ID_DPC 0x1D /* Downstream Port Containment */
+#define PCI_EXT_CAP_ID_MAX PCI_EXT_CAP_ID_DPC
#define PCI_EXT_CAP_DSN_SIZEOF 12
#define PCI_EXT_CAP_MCAST_ENDPOINT_SIZEOF 40
@@ -946,4 +947,21 @@
#define PCI_TPH_CAP_ST_SHIFT 16 /* st table shift */
#define PCI_TPH_BASE_SIZEOF 12 /* size with no st table */
+/* Downstream Port Containment */
+#define PCI_EXP_DPC_CAP 4 /* DPC Capability */
+#define PCI_EXP_DPC_CAP_RP_EXT 0x20 /* Root Port Extensions for DPC */
+#define PCI_EXP_DPC_CAP_POISONED_TLP 0x40 /* Poisoned TLP Egress Blocking Supported */
+#define PCI_EXP_DPC_CAP_SW_TRIGGER 0x80 /* Software Triggering Supported */
+#define PCI_EXP_DPC_CAP_DL_ACTIVE 0x1000 /* ERR_COR signal on DL_Active supported */
+
+#define PCI_EXP_DPC_CTL 6 /* DPC control */
+#define PCI_EXP_DPC_CTL_EN_NONFATAL 0x02 /* Enable trigger on ERR_NONFATAL message */
+#define PCI_EXP_DPC_CTL_INT_EN 0x08 /* DPC Interrupt Enable */
+
+#define PCI_EXP_DPC_STATUS 8 /* DPC Status */
+#define PCI_EXP_DPC_STATUS_TRIGGER 0x01 /* Trigger Status */
+#define PCI_EXP_DPC_STATUS_INTERRUPT 0x08 /* Interrupt Status */
+
+#define PCI_EXP_DPC_SOURCE_ID 10 /* DPC Source Identifier */
+
#endif /* LINUX_PCI_REGS_H */
diff --git a/include/uapi/linux/serial_core.h b/include/uapi/linux/serial_core.h
index e513a4e..99dbed8 100644
--- a/include/uapi/linux/serial_core.h
+++ b/include/uapi/linux/serial_core.h
@@ -264,4 +264,10 @@
/* MVEBU UART */
#define PORT_MVEBU 114
+/* Microchip PIC32 UART */
+#define PORT_PIC32 115
+
+/* MPS2 UART */
+#define PORT_MPS2UART 116
+
#endif /* _UAPILINUX_SERIAL_CORE_H */
diff --git a/include/uapi/linux/sync_file.h b/include/uapi/linux/sync_file.h
new file mode 100644
index 0000000..413303d
--- /dev/null
+++ b/include/uapi/linux/sync_file.h
@@ -0,0 +1,100 @@
+/*
+ * Copyright (C) 2012 Google, Inc.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#ifndef _UAPI_LINUX_SYNC_H
+#define _UAPI_LINUX_SYNC_H
+
+#include <linux/ioctl.h>
+#include <linux/types.h>
+
+/**
+ * struct sync_merge_data - data passed to merge ioctl
+ * @name: name of new fence
+ * @fd2: file descriptor of second fence
+ * @fence: returns the fd of the new fence to userspace
+ * @flags: merge_data flags
+ * @pad: padding for 64-bit alignment, should always be zero
+ */
+struct sync_merge_data {
+ char name[32];
+ __s32 fd2;
+ __s32 fence;
+ __u32 flags;
+ __u32 pad;
+};
+
+/**
+ * struct sync_fence_info - detailed fence information
+ * @obj_name: name of parent sync_timeline
+* @driver_name: name of driver implementing the parent
+* @status: status of the fence 0:active 1:signaled <0:error
+ * @flags: fence_info flags
+ * @timestamp_ns: timestamp of status change in nanoseconds
+ */
+struct sync_fence_info {
+ char obj_name[32];
+ char driver_name[32];
+ __s32 status;
+ __u32 flags;
+ __u64 timestamp_ns;
+};
+
+/**
+ * struct sync_file_info - data returned from fence info ioctl
+ * @name: name of fence
+ * @status: status of fence. 1: signaled 0:active <0:error
+ * @flags: sync_file_info flags
+ * @num_fences number of fences in the sync_file
+ * @pad: padding for 64-bit alignment, should always be zero
+ * @sync_fence_info: pointer to array of structs sync_fence_info with all
+ * fences in the sync_file
+ */
+struct sync_file_info {
+ char name[32];
+ __s32 status;
+ __u32 flags;
+ __u32 num_fences;
+ __u32 pad;
+
+ __u64 sync_fence_info;
+};
+
+#define SYNC_IOC_MAGIC '>'
+
+/**
+ * Opcodes 0, 1 and 2 were burned during a API change to avoid users of the
+ * old API to get weird errors when trying to handling sync_files. The API
+ * change happened during the de-stage of the Sync Framework when there was
+ * no upstream users available.
+ */
+
+/**
+ * DOC: SYNC_IOC_MERGE - merge two fences
+ *
+ * Takes a struct sync_merge_data. Creates a new fence containing copies of
+ * the sync_pts in both the calling fd and sync_merge_data.fd2. Returns the
+ * new fence's fd in sync_merge_data.fence
+ */
+#define SYNC_IOC_MERGE _IOWR(SYNC_IOC_MAGIC, 3, struct sync_merge_data)
+
+/**
+ * DOC: SYNC_IOC_FENCE_INFO - get detailed information on a fence
+ *
+ * Takes a struct sync_file_info_data with extra space allocated for pt_info.
+ * Caller should write the size of the buffer into len. On return, len is
+ * updated to reflect the total size of the sync_file_info_data including
+ * pt_info.
+ *
+ * pt_info is a buffer containing sync_pt_infos for every sync_pt in the fence.
+ * To iterate over the sync_pt_infos, use the sync_pt_info.len field.
+ */
+#define SYNC_IOC_FILE_INFO _IOWR(SYNC_IOC_MAGIC, 4, struct sync_file_info)
+
+#endif /* _UAPI_LINUX_SYNC_H */
diff --git a/include/uapi/linux/tty_flags.h b/include/uapi/linux/tty_flags.h
index 072e41e..66e4d8b 100644
--- a/include/uapi/linux/tty_flags.h
+++ b/include/uapi/linux/tty_flags.h
@@ -32,7 +32,13 @@
#define ASYNCB_MAGIC_MULTIPLIER 16 /* Use special CLK or divisor */
#define ASYNCB_LAST_USER 16
-/* Internal flags used only by kernel */
+/*
+ * Internal flags used only by kernel (read-only)
+ *
+ * WARNING: These flags are no longer used and have been superceded by the
+ * TTY_PORT_ flags in the iflags field (and not userspace-visible)
+ */
+#ifndef _KERNEL_
#define ASYNCB_INITIALIZED 31 /* Serial port was initialized */
#define ASYNCB_SUSPENDED 30 /* Serial port is suspended */
#define ASYNCB_NORMAL_ACTIVE 29 /* Normal device is active */
@@ -43,7 +49,9 @@
#define ASYNCB_SHARE_IRQ 24 /* for multifunction cards, no longer used */
#define ASYNCB_CONS_FLOW 23 /* flow control for console */
#define ASYNCB_FIRST_KERNEL 22
+#endif
+/* Masks */
#define ASYNC_HUP_NOTIFY (1U << ASYNCB_HUP_NOTIFY)
#define ASYNC_SUSPENDED (1U << ASYNCB_SUSPENDED)
#define ASYNC_FOURPORT (1U << ASYNCB_FOURPORT)
@@ -72,6 +80,8 @@
#define ASYNC_SPD_WARP (ASYNC_SPD_HI|ASYNC_SPD_SHI)
#define ASYNC_SPD_MASK (ASYNC_SPD_HI|ASYNC_SPD_VHI|ASYNC_SPD_SHI)
+#ifndef _KERNEL_
+/* These flags are no longer used (and were always masked from userspace) */
#define ASYNC_INITIALIZED (1U << ASYNCB_INITIALIZED)
#define ASYNC_NORMAL_ACTIVE (1U << ASYNCB_NORMAL_ACTIVE)
#define ASYNC_BOOT_AUTOCONF (1U << ASYNCB_BOOT_AUTOCONF)
@@ -81,5 +91,6 @@
#define ASYNC_SHARE_IRQ (1U << ASYNCB_SHARE_IRQ)
#define ASYNC_CONS_FLOW (1U << ASYNCB_CONS_FLOW)
#define ASYNC_INTERNAL_FLAGS (~((1U << ASYNCB_FIRST_KERNEL) - 1))
+#endif
#endif
diff --git a/include/uapi/linux/usb/ch9.h b/include/uapi/linux/usb/ch9.h
index d5ce716..a8acc24 100644
--- a/include/uapi/linux/usb/ch9.h
+++ b/include/uapi/linux/usb/ch9.h
@@ -105,6 +105,13 @@
#define USB_REQ_LOOPBACK_DATA_READ 0x16
#define USB_REQ_SET_INTERFACE_DS 0x17
+/* specific requests for USB Power Delivery */
+#define USB_REQ_GET_PARTNER_PDO 20
+#define USB_REQ_GET_BATTERY_STATUS 21
+#define USB_REQ_SET_PDO 22
+#define USB_REQ_GET_VDM 23
+#define USB_REQ_SEND_VDM 24
+
/* The Link Power Management (LPM) ECN defines USB_REQ_TEST_AND_SET command,
* used by hubs to put ports into a new L1 suspend state, except that it
* forgot to define its number ...
@@ -165,6 +172,22 @@
#define USB_DEV_STAT_U2_ENABLED 3 /* transition into U2 state */
#define USB_DEV_STAT_LTM_ENABLED 4 /* Latency tolerance messages */
+/*
+ * Feature selectors from Table 9-8 USB Power Delivery spec
+ */
+#define USB_DEVICE_BATTERY_WAKE_MASK 40
+#define USB_DEVICE_OS_IS_PD_AWARE 41
+#define USB_DEVICE_POLICY_MODE 42
+#define USB_PORT_PR_SWAP 43
+#define USB_PORT_GOTO_MIN 44
+#define USB_PORT_RETURN_POWER 45
+#define USB_PORT_ACCEPT_PD_REQUEST 46
+#define USB_PORT_REJECT_PD_REQUEST 47
+#define USB_PORT_PORT_PD_RESET 48
+#define USB_PORT_C_PORT_PD_CHANGE 49
+#define USB_PORT_CABLE_PD_RESET 50
+#define USB_DEVICE_CHARGING_POLICY 54
+
/**
* struct usb_ctrlrequest - SETUP data for a USB device control request
* @bRequestType: matches the USB bmRequestType field
@@ -914,6 +937,104 @@
} __attribute__((packed));
/*
+ * USB Power Delivery Capability Descriptor:
+ * Defines capabilities for PD
+ */
+/* Defines the various PD Capabilities of this device */
+#define USB_PD_POWER_DELIVERY_CAPABILITY 0x06
+/* Provides information on each battery supported by the device */
+#define USB_PD_BATTERY_INFO_CAPABILITY 0x07
+/* The Consumer characteristics of a Port on the device */
+#define USB_PD_PD_CONSUMER_PORT_CAPABILITY 0x08
+/* The provider characteristics of a Port on the device */
+#define USB_PD_PD_PROVIDER_PORT_CAPABILITY 0x09
+
+struct usb_pd_cap_descriptor {
+ __u8 bLength;
+ __u8 bDescriptorType;
+ __u8 bDevCapabilityType; /* set to USB_PD_POWER_DELIVERY_CAPABILITY */
+ __u8 bReserved;
+ __le32 bmAttributes;
+#define USB_PD_CAP_BATTERY_CHARGING (1 << 1) /* supports Battery Charging specification */
+#define USB_PD_CAP_USB_PD (1 << 2) /* supports USB Power Delivery specification */
+#define USB_PD_CAP_PROVIDER (1 << 3) /* can provide power */
+#define USB_PD_CAP_CONSUMER (1 << 4) /* can consume power */
+#define USB_PD_CAP_CHARGING_POLICY (1 << 5) /* supports CHARGING_POLICY feature */
+#define USB_PD_CAP_TYPE_C_CURRENT (1 << 6) /* supports power capabilities defined in the USB Type-C Specification */
+
+#define USB_PD_CAP_PWR_AC (1 << 8)
+#define USB_PD_CAP_PWR_BAT (1 << 9)
+#define USB_PD_CAP_PWR_USE_V_BUS (1 << 14)
+
+ __le16 bmProviderPorts; /* Bit zero refers to the UFP of the device */
+ __le16 bmConsumerPorts;
+ __le16 bcdBCVersion;
+ __le16 bcdPDVersion;
+ __le16 bcdUSBTypeCVersion;
+} __attribute__((packed));
+
+struct usb_pd_cap_battery_info_descriptor {
+ __u8 bLength;
+ __u8 bDescriptorType;
+ __u8 bDevCapabilityType;
+ /* Index of string descriptor shall contain the user friendly name for this battery */
+ __u8 iBattery;
+ /* Index of string descriptor shall contain the Serial Number String for this battery */
+ __u8 iSerial;
+ __u8 iManufacturer;
+ __u8 bBatteryId; /* uniquely identifies this battery in status Messages */
+ __u8 bReserved;
+ /*
+ * Shall contain the Battery Charge value above which this
+ * battery is considered to be fully charged but not necessarily
+ * “topped off.”
+ */
+ __le32 dwChargedThreshold; /* in mWh */
+ /*
+ * Shall contain the minimum charge level of this battery such
+ * that above this threshold, a device can be assured of being
+ * able to power up successfully (see Battery Charging 1.2).
+ */
+ __le32 dwWeakThreshold; /* in mWh */
+ __le32 dwBatteryDesignCapacity; /* in mWh */
+ __le32 dwBatteryLastFullchargeCapacity; /* in mWh */
+} __attribute__((packed));
+
+struct usb_pd_cap_consumer_port_descriptor {
+ __u8 bLength;
+ __u8 bDescriptorType;
+ __u8 bDevCapabilityType;
+ __u8 bReserved;
+ __u8 bmCapabilities;
+/* port will oerate under: */
+#define USB_PD_CAP_CONSUMER_BC (1 << 0) /* BC */
+#define USB_PD_CAP_CONSUMER_PD (1 << 1) /* PD */
+#define USB_PD_CAP_CONSUMER_TYPE_C (1 << 2) /* USB Type-C Current */
+ __le16 wMinVoltage; /* in 50mV units */
+ __le16 wMaxVoltage; /* in 50mV units */
+ __u16 wReserved;
+ __le32 dwMaxOperatingPower; /* in 10 mW - operating at steady state */
+ __le32 dwMaxPeakPower; /* in 10mW units - operating at peak power */
+ __le32 dwMaxPeakPowerTime; /* in 100ms units - duration of peak */
+#define USB_PD_CAP_CONSUMER_UNKNOWN_PEAK_POWER_TIME 0xffff
+} __attribute__((packed));
+
+struct usb_pd_cap_provider_port_descriptor {
+ __u8 bLength;
+ __u8 bDescriptorType;
+ __u8 bDevCapabilityType;
+ __u8 bReserved1;
+ __u8 bmCapabilities;
+/* port will oerate under: */
+#define USB_PD_CAP_PROVIDER_BC (1 << 0) /* BC */
+#define USB_PD_CAP_PROVIDER_PD (1 << 1) /* PD */
+#define USB_PD_CAP_PROVIDER_TYPE_C (1 << 2) /* USB Type-C Current */
+ __u8 bNumOfPDObjects;
+ __u8 bReserved2;
+ __le32 wPowerDataObject[];
+} __attribute__((packed));
+
+/*
* Precision time measurement capability descriptor: advertised by devices and
* hubs that support PTM
*/
diff --git a/include/uapi/linux/uuid.h b/include/uapi/linux/uuid.h
index 786f077..3738e5f 100644
--- a/include/uapi/linux/uuid.h
+++ b/include/uapi/linux/uuid.h
@@ -12,10 +12,6 @@
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#ifndef _UAPI_LINUX_UUID_H_
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index e895975..8f95191 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -138,10 +138,7 @@
V4L2_BUF_TYPE_VBI_OUTPUT = 5,
V4L2_BUF_TYPE_SLICED_VBI_CAPTURE = 6,
V4L2_BUF_TYPE_SLICED_VBI_OUTPUT = 7,
-#if 1
- /* Experimental */
V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY = 8,
-#endif
V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE = 9,
V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE = 10,
V4L2_BUF_TYPE_SDR_CAPTURE = 11,
@@ -657,8 +654,7 @@
#define V4L2_FMT_FLAG_COMPRESSED 0x0001
#define V4L2_FMT_FLAG_EMULATED 0x0002
-#if 1
- /* Experimental Frame Size and frame rate enumeration */
+ /* Frame Size and frame rate enumeration */
/*
* F R A M E S I Z E E N U M E R A T I O N
*/
@@ -724,7 +720,6 @@
__u32 reserved[2]; /* Reserved space for future use */
};
-#endif
/*
* T I M E C O D E
@@ -1728,8 +1723,6 @@
/*
* M P E G S E R V I C E S
- *
- * NOTE: EXPERIMENTAL API
*/
#if 1
#define V4L2_ENC_IDX_FRAME_I (0)
@@ -2259,46 +2252,35 @@
#define VIDIOC_ENCODER_CMD _IOWR('V', 77, struct v4l2_encoder_cmd)
#define VIDIOC_TRY_ENCODER_CMD _IOWR('V', 78, struct v4l2_encoder_cmd)
-/* Experimental, meant for debugging, testing and internal use.
- Only implemented if CONFIG_VIDEO_ADV_DEBUG is defined.
- You must be root to use these ioctls. Never use these in applications! */
+/*
+ * Experimental, meant for debugging, testing and internal use.
+ * Only implemented if CONFIG_VIDEO_ADV_DEBUG is defined.
+ * You must be root to use these ioctls. Never use these in applications!
+ */
#define VIDIOC_DBG_S_REGISTER _IOW('V', 79, struct v4l2_dbg_register)
#define VIDIOC_DBG_G_REGISTER _IOWR('V', 80, struct v4l2_dbg_register)
#define VIDIOC_S_HW_FREQ_SEEK _IOW('V', 82, struct v4l2_hw_freq_seek)
-
#define VIDIOC_S_DV_TIMINGS _IOWR('V', 87, struct v4l2_dv_timings)
#define VIDIOC_G_DV_TIMINGS _IOWR('V', 88, struct v4l2_dv_timings)
#define VIDIOC_DQEVENT _IOR('V', 89, struct v4l2_event)
#define VIDIOC_SUBSCRIBE_EVENT _IOW('V', 90, struct v4l2_event_subscription)
#define VIDIOC_UNSUBSCRIBE_EVENT _IOW('V', 91, struct v4l2_event_subscription)
-
-/* Experimental, the below two ioctls may change over the next couple of kernel
- versions */
#define VIDIOC_CREATE_BUFS _IOWR('V', 92, struct v4l2_create_buffers)
#define VIDIOC_PREPARE_BUF _IOWR('V', 93, struct v4l2_buffer)
-
-/* Experimental selection API */
#define VIDIOC_G_SELECTION _IOWR('V', 94, struct v4l2_selection)
#define VIDIOC_S_SELECTION _IOWR('V', 95, struct v4l2_selection)
-
-/* Experimental, these two ioctls may change over the next couple of kernel
- versions. */
#define VIDIOC_DECODER_CMD _IOWR('V', 96, struct v4l2_decoder_cmd)
#define VIDIOC_TRY_DECODER_CMD _IOWR('V', 97, struct v4l2_decoder_cmd)
-
-/* Experimental, these three ioctls may change over the next couple of kernel
- versions. */
#define VIDIOC_ENUM_DV_TIMINGS _IOWR('V', 98, struct v4l2_enum_dv_timings)
#define VIDIOC_QUERY_DV_TIMINGS _IOR('V', 99, struct v4l2_dv_timings)
#define VIDIOC_DV_TIMINGS_CAP _IOWR('V', 100, struct v4l2_dv_timings_cap)
-
-/* Experimental, this ioctl may change over the next couple of kernel
- versions. */
#define VIDIOC_ENUM_FREQ_BANDS _IOWR('V', 101, struct v4l2_frequency_band)
-/* Experimental, meant for debugging, testing and internal use.
- Never use these in applications! */
+/*
+ * Experimental, meant for debugging, testing and internal use.
+ * Never use this in applications!
+ */
#define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct v4l2_dbg_chip_info)
#define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl)
diff --git a/include/uapi/linux/vt.h b/include/uapi/linux/vt.h
index 978578b..f690348 100644
--- a/include/uapi/linux/vt.h
+++ b/include/uapi/linux/vt.h
@@ -8,7 +8,6 @@
*/
#define MIN_NR_CONSOLES 1 /* must be at least 1 */
#define MAX_NR_CONSOLES 63 /* serial lines start at 64 */
-#define MAX_NR_USER_CONSOLES 63 /* must be root to allocate above this */
/* Note: the ioctl VT_GETSTATE does not work for
consoles 16 and higher (since it returns a short) */
diff --git a/include/uapi/mtd/mtd-abi.h b/include/uapi/mtd/mtd-abi.h
index 763bb69..0ec1da2 100644
--- a/include/uapi/mtd/mtd-abi.h
+++ b/include/uapi/mtd/mtd-abi.h
@@ -228,7 +228,7 @@
* complete set of ECC information. The ioctl truncates the larger internal
* structure to retain binary compatibility with the static declaration of the
* ioctl. Note that the "MTD_MAX_..._ENTRIES" macros represent the max size of
- * the user struct, not the MAX size of the internal struct nand_ecclayout.
+ * the user struct, not the MAX size of the internal OOB layout representation.
*/
struct nand_ecclayout_user {
__u32 eccbytes;
diff --git a/include/uapi/rdma/ib_user_verbs.h b/include/uapi/rdma/ib_user_verbs.h
index 8126c14..b6543d7 100644
--- a/include/uapi/rdma/ib_user_verbs.h
+++ b/include/uapi/rdma/ib_user_verbs.h
@@ -226,6 +226,7 @@
struct ib_uverbs_odp_caps odp_caps;
__u64 timestamp_mask;
__u64 hca_core_clock; /* in KHZ */
+ __u64 device_cap_flags_ex;
};
struct ib_uverbs_query_port {
diff --git a/include/uapi/sound/asound.h b/include/uapi/sound/asound.h
index 67bf49d..609cadb 100644
--- a/include/uapi/sound/asound.h
+++ b/include/uapi/sound/asound.h
@@ -672,7 +672,7 @@
/* global timers (device member) */
#define SNDRV_TIMER_GLOBAL_SYSTEM 0
-#define SNDRV_TIMER_GLOBAL_RTC 1
+#define SNDRV_TIMER_GLOBAL_RTC 1 /* unused */
#define SNDRV_TIMER_GLOBAL_HPET 2
#define SNDRV_TIMER_GLOBAL_HRTIMER 3
diff --git a/include/video/exynos5433_decon.h b/include/video/exynos5433_decon.h
index c1c1ca1..0098a52 100644
--- a/include/video/exynos5433_decon.h
+++ b/include/video/exynos5433_decon.h
@@ -179,9 +179,9 @@
#define TRIGCON_TRIGMODE_W1BUF (1 << 10)
#define TRIGCON_SWTRIGCMD_W0BUF (1 << 6)
#define TRIGCON_TRIGMODE_W0BUF (1 << 5)
-#define TRIGCON_HWTRIGMASK_I80_RGB (1 << 4)
-#define TRIGCON_HWTRIGEN_I80_RGB (1 << 3)
-#define TRIGCON_HWTRIG_INV_I80_RGB (1 << 2)
+#define TRIGCON_HWTRIGMASK (1 << 4)
+#define TRIGCON_HWTRIGEN (1 << 3)
+#define TRIGCON_HWTRIG_INV (1 << 2)
#define TRIGCON_SWTRIGCMD (1 << 1)
#define TRIGCON_SWTRIGEN (1 << 0)
diff --git a/include/video/mipi_display.h b/include/video/mipi_display.h
index ddcc8ca..19aa65a 100644
--- a/include/video/mipi_display.h
+++ b/include/video/mipi_display.h
@@ -115,6 +115,14 @@
MIPI_DCS_READ_MEMORY_CONTINUE = 0x3E,
MIPI_DCS_SET_TEAR_SCANLINE = 0x44,
MIPI_DCS_GET_SCANLINE = 0x45,
+ MIPI_DCS_SET_DISPLAY_BRIGHTNESS = 0x51, /* MIPI DCS 1.3 */
+ MIPI_DCS_GET_DISPLAY_BRIGHTNESS = 0x52, /* MIPI DCS 1.3 */
+ MIPI_DCS_WRITE_CONTROL_DISPLAY = 0x53, /* MIPI DCS 1.3 */
+ MIPI_DCS_GET_CONTROL_DISPLAY = 0x54, /* MIPI DCS 1.3 */
+ MIPI_DCS_WRITE_POWER_SAVE = 0x55, /* MIPI DCS 1.3 */
+ MIPI_DCS_GET_POWER_SAVE = 0x56, /* MIPI DCS 1.3 */
+ MIPI_DCS_SET_CABC_MIN_BRIGHTNESS = 0x5E, /* MIPI DCS 1.3 */
+ MIPI_DCS_GET_CABC_MIN_BRIGHTNESS = 0x5F, /* MIPI DCS 1.3 */
MIPI_DCS_READ_DDB_START = 0xA1,
MIPI_DCS_READ_DDB_CONTINUE = 0xA8,
};
diff --git a/include/video/sh_mipi_dsi.h b/include/video/sh_mipi_dsi.h
deleted file mode 100644
index a01f197..0000000
--- a/include/video/sh_mipi_dsi.h
+++ /dev/null
@@ -1,59 +0,0 @@
-/*
- * Public SH-mobile MIPI DSI header
- *
- * Copyright (C) 2010 Guennadi Liakhovetski <g.liakhovetski@gmx.de>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-#ifndef VIDEO_SH_MIPI_DSI_H
-#define VIDEO_SH_MIPI_DSI_H
-
-enum sh_mipi_dsi_data_fmt {
- MIPI_RGB888,
- MIPI_RGB565,
- MIPI_RGB666_LP,
- MIPI_RGB666,
- MIPI_BGR888,
- MIPI_BGR565,
- MIPI_BGR666_LP,
- MIPI_BGR666,
- MIPI_YUYV,
- MIPI_UYVY,
- MIPI_YUV420_L,
- MIPI_YUV420,
-};
-
-#define SH_MIPI_DSI_HSABM (1 << 0)
-#define SH_MIPI_DSI_HBPBM (1 << 1)
-#define SH_MIPI_DSI_HFPBM (1 << 2)
-#define SH_MIPI_DSI_BL2E (1 << 3)
-#define SH_MIPI_DSI_VSEE (1 << 4)
-#define SH_MIPI_DSI_HSEE (1 << 5)
-#define SH_MIPI_DSI_HSAE (1 << 6)
-
-#define SH_MIPI_DSI_HSbyteCLK (1 << 24)
-#define SH_MIPI_DSI_HS6divCLK (1 << 25)
-#define SH_MIPI_DSI_HS4divCLK (1 << 26)
-
-#define SH_MIPI_DSI_SYNC_PULSES_MODE (SH_MIPI_DSI_VSEE | \
- SH_MIPI_DSI_HSEE | \
- SH_MIPI_DSI_HSAE)
-#define SH_MIPI_DSI_SYNC_EVENTS_MODE (0)
-#define SH_MIPI_DSI_SYNC_BURST_MODE (SH_MIPI_DSI_BL2E)
-
-struct sh_mipi_dsi_info {
- enum sh_mipi_dsi_data_fmt data_format;
- int channel;
- int lane;
- unsigned long flags;
- u32 clksrc;
- u32 phyctrl; /* for extra setting */
- unsigned int vsynw_offset;
- int (*set_dot_clock)(struct platform_device *pdev,
- void __iomem *base,
- int enable);
-};
-
-#endif
diff --git a/init/Kconfig b/init/Kconfig
index 0dfd09d..a9c4aefd 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -862,6 +862,28 @@
13 => 8 KB for each CPU
12 => 4 KB for each CPU
+config NMI_LOG_BUF_SHIFT
+ int "Temporary per-CPU NMI log buffer size (12 => 4KB, 13 => 8KB)"
+ range 10 21
+ default 13
+ depends on PRINTK_NMI
+ help
+ Select the size of a per-CPU buffer where NMI messages are temporary
+ stored. They are copied to the main log buffer in a safe context
+ to avoid a deadlock. The value defines the size as a power of 2.
+
+ NMI messages are rare and limited. The largest one is when
+ a backtrace is printed. It usually fits into 4KB. Select
+ 8KB if you want to be on the safe side.
+
+ Examples:
+ 17 => 128 KB for each CPU
+ 16 => 64 KB for each CPU
+ 15 => 32 KB for each CPU
+ 14 => 16 KB for each CPU
+ 13 => 8 KB for each CPU
+ 12 => 4 KB for each CPU
+
#
# Architectures with an unreliable sched_clock() should select this:
#
@@ -1454,6 +1476,11 @@
very difficult to diagnose system problems, saying N here is
strongly discouraged.
+config PRINTK_NMI
+ def_bool y
+ depends on PRINTK
+ depends on HAVE_NMI
+
config BUG
bool "BUG() support" if EXPERT
default y
@@ -1742,6 +1769,15 @@
endchoice
+config SLAB_FREELIST_RANDOM
+ default n
+ depends on SLAB
+ bool "SLAB freelist randomization"
+ help
+ Randomizes the freelist order used on creating new SLABs. This
+ security feature reduces the predictability of the kernel slab
+ allocator against heap overflows.
+
config SLUB_CPU_PARTIAL
default y
depends on SLUB && SMP
diff --git a/init/main.c b/init/main.c
index b3c6e36..bc0f9e0 100644
--- a/init/main.c
+++ b/init/main.c
@@ -569,6 +569,7 @@
timekeeping_init();
time_init();
sched_clock_postinit();
+ printk_nmi_init();
perf_event_init();
profile_init();
call_function_init();
@@ -606,7 +607,6 @@
initrd_start = 0;
}
#endif
- page_ext_init();
debug_objects_mem_init();
kmemleak_init();
setup_per_cpu_pageset();
@@ -706,21 +706,20 @@
static bool __init_or_module initcall_blacklisted(initcall_t fn)
{
struct blacklist_entry *entry;
- char *fn_name;
+ char fn_name[KSYM_SYMBOL_LEN];
- fn_name = kasprintf(GFP_KERNEL, "%pf", fn);
- if (!fn_name)
+ if (list_empty(&blacklisted_initcalls))
return false;
+ sprint_symbol_no_offset(fn_name, (unsigned long)fn);
+
list_for_each_entry(entry, &blacklisted_initcalls, next) {
if (!strcmp(fn_name, entry->buf)) {
pr_debug("initcall %s blacklisted\n", fn_name);
- kfree(fn_name);
return true;
}
}
- kfree(fn_name);
return false;
}
#else
@@ -1004,6 +1003,8 @@
sched_init_smp();
page_alloc_init_late();
+ /* Initialize page ext after all struct pages are initializaed */
+ page_ext_init();
do_basic_setup();
diff --git a/ipc/shm.c b/ipc/shm.c
index 331fc1b..1328251 100644
--- a/ipc/shm.c
+++ b/ipc/shm.c
@@ -1200,7 +1200,11 @@
if (err)
goto out_fput;
- down_write(¤t->mm->mmap_sem);
+ if (down_write_killable(¤t->mm->mmap_sem)) {
+ err = -EINTR;
+ goto out_fput;
+ }
+
if (addr && !(shmflg & SHM_REMAP)) {
err = -EINVAL;
if (addr + size < addr)
@@ -1271,7 +1275,8 @@
if (addr & ~PAGE_MASK)
return retval;
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
/*
* This function tries to be smart and unmap shm segments that
diff --git a/kernel/Makefile b/kernel/Makefile
index f0c40bf..e2ec54e 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -91,9 +91,7 @@
obj-$(CONFIG_TASKSTATS) += taskstats.o tsacct.o
obj-$(CONFIG_TRACEPOINTS) += tracepoint.o
obj-$(CONFIG_LATENCYTOP) += latencytop.o
-obj-$(CONFIG_BINFMT_ELF) += elfcore.o
-obj-$(CONFIG_COMPAT_BINFMT_ELF) += elfcore.o
-obj-$(CONFIG_BINFMT_ELF_FDPIC) += elfcore.o
+obj-$(CONFIG_ELFCORE) += elfcore.o
obj-$(CONFIG_FUNCTION_TRACER) += trace/
obj-$(CONFIG_TRACING) += trace/
obj-$(CONFIG_TRACE_CLOCK) += trace/
diff --git a/kernel/audit.c b/kernel/audit.c
index 678c3f0..22bb4f2 100644
--- a/kernel/audit.c
+++ b/kernel/audit.c
@@ -64,7 +64,6 @@
#include <linux/security.h>
#endif
#include <linux/freezer.h>
-#include <linux/tty.h>
#include <linux/pid_namespace.h>
#include <net/netns/generic.h>
@@ -430,7 +429,6 @@
attempts, audit_pid);
set_current_state(TASK_INTERRUPTIBLE);
schedule();
- __set_current_state(TASK_RUNNING);
goto restart;
}
}
@@ -1341,15 +1339,14 @@
static long wait_for_auditd(long sleep_time)
{
DECLARE_WAITQUEUE(wait, current);
- set_current_state(TASK_UNINTERRUPTIBLE);
- add_wait_queue_exclusive(&audit_backlog_wait, &wait);
if (audit_backlog_limit &&
- skb_queue_len(&audit_skb_queue) > audit_backlog_limit)
+ skb_queue_len(&audit_skb_queue) > audit_backlog_limit) {
+ add_wait_queue_exclusive(&audit_backlog_wait, &wait);
+ set_current_state(TASK_UNINTERRUPTIBLE);
sleep_time = schedule_timeout(sleep_time);
-
- __set_current_state(TASK_RUNNING);
- remove_wait_queue(&audit_backlog_wait, &wait);
+ remove_wait_queue(&audit_backlog_wait, &wait);
+ }
return sleep_time;
}
@@ -1890,21 +1887,14 @@
{
const struct cred *cred;
char comm[sizeof(tsk->comm)];
- char *tty;
+ struct tty_struct *tty;
if (!ab)
return;
/* tsk == current */
cred = current_cred();
-
- spin_lock_irq(&tsk->sighand->siglock);
- if (tsk->signal && tsk->signal->tty && tsk->signal->tty->name)
- tty = tsk->signal->tty->name;
- else
- tty = "(none)";
- spin_unlock_irq(&tsk->sighand->siglock);
-
+ tty = audit_get_tty(tsk);
audit_log_format(ab,
" ppid=%d pid=%d auid=%u uid=%u gid=%u"
" euid=%u suid=%u fsuid=%u"
@@ -1920,11 +1910,11 @@
from_kgid(&init_user_ns, cred->egid),
from_kgid(&init_user_ns, cred->sgid),
from_kgid(&init_user_ns, cred->fsgid),
- tty, audit_get_sessionid(tsk));
-
+ tty ? tty_name(tty) : "(none)",
+ audit_get_sessionid(tsk));
+ audit_put_tty(tty);
audit_log_format(ab, " comm=");
audit_log_untrustedstring(ab, get_task_comm(comm, tsk));
-
audit_log_d_path_exe(ab, tsk->mm);
audit_log_task_context(ab);
}
diff --git a/kernel/audit_tree.c b/kernel/audit_tree.c
index 5efe9b29..2577247 100644
--- a/kernel/audit_tree.c
+++ b/kernel/audit_tree.c
@@ -661,10 +661,10 @@
static int prune_tree_thread(void *unused)
{
for (;;) {
- set_current_state(TASK_INTERRUPTIBLE);
- if (list_empty(&prune_list))
+ if (list_empty(&prune_list)) {
+ set_current_state(TASK_INTERRUPTIBLE);
schedule();
- __set_current_state(TASK_RUNNING);
+ }
mutex_lock(&audit_cmd_mutex);
mutex_lock(&audit_filter_mutex);
@@ -693,16 +693,14 @@
{
if (prune_thread)
return 0;
- prune_thread = kthread_create(prune_tree_thread, NULL,
+ prune_thread = kthread_run(prune_tree_thread, NULL,
"audit_prune_tree");
if (IS_ERR(prune_thread)) {
pr_err("cannot start thread audit_prune_tree");
prune_thread = NULL;
return -ENOMEM;
- } else {
- wake_up_process(prune_thread);
- return 0;
}
+ return 0;
}
/* called with audit_filter_mutex */
diff --git a/kernel/auditsc.c b/kernel/auditsc.c
index 7d0e3cf..62ab53d 100644
--- a/kernel/auditsc.c
+++ b/kernel/auditsc.c
@@ -1980,6 +1980,7 @@
{
struct audit_buffer *ab;
uid_t uid, oldloginuid, loginuid;
+ struct tty_struct *tty;
if (!audit_enabled)
return;
@@ -1987,14 +1988,17 @@
uid = from_kuid(&init_user_ns, task_uid(current));
oldloginuid = from_kuid(&init_user_ns, koldloginuid);
loginuid = from_kuid(&init_user_ns, kloginuid),
+ tty = audit_get_tty(current);
ab = audit_log_start(NULL, GFP_KERNEL, AUDIT_LOGIN);
if (!ab)
return;
audit_log_format(ab, "pid=%d uid=%u", task_pid_nr(current), uid);
audit_log_task_context(ab);
- audit_log_format(ab, " old-auid=%u auid=%u old-ses=%u ses=%u res=%d",
- oldloginuid, loginuid, oldsessionid, sessionid, !rc);
+ audit_log_format(ab, " old-auid=%u auid=%u tty=%s old-ses=%u ses=%u res=%d",
+ oldloginuid, loginuid, tty ? tty_name(tty) : "(none)",
+ oldsessionid, sessionid, !rc);
+ audit_put_tty(tty);
audit_log_end(ab);
}
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index f1e8a0d..b94a365 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -231,7 +231,7 @@
hdr->pages = size / PAGE_SIZE;
hole = min_t(unsigned int, size - (proglen + sizeof(*hdr)),
PAGE_SIZE - sizeof(*hdr));
- start = (prandom_u32() % hole) & ~(alignment - 1);
+ start = (get_random_int() % hole) & ~(alignment - 1);
/* Leave a random number of instructions before BPF code. */
*image_ptr = &hdr->image[start];
@@ -251,7 +251,7 @@
struct bpf_insn *to_buff)
{
struct bpf_insn *to = to_buff;
- u32 imm_rnd = prandom_u32();
+ u32 imm_rnd = get_random_int();
s16 off;
BUILD_BUG_ON(BPF_REG_AX + 1 != MAX_BPF_JIT_REG);
diff --git a/kernel/bpf/inode.c b/kernel/bpf/inode.c
index 71b75d9..04be702 100644
--- a/kernel/bpf/inode.c
+++ b/kernel/bpf/inode.c
@@ -357,7 +357,7 @@
static struct dentry *bpf_mount(struct file_system_type *type, int flags,
const char *dev_name, void *data)
{
- return mount_ns(type, flags, current->nsproxy->mnt_ns, bpf_fill_super);
+ return mount_nodev(type, flags, data, bpf_fill_super);
}
static struct file_system_type bpf_fs_type = {
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index a08d662..668e079 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -683,15 +683,11 @@
{
struct reg_state *regs = env->cur_state.regs;
struct reg_state *reg = ®s[regno];
- int linear_size = (int) reg->range - (int) reg->off;
- if (linear_size < 0 || linear_size >= MAX_PACKET_OFF) {
- verbose("verifier bug\n");
- return -EFAULT;
- }
- if (off < 0 || off + size > linear_size) {
- verbose("invalid access to packet, off=%d size=%d, allowed=%d\n",
- off, size, linear_size);
+ off += reg->off;
+ if (off < 0 || off + size > reg->range) {
+ verbose("invalid access to packet, off=%d size=%d, R%d(id=%d,off=%d,r=%d)\n",
+ off, size, regno, reg->id, reg->off, reg->range);
return -EACCES;
}
return 0;
@@ -1249,6 +1245,7 @@
struct reg_state *regs = env->cur_state.regs;
struct reg_state *dst_reg = ®s[insn->dst_reg];
struct reg_state *src_reg = ®s[insn->src_reg];
+ struct reg_state tmp_reg;
s32 imm;
if (BPF_SRC(insn->code) == BPF_K) {
@@ -1271,6 +1268,19 @@
*/
dst_reg->off += imm;
} else {
+ if (src_reg->type == PTR_TO_PACKET) {
+ /* R6=pkt(id=0,off=0,r=62) R7=imm22; r7 += r6 */
+ tmp_reg = *dst_reg; /* save r7 state */
+ *dst_reg = *src_reg; /* copy pkt_ptr state r6 into r7 */
+ src_reg = &tmp_reg; /* pretend it's src_reg state */
+ /* if the checks below reject it, the copy won't matter,
+ * since we're rejecting the whole program. If all ok,
+ * then imm22 state will be added to r7
+ * and r7 will be pkt(id=0,off=22,r=62) while
+ * r6 will stay as pkt(id=0,off=0,r=62)
+ */
+ }
+
if (src_reg->type == CONST_IMM) {
/* pkt_ptr += reg where reg is known constant */
imm = src_reg->imm;
@@ -1569,7 +1579,9 @@
return 0;
} else if (opcode == BPF_ADD &&
BPF_CLASS(insn->code) == BPF_ALU64 &&
- dst_reg->type == PTR_TO_PACKET) {
+ (dst_reg->type == PTR_TO_PACKET ||
+ (BPF_SRC(insn->code) == BPF_X &&
+ regs[insn->src_reg].type == PTR_TO_PACKET))) {
/* ptr_to_packet += K|X */
return check_packet_ptr_add(env, insn);
} else if (BPF_CLASS(insn->code) == BPF_ALU64 &&
diff --git a/kernel/cpuset.c b/kernel/cpuset.c
index 1902956..73e93e5 100644
--- a/kernel/cpuset.c
+++ b/kernel/cpuset.c
@@ -61,7 +61,7 @@
#include <linux/cgroup.h>
#include <linux/wait.h>
-struct static_key cpusets_enabled_key __read_mostly = STATIC_KEY_INIT_FALSE;
+DEFINE_STATIC_KEY_FALSE(cpusets_enabled_key);
/* See "Frequency meter" comments, below. */
@@ -2528,27 +2528,27 @@
* GFP_KERNEL - any node in enclosing hardwalled cpuset ok
* GFP_USER - only nodes in current tasks mems allowed ok.
*/
-int __cpuset_node_allowed(int node, gfp_t gfp_mask)
+bool __cpuset_node_allowed(int node, gfp_t gfp_mask)
{
struct cpuset *cs; /* current cpuset ancestors */
int allowed; /* is allocation in zone z allowed? */
unsigned long flags;
if (in_interrupt())
- return 1;
+ return true;
if (node_isset(node, current->mems_allowed))
- return 1;
+ return true;
/*
* Allow tasks that have access to memory reserves because they have
* been OOM killed to get memory anywhere.
*/
if (unlikely(test_thread_flag(TIF_MEMDIE)))
- return 1;
+ return true;
if (gfp_mask & __GFP_HARDWALL) /* If hardwall request, stop here */
- return 0;
+ return false;
if (current->flags & PF_EXITING) /* Let dying task have memory */
- return 1;
+ return true;
/* Not hardwall and node outside mems_allowed: scan up cpusets */
spin_lock_irqsave(&callback_lock, flags);
@@ -2591,13 +2591,7 @@
static int cpuset_spread_node(int *rotor)
{
- int node;
-
- node = next_node(*rotor, current->mems_allowed);
- if (node == MAX_NUMNODES)
- node = first_node(current->mems_allowed);
- *rotor = node;
- return node;
+ return *rotor = next_node_in(*rotor, current->mems_allowed);
}
int cpuset_mem_spread_node(void)
diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
index 7edc95e..b7a525a 100644
--- a/kernel/events/uprobes.c
+++ b/kernel/events/uprobes.c
@@ -1130,7 +1130,9 @@
struct vm_area_struct *vma;
int ret;
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
+
if (mm->uprobes_state.xol_area) {
ret = -EALREADY;
goto fail;
@@ -1469,7 +1471,8 @@
if (current->flags & PF_EXITING)
return;
- if (!__create_xol_area(current->utask->dup_xol_addr))
+ if (!__create_xol_area(current->utask->dup_xol_addr) &&
+ !fatal_signal_pending(current))
uprobe_warn(current, "dup xol area");
}
@@ -1694,8 +1697,7 @@
int result;
pagefault_disable();
- result = __copy_from_user_inatomic(&opcode, (void __user*)vaddr,
- sizeof(opcode));
+ result = __get_user(opcode, (uprobe_opcode_t __user *)vaddr);
pagefault_enable();
if (likely(result == 0))
diff --git a/kernel/exit.c b/kernel/exit.c
index fd90195..9e6e135 100644
--- a/kernel/exit.c
+++ b/kernel/exit.c
@@ -746,7 +746,7 @@
disassociate_ctty(1);
exit_task_namespaces(tsk);
exit_task_work(tsk);
- exit_thread();
+ exit_thread(tsk);
/*
* Flush inherited counters to the parent - before the parent
@@ -918,17 +918,28 @@
task_pid_type(p, wo->wo_type) == wo->wo_pid;
}
-static int eligible_child(struct wait_opts *wo, struct task_struct *p)
+static int
+eligible_child(struct wait_opts *wo, bool ptrace, struct task_struct *p)
{
if (!eligible_pid(wo, p))
return 0;
- /* Wait for all children (clone and not) if __WALL is set;
- * otherwise, wait for clone children *only* if __WCLONE is
- * set; otherwise, wait for non-clone children *only*. (Note:
- * A "clone" child here is one that reports to its parent
- * using a signal other than SIGCHLD.) */
- if (((p->exit_signal != SIGCHLD) ^ !!(wo->wo_flags & __WCLONE))
- && !(wo->wo_flags & __WALL))
+
+ /*
+ * Wait for all children (clone and not) if __WALL is set or
+ * if it is traced by us.
+ */
+ if (ptrace || (wo->wo_flags & __WALL))
+ return 1;
+
+ /*
+ * Otherwise, wait for clone children *only* if __WCLONE is set;
+ * otherwise, wait for non-clone children *only*.
+ *
+ * Note: a "clone" child here is one that reports to its parent
+ * using a signal other than SIGCHLD, or a non-leader thread which
+ * we can only see if it is traced by us.
+ */
+ if ((p->exit_signal != SIGCHLD) ^ !!(wo->wo_flags & __WCLONE))
return 0;
return 1;
@@ -1300,7 +1311,7 @@
if (unlikely(exit_state == EXIT_DEAD))
return 0;
- ret = eligible_child(wo, p);
+ ret = eligible_child(wo, ptrace, p);
if (!ret)
return ret;
@@ -1524,7 +1535,8 @@
enum pid_type type;
long ret;
- if (options & ~(WNOHANG|WNOWAIT|WEXITED|WSTOPPED|WCONTINUED))
+ if (options & ~(WNOHANG|WNOWAIT|WEXITED|WSTOPPED|WCONTINUED|
+ __WNOTHREAD|__WCLONE|__WALL))
return -EINVAL;
if (!(options & (WEXITED|WSTOPPED|WCONTINUED)))
return -EINVAL;
diff --git a/kernel/fork.c b/kernel/fork.c
index 3e84515..47887bb 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -340,13 +340,14 @@
*stackend = STACK_END_MAGIC; /* for overflow detection */
}
-static struct task_struct *dup_task_struct(struct task_struct *orig)
+static struct task_struct *dup_task_struct(struct task_struct *orig, int node)
{
struct task_struct *tsk;
struct thread_info *ti;
- int node = tsk_fork_get_node(orig);
int err;
+ if (node == NUMA_NO_NODE)
+ node = tsk_fork_get_node(orig);
tsk = alloc_task_struct_node(node);
if (!tsk)
return NULL;
@@ -413,7 +414,10 @@
unsigned long charge;
uprobe_start_dup_mmap();
- down_write(&oldmm->mmap_sem);
+ if (down_write_killable(&oldmm->mmap_sem)) {
+ retval = -EINTR;
+ goto fail_uprobe_end;
+ }
flush_cache_dup_mm(oldmm);
uprobe_dup_mmap(oldmm, mm);
/*
@@ -525,6 +529,7 @@
up_write(&mm->mmap_sem);
flush_tlb_mm(oldmm);
up_write(&oldmm->mmap_sem);
+fail_uprobe_end:
uprobe_end_dup_mmap();
return retval;
fail_nomem_anon_vma_fork:
@@ -699,6 +704,26 @@
}
EXPORT_SYMBOL_GPL(__mmdrop);
+static inline void __mmput(struct mm_struct *mm)
+{
+ VM_BUG_ON(atomic_read(&mm->mm_users));
+
+ uprobe_clear_state(mm);
+ exit_aio(mm);
+ ksm_exit(mm);
+ khugepaged_exit(mm); /* must run before exit_mmap */
+ exit_mmap(mm);
+ set_mm_exe_file(mm, NULL);
+ if (!list_empty(&mm->mmlist)) {
+ spin_lock(&mmlist_lock);
+ list_del(&mm->mmlist);
+ spin_unlock(&mmlist_lock);
+ }
+ if (mm->binfmt)
+ module_put(mm->binfmt->module);
+ mmdrop(mm);
+}
+
/*
* Decrement the use count and release all resources for an mm.
*/
@@ -706,25 +731,25 @@
{
might_sleep();
- if (atomic_dec_and_test(&mm->mm_users)) {
- uprobe_clear_state(mm);
- exit_aio(mm);
- ksm_exit(mm);
- khugepaged_exit(mm); /* must run before exit_mmap */
- exit_mmap(mm);
- set_mm_exe_file(mm, NULL);
- if (!list_empty(&mm->mmlist)) {
- spin_lock(&mmlist_lock);
- list_del(&mm->mmlist);
- spin_unlock(&mmlist_lock);
- }
- if (mm->binfmt)
- module_put(mm->binfmt->module);
- mmdrop(mm);
- }
+ if (atomic_dec_and_test(&mm->mm_users))
+ __mmput(mm);
}
EXPORT_SYMBOL_GPL(mmput);
+static void mmput_async_fn(struct work_struct *work)
+{
+ struct mm_struct *mm = container_of(work, struct mm_struct, async_put_work);
+ __mmput(mm);
+}
+
+void mmput_async(struct mm_struct *mm)
+{
+ if (atomic_dec_and_test(&mm->mm_users)) {
+ INIT_WORK(&mm->async_put_work, mmput_async_fn);
+ schedule_work(&mm->async_put_work);
+ }
+}
+
/**
* set_mm_exe_file - change a reference to the mm's executable file
*
@@ -1256,7 +1281,8 @@
int __user *child_tidptr,
struct pid *pid,
int trace,
- unsigned long tls)
+ unsigned long tls,
+ int node)
{
int retval;
struct task_struct *p;
@@ -1308,7 +1334,7 @@
goto fork_out;
retval = -ENOMEM;
- p = dup_task_struct(current);
+ p = dup_task_struct(current, node);
if (!p)
goto fork_out;
@@ -1470,7 +1496,7 @@
pid = alloc_pid(p->nsproxy->pid_ns_for_children);
if (IS_ERR(pid)) {
retval = PTR_ERR(pid);
- goto bad_fork_cleanup_io;
+ goto bad_fork_cleanup_thread;
}
}
@@ -1632,6 +1658,8 @@
bad_fork_free_pid:
if (pid != &init_struct_pid)
free_pid(pid);
+bad_fork_cleanup_thread:
+ exit_thread(p);
bad_fork_cleanup_io:
if (p->io_context)
exit_io_context(p);
@@ -1684,7 +1712,8 @@
struct task_struct *fork_idle(int cpu)
{
struct task_struct *task;
- task = copy_process(CLONE_VM, 0, 0, NULL, &init_struct_pid, 0, 0);
+ task = copy_process(CLONE_VM, 0, 0, NULL, &init_struct_pid, 0, 0,
+ cpu_to_node(cpu));
if (!IS_ERR(task)) {
init_idle_pids(task->pids);
init_idle(task, cpu);
@@ -1729,7 +1758,7 @@
}
p = copy_process(clone_flags, stack_start, stack_size,
- child_tidptr, NULL, trace, tls);
+ child_tidptr, NULL, trace, tls, NUMA_NO_NODE);
/*
* Do this prior waking up the new thread - the thread pointer
* might get invalid after that point, if the thread exits quickly.
diff --git a/kernel/futex.c b/kernel/futex.c
index c20f06f..ee25f5b 100644
--- a/kernel/futex.c
+++ b/kernel/futex.c
@@ -729,7 +729,7 @@
int ret;
pagefault_disable();
- ret = __copy_from_user_inatomic(dest, from, sizeof(u32));
+ ret = __get_user(*dest, from);
pagefault_enable();
return ret ? -EFAULT : 0;
diff --git a/kernel/irq/irqdomain.c b/kernel/irq/irqdomain.c
index d65f6f3..8798b6c 100644
--- a/kernel/irq/irqdomain.c
+++ b/kernel/irq/irqdomain.c
@@ -139,12 +139,7 @@
{
mutex_lock(&irq_domain_mutex);
- /*
- * radix_tree_delete() takes care of destroying the root
- * node when all entries are removed. Shout if there are
- * any mappings left.
- */
- WARN_ON(domain->revmap_tree.height);
+ WARN_ON(!radix_tree_empty(&domain->revmap_tree));
list_del(&domain->link);
diff --git a/kernel/kexec.c b/kernel/kexec.c
index ee70aef..4384672 100644
--- a/kernel/kexec.c
+++ b/kernel/kexec.c
@@ -103,6 +103,65 @@
return ret;
}
+static int do_kexec_load(unsigned long entry, unsigned long nr_segments,
+ struct kexec_segment __user *segments, unsigned long flags)
+{
+ struct kimage **dest_image, *image;
+ unsigned long i;
+ int ret;
+
+ if (flags & KEXEC_ON_CRASH) {
+ dest_image = &kexec_crash_image;
+ if (kexec_crash_image)
+ arch_kexec_unprotect_crashkres();
+ } else {
+ dest_image = &kexec_image;
+ }
+
+ if (nr_segments == 0) {
+ /* Uninstall image */
+ kimage_free(xchg(dest_image, NULL));
+ return 0;
+ }
+ if (flags & KEXEC_ON_CRASH) {
+ /*
+ * Loading another kernel to switch to if this one
+ * crashes. Free any current crash dump kernel before
+ * we corrupt it.
+ */
+ kimage_free(xchg(&kexec_crash_image, NULL));
+ }
+
+ ret = kimage_alloc_init(&image, entry, nr_segments, segments, flags);
+ if (ret)
+ return ret;
+
+ if (flags & KEXEC_PRESERVE_CONTEXT)
+ image->preserve_context = 1;
+
+ ret = machine_kexec_prepare(image);
+ if (ret)
+ goto out;
+
+ for (i = 0; i < nr_segments; i++) {
+ ret = kimage_load_segment(image, &image->segment[i]);
+ if (ret)
+ goto out;
+ }
+
+ kimage_terminate(image);
+
+ /* Install the new kernel and uninstall the old */
+ image = xchg(dest_image, image);
+
+out:
+ if ((flags & KEXEC_ON_CRASH) && kexec_crash_image)
+ arch_kexec_protect_crashkres();
+
+ kimage_free(image);
+ return ret;
+}
+
/*
* Exec Kernel system call: for obvious reasons only root may call it.
*
@@ -127,7 +186,6 @@
SYSCALL_DEFINE4(kexec_load, unsigned long, entry, unsigned long, nr_segments,
struct kexec_segment __user *, segments, unsigned long, flags)
{
- struct kimage **dest_image, *image;
int result;
/* We only trust the superuser with rebooting the system. */
@@ -152,9 +210,6 @@
if (nr_segments > KEXEC_SEGMENT_MAX)
return -EINVAL;
- image = NULL;
- result = 0;
-
/* Because we write directly to the reserved memory
* region when loading crash kernels we need a mutex here to
* prevent multiple crash kernels from attempting to load
@@ -166,53 +221,9 @@
if (!mutex_trylock(&kexec_mutex))
return -EBUSY;
- dest_image = &kexec_image;
- if (flags & KEXEC_ON_CRASH)
- dest_image = &kexec_crash_image;
- if (nr_segments > 0) {
- unsigned long i;
+ result = do_kexec_load(entry, nr_segments, segments, flags);
- if (flags & KEXEC_ON_CRASH) {
- /*
- * Loading another kernel to switch to if this one
- * crashes. Free any current crash dump kernel before
- * we corrupt it.
- */
-
- kimage_free(xchg(&kexec_crash_image, NULL));
- result = kimage_alloc_init(&image, entry, nr_segments,
- segments, flags);
- crash_map_reserved_pages();
- } else {
- /* Loading another kernel to reboot into. */
-
- result = kimage_alloc_init(&image, entry, nr_segments,
- segments, flags);
- }
- if (result)
- goto out;
-
- if (flags & KEXEC_PRESERVE_CONTEXT)
- image->preserve_context = 1;
- result = machine_kexec_prepare(image);
- if (result)
- goto out;
-
- for (i = 0; i < nr_segments; i++) {
- result = kimage_load_segment(image, &image->segment[i]);
- if (result)
- goto out;
- }
- kimage_terminate(image);
- if (flags & KEXEC_ON_CRASH)
- crash_unmap_reserved_pages();
- }
- /* Install the new kernel, and Uninstall the old */
- image = xchg(dest_image, image);
-
-out:
mutex_unlock(&kexec_mutex);
- kimage_free(image);
return result;
}
diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c
index 1391d3e..56b3ed0 100644
--- a/kernel/kexec_core.c
+++ b/kernel/kexec_core.c
@@ -893,6 +893,7 @@
old_cpu = atomic_cmpxchg(&panic_cpu, PANIC_CPU_INVALID, this_cpu);
if (old_cpu == PANIC_CPU_INVALID) {
/* This is the 1st CPU which comes here, so go ahead. */
+ printk_nmi_flush_on_panic();
__crash_kexec(regs);
/*
@@ -953,7 +954,6 @@
start = roundup(start, KEXEC_CRASH_MEM_ALIGN);
end = roundup(start + new_size, KEXEC_CRASH_MEM_ALIGN);
- crash_map_reserved_pages();
crash_free_reserved_phys_range(end, crashk_res.end);
if ((start == end) && (crashk_res.parent != NULL))
@@ -967,7 +967,6 @@
crashk_res.end = end - 1;
insert_resource(&iomem_resource, ram_res);
- crash_unmap_reserved_pages();
unlock:
mutex_unlock(&kexec_mutex);
@@ -1410,7 +1409,7 @@
VMCOREINFO_STRUCT_SIZE(list_head);
VMCOREINFO_SIZE(nodemask_t);
VMCOREINFO_OFFSET(page, flags);
- VMCOREINFO_OFFSET(page, _count);
+ VMCOREINFO_OFFSET(page, _refcount);
VMCOREINFO_OFFSET(page, mapping);
VMCOREINFO_OFFSET(page, lru);
VMCOREINFO_OFFSET(page, _mapcount);
@@ -1552,13 +1551,14 @@
}
/*
- * Add and remove page tables for crashkernel memory
+ * Protection mechanism for crashkernel reserved memory after
+ * the kdump kernel is loaded.
*
* Provide an empty default implementation here -- architecture
* code may override this
*/
-void __weak crash_map_reserved_pages(void)
+void __weak arch_kexec_protect_crashkres(void)
{}
-void __weak crash_unmap_reserved_pages(void)
+void __weak arch_kexec_unprotect_crashkres(void)
{}
diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c
index c72d2ff..503bc2d 100644
--- a/kernel/kexec_file.c
+++ b/kernel/kexec_file.c
@@ -274,8 +274,11 @@
return -EBUSY;
dest_image = &kexec_image;
- if (flags & KEXEC_FILE_ON_CRASH)
+ if (flags & KEXEC_FILE_ON_CRASH) {
dest_image = &kexec_crash_image;
+ if (kexec_crash_image)
+ arch_kexec_unprotect_crashkres();
+ }
if (flags & KEXEC_FILE_UNLOAD)
goto exchange;
@@ -324,6 +327,9 @@
exchange:
image = xchg(dest_image, image);
out:
+ if ((flags & KEXEC_FILE_ON_CRASH) && kexec_crash_image)
+ arch_kexec_protect_crashkres();
+
mutex_unlock(&kexec_mutex);
kimage_free(image);
return ret;
diff --git a/kernel/locking/percpu-rwsem.c b/kernel/locking/percpu-rwsem.c
index f231e0b..bec0b64 100644
--- a/kernel/locking/percpu-rwsem.c
+++ b/kernel/locking/percpu-rwsem.c
@@ -37,6 +37,7 @@
free_percpu(brw->fast_read_ctr);
brw->fast_read_ctr = NULL; /* catch use after free bugs */
}
+EXPORT_SYMBOL_GPL(percpu_free_rwsem);
/*
* This is the fast-path for down_read/up_read. If it succeeds we rely
diff --git a/kernel/memremap.c b/kernel/memremap.c
index a6d3823..0175321 100644
--- a/kernel/memremap.c
+++ b/kernel/memremap.c
@@ -27,6 +27,13 @@
}
#endif
+#ifndef arch_memremap_wb
+static void *arch_memremap_wb(resource_size_t offset, unsigned long size)
+{
+ return (__force void *)ioremap_cache(offset, size);
+}
+#endif
+
static void *try_ram_remap(resource_size_t offset, size_t size)
{
unsigned long pfn = PHYS_PFN(offset);
@@ -34,7 +41,7 @@
/* In the simple case just return the existing linear address */
if (pfn_valid(pfn) && !PageHighMem(pfn_to_page(pfn)))
return __va(offset);
- return NULL; /* fallback to ioremap_cache */
+ return NULL; /* fallback to arch_memremap_wb */
}
/**
@@ -90,7 +97,7 @@
if (is_ram == REGION_INTERSECTS)
addr = try_ram_remap(offset, size);
if (!addr)
- addr = ioremap_cache(offset, size);
+ addr = arch_memremap_wb(offset, size);
}
/*
diff --git a/kernel/module_signing.c b/kernel/module_signing.c
index 64b9dea..937c844 100644
--- a/kernel/module_signing.c
+++ b/kernel/module_signing.c
@@ -12,7 +12,7 @@
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/string.h>
-#include <keys/system_keyring.h>
+#include <linux/verification.h>
#include <crypto/public_key.h>
#include "module-internal.h"
@@ -80,6 +80,7 @@
return -EBADMSG;
}
- return system_verify_data(mod, modlen, mod + modlen, sig_len,
- VERIFYING_MODULE_SIGNATURE);
+ return verify_pkcs7_signature(mod, modlen, mod + modlen, sig_len,
+ NULL, VERIFYING_MODULE_SIGNATURE,
+ NULL, NULL);
}
diff --git a/kernel/padata.c b/kernel/padata.c
index b38bea9..9932788 100644
--- a/kernel/padata.c
+++ b/kernel/padata.c
@@ -607,33 +607,6 @@
}
/**
- * padata_set_cpumasks - Set both parallel and serial cpumasks. The first
- * one is used by parallel workers and the second one
- * by the wokers doing serialization.
- *
- * @pinst: padata instance
- * @pcpumask: the cpumask to use for parallel workers
- * @cbcpumask: the cpumsak to use for serial workers
- */
-int padata_set_cpumasks(struct padata_instance *pinst, cpumask_var_t pcpumask,
- cpumask_var_t cbcpumask)
-{
- int err;
-
- mutex_lock(&pinst->lock);
- get_online_cpus();
-
- err = __padata_set_cpumasks(pinst, pcpumask, cbcpumask);
-
- put_online_cpus();
- mutex_unlock(&pinst->lock);
-
- return err;
-
-}
-EXPORT_SYMBOL(padata_set_cpumasks);
-
-/**
* padata_set_cpumask: Sets specified by @cpumask_type cpumask to the value
* equivalent to @cpumask.
*
@@ -674,6 +647,43 @@
}
EXPORT_SYMBOL(padata_set_cpumask);
+/**
+ * padata_start - start the parallel processing
+ *
+ * @pinst: padata instance to start
+ */
+int padata_start(struct padata_instance *pinst)
+{
+ int err = 0;
+
+ mutex_lock(&pinst->lock);
+
+ if (pinst->flags & PADATA_INVALID)
+ err = -EINVAL;
+
+ __padata_start(pinst);
+
+ mutex_unlock(&pinst->lock);
+
+ return err;
+}
+EXPORT_SYMBOL(padata_start);
+
+/**
+ * padata_stop - stop the parallel processing
+ *
+ * @pinst: padata instance to stop
+ */
+void padata_stop(struct padata_instance *pinst)
+{
+ mutex_lock(&pinst->lock);
+ __padata_stop(pinst);
+ mutex_unlock(&pinst->lock);
+}
+EXPORT_SYMBOL(padata_stop);
+
+#ifdef CONFIG_HOTPLUG_CPU
+
static int __padata_add_cpu(struct padata_instance *pinst, int cpu)
{
struct parallel_data *pd;
@@ -694,42 +704,6 @@
return 0;
}
- /**
- * padata_add_cpu - add a cpu to one or both(parallel and serial)
- * padata cpumasks.
- *
- * @pinst: padata instance
- * @cpu: cpu to add
- * @mask: bitmask of flags specifying to which cpumask @cpu shuld be added.
- * The @mask may be any combination of the following flags:
- * PADATA_CPU_SERIAL - serial cpumask
- * PADATA_CPU_PARALLEL - parallel cpumask
- */
-
-int padata_add_cpu(struct padata_instance *pinst, int cpu, int mask)
-{
- int err;
-
- if (!(mask & (PADATA_CPU_SERIAL | PADATA_CPU_PARALLEL)))
- return -EINVAL;
-
- mutex_lock(&pinst->lock);
-
- get_online_cpus();
- if (mask & PADATA_CPU_SERIAL)
- cpumask_set_cpu(cpu, pinst->cpumask.cbcpu);
- if (mask & PADATA_CPU_PARALLEL)
- cpumask_set_cpu(cpu, pinst->cpumask.pcpu);
-
- err = __padata_add_cpu(pinst, cpu);
- put_online_cpus();
-
- mutex_unlock(&pinst->lock);
-
- return err;
-}
-EXPORT_SYMBOL(padata_add_cpu);
-
static int __padata_remove_cpu(struct padata_instance *pinst, int cpu)
{
struct parallel_data *pd = NULL;
@@ -789,43 +763,6 @@
}
EXPORT_SYMBOL(padata_remove_cpu);
-/**
- * padata_start - start the parallel processing
- *
- * @pinst: padata instance to start
- */
-int padata_start(struct padata_instance *pinst)
-{
- int err = 0;
-
- mutex_lock(&pinst->lock);
-
- if (pinst->flags & PADATA_INVALID)
- err =-EINVAL;
-
- __padata_start(pinst);
-
- mutex_unlock(&pinst->lock);
-
- return err;
-}
-EXPORT_SYMBOL(padata_start);
-
-/**
- * padata_stop - stop the parallel processing
- *
- * @pinst: padata instance to stop
- */
-void padata_stop(struct padata_instance *pinst)
-{
- mutex_lock(&pinst->lock);
- __padata_stop(pinst);
- mutex_unlock(&pinst->lock);
-}
-EXPORT_SYMBOL(padata_stop);
-
-#ifdef CONFIG_HOTPLUG_CPU
-
static inline int pinst_has_cpu(struct padata_instance *pinst, int cpu)
{
return cpumask_test_cpu(cpu, pinst->cpumask.pcpu) ||
@@ -1091,7 +1028,6 @@
err:
return NULL;
}
-EXPORT_SYMBOL(padata_alloc);
/**
* padata_free - free a padata instance
diff --git a/kernel/panic.c b/kernel/panic.c
index 535c965..8aa7449 100644
--- a/kernel/panic.c
+++ b/kernel/panic.c
@@ -160,8 +160,10 @@
*
* Bypass the panic_cpu check and call __crash_kexec directly.
*/
- if (!crash_kexec_post_notifiers)
+ if (!crash_kexec_post_notifiers) {
+ printk_nmi_flush_on_panic();
__crash_kexec(NULL);
+ }
/*
* Note smp_send_stop is the usual smp shutdown function, which
@@ -176,6 +178,8 @@
*/
atomic_notifier_call_chain(&panic_notifier_list, 0, buf);
+ /* Call flush even twice. It tries harder with a single online CPU */
+ printk_nmi_flush_on_panic();
kmsg_dump(KMSG_DUMP_PANIC);
/*
diff --git a/kernel/printk/Makefile b/kernel/printk/Makefile
index 85405bd..abb0042 100644
--- a/kernel/printk/Makefile
+++ b/kernel/printk/Makefile
@@ -1,2 +1,3 @@
obj-y = printk.o
+obj-$(CONFIG_PRINTK_NMI) += nmi.o
obj-$(CONFIG_A11Y_BRAILLE_CONSOLE) += braille.o
diff --git a/kernel/printk/internal.h b/kernel/printk/internal.h
new file mode 100644
index 0000000..7fd2838
--- /dev/null
+++ b/kernel/printk/internal.h
@@ -0,0 +1,57 @@
+/*
+ * internal.h - printk internal definitions
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+#include <linux/percpu.h>
+
+typedef __printf(1, 0) int (*printk_func_t)(const char *fmt, va_list args);
+
+int __printf(1, 0) vprintk_default(const char *fmt, va_list args);
+
+#ifdef CONFIG_PRINTK_NMI
+
+extern raw_spinlock_t logbuf_lock;
+
+/*
+ * printk() could not take logbuf_lock in NMI context. Instead,
+ * it temporary stores the strings into a per-CPU buffer.
+ * The alternative implementation is chosen transparently
+ * via per-CPU variable.
+ */
+DECLARE_PER_CPU(printk_func_t, printk_func);
+static inline __printf(1, 0) int vprintk_func(const char *fmt, va_list args)
+{
+ return this_cpu_read(printk_func)(fmt, args);
+}
+
+extern atomic_t nmi_message_lost;
+static inline int get_nmi_message_lost(void)
+{
+ return atomic_xchg(&nmi_message_lost, 0);
+}
+
+#else /* CONFIG_PRINTK_NMI */
+
+static inline __printf(1, 0) int vprintk_func(const char *fmt, va_list args)
+{
+ return vprintk_default(fmt, args);
+}
+
+static inline int get_nmi_message_lost(void)
+{
+ return 0;
+}
+
+#endif /* CONFIG_PRINTK_NMI */
diff --git a/kernel/printk/nmi.c b/kernel/printk/nmi.c
new file mode 100644
index 0000000..b69eb8a
--- /dev/null
+++ b/kernel/printk/nmi.c
@@ -0,0 +1,260 @@
+/*
+ * nmi.c - Safe printk in NMI context
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/preempt.h>
+#include <linux/spinlock.h>
+#include <linux/debug_locks.h>
+#include <linux/smp.h>
+#include <linux/cpumask.h>
+#include <linux/irq_work.h>
+#include <linux/printk.h>
+
+#include "internal.h"
+
+/*
+ * printk() could not take logbuf_lock in NMI context. Instead,
+ * it uses an alternative implementation that temporary stores
+ * the strings into a per-CPU buffer. The content of the buffer
+ * is later flushed into the main ring buffer via IRQ work.
+ *
+ * The alternative implementation is chosen transparently
+ * via @printk_func per-CPU variable.
+ *
+ * The implementation allows to flush the strings also from another CPU.
+ * There are situations when we want to make sure that all buffers
+ * were handled or when IRQs are blocked.
+ */
+DEFINE_PER_CPU(printk_func_t, printk_func) = vprintk_default;
+static int printk_nmi_irq_ready;
+atomic_t nmi_message_lost;
+
+#define NMI_LOG_BUF_LEN ((1 << CONFIG_NMI_LOG_BUF_SHIFT) - \
+ sizeof(atomic_t) - sizeof(struct irq_work))
+
+struct nmi_seq_buf {
+ atomic_t len; /* length of written data */
+ struct irq_work work; /* IRQ work that flushes the buffer */
+ unsigned char buffer[NMI_LOG_BUF_LEN];
+};
+static DEFINE_PER_CPU(struct nmi_seq_buf, nmi_print_seq);
+
+/*
+ * Safe printk() for NMI context. It uses a per-CPU buffer to
+ * store the message. NMIs are not nested, so there is always only
+ * one writer running. But the buffer might get flushed from another
+ * CPU, so we need to be careful.
+ */
+static int vprintk_nmi(const char *fmt, va_list args)
+{
+ struct nmi_seq_buf *s = this_cpu_ptr(&nmi_print_seq);
+ int add = 0;
+ size_t len;
+
+again:
+ len = atomic_read(&s->len);
+
+ if (len >= sizeof(s->buffer)) {
+ atomic_inc(&nmi_message_lost);
+ return 0;
+ }
+
+ /*
+ * Make sure that all old data have been read before the buffer was
+ * reseted. This is not needed when we just append data.
+ */
+ if (!len)
+ smp_rmb();
+
+ add = vsnprintf(s->buffer + len, sizeof(s->buffer) - len, fmt, args);
+
+ /*
+ * Do it once again if the buffer has been flushed in the meantime.
+ * Note that atomic_cmpxchg() is an implicit memory barrier that
+ * makes sure that the data were written before updating s->len.
+ */
+ if (atomic_cmpxchg(&s->len, len, len + add) != len)
+ goto again;
+
+ /* Get flushed in a more safe context. */
+ if (add && printk_nmi_irq_ready) {
+ /* Make sure that IRQ work is really initialized. */
+ smp_rmb();
+ irq_work_queue(&s->work);
+ }
+
+ return add;
+}
+
+/*
+ * printk one line from the temporary buffer from @start index until
+ * and including the @end index.
+ */
+static void print_nmi_seq_line(struct nmi_seq_buf *s, int start, int end)
+{
+ const char *buf = s->buffer + start;
+
+ /*
+ * The buffers are flushed in NMI only on panic. The messages must
+ * go only into the ring buffer at this stage. Consoles will get
+ * explicitly called later when a crashdump is not generated.
+ */
+ if (in_nmi())
+ printk_deferred("%.*s", (end - start) + 1, buf);
+ else
+ printk("%.*s", (end - start) + 1, buf);
+
+}
+
+/*
+ * Flush data from the associated per_CPU buffer. The function
+ * can be called either via IRQ work or independently.
+ */
+static void __printk_nmi_flush(struct irq_work *work)
+{
+ static raw_spinlock_t read_lock =
+ __RAW_SPIN_LOCK_INITIALIZER(read_lock);
+ struct nmi_seq_buf *s = container_of(work, struct nmi_seq_buf, work);
+ unsigned long flags;
+ size_t len, size;
+ int i, last_i;
+
+ /*
+ * The lock has two functions. First, one reader has to flush all
+ * available message to make the lockless synchronization with
+ * writers easier. Second, we do not want to mix messages from
+ * different CPUs. This is especially important when printing
+ * a backtrace.
+ */
+ raw_spin_lock_irqsave(&read_lock, flags);
+
+ i = 0;
+more:
+ len = atomic_read(&s->len);
+
+ /*
+ * This is just a paranoid check that nobody has manipulated
+ * the buffer an unexpected way. If we printed something then
+ * @len must only increase.
+ */
+ if (i && i >= len)
+ pr_err("printk_nmi_flush: internal error: i=%d >= len=%zu\n",
+ i, len);
+
+ if (!len)
+ goto out; /* Someone else has already flushed the buffer. */
+
+ /* Make sure that data has been written up to the @len */
+ smp_rmb();
+
+ size = min(len, sizeof(s->buffer));
+ last_i = i;
+
+ /* Print line by line. */
+ for (; i < size; i++) {
+ if (s->buffer[i] == '\n') {
+ print_nmi_seq_line(s, last_i, i);
+ last_i = i + 1;
+ }
+ }
+ /* Check if there was a partial line. */
+ if (last_i < size) {
+ print_nmi_seq_line(s, last_i, size - 1);
+ pr_cont("\n");
+ }
+
+ /*
+ * Check that nothing has got added in the meantime and truncate
+ * the buffer. Note that atomic_cmpxchg() is an implicit memory
+ * barrier that makes sure that the data were copied before
+ * updating s->len.
+ */
+ if (atomic_cmpxchg(&s->len, len, 0) != len)
+ goto more;
+
+out:
+ raw_spin_unlock_irqrestore(&read_lock, flags);
+}
+
+/**
+ * printk_nmi_flush - flush all per-cpu nmi buffers.
+ *
+ * The buffers are flushed automatically via IRQ work. This function
+ * is useful only when someone wants to be sure that all buffers have
+ * been flushed at some point.
+ */
+void printk_nmi_flush(void)
+{
+ int cpu;
+
+ for_each_possible_cpu(cpu)
+ __printk_nmi_flush(&per_cpu(nmi_print_seq, cpu).work);
+}
+
+/**
+ * printk_nmi_flush_on_panic - flush all per-cpu nmi buffers when the system
+ * goes down.
+ *
+ * Similar to printk_nmi_flush() but it can be called even in NMI context when
+ * the system goes down. It does the best effort to get NMI messages into
+ * the main ring buffer.
+ *
+ * Note that it could try harder when there is only one CPU online.
+ */
+void printk_nmi_flush_on_panic(void)
+{
+ /*
+ * Make sure that we could access the main ring buffer.
+ * Do not risk a double release when more CPUs are up.
+ */
+ if (in_nmi() && raw_spin_is_locked(&logbuf_lock)) {
+ if (num_online_cpus() > 1)
+ return;
+
+ debug_locks_off();
+ raw_spin_lock_init(&logbuf_lock);
+ }
+
+ printk_nmi_flush();
+}
+
+void __init printk_nmi_init(void)
+{
+ int cpu;
+
+ for_each_possible_cpu(cpu) {
+ struct nmi_seq_buf *s = &per_cpu(nmi_print_seq, cpu);
+
+ init_irq_work(&s->work, __printk_nmi_flush);
+ }
+
+ /* Make sure that IRQ works are initialized before enabling. */
+ smp_wmb();
+ printk_nmi_irq_ready = 1;
+
+ /* Flush pending messages that did not have scheduled IRQ works. */
+ printk_nmi_flush();
+}
+
+void printk_nmi_enter(void)
+{
+ this_cpu_write(printk_func, vprintk_nmi);
+}
+
+void printk_nmi_exit(void)
+{
+ this_cpu_write(printk_func, vprintk_default);
+}
diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index bfbf284..60cdf63 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -55,6 +55,7 @@
#include "console_cmdline.h"
#include "braille.h"
+#include "internal.h"
int console_printk[4] = {
CONSOLE_LOGLEVEL_DEFAULT, /* console_loglevel */
@@ -244,7 +245,7 @@
* within the scheduler's rq lock. It must be released before calling
* console_unlock() or anything else that might wake up a process.
*/
-static DEFINE_RAW_SPINLOCK(logbuf_lock);
+DEFINE_RAW_SPINLOCK(logbuf_lock);
#ifdef CONFIG_PRINTK
DECLARE_WAIT_QUEUE_HEAD(log_wait);
@@ -1616,6 +1617,7 @@
unsigned long flags;
int this_cpu;
int printed_len = 0;
+ int nmi_message_lost;
bool in_sched = false;
/* cpu currently holding logbuf_lock in this function */
static unsigned int logbuf_cpu = UINT_MAX;
@@ -1666,6 +1668,15 @@
strlen(recursion_msg));
}
+ nmi_message_lost = get_nmi_message_lost();
+ if (unlikely(nmi_message_lost)) {
+ text_len = scnprintf(textbuf, sizeof(textbuf),
+ "BAD LUCK: lost %d message(s) from NMI context!",
+ nmi_message_lost);
+ printed_len += log_store(0, 2, LOG_PREFIX|LOG_NEWLINE, 0,
+ NULL, 0, textbuf, text_len);
+ }
+
/*
* The printf needs to come first; we need the syslog
* prefix which might be passed-in as a parameter.
@@ -1807,14 +1818,6 @@
}
EXPORT_SYMBOL_GPL(vprintk_default);
-/*
- * This allows printk to be diverted to another function per cpu.
- * This is useful for calling printk functions from within NMI
- * without worrying about race conditions that can lock up the
- * box.
- */
-DEFINE_PER_CPU(printk_func_t, printk_func) = vprintk_default;
-
/**
* printk - print a kernel message
* @fmt: format string
@@ -1838,21 +1841,11 @@
*/
asmlinkage __visible int printk(const char *fmt, ...)
{
- printk_func_t vprintk_func;
va_list args;
int r;
va_start(args, fmt);
-
- /*
- * If a caller overrides the per_cpu printk_func, then it needs
- * to disable preemption when calling printk(). Otherwise
- * the printk_func should be set to the default. No need to
- * disable preemption here.
- */
- vprintk_func = this_cpu_read(printk_func);
r = vprintk_func(fmt, args);
-
va_end(args);
return r;
diff --git a/kernel/rcu/update.c b/kernel/rcu/update.c
index 3ccdc8e..3e888cd 100644
--- a/kernel/rcu/update.c
+++ b/kernel/rcu/update.c
@@ -380,29 +380,9 @@
debug_object_free(head, &rcuhead_debug_descr);
}
-/*
- * fixup_activate is called when:
- * - an active object is activated
- * - an unknown object is activated (might be a statically initialized object)
- * Activation is performed internally by call_rcu().
- */
-static int rcuhead_fixup_activate(void *addr, enum debug_obj_state state)
+static bool rcuhead_is_static_object(void *addr)
{
- struct rcu_head *head = addr;
-
- switch (state) {
-
- case ODEBUG_STATE_NOTAVAILABLE:
- /*
- * This is not really a fixup. We just make sure that it is
- * tracked in the object tracker.
- */
- debug_object_init(head, &rcuhead_debug_descr);
- debug_object_activate(head, &rcuhead_debug_descr);
- return 0;
- default:
- return 1;
- }
+ return true;
}
/**
@@ -440,7 +420,7 @@
struct debug_obj_descr rcuhead_debug_descr = {
.name = "rcu_head",
- .fixup_activate = rcuhead_fixup_activate,
+ .is_static_object = rcuhead_is_static_object,
};
EXPORT_SYMBOL_GPL(rcuhead_debug_descr);
#endif /* #ifdef CONFIG_DEBUG_OBJECTS_RCU_HEAD */
diff --git a/kernel/seccomp.c b/kernel/seccomp.c
index e1e5a35..7002796 100644
--- a/kernel/seccomp.c
+++ b/kernel/seccomp.c
@@ -513,24 +513,17 @@
* To be fully secure this must be combined with rlimit
* to limit the stack allocations too.
*/
-static int mode1_syscalls[] = {
+static const int mode1_syscalls[] = {
__NR_seccomp_read, __NR_seccomp_write, __NR_seccomp_exit, __NR_seccomp_sigreturn,
0, /* null terminated */
};
-#ifdef CONFIG_COMPAT
-static int mode1_syscalls_32[] = {
- __NR_seccomp_read_32, __NR_seccomp_write_32, __NR_seccomp_exit_32, __NR_seccomp_sigreturn_32,
- 0, /* null terminated */
-};
-#endif
-
static void __secure_computing_strict(int this_syscall)
{
- int *syscall_whitelist = mode1_syscalls;
+ const int *syscall_whitelist = mode1_syscalls;
#ifdef CONFIG_COMPAT
if (in_compat_syscall())
- syscall_whitelist = mode1_syscalls_32;
+ syscall_whitelist = get_compat_mode1_syscalls();
#endif
do {
if (*syscall_whitelist == this_syscall)
@@ -915,7 +908,7 @@
fprog = filter->prog->orig_prog;
if (!fprog) {
- /* This must be a new non-cBPF filter, since we save every
+ /* This must be a new non-cBPF filter, since we save
* every cBPF filter's orig_prog above when
* CONFIG_CHECKPOINT_RESTORE is enabled.
*/
diff --git a/kernel/signal.c b/kernel/signal.c
index ab122a2..96e9bc4 100644
--- a/kernel/signal.c
+++ b/kernel/signal.c
@@ -224,7 +224,7 @@
if (!__ratelimit(&ratelimit_state))
return;
- printk(KERN_INFO "%s/%d: reached RLIMIT_SIGPENDING, dropped signal %d\n",
+ pr_info("%s/%d: reached RLIMIT_SIGPENDING, dropped signal %d\n",
current->comm, current->pid, sig);
}
@@ -1089,10 +1089,10 @@
static void print_fatal_signal(int signr)
{
struct pt_regs *regs = signal_pt_regs();
- printk(KERN_INFO "potentially unexpected fatal signal %d.\n", signr);
+ pr_info("potentially unexpected fatal signal %d.\n", signr);
#if defined(__i386__) && !defined(__arch_um__)
- printk(KERN_INFO "code at %08lx: ", regs->ip);
+ pr_info("code at %08lx: ", regs->ip);
{
int i;
for (i = 0; i < 16; i++) {
@@ -1100,10 +1100,10 @@
if (get_user(insn, (unsigned char *)(regs->ip + i)))
break;
- printk(KERN_CONT "%02x ", insn);
+ pr_cont("%02x ", insn);
}
}
- printk(KERN_CONT "\n");
+ pr_cont("\n");
#endif
preempt_disable();
show_regs(regs);
diff --git a/kernel/sys.c b/kernel/sys.c
index cf8ba54..89d5be4 100644
--- a/kernel/sys.c
+++ b/kernel/sys.c
@@ -2246,7 +2246,8 @@
case PR_SET_THP_DISABLE:
if (arg3 || arg4 || arg5)
return -EINVAL;
- down_write(&me->mm->mmap_sem);
+ if (down_write_killable(&me->mm->mmap_sem))
+ return -EINTR;
if (arg2)
me->mm->def_flags |= VM_NOHUGEPAGE;
else
diff --git a/kernel/sysctl.c b/kernel/sysctl.c
index c8b3186..2effd84 100644
--- a/kernel/sysctl.c
+++ b/kernel/sysctl.c
@@ -1521,6 +1521,13 @@
.mode = 0644,
.proc_handler = proc_dointvec_jiffies,
},
+ {
+ .procname = "stat_refresh",
+ .data = NULL,
+ .maxlen = 0,
+ .mode = 0600,
+ .proc_handler = vmstat_refresh,
+ },
#endif
#ifdef CONFIG_MMU
{
diff --git a/kernel/sysctl_binary.c b/kernel/sysctl_binary.c
index 10a1d7d..6eb99c1 100644
--- a/kernel/sysctl_binary.c
+++ b/kernel/sysctl_binary.c
@@ -13,6 +13,7 @@
#include <linux/ctype.h>
#include <linux/netdevice.h>
#include <linux/kernel.h>
+#include <linux/uuid.h>
#include <linux/slab.h>
#include <linux/compat.h>
@@ -1117,9 +1118,8 @@
/* Only supports reads */
if (oldval && oldlen) {
- char buf[40], *str = buf;
- unsigned char uuid[16];
- int i;
+ char buf[UUID_STRING_LEN + 1];
+ uuid_be uuid;
result = kernel_read(file, 0, buf, sizeof(buf) - 1);
if (result < 0)
@@ -1127,24 +1127,15 @@
buf[result] = '\0';
- /* Convert the uuid to from a string to binary */
- for (i = 0; i < 16; i++) {
- result = -EIO;
- if (!isxdigit(str[0]) || !isxdigit(str[1]))
- goto out;
-
- uuid[i] = (hex_to_bin(str[0]) << 4) |
- hex_to_bin(str[1]);
- str += 2;
- if (*str == '-')
- str++;
- }
+ result = -EIO;
+ if (uuid_be_to_bin(buf, &uuid))
+ goto out;
if (oldlen > 16)
oldlen = 16;
result = -EFAULT;
- if (copy_to_user(oldval, uuid, oldlen))
+ if (copy_to_user(oldval, &uuid, oldlen))
goto out;
copied = oldlen;
diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index fa0b983..8c7392c 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -334,7 +334,7 @@
* fixup_init is called when:
* - an active object is initialized
*/
-static int hrtimer_fixup_init(void *addr, enum debug_obj_state state)
+static bool hrtimer_fixup_init(void *addr, enum debug_obj_state state)
{
struct hrtimer *timer = addr;
@@ -342,30 +342,25 @@
case ODEBUG_STATE_ACTIVE:
hrtimer_cancel(timer);
debug_object_init(timer, &hrtimer_debug_descr);
- return 1;
+ return true;
default:
- return 0;
+ return false;
}
}
/*
* fixup_activate is called when:
* - an active object is activated
- * - an unknown object is activated (might be a statically initialized object)
+ * - an unknown non-static object is activated
*/
-static int hrtimer_fixup_activate(void *addr, enum debug_obj_state state)
+static bool hrtimer_fixup_activate(void *addr, enum debug_obj_state state)
{
switch (state) {
-
- case ODEBUG_STATE_NOTAVAILABLE:
- WARN_ON_ONCE(1);
- return 0;
-
case ODEBUG_STATE_ACTIVE:
WARN_ON(1);
default:
- return 0;
+ return false;
}
}
@@ -373,7 +368,7 @@
* fixup_free is called when:
* - an active object is freed
*/
-static int hrtimer_fixup_free(void *addr, enum debug_obj_state state)
+static bool hrtimer_fixup_free(void *addr, enum debug_obj_state state)
{
struct hrtimer *timer = addr;
@@ -381,9 +376,9 @@
case ODEBUG_STATE_ACTIVE:
hrtimer_cancel(timer);
debug_object_free(timer, &hrtimer_debug_descr);
- return 1;
+ return true;
default:
- return 0;
+ return false;
}
}
diff --git a/kernel/time/time.c b/kernel/time/time.c
index a4064b6..667b9335 100644
--- a/kernel/time/time.c
+++ b/kernel/time/time.c
@@ -769,3 +769,24 @@
return res;
}
+
+/*
+ * Add two timespec64 values and do a safety check for overflow.
+ * It's assumed that both values are valid (>= 0).
+ * And, each timespec64 is in normalized form.
+ */
+struct timespec64 timespec64_add_safe(const struct timespec64 lhs,
+ const struct timespec64 rhs)
+{
+ struct timespec64 res;
+
+ set_normalized_timespec64(&res, lhs.tv_sec + rhs.tv_sec,
+ lhs.tv_nsec + rhs.tv_nsec);
+
+ if (unlikely(res.tv_sec < lhs.tv_sec || res.tv_sec < rhs.tv_sec)) {
+ res.tv_sec = TIME64_MAX;
+ res.tv_nsec = 0;
+ }
+
+ return res;
+}
diff --git a/kernel/time/timer.c b/kernel/time/timer.c
index 73164c3..3a95f97 100644
--- a/kernel/time/timer.c
+++ b/kernel/time/timer.c
@@ -489,11 +489,19 @@
return ((struct timer_list *) addr)->function;
}
+static bool timer_is_static_object(void *addr)
+{
+ struct timer_list *timer = addr;
+
+ return (timer->entry.pprev == NULL &&
+ timer->entry.next == TIMER_ENTRY_STATIC);
+}
+
/*
* fixup_init is called when:
* - an active object is initialized
*/
-static int timer_fixup_init(void *addr, enum debug_obj_state state)
+static bool timer_fixup_init(void *addr, enum debug_obj_state state)
{
struct timer_list *timer = addr;
@@ -501,9 +509,9 @@
case ODEBUG_STATE_ACTIVE:
del_timer_sync(timer);
debug_object_init(timer, &timer_debug_descr);
- return 1;
+ return true;
default:
- return 0;
+ return false;
}
}
@@ -516,36 +524,22 @@
/*
* fixup_activate is called when:
* - an active object is activated
- * - an unknown object is activated (might be a statically initialized object)
+ * - an unknown non-static object is activated
*/
-static int timer_fixup_activate(void *addr, enum debug_obj_state state)
+static bool timer_fixup_activate(void *addr, enum debug_obj_state state)
{
struct timer_list *timer = addr;
switch (state) {
-
case ODEBUG_STATE_NOTAVAILABLE:
- /*
- * This is not really a fixup. The timer was
- * statically initialized. We just make sure that it
- * is tracked in the object tracker.
- */
- if (timer->entry.pprev == NULL &&
- timer->entry.next == TIMER_ENTRY_STATIC) {
- debug_object_init(timer, &timer_debug_descr);
- debug_object_activate(timer, &timer_debug_descr);
- return 0;
- } else {
- setup_timer(timer, stub_timer, 0);
- return 1;
- }
- return 0;
+ setup_timer(timer, stub_timer, 0);
+ return true;
case ODEBUG_STATE_ACTIVE:
WARN_ON(1);
default:
- return 0;
+ return false;
}
}
@@ -553,7 +547,7 @@
* fixup_free is called when:
* - an active object is freed
*/
-static int timer_fixup_free(void *addr, enum debug_obj_state state)
+static bool timer_fixup_free(void *addr, enum debug_obj_state state)
{
struct timer_list *timer = addr;
@@ -561,9 +555,9 @@
case ODEBUG_STATE_ACTIVE:
del_timer_sync(timer);
debug_object_free(timer, &timer_debug_descr);
- return 1;
+ return true;
default:
- return 0;
+ return false;
}
}
@@ -571,32 +565,23 @@
* fixup_assert_init is called when:
* - an untracked/uninit-ed object is found
*/
-static int timer_fixup_assert_init(void *addr, enum debug_obj_state state)
+static bool timer_fixup_assert_init(void *addr, enum debug_obj_state state)
{
struct timer_list *timer = addr;
switch (state) {
case ODEBUG_STATE_NOTAVAILABLE:
- if (timer->entry.next == TIMER_ENTRY_STATIC) {
- /*
- * This is not really a fixup. The timer was
- * statically initialized. We just make sure that it
- * is tracked in the object tracker.
- */
- debug_object_init(timer, &timer_debug_descr);
- return 0;
- } else {
- setup_timer(timer, stub_timer, 0);
- return 1;
- }
+ setup_timer(timer, stub_timer, 0);
+ return true;
default:
- return 0;
+ return false;
}
}
static struct debug_obj_descr timer_debug_descr = {
.name = "timer_list",
.debug_hint = timer_debug_hint,
+ .is_static_object = timer_is_static_object,
.fixup_init = timer_fixup_init,
.fixup_activate = timer_fixup_activate,
.fixup_free = timer_fixup_free,
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index e45db6b..fafeaf8 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -528,6 +528,32 @@
See Documentation/trace/mmiotrace.txt.
If you are not helping to develop drivers, say N.
+config TRACING_MAP
+ bool
+ depends on ARCH_HAVE_NMI_SAFE_CMPXCHG
+ help
+ tracing_map is a special-purpose lock-free map for tracing,
+ separated out as a stand-alone facility in order to allow it
+ to be shared between multiple tracers. It isn't meant to be
+ generally used outside of that context, and is normally
+ selected by tracers that use it.
+
+config HIST_TRIGGERS
+ bool "Histogram triggers"
+ depends on ARCH_HAVE_NMI_SAFE_CMPXCHG
+ select TRACING_MAP
+ default n
+ help
+ Hist triggers allow one or more arbitrary trace event fields
+ to be aggregated into hash tables and dumped to stdout by
+ reading a debugfs/tracefs file. They're useful for
+ gathering quick and dirty (though precise) summaries of
+ event activity as an initial guide for further investigation
+ using more advanced tools.
+
+ See Documentation/trace/events.txt.
+ If in doubt, say N.
+
config MMIOTRACE_TEST
tristate "Test module for mmiotrace"
depends on MMIOTRACE && m
diff --git a/kernel/trace/Makefile b/kernel/trace/Makefile
index 9b1044e..979e7bf 100644
--- a/kernel/trace/Makefile
+++ b/kernel/trace/Makefile
@@ -31,6 +31,7 @@
obj-$(CONFIG_TRACING) += trace_seq.o
obj-$(CONFIG_TRACING) += trace_stat.o
obj-$(CONFIG_TRACING) += trace_printk.o
+obj-$(CONFIG_TRACING_MAP) += tracing_map.o
obj-$(CONFIG_CONTEXT_SWITCH_TRACER) += trace_sched_switch.o
obj-$(CONFIG_FUNCTION_TRACER) += trace_functions.o
obj-$(CONFIG_IRQSOFF_TRACER) += trace_irqsoff.o
@@ -53,6 +54,7 @@
endif
obj-$(CONFIG_EVENT_TRACING) += trace_events_filter.o
obj-$(CONFIG_EVENT_TRACING) += trace_events_trigger.o
+obj-$(CONFIG_HIST_TRIGGERS) += trace_events_hist.o
obj-$(CONFIG_BPF_EVENTS) += bpf_trace.o
obj-$(CONFIG_KPROBE_EVENT) += trace_kprobe.o
obj-$(CONFIG_TRACEPOINTS) += power-traces.o
diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
index 7e8d792..900dbb1 100644
--- a/kernel/trace/ftrace.c
+++ b/kernel/trace/ftrace.c
@@ -3456,11 +3456,23 @@
int type;
};
+/*
+ * If symbols in an architecture don't correspond exactly to the user-visible
+ * name of what they represent, it is possible to define this function to
+ * perform the necessary adjustments.
+*/
+char * __weak arch_ftrace_match_adjust(char *str, const char *search)
+{
+ return str;
+}
+
static int ftrace_match(char *str, struct ftrace_glob *g)
{
int matched = 0;
int slen;
+ str = arch_ftrace_match_adjust(str, g->search);
+
switch (g->type) {
case MATCH_FULL:
if (strcmp(str, g->search) == 0)
@@ -5725,7 +5737,6 @@
{
int i;
int ret = 0;
- unsigned long flags;
int start = 0, end = FTRACE_RETSTACK_ALLOC_SIZE;
struct task_struct *g, *t;
@@ -5741,7 +5752,7 @@
}
}
- read_lock_irqsave(&tasklist_lock, flags);
+ read_lock(&tasklist_lock);
do_each_thread(g, t) {
if (start == end) {
ret = -EAGAIN;
@@ -5759,7 +5770,7 @@
} while_each_thread(g, t);
unlock:
- read_unlock_irqrestore(&tasklist_lock, flags);
+ read_unlock(&tasklist_lock);
free:
for (i = start; i < end; i++)
kfree(ret_stack_list[i]);
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index a2f0b9f..8a4bd6b 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -253,6 +253,9 @@
#define TOP_LEVEL_TRACE_FLAGS (TRACE_ITER_PRINTK | \
TRACE_ITER_PRINTK_MSGONLY | TRACE_ITER_RECORD_CMD)
+/* trace_flags that are default zero for instances */
+#define ZEROED_TRACE_FLAGS \
+ TRACE_ITER_EVENT_FORK
/*
* The global_trace is the descriptor that holds the tracing
@@ -303,33 +306,18 @@
mutex_unlock(&trace_types_lock);
}
-int filter_check_discard(struct trace_event_file *file, void *rec,
- struct ring_buffer *buffer,
- struct ring_buffer_event *event)
-{
- if (unlikely(file->flags & EVENT_FILE_FL_FILTERED) &&
- !filter_match_preds(file->filter, rec)) {
- ring_buffer_discard_commit(buffer, event);
- return 1;
- }
-
- return 0;
-}
-EXPORT_SYMBOL_GPL(filter_check_discard);
-
int call_filter_check_discard(struct trace_event_call *call, void *rec,
struct ring_buffer *buffer,
struct ring_buffer_event *event)
{
if (unlikely(call->flags & TRACE_EVENT_FL_FILTERED) &&
!filter_match_preds(call->filter, rec)) {
- ring_buffer_discard_commit(buffer, event);
+ __trace_event_discard_commit(buffer, event);
return 1;
}
return 0;
}
-EXPORT_SYMBOL_GPL(call_filter_check_discard);
static cycle_t buffer_ftrace_now(struct trace_buffer *buf, int cpu)
{
@@ -1672,6 +1660,16 @@
}
EXPORT_SYMBOL_GPL(tracing_generic_entry_update);
+static __always_inline void
+trace_event_setup(struct ring_buffer_event *event,
+ int type, unsigned long flags, int pc)
+{
+ struct trace_entry *ent = ring_buffer_event_data(event);
+
+ tracing_generic_entry_update(ent, flags, pc);
+ ent->type = type;
+}
+
struct ring_buffer_event *
trace_buffer_lock_reserve(struct ring_buffer *buffer,
int type,
@@ -1681,34 +1679,137 @@
struct ring_buffer_event *event;
event = ring_buffer_lock_reserve(buffer, len);
- if (event != NULL) {
- struct trace_entry *ent = ring_buffer_event_data(event);
-
- tracing_generic_entry_update(ent, flags, pc);
- ent->type = type;
- }
+ if (event != NULL)
+ trace_event_setup(event, type, flags, pc);
return event;
}
+DEFINE_PER_CPU(struct ring_buffer_event *, trace_buffered_event);
+DEFINE_PER_CPU(int, trace_buffered_event_cnt);
+static int trace_buffered_event_ref;
+
+/**
+ * trace_buffered_event_enable - enable buffering events
+ *
+ * When events are being filtered, it is quicker to use a temporary
+ * buffer to write the event data into if there's a likely chance
+ * that it will not be committed. The discard of the ring buffer
+ * is not as fast as committing, and is much slower than copying
+ * a commit.
+ *
+ * When an event is to be filtered, allocate per cpu buffers to
+ * write the event data into, and if the event is filtered and discarded
+ * it is simply dropped, otherwise, the entire data is to be committed
+ * in one shot.
+ */
+void trace_buffered_event_enable(void)
+{
+ struct ring_buffer_event *event;
+ struct page *page;
+ int cpu;
+
+ WARN_ON_ONCE(!mutex_is_locked(&event_mutex));
+
+ if (trace_buffered_event_ref++)
+ return;
+
+ for_each_tracing_cpu(cpu) {
+ page = alloc_pages_node(cpu_to_node(cpu),
+ GFP_KERNEL | __GFP_NORETRY, 0);
+ if (!page)
+ goto failed;
+
+ event = page_address(page);
+ memset(event, 0, sizeof(*event));
+
+ per_cpu(trace_buffered_event, cpu) = event;
+
+ preempt_disable();
+ if (cpu == smp_processor_id() &&
+ this_cpu_read(trace_buffered_event) !=
+ per_cpu(trace_buffered_event, cpu))
+ WARN_ON_ONCE(1);
+ preempt_enable();
+ }
+
+ return;
+ failed:
+ trace_buffered_event_disable();
+}
+
+static void enable_trace_buffered_event(void *data)
+{
+ /* Probably not needed, but do it anyway */
+ smp_rmb();
+ this_cpu_dec(trace_buffered_event_cnt);
+}
+
+static void disable_trace_buffered_event(void *data)
+{
+ this_cpu_inc(trace_buffered_event_cnt);
+}
+
+/**
+ * trace_buffered_event_disable - disable buffering events
+ *
+ * When a filter is removed, it is faster to not use the buffered
+ * events, and to commit directly into the ring buffer. Free up
+ * the temp buffers when there are no more users. This requires
+ * special synchronization with current events.
+ */
+void trace_buffered_event_disable(void)
+{
+ int cpu;
+
+ WARN_ON_ONCE(!mutex_is_locked(&event_mutex));
+
+ if (WARN_ON_ONCE(!trace_buffered_event_ref))
+ return;
+
+ if (--trace_buffered_event_ref)
+ return;
+
+ preempt_disable();
+ /* For each CPU, set the buffer as used. */
+ smp_call_function_many(tracing_buffer_mask,
+ disable_trace_buffered_event, NULL, 1);
+ preempt_enable();
+
+ /* Wait for all current users to finish */
+ synchronize_sched();
+
+ for_each_tracing_cpu(cpu) {
+ free_page((unsigned long)per_cpu(trace_buffered_event, cpu));
+ per_cpu(trace_buffered_event, cpu) = NULL;
+ }
+ /*
+ * Make sure trace_buffered_event is NULL before clearing
+ * trace_buffered_event_cnt.
+ */
+ smp_wmb();
+
+ preempt_disable();
+ /* Do the work on each cpu */
+ smp_call_function_many(tracing_buffer_mask,
+ enable_trace_buffered_event, NULL, 1);
+ preempt_enable();
+}
+
void
__buffer_unlock_commit(struct ring_buffer *buffer, struct ring_buffer_event *event)
{
__this_cpu_write(trace_cmdline_save, true);
- ring_buffer_unlock_commit(buffer, event);
-}
-void trace_buffer_unlock_commit(struct trace_array *tr,
- struct ring_buffer *buffer,
- struct ring_buffer_event *event,
- unsigned long flags, int pc)
-{
- __buffer_unlock_commit(buffer, event);
-
- ftrace_trace_stack(tr, buffer, flags, 6, pc, NULL);
- ftrace_trace_userstack(buffer, flags, pc);
+ /* If this is the temp buffer, we need to commit fully */
+ if (this_cpu_read(trace_buffered_event) == event) {
+ /* Length is in event->array[0] */
+ ring_buffer_write(buffer, event->array[0], &event->array[1]);
+ /* Release the temp buffer */
+ this_cpu_dec(trace_buffered_event_cnt);
+ } else
+ ring_buffer_unlock_commit(buffer, event);
}
-EXPORT_SYMBOL_GPL(trace_buffer_unlock_commit);
static struct ring_buffer *temp_buffer;
@@ -1719,8 +1820,23 @@
unsigned long flags, int pc)
{
struct ring_buffer_event *entry;
+ int val;
*current_rb = trace_file->tr->trace_buffer.buffer;
+
+ if ((trace_file->flags &
+ (EVENT_FILE_FL_SOFT_DISABLED | EVENT_FILE_FL_FILTERED)) &&
+ (entry = this_cpu_read(trace_buffered_event))) {
+ /* Try to use the per cpu buffer first */
+ val = this_cpu_inc_return(trace_buffered_event_cnt);
+ if (val == 1) {
+ trace_event_setup(entry, type, flags, pc);
+ entry->array[0] = len;
+ return entry;
+ }
+ this_cpu_dec(trace_buffered_event_cnt);
+ }
+
entry = trace_buffer_lock_reserve(*current_rb,
type, len, flags, pc);
/*
@@ -1738,17 +1854,6 @@
}
EXPORT_SYMBOL_GPL(trace_event_buffer_lock_reserve);
-struct ring_buffer_event *
-trace_current_buffer_lock_reserve(struct ring_buffer **current_rb,
- int type, unsigned long len,
- unsigned long flags, int pc)
-{
- *current_rb = global_trace.trace_buffer.buffer;
- return trace_buffer_lock_reserve(*current_rb,
- type, len, flags, pc);
-}
-EXPORT_SYMBOL_GPL(trace_current_buffer_lock_reserve);
-
void trace_buffer_unlock_commit_regs(struct trace_array *tr,
struct ring_buffer *buffer,
struct ring_buffer_event *event,
@@ -1760,14 +1865,6 @@
ftrace_trace_stack(tr, buffer, flags, 0, pc, regs);
ftrace_trace_userstack(buffer, flags, pc);
}
-EXPORT_SYMBOL_GPL(trace_buffer_unlock_commit_regs);
-
-void trace_current_buffer_discard_commit(struct ring_buffer *buffer,
- struct ring_buffer_event *event)
-{
- ring_buffer_discard_commit(buffer, event);
-}
-EXPORT_SYMBOL_GPL(trace_current_buffer_discard_commit);
void
trace_function(struct trace_array *tr,
@@ -3571,6 +3668,9 @@
if (mask == TRACE_ITER_RECORD_CMD)
trace_event_enable_cmd_record(enabled);
+ if (mask == TRACE_ITER_EVENT_FORK)
+ trace_event_follow_fork(tr, enabled);
+
if (mask == TRACE_ITER_OVERWRITE) {
ring_buffer_change_overwrite(tr->trace_buffer.buffer, enabled);
#ifdef CONFIG_TRACER_MAX_TRACE
@@ -3658,7 +3758,7 @@
if (cnt >= sizeof(buf))
return -EINVAL;
- if (copy_from_user(&buf, ubuf, cnt))
+ if (copy_from_user(buf, ubuf, cnt))
return -EFAULT;
buf[cnt] = 0;
@@ -3804,12 +3904,19 @@
"\t trigger: traceon, traceoff\n"
"\t enable_event:<system>:<event>\n"
"\t disable_event:<system>:<event>\n"
+#ifdef CONFIG_HIST_TRIGGERS
+ "\t enable_hist:<system>:<event>\n"
+ "\t disable_hist:<system>:<event>\n"
+#endif
#ifdef CONFIG_STACKTRACE
"\t\t stacktrace\n"
#endif
#ifdef CONFIG_TRACER_SNAPSHOT
"\t\t snapshot\n"
#endif
+#ifdef CONFIG_HIST_TRIGGERS
+ "\t\t hist (see below)\n"
+#endif
"\t example: echo traceoff > events/block/block_unplug/trigger\n"
"\t echo traceoff:3 > events/block/block_unplug/trigger\n"
"\t echo 'enable_event:kmem:kmalloc:3 if nr_rq > 1' > \\\n"
@@ -3825,6 +3932,56 @@
"\t To remove a trigger with a count:\n"
"\t echo '!<trigger>:0 > <system>/<event>/trigger\n"
"\t Filters can be ignored when removing a trigger.\n"
+#ifdef CONFIG_HIST_TRIGGERS
+ " hist trigger\t- If set, event hits are aggregated into a hash table\n"
+ "\t Format: hist:keys=<field1[,field2,...]>\n"
+ "\t [:values=<field1[,field2,...]>]\n"
+ "\t [:sort=<field1[,field2,...]>]\n"
+ "\t [:size=#entries]\n"
+ "\t [:pause][:continue][:clear]\n"
+ "\t [:name=histname1]\n"
+ "\t [if <filter>]\n\n"
+ "\t When a matching event is hit, an entry is added to a hash\n"
+ "\t table using the key(s) and value(s) named, and the value of a\n"
+ "\t sum called 'hitcount' is incremented. Keys and values\n"
+ "\t correspond to fields in the event's format description. Keys\n"
+ "\t can be any field, or the special string 'stacktrace'.\n"
+ "\t Compound keys consisting of up to two fields can be specified\n"
+ "\t by the 'keys' keyword. Values must correspond to numeric\n"
+ "\t fields. Sort keys consisting of up to two fields can be\n"
+ "\t specified using the 'sort' keyword. The sort direction can\n"
+ "\t be modified by appending '.descending' or '.ascending' to a\n"
+ "\t sort field. The 'size' parameter can be used to specify more\n"
+ "\t or fewer than the default 2048 entries for the hashtable size.\n"
+ "\t If a hist trigger is given a name using the 'name' parameter,\n"
+ "\t its histogram data will be shared with other triggers of the\n"
+ "\t same name, and trigger hits will update this common data.\n\n"
+ "\t Reading the 'hist' file for the event will dump the hash\n"
+ "\t table in its entirety to stdout. If there are multiple hist\n"
+ "\t triggers attached to an event, there will be a table for each\n"
+ "\t trigger in the output. The table displayed for a named\n"
+ "\t trigger will be the same as any other instance having the\n"
+ "\t same name. The default format used to display a given field\n"
+ "\t can be modified by appending any of the following modifiers\n"
+ "\t to the field name, as applicable:\n\n"
+ "\t .hex display a number as a hex value\n"
+ "\t .sym display an address as a symbol\n"
+ "\t .sym-offset display an address as a symbol and offset\n"
+ "\t .execname display a common_pid as a program name\n"
+ "\t .syscall display a syscall id as a syscall name\n\n"
+ "\t .log2 display log2 value rather than raw number\n\n"
+ "\t The 'pause' parameter can be used to pause an existing hist\n"
+ "\t trigger or to start a hist trigger but not log any events\n"
+ "\t until told to do so. 'continue' can be used to start or\n"
+ "\t restart a paused hist trigger.\n\n"
+ "\t The 'clear' parameter will clear the contents of a running\n"
+ "\t hist trigger and leave its current paused/active state\n"
+ "\t unchanged.\n\n"
+ "\t The enable_hist and disable_hist triggers can be used to\n"
+ "\t have one event conditionally start and stop another event's\n"
+ "\t already-attached hist trigger. The syntax is analagous to\n"
+ "\t the enable_event and disable_event triggers.\n"
+#endif
;
static ssize_t
@@ -4474,7 +4631,7 @@
if (cnt > MAX_TRACER_SIZE)
cnt = MAX_TRACER_SIZE;
- if (copy_from_user(&buf, ubuf, cnt))
+ if (copy_from_user(buf, ubuf, cnt))
return -EFAULT;
buf[cnt] = 0;
@@ -5264,7 +5421,7 @@
if (cnt >= sizeof(buf))
return -EINVAL;
- if (copy_from_user(&buf, ubuf, cnt))
+ if (copy_from_user(buf, ubuf, cnt))
return -EFAULT;
buf[cnt] = 0;
@@ -6650,7 +6807,7 @@
if (!alloc_cpumask_var(&tr->tracing_cpumask, GFP_KERNEL))
goto out_free_tr;
- tr->trace_flags = global_trace.trace_flags;
+ tr->trace_flags = global_trace.trace_flags & ~ZEROED_TRACE_FLAGS;
cpumask_copy(tr->tracing_cpumask, cpu_all_mask);
@@ -6724,6 +6881,12 @@
list_del(&tr->list);
+ /* Disable all the flags that were enabled coming in */
+ for (i = 0; i < TRACE_FLAGS_MAX_SIZE; i++) {
+ if ((1 << i) & ZEROED_TRACE_FLAGS)
+ set_tracer_flag(tr, 1 << i, 0);
+ }
+
tracing_set_nop(tr);
event_trace_del_tracer(tr);
ftrace_destroy_function_files(tr);
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index 3fff4ad..5167c36 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -177,9 +177,8 @@
};
struct trace_pid_list {
- unsigned int nr_pids;
- int order;
- pid_t *pids;
+ int pid_max;
+ unsigned long *pids;
};
/*
@@ -656,6 +655,7 @@
extern cycle_t ftrace_now(int cpu);
extern void trace_find_cmdline(int pid, char comm[]);
+extern void trace_event_follow_fork(struct trace_array *tr, bool enable);
#ifdef CONFIG_DYNAMIC_FTRACE
extern unsigned long ftrace_update_tot_cnt;
@@ -967,6 +967,7 @@
C(STOP_ON_FREE, "disable_on_free"), \
C(IRQ_INFO, "irq-info"), \
C(MARKERS, "markers"), \
+ C(EVENT_FORK, "event-fork"), \
FUNCTION_FLAGS \
FGRAPH_FLAGS \
STACK_FLAGS \
@@ -1064,6 +1065,137 @@
int nr_events;
};
+extern int call_filter_check_discard(struct trace_event_call *call, void *rec,
+ struct ring_buffer *buffer,
+ struct ring_buffer_event *event);
+
+void trace_buffer_unlock_commit_regs(struct trace_array *tr,
+ struct ring_buffer *buffer,
+ struct ring_buffer_event *event,
+ unsigned long flags, int pc,
+ struct pt_regs *regs);
+
+static inline void trace_buffer_unlock_commit(struct trace_array *tr,
+ struct ring_buffer *buffer,
+ struct ring_buffer_event *event,
+ unsigned long flags, int pc)
+{
+ trace_buffer_unlock_commit_regs(tr, buffer, event, flags, pc, NULL);
+}
+
+DECLARE_PER_CPU(struct ring_buffer_event *, trace_buffered_event);
+DECLARE_PER_CPU(int, trace_buffered_event_cnt);
+void trace_buffered_event_disable(void);
+void trace_buffered_event_enable(void);
+
+static inline void
+__trace_event_discard_commit(struct ring_buffer *buffer,
+ struct ring_buffer_event *event)
+{
+ if (this_cpu_read(trace_buffered_event) == event) {
+ /* Simply release the temp buffer */
+ this_cpu_dec(trace_buffered_event_cnt);
+ return;
+ }
+ ring_buffer_discard_commit(buffer, event);
+}
+
+/*
+ * Helper function for event_trigger_unlock_commit{_regs}().
+ * If there are event triggers attached to this event that requires
+ * filtering against its fields, then they wil be called as the
+ * entry already holds the field information of the current event.
+ *
+ * It also checks if the event should be discarded or not.
+ * It is to be discarded if the event is soft disabled and the
+ * event was only recorded to process triggers, or if the event
+ * filter is active and this event did not match the filters.
+ *
+ * Returns true if the event is discarded, false otherwise.
+ */
+static inline bool
+__event_trigger_test_discard(struct trace_event_file *file,
+ struct ring_buffer *buffer,
+ struct ring_buffer_event *event,
+ void *entry,
+ enum event_trigger_type *tt)
+{
+ unsigned long eflags = file->flags;
+
+ if (eflags & EVENT_FILE_FL_TRIGGER_COND)
+ *tt = event_triggers_call(file, entry);
+
+ if (test_bit(EVENT_FILE_FL_SOFT_DISABLED_BIT, &file->flags) ||
+ (unlikely(file->flags & EVENT_FILE_FL_FILTERED) &&
+ !filter_match_preds(file->filter, entry))) {
+ __trace_event_discard_commit(buffer, event);
+ return true;
+ }
+
+ return false;
+}
+
+/**
+ * event_trigger_unlock_commit - handle triggers and finish event commit
+ * @file: The file pointer assoctiated to the event
+ * @buffer: The ring buffer that the event is being written to
+ * @event: The event meta data in the ring buffer
+ * @entry: The event itself
+ * @irq_flags: The state of the interrupts at the start of the event
+ * @pc: The state of the preempt count at the start of the event.
+ *
+ * This is a helper function to handle triggers that require data
+ * from the event itself. It also tests the event against filters and
+ * if the event is soft disabled and should be discarded.
+ */
+static inline void
+event_trigger_unlock_commit(struct trace_event_file *file,
+ struct ring_buffer *buffer,
+ struct ring_buffer_event *event,
+ void *entry, unsigned long irq_flags, int pc)
+{
+ enum event_trigger_type tt = ETT_NONE;
+
+ if (!__event_trigger_test_discard(file, buffer, event, entry, &tt))
+ trace_buffer_unlock_commit(file->tr, buffer, event, irq_flags, pc);
+
+ if (tt)
+ event_triggers_post_call(file, tt, entry);
+}
+
+/**
+ * event_trigger_unlock_commit_regs - handle triggers and finish event commit
+ * @file: The file pointer assoctiated to the event
+ * @buffer: The ring buffer that the event is being written to
+ * @event: The event meta data in the ring buffer
+ * @entry: The event itself
+ * @irq_flags: The state of the interrupts at the start of the event
+ * @pc: The state of the preempt count at the start of the event.
+ *
+ * This is a helper function to handle triggers that require data
+ * from the event itself. It also tests the event against filters and
+ * if the event is soft disabled and should be discarded.
+ *
+ * Same as event_trigger_unlock_commit() but calls
+ * trace_buffer_unlock_commit_regs() instead of trace_buffer_unlock_commit().
+ */
+static inline void
+event_trigger_unlock_commit_regs(struct trace_event_file *file,
+ struct ring_buffer *buffer,
+ struct ring_buffer_event *event,
+ void *entry, unsigned long irq_flags, int pc,
+ struct pt_regs *regs)
+{
+ enum event_trigger_type tt = ETT_NONE;
+
+ if (!__event_trigger_test_discard(file, buffer, event, entry, &tt))
+ trace_buffer_unlock_commit_regs(file->tr, buffer, event,
+ irq_flags, pc, regs);
+
+ if (tt)
+ event_triggers_post_call(file, tt, entry);
+}
+
#define FILTER_PRED_INVALID ((unsigned short)-1)
#define FILTER_PRED_IS_RIGHT (1 << 15)
#define FILTER_PRED_FOLD (1 << 15)
@@ -1161,6 +1293,15 @@
extern struct list_head ftrace_events;
extern const struct file_operations event_trigger_fops;
+extern const struct file_operations event_hist_fops;
+
+#ifdef CONFIG_HIST_TRIGGERS
+extern int register_trigger_hist_cmd(void);
+extern int register_trigger_hist_enable_disable_cmds(void);
+#else
+static inline int register_trigger_hist_cmd(void) { return 0; }
+static inline int register_trigger_hist_enable_disable_cmds(void) { return 0; }
+#endif
extern int register_trigger_cmds(void);
extern void clear_event_triggers(struct trace_array *tr);
@@ -1174,9 +1315,41 @@
char *filter_str;
void *private_data;
bool paused;
+ bool paused_tmp;
struct list_head list;
+ char *name;
+ struct list_head named_list;
+ struct event_trigger_data *named_data;
};
+/* Avoid typos */
+#define ENABLE_EVENT_STR "enable_event"
+#define DISABLE_EVENT_STR "disable_event"
+#define ENABLE_HIST_STR "enable_hist"
+#define DISABLE_HIST_STR "disable_hist"
+
+struct enable_trigger_data {
+ struct trace_event_file *file;
+ bool enable;
+ bool hist;
+};
+
+extern int event_enable_trigger_print(struct seq_file *m,
+ struct event_trigger_ops *ops,
+ struct event_trigger_data *data);
+extern void event_enable_trigger_free(struct event_trigger_ops *ops,
+ struct event_trigger_data *data);
+extern int event_enable_trigger_func(struct event_command *cmd_ops,
+ struct trace_event_file *file,
+ char *glob, char *cmd, char *param);
+extern int event_enable_register_trigger(char *glob,
+ struct event_trigger_ops *ops,
+ struct event_trigger_data *data,
+ struct trace_event_file *file);
+extern void event_enable_unregister_trigger(char *glob,
+ struct event_trigger_ops *ops,
+ struct event_trigger_data *test,
+ struct trace_event_file *file);
extern void trigger_data_free(struct event_trigger_data *data);
extern int event_trigger_init(struct event_trigger_ops *ops,
struct event_trigger_data *data);
@@ -1189,7 +1362,18 @@
extern int set_trigger_filter(char *filter_str,
struct event_trigger_data *trigger_data,
struct trace_event_file *file);
+extern struct event_trigger_data *find_named_trigger(const char *name);
+extern bool is_named_trigger(struct event_trigger_data *test);
+extern int save_named_trigger(const char *name,
+ struct event_trigger_data *data);
+extern void del_named_trigger(struct event_trigger_data *data);
+extern void pause_named_trigger(struct event_trigger_data *data);
+extern void unpause_named_trigger(struct event_trigger_data *data);
+extern void set_named_trigger_data(struct event_trigger_data *data,
+ struct event_trigger_data *named_data);
extern int register_event_command(struct event_command *cmd);
+extern int unregister_event_command(struct event_command *cmd);
+extern int register_trigger_hist_enable_disable_cmds(void);
/**
* struct event_trigger_ops - callbacks for trace event triggers
diff --git a/kernel/trace/trace_events.c b/kernel/trace/trace_events.c
index b7b0760..3d41558 100644
--- a/kernel/trace/trace_events.c
+++ b/kernel/trace/trace_events.c
@@ -15,7 +15,7 @@
#include <linux/kthread.h>
#include <linux/tracefs.h>
#include <linux/uaccess.h>
-#include <linux/bsearch.h>
+#include <linux/vmalloc.h>
#include <linux/module.h>
#include <linux/ctype.h>
#include <linux/sort.h>
@@ -381,6 +381,7 @@
{
struct trace_event_call *call = file->event_call;
struct trace_array *tr = file->tr;
+ unsigned long file_flags = file->flags;
int ret = 0;
int disable;
@@ -463,6 +464,15 @@
break;
}
+ /* Enable or disable use of trace_buffered_event */
+ if ((file_flags & EVENT_FILE_FL_SOFT_DISABLED) !=
+ (file->flags & EVENT_FILE_FL_SOFT_DISABLED)) {
+ if (file->flags & EVENT_FILE_FL_SOFT_DISABLED)
+ trace_buffered_event_enable();
+ else
+ trace_buffered_event_disable();
+ }
+
return ret;
}
@@ -489,24 +499,26 @@
mutex_unlock(&event_mutex);
}
-static int cmp_pid(const void *key, const void *elt)
-{
- const pid_t *search_pid = key;
- const pid_t *pid = elt;
+/* Shouldn't this be in a header? */
+extern int pid_max;
- if (*search_pid == *pid)
- return 0;
- if (*search_pid < *pid)
- return -1;
- return 1;
+/* Returns true if found in filter */
+static bool
+find_filtered_pid(struct trace_pid_list *filtered_pids, pid_t search_pid)
+{
+ /*
+ * If pid_max changed after filtered_pids was created, we
+ * by default ignore all pids greater than the previous pid_max.
+ */
+ if (search_pid >= filtered_pids->pid_max)
+ return false;
+
+ return test_bit(search_pid, filtered_pids->pids);
}
static bool
-check_ignore_pid(struct trace_pid_list *filtered_pids, struct task_struct *task)
+ignore_this_task(struct trace_pid_list *filtered_pids, struct task_struct *task)
{
- pid_t search_pid;
- pid_t *pid;
-
/*
* Return false, because if filtered_pids does not exist,
* all pids are good to trace.
@@ -514,15 +526,68 @@
if (!filtered_pids)
return false;
- search_pid = task->pid;
+ return !find_filtered_pid(filtered_pids, task->pid);
+}
- pid = bsearch(&search_pid, filtered_pids->pids,
- filtered_pids->nr_pids, sizeof(pid_t),
- cmp_pid);
- if (!pid)
- return true;
+static void filter_add_remove_task(struct trace_pid_list *pid_list,
+ struct task_struct *self,
+ struct task_struct *task)
+{
+ if (!pid_list)
+ return;
- return false;
+ /* For forks, we only add if the forking task is listed */
+ if (self) {
+ if (!find_filtered_pid(pid_list, self->pid))
+ return;
+ }
+
+ /* Sorry, but we don't support pid_max changing after setting */
+ if (task->pid >= pid_list->pid_max)
+ return;
+
+ /* "self" is set for forks, and NULL for exits */
+ if (self)
+ set_bit(task->pid, pid_list->pids);
+ else
+ clear_bit(task->pid, pid_list->pids);
+}
+
+static void
+event_filter_pid_sched_process_exit(void *data, struct task_struct *task)
+{
+ struct trace_pid_list *pid_list;
+ struct trace_array *tr = data;
+
+ pid_list = rcu_dereference_sched(tr->filtered_pids);
+ filter_add_remove_task(pid_list, NULL, task);
+}
+
+static void
+event_filter_pid_sched_process_fork(void *data,
+ struct task_struct *self,
+ struct task_struct *task)
+{
+ struct trace_pid_list *pid_list;
+ struct trace_array *tr = data;
+
+ pid_list = rcu_dereference_sched(tr->filtered_pids);
+ filter_add_remove_task(pid_list, self, task);
+}
+
+void trace_event_follow_fork(struct trace_array *tr, bool enable)
+{
+ if (enable) {
+ register_trace_prio_sched_process_fork(event_filter_pid_sched_process_fork,
+ tr, INT_MIN);
+ register_trace_prio_sched_process_exit(event_filter_pid_sched_process_exit,
+ tr, INT_MAX);
+ } else {
+ unregister_trace_sched_process_fork(event_filter_pid_sched_process_fork,
+ tr);
+ unregister_trace_sched_process_exit(event_filter_pid_sched_process_exit,
+ tr);
+ }
}
static void
@@ -535,8 +600,8 @@
pid_list = rcu_dereference_sched(tr->filtered_pids);
this_cpu_write(tr->trace_buffer.data->ignore_pid,
- check_ignore_pid(pid_list, prev) &&
- check_ignore_pid(pid_list, next));
+ ignore_this_task(pid_list, prev) &&
+ ignore_this_task(pid_list, next));
}
static void
@@ -549,7 +614,7 @@
pid_list = rcu_dereference_sched(tr->filtered_pids);
this_cpu_write(tr->trace_buffer.data->ignore_pid,
- check_ignore_pid(pid_list, next));
+ ignore_this_task(pid_list, next));
}
static void
@@ -565,7 +630,7 @@
pid_list = rcu_dereference_sched(tr->filtered_pids);
this_cpu_write(tr->trace_buffer.data->ignore_pid,
- check_ignore_pid(pid_list, task));
+ ignore_this_task(pid_list, task));
}
static void
@@ -582,7 +647,7 @@
/* Set tracing if current is enabled */
this_cpu_write(tr->trace_buffer.data->ignore_pid,
- check_ignore_pid(pid_list, current));
+ ignore_this_task(pid_list, current));
}
static void __ftrace_clear_event_pids(struct trace_array *tr)
@@ -620,7 +685,7 @@
/* Wait till all users are no longer using pid filtering */
synchronize_sched();
- free_pages((unsigned long)pid_list->pids, pid_list->order);
+ vfree(pid_list->pids);
kfree(pid_list);
}
@@ -964,11 +1029,32 @@
mutex_unlock(&event_mutex);
}
+static void *
+p_next(struct seq_file *m, void *v, loff_t *pos)
+{
+ struct trace_array *tr = m->private;
+ struct trace_pid_list *pid_list = rcu_dereference_sched(tr->filtered_pids);
+ unsigned long pid = (unsigned long)v;
+
+ (*pos)++;
+
+ /* pid already is +1 of the actual prevous bit */
+ pid = find_next_bit(pid_list->pids, pid_list->pid_max, pid);
+
+ /* Return pid + 1 to allow zero to be represented */
+ if (pid < pid_list->pid_max)
+ return (void *)(pid + 1);
+
+ return NULL;
+}
+
static void *p_start(struct seq_file *m, loff_t *pos)
__acquires(RCU)
{
struct trace_pid_list *pid_list;
struct trace_array *tr = m->private;
+ unsigned long pid;
+ loff_t l = 0;
/*
* Grab the mutex, to keep calls to p_next() having the same
@@ -981,10 +1067,18 @@
pid_list = rcu_dereference_sched(tr->filtered_pids);
- if (!pid_list || *pos >= pid_list->nr_pids)
+ if (!pid_list)
return NULL;
- return (void *)&pid_list->pids[*pos];
+ pid = find_first_bit(pid_list->pids, pid_list->pid_max);
+ if (pid >= pid_list->pid_max)
+ return NULL;
+
+ /* Return pid + 1 so that zero can be the exit value */
+ for (pid++; pid && l < *pos;
+ pid = (unsigned long)p_next(m, (void *)pid, &l))
+ ;
+ return (void *)pid;
}
static void p_stop(struct seq_file *m, void *p)
@@ -994,25 +1088,11 @@
mutex_unlock(&event_mutex);
}
-static void *
-p_next(struct seq_file *m, void *v, loff_t *pos)
-{
- struct trace_array *tr = m->private;
- struct trace_pid_list *pid_list = rcu_dereference_sched(tr->filtered_pids);
-
- (*pos)++;
-
- if (*pos >= pid_list->nr_pids)
- return NULL;
-
- return (void *)&pid_list->pids[*pos];
-}
-
static int p_show(struct seq_file *m, void *v)
{
- pid_t *pid = v;
+ unsigned long pid = (unsigned long)v - 1;
- seq_printf(m, "%d\n", *pid);
+ seq_printf(m, "%lu\n", pid);
return 0;
}
@@ -1561,11 +1641,6 @@
return r;
}
-static int max_pids(struct trace_pid_list *pid_list)
-{
- return (PAGE_SIZE << pid_list->order) / sizeof(pid_t);
-}
-
static void ignore_task_cpu(void *data)
{
struct trace_array *tr = data;
@@ -1579,7 +1654,7 @@
mutex_is_locked(&event_mutex));
this_cpu_write(tr->trace_buffer.data->ignore_pid,
- check_ignore_pid(pid_list, current));
+ ignore_this_task(pid_list, current));
}
static ssize_t
@@ -1589,7 +1664,7 @@
struct seq_file *m = filp->private_data;
struct trace_array *tr = m->private;
struct trace_pid_list *filtered_pids = NULL;
- struct trace_pid_list *pid_list = NULL;
+ struct trace_pid_list *pid_list;
struct trace_event_file *file;
struct trace_parser parser;
unsigned long val;
@@ -1597,7 +1672,7 @@
ssize_t read = 0;
ssize_t ret = 0;
pid_t pid;
- int i;
+ int nr_pids = 0;
if (!cnt)
return 0;
@@ -1610,10 +1685,43 @@
return -ENOMEM;
mutex_lock(&event_mutex);
+ filtered_pids = rcu_dereference_protected(tr->filtered_pids,
+ lockdep_is_held(&event_mutex));
+
/*
- * Load as many pids into the array before doing a
- * swap from the tr->filtered_pids to the new list.
+ * Always recreate a new array. The write is an all or nothing
+ * operation. Always create a new array when adding new pids by
+ * the user. If the operation fails, then the current list is
+ * not modified.
*/
+ pid_list = kmalloc(sizeof(*pid_list), GFP_KERNEL);
+ if (!pid_list) {
+ read = -ENOMEM;
+ goto out;
+ }
+ pid_list->pid_max = READ_ONCE(pid_max);
+ /* Only truncating will shrink pid_max */
+ if (filtered_pids && filtered_pids->pid_max > pid_list->pid_max)
+ pid_list->pid_max = filtered_pids->pid_max;
+ pid_list->pids = vzalloc((pid_list->pid_max + 7) >> 3);
+ if (!pid_list->pids) {
+ kfree(pid_list);
+ read = -ENOMEM;
+ goto out;
+ }
+ if (filtered_pids) {
+ /* copy the current bits to the new max */
+ pid = find_first_bit(filtered_pids->pids,
+ filtered_pids->pid_max);
+ while (pid < filtered_pids->pid_max) {
+ set_bit(pid, pid_list->pids);
+ pid = find_next_bit(filtered_pids->pids,
+ filtered_pids->pid_max,
+ pid + 1);
+ nr_pids++;
+ }
+ }
+
while (cnt > 0) {
this_pos = 0;
@@ -1631,92 +1739,35 @@
ret = -EINVAL;
if (kstrtoul(parser.buffer, 0, &val))
break;
- if (val > INT_MAX)
+ if (val >= pid_list->pid_max)
break;
pid = (pid_t)val;
- ret = -ENOMEM;
- if (!pid_list) {
- pid_list = kmalloc(sizeof(*pid_list), GFP_KERNEL);
- if (!pid_list)
- break;
+ set_bit(pid, pid_list->pids);
+ nr_pids++;
- filtered_pids = rcu_dereference_protected(tr->filtered_pids,
- lockdep_is_held(&event_mutex));
- if (filtered_pids)
- pid_list->order = filtered_pids->order;
- else
- pid_list->order = 0;
-
- pid_list->pids = (void *)__get_free_pages(GFP_KERNEL,
- pid_list->order);
- if (!pid_list->pids)
- break;
-
- if (filtered_pids) {
- pid_list->nr_pids = filtered_pids->nr_pids;
- memcpy(pid_list->pids, filtered_pids->pids,
- pid_list->nr_pids * sizeof(pid_t));
- } else
- pid_list->nr_pids = 0;
- }
-
- if (pid_list->nr_pids >= max_pids(pid_list)) {
- pid_t *pid_page;
-
- pid_page = (void *)__get_free_pages(GFP_KERNEL,
- pid_list->order + 1);
- if (!pid_page)
- break;
- memcpy(pid_page, pid_list->pids,
- pid_list->nr_pids * sizeof(pid_t));
- free_pages((unsigned long)pid_list->pids, pid_list->order);
-
- pid_list->order++;
- pid_list->pids = pid_page;
- }
-
- pid_list->pids[pid_list->nr_pids++] = pid;
trace_parser_clear(&parser);
ret = 0;
}
trace_parser_put(&parser);
if (ret < 0) {
- if (pid_list)
- free_pages((unsigned long)pid_list->pids, pid_list->order);
+ vfree(pid_list->pids);
kfree(pid_list);
- mutex_unlock(&event_mutex);
- return ret;
+ read = ret;
+ goto out;
}
- if (!pid_list) {
- mutex_unlock(&event_mutex);
- return ret;
+ if (!nr_pids) {
+ /* Cleared the list of pids */
+ vfree(pid_list->pids);
+ kfree(pid_list);
+ read = ret;
+ if (!filtered_pids)
+ goto out;
+ pid_list = NULL;
}
-
- sort(pid_list->pids, pid_list->nr_pids, sizeof(pid_t), cmp_pid, NULL);
-
- /* Remove duplicates */
- for (i = 1; i < pid_list->nr_pids; i++) {
- int start = i;
-
- while (i < pid_list->nr_pids &&
- pid_list->pids[i - 1] == pid_list->pids[i])
- i++;
-
- if (start != i) {
- if (i < pid_list->nr_pids) {
- memmove(&pid_list->pids[start], &pid_list->pids[i],
- (pid_list->nr_pids - i) * sizeof(pid_t));
- pid_list->nr_pids -= i - start;
- i = start;
- } else
- pid_list->nr_pids = start;
- }
- }
-
rcu_assign_pointer(tr->filtered_pids, pid_list);
list_for_each_entry(file, &tr->events, list) {
@@ -1726,7 +1777,7 @@
if (filtered_pids) {
synchronize_sched();
- free_pages((unsigned long)filtered_pids->pids, filtered_pids->order);
+ vfree(filtered_pids->pids);
kfree(filtered_pids);
} else {
/*
@@ -1763,10 +1814,12 @@
*/
on_each_cpu(ignore_task_cpu, tr, 1);
+ out:
mutex_unlock(&event_mutex);
ret = read;
- *ppos += read;
+ if (read > 0)
+ *ppos += read;
return ret;
}
@@ -2121,6 +2174,10 @@
trace_create_file("trigger", 0644, file->dir, file,
&event_trigger_fops);
+#ifdef CONFIG_HIST_TRIGGERS
+ trace_create_file("hist", 0444, file->dir, file,
+ &event_hist_fops);
+#endif
trace_create_file("format", 0444, file->dir, call,
&ftrace_event_format_fops);
@@ -3368,7 +3425,7 @@
static DEFINE_PER_CPU(atomic_t, ftrace_test_event_disable);
-static struct trace_array *event_tr;
+static struct trace_event_file event_trace_file __initdata;
static void __init
function_test_events_call(unsigned long ip, unsigned long parent_ip,
@@ -3392,17 +3449,17 @@
local_save_flags(flags);
- event = trace_current_buffer_lock_reserve(&buffer,
- TRACE_FN, sizeof(*entry),
- flags, pc);
+ event = trace_event_buffer_lock_reserve(&buffer, &event_trace_file,
+ TRACE_FN, sizeof(*entry),
+ flags, pc);
if (!event)
goto out;
entry = ring_buffer_event_data(event);
entry->ip = ip;
entry->parent_ip = parent_ip;
- trace_buffer_unlock_commit(event_tr, buffer, event, flags, pc);
-
+ event_trigger_unlock_commit(&event_trace_file, buffer, event,
+ entry, flags, pc);
out:
atomic_dec(&per_cpu(ftrace_test_event_disable, cpu));
preempt_enable_notrace();
@@ -3417,9 +3474,11 @@
static __init void event_trace_self_test_with_function(void)
{
int ret;
- event_tr = top_trace_array();
- if (WARN_ON(!event_tr))
+
+ event_trace_file.tr = top_trace_array();
+ if (WARN_ON(!event_trace_file.tr))
return;
+
ret = register_ftrace_function(&trace_ops);
if (WARN_ON(ret < 0)) {
pr_info("Failed to enable function tracer for event tests\n");
diff --git a/kernel/trace/trace_events_filter.c b/kernel/trace/trace_events_filter.c
index b3f5051..9daa9b3 100644
--- a/kernel/trace/trace_events_filter.c
+++ b/kernel/trace/trace_events_filter.c
@@ -689,10 +689,7 @@
static inline struct event_filter *event_filter(struct trace_event_file *file)
{
- if (file->event_call->flags & TRACE_EVENT_FL_USE_CALL_FILTER)
- return file->event_call->filter;
- else
- return file->filter;
+ return file->filter;
}
/* caller must hold event_mutex */
@@ -826,12 +823,12 @@
static void filter_disable(struct trace_event_file *file)
{
- struct trace_event_call *call = file->event_call;
+ unsigned long old_flags = file->flags;
- if (call->flags & TRACE_EVENT_FL_USE_CALL_FILTER)
- call->flags &= ~TRACE_EVENT_FL_FILTERED;
- else
- file->flags &= ~EVENT_FILE_FL_FILTERED;
+ file->flags &= ~EVENT_FILE_FL_FILTERED;
+
+ if (old_flags != file->flags)
+ trace_buffered_event_disable();
}
static void __free_filter(struct event_filter *filter)
@@ -883,13 +880,8 @@
static inline void __remove_filter(struct trace_event_file *file)
{
- struct trace_event_call *call = file->event_call;
-
filter_disable(file);
- if (call->flags & TRACE_EVENT_FL_USE_CALL_FILTER)
- remove_filter_string(call->filter);
- else
- remove_filter_string(file->filter);
+ remove_filter_string(file->filter);
}
static void filter_free_subsystem_preds(struct trace_subsystem_dir *dir,
@@ -906,15 +898,8 @@
static inline void __free_subsystem_filter(struct trace_event_file *file)
{
- struct trace_event_call *call = file->event_call;
-
- if (call->flags & TRACE_EVENT_FL_USE_CALL_FILTER) {
- __free_filter(call->filter);
- call->filter = NULL;
- } else {
- __free_filter(file->filter);
- file->filter = NULL;
- }
+ __free_filter(file->filter);
+ file->filter = NULL;
}
static void filter_free_subsystem_filters(struct trace_subsystem_dir *dir,
@@ -1718,69 +1703,43 @@
static inline void event_set_filtered_flag(struct trace_event_file *file)
{
- struct trace_event_call *call = file->event_call;
+ unsigned long old_flags = file->flags;
- if (call->flags & TRACE_EVENT_FL_USE_CALL_FILTER)
- call->flags |= TRACE_EVENT_FL_FILTERED;
- else
- file->flags |= EVENT_FILE_FL_FILTERED;
+ file->flags |= EVENT_FILE_FL_FILTERED;
+
+ if (old_flags != file->flags)
+ trace_buffered_event_enable();
}
static inline void event_set_filter(struct trace_event_file *file,
struct event_filter *filter)
{
- struct trace_event_call *call = file->event_call;
-
- if (call->flags & TRACE_EVENT_FL_USE_CALL_FILTER)
- rcu_assign_pointer(call->filter, filter);
- else
- rcu_assign_pointer(file->filter, filter);
+ rcu_assign_pointer(file->filter, filter);
}
static inline void event_clear_filter(struct trace_event_file *file)
{
- struct trace_event_call *call = file->event_call;
-
- if (call->flags & TRACE_EVENT_FL_USE_CALL_FILTER)
- RCU_INIT_POINTER(call->filter, NULL);
- else
- RCU_INIT_POINTER(file->filter, NULL);
+ RCU_INIT_POINTER(file->filter, NULL);
}
static inline void
event_set_no_set_filter_flag(struct trace_event_file *file)
{
- struct trace_event_call *call = file->event_call;
-
- if (call->flags & TRACE_EVENT_FL_USE_CALL_FILTER)
- call->flags |= TRACE_EVENT_FL_NO_SET_FILTER;
- else
- file->flags |= EVENT_FILE_FL_NO_SET_FILTER;
+ file->flags |= EVENT_FILE_FL_NO_SET_FILTER;
}
static inline void
event_clear_no_set_filter_flag(struct trace_event_file *file)
{
- struct trace_event_call *call = file->event_call;
-
- if (call->flags & TRACE_EVENT_FL_USE_CALL_FILTER)
- call->flags &= ~TRACE_EVENT_FL_NO_SET_FILTER;
- else
- file->flags &= ~EVENT_FILE_FL_NO_SET_FILTER;
+ file->flags &= ~EVENT_FILE_FL_NO_SET_FILTER;
}
static inline bool
event_no_set_filter_flag(struct trace_event_file *file)
{
- struct trace_event_call *call = file->event_call;
-
if (file->flags & EVENT_FILE_FL_NO_SET_FILTER)
return true;
- if ((call->flags & TRACE_EVENT_FL_USE_CALL_FILTER) &&
- (call->flags & TRACE_EVENT_FL_NO_SET_FILTER))
- return true;
-
return false;
}
diff --git a/kernel/trace/trace_events_hist.c b/kernel/trace/trace_events_hist.c
new file mode 100644
index 0000000..0c05b8a
--- /dev/null
+++ b/kernel/trace/trace_events_hist.c
@@ -0,0 +1,1755 @@
+/*
+ * trace_events_hist - trace event hist triggers
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * Copyright (C) 2015 Tom Zanussi <tom.zanussi@linux.intel.com>
+ */
+
+#include <linux/module.h>
+#include <linux/kallsyms.h>
+#include <linux/mutex.h>
+#include <linux/slab.h>
+#include <linux/stacktrace.h>
+
+#include "tracing_map.h"
+#include "trace.h"
+
+struct hist_field;
+
+typedef u64 (*hist_field_fn_t) (struct hist_field *field, void *event);
+
+struct hist_field {
+ struct ftrace_event_field *field;
+ unsigned long flags;
+ hist_field_fn_t fn;
+ unsigned int size;
+ unsigned int offset;
+};
+
+static u64 hist_field_none(struct hist_field *field, void *event)
+{
+ return 0;
+}
+
+static u64 hist_field_counter(struct hist_field *field, void *event)
+{
+ return 1;
+}
+
+static u64 hist_field_string(struct hist_field *hist_field, void *event)
+{
+ char *addr = (char *)(event + hist_field->field->offset);
+
+ return (u64)(unsigned long)addr;
+}
+
+static u64 hist_field_dynstring(struct hist_field *hist_field, void *event)
+{
+ u32 str_item = *(u32 *)(event + hist_field->field->offset);
+ int str_loc = str_item & 0xffff;
+ char *addr = (char *)(event + str_loc);
+
+ return (u64)(unsigned long)addr;
+}
+
+static u64 hist_field_pstring(struct hist_field *hist_field, void *event)
+{
+ char **addr = (char **)(event + hist_field->field->offset);
+
+ return (u64)(unsigned long)*addr;
+}
+
+static u64 hist_field_log2(struct hist_field *hist_field, void *event)
+{
+ u64 val = *(u64 *)(event + hist_field->field->offset);
+
+ return (u64) ilog2(roundup_pow_of_two(val));
+}
+
+#define DEFINE_HIST_FIELD_FN(type) \
+static u64 hist_field_##type(struct hist_field *hist_field, void *event)\
+{ \
+ type *addr = (type *)(event + hist_field->field->offset); \
+ \
+ return (u64)(unsigned long)*addr; \
+}
+
+DEFINE_HIST_FIELD_FN(s64);
+DEFINE_HIST_FIELD_FN(u64);
+DEFINE_HIST_FIELD_FN(s32);
+DEFINE_HIST_FIELD_FN(u32);
+DEFINE_HIST_FIELD_FN(s16);
+DEFINE_HIST_FIELD_FN(u16);
+DEFINE_HIST_FIELD_FN(s8);
+DEFINE_HIST_FIELD_FN(u8);
+
+#define for_each_hist_field(i, hist_data) \
+ for ((i) = 0; (i) < (hist_data)->n_fields; (i)++)
+
+#define for_each_hist_val_field(i, hist_data) \
+ for ((i) = 0; (i) < (hist_data)->n_vals; (i)++)
+
+#define for_each_hist_key_field(i, hist_data) \
+ for ((i) = (hist_data)->n_vals; (i) < (hist_data)->n_fields; (i)++)
+
+#define HIST_STACKTRACE_DEPTH 16
+#define HIST_STACKTRACE_SIZE (HIST_STACKTRACE_DEPTH * sizeof(unsigned long))
+#define HIST_STACKTRACE_SKIP 5
+
+#define HITCOUNT_IDX 0
+#define HIST_KEY_SIZE_MAX (MAX_FILTER_STR_VAL + HIST_STACKTRACE_SIZE)
+
+enum hist_field_flags {
+ HIST_FIELD_FL_HITCOUNT = 1,
+ HIST_FIELD_FL_KEY = 2,
+ HIST_FIELD_FL_STRING = 4,
+ HIST_FIELD_FL_HEX = 8,
+ HIST_FIELD_FL_SYM = 16,
+ HIST_FIELD_FL_SYM_OFFSET = 32,
+ HIST_FIELD_FL_EXECNAME = 64,
+ HIST_FIELD_FL_SYSCALL = 128,
+ HIST_FIELD_FL_STACKTRACE = 256,
+ HIST_FIELD_FL_LOG2 = 512,
+};
+
+struct hist_trigger_attrs {
+ char *keys_str;
+ char *vals_str;
+ char *sort_key_str;
+ char *name;
+ bool pause;
+ bool cont;
+ bool clear;
+ unsigned int map_bits;
+};
+
+struct hist_trigger_data {
+ struct hist_field *fields[TRACING_MAP_FIELDS_MAX];
+ unsigned int n_vals;
+ unsigned int n_keys;
+ unsigned int n_fields;
+ unsigned int key_size;
+ struct tracing_map_sort_key sort_keys[TRACING_MAP_SORT_KEYS_MAX];
+ unsigned int n_sort_keys;
+ struct trace_event_file *event_file;
+ struct hist_trigger_attrs *attrs;
+ struct tracing_map *map;
+};
+
+static hist_field_fn_t select_value_fn(int field_size, int field_is_signed)
+{
+ hist_field_fn_t fn = NULL;
+
+ switch (field_size) {
+ case 8:
+ if (field_is_signed)
+ fn = hist_field_s64;
+ else
+ fn = hist_field_u64;
+ break;
+ case 4:
+ if (field_is_signed)
+ fn = hist_field_s32;
+ else
+ fn = hist_field_u32;
+ break;
+ case 2:
+ if (field_is_signed)
+ fn = hist_field_s16;
+ else
+ fn = hist_field_u16;
+ break;
+ case 1:
+ if (field_is_signed)
+ fn = hist_field_s8;
+ else
+ fn = hist_field_u8;
+ break;
+ }
+
+ return fn;
+}
+
+static int parse_map_size(char *str)
+{
+ unsigned long size, map_bits;
+ int ret;
+
+ strsep(&str, "=");
+ if (!str) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ ret = kstrtoul(str, 0, &size);
+ if (ret)
+ goto out;
+
+ map_bits = ilog2(roundup_pow_of_two(size));
+ if (map_bits < TRACING_MAP_BITS_MIN ||
+ map_bits > TRACING_MAP_BITS_MAX)
+ ret = -EINVAL;
+ else
+ ret = map_bits;
+ out:
+ return ret;
+}
+
+static void destroy_hist_trigger_attrs(struct hist_trigger_attrs *attrs)
+{
+ if (!attrs)
+ return;
+
+ kfree(attrs->name);
+ kfree(attrs->sort_key_str);
+ kfree(attrs->keys_str);
+ kfree(attrs->vals_str);
+ kfree(attrs);
+}
+
+static struct hist_trigger_attrs *parse_hist_trigger_attrs(char *trigger_str)
+{
+ struct hist_trigger_attrs *attrs;
+ int ret = 0;
+
+ attrs = kzalloc(sizeof(*attrs), GFP_KERNEL);
+ if (!attrs)
+ return ERR_PTR(-ENOMEM);
+
+ while (trigger_str) {
+ char *str = strsep(&trigger_str, ":");
+
+ if ((strncmp(str, "key=", strlen("key=")) == 0) ||
+ (strncmp(str, "keys=", strlen("keys=")) == 0))
+ attrs->keys_str = kstrdup(str, GFP_KERNEL);
+ else if ((strncmp(str, "val=", strlen("val=")) == 0) ||
+ (strncmp(str, "vals=", strlen("vals=")) == 0) ||
+ (strncmp(str, "values=", strlen("values=")) == 0))
+ attrs->vals_str = kstrdup(str, GFP_KERNEL);
+ else if (strncmp(str, "sort=", strlen("sort=")) == 0)
+ attrs->sort_key_str = kstrdup(str, GFP_KERNEL);
+ else if (strncmp(str, "name=", strlen("name=")) == 0)
+ attrs->name = kstrdup(str, GFP_KERNEL);
+ else if (strcmp(str, "pause") == 0)
+ attrs->pause = true;
+ else if ((strcmp(str, "cont") == 0) ||
+ (strcmp(str, "continue") == 0))
+ attrs->cont = true;
+ else if (strcmp(str, "clear") == 0)
+ attrs->clear = true;
+ else if (strncmp(str, "size=", strlen("size=")) == 0) {
+ int map_bits = parse_map_size(str);
+
+ if (map_bits < 0) {
+ ret = map_bits;
+ goto free;
+ }
+ attrs->map_bits = map_bits;
+ } else {
+ ret = -EINVAL;
+ goto free;
+ }
+ }
+
+ if (!attrs->keys_str) {
+ ret = -EINVAL;
+ goto free;
+ }
+
+ return attrs;
+ free:
+ destroy_hist_trigger_attrs(attrs);
+
+ return ERR_PTR(ret);
+}
+
+static inline void save_comm(char *comm, struct task_struct *task)
+{
+ if (!task->pid) {
+ strcpy(comm, "<idle>");
+ return;
+ }
+
+ if (WARN_ON_ONCE(task->pid < 0)) {
+ strcpy(comm, "<XXX>");
+ return;
+ }
+
+ memcpy(comm, task->comm, TASK_COMM_LEN);
+}
+
+static void hist_trigger_elt_comm_free(struct tracing_map_elt *elt)
+{
+ kfree((char *)elt->private_data);
+}
+
+static int hist_trigger_elt_comm_alloc(struct tracing_map_elt *elt)
+{
+ struct hist_trigger_data *hist_data = elt->map->private_data;
+ struct hist_field *key_field;
+ unsigned int i;
+
+ for_each_hist_key_field(i, hist_data) {
+ key_field = hist_data->fields[i];
+
+ if (key_field->flags & HIST_FIELD_FL_EXECNAME) {
+ unsigned int size = TASK_COMM_LEN + 1;
+
+ elt->private_data = kzalloc(size, GFP_KERNEL);
+ if (!elt->private_data)
+ return -ENOMEM;
+ break;
+ }
+ }
+
+ return 0;
+}
+
+static void hist_trigger_elt_comm_copy(struct tracing_map_elt *to,
+ struct tracing_map_elt *from)
+{
+ char *comm_from = from->private_data;
+ char *comm_to = to->private_data;
+
+ if (comm_from)
+ memcpy(comm_to, comm_from, TASK_COMM_LEN + 1);
+}
+
+static void hist_trigger_elt_comm_init(struct tracing_map_elt *elt)
+{
+ char *comm = elt->private_data;
+
+ if (comm)
+ save_comm(comm, current);
+}
+
+static const struct tracing_map_ops hist_trigger_elt_comm_ops = {
+ .elt_alloc = hist_trigger_elt_comm_alloc,
+ .elt_copy = hist_trigger_elt_comm_copy,
+ .elt_free = hist_trigger_elt_comm_free,
+ .elt_init = hist_trigger_elt_comm_init,
+};
+
+static void destroy_hist_field(struct hist_field *hist_field)
+{
+ kfree(hist_field);
+}
+
+static struct hist_field *create_hist_field(struct ftrace_event_field *field,
+ unsigned long flags)
+{
+ struct hist_field *hist_field;
+
+ if (field && is_function_field(field))
+ return NULL;
+
+ hist_field = kzalloc(sizeof(struct hist_field), GFP_KERNEL);
+ if (!hist_field)
+ return NULL;
+
+ if (flags & HIST_FIELD_FL_HITCOUNT) {
+ hist_field->fn = hist_field_counter;
+ goto out;
+ }
+
+ if (flags & HIST_FIELD_FL_STACKTRACE) {
+ hist_field->fn = hist_field_none;
+ goto out;
+ }
+
+ if (flags & HIST_FIELD_FL_LOG2) {
+ hist_field->fn = hist_field_log2;
+ goto out;
+ }
+
+ if (WARN_ON_ONCE(!field))
+ goto out;
+
+ if (is_string_field(field)) {
+ flags |= HIST_FIELD_FL_STRING;
+
+ if (field->filter_type == FILTER_STATIC_STRING)
+ hist_field->fn = hist_field_string;
+ else if (field->filter_type == FILTER_DYN_STRING)
+ hist_field->fn = hist_field_dynstring;
+ else
+ hist_field->fn = hist_field_pstring;
+ } else {
+ hist_field->fn = select_value_fn(field->size,
+ field->is_signed);
+ if (!hist_field->fn) {
+ destroy_hist_field(hist_field);
+ return NULL;
+ }
+ }
+ out:
+ hist_field->field = field;
+ hist_field->flags = flags;
+
+ return hist_field;
+}
+
+static void destroy_hist_fields(struct hist_trigger_data *hist_data)
+{
+ unsigned int i;
+
+ for (i = 0; i < TRACING_MAP_FIELDS_MAX; i++) {
+ if (hist_data->fields[i]) {
+ destroy_hist_field(hist_data->fields[i]);
+ hist_data->fields[i] = NULL;
+ }
+ }
+}
+
+static int create_hitcount_val(struct hist_trigger_data *hist_data)
+{
+ hist_data->fields[HITCOUNT_IDX] =
+ create_hist_field(NULL, HIST_FIELD_FL_HITCOUNT);
+ if (!hist_data->fields[HITCOUNT_IDX])
+ return -ENOMEM;
+
+ hist_data->n_vals++;
+
+ if (WARN_ON(hist_data->n_vals > TRACING_MAP_VALS_MAX))
+ return -EINVAL;
+
+ return 0;
+}
+
+static int create_val_field(struct hist_trigger_data *hist_data,
+ unsigned int val_idx,
+ struct trace_event_file *file,
+ char *field_str)
+{
+ struct ftrace_event_field *field = NULL;
+ unsigned long flags = 0;
+ char *field_name;
+ int ret = 0;
+
+ if (WARN_ON(val_idx >= TRACING_MAP_VALS_MAX))
+ return -EINVAL;
+
+ field_name = strsep(&field_str, ".");
+ if (field_str) {
+ if (strcmp(field_str, "hex") == 0)
+ flags |= HIST_FIELD_FL_HEX;
+ else {
+ ret = -EINVAL;
+ goto out;
+ }
+ }
+
+ field = trace_find_event_field(file->event_call, field_name);
+ if (!field) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ hist_data->fields[val_idx] = create_hist_field(field, flags);
+ if (!hist_data->fields[val_idx]) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ ++hist_data->n_vals;
+
+ if (WARN_ON(hist_data->n_vals > TRACING_MAP_VALS_MAX))
+ ret = -EINVAL;
+ out:
+ return ret;
+}
+
+static int create_val_fields(struct hist_trigger_data *hist_data,
+ struct trace_event_file *file)
+{
+ char *fields_str, *field_str;
+ unsigned int i, j;
+ int ret;
+
+ ret = create_hitcount_val(hist_data);
+ if (ret)
+ goto out;
+
+ fields_str = hist_data->attrs->vals_str;
+ if (!fields_str)
+ goto out;
+
+ strsep(&fields_str, "=");
+ if (!fields_str)
+ goto out;
+
+ for (i = 0, j = 1; i < TRACING_MAP_VALS_MAX &&
+ j < TRACING_MAP_VALS_MAX; i++) {
+ field_str = strsep(&fields_str, ",");
+ if (!field_str)
+ break;
+ if (strcmp(field_str, "hitcount") == 0)
+ continue;
+ ret = create_val_field(hist_data, j++, file, field_str);
+ if (ret)
+ goto out;
+ }
+ if (fields_str && (strcmp(fields_str, "hitcount") != 0))
+ ret = -EINVAL;
+ out:
+ return ret;
+}
+
+static int create_key_field(struct hist_trigger_data *hist_data,
+ unsigned int key_idx,
+ unsigned int key_offset,
+ struct trace_event_file *file,
+ char *field_str)
+{
+ struct ftrace_event_field *field = NULL;
+ unsigned long flags = 0;
+ unsigned int key_size;
+ int ret = 0;
+
+ if (WARN_ON(key_idx >= TRACING_MAP_FIELDS_MAX))
+ return -EINVAL;
+
+ flags |= HIST_FIELD_FL_KEY;
+
+ if (strcmp(field_str, "stacktrace") == 0) {
+ flags |= HIST_FIELD_FL_STACKTRACE;
+ key_size = sizeof(unsigned long) * HIST_STACKTRACE_DEPTH;
+ } else {
+ char *field_name = strsep(&field_str, ".");
+
+ if (field_str) {
+ if (strcmp(field_str, "hex") == 0)
+ flags |= HIST_FIELD_FL_HEX;
+ else if (strcmp(field_str, "sym") == 0)
+ flags |= HIST_FIELD_FL_SYM;
+ else if (strcmp(field_str, "sym-offset") == 0)
+ flags |= HIST_FIELD_FL_SYM_OFFSET;
+ else if ((strcmp(field_str, "execname") == 0) &&
+ (strcmp(field_name, "common_pid") == 0))
+ flags |= HIST_FIELD_FL_EXECNAME;
+ else if (strcmp(field_str, "syscall") == 0)
+ flags |= HIST_FIELD_FL_SYSCALL;
+ else if (strcmp(field_str, "log2") == 0)
+ flags |= HIST_FIELD_FL_LOG2;
+ else {
+ ret = -EINVAL;
+ goto out;
+ }
+ }
+
+ field = trace_find_event_field(file->event_call, field_name);
+ if (!field) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ if (is_string_field(field))
+ key_size = MAX_FILTER_STR_VAL;
+ else
+ key_size = field->size;
+ }
+
+ hist_data->fields[key_idx] = create_hist_field(field, flags);
+ if (!hist_data->fields[key_idx]) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ key_size = ALIGN(key_size, sizeof(u64));
+ hist_data->fields[key_idx]->size = key_size;
+ hist_data->fields[key_idx]->offset = key_offset;
+ hist_data->key_size += key_size;
+ if (hist_data->key_size > HIST_KEY_SIZE_MAX) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ hist_data->n_keys++;
+
+ if (WARN_ON(hist_data->n_keys > TRACING_MAP_KEYS_MAX))
+ return -EINVAL;
+
+ ret = key_size;
+ out:
+ return ret;
+}
+
+static int create_key_fields(struct hist_trigger_data *hist_data,
+ struct trace_event_file *file)
+{
+ unsigned int i, key_offset = 0, n_vals = hist_data->n_vals;
+ char *fields_str, *field_str;
+ int ret = -EINVAL;
+
+ fields_str = hist_data->attrs->keys_str;
+ if (!fields_str)
+ goto out;
+
+ strsep(&fields_str, "=");
+ if (!fields_str)
+ goto out;
+
+ for (i = n_vals; i < n_vals + TRACING_MAP_KEYS_MAX; i++) {
+ field_str = strsep(&fields_str, ",");
+ if (!field_str)
+ break;
+ ret = create_key_field(hist_data, i, key_offset,
+ file, field_str);
+ if (ret < 0)
+ goto out;
+ key_offset += ret;
+ }
+ if (fields_str) {
+ ret = -EINVAL;
+ goto out;
+ }
+ ret = 0;
+ out:
+ return ret;
+}
+
+static int create_hist_fields(struct hist_trigger_data *hist_data,
+ struct trace_event_file *file)
+{
+ int ret;
+
+ ret = create_val_fields(hist_data, file);
+ if (ret)
+ goto out;
+
+ ret = create_key_fields(hist_data, file);
+ if (ret)
+ goto out;
+
+ hist_data->n_fields = hist_data->n_vals + hist_data->n_keys;
+ out:
+ return ret;
+}
+
+static int is_descending(const char *str)
+{
+ if (!str)
+ return 0;
+
+ if (strcmp(str, "descending") == 0)
+ return 1;
+
+ if (strcmp(str, "ascending") == 0)
+ return 0;
+
+ return -EINVAL;
+}
+
+static int create_sort_keys(struct hist_trigger_data *hist_data)
+{
+ char *fields_str = hist_data->attrs->sort_key_str;
+ struct ftrace_event_field *field = NULL;
+ struct tracing_map_sort_key *sort_key;
+ int descending, ret = 0;
+ unsigned int i, j;
+
+ hist_data->n_sort_keys = 1; /* we always have at least one, hitcount */
+
+ if (!fields_str)
+ goto out;
+
+ strsep(&fields_str, "=");
+ if (!fields_str) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ for (i = 0; i < TRACING_MAP_SORT_KEYS_MAX; i++) {
+ char *field_str, *field_name;
+
+ sort_key = &hist_data->sort_keys[i];
+
+ field_str = strsep(&fields_str, ",");
+ if (!field_str) {
+ if (i == 0)
+ ret = -EINVAL;
+ break;
+ }
+
+ if ((i == TRACING_MAP_SORT_KEYS_MAX - 1) && fields_str) {
+ ret = -EINVAL;
+ break;
+ }
+
+ field_name = strsep(&field_str, ".");
+ if (!field_name) {
+ ret = -EINVAL;
+ break;
+ }
+
+ if (strcmp(field_name, "hitcount") == 0) {
+ descending = is_descending(field_str);
+ if (descending < 0) {
+ ret = descending;
+ break;
+ }
+ sort_key->descending = descending;
+ continue;
+ }
+
+ for (j = 1; j < hist_data->n_fields; j++) {
+ field = hist_data->fields[j]->field;
+ if (field && (strcmp(field_name, field->name) == 0)) {
+ sort_key->field_idx = j;
+ descending = is_descending(field_str);
+ if (descending < 0) {
+ ret = descending;
+ goto out;
+ }
+ sort_key->descending = descending;
+ break;
+ }
+ }
+ if (j == hist_data->n_fields) {
+ ret = -EINVAL;
+ break;
+ }
+ }
+ hist_data->n_sort_keys = i;
+ out:
+ return ret;
+}
+
+static void destroy_hist_data(struct hist_trigger_data *hist_data)
+{
+ destroy_hist_trigger_attrs(hist_data->attrs);
+ destroy_hist_fields(hist_data);
+ tracing_map_destroy(hist_data->map);
+ kfree(hist_data);
+}
+
+static int create_tracing_map_fields(struct hist_trigger_data *hist_data)
+{
+ struct tracing_map *map = hist_data->map;
+ struct ftrace_event_field *field;
+ struct hist_field *hist_field;
+ int i, idx;
+
+ for_each_hist_field(i, hist_data) {
+ hist_field = hist_data->fields[i];
+ if (hist_field->flags & HIST_FIELD_FL_KEY) {
+ tracing_map_cmp_fn_t cmp_fn;
+
+ field = hist_field->field;
+
+ if (hist_field->flags & HIST_FIELD_FL_STACKTRACE)
+ cmp_fn = tracing_map_cmp_none;
+ else if (is_string_field(field))
+ cmp_fn = tracing_map_cmp_string;
+ else
+ cmp_fn = tracing_map_cmp_num(field->size,
+ field->is_signed);
+ idx = tracing_map_add_key_field(map,
+ hist_field->offset,
+ cmp_fn);
+
+ } else
+ idx = tracing_map_add_sum_field(map);
+
+ if (idx < 0)
+ return idx;
+ }
+
+ return 0;
+}
+
+static bool need_tracing_map_ops(struct hist_trigger_data *hist_data)
+{
+ struct hist_field *key_field;
+ unsigned int i;
+
+ for_each_hist_key_field(i, hist_data) {
+ key_field = hist_data->fields[i];
+
+ if (key_field->flags & HIST_FIELD_FL_EXECNAME)
+ return true;
+ }
+
+ return false;
+}
+
+static struct hist_trigger_data *
+create_hist_data(unsigned int map_bits,
+ struct hist_trigger_attrs *attrs,
+ struct trace_event_file *file)
+{
+ const struct tracing_map_ops *map_ops = NULL;
+ struct hist_trigger_data *hist_data;
+ int ret = 0;
+
+ hist_data = kzalloc(sizeof(*hist_data), GFP_KERNEL);
+ if (!hist_data)
+ return ERR_PTR(-ENOMEM);
+
+ hist_data->attrs = attrs;
+
+ ret = create_hist_fields(hist_data, file);
+ if (ret)
+ goto free;
+
+ ret = create_sort_keys(hist_data);
+ if (ret)
+ goto free;
+
+ if (need_tracing_map_ops(hist_data))
+ map_ops = &hist_trigger_elt_comm_ops;
+
+ hist_data->map = tracing_map_create(map_bits, hist_data->key_size,
+ map_ops, hist_data);
+ if (IS_ERR(hist_data->map)) {
+ ret = PTR_ERR(hist_data->map);
+ hist_data->map = NULL;
+ goto free;
+ }
+
+ ret = create_tracing_map_fields(hist_data);
+ if (ret)
+ goto free;
+
+ ret = tracing_map_init(hist_data->map);
+ if (ret)
+ goto free;
+
+ hist_data->event_file = file;
+ out:
+ return hist_data;
+ free:
+ hist_data->attrs = NULL;
+
+ destroy_hist_data(hist_data);
+
+ hist_data = ERR_PTR(ret);
+
+ goto out;
+}
+
+static void hist_trigger_elt_update(struct hist_trigger_data *hist_data,
+ struct tracing_map_elt *elt,
+ void *rec)
+{
+ struct hist_field *hist_field;
+ unsigned int i;
+ u64 hist_val;
+
+ for_each_hist_val_field(i, hist_data) {
+ hist_field = hist_data->fields[i];
+ hist_val = hist_field->fn(hist_field, rec);
+ tracing_map_update_sum(elt, i, hist_val);
+ }
+}
+
+static inline void add_to_key(char *compound_key, void *key,
+ struct hist_field *key_field, void *rec)
+{
+ size_t size = key_field->size;
+
+ if (key_field->flags & HIST_FIELD_FL_STRING) {
+ struct ftrace_event_field *field;
+
+ field = key_field->field;
+ if (field->filter_type == FILTER_DYN_STRING)
+ size = *(u32 *)(rec + field->offset) >> 16;
+ else if (field->filter_type == FILTER_PTR_STRING)
+ size = strlen(key);
+ else if (field->filter_type == FILTER_STATIC_STRING)
+ size = field->size;
+
+ /* ensure NULL-termination */
+ if (size > key_field->size - 1)
+ size = key_field->size - 1;
+ }
+
+ memcpy(compound_key + key_field->offset, key, size);
+}
+
+static void event_hist_trigger(struct event_trigger_data *data, void *rec)
+{
+ struct hist_trigger_data *hist_data = data->private_data;
+ bool use_compound_key = (hist_data->n_keys > 1);
+ unsigned long entries[HIST_STACKTRACE_DEPTH];
+ char compound_key[HIST_KEY_SIZE_MAX];
+ struct stack_trace stacktrace;
+ struct hist_field *key_field;
+ struct tracing_map_elt *elt;
+ u64 field_contents;
+ void *key = NULL;
+ unsigned int i;
+
+ memset(compound_key, 0, hist_data->key_size);
+
+ for_each_hist_key_field(i, hist_data) {
+ key_field = hist_data->fields[i];
+
+ if (key_field->flags & HIST_FIELD_FL_STACKTRACE) {
+ stacktrace.max_entries = HIST_STACKTRACE_DEPTH;
+ stacktrace.entries = entries;
+ stacktrace.nr_entries = 0;
+ stacktrace.skip = HIST_STACKTRACE_SKIP;
+
+ memset(stacktrace.entries, 0, HIST_STACKTRACE_SIZE);
+ save_stack_trace(&stacktrace);
+
+ key = entries;
+ } else {
+ field_contents = key_field->fn(key_field, rec);
+ if (key_field->flags & HIST_FIELD_FL_STRING) {
+ key = (void *)(unsigned long)field_contents;
+ use_compound_key = true;
+ } else
+ key = (void *)&field_contents;
+ }
+
+ if (use_compound_key)
+ add_to_key(compound_key, key, key_field, rec);
+ }
+
+ if (use_compound_key)
+ key = compound_key;
+
+ elt = tracing_map_insert(hist_data->map, key);
+ if (elt)
+ hist_trigger_elt_update(hist_data, elt, rec);
+}
+
+static void hist_trigger_stacktrace_print(struct seq_file *m,
+ unsigned long *stacktrace_entries,
+ unsigned int max_entries)
+{
+ char str[KSYM_SYMBOL_LEN];
+ unsigned int spaces = 8;
+ unsigned int i;
+
+ for (i = 0; i < max_entries; i++) {
+ if (stacktrace_entries[i] == ULONG_MAX)
+ return;
+
+ seq_printf(m, "%*c", 1 + spaces, ' ');
+ sprint_symbol(str, stacktrace_entries[i]);
+ seq_printf(m, "%s\n", str);
+ }
+}
+
+static void
+hist_trigger_entry_print(struct seq_file *m,
+ struct hist_trigger_data *hist_data, void *key,
+ struct tracing_map_elt *elt)
+{
+ struct hist_field *key_field;
+ char str[KSYM_SYMBOL_LEN];
+ bool multiline = false;
+ unsigned int i;
+ u64 uval;
+
+ seq_puts(m, "{ ");
+
+ for_each_hist_key_field(i, hist_data) {
+ key_field = hist_data->fields[i];
+
+ if (i > hist_data->n_vals)
+ seq_puts(m, ", ");
+
+ if (key_field->flags & HIST_FIELD_FL_HEX) {
+ uval = *(u64 *)(key + key_field->offset);
+ seq_printf(m, "%s: %llx",
+ key_field->field->name, uval);
+ } else if (key_field->flags & HIST_FIELD_FL_SYM) {
+ uval = *(u64 *)(key + key_field->offset);
+ sprint_symbol_no_offset(str, uval);
+ seq_printf(m, "%s: [%llx] %-45s",
+ key_field->field->name, uval, str);
+ } else if (key_field->flags & HIST_FIELD_FL_SYM_OFFSET) {
+ uval = *(u64 *)(key + key_field->offset);
+ sprint_symbol(str, uval);
+ seq_printf(m, "%s: [%llx] %-55s",
+ key_field->field->name, uval, str);
+ } else if (key_field->flags & HIST_FIELD_FL_EXECNAME) {
+ char *comm = elt->private_data;
+
+ uval = *(u64 *)(key + key_field->offset);
+ seq_printf(m, "%s: %-16s[%10llu]",
+ key_field->field->name, comm, uval);
+ } else if (key_field->flags & HIST_FIELD_FL_SYSCALL) {
+ const char *syscall_name;
+
+ uval = *(u64 *)(key + key_field->offset);
+ syscall_name = get_syscall_name(uval);
+ if (!syscall_name)
+ syscall_name = "unknown_syscall";
+
+ seq_printf(m, "%s: %-30s[%3llu]",
+ key_field->field->name, syscall_name, uval);
+ } else if (key_field->flags & HIST_FIELD_FL_STACKTRACE) {
+ seq_puts(m, "stacktrace:\n");
+ hist_trigger_stacktrace_print(m,
+ key + key_field->offset,
+ HIST_STACKTRACE_DEPTH);
+ multiline = true;
+ } else if (key_field->flags & HIST_FIELD_FL_LOG2) {
+ seq_printf(m, "%s: ~ 2^%-2llu", key_field->field->name,
+ *(u64 *)(key + key_field->offset));
+ } else if (key_field->flags & HIST_FIELD_FL_STRING) {
+ seq_printf(m, "%s: %-50s", key_field->field->name,
+ (char *)(key + key_field->offset));
+ } else {
+ uval = *(u64 *)(key + key_field->offset);
+ seq_printf(m, "%s: %10llu", key_field->field->name,
+ uval);
+ }
+ }
+
+ if (!multiline)
+ seq_puts(m, " ");
+
+ seq_puts(m, "}");
+
+ seq_printf(m, " hitcount: %10llu",
+ tracing_map_read_sum(elt, HITCOUNT_IDX));
+
+ for (i = 1; i < hist_data->n_vals; i++) {
+ if (hist_data->fields[i]->flags & HIST_FIELD_FL_HEX) {
+ seq_printf(m, " %s: %10llx",
+ hist_data->fields[i]->field->name,
+ tracing_map_read_sum(elt, i));
+ } else {
+ seq_printf(m, " %s: %10llu",
+ hist_data->fields[i]->field->name,
+ tracing_map_read_sum(elt, i));
+ }
+ }
+
+ seq_puts(m, "\n");
+}
+
+static int print_entries(struct seq_file *m,
+ struct hist_trigger_data *hist_data)
+{
+ struct tracing_map_sort_entry **sort_entries = NULL;
+ struct tracing_map *map = hist_data->map;
+ int i, n_entries;
+
+ n_entries = tracing_map_sort_entries(map, hist_data->sort_keys,
+ hist_data->n_sort_keys,
+ &sort_entries);
+ if (n_entries < 0)
+ return n_entries;
+
+ for (i = 0; i < n_entries; i++)
+ hist_trigger_entry_print(m, hist_data,
+ sort_entries[i]->key,
+ sort_entries[i]->elt);
+
+ tracing_map_destroy_sort_entries(sort_entries, n_entries);
+
+ return n_entries;
+}
+
+static void hist_trigger_show(struct seq_file *m,
+ struct event_trigger_data *data, int n)
+{
+ struct hist_trigger_data *hist_data;
+ int n_entries, ret = 0;
+
+ if (n > 0)
+ seq_puts(m, "\n\n");
+
+ seq_puts(m, "# event histogram\n#\n# trigger info: ");
+ data->ops->print(m, data->ops, data);
+ seq_puts(m, "#\n\n");
+
+ hist_data = data->private_data;
+ n_entries = print_entries(m, hist_data);
+ if (n_entries < 0) {
+ ret = n_entries;
+ n_entries = 0;
+ }
+
+ seq_printf(m, "\nTotals:\n Hits: %llu\n Entries: %u\n Dropped: %llu\n",
+ (u64)atomic64_read(&hist_data->map->hits),
+ n_entries, (u64)atomic64_read(&hist_data->map->drops));
+}
+
+static int hist_show(struct seq_file *m, void *v)
+{
+ struct event_trigger_data *data;
+ struct trace_event_file *event_file;
+ int n = 0, ret = 0;
+
+ mutex_lock(&event_mutex);
+
+ event_file = event_file_data(m->private);
+ if (unlikely(!event_file)) {
+ ret = -ENODEV;
+ goto out_unlock;
+ }
+
+ list_for_each_entry_rcu(data, &event_file->triggers, list) {
+ if (data->cmd_ops->trigger_type == ETT_EVENT_HIST)
+ hist_trigger_show(m, data, n++);
+ }
+
+ out_unlock:
+ mutex_unlock(&event_mutex);
+
+ return ret;
+}
+
+static int event_hist_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, hist_show, file);
+}
+
+const struct file_operations event_hist_fops = {
+ .open = event_hist_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
+static const char *get_hist_field_flags(struct hist_field *hist_field)
+{
+ const char *flags_str = NULL;
+
+ if (hist_field->flags & HIST_FIELD_FL_HEX)
+ flags_str = "hex";
+ else if (hist_field->flags & HIST_FIELD_FL_SYM)
+ flags_str = "sym";
+ else if (hist_field->flags & HIST_FIELD_FL_SYM_OFFSET)
+ flags_str = "sym-offset";
+ else if (hist_field->flags & HIST_FIELD_FL_EXECNAME)
+ flags_str = "execname";
+ else if (hist_field->flags & HIST_FIELD_FL_SYSCALL)
+ flags_str = "syscall";
+ else if (hist_field->flags & HIST_FIELD_FL_LOG2)
+ flags_str = "log2";
+
+ return flags_str;
+}
+
+static void hist_field_print(struct seq_file *m, struct hist_field *hist_field)
+{
+ seq_printf(m, "%s", hist_field->field->name);
+ if (hist_field->flags) {
+ const char *flags_str = get_hist_field_flags(hist_field);
+
+ if (flags_str)
+ seq_printf(m, ".%s", flags_str);
+ }
+}
+
+static int event_hist_trigger_print(struct seq_file *m,
+ struct event_trigger_ops *ops,
+ struct event_trigger_data *data)
+{
+ struct hist_trigger_data *hist_data = data->private_data;
+ struct hist_field *key_field;
+ unsigned int i;
+
+ seq_puts(m, "hist:");
+
+ if (data->name)
+ seq_printf(m, "%s:", data->name);
+
+ seq_puts(m, "keys=");
+
+ for_each_hist_key_field(i, hist_data) {
+ key_field = hist_data->fields[i];
+
+ if (i > hist_data->n_vals)
+ seq_puts(m, ",");
+
+ if (key_field->flags & HIST_FIELD_FL_STACKTRACE)
+ seq_puts(m, "stacktrace");
+ else
+ hist_field_print(m, key_field);
+ }
+
+ seq_puts(m, ":vals=");
+
+ for_each_hist_val_field(i, hist_data) {
+ if (i == HITCOUNT_IDX)
+ seq_puts(m, "hitcount");
+ else {
+ seq_puts(m, ",");
+ hist_field_print(m, hist_data->fields[i]);
+ }
+ }
+
+ seq_puts(m, ":sort=");
+
+ for (i = 0; i < hist_data->n_sort_keys; i++) {
+ struct tracing_map_sort_key *sort_key;
+
+ sort_key = &hist_data->sort_keys[i];
+
+ if (i > 0)
+ seq_puts(m, ",");
+
+ if (sort_key->field_idx == HITCOUNT_IDX)
+ seq_puts(m, "hitcount");
+ else {
+ unsigned int idx = sort_key->field_idx;
+
+ if (WARN_ON(idx >= TRACING_MAP_FIELDS_MAX))
+ return -EINVAL;
+
+ hist_field_print(m, hist_data->fields[idx]);
+ }
+
+ if (sort_key->descending)
+ seq_puts(m, ".descending");
+ }
+
+ seq_printf(m, ":size=%u", (1 << hist_data->map->map_bits));
+
+ if (data->filter_str)
+ seq_printf(m, " if %s", data->filter_str);
+
+ if (data->paused)
+ seq_puts(m, " [paused]");
+ else
+ seq_puts(m, " [active]");
+
+ seq_putc(m, '\n');
+
+ return 0;
+}
+
+static int event_hist_trigger_init(struct event_trigger_ops *ops,
+ struct event_trigger_data *data)
+{
+ struct hist_trigger_data *hist_data = data->private_data;
+
+ if (!data->ref && hist_data->attrs->name)
+ save_named_trigger(hist_data->attrs->name, data);
+
+ data->ref++;
+
+ return 0;
+}
+
+static void event_hist_trigger_free(struct event_trigger_ops *ops,
+ struct event_trigger_data *data)
+{
+ struct hist_trigger_data *hist_data = data->private_data;
+
+ if (WARN_ON_ONCE(data->ref <= 0))
+ return;
+
+ data->ref--;
+ if (!data->ref) {
+ if (data->name)
+ del_named_trigger(data);
+ trigger_data_free(data);
+ destroy_hist_data(hist_data);
+ }
+}
+
+static struct event_trigger_ops event_hist_trigger_ops = {
+ .func = event_hist_trigger,
+ .print = event_hist_trigger_print,
+ .init = event_hist_trigger_init,
+ .free = event_hist_trigger_free,
+};
+
+static int event_hist_trigger_named_init(struct event_trigger_ops *ops,
+ struct event_trigger_data *data)
+{
+ data->ref++;
+
+ save_named_trigger(data->named_data->name, data);
+
+ event_hist_trigger_init(ops, data->named_data);
+
+ return 0;
+}
+
+static void event_hist_trigger_named_free(struct event_trigger_ops *ops,
+ struct event_trigger_data *data)
+{
+ if (WARN_ON_ONCE(data->ref <= 0))
+ return;
+
+ event_hist_trigger_free(ops, data->named_data);
+
+ data->ref--;
+ if (!data->ref) {
+ del_named_trigger(data);
+ trigger_data_free(data);
+ }
+}
+
+static struct event_trigger_ops event_hist_trigger_named_ops = {
+ .func = event_hist_trigger,
+ .print = event_hist_trigger_print,
+ .init = event_hist_trigger_named_init,
+ .free = event_hist_trigger_named_free,
+};
+
+static struct event_trigger_ops *event_hist_get_trigger_ops(char *cmd,
+ char *param)
+{
+ return &event_hist_trigger_ops;
+}
+
+static void hist_clear(struct event_trigger_data *data)
+{
+ struct hist_trigger_data *hist_data = data->private_data;
+
+ if (data->name)
+ pause_named_trigger(data);
+
+ synchronize_sched();
+
+ tracing_map_clear(hist_data->map);
+
+ if (data->name)
+ unpause_named_trigger(data);
+}
+
+static bool compatible_field(struct ftrace_event_field *field,
+ struct ftrace_event_field *test_field)
+{
+ if (field == test_field)
+ return true;
+ if (field == NULL || test_field == NULL)
+ return false;
+ if (strcmp(field->name, test_field->name) != 0)
+ return false;
+ if (strcmp(field->type, test_field->type) != 0)
+ return false;
+ if (field->size != test_field->size)
+ return false;
+ if (field->is_signed != test_field->is_signed)
+ return false;
+
+ return true;
+}
+
+static bool hist_trigger_match(struct event_trigger_data *data,
+ struct event_trigger_data *data_test,
+ struct event_trigger_data *named_data,
+ bool ignore_filter)
+{
+ struct tracing_map_sort_key *sort_key, *sort_key_test;
+ struct hist_trigger_data *hist_data, *hist_data_test;
+ struct hist_field *key_field, *key_field_test;
+ unsigned int i;
+
+ if (named_data && (named_data != data_test) &&
+ (named_data != data_test->named_data))
+ return false;
+
+ if (!named_data && is_named_trigger(data_test))
+ return false;
+
+ hist_data = data->private_data;
+ hist_data_test = data_test->private_data;
+
+ if (hist_data->n_vals != hist_data_test->n_vals ||
+ hist_data->n_fields != hist_data_test->n_fields ||
+ hist_data->n_sort_keys != hist_data_test->n_sort_keys)
+ return false;
+
+ if (!ignore_filter) {
+ if ((data->filter_str && !data_test->filter_str) ||
+ (!data->filter_str && data_test->filter_str))
+ return false;
+ }
+
+ for_each_hist_field(i, hist_data) {
+ key_field = hist_data->fields[i];
+ key_field_test = hist_data_test->fields[i];
+
+ if (key_field->flags != key_field_test->flags)
+ return false;
+ if (!compatible_field(key_field->field, key_field_test->field))
+ return false;
+ if (key_field->offset != key_field_test->offset)
+ return false;
+ }
+
+ for (i = 0; i < hist_data->n_sort_keys; i++) {
+ sort_key = &hist_data->sort_keys[i];
+ sort_key_test = &hist_data_test->sort_keys[i];
+
+ if (sort_key->field_idx != sort_key_test->field_idx ||
+ sort_key->descending != sort_key_test->descending)
+ return false;
+ }
+
+ if (!ignore_filter && data->filter_str &&
+ (strcmp(data->filter_str, data_test->filter_str) != 0))
+ return false;
+
+ return true;
+}
+
+static int hist_register_trigger(char *glob, struct event_trigger_ops *ops,
+ struct event_trigger_data *data,
+ struct trace_event_file *file)
+{
+ struct hist_trigger_data *hist_data = data->private_data;
+ struct event_trigger_data *test, *named_data = NULL;
+ int ret = 0;
+
+ if (hist_data->attrs->name) {
+ named_data = find_named_trigger(hist_data->attrs->name);
+ if (named_data) {
+ if (!hist_trigger_match(data, named_data, named_data,
+ true)) {
+ ret = -EINVAL;
+ goto out;
+ }
+ }
+ }
+
+ if (hist_data->attrs->name && !named_data)
+ goto new;
+
+ list_for_each_entry_rcu(test, &file->triggers, list) {
+ if (test->cmd_ops->trigger_type == ETT_EVENT_HIST) {
+ if (!hist_trigger_match(data, test, named_data, false))
+ continue;
+ if (hist_data->attrs->pause)
+ test->paused = true;
+ else if (hist_data->attrs->cont)
+ test->paused = false;
+ else if (hist_data->attrs->clear)
+ hist_clear(test);
+ else
+ ret = -EEXIST;
+ goto out;
+ }
+ }
+ new:
+ if (hist_data->attrs->cont || hist_data->attrs->clear) {
+ ret = -ENOENT;
+ goto out;
+ }
+
+ if (named_data) {
+ destroy_hist_data(data->private_data);
+ data->private_data = named_data->private_data;
+ set_named_trigger_data(data, named_data);
+ data->ops = &event_hist_trigger_named_ops;
+ }
+
+ if (hist_data->attrs->pause)
+ data->paused = true;
+
+ if (data->ops->init) {
+ ret = data->ops->init(data->ops, data);
+ if (ret < 0)
+ goto out;
+ }
+
+ list_add_rcu(&data->list, &file->triggers);
+ ret++;
+
+ update_cond_flag(file);
+
+ if (trace_event_trigger_enable_disable(file, 1) < 0) {
+ list_del_rcu(&data->list);
+ update_cond_flag(file);
+ ret--;
+ }
+ out:
+ return ret;
+}
+
+static void hist_unregister_trigger(char *glob, struct event_trigger_ops *ops,
+ struct event_trigger_data *data,
+ struct trace_event_file *file)
+{
+ struct hist_trigger_data *hist_data = data->private_data;
+ struct event_trigger_data *test, *named_data = NULL;
+ bool unregistered = false;
+
+ if (hist_data->attrs->name)
+ named_data = find_named_trigger(hist_data->attrs->name);
+
+ list_for_each_entry_rcu(test, &file->triggers, list) {
+ if (test->cmd_ops->trigger_type == ETT_EVENT_HIST) {
+ if (!hist_trigger_match(data, test, named_data, false))
+ continue;
+ unregistered = true;
+ list_del_rcu(&test->list);
+ trace_event_trigger_enable_disable(file, 0);
+ update_cond_flag(file);
+ break;
+ }
+ }
+
+ if (unregistered && test->ops->free)
+ test->ops->free(test->ops, test);
+}
+
+static void hist_unreg_all(struct trace_event_file *file)
+{
+ struct event_trigger_data *test;
+
+ list_for_each_entry_rcu(test, &file->triggers, list) {
+ if (test->cmd_ops->trigger_type == ETT_EVENT_HIST) {
+ list_del_rcu(&test->list);
+ trace_event_trigger_enable_disable(file, 0);
+ update_cond_flag(file);
+ if (test->ops->free)
+ test->ops->free(test->ops, test);
+ }
+ }
+}
+
+static int event_hist_trigger_func(struct event_command *cmd_ops,
+ struct trace_event_file *file,
+ char *glob, char *cmd, char *param)
+{
+ unsigned int hist_trigger_bits = TRACING_MAP_BITS_DEFAULT;
+ struct event_trigger_data *trigger_data;
+ struct hist_trigger_attrs *attrs;
+ struct event_trigger_ops *trigger_ops;
+ struct hist_trigger_data *hist_data;
+ char *trigger;
+ int ret = 0;
+
+ if (!param)
+ return -EINVAL;
+
+ /* separate the trigger from the filter (k:v [if filter]) */
+ trigger = strsep(¶m, " \t");
+ if (!trigger)
+ return -EINVAL;
+
+ attrs = parse_hist_trigger_attrs(trigger);
+ if (IS_ERR(attrs))
+ return PTR_ERR(attrs);
+
+ if (attrs->map_bits)
+ hist_trigger_bits = attrs->map_bits;
+
+ hist_data = create_hist_data(hist_trigger_bits, attrs, file);
+ if (IS_ERR(hist_data)) {
+ destroy_hist_trigger_attrs(attrs);
+ return PTR_ERR(hist_data);
+ }
+
+ trigger_ops = cmd_ops->get_trigger_ops(cmd, trigger);
+
+ ret = -ENOMEM;
+ trigger_data = kzalloc(sizeof(*trigger_data), GFP_KERNEL);
+ if (!trigger_data)
+ goto out_free;
+
+ trigger_data->count = -1;
+ trigger_data->ops = trigger_ops;
+ trigger_data->cmd_ops = cmd_ops;
+
+ INIT_LIST_HEAD(&trigger_data->list);
+ RCU_INIT_POINTER(trigger_data->filter, NULL);
+
+ trigger_data->private_data = hist_data;
+
+ /* if param is non-empty, it's supposed to be a filter */
+ if (param && cmd_ops->set_filter) {
+ ret = cmd_ops->set_filter(param, trigger_data, file);
+ if (ret < 0)
+ goto out_free;
+ }
+
+ if (glob[0] == '!') {
+ cmd_ops->unreg(glob+1, trigger_ops, trigger_data, file);
+ ret = 0;
+ goto out_free;
+ }
+
+ ret = cmd_ops->reg(glob, trigger_ops, trigger_data, file);
+ /*
+ * The above returns on success the # of triggers registered,
+ * but if it didn't register any it returns zero. Consider no
+ * triggers registered a failure too.
+ */
+ if (!ret) {
+ if (!(attrs->pause || attrs->cont || attrs->clear))
+ ret = -ENOENT;
+ goto out_free;
+ } else if (ret < 0)
+ goto out_free;
+ /* Just return zero, not the number of registered triggers */
+ ret = 0;
+ out:
+ return ret;
+ out_free:
+ if (cmd_ops->set_filter)
+ cmd_ops->set_filter(NULL, trigger_data, NULL);
+
+ kfree(trigger_data);
+
+ destroy_hist_data(hist_data);
+ goto out;
+}
+
+static struct event_command trigger_hist_cmd = {
+ .name = "hist",
+ .trigger_type = ETT_EVENT_HIST,
+ .flags = EVENT_CMD_FL_NEEDS_REC,
+ .func = event_hist_trigger_func,
+ .reg = hist_register_trigger,
+ .unreg = hist_unregister_trigger,
+ .unreg_all = hist_unreg_all,
+ .get_trigger_ops = event_hist_get_trigger_ops,
+ .set_filter = set_trigger_filter,
+};
+
+__init int register_trigger_hist_cmd(void)
+{
+ int ret;
+
+ ret = register_event_command(&trigger_hist_cmd);
+ WARN_ON(ret < 0);
+
+ return ret;
+}
+
+static void
+hist_enable_trigger(struct event_trigger_data *data, void *rec)
+{
+ struct enable_trigger_data *enable_data = data->private_data;
+ struct event_trigger_data *test;
+
+ list_for_each_entry_rcu(test, &enable_data->file->triggers, list) {
+ if (test->cmd_ops->trigger_type == ETT_EVENT_HIST) {
+ if (enable_data->enable)
+ test->paused = false;
+ else
+ test->paused = true;
+ }
+ }
+}
+
+static void
+hist_enable_count_trigger(struct event_trigger_data *data, void *rec)
+{
+ if (!data->count)
+ return;
+
+ if (data->count != -1)
+ (data->count)--;
+
+ hist_enable_trigger(data, rec);
+}
+
+static struct event_trigger_ops hist_enable_trigger_ops = {
+ .func = hist_enable_trigger,
+ .print = event_enable_trigger_print,
+ .init = event_trigger_init,
+ .free = event_enable_trigger_free,
+};
+
+static struct event_trigger_ops hist_enable_count_trigger_ops = {
+ .func = hist_enable_count_trigger,
+ .print = event_enable_trigger_print,
+ .init = event_trigger_init,
+ .free = event_enable_trigger_free,
+};
+
+static struct event_trigger_ops hist_disable_trigger_ops = {
+ .func = hist_enable_trigger,
+ .print = event_enable_trigger_print,
+ .init = event_trigger_init,
+ .free = event_enable_trigger_free,
+};
+
+static struct event_trigger_ops hist_disable_count_trigger_ops = {
+ .func = hist_enable_count_trigger,
+ .print = event_enable_trigger_print,
+ .init = event_trigger_init,
+ .free = event_enable_trigger_free,
+};
+
+static struct event_trigger_ops *
+hist_enable_get_trigger_ops(char *cmd, char *param)
+{
+ struct event_trigger_ops *ops;
+ bool enable;
+
+ enable = (strcmp(cmd, ENABLE_HIST_STR) == 0);
+
+ if (enable)
+ ops = param ? &hist_enable_count_trigger_ops :
+ &hist_enable_trigger_ops;
+ else
+ ops = param ? &hist_disable_count_trigger_ops :
+ &hist_disable_trigger_ops;
+
+ return ops;
+}
+
+static void hist_enable_unreg_all(struct trace_event_file *file)
+{
+ struct event_trigger_data *test;
+
+ list_for_each_entry_rcu(test, &file->triggers, list) {
+ if (test->cmd_ops->trigger_type == ETT_HIST_ENABLE) {
+ list_del_rcu(&test->list);
+ update_cond_flag(file);
+ trace_event_trigger_enable_disable(file, 0);
+ if (test->ops->free)
+ test->ops->free(test->ops, test);
+ }
+ }
+}
+
+static struct event_command trigger_hist_enable_cmd = {
+ .name = ENABLE_HIST_STR,
+ .trigger_type = ETT_HIST_ENABLE,
+ .func = event_enable_trigger_func,
+ .reg = event_enable_register_trigger,
+ .unreg = event_enable_unregister_trigger,
+ .unreg_all = hist_enable_unreg_all,
+ .get_trigger_ops = hist_enable_get_trigger_ops,
+ .set_filter = set_trigger_filter,
+};
+
+static struct event_command trigger_hist_disable_cmd = {
+ .name = DISABLE_HIST_STR,
+ .trigger_type = ETT_HIST_ENABLE,
+ .func = event_enable_trigger_func,
+ .reg = event_enable_register_trigger,
+ .unreg = event_enable_unregister_trigger,
+ .unreg_all = hist_enable_unreg_all,
+ .get_trigger_ops = hist_enable_get_trigger_ops,
+ .set_filter = set_trigger_filter,
+};
+
+static __init void unregister_trigger_hist_enable_disable_cmds(void)
+{
+ unregister_event_command(&trigger_hist_enable_cmd);
+ unregister_event_command(&trigger_hist_disable_cmd);
+}
+
+__init int register_trigger_hist_enable_disable_cmds(void)
+{
+ int ret;
+
+ ret = register_event_command(&trigger_hist_enable_cmd);
+ if (WARN_ON(ret < 0))
+ return ret;
+ ret = register_event_command(&trigger_hist_disable_cmd);
+ if (WARN_ON(ret < 0))
+ unregister_trigger_hist_enable_disable_cmds();
+
+ return ret;
+}
diff --git a/kernel/trace/trace_events_trigger.c b/kernel/trace/trace_events_trigger.c
index d67992f..a975571 100644
--- a/kernel/trace/trace_events_trigger.c
+++ b/kernel/trace/trace_events_trigger.c
@@ -347,7 +347,7 @@
* Currently we only unregister event commands from __init, so mark
* this __init too.
*/
-static __init int unregister_event_command(struct event_command *cmd)
+__init int unregister_event_command(struct event_command *cmd)
{
struct event_command *p, *n;
int ret = -ENODEV;
@@ -641,6 +641,7 @@
trigger_data->ops = trigger_ops;
trigger_data->cmd_ops = cmd_ops;
INIT_LIST_HEAD(&trigger_data->list);
+ INIT_LIST_HEAD(&trigger_data->named_list);
if (glob[0] == '!') {
cmd_ops->unreg(glob+1, trigger_ops, trigger_data, file);
@@ -764,6 +765,148 @@
return ret;
}
+static LIST_HEAD(named_triggers);
+
+/**
+ * find_named_trigger - Find the common named trigger associated with @name
+ * @name: The name of the set of named triggers to find the common data for
+ *
+ * Named triggers are sets of triggers that share a common set of
+ * trigger data. The first named trigger registered with a given name
+ * owns the common trigger data that the others subsequently
+ * registered with the same name will reference. This function
+ * returns the common trigger data associated with that first
+ * registered instance.
+ *
+ * Return: the common trigger data for the given named trigger on
+ * success, NULL otherwise.
+ */
+struct event_trigger_data *find_named_trigger(const char *name)
+{
+ struct event_trigger_data *data;
+
+ if (!name)
+ return NULL;
+
+ list_for_each_entry(data, &named_triggers, named_list) {
+ if (data->named_data)
+ continue;
+ if (strcmp(data->name, name) == 0)
+ return data;
+ }
+
+ return NULL;
+}
+
+/**
+ * is_named_trigger - determine if a given trigger is a named trigger
+ * @test: The trigger data to test
+ *
+ * Return: true if 'test' is a named trigger, false otherwise.
+ */
+bool is_named_trigger(struct event_trigger_data *test)
+{
+ struct event_trigger_data *data;
+
+ list_for_each_entry(data, &named_triggers, named_list) {
+ if (test == data)
+ return true;
+ }
+
+ return false;
+}
+
+/**
+ * save_named_trigger - save the trigger in the named trigger list
+ * @name: The name of the named trigger set
+ * @data: The trigger data to save
+ *
+ * Return: 0 if successful, negative error otherwise.
+ */
+int save_named_trigger(const char *name, struct event_trigger_data *data)
+{
+ data->name = kstrdup(name, GFP_KERNEL);
+ if (!data->name)
+ return -ENOMEM;
+
+ list_add(&data->named_list, &named_triggers);
+
+ return 0;
+}
+
+/**
+ * del_named_trigger - delete a trigger from the named trigger list
+ * @data: The trigger data to delete
+ */
+void del_named_trigger(struct event_trigger_data *data)
+{
+ kfree(data->name);
+ data->name = NULL;
+
+ list_del(&data->named_list);
+}
+
+static void __pause_named_trigger(struct event_trigger_data *data, bool pause)
+{
+ struct event_trigger_data *test;
+
+ list_for_each_entry(test, &named_triggers, named_list) {
+ if (strcmp(test->name, data->name) == 0) {
+ if (pause) {
+ test->paused_tmp = test->paused;
+ test->paused = true;
+ } else {
+ test->paused = test->paused_tmp;
+ }
+ }
+ }
+}
+
+/**
+ * pause_named_trigger - Pause all named triggers with the same name
+ * @data: The trigger data of a named trigger to pause
+ *
+ * Pauses a named trigger along with all other triggers having the
+ * same name. Because named triggers share a common set of data,
+ * pausing only one is meaningless, so pausing one named trigger needs
+ * to pause all triggers with the same name.
+ */
+void pause_named_trigger(struct event_trigger_data *data)
+{
+ __pause_named_trigger(data, true);
+}
+
+/**
+ * unpause_named_trigger - Un-pause all named triggers with the same name
+ * @data: The trigger data of a named trigger to unpause
+ *
+ * Un-pauses a named trigger along with all other triggers having the
+ * same name. Because named triggers share a common set of data,
+ * unpausing only one is meaningless, so unpausing one named trigger
+ * needs to unpause all triggers with the same name.
+ */
+void unpause_named_trigger(struct event_trigger_data *data)
+{
+ __pause_named_trigger(data, false);
+}
+
+/**
+ * set_named_trigger_data - Associate common named trigger data
+ * @data: The trigger data of a named trigger to unpause
+ *
+ * Named triggers are sets of triggers that share a common set of
+ * trigger data. The first named trigger registered with a given name
+ * owns the common trigger data that the others subsequently
+ * registered with the same name will reference. This function
+ * associates the common trigger data from the first trigger with the
+ * given trigger.
+ */
+void set_named_trigger_data(struct event_trigger_data *data,
+ struct event_trigger_data *named_data)
+{
+ data->named_data = named_data;
+}
+
static void
traceon_trigger(struct event_trigger_data *data, void *rec)
{
@@ -1062,15 +1205,6 @@
unregister_event_command(&trigger_traceoff_cmd);
}
-/* Avoid typos */
-#define ENABLE_EVENT_STR "enable_event"
-#define DISABLE_EVENT_STR "disable_event"
-
-struct enable_trigger_data {
- struct trace_event_file *file;
- bool enable;
-};
-
static void
event_enable_trigger(struct event_trigger_data *data, void *rec)
{
@@ -1100,14 +1234,16 @@
event_enable_trigger(data, rec);
}
-static int
-event_enable_trigger_print(struct seq_file *m, struct event_trigger_ops *ops,
- struct event_trigger_data *data)
+int event_enable_trigger_print(struct seq_file *m,
+ struct event_trigger_ops *ops,
+ struct event_trigger_data *data)
{
struct enable_trigger_data *enable_data = data->private_data;
seq_printf(m, "%s:%s:%s",
- enable_data->enable ? ENABLE_EVENT_STR : DISABLE_EVENT_STR,
+ enable_data->hist ?
+ (enable_data->enable ? ENABLE_HIST_STR : DISABLE_HIST_STR) :
+ (enable_data->enable ? ENABLE_EVENT_STR : DISABLE_EVENT_STR),
enable_data->file->event_call->class->system,
trace_event_name(enable_data->file->event_call));
@@ -1124,9 +1260,8 @@
return 0;
}
-static void
-event_enable_trigger_free(struct event_trigger_ops *ops,
- struct event_trigger_data *data)
+void event_enable_trigger_free(struct event_trigger_ops *ops,
+ struct event_trigger_data *data)
{
struct enable_trigger_data *enable_data = data->private_data;
@@ -1171,10 +1306,9 @@
.free = event_enable_trigger_free,
};
-static int
-event_enable_trigger_func(struct event_command *cmd_ops,
- struct trace_event_file *file,
- char *glob, char *cmd, char *param)
+int event_enable_trigger_func(struct event_command *cmd_ops,
+ struct trace_event_file *file,
+ char *glob, char *cmd, char *param)
{
struct trace_event_file *event_enable_file;
struct enable_trigger_data *enable_data;
@@ -1183,6 +1317,7 @@
struct trace_array *tr = file->tr;
const char *system;
const char *event;
+ bool hist = false;
char *trigger;
char *number;
bool enable;
@@ -1207,8 +1342,15 @@
if (!event_enable_file)
goto out;
- enable = strcmp(cmd, ENABLE_EVENT_STR) == 0;
+#ifdef CONFIG_HIST_TRIGGERS
+ hist = ((strcmp(cmd, ENABLE_HIST_STR) == 0) ||
+ (strcmp(cmd, DISABLE_HIST_STR) == 0));
+ enable = ((strcmp(cmd, ENABLE_EVENT_STR) == 0) ||
+ (strcmp(cmd, ENABLE_HIST_STR) == 0));
+#else
+ enable = strcmp(cmd, ENABLE_EVENT_STR) == 0;
+#endif
trigger_ops = cmd_ops->get_trigger_ops(cmd, trigger);
ret = -ENOMEM;
@@ -1228,6 +1370,7 @@
INIT_LIST_HEAD(&trigger_data->list);
RCU_INIT_POINTER(trigger_data->filter, NULL);
+ enable_data->hist = hist;
enable_data->enable = enable;
enable_data->file = event_enable_file;
trigger_data->private_data = enable_data;
@@ -1305,10 +1448,10 @@
goto out;
}
-static int event_enable_register_trigger(char *glob,
- struct event_trigger_ops *ops,
- struct event_trigger_data *data,
- struct trace_event_file *file)
+int event_enable_register_trigger(char *glob,
+ struct event_trigger_ops *ops,
+ struct event_trigger_data *data,
+ struct trace_event_file *file)
{
struct enable_trigger_data *enable_data = data->private_data;
struct enable_trigger_data *test_enable_data;
@@ -1318,6 +1461,8 @@
list_for_each_entry_rcu(test, &file->triggers, list) {
test_enable_data = test->private_data;
if (test_enable_data &&
+ (test->cmd_ops->trigger_type ==
+ data->cmd_ops->trigger_type) &&
(test_enable_data->file == enable_data->file)) {
ret = -EEXIST;
goto out;
@@ -1343,10 +1488,10 @@
return ret;
}
-static void event_enable_unregister_trigger(char *glob,
- struct event_trigger_ops *ops,
- struct event_trigger_data *test,
- struct trace_event_file *file)
+void event_enable_unregister_trigger(char *glob,
+ struct event_trigger_ops *ops,
+ struct event_trigger_data *test,
+ struct trace_event_file *file)
{
struct enable_trigger_data *test_enable_data = test->private_data;
struct enable_trigger_data *enable_data;
@@ -1356,6 +1501,8 @@
list_for_each_entry_rcu(data, &file->triggers, list) {
enable_data = data->private_data;
if (enable_data &&
+ (data->cmd_ops->trigger_type ==
+ test->cmd_ops->trigger_type) &&
(enable_data->file == test_enable_data->file)) {
unregistered = true;
list_del_rcu(&data->list);
@@ -1375,8 +1522,12 @@
struct event_trigger_ops *ops;
bool enable;
+#ifdef CONFIG_HIST_TRIGGERS
+ enable = ((strcmp(cmd, ENABLE_EVENT_STR) == 0) ||
+ (strcmp(cmd, ENABLE_HIST_STR) == 0));
+#else
enable = strcmp(cmd, ENABLE_EVENT_STR) == 0;
-
+#endif
if (enable)
ops = param ? &event_enable_count_trigger_ops :
&event_enable_trigger_ops;
@@ -1447,6 +1598,8 @@
register_trigger_snapshot_cmd();
register_trigger_stacktrace_cmd();
register_trigger_enable_disable_cmds();
+ register_trigger_hist_enable_disable_cmds();
+ register_trigger_hist_cmd();
return 0;
}
diff --git a/kernel/trace/tracing_map.c b/kernel/trace/tracing_map.c
new file mode 100644
index 0000000..0a689bb
--- /dev/null
+++ b/kernel/trace/tracing_map.c
@@ -0,0 +1,1062 @@
+/*
+ * tracing_map - lock-free map for tracing
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * Copyright (C) 2015 Tom Zanussi <tom.zanussi@linux.intel.com>
+ *
+ * tracing_map implementation inspired by lock-free map algorithms
+ * originated by Dr. Cliff Click:
+ *
+ * http://www.azulsystems.com/blog/cliff/2007-03-26-non-blocking-hashtable
+ * http://www.azulsystems.com/events/javaone_2007/2007_LockFreeHash.pdf
+ */
+
+#include <linux/vmalloc.h>
+#include <linux/jhash.h>
+#include <linux/slab.h>
+#include <linux/sort.h>
+
+#include "tracing_map.h"
+#include "trace.h"
+
+/*
+ * NOTE: For a detailed description of the data structures used by
+ * these functions (such as tracing_map_elt) please see the overview
+ * of tracing_map data structures at the beginning of tracing_map.h.
+ */
+
+/**
+ * tracing_map_update_sum - Add a value to a tracing_map_elt's sum field
+ * @elt: The tracing_map_elt
+ * @i: The index of the given sum associated with the tracing_map_elt
+ * @n: The value to add to the sum
+ *
+ * Add n to sum i associated with the specified tracing_map_elt
+ * instance. The index i is the index returned by the call to
+ * tracing_map_add_sum_field() when the tracing map was set up.
+ */
+void tracing_map_update_sum(struct tracing_map_elt *elt, unsigned int i, u64 n)
+{
+ atomic64_add(n, &elt->fields[i].sum);
+}
+
+/**
+ * tracing_map_read_sum - Return the value of a tracing_map_elt's sum field
+ * @elt: The tracing_map_elt
+ * @i: The index of the given sum associated with the tracing_map_elt
+ *
+ * Retrieve the value of the sum i associated with the specified
+ * tracing_map_elt instance. The index i is the index returned by the
+ * call to tracing_map_add_sum_field() when the tracing map was set
+ * up.
+ *
+ * Return: The sum associated with field i for elt.
+ */
+u64 tracing_map_read_sum(struct tracing_map_elt *elt, unsigned int i)
+{
+ return (u64)atomic64_read(&elt->fields[i].sum);
+}
+
+int tracing_map_cmp_string(void *val_a, void *val_b)
+{
+ char *a = val_a;
+ char *b = val_b;
+
+ return strcmp(a, b);
+}
+
+int tracing_map_cmp_none(void *val_a, void *val_b)
+{
+ return 0;
+}
+
+static int tracing_map_cmp_atomic64(void *val_a, void *val_b)
+{
+ u64 a = atomic64_read((atomic64_t *)val_a);
+ u64 b = atomic64_read((atomic64_t *)val_b);
+
+ return (a > b) ? 1 : ((a < b) ? -1 : 0);
+}
+
+#define DEFINE_TRACING_MAP_CMP_FN(type) \
+static int tracing_map_cmp_##type(void *val_a, void *val_b) \
+{ \
+ type a = *(type *)val_a; \
+ type b = *(type *)val_b; \
+ \
+ return (a > b) ? 1 : ((a < b) ? -1 : 0); \
+}
+
+DEFINE_TRACING_MAP_CMP_FN(s64);
+DEFINE_TRACING_MAP_CMP_FN(u64);
+DEFINE_TRACING_MAP_CMP_FN(s32);
+DEFINE_TRACING_MAP_CMP_FN(u32);
+DEFINE_TRACING_MAP_CMP_FN(s16);
+DEFINE_TRACING_MAP_CMP_FN(u16);
+DEFINE_TRACING_MAP_CMP_FN(s8);
+DEFINE_TRACING_MAP_CMP_FN(u8);
+
+tracing_map_cmp_fn_t tracing_map_cmp_num(int field_size,
+ int field_is_signed)
+{
+ tracing_map_cmp_fn_t fn = tracing_map_cmp_none;
+
+ switch (field_size) {
+ case 8:
+ if (field_is_signed)
+ fn = tracing_map_cmp_s64;
+ else
+ fn = tracing_map_cmp_u64;
+ break;
+ case 4:
+ if (field_is_signed)
+ fn = tracing_map_cmp_s32;
+ else
+ fn = tracing_map_cmp_u32;
+ break;
+ case 2:
+ if (field_is_signed)
+ fn = tracing_map_cmp_s16;
+ else
+ fn = tracing_map_cmp_u16;
+ break;
+ case 1:
+ if (field_is_signed)
+ fn = tracing_map_cmp_s8;
+ else
+ fn = tracing_map_cmp_u8;
+ break;
+ }
+
+ return fn;
+}
+
+static int tracing_map_add_field(struct tracing_map *map,
+ tracing_map_cmp_fn_t cmp_fn)
+{
+ int ret = -EINVAL;
+
+ if (map->n_fields < TRACING_MAP_FIELDS_MAX) {
+ ret = map->n_fields;
+ map->fields[map->n_fields++].cmp_fn = cmp_fn;
+ }
+
+ return ret;
+}
+
+/**
+ * tracing_map_add_sum_field - Add a field describing a tracing_map sum
+ * @map: The tracing_map
+ *
+ * Add a sum field to the key and return the index identifying it in
+ * the map and associated tracing_map_elts. This is the index used
+ * for instance to update a sum for a particular tracing_map_elt using
+ * tracing_map_update_sum() or reading it via tracing_map_read_sum().
+ *
+ * Return: The index identifying the field in the map and associated
+ * tracing_map_elts, or -EINVAL on error.
+ */
+int tracing_map_add_sum_field(struct tracing_map *map)
+{
+ return tracing_map_add_field(map, tracing_map_cmp_atomic64);
+}
+
+/**
+ * tracing_map_add_key_field - Add a field describing a tracing_map key
+ * @map: The tracing_map
+ * @offset: The offset within the key
+ * @cmp_fn: The comparison function that will be used to sort on the key
+ *
+ * Let the map know there is a key and that if it's used as a sort key
+ * to use cmp_fn.
+ *
+ * A key can be a subset of a compound key; for that purpose, the
+ * offset param is used to describe where within the the compound key
+ * the key referenced by this key field resides.
+ *
+ * Return: The index identifying the field in the map and associated
+ * tracing_map_elts, or -EINVAL on error.
+ */
+int tracing_map_add_key_field(struct tracing_map *map,
+ unsigned int offset,
+ tracing_map_cmp_fn_t cmp_fn)
+
+{
+ int idx = tracing_map_add_field(map, cmp_fn);
+
+ if (idx < 0)
+ return idx;
+
+ map->fields[idx].offset = offset;
+
+ map->key_idx[map->n_keys++] = idx;
+
+ return idx;
+}
+
+void tracing_map_array_clear(struct tracing_map_array *a)
+{
+ unsigned int i;
+
+ if (!a->pages)
+ return;
+
+ for (i = 0; i < a->n_pages; i++)
+ memset(a->pages[i], 0, PAGE_SIZE);
+}
+
+void tracing_map_array_free(struct tracing_map_array *a)
+{
+ unsigned int i;
+
+ if (!a)
+ return;
+
+ if (!a->pages) {
+ kfree(a);
+ return;
+ }
+
+ for (i = 0; i < a->n_pages; i++) {
+ if (!a->pages[i])
+ break;
+ free_page((unsigned long)a->pages[i]);
+ }
+}
+
+struct tracing_map_array *tracing_map_array_alloc(unsigned int n_elts,
+ unsigned int entry_size)
+{
+ struct tracing_map_array *a;
+ unsigned int i;
+
+ a = kzalloc(sizeof(*a), GFP_KERNEL);
+ if (!a)
+ return NULL;
+
+ a->entry_size_shift = fls(roundup_pow_of_two(entry_size) - 1);
+ a->entries_per_page = PAGE_SIZE / (1 << a->entry_size_shift);
+ a->n_pages = n_elts / a->entries_per_page;
+ if (!a->n_pages)
+ a->n_pages = 1;
+ a->entry_shift = fls(a->entries_per_page) - 1;
+ a->entry_mask = (1 << a->entry_shift) - 1;
+
+ a->pages = kcalloc(a->n_pages, sizeof(void *), GFP_KERNEL);
+ if (!a->pages)
+ goto free;
+
+ for (i = 0; i < a->n_pages; i++) {
+ a->pages[i] = (void *)get_zeroed_page(GFP_KERNEL);
+ if (!a->pages[i])
+ goto free;
+ }
+ out:
+ return a;
+ free:
+ tracing_map_array_free(a);
+ a = NULL;
+
+ goto out;
+}
+
+static void tracing_map_elt_clear(struct tracing_map_elt *elt)
+{
+ unsigned i;
+
+ for (i = 0; i < elt->map->n_fields; i++)
+ if (elt->fields[i].cmp_fn == tracing_map_cmp_atomic64)
+ atomic64_set(&elt->fields[i].sum, 0);
+
+ if (elt->map->ops && elt->map->ops->elt_clear)
+ elt->map->ops->elt_clear(elt);
+}
+
+static void tracing_map_elt_init_fields(struct tracing_map_elt *elt)
+{
+ unsigned int i;
+
+ tracing_map_elt_clear(elt);
+
+ for (i = 0; i < elt->map->n_fields; i++) {
+ elt->fields[i].cmp_fn = elt->map->fields[i].cmp_fn;
+
+ if (elt->fields[i].cmp_fn != tracing_map_cmp_atomic64)
+ elt->fields[i].offset = elt->map->fields[i].offset;
+ }
+}
+
+static void tracing_map_elt_free(struct tracing_map_elt *elt)
+{
+ if (!elt)
+ return;
+
+ if (elt->map->ops && elt->map->ops->elt_free)
+ elt->map->ops->elt_free(elt);
+ kfree(elt->fields);
+ kfree(elt->key);
+ kfree(elt);
+}
+
+static struct tracing_map_elt *tracing_map_elt_alloc(struct tracing_map *map)
+{
+ struct tracing_map_elt *elt;
+ int err = 0;
+
+ elt = kzalloc(sizeof(*elt), GFP_KERNEL);
+ if (!elt)
+ return ERR_PTR(-ENOMEM);
+
+ elt->map = map;
+
+ elt->key = kzalloc(map->key_size, GFP_KERNEL);
+ if (!elt->key) {
+ err = -ENOMEM;
+ goto free;
+ }
+
+ elt->fields = kcalloc(map->n_fields, sizeof(*elt->fields), GFP_KERNEL);
+ if (!elt->fields) {
+ err = -ENOMEM;
+ goto free;
+ }
+
+ tracing_map_elt_init_fields(elt);
+
+ if (map->ops && map->ops->elt_alloc) {
+ err = map->ops->elt_alloc(elt);
+ if (err)
+ goto free;
+ }
+ return elt;
+ free:
+ tracing_map_elt_free(elt);
+
+ return ERR_PTR(err);
+}
+
+static struct tracing_map_elt *get_free_elt(struct tracing_map *map)
+{
+ struct tracing_map_elt *elt = NULL;
+ int idx;
+
+ idx = atomic_inc_return(&map->next_elt);
+ if (idx < map->max_elts) {
+ elt = *(TRACING_MAP_ELT(map->elts, idx));
+ if (map->ops && map->ops->elt_init)
+ map->ops->elt_init(elt);
+ }
+
+ return elt;
+}
+
+static void tracing_map_free_elts(struct tracing_map *map)
+{
+ unsigned int i;
+
+ if (!map->elts)
+ return;
+
+ for (i = 0; i < map->max_elts; i++) {
+ tracing_map_elt_free(*(TRACING_MAP_ELT(map->elts, i)));
+ *(TRACING_MAP_ELT(map->elts, i)) = NULL;
+ }
+
+ tracing_map_array_free(map->elts);
+ map->elts = NULL;
+}
+
+static int tracing_map_alloc_elts(struct tracing_map *map)
+{
+ unsigned int i;
+
+ map->elts = tracing_map_array_alloc(map->max_elts,
+ sizeof(struct tracing_map_elt *));
+ if (!map->elts)
+ return -ENOMEM;
+
+ for (i = 0; i < map->max_elts; i++) {
+ *(TRACING_MAP_ELT(map->elts, i)) = tracing_map_elt_alloc(map);
+ if (IS_ERR(*(TRACING_MAP_ELT(map->elts, i)))) {
+ *(TRACING_MAP_ELT(map->elts, i)) = NULL;
+ tracing_map_free_elts(map);
+
+ return -ENOMEM;
+ }
+ }
+
+ return 0;
+}
+
+static inline bool keys_match(void *key, void *test_key, unsigned key_size)
+{
+ bool match = true;
+
+ if (memcmp(key, test_key, key_size))
+ match = false;
+
+ return match;
+}
+
+static inline struct tracing_map_elt *
+__tracing_map_insert(struct tracing_map *map, void *key, bool lookup_only)
+{
+ u32 idx, key_hash, test_key;
+ struct tracing_map_entry *entry;
+
+ key_hash = jhash(key, map->key_size, 0);
+ if (key_hash == 0)
+ key_hash = 1;
+ idx = key_hash >> (32 - (map->map_bits + 1));
+
+ while (1) {
+ idx &= (map->map_size - 1);
+ entry = TRACING_MAP_ENTRY(map->map, idx);
+ test_key = entry->key;
+
+ if (test_key && test_key == key_hash && entry->val &&
+ keys_match(key, entry->val->key, map->key_size)) {
+ atomic64_inc(&map->hits);
+ return entry->val;
+ }
+
+ if (!test_key) {
+ if (lookup_only)
+ break;
+
+ if (!cmpxchg(&entry->key, 0, key_hash)) {
+ struct tracing_map_elt *elt;
+
+ elt = get_free_elt(map);
+ if (!elt) {
+ atomic64_inc(&map->drops);
+ entry->key = 0;
+ break;
+ }
+
+ memcpy(elt->key, key, map->key_size);
+ entry->val = elt;
+ atomic64_inc(&map->hits);
+
+ return entry->val;
+ }
+ }
+
+ idx++;
+ }
+
+ return NULL;
+}
+
+/**
+ * tracing_map_insert - Insert key and/or retrieve val from a tracing_map
+ * @map: The tracing_map to insert into
+ * @key: The key to insert
+ *
+ * Inserts a key into a tracing_map and creates and returns a new
+ * tracing_map_elt for it, or if the key has already been inserted by
+ * a previous call, returns the tracing_map_elt already associated
+ * with it. When the map was created, the number of elements to be
+ * allocated for the map was specified (internally maintained as
+ * 'max_elts' in struct tracing_map), and that number of
+ * tracing_map_elts was created by tracing_map_init(). This is the
+ * pre-allocated pool of tracing_map_elts that tracing_map_insert()
+ * will allocate from when adding new keys. Once that pool is
+ * exhausted, tracing_map_insert() is useless and will return NULL to
+ * signal that state. There are two user-visible tracing_map
+ * variables, 'hits' and 'drops', which are updated by this function.
+ * Every time an element is either successfully inserted or retrieved,
+ * the 'hits' value is incrememented. Every time an element insertion
+ * fails, the 'drops' value is incremented.
+ *
+ * This is a lock-free tracing map insertion function implementing a
+ * modified form of Cliff Click's basic insertion algorithm. It
+ * requires the table size be a power of two. To prevent any
+ * possibility of an infinite loop we always make the internal table
+ * size double the size of the requested table size (max_elts * 2).
+ * Likewise, we never reuse a slot or resize or delete elements - when
+ * we've reached max_elts entries, we simply return NULL once we've
+ * run out of entries. Readers can at any point in time traverse the
+ * tracing map and safely access the key/val pairs.
+ *
+ * Return: the tracing_map_elt pointer val associated with the key.
+ * If this was a newly inserted key, the val will be a newly allocated
+ * and associated tracing_map_elt pointer val. If the key wasn't
+ * found and the pool of tracing_map_elts has been exhausted, NULL is
+ * returned and no further insertions will succeed.
+ */
+struct tracing_map_elt *tracing_map_insert(struct tracing_map *map, void *key)
+{
+ return __tracing_map_insert(map, key, false);
+}
+
+/**
+ * tracing_map_lookup - Retrieve val from a tracing_map
+ * @map: The tracing_map to perform the lookup on
+ * @key: The key to look up
+ *
+ * Looks up key in tracing_map and if found returns the matching
+ * tracing_map_elt. This is a lock-free lookup; see
+ * tracing_map_insert() for details on tracing_map and how it works.
+ * Every time an element is retrieved, the 'hits' value is
+ * incrememented. There is one user-visible tracing_map variable,
+ * 'hits', which is updated by this function. Every time an element
+ * is successfully retrieved, the 'hits' value is incrememented. The
+ * 'drops' value is never updated by this function.
+ *
+ * Return: the tracing_map_elt pointer val associated with the key.
+ * If the key wasn't found, NULL is returned.
+ */
+struct tracing_map_elt *tracing_map_lookup(struct tracing_map *map, void *key)
+{
+ return __tracing_map_insert(map, key, true);
+}
+
+/**
+ * tracing_map_destroy - Destroy a tracing_map
+ * @map: The tracing_map to destroy
+ *
+ * Frees a tracing_map along with its associated array of
+ * tracing_map_elts.
+ *
+ * Callers should make sure there are no readers or writers actively
+ * reading or inserting into the map before calling this.
+ */
+void tracing_map_destroy(struct tracing_map *map)
+{
+ if (!map)
+ return;
+
+ tracing_map_free_elts(map);
+
+ tracing_map_array_free(map->map);
+ kfree(map);
+}
+
+/**
+ * tracing_map_clear - Clear a tracing_map
+ * @map: The tracing_map to clear
+ *
+ * Resets the tracing map to a cleared or initial state. The
+ * tracing_map_elts are all cleared, and the array of struct
+ * tracing_map_entry is reset to an initialized state.
+ *
+ * Callers should make sure there are no writers actively inserting
+ * into the map before calling this.
+ */
+void tracing_map_clear(struct tracing_map *map)
+{
+ unsigned int i;
+
+ atomic_set(&map->next_elt, -1);
+ atomic64_set(&map->hits, 0);
+ atomic64_set(&map->drops, 0);
+
+ tracing_map_array_clear(map->map);
+
+ for (i = 0; i < map->max_elts; i++)
+ tracing_map_elt_clear(*(TRACING_MAP_ELT(map->elts, i)));
+}
+
+static void set_sort_key(struct tracing_map *map,
+ struct tracing_map_sort_key *sort_key)
+{
+ map->sort_key = *sort_key;
+}
+
+/**
+ * tracing_map_create - Create a lock-free map and element pool
+ * @map_bits: The size of the map (2 ** map_bits)
+ * @key_size: The size of the key for the map in bytes
+ * @ops: Optional client-defined tracing_map_ops instance
+ * @private_data: Client data associated with the map
+ *
+ * Creates and sets up a map to contain 2 ** map_bits number of
+ * elements (internally maintained as 'max_elts' in struct
+ * tracing_map). Before using, map fields should be added to the map
+ * with tracing_map_add_sum_field() and tracing_map_add_key_field().
+ * tracing_map_init() should then be called to allocate the array of
+ * tracing_map_elts, in order to avoid allocating anything in the map
+ * insertion path. The user-specified map size reflects the maximum
+ * number of elements that can be contained in the table requested by
+ * the user - internally we double that in order to keep the table
+ * sparse and keep collisions manageable.
+ *
+ * A tracing_map is a special-purpose map designed to aggregate or
+ * 'sum' one or more values associated with a specific object of type
+ * tracing_map_elt, which is attached by the map to a given key.
+ *
+ * tracing_map_create() sets up the map itself, and provides
+ * operations for inserting tracing_map_elts, but doesn't allocate the
+ * tracing_map_elts themselves, or provide a means for describing the
+ * keys or sums associated with the tracing_map_elts. All
+ * tracing_map_elts for a given map have the same set of sums and
+ * keys, which are defined by the client using the functions
+ * tracing_map_add_key_field() and tracing_map_add_sum_field(). Once
+ * the fields are defined, the pool of elements allocated for the map
+ * can be created, which occurs when the client code calls
+ * tracing_map_init().
+ *
+ * When tracing_map_init() returns, tracing_map_elt elements can be
+ * inserted into the map using tracing_map_insert(). When called,
+ * tracing_map_insert() grabs a free tracing_map_elt from the pool, or
+ * finds an existing match in the map and in either case returns it.
+ * The client can then use tracing_map_update_sum() and
+ * tracing_map_read_sum() to update or read a given sum field for the
+ * tracing_map_elt.
+ *
+ * The client can at any point retrieve and traverse the current set
+ * of inserted tracing_map_elts in a tracing_map, via
+ * tracing_map_sort_entries(). Sorting can be done on any field,
+ * including keys.
+ *
+ * See tracing_map.h for a description of tracing_map_ops.
+ *
+ * Return: the tracing_map pointer if successful, ERR_PTR if not.
+ */
+struct tracing_map *tracing_map_create(unsigned int map_bits,
+ unsigned int key_size,
+ const struct tracing_map_ops *ops,
+ void *private_data)
+{
+ struct tracing_map *map;
+ unsigned int i;
+
+ if (map_bits < TRACING_MAP_BITS_MIN ||
+ map_bits > TRACING_MAP_BITS_MAX)
+ return ERR_PTR(-EINVAL);
+
+ map = kzalloc(sizeof(*map), GFP_KERNEL);
+ if (!map)
+ return ERR_PTR(-ENOMEM);
+
+ map->map_bits = map_bits;
+ map->max_elts = (1 << map_bits);
+ atomic_set(&map->next_elt, -1);
+
+ map->map_size = (1 << (map_bits + 1));
+ map->ops = ops;
+
+ map->private_data = private_data;
+
+ map->map = tracing_map_array_alloc(map->map_size,
+ sizeof(struct tracing_map_entry));
+ if (!map->map)
+ goto free;
+
+ map->key_size = key_size;
+ for (i = 0; i < TRACING_MAP_KEYS_MAX; i++)
+ map->key_idx[i] = -1;
+ out:
+ return map;
+ free:
+ tracing_map_destroy(map);
+ map = ERR_PTR(-ENOMEM);
+
+ goto out;
+}
+
+/**
+ * tracing_map_init - Allocate and clear a map's tracing_map_elts
+ * @map: The tracing_map to initialize
+ *
+ * Allocates a clears a pool of tracing_map_elts equal to the
+ * user-specified size of 2 ** map_bits (internally maintained as
+ * 'max_elts' in struct tracing_map). Before using, the map fields
+ * should be added to the map with tracing_map_add_sum_field() and
+ * tracing_map_add_key_field(). tracing_map_init() should then be
+ * called to allocate the array of tracing_map_elts, in order to avoid
+ * allocating anything in the map insertion path. The user-specified
+ * map size reflects the max number of elements requested by the user
+ * - internally we double that in order to keep the table sparse and
+ * keep collisions manageable.
+ *
+ * See tracing_map.h for a description of tracing_map_ops.
+ *
+ * Return: the tracing_map pointer if successful, ERR_PTR if not.
+ */
+int tracing_map_init(struct tracing_map *map)
+{
+ int err;
+
+ if (map->n_fields < 2)
+ return -EINVAL; /* need at least 1 key and 1 val */
+
+ err = tracing_map_alloc_elts(map);
+ if (err)
+ return err;
+
+ tracing_map_clear(map);
+
+ return err;
+}
+
+static int cmp_entries_dup(const struct tracing_map_sort_entry **a,
+ const struct tracing_map_sort_entry **b)
+{
+ int ret = 0;
+
+ if (memcmp((*a)->key, (*b)->key, (*a)->elt->map->key_size))
+ ret = 1;
+
+ return ret;
+}
+
+static int cmp_entries_sum(const struct tracing_map_sort_entry **a,
+ const struct tracing_map_sort_entry **b)
+{
+ const struct tracing_map_elt *elt_a, *elt_b;
+ struct tracing_map_sort_key *sort_key;
+ struct tracing_map_field *field;
+ tracing_map_cmp_fn_t cmp_fn;
+ void *val_a, *val_b;
+ int ret = 0;
+
+ elt_a = (*a)->elt;
+ elt_b = (*b)->elt;
+
+ sort_key = &elt_a->map->sort_key;
+
+ field = &elt_a->fields[sort_key->field_idx];
+ cmp_fn = field->cmp_fn;
+
+ val_a = &elt_a->fields[sort_key->field_idx].sum;
+ val_b = &elt_b->fields[sort_key->field_idx].sum;
+
+ ret = cmp_fn(val_a, val_b);
+ if (sort_key->descending)
+ ret = -ret;
+
+ return ret;
+}
+
+static int cmp_entries_key(const struct tracing_map_sort_entry **a,
+ const struct tracing_map_sort_entry **b)
+{
+ const struct tracing_map_elt *elt_a, *elt_b;
+ struct tracing_map_sort_key *sort_key;
+ struct tracing_map_field *field;
+ tracing_map_cmp_fn_t cmp_fn;
+ void *val_a, *val_b;
+ int ret = 0;
+
+ elt_a = (*a)->elt;
+ elt_b = (*b)->elt;
+
+ sort_key = &elt_a->map->sort_key;
+
+ field = &elt_a->fields[sort_key->field_idx];
+
+ cmp_fn = field->cmp_fn;
+
+ val_a = elt_a->key + field->offset;
+ val_b = elt_b->key + field->offset;
+
+ ret = cmp_fn(val_a, val_b);
+ if (sort_key->descending)
+ ret = -ret;
+
+ return ret;
+}
+
+static void destroy_sort_entry(struct tracing_map_sort_entry *entry)
+{
+ if (!entry)
+ return;
+
+ if (entry->elt_copied)
+ tracing_map_elt_free(entry->elt);
+
+ kfree(entry);
+}
+
+/**
+ * tracing_map_destroy_sort_entries - Destroy an array of sort entries
+ * @entries: The entries to destroy
+ * @n_entries: The number of entries in the array
+ *
+ * Destroy the elements returned by a tracing_map_sort_entries() call.
+ */
+void tracing_map_destroy_sort_entries(struct tracing_map_sort_entry **entries,
+ unsigned int n_entries)
+{
+ unsigned int i;
+
+ for (i = 0; i < n_entries; i++)
+ destroy_sort_entry(entries[i]);
+
+ vfree(entries);
+}
+
+static struct tracing_map_sort_entry *
+create_sort_entry(void *key, struct tracing_map_elt *elt)
+{
+ struct tracing_map_sort_entry *sort_entry;
+
+ sort_entry = kzalloc(sizeof(*sort_entry), GFP_KERNEL);
+ if (!sort_entry)
+ return NULL;
+
+ sort_entry->key = key;
+ sort_entry->elt = elt;
+
+ return sort_entry;
+}
+
+static struct tracing_map_elt *copy_elt(struct tracing_map_elt *elt)
+{
+ struct tracing_map_elt *dup_elt;
+ unsigned int i;
+
+ dup_elt = tracing_map_elt_alloc(elt->map);
+ if (IS_ERR(dup_elt))
+ return NULL;
+
+ if (elt->map->ops && elt->map->ops->elt_copy)
+ elt->map->ops->elt_copy(dup_elt, elt);
+
+ dup_elt->private_data = elt->private_data;
+ memcpy(dup_elt->key, elt->key, elt->map->key_size);
+
+ for (i = 0; i < elt->map->n_fields; i++) {
+ atomic64_set(&dup_elt->fields[i].sum,
+ atomic64_read(&elt->fields[i].sum));
+ dup_elt->fields[i].cmp_fn = elt->fields[i].cmp_fn;
+ }
+
+ return dup_elt;
+}
+
+static int merge_dup(struct tracing_map_sort_entry **sort_entries,
+ unsigned int target, unsigned int dup)
+{
+ struct tracing_map_elt *target_elt, *elt;
+ bool first_dup = (target - dup) == 1;
+ int i;
+
+ if (first_dup) {
+ elt = sort_entries[target]->elt;
+ target_elt = copy_elt(elt);
+ if (!target_elt)
+ return -ENOMEM;
+ sort_entries[target]->elt = target_elt;
+ sort_entries[target]->elt_copied = true;
+ } else
+ target_elt = sort_entries[target]->elt;
+
+ elt = sort_entries[dup]->elt;
+
+ for (i = 0; i < elt->map->n_fields; i++)
+ atomic64_add(atomic64_read(&elt->fields[i].sum),
+ &target_elt->fields[i].sum);
+
+ sort_entries[dup]->dup = true;
+
+ return 0;
+}
+
+static int merge_dups(struct tracing_map_sort_entry **sort_entries,
+ int n_entries, unsigned int key_size)
+{
+ unsigned int dups = 0, total_dups = 0;
+ int err, i, j;
+ void *key;
+
+ if (n_entries < 2)
+ return total_dups;
+
+ sort(sort_entries, n_entries, sizeof(struct tracing_map_sort_entry *),
+ (int (*)(const void *, const void *))cmp_entries_dup, NULL);
+
+ key = sort_entries[0]->key;
+ for (i = 1; i < n_entries; i++) {
+ if (!memcmp(sort_entries[i]->key, key, key_size)) {
+ dups++; total_dups++;
+ err = merge_dup(sort_entries, i - dups, i);
+ if (err)
+ return err;
+ continue;
+ }
+ key = sort_entries[i]->key;
+ dups = 0;
+ }
+
+ if (!total_dups)
+ return total_dups;
+
+ for (i = 0, j = 0; i < n_entries; i++) {
+ if (!sort_entries[i]->dup) {
+ sort_entries[j] = sort_entries[i];
+ if (j++ != i)
+ sort_entries[i] = NULL;
+ } else {
+ destroy_sort_entry(sort_entries[i]);
+ sort_entries[i] = NULL;
+ }
+ }
+
+ return total_dups;
+}
+
+static bool is_key(struct tracing_map *map, unsigned int field_idx)
+{
+ unsigned int i;
+
+ for (i = 0; i < map->n_keys; i++)
+ if (map->key_idx[i] == field_idx)
+ return true;
+ return false;
+}
+
+static void sort_secondary(struct tracing_map *map,
+ const struct tracing_map_sort_entry **entries,
+ unsigned int n_entries,
+ struct tracing_map_sort_key *primary_key,
+ struct tracing_map_sort_key *secondary_key)
+{
+ int (*primary_fn)(const struct tracing_map_sort_entry **,
+ const struct tracing_map_sort_entry **);
+ int (*secondary_fn)(const struct tracing_map_sort_entry **,
+ const struct tracing_map_sort_entry **);
+ unsigned i, start = 0, n_sub = 1;
+
+ if (is_key(map, primary_key->field_idx))
+ primary_fn = cmp_entries_key;
+ else
+ primary_fn = cmp_entries_sum;
+
+ if (is_key(map, secondary_key->field_idx))
+ secondary_fn = cmp_entries_key;
+ else
+ secondary_fn = cmp_entries_sum;
+
+ for (i = 0; i < n_entries - 1; i++) {
+ const struct tracing_map_sort_entry **a = &entries[i];
+ const struct tracing_map_sort_entry **b = &entries[i + 1];
+
+ if (primary_fn(a, b) == 0) {
+ n_sub++;
+ if (i < n_entries - 2)
+ continue;
+ }
+
+ if (n_sub < 2) {
+ start = i + 1;
+ n_sub = 1;
+ continue;
+ }
+
+ set_sort_key(map, secondary_key);
+ sort(&entries[start], n_sub,
+ sizeof(struct tracing_map_sort_entry *),
+ (int (*)(const void *, const void *))secondary_fn, NULL);
+ set_sort_key(map, primary_key);
+
+ start = i + 1;
+ n_sub = 1;
+ }
+}
+
+/**
+ * tracing_map_sort_entries - Sort the current set of tracing_map_elts in a map
+ * @map: The tracing_map
+ * @sort_key: The sort key to use for sorting
+ * @sort_entries: outval: pointer to allocated and sorted array of entries
+ *
+ * tracing_map_sort_entries() sorts the current set of entries in the
+ * map and returns the list of tracing_map_sort_entries containing
+ * them to the client in the sort_entries param. The client can
+ * access the struct tracing_map_elt element of interest directly as
+ * the 'elt' field of a returned struct tracing_map_sort_entry object.
+ *
+ * The sort_key has only two fields: idx and descending. 'idx' refers
+ * to the index of the field added via tracing_map_add_sum_field() or
+ * tracing_map_add_key_field() when the tracing_map was initialized.
+ * 'descending' is a flag that if set reverses the sort order, which
+ * by default is ascending.
+ *
+ * The client should not hold on to the returned array but should use
+ * it and call tracing_map_destroy_sort_entries() when done.
+ *
+ * Return: the number of sort_entries in the struct tracing_map_sort_entry
+ * array, negative on error
+ */
+int tracing_map_sort_entries(struct tracing_map *map,
+ struct tracing_map_sort_key *sort_keys,
+ unsigned int n_sort_keys,
+ struct tracing_map_sort_entry ***sort_entries)
+{
+ int (*cmp_entries_fn)(const struct tracing_map_sort_entry **,
+ const struct tracing_map_sort_entry **);
+ struct tracing_map_sort_entry *sort_entry, **entries;
+ int i, n_entries, ret;
+
+ entries = vmalloc(map->max_elts * sizeof(sort_entry));
+ if (!entries)
+ return -ENOMEM;
+
+ for (i = 0, n_entries = 0; i < map->map_size; i++) {
+ struct tracing_map_entry *entry;
+
+ entry = TRACING_MAP_ENTRY(map->map, i);
+
+ if (!entry->key || !entry->val)
+ continue;
+
+ entries[n_entries] = create_sort_entry(entry->val->key,
+ entry->val);
+ if (!entries[n_entries++]) {
+ ret = -ENOMEM;
+ goto free;
+ }
+ }
+
+ if (n_entries == 0) {
+ ret = 0;
+ goto free;
+ }
+
+ if (n_entries == 1) {
+ *sort_entries = entries;
+ return 1;
+ }
+
+ ret = merge_dups(entries, n_entries, map->key_size);
+ if (ret < 0)
+ goto free;
+ n_entries -= ret;
+
+ if (is_key(map, sort_keys[0].field_idx))
+ cmp_entries_fn = cmp_entries_key;
+ else
+ cmp_entries_fn = cmp_entries_sum;
+
+ set_sort_key(map, &sort_keys[0]);
+
+ sort(entries, n_entries, sizeof(struct tracing_map_sort_entry *),
+ (int (*)(const void *, const void *))cmp_entries_fn, NULL);
+
+ if (n_sort_keys > 1)
+ sort_secondary(map,
+ (const struct tracing_map_sort_entry **)entries,
+ n_entries,
+ &sort_keys[0],
+ &sort_keys[1]);
+
+ *sort_entries = entries;
+
+ return n_entries;
+ free:
+ tracing_map_destroy_sort_entries(entries, n_entries);
+
+ return ret;
+}
diff --git a/kernel/trace/tracing_map.h b/kernel/trace/tracing_map.h
new file mode 100644
index 0000000..618838f
--- /dev/null
+++ b/kernel/trace/tracing_map.h
@@ -0,0 +1,283 @@
+#ifndef __TRACING_MAP_H
+#define __TRACING_MAP_H
+
+#define TRACING_MAP_BITS_DEFAULT 11
+#define TRACING_MAP_BITS_MAX 17
+#define TRACING_MAP_BITS_MIN 7
+
+#define TRACING_MAP_KEYS_MAX 2
+#define TRACING_MAP_VALS_MAX 3
+#define TRACING_MAP_FIELDS_MAX (TRACING_MAP_KEYS_MAX + \
+ TRACING_MAP_VALS_MAX)
+#define TRACING_MAP_SORT_KEYS_MAX 2
+
+typedef int (*tracing_map_cmp_fn_t) (void *val_a, void *val_b);
+
+/*
+ * This is an overview of the tracing_map data structures and how they
+ * relate to the tracing_map API. The details of the algorithms
+ * aren't discussed here - this is just a general overview of the data
+ * structures and how they interact with the API.
+ *
+ * The central data structure of the tracing_map is an initially
+ * zeroed array of struct tracing_map_entry (stored in the map field
+ * of struct tracing_map). tracing_map_entry is a very simple data
+ * structure containing only two fields: a 32-bit unsigned 'key'
+ * variable and a pointer named 'val'. This array of struct
+ * tracing_map_entry is essentially a hash table which will be
+ * modified by a single function, tracing_map_insert(), but which can
+ * be traversed and read by a user at any time (though the user does
+ * this indirectly via an array of tracing_map_sort_entry - see the
+ * explanation of that data structure in the discussion of the
+ * sorting-related data structures below).
+ *
+ * The central function of the tracing_map API is
+ * tracing_map_insert(). tracing_map_insert() hashes the
+ * arbitrarily-sized key passed into it into a 32-bit unsigned key.
+ * It then uses this key, truncated to the array size, as an index
+ * into the array of tracing_map_entries. If the value of the 'key'
+ * field of the tracing_map_entry found at that location is 0, then
+ * that entry is considered to be free and can be claimed, by
+ * replacing the 0 in the 'key' field of the tracing_map_entry with
+ * the new 32-bit hashed key. Once claimed, that tracing_map_entry's
+ * 'val' field is then used to store a unique element which will be
+ * forever associated with that 32-bit hashed key in the
+ * tracing_map_entry.
+ *
+ * That unique element now in the tracing_map_entry's 'val' field is
+ * an instance of tracing_map_elt, where 'elt' in the latter part of
+ * that variable name is short for 'element'. The purpose of a
+ * tracing_map_elt is to hold values specific to the particular
+ * 32-bit hashed key it's assocated with. Things such as the unique
+ * set of aggregated sums associated with the 32-bit hashed key, along
+ * with a copy of the full key associated with the entry, and which
+ * was used to produce the 32-bit hashed key.
+ *
+ * When tracing_map_create() is called to create the tracing map, the
+ * user specifies (indirectly via the map_bits param, the details are
+ * unimportant for this discussion) the maximum number of elements
+ * that the map can hold (stored in the max_elts field of struct
+ * tracing_map). This is the maximum possible number of
+ * tracing_map_entries in the tracing_map_entry array which can be
+ * 'claimed' as described in the above discussion, and therefore is
+ * also the maximum number of tracing_map_elts that can be associated
+ * with the tracing_map_entry array in the tracing_map. Because of
+ * the way the insertion algorithm works, the size of the allocated
+ * tracing_map_entry array is always twice the maximum number of
+ * elements (2 * max_elts). This value is stored in the map_size
+ * field of struct tracing_map.
+ *
+ * Because tracing_map_insert() needs to work from any context,
+ * including from within the memory allocation functions themselves,
+ * both the tracing_map_entry array and a pool of max_elts
+ * tracing_map_elts are pre-allocated before any call is made to
+ * tracing_map_insert().
+ *
+ * The tracing_map_entry array is allocated as a single block by
+ * tracing_map_create().
+ *
+ * Because the tracing_map_elts are much larger objects and can't
+ * generally be allocated together as a single large array without
+ * failure, they're allocated individually, by tracing_map_init().
+ *
+ * The pool of tracing_map_elts are allocated by tracing_map_init()
+ * rather than by tracing_map_create() because at the time
+ * tracing_map_create() is called, there isn't enough information to
+ * create the tracing_map_elts. Specifically,the user first needs to
+ * tell the tracing_map implementation how many fields the
+ * tracing_map_elts contain, and which types of fields they are (key
+ * or sum). The user does this via the tracing_map_add_sum_field()
+ * and tracing_map_add_key_field() functions, following which the user
+ * calls tracing_map_init() to finish up the tracing map setup. The
+ * array holding the pointers which make up the pre-allocated pool of
+ * tracing_map_elts is allocated as a single block and is stored in
+ * the elts field of struct tracing_map.
+ *
+ * There is also a set of structures used for sorting that might
+ * benefit from some minimal explanation.
+ *
+ * struct tracing_map_sort_key is used to drive the sort at any given
+ * time. By 'any given time' we mean that a different
+ * tracing_map_sort_key will be used at different times depending on
+ * whether the sort currently being performed is a primary or a
+ * secondary sort.
+ *
+ * The sort key is very simple, consisting of the field index of the
+ * tracing_map_elt field to sort on (which the user saved when adding
+ * the field), and whether the sort should be done in an ascending or
+ * descending order.
+ *
+ * For the convenience of the sorting code, a tracing_map_sort_entry
+ * is created for each tracing_map_elt, again individually allocated
+ * to avoid failures that might be expected if allocated as a single
+ * large array of struct tracing_map_sort_entry.
+ * tracing_map_sort_entry instances are the objects expected by the
+ * various internal sorting functions, and are also what the user
+ * ultimately receives after calling tracing_map_sort_entries().
+ * Because it doesn't make sense for users to access an unordered and
+ * sparsely populated tracing_map directly, the
+ * tracing_map_sort_entries() function is provided so that users can
+ * retrieve a sorted list of all existing elements. In addition to
+ * the associated tracing_map_elt 'elt' field contained within the
+ * tracing_map_sort_entry, which is the object of interest to the
+ * user, tracing_map_sort_entry objects contain a number of additional
+ * fields which are used for caching and internal purposes and can
+ * safely be ignored.
+*/
+
+struct tracing_map_field {
+ tracing_map_cmp_fn_t cmp_fn;
+ union {
+ atomic64_t sum;
+ unsigned int offset;
+ };
+};
+
+struct tracing_map_elt {
+ struct tracing_map *map;
+ struct tracing_map_field *fields;
+ void *key;
+ void *private_data;
+};
+
+struct tracing_map_entry {
+ u32 key;
+ struct tracing_map_elt *val;
+};
+
+struct tracing_map_sort_key {
+ unsigned int field_idx;
+ bool descending;
+};
+
+struct tracing_map_sort_entry {
+ void *key;
+ struct tracing_map_elt *elt;
+ bool elt_copied;
+ bool dup;
+};
+
+struct tracing_map_array {
+ unsigned int entries_per_page;
+ unsigned int entry_size_shift;
+ unsigned int entry_shift;
+ unsigned int entry_mask;
+ unsigned int n_pages;
+ void **pages;
+};
+
+#define TRACING_MAP_ARRAY_ELT(array, idx) \
+ (array->pages[idx >> array->entry_shift] + \
+ ((idx & array->entry_mask) << array->entry_size_shift))
+
+#define TRACING_MAP_ENTRY(array, idx) \
+ ((struct tracing_map_entry *)TRACING_MAP_ARRAY_ELT(array, idx))
+
+#define TRACING_MAP_ELT(array, idx) \
+ ((struct tracing_map_elt **)TRACING_MAP_ARRAY_ELT(array, idx))
+
+struct tracing_map {
+ unsigned int key_size;
+ unsigned int map_bits;
+ unsigned int map_size;
+ unsigned int max_elts;
+ atomic_t next_elt;
+ struct tracing_map_array *elts;
+ struct tracing_map_array *map;
+ const struct tracing_map_ops *ops;
+ void *private_data;
+ struct tracing_map_field fields[TRACING_MAP_FIELDS_MAX];
+ unsigned int n_fields;
+ int key_idx[TRACING_MAP_KEYS_MAX];
+ unsigned int n_keys;
+ struct tracing_map_sort_key sort_key;
+ atomic64_t hits;
+ atomic64_t drops;
+};
+
+/**
+ * struct tracing_map_ops - callbacks for tracing_map
+ *
+ * The methods in this structure define callback functions for various
+ * operations on a tracing_map or objects related to a tracing_map.
+ *
+ * For a detailed description of tracing_map_elt objects please see
+ * the overview of tracing_map data structures at the beginning of
+ * this file.
+ *
+ * All the methods below are optional.
+ *
+ * @elt_alloc: When a tracing_map_elt is allocated, this function, if
+ * defined, will be called and gives clients the opportunity to
+ * allocate additional data and attach it to the element
+ * (tracing_map_elt->private_data is meant for that purpose).
+ * Element allocation occurs before tracing begins, when the
+ * tracing_map_init() call is made by client code.
+ *
+ * @elt_copy: At certain points in the lifetime of an element, it may
+ * need to be copied. The copy should include a copy of the
+ * client-allocated data, which can be copied into the 'to'
+ * element from the 'from' element.
+ *
+ * @elt_free: When a tracing_map_elt is freed, this function is called
+ * and allows client-allocated per-element data to be freed.
+ *
+ * @elt_clear: This callback allows per-element client-defined data to
+ * be cleared, if applicable.
+ *
+ * @elt_init: This callback allows per-element client-defined data to
+ * be initialized when used i.e. when the element is actually
+ * claimed by tracing_map_insert() in the context of the map
+ * insertion.
+ */
+struct tracing_map_ops {
+ int (*elt_alloc)(struct tracing_map_elt *elt);
+ void (*elt_copy)(struct tracing_map_elt *to,
+ struct tracing_map_elt *from);
+ void (*elt_free)(struct tracing_map_elt *elt);
+ void (*elt_clear)(struct tracing_map_elt *elt);
+ void (*elt_init)(struct tracing_map_elt *elt);
+};
+
+extern struct tracing_map *
+tracing_map_create(unsigned int map_bits,
+ unsigned int key_size,
+ const struct tracing_map_ops *ops,
+ void *private_data);
+extern int tracing_map_init(struct tracing_map *map);
+
+extern int tracing_map_add_sum_field(struct tracing_map *map);
+extern int tracing_map_add_key_field(struct tracing_map *map,
+ unsigned int offset,
+ tracing_map_cmp_fn_t cmp_fn);
+
+extern void tracing_map_destroy(struct tracing_map *map);
+extern void tracing_map_clear(struct tracing_map *map);
+
+extern struct tracing_map_elt *
+tracing_map_insert(struct tracing_map *map, void *key);
+extern struct tracing_map_elt *
+tracing_map_lookup(struct tracing_map *map, void *key);
+
+extern tracing_map_cmp_fn_t tracing_map_cmp_num(int field_size,
+ int field_is_signed);
+extern int tracing_map_cmp_string(void *val_a, void *val_b);
+extern int tracing_map_cmp_none(void *val_a, void *val_b);
+
+extern void tracing_map_update_sum(struct tracing_map_elt *elt,
+ unsigned int i, u64 n);
+extern u64 tracing_map_read_sum(struct tracing_map_elt *elt, unsigned int i);
+extern void tracing_map_set_field_descr(struct tracing_map *map,
+ unsigned int i,
+ unsigned int key_offset,
+ tracing_map_cmp_fn_t cmp_fn);
+extern int
+tracing_map_sort_entries(struct tracing_map *map,
+ struct tracing_map_sort_key *sort_keys,
+ unsigned int n_sort_keys,
+ struct tracing_map_sort_entry ***sort_entries);
+
+extern void
+tracing_map_destroy_sort_entries(struct tracing_map_sort_entry **entries,
+ unsigned int n_entries);
+#endif /* __TRACING_MAP_H */
diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 5f5068e..e1c0e99 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -433,11 +433,18 @@
return ((struct work_struct *) addr)->func;
}
+static bool work_is_static_object(void *addr)
+{
+ struct work_struct *work = addr;
+
+ return test_bit(WORK_STRUCT_STATIC_BIT, work_data_bits(work));
+}
+
/*
* fixup_init is called when:
* - an active object is initialized
*/
-static int work_fixup_init(void *addr, enum debug_obj_state state)
+static bool work_fixup_init(void *addr, enum debug_obj_state state)
{
struct work_struct *work = addr;
@@ -445,42 +452,9 @@
case ODEBUG_STATE_ACTIVE:
cancel_work_sync(work);
debug_object_init(work, &work_debug_descr);
- return 1;
+ return true;
default:
- return 0;
- }
-}
-
-/*
- * fixup_activate is called when:
- * - an active object is activated
- * - an unknown object is activated (might be a statically initialized object)
- */
-static int work_fixup_activate(void *addr, enum debug_obj_state state)
-{
- struct work_struct *work = addr;
-
- switch (state) {
-
- case ODEBUG_STATE_NOTAVAILABLE:
- /*
- * This is not really a fixup. The work struct was
- * statically initialized. We just make sure that it
- * is tracked in the object tracker.
- */
- if (test_bit(WORK_STRUCT_STATIC_BIT, work_data_bits(work))) {
- debug_object_init(work, &work_debug_descr);
- debug_object_activate(work, &work_debug_descr);
- return 0;
- }
- WARN_ON_ONCE(1);
- return 0;
-
- case ODEBUG_STATE_ACTIVE:
- WARN_ON(1);
-
- default:
- return 0;
+ return false;
}
}
@@ -488,7 +462,7 @@
* fixup_free is called when:
* - an active object is freed
*/
-static int work_fixup_free(void *addr, enum debug_obj_state state)
+static bool work_fixup_free(void *addr, enum debug_obj_state state)
{
struct work_struct *work = addr;
@@ -496,17 +470,17 @@
case ODEBUG_STATE_ACTIVE:
cancel_work_sync(work);
debug_object_free(work, &work_debug_descr);
- return 1;
+ return true;
default:
- return 0;
+ return false;
}
}
static struct debug_obj_descr work_debug_descr = {
.name = "work_struct",
.debug_hint = work_debug_hint,
+ .is_static_object = work_is_static_object,
.fixup_init = work_fixup_init,
- .fixup_activate = work_fixup_activate,
.fixup_free = work_fixup_free,
};
diff --git a/lib/Kconfig b/lib/Kconfig
index 3cca122..d79909d 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -362,6 +362,9 @@
for more information.
+config RADIX_TREE_MULTIORDER
+ bool
+
config ASSOCIATIVE_ARRAY
bool
help
@@ -523,6 +526,13 @@
a scatterlist. This should be selected by a driver or an API which
whishes to split a scatterlist amongst multiple DMA channels.
+config SG_POOL
+ def_bool n
+ help
+ Provides a helper to allocate chained scatterlists. This should be
+ selected by a driver or an API which whishes to allocate chained
+ scatterlist.
+
#
# sg chaining option
#
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index f4b797a..e707ab3 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -257,6 +257,7 @@
config DEBUG_FS
bool "Debug Filesystem"
+ select SRCU
help
debugfs is a virtual file system that kernel developers use to put
debugging files into. Enable this option to be able to read and
diff --git a/lib/Kconfig.kgdb b/lib/Kconfig.kgdb
index c635a10..533f912 100644
--- a/lib/Kconfig.kgdb
+++ b/lib/Kconfig.kgdb
@@ -22,7 +22,7 @@
tristate "KGDB: use kgdb over the serial console"
select CONSOLE_POLL
select MAGIC_SYSRQ
- depends on TTY
+ depends on TTY && HW_CONSOLE
default y
help
Share a serial console with kgdb. Sysrq-g must be used
diff --git a/lib/Makefile b/lib/Makefile
index a65e9a8..42b6918 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -25,7 +25,7 @@
sha1.o md5.o irq_regs.o argv_split.o \
flex_proportions.o ratelimit.o show_mem.o \
is_single_threaded.o plist.o decompress.o kobject_uevent.o \
- earlycpio.o seq_buf.o nmi_backtrace.o
+ earlycpio.o seq_buf.o nmi_backtrace.o nodemask.o
obj-$(CONFIG_ARCH_HAS_DEBUG_STRICT_USER_COPY_CHECKS) += usercopy.o
lib-$(CONFIG_MMU) += ioremap.o
@@ -178,6 +178,7 @@
obj-$(CONFIG_GENERIC_NET_UTILS) += net_utils.o
obj-$(CONFIG_SG_SPLIT) += sg_split.o
+obj-$(CONFIG_SG_POOL) += sg_pool.o
obj-$(CONFIG_STMP_DEVICE) += stmp_device.o
obj-$(CONFIG_IRQ_POLL) += irq_poll.o
diff --git a/lib/debugobjects.c b/lib/debugobjects.c
index 519b5a1..a8e1260 100644
--- a/lib/debugobjects.c
+++ b/lib/debugobjects.c
@@ -269,16 +269,15 @@
* Try to repair the damage, so we have a better chance to get useful
* debug output.
*/
-static int
-debug_object_fixup(int (*fixup)(void *addr, enum debug_obj_state state),
+static bool
+debug_object_fixup(bool (*fixup)(void *addr, enum debug_obj_state state),
void * addr, enum debug_obj_state state)
{
- int fixed = 0;
-
- if (fixup)
- fixed = fixup(addr, state);
- debug_objects_fixups += fixed;
- return fixed;
+ if (fixup && fixup(addr, state)) {
+ debug_objects_fixups++;
+ return true;
+ }
+ return false;
}
static void debug_object_is_on_stack(void *addr, int onstack)
@@ -416,7 +415,7 @@
state = obj->state;
raw_spin_unlock_irqrestore(&db->lock, flags);
ret = debug_object_fixup(descr->fixup_activate, addr, state);
- return ret ? -EINVAL : 0;
+ return ret ? 0 : -EINVAL;
case ODEBUG_STATE_DESTROYED:
debug_print_object(obj, "activate");
@@ -432,14 +431,21 @@
raw_spin_unlock_irqrestore(&db->lock, flags);
/*
- * This happens when a static object is activated. We
- * let the type specific code decide whether this is
- * true or not.
+ * We are here when a static object is activated. We
+ * let the type specific code confirm whether this is
+ * true or not. if true, we just make sure that the
+ * static object is tracked in the object tracker. If
+ * not, this must be a bug, so we try to fix it up.
*/
- if (debug_object_fixup(descr->fixup_activate, addr,
- ODEBUG_STATE_NOTAVAILABLE)) {
+ if (descr->is_static_object && descr->is_static_object(addr)) {
+ /* track this static object */
+ debug_object_init(addr, descr);
+ debug_object_activate(addr, descr);
+ } else {
debug_print_object(&o, "activate");
- return -EINVAL;
+ ret = debug_object_fixup(descr->fixup_activate, addr,
+ ODEBUG_STATE_NOTAVAILABLE);
+ return ret ? 0 : -EINVAL;
}
return 0;
}
@@ -603,12 +609,18 @@
raw_spin_unlock_irqrestore(&db->lock, flags);
/*
- * Maybe the object is static. Let the type specific
- * code decide what to do.
+ * Maybe the object is static, and we let the type specific
+ * code confirm. Track this static object if true, else invoke
+ * fixup.
*/
- if (debug_object_fixup(descr->fixup_assert_init, addr,
- ODEBUG_STATE_NOTAVAILABLE))
+ if (descr->is_static_object && descr->is_static_object(addr)) {
+ /* Track this static object */
+ debug_object_init(addr, descr);
+ } else {
debug_print_object(&o, "assert_init");
+ debug_object_fixup(descr->fixup_assert_init, addr,
+ ODEBUG_STATE_NOTAVAILABLE);
+ }
return;
}
@@ -793,11 +805,18 @@
static __initdata struct debug_obj_descr descr_type_test;
+static bool __init is_static_object(void *addr)
+{
+ struct self_test *obj = addr;
+
+ return obj->static_init;
+}
+
/*
* fixup_init is called when:
* - an active object is initialized
*/
-static int __init fixup_init(void *addr, enum debug_obj_state state)
+static bool __init fixup_init(void *addr, enum debug_obj_state state)
{
struct self_test *obj = addr;
@@ -805,37 +824,31 @@
case ODEBUG_STATE_ACTIVE:
debug_object_deactivate(obj, &descr_type_test);
debug_object_init(obj, &descr_type_test);
- return 1;
+ return true;
default:
- return 0;
+ return false;
}
}
/*
* fixup_activate is called when:
* - an active object is activated
- * - an unknown object is activated (might be a statically initialized object)
+ * - an unknown non-static object is activated
*/
-static int __init fixup_activate(void *addr, enum debug_obj_state state)
+static bool __init fixup_activate(void *addr, enum debug_obj_state state)
{
struct self_test *obj = addr;
switch (state) {
case ODEBUG_STATE_NOTAVAILABLE:
- if (obj->static_init == 1) {
- debug_object_init(obj, &descr_type_test);
- debug_object_activate(obj, &descr_type_test);
- return 0;
- }
- return 1;
-
+ return true;
case ODEBUG_STATE_ACTIVE:
debug_object_deactivate(obj, &descr_type_test);
debug_object_activate(obj, &descr_type_test);
- return 1;
+ return true;
default:
- return 0;
+ return false;
}
}
@@ -843,7 +856,7 @@
* fixup_destroy is called when:
* - an active object is destroyed
*/
-static int __init fixup_destroy(void *addr, enum debug_obj_state state)
+static bool __init fixup_destroy(void *addr, enum debug_obj_state state)
{
struct self_test *obj = addr;
@@ -851,9 +864,9 @@
case ODEBUG_STATE_ACTIVE:
debug_object_deactivate(obj, &descr_type_test);
debug_object_destroy(obj, &descr_type_test);
- return 1;
+ return true;
default:
- return 0;
+ return false;
}
}
@@ -861,7 +874,7 @@
* fixup_free is called when:
* - an active object is freed
*/
-static int __init fixup_free(void *addr, enum debug_obj_state state)
+static bool __init fixup_free(void *addr, enum debug_obj_state state)
{
struct self_test *obj = addr;
@@ -869,9 +882,9 @@
case ODEBUG_STATE_ACTIVE:
debug_object_deactivate(obj, &descr_type_test);
debug_object_free(obj, &descr_type_test);
- return 1;
+ return true;
default:
- return 0;
+ return false;
}
}
@@ -917,6 +930,7 @@
static __initdata struct debug_obj_descr descr_type_test = {
.name = "selftest",
+ .is_static_object = is_static_object,
.fixup_init = fixup_init,
.fixup_activate = fixup_activate,
.fixup_destroy = fixup_destroy,
diff --git a/lib/gcd.c b/lib/gcd.c
index 3657f12..135ee64 100644
--- a/lib/gcd.c
+++ b/lib/gcd.c
@@ -2,20 +2,77 @@
#include <linux/gcd.h>
#include <linux/export.h>
-/* Greatest common divisor */
+/*
+ * This implements the binary GCD algorithm. (Often attributed to Stein,
+ * but as Knuth has noted, appears in a first-century Chinese math text.)
+ *
+ * This is faster than the division-based algorithm even on x86, which
+ * has decent hardware division.
+ */
+
+#if !defined(CONFIG_CPU_NO_EFFICIENT_FFS) && !defined(CPU_NO_EFFICIENT_FFS)
+
+/* If __ffs is available, the even/odd algorithm benchmarks slower. */
unsigned long gcd(unsigned long a, unsigned long b)
{
- unsigned long r;
+ unsigned long r = a | b;
- if (a < b)
- swap(a, b);
+ if (!a || !b)
+ return r;
- if (!b)
- return a;
- while ((r = a % b) != 0) {
- a = b;
- b = r;
+ b >>= __ffs(b);
+ if (b == 1)
+ return r & -r;
+
+ for (;;) {
+ a >>= __ffs(a);
+ if (a == 1)
+ return r & -r;
+ if (a == b)
+ return a << __ffs(r);
+
+ if (a < b)
+ swap(a, b);
+ a -= b;
}
- return b;
}
+
+#else
+
+/* If normalization is done by loops, the even/odd algorithm is a win. */
+unsigned long gcd(unsigned long a, unsigned long b)
+{
+ unsigned long r = a | b;
+
+ if (!a || !b)
+ return r;
+
+ /* Isolate lsbit of r */
+ r &= -r;
+
+ while (!(b & r))
+ b >>= 1;
+ if (b == r)
+ return r;
+
+ for (;;) {
+ while (!(a & r))
+ a >>= 1;
+ if (a == r)
+ return r;
+ if (a == b)
+ return a;
+
+ if (a < b)
+ swap(a, b);
+ a -= b;
+ a >>= 1;
+ if (a & r)
+ a += b;
+ a >>= 1;
+ }
+}
+
+#endif
+
EXPORT_SYMBOL_GPL(gcd);
diff --git a/lib/nmi_backtrace.c b/lib/nmi_backtrace.c
index 6019c53..26caf51 100644
--- a/lib/nmi_backtrace.c
+++ b/lib/nmi_backtrace.c
@@ -16,33 +16,14 @@
#include <linux/delay.h>
#include <linux/kprobes.h>
#include <linux/nmi.h>
-#include <linux/seq_buf.h>
#ifdef arch_trigger_all_cpu_backtrace
/* For reliability, we're prepared to waste bits here. */
static DECLARE_BITMAP(backtrace_mask, NR_CPUS) __read_mostly;
-static cpumask_t printtrace_mask;
-
-#define NMI_BUF_SIZE 4096
-
-struct nmi_seq_buf {
- unsigned char buffer[NMI_BUF_SIZE];
- struct seq_buf seq;
-};
-
-/* Safe printing in NMI context */
-static DEFINE_PER_CPU(struct nmi_seq_buf, nmi_print_seq);
/* "in progress" flag of arch_trigger_all_cpu_backtrace */
static unsigned long backtrace_flag;
-static void print_seq_line(struct nmi_seq_buf *s, int start, int end)
-{
- const char *buf = s->buffer + start;
-
- printk("%.*s", (end - start) + 1, buf);
-}
-
/*
* When raise() is called it will be is passed a pointer to the
* backtrace_mask. Architectures that call nmi_cpu_backtrace()
@@ -52,8 +33,7 @@
void nmi_trigger_all_cpu_backtrace(bool include_self,
void (*raise)(cpumask_t *mask))
{
- struct nmi_seq_buf *s;
- int i, cpu, this_cpu = get_cpu();
+ int i, this_cpu = get_cpu();
if (test_and_set_bit(0, &backtrace_flag)) {
/*
@@ -68,17 +48,6 @@
if (!include_self)
cpumask_clear_cpu(this_cpu, to_cpumask(backtrace_mask));
- cpumask_copy(&printtrace_mask, to_cpumask(backtrace_mask));
-
- /*
- * Set up per_cpu seq_buf buffers that the NMIs running on the other
- * CPUs will write to.
- */
- for_each_cpu(cpu, to_cpumask(backtrace_mask)) {
- s = &per_cpu(nmi_print_seq, cpu);
- seq_buf_init(&s->seq, s->buffer, NMI_BUF_SIZE);
- }
-
if (!cpumask_empty(to_cpumask(backtrace_mask))) {
pr_info("Sending NMI to %s CPUs:\n",
(include_self ? "all" : "other"));
@@ -94,73 +63,25 @@
}
/*
- * Now that all the NMIs have triggered, we can dump out their
- * back traces safely to the console.
+ * Force flush any remote buffers that might be stuck in IRQ context
+ * and therefore could not run their irq_work.
*/
- for_each_cpu(cpu, &printtrace_mask) {
- int len, last_i = 0;
+ printk_nmi_flush();
- s = &per_cpu(nmi_print_seq, cpu);
- len = seq_buf_used(&s->seq);
- if (!len)
- continue;
-
- /* Print line by line. */
- for (i = 0; i < len; i++) {
- if (s->buffer[i] == '\n') {
- print_seq_line(s, last_i, i);
- last_i = i + 1;
- }
- }
- /* Check if there was a partial line. */
- if (last_i < len) {
- print_seq_line(s, last_i, len - 1);
- pr_cont("\n");
- }
- }
-
- clear_bit(0, &backtrace_flag);
- smp_mb__after_atomic();
+ clear_bit_unlock(0, &backtrace_flag);
put_cpu();
}
-/*
- * It is not safe to call printk() directly from NMI handlers.
- * It may be fine if the NMI detected a lock up and we have no choice
- * but to do so, but doing a NMI on all other CPUs to get a back trace
- * can be done with a sysrq-l. We don't want that to lock up, which
- * can happen if the NMI interrupts a printk in progress.
- *
- * Instead, we redirect the vprintk() to this nmi_vprintk() that writes
- * the content into a per cpu seq_buf buffer. Then when the NMIs are
- * all done, we can safely dump the contents of the seq_buf to a printk()
- * from a non NMI context.
- */
-static int nmi_vprintk(const char *fmt, va_list args)
-{
- struct nmi_seq_buf *s = this_cpu_ptr(&nmi_print_seq);
- unsigned int len = seq_buf_used(&s->seq);
-
- seq_buf_vprintf(&s->seq, fmt, args);
- return seq_buf_used(&s->seq) - len;
-}
-
bool nmi_cpu_backtrace(struct pt_regs *regs)
{
int cpu = smp_processor_id();
if (cpumask_test_cpu(cpu, to_cpumask(backtrace_mask))) {
- printk_func_t printk_func_save = this_cpu_read(printk_func);
-
- /* Replace printk to write into the NMI seq */
- this_cpu_write(printk_func, nmi_vprintk);
pr_warn("NMI backtrace for cpu %d\n", cpu);
if (regs)
show_regs(regs);
else
dump_stack();
- this_cpu_write(printk_func, printk_func_save);
-
cpumask_clear_cpu(cpu, to_cpumask(backtrace_mask));
return true;
}
diff --git a/lib/nodemask.c b/lib/nodemask.c
new file mode 100644
index 0000000..e42a5bf4
--- /dev/null
+++ b/lib/nodemask.c
@@ -0,0 +1,30 @@
+#include <linux/nodemask.h>
+#include <linux/module.h>
+#include <linux/random.h>
+
+int __next_node_in(int node, const nodemask_t *srcp)
+{
+ int ret = __next_node(node, srcp);
+
+ if (ret == MAX_NUMNODES)
+ ret = __first_node(srcp);
+ return ret;
+}
+EXPORT_SYMBOL(__next_node_in);
+
+#ifdef CONFIG_NUMA
+/*
+ * Return the bit number of a random bit set in the nodemask.
+ * (returns NUMA_NO_NODE if nodemask is empty)
+ */
+int node_random(const nodemask_t *maskp)
+{
+ int w, bit = NUMA_NO_NODE;
+
+ w = nodes_weight(*maskp);
+ if (w)
+ bit = bitmap_ord_to_pos(maskp->bits,
+ get_random_int() % w, MAX_NUMNODES);
+ return bit;
+}
+#endif
diff --git a/lib/percpu_counter.c b/lib/percpu_counter.c
index f051d69..72d3611 100644
--- a/lib/percpu_counter.c
+++ b/lib/percpu_counter.c
@@ -19,7 +19,7 @@
static struct debug_obj_descr percpu_counter_debug_descr;
-static int percpu_counter_fixup_free(void *addr, enum debug_obj_state state)
+static bool percpu_counter_fixup_free(void *addr, enum debug_obj_state state)
{
struct percpu_counter *fbc = addr;
@@ -27,9 +27,9 @@
case ODEBUG_STATE_ACTIVE:
percpu_counter_destroy(fbc);
debug_object_free(fbc, &percpu_counter_debug_descr);
- return 1;
+ return true;
default:
- return 0;
+ return false;
}
}
diff --git a/lib/radix-tree.c b/lib/radix-tree.c
index 1624c41..8b7d845 100644
--- a/lib/radix-tree.c
+++ b/lib/radix-tree.c
@@ -4,6 +4,8 @@
* Copyright (C) 2005 SGI, Christoph Lameter
* Copyright (C) 2006 Nick Piggin
* Copyright (C) 2012 Konstantin Khlebnikov
+ * Copyright (C) 2016 Intel, Matthew Wilcox
+ * Copyright (C) 2016 Intel, Ross Zwisler
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License as
@@ -37,12 +39,6 @@
/*
- * The height_to_maxindex array needs to be one deeper than the maximum
- * path as height 0 holds only 1 entry.
- */
-static unsigned long height_to_maxindex[RADIX_TREE_MAX_PATH + 1] __read_mostly;
-
-/*
* Radix tree node cache.
*/
static struct kmem_cache *radix_tree_node_cachep;
@@ -64,20 +60,58 @@
* Per-cpu pool of preloaded nodes
*/
struct radix_tree_preload {
- int nr;
+ unsigned nr;
/* nodes->private_data points to next preallocated node */
struct radix_tree_node *nodes;
};
static DEFINE_PER_CPU(struct radix_tree_preload, radix_tree_preloads) = { 0, };
-static inline void *ptr_to_indirect(void *ptr)
+static inline void *node_to_entry(void *ptr)
{
- return (void *)((unsigned long)ptr | RADIX_TREE_INDIRECT_PTR);
+ return (void *)((unsigned long)ptr | RADIX_TREE_INTERNAL_NODE);
}
-static inline void *indirect_to_ptr(void *ptr)
+#define RADIX_TREE_RETRY node_to_entry(NULL)
+
+#ifdef CONFIG_RADIX_TREE_MULTIORDER
+/* Sibling slots point directly to another slot in the same node */
+static inline bool is_sibling_entry(struct radix_tree_node *parent, void *node)
{
- return (void *)((unsigned long)ptr & ~RADIX_TREE_INDIRECT_PTR);
+ void **ptr = node;
+ return (parent->slots <= ptr) &&
+ (ptr < parent->slots + RADIX_TREE_MAP_SIZE);
+}
+#else
+static inline bool is_sibling_entry(struct radix_tree_node *parent, void *node)
+{
+ return false;
+}
+#endif
+
+static inline unsigned long get_slot_offset(struct radix_tree_node *parent,
+ void **slot)
+{
+ return slot - parent->slots;
+}
+
+static unsigned int radix_tree_descend(struct radix_tree_node *parent,
+ struct radix_tree_node **nodep, unsigned long index)
+{
+ unsigned int offset = (index >> parent->shift) & RADIX_TREE_MAP_MASK;
+ void **entry = rcu_dereference_raw(parent->slots[offset]);
+
+#ifdef CONFIG_RADIX_TREE_MULTIORDER
+ if (radix_tree_is_internal_node(entry)) {
+ unsigned long siboff = get_slot_offset(parent, entry);
+ if (siboff < RADIX_TREE_MAP_SIZE) {
+ offset = siboff;
+ entry = rcu_dereference_raw(parent->slots[offset]);
+ }
+ }
+#endif
+
+ *nodep = (void *)entry;
+ return offset;
}
static inline gfp_t root_gfp_mask(struct radix_tree_root *root)
@@ -108,7 +142,7 @@
root->gfp_mask |= (__force gfp_t)(1 << (tag + __GFP_BITS_SHIFT));
}
-static inline void root_tag_clear(struct radix_tree_root *root, unsigned int tag)
+static inline void root_tag_clear(struct radix_tree_root *root, unsigned tag)
{
root->gfp_mask &= (__force gfp_t)~(1 << (tag + __GFP_BITS_SHIFT));
}
@@ -120,7 +154,12 @@
static inline int root_tag_get(struct radix_tree_root *root, unsigned int tag)
{
- return (__force unsigned)root->gfp_mask & (1 << (tag + __GFP_BITS_SHIFT));
+ return (__force int)root->gfp_mask & (1 << (tag + __GFP_BITS_SHIFT));
+}
+
+static inline unsigned root_tags_get(struct radix_tree_root *root)
+{
+ return (__force unsigned)root->gfp_mask >> __GFP_BITS_SHIFT;
}
/*
@@ -129,7 +168,7 @@
*/
static inline int any_tag_set(struct radix_tree_node *node, unsigned int tag)
{
- int idx;
+ unsigned idx;
for (idx = 0; idx < RADIX_TREE_TAG_LONGS; idx++) {
if (node->tags[tag][idx])
return 1;
@@ -173,38 +212,45 @@
return size;
}
-#if 0
-static void dump_node(void *slot, int height, int offset)
+#ifndef __KERNEL__
+static void dump_node(struct radix_tree_node *node, unsigned long index)
{
- struct radix_tree_node *node;
- int i;
+ unsigned long i;
- if (!slot)
- return;
+ pr_debug("radix node: %p offset %d tags %lx %lx %lx shift %d count %d parent %p\n",
+ node, node->offset,
+ node->tags[0][0], node->tags[1][0], node->tags[2][0],
+ node->shift, node->count, node->parent);
- if (height == 0) {
- pr_debug("radix entry %p offset %d\n", slot, offset);
- return;
+ for (i = 0; i < RADIX_TREE_MAP_SIZE; i++) {
+ unsigned long first = index | (i << node->shift);
+ unsigned long last = first | ((1UL << node->shift) - 1);
+ void *entry = node->slots[i];
+ if (!entry)
+ continue;
+ if (is_sibling_entry(node, entry)) {
+ pr_debug("radix sblng %p offset %ld val %p indices %ld-%ld\n",
+ entry, i,
+ *(void **)entry_to_node(entry),
+ first, last);
+ } else if (!radix_tree_is_internal_node(entry)) {
+ pr_debug("radix entry %p offset %ld indices %ld-%ld\n",
+ entry, i, first, last);
+ } else {
+ dump_node(entry_to_node(entry), first);
+ }
}
-
- node = indirect_to_ptr(slot);
- pr_debug("radix node: %p offset %d tags %lx %lx %lx path %x count %d parent %p\n",
- slot, offset, node->tags[0][0], node->tags[1][0],
- node->tags[2][0], node->path, node->count, node->parent);
-
- for (i = 0; i < RADIX_TREE_MAP_SIZE; i++)
- dump_node(node->slots[i], height - 1, i);
}
/* For debug */
static void radix_tree_dump(struct radix_tree_root *root)
{
- pr_debug("radix root: %p height %d rnode %p tags %x\n",
- root, root->height, root->rnode,
+ pr_debug("radix root: %p rnode %p tags %x\n",
+ root, root->rnode,
root->gfp_mask >> __GFP_BITS_SHIFT);
- if (!radix_tree_is_indirect_ptr(root->rnode))
+ if (!radix_tree_is_internal_node(root->rnode))
return;
- dump_node(root->rnode, root->height, 0);
+ dump_node(entry_to_node(root->rnode), 0);
}
#endif
@@ -219,9 +265,9 @@
gfp_t gfp_mask = root_gfp_mask(root);
/*
- * Preload code isn't irq safe and it doesn't make sence to use
- * preloading in the interrupt anyway as all the allocations have to
- * be atomic. So just do normal allocation when in interrupt.
+ * Preload code isn't irq safe and it doesn't make sense to use
+ * preloading during an interrupt anyway as all the allocations have
+ * to be atomic. So just do normal allocation when in interrupt.
*/
if (!gfpflags_allow_blocking(gfp_mask) && !in_interrupt()) {
struct radix_tree_preload *rtp;
@@ -257,7 +303,7 @@
ret = kmem_cache_alloc(radix_tree_node_cachep,
gfp_mask | __GFP_ACCOUNT);
out:
- BUG_ON(radix_tree_is_indirect_ptr(ret));
+ BUG_ON(radix_tree_is_internal_node(ret));
return ret;
}
@@ -357,38 +403,58 @@
EXPORT_SYMBOL(radix_tree_maybe_preload);
/*
- * Return the maximum key which can be store into a
- * radix tree with height HEIGHT.
+ * The maximum index which can be stored in a radix tree
*/
-static inline unsigned long radix_tree_maxindex(unsigned int height)
+static inline unsigned long shift_maxindex(unsigned int shift)
{
- return height_to_maxindex[height];
+ return (RADIX_TREE_MAP_SIZE << shift) - 1;
+}
+
+static inline unsigned long node_maxindex(struct radix_tree_node *node)
+{
+ return shift_maxindex(node->shift);
+}
+
+static unsigned radix_tree_load_root(struct radix_tree_root *root,
+ struct radix_tree_node **nodep, unsigned long *maxindex)
+{
+ struct radix_tree_node *node = rcu_dereference_raw(root->rnode);
+
+ *nodep = node;
+
+ if (likely(radix_tree_is_internal_node(node))) {
+ node = entry_to_node(node);
+ *maxindex = node_maxindex(node);
+ return node->shift + RADIX_TREE_MAP_SHIFT;
+ }
+
+ *maxindex = 0;
+ return 0;
}
/*
* Extend a radix tree so it can store key @index.
*/
static int radix_tree_extend(struct radix_tree_root *root,
- unsigned long index, unsigned order)
+ unsigned long index, unsigned int shift)
{
- struct radix_tree_node *node;
struct radix_tree_node *slot;
- unsigned int height;
+ unsigned int maxshift;
int tag;
- /* Figure out what the height should be. */
- height = root->height + 1;
- while (index > radix_tree_maxindex(height))
- height++;
+ /* Figure out what the shift should be. */
+ maxshift = shift;
+ while (index > shift_maxindex(maxshift))
+ maxshift += RADIX_TREE_MAP_SHIFT;
- if ((root->rnode == NULL) && (order == 0)) {
- root->height = height;
+ slot = root->rnode;
+ if (!slot)
goto out;
- }
do {
- unsigned int newheight;
- if (!(node = radix_tree_node_alloc(root)))
+ struct radix_tree_node *node = radix_tree_node_alloc(root);
+
+ if (!node)
return -ENOMEM;
/* Propagate the aggregated tag info into the new root */
@@ -397,25 +463,20 @@
tag_set(node, tag, 0);
}
- /* Increase the height. */
- newheight = root->height+1;
- BUG_ON(newheight & ~RADIX_TREE_HEIGHT_MASK);
- node->path = newheight;
+ BUG_ON(shift > BITS_PER_LONG);
+ node->shift = shift;
+ node->offset = 0;
node->count = 1;
node->parent = NULL;
- slot = root->rnode;
- if (radix_tree_is_indirect_ptr(slot) && newheight > 1) {
- slot = indirect_to_ptr(slot);
- slot->parent = node;
- slot = ptr_to_indirect(slot);
- }
+ if (radix_tree_is_internal_node(slot))
+ entry_to_node(slot)->parent = node;
node->slots[0] = slot;
- node = ptr_to_indirect(node);
- rcu_assign_pointer(root->rnode, node);
- root->height = newheight;
- } while (height > root->height);
+ slot = node_to_entry(node);
+ rcu_assign_pointer(root->rnode, slot);
+ shift += RADIX_TREE_MAP_SHIFT;
+ } while (shift <= maxshift);
out:
- return 0;
+ return maxshift + RADIX_TREE_MAP_SHIFT;
}
/**
@@ -439,71 +500,70 @@
unsigned order, struct radix_tree_node **nodep,
void ***slotp)
{
- struct radix_tree_node *node = NULL, *slot;
- unsigned int height, shift, offset;
- int error;
+ struct radix_tree_node *node = NULL, *child;
+ void **slot = (void **)&root->rnode;
+ unsigned long maxindex;
+ unsigned int shift, offset = 0;
+ unsigned long max = index | ((1UL << order) - 1);
- BUG_ON((0 < order) && (order < RADIX_TREE_MAP_SHIFT));
+ shift = radix_tree_load_root(root, &child, &maxindex);
/* Make sure the tree is high enough. */
- if (index > radix_tree_maxindex(root->height)) {
- error = radix_tree_extend(root, index, order);
- if (error)
+ if (max > maxindex) {
+ int error = radix_tree_extend(root, max, shift);
+ if (error < 0)
return error;
+ shift = error;
+ child = root->rnode;
+ if (order == shift)
+ shift += RADIX_TREE_MAP_SHIFT;
}
- slot = root->rnode;
-
- height = root->height;
- shift = height * RADIX_TREE_MAP_SHIFT;
-
- offset = 0; /* uninitialised var warning */
while (shift > order) {
- if (slot == NULL) {
+ shift -= RADIX_TREE_MAP_SHIFT;
+ if (child == NULL) {
/* Have to add a child node. */
- if (!(slot = radix_tree_node_alloc(root)))
+ child = radix_tree_node_alloc(root);
+ if (!child)
return -ENOMEM;
- slot->path = height;
- slot->parent = node;
- if (node) {
- rcu_assign_pointer(node->slots[offset],
- ptr_to_indirect(slot));
+ child->shift = shift;
+ child->offset = offset;
+ child->parent = node;
+ rcu_assign_pointer(*slot, node_to_entry(child));
+ if (node)
node->count++;
- slot->path |= offset << RADIX_TREE_HEIGHT_SHIFT;
- } else
- rcu_assign_pointer(root->rnode,
- ptr_to_indirect(slot));
- } else if (!radix_tree_is_indirect_ptr(slot))
+ } else if (!radix_tree_is_internal_node(child))
break;
/* Go a level down */
- height--;
- shift -= RADIX_TREE_MAP_SHIFT;
- offset = (index >> shift) & RADIX_TREE_MAP_MASK;
- node = indirect_to_ptr(slot);
- slot = node->slots[offset];
+ node = entry_to_node(child);
+ offset = radix_tree_descend(node, &child, index);
+ slot = &node->slots[offset];
}
+#ifdef CONFIG_RADIX_TREE_MULTIORDER
/* Insert pointers to the canonical entry */
- if ((shift - order) > 0) {
- int i, n = 1 << (shift - order);
+ if (order > shift) {
+ unsigned i, n = 1 << (order - shift);
offset = offset & ~(n - 1);
- slot = ptr_to_indirect(&node->slots[offset]);
+ slot = &node->slots[offset];
+ child = node_to_entry(slot);
for (i = 0; i < n; i++) {
- if (node->slots[offset + i])
+ if (slot[i])
return -EEXIST;
}
for (i = 1; i < n; i++) {
- rcu_assign_pointer(node->slots[offset + i], slot);
+ rcu_assign_pointer(slot[i], child);
node->count++;
}
}
+#endif
if (nodep)
*nodep = node;
if (slotp)
- *slotp = node ? node->slots + offset : (void **)&root->rnode;
+ *slotp = slot;
return 0;
}
@@ -523,7 +583,7 @@
void **slot;
int error;
- BUG_ON(radix_tree_is_indirect_ptr(item));
+ BUG_ON(radix_tree_is_internal_node(item));
error = __radix_tree_create(root, index, order, &node, &slot);
if (error)
@@ -533,12 +593,13 @@
rcu_assign_pointer(*slot, item);
if (node) {
+ unsigned offset = get_slot_offset(node, slot);
node->count++;
- BUG_ON(tag_get(node, 0, index & RADIX_TREE_MAP_MASK));
- BUG_ON(tag_get(node, 1, index & RADIX_TREE_MAP_MASK));
+ BUG_ON(tag_get(node, 0, offset));
+ BUG_ON(tag_get(node, 1, offset));
+ BUG_ON(tag_get(node, 2, offset));
} else {
- BUG_ON(root_tag_get(root, 0));
- BUG_ON(root_tag_get(root, 1));
+ BUG_ON(root_tags_get(root));
}
return 0;
@@ -563,44 +624,25 @@
struct radix_tree_node **nodep, void ***slotp)
{
struct radix_tree_node *node, *parent;
- unsigned int height, shift;
+ unsigned long maxindex;
void **slot;
- node = rcu_dereference_raw(root->rnode);
- if (node == NULL)
+ restart:
+ parent = NULL;
+ slot = (void **)&root->rnode;
+ radix_tree_load_root(root, &node, &maxindex);
+ if (index > maxindex)
return NULL;
- if (!radix_tree_is_indirect_ptr(node)) {
- if (index > 0)
- return NULL;
+ while (radix_tree_is_internal_node(node)) {
+ unsigned offset;
- if (nodep)
- *nodep = NULL;
- if (slotp)
- *slotp = (void **)&root->rnode;
- return node;
+ if (node == RADIX_TREE_RETRY)
+ goto restart;
+ parent = entry_to_node(node);
+ offset = radix_tree_descend(parent, &node, index);
+ slot = parent->slots + offset;
}
- node = indirect_to_ptr(node);
-
- height = node->path & RADIX_TREE_HEIGHT_MASK;
- if (index > radix_tree_maxindex(height))
- return NULL;
-
- shift = (height-1) * RADIX_TREE_MAP_SHIFT;
-
- do {
- parent = node;
- slot = node->slots + ((index >> shift) & RADIX_TREE_MAP_MASK);
- node = rcu_dereference_raw(*slot);
- if (node == NULL)
- return NULL;
- if (!radix_tree_is_indirect_ptr(node))
- break;
- node = indirect_to_ptr(node);
-
- shift -= RADIX_TREE_MAP_SHIFT;
- height--;
- } while (height > 0);
if (nodep)
*nodep = parent;
@@ -654,59 +696,72 @@
* radix_tree_tag_set - set a tag on a radix tree node
* @root: radix tree root
* @index: index key
- * @tag: tag index
+ * @tag: tag index
*
* Set the search tag (which must be < RADIX_TREE_MAX_TAGS)
* corresponding to @index in the radix tree. From
* the root all the way down to the leaf node.
*
- * Returns the address of the tagged item. Setting a tag on a not-present
+ * Returns the address of the tagged item. Setting a tag on a not-present
* item is a bug.
*/
void *radix_tree_tag_set(struct radix_tree_root *root,
unsigned long index, unsigned int tag)
{
- unsigned int height, shift;
- struct radix_tree_node *slot;
+ struct radix_tree_node *node, *parent;
+ unsigned long maxindex;
- height = root->height;
- BUG_ON(index > radix_tree_maxindex(height));
+ radix_tree_load_root(root, &node, &maxindex);
+ BUG_ON(index > maxindex);
- slot = indirect_to_ptr(root->rnode);
- shift = (height - 1) * RADIX_TREE_MAP_SHIFT;
+ while (radix_tree_is_internal_node(node)) {
+ unsigned offset;
- while (height > 0) {
- int offset;
+ parent = entry_to_node(node);
+ offset = radix_tree_descend(parent, &node, index);
+ BUG_ON(!node);
- offset = (index >> shift) & RADIX_TREE_MAP_MASK;
- if (!tag_get(slot, tag, offset))
- tag_set(slot, tag, offset);
- slot = slot->slots[offset];
- BUG_ON(slot == NULL);
- if (!radix_tree_is_indirect_ptr(slot))
- break;
- slot = indirect_to_ptr(slot);
- shift -= RADIX_TREE_MAP_SHIFT;
- height--;
+ if (!tag_get(parent, tag, offset))
+ tag_set(parent, tag, offset);
}
/* set the root's tag bit */
- if (slot && !root_tag_get(root, tag))
+ if (!root_tag_get(root, tag))
root_tag_set(root, tag);
- return slot;
+ return node;
}
EXPORT_SYMBOL(radix_tree_tag_set);
+static void node_tag_clear(struct radix_tree_root *root,
+ struct radix_tree_node *node,
+ unsigned int tag, unsigned int offset)
+{
+ while (node) {
+ if (!tag_get(node, tag, offset))
+ return;
+ tag_clear(node, tag, offset);
+ if (any_tag_set(node, tag))
+ return;
+
+ offset = node->offset;
+ node = node->parent;
+ }
+
+ /* clear the root's tag bit */
+ if (root_tag_get(root, tag))
+ root_tag_clear(root, tag);
+}
+
/**
* radix_tree_tag_clear - clear a tag on a radix tree node
* @root: radix tree root
* @index: index key
- * @tag: tag index
+ * @tag: tag index
*
* Clear the search tag (which must be < RADIX_TREE_MAX_TAGS)
- * corresponding to @index in the radix tree. If
- * this causes the leaf node to have no tags set then clear the tag in the
+ * corresponding to @index in the radix tree. If this causes
+ * the leaf node to have no tags set then clear the tag in the
* next-to-leaf node, etc.
*
* Returns the address of the tagged item on success, else NULL. ie:
@@ -715,52 +770,25 @@
void *radix_tree_tag_clear(struct radix_tree_root *root,
unsigned long index, unsigned int tag)
{
- struct radix_tree_node *node = NULL;
- struct radix_tree_node *slot = NULL;
- unsigned int height, shift;
+ struct radix_tree_node *node, *parent;
+ unsigned long maxindex;
int uninitialized_var(offset);
- height = root->height;
- if (index > radix_tree_maxindex(height))
- goto out;
+ radix_tree_load_root(root, &node, &maxindex);
+ if (index > maxindex)
+ return NULL;
- shift = height * RADIX_TREE_MAP_SHIFT;
- slot = root->rnode;
+ parent = NULL;
- while (shift) {
- if (slot == NULL)
- goto out;
- if (!radix_tree_is_indirect_ptr(slot))
- break;
- slot = indirect_to_ptr(slot);
-
- shift -= RADIX_TREE_MAP_SHIFT;
- offset = (index >> shift) & RADIX_TREE_MAP_MASK;
- node = slot;
- slot = slot->slots[offset];
+ while (radix_tree_is_internal_node(node)) {
+ parent = entry_to_node(node);
+ offset = radix_tree_descend(parent, &node, index);
}
- if (slot == NULL)
- goto out;
+ if (node)
+ node_tag_clear(root, parent, tag, offset);
- while (node) {
- if (!tag_get(node, tag, offset))
- goto out;
- tag_clear(node, tag, offset);
- if (any_tag_set(node, tag))
- goto out;
-
- index >>= RADIX_TREE_MAP_SHIFT;
- offset = index & RADIX_TREE_MAP_MASK;
- node = node->parent;
- }
-
- /* clear the root's tag bit */
- if (root_tag_get(root, tag))
- root_tag_clear(root, tag);
-
-out:
- return slot;
+ return node;
}
EXPORT_SYMBOL(radix_tree_tag_clear);
@@ -768,7 +796,7 @@
* radix_tree_tag_get - get a tag on a radix tree node
* @root: radix tree root
* @index: index key
- * @tag: tag index (< RADIX_TREE_MAX_TAGS)
+ * @tag: tag index (< RADIX_TREE_MAX_TAGS)
*
* Return values:
*
@@ -782,48 +810,44 @@
int radix_tree_tag_get(struct radix_tree_root *root,
unsigned long index, unsigned int tag)
{
- unsigned int height, shift;
- struct radix_tree_node *node;
+ struct radix_tree_node *node, *parent;
+ unsigned long maxindex;
- /* check the root's tag bit */
if (!root_tag_get(root, tag))
return 0;
- node = rcu_dereference_raw(root->rnode);
+ radix_tree_load_root(root, &node, &maxindex);
+ if (index > maxindex)
+ return 0;
if (node == NULL)
return 0;
- if (!radix_tree_is_indirect_ptr(node))
- return (index == 0);
- node = indirect_to_ptr(node);
+ while (radix_tree_is_internal_node(node)) {
+ unsigned offset;
- height = node->path & RADIX_TREE_HEIGHT_MASK;
- if (index > radix_tree_maxindex(height))
- return 0;
+ parent = entry_to_node(node);
+ offset = radix_tree_descend(parent, &node, index);
- shift = (height - 1) * RADIX_TREE_MAP_SHIFT;
-
- for ( ; ; ) {
- int offset;
-
- if (node == NULL)
+ if (!node)
return 0;
- node = indirect_to_ptr(node);
-
- offset = (index >> shift) & RADIX_TREE_MAP_MASK;
- if (!tag_get(node, tag, offset))
+ if (!tag_get(parent, tag, offset))
return 0;
- if (height == 1)
- return 1;
- node = rcu_dereference_raw(node->slots[offset]);
- if (!radix_tree_is_indirect_ptr(node))
- return 1;
- shift -= RADIX_TREE_MAP_SHIFT;
- height--;
+ if (node == RADIX_TREE_RETRY)
+ break;
}
+
+ return 1;
}
EXPORT_SYMBOL(radix_tree_tag_get);
+static inline void __set_iter_shift(struct radix_tree_iter *iter,
+ unsigned int shift)
+{
+#ifdef CONFIG_RADIX_TREE_MULTIORDER
+ iter->shift = shift;
+#endif
+}
+
/**
* radix_tree_next_chunk - find next chunk of slots for iteration
*
@@ -835,9 +859,9 @@
void **radix_tree_next_chunk(struct radix_tree_root *root,
struct radix_tree_iter *iter, unsigned flags)
{
- unsigned shift, tag = flags & RADIX_TREE_ITER_TAG_MASK;
- struct radix_tree_node *rnode, *node;
- unsigned long index, offset, height;
+ unsigned tag = flags & RADIX_TREE_ITER_TAG_MASK;
+ struct radix_tree_node *node, *child;
+ unsigned long index, offset, maxindex;
if ((flags & RADIX_TREE_ITER_TAGGED) && !root_tag_get(root, tag))
return NULL;
@@ -855,33 +879,28 @@
if (!index && iter->index)
return NULL;
- rnode = rcu_dereference_raw(root->rnode);
- if (radix_tree_is_indirect_ptr(rnode)) {
- rnode = indirect_to_ptr(rnode);
- } else if (rnode && !index) {
+ restart:
+ radix_tree_load_root(root, &child, &maxindex);
+ if (index > maxindex)
+ return NULL;
+ if (!child)
+ return NULL;
+
+ if (!radix_tree_is_internal_node(child)) {
/* Single-slot tree */
- iter->index = 0;
- iter->next_index = 1;
+ iter->index = index;
+ iter->next_index = maxindex + 1;
iter->tags = 1;
+ __set_iter_shift(iter, 0);
return (void **)&root->rnode;
- } else
- return NULL;
+ }
-restart:
- height = rnode->path & RADIX_TREE_HEIGHT_MASK;
- shift = (height - 1) * RADIX_TREE_MAP_SHIFT;
- offset = index >> shift;
+ do {
+ node = entry_to_node(child);
+ offset = radix_tree_descend(node, &child, index);
- /* Index outside of the tree */
- if (offset >= RADIX_TREE_MAP_SIZE)
- return NULL;
-
- node = rnode;
- while (1) {
- struct radix_tree_node *slot;
if ((flags & RADIX_TREE_ITER_TAGGED) ?
- !test_bit(offset, node->tags[tag]) :
- !node->slots[offset]) {
+ !tag_get(node, tag, offset) : !child) {
/* Hole detected */
if (flags & RADIX_TREE_ITER_CONTIG)
return NULL;
@@ -893,35 +912,30 @@
offset + 1);
else
while (++offset < RADIX_TREE_MAP_SIZE) {
- if (node->slots[offset])
+ void *slot = node->slots[offset];
+ if (is_sibling_entry(node, slot))
+ continue;
+ if (slot)
break;
}
- index &= ~((RADIX_TREE_MAP_SIZE << shift) - 1);
- index += offset << shift;
+ index &= ~node_maxindex(node);
+ index += offset << node->shift;
/* Overflow after ~0UL */
if (!index)
return NULL;
if (offset == RADIX_TREE_MAP_SIZE)
goto restart;
+ child = rcu_dereference_raw(node->slots[offset]);
}
- /* This is leaf-node */
- if (!shift)
- break;
-
- slot = rcu_dereference_raw(node->slots[offset]);
- if (slot == NULL)
+ if ((child == NULL) || (child == RADIX_TREE_RETRY))
goto restart;
- if (!radix_tree_is_indirect_ptr(slot))
- break;
- node = indirect_to_ptr(slot);
- shift -= RADIX_TREE_MAP_SHIFT;
- offset = (index >> shift) & RADIX_TREE_MAP_MASK;
- }
+ } while (radix_tree_is_internal_node(child));
/* Update the iterator state */
- iter->index = index;
- iter->next_index = (index | RADIX_TREE_MAP_MASK) + 1;
+ iter->index = (index &~ node_maxindex(node)) | (offset << node->shift);
+ iter->next_index = (index | node_maxindex(node)) + 1;
+ __set_iter_shift(iter, node->shift);
/* Construct iter->tags bit-mask from node->tags[tag] array */
if (flags & RADIX_TREE_ITER_TAGGED) {
@@ -967,7 +981,7 @@
* set is outside the range we are scanning. This reults in dangling tags and
* can lead to problems with later tag operations (e.g. livelocks on lookups).
*
- * The function returns number of leaves where the tag was set and sets
+ * The function returns the number of leaves where the tag was set and sets
* *first_indexp to the first unscanned index.
* WARNING! *first_indexp can wrap if last_index is ULONG_MAX. Caller must
* be prepared to handle that.
@@ -977,14 +991,13 @@
unsigned long nr_to_tag,
unsigned int iftag, unsigned int settag)
{
- unsigned int height = root->height;
- struct radix_tree_node *node = NULL;
- struct radix_tree_node *slot;
- unsigned int shift;
+ struct radix_tree_node *parent, *node, *child;
+ unsigned long maxindex;
unsigned long tagged = 0;
unsigned long index = *first_indexp;
- last_index = min(last_index, radix_tree_maxindex(height));
+ radix_tree_load_root(root, &child, &maxindex);
+ last_index = min(last_index, maxindex);
if (index > last_index)
return 0;
if (!nr_to_tag)
@@ -993,80 +1006,62 @@
*first_indexp = last_index + 1;
return 0;
}
- if (height == 0) {
+ if (!radix_tree_is_internal_node(child)) {
*first_indexp = last_index + 1;
root_tag_set(root, settag);
return 1;
}
- shift = (height - 1) * RADIX_TREE_MAP_SHIFT;
- slot = indirect_to_ptr(root->rnode);
+ node = entry_to_node(child);
for (;;) {
- unsigned long upindex;
- int offset;
-
- offset = (index >> shift) & RADIX_TREE_MAP_MASK;
- if (!slot->slots[offset])
+ unsigned offset = radix_tree_descend(node, &child, index);
+ if (!child)
goto next;
- if (!tag_get(slot, iftag, offset))
+ if (!tag_get(node, iftag, offset))
goto next;
- if (shift) {
- node = slot;
- slot = slot->slots[offset];
- if (radix_tree_is_indirect_ptr(slot)) {
- slot = indirect_to_ptr(slot);
- shift -= RADIX_TREE_MAP_SHIFT;
- continue;
- } else {
- slot = node;
- node = node->parent;
- }
+ /* Sibling slots never have tags set on them */
+ if (radix_tree_is_internal_node(child)) {
+ node = entry_to_node(child);
+ continue;
}
/* tag the leaf */
- tagged += 1 << shift;
- tag_set(slot, settag, offset);
+ tagged++;
+ tag_set(node, settag, offset);
/* walk back up the path tagging interior nodes */
- upindex = index;
- while (node) {
- upindex >>= RADIX_TREE_MAP_SHIFT;
- offset = upindex & RADIX_TREE_MAP_MASK;
-
- /* stop if we find a node with the tag already set */
- if (tag_get(node, settag, offset))
+ parent = node;
+ for (;;) {
+ offset = parent->offset;
+ parent = parent->parent;
+ if (!parent)
break;
- tag_set(node, settag, offset);
- node = node->parent;
+ /* stop if we find a node with the tag already set */
+ if (tag_get(parent, settag, offset))
+ break;
+ tag_set(parent, settag, offset);
}
-
- /*
- * Small optimization: now clear that node pointer.
- * Since all of this slot's ancestors now have the tag set
- * from setting it above, we have no further need to walk
- * back up the tree setting tags, until we update slot to
- * point to another radix_tree_node.
- */
- node = NULL;
-
-next:
- /* Go to next item at level determined by 'shift' */
- index = ((index >> shift) + 1) << shift;
+ next:
+ /* Go to next entry in node */
+ index = ((index >> node->shift) + 1) << node->shift;
/* Overflow can happen when last_index is ~0UL... */
if (index > last_index || !index)
break;
- if (tagged >= nr_to_tag)
- break;
- while (((index >> shift) & RADIX_TREE_MAP_MASK) == 0) {
+ offset = (index >> node->shift) & RADIX_TREE_MAP_MASK;
+ while (offset == 0) {
/*
* We've fully scanned this node. Go up. Because
* last_index is guaranteed to be in the tree, what
* we do below cannot wander astray.
*/
- slot = slot->parent;
- shift += RADIX_TREE_MAP_SHIFT;
+ node = node->parent;
+ offset = (index >> node->shift) & RADIX_TREE_MAP_MASK;
}
+ if (is_sibling_entry(node, node->slots[offset]))
+ goto next;
+ if (tagged >= nr_to_tag)
+ break;
}
/*
* We need not to tag the root tag if there is no tag which is set with
@@ -1095,9 +1090,10 @@
*
* Like radix_tree_lookup, radix_tree_gang_lookup may be called under
* rcu_read_lock. In this case, rather than the returned results being
- * an atomic snapshot of the tree at a single point in time, the semantics
- * of an RCU protected gang lookup are as though multiple radix_tree_lookups
- * have been issued in individual locks, and results stored in 'results'.
+ * an atomic snapshot of the tree at a single point in time, the
+ * semantics of an RCU protected gang lookup are as though multiple
+ * radix_tree_lookups have been issued in individual locks, and results
+ * stored in 'results'.
*/
unsigned int
radix_tree_gang_lookup(struct radix_tree_root *root, void **results,
@@ -1114,7 +1110,7 @@
results[ret] = rcu_dereference_raw(*slot);
if (!results[ret])
continue;
- if (radix_tree_is_indirect_ptr(results[ret])) {
+ if (radix_tree_is_internal_node(results[ret])) {
slot = radix_tree_iter_retry(&iter);
continue;
}
@@ -1197,7 +1193,7 @@
results[ret] = rcu_dereference_raw(*slot);
if (!results[ret])
continue;
- if (radix_tree_is_indirect_ptr(results[ret])) {
+ if (radix_tree_is_internal_node(results[ret])) {
slot = radix_tree_iter_retry(&iter);
continue;
}
@@ -1247,58 +1243,48 @@
#if defined(CONFIG_SHMEM) && defined(CONFIG_SWAP)
#include <linux/sched.h> /* for cond_resched() */
+struct locate_info {
+ unsigned long found_index;
+ bool stop;
+};
+
/*
* This linear search is at present only useful to shmem_unuse_inode().
*/
static unsigned long __locate(struct radix_tree_node *slot, void *item,
- unsigned long index, unsigned long *found_index)
+ unsigned long index, struct locate_info *info)
{
- unsigned int shift, height;
unsigned long i;
- height = slot->path & RADIX_TREE_HEIGHT_MASK;
- shift = (height-1) * RADIX_TREE_MAP_SHIFT;
+ do {
+ unsigned int shift = slot->shift;
- for ( ; height > 1; height--) {
- i = (index >> shift) & RADIX_TREE_MAP_MASK;
- for (;;) {
- if (slot->slots[i] != NULL)
- break;
- index &= ~((1UL << shift) - 1);
- index += 1UL << shift;
- if (index == 0)
- goto out; /* 32-bit wraparound */
- i++;
- if (i == RADIX_TREE_MAP_SIZE)
+ for (i = (index >> shift) & RADIX_TREE_MAP_MASK;
+ i < RADIX_TREE_MAP_SIZE;
+ i++, index += (1UL << shift)) {
+ struct radix_tree_node *node =
+ rcu_dereference_raw(slot->slots[i]);
+ if (node == RADIX_TREE_RETRY)
goto out;
- }
-
- slot = rcu_dereference_raw(slot->slots[i]);
- if (slot == NULL)
- goto out;
- if (!radix_tree_is_indirect_ptr(slot)) {
- if (slot == item) {
- *found_index = index + i;
- index = 0;
- } else {
- index += shift;
+ if (!radix_tree_is_internal_node(node)) {
+ if (node == item) {
+ info->found_index = index;
+ info->stop = true;
+ goto out;
+ }
+ continue;
}
- goto out;
+ node = entry_to_node(node);
+ if (is_sibling_entry(slot, node))
+ continue;
+ slot = node;
+ break;
}
- slot = indirect_to_ptr(slot);
- shift -= RADIX_TREE_MAP_SHIFT;
- }
+ } while (i < RADIX_TREE_MAP_SIZE);
- /* Bottom level: check items */
- for (i = 0; i < RADIX_TREE_MAP_SIZE; i++) {
- if (slot->slots[i] == item) {
- *found_index = index + i;
- index = 0;
- goto out;
- }
- }
- index += RADIX_TREE_MAP_SIZE;
out:
+ if ((index == 0) && (i == RADIX_TREE_MAP_SIZE))
+ info->stop = true;
return index;
}
@@ -1316,32 +1302,35 @@
struct radix_tree_node *node;
unsigned long max_index;
unsigned long cur_index = 0;
- unsigned long found_index = -1;
+ struct locate_info info = {
+ .found_index = -1,
+ .stop = false,
+ };
do {
rcu_read_lock();
node = rcu_dereference_raw(root->rnode);
- if (!radix_tree_is_indirect_ptr(node)) {
+ if (!radix_tree_is_internal_node(node)) {
rcu_read_unlock();
if (node == item)
- found_index = 0;
+ info.found_index = 0;
break;
}
- node = indirect_to_ptr(node);
- max_index = radix_tree_maxindex(node->path &
- RADIX_TREE_HEIGHT_MASK);
+ node = entry_to_node(node);
+
+ max_index = node_maxindex(node);
if (cur_index > max_index) {
rcu_read_unlock();
break;
}
- cur_index = __locate(node, item, cur_index, &found_index);
+ cur_index = __locate(node, item, cur_index, &info);
rcu_read_unlock();
cond_resched();
- } while (cur_index != 0 && cur_index <= max_index);
+ } while (!info.stop && cur_index <= max_index);
- return found_index;
+ return info.found_index;
}
#else
unsigned long radix_tree_locate_item(struct radix_tree_root *root, void *item)
@@ -1351,47 +1340,45 @@
#endif /* CONFIG_SHMEM && CONFIG_SWAP */
/**
- * radix_tree_shrink - shrink height of a radix tree to minimal
+ * radix_tree_shrink - shrink radix tree to minimum height
* @root radix tree root
*/
-static inline void radix_tree_shrink(struct radix_tree_root *root)
+static inline bool radix_tree_shrink(struct radix_tree_root *root)
{
- /* try to shrink tree height */
- while (root->height > 0) {
- struct radix_tree_node *to_free = root->rnode;
- struct radix_tree_node *slot;
+ bool shrunk = false;
- BUG_ON(!radix_tree_is_indirect_ptr(to_free));
- to_free = indirect_to_ptr(to_free);
+ for (;;) {
+ struct radix_tree_node *node = root->rnode;
+ struct radix_tree_node *child;
+
+ if (!radix_tree_is_internal_node(node))
+ break;
+ node = entry_to_node(node);
/*
* The candidate node has more than one child, or its child
- * is not at the leftmost slot, or it is a multiorder entry,
- * we cannot shrink.
+ * is not at the leftmost slot, or the child is a multiorder
+ * entry, we cannot shrink.
*/
- if (to_free->count != 1)
+ if (node->count != 1)
break;
- slot = to_free->slots[0];
- if (!slot)
+ child = node->slots[0];
+ if (!child)
break;
+ if (!radix_tree_is_internal_node(child) && node->shift)
+ break;
+
+ if (radix_tree_is_internal_node(child))
+ entry_to_node(child)->parent = NULL;
/*
* We don't need rcu_assign_pointer(), since we are simply
* moving the node from one part of the tree to another: if it
* was safe to dereference the old pointer to it
- * (to_free->slots[0]), it will be safe to dereference the new
+ * (node->slots[0]), it will be safe to dereference the new
* one (root->rnode) as far as dependent read barriers go.
*/
- if (root->height > 1) {
- if (!radix_tree_is_indirect_ptr(slot))
- break;
-
- slot = indirect_to_ptr(slot);
- slot->parent = NULL;
- slot = ptr_to_indirect(slot);
- }
- root->rnode = slot;
- root->height--;
+ root->rnode = child;
/*
* We have a dilemma here. The node's slot[0] must not be
@@ -1403,7 +1390,7 @@
* their slot to become empty sooner or later.
*
* For example, lockless pagecache will look up a slot, deref
- * the page pointer, and if the page is 0 refcount it means it
+ * the page pointer, and if the page has 0 refcount it means it
* was concurrently deleted from pagecache so try the deref
* again. Fortunately there is already a requirement for logic
* to retry the entire slot lookup -- the indirect pointer
@@ -1411,12 +1398,14 @@
* also results in a stale slot). So tag the slot as indirect
* to force callers to retry.
*/
- if (root->height == 0)
- *((unsigned long *)&to_free->slots[0]) |=
- RADIX_TREE_INDIRECT_PTR;
+ if (!radix_tree_is_internal_node(child))
+ node->slots[0] = RADIX_TREE_RETRY;
- radix_tree_node_free(to_free);
+ radix_tree_node_free(node);
+ shrunk = true;
}
+
+ return shrunk;
}
/**
@@ -1439,24 +1428,17 @@
struct radix_tree_node *parent;
if (node->count) {
- if (node == indirect_to_ptr(root->rnode)) {
- radix_tree_shrink(root);
- if (root->height == 0)
- deleted = true;
- }
+ if (node == entry_to_node(root->rnode))
+ deleted |= radix_tree_shrink(root);
return deleted;
}
parent = node->parent;
if (parent) {
- unsigned int offset;
-
- offset = node->path >> RADIX_TREE_HEIGHT_SHIFT;
- parent->slots[offset] = NULL;
+ parent->slots[node->offset] = NULL;
parent->count--;
} else {
root_tag_clear_all(root);
- root->height = 0;
root->rnode = NULL;
}
@@ -1469,6 +1451,20 @@
return deleted;
}
+static inline void delete_sibling_entries(struct radix_tree_node *node,
+ void *ptr, unsigned offset)
+{
+#ifdef CONFIG_RADIX_TREE_MULTIORDER
+ int i;
+ for (i = 1; offset + i < RADIX_TREE_MAP_SIZE; i++) {
+ if (node->slots[offset + i] != ptr)
+ break;
+ node->slots[offset + i] = NULL;
+ node->count--;
+ }
+#endif
+}
+
/**
* radix_tree_delete_item - delete an item from a radix tree
* @root: radix tree root
@@ -1484,7 +1480,7 @@
unsigned long index, void *item)
{
struct radix_tree_node *node;
- unsigned int offset, i;
+ unsigned int offset;
void **slot;
void *entry;
int tag;
@@ -1502,24 +1498,13 @@
return entry;
}
- offset = index & RADIX_TREE_MAP_MASK;
+ offset = get_slot_offset(node, slot);
- /*
- * Clear all tags associated with the item to be deleted.
- * This way of doing it would be inefficient, but seldom is any set.
- */
- for (tag = 0; tag < RADIX_TREE_MAX_TAGS; tag++) {
- if (tag_get(node, tag, offset))
- radix_tree_tag_clear(root, index, tag);
- }
+ /* Clear all tags associated with the item to be deleted. */
+ for (tag = 0; tag < RADIX_TREE_MAX_TAGS; tag++)
+ node_tag_clear(root, node, tag, offset);
- /* Delete any sibling slots pointing to this slot */
- for (i = 1; offset + i < RADIX_TREE_MAP_SIZE; i++) {
- if (node->slots[offset + i] != ptr_to_indirect(slot))
- break;
- node->slots[offset + i] = NULL;
- node->count--;
- }
+ delete_sibling_entries(node, node_to_entry(slot), offset);
node->slots[offset] = NULL;
node->count--;
@@ -1544,6 +1529,28 @@
}
EXPORT_SYMBOL(radix_tree_delete);
+struct radix_tree_node *radix_tree_replace_clear_tags(
+ struct radix_tree_root *root,
+ unsigned long index, void *entry)
+{
+ struct radix_tree_node *node;
+ void **slot;
+
+ __radix_tree_lookup(root, index, &node, &slot);
+
+ if (node) {
+ unsigned int tag, offset = get_slot_offset(node, slot);
+ for (tag = 0; tag < RADIX_TREE_MAX_TAGS; tag++)
+ node_tag_clear(root, node, tag, offset);
+ } else {
+ /* Clear root node tags */
+ root->gfp_mask &= __GFP_BITS_MASK;
+ }
+
+ radix_tree_replace_slot(slot, entry);
+ return node;
+}
+
/**
* radix_tree_tagged - test whether any items in the tree are tagged
* @root: radix tree root
@@ -1564,45 +1571,24 @@
INIT_LIST_HEAD(&node->private_list);
}
-static __init unsigned long __maxindex(unsigned int height)
-{
- unsigned int width = height * RADIX_TREE_MAP_SHIFT;
- int shift = RADIX_TREE_INDEX_BITS - width;
-
- if (shift < 0)
- return ~0UL;
- if (shift >= BITS_PER_LONG)
- return 0UL;
- return ~0UL >> shift;
-}
-
-static __init void radix_tree_init_maxindex(void)
-{
- unsigned int i;
-
- for (i = 0; i < ARRAY_SIZE(height_to_maxindex); i++)
- height_to_maxindex[i] = __maxindex(i);
-}
-
static int radix_tree_callback(struct notifier_block *nfb,
- unsigned long action,
- void *hcpu)
+ unsigned long action, void *hcpu)
{
- int cpu = (long)hcpu;
- struct radix_tree_preload *rtp;
- struct radix_tree_node *node;
+ int cpu = (long)hcpu;
+ struct radix_tree_preload *rtp;
+ struct radix_tree_node *node;
- /* Free per-cpu pool of perloaded nodes */
- if (action == CPU_DEAD || action == CPU_DEAD_FROZEN) {
- rtp = &per_cpu(radix_tree_preloads, cpu);
- while (rtp->nr) {
+ /* Free per-cpu pool of preloaded nodes */
+ if (action == CPU_DEAD || action == CPU_DEAD_FROZEN) {
+ rtp = &per_cpu(radix_tree_preloads, cpu);
+ while (rtp->nr) {
node = rtp->nodes;
rtp->nodes = node->private_data;
kmem_cache_free(radix_tree_node_cachep, node);
rtp->nr--;
- }
- }
- return NOTIFY_OK;
+ }
+ }
+ return NOTIFY_OK;
}
void __init radix_tree_init(void)
@@ -1611,6 +1597,5 @@
sizeof(struct radix_tree_node), 0,
SLAB_PANIC | SLAB_RECLAIM_ACCOUNT,
radix_tree_node_ctor);
- radix_tree_init_maxindex();
hotcpu_notifier(radix_tree_callback, 0);
}
diff --git a/lib/sg_pool.c b/lib/sg_pool.c
new file mode 100644
index 0000000..6dd3061
--- /dev/null
+++ b/lib/sg_pool.c
@@ -0,0 +1,172 @@
+#include <linux/module.h>
+#include <linux/scatterlist.h>
+#include <linux/mempool.h>
+#include <linux/slab.h>
+
+#define SG_MEMPOOL_NR ARRAY_SIZE(sg_pools)
+#define SG_MEMPOOL_SIZE 2
+
+struct sg_pool {
+ size_t size;
+ char *name;
+ struct kmem_cache *slab;
+ mempool_t *pool;
+};
+
+#define SP(x) { .size = x, "sgpool-" __stringify(x) }
+#if (SG_CHUNK_SIZE < 32)
+#error SG_CHUNK_SIZE is too small (must be 32 or greater)
+#endif
+static struct sg_pool sg_pools[] = {
+ SP(8),
+ SP(16),
+#if (SG_CHUNK_SIZE > 32)
+ SP(32),
+#if (SG_CHUNK_SIZE > 64)
+ SP(64),
+#if (SG_CHUNK_SIZE > 128)
+ SP(128),
+#if (SG_CHUNK_SIZE > 256)
+#error SG_CHUNK_SIZE is too large (256 MAX)
+#endif
+#endif
+#endif
+#endif
+ SP(SG_CHUNK_SIZE)
+};
+#undef SP
+
+static inline unsigned int sg_pool_index(unsigned short nents)
+{
+ unsigned int index;
+
+ BUG_ON(nents > SG_CHUNK_SIZE);
+
+ if (nents <= 8)
+ index = 0;
+ else
+ index = get_count_order(nents) - 3;
+
+ return index;
+}
+
+static void sg_pool_free(struct scatterlist *sgl, unsigned int nents)
+{
+ struct sg_pool *sgp;
+
+ sgp = sg_pools + sg_pool_index(nents);
+ mempool_free(sgl, sgp->pool);
+}
+
+static struct scatterlist *sg_pool_alloc(unsigned int nents, gfp_t gfp_mask)
+{
+ struct sg_pool *sgp;
+
+ sgp = sg_pools + sg_pool_index(nents);
+ return mempool_alloc(sgp->pool, gfp_mask);
+}
+
+/**
+ * sg_free_table_chained - Free a previously mapped sg table
+ * @table: The sg table header to use
+ * @first_chunk: was first_chunk not NULL in sg_alloc_table_chained?
+ *
+ * Description:
+ * Free an sg table previously allocated and setup with
+ * sg_alloc_table_chained().
+ *
+ **/
+void sg_free_table_chained(struct sg_table *table, bool first_chunk)
+{
+ if (first_chunk && table->orig_nents <= SG_CHUNK_SIZE)
+ return;
+ __sg_free_table(table, SG_CHUNK_SIZE, first_chunk, sg_pool_free);
+}
+EXPORT_SYMBOL_GPL(sg_free_table_chained);
+
+/**
+ * sg_alloc_table_chained - Allocate and chain SGLs in an sg table
+ * @table: The sg table header to use
+ * @nents: Number of entries in sg list
+ * @first_chunk: first SGL
+ *
+ * Description:
+ * Allocate and chain SGLs in an sg table. If @nents@ is larger than
+ * SG_CHUNK_SIZE a chained sg table will be setup.
+ *
+ **/
+int sg_alloc_table_chained(struct sg_table *table, int nents,
+ struct scatterlist *first_chunk)
+{
+ int ret;
+
+ BUG_ON(!nents);
+
+ if (first_chunk) {
+ if (nents <= SG_CHUNK_SIZE) {
+ table->nents = table->orig_nents = nents;
+ sg_init_table(table->sgl, nents);
+ return 0;
+ }
+ }
+
+ ret = __sg_alloc_table(table, nents, SG_CHUNK_SIZE,
+ first_chunk, GFP_ATOMIC, sg_pool_alloc);
+ if (unlikely(ret))
+ sg_free_table_chained(table, (bool)first_chunk);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(sg_alloc_table_chained);
+
+static __init int sg_pool_init(void)
+{
+ int i;
+
+ for (i = 0; i < SG_MEMPOOL_NR; i++) {
+ struct sg_pool *sgp = sg_pools + i;
+ int size = sgp->size * sizeof(struct scatterlist);
+
+ sgp->slab = kmem_cache_create(sgp->name, size, 0,
+ SLAB_HWCACHE_ALIGN, NULL);
+ if (!sgp->slab) {
+ printk(KERN_ERR "SG_POOL: can't init sg slab %s\n",
+ sgp->name);
+ goto cleanup_sdb;
+ }
+
+ sgp->pool = mempool_create_slab_pool(SG_MEMPOOL_SIZE,
+ sgp->slab);
+ if (!sgp->pool) {
+ printk(KERN_ERR "SG_POOL: can't init sg mempool %s\n",
+ sgp->name);
+ goto cleanup_sdb;
+ }
+ }
+
+ return 0;
+
+cleanup_sdb:
+ for (i = 0; i < SG_MEMPOOL_NR; i++) {
+ struct sg_pool *sgp = sg_pools + i;
+ if (sgp->pool)
+ mempool_destroy(sgp->pool);
+ if (sgp->slab)
+ kmem_cache_destroy(sgp->slab);
+ }
+
+ return -ENOMEM;
+}
+
+static __exit void sg_pool_exit(void)
+{
+ int i;
+
+ for (i = 0; i < SG_MEMPOOL_NR; i++) {
+ struct sg_pool *sgp = sg_pools + i;
+ mempool_destroy(sgp->pool);
+ kmem_cache_destroy(sgp->slab);
+ }
+}
+
+module_init(sg_pool_init);
+module_exit(sg_pool_exit);
diff --git a/lib/string_helpers.c b/lib/string_helpers.c
index 5c88204..ecaac2c 100644
--- a/lib/string_helpers.c
+++ b/lib/string_helpers.c
@@ -10,6 +10,10 @@
#include <linux/export.h>
#include <linux/ctype.h>
#include <linux/errno.h>
+#include <linux/fs.h>
+#include <linux/limits.h>
+#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/string_helpers.h>
@@ -534,3 +538,91 @@
return p - dst;
}
EXPORT_SYMBOL(string_escape_mem);
+
+/*
+ * Return an allocated string that has been escaped of special characters
+ * and double quotes, making it safe to log in quotes.
+ */
+char *kstrdup_quotable(const char *src, gfp_t gfp)
+{
+ size_t slen, dlen;
+ char *dst;
+ const int flags = ESCAPE_HEX;
+ const char esc[] = "\f\n\r\t\v\a\e\\\"";
+
+ if (!src)
+ return NULL;
+ slen = strlen(src);
+
+ dlen = string_escape_mem(src, slen, NULL, 0, flags, esc);
+ dst = kmalloc(dlen + 1, gfp);
+ if (!dst)
+ return NULL;
+
+ WARN_ON(string_escape_mem(src, slen, dst, dlen, flags, esc) != dlen);
+ dst[dlen] = '\0';
+
+ return dst;
+}
+EXPORT_SYMBOL_GPL(kstrdup_quotable);
+
+/*
+ * Returns allocated NULL-terminated string containing process
+ * command line, with inter-argument NULLs replaced with spaces,
+ * and other special characters escaped.
+ */
+char *kstrdup_quotable_cmdline(struct task_struct *task, gfp_t gfp)
+{
+ char *buffer, *quoted;
+ int i, res;
+
+ buffer = kmalloc(PAGE_SIZE, GFP_TEMPORARY);
+ if (!buffer)
+ return NULL;
+
+ res = get_cmdline(task, buffer, PAGE_SIZE - 1);
+ buffer[res] = '\0';
+
+ /* Collapse trailing NULLs, leave res pointing to last non-NULL. */
+ while (--res >= 0 && buffer[res] == '\0')
+ ;
+
+ /* Replace inter-argument NULLs. */
+ for (i = 0; i <= res; i++)
+ if (buffer[i] == '\0')
+ buffer[i] = ' ';
+
+ /* Make sure result is printable. */
+ quoted = kstrdup_quotable(buffer, gfp);
+ kfree(buffer);
+ return quoted;
+}
+EXPORT_SYMBOL_GPL(kstrdup_quotable_cmdline);
+
+/*
+ * Returns allocated NULL-terminated string containing pathname,
+ * with special characters escaped, able to be safely logged. If
+ * there is an error, the leading character will be "<".
+ */
+char *kstrdup_quotable_file(struct file *file, gfp_t gfp)
+{
+ char *temp, *pathname;
+
+ if (!file)
+ return kstrdup("<unknown>", gfp);
+
+ /* We add 11 spaces for ' (deleted)' to be appended */
+ temp = kmalloc(PATH_MAX + 11, GFP_TEMPORARY);
+ if (!temp)
+ return kstrdup("<no_memory>", gfp);
+
+ pathname = file_path(file, temp, PATH_MAX + 11);
+ if (IS_ERR(pathname))
+ pathname = kstrdup("<too_long>", gfp);
+ else
+ pathname = kstrdup_quotable(pathname, gfp);
+
+ kfree(temp);
+ return pathname;
+}
+EXPORT_SYMBOL_GPL(kstrdup_quotable_file);
diff --git a/lib/strncpy_from_user.c b/lib/strncpy_from_user.c
index 3384032..33f655e 100644
--- a/lib/strncpy_from_user.c
+++ b/lib/strncpy_from_user.c
@@ -1,5 +1,6 @@
#include <linux/compiler.h>
#include <linux/export.h>
+#include <linux/kasan-checks.h>
#include <linux/uaccess.h>
#include <linux/kernel.h>
#include <linux/errno.h>
@@ -109,6 +110,7 @@
unsigned long max = max_addr - src_addr;
long retval;
+ kasan_check_write(dst, count);
user_access_begin();
retval = do_strncpy_from_user(dst, src, count, max);
user_access_end();
diff --git a/lib/test_kasan.c b/lib/test_kasan.c
index 82169fb..5e51872b 100644
--- a/lib/test_kasan.c
+++ b/lib/test_kasan.c
@@ -12,9 +12,12 @@
#define pr_fmt(fmt) "kasan test: %s " fmt, __func__
#include <linux/kernel.h>
+#include <linux/mman.h>
+#include <linux/mm.h>
#include <linux/printk.h>
#include <linux/slab.h>
#include <linux/string.h>
+#include <linux/uaccess.h>
#include <linux/module.h>
static noinline void __init kmalloc_oob_right(void)
@@ -344,6 +347,70 @@
*(volatile char *)p;
}
+static noinline void __init ksize_unpoisons_memory(void)
+{
+ char *ptr;
+ size_t size = 123, real_size = size;
+
+ pr_info("ksize() unpoisons the whole allocated chunk\n");
+ ptr = kmalloc(size, GFP_KERNEL);
+ if (!ptr) {
+ pr_err("Allocation failed\n");
+ return;
+ }
+ real_size = ksize(ptr);
+ /* This access doesn't trigger an error. */
+ ptr[size] = 'x';
+ /* This one does. */
+ ptr[real_size] = 'y';
+ kfree(ptr);
+}
+
+static noinline void __init copy_user_test(void)
+{
+ char *kmem;
+ char __user *usermem;
+ size_t size = 10;
+ int unused;
+
+ kmem = kmalloc(size, GFP_KERNEL);
+ if (!kmem)
+ return;
+
+ usermem = (char __user *)vm_mmap(NULL, 0, PAGE_SIZE,
+ PROT_READ | PROT_WRITE | PROT_EXEC,
+ MAP_ANONYMOUS | MAP_PRIVATE, 0);
+ if (IS_ERR(usermem)) {
+ pr_err("Failed to allocate user memory\n");
+ kfree(kmem);
+ return;
+ }
+
+ pr_info("out-of-bounds in copy_from_user()\n");
+ unused = copy_from_user(kmem, usermem, size + 1);
+
+ pr_info("out-of-bounds in copy_to_user()\n");
+ unused = copy_to_user(usermem, kmem, size + 1);
+
+ pr_info("out-of-bounds in __copy_from_user()\n");
+ unused = __copy_from_user(kmem, usermem, size + 1);
+
+ pr_info("out-of-bounds in __copy_to_user()\n");
+ unused = __copy_to_user(usermem, kmem, size + 1);
+
+ pr_info("out-of-bounds in __copy_from_user_inatomic()\n");
+ unused = __copy_from_user_inatomic(kmem, usermem, size + 1);
+
+ pr_info("out-of-bounds in __copy_to_user_inatomic()\n");
+ unused = __copy_to_user_inatomic(usermem, kmem, size + 1);
+
+ pr_info("out-of-bounds in strncpy_from_user()\n");
+ unused = strncpy_from_user(kmem, usermem, size + 1);
+
+ vm_munmap((unsigned long)usermem, PAGE_SIZE);
+ kfree(kmem);
+}
+
static int __init kmalloc_tests_init(void)
{
kmalloc_oob_right();
@@ -367,6 +434,8 @@
kmem_cache_oob();
kasan_stack_oob();
kasan_global_oob();
+ ksize_unpoisons_memory();
+ copy_user_test();
return -EAGAIN;
}
diff --git a/lib/uuid.c b/lib/uuid.c
index 398821e..e116ae5 100644
--- a/lib/uuid.c
+++ b/lib/uuid.c
@@ -1,7 +1,7 @@
/*
* Unified UUID/GUID definition
*
- * Copyright (C) 2009, Intel Corp.
+ * Copyright (C) 2009, 2016 Intel Corp.
* Huang Ying <ying.huang@intel.com>
*
* This program is free software; you can redistribute it and/or
@@ -12,17 +12,40 @@
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
+#include <linux/ctype.h>
+#include <linux/errno.h>
#include <linux/export.h>
#include <linux/uuid.h>
#include <linux/random.h>
+const u8 uuid_le_index[16] = {3,2,1,0,5,4,7,6,8,9,10,11,12,13,14,15};
+EXPORT_SYMBOL(uuid_le_index);
+const u8 uuid_be_index[16] = {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15};
+EXPORT_SYMBOL(uuid_be_index);
+
+/***************************************************************
+ * Random UUID interface
+ *
+ * Used here for a Boot ID, but can be useful for other kernel
+ * drivers.
+ ***************************************************************/
+
+/*
+ * Generate random UUID
+ */
+void generate_random_uuid(unsigned char uuid[16])
+{
+ get_random_bytes(uuid, 16);
+ /* Set UUID version to 4 --- truly random generation */
+ uuid[6] = (uuid[6] & 0x0F) | 0x40;
+ /* Set the UUID variant to DCE */
+ uuid[8] = (uuid[8] & 0x3F) | 0x80;
+}
+EXPORT_SYMBOL(generate_random_uuid);
+
static void __uuid_gen_common(__u8 b[16])
{
prandom_bytes(b, 16);
@@ -45,3 +68,61 @@
bu->b[6] = (bu->b[6] & 0x0F) | 0x40;
}
EXPORT_SYMBOL_GPL(uuid_be_gen);
+
+/**
+ * uuid_is_valid - checks if UUID string valid
+ * @uuid: UUID string to check
+ *
+ * Description:
+ * It checks if the UUID string is following the format:
+ * xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
+ * where x is a hex digit.
+ *
+ * Return: true if input is valid UUID string.
+ */
+bool uuid_is_valid(const char *uuid)
+{
+ unsigned int i;
+
+ for (i = 0; i < UUID_STRING_LEN; i++) {
+ if (i == 8 || i == 13 || i == 18 || i == 23) {
+ if (uuid[i] != '-')
+ return false;
+ } else if (!isxdigit(uuid[i])) {
+ return false;
+ }
+ }
+
+ return true;
+}
+EXPORT_SYMBOL(uuid_is_valid);
+
+static int __uuid_to_bin(const char *uuid, __u8 b[16], const u8 ei[16])
+{
+ static const u8 si[16] = {0,2,4,6,9,11,14,16,19,21,24,26,28,30,32,34};
+ unsigned int i;
+
+ if (!uuid_is_valid(uuid))
+ return -EINVAL;
+
+ for (i = 0; i < 16; i++) {
+ int hi = hex_to_bin(uuid[si[i]] + 0);
+ int lo = hex_to_bin(uuid[si[i]] + 1);
+
+ b[ei[i]] = (hi << 4) | lo;
+ }
+
+ return 0;
+}
+
+int uuid_le_to_bin(const char *uuid, uuid_le *u)
+{
+ return __uuid_to_bin(uuid, u->b, uuid_le_index);
+}
+EXPORT_SYMBOL(uuid_le_to_bin);
+
+int uuid_be_to_bin(const char *uuid, uuid_be *u)
+{
+ return __uuid_to_bin(uuid, u->b, uuid_be_index);
+}
+EXPORT_SYMBOL(uuid_be_to_bin);
diff --git a/lib/vsprintf.c b/lib/vsprintf.c
index ccb664b..0967771 100644
--- a/lib/vsprintf.c
+++ b/lib/vsprintf.c
@@ -30,6 +30,7 @@
#include <linux/ioport.h>
#include <linux/dcache.h>
#include <linux/cred.h>
+#include <linux/uuid.h>
#include <net/addrconf.h>
#ifdef CONFIG_BLOCK
#include <linux/blkdev.h>
@@ -1304,19 +1305,17 @@
char *uuid_string(char *buf, char *end, const u8 *addr,
struct printf_spec spec, const char *fmt)
{
- char uuid[sizeof("xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx")];
+ char uuid[UUID_STRING_LEN + 1];
char *p = uuid;
int i;
- static const u8 be[16] = {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15};
- static const u8 le[16] = {3,2,1,0,5,4,7,6,8,9,10,11,12,13,14,15};
- const u8 *index = be;
+ const u8 *index = uuid_be_index;
bool uc = false;
switch (*(++fmt)) {
case 'L':
uc = true; /* fall-through */
case 'l':
- index = le;
+ index = uuid_le_index;
break;
case 'B':
uc = true;
@@ -1324,7 +1323,10 @@
}
for (i = 0; i < 16; i++) {
- p = hex_byte_pack(p, addr[index[i]]);
+ if (uc)
+ p = hex_byte_pack_upper(p, addr[index[i]]);
+ else
+ p = hex_byte_pack(p, addr[index[i]]);
switch (i) {
case 3:
case 5:
@@ -1337,13 +1339,6 @@
*p = 0;
- if (uc) {
- p = uuid;
- do {
- *p = toupper(*p);
- } while (*(++p));
- }
-
return string(buf, end, uuid, spec);
}
diff --git a/mm/Kconfig b/mm/Kconfig
index 989f8f3..2664c11 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -192,6 +192,22 @@
def_bool y
depends on SPARSEMEM && MEMORY_HOTPLUG
+config MEMORY_HOTPLUG_DEFAULT_ONLINE
+ bool "Online the newly added memory blocks by default"
+ default n
+ depends on MEMORY_HOTPLUG
+ help
+ This option sets the default policy setting for memory hotplug
+ onlining policy (/sys/devices/system/memory/auto_online_blocks) which
+ determines what happens to newly added memory regions. Policy setting
+ can always be changed at runtime.
+ See Documentation/memory-hotplug.txt for more information.
+
+ Say Y here if you want all hot-plugged memory blocks to appear in
+ 'online' state by default.
+ Say N here if you want the default policy to keep all hot-plugged
+ memory blocks in 'offline' state.
+
config MEMORY_HOTREMOVE
bool "Allow for memory hot remove"
select MEMORY_ISOLATION
@@ -268,11 +284,6 @@
config PHYS_ADDR_T_64BIT
def_bool 64BIT || ARCH_PHYS_ADDR_T_64BIT
-config ZONE_DMA_FLAG
- int
- default "0" if !ZONE_DMA
- default "1"
-
config BOUNCE
bool "Enable bounce buffers"
default y
@@ -393,6 +404,7 @@
bool "Transparent Hugepage Support"
depends on HAVE_ARCH_TRANSPARENT_HUGEPAGE
select COMPACTION
+ select RADIX_TREE_MULTIORDER
help
Transparent Hugepages allows the kernel to use huge pages and
huge tlb transparently to the applications whenever possible.
@@ -556,7 +568,7 @@
zsmalloc.
config ZBUD
- tristate "Low density storage for compressed pages"
+ tristate "Low (Up to 2x) density storage for compressed pages"
default n
help
A special purpose allocator for storing compressed pages.
@@ -565,6 +577,16 @@
deterministic reclaim properties that make it preferable to a higher
density approach when reclaim will be used.
+config Z3FOLD
+ tristate "Up to 3x density storage for compressed pages"
+ depends on ZPOOL
+ default n
+ help
+ A special purpose allocator for storing compressed pages.
+ It is designed to store up to three compressed pages per physical
+ page. It is a ZBUD derivative so the simplicity and determinism are
+ still there.
+
config ZSMALLOC
tristate "Memory allocator for compressed pages"
depends on MMU
diff --git a/mm/Makefile b/mm/Makefile
index deb467e..78c6f7d 100644
--- a/mm/Makefile
+++ b/mm/Makefile
@@ -89,6 +89,7 @@
obj-$(CONFIG_ZPOOL) += zpool.o
obj-$(CONFIG_ZBUD) += zbud.o
obj-$(CONFIG_ZSMALLOC) += zsmalloc.o
+obj-$(CONFIG_Z3FOLD) += z3fold.o
obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o
obj-$(CONFIG_CMA) += cma.o
obj-$(CONFIG_MEMORY_BALLOON) += balloon_compaction.o
diff --git a/mm/backing-dev.c b/mm/backing-dev.c
index 0c6317b..ed173b8 100644
--- a/mm/backing-dev.c
+++ b/mm/backing-dev.c
@@ -957,9 +957,8 @@
* jiffies for either a BDI to exit congestion of the given @sync queue
* or a write to complete.
*
- * In the absence of zone congestion, a short sleep or a cond_resched is
- * performed to yield the processor and to allow other subsystems to make
- * a forward progress.
+ * In the absence of zone congestion, cond_resched() is called to yield
+ * the processor if necessary but otherwise does not sleep.
*
* The return value is 0 if the sleep is for the full timeout. Otherwise,
* it is the number of jiffies that were still remaining when the function
@@ -979,20 +978,7 @@
*/
if (atomic_read(&nr_wb_congested[sync]) == 0 ||
!test_bit(ZONE_CONGESTED, &zone->flags)) {
-
- /*
- * Memory allocation/reclaim might be called from a WQ
- * context and the current implementation of the WQ
- * concurrency control doesn't recognize that a particular
- * WQ is congested if the worker thread is looping without
- * ever sleeping. Therefore we have to do a short sleep
- * here rather than calling cond_resched().
- */
- if (current->flags & PF_WQ_WORKER)
- schedule_timeout_uninterruptible(1);
- else
- cond_resched();
-
+ cond_resched();
/* In case we scheduled, work out time remaining */
ret = timeout - (jiffies - start);
if (ret < 0)
diff --git a/mm/compaction.c b/mm/compaction.c
index 8fa2540..1427366 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -42,6 +42,11 @@
#define CREATE_TRACE_POINTS
#include <trace/events/compaction.h>
+#define block_start_pfn(pfn, order) round_down(pfn, 1UL << (order))
+#define block_end_pfn(pfn, order) ALIGN((pfn) + 1, 1UL << (order))
+#define pageblock_start_pfn(pfn) block_start_pfn(pfn, pageblock_order)
+#define pageblock_end_pfn(pfn) block_end_pfn(pfn, pageblock_order)
+
static unsigned long release_freepages(struct list_head *freelist)
{
struct page *page, *next;
@@ -161,7 +166,7 @@
zone->compact_cached_migrate_pfn[0] = zone->zone_start_pfn;
zone->compact_cached_migrate_pfn[1] = zone->zone_start_pfn;
zone->compact_cached_free_pfn =
- round_down(zone_end_pfn(zone) - 1, pageblock_nr_pages);
+ pageblock_start_pfn(zone_end_pfn(zone) - 1);
}
/*
@@ -519,10 +524,10 @@
LIST_HEAD(freelist);
pfn = start_pfn;
- block_start_pfn = pfn & ~(pageblock_nr_pages - 1);
+ block_start_pfn = pageblock_start_pfn(pfn);
if (block_start_pfn < cc->zone->zone_start_pfn)
block_start_pfn = cc->zone->zone_start_pfn;
- block_end_pfn = ALIGN(pfn + 1, pageblock_nr_pages);
+ block_end_pfn = pageblock_end_pfn(pfn);
for (; pfn < end_pfn; pfn += isolated,
block_start_pfn = block_end_pfn,
@@ -538,8 +543,8 @@
* scanning range to right one.
*/
if (pfn >= block_end_pfn) {
- block_start_pfn = pfn & ~(pageblock_nr_pages - 1);
- block_end_pfn = ALIGN(pfn + 1, pageblock_nr_pages);
+ block_start_pfn = pageblock_start_pfn(pfn);
+ block_end_pfn = pageblock_end_pfn(pfn);
block_end_pfn = min(block_end_pfn, end_pfn);
}
@@ -633,12 +638,13 @@
{
struct zone *zone = cc->zone;
unsigned long nr_scanned = 0, nr_isolated = 0;
- struct list_head *migratelist = &cc->migratepages;
struct lruvec *lruvec;
unsigned long flags = 0;
bool locked = false;
struct page *page = NULL, *valid_page = NULL;
unsigned long start_pfn = low_pfn;
+ bool skip_on_failure = false;
+ unsigned long next_skip_pfn = 0;
/*
* Ensure that there are not too many pages isolated from the LRU
@@ -659,10 +665,37 @@
if (compact_should_abort(cc))
return 0;
+ if (cc->direct_compaction && (cc->mode == MIGRATE_ASYNC)) {
+ skip_on_failure = true;
+ next_skip_pfn = block_end_pfn(low_pfn, cc->order);
+ }
+
/* Time to isolate some pages for migration */
for (; low_pfn < end_pfn; low_pfn++) {
bool is_lru;
+ if (skip_on_failure && low_pfn >= next_skip_pfn) {
+ /*
+ * We have isolated all migration candidates in the
+ * previous order-aligned block, and did not skip it due
+ * to failure. We should migrate the pages now and
+ * hopefully succeed compaction.
+ */
+ if (nr_isolated)
+ break;
+
+ /*
+ * We failed to isolate in the previous order-aligned
+ * block. Set the new boundary to the end of the
+ * current block. Note we can't simply increase
+ * next_skip_pfn by 1 << order, as low_pfn might have
+ * been incremented by a higher number due to skipping
+ * a compound or a high-order buddy page in the
+ * previous loop iteration.
+ */
+ next_skip_pfn = block_end_pfn(low_pfn, cc->order);
+ }
+
/*
* Periodically drop the lock (if held) regardless of its
* contention, to give chance to IRQs. Abort async compaction
@@ -674,7 +707,7 @@
break;
if (!pfn_valid_within(low_pfn))
- continue;
+ goto isolate_fail;
nr_scanned++;
page = pfn_to_page(low_pfn);
@@ -729,11 +762,11 @@
if (likely(comp_order < MAX_ORDER))
low_pfn += (1UL << comp_order) - 1;
- continue;
+ goto isolate_fail;
}
if (!is_lru)
- continue;
+ goto isolate_fail;
/*
* Migration will fail if an anonymous page is pinned in memory,
@@ -742,7 +775,7 @@
*/
if (!page_mapping(page) &&
page_count(page) > page_mapcount(page))
- continue;
+ goto isolate_fail;
/* If we already hold the lock, we can skip some rechecking */
if (!locked) {
@@ -753,7 +786,7 @@
/* Recheck PageLRU and PageCompound under lock */
if (!PageLRU(page))
- continue;
+ goto isolate_fail;
/*
* Page become compound since the non-locked check,
@@ -762,7 +795,7 @@
*/
if (unlikely(PageCompound(page))) {
low_pfn += (1UL << compound_order(page)) - 1;
- continue;
+ goto isolate_fail;
}
}
@@ -770,7 +803,7 @@
/* Try isolate the page */
if (__isolate_lru_page(page, isolate_mode) != 0)
- continue;
+ goto isolate_fail;
VM_BUG_ON_PAGE(PageCompound(page), page);
@@ -778,15 +811,55 @@
del_page_from_lru_list(page, lruvec, page_lru(page));
isolate_success:
- list_add(&page->lru, migratelist);
+ list_add(&page->lru, &cc->migratepages);
cc->nr_migratepages++;
nr_isolated++;
+ /*
+ * Record where we could have freed pages by migration and not
+ * yet flushed them to buddy allocator.
+ * - this is the lowest page that was isolated and likely be
+ * then freed by migration.
+ */
+ if (!cc->last_migrated_pfn)
+ cc->last_migrated_pfn = low_pfn;
+
/* Avoid isolating too much */
if (cc->nr_migratepages == COMPACT_CLUSTER_MAX) {
++low_pfn;
break;
}
+
+ continue;
+isolate_fail:
+ if (!skip_on_failure)
+ continue;
+
+ /*
+ * We have isolated some pages, but then failed. Release them
+ * instead of migrating, as we cannot form the cc->order buddy
+ * page anyway.
+ */
+ if (nr_isolated) {
+ if (locked) {
+ spin_unlock_irqrestore(&zone->lru_lock, flags);
+ locked = false;
+ }
+ acct_isolated(zone, cc);
+ putback_movable_pages(&cc->migratepages);
+ cc->nr_migratepages = 0;
+ cc->last_migrated_pfn = 0;
+ nr_isolated = 0;
+ }
+
+ if (low_pfn < next_skip_pfn) {
+ low_pfn = next_skip_pfn - 1;
+ /*
+ * The check near the loop beginning would have updated
+ * next_skip_pfn too, but this is a bit simpler.
+ */
+ next_skip_pfn += 1UL << cc->order;
+ }
}
/*
@@ -834,10 +907,10 @@
/* Scan block by block. First and last block may be incomplete */
pfn = start_pfn;
- block_start_pfn = pfn & ~(pageblock_nr_pages - 1);
+ block_start_pfn = pageblock_start_pfn(pfn);
if (block_start_pfn < cc->zone->zone_start_pfn)
block_start_pfn = cc->zone->zone_start_pfn;
- block_end_pfn = ALIGN(pfn + 1, pageblock_nr_pages);
+ block_end_pfn = pageblock_end_pfn(pfn);
for (; pfn < end_pfn; pfn = block_end_pfn,
block_start_pfn = block_end_pfn,
@@ -924,10 +997,10 @@
* is using.
*/
isolate_start_pfn = cc->free_pfn;
- block_start_pfn = cc->free_pfn & ~(pageblock_nr_pages-1);
+ block_start_pfn = pageblock_start_pfn(cc->free_pfn);
block_end_pfn = min(block_start_pfn + pageblock_nr_pages,
zone_end_pfn(zone));
- low_pfn = ALIGN(cc->migrate_pfn + 1, pageblock_nr_pages);
+ low_pfn = pageblock_end_pfn(cc->migrate_pfn);
/*
* Isolate free pages until enough are available to migrate the
@@ -1070,7 +1143,6 @@
unsigned long block_start_pfn;
unsigned long block_end_pfn;
unsigned long low_pfn;
- unsigned long isolate_start_pfn;
struct page *page;
const isolate_mode_t isolate_mode =
(sysctl_compact_unevictable_allowed ? ISOLATE_UNEVICTABLE : 0) |
@@ -1081,12 +1153,12 @@
* initialized by compact_zone()
*/
low_pfn = cc->migrate_pfn;
- block_start_pfn = cc->migrate_pfn & ~(pageblock_nr_pages - 1);
+ block_start_pfn = pageblock_start_pfn(low_pfn);
if (block_start_pfn < zone->zone_start_pfn)
block_start_pfn = zone->zone_start_pfn;
/* Only scan within a pageblock boundary */
- block_end_pfn = ALIGN(low_pfn + 1, pageblock_nr_pages);
+ block_end_pfn = pageblock_end_pfn(low_pfn);
/*
* Iterate over whole pageblocks until we find the first suitable.
@@ -1125,7 +1197,6 @@
continue;
/* Perform the isolation */
- isolate_start_pfn = low_pfn;
low_pfn = isolate_migratepages_block(cc, low_pfn,
block_end_pfn, isolate_mode);
@@ -1135,15 +1206,6 @@
}
/*
- * Record where we could have freed pages by migration and not
- * yet flushed them to buddy allocator.
- * - this is the lowest page that could have been isolated and
- * then freed by migration.
- */
- if (cc->nr_migratepages && !cc->last_migrated_pfn)
- cc->last_migrated_pfn = isolate_start_pfn;
-
- /*
* Either we isolated something and proceed with migration. Or
* we failed and compact_zone should decide if we should
* continue or not.
@@ -1167,7 +1229,7 @@
return order == -1;
}
-static int __compact_finished(struct zone *zone, struct compact_control *cc,
+static enum compact_result __compact_finished(struct zone *zone, struct compact_control *cc,
const int migratetype)
{
unsigned int order;
@@ -1190,7 +1252,10 @@
if (cc->direct_compaction)
zone->compact_blockskip_flush = true;
- return COMPACT_COMPLETE;
+ if (cc->whole_zone)
+ return COMPACT_COMPLETE;
+ else
+ return COMPACT_PARTIAL_SKIPPED;
}
if (is_via_compact_memory(cc->order))
@@ -1230,8 +1295,9 @@
return COMPACT_NO_SUITABLE_PAGE;
}
-static int compact_finished(struct zone *zone, struct compact_control *cc,
- const int migratetype)
+static enum compact_result compact_finished(struct zone *zone,
+ struct compact_control *cc,
+ const int migratetype)
{
int ret;
@@ -1250,8 +1316,10 @@
* COMPACT_PARTIAL - If the allocation would succeed without compaction
* COMPACT_CONTINUE - If compaction should run now
*/
-static unsigned long __compaction_suitable(struct zone *zone, int order,
- int alloc_flags, int classzone_idx)
+static enum compact_result __compaction_suitable(struct zone *zone, int order,
+ unsigned int alloc_flags,
+ int classzone_idx,
+ unsigned long wmark_target)
{
int fragindex;
unsigned long watermark;
@@ -1274,7 +1342,8 @@
* allocated and for a short time, the footprint is higher
*/
watermark += (2UL << order);
- if (!zone_watermark_ok(zone, 0, watermark, classzone_idx, alloc_flags))
+ if (!__zone_watermark_ok(zone, 0, watermark, classzone_idx,
+ alloc_flags, wmark_target))
return COMPACT_SKIPPED;
/*
@@ -1295,12 +1364,14 @@
return COMPACT_CONTINUE;
}
-unsigned long compaction_suitable(struct zone *zone, int order,
- int alloc_flags, int classzone_idx)
+enum compact_result compaction_suitable(struct zone *zone, int order,
+ unsigned int alloc_flags,
+ int classzone_idx)
{
- unsigned long ret;
+ enum compact_result ret;
- ret = __compaction_suitable(zone, order, alloc_flags, classzone_idx);
+ ret = __compaction_suitable(zone, order, alloc_flags, classzone_idx,
+ zone_page_state(zone, NR_FREE_PAGES));
trace_mm_compaction_suitable(zone, order, ret);
if (ret == COMPACT_NOT_SUITABLE_ZONE)
ret = COMPACT_SKIPPED;
@@ -1308,9 +1379,42 @@
return ret;
}
-static int compact_zone(struct zone *zone, struct compact_control *cc)
+bool compaction_zonelist_suitable(struct alloc_context *ac, int order,
+ int alloc_flags)
{
- int ret;
+ struct zone *zone;
+ struct zoneref *z;
+
+ /*
+ * Make sure at least one zone would pass __compaction_suitable if we continue
+ * retrying the reclaim.
+ */
+ for_each_zone_zonelist_nodemask(zone, z, ac->zonelist, ac->high_zoneidx,
+ ac->nodemask) {
+ unsigned long available;
+ enum compact_result compact_result;
+
+ /*
+ * Do not consider all the reclaimable memory because we do not
+ * want to trash just for a single high order allocation which
+ * is even not guaranteed to appear even if __compaction_suitable
+ * is happy about the watermark check.
+ */
+ available = zone_reclaimable_pages(zone) / order;
+ available += zone_page_state_snapshot(zone, NR_FREE_PAGES);
+ compact_result = __compaction_suitable(zone, order, alloc_flags,
+ ac_classzone_idx(ac), available);
+ if (compact_result != COMPACT_SKIPPED &&
+ compact_result != COMPACT_NOT_SUITABLE_ZONE)
+ return true;
+ }
+
+ return false;
+}
+
+static enum compact_result compact_zone(struct zone *zone, struct compact_control *cc)
+{
+ enum compact_result ret;
unsigned long start_pfn = zone->zone_start_pfn;
unsigned long end_pfn = zone_end_pfn(zone);
const int migratetype = gfpflags_to_migratetype(cc->gfp_mask);
@@ -1318,15 +1422,12 @@
ret = compaction_suitable(zone, cc->order, cc->alloc_flags,
cc->classzone_idx);
- switch (ret) {
- case COMPACT_PARTIAL:
- case COMPACT_SKIPPED:
- /* Compaction is likely to fail */
+ /* Compaction is likely to fail */
+ if (ret == COMPACT_PARTIAL || ret == COMPACT_SKIPPED)
return ret;
- case COMPACT_CONTINUE:
- /* Fall through to compaction */
- ;
- }
+
+ /* huh, compaction_suitable is returning something unexpected */
+ VM_BUG_ON(ret != COMPACT_CONTINUE);
/*
* Clear pageblock skip if there were failures recently and compaction
@@ -1343,7 +1444,7 @@
cc->migrate_pfn = zone->compact_cached_migrate_pfn[sync];
cc->free_pfn = zone->compact_cached_free_pfn;
if (cc->free_pfn < start_pfn || cc->free_pfn >= end_pfn) {
- cc->free_pfn = round_down(end_pfn - 1, pageblock_nr_pages);
+ cc->free_pfn = pageblock_start_pfn(end_pfn - 1);
zone->compact_cached_free_pfn = cc->free_pfn;
}
if (cc->migrate_pfn < start_pfn || cc->migrate_pfn >= end_pfn) {
@@ -1351,6 +1452,10 @@
zone->compact_cached_migrate_pfn[0] = cc->migrate_pfn;
zone->compact_cached_migrate_pfn[1] = cc->migrate_pfn;
}
+
+ if (cc->migrate_pfn == start_pfn)
+ cc->whole_zone = true;
+
cc->last_migrated_pfn = 0;
trace_mm_compaction_begin(start_pfn, cc->migrate_pfn,
@@ -1398,6 +1503,18 @@
ret = COMPACT_CONTENDED;
goto out;
}
+ /*
+ * We failed to migrate at least one page in the current
+ * order-aligned block, so skip the rest of it.
+ */
+ if (cc->direct_compaction &&
+ (cc->mode == MIGRATE_ASYNC)) {
+ cc->migrate_pfn = block_end_pfn(
+ cc->migrate_pfn - 1, cc->order);
+ /* Draining pcplists is useless in this case */
+ cc->last_migrated_pfn = 0;
+
+ }
}
check_drain:
@@ -1411,7 +1528,7 @@
if (cc->order > 0 && cc->last_migrated_pfn) {
int cpu;
unsigned long current_block_start =
- cc->migrate_pfn & ~((1UL << cc->order) - 1);
+ block_start_pfn(cc->migrate_pfn, cc->order);
if (cc->last_migrated_pfn < current_block_start) {
cpu = get_cpu();
@@ -1436,7 +1553,7 @@
cc->nr_freepages = 0;
VM_BUG_ON(free_pfn == 0);
/* The cached pfn is always the first in a pageblock */
- free_pfn &= ~(pageblock_nr_pages-1);
+ free_pfn = pageblock_start_pfn(free_pfn);
/*
* Only go back, not forward. The cached pfn might have been
* already reset to zone end in compact_finished()
@@ -1454,11 +1571,11 @@
return ret;
}
-static unsigned long compact_zone_order(struct zone *zone, int order,
+static enum compact_result compact_zone_order(struct zone *zone, int order,
gfp_t gfp_mask, enum migrate_mode mode, int *contended,
- int alloc_flags, int classzone_idx)
+ unsigned int alloc_flags, int classzone_idx)
{
- unsigned long ret;
+ enum compact_result ret;
struct compact_control cc = {
.nr_freepages = 0,
.nr_migratepages = 0,
@@ -1496,15 +1613,15 @@
*
* This is the main entry point for direct page compaction.
*/
-unsigned long try_to_compact_pages(gfp_t gfp_mask, unsigned int order,
- int alloc_flags, const struct alloc_context *ac,
- enum migrate_mode mode, int *contended)
+enum compact_result try_to_compact_pages(gfp_t gfp_mask, unsigned int order,
+ unsigned int alloc_flags, const struct alloc_context *ac,
+ enum migrate_mode mode, int *contended)
{
int may_enter_fs = gfp_mask & __GFP_FS;
int may_perform_io = gfp_mask & __GFP_IO;
struct zoneref *z;
struct zone *zone;
- int rc = COMPACT_DEFERRED;
+ enum compact_result rc = COMPACT_SKIPPED;
int all_zones_contended = COMPACT_CONTENDED_LOCK; /* init for &= op */
*contended = COMPACT_CONTENDED_NONE;
@@ -1518,15 +1635,17 @@
/* Compact each zone in the list */
for_each_zone_zonelist_nodemask(zone, z, ac->zonelist, ac->high_zoneidx,
ac->nodemask) {
- int status;
+ enum compact_result status;
int zone_contended;
- if (compaction_deferred(zone, order))
+ if (compaction_deferred(zone, order)) {
+ rc = max_t(enum compact_result, COMPACT_DEFERRED, rc);
continue;
+ }
status = compact_zone_order(zone, order, gfp_mask, mode,
&zone_contended, alloc_flags,
- ac->classzone_idx);
+ ac_classzone_idx(ac));
rc = max(status, rc);
/*
* It takes at least one zone that wasn't lock contended
@@ -1536,7 +1655,7 @@
/* If a normal allocation would succeed, stop compacting */
if (zone_watermark_ok(zone, order, low_wmark_pages(zone),
- ac->classzone_idx, alloc_flags)) {
+ ac_classzone_idx(ac), alloc_flags)) {
/*
* We think the allocation will succeed in this zone,
* but it is not certain, hence the false. The caller
@@ -1558,7 +1677,8 @@
goto break_loop;
}
- if (mode != MIGRATE_ASYNC && status == COMPACT_COMPLETE) {
+ if (mode != MIGRATE_ASYNC && (status == COMPACT_COMPLETE ||
+ status == COMPACT_PARTIAL_SKIPPED)) {
/*
* We think that allocation won't succeed in this zone
* so we defer compaction there. If it ends up
@@ -1593,7 +1713,7 @@
* If at least one zone wasn't deferred or skipped, we report if all
* zones that were tried were lock contended.
*/
- if (rc > COMPACT_SKIPPED && all_zones_contended)
+ if (rc > COMPACT_INACTIVE && all_zones_contended)
*contended = COMPACT_CONTENDED_LOCK;
return rc;
@@ -1742,7 +1862,7 @@
struct zone *zone;
enum zone_type classzone_idx = pgdat->kcompactd_classzone_idx;
- for (zoneid = 0; zoneid < classzone_idx; zoneid++) {
+ for (zoneid = 0; zoneid <= classzone_idx; zoneid++) {
zone = &pgdat->node_zones[zoneid];
if (!populated_zone(zone))
@@ -1777,7 +1897,7 @@
cc.classzone_idx);
count_vm_event(KCOMPACTD_WAKE);
- for (zoneid = 0; zoneid < cc.classzone_idx; zoneid++) {
+ for (zoneid = 0; zoneid <= cc.classzone_idx; zoneid++) {
int status;
zone = &pgdat->node_zones[zoneid];
@@ -1805,7 +1925,7 @@
cc.classzone_idx, 0)) {
success = true;
compaction_defer_reset(zone, cc.order, false);
- } else if (status == COMPACT_COMPLETE) {
+ } else if (status == COMPACT_PARTIAL_SKIPPED || status == COMPACT_COMPLETE) {
/*
* We use sync migration mode here, so we defer like
* sync direct compaction does.
diff --git a/mm/filemap.c b/mm/filemap.c
index 182b218..9665b1d 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -114,14 +114,11 @@
struct page *page, void *shadow)
{
struct radix_tree_node *node;
- unsigned long index;
- unsigned int offset;
- unsigned int tag;
- void **slot;
VM_BUG_ON(!PageLocked(page));
- __radix_tree_lookup(&mapping->page_tree, page->index, &node, &slot);
+ node = radix_tree_replace_clear_tags(&mapping->page_tree, page->index,
+ shadow);
if (shadow) {
mapping->nrexceptional++;
@@ -135,23 +132,9 @@
}
mapping->nrpages--;
- if (!node) {
- /* Clear direct pointer tags in root node */
- mapping->page_tree.gfp_mask &= __GFP_BITS_MASK;
- radix_tree_replace_slot(slot, shadow);
+ if (!node)
return;
- }
- /* Clear tree tags for the removed page */
- index = page->index;
- offset = index & RADIX_TREE_MAP_MASK;
- for (tag = 0; tag < RADIX_TREE_MAX_TAGS; tag++) {
- if (test_bit(offset, node->tags[tag]))
- radix_tree_tag_clear(&mapping->page_tree, index, tag);
- }
-
- /* Delete page, swap shadow entry */
- radix_tree_replace_slot(slot, shadow);
workingset_node_pages_dec(node);
if (shadow)
workingset_node_shadows_inc(node);
@@ -213,7 +196,7 @@
* some other bad page check should catch it later.
*/
page_mapcount_reset(page);
- atomic_sub(mapcount, &page->_count);
+ page_ref_sub(page, mapcount);
}
}
@@ -713,8 +696,12 @@
* The page might have been evicted from cache only
* recently, in which case it should be activated like
* any other repeatedly accessed page.
+ * The exception is pages getting rewritten; evicting other
+ * data from the working set, only to cache data that will
+ * get overwritten with something else, is a waste of memory.
*/
- if (shadow && workingset_refault(shadow)) {
+ if (!(gfp_mask & __GFP_WRITE) &&
+ shadow && workingset_refault(shadow)) {
SetPageActive(page);
workingset_activation(page);
} else
@@ -2187,7 +2174,7 @@
if (file->f_ra.mmap_miss > 0)
file->f_ra.mmap_miss--;
addr = address + (page->index - vmf->pgoff) * PAGE_SIZE;
- do_set_pte(vma, addr, page, pte, false, false);
+ do_set_pte(vma, addr, page, pte, false, false, true);
unlock_page(page);
goto next;
unlock:
@@ -2574,7 +2561,7 @@
pgoff_t index, unsigned flags)
{
struct page *page;
- int fgp_flags = FGP_LOCK|FGP_ACCESSED|FGP_WRITE|FGP_CREAT;
+ int fgp_flags = FGP_LOCK|FGP_WRITE|FGP_CREAT;
if (flags & AOP_FLAG_NOFS)
fgp_flags |= FGP_NOFS;
diff --git a/mm/highmem.c b/mm/highmem.c
index 123bcd3..50b4ca6 100644
--- a/mm/highmem.c
+++ b/mm/highmem.c
@@ -112,16 +112,12 @@
unsigned int nr_free_highpages (void)
{
- pg_data_t *pgdat;
+ struct zone *zone;
unsigned int pages = 0;
- for_each_online_pgdat(pgdat) {
- pages += zone_page_state(&pgdat->node_zones[ZONE_HIGHMEM],
- NR_FREE_PAGES);
- if (zone_movable_is_highmem())
- pages += zone_page_state(
- &pgdat->node_zones[ZONE_MOVABLE],
- NR_FREE_PAGES);
+ for_each_populated_zone(zone) {
+ if (is_highmem(zone))
+ pages += zone_page_state(zone, NR_FREE_PAGES);
}
return pages;
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index b49ee12..9ed58530 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -89,6 +89,7 @@
static unsigned int khugepaged_scan_sleep_millisecs __read_mostly = 10000;
/* during fragmentation poll the hugepage allocator once every minute */
static unsigned int khugepaged_alloc_sleep_millisecs __read_mostly = 60000;
+static unsigned long khugepaged_sleep_expire;
static struct task_struct *khugepaged_thread __read_mostly;
static DEFINE_MUTEX(khugepaged_mutex);
static DEFINE_SPINLOCK(khugepaged_mm_lock);
@@ -467,6 +468,7 @@
return -EINVAL;
khugepaged_scan_sleep_millisecs = msecs;
+ khugepaged_sleep_expire = 0;
wake_up_interruptible(&khugepaged_wait);
return count;
@@ -494,6 +496,7 @@
return -EINVAL;
khugepaged_alloc_sleep_millisecs = msecs;
+ khugepaged_sleep_expire = 0;
wake_up_interruptible(&khugepaged_wait);
return count;
@@ -764,10 +767,7 @@
static inline pmd_t mk_huge_pmd(struct page *page, pgprot_t prot)
{
- pmd_t entry;
- entry = mk_pmd(page, prot);
- entry = pmd_mkhuge(entry);
- return entry;
+ return pmd_mkhuge(mk_pmd(page, prot));
}
static inline struct list_head *page_deferred_list(struct page *page)
@@ -1013,6 +1013,7 @@
insert_pfn_pmd(vma, addr, pmd, pfn, pgprot, write);
return VM_FAULT_NOPAGE;
}
+EXPORT_SYMBOL_GPL(vmf_insert_pfn_pmd);
static void touch_pmd(struct vm_area_struct *vma, unsigned long addr,
pmd_t *pmd)
@@ -1698,20 +1699,17 @@
return 1;
}
-bool move_huge_pmd(struct vm_area_struct *vma, struct vm_area_struct *new_vma,
- unsigned long old_addr,
+bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr,
unsigned long new_addr, unsigned long old_end,
pmd_t *old_pmd, pmd_t *new_pmd)
{
spinlock_t *old_ptl, *new_ptl;
pmd_t pmd;
-
struct mm_struct *mm = vma->vm_mm;
if ((old_addr & ~HPAGE_PMD_MASK) ||
(new_addr & ~HPAGE_PMD_MASK) ||
- old_end - old_addr < HPAGE_PMD_SIZE ||
- (new_vma->vm_flags & VM_NOHUGEPAGE))
+ old_end - old_addr < HPAGE_PMD_SIZE)
return false;
/*
@@ -2797,15 +2795,25 @@
put_page(hpage);
}
+static bool khugepaged_should_wakeup(void)
+{
+ return kthread_should_stop() ||
+ time_after_eq(jiffies, khugepaged_sleep_expire);
+}
+
static void khugepaged_wait_work(void)
{
if (khugepaged_has_work()) {
- if (!khugepaged_scan_sleep_millisecs)
+ const unsigned long scan_sleep_jiffies =
+ msecs_to_jiffies(khugepaged_scan_sleep_millisecs);
+
+ if (!scan_sleep_jiffies)
return;
+ khugepaged_sleep_expire = jiffies + scan_sleep_jiffies;
wait_event_freezable_timeout(khugepaged_wait,
- kthread_should_stop(),
- msecs_to_jiffies(khugepaged_scan_sleep_millisecs));
+ khugepaged_should_wakeup(),
+ scan_sleep_jiffies);
return;
}
@@ -3029,8 +3037,10 @@
return;
/*
- * Caller holds the mmap_sem write mode, so a huge pmd cannot
- * materialize from under us.
+ * Caller holds the mmap_sem write mode or the anon_vma lock,
+ * so a huge pmd cannot materialize from under us (khugepaged
+ * holds both the mmap_sem write mode and the anon_vma lock
+ * write mode).
*/
__split_huge_pmd(vma, pmd, address, freeze);
}
@@ -3113,7 +3123,7 @@
VM_BUG_ON_PAGE(page_ref_count(page_tail) != 0, page_tail);
/*
- * tail_page->_count is zero and not changing from under us. But
+ * tail_page->_refcount is zero and not changing from under us. But
* get_page_unless_zero() may be running from under us on the
* tail_page. If we used atomic_set() below instead of atomic_inc(), we
* would then run atomic_set() concurrently with
@@ -3340,7 +3350,7 @@
if (mlocked)
lru_add_drain();
- /* Prevent deferred_split_scan() touching ->_count */
+ /* Prevent deferred_split_scan() touching ->_refcount */
spin_lock_irqsave(&pgdata->split_queue_lock, flags);
count = page_count(head);
mapcount = total_mapcount(head);
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 19d0d08..d26162e 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -51,6 +51,7 @@
static struct hstate * __initdata parsed_hstate;
static unsigned long __initdata default_hstate_max_huge_pages;
static unsigned long __initdata default_hstate_size;
+static bool __initdata parsed_valid_hugepagesz = true;
/*
* Protects updates to hugepage_freelists, hugepage_activelist, nr_huge_pages,
@@ -144,7 +145,8 @@
}
}
- if (spool->min_hpages != -1) { /* minimum size accounting */
+ /* minimum size accounting */
+ if (spool->min_hpages != -1 && spool->rsv_hpages) {
if (delta > spool->rsv_hpages) {
/*
* Asking for more reserves than those already taken on
@@ -182,7 +184,8 @@
if (spool->max_hpages != -1) /* maximum size accounting */
spool->used_hpages -= delta;
- if (spool->min_hpages != -1) { /* minimum size accounting */
+ /* minimum size accounting */
+ if (spool->min_hpages != -1 && spool->used_hpages < spool->min_hpages) {
if (spool->rsv_hpages + delta <= spool->min_hpages)
ret = 0;
else
@@ -624,6 +627,7 @@
{
return vma_hugecache_offset(hstate_vma(vma), vma, address);
}
+EXPORT_SYMBOL_GPL(linear_hugepage_index);
/*
* Return the size of the pages allocated when backing a VMA. In the majority
@@ -937,9 +941,7 @@
*/
static int next_node_allowed(int nid, nodemask_t *nodes_allowed)
{
- nid = next_node(nid, *nodes_allowed);
- if (nid == MAX_NUMNODES)
- nid = first_node(*nodes_allowed);
+ nid = next_node_in(nid, *nodes_allowed);
VM_BUG_ON(nid >= MAX_NUMNODES);
return nid;
@@ -1030,8 +1032,8 @@
return alloc_contig_range(start_pfn, end_pfn, MIGRATE_MOVABLE);
}
-static bool pfn_range_valid_gigantic(unsigned long start_pfn,
- unsigned long nr_pages)
+static bool pfn_range_valid_gigantic(struct zone *z,
+ unsigned long start_pfn, unsigned long nr_pages)
{
unsigned long i, end_pfn = start_pfn + nr_pages;
struct page *page;
@@ -1042,6 +1044,9 @@
page = pfn_to_page(i);
+ if (page_zone(page) != z)
+ return false;
+
if (PageReserved(page))
return false;
@@ -1074,7 +1079,7 @@
pfn = ALIGN(z->zone_start_pfn, nr_pages);
while (zone_spans_last_pfn(z, pfn, nr_pages)) {
- if (pfn_range_valid_gigantic(pfn, nr_pages)) {
+ if (pfn_range_valid_gigantic(z, pfn, nr_pages)) {
/*
* We release the zone lock here because
* alloc_contig_range() will also lock the zone
@@ -2659,6 +2664,11 @@
subsys_initcall(hugetlb_init);
/* Should be called on processing a hugepagesz=... option */
+void __init hugetlb_bad_size(void)
+{
+ parsed_valid_hugepagesz = false;
+}
+
void __init hugetlb_add_hstate(unsigned int order)
{
struct hstate *h;
@@ -2678,8 +2688,8 @@
for (i = 0; i < MAX_NUMNODES; ++i)
INIT_LIST_HEAD(&h->hugepage_freelists[i]);
INIT_LIST_HEAD(&h->hugepage_activelist);
- h->next_nid_to_alloc = first_node(node_states[N_MEMORY]);
- h->next_nid_to_free = first_node(node_states[N_MEMORY]);
+ h->next_nid_to_alloc = first_memory_node;
+ h->next_nid_to_free = first_memory_node;
snprintf(h->name, HSTATE_NAME_LEN, "hugepages-%lukB",
huge_page_size(h)/1024);
@@ -2691,11 +2701,17 @@
unsigned long *mhp;
static unsigned long *last_mhp;
+ if (!parsed_valid_hugepagesz) {
+ pr_warn("hugepages = %s preceded by "
+ "an unsupported hugepagesz, ignoring\n", s);
+ parsed_valid_hugepagesz = true;
+ return 1;
+ }
/*
* !hugetlb_max_hstate means we haven't parsed a hugepagesz= parameter yet,
* so this hugepages= parameter goes to the "default hstate".
*/
- if (!hugetlb_max_hstate)
+ else if (!hugetlb_max_hstate)
mhp = &default_hstate_max_huge_pages;
else
mhp = &parsed_hstate->max_huge_pages;
diff --git a/mm/hugetlb_cgroup.c b/mm/hugetlb_cgroup.c
index d8fb10d..eec1150 100644
--- a/mm/hugetlb_cgroup.c
+++ b/mm/hugetlb_cgroup.c
@@ -67,26 +67,42 @@
return false;
}
+static void hugetlb_cgroup_init(struct hugetlb_cgroup *h_cgroup,
+ struct hugetlb_cgroup *parent_h_cgroup)
+{
+ int idx;
+
+ for (idx = 0; idx < HUGE_MAX_HSTATE; idx++) {
+ struct page_counter *counter = &h_cgroup->hugepage[idx];
+ struct page_counter *parent = NULL;
+ unsigned long limit;
+ int ret;
+
+ if (parent_h_cgroup)
+ parent = &parent_h_cgroup->hugepage[idx];
+ page_counter_init(counter, parent);
+
+ limit = round_down(PAGE_COUNTER_MAX,
+ 1 << huge_page_order(&hstates[idx]));
+ ret = page_counter_limit(counter, limit);
+ VM_BUG_ON(ret);
+ }
+}
+
static struct cgroup_subsys_state *
hugetlb_cgroup_css_alloc(struct cgroup_subsys_state *parent_css)
{
struct hugetlb_cgroup *parent_h_cgroup = hugetlb_cgroup_from_css(parent_css);
struct hugetlb_cgroup *h_cgroup;
- int idx;
h_cgroup = kzalloc(sizeof(*h_cgroup), GFP_KERNEL);
if (!h_cgroup)
return ERR_PTR(-ENOMEM);
- if (parent_h_cgroup) {
- for (idx = 0; idx < HUGE_MAX_HSTATE; idx++)
- page_counter_init(&h_cgroup->hugepage[idx],
- &parent_h_cgroup->hugepage[idx]);
- } else {
+ if (!parent_h_cgroup)
root_h_cgroup = h_cgroup;
- for (idx = 0; idx < HUGE_MAX_HSTATE; idx++)
- page_counter_init(&h_cgroup->hugepage[idx], NULL);
- }
+
+ hugetlb_cgroup_init(h_cgroup, parent_h_cgroup);
return &h_cgroup->css;
}
@@ -285,6 +301,7 @@
return ret;
idx = MEMFILE_IDX(of_cft(of)->private);
+ nr_pages = round_down(nr_pages, 1 << huge_page_order(&hstates[idx]));
switch (MEMFILE_ATTR(of_cft(of)->private)) {
case RES_LIMIT:
diff --git a/mm/internal.h b/mm/internal.h
index b79abb6..a37e5b6 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -58,7 +58,7 @@
}
/*
- * Turn a non-refcounted page (->_count == 0) into refcounted with
+ * Turn a non-refcounted page (->_refcount == 0) into refcounted with
* a count of one.
*/
static inline void set_page_refcounted(struct page *page)
@@ -102,13 +102,14 @@
struct alloc_context {
struct zonelist *zonelist;
nodemask_t *nodemask;
- struct zone *preferred_zone;
- int classzone_idx;
+ struct zoneref *preferred_zoneref;
int migratetype;
enum zone_type high_zoneidx;
bool spread_dirty_pages;
};
+#define ac_classzone_idx(ac) zonelist_zone_idx(ac->preferred_zoneref)
+
/*
* Locate the struct page for both the matching buddy in our
* pair (buddy1) and the combined O(n+1) page they form (page).
@@ -173,9 +174,10 @@
enum migrate_mode mode; /* Async or sync migration mode */
bool ignore_skip_hint; /* Scan blocks even if marked skip */
bool direct_compaction; /* False from kcompactd or /proc/... */
+ bool whole_zone; /* Whole zone has been scanned */
int order; /* order a direct compactor needs */
const gfp_t gfp_mask; /* gfp mask of a direct compactor */
- const int alloc_flags; /* alloc flags of a direct compactor */
+ const unsigned int alloc_flags; /* alloc flags of a direct compactor */
const int classzone_idx; /* zone index of a direct compactor */
struct zone *zone;
int contended; /* Signal need_sched() or lock
@@ -440,7 +442,7 @@
extern u64 hwpoison_filter_memcg;
extern u32 hwpoison_filter_enable;
-extern unsigned long vm_mmap_pgoff(struct file *, unsigned long,
+extern unsigned long __must_check vm_mmap_pgoff(struct file *, unsigned long,
unsigned long, unsigned long,
unsigned long, unsigned long);
diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile
index 131daad..1548749 100644
--- a/mm/kasan/Makefile
+++ b/mm/kasan/Makefile
@@ -8,3 +8,4 @@
CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector)
obj-y := kasan.o report.o kasan_init.o
+obj-$(CONFIG_SLAB) += quarantine.o
diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c
index 38f1dd7..18b6a2b 100644
--- a/mm/kasan/kasan.c
+++ b/mm/kasan/kasan.c
@@ -273,32 +273,48 @@
return memory_is_poisoned_n(addr, size);
}
-
-static __always_inline void check_memory_region(unsigned long addr,
- size_t size, bool write)
+static __always_inline void check_memory_region_inline(unsigned long addr,
+ size_t size, bool write,
+ unsigned long ret_ip)
{
if (unlikely(size == 0))
return;
if (unlikely((void *)addr <
kasan_shadow_to_mem((void *)KASAN_SHADOW_START))) {
- kasan_report(addr, size, write, _RET_IP_);
+ kasan_report(addr, size, write, ret_ip);
return;
}
if (likely(!memory_is_poisoned(addr, size)))
return;
- kasan_report(addr, size, write, _RET_IP_);
+ kasan_report(addr, size, write, ret_ip);
}
-void __asan_loadN(unsigned long addr, size_t size);
-void __asan_storeN(unsigned long addr, size_t size);
+static void check_memory_region(unsigned long addr,
+ size_t size, bool write,
+ unsigned long ret_ip)
+{
+ check_memory_region_inline(addr, size, write, ret_ip);
+}
+
+void kasan_check_read(const void *p, unsigned int size)
+{
+ check_memory_region((unsigned long)p, size, false, _RET_IP_);
+}
+EXPORT_SYMBOL(kasan_check_read);
+
+void kasan_check_write(const void *p, unsigned int size)
+{
+ check_memory_region((unsigned long)p, size, true, _RET_IP_);
+}
+EXPORT_SYMBOL(kasan_check_write);
#undef memset
void *memset(void *addr, int c, size_t len)
{
- __asan_storeN((unsigned long)addr, len);
+ check_memory_region((unsigned long)addr, len, true, _RET_IP_);
return __memset(addr, c, len);
}
@@ -306,8 +322,8 @@
#undef memmove
void *memmove(void *dest, const void *src, size_t len)
{
- __asan_loadN((unsigned long)src, len);
- __asan_storeN((unsigned long)dest, len);
+ check_memory_region((unsigned long)src, len, false, _RET_IP_);
+ check_memory_region((unsigned long)dest, len, true, _RET_IP_);
return __memmove(dest, src, len);
}
@@ -315,8 +331,8 @@
#undef memcpy
void *memcpy(void *dest, const void *src, size_t len)
{
- __asan_loadN((unsigned long)src, len);
- __asan_storeN((unsigned long)dest, len);
+ check_memory_region((unsigned long)src, len, false, _RET_IP_);
+ check_memory_region((unsigned long)dest, len, true, _RET_IP_);
return __memcpy(dest, src, len);
}
@@ -388,6 +404,16 @@
}
#endif
+void kasan_cache_shrink(struct kmem_cache *cache)
+{
+ quarantine_remove_cache(cache);
+}
+
+void kasan_cache_destroy(struct kmem_cache *cache)
+{
+ quarantine_remove_cache(cache);
+}
+
void kasan_poison_slab(struct page *page)
{
kasan_poison_shadow(page_address(page),
@@ -482,7 +508,7 @@
kasan_kmalloc(cache, object, cache->object_size, flags);
}
-void kasan_slab_free(struct kmem_cache *cache, void *object)
+void kasan_poison_slab_free(struct kmem_cache *cache, void *object)
{
unsigned long size = cache->object_size;
unsigned long rounded_up_size = round_up(size, KASAN_SHADOW_SCALE_SIZE);
@@ -491,18 +517,43 @@
if (unlikely(cache->flags & SLAB_DESTROY_BY_RCU))
return;
+ kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE);
+}
+
+bool kasan_slab_free(struct kmem_cache *cache, void *object)
+{
#ifdef CONFIG_SLAB
- if (cache->flags & SLAB_KASAN) {
- struct kasan_free_meta *free_info =
- get_free_info(cache, object);
+ /* RCU slabs could be legally used after free within the RCU period */
+ if (unlikely(cache->flags & SLAB_DESTROY_BY_RCU))
+ return false;
+
+ if (likely(cache->flags & SLAB_KASAN)) {
struct kasan_alloc_meta *alloc_info =
get_alloc_info(cache, object);
- alloc_info->state = KASAN_STATE_FREE;
- set_track(&free_info->track, GFP_NOWAIT);
- }
-#endif
+ struct kasan_free_meta *free_info =
+ get_free_info(cache, object);
- kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE);
+ switch (alloc_info->state) {
+ case KASAN_STATE_ALLOC:
+ alloc_info->state = KASAN_STATE_QUARANTINE;
+ quarantine_put(free_info, cache);
+ set_track(&free_info->track, GFP_NOWAIT);
+ kasan_poison_slab_free(cache, object);
+ return true;
+ case KASAN_STATE_QUARANTINE:
+ case KASAN_STATE_FREE:
+ pr_err("Double free");
+ dump_stack();
+ break;
+ default:
+ break;
+ }
+ }
+ return false;
+#else
+ kasan_poison_slab_free(cache, object);
+ return false;
+#endif
}
void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size,
@@ -511,6 +562,9 @@
unsigned long redzone_start;
unsigned long redzone_end;
+ if (flags & __GFP_RECLAIM)
+ quarantine_reduce();
+
if (unlikely(object == NULL))
return;
@@ -541,6 +595,9 @@
unsigned long redzone_start;
unsigned long redzone_end;
+ if (flags & __GFP_RECLAIM)
+ quarantine_reduce();
+
if (unlikely(ptr == NULL))
return;
@@ -649,22 +706,22 @@
}
EXPORT_SYMBOL(__asan_unregister_globals);
-#define DEFINE_ASAN_LOAD_STORE(size) \
- void __asan_load##size(unsigned long addr) \
- { \
- check_memory_region(addr, size, false); \
- } \
- EXPORT_SYMBOL(__asan_load##size); \
- __alias(__asan_load##size) \
- void __asan_load##size##_noabort(unsigned long); \
- EXPORT_SYMBOL(__asan_load##size##_noabort); \
- void __asan_store##size(unsigned long addr) \
- { \
- check_memory_region(addr, size, true); \
- } \
- EXPORT_SYMBOL(__asan_store##size); \
- __alias(__asan_store##size) \
- void __asan_store##size##_noabort(unsigned long); \
+#define DEFINE_ASAN_LOAD_STORE(size) \
+ void __asan_load##size(unsigned long addr) \
+ { \
+ check_memory_region_inline(addr, size, false, _RET_IP_);\
+ } \
+ EXPORT_SYMBOL(__asan_load##size); \
+ __alias(__asan_load##size) \
+ void __asan_load##size##_noabort(unsigned long); \
+ EXPORT_SYMBOL(__asan_load##size##_noabort); \
+ void __asan_store##size(unsigned long addr) \
+ { \
+ check_memory_region_inline(addr, size, true, _RET_IP_); \
+ } \
+ EXPORT_SYMBOL(__asan_store##size); \
+ __alias(__asan_store##size) \
+ void __asan_store##size##_noabort(unsigned long); \
EXPORT_SYMBOL(__asan_store##size##_noabort)
DEFINE_ASAN_LOAD_STORE(1);
@@ -675,7 +732,7 @@
void __asan_loadN(unsigned long addr, size_t size)
{
- check_memory_region(addr, size, false);
+ check_memory_region(addr, size, false, _RET_IP_);
}
EXPORT_SYMBOL(__asan_loadN);
@@ -685,7 +742,7 @@
void __asan_storeN(unsigned long addr, size_t size)
{
- check_memory_region(addr, size, true);
+ check_memory_region(addr, size, true, _RET_IP_);
}
EXPORT_SYMBOL(__asan_storeN);
diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index 30a2f0b..7f7ac51 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -62,6 +62,7 @@
enum kasan_state {
KASAN_STATE_INIT,
KASAN_STATE_ALLOC,
+ KASAN_STATE_QUARANTINE,
KASAN_STATE_FREE
};
@@ -79,9 +80,14 @@
u32 reserved;
};
+struct qlist_node {
+ struct qlist_node *next;
+};
struct kasan_free_meta {
- /* Allocator freelist pointer, unused by KASAN. */
- void **freelist;
+ /* This field is used while the object is in the quarantine.
+ * Otherwise it might be used for the allocator freelist.
+ */
+ struct qlist_node quarantine_link;
struct kasan_track track;
};
@@ -105,4 +111,15 @@
void kasan_report(unsigned long addr, size_t size,
bool is_write, unsigned long ip);
+#ifdef CONFIG_SLAB
+void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache);
+void quarantine_reduce(void);
+void quarantine_remove_cache(struct kmem_cache *cache);
+#else
+static inline void quarantine_put(struct kasan_free_meta *info,
+ struct kmem_cache *cache) { }
+static inline void quarantine_reduce(void) { }
+static inline void quarantine_remove_cache(struct kmem_cache *cache) { }
+#endif
+
#endif
diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c
new file mode 100644
index 0000000..4973505
--- /dev/null
+++ b/mm/kasan/quarantine.c
@@ -0,0 +1,291 @@
+/*
+ * KASAN quarantine.
+ *
+ * Author: Alexander Potapenko <glider@google.com>
+ * Copyright (C) 2016 Google, Inc.
+ *
+ * Based on code by Dmitry Chernenkov.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ */
+
+#include <linux/gfp.h>
+#include <linux/hash.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/percpu.h>
+#include <linux/printk.h>
+#include <linux/shrinker.h>
+#include <linux/slab.h>
+#include <linux/string.h>
+#include <linux/types.h>
+
+#include "../slab.h"
+#include "kasan.h"
+
+/* Data structure and operations for quarantine queues. */
+
+/*
+ * Each queue is a signle-linked list, which also stores the total size of
+ * objects inside of it.
+ */
+struct qlist_head {
+ struct qlist_node *head;
+ struct qlist_node *tail;
+ size_t bytes;
+};
+
+#define QLIST_INIT { NULL, NULL, 0 }
+
+static bool qlist_empty(struct qlist_head *q)
+{
+ return !q->head;
+}
+
+static void qlist_init(struct qlist_head *q)
+{
+ q->head = q->tail = NULL;
+ q->bytes = 0;
+}
+
+static void qlist_put(struct qlist_head *q, struct qlist_node *qlink,
+ size_t size)
+{
+ if (unlikely(qlist_empty(q)))
+ q->head = qlink;
+ else
+ q->tail->next = qlink;
+ q->tail = qlink;
+ qlink->next = NULL;
+ q->bytes += size;
+}
+
+static void qlist_move_all(struct qlist_head *from, struct qlist_head *to)
+{
+ if (unlikely(qlist_empty(from)))
+ return;
+
+ if (qlist_empty(to)) {
+ *to = *from;
+ qlist_init(from);
+ return;
+ }
+
+ to->tail->next = from->head;
+ to->tail = from->tail;
+ to->bytes += from->bytes;
+
+ qlist_init(from);
+}
+
+static void qlist_move(struct qlist_head *from, struct qlist_node *last,
+ struct qlist_head *to, size_t size)
+{
+ if (unlikely(last == from->tail)) {
+ qlist_move_all(from, to);
+ return;
+ }
+ if (qlist_empty(to))
+ to->head = from->head;
+ else
+ to->tail->next = from->head;
+ to->tail = last;
+ from->head = last->next;
+ last->next = NULL;
+ from->bytes -= size;
+ to->bytes += size;
+}
+
+
+/*
+ * The object quarantine consists of per-cpu queues and a global queue,
+ * guarded by quarantine_lock.
+ */
+static DEFINE_PER_CPU(struct qlist_head, cpu_quarantine);
+
+static struct qlist_head global_quarantine;
+static DEFINE_SPINLOCK(quarantine_lock);
+
+/* Maximum size of the global queue. */
+static unsigned long quarantine_size;
+
+/*
+ * The fraction of physical memory the quarantine is allowed to occupy.
+ * Quarantine doesn't support memory shrinker with SLAB allocator, so we keep
+ * the ratio low to avoid OOM.
+ */
+#define QUARANTINE_FRACTION 32
+
+#define QUARANTINE_LOW_SIZE (READ_ONCE(quarantine_size) * 3 / 4)
+#define QUARANTINE_PERCPU_SIZE (1 << 20)
+
+static struct kmem_cache *qlink_to_cache(struct qlist_node *qlink)
+{
+ return virt_to_head_page(qlink)->slab_cache;
+}
+
+static void *qlink_to_object(struct qlist_node *qlink, struct kmem_cache *cache)
+{
+ struct kasan_free_meta *free_info =
+ container_of(qlink, struct kasan_free_meta,
+ quarantine_link);
+
+ return ((void *)free_info) - cache->kasan_info.free_meta_offset;
+}
+
+static void qlink_free(struct qlist_node *qlink, struct kmem_cache *cache)
+{
+ void *object = qlink_to_object(qlink, cache);
+ struct kasan_alloc_meta *alloc_info = get_alloc_info(cache, object);
+ unsigned long flags;
+
+ local_irq_save(flags);
+ alloc_info->state = KASAN_STATE_FREE;
+ ___cache_free(cache, object, _THIS_IP_);
+ local_irq_restore(flags);
+}
+
+static void qlist_free_all(struct qlist_head *q, struct kmem_cache *cache)
+{
+ struct qlist_node *qlink;
+
+ if (unlikely(qlist_empty(q)))
+ return;
+
+ qlink = q->head;
+ while (qlink) {
+ struct kmem_cache *obj_cache =
+ cache ? cache : qlink_to_cache(qlink);
+ struct qlist_node *next = qlink->next;
+
+ qlink_free(qlink, obj_cache);
+ qlink = next;
+ }
+ qlist_init(q);
+}
+
+void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache)
+{
+ unsigned long flags;
+ struct qlist_head *q;
+ struct qlist_head temp = QLIST_INIT;
+
+ local_irq_save(flags);
+
+ q = this_cpu_ptr(&cpu_quarantine);
+ qlist_put(q, &info->quarantine_link, cache->size);
+ if (unlikely(q->bytes > QUARANTINE_PERCPU_SIZE))
+ qlist_move_all(q, &temp);
+
+ local_irq_restore(flags);
+
+ if (unlikely(!qlist_empty(&temp))) {
+ spin_lock_irqsave(&quarantine_lock, flags);
+ qlist_move_all(&temp, &global_quarantine);
+ spin_unlock_irqrestore(&quarantine_lock, flags);
+ }
+}
+
+void quarantine_reduce(void)
+{
+ size_t new_quarantine_size;
+ unsigned long flags;
+ struct qlist_head to_free = QLIST_INIT;
+ size_t size_to_free = 0;
+ struct qlist_node *last;
+
+ if (likely(READ_ONCE(global_quarantine.bytes) <=
+ READ_ONCE(quarantine_size)))
+ return;
+
+ spin_lock_irqsave(&quarantine_lock, flags);
+
+ /*
+ * Update quarantine size in case of hotplug. Allocate a fraction of
+ * the installed memory to quarantine minus per-cpu queue limits.
+ */
+ new_quarantine_size = (READ_ONCE(totalram_pages) << PAGE_SHIFT) /
+ QUARANTINE_FRACTION;
+ new_quarantine_size -= QUARANTINE_PERCPU_SIZE * num_online_cpus();
+ WRITE_ONCE(quarantine_size, new_quarantine_size);
+
+ last = global_quarantine.head;
+ while (last) {
+ struct kmem_cache *cache = qlink_to_cache(last);
+
+ size_to_free += cache->size;
+ if (!last->next || size_to_free >
+ global_quarantine.bytes - QUARANTINE_LOW_SIZE)
+ break;
+ last = last->next;
+ }
+ qlist_move(&global_quarantine, last, &to_free, size_to_free);
+
+ spin_unlock_irqrestore(&quarantine_lock, flags);
+
+ qlist_free_all(&to_free, NULL);
+}
+
+static void qlist_move_cache(struct qlist_head *from,
+ struct qlist_head *to,
+ struct kmem_cache *cache)
+{
+ struct qlist_node *prev = NULL, *curr;
+
+ if (unlikely(qlist_empty(from)))
+ return;
+
+ curr = from->head;
+ while (curr) {
+ struct qlist_node *qlink = curr;
+ struct kmem_cache *obj_cache = qlink_to_cache(qlink);
+
+ if (obj_cache == cache) {
+ if (unlikely(from->head == qlink)) {
+ from->head = curr->next;
+ prev = curr;
+ } else
+ prev->next = curr->next;
+ if (unlikely(from->tail == qlink))
+ from->tail = curr->next;
+ from->bytes -= cache->size;
+ qlist_put(to, qlink, cache->size);
+ } else {
+ prev = curr;
+ }
+ curr = curr->next;
+ }
+}
+
+static void per_cpu_remove_cache(void *arg)
+{
+ struct kmem_cache *cache = arg;
+ struct qlist_head to_free = QLIST_INIT;
+ struct qlist_head *q;
+
+ q = this_cpu_ptr(&cpu_quarantine);
+ qlist_move_cache(q, &to_free, cache);
+ qlist_free_all(&to_free, cache);
+}
+
+void quarantine_remove_cache(struct kmem_cache *cache)
+{
+ unsigned long flags;
+ struct qlist_head to_free = QLIST_INIT;
+
+ on_each_cpu(per_cpu_remove_cache, cache, 1);
+
+ spin_lock_irqsave(&quarantine_lock, flags);
+ qlist_move_cache(&global_quarantine, &to_free, cache);
+ spin_unlock_irqrestore(&quarantine_lock, flags);
+
+ qlist_free_all(&to_free, cache);
+}
diff --git a/mm/kasan/report.c b/mm/kasan/report.c
index 60869a5a..b3c122d 100644
--- a/mm/kasan/report.c
+++ b/mm/kasan/report.c
@@ -151,6 +151,7 @@
print_track(&alloc_info->track);
break;
case KASAN_STATE_FREE:
+ case KASAN_STATE_QUARANTINE:
pr_err("Object freed, allocated with size %u bytes\n",
alloc_info->alloc_size);
free_info = get_free_info(cache, object);
diff --git a/mm/maccess.c b/mm/maccess.c
index d159b1c..78f9274 100644
--- a/mm/maccess.c
+++ b/mm/maccess.c
@@ -96,8 +96,7 @@
pagefault_disable();
do {
- ret = __copy_from_user_inatomic(dst++,
- (const void __user __force *)src++, 1);
+ ret = __get_user(*dst++, (const char __user __force *)src++);
} while (dst[-1] && ret == 0 && src - unsafe_addr < count);
dst[-1] = '\0';
diff --git a/mm/madvise.c b/mm/madvise.c
index 07427d3..93fb63e 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -707,10 +707,12 @@
return error;
write = madvise_need_mmap_write(behavior);
- if (write)
- down_write(¤t->mm->mmap_sem);
- else
+ if (write) {
+ if (down_write_killable(¤t->mm->mmap_sem))
+ return -EINTR;
+ } else {
down_read(¤t->mm->mmap_sem);
+ }
/*
* If the interval [start,end) covers some unmapped address
diff --git a/mm/memblock.c b/mm/memblock.c
index b570ddd..ac12489 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -606,22 +606,14 @@
return memblock_add_range(&memblock.memory, base, size, nid, 0);
}
-static int __init_memblock memblock_add_region(phys_addr_t base,
- phys_addr_t size,
- int nid,
- unsigned long flags)
+int __init_memblock memblock_add(phys_addr_t base, phys_addr_t size)
{
memblock_dbg("memblock_add: [%#016llx-%#016llx] flags %#02lx %pF\n",
(unsigned long long)base,
(unsigned long long)base + size - 1,
- flags, (void *)_RET_IP_);
+ 0UL, (void *)_RET_IP_);
- return memblock_add_range(&memblock.memory, base, size, nid, flags);
-}
-
-int __init_memblock memblock_add(phys_addr_t base, phys_addr_t size)
-{
- return memblock_add_region(base, size, MAX_NUMNODES, 0);
+ return memblock_add_range(&memblock.memory, base, size, MAX_NUMNODES, 0);
}
/**
@@ -732,22 +724,14 @@
return memblock_remove_range(&memblock.reserved, base, size);
}
-static int __init_memblock memblock_reserve_region(phys_addr_t base,
- phys_addr_t size,
- int nid,
- unsigned long flags)
+int __init_memblock memblock_reserve(phys_addr_t base, phys_addr_t size)
{
memblock_dbg("memblock_reserve: [%#016llx-%#016llx] flags %#02lx %pF\n",
(unsigned long long)base,
(unsigned long long)base + size - 1,
- flags, (void *)_RET_IP_);
+ 0UL, (void *)_RET_IP_);
- return memblock_add_range(&memblock.reserved, base, size, nid, flags);
-}
-
-int __init_memblock memblock_reserve(phys_addr_t base, phys_addr_t size)
-{
- return memblock_reserve_region(base, size, MAX_NUMNODES, 0);
+ return memblock_add_range(&memblock.reserved, base, size, MAX_NUMNODES, 0);
}
/**
@@ -840,7 +824,7 @@
{
struct memblock_type *type = &memblock.reserved;
- if (*idx >= 0 && *idx < type->cnt) {
+ if (*idx < type->cnt) {
struct memblock_region *r = &type->regions[*idx];
phys_addr_t base = r->base;
phys_addr_t size = r->size;
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index fe787f5..cf428d7 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -1023,22 +1023,40 @@
* @lru: index of lru list the page is sitting on
* @nr_pages: positive when adding or negative when removing
*
- * This function must be called when a page is added to or removed from an
- * lru list.
+ * This function must be called under lru_lock, just before a page is added
+ * to or just after a page is removed from an lru list (that ordering being
+ * so as to allow it to check that lru_size 0 is consistent with list_empty).
*/
void mem_cgroup_update_lru_size(struct lruvec *lruvec, enum lru_list lru,
int nr_pages)
{
struct mem_cgroup_per_zone *mz;
unsigned long *lru_size;
+ long size;
+ bool empty;
+
+ __update_lru_size(lruvec, lru, nr_pages);
if (mem_cgroup_disabled())
return;
mz = container_of(lruvec, struct mem_cgroup_per_zone, lruvec);
lru_size = mz->lru_size + lru;
- *lru_size += nr_pages;
- VM_BUG_ON((long)(*lru_size) < 0);
+ empty = list_empty(lruvec->lists + lru);
+
+ if (nr_pages < 0)
+ *lru_size += nr_pages;
+
+ size = *lru_size;
+ if (WARN_ONCE(size < 0 || empty != !size,
+ "%s(%p, %d, %d): lru_size %ld but %sempty\n",
+ __func__, lruvec, lru, nr_pages, size, empty ? "" : "not ")) {
+ VM_BUG_ON(1);
+ *lru_size = 0;
+ }
+
+ if (nr_pages > 0)
+ *lru_size += nr_pages;
}
bool task_in_mem_cgroup(struct task_struct *task, struct mem_cgroup *memcg)
@@ -1257,6 +1275,7 @@
*/
if (fatal_signal_pending(current) || task_will_free_mem(current)) {
mark_oom_victim(current);
+ try_oom_reaper(current);
goto unlock;
}
@@ -1389,14 +1408,11 @@
mem_cgroup_may_update_nodemask(memcg);
node = memcg->last_scanned_node;
- node = next_node(node, memcg->scan_nodes);
- if (node == MAX_NUMNODES)
- node = first_node(memcg->scan_nodes);
+ node = next_node_in(node, memcg->scan_nodes);
/*
- * We call this when we hit limit, not when pages are added to LRU.
- * No LRU may hold pages because all pages are UNEVICTABLE or
- * memcg is too small and all pages are not on LRU. In that case,
- * we use curret node.
+ * mem_cgroup_may_update_nodemask might have seen no reclaimmable pages
+ * last time it really checked all the LRUs due to rate limiting.
+ * Fallback to the current node in that case for simplicity.
*/
if (unlikely(node == MAX_NUMNODES))
node = numa_node_id();
@@ -1588,7 +1604,7 @@
static void mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int order)
{
- if (!current->memcg_may_oom)
+ if (!current->memcg_may_oom || current->memcg_in_oom)
return;
/*
* We are in the middle of the charge context here, so we
@@ -2636,8 +2652,7 @@
}
/*
- * Reclaims as many pages from the given memcg as possible and moves
- * the rest to the parent.
+ * Reclaims as many pages from the given memcg as possible.
*
* Caller is responsible for holding css reference for memcg.
*/
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index ca5acee..2fcca6b 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -184,8 +184,8 @@
struct siginfo si;
int ret;
- pr_err("MCE %#lx: Killing %s:%d due to hardware memory corruption\n",
- pfn, t->comm, t->pid);
+ pr_err("Memory failure: %#lx: Killing %s:%d due to hardware memory corruption\n",
+ pfn, t->comm, t->pid);
si.si_signo = SIGBUS;
si.si_errno = 0;
si.si_addr = (void *)addr;
@@ -208,7 +208,7 @@
ret = send_sig_info(SIGBUS, &si, t); /* synchronous? */
}
if (ret < 0)
- pr_info("MCE: Error sending signal to %s:%d: %d\n",
+ pr_info("Memory failure: Error sending signal to %s:%d: %d\n",
t->comm, t->pid, ret);
return ret;
}
@@ -289,7 +289,7 @@
} else {
tk = kmalloc(sizeof(struct to_kill), GFP_ATOMIC);
if (!tk) {
- pr_err("MCE: Out of memory while machine check handling\n");
+ pr_err("Memory failure: Out of memory while machine check handling\n");
return;
}
}
@@ -303,7 +303,7 @@
* a SIGKILL because the error is not contained anymore.
*/
if (tk->addr == -EFAULT) {
- pr_info("MCE: Unable to find user space address %lx in %s\n",
+ pr_info("Memory failure: Unable to find user space address %lx in %s\n",
page_to_pfn(p), tsk->comm);
tk->addr_valid = 0;
}
@@ -334,7 +334,7 @@
* signal and then access the memory. Just kill it.
*/
if (fail || tk->addr_valid == 0) {
- pr_err("MCE %#lx: forcibly killing %s:%d because of failure to unmap corrupted page\n",
+ pr_err("Memory failure: %#lx: forcibly killing %s:%d because of failure to unmap corrupted page\n",
pfn, tk->tsk->comm, tk->tsk->pid);
force_sig(SIGKILL, tk->tsk);
}
@@ -347,7 +347,7 @@
*/
else if (kill_proc(tk->tsk, tk->addr, trapno,
pfn, page, flags) < 0)
- pr_err("MCE %#lx: Cannot send advisory machine check signal to %s:%d\n",
+ pr_err("Memory failure: %#lx: Cannot send advisory machine check signal to %s:%d\n",
pfn, tk->tsk->comm, tk->tsk->pid);
}
put_task_struct(tk->tsk);
@@ -559,7 +559,7 @@
*/
static int me_unknown(struct page *p, unsigned long pfn)
{
- pr_err("MCE %#lx: Unknown page state\n", pfn);
+ pr_err("Memory failure: %#lx: Unknown page state\n", pfn);
return MF_FAILED;
}
@@ -604,11 +604,12 @@
if (mapping->a_ops->error_remove_page) {
err = mapping->a_ops->error_remove_page(mapping, p);
if (err != 0) {
- pr_info("MCE %#lx: Failed to punch page: %d\n",
+ pr_info("Memory failure: %#lx: Failed to punch page: %d\n",
pfn, err);
} else if (page_has_private(p) &&
!try_to_release_page(p, GFP_NOIO)) {
- pr_info("MCE %#lx: failed to release buffers\n", pfn);
+ pr_info("Memory failure: %#lx: failed to release buffers\n",
+ pfn);
} else {
ret = MF_RECOVERED;
}
@@ -620,7 +621,8 @@
if (invalidate_inode_page(p))
ret = MF_RECOVERED;
else
- pr_info("MCE %#lx: Failed to invalidate\n", pfn);
+ pr_info("Memory failure: %#lx: Failed to invalidate\n",
+ pfn);
}
return ret;
}
@@ -833,7 +835,7 @@
{
trace_memory_failure_event(pfn, type, result);
- pr_err("MCE %#lx: recovery action for %s: %s\n",
+ pr_err("Memory failure: %#lx: recovery action for %s: %s\n",
pfn, action_page_types[type], action_name[result]);
}
@@ -849,7 +851,7 @@
if (ps->action == me_swapcache_dirty && result == MF_DELAYED)
count--;
if (count != 0) {
- pr_err("MCE %#lx: %s still referenced by %d users\n",
+ pr_err("Memory failure: %#lx: %s still referenced by %d users\n",
pfn, action_page_types[ps->type], count);
result = MF_FAILED;
}
@@ -882,7 +884,7 @@
* tries to touch the "partially handled" page.
*/
if (!PageAnon(head)) {
- pr_err("MCE: %#lx: non anonymous thp\n",
+ pr_err("Memory failure: %#lx: non anonymous thp\n",
page_to_pfn(page));
return 0;
}
@@ -892,7 +894,8 @@
if (head == compound_head(page))
return 1;
- pr_info("MCE: %#lx cannot catch tail\n", page_to_pfn(page));
+ pr_info("Memory failure: %#lx cannot catch tail\n",
+ page_to_pfn(page));
put_page(head);
}
@@ -931,12 +934,13 @@
return SWAP_SUCCESS;
if (PageKsm(p)) {
- pr_err("MCE %#lx: can't handle KSM pages.\n", pfn);
+ pr_err("Memory failure: %#lx: can't handle KSM pages.\n", pfn);
return SWAP_FAIL;
}
if (PageSwapCache(p)) {
- pr_err("MCE %#lx: keeping poisoned page in swap cache\n", pfn);
+ pr_err("Memory failure: %#lx: keeping poisoned page in swap cache\n",
+ pfn);
ttu |= TTU_IGNORE_HWPOISON;
}
@@ -954,7 +958,7 @@
} else {
kill = 0;
ttu |= TTU_IGNORE_HWPOISON;
- pr_info("MCE %#lx: corrupted page was clean: dropped without side effects\n",
+ pr_info("Memory failure: %#lx: corrupted page was clean: dropped without side effects\n",
pfn);
}
}
@@ -972,7 +976,7 @@
ret = try_to_unmap(hpage, ttu);
if (ret != SWAP_SUCCESS)
- pr_err("MCE %#lx: failed to unmap page (mapcount=%d)\n",
+ pr_err("Memory failure: %#lx: failed to unmap page (mapcount=%d)\n",
pfn, page_mapcount(hpage));
/*
@@ -1040,14 +1044,16 @@
panic("Memory failure from trap %d on page %lx", trapno, pfn);
if (!pfn_valid(pfn)) {
- pr_err("MCE %#lx: memory outside kernel control\n", pfn);
+ pr_err("Memory failure: %#lx: memory outside kernel control\n",
+ pfn);
return -ENXIO;
}
p = pfn_to_page(pfn);
orig_head = hpage = compound_head(p);
if (TestSetPageHWPoison(p)) {
- pr_err("MCE %#lx: already hardware poisoned\n", pfn);
+ pr_err("Memory failure: %#lx: already hardware poisoned\n",
+ pfn);
return 0;
}
@@ -1112,9 +1118,11 @@
if (!PageAnon(hpage) || unlikely(split_huge_page(hpage))) {
unlock_page(hpage);
if (!PageAnon(hpage))
- pr_err("MCE: %#lx: non anonymous thp\n", pfn);
+ pr_err("Memory failure: %#lx: non anonymous thp\n",
+ pfn);
else
- pr_err("MCE: %#lx: thp split failed\n", pfn);
+ pr_err("Memory failure: %#lx: thp split failed\n",
+ pfn);
if (TestClearPageHWPoison(p))
num_poisoned_pages_sub(nr_pages);
put_hwpoison_page(p);
@@ -1178,7 +1186,7 @@
* unpoison always clear PG_hwpoison inside page lock
*/
if (!PageHWPoison(p)) {
- pr_err("MCE %#lx: just unpoisoned\n", pfn);
+ pr_err("Memory failure: %#lx: just unpoisoned\n", pfn);
num_poisoned_pages_sub(nr_pages);
unlock_page(hpage);
put_hwpoison_page(hpage);
@@ -1395,25 +1403,25 @@
page = compound_head(p);
if (!PageHWPoison(p)) {
- unpoison_pr_info("MCE: Page was already unpoisoned %#lx\n",
+ unpoison_pr_info("Unpoison: Page was already unpoisoned %#lx\n",
pfn, &unpoison_rs);
return 0;
}
if (page_count(page) > 1) {
- unpoison_pr_info("MCE: Someone grabs the hwpoison page %#lx\n",
+ unpoison_pr_info("Unpoison: Someone grabs the hwpoison page %#lx\n",
pfn, &unpoison_rs);
return 0;
}
if (page_mapped(page)) {
- unpoison_pr_info("MCE: Someone maps the hwpoison page %#lx\n",
+ unpoison_pr_info("Unpoison: Someone maps the hwpoison page %#lx\n",
pfn, &unpoison_rs);
return 0;
}
if (page_mapping(page)) {
- unpoison_pr_info("MCE: the hwpoison page has non-NULL mapping %#lx\n",
+ unpoison_pr_info("Unpoison: the hwpoison page has non-NULL mapping %#lx\n",
pfn, &unpoison_rs);
return 0;
}
@@ -1424,7 +1432,7 @@
* In such case, we yield to memory_failure() and make unpoison fail.
*/
if (!PageHuge(page) && PageTransHuge(page)) {
- unpoison_pr_info("MCE: Memory failure is now running on %#lx\n",
+ unpoison_pr_info("Unpoison: Memory failure is now running on %#lx\n",
pfn, &unpoison_rs);
return 0;
}
@@ -1439,13 +1447,13 @@
* to the end.
*/
if (PageHuge(page)) {
- unpoison_pr_info("MCE: Memory failure is now running on free hugepage %#lx\n",
+ unpoison_pr_info("Unpoison: Memory failure is now running on free hugepage %#lx\n",
pfn, &unpoison_rs);
return 0;
}
if (TestClearPageHWPoison(p))
num_poisoned_pages_dec();
- unpoison_pr_info("MCE: Software-unpoisoned free page %#lx\n",
+ unpoison_pr_info("Unpoison: Software-unpoisoned free page %#lx\n",
pfn, &unpoison_rs);
return 0;
}
@@ -1458,7 +1466,7 @@
* the free buddy page pool.
*/
if (TestClearPageHWPoison(page)) {
- unpoison_pr_info("MCE: Software-unpoisoned page %#lx\n",
+ unpoison_pr_info("Unpoison: Software-unpoisoned page %#lx\n",
pfn, &unpoison_rs);
num_poisoned_pages_sub(nr_pages);
freeit = 1;
diff --git a/mm/memory.c b/mm/memory.c
index 07493e3..a1b93d9 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -1744,6 +1744,7 @@
unsigned long next;
unsigned long end = addr + PAGE_ALIGN(size);
struct mm_struct *mm = vma->vm_mm;
+ unsigned long remap_pfn = pfn;
int err;
/*
@@ -1770,7 +1771,7 @@
vma->vm_pgoff = pfn;
}
- err = track_pfn_remap(vma, &prot, pfn, addr, PAGE_ALIGN(size));
+ err = track_pfn_remap(vma, &prot, remap_pfn, addr, PAGE_ALIGN(size));
if (err)
return -EINVAL;
@@ -1789,7 +1790,7 @@
} while (pgd++, addr = next, addr != end);
if (err)
- untrack_pfn(vma, pfn, PAGE_ALIGN(size));
+ untrack_pfn(vma, remap_pfn, PAGE_ALIGN(size));
return err;
}
@@ -2875,7 +2876,7 @@
* vm_ops->map_pages.
*/
void do_set_pte(struct vm_area_struct *vma, unsigned long address,
- struct page *page, pte_t *pte, bool write, bool anon)
+ struct page *page, pte_t *pte, bool write, bool anon, bool old)
{
pte_t entry;
@@ -2883,6 +2884,8 @@
entry = mk_pte(page, vma->vm_page_prot);
if (write)
entry = maybe_mkwrite(pte_mkdirty(entry), vma);
+ if (old)
+ entry = pte_mkold(entry);
if (anon) {
inc_mm_counter_fast(vma->vm_mm, MM_ANONPAGES);
page_add_new_anon_rmap(page, vma, address, false);
@@ -2896,8 +2899,16 @@
update_mmu_cache(vma, address, pte);
}
+/*
+ * If architecture emulates "accessed" or "young" bit without HW support,
+ * there is no much gain with fault_around.
+ */
static unsigned long fault_around_bytes __read_mostly =
+#ifndef __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS
+ PAGE_SIZE;
+#else
rounddown_pow_of_two(65536);
+#endif
#ifdef CONFIG_DEBUG_FS
static int fault_around_bytes_get(void *data, u64 *val)
@@ -3020,9 +3031,20 @@
*/
if (vma->vm_ops->map_pages && fault_around_bytes >> PAGE_SHIFT > 1) {
pte = pte_offset_map_lock(mm, pmd, address, &ptl);
- do_fault_around(vma, address, pte, pgoff, flags);
if (!pte_same(*pte, orig_pte))
goto unlock_out;
+ do_fault_around(vma, address, pte, pgoff, flags);
+ /* Check if the fault is handled by faultaround */
+ if (!pte_same(*pte, orig_pte)) {
+ /*
+ * Faultaround produce old pte, but the pte we've
+ * handler fault for should be young.
+ */
+ pte_t entry = pte_mkyoung(*pte);
+ if (ptep_set_access_flags(vma, address, pte, entry, 0))
+ update_mmu_cache(vma, address, pte);
+ goto unlock_out;
+ }
pte_unmap_unlock(pte, ptl);
}
@@ -3037,7 +3059,7 @@
put_page(fault_page);
return ret;
}
- do_set_pte(vma, address, fault_page, pte, false, false);
+ do_set_pte(vma, address, fault_page, pte, false, false, false);
unlock_page(fault_page);
unlock_out:
pte_unmap_unlock(pte, ptl);
@@ -3089,7 +3111,7 @@
}
goto uncharge_out;
}
- do_set_pte(vma, address, new_page, pte, true, true);
+ do_set_pte(vma, address, new_page, pte, true, true, false);
mem_cgroup_commit_charge(new_page, memcg, false, false);
lru_cache_add_active_or_unevictable(new_page, vma);
pte_unmap_unlock(pte, ptl);
@@ -3146,7 +3168,7 @@
put_page(fault_page);
return ret;
}
- do_set_pte(vma, address, fault_page, pte, true, false);
+ do_set_pte(vma, address, fault_page, pte, true, false, false);
pte_unmap_unlock(pte, ptl);
if (set_page_dirty(fault_page))
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index aa34431..caf2a14 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -78,9 +78,24 @@
#define memhp_lock_acquire() lock_map_acquire(&mem_hotplug.dep_map)
#define memhp_lock_release() lock_map_release(&mem_hotplug.dep_map)
+#ifndef CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE
bool memhp_auto_online;
+#else
+bool memhp_auto_online = true;
+#endif
EXPORT_SYMBOL_GPL(memhp_auto_online);
+static int __init setup_memhp_default_state(char *str)
+{
+ if (!strcmp(str, "online"))
+ memhp_auto_online = true;
+ else if (!strcmp(str, "offline"))
+ memhp_auto_online = false;
+
+ return 1;
+}
+__setup("memhp_default_state=", setup_memhp_default_state);
+
void get_online_mems(void)
{
might_sleep();
@@ -1410,7 +1425,7 @@
}
/* Checks if this range of memory is likely to be hot-removable. */
-int is_mem_section_removable(unsigned long start_pfn, unsigned long nr_pages)
+bool is_mem_section_removable(unsigned long start_pfn, unsigned long nr_pages)
{
struct page *page = pfn_to_page(start_pfn);
struct page *end_page = page + nr_pages;
@@ -1418,12 +1433,12 @@
/* Check the starting page of each pageblock within the range */
for (; page < end_page; page = next_active_pageblock(page)) {
if (!is_pageblock_removable_nolock(page))
- return 0;
+ return false;
cond_resched();
}
/* All pageblocks in the memory block are likely to be hot-removable */
- return 1;
+ return true;
}
/*
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 36cc01b..297d685 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -97,7 +97,6 @@
#include <asm/tlbflush.h>
#include <asm/uaccess.h>
-#include <linux/random.h>
#include "internal.h"
@@ -347,9 +346,7 @@
BUG();
if (!node_isset(current->il_next, tmp)) {
- current->il_next = next_node(current->il_next, tmp);
- if (current->il_next >= MAX_NUMNODES)
- current->il_next = first_node(tmp);
+ current->il_next = next_node_in(current->il_next, tmp);
if (current->il_next >= MAX_NUMNODES)
current->il_next = numa_node_id();
}
@@ -1709,9 +1706,7 @@
struct task_struct *me = current;
nid = me->il_next;
- next = next_node(nid, policy->v.nodes);
- if (next >= MAX_NUMNODES)
- next = first_node(policy->v.nodes);
+ next = next_node_in(nid, policy->v.nodes);
if (next < MAX_NUMNODES)
me->il_next = next;
return nid;
@@ -1744,18 +1739,18 @@
return interleave_nodes(policy);
case MPOL_BIND: {
+ struct zoneref *z;
+
/*
* Follow bind policy behavior and start allocation at the
* first node.
*/
struct zonelist *zonelist;
- struct zone *zone;
enum zone_type highest_zoneidx = gfp_zone(GFP_KERNEL);
zonelist = &NODE_DATA(node)->node_zonelists[0];
- (void)first_zones_zonelist(zonelist, highest_zoneidx,
- &policy->v.nodes,
- &zone);
- return zone ? zone->node : node;
+ z = first_zones_zonelist(zonelist, highest_zoneidx,
+ &policy->v.nodes);
+ return z->zone ? z->zone->node : node;
}
default:
@@ -1763,23 +1758,25 @@
}
}
-/* Do static interleaving for a VMA with known offset. */
+/*
+ * Do static interleaving for a VMA with known offset @n. Returns the n'th
+ * node in pol->v.nodes (starting from n=0), wrapping around if n exceeds the
+ * number of present nodes.
+ */
static unsigned offset_il_node(struct mempolicy *pol,
- struct vm_area_struct *vma, unsigned long off)
+ struct vm_area_struct *vma, unsigned long n)
{
unsigned nnodes = nodes_weight(pol->v.nodes);
unsigned target;
- int c;
- int nid = NUMA_NO_NODE;
+ int i;
+ int nid;
if (!nnodes)
return numa_node_id();
- target = (unsigned int)off % nnodes;
- c = 0;
- do {
+ target = (unsigned int)n % nnodes;
+ nid = first_node(pol->v.nodes);
+ for (i = 0; i < target; i++)
nid = next_node(nid, pol->v.nodes);
- c++;
- } while (c <= target);
return nid;
}
@@ -1805,21 +1802,6 @@
return interleave_nodes(pol);
}
-/*
- * Return the bit number of a random bit set in the nodemask.
- * (returns NUMA_NO_NODE if nodemask is empty)
- */
-int node_random(const nodemask_t *maskp)
-{
- int w, bit = NUMA_NO_NODE;
-
- w = nodes_weight(*maskp);
- if (w)
- bit = bitmap_ord_to_pos(maskp->bits,
- get_random_int() % w, MAX_NUMNODES);
- return bit;
-}
-
#ifdef CONFIG_HUGETLBFS
/*
* huge_zonelist(@vma, @addr, @gfp_flags, @mpol)
@@ -2284,7 +2266,7 @@
int mpol_misplaced(struct page *page, struct vm_area_struct *vma, unsigned long addr)
{
struct mempolicy *pol;
- struct zone *zone;
+ struct zoneref *z;
int curnid = page_to_nid(page);
unsigned long pgoff;
int thiscpu = raw_smp_processor_id();
@@ -2316,6 +2298,7 @@
break;
case MPOL_BIND:
+
/*
* allows binding to multiple nodes.
* use current page if in policy nodemask,
@@ -2324,11 +2307,11 @@
*/
if (node_isset(curnid, pol->v.nodes))
goto out;
- (void)first_zones_zonelist(
+ z = first_zones_zonelist(
node_zonelist(numa_node_id(), GFP_HIGHUSER),
gfp_zone(GFP_HIGHUSER),
- &pol->v.nodes, &zone);
- polnid = zone->node;
+ &pol->v.nodes);
+ polnid = z->zone->node;
break;
default:
diff --git a/mm/mempool.c b/mm/mempool.c
index 9b7a14a..9e075f8 100644
--- a/mm/mempool.c
+++ b/mm/mempool.c
@@ -105,7 +105,7 @@
static void kasan_poison_element(mempool_t *pool, void *element)
{
if (pool->alloc == mempool_alloc_slab)
- kasan_slab_free(pool->pool_data, element);
+ kasan_poison_slab_free(pool->pool_data, element);
if (pool->alloc == mempool_kmalloc)
kasan_kfree(element);
if (pool->alloc == mempool_alloc_pages)
diff --git a/mm/migrate.c b/mm/migrate.c
index f9dfb18..9baf41c 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -332,7 +332,7 @@
newpage->index = page->index;
newpage->mapping = page->mapping;
if (PageSwapBacked(page))
- SetPageSwapBacked(newpage);
+ __SetPageSwapBacked(newpage);
return MIGRATEPAGE_SUCCESS;
}
@@ -378,7 +378,7 @@
newpage->index = page->index;
newpage->mapping = page->mapping;
if (PageSwapBacked(page))
- SetPageSwapBacked(newpage);
+ __SetPageSwapBacked(newpage);
get_page(newpage); /* add cache reference */
if (PageSwapCache(page)) {
@@ -1171,6 +1171,7 @@
switch(rc) {
case -ENOMEM:
+ nr_failed++;
goto out;
case -EAGAIN:
retry++;
@@ -1791,7 +1792,7 @@
/* Prepare a page as a migration target */
__SetPageLocked(new_page);
- SetPageSwapBacked(new_page);
+ __SetPageSwapBacked(new_page);
/* anon mapping, we can simply copy page->mapping to the new page: */
new_page->mapping = page->mapping;
diff --git a/mm/mlock.c b/mm/mlock.c
index 96f0010..ef8dc9f 100644
--- a/mm/mlock.c
+++ b/mm/mlock.c
@@ -617,7 +617,7 @@
return error;
}
-static int do_mlock(unsigned long start, size_t len, vm_flags_t flags)
+static __must_check int do_mlock(unsigned long start, size_t len, vm_flags_t flags)
{
unsigned long locked;
unsigned long lock_limit;
@@ -635,7 +635,8 @@
lock_limit >>= PAGE_SHIFT;
locked = len >> PAGE_SHIFT;
- down_write(¤t->mm->mmap_sem);
+ if (down_write_killable(¤t->mm->mmap_sem))
+ return -EINTR;
locked += current->mm->locked_vm;
@@ -678,7 +679,8 @@
len = PAGE_ALIGN(len + (offset_in_page(start)));
start &= PAGE_MASK;
- down_write(¤t->mm->mmap_sem);
+ if (down_write_killable(¤t->mm->mmap_sem))
+ return -EINTR;
ret = apply_vma_lock_flags(start, len, 0);
up_write(¤t->mm->mmap_sem);
@@ -748,9 +750,10 @@
lock_limit = rlimit(RLIMIT_MEMLOCK);
lock_limit >>= PAGE_SHIFT;
- ret = -ENOMEM;
- down_write(¤t->mm->mmap_sem);
+ if (down_write_killable(¤t->mm->mmap_sem))
+ return -EINTR;
+ ret = -ENOMEM;
if (!(flags & MCL_CURRENT) || (current->mm->total_vm <= lock_limit) ||
capable(CAP_IPC_LOCK))
ret = apply_mlockall_flags(flags);
@@ -765,7 +768,8 @@
{
int ret;
- down_write(¤t->mm->mmap_sem);
+ if (down_write_killable(¤t->mm->mmap_sem))
+ return -EINTR;
ret = apply_mlockall_flags(0);
up_write(¤t->mm->mmap_sem);
return ret;
diff --git a/mm/mmap.c b/mm/mmap.c
index bd2e1a53..d3d9a94 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -55,10 +55,6 @@
#define arch_mmap_check(addr, len, flags) (0)
#endif
-#ifndef arch_rebalance_pgtables
-#define arch_rebalance_pgtables(addr, len) (addr)
-#endif
-
#ifdef CONFIG_HAVE_ARCH_MMAP_RND_BITS
const int mmap_rnd_bits_min = CONFIG_ARCH_MMAP_RND_BITS_MIN;
const int mmap_rnd_bits_max = CONFIG_ARCH_MMAP_RND_BITS_MAX;
@@ -70,7 +66,7 @@
int mmap_rnd_compat_bits __read_mostly = CONFIG_ARCH_MMAP_RND_COMPAT_BITS;
#endif
-static bool ignore_rlimit_data = true;
+static bool ignore_rlimit_data;
core_param(ignore_rlimit_data, ignore_rlimit_data, bool, 0644);
static void unmap_region(struct mm_struct *mm,
@@ -182,7 +178,8 @@
unsigned long min_brk;
bool populate;
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
#ifdef CONFIG_COMPAT_BRK
/*
@@ -1911,7 +1908,6 @@
if (offset_in_page(addr))
return -EINVAL;
- addr = arch_rebalance_pgtables(addr, len);
error = security_mmap_addr(addr);
return error ? error : addr;
}
@@ -2498,7 +2494,9 @@
int ret;
struct mm_struct *mm = current->mm;
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
+
ret = do_munmap(mm, start, len);
up_write(&mm->mmap_sem);
return ret;
@@ -2507,8 +2505,15 @@
SYSCALL_DEFINE2(munmap, unsigned long, addr, size_t, len)
{
+ int ret;
+ struct mm_struct *mm = current->mm;
+
profile_munmap(addr);
- return vm_munmap(addr, len);
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
+ ret = do_munmap(mm, addr, len);
+ up_write(&mm->mmap_sem);
+ return ret;
}
@@ -2540,7 +2545,9 @@
if (pgoff + (size >> PAGE_SHIFT) < pgoff)
return ret;
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
+
vma = find_vma(mm, start);
if (!vma || !(vma->vm_flags & VM_SHARED))
@@ -2705,7 +2712,9 @@
unsigned long ret;
bool populate;
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
+
ret = do_brk(addr, len);
populate = ((mm->def_flags & VM_LOCKED) != 0);
up_write(&mm->mmap_sem);
@@ -2891,13 +2900,17 @@
if (is_data_mapping(flags) &&
mm->data_vm + npages > rlimit(RLIMIT_DATA) >> PAGE_SHIFT) {
- if (ignore_rlimit_data)
- pr_warn_once("%s (%d): VmData %lu exceed data ulimit %lu. Will be forbidden soon.\n",
+ /* Workaround for Valgrind */
+ if (rlimit(RLIMIT_DATA) == 0 &&
+ mm->data_vm + npages <= rlimit_max(RLIMIT_DATA) >> PAGE_SHIFT)
+ return true;
+ if (!ignore_rlimit_data) {
+ pr_warn_once("%s (%d): VmData %lu exceed data ulimit %lu. Update limits or use boot option ignore_rlimit_data.\n",
current->comm, current->pid,
(mm->data_vm + npages) << PAGE_SHIFT,
rlimit(RLIMIT_DATA));
- else
return false;
+ }
}
return true;
diff --git a/mm/mmzone.c b/mm/mmzone.c
index 52687fb..5652be8 100644
--- a/mm/mmzone.c
+++ b/mm/mmzone.c
@@ -52,7 +52,7 @@
}
/* Returns the next zone at or below highest_zoneidx in a zonelist */
-struct zoneref *next_zones_zonelist(struct zoneref *z,
+struct zoneref *__next_zones_zonelist(struct zoneref *z,
enum zone_type highest_zoneidx,
nodemask_t *nodes)
{
diff --git a/mm/mprotect.c b/mm/mprotect.c
index b650c54..5019a1e 100644
--- a/mm/mprotect.c
+++ b/mm/mprotect.c
@@ -379,7 +379,8 @@
reqprot = prot;
- down_write(¤t->mm->mmap_sem);
+ if (down_write_killable(¤t->mm->mmap_sem))
+ return -EINTR;
vma = find_vma(current->mm, start);
error = -ENOMEM;
diff --git a/mm/mremap.c b/mm/mremap.c
index 3fa0a467..1f157ad 100644
--- a/mm/mremap.c
+++ b/mm/mremap.c
@@ -70,6 +70,22 @@
return pmd;
}
+static void take_rmap_locks(struct vm_area_struct *vma)
+{
+ if (vma->vm_file)
+ i_mmap_lock_write(vma->vm_file->f_mapping);
+ if (vma->anon_vma)
+ anon_vma_lock_write(vma->anon_vma);
+}
+
+static void drop_rmap_locks(struct vm_area_struct *vma)
+{
+ if (vma->anon_vma)
+ anon_vma_unlock_write(vma->anon_vma);
+ if (vma->vm_file)
+ i_mmap_unlock_write(vma->vm_file->f_mapping);
+}
+
static pte_t move_soft_dirty_pte(pte_t pte)
{
/*
@@ -90,8 +106,6 @@
struct vm_area_struct *new_vma, pmd_t *new_pmd,
unsigned long new_addr, bool need_rmap_locks)
{
- struct address_space *mapping = NULL;
- struct anon_vma *anon_vma = NULL;
struct mm_struct *mm = vma->vm_mm;
pte_t *old_pte, *new_pte, pte;
spinlock_t *old_ptl, *new_ptl;
@@ -114,16 +128,8 @@
* serialize access to individual ptes, but only rmap traversal
* order guarantees that we won't miss both the old and new ptes).
*/
- if (need_rmap_locks) {
- if (vma->vm_file) {
- mapping = vma->vm_file->f_mapping;
- i_mmap_lock_write(mapping);
- }
- if (vma->anon_vma) {
- anon_vma = vma->anon_vma;
- anon_vma_lock_write(anon_vma);
- }
- }
+ if (need_rmap_locks)
+ take_rmap_locks(vma);
/*
* We don't have to worry about the ordering of src and dst
@@ -151,10 +157,8 @@
spin_unlock(new_ptl);
pte_unmap(new_pte - 1);
pte_unmap_unlock(old_pte - 1, old_ptl);
- if (anon_vma)
- anon_vma_unlock_write(anon_vma);
- if (mapping)
- i_mmap_unlock_write(mapping);
+ if (need_rmap_locks)
+ drop_rmap_locks(vma);
}
#define LATENCY_LIMIT (64 * PAGE_SIZE)
@@ -193,16 +197,13 @@
if (pmd_trans_huge(*old_pmd)) {
if (extent == HPAGE_PMD_SIZE) {
bool moved;
- VM_BUG_ON_VMA(vma->vm_file || !vma->anon_vma,
- vma);
/* See comment in move_ptes() */
if (need_rmap_locks)
- anon_vma_lock_write(vma->anon_vma);
- moved = move_huge_pmd(vma, new_vma, old_addr,
- new_addr, old_end,
- old_pmd, new_pmd);
+ take_rmap_locks(vma);
+ moved = move_huge_pmd(vma, old_addr, new_addr,
+ old_end, old_pmd, new_pmd);
if (need_rmap_locks)
- anon_vma_unlock_write(vma->anon_vma);
+ drop_rmap_locks(vma);
if (moved) {
need_flush = true;
continue;
@@ -502,7 +503,8 @@
if (!new_len)
return ret;
- down_write(¤t->mm->mmap_sem);
+ if (down_write_killable(¤t->mm->mmap_sem))
+ return -EINTR;
if (flags & MREMAP_FIXED) {
ret = mremap_to(addr, old_len, new_addr, new_len,
diff --git a/mm/oom_kill.c b/mm/oom_kill.c
index 8634958..5bb2f76 100644
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -174,8 +174,13 @@
if (!p)
return 0;
+ /*
+ * Do not even consider tasks which are explicitly marked oom
+ * unkillable or have been already oom reaped.
+ */
adj = (long)p->signal->oom_score_adj;
- if (adj == OOM_SCORE_ADJ_MIN) {
+ if (adj == OOM_SCORE_ADJ_MIN ||
+ test_bit(MMF_OOM_REAPED, &p->mm->flags)) {
task_unlock(p);
return 0;
}
@@ -278,12 +283,8 @@
* This task already has access to memory reserves and is being killed.
* Don't allow any other task to have access to the reserves.
*/
- if (test_tsk_thread_flag(task, TIF_MEMDIE)) {
- if (!is_sysrq_oom(oc))
- return OOM_SCAN_ABORT;
- }
- if (!task->mm)
- return OOM_SCAN_CONTINUE;
+ if (!is_sysrq_oom(oc) && atomic_read(&task->signal->oom_victims))
+ return OOM_SCAN_ABORT;
/*
* If task is allocating a lot of memory and has been marked to be
@@ -302,12 +303,12 @@
static struct task_struct *select_bad_process(struct oom_control *oc,
unsigned int *ppoints, unsigned long totalpages)
{
- struct task_struct *g, *p;
+ struct task_struct *p;
struct task_struct *chosen = NULL;
unsigned long chosen_points = 0;
rcu_read_lock();
- for_each_process_thread(g, p) {
+ for_each_process(p) {
unsigned int points;
switch (oom_scan_process_thread(oc, p, totalpages)) {
@@ -326,9 +327,6 @@
points = oom_badness(p, NULL, oc->nodemask, totalpages);
if (!points || points < chosen_points)
continue;
- /* Prefer thread group leaders for display purposes */
- if (points == chosen_points && thread_group_leader(chosen))
- continue;
chosen = p;
chosen_points = points;
@@ -412,6 +410,25 @@
#define K(x) ((x) << (PAGE_SHIFT-10))
+/*
+ * task->mm can be NULL if the task is the exited group leader. So to
+ * determine whether the task is using a particular mm, we examine all the
+ * task's threads: if one of those is using this mm then this task was also
+ * using it.
+ */
+static bool process_shares_mm(struct task_struct *p, struct mm_struct *mm)
+{
+ struct task_struct *t;
+
+ for_each_thread(p, t) {
+ struct mm_struct *t_mm = READ_ONCE(t->mm);
+ if (t_mm)
+ return t_mm == mm;
+ }
+ return false;
+}
+
+
#ifdef CONFIG_MMU
/*
* OOM Reaper kernel thread which tries to reap the memory used by the OOM
@@ -422,7 +439,6 @@
static struct task_struct *oom_reaper_list;
static DEFINE_SPINLOCK(oom_reaper_lock);
-
static bool __oom_reap_task(struct task_struct *tsk)
{
struct mmu_gather tlb;
@@ -491,16 +507,17 @@
up_read(&mm->mmap_sem);
/*
- * Clear TIF_MEMDIE because the task shouldn't be sitting on a
- * reasonably reclaimable memory anymore. OOM killer can continue
- * by selecting other victim if unmapping hasn't led to any
- * improvements. This also means that selecting this task doesn't
- * make any sense.
+ * This task can be safely ignored because we cannot do much more
+ * to release its memory.
*/
- tsk->signal->oom_score_adj = OOM_SCORE_ADJ_MIN;
- exit_oom_victim(tsk);
+ set_bit(MMF_OOM_REAPED, &mm->flags);
out:
- mmput(mm);
+ /*
+ * Drop our reference but make sure the mmput slow path is called from a
+ * different context because we shouldn't risk we get stuck there and
+ * put the oom_reaper out of the way.
+ */
+ mmput_async(mm);
return ret;
}
@@ -519,6 +536,15 @@
debug_show_all_locks();
}
+ /*
+ * Clear TIF_MEMDIE because the task shouldn't be sitting on a
+ * reasonably reclaimable memory anymore or it is not a good candidate
+ * for the oom victim right now because it cannot release its memory
+ * itself nor by the oom reaper.
+ */
+ tsk->oom_reaper_list = NULL;
+ exit_oom_victim(tsk);
+
/* Drop a reference taken by wake_oom_reaper */
put_task_struct(tsk);
}
@@ -563,6 +589,53 @@
wake_up(&oom_reaper_wait);
}
+/* Check if we can reap the given task. This has to be called with stable
+ * tsk->mm
+ */
+void try_oom_reaper(struct task_struct *tsk)
+{
+ struct mm_struct *mm = tsk->mm;
+ struct task_struct *p;
+
+ if (!mm)
+ return;
+
+ /*
+ * There might be other threads/processes which are either not
+ * dying or even not killable.
+ */
+ if (atomic_read(&mm->mm_users) > 1) {
+ rcu_read_lock();
+ for_each_process(p) {
+ bool exiting;
+
+ if (!process_shares_mm(p, mm))
+ continue;
+ if (same_thread_group(p, tsk))
+ continue;
+ if (fatal_signal_pending(p))
+ continue;
+
+ /*
+ * If the task is exiting make sure the whole thread group
+ * is exiting and cannot acces mm anymore.
+ */
+ spin_lock_irq(&p->sighand->siglock);
+ exiting = signal_group_exit(p->signal);
+ spin_unlock_irq(&p->sighand->siglock);
+ if (exiting)
+ continue;
+
+ /* Give up */
+ rcu_read_unlock();
+ return;
+ }
+ rcu_read_unlock();
+ }
+
+ wake_oom_reaper(tsk);
+}
+
static int __init oom_init(void)
{
oom_reaper_th = kthread_run(oom_reaper, NULL, "oom_reaper");
@@ -593,6 +666,7 @@
/* OOM killer might race with memcg OOM */
if (test_and_set_tsk_thread_flag(tsk, TIF_MEMDIE))
return;
+ atomic_inc(&tsk->signal->oom_victims);
/*
* Make sure that the task is woken up from uninterruptible sleep
* if it is frozen because OOM killer wouldn't be able to free
@@ -610,6 +684,7 @@
{
if (!test_and_clear_tsk_thread_flag(tsk, TIF_MEMDIE))
return;
+ atomic_dec(&tsk->signal->oom_victims);
if (!atomic_dec_return(&oom_victims))
wake_up_all(&oom_victims_wait);
@@ -653,24 +728,6 @@
}
/*
- * task->mm can be NULL if the task is the exited group leader. So to
- * determine whether the task is using a particular mm, we examine all the
- * task's threads: if one of those is using this mm then this task was also
- * using it.
- */
-static bool process_shares_mm(struct task_struct *p, struct mm_struct *mm)
-{
- struct task_struct *t;
-
- for_each_thread(p, t) {
- struct mm_struct *t_mm = READ_ONCE(t->mm);
- if (t_mm)
- return t_mm == mm;
- }
- return false;
-}
-
-/*
* Must be called while holding a reference to p, which will be released upon
* returning.
*/
@@ -694,6 +751,7 @@
task_lock(p);
if (p->mm && task_will_free_mem(p)) {
mark_oom_victim(p);
+ try_oom_reaper(p);
task_unlock(p);
put_task_struct(p);
return;
@@ -873,10 +931,20 @@
if (current->mm &&
(fatal_signal_pending(current) || task_will_free_mem(current))) {
mark_oom_victim(current);
+ try_oom_reaper(current);
return true;
}
/*
+ * The OOM killer does not compensate for IO-less reclaim.
+ * pagefault_out_of_memory lost its gfp context so we have to
+ * make sure exclude 0 mask - all other users should have at least
+ * ___GFP_DIRECT_RECLAIM to get here.
+ */
+ if (oc->gfp_mask && !(oc->gfp_mask & (__GFP_FS|__GFP_NOFAIL)))
+ return true;
+
+ /*
* Check if there were limitations on the allocation (only relevant for
* NUMA) that may require different handling.
*/
diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index bc5149d..b9956fd 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -296,11 +296,15 @@
#ifdef CONFIG_HIGHMEM
int node;
unsigned long x = 0;
+ int i;
for_each_node_state(node, N_HIGH_MEMORY) {
- struct zone *z = &NODE_DATA(node)->node_zones[ZONE_HIGHMEM];
+ for (i = 0; i < MAX_NR_ZONES; i++) {
+ struct zone *z = &NODE_DATA(node)->node_zones[i];
- x += zone_dirtyable_memory(z);
+ if (is_highmem(z))
+ x += zone_dirtyable_memory(z);
+ }
}
/*
* Unreclaimable memory (kernel memory or anonymous memory
@@ -407,8 +411,8 @@
bg_thresh = thresh / 2;
tsk = current;
if (tsk->flags & PF_LESS_THROTTLE || rt_task(tsk)) {
- bg_thresh += bg_thresh / 4;
- thresh += thresh / 4;
+ bg_thresh += bg_thresh / 4 + global_wb_domain.dirty_limit / 32;
+ thresh += thresh / 4 + global_wb_domain.dirty_limit / 32;
}
dtc->thresh = thresh;
dtc->bg_thresh = bg_thresh;
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index c1069ef..f8f3bfc 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -352,6 +352,106 @@
}
#endif
+/* Return a pointer to the bitmap storing bits affecting a block of pages */
+static inline unsigned long *get_pageblock_bitmap(struct page *page,
+ unsigned long pfn)
+{
+#ifdef CONFIG_SPARSEMEM
+ return __pfn_to_section(pfn)->pageblock_flags;
+#else
+ return page_zone(page)->pageblock_flags;
+#endif /* CONFIG_SPARSEMEM */
+}
+
+static inline int pfn_to_bitidx(struct page *page, unsigned long pfn)
+{
+#ifdef CONFIG_SPARSEMEM
+ pfn &= (PAGES_PER_SECTION-1);
+ return (pfn >> pageblock_order) * NR_PAGEBLOCK_BITS;
+#else
+ pfn = pfn - round_down(page_zone(page)->zone_start_pfn, pageblock_nr_pages);
+ return (pfn >> pageblock_order) * NR_PAGEBLOCK_BITS;
+#endif /* CONFIG_SPARSEMEM */
+}
+
+/**
+ * get_pfnblock_flags_mask - Return the requested group of flags for the pageblock_nr_pages block of pages
+ * @page: The page within the block of interest
+ * @pfn: The target page frame number
+ * @end_bitidx: The last bit of interest to retrieve
+ * @mask: mask of bits that the caller is interested in
+ *
+ * Return: pageblock_bits flags
+ */
+static __always_inline unsigned long __get_pfnblock_flags_mask(struct page *page,
+ unsigned long pfn,
+ unsigned long end_bitidx,
+ unsigned long mask)
+{
+ unsigned long *bitmap;
+ unsigned long bitidx, word_bitidx;
+ unsigned long word;
+
+ bitmap = get_pageblock_bitmap(page, pfn);
+ bitidx = pfn_to_bitidx(page, pfn);
+ word_bitidx = bitidx / BITS_PER_LONG;
+ bitidx &= (BITS_PER_LONG-1);
+
+ word = bitmap[word_bitidx];
+ bitidx += end_bitidx;
+ return (word >> (BITS_PER_LONG - bitidx - 1)) & mask;
+}
+
+unsigned long get_pfnblock_flags_mask(struct page *page, unsigned long pfn,
+ unsigned long end_bitidx,
+ unsigned long mask)
+{
+ return __get_pfnblock_flags_mask(page, pfn, end_bitidx, mask);
+}
+
+static __always_inline int get_pfnblock_migratetype(struct page *page, unsigned long pfn)
+{
+ return __get_pfnblock_flags_mask(page, pfn, PB_migrate_end, MIGRATETYPE_MASK);
+}
+
+/**
+ * set_pfnblock_flags_mask - Set the requested group of flags for a pageblock_nr_pages block of pages
+ * @page: The page within the block of interest
+ * @flags: The flags to set
+ * @pfn: The target page frame number
+ * @end_bitidx: The last bit of interest
+ * @mask: mask of bits that the caller is interested in
+ */
+void set_pfnblock_flags_mask(struct page *page, unsigned long flags,
+ unsigned long pfn,
+ unsigned long end_bitidx,
+ unsigned long mask)
+{
+ unsigned long *bitmap;
+ unsigned long bitidx, word_bitidx;
+ unsigned long old_word, word;
+
+ BUILD_BUG_ON(NR_PAGEBLOCK_BITS != 4);
+
+ bitmap = get_pageblock_bitmap(page, pfn);
+ bitidx = pfn_to_bitidx(page, pfn);
+ word_bitidx = bitidx / BITS_PER_LONG;
+ bitidx &= (BITS_PER_LONG-1);
+
+ VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page);
+
+ bitidx += end_bitidx;
+ mask <<= (BITS_PER_LONG - bitidx - 1);
+ flags <<= (BITS_PER_LONG - bitidx - 1);
+
+ word = READ_ONCE(bitmap[word_bitidx]);
+ for (;;) {
+ old_word = cmpxchg(&bitmap[word_bitidx], word, (word & ~mask) | flags);
+ if (word == old_word)
+ break;
+ word = old_word;
+ }
+}
void set_pageblock_migratetype(struct page *page, int migratetype)
{
@@ -422,12 +522,6 @@
static unsigned long nr_shown;
static unsigned long nr_unshown;
- /* Don't complain about poisoned pages */
- if (PageHWPoison(page)) {
- page_mapcount_reset(page); /* remove PageBuddy */
- return;
- }
-
/*
* Allow a burst of 60 reports, then keep quiet for that minute;
* or allow a steady drip of one report per second.
@@ -513,14 +607,7 @@
{
if (!buf)
return -EINVAL;
-
- if (strcmp(buf, "on") == 0)
- _debug_pagealloc_enabled = true;
-
- if (strcmp(buf, "off") == 0)
- _debug_pagealloc_enabled = false;
-
- return 0;
+ return kstrtobool(buf, &_debug_pagealloc_enabled);
}
early_param("debug_pagealloc", early_debug_pagealloc);
@@ -784,17 +871,42 @@
zone->free_area[order].nr_free++;
}
-static inline int free_pages_check(struct page *page)
+/*
+ * A bad page could be due to a number of fields. Instead of multiple branches,
+ * try and check multiple fields with one check. The caller must do a detailed
+ * check if necessary.
+ */
+static inline bool page_expected_state(struct page *page,
+ unsigned long check_flags)
{
- const char *bad_reason = NULL;
- unsigned long bad_flags = 0;
+ if (unlikely(atomic_read(&page->_mapcount) != -1))
+ return false;
+
+ if (unlikely((unsigned long)page->mapping |
+ page_ref_count(page) |
+#ifdef CONFIG_MEMCG
+ (unsigned long)page->mem_cgroup |
+#endif
+ (page->flags & check_flags)))
+ return false;
+
+ return true;
+}
+
+static void free_pages_check_bad(struct page *page)
+{
+ const char *bad_reason;
+ unsigned long bad_flags;
+
+ bad_reason = NULL;
+ bad_flags = 0;
if (unlikely(atomic_read(&page->_mapcount) != -1))
bad_reason = "nonzero mapcount";
if (unlikely(page->mapping != NULL))
bad_reason = "non-NULL mapping";
if (unlikely(page_ref_count(page) != 0))
- bad_reason = "nonzero _count";
+ bad_reason = "nonzero _refcount";
if (unlikely(page->flags & PAGE_FLAGS_CHECK_AT_FREE)) {
bad_reason = "PAGE_FLAGS_CHECK_AT_FREE flag(s) set";
bad_flags = PAGE_FLAGS_CHECK_AT_FREE;
@@ -803,100 +915,17 @@
if (unlikely(page->mem_cgroup))
bad_reason = "page still charged to cgroup";
#endif
- if (unlikely(bad_reason)) {
- bad_page(page, bad_reason, bad_flags);
- return 1;
- }
- page_cpupid_reset_last(page);
- if (page->flags & PAGE_FLAGS_CHECK_AT_PREP)
- page->flags &= ~PAGE_FLAGS_CHECK_AT_PREP;
- return 0;
+ bad_page(page, bad_reason, bad_flags);
}
-/*
- * Frees a number of pages from the PCP lists
- * Assumes all pages on list are in same zone, and of same order.
- * count is the number of pages to free.
- *
- * If the zone was previously in an "all pages pinned" state then look to
- * see if this freeing clears that state.
- *
- * And clear the zone's pages_scanned counter, to hold off the "all pages are
- * pinned" detection logic.
- */
-static void free_pcppages_bulk(struct zone *zone, int count,
- struct per_cpu_pages *pcp)
+static inline int free_pages_check(struct page *page)
{
- int migratetype = 0;
- int batch_free = 0;
- int to_free = count;
- unsigned long nr_scanned;
+ if (likely(page_expected_state(page, PAGE_FLAGS_CHECK_AT_FREE)))
+ return 0;
- spin_lock(&zone->lock);
- nr_scanned = zone_page_state(zone, NR_PAGES_SCANNED);
- if (nr_scanned)
- __mod_zone_page_state(zone, NR_PAGES_SCANNED, -nr_scanned);
-
- while (to_free) {
- struct page *page;
- struct list_head *list;
-
- /*
- * Remove pages from lists in a round-robin fashion. A
- * batch_free count is maintained that is incremented when an
- * empty list is encountered. This is so more pages are freed
- * off fuller lists instead of spinning excessively around empty
- * lists
- */
- do {
- batch_free++;
- if (++migratetype == MIGRATE_PCPTYPES)
- migratetype = 0;
- list = &pcp->lists[migratetype];
- } while (list_empty(list));
-
- /* This is the only non-empty list. Free them all. */
- if (batch_free == MIGRATE_PCPTYPES)
- batch_free = to_free;
-
- do {
- int mt; /* migratetype of the to-be-freed page */
-
- page = list_last_entry(list, struct page, lru);
- /* must delete as __free_one_page list manipulates */
- list_del(&page->lru);
-
- mt = get_pcppage_migratetype(page);
- /* MIGRATE_ISOLATE page should not go to pcplists */
- VM_BUG_ON_PAGE(is_migrate_isolate(mt), page);
- /* Pageblock could have been isolated meanwhile */
- if (unlikely(has_isolate_pageblock(zone)))
- mt = get_pageblock_migratetype(page);
-
- __free_one_page(page, page_to_pfn(page), zone, 0, mt);
- trace_mm_page_pcpu_drain(page, 0, mt);
- } while (--to_free && --batch_free && !list_empty(list));
- }
- spin_unlock(&zone->lock);
-}
-
-static void free_one_page(struct zone *zone,
- struct page *page, unsigned long pfn,
- unsigned int order,
- int migratetype)
-{
- unsigned long nr_scanned;
- spin_lock(&zone->lock);
- nr_scanned = zone_page_state(zone, NR_PAGES_SCANNED);
- if (nr_scanned)
- __mod_zone_page_state(zone, NR_PAGES_SCANNED, -nr_scanned);
-
- if (unlikely(has_isolate_pageblock(zone) ||
- is_migrate_isolate(migratetype))) {
- migratetype = get_pfnblock_migratetype(page, pfn);
- }
- __free_one_page(page, pfn, zone, order, migratetype);
- spin_unlock(&zone->lock);
+ /* Something has gone sideways, find it */
+ free_pages_check_bad(page);
+ return 1;
}
static int free_tail_pages_check(struct page *head_page, struct page *page)
@@ -949,6 +978,173 @@
return ret;
}
+static __always_inline bool free_pages_prepare(struct page *page,
+ unsigned int order, bool check_free)
+{
+ int bad = 0;
+
+ VM_BUG_ON_PAGE(PageTail(page), page);
+
+ trace_mm_page_free(page, order);
+ kmemcheck_free_shadow(page, order);
+
+ /*
+ * Check tail pages before head page information is cleared to
+ * avoid checking PageCompound for order-0 pages.
+ */
+ if (unlikely(order)) {
+ bool compound = PageCompound(page);
+ int i;
+
+ VM_BUG_ON_PAGE(compound && compound_order(page) != order, page);
+
+ for (i = 1; i < (1 << order); i++) {
+ if (compound)
+ bad += free_tail_pages_check(page, page + i);
+ if (unlikely(free_pages_check(page + i))) {
+ bad++;
+ continue;
+ }
+ (page + i)->flags &= ~PAGE_FLAGS_CHECK_AT_PREP;
+ }
+ }
+ if (PageAnonHead(page))
+ page->mapping = NULL;
+ if (check_free)
+ bad += free_pages_check(page);
+ if (bad)
+ return false;
+
+ page_cpupid_reset_last(page);
+ page->flags &= ~PAGE_FLAGS_CHECK_AT_PREP;
+ reset_page_owner(page, order);
+
+ if (!PageHighMem(page)) {
+ debug_check_no_locks_freed(page_address(page),
+ PAGE_SIZE << order);
+ debug_check_no_obj_freed(page_address(page),
+ PAGE_SIZE << order);
+ }
+ arch_free_page(page, order);
+ kernel_poison_pages(page, 1 << order, 0);
+ kernel_map_pages(page, 1 << order, 0);
+ kasan_free_pages(page, order);
+
+ return true;
+}
+
+#ifdef CONFIG_DEBUG_VM
+static inline bool free_pcp_prepare(struct page *page)
+{
+ return free_pages_prepare(page, 0, true);
+}
+
+static inline bool bulkfree_pcp_prepare(struct page *page)
+{
+ return false;
+}
+#else
+static bool free_pcp_prepare(struct page *page)
+{
+ return free_pages_prepare(page, 0, false);
+}
+
+static bool bulkfree_pcp_prepare(struct page *page)
+{
+ return free_pages_check(page);
+}
+#endif /* CONFIG_DEBUG_VM */
+
+/*
+ * Frees a number of pages from the PCP lists
+ * Assumes all pages on list are in same zone, and of same order.
+ * count is the number of pages to free.
+ *
+ * If the zone was previously in an "all pages pinned" state then look to
+ * see if this freeing clears that state.
+ *
+ * And clear the zone's pages_scanned counter, to hold off the "all pages are
+ * pinned" detection logic.
+ */
+static void free_pcppages_bulk(struct zone *zone, int count,
+ struct per_cpu_pages *pcp)
+{
+ int migratetype = 0;
+ int batch_free = 0;
+ unsigned long nr_scanned;
+ bool isolated_pageblocks;
+
+ spin_lock(&zone->lock);
+ isolated_pageblocks = has_isolate_pageblock(zone);
+ nr_scanned = zone_page_state(zone, NR_PAGES_SCANNED);
+ if (nr_scanned)
+ __mod_zone_page_state(zone, NR_PAGES_SCANNED, -nr_scanned);
+
+ while (count) {
+ struct page *page;
+ struct list_head *list;
+
+ /*
+ * Remove pages from lists in a round-robin fashion. A
+ * batch_free count is maintained that is incremented when an
+ * empty list is encountered. This is so more pages are freed
+ * off fuller lists instead of spinning excessively around empty
+ * lists
+ */
+ do {
+ batch_free++;
+ if (++migratetype == MIGRATE_PCPTYPES)
+ migratetype = 0;
+ list = &pcp->lists[migratetype];
+ } while (list_empty(list));
+
+ /* This is the only non-empty list. Free them all. */
+ if (batch_free == MIGRATE_PCPTYPES)
+ batch_free = count;
+
+ do {
+ int mt; /* migratetype of the to-be-freed page */
+
+ page = list_last_entry(list, struct page, lru);
+ /* must delete as __free_one_page list manipulates */
+ list_del(&page->lru);
+
+ mt = get_pcppage_migratetype(page);
+ /* MIGRATE_ISOLATE page should not go to pcplists */
+ VM_BUG_ON_PAGE(is_migrate_isolate(mt), page);
+ /* Pageblock could have been isolated meanwhile */
+ if (unlikely(isolated_pageblocks))
+ mt = get_pageblock_migratetype(page);
+
+ if (bulkfree_pcp_prepare(page))
+ continue;
+
+ __free_one_page(page, page_to_pfn(page), zone, 0, mt);
+ trace_mm_page_pcpu_drain(page, 0, mt);
+ } while (--count && --batch_free && !list_empty(list));
+ }
+ spin_unlock(&zone->lock);
+}
+
+static void free_one_page(struct zone *zone,
+ struct page *page, unsigned long pfn,
+ unsigned int order,
+ int migratetype)
+{
+ unsigned long nr_scanned;
+ spin_lock(&zone->lock);
+ nr_scanned = zone_page_state(zone, NR_PAGES_SCANNED);
+ if (nr_scanned)
+ __mod_zone_page_state(zone, NR_PAGES_SCANNED, -nr_scanned);
+
+ if (unlikely(has_isolate_pageblock(zone) ||
+ is_migrate_isolate(migratetype))) {
+ migratetype = get_pfnblock_migratetype(page, pfn);
+ }
+ __free_one_page(page, pfn, zone, order, migratetype);
+ spin_unlock(&zone->lock);
+}
+
static void __meminit __init_single_page(struct page *page, unsigned long pfn,
unsigned long zone, int nid)
{
@@ -1003,7 +1199,7 @@
* marks the pages PageReserved. The remaining valid pages are later
* sent to the buddy page allocator.
*/
-void __meminit reserve_bootmem_region(unsigned long start, unsigned long end)
+void __meminit reserve_bootmem_region(phys_addr_t start, phys_addr_t end)
{
unsigned long start_pfn = PFN_DOWN(start);
unsigned long end_pfn = PFN_UP(end);
@@ -1022,51 +1218,13 @@
}
}
-static bool free_pages_prepare(struct page *page, unsigned int order)
-{
- bool compound = PageCompound(page);
- int i, bad = 0;
-
- VM_BUG_ON_PAGE(PageTail(page), page);
- VM_BUG_ON_PAGE(compound && compound_order(page) != order, page);
-
- trace_mm_page_free(page, order);
- kmemcheck_free_shadow(page, order);
- kasan_free_pages(page, order);
-
- if (PageAnon(page))
- page->mapping = NULL;
- bad += free_pages_check(page);
- for (i = 1; i < (1 << order); i++) {
- if (compound)
- bad += free_tail_pages_check(page, page + i);
- bad += free_pages_check(page + i);
- }
- if (bad)
- return false;
-
- reset_page_owner(page, order);
-
- if (!PageHighMem(page)) {
- debug_check_no_locks_freed(page_address(page),
- PAGE_SIZE << order);
- debug_check_no_obj_freed(page_address(page),
- PAGE_SIZE << order);
- }
- arch_free_page(page, order);
- kernel_poison_pages(page, 1 << order, 0);
- kernel_map_pages(page, 1 << order, 0);
-
- return true;
-}
-
static void __free_pages_ok(struct page *page, unsigned int order)
{
unsigned long flags;
int migratetype;
unsigned long pfn = page_to_pfn(page);
- if (!free_pages_prepare(page, order))
+ if (!free_pages_prepare(page, order, true))
return;
migratetype = get_pfnblock_migratetype(page, pfn);
@@ -1076,8 +1234,7 @@
local_irq_restore(flags);
}
-static void __init __free_pages_boot_core(struct page *page,
- unsigned long pfn, unsigned int order)
+static void __init __free_pages_boot_core(struct page *page, unsigned int order)
{
unsigned int nr_pages = 1 << order;
struct page *p = page;
@@ -1154,7 +1311,7 @@
{
if (early_page_uninitialised(pfn))
return;
- return __free_pages_boot_core(page, pfn, order);
+ return __free_pages_boot_core(page, order);
}
/*
@@ -1239,12 +1396,12 @@
if (nr_pages == MAX_ORDER_NR_PAGES &&
(pfn & (MAX_ORDER_NR_PAGES-1)) == 0) {
set_pageblock_migratetype(page, MIGRATE_MOVABLE);
- __free_pages_boot_core(page, pfn, MAX_ORDER-1);
+ __free_pages_boot_core(page, MAX_ORDER-1);
return;
}
- for (i = 0; i < nr_pages; i++, page++, pfn++)
- __free_pages_boot_core(page, pfn, 0);
+ for (i = 0; i < nr_pages; i++, page++)
+ __free_pages_boot_core(page, 0);
}
/* Completion tracking for deferred_init_memmap() threads */
@@ -1477,10 +1634,7 @@
}
}
-/*
- * This page is about to be returned from the page allocator
- */
-static inline int check_new_page(struct page *page)
+static void check_new_page_bad(struct page *page)
{
const char *bad_reason = NULL;
unsigned long bad_flags = 0;
@@ -1494,6 +1648,9 @@
if (unlikely(page->flags & __PG_HWPOISON)) {
bad_reason = "HWPoisoned (hardware-corrupted)";
bad_flags = __PG_HWPOISON;
+ /* Don't complain about hwpoisoned pages */
+ page_mapcount_reset(page); /* remove PageBuddy */
+ return;
}
if (unlikely(page->flags & PAGE_FLAGS_CHECK_AT_PREP)) {
bad_reason = "PAGE_FLAGS_CHECK_AT_PREP flag set";
@@ -1503,11 +1660,20 @@
if (unlikely(page->mem_cgroup))
bad_reason = "page still charged to cgroup";
#endif
- if (unlikely(bad_reason)) {
- bad_page(page, bad_reason, bad_flags);
- return 1;
- }
- return 0;
+ bad_page(page, bad_reason, bad_flags);
+}
+
+/*
+ * This page is about to be returned from the page allocator
+ */
+static inline int check_new_page(struct page *page)
+{
+ if (likely(page_expected_state(page,
+ PAGE_FLAGS_CHECK_AT_PREP|__PG_HWPOISON)))
+ return 0;
+
+ check_new_page_bad(page);
+ return 1;
}
static inline bool free_pages_prezeroed(bool poisoned)
@@ -1516,16 +1682,48 @@
page_poisoning_enabled() && poisoned;
}
-static int prep_new_page(struct page *page, unsigned int order, gfp_t gfp_flags,
- int alloc_flags)
+#ifdef CONFIG_DEBUG_VM
+static bool check_pcp_refill(struct page *page)
+{
+ return false;
+}
+
+static bool check_new_pcp(struct page *page)
+{
+ return check_new_page(page);
+}
+#else
+static bool check_pcp_refill(struct page *page)
+{
+ return check_new_page(page);
+}
+static bool check_new_pcp(struct page *page)
+{
+ return false;
+}
+#endif /* CONFIG_DEBUG_VM */
+
+static bool check_new_pages(struct page *page, unsigned int order)
+{
+ int i;
+ for (i = 0; i < (1 << order); i++) {
+ struct page *p = page + i;
+
+ if (unlikely(check_new_page(p)))
+ return true;
+ }
+
+ return false;
+}
+
+static void prep_new_page(struct page *page, unsigned int order, gfp_t gfp_flags,
+ unsigned int alloc_flags)
{
int i;
bool poisoned = true;
for (i = 0; i < (1 << order); i++) {
struct page *p = page + i;
- if (unlikely(check_new_page(p)))
- return 1;
if (poisoned)
poisoned &= page_is_poisoned(p);
}
@@ -1557,8 +1755,6 @@
set_page_pfmemalloc(page);
else
clear_page_pfmemalloc(page);
-
- return 0;
}
/*
@@ -1980,6 +2176,9 @@
if (unlikely(page == NULL))
break;
+ if (unlikely(check_pcp_refill(page)))
+ continue;
+
/*
* Split buddy pages returned by expand() are received here
* in physical page order. The page is added to the callers and
@@ -2157,6 +2356,10 @@
for (pfn = zone->zone_start_pfn; pfn < max_zone_pfn; pfn++)
if (pfn_valid(pfn)) {
page = pfn_to_page(pfn);
+
+ if (page_zone(page) != zone)
+ continue;
+
if (!swsusp_page_is_forbidden(page))
swsusp_unset_page_free(page);
}
@@ -2187,7 +2390,7 @@
unsigned long pfn = page_to_pfn(page);
int migratetype;
- if (!free_pages_prepare(page, 0))
+ if (!free_pcp_prepare(page))
return;
migratetype = get_pfnblock_migratetype(page, pfn);
@@ -2343,12 +2546,44 @@
}
/*
+ * Update NUMA hit/miss statistics
+ *
+ * Must be called with interrupts disabled.
+ *
+ * When __GFP_OTHER_NODE is set assume the node of the preferred
+ * zone is the local node. This is useful for daemons who allocate
+ * memory on behalf of other processes.
+ */
+static inline void zone_statistics(struct zone *preferred_zone, struct zone *z,
+ gfp_t flags)
+{
+#ifdef CONFIG_NUMA
+ int local_nid = numa_node_id();
+ enum zone_stat_item local_stat = NUMA_LOCAL;
+
+ if (unlikely(flags & __GFP_OTHER_NODE)) {
+ local_stat = NUMA_OTHER;
+ local_nid = preferred_zone->node;
+ }
+
+ if (z->node == local_nid) {
+ __inc_zone_state(z, NUMA_HIT);
+ __inc_zone_state(z, local_stat);
+ } else {
+ __inc_zone_state(z, NUMA_MISS);
+ __inc_zone_state(preferred_zone, NUMA_FOREIGN);
+ }
+#endif
+}
+
+/*
* Allocate a page from the given zone. Use pcplists for order-0 allocations.
*/
static inline
struct page *buffered_rmqueue(struct zone *preferred_zone,
struct zone *zone, unsigned int order,
- gfp_t gfp_flags, int alloc_flags, int migratetype)
+ gfp_t gfp_flags, unsigned int alloc_flags,
+ int migratetype)
{
unsigned long flags;
struct page *page;
@@ -2359,21 +2594,24 @@
struct list_head *list;
local_irq_save(flags);
- pcp = &this_cpu_ptr(zone->pageset)->pcp;
- list = &pcp->lists[migratetype];
- if (list_empty(list)) {
- pcp->count += rmqueue_bulk(zone, 0,
- pcp->batch, list,
- migratetype, cold);
- if (unlikely(list_empty(list)))
- goto failed;
- }
+ do {
+ pcp = &this_cpu_ptr(zone->pageset)->pcp;
+ list = &pcp->lists[migratetype];
+ if (list_empty(list)) {
+ pcp->count += rmqueue_bulk(zone, 0,
+ pcp->batch, list,
+ migratetype, cold);
+ if (unlikely(list_empty(list)))
+ goto failed;
+ }
- if (cold)
- page = list_last_entry(list, struct page, lru);
- else
- page = list_first_entry(list, struct page, lru);
+ if (cold)
+ page = list_last_entry(list, struct page, lru);
+ else
+ page = list_first_entry(list, struct page, lru);
+ } while (page && check_new_pcp(page));
+ __dec_zone_state(zone, NR_ALLOC_BATCH);
list_del(&page->lru);
pcp->count--;
} else {
@@ -2384,22 +2622,24 @@
WARN_ON_ONCE((gfp_flags & __GFP_NOFAIL) && (order > 1));
spin_lock_irqsave(&zone->lock, flags);
- page = NULL;
- if (alloc_flags & ALLOC_HARDER) {
- page = __rmqueue_smallest(zone, order, MIGRATE_HIGHATOMIC);
- if (page)
- trace_mm_page_alloc_zone_locked(page, order, migratetype);
- }
- if (!page)
- page = __rmqueue(zone, order, migratetype);
+ do {
+ page = NULL;
+ if (alloc_flags & ALLOC_HARDER) {
+ page = __rmqueue_smallest(zone, order, MIGRATE_HIGHATOMIC);
+ if (page)
+ trace_mm_page_alloc_zone_locked(page, order, migratetype);
+ }
+ if (!page)
+ page = __rmqueue(zone, order, migratetype);
+ } while (page && check_new_pages(page, order));
spin_unlock(&zone->lock);
if (!page)
goto failed;
+ __mod_zone_page_state(zone, NR_ALLOC_BATCH, -(1 << order));
__mod_zone_freepage_state(zone, -(1 << order),
get_pcppage_migratetype(page));
}
- __mod_zone_page_state(zone, NR_ALLOC_BATCH, -(1 << order));
if (atomic_long_read(&zone->vm_stat[NR_ALLOC_BATCH]) <= 0 &&
!test_bit(ZONE_FAIR_DEPLETED, &zone->flags))
set_bit(ZONE_FAIR_DEPLETED, &zone->flags);
@@ -2500,13 +2740,13 @@
* one free page of a suitable size. Checking now avoids taking the zone lock
* to check in the allocation paths if no pages are free.
*/
-static bool __zone_watermark_ok(struct zone *z, unsigned int order,
- unsigned long mark, int classzone_idx, int alloc_flags,
- long free_pages)
+bool __zone_watermark_ok(struct zone *z, unsigned int order, unsigned long mark,
+ int classzone_idx, unsigned int alloc_flags,
+ long free_pages)
{
long min = mark;
int o;
- const int alloc_harder = (alloc_flags & ALLOC_HARDER);
+ const bool alloc_harder = (alloc_flags & ALLOC_HARDER);
/* free_pages may go negative - that's OK */
free_pages -= (1 << order) - 1;
@@ -2569,12 +2809,38 @@
}
bool zone_watermark_ok(struct zone *z, unsigned int order, unsigned long mark,
- int classzone_idx, int alloc_flags)
+ int classzone_idx, unsigned int alloc_flags)
{
return __zone_watermark_ok(z, order, mark, classzone_idx, alloc_flags,
zone_page_state(z, NR_FREE_PAGES));
}
+static inline bool zone_watermark_fast(struct zone *z, unsigned int order,
+ unsigned long mark, int classzone_idx, unsigned int alloc_flags)
+{
+ long free_pages = zone_page_state(z, NR_FREE_PAGES);
+ long cma_pages = 0;
+
+#ifdef CONFIG_CMA
+ /* If allocation can't use CMA areas don't use free CMA pages */
+ if (!(alloc_flags & ALLOC_CMA))
+ cma_pages = zone_page_state(z, NR_FREE_CMA_PAGES);
+#endif
+
+ /*
+ * Fast check for order-0 only. If this fails then the reserves
+ * need to be calculated. There is a corner case where the check
+ * passes but only the high-order atomic reserve are free. If
+ * the caller is !atomic then it'll uselessly search the free
+ * list. That corner case is then slower but it is harmless.
+ */
+ if (!order && (free_pages - cma_pages) > mark + z->lowmem_reserve[classzone_idx])
+ return true;
+
+ return __zone_watermark_ok(z, order, mark, classzone_idx, alloc_flags,
+ free_pages);
+}
+
bool zone_watermark_ok_safe(struct zone *z, unsigned int order,
unsigned long mark, int classzone_idx)
{
@@ -2630,27 +2896,24 @@
get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags,
const struct alloc_context *ac)
{
- struct zonelist *zonelist = ac->zonelist;
- struct zoneref *z;
- struct page *page = NULL;
+ struct zoneref *z = ac->preferred_zoneref;
struct zone *zone;
- int nr_fair_skipped = 0;
- bool zonelist_rescan;
+ bool fair_skipped = false;
+ bool apply_fair = (alloc_flags & ALLOC_FAIR);
zonelist_scan:
- zonelist_rescan = false;
-
/*
* Scan zonelist, looking for a zone with enough free.
* See also __cpuset_node_allowed() comment in kernel/cpuset.c.
*/
- for_each_zone_zonelist_nodemask(zone, z, zonelist, ac->high_zoneidx,
+ for_next_zone_zonelist_nodemask(zone, z, ac->zonelist, ac->high_zoneidx,
ac->nodemask) {
+ struct page *page;
unsigned long mark;
if (cpusets_enabled() &&
(alloc_flags & ALLOC_CPUSET) &&
- !cpuset_zone_allowed(zone, gfp_mask))
+ !__cpuset_zone_allowed(zone, gfp_mask))
continue;
/*
* Distribute pages in proportion to the individual
@@ -2658,13 +2921,16 @@
* page was allocated in should have no effect on the
* time the page has in memory before being reclaimed.
*/
- if (alloc_flags & ALLOC_FAIR) {
- if (!zone_local(ac->preferred_zone, zone))
- break;
+ if (apply_fair) {
if (test_bit(ZONE_FAIR_DEPLETED, &zone->flags)) {
- nr_fair_skipped++;
+ fair_skipped = true;
continue;
}
+ if (!zone_local(ac->preferred_zoneref->zone, zone)) {
+ if (fair_skipped)
+ goto reset_fair;
+ apply_fair = false;
+ }
}
/*
* When allocating a page cache page for writing, we
@@ -2696,8 +2962,8 @@
continue;
mark = zone->watermark[alloc_flags & ALLOC_WMARK_MASK];
- if (!zone_watermark_ok(zone, order, mark,
- ac->classzone_idx, alloc_flags)) {
+ if (!zone_watermark_fast(zone, order, mark,
+ ac_classzone_idx(ac), alloc_flags)) {
int ret;
/* Checked here to keep the fast path fast */
@@ -2706,7 +2972,7 @@
goto try_this_zone;
if (zone_reclaim_mode == 0 ||
- !zone_allows_reclaim(ac->preferred_zone, zone))
+ !zone_allows_reclaim(ac->preferred_zoneref->zone, zone))
continue;
ret = zone_reclaim(zone, gfp_mask, order);
@@ -2720,7 +2986,7 @@
default:
/* did we reclaim enough */
if (zone_watermark_ok(zone, order, mark,
- ac->classzone_idx, alloc_flags))
+ ac_classzone_idx(ac), alloc_flags))
goto try_this_zone;
continue;
@@ -2728,11 +2994,10 @@
}
try_this_zone:
- page = buffered_rmqueue(ac->preferred_zone, zone, order,
+ page = buffered_rmqueue(ac->preferred_zoneref->zone, zone, order,
gfp_mask, alloc_flags, ac->migratetype);
if (page) {
- if (prep_new_page(page, order, gfp_mask, alloc_flags))
- goto try_this_zone;
+ prep_new_page(page, order, gfp_mask, alloc_flags);
/*
* If this is a high-order atomic allocation then check
@@ -2753,18 +3018,13 @@
* include remote zones now, before entering the slowpath and waking
* kswapd: prefer spilling to a remote zone over swapping locally.
*/
- if (alloc_flags & ALLOC_FAIR) {
- alloc_flags &= ~ALLOC_FAIR;
- if (nr_fair_skipped) {
- zonelist_rescan = true;
- reset_alloc_batches(ac->preferred_zone);
- }
- if (nr_online_nodes > 1)
- zonelist_rescan = true;
- }
-
- if (zonelist_rescan)
+ if (fair_skipped) {
+reset_fair:
+ apply_fair = false;
+ fair_skipped = false;
+ reset_alloc_batches(ac->preferred_zoneref->zone);
goto zonelist_scan;
+ }
return NULL;
}
@@ -2872,22 +3132,18 @@
/* The OOM killer does not needlessly kill tasks for lowmem */
if (ac->high_zoneidx < ZONE_NORMAL)
goto out;
- /* The OOM killer does not compensate for IO-less reclaim */
- if (!(gfp_mask & __GFP_FS)) {
- /*
- * XXX: Page reclaim didn't yield anything,
- * and the OOM killer can't be invoked, but
- * keep looping as per tradition.
- *
- * But do not keep looping if oom_killer_disable()
- * was already called, for the system is trying to
- * enter a quiescent state during suspend.
- */
- *did_some_progress = !oom_killer_disabled;
- goto out;
- }
if (pm_suspended_storage())
goto out;
+ /*
+ * XXX: GFP_NOFS allocations should rather fail than rely on
+ * other request to make a forward progress.
+ * We are in an unfortunate situation where out_of_memory cannot
+ * do much for this context but let's try it to at least get
+ * access to memory reserved if the current task is killed (see
+ * out_of_memory). Once filesystems are ready to handle allocation
+ * failures more gracefully we should just bail out here.
+ */
+
/* The OOM killer may not free memory on a specific node */
if (gfp_mask & __GFP_THISNODE)
goto out;
@@ -2913,34 +3169,33 @@
return page;
}
+
+/*
+ * Maximum number of compaction retries wit a progress before OOM
+ * killer is consider as the only way to move forward.
+ */
+#define MAX_COMPACT_RETRIES 16
+
#ifdef CONFIG_COMPACTION
/* Try memory compaction for high-order allocations before reclaim */
static struct page *
__alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
- int alloc_flags, const struct alloc_context *ac,
- enum migrate_mode mode, int *contended_compaction,
- bool *deferred_compaction)
+ unsigned int alloc_flags, const struct alloc_context *ac,
+ enum migrate_mode mode, enum compact_result *compact_result)
{
- unsigned long compact_result;
struct page *page;
+ int contended_compaction;
if (!order)
return NULL;
current->flags |= PF_MEMALLOC;
- compact_result = try_to_compact_pages(gfp_mask, order, alloc_flags, ac,
- mode, contended_compaction);
+ *compact_result = try_to_compact_pages(gfp_mask, order, alloc_flags, ac,
+ mode, &contended_compaction);
current->flags &= ~PF_MEMALLOC;
- switch (compact_result) {
- case COMPACT_DEFERRED:
- *deferred_compaction = true;
- /* fall-through */
- case COMPACT_SKIPPED:
+ if (*compact_result <= COMPACT_INACTIVE)
return NULL;
- default:
- break;
- }
/*
* At least in one zone compaction wasn't deferred or skipped, so let's
@@ -2966,19 +3221,112 @@
*/
count_vm_event(COMPACTFAIL);
+ /*
+ * In all zones where compaction was attempted (and not
+ * deferred or skipped), lock contention has been detected.
+ * For THP allocation we do not want to disrupt the others
+ * so we fallback to base pages instead.
+ */
+ if (contended_compaction == COMPACT_CONTENDED_LOCK)
+ *compact_result = COMPACT_CONTENDED;
+
+ /*
+ * If compaction was aborted due to need_resched(), we do not
+ * want to further increase allocation latency, unless it is
+ * khugepaged trying to collapse.
+ */
+ if (contended_compaction == COMPACT_CONTENDED_SCHED
+ && !(current->flags & PF_KTHREAD))
+ *compact_result = COMPACT_CONTENDED;
+
cond_resched();
return NULL;
}
+
+static inline bool
+should_compact_retry(struct alloc_context *ac, int order, int alloc_flags,
+ enum compact_result compact_result, enum migrate_mode *migrate_mode,
+ int compaction_retries)
+{
+ int max_retries = MAX_COMPACT_RETRIES;
+
+ if (!order)
+ return false;
+
+ /*
+ * compaction considers all the zone as desperately out of memory
+ * so it doesn't really make much sense to retry except when the
+ * failure could be caused by weak migration mode.
+ */
+ if (compaction_failed(compact_result)) {
+ if (*migrate_mode == MIGRATE_ASYNC) {
+ *migrate_mode = MIGRATE_SYNC_LIGHT;
+ return true;
+ }
+ return false;
+ }
+
+ /*
+ * make sure the compaction wasn't deferred or didn't bail out early
+ * due to locks contention before we declare that we should give up.
+ * But do not retry if the given zonelist is not suitable for
+ * compaction.
+ */
+ if (compaction_withdrawn(compact_result))
+ return compaction_zonelist_suitable(ac, order, alloc_flags);
+
+ /*
+ * !costly requests are much more important than __GFP_REPEAT
+ * costly ones because they are de facto nofail and invoke OOM
+ * killer to move on while costly can fail and users are ready
+ * to cope with that. 1/4 retries is rather arbitrary but we
+ * would need much more detailed feedback from compaction to
+ * make a better decision.
+ */
+ if (order > PAGE_ALLOC_COSTLY_ORDER)
+ max_retries /= 4;
+ if (compaction_retries <= max_retries)
+ return true;
+
+ return false;
+}
#else
static inline struct page *
__alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
- int alloc_flags, const struct alloc_context *ac,
- enum migrate_mode mode, int *contended_compaction,
- bool *deferred_compaction)
+ unsigned int alloc_flags, const struct alloc_context *ac,
+ enum migrate_mode mode, enum compact_result *compact_result)
{
+ *compact_result = COMPACT_SKIPPED;
return NULL;
}
+
+static inline bool
+should_compact_retry(struct alloc_context *ac, unsigned int order, int alloc_flags,
+ enum compact_result compact_result,
+ enum migrate_mode *migrate_mode,
+ int compaction_retries)
+{
+ struct zone *zone;
+ struct zoneref *z;
+
+ if (!order || order > PAGE_ALLOC_COSTLY_ORDER)
+ return false;
+
+ /*
+ * There are setups with compaction disabled which would prefer to loop
+ * inside the allocator rather than hit the oom killer prematurely.
+ * Let's give them a good hope and keep retrying while the order-0
+ * watermarks are OK.
+ */
+ for_each_zone_zonelist_nodemask(zone, z, ac->zonelist, ac->high_zoneidx,
+ ac->nodemask) {
+ if (zone_watermark_ok(zone, 0, min_wmark_pages(zone),
+ ac_classzone_idx(ac), alloc_flags))
+ return true;
+ }
+ return false;
+}
#endif /* CONFIG_COMPACTION */
/* Perform direct synchronous page reclaim */
@@ -3013,7 +3361,7 @@
/* The really slow allocator path where we enter direct reclaim */
static inline struct page *
__alloc_pages_direct_reclaim(gfp_t gfp_mask, unsigned int order,
- int alloc_flags, const struct alloc_context *ac,
+ unsigned int alloc_flags, const struct alloc_context *ac,
unsigned long *did_some_progress)
{
struct page *page = NULL;
@@ -3049,13 +3397,13 @@
for_each_zone_zonelist_nodemask(zone, z, ac->zonelist,
ac->high_zoneidx, ac->nodemask)
- wakeup_kswapd(zone, order, zone_idx(ac->preferred_zone));
+ wakeup_kswapd(zone, order, ac_classzone_idx(ac));
}
-static inline int
+static inline unsigned int
gfp_to_alloc_flags(gfp_t gfp_mask)
{
- int alloc_flags = ALLOC_WMARK_MIN | ALLOC_CPUSET;
+ unsigned int alloc_flags = ALLOC_WMARK_MIN | ALLOC_CPUSET;
/* __GFP_HIGH is assumed to be the same as ALLOC_HIGH to save a branch. */
BUILD_BUG_ON(__GFP_HIGH != (__force gfp_t) ALLOC_HIGH);
@@ -3110,18 +3458,113 @@
return (gfp_mask & (GFP_TRANSHUGE | __GFP_KSWAPD_RECLAIM)) == GFP_TRANSHUGE;
}
+/*
+ * Maximum number of reclaim retries without any progress before OOM killer
+ * is consider as the only way to move forward.
+ */
+#define MAX_RECLAIM_RETRIES 16
+
+/*
+ * Checks whether it makes sense to retry the reclaim to make a forward progress
+ * for the given allocation request.
+ * The reclaim feedback represented by did_some_progress (any progress during
+ * the last reclaim round) and no_progress_loops (number of reclaim rounds without
+ * any progress in a row) is considered as well as the reclaimable pages on the
+ * applicable zone list (with a backoff mechanism which is a function of
+ * no_progress_loops).
+ *
+ * Returns true if a retry is viable or false to enter the oom path.
+ */
+static inline bool
+should_reclaim_retry(gfp_t gfp_mask, unsigned order,
+ struct alloc_context *ac, int alloc_flags,
+ bool did_some_progress, int no_progress_loops)
+{
+ struct zone *zone;
+ struct zoneref *z;
+
+ /*
+ * Make sure we converge to OOM if we cannot make any progress
+ * several times in the row.
+ */
+ if (no_progress_loops > MAX_RECLAIM_RETRIES)
+ return false;
+
+ /*
+ * Keep reclaiming pages while there is a chance this will lead somewhere.
+ * If none of the target zones can satisfy our allocation request even
+ * if all reclaimable pages are considered then we are screwed and have
+ * to go OOM.
+ */
+ for_each_zone_zonelist_nodemask(zone, z, ac->zonelist, ac->high_zoneidx,
+ ac->nodemask) {
+ unsigned long available;
+ unsigned long reclaimable;
+
+ available = reclaimable = zone_reclaimable_pages(zone);
+ available -= DIV_ROUND_UP(no_progress_loops * available,
+ MAX_RECLAIM_RETRIES);
+ available += zone_page_state_snapshot(zone, NR_FREE_PAGES);
+
+ /*
+ * Would the allocation succeed if we reclaimed the whole
+ * available?
+ */
+ if (__zone_watermark_ok(zone, order, min_wmark_pages(zone),
+ ac_classzone_idx(ac), alloc_flags, available)) {
+ /*
+ * If we didn't make any progress and have a lot of
+ * dirty + writeback pages then we should wait for
+ * an IO to complete to slow down the reclaim and
+ * prevent from pre mature OOM
+ */
+ if (!did_some_progress) {
+ unsigned long writeback;
+ unsigned long dirty;
+
+ writeback = zone_page_state_snapshot(zone,
+ NR_WRITEBACK);
+ dirty = zone_page_state_snapshot(zone, NR_FILE_DIRTY);
+
+ if (2*(writeback + dirty) > reclaimable) {
+ congestion_wait(BLK_RW_ASYNC, HZ/10);
+ return true;
+ }
+ }
+
+ /*
+ * Memory allocation/reclaim might be called from a WQ
+ * context and the current implementation of the WQ
+ * concurrency control doesn't recognize that
+ * a particular WQ is congested if the worker thread is
+ * looping without ever sleeping. Therefore we have to
+ * do a short sleep here rather than calling
+ * cond_resched().
+ */
+ if (current->flags & PF_WQ_WORKER)
+ schedule_timeout_uninterruptible(1);
+ else
+ cond_resched();
+
+ return true;
+ }
+ }
+
+ return false;
+}
+
static inline struct page *
__alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
struct alloc_context *ac)
{
bool can_direct_reclaim = gfp_mask & __GFP_DIRECT_RECLAIM;
struct page *page = NULL;
- int alloc_flags;
- unsigned long pages_reclaimed = 0;
+ unsigned int alloc_flags;
unsigned long did_some_progress;
enum migrate_mode migration_mode = MIGRATE_ASYNC;
- bool deferred_compaction = false;
- int contended_compaction = COMPACT_CONTENDED_NONE;
+ enum compact_result compact_result;
+ int compaction_retries = 0;
+ int no_progress_loops = 0;
/*
* In the slowpath, we sanity check order to avoid ever trying to
@@ -3153,17 +3596,6 @@
*/
alloc_flags = gfp_to_alloc_flags(gfp_mask);
- /*
- * Find the true preferred zone if the allocation is unconstrained by
- * cpusets.
- */
- if (!(alloc_flags & ALLOC_CPUSET) && !ac->nodemask) {
- struct zoneref *preferred_zoneref;
- preferred_zoneref = first_zones_zonelist(ac->zonelist,
- ac->high_zoneidx, NULL, &ac->preferred_zone);
- ac->classzone_idx = zonelist_zone_idx(preferred_zoneref);
- }
-
/* This is the last chance, in general, before the goto nopage. */
page = get_page_from_freelist(gfp_mask, order,
alloc_flags & ~ALLOC_NO_WATERMARKS, ac);
@@ -3219,8 +3651,7 @@
*/
page = __alloc_pages_direct_compact(gfp_mask, order, alloc_flags, ac,
migration_mode,
- &contended_compaction,
- &deferred_compaction);
+ &compact_result);
if (page)
goto got_pg;
@@ -3233,35 +3664,19 @@
* to heavily disrupt the system, so we fail the allocation
* instead of entering direct reclaim.
*/
- if (deferred_compaction)
+ if (compact_result == COMPACT_DEFERRED)
goto nopage;
/*
- * In all zones where compaction was attempted (and not
- * deferred or skipped), lock contention has been detected.
- * For THP allocation we do not want to disrupt the others
- * so we fallback to base pages instead.
+ * Compaction is contended so rather back off than cause
+ * excessive stalls.
*/
- if (contended_compaction == COMPACT_CONTENDED_LOCK)
- goto nopage;
-
- /*
- * If compaction was aborted due to need_resched(), we do not
- * want to further increase allocation latency, unless it is
- * khugepaged trying to collapse.
- */
- if (contended_compaction == COMPACT_CONTENDED_SCHED
- && !(current->flags & PF_KTHREAD))
+ if(compact_result == COMPACT_CONTENDED)
goto nopage;
}
- /*
- * It can become very expensive to allocate transparent hugepages at
- * fault, so use asynchronous memory compaction for THP unless it is
- * khugepaged trying to collapse.
- */
- if (!is_thp_gfp_mask(gfp_mask) || (current->flags & PF_KTHREAD))
- migration_mode = MIGRATE_SYNC_LIGHT;
+ if (order && compaction_made_progress(compact_result))
+ compaction_retries++;
/* Try direct reclaim and then allocating */
page = __alloc_pages_direct_reclaim(gfp_mask, order, alloc_flags, ac,
@@ -3273,14 +3688,38 @@
if (gfp_mask & __GFP_NORETRY)
goto noretry;
- /* Keep reclaiming pages as long as there is reasonable progress */
- pages_reclaimed += did_some_progress;
- if ((did_some_progress && order <= PAGE_ALLOC_COSTLY_ORDER) ||
- ((gfp_mask & __GFP_REPEAT) && pages_reclaimed < (1 << order))) {
- /* Wait for some write requests to complete then retry */
- wait_iff_congested(ac->preferred_zone, BLK_RW_ASYNC, HZ/50);
+ /*
+ * Do not retry costly high order allocations unless they are
+ * __GFP_REPEAT
+ */
+ if (order > PAGE_ALLOC_COSTLY_ORDER && !(gfp_mask & __GFP_REPEAT))
+ goto noretry;
+
+ /*
+ * Costly allocations might have made a progress but this doesn't mean
+ * their order will become available due to high fragmentation so
+ * always increment the no progress counter for them
+ */
+ if (did_some_progress && order <= PAGE_ALLOC_COSTLY_ORDER)
+ no_progress_loops = 0;
+ else
+ no_progress_loops++;
+
+ if (should_reclaim_retry(gfp_mask, order, ac, alloc_flags,
+ did_some_progress > 0, no_progress_loops))
goto retry;
- }
+
+ /*
+ * It doesn't make any sense to retry for the compaction if the order-0
+ * reclaim is not able to make any progress because the current
+ * implementation of the compaction depends on the sufficient amount
+ * of free memory (see __compaction_suitable)
+ */
+ if (did_some_progress > 0 &&
+ should_compact_retry(ac, order, alloc_flags,
+ compact_result, &migration_mode,
+ compaction_retries))
+ goto retry;
/* Reclaim has failed us, start killing things */
page = __alloc_pages_may_oom(gfp_mask, order, ac, &did_some_progress);
@@ -3288,19 +3727,28 @@
goto got_pg;
/* Retry as long as the OOM killer is making progress */
- if (did_some_progress)
+ if (did_some_progress) {
+ no_progress_loops = 0;
goto retry;
+ }
noretry:
/*
- * High-order allocations do not necessarily loop after
- * direct reclaim and reclaim/compaction depends on compaction
- * being called after reclaim so call directly if necessary
+ * High-order allocations do not necessarily loop after direct reclaim
+ * and reclaim/compaction depends on compaction being called after
+ * reclaim so call directly if necessary.
+ * It can become very expensive to allocate transparent hugepages at
+ * fault, so use asynchronous memory compaction for THP unless it is
+ * khugepaged trying to collapse. All other requests should tolerate
+ * at least light sync migration.
*/
+ if (is_thp_gfp_mask(gfp_mask) && !(current->flags & PF_KTHREAD))
+ migration_mode = MIGRATE_ASYNC;
+ else
+ migration_mode = MIGRATE_SYNC_LIGHT;
page = __alloc_pages_direct_compact(gfp_mask, order, alloc_flags,
ac, migration_mode,
- &contended_compaction,
- &deferred_compaction);
+ &compact_result);
if (page)
goto got_pg;
nopage:
@@ -3316,17 +3764,24 @@
__alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order,
struct zonelist *zonelist, nodemask_t *nodemask)
{
- struct zoneref *preferred_zoneref;
- struct page *page = NULL;
+ struct page *page;
unsigned int cpuset_mems_cookie;
- int alloc_flags = ALLOC_WMARK_LOW|ALLOC_CPUSET|ALLOC_FAIR;
- gfp_t alloc_mask; /* The gfp_t that was actually used for allocation */
+ unsigned int alloc_flags = ALLOC_WMARK_LOW|ALLOC_FAIR;
+ gfp_t alloc_mask = gfp_mask; /* The gfp_t that was actually used for allocation */
struct alloc_context ac = {
.high_zoneidx = gfp_zone(gfp_mask),
+ .zonelist = zonelist,
.nodemask = nodemask,
.migratetype = gfpflags_to_migratetype(gfp_mask),
};
+ if (cpusets_enabled()) {
+ alloc_mask |= __GFP_HARDWALL;
+ alloc_flags |= ALLOC_CPUSET;
+ if (!ac.nodemask)
+ ac.nodemask = &cpuset_current_mems_allowed;
+ }
+
gfp_mask &= gfp_allowed_mask;
lockdep_trace_alloc(gfp_mask);
@@ -3350,49 +3805,54 @@
retry_cpuset:
cpuset_mems_cookie = read_mems_allowed_begin();
- /* We set it here, as __alloc_pages_slowpath might have changed it */
- ac.zonelist = zonelist;
-
/* Dirty zone balancing only done in the fast path */
ac.spread_dirty_pages = (gfp_mask & __GFP_WRITE);
/* The preferred zone is used for statistics later */
- preferred_zoneref = first_zones_zonelist(ac.zonelist, ac.high_zoneidx,
- ac.nodemask ? : &cpuset_current_mems_allowed,
- &ac.preferred_zone);
- if (!ac.preferred_zone)
- goto out;
- ac.classzone_idx = zonelist_zone_idx(preferred_zoneref);
-
- /* First allocation attempt */
- alloc_mask = gfp_mask|__GFP_HARDWALL;
- page = get_page_from_freelist(alloc_mask, order, alloc_flags, &ac);
- if (unlikely(!page)) {
- /*
- * Runtime PM, block IO and its error handling path
- * can deadlock because I/O on the device might not
- * complete.
- */
- alloc_mask = memalloc_noio_flags(gfp_mask);
- ac.spread_dirty_pages = false;
-
- page = __alloc_pages_slowpath(alloc_mask, order, &ac);
+ ac.preferred_zoneref = first_zones_zonelist(ac.zonelist,
+ ac.high_zoneidx, ac.nodemask);
+ if (!ac.preferred_zoneref) {
+ page = NULL;
+ goto no_zone;
}
- if (kmemcheck_enabled && page)
- kmemcheck_pagealloc_alloc(page, order, gfp_mask);
+ /* First allocation attempt */
+ page = get_page_from_freelist(alloc_mask, order, alloc_flags, &ac);
+ if (likely(page))
+ goto out;
- trace_mm_page_alloc(page, order, alloc_mask, ac.migratetype);
+ /*
+ * Runtime PM, block IO and its error handling path can deadlock
+ * because I/O on the device might not complete.
+ */
+ alloc_mask = memalloc_noio_flags(gfp_mask);
+ ac.spread_dirty_pages = false;
-out:
+ /*
+ * Restore the original nodemask if it was potentially replaced with
+ * &cpuset_current_mems_allowed to optimize the fast-path attempt.
+ */
+ if (cpusets_enabled())
+ ac.nodemask = nodemask;
+ page = __alloc_pages_slowpath(alloc_mask, order, &ac);
+
+no_zone:
/*
* When updating a task's mems_allowed, it is possible to race with
* parallel threads in such a way that an allocation can fail while
* the mask is being updated. If a page allocation is about to fail,
* check if the cpuset changed during allocation and if so, retry.
*/
- if (unlikely(!page && read_mems_allowed_retry(cpuset_mems_cookie)))
+ if (unlikely(!page && read_mems_allowed_retry(cpuset_mems_cookie))) {
+ alloc_mask = gfp_mask;
goto retry_cpuset;
+ }
+
+out:
+ if (kmemcheck_enabled && page)
+ kmemcheck_pagealloc_alloc(page, order, gfp_mask);
+
+ trace_mm_page_alloc(page, order, alloc_mask, ac.migratetype);
return page;
}
@@ -3790,6 +4250,8 @@
{
int zone_type; /* needs to be signed */
unsigned long managed_pages = 0;
+ unsigned long managed_highpages = 0;
+ unsigned long free_highpages = 0;
pg_data_t *pgdat = NODE_DATA(nid);
for (zone_type = 0; zone_type < MAX_NR_ZONES; zone_type++)
@@ -3798,12 +4260,19 @@
val->sharedram = node_page_state(nid, NR_SHMEM);
val->freeram = node_page_state(nid, NR_FREE_PAGES);
#ifdef CONFIG_HIGHMEM
- val->totalhigh = pgdat->node_zones[ZONE_HIGHMEM].managed_pages;
- val->freehigh = zone_page_state(&pgdat->node_zones[ZONE_HIGHMEM],
- NR_FREE_PAGES);
+ for (zone_type = 0; zone_type < MAX_NR_ZONES; zone_type++) {
+ struct zone *zone = &pgdat->node_zones[zone_type];
+
+ if (is_highmem(zone)) {
+ managed_highpages += zone->managed_pages;
+ free_highpages += zone_page_state(zone, NR_FREE_PAGES);
+ }
+ }
+ val->totalhigh = managed_highpages;
+ val->freehigh = free_highpages;
#else
- val->totalhigh = 0;
- val->freehigh = 0;
+ val->totalhigh = managed_highpages;
+ val->freehigh = free_highpages;
#endif
val->mem_unit = PAGE_SIZE;
}
@@ -4390,13 +4859,12 @@
*/
int local_memory_node(int node)
{
- struct zone *zone;
+ struct zoneref *z;
- (void)first_zones_zonelist(node_zonelist(node, GFP_KERNEL),
+ z = first_zones_zonelist(node_zonelist(node, GFP_KERNEL),
gfp_zone(GFP_KERNEL),
- NULL,
- &zone);
- return zone->node;
+ NULL);
+ return z->zone->node;
}
#endif
@@ -6395,49 +6863,6 @@
}
/*
- * The inactive anon list should be small enough that the VM never has to
- * do too much work, but large enough that each inactive page has a chance
- * to be referenced again before it is swapped out.
- *
- * The inactive_anon ratio is the target ratio of ACTIVE_ANON to
- * INACTIVE_ANON pages on this zone's LRU, maintained by the
- * pageout code. A zone->inactive_ratio of 3 means 3:1 or 25% of
- * the anonymous pages are kept on the inactive list.
- *
- * total target max
- * memory ratio inactive anon
- * -------------------------------------
- * 10MB 1 5MB
- * 100MB 1 50MB
- * 1GB 3 250MB
- * 10GB 10 0.9GB
- * 100GB 31 3GB
- * 1TB 101 10GB
- * 10TB 320 32GB
- */
-static void __meminit calculate_zone_inactive_ratio(struct zone *zone)
-{
- unsigned int gb, ratio;
-
- /* Zone size in gigabytes */
- gb = zone->managed_pages >> (30 - PAGE_SHIFT);
- if (gb)
- ratio = int_sqrt(10 * gb);
- else
- ratio = 1;
-
- zone->inactive_ratio = ratio;
-}
-
-static void __meminit setup_per_zone_inactive_ratio(void)
-{
- struct zone *zone;
-
- for_each_zone(zone)
- calculate_zone_inactive_ratio(zone);
-}
-
-/*
* Initialise min_free_kbytes.
*
* For small machines we want it small (128k min). For large machines
@@ -6482,7 +6907,6 @@
setup_per_zone_wmarks();
refresh_zone_stat_thresholds();
setup_per_zone_lowmem_reserve();
- setup_per_zone_inactive_ratio();
return 0;
}
core_initcall(init_per_zone_wmark_min)
@@ -6725,98 +7149,6 @@
return table;
}
-/* Return a pointer to the bitmap storing bits affecting a block of pages */
-static inline unsigned long *get_pageblock_bitmap(struct zone *zone,
- unsigned long pfn)
-{
-#ifdef CONFIG_SPARSEMEM
- return __pfn_to_section(pfn)->pageblock_flags;
-#else
- return zone->pageblock_flags;
-#endif /* CONFIG_SPARSEMEM */
-}
-
-static inline int pfn_to_bitidx(struct zone *zone, unsigned long pfn)
-{
-#ifdef CONFIG_SPARSEMEM
- pfn &= (PAGES_PER_SECTION-1);
- return (pfn >> pageblock_order) * NR_PAGEBLOCK_BITS;
-#else
- pfn = pfn - round_down(zone->zone_start_pfn, pageblock_nr_pages);
- return (pfn >> pageblock_order) * NR_PAGEBLOCK_BITS;
-#endif /* CONFIG_SPARSEMEM */
-}
-
-/**
- * get_pfnblock_flags_mask - Return the requested group of flags for the pageblock_nr_pages block of pages
- * @page: The page within the block of interest
- * @pfn: The target page frame number
- * @end_bitidx: The last bit of interest to retrieve
- * @mask: mask of bits that the caller is interested in
- *
- * Return: pageblock_bits flags
- */
-unsigned long get_pfnblock_flags_mask(struct page *page, unsigned long pfn,
- unsigned long end_bitidx,
- unsigned long mask)
-{
- struct zone *zone;
- unsigned long *bitmap;
- unsigned long bitidx, word_bitidx;
- unsigned long word;
-
- zone = page_zone(page);
- bitmap = get_pageblock_bitmap(zone, pfn);
- bitidx = pfn_to_bitidx(zone, pfn);
- word_bitidx = bitidx / BITS_PER_LONG;
- bitidx &= (BITS_PER_LONG-1);
-
- word = bitmap[word_bitidx];
- bitidx += end_bitidx;
- return (word >> (BITS_PER_LONG - bitidx - 1)) & mask;
-}
-
-/**
- * set_pfnblock_flags_mask - Set the requested group of flags for a pageblock_nr_pages block of pages
- * @page: The page within the block of interest
- * @flags: The flags to set
- * @pfn: The target page frame number
- * @end_bitidx: The last bit of interest
- * @mask: mask of bits that the caller is interested in
- */
-void set_pfnblock_flags_mask(struct page *page, unsigned long flags,
- unsigned long pfn,
- unsigned long end_bitidx,
- unsigned long mask)
-{
- struct zone *zone;
- unsigned long *bitmap;
- unsigned long bitidx, word_bitidx;
- unsigned long old_word, word;
-
- BUILD_BUG_ON(NR_PAGEBLOCK_BITS != 4);
-
- zone = page_zone(page);
- bitmap = get_pageblock_bitmap(zone, pfn);
- bitidx = pfn_to_bitidx(zone, pfn);
- word_bitidx = bitidx / BITS_PER_LONG;
- bitidx &= (BITS_PER_LONG-1);
-
- VM_BUG_ON_PAGE(!zone_spans_pfn(zone, pfn), page);
-
- bitidx += end_bitidx;
- mask <<= (BITS_PER_LONG - bitidx - 1);
- flags <<= (BITS_PER_LONG - bitidx - 1);
-
- word = READ_ONCE(bitmap[word_bitidx]);
- for (;;) {
- old_word = cmpxchg(&bitmap[word_bitidx], word, (word & ~mask) | flags);
- if (word == old_word)
- break;
- word = old_word;
- }
-}
-
/*
* This function checks whether pageblock includes unmovable pages or not.
* If @count is not zero, it is okay to include less @count unmovable pages
@@ -6864,7 +7196,7 @@
* We can't use page_count without pin a page
* because another CPU can free compound page.
* This check already skips compound tails of THP
- * because their page->_count is zero at all time.
+ * because their page->_refcount is zero at all time.
*/
if (!page_ref_count(page)) {
if (PageBuddy(page))
@@ -7177,7 +7509,8 @@
#ifdef CONFIG_MEMORY_HOTREMOVE
/*
- * All pages in the range must be isolated before calling this.
+ * All pages in the range must be in a single zone and isolated
+ * before calling this.
*/
void
__offline_isolated_pages(unsigned long start_pfn, unsigned long end_pfn)
diff --git a/mm/page_isolation.c b/mm/page_isolation.c
index c4f5682..612122b 100644
--- a/mm/page_isolation.c
+++ b/mm/page_isolation.c
@@ -246,6 +246,7 @@
return pfn;
}
+/* Caller should ensure that requested range is in a single zone */
int test_pages_isolated(unsigned long start_pfn, unsigned long end_pfn,
bool skip_hwpoisoned_pages)
{
@@ -288,13 +289,10 @@
* accordance with memory policy of the user process if possible. For
* now as a simple work-around, we use the next node for destination.
*/
- if (PageHuge(page)) {
- int node = next_online_node(page_to_nid(page));
- if (node == MAX_NUMNODES)
- node = first_online_node;
+ if (PageHuge(page))
return alloc_huge_page_node(page_hstate(compound_head(page)),
- node);
- }
+ next_node_in(page_to_nid(page),
+ node_online_map));
if (PageHighMem(page))
gfp_mask |= __GFP_HIGHMEM;
diff --git a/mm/page_owner.c b/mm/page_owner.c
index ac3d8d1..792b56d 100644
--- a/mm/page_owner.c
+++ b/mm/page_owner.c
@@ -143,7 +143,7 @@
goto err;
/* Print information relevant to grouping pages by mobility */
- pageblock_mt = get_pfnblock_migratetype(page, pfn);
+ pageblock_mt = get_pageblock_migratetype(page);
page_mt = gfpflags_to_migratetype(page_ext->gfp_mask);
ret += snprintf(kbuf + ret, count - ret,
"PFN %lu type %s Block %lu type %s Flags %#lx(%pGp)\n",
@@ -301,6 +301,9 @@
page = pfn_to_page(pfn);
+ if (page_zone(page) != zone)
+ continue;
+
/*
* We are safe to check buddy flag and order, because
* this is init stage and only single thread runs.
diff --git a/mm/page_poison.c b/mm/page_poison.c
index 479e7ea..1eae5fa 100644
--- a/mm/page_poison.c
+++ b/mm/page_poison.c
@@ -13,13 +13,7 @@
{
if (!buf)
return -EINVAL;
-
- if (strcmp(buf, "on") == 0)
- want_page_poisoning = true;
- else if (strcmp(buf, "off") == 0)
- want_page_poisoning = false;
-
- return 0;
+ return strtobool(buf, &want_page_poisoning);
}
early_param("page_poison", early_page_poison_param);
diff --git a/mm/rmap.c b/mm/rmap.c
index 307b555..8a83993 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -409,7 +409,7 @@
list_for_each_entry_safe(avc, next, &vma->anon_vma_chain, same_vma) {
struct anon_vma *anon_vma = avc->anon_vma;
- BUG_ON(anon_vma->degree);
+ VM_WARN_ON(anon_vma->degree);
put_anon_vma(anon_vma);
list_del(&avc->same_vma);
@@ -1249,7 +1249,7 @@
int nr = compound ? hpage_nr_pages(page) : 1;
VM_BUG_ON_VMA(address < vma->vm_start || address >= vma->vm_end, vma);
- SetPageSwapBacked(page);
+ __SetPageSwapBacked(page);
if (compound) {
VM_BUG_ON_PAGE(!PageTransHuge(page), page);
/* increment count (starts at -1) */
diff --git a/mm/shmem.c b/mm/shmem.c
index e684a91..e418a99 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -101,7 +101,6 @@
enum sgp_type {
SGP_READ, /* don't exceed i_size, don't allocate page */
SGP_CACHE, /* don't exceed i_size, may allocate page */
- SGP_DIRTY, /* like SGP_CACHE, but set new page dirty */
SGP_WRITE, /* may exceed i_size, may allocate !Uptodate page */
SGP_FALLOC, /* like SGP_WRITE, but make existing page Uptodate */
};
@@ -122,13 +121,14 @@
static int shmem_replace_page(struct page **pagep, gfp_t gfp,
struct shmem_inode_info *info, pgoff_t index);
static int shmem_getpage_gfp(struct inode *inode, pgoff_t index,
- struct page **pagep, enum sgp_type sgp, gfp_t gfp, int *fault_type);
+ struct page **pagep, enum sgp_type sgp,
+ gfp_t gfp, struct mm_struct *fault_mm, int *fault_type);
static inline int shmem_getpage(struct inode *inode, pgoff_t index,
- struct page **pagep, enum sgp_type sgp, int *fault_type)
+ struct page **pagep, enum sgp_type sgp)
{
return shmem_getpage_gfp(inode, index, pagep, sgp,
- mapping_gfp_mask(inode->i_mapping), fault_type);
+ mapping_gfp_mask(inode->i_mapping), NULL, NULL);
}
static inline struct shmem_sb_info *SHMEM_SB(struct super_block *sb)
@@ -169,7 +169,7 @@
/*
* ... whereas tmpfs objects are accounted incrementally as
- * pages are allocated, in order to allow huge sparse files.
+ * pages are allocated, in order to allow large sparse files.
* shmem_getpage reports shmem_acct_block failure as -ENOSPC not -ENOMEM,
* so that a failure on a sparse tmpfs mapping will give SIGBUS not OOM.
*/
@@ -528,7 +528,7 @@
if (partial_start) {
struct page *page = NULL;
- shmem_getpage(inode, start - 1, &page, SGP_READ, NULL);
+ shmem_getpage(inode, start - 1, &page, SGP_READ);
if (page) {
unsigned int top = PAGE_SIZE;
if (start > end) {
@@ -543,7 +543,7 @@
}
if (partial_end) {
struct page *page = NULL;
- shmem_getpage(inode, end, &page, SGP_READ, NULL);
+ shmem_getpage(inode, end, &page, SGP_READ);
if (page) {
zero_user_segment(page, 0, partial_end);
set_page_dirty(page);
@@ -947,8 +947,7 @@
return 0;
}
-#ifdef CONFIG_NUMA
-#ifdef CONFIG_TMPFS
+#if defined(CONFIG_NUMA) && defined(CONFIG_TMPFS)
static void shmem_show_mpol(struct seq_file *seq, struct mempolicy *mpol)
{
char buffer[64];
@@ -972,7 +971,18 @@
}
return mpol;
}
-#endif /* CONFIG_TMPFS */
+#else /* !CONFIG_NUMA || !CONFIG_TMPFS */
+static inline void shmem_show_mpol(struct seq_file *seq, struct mempolicy *mpol)
+{
+}
+static inline struct mempolicy *shmem_get_sbmpol(struct shmem_sb_info *sbinfo)
+{
+ return NULL;
+}
+#endif /* CONFIG_NUMA && CONFIG_TMPFS */
+#ifndef CONFIG_NUMA
+#define vm_policy vm_private_data
+#endif
static struct page *shmem_swapin(swp_entry_t swap, gfp_t gfp,
struct shmem_inode_info *info, pgoff_t index)
@@ -1008,39 +1018,17 @@
pvma.vm_ops = NULL;
pvma.vm_policy = mpol_shared_policy_lookup(&info->policy, index);
- page = alloc_page_vma(gfp, &pvma, 0);
+ page = alloc_pages_vma(gfp, 0, &pvma, 0, numa_node_id(), false);
+ if (page) {
+ __SetPageLocked(page);
+ __SetPageSwapBacked(page);
+ }
/* Drop reference taken by mpol_shared_policy_lookup() */
mpol_cond_put(pvma.vm_policy);
return page;
}
-#else /* !CONFIG_NUMA */
-#ifdef CONFIG_TMPFS
-static inline void shmem_show_mpol(struct seq_file *seq, struct mempolicy *mpol)
-{
-}
-#endif /* CONFIG_TMPFS */
-
-static inline struct page *shmem_swapin(swp_entry_t swap, gfp_t gfp,
- struct shmem_inode_info *info, pgoff_t index)
-{
- return swapin_readahead(swap, gfp, NULL, 0);
-}
-
-static inline struct page *shmem_alloc_page(gfp_t gfp,
- struct shmem_inode_info *info, pgoff_t index)
-{
- return alloc_page(gfp);
-}
-#endif /* CONFIG_NUMA */
-
-#if !defined(CONFIG_NUMA) || !defined(CONFIG_TMPFS)
-static inline struct mempolicy *shmem_get_sbmpol(struct shmem_sb_info *sbinfo)
-{
- return NULL;
-}
-#endif
/*
* When a page is moved from swapcache to shmem filecache (either by the
@@ -1084,9 +1072,7 @@
copy_highpage(newpage, oldpage);
flush_dcache_page(newpage);
- __SetPageLocked(newpage);
SetPageUptodate(newpage);
- SetPageSwapBacked(newpage);
set_page_private(newpage, swap_index);
SetPageSwapCache(newpage);
@@ -1130,14 +1116,19 @@
*
* If we allocate a new one we do not mark it dirty. That's up to the
* vm. If we swap it in we mark it dirty since we also free the swap
- * entry since a page cannot live in both the swap and page cache
+ * entry since a page cannot live in both the swap and page cache.
+ *
+ * fault_mm and fault_type are only supplied by shmem_fault:
+ * otherwise they are NULL.
*/
static int shmem_getpage_gfp(struct inode *inode, pgoff_t index,
- struct page **pagep, enum sgp_type sgp, gfp_t gfp, int *fault_type)
+ struct page **pagep, enum sgp_type sgp, gfp_t gfp,
+ struct mm_struct *fault_mm, int *fault_type)
{
struct address_space *mapping = inode->i_mapping;
struct shmem_inode_info *info;
struct shmem_sb_info *sbinfo;
+ struct mm_struct *charge_mm;
struct mem_cgroup *memcg;
struct page *page;
swp_entry_t swap;
@@ -1155,7 +1146,7 @@
page = NULL;
}
- if (sgp != SGP_WRITE && sgp != SGP_FALLOC &&
+ if (sgp <= SGP_CACHE &&
((loff_t)index << PAGE_SHIFT) >= i_size_read(inode)) {
error = -EINVAL;
goto unlock;
@@ -1183,14 +1174,19 @@
*/
info = SHMEM_I(inode);
sbinfo = SHMEM_SB(inode->i_sb);
+ charge_mm = fault_mm ? : current->mm;
if (swap.val) {
/* Look it up and read it in.. */
page = lookup_swap_cache(swap);
if (!page) {
- /* here we actually do the io */
- if (fault_type)
+ /* Or update major stats only when swapin succeeds?? */
+ if (fault_type) {
*fault_type |= VM_FAULT_MAJOR;
+ count_vm_event(PGMAJFAULT);
+ mem_cgroup_count_vm_event(fault_mm, PGMAJFAULT);
+ }
+ /* Here we actually start the io */
page = shmem_swapin(swap, gfp, info, index);
if (!page) {
error = -ENOMEM;
@@ -1217,7 +1213,7 @@
goto failed;
}
- error = mem_cgroup_try_charge(page, current->mm, gfp, &memcg,
+ error = mem_cgroup_try_charge(page, charge_mm, gfp, &memcg,
false);
if (!error) {
error = shmem_add_to_page_cache(page, mapping, index,
@@ -1275,13 +1271,10 @@
error = -ENOMEM;
goto decused;
}
-
- __SetPageSwapBacked(page);
- __SetPageLocked(page);
if (sgp == SGP_WRITE)
__SetPageReferenced(page);
- error = mem_cgroup_try_charge(page, current->mm, gfp, &memcg,
+ error = mem_cgroup_try_charge(page, charge_mm, gfp, &memcg,
false);
if (error)
goto decused;
@@ -1321,12 +1314,10 @@
flush_dcache_page(page);
SetPageUptodate(page);
}
- if (sgp == SGP_DIRTY)
- set_page_dirty(page);
}
/* Perhaps the file has been truncated since we checked */
- if (sgp != SGP_WRITE && sgp != SGP_FALLOC &&
+ if (sgp <= SGP_CACHE &&
((loff_t)index << PAGE_SHIFT) >= i_size_read(inode)) {
if (alloced) {
ClearPageDirty(page);
@@ -1372,6 +1363,7 @@
static int shmem_fault(struct vm_area_struct *vma, struct vm_fault *vmf)
{
struct inode *inode = file_inode(vma->vm_file);
+ gfp_t gfp = mapping_gfp_mask(inode->i_mapping);
int error;
int ret = VM_FAULT_LOCKED;
@@ -1433,14 +1425,10 @@
spin_unlock(&inode->i_lock);
}
- error = shmem_getpage(inode, vmf->pgoff, &vmf->page, SGP_CACHE, &ret);
+ error = shmem_getpage_gfp(inode, vmf->pgoff, &vmf->page, SGP_CACHE,
+ gfp, vma->vm_mm, &ret);
if (error)
return ((error == -ENOMEM) ? VM_FAULT_OOM : VM_FAULT_SIGBUS);
-
- if (ret & VM_FAULT_MAJOR) {
- count_vm_event(PGMAJFAULT);
- mem_cgroup_count_vm_event(vma->vm_mm, PGMAJFAULT);
- }
return ret;
}
@@ -1587,7 +1575,7 @@
return -EPERM;
}
- return shmem_getpage(inode, index, pagep, SGP_WRITE, NULL);
+ return shmem_getpage(inode, index, pagep, SGP_WRITE);
}
static int
@@ -1633,7 +1621,7 @@
* and even mark them dirty, so it cannot exceed the max_blocks limit.
*/
if (!iter_is_iovec(to))
- sgp = SGP_DIRTY;
+ sgp = SGP_CACHE;
index = *ppos >> PAGE_SHIFT;
offset = *ppos & ~PAGE_MASK;
@@ -1653,14 +1641,17 @@
break;
}
- error = shmem_getpage(inode, index, &page, sgp, NULL);
+ error = shmem_getpage(inode, index, &page, sgp);
if (error) {
if (error == -EINVAL)
error = 0;
break;
}
- if (page)
+ if (page) {
+ if (sgp == SGP_CACHE)
+ set_page_dirty(page);
unlock_page(page);
+ }
/*
* We must evaluate after, since reads (unlike writes)
@@ -1766,7 +1757,7 @@
error = 0;
while (spd.nr_pages < nr_pages) {
- error = shmem_getpage(inode, index, &page, SGP_CACHE, NULL);
+ error = shmem_getpage(inode, index, &page, SGP_CACHE);
if (error)
break;
unlock_page(page);
@@ -1788,8 +1779,7 @@
page = spd.pages[page_nr];
if (!PageUptodate(page) || page->mapping != mapping) {
- error = shmem_getpage(inode, index, &page,
- SGP_CACHE, NULL);
+ error = shmem_getpage(inode, index, &page, SGP_CACHE);
if (error)
break;
unlock_page(page);
@@ -2232,8 +2222,7 @@
else if (shmem_falloc.nr_unswapped > shmem_falloc.nr_falloced)
error = -ENOMEM;
else
- error = shmem_getpage(inode, index, &page, SGP_FALLOC,
- NULL);
+ error = shmem_getpage(inode, index, &page, SGP_FALLOC);
if (error) {
/* Remove the !PageUptodate pages we added */
shmem_undo_range(inode,
@@ -2551,7 +2540,7 @@
inode->i_op = &shmem_short_symlink_operations;
} else {
inode_nohighmem(inode);
- error = shmem_getpage(inode, 0, &page, SGP_WRITE, NULL);
+ error = shmem_getpage(inode, 0, &page, SGP_WRITE);
if (error) {
iput(inode);
return error;
@@ -2592,7 +2581,7 @@
return ERR_PTR(-ECHILD);
}
} else {
- error = shmem_getpage(inode, 0, &page, SGP_READ, NULL);
+ error = shmem_getpage(inode, 0, &page, SGP_READ);
if (error)
return ERR_PTR(error);
unlock_page(page);
@@ -3496,7 +3485,8 @@
int error;
BUG_ON(mapping->a_ops != &shmem_aops);
- error = shmem_getpage_gfp(inode, index, &page, SGP_CACHE, gfp, NULL);
+ error = shmem_getpage_gfp(inode, index, &page, SGP_CACHE,
+ gfp, NULL, NULL);
if (error)
page = ERR_PTR(error);
else
diff --git a/mm/slab.c b/mm/slab.c
index 17e2848..cc8bbc1 100644
--- a/mm/slab.c
+++ b/mm/slab.c
@@ -213,6 +213,11 @@
static int enable_cpucache(struct kmem_cache *cachep, gfp_t gfp);
static void cache_reap(struct work_struct *unused);
+static inline void fixup_objfreelist_debug(struct kmem_cache *cachep,
+ void **list);
+static inline void fixup_slab_list(struct kmem_cache *cachep,
+ struct kmem_cache_node *n, struct page *page,
+ void **list);
static int slab_early_init = 1;
#define INDEX_NODE kmalloc_index(sizeof(struct kmem_cache_node))
@@ -421,8 +426,6 @@
.name = "kmem_cache",
};
-#define BAD_ALIEN_MAGIC 0x01020304ul
-
static DEFINE_PER_CPU(struct delayed_work, slab_reap_work);
static inline struct array_cache *cpu_cache_get(struct kmem_cache *cachep)
@@ -519,22 +522,15 @@
static void init_reap_node(int cpu)
{
- int node;
-
- node = next_node(cpu_to_mem(cpu), node_online_map);
- if (node == MAX_NUMNODES)
- node = first_node(node_online_map);
-
- per_cpu(slab_reap_node, cpu) = node;
+ per_cpu(slab_reap_node, cpu) = next_node_in(cpu_to_mem(cpu),
+ node_online_map);
}
static void next_reap_node(void)
{
int node = __this_cpu_read(slab_reap_node);
- node = next_node(node, node_online_map);
- if (unlikely(node >= MAX_NUMNODES))
- node = first_node(node_online_map);
+ node = next_node_in(node, node_online_map);
__this_cpu_write(slab_reap_node, node);
}
@@ -644,7 +640,7 @@
static inline struct alien_cache **alloc_alien_cache(int node,
int limit, gfp_t gfp)
{
- return (struct alien_cache **)BAD_ALIEN_MAGIC;
+ return NULL;
}
static inline void free_alien_cache(struct alien_cache **ac_ptr)
@@ -850,6 +846,46 @@
}
#endif
+static int init_cache_node(struct kmem_cache *cachep, int node, gfp_t gfp)
+{
+ struct kmem_cache_node *n;
+
+ /*
+ * Set up the kmem_cache_node for cpu before we can
+ * begin anything. Make sure some other cpu on this
+ * node has not already allocated this
+ */
+ n = get_node(cachep, node);
+ if (n) {
+ spin_lock_irq(&n->list_lock);
+ n->free_limit = (1 + nr_cpus_node(node)) * cachep->batchcount +
+ cachep->num;
+ spin_unlock_irq(&n->list_lock);
+
+ return 0;
+ }
+
+ n = kmalloc_node(sizeof(struct kmem_cache_node), gfp, node);
+ if (!n)
+ return -ENOMEM;
+
+ kmem_cache_node_init(n);
+ n->next_reap = jiffies + REAPTIMEOUT_NODE +
+ ((unsigned long)cachep) % REAPTIMEOUT_NODE;
+
+ n->free_limit =
+ (1 + nr_cpus_node(node)) * cachep->batchcount + cachep->num;
+
+ /*
+ * The kmem_cache_nodes don't come and go as CPUs
+ * come and go. slab_mutex is sufficient
+ * protection here.
+ */
+ cachep->node[node] = n;
+
+ return 0;
+}
+
/*
* Allocates and initializes node for a node on each slab cache, used for
* either memory or cpu hotplug. If memory is being hot-added, the kmem_cache_node
@@ -861,46 +897,82 @@
*/
static int init_cache_node_node(int node)
{
+ int ret;
struct kmem_cache *cachep;
- struct kmem_cache_node *n;
- const size_t memsize = sizeof(struct kmem_cache_node);
list_for_each_entry(cachep, &slab_caches, list) {
- /*
- * Set up the kmem_cache_node for cpu before we can
- * begin anything. Make sure some other cpu on this
- * node has not already allocated this
- */
- n = get_node(cachep, node);
- if (!n) {
- n = kmalloc_node(memsize, GFP_KERNEL, node);
- if (!n)
- return -ENOMEM;
- kmem_cache_node_init(n);
- n->next_reap = jiffies + REAPTIMEOUT_NODE +
- ((unsigned long)cachep) % REAPTIMEOUT_NODE;
-
- /*
- * The kmem_cache_nodes don't come and go as CPUs
- * come and go. slab_mutex is sufficient
- * protection here.
- */
- cachep->node[node] = n;
- }
-
- spin_lock_irq(&n->list_lock);
- n->free_limit =
- (1 + nr_cpus_node(node)) *
- cachep->batchcount + cachep->num;
- spin_unlock_irq(&n->list_lock);
+ ret = init_cache_node(cachep, node, GFP_KERNEL);
+ if (ret)
+ return ret;
}
+
return 0;
}
-static inline int slabs_tofree(struct kmem_cache *cachep,
- struct kmem_cache_node *n)
+static int setup_kmem_cache_node(struct kmem_cache *cachep,
+ int node, gfp_t gfp, bool force_change)
{
- return (n->free_objects + cachep->num - 1) / cachep->num;
+ int ret = -ENOMEM;
+ struct kmem_cache_node *n;
+ struct array_cache *old_shared = NULL;
+ struct array_cache *new_shared = NULL;
+ struct alien_cache **new_alien = NULL;
+ LIST_HEAD(list);
+
+ if (use_alien_caches) {
+ new_alien = alloc_alien_cache(node, cachep->limit, gfp);
+ if (!new_alien)
+ goto fail;
+ }
+
+ if (cachep->shared) {
+ new_shared = alloc_arraycache(node,
+ cachep->shared * cachep->batchcount, 0xbaadf00d, gfp);
+ if (!new_shared)
+ goto fail;
+ }
+
+ ret = init_cache_node(cachep, node, gfp);
+ if (ret)
+ goto fail;
+
+ n = get_node(cachep, node);
+ spin_lock_irq(&n->list_lock);
+ if (n->shared && force_change) {
+ free_block(cachep, n->shared->entry,
+ n->shared->avail, node, &list);
+ n->shared->avail = 0;
+ }
+
+ if (!n->shared || force_change) {
+ old_shared = n->shared;
+ n->shared = new_shared;
+ new_shared = NULL;
+ }
+
+ if (!n->alien) {
+ n->alien = new_alien;
+ new_alien = NULL;
+ }
+
+ spin_unlock_irq(&n->list_lock);
+ slabs_destroy(cachep, &list);
+
+ /*
+ * To protect lockless access to n->shared during irq disabled context.
+ * If n->shared isn't NULL in irq disabled context, accessing to it is
+ * guaranteed to be valid until irq is re-enabled, because it will be
+ * freed after synchronize_sched().
+ */
+ if (force_change)
+ synchronize_sched();
+
+fail:
+ kfree(old_shared);
+ kfree(new_shared);
+ free_alien_cache(new_alien);
+
+ return ret;
}
static void cpuup_canceled(long cpu)
@@ -967,14 +1039,13 @@
n = get_node(cachep, node);
if (!n)
continue;
- drain_freelist(cachep, n, slabs_tofree(cachep, n));
+ drain_freelist(cachep, n, INT_MAX);
}
}
static int cpuup_prepare(long cpu)
{
struct kmem_cache *cachep;
- struct kmem_cache_node *n = NULL;
int node = cpu_to_mem(cpu);
int err;
@@ -993,44 +1064,9 @@
* array caches
*/
list_for_each_entry(cachep, &slab_caches, list) {
- struct array_cache *shared = NULL;
- struct alien_cache **alien = NULL;
-
- if (cachep->shared) {
- shared = alloc_arraycache(node,
- cachep->shared * cachep->batchcount,
- 0xbaadf00d, GFP_KERNEL);
- if (!shared)
- goto bad;
- }
- if (use_alien_caches) {
- alien = alloc_alien_cache(node, cachep->limit, GFP_KERNEL);
- if (!alien) {
- kfree(shared);
- goto bad;
- }
- }
- n = get_node(cachep, node);
- BUG_ON(!n);
-
- spin_lock_irq(&n->list_lock);
- if (!n->shared) {
- /*
- * We are serialised from CPU_DEAD or
- * CPU_UP_CANCELLED by the cpucontrol lock
- */
- n->shared = shared;
- shared = NULL;
- }
-#ifdef CONFIG_NUMA
- if (!n->alien) {
- n->alien = alien;
- alien = NULL;
- }
-#endif
- spin_unlock_irq(&n->list_lock);
- kfree(shared);
- free_alien_cache(alien);
+ err = setup_kmem_cache_node(cachep, node, GFP_KERNEL, false);
+ if (err)
+ goto bad;
}
return 0;
@@ -1119,7 +1155,7 @@
if (!n)
continue;
- drain_freelist(cachep, n, slabs_tofree(cachep, n));
+ drain_freelist(cachep, n, INT_MAX);
if (!list_empty(&n->slabs_full) ||
!list_empty(&n->slabs_partial)) {
@@ -1200,6 +1236,61 @@
}
}
+#ifdef CONFIG_SLAB_FREELIST_RANDOM
+static void freelist_randomize(struct rnd_state *state, freelist_idx_t *list,
+ size_t count)
+{
+ size_t i;
+ unsigned int rand;
+
+ for (i = 0; i < count; i++)
+ list[i] = i;
+
+ /* Fisher-Yates shuffle */
+ for (i = count - 1; i > 0; i--) {
+ rand = prandom_u32_state(state);
+ rand %= (i + 1);
+ swap(list[i], list[rand]);
+ }
+}
+
+/* Create a random sequence per cache */
+static int cache_random_seq_create(struct kmem_cache *cachep, gfp_t gfp)
+{
+ unsigned int seed, count = cachep->num;
+ struct rnd_state state;
+
+ if (count < 2)
+ return 0;
+
+ /* If it fails, we will just use the global lists */
+ cachep->random_seq = kcalloc(count, sizeof(freelist_idx_t), gfp);
+ if (!cachep->random_seq)
+ return -ENOMEM;
+
+ /* Get best entropy at this stage */
+ get_random_bytes_arch(&seed, sizeof(seed));
+ prandom_seed_state(&state, seed);
+
+ freelist_randomize(&state, cachep->random_seq, count);
+ return 0;
+}
+
+/* Destroy the per-cache random freelist sequence */
+static void cache_random_seq_destroy(struct kmem_cache *cachep)
+{
+ kfree(cachep->random_seq);
+ cachep->random_seq = NULL;
+}
+#else
+static inline int cache_random_seq_create(struct kmem_cache *cachep, gfp_t gfp)
+{
+ return 0;
+}
+static inline void cache_random_seq_destroy(struct kmem_cache *cachep) { }
+#endif /* CONFIG_SLAB_FREELIST_RANDOM */
+
+
/*
* Initialisation. Called after the page allocator have been initialised and
* before smp_init().
@@ -1212,7 +1303,7 @@
sizeof(struct rcu_head));
kmem_cache = &kmem_cache_boot;
- if (num_possible_nodes() == 1)
+ if (!IS_ENABLED(CONFIG_NUMA) || num_possible_nodes() == 1)
use_alien_caches = 0;
for (i = 0; i < NUM_INIT_LISTS; i++)
@@ -1781,7 +1872,7 @@
/*
* Needed to avoid possible looping condition
- * in cache_grow()
+ * in cache_grow_begin()
*/
if (OFF_SLAB(freelist_cache))
continue;
@@ -2138,7 +2229,7 @@
cachep->freelist_size = cachep->num * sizeof(freelist_idx_t);
cachep->flags = flags;
cachep->allocflags = __GFP_COMP;
- if (CONFIG_ZONE_DMA_FLAG && (flags & SLAB_CACHE_DMA))
+ if (flags & SLAB_CACHE_DMA)
cachep->allocflags |= GFP_DMA;
cachep->size = size;
cachep->reciprocal_buffer_size = reciprocal_value(size);
@@ -2180,6 +2271,11 @@
BUG_ON(irqs_disabled());
}
+static void check_mutex_acquired(void)
+{
+ BUG_ON(!mutex_is_locked(&slab_mutex));
+}
+
static void check_spinlock_acquired(struct kmem_cache *cachep)
{
#ifdef CONFIG_SMP
@@ -2199,13 +2295,27 @@
#else
#define check_irq_off() do { } while(0)
#define check_irq_on() do { } while(0)
+#define check_mutex_acquired() do { } while(0)
#define check_spinlock_acquired(x) do { } while(0)
#define check_spinlock_acquired_node(x, y) do { } while(0)
#endif
-static void drain_array(struct kmem_cache *cachep, struct kmem_cache_node *n,
- struct array_cache *ac,
- int force, int node);
+static void drain_array_locked(struct kmem_cache *cachep, struct array_cache *ac,
+ int node, bool free_all, struct list_head *list)
+{
+ int tofree;
+
+ if (!ac || !ac->avail)
+ return;
+
+ tofree = free_all ? ac->avail : (ac->limit + 4) / 5;
+ if (tofree > ac->avail)
+ tofree = (ac->avail + 1) / 2;
+
+ free_block(cachep, ac->entry, tofree, node, list);
+ ac->avail -= tofree;
+ memmove(ac->entry, &(ac->entry[tofree]), sizeof(void *) * ac->avail);
+}
static void do_drain(void *arg)
{
@@ -2229,6 +2339,7 @@
{
struct kmem_cache_node *n;
int node;
+ LIST_HEAD(list);
on_each_cpu(do_drain, cachep, 1);
check_irq_on();
@@ -2236,8 +2347,13 @@
if (n->alien)
drain_alien_cache(cachep, n->alien);
- for_each_kmem_cache_node(cachep, node, n)
- drain_array(cachep, n, n->shared, 1, node);
+ for_each_kmem_cache_node(cachep, node, n) {
+ spin_lock_irq(&n->list_lock);
+ drain_array_locked(cachep, n->shared, node, true, &list);
+ spin_unlock_irq(&n->list_lock);
+
+ slabs_destroy(cachep, &list);
+ }
}
/*
@@ -2288,7 +2404,7 @@
check_irq_on();
for_each_kmem_cache_node(cachep, node, n) {
- drain_freelist(cachep, n, slabs_tofree(cachep, n));
+ drain_freelist(cachep, n, INT_MAX);
ret += !list_empty(&n->slabs_full) ||
!list_empty(&n->slabs_partial);
@@ -2306,6 +2422,8 @@
int i;
struct kmem_cache_node *n;
+ cache_random_seq_destroy(cachep);
+
free_percpu(cachep->cpu_cache);
/* NUMA: free the node structures */
@@ -2412,15 +2530,115 @@
#endif
}
+#ifdef CONFIG_SLAB_FREELIST_RANDOM
+/* Hold information during a freelist initialization */
+union freelist_init_state {
+ struct {
+ unsigned int pos;
+ freelist_idx_t *list;
+ unsigned int count;
+ unsigned int rand;
+ };
+ struct rnd_state rnd_state;
+};
+
+/*
+ * Initialize the state based on the randomization methode available.
+ * return true if the pre-computed list is available, false otherwize.
+ */
+static bool freelist_state_initialize(union freelist_init_state *state,
+ struct kmem_cache *cachep,
+ unsigned int count)
+{
+ bool ret;
+ unsigned int rand;
+
+ /* Use best entropy available to define a random shift */
+ get_random_bytes_arch(&rand, sizeof(rand));
+
+ /* Use a random state if the pre-computed list is not available */
+ if (!cachep->random_seq) {
+ prandom_seed_state(&state->rnd_state, rand);
+ ret = false;
+ } else {
+ state->list = cachep->random_seq;
+ state->count = count;
+ state->pos = 0;
+ state->rand = rand;
+ ret = true;
+ }
+ return ret;
+}
+
+/* Get the next entry on the list and randomize it using a random shift */
+static freelist_idx_t next_random_slot(union freelist_init_state *state)
+{
+ return (state->list[state->pos++] + state->rand) % state->count;
+}
+
+/*
+ * Shuffle the freelist initialization state based on pre-computed lists.
+ * return true if the list was successfully shuffled, false otherwise.
+ */
+static bool shuffle_freelist(struct kmem_cache *cachep, struct page *page)
+{
+ unsigned int objfreelist = 0, i, count = cachep->num;
+ union freelist_init_state state;
+ bool precomputed;
+
+ if (count < 2)
+ return false;
+
+ precomputed = freelist_state_initialize(&state, cachep, count);
+
+ /* Take a random entry as the objfreelist */
+ if (OBJFREELIST_SLAB(cachep)) {
+ if (!precomputed)
+ objfreelist = count - 1;
+ else
+ objfreelist = next_random_slot(&state);
+ page->freelist = index_to_obj(cachep, page, objfreelist) +
+ obj_offset(cachep);
+ count--;
+ }
+
+ /*
+ * On early boot, generate the list dynamically.
+ * Later use a pre-computed list for speed.
+ */
+ if (!precomputed) {
+ freelist_randomize(&state.rnd_state, page->freelist, count);
+ } else {
+ for (i = 0; i < count; i++)
+ set_free_obj(page, i, next_random_slot(&state));
+ }
+
+ if (OBJFREELIST_SLAB(cachep))
+ set_free_obj(page, cachep->num - 1, objfreelist);
+
+ return true;
+}
+#else
+static inline bool shuffle_freelist(struct kmem_cache *cachep,
+ struct page *page)
+{
+ return false;
+}
+#endif /* CONFIG_SLAB_FREELIST_RANDOM */
+
static void cache_init_objs(struct kmem_cache *cachep,
struct page *page)
{
int i;
void *objp;
+ bool shuffled;
cache_init_objs_debug(cachep, page);
- if (OBJFREELIST_SLAB(cachep)) {
+ /* Try to randomize the freelist if enabled */
+ shuffled = shuffle_freelist(cachep, page);
+
+ if (!shuffled && OBJFREELIST_SLAB(cachep)) {
page->freelist = index_to_obj(cachep, page, cachep->num - 1) +
obj_offset(cachep);
}
@@ -2434,17 +2652,8 @@
kasan_poison_object_data(cachep, objp);
}
- set_free_obj(page, i, i);
- }
-}
-
-static void kmem_flagcheck(struct kmem_cache *cachep, gfp_t flags)
-{
- if (CONFIG_ZONE_DMA_FLAG) {
- if (flags & GFP_DMA)
- BUG_ON(!(cachep->allocflags & GFP_DMA));
- else
- BUG_ON(cachep->allocflags & GFP_DMA);
+ if (!shuffled)
+ set_free_obj(page, i, i);
}
}
@@ -2502,13 +2711,15 @@
* Grow (by 1) the number of slabs within a cache. This is called by
* kmem_cache_alloc() when there are no active objs left in a cache.
*/
-static int cache_grow(struct kmem_cache *cachep,
- gfp_t flags, int nodeid, struct page *page)
+static struct page *cache_grow_begin(struct kmem_cache *cachep,
+ gfp_t flags, int nodeid)
{
void *freelist;
size_t offset;
gfp_t local_flags;
+ int page_node;
struct kmem_cache_node *n;
+ struct page *page;
/*
* Be lazy and only check for valid flags here, keeping it out of the
@@ -2520,43 +2731,35 @@
}
local_flags = flags & (GFP_CONSTRAINT_MASK|GFP_RECLAIM_MASK);
- /* Take the node list lock to change the colour_next on this node */
check_irq_off();
- n = get_node(cachep, nodeid);
- spin_lock(&n->list_lock);
-
- /* Get colour for the slab, and cal the next value. */
- offset = n->colour_next;
- n->colour_next++;
- if (n->colour_next >= cachep->colour)
- n->colour_next = 0;
- spin_unlock(&n->list_lock);
-
- offset *= cachep->colour_off;
-
if (gfpflags_allow_blocking(local_flags))
local_irq_enable();
/*
- * The test for missing atomic flag is performed here, rather than
- * the more obvious place, simply to reduce the critical path length
- * in kmem_cache_alloc(). If a caller is seriously mis-behaving they
- * will eventually be caught here (where it matters).
- */
- kmem_flagcheck(cachep, flags);
-
- /*
* Get mem for the objs. Attempt to allocate a physical page from
* 'nodeid'.
*/
- if (!page)
- page = kmem_getpages(cachep, local_flags, nodeid);
+ page = kmem_getpages(cachep, local_flags, nodeid);
if (!page)
goto failed;
+ page_node = page_to_nid(page);
+ n = get_node(cachep, page_node);
+
+ /* Get colour for the slab, and cal the next value. */
+ n->colour_next++;
+ if (n->colour_next >= cachep->colour)
+ n->colour_next = 0;
+
+ offset = n->colour_next;
+ if (offset >= cachep->colour)
+ offset = 0;
+
+ offset *= cachep->colour_off;
+
/* Get slab management. */
freelist = alloc_slabmgmt(cachep, page, offset,
- local_flags & ~GFP_CONSTRAINT_MASK, nodeid);
+ local_flags & ~GFP_CONSTRAINT_MASK, page_node);
if (OFF_SLAB(cachep) && !freelist)
goto opps1;
@@ -2567,21 +2770,40 @@
if (gfpflags_allow_blocking(local_flags))
local_irq_disable();
- check_irq_off();
- spin_lock(&n->list_lock);
- /* Make slab active. */
- list_add_tail(&page->lru, &(n->slabs_free));
- STATS_INC_GROWN(cachep);
- n->free_objects += cachep->num;
- spin_unlock(&n->list_lock);
- return 1;
+ return page;
+
opps1:
kmem_freepages(cachep, page);
failed:
if (gfpflags_allow_blocking(local_flags))
local_irq_disable();
- return 0;
+ return NULL;
+}
+
+static void cache_grow_end(struct kmem_cache *cachep, struct page *page)
+{
+ struct kmem_cache_node *n;
+ void *list = NULL;
+
+ check_irq_off();
+
+ if (!page)
+ return;
+
+ INIT_LIST_HEAD(&page->lru);
+ n = get_node(cachep, page_to_nid(page));
+
+ spin_lock(&n->list_lock);
+ if (!page->active)
+ list_add_tail(&page->lru, &(n->slabs_free));
+ else
+ fixup_slab_list(cachep, n, page, &list);
+ STATS_INC_GROWN(cachep);
+ n->free_objects += cachep->num - page->active;
+ spin_unlock(&n->list_lock);
+
+ fixup_objfreelist_debug(cachep, &list);
}
#if DEBUG
@@ -2785,18 +3007,42 @@
return obj;
}
+/*
+ * Slab list should be fixed up by fixup_slab_list() for existing slab
+ * or cache_grow_end() for new slab
+ */
+static __always_inline int alloc_block(struct kmem_cache *cachep,
+ struct array_cache *ac, struct page *page, int batchcount)
+{
+ /*
+ * There must be at least one object available for
+ * allocation.
+ */
+ BUG_ON(page->active >= cachep->num);
+
+ while (page->active < cachep->num && batchcount--) {
+ STATS_INC_ALLOCED(cachep);
+ STATS_INC_ACTIVE(cachep);
+ STATS_SET_HIGH(cachep);
+
+ ac->entry[ac->avail++] = slab_get_obj(cachep, page);
+ }
+
+ return batchcount;
+}
+
static void *cache_alloc_refill(struct kmem_cache *cachep, gfp_t flags)
{
int batchcount;
struct kmem_cache_node *n;
- struct array_cache *ac;
+ struct array_cache *ac, *shared;
int node;
void *list = NULL;
+ struct page *page;
check_irq_off();
node = numa_mem_id();
-retry:
ac = cpu_cache_get(cachep);
batchcount = ac->batchcount;
if (!ac->touched && batchcount > BATCHREFILL_LIMIT) {
@@ -2810,16 +3056,20 @@
n = get_node(cachep, node);
BUG_ON(ac->avail > 0 || !n);
+ shared = READ_ONCE(n->shared);
+ if (!n->free_objects && (!shared || !shared->avail))
+ goto direct_grow;
+
spin_lock(&n->list_lock);
+ shared = READ_ONCE(n->shared);
/* See if we can refill from the shared array */
- if (n->shared && transfer_objects(ac, n->shared, batchcount)) {
- n->shared->touched = 1;
+ if (shared && transfer_objects(ac, shared, batchcount)) {
+ shared->touched = 1;
goto alloc_done;
}
while (batchcount > 0) {
- struct page *page;
/* Get slab alloc is to come from. */
page = get_first_slab(n, false);
if (!page)
@@ -2827,21 +3077,7 @@
check_spinlock_acquired(cachep);
- /*
- * The slab was either on partial or free list so
- * there must be at least one object available for
- * allocation.
- */
- BUG_ON(page->active >= cachep->num);
-
- while (page->active < cachep->num && batchcount--) {
- STATS_INC_ALLOCED(cachep);
- STATS_INC_ACTIVE(cachep);
- STATS_SET_HIGH(cachep);
-
- ac->entry[ac->avail++] = slab_get_obj(cachep, page);
- }
-
+ batchcount = alloc_block(cachep, ac, page, batchcount);
fixup_slab_list(cachep, n, page, &list);
}
@@ -2851,9 +3087,8 @@
spin_unlock(&n->list_lock);
fixup_objfreelist_debug(cachep, &list);
+direct_grow:
if (unlikely(!ac->avail)) {
- int x;
-
/* Check if we can use obj in pfmemalloc slab */
if (sk_memalloc_socks()) {
void *obj = cache_alloc_pfmemalloc(cachep, n, flags);
@@ -2862,18 +3097,19 @@
return obj;
}
- x = cache_grow(cachep, gfp_exact_node(flags), node, NULL);
+ page = cache_grow_begin(cachep, gfp_exact_node(flags), node);
- /* cache_grow can reenable interrupts, then ac could change. */
+ /*
+ * cache_grow_begin() can reenable interrupts,
+ * then ac could change.
+ */
ac = cpu_cache_get(cachep);
- node = numa_mem_id();
+ if (!ac->avail && page)
+ alloc_block(cachep, ac, page, batchcount);
+ cache_grow_end(cachep, page);
- /* no objects in sight? abort */
- if (!x && ac->avail == 0)
+ if (!ac->avail)
return NULL;
-
- if (!ac->avail) /* objects refilled by interrupt? */
- goto retry;
}
ac->touched = 1;
@@ -2884,9 +3120,6 @@
gfp_t flags)
{
might_sleep_if(gfpflags_allow_blocking(flags));
-#if DEBUG
- kmem_flagcheck(cachep, flags);
-#endif
}
#if DEBUG
@@ -2998,19 +3231,17 @@
static void *fallback_alloc(struct kmem_cache *cache, gfp_t flags)
{
struct zonelist *zonelist;
- gfp_t local_flags;
struct zoneref *z;
struct zone *zone;
enum zone_type high_zoneidx = gfp_zone(flags);
void *obj = NULL;
+ struct page *page;
int nid;
unsigned int cpuset_mems_cookie;
if (flags & __GFP_THISNODE)
return NULL;
- local_flags = flags & (GFP_CONSTRAINT_MASK|GFP_RECLAIM_MASK);
-
retry_cpuset:
cpuset_mems_cookie = read_mems_allowed_begin();
zonelist = node_zonelist(mempolicy_slab_node(), flags);
@@ -3040,33 +3271,19 @@
* We may trigger various forms of reclaim on the allowed
* set and go into memory reserves if necessary.
*/
- struct page *page;
-
- if (gfpflags_allow_blocking(local_flags))
- local_irq_enable();
- kmem_flagcheck(cache, flags);
- page = kmem_getpages(cache, local_flags, numa_mem_id());
- if (gfpflags_allow_blocking(local_flags))
- local_irq_disable();
+ page = cache_grow_begin(cache, flags, numa_mem_id());
+ cache_grow_end(cache, page);
if (page) {
- /*
- * Insert into the appropriate per node queues
- */
nid = page_to_nid(page);
- if (cache_grow(cache, flags, nid, page)) {
- obj = ____cache_alloc_node(cache,
- gfp_exact_node(flags), nid);
- if (!obj)
- /*
- * Another processor may allocate the
- * objects in the slab since we are
- * not holding any locks.
- */
- goto retry;
- } else {
- /* cache_grow already freed obj */
- obj = NULL;
- }
+ obj = ____cache_alloc_node(cache,
+ gfp_exact_node(flags), nid);
+
+ /*
+ * Another processor may allocate the objects in
+ * the slab since we are not holding any locks.
+ */
+ if (!obj)
+ goto retry;
}
}
@@ -3083,15 +3300,13 @@
{
struct page *page;
struct kmem_cache_node *n;
- void *obj;
+ void *obj = NULL;
void *list = NULL;
- int x;
VM_BUG_ON(nodeid < 0 || nodeid >= MAX_NUMNODES);
n = get_node(cachep, nodeid);
BUG_ON(!n);
-retry:
check_irq_off();
spin_lock(&n->list_lock);
page = get_first_slab(n, false);
@@ -3113,18 +3328,18 @@
spin_unlock(&n->list_lock);
fixup_objfreelist_debug(cachep, &list);
- goto done;
+ return obj;
must_grow:
spin_unlock(&n->list_lock);
- x = cache_grow(cachep, gfp_exact_node(flags), nodeid, NULL);
- if (x)
- goto retry;
+ page = cache_grow_begin(cachep, gfp_exact_node(flags), nodeid);
+ if (page) {
+ /* This slab isn't counted yet so don't update free_objects */
+ obj = slab_get_obj(cachep, page);
+ }
+ cache_grow_end(cachep, page);
- return fallback_alloc(cachep, flags);
-
-done:
- return obj;
+ return obj ? obj : fallback_alloc(cachep, flags);
}
static __always_inline void *
@@ -3242,6 +3457,9 @@
{
int i;
struct kmem_cache_node *n = get_node(cachep, node);
+ struct page *page;
+
+ n->free_objects += nr_objects;
for (i = 0; i < nr_objects; i++) {
void *objp;
@@ -3254,17 +3472,11 @@
check_spinlock_acquired_node(cachep, node);
slab_put_obj(cachep, page, objp);
STATS_DEC_ACTIVE(cachep);
- n->free_objects++;
/* fixup slab chains */
- if (page->active == 0) {
- if (n->free_objects > n->free_limit) {
- n->free_objects -= cachep->num;
- list_add_tail(&page->lru, list);
- } else {
- list_add(&page->lru, &n->slabs_free);
- }
- } else {
+ if (page->active == 0)
+ list_add(&page->lru, &n->slabs_free);
+ else {
/* Unconditionally move a slab to the end of the
* partial list on free - maximum time for the
* other objects to be freed, too.
@@ -3272,6 +3484,14 @@
list_add_tail(&page->lru, &n->slabs_partial);
}
}
+
+ while (n->free_objects > n->free_limit && !list_empty(&n->slabs_free)) {
+ n->free_objects -= cachep->num;
+
+ page = list_last_entry(&n->slabs_free, struct page, lru);
+ list_del(&page->lru);
+ list_add(&page->lru, list);
+ }
}
static void cache_flusharray(struct kmem_cache *cachep, struct array_cache *ac)
@@ -3327,9 +3547,17 @@
static inline void __cache_free(struct kmem_cache *cachep, void *objp,
unsigned long caller)
{
- struct array_cache *ac = cpu_cache_get(cachep);
+ /* Put the object into the quarantine, don't touch it for now. */
+ if (kasan_slab_free(cachep, objp))
+ return;
- kasan_slab_free(cachep, objp);
+ ___cache_free(cachep, objp, caller);
+}
+
+void ___cache_free(struct kmem_cache *cachep, void *objp,
+ unsigned long caller)
+{
+ struct array_cache *ac = cpu_cache_get(cachep);
check_irq_off();
kmemleak_free_recursive(objp, cachep->flags);
@@ -3645,72 +3873,19 @@
/*
* This initializes kmem_cache_node or resizes various caches for all nodes.
*/
-static int alloc_kmem_cache_node(struct kmem_cache *cachep, gfp_t gfp)
+static int setup_kmem_cache_nodes(struct kmem_cache *cachep, gfp_t gfp)
{
+ int ret;
int node;
struct kmem_cache_node *n;
- struct array_cache *new_shared;
- struct alien_cache **new_alien = NULL;
for_each_online_node(node) {
-
- if (use_alien_caches) {
- new_alien = alloc_alien_cache(node, cachep->limit, gfp);
- if (!new_alien)
- goto fail;
- }
-
- new_shared = NULL;
- if (cachep->shared) {
- new_shared = alloc_arraycache(node,
- cachep->shared*cachep->batchcount,
- 0xbaadf00d, gfp);
- if (!new_shared) {
- free_alien_cache(new_alien);
- goto fail;
- }
- }
-
- n = get_node(cachep, node);
- if (n) {
- struct array_cache *shared = n->shared;
- LIST_HEAD(list);
-
- spin_lock_irq(&n->list_lock);
-
- if (shared)
- free_block(cachep, shared->entry,
- shared->avail, node, &list);
-
- n->shared = new_shared;
- if (!n->alien) {
- n->alien = new_alien;
- new_alien = NULL;
- }
- n->free_limit = (1 + nr_cpus_node(node)) *
- cachep->batchcount + cachep->num;
- spin_unlock_irq(&n->list_lock);
- slabs_destroy(cachep, &list);
- kfree(shared);
- free_alien_cache(new_alien);
- continue;
- }
- n = kmalloc_node(sizeof(struct kmem_cache_node), gfp, node);
- if (!n) {
- free_alien_cache(new_alien);
- kfree(new_shared);
+ ret = setup_kmem_cache_node(cachep, node, gfp, true);
+ if (ret)
goto fail;
- }
- kmem_cache_node_init(n);
- n->next_reap = jiffies + REAPTIMEOUT_NODE +
- ((unsigned long)cachep) % REAPTIMEOUT_NODE;
- n->shared = new_shared;
- n->alien = new_alien;
- n->free_limit = (1 + nr_cpus_node(node)) *
- cachep->batchcount + cachep->num;
- cachep->node[node] = n;
}
+
return 0;
fail:
@@ -3752,7 +3927,7 @@
cachep->shared = shared;
if (!prev)
- goto alloc_node;
+ goto setup_node;
for_each_online_cpu(cpu) {
LIST_HEAD(list);
@@ -3769,8 +3944,8 @@
}
free_percpu(prev);
-alloc_node:
- return alloc_kmem_cache_node(cachep, gfp);
+setup_node:
+ return setup_kmem_cache_nodes(cachep, gfp);
}
static int do_tune_cpucache(struct kmem_cache *cachep, int limit,
@@ -3804,6 +3979,10 @@
int shared = 0;
int batchcount = 0;
+ err = cache_random_seq_create(cachep, gfp);
+ if (err)
+ goto end;
+
if (!is_root_cache(cachep)) {
struct kmem_cache *root = memcg_root_cache(cachep);
limit = root->limit;
@@ -3857,6 +4036,7 @@
batchcount = (limit + 1) / 2;
skip_setup:
err = do_tune_cpucache(cachep, limit, batchcount, shared, gfp);
+end:
if (err)
pr_err("enable_cpucache failed for %s, error %d\n",
cachep->name, -err);
@@ -3869,29 +4049,26 @@
* if drain_array() is used on the shared array.
*/
static void drain_array(struct kmem_cache *cachep, struct kmem_cache_node *n,
- struct array_cache *ac, int force, int node)
+ struct array_cache *ac, int node)
{
LIST_HEAD(list);
- int tofree;
+
+ /* ac from n->shared can be freed if we don't hold the slab_mutex. */
+ check_mutex_acquired();
if (!ac || !ac->avail)
return;
- if (ac->touched && !force) {
+
+ if (ac->touched) {
ac->touched = 0;
- } else {
- spin_lock_irq(&n->list_lock);
- if (ac->avail) {
- tofree = force ? ac->avail : (ac->limit + 4) / 5;
- if (tofree > ac->avail)
- tofree = (ac->avail + 1) / 2;
- free_block(cachep, ac->entry, tofree, node, &list);
- ac->avail -= tofree;
- memmove(ac->entry, &(ac->entry[tofree]),
- sizeof(void *) * ac->avail);
- }
- spin_unlock_irq(&n->list_lock);
- slabs_destroy(cachep, &list);
+ return;
}
+
+ spin_lock_irq(&n->list_lock);
+ drain_array_locked(cachep, ac, node, false, &list);
+ spin_unlock_irq(&n->list_lock);
+
+ slabs_destroy(cachep, &list);
}
/**
@@ -3929,7 +4106,7 @@
reap_alien(searchp, n);
- drain_array(searchp, n, cpu_cache_get(searchp), 0, node);
+ drain_array(searchp, n, cpu_cache_get(searchp), node);
/*
* These are racy checks but it does not matter
@@ -3940,7 +4117,7 @@
n->next_reap = jiffies + REAPTIMEOUT_NODE;
- drain_array(searchp, n, n->shared, 0, node);
+ drain_array(searchp, n, n->shared, node);
if (n->free_touched)
n->free_touched = 0;
@@ -4324,7 +4501,7 @@
/* We assume that ksize callers could use the whole allocated area,
* so we need to unpoison this area.
*/
- kasan_krealloc(objp, size, GFP_NOWAIT);
+ kasan_unpoison_shadow(objp, size);
return size;
}
diff --git a/mm/slab.h b/mm/slab.h
index 5969769..dedb1a9 100644
--- a/mm/slab.h
+++ b/mm/slab.h
@@ -462,4 +462,6 @@
void slab_stop(struct seq_file *m, void *p);
int memcg_slab_show(struct seq_file *m, void *p);
+void ___cache_free(struct kmem_cache *cache, void *x, unsigned long addr);
+
#endif /* MM_SLAB_H */
diff --git a/mm/slab_common.c b/mm/slab_common.c
index 3239bfd..a65dad7 100644
--- a/mm/slab_common.c
+++ b/mm/slab_common.c
@@ -715,6 +715,7 @@
get_online_cpus();
get_online_mems();
+ kasan_cache_destroy(s);
mutex_lock(&slab_mutex);
s->refcount--;
@@ -753,6 +754,7 @@
get_online_cpus();
get_online_mems();
+ kasan_cache_shrink(cachep);
ret = __kmem_cache_shrink(cachep, false);
put_online_mems();
put_online_cpus();
diff --git a/mm/slub.c b/mm/slub.c
index 4dbb109e..825ff45 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -329,8 +329,8 @@
tmp.counters = counters_new;
/*
* page->counters can cover frozen/inuse/objects as well
- * as page->_count. If we assign to ->counters directly
- * we run the risk of losing updates to page->_count, so
+ * as page->_refcount. If we assign to ->counters directly
+ * we run the risk of losing updates to page->_refcount, so
* be careful and only assign to the fields we need.
*/
page->frozen = tmp.frozen;
@@ -1735,11 +1735,11 @@
* may return off node objects because partial slabs are obtained
* from other nodes and filled up.
*
- * If /sys/kernel/slab/xx/defrag_ratio is set to 100 (which makes
- * defrag_ratio = 1000) then every (well almost) allocation will
- * first attempt to defrag slab caches on other nodes. This means
- * scanning over all nodes to look for partial slabs which may be
- * expensive if we do it every time we are trying to find a slab
+ * If /sys/kernel/slab/xx/remote_node_defrag_ratio is set to 100
+ * (which makes defrag_ratio = 1000) then every (well almost)
+ * allocation will first attempt to defrag slab caches on other nodes.
+ * This means scanning over all nodes to look for partial slabs which
+ * may be expensive if we do it every time we are trying to find a slab
* with available objects.
*/
if (!s->remote_node_defrag_ratio ||
@@ -3635,8 +3635,9 @@
{
size_t size = __ksize(object);
/* We assume that ksize callers could use whole allocated area,
- so we need unpoison this area. */
- kasan_krealloc(object, size, GFP_NOWAIT);
+ * so we need to unpoison this area.
+ */
+ kasan_unpoison_shadow(object, size);
return size;
}
EXPORT_SYMBOL(ksize);
@@ -3697,7 +3698,7 @@
* s->cpu_partial is checked locklessly (see put_cpu_partial),
* so we have to make sure the change is visible.
*/
- kick_all_cpus_sync();
+ synchronize_sched();
}
flush_all(s);
diff --git a/mm/swap.c b/mm/swap.c
index 03aacbc..9591614 100644
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -47,6 +47,9 @@
static DEFINE_PER_CPU(struct pagevec, lru_rotate_pvecs);
static DEFINE_PER_CPU(struct pagevec, lru_deactivate_file_pvecs);
static DEFINE_PER_CPU(struct pagevec, lru_deactivate_pvecs);
+#ifdef CONFIG_SMP
+static DEFINE_PER_CPU(struct pagevec, activate_page_pvecs);
+#endif
/*
* This path almost never happens for VM activity - pages are normally
@@ -274,8 +277,6 @@
}
#ifdef CONFIG_SMP
-static DEFINE_PER_CPU(struct pagevec, activate_page_pvecs);
-
static void activate_page_drain(int cpu)
{
struct pagevec *pvec = &per_cpu(activate_page_pvecs, cpu);
diff --git a/mm/swap_state.c b/mm/swap_state.c
index 366ce35..0d457e7 100644
--- a/mm/swap_state.c
+++ b/mm/swap_state.c
@@ -358,7 +358,7 @@
/* May fail (-ENOMEM) if radix-tree node allocation failed. */
__SetPageLocked(new_page);
- SetPageSwapBacked(new_page);
+ __SetPageSwapBacked(new_page);
err = __add_to_swap_cache(new_page, entry);
if (likely(!err)) {
radix_tree_preload_end();
@@ -370,7 +370,6 @@
return new_page;
}
radix_tree_preload_end();
- ClearPageSwapBacked(new_page);
__ClearPageLocked(new_page);
/*
* add_to_swap_cache() doesn't return -EEXIST, so we can safely
diff --git a/mm/util.c b/mm/util.c
index 6cc81e7..917e0e3 100644
--- a/mm/util.c
+++ b/mm/util.c
@@ -297,7 +297,8 @@
ret = security_mmap_file(file, prot, flag);
if (!ret) {
- down_write(&mm->mmap_sem);
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
ret = do_mmap_pgoff(file, addr, len, prot, flag, pgoff,
&populate);
up_write(&mm->mmap_sem);
@@ -346,6 +347,29 @@
return __page_rmapping(page);
}
+/*
+ * Return true if this page is mapped into pagetables.
+ * For compound page it returns true if any subpage of compound page is mapped.
+ */
+bool page_mapped(struct page *page)
+{
+ int i;
+
+ if (likely(!PageCompound(page)))
+ return atomic_read(&page->_mapcount) >= 0;
+ page = compound_head(page);
+ if (atomic_read(compound_mapcount_ptr(page)) >= 0)
+ return true;
+ if (PageHuge(page))
+ return false;
+ for (i = 0; i < hpage_nr_pages(page); i++) {
+ if (atomic_read(&page[i]._mapcount) >= 0)
+ return true;
+ }
+ return false;
+}
+EXPORT_SYMBOL(page_mapped);
+
struct anon_vma *page_anon_vma(struct page *page)
{
unsigned long mapping;
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index ae7d20b..cf7ad1a 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -21,6 +21,7 @@
#include <linux/debugobjects.h>
#include <linux/kallsyms.h>
#include <linux/list.h>
+#include <linux/notifier.h>
#include <linux/rbtree.h>
#include <linux/radix-tree.h>
#include <linux/rcupdate.h>
@@ -274,13 +275,12 @@
/*** Global kva allocator ***/
-#define VM_LAZY_FREE 0x01
-#define VM_LAZY_FREEING 0x02
#define VM_VM_AREA 0x04
static DEFINE_SPINLOCK(vmap_area_lock);
/* Export for kexec only */
LIST_HEAD(vmap_area_list);
+static LLIST_HEAD(vmap_purge_list);
static struct rb_root vmap_area_root = RB_ROOT;
/* The vmap cache globals are protected by vmap_area_lock */
@@ -344,6 +344,8 @@
static void purge_vmap_area_lazy(void);
+static BLOCKING_NOTIFIER_HEAD(vmap_notify_list);
+
/*
* Allocate a region of KVA of the specified size and alignment, within the
* vstart and vend.
@@ -363,6 +365,8 @@
BUG_ON(offset_in_page(size));
BUG_ON(!is_power_of_2(align));
+ might_sleep_if(gfpflags_allow_blocking(gfp_mask));
+
va = kmalloc_node(sizeof(struct vmap_area),
gfp_mask & GFP_RECLAIM_MASK, node);
if (unlikely(!va))
@@ -468,6 +472,16 @@
purged = 1;
goto retry;
}
+
+ if (gfpflags_allow_blocking(gfp_mask)) {
+ unsigned long freed = 0;
+ blocking_notifier_call_chain(&vmap_notify_list, 0, &freed);
+ if (freed > 0) {
+ purged = 0;
+ goto retry;
+ }
+ }
+
if (printk_ratelimit())
pr_warn("vmap allocation for size %lu failed: use vmalloc=<size> to increase size\n",
size);
@@ -475,6 +489,18 @@
return ERR_PTR(-EBUSY);
}
+int register_vmap_purge_notifier(struct notifier_block *nb)
+{
+ return blocking_notifier_chain_register(&vmap_notify_list, nb);
+}
+EXPORT_SYMBOL_GPL(register_vmap_purge_notifier);
+
+int unregister_vmap_purge_notifier(struct notifier_block *nb)
+{
+ return blocking_notifier_chain_unregister(&vmap_notify_list, nb);
+}
+EXPORT_SYMBOL_GPL(unregister_vmap_purge_notifier);
+
static void __free_vmap_area(struct vmap_area *va)
{
BUG_ON(RB_EMPTY_NODE(&va->rb_node));
@@ -601,7 +627,7 @@
int sync, int force_flush)
{
static DEFINE_SPINLOCK(purge_lock);
- LIST_HEAD(valist);
+ struct llist_node *valist;
struct vmap_area *va;
struct vmap_area *n_va;
int nr = 0;
@@ -620,20 +646,14 @@
if (sync)
purge_fragmented_blocks_allcpus();
- rcu_read_lock();
- list_for_each_entry_rcu(va, &vmap_area_list, list) {
- if (va->flags & VM_LAZY_FREE) {
- if (va->va_start < *start)
- *start = va->va_start;
- if (va->va_end > *end)
- *end = va->va_end;
- nr += (va->va_end - va->va_start) >> PAGE_SHIFT;
- list_add_tail(&va->purge_list, &valist);
- va->flags |= VM_LAZY_FREEING;
- va->flags &= ~VM_LAZY_FREE;
- }
+ valist = llist_del_all(&vmap_purge_list);
+ llist_for_each_entry(va, valist, purge_list) {
+ if (va->va_start < *start)
+ *start = va->va_start;
+ if (va->va_end > *end)
+ *end = va->va_end;
+ nr += (va->va_end - va->va_start) >> PAGE_SHIFT;
}
- rcu_read_unlock();
if (nr)
atomic_sub(nr, &vmap_lazy_nr);
@@ -643,7 +663,7 @@
if (nr) {
spin_lock(&vmap_area_lock);
- list_for_each_entry_safe(va, n_va, &valist, purge_list)
+ llist_for_each_entry_safe(va, n_va, valist, purge_list)
__free_vmap_area(va);
spin_unlock(&vmap_area_lock);
}
@@ -678,9 +698,15 @@
*/
static void free_vmap_area_noflush(struct vmap_area *va)
{
- va->flags |= VM_LAZY_FREE;
- atomic_add((va->va_end - va->va_start) >> PAGE_SHIFT, &vmap_lazy_nr);
- if (unlikely(atomic_read(&vmap_lazy_nr) > lazy_max_pages()))
+ int nr_lazy;
+
+ nr_lazy = atomic_add_return((va->va_end - va->va_start) >> PAGE_SHIFT,
+ &vmap_lazy_nr);
+
+ /* After this point, we may free va at any time */
+ llist_add(&va->purge_list, &vmap_purge_list);
+
+ if (unlikely(nr_lazy > lazy_max_pages()))
try_purge_vmap_area_lazy();
}
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 142cb61..c4a2f451 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -191,7 +191,7 @@
}
#endif
-static unsigned long zone_reclaimable_pages(struct zone *zone)
+unsigned long zone_reclaimable_pages(struct zone *zone)
{
unsigned long nr;
@@ -633,7 +633,7 @@
*
* Reversing the order of the tests ensures such a situation cannot
* escape unnoticed. The smp_rmb is needed to ensure the page->flags
- * load is not satisfied before that of page->_count.
+ * load is not satisfied before that of page->_refcount.
*
* Note that if SetPageDirty is always performed via set_page_dirty,
* and thus under tree_lock, then this ordering is not required.
@@ -1374,7 +1374,6 @@
for (scan = 0; scan < nr_to_scan && nr_taken < nr_to_scan &&
!list_empty(src); scan++) {
struct page *page;
- int nr_pages;
page = lru_to_page(src);
prefetchw_prev_lru_page(page, src, flags);
@@ -1383,10 +1382,8 @@
switch (__isolate_lru_page(page, mode)) {
case 0:
- nr_pages = hpage_nr_pages(page);
- mem_cgroup_update_lru_size(lruvec, lru, -nr_pages);
+ nr_taken += hpage_nr_pages(page);
list_move(&page->lru, dst);
- nr_taken += nr_pages;
break;
case -EBUSY:
@@ -1602,8 +1599,9 @@
nr_taken = isolate_lru_pages(nr_to_scan, lruvec, &page_list,
&nr_scanned, sc, isolate_mode, lru);
- __mod_zone_page_state(zone, NR_LRU_BASE + lru, -nr_taken);
+ update_lru_size(lruvec, lru, -nr_taken);
__mod_zone_page_state(zone, NR_ISOLATED_ANON + file, nr_taken);
+ reclaim_stat->recent_scanned[file] += nr_taken;
if (global_reclaim(sc)) {
__mod_zone_page_state(zone, NR_PAGES_SCANNED, nr_scanned);
@@ -1624,8 +1622,6 @@
spin_lock_irq(&zone->lru_lock);
- reclaim_stat->recent_scanned[file] += nr_taken;
-
if (global_reclaim(sc)) {
if (current_is_kswapd())
__count_zone_vm_events(PGSTEAL_KSWAPD, zone,
@@ -1720,7 +1716,7 @@
* It is safe to rely on PG_active against the non-LRU pages in here because
* nobody will play with that bit on a non-LRU page.
*
- * The downside is that we have to touch page->_count against each page.
+ * The downside is that we have to touch page->_refcount against each page.
* But we had to alter page->flags anyway.
*/
@@ -1742,7 +1738,7 @@
SetPageLRU(page);
nr_pages = hpage_nr_pages(page);
- mem_cgroup_update_lru_size(lruvec, lru, nr_pages);
+ update_lru_size(lruvec, lru, nr_pages);
list_move(&page->lru, &lruvec->lists[lru]);
pgmoved += nr_pages;
@@ -1760,7 +1756,7 @@
list_add(&page->lru, pages_to_free);
}
}
- __mod_zone_page_state(zone, NR_LRU_BASE + lru, pgmoved);
+
if (!is_active_lru(lru))
__count_vm_events(PGDEACTIVATE, pgmoved);
}
@@ -1794,14 +1790,15 @@
nr_taken = isolate_lru_pages(nr_to_scan, lruvec, &l_hold,
&nr_scanned, sc, isolate_mode, lru);
- if (global_reclaim(sc))
- __mod_zone_page_state(zone, NR_PAGES_SCANNED, nr_scanned);
+ update_lru_size(lruvec, lru, -nr_taken);
+ __mod_zone_page_state(zone, NR_ISOLATED_ANON + file, nr_taken);
reclaim_stat->recent_scanned[file] += nr_taken;
+ if (global_reclaim(sc))
+ __mod_zone_page_state(zone, NR_PAGES_SCANNED, nr_scanned);
__count_zone_vm_events(PGREFILL, zone, nr_scanned);
- __mod_zone_page_state(zone, NR_LRU_BASE + lru, -nr_taken);
- __mod_zone_page_state(zone, NR_ISOLATED_ANON + file, nr_taken);
+
spin_unlock_irq(&zone->lru_lock);
while (!list_empty(&l_hold)) {
@@ -1865,83 +1862,63 @@
free_hot_cold_page_list(&l_hold, true);
}
-#ifdef CONFIG_SWAP
-static bool inactive_anon_is_low_global(struct zone *zone)
-{
- unsigned long active, inactive;
-
- active = zone_page_state(zone, NR_ACTIVE_ANON);
- inactive = zone_page_state(zone, NR_INACTIVE_ANON);
-
- return inactive * zone->inactive_ratio < active;
-}
-
-/**
- * inactive_anon_is_low - check if anonymous pages need to be deactivated
- * @lruvec: LRU vector to check
+/*
+ * The inactive anon list should be small enough that the VM never has
+ * to do too much work.
*
- * Returns true if the zone does not have enough inactive anon pages,
- * meaning some active anon pages need to be deactivated.
+ * The inactive file list should be small enough to leave most memory
+ * to the established workingset on the scan-resistant active list,
+ * but large enough to avoid thrashing the aggregate readahead window.
+ *
+ * Both inactive lists should also be large enough that each inactive
+ * page has a chance to be referenced again before it is reclaimed.
+ *
+ * The inactive_ratio is the target ratio of ACTIVE to INACTIVE pages
+ * on this LRU, maintained by the pageout code. A zone->inactive_ratio
+ * of 3 means 3:1 or 25% of the pages are kept on the inactive list.
+ *
+ * total target max
+ * memory ratio inactive
+ * -------------------------------------
+ * 10MB 1 5MB
+ * 100MB 1 50MB
+ * 1GB 3 250MB
+ * 10GB 10 0.9GB
+ * 100GB 31 3GB
+ * 1TB 101 10GB
+ * 10TB 320 32GB
*/
-static bool inactive_anon_is_low(struct lruvec *lruvec)
+static bool inactive_list_is_low(struct lruvec *lruvec, bool file)
{
+ unsigned long inactive_ratio;
+ unsigned long inactive;
+ unsigned long active;
+ unsigned long gb;
+
/*
* If we don't have swap space, anonymous page deactivation
* is pointless.
*/
- if (!total_swap_pages)
+ if (!file && !total_swap_pages)
return false;
- if (!mem_cgroup_disabled())
- return mem_cgroup_inactive_anon_is_low(lruvec);
+ inactive = lruvec_lru_size(lruvec, file * LRU_FILE);
+ active = lruvec_lru_size(lruvec, file * LRU_FILE + LRU_ACTIVE);
- return inactive_anon_is_low_global(lruvec_zone(lruvec));
-}
-#else
-static inline bool inactive_anon_is_low(struct lruvec *lruvec)
-{
- return false;
-}
-#endif
-
-/**
- * inactive_file_is_low - check if file pages need to be deactivated
- * @lruvec: LRU vector to check
- *
- * When the system is doing streaming IO, memory pressure here
- * ensures that active file pages get deactivated, until more
- * than half of the file pages are on the inactive list.
- *
- * Once we get to that situation, protect the system's working
- * set from being evicted by disabling active file page aging.
- *
- * This uses a different ratio than the anonymous pages, because
- * the page cache uses a use-once replacement algorithm.
- */
-static bool inactive_file_is_low(struct lruvec *lruvec)
-{
- unsigned long inactive;
- unsigned long active;
-
- inactive = lruvec_lru_size(lruvec, LRU_INACTIVE_FILE);
- active = lruvec_lru_size(lruvec, LRU_ACTIVE_FILE);
-
- return active > inactive;
-}
-
-static bool inactive_list_is_low(struct lruvec *lruvec, enum lru_list lru)
-{
- if (is_file_lru(lru))
- return inactive_file_is_low(lruvec);
+ gb = (inactive + active) >> (30 - PAGE_SHIFT);
+ if (gb)
+ inactive_ratio = int_sqrt(10 * gb);
else
- return inactive_anon_is_low(lruvec);
+ inactive_ratio = 1;
+
+ return inactive * inactive_ratio < active;
}
static unsigned long shrink_list(enum lru_list lru, unsigned long nr_to_scan,
struct lruvec *lruvec, struct scan_control *sc)
{
if (is_active_lru(lru)) {
- if (inactive_list_is_low(lruvec, lru))
+ if (inactive_list_is_low(lruvec, is_file_lru(lru)))
shrink_active_list(nr_to_scan, lruvec, sc, lru);
return 0;
}
@@ -2062,7 +2039,7 @@
* lruvec even if it has plenty of old anonymous pages unless the
* system is under heavy pressure.
*/
- if (!inactive_file_is_low(lruvec) &&
+ if (!inactive_list_is_low(lruvec, true) &&
lruvec_lru_size(lruvec, LRU_INACTIVE_FILE) >> sc->priority) {
scan_balance = SCAN_FILE;
goto out;
@@ -2304,7 +2281,7 @@
* Even if we did not try to evict anon pages at all, we want to
* rebalance the anon lru active/inactive ratio.
*/
- if (inactive_anon_is_low(lruvec))
+ if (inactive_list_is_low(lruvec, false))
shrink_active_list(SWAP_CLUSTER_MAX, lruvec,
sc, LRU_ACTIVE_ANON);
@@ -2482,7 +2459,7 @@
* Returns true if compaction should go ahead for a high-order request, or
* the high-order allocation would succeed without compaction.
*/
-static inline bool compaction_ready(struct zone *zone, int order)
+static inline bool compaction_ready(struct zone *zone, int order, int classzone_idx)
{
unsigned long balance_gap, watermark;
bool watermark_ok;
@@ -2496,7 +2473,7 @@
balance_gap = min(low_wmark_pages(zone), DIV_ROUND_UP(
zone->managed_pages, KSWAPD_ZONE_BALANCE_GAP_RATIO));
watermark = high_wmark_pages(zone) + balance_gap + (2UL << order);
- watermark_ok = zone_watermark_ok_safe(zone, 0, watermark, 0);
+ watermark_ok = zone_watermark_ok_safe(zone, 0, watermark, classzone_idx);
/*
* If compaction is deferred, reclaim up to a point where
@@ -2509,7 +2486,7 @@
* If compaction is not ready to start and allocation is not likely
* to succeed without it, then keep reclaiming.
*/
- if (compaction_suitable(zone, order, 0, 0) == COMPACT_SKIPPED)
+ if (compaction_suitable(zone, order, 0, classzone_idx) == COMPACT_SKIPPED)
return false;
return watermark_ok;
@@ -2530,10 +2507,8 @@
*
* If a zone is deemed to be full of pinned pages then just give it a light
* scan then give up on it.
- *
- * Returns true if a zone was reclaimable.
*/
-static bool shrink_zones(struct zonelist *zonelist, struct scan_control *sc)
+static void shrink_zones(struct zonelist *zonelist, struct scan_control *sc)
{
struct zoneref *z;
struct zone *zone;
@@ -2541,7 +2516,6 @@
unsigned long nr_soft_scanned;
gfp_t orig_mask;
enum zone_type requested_highidx = gfp_zone(sc->gfp_mask);
- bool reclaimable = false;
/*
* If the number of buffer_heads in the machine exceeds the maximum
@@ -2589,7 +2563,7 @@
if (IS_ENABLED(CONFIG_COMPACTION) &&
sc->order > PAGE_ALLOC_COSTLY_ORDER &&
zonelist_zone_idx(z) <= requested_highidx &&
- compaction_ready(zone, sc->order)) {
+ compaction_ready(zone, sc->order, requested_highidx)) {
sc->compaction_ready = true;
continue;
}
@@ -2606,17 +2580,10 @@
&nr_soft_scanned);
sc->nr_reclaimed += nr_soft_reclaimed;
sc->nr_scanned += nr_soft_scanned;
- if (nr_soft_reclaimed)
- reclaimable = true;
/* need some check for avoid more shrink_zone() */
}
- if (shrink_zone(zone, sc, zone_idx(zone) == classzone_idx))
- reclaimable = true;
-
- if (global_reclaim(sc) &&
- !reclaimable && zone_reclaimable(zone))
- reclaimable = true;
+ shrink_zone(zone, sc, zone_idx(zone) == classzone_idx);
}
/*
@@ -2624,8 +2591,6 @@
* promoted it to __GFP_HIGHMEM.
*/
sc->gfp_mask = orig_mask;
-
- return reclaimable;
}
/*
@@ -2650,7 +2615,6 @@
int initial_priority = sc->priority;
unsigned long total_scanned = 0;
unsigned long writeback_threshold;
- bool zones_reclaimable;
retry:
delayacct_freepages_start();
@@ -2661,7 +2625,7 @@
vmpressure_prio(sc->gfp_mask, sc->target_mem_cgroup,
sc->priority);
sc->nr_scanned = 0;
- zones_reclaimable = shrink_zones(zonelist, sc);
+ shrink_zones(zonelist, sc);
total_scanned += sc->nr_scanned;
if (sc->nr_reclaimed >= sc->nr_to_reclaim)
@@ -2708,10 +2672,6 @@
goto retry;
}
- /* Any of the zones still reclaimable? Don't OOM. */
- if (zones_reclaimable)
- return 1;
-
return 0;
}
@@ -2965,7 +2925,7 @@
do {
struct lruvec *lruvec = mem_cgroup_zone_lruvec(zone, memcg);
- if (inactive_anon_is_low(lruvec))
+ if (inactive_list_is_low(lruvec, false))
shrink_active_list(SWAP_CLUSTER_MAX, lruvec,
sc, LRU_ACTIVE_ANON);
diff --git a/mm/vmstat.c b/mm/vmstat.c
index 5e43004..77e42ef 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -570,49 +570,18 @@
#ifdef CONFIG_NUMA
/*
- * zonelist = the list of zones passed to the allocator
- * z = the zone from which the allocation occurred.
- *
- * Must be called with interrupts disabled.
- *
- * When __GFP_OTHER_NODE is set assume the node of the preferred
- * zone is the local node. This is useful for daemons who allocate
- * memory on behalf of other processes.
- */
-void zone_statistics(struct zone *preferred_zone, struct zone *z, gfp_t flags)
-{
- if (z->zone_pgdat == preferred_zone->zone_pgdat) {
- __inc_zone_state(z, NUMA_HIT);
- } else {
- __inc_zone_state(z, NUMA_MISS);
- __inc_zone_state(preferred_zone, NUMA_FOREIGN);
- }
- if (z->node == ((flags & __GFP_OTHER_NODE) ?
- preferred_zone->node : numa_node_id()))
- __inc_zone_state(z, NUMA_LOCAL);
- else
- __inc_zone_state(z, NUMA_OTHER);
-}
-
-/*
* Determine the per node value of a stat item.
*/
unsigned long node_page_state(int node, enum zone_stat_item item)
{
struct zone *zones = NODE_DATA(node)->node_zones;
+ int i;
+ unsigned long count = 0;
- return
-#ifdef CONFIG_ZONE_DMA
- zone_page_state(&zones[ZONE_DMA], item) +
-#endif
-#ifdef CONFIG_ZONE_DMA32
- zone_page_state(&zones[ZONE_DMA32], item) +
-#endif
-#ifdef CONFIG_HIGHMEM
- zone_page_state(&zones[ZONE_HIGHMEM], item) +
-#endif
- zone_page_state(&zones[ZONE_NORMAL], item) +
- zone_page_state(&zones[ZONE_MOVABLE], item);
+ for (i = 0; i < MAX_NR_ZONES; i++)
+ count += zone_page_state(zones + i, item);
+
+ return count;
}
#endif
@@ -1010,6 +979,9 @@
if (!memmap_valid_within(pfn, page, zone))
continue;
+ if (page_zone(page) != zone)
+ continue;
+
mtype = get_pageblock_migratetype(page);
if (mtype < MIGRATE_TYPES)
@@ -1069,13 +1041,17 @@
block_end_pfn = min(block_end_pfn, end_pfn);
page = pfn_to_page(pfn);
- pageblock_mt = get_pfnblock_migratetype(page, pfn);
+ pageblock_mt = get_pageblock_migratetype(page);
for (; pfn < block_end_pfn; pfn++) {
if (!pfn_valid_within(pfn))
continue;
page = pfn_to_page(pfn);
+
+ if (page_zone(page) != zone)
+ continue;
+
if (PageBuddy(page)) {
pfn += (1UL << page_order(page)) - 1;
continue;
@@ -1376,7 +1352,66 @@
static struct workqueue_struct *vmstat_wq;
static DEFINE_PER_CPU(struct delayed_work, vmstat_work);
int sysctl_stat_interval __read_mostly = HZ;
-static cpumask_var_t cpu_stat_off;
+
+#ifdef CONFIG_PROC_FS
+static void refresh_vm_stats(struct work_struct *work)
+{
+ refresh_cpu_vm_stats(true);
+}
+
+int vmstat_refresh(struct ctl_table *table, int write,
+ void __user *buffer, size_t *lenp, loff_t *ppos)
+{
+ long val;
+ int err;
+ int i;
+
+ /*
+ * The regular update, every sysctl_stat_interval, may come later
+ * than expected: leaving a significant amount in per_cpu buckets.
+ * This is particularly misleading when checking a quantity of HUGE
+ * pages, immediately after running a test. /proc/sys/vm/stat_refresh,
+ * which can equally be echo'ed to or cat'ted from (by root),
+ * can be used to update the stats just before reading them.
+ *
+ * Oh, and since global_page_state() etc. are so careful to hide
+ * transiently negative values, report an error here if any of
+ * the stats is negative, so we know to go looking for imbalance.
+ */
+ err = schedule_on_each_cpu(refresh_vm_stats);
+ if (err)
+ return err;
+ for (i = 0; i < NR_VM_ZONE_STAT_ITEMS; i++) {
+ val = atomic_long_read(&vm_stat[i]);
+ if (val < 0) {
+ switch (i) {
+ case NR_ALLOC_BATCH:
+ case NR_PAGES_SCANNED:
+ /*
+ * These are often seen to go negative in
+ * recent kernels, but not to go permanently
+ * negative. Whilst it would be nicer not to
+ * have exceptions, rooting them out would be
+ * another task, of rather low priority.
+ */
+ break;
+ default:
+ pr_warn("%s: %s %ld\n",
+ __func__, vmstat_text[i], val);
+ err = -EINVAL;
+ break;
+ }
+ }
+ }
+ if (err)
+ return err;
+ if (write)
+ *ppos += *lenp;
+ else
+ *lenp = 0;
+ return 0;
+}
+#endif /* CONFIG_PROC_FS */
static void vmstat_update(struct work_struct *w)
{
@@ -1385,24 +1420,10 @@
* Counters were updated so we expect more updates
* to occur in the future. Keep on running the
* update worker thread.
- * If we were marked on cpu_stat_off clear the flag
- * so that vmstat_shepherd doesn't schedule us again.
*/
- if (!cpumask_test_and_clear_cpu(smp_processor_id(),
- cpu_stat_off)) {
- queue_delayed_work_on(smp_processor_id(), vmstat_wq,
+ queue_delayed_work_on(smp_processor_id(), vmstat_wq,
this_cpu_ptr(&vmstat_work),
round_jiffies_relative(sysctl_stat_interval));
- }
- } else {
- /*
- * We did not update any counters so the app may be in
- * a mode where it does not cause counter updates.
- * We may be uselessly running vmstat_update.
- * Defer the checking for differentials to the
- * shepherd thread on a different processor.
- */
- cpumask_set_cpu(smp_processor_id(), cpu_stat_off);
}
}
@@ -1434,16 +1455,17 @@
return false;
}
+/*
+ * Switch off vmstat processing and then fold all the remaining differentials
+ * until the diffs stay at zero. The function is used by NOHZ and can only be
+ * invoked when tick processing is not active.
+ */
void quiet_vmstat(void)
{
if (system_state != SYSTEM_RUNNING)
return;
- /*
- * If we are already in hands of the shepherd then there
- * is nothing for us to do here.
- */
- if (cpumask_test_and_set_cpu(smp_processor_id(), cpu_stat_off))
+ if (!delayed_work_pending(this_cpu_ptr(&vmstat_work)))
return;
if (!need_update(smp_processor_id()))
@@ -1458,7 +1480,6 @@
refresh_cpu_vm_stats(false);
}
-
/*
* Shepherd worker thread that checks the
* differentials of processors that have their worker
@@ -1475,20 +1496,11 @@
get_online_cpus();
/* Check processors whose vmstat worker threads have been disabled */
- for_each_cpu(cpu, cpu_stat_off) {
+ for_each_online_cpu(cpu) {
struct delayed_work *dw = &per_cpu(vmstat_work, cpu);
- if (need_update(cpu)) {
- if (cpumask_test_and_clear_cpu(cpu, cpu_stat_off))
- queue_delayed_work_on(cpu, vmstat_wq, dw, 0);
- } else {
- /*
- * Cancel the work if quiet_vmstat has put this
- * cpu on cpu_stat_off because the work item might
- * be still scheduled
- */
- cancel_delayed_work(dw);
- }
+ if (!delayed_work_pending(dw) && need_update(cpu))
+ queue_delayed_work_on(cpu, vmstat_wq, dw, 0);
}
put_online_cpus();
@@ -1504,10 +1516,6 @@
INIT_DEFERRABLE_WORK(per_cpu_ptr(&vmstat_work, cpu),
vmstat_update);
- if (!alloc_cpumask_var(&cpu_stat_off, GFP_KERNEL))
- BUG();
- cpumask_copy(cpu_stat_off, cpu_online_mask);
-
vmstat_wq = alloc_workqueue("vmstat", WQ_FREEZABLE|WQ_MEM_RECLAIM, 0);
schedule_delayed_work(&shepherd,
round_jiffies_relative(sysctl_stat_interval));
@@ -1542,16 +1550,13 @@
case CPU_ONLINE_FROZEN:
refresh_zone_stat_thresholds();
node_set_state(cpu_to_node(cpu), N_CPU);
- cpumask_set_cpu(cpu, cpu_stat_off);
break;
case CPU_DOWN_PREPARE:
case CPU_DOWN_PREPARE_FROZEN:
cancel_delayed_work_sync(&per_cpu(vmstat_work, cpu));
- cpumask_clear_cpu(cpu, cpu_stat_off);
break;
case CPU_DOWN_FAILED:
case CPU_DOWN_FAILED_FROZEN:
- cpumask_set_cpu(cpu, cpu_stat_off);
break;
case CPU_DEAD:
case CPU_DEAD_FROZEN:
diff --git a/mm/z3fold.c b/mm/z3fold.c
new file mode 100644
index 0000000..34917d5
--- /dev/null
+++ b/mm/z3fold.c
@@ -0,0 +1,792 @@
+/*
+ * z3fold.c
+ *
+ * Author: Vitaly Wool <vitaly.wool@konsulko.com>
+ * Copyright (C) 2016, Sony Mobile Communications Inc.
+ *
+ * This implementation is based on zbud written by Seth Jennings.
+ *
+ * z3fold is an special purpose allocator for storing compressed pages. It
+ * can store up to three compressed pages per page which improves the
+ * compression ratio of zbud while retaining its main concepts (e. g. always
+ * storing an integral number of objects per page) and simplicity.
+ * It still has simple and deterministic reclaim properties that make it
+ * preferable to a higher density approach (with no requirement on integral
+ * number of object per page) when reclaim is used.
+ *
+ * As in zbud, pages are divided into "chunks". The size of the chunks is
+ * fixed at compile time and is determined by NCHUNKS_ORDER below.
+ *
+ * z3fold doesn't export any API and is meant to be used via zpool API.
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/atomic.h>
+#include <linux/list.h>
+#include <linux/mm.h>
+#include <linux/module.h>
+#include <linux/preempt.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/zpool.h>
+
+/*****************
+ * Structures
+*****************/
+/*
+ * NCHUNKS_ORDER determines the internal allocation granularity, effectively
+ * adjusting internal fragmentation. It also determines the number of
+ * freelists maintained in each pool. NCHUNKS_ORDER of 6 means that the
+ * allocation granularity will be in chunks of size PAGE_SIZE/64. As one chunk
+ * in allocated page is occupied by z3fold header, NCHUNKS will be calculated
+ * to 63 which shows the max number of free chunks in z3fold page, also there
+ * will be 63 freelists per pool.
+ */
+#define NCHUNKS_ORDER 6
+
+#define CHUNK_SHIFT (PAGE_SHIFT - NCHUNKS_ORDER)
+#define CHUNK_SIZE (1 << CHUNK_SHIFT)
+#define ZHDR_SIZE_ALIGNED CHUNK_SIZE
+#define NCHUNKS ((PAGE_SIZE - ZHDR_SIZE_ALIGNED) >> CHUNK_SHIFT)
+
+#define BUDDY_MASK ((1 << NCHUNKS_ORDER) - 1)
+
+struct z3fold_pool;
+struct z3fold_ops {
+ int (*evict)(struct z3fold_pool *pool, unsigned long handle);
+};
+
+/**
+ * struct z3fold_pool - stores metadata for each z3fold pool
+ * @lock: protects all pool fields and first|last_chunk fields of any
+ * z3fold page in the pool
+ * @unbuddied: array of lists tracking z3fold pages that contain 2- buddies;
+ * the lists each z3fold page is added to depends on the size of
+ * its free region.
+ * @buddied: list tracking the z3fold pages that contain 3 buddies;
+ * these z3fold pages are full
+ * @lru: list tracking the z3fold pages in LRU order by most recently
+ * added buddy.
+ * @pages_nr: number of z3fold pages in the pool.
+ * @ops: pointer to a structure of user defined operations specified at
+ * pool creation time.
+ *
+ * This structure is allocated at pool creation time and maintains metadata
+ * pertaining to a particular z3fold pool.
+ */
+struct z3fold_pool {
+ spinlock_t lock;
+ struct list_head unbuddied[NCHUNKS];
+ struct list_head buddied;
+ struct list_head lru;
+ u64 pages_nr;
+ const struct z3fold_ops *ops;
+ struct zpool *zpool;
+ const struct zpool_ops *zpool_ops;
+};
+
+enum buddy {
+ HEADLESS = 0,
+ FIRST,
+ MIDDLE,
+ LAST,
+ BUDDIES_MAX
+};
+
+/*
+ * struct z3fold_header - z3fold page metadata occupying the first chunk of each
+ * z3fold page, except for HEADLESS pages
+ * @buddy: links the z3fold page into the relevant list in the pool
+ * @first_chunks: the size of the first buddy in chunks, 0 if free
+ * @middle_chunks: the size of the middle buddy in chunks, 0 if free
+ * @last_chunks: the size of the last buddy in chunks, 0 if free
+ * @first_num: the starting number (for the first handle)
+ */
+struct z3fold_header {
+ struct list_head buddy;
+ unsigned short first_chunks;
+ unsigned short middle_chunks;
+ unsigned short last_chunks;
+ unsigned short start_middle;
+ unsigned short first_num:NCHUNKS_ORDER;
+};
+
+/*
+ * Internal z3fold page flags
+ */
+enum z3fold_page_flags {
+ UNDER_RECLAIM = 0,
+ PAGE_HEADLESS,
+ MIDDLE_CHUNK_MAPPED,
+};
+
+/*****************
+ * Helpers
+*****************/
+
+/* Converts an allocation size in bytes to size in z3fold chunks */
+static int size_to_chunks(size_t size)
+{
+ return (size + CHUNK_SIZE - 1) >> CHUNK_SHIFT;
+}
+
+#define for_each_unbuddied_list(_iter, _begin) \
+ for ((_iter) = (_begin); (_iter) < NCHUNKS; (_iter)++)
+
+/* Initializes the z3fold header of a newly allocated z3fold page */
+static struct z3fold_header *init_z3fold_page(struct page *page)
+{
+ struct z3fold_header *zhdr = page_address(page);
+
+ INIT_LIST_HEAD(&page->lru);
+ clear_bit(UNDER_RECLAIM, &page->private);
+ clear_bit(PAGE_HEADLESS, &page->private);
+ clear_bit(MIDDLE_CHUNK_MAPPED, &page->private);
+
+ zhdr->first_chunks = 0;
+ zhdr->middle_chunks = 0;
+ zhdr->last_chunks = 0;
+ zhdr->first_num = 0;
+ zhdr->start_middle = 0;
+ INIT_LIST_HEAD(&zhdr->buddy);
+ return zhdr;
+}
+
+/* Resets the struct page fields and frees the page */
+static void free_z3fold_page(struct z3fold_header *zhdr)
+{
+ __free_page(virt_to_page(zhdr));
+}
+
+/*
+ * Encodes the handle of a particular buddy within a z3fold page
+ * Pool lock should be held as this function accesses first_num
+ */
+static unsigned long encode_handle(struct z3fold_header *zhdr, enum buddy bud)
+{
+ unsigned long handle;
+
+ handle = (unsigned long)zhdr;
+ if (bud != HEADLESS)
+ handle += (bud + zhdr->first_num) & BUDDY_MASK;
+ return handle;
+}
+
+/* Returns the z3fold page where a given handle is stored */
+static struct z3fold_header *handle_to_z3fold_header(unsigned long handle)
+{
+ return (struct z3fold_header *)(handle & PAGE_MASK);
+}
+
+/* Returns buddy number */
+static enum buddy handle_to_buddy(unsigned long handle)
+{
+ struct z3fold_header *zhdr = handle_to_z3fold_header(handle);
+ return (handle - zhdr->first_num) & BUDDY_MASK;
+}
+
+/*
+ * Returns the number of free chunks in a z3fold page.
+ * NB: can't be used with HEADLESS pages.
+ */
+static int num_free_chunks(struct z3fold_header *zhdr)
+{
+ int nfree;
+ /*
+ * If there is a middle object, pick up the bigger free space
+ * either before or after it. Otherwise just subtract the number
+ * of chunks occupied by the first and the last objects.
+ */
+ if (zhdr->middle_chunks != 0) {
+ int nfree_before = zhdr->first_chunks ?
+ 0 : zhdr->start_middle - 1;
+ int nfree_after = zhdr->last_chunks ?
+ 0 : NCHUNKS - zhdr->start_middle - zhdr->middle_chunks;
+ nfree = max(nfree_before, nfree_after);
+ } else
+ nfree = NCHUNKS - zhdr->first_chunks - zhdr->last_chunks;
+ return nfree;
+}
+
+/*****************
+ * API Functions
+*****************/
+/**
+ * z3fold_create_pool() - create a new z3fold pool
+ * @gfp: gfp flags when allocating the z3fold pool structure
+ * @ops: user-defined operations for the z3fold pool
+ *
+ * Return: pointer to the new z3fold pool or NULL if the metadata allocation
+ * failed.
+ */
+static struct z3fold_pool *z3fold_create_pool(gfp_t gfp,
+ const struct z3fold_ops *ops)
+{
+ struct z3fold_pool *pool;
+ int i;
+
+ pool = kzalloc(sizeof(struct z3fold_pool), gfp);
+ if (!pool)
+ return NULL;
+ spin_lock_init(&pool->lock);
+ for_each_unbuddied_list(i, 0)
+ INIT_LIST_HEAD(&pool->unbuddied[i]);
+ INIT_LIST_HEAD(&pool->buddied);
+ INIT_LIST_HEAD(&pool->lru);
+ pool->pages_nr = 0;
+ pool->ops = ops;
+ return pool;
+}
+
+/**
+ * z3fold_destroy_pool() - destroys an existing z3fold pool
+ * @pool: the z3fold pool to be destroyed
+ *
+ * The pool should be emptied before this function is called.
+ */
+static void z3fold_destroy_pool(struct z3fold_pool *pool)
+{
+ kfree(pool);
+}
+
+/* Has to be called with lock held */
+static int z3fold_compact_page(struct z3fold_header *zhdr)
+{
+ struct page *page = virt_to_page(zhdr);
+ void *beg = zhdr;
+
+
+ if (!test_bit(MIDDLE_CHUNK_MAPPED, &page->private) &&
+ zhdr->middle_chunks != 0 &&
+ zhdr->first_chunks == 0 && zhdr->last_chunks == 0) {
+ memmove(beg + ZHDR_SIZE_ALIGNED,
+ beg + (zhdr->start_middle << CHUNK_SHIFT),
+ zhdr->middle_chunks << CHUNK_SHIFT);
+ zhdr->first_chunks = zhdr->middle_chunks;
+ zhdr->middle_chunks = 0;
+ zhdr->start_middle = 0;
+ zhdr->first_num++;
+ return 1;
+ }
+ return 0;
+}
+
+/**
+ * z3fold_alloc() - allocates a region of a given size
+ * @pool: z3fold pool from which to allocate
+ * @size: size in bytes of the desired allocation
+ * @gfp: gfp flags used if the pool needs to grow
+ * @handle: handle of the new allocation
+ *
+ * This function will attempt to find a free region in the pool large enough to
+ * satisfy the allocation request. A search of the unbuddied lists is
+ * performed first. If no suitable free region is found, then a new page is
+ * allocated and added to the pool to satisfy the request.
+ *
+ * gfp should not set __GFP_HIGHMEM as highmem pages cannot be used
+ * as z3fold pool pages.
+ *
+ * Return: 0 if success and handle is set, otherwise -EINVAL if the size or
+ * gfp arguments are invalid or -ENOMEM if the pool was unable to allocate
+ * a new page.
+ */
+static int z3fold_alloc(struct z3fold_pool *pool, size_t size, gfp_t gfp,
+ unsigned long *handle)
+{
+ int chunks = 0, i, freechunks;
+ struct z3fold_header *zhdr = NULL;
+ enum buddy bud;
+ struct page *page;
+
+ if (!size || (gfp & __GFP_HIGHMEM))
+ return -EINVAL;
+
+ if (size > PAGE_SIZE)
+ return -ENOSPC;
+
+ if (size > PAGE_SIZE - ZHDR_SIZE_ALIGNED - CHUNK_SIZE)
+ bud = HEADLESS;
+ else {
+ chunks = size_to_chunks(size);
+ spin_lock(&pool->lock);
+
+ /* First, try to find an unbuddied z3fold page. */
+ zhdr = NULL;
+ for_each_unbuddied_list(i, chunks) {
+ if (!list_empty(&pool->unbuddied[i])) {
+ zhdr = list_first_entry(&pool->unbuddied[i],
+ struct z3fold_header, buddy);
+ page = virt_to_page(zhdr);
+ if (zhdr->first_chunks == 0) {
+ if (zhdr->middle_chunks != 0 &&
+ chunks >= zhdr->start_middle)
+ bud = LAST;
+ else
+ bud = FIRST;
+ } else if (zhdr->last_chunks == 0)
+ bud = LAST;
+ else if (zhdr->middle_chunks == 0)
+ bud = MIDDLE;
+ else {
+ pr_err("No free chunks in unbuddied\n");
+ WARN_ON(1);
+ continue;
+ }
+ list_del(&zhdr->buddy);
+ goto found;
+ }
+ }
+ bud = FIRST;
+ spin_unlock(&pool->lock);
+ }
+
+ /* Couldn't find unbuddied z3fold page, create new one */
+ page = alloc_page(gfp);
+ if (!page)
+ return -ENOMEM;
+ spin_lock(&pool->lock);
+ pool->pages_nr++;
+ zhdr = init_z3fold_page(page);
+
+ if (bud == HEADLESS) {
+ set_bit(PAGE_HEADLESS, &page->private);
+ goto headless;
+ }
+
+found:
+ if (bud == FIRST)
+ zhdr->first_chunks = chunks;
+ else if (bud == LAST)
+ zhdr->last_chunks = chunks;
+ else {
+ zhdr->middle_chunks = chunks;
+ zhdr->start_middle = zhdr->first_chunks + 1;
+ }
+
+ if (zhdr->first_chunks == 0 || zhdr->last_chunks == 0 ||
+ zhdr->middle_chunks == 0) {
+ /* Add to unbuddied list */
+ freechunks = num_free_chunks(zhdr);
+ list_add(&zhdr->buddy, &pool->unbuddied[freechunks]);
+ } else {
+ /* Add to buddied list */
+ list_add(&zhdr->buddy, &pool->buddied);
+ }
+
+headless:
+ /* Add/move z3fold page to beginning of LRU */
+ if (!list_empty(&page->lru))
+ list_del(&page->lru);
+
+ list_add(&page->lru, &pool->lru);
+
+ *handle = encode_handle(zhdr, bud);
+ spin_unlock(&pool->lock);
+
+ return 0;
+}
+
+/**
+ * z3fold_free() - frees the allocation associated with the given handle
+ * @pool: pool in which the allocation resided
+ * @handle: handle associated with the allocation returned by z3fold_alloc()
+ *
+ * In the case that the z3fold page in which the allocation resides is under
+ * reclaim, as indicated by the PG_reclaim flag being set, this function
+ * only sets the first|last_chunks to 0. The page is actually freed
+ * once both buddies are evicted (see z3fold_reclaim_page() below).
+ */
+static void z3fold_free(struct z3fold_pool *pool, unsigned long handle)
+{
+ struct z3fold_header *zhdr;
+ int freechunks;
+ struct page *page;
+ enum buddy bud;
+
+ spin_lock(&pool->lock);
+ zhdr = handle_to_z3fold_header(handle);
+ page = virt_to_page(zhdr);
+
+ if (test_bit(PAGE_HEADLESS, &page->private)) {
+ /* HEADLESS page stored */
+ bud = HEADLESS;
+ } else {
+ bud = (handle - zhdr->first_num) & BUDDY_MASK;
+
+ switch (bud) {
+ case FIRST:
+ zhdr->first_chunks = 0;
+ break;
+ case MIDDLE:
+ zhdr->middle_chunks = 0;
+ zhdr->start_middle = 0;
+ break;
+ case LAST:
+ zhdr->last_chunks = 0;
+ break;
+ default:
+ pr_err("%s: unknown bud %d\n", __func__, bud);
+ WARN_ON(1);
+ spin_unlock(&pool->lock);
+ return;
+ }
+ }
+
+ if (test_bit(UNDER_RECLAIM, &page->private)) {
+ /* z3fold page is under reclaim, reclaim will free */
+ spin_unlock(&pool->lock);
+ return;
+ }
+
+ if (bud != HEADLESS) {
+ /* Remove from existing buddy list */
+ list_del(&zhdr->buddy);
+ }
+
+ if (bud == HEADLESS ||
+ (zhdr->first_chunks == 0 && zhdr->middle_chunks == 0 &&
+ zhdr->last_chunks == 0)) {
+ /* z3fold page is empty, free */
+ list_del(&page->lru);
+ clear_bit(PAGE_HEADLESS, &page->private);
+ free_z3fold_page(zhdr);
+ pool->pages_nr--;
+ } else {
+ z3fold_compact_page(zhdr);
+ /* Add to the unbuddied list */
+ freechunks = num_free_chunks(zhdr);
+ list_add(&zhdr->buddy, &pool->unbuddied[freechunks]);
+ }
+
+ spin_unlock(&pool->lock);
+}
+
+/**
+ * z3fold_reclaim_page() - evicts allocations from a pool page and frees it
+ * @pool: pool from which a page will attempt to be evicted
+ * @retires: number of pages on the LRU list for which eviction will
+ * be attempted before failing
+ *
+ * z3fold reclaim is different from normal system reclaim in that it is done
+ * from the bottom, up. This is because only the bottom layer, z3fold, has
+ * information on how the allocations are organized within each z3fold page.
+ * This has the potential to create interesting locking situations between
+ * z3fold and the user, however.
+ *
+ * To avoid these, this is how z3fold_reclaim_page() should be called:
+
+ * The user detects a page should be reclaimed and calls z3fold_reclaim_page().
+ * z3fold_reclaim_page() will remove a z3fold page from the pool LRU list and
+ * call the user-defined eviction handler with the pool and handle as
+ * arguments.
+ *
+ * If the handle can not be evicted, the eviction handler should return
+ * non-zero. z3fold_reclaim_page() will add the z3fold page back to the
+ * appropriate list and try the next z3fold page on the LRU up to
+ * a user defined number of retries.
+ *
+ * If the handle is successfully evicted, the eviction handler should
+ * return 0 _and_ should have called z3fold_free() on the handle. z3fold_free()
+ * contains logic to delay freeing the page if the page is under reclaim,
+ * as indicated by the setting of the PG_reclaim flag on the underlying page.
+ *
+ * If all buddies in the z3fold page are successfully evicted, then the
+ * z3fold page can be freed.
+ *
+ * Returns: 0 if page is successfully freed, otherwise -EINVAL if there are
+ * no pages to evict or an eviction handler is not registered, -EAGAIN if
+ * the retry limit was hit.
+ */
+static int z3fold_reclaim_page(struct z3fold_pool *pool, unsigned int retries)
+{
+ int i, ret = 0, freechunks;
+ struct z3fold_header *zhdr;
+ struct page *page;
+ unsigned long first_handle = 0, middle_handle = 0, last_handle = 0;
+
+ spin_lock(&pool->lock);
+ if (!pool->ops || !pool->ops->evict || list_empty(&pool->lru) ||
+ retries == 0) {
+ spin_unlock(&pool->lock);
+ return -EINVAL;
+ }
+ for (i = 0; i < retries; i++) {
+ page = list_last_entry(&pool->lru, struct page, lru);
+ list_del(&page->lru);
+
+ /* Protect z3fold page against free */
+ set_bit(UNDER_RECLAIM, &page->private);
+ zhdr = page_address(page);
+ if (!test_bit(PAGE_HEADLESS, &page->private)) {
+ list_del(&zhdr->buddy);
+ /*
+ * We need encode the handles before unlocking, since
+ * we can race with free that will set
+ * (first|last)_chunks to 0
+ */
+ first_handle = 0;
+ last_handle = 0;
+ middle_handle = 0;
+ if (zhdr->first_chunks)
+ first_handle = encode_handle(zhdr, FIRST);
+ if (zhdr->middle_chunks)
+ middle_handle = encode_handle(zhdr, MIDDLE);
+ if (zhdr->last_chunks)
+ last_handle = encode_handle(zhdr, LAST);
+ } else {
+ first_handle = encode_handle(zhdr, HEADLESS);
+ last_handle = middle_handle = 0;
+ }
+
+ spin_unlock(&pool->lock);
+
+ /* Issue the eviction callback(s) */
+ if (middle_handle) {
+ ret = pool->ops->evict(pool, middle_handle);
+ if (ret)
+ goto next;
+ }
+ if (first_handle) {
+ ret = pool->ops->evict(pool, first_handle);
+ if (ret)
+ goto next;
+ }
+ if (last_handle) {
+ ret = pool->ops->evict(pool, last_handle);
+ if (ret)
+ goto next;
+ }
+next:
+ spin_lock(&pool->lock);
+ clear_bit(UNDER_RECLAIM, &page->private);
+ if ((test_bit(PAGE_HEADLESS, &page->private) && ret == 0) ||
+ (zhdr->first_chunks == 0 && zhdr->last_chunks == 0 &&
+ zhdr->middle_chunks == 0)) {
+ /*
+ * All buddies are now free, free the z3fold page and
+ * return success.
+ */
+ clear_bit(PAGE_HEADLESS, &page->private);
+ free_z3fold_page(zhdr);
+ pool->pages_nr--;
+ spin_unlock(&pool->lock);
+ return 0;
+ } else if (zhdr->first_chunks != 0 &&
+ zhdr->last_chunks != 0 && zhdr->middle_chunks != 0) {
+ /* Full, add to buddied list */
+ list_add(&zhdr->buddy, &pool->buddied);
+ } else if (!test_bit(PAGE_HEADLESS, &page->private)) {
+ z3fold_compact_page(zhdr);
+ /* add to unbuddied list */
+ freechunks = num_free_chunks(zhdr);
+ list_add(&zhdr->buddy, &pool->unbuddied[freechunks]);
+ }
+
+ /* add to beginning of LRU */
+ list_add(&page->lru, &pool->lru);
+ }
+ spin_unlock(&pool->lock);
+ return -EAGAIN;
+}
+
+/**
+ * z3fold_map() - maps the allocation associated with the given handle
+ * @pool: pool in which the allocation resides
+ * @handle: handle associated with the allocation to be mapped
+ *
+ * Extracts the buddy number from handle and constructs the pointer to the
+ * correct starting chunk within the page.
+ *
+ * Returns: a pointer to the mapped allocation
+ */
+static void *z3fold_map(struct z3fold_pool *pool, unsigned long handle)
+{
+ struct z3fold_header *zhdr;
+ struct page *page;
+ void *addr;
+ enum buddy buddy;
+
+ spin_lock(&pool->lock);
+ zhdr = handle_to_z3fold_header(handle);
+ addr = zhdr;
+ page = virt_to_page(zhdr);
+
+ if (test_bit(PAGE_HEADLESS, &page->private))
+ goto out;
+
+ buddy = handle_to_buddy(handle);
+ switch (buddy) {
+ case FIRST:
+ addr += ZHDR_SIZE_ALIGNED;
+ break;
+ case MIDDLE:
+ addr += zhdr->start_middle << CHUNK_SHIFT;
+ set_bit(MIDDLE_CHUNK_MAPPED, &page->private);
+ break;
+ case LAST:
+ addr += PAGE_SIZE - (zhdr->last_chunks << CHUNK_SHIFT);
+ break;
+ default:
+ pr_err("unknown buddy id %d\n", buddy);
+ WARN_ON(1);
+ addr = NULL;
+ break;
+ }
+out:
+ spin_unlock(&pool->lock);
+ return addr;
+}
+
+/**
+ * z3fold_unmap() - unmaps the allocation associated with the given handle
+ * @pool: pool in which the allocation resides
+ * @handle: handle associated with the allocation to be unmapped
+ */
+static void z3fold_unmap(struct z3fold_pool *pool, unsigned long handle)
+{
+ struct z3fold_header *zhdr;
+ struct page *page;
+ enum buddy buddy;
+
+ spin_lock(&pool->lock);
+ zhdr = handle_to_z3fold_header(handle);
+ page = virt_to_page(zhdr);
+
+ if (test_bit(PAGE_HEADLESS, &page->private)) {
+ spin_unlock(&pool->lock);
+ return;
+ }
+
+ buddy = handle_to_buddy(handle);
+ if (buddy == MIDDLE)
+ clear_bit(MIDDLE_CHUNK_MAPPED, &page->private);
+ spin_unlock(&pool->lock);
+}
+
+/**
+ * z3fold_get_pool_size() - gets the z3fold pool size in pages
+ * @pool: pool whose size is being queried
+ *
+ * Returns: size in pages of the given pool. The pool lock need not be
+ * taken to access pages_nr.
+ */
+static u64 z3fold_get_pool_size(struct z3fold_pool *pool)
+{
+ return pool->pages_nr;
+}
+
+/*****************
+ * zpool
+ ****************/
+
+static int z3fold_zpool_evict(struct z3fold_pool *pool, unsigned long handle)
+{
+ if (pool->zpool && pool->zpool_ops && pool->zpool_ops->evict)
+ return pool->zpool_ops->evict(pool->zpool, handle);
+ else
+ return -ENOENT;
+}
+
+static const struct z3fold_ops z3fold_zpool_ops = {
+ .evict = z3fold_zpool_evict
+};
+
+static void *z3fold_zpool_create(const char *name, gfp_t gfp,
+ const struct zpool_ops *zpool_ops,
+ struct zpool *zpool)
+{
+ struct z3fold_pool *pool;
+
+ pool = z3fold_create_pool(gfp, zpool_ops ? &z3fold_zpool_ops : NULL);
+ if (pool) {
+ pool->zpool = zpool;
+ pool->zpool_ops = zpool_ops;
+ }
+ return pool;
+}
+
+static void z3fold_zpool_destroy(void *pool)
+{
+ z3fold_destroy_pool(pool);
+}
+
+static int z3fold_zpool_malloc(void *pool, size_t size, gfp_t gfp,
+ unsigned long *handle)
+{
+ return z3fold_alloc(pool, size, gfp, handle);
+}
+static void z3fold_zpool_free(void *pool, unsigned long handle)
+{
+ z3fold_free(pool, handle);
+}
+
+static int z3fold_zpool_shrink(void *pool, unsigned int pages,
+ unsigned int *reclaimed)
+{
+ unsigned int total = 0;
+ int ret = -EINVAL;
+
+ while (total < pages) {
+ ret = z3fold_reclaim_page(pool, 8);
+ if (ret < 0)
+ break;
+ total++;
+ }
+
+ if (reclaimed)
+ *reclaimed = total;
+
+ return ret;
+}
+
+static void *z3fold_zpool_map(void *pool, unsigned long handle,
+ enum zpool_mapmode mm)
+{
+ return z3fold_map(pool, handle);
+}
+static void z3fold_zpool_unmap(void *pool, unsigned long handle)
+{
+ z3fold_unmap(pool, handle);
+}
+
+static u64 z3fold_zpool_total_size(void *pool)
+{
+ return z3fold_get_pool_size(pool) * PAGE_SIZE;
+}
+
+static struct zpool_driver z3fold_zpool_driver = {
+ .type = "z3fold",
+ .owner = THIS_MODULE,
+ .create = z3fold_zpool_create,
+ .destroy = z3fold_zpool_destroy,
+ .malloc = z3fold_zpool_malloc,
+ .free = z3fold_zpool_free,
+ .shrink = z3fold_zpool_shrink,
+ .map = z3fold_zpool_map,
+ .unmap = z3fold_zpool_unmap,
+ .total_size = z3fold_zpool_total_size,
+};
+
+MODULE_ALIAS("zpool-z3fold");
+
+static int __init init_z3fold(void)
+{
+ /* Make sure the z3fold header will fit in one chunk */
+ BUILD_BUG_ON(sizeof(struct z3fold_header) > ZHDR_SIZE_ALIGNED);
+ zpool_register_driver(&z3fold_zpool_driver);
+
+ return 0;
+}
+
+static void __exit exit_z3fold(void)
+{
+ zpool_unregister_driver(&z3fold_zpool_driver);
+}
+
+module_init(init_z3fold);
+module_exit(exit_z3fold);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Vitaly Wool <vitalywool@gmail.com>");
+MODULE_DESCRIPTION("3-Fold Allocator for Compressed Pages");
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index fe47fbb..72698db 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -247,7 +247,6 @@
struct size_class **size_class;
struct kmem_cache *handle_cachep;
- gfp_t flags; /* allocation flags used when growing pool */
atomic_long_t pages_allocated;
struct zs_pool_stats stats;
@@ -295,10 +294,10 @@
kmem_cache_destroy(pool->handle_cachep);
}
-static unsigned long alloc_handle(struct zs_pool *pool)
+static unsigned long alloc_handle(struct zs_pool *pool, gfp_t gfp)
{
return (unsigned long)kmem_cache_alloc(pool->handle_cachep,
- pool->flags & ~__GFP_HIGHMEM);
+ gfp & ~__GFP_HIGHMEM);
}
static void free_handle(struct zs_pool *pool, unsigned long handle)
@@ -324,7 +323,12 @@
const struct zpool_ops *zpool_ops,
struct zpool *zpool)
{
- return zs_create_pool(name, gfp);
+ /*
+ * Ignore global gfp flags: zs_malloc() may be invoked from
+ * different contexts and its caller must provide a valid
+ * gfp mask.
+ */
+ return zs_create_pool(name);
}
static void zs_zpool_destroy(void *pool)
@@ -335,7 +339,7 @@
static int zs_zpool_malloc(void *pool, size_t size, gfp_t gfp,
unsigned long *handle)
{
- *handle = zs_malloc(pool, size);
+ *handle = zs_malloc(pool, size, gfp);
return *handle ? 0 : -1;
}
static void zs_zpool_free(void *pool, unsigned long handle)
@@ -413,26 +417,28 @@
return PagePrivate2(page);
}
-static void get_zspage_mapping(struct page *page, unsigned int *class_idx,
+static void get_zspage_mapping(struct page *first_page,
+ unsigned int *class_idx,
enum fullness_group *fullness)
{
unsigned long m;
- BUG_ON(!is_first_page(page));
+ VM_BUG_ON_PAGE(!is_first_page(first_page), first_page);
- m = (unsigned long)page->mapping;
+ m = (unsigned long)first_page->mapping;
*fullness = m & FULLNESS_MASK;
*class_idx = (m >> FULLNESS_BITS) & CLASS_IDX_MASK;
}
-static void set_zspage_mapping(struct page *page, unsigned int class_idx,
+static void set_zspage_mapping(struct page *first_page,
+ unsigned int class_idx,
enum fullness_group fullness)
{
unsigned long m;
- BUG_ON(!is_first_page(page));
+ VM_BUG_ON_PAGE(!is_first_page(first_page), first_page);
m = ((class_idx & CLASS_IDX_MASK) << FULLNESS_BITS) |
(fullness & FULLNESS_MASK);
- page->mapping = (struct address_space *)m;
+ first_page->mapping = (struct address_space *)m;
}
/*
@@ -567,17 +573,17 @@
.release = single_release,
};
-static int zs_pool_stat_create(const char *name, struct zs_pool *pool)
+static void zs_pool_stat_create(struct zs_pool *pool, const char *name)
{
struct dentry *entry;
if (!zs_stat_root)
- return -ENODEV;
+ return;
entry = debugfs_create_dir(name, zs_stat_root);
if (!entry) {
pr_warn("debugfs dir <%s> creation failed\n", name);
- return -ENOMEM;
+ return;
}
pool->stat_dentry = entry;
@@ -586,10 +592,8 @@
if (!entry) {
pr_warn("%s: debugfs file entry <%s> creation failed\n",
name, "classes");
- return -ENOMEM;
+ return;
}
-
- return 0;
}
static void zs_pool_stat_destroy(struct zs_pool *pool)
@@ -607,9 +611,8 @@
{
}
-static inline int zs_pool_stat_create(const char *name, struct zs_pool *pool)
+static inline void zs_pool_stat_create(struct zs_pool *pool, const char *name)
{
- return 0;
}
static inline void zs_pool_stat_destroy(struct zs_pool *pool)
@@ -617,7 +620,6 @@
}
#endif
-
/*
* For each size class, zspages are divided into different groups
* depending on how "full" they are. This was done so that we could
@@ -625,14 +627,15 @@
* the pool (not yet implemented). This function returns fullness
* status of the given page.
*/
-static enum fullness_group get_fullness_group(struct page *page)
+static enum fullness_group get_fullness_group(struct page *first_page)
{
int inuse, max_objects;
enum fullness_group fg;
- BUG_ON(!is_first_page(page));
- inuse = page->inuse;
- max_objects = page->objects;
+ VM_BUG_ON_PAGE(!is_first_page(first_page), first_page);
+
+ inuse = first_page->inuse;
+ max_objects = first_page->objects;
if (inuse == 0)
fg = ZS_EMPTY;
@@ -652,12 +655,13 @@
* have. This functions inserts the given zspage into the freelist
* identified by <class, fullness_group>.
*/
-static void insert_zspage(struct page *page, struct size_class *class,
- enum fullness_group fullness)
+static void insert_zspage(struct size_class *class,
+ enum fullness_group fullness,
+ struct page *first_page)
{
struct page **head;
- BUG_ON(!is_first_page(page));
+ VM_BUG_ON_PAGE(!is_first_page(first_page), first_page);
if (fullness >= _ZS_NR_FULLNESS_GROUPS)
return;
@@ -667,7 +671,7 @@
head = &class->fullness_list[fullness];
if (!*head) {
- *head = page;
+ *head = first_page;
return;
}
@@ -675,34 +679,35 @@
* We want to see more ZS_FULL pages and less almost
* empty/full. Put pages with higher ->inuse first.
*/
- list_add_tail(&page->lru, &(*head)->lru);
- if (page->inuse >= (*head)->inuse)
- *head = page;
+ list_add_tail(&first_page->lru, &(*head)->lru);
+ if (first_page->inuse >= (*head)->inuse)
+ *head = first_page;
}
/*
* This function removes the given zspage from the freelist identified
* by <class, fullness_group>.
*/
-static void remove_zspage(struct page *page, struct size_class *class,
- enum fullness_group fullness)
+static void remove_zspage(struct size_class *class,
+ enum fullness_group fullness,
+ struct page *first_page)
{
struct page **head;
- BUG_ON(!is_first_page(page));
+ VM_BUG_ON_PAGE(!is_first_page(first_page), first_page);
if (fullness >= _ZS_NR_FULLNESS_GROUPS)
return;
head = &class->fullness_list[fullness];
- BUG_ON(!*head);
+ VM_BUG_ON_PAGE(!*head, first_page);
if (list_empty(&(*head)->lru))
*head = NULL;
- else if (*head == page)
+ else if (*head == first_page)
*head = (struct page *)list_entry((*head)->lru.next,
struct page, lru);
- list_del_init(&page->lru);
+ list_del_init(&first_page->lru);
zs_stat_dec(class, fullness == ZS_ALMOST_EMPTY ?
CLASS_ALMOST_EMPTY : CLASS_ALMOST_FULL, 1);
}
@@ -717,21 +722,19 @@
* fullness group.
*/
static enum fullness_group fix_fullness_group(struct size_class *class,
- struct page *page)
+ struct page *first_page)
{
int class_idx;
enum fullness_group currfg, newfg;
- BUG_ON(!is_first_page(page));
-
- get_zspage_mapping(page, &class_idx, &currfg);
- newfg = get_fullness_group(page);
+ get_zspage_mapping(first_page, &class_idx, &currfg);
+ newfg = get_fullness_group(first_page);
if (newfg == currfg)
goto out;
- remove_zspage(page, class, currfg);
- insert_zspage(page, class, newfg);
- set_zspage_mapping(page, class_idx, newfg);
+ remove_zspage(class, currfg, first_page);
+ insert_zspage(class, newfg, first_page);
+ set_zspage_mapping(first_page, class_idx, newfg);
out:
return newfg;
@@ -809,7 +812,7 @@
unsigned long obj;
if (!page) {
- BUG_ON(obj_idx);
+ VM_BUG_ON(obj_idx);
return NULL;
}
@@ -842,7 +845,7 @@
void *obj)
{
if (class->huge) {
- VM_BUG_ON(!is_first_page(page));
+ VM_BUG_ON_PAGE(!is_first_page(page), page);
return page_private(page);
} else
return *(unsigned long *)obj;
@@ -892,8 +895,8 @@
{
struct page *nextp, *tmp, *head_extra;
- BUG_ON(!is_first_page(first_page));
- BUG_ON(first_page->inuse);
+ VM_BUG_ON_PAGE(!is_first_page(first_page), first_page);
+ VM_BUG_ON_PAGE(first_page->inuse, first_page);
head_extra = (struct page *)page_private(first_page);
@@ -914,12 +917,13 @@
}
/* Initialize a newly allocated zspage */
-static void init_zspage(struct page *first_page, struct size_class *class)
+static void init_zspage(struct size_class *class, struct page *first_page)
{
unsigned long off = 0;
struct page *page = first_page;
- BUG_ON(!is_first_page(first_page));
+ VM_BUG_ON_PAGE(!is_first_page(first_page), first_page);
+
while (page) {
struct page *next_page;
struct link_free *link;
@@ -1001,7 +1005,7 @@
prev_page = page;
}
- init_zspage(first_page, class);
+ init_zspage(class, first_page);
first_page->freelist = location_to_obj(first_page, 0);
/* Maximum number of objects we can store in this zspage */
@@ -1234,11 +1238,11 @@
return true;
}
-static bool zspage_full(struct page *page)
+static bool zspage_full(struct page *first_page)
{
- BUG_ON(!is_first_page(page));
+ VM_BUG_ON_PAGE(!is_first_page(first_page), first_page);
- return page->inuse == page->objects;
+ return first_page->inuse == first_page->objects;
}
unsigned long zs_get_total_pages(struct zs_pool *pool)
@@ -1274,14 +1278,12 @@
struct page *pages[2];
void *ret;
- BUG_ON(!handle);
-
/*
* Because we use per-cpu mapping areas shared among the
* pools/users, we can't allow mapping in interrupt context
* because it can corrupt another users mappings.
*/
- BUG_ON(in_interrupt());
+ WARN_ON_ONCE(in_interrupt());
/* From now on, migration cannot move the object */
pin_tag(handle);
@@ -1325,8 +1327,6 @@
struct size_class *class;
struct mapping_area *area;
- BUG_ON(!handle);
-
obj = handle_to_obj(handle);
obj_to_location(obj, &page, &obj_idx);
get_zspage_mapping(get_first_page(page), &class_idx, &fg);
@@ -1350,8 +1350,8 @@
}
EXPORT_SYMBOL_GPL(zs_unmap_object);
-static unsigned long obj_malloc(struct page *first_page,
- struct size_class *class, unsigned long handle)
+static unsigned long obj_malloc(struct size_class *class,
+ struct page *first_page, unsigned long handle)
{
unsigned long obj;
struct link_free *link;
@@ -1391,7 +1391,7 @@
* otherwise 0.
* Allocation requests with size > ZS_MAX_ALLOC_SIZE will fail.
*/
-unsigned long zs_malloc(struct zs_pool *pool, size_t size)
+unsigned long zs_malloc(struct zs_pool *pool, size_t size, gfp_t gfp)
{
unsigned long handle, obj;
struct size_class *class;
@@ -1400,7 +1400,7 @@
if (unlikely(!size || size > ZS_MAX_ALLOC_SIZE))
return 0;
- handle = alloc_handle(pool);
+ handle = alloc_handle(pool, gfp);
if (!handle)
return 0;
@@ -1413,7 +1413,7 @@
if (!first_page) {
spin_unlock(&class->lock);
- first_page = alloc_zspage(class, pool->flags);
+ first_page = alloc_zspage(class, gfp);
if (unlikely(!first_page)) {
free_handle(pool, handle);
return 0;
@@ -1428,7 +1428,7 @@
class->size, class->pages_per_zspage));
}
- obj = obj_malloc(first_page, class, handle);
+ obj = obj_malloc(class, first_page, handle);
/* Now move the zspage to another fullness group, if required */
fix_fullness_group(class, first_page);
record_obj(handle, obj);
@@ -1438,16 +1438,13 @@
}
EXPORT_SYMBOL_GPL(zs_malloc);
-static void obj_free(struct zs_pool *pool, struct size_class *class,
- unsigned long obj)
+static void obj_free(struct size_class *class, unsigned long obj)
{
struct link_free *link;
struct page *first_page, *f_page;
unsigned long f_objidx, f_offset;
void *vaddr;
- BUG_ON(!obj);
-
obj &= ~OBJ_ALLOCATED_TAG;
obj_to_location(obj, &f_page, &f_objidx);
first_page = get_first_page(f_page);
@@ -1487,7 +1484,7 @@
class = pool->size_class[class_idx];
spin_lock(&class->lock);
- obj_free(pool, class, obj);
+ obj_free(class, obj);
fullness = fix_fullness_group(class, first_page);
if (fullness == ZS_EMPTY) {
zs_stat_dec(class, OBJ_ALLOCATED, get_maxobj_per_zspage(
@@ -1503,8 +1500,8 @@
}
EXPORT_SYMBOL_GPL(zs_free);
-static void zs_object_copy(unsigned long dst, unsigned long src,
- struct size_class *class)
+static void zs_object_copy(struct size_class *class, unsigned long dst,
+ unsigned long src)
{
struct page *s_page, *d_page;
unsigned long s_objidx, d_objidx;
@@ -1547,7 +1544,6 @@
kunmap_atomic(d_addr);
kunmap_atomic(s_addr);
s_page = get_next_page(s_page);
- BUG_ON(!s_page);
s_addr = kmap_atomic(s_page);
d_addr = kmap_atomic(d_page);
s_size = class->size - written;
@@ -1557,7 +1553,6 @@
if (d_off >= PAGE_SIZE) {
kunmap_atomic(d_addr);
d_page = get_next_page(d_page);
- BUG_ON(!d_page);
d_addr = kmap_atomic(d_page);
d_size = class->size - written;
d_off = 0;
@@ -1572,8 +1567,8 @@
* Find alloced object in zspage from index object and
* return handle.
*/
-static unsigned long find_alloced_obj(struct page *page, int index,
- struct size_class *class)
+static unsigned long find_alloced_obj(struct size_class *class,
+ struct page *page, int index)
{
unsigned long head;
int offset = 0;
@@ -1623,7 +1618,7 @@
int ret = 0;
while (1) {
- handle = find_alloced_obj(s_page, index, class);
+ handle = find_alloced_obj(class, s_page, index);
if (!handle) {
s_page = get_next_page(s_page);
if (!s_page)
@@ -1640,8 +1635,8 @@
}
used_obj = handle_to_obj(handle);
- free_obj = obj_malloc(d_page, class, handle);
- zs_object_copy(free_obj, used_obj, class);
+ free_obj = obj_malloc(class, d_page, handle);
+ zs_object_copy(class, free_obj, used_obj);
index++;
/*
* record_obj updates handle's value to free_obj and it will
@@ -1652,7 +1647,7 @@
free_obj |= BIT(HANDLE_PIN_BIT);
record_obj(handle, free_obj);
unpin_tag(handle);
- obj_free(pool, class, used_obj);
+ obj_free(class, used_obj);
}
/* Remember last position in this iteration */
@@ -1670,7 +1665,7 @@
for (i = 0; i < _ZS_NR_FULLNESS_GROUPS; i++) {
page = class->fullness_list[i];
if (page) {
- remove_zspage(page, class, i);
+ remove_zspage(class, i, page);
break;
}
}
@@ -1692,10 +1687,8 @@
{
enum fullness_group fullness;
- BUG_ON(!is_first_page(first_page));
-
fullness = get_fullness_group(first_page);
- insert_zspage(first_page, class, fullness);
+ insert_zspage(class, fullness, first_page);
set_zspage_mapping(first_page, class->index, fullness);
if (fullness == ZS_EMPTY) {
@@ -1720,7 +1713,7 @@
if (!page)
continue;
- remove_zspage(page, class, i);
+ remove_zspage(class, i, page);
break;
}
@@ -1757,8 +1750,6 @@
spin_lock(&class->lock);
while ((src_page = isolate_source_page(class))) {
- BUG_ON(!is_first_page(src_page));
-
if (!zs_can_compact(class))
break;
@@ -1887,7 +1878,7 @@
* On success, a pointer to the newly created pool is returned,
* otherwise NULL.
*/
-struct zs_pool *zs_create_pool(const char *name, gfp_t flags)
+struct zs_pool *zs_create_pool(const char *name)
{
int i;
struct zs_pool *pool;
@@ -1957,10 +1948,8 @@
prev_class = class;
}
- pool->flags = flags;
-
- if (zs_pool_stat_create(name, pool))
- goto err;
+ /* debug only, don't abort if it fails */
+ zs_pool_stat_create(pool, name);
/*
* Not critical, we still can use the pool
diff --git a/mm/zswap.c b/mm/zswap.c
index de0f119b..275b22c 100644
--- a/mm/zswap.c
+++ b/mm/zswap.c
@@ -117,7 +117,7 @@
struct crypto_comp * __percpu *tfm;
struct kref kref;
struct list_head list;
- struct rcu_head rcu_head;
+ struct work_struct work;
struct notifier_block notifier;
char tfm_name[CRYPTO_MAX_ALG_NAME];
};
@@ -658,9 +658,11 @@
return kref_get_unless_zero(&pool->kref);
}
-static void __zswap_pool_release(struct rcu_head *head)
+static void __zswap_pool_release(struct work_struct *work)
{
- struct zswap_pool *pool = container_of(head, typeof(*pool), rcu_head);
+ struct zswap_pool *pool = container_of(work, typeof(*pool), work);
+
+ synchronize_rcu();
/* nobody should have been able to get a kref... */
WARN_ON(kref_get_unless_zero(&pool->kref));
@@ -680,7 +682,9 @@
WARN_ON(pool == zswap_pool_current());
list_del_rcu(&pool->list);
- call_rcu(&pool->rcu_head, __zswap_pool_release);
+
+ INIT_WORK(&pool->work, __zswap_pool_release);
+ schedule_work(&pool->work);
spin_unlock(&zswap_pools_lock);
}
diff --git a/net/batman-adv/bat_iv_ogm.c b/net/batman-adv/bat_iv_ogm.c
index 7f98a9d..ce2f203 100644
--- a/net/batman-adv/bat_iv_ogm.c
+++ b/net/batman-adv/bat_iv_ogm.c
@@ -157,10 +157,8 @@
orig_node->bat_iv.bcast_own = data_ptr;
data_ptr = kmalloc_array(max_if_num, sizeof(u8), GFP_ATOMIC);
- if (!data_ptr) {
- kfree(orig_node->bat_iv.bcast_own);
+ if (!data_ptr)
goto unlock;
- }
memcpy(data_ptr, orig_node->bat_iv.bcast_own_sum,
(max_if_num - 1) * sizeof(u8));
@@ -1182,9 +1180,10 @@
u8 total_count;
u8 orig_eq_count, neigh_rq_count, neigh_rq_inv, tq_own;
unsigned int neigh_rq_inv_cube, neigh_rq_max_cube;
- int tq_asym_penalty, inv_asym_penalty, if_num;
+ int if_num;
+ unsigned int tq_asym_penalty, inv_asym_penalty;
unsigned int combined_tq;
- int tq_iface_penalty;
+ unsigned int tq_iface_penalty;
bool ret = false;
/* find corresponding one hop neighbor */
diff --git a/net/batman-adv/bat_v.c b/net/batman-adv/bat_v.c
index 3ff8bd1..0a12e5c 100644
--- a/net/batman-adv/bat_v.c
+++ b/net/batman-adv/bat_v.c
@@ -27,6 +27,7 @@
#include <linux/rculist.h>
#include <linux/rcupdate.h>
#include <linux/seq_file.h>
+#include <linux/stddef.h>
#include <linux/types.h>
#include <linux/workqueue.h>
@@ -39,6 +40,16 @@
static void batadv_v_iface_activate(struct batadv_hard_iface *hard_iface)
{
+ struct batadv_priv *bat_priv = netdev_priv(hard_iface->soft_iface);
+ struct batadv_hard_iface *primary_if;
+
+ primary_if = batadv_primary_if_get_selected(bat_priv);
+
+ if (primary_if) {
+ batadv_v_elp_iface_activate(primary_if, hard_iface);
+ batadv_hardif_put(primary_if);
+ }
+
/* B.A.T.M.A.N. V does not use any queuing mechanism, therefore it can
* set the interface as ACTIVE right away, without any risk of race
* condition
@@ -72,16 +83,34 @@
batadv_v_elp_iface_disable(hard_iface);
}
-static void batadv_v_iface_update_mac(struct batadv_hard_iface *hard_iface)
-{
-}
-
static void batadv_v_primary_iface_set(struct batadv_hard_iface *hard_iface)
{
batadv_v_elp_primary_iface_set(hard_iface);
batadv_v_ogm_primary_iface_set(hard_iface);
}
+/**
+ * batadv_v_iface_update_mac - react to hard-interface MAC address change
+ * @hard_iface: the modified interface
+ *
+ * If the modified interface is the primary one, update the originator
+ * address in the ELP and OGM messages to reflect the new MAC address.
+ */
+static void batadv_v_iface_update_mac(struct batadv_hard_iface *hard_iface)
+{
+ struct batadv_priv *bat_priv = netdev_priv(hard_iface->soft_iface);
+ struct batadv_hard_iface *primary_if;
+
+ primary_if = batadv_primary_if_get_selected(bat_priv);
+ if (primary_if != hard_iface)
+ goto out;
+
+ batadv_v_primary_iface_set(hard_iface);
+out:
+ if (primary_if)
+ batadv_hardif_put(primary_if);
+}
+
static void
batadv_v_hardif_neigh_init(struct batadv_hardif_neigh_node *hardif_neigh)
{
@@ -255,14 +284,23 @@
struct batadv_hard_iface *if_outgoing2)
{
struct batadv_neigh_ifinfo *ifinfo1, *ifinfo2;
+ int ret = 0;
ifinfo1 = batadv_neigh_ifinfo_get(neigh1, if_outgoing1);
+ if (WARN_ON(!ifinfo1))
+ goto err_ifinfo1;
+
ifinfo2 = batadv_neigh_ifinfo_get(neigh2, if_outgoing2);
+ if (WARN_ON(!ifinfo2))
+ goto err_ifinfo2;
- if (WARN_ON(!ifinfo1 || !ifinfo2))
- return 0;
+ ret = ifinfo1->bat_v.throughput - ifinfo2->bat_v.throughput;
- return ifinfo1->bat_v.throughput - ifinfo2->bat_v.throughput;
+ batadv_neigh_ifinfo_put(ifinfo2);
+err_ifinfo2:
+ batadv_neigh_ifinfo_put(ifinfo1);
+err_ifinfo1:
+ return ret;
}
static bool batadv_v_neigh_is_sob(struct batadv_neigh_node *neigh1,
@@ -272,14 +310,26 @@
{
struct batadv_neigh_ifinfo *ifinfo1, *ifinfo2;
u32 threshold;
+ bool ret = false;
ifinfo1 = batadv_neigh_ifinfo_get(neigh1, if_outgoing1);
+ if (WARN_ON(!ifinfo1))
+ goto err_ifinfo1;
+
ifinfo2 = batadv_neigh_ifinfo_get(neigh2, if_outgoing2);
+ if (WARN_ON(!ifinfo2))
+ goto err_ifinfo2;
threshold = ifinfo1->bat_v.throughput / 4;
threshold = ifinfo1->bat_v.throughput - threshold;
- return ifinfo2->bat_v.throughput > threshold;
+ ret = ifinfo2->bat_v.throughput > threshold;
+
+ batadv_neigh_ifinfo_put(ifinfo2);
+err_ifinfo2:
+ batadv_neigh_ifinfo_put(ifinfo1);
+err_ifinfo1:
+ return ret;
}
static struct batadv_algo_ops batadv_batman_v __read_mostly = {
diff --git a/net/batman-adv/bat_v_elp.c b/net/batman-adv/bat_v_elp.c
index 3844e7e..df42eb1 100644
--- a/net/batman-adv/bat_v_elp.c
+++ b/net/batman-adv/bat_v_elp.c
@@ -377,6 +377,27 @@
}
/**
+ * batadv_v_elp_iface_activate - update the ELP buffer belonging to the given
+ * hard-interface
+ * @primary_iface: the new primary interface
+ * @hard_iface: interface holding the to-be-updated buffer
+ */
+void batadv_v_elp_iface_activate(struct batadv_hard_iface *primary_iface,
+ struct batadv_hard_iface *hard_iface)
+{
+ struct batadv_elp_packet *elp_packet;
+ struct sk_buff *skb;
+
+ if (!hard_iface->bat_v.elp_skb)
+ return;
+
+ skb = hard_iface->bat_v.elp_skb;
+ elp_packet = (struct batadv_elp_packet *)skb->data;
+ ether_addr_copy(elp_packet->orig,
+ primary_iface->net_dev->dev_addr);
+}
+
+/**
* batadv_v_elp_primary_iface_set - change internal data to reflect the new
* primary interface
* @primary_iface: the new primary interface
@@ -384,8 +405,6 @@
void batadv_v_elp_primary_iface_set(struct batadv_hard_iface *primary_iface)
{
struct batadv_hard_iface *hard_iface;
- struct batadv_elp_packet *elp_packet;
- struct sk_buff *skb;
/* update orig field of every elp iface belonging to this mesh */
rcu_read_lock();
@@ -393,13 +412,7 @@
if (primary_iface->soft_iface != hard_iface->soft_iface)
continue;
- if (!hard_iface->bat_v.elp_skb)
- continue;
-
- skb = hard_iface->bat_v.elp_skb;
- elp_packet = (struct batadv_elp_packet *)skb->data;
- ether_addr_copy(elp_packet->orig,
- primary_iface->net_dev->dev_addr);
+ batadv_v_elp_iface_activate(primary_iface, hard_iface);
}
rcu_read_unlock();
}
diff --git a/net/batman-adv/bat_v_elp.h b/net/batman-adv/bat_v_elp.h
index e95f1bc..cc130b2 100644
--- a/net/batman-adv/bat_v_elp.h
+++ b/net/batman-adv/bat_v_elp.h
@@ -25,6 +25,8 @@
int batadv_v_elp_iface_enable(struct batadv_hard_iface *hard_iface);
void batadv_v_elp_iface_disable(struct batadv_hard_iface *hard_iface);
+void batadv_v_elp_iface_activate(struct batadv_hard_iface *primary_iface,
+ struct batadv_hard_iface *hard_iface);
void batadv_v_elp_primary_iface_set(struct batadv_hard_iface *primary_iface);
int batadv_v_elp_packet_recv(struct sk_buff *skb,
struct batadv_hard_iface *if_incoming);
diff --git a/net/batman-adv/originator.c b/net/batman-adv/originator.c
index 1ff4ee4..7f51bc2 100644
--- a/net/batman-adv/originator.c
+++ b/net/batman-adv/originator.c
@@ -619,6 +619,8 @@
struct batadv_neigh_node *neigh_node;
struct batadv_hardif_neigh_node *hardif_neigh = NULL;
+ spin_lock_bh(&orig_node->neigh_list_lock);
+
neigh_node = batadv_neigh_node_get(orig_node, hard_iface, neigh_addr);
if (neigh_node)
goto out;
@@ -650,15 +652,15 @@
kref_init(&neigh_node->refcount);
kref_get(&neigh_node->refcount);
- spin_lock_bh(&orig_node->neigh_list_lock);
hlist_add_head_rcu(&neigh_node->list, &orig_node->neigh_list);
- spin_unlock_bh(&orig_node->neigh_list_lock);
batadv_dbg(BATADV_DBG_BATMAN, orig_node->bat_priv,
"Creating new neighbor %pM for orig_node %pM on interface %s\n",
neigh_addr, orig_node->orig, hard_iface->net_dev->name);
out:
+ spin_unlock_bh(&orig_node->neigh_list_lock);
+
if (hardif_neigh)
batadv_hardif_neigh_put(hardif_neigh);
return neigh_node;
diff --git a/net/batman-adv/routing.c b/net/batman-adv/routing.c
index ae850f2..e3857ed 100644
--- a/net/batman-adv/routing.c
+++ b/net/batman-adv/routing.c
@@ -601,6 +601,7 @@
struct batadv_unicast_packet *unicast_packet;
struct ethhdr *ethhdr = eth_hdr(skb);
int res, hdr_len, ret = NET_RX_DROP;
+ unsigned int len;
unicast_packet = (struct batadv_unicast_packet *)skb->data;
@@ -641,6 +642,7 @@
if (hdr_len > 0)
batadv_skb_set_priority(skb, hdr_len);
+ len = skb->len;
res = batadv_send_skb_to_orig(skb, orig_node, recv_if);
/* translate transmit result into receive result */
@@ -648,7 +650,7 @@
/* skb was transmitted and consumed */
batadv_inc_counter(bat_priv, BATADV_CNT_FORWARD);
batadv_add_counter(bat_priv, BATADV_CNT_FORWARD_BYTES,
- skb->len + ETH_HLEN);
+ len + ETH_HLEN);
ret = NET_RX_SUCCESS;
} else if (res == NET_XMIT_POLICED) {
diff --git a/net/core/ethtool.c b/net/core/ethtool.c
index bdb4013..f403481 100644
--- a/net/core/ethtool.c
+++ b/net/core/ethtool.c
@@ -84,8 +84,8 @@
[NETIF_F_FSO_BIT] = "tx-fcoe-segmentation",
[NETIF_F_GSO_GRE_BIT] = "tx-gre-segmentation",
[NETIF_F_GSO_GRE_CSUM_BIT] = "tx-gre-csum-segmentation",
- [NETIF_F_GSO_IPIP_BIT] = "tx-ipip-segmentation",
- [NETIF_F_GSO_SIT_BIT] = "tx-sit-segmentation",
+ [NETIF_F_GSO_IPXIP4_BIT] = "tx-ipxip4-segmentation",
+ [NETIF_F_GSO_IPXIP6_BIT] = "tx-ipxip6-segmentation",
[NETIF_F_GSO_UDP_TUNNEL_BIT] = "tx-udp_tnl-segmentation",
[NETIF_F_GSO_UDP_TUNNEL_CSUM_BIT] = "tx-udp_tnl-csum-segmentation",
[NETIF_F_GSO_PARTIAL_BIT] = "tx-gso-partial",
diff --git a/net/dns_resolver/dns_key.c b/net/dns_resolver/dns_key.c
index c79b85e..8737412 100644
--- a/net/dns_resolver/dns_key.c
+++ b/net/dns_resolver/dns_key.c
@@ -281,7 +281,7 @@
GLOBAL_ROOT_UID, GLOBAL_ROOT_GID, cred,
(KEY_POS_ALL & ~KEY_POS_SETATTR) |
KEY_USR_VIEW | KEY_USR_READ,
- KEY_ALLOC_NOT_IN_QUOTA, NULL);
+ KEY_ALLOC_NOT_IN_QUOTA, NULL, NULL);
if (IS_ERR(keyring)) {
ret = PTR_ERR(keyring);
goto failed_put_cred;
diff --git a/net/ipv4/af_inet.c b/net/ipv4/af_inet.c
index 2e6e65f..377424e 100644
--- a/net/ipv4/af_inet.c
+++ b/net/ipv4/af_inet.c
@@ -1192,8 +1192,8 @@
}
EXPORT_SYMBOL(inet_sk_rebuild_header);
-static struct sk_buff *inet_gso_segment(struct sk_buff *skb,
- netdev_features_t features)
+struct sk_buff *inet_gso_segment(struct sk_buff *skb,
+ netdev_features_t features)
{
bool udpfrag = false, fixedid = false, encap;
struct sk_buff *segs = ERR_PTR(-EINVAL);
@@ -1205,24 +1205,6 @@
int ihl;
int id;
- if (unlikely(skb_shinfo(skb)->gso_type &
- ~(SKB_GSO_TCPV4 |
- SKB_GSO_UDP |
- SKB_GSO_DODGY |
- SKB_GSO_TCP_ECN |
- SKB_GSO_GRE |
- SKB_GSO_GRE_CSUM |
- SKB_GSO_IPIP |
- SKB_GSO_SIT |
- SKB_GSO_TCPV6 |
- SKB_GSO_UDP_TUNNEL |
- SKB_GSO_UDP_TUNNEL_CSUM |
- SKB_GSO_TCP_FIXEDID |
- SKB_GSO_TUNNEL_REMCSUM |
- SKB_GSO_PARTIAL |
- 0)))
- goto out;
-
skb_reset_network_header(skb);
nhoff = skb_network_header(skb) - skb_mac_header(skb);
if (unlikely(!pskb_may_pull(skb, sizeof(*iph))))
@@ -1298,9 +1280,9 @@
out:
return segs;
}
+EXPORT_SYMBOL(inet_gso_segment);
-static struct sk_buff **inet_gro_receive(struct sk_buff **head,
- struct sk_buff *skb)
+struct sk_buff **inet_gro_receive(struct sk_buff **head, struct sk_buff *skb)
{
const struct net_offload *ops;
struct sk_buff **pp = NULL;
@@ -1416,6 +1398,7 @@
return pp;
}
+EXPORT_SYMBOL(inet_gro_receive);
static struct sk_buff **ipip_gro_receive(struct sk_buff **head,
struct sk_buff *skb)
@@ -1467,7 +1450,7 @@
return -EINVAL;
}
-static int inet_gro_complete(struct sk_buff *skb, int nhoff)
+int inet_gro_complete(struct sk_buff *skb, int nhoff)
{
__be16 newlen = htons(skb->len - nhoff);
struct iphdr *iph = (struct iphdr *)(skb->data + nhoff);
@@ -1497,11 +1480,12 @@
return err;
}
+EXPORT_SYMBOL(inet_gro_complete);
static int ipip_gro_complete(struct sk_buff *skb, int nhoff)
{
skb->encapsulation = 1;
- skb_shinfo(skb)->gso_type |= SKB_GSO_IPIP;
+ skb_shinfo(skb)->gso_type |= SKB_GSO_IPXIP4;
return inet_gro_complete(skb, nhoff);
}
diff --git a/net/ipv4/fou.c b/net/ipv4/fou.c
index eeec7d6..5f9207c 100644
--- a/net/ipv4/fou.c
+++ b/net/ipv4/fou.c
@@ -21,6 +21,7 @@
u8 protocol;
u8 flags;
__be16 port;
+ u8 family;
u16 type;
struct list_head list;
struct rcu_head rcu;
@@ -47,14 +48,17 @@
return sk->sk_user_data;
}
-static int fou_recv_pull(struct sk_buff *skb, size_t len)
+static int fou_recv_pull(struct sk_buff *skb, struct fou *fou, size_t len)
{
- struct iphdr *iph = ip_hdr(skb);
-
/* Remove 'len' bytes from the packet (UDP header and
* FOU header if present).
*/
- iph->tot_len = htons(ntohs(iph->tot_len) - len);
+ if (fou->family == AF_INET)
+ ip_hdr(skb)->tot_len = htons(ntohs(ip_hdr(skb)->tot_len) - len);
+ else
+ ipv6_hdr(skb)->payload_len =
+ htons(ntohs(ipv6_hdr(skb)->payload_len) - len);
+
__skb_pull(skb, len);
skb_postpull_rcsum(skb, udp_hdr(skb), len);
skb_reset_transport_header(skb);
@@ -68,7 +72,7 @@
if (!fou)
return 1;
- if (fou_recv_pull(skb, sizeof(struct udphdr)))
+ if (fou_recv_pull(skb, fou, sizeof(struct udphdr)))
goto drop;
return -fou->protocol;
@@ -141,7 +145,11 @@
hdrlen = sizeof(struct guehdr) + optlen;
- ip_hdr(skb)->tot_len = htons(ntohs(ip_hdr(skb)->tot_len) - len);
+ if (fou->family == AF_INET)
+ ip_hdr(skb)->tot_len = htons(ntohs(ip_hdr(skb)->tot_len) - len);
+ else
+ ipv6_hdr(skb)->payload_len =
+ htons(ntohs(ipv6_hdr(skb)->payload_len) - len);
/* Pull csum through the guehdr now . This can be used if
* there is a remote checksum offload.
@@ -426,7 +434,8 @@
mutex_lock(&fn->fou_lock);
list_for_each_entry(fout, &fn->fou_list, list) {
- if (fou->port == fout->port) {
+ if (fou->port == fout->port &&
+ fou->family == fout->family) {
mutex_unlock(&fn->fou_lock);
return -EALREADY;
}
@@ -448,31 +457,13 @@
kfree_rcu(fou, rcu);
}
-static int fou_encap_init(struct sock *sk, struct fou *fou, struct fou_cfg *cfg)
-{
- udp_sk(sk)->encap_rcv = fou_udp_recv;
- udp_sk(sk)->gro_receive = fou_gro_receive;
- udp_sk(sk)->gro_complete = fou_gro_complete;
- fou_from_sock(sk)->protocol = cfg->protocol;
-
- return 0;
-}
-
-static int gue_encap_init(struct sock *sk, struct fou *fou, struct fou_cfg *cfg)
-{
- udp_sk(sk)->encap_rcv = gue_udp_recv;
- udp_sk(sk)->gro_receive = gue_gro_receive;
- udp_sk(sk)->gro_complete = gue_gro_complete;
-
- return 0;
-}
-
static int fou_create(struct net *net, struct fou_cfg *cfg,
struct socket **sockp)
{
struct socket *sock = NULL;
struct fou *fou = NULL;
struct sock *sk;
+ struct udp_tunnel_sock_cfg tunnel_cfg;
int err;
/* Open UDP socket */
@@ -489,35 +480,36 @@
sk = sock->sk;
- fou->flags = cfg->flags;
fou->port = cfg->udp_config.local_udp_port;
+ fou->family = cfg->udp_config.family;
+ fou->flags = cfg->flags;
+ fou->type = cfg->type;
+ fou->sock = sock;
+
+ memset(&tunnel_cfg, 0, sizeof(tunnel_cfg));
+ tunnel_cfg.encap_type = 1;
+ tunnel_cfg.sk_user_data = fou;
+ tunnel_cfg.encap_destroy = NULL;
/* Initial for fou type */
switch (cfg->type) {
case FOU_ENCAP_DIRECT:
- err = fou_encap_init(sk, fou, cfg);
- if (err)
- goto error;
+ tunnel_cfg.encap_rcv = fou_udp_recv;
+ tunnel_cfg.gro_receive = fou_gro_receive;
+ tunnel_cfg.gro_complete = fou_gro_complete;
+ fou->protocol = cfg->protocol;
break;
case FOU_ENCAP_GUE:
- err = gue_encap_init(sk, fou, cfg);
- if (err)
- goto error;
+ tunnel_cfg.encap_rcv = gue_udp_recv;
+ tunnel_cfg.gro_receive = gue_gro_receive;
+ tunnel_cfg.gro_complete = gue_gro_complete;
break;
default:
err = -EINVAL;
goto error;
}
- fou->type = cfg->type;
-
- udp_sk(sk)->encap_type = 1;
- udp_encap_enable();
-
- sk->sk_user_data = fou;
- fou->sock = sock;
-
- inet_inc_convert_csum(sk);
+ setup_udp_tunnel_sock(net, sock, &tunnel_cfg);
sk->sk_allocation = GFP_ATOMIC;
@@ -542,12 +534,13 @@
{
struct fou_net *fn = net_generic(net, fou_net_id);
__be16 port = cfg->udp_config.local_udp_port;
+ u8 family = cfg->udp_config.family;
int err = -EINVAL;
struct fou *fou;
mutex_lock(&fn->fou_lock);
list_for_each_entry(fou, &fn->fou_list, list) {
- if (fou->port == port) {
+ if (fou->port == port && fou->family == family) {
fou_release(fou);
err = 0;
break;
@@ -585,8 +578,15 @@
if (info->attrs[FOU_ATTR_AF]) {
u8 family = nla_get_u8(info->attrs[FOU_ATTR_AF]);
- if (family != AF_INET)
- return -EINVAL;
+ switch (family) {
+ case AF_INET:
+ break;
+ case AF_INET6:
+ cfg->udp_config.ipv6_v6only = 1;
+ break;
+ default:
+ return -EAFNOSUPPORT;
+ }
cfg->udp_config.family = family;
}
@@ -677,6 +677,7 @@
struct fou_cfg cfg;
struct fou *fout;
__be16 port;
+ u8 family;
int ret;
ret = parse_nl_config(info, &cfg);
@@ -686,6 +687,10 @@
if (port == 0)
return -EINVAL;
+ family = cfg.udp_config.family;
+ if (family != AF_INET && family != AF_INET6)
+ return -EINVAL;
+
msg = nlmsg_new(NLMSG_DEFAULT_SIZE, GFP_KERNEL);
if (!msg)
return -ENOMEM;
@@ -693,7 +698,7 @@
ret = -ESRCH;
mutex_lock(&fn->fou_lock);
list_for_each_entry(fout, &fn->fou_list, list) {
- if (port == fout->port) {
+ if (port == fout->port && family == fout->family) {
ret = fou_dump_info(fout, info->snd_portid,
info->snd_seq, 0, msg,
info->genlhdr->cmd);
@@ -798,6 +803,22 @@
*protocol = IPPROTO_UDP;
}
+int __fou_build_header(struct sk_buff *skb, struct ip_tunnel_encap *e,
+ u8 *protocol, __be16 *sport, int type)
+{
+ int err;
+
+ err = iptunnel_handle_offloads(skb, type);
+ if (err)
+ return err;
+
+ *sport = e->sport ? : udp_flow_src_port(dev_net(skb->dev),
+ skb, 0, 0, false);
+
+ return 0;
+}
+EXPORT_SYMBOL(__fou_build_header);
+
int fou_build_header(struct sk_buff *skb, struct ip_tunnel_encap *e,
u8 *protocol, struct flowi4 *fl4)
{
@@ -806,26 +827,21 @@
__be16 sport;
int err;
- err = iptunnel_handle_offloads(skb, type);
+ err = __fou_build_header(skb, e, protocol, &sport, type);
if (err)
return err;
- sport = e->sport ? : udp_flow_src_port(dev_net(skb->dev),
- skb, 0, 0, false);
fou_build_udp(skb, e, fl4, protocol, sport);
return 0;
}
EXPORT_SYMBOL(fou_build_header);
-int gue_build_header(struct sk_buff *skb, struct ip_tunnel_encap *e,
- u8 *protocol, struct flowi4 *fl4)
+int __gue_build_header(struct sk_buff *skb, struct ip_tunnel_encap *e,
+ u8 *protocol, __be16 *sport, int type)
{
- int type = e->flags & TUNNEL_ENCAP_FLAG_CSUM ? SKB_GSO_UDP_TUNNEL_CSUM :
- SKB_GSO_UDP_TUNNEL;
struct guehdr *guehdr;
size_t hdrlen, optlen = 0;
- __be16 sport;
void *data;
bool need_priv = false;
int err;
@@ -844,8 +860,8 @@
return err;
/* Get source port (based on flow hash) before skb_push */
- sport = e->sport ? : udp_flow_src_port(dev_net(skb->dev),
- skb, 0, 0, false);
+ *sport = e->sport ? : udp_flow_src_port(dev_net(skb->dev),
+ skb, 0, 0, false);
hdrlen = sizeof(struct guehdr) + optlen;
@@ -890,6 +906,22 @@
}
+ return 0;
+}
+EXPORT_SYMBOL(__gue_build_header);
+
+int gue_build_header(struct sk_buff *skb, struct ip_tunnel_encap *e,
+ u8 *protocol, struct flowi4 *fl4)
+{
+ int type = e->flags & TUNNEL_ENCAP_FLAG_CSUM ? SKB_GSO_UDP_TUNNEL_CSUM :
+ SKB_GSO_UDP_TUNNEL;
+ __be16 sport;
+ int err;
+
+ err = __gue_build_header(skb, e, protocol, &sport, type);
+ if (err)
+ return err;
+
fou_build_udp(skb, e, fl4, protocol, sport);
return 0;
diff --git a/net/ipv4/gre_offload.c b/net/ipv4/gre_offload.c
index e88190a..ecd1e09 100644
--- a/net/ipv4/gre_offload.c
+++ b/net/ipv4/gre_offload.c
@@ -26,20 +26,6 @@
int gre_offset, outer_hlen;
bool need_csum, ufo;
- if (unlikely(skb_shinfo(skb)->gso_type &
- ~(SKB_GSO_TCPV4 |
- SKB_GSO_TCPV6 |
- SKB_GSO_UDP |
- SKB_GSO_DODGY |
- SKB_GSO_TCP_ECN |
- SKB_GSO_TCP_FIXEDID |
- SKB_GSO_GRE |
- SKB_GSO_GRE_CSUM |
- SKB_GSO_IPIP |
- SKB_GSO_SIT |
- SKB_GSO_PARTIAL)))
- goto out;
-
if (!skb->encapsulation)
goto out;
diff --git a/net/ipv4/ip_tunnel.c b/net/ipv4/ip_tunnel.c
index a69ed94..d8f5e0a 100644
--- a/net/ipv4/ip_tunnel.c
+++ b/net/ipv4/ip_tunnel.c
@@ -443,29 +443,6 @@
}
EXPORT_SYMBOL_GPL(ip_tunnel_rcv);
-static int ip_encap_hlen(struct ip_tunnel_encap *e)
-{
- const struct ip_tunnel_encap_ops *ops;
- int hlen = -EINVAL;
-
- if (e->type == TUNNEL_ENCAP_NONE)
- return 0;
-
- if (e->type >= MAX_IPTUN_ENCAP_OPS)
- return -EINVAL;
-
- rcu_read_lock();
- ops = rcu_dereference(iptun_encaps[e->type]);
- if (likely(ops && ops->encap_hlen))
- hlen = ops->encap_hlen(e);
- rcu_read_unlock();
-
- return hlen;
-}
-
-const struct ip_tunnel_encap_ops __rcu *
- iptun_encaps[MAX_IPTUN_ENCAP_OPS] __read_mostly;
-
int ip_tunnel_encap_add_ops(const struct ip_tunnel_encap_ops *ops,
unsigned int num)
{
@@ -519,28 +496,6 @@
}
EXPORT_SYMBOL_GPL(ip_tunnel_encap_setup);
-int ip_tunnel_encap(struct sk_buff *skb, struct ip_tunnel *t,
- u8 *protocol, struct flowi4 *fl4)
-{
- const struct ip_tunnel_encap_ops *ops;
- int ret = -EINVAL;
-
- if (t->encap.type == TUNNEL_ENCAP_NONE)
- return 0;
-
- if (t->encap.type >= MAX_IPTUN_ENCAP_OPS)
- return -EINVAL;
-
- rcu_read_lock();
- ops = rcu_dereference(iptun_encaps[t->encap.type]);
- if (likely(ops && ops->build_header))
- ret = ops->build_header(skb, &t->encap, protocol, fl4);
- rcu_read_unlock();
-
- return ret;
-}
-EXPORT_SYMBOL(ip_tunnel_encap);
-
static int tnl_update_pmtu(struct net_device *dev, struct sk_buff *skb,
struct rtable *rt, __be16 df,
const struct iphdr *inner_iph)
diff --git a/net/ipv4/ip_tunnel_core.c b/net/ipv4/ip_tunnel_core.c
index 9118b0e..afd6b59 100644
--- a/net/ipv4/ip_tunnel_core.c
+++ b/net/ipv4/ip_tunnel_core.c
@@ -37,6 +37,7 @@
#include <net/icmp.h>
#include <net/protocol.h>
#include <net/ip_tunnels.h>
+#include <net/ip6_tunnel.h>
#include <net/arp.h>
#include <net/checksum.h>
#include <net/dsfield.h>
@@ -47,6 +48,14 @@
#include <net/rtnetlink.h>
#include <net/dst_metadata.h>
+const struct ip_tunnel_encap_ops __rcu *
+ iptun_encaps[MAX_IPTUN_ENCAP_OPS] __read_mostly;
+EXPORT_SYMBOL(iptun_encaps);
+
+const struct ip6_tnl_encap_ops __rcu *
+ ip6tun_encaps[MAX_IPTUN_ENCAP_OPS] __read_mostly;
+EXPORT_SYMBOL(ip6tun_encaps);
+
void iptunnel_xmit(struct sock *sk, struct rtable *rt, struct sk_buff *skb,
__be32 src, __be32 dst, __u8 proto,
__u8 tos, __u8 ttl, __be16 df, bool xnet)
diff --git a/net/ipv4/ipip.c b/net/ipv4/ipip.c
index 9282748..9783701 100644
--- a/net/ipv4/ipip.c
+++ b/net/ipv4/ipip.c
@@ -219,7 +219,7 @@
if (unlikely(skb->protocol != htons(ETH_P_IP)))
goto tx_error;
- if (iptunnel_handle_offloads(skb, SKB_GSO_IPIP))
+ if (iptunnel_handle_offloads(skb, SKB_GSO_IPXIP4))
goto tx_error;
skb_set_inner_ipproto(skb, IPPROTO_IPIP);
diff --git a/net/ipv4/tcp_offload.c b/net/ipv4/tcp_offload.c
index 02737b6..5c59649 100644
--- a/net/ipv4/tcp_offload.c
+++ b/net/ipv4/tcp_offload.c
@@ -83,25 +83,6 @@
if (skb_gso_ok(skb, features | NETIF_F_GSO_ROBUST)) {
/* Packet is from an untrusted source, reset gso_segs. */
- int type = skb_shinfo(skb)->gso_type;
-
- if (unlikely(type &
- ~(SKB_GSO_TCPV4 |
- SKB_GSO_DODGY |
- SKB_GSO_TCP_ECN |
- SKB_GSO_TCP_FIXEDID |
- SKB_GSO_TCPV6 |
- SKB_GSO_GRE |
- SKB_GSO_GRE_CSUM |
- SKB_GSO_IPIP |
- SKB_GSO_SIT |
- SKB_GSO_UDP_TUNNEL |
- SKB_GSO_UDP_TUNNEL_CSUM |
- SKB_GSO_TUNNEL_REMCSUM |
- 0) ||
- !(type & (SKB_GSO_TCPV4 |
- SKB_GSO_TCPV6))))
- goto out;
skb_shinfo(skb)->gso_segs = DIV_ROUND_UP(skb->len, mss);
diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index 2e3ebfe..d56c055 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -1565,7 +1565,7 @@
/* if we're overly short, let UDP handle it */
encap_rcv = ACCESS_ONCE(up->encap_rcv);
- if (skb->len > sizeof(struct udphdr) && encap_rcv) {
+ if (encap_rcv) {
int ret;
/* Verify checksum before giving to encap */
diff --git a/net/ipv4/udp_offload.c b/net/ipv4/udp_offload.c
index 6b7459c..81f253b 100644
--- a/net/ipv4/udp_offload.c
+++ b/net/ipv4/udp_offload.c
@@ -209,16 +209,6 @@
if (skb_gso_ok(skb, features | NETIF_F_GSO_ROBUST)) {
/* Packet is from an untrusted source, reset gso_segs. */
- int type = skb_shinfo(skb)->gso_type;
-
- if (unlikely(type & ~(SKB_GSO_UDP | SKB_GSO_DODGY |
- SKB_GSO_UDP_TUNNEL |
- SKB_GSO_UDP_TUNNEL_CSUM |
- SKB_GSO_TUNNEL_REMCSUM |
- SKB_GSO_IPIP |
- SKB_GSO_GRE | SKB_GSO_GRE_CSUM) ||
- !(type & (SKB_GSO_UDP))))
- goto out;
skb_shinfo(skb)->gso_segs = DIV_ROUND_UP(skb->len, mss);
diff --git a/net/ipv6/Makefile b/net/ipv6/Makefile
index 5e9d6bf..7ec3129 100644
--- a/net/ipv6/Makefile
+++ b/net/ipv6/Makefile
@@ -42,6 +42,7 @@
obj-$(CONFIG_IPV6_SIT) += sit.o
obj-$(CONFIG_IPV6_TUNNEL) += ip6_tunnel.o
obj-$(CONFIG_IPV6_GRE) += ip6_gre.o
+obj-$(CONFIG_NET_FOU) += fou6.o
obj-y += addrconf_core.o exthdrs_core.o ip6_checksum.o ip6_icmp.o
obj-$(CONFIG_INET) += output_core.o protocol.o $(ipv6-offload)
diff --git a/net/ipv6/fou6.c b/net/ipv6/fou6.c
new file mode 100644
index 0000000..c972d0b
--- /dev/null
+++ b/net/ipv6/fou6.c
@@ -0,0 +1,140 @@
+#include <linux/module.h>
+#include <linux/errno.h>
+#include <linux/socket.h>
+#include <linux/skbuff.h>
+#include <linux/ip.h>
+#include <linux/udp.h>
+#include <linux/types.h>
+#include <linux/kernel.h>
+#include <net/fou.h>
+#include <net/ip.h>
+#include <net/ip6_tunnel.h>
+#include <net/ip6_checksum.h>
+#include <net/protocol.h>
+#include <net/udp.h>
+#include <net/udp_tunnel.h>
+
+static void fou6_build_udp(struct sk_buff *skb, struct ip_tunnel_encap *e,
+ struct flowi6 *fl6, u8 *protocol, __be16 sport)
+{
+ struct udphdr *uh;
+
+ skb_push(skb, sizeof(struct udphdr));
+ skb_reset_transport_header(skb);
+
+ uh = udp_hdr(skb);
+
+ uh->dest = e->dport;
+ uh->source = sport;
+ uh->len = htons(skb->len);
+ udp6_set_csum(!(e->flags & TUNNEL_ENCAP_FLAG_CSUM6), skb,
+ &fl6->saddr, &fl6->daddr, skb->len);
+
+ *protocol = IPPROTO_UDP;
+}
+
+int fou6_build_header(struct sk_buff *skb, struct ip_tunnel_encap *e,
+ u8 *protocol, struct flowi6 *fl6)
+{
+ __be16 sport;
+ int err;
+ int type = e->flags & TUNNEL_ENCAP_FLAG_CSUM6 ?
+ SKB_GSO_UDP_TUNNEL_CSUM : SKB_GSO_UDP_TUNNEL;
+
+ err = __fou_build_header(skb, e, protocol, &sport, type);
+ if (err)
+ return err;
+
+ fou6_build_udp(skb, e, fl6, protocol, sport);
+
+ return 0;
+}
+EXPORT_SYMBOL(fou6_build_header);
+
+int gue6_build_header(struct sk_buff *skb, struct ip_tunnel_encap *e,
+ u8 *protocol, struct flowi6 *fl6)
+{
+ __be16 sport;
+ int err;
+ int type = e->flags & TUNNEL_ENCAP_FLAG_CSUM6 ?
+ SKB_GSO_UDP_TUNNEL_CSUM : SKB_GSO_UDP_TUNNEL;
+
+ err = __gue_build_header(skb, e, protocol, &sport, type);
+ if (err)
+ return err;
+
+ fou6_build_udp(skb, e, fl6, protocol, sport);
+
+ return 0;
+}
+EXPORT_SYMBOL(gue6_build_header);
+
+#ifdef CONFIG_NET_FOU_IP_TUNNELS
+
+static const struct ip6_tnl_encap_ops fou_ip6tun_ops = {
+ .encap_hlen = fou_encap_hlen,
+ .build_header = fou6_build_header,
+};
+
+static const struct ip6_tnl_encap_ops gue_ip6tun_ops = {
+ .encap_hlen = gue_encap_hlen,
+ .build_header = gue6_build_header,
+};
+
+static int ip6_tnl_encap_add_fou_ops(void)
+{
+ int ret;
+
+ ret = ip6_tnl_encap_add_ops(&fou_ip6tun_ops, TUNNEL_ENCAP_FOU);
+ if (ret < 0) {
+ pr_err("can't add fou6 ops\n");
+ return ret;
+ }
+
+ ret = ip6_tnl_encap_add_ops(&gue_ip6tun_ops, TUNNEL_ENCAP_GUE);
+ if (ret < 0) {
+ pr_err("can't add gue6 ops\n");
+ ip6_tnl_encap_del_ops(&fou_ip6tun_ops, TUNNEL_ENCAP_FOU);
+ return ret;
+ }
+
+ return 0;
+}
+
+static void ip6_tnl_encap_del_fou_ops(void)
+{
+ ip6_tnl_encap_del_ops(&fou_ip6tun_ops, TUNNEL_ENCAP_FOU);
+ ip6_tnl_encap_del_ops(&gue_ip6tun_ops, TUNNEL_ENCAP_GUE);
+}
+
+#else
+
+static int ip6_tnl_encap_add_fou_ops(void)
+{
+ return 0;
+}
+
+static void ip6_tnl_encap_del_fou_ops(void)
+{
+}
+
+#endif
+
+static int __init fou6_init(void)
+{
+ int ret;
+
+ ret = ip6_tnl_encap_add_fou_ops();
+
+ return ret;
+}
+
+static void __exit fou6_fini(void)
+{
+ ip6_tnl_encap_del_fou_ops();
+}
+
+module_init(fou6_init);
+module_exit(fou6_fini);
+MODULE_AUTHOR("Tom Herbert <therbert@google.com>");
+MODULE_LICENSE("GPL");
diff --git a/net/ipv6/ip6_gre.c b/net/ipv6/ip6_gre.c
index 4541fa5..af503f5 100644
--- a/net/ipv6/ip6_gre.c
+++ b/net/ipv6/ip6_gre.c
@@ -729,7 +729,7 @@
t->tun_hlen = gre_calc_hlen(t->parms.o_flags);
- t->hlen = t->tun_hlen;
+ t->hlen = t->encap_hlen + t->tun_hlen;
t_hlen = t->hlen + sizeof(struct ipv6hdr);
@@ -1022,9 +1022,7 @@
}
tunnel->tun_hlen = gre_calc_hlen(tunnel->parms.o_flags);
-
- tunnel->hlen = tunnel->tun_hlen;
-
+ tunnel->hlen = tunnel->tun_hlen + tunnel->encap_hlen;
t_hlen = tunnel->hlen + sizeof(struct ipv6hdr);
dev->hard_header_len = LL_MAX_HEADER + t_hlen;
@@ -1290,15 +1288,57 @@
dev->priv_flags &= ~IFF_TX_SKB_SHARING;
}
+static bool ip6gre_netlink_encap_parms(struct nlattr *data[],
+ struct ip_tunnel_encap *ipencap)
+{
+ bool ret = false;
+
+ memset(ipencap, 0, sizeof(*ipencap));
+
+ if (!data)
+ return ret;
+
+ if (data[IFLA_GRE_ENCAP_TYPE]) {
+ ret = true;
+ ipencap->type = nla_get_u16(data[IFLA_GRE_ENCAP_TYPE]);
+ }
+
+ if (data[IFLA_GRE_ENCAP_FLAGS]) {
+ ret = true;
+ ipencap->flags = nla_get_u16(data[IFLA_GRE_ENCAP_FLAGS]);
+ }
+
+ if (data[IFLA_GRE_ENCAP_SPORT]) {
+ ret = true;
+ ipencap->sport = nla_get_be16(data[IFLA_GRE_ENCAP_SPORT]);
+ }
+
+ if (data[IFLA_GRE_ENCAP_DPORT]) {
+ ret = true;
+ ipencap->dport = nla_get_be16(data[IFLA_GRE_ENCAP_DPORT]);
+ }
+
+ return ret;
+}
+
static int ip6gre_newlink(struct net *src_net, struct net_device *dev,
struct nlattr *tb[], struct nlattr *data[])
{
struct ip6_tnl *nt;
struct net *net = dev_net(dev);
struct ip6gre_net *ign = net_generic(net, ip6gre_net_id);
+ struct ip_tunnel_encap ipencap;
int err;
nt = netdev_priv(dev);
+
+ if (ip6gre_netlink_encap_parms(data, &ipencap)) {
+ int err = ip6_tnl_encap_setup(nt, &ipencap);
+
+ if (err < 0)
+ return err;
+ }
+
ip6gre_netlink_parms(data, &nt->parms);
if (ip6gre_tunnel_find(net, &nt->parms, dev->type))
@@ -1315,11 +1355,15 @@
dev->hw_features |= GRE6_FEATURES;
if (!(nt->parms.o_flags & TUNNEL_SEQ)) {
- /* TCP segmentation offload is not supported when we
- * generate output sequences.
+ /* TCP offload with GRE SEQ is not supported, nor
+ * can we support 2 levels of outer headers requiring
+ * an update.
*/
- dev->features |= NETIF_F_GSO_SOFTWARE;
- dev->hw_features |= NETIF_F_GSO_SOFTWARE;
+ if (!(nt->parms.o_flags & TUNNEL_CSUM) ||
+ (nt->encap.type == TUNNEL_ENCAP_NONE)) {
+ dev->features |= NETIF_F_GSO_SOFTWARE;
+ dev->hw_features |= NETIF_F_GSO_SOFTWARE;
+ }
/* Can use a lockless transmit, unless we generate
* output sequences
@@ -1345,10 +1389,18 @@
struct net *net = nt->net;
struct ip6gre_net *ign = net_generic(net, ip6gre_net_id);
struct __ip6_tnl_parm p;
+ struct ip_tunnel_encap ipencap;
if (dev == ign->fb_tunnel_dev)
return -EINVAL;
+ if (ip6gre_netlink_encap_parms(data, &ipencap)) {
+ int err = ip6_tnl_encap_setup(nt, &ipencap);
+
+ if (err < 0)
+ return err;
+ }
+
ip6gre_netlink_parms(data, &p);
t = ip6gre_tunnel_locate(net, &p, 0);
@@ -1400,6 +1452,14 @@
nla_total_size(4) +
/* IFLA_GRE_FLAGS */
nla_total_size(4) +
+ /* IFLA_GRE_ENCAP_TYPE */
+ nla_total_size(2) +
+ /* IFLA_GRE_ENCAP_FLAGS */
+ nla_total_size(2) +
+ /* IFLA_GRE_ENCAP_SPORT */
+ nla_total_size(2) +
+ /* IFLA_GRE_ENCAP_DPORT */
+ nla_total_size(2) +
0;
}
@@ -1422,6 +1482,17 @@
nla_put_be32(skb, IFLA_GRE_FLOWINFO, p->flowinfo) ||
nla_put_u32(skb, IFLA_GRE_FLAGS, p->flags))
goto nla_put_failure;
+
+ if (nla_put_u16(skb, IFLA_GRE_ENCAP_TYPE,
+ t->encap.type) ||
+ nla_put_be16(skb, IFLA_GRE_ENCAP_SPORT,
+ t->encap.sport) ||
+ nla_put_be16(skb, IFLA_GRE_ENCAP_DPORT,
+ t->encap.dport) ||
+ nla_put_u16(skb, IFLA_GRE_ENCAP_FLAGS,
+ t->encap.flags))
+ goto nla_put_failure;
+
return 0;
nla_put_failure:
@@ -1440,6 +1511,10 @@
[IFLA_GRE_ENCAP_LIMIT] = { .type = NLA_U8 },
[IFLA_GRE_FLOWINFO] = { .type = NLA_U32 },
[IFLA_GRE_FLAGS] = { .type = NLA_U32 },
+ [IFLA_GRE_ENCAP_TYPE] = { .type = NLA_U16 },
+ [IFLA_GRE_ENCAP_FLAGS] = { .type = NLA_U16 },
+ [IFLA_GRE_ENCAP_SPORT] = { .type = NLA_U16 },
+ [IFLA_GRE_ENCAP_DPORT] = { .type = NLA_U16 },
};
static struct rtnl_link_ops ip6gre_link_ops __read_mostly = {
diff --git a/net/ipv6/ip6_input.c b/net/ipv6/ip6_input.c
index f185cbc..94611e4 100644
--- a/net/ipv6/ip6_input.c
+++ b/net/ipv6/ip6_input.c
@@ -223,6 +223,7 @@
unsigned int nhoff;
int nexthdr;
bool raw;
+ bool have_final = false;
/*
* Parse extension headers
@@ -236,14 +237,27 @@
nhoff = IP6CB(skb)->nhoff;
nexthdr = skb_network_header(skb)[nhoff];
+resubmit_final:
raw = raw6_local_deliver(skb, nexthdr);
ipprot = rcu_dereference(inet6_protos[nexthdr]);
if (ipprot) {
int ret;
- if (ipprot->flags & INET6_PROTO_FINAL) {
+ if (have_final) {
+ if (!(ipprot->flags & INET6_PROTO_FINAL)) {
+ /* Once we've seen a final protocol don't
+ * allow encapsulation on any non-final
+ * ones. This allows foo in UDP encapsulation
+ * to work.
+ */
+ goto discard;
+ }
+ } else if (ipprot->flags & INET6_PROTO_FINAL) {
const struct ipv6hdr *hdr;
+ /* Only do this once for first final protocol */
+ have_final = true;
+
/* Free reference early: we don't need it any more,
and it may hold ip_conntrack module loaded
indefinitely. */
@@ -263,10 +277,21 @@
goto discard;
ret = ipprot->handler(skb);
- if (ret > 0)
- goto resubmit;
- else if (ret == 0)
+ if (ret > 0) {
+ if (ipprot->flags & INET6_PROTO_FINAL) {
+ /* Not an extension header, most likely UDP
+ * encapsulation. Use return value as nexthdr
+ * protocol not nhoff (which presumably is
+ * not set by handler).
+ */
+ nexthdr = ret;
+ goto resubmit_final;
+ } else {
+ goto resubmit;
+ }
+ } else if (ret == 0) {
__IP6_INC_STATS(net, idev, IPSTATS_MIB_INDELIVERS);
+ }
} else {
if (!raw) {
if (xfrm6_policy_check(NULL, XFRM_POLICY_IN, skb)) {
diff --git a/net/ipv6/ip6_offload.c b/net/ipv6/ip6_offload.c
index f5eb184..22e90e5 100644
--- a/net/ipv6/ip6_offload.c
+++ b/net/ipv6/ip6_offload.c
@@ -16,6 +16,7 @@
#include <net/protocol.h>
#include <net/ipv6.h>
+#include <net/inet_common.h>
#include "ip6_offload.h"
@@ -69,24 +70,6 @@
bool encap, udpfrag;
int nhoff;
- if (unlikely(skb_shinfo(skb)->gso_type &
- ~(SKB_GSO_TCPV4 |
- SKB_GSO_UDP |
- SKB_GSO_DODGY |
- SKB_GSO_TCP_ECN |
- SKB_GSO_TCP_FIXEDID |
- SKB_GSO_TCPV6 |
- SKB_GSO_GRE |
- SKB_GSO_GRE_CSUM |
- SKB_GSO_IPIP |
- SKB_GSO_SIT |
- SKB_GSO_UDP_TUNNEL |
- SKB_GSO_UDP_TUNNEL_CSUM |
- SKB_GSO_TUNNEL_REMCSUM |
- SKB_GSO_PARTIAL |
- 0)))
- goto out;
-
skb_reset_network_header(skb);
nhoff = skb_network_header(skb) - skb_mac_header(skb);
if (unlikely(!pskb_may_pull(skb, sizeof(*ipv6h))))
@@ -104,7 +87,7 @@
proto = ipv6_gso_pull_exthdrs(skb, ipv6h->nexthdr);
if (skb->encapsulation &&
- skb_shinfo(skb)->gso_type & (SKB_GSO_SIT|SKB_GSO_IPIP))
+ skb_shinfo(skb)->gso_type & (SKB_GSO_IPXIP4 | SKB_GSO_IPXIP6))
udpfrag = proto == IPPROTO_UDP && encap;
else
udpfrag = proto == IPPROTO_UDP && !skb->encapsulation;
@@ -271,9 +254,11 @@
return pp;
}
-static struct sk_buff **sit_gro_receive(struct sk_buff **head,
- struct sk_buff *skb)
+static struct sk_buff **sit_ip6ip6_gro_receive(struct sk_buff **head,
+ struct sk_buff *skb)
{
+ /* Common GRO receive for SIT and IP6IP6 */
+
if (NAPI_GRO_CB(skb)->encap_mark) {
NAPI_GRO_CB(skb)->flush = 1;
return NULL;
@@ -284,6 +269,21 @@
return ipv6_gro_receive(head, skb);
}
+static struct sk_buff **ip4ip6_gro_receive(struct sk_buff **head,
+ struct sk_buff *skb)
+{
+ /* Common GRO receive for SIT and IP6IP6 */
+
+ if (NAPI_GRO_CB(skb)->encap_mark) {
+ NAPI_GRO_CB(skb)->flush = 1;
+ return NULL;
+ }
+
+ NAPI_GRO_CB(skb)->encap_mark = 1;
+
+ return inet_gro_receive(head, skb);
+}
+
static int ipv6_gro_complete(struct sk_buff *skb, int nhoff)
{
const struct net_offload *ops;
@@ -312,10 +312,24 @@
static int sit_gro_complete(struct sk_buff *skb, int nhoff)
{
skb->encapsulation = 1;
- skb_shinfo(skb)->gso_type |= SKB_GSO_SIT;
+ skb_shinfo(skb)->gso_type |= SKB_GSO_IPXIP4;
return ipv6_gro_complete(skb, nhoff);
}
+static int ip6ip6_gro_complete(struct sk_buff *skb, int nhoff)
+{
+ skb->encapsulation = 1;
+ skb_shinfo(skb)->gso_type |= SKB_GSO_IPXIP6;
+ return ipv6_gro_complete(skb, nhoff);
+}
+
+static int ip4ip6_gro_complete(struct sk_buff *skb, int nhoff)
+{
+ skb->encapsulation = 1;
+ skb_shinfo(skb)->gso_type |= SKB_GSO_IPXIP6;
+ return inet_gro_complete(skb, nhoff);
+}
+
static struct packet_offload ipv6_packet_offload __read_mostly = {
.type = cpu_to_be16(ETH_P_IPV6),
.callbacks = {
@@ -328,11 +342,26 @@
static const struct net_offload sit_offload = {
.callbacks = {
.gso_segment = ipv6_gso_segment,
- .gro_receive = sit_gro_receive,
+ .gro_receive = sit_ip6ip6_gro_receive,
.gro_complete = sit_gro_complete,
},
};
+static const struct net_offload ip4ip6_offload = {
+ .callbacks = {
+ .gso_segment = inet_gso_segment,
+ .gro_receive = ip4ip6_gro_receive,
+ .gro_complete = ip4ip6_gro_complete,
+ },
+};
+
+static const struct net_offload ip6ip6_offload = {
+ .callbacks = {
+ .gso_segment = ipv6_gso_segment,
+ .gro_receive = sit_ip6ip6_gro_receive,
+ .gro_complete = ip6ip6_gro_complete,
+ },
+};
static int __init ipv6_offload_init(void)
{
@@ -344,6 +373,8 @@
dev_add_offload(&ipv6_packet_offload);
inet_add_offload(&sit_offload, IPPROTO_IPV6);
+ inet6_add_offload(&ip6ip6_offload, IPPROTO_IPV6);
+ inet6_add_offload(&ip4ip6_offload, IPPROTO_IPIP);
return 0;
}
diff --git a/net/ipv6/ip6_tunnel.c b/net/ipv6/ip6_tunnel.c
index e79330f..7b0481e 100644
--- a/net/ipv6/ip6_tunnel.c
+++ b/net/ipv6/ip6_tunnel.c
@@ -1010,7 +1010,8 @@
struct dst_entry *dst = NULL, *ndst = NULL;
struct net_device *tdev;
int mtu;
- unsigned int max_headroom = sizeof(struct ipv6hdr);
+ unsigned int psh_hlen = sizeof(struct ipv6hdr) + t->encap_hlen;
+ unsigned int max_headroom = psh_hlen;
int err = -1;
/* NBMA tunnel */
@@ -1063,7 +1064,7 @@
t->parms.name);
goto tx_err_dst_release;
}
- mtu = dst_mtu(dst) - sizeof(*ipv6h);
+ mtu = dst_mtu(dst) - psh_hlen;
if (encap_limit >= 0) {
max_headroom += 8;
mtu -= 8;
@@ -1119,16 +1120,18 @@
ipv6_push_nfrag_opts(skb, &opt.ops, &proto, NULL);
}
- if (likely(!skb->encapsulation)) {
- skb_reset_inner_headers(skb);
- skb->encapsulation = 1;
- }
-
+ /* Calculate max headroom for all the headers and adjust
+ * needed_headroom if necessary.
+ */
max_headroom = LL_RESERVED_SPACE(dst->dev) + sizeof(struct ipv6hdr)
- + dst->header_len;
+ + dst->header_len + t->hlen;
if (max_headroom > dev->needed_headroom)
dev->needed_headroom = max_headroom;
+ err = ip6_tnl_encap(skb, t, &proto, fl6);
+ if (err)
+ return err;
+
skb_push(skb, sizeof(struct ipv6hdr));
skb_reset_network_header(skb);
ipv6h = ipv6_hdr(skb);
@@ -1180,6 +1183,11 @@
if (t->parms.flags & IP6_TNL_F_USE_ORIG_FWMARK)
fl6.flowi6_mark = skb->mark;
+ if (iptunnel_handle_offloads(skb, SKB_GSO_IPXIP6))
+ return -1;
+
+ skb_set_inner_ipproto(skb, IPPROTO_IPIP);
+
err = ip6_tnl_xmit(skb, dev, dsfield, &fl6, encap_limit, &mtu,
IPPROTO_IPIP);
if (err != 0) {
@@ -1234,6 +1242,11 @@
if (t->parms.flags & IP6_TNL_F_USE_ORIG_FWMARK)
fl6.flowi6_mark = skb->mark;
+ if (iptunnel_handle_offloads(skb, SKB_GSO_IPXIP6))
+ return -1;
+
+ skb_set_inner_ipproto(skb, IPPROTO_IPV6);
+
err = ip6_tnl_xmit(skb, dev, dsfield, &fl6, encap_limit, &mtu,
IPPROTO_IPV6);
if (err != 0) {
@@ -1280,6 +1293,7 @@
struct net_device *dev = t->dev;
struct __ip6_tnl_parm *p = &t->parms;
struct flowi6 *fl6 = &t->fl.u.ip6;
+ int t_hlen;
memcpy(dev->dev_addr, &p->laddr, sizeof(struct in6_addr));
memcpy(dev->broadcast, &p->raddr, sizeof(struct in6_addr));
@@ -1303,6 +1317,10 @@
else
dev->flags &= ~IFF_POINTOPOINT;
+ t->tun_hlen = 0;
+ t->hlen = t->encap_hlen + t->tun_hlen;
+ t_hlen = t->hlen + sizeof(struct ipv6hdr);
+
if (p->flags & IP6_TNL_F_CAP_XMIT) {
int strict = (ipv6_addr_type(&p->raddr) &
(IPV6_ADDR_MULTICAST|IPV6_ADDR_LINKLOCAL));
@@ -1316,9 +1334,9 @@
if (rt->dst.dev) {
dev->hard_header_len = rt->dst.dev->hard_header_len +
- sizeof(struct ipv6hdr);
+ t_hlen;
- dev->mtu = rt->dst.dev->mtu - sizeof(struct ipv6hdr);
+ dev->mtu = rt->dst.dev->mtu - t_hlen;
if (!(t->parms.flags & IP6_TNL_F_IGN_ENCAP_LIMIT))
dev->mtu -= 8;
@@ -1564,6 +1582,59 @@
}
EXPORT_SYMBOL(ip6_tnl_get_iflink);
+int ip6_tnl_encap_add_ops(const struct ip6_tnl_encap_ops *ops,
+ unsigned int num)
+{
+ if (num >= MAX_IPTUN_ENCAP_OPS)
+ return -ERANGE;
+
+ return !cmpxchg((const struct ip6_tnl_encap_ops **)
+ &ip6tun_encaps[num],
+ NULL, ops) ? 0 : -1;
+}
+EXPORT_SYMBOL(ip6_tnl_encap_add_ops);
+
+int ip6_tnl_encap_del_ops(const struct ip6_tnl_encap_ops *ops,
+ unsigned int num)
+{
+ int ret;
+
+ if (num >= MAX_IPTUN_ENCAP_OPS)
+ return -ERANGE;
+
+ ret = (cmpxchg((const struct ip6_tnl_encap_ops **)
+ &ip6tun_encaps[num],
+ ops, NULL) == ops) ? 0 : -1;
+
+ synchronize_net();
+
+ return ret;
+}
+EXPORT_SYMBOL(ip6_tnl_encap_del_ops);
+
+int ip6_tnl_encap_setup(struct ip6_tnl *t,
+ struct ip_tunnel_encap *ipencap)
+{
+ int hlen;
+
+ memset(&t->encap, 0, sizeof(t->encap));
+
+ hlen = ip6_encap_hlen(ipencap);
+ if (hlen < 0)
+ return hlen;
+
+ t->encap.type = ipencap->type;
+ t->encap.sport = ipencap->sport;
+ t->encap.dport = ipencap->dport;
+ t->encap.flags = ipencap->flags;
+
+ t->encap_hlen = hlen;
+ t->hlen = t->encap_hlen + t->tun_hlen;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(ip6_tnl_encap_setup);
+
static const struct net_device_ops ip6_tnl_netdev_ops = {
.ndo_init = ip6_tnl_dev_init,
.ndo_uninit = ip6_tnl_dev_uninit,
@@ -1574,6 +1645,11 @@
.ndo_get_iflink = ip6_tnl_get_iflink,
};
+#define IPXIPX_FEATURES (NETIF_F_SG | \
+ NETIF_F_FRAGLIST | \
+ NETIF_F_HIGHDMA | \
+ NETIF_F_GSO_SOFTWARE | \
+ NETIF_F_HW_CSUM)
/**
* ip6_tnl_dev_setup - setup virtual tunnel device
@@ -1585,20 +1661,18 @@
static void ip6_tnl_dev_setup(struct net_device *dev)
{
- struct ip6_tnl *t;
-
dev->netdev_ops = &ip6_tnl_netdev_ops;
dev->destructor = ip6_dev_free;
dev->type = ARPHRD_TUNNEL6;
- dev->hard_header_len = LL_MAX_HEADER + sizeof(struct ipv6hdr);
- dev->mtu = ETH_DATA_LEN - sizeof(struct ipv6hdr);
- t = netdev_priv(dev);
- if (!(t->parms.flags & IP6_TNL_F_IGN_ENCAP_LIMIT))
- dev->mtu -= 8;
dev->flags |= IFF_NOARP;
dev->addr_len = sizeof(struct in6_addr);
+ dev->features |= NETIF_F_LLTX;
netif_keep_dst(dev);
+
+ dev->features |= IPXIPX_FEATURES;
+ dev->hw_features |= IPXIPX_FEATURES;
+
/* This perm addr will be used as interface identifier by IPv6 */
dev->addr_assign_type = NET_ADDR_RANDOM;
eth_random_addr(dev->perm_addr);
@@ -1615,6 +1689,7 @@
{
struct ip6_tnl *t = netdev_priv(dev);
int ret;
+ int t_hlen;
t->dev = dev;
t->net = dev_net(dev);
@@ -1630,8 +1705,15 @@
if (ret)
goto destroy_dst;
- t->hlen = 0;
t->tun_hlen = 0;
+ t->hlen = t->encap_hlen + t->tun_hlen;
+ t_hlen = t->hlen + sizeof(struct ipv6hdr);
+
+ dev->type = ARPHRD_TUNNEL6;
+ dev->hard_header_len = LL_MAX_HEADER + t_hlen;
+ dev->mtu = ETH_DATA_LEN - t_hlen;
+ if (!(t->parms.flags & IP6_TNL_F_IGN_ENCAP_LIMIT))
+ dev->mtu -= 8;
return 0;
@@ -1729,13 +1811,55 @@
parms->proto = nla_get_u8(data[IFLA_IPTUN_PROTO]);
}
+static bool ip6_tnl_netlink_encap_parms(struct nlattr *data[],
+ struct ip_tunnel_encap *ipencap)
+{
+ bool ret = false;
+
+ memset(ipencap, 0, sizeof(*ipencap));
+
+ if (!data)
+ return ret;
+
+ if (data[IFLA_IPTUN_ENCAP_TYPE]) {
+ ret = true;
+ ipencap->type = nla_get_u16(data[IFLA_IPTUN_ENCAP_TYPE]);
+ }
+
+ if (data[IFLA_IPTUN_ENCAP_FLAGS]) {
+ ret = true;
+ ipencap->flags = nla_get_u16(data[IFLA_IPTUN_ENCAP_FLAGS]);
+ }
+
+ if (data[IFLA_IPTUN_ENCAP_SPORT]) {
+ ret = true;
+ ipencap->sport = nla_get_be16(data[IFLA_IPTUN_ENCAP_SPORT]);
+ }
+
+ if (data[IFLA_IPTUN_ENCAP_DPORT]) {
+ ret = true;
+ ipencap->dport = nla_get_be16(data[IFLA_IPTUN_ENCAP_DPORT]);
+ }
+
+ return ret;
+}
+
static int ip6_tnl_newlink(struct net *src_net, struct net_device *dev,
struct nlattr *tb[], struct nlattr *data[])
{
struct net *net = dev_net(dev);
struct ip6_tnl *nt, *t;
+ struct ip_tunnel_encap ipencap;
nt = netdev_priv(dev);
+
+ if (ip6_tnl_netlink_encap_parms(data, &ipencap)) {
+ int err = ip6_tnl_encap_setup(nt, &ipencap);
+
+ if (err < 0)
+ return err;
+ }
+
ip6_tnl_netlink_parms(data, &nt->parms);
t = ip6_tnl_locate(net, &nt->parms, 0);
@@ -1752,10 +1876,17 @@
struct __ip6_tnl_parm p;
struct net *net = t->net;
struct ip6_tnl_net *ip6n = net_generic(net, ip6_tnl_net_id);
+ struct ip_tunnel_encap ipencap;
if (dev == ip6n->fb_tnl_dev)
return -EINVAL;
+ if (ip6_tnl_netlink_encap_parms(data, &ipencap)) {
+ int err = ip6_tnl_encap_setup(t, &ipencap);
+
+ if (err < 0)
+ return err;
+ }
ip6_tnl_netlink_parms(data, &p);
t = ip6_tnl_locate(net, &p, 0);
@@ -1796,6 +1927,14 @@
nla_total_size(4) +
/* IFLA_IPTUN_PROTO */
nla_total_size(1) +
+ /* IFLA_IPTUN_ENCAP_TYPE */
+ nla_total_size(2) +
+ /* IFLA_IPTUN_ENCAP_FLAGS */
+ nla_total_size(2) +
+ /* IFLA_IPTUN_ENCAP_SPORT */
+ nla_total_size(2) +
+ /* IFLA_IPTUN_ENCAP_DPORT */
+ nla_total_size(2) +
0;
}
@@ -1813,6 +1952,17 @@
nla_put_u32(skb, IFLA_IPTUN_FLAGS, parm->flags) ||
nla_put_u8(skb, IFLA_IPTUN_PROTO, parm->proto))
goto nla_put_failure;
+
+ if (nla_put_u16(skb, IFLA_IPTUN_ENCAP_TYPE,
+ tunnel->encap.type) ||
+ nla_put_be16(skb, IFLA_IPTUN_ENCAP_SPORT,
+ tunnel->encap.sport) ||
+ nla_put_be16(skb, IFLA_IPTUN_ENCAP_DPORT,
+ tunnel->encap.dport) ||
+ nla_put_u16(skb, IFLA_IPTUN_ENCAP_FLAGS,
+ tunnel->encap.flags))
+ goto nla_put_failure;
+
return 0;
nla_put_failure:
@@ -1836,6 +1986,10 @@
[IFLA_IPTUN_FLOWINFO] = { .type = NLA_U32 },
[IFLA_IPTUN_FLAGS] = { .type = NLA_U32 },
[IFLA_IPTUN_PROTO] = { .type = NLA_U8 },
+ [IFLA_IPTUN_ENCAP_TYPE] = { .type = NLA_U16 },
+ [IFLA_IPTUN_ENCAP_FLAGS] = { .type = NLA_U16 },
+ [IFLA_IPTUN_ENCAP_SPORT] = { .type = NLA_U16 },
+ [IFLA_IPTUN_ENCAP_DPORT] = { .type = NLA_U16 },
};
static struct rtnl_link_ops ip6_link_ops __read_mostly = {
diff --git a/net/ipv6/sit.c b/net/ipv6/sit.c
index a13d8c1..0a5a255 100644
--- a/net/ipv6/sit.c
+++ b/net/ipv6/sit.c
@@ -913,7 +913,7 @@
goto tx_error;
}
- if (iptunnel_handle_offloads(skb, SKB_GSO_SIT)) {
+ if (iptunnel_handle_offloads(skb, SKB_GSO_IPXIP4)) {
ip_rt_put(rt);
goto tx_error;
}
@@ -1000,7 +1000,7 @@
struct ip_tunnel *tunnel = netdev_priv(dev);
const struct iphdr *tiph = &tunnel->parms.iph;
- if (iptunnel_handle_offloads(skb, SKB_GSO_IPIP))
+ if (iptunnel_handle_offloads(skb, SKB_GSO_IPXIP4))
goto tx_error;
skb_set_inner_ipproto(skb, IPPROTO_IPIP);
diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c
index 2ba6a77..2da1896 100644
--- a/net/ipv6/udp.c
+++ b/net/ipv6/udp.c
@@ -617,7 +617,7 @@
/* if we're overly short, let UDP handle it */
encap_rcv = ACCESS_ONCE(up->encap_rcv);
- if (skb->len > sizeof(struct udphdr) && encap_rcv) {
+ if (encap_rcv) {
int ret;
/* Verify checksum before giving to encap */
diff --git a/net/ipv6/udp_offload.c b/net/ipv6/udp_offload.c
index 5429f6b..ac858c4 100644
--- a/net/ipv6/udp_offload.c
+++ b/net/ipv6/udp_offload.c
@@ -36,19 +36,6 @@
if (skb_gso_ok(skb, features | NETIF_F_GSO_ROBUST)) {
/* Packet is from an untrusted source, reset gso_segs. */
- int type = skb_shinfo(skb)->gso_type;
-
- if (unlikely(type & ~(SKB_GSO_UDP |
- SKB_GSO_DODGY |
- SKB_GSO_UDP_TUNNEL |
- SKB_GSO_UDP_TUNNEL_CSUM |
- SKB_GSO_TUNNEL_REMCSUM |
- SKB_GSO_GRE |
- SKB_GSO_GRE_CSUM |
- SKB_GSO_IPIP |
- SKB_GSO_SIT) ||
- !(type & (SKB_GSO_UDP))))
- goto out;
skb_shinfo(skb)->gso_segs = DIV_ROUND_UP(skb->len, mss);
diff --git a/net/irda/ircomm/ircomm_tty.c b/net/irda/ircomm/ircomm_tty.c
index da126ee..873c4b7 100644
--- a/net/irda/ircomm/ircomm_tty.c
+++ b/net/irda/ircomm/ircomm_tty.c
@@ -220,10 +220,11 @@
IRDA_ASSERT(self->magic == IRCOMM_TTY_MAGIC, return -1;);
/* Check if already open */
- if (test_and_set_bit(ASYNCB_INITIALIZED, &self->port.flags)) {
+ if (tty_port_initialized(&self->port)) {
pr_debug("%s(), already open so break out!\n", __func__);
return 0;
}
+ tty_port_set_initialized(&self->port, 1);
/* Register with IrCOMM */
irda_notify_init(¬ify);
@@ -257,7 +258,7 @@
return 0;
err:
- clear_bit(ASYNCB_INITIALIZED, &self->port.flags);
+ tty_port_set_initialized(&self->port, 0);
return ret;
}
@@ -280,8 +281,8 @@
* If non-blocking mode is set, or the port is not enabled,
* then make the check up front and then exit.
*/
- if (test_bit(TTY_IO_ERROR, &tty->flags)) {
- port->flags |= ASYNC_NORMAL_ACTIVE;
+ if (tty_io_error(tty)) {
+ tty_port_set_active(port, 1);
return 0;
}
@@ -289,7 +290,7 @@
/* nonblock mode is set */
if (C_BAUD(tty))
tty_port_raise_dtr_rts(port);
- port->flags |= ASYNC_NORMAL_ACTIVE;
+ tty_port_set_active(port, 1);
pr_debug("%s(), O_NONBLOCK requested!\n", __func__);
return 0;
}
@@ -318,13 +319,12 @@
spin_unlock_irqrestore(&port->lock, flags);
while (1) {
- if (C_BAUD(tty) && test_bit(ASYNCB_INITIALIZED, &port->flags))
+ if (C_BAUD(tty) && tty_port_initialized(port))
tty_port_raise_dtr_rts(port);
set_current_state(TASK_INTERRUPTIBLE);
- if (tty_hung_up_p(filp) ||
- !test_bit(ASYNCB_INITIALIZED, &port->flags)) {
+ if (tty_hung_up_p(filp) || !tty_port_initialized(port)) {
retval = (port->flags & ASYNC_HUP_NOTIFY) ?
-EAGAIN : -ERESTARTSYS;
break;
@@ -365,7 +365,7 @@
__FILE__, __LINE__, tty->driver->name, port->count);
if (!retval)
- port->flags |= ASYNC_NORMAL_ACTIVE;
+ tty_port_set_active(port, 1);
return retval;
}
@@ -876,8 +876,9 @@
IRDA_ASSERT(self != NULL, return;);
IRDA_ASSERT(self->magic == IRCOMM_TTY_MAGIC, return;);
- if (!test_and_clear_bit(ASYNCB_INITIALIZED, &self->port.flags))
+ if (!tty_port_initialized(&self->port))
return;
+ tty_port_set_initialized(&self->port, 0);
ircomm_tty_detach_cable(self);
@@ -925,7 +926,6 @@
ircomm_tty_shutdown(self);
spin_lock_irqsave(&port->lock, flags);
- port->flags &= ~ASYNC_NORMAL_ACTIVE;
if (port->tty) {
set_bit(TTY_IO_ERROR, &port->tty->flags);
tty_kref_put(port->tty);
@@ -933,6 +933,7 @@
port->tty = NULL;
port->count = 0;
spin_unlock_irqrestore(&port->lock, flags);
+ tty_port_set_active(port, 0);
wake_up_interruptible(&port->open_wait);
}
@@ -999,7 +1000,7 @@
if (status & IRCOMM_DCE_DELTA_ANY) {
/*wake_up_interruptible(&self->delta_msr_wait);*/
}
- if ((self->port.flags & ASYNC_CHECK_CD) && (status & IRCOMM_DELTA_CD)) {
+ if (tty_port_check_carrier(&self->port) && (status & IRCOMM_DELTA_CD)) {
pr_debug("%s(), ircomm%d CD now %s...\n", __func__ , self->line,
(status & IRCOMM_CD) ? "on" : "off");
@@ -1255,11 +1256,11 @@
seq_printf(m, "%cASYNC_CTS_FLOW", sep);
sep = '|';
}
- if (self->port.flags & ASYNC_CHECK_CD) {
+ if (tty_port_check_carrier(&self->port)) {
seq_printf(m, "%cASYNC_CHECK_CD", sep);
sep = '|';
}
- if (self->port.flags & ASYNC_INITIALIZED) {
+ if (tty_port_initialized(&self->port)) {
seq_printf(m, "%cASYNC_INITIALIZED", sep);
sep = '|';
}
@@ -1267,7 +1268,7 @@
seq_printf(m, "%cASYNC_LOW_LATENCY", sep);
sep = '|';
}
- if (self->port.flags & ASYNC_NORMAL_ACTIVE) {
+ if (tty_port_active(&self->port)) {
seq_printf(m, "%cASYNC_NORMAL_ACTIVE", sep);
sep = '|';
}
diff --git a/net/irda/ircomm/ircomm_tty_attach.c b/net/irda/ircomm/ircomm_tty_attach.c
index 61137f8..0a41101 100644
--- a/net/irda/ircomm/ircomm_tty_attach.c
+++ b/net/irda/ircomm/ircomm_tty_attach.c
@@ -968,7 +968,7 @@
ircomm_tty_next_state(self, IRCOMM_TTY_SEARCH);
ircomm_tty_start_watchdog_timer(self, 3*HZ);
- if (self->port.flags & ASYNC_CHECK_CD) {
+ if (tty_port_check_carrier(&self->port)) {
/* Drop carrier */
self->settings.dce = IRCOMM_DELTA_CD;
ircomm_tty_check_modem_status(self);
diff --git a/net/irda/ircomm/ircomm_tty_ioctl.c b/net/irda/ircomm/ircomm_tty_ioctl.c
index d3687aa..d4fdf8f 100644
--- a/net/irda/ircomm/ircomm_tty_ioctl.c
+++ b/net/irda/ircomm/ircomm_tty_ioctl.c
@@ -86,21 +86,17 @@
ircomm_param_request(self, IRCOMM_DATA_RATE, FALSE);
/* CTS flow control flag and modem status interrupts */
+ tty_port_set_cts_flow(&self->port, cflag & CRTSCTS);
if (cflag & CRTSCTS) {
- self->port.flags |= ASYNC_CTS_FLOW;
self->settings.flow_control |= IRCOMM_RTS_CTS_IN;
/* This got me. Bummer. Jean II */
if (self->service_type == IRCOMM_3_WIRE_RAW)
net_warn_ratelimited("%s(), enabling RTS/CTS on link that doesn't support it (3-wire-raw)\n",
__func__);
} else {
- self->port.flags &= ~ASYNC_CTS_FLOW;
self->settings.flow_control &= ~IRCOMM_RTS_CTS_IN;
}
- if (cflag & CLOCAL)
- self->port.flags &= ~ASYNC_CHECK_CD;
- else
- self->port.flags |= ASYNC_CHECK_CD;
+ tty_port_set_check_carrier(&self->port, ~cflag & CLOCAL);
#if 0
/*
* Set up parity check flag
@@ -166,7 +162,7 @@
/* Handle transition away from B0 status */
if (!(old_termios->c_cflag & CBAUD) && (cflag & CBAUD)) {
self->settings.dte |= IRCOMM_DTR;
- if (!C_CRTSCTS(tty) || !test_bit(TTY_THROTTLED, &tty->flags))
+ if (!C_CRTSCTS(tty) || !tty_throttled(tty))
self->settings.dte |= IRCOMM_RTS;
ircomm_param_request(self, IRCOMM_DTE, TRUE);
}
@@ -190,7 +186,7 @@
struct ircomm_tty_cb *self = (struct ircomm_tty_cb *) tty->driver_data;
unsigned int result;
- if (tty->flags & (1 << TTY_IO_ERROR))
+ if (tty_io_error(tty))
return -EIO;
result = ((self->settings.dte & IRCOMM_RTS) ? TIOCM_RTS : 0)
@@ -213,7 +209,7 @@
{
struct ircomm_tty_cb *self = (struct ircomm_tty_cb *) tty->driver_data;
- if (tty->flags & (1 << TTY_IO_ERROR))
+ if (tty_io_error(tty))
return -EIO;
IRDA_ASSERT(self != NULL, return -1;);
@@ -328,7 +324,7 @@
check_and_exit:
- if (self->flags & ASYNC_INITIALIZED) {
+ if (tty_port_initialized(self)) {
if (((old_state.flags & ASYNC_SPD_MASK) !=
(self->flags & ASYNC_SPD_MASK)) ||
(old_driver.custom_divisor != driver->custom_divisor)) {
@@ -362,7 +358,7 @@
if ((cmd != TIOCGSERIAL) && (cmd != TIOCSSERIAL) &&
(cmd != TIOCSERCONFIG) && (cmd != TIOCSERGSTRUCT) &&
(cmd != TIOCMIWAIT) && (cmd != TIOCGICOUNT)) {
- if (tty->flags & (1 << TTY_IO_ERROR))
+ if (tty_io_error(tty))
return -EIO;
}
diff --git a/net/kcm/kcmsock.c b/net/kcm/kcmsock.c
index 40662d73..0b68ba7 100644
--- a/net/kcm/kcmsock.c
+++ b/net/kcm/kcmsock.c
@@ -1483,7 +1483,7 @@
long timeo;
struct kcm_rx_msg *rxm;
int err = 0;
- size_t copied;
+ ssize_t copied;
struct sk_buff *skb;
/* Only support splice for SOCKSEQPACKET */
diff --git a/net/mpls/mpls_gso.c b/net/mpls/mpls_gso.c
index bbcf604..2055e57 100644
--- a/net/mpls/mpls_gso.c
+++ b/net/mpls/mpls_gso.c
@@ -26,15 +26,6 @@
netdev_features_t mpls_features;
__be16 mpls_protocol;
- if (unlikely(skb_shinfo(skb)->gso_type &
- ~(SKB_GSO_TCPV4 |
- SKB_GSO_TCPV6 |
- SKB_GSO_UDP |
- SKB_GSO_DODGY |
- SKB_GSO_TCP_FIXEDID |
- SKB_GSO_TCP_ECN)))
- goto out;
-
/* Setup inner SKB. */
mpls_protocol = skb->protocol;
skb->protocol = skb->inner_protocol;
@@ -57,7 +48,7 @@
* skb_mac_gso_segment(), an indirect caller of this function.
*/
__skb_pull(skb, skb->data - skb_mac_header(skb));
-out:
+
return segs;
}
diff --git a/net/netfilter/ipvs/ip_vs_xmit.c b/net/netfilter/ipvs/ip_vs_xmit.c
index 6d19d2e..01d3d89 100644
--- a/net/netfilter/ipvs/ip_vs_xmit.c
+++ b/net/netfilter/ipvs/ip_vs_xmit.c
@@ -932,17 +932,14 @@
static inline int __tun_gso_type_mask(int encaps_af, int orig_af)
{
- if (encaps_af == AF_INET) {
- if (orig_af == AF_INET)
- return SKB_GSO_IPIP;
-
- return SKB_GSO_SIT;
+ switch (encaps_af) {
+ case AF_INET:
+ return SKB_GSO_IPXIP4;
+ case AF_INET6:
+ return SKB_GSO_IPXIP6;
+ default:
+ return 0;
}
-
- /* GSO: we need to provide proper SKB_GSO_ value for IPv6:
- * SKB_GSO_SIT/IPV6
- */
- return 0;
}
/*
diff --git a/net/netlabel/netlabel_kapi.c b/net/netlabel/netlabel_kapi.c
index 28cddc8..1325776 100644
--- a/net/netlabel/netlabel_kapi.c
+++ b/net/netlabel/netlabel_kapi.c
@@ -677,7 +677,7 @@
u32 spot = start;
while (rc == 0 && spot <= end) {
- if (((spot & (BITS_PER_LONG - 1)) != 0) &&
+ if (((spot & (BITS_PER_LONG - 1)) == 0) &&
((end - spot) > BITS_PER_LONG)) {
rc = netlbl_catmap_setlong(catmap,
spot,
diff --git a/net/rds/ib_frmr.c b/net/rds/ib_frmr.c
index 93ff038..d921adc 100644
--- a/net/rds/ib_frmr.c
+++ b/net/rds/ib_frmr.c
@@ -111,7 +111,7 @@
cpu_relax();
}
- ret = ib_map_mr_sg_zbva(frmr->mr, ibmr->sg, ibmr->sg_len, PAGE_SIZE);
+ ret = ib_map_mr_sg_zbva(frmr->mr, ibmr->sg, ibmr->sg_len, 0, PAGE_SIZE);
if (unlikely(ret != ibmr->sg_len))
return ret < 0 ? ret : -EINVAL;
diff --git a/net/rds/tcp_connect.c b/net/rds/tcp_connect.c
index 49a3fcf..fb82e0a 100644
--- a/net/rds/tcp_connect.c
+++ b/net/rds/tcp_connect.c
@@ -43,7 +43,7 @@
struct rds_connection *conn;
struct rds_tcp_connection *tc;
- read_lock(&sk->sk_callback_lock);
+ read_lock_bh(&sk->sk_callback_lock);
conn = sk->sk_user_data;
if (!conn) {
state_change = sk->sk_state_change;
@@ -69,7 +69,7 @@
break;
}
out:
- read_unlock(&sk->sk_callback_lock);
+ read_unlock_bh(&sk->sk_callback_lock);
state_change(sk);
}
diff --git a/net/rds/tcp_listen.c b/net/rds/tcp_listen.c
index be263cd..4bf4bef 100644
--- a/net/rds/tcp_listen.c
+++ b/net/rds/tcp_listen.c
@@ -80,6 +80,9 @@
int conn_state;
struct sock *nsk;
+ if (!sock) /* module unload or netns delete in progress */
+ return -ENETUNREACH;
+
ret = sock_create_kern(sock_net(sock->sk), sock->sk->sk_family,
sock->sk->sk_type, sock->sk->sk_protocol,
&new_sock);
@@ -129,11 +132,13 @@
* so we must quiesce any send threads before resetting
* c_transport_data.
*/
- wait_event(conn->c_waitq,
- !test_bit(RDS_IN_XMIT, &conn->c_flags));
- if (ntohl(inet->inet_saddr) < ntohl(inet->inet_daddr)) {
+ if (ntohl(inet->inet_saddr) < ntohl(inet->inet_daddr) ||
+ !conn->c_outgoing) {
goto rst_nsk;
- } else if (rs_tcp->t_sock) {
+ } else {
+ atomic_set(&conn->c_state, RDS_CONN_CONNECTING);
+ wait_event(conn->c_waitq,
+ !test_bit(RDS_IN_XMIT, &conn->c_flags));
rds_tcp_restore_callbacks(rs_tcp->t_sock, rs_tcp);
conn->c_outgoing = 0;
}
@@ -166,7 +171,7 @@
rdsdebug("listen data ready sk %p\n", sk);
- read_lock(&sk->sk_callback_lock);
+ read_lock_bh(&sk->sk_callback_lock);
ready = sk->sk_user_data;
if (!ready) { /* check for teardown race */
ready = sk->sk_data_ready;
@@ -183,7 +188,7 @@
rds_tcp_accept_work(sk);
out:
- read_unlock(&sk->sk_callback_lock);
+ read_unlock_bh(&sk->sk_callback_lock);
ready(sk);
}
diff --git a/net/rds/tcp_recv.c b/net/rds/tcp_recv.c
index d75d8b5..c3196f9 100644
--- a/net/rds/tcp_recv.c
+++ b/net/rds/tcp_recv.c
@@ -301,7 +301,7 @@
rdsdebug("data ready sk %p\n", sk);
- read_lock(&sk->sk_callback_lock);
+ read_lock_bh(&sk->sk_callback_lock);
conn = sk->sk_user_data;
if (!conn) { /* check for teardown race */
ready = sk->sk_data_ready;
@@ -315,7 +315,7 @@
if (rds_tcp_read_sock(conn, GFP_ATOMIC) == -ENOMEM)
queue_delayed_work(rds_wq, &conn->c_recv_w, 0);
out:
- read_unlock(&sk->sk_callback_lock);
+ read_unlock_bh(&sk->sk_callback_lock);
ready(sk);
}
diff --git a/net/rds/tcp_send.c b/net/rds/tcp_send.c
index 2894e60..22d0f20 100644
--- a/net/rds/tcp_send.c
+++ b/net/rds/tcp_send.c
@@ -180,7 +180,7 @@
struct rds_connection *conn;
struct rds_tcp_connection *tc;
- read_lock(&sk->sk_callback_lock);
+ read_lock_bh(&sk->sk_callback_lock);
conn = sk->sk_user_data;
if (!conn) {
write_space = sk->sk_write_space;
@@ -200,7 +200,7 @@
queue_delayed_work(rds_wq, &conn->c_send_w, 0);
out:
- read_unlock(&sk->sk_callback_lock);
+ read_unlock_bh(&sk->sk_callback_lock);
/*
* write_space is only called when data leaves tcp's send queue if
diff --git a/net/rxrpc/ar-key.c b/net/rxrpc/ar-key.c
index 3fb492e..1021b4c 100644
--- a/net/rxrpc/ar-key.c
+++ b/net/rxrpc/ar-key.c
@@ -965,7 +965,7 @@
key = key_alloc(&key_type_rxrpc, "x",
GLOBAL_ROOT_UID, GLOBAL_ROOT_GID, cred, 0,
- KEY_ALLOC_NOT_IN_QUOTA);
+ KEY_ALLOC_NOT_IN_QUOTA, NULL);
if (IS_ERR(key)) {
_leave(" = -ENOMEM [alloc %ld]", PTR_ERR(key));
return -ENOMEM;
@@ -1012,7 +1012,7 @@
key = key_alloc(&key_type_rxrpc, keyname,
GLOBAL_ROOT_UID, GLOBAL_ROOT_GID, cred,
- KEY_POS_SEARCH, KEY_ALLOC_NOT_IN_QUOTA);
+ KEY_POS_SEARCH, KEY_ALLOC_NOT_IN_QUOTA, NULL);
if (IS_ERR(key))
return key;
diff --git a/net/socket.c b/net/socket.c
index e7793f5..a1bd161 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -2168,7 +2168,8 @@
struct mmsghdr __user *entry;
struct compat_mmsghdr __user *compat_entry;
struct msghdr msg_sys;
- struct timespec end_time;
+ struct timespec64 end_time;
+ struct timespec64 timeout64;
if (timeout &&
poll_select_set_timeout(&end_time, timeout->tv_sec,
@@ -2220,8 +2221,9 @@
flags |= MSG_DONTWAIT;
if (timeout) {
- ktime_get_ts(timeout);
- *timeout = timespec_sub(end_time, *timeout);
+ ktime_get_ts64(&timeout64);
+ *timeout = timespec64_to_timespec(
+ timespec64_sub(end_time, timeout64));
if (timeout->tv_sec < 0) {
timeout->tv_sec = timeout->tv_nsec = 0;
break;
diff --git a/net/sunrpc/auth_gss/svcauth_gss.c b/net/sunrpc/auth_gss/svcauth_gss.c
index 1095be9..e085f5a 100644
--- a/net/sunrpc/auth_gss/svcauth_gss.c
+++ b/net/sunrpc/auth_gss/svcauth_gss.c
@@ -569,10 +569,9 @@
struct rsc *found;
memset(&rsci, 0, sizeof(rsci));
- if (dup_to_netobj(&rsci.handle, handle->data, handle->len))
- return NULL;
+ rsci.handle.data = handle->data;
+ rsci.handle.len = handle->len;
found = rsc_lookup(cd, &rsci);
- rsc_free(&rsci);
if (!found)
return NULL;
if (cache_check(cd, &found->h, NULL))
@@ -857,8 +856,8 @@
goto out;
if (svc_getnl(&buf->head[0]) != seq)
goto out;
- /* trim off the mic at the end before returning */
- xdr_buf_trim(buf, mic.len + 4);
+ /* trim off the mic and padding at the end before returning */
+ xdr_buf_trim(buf, round_up_to_quad(mic.len) + 4);
stat = 0;
out:
kfree(mic.data);
diff --git a/net/sunrpc/svc_xprt.c b/net/sunrpc/svc_xprt.c
index 7422f28..f5572e3 100644
--- a/net/sunrpc/svc_xprt.c
+++ b/net/sunrpc/svc_xprt.c
@@ -244,13 +244,12 @@
svc_xprt_received(new);
}
-int svc_create_xprt(struct svc_serv *serv, const char *xprt_name,
+int _svc_create_xprt(struct svc_serv *serv, const char *xprt_name,
struct net *net, const int family,
const unsigned short port, int flags)
{
struct svc_xprt_class *xcl;
- dprintk("svc: creating transport %s[%d]\n", xprt_name, port);
spin_lock(&svc_xprt_class_lock);
list_for_each_entry(xcl, &svc_xprt_class_list, xcl_list) {
struct svc_xprt *newxprt;
@@ -274,12 +273,28 @@
}
err:
spin_unlock(&svc_xprt_class_lock);
- dprintk("svc: transport %s not found\n", xprt_name);
-
/* This errno is exposed to user space. Provide a reasonable
* perror msg for a bad transport. */
return -EPROTONOSUPPORT;
}
+
+int svc_create_xprt(struct svc_serv *serv, const char *xprt_name,
+ struct net *net, const int family,
+ const unsigned short port, int flags)
+{
+ int err;
+
+ dprintk("svc: creating transport %s[%d]\n", xprt_name, port);
+ err = _svc_create_xprt(serv, xprt_name, net, family, port, flags);
+ if (err == -EPROTONOSUPPORT) {
+ request_module("svc%s", xprt_name);
+ err = _svc_create_xprt(serv, xprt_name, net, family, port, flags);
+ }
+ if (err)
+ dprintk("svc: transport %s not found, err %d\n",
+ xprt_name, err);
+ return err;
+}
EXPORT_SYMBOL_GPL(svc_create_xprt);
/*
diff --git a/net/sunrpc/xprtrdma/frwr_ops.c b/net/sunrpc/xprtrdma/frwr_ops.c
index c250924..94c3fa9 100644
--- a/net/sunrpc/xprtrdma/frwr_ops.c
+++ b/net/sunrpc/xprtrdma/frwr_ops.c
@@ -421,7 +421,7 @@
return -ENOMEM;
}
- n = ib_map_mr_sg(mr, frmr->sg, frmr->sg_nents, PAGE_SIZE);
+ n = ib_map_mr_sg(mr, frmr->sg, frmr->sg_nents, NULL, PAGE_SIZE);
if (unlikely(n != frmr->sg_nents)) {
pr_err("RPC: %s: failed to map mr %p (%u/%u)\n",
__func__, frmr->fr_mr, n, frmr->sg_nents);
diff --git a/net/sunrpc/xprtrdma/svc_rdma_marshal.c b/net/sunrpc/xprtrdma/svc_rdma_marshal.c
index 765bca4..0ba9887 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_marshal.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_marshal.c
@@ -145,19 +145,32 @@
return (__be32 *)&ary->wc_array[nchunks];
}
-int svc_rdma_xdr_decode_req(struct rpcrdma_msg *rmsgp, struct svc_rqst *rqstp)
+/**
+ * svc_rdma_xdr_decode_req - Parse incoming RPC-over-RDMA header
+ * @rq_arg: Receive buffer
+ *
+ * On entry, xdr->head[0].iov_base points to first byte in the
+ * RPC-over-RDMA header.
+ *
+ * On successful exit, head[0] points to first byte past the
+ * RPC-over-RDMA header. For RDMA_MSG, this is the RPC message.
+ * The length of the RPC-over-RDMA header is returned.
+ */
+int svc_rdma_xdr_decode_req(struct xdr_buf *rq_arg)
{
+ struct rpcrdma_msg *rmsgp;
__be32 *va, *vaend;
unsigned int len;
u32 hdr_len;
/* Verify that there's enough bytes for header + something */
- if (rqstp->rq_arg.len <= RPCRDMA_HDRLEN_ERR) {
+ if (rq_arg->len <= RPCRDMA_HDRLEN_ERR) {
dprintk("svcrdma: header too short = %d\n",
- rqstp->rq_arg.len);
+ rq_arg->len);
return -EINVAL;
}
+ rmsgp = (struct rpcrdma_msg *)rq_arg->head[0].iov_base;
if (rmsgp->rm_vers != rpcrdma_version) {
dprintk("%s: bad version %u\n", __func__,
be32_to_cpu(rmsgp->rm_vers));
@@ -189,10 +202,10 @@
be32_to_cpu(rmsgp->rm_body.rm_padded.rm_thresh);
va = &rmsgp->rm_body.rm_padded.rm_pempty[4];
- rqstp->rq_arg.head[0].iov_base = va;
+ rq_arg->head[0].iov_base = va;
len = (u32)((unsigned long)va - (unsigned long)rmsgp);
- rqstp->rq_arg.head[0].iov_len -= len;
- if (len > rqstp->rq_arg.len)
+ rq_arg->head[0].iov_len -= len;
+ if (len > rq_arg->len)
return -EINVAL;
return len;
default:
@@ -205,7 +218,7 @@
* chunk list and a reply chunk list.
*/
va = &rmsgp->rm_body.rm_chunks[0];
- vaend = (__be32 *)((unsigned long)rmsgp + rqstp->rq_arg.len);
+ vaend = (__be32 *)((unsigned long)rmsgp + rq_arg->len);
va = decode_read_list(va, vaend);
if (!va) {
dprintk("svcrdma: failed to decode read list\n");
@@ -222,10 +235,9 @@
return -EINVAL;
}
- rqstp->rq_arg.head[0].iov_base = va;
+ rq_arg->head[0].iov_base = va;
hdr_len = (unsigned long)va - (unsigned long)rmsgp;
- rqstp->rq_arg.head[0].iov_len -= hdr_len;
-
+ rq_arg->head[0].iov_len -= hdr_len;
return hdr_len;
}
diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
index 3b24a64..2c25606 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
@@ -281,7 +281,7 @@
}
atomic_inc(&xprt->sc_dma_used);
- n = ib_map_mr_sg(frmr->mr, frmr->sg, frmr->sg_nents, PAGE_SIZE);
+ n = ib_map_mr_sg(frmr->mr, frmr->sg, frmr->sg_nents, NULL, PAGE_SIZE);
if (unlikely(n != frmr->sg_nents)) {
pr_err("svcrdma: failed to map mr %p (%d/%d elements)\n",
frmr->mr, n, frmr->sg_nents);
@@ -447,10 +447,8 @@
head->arg.len = rqstp->rq_arg.len;
head->arg.buflen = rqstp->rq_arg.buflen;
- ch = (struct rpcrdma_read_chunk *)&rmsgp->rm_body.rm_chunks[0];
- position = be32_to_cpu(ch->rc_position);
-
/* RDMA_NOMSG: RDMA READ data should land just after RDMA RECV data */
+ position = be32_to_cpu(ch->rc_position);
if (position == 0) {
head->arg.pages = &head->pages[0];
page_offset = head->byte_len;
@@ -488,7 +486,7 @@
if (page_offset & 3) {
u32 pad = 4 - (page_offset & 3);
- head->arg.page_len += pad;
+ head->arg.tail[0].iov_len += pad;
head->arg.len += pad;
head->arg.buflen += pad;
page_offset += pad;
@@ -510,11 +508,10 @@
return ret;
}
-static int rdma_read_complete(struct svc_rqst *rqstp,
- struct svc_rdma_op_ctxt *head)
+static void rdma_read_complete(struct svc_rqst *rqstp,
+ struct svc_rdma_op_ctxt *head)
{
int page_no;
- int ret;
/* Copy RPC pages */
for (page_no = 0; page_no < head->count; page_no++) {
@@ -550,23 +547,6 @@
rqstp->rq_arg.tail[0] = head->arg.tail[0];
rqstp->rq_arg.len = head->arg.len;
rqstp->rq_arg.buflen = head->arg.buflen;
-
- /* Free the context */
- svc_rdma_put_context(head, 0);
-
- /* XXX: What should this be? */
- rqstp->rq_prot = IPPROTO_MAX;
- svc_xprt_copy_addrs(rqstp, rqstp->rq_xprt);
-
- ret = rqstp->rq_arg.head[0].iov_len
- + rqstp->rq_arg.page_len
- + rqstp->rq_arg.tail[0].iov_len;
- dprintk("svcrdma: deferred read ret=%d, rq_arg.len=%u, "
- "rq_arg.head[0].iov_base=%p, rq_arg.head[0].iov_len=%zu\n",
- ret, rqstp->rq_arg.len, rqstp->rq_arg.head[0].iov_base,
- rqstp->rq_arg.head[0].iov_len);
-
- return ret;
}
/* By convention, backchannel calls arrive via rdma_msg type
@@ -624,7 +604,8 @@
dto_q);
list_del_init(&ctxt->dto_q);
spin_unlock_bh(&rdma_xprt->sc_rq_dto_lock);
- return rdma_read_complete(rqstp, ctxt);
+ rdma_read_complete(rqstp, ctxt);
+ goto complete;
} else if (!list_empty(&rdma_xprt->sc_rq_dto_q)) {
ctxt = list_entry(rdma_xprt->sc_rq_dto_q.next,
struct svc_rdma_op_ctxt,
@@ -655,7 +636,7 @@
/* Decode the RDMA header. */
rmsgp = (struct rpcrdma_msg *)rqstp->rq_arg.head[0].iov_base;
- ret = svc_rdma_xdr_decode_req(rmsgp, rqstp);
+ ret = svc_rdma_xdr_decode_req(&rqstp->rq_arg);
if (ret < 0)
goto out_err;
if (ret == 0)
@@ -682,6 +663,7 @@
return 0;
}
+complete:
ret = rqstp->rq_arg.head[0].iov_len
+ rqstp->rq_arg.page_len
+ rqstp->rq_arg.tail[0].iov_len;
diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
index 4f1b1c4..54d53330 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
@@ -463,25 +463,21 @@
struct svc_rqst *rqstp,
struct page *page,
struct rpcrdma_msg *rdma_resp,
- struct svc_rdma_op_ctxt *ctxt,
struct svc_rdma_req_map *vec,
int byte_count)
{
+ struct svc_rdma_op_ctxt *ctxt;
struct ib_send_wr send_wr;
u32 xdr_off;
int sge_no;
int sge_bytes;
int page_no;
int pages;
- int ret;
-
- ret = svc_rdma_repost_recv(rdma, GFP_KERNEL);
- if (ret) {
- svc_rdma_put_context(ctxt, 0);
- return -ENOTCONN;
- }
+ int ret = -EIO;
/* Prepare the context */
+ ctxt = svc_rdma_get_context(rdma);
+ ctxt->direction = DMA_TO_DEVICE;
ctxt->pages[0] = page;
ctxt->count = 1;
@@ -565,8 +561,7 @@
err:
svc_rdma_unmap_dma(ctxt);
svc_rdma_put_context(ctxt, 1);
- pr_err("svcrdma: failed to send reply, rc=%d\n", ret);
- return -EIO;
+ return ret;
}
void svc_rdma_prep_reply_hdr(struct svc_rqst *rqstp)
@@ -585,7 +580,6 @@
int ret;
int inline_bytes;
struct page *res_page;
- struct svc_rdma_op_ctxt *ctxt;
struct svc_rdma_req_map *vec;
dprintk("svcrdma: sending response for rqstp=%p\n", rqstp);
@@ -598,8 +592,6 @@
rp_ary = svc_rdma_get_reply_array(rdma_argp, wr_ary);
/* Build an req vec for the XDR */
- ctxt = svc_rdma_get_context(rdma);
- ctxt->direction = DMA_TO_DEVICE;
vec = svc_rdma_get_req_map(rdma);
ret = svc_rdma_map_xdr(rdma, &rqstp->rq_res, vec, wr_ary != NULL);
if (ret)
@@ -635,7 +627,12 @@
inline_bytes -= ret;
}
- ret = send_reply(rdma, rqstp, res_page, rdma_resp, ctxt, vec,
+ /* Post a fresh Receive buffer _before_ sending the reply */
+ ret = svc_rdma_post_recv(rdma, GFP_KERNEL);
+ if (ret)
+ goto err1;
+
+ ret = send_reply(rdma, rqstp, res_page, rdma_resp, vec,
inline_bytes);
if (ret < 0)
goto err1;
@@ -648,7 +645,8 @@
put_page(res_page);
err0:
svc_rdma_put_req_map(rdma, vec);
- svc_rdma_put_context(ctxt, 0);
+ pr_err("svcrdma: Could not send reply, err=%d. Closing transport.\n",
+ ret);
set_bit(XPT_CLOSE, &rdma->sc_xprt.xpt_flags);
return -ENOTCONN;
}
diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c
index 9066896..dd94401 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_transport.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c
@@ -789,7 +789,7 @@
int ret;
dprintk("svcrdma: Creating RDMA socket\n");
- if (sa->sa_family != AF_INET) {
+ if ((sa->sa_family != AF_INET) && (sa->sa_family != AF_INET6)) {
dprintk("svcrdma: Address family %d is not supported.\n", sa->sa_family);
return ERR_PTR(-EAFNOSUPPORT);
}
@@ -805,6 +805,16 @@
goto err0;
}
+ /* Allow both IPv4 and IPv6 sockets to bind a single port
+ * at the same time.
+ */
+#if IS_ENABLED(CONFIG_IPV6)
+ ret = rdma_set_afonly(listen_id, 1);
+ if (ret) {
+ dprintk("svcrdma: rdma_set_afonly failed = %d\n", ret);
+ goto err1;
+ }
+#endif
ret = rdma_bind_addr(listen_id, sa);
if (ret) {
dprintk("svcrdma: rdma_bind_addr failed = %d\n", ret);
@@ -1073,7 +1083,7 @@
newxprt->sc_dev_caps |= SVCRDMA_DEVCAP_READ_W_INV;
/* Post receive buffers */
- for (i = 0; i < newxprt->sc_rq_depth; i++) {
+ for (i = 0; i < newxprt->sc_max_requests; i++) {
ret = svc_rdma_post_recv(newxprt, GFP_KERNEL);
if (ret) {
dprintk("svcrdma: failure posting receive buffers\n");
@@ -1170,6 +1180,9 @@
dprintk("svcrdma: %s(%p)\n", __func__, rdma);
+ if (rdma->sc_qp && !IS_ERR(rdma->sc_qp))
+ ib_drain_qp(rdma->sc_qp);
+
/* We should only be called from kref_put */
if (atomic_read(&xprt->xpt_ref.refcount) != 0)
pr_err("svcrdma: sc_xprt still in use? (%d)\n",
diff --git a/net/tipc/server.c b/net/tipc/server.c
index 7a0af2d..272d20a 100644
--- a/net/tipc/server.c
+++ b/net/tipc/server.c
@@ -138,28 +138,28 @@
{
struct tipc_conn *con;
- read_lock(&sk->sk_callback_lock);
+ read_lock_bh(&sk->sk_callback_lock);
con = sock2con(sk);
if (con && test_bit(CF_CONNECTED, &con->flags)) {
conn_get(con);
if (!queue_work(con->server->rcv_wq, &con->rwork))
conn_put(con);
}
- read_unlock(&sk->sk_callback_lock);
+ read_unlock_bh(&sk->sk_callback_lock);
}
static void sock_write_space(struct sock *sk)
{
struct tipc_conn *con;
- read_lock(&sk->sk_callback_lock);
+ read_lock_bh(&sk->sk_callback_lock);
con = sock2con(sk);
if (con && test_bit(CF_CONNECTED, &con->flags)) {
conn_get(con);
if (!queue_work(con->server->send_wq, &con->swork))
conn_put(con);
}
- read_unlock(&sk->sk_callback_lock);
+ read_unlock_bh(&sk->sk_callback_lock);
}
static void tipc_register_callbacks(struct socket *sock, struct tipc_conn *con)
diff --git a/net/wireless/util.c b/net/wireless/util.c
index 219bd19..4e809e9 100644
--- a/net/wireless/util.c
+++ b/net/wireless/util.c
@@ -651,7 +651,7 @@
struct skb_shared_info *sh = skb_shinfo(skb);
int page_offset;
- atomic_inc(&page->_count);
+ page_ref_inc(page);
page_offset = ptr - page_address(page);
skb_add_rx_frag(skb, sh->nr_frags, page, page_offset, len, size);
}
diff --git a/samples/Kconfig b/samples/Kconfig
index d54f28c..559a58b 100644
--- a/samples/Kconfig
+++ b/samples/Kconfig
@@ -76,4 +76,13 @@
help
Builds a sample configfs interface.
+config SAMPLE_CONNECTOR
+ tristate "Build connector sample -- loadable modules only"
+ depends on CONNECTOR && m
+ help
+ When enabled, this builds both a sample kernel module for
+ the connector interface and a user space tool to communicate
+ with it.
+ See also Documentation/connector/connector.txt
+
endif # SAMPLES
diff --git a/samples/Makefile b/samples/Makefile
index 48001d7..2e3b523 100644
--- a/samples/Makefile
+++ b/samples/Makefile
@@ -2,4 +2,4 @@
obj-$(CONFIG_SAMPLES) += kobject/ kprobes/ trace_events/ livepatch/ \
hw_breakpoint/ kfifo/ kdb/ hidraw/ rpmsg/ seccomp/ \
- configfs/
+ configfs/ connector/ v4l/
diff --git a/Documentation/connector/.gitignore b/samples/connector/.gitignore
similarity index 100%
rename from Documentation/connector/.gitignore
rename to samples/connector/.gitignore
diff --git a/samples/connector/Makefile b/samples/connector/Makefile
new file mode 100644
index 0000000..04b9622
--- /dev/null
+++ b/samples/connector/Makefile
@@ -0,0 +1,16 @@
+obj-$(CONFIG_SAMPLE_CONNECTOR) += cn_test.o
+
+# List of programs to build
+ifdef CONFIG_SAMPLE_CONNECTOR
+hostprogs-y := ucon
+endif
+
+# Tell kbuild to always build the programs
+always := $(hostprogs-y)
+
+HOSTCFLAGS_ucon.o += -I$(objtree)/usr/include
+
+all: modules
+
+modules clean:
+ $(MAKE) -C ../.. SUBDIRS=$(PWD) $@
diff --git a/Documentation/connector/cn_test.c b/samples/connector/cn_test.c
similarity index 100%
rename from Documentation/connector/cn_test.c
rename to samples/connector/cn_test.c
diff --git a/Documentation/connector/ucon.c b/samples/connector/ucon.c
similarity index 100%
rename from Documentation/connector/ucon.c
rename to samples/connector/ucon.c
diff --git a/samples/kprobes/jprobe_example.c b/samples/kprobes/jprobe_example.c
index c285a3b..c3108bb 100644
--- a/samples/kprobes/jprobe_example.c
+++ b/samples/kprobes/jprobe_example.c
@@ -25,7 +25,7 @@
/* Proxy routine having the same arguments as actual _do_fork() routine */
static long j_do_fork(unsigned long clone_flags, unsigned long stack_start,
unsigned long stack_size, int __user *parent_tidptr,
- int __user *child_tidptr)
+ int __user *child_tidptr, unsigned long tls)
{
pr_info("jprobe: clone_flags = 0x%lx, stack_start = 0x%lx "
"stack_size = 0x%lx\n", clone_flags, stack_start, stack_size);
diff --git a/samples/kprobes/kprobe_example.c b/samples/kprobes/kprobe_example.c
index 727eb21..ed0ca0c 100644
--- a/samples/kprobes/kprobe_example.c
+++ b/samples/kprobes/kprobe_example.c
@@ -14,33 +14,37 @@
#include <linux/module.h>
#include <linux/kprobes.h>
+#define MAX_SYMBOL_LEN 64
+static char symbol[MAX_SYMBOL_LEN] = "_do_fork";
+module_param_string(symbol, symbol, sizeof(symbol), 0644);
+
/* For each probe you need to allocate a kprobe structure */
static struct kprobe kp = {
- .symbol_name = "_do_fork",
+ .symbol_name = symbol,
};
/* kprobe pre_handler: called just before the probed instruction is executed */
static int handler_pre(struct kprobe *p, struct pt_regs *regs)
{
#ifdef CONFIG_X86
- printk(KERN_INFO "pre_handler: p->addr = 0x%p, ip = %lx,"
+ printk(KERN_INFO "<%s> pre_handler: p->addr = 0x%p, ip = %lx,"
" flags = 0x%lx\n",
- p->addr, regs->ip, regs->flags);
+ p->symbol_name, p->addr, regs->ip, regs->flags);
#endif
#ifdef CONFIG_PPC
- printk(KERN_INFO "pre_handler: p->addr = 0x%p, nip = 0x%lx,"
+ printk(KERN_INFO "<%s> pre_handler: p->addr = 0x%p, nip = 0x%lx,"
" msr = 0x%lx\n",
- p->addr, regs->nip, regs->msr);
+ p->symbol_name, p->addr, regs->nip, regs->msr);
#endif
#ifdef CONFIG_MIPS
- printk(KERN_INFO "pre_handler: p->addr = 0x%p, epc = 0x%lx,"
+ printk(KERN_INFO "<%s> pre_handler: p->addr = 0x%p, epc = 0x%lx,"
" status = 0x%lx\n",
- p->addr, regs->cp0_epc, regs->cp0_status);
+ p->symbol_name, p->addr, regs->cp0_epc, regs->cp0_status);
#endif
#ifdef CONFIG_TILEGX
- printk(KERN_INFO "pre_handler: p->addr = 0x%p, pc = 0x%lx,"
+ printk(KERN_INFO "<%s> pre_handler: p->addr = 0x%p, pc = 0x%lx,"
" ex1 = 0x%lx\n",
- p->addr, regs->pc, regs->ex1);
+ p->symbol_name, p->addr, regs->pc, regs->ex1);
#endif
/* A dump_stack() here will give a stack backtrace */
@@ -52,20 +56,20 @@
unsigned long flags)
{
#ifdef CONFIG_X86
- printk(KERN_INFO "post_handler: p->addr = 0x%p, flags = 0x%lx\n",
- p->addr, regs->flags);
+ printk(KERN_INFO "<%s> post_handler: p->addr = 0x%p, flags = 0x%lx\n",
+ p->symbol_name, p->addr, regs->flags);
#endif
#ifdef CONFIG_PPC
- printk(KERN_INFO "post_handler: p->addr = 0x%p, msr = 0x%lx\n",
- p->addr, regs->msr);
+ printk(KERN_INFO "<%s> post_handler: p->addr = 0x%p, msr = 0x%lx\n",
+ p->symbol_name, p->addr, regs->msr);
#endif
#ifdef CONFIG_MIPS
- printk(KERN_INFO "post_handler: p->addr = 0x%p, status = 0x%lx\n",
- p->addr, regs->cp0_status);
+ printk(KERN_INFO "<%s> post_handler: p->addr = 0x%p, status = 0x%lx\n",
+ p->symbol_name, p->addr, regs->cp0_status);
#endif
#ifdef CONFIG_TILEGX
- printk(KERN_INFO "post_handler: p->addr = 0x%p, ex1 = 0x%lx\n",
- p->addr, regs->ex1);
+ printk(KERN_INFO "<%s> post_handler: p->addr = 0x%p, ex1 = 0x%lx\n",
+ p->symbol_name, p->addr, regs->ex1);
#endif
}
diff --git a/samples/rpmsg/rpmsg_client_sample.c b/samples/rpmsg/rpmsg_client_sample.c
index 59b1344..d0e249c9 100644
--- a/samples/rpmsg/rpmsg_client_sample.c
+++ b/samples/rpmsg/rpmsg_client_sample.c
@@ -77,24 +77,12 @@
static struct rpmsg_driver rpmsg_sample_client = {
.drv.name = KBUILD_MODNAME,
- .drv.owner = THIS_MODULE,
.id_table = rpmsg_driver_sample_id_table,
.probe = rpmsg_sample_probe,
.callback = rpmsg_sample_cb,
.remove = rpmsg_sample_remove,
};
-
-static int __init rpmsg_client_sample_init(void)
-{
- return register_rpmsg_driver(&rpmsg_sample_client);
-}
-module_init(rpmsg_client_sample_init);
-
-static void __exit rpmsg_client_sample_fini(void)
-{
- unregister_rpmsg_driver(&rpmsg_sample_client);
-}
-module_exit(rpmsg_client_sample_fini);
+module_rpmsg_driver(rpmsg_sample_client);
MODULE_DESCRIPTION("Remote processor messaging sample client driver");
MODULE_LICENSE("GPL v2");
diff --git a/Documentation/video4linux/Makefile b/samples/v4l/Makefile
similarity index 100%
rename from Documentation/video4linux/Makefile
rename to samples/v4l/Makefile
diff --git a/samples/v4l/v4l2-pci-skeleton.c b/samples/v4l/v4l2-pci-skeleton.c
new file mode 100644
index 0000000..a55cf94
--- /dev/null
+++ b/samples/v4l/v4l2-pci-skeleton.c
@@ -0,0 +1,922 @@
+/*
+ * This is a V4L2 PCI Skeleton Driver. It gives an initial skeleton source
+ * for use with other PCI drivers.
+ *
+ * This skeleton PCI driver assumes that the card has an S-Video connector as
+ * input 0 and an HDMI connector as input 1.
+ *
+ * Copyright 2014 Cisco Systems, Inc. and/or its affiliates. All rights reserved.
+ *
+ * This program is free software; you may redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <linux/types.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/kmod.h>
+#include <linux/mutex.h>
+#include <linux/pci.h>
+#include <linux/interrupt.h>
+#include <linux/videodev2.h>
+#include <linux/v4l2-dv-timings.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-dev.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-dv-timings.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-event.h>
+#include <media/videobuf2-v4l2.h>
+#include <media/videobuf2-dma-contig.h>
+
+MODULE_DESCRIPTION("V4L2 PCI Skeleton Driver");
+MODULE_AUTHOR("Hans Verkuil");
+MODULE_LICENSE("GPL v2");
+
+/**
+ * struct skeleton - All internal data for one instance of device
+ * @pdev: PCI device
+ * @v4l2_dev: top-level v4l2 device struct
+ * @vdev: video node structure
+ * @ctrl_handler: control handler structure
+ * @lock: ioctl serialization mutex
+ * @std: current SDTV standard
+ * @timings: current HDTV timings
+ * @format: current pix format
+ * @input: current video input (0 = SDTV, 1 = HDTV)
+ * @queue: vb2 video capture queue
+ * @alloc_ctx: vb2 contiguous DMA context
+ * @qlock: spinlock controlling access to buf_list and sequence
+ * @buf_list: list of buffers queued for DMA
+ * @sequence: frame sequence counter
+ */
+struct skeleton {
+ struct pci_dev *pdev;
+ struct v4l2_device v4l2_dev;
+ struct video_device vdev;
+ struct v4l2_ctrl_handler ctrl_handler;
+ struct mutex lock;
+ v4l2_std_id std;
+ struct v4l2_dv_timings timings;
+ struct v4l2_pix_format format;
+ unsigned input;
+
+ struct vb2_queue queue;
+ struct vb2_alloc_ctx *alloc_ctx;
+
+ spinlock_t qlock;
+ struct list_head buf_list;
+ unsigned field;
+ unsigned sequence;
+};
+
+struct skel_buffer {
+ struct vb2_buffer vb;
+ struct list_head list;
+};
+
+static inline struct skel_buffer *to_skel_buffer(struct vb2_buffer *vb2)
+{
+ return container_of(vb2, struct skel_buffer, vb);
+}
+
+static const struct pci_device_id skeleton_pci_tbl[] = {
+ /* { PCI_DEVICE(PCI_VENDOR_ID_, PCI_DEVICE_ID_) }, */
+ { 0, }
+};
+MODULE_DEVICE_TABLE(pci, skeleton_pci_tbl);
+
+/*
+ * HDTV: this structure has the capabilities of the HDTV receiver.
+ * It is used to constrain the huge list of possible formats based
+ * upon the hardware capabilities.
+ */
+static const struct v4l2_dv_timings_cap skel_timings_cap = {
+ .type = V4L2_DV_BT_656_1120,
+ /* keep this initialization for compatibility with GCC < 4.4.6 */
+ .reserved = { 0 },
+ V4L2_INIT_BT_TIMINGS(
+ 720, 1920, /* min/max width */
+ 480, 1080, /* min/max height */
+ 27000000, 74250000, /* min/max pixelclock*/
+ V4L2_DV_BT_STD_CEA861, /* Supported standards */
+ /* capabilities */
+ V4L2_DV_BT_CAP_INTERLACED | V4L2_DV_BT_CAP_PROGRESSIVE
+ )
+};
+
+/*
+ * Supported SDTV standards. This does the same job as skel_timings_cap, but
+ * for standard TV formats.
+ */
+#define SKEL_TVNORMS V4L2_STD_ALL
+
+/*
+ * Interrupt handler: typically interrupts happen after a new frame has been
+ * captured. It is the job of the handler to remove the new frame from the
+ * internal list and give it back to the vb2 framework, updating the sequence
+ * counter, field and timestamp at the same time.
+ */
+static irqreturn_t skeleton_irq(int irq, void *dev_id)
+{
+#ifdef TODO
+ struct skeleton *skel = dev_id;
+
+ /* handle interrupt */
+
+ /* Once a new frame has been captured, mark it as done like this: */
+ if (captured_new_frame) {
+ ...
+ spin_lock(&skel->qlock);
+ list_del(&new_buf->list);
+ spin_unlock(&skel->qlock);
+ v4l2_get_timestamp(&new_buf->vb.v4l2_buf.timestamp);
+ new_buf->vb.v4l2_buf.sequence = skel->sequence++;
+ new_buf->vb.v4l2_buf.field = skel->field;
+ if (skel->format.field == V4L2_FIELD_ALTERNATE) {
+ if (skel->field == V4L2_FIELD_BOTTOM)
+ skel->field = V4L2_FIELD_TOP;
+ else if (skel->field == V4L2_FIELD_TOP)
+ skel->field = V4L2_FIELD_BOTTOM;
+ }
+ vb2_buffer_done(&new_buf->vb, VB2_BUF_STATE_DONE);
+ }
+#endif
+ return IRQ_HANDLED;
+}
+
+/*
+ * Setup the constraints of the queue: besides setting the number of planes
+ * per buffer and the size and allocation context of each plane, it also
+ * checks if sufficient buffers have been allocated. Usually 3 is a good
+ * minimum number: many DMA engines need a minimum of 2 buffers in the
+ * queue and you need to have another available for userspace processing.
+ */
+static int queue_setup(struct vb2_queue *vq,
+ unsigned int *nbuffers, unsigned int *nplanes,
+ unsigned int sizes[], void *alloc_ctxs[])
+{
+ struct skeleton *skel = vb2_get_drv_priv(vq);
+
+ skel->field = skel->format.field;
+ if (skel->field == V4L2_FIELD_ALTERNATE) {
+ /*
+ * You cannot use read() with FIELD_ALTERNATE since the field
+ * information (TOP/BOTTOM) cannot be passed back to the user.
+ */
+ if (vb2_fileio_is_active(vq))
+ return -EINVAL;
+ skel->field = V4L2_FIELD_TOP;
+ }
+
+ if (vq->num_buffers + *nbuffers < 3)
+ *nbuffers = 3 - vq->num_buffers;
+ alloc_ctxs[0] = skel->alloc_ctx;
+
+ if (*nplanes)
+ return sizes[0] < skel->format.sizeimage ? -EINVAL : 0;
+ *nplanes = 1;
+ sizes[0] = skel->format.sizeimage;
+ return 0;
+}
+
+/*
+ * Prepare the buffer for queueing to the DMA engine: check and set the
+ * payload size.
+ */
+static int buffer_prepare(struct vb2_buffer *vb)
+{
+ struct skeleton *skel = vb2_get_drv_priv(vb->vb2_queue);
+ unsigned long size = skel->format.sizeimage;
+
+ if (vb2_plane_size(vb, 0) < size) {
+ dev_err(&skel->pdev->dev, "buffer too small (%lu < %lu)\n",
+ vb2_plane_size(vb, 0), size);
+ return -EINVAL;
+ }
+
+ vb2_set_plane_payload(vb, 0, size);
+ return 0;
+}
+
+/*
+ * Queue this buffer to the DMA engine.
+ */
+static void buffer_queue(struct vb2_buffer *vb)
+{
+ struct skeleton *skel = vb2_get_drv_priv(vb->vb2_queue);
+ struct skel_buffer *buf = to_skel_buffer(vb);
+ unsigned long flags;
+
+ spin_lock_irqsave(&skel->qlock, flags);
+ list_add_tail(&buf->list, &skel->buf_list);
+
+ /* TODO: Update any DMA pointers if necessary */
+
+ spin_unlock_irqrestore(&skel->qlock, flags);
+}
+
+static void return_all_buffers(struct skeleton *skel,
+ enum vb2_buffer_state state)
+{
+ struct skel_buffer *buf, *node;
+ unsigned long flags;
+
+ spin_lock_irqsave(&skel->qlock, flags);
+ list_for_each_entry_safe(buf, node, &skel->buf_list, list) {
+ vb2_buffer_done(&buf->vb, state);
+ list_del(&buf->list);
+ }
+ spin_unlock_irqrestore(&skel->qlock, flags);
+}
+
+/*
+ * Start streaming. First check if the minimum number of buffers have been
+ * queued. If not, then return -ENOBUFS and the vb2 framework will call
+ * this function again the next time a buffer has been queued until enough
+ * buffers are available to actually start the DMA engine.
+ */
+static int start_streaming(struct vb2_queue *vq, unsigned int count)
+{
+ struct skeleton *skel = vb2_get_drv_priv(vq);
+ int ret = 0;
+
+ skel->sequence = 0;
+
+ /* TODO: start DMA */
+
+ if (ret) {
+ /*
+ * In case of an error, return all active buffers to the
+ * QUEUED state
+ */
+ return_all_buffers(skel, VB2_BUF_STATE_QUEUED);
+ }
+ return ret;
+}
+
+/*
+ * Stop the DMA engine. Any remaining buffers in the DMA queue are dequeued
+ * and passed on to the vb2 framework marked as STATE_ERROR.
+ */
+static void stop_streaming(struct vb2_queue *vq)
+{
+ struct skeleton *skel = vb2_get_drv_priv(vq);
+
+ /* TODO: stop DMA */
+
+ /* Release all active buffers */
+ return_all_buffers(skel, VB2_BUF_STATE_ERROR);
+}
+
+/*
+ * The vb2 queue ops. Note that since q->lock is set we can use the standard
+ * vb2_ops_wait_prepare/finish helper functions. If q->lock would be NULL,
+ * then this driver would have to provide these ops.
+ */
+static struct vb2_ops skel_qops = {
+ .queue_setup = queue_setup,
+ .buf_prepare = buffer_prepare,
+ .buf_queue = buffer_queue,
+ .start_streaming = start_streaming,
+ .stop_streaming = stop_streaming,
+ .wait_prepare = vb2_ops_wait_prepare,
+ .wait_finish = vb2_ops_wait_finish,
+};
+
+/*
+ * Required ioctl querycap. Note that the version field is prefilled with
+ * the version of the kernel.
+ */
+static int skeleton_querycap(struct file *file, void *priv,
+ struct v4l2_capability *cap)
+{
+ struct skeleton *skel = video_drvdata(file);
+
+ strlcpy(cap->driver, KBUILD_MODNAME, sizeof(cap->driver));
+ strlcpy(cap->card, "V4L2 PCI Skeleton", sizeof(cap->card));
+ snprintf(cap->bus_info, sizeof(cap->bus_info), "PCI:%s",
+ pci_name(skel->pdev));
+ return 0;
+}
+
+/*
+ * Helper function to check and correct struct v4l2_pix_format. It's used
+ * not only in VIDIOC_TRY/S_FMT, but also elsewhere if changes to the SDTV
+ * standard, HDTV timings or the video input would require updating the
+ * current format.
+ */
+static void skeleton_fill_pix_format(struct skeleton *skel,
+ struct v4l2_pix_format *pix)
+{
+ pix->pixelformat = V4L2_PIX_FMT_YUYV;
+ if (skel->input == 0) {
+ /* S-Video input */
+ pix->width = 720;
+ pix->height = (skel->std & V4L2_STD_525_60) ? 480 : 576;
+ pix->field = V4L2_FIELD_INTERLACED;
+ pix->colorspace = V4L2_COLORSPACE_SMPTE170M;
+ } else {
+ /* HDMI input */
+ pix->width = skel->timings.bt.width;
+ pix->height = skel->timings.bt.height;
+ if (skel->timings.bt.interlaced) {
+ pix->field = V4L2_FIELD_ALTERNATE;
+ pix->height /= 2;
+ } else {
+ pix->field = V4L2_FIELD_NONE;
+ }
+ pix->colorspace = V4L2_COLORSPACE_REC709;
+ }
+
+ /*
+ * The YUYV format is four bytes for every two pixels, so bytesperline
+ * is width * 2.
+ */
+ pix->bytesperline = pix->width * 2;
+ pix->sizeimage = pix->bytesperline * pix->height;
+ pix->priv = 0;
+}
+
+static int skeleton_try_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct skeleton *skel = video_drvdata(file);
+ struct v4l2_pix_format *pix = &f->fmt.pix;
+
+ /*
+ * Due to historical reasons providing try_fmt with an unsupported
+ * pixelformat will return -EINVAL for video receivers. Webcam drivers,
+ * however, will silently correct the pixelformat. Some video capture
+ * applications rely on this behavior...
+ */
+ if (pix->pixelformat != V4L2_PIX_FMT_YUYV)
+ return -EINVAL;
+ skeleton_fill_pix_format(skel, pix);
+ return 0;
+}
+
+static int skeleton_s_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct skeleton *skel = video_drvdata(file);
+ int ret;
+
+ ret = skeleton_try_fmt_vid_cap(file, priv, f);
+ if (ret)
+ return ret;
+
+ /*
+ * It is not allowed to change the format while buffers for use with
+ * streaming have already been allocated.
+ */
+ if (vb2_is_busy(&skel->queue))
+ return -EBUSY;
+
+ /* TODO: change format */
+ skel->format = f->fmt.pix;
+ return 0;
+}
+
+static int skeleton_g_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct skeleton *skel = video_drvdata(file);
+
+ f->fmt.pix = skel->format;
+ return 0;
+}
+
+static int skeleton_enum_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_fmtdesc *f)
+{
+ if (f->index != 0)
+ return -EINVAL;
+
+ f->pixelformat = V4L2_PIX_FMT_YUYV;
+ return 0;
+}
+
+static int skeleton_s_std(struct file *file, void *priv, v4l2_std_id std)
+{
+ struct skeleton *skel = video_drvdata(file);
+
+ /* S_STD is not supported on the HDMI input */
+ if (skel->input)
+ return -ENODATA;
+
+ /*
+ * No change, so just return. Some applications call S_STD again after
+ * the buffers for streaming have been set up, so we have to allow for
+ * this behavior.
+ */
+ if (std == skel->std)
+ return 0;
+
+ /*
+ * Changing the standard implies a format change, which is not allowed
+ * while buffers for use with streaming have already been allocated.
+ */
+ if (vb2_is_busy(&skel->queue))
+ return -EBUSY;
+
+ /* TODO: handle changing std */
+
+ skel->std = std;
+
+ /* Update the internal format */
+ skeleton_fill_pix_format(skel, &skel->format);
+ return 0;
+}
+
+static int skeleton_g_std(struct file *file, void *priv, v4l2_std_id *std)
+{
+ struct skeleton *skel = video_drvdata(file);
+
+ /* G_STD is not supported on the HDMI input */
+ if (skel->input)
+ return -ENODATA;
+
+ *std = skel->std;
+ return 0;
+}
+
+/*
+ * Query the current standard as seen by the hardware. This function shall
+ * never actually change the standard, it just detects and reports.
+ * The framework will initially set *std to tvnorms (i.e. the set of
+ * supported standards by this input), and this function should just AND
+ * this value. If there is no signal, then *std should be set to 0.
+ */
+static int skeleton_querystd(struct file *file, void *priv, v4l2_std_id *std)
+{
+ struct skeleton *skel = video_drvdata(file);
+
+ /* QUERY_STD is not supported on the HDMI input */
+ if (skel->input)
+ return -ENODATA;
+
+#ifdef TODO
+ /*
+ * Query currently seen standard. Initial value of *std is
+ * V4L2_STD_ALL. This function should look something like this:
+ */
+ get_signal_info();
+ if (no_signal) {
+ *std = 0;
+ return 0;
+ }
+ /* Use signal information to reduce the number of possible standards */
+ if (signal_has_525_lines)
+ *std &= V4L2_STD_525_60;
+ else
+ *std &= V4L2_STD_625_50;
+#endif
+ return 0;
+}
+
+static int skeleton_s_dv_timings(struct file *file, void *_fh,
+ struct v4l2_dv_timings *timings)
+{
+ struct skeleton *skel = video_drvdata(file);
+
+ /* S_DV_TIMINGS is not supported on the S-Video input */
+ if (skel->input == 0)
+ return -ENODATA;
+
+ /* Quick sanity check */
+ if (!v4l2_valid_dv_timings(timings, &skel_timings_cap, NULL, NULL))
+ return -EINVAL;
+
+ /* Check if the timings are part of the CEA-861 timings. */
+ if (!v4l2_find_dv_timings_cap(timings, &skel_timings_cap,
+ 0, NULL, NULL))
+ return -EINVAL;
+
+ /* Return 0 if the new timings are the same as the current timings. */
+ if (v4l2_match_dv_timings(timings, &skel->timings, 0, false))
+ return 0;
+
+ /*
+ * Changing the timings implies a format change, which is not allowed
+ * while buffers for use with streaming have already been allocated.
+ */
+ if (vb2_is_busy(&skel->queue))
+ return -EBUSY;
+
+ /* TODO: Configure new timings */
+
+ /* Save timings */
+ skel->timings = *timings;
+
+ /* Update the internal format */
+ skeleton_fill_pix_format(skel, &skel->format);
+ return 0;
+}
+
+static int skeleton_g_dv_timings(struct file *file, void *_fh,
+ struct v4l2_dv_timings *timings)
+{
+ struct skeleton *skel = video_drvdata(file);
+
+ /* G_DV_TIMINGS is not supported on the S-Video input */
+ if (skel->input == 0)
+ return -ENODATA;
+
+ *timings = skel->timings;
+ return 0;
+}
+
+static int skeleton_enum_dv_timings(struct file *file, void *_fh,
+ struct v4l2_enum_dv_timings *timings)
+{
+ struct skeleton *skel = video_drvdata(file);
+
+ /* ENUM_DV_TIMINGS is not supported on the S-Video input */
+ if (skel->input == 0)
+ return -ENODATA;
+
+ return v4l2_enum_dv_timings_cap(timings, &skel_timings_cap,
+ NULL, NULL);
+}
+
+/*
+ * Query the current timings as seen by the hardware. This function shall
+ * never actually change the timings, it just detects and reports.
+ * If no signal is detected, then return -ENOLINK. If the hardware cannot
+ * lock to the signal, then return -ENOLCK. If the signal is out of range
+ * of the capabilities of the system (e.g., it is possible that the receiver
+ * can lock but that the DMA engine it is connected to cannot handle
+ * pixelclocks above a certain frequency), then -ERANGE is returned.
+ */
+static int skeleton_query_dv_timings(struct file *file, void *_fh,
+ struct v4l2_dv_timings *timings)
+{
+ struct skeleton *skel = video_drvdata(file);
+
+ /* QUERY_DV_TIMINGS is not supported on the S-Video input */
+ if (skel->input == 0)
+ return -ENODATA;
+
+#ifdef TODO
+ /*
+ * Query currently seen timings. This function should look
+ * something like this:
+ */
+ detect_timings();
+ if (no_signal)
+ return -ENOLINK;
+ if (cannot_lock_to_signal)
+ return -ENOLCK;
+ if (signal_out_of_range_of_capabilities)
+ return -ERANGE;
+
+ /* Useful for debugging */
+ v4l2_print_dv_timings(skel->v4l2_dev.name, "query_dv_timings:",
+ timings, true);
+#endif
+ return 0;
+}
+
+static int skeleton_dv_timings_cap(struct file *file, void *fh,
+ struct v4l2_dv_timings_cap *cap)
+{
+ struct skeleton *skel = video_drvdata(file);
+
+ /* DV_TIMINGS_CAP is not supported on the S-Video input */
+ if (skel->input == 0)
+ return -ENODATA;
+ *cap = skel_timings_cap;
+ return 0;
+}
+
+static int skeleton_enum_input(struct file *file, void *priv,
+ struct v4l2_input *i)
+{
+ if (i->index > 1)
+ return -EINVAL;
+
+ i->type = V4L2_INPUT_TYPE_CAMERA;
+ if (i->index == 0) {
+ i->std = SKEL_TVNORMS;
+ strlcpy(i->name, "S-Video", sizeof(i->name));
+ i->capabilities = V4L2_IN_CAP_STD;
+ } else {
+ i->std = 0;
+ strlcpy(i->name, "HDMI", sizeof(i->name));
+ i->capabilities = V4L2_IN_CAP_DV_TIMINGS;
+ }
+ return 0;
+}
+
+static int skeleton_s_input(struct file *file, void *priv, unsigned int i)
+{
+ struct skeleton *skel = video_drvdata(file);
+
+ if (i > 1)
+ return -EINVAL;
+
+ /*
+ * Changing the input implies a format change, which is not allowed
+ * while buffers for use with streaming have already been allocated.
+ */
+ if (vb2_is_busy(&skel->queue))
+ return -EBUSY;
+
+ skel->input = i;
+ /*
+ * Update tvnorms. The tvnorms value is used by the core to implement
+ * VIDIOC_ENUMSTD so it has to be correct. If tvnorms == 0, then
+ * ENUMSTD will return -ENODATA.
+ */
+ skel->vdev.tvnorms = i ? 0 : SKEL_TVNORMS;
+
+ /* Update the internal format */
+ skeleton_fill_pix_format(skel, &skel->format);
+ return 0;
+}
+
+static int skeleton_g_input(struct file *file, void *priv, unsigned int *i)
+{
+ struct skeleton *skel = video_drvdata(file);
+
+ *i = skel->input;
+ return 0;
+}
+
+/* The control handler. */
+static int skeleton_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ /*struct skeleton *skel =
+ container_of(ctrl->handler, struct skeleton, ctrl_handler);*/
+
+ switch (ctrl->id) {
+ case V4L2_CID_BRIGHTNESS:
+ /* TODO: set brightness to ctrl->val */
+ break;
+ case V4L2_CID_CONTRAST:
+ /* TODO: set contrast to ctrl->val */
+ break;
+ case V4L2_CID_SATURATION:
+ /* TODO: set saturation to ctrl->val */
+ break;
+ case V4L2_CID_HUE:
+ /* TODO: set hue to ctrl->val */
+ break;
+ default:
+ return -EINVAL;
+ }
+ return 0;
+}
+
+/* ------------------------------------------------------------------
+ File operations for the device
+ ------------------------------------------------------------------*/
+
+static const struct v4l2_ctrl_ops skel_ctrl_ops = {
+ .s_ctrl = skeleton_s_ctrl,
+};
+
+/*
+ * The set of all supported ioctls. Note that all the streaming ioctls
+ * use the vb2 helper functions that take care of all the locking and
+ * that also do ownership tracking (i.e. only the filehandle that requested
+ * the buffers can call the streaming ioctls, all other filehandles will
+ * receive -EBUSY if they attempt to call the same streaming ioctls).
+ *
+ * The last three ioctls also use standard helper functions: these implement
+ * standard behavior for drivers with controls.
+ */
+static const struct v4l2_ioctl_ops skel_ioctl_ops = {
+ .vidioc_querycap = skeleton_querycap,
+ .vidioc_try_fmt_vid_cap = skeleton_try_fmt_vid_cap,
+ .vidioc_s_fmt_vid_cap = skeleton_s_fmt_vid_cap,
+ .vidioc_g_fmt_vid_cap = skeleton_g_fmt_vid_cap,
+ .vidioc_enum_fmt_vid_cap = skeleton_enum_fmt_vid_cap,
+
+ .vidioc_g_std = skeleton_g_std,
+ .vidioc_s_std = skeleton_s_std,
+ .vidioc_querystd = skeleton_querystd,
+
+ .vidioc_s_dv_timings = skeleton_s_dv_timings,
+ .vidioc_g_dv_timings = skeleton_g_dv_timings,
+ .vidioc_enum_dv_timings = skeleton_enum_dv_timings,
+ .vidioc_query_dv_timings = skeleton_query_dv_timings,
+ .vidioc_dv_timings_cap = skeleton_dv_timings_cap,
+
+ .vidioc_enum_input = skeleton_enum_input,
+ .vidioc_g_input = skeleton_g_input,
+ .vidioc_s_input = skeleton_s_input,
+
+ .vidioc_reqbufs = vb2_ioctl_reqbufs,
+ .vidioc_create_bufs = vb2_ioctl_create_bufs,
+ .vidioc_querybuf = vb2_ioctl_querybuf,
+ .vidioc_qbuf = vb2_ioctl_qbuf,
+ .vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_expbuf = vb2_ioctl_expbuf,
+ .vidioc_streamon = vb2_ioctl_streamon,
+ .vidioc_streamoff = vb2_ioctl_streamoff,
+
+ .vidioc_log_status = v4l2_ctrl_log_status,
+ .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+/*
+ * The set of file operations. Note that all these ops are standard core
+ * helper functions.
+ */
+static const struct v4l2_file_operations skel_fops = {
+ .owner = THIS_MODULE,
+ .open = v4l2_fh_open,
+ .release = vb2_fop_release,
+ .unlocked_ioctl = video_ioctl2,
+ .read = vb2_fop_read,
+ .mmap = vb2_fop_mmap,
+ .poll = vb2_fop_poll,
+};
+
+/*
+ * The initial setup of this device instance. Note that the initial state of
+ * the driver should be complete. So the initial format, standard, timings
+ * and video input should all be initialized to some reasonable value.
+ */
+static int skeleton_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
+{
+ /* The initial timings are chosen to be 720p60. */
+ static const struct v4l2_dv_timings timings_def =
+ V4L2_DV_BT_CEA_1280X720P60;
+ struct skeleton *skel;
+ struct video_device *vdev;
+ struct v4l2_ctrl_handler *hdl;
+ struct vb2_queue *q;
+ int ret;
+
+ /* Enable PCI */
+ ret = pci_enable_device(pdev);
+ if (ret)
+ return ret;
+ ret = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
+ if (ret) {
+ dev_err(&pdev->dev, "no suitable DMA available.\n");
+ goto disable_pci;
+ }
+
+ /* Allocate a new instance */
+ skel = devm_kzalloc(&pdev->dev, sizeof(struct skeleton), GFP_KERNEL);
+ if (!skel)
+ return -ENOMEM;
+
+ /* Allocate the interrupt */
+ ret = devm_request_irq(&pdev->dev, pdev->irq,
+ skeleton_irq, 0, KBUILD_MODNAME, skel);
+ if (ret) {
+ dev_err(&pdev->dev, "request_irq failed\n");
+ goto disable_pci;
+ }
+ skel->pdev = pdev;
+
+ /* Fill in the initial format-related settings */
+ skel->timings = timings_def;
+ skel->std = V4L2_STD_625_50;
+ skeleton_fill_pix_format(skel, &skel->format);
+
+ /* Initialize the top-level structure */
+ ret = v4l2_device_register(&pdev->dev, &skel->v4l2_dev);
+ if (ret)
+ goto disable_pci;
+
+ mutex_init(&skel->lock);
+
+ /* Add the controls */
+ hdl = &skel->ctrl_handler;
+ v4l2_ctrl_handler_init(hdl, 4);
+ v4l2_ctrl_new_std(hdl, &skel_ctrl_ops,
+ V4L2_CID_BRIGHTNESS, 0, 255, 1, 127);
+ v4l2_ctrl_new_std(hdl, &skel_ctrl_ops,
+ V4L2_CID_CONTRAST, 0, 255, 1, 16);
+ v4l2_ctrl_new_std(hdl, &skel_ctrl_ops,
+ V4L2_CID_SATURATION, 0, 255, 1, 127);
+ v4l2_ctrl_new_std(hdl, &skel_ctrl_ops,
+ V4L2_CID_HUE, -128, 127, 1, 0);
+ if (hdl->error) {
+ ret = hdl->error;
+ goto free_hdl;
+ }
+ skel->v4l2_dev.ctrl_handler = hdl;
+
+ /* Initialize the vb2 queue */
+ q = &skel->queue;
+ q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+ q->io_modes = VB2_MMAP | VB2_DMABUF | VB2_READ;
+ q->drv_priv = skel;
+ q->buf_struct_size = sizeof(struct skel_buffer);
+ q->ops = &skel_qops;
+ q->mem_ops = &vb2_dma_contig_memops;
+ q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+ /*
+ * Assume that this DMA engine needs to have at least two buffers
+ * available before it can be started. The start_streaming() op
+ * won't be called until at least this many buffers are queued up.
+ */
+ q->min_buffers_needed = 2;
+ /*
+ * The serialization lock for the streaming ioctls. This is the same
+ * as the main serialization lock, but if some of the non-streaming
+ * ioctls could take a long time to execute, then you might want to
+ * have a different lock here to prevent VIDIOC_DQBUF from being
+ * blocked while waiting for another action to finish. This is
+ * generally not needed for PCI devices, but USB devices usually do
+ * want a separate lock here.
+ */
+ q->lock = &skel->lock;
+ /*
+ * Since this driver can only do 32-bit DMA we must make sure that
+ * the vb2 core will allocate the buffers in 32-bit DMA memory.
+ */
+ q->gfp_flags = GFP_DMA32;
+ ret = vb2_queue_init(q);
+ if (ret)
+ goto free_hdl;
+
+ skel->alloc_ctx = vb2_dma_contig_init_ctx(&pdev->dev);
+ if (IS_ERR(skel->alloc_ctx)) {
+ dev_err(&pdev->dev, "Can't allocate buffer context");
+ ret = PTR_ERR(skel->alloc_ctx);
+ goto free_hdl;
+ }
+ INIT_LIST_HEAD(&skel->buf_list);
+ spin_lock_init(&skel->qlock);
+
+ /* Initialize the video_device structure */
+ vdev = &skel->vdev;
+ strlcpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name));
+ /*
+ * There is nothing to clean up, so release is set to an empty release
+ * function. The release callback must be non-NULL.
+ */
+ vdev->release = video_device_release_empty;
+ vdev->fops = &skel_fops,
+ vdev->ioctl_ops = &skel_ioctl_ops,
+ vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_READWRITE |
+ V4L2_CAP_STREAMING;
+ /*
+ * The main serialization lock. All ioctls are serialized by this
+ * lock. Exception: if q->lock is set, then the streaming ioctls
+ * are serialized by that separate lock.
+ */
+ vdev->lock = &skel->lock;
+ vdev->queue = q;
+ vdev->v4l2_dev = &skel->v4l2_dev;
+ /* Supported SDTV standards, if any */
+ vdev->tvnorms = SKEL_TVNORMS;
+ video_set_drvdata(vdev, skel);
+
+ ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+ if (ret)
+ goto free_ctx;
+
+ dev_info(&pdev->dev, "V4L2 PCI Skeleton Driver loaded\n");
+ return 0;
+
+free_ctx:
+ vb2_dma_contig_cleanup_ctx(skel->alloc_ctx);
+free_hdl:
+ v4l2_ctrl_handler_free(&skel->ctrl_handler);
+ v4l2_device_unregister(&skel->v4l2_dev);
+disable_pci:
+ pci_disable_device(pdev);
+ return ret;
+}
+
+static void skeleton_remove(struct pci_dev *pdev)
+{
+ struct v4l2_device *v4l2_dev = pci_get_drvdata(pdev);
+ struct skeleton *skel = container_of(v4l2_dev, struct skeleton, v4l2_dev);
+
+ video_unregister_device(&skel->vdev);
+ v4l2_ctrl_handler_free(&skel->ctrl_handler);
+ vb2_dma_contig_cleanup_ctx(skel->alloc_ctx);
+ v4l2_device_unregister(&skel->v4l2_dev);
+ pci_disable_device(skel->pdev);
+}
+
+static struct pci_driver skeleton_driver = {
+ .name = KBUILD_MODNAME,
+ .probe = skeleton_probe,
+ .remove = skeleton_remove,
+ .id_table = skeleton_pci_tbl,
+};
+
+module_pci_driver(skeleton_driver);
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index ddf83d0..ed1b7c4 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -277,6 +277,11 @@
# ---------------------------------------------------------------------------
DTC ?= $(objtree)/scripts/dtc/dtc
+# Disable noisy checks by default
+ifeq ($(KBUILD_ENABLE_EXTRA_GCC_CHECKS),)
+DTC_FLAGS += -Wno-unit_address_vs_reg
+endif
+
# Generate an assembly file to wrap the output of the device tree compiler
quiet_cmd_dt_S_dtb= DTB $@
cmd_dt_S_dtb= \
diff --git a/scripts/bloat-o-meter b/scripts/bloat-o-meter
index 38b64f4..0254f3b 100755
--- a/scripts/bloat-o-meter
+++ b/scripts/bloat-o-meter
@@ -32,18 +32,21 @@
new = getsizes(sys.argv[2])
grow, shrink, add, remove, up, down = 0, 0, 0, 0, 0, 0
delta, common = [], {}
+otot, ntot = 0, 0
for a in old:
if a in new:
common[a] = 1
for name in old:
+ otot += old[name]
if name not in common:
remove += 1
down += old[name]
delta.append((-old[name], name))
for name in new:
+ ntot += new[name]
if name not in common:
add += 1
up += new[name]
@@ -63,3 +66,6 @@
print("%-40s %7s %7s %+7s" % ("function", "old", "new", "delta"))
for d, n in delta:
if d: print("%-40s %7s %7s %+7d" % (n, old.get(n,"-"), new.get(n,"-"), d))
+
+print("Total: Before=%d, After=%d, chg %f%%" % \
+ (otot, ntot, (ntot - otot)*100/otot))
diff --git a/scripts/checkkconfigsymbols.py b/scripts/checkkconfigsymbols.py
index d8f6c09..df643f6 100755
--- a/scripts/checkkconfigsymbols.py
+++ b/scripts/checkkconfigsymbols.py
@@ -89,7 +89,7 @@
if opts.diff and not re.match(r"^[\w\-\.]+\.\.[\w\-\.]+$", opts.diff):
sys.exit("Please specify valid input in the following format: "
- "\'commmit1..commit2\'")
+ "\'commit1..commit2\'")
if opts.commit or opts.diff:
if not opts.force and tree_is_dirty():
diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index d574d13..6750595 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -27,12 +27,15 @@
my $terse = 0;
my $showfile = 0;
my $file = 0;
+my $git = 0;
+my %git_commits = ();
my $check = 0;
my $check_orig = 0;
my $summary = 1;
my $mailback = 0;
my $summary_file = 0;
my $show_types = 0;
+my $list_types = 0;
my $fix = 0;
my $fix_inplace = 0;
my $root;
@@ -68,13 +71,24 @@
--emacs emacs compile window format
--terse one line per report
--showfile emit diffed file position, not input file position
+ -g, --git treat FILE as a single commit or git revision range
+ single git commit with:
+ <rev>
+ <rev>^
+ <rev>~n
+ multiple git commits with:
+ <rev1>..<rev2>
+ <rev1>...<rev2>
+ <rev>-<count>
+ git merges are ignored
-f, --file treat FILE as regular source file
--subjective, --strict enable more subjective tests
+ --list-types list the possible message types
--types TYPE(,TYPE2...) show only these comma separated message types
--ignore TYPE(,TYPE2...) ignore various comma separated message types
+ --show-types show the specific message type in the output
--max-line-length=n set the maximum line length, if exceeded, warn
--min-conf-desc-length=n set the min description length, if shorter, warn
- --show-types show the message "types" in the output
--root=PATH PATH to the kernel tree root
--no-summary suppress the per-file summary
--mailback only produce a report in case of warnings/errors
@@ -106,6 +120,37 @@
exit($exitcode);
}
+sub uniq {
+ my %seen;
+ return grep { !$seen{$_}++ } @_;
+}
+
+sub list_types {
+ my ($exitcode) = @_;
+
+ my $count = 0;
+
+ local $/ = undef;
+
+ open(my $script, '<', abs_path($P)) or
+ die "$P: Can't read '$P' $!\n";
+
+ my $text = <$script>;
+ close($script);
+
+ my @types = ();
+ for ($text =~ /\b(?:(?:CHK|WARN|ERROR)\s*\(\s*"([^"]+)")/g) {
+ push (@types, $_);
+ }
+ @types = sort(uniq(@types));
+ print("#\tMessage type\n\n");
+ foreach my $type (@types) {
+ print(++$count . "\t" . $type . "\n");
+ }
+
+ exit($exitcode);
+}
+
my $conf = which_conf($configuration_file);
if (-f $conf) {
my @conf_args;
@@ -141,11 +186,13 @@
'terse!' => \$terse,
'showfile!' => \$showfile,
'f|file!' => \$file,
+ 'g|git!' => \$git,
'subjective!' => \$check,
'strict!' => \$check,
'ignore=s' => \@ignore,
'types=s' => \@use,
'show-types!' => \$show_types,
+ 'list-types!' => \$list_types,
'max-line-length=i' => \$max_line_length,
'min-conf-desc-length=i' => \$min_conf_desc_length,
'root=s' => \$root,
@@ -166,6 +213,8 @@
help(0) if ($help);
+list_types(0) if ($list_types);
+
$fix = 1 if ($fix_inplace);
$check_orig = $check;
@@ -752,10 +801,42 @@
my @fixed_deleted = ();
my $fixlinenr = -1;
+# If input is git commits, extract all commits from the commit expressions.
+# For example, HEAD-3 means we need check 'HEAD, HEAD~1, HEAD~2'.
+die "$P: No git repository found\n" if ($git && !-e ".git");
+
+if ($git) {
+ my @commits = ();
+ foreach my $commit_expr (@ARGV) {
+ my $git_range;
+ if ($commit_expr =~ m/^(.*)-(\d+)$/) {
+ $git_range = "-$2 $1";
+ } elsif ($commit_expr =~ m/\.\./) {
+ $git_range = "$commit_expr";
+ } else {
+ $git_range = "-1 $commit_expr";
+ }
+ my $lines = `git log --no-color --no-merges --pretty=format:'%H %s' $git_range`;
+ foreach my $line (split(/\n/, $lines)) {
+ $line =~ /^([0-9a-fA-F]{40,40}) (.*)$/;
+ next if (!defined($1) || !defined($2));
+ my $sha1 = $1;
+ my $subject = $2;
+ unshift(@commits, $sha1);
+ $git_commits{$sha1} = $subject;
+ }
+ }
+ die "$P: no git commits after extraction!\n" if (@commits == 0);
+ @ARGV = @commits;
+}
+
my $vname;
for my $filename (@ARGV) {
my $FILE;
- if ($file) {
+ if ($git) {
+ open($FILE, '-|', "git format-patch -M --stdout -1 $filename") ||
+ die "$P: $filename: git format-patch failed - $!\n";
+ } elsif ($file) {
open($FILE, '-|', "diff -u /dev/null $filename") ||
die "$P: $filename: diff failed - $!\n";
} elsif ($filename eq '-') {
@@ -766,6 +847,8 @@
}
if ($filename eq '-') {
$vname = 'Your patch';
+ } elsif ($git) {
+ $vname = "Commit " . substr($filename, 0, 12) . ' ("' . $git_commits{$filename} . '")';
} else {
$vname = $filename;
}
@@ -2755,6 +2838,19 @@
"Logical continuations should be on the previous line\n" . $hereprev);
}
+# check indentation starts on a tab stop
+ if ($^V && $^V ge 5.10.0 &&
+ $sline =~ /^\+\t+( +)(?:$c90_Keywords\b|\{\s*$|\}\s*(?:else\b|while\b|\s*$))/) {
+ my $indent = length($1);
+ if ($indent % 8) {
+ if (WARN("TABSTOP",
+ "Statements should start on a tabstop\n" . $herecurr) &&
+ $fix) {
+ $fixed[$fixlinenr] =~ s@(^\+\t+) +@$1 . "\t" x ($indent/8)@e;
+ }
+ }
+ }
+
# check multi-line statement indentation matches previous line
if ($^V && $^V ge 5.10.0 &&
$prevline =~ /^\+([ \t]*)((?:$c90_Keywords(?:\s+if)\s*)|(?:$Declare\s*)?(?:$Ident|\(\s*\*\s*$Ident\s*\))\s*|$Ident\s*=\s*$Ident\s*)\(.*(\&\&|\|\||,)\s*$/) {
@@ -4291,7 +4387,7 @@
my $comp = $3;
my $to = $4;
my $newcomp = $comp;
- if ($lead !~ /$Operators\s*$/ &&
+ if ($lead !~ /(?:$Operators|\.)\s*$/ &&
$to !~ /^(?:Constant|[A-Z_][A-Z0-9_]*)$/ &&
WARN("CONSTANT_COMPARISON",
"Comparisons should place the constant on the right side of the test\n" . $herecurr) &&
@@ -5637,6 +5733,16 @@
}
}
+# check for #if defined CONFIG_<FOO> || defined CONFIG_<FOO>_MODULE
+ if ($line =~ /^\+\s*#\s*if\s+defined(?:\s*\(?\s*|\s+)(CONFIG_[A-Z_]+)\s*\)?\s*\|\|\s*defined(?:\s*\(?\s*|\s+)\1_MODULE\s*\)?\s*$/) {
+ my $config = $1;
+ if (WARN("PREFER_IS_ENABLED",
+ "Prefer IS_ENABLED(<FOO>) to CONFIG_<FOO> || CONFIG_<FOO>_MODULE\n" . $herecurr) &&
+ $fix) {
+ $fixed[$fixlinenr] = "\+#if IS_ENABLED($config)";
+ }
+ }
+
# check for case / default statements not preceded by break/fallthrough/switch
if ($line =~ /^.\s*(?:case\s+(?:$Ident|$Constant)\s*|default):/) {
my $has_break = 0;
@@ -5827,6 +5933,28 @@
}
}
+# whine about ACCESS_ONCE
+ if ($^V && $^V ge 5.10.0 &&
+ $line =~ /\bACCESS_ONCE\s*$balanced_parens\s*(=(?!=))?\s*($FuncArg)?/) {
+ my $par = $1;
+ my $eq = $2;
+ my $fun = $3;
+ $par =~ s/^\(\s*(.*)\s*\)$/$1/;
+ if (defined($eq)) {
+ if (WARN("PREFER_WRITE_ONCE",
+ "Prefer WRITE_ONCE(<FOO>, <BAR>) over ACCESS_ONCE(<FOO>) = <BAR>\n" . $herecurr) &&
+ $fix) {
+ $fixed[$fixlinenr] =~ s/\bACCESS_ONCE\s*\(\s*\Q$par\E\s*\)\s*$eq\s*\Q$fun\E/WRITE_ONCE($par, $fun)/;
+ }
+ } else {
+ if (WARN("PREFER_READ_ONCE",
+ "Prefer READ_ONCE(<FOO>) over ACCESS_ONCE(<FOO>)\n" . $herecurr) &&
+ $fix) {
+ $fixed[$fixlinenr] =~ s/\bACCESS_ONCE\s*\(\s*\Q$par\E\s*\)/READ_ONCE($par)/;
+ }
+ }
+ }
+
# check for lockdep_set_novalidate_class
if ($line =~ /^.\s*lockdep_set_novalidate_class\s*\(/ ||
$line =~ /__lockdep_no_validate__\s*\)/ ) {
@@ -5930,6 +6058,14 @@
}
if ($quiet == 0) {
+ # If there were any defects found and not already fixing them
+ if (!$clean and !$fix) {
+ print << "EOM"
+
+NOTE: For some of the reported defects, checkpatch may be able to
+ mechanically convert to the typical style using --fix or --fix-inplace.
+EOM
+ }
# If there were whitespace errors which cleanpatch can fix
# then suggest that.
if ($rpt_cleaners) {
diff --git a/scripts/coccinelle/api/debugfs/debugfs_simple_attr.cocci b/scripts/coccinelle/api/debugfs/debugfs_simple_attr.cocci
new file mode 100644
index 0000000..85cf540
--- /dev/null
+++ b/scripts/coccinelle/api/debugfs/debugfs_simple_attr.cocci
@@ -0,0 +1,67 @@
+/// Use DEFINE_DEBUGFS_ATTRIBUTE rather than DEFINE_SIMPLE_ATTRIBUTE
+/// for debugfs files.
+///
+//# Rationale: DEFINE_SIMPLE_ATTRIBUTE + debugfs_create_file()
+//# imposes some significant overhead as compared to
+//# DEFINE_DEBUGFS_ATTRIBUTE + debugfs_create_file_unsafe().
+//
+// Copyright (C): 2016 Nicolai Stange
+// Options: --no-includes
+//
+
+virtual context
+virtual patch
+virtual org
+virtual report
+
+@dsa@
+declarer name DEFINE_SIMPLE_ATTRIBUTE;
+identifier dsa_fops;
+expression dsa_get, dsa_set, dsa_fmt;
+position p;
+@@
+DEFINE_SIMPLE_ATTRIBUTE@p(dsa_fops, dsa_get, dsa_set, dsa_fmt);
+
+@dcf@
+expression name, mode, parent, data;
+identifier dsa.dsa_fops;
+@@
+debugfs_create_file(name, mode, parent, data, &dsa_fops)
+
+
+@context_dsa depends on context && dcf@
+declarer name DEFINE_DEBUGFS_ATTRIBUTE;
+identifier dsa.dsa_fops;
+expression dsa.dsa_get, dsa.dsa_set, dsa.dsa_fmt;
+@@
+* DEFINE_SIMPLE_ATTRIBUTE(dsa_fops, dsa_get, dsa_set, dsa_fmt);
+
+
+@patch_dcf depends on patch expression@
+expression name, mode, parent, data;
+identifier dsa.dsa_fops;
+@@
+- debugfs_create_file(name, mode, parent, data, &dsa_fops)
++ debugfs_create_file_unsafe(name, mode, parent, data, &dsa_fops)
+
+@patch_dsa depends on patch_dcf && patch@
+identifier dsa.dsa_fops;
+expression dsa.dsa_get, dsa.dsa_set, dsa.dsa_fmt;
+@@
+- DEFINE_SIMPLE_ATTRIBUTE(dsa_fops, dsa_get, dsa_set, dsa_fmt);
++ DEFINE_DEBUGFS_ATTRIBUTE(dsa_fops, dsa_get, dsa_set, dsa_fmt);
+
+
+@script:python depends on org && dcf@
+fops << dsa.dsa_fops;
+p << dsa.p;
+@@
+msg="%s should be defined with DEFINE_DEBUGFS_ATTRIBUTE" % (fops)
+coccilib.org.print_todo(p[0], msg)
+
+@script:python depends on report && dcf@
+fops << dsa.dsa_fops;
+p << dsa.p;
+@@
+msg="WARNING: %s should be defined with DEFINE_DEBUGFS_ATTRIBUTE" % (fops)
+coccilib.report.print_report(p[0], msg)
diff --git a/scripts/decode_stacktrace.sh b/scripts/decode_stacktrace.sh
index 00d6d53c..c332684 100755
--- a/scripts/decode_stacktrace.sh
+++ b/scripts/decode_stacktrace.sh
@@ -2,15 +2,17 @@
# (c) 2014, Sasha Levin <sasha.levin@oracle.com>
#set -x
-if [[ $# != 2 ]]; then
+if [[ $# < 2 ]]; then
echo "Usage:"
- echo " $0 [vmlinux] [base path]"
+ echo " $0 [vmlinux] [base path] [modules path]"
exit 1
fi
vmlinux=$1
basepath=$2
+modpath=$3
declare -A cache
+declare -A modcache
parse_symbol() {
# The structure of symbol at this point is:
@@ -19,6 +21,17 @@
# For example:
# do_basic_setup+0x9c/0xbf
+ if [[ $module == "" ]] ; then
+ local objfile=$vmlinux
+ elif [[ "${modcache[$module]+isset}" == "isset" ]]; then
+ local objfile=${modcache[$module]}
+ else
+ [[ $modpath == "" ]] && return
+ local objfile=$(find "$modpath" -name $module.ko -print -quit)
+ [[ $objfile == "" ]] && return
+ modcache[$module]=$objfile
+ fi
+
# Remove the englobing parenthesis
symbol=${symbol#\(}
symbol=${symbol%\)}
@@ -29,11 +42,11 @@
# Use 'nm vmlinux' to figure out the base address of said symbol.
# It's actually faster to call it every time than to load it
# all into bash.
- if [[ "${cache[$name]+isset}" == "isset" ]]; then
- local base_addr=${cache[$name]}
+ if [[ "${cache[$module,$name]+isset}" == "isset" ]]; then
+ local base_addr=${cache[$module,$name]}
else
- local base_addr=$(nm "$vmlinux" | grep -i ' t ' | awk "/ $name\$/ {print \$1}" | head -n1)
- cache["$name"]="$base_addr"
+ local base_addr=$(nm "$objfile" | grep -i ' t ' | awk "/ $name\$/ {print \$1}" | head -n1)
+ cache[$module,$name]="$base_addr"
fi
# Let's start doing the math to get the exact address into the
# symbol. First, strip out the symbol total length.
@@ -48,12 +61,12 @@
local address=$(printf "%x\n" "$expr")
# Pass it to addr2line to get filename and line number
- # Could get more than one result
- if [[ "${cache[$address]+isset}" == "isset" ]]; then
- local code=${cache[$address]}
+ # Could get more than one result
+ if [[ "${cache[$module,$address]+isset}" == "isset" ]]; then
+ local code=${cache[$module,$address]}
else
- local code=$(addr2line -i -e "$vmlinux" "$address")
- cache[$address]=$code
+ local code=$(addr2line -i -e "$objfile" "$address")
+ cache[$module,$address]=$code
fi
# addr2line doesn't return a proper error code if it fails, so
@@ -105,13 +118,23 @@
fi
done
- # The symbol is the last element, process it
- symbol=${words[$last]}
+ if [[ ${words[$last]} =~ \[([^]]+)\] ]]; then
+ module=${words[$last]}
+ module=${module#\[}
+ module=${module%\]}
+ symbol=${words[$last-1]}
+ unset words[$last-1]
+ else
+ # The symbol is the last element, process it
+ symbol=${words[$last]}
+ module=
+ fi
+
unset words[$last]
parse_symbol # modifies $symbol
# Add up the line number to the symbol
- echo "${words[@]}" "$symbol"
+ echo "${words[@]}" "$symbol $module"
}
while read line; do
@@ -121,8 +144,8 @@
handle_line "$line"
# Is it a code line?
elif [[ $line == *Code:* ]]; then
- decode_code "$line"
- else
+ decode_code "$line"
+ else
# Nothing special in this line, show it as is
echo "$line"
fi
diff --git a/scripts/docproc.c b/scripts/docproc.c
index e267e621..0a12593b 100644
--- a/scripts/docproc.c
+++ b/scripts/docproc.c
@@ -42,8 +42,10 @@
#include <unistd.h>
#include <limits.h>
#include <errno.h>
+#include <getopt.h>
#include <sys/types.h>
#include <sys/wait.h>
+#include <time.h>
/* exitstatus is used to keep track of any failing calls to kernel-doc,
* but execution continues. */
@@ -68,12 +70,23 @@
#define KERNELDOCPATH "scripts/"
#define KERNELDOC "kernel-doc"
#define DOCBOOK "-docbook"
+#define RST "-rst"
#define LIST "-list"
#define FUNCTION "-function"
#define NOFUNCTION "-nofunction"
#define NODOCSECTIONS "-no-doc-sections"
#define SHOWNOTFOUND "-show-not-found"
+enum file_format {
+ FORMAT_AUTO,
+ FORMAT_DOCBOOK,
+ FORMAT_RST,
+};
+
+static enum file_format file_format = FORMAT_AUTO;
+
+#define KERNELDOC_FORMAT (file_format == FORMAT_RST ? RST : DOCBOOK)
+
static char *srctree, *kernsrctree;
static char **all_list = NULL;
@@ -95,7 +108,7 @@
static void usage (void)
{
- fprintf(stderr, "Usage: docproc {doc|depend} file\n");
+ fprintf(stderr, "Usage: docproc [{--docbook|--rst}] {doc|depend} file\n");
fprintf(stderr, "Input is read from file.tmpl. Output is sent to stdout\n");
fprintf(stderr, "doc: frontend when generating kernel documentation\n");
fprintf(stderr, "depend: generate list of files referenced within file\n");
@@ -242,7 +255,7 @@
/*
* Document all external or internal functions in a file.
* Call kernel-doc with following parameters:
- * kernel-doc -docbook -nofunction function_name1 filename
+ * kernel-doc [-docbook|-rst] -nofunction function_name1 filename
* Function names are obtained from all the src files
* by find_export_symbols.
* intfunc uses -nofunction
@@ -263,7 +276,7 @@
exit(1);
}
vec[idx++] = KERNELDOC;
- vec[idx++] = DOCBOOK;
+ vec[idx++] = KERNELDOC_FORMAT;
vec[idx++] = NODOCSECTIONS;
for (i=0; i < symfilecnt; i++) {
struct symfile * sym = &symfilelist[i];
@@ -275,7 +288,10 @@
}
vec[idx++] = filename;
vec[idx] = NULL;
- printf("<!-- %s -->\n", filename);
+ if (file_format == FORMAT_RST)
+ printf(".. %s\n", filename);
+ else
+ printf("<!-- %s -->\n", filename);
exec_kernel_doc(vec);
fflush(stdout);
free(vec);
@@ -294,7 +310,7 @@
int i, idx = 0;
int startofsym = 1;
vec[idx++] = KERNELDOC;
- vec[idx++] = DOCBOOK;
+ vec[idx++] = KERNELDOC_FORMAT;
vec[idx++] = SHOWNOTFOUND;
/* Split line up in individual parameters preceded by FUNCTION */
@@ -343,7 +359,7 @@
free(s);
vec[0] = KERNELDOC;
- vec[1] = DOCBOOK;
+ vec[1] = KERNELDOC_FORMAT;
vec[2] = SHOWNOTFOUND;
vec[3] = FUNCTION;
vec[4] = line;
@@ -431,6 +447,32 @@
}
/*
+ * Terminate s at first space, if any. If there was a space, return pointer to
+ * the character after that. Otherwise, return pointer to the terminating NUL.
+ */
+static char *chomp(char *s)
+{
+ while (*s && !isspace(*s))
+ s++;
+
+ if (*s)
+ *s++ = '\0';
+
+ return s;
+}
+
+/* Return pointer to directive content, or NULL if not a directive. */
+static char *is_directive(char *line)
+{
+ if (file_format == FORMAT_DOCBOOK && line[0] == '!')
+ return line + 1;
+ else if (file_format == FORMAT_RST && !strncmp(line, ".. !", 4))
+ return line + 4;
+
+ return NULL;
+}
+
+/*
* Parse file, calling action specific functions for:
* 1) Lines containing !E
* 2) Lines containing !I
@@ -443,63 +485,75 @@
static void parse_file(FILE *infile)
{
char line[MAXLINESZ];
- char * s;
+ char *p, *s;
while (fgets(line, MAXLINESZ, infile)) {
- if (line[0] == '!') {
- s = line + 2;
- switch (line[1]) {
- case 'E':
- while (*s && !isspace(*s)) s++;
- *s = '\0';
- externalfunctions(line+2);
- break;
- case 'I':
- while (*s && !isspace(*s)) s++;
- *s = '\0';
- internalfunctions(line+2);
- break;
- case 'D':
- while (*s && !isspace(*s)) s++;
- *s = '\0';
- symbolsonly(line+2);
- break;
- case 'F':
- /* filename */
- while (*s && !isspace(*s)) s++;
- *s++ = '\0';
- /* function names */
- while (isspace(*s))
- s++;
- singlefunctions(line +2, s);
- break;
- case 'P':
- /* filename */
- while (*s && !isspace(*s)) s++;
- *s++ = '\0';
- /* DOC: section name */
- while (isspace(*s))
- s++;
- docsection(line + 2, s);
- break;
- case 'C':
- while (*s && !isspace(*s)) s++;
- *s = '\0';
- if (findall)
- findall(line+2);
- break;
- default:
- defaultline(line);
- }
- } else {
+ p = is_directive(line);
+ if (!p) {
+ defaultline(line);
+ continue;
+ }
+
+ switch (*p++) {
+ case 'E':
+ chomp(p);
+ externalfunctions(p);
+ break;
+ case 'I':
+ chomp(p);
+ internalfunctions(p);
+ break;
+ case 'D':
+ chomp(p);
+ symbolsonly(p);
+ break;
+ case 'F':
+ /* filename */
+ s = chomp(p);
+ /* function names */
+ while (isspace(*s))
+ s++;
+ singlefunctions(p, s);
+ break;
+ case 'P':
+ /* filename */
+ s = chomp(p);
+ /* DOC: section name */
+ while (isspace(*s))
+ s++;
+ docsection(p, s);
+ break;
+ case 'C':
+ chomp(p);
+ if (findall)
+ findall(p);
+ break;
+ default:
defaultline(line);
}
}
fflush(stdout);
}
+/*
+ * Is this a RestructuredText template? Answer the question by seeing if its
+ * name ends in ".rst".
+ */
+static int is_rst(const char *file)
+{
+ char *dot = strrchr(file, '.');
+
+ return dot && !strcmp(dot + 1, "rst");
+}
+
+enum opts {
+ OPT_DOCBOOK,
+ OPT_RST,
+ OPT_HELP,
+};
int main(int argc, char *argv[])
{
+ const char *subcommand, *filename;
FILE * infile;
int i;
@@ -509,19 +563,66 @@
kernsrctree = getenv("KBUILD_SRC");
if (!kernsrctree || !*kernsrctree)
kernsrctree = srctree;
- if (argc != 3) {
+
+ for (;;) {
+ int c;
+ struct option opts[] = {
+ { "docbook", no_argument, NULL, OPT_DOCBOOK },
+ { "rst", no_argument, NULL, OPT_RST },
+ { "help", no_argument, NULL, OPT_HELP },
+ {}
+ };
+
+ c = getopt_long_only(argc, argv, "", opts, NULL);
+ if (c == -1)
+ break;
+
+ switch (c) {
+ case OPT_DOCBOOK:
+ file_format = FORMAT_DOCBOOK;
+ break;
+ case OPT_RST:
+ file_format = FORMAT_RST;
+ break;
+ case OPT_HELP:
+ usage();
+ return 0;
+ default:
+ case '?':
+ usage();
+ return 1;
+ }
+ }
+
+ argc -= optind;
+ argv += optind;
+
+ if (argc != 2) {
usage();
exit(1);
}
+
+ subcommand = argv[0];
+ filename = argv[1];
+
+ if (file_format == FORMAT_AUTO)
+ file_format = is_rst(filename) ? FORMAT_RST : FORMAT_DOCBOOK;
+
/* Open file, exit on error */
- infile = fopen(argv[2], "r");
+ infile = fopen(filename, "r");
if (infile == NULL) {
fprintf(stderr, "docproc: ");
- perror(argv[2]);
+ perror(filename);
exit(2);
}
- if (strcmp("doc", argv[1]) == 0) {
+ if (strcmp("doc", subcommand) == 0) {
+ if (file_format == FORMAT_RST) {
+ time_t t = time(NULL);
+ printf(".. generated from %s by docproc %s\n",
+ filename, ctime(&t));
+ }
+
/* Need to do this in two passes.
* First pass is used to collect all symbols exported
* in the various files;
@@ -557,10 +658,10 @@
fprintf(stderr, "Warning: didn't use docs for %s\n",
all_list[i]);
}
- } else if (strcmp("depend", argv[1]) == 0) {
+ } else if (strcmp("depend", subcommand) == 0) {
/* Create first part of dependency chain
* file.tmpl */
- printf("%s\t", argv[2]);
+ printf("%s\t", filename);
defaultline = noaction;
internalfunctions = adddep;
externalfunctions = adddep;
@@ -571,7 +672,7 @@
parse_file(infile);
printf("\n");
} else {
- fprintf(stderr, "Unknown option: %s\n", argv[1]);
+ fprintf(stderr, "Unknown option: %s\n", subcommand);
exit(1);
}
fclose(infile);
diff --git a/scripts/dtc/checks.c b/scripts/dtc/checks.c
index 0c03ac9..386f956 100644
--- a/scripts/dtc/checks.c
+++ b/scripts/dtc/checks.c
@@ -294,6 +294,30 @@
}
NODE_ERROR(node_name_format, NULL, &node_name_chars);
+static void check_unit_address_vs_reg(struct check *c, struct node *dt,
+ struct node *node)
+{
+ const char *unitname = get_unitname(node);
+ struct property *prop = get_property(node, "reg");
+
+ if (!prop) {
+ prop = get_property(node, "ranges");
+ if (prop && !prop->val.len)
+ prop = NULL;
+ }
+
+ if (prop) {
+ if (!unitname[0])
+ FAIL(c, "Node %s has a reg or ranges property, but no unit name",
+ node->fullpath);
+ } else {
+ if (unitname[0])
+ FAIL(c, "Node %s has a unit name, but no reg property",
+ node->fullpath);
+ }
+}
+NODE_WARNING(unit_address_vs_reg, NULL);
+
static void check_property_name_chars(struct check *c, struct node *dt,
struct node *node, struct property *prop)
{
@@ -667,6 +691,8 @@
&addr_size_cells, ®_format, &ranges_format,
+ &unit_address_vs_reg,
+
&avoid_default_addr_size,
&obsolete_chosen_interrupt_controller,
diff --git a/scripts/dtc/flattree.c b/scripts/dtc/flattree.c
index bd99fa2..ec14954 100644
--- a/scripts/dtc/flattree.c
+++ b/scripts/dtc/flattree.c
@@ -889,7 +889,7 @@
if (version >= 3) {
uint32_t size_str = fdt32_to_cpu(fdt->size_dt_strings);
- if (off_str+size_str > totalsize)
+ if ((off_str+size_str < off_str) || (off_str+size_str > totalsize))
die("String table extends past total size\n");
inbuf_init(&strbuf, blob + off_str, blob + off_str + size_str);
} else {
@@ -898,7 +898,7 @@
if (version >= 17) {
size_dt = fdt32_to_cpu(fdt->size_dt_struct);
- if (off_dt+size_dt > totalsize)
+ if ((off_dt+size_dt < off_dt) || (off_dt+size_dt > totalsize))
die("Structure block extends past total size\n");
}
diff --git a/scripts/dtc/libfdt/fdt_ro.c b/scripts/dtc/libfdt/fdt_ro.c
index e5b3136..50cce86 100644
--- a/scripts/dtc/libfdt/fdt_ro.c
+++ b/scripts/dtc/libfdt/fdt_ro.c
@@ -647,10 +647,8 @@
prop = fdt_getprop(fdt, nodeoffset, "compatible", &len);
if (!prop)
return len;
- if (fdt_stringlist_contains(prop, len, compatible))
- return 0;
- else
- return 1;
+
+ return !fdt_stringlist_contains(prop, len, compatible);
}
int fdt_node_offset_by_compatible(const void *fdt, int startoffset,
diff --git a/scripts/dtc/version_gen.h b/scripts/dtc/version_gen.h
index 11d93e6..ad9b05a 100644
--- a/scripts/dtc/version_gen.h
+++ b/scripts/dtc/version_gen.h
@@ -1 +1 @@
-#define DTC_VERSION "DTC 1.4.1-gb06e55c8"
+#define DTC_VERSION "DTC 1.4.1-g53bf130b"
diff --git a/scripts/gdb/linux/Makefile b/scripts/gdb/linux/Makefile
index 6cf1ecf..cd129e6 100644
--- a/scripts/gdb/linux/Makefile
+++ b/scripts/gdb/linux/Makefile
@@ -8,4 +8,14 @@
endif
@:
-clean-files := *.pyc *.pyo $(if $(KBUILD_SRC),*.py)
+quiet_cmd_gen_constants_py = GEN $@
+ cmd_gen_constants_py = \
+ $(CPP) -E -x c -P $(c_flags) $< > $@ ;\
+ sed -i '1,/<!-- end-c-headers -->/d;' $@
+
+$(obj)/constants.py: $(SRCTREE)/$(obj)/constants.py.in
+ $(call if_changed,gen_constants_py)
+
+build_constants_py: $(obj)/constants.py
+
+clean-files := *.pyc *.pyo $(if $(KBUILD_SRC),*.py) $(obj)/constants.py
diff --git a/scripts/gdb/linux/constants.py.in b/scripts/gdb/linux/constants.py.in
new file mode 100644
index 0000000..07e6c2b
--- /dev/null
+++ b/scripts/gdb/linux/constants.py.in
@@ -0,0 +1,59 @@
+/*
+ * gdb helper commands and functions for Linux kernel debugging
+ *
+ * Kernel constants derived from include files.
+ *
+ * Copyright (c) 2016 Linaro Ltd
+ *
+ * Authors:
+ * Kieran Bingham <kieran.bingham@linaro.org>
+ *
+ * This work is licensed under the terms of the GNU GPL version 2.
+ *
+ */
+
+#include <linux/fs.h>
+#include <linux/mount.h>
+#include <linux/radix-tree.h>
+
+/* We need to stringify expanded macros so that they can be parsed */
+
+#define STRING(x) #x
+#define XSTRING(x) STRING(x)
+
+#define LX_VALUE(x) LX_##x = x
+#define LX_GDBPARSED(x) LX_##x = gdb.parse_and_eval(XSTRING(x))
+
+/*
+ * IS_ENABLED generates (a || b) which is not compatible with python
+ * We can only switch on configuration items we know are available
+ * Therefore - IS_BUILTIN() is more appropriate
+ */
+#define LX_CONFIG(x) LX_##x = IS_BUILTIN(x)
+
+/* The build system will take care of deleting everything above this marker */
+<!-- end-c-headers -->
+
+import gdb
+
+/* linux/fs.h */
+LX_VALUE(MS_RDONLY)
+LX_VALUE(MS_SYNCHRONOUS)
+LX_VALUE(MS_MANDLOCK)
+LX_VALUE(MS_DIRSYNC)
+LX_VALUE(MS_NOATIME)
+LX_VALUE(MS_NODIRATIME)
+
+/* linux/mount.h */
+LX_VALUE(MNT_NOSUID)
+LX_VALUE(MNT_NODEV)
+LX_VALUE(MNT_NOEXEC)
+LX_VALUE(MNT_NOATIME)
+LX_VALUE(MNT_NODIRATIME)
+LX_VALUE(MNT_RELATIME)
+
+/* linux/radix-tree.h */
+LX_VALUE(RADIX_TREE_INDIRECT_PTR)
+LX_GDBPARSED(RADIX_TREE_HEIGHT_MASK)
+LX_GDBPARSED(RADIX_TREE_MAP_SHIFT)
+LX_GDBPARSED(RADIX_TREE_MAP_MASK)
diff --git a/scripts/gdb/linux/cpus.py b/scripts/gdb/linux/cpus.py
index 4297b83..ca11e8d 100644
--- a/scripts/gdb/linux/cpus.py
+++ b/scripts/gdb/linux/cpus.py
@@ -97,9 +97,47 @@
bits >>= 1
bit += 1
+ yield int(cpu)
+
+
+def each_online_cpu():
+ for cpu in cpu_list("__cpu_online_mask"):
yield cpu
+def each_present_cpu():
+ for cpu in cpu_list("__cpu_present_mask"):
+ yield cpu
+
+
+def each_possible_cpu():
+ for cpu in cpu_list("__cpu_possible_mask"):
+ yield cpu
+
+
+def each_active_cpu():
+ for cpu in cpu_list("__cpu_active_mask"):
+ yield cpu
+
+
+class LxCpus(gdb.Command):
+ """List CPU status arrays
+
+Displays the known state of each CPU based on the kernel masks
+and can help identify the state of hotplugged CPUs"""
+
+ def __init__(self):
+ super(LxCpus, self).__init__("lx-cpus", gdb.COMMAND_DATA)
+
+ def invoke(self, arg, from_tty):
+ gdb.write("Possible CPUs : {}\n".format(list(each_possible_cpu())))
+ gdb.write("Present CPUs : {}\n".format(list(each_present_cpu())))
+ gdb.write("Online CPUs : {}\n".format(list(each_online_cpu())))
+ gdb.write("Active CPUs : {}\n".format(list(each_active_cpu())))
+
+LxCpus()
+
+
class PerCpu(gdb.Function):
"""Return per-cpu variable.
diff --git a/scripts/gdb/linux/dmesg.py b/scripts/gdb/linux/dmesg.py
index 927d0d2..f9b92ec 100644
--- a/scripts/gdb/linux/dmesg.py
+++ b/scripts/gdb/linux/dmesg.py
@@ -33,11 +33,12 @@
if log_first_idx < log_next_idx:
log_buf_2nd_half = -1
length = log_next_idx - log_first_idx
- log_buf = inf.read_memory(start, length)
+ log_buf = utils.read_memoryview(inf, start, length).tobytes()
else:
log_buf_2nd_half = log_buf_len - log_first_idx
- log_buf = inf.read_memory(start, log_buf_2nd_half) + \
- inf.read_memory(log_buf_addr, log_next_idx)
+ a = utils.read_memoryview(inf, start, log_buf_2nd_half)
+ b = utils.read_memoryview(inf, log_buf_addr, log_next_idx)
+ log_buf = a.tobytes() + b.tobytes()
pos = 0
while pos < log_buf.__len__():
@@ -50,10 +51,10 @@
continue
text_len = utils.read_u16(log_buf[pos + 10:pos + 12])
- text = log_buf[pos + 16:pos + 16 + text_len]
+ text = log_buf[pos + 16:pos + 16 + text_len].decode()
time_stamp = utils.read_u64(log_buf[pos:pos + 8])
- for line in memoryview(text).tobytes().splitlines():
+ for line in text.splitlines():
gdb.write("[{time:12.6f}] {line}\n".format(
time=time_stamp / 1000000000.0,
line=line))
diff --git a/scripts/gdb/linux/lists.py b/scripts/gdb/linux/lists.py
index 3a3775b..2f335fb 100644
--- a/scripts/gdb/linux/lists.py
+++ b/scripts/gdb/linux/lists.py
@@ -18,6 +18,27 @@
list_head = utils.CachedType("struct list_head")
+def list_for_each(head):
+ if head.type == list_head.get_type().pointer():
+ head = head.dereference()
+ elif head.type != list_head.get_type():
+ raise gdb.GdbError("Must be struct list_head not {}"
+ .format(head.type))
+
+ node = head['next'].dereference()
+ while node.address != head.address:
+ yield node.address
+ node = node['next'].dereference()
+
+
+def list_for_each_entry(head, gdbtype, member):
+ for node in list_for_each(head):
+ if node.type != list_head.get_type().pointer():
+ raise TypeError("Type {} found. Expected struct list_head *."
+ .format(node.type))
+ yield utils.container_of(node, gdbtype, member)
+
+
def list_check(head):
nb = 0
if (head.type == list_head.get_type().pointer()):
diff --git a/scripts/gdb/linux/modules.py b/scripts/gdb/linux/modules.py
index 0a35d6d..441b2323 100644
--- a/scripts/gdb/linux/modules.py
+++ b/scripts/gdb/linux/modules.py
@@ -13,7 +13,7 @@
import gdb
-from linux import cpus, utils
+from linux import cpus, utils, lists
module_type = utils.CachedType("struct module")
@@ -21,14 +21,14 @@
def module_list():
global module_type
- module_ptr_type = module_type.get_type().pointer()
- modules = gdb.parse_and_eval("modules")
- entry = modules['next']
- end_of_list = modules.address
+ modules = utils.gdb_eval_or_none("modules")
+ if modules is None:
+ return
- while entry != end_of_list:
- yield utils.container_of(entry, module_ptr_type, "list")
- entry = entry['next']
+ module_ptr_type = module_type.get_type().pointer()
+
+ for module in lists.list_for_each_entry(modules, module_ptr_type, "list"):
+ yield module
def find_module_by_name(name):
@@ -78,19 +78,17 @@
address=str(layout['base']).split()[0],
name=module['name'].string(),
size=str(layout['size']),
- ref=str(module['refcnt']['counter'])))
+ ref=str(module['refcnt']['counter'] - 1)))
- source_list = module['source_list']
t = self._module_use_type.get_type().pointer()
- entry = source_list['next']
first = True
- while entry != source_list.address:
- use = utils.container_of(entry, t, "source_list")
+ sources = module['source_list']
+ for use in lists.list_for_each_entry(sources, t, "source_list"):
gdb.write("{separator}{name}".format(
separator=" " if first else ",",
name=use['source']['name'].string()))
first = False
- entry = entry['next']
+
gdb.write("\n")
diff --git a/scripts/gdb/linux/proc.py b/scripts/gdb/linux/proc.py
index 6e6709c1..38b1f09 100644
--- a/scripts/gdb/linux/proc.py
+++ b/scripts/gdb/linux/proc.py
@@ -12,6 +12,10 @@
#
import gdb
+from linux import constants
+from linux import utils
+from linux import tasks
+from linux import lists
class LxCmdLine(gdb.Command):
@@ -39,3 +43,155 @@
gdb.write(gdb.parse_and_eval("linux_banner").string())
LxVersion()
+
+
+# Resource Structure Printers
+# /proc/iomem
+# /proc/ioports
+
+def get_resources(resource, depth):
+ while resource:
+ yield resource, depth
+
+ child = resource['child']
+ if child:
+ for res, deep in get_resources(child, depth + 1):
+ yield res, deep
+
+ resource = resource['sibling']
+
+
+def show_lx_resources(resource_str):
+ resource = gdb.parse_and_eval(resource_str)
+ width = 4 if resource['end'] < 0x10000 else 8
+ # Iterate straight to the first child
+ for res, depth in get_resources(resource['child'], 0):
+ start = int(res['start'])
+ end = int(res['end'])
+ gdb.write(" " * depth * 2 +
+ "{0:0{1}x}-".format(start, width) +
+ "{0:0{1}x} : ".format(end, width) +
+ res['name'].string() + "\n")
+
+
+class LxIOMem(gdb.Command):
+ """Identify the IO memory resource locations defined by the kernel
+
+Equivalent to cat /proc/iomem on a running target"""
+
+ def __init__(self):
+ super(LxIOMem, self).__init__("lx-iomem", gdb.COMMAND_DATA)
+
+ def invoke(self, arg, from_tty):
+ return show_lx_resources("iomem_resource")
+
+LxIOMem()
+
+
+class LxIOPorts(gdb.Command):
+ """Identify the IO port resource locations defined by the kernel
+
+Equivalent to cat /proc/ioports on a running target"""
+
+ def __init__(self):
+ super(LxIOPorts, self).__init__("lx-ioports", gdb.COMMAND_DATA)
+
+ def invoke(self, arg, from_tty):
+ return show_lx_resources("ioport_resource")
+
+LxIOPorts()
+
+
+# Mount namespace viewer
+# /proc/mounts
+
+def info_opts(lst, opt):
+ opts = ""
+ for key, string in lst.items():
+ if opt & key:
+ opts += string
+ return opts
+
+
+FS_INFO = {constants.LX_MS_SYNCHRONOUS: ",sync",
+ constants.LX_MS_MANDLOCK: ",mand",
+ constants.LX_MS_DIRSYNC: ",dirsync",
+ constants.LX_MS_NOATIME: ",noatime",
+ constants.LX_MS_NODIRATIME: ",nodiratime"}
+
+MNT_INFO = {constants.LX_MNT_NOSUID: ",nosuid",
+ constants.LX_MNT_NODEV: ",nodev",
+ constants.LX_MNT_NOEXEC: ",noexec",
+ constants.LX_MNT_NOATIME: ",noatime",
+ constants.LX_MNT_NODIRATIME: ",nodiratime",
+ constants.LX_MNT_RELATIME: ",relatime"}
+
+mount_type = utils.CachedType("struct mount")
+mount_ptr_type = mount_type.get_type().pointer()
+
+
+class LxMounts(gdb.Command):
+ """Report the VFS mounts of the current process namespace.
+
+Equivalent to cat /proc/mounts on a running target
+An integer value can be supplied to display the mount
+values of that process namespace"""
+
+ def __init__(self):
+ super(LxMounts, self).__init__("lx-mounts", gdb.COMMAND_DATA)
+
+ # Equivalent to proc_namespace.c:show_vfsmnt
+ # However, that has the ability to call into s_op functions
+ # whereas we cannot and must make do with the information we can obtain.
+ def invoke(self, arg, from_tty):
+ argv = gdb.string_to_argv(arg)
+ if len(argv) >= 1:
+ try:
+ pid = int(argv[0])
+ except:
+ raise gdb.GdbError("Provide a PID as integer value")
+ else:
+ pid = 1
+
+ task = tasks.get_task_by_pid(pid)
+ if not task:
+ raise gdb.GdbError("Couldn't find a process with PID {}"
+ .format(pid))
+
+ namespace = task['nsproxy']['mnt_ns']
+ if not namespace:
+ raise gdb.GdbError("No namespace for current process")
+
+ for vfs in lists.list_for_each_entry(namespace['list'],
+ mount_ptr_type, "mnt_list"):
+ devname = vfs['mnt_devname'].string()
+ devname = devname if devname else "none"
+
+ pathname = ""
+ parent = vfs
+ while True:
+ mntpoint = parent['mnt_mountpoint']
+ pathname = utils.dentry_name(mntpoint) + pathname
+ if (parent == parent['mnt_parent']):
+ break
+ parent = parent['mnt_parent']
+
+ if (pathname == ""):
+ pathname = "/"
+
+ superblock = vfs['mnt']['mnt_sb']
+ fstype = superblock['s_type']['name'].string()
+ s_flags = int(superblock['s_flags'])
+ m_flags = int(vfs['mnt']['mnt_flags'])
+ rd = "ro" if (s_flags & constants.LX_MS_RDONLY) else "rw"
+
+ gdb.write(
+ "{} {} {} {}{}{} 0 0\n"
+ .format(devname,
+ pathname,
+ fstype,
+ rd,
+ info_opts(FS_INFO, s_flags),
+ info_opts(MNT_INFO, m_flags)))
+
+LxMounts()
diff --git a/scripts/gdb/linux/radixtree.py b/scripts/gdb/linux/radixtree.py
new file mode 100644
index 0000000..0fdef4e
--- /dev/null
+++ b/scripts/gdb/linux/radixtree.py
@@ -0,0 +1,97 @@
+#
+# gdb helper commands and functions for Linux kernel debugging
+#
+# Radix Tree Parser
+#
+# Copyright (c) 2016 Linaro Ltd
+#
+# Authors:
+# Kieran Bingham <kieran.bingham@linaro.org>
+#
+# This work is licensed under the terms of the GNU GPL version 2.
+#
+
+import gdb
+
+from linux import utils
+from linux import constants
+
+radix_tree_root_type = utils.CachedType("struct radix_tree_root")
+radix_tree_node_type = utils.CachedType("struct radix_tree_node")
+
+
+def is_indirect_ptr(node):
+ long_type = utils.get_long_type()
+ return (node.cast(long_type) & constants.LX_RADIX_TREE_INDIRECT_PTR)
+
+
+def indirect_to_ptr(node):
+ long_type = utils.get_long_type()
+ node_type = node.type
+ indirect_ptr = node.cast(long_type) & ~constants.LX_RADIX_TREE_INDIRECT_PTR
+ return indirect_ptr.cast(node_type)
+
+
+def maxindex(height):
+ height = height & constants.LX_RADIX_TREE_HEIGHT_MASK
+ return gdb.parse_and_eval("height_to_maxindex["+str(height)+"]")
+
+
+def lookup(root, index):
+ if root.type == radix_tree_root_type.get_type().pointer():
+ root = root.dereference()
+ elif root.type != radix_tree_root_type.get_type():
+ raise gdb.GdbError("Must be struct radix_tree_root not {}"
+ .format(root.type))
+
+ node = root['rnode']
+ if node is 0:
+ return None
+
+ if not (is_indirect_ptr(node)):
+ if (index > 0):
+ return None
+ return node
+
+ node = indirect_to_ptr(node)
+
+ height = node['path'] & constants.LX_RADIX_TREE_HEIGHT_MASK
+ if (index > maxindex(height)):
+ return None
+
+ shift = (height-1) * constants.LX_RADIX_TREE_MAP_SHIFT
+
+ while True:
+ new_index = (index >> shift) & constants.LX_RADIX_TREE_MAP_MASK
+ slot = node['slots'][new_index]
+
+ node = slot.cast(node.type.pointer()).dereference()
+ if node is 0:
+ return None
+
+ shift -= constants.LX_RADIX_TREE_MAP_SHIFT
+ height -= 1
+
+ if (height <= 0):
+ break
+
+ return node
+
+
+class LxRadixTree(gdb.Function):
+ """ Lookup and return a node from a RadixTree.
+
+$lx_radix_tree_lookup(root_node [, index]): Return the node at the given index.
+If index is omitted, the root node is dereferenced and returned."""
+
+ def __init__(self):
+ super(LxRadixTree, self).__init__("lx_radix_tree_lookup")
+
+ def invoke(self, root, index=0):
+ result = lookup(root, index)
+ if result is None:
+ raise gdb.GdbError("No entry in tree at index {}".format(index))
+
+ return result
+
+LxRadixTree()
diff --git a/scripts/gdb/linux/tasks.py b/scripts/gdb/linux/tasks.py
index 862a4ae..1bf949c 100644
--- a/scripts/gdb/linux/tasks.py
+++ b/scripts/gdb/linux/tasks.py
@@ -114,3 +114,22 @@
LxThreadInfoFunc()
+
+
+class LxThreadInfoByPidFunc (gdb.Function):
+ """Calculate Linux thread_info from task variable found by pid
+
+$lx_thread_info_by_pid(PID): Given PID, return the corresponding thread_info
+variable."""
+
+ def __init__(self):
+ super(LxThreadInfoByPidFunc, self).__init__("lx_thread_info_by_pid")
+
+ def invoke(self, pid):
+ task = get_task_by_pid(pid)
+ if task:
+ return get_thread_info(task.dereference())
+ else:
+ raise gdb.GdbError("No task of PID " + str(pid))
+
+LxThreadInfoByPidFunc()
diff --git a/scripts/gdb/linux/utils.py b/scripts/gdb/linux/utils.py
index 0893b32..5080587 100644
--- a/scripts/gdb/linux/utils.py
+++ b/scripts/gdb/linux/utils.py
@@ -87,11 +87,24 @@
return target_endianness
+def read_memoryview(inf, start, length):
+ return memoryview(inf.read_memory(start, length))
+
+
def read_u16(buffer):
- if get_target_endianness() == LITTLE_ENDIAN:
- return ord(buffer[0]) + (ord(buffer[1]) << 8)
+ value = [0, 0]
+
+ if type(buffer[0]) is str:
+ value[0] = ord(buffer[0])
+ value[1] = ord(buffer[1])
else:
- return ord(buffer[1]) + (ord(buffer[0]) << 8)
+ value[0] = buffer[0]
+ value[1] = buffer[1]
+
+ if get_target_endianness() == LITTLE_ENDIAN:
+ return value[0] + (value[1] << 8)
+ else:
+ return value[1] + (value[0] << 8)
def read_u32(buffer):
@@ -154,3 +167,18 @@
if gdbserver_type is not None and hasattr(gdb, 'events'):
gdb.events.exited.connect(exit_handler)
return gdbserver_type
+
+
+def gdb_eval_or_none(expresssion):
+ try:
+ return gdb.parse_and_eval(expresssion)
+ except:
+ return None
+
+
+def dentry_name(d):
+ parent = d['d_parent']
+ if parent == d or parent == 0:
+ return ""
+ p = dentry_name(d['d_parent']) + "/"
+ return p + d['d_iname'].string()
diff --git a/scripts/gdb/vmlinux-gdb.py b/scripts/gdb/vmlinux-gdb.py
index d5943ec..3a80ad6 100644
--- a/scripts/gdb/vmlinux-gdb.py
+++ b/scripts/gdb/vmlinux-gdb.py
@@ -30,3 +30,5 @@
import linux.cpus
import linux.lists
import linux.proc
+ import linux.constants
+ import linux.radixtree
diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c
index 638b143e..1f22a18 100644
--- a/scripts/kallsyms.c
+++ b/scripts/kallsyms.c
@@ -63,7 +63,6 @@
static int all_symbols = 0;
static int absolute_percpu = 0;
static char symbol_prefix_char = '\0';
-static unsigned long long kernel_start_addr = 0;
static int base_relative = 0;
int token_profit[0x10000];
@@ -223,15 +222,13 @@
static char *special_suffixes[] = {
"_veneer", /* arm */
+ "_from_arm", /* arm */
+ "_from_thumb", /* arm */
NULL };
int i;
char *sym_name = (char *)s->sym + 1;
-
- if (s->addr < kernel_start_addr)
- return 0;
-
/* skip prefix char */
if (symbol_prefix_char && *sym_name == symbol_prefix_char)
sym_name++;
@@ -765,9 +762,6 @@
if ((*p == '"' && *(p+2) == '"') || (*p == '\'' && *(p+2) == '\''))
p++;
symbol_prefix_char = *p;
- } else if (strncmp(argv[i], "--page-offset=", 14) == 0) {
- const char *p = &argv[i][14];
- kernel_start_addr = strtoull(p, NULL, 16);
} else if (strcmp(argv[i], "--base-relative") == 0)
base_relative = 1;
else
diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
index f3d3fb4..b8c7b29 100755
--- a/scripts/kconfig/streamline_config.pl
+++ b/scripts/kconfig/streamline_config.pl
@@ -188,7 +188,7 @@
$cont = 0;
# collect any Kconfig sources
- if (/^source\s*"(.*)"/) {
+ if (/^source\s+"?([^"]+)/) {
my $kconfig = $1;
# prevent reading twice.
if (!defined($read_kconfigs{$kconfig})) {
@@ -237,7 +237,7 @@
}
# configs without prompts must be selected
- } elsif ($state ne "NONE" && /^\s*tristate\s\S/) {
+ } elsif ($state ne "NONE" && /^\s*(tristate\s+\S|prompt\b)/) {
# note if the config has a prompt
$prompts{$config} = 1;
@@ -256,8 +256,8 @@
$iflevel-- if ($iflevel);
- # stop on "help"
- } elsif (/^\s*help\s*$/) {
+ # stop on "help" and keywords that end a menu entry
+ } elsif (/^\s*(---)?help(---)?\s*$/ || /^(comment|choice|menu)\b/) {
$state = "NONE";
}
}
@@ -454,7 +454,7 @@
$p =~ s/^[^$valid]*[$valid]+//;
# We only need to process if the depend config is a module
- if (!defined($orig_configs{$conf}) || !$orig_configs{conf} eq "m") {
+ if (!defined($orig_configs{$conf}) || $orig_configs{$conf} eq "y") {
next;
}
@@ -610,6 +610,40 @@
next;
}
+ if (/CONFIG_MODULE_SIG_KEY="(.+)"/) {
+ my $orig_cert = $1;
+ my $default_cert = "certs/signing_key.pem";
+
+ # Check that the logic in this script still matches the one in Kconfig
+ if (!defined($depends{"MODULE_SIG_KEY"}) ||
+ $depends{"MODULE_SIG_KEY"} !~ /"\Q$default_cert\E"/) {
+ print STDERR "WARNING: MODULE_SIG_KEY assertion failure, ",
+ "update needed to ", __FILE__, " line ", __LINE__, "\n";
+ print;
+ } elsif ($orig_cert ne $default_cert && ! -f $orig_cert) {
+ print STDERR "Module signature verification enabled but ",
+ "module signing key \"$orig_cert\" not found. Resetting ",
+ "signing key to default value.\n";
+ print "CONFIG_MODULE_SIG_KEY=\"$default_cert\"\n";
+ } else {
+ print;
+ }
+ next;
+ }
+
+ if (/CONFIG_SYSTEM_TRUSTED_KEYS="(.+)"/) {
+ my $orig_keys = $1;
+
+ if (! -f $orig_keys) {
+ print STDERR "System keyring enabled but keys \"$orig_keys\" ",
+ "not found. Resetting keys to default value.\n";
+ print "CONFIG_SYSTEM_TRUSTED_KEYS=\"\"\n";
+ } else {
+ print;
+ }
+ next;
+ }
+
if (/^(CONFIG.*)=(m|y)/) {
if (defined($configs{$1})) {
if ($localyesconfig) {
diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index c37255b..2fc8fad 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -39,41 +39,44 @@
# 25/07/2012 - Added support for HTML5
# -- Dan Luedtke <mail@danrl.de>
-#
-# This will read a 'c' file and scan for embedded comments in the
-# style of gnome comments (+minor extensions - see below).
-#
+sub usage {
+ my $message = <<"EOF";
+Usage: $0 [OPTION ...] FILE ...
-# Note: This only supports 'c'.
+Read C language source or header FILEs, extract embedded documentation comments,
+and print formatted documentation to standard output.
-# usage:
-# kernel-doc [ -docbook | -html | -html5 | -text | -man | -list ]
-# [ -no-doc-sections ]
-# [ -function funcname [ -function funcname ...] ]
-# c file(s)s > outputfile
-# or
-# [ -nofunction funcname [ -function funcname ...] ]
-# c file(s)s > outputfile
-#
-# Set output format using one of -docbook -html -html5 -text or -man.
-# Default is man.
-# The -list format is for internal use by docproc.
-#
-# -no-doc-sections
-# Do not output DOC: sections
-#
-# -function funcname
-# If set, then only generate documentation for the given function(s) or
-# DOC: section titles. All other functions and DOC: sections are ignored.
-#
-# -nofunction funcname
-# If set, then only generate documentation for the other function(s)/DOC:
-# sections. Cannot be used together with -function (yes, that's a bug --
-# perl hackers can fix it 8))
-#
-# c files - list of 'c' files to process
-#
-# All output goes to stdout, with errors to stderr.
+The documentation comments are identified by "/**" opening comment mark. See
+Documentation/kernel-doc-nano-HOWTO.txt for the documentation comment syntax.
+
+Output format selection (mutually exclusive):
+ -docbook Output DocBook format.
+ -html Output HTML format.
+ -html5 Output HTML5 format.
+ -list Output symbol list format. This is for use by docproc.
+ -man Output troff manual page format. This is the default.
+ -rst Output reStructuredText format.
+ -text Output plain text format.
+
+Output selection (mutually exclusive):
+ -function NAME Only output documentation for the given function(s)
+ or DOC: section title(s). All other functions and DOC:
+ sections are ignored. May be specified multiple times.
+ -nofunction NAME Do NOT output documentation for the given function(s);
+ only output documentation for the other functions and
+ DOC: sections. May be specified multiple times.
+
+Output selection modifiers:
+ -no-doc-sections Do not output DOC: sections.
+
+Other parameters:
+ -v Verbose output, more warnings and other information.
+ -h Print this help.
+
+EOF
+ print $message;
+ exit 1;
+}
#
# format of comments.
@@ -201,6 +204,8 @@
my $type_struct = '\&((struct\s*)*[_\w]+)';
my $type_struct_xml = '\\&((struct\s*)*[_\w]+)';
my $type_env = '(\$\w+)';
+my $type_enum_full = '\&(enum)\s*([_\w]+)';
+my $type_struct_full = '\&(struct)\s*([_\w]+)';
# Output conversion substitutions.
# One for each output format
@@ -266,6 +271,17 @@
);
my $blankline_text = "";
+# rst-mode
+my @highlights_rst = (
+ [$type_constant, "``\$1``"],
+ [$type_func, "\\:c\\:func\\:`\$1`"],
+ [$type_struct_full, "\\:c\\:type\\:`\$1 \$2 <\$2>`"],
+ [$type_enum_full, "\\:c\\:type\\:`\$1 \$2 <\$2>`"],
+ [$type_struct, "\\:c\\:type\\:`struct \$1 <\$1>`"],
+ [$type_param, "**\$1**"]
+ );
+my $blankline_rst = "\n";
+
# list mode
my @highlights_list = (
[$type_constant, "\$1"],
@@ -402,6 +418,10 @@
$output_mode = "text";
@highlights = @highlights_text;
$blankline = $blankline_text;
+ } elsif ($cmd eq "-rst") {
+ $output_mode = "rst";
+ @highlights = @highlights_rst;
+ $blankline = $blankline_rst;
} elsif ($cmd eq "-docbook") {
$output_mode = "xml";
@highlights = @highlights_xml;
@@ -437,17 +457,6 @@
# continue execution near EOF;
-sub usage {
- print "Usage: $0 [ -docbook | -html | -html5 | -text | -man | -list ]\n";
- print " [ -no-doc-sections ]\n";
- print " [ -function funcname [ -function funcname ...] ]\n";
- print " [ -nofunction funcname [ -nofunction funcname ...] ]\n";
- print " [ -v ]\n";
- print " c source file(s) > outputfile\n";
- print " -v : verbose output, more warnings & other info listed\n";
- exit 1;
-}
-
# get kernel version from env
sub get_kernel_version() {
my $version = 'unknown kernel version';
@@ -1713,6 +1722,208 @@
}
}
+##
+# output in restructured text
+#
+
+#
+# This could use some work; it's used to output the DOC: sections, and
+# starts by putting out the name of the doc section itself, but that tends
+# to duplicate a header already in the template file.
+#
+sub output_blockhead_rst(%) {
+ my %args = %{$_[0]};
+ my ($parameter, $section);
+
+ foreach $section (@{$args{'sectionlist'}}) {
+ print "**$section**\n\n";
+ output_highlight_rst($args{'sections'}{$section});
+ print "\n";
+ }
+}
+
+sub output_highlight_rst {
+ my $contents = join "\n",@_;
+ my $line;
+
+ # undo the evil effects of xml_escape() earlier
+ $contents = xml_unescape($contents);
+
+ eval $dohighlight;
+ die $@ if $@;
+
+ foreach $line (split "\n", $contents) {
+ if ($line eq "") {
+ print $lineprefix, $blankline;
+ } else {
+ $line =~ s/\\\\\\/\&/g;
+ print $lineprefix, $line;
+ }
+ print "\n";
+ }
+}
+
+sub output_function_rst(%) {
+ my %args = %{$_[0]};
+ my ($parameter, $section);
+ my $start;
+
+ print ".. c:function:: ";
+ if ($args{'functiontype'} ne "") {
+ $start = $args{'functiontype'} . " " . $args{'function'} . " (";
+ } else {
+ $start = $args{'function'} . " (";
+ }
+ print $start;
+
+ my $count = 0;
+ foreach my $parameter (@{$args{'parameterlist'}}) {
+ if ($count ne 0) {
+ print ", ";
+ }
+ $count++;
+ $type = $args{'parametertypes'}{$parameter};
+ if ($type =~ m/([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)/) {
+ # pointer-to-function
+ print $1 . $parameter . ") (" . $2;
+ } else {
+ print $type . " " . $parameter;
+ }
+ }
+ print ")\n\n " . $args{'purpose'} . "\n\n";
+
+ print ":Parameters:\n\n";
+ foreach $parameter (@{$args{'parameterlist'}}) {
+ my $parameter_name = $parameter;
+ #$parameter_name =~ s/\[.*//;
+ $type = $args{'parametertypes'}{$parameter};
+
+ if ($type ne "") {
+ print " ``$type $parameter``\n";
+ } else {
+ print " ``$parameter``\n";
+ }
+ if ($args{'parameterdescs'}{$parameter_name} ne $undescribed) {
+ my $oldprefix = $lineprefix;
+ $lineprefix = " ";
+ output_highlight_rst($args{'parameterdescs'}{$parameter_name});
+ $lineprefix = $oldprefix;
+ } else {
+ print "\n _undescribed_\n";
+ }
+ print "\n";
+ }
+ output_section_rst(@_);
+}
+
+sub output_section_rst(%) {
+ my %args = %{$_[0]};
+ my $section;
+ my $oldprefix = $lineprefix;
+ $lineprefix = " ";
+
+ foreach $section (@{$args{'sectionlist'}}) {
+ print ":$section:\n\n";
+ output_highlight_rst($args{'sections'}{$section});
+ print "\n";
+ }
+ print "\n";
+ $lineprefix = $oldprefix;
+}
+
+sub output_enum_rst(%) {
+ my %args = %{$_[0]};
+ my ($parameter);
+ my $count;
+ my $name = "enum " . $args{'enum'};
+
+ print "\n\n.. c:type:: " . $name . "\n\n";
+ print " " . $args{'purpose'} . "\n\n";
+
+ print "..\n\n:Constants:\n\n";
+ my $oldprefix = $lineprefix;
+ $lineprefix = " ";
+ foreach $parameter (@{$args{'parameterlist'}}) {
+ print " `$parameter`\n";
+ if ($args{'parameterdescs'}{$parameter} ne $undescribed) {
+ output_highlight_rst($args{'parameterdescs'}{$parameter});
+ } else {
+ print " undescribed\n";
+ }
+ print "\n";
+ }
+ $lineprefix = $oldprefix;
+ output_section_rst(@_);
+}
+
+sub output_typedef_rst(%) {
+ my %args = %{$_[0]};
+ my ($parameter);
+ my $count;
+ my $name = "typedef " . $args{'typedef'};
+
+ ### FIXME: should the name below contain "typedef" or not?
+ print "\n\n.. c:type:: " . $name . "\n\n";
+ print " " . $args{'purpose'} . "\n\n";
+
+ output_section_rst(@_);
+}
+
+sub output_struct_rst(%) {
+ my %args = %{$_[0]};
+ my ($parameter);
+ my $name = $args{'type'} . " " . $args{'struct'};
+
+ print "\n\n.. c:type:: " . $name . "\n\n";
+ print " " . $args{'purpose'} . "\n\n";
+
+ print ":Definition:\n\n";
+ print " ::\n\n";
+ print " " . $args{'type'} . " " . $args{'struct'} . " {\n";
+ foreach $parameter (@{$args{'parameterlist'}}) {
+ if ($parameter =~ /^#/) {
+ print " " . "$parameter\n";
+ next;
+ }
+
+ my $parameter_name = $parameter;
+ $parameter_name =~ s/\[.*//;
+
+ ($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
+ $type = $args{'parametertypes'}{$parameter};
+ if ($type =~ m/([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)/) {
+ # pointer-to-function
+ print " $1 $parameter) ($2);\n";
+ } elsif ($type =~ m/^(.*?)\s*(:.*)/) {
+ # bitfield
+ print " $1 $parameter$2;\n";
+ } else {
+ print " " . $type . " " . $parameter . ";\n";
+ }
+ }
+ print " };\n\n";
+
+ print ":Members:\n\n";
+ foreach $parameter (@{$args{'parameterlist'}}) {
+ ($parameter =~ /^#/) && next;
+
+ my $parameter_name = $parameter;
+ $parameter_name =~ s/\[.*//;
+
+ ($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
+ $type = $args{'parametertypes'}{$parameter};
+ print " `$type $parameter`" . "\n";
+ my $oldprefix = $lineprefix;
+ $lineprefix = " ";
+ output_highlight_rst($args{'parameterdescs'}{$parameter_name});
+ $lineprefix = $oldprefix;
+ print "\n";
+ }
+ print "\n";
+ output_section_rst(@_);
+}
+
+
## list mode output functions
sub output_function_list(%) {
@@ -2414,6 +2625,18 @@
return $text;
}
+# xml_unescape: reverse the effects of xml_escape
+sub xml_unescape($) {
+ my $text = shift;
+ if (($output_mode eq "text") || ($output_mode eq "man")) {
+ return $text;
+ }
+ $text =~ s/\\\\\\amp;/\&/g;
+ $text =~ s/\\\\\\lt;/</g;
+ $text =~ s/\\\\\\gt;/>/g;
+ return $text;
+}
+
# convert local escape strings to html
# local escape strings look like: '\\\\menmonic:' (that's 4 backslashes)
sub local_unescape($) {
diff --git a/scripts/ld-version.sh b/scripts/ld-version.sh
index 7bfe9fa..d135882 100755
--- a/scripts/ld-version.sh
+++ b/scripts/ld-version.sh
@@ -5,6 +5,6 @@
gsub(".*version ", "");
gsub("-.*", "");
split($1,a, ".");
- print a[1]*100000000 + a[2]*1000000 + a[3]*10000 + a[4]*100 + a[5];
+ print a[1]*100000000 + a[2]*1000000 + a[3]*10000;
exit
}
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index 49d61ad..f0f6d9d 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -82,10 +82,6 @@
kallsymopt="${kallsymopt} --all-symbols"
fi
- if [ -n "${CONFIG_ARM}" ] && [ -z "${CONFIG_XIP_KERNEL}" ] && [ -n "${CONFIG_PAGE_OFFSET}" ]; then
- kallsymopt="${kallsymopt} --page-offset=$CONFIG_PAGE_OFFSET"
- fi
-
if [ -n "${CONFIG_KALLSYMS_ABSOLUTE_PERCPU}" ]; then
kallsymopt="${kallsymopt} --absolute-percpu"
fi
diff --git a/scripts/spelling.txt b/scripts/spelling.txt
index 946caf3..fa79c6d 100644
--- a/scripts/spelling.txt
+++ b/scripts/spelling.txt
@@ -428,6 +428,7 @@
fetaure||feature
fetaures||features
fileystem||filesystem
+fimware||firmware
finanize||finalize
findn||find
finilizes||finalizes
diff --git a/security/Kconfig b/security/Kconfig
index e4523789..176758c 100644
--- a/security/Kconfig
+++ b/security/Kconfig
@@ -122,6 +122,7 @@
source security/smack/Kconfig
source security/tomoyo/Kconfig
source security/apparmor/Kconfig
+source security/loadpin/Kconfig
source security/yama/Kconfig
source security/integrity/Kconfig
diff --git a/security/Makefile b/security/Makefile
index c9bfbc84..f2d71cd 100644
--- a/security/Makefile
+++ b/security/Makefile
@@ -8,6 +8,7 @@
subdir-$(CONFIG_SECURITY_TOMOYO) += tomoyo
subdir-$(CONFIG_SECURITY_APPARMOR) += apparmor
subdir-$(CONFIG_SECURITY_YAMA) += yama
+subdir-$(CONFIG_SECURITY_LOADPIN) += loadpin
# always enable default capabilities
obj-y += commoncap.o
@@ -22,6 +23,7 @@
obj-$(CONFIG_SECURITY_TOMOYO) += tomoyo/
obj-$(CONFIG_SECURITY_APPARMOR) += apparmor/
obj-$(CONFIG_SECURITY_YAMA) += yama/
+obj-$(CONFIG_SECURITY_LOADPIN) += loadpin/
obj-$(CONFIG_CGROUP_DEVICE) += device_cgroup.o
# Object integrity file lists
diff --git a/security/integrity/Kconfig b/security/integrity/Kconfig
index 979be65..da95658 100644
--- a/security/integrity/Kconfig
+++ b/security/integrity/Kconfig
@@ -35,7 +35,6 @@
default n
select ASYMMETRIC_KEY_TYPE
select ASYMMETRIC_PUBLIC_KEY_SUBTYPE
- select PUBLIC_KEY_ALGO_RSA
select CRYPTO_RSA
select X509_CERTIFICATE_PARSER
help
diff --git a/security/integrity/digsig.c b/security/integrity/digsig.c
index 8ef1511..4304372 100644
--- a/security/integrity/digsig.c
+++ b/security/integrity/digsig.c
@@ -18,6 +18,8 @@
#include <linux/cred.h>
#include <linux/key-type.h>
#include <linux/digsig.h>
+#include <crypto/public_key.h>
+#include <keys/system_keyring.h>
#include "integrity.h"
@@ -40,6 +42,12 @@
static bool init_keyring __initdata;
#endif
+#ifdef CONFIG_IMA_KEYRINGS_PERMIT_SIGNED_BY_BUILTIN_OR_SECONDARY
+#define restrict_link_to_ima restrict_link_by_builtin_and_secondary_trusted
+#else
+#define restrict_link_to_ima restrict_link_by_builtin_trusted
+#endif
+
int integrity_digsig_verify(const unsigned int id, const char *sig, int siglen,
const char *digest, int digestlen)
{
@@ -83,10 +91,9 @@
((KEY_POS_ALL & ~KEY_POS_SETATTR) |
KEY_USR_VIEW | KEY_USR_READ |
KEY_USR_WRITE | KEY_USR_SEARCH),
- KEY_ALLOC_NOT_IN_QUOTA, NULL);
- if (!IS_ERR(keyring[id]))
- set_bit(KEY_FLAG_TRUSTED_ONLY, &keyring[id]->flags);
- else {
+ KEY_ALLOC_NOT_IN_QUOTA,
+ restrict_link_to_ima, NULL);
+ if (IS_ERR(keyring[id])) {
err = PTR_ERR(keyring[id]);
pr_info("Can't allocate %s keyring (%d)\n",
keyring_name[id], err);
diff --git a/security/integrity/ima/Kconfig b/security/integrity/ima/Kconfig
index e54a8a8..5487827 100644
--- a/security/integrity/ima/Kconfig
+++ b/security/integrity/ima/Kconfig
@@ -155,23 +155,33 @@
This option is deprecated in favor of INTEGRITY_TRUSTED_KEYRING
-config IMA_MOK_KEYRING
- bool "Create IMA machine owner keys (MOK) and blacklist keyrings"
+config IMA_KEYRINGS_PERMIT_SIGNED_BY_BUILTIN_OR_SECONDARY
+ bool "Permit keys validly signed by a built-in or secondary CA cert (EXPERIMENTAL)"
+ depends on SYSTEM_TRUSTED_KEYRING
+ depends on SECONDARY_TRUSTED_KEYRING
+ depends on INTEGRITY_ASYMMETRIC_KEYS
+ select INTEGRITY_TRUSTED_KEYRING
+ default n
+ help
+ Keys may be added to the IMA or IMA blacklist keyrings, if the
+ key is validly signed by a CA cert in the system built-in or
+ secondary trusted keyrings.
+
+ Intermediate keys between those the kernel has compiled in and the
+ IMA keys to be added may be added to the system secondary keyring,
+ provided they are validly signed by a key already resident in the
+ built-in or secondary trusted keyrings.
+
+config IMA_BLACKLIST_KEYRING
+ bool "Create IMA machine owner blacklist keyrings (EXPERIMENTAL)"
depends on SYSTEM_TRUSTED_KEYRING
depends on IMA_TRUSTED_KEYRING
default n
help
- This option creates IMA MOK and blacklist keyrings. IMA MOK is an
- intermediate keyring that sits between .system and .ima keyrings,
- effectively forming a simple CA hierarchy. To successfully import a
- key into .ima_mok it must be signed by a key which CA is in .system
- keyring. On turn any key that needs to go in .ima keyring must be
- signed by CA in either .system or .ima_mok keyrings. IMA MOK is empty
- at kernel boot.
-
- IMA blacklist keyring contains all revoked IMA keys. It is consulted
- before any other keyring. If the search is successful the requested
- operation is rejected and error is returned to the caller.
+ This option creates an IMA blacklist keyring, which contains all
+ revoked IMA keys. It is consulted before any other keyring. If
+ the search is successful the requested operation is rejected and
+ an error is returned to the caller.
config IMA_LOAD_X509
bool "Load X509 certificate onto the '.ima' trusted keyring"
diff --git a/security/integrity/ima/Makefile b/security/integrity/ima/Makefile
index a8539f9..9aeaeda 100644
--- a/security/integrity/ima/Makefile
+++ b/security/integrity/ima/Makefile
@@ -8,4 +8,4 @@
ima-y := ima_fs.o ima_queue.o ima_init.o ima_main.o ima_crypto.o ima_api.o \
ima_policy.o ima_template.o ima_template_lib.o
ima-$(CONFIG_IMA_APPRAISE) += ima_appraise.o
-obj-$(CONFIG_IMA_MOK_KEYRING) += ima_mok.o
+obj-$(CONFIG_IMA_BLACKLIST_KEYRING) += ima_mok.o
diff --git a/security/integrity/ima/ima_appraise.c b/security/integrity/ima/ima_appraise.c
index 6b4694a..1bcbc12 100644
--- a/security/integrity/ima/ima_appraise.c
+++ b/security/integrity/ima/ima_appraise.c
@@ -275,6 +275,11 @@
xattr_value->type != EVM_IMA_XATTR_DIGSIG)) {
if (!ima_fix_xattr(dentry, iint))
status = INTEGRITY_PASS;
+ } else if ((inode->i_size == 0) &&
+ (iint->flags & IMA_NEW_FILE) &&
+ (xattr_value &&
+ xattr_value->type == EVM_IMA_XATTR_DIGSIG)) {
+ status = INTEGRITY_PASS;
}
integrity_audit_msg(AUDIT_INTEGRITY_DATA, inode, filename,
op, cause, rc, 0);
@@ -328,7 +333,7 @@
if (iint) {
iint->flags &= ~(IMA_APPRAISE | IMA_APPRAISED |
IMA_APPRAISE_SUBMASK | IMA_APPRAISED_SUBMASK |
- IMA_ACTION_FLAGS);
+ IMA_ACTION_RULE_FLAGS);
if (must_appraise)
iint->flags |= IMA_APPRAISE;
}
diff --git a/security/integrity/ima/ima_main.c b/security/integrity/ima/ima_main.c
index 391f417..68b26c3 100644
--- a/security/integrity/ima/ima_main.c
+++ b/security/integrity/ima/ima_main.c
@@ -246,7 +246,8 @@
ima_audit_measurement(iint, pathname);
out_digsig:
- if ((mask & MAY_WRITE) && (iint->flags & IMA_DIGSIG))
+ if ((mask & MAY_WRITE) && (iint->flags & IMA_DIGSIG) &&
+ !(iint->flags & IMA_NEW_FILE))
rc = -EACCES;
kfree(xattr_value);
out_free:
@@ -316,6 +317,28 @@
EXPORT_SYMBOL_GPL(ima_file_check);
/**
+ * ima_post_path_mknod - mark as a new inode
+ * @dentry: newly created dentry
+ *
+ * Mark files created via the mknodat syscall as new, so that the
+ * file data can be written later.
+ */
+void ima_post_path_mknod(struct dentry *dentry)
+{
+ struct integrity_iint_cache *iint;
+ struct inode *inode = dentry->d_inode;
+ int must_appraise;
+
+ must_appraise = ima_must_appraise(inode, MAY_ACCESS, FILE_CHECK);
+ if (!must_appraise)
+ return;
+
+ iint = integrity_inode_get(inode);
+ if (iint)
+ iint->flags |= IMA_NEW_FILE;
+}
+
+/**
* ima_read_file - pre-measure/appraise hook decision based on policy
* @file: pointer to the file to be measured/appraised/audit
* @read_id: caller identifier
diff --git a/security/integrity/ima/ima_mok.c b/security/integrity/ima/ima_mok.c
index 676885e..74a27995 100644
--- a/security/integrity/ima/ima_mok.c
+++ b/security/integrity/ima/ima_mok.c
@@ -17,38 +17,29 @@
#include <linux/cred.h>
#include <linux/err.h>
#include <linux/init.h>
-#include <keys/asymmetric-type.h>
+#include <keys/system_keyring.h>
-struct key *ima_mok_keyring;
struct key *ima_blacklist_keyring;
/*
- * Allocate the IMA MOK and blacklist keyrings
+ * Allocate the IMA blacklist keyring
*/
__init int ima_mok_init(void)
{
- pr_notice("Allocating IMA MOK and blacklist keyrings.\n");
-
- ima_mok_keyring = keyring_alloc(".ima_mok",
- KUIDT_INIT(0), KGIDT_INIT(0), current_cred(),
- (KEY_POS_ALL & ~KEY_POS_SETATTR) |
- KEY_USR_VIEW | KEY_USR_READ |
- KEY_USR_WRITE | KEY_USR_SEARCH,
- KEY_ALLOC_NOT_IN_QUOTA, NULL);
+ pr_notice("Allocating IMA blacklist keyring.\n");
ima_blacklist_keyring = keyring_alloc(".ima_blacklist",
KUIDT_INIT(0), KGIDT_INIT(0), current_cred(),
(KEY_POS_ALL & ~KEY_POS_SETATTR) |
KEY_USR_VIEW | KEY_USR_READ |
KEY_USR_WRITE | KEY_USR_SEARCH,
- KEY_ALLOC_NOT_IN_QUOTA, NULL);
+ KEY_ALLOC_NOT_IN_QUOTA,
+ restrict_link_by_builtin_trusted, NULL);
- if (IS_ERR(ima_mok_keyring) || IS_ERR(ima_blacklist_keyring))
- panic("Can't allocate IMA MOK or blacklist keyrings.");
- set_bit(KEY_FLAG_TRUSTED_ONLY, &ima_mok_keyring->flags);
+ if (IS_ERR(ima_blacklist_keyring))
+ panic("Can't allocate IMA blacklist keyring.");
- set_bit(KEY_FLAG_TRUSTED_ONLY, &ima_blacklist_keyring->flags);
set_bit(KEY_FLAG_KEEP, &ima_blacklist_keyring->flags);
return 0;
}
diff --git a/security/integrity/ima/ima_policy.c b/security/integrity/ima/ima_policy.c
index 3cd0a58..0f887a5 100644
--- a/security/integrity/ima/ima_policy.c
+++ b/security/integrity/ima/ima_policy.c
@@ -972,7 +972,7 @@
int ima_policy_show(struct seq_file *m, void *v)
{
struct ima_rule_entry *entry = v;
- int i = 0;
+ int i;
char tbuf[64] = {0,};
rcu_read_lock();
@@ -1012,17 +1012,7 @@
}
if (entry->flags & IMA_FSUUID) {
- seq_puts(m, "fsuuid=");
- for (i = 0; i < ARRAY_SIZE(entry->fsuuid); ++i) {
- switch (i) {
- case 4:
- case 6:
- case 8:
- case 10:
- seq_puts(m, "-");
- }
- seq_printf(m, "%x", entry->fsuuid[i]);
- }
+ seq_printf(m, "fsuuid=%pU", entry->fsuuid);
seq_puts(m, " ");
}
diff --git a/security/integrity/integrity.h b/security/integrity/integrity.h
index e08935c..90bc57d 100644
--- a/security/integrity/integrity.h
+++ b/security/integrity/integrity.h
@@ -28,6 +28,7 @@
/* iint cache flags */
#define IMA_ACTION_FLAGS 0xff000000
+#define IMA_ACTION_RULE_FLAGS 0x06000000
#define IMA_DIGSIG 0x01000000
#define IMA_DIGSIG_REQUIRED 0x02000000
#define IMA_PERMIT_DIRECTIO 0x04000000
diff --git a/security/keys/Kconfig b/security/keys/Kconfig
index fe4d74e..f826e87 100644
--- a/security/keys/Kconfig
+++ b/security/keys/Kconfig
@@ -41,6 +41,10 @@
bool "Large payload keys"
depends on KEYS
depends on TMPFS
+ select CRYPTO
+ select CRYPTO_AES
+ select CRYPTO_ECB
+ select CRYPTO_RNG
help
This option provides support for holding large keys within the kernel
(for example Kerberos ticket caches). The data may be stored out to
@@ -81,3 +85,14 @@
Userspace only ever sees/stores encrypted blobs.
If you are unsure as to whether this is required, answer N.
+
+config KEY_DH_OPERATIONS
+ bool "Diffie-Hellman operations on retained keys"
+ depends on KEYS
+ select MPILIB
+ help
+ This option provides support for calculating Diffie-Hellman
+ public keys and shared secrets using values stored as keys
+ in the kernel.
+
+ If you are unsure as to whether this is required, answer N.
diff --git a/security/keys/Makefile b/security/keys/Makefile
index dfb3a7b..1fd4a16 100644
--- a/security/keys/Makefile
+++ b/security/keys/Makefile
@@ -19,6 +19,7 @@
obj-$(CONFIG_PROC_FS) += proc.o
obj-$(CONFIG_SYSCTL) += sysctl.o
obj-$(CONFIG_PERSISTENT_KEYRINGS) += persistent.o
+obj-$(CONFIG_KEY_DH_OPERATIONS) += dh.o
#
# Key types
diff --git a/security/keys/big_key.c b/security/keys/big_key.c
index c721e39..9e443fc 100644
--- a/security/keys/big_key.c
+++ b/security/keys/big_key.c
@@ -14,8 +14,10 @@
#include <linux/file.h>
#include <linux/shmem_fs.h>
#include <linux/err.h>
+#include <linux/scatterlist.h>
#include <keys/user-type.h>
#include <keys/big_key-type.h>
+#include <crypto/rng.h>
/*
* Layout of key payload words.
@@ -28,6 +30,14 @@
};
/*
+ * Crypto operation with big_key data
+ */
+enum big_key_op {
+ BIG_KEY_ENC,
+ BIG_KEY_DEC,
+};
+
+/*
* If the data is under this limit, there's no point creating a shm file to
* hold it as the permanently resident metadata for the shmem fs will be at
* least as large as the data.
@@ -35,6 +45,11 @@
#define BIG_KEY_FILE_THRESHOLD (sizeof(struct inode) + sizeof(struct dentry))
/*
+ * Key size for big_key data encryption
+ */
+#define ENC_KEY_SIZE 16
+
+/*
* big_key defined keys take an arbitrary string as the description and an
* arbitrary blob of data as the payload
*/
@@ -50,12 +65,62 @@
};
/*
+ * Crypto names for big_key data encryption
+ */
+static const char big_key_rng_name[] = "stdrng";
+static const char big_key_alg_name[] = "ecb(aes)";
+
+/*
+ * Crypto algorithms for big_key data encryption
+ */
+static struct crypto_rng *big_key_rng;
+static struct crypto_blkcipher *big_key_blkcipher;
+
+/*
+ * Generate random key to encrypt big_key data
+ */
+static inline int big_key_gen_enckey(u8 *key)
+{
+ return crypto_rng_get_bytes(big_key_rng, key, ENC_KEY_SIZE);
+}
+
+/*
+ * Encrypt/decrypt big_key data
+ */
+static int big_key_crypt(enum big_key_op op, u8 *data, size_t datalen, u8 *key)
+{
+ int ret = -EINVAL;
+ struct scatterlist sgio;
+ struct blkcipher_desc desc;
+
+ if (crypto_blkcipher_setkey(big_key_blkcipher, key, ENC_KEY_SIZE)) {
+ ret = -EAGAIN;
+ goto error;
+ }
+
+ desc.flags = 0;
+ desc.tfm = big_key_blkcipher;
+
+ sg_init_one(&sgio, data, datalen);
+
+ if (op == BIG_KEY_ENC)
+ ret = crypto_blkcipher_encrypt(&desc, &sgio, &sgio, datalen);
+ else
+ ret = crypto_blkcipher_decrypt(&desc, &sgio, &sgio, datalen);
+
+error:
+ return ret;
+}
+
+/*
* Preparse a big key
*/
int big_key_preparse(struct key_preparsed_payload *prep)
{
struct path *path = (struct path *)&prep->payload.data[big_key_path];
struct file *file;
+ u8 *enckey;
+ u8 *data = NULL;
ssize_t written;
size_t datalen = prep->datalen;
int ret;
@@ -73,16 +138,43 @@
/* Create a shmem file to store the data in. This will permit the data
* to be swapped out if needed.
*
- * TODO: Encrypt the stored data with a temporary key.
+ * File content is stored encrypted with randomly generated key.
*/
- file = shmem_kernel_file_setup("", datalen, 0);
- if (IS_ERR(file)) {
- ret = PTR_ERR(file);
+ size_t enclen = ALIGN(datalen, crypto_blkcipher_blocksize(big_key_blkcipher));
+
+ /* prepare aligned data to encrypt */
+ data = kmalloc(enclen, GFP_KERNEL);
+ if (!data)
+ return -ENOMEM;
+
+ memcpy(data, prep->data, datalen);
+ memset(data + datalen, 0x00, enclen - datalen);
+
+ /* generate random key */
+ enckey = kmalloc(ENC_KEY_SIZE, GFP_KERNEL);
+ if (!enckey) {
+ ret = -ENOMEM;
goto error;
}
- written = kernel_write(file, prep->data, prep->datalen, 0);
- if (written != datalen) {
+ ret = big_key_gen_enckey(enckey);
+ if (ret)
+ goto err_enckey;
+
+ /* encrypt aligned data */
+ ret = big_key_crypt(BIG_KEY_ENC, data, enclen, enckey);
+ if (ret)
+ goto err_enckey;
+
+ /* save aligned data to file */
+ file = shmem_kernel_file_setup("", enclen, 0);
+ if (IS_ERR(file)) {
+ ret = PTR_ERR(file);
+ goto err_enckey;
+ }
+
+ written = kernel_write(file, data, enclen, 0);
+ if (written != enclen) {
ret = written;
if (written >= 0)
ret = -ENOMEM;
@@ -92,12 +184,15 @@
/* Pin the mount and dentry to the key so that we can open it again
* later
*/
+ prep->payload.data[big_key_data] = enckey;
*path = file->f_path;
path_get(path);
fput(file);
+ kfree(data);
} else {
/* Just store the data in a buffer */
void *data = kmalloc(datalen, GFP_KERNEL);
+
if (!data)
return -ENOMEM;
@@ -108,7 +203,10 @@
err_fput:
fput(file);
+err_enckey:
+ kfree(enckey);
error:
+ kfree(data);
return ret;
}
@@ -119,10 +217,10 @@
{
if (prep->datalen > BIG_KEY_FILE_THRESHOLD) {
struct path *path = (struct path *)&prep->payload.data[big_key_path];
+
path_put(path);
- } else {
- kfree(prep->payload.data[big_key_data]);
}
+ kfree(prep->payload.data[big_key_data]);
}
/*
@@ -147,15 +245,15 @@
{
size_t datalen = (size_t)key->payload.data[big_key_len];
- if (datalen) {
+ if (datalen > BIG_KEY_FILE_THRESHOLD) {
struct path *path = (struct path *)&key->payload.data[big_key_path];
+
path_put(path);
path->mnt = NULL;
path->dentry = NULL;
- } else {
- kfree(key->payload.data[big_key_data]);
- key->payload.data[big_key_data] = NULL;
}
+ kfree(key->payload.data[big_key_data]);
+ key->payload.data[big_key_data] = NULL;
}
/*
@@ -188,17 +286,41 @@
if (datalen > BIG_KEY_FILE_THRESHOLD) {
struct path *path = (struct path *)&key->payload.data[big_key_path];
struct file *file;
- loff_t pos;
+ u8 *data;
+ u8 *enckey = (u8 *)key->payload.data[big_key_data];
+ size_t enclen = ALIGN(datalen, crypto_blkcipher_blocksize(big_key_blkcipher));
+
+ data = kmalloc(enclen, GFP_KERNEL);
+ if (!data)
+ return -ENOMEM;
file = dentry_open(path, O_RDONLY, current_cred());
- if (IS_ERR(file))
- return PTR_ERR(file);
+ if (IS_ERR(file)) {
+ ret = PTR_ERR(file);
+ goto error;
+ }
- pos = 0;
- ret = vfs_read(file, buffer, datalen, &pos);
- fput(file);
- if (ret >= 0 && ret != datalen)
+ /* read file to kernel and decrypt */
+ ret = kernel_read(file, 0, data, enclen);
+ if (ret >= 0 && ret != enclen) {
ret = -EIO;
+ goto err_fput;
+ }
+
+ ret = big_key_crypt(BIG_KEY_DEC, data, enclen, enckey);
+ if (ret)
+ goto err_fput;
+
+ ret = datalen;
+
+ /* copy decrypted data to user */
+ if (copy_to_user(buffer, data, datalen) != 0)
+ ret = -EFAULT;
+
+err_fput:
+ fput(file);
+error:
+ kfree(data);
} else {
ret = datalen;
if (copy_to_user(buffer, key->payload.data[big_key_data],
@@ -209,8 +331,48 @@
return ret;
}
+/*
+ * Register key type
+ */
static int __init big_key_init(void)
{
return register_key_type(&key_type_big_key);
}
+
+/*
+ * Initialize big_key crypto and RNG algorithms
+ */
+static int __init big_key_crypto_init(void)
+{
+ int ret = -EINVAL;
+
+ /* init RNG */
+ big_key_rng = crypto_alloc_rng(big_key_rng_name, 0, 0);
+ if (IS_ERR(big_key_rng)) {
+ big_key_rng = NULL;
+ return -EFAULT;
+ }
+
+ /* seed RNG */
+ ret = crypto_rng_reset(big_key_rng, NULL, crypto_rng_seedsize(big_key_rng));
+ if (ret)
+ goto error;
+
+ /* init block cipher */
+ big_key_blkcipher = crypto_alloc_blkcipher(big_key_alg_name, 0, 0);
+ if (IS_ERR(big_key_blkcipher)) {
+ big_key_blkcipher = NULL;
+ ret = -EFAULT;
+ goto error;
+ }
+
+ return 0;
+
+error:
+ crypto_free_rng(big_key_rng);
+ big_key_rng = NULL;
+ return ret;
+}
+
device_initcall(big_key_init);
+late_initcall(big_key_crypto_init);
diff --git a/security/keys/compat.c b/security/keys/compat.c
index 25430a3..c8783b3 100644
--- a/security/keys/compat.c
+++ b/security/keys/compat.c
@@ -132,6 +132,10 @@
case KEYCTL_GET_PERSISTENT:
return keyctl_get_persistent(arg2, arg3);
+ case KEYCTL_DH_COMPUTE:
+ return keyctl_dh_compute(compat_ptr(arg2), compat_ptr(arg3),
+ arg4);
+
default:
return -EOPNOTSUPP;
}
diff --git a/security/keys/dh.c b/security/keys/dh.c
new file mode 100644
index 0000000..880505a
--- /dev/null
+++ b/security/keys/dh.c
@@ -0,0 +1,160 @@
+/* Crypto operations using stored keys
+ *
+ * Copyright (c) 2016, Intel Corporation
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <linux/mpi.h>
+#include <linux/slab.h>
+#include <linux/uaccess.h>
+#include <keys/user-type.h>
+#include "internal.h"
+
+/*
+ * Public key or shared secret generation function [RFC2631 sec 2.1.1]
+ *
+ * ya = g^xa mod p;
+ * or
+ * ZZ = yb^xa mod p;
+ *
+ * where xa is the local private key, ya is the local public key, g is
+ * the generator, p is the prime, yb is the remote public key, and ZZ
+ * is the shared secret.
+ *
+ * Both are the same calculation, so g or yb are the "base" and ya or
+ * ZZ are the "result".
+ */
+static int do_dh(MPI result, MPI base, MPI xa, MPI p)
+{
+ return mpi_powm(result, base, xa, p);
+}
+
+static ssize_t mpi_from_key(key_serial_t keyid, size_t maxlen, MPI *mpi)
+{
+ struct key *key;
+ key_ref_t key_ref;
+ long status;
+ ssize_t ret;
+
+ key_ref = lookup_user_key(keyid, 0, KEY_NEED_READ);
+ if (IS_ERR(key_ref)) {
+ ret = -ENOKEY;
+ goto error;
+ }
+
+ key = key_ref_to_ptr(key_ref);
+
+ ret = -EOPNOTSUPP;
+ if (key->type == &key_type_user) {
+ down_read(&key->sem);
+ status = key_validate(key);
+ if (status == 0) {
+ const struct user_key_payload *payload;
+
+ payload = user_key_payload(key);
+
+ if (maxlen == 0) {
+ *mpi = NULL;
+ ret = payload->datalen;
+ } else if (payload->datalen <= maxlen) {
+ *mpi = mpi_read_raw_data(payload->data,
+ payload->datalen);
+ if (*mpi)
+ ret = payload->datalen;
+ } else {
+ ret = -EINVAL;
+ }
+ }
+ up_read(&key->sem);
+ }
+
+ key_put(key);
+error:
+ return ret;
+}
+
+long keyctl_dh_compute(struct keyctl_dh_params __user *params,
+ char __user *buffer, size_t buflen)
+{
+ long ret;
+ MPI base, private, prime, result;
+ unsigned nbytes;
+ struct keyctl_dh_params pcopy;
+ uint8_t *kbuf;
+ ssize_t keylen;
+ size_t resultlen;
+
+ if (!params || (!buffer && buflen)) {
+ ret = -EINVAL;
+ goto out;
+ }
+ if (copy_from_user(&pcopy, params, sizeof(pcopy)) != 0) {
+ ret = -EFAULT;
+ goto out;
+ }
+
+ keylen = mpi_from_key(pcopy.prime, buflen, &prime);
+ if (keylen < 0 || !prime) {
+ /* buflen == 0 may be used to query the required buffer size,
+ * which is the prime key length.
+ */
+ ret = keylen;
+ goto out;
+ }
+
+ /* The result is never longer than the prime */
+ resultlen = keylen;
+
+ keylen = mpi_from_key(pcopy.base, SIZE_MAX, &base);
+ if (keylen < 0 || !base) {
+ ret = keylen;
+ goto error1;
+ }
+
+ keylen = mpi_from_key(pcopy.private, SIZE_MAX, &private);
+ if (keylen < 0 || !private) {
+ ret = keylen;
+ goto error2;
+ }
+
+ result = mpi_alloc(0);
+ if (!result) {
+ ret = -ENOMEM;
+ goto error3;
+ }
+
+ kbuf = kmalloc(resultlen, GFP_KERNEL);
+ if (!kbuf) {
+ ret = -ENOMEM;
+ goto error4;
+ }
+
+ ret = do_dh(result, base, private, prime);
+ if (ret)
+ goto error5;
+
+ ret = mpi_read_buffer(result, kbuf, resultlen, &nbytes, NULL);
+ if (ret != 0)
+ goto error5;
+
+ ret = nbytes;
+ if (copy_to_user(buffer, kbuf, nbytes) != 0)
+ ret = -EFAULT;
+
+error5:
+ kfree(kbuf);
+error4:
+ mpi_free(result);
+error3:
+ mpi_free(private);
+error2:
+ mpi_free(base);
+error1:
+ mpi_free(prime);
+out:
+ return ret;
+}
diff --git a/security/keys/internal.h b/security/keys/internal.h
index 5105c2c..8ec7a52 100644
--- a/security/keys/internal.h
+++ b/security/keys/internal.h
@@ -15,6 +15,7 @@
#include <linux/sched.h>
#include <linux/key-type.h>
#include <linux/task_work.h>
+#include <linux/keyctl.h>
struct iovec;
@@ -257,6 +258,17 @@
}
#endif
+#ifdef CONFIG_KEY_DH_OPERATIONS
+extern long keyctl_dh_compute(struct keyctl_dh_params __user *, char __user *,
+ size_t);
+#else
+static inline long keyctl_dh_compute(struct keyctl_dh_params __user *params,
+ char __user *buffer, size_t buflen)
+{
+ return -EOPNOTSUPP;
+}
+#endif
+
/*
* Debugging key validation
*/
diff --git a/security/keys/key.c b/security/keys/key.c
index b287551..bd5a272 100644
--- a/security/keys/key.c
+++ b/security/keys/key.c
@@ -201,6 +201,7 @@
* @cred: The credentials specifying UID namespace.
* @perm: The permissions mask of the new key.
* @flags: Flags specifying quota properties.
+ * @restrict_link: Optional link restriction method for new keyrings.
*
* Allocate a key of the specified type with the attributes given. The key is
* returned in an uninstantiated state and the caller needs to instantiate the
@@ -223,7 +224,10 @@
*/
struct key *key_alloc(struct key_type *type, const char *desc,
kuid_t uid, kgid_t gid, const struct cred *cred,
- key_perm_t perm, unsigned long flags)
+ key_perm_t perm, unsigned long flags,
+ int (*restrict_link)(struct key *,
+ const struct key_type *,
+ const union key_payload *))
{
struct key_user *user = NULL;
struct key *key;
@@ -291,11 +295,10 @@
key->uid = uid;
key->gid = gid;
key->perm = perm;
+ key->restrict_link = restrict_link;
if (!(flags & KEY_ALLOC_NOT_IN_QUOTA))
key->flags |= 1 << KEY_FLAG_IN_QUOTA;
- if (flags & KEY_ALLOC_TRUSTED)
- key->flags |= 1 << KEY_FLAG_TRUSTED;
if (flags & KEY_ALLOC_BUILT_IN)
key->flags |= 1 << KEY_FLAG_BUILTIN;
@@ -496,6 +499,12 @@
}
if (keyring) {
+ if (keyring->restrict_link) {
+ ret = keyring->restrict_link(keyring, key->type,
+ &prep.payload);
+ if (ret < 0)
+ goto error;
+ }
ret = __key_link_begin(keyring, &key->index_key, &edit);
if (ret < 0)
goto error;
@@ -551,8 +560,12 @@
awaken = 0;
ret = -EBUSY;
- if (keyring)
+ if (keyring) {
+ if (keyring->restrict_link)
+ return -EPERM;
+
link_ret = __key_link_begin(keyring, &key->index_key, &edit);
+ }
mutex_lock(&key_construction_mutex);
@@ -793,6 +806,9 @@
struct key *keyring, *key = NULL;
key_ref_t key_ref;
int ret;
+ int (*restrict_link)(struct key *,
+ const struct key_type *,
+ const union key_payload *) = NULL;
/* look up the key type to see if it's one of the registered kernel
* types */
@@ -811,6 +827,10 @@
key_check(keyring);
+ key_ref = ERR_PTR(-EPERM);
+ if (!(flags & KEY_ALLOC_BYPASS_RESTRICTION))
+ restrict_link = keyring->restrict_link;
+
key_ref = ERR_PTR(-ENOTDIR);
if (keyring->type != &key_type_keyring)
goto error_put_type;
@@ -819,7 +839,6 @@
prep.data = payload;
prep.datalen = plen;
prep.quotalen = index_key.type->def_datalen;
- prep.trusted = flags & KEY_ALLOC_TRUSTED;
prep.expiry = TIME_T_MAX;
if (index_key.type->preparse) {
ret = index_key.type->preparse(&prep);
@@ -835,10 +854,13 @@
}
index_key.desc_len = strlen(index_key.description);
- key_ref = ERR_PTR(-EPERM);
- if (!prep.trusted && test_bit(KEY_FLAG_TRUSTED_ONLY, &keyring->flags))
- goto error_free_prep;
- flags |= prep.trusted ? KEY_ALLOC_TRUSTED : 0;
+ if (restrict_link) {
+ ret = restrict_link(keyring, index_key.type, &prep.payload);
+ if (ret < 0) {
+ key_ref = ERR_PTR(ret);
+ goto error_free_prep;
+ }
+ }
ret = __key_link_begin(keyring, &index_key, &edit);
if (ret < 0) {
@@ -879,7 +901,7 @@
/* allocate a new key */
key = key_alloc(index_key.type, index_key.description,
- cred->fsuid, cred->fsgid, cred, perm, flags);
+ cred->fsuid, cred->fsgid, cred, perm, flags, NULL);
if (IS_ERR(key)) {
key_ref = ERR_CAST(key);
goto error_link_end;
diff --git a/security/keys/keyctl.c b/security/keys/keyctl.c
index ed73c6c..3b135a0 100644
--- a/security/keys/keyctl.c
+++ b/security/keys/keyctl.c
@@ -1686,6 +1686,11 @@
case KEYCTL_GET_PERSISTENT:
return keyctl_get_persistent((uid_t)arg2, (key_serial_t)arg3);
+ case KEYCTL_DH_COMPUTE:
+ return keyctl_dh_compute((struct keyctl_dh_params __user *) arg2,
+ (char __user *) arg3,
+ (size_t) arg4);
+
default:
return -EOPNOTSUPP;
}
diff --git a/security/keys/keyring.c b/security/keys/keyring.c
index f931ccf..c91e4e0 100644
--- a/security/keys/keyring.c
+++ b/security/keys/keyring.c
@@ -491,13 +491,17 @@
*/
struct key *keyring_alloc(const char *description, kuid_t uid, kgid_t gid,
const struct cred *cred, key_perm_t perm,
- unsigned long flags, struct key *dest)
+ unsigned long flags,
+ int (*restrict_link)(struct key *,
+ const struct key_type *,
+ const union key_payload *),
+ struct key *dest)
{
struct key *keyring;
int ret;
keyring = key_alloc(&key_type_keyring, description,
- uid, gid, cred, perm, flags);
+ uid, gid, cred, perm, flags, restrict_link);
if (!IS_ERR(keyring)) {
ret = key_instantiate_and_link(keyring, NULL, 0, dest, NULL);
if (ret < 0) {
@@ -510,6 +514,26 @@
}
EXPORT_SYMBOL(keyring_alloc);
+/**
+ * restrict_link_reject - Give -EPERM to restrict link
+ * @keyring: The keyring being added to.
+ * @type: The type of key being added.
+ * @payload: The payload of the key intended to be added.
+ *
+ * Reject the addition of any links to a keyring. It can be overridden by
+ * passing KEY_ALLOC_BYPASS_RESTRICTION to key_instantiate_and_link() when
+ * adding a key to a keyring.
+ *
+ * This is meant to be passed as the restrict_link parameter to
+ * keyring_alloc().
+ */
+int restrict_link_reject(struct key *keyring,
+ const struct key_type *type,
+ const union key_payload *payload)
+{
+ return -EPERM;
+}
+
/*
* By default, we keys found by getting an exact match on their descriptions.
*/
@@ -1191,6 +1215,16 @@
up_write(&keyring->sem);
}
+/*
+ * Check addition of keys to restricted keyrings.
+ */
+static int __key_link_check_restriction(struct key *keyring, struct key *key)
+{
+ if (!keyring->restrict_link)
+ return 0;
+ return keyring->restrict_link(keyring, key->type, &key->payload);
+}
+
/**
* key_link - Link a key to a keyring
* @keyring: The keyring to make the link in.
@@ -1221,14 +1255,12 @@
key_check(keyring);
key_check(key);
- if (test_bit(KEY_FLAG_TRUSTED_ONLY, &keyring->flags) &&
- !test_bit(KEY_FLAG_TRUSTED, &key->flags))
- return -EPERM;
-
ret = __key_link_begin(keyring, &key->index_key, &edit);
if (ret == 0) {
kdebug("begun {%d,%d}", keyring->serial, atomic_read(&keyring->usage));
- ret = __key_link_check_live_key(keyring, key);
+ ret = __key_link_check_restriction(keyring, key);
+ if (ret == 0)
+ ret = __key_link_check_live_key(keyring, key);
if (ret == 0)
__key_link(key, &edit);
__key_link_end(keyring, &key->index_key, edit);
diff --git a/security/keys/persistent.c b/security/keys/persistent.c
index c9fae5e..2ef45b3 100644
--- a/security/keys/persistent.c
+++ b/security/keys/persistent.c
@@ -26,7 +26,7 @@
current_cred(),
((KEY_POS_ALL & ~KEY_POS_SETATTR) |
KEY_USR_VIEW | KEY_USR_READ),
- KEY_ALLOC_NOT_IN_QUOTA, NULL);
+ KEY_ALLOC_NOT_IN_QUOTA, NULL, NULL);
if (IS_ERR(reg))
return PTR_ERR(reg);
@@ -60,7 +60,7 @@
uid, INVALID_GID, current_cred(),
((KEY_POS_ALL & ~KEY_POS_SETATTR) |
KEY_USR_VIEW | KEY_USR_READ),
- KEY_ALLOC_NOT_IN_QUOTA,
+ KEY_ALLOC_NOT_IN_QUOTA, NULL,
ns->persistent_keyring_register);
if (IS_ERR(persistent))
return ERR_CAST(persistent);
diff --git a/security/keys/process_keys.c b/security/keys/process_keys.c
index e6d50172..40a88523 100644
--- a/security/keys/process_keys.c
+++ b/security/keys/process_keys.c
@@ -76,7 +76,8 @@
if (IS_ERR(uid_keyring)) {
uid_keyring = keyring_alloc(buf, user->uid, INVALID_GID,
cred, user_keyring_perm,
- KEY_ALLOC_IN_QUOTA, NULL);
+ KEY_ALLOC_IN_QUOTA,
+ NULL, NULL);
if (IS_ERR(uid_keyring)) {
ret = PTR_ERR(uid_keyring);
goto error;
@@ -92,7 +93,8 @@
session_keyring =
keyring_alloc(buf, user->uid, INVALID_GID,
cred, user_keyring_perm,
- KEY_ALLOC_IN_QUOTA, NULL);
+ KEY_ALLOC_IN_QUOTA,
+ NULL, NULL);
if (IS_ERR(session_keyring)) {
ret = PTR_ERR(session_keyring);
goto error_release;
@@ -134,7 +136,8 @@
keyring = keyring_alloc("_tid", new->uid, new->gid, new,
KEY_POS_ALL | KEY_USR_VIEW,
- KEY_ALLOC_QUOTA_OVERRUN, NULL);
+ KEY_ALLOC_QUOTA_OVERRUN,
+ NULL, NULL);
if (IS_ERR(keyring))
return PTR_ERR(keyring);
@@ -180,7 +183,8 @@
keyring = keyring_alloc("_pid", new->uid, new->gid, new,
KEY_POS_ALL | KEY_USR_VIEW,
- KEY_ALLOC_QUOTA_OVERRUN, NULL);
+ KEY_ALLOC_QUOTA_OVERRUN,
+ NULL, NULL);
if (IS_ERR(keyring))
return PTR_ERR(keyring);
@@ -231,7 +235,7 @@
keyring = keyring_alloc("_ses", cred->uid, cred->gid, cred,
KEY_POS_ALL | KEY_USR_VIEW | KEY_USR_READ,
- flags, NULL);
+ flags, NULL, NULL);
if (IS_ERR(keyring))
return PTR_ERR(keyring);
} else {
@@ -785,7 +789,7 @@
keyring = keyring_alloc(
name, old->uid, old->gid, old,
KEY_POS_ALL | KEY_USR_VIEW | KEY_USR_READ | KEY_USR_LINK,
- KEY_ALLOC_IN_QUOTA, NULL);
+ KEY_ALLOC_IN_QUOTA, NULL, NULL);
if (IS_ERR(keyring)) {
ret = PTR_ERR(keyring);
goto error2;
diff --git a/security/keys/request_key.c b/security/keys/request_key.c
index c7a117c..a29e355 100644
--- a/security/keys/request_key.c
+++ b/security/keys/request_key.c
@@ -116,7 +116,7 @@
cred = get_current_cred();
keyring = keyring_alloc(desc, cred->fsuid, cred->fsgid, cred,
KEY_POS_ALL | KEY_USR_VIEW | KEY_USR_READ,
- KEY_ALLOC_QUOTA_OVERRUN, NULL);
+ KEY_ALLOC_QUOTA_OVERRUN, NULL, NULL);
put_cred(cred);
if (IS_ERR(keyring)) {
ret = PTR_ERR(keyring);
@@ -355,7 +355,7 @@
key = key_alloc(ctx->index_key.type, ctx->index_key.description,
ctx->cred->fsuid, ctx->cred->fsgid, ctx->cred,
- perm, flags);
+ perm, flags, NULL);
if (IS_ERR(key))
goto alloc_failed;
diff --git a/security/keys/request_key_auth.c b/security/keys/request_key_auth.c
index 4f0f112..9db8b4a 100644
--- a/security/keys/request_key_auth.c
+++ b/security/keys/request_key_auth.c
@@ -202,7 +202,7 @@
authkey = key_alloc(&key_type_request_key_auth, desc,
cred->fsuid, cred->fsgid, cred,
KEY_POS_VIEW | KEY_POS_READ | KEY_POS_SEARCH |
- KEY_USR_VIEW, KEY_ALLOC_NOT_IN_QUOTA);
+ KEY_USR_VIEW, KEY_ALLOC_NOT_IN_QUOTA, NULL);
if (IS_ERR(authkey)) {
ret = PTR_ERR(authkey);
goto error_alloc;
diff --git a/security/keys/user_defined.c b/security/keys/user_defined.c
index 8705d79..66b1840 100644
--- a/security/keys/user_defined.c
+++ b/security/keys/user_defined.c
@@ -96,45 +96,25 @@
*/
int user_update(struct key *key, struct key_preparsed_payload *prep)
{
- struct user_key_payload *upayload, *zap;
- size_t datalen = prep->datalen;
+ struct user_key_payload *zap = NULL;
int ret;
- ret = -EINVAL;
- if (datalen <= 0 || datalen > 32767 || !prep->data)
- goto error;
-
- /* construct a replacement payload */
- ret = -ENOMEM;
- upayload = kmalloc(sizeof(*upayload) + datalen, GFP_KERNEL);
- if (!upayload)
- goto error;
-
- upayload->datalen = datalen;
- memcpy(upayload->data, prep->data, datalen);
-
/* check the quota and attach the new data */
- zap = upayload;
+ ret = key_payload_reserve(key, prep->datalen);
+ if (ret < 0)
+ return ret;
- ret = key_payload_reserve(key, datalen);
-
- if (ret == 0) {
- /* attach the new data, displacing the old */
- if (!test_bit(KEY_FLAG_NEGATIVE, &key->flags))
- zap = key->payload.data[0];
- else
- zap = NULL;
- rcu_assign_keypointer(key, upayload);
- key->expiry = 0;
- }
+ /* attach the new data, displacing the old */
+ key->expiry = prep->expiry;
+ if (!test_bit(KEY_FLAG_NEGATIVE, &key->flags))
+ zap = rcu_dereference_key(key);
+ rcu_assign_keypointer(key, prep->payload.data[0]);
+ prep->payload.data[0] = NULL;
if (zap)
kfree_rcu(zap, rcu);
-
-error:
return ret;
}
-
EXPORT_SYMBOL_GPL(user_update);
/*
diff --git a/security/loadpin/Kconfig b/security/loadpin/Kconfig
new file mode 100644
index 0000000..dd01aa9
--- /dev/null
+++ b/security/loadpin/Kconfig
@@ -0,0 +1,19 @@
+config SECURITY_LOADPIN
+ bool "Pin load of kernel files (modules, fw, etc) to one filesystem"
+ depends on SECURITY && BLOCK
+ help
+ Any files read through the kernel file reading interface
+ (kernel modules, firmware, kexec images, security policy)
+ can be pinned to the first filesystem used for loading. When
+ enabled, any files that come from other filesystems will be
+ rejected. This is best used on systems without an initrd that
+ have a root filesystem backed by a read-only device such as
+ dm-verity or a CDROM.
+
+config SECURITY_LOADPIN_ENABLED
+ bool "Enforce LoadPin at boot"
+ depends on SECURITY_LOADPIN
+ help
+ If selected, LoadPin will enforce pinning at boot. If not
+ selected, it can be enabled at boot with the kernel parameter
+ "loadpin.enabled=1".
diff --git a/security/loadpin/Makefile b/security/loadpin/Makefile
new file mode 100644
index 0000000..c2d77f8
--- /dev/null
+++ b/security/loadpin/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_SECURITY_LOADPIN) += loadpin.o
diff --git a/security/loadpin/loadpin.c b/security/loadpin/loadpin.c
new file mode 100644
index 0000000..89a46f1
--- /dev/null
+++ b/security/loadpin/loadpin.c
@@ -0,0 +1,190 @@
+/*
+ * Module and Firmware Pinning Security Module
+ *
+ * Copyright 2011-2016 Google Inc.
+ *
+ * Author: Kees Cook <keescook@chromium.org>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#define pr_fmt(fmt) "LoadPin: " fmt
+
+#include <linux/module.h>
+#include <linux/fs.h>
+#include <linux/fs_struct.h>
+#include <linux/lsm_hooks.h>
+#include <linux/mount.h>
+#include <linux/path.h>
+#include <linux/sched.h> /* current */
+#include <linux/string_helpers.h>
+
+static void report_load(const char *origin, struct file *file, char *operation)
+{
+ char *cmdline, *pathname;
+
+ pathname = kstrdup_quotable_file(file, GFP_KERNEL);
+ cmdline = kstrdup_quotable_cmdline(current, GFP_KERNEL);
+
+ pr_notice("%s %s obj=%s%s%s pid=%d cmdline=%s%s%s\n",
+ origin, operation,
+ (pathname && pathname[0] != '<') ? "\"" : "",
+ pathname,
+ (pathname && pathname[0] != '<') ? "\"" : "",
+ task_pid_nr(current),
+ cmdline ? "\"" : "", cmdline, cmdline ? "\"" : "");
+
+ kfree(cmdline);
+ kfree(pathname);
+}
+
+static int enabled = IS_ENABLED(CONFIG_SECURITY_LOADPIN_ENABLED);
+static struct super_block *pinned_root;
+static DEFINE_SPINLOCK(pinned_root_spinlock);
+
+#ifdef CONFIG_SYSCTL
+static int zero;
+static int one = 1;
+
+static struct ctl_path loadpin_sysctl_path[] = {
+ { .procname = "kernel", },
+ { .procname = "loadpin", },
+ { }
+};
+
+static struct ctl_table loadpin_sysctl_table[] = {
+ {
+ .procname = "enabled",
+ .data = &enabled,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec_minmax,
+ .extra1 = &zero,
+ .extra2 = &one,
+ },
+ { }
+};
+
+/*
+ * This must be called after early kernel init, since then the rootdev
+ * is available.
+ */
+static void check_pinning_enforcement(struct super_block *mnt_sb)
+{
+ bool ro = false;
+
+ /*
+ * If load pinning is not enforced via a read-only block
+ * device, allow sysctl to change modes for testing.
+ */
+ if (mnt_sb->s_bdev) {
+ ro = bdev_read_only(mnt_sb->s_bdev);
+ pr_info("dev(%u,%u): %s\n",
+ MAJOR(mnt_sb->s_bdev->bd_dev),
+ MINOR(mnt_sb->s_bdev->bd_dev),
+ ro ? "read-only" : "writable");
+ } else
+ pr_info("mnt_sb lacks block device, treating as: writable\n");
+
+ if (!ro) {
+ if (!register_sysctl_paths(loadpin_sysctl_path,
+ loadpin_sysctl_table))
+ pr_notice("sysctl registration failed!\n");
+ else
+ pr_info("load pinning can be disabled.\n");
+ } else
+ pr_info("load pinning engaged.\n");
+}
+#else
+static void check_pinning_enforcement(struct super_block *mnt_sb)
+{
+ pr_info("load pinning engaged.\n");
+}
+#endif
+
+static void loadpin_sb_free_security(struct super_block *mnt_sb)
+{
+ /*
+ * When unmounting the filesystem we were using for load
+ * pinning, we acknowledge the superblock release, but make sure
+ * no other modules or firmware can be loaded.
+ */
+ if (!IS_ERR_OR_NULL(pinned_root) && mnt_sb == pinned_root) {
+ pinned_root = ERR_PTR(-EIO);
+ pr_info("umount pinned fs: refusing further loads\n");
+ }
+}
+
+static int loadpin_read_file(struct file *file, enum kernel_read_file_id id)
+{
+ struct super_block *load_root;
+ const char *origin = kernel_read_file_id_str(id);
+
+ /* This handles the older init_module API that has a NULL file. */
+ if (!file) {
+ if (!enabled) {
+ report_load(origin, NULL, "old-api-pinning-ignored");
+ return 0;
+ }
+
+ report_load(origin, NULL, "old-api-denied");
+ return -EPERM;
+ }
+
+ load_root = file->f_path.mnt->mnt_sb;
+
+ /* First loaded module/firmware defines the root for all others. */
+ spin_lock(&pinned_root_spinlock);
+ /*
+ * pinned_root is only NULL at startup. Otherwise, it is either
+ * a valid reference, or an ERR_PTR.
+ */
+ if (!pinned_root) {
+ pinned_root = load_root;
+ /*
+ * Unlock now since it's only pinned_root we care about.
+ * In the worst case, we will (correctly) report pinning
+ * failures before we have announced that pinning is
+ * enabled. This would be purely cosmetic.
+ */
+ spin_unlock(&pinned_root_spinlock);
+ check_pinning_enforcement(pinned_root);
+ report_load(origin, file, "pinned");
+ } else {
+ spin_unlock(&pinned_root_spinlock);
+ }
+
+ if (IS_ERR_OR_NULL(pinned_root) || load_root != pinned_root) {
+ if (unlikely(!enabled)) {
+ report_load(origin, file, "pinning-ignored");
+ return 0;
+ }
+
+ report_load(origin, file, "denied");
+ return -EPERM;
+ }
+
+ return 0;
+}
+
+static struct security_hook_list loadpin_hooks[] = {
+ LSM_HOOK_INIT(sb_free_security, loadpin_sb_free_security),
+ LSM_HOOK_INIT(kernel_read_file, loadpin_read_file),
+};
+
+void __init loadpin_add_hooks(void)
+{
+ pr_info("ready to pin (currently %sabled)", enabled ? "en" : "dis");
+ security_add_hooks(loadpin_hooks, ARRAY_SIZE(loadpin_hooks));
+}
+
+/* Should not be mutable after boot, so not listed in sysfs (perm == 0). */
+module_param(enabled, int, 0);
+MODULE_PARM_DESC(enabled, "Pin module/firmware loading (default: true)");
diff --git a/security/security.c b/security/security.c
index d17e4a6..7095693 100644
--- a/security/security.c
+++ b/security/security.c
@@ -60,6 +60,7 @@
*/
capability_add_hooks();
yama_add_hooks();
+ loadpin_add_hooks();
/*
* Load all the remaining security modules.
@@ -1848,7 +1849,6 @@
.tun_dev_attach =
LIST_HEAD_INIT(security_hook_heads.tun_dev_attach),
.tun_dev_open = LIST_HEAD_INIT(security_hook_heads.tun_dev_open),
- .skb_owned_by = LIST_HEAD_INIT(security_hook_heads.skb_owned_by),
#endif /* CONFIG_SECURITY_NETWORK */
#ifdef CONFIG_SECURITY_NETWORK_XFRM
.xfrm_policy_alloc_security =
diff --git a/security/selinux/hooks.c b/security/selinux/hooks.c
index 3140efa7..a86d537 100644
--- a/security/selinux/hooks.c
+++ b/security/selinux/hooks.c
@@ -259,7 +259,7 @@
might_sleep_if(may_sleep);
- if (isec->initialized == LABEL_INVALID) {
+ if (ss_initialized && isec->initialized != LABEL_INITIALIZED) {
if (!may_sleep)
return -ECHILD;
@@ -297,6 +297,13 @@
return inode->i_security;
}
+static struct inode_security_struct *backing_inode_security_novalidate(struct dentry *dentry)
+{
+ struct inode *inode = d_backing_inode(dentry);
+
+ return inode->i_security;
+}
+
/*
* Get the security label of a dentry's backing inode.
*/
@@ -687,7 +694,7 @@
struct superblock_security_struct *sbsec = sb->s_security;
const char *name = sb->s_type->name;
struct dentry *root = sbsec->sb->s_root;
- struct inode_security_struct *root_isec = backing_inode_security(root);
+ struct inode_security_struct *root_isec;
u32 fscontext_sid = 0, context_sid = 0, rootcontext_sid = 0;
u32 defcontext_sid = 0;
char **mount_options = opts->mnt_opts;
@@ -730,6 +737,8 @@
&& (num_opts == 0))
goto out;
+ root_isec = backing_inode_security_novalidate(root);
+
/*
* parse the mount options, check if they are valid sids.
* also check if someone is trying to mount the same sb more
@@ -1623,7 +1632,7 @@
/* Check whether a task is allowed to use a capability. */
static int cred_has_capability(const struct cred *cred,
- int cap, int audit)
+ int cap, int audit, bool initns)
{
struct common_audit_data ad;
struct av_decision avd;
@@ -1637,10 +1646,10 @@
switch (CAP_TO_INDEX(cap)) {
case 0:
- sclass = SECCLASS_CAPABILITY;
+ sclass = initns ? SECCLASS_CAPABILITY : SECCLASS_CAP_USERNS;
break;
case 1:
- sclass = SECCLASS_CAPABILITY2;
+ sclass = initns ? SECCLASS_CAPABILITY2 : SECCLASS_CAP2_USERNS;
break;
default:
printk(KERN_ERR
@@ -1782,7 +1791,6 @@
u32 *_new_isid)
{
const struct superblock_security_struct *sbsec = dir->i_sb->s_security;
- const struct inode_security_struct *dsec = inode_security(dir);
const struct task_security_struct *tsec = current_security();
if ((sbsec->flags & SE_SBINITIALIZED) &&
@@ -1792,6 +1800,7 @@
tsec->create_sid) {
*_new_isid = tsec->create_sid;
} else {
+ const struct inode_security_struct *dsec = inode_security(dir);
return security_transition_sid(tsec->sid, dsec->sid, tclass,
name, _new_isid);
}
@@ -2076,7 +2085,7 @@
u32 sid = task_sid(to);
struct file_security_struct *fsec = file->f_security;
struct dentry *dentry = file->f_path.dentry;
- struct inode_security_struct *isec = backing_inode_security(dentry);
+ struct inode_security_struct *isec;
struct common_audit_data ad;
int rc;
@@ -2095,6 +2104,7 @@
if (unlikely(IS_PRIVATE(d_backing_inode(dentry))))
return 0;
+ isec = backing_inode_security(dentry);
return avc_has_perm(sid, isec->sid, isec->sclass, file_to_av(file),
&ad);
}
@@ -2143,7 +2153,7 @@
static int selinux_capable(const struct cred *cred, struct user_namespace *ns,
int cap, int audit)
{
- return cred_has_capability(cred, cap, audit);
+ return cred_has_capability(cred, cap, audit, ns == &init_user_ns);
}
static int selinux_quotactl(int cmds, int type, int id, struct super_block *sb)
@@ -2221,7 +2231,7 @@
int rc, cap_sys_admin = 0;
rc = cred_has_capability(current_cred(), CAP_SYS_ADMIN,
- SECURITY_CAP_NOAUDIT);
+ SECURITY_CAP_NOAUDIT, true);
if (rc == 0)
cap_sys_admin = 1;
@@ -2230,6 +2240,20 @@
/* binprm security operations */
+static u32 ptrace_parent_sid(struct task_struct *task)
+{
+ u32 sid = 0;
+ struct task_struct *tracer;
+
+ rcu_read_lock();
+ tracer = ptrace_parent(task);
+ if (tracer)
+ sid = task_sid(tracer);
+ rcu_read_unlock();
+
+ return sid;
+}
+
static int check_nnp_nosuid(const struct linux_binprm *bprm,
const struct task_security_struct *old_tsec,
const struct task_security_struct *new_tsec)
@@ -2351,18 +2375,7 @@
* changes its SID has the appropriate permit */
if (bprm->unsafe &
(LSM_UNSAFE_PTRACE | LSM_UNSAFE_PTRACE_CAP)) {
- struct task_struct *tracer;
- struct task_security_struct *sec;
- u32 ptsid = 0;
-
- rcu_read_lock();
- tracer = ptrace_parent(current);
- if (likely(tracer != NULL)) {
- sec = __task_cred(tracer)->security;
- ptsid = sec->sid;
- }
- rcu_read_unlock();
-
+ u32 ptsid = ptrace_parent_sid(current);
if (ptsid != 0) {
rc = avc_has_perm(ptsid, new_tsec->sid,
SECCLASS_PROCESS,
@@ -3046,7 +3059,7 @@
const void *value, size_t size, int flags)
{
struct inode *inode = d_backing_inode(dentry);
- struct inode_security_struct *isec = backing_inode_security(dentry);
+ struct inode_security_struct *isec;
struct superblock_security_struct *sbsec;
struct common_audit_data ad;
u32 newsid, sid = current_sid();
@@ -3065,6 +3078,7 @@
ad.type = LSM_AUDIT_DATA_DENTRY;
ad.u.dentry = dentry;
+ isec = backing_inode_security(dentry);
rc = avc_has_perm(sid, isec->sid, isec->sclass,
FILE__RELABELFROM, &ad);
if (rc)
@@ -3123,7 +3137,7 @@
int flags)
{
struct inode *inode = d_backing_inode(dentry);
- struct inode_security_struct *isec = backing_inode_security(dentry);
+ struct inode_security_struct *isec;
u32 newsid;
int rc;
@@ -3140,6 +3154,7 @@
return;
}
+ isec = backing_inode_security(dentry);
isec->sclass = inode_mode_to_security_class(inode->i_mode);
isec->sid = newsid;
isec->initialized = LABEL_INITIALIZED;
@@ -3181,7 +3196,7 @@
u32 size;
int error;
char *context = NULL;
- struct inode_security_struct *isec = inode_security(inode);
+ struct inode_security_struct *isec;
if (strcmp(name, XATTR_SELINUX_SUFFIX))
return -EOPNOTSUPP;
@@ -3199,7 +3214,8 @@
SECURITY_CAP_NOAUDIT);
if (!error)
error = cred_has_capability(current_cred(), CAP_MAC_ADMIN,
- SECURITY_CAP_NOAUDIT);
+ SECURITY_CAP_NOAUDIT, true);
+ isec = inode_security(inode);
if (!error)
error = security_sid_to_context_force(isec->sid, &context,
&size);
@@ -3220,7 +3236,7 @@
static int selinux_inode_setsecurity(struct inode *inode, const char *name,
const void *value, size_t size, int flags)
{
- struct inode_security_struct *isec = inode_security(inode);
+ struct inode_security_struct *isec = inode_security_novalidate(inode);
u32 newsid;
int rc;
@@ -3309,7 +3325,7 @@
struct common_audit_data ad;
struct file_security_struct *fsec = file->f_security;
struct inode *inode = file_inode(file);
- struct inode_security_struct *isec = inode_security(inode);
+ struct inode_security_struct *isec;
struct lsm_ioctlop_audit ioctl;
u32 ssid = cred_sid(cred);
int rc;
@@ -3333,6 +3349,7 @@
if (unlikely(IS_PRIVATE(inode)))
return 0;
+ isec = inode_security(inode);
rc = avc_has_extended_perms(ssid, isec->sid, isec->sclass,
requested, driver, xperm, &ad);
out:
@@ -3374,7 +3391,7 @@
case KDSKBENT:
case KDSKBSENT:
error = cred_has_capability(cred, CAP_SYS_TTY_CONFIG,
- SECURITY_CAP_AUDIT);
+ SECURITY_CAP_AUDIT, true);
break;
/* default case assumes that the command will go
@@ -3463,8 +3480,9 @@
vma->vm_end <= vma->vm_mm->brk) {
rc = cred_has_perm(cred, cred, PROCESS__EXECHEAP);
} else if (!vma->vm_file &&
- vma->vm_start <= vma->vm_mm->start_stack &&
- vma->vm_end >= vma->vm_mm->start_stack) {
+ ((vma->vm_start <= vma->vm_mm->start_stack &&
+ vma->vm_end >= vma->vm_mm->start_stack) ||
+ vma_is_stack_for_task(vma, current))) {
rc = current_has_perm(current, PROCESS__EXECSTACK);
} else if (vma->vm_file && vma->anon_vma) {
/*
@@ -3720,6 +3738,52 @@
SYSTEM__MODULE_REQUEST, &ad);
}
+static int selinux_kernel_module_from_file(struct file *file)
+{
+ struct common_audit_data ad;
+ struct inode_security_struct *isec;
+ struct file_security_struct *fsec;
+ u32 sid = current_sid();
+ int rc;
+
+ /* init_module */
+ if (file == NULL)
+ return avc_has_perm(sid, sid, SECCLASS_SYSTEM,
+ SYSTEM__MODULE_LOAD, NULL);
+
+ /* finit_module */
+
+ ad.type = LSM_AUDIT_DATA_PATH;
+ ad.u.path = file->f_path;
+
+ fsec = file->f_security;
+ if (sid != fsec->sid) {
+ rc = avc_has_perm(sid, fsec->sid, SECCLASS_FD, FD__USE, &ad);
+ if (rc)
+ return rc;
+ }
+
+ isec = inode_security(file_inode(file));
+ return avc_has_perm(sid, isec->sid, SECCLASS_SYSTEM,
+ SYSTEM__MODULE_LOAD, &ad);
+}
+
+static int selinux_kernel_read_file(struct file *file,
+ enum kernel_read_file_id id)
+{
+ int rc = 0;
+
+ switch (id) {
+ case READING_MODULE:
+ rc = selinux_kernel_module_from_file(file);
+ break;
+ default:
+ break;
+ }
+
+ return rc;
+}
+
static int selinux_task_setpgid(struct task_struct *p, pid_t pgid)
{
return current_has_perm(p, PROCESS__SETPGID);
@@ -4599,6 +4663,7 @@
{
u32 peer_secid = SECSID_NULL;
u16 family;
+ struct inode_security_struct *isec;
if (skb && skb->protocol == htons(ETH_P_IP))
family = PF_INET;
@@ -4609,9 +4674,10 @@
else
goto out;
- if (sock && family == PF_UNIX)
- selinux_inode_getsecid(SOCK_INODE(sock), &peer_secid);
- else if (skb)
+ if (sock && family == PF_UNIX) {
+ isec = inode_security_novalidate(SOCK_INODE(sock));
+ peer_secid = isec->sid;
+ } else if (skb)
selinux_skb_peerlbl_sid(skb, family, &peer_secid);
out:
@@ -5676,7 +5742,6 @@
char *name, void *value, size_t size)
{
struct task_security_struct *tsec;
- struct task_struct *tracer;
struct cred *new;
u32 sid = 0, ptsid;
int error;
@@ -5783,14 +5848,8 @@
/* Check for ptracing, and update the task SID if ok.
Otherwise, leave SID unchanged and fail. */
- ptsid = 0;
- rcu_read_lock();
- tracer = ptrace_parent(p);
- if (tracer)
- ptsid = task_sid(tracer);
- rcu_read_unlock();
-
- if (tracer) {
+ ptsid = ptrace_parent_sid(p);
+ if (ptsid != 0) {
error = avc_has_perm(ptsid, sid, SECCLASS_PROCESS,
PROCESS__PTRACE, NULL);
if (error)
@@ -6021,6 +6080,7 @@
LSM_HOOK_INIT(kernel_act_as, selinux_kernel_act_as),
LSM_HOOK_INIT(kernel_create_files_as, selinux_kernel_create_files_as),
LSM_HOOK_INIT(kernel_module_request, selinux_kernel_module_request),
+ LSM_HOOK_INIT(kernel_read_file, selinux_kernel_read_file),
LSM_HOOK_INIT(task_setpgid, selinux_task_setpgid),
LSM_HOOK_INIT(task_getpgid, selinux_task_getpgid),
LSM_HOOK_INIT(task_getsid, selinux_task_getsid),
diff --git a/security/selinux/include/classmap.h b/security/selinux/include/classmap.h
index ef83c4b..1f1f4b2 100644
--- a/security/selinux/include/classmap.h
+++ b/security/selinux/include/classmap.h
@@ -12,6 +12,18 @@
#define COMMON_IPC_PERMS "create", "destroy", "getattr", "setattr", "read", \
"write", "associate", "unix_read", "unix_write"
+#define COMMON_CAP_PERMS "chown", "dac_override", "dac_read_search", \
+ "fowner", "fsetid", "kill", "setgid", "setuid", "setpcap", \
+ "linux_immutable", "net_bind_service", "net_broadcast", \
+ "net_admin", "net_raw", "ipc_lock", "ipc_owner", "sys_module", \
+ "sys_rawio", "sys_chroot", "sys_ptrace", "sys_pacct", "sys_admin", \
+ "sys_boot", "sys_nice", "sys_resource", "sys_time", \
+ "sys_tty_config", "mknod", "lease", "audit_write", \
+ "audit_control", "setfcap"
+
+#define COMMON_CAP2_PERMS "mac_override", "mac_admin", "syslog", \
+ "wake_alarm", "block_suspend", "audit_read"
+
/*
* Note: The name for any socket class should be suffixed by "socket",
* and doesn't contain more than one substr of "socket".
@@ -32,16 +44,9 @@
"setsockcreate", NULL } },
{ "system",
{ "ipc_info", "syslog_read", "syslog_mod",
- "syslog_console", "module_request", NULL } },
+ "syslog_console", "module_request", "module_load", NULL } },
{ "capability",
- { "chown", "dac_override", "dac_read_search",
- "fowner", "fsetid", "kill", "setgid", "setuid", "setpcap",
- "linux_immutable", "net_bind_service", "net_broadcast",
- "net_admin", "net_raw", "ipc_lock", "ipc_owner", "sys_module",
- "sys_rawio", "sys_chroot", "sys_ptrace", "sys_pacct", "sys_admin",
- "sys_boot", "sys_nice", "sys_resource", "sys_time",
- "sys_tty_config", "mknod", "lease", "audit_write",
- "audit_control", "setfcap", NULL } },
+ { COMMON_CAP_PERMS, NULL } },
{ "filesystem",
{ "mount", "remount", "unmount", "getattr",
"relabelfrom", "relabelto", "associate", "quotamod",
@@ -150,12 +155,15 @@
{ "memprotect", { "mmap_zero", NULL } },
{ "peer", { "recv", NULL } },
{ "capability2",
- { "mac_override", "mac_admin", "syslog", "wake_alarm", "block_suspend",
- "audit_read", NULL } },
+ { COMMON_CAP2_PERMS, NULL } },
{ "kernel_service", { "use_as_override", "create_files_as", NULL } },
{ "tun_socket",
{ COMMON_SOCK_PERMS, "attach_queue", NULL } },
{ "binder", { "impersonate", "call", "set_context_mgr", "transfer",
NULL } },
+ { "cap_userns",
+ { COMMON_CAP_PERMS, NULL } },
+ { "cap2_userns",
+ { COMMON_CAP2_PERMS, NULL } },
{ NULL }
};
diff --git a/security/selinux/include/conditional.h b/security/selinux/include/conditional.h
index 67ce7a8..ff4fddc 100644
--- a/security/selinux/include/conditional.h
+++ b/security/selinux/include/conditional.h
@@ -17,6 +17,6 @@
int security_set_bools(int len, int *values);
-int security_get_bool_value(int bool);
+int security_get_bool_value(int index);
#endif
diff --git a/security/selinux/include/objsec.h b/security/selinux/include/objsec.h
index a2ae054..c21e135 100644
--- a/security/selinux/include/objsec.h
+++ b/security/selinux/include/objsec.h
@@ -38,9 +38,8 @@
};
enum label_initialized {
- LABEL_MISSING, /* not initialized */
- LABEL_INITIALIZED, /* inizialized */
- LABEL_INVALID /* invalid */
+ LABEL_INVALID, /* invalid or not initialized */
+ LABEL_INITIALIZED /* initialized */
};
struct inode_security_struct {
diff --git a/security/selinux/ss/services.c b/security/selinux/ss/services.c
index ebda973..89df646 100644
--- a/security/selinux/ss/services.c
+++ b/security/selinux/ss/services.c
@@ -2696,7 +2696,7 @@
return rc;
}
-int security_get_bool_value(int bool)
+int security_get_bool_value(int index)
{
int rc;
int len;
@@ -2705,10 +2705,10 @@
rc = -EFAULT;
len = policydb.p_bools.nprim;
- if (bool >= len)
+ if (index >= len)
goto out;
- rc = policydb.bool_val_to_struct[bool]->state;
+ rc = policydb.bool_val_to_struct[index]->state;
out:
read_unlock(&policy_rwlock);
return rc;
diff --git a/security/yama/yama_lsm.c b/security/yama/yama_lsm.c
index cb6ed10..9b756b1 100644
--- a/security/yama/yama_lsm.c
+++ b/security/yama/yama_lsm.c
@@ -18,6 +18,7 @@
#include <linux/prctl.h>
#include <linux/ratelimit.h>
#include <linux/workqueue.h>
+#include <linux/string_helpers.h>
#define YAMA_SCOPE_DISABLED 0
#define YAMA_SCOPE_RELATIONAL 1
@@ -41,6 +42,22 @@
static void yama_relation_cleanup(struct work_struct *work);
static DECLARE_WORK(yama_relation_work, yama_relation_cleanup);
+static void report_access(const char *access, struct task_struct *target,
+ struct task_struct *agent)
+{
+ char *target_cmd, *agent_cmd;
+
+ target_cmd = kstrdup_quotable_cmdline(target, GFP_ATOMIC);
+ agent_cmd = kstrdup_quotable_cmdline(agent, GFP_ATOMIC);
+
+ pr_notice_ratelimited(
+ "ptrace %s of \"%s\"[%d] was attempted by \"%s\"[%d]\n",
+ access, target_cmd, target->pid, agent_cmd, agent->pid);
+
+ kfree(agent_cmd);
+ kfree(target_cmd);
+}
+
/**
* yama_relation_cleanup - remove invalid entries from the relation list
*
@@ -307,11 +324,8 @@
}
}
- if (rc && (mode & PTRACE_MODE_NOAUDIT) == 0) {
- printk_ratelimited(KERN_NOTICE
- "ptrace of pid %d was attempted by: %s (pid %d)\n",
- child->pid, current->comm, current->pid);
- }
+ if (rc && (mode & PTRACE_MODE_NOAUDIT) == 0)
+ report_access("attach", child, current);
return rc;
}
@@ -337,11 +351,8 @@
break;
}
- if (rc) {
- printk_ratelimited(KERN_NOTICE
- "ptraceme of pid %d was attempted by: %s (pid %d)\n",
- current->pid, parent->comm, parent->pid);
- }
+ if (rc)
+ report_access("traceme", current, parent);
return rc;
}
diff --git a/sound/core/Kconfig b/sound/core/Kconfig
index 6d12ca9..9749f9e 100644
--- a/sound/core/Kconfig
+++ b/sound/core/Kconfig
@@ -141,35 +141,6 @@
Say Y here to use the HR-timer backend as the default sequencer
timer.
-config SND_RTCTIMER
- tristate "RTC Timer support"
- depends on RTC
- select SND_TIMER
- help
- Say Y here to enable RTC timer support for ALSA. ALSA uses
- the RTC timer as a precise timing source and maps the RTC
- timer to ALSA's timer interface. The ALSA sequencer code also
- can use this timing source.
-
- To compile this driver as a module, choose M here: the module
- will be called snd-rtctimer.
-
- Note that this option is exclusive with the new RTC drivers
- (CONFIG_RTC_CLASS) since this requires the old API.
-
-config SND_SEQ_RTCTIMER_DEFAULT
- bool "Use RTC as default sequencer timer"
- depends on SND_RTCTIMER && SND_SEQUENCER
- depends on !SND_SEQ_HRTIMER_DEFAULT
- default y
- help
- Say Y here to use the RTC timer as the default sequencer
- timer. This is strongly recommended because it ensures
- precise MIDI timing even when the system timer runs at less
- than 1000 Hz.
-
- If in doubt, say Y.
-
config SND_DYNAMIC_MINORS
bool "Dynamic device file minor numbers"
help
diff --git a/sound/core/Makefile b/sound/core/Makefile
index 48ab4b8f8..e85d9dd 100644
--- a/sound/core/Makefile
+++ b/sound/core/Makefile
@@ -37,7 +37,6 @@
obj-$(CONFIG_SND_HWDEP) += snd-hwdep.o
obj-$(CONFIG_SND_TIMER) += snd-timer.o
obj-$(CONFIG_SND_HRTIMER) += snd-hrtimer.o
-obj-$(CONFIG_SND_RTCTIMER) += snd-rtctimer.o
obj-$(CONFIG_SND_PCM) += snd-pcm.o
obj-$(CONFIG_SND_DMAENGINE_PCM) += snd-pcm-dmaengine.o
obj-$(CONFIG_SND_RAWMIDI) += snd-rawmidi.o
diff --git a/sound/core/compress_offload.c b/sound/core/compress_offload.c
index a9933c0..9b3334b 100644
--- a/sound/core/compress_offload.c
+++ b/sound/core/compress_offload.c
@@ -288,9 +288,12 @@
stream = &data->stream;
mutex_lock(&stream->device->lock);
/* write is allowed when stream is running or has been steup */
- if (stream->runtime->state != SNDRV_PCM_STATE_SETUP &&
- stream->runtime->state != SNDRV_PCM_STATE_PREPARED &&
- stream->runtime->state != SNDRV_PCM_STATE_RUNNING) {
+ switch (stream->runtime->state) {
+ case SNDRV_PCM_STATE_SETUP:
+ case SNDRV_PCM_STATE_PREPARED:
+ case SNDRV_PCM_STATE_RUNNING:
+ break;
+ default:
mutex_unlock(&stream->device->lock);
return -EBADFD;
}
@@ -391,14 +394,13 @@
int retval = 0;
if (snd_BUG_ON(!data))
- return -EFAULT;
+ return POLLERR;
+
stream = &data->stream;
- if (snd_BUG_ON(!stream))
- return -EFAULT;
mutex_lock(&stream->device->lock);
if (stream->runtime->state == SNDRV_PCM_STATE_OPEN) {
- retval = -EBADFD;
+ retval = snd_compr_get_poll(stream) | POLLERR;
goto out;
}
poll_wait(f, &stream->runtime->sleep, wait);
@@ -421,10 +423,7 @@
retval = snd_compr_get_poll(stream);
break;
default:
- if (stream->direction == SND_COMPRESS_PLAYBACK)
- retval = POLLOUT | POLLWRNORM | POLLERR;
- else
- retval = POLLIN | POLLRDNORM | POLLERR;
+ retval = snd_compr_get_poll(stream) | POLLERR;
break;
}
out:
@@ -802,9 +801,9 @@
if (snd_BUG_ON(!data))
return -EFAULT;
+
stream = &data->stream;
- if (snd_BUG_ON(!stream))
- return -EFAULT;
+
mutex_lock(&stream->device->lock);
switch (_IOC_NR(cmd)) {
case _IOC_NR(SNDRV_COMPRESS_IOCTL_VERSION):
diff --git a/sound/core/hrtimer.c b/sound/core/hrtimer.c
index 656d9a9..e2f2702 100644
--- a/sound/core/hrtimer.c
+++ b/sound/core/hrtimer.c
@@ -38,37 +38,53 @@
struct snd_hrtimer {
struct snd_timer *timer;
struct hrtimer hrt;
- atomic_t running;
+ bool in_callback;
};
static enum hrtimer_restart snd_hrtimer_callback(struct hrtimer *hrt)
{
struct snd_hrtimer *stime = container_of(hrt, struct snd_hrtimer, hrt);
struct snd_timer *t = stime->timer;
- unsigned long oruns;
+ ktime_t delta;
+ unsigned long ticks;
+ enum hrtimer_restart ret = HRTIMER_NORESTART;
- if (!atomic_read(&stime->running))
- return HRTIMER_NORESTART;
+ spin_lock(&t->lock);
+ if (!t->running)
+ goto out; /* fast path */
+ stime->in_callback = true;
+ ticks = t->sticks;
+ spin_unlock(&t->lock);
- oruns = hrtimer_forward_now(hrt, ns_to_ktime(t->sticks * resolution));
- snd_timer_interrupt(stime->timer, t->sticks * oruns);
+ /* calculate the drift */
+ delta = ktime_sub(hrt->base->get_time(), hrtimer_get_expires(hrt));
+ if (delta.tv64 > 0)
+ ticks += ktime_divns(delta, ticks * resolution);
- if (!atomic_read(&stime->running))
- return HRTIMER_NORESTART;
- return HRTIMER_RESTART;
+ snd_timer_interrupt(stime->timer, ticks);
+
+ spin_lock(&t->lock);
+ if (t->running) {
+ hrtimer_add_expires_ns(hrt, t->sticks * resolution);
+ ret = HRTIMER_RESTART;
+ }
+
+ stime->in_callback = false;
+ out:
+ spin_unlock(&t->lock);
+ return ret;
}
static int snd_hrtimer_open(struct snd_timer *t)
{
struct snd_hrtimer *stime;
- stime = kmalloc(sizeof(*stime), GFP_KERNEL);
+ stime = kzalloc(sizeof(*stime), GFP_KERNEL);
if (!stime)
return -ENOMEM;
hrtimer_init(&stime->hrt, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
stime->timer = t;
stime->hrt.function = snd_hrtimer_callback;
- atomic_set(&stime->running, 0);
t->private_data = stime;
return 0;
}
@@ -78,6 +94,11 @@
struct snd_hrtimer *stime = t->private_data;
if (stime) {
+ spin_lock_irq(&t->lock);
+ t->running = 0; /* just to be sure */
+ stime->in_callback = 1; /* skip start/stop */
+ spin_unlock_irq(&t->lock);
+
hrtimer_cancel(&stime->hrt);
kfree(stime);
t->private_data = NULL;
@@ -89,18 +110,19 @@
{
struct snd_hrtimer *stime = t->private_data;
- atomic_set(&stime->running, 0);
- hrtimer_try_to_cancel(&stime->hrt);
+ if (stime->in_callback)
+ return 0;
hrtimer_start(&stime->hrt, ns_to_ktime(t->sticks * resolution),
HRTIMER_MODE_REL);
- atomic_set(&stime->running, 1);
return 0;
}
static int snd_hrtimer_stop(struct snd_timer *t)
{
struct snd_hrtimer *stime = t->private_data;
- atomic_set(&stime->running, 0);
+
+ if (stime->in_callback)
+ return 0;
hrtimer_try_to_cancel(&stime->hrt);
return 0;
}
diff --git a/sound/core/pcm_dmaengine.c b/sound/core/pcm_dmaengine.c
index 697c166..8eb58c7 100644
--- a/sound/core/pcm_dmaengine.c
+++ b/sound/core/pcm_dmaengine.c
@@ -106,8 +106,9 @@
* direction of the substream. If the substream is a playback stream the dst
* fields will be initialized, if it is a capture stream the src fields will be
* initialized. The {dst,src}_addr_width field will only be initialized if the
- * addr_width field of the DAI DMA data struct is not equal to
- * DMA_SLAVE_BUSWIDTH_UNDEFINED.
+ * SND_DMAENGINE_PCM_DAI_FLAG_PACK flag is set or if the addr_width field of
+ * the DAI DMA data struct is not equal to DMA_SLAVE_BUSWIDTH_UNDEFINED. If
+ * both conditions are met the latter takes priority.
*/
void snd_dmaengine_pcm_set_config_from_dai_data(
const struct snd_pcm_substream *substream,
@@ -117,11 +118,17 @@
if (substream->stream == SNDRV_PCM_STREAM_PLAYBACK) {
slave_config->dst_addr = dma_data->addr;
slave_config->dst_maxburst = dma_data->maxburst;
+ if (dma_data->flags & SND_DMAENGINE_PCM_DAI_FLAG_PACK)
+ slave_config->dst_addr_width =
+ DMA_SLAVE_BUSWIDTH_UNDEFINED;
if (dma_data->addr_width != DMA_SLAVE_BUSWIDTH_UNDEFINED)
slave_config->dst_addr_width = dma_data->addr_width;
} else {
slave_config->src_addr = dma_data->addr;
slave_config->src_maxburst = dma_data->maxburst;
+ if (dma_data->flags & SND_DMAENGINE_PCM_DAI_FLAG_PACK)
+ slave_config->src_addr_width =
+ DMA_SLAVE_BUSWIDTH_UNDEFINED;
if (dma_data->addr_width != DMA_SLAVE_BUSWIDTH_UNDEFINED)
slave_config->src_addr_width = dma_data->addr_width;
}
diff --git a/sound/core/pcm_iec958.c b/sound/core/pcm_iec958.c
index 36b2d7a..5e6aed6 100644
--- a/sound/core/pcm_iec958.c
+++ b/sound/core/pcm_iec958.c
@@ -9,30 +9,18 @@
#include <linux/types.h>
#include <sound/asoundef.h>
#include <sound/pcm.h>
+#include <sound/pcm_params.h>
#include <sound/pcm_iec958.h>
-/**
- * snd_pcm_create_iec958_consumer - create consumer format IEC958 channel status
- * @runtime: pcm runtime structure with ->rate filled in
- * @cs: channel status buffer, at least four bytes
- * @len: length of channel status buffer
- *
- * Create the consumer format channel status data in @cs of maximum size
- * @len corresponding to the parameters of the PCM runtime @runtime.
- *
- * Drivers may wish to tweak the contents of the buffer after creation.
- *
- * Returns: length of buffer, or negative error code if something failed.
- */
-int snd_pcm_create_iec958_consumer(struct snd_pcm_runtime *runtime, u8 *cs,
- size_t len)
+static int create_iec958_consumer(uint rate, uint sample_width,
+ u8 *cs, size_t len)
{
unsigned int fs, ws;
if (len < 4)
return -EINVAL;
- switch (runtime->rate) {
+ switch (rate) {
case 32000:
fs = IEC958_AES3_CON_FS_32000;
break;
@@ -59,7 +47,7 @@
}
if (len > 4) {
- switch (snd_pcm_format_width(runtime->format)) {
+ switch (sample_width) {
case 16:
ws = IEC958_AES4_CON_WORDLEN_20_16;
break;
@@ -71,6 +59,7 @@
IEC958_AES4_CON_MAX_WORDLEN_24;
break;
case 24:
+ case 32: /* Assume 24-bit width for 32-bit samples. */
ws = IEC958_AES4_CON_WORDLEN_24_20 |
IEC958_AES4_CON_MAX_WORDLEN_24;
break;
@@ -92,4 +81,46 @@
return len;
}
+
+/**
+ * snd_pcm_create_iec958_consumer - create consumer format IEC958 channel status
+ * @runtime: pcm runtime structure with ->rate filled in
+ * @cs: channel status buffer, at least four bytes
+ * @len: length of channel status buffer
+ *
+ * Create the consumer format channel status data in @cs of maximum size
+ * @len corresponding to the parameters of the PCM runtime @runtime.
+ *
+ * Drivers may wish to tweak the contents of the buffer after creation.
+ *
+ * Returns: length of buffer, or negative error code if something failed.
+ */
+int snd_pcm_create_iec958_consumer(struct snd_pcm_runtime *runtime, u8 *cs,
+ size_t len)
+{
+ return create_iec958_consumer(runtime->rate,
+ snd_pcm_format_width(runtime->format),
+ cs, len);
+}
EXPORT_SYMBOL(snd_pcm_create_iec958_consumer);
+
+/**
+ * snd_pcm_create_iec958_consumer_hw_params - create IEC958 channel status
+ * @hw_params: the hw_params instance for extracting rate and sample format
+ * @cs: channel status buffer, at least four bytes
+ * @len: length of channel status buffer
+ *
+ * Create the consumer format channel status data in @cs of maximum size
+ * @len corresponding to the parameters of the PCM runtime @runtime.
+ *
+ * Drivers may wish to tweak the contents of the buffer after creation.
+ *
+ * Returns: length of buffer, or negative error code if something failed.
+ */
+int snd_pcm_create_iec958_consumer_hw_params(struct snd_pcm_hw_params *params,
+ u8 *cs, size_t len)
+{
+ return create_iec958_consumer(params_rate(params), params_width(params),
+ cs, len);
+}
+EXPORT_SYMBOL(snd_pcm_create_iec958_consumer_hw_params);
diff --git a/sound/core/pcm_lib.c b/sound/core/pcm_lib.c
index 3a9b66c..bb12615 100644
--- a/sound/core/pcm_lib.c
+++ b/sound/core/pcm_lib.c
@@ -1886,8 +1886,8 @@
snd_timer_interrupt(substream->timer, 1);
#endif
_end:
- snd_pcm_stream_unlock_irqrestore(substream, flags);
kill_fasync(&runtime->fasync, SIGIO, POLL_IN);
+ snd_pcm_stream_unlock_irqrestore(substream, flags);
}
EXPORT_SYMBOL(snd_pcm_period_elapsed);
@@ -2595,6 +2595,8 @@
};
int err;
+ if (WARN_ON(pcm->streams[stream].chmap_kctl))
+ return -EBUSY;
info = kzalloc(sizeof(*info), GFP_KERNEL);
if (!info)
return -ENOMEM;
diff --git a/sound/core/pcm_native.c b/sound/core/pcm_native.c
index 9106d8e..c61fd50 100644
--- a/sound/core/pcm_native.c
+++ b/sound/core/pcm_native.c
@@ -3161,7 +3161,7 @@
substream = pcm_file->substream;
if (PCM_RUNTIME_CHECK(substream))
- return -ENXIO;
+ return POLLOUT | POLLWRNORM | POLLERR;
runtime = substream->runtime;
poll_wait(file, &runtime->sleep, wait);
@@ -3200,7 +3200,7 @@
substream = pcm_file->substream;
if (PCM_RUNTIME_CHECK(substream))
- return -ENXIO;
+ return POLLIN | POLLRDNORM | POLLERR;
runtime = substream->runtime;
poll_wait(file, &runtime->sleep, wait);
diff --git a/sound/core/rtctimer.c b/sound/core/rtctimer.c
deleted file mode 100644
index f3420d1..0000000
--- a/sound/core/rtctimer.c
+++ /dev/null
@@ -1,187 +0,0 @@
-/*
- * RTC based high-frequency timer
- *
- * Copyright (C) 2000 Takashi Iwai
- * based on rtctimer.c by Steve Ratcliffe
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
- */
-
-#include <linux/init.h>
-#include <linux/interrupt.h>
-#include <linux/module.h>
-#include <linux/log2.h>
-#include <sound/core.h>
-#include <sound/timer.h>
-
-#if IS_ENABLED(CONFIG_RTC)
-
-#include <linux/mc146818rtc.h>
-
-#define RTC_FREQ 1024 /* default frequency */
-#define NANO_SEC 1000000000L /* 10^9 in sec */
-
-/*
- * prototypes
- */
-static int rtctimer_open(struct snd_timer *t);
-static int rtctimer_close(struct snd_timer *t);
-static int rtctimer_start(struct snd_timer *t);
-static int rtctimer_stop(struct snd_timer *t);
-
-
-/*
- * The hardware dependent description for this timer.
- */
-static struct snd_timer_hardware rtc_hw = {
- .flags = SNDRV_TIMER_HW_AUTO |
- SNDRV_TIMER_HW_FIRST |
- SNDRV_TIMER_HW_TASKLET,
- .ticks = 100000000L, /* FIXME: XXX */
- .open = rtctimer_open,
- .close = rtctimer_close,
- .start = rtctimer_start,
- .stop = rtctimer_stop,
-};
-
-static int rtctimer_freq = RTC_FREQ; /* frequency */
-static struct snd_timer *rtctimer;
-static struct tasklet_struct rtc_tasklet;
-static rtc_task_t rtc_task;
-
-
-static int
-rtctimer_open(struct snd_timer *t)
-{
- int err;
-
- err = rtc_register(&rtc_task);
- if (err < 0)
- return err;
- t->private_data = &rtc_task;
- return 0;
-}
-
-static int
-rtctimer_close(struct snd_timer *t)
-{
- rtc_task_t *rtc = t->private_data;
- if (rtc) {
- rtc_unregister(rtc);
- tasklet_kill(&rtc_tasklet);
- t->private_data = NULL;
- }
- return 0;
-}
-
-static int
-rtctimer_start(struct snd_timer *timer)
-{
- rtc_task_t *rtc = timer->private_data;
- if (snd_BUG_ON(!rtc))
- return -EINVAL;
- rtc_control(rtc, RTC_IRQP_SET, rtctimer_freq);
- rtc_control(rtc, RTC_PIE_ON, 0);
- return 0;
-}
-
-static int
-rtctimer_stop(struct snd_timer *timer)
-{
- rtc_task_t *rtc = timer->private_data;
- if (snd_BUG_ON(!rtc))
- return -EINVAL;
- rtc_control(rtc, RTC_PIE_OFF, 0);
- return 0;
-}
-
-static void rtctimer_tasklet(unsigned long data)
-{
- snd_timer_interrupt((struct snd_timer *)data, 1);
-}
-
-/*
- * interrupt
- */
-static void rtctimer_interrupt(void *private_data)
-{
- tasklet_schedule(private_data);
-}
-
-
-/*
- * ENTRY functions
- */
-static int __init rtctimer_init(void)
-{
- int err;
- struct snd_timer *timer;
-
- if (rtctimer_freq < 2 || rtctimer_freq > 8192 ||
- !is_power_of_2(rtctimer_freq)) {
- pr_err("ALSA: rtctimer: invalid frequency %d\n", rtctimer_freq);
- return -EINVAL;
- }
-
- /* Create a new timer and set up the fields */
- err = snd_timer_global_new("rtc", SNDRV_TIMER_GLOBAL_RTC, &timer);
- if (err < 0)
- return err;
-
- timer->module = THIS_MODULE;
- strcpy(timer->name, "RTC timer");
- timer->hw = rtc_hw;
- timer->hw.resolution = NANO_SEC / rtctimer_freq;
-
- tasklet_init(&rtc_tasklet, rtctimer_tasklet, (unsigned long)timer);
-
- /* set up RTC callback */
- rtc_task.func = rtctimer_interrupt;
- rtc_task.private_data = &rtc_tasklet;
-
- err = snd_timer_global_register(timer);
- if (err < 0) {
- snd_timer_global_free(timer);
- return err;
- }
- rtctimer = timer; /* remember this */
-
- return 0;
-}
-
-static void __exit rtctimer_exit(void)
-{
- if (rtctimer) {
- snd_timer_global_free(rtctimer);
- rtctimer = NULL;
- }
-}
-
-
-/*
- * exported stuff
- */
-module_init(rtctimer_init)
-module_exit(rtctimer_exit)
-
-module_param(rtctimer_freq, int, 0444);
-MODULE_PARM_DESC(rtctimer_freq, "timer frequency in Hz");
-
-MODULE_LICENSE("GPL");
-
-MODULE_ALIAS("snd-timer-" __stringify(SNDRV_TIMER_GLOBAL_RTC));
-
-#endif /* IS_ENABLED(CONFIG_RTC) */
diff --git a/sound/core/seq/seq.c b/sound/core/seq/seq.c
index 7e0aabb..639544b 100644
--- a/sound/core/seq/seq.c
+++ b/sound/core/seq/seq.c
@@ -47,8 +47,6 @@
int seq_default_timer_device =
#ifdef CONFIG_SND_SEQ_HRTIMER_DEFAULT
SNDRV_TIMER_GLOBAL_HRTIMER
-#elif defined(CONFIG_SND_SEQ_RTCTIMER_DEFAULT)
- SNDRV_TIMER_GLOBAL_RTC
#else
SNDRV_TIMER_GLOBAL_SYSTEM
#endif
diff --git a/sound/core/timer.c b/sound/core/timer.c
index 6469bed..e722022 100644
--- a/sound/core/timer.c
+++ b/sound/core/timer.c
@@ -37,8 +37,6 @@
#if IS_ENABLED(CONFIG_SND_HRTIMER)
#define DEFAULT_TIMER_LIMIT 4
-#elif IS_ENABLED(CONFIG_SND_RTCTIMER)
-#define DEFAULT_TIMER_LIMIT 2
#else
#define DEFAULT_TIMER_LIMIT 1
#endif
@@ -1225,6 +1223,7 @@
tu->tstamp = *tstamp;
if ((tu->filter & (1 << event)) == 0 || !tu->tread)
return;
+ memset(&r1, 0, sizeof(r1));
r1.event = event;
r1.tstamp = *tstamp;
r1.val = resolution;
@@ -1267,6 +1266,7 @@
}
if ((tu->filter & (1 << SNDRV_TIMER_EVENT_RESOLUTION)) &&
tu->last_resolution != resolution) {
+ memset(&r1, 0, sizeof(r1));
r1.event = SNDRV_TIMER_EVENT_RESOLUTION;
r1.tstamp = tstamp;
r1.val = resolution;
@@ -1739,6 +1739,7 @@
if (tu->timeri->flags & SNDRV_TIMER_IFLG_EARLY_EVENT) {
if (tu->tread) {
struct snd_timer_tread tread;
+ memset(&tread, 0, sizeof(tread));
tread.event = SNDRV_TIMER_EVENT_EARLY;
tread.tstamp.tv_sec = 0;
tread.tstamp.tv_nsec = 0;
diff --git a/sound/firewire/Kconfig b/sound/firewire/Kconfig
index 2a779c2..ab894ed 100644
--- a/sound/firewire/Kconfig
+++ b/sound/firewire/Kconfig
@@ -134,6 +134,7 @@
Say Y here to include support for TASCAM.
* FW-1884
* FW-1082
+ * FW-1804
To compile this driver as a module, choose M here: the module
will be called snd-firewire-tascam.
diff --git a/sound/firewire/Makefile b/sound/firewire/Makefile
index 003c090..0ee1fb1 100644
--- a/sound/firewire/Makefile
+++ b/sound/firewire/Makefile
@@ -1,3 +1,6 @@
+# To find a header included by define_trace.h.
+CFLAGS_amdtp-stream.o := -I$(src)
+
snd-firewire-lib-objs := lib.o iso-resources.o packets-buffer.o \
fcp.o cmp.o amdtp-stream.o amdtp-am824.o
snd-isight-objs := isight.o
diff --git a/sound/firewire/amdtp-stream-trace.h b/sound/firewire/amdtp-stream-trace.h
new file mode 100644
index 0000000..9c04faf
--- /dev/null
+++ b/sound/firewire/amdtp-stream-trace.h
@@ -0,0 +1,110 @@
+/*
+ * amdtp-stream-trace.h - tracepoint definitions to dump a part of packet data
+ *
+ * Copyright (c) 2016 Takashi Sakamoto
+ * Licensed under the terms of the GNU General Public License, version 2.
+ */
+
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM snd_firewire_lib
+
+#if !defined(_AMDTP_STREAM_TRACE_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _AMDTP_STREAM_TRACE_H
+
+#include <linux/tracepoint.h>
+
+TRACE_EVENT(in_packet,
+ TP_PROTO(const struct amdtp_stream *s, u32 cycles, u32 cip_header[2], unsigned int payload_quadlets, unsigned int index),
+ TP_ARGS(s, cycles, cip_header, payload_quadlets, index),
+ TP_STRUCT__entry(
+ __field(unsigned int, second)
+ __field(unsigned int, cycle)
+ __field(int, channel)
+ __field(int, src)
+ __field(int, dest)
+ __field(u32, cip_header0)
+ __field(u32, cip_header1)
+ __field(unsigned int, payload_quadlets)
+ __field(unsigned int, packet_index)
+ __field(unsigned int, irq)
+ __field(unsigned int, index)
+ ),
+ TP_fast_assign(
+ __entry->second = cycles / CYCLES_PER_SECOND;
+ __entry->cycle = cycles % CYCLES_PER_SECOND;
+ __entry->channel = s->context->channel;
+ __entry->src = fw_parent_device(s->unit)->node_id;
+ __entry->dest = fw_parent_device(s->unit)->card->node_id;
+ __entry->cip_header0 = cip_header[0];
+ __entry->cip_header1 = cip_header[1];
+ __entry->payload_quadlets = payload_quadlets;
+ __entry->packet_index = s->packet_index;
+ __entry->irq = !!in_interrupt();
+ __entry->index = index;
+ ),
+ TP_printk(
+ "%02u %04u %04x %04x %02d %08x %08x %03u %02u %01u %02u",
+ __entry->second,
+ __entry->cycle,
+ __entry->src,
+ __entry->dest,
+ __entry->channel,
+ __entry->cip_header0,
+ __entry->cip_header1,
+ __entry->payload_quadlets,
+ __entry->packet_index,
+ __entry->irq,
+ __entry->index)
+);
+
+TRACE_EVENT(out_packet,
+ TP_PROTO(const struct amdtp_stream *s, u32 cycles, __be32 *cip_header, unsigned int payload_length, unsigned int index),
+ TP_ARGS(s, cycles, cip_header, payload_length, index),
+ TP_STRUCT__entry(
+ __field(unsigned int, second)
+ __field(unsigned int, cycle)
+ __field(int, channel)
+ __field(int, src)
+ __field(int, dest)
+ __field(u32, cip_header0)
+ __field(u32, cip_header1)
+ __field(unsigned int, payload_quadlets)
+ __field(unsigned int, packet_index)
+ __field(unsigned int, irq)
+ __field(unsigned int, index)
+ ),
+ TP_fast_assign(
+ __entry->second = cycles / CYCLES_PER_SECOND;
+ __entry->cycle = cycles % CYCLES_PER_SECOND;
+ __entry->channel = s->context->channel;
+ __entry->src = fw_parent_device(s->unit)->card->node_id;
+ __entry->dest = fw_parent_device(s->unit)->node_id;
+ __entry->cip_header0 = be32_to_cpu(cip_header[0]);
+ __entry->cip_header1 = be32_to_cpu(cip_header[1]);
+ __entry->payload_quadlets = payload_length / 4;
+ __entry->packet_index = s->packet_index;
+ __entry->irq = !!in_interrupt();
+ __entry->index = index;
+ ),
+ TP_printk(
+ "%02u %04u %04x %04x %02d %08x %08x %03u %02u %01u %02u",
+ __entry->second,
+ __entry->cycle,
+ __entry->src,
+ __entry->dest,
+ __entry->channel,
+ __entry->cip_header0,
+ __entry->cip_header1,
+ __entry->payload_quadlets,
+ __entry->packet_index,
+ __entry->irq,
+ __entry->index)
+);
+
+#endif
+
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH .
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_FILE amdtp-stream-trace
+#include <trace/define_trace.h>
diff --git a/sound/firewire/amdtp-stream.c b/sound/firewire/amdtp-stream.c
index ed29026..00060c4 100644
--- a/sound/firewire/amdtp-stream.c
+++ b/sound/firewire/amdtp-stream.c
@@ -19,6 +19,10 @@
#define CYCLES_PER_SECOND 8000
#define TICKS_PER_SECOND (TICKS_PER_CYCLE * CYCLES_PER_SECOND)
+/* Always support Linux tracing subsystem. */
+#define CREATE_TRACE_POINTS
+#include "amdtp-stream-trace.h"
+
#define TRANSFER_DELAY_TICKS 0x2e00 /* 479.17 microseconds */
/* isochronous header parameters */
@@ -87,7 +91,6 @@
init_waitqueue_head(&s->callback_wait);
s->callbacked = false;
- s->sync_slave = NULL;
s->fmt = fmt;
s->process_data_blocks = process_data_blocks;
@@ -102,6 +105,10 @@
*/
void amdtp_stream_destroy(struct amdtp_stream *s)
{
+ /* Not initialized. */
+ if (s->protocol == NULL)
+ return;
+
WARN_ON(amdtp_stream_running(s));
kfree(s->protocol);
mutex_destroy(&s->mutex);
@@ -244,7 +251,6 @@
tasklet_kill(&s->period_tasklet);
s->pcm_buffer_pointer = 0;
s->pcm_period_pointer = 0;
- s->pointer_flush = true;
}
EXPORT_SYMBOL(amdtp_stream_pcm_prepare);
@@ -349,7 +355,6 @@
s->pcm_period_pointer += frames;
if (s->pcm_period_pointer >= pcm->runtime->period_size) {
s->pcm_period_pointer -= pcm->runtime->period_size;
- s->pointer_flush = false;
tasklet_hi_schedule(&s->period_tasklet);
}
}
@@ -363,9 +368,8 @@
snd_pcm_period_elapsed(pcm);
}
-static int queue_packet(struct amdtp_stream *s,
- unsigned int header_length,
- unsigned int payload_length, bool skip)
+static int queue_packet(struct amdtp_stream *s, unsigned int header_length,
+ unsigned int payload_length)
{
struct fw_iso_packet p = {0};
int err = 0;
@@ -376,8 +380,10 @@
p.interrupt = IS_ALIGNED(s->packet_index + 1, INTERRUPT_INTERVAL);
p.tag = TAG_CIP;
p.header_length = header_length;
- p.payload_length = (!skip) ? payload_length : 0;
- p.skip = skip;
+ if (payload_length > 0)
+ p.payload_length = payload_length;
+ else
+ p.skip = true;
err = fw_iso_context_queue(s->context, &p, &s->buffer.iso_buffer,
s->buffer.packets[s->packet_index].offset);
if (err < 0) {
@@ -392,27 +398,30 @@
}
static inline int queue_out_packet(struct amdtp_stream *s,
- unsigned int payload_length, bool skip)
+ unsigned int payload_length)
{
- return queue_packet(s, OUT_PACKET_HEADER_SIZE,
- payload_length, skip);
+ return queue_packet(s, OUT_PACKET_HEADER_SIZE, payload_length);
}
static inline int queue_in_packet(struct amdtp_stream *s)
{
return queue_packet(s, IN_PACKET_HEADER_SIZE,
- amdtp_stream_get_max_payload(s), false);
+ amdtp_stream_get_max_payload(s));
}
-static int handle_out_packet(struct amdtp_stream *s, unsigned int data_blocks,
- unsigned int syt)
+static int handle_out_packet(struct amdtp_stream *s, unsigned int cycle,
+ unsigned int index)
{
__be32 *buffer;
+ unsigned int syt;
+ unsigned int data_blocks;
unsigned int payload_length;
unsigned int pcm_frames;
struct snd_pcm_substream *pcm;
buffer = s->buffer.packets[s->packet_index].buffer;
+ syt = calculate_syt(s, cycle);
+ data_blocks = calculate_data_blocks(s, syt);
pcm_frames = s->process_data_blocks(s, buffer + 2, data_blocks, &syt);
buffer[0] = cpu_to_be32(ACCESS_ONCE(s->source_node_id_field) |
@@ -424,9 +433,11 @@
(syt & CIP_SYT_MASK));
s->data_block_counter = (s->data_block_counter + data_blocks) & 0xff;
-
payload_length = 8 + data_blocks * 4 * s->data_block_quadlets;
- if (queue_out_packet(s, payload_length, false) < 0)
+
+ trace_out_packet(s, cycle, buffer, payload_length, index);
+
+ if (queue_out_packet(s, payload_length) < 0)
return -EIO;
pcm = ACCESS_ONCE(s->pcm);
@@ -438,19 +449,24 @@
}
static int handle_in_packet(struct amdtp_stream *s,
- unsigned int payload_quadlets, __be32 *buffer,
- unsigned int *data_blocks, unsigned int syt)
+ unsigned int payload_quadlets, unsigned int cycle,
+ unsigned int index)
{
+ __be32 *buffer;
u32 cip_header[2];
- unsigned int fmt, fdf;
+ unsigned int fmt, fdf, syt;
unsigned int data_block_quadlets, data_block_counter, dbc_interval;
+ unsigned int data_blocks;
struct snd_pcm_substream *pcm;
unsigned int pcm_frames;
bool lost;
+ buffer = s->buffer.packets[s->packet_index].buffer;
cip_header[0] = be32_to_cpu(buffer[0]);
cip_header[1] = be32_to_cpu(buffer[1]);
+ trace_in_packet(s, cycle, cip_header, payload_quadlets, index);
+
/*
* This module supports 'Two-quadlet CIP header with SYT field'.
* For convenience, also check FMT field is AM824 or not.
@@ -460,7 +476,7 @@
dev_info_ratelimited(&s->unit->device,
"Invalid CIP header for AMDTP: %08X:%08X\n",
cip_header[0], cip_header[1]);
- *data_blocks = 0;
+ data_blocks = 0;
pcm_frames = 0;
goto end;
}
@@ -471,7 +487,7 @@
dev_info_ratelimited(&s->unit->device,
"Detect unexpected protocol: %08x %08x\n",
cip_header[0], cip_header[1]);
- *data_blocks = 0;
+ data_blocks = 0;
pcm_frames = 0;
goto end;
}
@@ -480,7 +496,7 @@
fdf = (cip_header[1] & CIP_FDF_MASK) >> CIP_FDF_SHIFT;
if (payload_quadlets < 3 ||
(fmt == CIP_FMT_AM && fdf == AMDTP_FDF_NO_DATA)) {
- *data_blocks = 0;
+ data_blocks = 0;
} else {
data_block_quadlets =
(cip_header[0] & CIP_DBS_MASK) >> CIP_DBS_SHIFT;
@@ -494,12 +510,12 @@
if (s->flags & CIP_WRONG_DBS)
data_block_quadlets = s->data_block_quadlets;
- *data_blocks = (payload_quadlets - 2) / data_block_quadlets;
+ data_blocks = (payload_quadlets - 2) / data_block_quadlets;
}
/* Check data block counter continuity */
data_block_counter = cip_header[0] & CIP_DBC_MASK;
- if (*data_blocks == 0 && (s->flags & CIP_EMPTY_HAS_WRONG_DBC) &&
+ if (data_blocks == 0 && (s->flags & CIP_EMPTY_HAS_WRONG_DBC) &&
s->data_block_counter != UINT_MAX)
data_block_counter = s->data_block_counter;
@@ -510,10 +526,10 @@
} else if (!(s->flags & CIP_DBC_IS_END_EVENT)) {
lost = data_block_counter != s->data_block_counter;
} else {
- if ((*data_blocks > 0) && (s->tx_dbc_interval > 0))
+ if (data_blocks > 0 && s->tx_dbc_interval > 0)
dbc_interval = s->tx_dbc_interval;
else
- dbc_interval = *data_blocks;
+ dbc_interval = data_blocks;
lost = data_block_counter !=
((s->data_block_counter + dbc_interval) & 0xff);
@@ -526,13 +542,14 @@
return -EIO;
}
- pcm_frames = s->process_data_blocks(s, buffer + 2, *data_blocks, &syt);
+ syt = be32_to_cpu(buffer[1]) & CIP_SYT_MASK;
+ pcm_frames = s->process_data_blocks(s, buffer + 2, data_blocks, &syt);
if (s->flags & CIP_DBC_IS_END_EVENT)
s->data_block_counter = data_block_counter;
else
s->data_block_counter =
- (data_block_counter + *data_blocks) & 0xff;
+ (data_block_counter + data_blocks) & 0xff;
end:
if (queue_in_packet(s) < 0)
return -EIO;
@@ -544,29 +561,50 @@
return 0;
}
-static void out_stream_callback(struct fw_iso_context *context, u32 cycle,
+/*
+ * In CYCLE_TIMER register of IEEE 1394, 7 bits are used to represent second. On
+ * the other hand, in DMA descriptors of 1394 OHCI, 3 bits are used to represent
+ * it. Thus, via Linux firewire subsystem, we can get the 3 bits for second.
+ */
+static inline u32 compute_cycle_count(u32 tstamp)
+{
+ return (((tstamp >> 13) & 0x07) * 8000) + (tstamp & 0x1fff);
+}
+
+static inline u32 increment_cycle_count(u32 cycle, unsigned int addend)
+{
+ cycle += addend;
+ if (cycle >= 8 * CYCLES_PER_SECOND)
+ cycle -= 8 * CYCLES_PER_SECOND;
+ return cycle;
+}
+
+static inline u32 decrement_cycle_count(u32 cycle, unsigned int subtrahend)
+{
+ if (cycle < subtrahend)
+ cycle += 8 * CYCLES_PER_SECOND;
+ return cycle - subtrahend;
+}
+
+static void out_stream_callback(struct fw_iso_context *context, u32 tstamp,
size_t header_length, void *header,
void *private_data)
{
struct amdtp_stream *s = private_data;
- unsigned int i, syt, packets = header_length / 4;
- unsigned int data_blocks;
+ unsigned int i, packets = header_length / 4;
+ u32 cycle;
if (s->packet_index < 0)
return;
- /*
- * Compute the cycle of the last queued packet.
- * (We need only the four lowest bits for the SYT, so we can ignore
- * that bits 0-11 must wrap around at 3072.)
- */
- cycle += QUEUE_LENGTH - packets;
+ cycle = compute_cycle_count(tstamp);
+
+ /* Align to actual cycle count for the last packet. */
+ cycle = increment_cycle_count(cycle, QUEUE_LENGTH - packets);
for (i = 0; i < packets; ++i) {
- syt = calculate_syt(s, ++cycle);
- data_blocks = calculate_data_blocks(s, syt);
-
- if (handle_out_packet(s, data_blocks, syt) < 0) {
+ cycle = increment_cycle_count(cycle, 1);
+ if (handle_out_packet(s, cycle, i) < 0) {
s->packet_index = -1;
amdtp_stream_pcm_abort(s);
return;
@@ -576,15 +614,15 @@
fw_iso_context_queue_flush(s->context);
}
-static void in_stream_callback(struct fw_iso_context *context, u32 cycle,
+static void in_stream_callback(struct fw_iso_context *context, u32 tstamp,
size_t header_length, void *header,
void *private_data)
{
struct amdtp_stream *s = private_data;
- unsigned int p, syt, packets;
+ unsigned int i, packets;
unsigned int payload_quadlets, max_payload_quadlets;
- unsigned int data_blocks;
- __be32 *buffer, *headers = header;
+ __be32 *headers = header;
+ u32 cycle;
if (s->packet_index < 0)
return;
@@ -592,70 +630,44 @@
/* The number of packets in buffer */
packets = header_length / IN_PACKET_HEADER_SIZE;
+ cycle = compute_cycle_count(tstamp);
+
+ /* Align to actual cycle count for the last packet. */
+ cycle = decrement_cycle_count(cycle, packets);
+
/* For buffer-over-run prevention. */
max_payload_quadlets = amdtp_stream_get_max_payload(s) / 4;
- for (p = 0; p < packets; p++) {
- buffer = s->buffer.packets[s->packet_index].buffer;
+ for (i = 0; i < packets; i++) {
+ cycle = increment_cycle_count(cycle, 1);
/* The number of quadlets in this packet */
payload_quadlets =
- (be32_to_cpu(headers[p]) >> ISO_DATA_LENGTH_SHIFT) / 4;
+ (be32_to_cpu(headers[i]) >> ISO_DATA_LENGTH_SHIFT) / 4;
if (payload_quadlets > max_payload_quadlets) {
dev_err(&s->unit->device,
"Detect jumbo payload: %02x %02x\n",
payload_quadlets, max_payload_quadlets);
- s->packet_index = -1;
break;
}
- syt = be32_to_cpu(buffer[1]) & CIP_SYT_MASK;
- if (handle_in_packet(s, payload_quadlets, buffer,
- &data_blocks, syt) < 0) {
- s->packet_index = -1;
+ if (handle_in_packet(s, payload_quadlets, cycle, i) < 0)
break;
- }
-
- /* Process sync slave stream */
- if (s->sync_slave && s->sync_slave->callbacked) {
- if (handle_out_packet(s->sync_slave,
- data_blocks, syt) < 0) {
- s->packet_index = -1;
- break;
- }
- }
}
- /* Queueing error or detecting discontinuity */
- if (s->packet_index < 0) {
+ /* Queueing error or detecting invalid payload. */
+ if (i < packets) {
+ s->packet_index = -1;
amdtp_stream_pcm_abort(s);
-
- /* Abort sync slave. */
- if (s->sync_slave) {
- s->sync_slave->packet_index = -1;
- amdtp_stream_pcm_abort(s->sync_slave);
- }
return;
}
- /* when sync to device, flush the packets for slave stream */
- if (s->sync_slave && s->sync_slave->callbacked)
- fw_iso_context_queue_flush(s->sync_slave->context);
-
fw_iso_context_queue_flush(s->context);
}
-/* processing is done by master callback */
-static void slave_stream_callback(struct fw_iso_context *context, u32 cycle,
- size_t header_length, void *header,
- void *private_data)
-{
- return;
-}
-
/* this is executed one time */
static void amdtp_stream_first_callback(struct fw_iso_context *context,
- u32 cycle, size_t header_length,
+ u32 tstamp, size_t header_length,
void *header, void *private_data)
{
struct amdtp_stream *s = private_data;
@@ -669,12 +681,10 @@
if (s->direction == AMDTP_IN_STREAM)
context->callback.sc = in_stream_callback;
- else if (s->flags & CIP_SYNC_TO_DEVICE)
- context->callback.sc = slave_stream_callback;
else
context->callback.sc = out_stream_callback;
- context->callback.sc(context, cycle, header_length, header, s);
+ context->callback.sc(context, tstamp, header_length, header, s);
}
/**
@@ -713,8 +723,7 @@
goto err_unlock;
}
- if (s->direction == AMDTP_IN_STREAM &&
- s->flags & CIP_SKIP_INIT_DBC_CHECK)
+ if (s->direction == AMDTP_IN_STREAM)
s->data_block_counter = UINT_MAX;
else
s->data_block_counter = 0;
@@ -755,7 +764,7 @@
if (s->direction == AMDTP_IN_STREAM)
err = queue_in_packet(s);
else
- err = queue_out_packet(s, 0, true);
+ err = queue_out_packet(s, 0);
if (err < 0)
goto err_context;
} while (s->packet_index > 0);
@@ -794,11 +803,24 @@
*/
unsigned long amdtp_stream_pcm_pointer(struct amdtp_stream *s)
{
- /* this optimization is allowed to be racy */
- if (s->pointer_flush && amdtp_stream_running(s))
+ /*
+ * This function is called in software IRQ context of period_tasklet or
+ * process context.
+ *
+ * When the software IRQ context was scheduled by software IRQ context
+ * of IR/IT contexts, queued packets were already handled. Therefore,
+ * no need to flush the queue in buffer anymore.
+ *
+ * When the process context reach here, some packets will be already
+ * queued in the buffer. These packets should be handled immediately
+ * to keep better granularity of PCM pointer.
+ *
+ * Later, the process context will sometimes schedules software IRQ
+ * context of the period_tasklet. Then, no need to flush the queue by
+ * the same reason as described for IR/IT contexts.
+ */
+ if (!in_interrupt() && amdtp_stream_running(s))
fw_iso_context_flush_completions(s->context);
- else
- s->pointer_flush = true;
return ACCESS_ONCE(s->pcm_buffer_pointer);
}
diff --git a/sound/firewire/amdtp-stream.h b/sound/firewire/amdtp-stream.h
index 8775704..c1bc7fa 100644
--- a/sound/firewire/amdtp-stream.h
+++ b/sound/firewire/amdtp-stream.h
@@ -17,8 +17,6 @@
* @CIP_BLOCKING: In blocking mode, each packet contains either zero or
* SYT_INTERVAL samples, with these two types alternating so that
* the overall sample rate comes out right.
- * @CIP_SYNC_TO_DEVICE: In sync to device mode, time stamp in out packets is
- * generated by in packets. Defaultly this driver generates timestamp.
* @CIP_EMPTY_WITH_TAG0: Only for in-stream. Empty in-packets have TAG0.
* @CIP_DBC_IS_END_EVENT: Only for in-stream. The value of dbc in an in-packet
* corresponds to the end of event in the packet. Out of IEC 61883.
@@ -26,8 +24,6 @@
* The value of data_block_quadlets is used instead of reported value.
* @CIP_SKIP_DBC_ZERO_CHECK: Only for in-stream. Packets with zero in dbc is
* skipped for detecting discontinuity.
- * @CIP_SKIP_INIT_DBC_CHECK: Only for in-stream. The value of dbc in first
- * packet is not continuous from an initial value.
* @CIP_EMPTY_HAS_WRONG_DBC: Only for in-stream. The value of dbc in empty
* packet is wrong but the others are correct.
* @CIP_JUMBO_PAYLOAD: Only for in-stream. The number of data blocks in an
@@ -37,14 +33,12 @@
enum cip_flags {
CIP_NONBLOCKING = 0x00,
CIP_BLOCKING = 0x01,
- CIP_SYNC_TO_DEVICE = 0x02,
- CIP_EMPTY_WITH_TAG0 = 0x04,
- CIP_DBC_IS_END_EVENT = 0x08,
- CIP_WRONG_DBS = 0x10,
- CIP_SKIP_DBC_ZERO_CHECK = 0x20,
- CIP_SKIP_INIT_DBC_CHECK = 0x40,
- CIP_EMPTY_HAS_WRONG_DBC = 0x80,
- CIP_JUMBO_PAYLOAD = 0x100,
+ CIP_EMPTY_WITH_TAG0 = 0x02,
+ CIP_DBC_IS_END_EVENT = 0x04,
+ CIP_WRONG_DBS = 0x08,
+ CIP_SKIP_DBC_ZERO_CHECK = 0x10,
+ CIP_EMPTY_HAS_WRONG_DBC = 0x20,
+ CIP_JUMBO_PAYLOAD = 0x40,
};
/**
@@ -132,12 +126,10 @@
struct tasklet_struct period_tasklet;
unsigned int pcm_buffer_pointer;
unsigned int pcm_period_pointer;
- bool pointer_flush;
/* To wait for first packet. */
bool callbacked;
wait_queue_head_t callback_wait;
- struct amdtp_stream *sync_slave;
/* For backends to process data blocks. */
void *protocol;
@@ -223,23 +215,6 @@
return sfc & 1;
}
-static inline void amdtp_stream_set_sync(enum cip_flags sync_mode,
- struct amdtp_stream *master,
- struct amdtp_stream *slave)
-{
- if (sync_mode == CIP_SYNC_TO_DEVICE) {
- master->flags |= CIP_SYNC_TO_DEVICE;
- slave->flags |= CIP_SYNC_TO_DEVICE;
- master->sync_slave = slave;
- } else {
- master->flags &= ~CIP_SYNC_TO_DEVICE;
- slave->flags &= ~CIP_SYNC_TO_DEVICE;
- master->sync_slave = NULL;
- }
-
- slave->sync_slave = NULL;
-}
-
/**
* amdtp_stream_wait_callback - sleep till callbacked or timeout
* @s: the AMDTP stream
diff --git a/sound/firewire/bebob/bebob.c b/sound/firewire/bebob/bebob.c
index 3e4e075..f7e2cbd 100644
--- a/sound/firewire/bebob/bebob.c
+++ b/sound/firewire/bebob/bebob.c
@@ -67,7 +67,7 @@
#define MODEL_MAUDIO_PROJECTMIX 0x00010091
static int
-name_device(struct snd_bebob *bebob, unsigned int vendor_id)
+name_device(struct snd_bebob *bebob)
{
struct fw_device *fw_dev = fw_parent_device(bebob->unit);
char vendor[24] = {0};
@@ -126,6 +126,17 @@
return err;
}
+static void bebob_free(struct snd_bebob *bebob)
+{
+ snd_bebob_stream_destroy_duplex(bebob);
+ fw_unit_put(bebob->unit);
+
+ kfree(bebob->maudio_special_quirk);
+
+ mutex_destroy(&bebob->mutex);
+ kfree(bebob);
+}
+
/*
* This module releases the FireWire unit data after all ALSA character devices
* are released by applications. This is for releasing stream data or finishing
@@ -137,18 +148,11 @@
{
struct snd_bebob *bebob = card->private_data;
- snd_bebob_stream_destroy_duplex(bebob);
- fw_unit_put(bebob->unit);
+ mutex_lock(&devices_mutex);
+ clear_bit(bebob->card_index, devices_used);
+ mutex_unlock(&devices_mutex);
- kfree(bebob->maudio_special_quirk);
-
- if (bebob->card_index >= 0) {
- mutex_lock(&devices_mutex);
- clear_bit(bebob->card_index, devices_used);
- mutex_unlock(&devices_mutex);
- }
-
- mutex_destroy(&bebob->mutex);
+ bebob_free(card->private_data);
}
static const struct snd_bebob_spec *
@@ -176,16 +180,17 @@
return strncmp(name, "FW Audiophile Bootloader", 15) != 0;
}
-static int
-bebob_probe(struct fw_unit *unit,
- const struct ieee1394_device_id *entry)
+static void
+do_registration(struct work_struct *work)
{
- struct snd_card *card;
- struct snd_bebob *bebob;
- const struct snd_bebob_spec *spec;
+ struct snd_bebob *bebob =
+ container_of(work, struct snd_bebob, dwork.work);
unsigned int card_index;
int err;
+ if (bebob->registered)
+ return;
+
mutex_lock(&devices_mutex);
for (card_index = 0; card_index < SNDRV_CARDS; card_index++) {
@@ -193,64 +198,39 @@
break;
}
if (card_index >= SNDRV_CARDS) {
- err = -ENOENT;
- goto end;
+ mutex_unlock(&devices_mutex);
+ return;
}
- if ((entry->vendor_id == VEN_FOCUSRITE) &&
- (entry->model_id == MODEL_FOCUSRITE_SAFFIRE_BOTH))
- spec = get_saffire_spec(unit);
- else if ((entry->vendor_id == VEN_MAUDIO1) &&
- (entry->model_id == MODEL_MAUDIO_AUDIOPHILE_BOTH) &&
- !check_audiophile_booted(unit))
- spec = NULL;
- else
- spec = (const struct snd_bebob_spec *)entry->driver_data;
-
- if (spec == NULL) {
- if ((entry->vendor_id == VEN_MAUDIO1) ||
- (entry->vendor_id == VEN_MAUDIO2))
- err = snd_bebob_maudio_load_firmware(unit);
- else
- err = -ENOSYS;
- goto end;
+ err = snd_card_new(&bebob->unit->device, index[card_index],
+ id[card_index], THIS_MODULE, 0, &bebob->card);
+ if (err < 0) {
+ mutex_unlock(&devices_mutex);
+ return;
}
- err = snd_card_new(&unit->device, index[card_index], id[card_index],
- THIS_MODULE, sizeof(struct snd_bebob), &card);
- if (err < 0)
- goto end;
- bebob = card->private_data;
- bebob->card_index = card_index;
- set_bit(card_index, devices_used);
- card->private_free = bebob_card_free;
-
- bebob->card = card;
- bebob->unit = fw_unit_get(unit);
- bebob->spec = spec;
- mutex_init(&bebob->mutex);
- spin_lock_init(&bebob->lock);
- init_waitqueue_head(&bebob->hwdep_wait);
-
- err = name_device(bebob, entry->vendor_id);
+ err = name_device(bebob);
if (err < 0)
goto error;
- if ((entry->vendor_id == VEN_MAUDIO1) &&
- (entry->model_id == MODEL_MAUDIO_FW1814))
- err = snd_bebob_maudio_special_discover(bebob, true);
- else if ((entry->vendor_id == VEN_MAUDIO1) &&
- (entry->model_id == MODEL_MAUDIO_PROJECTMIX))
- err = snd_bebob_maudio_special_discover(bebob, false);
- else
+ if (bebob->spec == &maudio_special_spec) {
+ if (bebob->entry->model_id == MODEL_MAUDIO_FW1814)
+ err = snd_bebob_maudio_special_discover(bebob, true);
+ else
+ err = snd_bebob_maudio_special_discover(bebob, false);
+ } else {
err = snd_bebob_stream_discover(bebob);
+ }
+ if (err < 0)
+ goto error;
+
+ err = snd_bebob_stream_init_duplex(bebob);
if (err < 0)
goto error;
snd_bebob_proc_init(bebob);
- if ((bebob->midi_input_ports > 0) ||
- (bebob->midi_output_ports > 0)) {
+ if (bebob->midi_input_ports > 0 || bebob->midi_output_ports > 0) {
err = snd_bebob_create_midi_devices(bebob);
if (err < 0)
goto error;
@@ -264,16 +244,75 @@
if (err < 0)
goto error;
- err = snd_bebob_stream_init_duplex(bebob);
+ err = snd_card_register(bebob->card);
if (err < 0)
goto error;
- if (!bebob->maudio_special_quirk) {
- err = snd_card_register(card);
- if (err < 0) {
- snd_bebob_stream_destroy_duplex(bebob);
- goto error;
- }
+ set_bit(card_index, devices_used);
+ mutex_unlock(&devices_mutex);
+
+ /*
+ * After registered, bebob instance can be released corresponding to
+ * releasing the sound card instance.
+ */
+ bebob->card->private_free = bebob_card_free;
+ bebob->card->private_data = bebob;
+ bebob->registered = true;
+
+ return;
+error:
+ mutex_unlock(&devices_mutex);
+ snd_bebob_stream_destroy_duplex(bebob);
+ snd_card_free(bebob->card);
+ dev_info(&bebob->unit->device,
+ "Sound card registration failed: %d\n", err);
+}
+
+static int
+bebob_probe(struct fw_unit *unit, const struct ieee1394_device_id *entry)
+{
+ struct snd_bebob *bebob;
+ const struct snd_bebob_spec *spec;
+
+ if (entry->vendor_id == VEN_FOCUSRITE &&
+ entry->model_id == MODEL_FOCUSRITE_SAFFIRE_BOTH)
+ spec = get_saffire_spec(unit);
+ else if (entry->vendor_id == VEN_MAUDIO1 &&
+ entry->model_id == MODEL_MAUDIO_AUDIOPHILE_BOTH &&
+ !check_audiophile_booted(unit))
+ spec = NULL;
+ else
+ spec = (const struct snd_bebob_spec *)entry->driver_data;
+
+ if (spec == NULL) {
+ if (entry->vendor_id == VEN_MAUDIO1 ||
+ entry->vendor_id == VEN_MAUDIO2)
+ return snd_bebob_maudio_load_firmware(unit);
+ else
+ return -ENODEV;
+ }
+
+ /* Allocate this independent of sound card instance. */
+ bebob = kzalloc(sizeof(struct snd_bebob), GFP_KERNEL);
+ if (bebob == NULL)
+ return -ENOMEM;
+
+ bebob->unit = fw_unit_get(unit);
+ bebob->entry = entry;
+ bebob->spec = spec;
+ dev_set_drvdata(&unit->device, bebob);
+
+ mutex_init(&bebob->mutex);
+ spin_lock_init(&bebob->lock);
+ init_waitqueue_head(&bebob->hwdep_wait);
+
+ /* Allocate and register this sound card later. */
+ INIT_DEFERRABLE_WORK(&bebob->dwork, do_registration);
+
+ if (entry->vendor_id != VEN_MAUDIO1 ||
+ (entry->model_id != MODEL_MAUDIO_FW1814 &&
+ entry->model_id != MODEL_MAUDIO_PROJECTMIX)) {
+ snd_fw_schedule_registration(unit, &bebob->dwork);
} else {
/*
* This is a workaround. This bus reset seems to have an effect
@@ -285,19 +324,11 @@
* signals from dbus and starts I/Os. To avoid I/Os till the
* future bus reset, registration is done in next update().
*/
- bebob->deferred_registration = true;
fw_schedule_bus_reset(fw_parent_device(bebob->unit)->card,
false, true);
}
- dev_set_drvdata(&unit->device, bebob);
-end:
- mutex_unlock(&devices_mutex);
- return err;
-error:
- mutex_unlock(&devices_mutex);
- snd_card_free(card);
- return err;
+ return 0;
}
/*
@@ -324,15 +355,11 @@
if (bebob == NULL)
return;
- fcp_bus_reset(bebob->unit);
-
- if (bebob->deferred_registration) {
- if (snd_card_register(bebob->card) < 0) {
- snd_bebob_stream_destroy_duplex(bebob);
- snd_card_free(bebob->card);
- }
- bebob->deferred_registration = false;
- }
+ /* Postpone a workqueue for deferred registration. */
+ if (!bebob->registered)
+ snd_fw_schedule_registration(unit, &bebob->dwork);
+ else
+ fcp_bus_reset(bebob->unit);
}
static void bebob_remove(struct fw_unit *unit)
@@ -342,8 +369,20 @@
if (bebob == NULL)
return;
- /* No need to wait for releasing card object in this context. */
- snd_card_free_when_closed(bebob->card);
+ /*
+ * Confirm to stop the work for registration before the sound card is
+ * going to be released. The work is not scheduled again because bus
+ * reset handler is not called anymore.
+ */
+ cancel_delayed_work_sync(&bebob->dwork);
+
+ if (bebob->registered) {
+ /* No need to wait for releasing card object in this context. */
+ snd_card_free_when_closed(bebob->card);
+ } else {
+ /* Don't forget this case. */
+ bebob_free(bebob);
+ }
}
static const struct snd_bebob_rate_spec normal_rate_spec = {
diff --git a/sound/firewire/bebob/bebob.h b/sound/firewire/bebob/bebob.h
index b50bb33d..e7f1bb9 100644
--- a/sound/firewire/bebob/bebob.h
+++ b/sound/firewire/bebob/bebob.h
@@ -83,6 +83,10 @@
struct mutex mutex;
spinlock_t lock;
+ bool registered;
+ struct delayed_work dwork;
+
+ const struct ieee1394_device_id *entry;
const struct snd_bebob_spec *spec;
unsigned int midi_input_ports;
@@ -90,7 +94,6 @@
bool connected;
- struct amdtp_stream *master;
struct amdtp_stream tx_stream;
struct amdtp_stream rx_stream;
struct cmp_connection out_conn;
@@ -111,7 +114,6 @@
/* for M-Audio special devices */
void *maudio_special_quirk;
- bool deferred_registration;
/* For BeBoB version quirk. */
unsigned int version;
diff --git a/sound/firewire/bebob/bebob_stream.c b/sound/firewire/bebob/bebob_stream.c
index 77cbb02..4d3034a 100644
--- a/sound/firewire/bebob/bebob_stream.c
+++ b/sound/firewire/bebob/bebob_stream.c
@@ -484,30 +484,6 @@
}
static int
-get_sync_mode(struct snd_bebob *bebob, enum cip_flags *sync_mode)
-{
- enum snd_bebob_clock_type src;
- int err;
-
- err = snd_bebob_stream_get_clock_src(bebob, &src);
- if (err < 0)
- return err;
-
- switch (src) {
- case SND_BEBOB_CLOCK_TYPE_INTERNAL:
- case SND_BEBOB_CLOCK_TYPE_EXTERNAL:
- *sync_mode = CIP_SYNC_TO_DEVICE;
- break;
- default:
- case SND_BEBOB_CLOCK_TYPE_SYT:
- *sync_mode = 0;
- break;
- }
-
- return 0;
-}
-
-static int
start_stream(struct snd_bebob *bebob, struct amdtp_stream *stream,
unsigned int rate)
{
@@ -550,8 +526,6 @@
goto end;
}
- bebob->tx_stream.flags |= CIP_SKIP_INIT_DBC_CHECK;
-
/*
* BeBoB v3 transfers packets with these qurks:
* - In the beginning of streaming, the value of dbc is incremented
@@ -584,8 +558,6 @@
int snd_bebob_stream_start_duplex(struct snd_bebob *bebob, unsigned int rate)
{
const struct snd_bebob_rate_spec *rate_spec = bebob->spec->rate;
- struct amdtp_stream *master, *slave;
- enum cip_flags sync_mode;
unsigned int curr_rate;
int err = 0;
@@ -593,22 +565,11 @@
if (bebob->substreams_counter == 0)
goto end;
- err = get_sync_mode(bebob, &sync_mode);
- if (err < 0)
- goto end;
- if (sync_mode == CIP_SYNC_TO_DEVICE) {
- master = &bebob->tx_stream;
- slave = &bebob->rx_stream;
- } else {
- master = &bebob->rx_stream;
- slave = &bebob->tx_stream;
- }
-
/*
* Considering JACK/FFADO streaming:
* TODO: This can be removed hwdep functionality becomes popular.
*/
- err = check_connection_used_by_others(bebob, master);
+ err = check_connection_used_by_others(bebob, &bebob->rx_stream);
if (err < 0)
goto end;
@@ -618,11 +579,12 @@
* At bus reset, connections should not be broken here. So streams need
* to be re-started. This is a reason to use SKIP_INIT_DBC_CHECK flag.
*/
- if (amdtp_streaming_error(master))
- amdtp_stream_stop(master);
- if (amdtp_streaming_error(slave))
- amdtp_stream_stop(slave);
- if (!amdtp_stream_running(master) && !amdtp_stream_running(slave))
+ if (amdtp_streaming_error(&bebob->rx_stream))
+ amdtp_stream_stop(&bebob->rx_stream);
+ if (amdtp_streaming_error(&bebob->tx_stream))
+ amdtp_stream_stop(&bebob->tx_stream);
+ if (!amdtp_stream_running(&bebob->rx_stream) &&
+ !amdtp_stream_running(&bebob->tx_stream))
break_both_connections(bebob);
/* stop streams if rate is different */
@@ -635,16 +597,13 @@
if (rate == 0)
rate = curr_rate;
if (rate != curr_rate) {
- amdtp_stream_stop(master);
- amdtp_stream_stop(slave);
+ amdtp_stream_stop(&bebob->rx_stream);
+ amdtp_stream_stop(&bebob->tx_stream);
break_both_connections(bebob);
}
/* master should be always running */
- if (!amdtp_stream_running(master)) {
- amdtp_stream_set_sync(sync_mode, master, slave);
- bebob->master = master;
-
+ if (!amdtp_stream_running(&bebob->rx_stream)) {
/*
* NOTE:
* If establishing connections at first, Yamaha GO46
@@ -666,7 +625,7 @@
if (err < 0)
goto end;
- err = start_stream(bebob, master, rate);
+ err = start_stream(bebob, &bebob->rx_stream, rate);
if (err < 0) {
dev_err(&bebob->unit->device,
"fail to run AMDTP master stream:%d\n", err);
@@ -685,15 +644,16 @@
dev_err(&bebob->unit->device,
"fail to ensure sampling rate: %d\n",
err);
- amdtp_stream_stop(master);
+ amdtp_stream_stop(&bebob->rx_stream);
break_both_connections(bebob);
goto end;
}
}
/* wait first callback */
- if (!amdtp_stream_wait_callback(master, CALLBACK_TIMEOUT)) {
- amdtp_stream_stop(master);
+ if (!amdtp_stream_wait_callback(&bebob->rx_stream,
+ CALLBACK_TIMEOUT)) {
+ amdtp_stream_stop(&bebob->rx_stream);
break_both_connections(bebob);
err = -ETIMEDOUT;
goto end;
@@ -701,20 +661,21 @@
}
/* start slave if needed */
- if (!amdtp_stream_running(slave)) {
- err = start_stream(bebob, slave, rate);
+ if (!amdtp_stream_running(&bebob->tx_stream)) {
+ err = start_stream(bebob, &bebob->tx_stream, rate);
if (err < 0) {
dev_err(&bebob->unit->device,
"fail to run AMDTP slave stream:%d\n", err);
- amdtp_stream_stop(master);
+ amdtp_stream_stop(&bebob->rx_stream);
break_both_connections(bebob);
goto end;
}
/* wait first callback */
- if (!amdtp_stream_wait_callback(slave, CALLBACK_TIMEOUT)) {
- amdtp_stream_stop(slave);
- amdtp_stream_stop(master);
+ if (!amdtp_stream_wait_callback(&bebob->tx_stream,
+ CALLBACK_TIMEOUT)) {
+ amdtp_stream_stop(&bebob->tx_stream);
+ amdtp_stream_stop(&bebob->rx_stream);
break_both_connections(bebob);
err = -ETIMEDOUT;
}
@@ -725,22 +686,12 @@
void snd_bebob_stream_stop_duplex(struct snd_bebob *bebob)
{
- struct amdtp_stream *master, *slave;
-
- if (bebob->master == &bebob->rx_stream) {
- slave = &bebob->tx_stream;
- master = &bebob->rx_stream;
- } else {
- slave = &bebob->rx_stream;
- master = &bebob->tx_stream;
- }
-
if (bebob->substreams_counter == 0) {
- amdtp_stream_pcm_abort(master);
- amdtp_stream_stop(master);
+ amdtp_stream_pcm_abort(&bebob->rx_stream);
+ amdtp_stream_stop(&bebob->rx_stream);
- amdtp_stream_pcm_abort(slave);
- amdtp_stream_stop(slave);
+ amdtp_stream_pcm_abort(&bebob->tx_stream);
+ amdtp_stream_stop(&bebob->tx_stream);
break_both_connections(bebob);
}
diff --git a/sound/firewire/dice/dice.c b/sound/firewire/dice/dice.c
index 8b64aef..25e9f77 100644
--- a/sound/firewire/dice/dice.c
+++ b/sound/firewire/dice/dice.c
@@ -20,8 +20,6 @@
#define WEISS_CATEGORY_ID 0x00
#define LOUD_CATEGORY_ID 0x10
-#define PROBE_DELAY_MS (2 * MSEC_PER_SEC)
-
/*
* Some models support several isochronous channels, while these streams are not
* always available. In this case, add the model name to this list.
@@ -201,6 +199,10 @@
dice_card_strings(dice);
+ err = snd_dice_stream_init_duplex(dice);
+ if (err < 0)
+ goto error;
+
snd_dice_create_proc(dice);
err = snd_dice_create_pcm(dice);
@@ -229,28 +231,14 @@
return;
error:
+ snd_dice_stream_destroy_duplex(dice);
snd_dice_transaction_destroy(dice);
+ snd_dice_stream_destroy_duplex(dice);
snd_card_free(dice->card);
dev_info(&dice->unit->device,
"Sound card registration failed: %d\n", err);
}
-static void schedule_registration(struct snd_dice *dice)
-{
- struct fw_card *fw_card = fw_parent_device(dice->unit)->card;
- u64 now, delay;
-
- now = get_jiffies_64();
- delay = fw_card->reset_jiffies + msecs_to_jiffies(PROBE_DELAY_MS);
-
- if (time_after64(delay, now))
- delay -= now;
- else
- delay = 0;
-
- mod_delayed_work(system_wq, &dice->dwork, delay);
-}
-
static int dice_probe(struct fw_unit *unit, const struct ieee1394_device_id *id)
{
struct snd_dice *dice;
@@ -273,15 +261,9 @@
init_completion(&dice->clock_accepted);
init_waitqueue_head(&dice->hwdep_wait);
- err = snd_dice_stream_init_duplex(dice);
- if (err < 0) {
- dice_free(dice);
- return err;
- }
-
/* Allocate and register this sound card later. */
INIT_DEFERRABLE_WORK(&dice->dwork, do_registration);
- schedule_registration(dice);
+ snd_fw_schedule_registration(unit, &dice->dwork);
return 0;
}
@@ -312,7 +294,7 @@
/* Postpone a workqueue for deferred registration. */
if (!dice->registered)
- schedule_registration(dice);
+ snd_fw_schedule_registration(unit, &dice->dwork);
/* The handler address register becomes initialized. */
snd_dice_transaction_reinit(dice);
@@ -335,6 +317,13 @@
.match_flags = IEEE1394_MATCH_VERSION,
.version = DICE_INTERFACE,
},
+ /* M-Audio Profire 610/2626 has a different value in version field. */
+ {
+ .match_flags = IEEE1394_MATCH_VENDOR_ID |
+ IEEE1394_MATCH_SPECIFIER_ID,
+ .vendor_id = 0x000d6c,
+ .specifier_id = 0x000d6c,
+ },
{ }
};
MODULE_DEVICE_TABLE(ieee1394, dice_id_table);
diff --git a/sound/firewire/digi00x/amdtp-dot.c b/sound/firewire/digi00x/amdtp-dot.c
index 0ac92ab..b3cffd0 100644
--- a/sound/firewire/digi00x/amdtp-dot.c
+++ b/sound/firewire/digi00x/amdtp-dot.c
@@ -421,7 +421,7 @@
/* Use different mode between incoming/outgoing. */
if (dir == AMDTP_IN_STREAM) {
- flags = CIP_NONBLOCKING | CIP_SKIP_INIT_DBC_CHECK;
+ flags = CIP_NONBLOCKING;
process_data_blocks = process_tx_data_blocks;
} else {
flags = CIP_BLOCKING;
diff --git a/sound/firewire/digi00x/digi00x-transaction.c b/sound/firewire/digi00x/digi00x-transaction.c
index 554324d..735d356 100644
--- a/sound/firewire/digi00x/digi00x-transaction.c
+++ b/sound/firewire/digi00x/digi00x-transaction.c
@@ -126,12 +126,17 @@
return err;
error:
fw_core_remove_address_handler(&dg00x->async_handler);
- dg00x->async_handler.address_callback = NULL;
+ dg00x->async_handler.callback_data = NULL;
return err;
}
void snd_dg00x_transaction_unregister(struct snd_dg00x *dg00x)
{
+ if (dg00x->async_handler.callback_data == NULL)
+ return;
+
snd_fw_async_midi_port_destroy(&dg00x->out_control);
fw_core_remove_address_handler(&dg00x->async_handler);
+
+ dg00x->async_handler.callback_data = NULL;
}
diff --git a/sound/firewire/digi00x/digi00x.c b/sound/firewire/digi00x/digi00x.c
index 1f33b7a..cc4776c 100644
--- a/sound/firewire/digi00x/digi00x.c
+++ b/sound/firewire/digi00x/digi00x.c
@@ -40,10 +40,8 @@
return 0;
}
-static void dg00x_card_free(struct snd_card *card)
+static void dg00x_free(struct snd_dg00x *dg00x)
{
- struct snd_dg00x *dg00x = card->private_data;
-
snd_dg00x_stream_destroy_duplex(dg00x);
snd_dg00x_transaction_unregister(dg00x);
@@ -52,28 +50,24 @@
mutex_destroy(&dg00x->mutex);
}
-static int snd_dg00x_probe(struct fw_unit *unit,
- const struct ieee1394_device_id *entry)
+static void dg00x_card_free(struct snd_card *card)
{
- struct snd_card *card;
- struct snd_dg00x *dg00x;
+ dg00x_free(card->private_data);
+}
+
+static void do_registration(struct work_struct *work)
+{
+ struct snd_dg00x *dg00x =
+ container_of(work, struct snd_dg00x, dwork.work);
int err;
- /* create card */
- err = snd_card_new(&unit->device, -1, NULL, THIS_MODULE,
- sizeof(struct snd_dg00x), &card);
+ if (dg00x->registered)
+ return;
+
+ err = snd_card_new(&dg00x->unit->device, -1, NULL, THIS_MODULE, 0,
+ &dg00x->card);
if (err < 0)
- return err;
- card->private_free = dg00x_card_free;
-
- /* initialize myself */
- dg00x = card->private_data;
- dg00x->card = card;
- dg00x->unit = fw_unit_get(unit);
-
- mutex_init(&dg00x->mutex);
- spin_lock_init(&dg00x->lock);
- init_waitqueue_head(&dg00x->hwdep_wait);
+ return;
err = name_card(dg00x);
if (err < 0)
@@ -101,35 +95,86 @@
if (err < 0)
goto error;
- err = snd_card_register(card);
+ err = snd_card_register(dg00x->card);
if (err < 0)
goto error;
+ dg00x->card->private_free = dg00x_card_free;
+ dg00x->card->private_data = dg00x;
+ dg00x->registered = true;
+
+ return;
+error:
+ snd_dg00x_transaction_unregister(dg00x);
+ snd_dg00x_stream_destroy_duplex(dg00x);
+ snd_card_free(dg00x->card);
+ dev_info(&dg00x->unit->device,
+ "Sound card registration failed: %d\n", err);
+}
+
+static int snd_dg00x_probe(struct fw_unit *unit,
+ const struct ieee1394_device_id *entry)
+{
+ struct snd_dg00x *dg00x;
+
+ /* Allocate this independent of sound card instance. */
+ dg00x = kzalloc(sizeof(struct snd_dg00x), GFP_KERNEL);
+ if (dg00x == NULL)
+ return -ENOMEM;
+
+ dg00x->unit = fw_unit_get(unit);
dev_set_drvdata(&unit->device, dg00x);
- return err;
-error:
- snd_card_free(card);
- return err;
+ mutex_init(&dg00x->mutex);
+ spin_lock_init(&dg00x->lock);
+ init_waitqueue_head(&dg00x->hwdep_wait);
+
+ /* Allocate and register this sound card later. */
+ INIT_DEFERRABLE_WORK(&dg00x->dwork, do_registration);
+ snd_fw_schedule_registration(unit, &dg00x->dwork);
+
+ return 0;
}
static void snd_dg00x_update(struct fw_unit *unit)
{
struct snd_dg00x *dg00x = dev_get_drvdata(&unit->device);
+ /* Postpone a workqueue for deferred registration. */
+ if (!dg00x->registered)
+ snd_fw_schedule_registration(unit, &dg00x->dwork);
+
snd_dg00x_transaction_reregister(dg00x);
- mutex_lock(&dg00x->mutex);
- snd_dg00x_stream_update_duplex(dg00x);
- mutex_unlock(&dg00x->mutex);
+ /*
+ * After registration, userspace can start packet streaming, then this
+ * code block works fine.
+ */
+ if (dg00x->registered) {
+ mutex_lock(&dg00x->mutex);
+ snd_dg00x_stream_update_duplex(dg00x);
+ mutex_unlock(&dg00x->mutex);
+ }
}
static void snd_dg00x_remove(struct fw_unit *unit)
{
struct snd_dg00x *dg00x = dev_get_drvdata(&unit->device);
- /* No need to wait for releasing card object in this context. */
- snd_card_free_when_closed(dg00x->card);
+ /*
+ * Confirm to stop the work for registration before the sound card is
+ * going to be released. The work is not scheduled again because bus
+ * reset handler is not called anymore.
+ */
+ cancel_delayed_work_sync(&dg00x->dwork);
+
+ if (dg00x->registered) {
+ /* No need to wait for releasing card object in this context. */
+ snd_card_free_when_closed(dg00x->card);
+ } else {
+ /* Don't forget this case. */
+ dg00x_free(dg00x);
+ }
}
static const struct ieee1394_device_id snd_dg00x_id_table[] = {
diff --git a/sound/firewire/digi00x/digi00x.h b/sound/firewire/digi00x/digi00x.h
index 907e739..2cd465c0 100644
--- a/sound/firewire/digi00x/digi00x.h
+++ b/sound/firewire/digi00x/digi00x.h
@@ -37,6 +37,9 @@
struct mutex mutex;
spinlock_t lock;
+ bool registered;
+ struct delayed_work dwork;
+
struct amdtp_stream tx_stream;
struct fw_iso_resources tx_resources;
diff --git a/sound/firewire/fireworks/fireworks.c b/sound/firewire/fireworks/fireworks.c
index 8f27b67..71a0613 100644
--- a/sound/firewire/fireworks/fireworks.c
+++ b/sound/firewire/fireworks/fireworks.c
@@ -168,11 +168,34 @@
sizeof(struct snd_efw_phys_grp) * hwinfo->phys_in_grp_count);
memcpy(&efw->phys_out_grps, hwinfo->phys_out_grps,
sizeof(struct snd_efw_phys_grp) * hwinfo->phys_out_grp_count);
+
+ /* AudioFire8 (since 2009) and AudioFirePre8 */
+ if (hwinfo->type == MODEL_ECHO_AUDIOFIRE_9)
+ efw->is_af9 = true;
+ /* These models uses the same firmware. */
+ if (hwinfo->type == MODEL_ECHO_AUDIOFIRE_2 ||
+ hwinfo->type == MODEL_ECHO_AUDIOFIRE_4 ||
+ hwinfo->type == MODEL_ECHO_AUDIOFIRE_9 ||
+ hwinfo->type == MODEL_GIBSON_RIP ||
+ hwinfo->type == MODEL_GIBSON_GOLDTOP)
+ efw->is_fireworks3 = true;
end:
kfree(hwinfo);
return err;
}
+static void efw_free(struct snd_efw *efw)
+{
+ snd_efw_stream_destroy_duplex(efw);
+ snd_efw_transaction_remove_instance(efw);
+ fw_unit_put(efw->unit);
+
+ kfree(efw->resp_buf);
+
+ mutex_destroy(&efw->mutex);
+ kfree(efw);
+}
+
/*
* This module releases the FireWire unit data after all ALSA character devices
* are released by applications. This is for releasing stream data or finishing
@@ -184,28 +207,24 @@
{
struct snd_efw *efw = card->private_data;
- snd_efw_stream_destroy_duplex(efw);
- snd_efw_transaction_remove_instance(efw);
- fw_unit_put(efw->unit);
-
- kfree(efw->resp_buf);
-
if (efw->card_index >= 0) {
mutex_lock(&devices_mutex);
clear_bit(efw->card_index, devices_used);
mutex_unlock(&devices_mutex);
}
- mutex_destroy(&efw->mutex);
+ efw_free(card->private_data);
}
-static int
-efw_probe(struct fw_unit *unit,
- const struct ieee1394_device_id *entry)
+static void
+do_registration(struct work_struct *work)
{
- struct snd_card *card;
- struct snd_efw *efw;
- int card_index, err;
+ struct snd_efw *efw = container_of(work, struct snd_efw, dwork.work);
+ unsigned int card_index;
+ int err;
+
+ if (efw->registered)
+ return;
mutex_lock(&devices_mutex);
@@ -215,24 +234,16 @@
break;
}
if (card_index >= SNDRV_CARDS) {
- err = -ENOENT;
- goto end;
+ mutex_unlock(&devices_mutex);
+ return;
}
- err = snd_card_new(&unit->device, index[card_index], id[card_index],
- THIS_MODULE, sizeof(struct snd_efw), &card);
- if (err < 0)
- goto end;
- efw = card->private_data;
- efw->card_index = card_index;
- set_bit(card_index, devices_used);
- card->private_free = efw_card_free;
-
- efw->card = card;
- efw->unit = fw_unit_get(unit);
- mutex_init(&efw->mutex);
- spin_lock_init(&efw->lock);
- init_waitqueue_head(&efw->hwdep_wait);
+ err = snd_card_new(&efw->unit->device, index[card_index],
+ id[card_index], THIS_MODULE, 0, &efw->card);
+ if (err < 0) {
+ mutex_unlock(&devices_mutex);
+ return;
+ }
/* prepare response buffer */
snd_efw_resp_buf_size = clamp(snd_efw_resp_buf_size,
@@ -248,16 +259,10 @@
err = get_hardware_info(efw);
if (err < 0)
goto error;
- /* AudioFire8 (since 2009) and AudioFirePre8 */
- if (entry->model_id == MODEL_ECHO_AUDIOFIRE_9)
- efw->is_af9 = true;
- /* These models uses the same firmware. */
- if (entry->model_id == MODEL_ECHO_AUDIOFIRE_2 ||
- entry->model_id == MODEL_ECHO_AUDIOFIRE_4 ||
- entry->model_id == MODEL_ECHO_AUDIOFIRE_9 ||
- entry->model_id == MODEL_GIBSON_RIP ||
- entry->model_id == MODEL_GIBSON_GOLDTOP)
- efw->is_fireworks3 = true;
+
+ err = snd_efw_stream_init_duplex(efw);
+ if (err < 0)
+ goto error;
snd_efw_proc_init(efw);
@@ -275,44 +280,93 @@
if (err < 0)
goto error;
- err = snd_efw_stream_init_duplex(efw);
+ err = snd_card_register(efw->card);
if (err < 0)
goto error;
- err = snd_card_register(card);
- if (err < 0) {
- snd_efw_stream_destroy_duplex(efw);
- goto error;
- }
+ set_bit(card_index, devices_used);
+ mutex_unlock(&devices_mutex);
- dev_set_drvdata(&unit->device, efw);
-end:
- mutex_unlock(&devices_mutex);
- return err;
+ /*
+ * After registered, efw instance can be released corresponding to
+ * releasing the sound card instance.
+ */
+ efw->card->private_free = efw_card_free;
+ efw->card->private_data = efw;
+ efw->registered = true;
+
+ return;
error:
- snd_efw_transaction_remove_instance(efw);
mutex_unlock(&devices_mutex);
- snd_card_free(card);
- return err;
+ snd_efw_transaction_remove_instance(efw);
+ snd_efw_stream_destroy_duplex(efw);
+ snd_card_free(efw->card);
+ dev_info(&efw->unit->device,
+ "Sound card registration failed: %d\n", err);
+}
+
+static int
+efw_probe(struct fw_unit *unit, const struct ieee1394_device_id *entry)
+{
+ struct snd_efw *efw;
+
+ efw = kzalloc(sizeof(struct snd_efw), GFP_KERNEL);
+ if (efw == NULL)
+ return -ENOMEM;
+
+ efw->unit = fw_unit_get(unit);
+ dev_set_drvdata(&unit->device, efw);
+
+ mutex_init(&efw->mutex);
+ spin_lock_init(&efw->lock);
+ init_waitqueue_head(&efw->hwdep_wait);
+
+ /* Allocate and register this sound card later. */
+ INIT_DEFERRABLE_WORK(&efw->dwork, do_registration);
+ snd_fw_schedule_registration(unit, &efw->dwork);
+
+ return 0;
}
static void efw_update(struct fw_unit *unit)
{
struct snd_efw *efw = dev_get_drvdata(&unit->device);
+ /* Postpone a workqueue for deferred registration. */
+ if (!efw->registered)
+ snd_fw_schedule_registration(unit, &efw->dwork);
+
snd_efw_transaction_bus_reset(efw->unit);
- mutex_lock(&efw->mutex);
- snd_efw_stream_update_duplex(efw);
- mutex_unlock(&efw->mutex);
+ /*
+ * After registration, userspace can start packet streaming, then this
+ * code block works fine.
+ */
+ if (efw->registered) {
+ mutex_lock(&efw->mutex);
+ snd_efw_stream_update_duplex(efw);
+ mutex_unlock(&efw->mutex);
+ }
}
static void efw_remove(struct fw_unit *unit)
{
struct snd_efw *efw = dev_get_drvdata(&unit->device);
- /* No need to wait for releasing card object in this context. */
- snd_card_free_when_closed(efw->card);
+ /*
+ * Confirm to stop the work for registration before the sound card is
+ * going to be released. The work is not scheduled again because bus
+ * reset handler is not called anymore.
+ */
+ cancel_delayed_work_sync(&efw->dwork);
+
+ if (efw->registered) {
+ /* No need to wait for releasing card object in this context. */
+ snd_card_free_when_closed(efw->card);
+ } else {
+ /* Don't forget this case. */
+ efw_free(efw);
+ }
}
static const struct ieee1394_device_id efw_id_table[] = {
diff --git a/sound/firewire/fireworks/fireworks.h b/sound/firewire/fireworks/fireworks.h
index 96c4e0c..03ed352 100644
--- a/sound/firewire/fireworks/fireworks.h
+++ b/sound/firewire/fireworks/fireworks.h
@@ -65,6 +65,9 @@
struct mutex mutex;
spinlock_t lock;
+ bool registered;
+ struct delayed_work dwork;
+
/* for transaction */
u32 seqnum;
bool resp_addr_changable;
@@ -81,7 +84,6 @@
unsigned int pcm_capture_channels[SND_EFW_MULTIPLIER_MODES];
unsigned int pcm_playback_channels[SND_EFW_MULTIPLIER_MODES];
- struct amdtp_stream *master;
struct amdtp_stream tx_stream;
struct amdtp_stream rx_stream;
struct cmp_connection out_conn;
diff --git a/sound/firewire/fireworks/fireworks_stream.c b/sound/firewire/fireworks/fireworks_stream.c
index 425db8d..ee47924 100644
--- a/sound/firewire/fireworks/fireworks_stream.c
+++ b/sound/firewire/fireworks/fireworks_stream.c
@@ -121,23 +121,6 @@
}
static int
-get_sync_mode(struct snd_efw *efw, enum cip_flags *sync_mode)
-{
- enum snd_efw_clock_source clock_source;
- int err;
-
- err = snd_efw_command_get_clock_source(efw, &clock_source);
- if (err < 0)
- return err;
-
- if (clock_source == SND_EFW_CLOCK_SOURCE_SYTMATCH)
- return -ENOSYS;
-
- *sync_mode = CIP_SYNC_TO_DEVICE;
- return 0;
-}
-
-static int
check_connection_used_by_others(struct snd_efw *efw, struct amdtp_stream *s)
{
struct cmp_connection *conn;
@@ -208,9 +191,6 @@
int snd_efw_stream_start_duplex(struct snd_efw *efw, unsigned int rate)
{
- struct amdtp_stream *master, *slave;
- unsigned int slave_substreams;
- enum cip_flags sync_mode;
unsigned int curr_rate;
int err = 0;
@@ -218,32 +198,19 @@
if (efw->playback_substreams == 0 && efw->capture_substreams == 0)
goto end;
- err = get_sync_mode(efw, &sync_mode);
- if (err < 0)
- goto end;
- if (sync_mode == CIP_SYNC_TO_DEVICE) {
- master = &efw->tx_stream;
- slave = &efw->rx_stream;
- slave_substreams = efw->playback_substreams;
- } else {
- master = &efw->rx_stream;
- slave = &efw->tx_stream;
- slave_substreams = efw->capture_substreams;
- }
-
/*
* Considering JACK/FFADO streaming:
* TODO: This can be removed hwdep functionality becomes popular.
*/
- err = check_connection_used_by_others(efw, master);
+ err = check_connection_used_by_others(efw, &efw->rx_stream);
if (err < 0)
goto end;
/* packet queueing error */
- if (amdtp_streaming_error(slave))
- stop_stream(efw, slave);
- if (amdtp_streaming_error(master))
- stop_stream(efw, master);
+ if (amdtp_streaming_error(&efw->tx_stream))
+ stop_stream(efw, &efw->tx_stream);
+ if (amdtp_streaming_error(&efw->rx_stream))
+ stop_stream(efw, &efw->rx_stream);
/* stop streams if rate is different */
err = snd_efw_command_get_sampling_rate(efw, &curr_rate);
@@ -252,20 +219,17 @@
if (rate == 0)
rate = curr_rate;
if (rate != curr_rate) {
- stop_stream(efw, slave);
- stop_stream(efw, master);
+ stop_stream(efw, &efw->tx_stream);
+ stop_stream(efw, &efw->rx_stream);
}
/* master should be always running */
- if (!amdtp_stream_running(master)) {
- amdtp_stream_set_sync(sync_mode, master, slave);
- efw->master = master;
-
+ if (!amdtp_stream_running(&efw->rx_stream)) {
err = snd_efw_command_set_sampling_rate(efw, rate);
if (err < 0)
goto end;
- err = start_stream(efw, master, rate);
+ err = start_stream(efw, &efw->rx_stream, rate);
if (err < 0) {
dev_err(&efw->unit->device,
"fail to start AMDTP master stream:%d\n", err);
@@ -274,12 +238,13 @@
}
/* start slave if needed */
- if (slave_substreams > 0 && !amdtp_stream_running(slave)) {
- err = start_stream(efw, slave, rate);
+ if (efw->capture_substreams > 0 &&
+ !amdtp_stream_running(&efw->tx_stream)) {
+ err = start_stream(efw, &efw->tx_stream, rate);
if (err < 0) {
dev_err(&efw->unit->device,
"fail to start AMDTP slave stream:%d\n", err);
- stop_stream(efw, master);
+ stop_stream(efw, &efw->rx_stream);
}
}
end:
@@ -288,26 +253,11 @@
void snd_efw_stream_stop_duplex(struct snd_efw *efw)
{
- struct amdtp_stream *master, *slave;
- unsigned int master_substreams, slave_substreams;
+ if (efw->capture_substreams == 0) {
+ stop_stream(efw, &efw->tx_stream);
- if (efw->master == &efw->rx_stream) {
- slave = &efw->tx_stream;
- master = &efw->rx_stream;
- slave_substreams = efw->capture_substreams;
- master_substreams = efw->playback_substreams;
- } else {
- slave = &efw->rx_stream;
- master = &efw->tx_stream;
- slave_substreams = efw->playback_substreams;
- master_substreams = efw->capture_substreams;
- }
-
- if (slave_substreams == 0) {
- stop_stream(efw, slave);
-
- if (master_substreams == 0)
- stop_stream(efw, master);
+ if (efw->playback_substreams == 0)
+ stop_stream(efw, &efw->rx_stream);
}
}
diff --git a/sound/firewire/lib.c b/sound/firewire/lib.c
index f80aafa..ca4dfcf 100644
--- a/sound/firewire/lib.c
+++ b/sound/firewire/lib.c
@@ -67,6 +67,38 @@
}
EXPORT_SYMBOL(snd_fw_transaction);
+#define PROBE_DELAY_MS (2 * MSEC_PER_SEC)
+
+/**
+ * snd_fw_schedule_registration - schedule work for sound card registration
+ * @unit: an instance for unit on IEEE 1394 bus
+ * @dwork: delayed work with callback function
+ *
+ * This function is not designed for general purposes. When new unit is
+ * connected to IEEE 1394 bus, the bus is under bus-reset state because of
+ * topological change. In this state, units tend to fail both of asynchronous
+ * and isochronous communication. To avoid this problem, this function is used
+ * to postpone sound card registration after the state. The callers must
+ * set up instance of delayed work in advance.
+ */
+void snd_fw_schedule_registration(struct fw_unit *unit,
+ struct delayed_work *dwork)
+{
+ u64 now, delay;
+
+ now = get_jiffies_64();
+ delay = fw_parent_device(unit)->card->reset_jiffies
+ + msecs_to_jiffies(PROBE_DELAY_MS);
+
+ if (time_after64(delay, now))
+ delay -= now;
+ else
+ delay = 0;
+
+ mod_delayed_work(system_wq, dwork, delay);
+}
+EXPORT_SYMBOL(snd_fw_schedule_registration);
+
static void async_midi_port_callback(struct fw_card *card, int rcode,
void *data, size_t length,
void *callback_data)
diff --git a/sound/firewire/lib.h b/sound/firewire/lib.h
index f3f6f84..f676931 100644
--- a/sound/firewire/lib.h
+++ b/sound/firewire/lib.h
@@ -22,6 +22,9 @@
return rcode == RCODE_TYPE_ERROR || rcode == RCODE_ADDRESS_ERROR;
}
+void snd_fw_schedule_registration(struct fw_unit *unit,
+ struct delayed_work *dwork);
+
struct snd_fw_async_midi_port;
typedef int (*snd_fw_async_midi_port_fill)(
struct snd_rawmidi_substream *substream,
diff --git a/sound/firewire/oxfw/oxfw-stream.c b/sound/firewire/oxfw/oxfw-stream.c
index 7cb5743..d9361f3 100644
--- a/sound/firewire/oxfw/oxfw-stream.c
+++ b/sound/firewire/oxfw/oxfw-stream.c
@@ -242,8 +242,7 @@
* blocks than IEC 61883-6 defines.
*/
if (stream == &oxfw->tx_stream) {
- oxfw->tx_stream.flags |= CIP_SKIP_INIT_DBC_CHECK |
- CIP_JUMBO_PAYLOAD;
+ oxfw->tx_stream.flags |= CIP_JUMBO_PAYLOAD;
if (oxfw->wrong_dbs)
oxfw->tx_stream.flags |= CIP_WRONG_DBS;
}
diff --git a/sound/firewire/oxfw/oxfw.c b/sound/firewire/oxfw/oxfw.c
index abedc22..e629b88 100644
--- a/sound/firewire/oxfw/oxfw.c
+++ b/sound/firewire/oxfw/oxfw.c
@@ -118,15 +118,8 @@
return err;
}
-/*
- * This module releases the FireWire unit data after all ALSA character devices
- * are released by applications. This is for releasing stream data or finishing
- * transactions safely. Thus at returning from .remove(), this module still keep
- * references for the unit.
- */
-static void oxfw_card_free(struct snd_card *card)
+static void oxfw_free(struct snd_oxfw *oxfw)
{
- struct snd_oxfw *oxfw = card->private_data;
unsigned int i;
snd_oxfw_stream_destroy_simplex(oxfw, &oxfw->rx_stream);
@@ -144,6 +137,17 @@
mutex_destroy(&oxfw->mutex);
}
+/*
+ * This module releases the FireWire unit data after all ALSA character devices
+ * are released by applications. This is for releasing stream data or finishing
+ * transactions safely. Thus at returning from .remove(), this module still keep
+ * references for the unit.
+ */
+static void oxfw_card_free(struct snd_card *card)
+{
+ oxfw_free(card->private_data);
+}
+
static int detect_quirks(struct snd_oxfw *oxfw)
{
struct fw_device *fw_dev = fw_parent_device(oxfw->unit);
@@ -205,33 +209,18 @@
return 0;
}
-static int oxfw_probe(struct fw_unit *unit,
- const struct ieee1394_device_id *entry)
+static void do_registration(struct work_struct *work)
{
- struct snd_card *card;
- struct snd_oxfw *oxfw;
+ struct snd_oxfw *oxfw = container_of(work, struct snd_oxfw, dwork.work);
int err;
- if (entry->vendor_id == VENDOR_LOUD && !detect_loud_models(unit))
- return -ENODEV;
+ if (oxfw->registered)
+ return;
- err = snd_card_new(&unit->device, -1, NULL, THIS_MODULE,
- sizeof(*oxfw), &card);
+ err = snd_card_new(&oxfw->unit->device, -1, NULL, THIS_MODULE, 0,
+ &oxfw->card);
if (err < 0)
- return err;
-
- card->private_free = oxfw_card_free;
- oxfw = card->private_data;
- oxfw->card = card;
- mutex_init(&oxfw->mutex);
- oxfw->unit = fw_unit_get(unit);
- oxfw->entry = entry;
- spin_lock_init(&oxfw->lock);
- init_waitqueue_head(&oxfw->hwdep_wait);
-
- err = snd_oxfw_stream_discover(oxfw);
- if (err < 0)
- goto error;
+ return;
err = name_card(oxfw);
if (err < 0)
@@ -241,6 +230,19 @@
if (err < 0)
goto error;
+ err = snd_oxfw_stream_discover(oxfw);
+ if (err < 0)
+ goto error;
+
+ err = snd_oxfw_stream_init_simplex(oxfw, &oxfw->rx_stream);
+ if (err < 0)
+ goto error;
+ if (oxfw->has_output) {
+ err = snd_oxfw_stream_init_simplex(oxfw, &oxfw->tx_stream);
+ if (err < 0)
+ goto error;
+ }
+
err = snd_oxfw_create_pcm(oxfw);
if (err < 0)
goto error;
@@ -255,54 +257,97 @@
if (err < 0)
goto error;
- err = snd_oxfw_stream_init_simplex(oxfw, &oxfw->rx_stream);
+ err = snd_card_register(oxfw->card);
if (err < 0)
goto error;
- if (oxfw->has_output) {
- err = snd_oxfw_stream_init_simplex(oxfw, &oxfw->tx_stream);
- if (err < 0)
- goto error;
- }
- err = snd_card_register(card);
- if (err < 0) {
- snd_oxfw_stream_destroy_simplex(oxfw, &oxfw->rx_stream);
- if (oxfw->has_output)
- snd_oxfw_stream_destroy_simplex(oxfw, &oxfw->tx_stream);
- goto error;
- }
+ /*
+ * After registered, oxfw instance can be released corresponding to
+ * releasing the sound card instance.
+ */
+ oxfw->card->private_free = oxfw_card_free;
+ oxfw->card->private_data = oxfw;
+ oxfw->registered = true;
+
+ return;
+error:
+ snd_oxfw_stream_destroy_simplex(oxfw, &oxfw->rx_stream);
+ if (oxfw->has_output)
+ snd_oxfw_stream_destroy_simplex(oxfw, &oxfw->tx_stream);
+ snd_card_free(oxfw->card);
+ dev_info(&oxfw->unit->device,
+ "Sound card registration failed: %d\n", err);
+}
+
+static int oxfw_probe(struct fw_unit *unit,
+ const struct ieee1394_device_id *entry)
+{
+ struct snd_oxfw *oxfw;
+
+ if (entry->vendor_id == VENDOR_LOUD && !detect_loud_models(unit))
+ return -ENODEV;
+
+ /* Allocate this independent of sound card instance. */
+ oxfw = kzalloc(sizeof(struct snd_oxfw), GFP_KERNEL);
+ if (oxfw == NULL)
+ return -ENOMEM;
+
+ oxfw->entry = entry;
+ oxfw->unit = fw_unit_get(unit);
dev_set_drvdata(&unit->device, oxfw);
+ mutex_init(&oxfw->mutex);
+ spin_lock_init(&oxfw->lock);
+ init_waitqueue_head(&oxfw->hwdep_wait);
+
+ /* Allocate and register this sound card later. */
+ INIT_DEFERRABLE_WORK(&oxfw->dwork, do_registration);
+ snd_fw_schedule_registration(unit, &oxfw->dwork);
+
return 0;
-error:
- snd_card_free(card);
- return err;
}
static void oxfw_bus_reset(struct fw_unit *unit)
{
struct snd_oxfw *oxfw = dev_get_drvdata(&unit->device);
+ if (!oxfw->registered)
+ snd_fw_schedule_registration(unit, &oxfw->dwork);
+
fcp_bus_reset(oxfw->unit);
- mutex_lock(&oxfw->mutex);
+ if (oxfw->registered) {
+ mutex_lock(&oxfw->mutex);
- snd_oxfw_stream_update_simplex(oxfw, &oxfw->rx_stream);
- if (oxfw->has_output)
- snd_oxfw_stream_update_simplex(oxfw, &oxfw->tx_stream);
+ snd_oxfw_stream_update_simplex(oxfw, &oxfw->rx_stream);
+ if (oxfw->has_output)
+ snd_oxfw_stream_update_simplex(oxfw, &oxfw->tx_stream);
- mutex_unlock(&oxfw->mutex);
+ mutex_unlock(&oxfw->mutex);
- if (oxfw->entry->vendor_id == OUI_STANTON)
- snd_oxfw_scs1x_update(oxfw);
+ if (oxfw->entry->vendor_id == OUI_STANTON)
+ snd_oxfw_scs1x_update(oxfw);
+ }
}
static void oxfw_remove(struct fw_unit *unit)
{
struct snd_oxfw *oxfw = dev_get_drvdata(&unit->device);
- /* No need to wait for releasing card object in this context. */
- snd_card_free_when_closed(oxfw->card);
+ /*
+ * Confirm to stop the work for registration before the sound card is
+ * going to be released. The work is not scheduled again because bus
+ * reset handler is not called anymore.
+ */
+ cancel_delayed_work_sync(&oxfw->dwork);
+
+ if (oxfw->registered) {
+ /* No need to wait for releasing card object in this context. */
+ snd_card_free_when_closed(oxfw->card);
+ } else {
+ /* Don't forget this case. */
+ oxfw_free(oxfw);
+ }
}
static const struct compat_info griffin_firewave = {
diff --git a/sound/firewire/oxfw/oxfw.h b/sound/firewire/oxfw/oxfw.h
index 9beecc2..2047dcb 100644
--- a/sound/firewire/oxfw/oxfw.h
+++ b/sound/firewire/oxfw/oxfw.h
@@ -36,10 +36,12 @@
struct snd_oxfw {
struct snd_card *card;
struct fw_unit *unit;
- const struct device_info *device_info;
struct mutex mutex;
spinlock_t lock;
+ bool registered;
+ struct delayed_work dwork;
+
bool wrong_dbs;
bool has_output;
u8 *tx_stream_formats[SND_OXFW_STREAM_FORMAT_ENTRIES];
diff --git a/sound/firewire/tascam/tascam-stream.c b/sound/firewire/tascam/tascam-stream.c
index 0e6dd5c6..4ad3bd7 100644
--- a/sound/firewire/tascam/tascam-stream.c
+++ b/sound/firewire/tascam/tascam-stream.c
@@ -381,19 +381,17 @@
if (err < 0)
return err;
if (curr_rate != rate ||
- amdtp_streaming_error(&tscm->tx_stream) ||
- amdtp_streaming_error(&tscm->rx_stream)) {
+ amdtp_streaming_error(&tscm->rx_stream) ||
+ amdtp_streaming_error(&tscm->tx_stream)) {
finish_session(tscm);
- amdtp_stream_stop(&tscm->tx_stream);
amdtp_stream_stop(&tscm->rx_stream);
+ amdtp_stream_stop(&tscm->tx_stream);
release_resources(tscm);
}
- if (!amdtp_stream_running(&tscm->tx_stream)) {
- amdtp_stream_set_sync(CIP_SYNC_TO_DEVICE,
- &tscm->tx_stream, &tscm->rx_stream);
+ if (!amdtp_stream_running(&tscm->rx_stream)) {
err = keep_resources(tscm, rate);
if (err < 0)
goto error;
@@ -406,20 +404,6 @@
if (err < 0)
goto error;
- err = amdtp_stream_start(&tscm->tx_stream,
- tscm->tx_resources.channel,
- fw_parent_device(tscm->unit)->max_speed);
- if (err < 0)
- goto error;
-
- if (!amdtp_stream_wait_callback(&tscm->tx_stream,
- CALLBACK_TIMEOUT)) {
- err = -ETIMEDOUT;
- goto error;
- }
- }
-
- if (!amdtp_stream_running(&tscm->rx_stream)) {
err = amdtp_stream_start(&tscm->rx_stream,
tscm->rx_resources.channel,
fw_parent_device(tscm->unit)->max_speed);
@@ -433,10 +417,24 @@
}
}
+ if (!amdtp_stream_running(&tscm->tx_stream)) {
+ err = amdtp_stream_start(&tscm->tx_stream,
+ tscm->tx_resources.channel,
+ fw_parent_device(tscm->unit)->max_speed);
+ if (err < 0)
+ goto error;
+
+ if (!amdtp_stream_wait_callback(&tscm->tx_stream,
+ CALLBACK_TIMEOUT)) {
+ err = -ETIMEDOUT;
+ goto error;
+ }
+ }
+
return 0;
error:
- amdtp_stream_stop(&tscm->tx_stream);
amdtp_stream_stop(&tscm->rx_stream);
+ amdtp_stream_stop(&tscm->tx_stream);
finish_session(tscm);
release_resources(tscm);
diff --git a/sound/firewire/tascam/tascam.c b/sound/firewire/tascam/tascam.c
index e281c33..9dc93a7 100644
--- a/sound/firewire/tascam/tascam.c
+++ b/sound/firewire/tascam/tascam.c
@@ -85,10 +85,8 @@
return 0;
}
-static void tscm_card_free(struct snd_card *card)
+static void tscm_free(struct snd_tscm *tscm)
{
- struct snd_tscm *tscm = card->private_data;
-
snd_tscm_transaction_unregister(tscm);
snd_tscm_stream_destroy_duplex(tscm);
@@ -97,44 +95,36 @@
mutex_destroy(&tscm->mutex);
}
-static int snd_tscm_probe(struct fw_unit *unit,
- const struct ieee1394_device_id *entry)
+static void tscm_card_free(struct snd_card *card)
{
- struct snd_card *card;
- struct snd_tscm *tscm;
+ tscm_free(card->private_data);
+}
+
+static void do_registration(struct work_struct *work)
+{
+ struct snd_tscm *tscm = container_of(work, struct snd_tscm, dwork.work);
int err;
- /* create card */
- err = snd_card_new(&unit->device, -1, NULL, THIS_MODULE,
- sizeof(struct snd_tscm), &card);
+ err = snd_card_new(&tscm->unit->device, -1, NULL, THIS_MODULE, 0,
+ &tscm->card);
if (err < 0)
- return err;
- card->private_free = tscm_card_free;
-
- /* initialize myself */
- tscm = card->private_data;
- tscm->card = card;
- tscm->unit = fw_unit_get(unit);
-
- mutex_init(&tscm->mutex);
- spin_lock_init(&tscm->lock);
- init_waitqueue_head(&tscm->hwdep_wait);
+ return;
err = identify_model(tscm);
if (err < 0)
goto error;
- snd_tscm_proc_init(tscm);
+ err = snd_tscm_transaction_register(tscm);
+ if (err < 0)
+ goto error;
err = snd_tscm_stream_init_duplex(tscm);
if (err < 0)
goto error;
- err = snd_tscm_create_pcm_devices(tscm);
- if (err < 0)
- goto error;
+ snd_tscm_proc_init(tscm);
- err = snd_tscm_transaction_register(tscm);
+ err = snd_tscm_create_pcm_devices(tscm);
if (err < 0)
goto error;
@@ -146,35 +136,91 @@
if (err < 0)
goto error;
- err = snd_card_register(card);
+ err = snd_card_register(tscm->card);
if (err < 0)
goto error;
+ /*
+ * After registered, tscm instance can be released corresponding to
+ * releasing the sound card instance.
+ */
+ tscm->card->private_free = tscm_card_free;
+ tscm->card->private_data = tscm;
+ tscm->registered = true;
+
+ return;
+error:
+ snd_tscm_transaction_unregister(tscm);
+ snd_tscm_stream_destroy_duplex(tscm);
+ snd_card_free(tscm->card);
+ dev_info(&tscm->unit->device,
+ "Sound card registration failed: %d\n", err);
+}
+
+static int snd_tscm_probe(struct fw_unit *unit,
+ const struct ieee1394_device_id *entry)
+{
+ struct snd_tscm *tscm;
+
+ /* Allocate this independent of sound card instance. */
+ tscm = kzalloc(sizeof(struct snd_tscm), GFP_KERNEL);
+ if (tscm == NULL)
+ return -ENOMEM;
+
+ /* initialize myself */
+ tscm->unit = fw_unit_get(unit);
dev_set_drvdata(&unit->device, tscm);
- return err;
-error:
- snd_card_free(card);
- return err;
+ mutex_init(&tscm->mutex);
+ spin_lock_init(&tscm->lock);
+ init_waitqueue_head(&tscm->hwdep_wait);
+
+ /* Allocate and register this sound card later. */
+ INIT_DEFERRABLE_WORK(&tscm->dwork, do_registration);
+ snd_fw_schedule_registration(unit, &tscm->dwork);
+
+ return 0;
}
static void snd_tscm_update(struct fw_unit *unit)
{
struct snd_tscm *tscm = dev_get_drvdata(&unit->device);
+ /* Postpone a workqueue for deferred registration. */
+ if (!tscm->registered)
+ snd_fw_schedule_registration(unit, &tscm->dwork);
+
snd_tscm_transaction_reregister(tscm);
- mutex_lock(&tscm->mutex);
- snd_tscm_stream_update_duplex(tscm);
- mutex_unlock(&tscm->mutex);
+ /*
+ * After registration, userspace can start packet streaming, then this
+ * code block works fine.
+ */
+ if (tscm->registered) {
+ mutex_lock(&tscm->mutex);
+ snd_tscm_stream_update_duplex(tscm);
+ mutex_unlock(&tscm->mutex);
+ }
}
static void snd_tscm_remove(struct fw_unit *unit)
{
struct snd_tscm *tscm = dev_get_drvdata(&unit->device);
- /* No need to wait for releasing card object in this context. */
- snd_card_free_when_closed(tscm->card);
+ /*
+ * Confirm to stop the work for registration before the sound card is
+ * going to be released. The work is not scheduled again because bus
+ * reset handler is not called anymore.
+ */
+ cancel_delayed_work_sync(&tscm->dwork);
+
+ if (tscm->registered) {
+ /* No need to wait for releasing card object in this context. */
+ snd_card_free_when_closed(tscm->card);
+ } else {
+ /* Don't forget this case. */
+ tscm_free(tscm);
+ }
}
static const struct ieee1394_device_id snd_tscm_id_table[] = {
diff --git a/sound/firewire/tascam/tascam.h b/sound/firewire/tascam/tascam.h
index 30ab77e..1f61011 100644
--- a/sound/firewire/tascam/tascam.h
+++ b/sound/firewire/tascam/tascam.h
@@ -51,6 +51,8 @@
struct mutex mutex;
spinlock_t lock;
+ bool registered;
+ struct delayed_work dwork;
const struct snd_tscm_spec *spec;
struct fw_iso_resources tx_resources;
diff --git a/sound/hda/ext/hdac_ext_bus.c b/sound/hda/ext/hdac_ext_bus.c
index 2433f7c..31b510c 100644
--- a/sound/hda/ext/hdac_ext_bus.c
+++ b/sound/hda/ext/hdac_ext_bus.c
@@ -105,6 +105,9 @@
INIT_LIST_HEAD(&ebus->hlink_list);
ebus->idx = idx++;
+ mutex_init(&ebus->lock);
+ ebus->cmd_dma_state = true;
+
return 0;
}
EXPORT_SYMBOL_GPL(snd_hdac_ext_bus_init);
@@ -144,6 +147,7 @@
if (!edev)
return -ENOMEM;
hdev = &edev->hdac;
+ edev->ebus = ebus;
snprintf(name, sizeof(name), "ehdaudio%dD%d", ebus->idx, addr);
diff --git a/sound/hda/ext/hdac_ext_controller.c b/sound/hda/ext/hdac_ext_controller.c
index 548cc1e..860f8ca 100644
--- a/sound/hda/ext/hdac_ext_controller.c
+++ b/sound/hda/ext/hdac_ext_controller.c
@@ -186,6 +186,9 @@
hlink->lcaps = readl(hlink->ml_addr + AZX_REG_ML_LCAP);
hlink->lsdiid = readw(hlink->ml_addr + AZX_REG_ML_LSDIID);
+ /* since link in On, update the ref */
+ hlink->ref_count = 1;
+
list_add_tail(&hlink->list, &ebus->hlink_list);
}
@@ -327,3 +330,66 @@
return 0;
}
EXPORT_SYMBOL_GPL(snd_hdac_ext_bus_link_power_down_all);
+
+int snd_hdac_ext_bus_link_get(struct hdac_ext_bus *ebus,
+ struct hdac_ext_link *link)
+{
+ int ret = 0;
+
+ mutex_lock(&ebus->lock);
+
+ /*
+ * if we move from 0 to 1, count will be 1 so power up this link
+ * as well, also check the dma status and trigger that
+ */
+ if (++link->ref_count == 1) {
+ if (!ebus->cmd_dma_state) {
+ snd_hdac_bus_init_cmd_io(&ebus->bus);
+ ebus->cmd_dma_state = true;
+ }
+
+ ret = snd_hdac_ext_bus_link_power_up(link);
+ }
+
+ mutex_unlock(&ebus->lock);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(snd_hdac_ext_bus_link_get);
+
+int snd_hdac_ext_bus_link_put(struct hdac_ext_bus *ebus,
+ struct hdac_ext_link *link)
+{
+ int ret = 0;
+ struct hdac_ext_link *hlink;
+ bool link_up = false;
+
+ mutex_lock(&ebus->lock);
+
+ /*
+ * if we move from 1 to 0, count will be 0
+ * so power down this link as well
+ */
+ if (--link->ref_count == 0) {
+ ret = snd_hdac_ext_bus_link_power_down(link);
+
+ /*
+ * now check if all links are off, if so turn off
+ * cmd dma as well
+ */
+ list_for_each_entry(hlink, &ebus->hlink_list, list) {
+ if (hlink->ref_count) {
+ link_up = true;
+ break;
+ }
+ }
+
+ if (!link_up) {
+ snd_hdac_bus_stop_cmd_io(&ebus->bus);
+ ebus->cmd_dma_state = false;
+ }
+ }
+
+ mutex_unlock(&ebus->lock);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(snd_hdac_ext_bus_link_put);
diff --git a/sound/hda/hdac_controller.c b/sound/hda/hdac_controller.c
index 8c48623..9fee464 100644
--- a/sound/hda/hdac_controller.c
+++ b/sound/hda/hdac_controller.c
@@ -80,6 +80,22 @@
}
EXPORT_SYMBOL_GPL(snd_hdac_bus_init_cmd_io);
+/* wait for cmd dmas till they are stopped */
+static void hdac_wait_for_cmd_dmas(struct hdac_bus *bus)
+{
+ unsigned long timeout;
+
+ timeout = jiffies + msecs_to_jiffies(100);
+ while ((snd_hdac_chip_readb(bus, RIRBCTL) & AZX_RBCTL_DMA_EN)
+ && time_before(jiffies, timeout))
+ udelay(10);
+
+ timeout = jiffies + msecs_to_jiffies(100);
+ while ((snd_hdac_chip_readb(bus, CORBCTL) & AZX_CORBCTL_RUN)
+ && time_before(jiffies, timeout))
+ udelay(10);
+}
+
/**
* snd_hdac_bus_stop_cmd_io - clean up CORB/RIRB buffers
* @bus: HD-audio core bus
@@ -90,6 +106,7 @@
/* disable ringbuffer DMAs */
snd_hdac_chip_writeb(bus, RIRBCTL, 0);
snd_hdac_chip_writeb(bus, CORBCTL, 0);
+ hdac_wait_for_cmd_dmas(bus);
/* disable unsolicited responses */
snd_hdac_chip_updatel(bus, GCTL, AZX_GCTL_UNSOL, 0);
spin_unlock_irq(&bus->reg_lock);
diff --git a/sound/hda/hdac_i915.c b/sound/hda/hdac_i915.c
index 607bbea..c9af022 100644
--- a/sound/hda/hdac_i915.c
+++ b/sound/hda/hdac_i915.c
@@ -158,22 +158,40 @@
}
EXPORT_SYMBOL_GPL(snd_hdac_i915_set_bclk);
-/* There is a fixed mapping between audio pin node and display port
- * on current Intel platforms:
+/* There is a fixed mapping between audio pin node and display port.
+ * on SNB, IVY, HSW, BSW, SKL, BXT, KBL:
* Pin Widget 5 - PORT B (port = 1 in i915 driver)
* Pin Widget 6 - PORT C (port = 2 in i915 driver)
* Pin Widget 7 - PORT D (port = 3 in i915 driver)
+ *
+ * on VLV, ILK:
+ * Pin Widget 4 - PORT B (port = 1 in i915 driver)
+ * Pin Widget 5 - PORT C (port = 2 in i915 driver)
+ * Pin Widget 6 - PORT D (port = 3 in i915 driver)
*/
-static int pin2port(hda_nid_t pin_nid)
+static int pin2port(struct hdac_device *codec, hda_nid_t pin_nid)
{
- if (WARN_ON(pin_nid < 5 || pin_nid > 7))
+ int base_nid;
+
+ switch (codec->vendor_id) {
+ case 0x80860054: /* ILK */
+ case 0x80862804: /* ILK */
+ case 0x80862882: /* VLV */
+ base_nid = 3;
+ break;
+ default:
+ base_nid = 4;
+ break;
+ }
+
+ if (WARN_ON(pin_nid <= base_nid || pin_nid > base_nid + 3))
return -1;
- return pin_nid - 4;
+ return pin_nid - base_nid;
}
/**
* snd_hdac_sync_audio_rate - Set N/CTS based on the sample rate
- * @bus: HDA core bus
+ * @codec: HDA codec
* @nid: the pin widget NID
* @rate: the sample rate to set
*
@@ -183,14 +201,15 @@
* This function sets N/CTS value based on the given sample rate.
* Returns zero for success, or a negative error code.
*/
-int snd_hdac_sync_audio_rate(struct hdac_bus *bus, hda_nid_t nid, int rate)
+int snd_hdac_sync_audio_rate(struct hdac_device *codec, hda_nid_t nid, int rate)
{
+ struct hdac_bus *bus = codec->bus;
struct i915_audio_component *acomp = bus->audio_component;
int port;
if (!acomp || !acomp->ops || !acomp->ops->sync_audio_rate)
return -ENODEV;
- port = pin2port(nid);
+ port = pin2port(codec, nid);
if (port < 0)
return -EINVAL;
return acomp->ops->sync_audio_rate(acomp->dev, port, rate);
@@ -199,7 +218,7 @@
/**
* snd_hdac_acomp_get_eld - Get the audio state and ELD via component
- * @bus: HDA core bus
+ * @codec: HDA codec
* @nid: the pin widget NID
* @audio_enabled: the pointer to store the current audio state
* @buffer: the buffer pointer to store ELD bytes
@@ -217,16 +236,17 @@
* thus it may be over @max_bytes. If it's over @max_bytes, it implies
* that only a part of ELD bytes have been fetched.
*/
-int snd_hdac_acomp_get_eld(struct hdac_bus *bus, hda_nid_t nid,
+int snd_hdac_acomp_get_eld(struct hdac_device *codec, hda_nid_t nid,
bool *audio_enabled, char *buffer, int max_bytes)
{
+ struct hdac_bus *bus = codec->bus;
struct i915_audio_component *acomp = bus->audio_component;
int port;
if (!acomp || !acomp->ops || !acomp->ops->get_eld)
return -ENODEV;
- port = pin2port(nid);
+ port = pin2port(codec, nid);
if (port < 0)
return -EINVAL;
return acomp->ops->get_eld(acomp->dev, port, audio_enabled,
@@ -338,6 +358,9 @@
struct i915_audio_component *acomp;
int ret;
+ if (WARN_ON(hdac_acomp))
+ return -EBUSY;
+
if (!i915_gfx_present())
return -ENODEV;
@@ -371,6 +394,7 @@
out_err:
kfree(acomp);
bus->audio_component = NULL;
+ hdac_acomp = NULL;
dev_info(dev, "failed to add i915 component master (%d)\n", ret);
return ret;
@@ -404,6 +428,7 @@
kfree(acomp);
bus->audio_component = NULL;
+ hdac_acomp = NULL;
return 0;
}
diff --git a/sound/hda/hdmi_chmap.c b/sound/hda/hdmi_chmap.c
index d7ec862..c6c75e7 100644
--- a/sound/hda/hdmi_chmap.c
+++ b/sound/hda/hdmi_chmap.c
@@ -625,13 +625,30 @@
WARN_ON(count != channels);
}
+static int spk_mask_from_spk_alloc(int spk_alloc)
+{
+ int i;
+ int spk_mask = eld_speaker_allocation_bits[0];
+
+ for (i = 0; i < ARRAY_SIZE(eld_speaker_allocation_bits); i++) {
+ if (spk_alloc & (1 << i))
+ spk_mask |= eld_speaker_allocation_bits[i];
+ }
+
+ return spk_mask;
+}
+
static int hdmi_chmap_ctl_tlv(struct snd_kcontrol *kcontrol, int op_flag,
unsigned int size, unsigned int __user *tlv)
{
struct snd_pcm_chmap *info = snd_kcontrol_chip(kcontrol);
struct hdac_chmap *chmap = info->private_data;
+ int pcm_idx = kcontrol->private_value;
unsigned int __user *dst;
int chs, count = 0;
+ unsigned long max_chs;
+ int type;
+ int spk_alloc, spk_mask;
if (size < 8)
return -ENOMEM;
@@ -639,40 +656,59 @@
return -EFAULT;
size -= 8;
dst = tlv + 2;
- for (chs = 2; chs <= chmap->channels_max; chs++) {
+
+ spk_alloc = chmap->ops.get_spk_alloc(chmap->hdac, pcm_idx);
+ spk_mask = spk_mask_from_spk_alloc(spk_alloc);
+
+ max_chs = hweight_long(spk_mask);
+
+ for (chs = 2; chs <= max_chs; chs++) {
int i;
struct hdac_cea_channel_speaker_allocation *cap;
cap = channel_allocations;
for (i = 0; i < ARRAY_SIZE(channel_allocations); i++, cap++) {
int chs_bytes = chs * 4;
- int type = chmap->ops.chmap_cea_alloc_validate_get_type(
- chmap, cap, chs);
unsigned int tlv_chmap[8];
- if (type < 0)
+ if (cap->channels != chs)
continue;
+
+ if (!(cap->spk_mask == (spk_mask & cap->spk_mask)))
+ continue;
+
+ type = chmap->ops.chmap_cea_alloc_validate_get_type(
+ chmap, cap, chs);
+ if (type < 0)
+ return -ENODEV;
if (size < 8)
return -ENOMEM;
+
if (put_user(type, dst) ||
put_user(chs_bytes, dst + 1))
return -EFAULT;
+
dst += 2;
size -= 8;
count += 8;
+
if (size < chs_bytes)
return -ENOMEM;
+
size -= chs_bytes;
count += chs_bytes;
chmap->ops.cea_alloc_to_tlv_chmap(chmap, cap,
tlv_chmap, chs);
+
if (copy_to_user(dst, tlv_chmap, chs_bytes))
return -EFAULT;
dst += chs;
}
}
+
if (put_user(count, tlv + 1))
return -EFAULT;
+
return 0;
}
diff --git a/sound/hda/local.h b/sound/hda/local.h
index d692f41..0d5bb15 100644
--- a/sound/hda/local.h
+++ b/sound/hda/local.h
@@ -16,6 +16,16 @@
return (wcaps & AC_WCAP_TYPE) >> AC_WCAP_TYPE_SHIFT;
}
+static inline unsigned int get_wcaps_channels(u32 wcaps)
+{
+ unsigned int chans;
+
+ chans = (wcaps & AC_WCAP_CHAN_CNT_EXT) >> 13;
+ chans = (chans + 1) * 2;
+
+ return chans;
+}
+
extern const struct attribute_group *hdac_dev_attr_groups[];
int hda_widget_sysfs_init(struct hdac_device *codec);
void hda_widget_sysfs_exit(struct hdac_device *codec);
diff --git a/sound/isa/wavefront/wavefront_synth.c b/sound/isa/wavefront/wavefront_synth.c
index 69f76ff..718d5e3 100644
--- a/sound/isa/wavefront/wavefront_synth.c
+++ b/sound/isa/wavefront/wavefront_synth.c
@@ -785,6 +785,9 @@
DPRINT (WF_DEBUG_LOAD_PATCH, "downloading patch %d\n",
header->number);
+ if (header->number >= ARRAY_SIZE(dev->patch_status))
+ return -EINVAL;
+
dev->patch_status[header->number] |= WF_SLOT_FILLED;
bptr = buf;
@@ -809,6 +812,9 @@
DPRINT (WF_DEBUG_LOAD_PATCH, "downloading program %d\n",
header->number);
+ if (header->number >= ARRAY_SIZE(dev->prog_status))
+ return -EINVAL;
+
dev->prog_status[header->number] = WF_SLOT_USED;
/* XXX need to zero existing SLOT_USED bit for program_status[i]
@@ -898,6 +904,9 @@
header->number = x;
}
+ if (header->number >= WF_MAX_SAMPLE)
+ return -EINVAL;
+
if (header->size) {
/* XXX it's a debatable point whether or not RDONLY semantics
diff --git a/sound/oss/waveartist.c b/sound/oss/waveartist.c
index b36ea47..0b8d0de 100644
--- a/sound/oss/waveartist.c
+++ b/sound/oss/waveartist.c
@@ -1414,11 +1414,9 @@
else {
#ifdef CONFIG_ARCH_NETWINDER
if (machine_is_netwinder()) {
- init_timer(&vnc_timer);
- vnc_timer.function = vnc_slider_tick;
- vnc_timer.expires = jiffies;
- vnc_timer.data = nr_waveartist_devs;
- add_timer(&vnc_timer);
+ setup_timer(&vnc_timer, vnc_slider_tick,
+ nr_waveartist_devs);
+ mod_timer(&vnc_timer, jiffies);
vnc_configure_mixer(devc, 0);
diff --git a/sound/pci/au88x0/au88x0_core.c b/sound/pci/au88x0/au88x0_core.c
index 4667c32..4a054d7 100644
--- a/sound/pci/au88x0/au88x0_core.c
+++ b/sound/pci/au88x0/au88x0_core.c
@@ -2151,8 +2151,7 @@
stream->resources, en,
VORTEX_RESOURCE_SRC)) < 0) {
memset(stream->resources, 0,
- sizeof(unsigned char) *
- VORTEX_RESOURCE_LAST);
+ sizeof(stream->resources));
return -EBUSY;
}
if (stream->type != VORTEX_PCM_A3D) {
@@ -2162,7 +2161,7 @@
VORTEX_RESOURCE_MIXIN)) < 0) {
memset(stream->resources,
0,
- sizeof(unsigned char) * VORTEX_RESOURCE_LAST);
+ sizeof(stream->resources));
return -EBUSY;
}
}
@@ -2175,8 +2174,7 @@
stream->resources, en,
VORTEX_RESOURCE_A3D)) < 0) {
memset(stream->resources, 0,
- sizeof(unsigned char) *
- VORTEX_RESOURCE_LAST);
+ sizeof(stream->resources));
dev_err(vortex->card->dev,
"out of A3D sources. Sorry\n");
return -EBUSY;
@@ -2290,8 +2288,7 @@
VORTEX_RESOURCE_MIXOUT))
< 0) {
memset(stream->resources, 0,
- sizeof(unsigned char) *
- VORTEX_RESOURCE_LAST);
+ sizeof(stream->resources));
return -EBUSY;
}
if ((src[i] =
@@ -2299,8 +2296,7 @@
stream->resources, en,
VORTEX_RESOURCE_SRC)) < 0) {
memset(stream->resources, 0,
- sizeof(unsigned char) *
- VORTEX_RESOURCE_LAST);
+ sizeof(stream->resources));
return -EBUSY;
}
}
diff --git a/sound/pci/au88x0/au88x0_pcm.c b/sound/pci/au88x0/au88x0_pcm.c
index a6d6d8d..df5741a 100644
--- a/sound/pci/au88x0/au88x0_pcm.c
+++ b/sound/pci/au88x0/au88x0_pcm.c
@@ -432,7 +432,10 @@
#endif
//printk(KERN_INFO "vortex: pointer = 0x%x\n", current_ptr);
spin_unlock(&chip->lock);
- return (bytes_to_frames(substream->runtime, current_ptr));
+ current_ptr = bytes_to_frames(substream->runtime, current_ptr);
+ if (current_ptr >= substream->runtime->buffer_size)
+ current_ptr = 0;
+ return current_ptr;
}
/* operators */
diff --git a/sound/pci/ctxfi/cttimer.c b/sound/pci/ctxfi/cttimer.c
index a5d4604..8f94534 100644
--- a/sound/pci/ctxfi/cttimer.c
+++ b/sound/pci/ctxfi/cttimer.c
@@ -49,7 +49,7 @@
spinlock_t lock; /* global timer lock (for xfitimer) */
spinlock_t list_lock; /* lock for instance list */
struct ct_atc *atc;
- struct ct_timer_ops *ops;
+ const struct ct_timer_ops *ops;
struct list_head instance_head;
struct list_head running_head;
unsigned int wc; /* current wallclock */
@@ -128,7 +128,7 @@
#define ct_systimer_free ct_systimer_prepare
-static struct ct_timer_ops ct_systimer_ops = {
+static const struct ct_timer_ops ct_systimer_ops = {
.init = ct_systimer_init,
.free_instance = ct_systimer_free,
.prepare = ct_systimer_prepare,
@@ -322,7 +322,7 @@
ct_xfitimer_irq_stop(atimer);
}
-static struct ct_timer_ops ct_xfitimer_ops = {
+static const struct ct_timer_ops ct_xfitimer_ops = {
.prepare = ct_xfitimer_prepare,
.start = ct_xfitimer_start,
.stop = ct_xfitimer_stop,
diff --git a/sound/pci/ens1370.c b/sound/pci/ens1370.c
index 0dc44eb..626cd21 100644
--- a/sound/pci/ens1370.c
+++ b/sound/pci/ens1370.c
@@ -1548,7 +1548,7 @@
int val = 0;
spin_lock_irq(&ensoniq->reg_lock);
- if ((ensoniq->ctrl & ES_1371_GPIO_OUTM) >= 4)
+ if (ensoniq->ctrl & ES_1371_GPIO_OUT(4))
val = 1;
ucontrol->value.integer.value[0] = val;
spin_unlock_irq(&ensoniq->reg_lock);
diff --git a/sound/pci/hda/Kconfig b/sound/pci/hda/Kconfig
index bb02c2d..7f3b5ed 100644
--- a/sound/pci/hda/Kconfig
+++ b/sound/pci/hda/Kconfig
@@ -50,9 +50,13 @@
bool "Allow dynamic codec reconfiguration"
help
Say Y here to enable the HD-audio codec re-configuration feature.
- This adds the sysfs interfaces to allow user to clear the whole
- codec configuration, change the codec setup, add extra verbs,
- and re-configure the codec dynamically.
+ It allows user to clear the whole codec configuration, change the
+ codec setup, add extra verbs, and re-configure the codec dynamically.
+
+ Note that this item alone doesn't provide the sysfs interface, but
+ enables the feature just for the patch loader below.
+ If you need the traditional sysfs entries for the manual interaction,
+ turn on CONFIG_SND_HDA_HWDEP as well.
config SND_HDA_INPUT_BEEP
bool "Support digital beep via input layer"
diff --git a/sound/pci/hda/hda_generic.c b/sound/pci/hda/hda_generic.c
index dfaf1a9..320445f 100644
--- a/sound/pci/hda/hda_generic.c
+++ b/sound/pci/hda/hda_generic.c
@@ -5434,6 +5434,7 @@
spec->cur_adc_stream_tag = stream_tag;
spec->cur_adc_format = format;
snd_hda_codec_setup_stream(codec, spec->cur_adc, stream_tag, 0, format);
+ call_pcm_capture_hook(hinfo, codec, substream, HDA_GEN_PCM_ACT_PREPARE);
return 0;
}
@@ -5444,6 +5445,7 @@
struct hda_gen_spec *spec = codec->spec;
snd_hda_codec_cleanup_stream(codec, spec->cur_adc);
spec->cur_adc = 0;
+ call_pcm_capture_hook(hinfo, codec, substream, HDA_GEN_PCM_ACT_CLEANUP);
return 0;
}
diff --git a/sound/pci/hda/patch_hdmi.c b/sound/pci/hda/patch_hdmi.c
index a010d70..d0d5ad8 100644
--- a/sound/pci/hda/patch_hdmi.c
+++ b/sound/pci/hda/patch_hdmi.c
@@ -114,6 +114,9 @@
int (*setup_stream)(struct hda_codec *codec, hda_nid_t cvt_nid,
hda_nid_t pin_nid, u32 stream_tag, int format);
+ void (*pin_cvt_fixup)(struct hda_codec *codec,
+ struct hdmi_spec_per_pin *per_pin,
+ hda_nid_t cvt_nid);
};
struct hdmi_pcm {
@@ -684,7 +687,8 @@
if (!channels)
return;
- if (is_haswell_plus(codec))
+ /* some HW (e.g. HSW+) needs reprogramming the amp at each time */
+ if (get_wcaps(codec, pin_nid) & AC_WCAP_OUT_AMP)
snd_hda_codec_write(codec, pin_nid, 0,
AC_VERB_SET_AMP_GAIN_MUTE,
AMP_OUT_UNMUTE);
@@ -864,9 +868,6 @@
struct hdmi_spec *spec = codec->spec;
int err;
- if (is_haswell_plus(codec))
- haswell_verify_D0(codec, cvt_nid, pin_nid);
-
err = spec->ops.pin_hbr_setup(codec, pin_nid, is_hbr_format(format));
if (err) {
@@ -884,7 +885,7 @@
* of the pin.
*/
static int hdmi_choose_cvt(struct hda_codec *codec,
- int pin_idx, int *cvt_id, int *mux_id)
+ int pin_idx, int *cvt_id)
{
struct hdmi_spec *spec = codec->spec;
struct hdmi_spec_per_pin *per_pin;
@@ -925,8 +926,6 @@
if (cvt_id)
*cvt_id = cvt_idx;
- if (mux_id)
- *mux_id = mux_idx;
return 0;
}
@@ -1019,9 +1018,6 @@
int mux_idx;
struct hdmi_spec *spec = codec->spec;
- if (!is_haswell_plus(codec) && !is_valleyview_plus(codec))
- return;
-
/* On Intel platform, the mapping of converter nid to
* mux index of the pins are always the same.
* The pin nid may be 0, this means all pins will not
@@ -1032,6 +1028,17 @@
intel_not_share_assigned_cvt(codec, pin_nid, mux_idx);
}
+/* skeleton caller of pin_cvt_fixup ops */
+static void pin_cvt_fixup(struct hda_codec *codec,
+ struct hdmi_spec_per_pin *per_pin,
+ hda_nid_t cvt_nid)
+{
+ struct hdmi_spec *spec = codec->spec;
+
+ if (spec->ops.pin_cvt_fixup)
+ spec->ops.pin_cvt_fixup(codec, per_pin, cvt_nid);
+}
+
/* called in hdmi_pcm_open when no pin is assigned to the PCM
* in dyn_pcm_assign mode.
*/
@@ -1049,7 +1056,7 @@
if (pcm_idx < 0)
return -EINVAL;
- err = hdmi_choose_cvt(codec, -1, &cvt_idx, NULL);
+ err = hdmi_choose_cvt(codec, -1, &cvt_idx);
if (err)
return err;
@@ -1057,7 +1064,7 @@
per_cvt->assigned = 1;
hinfo->nid = per_cvt->cvt_nid;
- intel_not_share_assigned_cvt_nid(codec, 0, per_cvt->cvt_nid);
+ pin_cvt_fixup(codec, NULL, per_cvt->cvt_nid);
set_bit(pcm_idx, &spec->pcm_in_use);
/* todo: setup spdif ctls assign */
@@ -1089,7 +1096,7 @@
{
struct hdmi_spec *spec = codec->spec;
struct snd_pcm_runtime *runtime = substream->runtime;
- int pin_idx, cvt_idx, pcm_idx, mux_idx = 0;
+ int pin_idx, cvt_idx, pcm_idx;
struct hdmi_spec_per_pin *per_pin;
struct hdmi_eld *eld;
struct hdmi_spec_per_cvt *per_cvt = NULL;
@@ -1118,7 +1125,7 @@
}
}
- err = hdmi_choose_cvt(codec, pin_idx, &cvt_idx, &mux_idx);
+ err = hdmi_choose_cvt(codec, pin_idx, &cvt_idx);
if (err < 0) {
mutex_unlock(&spec->pcm_lock);
return err;
@@ -1135,11 +1142,10 @@
snd_hda_codec_write_cache(codec, per_pin->pin_nid, 0,
AC_VERB_SET_CONNECT_SEL,
- mux_idx);
+ per_pin->mux_idx);
/* configure unused pins to choose other converters */
- if (is_haswell_plus(codec) || is_valleyview_plus(codec))
- intel_not_share_assigned_cvt(codec, per_pin->pin_nid, mux_idx);
+ pin_cvt_fixup(codec, per_pin, 0);
snd_hda_spdif_ctls_assign(codec, pcm_idx, per_cvt->cvt_nid);
@@ -1372,12 +1378,7 @@
* and this can make HW reset converter selection on a pin.
*/
if (eld->eld_valid && !old_eld_valid && per_pin->setup) {
- if (is_haswell_plus(codec) || is_valleyview_plus(codec)) {
- intel_verify_pin_cvt_connect(codec, per_pin);
- intel_not_share_assigned_cvt(codec, per_pin->pin_nid,
- per_pin->mux_idx);
- }
-
+ pin_cvt_fixup(codec, per_pin, 0);
hdmi_setup_audio_infoframe(codec, per_pin, per_pin->non_pcm);
}
@@ -1484,7 +1485,7 @@
mutex_lock(&per_pin->lock);
eld->monitor_present = false;
- size = snd_hdac_acomp_get_eld(&codec->bus->core, per_pin->pin_nid,
+ size = snd_hdac_acomp_get_eld(&codec->core, per_pin->pin_nid,
&eld->monitor_present, eld->eld_buffer,
ELD_MAX_SIZE);
if (size > 0) {
@@ -1711,7 +1712,7 @@
* skip pin setup and return 0 to make audio playback
* be ongoing
*/
- intel_not_share_assigned_cvt_nid(codec, 0, cvt_nid);
+ pin_cvt_fixup(codec, NULL, cvt_nid);
snd_hda_codec_setup_stream(codec, cvt_nid,
stream_tag, 0, format);
mutex_unlock(&spec->pcm_lock);
@@ -1724,23 +1725,21 @@
}
per_pin = get_pin(spec, pin_idx);
pin_nid = per_pin->pin_nid;
- if (is_haswell_plus(codec) || is_valleyview_plus(codec)) {
- /* Verify pin:cvt selections to avoid silent audio after S3.
- * After S3, the audio driver restores pin:cvt selections
- * but this can happen before gfx is ready and such selection
- * is overlooked by HW. Thus multiple pins can share a same
- * default convertor and mute control will affect each other,
- * which can cause a resumed audio playback become silent
- * after S3.
- */
- intel_verify_pin_cvt_connect(codec, per_pin);
- intel_not_share_assigned_cvt(codec, pin_nid, per_pin->mux_idx);
- }
+
+ /* Verify pin:cvt selections to avoid silent audio after S3.
+ * After S3, the audio driver restores pin:cvt selections
+ * but this can happen before gfx is ready and such selection
+ * is overlooked by HW. Thus multiple pins can share a same
+ * default convertor and mute control will affect each other,
+ * which can cause a resumed audio playback become silent
+ * after S3.
+ */
+ pin_cvt_fixup(codec, per_pin, 0);
/* Call sync_audio_rate to set the N/CTS/M manually if necessary */
/* Todo: add DP1.2 MST audio support later */
if (codec_has_acomp(codec))
- snd_hdac_sync_audio_rate(&codec->bus->core, pin_nid, runtime->rate);
+ snd_hdac_sync_audio_rate(&codec->core, pin_nid, runtime->rate);
non_pcm = check_non_pcm_per_cvt(codec, cvt_nid);
mutex_lock(&per_pin->lock);
@@ -1837,6 +1836,18 @@
.cleanup = generic_hdmi_playback_pcm_cleanup,
};
+static int hdmi_get_spk_alloc(struct hdac_device *hdac, int pcm_idx)
+{
+ struct hda_codec *codec = container_of(hdac, struct hda_codec, core);
+ struct hdmi_spec *spec = codec->spec;
+ struct hdmi_spec_per_pin *per_pin = pcm_idx_to_pin(spec, pcm_idx);
+
+ if (!per_pin)
+ return 0;
+
+ return per_pin->sink_eld.info.spk_alloc;
+}
+
static void hdmi_get_chmap(struct hdac_device *hdac, int pcm_idx,
unsigned char *chmap)
{
@@ -2075,6 +2086,20 @@
snd_array_free(&spec->cvts);
}
+static void generic_spec_free(struct hda_codec *codec)
+{
+ struct hdmi_spec *spec = codec->spec;
+
+ if (spec) {
+ if (spec->i915_bound)
+ snd_hdac_i915_exit(&codec->bus->core);
+ hdmi_array_free(spec);
+ kfree(spec);
+ codec->spec = NULL;
+ }
+ codec->dp_mst = false;
+}
+
static void generic_hdmi_free(struct hda_codec *codec)
{
struct hdmi_spec *spec = codec->spec;
@@ -2099,10 +2124,7 @@
spec->pcm_rec[pcm_idx].jack = NULL;
}
- if (spec->i915_bound)
- snd_hdac_i915_exit(&codec->bus->core);
- hdmi_array_free(spec);
- kfree(spec);
+ generic_spec_free(codec);
}
#ifdef CONFIG_PM
@@ -2140,6 +2162,55 @@
.setup_stream = hdmi_setup_stream,
};
+/* allocate codec->spec and assign/initialize generic parser ops */
+static int alloc_generic_hdmi(struct hda_codec *codec)
+{
+ struct hdmi_spec *spec;
+
+ spec = kzalloc(sizeof(*spec), GFP_KERNEL);
+ if (!spec)
+ return -ENOMEM;
+
+ spec->ops = generic_standard_hdmi_ops;
+ mutex_init(&spec->pcm_lock);
+ snd_hdac_register_chmap_ops(&codec->core, &spec->chmap);
+
+ spec->chmap.ops.get_chmap = hdmi_get_chmap;
+ spec->chmap.ops.set_chmap = hdmi_set_chmap;
+ spec->chmap.ops.is_pcm_attached = is_hdmi_pcm_attached;
+ spec->chmap.ops.get_spk_alloc = hdmi_get_spk_alloc,
+
+ codec->spec = spec;
+ hdmi_array_init(spec, 4);
+
+ codec->patch_ops = generic_hdmi_patch_ops;
+
+ return 0;
+}
+
+/* generic HDMI parser */
+static int patch_generic_hdmi(struct hda_codec *codec)
+{
+ int err;
+
+ err = alloc_generic_hdmi(codec);
+ if (err < 0)
+ return err;
+
+ err = hdmi_parse_codec(codec);
+ if (err < 0) {
+ generic_spec_free(codec);
+ return err;
+ }
+
+ generic_hdmi_init_per_pins(codec);
+ return 0;
+}
+
+/*
+ * Intel codec parsers and helpers
+ */
+
static void intel_haswell_fixup_connect_list(struct hda_codec *codec,
hda_nid_t nid)
{
@@ -2217,12 +2288,23 @@
static void intel_pin_eld_notify(void *audio_ptr, int port)
{
struct hda_codec *codec = audio_ptr;
- int pin_nid = port + 0x04;
+ int pin_nid;
/* we assume only from port-B to port-D */
if (port < 1 || port > 3)
return;
+ switch (codec->core.vendor_id) {
+ case 0x80860054: /* ILK */
+ case 0x80862804: /* ILK */
+ case 0x80862882: /* VLV */
+ pin_nid = port + 0x03;
+ break;
+ default:
+ pin_nid = port + 0x04;
+ break;
+ }
+
/* skip notification during system suspend (but not in runtime PM);
* the state will be updated at resume
*/
@@ -2236,95 +2318,161 @@
check_presence_and_report(codec, pin_nid);
}
-static int patch_generic_hdmi(struct hda_codec *codec)
+/* register i915 component pin_eld_notify callback */
+static void register_i915_notifier(struct hda_codec *codec)
+{
+ struct hdmi_spec *spec = codec->spec;
+
+ spec->use_acomp_notifier = true;
+ spec->i915_audio_ops.audio_ptr = codec;
+ /* intel_audio_codec_enable() or intel_audio_codec_disable()
+ * will call pin_eld_notify with using audio_ptr pointer
+ * We need make sure audio_ptr is really setup
+ */
+ wmb();
+ spec->i915_audio_ops.pin_eld_notify = intel_pin_eld_notify;
+ snd_hdac_i915_register_notifier(&spec->i915_audio_ops);
+}
+
+/* setup_stream ops override for HSW+ */
+static int i915_hsw_setup_stream(struct hda_codec *codec, hda_nid_t cvt_nid,
+ hda_nid_t pin_nid, u32 stream_tag, int format)
+{
+ haswell_verify_D0(codec, cvt_nid, pin_nid);
+ return hdmi_setup_stream(codec, cvt_nid, pin_nid, stream_tag, format);
+}
+
+/* pin_cvt_fixup ops override for HSW+ and VLV+ */
+static void i915_pin_cvt_fixup(struct hda_codec *codec,
+ struct hdmi_spec_per_pin *per_pin,
+ hda_nid_t cvt_nid)
+{
+ if (per_pin) {
+ intel_verify_pin_cvt_connect(codec, per_pin);
+ intel_not_share_assigned_cvt(codec, per_pin->pin_nid,
+ per_pin->mux_idx);
+ } else {
+ intel_not_share_assigned_cvt_nid(codec, 0, cvt_nid);
+ }
+}
+
+/* Intel Haswell and onwards; audio component with eld notifier */
+static int patch_i915_hsw_hdmi(struct hda_codec *codec)
{
struct hdmi_spec *spec;
+ int err;
- spec = kzalloc(sizeof(*spec), GFP_KERNEL);
- if (spec == NULL)
- return -ENOMEM;
-
- spec->ops = generic_standard_hdmi_ops;
- mutex_init(&spec->pcm_lock);
- snd_hdac_register_chmap_ops(&codec->core, &spec->chmap);
-
- spec->chmap.ops.get_chmap = hdmi_get_chmap;
- spec->chmap.ops.set_chmap = hdmi_set_chmap;
- spec->chmap.ops.is_pcm_attached = is_hdmi_pcm_attached;
-
- codec->spec = spec;
- hdmi_array_init(spec, 4);
-
-#ifdef CONFIG_SND_HDA_I915
- /* Try to bind with i915 for Intel HSW+ codecs (if not done yet) */
- if ((codec->core.vendor_id >> 16) == 0x8086 &&
- is_haswell_plus(codec)) {
-#if 0
- /* on-demand binding leads to an unbalanced refcount when
- * both i915 and hda drivers are probed concurrently;
- * disabled temporarily for now
- */
- if (!codec->bus->core.audio_component)
- if (!snd_hdac_i915_init(&codec->bus->core))
- spec->i915_bound = true;
-#endif
- /* use i915 audio component notifier for hotplug */
- if (codec->bus->core.audio_component)
- spec->use_acomp_notifier = true;
+ /* HSW+ requires i915 binding */
+ if (!codec->bus->core.audio_component) {
+ codec_info(codec, "No i915 binding for Intel HDMI/DP codec\n");
+ return -ENODEV;
}
-#endif
- if (is_haswell_plus(codec)) {
- intel_haswell_enable_all_pins(codec, true);
- intel_haswell_fixup_enable_dp12(codec);
+ err = alloc_generic_hdmi(codec);
+ if (err < 0)
+ return err;
+ spec = codec->spec;
+
+ intel_haswell_enable_all_pins(codec, true);
+ intel_haswell_fixup_enable_dp12(codec);
+
+ /* For Haswell/Broadwell, the controller is also in the power well and
+ * can cover the codec power request, and so need not set this flag.
+ */
+ if (!is_haswell(codec) && !is_broadwell(codec))
+ codec->core.link_power_control = 1;
+
+ codec->patch_ops.set_power_state = haswell_set_power_state;
+ codec->dp_mst = true;
+ codec->depop_delay = 0;
+ codec->auto_runtime_pm = 1;
+
+ spec->ops.setup_stream = i915_hsw_setup_stream;
+ spec->ops.pin_cvt_fixup = i915_pin_cvt_fixup;
+
+ err = hdmi_parse_codec(codec);
+ if (err < 0) {
+ generic_spec_free(codec);
+ return err;
}
+ generic_hdmi_init_per_pins(codec);
+ register_i915_notifier(codec);
+ return 0;
+}
+
+/* Intel Baytrail and Braswell; with eld notifier */
+static int patch_i915_byt_hdmi(struct hda_codec *codec)
+{
+ struct hdmi_spec *spec;
+ int err;
+
+ /* requires i915 binding */
+ if (!codec->bus->core.audio_component) {
+ codec_info(codec, "No i915 binding for Intel HDMI/DP codec\n");
+ return -ENODEV;
+ }
+
+ err = alloc_generic_hdmi(codec);
+ if (err < 0)
+ return err;
+ spec = codec->spec;
+
/* For Valleyview/Cherryview, only the display codec is in the display
* power well and can use link_power ops to request/release the power.
- * For Haswell/Broadwell, the controller is also in the power well and
- * can cover the codec power request, and so need not set this flag.
- * For previous platforms, there is no such power well feature.
*/
- if (is_valleyview_plus(codec) || is_skylake(codec) ||
- is_broxton(codec))
- codec->core.link_power_control = 1;
+ codec->core.link_power_control = 1;
- if (hdmi_parse_codec(codec) < 0) {
- if (spec->i915_bound)
- snd_hdac_i915_exit(&codec->bus->core);
- codec->spec = NULL;
- kfree(spec);
- return -EINVAL;
- }
- codec->patch_ops = generic_hdmi_patch_ops;
- if (is_haswell_plus(codec)) {
- codec->patch_ops.set_power_state = haswell_set_power_state;
- codec->dp_mst = true;
- }
+ codec->depop_delay = 0;
+ codec->auto_runtime_pm = 1;
- /* Enable runtime pm for HDMI audio codec of HSW/BDW/SKL/BYT/BSW */
- if (is_haswell_plus(codec) || is_valleyview_plus(codec))
- codec->auto_runtime_pm = 1;
+ spec->ops.pin_cvt_fixup = i915_pin_cvt_fixup;
+
+ err = hdmi_parse_codec(codec);
+ if (err < 0) {
+ generic_spec_free(codec);
+ return err;
+ }
generic_hdmi_init_per_pins(codec);
-
-
- if (codec_has_acomp(codec)) {
- codec->depop_delay = 0;
- spec->i915_audio_ops.audio_ptr = codec;
- /* intel_audio_codec_enable() or intel_audio_codec_disable()
- * will call pin_eld_notify with using audio_ptr pointer
- * We need make sure audio_ptr is really setup
- */
- wmb();
- spec->i915_audio_ops.pin_eld_notify = intel_pin_eld_notify;
- snd_hdac_i915_register_notifier(&spec->i915_audio_ops);
- }
-
- WARN_ON(spec->dyn_pcm_assign && !codec_has_acomp(codec));
+ register_i915_notifier(codec);
return 0;
}
+/* Intel IronLake, SandyBridge and IvyBridge; with eld notifier */
+static int patch_i915_cpt_hdmi(struct hda_codec *codec)
+{
+ struct hdmi_spec *spec;
+ int err;
+
+ /* no i915 component should have been bound before this */
+ if (WARN_ON(codec->bus->core.audio_component))
+ return -EBUSY;
+
+ err = alloc_generic_hdmi(codec);
+ if (err < 0)
+ return err;
+ spec = codec->spec;
+
+ /* Try to bind with i915 now */
+ err = snd_hdac_i915_init(&codec->bus->core);
+ if (err < 0)
+ goto error;
+ spec->i915_bound = true;
+
+ err = hdmi_parse_codec(codec);
+ if (err < 0)
+ goto error;
+
+ generic_hdmi_init_per_pins(codec);
+ register_i915_notifier(codec);
+ return 0;
+
+ error:
+ generic_spec_free(codec);
+ return err;
+}
+
/*
* Shared non-generic implementations
*/
@@ -3492,21 +3640,21 @@
HDA_CODEC_ENTRY(0x11069f81, "VX900 HDMI/DP", patch_via_hdmi),
HDA_CODEC_ENTRY(0x11069f84, "VX11 HDMI/DP", patch_generic_hdmi),
HDA_CODEC_ENTRY(0x11069f85, "VX11 HDMI/DP", patch_generic_hdmi),
-HDA_CODEC_ENTRY(0x80860054, "IbexPeak HDMI", patch_generic_hdmi),
+HDA_CODEC_ENTRY(0x80860054, "IbexPeak HDMI", patch_i915_cpt_hdmi),
HDA_CODEC_ENTRY(0x80862801, "Bearlake HDMI", patch_generic_hdmi),
HDA_CODEC_ENTRY(0x80862802, "Cantiga HDMI", patch_generic_hdmi),
HDA_CODEC_ENTRY(0x80862803, "Eaglelake HDMI", patch_generic_hdmi),
-HDA_CODEC_ENTRY(0x80862804, "IbexPeak HDMI", patch_generic_hdmi),
-HDA_CODEC_ENTRY(0x80862805, "CougarPoint HDMI", patch_generic_hdmi),
-HDA_CODEC_ENTRY(0x80862806, "PantherPoint HDMI", patch_generic_hdmi),
-HDA_CODEC_ENTRY(0x80862807, "Haswell HDMI", patch_generic_hdmi),
-HDA_CODEC_ENTRY(0x80862808, "Broadwell HDMI", patch_generic_hdmi),
-HDA_CODEC_ENTRY(0x80862809, "Skylake HDMI", patch_generic_hdmi),
-HDA_CODEC_ENTRY(0x8086280a, "Broxton HDMI", patch_generic_hdmi),
-HDA_CODEC_ENTRY(0x8086280b, "Kabylake HDMI", patch_generic_hdmi),
+HDA_CODEC_ENTRY(0x80862804, "IbexPeak HDMI", patch_i915_cpt_hdmi),
+HDA_CODEC_ENTRY(0x80862805, "CougarPoint HDMI", patch_i915_cpt_hdmi),
+HDA_CODEC_ENTRY(0x80862806, "PantherPoint HDMI", patch_i915_cpt_hdmi),
+HDA_CODEC_ENTRY(0x80862807, "Haswell HDMI", patch_i915_hsw_hdmi),
+HDA_CODEC_ENTRY(0x80862808, "Broadwell HDMI", patch_i915_hsw_hdmi),
+HDA_CODEC_ENTRY(0x80862809, "Skylake HDMI", patch_i915_hsw_hdmi),
+HDA_CODEC_ENTRY(0x8086280a, "Broxton HDMI", patch_i915_hsw_hdmi),
+HDA_CODEC_ENTRY(0x8086280b, "Kabylake HDMI", patch_i915_hsw_hdmi),
HDA_CODEC_ENTRY(0x80862880, "CedarTrail HDMI", patch_generic_hdmi),
-HDA_CODEC_ENTRY(0x80862882, "Valleyview2 HDMI", patch_generic_hdmi),
-HDA_CODEC_ENTRY(0x80862883, "Braswell HDMI", patch_generic_hdmi),
+HDA_CODEC_ENTRY(0x80862882, "Valleyview2 HDMI", patch_i915_byt_hdmi),
+HDA_CODEC_ENTRY(0x80862883, "Braswell HDMI", patch_i915_byt_hdmi),
HDA_CODEC_ENTRY(0x808629fb, "Crestline HDMI", patch_generic_hdmi),
/* special ID for generic HDMI */
HDA_CODEC_ENTRY(HDA_CODEC_ID_GENERIC_HDMI, "Generic HDMI", patch_generic_hdmi),
diff --git a/sound/pci/hda/patch_realtek.c b/sound/pci/hda/patch_realtek.c
index 4918ffa..002f153 100644
--- a/sound/pci/hda/patch_realtek.c
+++ b/sound/pci/hda/patch_realtek.c
@@ -342,6 +342,11 @@
case 0x10ec0293:
alc_update_coef_idx(codec, 0xa, 1<<13, 0);
break;
+ case 0x10ec0234:
+ case 0x10ec0274:
+ case 0x10ec0294:
+ alc_update_coef_idx(codec, 0x10, 1<<15, 0);
+ break;
case 0x10ec0662:
if ((coef & 0x00f0) == 0x0030)
alc_update_coef_idx(codec, 0x4, 1<<10, 0); /* EAPD Ctrl */
@@ -2647,6 +2652,7 @@
ALC269_TYPE_ALC255,
ALC269_TYPE_ALC256,
ALC269_TYPE_ALC225,
+ ALC269_TYPE_ALC294,
};
/*
@@ -2677,6 +2683,7 @@
case ALC269_TYPE_ALC255:
case ALC269_TYPE_ALC256:
case ALC269_TYPE_ALC225:
+ case ALC269_TYPE_ALC294:
ssids = alc269_ssids;
break;
default:
@@ -6028,6 +6035,11 @@
case 0x10ec0225:
spec->codec_variant = ALC269_TYPE_ALC225;
break;
+ case 0x10ec0234:
+ case 0x10ec0274:
+ case 0x10ec0294:
+ spec->codec_variant = ALC269_TYPE_ALC294;
+ break;
}
if (snd_hda_codec_read(codec, 0x51, 0, AC_VERB_PARAMETERS, 0) == 0x10ec5505) {
@@ -6942,6 +6954,7 @@
HDA_CODEC_ENTRY(0x10ec0225, "ALC225", patch_alc269),
HDA_CODEC_ENTRY(0x10ec0231, "ALC231", patch_alc269),
HDA_CODEC_ENTRY(0x10ec0233, "ALC233", patch_alc269),
+ HDA_CODEC_ENTRY(0x10ec0234, "ALC234", patch_alc269),
HDA_CODEC_ENTRY(0x10ec0235, "ALC233", patch_alc269),
HDA_CODEC_ENTRY(0x10ec0255, "ALC255", patch_alc269),
HDA_CODEC_ENTRY(0x10ec0256, "ALC256", patch_alc269),
@@ -6952,6 +6965,7 @@
HDA_CODEC_ENTRY(0x10ec0269, "ALC269", patch_alc269),
HDA_CODEC_ENTRY(0x10ec0270, "ALC270", patch_alc269),
HDA_CODEC_ENTRY(0x10ec0272, "ALC272", patch_alc662),
+ HDA_CODEC_ENTRY(0x10ec0274, "ALC274", patch_alc269),
HDA_CODEC_ENTRY(0x10ec0275, "ALC275", patch_alc269),
HDA_CODEC_ENTRY(0x10ec0276, "ALC276", patch_alc269),
HDA_CODEC_ENTRY(0x10ec0280, "ALC280", patch_alc269),
@@ -6964,6 +6978,7 @@
HDA_CODEC_ENTRY(0x10ec0290, "ALC290", patch_alc269),
HDA_CODEC_ENTRY(0x10ec0292, "ALC292", patch_alc269),
HDA_CODEC_ENTRY(0x10ec0293, "ALC293", patch_alc269),
+ HDA_CODEC_ENTRY(0x10ec0294, "ALC294", patch_alc269),
HDA_CODEC_ENTRY(0x10ec0298, "ALC298", patch_alc269),
HDA_CODEC_REV_ENTRY(0x10ec0861, 0x100340, "ALC660", patch_alc861),
HDA_CODEC_ENTRY(0x10ec0660, "ALC660-VD", patch_alc861vd),
diff --git a/sound/pci/intel8x0.c b/sound/pci/intel8x0.c
index 8151318..9720a30 100644
--- a/sound/pci/intel8x0.c
+++ b/sound/pci/intel8x0.c
@@ -42,12 +42,6 @@
#include <asm/pgtable.h>
#include <asm/cacheflush.h>
-#ifdef CONFIG_KVM_GUEST
-#include <linux/kvm_para.h>
-#else
-#define kvm_para_available() (0)
-#endif
-
MODULE_AUTHOR("Jaroslav Kysela <perex@perex.cz>");
MODULE_DESCRIPTION("Intel 82801AA,82901AB,i810,i820,i830,i840,i845,MX440; SiS 7012; Ali 5455");
MODULE_LICENSE("GPL");
@@ -2972,25 +2966,17 @@
goto fini;
}
- /* detect KVM and Parallels virtual environments */
- result = kvm_para_available();
-#ifdef X86_FEATURE_HYPERVISOR
- result = result || boot_cpu_has(X86_FEATURE_HYPERVISOR);
-#endif
- if (!result)
- goto fini;
-
/* check for known (emulated) devices */
+ result = 0;
if (pci->subsystem_vendor == PCI_SUBVENDOR_ID_REDHAT_QUMRANET &&
pci->subsystem_device == PCI_SUBDEVICE_ID_QEMU) {
/* KVM emulated sound, PCI SSID: 1af4:1100 */
msg = "enable KVM";
+ result = 1;
} else if (pci->subsystem_vendor == 0x1ab8) {
/* Parallels VM emulated sound, PCI SSID: 1ab8:xxxx */
msg = "enable Parallels VM";
- } else {
- msg = "disable (unknown or VT-d) VM";
- result = 0;
+ result = 1;
}
fini:
diff --git a/sound/pci/lx6464es/lx_core.c b/sound/pci/lx6464es/lx_core.c
index f3d6202..a80684b 100644
--- a/sound/pci/lx6464es/lx_core.c
+++ b/sound/pci/lx6464es/lx_core.c
@@ -644,7 +644,7 @@
if (err < 0)
return err;
- if (current_state == state)
+ if (!err && current_state == state)
return 0;
mdelay(1);
diff --git a/sound/soc/atmel/atmel_ssc_dai.c b/sound/soc/atmel/atmel_ssc_dai.c
index 2768970..1267e1a 100644
--- a/sound/soc/atmel/atmel_ssc_dai.c
+++ b/sound/soc/atmel/atmel_ssc_dai.c
@@ -652,7 +652,7 @@
rcmr = SSC_BF(RCMR_PERIOD, ssc_p->rcmr_period)
| SSC_BF(RCMR_STTDLY, 1)
| SSC_BF(RCMR_START, SSC_START_RISING_RF)
- | SSC_BF(RCMR_CKI, SSC_CKI_FALLING)
+ | SSC_BF(RCMR_CKI, SSC_CKI_RISING)
| SSC_BF(RCMR_CKO, SSC_CKO_NONE)
| SSC_BF(RCMR_CKS, SSC_CKS_DIV);
@@ -692,7 +692,7 @@
rcmr = SSC_BF(RCMR_PERIOD, 0)
| SSC_BF(RCMR_STTDLY, START_DELAY)
| SSC_BF(RCMR_START, SSC_START_RISING_RF)
- | SSC_BF(RCMR_CKI, SSC_CKI_FALLING)
+ | SSC_BF(RCMR_CKI, SSC_CKI_RISING)
| SSC_BF(RCMR_CKO, SSC_CKO_NONE)
| SSC_BF(RCMR_CKS, ssc->clk_from_rk_pin ?
SSC_CKS_PIN : SSC_CKS_CLOCK);
diff --git a/sound/soc/au1x/dbdma2.c b/sound/soc/au1x/dbdma2.c
index 5741c0a..b5d1caa 100644
--- a/sound/soc/au1x/dbdma2.c
+++ b/sound/soc/au1x/dbdma2.c
@@ -206,8 +206,8 @@
stype = substream->stream;
pcd = to_dmadata(substream);
- DBG("runtime->dma_area = 0x%08lx dma_addr_t = 0x%08lx dma_size = %d "
- "runtime->min_align %d\n",
+ DBG("runtime->dma_area = 0x%08lx dma_addr_t = 0x%08lx dma_size = %zu "
+ "runtime->min_align %lu\n",
(unsigned long)runtime->dma_area,
(unsigned long)runtime->dma_addr, runtime->dma_bytes,
runtime->min_align);
diff --git a/sound/soc/bcm/bcm2835-i2s.c b/sound/soc/bcm/bcm2835-i2s.c
index 1c1f221..6ba2049 100644
--- a/sound/soc/bcm/bcm2835-i2s.c
+++ b/sound/soc/bcm/bcm2835-i2s.c
@@ -259,6 +259,9 @@
case SNDRV_PCM_FORMAT_S16_LE:
data_length = 16;
break;
+ case SNDRV_PCM_FORMAT_S24_LE:
+ data_length = 24;
+ break;
case SNDRV_PCM_FORMAT_S32_LE:
data_length = 32;
break;
@@ -273,13 +276,20 @@
/* otherwise calculate a fitting block ratio */
bclk_ratio = 2 * data_length;
- /* set target clock rate*/
- clk_set_rate(dev->clk, sampling_rate * bclk_ratio);
+ /* Clock should only be set up here if CPU is clock master */
+ switch (dev->fmt & SND_SOC_DAIFMT_MASTER_MASK) {
+ case SND_SOC_DAIFMT_CBS_CFS:
+ case SND_SOC_DAIFMT_CBS_CFM:
+ clk_set_rate(dev->clk, sampling_rate * bclk_ratio);
+ break;
+ default:
+ break;
+ }
/* Setup the frame format */
format = BCM2835_I2S_CHEN;
- if (data_length > 24)
+ if (data_length >= 24)
format |= BCM2835_I2S_CHWEX;
format |= BCM2835_I2S_CHWID((data_length-8)&0xf);
@@ -570,6 +580,7 @@
.channels_max = 2,
.rates = SNDRV_PCM_RATE_8000_192000,
.formats = SNDRV_PCM_FMTBIT_S16_LE
+ | SNDRV_PCM_FMTBIT_S24_LE
| SNDRV_PCM_FMTBIT_S32_LE
},
.capture = {
@@ -577,6 +588,7 @@
.channels_max = 2,
.rates = SNDRV_PCM_RATE_8000_192000,
.formats = SNDRV_PCM_FMTBIT_S16_LE
+ | SNDRV_PCM_FMTBIT_S24_LE
| SNDRV_PCM_FMTBIT_S32_LE
},
.ops = &bcm2835_i2s_dai_ops,
@@ -678,6 +690,15 @@
dev->dma_data[SNDRV_PCM_STREAM_PLAYBACK].maxburst = 2;
dev->dma_data[SNDRV_PCM_STREAM_CAPTURE].maxburst = 2;
+ /*
+ * Set the PACK flag to enable S16_LE support (2 S16_LE values
+ * packed into 32-bit transfers).
+ */
+ dev->dma_data[SNDRV_PCM_STREAM_PLAYBACK].flags =
+ SND_DMAENGINE_PCM_DAI_FLAG_PACK;
+ dev->dma_data[SNDRV_PCM_STREAM_CAPTURE].flags =
+ SND_DMAENGINE_PCM_DAI_FLAG_PACK;
+
/* BCLK ratio - use default */
dev->bclk_ratio = 0;
diff --git a/sound/soc/codecs/Kconfig b/sound/soc/codecs/Kconfig
index 7ef3a0c..b3afae9 100644
--- a/sound/soc/codecs/Kconfig
+++ b/sound/soc/codecs/Kconfig
@@ -88,12 +88,14 @@
select SND_SOC_MC13783 if MFD_MC13XXX
select SND_SOC_ML26124 if I2C
select SND_SOC_NAU8825 if I2C
+ select SND_SOC_HDMI_CODEC
select SND_SOC_PCM1681 if I2C
select SND_SOC_PCM179X_I2C if I2C
select SND_SOC_PCM179X_SPI if SPI_MASTER
select SND_SOC_PCM3008
select SND_SOC_PCM3168A_I2C if I2C
select SND_SOC_PCM3168A_SPI if SPI_MASTER
+ select SND_SOC_PCM5102A
select SND_SOC_PCM512x_I2C if I2C
select SND_SOC_PCM512x_SPI if SPI_MASTER
select SND_SOC_RT286 if I2C
@@ -477,6 +479,11 @@
config SND_SOC_DMIC
tristate
+config SND_SOC_HDMI_CODEC
+ tristate
+ select SND_PCM_ELD
+ select SND_PCM_IEC958
+
config SND_SOC_ES8328
tristate "Everest Semi ES8328 CODEC"
@@ -575,6 +582,9 @@
select SND_SOC_PCM3168A
select REGMAP_SPI
+config SND_SOC_PCM5102A
+ tristate
+
config SND_SOC_PCM512x
tristate
diff --git a/sound/soc/codecs/Makefile b/sound/soc/codecs/Makefile
index 185a712..b7b9941 100644
--- a/sound/soc/codecs/Makefile
+++ b/sound/soc/codecs/Makefile
@@ -81,6 +81,7 @@
snd-soc-mc13783-objs := mc13783.o
snd-soc-ml26124-objs := ml26124.o
snd-soc-nau8825-objs := nau8825.o
+snd-soc-hdmi-codec-objs := hdmi-codec.o
snd-soc-pcm1681-objs := pcm1681.o
snd-soc-pcm179x-codec-objs := pcm179x.o
snd-soc-pcm179x-i2c-objs := pcm179x-i2c.o
@@ -89,6 +90,7 @@
snd-soc-pcm3168a-objs := pcm3168a.o
snd-soc-pcm3168a-i2c-objs := pcm3168a-i2c.o
snd-soc-pcm3168a-spi-objs := pcm3168a-spi.o
+snd-soc-pcm5102a-objs := pcm5102a.o
snd-soc-pcm512x-objs := pcm512x.o
snd-soc-pcm512x-i2c-objs := pcm512x-i2c.o
snd-soc-pcm512x-spi-objs := pcm512x-spi.o
@@ -290,6 +292,7 @@
obj-$(CONFIG_SND_SOC_MC13783) += snd-soc-mc13783.o
obj-$(CONFIG_SND_SOC_ML26124) += snd-soc-ml26124.o
obj-$(CONFIG_SND_SOC_NAU8825) += snd-soc-nau8825.o
+obj-$(CONFIG_SND_SOC_HDMI_CODEC) += snd-soc-hdmi-codec.o
obj-$(CONFIG_SND_SOC_PCM1681) += snd-soc-pcm1681.o
obj-$(CONFIG_SND_SOC_PCM179X) += snd-soc-pcm179x-codec.o
obj-$(CONFIG_SND_SOC_PCM179X_I2C) += snd-soc-pcm179x-i2c.o
@@ -298,6 +301,7 @@
obj-$(CONFIG_SND_SOC_PCM3168A) += snd-soc-pcm3168a.o
obj-$(CONFIG_SND_SOC_PCM3168A_I2C) += snd-soc-pcm3168a-i2c.o
obj-$(CONFIG_SND_SOC_PCM3168A_SPI) += snd-soc-pcm3168a-spi.o
+obj-$(CONFIG_SND_SOC_PCM5102A) += snd-soc-pcm5102a.o
obj-$(CONFIG_SND_SOC_PCM512x) += snd-soc-pcm512x.o
obj-$(CONFIG_SND_SOC_PCM512x_I2C) += snd-soc-pcm512x-i2c.o
obj-$(CONFIG_SND_SOC_PCM512x_SPI) += snd-soc-pcm512x-spi.o
diff --git a/sound/soc/codecs/ak4642.c b/sound/soc/codecs/ak4642.c
index cda27c2..1ee8506 100644
--- a/sound/soc/codecs/ak4642.c
+++ b/sound/soc/codecs/ak4642.c
@@ -608,9 +608,7 @@
of_property_read_string(np, "clock-output-names", &clk_name);
- clk = clk_register_fixed_rate(dev, clk_name, parent_clk_name,
- (parent_clk_name) ? 0 : CLK_IS_ROOT,
- rate);
+ clk = clk_register_fixed_rate(dev, clk_name, parent_clk_name, 0, rate);
if (!IS_ERR(clk))
of_clk_add_provider(np, of_clk_src_simple_get, clk);
diff --git a/sound/soc/codecs/arizona.c b/sound/soc/codecs/arizona.c
index 8395931..664a8c0 100644
--- a/sound/soc/codecs/arizona.c
+++ b/sound/soc/codecs/arizona.c
@@ -221,6 +221,8 @@
switch (arizona->type) {
case WM8997:
+ case CS47L24:
+ case WM1831:
break;
default:
ret = snd_soc_dapm_new_controls(dapm, &arizona_spkr, 1);
@@ -1134,7 +1136,6 @@
int event)
{
struct snd_soc_codec *codec = snd_soc_dapm_to_codec(w->dapm);
- unsigned int mask = 0x3 << w->shift;
unsigned int val;
switch (event) {
@@ -1148,7 +1149,7 @@
return 0;
}
- snd_soc_update_bits(codec, ARIZONA_CLOCK_CONTROL, mask, val);
+ snd_soc_write(codec, ARIZONA_CLOCK_CONTROL, val);
return 0;
}
@@ -2047,7 +2048,21 @@
init_ratio, Fref, refdiv);
while (div <= ARIZONA_FLL_MAX_REFDIV) {
- for (ratio = init_ratio; ratio <= ARIZONA_FLL_MAX_FRATIO;
+ /* start from init_ratio because this may already give a
+ * fractional N.K
+ */
+ for (ratio = init_ratio; ratio > 0; ratio--) {
+ if (target % (ratio * Fref)) {
+ cfg->refdiv = refdiv;
+ cfg->fratio = ratio - 1;
+ arizona_fll_dbg(fll,
+ "pseudo: found fref=%u refdiv=%d(%d) ratio=%d\n",
+ Fref, refdiv, div, ratio);
+ return ratio;
+ }
+ }
+
+ for (ratio = init_ratio + 1; ratio <= ARIZONA_FLL_MAX_FRATIO;
ratio++) {
if ((ARIZONA_FLL_VCO_CORNER / 2) /
(fll->vco_mult * ratio) < Fref) {
@@ -2073,17 +2088,6 @@
}
}
- for (ratio = init_ratio - 1; ratio > 0; ratio--) {
- if (target % (ratio * Fref)) {
- cfg->refdiv = refdiv;
- cfg->fratio = ratio - 1;
- arizona_fll_dbg(fll,
- "pseudo: found fref=%u refdiv=%d(%d) ratio=%d\n",
- Fref, refdiv, div, ratio);
- return ratio;
- }
- }
-
div *= 2;
Fref /= 2;
refdiv++;
diff --git a/sound/soc/codecs/cs42l56.c b/sound/soc/codecs/cs42l56.c
index 7cd5f76..eec1ff8 100644
--- a/sound/soc/codecs/cs42l56.c
+++ b/sound/soc/codecs/cs42l56.c
@@ -56,7 +56,7 @@
u8 iface;
u8 iface_fmt;
u8 iface_inv;
-#if defined(CONFIG_INPUT) || defined(CONFIG_INPUT_MODULE)
+#if IS_ENABLED(CONFIG_INPUT)
struct input_dev *beep;
struct work_struct beep_work;
int beep_rate;
diff --git a/sound/soc/codecs/cs47l24.c b/sound/soc/codecs/cs47l24.c
index 00e9b6fc..5ec5a68 100644
--- a/sound/soc/codecs/cs47l24.c
+++ b/sound/soc/codecs/cs47l24.c
@@ -807,6 +807,9 @@
{ "IN2L PGA", NULL, "IN2L" },
{ "IN2R PGA", NULL, "IN2R" },
+ { "Audio Trace DSP", NULL, "DSP2" },
+ { "Audio Trace DSP", NULL, "SYSCLK" },
+
ARIZONA_MIXER_ROUTES("OUT1L", "HPOUT1L"),
ARIZONA_MIXER_ROUTES("OUT1R", "HPOUT1R"),
@@ -1016,6 +1019,27 @@
.formats = CS47L24_FORMATS,
},
},
+ {
+ .name = "cs47l24-cpu-trace",
+ .capture = {
+ .stream_name = "Audio Trace CPU",
+ .channels_min = 1,
+ .channels_max = 6,
+ .rates = CS47L24_RATES,
+ .formats = CS47L24_FORMATS,
+ },
+ .compress_new = snd_soc_new_compress,
+ },
+ {
+ .name = "cs47l24-dsp-trace",
+ .capture = {
+ .stream_name = "Audio Trace DSP",
+ .channels_min = 1,
+ .channels_max = 6,
+ .rates = CS47L24_RATES,
+ .formats = CS47L24_FORMATS,
+ },
+ },
};
static int cs47l24_open(struct snd_compr_stream *stream)
@@ -1027,6 +1051,8 @@
if (strcmp(rtd->codec_dai->name, "cs47l24-dsp-voicectrl") == 0) {
n_adsp = 2;
+ } else if (strcmp(rtd->codec_dai->name, "cs47l24-dsp-trace") == 0) {
+ n_adsp = 1;
} else {
dev_err(arizona->dev,
"No suitable compressed stream for DAI '%s'\n",
@@ -1041,10 +1067,16 @@
{
struct cs47l24_priv *priv = data;
struct arizona *arizona = priv->core.arizona;
- int ret;
+ int serviced = 0;
+ int i, ret;
- ret = wm_adsp_compr_handle_irq(&priv->core.adsp[2]);
- if (ret == -ENODEV) {
+ for (i = 1; i <= 2; ++i) {
+ ret = wm_adsp_compr_handle_irq(&priv->core.adsp[i]);
+ if (ret != -ENODEV)
+ serviced++;
+ }
+
+ if (!serviced) {
dev_err(arizona->dev, "Spurious compressed data IRQ\n");
return IRQ_NONE;
}
@@ -1160,6 +1192,7 @@
static struct snd_soc_platform_driver cs47l24_compr_platform = {
.compr_ops = &cs47l24_compr_ops,
};
+
static int cs47l24_probe(struct platform_device *pdev)
{
struct arizona *arizona = dev_get_drvdata(pdev->dev.parent);
@@ -1228,9 +1261,9 @@
dev_err(&pdev->dev, "Failed to register platform: %d\n", ret);
return ret;
}
+
ret = snd_soc_register_codec(&pdev->dev, &soc_codec_dev_cs47l24,
cs47l24_dai, ARRAY_SIZE(cs47l24_dai));
-
if (ret < 0) {
dev_err(&pdev->dev, "Failed to register codec: %d\n", ret);
snd_soc_unregister_platform(&pdev->dev);
@@ -1241,10 +1274,15 @@
static int cs47l24_remove(struct platform_device *pdev)
{
+ struct cs47l24_priv *cs47l24 = platform_get_drvdata(pdev);
+
snd_soc_unregister_platform(&pdev->dev);
snd_soc_unregister_codec(&pdev->dev);
pm_runtime_disable(&pdev->dev);
+ wm_adsp2_remove(&cs47l24->core.adsp[1]);
+ wm_adsp2_remove(&cs47l24->core.adsp[2]);
+
return 0;
}
diff --git a/sound/soc/codecs/da7213.c b/sound/soc/codecs/da7213.c
index 7278f93..e5527bc 100644
--- a/sound/soc/codecs/da7213.c
+++ b/sound/soc/codecs/da7213.c
@@ -726,6 +726,68 @@
/*
+ * DAPM Events
+ */
+
+static int da7213_dai_event(struct snd_soc_dapm_widget *w,
+ struct snd_kcontrol *kcontrol, int event)
+{
+ struct snd_soc_codec *codec = snd_soc_dapm_to_codec(w->dapm);
+ struct da7213_priv *da7213 = snd_soc_codec_get_drvdata(codec);
+ u8 pll_ctrl, pll_status;
+ int i = 0;
+ bool srm_lock = false;
+
+ switch (event) {
+ case SND_SOC_DAPM_PRE_PMU:
+ /* Enable DAI clks for master mode */
+ if (da7213->master)
+ snd_soc_update_bits(codec, DA7213_DAI_CLK_MODE,
+ DA7213_DAI_CLK_EN_MASK,
+ DA7213_DAI_CLK_EN_MASK);
+
+ /* PC synchronised to DAI */
+ snd_soc_update_bits(codec, DA7213_PC_COUNT,
+ DA7213_PC_FREERUN_MASK, 0);
+
+ /* Slave mode, if SRM not enabled no need for status checks */
+ pll_ctrl = snd_soc_read(codec, DA7213_PLL_CTRL);
+ if (!(pll_ctrl & DA7213_PLL_SRM_EN))
+ return 0;
+
+ /* Check SRM has locked */
+ do {
+ pll_status = snd_soc_read(codec, DA7213_PLL_STATUS);
+ if (pll_status & DA7219_PLL_SRM_LOCK) {
+ srm_lock = true;
+ } else {
+ ++i;
+ msleep(50);
+ }
+ } while ((i < DA7213_SRM_CHECK_RETRIES) & (!srm_lock));
+
+ if (!srm_lock)
+ dev_warn(codec->dev, "SRM failed to lock\n");
+
+ return 0;
+ case SND_SOC_DAPM_POST_PMD:
+ /* PC free-running */
+ snd_soc_update_bits(codec, DA7213_PC_COUNT,
+ DA7213_PC_FREERUN_MASK,
+ DA7213_PC_FREERUN_MASK);
+
+ /* Disable DAI clks if in master mode */
+ if (da7213->master)
+ snd_soc_update_bits(codec, DA7213_DAI_CLK_MODE,
+ DA7213_DAI_CLK_EN_MASK, 0);
+ return 0;
+ default:
+ return -EINVAL;
+ }
+}
+
+
+/*
* DAPM widgets
*/
@@ -736,7 +798,8 @@
/* Use a supply here as this controls both input & output DAIs */
SND_SOC_DAPM_SUPPLY("DAI", DA7213_DAI_CTRL, DA7213_DAI_EN_SHIFT,
- DA7213_NO_INVERT, NULL, 0),
+ DA7213_NO_INVERT, da7213_dai_event,
+ SND_SOC_DAPM_PRE_PMU | SND_SOC_DAPM_POST_PMD),
/*
* Input
@@ -1143,11 +1206,9 @@
/* Set master/slave mode */
switch (fmt & SND_SOC_DAIFMT_MASTER_MASK) {
case SND_SOC_DAIFMT_CBM_CFM:
- dai_clk_mode |= DA7213_DAI_CLK_EN_MASTER_MODE;
da7213->master = true;
break;
case SND_SOC_DAIFMT_CBS_CFS:
- dai_clk_mode |= DA7213_DAI_CLK_EN_SLAVE_MODE;
da7213->master = false;
break;
default:
@@ -1281,28 +1342,28 @@
pll_ctrl = 0;
/* Workout input divider based on MCLK rate */
- if ((da7213->mclk_rate == 32768) && (source == DA7213_SYSCLK_PLL)) {
+ if (da7213->mclk_rate == 32768) {
/* 32KHz PLL Mode */
- indiv_bits = DA7213_PLL_INDIV_10_20_MHZ;
- indiv = DA7213_PLL_INDIV_10_20_MHZ_VAL;
+ indiv_bits = DA7213_PLL_INDIV_9_TO_18_MHZ;
+ indiv = DA7213_PLL_INDIV_9_TO_18_MHZ_VAL;
freq_ref = 3750000;
pll_ctrl |= DA7213_PLL_32K_MODE;
} else {
/* 5 - 54MHz MCLK */
if (da7213->mclk_rate < 5000000) {
goto pll_err;
- } else if (da7213->mclk_rate <= 10000000) {
- indiv_bits = DA7213_PLL_INDIV_5_10_MHZ;
- indiv = DA7213_PLL_INDIV_5_10_MHZ_VAL;
- } else if (da7213->mclk_rate <= 20000000) {
- indiv_bits = DA7213_PLL_INDIV_10_20_MHZ;
- indiv = DA7213_PLL_INDIV_10_20_MHZ_VAL;
- } else if (da7213->mclk_rate <= 40000000) {
- indiv_bits = DA7213_PLL_INDIV_20_40_MHZ;
- indiv = DA7213_PLL_INDIV_20_40_MHZ_VAL;
+ } else if (da7213->mclk_rate <= 9000000) {
+ indiv_bits = DA7213_PLL_INDIV_5_TO_9_MHZ;
+ indiv = DA7213_PLL_INDIV_5_TO_9_MHZ_VAL;
+ } else if (da7213->mclk_rate <= 18000000) {
+ indiv_bits = DA7213_PLL_INDIV_9_TO_18_MHZ;
+ indiv = DA7213_PLL_INDIV_9_TO_18_MHZ_VAL;
+ } else if (da7213->mclk_rate <= 36000000) {
+ indiv_bits = DA7213_PLL_INDIV_18_TO_36_MHZ;
+ indiv = DA7213_PLL_INDIV_18_TO_36_MHZ_VAL;
} else if (da7213->mclk_rate <= 54000000) {
- indiv_bits = DA7213_PLL_INDIV_40_54_MHZ;
- indiv = DA7213_PLL_INDIV_40_54_MHZ_VAL;
+ indiv_bits = DA7213_PLL_INDIV_36_TO_54_MHZ;
+ indiv = DA7213_PLL_INDIV_36_TO_54_MHZ_VAL;
} else {
goto pll_err;
}
@@ -1547,6 +1608,10 @@
/* Default to using SRM for slave mode */
da7213->srm_en = true;
+ /* Default PC counter to free-running */
+ snd_soc_update_bits(codec, DA7213_PC_COUNT, DA7213_PC_FREERUN_MASK,
+ DA7213_PC_FREERUN_MASK);
+
/* Enable all Gain Ramps */
snd_soc_update_bits(codec, DA7213_AUX_L_CTRL,
DA7213_GAIN_RAMP_EN, DA7213_GAIN_RAMP_EN);
diff --git a/sound/soc/codecs/da7213.h b/sound/soc/codecs/da7213.h
index 030fd69..fbb7a35 100644
--- a/sound/soc/codecs/da7213.h
+++ b/sound/soc/codecs/da7213.h
@@ -142,6 +142,9 @@
* Bit fields
*/
+/* DA7213_PLL_STATUS = 0x03 */
+#define DA7219_PLL_SRM_LOCK (0x1 << 1)
+
/* DA7213_SR = 0x22 */
#define DA7213_SR_8000 (0x1 << 0)
#define DA7213_SR_11025 (0x2 << 0)
@@ -160,10 +163,10 @@
#define DA7213_VMID_EN (0x1 << 7)
/* DA7213_PLL_CTRL = 0x27 */
-#define DA7213_PLL_INDIV_5_10_MHZ (0x0 << 2)
-#define DA7213_PLL_INDIV_10_20_MHZ (0x1 << 2)
-#define DA7213_PLL_INDIV_20_40_MHZ (0x2 << 2)
-#define DA7213_PLL_INDIV_40_54_MHZ (0x3 << 2)
+#define DA7213_PLL_INDIV_5_TO_9_MHZ (0x0 << 2)
+#define DA7213_PLL_INDIV_9_TO_18_MHZ (0x1 << 2)
+#define DA7213_PLL_INDIV_18_TO_36_MHZ (0x2 << 2)
+#define DA7213_PLL_INDIV_36_TO_54_MHZ (0x3 << 2)
#define DA7213_PLL_INDIV_MASK (0x3 << 2)
#define DA7213_PLL_MCLK_SQR_EN (0x1 << 4)
#define DA7213_PLL_32K_MODE (0x1 << 5)
@@ -178,8 +181,6 @@
#define DA7213_DAI_BCLKS_PER_WCLK_MASK (0x3 << 0)
#define DA7213_DAI_CLK_POL_INV (0x1 << 2)
#define DA7213_DAI_WCLK_POL_INV (0x1 << 3)
-#define DA7213_DAI_CLK_EN_SLAVE_MODE (0x0 << 7)
-#define DA7213_DAI_CLK_EN_MASTER_MODE (0x1 << 7)
#define DA7213_DAI_CLK_EN_MASK (0x1 << 7)
/* DA7213_DAI_CTRL = 0x29 */
@@ -412,6 +413,9 @@
#define DA7213_DMIC_CLK_RATE_SHIFT 2
#define DA7213_DMIC_CLK_RATE_MASK (0x1 << 2)
+/* DA7213_PC_COUNT = 0x94 */
+#define DA7213_PC_FREERUN_MASK (0x1 << 0)
+
/* DA7213_DIG_CTRL = 0x99 */
#define DA7213_DAC_L_INV_SHIFT 3
#define DA7213_DAC_R_INV_SHIFT 7
@@ -495,15 +499,16 @@
#define DA7213_ALC_AVG_ITERATIONS 5
/* PLL related */
-#define DA7213_SYSCLK_MCLK 0
-#define DA7213_SYSCLK_PLL 1
-#define DA7213_PLL_FREQ_OUT_90316800 90316800
-#define DA7213_PLL_FREQ_OUT_98304000 98304000
-#define DA7213_PLL_FREQ_OUT_94310400 94310400
-#define DA7213_PLL_INDIV_5_10_MHZ_VAL 2
-#define DA7213_PLL_INDIV_10_20_MHZ_VAL 4
-#define DA7213_PLL_INDIV_20_40_MHZ_VAL 8
-#define DA7213_PLL_INDIV_40_54_MHZ_VAL 16
+#define DA7213_SYSCLK_MCLK 0
+#define DA7213_SYSCLK_PLL 1
+#define DA7213_PLL_FREQ_OUT_90316800 90316800
+#define DA7213_PLL_FREQ_OUT_98304000 98304000
+#define DA7213_PLL_FREQ_OUT_94310400 94310400
+#define DA7213_PLL_INDIV_5_TO_9_MHZ_VAL 2
+#define DA7213_PLL_INDIV_9_TO_18_MHZ_VAL 4
+#define DA7213_PLL_INDIV_18_TO_36_MHZ_VAL 8
+#define DA7213_PLL_INDIV_36_TO_54_MHZ_VAL 16
+#define DA7213_SRM_CHECK_RETRIES 8
enum da7213_clk_src {
DA7213_CLKSRC_MCLK = 0,
diff --git a/sound/soc/codecs/da7218.c b/sound/soc/codecs/da7218.c
index 93575f2..99ce23e 100644
--- a/sound/soc/codecs/da7218.c
+++ b/sound/soc/codecs/da7218.c
@@ -1868,27 +1868,27 @@
/* Verify 32KHz, 2MHz - 54MHz MCLK provided, and set input divider */
if (da7218->mclk_rate == 32768) {
- indiv_bits = DA7218_PLL_INDIV_2_5_MHZ;
- indiv = DA7218_PLL_INDIV_2_10_MHZ_VAL;
+ indiv_bits = DA7218_PLL_INDIV_9_TO_18_MHZ;
+ indiv = DA7218_PLL_INDIV_9_TO_18_MHZ_VAL;
} else if (da7218->mclk_rate < 2000000) {
dev_err(codec->dev, "PLL input clock %d below valid range\n",
da7218->mclk_rate);
return -EINVAL;
- } else if (da7218->mclk_rate <= 5000000) {
- indiv_bits = DA7218_PLL_INDIV_2_5_MHZ;
- indiv = DA7218_PLL_INDIV_2_10_MHZ_VAL;
- } else if (da7218->mclk_rate <= 10000000) {
- indiv_bits = DA7218_PLL_INDIV_5_10_MHZ;
- indiv = DA7218_PLL_INDIV_2_10_MHZ_VAL;
- } else if (da7218->mclk_rate <= 20000000) {
- indiv_bits = DA7218_PLL_INDIV_10_20_MHZ;
- indiv = DA7218_PLL_INDIV_10_20_MHZ_VAL;
- } else if (da7218->mclk_rate <= 40000000) {
- indiv_bits = DA7218_PLL_INDIV_20_40_MHZ;
- indiv = DA7218_PLL_INDIV_20_40_MHZ_VAL;
+ } else if (da7218->mclk_rate <= 4500000) {
+ indiv_bits = DA7218_PLL_INDIV_2_TO_4_5_MHZ;
+ indiv = DA7218_PLL_INDIV_2_TO_4_5_MHZ_VAL;
+ } else if (da7218->mclk_rate <= 9000000) {
+ indiv_bits = DA7218_PLL_INDIV_4_5_TO_9_MHZ;
+ indiv = DA7218_PLL_INDIV_4_5_TO_9_MHZ_VAL;
+ } else if (da7218->mclk_rate <= 18000000) {
+ indiv_bits = DA7218_PLL_INDIV_9_TO_18_MHZ;
+ indiv = DA7218_PLL_INDIV_9_TO_18_MHZ_VAL;
+ } else if (da7218->mclk_rate <= 36000000) {
+ indiv_bits = DA7218_PLL_INDIV_18_TO_36_MHZ;
+ indiv = DA7218_PLL_INDIV_18_TO_36_MHZ_VAL;
} else if (da7218->mclk_rate <= 54000000) {
- indiv_bits = DA7218_PLL_INDIV_40_54_MHZ;
- indiv = DA7218_PLL_INDIV_40_54_MHZ_VAL;
+ indiv_bits = DA7218_PLL_INDIV_36_TO_54_MHZ;
+ indiv = DA7218_PLL_INDIV_36_TO_54_MHZ_VAL;
} else {
dev_err(codec->dev, "PLL input clock %d above valid range\n",
da7218->mclk_rate);
diff --git a/sound/soc/codecs/da7218.h b/sound/soc/codecs/da7218.h
index c2c5904..477cd37 100644
--- a/sound/soc/codecs/da7218.h
+++ b/sound/soc/codecs/da7218.h
@@ -876,15 +876,11 @@
/* DA7218_PLL_CTRL = 0x91 */
#define DA7218_PLL_INDIV_SHIFT 0
#define DA7218_PLL_INDIV_MASK (0x7 << 0)
-#define DA7218_PLL_INDIV_2_5_MHZ (0x0 << 0)
-#define DA7218_PLL_INDIV_5_10_MHZ (0x1 << 0)
-#define DA7218_PLL_INDIV_10_20_MHZ (0x2 << 0)
-#define DA7218_PLL_INDIV_20_40_MHZ (0x3 << 0)
-#define DA7218_PLL_INDIV_40_54_MHZ (0x4 << 0)
-#define DA7218_PLL_INDIV_2_10_MHZ_VAL 2
-#define DA7218_PLL_INDIV_10_20_MHZ_VAL 4
-#define DA7218_PLL_INDIV_20_40_MHZ_VAL 8
-#define DA7218_PLL_INDIV_40_54_MHZ_VAL 16
+#define DA7218_PLL_INDIV_2_TO_4_5_MHZ (0x0 << 0)
+#define DA7218_PLL_INDIV_4_5_TO_9_MHZ (0x1 << 0)
+#define DA7218_PLL_INDIV_9_TO_18_MHZ (0x2 << 0)
+#define DA7218_PLL_INDIV_18_TO_36_MHZ (0x3 << 0)
+#define DA7218_PLL_INDIV_36_TO_54_MHZ (0x4 << 0)
#define DA7218_PLL_MCLK_SQR_EN_SHIFT 4
#define DA7218_PLL_MCLK_SQR_EN_MASK (0x1 << 4)
#define DA7218_PLL_MODE_SHIFT 6
@@ -1336,6 +1332,13 @@
#define DA7218_PLL_FREQ_OUT_90316 90316800
#define DA7218_PLL_FREQ_OUT_98304 98304000
+/* PLL Frequency Dividers */
+#define DA7218_PLL_INDIV_2_TO_4_5_MHZ_VAL 1
+#define DA7218_PLL_INDIV_4_5_TO_9_MHZ_VAL 2
+#define DA7218_PLL_INDIV_9_TO_18_MHZ_VAL 4
+#define DA7218_PLL_INDIV_18_TO_36_MHZ_VAL 8
+#define DA7218_PLL_INDIV_36_TO_54_MHZ_VAL 16
+
/* ALC Calibration */
#define DA7218_ALC_CALIB_DELAY_MIN 2500
#define DA7218_ALC_CALIB_DELAY_MAX 5000
diff --git a/sound/soc/codecs/da7219.c b/sound/soc/codecs/da7219.c
index 81c0708..5c93899 100644
--- a/sound/soc/codecs/da7219.c
+++ b/sound/soc/codecs/da7219.c
@@ -11,6 +11,7 @@
* option) any later version.
*/
+#include <linux/acpi.h>
#include <linux/clk.h>
#include <linux/i2c.h>
#include <linux/of_device.h>
@@ -1025,7 +1026,7 @@
if ((da7219->clk_src == clk_id) && (da7219->mclk_rate == freq))
return 0;
- if (((freq < 2000000) && (freq != 32768)) || (freq > 54000000)) {
+ if ((freq < 2000000) || (freq > 54000000)) {
dev_err(codec_dai->dev, "Unsupported MCLK value %d\n",
freq);
return -EINVAL;
@@ -1079,21 +1080,21 @@
dev_err(codec->dev, "PLL input clock %d below valid range\n",
da7219->mclk_rate);
return -EINVAL;
- } else if (da7219->mclk_rate <= 5000000) {
- indiv_bits = DA7219_PLL_INDIV_2_5_MHZ;
- indiv = DA7219_PLL_INDIV_2_5_MHZ_VAL;
- } else if (da7219->mclk_rate <= 10000000) {
- indiv_bits = DA7219_PLL_INDIV_5_10_MHZ;
- indiv = DA7219_PLL_INDIV_5_10_MHZ_VAL;
- } else if (da7219->mclk_rate <= 20000000) {
- indiv_bits = DA7219_PLL_INDIV_10_20_MHZ;
- indiv = DA7219_PLL_INDIV_10_20_MHZ_VAL;
- } else if (da7219->mclk_rate <= 40000000) {
- indiv_bits = DA7219_PLL_INDIV_20_40_MHZ;
- indiv = DA7219_PLL_INDIV_20_40_MHZ_VAL;
+ } else if (da7219->mclk_rate <= 4500000) {
+ indiv_bits = DA7219_PLL_INDIV_2_TO_4_5_MHZ;
+ indiv = DA7219_PLL_INDIV_2_TO_4_5_MHZ_VAL;
+ } else if (da7219->mclk_rate <= 9000000) {
+ indiv_bits = DA7219_PLL_INDIV_4_5_TO_9_MHZ;
+ indiv = DA7219_PLL_INDIV_4_5_TO_9_MHZ_VAL;
+ } else if (da7219->mclk_rate <= 18000000) {
+ indiv_bits = DA7219_PLL_INDIV_9_TO_18_MHZ;
+ indiv = DA7219_PLL_INDIV_9_TO_18_MHZ_VAL;
+ } else if (da7219->mclk_rate <= 36000000) {
+ indiv_bits = DA7219_PLL_INDIV_18_TO_36_MHZ;
+ indiv = DA7219_PLL_INDIV_18_TO_36_MHZ_VAL;
} else if (da7219->mclk_rate <= 54000000) {
- indiv_bits = DA7219_PLL_INDIV_40_54_MHZ;
- indiv = DA7219_PLL_INDIV_40_54_MHZ_VAL;
+ indiv_bits = DA7219_PLL_INDIV_36_TO_54_MHZ;
+ indiv = DA7219_PLL_INDIV_36_TO_54_MHZ_VAL;
} else {
dev_err(codec->dev, "PLL input clock %d above valid range\n",
da7219->mclk_rate);
@@ -1426,6 +1427,12 @@
};
MODULE_DEVICE_TABLE(of, da7219_of_match);
+static const struct acpi_device_id da7219_acpi_match[] = {
+ { .id = "DLGS7219", },
+ { }
+};
+MODULE_DEVICE_TABLE(acpi, da7219_acpi_match);
+
static enum da7219_micbias_voltage
da7219_of_micbias_lvl(struct snd_soc_codec *codec, u32 val)
{
@@ -1955,6 +1962,7 @@
.driver = {
.name = "da7219",
.of_match_table = of_match_ptr(da7219_of_match),
+ .acpi_match_table = ACPI_PTR(da7219_acpi_match),
},
.probe = da7219_i2c_probe,
.remove = da7219_i2c_remove,
diff --git a/sound/soc/codecs/da7219.h b/sound/soc/codecs/da7219.h
index 5a787e7..ff2a2f0 100644
--- a/sound/soc/codecs/da7219.h
+++ b/sound/soc/codecs/da7219.h
@@ -194,11 +194,11 @@
/* DA7219_PLL_CTRL = 0x20 */
#define DA7219_PLL_INDIV_SHIFT 2
#define DA7219_PLL_INDIV_MASK (0x7 << 2)
-#define DA7219_PLL_INDIV_2_5_MHZ (0x0 << 2)
-#define DA7219_PLL_INDIV_5_10_MHZ (0x1 << 2)
-#define DA7219_PLL_INDIV_10_20_MHZ (0x2 << 2)
-#define DA7219_PLL_INDIV_20_40_MHZ (0x3 << 2)
-#define DA7219_PLL_INDIV_40_54_MHZ (0x4 << 2)
+#define DA7219_PLL_INDIV_2_TO_4_5_MHZ (0x0 << 2)
+#define DA7219_PLL_INDIV_4_5_TO_9_MHZ (0x1 << 2)
+#define DA7219_PLL_INDIV_9_TO_18_MHZ (0x2 << 2)
+#define DA7219_PLL_INDIV_18_TO_36_MHZ (0x3 << 2)
+#define DA7219_PLL_INDIV_36_TO_54_MHZ (0x4 << 2)
#define DA7219_PLL_MCLK_SQR_EN_SHIFT 5
#define DA7219_PLL_MCLK_SQR_EN_MASK (0x1 << 5)
#define DA7219_PLL_MODE_SHIFT 6
@@ -761,11 +761,11 @@
#define DA7219_PLL_FREQ_OUT_98304 98304000
/* PLL Frequency Dividers */
-#define DA7219_PLL_INDIV_2_5_MHZ_VAL 1
-#define DA7219_PLL_INDIV_5_10_MHZ_VAL 2
-#define DA7219_PLL_INDIV_10_20_MHZ_VAL 4
-#define DA7219_PLL_INDIV_20_40_MHZ_VAL 8
-#define DA7219_PLL_INDIV_40_54_MHZ_VAL 16
+#define DA7219_PLL_INDIV_2_TO_4_5_MHZ_VAL 1
+#define DA7219_PLL_INDIV_4_5_TO_9_MHZ_VAL 2
+#define DA7219_PLL_INDIV_9_TO_18_MHZ_VAL 4
+#define DA7219_PLL_INDIV_18_TO_36_MHZ_VAL 8
+#define DA7219_PLL_INDIV_36_TO_54_MHZ_VAL 16
/* SRM */
#define DA7219_SRM_CHECK_RETRIES 8
diff --git a/sound/soc/codecs/es8328.c b/sound/soc/codecs/es8328.c
index afa6c5d..2086d71 100644
--- a/sound/soc/codecs/es8328.c
+++ b/sound/soc/codecs/es8328.c
@@ -26,18 +26,30 @@
#include <sound/tlv.h>
#include "es8328.h"
-#define ES8328_SYSCLK_RATE_1X 11289600
-#define ES8328_SYSCLK_RATE_2X 22579200
+static const unsigned int rates_12288[] = {
+ 8000, 12000, 16000, 24000, 32000, 48000, 96000,
+};
-/* Run the codec at 22.5792 or 11.2896 MHz to support these rates */
-static struct {
- int rate;
- u8 ratio;
-} mclk_ratios[] = {
- { 8000, 9 },
- {11025, 7 },
- {22050, 4 },
- {44100, 2 },
+static const int ratios_12288[] = {
+ 10, 7, 6, 4, 3, 2, 0,
+};
+
+static const struct snd_pcm_hw_constraint_list constraints_12288 = {
+ .count = ARRAY_SIZE(rates_12288),
+ .list = rates_12288,
+};
+
+static const unsigned int rates_11289[] = {
+ 8018, 11025, 22050, 44100, 88200,
+};
+
+static const int ratios_11289[] = {
+ 9, 7, 4, 2, 0,
+};
+
+static const struct snd_pcm_hw_constraint_list constraints_11289 = {
+ .count = ARRAY_SIZE(rates_11289),
+ .list = rates_11289,
};
/* regulator supplies for sgtl5000, VDDD is an optional external supply */
@@ -57,16 +69,28 @@
"HPVDD",
};
-#define ES8328_RATES (SNDRV_PCM_RATE_44100 | \
+#define ES8328_RATES (SNDRV_PCM_RATE_96000 | \
+ SNDRV_PCM_RATE_48000 | \
+ SNDRV_PCM_RATE_44100 | \
+ SNDRV_PCM_RATE_32000 | \
SNDRV_PCM_RATE_22050 | \
- SNDRV_PCM_RATE_11025)
-#define ES8328_FORMATS (SNDRV_PCM_FMTBIT_S16_LE)
+ SNDRV_PCM_RATE_16000 | \
+ SNDRV_PCM_RATE_11025 | \
+ SNDRV_PCM_RATE_8000)
+#define ES8328_FORMATS (SNDRV_PCM_FMTBIT_S16_LE | \
+ SNDRV_PCM_FMTBIT_S18_3LE | \
+ SNDRV_PCM_FMTBIT_S20_3LE | \
+ SNDRV_PCM_FMTBIT_S24_LE | \
+ SNDRV_PCM_FMTBIT_S32_LE)
struct es8328_priv {
struct regmap *regmap;
struct clk *clk;
int playback_fs;
bool deemph;
+ int mclkdiv2;
+ const struct snd_pcm_hw_constraint_list *sysclk_constraints;
+ const int *mclk_ratios;
struct regulator_bulk_data supplies[ES8328_SUPPLY_NUM];
};
@@ -439,54 +463,131 @@
mute ? ES8328_DACCONTROL3_DACMUTE : 0);
}
+static int es8328_startup(struct snd_pcm_substream *substream,
+ struct snd_soc_dai *dai)
+{
+ struct snd_soc_codec *codec = dai->codec;
+ struct es8328_priv *es8328 = snd_soc_codec_get_drvdata(codec);
+
+ if (es8328->sysclk_constraints)
+ snd_pcm_hw_constraint_list(substream->runtime, 0,
+ SNDRV_PCM_HW_PARAM_RATE,
+ es8328->sysclk_constraints);
+
+ return 0;
+}
+
static int es8328_hw_params(struct snd_pcm_substream *substream,
struct snd_pcm_hw_params *params,
struct snd_soc_dai *dai)
{
struct snd_soc_codec *codec = dai->codec;
struct es8328_priv *es8328 = snd_soc_codec_get_drvdata(codec);
- int clk_rate;
int i;
int reg;
- u8 ratio;
+ int wl;
+ int ratio;
+
+ if (!es8328->sysclk_constraints) {
+ dev_err(codec->dev, "No MCLK configured\n");
+ return -EINVAL;
+ }
if (substream->stream == SNDRV_PCM_STREAM_PLAYBACK)
reg = ES8328_DACCONTROL2;
else
reg = ES8328_ADCCONTROL5;
- clk_rate = clk_get_rate(es8328->clk);
+ for (i = 0; i < es8328->sysclk_constraints->count; i++)
+ if (es8328->sysclk_constraints->list[i] == params_rate(params))
+ break;
- if ((clk_rate != ES8328_SYSCLK_RATE_1X) &&
- (clk_rate != ES8328_SYSCLK_RATE_2X)) {
- dev_err(codec->dev,
- "%s: clock is running at %d Hz, not %d or %d Hz\n",
- __func__, clk_rate,
- ES8328_SYSCLK_RATE_1X, ES8328_SYSCLK_RATE_2X);
+ if (i == es8328->sysclk_constraints->count) {
+ dev_err(codec->dev, "LRCLK %d unsupported with current clock\n",
+ params_rate(params));
return -EINVAL;
}
- /* find master mode MCLK to sampling frequency ratio */
- ratio = mclk_ratios[0].rate;
- for (i = 1; i < ARRAY_SIZE(mclk_ratios); i++)
- if (params_rate(params) <= mclk_ratios[i].rate)
- ratio = mclk_ratios[i].ratio;
+ ratio = es8328->mclk_ratios[i];
+ snd_soc_update_bits(codec, ES8328_MASTERMODE,
+ ES8328_MASTERMODE_MCLKDIV2,
+ es8328->mclkdiv2 ? ES8328_MASTERMODE_MCLKDIV2 : 0);
- if (substream->stream == SNDRV_PCM_STREAM_PLAYBACK) {
- es8328->playback_fs = params_rate(params);
- es8328_set_deemph(codec);
+ switch (params_width(params)) {
+ case 16:
+ wl = 3;
+ break;
+ case 18:
+ wl = 2;
+ break;
+ case 20:
+ wl = 1;
+ break;
+ case 24:
+ wl = 0;
+ break;
+ case 32:
+ wl = 4;
+ break;
+ default:
+ return -EINVAL;
}
+ if (substream->stream == SNDRV_PCM_STREAM_PLAYBACK) {
+ snd_soc_update_bits(codec, ES8328_DACCONTROL1,
+ ES8328_DACCONTROL1_DACWL_MASK,
+ wl << ES8328_DACCONTROL1_DACWL_SHIFT);
+
+ es8328->playback_fs = params_rate(params);
+ es8328_set_deemph(codec);
+ } else
+ snd_soc_update_bits(codec, ES8328_ADCCONTROL4,
+ ES8328_ADCCONTROL4_ADCWL_MASK,
+ wl << ES8328_ADCCONTROL4_ADCWL_SHIFT);
+
return snd_soc_update_bits(codec, reg, ES8328_RATEMASK, ratio);
}
+static int es8328_set_sysclk(struct snd_soc_dai *codec_dai,
+ int clk_id, unsigned int freq, int dir)
+{
+ struct snd_soc_codec *codec = codec_dai->codec;
+ struct es8328_priv *es8328 = snd_soc_codec_get_drvdata(codec);
+ int mclkdiv2 = 0;
+
+ switch (freq) {
+ case 0:
+ es8328->sysclk_constraints = NULL;
+ es8328->mclk_ratios = NULL;
+ break;
+ case 22579200:
+ mclkdiv2 = 1;
+ /* fallthru */
+ case 11289600:
+ es8328->sysclk_constraints = &constraints_11289;
+ es8328->mclk_ratios = ratios_11289;
+ break;
+ case 24576000:
+ mclkdiv2 = 1;
+ /* fallthru */
+ case 12288000:
+ es8328->sysclk_constraints = &constraints_12288;
+ es8328->mclk_ratios = ratios_12288;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ es8328->mclkdiv2 = mclkdiv2;
+ return 0;
+}
+
static int es8328_set_dai_fmt(struct snd_soc_dai *codec_dai,
unsigned int fmt)
{
struct snd_soc_codec *codec = codec_dai->codec;
- struct es8328_priv *es8328 = snd_soc_codec_get_drvdata(codec);
- int clk_rate;
- u8 mode = ES8328_DACCONTROL1_DACWL_16;
+ u8 dac_mode = 0;
+ u8 adc_mode = 0;
/* set master/slave audio interface */
if ((fmt & SND_SOC_DAIFMT_MASTER_MASK) != SND_SOC_DAIFMT_CBM_CFM)
@@ -495,13 +596,16 @@
/* interface format */
switch (fmt & SND_SOC_DAIFMT_FORMAT_MASK) {
case SND_SOC_DAIFMT_I2S:
- mode |= ES8328_DACCONTROL1_DACFORMAT_I2S;
+ dac_mode |= ES8328_DACCONTROL1_DACFORMAT_I2S;
+ adc_mode |= ES8328_ADCCONTROL4_ADCFORMAT_I2S;
break;
case SND_SOC_DAIFMT_RIGHT_J:
- mode |= ES8328_DACCONTROL1_DACFORMAT_RJUST;
+ dac_mode |= ES8328_DACCONTROL1_DACFORMAT_RJUST;
+ adc_mode |= ES8328_ADCCONTROL4_ADCFORMAT_RJUST;
break;
case SND_SOC_DAIFMT_LEFT_J:
- mode |= ES8328_DACCONTROL1_DACFORMAT_LJUST;
+ dac_mode |= ES8328_DACCONTROL1_DACFORMAT_LJUST;
+ adc_mode |= ES8328_ADCCONTROL4_ADCFORMAT_LJUST;
break;
default:
return -EINVAL;
@@ -511,18 +615,14 @@
if ((fmt & SND_SOC_DAIFMT_INV_MASK) != SND_SOC_DAIFMT_NB_NF)
return -EINVAL;
- snd_soc_write(codec, ES8328_DACCONTROL1, mode);
- snd_soc_write(codec, ES8328_ADCCONTROL4, mode);
+ snd_soc_update_bits(codec, ES8328_DACCONTROL1,
+ ES8328_DACCONTROL1_DACFORMAT_MASK, dac_mode);
+ snd_soc_update_bits(codec, ES8328_ADCCONTROL4,
+ ES8328_ADCCONTROL4_ADCFORMAT_MASK, adc_mode);
/* Master serial port mode, with BCLK generated automatically */
- clk_rate = clk_get_rate(es8328->clk);
- if (clk_rate == ES8328_SYSCLK_RATE_1X)
- snd_soc_write(codec, ES8328_MASTERMODE,
- ES8328_MASTERMODE_MSC);
- else
- snd_soc_write(codec, ES8328_MASTERMODE,
- ES8328_MASTERMODE_MCLKDIV2 |
- ES8328_MASTERMODE_MSC);
+ snd_soc_update_bits(codec, ES8328_MASTERMODE,
+ ES8328_MASTERMODE_MSC, ES8328_MASTERMODE_MSC);
return 0;
}
@@ -579,8 +679,10 @@
}
static const struct snd_soc_dai_ops es8328_dai_ops = {
+ .startup = es8328_startup,
.hw_params = es8328_hw_params,
.digital_mute = es8328_mute,
+ .set_sysclk = es8328_set_sysclk,
.set_fmt = es8328_set_dai_fmt,
};
@@ -601,6 +703,7 @@
.formats = ES8328_FORMATS,
},
.ops = &es8328_dai_ops,
+ .symmetric_rates = 1,
};
static int es8328_suspend(struct snd_soc_codec *codec)
@@ -708,6 +811,7 @@
.val_bits = 8,
.max_register = ES8328_REG_MAX,
.cache_type = REGCACHE_RBTREE,
+ .use_single_rw = true,
};
EXPORT_SYMBOL_GPL(es8328_regmap_config);
diff --git a/sound/soc/codecs/es8328.h b/sound/soc/codecs/es8328.h
index 156c748..1a736e7 100644
--- a/sound/soc/codecs/es8328.h
+++ b/sound/soc/codecs/es8328.h
@@ -22,7 +22,7 @@
#define ES8328_CONTROL1_VMIDSEL_50k (1 << 0)
#define ES8328_CONTROL1_VMIDSEL_500k (2 << 0)
#define ES8328_CONTROL1_VMIDSEL_5k (3 << 0)
-#define ES8328_CONTROL1_VMIDSEL_MASK (7 << 0)
+#define ES8328_CONTROL1_VMIDSEL_MASK (3 << 0)
#define ES8328_CONTROL1_ENREF (1 << 2)
#define ES8328_CONTROL1_SEQEN (1 << 3)
#define ES8328_CONTROL1_SAMEFS (1 << 4)
@@ -84,7 +84,20 @@
#define ES8328_ADCCONTROL1 0x09
#define ES8328_ADCCONTROL2 0x0a
#define ES8328_ADCCONTROL3 0x0b
+
#define ES8328_ADCCONTROL4 0x0c
+#define ES8328_ADCCONTROL4_ADCFORMAT_MASK (3 << 0)
+#define ES8328_ADCCONTROL4_ADCFORMAT_I2S (0 << 0)
+#define ES8328_ADCCONTROL4_ADCFORMAT_LJUST (1 << 0)
+#define ES8328_ADCCONTROL4_ADCFORMAT_RJUST (2 << 0)
+#define ES8328_ADCCONTROL4_ADCFORMAT_PCM (3 << 0)
+#define ES8328_ADCCONTROL4_ADCWL_SHIFT 2
+#define ES8328_ADCCONTROL4_ADCWL_MASK (7 << 2)
+#define ES8328_ADCCONTROL4_ADCLRP_I2S_POL_NORMAL (0 << 5)
+#define ES8328_ADCCONTROL4_ADCLRP_I2S_POL_INV (1 << 5)
+#define ES8328_ADCCONTROL4_ADCLRP_PCM_MSB_CLK2 (0 << 5)
+#define ES8328_ADCCONTROL4_ADCLRP_PCM_MSB_CLK1 (1 << 5)
+
#define ES8328_ADCCONTROL5 0x0d
#define ES8328_ADCCONTROL5_RATEMASK (0x1f << 0)
@@ -109,15 +122,13 @@
#define ES8328_ADCCONTROL14 0x16
#define ES8328_DACCONTROL1 0x17
+#define ES8328_DACCONTROL1_DACFORMAT_MASK (3 << 1)
#define ES8328_DACCONTROL1_DACFORMAT_I2S (0 << 1)
#define ES8328_DACCONTROL1_DACFORMAT_LJUST (1 << 1)
#define ES8328_DACCONTROL1_DACFORMAT_RJUST (2 << 1)
#define ES8328_DACCONTROL1_DACFORMAT_PCM (3 << 1)
-#define ES8328_DACCONTROL1_DACWL_24 (0 << 3)
-#define ES8328_DACCONTROL1_DACWL_20 (1 << 3)
-#define ES8328_DACCONTROL1_DACWL_18 (2 << 3)
-#define ES8328_DACCONTROL1_DACWL_16 (3 << 3)
-#define ES8328_DACCONTROL1_DACWL_32 (4 << 3)
+#define ES8328_DACCONTROL1_DACWL_SHIFT 3
+#define ES8328_DACCONTROL1_DACWL_MASK (7 << 3)
#define ES8328_DACCONTROL1_DACLRP_I2S_POL_NORMAL (0 << 6)
#define ES8328_DACCONTROL1_DACLRP_I2S_POL_INV (1 << 6)
#define ES8328_DACCONTROL1_DACLRP_PCM_MSB_CLK2 (0 << 6)
diff --git a/sound/soc/codecs/hdac_hdmi.c b/sound/soc/codecs/hdac_hdmi.c
index aaa038f..181cd3b 100644
--- a/sound/soc/codecs/hdac_hdmi.c
+++ b/sound/soc/codecs/hdac_hdmi.c
@@ -29,6 +29,7 @@
#include <sound/hdaudio_ext.h>
#include <sound/hda_i915.h>
#include <sound/pcm_drm_eld.h>
+#include <sound/hda_chmap.h>
#include "../../hda/local.h"
#include "hdac_hdmi.h"
@@ -60,11 +61,17 @@
struct hdac_hdmi_cvt_params params;
};
+/* Currently only spk_alloc, more to be added */
+struct hdac_hdmi_parsed_eld {
+ u8 spk_alloc;
+};
+
struct hdac_hdmi_eld {
bool monitor_present;
bool eld_valid;
int eld_size;
char eld_buffer[ELD_MAX_SIZE];
+ struct hdac_hdmi_parsed_eld info;
};
struct hdac_hdmi_pin {
@@ -76,6 +83,10 @@
struct hdac_ext_device *edev;
int repoll_count;
struct delayed_work work;
+ struct mutex lock;
+ bool chmap_set;
+ unsigned char chmap[8]; /* ALSA API channel-map */
+ int channels; /* current number of channels */
};
struct hdac_hdmi_pcm {
@@ -100,8 +111,22 @@
int num_pin;
int num_cvt;
struct mutex pin_mutex;
+ struct hdac_chmap chmap;
};
+static struct hdac_hdmi_pcm *get_hdmi_pcm_from_id(struct hdac_hdmi_priv *hdmi,
+ int pcm_idx)
+{
+ struct hdac_hdmi_pcm *pcm;
+
+ list_for_each_entry(pcm, &hdmi->pcm_list, head) {
+ if (pcm->pcm_id == pcm_idx)
+ return pcm;
+ }
+
+ return NULL;
+}
+
static inline struct hdac_ext_device *to_hda_ext_device(struct device *dev)
{
struct hdac_device *hdac = dev_to_hdac_dev(dev);
@@ -278,26 +303,31 @@
int i;
const u8 *eld_buf;
u8 conn_type;
- int channels = 2;
+ int channels, ca;
list_for_each_entry(pin, &hdmi->pin_list, head) {
if (pin->nid == pin_nid)
break;
}
+ ca = snd_hdac_channel_allocation(&hdac->hdac, pin->eld.info.spk_alloc,
+ pin->channels, pin->chmap_set, true, pin->chmap);
+
+ channels = snd_hdac_get_active_channels(ca);
+ hdmi->chmap.ops.set_channel_count(&hdac->hdac, cvt_nid, channels);
+
+ snd_hdac_setup_channel_mapping(&hdmi->chmap, pin->nid, false, ca,
+ pin->channels, pin->chmap, pin->chmap_set);
+
eld_buf = pin->eld.eld_buffer;
conn_type = drm_eld_get_conn_type(eld_buf);
- /* setup channel count */
- snd_hdac_codec_write(&hdac->hdac, cvt_nid, 0,
- AC_VERB_SET_CVT_CHAN_COUNT, channels - 1);
-
switch (conn_type) {
case DRM_ELD_CONN_TYPE_HDMI:
hdmi_audio_infoframe_init(&frame);
- /* Default stereo for now */
frame.channels = channels;
+ frame.channel_allocation = ca;
ret = hdmi_audio_infoframe_pack(&frame, buffer, sizeof(buffer));
if (ret < 0)
@@ -311,7 +341,7 @@
dp_ai.len = 0x1b;
dp_ai.ver = 0x11 << 2;
dp_ai.CC02_CT47 = channels - 1;
- dp_ai.CA = 0;
+ dp_ai.CA = ca;
dip = (u8 *)&dp_ai;
break;
@@ -370,17 +400,23 @@
struct hdac_ext_device *hdac = snd_soc_dai_get_drvdata(dai);
struct hdac_hdmi_priv *hdmi = hdac->private_data;
struct hdac_hdmi_dai_pin_map *dai_map;
+ struct hdac_hdmi_pin *pin;
struct hdac_ext_dma_params *dd;
int ret;
dai_map = &hdmi->dai_map[dai->id];
+ pin = dai_map->pin;
dd = (struct hdac_ext_dma_params *)snd_soc_dai_get_dma_data(dai, substream);
dev_dbg(&hdac->hdac.dev, "stream tag from cpu dai %d format in cvt 0x%x\n",
dd->stream_tag, dd->format);
+ mutex_lock(&pin->lock);
+ pin->channels = substream->runtime->channels;
+
ret = hdac_hdmi_setup_audio_infoframe(hdac, dai_map->cvt->nid,
dai_map->pin->nid);
+ mutex_unlock(&pin->lock);
if (ret < 0)
return ret;
@@ -640,6 +676,12 @@
snd_hdac_codec_write(&hdac->hdac, dai_map->pin->nid, 0,
AC_VERB_SET_AMP_GAIN_MUTE, AMP_OUT_MUTE);
+ mutex_lock(&dai_map->pin->lock);
+ dai_map->pin->chmap_set = false;
+ memset(dai_map->pin->chmap, 0, sizeof(dai_map->pin->chmap));
+ dai_map->pin->channels = 0;
+ mutex_unlock(&dai_map->pin->lock);
+
dai_map->pin = NULL;
}
}
@@ -647,10 +689,19 @@
static int
hdac_hdmi_query_cvt_params(struct hdac_device *hdac, struct hdac_hdmi_cvt *cvt)
{
+ unsigned int chans;
+ struct hdac_ext_device *edev = to_ehdac_device(hdac);
+ struct hdac_hdmi_priv *hdmi = edev->private_data;
int err;
- /* Only stereo supported as of now */
- cvt->params.channels_min = cvt->params.channels_max = 2;
+ chans = get_wcaps(hdac, cvt->nid);
+ chans = get_wcaps_channels(chans);
+
+ cvt->params.channels_min = 2;
+
+ cvt->params.channels_max = chans;
+ if (chans > hdmi->chmap.channels_max)
+ hdmi->chmap.channels_max = chans;
err = snd_hdac_query_supported_pcm(hdac, cvt->nid,
&cvt->params.rates,
@@ -1008,6 +1059,12 @@
return hdac_hdmi_query_cvt_params(&edev->hdac, cvt);
}
+static void hdac_hdmi_parse_eld(struct hdac_ext_device *edev,
+ struct hdac_hdmi_pin *pin)
+{
+ pin->eld.info.spk_alloc = pin->eld.eld_buffer[DRM_ELD_SPEAKER];
+}
+
static void hdac_hdmi_present_sense(struct hdac_hdmi_pin *pin, int repoll)
{
struct hdac_ext_device *edev = pin->edev;
@@ -1065,6 +1122,7 @@
snd_jack_report(pcm->jack, SND_JACK_AVOUT);
}
+ hdac_hdmi_parse_eld(edev, pin);
print_hex_dump_bytes("ELD: ", DUMP_PREFIX_OFFSET,
pin->eld.eld_buffer, pin->eld.eld_size);
@@ -1123,6 +1181,7 @@
hdmi->num_pin++;
pin->edev = edev;
+ mutex_init(&pin->lock);
INIT_DELAYED_WORK(&pin->work, hdac_hdmi_repoll_eld);
return 0;
@@ -1342,6 +1401,19 @@
.pin_eld_notify = hdac_hdmi_eld_notify_cb,
};
+static struct snd_pcm *hdac_hdmi_get_pcm_from_id(struct snd_soc_card *card,
+ int device)
+{
+ struct snd_soc_pcm_runtime *rtd;
+
+ list_for_each_entry(rtd, &card->rtd_list, list) {
+ if (rtd->pcm && (rtd->pcm->device == device))
+ return rtd->pcm;
+ }
+
+ return NULL;
+}
+
int hdac_hdmi_jack_init(struct snd_soc_dai *dai, int device)
{
char jack_name[NAME_SIZE];
@@ -1351,6 +1423,8 @@
snd_soc_component_get_dapm(&codec->component);
struct hdac_hdmi_priv *hdmi = edev->private_data;
struct hdac_hdmi_pcm *pcm;
+ struct snd_pcm *snd_pcm;
+ int err;
/*
* this is a new PCM device, create new pcm and
@@ -1362,6 +1436,18 @@
pcm->pcm_id = device;
pcm->cvt = hdmi->dai_map[dai->id].cvt;
+ snd_pcm = hdac_hdmi_get_pcm_from_id(dai->component->card, device);
+ if (snd_pcm) {
+ err = snd_hdac_add_chmap_ctls(snd_pcm, device, &hdmi->chmap);
+ if (err < 0) {
+ dev_err(&edev->hdac.dev,
+ "chmap control add failed with err: %d for pcm: %d\n",
+ err, device);
+ kfree(pcm);
+ return err;
+ }
+ }
+
list_add_tail(&pcm->head, &hdmi->pcm_list);
sprintf(jack_name, "HDMI/DP, pcm=%d Jack", device);
@@ -1378,10 +1464,18 @@
struct snd_soc_dapm_context *dapm =
snd_soc_component_get_dapm(&codec->component);
struct hdac_hdmi_pin *pin;
+ struct hdac_ext_link *hlink = NULL;
int ret;
edev->scodec = codec;
+ /*
+ * hold the ref while we probe, also no need to drop the ref on
+ * exit, we call pm_runtime_suspend() so that will do for us
+ */
+ hlink = snd_hdac_ext_bus_get_link(edev->ebus, dev_name(&edev->hdac.dev));
+ snd_hdac_ext_bus_link_get(edev->ebus, hlink);
+
ret = create_fill_widget_route_map(dapm);
if (ret < 0)
return ret;
@@ -1475,19 +1569,83 @@
.idle_bias_off = true,
};
+static void hdac_hdmi_get_chmap(struct hdac_device *hdac, int pcm_idx,
+ unsigned char *chmap)
+{
+ struct hdac_ext_device *edev = to_ehdac_device(hdac);
+ struct hdac_hdmi_priv *hdmi = edev->private_data;
+ struct hdac_hdmi_pcm *pcm = get_hdmi_pcm_from_id(hdmi, pcm_idx);
+ struct hdac_hdmi_pin *pin = pcm->pin;
+
+ /* chmap is already set to 0 in caller */
+ if (!pin)
+ return;
+
+ memcpy(chmap, pin->chmap, ARRAY_SIZE(pin->chmap));
+}
+
+static void hdac_hdmi_set_chmap(struct hdac_device *hdac, int pcm_idx,
+ unsigned char *chmap, int prepared)
+{
+ struct hdac_ext_device *edev = to_ehdac_device(hdac);
+ struct hdac_hdmi_priv *hdmi = edev->private_data;
+ struct hdac_hdmi_pcm *pcm = get_hdmi_pcm_from_id(hdmi, pcm_idx);
+ struct hdac_hdmi_pin *pin = pcm->pin;
+
+ mutex_lock(&pin->lock);
+ pin->chmap_set = true;
+ memcpy(pin->chmap, chmap, ARRAY_SIZE(pin->chmap));
+ if (prepared)
+ hdac_hdmi_setup_audio_infoframe(edev, pcm->cvt->nid, pin->nid);
+ mutex_unlock(&pin->lock);
+}
+
+static bool is_hdac_hdmi_pcm_attached(struct hdac_device *hdac, int pcm_idx)
+{
+ struct hdac_ext_device *edev = to_ehdac_device(hdac);
+ struct hdac_hdmi_priv *hdmi = edev->private_data;
+ struct hdac_hdmi_pcm *pcm = get_hdmi_pcm_from_id(hdmi, pcm_idx);
+ struct hdac_hdmi_pin *pin = pcm->pin;
+
+ return pin ? true:false;
+}
+
+static int hdac_hdmi_get_spk_alloc(struct hdac_device *hdac, int pcm_idx)
+{
+ struct hdac_ext_device *edev = to_ehdac_device(hdac);
+ struct hdac_hdmi_priv *hdmi = edev->private_data;
+ struct hdac_hdmi_pcm *pcm = get_hdmi_pcm_from_id(hdmi, pcm_idx);
+ struct hdac_hdmi_pin *pin = pcm->pin;
+
+ if (!pin || !pin->eld.eld_valid)
+ return 0;
+
+ return pin->eld.info.spk_alloc;
+}
+
static int hdac_hdmi_dev_probe(struct hdac_ext_device *edev)
{
struct hdac_device *codec = &edev->hdac;
struct hdac_hdmi_priv *hdmi_priv;
struct snd_soc_dai_driver *hdmi_dais = NULL;
+ struct hdac_ext_link *hlink = NULL;
int num_dais = 0;
int ret = 0;
+ /* hold the ref while we probe */
+ hlink = snd_hdac_ext_bus_get_link(edev->ebus, dev_name(&edev->hdac.dev));
+ snd_hdac_ext_bus_link_get(edev->ebus, hlink);
+
hdmi_priv = devm_kzalloc(&codec->dev, sizeof(*hdmi_priv), GFP_KERNEL);
if (hdmi_priv == NULL)
return -ENOMEM;
edev->private_data = hdmi_priv;
+ snd_hdac_register_chmap_ops(codec, &hdmi_priv->chmap);
+ hdmi_priv->chmap.ops.get_chmap = hdac_hdmi_get_chmap;
+ hdmi_priv->chmap.ops.set_chmap = hdac_hdmi_set_chmap;
+ hdmi_priv->chmap.ops.is_pcm_attached = is_hdac_hdmi_pcm_attached;
+ hdmi_priv->chmap.ops.get_spk_alloc = hdac_hdmi_get_spk_alloc;
dev_set_drvdata(&codec->dev, edev);
@@ -1516,8 +1674,12 @@
}
/* ASoC specific initialization */
- return snd_soc_register_codec(&codec->dev, &hdmi_hda_codec,
- hdmi_dais, num_dais);
+ ret = snd_soc_register_codec(&codec->dev, &hdmi_hda_codec,
+ hdmi_dais, num_dais);
+
+ snd_hdac_ext_bus_link_put(edev->ebus, hlink);
+
+ return ret;
}
static int hdac_hdmi_dev_remove(struct hdac_ext_device *edev)
@@ -1556,6 +1718,8 @@
struct hdac_ext_device *edev = to_hda_ext_device(dev);
struct hdac_device *hdac = &edev->hdac;
struct hdac_bus *bus = hdac->bus;
+ struct hdac_ext_bus *ebus = hbus_to_ebus(bus);
+ struct hdac_ext_link *hlink = NULL;
int err;
dev_dbg(dev, "Enter: %s\n", __func__);
@@ -1579,6 +1743,9 @@
return err;
}
+ hlink = snd_hdac_ext_bus_get_link(ebus, dev_name(dev));
+ snd_hdac_ext_bus_link_put(ebus, hlink);
+
return 0;
}
@@ -1587,6 +1754,8 @@
struct hdac_ext_device *edev = to_hda_ext_device(dev);
struct hdac_device *hdac = &edev->hdac;
struct hdac_bus *bus = hdac->bus;
+ struct hdac_ext_bus *ebus = hbus_to_ebus(bus);
+ struct hdac_ext_link *hlink = NULL;
int err;
dev_dbg(dev, "Enter: %s\n", __func__);
@@ -1595,6 +1764,9 @@
if (!bus)
return 0;
+ hlink = snd_hdac_ext_bus_get_link(ebus, dev_name(dev));
+ snd_hdac_ext_bus_link_get(ebus, hlink);
+
err = snd_hdac_display_power(bus, true);
if (err < 0) {
dev_err(bus->dev, "Cannot turn on display power on i915\n");
diff --git a/sound/soc/codecs/hdmi-codec.c b/sound/soc/codecs/hdmi-codec.c
new file mode 100644
index 0000000..8e36e88
--- /dev/null
+++ b/sound/soc/codecs/hdmi-codec.c
@@ -0,0 +1,432 @@
+/*
+ * ALSA SoC codec for HDMI encoder drivers
+ * Copyright (C) 2015 Texas Instruments Incorporated - http://www.ti.com/
+ * Author: Jyri Sarha <jsarha@ti.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ */
+#include <linux/module.h>
+#include <linux/string.h>
+#include <sound/core.h>
+#include <sound/pcm.h>
+#include <sound/pcm_params.h>
+#include <sound/soc.h>
+#include <sound/pcm_drm_eld.h>
+#include <sound/hdmi-codec.h>
+#include <sound/pcm_iec958.h>
+
+#include <drm/drm_crtc.h> /* This is only to get MAX_ELD_BYTES */
+
+struct hdmi_codec_priv {
+ struct hdmi_codec_pdata hcd;
+ struct snd_soc_dai_driver *daidrv;
+ struct hdmi_codec_daifmt daifmt[2];
+ struct mutex current_stream_lock;
+ struct snd_pcm_substream *current_stream;
+ struct snd_pcm_hw_constraint_list ratec;
+ uint8_t eld[MAX_ELD_BYTES];
+};
+
+static const struct snd_soc_dapm_widget hdmi_widgets[] = {
+ SND_SOC_DAPM_OUTPUT("TX"),
+};
+
+static const struct snd_soc_dapm_route hdmi_routes[] = {
+ { "TX", NULL, "Playback" },
+};
+
+enum {
+ DAI_ID_I2S = 0,
+ DAI_ID_SPDIF,
+};
+
+static int hdmi_eld_ctl_info(struct snd_kcontrol *kcontrol,
+ struct snd_ctl_elem_info *uinfo)
+{
+ struct snd_soc_component *component = snd_kcontrol_chip(kcontrol);
+ struct hdmi_codec_priv *hcp = snd_soc_component_get_drvdata(component);
+
+ uinfo->type = SNDRV_CTL_ELEM_TYPE_BYTES;
+ uinfo->count = sizeof(hcp->eld);
+
+ return 0;
+}
+
+static int hdmi_eld_ctl_get(struct snd_kcontrol *kcontrol,
+ struct snd_ctl_elem_value *ucontrol)
+{
+ struct snd_soc_component *component = snd_kcontrol_chip(kcontrol);
+ struct hdmi_codec_priv *hcp = snd_soc_component_get_drvdata(component);
+
+ memcpy(ucontrol->value.bytes.data, hcp->eld, sizeof(hcp->eld));
+
+ return 0;
+}
+
+static const struct snd_kcontrol_new hdmi_controls[] = {
+ {
+ .access = SNDRV_CTL_ELEM_ACCESS_READ |
+ SNDRV_CTL_ELEM_ACCESS_VOLATILE,
+ .iface = SNDRV_CTL_ELEM_IFACE_PCM,
+ .name = "ELD",
+ .info = hdmi_eld_ctl_info,
+ .get = hdmi_eld_ctl_get,
+ },
+};
+
+static int hdmi_codec_new_stream(struct snd_pcm_substream *substream,
+ struct snd_soc_dai *dai)
+{
+ struct hdmi_codec_priv *hcp = snd_soc_dai_get_drvdata(dai);
+ int ret = 0;
+
+ mutex_lock(&hcp->current_stream_lock);
+ if (!hcp->current_stream) {
+ hcp->current_stream = substream;
+ } else if (hcp->current_stream != substream) {
+ dev_err(dai->dev, "Only one simultaneous stream supported!\n");
+ ret = -EINVAL;
+ }
+ mutex_unlock(&hcp->current_stream_lock);
+
+ return ret;
+}
+
+static int hdmi_codec_startup(struct snd_pcm_substream *substream,
+ struct snd_soc_dai *dai)
+{
+ struct hdmi_codec_priv *hcp = snd_soc_dai_get_drvdata(dai);
+ int ret = 0;
+
+ dev_dbg(dai->dev, "%s()\n", __func__);
+
+ ret = hdmi_codec_new_stream(substream, dai);
+ if (ret)
+ return ret;
+
+ if (hcp->hcd.ops->audio_startup) {
+ ret = hcp->hcd.ops->audio_startup(dai->dev->parent);
+ if (ret) {
+ mutex_lock(&hcp->current_stream_lock);
+ hcp->current_stream = NULL;
+ mutex_unlock(&hcp->current_stream_lock);
+ return ret;
+ }
+ }
+
+ if (hcp->hcd.ops->get_eld) {
+ ret = hcp->hcd.ops->get_eld(dai->dev->parent, hcp->eld,
+ sizeof(hcp->eld));
+
+ if (!ret) {
+ ret = snd_pcm_hw_constraint_eld(substream->runtime,
+ hcp->eld);
+ if (ret)
+ return ret;
+ }
+ }
+ return 0;
+}
+
+static void hdmi_codec_shutdown(struct snd_pcm_substream *substream,
+ struct snd_soc_dai *dai)
+{
+ struct hdmi_codec_priv *hcp = snd_soc_dai_get_drvdata(dai);
+
+ dev_dbg(dai->dev, "%s()\n", __func__);
+
+ WARN_ON(hcp->current_stream != substream);
+
+ hcp->hcd.ops->audio_shutdown(dai->dev->parent);
+
+ mutex_lock(&hcp->current_stream_lock);
+ hcp->current_stream = NULL;
+ mutex_unlock(&hcp->current_stream_lock);
+}
+
+static int hdmi_codec_hw_params(struct snd_pcm_substream *substream,
+ struct snd_pcm_hw_params *params,
+ struct snd_soc_dai *dai)
+{
+ struct hdmi_codec_priv *hcp = snd_soc_dai_get_drvdata(dai);
+ struct hdmi_codec_params hp = {
+ .iec = {
+ .status = { 0 },
+ .subcode = { 0 },
+ .pad = 0,
+ .dig_subframe = { 0 },
+ }
+ };
+ int ret;
+
+ dev_dbg(dai->dev, "%s() width %d rate %d channels %d\n", __func__,
+ params_width(params), params_rate(params),
+ params_channels(params));
+
+ if (params_width(params) > 24)
+ params->msbits = 24;
+
+ ret = snd_pcm_create_iec958_consumer_hw_params(params, hp.iec.status,
+ sizeof(hp.iec.status));
+ if (ret < 0) {
+ dev_err(dai->dev, "Creating IEC958 channel status failed %d\n",
+ ret);
+ return ret;
+ }
+
+ ret = hdmi_codec_new_stream(substream, dai);
+ if (ret)
+ return ret;
+
+ hdmi_audio_infoframe_init(&hp.cea);
+ hp.cea.channels = params_channels(params);
+ hp.cea.coding_type = HDMI_AUDIO_CODING_TYPE_STREAM;
+ hp.cea.sample_size = HDMI_AUDIO_SAMPLE_SIZE_STREAM;
+ hp.cea.sample_frequency = HDMI_AUDIO_SAMPLE_FREQUENCY_STREAM;
+
+ hp.sample_width = params_width(params);
+ hp.sample_rate = params_rate(params);
+ hp.channels = params_channels(params);
+
+ return hcp->hcd.ops->hw_params(dai->dev->parent, &hcp->daifmt[dai->id],
+ &hp);
+}
+
+static int hdmi_codec_set_fmt(struct snd_soc_dai *dai,
+ unsigned int fmt)
+{
+ struct hdmi_codec_priv *hcp = snd_soc_dai_get_drvdata(dai);
+ struct hdmi_codec_daifmt cf = { 0 };
+ int ret = 0;
+
+ dev_dbg(dai->dev, "%s()\n", __func__);
+
+ if (dai->id == DAI_ID_SPDIF) {
+ cf.fmt = HDMI_SPDIF;
+ } else {
+ switch (fmt & SND_SOC_DAIFMT_MASTER_MASK) {
+ case SND_SOC_DAIFMT_CBM_CFM:
+ cf.bit_clk_master = 1;
+ cf.frame_clk_master = 1;
+ break;
+ case SND_SOC_DAIFMT_CBS_CFM:
+ cf.frame_clk_master = 1;
+ break;
+ case SND_SOC_DAIFMT_CBM_CFS:
+ cf.bit_clk_master = 1;
+ break;
+ case SND_SOC_DAIFMT_CBS_CFS:
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ switch (fmt & SND_SOC_DAIFMT_INV_MASK) {
+ case SND_SOC_DAIFMT_NB_NF:
+ break;
+ case SND_SOC_DAIFMT_NB_IF:
+ cf.frame_clk_inv = 1;
+ break;
+ case SND_SOC_DAIFMT_IB_NF:
+ cf.bit_clk_inv = 1;
+ break;
+ case SND_SOC_DAIFMT_IB_IF:
+ cf.frame_clk_inv = 1;
+ cf.bit_clk_inv = 1;
+ break;
+ }
+
+ switch (fmt & SND_SOC_DAIFMT_FORMAT_MASK) {
+ case SND_SOC_DAIFMT_I2S:
+ cf.fmt = HDMI_I2S;
+ break;
+ case SND_SOC_DAIFMT_DSP_A:
+ cf.fmt = HDMI_DSP_A;
+ break;
+ case SND_SOC_DAIFMT_DSP_B:
+ cf.fmt = HDMI_DSP_B;
+ break;
+ case SND_SOC_DAIFMT_RIGHT_J:
+ cf.fmt = HDMI_RIGHT_J;
+ break;
+ case SND_SOC_DAIFMT_LEFT_J:
+ cf.fmt = HDMI_LEFT_J;
+ break;
+ case SND_SOC_DAIFMT_AC97:
+ cf.fmt = HDMI_AC97;
+ break;
+ default:
+ dev_err(dai->dev, "Invalid DAI interface format\n");
+ return -EINVAL;
+ }
+ }
+
+ hcp->daifmt[dai->id] = cf;
+
+ return ret;
+}
+
+static int hdmi_codec_digital_mute(struct snd_soc_dai *dai, int mute)
+{
+ struct hdmi_codec_priv *hcp = snd_soc_dai_get_drvdata(dai);
+
+ dev_dbg(dai->dev, "%s()\n", __func__);
+
+ if (hcp->hcd.ops->digital_mute)
+ return hcp->hcd.ops->digital_mute(dai->dev->parent, mute);
+
+ return 0;
+}
+
+static const struct snd_soc_dai_ops hdmi_dai_ops = {
+ .startup = hdmi_codec_startup,
+ .shutdown = hdmi_codec_shutdown,
+ .hw_params = hdmi_codec_hw_params,
+ .set_fmt = hdmi_codec_set_fmt,
+ .digital_mute = hdmi_codec_digital_mute,
+};
+
+
+#define HDMI_RATES (SNDRV_PCM_RATE_32000 | SNDRV_PCM_RATE_44100 |\
+ SNDRV_PCM_RATE_48000 | SNDRV_PCM_RATE_88200 |\
+ SNDRV_PCM_RATE_96000 | SNDRV_PCM_RATE_176400 |\
+ SNDRV_PCM_RATE_192000)
+
+#define SPDIF_FORMATS (SNDRV_PCM_FMTBIT_S16_LE | SNDRV_PCM_FMTBIT_S16_BE |\
+ SNDRV_PCM_FMTBIT_S20_3LE | SNDRV_PCM_FMTBIT_S20_3BE |\
+ SNDRV_PCM_FMTBIT_S24_3LE | SNDRV_PCM_FMTBIT_S24_3BE |\
+ SNDRV_PCM_FMTBIT_S24_LE | SNDRV_PCM_FMTBIT_S24_BE)
+
+/*
+ * This list is only for formats allowed on the I2S bus. So there is
+ * some formats listed that are not supported by HDMI interface. For
+ * instance allowing the 32-bit formats enables 24-precision with CPU
+ * DAIs that do not support 24-bit formats. If the extra formats cause
+ * problems, we should add the video side driver an option to disable
+ * them.
+ */
+#define I2S_FORMATS (SNDRV_PCM_FMTBIT_S16_LE | SNDRV_PCM_FMTBIT_S16_BE |\
+ SNDRV_PCM_FMTBIT_S20_3LE | SNDRV_PCM_FMTBIT_S20_3BE |\
+ SNDRV_PCM_FMTBIT_S24_3LE | SNDRV_PCM_FMTBIT_S24_3BE |\
+ SNDRV_PCM_FMTBIT_S24_LE | SNDRV_PCM_FMTBIT_S24_BE |\
+ SNDRV_PCM_FMTBIT_S32_LE | SNDRV_PCM_FMTBIT_S32_BE)
+
+static struct snd_soc_dai_driver hdmi_i2s_dai = {
+ .name = "i2s-hifi",
+ .id = DAI_ID_I2S,
+ .playback = {
+ .stream_name = "Playback",
+ .channels_min = 2,
+ .channels_max = 8,
+ .rates = HDMI_RATES,
+ .formats = I2S_FORMATS,
+ .sig_bits = 24,
+ },
+ .ops = &hdmi_dai_ops,
+};
+
+static const struct snd_soc_dai_driver hdmi_spdif_dai = {
+ .name = "spdif-hifi",
+ .id = DAI_ID_SPDIF,
+ .playback = {
+ .stream_name = "Playback",
+ .channels_min = 2,
+ .channels_max = 2,
+ .rates = HDMI_RATES,
+ .formats = SPDIF_FORMATS,
+ },
+ .ops = &hdmi_dai_ops,
+};
+
+static struct snd_soc_codec_driver hdmi_codec = {
+ .controls = hdmi_controls,
+ .num_controls = ARRAY_SIZE(hdmi_controls),
+ .dapm_widgets = hdmi_widgets,
+ .num_dapm_widgets = ARRAY_SIZE(hdmi_widgets),
+ .dapm_routes = hdmi_routes,
+ .num_dapm_routes = ARRAY_SIZE(hdmi_routes),
+};
+
+static int hdmi_codec_probe(struct platform_device *pdev)
+{
+ struct hdmi_codec_pdata *hcd = pdev->dev.platform_data;
+ struct device *dev = &pdev->dev;
+ struct hdmi_codec_priv *hcp;
+ int dai_count, i = 0;
+ int ret;
+
+ dev_dbg(dev, "%s()\n", __func__);
+
+ if (!hcd) {
+ dev_err(dev, "%s: No plalform data\n", __func__);
+ return -EINVAL;
+ }
+
+ dai_count = hcd->i2s + hcd->spdif;
+ if (dai_count < 1 || !hcd->ops || !hcd->ops->hw_params ||
+ !hcd->ops->audio_shutdown) {
+ dev_err(dev, "%s: Invalid parameters\n", __func__);
+ return -EINVAL;
+ }
+
+ hcp = devm_kzalloc(dev, sizeof(*hcp), GFP_KERNEL);
+ if (!hcp)
+ return -ENOMEM;
+
+ hcp->hcd = *hcd;
+ mutex_init(&hcp->current_stream_lock);
+
+ hcp->daidrv = devm_kzalloc(dev, dai_count * sizeof(*hcp->daidrv),
+ GFP_KERNEL);
+ if (!hcp->daidrv)
+ return -ENOMEM;
+
+ if (hcd->i2s) {
+ hcp->daidrv[i] = hdmi_i2s_dai;
+ hcp->daidrv[i].playback.channels_max =
+ hcd->max_i2s_channels;
+ i++;
+ }
+
+ if (hcd->spdif)
+ hcp->daidrv[i] = hdmi_spdif_dai;
+
+ ret = snd_soc_register_codec(dev, &hdmi_codec, hcp->daidrv,
+ dai_count);
+ if (ret) {
+ dev_err(dev, "%s: snd_soc_register_codec() failed (%d)\n",
+ __func__, ret);
+ return ret;
+ }
+
+ dev_set_drvdata(dev, hcp);
+ return 0;
+}
+
+static int hdmi_codec_remove(struct platform_device *pdev)
+{
+ snd_soc_unregister_codec(&pdev->dev);
+ return 0;
+}
+
+static struct platform_driver hdmi_codec_driver = {
+ .driver = {
+ .name = HDMI_CODEC_DRV_NAME,
+ },
+ .probe = hdmi_codec_probe,
+ .remove = hdmi_codec_remove,
+};
+
+module_platform_driver(hdmi_codec_driver);
+
+MODULE_AUTHOR("Jyri Sarha <jsarha@ti.com>");
+MODULE_DESCRIPTION("HDMI Audio Codec Driver");
+MODULE_LICENSE("GPL");
+MODULE_ALIAS("platform:" HDMI_CODEC_DRV_NAME);
diff --git a/sound/soc/codecs/pcm5102a.c b/sound/soc/codecs/pcm5102a.c
new file mode 100644
index 0000000..ed51567
--- /dev/null
+++ b/sound/soc/codecs/pcm5102a.c
@@ -0,0 +1,69 @@
+/*
+ * Driver for the PCM5102A codec
+ *
+ * Author: Florian Meier <florian.meier@koalo.de>
+ * Copyright 2013
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ */
+
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+
+#include <sound/soc.h>
+
+static struct snd_soc_dai_driver pcm5102a_dai = {
+ .name = "pcm5102a-hifi",
+ .playback = {
+ .channels_min = 2,
+ .channels_max = 2,
+ .rates = SNDRV_PCM_RATE_8000_192000,
+ .formats = SNDRV_PCM_FMTBIT_S16_LE |
+ SNDRV_PCM_FMTBIT_S24_LE |
+ SNDRV_PCM_FMTBIT_S32_LE
+ },
+};
+
+static struct snd_soc_codec_driver soc_codec_dev_pcm5102a;
+
+static int pcm5102a_probe(struct platform_device *pdev)
+{
+ return snd_soc_register_codec(&pdev->dev, &soc_codec_dev_pcm5102a,
+ &pcm5102a_dai, 1);
+}
+
+static int pcm5102a_remove(struct platform_device *pdev)
+{
+ snd_soc_unregister_codec(&pdev->dev);
+ return 0;
+}
+
+static const struct of_device_id pcm5102a_of_match[] = {
+ { .compatible = "ti,pcm5102a", },
+ { }
+};
+MODULE_DEVICE_TABLE(of, pcm5102a_of_match);
+
+static struct platform_driver pcm5102a_codec_driver = {
+ .probe = pcm5102a_probe,
+ .remove = pcm5102a_remove,
+ .driver = {
+ .name = "pcm5102a-codec",
+ .owner = THIS_MODULE,
+ .of_match_table = pcm5102a_of_match,
+ },
+};
+
+module_platform_driver(pcm5102a_codec_driver);
+
+MODULE_DESCRIPTION("ASoC PCM5102A codec driver");
+MODULE_AUTHOR("Florian Meier <florian.meier@koalo.de>");
+MODULE_LICENSE("GPL v2");
diff --git a/sound/soc/codecs/rt298.c b/sound/soc/codecs/rt298.c
index f0e6c06..a1aaffc 100644
--- a/sound/soc/codecs/rt298.c
+++ b/sound/soc/codecs/rt298.c
@@ -17,6 +17,7 @@
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/spi/spi.h>
+#include <linux/dmi.h>
#include <linux/acpi.h>
#include <sound/core.h>
#include <sound/pcm.h>
@@ -1132,6 +1133,17 @@
};
MODULE_DEVICE_TABLE(acpi, rt298_acpi_match);
+static const struct dmi_system_id force_combo_jack_table[] = {
+ {
+ .ident = "Intel Broxton P",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "Intel Corp"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "Broxton P")
+ }
+ },
+ { }
+};
+
static int rt298_i2c_probe(struct i2c_client *i2c,
const struct i2c_device_id *id)
{
@@ -1184,11 +1196,16 @@
/* enable jack combo mode on supported devices */
acpiid = acpi_match_device(dev->driver->acpi_match_table, dev);
- if (acpiid) {
+ if (acpiid && acpiid->driver_data) {
rt298->pdata = *(struct rt298_platform_data *)
acpiid->driver_data;
}
+ if (dmi_check_system(force_combo_jack_table)) {
+ rt298->pdata.cbj_en = true;
+ rt298->pdata.gpio2_en = false;
+ }
+
/* VREF Charging */
regmap_update_bits(rt298->regmap, 0x04, 0x80, 0x80);
regmap_update_bits(rt298->regmap, 0x1b, 0x860, 0x860);
diff --git a/sound/soc/codecs/rt5645.c b/sound/soc/codecs/rt5645.c
index 7af5e73..3c6594d 100644
--- a/sound/soc/codecs/rt5645.c
+++ b/sound/soc/codecs/rt5645.c
@@ -3286,10 +3286,8 @@
if (btn_type == 0)/* button release */
report = rt5645->jack_type;
else {
- if (rt5645->pdata.jd_invert) {
- mod_timer(&rt5645->btn_check_timer,
- msecs_to_jiffies(100));
- }
+ mod_timer(&rt5645->btn_check_timer,
+ msecs_to_jiffies(100));
}
break;
@@ -3557,6 +3555,12 @@
DMI_MATCH(DMI_SYS_VENDOR, "GOOGLE"),
},
},
+ {
+ .ident = "Google Setzer",
+ .matches = {
+ DMI_MATCH(DMI_PRODUCT_NAME, "Setzer"),
+ },
+ },
{ }
};
@@ -3810,9 +3814,9 @@
if (rt5645->pdata.jd_invert) {
regmap_update_bits(rt5645->regmap, RT5645_IRQ_CTRL2,
RT5645_JD_1_1_MASK, RT5645_JD_1_1_INV);
- setup_timer(&rt5645->btn_check_timer,
- rt5645_btn_check_callback, (unsigned long)rt5645);
}
+ setup_timer(&rt5645->btn_check_timer,
+ rt5645_btn_check_callback, (unsigned long)rt5645);
INIT_DELAYED_WORK(&rt5645->jack_detect_work, rt5645_jack_detect_work);
INIT_DELAYED_WORK(&rt5645->rcclock_work, rt5645_rcclock_work);
diff --git a/sound/soc/codecs/wm5102.c b/sound/soc/codecs/wm5102.c
index 1bae17e..da60e3f 100644
--- a/sound/soc/codecs/wm5102.c
+++ b/sound/soc/codecs/wm5102.c
@@ -2098,10 +2098,14 @@
static int wm5102_remove(struct platform_device *pdev)
{
+ struct wm5102_priv *wm5102 = platform_get_drvdata(pdev);
+
snd_soc_unregister_platform(&pdev->dev);
snd_soc_unregister_codec(&pdev->dev);
pm_runtime_disable(&pdev->dev);
+ wm_adsp2_remove(&wm5102->core.adsp[0]);
+
return 0;
}
diff --git a/sound/soc/codecs/wm5110.c b/sound/soc/codecs/wm5110.c
index 2728ac5..b5820e4 100644
--- a/sound/soc/codecs/wm5110.c
+++ b/sound/soc/codecs/wm5110.c
@@ -2437,10 +2437,16 @@
static int wm5110_remove(struct platform_device *pdev)
{
+ struct wm5110_priv *wm5110 = platform_get_drvdata(pdev);
+ int i;
+
snd_soc_unregister_platform(&pdev->dev);
snd_soc_unregister_codec(&pdev->dev);
pm_runtime_disable(&pdev->dev);
+ for (i = 0; i < WM5110_NUM_ADSP; i++)
+ wm_adsp2_remove(&wm5110->core.adsp[i]);
+
return 0;
}
diff --git a/sound/soc/codecs/wm_adsp.c b/sound/soc/codecs/wm_adsp.c
index d3b1cb1..a07bd7c 100644
--- a/sound/soc/codecs/wm_adsp.c
+++ b/sound/soc/codecs/wm_adsp.c
@@ -160,6 +160,8 @@
#define ADSP2_RAM_RDY_SHIFT 0
#define ADSP2_RAM_RDY_WIDTH 1
+#define ADSP_MAX_STD_CTRL_SIZE 512
+
struct wm_adsp_buf {
struct list_head list;
void *buf;
@@ -271,8 +273,11 @@
__be32 words_written[2]; /* total words written (64 bit) */
};
+struct wm_adsp_compr;
+
struct wm_adsp_compr_buf {
struct wm_adsp *dsp;
+ struct wm_adsp_compr *compr;
struct wm_adsp_buffer_region *regions;
u32 host_buf_ptr;
@@ -435,6 +440,7 @@
size_t len;
unsigned int set:1;
struct snd_kcontrol *kcontrol;
+ struct soc_bytes_ext bytes_ext;
unsigned int flags;
};
@@ -711,10 +717,17 @@
be16_to_cpu(scratch[3]));
}
+static inline struct wm_coeff_ctl *bytes_ext_to_ctl(struct soc_bytes_ext *ext)
+{
+ return container_of(ext, struct wm_coeff_ctl, bytes_ext);
+}
+
static int wm_coeff_info(struct snd_kcontrol *kctl,
struct snd_ctl_elem_info *uinfo)
{
- struct wm_coeff_ctl *ctl = (struct wm_coeff_ctl *)kctl->private_value;
+ struct soc_bytes_ext *bytes_ext =
+ (struct soc_bytes_ext *)kctl->private_value;
+ struct wm_coeff_ctl *ctl = bytes_ext_to_ctl(bytes_ext);
uinfo->type = SNDRV_CTL_ELEM_TYPE_BYTES;
uinfo->count = ctl->len;
@@ -763,7 +776,9 @@
static int wm_coeff_put(struct snd_kcontrol *kctl,
struct snd_ctl_elem_value *ucontrol)
{
- struct wm_coeff_ctl *ctl = (struct wm_coeff_ctl *)kctl->private_value;
+ struct soc_bytes_ext *bytes_ext =
+ (struct soc_bytes_ext *)kctl->private_value;
+ struct wm_coeff_ctl *ctl = bytes_ext_to_ctl(bytes_ext);
char *p = ucontrol->value.bytes.data;
int ret = 0;
@@ -780,6 +795,29 @@
return ret;
}
+static int wm_coeff_tlv_put(struct snd_kcontrol *kctl,
+ const unsigned int __user *bytes, unsigned int size)
+{
+ struct soc_bytes_ext *bytes_ext =
+ (struct soc_bytes_ext *)kctl->private_value;
+ struct wm_coeff_ctl *ctl = bytes_ext_to_ctl(bytes_ext);
+ int ret = 0;
+
+ mutex_lock(&ctl->dsp->pwr_lock);
+
+ if (copy_from_user(ctl->cache, bytes, size)) {
+ ret = -EFAULT;
+ } else {
+ ctl->set = 1;
+ if (ctl->enabled)
+ ret = wm_coeff_write_control(ctl, ctl->cache, size);
+ }
+
+ mutex_unlock(&ctl->dsp->pwr_lock);
+
+ return ret;
+}
+
static int wm_coeff_read_control(struct wm_coeff_ctl *ctl,
void *buf, size_t len)
{
@@ -822,7 +860,9 @@
static int wm_coeff_get(struct snd_kcontrol *kctl,
struct snd_ctl_elem_value *ucontrol)
{
- struct wm_coeff_ctl *ctl = (struct wm_coeff_ctl *)kctl->private_value;
+ struct soc_bytes_ext *bytes_ext =
+ (struct soc_bytes_ext *)kctl->private_value;
+ struct wm_coeff_ctl *ctl = bytes_ext_to_ctl(bytes_ext);
char *p = ucontrol->value.bytes.data;
int ret = 0;
@@ -845,12 +885,72 @@
return ret;
}
+static int wm_coeff_tlv_get(struct snd_kcontrol *kctl,
+ unsigned int __user *bytes, unsigned int size)
+{
+ struct soc_bytes_ext *bytes_ext =
+ (struct soc_bytes_ext *)kctl->private_value;
+ struct wm_coeff_ctl *ctl = bytes_ext_to_ctl(bytes_ext);
+ int ret = 0;
+
+ mutex_lock(&ctl->dsp->pwr_lock);
+
+ if (ctl->flags & WMFW_CTL_FLAG_VOLATILE) {
+ if (ctl->enabled)
+ ret = wm_coeff_read_control(ctl, ctl->cache, size);
+ else
+ ret = -EPERM;
+ } else {
+ if (!ctl->flags && ctl->enabled)
+ ret = wm_coeff_read_control(ctl, ctl->cache, size);
+ }
+
+ if (!ret && copy_to_user(bytes, ctl->cache, size))
+ ret = -EFAULT;
+
+ mutex_unlock(&ctl->dsp->pwr_lock);
+
+ return ret;
+}
+
struct wmfw_ctl_work {
struct wm_adsp *dsp;
struct wm_coeff_ctl *ctl;
struct work_struct work;
};
+static unsigned int wmfw_convert_flags(unsigned int in, unsigned int len)
+{
+ unsigned int out, rd, wr, vol;
+
+ if (len > ADSP_MAX_STD_CTRL_SIZE) {
+ rd = SNDRV_CTL_ELEM_ACCESS_TLV_READ;
+ wr = SNDRV_CTL_ELEM_ACCESS_TLV_WRITE;
+ vol = SNDRV_CTL_ELEM_ACCESS_VOLATILE;
+
+ out = SNDRV_CTL_ELEM_ACCESS_TLV_CALLBACK;
+ } else {
+ rd = SNDRV_CTL_ELEM_ACCESS_READ;
+ wr = SNDRV_CTL_ELEM_ACCESS_WRITE;
+ vol = SNDRV_CTL_ELEM_ACCESS_VOLATILE;
+
+ out = 0;
+ }
+
+ if (in) {
+ if (in & WMFW_CTL_FLAG_READABLE)
+ out |= rd;
+ if (in & WMFW_CTL_FLAG_WRITEABLE)
+ out |= wr;
+ if (in & WMFW_CTL_FLAG_VOLATILE)
+ out |= vol;
+ } else {
+ out |= rd | wr | vol;
+ }
+
+ return out;
+}
+
static int wmfw_add_ctl(struct wm_adsp *dsp, struct wm_coeff_ctl *ctl)
{
struct snd_kcontrol_new *kcontrol;
@@ -868,19 +968,15 @@
kcontrol->info = wm_coeff_info;
kcontrol->get = wm_coeff_get;
kcontrol->put = wm_coeff_put;
- kcontrol->private_value = (unsigned long)ctl;
+ kcontrol->iface = SNDRV_CTL_ELEM_IFACE_MIXER;
+ kcontrol->tlv.c = snd_soc_bytes_tlv_callback;
+ kcontrol->private_value = (unsigned long)&ctl->bytes_ext;
- if (ctl->flags) {
- if (ctl->flags & WMFW_CTL_FLAG_WRITEABLE)
- kcontrol->access |= SNDRV_CTL_ELEM_ACCESS_WRITE;
- if (ctl->flags & WMFW_CTL_FLAG_READABLE)
- kcontrol->access |= SNDRV_CTL_ELEM_ACCESS_READ;
- if (ctl->flags & WMFW_CTL_FLAG_VOLATILE)
- kcontrol->access |= SNDRV_CTL_ELEM_ACCESS_VOLATILE;
- } else {
- kcontrol->access = SNDRV_CTL_ELEM_ACCESS_READWRITE;
- kcontrol->access |= SNDRV_CTL_ELEM_ACCESS_VOLATILE;
- }
+ ctl->bytes_ext.max = ctl->len;
+ ctl->bytes_ext.get = wm_coeff_tlv_get;
+ ctl->bytes_ext.put = wm_coeff_tlv_put;
+
+ kcontrol->access = wmfw_convert_flags(ctl->flags, ctl->len);
ret = snd_soc_add_card_controls(dsp->card, kcontrol, 1);
if (ret < 0)
@@ -944,6 +1040,13 @@
kfree(ctl_work);
}
+static void wm_adsp_free_ctl_blk(struct wm_coeff_ctl *ctl)
+{
+ kfree(ctl->cache);
+ kfree(ctl->name);
+ kfree(ctl);
+}
+
static int wm_adsp_create_control(struct wm_adsp *dsp,
const struct wm_adsp_alg_region *alg_region,
unsigned int offset, unsigned int len,
@@ -1032,11 +1135,6 @@
ctl->flags = flags;
ctl->offset = offset;
- if (len > 512) {
- adsp_warn(dsp, "Truncating control %s from %d\n",
- ctl->name, len);
- len = 512;
- }
ctl->len = len;
ctl->cache = kzalloc(ctl->len, GFP_KERNEL);
if (!ctl->cache) {
@@ -1564,6 +1662,19 @@
return alg_region;
}
+static void wm_adsp_free_alg_regions(struct wm_adsp *dsp)
+{
+ struct wm_adsp_alg_region *alg_region;
+
+ while (!list_empty(&dsp->alg_regions)) {
+ alg_region = list_first_entry(&dsp->alg_regions,
+ struct wm_adsp_alg_region,
+ list);
+ list_del(&alg_region->list);
+ kfree(alg_region);
+ }
+}
+
static int wm_adsp1_setup_algs(struct wm_adsp *dsp)
{
struct wmfw_adsp1_id_hdr adsp1_id;
@@ -1994,7 +2105,6 @@
struct snd_soc_codec *codec = snd_soc_dapm_to_codec(w->dapm);
struct wm_adsp *dsps = snd_soc_codec_get_drvdata(codec);
struct wm_adsp *dsp = &dsps[w->shift];
- struct wm_adsp_alg_region *alg_region;
struct wm_coeff_ctl *ctl;
int ret;
unsigned int val;
@@ -2074,13 +2184,8 @@
list_for_each_entry(ctl, &dsp->ctl_list, list)
ctl->enabled = 0;
- while (!list_empty(&dsp->alg_regions)) {
- alg_region = list_first_entry(&dsp->alg_regions,
- struct wm_adsp_alg_region,
- list);
- list_del(&alg_region->list);
- kfree(alg_region);
- }
+
+ wm_adsp_free_alg_regions(dsp);
break;
default:
@@ -2222,7 +2327,6 @@
struct snd_soc_codec *codec = snd_soc_dapm_to_codec(w->dapm);
struct wm_adsp *dsps = snd_soc_codec_get_drvdata(codec);
struct wm_adsp *dsp = &dsps[w->shift];
- struct wm_adsp_alg_region *alg_region;
struct wm_coeff_ctl *ctl;
int ret;
@@ -2240,9 +2344,13 @@
if (ret != 0)
goto err;
+ mutex_lock(&dsp->pwr_lock);
+
if (wm_adsp_fw[dsp->fw].num_caps != 0)
ret = wm_adsp_buffer_init(dsp);
+ mutex_unlock(&dsp->pwr_lock);
+
break;
case SND_SOC_DAPM_PRE_PMD:
@@ -2269,13 +2377,7 @@
list_for_each_entry(ctl, &dsp->ctl_list, list)
ctl->enabled = 0;
- while (!list_empty(&dsp->alg_regions)) {
- alg_region = list_first_entry(&dsp->alg_regions,
- struct wm_adsp_alg_region,
- list);
- list_del(&alg_region->list);
- kfree(alg_region);
- }
+ wm_adsp_free_alg_regions(dsp);
if (wm_adsp_fw[dsp->fw].num_caps != 0)
wm_adsp_buffer_free(dsp);
@@ -2340,6 +2442,54 @@
}
EXPORT_SYMBOL_GPL(wm_adsp2_init);
+void wm_adsp2_remove(struct wm_adsp *dsp)
+{
+ struct wm_coeff_ctl *ctl;
+
+ while (!list_empty(&dsp->ctl_list)) {
+ ctl = list_first_entry(&dsp->ctl_list, struct wm_coeff_ctl,
+ list);
+ list_del(&ctl->list);
+ wm_adsp_free_ctl_blk(ctl);
+ }
+}
+EXPORT_SYMBOL_GPL(wm_adsp2_remove);
+
+static inline int wm_adsp_compr_attached(struct wm_adsp_compr *compr)
+{
+ return compr->buf != NULL;
+}
+
+static int wm_adsp_compr_attach(struct wm_adsp_compr *compr)
+{
+ /*
+ * Note this will be more complex once each DSP can support multiple
+ * streams
+ */
+ if (!compr->dsp->buffer)
+ return -EINVAL;
+
+ compr->buf = compr->dsp->buffer;
+ compr->buf->compr = compr;
+
+ return 0;
+}
+
+static void wm_adsp_compr_detach(struct wm_adsp_compr *compr)
+{
+ if (!compr)
+ return;
+
+ /* Wake the poll so it can see buffer is no longer attached */
+ if (compr->stream)
+ snd_compr_fragment_elapsed(compr->stream);
+
+ if (wm_adsp_compr_attached(compr)) {
+ compr->buf->compr = NULL;
+ compr->buf = NULL;
+ }
+}
+
int wm_adsp_compr_open(struct wm_adsp *dsp, struct snd_compr_stream *stream)
{
struct wm_adsp_compr *compr;
@@ -2393,6 +2543,7 @@
mutex_lock(&dsp->pwr_lock);
+ wm_adsp_compr_detach(compr);
dsp->compr = NULL;
kfree(compr->raw_buf);
@@ -2689,6 +2840,8 @@
static int wm_adsp_buffer_free(struct wm_adsp *dsp)
{
if (dsp->buffer) {
+ wm_adsp_compr_detach(dsp->buffer->compr);
+
kfree(dsp->buffer->regions);
kfree(dsp->buffer);
@@ -2698,25 +2851,6 @@
return 0;
}
-static inline int wm_adsp_compr_attached(struct wm_adsp_compr *compr)
-{
- return compr->buf != NULL;
-}
-
-static int wm_adsp_compr_attach(struct wm_adsp_compr *compr)
-{
- /*
- * Note this will be more complex once each DSP can support multiple
- * streams
- */
- if (!compr->dsp->buffer)
- return -EINVAL;
-
- compr->buf = compr->dsp->buffer;
-
- return 0;
-}
-
int wm_adsp_compr_trigger(struct snd_compr_stream *stream, int cmd)
{
struct wm_adsp_compr *compr = stream->runtime->private_data;
@@ -2805,21 +2939,41 @@
avail += wm_adsp_buffer_size(buf);
adsp_dbg(buf->dsp, "readindex=0x%x, writeindex=0x%x, avail=%d\n",
- buf->read_index, write_index, avail);
+ buf->read_index, write_index, avail * WM_ADSP_DATA_WORD_SIZE);
buf->avail = avail;
return 0;
}
+static int wm_adsp_buffer_get_error(struct wm_adsp_compr_buf *buf)
+{
+ int ret;
+
+ ret = wm_adsp_buffer_read(buf, HOST_BUFFER_FIELD(error), &buf->error);
+ if (ret < 0) {
+ adsp_err(buf->dsp, "Failed to check buffer error: %d\n", ret);
+ return ret;
+ }
+ if (buf->error != 0) {
+ adsp_err(buf->dsp, "Buffer error occurred: %d\n", buf->error);
+ return -EIO;
+ }
+
+ return 0;
+}
+
int wm_adsp_compr_handle_irq(struct wm_adsp *dsp)
{
- struct wm_adsp_compr_buf *buf = dsp->buffer;
- struct wm_adsp_compr *compr = dsp->compr;
+ struct wm_adsp_compr_buf *buf;
+ struct wm_adsp_compr *compr;
int ret = 0;
mutex_lock(&dsp->pwr_lock);
+ buf = dsp->buffer;
+ compr = dsp->compr;
+
if (!buf) {
ret = -ENODEV;
goto out;
@@ -2827,16 +2981,9 @@
adsp_dbg(dsp, "Handling buffer IRQ\n");
- ret = wm_adsp_buffer_read(buf, HOST_BUFFER_FIELD(error), &buf->error);
- if (ret < 0) {
- adsp_err(dsp, "Failed to check buffer error: %d\n", ret);
- goto out;
- }
- if (buf->error != 0) {
- adsp_err(dsp, "Buffer error occurred: %d\n", buf->error);
- ret = -EIO;
- goto out;
- }
+ ret = wm_adsp_buffer_get_error(buf);
+ if (ret < 0)
+ goto out_notify; /* Wake poll to report error */
ret = wm_adsp_buffer_read(buf, HOST_BUFFER_FIELD(irq_count),
&buf->irq_count);
@@ -2851,6 +2998,7 @@
goto out;
}
+out_notify:
if (compr && compr->stream)
snd_compr_fragment_elapsed(compr->stream);
@@ -2879,14 +3027,16 @@
struct snd_compr_tstamp *tstamp)
{
struct wm_adsp_compr *compr = stream->runtime->private_data;
- struct wm_adsp_compr_buf *buf = compr->buf;
struct wm_adsp *dsp = compr->dsp;
+ struct wm_adsp_compr_buf *buf;
int ret = 0;
adsp_dbg(dsp, "Pointer request\n");
mutex_lock(&dsp->pwr_lock);
+ buf = compr->buf;
+
if (!compr->buf) {
ret = -ENXIO;
goto out;
@@ -2909,6 +3059,10 @@
* DSP to inform us once a whole fragment is available.
*/
if (buf->avail < wm_adsp_compr_frag_words(compr)) {
+ ret = wm_adsp_buffer_get_error(buf);
+ if (ret < 0)
+ goto out;
+
ret = wm_adsp_buffer_reenable_irq(buf);
if (ret < 0) {
adsp_err(dsp,
diff --git a/sound/soc/codecs/wm_adsp.h b/sound/soc/codecs/wm_adsp.h
index b61cb57..feb61e2 100644
--- a/sound/soc/codecs/wm_adsp.h
+++ b/sound/soc/codecs/wm_adsp.h
@@ -92,6 +92,7 @@
int wm_adsp1_init(struct wm_adsp *dsp);
int wm_adsp2_init(struct wm_adsp *dsp);
+void wm_adsp2_remove(struct wm_adsp *dsp);
int wm_adsp2_codec_probe(struct wm_adsp *dsp, struct snd_soc_codec *codec);
int wm_adsp2_codec_remove(struct wm_adsp *dsp, struct snd_soc_codec *codec);
int wm_adsp1_event(struct snd_soc_dapm_widget *w,
diff --git a/sound/soc/davinci/Kconfig b/sound/soc/davinci/Kconfig
index 50ca291..6b732d8 100644
--- a/sound/soc/davinci/Kconfig
+++ b/sound/soc/davinci/Kconfig
@@ -16,7 +16,11 @@
- DRA7xx family
config SND_DAVINCI_SOC_I2S
- tristate
+ tristate "DaVinci Multichannel Buffered Serial Port (McBSP) support"
+ depends on SND_EDMA_SOC
+ help
+ Say Y or M here if you want to have support for McBSP IP found in
+ Texas Instruments DaVinci DA850 SoCs.
config SND_DAVINCI_SOC_MCASP
tristate "Multichannel Audio Serial Port (McASP) support"
diff --git a/sound/soc/davinci/davinci-i2s.c b/sound/soc/davinci/davinci-i2s.c
index ec98548..3849616 100644
--- a/sound/soc/davinci/davinci-i2s.c
+++ b/sound/soc/davinci/davinci-i2s.c
@@ -4,9 +4,15 @@
* Author: Vladimir Barinov, <vbarinov@embeddedalley.com>
* Copyright: (C) 2007 MontaVista Software, Inc., <source@mvista.com>
*
+ * DT support (c) 2016 Petr Kulhavy, Barix AG <petr@barix.com>
+ * based on davinci-mcasp.c DT support
+ *
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
+ *
+ * TODO:
+ * on DA850 implement HW FIFOs instead of DMA into DXR and DRR registers
*/
#include <linux/init.h>
@@ -650,13 +656,24 @@
static int davinci_i2s_probe(struct platform_device *pdev)
{
+ struct snd_dmaengine_dai_dma_data *dma_data;
struct davinci_mcbsp_dev *dev;
struct resource *mem, *res;
void __iomem *io_base;
int *dma;
int ret;
- mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ mem = platform_get_resource_byname(pdev, IORESOURCE_MEM, "mpu");
+ if (!mem) {
+ dev_warn(&pdev->dev,
+ "\"mpu\" mem resource not found, using index 0\n");
+ mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (!mem) {
+ dev_err(&pdev->dev, "no mem resource?\n");
+ return -ENODEV;
+ }
+ }
+
io_base = devm_ioremap_resource(&pdev->dev, mem);
if (IS_ERR(io_base))
return PTR_ERR(io_base);
@@ -666,40 +683,44 @@
if (!dev)
return -ENOMEM;
+ dev->base = io_base;
+
+ /* setup DMA, first TX, then RX */
+ dma_data = &dev->dma_data[SNDRV_PCM_STREAM_PLAYBACK];
+ dma_data->addr = (dma_addr_t)(mem->start + DAVINCI_MCBSP_DXR_REG);
+
+ res = platform_get_resource(pdev, IORESOURCE_DMA, 0);
+ if (res) {
+ dma = &dev->dma_request[SNDRV_PCM_STREAM_PLAYBACK];
+ *dma = res->start;
+ dma_data->filter_data = dma;
+ } else if (IS_ENABLED(CONFIG_OF) && pdev->dev.of_node) {
+ dma_data->filter_data = "tx";
+ } else {
+ dev_err(&pdev->dev, "Missing DMA tx resource\n");
+ return -ENODEV;
+ }
+
+ dma_data = &dev->dma_data[SNDRV_PCM_STREAM_CAPTURE];
+ dma_data->addr = (dma_addr_t)(mem->start + DAVINCI_MCBSP_DRR_REG);
+
+ res = platform_get_resource(pdev, IORESOURCE_DMA, 1);
+ if (res) {
+ dma = &dev->dma_request[SNDRV_PCM_STREAM_CAPTURE];
+ *dma = res->start;
+ dma_data->filter_data = dma;
+ } else if (IS_ENABLED(CONFIG_OF) && pdev->dev.of_node) {
+ dma_data->filter_data = "rx";
+ } else {
+ dev_err(&pdev->dev, "Missing DMA rx resource\n");
+ return -ENODEV;
+ }
+
dev->clk = clk_get(&pdev->dev, NULL);
if (IS_ERR(dev->clk))
return -ENODEV;
clk_enable(dev->clk);
- dev->base = io_base;
-
- dev->dma_data[SNDRV_PCM_STREAM_PLAYBACK].addr =
- (dma_addr_t)(mem->start + DAVINCI_MCBSP_DXR_REG);
-
- dev->dma_data[SNDRV_PCM_STREAM_CAPTURE].addr =
- (dma_addr_t)(mem->start + DAVINCI_MCBSP_DRR_REG);
-
- /* first TX, then RX */
- res = platform_get_resource(pdev, IORESOURCE_DMA, 0);
- if (!res) {
- dev_err(&pdev->dev, "no DMA resource\n");
- ret = -ENXIO;
- goto err_release_clk;
- }
- dma = &dev->dma_request[SNDRV_PCM_STREAM_PLAYBACK];
- *dma = res->start;
- dev->dma_data[SNDRV_PCM_STREAM_PLAYBACK].filter_data = dma;
-
- res = platform_get_resource(pdev, IORESOURCE_DMA, 1);
- if (!res) {
- dev_err(&pdev->dev, "no DMA resource\n");
- ret = -ENXIO;
- goto err_release_clk;
- }
- dma = &dev->dma_request[SNDRV_PCM_STREAM_CAPTURE];
- *dma = res->start;
- dev->dma_data[SNDRV_PCM_STREAM_CAPTURE].filter_data = dma;
-
dev->dev = &pdev->dev;
dev_set_drvdata(&pdev->dev, dev);
@@ -737,11 +758,18 @@
return 0;
}
+static const struct of_device_id davinci_i2s_match[] = {
+ { .compatible = "ti,da850-mcbsp" },
+ {},
+};
+MODULE_DEVICE_TABLE(of, davinci_i2s_match);
+
static struct platform_driver davinci_mcbsp_driver = {
.probe = davinci_i2s_probe,
.remove = davinci_i2s_remove,
.driver = {
.name = "davinci-mcbsp",
+ .of_match_table = of_match_ptr(davinci_i2s_match),
},
};
diff --git a/sound/soc/davinci/davinci-mcasp.c b/sound/soc/davinci/davinci-mcasp.c
index e132498..0f66fda 100644
--- a/sound/soc/davinci/davinci-mcasp.c
+++ b/sound/soc/davinci/davinci-mcasp.c
@@ -489,7 +489,7 @@
mcasp_clr_bits(mcasp, DAVINCI_MCASP_RXFMCTL_REG, AFSRE);
mcasp_clr_bits(mcasp, DAVINCI_MCASP_PDIR_REG,
- ACLKX | AHCLKX | AFSX | ACLKR | AHCLKR | AFSR);
+ ACLKX | AFSX | ACLKR | AHCLKR | AFSR);
mcasp->bclk_master = 0;
break;
default:
@@ -540,21 +540,19 @@
return ret;
}
-static int __davinci_mcasp_set_clkdiv(struct snd_soc_dai *dai, int div_id,
+static int __davinci_mcasp_set_clkdiv(struct davinci_mcasp *mcasp, int div_id,
int div, bool explicit)
{
- struct davinci_mcasp *mcasp = snd_soc_dai_get_drvdata(dai);
-
pm_runtime_get_sync(mcasp->dev);
switch (div_id) {
- case 0: /* MCLK divider */
+ case MCASP_CLKDIV_AUXCLK: /* MCLK divider */
mcasp_mod_bits(mcasp, DAVINCI_MCASP_AHCLKXCTL_REG,
AHCLKXDIV(div - 1), AHCLKXDIV_MASK);
mcasp_mod_bits(mcasp, DAVINCI_MCASP_AHCLKRCTL_REG,
AHCLKRDIV(div - 1), AHCLKRDIV_MASK);
break;
- case 1: /* BCLK divider */
+ case MCASP_CLKDIV_BCLK: /* BCLK divider */
mcasp_mod_bits(mcasp, DAVINCI_MCASP_ACLKXCTL_REG,
ACLKXDIV(div - 1), ACLKXDIV_MASK);
mcasp_mod_bits(mcasp, DAVINCI_MCASP_ACLKRCTL_REG,
@@ -563,7 +561,8 @@
mcasp->bclk_div = div;
break;
- case 2: /*
+ case MCASP_CLKDIV_BCLK_FS_RATIO:
+ /*
* BCLK/LRCLK ratio descries how many bit-clock cycles
* fit into one frame. The clock ratio is given for a
* full period of data (for I2S format both left and
@@ -591,7 +590,9 @@
static int davinci_mcasp_set_clkdiv(struct snd_soc_dai *dai, int div_id,
int div)
{
- return __davinci_mcasp_set_clkdiv(dai, div_id, div, 1);
+ struct davinci_mcasp *mcasp = snd_soc_dai_get_drvdata(dai);
+
+ return __davinci_mcasp_set_clkdiv(mcasp, div_id, div, 1);
}
static int davinci_mcasp_set_sysclk(struct snd_soc_dai *dai, int clk_id,
@@ -999,27 +1000,53 @@
}
static int davinci_mcasp_calc_clk_div(struct davinci_mcasp *mcasp,
- unsigned int bclk_freq,
- int *error_ppm)
+ unsigned int bclk_freq, bool set)
{
- int div = mcasp->sysclk_freq / bclk_freq;
- int rem = mcasp->sysclk_freq % bclk_freq;
+ int error_ppm;
+ unsigned int sysclk_freq = mcasp->sysclk_freq;
+ u32 reg = mcasp_get_reg(mcasp, DAVINCI_MCASP_AHCLKXCTL_REG);
+ int div = sysclk_freq / bclk_freq;
+ int rem = sysclk_freq % bclk_freq;
+ int aux_div = 1;
+
+ if (div > (ACLKXDIV_MASK + 1)) {
+ if (reg & AHCLKXE) {
+ aux_div = div / (ACLKXDIV_MASK + 1);
+ if (div % (ACLKXDIV_MASK + 1))
+ aux_div++;
+
+ sysclk_freq /= aux_div;
+ div = sysclk_freq / bclk_freq;
+ rem = sysclk_freq % bclk_freq;
+ } else if (set) {
+ dev_warn(mcasp->dev, "Too fast reference clock (%u)\n",
+ sysclk_freq);
+ }
+ }
if (rem != 0) {
if (div == 0 ||
- ((mcasp->sysclk_freq / div) - bclk_freq) >
- (bclk_freq - (mcasp->sysclk_freq / (div+1)))) {
+ ((sysclk_freq / div) - bclk_freq) >
+ (bclk_freq - (sysclk_freq / (div+1)))) {
div++;
rem = rem - bclk_freq;
}
}
- if (error_ppm)
- *error_ppm =
- (div*1000000 + (int)div64_long(1000000LL*rem,
- (int)bclk_freq))
- /div - 1000000;
+ error_ppm = (div*1000000 + (int)div64_long(1000000LL*rem,
+ (int)bclk_freq)) / div - 1000000;
- return div;
+ if (set) {
+ if (error_ppm)
+ dev_info(mcasp->dev, "Sample-rate is off by %d PPM\n",
+ error_ppm);
+
+ __davinci_mcasp_set_clkdiv(mcasp, MCASP_CLKDIV_BCLK, div, 0);
+ if (reg & AHCLKXE)
+ __davinci_mcasp_set_clkdiv(mcasp, MCASP_CLKDIV_AUXCLK,
+ aux_div, 0);
+ }
+
+ return error_ppm;
}
static int davinci_mcasp_hw_params(struct snd_pcm_substream *substream,
@@ -1044,18 +1071,11 @@
int slots = mcasp->tdm_slots;
int rate = params_rate(params);
int sbits = params_width(params);
- int ppm, div;
if (mcasp->slot_width)
sbits = mcasp->slot_width;
- div = davinci_mcasp_calc_clk_div(mcasp, rate*sbits*slots,
- &ppm);
- if (ppm)
- dev_info(mcasp->dev, "Sample-rate is off by %d PPM\n",
- ppm);
-
- __davinci_mcasp_set_clkdiv(cpu_dai, 1, div, 0);
+ davinci_mcasp_calc_clk_div(mcasp, rate * sbits * slots, true);
}
ret = mcasp_common_hw_param(mcasp, substream->stream,
@@ -1166,7 +1186,8 @@
davinci_mcasp_dai_rates[i];
int ppm;
- davinci_mcasp_calc_clk_div(rd->mcasp, bclk_freq, &ppm);
+ ppm = davinci_mcasp_calc_clk_div(rd->mcasp, bclk_freq,
+ false);
if (abs(ppm) < DAVINCI_MAX_RATE_ERROR_PPM) {
if (range.empty) {
range.min = davinci_mcasp_dai_rates[i];
@@ -1205,8 +1226,9 @@
if (rd->mcasp->slot_width)
sbits = rd->mcasp->slot_width;
- davinci_mcasp_calc_clk_div(rd->mcasp, sbits*slots*rate,
- &ppm);
+ ppm = davinci_mcasp_calc_clk_div(rd->mcasp,
+ sbits * slots * rate,
+ false);
if (abs(ppm) < DAVINCI_MAX_RATE_ERROR_PPM) {
snd_mask_set(&nfmt, i);
count++;
@@ -1230,11 +1252,15 @@
int i, dir;
int tdm_slots = mcasp->tdm_slots;
- if (mcasp->tdm_mask[substream->stream])
- tdm_slots = hweight32(mcasp->tdm_mask[substream->stream]);
+ /* Do not allow more then one stream per direction */
+ if (mcasp->substreams[substream->stream])
+ return -EBUSY;
mcasp->substreams[substream->stream] = substream;
+ if (mcasp->tdm_mask[substream->stream])
+ tdm_slots = hweight32(mcasp->tdm_mask[substream->stream]);
+
if (mcasp->op_mode == DAVINCI_MCASP_DIT_MODE)
return 0;
diff --git a/sound/soc/davinci/davinci-mcasp.h b/sound/soc/davinci/davinci-mcasp.h
index a3be108..1e8787f 100644
--- a/sound/soc/davinci/davinci-mcasp.h
+++ b/sound/soc/davinci/davinci-mcasp.h
@@ -306,4 +306,9 @@
#define NUMEVT(x) (((x) & 0xFF) << 8)
#define NUMDMA_MASK (0xFF)
+/* clock divider IDs */
+#define MCASP_CLKDIV_AUXCLK 0 /* HCLK divider from AUXCLK */
+#define MCASP_CLKDIV_BCLK 1 /* BCLK divider from HCLK */
+#define MCASP_CLKDIV_BCLK_FS_RATIO 2 /* to set BCLK FS ration */
+
#endif /* DAVINCI_MCASP_H */
diff --git a/sound/soc/dwc/designware_i2s.c b/sound/soc/dwc/designware_i2s.c
index bff258d..0db69b7 100644
--- a/sound/soc/dwc/designware_i2s.c
+++ b/sound/soc/dwc/designware_i2s.c
@@ -100,6 +100,7 @@
struct device *dev;
u32 ccr;
u32 xfer_resolution;
+ u32 fifo_th;
/* data related to DMA transfers b/w i2s and DMAC */
union dw_i2s_snd_dma_data play_dma_data;
@@ -147,17 +148,18 @@
static void i2s_start(struct dw_i2s_dev *dev,
struct snd_pcm_substream *substream)
{
+ struct i2s_clk_config_data *config = &dev->config;
u32 i, irq;
i2s_write_reg(dev->i2s_base, IER, 1);
if (substream->stream == SNDRV_PCM_STREAM_PLAYBACK) {
- for (i = 0; i < 4; i++) {
+ for (i = 0; i < (config->chan_nr / 2); i++) {
irq = i2s_read_reg(dev->i2s_base, IMR(i));
i2s_write_reg(dev->i2s_base, IMR(i), irq & ~0x30);
}
i2s_write_reg(dev->i2s_base, ITER, 1);
} else {
- for (i = 0; i < 4; i++) {
+ for (i = 0; i < (config->chan_nr / 2); i++) {
irq = i2s_read_reg(dev->i2s_base, IMR(i));
i2s_write_reg(dev->i2s_base, IMR(i), irq & ~0x03);
}
@@ -231,14 +233,16 @@
if (stream == SNDRV_PCM_STREAM_PLAYBACK) {
i2s_write_reg(dev->i2s_base, TCR(ch_reg),
dev->xfer_resolution);
- i2s_write_reg(dev->i2s_base, TFCR(ch_reg), 0x02);
+ i2s_write_reg(dev->i2s_base, TFCR(ch_reg),
+ dev->fifo_th - 1);
irq = i2s_read_reg(dev->i2s_base, IMR(ch_reg));
i2s_write_reg(dev->i2s_base, IMR(ch_reg), irq & ~0x30);
i2s_write_reg(dev->i2s_base, TER(ch_reg), 1);
} else {
i2s_write_reg(dev->i2s_base, RCR(ch_reg),
dev->xfer_resolution);
- i2s_write_reg(dev->i2s_base, RFCR(ch_reg), 0x07);
+ i2s_write_reg(dev->i2s_base, RFCR(ch_reg),
+ dev->fifo_th - 1);
irq = i2s_read_reg(dev->i2s_base, IMR(ch_reg));
i2s_write_reg(dev->i2s_base, IMR(ch_reg), irq & ~0x03);
i2s_write_reg(dev->i2s_base, RER(ch_reg), 1);
@@ -498,6 +502,7 @@
*/
u32 comp1 = i2s_read_reg(dev->i2s_base, dev->i2s_reg_comp1);
u32 comp2 = i2s_read_reg(dev->i2s_base, dev->i2s_reg_comp2);
+ u32 fifo_depth = 1 << (1 + COMP1_FIFO_DEPTH_GLOBAL(comp1));
u32 idx;
if (dev->capability & DWC_I2S_RECORD &&
@@ -536,6 +541,7 @@
dev->capability |= DW_I2S_SLAVE;
}
+ dev->fifo_th = fifo_depth / 2;
return 0;
}
diff --git a/sound/soc/fsl/fsl_sai.c b/sound/soc/fsl/fsl_sai.c
index 0754df7..2147994 100644
--- a/sound/soc/fsl/fsl_sai.c
+++ b/sound/soc/fsl/fsl_sai.c
@@ -21,6 +21,8 @@
#include <sound/core.h>
#include <sound/dmaengine_pcm.h>
#include <sound/pcm_params.h>
+#include <linux/mfd/syscon.h>
+#include <linux/mfd/syscon/imx6q-iomuxc-gpr.h>
#include "fsl_sai.h"
#include "imx-pcm.h"
@@ -786,10 +788,12 @@
{
struct device_node *np = pdev->dev.of_node;
struct fsl_sai *sai;
+ struct regmap *gpr;
struct resource *res;
void __iomem *base;
char tmp[8];
int irq, ret, i;
+ int index;
sai = devm_kzalloc(&pdev->dev, sizeof(*sai), GFP_KERNEL);
if (!sai)
@@ -797,7 +801,8 @@
sai->pdev = pdev;
- if (of_device_is_compatible(pdev->dev.of_node, "fsl,imx6sx-sai"))
+ if (of_device_is_compatible(pdev->dev.of_node, "fsl,imx6sx-sai") ||
+ of_device_is_compatible(pdev->dev.of_node, "fsl,imx6ul-sai"))
sai->sai_on_imx = true;
sai->is_lsb_first = of_property_read_bool(np, "lsb-first");
@@ -877,6 +882,22 @@
fsl_sai_dai.symmetric_samplebits = 0;
}
+ if (of_find_property(np, "fsl,sai-mclk-direction-output", NULL) &&
+ of_device_is_compatible(pdev->dev.of_node, "fsl,imx6ul-sai")) {
+ gpr = syscon_regmap_lookup_by_compatible("fsl,imx6ul-iomuxc-gpr");
+ if (IS_ERR(gpr)) {
+ dev_err(&pdev->dev, "cannot find iomuxc registers\n");
+ return PTR_ERR(gpr);
+ }
+
+ index = of_alias_get_id(np, "sai");
+ if (index < 0)
+ return index;
+
+ regmap_update_bits(gpr, IOMUXC_GPR1, MCLK_DIR(index),
+ MCLK_DIR(index));
+ }
+
sai->dma_params_rx.addr = res->start + FSL_SAI_RDR;
sai->dma_params_tx.addr = res->start + FSL_SAI_TDR;
sai->dma_params_rx.maxburst = FSL_SAI_MAXBURST_RX;
@@ -898,6 +919,7 @@
static const struct of_device_id fsl_sai_ids[] = {
{ .compatible = "fsl,vf610-sai", },
{ .compatible = "fsl,imx6sx-sai", },
+ { .compatible = "fsl,imx6ul-sai", },
{ /* sentinel */ }
};
MODULE_DEVICE_TABLE(of, fsl_sai_ids);
diff --git a/sound/soc/fsl/fsl_ssi.c b/sound/soc/fsl/fsl_ssi.c
index ed8de10..632ecc0 100644
--- a/sound/soc/fsl/fsl_ssi.c
+++ b/sound/soc/fsl/fsl_ssi.c
@@ -137,6 +137,7 @@
case CCSR_SSI_SACDAT:
case CCSR_SSI_SATAG:
case CCSR_SSI_SACCST:
+ case CCSR_SSI_SOR:
return true;
default:
return false;
@@ -261,6 +262,7 @@
struct fsl_ssi_dbg dbg_stats;
const struct fsl_ssi_soc_data *soc;
+ struct device *dev;
};
/*
@@ -400,6 +402,26 @@
}
/*
+ * Clear RX or TX FIFO to remove samples from the previous
+ * stream session which may be still present in the FIFO and
+ * may introduce bad samples and/or channel slipping.
+ *
+ * Note: The SOR is not documented in recent IMX datasheet, but
+ * is described in IMX51 reference manual at section 56.3.3.15.
+ */
+static void fsl_ssi_fifo_clear(struct fsl_ssi_private *ssi_private,
+ bool is_rx)
+{
+ if (is_rx) {
+ regmap_update_bits(ssi_private->regs, CCSR_SSI_SOR,
+ CCSR_SSI_SOR_RX_CLR, CCSR_SSI_SOR_RX_CLR);
+ } else {
+ regmap_update_bits(ssi_private->regs, CCSR_SSI_SOR,
+ CCSR_SSI_SOR_TX_CLR, CCSR_SSI_SOR_TX_CLR);
+ }
+}
+
+/*
* Calculate the bits that have to be disabled for the current stream that is
* getting disabled. This keeps the bits enabled that are necessary for the
* second stream to work if 'stream_active' is true.
@@ -474,9 +496,11 @@
* (online configuration)
*/
if (enable) {
- regmap_update_bits(regs, CCSR_SSI_SIER, vals->sier, vals->sier);
+ fsl_ssi_fifo_clear(ssi_private, vals->scr & CCSR_SSI_SCR_RE);
+
regmap_update_bits(regs, CCSR_SSI_SRCR, vals->srcr, vals->srcr);
regmap_update_bits(regs, CCSR_SSI_STCR, vals->stcr, vals->stcr);
+ regmap_update_bits(regs, CCSR_SSI_SIER, vals->sier, vals->sier);
} else {
u32 sier;
u32 srcr;
@@ -506,8 +530,40 @@
config_done:
/* Enabling of subunits is done after configuration */
- if (enable)
+ if (enable) {
+ if (ssi_private->use_dma && (vals->scr & CCSR_SSI_SCR_TE)) {
+ /*
+ * Be sure the Tx FIFO is filled when TE is set.
+ * Otherwise, there are some chances to start the
+ * playback with some void samples inserted first,
+ * generating a channel slip.
+ *
+ * First, SSIEN must be set, to let the FIFO be filled.
+ *
+ * Notes:
+ * - Limit this fix to the DMA case until FIQ cases can
+ * be tested.
+ * - Limit the length of the busy loop to not lock the
+ * system too long, even if 1-2 loops are sufficient
+ * in general.
+ */
+ int i;
+ int max_loop = 100;
+ regmap_update_bits(regs, CCSR_SSI_SCR,
+ CCSR_SSI_SCR_SSIEN, CCSR_SSI_SCR_SSIEN);
+ for (i = 0; i < max_loop; i++) {
+ u32 sfcsr;
+ regmap_read(regs, CCSR_SSI_SFCSR, &sfcsr);
+ if (CCSR_SSI_SFCSR_TFCNT0(sfcsr))
+ break;
+ }
+ if (i == max_loop) {
+ dev_err(ssi_private->dev,
+ "Timeout waiting TX FIFO filling\n");
+ }
+ }
regmap_update_bits(regs, CCSR_SSI_SCR, vals->scr, vals->scr);
+ }
}
@@ -670,6 +726,15 @@
if (IS_ERR(ssi_private->baudclk))
return -EINVAL;
+ /*
+ * Hardware limitation: The bclk rate must be
+ * never greater than 1/5 IPG clock rate
+ */
+ if (freq * 5 > clk_get_rate(ssi_private->clk)) {
+ dev_err(cpu_dai->dev, "bitclk > ipgclk/5\n");
+ return -EINVAL;
+ }
+
baudclk_is_used = ssi_private->baudclk_streams & ~(BIT(substream->stream));
/* It should be already enough to divide clock by setting pm alone */
@@ -686,13 +751,6 @@
else
clkrate = clk_round_rate(ssi_private->baudclk, tmprate);
- /*
- * Hardware limitation: The bclk rate must be
- * never greater than 1/5 IPG clock rate
- */
- if (clkrate * 5 > clk_get_rate(ssi_private->clk))
- continue;
-
clkrate /= factor;
afreq = clkrate / (i + 1);
@@ -1158,14 +1216,14 @@
.playback = {
.stream_name = "CPU-Playback",
.channels_min = 1,
- .channels_max = 2,
+ .channels_max = 32,
.rates = FSLSSI_I2S_RATES,
.formats = FSLSSI_I2S_FORMATS,
},
.capture = {
.stream_name = "CPU-Capture",
.channels_min = 1,
- .channels_max = 2,
+ .channels_max = 32,
.rates = FSLSSI_I2S_RATES,
.formats = FSLSSI_I2S_FORMATS,
},
@@ -1402,6 +1460,7 @@
}
ssi_private->soc = of_id->data;
+ ssi_private->dev = &pdev->dev;
sprop = of_get_property(np, "fsl,mode", NULL);
if (sprop) {
diff --git a/sound/soc/fsl/imx-pcm-fiq.c b/sound/soc/fsl/imx-pcm-fiq.c
index e63cd5e..dac6688 100644
--- a/sound/soc/fsl/imx-pcm-fiq.c
+++ b/sound/soc/fsl/imx-pcm-fiq.c
@@ -220,7 +220,7 @@
ret = dma_mmap_wc(substream->pcm->card->dev, vma, runtime->dma_area,
runtime->dma_addr, runtime->dma_bytes);
- pr_debug("%s: ret: %d %p %pad 0x%08x\n", __func__, ret,
+ pr_debug("%s: ret: %d %p %pad 0x%08zx\n", __func__, ret,
runtime->dma_area,
&runtime->dma_addr,
runtime->dma_bytes);
diff --git a/sound/soc/intel/Kconfig b/sound/soc/intel/Kconfig
index 1120f4f..91c15ab 100644
--- a/sound/soc/intel/Kconfig
+++ b/sound/soc/intel/Kconfig
@@ -58,6 +58,21 @@
Say Y if you have such a device
If unsure select "N".
+config SND_SOC_INTEL_BXT_RT298_MACH
+ tristate "ASoC Audio driver for Broxton with RT298 I2S mode"
+ depends on X86 && ACPI && I2C
+ select SND_SOC_INTEL_SST
+ select SND_SOC_INTEL_SKYLAKE
+ select SND_SOC_RT298
+ select SND_SOC_DMIC
+ select SND_SOC_HDAC_HDMI
+ select SND_HDA_DSP_LOADER
+ help
+ This adds support for ASoC machine driver for Broxton platforms
+ with RT286 I2S audio codec.
+ Say Y if you have such a device
+ If unsure select "N".
+
config SND_SOC_INTEL_BYT_RT5640_MACH
tristate "ASoC Audio driver for Intel Baytrail with RT5640 codec"
depends on X86_INTEL_LPSS && I2C
@@ -162,6 +177,7 @@
config SND_SOC_INTEL_SKYLAKE
tristate
select SND_HDA_EXT_CORE
+ select SND_HDA_DSP_LOADER
select SND_SOC_TOPOLOGY
select SND_SOC_INTEL_SST
diff --git a/sound/soc/intel/atom/sst-atom-controls.c b/sound/soc/intel/atom/sst-atom-controls.c
index b97e6ad..98720a9 100644
--- a/sound/soc/intel/atom/sst-atom-controls.c
+++ b/sound/soc/intel/atom/sst-atom-controls.c
@@ -195,7 +195,7 @@
if (e->w && e->w->power)
ret = sst_send_slot_map(drv);
- else
+ else if (!e->w)
dev_err(&drv->pdev->dev, "Slot control: %s doesn't have DAPM widget!!!\n",
kcontrol->id.name);
return ret;
diff --git a/sound/soc/intel/boards/Makefile b/sound/soc/intel/boards/Makefile
index 3310c0f..a850677 100644
--- a/sound/soc/intel/boards/Makefile
+++ b/sound/soc/intel/boards/Makefile
@@ -2,6 +2,7 @@
snd-soc-sst-byt-rt5640-mach-objs := byt-rt5640.o
snd-soc-sst-byt-max98090-mach-objs := byt-max98090.o
snd-soc-sst-broadwell-objs := broadwell.o
+snd-soc-sst-bxt-rt298-objs := bxt_rt298.o
snd-soc-sst-bytcr-rt5640-objs := bytcr_rt5640.o
snd-soc-sst-bytcr-rt5651-objs := bytcr_rt5651.o
snd-soc-sst-cht-bsw-rt5672-objs := cht_bsw_rt5672.o
@@ -14,6 +15,7 @@
obj-$(CONFIG_SND_SOC_INTEL_HASWELL_MACH) += snd-soc-sst-haswell.o
obj-$(CONFIG_SND_SOC_INTEL_BYT_RT5640_MACH) += snd-soc-sst-byt-rt5640-mach.o
obj-$(CONFIG_SND_SOC_INTEL_BYT_MAX98090_MACH) += snd-soc-sst-byt-max98090-mach.o
+obj-$(CONFIG_SND_SOC_INTEL_BXT_RT298_MACH) += snd-soc-sst-bxt-rt298.o
obj-$(CONFIG_SND_SOC_INTEL_BROADWELL_MACH) += snd-soc-sst-broadwell.o
obj-$(CONFIG_SND_SOC_INTEL_BYTCR_RT5640_MACH) += snd-soc-sst-bytcr-rt5640.o
obj-$(CONFIG_SND_SOC_INTEL_BYTCR_RT5651_MACH) += snd-soc-sst-bytcr-rt5651.o
diff --git a/sound/soc/intel/boards/broadwell.c b/sound/soc/intel/boards/broadwell.c
index 3f8a1e1..7486a002 100644
--- a/sound/soc/intel/boards/broadwell.c
+++ b/sound/soc/intel/boards/broadwell.c
@@ -201,7 +201,7 @@
{
/* SSP0 - Codec */
.name = "Codec",
- .be_id = 0,
+ .id = 0,
.cpu_dai_name = "snd-soc-dummy-dai",
.platform_name = "snd-soc-dummy",
.no_pcm = 1,
diff --git a/sound/soc/intel/boards/bxt_rt298.c b/sound/soc/intel/boards/bxt_rt298.c
new file mode 100644
index 0000000..f478751
--- /dev/null
+++ b/sound/soc/intel/boards/bxt_rt298.c
@@ -0,0 +1,353 @@
+/*
+ * Intel Broxton-P I2S Machine Driver
+ *
+ * Copyright (C) 2014-2016, Intel Corporation. All rights reserved.
+ *
+ * Modified from:
+ * Intel Skylake I2S Machine driver
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version
+ * 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <sound/core.h>
+#include <sound/pcm.h>
+#include <sound/soc.h>
+#include <sound/jack.h>
+#include <sound/pcm_params.h>
+#include "../../codecs/hdac_hdmi.h"
+#include "../../codecs/rt298.h"
+
+static struct snd_soc_jack broxton_headset;
+/* Headset jack detection DAPM pins */
+
+enum {
+ BXT_DPCM_AUDIO_PB = 0,
+ BXT_DPCM_AUDIO_CP,
+ BXT_DPCM_AUDIO_REF_CP,
+ BXT_DPCM_AUDIO_HDMI1_PB,
+ BXT_DPCM_AUDIO_HDMI2_PB,
+ BXT_DPCM_AUDIO_HDMI3_PB,
+};
+
+static struct snd_soc_jack_pin broxton_headset_pins[] = {
+ {
+ .pin = "Mic Jack",
+ .mask = SND_JACK_MICROPHONE,
+ },
+ {
+ .pin = "Headphone Jack",
+ .mask = SND_JACK_HEADPHONE,
+ },
+};
+
+static const struct snd_kcontrol_new broxton_controls[] = {
+ SOC_DAPM_PIN_SWITCH("Speaker"),
+ SOC_DAPM_PIN_SWITCH("Headphone Jack"),
+ SOC_DAPM_PIN_SWITCH("Mic Jack"),
+};
+
+static const struct snd_soc_dapm_widget broxton_widgets[] = {
+ SND_SOC_DAPM_HP("Headphone Jack", NULL),
+ SND_SOC_DAPM_SPK("Speaker", NULL),
+ SND_SOC_DAPM_MIC("Mic Jack", NULL),
+ SND_SOC_DAPM_MIC("DMIC2", NULL),
+ SND_SOC_DAPM_MIC("SoC DMIC", NULL),
+ SND_SOC_DAPM_SPK("HDMI1", NULL),
+ SND_SOC_DAPM_SPK("HDMI2", NULL),
+ SND_SOC_DAPM_SPK("HDMI3", NULL),
+};
+
+static const struct snd_soc_dapm_route broxton_rt298_map[] = {
+ /* speaker */
+ {"Speaker", NULL, "SPOR"},
+ {"Speaker", NULL, "SPOL"},
+
+ /* HP jack connectors - unknown if we have jack detect */
+ {"Headphone Jack", NULL, "HPO Pin"},
+
+ /* other jacks */
+ {"MIC1", NULL, "Mic Jack"},
+
+ /* digital mics */
+ {"DMIC1 Pin", NULL, "DMIC2"},
+ {"DMic", NULL, "SoC DMIC"},
+
+ {"HDMI1", NULL, "hif5 Output"},
+ {"HDMI2", NULL, "hif6 Output"},
+ {"HDMI3", NULL, "hif7 Output"},
+
+ /* CODEC BE connections */
+ { "AIF1 Playback", NULL, "ssp5 Tx"},
+ { "ssp5 Tx", NULL, "codec0_out"},
+
+ { "codec0_in", NULL, "ssp5 Rx" },
+ { "ssp5 Rx", NULL, "AIF1 Capture" },
+
+ { "dmic01_hifi", NULL, "DMIC01 Rx" },
+ { "DMIC01 Rx", NULL, "Capture" },
+
+ { "hifi3", NULL, "iDisp3 Tx"},
+ { "iDisp3 Tx", NULL, "iDisp3_out"},
+ { "hifi2", NULL, "iDisp2 Tx"},
+ { "iDisp2 Tx", NULL, "iDisp2_out"},
+ { "hifi1", NULL, "iDisp1 Tx"},
+ { "iDisp1 Tx", NULL, "iDisp1_out"},
+
+};
+
+static int broxton_rt298_codec_init(struct snd_soc_pcm_runtime *rtd)
+{
+ struct snd_soc_codec *codec = rtd->codec;
+ int ret = 0;
+
+ ret = snd_soc_card_jack_new(rtd->card, "Headset",
+ SND_JACK_HEADSET | SND_JACK_BTN_0,
+ &broxton_headset,
+ broxton_headset_pins, ARRAY_SIZE(broxton_headset_pins));
+
+ if (ret)
+ return ret;
+
+ rt298_mic_detect(codec, &broxton_headset);
+ return 0;
+}
+
+static int broxton_hdmi_init(struct snd_soc_pcm_runtime *rtd)
+{
+ struct snd_soc_dai *dai = rtd->codec_dai;
+
+ return hdac_hdmi_jack_init(dai, BXT_DPCM_AUDIO_HDMI1_PB + dai->id);
+}
+
+static int broxton_ssp5_fixup(struct snd_soc_pcm_runtime *rtd,
+ struct snd_pcm_hw_params *params)
+{
+ struct snd_interval *rate = hw_param_interval(params,
+ SNDRV_PCM_HW_PARAM_RATE);
+ struct snd_interval *channels = hw_param_interval(params,
+ SNDRV_PCM_HW_PARAM_CHANNELS);
+ struct snd_mask *fmt = hw_param_mask(params, SNDRV_PCM_HW_PARAM_FORMAT);
+
+ /* The ADSP will covert the FE rate to 48k, stereo */
+ rate->min = rate->max = 48000;
+ channels->min = channels->max = 2;
+
+ /* set SSP5 to 24 bit */
+ snd_mask_none(fmt);
+ snd_mask_set(fmt, SNDRV_PCM_FORMAT_S24_LE);
+
+ return 0;
+}
+
+static int broxton_rt298_hw_params(struct snd_pcm_substream *substream,
+ struct snd_pcm_hw_params *params)
+{
+ struct snd_soc_pcm_runtime *rtd = substream->private_data;
+ struct snd_soc_dai *codec_dai = rtd->codec_dai;
+ int ret;
+
+ ret = snd_soc_dai_set_sysclk(codec_dai, RT298_SCLK_S_PLL,
+ 19200000, SND_SOC_CLOCK_IN);
+ if (ret < 0) {
+ dev_err(rtd->dev, "can't set codec sysclk configuration\n");
+ return ret;
+ }
+
+ return ret;
+}
+
+static struct snd_soc_ops broxton_rt298_ops = {
+ .hw_params = broxton_rt298_hw_params,
+};
+
+/* broxton digital audio interface glue - connects codec <--> CPU */
+static struct snd_soc_dai_link broxton_rt298_dais[] = {
+ /* Front End DAI links */
+ [BXT_DPCM_AUDIO_PB]
+ {
+ .name = "Bxt Audio Port",
+ .stream_name = "Audio",
+ .cpu_dai_name = "System Pin",
+ .platform_name = "0000:00:0e.0",
+ .nonatomic = 1,
+ .dynamic = 1,
+ .codec_name = "snd-soc-dummy",
+ .codec_dai_name = "snd-soc-dummy-dai",
+ .trigger = {SND_SOC_DPCM_TRIGGER_POST, SND_SOC_DPCM_TRIGGER_POST},
+ .dpcm_playback = 1,
+ },
+ [BXT_DPCM_AUDIO_CP]
+ {
+ .name = "Bxt Audio Capture Port",
+ .stream_name = "Audio Record",
+ .cpu_dai_name = "System Pin",
+ .platform_name = "0000:00:0e.0",
+ .nonatomic = 1,
+ .dynamic = 1,
+ .codec_name = "snd-soc-dummy",
+ .codec_dai_name = "snd-soc-dummy-dai",
+ .trigger = {SND_SOC_DPCM_TRIGGER_POST, SND_SOC_DPCM_TRIGGER_POST},
+ .dpcm_capture = 1,
+ },
+ [BXT_DPCM_AUDIO_REF_CP]
+ {
+ .name = "Bxt Audio Reference cap",
+ .stream_name = "refcap",
+ .cpu_dai_name = "Reference Pin",
+ .codec_name = "snd-soc-dummy",
+ .codec_dai_name = "snd-soc-dummy-dai",
+ .platform_name = "0000:00:0e.0",
+ .init = NULL,
+ .dpcm_capture = 1,
+ .nonatomic = 1,
+ .dynamic = 1,
+ },
+ [BXT_DPCM_AUDIO_HDMI1_PB]
+ {
+ .name = "Bxt HDMI Port1",
+ .stream_name = "Hdmi1",
+ .cpu_dai_name = "HDMI1 Pin",
+ .codec_name = "snd-soc-dummy",
+ .codec_dai_name = "snd-soc-dummy-dai",
+ .platform_name = "0000:00:0e.0",
+ .dpcm_playback = 1,
+ .init = NULL,
+ .nonatomic = 1,
+ .dynamic = 1,
+ },
+ [BXT_DPCM_AUDIO_HDMI2_PB]
+ {
+ .name = "Bxt HDMI Port2",
+ .stream_name = "Hdmi2",
+ .cpu_dai_name = "HDMI2 Pin",
+ .codec_name = "snd-soc-dummy",
+ .codec_dai_name = "snd-soc-dummy-dai",
+ .platform_name = "0000:00:0e.0",
+ .dpcm_playback = 1,
+ .init = NULL,
+ .nonatomic = 1,
+ .dynamic = 1,
+ },
+ [BXT_DPCM_AUDIO_HDMI3_PB]
+ {
+ .name = "Bxt HDMI Port3",
+ .stream_name = "Hdmi3",
+ .cpu_dai_name = "HDMI3 Pin",
+ .codec_name = "snd-soc-dummy",
+ .codec_dai_name = "snd-soc-dummy-dai",
+ .platform_name = "0000:00:0e.0",
+ .dpcm_playback = 1,
+ .init = NULL,
+ .nonatomic = 1,
+ .dynamic = 1,
+ },
+ /* Back End DAI links */
+ {
+ /* SSP5 - Codec */
+ .name = "SSP5-Codec",
+ .id = 0,
+ .cpu_dai_name = "SSP5 Pin",
+ .platform_name = "0000:00:0e.0",
+ .no_pcm = 1,
+ .codec_name = "i2c-INT343A:00",
+ .codec_dai_name = "rt298-aif1",
+ .init = broxton_rt298_codec_init,
+ .dai_fmt = SND_SOC_DAIFMT_DSP_A | SND_SOC_DAIFMT_NB_NF |
+ SND_SOC_DAIFMT_CBS_CFS,
+ .ignore_pmdown_time = 1,
+ .be_hw_params_fixup = broxton_ssp5_fixup,
+ .ops = &broxton_rt298_ops,
+ .dpcm_playback = 1,
+ .dpcm_capture = 1,
+ },
+ {
+ .name = "dmic01",
+ .id = 1,
+ .cpu_dai_name = "DMIC01 Pin",
+ .codec_name = "dmic-codec",
+ .codec_dai_name = "dmic-hifi",
+ .platform_name = "0000:00:0e.0",
+ .ignore_suspend = 1,
+ .dpcm_capture = 1,
+ .no_pcm = 1,
+ },
+ {
+ .name = "iDisp1",
+ .id = 3,
+ .cpu_dai_name = "iDisp1 Pin",
+ .codec_name = "ehdaudio0D2",
+ .codec_dai_name = "intel-hdmi-hifi1",
+ .platform_name = "0000:00:0e.0",
+ .init = broxton_hdmi_init,
+ .dpcm_playback = 1,
+ .no_pcm = 1,
+ },
+ {
+ .name = "iDisp2",
+ .id = 4,
+ .cpu_dai_name = "iDisp2 Pin",
+ .codec_name = "ehdaudio0D2",
+ .codec_dai_name = "intel-hdmi-hifi2",
+ .platform_name = "0000:00:0e.0",
+ .init = broxton_hdmi_init,
+ .dpcm_playback = 1,
+ .no_pcm = 1,
+ },
+ {
+ .name = "iDisp3",
+ .id = 5,
+ .cpu_dai_name = "iDisp3 Pin",
+ .codec_name = "ehdaudio0D2",
+ .codec_dai_name = "intel-hdmi-hifi3",
+ .platform_name = "0000:00:0e.0",
+ .init = broxton_hdmi_init,
+ .dpcm_playback = 1,
+ .no_pcm = 1,
+ },
+};
+
+/* broxton audio machine driver for SPT + RT298S */
+static struct snd_soc_card broxton_rt298 = {
+ .name = "broxton-rt298",
+ .owner = THIS_MODULE,
+ .dai_link = broxton_rt298_dais,
+ .num_links = ARRAY_SIZE(broxton_rt298_dais),
+ .controls = broxton_controls,
+ .num_controls = ARRAY_SIZE(broxton_controls),
+ .dapm_widgets = broxton_widgets,
+ .num_dapm_widgets = ARRAY_SIZE(broxton_widgets),
+ .dapm_routes = broxton_rt298_map,
+ .num_dapm_routes = ARRAY_SIZE(broxton_rt298_map),
+ .fully_routed = true,
+};
+
+static int broxton_audio_probe(struct platform_device *pdev)
+{
+ broxton_rt298.dev = &pdev->dev;
+
+ return devm_snd_soc_register_card(&pdev->dev, &broxton_rt298);
+}
+
+static struct platform_driver broxton_audio = {
+ .probe = broxton_audio_probe,
+ .driver = {
+ .name = "bxt_alc298s_i2s",
+ },
+};
+module_platform_driver(broxton_audio)
+
+/* Module information */
+MODULE_AUTHOR("Ramesh Babu <Ramesh.Babu@intel.com>");
+MODULE_AUTHOR("Senthilnathan Veppur <senthilnathanx.veppur@intel.com>");
+MODULE_DESCRIPTION("Intel SST Audio for Broxton");
+MODULE_LICENSE("GPL v2");
+MODULE_ALIAS("platform:bxt_alc298s_i2s");
diff --git a/sound/soc/intel/boards/bytcr_rt5640.c b/sound/soc/intel/boards/bytcr_rt5640.c
index 032a2e7..88efb62 100644
--- a/sound/soc/intel/boards/bytcr_rt5640.c
+++ b/sound/soc/intel/boards/bytcr_rt5640.c
@@ -304,7 +304,7 @@
/* back ends */
{
.name = "SSP2-Codec",
- .be_id = 1,
+ .id = 1,
.cpu_dai_name = "ssp2-port",
.platform_name = "sst-mfld-platform",
.no_pcm = 1,
diff --git a/sound/soc/intel/boards/bytcr_rt5651.c b/sound/soc/intel/boards/bytcr_rt5651.c
index 1c95ccc..35f591e 100644
--- a/sound/soc/intel/boards/bytcr_rt5651.c
+++ b/sound/soc/intel/boards/bytcr_rt5651.c
@@ -267,7 +267,7 @@
/* back ends */
{
.name = "SSP2-Codec",
- .be_id = 1,
+ .id = 1,
.cpu_dai_name = "ssp2-port",
.platform_name = "sst-mfld-platform",
.no_pcm = 1,
diff --git a/sound/soc/intel/boards/cht_bsw_max98090_ti.c b/sound/soc/intel/boards/cht_bsw_max98090_ti.c
index ac60b04..cdcced9 100644
--- a/sound/soc/intel/boards/cht_bsw_max98090_ti.c
+++ b/sound/soc/intel/boards/cht_bsw_max98090_ti.c
@@ -255,7 +255,7 @@
/* back ends */
{
.name = "SSP2-Codec",
- .be_id = 1,
+ .id = 1,
.cpu_dai_name = "ssp2-port",
.platform_name = "sst-mfld-platform",
.no_pcm = 1,
diff --git a/sound/soc/intel/boards/cht_bsw_rt5645.c b/sound/soc/intel/boards/cht_bsw_rt5645.c
index 3f2c1ea..d7ef292 100644
--- a/sound/soc/intel/boards/cht_bsw_rt5645.c
+++ b/sound/soc/intel/boards/cht_bsw_rt5645.c
@@ -295,7 +295,7 @@
/* back ends */
{
.name = "SSP2-Codec",
- .be_id = 1,
+ .id = 1,
.cpu_dai_name = "ssp2-port",
.platform_name = "sst-mfld-platform",
.no_pcm = 1,
diff --git a/sound/soc/intel/boards/cht_bsw_rt5672.c b/sound/soc/intel/boards/cht_bsw_rt5672.c
index 2e5347f..df9d254 100644
--- a/sound/soc/intel/boards/cht_bsw_rt5672.c
+++ b/sound/soc/intel/boards/cht_bsw_rt5672.c
@@ -273,7 +273,7 @@
{
/* SSP2 - Codec */
.name = "SSP2-Codec",
- .be_id = 1,
+ .id = 1,
.cpu_dai_name = "ssp2-port",
.platform_name = "sst-mfld-platform",
.no_pcm = 1,
diff --git a/sound/soc/intel/boards/haswell.c b/sound/soc/intel/boards/haswell.c
index 2255857..863f1d5 100644
--- a/sound/soc/intel/boards/haswell.c
+++ b/sound/soc/intel/boards/haswell.c
@@ -156,7 +156,7 @@
{
/* SSP0 - Codec */
.name = "Codec",
- .be_id = 0,
+ .id = 0,
.cpu_dai_name = "snd-soc-dummy-dai",
.platform_name = "snd-soc-dummy",
.no_pcm = 1,
diff --git a/sound/soc/intel/boards/skl_nau88l25_max98357a.c b/sound/soc/intel/boards/skl_nau88l25_max98357a.c
index 72176b7..d280865 100644
--- a/sound/soc/intel/boards/skl_nau88l25_max98357a.c
+++ b/sound/soc/intel/boards/skl_nau88l25_max98357a.c
@@ -30,6 +30,16 @@
static struct snd_soc_jack skylake_headset;
static struct snd_soc_card skylake_audio_card;
+struct skl_hdmi_pcm {
+ struct list_head head;
+ struct snd_soc_dai *codec_dai;
+ int device;
+};
+
+struct skl_nau8825_private {
+ struct list_head hdmi_pcm_list;
+};
+
enum {
SKL_DPCM_AUDIO_PB = 0,
SKL_DPCM_AUDIO_CP,
@@ -192,23 +202,56 @@
static int skylake_hdmi1_init(struct snd_soc_pcm_runtime *rtd)
{
+ struct skl_nau8825_private *ctx = snd_soc_card_get_drvdata(rtd->card);
struct snd_soc_dai *dai = rtd->codec_dai;
+ struct skl_hdmi_pcm *pcm;
- return hdac_hdmi_jack_init(dai, SKL_DPCM_AUDIO_HDMI1_PB);
+ pcm = devm_kzalloc(rtd->card->dev, sizeof(*pcm), GFP_KERNEL);
+ if (!pcm)
+ return -ENOMEM;
+
+ pcm->device = SKL_DPCM_AUDIO_HDMI1_PB;
+ pcm->codec_dai = dai;
+
+ list_add_tail(&pcm->head, &ctx->hdmi_pcm_list);
+
+ return 0;
}
static int skylake_hdmi2_init(struct snd_soc_pcm_runtime *rtd)
{
+ struct skl_nau8825_private *ctx = snd_soc_card_get_drvdata(rtd->card);
struct snd_soc_dai *dai = rtd->codec_dai;
+ struct skl_hdmi_pcm *pcm;
- return hdac_hdmi_jack_init(dai, SKL_DPCM_AUDIO_HDMI2_PB);
+ pcm = devm_kzalloc(rtd->card->dev, sizeof(*pcm), GFP_KERNEL);
+ if (!pcm)
+ return -ENOMEM;
+
+ pcm->device = SKL_DPCM_AUDIO_HDMI2_PB;
+ pcm->codec_dai = dai;
+
+ list_add_tail(&pcm->head, &ctx->hdmi_pcm_list);
+
+ return 0;
}
static int skylake_hdmi3_init(struct snd_soc_pcm_runtime *rtd)
{
+ struct skl_nau8825_private *ctx = snd_soc_card_get_drvdata(rtd->card);
struct snd_soc_dai *dai = rtd->codec_dai;
+ struct skl_hdmi_pcm *pcm;
- return hdac_hdmi_jack_init(dai, SKL_DPCM_AUDIO_HDMI3_PB);
+ pcm = devm_kzalloc(rtd->card->dev, sizeof(*pcm), GFP_KERNEL);
+ if (!pcm)
+ return -ENOMEM;
+
+ pcm->device = SKL_DPCM_AUDIO_HDMI3_PB;
+ pcm->codec_dai = dai;
+
+ list_add_tail(&pcm->head, &ctx->hdmi_pcm_list);
+
+ return 0;
}
static int skylake_nau8825_fe_init(struct snd_soc_pcm_runtime *rtd)
@@ -391,7 +434,6 @@
.platform_name = "0000:00:1f.3",
.init = NULL,
.dpcm_capture = 1,
- .ignore_suspend = 1,
.nonatomic = 1,
.dynamic = 1,
.ops = &skylaye_refcap_ops,
@@ -456,7 +498,7 @@
{
/* SSP0 - Codec */
.name = "SSP0-Codec",
- .be_id = 0,
+ .id = 0,
.cpu_dai_name = "SSP0 Pin",
.platform_name = "0000:00:1f.3",
.no_pcm = 1,
@@ -472,7 +514,7 @@
{
/* SSP1 - Codec */
.name = "SSP1-Codec",
- .be_id = 1,
+ .id = 1,
.cpu_dai_name = "SSP1 Pin",
.platform_name = "0000:00:1f.3",
.no_pcm = 1,
@@ -489,7 +531,7 @@
},
{
.name = "dmic01",
- .be_id = 2,
+ .id = 2,
.cpu_dai_name = "DMIC01 Pin",
.codec_name = "dmic-codec",
.codec_dai_name = "dmic-hifi",
@@ -501,7 +543,7 @@
},
{
.name = "iDisp1",
- .be_id = 3,
+ .id = 3,
.cpu_dai_name = "iDisp1 Pin",
.codec_name = "ehdaudio0D2",
.codec_dai_name = "intel-hdmi-hifi1",
@@ -512,7 +554,7 @@
},
{
.name = "iDisp2",
- .be_id = 4,
+ .id = 4,
.cpu_dai_name = "iDisp2 Pin",
.codec_name = "ehdaudio0D2",
.codec_dai_name = "intel-hdmi-hifi2",
@@ -523,7 +565,7 @@
},
{
.name = "iDisp3",
- .be_id = 5,
+ .id = 5,
.cpu_dai_name = "iDisp3 Pin",
.codec_name = "ehdaudio0D2",
.codec_dai_name = "intel-hdmi-hifi3",
@@ -534,6 +576,21 @@
},
};
+static int skylake_card_late_probe(struct snd_soc_card *card)
+{
+ struct skl_nau8825_private *ctx = snd_soc_card_get_drvdata(card);
+ struct skl_hdmi_pcm *pcm;
+ int err;
+
+ list_for_each_entry(pcm, &ctx->hdmi_pcm_list, head) {
+ err = hdac_hdmi_jack_init(pcm->codec_dai, pcm->device);
+ if (err < 0)
+ return err;
+ }
+
+ return 0;
+}
+
/* skylake audio machine driver for SPT + NAU88L25 */
static struct snd_soc_card skylake_audio_card = {
.name = "sklnau8825max",
@@ -547,11 +604,21 @@
.dapm_routes = skylake_map,
.num_dapm_routes = ARRAY_SIZE(skylake_map),
.fully_routed = true,
+ .late_probe = skylake_card_late_probe,
};
static int skylake_audio_probe(struct platform_device *pdev)
{
+ struct skl_nau8825_private *ctx;
+
+ ctx = devm_kzalloc(&pdev->dev, sizeof(*ctx), GFP_ATOMIC);
+ if (!ctx)
+ return -ENOMEM;
+
+ INIT_LIST_HEAD(&ctx->hdmi_pcm_list);
+
skylake_audio_card.dev = &pdev->dev;
+ snd_soc_card_set_drvdata(&skylake_audio_card, ctx);
return devm_snd_soc_register_card(&pdev->dev, &skylake_audio_card);
}
diff --git a/sound/soc/intel/boards/skl_nau88l25_ssm4567.c b/sound/soc/intel/boards/skl_nau88l25_ssm4567.c
index 5f1ca99..e19aa99 100644
--- a/sound/soc/intel/boards/skl_nau88l25_ssm4567.c
+++ b/sound/soc/intel/boards/skl_nau88l25_ssm4567.c
@@ -34,6 +34,15 @@
static struct snd_soc_jack skylake_headset;
static struct snd_soc_card skylake_audio_card;
+struct skl_hdmi_pcm {
+ struct list_head head;
+ struct snd_soc_dai *codec_dai;
+ int device;
+};
+
+struct skl_nau88125_private {
+ struct list_head hdmi_pcm_list;
+};
enum {
SKL_DPCM_AUDIO_PB = 0,
SKL_DPCM_AUDIO_CP,
@@ -222,24 +231,57 @@
static int skylake_hdmi1_init(struct snd_soc_pcm_runtime *rtd)
{
+ struct skl_nau88125_private *ctx = snd_soc_card_get_drvdata(rtd->card);
struct snd_soc_dai *dai = rtd->codec_dai;
+ struct skl_hdmi_pcm *pcm;
- return hdac_hdmi_jack_init(dai, SKL_DPCM_AUDIO_HDMI1_PB);
+ pcm = devm_kzalloc(rtd->card->dev, sizeof(*pcm), GFP_KERNEL);
+ if (!pcm)
+ return -ENOMEM;
+
+ pcm->device = SKL_DPCM_AUDIO_HDMI1_PB;
+ pcm->codec_dai = dai;
+
+ list_add_tail(&pcm->head, &ctx->hdmi_pcm_list);
+
+ return 0;
}
static int skylake_hdmi2_init(struct snd_soc_pcm_runtime *rtd)
{
+ struct skl_nau88125_private *ctx = snd_soc_card_get_drvdata(rtd->card);
struct snd_soc_dai *dai = rtd->codec_dai;
+ struct skl_hdmi_pcm *pcm;
- return hdac_hdmi_jack_init(dai, SKL_DPCM_AUDIO_HDMI2_PB);
+ pcm = devm_kzalloc(rtd->card->dev, sizeof(*pcm), GFP_KERNEL);
+ if (!pcm)
+ return -ENOMEM;
+
+ pcm->device = SKL_DPCM_AUDIO_HDMI2_PB;
+ pcm->codec_dai = dai;
+
+ list_add_tail(&pcm->head, &ctx->hdmi_pcm_list);
+
+ return 0;
}
static int skylake_hdmi3_init(struct snd_soc_pcm_runtime *rtd)
{
+ struct skl_nau88125_private *ctx = snd_soc_card_get_drvdata(rtd->card);
struct snd_soc_dai *dai = rtd->codec_dai;
+ struct skl_hdmi_pcm *pcm;
- return hdac_hdmi_jack_init(dai, SKL_DPCM_AUDIO_HDMI3_PB);
+ pcm = devm_kzalloc(rtd->card->dev, sizeof(*pcm), GFP_KERNEL);
+ if (!pcm)
+ return -ENOMEM;
+
+ pcm->device = SKL_DPCM_AUDIO_HDMI3_PB;
+ pcm->codec_dai = dai;
+
+ list_add_tail(&pcm->head, &ctx->hdmi_pcm_list);
+
+ return 0;
}
static int skylake_nau8825_fe_init(struct snd_soc_pcm_runtime *rtd)
@@ -440,7 +482,6 @@
.platform_name = "0000:00:1f.3",
.init = NULL,
.dpcm_capture = 1,
- .ignore_suspend = 1,
.nonatomic = 1,
.dynamic = 1,
.ops = &skylaye_refcap_ops,
@@ -505,7 +546,7 @@
{
/* SSP0 - Codec */
.name = "SSP0-Codec",
- .be_id = 0,
+ .id = 0,
.cpu_dai_name = "SSP0 Pin",
.platform_name = "0000:00:1f.3",
.no_pcm = 1,
@@ -523,7 +564,7 @@
{
/* SSP1 - Codec */
.name = "SSP1-Codec",
- .be_id = 1,
+ .id = 1,
.cpu_dai_name = "SSP1 Pin",
.platform_name = "0000:00:1f.3",
.no_pcm = 1,
@@ -540,7 +581,7 @@
},
{
.name = "dmic01",
- .be_id = 2,
+ .id = 2,
.cpu_dai_name = "DMIC01 Pin",
.codec_name = "dmic-codec",
.codec_dai_name = "dmic-hifi",
@@ -552,7 +593,7 @@
},
{
.name = "iDisp1",
- .be_id = 3,
+ .id = 3,
.cpu_dai_name = "iDisp1 Pin",
.codec_name = "ehdaudio0D2",
.codec_dai_name = "intel-hdmi-hifi1",
@@ -563,7 +604,7 @@
},
{
.name = "iDisp2",
- .be_id = 4,
+ .id = 4,
.cpu_dai_name = "iDisp2 Pin",
.codec_name = "ehdaudio0D2",
.codec_dai_name = "intel-hdmi-hifi2",
@@ -574,7 +615,7 @@
},
{
.name = "iDisp3",
- .be_id = 5,
+ .id = 5,
.cpu_dai_name = "iDisp3 Pin",
.codec_name = "ehdaudio0D2",
.codec_dai_name = "intel-hdmi-hifi3",
@@ -585,6 +626,21 @@
},
};
+static int skylake_card_late_probe(struct snd_soc_card *card)
+{
+ struct skl_nau88125_private *ctx = snd_soc_card_get_drvdata(card);
+ struct skl_hdmi_pcm *pcm;
+ int err;
+
+ list_for_each_entry(pcm, &ctx->hdmi_pcm_list, head) {
+ err = hdac_hdmi_jack_init(pcm->codec_dai, pcm->device);
+ if (err < 0)
+ return err;
+ }
+
+ return 0;
+}
+
/* skylake audio machine driver for SPT + NAU88L25 */
static struct snd_soc_card skylake_audio_card = {
.name = "sklnau8825adi",
@@ -600,11 +656,21 @@
.codec_conf = ssm4567_codec_conf,
.num_configs = ARRAY_SIZE(ssm4567_codec_conf),
.fully_routed = true,
+ .late_probe = skylake_card_late_probe,
};
static int skylake_audio_probe(struct platform_device *pdev)
{
+ struct skl_nau88125_private *ctx;
+
+ ctx = devm_kzalloc(&pdev->dev, sizeof(*ctx), GFP_ATOMIC);
+ if (!ctx)
+ return -ENOMEM;
+
+ INIT_LIST_HEAD(&ctx->hdmi_pcm_list);
+
skylake_audio_card.dev = &pdev->dev;
+ snd_soc_card_set_drvdata(&skylake_audio_card, ctx);
return devm_snd_soc_register_card(&pdev->dev, &skylake_audio_card);
}
diff --git a/sound/soc/intel/boards/skl_rt286.c b/sound/soc/intel/boards/skl_rt286.c
index 2016397a..426b482 100644
--- a/sound/soc/intel/boards/skl_rt286.c
+++ b/sound/soc/intel/boards/skl_rt286.c
@@ -30,6 +30,16 @@
static struct snd_soc_jack skylake_headset;
+struct skl_hdmi_pcm {
+ struct list_head head;
+ struct snd_soc_dai *codec_dai;
+ int device;
+};
+
+struct skl_rt286_private {
+ struct list_head hdmi_pcm_list;
+};
+
enum {
SKL_DPCM_AUDIO_PB = 0,
SKL_DPCM_AUDIO_CP,
@@ -142,9 +152,20 @@
static int skylake_hdmi_init(struct snd_soc_pcm_runtime *rtd)
{
+ struct skl_rt286_private *ctx = snd_soc_card_get_drvdata(rtd->card);
struct snd_soc_dai *dai = rtd->codec_dai;
+ struct skl_hdmi_pcm *pcm;
- return hdac_hdmi_jack_init(dai, SKL_DPCM_AUDIO_HDMI1_PB + dai->id);
+ pcm = devm_kzalloc(rtd->card->dev, sizeof(*pcm), GFP_KERNEL);
+ if (!pcm)
+ return -ENOMEM;
+
+ pcm->device = SKL_DPCM_AUDIO_HDMI1_PB + dai->id;
+ pcm->codec_dai = dai;
+
+ list_add_tail(&pcm->head, &ctx->hdmi_pcm_list);
+
+ return 0;
}
static unsigned int rates[] = {
@@ -317,7 +338,6 @@
.platform_name = "0000:00:1f.3",
.init = NULL,
.dpcm_capture = 1,
- .ignore_suspend = 1,
.nonatomic = 1,
.dynamic = 1,
},
@@ -375,7 +395,7 @@
{
/* SSP0 - Codec */
.name = "SSP0-Codec",
- .be_id = 0,
+ .id = 0,
.cpu_dai_name = "SSP0 Pin",
.platform_name = "0000:00:1f.3",
.no_pcm = 1,
@@ -393,7 +413,7 @@
},
{
.name = "dmic01",
- .be_id = 1,
+ .id = 1,
.cpu_dai_name = "DMIC01 Pin",
.codec_name = "dmic-codec",
.codec_dai_name = "dmic-hifi",
@@ -405,7 +425,7 @@
},
{
.name = "iDisp1",
- .be_id = 2,
+ .id = 2,
.cpu_dai_name = "iDisp1 Pin",
.codec_name = "ehdaudio0D2",
.codec_dai_name = "intel-hdmi-hifi1",
@@ -416,7 +436,7 @@
},
{
.name = "iDisp2",
- .be_id = 3,
+ .id = 3,
.cpu_dai_name = "iDisp2 Pin",
.codec_name = "ehdaudio0D2",
.codec_dai_name = "intel-hdmi-hifi2",
@@ -427,7 +447,7 @@
},
{
.name = "iDisp3",
- .be_id = 4,
+ .id = 4,
.cpu_dai_name = "iDisp3 Pin",
.codec_name = "ehdaudio0D2",
.codec_dai_name = "intel-hdmi-hifi3",
@@ -438,6 +458,21 @@
},
};
+static int skylake_card_late_probe(struct snd_soc_card *card)
+{
+ struct skl_rt286_private *ctx = snd_soc_card_get_drvdata(card);
+ struct skl_hdmi_pcm *pcm;
+ int err;
+
+ list_for_each_entry(pcm, &ctx->hdmi_pcm_list, head) {
+ err = hdac_hdmi_jack_init(pcm->codec_dai, pcm->device);
+ if (err < 0)
+ return err;
+ }
+
+ return 0;
+}
+
/* skylake audio machine driver for SPT + RT286S */
static struct snd_soc_card skylake_rt286 = {
.name = "skylake-rt286",
@@ -451,11 +486,21 @@
.dapm_routes = skylake_rt286_map,
.num_dapm_routes = ARRAY_SIZE(skylake_rt286_map),
.fully_routed = true,
+ .late_probe = skylake_card_late_probe,
};
static int skylake_audio_probe(struct platform_device *pdev)
{
+ struct skl_rt286_private *ctx;
+
+ ctx = devm_kzalloc(&pdev->dev, sizeof(*ctx), GFP_ATOMIC);
+ if (!ctx)
+ return -ENOMEM;
+
+ INIT_LIST_HEAD(&ctx->hdmi_pcm_list);
+
skylake_rt286.dev = &pdev->dev;
+ snd_soc_card_set_drvdata(&skylake_rt286, ctx);
return devm_snd_soc_register_card(&pdev->dev, &skylake_rt286);
}
diff --git a/sound/soc/intel/common/sst-acpi.h b/sound/soc/intel/common/sst-acpi.h
index 4dcfb7e..8398cb2 100644
--- a/sound/soc/intel/common/sst-acpi.h
+++ b/sound/soc/intel/common/sst-acpi.h
@@ -12,10 +12,19 @@
*
*/
+#include <linux/kconfig.h>
+#include <linux/stddef.h>
#include <linux/acpi.h>
/* translation fron HID to I2C name, needed for DAI codec_name */
+#if IS_ENABLED(CONFIG_ACPI)
const char *sst_acpi_find_name_from_hid(const u8 hid[ACPI_ID_LEN]);
+#else
+inline const char *sst_acpi_find_name_from_hid(const u8 hid[ACPI_ID_LEN])
+{
+ return NULL;
+}
+#endif
/* acpi match */
struct sst_acpi_mach *sst_acpi_find_machine(struct sst_acpi_mach *machines);
diff --git a/sound/soc/intel/common/sst-firmware.c b/sound/soc/intel/common/sst-firmware.c
index ef4881e..2599352 100644
--- a/sound/soc/intel/common/sst-firmware.c
+++ b/sound/soc/intel/common/sst-firmware.c
@@ -203,7 +203,7 @@
chip->dev = dev;
- err = dw_dma_probe(chip, NULL);
+ err = dw_dma_probe(chip);
if (err)
return ERR_PTR(err);
diff --git a/sound/soc/intel/haswell/sst-haswell-pcm.c b/sound/soc/intel/haswell/sst-haswell-pcm.c
index 1aa819c..994256b 100644
--- a/sound/soc/intel/haswell/sst-haswell-pcm.c
+++ b/sound/soc/intel/haswell/sst-haswell-pcm.c
@@ -445,7 +445,7 @@
pages = snd_sgbuf_aligned_pages(size);
- dev_dbg(rtd->dev, "generating page table for %p size 0x%zu pages %d\n",
+ dev_dbg(rtd->dev, "generating page table for %p size 0x%zx pages %d\n",
dma_area, size, pages);
for (i = 0; i < pages; i++) {
diff --git a/sound/soc/intel/skylake/Makefile b/sound/soc/intel/skylake/Makefile
index 914b6da..c28f5d0 100644
--- a/sound/soc/intel/skylake/Makefile
+++ b/sound/soc/intel/skylake/Makefile
@@ -5,6 +5,6 @@
# Skylake IPC Support
snd-soc-skl-ipc-objs := skl-sst-ipc.o skl-sst-dsp.o skl-sst-cldma.o \
- skl-sst.o
+ skl-sst.o bxt-sst.o
obj-$(CONFIG_SND_SOC_INTEL_SKYLAKE) += snd-soc-skl-ipc.o
diff --git a/sound/soc/intel/skylake/bxt-sst.c b/sound/soc/intel/skylake/bxt-sst.c
new file mode 100644
index 0000000..965ce40
--- /dev/null
+++ b/sound/soc/intel/skylake/bxt-sst.c
@@ -0,0 +1,328 @@
+/*
+ * bxt-sst.c - DSP library functions for BXT platform
+ *
+ * Copyright (C) 2015-16 Intel Corp
+ * Author:Rafal Redzimski <rafal.f.redzimski@intel.com>
+ * Jeeja KP <jeeja.kp@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ */
+
+#include <linux/module.h>
+#include <linux/delay.h>
+#include <linux/firmware.h>
+#include <linux/device.h>
+
+#include "../common/sst-dsp.h"
+#include "../common/sst-dsp-priv.h"
+#include "skl-sst-ipc.h"
+
+#define BXT_BASEFW_TIMEOUT 3000
+#define BXT_INIT_TIMEOUT 500
+#define BXT_IPC_PURGE_FW 0x01004000
+
+#define BXT_ROM_INIT 0x5
+#define BXT_ADSP_SRAM0_BASE 0x80000
+
+/* Firmware status window */
+#define BXT_ADSP_FW_STATUS BXT_ADSP_SRAM0_BASE
+#define BXT_ADSP_ERROR_CODE (BXT_ADSP_FW_STATUS + 0x4)
+
+#define BXT_ADSP_SRAM1_BASE 0xA0000
+
+static unsigned int bxt_get_errorcode(struct sst_dsp *ctx)
+{
+ return sst_dsp_shim_read(ctx, BXT_ADSP_ERROR_CODE);
+}
+
+static int sst_bxt_prepare_fw(struct sst_dsp *ctx,
+ const void *fwdata, u32 fwsize)
+{
+ int stream_tag, ret, i;
+ u32 reg;
+
+ stream_tag = ctx->dsp_ops.prepare(ctx->dev, 0x40, fwsize, &ctx->dmab);
+ if (stream_tag < 0) {
+ dev_err(ctx->dev, "Failed to prepare DMA FW loading err: %x\n",
+ stream_tag);
+ return stream_tag;
+ }
+
+ ctx->dsp_ops.stream_tag = stream_tag;
+ memcpy(ctx->dmab.area, fwdata, fwsize);
+
+ /* Purge FW request */
+ sst_dsp_shim_write(ctx, SKL_ADSP_REG_HIPCI, SKL_ADSP_REG_HIPCI_BUSY |
+ BXT_IPC_PURGE_FW | (stream_tag - 1));
+
+ ret = skl_dsp_enable_core(ctx);
+ if (ret < 0) {
+ dev_err(ctx->dev, "Boot dsp core failed ret: %d\n", ret);
+ ret = -EIO;
+ goto base_fw_load_failed;
+ }
+
+ for (i = BXT_INIT_TIMEOUT; i > 0; --i) {
+ reg = sst_dsp_shim_read(ctx, SKL_ADSP_REG_HIPCIE);
+
+ if (reg & SKL_ADSP_REG_HIPCIE_DONE) {
+ sst_dsp_shim_update_bits_forced(ctx,
+ SKL_ADSP_REG_HIPCIE,
+ SKL_ADSP_REG_HIPCIE_DONE,
+ SKL_ADSP_REG_HIPCIE_DONE);
+ break;
+ }
+ mdelay(1);
+ }
+ if (!i) {
+ dev_info(ctx->dev, "Waiting for HIPCIE done, reg: 0x%x\n", reg);
+ sst_dsp_shim_update_bits(ctx, SKL_ADSP_REG_HIPCIE,
+ SKL_ADSP_REG_HIPCIE_DONE,
+ SKL_ADSP_REG_HIPCIE_DONE);
+ }
+
+ /* enable Interrupt */
+ skl_ipc_int_enable(ctx);
+ skl_ipc_op_int_enable(ctx);
+
+ for (i = BXT_INIT_TIMEOUT; i > 0; --i) {
+ if (SKL_FW_INIT ==
+ (sst_dsp_shim_read(ctx, BXT_ADSP_FW_STATUS) &
+ SKL_FW_STS_MASK)) {
+
+ dev_info(ctx->dev, "ROM loaded, continue FW loading\n");
+ break;
+ }
+ mdelay(1);
+ }
+ if (!i) {
+ dev_err(ctx->dev, "Timeout for ROM init, HIPCIE: 0x%x\n", reg);
+ ret = -EIO;
+ goto base_fw_load_failed;
+ }
+
+ return ret;
+
+base_fw_load_failed:
+ ctx->dsp_ops.cleanup(ctx->dev, &ctx->dmab, stream_tag);
+ skl_dsp_disable_core(ctx);
+ return ret;
+}
+
+static int sst_transfer_fw_host_dma(struct sst_dsp *ctx)
+{
+ int ret;
+
+ ctx->dsp_ops.trigger(ctx->dev, true, ctx->dsp_ops.stream_tag);
+ ret = sst_dsp_register_poll(ctx, BXT_ADSP_FW_STATUS, SKL_FW_STS_MASK,
+ BXT_ROM_INIT, BXT_BASEFW_TIMEOUT, "Firmware boot");
+
+ ctx->dsp_ops.trigger(ctx->dev, false, ctx->dsp_ops.stream_tag);
+ ctx->dsp_ops.cleanup(ctx->dev, &ctx->dmab, ctx->dsp_ops.stream_tag);
+
+ return ret;
+}
+
+static int bxt_load_base_firmware(struct sst_dsp *ctx)
+{
+ const struct firmware *fw = NULL;
+ struct skl_sst *skl = ctx->thread_context;
+ int ret;
+
+ ret = request_firmware(&fw, ctx->fw_name, ctx->dev);
+ if (ret < 0) {
+ dev_err(ctx->dev, "Request firmware failed %d\n", ret);
+ goto sst_load_base_firmware_failed;
+ }
+
+ ret = sst_bxt_prepare_fw(ctx, fw->data, fw->size);
+ /* Retry Enabling core and ROM load. Retry seemed to help */
+ if (ret < 0) {
+ ret = sst_bxt_prepare_fw(ctx, fw->data, fw->size);
+ if (ret < 0) {
+ dev_err(ctx->dev, "Core En/ROM load fail:%d\n", ret);
+ goto sst_load_base_firmware_failed;
+ }
+ }
+
+ ret = sst_transfer_fw_host_dma(ctx);
+ if (ret < 0) {
+ dev_err(ctx->dev, "Transfer firmware failed %d\n", ret);
+ dev_info(ctx->dev, "Error code=0x%x: FW status=0x%x\n",
+ sst_dsp_shim_read(ctx, BXT_ADSP_ERROR_CODE),
+ sst_dsp_shim_read(ctx, BXT_ADSP_FW_STATUS));
+
+ skl_dsp_disable_core(ctx);
+ } else {
+ dev_dbg(ctx->dev, "Firmware download successful\n");
+ ret = wait_event_timeout(skl->boot_wait, skl->boot_complete,
+ msecs_to_jiffies(SKL_IPC_BOOT_MSECS));
+ if (ret == 0) {
+ dev_err(ctx->dev, "DSP boot fail, FW Ready timeout\n");
+ skl_dsp_disable_core(ctx);
+ ret = -EIO;
+ } else {
+ skl_dsp_set_state_locked(ctx, SKL_DSP_RUNNING);
+ ret = 0;
+ }
+ }
+
+sst_load_base_firmware_failed:
+ release_firmware(fw);
+ return ret;
+}
+
+static int bxt_set_dsp_D0(struct sst_dsp *ctx)
+{
+ struct skl_sst *skl = ctx->thread_context;
+ int ret;
+
+ skl->boot_complete = false;
+
+ ret = skl_dsp_enable_core(ctx);
+ if (ret < 0) {
+ dev_err(ctx->dev, "enable dsp core failed ret: %d\n", ret);
+ return ret;
+ }
+
+ /* enable interrupt */
+ skl_ipc_int_enable(ctx);
+ skl_ipc_op_int_enable(ctx);
+
+ ret = wait_event_timeout(skl->boot_wait, skl->boot_complete,
+ msecs_to_jiffies(SKL_IPC_BOOT_MSECS));
+ if (ret == 0) {
+ dev_err(ctx->dev, "ipc: error DSP boot timeout\n");
+ dev_err(ctx->dev, "Error code=0x%x: FW status=0x%x\n",
+ sst_dsp_shim_read(ctx, BXT_ADSP_ERROR_CODE),
+ sst_dsp_shim_read(ctx, BXT_ADSP_FW_STATUS));
+ return -EIO;
+ }
+
+ skl_dsp_set_state_locked(ctx, SKL_DSP_RUNNING);
+ return 0;
+}
+
+static int bxt_set_dsp_D3(struct sst_dsp *ctx)
+{
+ struct skl_ipc_dxstate_info dx;
+ struct skl_sst *skl = ctx->thread_context;
+ int ret = 0;
+
+ if (!is_skl_dsp_running(ctx))
+ return ret;
+
+ dx.core_mask = SKL_DSP_CORE0_MASK;
+ dx.dx_mask = SKL_IPC_D3_MASK;
+
+ ret = skl_ipc_set_dx(&skl->ipc, SKL_INSTANCE_ID,
+ SKL_BASE_FW_MODULE_ID, &dx);
+ if (ret < 0) {
+ dev_err(ctx->dev, "Failed to set DSP to D3 state: %d\n", ret);
+ return ret;
+ }
+
+ ret = skl_dsp_disable_core(ctx);
+ if (ret < 0) {
+ dev_err(ctx->dev, "disbale dsp core failed: %d\n", ret);
+ ret = -EIO;
+ }
+
+ skl_dsp_set_state_locked(ctx, SKL_DSP_RESET);
+ return 0;
+}
+
+static struct skl_dsp_fw_ops bxt_fw_ops = {
+ .set_state_D0 = bxt_set_dsp_D0,
+ .set_state_D3 = bxt_set_dsp_D3,
+ .load_fw = bxt_load_base_firmware,
+ .get_fw_errcode = bxt_get_errorcode,
+};
+
+static struct sst_ops skl_ops = {
+ .irq_handler = skl_dsp_sst_interrupt,
+ .write = sst_shim32_write,
+ .read = sst_shim32_read,
+ .ram_read = sst_memcpy_fromio_32,
+ .ram_write = sst_memcpy_toio_32,
+ .free = skl_dsp_free,
+};
+
+static struct sst_dsp_device skl_dev = {
+ .thread = skl_dsp_irq_thread_handler,
+ .ops = &skl_ops,
+};
+
+int bxt_sst_dsp_init(struct device *dev, void __iomem *mmio_base, int irq,
+ const char *fw_name, struct skl_dsp_loader_ops dsp_ops,
+ struct skl_sst **dsp)
+{
+ struct skl_sst *skl;
+ struct sst_dsp *sst;
+ int ret;
+
+ skl = devm_kzalloc(dev, sizeof(*skl), GFP_KERNEL);
+ if (skl == NULL)
+ return -ENOMEM;
+
+ skl->dev = dev;
+ skl_dev.thread_context = skl;
+
+ skl->dsp = skl_dsp_ctx_init(dev, &skl_dev, irq);
+ if (!skl->dsp) {
+ dev_err(skl->dev, "skl_dsp_ctx_init failed\n");
+ return -ENODEV;
+ }
+
+ sst = skl->dsp;
+ sst->fw_name = fw_name;
+ sst->dsp_ops = dsp_ops;
+ sst->fw_ops = bxt_fw_ops;
+ sst->addr.lpe = mmio_base;
+ sst->addr.shim = mmio_base;
+
+ sst_dsp_mailbox_init(sst, (BXT_ADSP_SRAM0_BASE + SKL_ADSP_W0_STAT_SZ),
+ SKL_ADSP_W0_UP_SZ, BXT_ADSP_SRAM1_BASE, SKL_ADSP_W1_SZ);
+
+ ret = skl_ipc_init(dev, skl);
+ if (ret)
+ return ret;
+
+ skl->boot_complete = false;
+ init_waitqueue_head(&skl->boot_wait);
+
+ ret = sst->fw_ops.load_fw(sst);
+ if (ret < 0) {
+ dev_err(dev, "Load base fw failed: %x", ret);
+ return ret;
+ }
+
+ if (dsp)
+ *dsp = skl;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(bxt_sst_dsp_init);
+
+
+void bxt_sst_dsp_cleanup(struct device *dev, struct skl_sst *ctx)
+{
+ skl_ipc_free(&ctx->ipc);
+ ctx->dsp->cl_dev.ops.cl_cleanup_controller(ctx->dsp);
+
+ if (ctx->dsp->addr.lpe)
+ iounmap(ctx->dsp->addr.lpe);
+
+ ctx->dsp->ops->free(ctx->dsp);
+}
+EXPORT_SYMBOL_GPL(bxt_sst_dsp_cleanup);
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Intel Broxton IPC driver");
diff --git a/sound/soc/intel/skylake/skl-messages.c b/sound/soc/intel/skylake/skl-messages.c
index 79c5089..226db84 100644
--- a/sound/soc/intel/skylake/skl-messages.c
+++ b/sound/soc/intel/skylake/skl-messages.c
@@ -72,6 +72,105 @@
skl_ipc_set_large_config(&ctx->ipc, &msg, (u32 *)&mask);
}
+static int skl_dsp_setup_spib(struct device *dev, unsigned int size,
+ int stream_tag, int enable)
+{
+ struct hdac_ext_bus *ebus = dev_get_drvdata(dev);
+ struct hdac_bus *bus = ebus_to_hbus(ebus);
+ struct hdac_stream *stream = snd_hdac_get_stream(bus,
+ SNDRV_PCM_STREAM_PLAYBACK, stream_tag);
+ struct hdac_ext_stream *estream;
+
+ if (!stream)
+ return -EINVAL;
+
+ estream = stream_to_hdac_ext_stream(stream);
+ /* enable/disable SPIB for this hdac stream */
+ snd_hdac_ext_stream_spbcap_enable(ebus, enable, stream->index);
+
+ /* set the spib value */
+ snd_hdac_ext_stream_set_spib(ebus, estream, size);
+
+ return 0;
+}
+
+static int skl_dsp_prepare(struct device *dev, unsigned int format,
+ unsigned int size, struct snd_dma_buffer *dmab)
+{
+ struct hdac_ext_bus *ebus = dev_get_drvdata(dev);
+ struct hdac_bus *bus = ebus_to_hbus(ebus);
+ struct hdac_ext_stream *estream;
+ struct hdac_stream *stream;
+ struct snd_pcm_substream substream;
+ int ret;
+
+ if (!bus)
+ return -ENODEV;
+
+ memset(&substream, 0, sizeof(substream));
+ substream.stream = SNDRV_PCM_STREAM_PLAYBACK;
+
+ estream = snd_hdac_ext_stream_assign(ebus, &substream,
+ HDAC_EXT_STREAM_TYPE_HOST);
+ if (!estream)
+ return -ENODEV;
+
+ stream = hdac_stream(estream);
+
+ /* assign decouple host dma channel */
+ ret = snd_hdac_dsp_prepare(stream, format, size, dmab);
+ if (ret < 0)
+ return ret;
+
+ skl_dsp_setup_spib(dev, size, stream->stream_tag, true);
+
+ return stream->stream_tag;
+}
+
+static int skl_dsp_trigger(struct device *dev, bool start, int stream_tag)
+{
+ struct hdac_ext_bus *ebus = dev_get_drvdata(dev);
+ struct hdac_stream *stream;
+ struct hdac_bus *bus = ebus_to_hbus(ebus);
+
+ if (!bus)
+ return -ENODEV;
+
+ stream = snd_hdac_get_stream(bus,
+ SNDRV_PCM_STREAM_PLAYBACK, stream_tag);
+ if (!stream)
+ return -EINVAL;
+
+ snd_hdac_dsp_trigger(stream, start);
+
+ return 0;
+}
+
+static int skl_dsp_cleanup(struct device *dev,
+ struct snd_dma_buffer *dmab, int stream_tag)
+{
+ struct hdac_ext_bus *ebus = dev_get_drvdata(dev);
+ struct hdac_stream *stream;
+ struct hdac_ext_stream *estream;
+ struct hdac_bus *bus = ebus_to_hbus(ebus);
+
+ if (!bus)
+ return -ENODEV;
+
+ stream = snd_hdac_get_stream(bus,
+ SNDRV_PCM_STREAM_PLAYBACK, stream_tag);
+ if (!stream)
+ return -EINVAL;
+
+ estream = stream_to_hdac_ext_stream(stream);
+ skl_dsp_setup_spib(dev, 0, stream_tag, false);
+ snd_hdac_ext_stream_release(estream, HDAC_EXT_STREAM_TYPE_HOST);
+
+ snd_hdac_dsp_cleanup(stream, dmab);
+
+ return 0;
+}
+
static struct skl_dsp_loader_ops skl_get_loader_ops(void)
{
struct skl_dsp_loader_ops loader_ops;
@@ -84,6 +183,21 @@
return loader_ops;
};
+static struct skl_dsp_loader_ops bxt_get_loader_ops(void)
+{
+ struct skl_dsp_loader_ops loader_ops;
+
+ memset(&loader_ops, 0, sizeof(loader_ops));
+
+ loader_ops.alloc_dma_buf = skl_alloc_dma_buf;
+ loader_ops.free_dma_buf = skl_free_dma_buf;
+ loader_ops.prepare = skl_dsp_prepare;
+ loader_ops.trigger = skl_dsp_trigger;
+ loader_ops.cleanup = skl_dsp_cleanup;
+
+ return loader_ops;
+};
+
static const struct skl_dsp_ops dsp_ops[] = {
{
.id = 0x9d70,
@@ -91,6 +205,12 @@
.init = skl_sst_dsp_init,
.cleanup = skl_sst_dsp_cleanup
},
+ {
+ .id = 0x5a98,
+ .loader_ops = bxt_get_loader_ops,
+ .init = bxt_sst_dsp_init,
+ .cleanup = bxt_sst_dsp_cleanup
+ },
};
static int skl_get_dsp_ops(int pci_id)
@@ -744,7 +864,7 @@
return ret;
}
mconfig->m_state = SKL_MODULE_INIT_DONE;
-
+ kfree(param_data);
return ret;
}
diff --git a/sound/soc/intel/skylake/skl-nhlt.c b/sound/soc/intel/skylake/skl-nhlt.c
index 14d1916e..7d73648 100644
--- a/sound/soc/intel/skylake/skl-nhlt.c
+++ b/sound/soc/intel/skylake/skl-nhlt.c
@@ -25,11 +25,12 @@
#define DSDT_NHLT_PATH "\\_SB.PCI0.HDAS"
-void *skl_nhlt_init(struct device *dev)
+struct nhlt_acpi_table *skl_nhlt_init(struct device *dev)
{
acpi_handle handle;
union acpi_object *obj;
struct nhlt_resource_desc *nhlt_ptr = NULL;
+ struct nhlt_acpi_table *nhlt_table = NULL;
if (ACPI_FAILURE(acpi_get_handle(NULL, DSDT_NHLT_PATH, &handle))) {
dev_err(dev, "Requested NHLT device not found\n");
@@ -39,18 +40,20 @@
obj = acpi_evaluate_dsm(handle, OSC_UUID, 1, 1, NULL);
if (obj && obj->type == ACPI_TYPE_BUFFER) {
nhlt_ptr = (struct nhlt_resource_desc *)obj->buffer.pointer;
-
- return memremap(nhlt_ptr->min_addr, nhlt_ptr->length,
+ nhlt_table = (struct nhlt_acpi_table *)
+ memremap(nhlt_ptr->min_addr, nhlt_ptr->length,
MEMREMAP_WB);
+ ACPI_FREE(obj);
+ return nhlt_table;
}
dev_err(dev, "device specific method to extract NHLT blob failed\n");
return NULL;
}
-void skl_nhlt_free(void *addr)
+void skl_nhlt_free(struct nhlt_acpi_table *nhlt)
{
- memunmap(addr);
+ memunmap((void *) nhlt);
}
static struct nhlt_specific_cfg *skl_get_specific_cfg(
@@ -120,7 +123,7 @@
struct hdac_bus *bus = ebus_to_hbus(&skl->ebus);
struct device *dev = bus->dev;
struct nhlt_specific_cfg *sp_config;
- struct nhlt_acpi_table *nhlt = (struct nhlt_acpi_table *)skl->nhlt;
+ struct nhlt_acpi_table *nhlt = skl->nhlt;
u16 bps = (s_fmt == 16) ? 16 : 32;
u8 j;
diff --git a/sound/soc/intel/skylake/skl-pcm.c b/sound/soc/intel/skylake/skl-pcm.c
index dab0900..7c81b31 100644
--- a/sound/soc/intel/skylake/skl-pcm.c
+++ b/sound/soc/intel/skylake/skl-pcm.c
@@ -51,7 +51,7 @@
.rate_min = 8000,
.rate_max = 48000,
.channels_min = 1,
- .channels_max = HDA_QUAD,
+ .channels_max = 8,
.buffer_bytes_max = AZX_MAX_BUF_SIZE,
.period_bytes_min = 128,
.period_bytes_max = AZX_MAX_BUF_SIZE / 2,
@@ -213,7 +213,7 @@
struct skl_sst *ctx = skl->skl_sst;
struct skl_module_cfg *mconfig;
- if ((dai->playback_active > 1) || (dai->capture_active > 1))
+ if (dai->playback_widget->power || dai->capture_widget->power)
return 0;
mconfig = skl_tplg_be_get_cpr_module(dai, substream->stream);
@@ -402,23 +402,33 @@
struct skl_module_cfg *mconfig;
struct hdac_ext_bus *ebus = get_bus_ctx(substream);
struct hdac_ext_stream *stream = get_hdac_ext_stream(substream);
+ struct snd_soc_dapm_widget *w;
int ret;
mconfig = skl_tplg_fe_get_cpr_module(dai, substream->stream);
if (!mconfig)
return -EIO;
+ if (substream->stream == SNDRV_PCM_STREAM_PLAYBACK)
+ w = dai->playback_widget;
+ else
+ w = dai->capture_widget;
+
switch (cmd) {
case SNDRV_PCM_TRIGGER_RESUME:
- skl_pcm_prepare(substream, dai);
- /*
- * enable DMA Resume enable bit for the stream, set the dpib
- * & lpib position to resune before starting the DMA
- */
- snd_hdac_ext_stream_drsm_enable(ebus, true,
- hdac_stream(stream)->index);
- snd_hdac_ext_stream_set_dpibr(ebus, stream, stream->dpib);
- snd_hdac_ext_stream_set_lpib(stream, stream->lpib);
+ if (!w->ignore_suspend) {
+ skl_pcm_prepare(substream, dai);
+ /*
+ * enable DMA Resume enable bit for the stream, set the
+ * dpib & lpib position to resume before starting the
+ * DMA
+ */
+ snd_hdac_ext_stream_drsm_enable(ebus, true,
+ hdac_stream(stream)->index);
+ snd_hdac_ext_stream_set_dpibr(ebus, stream,
+ stream->dpib);
+ snd_hdac_ext_stream_set_lpib(stream, stream->lpib);
+ }
case SNDRV_PCM_TRIGGER_START:
case SNDRV_PCM_TRIGGER_PAUSE_RELEASE:
@@ -448,7 +458,7 @@
return ret;
ret = skl_decoupled_trigger(substream, cmd);
- if (cmd == SNDRV_PCM_TRIGGER_SUSPEND) {
+ if ((cmd == SNDRV_PCM_TRIGGER_SUSPEND) && !w->ignore_suspend) {
/* save the dpib and lpib positions */
stream->dpib = readl(ebus->bus.remap_addr +
AZX_REG_VS_SDXDPIB_XBASE +
@@ -523,7 +533,6 @@
if (!link)
return -EINVAL;
- snd_hdac_ext_bus_link_power_up(link);
snd_hdac_ext_link_stream_reset(link_dev);
snd_hdac_ext_link_stream_setup(link_dev, format_val);
@@ -682,7 +691,7 @@
.playback = {
.stream_name = "HDMI1 Playback",
.channels_min = HDA_STEREO,
- .channels_max = HDA_STEREO,
+ .channels_max = 8,
.rates = SNDRV_PCM_RATE_32000 | SNDRV_PCM_RATE_44100 |
SNDRV_PCM_RATE_48000 | SNDRV_PCM_RATE_88200 |
SNDRV_PCM_RATE_96000 | SNDRV_PCM_RATE_176400 |
@@ -697,7 +706,7 @@
.playback = {
.stream_name = "HDMI2 Playback",
.channels_min = HDA_STEREO,
- .channels_max = HDA_STEREO,
+ .channels_max = 8,
.rates = SNDRV_PCM_RATE_32000 | SNDRV_PCM_RATE_44100 |
SNDRV_PCM_RATE_48000 | SNDRV_PCM_RATE_88200 |
SNDRV_PCM_RATE_96000 | SNDRV_PCM_RATE_176400 |
@@ -712,7 +721,7 @@
.playback = {
.stream_name = "HDMI3 Playback",
.channels_min = HDA_STEREO,
- .channels_max = HDA_STEREO,
+ .channels_max = 8,
.rates = SNDRV_PCM_RATE_32000 | SNDRV_PCM_RATE_44100 |
SNDRV_PCM_RATE_48000 | SNDRV_PCM_RATE_88200 |
SNDRV_PCM_RATE_96000 | SNDRV_PCM_RATE_176400 |
@@ -760,12 +769,84 @@
},
},
{
+ .name = "SSP2 Pin",
+ .ops = &skl_be_ssp_dai_ops,
+ .playback = {
+ .stream_name = "ssp2 Tx",
+ .channels_min = HDA_STEREO,
+ .channels_max = HDA_STEREO,
+ .rates = SNDRV_PCM_RATE_48000,
+ .formats = SNDRV_PCM_FMTBIT_S16_LE,
+ },
+ .capture = {
+ .stream_name = "ssp2 Rx",
+ .channels_min = HDA_STEREO,
+ .channels_max = HDA_STEREO,
+ .rates = SNDRV_PCM_RATE_48000,
+ .formats = SNDRV_PCM_FMTBIT_S16_LE,
+ },
+},
+{
+ .name = "SSP3 Pin",
+ .ops = &skl_be_ssp_dai_ops,
+ .playback = {
+ .stream_name = "ssp3 Tx",
+ .channels_min = HDA_STEREO,
+ .channels_max = HDA_STEREO,
+ .rates = SNDRV_PCM_RATE_48000,
+ .formats = SNDRV_PCM_FMTBIT_S16_LE,
+ },
+ .capture = {
+ .stream_name = "ssp3 Rx",
+ .channels_min = HDA_STEREO,
+ .channels_max = HDA_STEREO,
+ .rates = SNDRV_PCM_RATE_48000,
+ .formats = SNDRV_PCM_FMTBIT_S16_LE,
+ },
+},
+{
+ .name = "SSP4 Pin",
+ .ops = &skl_be_ssp_dai_ops,
+ .playback = {
+ .stream_name = "ssp4 Tx",
+ .channels_min = HDA_STEREO,
+ .channels_max = HDA_STEREO,
+ .rates = SNDRV_PCM_RATE_48000,
+ .formats = SNDRV_PCM_FMTBIT_S16_LE,
+ },
+ .capture = {
+ .stream_name = "ssp4 Rx",
+ .channels_min = HDA_STEREO,
+ .channels_max = HDA_STEREO,
+ .rates = SNDRV_PCM_RATE_48000,
+ .formats = SNDRV_PCM_FMTBIT_S16_LE,
+ },
+},
+{
+ .name = "SSP5 Pin",
+ .ops = &skl_be_ssp_dai_ops,
+ .playback = {
+ .stream_name = "ssp5 Tx",
+ .channels_min = HDA_STEREO,
+ .channels_max = HDA_STEREO,
+ .rates = SNDRV_PCM_RATE_48000,
+ .formats = SNDRV_PCM_FMTBIT_S16_LE,
+ },
+ .capture = {
+ .stream_name = "ssp5 Rx",
+ .channels_min = HDA_STEREO,
+ .channels_max = HDA_STEREO,
+ .rates = SNDRV_PCM_RATE_48000,
+ .formats = SNDRV_PCM_FMTBIT_S16_LE,
+ },
+},
+{
.name = "iDisp1 Pin",
.ops = &skl_link_dai_ops,
.playback = {
.stream_name = "iDisp1 Tx",
.channels_min = HDA_STEREO,
- .channels_max = HDA_STEREO,
+ .channels_max = 8,
.rates = SNDRV_PCM_RATE_8000|SNDRV_PCM_RATE_16000|SNDRV_PCM_RATE_48000,
.formats = SNDRV_PCM_FMTBIT_S16_LE | SNDRV_PCM_FMTBIT_S32_LE |
SNDRV_PCM_FMTBIT_S24_LE,
@@ -777,7 +858,7 @@
.playback = {
.stream_name = "iDisp2 Tx",
.channels_min = HDA_STEREO,
- .channels_max = HDA_STEREO,
+ .channels_max = 8,
.rates = SNDRV_PCM_RATE_8000|SNDRV_PCM_RATE_16000|
SNDRV_PCM_RATE_48000,
.formats = SNDRV_PCM_FMTBIT_S16_LE | SNDRV_PCM_FMTBIT_S32_LE |
@@ -790,7 +871,7 @@
.playback = {
.stream_name = "iDisp3 Tx",
.channels_min = HDA_STEREO,
- .channels_max = HDA_STEREO,
+ .channels_max = 8,
.rates = SNDRV_PCM_RATE_8000|SNDRV_PCM_RATE_16000|
SNDRV_PCM_RATE_48000,
.formats = SNDRV_PCM_FMTBIT_S16_LE | SNDRV_PCM_FMTBIT_S32_LE |
diff --git a/sound/soc/intel/skylake/skl-sst-dsp.c b/sound/soc/intel/skylake/skl-sst-dsp.c
index 2962ef2..13c1985 100644
--- a/sound/soc/intel/skylake/skl-sst-dsp.c
+++ b/sound/soc/intel/skylake/skl-sst-dsp.c
@@ -336,8 +336,6 @@
skl_ipc_int_disable(dsp);
free_irq(dsp->irq, dsp);
- dsp->cl_dev.ops.cl_cleanup_controller(dsp);
- skl_cldma_int_disable(dsp);
skl_ipc_op_int_disable(dsp);
skl_ipc_int_disable(dsp);
diff --git a/sound/soc/intel/skylake/skl-sst-dsp.h b/sound/soc/intel/skylake/skl-sst-dsp.h
index b6e310d..deabe73 100644
--- a/sound/soc/intel/skylake/skl-sst-dsp.h
+++ b/sound/soc/intel/skylake/skl-sst-dsp.h
@@ -118,16 +118,25 @@
int (*set_state_D0)(struct sst_dsp *ctx);
int (*set_state_D3)(struct sst_dsp *ctx);
unsigned int (*get_fw_errcode)(struct sst_dsp *ctx);
- int (*load_mod)(struct sst_dsp *ctx, u16 mod_id, char *mod_name);
+ int (*load_mod)(struct sst_dsp *ctx, u16 mod_id, u8 *mod_name);
int (*unload_mod)(struct sst_dsp *ctx, u16 mod_id);
};
struct skl_dsp_loader_ops {
+ int stream_tag;
+
int (*alloc_dma_buf)(struct device *dev,
struct snd_dma_buffer *dmab, size_t size);
int (*free_dma_buf)(struct device *dev,
struct snd_dma_buffer *dmab);
+ int (*prepare)(struct device *dev, unsigned int format,
+ unsigned int byte_size,
+ struct snd_dma_buffer *bufp);
+ int (*trigger)(struct device *dev, bool start, int stream_tag);
+
+ int (*cleanup)(struct device *dev, struct snd_dma_buffer *dmab,
+ int stream_tag);
};
struct skl_load_module_info {
@@ -160,6 +169,10 @@
int skl_sst_dsp_init(struct device *dev, void __iomem *mmio_base, int irq,
const char *fw_name, struct skl_dsp_loader_ops dsp_ops,
struct skl_sst **dsp);
+int bxt_sst_dsp_init(struct device *dev, void __iomem *mmio_base, int irq,
+ const char *fw_name, struct skl_dsp_loader_ops dsp_ops,
+ struct skl_sst **dsp);
void skl_sst_dsp_cleanup(struct device *dev, struct skl_sst *ctx);
+void bxt_sst_dsp_cleanup(struct device *dev, struct skl_sst *ctx);
#endif /*__SKL_SST_DSP_H__*/
diff --git a/sound/soc/intel/skylake/skl-sst.c b/sound/soc/intel/skylake/skl-sst.c
index 348a734..13ec8d5 100644
--- a/sound/soc/intel/skylake/skl-sst.c
+++ b/sound/soc/intel/skylake/skl-sst.c
@@ -20,6 +20,7 @@
#include <linux/delay.h>
#include <linux/device.h>
#include <linux/err.h>
+#include <linux/uuid.h>
#include "../common/sst-dsp.h"
#include "../common/sst-dsp-priv.h"
#include "../common/sst-ipc.h"
@@ -304,14 +305,16 @@
return ret;
}
-static int skl_load_module(struct sst_dsp *ctx, u16 mod_id, char *guid)
+static int skl_load_module(struct sst_dsp *ctx, u16 mod_id, u8 *guid)
{
struct skl_module_table *module_entry = NULL;
int ret = 0;
char mod_name[64]; /* guid str = 32 chars + 4 hyphens */
+ uuid_le *uuid_mod;
- snprintf(mod_name, sizeof(mod_name), "%s%s%s",
- "intel/dsp_fw_", guid, ".bin");
+ uuid_mod = (uuid_le *)guid;
+ snprintf(mod_name, sizeof(mod_name), "%s%pUL%s",
+ "intel/dsp_fw_", uuid_mod, ".bin");
module_entry = skl_module_get_from_id(ctx, mod_id);
if (module_entry == NULL) {
@@ -451,6 +454,10 @@
skl_clear_module_table(ctx->dsp);
skl_ipc_free(&ctx->ipc);
ctx->dsp->ops->free(ctx->dsp);
+ if (ctx->boot_complete) {
+ ctx->dsp->cl_dev.ops.cl_cleanup_controller(ctx->dsp);
+ skl_cldma_int_disable(ctx->dsp);
+ }
}
EXPORT_SYMBOL_GPL(skl_sst_dsp_cleanup);
diff --git a/sound/soc/intel/skylake/skl-topology.c b/sound/soc/intel/skylake/skl-topology.c
index cdb78b7..3e036b0 100644
--- a/sound/soc/intel/skylake/skl-topology.c
+++ b/sound/soc/intel/skylake/skl-topology.c
@@ -154,13 +154,32 @@
dev_dbg(ctx->dev, "ch_cfg = %d\n", mcfg->out_fmt[0].ch_cfg);
}
+static void skl_tplg_update_chmap(struct skl_module_fmt *fmt, int chs)
+{
+ int slot_map = 0xFFFFFFFF;
+ int start_slot = 0;
+ int i;
+
+ for (i = 0; i < chs; i++) {
+ /*
+ * For 2 channels with starting slot as 0, slot map will
+ * look like 0xFFFFFF10.
+ */
+ slot_map &= (~(0xF << (4 * i)) | (start_slot << (4 * i)));
+ start_slot++;
+ }
+ fmt->ch_map = slot_map;
+}
+
static void skl_tplg_update_params(struct skl_module_fmt *fmt,
struct skl_pipe_params *params, int fixup)
{
if (fixup & SKL_RATE_FIXUP_MASK)
fmt->s_freq = params->s_freq;
- if (fixup & SKL_CH_FIXUP_MASK)
+ if (fixup & SKL_CH_FIXUP_MASK) {
fmt->channels = params->ch;
+ skl_tplg_update_chmap(fmt, fmt->channels);
+ }
if (fixup & SKL_FMT_FIXUP_MASK) {
fmt->valid_bit_depth = skl_get_bit_depth(params->s_fmt);
@@ -1564,6 +1583,8 @@
return -ENOMEM;
w->priv = mconfig;
+ memcpy(&mconfig->guid, &dfw_config->uuid, 16);
+
mconfig->id.module_id = dfw_config->module_id;
mconfig->id.instance_id = dfw_config->instance_id;
mconfig->mcps = dfw_config->max_mcps;
@@ -1593,10 +1614,6 @@
mconfig->time_slot = dfw_config->time_slot;
mconfig->formats_config.caps_size = dfw_config->caps.caps_size;
- if (dfw_config->is_loadable)
- memcpy(mconfig->guid, dfw_config->uuid,
- ARRAY_SIZE(dfw_config->uuid));
-
mconfig->m_in_pin = devm_kzalloc(bus->dev, (mconfig->max_in_queue) *
sizeof(*mconfig->m_in_pin),
GFP_KERNEL);
diff --git a/sound/soc/intel/skylake/skl-topology.h b/sound/soc/intel/skylake/skl-topology.h
index d2d9230..e4b399c 100644
--- a/sound/soc/intel/skylake/skl-topology.h
+++ b/sound/soc/intel/skylake/skl-topology.h
@@ -281,7 +281,7 @@
};
struct skl_module_cfg {
- char guid[SKL_UUID_STR_SZ];
+ u8 guid[16];
struct skl_module_inst_id id;
u8 domain;
bool homogenous_inputs;
diff --git a/sound/soc/intel/skylake/skl-tplg-interface.h b/sound/soc/intel/skylake/skl-tplg-interface.h
index 1db88a6..a32e5e9 100644
--- a/sound/soc/intel/skylake/skl-tplg-interface.h
+++ b/sound/soc/intel/skylake/skl-tplg-interface.h
@@ -181,7 +181,7 @@
} __packed;
struct skl_dfw_module {
- char uuid[SKL_UUID_STR_SZ];
+ u8 uuid[16];
u16 module_id;
u16 instance_id;
diff --git a/sound/soc/intel/skylake/skl.c b/sound/soc/intel/skylake/skl.c
index 3982f55..06d8c26 100644
--- a/sound/soc/intel/skylake/skl.c
+++ b/sound/soc/intel/skylake/skl.c
@@ -229,7 +229,12 @@
* running, we need to save the state for these and continue
*/
if (skl->supend_active) {
+ /* turn off the links and stop the CORB/RIRB DMA if it is On */
snd_hdac_ext_bus_link_power_down_all(ebus);
+
+ if (ebus->cmd_dma_state)
+ snd_hdac_bus_stop_cmd_io(&ebus->bus);
+
enable_irq_wake(bus->irq);
pci_save_state(pci);
pci_disable_device(pci);
@@ -255,6 +260,7 @@
struct hdac_ext_bus *ebus = pci_get_drvdata(pci);
struct skl *skl = ebus_to_skl(ebus);
struct hdac_bus *bus = ebus_to_hbus(ebus);
+ struct hdac_ext_link *hlink = NULL;
int ret;
/* Turned OFF in HDMI codec driver after codec reconfiguration */
@@ -276,8 +282,29 @@
ret = pci_enable_device(pci);
snd_hdac_ext_bus_link_power_up_all(ebus);
disable_irq_wake(bus->irq);
+ /*
+ * turn On the links which are On before active suspend
+ * and start the CORB/RIRB DMA if On before
+ * active suspend.
+ */
+ list_for_each_entry(hlink, &ebus->hlink_list, list) {
+ if (hlink->ref_count)
+ snd_hdac_ext_bus_link_power_up(hlink);
+ }
+
+ if (ebus->cmd_dma_state)
+ snd_hdac_bus_init_cmd_io(&ebus->bus);
} else {
ret = _skl_resume(ebus);
+
+ /* turn off the links which are off before suspend */
+ list_for_each_entry(hlink, &ebus->hlink_list, list) {
+ if (!hlink->ref_count)
+ snd_hdac_ext_bus_link_power_down(hlink);
+ }
+
+ if (!ebus->cmd_dma_state)
+ snd_hdac_bus_stop_cmd_io(&ebus->bus);
}
return ret;
@@ -613,6 +640,7 @@
struct skl *skl;
struct hdac_ext_bus *ebus = NULL;
struct hdac_bus *bus = NULL;
+ struct hdac_ext_link *hlink = NULL;
int err;
/* we use ext core ops, so provide NULL for ops here */
@@ -643,7 +671,7 @@
err = skl_machine_device_register(skl,
(void *)pci_id->driver_data);
if (err < 0)
- goto out_free;
+ goto out_nhlt_free;
err = skl_init_dsp(skl);
if (err < 0) {
@@ -679,6 +707,12 @@
}
}
+ /*
+ * we are done probling so decrement link counts
+ */
+ list_for_each_entry(hlink, &ebus->hlink_list, list)
+ snd_hdac_ext_bus_link_put(ebus, hlink);
+
/*configure PM */
pm_runtime_put_noidle(bus->dev);
pm_runtime_allow(bus->dev);
@@ -693,6 +727,8 @@
skl_free_dsp(skl);
out_mach_free:
skl_machine_device_unregister(skl);
+out_nhlt_free:
+ skl_nhlt_free(skl->nhlt);
out_free:
skl->init_failed = 1;
skl_free(ebus);
@@ -743,6 +779,7 @@
skl_free_dsp(skl);
skl_machine_device_unregister(skl);
skl_dmic_device_unregister(skl);
+ skl_nhlt_free(skl->nhlt);
skl_free(ebus);
dev_set_drvdata(&pci->dev, NULL);
}
diff --git a/sound/soc/intel/skylake/skl.h b/sound/soc/intel/skylake/skl.h
index 39e16fa..4b4b387 100644
--- a/sound/soc/intel/skylake/skl.h
+++ b/sound/soc/intel/skylake/skl.h
@@ -66,7 +66,7 @@
struct platform_device *dmic_dev;
struct platform_device *i2s_dev;
- void *nhlt; /* nhlt ptr */
+ struct nhlt_acpi_table *nhlt; /* nhlt ptr */
struct skl_sst *skl_sst; /* sst skl ctx */
struct skl_dsp_resource resource;
@@ -103,8 +103,8 @@
int skl_platform_unregister(struct device *dev);
int skl_platform_register(struct device *dev);
-void *skl_nhlt_init(struct device *dev);
-void skl_nhlt_free(void *addr);
+struct nhlt_acpi_table *skl_nhlt_init(struct device *dev);
+void skl_nhlt_free(struct nhlt_acpi_table *addr);
struct nhlt_specific_cfg *skl_get_ep_blob(struct skl *skl, u32 instance,
u8 link_type, u8 s_fmt, u8 no_ch, u32 s_rate, u8 dirn);
diff --git a/sound/soc/rockchip/rockchip_i2s.c b/sound/soc/rockchip/rockchip_i2s.c
index 2f8e204..574c6af 100644
--- a/sound/soc/rockchip/rockchip_i2s.c
+++ b/sound/soc/rockchip/rockchip_i2s.c
@@ -34,6 +34,13 @@
struct regmap *regmap;
+/*
+ * Used to indicate the tx/rx status.
+ * I2S controller hopes to start the tx and rx together,
+ * also to stop them when they are both try to stop.
+*/
+ bool tx_start;
+ bool rx_start;
bool is_master_mode;
};
@@ -75,29 +82,37 @@
I2S_DMACR_TDE_ENABLE, I2S_DMACR_TDE_ENABLE);
regmap_update_bits(i2s->regmap, I2S_XFER,
- I2S_XFER_TXS_START,
- I2S_XFER_TXS_START);
+ I2S_XFER_TXS_START | I2S_XFER_RXS_START,
+ I2S_XFER_TXS_START | I2S_XFER_RXS_START);
+
+ i2s->tx_start = true;
} else {
+ i2s->tx_start = false;
+
regmap_update_bits(i2s->regmap, I2S_DMACR,
I2S_DMACR_TDE_ENABLE, I2S_DMACR_TDE_DISABLE);
- regmap_update_bits(i2s->regmap, I2S_XFER,
- I2S_XFER_TXS_START,
- I2S_XFER_TXS_STOP);
+ if (!i2s->rx_start) {
+ regmap_update_bits(i2s->regmap, I2S_XFER,
+ I2S_XFER_TXS_START |
+ I2S_XFER_RXS_START,
+ I2S_XFER_TXS_STOP |
+ I2S_XFER_RXS_STOP);
- regmap_update_bits(i2s->regmap, I2S_CLR,
- I2S_CLR_TXC,
- I2S_CLR_TXC);
+ regmap_update_bits(i2s->regmap, I2S_CLR,
+ I2S_CLR_TXC | I2S_CLR_RXC,
+ I2S_CLR_TXC | I2S_CLR_RXC);
- regmap_read(i2s->regmap, I2S_CLR, &val);
-
- /* Should wait for clear operation to finish */
- while (val & I2S_CLR_TXC) {
regmap_read(i2s->regmap, I2S_CLR, &val);
- retry--;
- if (!retry) {
- dev_warn(i2s->dev, "fail to clear\n");
- break;
+
+ /* Should wait for clear operation to finish */
+ while (val) {
+ regmap_read(i2s->regmap, I2S_CLR, &val);
+ retry--;
+ if (!retry) {
+ dev_warn(i2s->dev, "fail to clear\n");
+ break;
+ }
}
}
}
@@ -113,29 +128,37 @@
I2S_DMACR_RDE_ENABLE, I2S_DMACR_RDE_ENABLE);
regmap_update_bits(i2s->regmap, I2S_XFER,
- I2S_XFER_RXS_START,
- I2S_XFER_RXS_START);
+ I2S_XFER_TXS_START | I2S_XFER_RXS_START,
+ I2S_XFER_TXS_START | I2S_XFER_RXS_START);
+
+ i2s->rx_start = true;
} else {
+ i2s->rx_start = false;
+
regmap_update_bits(i2s->regmap, I2S_DMACR,
I2S_DMACR_RDE_ENABLE, I2S_DMACR_RDE_DISABLE);
- regmap_update_bits(i2s->regmap, I2S_XFER,
- I2S_XFER_RXS_START,
- I2S_XFER_RXS_STOP);
+ if (!i2s->tx_start) {
+ regmap_update_bits(i2s->regmap, I2S_XFER,
+ I2S_XFER_TXS_START |
+ I2S_XFER_RXS_START,
+ I2S_XFER_TXS_STOP |
+ I2S_XFER_RXS_STOP);
- regmap_update_bits(i2s->regmap, I2S_CLR,
- I2S_CLR_RXC,
- I2S_CLR_RXC);
+ regmap_update_bits(i2s->regmap, I2S_CLR,
+ I2S_CLR_TXC | I2S_CLR_RXC,
+ I2S_CLR_TXC | I2S_CLR_RXC);
- regmap_read(i2s->regmap, I2S_CLR, &val);
-
- /* Should wait for clear operation to finish */
- while (val & I2S_CLR_RXC) {
regmap_read(i2s->regmap, I2S_CLR, &val);
- retry--;
- if (!retry) {
- dev_warn(i2s->dev, "fail to clear\n");
- break;
+
+ /* Should wait for clear operation to finish */
+ while (val) {
+ regmap_read(i2s->regmap, I2S_CLR, &val);
+ retry--;
+ if (!retry) {
+ dev_warn(i2s->dev, "fail to clear\n");
+ break;
+ }
}
}
}
diff --git a/sound/soc/soc-core.c b/sound/soc/soc-core.c
index d2e62b15..16369ca 100644
--- a/sound/soc/soc-core.c
+++ b/sound/soc/soc-core.c
@@ -930,7 +930,18 @@
return NULL;
}
-static struct snd_soc_dai *snd_soc_find_dai(
+/**
+ * snd_soc_find_dai - Find a registered DAI
+ *
+ * @dlc: name of the DAI and optional component info to match
+ *
+ * This function will search all regsitered components and their DAIs to
+ * find the DAI of the same name. The component's of_node and name
+ * should also match if being specified.
+ *
+ * Return: pointer of DAI, or NULL if not found.
+ */
+struct snd_soc_dai *snd_soc_find_dai(
const struct snd_soc_dai_link_component *dlc)
{
struct snd_soc_component *component;
@@ -959,6 +970,7 @@
return NULL;
}
+EXPORT_SYMBOL_GPL(snd_soc_find_dai);
static bool soc_is_dai_link_bound(struct snd_soc_card *card,
struct snd_soc_dai_link *dai_link)
diff --git a/sound/soc/soc-generic-dmaengine-pcm.c b/sound/soc/soc-generic-dmaengine-pcm.c
index 6fd1906..6cef397 100644
--- a/sound/soc/soc-generic-dmaengine-pcm.c
+++ b/sound/soc/soc-generic-dmaengine-pcm.c
@@ -163,31 +163,42 @@
}
/*
- * Prepare formats mask for valid/allowed sample types. If the dma does
- * not have support for the given physical word size, it needs to be
- * masked out so user space can not use the format which produces
- * corrupted audio.
- * In case the dma driver does not implement the slave_caps the default
- * assumption is that it supports 1, 2 and 4 bytes widths.
+ * If SND_DMAENGINE_PCM_DAI_FLAG_PACK is set keep
+ * hw.formats set to 0, meaning no restrictions are in place.
+ * In this case it's the responsibility of the DAI driver to
+ * provide the supported format information.
*/
- for (i = 0; i <= SNDRV_PCM_FORMAT_LAST; i++) {
- int bits = snd_pcm_format_physical_width(i);
+ if (!(dma_data->flags & SND_DMAENGINE_PCM_DAI_FLAG_PACK))
+ /*
+ * Prepare formats mask for valid/allowed sample types. If the
+ * dma does not have support for the given physical word size,
+ * it needs to be masked out so user space can not use the
+ * format which produces corrupted audio.
+ * In case the dma driver does not implement the slave_caps the
+ * default assumption is that it supports 1, 2 and 4 bytes
+ * widths.
+ */
+ for (i = 0; i <= SNDRV_PCM_FORMAT_LAST; i++) {
+ int bits = snd_pcm_format_physical_width(i);
- /* Enable only samples with DMA supported physical widths */
- switch (bits) {
- case 8:
- case 16:
- case 24:
- case 32:
- case 64:
- if (addr_widths & (1 << (bits / 8)))
- hw.formats |= (1LL << i);
- break;
- default:
- /* Unsupported types */
- break;
+ /*
+ * Enable only samples with DMA supported physical
+ * widths
+ */
+ switch (bits) {
+ case 8:
+ case 16:
+ case 24:
+ case 32:
+ case 64:
+ if (addr_widths & (1 << (bits / 8)))
+ hw.formats |= (1LL << i);
+ break;
+ default:
+ /* Unsupported types */
+ break;
+ }
}
- }
return snd_soc_set_runtime_hwparams(substream, &hw);
}
diff --git a/sound/usb/card.c b/sound/usb/card.c
index 3fc6358..69860da 100644
--- a/sound/usb/card.c
+++ b/sound/usb/card.c
@@ -350,6 +350,7 @@
case USB_SPEED_HIGH:
case USB_SPEED_WIRELESS:
case USB_SPEED_SUPER:
+ case USB_SPEED_SUPER_PLUS:
break;
default:
dev_err(&dev->dev, "unknown device speed %d\n", snd_usb_get_speed(dev));
@@ -450,6 +451,9 @@
case USB_SPEED_SUPER:
strlcat(card->longname, ", super speed", sizeof(card->longname));
break;
+ case USB_SPEED_SUPER_PLUS:
+ strlcat(card->longname, ", super speed plus", sizeof(card->longname));
+ break;
default:
break;
}
diff --git a/sound/usb/clock.c b/sound/usb/clock.c
index 7ccbcaf..26dd5f2 100644
--- a/sound/usb/clock.c
+++ b/sound/usb/clock.c
@@ -309,6 +309,9 @@
* support reading */
if (snd_usb_get_sample_rate_quirk(chip))
return 0;
+ /* the firmware is likely buggy, don't repeat to fail too many times */
+ if (chip->sample_rate_read_error > 2)
+ return 0;
if ((err = snd_usb_ctl_msg(dev, usb_rcvctrlpipe(dev, 0), UAC_GET_CUR,
USB_TYPE_CLASS | USB_RECIP_ENDPOINT | USB_DIR_IN,
@@ -316,6 +319,7 @@
data, sizeof(data))) < 0) {
dev_err(&dev->dev, "%d:%d: cannot get freq at ep %#x\n",
iface, fmt->altsetting, ep);
+ chip->sample_rate_read_error++;
return 0; /* some devices don't support reading */
}
diff --git a/sound/usb/helper.c b/sound/usb/helper.c
index 51ed1ac..7712e2b 100644
--- a/sound/usb/helper.c
+++ b/sound/usb/helper.c
@@ -120,6 +120,7 @@
case USB_SPEED_HIGH:
case USB_SPEED_WIRELESS:
case USB_SPEED_SUPER:
+ case USB_SPEED_SUPER_PLUS:
if (get_endpoint(alts, 0)->bInterval >= 1 &&
get_endpoint(alts, 0)->bInterval <= 4)
return get_endpoint(alts, 0)->bInterval - 1;
diff --git a/sound/usb/midi.c b/sound/usb/midi.c
index 47de8af..7ba9292 100644
--- a/sound/usb/midi.c
+++ b/sound/usb/midi.c
@@ -911,6 +911,7 @@
switch (snd_usb_get_speed(ep->umidi->dev)) {
case USB_SPEED_HIGH:
case USB_SPEED_SUPER:
+ case USB_SPEED_SUPER_PLUS:
count = 1;
break;
default:
diff --git a/sound/usb/mixer.c b/sound/usb/mixer.c
index 4f85757..2f8c388 100644
--- a/sound/usb/mixer.c
+++ b/sound/usb/mixer.c
@@ -45,6 +45,7 @@
#include <linux/bitops.h>
#include <linux/init.h>
#include <linux/list.h>
+#include <linux/log2.h>
#include <linux/slab.h>
#include <linux/string.h>
#include <linux/usb.h>
@@ -1378,6 +1379,71 @@
snd_usb_mixer_add_control(&cval->head, kctl);
}
+static int parse_clock_source_unit(struct mixer_build *state, int unitid,
+ void *_ftr)
+{
+ struct uac_clock_source_descriptor *hdr = _ftr;
+ struct usb_mixer_elem_info *cval;
+ struct snd_kcontrol *kctl;
+ char name[SNDRV_CTL_ELEM_ID_NAME_MAXLEN];
+ int ret;
+
+ if (state->mixer->protocol != UAC_VERSION_2)
+ return -EINVAL;
+
+ if (hdr->bLength != sizeof(*hdr)) {
+ usb_audio_dbg(state->chip,
+ "Bogus clock source descriptor length of %d, ignoring.\n",
+ hdr->bLength);
+ return 0;
+ }
+
+ /*
+ * The only property of this unit we are interested in is the
+ * clock source validity. If that isn't readable, just bail out.
+ */
+ if (!uac2_control_is_readable(hdr->bmControls,
+ ilog2(UAC2_CS_CONTROL_CLOCK_VALID)))
+ return 0;
+
+ cval = kzalloc(sizeof(*cval), GFP_KERNEL);
+ if (!cval)
+ return -ENOMEM;
+
+ snd_usb_mixer_elem_init_std(&cval->head, state->mixer, hdr->bClockID);
+
+ cval->min = 0;
+ cval->max = 1;
+ cval->channels = 1;
+ cval->val_type = USB_MIXER_BOOLEAN;
+ cval->control = UAC2_CS_CONTROL_CLOCK_VALID;
+
+ if (uac2_control_is_writeable(hdr->bmControls,
+ ilog2(UAC2_CS_CONTROL_CLOCK_VALID)))
+ kctl = snd_ctl_new1(&usb_feature_unit_ctl, cval);
+ else {
+ cval->master_readonly = 1;
+ kctl = snd_ctl_new1(&usb_feature_unit_ctl_ro, cval);
+ }
+
+ if (!kctl) {
+ kfree(cval);
+ return -ENOMEM;
+ }
+
+ kctl->private_free = snd_usb_mixer_elem_free;
+ ret = snd_usb_copy_string_desc(state, hdr->iClockSource,
+ name, sizeof(name));
+ if (ret > 0)
+ snprintf(kctl->id.name, sizeof(kctl->id.name),
+ "%s Validity", name);
+ else
+ snprintf(kctl->id.name, sizeof(kctl->id.name),
+ "Clock Source %d Validity", hdr->bClockID);
+
+ return snd_usb_mixer_add_control(&cval->head, kctl);
+}
+
/*
* parse a feature unit
*
@@ -2126,10 +2192,11 @@
switch (p1[2]) {
case UAC_INPUT_TERMINAL:
- case UAC2_CLOCK_SOURCE:
return 0; /* NOP */
case UAC_MIXER_UNIT:
return parse_audio_mixer_unit(state, unitid, p1);
+ case UAC2_CLOCK_SOURCE:
+ return parse_clock_source_unit(state, unitid, p1);
case UAC_SELECTOR_UNIT:
case UAC2_CLOCK_SELECTOR:
return parse_audio_selector_unit(state, unitid, p1);
@@ -2307,6 +2374,7 @@
__u8 unitid = (index >> 8) & 0xff;
__u8 control = (value >> 8) & 0xff;
__u8 channel = value & 0xff;
+ unsigned int count = 0;
if (channel >= MAX_CHANNELS) {
usb_audio_dbg(mixer->chip,
@@ -2315,6 +2383,12 @@
return;
}
+ for (list = mixer->id_elems[unitid]; list; list = list->next_id_elem)
+ count++;
+
+ if (count == 0)
+ return;
+
for (list = mixer->id_elems[unitid]; list; list = list->next_id_elem) {
struct usb_mixer_elem_info *info;
@@ -2322,7 +2396,7 @@
continue;
info = (struct usb_mixer_elem_info *)list;
- if (info->control != control)
+ if (count > 1 && info->control != control)
continue;
switch (attribute) {
diff --git a/sound/usb/usbaudio.h b/sound/usb/usbaudio.h
index b665d85..4d5c89a 100644
--- a/sound/usb/usbaudio.h
+++ b/sound/usb/usbaudio.h
@@ -47,6 +47,7 @@
int num_interfaces;
int num_suspended_intf;
+ int sample_rate_read_error;
struct list_head pcm_list; /* list of pcm streams */
struct list_head ep_list; /* list of audio-related endpoints */
diff --git a/tools/hv/lsvmbus b/tools/hv/lsvmbus
index 162a378..e8fecd6 100644
--- a/tools/hv/lsvmbus
+++ b/tools/hv/lsvmbus
@@ -35,6 +35,7 @@
'{ba6163d9-04a1-4d29-b605-72e2ffb1dc7f}' : 'Synthetic SCSI Controller',
'{2f9bcc4a-0069-4af3-b76b-6fd0be528cda}' : 'Synthetic fiber channel adapter',
'{8c2eaf3d-32a7-4b09-ab99-bd1f1c86b501}' : 'Synthetic RDMA adapter',
+ '{44c4f61d-4444-4400-9d52-802e27ede19f}' : 'PCI Express pass-through',
'{276aacf4-ac15-426c-98dd-7521ad3f01fe}' : '[Reserved system device]',
'{f8e65716-3cb3-4a06-9a60-1889c5cccab5}' : '[Reserved system device]',
'{3375baf4-9e15-4b30-b765-67acb10d607b}' : '[Reserved system device]',
diff --git a/tools/iio/generic_buffer.c b/tools/iio/generic_buffer.c
index 01c4f67..2429c78 100644
--- a/tools/iio/generic_buffer.c
+++ b/tools/iio/generic_buffer.c
@@ -35,6 +35,15 @@
#include "iio_utils.h"
/**
+ * enum autochan - state for the automatic channel enabling mechanism
+ */
+enum autochan {
+ AUTOCHANNELS_DISABLED,
+ AUTOCHANNELS_ENABLED,
+ AUTOCHANNELS_ACTIVE,
+};
+
+/**
* size_from_channelarray() - calculate the storage size of a scan
* @channels: the channel info array
* @num_channels: number of channels
@@ -191,10 +200,51 @@
printf("\n");
}
+static int enable_disable_all_channels(char *dev_dir_name, int enable)
+{
+ const struct dirent *ent;
+ char scanelemdir[256];
+ DIR *dp;
+ int ret;
+
+ snprintf(scanelemdir, sizeof(scanelemdir),
+ FORMAT_SCAN_ELEMENTS_DIR, dev_dir_name);
+ scanelemdir[sizeof(scanelemdir)-1] = '\0';
+
+ dp = opendir(scanelemdir);
+ if (!dp) {
+ fprintf(stderr, "Enabling/disabling channels: can't open %s\n",
+ scanelemdir);
+ return -EIO;
+ }
+
+ ret = -ENOENT;
+ while (ent = readdir(dp), ent) {
+ if (iioutils_check_suffix(ent->d_name, "_en")) {
+ printf("%sabling: %s\n",
+ enable ? "En" : "Dis",
+ ent->d_name);
+ ret = write_sysfs_int(ent->d_name, scanelemdir,
+ enable);
+ if (ret < 0)
+ fprintf(stderr, "Failed to enable/disable %s\n",
+ ent->d_name);
+ }
+ }
+
+ if (closedir(dp) == -1) {
+ perror("Enabling/disabling channels: "
+ "Failed to close directory");
+ return -errno;
+ }
+ return 0;
+}
+
void print_usage(void)
{
fprintf(stderr, "Usage: generic_buffer [options]...\n"
"Capture, convert and output data from IIO device buffer\n"
+ " -a Auto-activate all available channels\n"
" -c <n> Do n conversions\n"
" -e Disable wait for event (new data)\n"
" -g Use trigger-less mode\n"
@@ -225,12 +275,16 @@
int scan_size;
int noevents = 0;
int notrigger = 0;
+ enum autochan autochannels = AUTOCHANNELS_DISABLED;
char *dummy;
struct iio_channel_info *channels;
- while ((c = getopt(argc, argv, "c:egl:n:t:w:")) != -1) {
+ while ((c = getopt(argc, argv, "ac:egl:n:t:w:")) != -1) {
switch (c) {
+ case 'a':
+ autochannels = AUTOCHANNELS_ENABLED;
+ break;
case 'c':
errno = 0;
num_loops = strtoul(optarg, &dummy, 10);
@@ -304,7 +358,19 @@
}
}
- /* Verify the trigger exists */
+ /* Look for this "-devN" trigger */
+ trig_num = find_type_by_name(trigger_name, "trigger");
+ if (trig_num < 0) {
+ /* OK try the simpler "-trigger" suffix instead */
+ free(trigger_name);
+ ret = asprintf(&trigger_name,
+ "%s-trigger", device_name);
+ if (ret < 0) {
+ ret = -ENOMEM;
+ goto error_free_dev_dir_name;
+ }
+ }
+
trig_num = find_type_by_name(trigger_name, "trigger");
if (trig_num < 0) {
fprintf(stderr, "Failed to find the trigger %s\n",
@@ -328,12 +394,47 @@
"diag %s\n", dev_dir_name);
goto error_free_triggername;
}
- if (!num_channels) {
+ if (num_channels && autochannels == AUTOCHANNELS_ENABLED) {
+ fprintf(stderr, "Auto-channels selected but some channels "
+ "are already activated in sysfs\n");
+ fprintf(stderr, "Proceeding without activating any channels\n");
+ }
+
+ if (!num_channels && autochannels == AUTOCHANNELS_ENABLED) {
+ fprintf(stderr,
+ "No channels are enabled, enabling all channels\n");
+
+ ret = enable_disable_all_channels(dev_dir_name, 1);
+ if (ret) {
+ fprintf(stderr, "Failed to enable all channels\n");
+ goto error_free_triggername;
+ }
+
+ /* This flags that we need to disable the channels again */
+ autochannels = AUTOCHANNELS_ACTIVE;
+
+ ret = build_channel_array(dev_dir_name, &channels,
+ &num_channels);
+ if (ret) {
+ fprintf(stderr, "Problem reading scan element "
+ "information\n"
+ "diag %s\n", dev_dir_name);
+ goto error_disable_channels;
+ }
+ if (!num_channels) {
+ fprintf(stderr, "Still no channels after "
+ "auto-enabling, giving up\n");
+ goto error_disable_channels;
+ }
+ }
+
+ if (!num_channels && autochannels == AUTOCHANNELS_DISABLED) {
fprintf(stderr,
"No channels are enabled, we have nothing to scan.\n");
fprintf(stderr, "Enable channels manually in "
FORMAT_SCAN_ELEMENTS_DIR
- "/*_en and try again.\n", dev_dir_name);
+ "/*_en or pass -a to autoenable channels and "
+ "try again.\n", dev_dir_name);
ret = -ENOENT;
goto error_free_triggername;
}
@@ -467,7 +568,12 @@
error_free_triggername:
if (datardytrigger)
free(trigger_name);
-
+error_disable_channels:
+ if (autochannels == AUTOCHANNELS_ACTIVE) {
+ ret = enable_disable_all_channels(dev_dir_name, 0);
+ if (ret)
+ fprintf(stderr, "Failed to disable all channels\n");
+ }
error_free_dev_dir_name:
free(dev_dir_name);
diff --git a/tools/iio/iio_event_monitor.c b/tools/iio/iio_event_monitor.c
index d51eb04..d9b7e0f 100644
--- a/tools/iio/iio_event_monitor.c
+++ b/tools/iio/iio_event_monitor.c
@@ -53,6 +53,10 @@
[IIO_ENERGY] = "energy",
[IIO_DISTANCE] = "distance",
[IIO_VELOCITY] = "velocity",
+ [IIO_CONCENTRATION] = "concentration",
+ [IIO_RESISTANCE] = "resistance",
+ [IIO_PH] = "ph",
+ [IIO_UVINDEX] = "uvindex",
};
static const char * const iio_ev_type_text[] = {
@@ -90,6 +94,7 @@
[IIO_MOD_LIGHT_RED] = "red",
[IIO_MOD_LIGHT_GREEN] = "green",
[IIO_MOD_LIGHT_BLUE] = "blue",
+ [IIO_MOD_LIGHT_UV] = "uv",
[IIO_MOD_QUATERNION] = "quaternion",
[IIO_MOD_TEMP_AMBIENT] = "ambient",
[IIO_MOD_TEMP_OBJECT] = "object",
@@ -102,6 +107,10 @@
[IIO_MOD_WALKING] = "walking",
[IIO_MOD_STILL] = "still",
[IIO_MOD_ROOT_SUM_SQUARED_X_Y_Z] = "sqrt(x^2+y^2+z^2)",
+ [IIO_MOD_I] = "i",
+ [IIO_MOD_Q] = "q",
+ [IIO_MOD_CO2] = "co2",
+ [IIO_MOD_VOC] = "voc",
};
static bool event_is_known(struct iio_event_data *event)
@@ -136,6 +145,10 @@
case IIO_ENERGY:
case IIO_DISTANCE:
case IIO_VELOCITY:
+ case IIO_CONCENTRATION:
+ case IIO_RESISTANCE:
+ case IIO_PH:
+ case IIO_UVINDEX:
break;
default:
return false;
@@ -162,6 +175,7 @@
case IIO_MOD_LIGHT_RED:
case IIO_MOD_LIGHT_GREEN:
case IIO_MOD_LIGHT_BLUE:
+ case IIO_MOD_LIGHT_UV:
case IIO_MOD_QUATERNION:
case IIO_MOD_TEMP_AMBIENT:
case IIO_MOD_TEMP_OBJECT:
@@ -174,6 +188,10 @@
case IIO_MOD_WALKING:
case IIO_MOD_STILL:
case IIO_MOD_ROOT_SUM_SQUARED_X_Y_Z:
+ case IIO_MOD_I:
+ case IIO_MOD_Q:
+ case IIO_MOD_CO2:
+ case IIO_MOD_VOC:
break;
default:
return false;
diff --git a/tools/iio/iio_utils.h b/tools/iio/iio_utils.h
index e3503bf..780f201 100644
--- a/tools/iio/iio_utils.h
+++ b/tools/iio/iio_utils.h
@@ -52,6 +52,13 @@
unsigned location;
};
+static inline int iioutils_check_suffix(const char *str, const char *suffix)
+{
+ return strlen(str) >= strlen(suffix) &&
+ strncmp(str+strlen(str)-strlen(suffix),
+ suffix, strlen(suffix)) == 0;
+}
+
int iioutils_break_up_name(const char *full_name, char **generic_name);
int iioutils_get_type(unsigned *is_signed, unsigned *bytes, unsigned *bits_used,
unsigned *shift, uint64_t *mask, unsigned *be,
diff --git a/tools/perf/arch/powerpc/include/perf_regs.h b/tools/perf/arch/powerpc/include/perf_regs.h
new file mode 100644
index 0000000..75de0e9
--- /dev/null
+++ b/tools/perf/arch/powerpc/include/perf_regs.h
@@ -0,0 +1,69 @@
+#ifndef ARCH_PERF_REGS_H
+#define ARCH_PERF_REGS_H
+
+#include <stdlib.h>
+#include <linux/types.h>
+#include <asm/perf_regs.h>
+
+#define PERF_REGS_MASK ((1ULL << PERF_REG_POWERPC_MAX) - 1)
+#define PERF_REGS_MAX PERF_REG_POWERPC_MAX
+#ifdef __powerpc64__
+ #define PERF_SAMPLE_REGS_ABI PERF_SAMPLE_REGS_ABI_64
+#else
+ #define PERF_SAMPLE_REGS_ABI PERF_SAMPLE_REGS_ABI_32
+#endif
+
+#define PERF_REG_IP PERF_REG_POWERPC_NIP
+#define PERF_REG_SP PERF_REG_POWERPC_R1
+
+static const char *reg_names[] = {
+ [PERF_REG_POWERPC_R0] = "r0",
+ [PERF_REG_POWERPC_R1] = "r1",
+ [PERF_REG_POWERPC_R2] = "r2",
+ [PERF_REG_POWERPC_R3] = "r3",
+ [PERF_REG_POWERPC_R4] = "r4",
+ [PERF_REG_POWERPC_R5] = "r5",
+ [PERF_REG_POWERPC_R6] = "r6",
+ [PERF_REG_POWERPC_R7] = "r7",
+ [PERF_REG_POWERPC_R8] = "r8",
+ [PERF_REG_POWERPC_R9] = "r9",
+ [PERF_REG_POWERPC_R10] = "r10",
+ [PERF_REG_POWERPC_R11] = "r11",
+ [PERF_REG_POWERPC_R12] = "r12",
+ [PERF_REG_POWERPC_R13] = "r13",
+ [PERF_REG_POWERPC_R14] = "r14",
+ [PERF_REG_POWERPC_R15] = "r15",
+ [PERF_REG_POWERPC_R16] = "r16",
+ [PERF_REG_POWERPC_R17] = "r17",
+ [PERF_REG_POWERPC_R18] = "r18",
+ [PERF_REG_POWERPC_R19] = "r19",
+ [PERF_REG_POWERPC_R20] = "r20",
+ [PERF_REG_POWERPC_R21] = "r21",
+ [PERF_REG_POWERPC_R22] = "r22",
+ [PERF_REG_POWERPC_R23] = "r23",
+ [PERF_REG_POWERPC_R24] = "r24",
+ [PERF_REG_POWERPC_R25] = "r25",
+ [PERF_REG_POWERPC_R26] = "r26",
+ [PERF_REG_POWERPC_R27] = "r27",
+ [PERF_REG_POWERPC_R28] = "r28",
+ [PERF_REG_POWERPC_R29] = "r29",
+ [PERF_REG_POWERPC_R30] = "r30",
+ [PERF_REG_POWERPC_R31] = "r31",
+ [PERF_REG_POWERPC_NIP] = "nip",
+ [PERF_REG_POWERPC_MSR] = "msr",
+ [PERF_REG_POWERPC_ORIG_R3] = "orig_r3",
+ [PERF_REG_POWERPC_CTR] = "ctr",
+ [PERF_REG_POWERPC_LINK] = "link",
+ [PERF_REG_POWERPC_XER] = "xer",
+ [PERF_REG_POWERPC_CCR] = "ccr",
+ [PERF_REG_POWERPC_SOFTE] = "softe",
+ [PERF_REG_POWERPC_TRAP] = "trap",
+ [PERF_REG_POWERPC_DAR] = "dar",
+ [PERF_REG_POWERPC_DSISR] = "dsisr"
+};
+
+static inline const char *perf_reg_name(int id)
+{
+ return reg_names[id];
+}
+#endif /* ARCH_PERF_REGS_H */
diff --git a/tools/perf/arch/powerpc/util/Build b/tools/perf/arch/powerpc/util/Build
index c8fe207..90ad64b 100644
--- a/tools/perf/arch/powerpc/util/Build
+++ b/tools/perf/arch/powerpc/util/Build
@@ -1,6 +1,8 @@
libperf-y += header.o
libperf-y += sym-handling.o
libperf-y += kvm-stat.o
+libperf-y += perf_regs.o
libperf-$(CONFIG_DWARF) += dwarf-regs.o
libperf-$(CONFIG_DWARF) += skip-callchain-idx.o
+libperf-$(CONFIG_LIBUNWIND) += unwind-libunwind.o
diff --git a/tools/perf/arch/powerpc/util/perf_regs.c b/tools/perf/arch/powerpc/util/perf_regs.c
new file mode 100644
index 0000000..a3c3e1c
--- /dev/null
+++ b/tools/perf/arch/powerpc/util/perf_regs.c
@@ -0,0 +1,49 @@
+#include "../../perf.h"
+#include "../../util/perf_regs.h"
+
+const struct sample_reg sample_reg_masks[] = {
+ SMPL_REG(r0, PERF_REG_POWERPC_R0),
+ SMPL_REG(r1, PERF_REG_POWERPC_R1),
+ SMPL_REG(r2, PERF_REG_POWERPC_R2),
+ SMPL_REG(r3, PERF_REG_POWERPC_R3),
+ SMPL_REG(r4, PERF_REG_POWERPC_R4),
+ SMPL_REG(r5, PERF_REG_POWERPC_R5),
+ SMPL_REG(r6, PERF_REG_POWERPC_R6),
+ SMPL_REG(r7, PERF_REG_POWERPC_R7),
+ SMPL_REG(r8, PERF_REG_POWERPC_R8),
+ SMPL_REG(r9, PERF_REG_POWERPC_R9),
+ SMPL_REG(r10, PERF_REG_POWERPC_R10),
+ SMPL_REG(r11, PERF_REG_POWERPC_R11),
+ SMPL_REG(r12, PERF_REG_POWERPC_R12),
+ SMPL_REG(r13, PERF_REG_POWERPC_R13),
+ SMPL_REG(r14, PERF_REG_POWERPC_R14),
+ SMPL_REG(r15, PERF_REG_POWERPC_R15),
+ SMPL_REG(r16, PERF_REG_POWERPC_R16),
+ SMPL_REG(r17, PERF_REG_POWERPC_R17),
+ SMPL_REG(r18, PERF_REG_POWERPC_R18),
+ SMPL_REG(r19, PERF_REG_POWERPC_R19),
+ SMPL_REG(r20, PERF_REG_POWERPC_R20),
+ SMPL_REG(r21, PERF_REG_POWERPC_R21),
+ SMPL_REG(r22, PERF_REG_POWERPC_R22),
+ SMPL_REG(r23, PERF_REG_POWERPC_R23),
+ SMPL_REG(r24, PERF_REG_POWERPC_R24),
+ SMPL_REG(r25, PERF_REG_POWERPC_R25),
+ SMPL_REG(r26, PERF_REG_POWERPC_R26),
+ SMPL_REG(r27, PERF_REG_POWERPC_R27),
+ SMPL_REG(r28, PERF_REG_POWERPC_R28),
+ SMPL_REG(r29, PERF_REG_POWERPC_R29),
+ SMPL_REG(r30, PERF_REG_POWERPC_R30),
+ SMPL_REG(r31, PERF_REG_POWERPC_R31),
+ SMPL_REG(nip, PERF_REG_POWERPC_NIP),
+ SMPL_REG(msr, PERF_REG_POWERPC_MSR),
+ SMPL_REG(orig_r3, PERF_REG_POWERPC_ORIG_R3),
+ SMPL_REG(ctr, PERF_REG_POWERPC_CTR),
+ SMPL_REG(link, PERF_REG_POWERPC_LINK),
+ SMPL_REG(xer, PERF_REG_POWERPC_XER),
+ SMPL_REG(ccr, PERF_REG_POWERPC_CCR),
+ SMPL_REG(softe, PERF_REG_POWERPC_SOFTE),
+ SMPL_REG(trap, PERF_REG_POWERPC_TRAP),
+ SMPL_REG(dar, PERF_REG_POWERPC_DAR),
+ SMPL_REG(dsisr, PERF_REG_POWERPC_DSISR),
+ SMPL_REG_END
+};
diff --git a/tools/perf/arch/powerpc/util/unwind-libunwind.c b/tools/perf/arch/powerpc/util/unwind-libunwind.c
new file mode 100644
index 0000000..9e15f92
--- /dev/null
+++ b/tools/perf/arch/powerpc/util/unwind-libunwind.c
@@ -0,0 +1,96 @@
+/*
+ * Copyright 2016 Chandan Kumar, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <errno.h>
+#include <libunwind.h>
+#include <asm/perf_regs.h>
+#include "../../util/unwind.h"
+#include "../../util/debug.h"
+
+int libunwind__arch_reg_id(int regnum)
+{
+ switch (regnum) {
+ case UNW_PPC64_R0:
+ return PERF_REG_POWERPC_R0;
+ case UNW_PPC64_R1:
+ return PERF_REG_POWERPC_R1;
+ case UNW_PPC64_R2:
+ return PERF_REG_POWERPC_R2;
+ case UNW_PPC64_R3:
+ return PERF_REG_POWERPC_R3;
+ case UNW_PPC64_R4:
+ return PERF_REG_POWERPC_R4;
+ case UNW_PPC64_R5:
+ return PERF_REG_POWERPC_R5;
+ case UNW_PPC64_R6:
+ return PERF_REG_POWERPC_R6;
+ case UNW_PPC64_R7:
+ return PERF_REG_POWERPC_R7;
+ case UNW_PPC64_R8:
+ return PERF_REG_POWERPC_R8;
+ case UNW_PPC64_R9:
+ return PERF_REG_POWERPC_R9;
+ case UNW_PPC64_R10:
+ return PERF_REG_POWERPC_R10;
+ case UNW_PPC64_R11:
+ return PERF_REG_POWERPC_R11;
+ case UNW_PPC64_R12:
+ return PERF_REG_POWERPC_R12;
+ case UNW_PPC64_R13:
+ return PERF_REG_POWERPC_R13;
+ case UNW_PPC64_R14:
+ return PERF_REG_POWERPC_R14;
+ case UNW_PPC64_R15:
+ return PERF_REG_POWERPC_R15;
+ case UNW_PPC64_R16:
+ return PERF_REG_POWERPC_R16;
+ case UNW_PPC64_R17:
+ return PERF_REG_POWERPC_R17;
+ case UNW_PPC64_R18:
+ return PERF_REG_POWERPC_R18;
+ case UNW_PPC64_R19:
+ return PERF_REG_POWERPC_R19;
+ case UNW_PPC64_R20:
+ return PERF_REG_POWERPC_R20;
+ case UNW_PPC64_R21:
+ return PERF_REG_POWERPC_R21;
+ case UNW_PPC64_R22:
+ return PERF_REG_POWERPC_R22;
+ case UNW_PPC64_R23:
+ return PERF_REG_POWERPC_R23;
+ case UNW_PPC64_R24:
+ return PERF_REG_POWERPC_R24;
+ case UNW_PPC64_R25:
+ return PERF_REG_POWERPC_R25;
+ case UNW_PPC64_R26:
+ return PERF_REG_POWERPC_R26;
+ case UNW_PPC64_R27:
+ return PERF_REG_POWERPC_R27;
+ case UNW_PPC64_R28:
+ return PERF_REG_POWERPC_R28;
+ case UNW_PPC64_R29:
+ return PERF_REG_POWERPC_R29;
+ case UNW_PPC64_R30:
+ return PERF_REG_POWERPC_R30;
+ case UNW_PPC64_R31:
+ return PERF_REG_POWERPC_R31;
+ case UNW_PPC64_LR:
+ return PERF_REG_POWERPC_LINK;
+ case UNW_PPC64_CTR:
+ return PERF_REG_POWERPC_CTR;
+ case UNW_PPC64_XER:
+ return PERF_REG_POWERPC_XER;
+ case UNW_PPC64_NIP:
+ return PERF_REG_POWERPC_NIP;
+ default:
+ pr_err("unwind: invalid reg id %d\n", regnum);
+ return -EINVAL;
+ }
+ return -EINVAL;
+}
diff --git a/tools/perf/config/Makefile b/tools/perf/config/Makefile
index 1e46277..5ad0255 100644
--- a/tools/perf/config/Makefile
+++ b/tools/perf/config/Makefile
@@ -23,6 +23,12 @@
NO_PERF_REGS := 1
+# Additional ARCH settings for ppc
+ifeq ($(ARCH),powerpc)
+ NO_PERF_REGS := 0
+ LIBUNWIND_LIBS := -lunwind -lunwind-ppc64
+endif
+
# Additional ARCH settings for x86
ifeq ($(ARCH),x86)
$(call detected,CONFIG_X86)
diff --git a/tools/perf/util/perf_regs.c b/tools/perf/util/perf_regs.c
index 6b8eb13..c4023f2 100644
--- a/tools/perf/util/perf_regs.c
+++ b/tools/perf/util/perf_regs.c
@@ -12,18 +12,18 @@
int i, idx = 0;
u64 mask = regs->mask;
- if (regs->cache_mask & (1 << id))
+ if (regs->cache_mask & (1ULL << id))
goto out;
- if (!(mask & (1 << id)))
+ if (!(mask & (1ULL << id)))
return -EINVAL;
for (i = 0; i < id; i++) {
- if (mask & (1 << i))
+ if (mask & (1ULL << i))
idx++;
}
- regs->cache_mask |= (1 << id);
+ regs->cache_mask |= (1ULL << id);
regs->cache_regs[id] = regs->regs[idx];
out:
diff --git a/tools/testing/nvdimm/Kbuild b/tools/testing/nvdimm/Kbuild
index a34bfd0..7859856 100644
--- a/tools/testing/nvdimm/Kbuild
+++ b/tools/testing/nvdimm/Kbuild
@@ -7,6 +7,7 @@
ldflags-y += --wrap=iounmap
ldflags-y += --wrap=memunmap
ldflags-y += --wrap=__devm_request_region
+ldflags-y += --wrap=__devm_release_region
ldflags-y += --wrap=__request_region
ldflags-y += --wrap=__release_region
ldflags-y += --wrap=devm_memremap_pages
@@ -15,6 +16,7 @@
DRIVERS := ../../../drivers
NVDIMM_SRC := $(DRIVERS)/nvdimm
ACPI_SRC := $(DRIVERS)/acpi
+DAX_SRC := $(DRIVERS)/dax
obj-$(CONFIG_LIBNVDIMM) += libnvdimm.o
obj-$(CONFIG_BLK_DEV_PMEM) += nd_pmem.o
@@ -22,6 +24,8 @@
obj-$(CONFIG_ND_BLK) += nd_blk.o
obj-$(CONFIG_X86_PMEM_LEGACY) += nd_e820.o
obj-$(CONFIG_ACPI_NFIT) += nfit.o
+obj-$(CONFIG_DEV_DAX) += dax.o
+obj-$(CONFIG_DEV_DAX_PMEM) += dax_pmem.o
nfit-y := $(ACPI_SRC)/nfit.o
nfit-y += config_check.o
@@ -38,6 +42,12 @@
nd_e820-y := $(NVDIMM_SRC)/e820.o
nd_e820-y += config_check.o
+dax-y := $(DAX_SRC)/dax.o
+dax-y += config_check.o
+
+dax_pmem-y := $(DAX_SRC)/pmem.o
+dax_pmem-y += config_check.o
+
libnvdimm-y := $(NVDIMM_SRC)/core.o
libnvdimm-y += $(NVDIMM_SRC)/bus.o
libnvdimm-y += $(NVDIMM_SRC)/dimm_devs.o
@@ -49,6 +59,7 @@
libnvdimm-$(CONFIG_ND_CLAIM) += $(NVDIMM_SRC)/claim.o
libnvdimm-$(CONFIG_BTT) += $(NVDIMM_SRC)/btt_devs.o
libnvdimm-$(CONFIG_NVDIMM_PFN) += $(NVDIMM_SRC)/pfn_devs.o
+libnvdimm-$(CONFIG_NVDIMM_DAX) += $(NVDIMM_SRC)/dax_devs.o
libnvdimm-y += config_check.o
obj-m += test/
diff --git a/tools/testing/nvdimm/config_check.c b/tools/testing/nvdimm/config_check.c
index f2c7615..adf18bf 100644
--- a/tools/testing/nvdimm/config_check.c
+++ b/tools/testing/nvdimm/config_check.c
@@ -12,4 +12,6 @@
BUILD_BUG_ON(!IS_MODULE(CONFIG_ND_BTT));
BUILD_BUG_ON(!IS_MODULE(CONFIG_ND_BLK));
BUILD_BUG_ON(!IS_MODULE(CONFIG_ACPI_NFIT));
+ BUILD_BUG_ON(!IS_MODULE(CONFIG_DEV_DAX));
+ BUILD_BUG_ON(!IS_MODULE(CONFIG_DEV_DAX_PMEM));
}
diff --git a/tools/testing/nvdimm/test/iomap.c b/tools/testing/nvdimm/test/iomap.c
index 0c1a7e6..c842095 100644
--- a/tools/testing/nvdimm/test/iomap.c
+++ b/tools/testing/nvdimm/test/iomap.c
@@ -239,13 +239,11 @@
}
EXPORT_SYMBOL(__wrap___devm_request_region);
-void __wrap___release_region(struct resource *parent, resource_size_t start,
- resource_size_t n)
+static bool nfit_test_release_region(struct resource *parent,
+ resource_size_t start, resource_size_t n)
{
- struct nfit_test_resource *nfit_res;
-
if (parent == &iomem_resource) {
- nfit_res = get_nfit_res(start);
+ struct nfit_test_resource *nfit_res = get_nfit_res(start);
if (nfit_res) {
struct resource *res = nfit_res->res + 1;
@@ -254,11 +252,26 @@
__func__, start, n, res);
else
memset(res, 0, sizeof(*res));
- return;
+ return true;
}
}
- __release_region(parent, start, n);
+ return false;
+}
+
+void __wrap___release_region(struct resource *parent, resource_size_t start,
+ resource_size_t n)
+{
+ if (!nfit_test_release_region(parent, start, n))
+ __release_region(parent, start, n);
}
EXPORT_SYMBOL(__wrap___release_region);
+void __wrap___devm_release_region(struct device *dev, struct resource *parent,
+ resource_size_t start, resource_size_t n)
+{
+ if (!nfit_test_release_region(parent, start, n))
+ __devm_release_region(dev, parent, start, n);
+}
+EXPORT_SYMBOL(__wrap___devm_release_region);
+
MODULE_LICENSE("GPL v2");
diff --git a/tools/testing/nvdimm/test/nfit.c b/tools/testing/nvdimm/test/nfit.c
index 3187322..c919866 100644
--- a/tools/testing/nvdimm/test/nfit.c
+++ b/tools/testing/nvdimm/test/nfit.c
@@ -330,12 +330,49 @@
return 0;
}
+static int nfit_test_cmd_smart(struct nd_cmd_smart *smart, unsigned int buf_len)
+{
+ static const struct nd_smart_payload smart_data = {
+ .flags = ND_SMART_HEALTH_VALID | ND_SMART_TEMP_VALID
+ | ND_SMART_SPARES_VALID | ND_SMART_ALARM_VALID
+ | ND_SMART_USED_VALID | ND_SMART_SHUTDOWN_VALID,
+ .health = ND_SMART_NON_CRITICAL_HEALTH,
+ .temperature = 23 * 16,
+ .spares = 75,
+ .alarm_flags = ND_SMART_SPARE_TRIP | ND_SMART_TEMP_TRIP,
+ .life_used = 5,
+ .shutdown_state = 0,
+ .vendor_size = 0,
+ };
+
+ if (buf_len < sizeof(*smart))
+ return -EINVAL;
+ memcpy(smart->data, &smart_data, sizeof(smart_data));
+ return 0;
+}
+
+static int nfit_test_cmd_smart_threshold(struct nd_cmd_smart_threshold *smart_t,
+ unsigned int buf_len)
+{
+ static const struct nd_smart_threshold_payload smart_t_data = {
+ .alarm_control = ND_SMART_SPARE_TRIP | ND_SMART_TEMP_TRIP,
+ .temperature = 40 * 16,
+ .spares = 5,
+ };
+
+ if (buf_len < sizeof(*smart_t))
+ return -EINVAL;
+ memcpy(smart_t->data, &smart_t_data, sizeof(smart_t_data));
+ return 0;
+}
+
static int nfit_test_ctl(struct nvdimm_bus_descriptor *nd_desc,
struct nvdimm *nvdimm, unsigned int cmd, void *buf,
unsigned int buf_len, int *cmd_rc)
{
struct acpi_nfit_desc *acpi_desc = to_acpi_desc(nd_desc);
struct nfit_test *t = container_of(acpi_desc, typeof(*t), acpi_desc);
+ unsigned int func = cmd;
int i, rc = 0, __cmd_rc;
if (!cmd_rc)
@@ -344,8 +381,23 @@
if (nvdimm) {
struct nfit_mem *nfit_mem = nvdimm_provider_data(nvdimm);
+ unsigned long cmd_mask = nvdimm_cmd_mask(nvdimm);
- if (!nfit_mem || !test_bit(cmd, &nfit_mem->dsm_mask))
+ if (!nfit_mem)
+ return -ENOTTY;
+
+ if (cmd == ND_CMD_CALL) {
+ struct nd_cmd_pkg *call_pkg = buf;
+
+ buf_len = call_pkg->nd_size_in + call_pkg->nd_size_out;
+ buf = (void *) call_pkg->nd_payload;
+ func = call_pkg->nd_command;
+ if (call_pkg->nd_family != nfit_mem->family)
+ return -ENOTTY;
+ }
+
+ if (!test_bit(cmd, &cmd_mask)
+ || !test_bit(func, &nfit_mem->dsm_mask))
return -ENOTTY;
/* lookup label space for the given dimm */
@@ -356,7 +408,7 @@
if (i >= ARRAY_SIZE(handle))
return -ENXIO;
- switch (cmd) {
+ switch (func) {
case ND_CMD_GET_CONFIG_SIZE:
rc = nfit_test_cmd_get_config_size(buf, buf_len);
break;
@@ -368,16 +420,22 @@
rc = nfit_test_cmd_set_config_data(buf, buf_len,
t->label[i]);
break;
+ case ND_CMD_SMART:
+ rc = nfit_test_cmd_smart(buf, buf_len);
+ break;
+ case ND_CMD_SMART_THRESHOLD:
+ rc = nfit_test_cmd_smart_threshold(buf, buf_len);
+ break;
default:
return -ENOTTY;
}
} else {
struct ars_state *ars_state = &t->ars_state;
- if (!nd_desc || !test_bit(cmd, &nd_desc->dsm_mask))
+ if (!nd_desc || !test_bit(cmd, &nd_desc->cmd_mask))
return -ENOTTY;
- switch (cmd) {
+ switch (func) {
case ND_CMD_ARS_CAP:
rc = nfit_test_cmd_ars_cap(buf, buf_len);
break;
@@ -1251,13 +1309,15 @@
post_ars_status(&t->ars_state, t->spa_set_dma[0], SPA0_SIZE);
acpi_desc = &t->acpi_desc;
- set_bit(ND_CMD_GET_CONFIG_SIZE, &acpi_desc->dimm_dsm_force_en);
- set_bit(ND_CMD_GET_CONFIG_DATA, &acpi_desc->dimm_dsm_force_en);
- set_bit(ND_CMD_SET_CONFIG_DATA, &acpi_desc->dimm_dsm_force_en);
- set_bit(ND_CMD_ARS_CAP, &acpi_desc->bus_dsm_force_en);
- set_bit(ND_CMD_ARS_START, &acpi_desc->bus_dsm_force_en);
- set_bit(ND_CMD_ARS_STATUS, &acpi_desc->bus_dsm_force_en);
- set_bit(ND_CMD_CLEAR_ERROR, &acpi_desc->bus_dsm_force_en);
+ set_bit(ND_CMD_GET_CONFIG_SIZE, &acpi_desc->dimm_cmd_force_en);
+ set_bit(ND_CMD_GET_CONFIG_DATA, &acpi_desc->dimm_cmd_force_en);
+ set_bit(ND_CMD_SET_CONFIG_DATA, &acpi_desc->dimm_cmd_force_en);
+ set_bit(ND_CMD_SMART, &acpi_desc->dimm_cmd_force_en);
+ set_bit(ND_CMD_ARS_CAP, &acpi_desc->bus_cmd_force_en);
+ set_bit(ND_CMD_ARS_START, &acpi_desc->bus_cmd_force_en);
+ set_bit(ND_CMD_ARS_STATUS, &acpi_desc->bus_cmd_force_en);
+ set_bit(ND_CMD_CLEAR_ERROR, &acpi_desc->bus_cmd_force_en);
+ set_bit(ND_CMD_SMART_THRESHOLD, &acpi_desc->dimm_cmd_force_en);
}
static void nfit_test1_setup(struct nfit_test *t)
@@ -1315,10 +1375,10 @@
post_ars_status(&t->ars_state, t->spa_set_dma[0], SPA2_SIZE);
acpi_desc = &t->acpi_desc;
- set_bit(ND_CMD_ARS_CAP, &acpi_desc->bus_dsm_force_en);
- set_bit(ND_CMD_ARS_START, &acpi_desc->bus_dsm_force_en);
- set_bit(ND_CMD_ARS_STATUS, &acpi_desc->bus_dsm_force_en);
- set_bit(ND_CMD_CLEAR_ERROR, &acpi_desc->bus_dsm_force_en);
+ set_bit(ND_CMD_ARS_CAP, &acpi_desc->bus_cmd_force_en);
+ set_bit(ND_CMD_ARS_START, &acpi_desc->bus_cmd_force_en);
+ set_bit(ND_CMD_ARS_STATUS, &acpi_desc->bus_cmd_force_en);
+ set_bit(ND_CMD_CLEAR_ERROR, &acpi_desc->bus_cmd_force_en);
}
static int nfit_test_blk_do_io(struct nd_blk_region *ndbr, resource_size_t dpa,
diff --git a/tools/testing/radix-tree/Makefile b/tools/testing/radix-tree/Makefile
index 604212d..3b53046 100644
--- a/tools/testing/radix-tree/Makefile
+++ b/tools/testing/radix-tree/Makefile
@@ -3,7 +3,7 @@
LDFLAGS += -lpthread -lurcu
TARGETS = main
OFILES = main.o radix-tree.o linux.o test.o tag_check.o find_next_bit.o \
- regression1.o regression2.o regression3.o
+ regression1.o regression2.o regression3.o multiorder.o
targets: $(TARGETS)
@@ -13,7 +13,7 @@
clean:
$(RM) -f $(TARGETS) *.o radix-tree.c
-$(OFILES): *.h */*.h
+$(OFILES): *.h */*.h ../../../include/linux/radix-tree.h ../../include/linux/*.h
radix-tree.c: ../../../lib/radix-tree.c
sed -e 's/^static //' -e 's/__always_inline //' -e 's/inline //' < $< > $@
diff --git a/tools/testing/radix-tree/generated/autoconf.h b/tools/testing/radix-tree/generated/autoconf.h
new file mode 100644
index 0000000..ad18cf5
--- /dev/null
+++ b/tools/testing/radix-tree/generated/autoconf.h
@@ -0,0 +1,3 @@
+#define CONFIG_RADIX_TREE_MULTIORDER 1
+#define CONFIG_SHMEM 1
+#define CONFIG_SWAP 1
diff --git a/tools/testing/radix-tree/linux/init.h b/tools/testing/radix-tree/linux/init.h
new file mode 100644
index 0000000..360cabb
--- /dev/null
+++ b/tools/testing/radix-tree/linux/init.h
@@ -0,0 +1 @@
+/* An empty file stub that allows radix-tree.c to compile. */
diff --git a/tools/testing/radix-tree/linux/kernel.h b/tools/testing/radix-tree/linux/kernel.h
index ae013b0..be98a47b 100644
--- a/tools/testing/radix-tree/linux/kernel.h
+++ b/tools/testing/radix-tree/linux/kernel.h
@@ -7,19 +7,28 @@
#include <stddef.h>
#include <limits.h>
+#include "../../include/linux/compiler.h"
+#include "../../../include/linux/kconfig.h"
+
+#define RADIX_TREE_MAP_SHIFT 3
+
#ifndef NULL
#define NULL 0
#endif
#define BUG_ON(expr) assert(!(expr))
+#define WARN_ON(expr) assert(!(expr))
#define __init
#define __must_check
#define panic(expr)
#define printk printf
#define __force
-#define likely(c) (c)
-#define unlikely(c) (c)
#define DIV_ROUND_UP(n,d) (((n) + (d) - 1) / (d))
+#define pr_debug printk
+
+#define smp_rmb() barrier()
+#define smp_wmb() barrier()
+#define cpu_relax() barrier()
#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
@@ -28,6 +37,8 @@
(type *)( (char *)__mptr - offsetof(type, member) );})
#define min(a, b) ((a) < (b) ? (a) : (b))
+#define cond_resched() sched_yield()
+
static inline int in_interrupt(void)
{
return 0;
diff --git a/tools/testing/radix-tree/linux/slab.h b/tools/testing/radix-tree/linux/slab.h
index 57282506..6d5a347 100644
--- a/tools/testing/radix-tree/linux/slab.h
+++ b/tools/testing/radix-tree/linux/slab.h
@@ -3,7 +3,6 @@
#include <linux/types.h>
-#define GFP_KERNEL 1
#define SLAB_HWCACHE_ALIGN 1
#define SLAB_PANIC 2
#define SLAB_RECLAIM_ACCOUNT 0x00020000UL /* Objects are reclaimable */
diff --git a/tools/testing/radix-tree/linux/types.h b/tools/testing/radix-tree/linux/types.h
index 72a9d85..faa0b6f 100644
--- a/tools/testing/radix-tree/linux/types.h
+++ b/tools/testing/radix-tree/linux/types.h
@@ -1,15 +1,13 @@
#ifndef _TYPES_H
#define _TYPES_H
+#include "../../include/linux/types.h"
+
#define __rcu
#define __read_mostly
#define BITS_PER_LONG (sizeof(long) * 8)
-struct list_head {
- struct list_head *next, *prev;
-};
-
static inline void INIT_LIST_HEAD(struct list_head *list)
{
list->next = list;
@@ -22,7 +20,6 @@
#define uninitialized_var(x) x = x
-typedef unsigned gfp_t;
#include <linux/gfp.h>
#endif
diff --git a/tools/testing/radix-tree/main.c b/tools/testing/radix-tree/main.c
index 0e83cad..b7619ff 100644
--- a/tools/testing/radix-tree/main.c
+++ b/tools/testing/radix-tree/main.c
@@ -61,11 +61,11 @@
} while (!wrapped);
}
-void big_gang_check(void)
+void big_gang_check(bool long_run)
{
int i;
- for (i = 0; i < 1000; i++) {
+ for (i = 0; i < (long_run ? 1000 : 3); i++) {
__big_gang_check();
srand(time(0));
printf("%d ", i);
@@ -232,10 +232,72 @@
item_kill_tree(&tree);
}
-static void single_thread_tests(void)
+static void __locate_check(struct radix_tree_root *tree, unsigned long index,
+ unsigned order)
+{
+ struct item *item;
+ unsigned long index2;
+
+ item_insert_order(tree, index, order);
+ item = item_lookup(tree, index);
+ index2 = radix_tree_locate_item(tree, item);
+ if (index != index2) {
+ printf("index %ld order %d inserted; found %ld\n",
+ index, order, index2);
+ abort();
+ }
+}
+
+static void __order_0_locate_check(void)
+{
+ RADIX_TREE(tree, GFP_KERNEL);
+ int i;
+
+ for (i = 0; i < 50; i++)
+ __locate_check(&tree, rand() % INT_MAX, 0);
+
+ item_kill_tree(&tree);
+}
+
+static void locate_check(void)
+{
+ RADIX_TREE(tree, GFP_KERNEL);
+ unsigned order;
+ unsigned long offset, index;
+
+ __order_0_locate_check();
+
+ for (order = 0; order < 20; order++) {
+ for (offset = 0; offset < (1 << (order + 3));
+ offset += (1UL << order)) {
+ for (index = 0; index < (1UL << (order + 5));
+ index += (1UL << order)) {
+ __locate_check(&tree, index + offset, order);
+ }
+ if (radix_tree_locate_item(&tree, &tree) != -1)
+ abort();
+
+ item_kill_tree(&tree);
+ }
+ }
+
+ if (radix_tree_locate_item(&tree, &tree) != -1)
+ abort();
+ __locate_check(&tree, -1, 0);
+ if (radix_tree_locate_item(&tree, &tree) != -1)
+ abort();
+ item_kill_tree(&tree);
+}
+
+static void single_thread_tests(bool long_run)
{
int i;
+ printf("starting single_thread_tests: %d allocated\n", nr_allocated);
+ multiorder_checks();
+ printf("after multiorder_check: %d allocated\n", nr_allocated);
+ locate_check();
+ printf("after locate_check: %d allocated\n", nr_allocated);
tag_check();
printf("after tag_check: %d allocated\n", nr_allocated);
gang_check();
@@ -244,9 +306,9 @@
printf("after add_and_check: %d allocated\n", nr_allocated);
dynamic_height_check();
printf("after dynamic_height_check: %d allocated\n", nr_allocated);
- big_gang_check();
+ big_gang_check(long_run);
printf("after big_gang_check: %d allocated\n", nr_allocated);
- for (i = 0; i < 2000; i++) {
+ for (i = 0; i < (long_run ? 2000 : 3); i++) {
copy_tag_check();
printf("%d ", i);
fflush(stdout);
@@ -254,15 +316,23 @@
printf("after copy_tag_check: %d allocated\n", nr_allocated);
}
-int main(void)
+int main(int argc, char **argv)
{
+ bool long_run = false;
+ int opt;
+
+ while ((opt = getopt(argc, argv, "l")) != -1) {
+ if (opt == 'l')
+ long_run = true;
+ }
+
rcu_register_thread();
radix_tree_init();
regression1_test();
regression2_test();
regression3_test();
- single_thread_tests();
+ single_thread_tests(long_run);
sleep(1);
printf("after sleep(1): %d allocated\n", nr_allocated);
diff --git a/tools/testing/radix-tree/multiorder.c b/tools/testing/radix-tree/multiorder.c
new file mode 100644
index 0000000..39d9b95
--- /dev/null
+++ b/tools/testing/radix-tree/multiorder.c
@@ -0,0 +1,337 @@
+/*
+ * multiorder.c: Multi-order radix tree entry testing
+ * Copyright (c) 2016 Intel Corporation
+ * Author: Ross Zwisler <ross.zwisler@linux.intel.com>
+ * Author: Matthew Wilcox <matthew.r.wilcox@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ */
+#include <linux/radix-tree.h>
+#include <linux/slab.h>
+#include <linux/errno.h>
+
+#include "test.h"
+
+#define for_each_index(i, base, order) \
+ for (i = base; i < base + (1 << order); i++)
+
+static void __multiorder_tag_test(int index, int order)
+{
+ RADIX_TREE(tree, GFP_KERNEL);
+ int base, err, i;
+ unsigned long first = 0;
+
+ /* our canonical entry */
+ base = index & ~((1 << order) - 1);
+
+ printf("Multiorder tag test with index %d, canonical entry %d\n",
+ index, base);
+
+ err = item_insert_order(&tree, index, order);
+ assert(!err);
+
+ /*
+ * Verify we get collisions for covered indices. We try and fail to
+ * insert an exceptional entry so we don't leak memory via
+ * item_insert_order().
+ */
+ for_each_index(i, base, order) {
+ err = __radix_tree_insert(&tree, i, order,
+ (void *)(0xA0 | RADIX_TREE_EXCEPTIONAL_ENTRY));
+ assert(err == -EEXIST);
+ }
+
+ for_each_index(i, base, order) {
+ assert(!radix_tree_tag_get(&tree, i, 0));
+ assert(!radix_tree_tag_get(&tree, i, 1));
+ }
+
+ assert(radix_tree_tag_set(&tree, index, 0));
+
+ for_each_index(i, base, order) {
+ assert(radix_tree_tag_get(&tree, i, 0));
+ assert(!radix_tree_tag_get(&tree, i, 1));
+ }
+
+ assert(radix_tree_range_tag_if_tagged(&tree, &first, ~0UL, 10, 0, 1) == 1);
+ assert(radix_tree_tag_clear(&tree, index, 0));
+
+ for_each_index(i, base, order) {
+ assert(!radix_tree_tag_get(&tree, i, 0));
+ assert(radix_tree_tag_get(&tree, i, 1));
+ }
+
+ assert(radix_tree_tag_clear(&tree, index, 1));
+
+ assert(!radix_tree_tagged(&tree, 0));
+ assert(!radix_tree_tagged(&tree, 1));
+
+ item_kill_tree(&tree);
+}
+
+static void multiorder_tag_tests(void)
+{
+ /* test multi-order entry for indices 0-7 with no sibling pointers */
+ __multiorder_tag_test(0, 3);
+ __multiorder_tag_test(5, 3);
+
+ /* test multi-order entry for indices 8-15 with no sibling pointers */
+ __multiorder_tag_test(8, 3);
+ __multiorder_tag_test(15, 3);
+
+ /*
+ * Our order 5 entry covers indices 0-31 in a tree with height=2.
+ * This is broken up as follows:
+ * 0-7: canonical entry
+ * 8-15: sibling 1
+ * 16-23: sibling 2
+ * 24-31: sibling 3
+ */
+ __multiorder_tag_test(0, 5);
+ __multiorder_tag_test(29, 5);
+
+ /* same test, but with indices 32-63 */
+ __multiorder_tag_test(32, 5);
+ __multiorder_tag_test(44, 5);
+
+ /*
+ * Our order 8 entry covers indices 0-255 in a tree with height=3.
+ * This is broken up as follows:
+ * 0-63: canonical entry
+ * 64-127: sibling 1
+ * 128-191: sibling 2
+ * 192-255: sibling 3
+ */
+ __multiorder_tag_test(0, 8);
+ __multiorder_tag_test(190, 8);
+
+ /* same test, but with indices 256-511 */
+ __multiorder_tag_test(256, 8);
+ __multiorder_tag_test(300, 8);
+
+ __multiorder_tag_test(0x12345678UL, 8);
+}
+
+static void multiorder_check(unsigned long index, int order)
+{
+ unsigned long i;
+ unsigned long min = index & ~((1UL << order) - 1);
+ unsigned long max = min + (1UL << order);
+ RADIX_TREE(tree, GFP_KERNEL);
+
+ printf("Multiorder index %ld, order %d\n", index, order);
+
+ assert(item_insert_order(&tree, index, order) == 0);
+
+ for (i = min; i < max; i++) {
+ struct item *item = item_lookup(&tree, i);
+ assert(item != 0);
+ assert(item->index == index);
+ }
+ for (i = 0; i < min; i++)
+ item_check_absent(&tree, i);
+ for (i = max; i < 2*max; i++)
+ item_check_absent(&tree, i);
+ for (i = min; i < max; i++) {
+ static void *entry = (void *)
+ (0xA0 | RADIX_TREE_EXCEPTIONAL_ENTRY);
+ assert(radix_tree_insert(&tree, i, entry) == -EEXIST);
+ }
+
+ assert(item_delete(&tree, index) != 0);
+
+ for (i = 0; i < 2*max; i++)
+ item_check_absent(&tree, i);
+}
+
+static void multiorder_shrink(unsigned long index, int order)
+{
+ unsigned long i;
+ unsigned long max = 1 << order;
+ RADIX_TREE(tree, GFP_KERNEL);
+ struct radix_tree_node *node;
+
+ printf("Multiorder shrink index %ld, order %d\n", index, order);
+
+ assert(item_insert_order(&tree, 0, order) == 0);
+
+ node = tree.rnode;
+
+ assert(item_insert(&tree, index) == 0);
+ assert(node != tree.rnode);
+
+ assert(item_delete(&tree, index) != 0);
+ assert(node == tree.rnode);
+
+ for (i = 0; i < max; i++) {
+ struct item *item = item_lookup(&tree, i);
+ assert(item != 0);
+ assert(item->index == 0);
+ }
+ for (i = max; i < 2*max; i++)
+ item_check_absent(&tree, i);
+
+ if (!item_delete(&tree, 0)) {
+ printf("failed to delete index %ld (order %d)\n", index, order); abort();
+ }
+
+ for (i = 0; i < 2*max; i++)
+ item_check_absent(&tree, i);
+}
+
+static void multiorder_insert_bug(void)
+{
+ RADIX_TREE(tree, GFP_KERNEL);
+
+ item_insert(&tree, 0);
+ radix_tree_tag_set(&tree, 0, 0);
+ item_insert_order(&tree, 3 << 6, 6);
+
+ item_kill_tree(&tree);
+}
+
+void multiorder_iteration(void)
+{
+ RADIX_TREE(tree, GFP_KERNEL);
+ struct radix_tree_iter iter;
+ void **slot;
+ int i, j, err;
+
+ printf("Multiorder iteration test\n");
+
+#define NUM_ENTRIES 11
+ int index[NUM_ENTRIES] = {0, 2, 4, 8, 16, 32, 34, 36, 64, 72, 128};
+ int order[NUM_ENTRIES] = {1, 1, 2, 3, 4, 1, 0, 1, 3, 0, 7};
+
+ for (i = 0; i < NUM_ENTRIES; i++) {
+ err = item_insert_order(&tree, index[i], order[i]);
+ assert(!err);
+ }
+
+ for (j = 0; j < 256; j++) {
+ for (i = 0; i < NUM_ENTRIES; i++)
+ if (j <= (index[i] | ((1 << order[i]) - 1)))
+ break;
+
+ radix_tree_for_each_slot(slot, &tree, &iter, j) {
+ int height = order[i] / RADIX_TREE_MAP_SHIFT;
+ int shift = height * RADIX_TREE_MAP_SHIFT;
+ int mask = (1 << order[i]) - 1;
+
+ assert(iter.index >= (index[i] &~ mask));
+ assert(iter.index <= (index[i] | mask));
+ assert(iter.shift == shift);
+ i++;
+ }
+ }
+
+ item_kill_tree(&tree);
+}
+
+void multiorder_tagged_iteration(void)
+{
+ RADIX_TREE(tree, GFP_KERNEL);
+ struct radix_tree_iter iter;
+ void **slot;
+ unsigned long first = 0;
+ int i, j;
+
+ printf("Multiorder tagged iteration test\n");
+
+#define MT_NUM_ENTRIES 9
+ int index[MT_NUM_ENTRIES] = {0, 2, 4, 16, 32, 40, 64, 72, 128};
+ int order[MT_NUM_ENTRIES] = {1, 0, 2, 4, 3, 1, 3, 0, 7};
+
+#define TAG_ENTRIES 7
+ int tag_index[TAG_ENTRIES] = {0, 4, 16, 40, 64, 72, 128};
+
+ for (i = 0; i < MT_NUM_ENTRIES; i++)
+ assert(!item_insert_order(&tree, index[i], order[i]));
+
+ assert(!radix_tree_tagged(&tree, 1));
+
+ for (i = 0; i < TAG_ENTRIES; i++)
+ assert(radix_tree_tag_set(&tree, tag_index[i], 1));
+
+ for (j = 0; j < 256; j++) {
+ int mask, k;
+
+ for (i = 0; i < TAG_ENTRIES; i++) {
+ for (k = i; index[k] < tag_index[i]; k++)
+ ;
+ if (j <= (index[k] | ((1 << order[k]) - 1)))
+ break;
+ }
+
+ radix_tree_for_each_tagged(slot, &tree, &iter, j, 1) {
+ for (k = i; index[k] < tag_index[i]; k++)
+ ;
+ mask = (1 << order[k]) - 1;
+
+ assert(iter.index >= (tag_index[i] &~ mask));
+ assert(iter.index <= (tag_index[i] | mask));
+ i++;
+ }
+ }
+
+ radix_tree_range_tag_if_tagged(&tree, &first, ~0UL,
+ MT_NUM_ENTRIES, 1, 2);
+
+ for (j = 0; j < 256; j++) {
+ int mask, k;
+
+ for (i = 0; i < TAG_ENTRIES; i++) {
+ for (k = i; index[k] < tag_index[i]; k++)
+ ;
+ if (j <= (index[k] | ((1 << order[k]) - 1)))
+ break;
+ }
+
+ radix_tree_for_each_tagged(slot, &tree, &iter, j, 2) {
+ for (k = i; index[k] < tag_index[i]; k++)
+ ;
+ mask = (1 << order[k]) - 1;
+
+ assert(iter.index >= (tag_index[i] &~ mask));
+ assert(iter.index <= (tag_index[i] | mask));
+ i++;
+ }
+ }
+
+ first = 1;
+ radix_tree_range_tag_if_tagged(&tree, &first, ~0UL,
+ MT_NUM_ENTRIES, 1, 0);
+ i = 0;
+ radix_tree_for_each_tagged(slot, &tree, &iter, 0, 0) {
+ assert(iter.index == tag_index[i]);
+ i++;
+ }
+
+ item_kill_tree(&tree);
+}
+
+void multiorder_checks(void)
+{
+ int i;
+
+ for (i = 0; i < 20; i++) {
+ multiorder_check(200, i);
+ multiorder_check(0, i);
+ multiorder_check((1UL << i) + 1, i);
+ }
+
+ for (i = 0; i < 15; i++)
+ multiorder_shrink((1UL << (i + RADIX_TREE_MAP_SHIFT)), i);
+
+ multiorder_insert_bug();
+ multiorder_tag_tests();
+ multiorder_iteration();
+ multiorder_tagged_iteration();
+}
diff --git a/tools/testing/radix-tree/regression2.c b/tools/testing/radix-tree/regression2.c
index 5d2fa28..63bf347 100644
--- a/tools/testing/radix-tree/regression2.c
+++ b/tools/testing/radix-tree/regression2.c
@@ -51,13 +51,6 @@
#include "regression.h"
-#ifdef __KERNEL__
-#define RADIX_TREE_MAP_SHIFT (CONFIG_BASE_SMALL ? 4 : 6)
-#else
-#define RADIX_TREE_MAP_SHIFT 3 /* For more stressful testing */
-#endif
-
-#define RADIX_TREE_MAP_SIZE (1UL << RADIX_TREE_MAP_SHIFT)
#define PAGECACHE_TAG_DIRTY 0
#define PAGECACHE_TAG_WRITEBACK 1
#define PAGECACHE_TAG_TOWRITE 2
diff --git a/tools/testing/radix-tree/tag_check.c b/tools/testing/radix-tree/tag_check.c
index 83136be..b7447ce 100644
--- a/tools/testing/radix-tree/tag_check.c
+++ b/tools/testing/radix-tree/tag_check.c
@@ -12,6 +12,7 @@
static void
__simple_checks(struct radix_tree_root *tree, unsigned long index, int tag)
{
+ unsigned long first = 0;
int ret;
item_check_absent(tree, index);
@@ -22,6 +23,10 @@
item_tag_set(tree, index, tag);
ret = item_tag_get(tree, index, tag);
assert(ret != 0);
+ ret = radix_tree_range_tag_if_tagged(tree, &first, ~0UL, 10, tag, !tag);
+ assert(ret == 1);
+ ret = item_tag_get(tree, index, !tag);
+ assert(ret != 0);
ret = item_delete(tree, index);
assert(ret != 0);
item_insert(tree, index);
@@ -304,6 +309,7 @@
struct item *items[BATCH];
RADIX_TREE(tree, GFP_KERNEL);
int ret;
+ unsigned long first = 0;
item_insert(&tree, 0);
item_tag_set(&tree, 0, 0);
@@ -313,6 +319,10 @@
assert(ret == 0);
verify_tag_consistency(&tree, 0);
verify_tag_consistency(&tree, 1);
+ ret = radix_tree_range_tag_if_tagged(&tree, &first, 10, 10, 0, 1);
+ assert(ret == 1);
+ ret = radix_tree_gang_lookup_tag(&tree, (void **)items, 0, BATCH, 1);
+ assert(ret == 1);
item_kill_tree(&tree);
}
diff --git a/tools/testing/radix-tree/test.c b/tools/testing/radix-tree/test.c
index 2bebf34..a6e8099 100644
--- a/tools/testing/radix-tree/test.c
+++ b/tools/testing/radix-tree/test.c
@@ -24,14 +24,21 @@
return radix_tree_tag_get(root, index, tag);
}
-int __item_insert(struct radix_tree_root *root, struct item *item)
+int __item_insert(struct radix_tree_root *root, struct item *item,
+ unsigned order)
{
- return radix_tree_insert(root, item->index, item);
+ return __radix_tree_insert(root, item->index, order, item);
}
int item_insert(struct radix_tree_root *root, unsigned long index)
{
- return __item_insert(root, item_create(index));
+ return __item_insert(root, item_create(index), 0);
+}
+
+int item_insert_order(struct radix_tree_root *root, unsigned long index,
+ unsigned order)
+{
+ return __item_insert(root, item_create(index), order);
}
int item_delete(struct radix_tree_root *root, unsigned long index)
@@ -136,13 +143,13 @@
}
static int verify_node(struct radix_tree_node *slot, unsigned int tag,
- unsigned int height, int tagged)
+ int tagged)
{
int anyset = 0;
int i;
int j;
- slot = indirect_to_ptr(slot);
+ slot = entry_to_node(slot);
/* Verify consistency at this level */
for (i = 0; i < RADIX_TREE_TAG_LONGS; i++) {
@@ -152,7 +159,8 @@
}
}
if (tagged != anyset) {
- printf("tag: %u, height %u, tagged: %d, anyset: %d\n", tag, height, tagged, anyset);
+ printf("tag: %u, shift %u, tagged: %d, anyset: %d\n",
+ tag, slot->shift, tagged, anyset);
for (j = 0; j < RADIX_TREE_MAX_TAGS; j++) {
printf("tag %d: ", j);
for (i = 0; i < RADIX_TREE_TAG_LONGS; i++)
@@ -164,10 +172,10 @@
assert(tagged == anyset);
/* Go for next level */
- if (height > 1) {
+ if (slot->shift > 0) {
for (i = 0; i < RADIX_TREE_MAP_SIZE; i++)
if (slot->slots[i])
- if (verify_node(slot->slots[i], tag, height - 1,
+ if (verify_node(slot->slots[i], tag,
!!test_bit(i, slot->tags[tag]))) {
printf("Failure at off %d\n", i);
for (j = 0; j < RADIX_TREE_MAX_TAGS; j++) {
@@ -184,9 +192,10 @@
void verify_tag_consistency(struct radix_tree_root *root, unsigned int tag)
{
- if (!root->height)
+ struct radix_tree_node *node = root->rnode;
+ if (!radix_tree_is_internal_node(node))
return;
- verify_node(root->rnode, tag, root->height, !!root_tag_get(root, tag));
+ verify_node(node, tag, !!root_tag_get(root, tag));
}
void item_kill_tree(struct radix_tree_root *root)
@@ -211,9 +220,19 @@
void tree_verify_min_height(struct radix_tree_root *root, int maxindex)
{
- assert(radix_tree_maxindex(root->height) >= maxindex);
- if (root->height > 1)
- assert(radix_tree_maxindex(root->height-1) < maxindex);
- else if (root->height == 1)
- assert(radix_tree_maxindex(root->height-1) <= maxindex);
+ unsigned shift;
+ struct radix_tree_node *node = root->rnode;
+ if (!radix_tree_is_internal_node(node)) {
+ assert(maxindex == 0);
+ return;
+ }
+
+ node = entry_to_node(node);
+ assert(maxindex <= node_maxindex(node));
+
+ shift = node->shift;
+ if (shift > 0)
+ assert(maxindex > shift_maxindex(shift - RADIX_TREE_MAP_SHIFT));
+ else
+ assert(maxindex > 0);
}
diff --git a/tools/testing/radix-tree/test.h b/tools/testing/radix-tree/test.h
index 4e1d95f..e851313 100644
--- a/tools/testing/radix-tree/test.h
+++ b/tools/testing/radix-tree/test.h
@@ -8,8 +8,11 @@
};
struct item *item_create(unsigned long index);
-int __item_insert(struct radix_tree_root *root, struct item *item);
+int __item_insert(struct radix_tree_root *root, struct item *item,
+ unsigned order);
int item_insert(struct radix_tree_root *root, unsigned long index);
+int item_insert_order(struct radix_tree_root *root, unsigned long index,
+ unsigned order);
int item_delete(struct radix_tree_root *root, unsigned long index);
struct item *item_lookup(struct radix_tree_root *root, unsigned long index);
@@ -23,6 +26,7 @@
void item_kill_tree(struct radix_tree_root *root);
void tag_check(void);
+void multiorder_checks(void);
struct item *
item_tag_set(struct radix_tree_root *root, unsigned long index, int tag);
@@ -35,6 +39,7 @@
extern int nr_allocated;
/* Normally private parts of lib/radix-tree.c */
-void *indirect_to_ptr(void *ptr);
+void radix_tree_dump(struct radix_tree_root *root);
int root_tag_get(struct radix_tree_root *root, unsigned int tag);
-unsigned long radix_tree_maxindex(unsigned int height);
+unsigned long node_maxindex(struct radix_tree_node *);
+unsigned long shift_maxindex(unsigned int shift);
diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
index da48812..4c6a0bf 100755
--- a/tools/testing/selftests/ftrace/ftracetest
+++ b/tools/testing/selftests/ftrace/ftracetest
@@ -88,7 +88,12 @@
# Parameters
DEBUGFS_DIR=`grep debugfs /proc/mounts | cut -f2 -d' ' | head -1`
-TRACING_DIR=$DEBUGFS_DIR/tracing
+if [ -z "$DEBUGFS_DIR" ]; then
+ TRACING_DIR=`grep tracefs /proc/mounts | cut -f2 -d' ' | head -1`
+else
+ TRACING_DIR=$DEBUGFS_DIR/tracing
+fi
+
TOP_DIR=`absdir $0`
TEST_DIR=$TOP_DIR/test.d
TEST_CASES=`find_testcases $TEST_DIR`
@@ -102,7 +107,7 @@
[ $DEBUG -ne 0 ] && set -x
# Verify parameters
-if [ -z "$DEBUGFS_DIR" -o ! -d "$TRACING_DIR" ]; then
+if [ -z "$TRACING_DIR" -o ! -d "$TRACING_DIR" ]; then
errexit "No ftrace directory found"
fi
diff --git a/tools/testing/selftests/ftrace/test.d/event/event-pid.tc b/tools/testing/selftests/ftrace/test.d/event/event-pid.tc
new file mode 100644
index 0000000..d4ab27b5
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/event/event-pid.tc
@@ -0,0 +1,72 @@
+#!/bin/sh
+# description: event tracing - restricts events based on pid
+
+do_reset() {
+ echo > set_event
+ echo > set_event_pid
+ echo 0 > options/event-fork
+ clear_trace
+}
+
+fail() { #msg
+ do_reset
+ echo $1
+ exit $FAIL
+}
+
+yield() {
+ ping localhost -c 1 || sleep .001 || usleep 1 || sleep 1
+}
+
+if [ ! -f set_event -o ! -d events/sched ]; then
+ echo "event tracing is not supported"
+ exit_unsupported
+fi
+
+if [ ! -f set_event_pid ]; then
+ echo "event pid filtering is not supported"
+ exit_unsupported
+fi
+
+reset_tracer
+do_reset
+
+echo 1 > events/sched/sched_switch/enable
+
+yield
+
+count=`cat trace | grep sched_switch | wc -l`
+if [ $count -eq 0 ]; then
+ fail "sched_switch events are not recorded"
+fi
+
+do_reset
+
+read mypid rest < /proc/self/stat
+
+echo $mypid > set_event_pid
+echo 'sched:sched_switch' > set_event
+
+yield
+
+count=`cat trace | grep sched_switch | grep -v "pid=$mypid" | wc -l`
+if [ $count -ne 0 ]; then
+ fail "sched_switch events from other task are recorded"
+fi
+
+do_reset
+
+echo $mypid > set_event_pid
+echo 1 > options/event-fork
+echo 1 > events/sched/sched_switch/enable
+
+yield
+
+count=`cat trace | grep sched_switch | grep -v "pid=$mypid" | wc -l`
+if [ $count -eq 0 ]; then
+ fail "sched_switch events from other task are not recorded"
+fi
+
+do_reset
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/functions b/tools/testing/selftests/ftrace/test.d/functions
index 5d8cd06..c37262f 100644
--- a/tools/testing/selftests/ftrace/test.d/functions
+++ b/tools/testing/selftests/ftrace/test.d/functions
@@ -14,3 +14,12 @@
reset_tracer() { # reset the current tracer
echo nop > current_tracer
}
+
+reset_trigger() { # reset all current setting triggers
+ grep -v ^# events/*/*/trigger |
+ while read line; do
+ cmd=`echo $line | cut -f2- -d: | cut -f1 -d" "`
+ echo "!$cmd" > `echo $line | cut -f1 -d:`
+ done
+}
+
diff --git a/tools/testing/selftests/ftrace/test.d/instances/instance-event.tc b/tools/testing/selftests/ftrace/test.d/instances/instance-event.tc
new file mode 100644
index 0000000..4c5a061
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/instances/instance-event.tc
@@ -0,0 +1,138 @@
+#!/bin/sh
+# description: Test creation and deletion of trace instances while setting an event
+
+if [ ! -d instances ] ; then
+ echo "no instance directory with this kernel"
+ exit_unsupported;
+fi
+
+fail() { # mesg
+ rmdir foo 2>/dev/null
+ echo $1
+ set -e
+ exit $FAIL
+}
+
+cd instances
+
+# we don't want to fail on error
+set +e
+
+mkdir x
+rmdir x
+result=$?
+
+if [ $result -ne 0 ]; then
+ echo "instance rmdir not supported"
+ exit_unsupported
+fi
+
+instance_slam() {
+ while :; do
+ mkdir foo 2> /dev/null
+ rmdir foo 2> /dev/null
+ done
+}
+
+instance_read() {
+ while :; do
+ cat foo/trace 1> /dev/null 2>&1
+ done
+}
+
+instance_set() {
+ while :; do
+ echo 1 > foo/events/sched/sched_switch
+ done 2> /dev/null
+}
+
+instance_slam &
+p1=$!
+echo $p1
+
+instance_set &
+p2=$!
+echo $p2
+
+instance_read &
+p3=$!
+echo $p3
+
+sleep 1
+
+kill -1 $p3
+kill -1 $p2
+kill -1 $p1
+
+echo "Wait for processes to finish"
+wait $p1 $p2 $p3
+echo "all processes finished, wait for cleanup"
+sleep 1
+
+mkdir foo
+ls foo > /dev/null
+rmdir foo
+if [ -d foo ]; then
+ fail "foo still exists"
+fi
+exit 0
+
+
+
+
+instance_slam() {
+ while :; do
+ mkdir x
+ mkdir y
+ mkdir z
+ rmdir x
+ rmdir y
+ rmdir z
+ done 2>/dev/null
+}
+
+instance_slam &
+p1=$!
+echo $p1
+
+instance_slam &
+p2=$!
+echo $p2
+
+instance_slam &
+p3=$!
+echo $p3
+
+instance_slam &
+p4=$!
+echo $p4
+
+instance_slam &
+p5=$!
+echo $p5
+
+ls -lR >/dev/null
+sleep 1
+
+kill -1 $p1
+kill -1 $p2
+kill -1 $p3
+kill -1 $p4
+kill -1 $p5
+
+echo "Wait for processes to finish"
+wait $p1 $p2 $p3 $p4 $p5
+echo "all processes finished, wait for cleanup"
+
+mkdir x y z
+ls x y z
+rmdir x y z
+for d in x y z; do
+ if [ -d $d ]; then
+ fail "instance $d still exists"
+ fi
+done
+
+set -e
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-eventonoff.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-eventonoff.tc
new file mode 100644
index 0000000..1a94450
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-eventonoff.tc
@@ -0,0 +1,64 @@
+#!/bin/sh
+# description: event trigger - test event enable/disable trigger
+
+do_reset() {
+ reset_trigger
+ echo > set_event
+ clear_trace
+}
+
+fail() { #msg
+ do_reset
+ echo $1
+ exit $FAIL
+}
+
+if [ ! -f set_event -o ! -d events/sched ]; then
+ echo "event tracing is not supported"
+ exit_unsupported
+fi
+
+if [ ! -f events/sched/sched_process_fork/trigger ]; then
+ echo "event trigger is not supported"
+ exit_unsupported
+fi
+
+reset_tracer
+do_reset
+
+FEATURE=`grep enable_event events/sched/sched_process_fork/trigger`
+if [ -z "$FEATURE" ]; then
+ echo "event enable/disable trigger is not supported"
+ exit_unsupported
+fi
+
+echo "Test enable_event trigger"
+echo 0 > events/sched/sched_switch/enable
+echo 'enable_event:sched:sched_switch' > events/sched/sched_process_fork/trigger
+( echo "forked")
+if [ `cat events/sched/sched_switch/enable` != '1*' ]; then
+ fail "enable_event trigger on sched_process_fork did not work"
+fi
+
+reset_trigger
+
+echo "Test disable_event trigger"
+echo 1 > events/sched/sched_switch/enable
+echo 'disable_event:sched:sched_switch' > events/sched/sched_process_fork/trigger
+( echo "forked")
+if [ `cat events/sched/sched_switch/enable` != '0*' ]; then
+ fail "disable_event trigger on sched_process_fork did not work"
+fi
+
+reset_trigger
+
+echo "Test semantic error for event enable/disable trigger"
+! echo 'enable_event:nogroup:noevent' > events/sched/sched_process_fork/trigger
+! echo 'disable_event+1' > events/sched/sched_process_fork/trigger
+echo 'enable_event:sched:sched_switch' > events/sched/sched_process_fork/trigger
+! echo 'enable_event:sched:sched_switch' > events/sched/sched_process_fork/trigger
+! echo 'disable_event:sched:sched_switch' > events/sched/sched_process_fork/trigger
+
+do_reset
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-filter.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-filter.tc
new file mode 100644
index 0000000..514e466
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-filter.tc
@@ -0,0 +1,59 @@
+#!/bin/sh
+# description: event trigger - test trigger filter
+
+do_reset() {
+ reset_trigger
+ echo > set_event
+ clear_trace
+}
+
+fail() { #msg
+ do_reset
+ echo $1
+ exit $FAIL
+}
+
+if [ ! -f set_event -o ! -d events/sched ]; then
+ echo "event tracing is not supported"
+ exit_unsupported
+fi
+
+if [ ! -f events/sched/sched_process_fork/trigger ]; then
+ echo "event trigger is not supported"
+ exit_unsupported
+fi
+
+reset_tracer
+do_reset
+
+echo "Test trigger filter"
+echo 1 > tracing_on
+echo 'traceoff if child_pid == 0' > events/sched/sched_process_fork/trigger
+( echo "forked")
+if [ `cat tracing_on` -ne 1 ]; then
+ fail "traceoff trigger on sched_process_fork did not work"
+fi
+
+reset_trigger
+
+echo "Test semantic error for trigger filter"
+! echo 'traceoff if a' > events/sched/sched_process_fork/trigger
+! echo 'traceoff if common_pid=0' > events/sched/sched_process_fork/trigger
+! echo 'traceoff if common_pid==b' > events/sched/sched_process_fork/trigger
+echo 'traceoff if common_pid == 0' > events/sched/sched_process_fork/trigger
+echo '!traceoff' > events/sched/sched_process_fork/trigger
+! echo 'traceoff if common_pid == child_pid' > events/sched/sched_process_fork/trigger
+echo 'traceoff if common_pid <= 0' > events/sched/sched_process_fork/trigger
+echo '!traceoff' > events/sched/sched_process_fork/trigger
+echo 'traceoff if common_pid >= 0' > events/sched/sched_process_fork/trigger
+echo '!traceoff' > events/sched/sched_process_fork/trigger
+echo 'traceoff if parent_pid >= 0 && child_pid >= 0' > events/sched/sched_process_fork/trigger
+echo '!traceoff' > events/sched/sched_process_fork/trigger
+echo 'traceoff if parent_pid >= 0 || child_pid >= 0' > events/sched/sched_process_fork/trigger
+echo '!traceoff' > events/sched/sched_process_fork/trigger
+
+
+
+do_reset
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-hist-mod.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-hist-mod.tc
new file mode 100644
index 0000000..c2b61c4
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-hist-mod.tc
@@ -0,0 +1,75 @@
+#!/bin/sh
+# description: event trigger - test histogram modifiers
+
+do_reset() {
+ reset_trigger
+ echo > set_event
+ clear_trace
+}
+
+fail() { #msg
+ do_reset
+ echo $1
+ exit $FAIL
+}
+
+if [ ! -f set_event -o ! -d events/sched ]; then
+ echo "event tracing is not supported"
+ exit_unsupported
+fi
+
+if [ ! -f events/sched/sched_process_fork/trigger ]; then
+ echo "event trigger is not supported"
+ exit_unsupported
+fi
+
+reset_tracer
+do_reset
+
+FEATURE=`grep hist events/sched/sched_process_fork/trigger`
+if [ -z "$FEATURE" ]; then
+ echo "hist trigger is not supported"
+ exit_unsupported
+fi
+
+echo "Test histogram with execname modifier"
+
+echo 'hist:keys=common_pid.execname' > events/sched/sched_process_fork/trigger
+for i in `seq 1 10` ; do ( echo "forked" > /dev/null); done
+COMM=`cat /proc/$$/comm`
+grep "common_pid: $COMM" events/sched/sched_process_fork/hist > /dev/null || \
+ fail "execname modifier on sched_process_fork did not work"
+
+reset_trigger
+
+echo "Test histogram with hex modifier"
+
+echo 'hist:keys=parent_pid.hex' > events/sched/sched_process_fork/trigger
+for i in `seq 1 10` ; do ( echo "forked" > /dev/null); done
+# Note that $$ is the parent pid. $PID is current PID.
+HEX=`printf %x $PID`
+grep "parent_pid: $HEX" events/sched/sched_process_fork/hist > /dev/null || \
+ fail "hex modifier on sched_process_fork did not work"
+
+reset_trigger
+
+echo "Test histogram with syscall modifier"
+
+echo 'hist:keys=id.syscall' > events/raw_syscalls/sys_exit/trigger
+for i in `seq 1 10` ; do ( echo "forked" > /dev/null); done
+grep "id: sys_" events/raw_syscalls/sys_exit/hist > /dev/null || \
+ fail "syscall modifier on raw_syscalls/sys_exit did not work"
+
+
+reset_trigger
+
+echo "Test histgram with log2 modifier"
+
+echo 'hist:keys=bytes_req.log2' > events/kmem/kmalloc/trigger
+for i in `seq 1 10` ; do ( echo "forked" > /dev/null); done
+grep 'bytes_req: ~ 2^[0-9]*' events/kmem/kmalloc/hist > /dev/null || \
+ fail "log2 modifier on kmem/kmalloc did not work"
+
+do_reset
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-hist.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-hist.tc
new file mode 100644
index 0000000..b2902d4
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-hist.tc
@@ -0,0 +1,83 @@
+#!/bin/sh
+# description: event trigger - test histogram trigger
+
+do_reset() {
+ reset_trigger
+ echo > set_event
+ clear_trace
+}
+
+fail() { #msg
+ do_reset
+ echo $1
+ exit $FAIL
+}
+
+if [ ! -f set_event -o ! -d events/sched ]; then
+ echo "event tracing is not supported"
+ exit_unsupported
+fi
+
+if [ ! -f events/sched/sched_process_fork/trigger ]; then
+ echo "event trigger is not supported"
+ exit_unsupported
+fi
+
+reset_tracer
+do_reset
+
+FEATURE=`grep hist events/sched/sched_process_fork/trigger`
+if [ -z "$FEATURE" ]; then
+ echo "hist trigger is not supported"
+ exit_unsupported
+fi
+
+echo "Test histogram basic tigger"
+
+echo 'hist:keys=parent_pid:vals=child_pid' > events/sched/sched_process_fork/trigger
+for i in `seq 1 10` ; do ( echo "forked" > /dev/null); done
+grep parent_pid events/sched/sched_process_fork/hist > /dev/null || \
+ fail "hist trigger on sched_process_fork did not work"
+grep child events/sched/sched_process_fork/hist > /dev/null || \
+ fail "hist trigger on sched_process_fork did not work"
+
+reset_trigger
+
+echo "Test histogram with compound keys"
+
+echo 'hist:keys=parent_pid,child_pid' > events/sched/sched_process_fork/trigger
+for i in `seq 1 10` ; do ( echo "forked" > /dev/null); done
+grep '^{ parent_pid:.*, child_pid:.*}' events/sched/sched_process_fork/hist > /dev/null || \
+ fail "compound keys on sched_process_fork did not work"
+
+reset_trigger
+
+echo "Test histogram with string key"
+
+echo 'hist:keys=parent_comm' > events/sched/sched_process_fork/trigger
+for i in `seq 1 10` ; do ( echo "forked" > /dev/null); done
+COMM=`cat /proc/$$/comm`
+grep "parent_comm: $COMM" events/sched/sched_process_fork/hist > /dev/null || \
+ fail "string key on sched_process_fork did not work"
+
+reset_trigger
+
+echo "Test histogram with sort key"
+
+echo 'hist:keys=parent_pid,child_pid:sort=child_pid.ascending' > events/sched/sched_process_fork/trigger
+for i in `seq 1 10` ; do ( echo "forked" > /dev/null); done
+
+check_inc() {
+ while [ $# -gt 1 ]; do
+ [ $1 -gt $2 ] && return 1
+ shift 1
+ done
+ return 0
+}
+check_inc `grep -o "child_pid:[[:space:]]*[[:digit:]]*" \
+ events/sched/sched_process_fork/hist | cut -d: -f2 ` ||
+ fail "sort param on sched_process_fork did not work"
+
+do_reset
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-multihist.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-multihist.tc
new file mode 100644
index 0000000..03c4a46
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-multihist.tc
@@ -0,0 +1,73 @@
+#!/bin/sh
+# description: event trigger - test multiple histogram triggers
+
+do_reset() {
+ reset_trigger
+ echo > set_event
+ clear_trace
+}
+
+fail() { #msg
+ do_reset
+ echo $1
+ exit $FAIL
+}
+
+if [ ! -f set_event -o ! -d events/sched ]; then
+ echo "event tracing is not supported"
+ exit_unsupported
+fi
+
+if [ ! -f events/sched/sched_process_fork/trigger ]; then
+ echo "event trigger is not supported"
+ exit_unsupported
+fi
+
+reset_tracer
+do_reset
+
+FEATURE=`grep hist events/sched/sched_process_fork/trigger`
+if [ -z "$FEATURE" ]; then
+ echo "hist trigger is not supported"
+ exit_unsupported
+fi
+
+reset_trigger
+
+echo "Test histogram multiple tiggers"
+
+echo 'hist:keys=parent_pid:vals=child_pid' > events/sched/sched_process_fork/trigger
+echo 'hist:keys=parent_comm:vals=child_pid' >> events/sched/sched_process_fork/trigger
+for i in `seq 1 10` ; do ( echo "forked" > /dev/null); done
+grep parent_pid events/sched/sched_process_fork/hist > /dev/null || \
+ fail "hist trigger on sched_process_fork did not work"
+grep child events/sched/sched_process_fork/hist > /dev/null || \
+ fail "hist trigger on sched_process_fork did not work"
+COMM=`cat /proc/$$/comm`
+grep "parent_comm: $COMM" events/sched/sched_process_fork/hist > /dev/null || \
+ fail "string key on sched_process_fork did not work"
+
+reset_trigger
+
+echo "Test histogram with its name"
+
+echo 'hist:name=test_hist:keys=common_pid' > events/sched/sched_process_fork/trigger
+for i in `seq 1 10` ; do ( echo "forked" > /dev/null); done
+grep test_hist events/sched/sched_process_fork/hist > /dev/null || \
+ fail "named event on sched_process_fork did not work"
+
+echo "Test same named histogram on different events"
+
+echo 'hist:name=test_hist:keys=common_pid' > events/sched/sched_process_exit/trigger
+for i in `seq 1 10` ; do ( echo "forked" > /dev/null); done
+grep test_hist events/sched/sched_process_exit/hist > /dev/null || \
+ fail "named event on sched_process_fork did not work"
+
+diffs=`diff events/sched/sched_process_exit/hist events/sched/sched_process_fork/hist | wc -l`
+test $diffs -eq 0 || fail "Same name histograms are not same"
+
+reset_trigger
+
+do_reset
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-snapshot.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-snapshot.tc
new file mode 100644
index 0000000..f84b80d
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-snapshot.tc
@@ -0,0 +1,56 @@
+#!/bin/sh
+# description: event trigger - test snapshot-trigger
+
+do_reset() {
+ reset_trigger
+ echo > set_event
+ clear_trace
+}
+
+fail() { #msg
+ do_reset
+ echo $1
+ exit $FAIL
+}
+
+if [ ! -f set_event -o ! -d events/sched ]; then
+ echo "event tracing is not supported"
+ exit_unsupported
+fi
+
+if [ ! -f events/sched/sched_process_fork/trigger ]; then
+ echo "event trigger is not supported"
+ exit_unsupported
+fi
+
+reset_tracer
+do_reset
+
+FEATURE=`grep snapshot events/sched/sched_process_fork/trigger`
+if [ -z "$FEATURE" ]; then
+ echo "snapshot trigger is not supported"
+ exit_unsupported
+fi
+
+echo "Test snapshot tigger"
+echo 0 > snapshot
+echo 1 > events/sched/sched_process_fork/enable
+( echo "forked")
+echo 'snapshot:1' > events/sched/sched_process_fork/trigger
+( echo "forked")
+grep sched_process_fork snapshot > /dev/null || \
+ fail "snapshot trigger on sched_process_fork did not work"
+
+reset_trigger
+echo 0 > snapshot
+echo 0 > events/sched/sched_process_fork/enable
+
+echo "Test snapshot semantic errors"
+
+! echo "snapshot+1" > events/sched/sched_process_fork/trigger
+echo "snapshot" > events/sched/sched_process_fork/trigger
+! echo "snapshot" > events/sched/sched_process_fork/trigger
+
+do_reset
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-stacktrace.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-stacktrace.tc
new file mode 100644
index 0000000..9fa23b0
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-stacktrace.tc
@@ -0,0 +1,53 @@
+#!/bin/sh
+# description: event trigger - test stacktrace-trigger
+
+do_reset() {
+ reset_trigger
+ echo > set_event
+ clear_trace
+}
+
+fail() { #msg
+ do_reset
+ echo $1
+ exit $FAIL
+}
+
+if [ ! -f set_event -o ! -d events/sched ]; then
+ echo "event tracing is not supported"
+ exit_unsupported
+fi
+
+if [ ! -f events/sched/sched_process_fork/trigger ]; then
+ echo "event trigger is not supported"
+ exit_unsupported
+fi
+
+reset_tracer
+do_reset
+
+FEATURE=`grep stacktrace events/sched/sched_process_fork/trigger`
+if [ -z "$FEATURE" ]; then
+ echo "stacktrace trigger is not supported"
+ exit_unsupported
+fi
+
+echo "Test stacktrace tigger"
+echo 0 > trace
+echo 0 > options/stacktrace
+echo 'stacktrace' > events/sched/sched_process_fork/trigger
+( echo "forked")
+grep "<stack trace>" trace > /dev/null || \
+ fail "stacktrace trigger on sched_process_fork did not work"
+
+reset_trigger
+
+echo "Test stacktrace semantic errors"
+
+! echo "stacktrace:foo" > events/sched/sched_process_fork/trigger
+echo "stacktrace" > events/sched/sched_process_fork/trigger
+! echo "stacktrace" > events/sched/sched_process_fork/trigger
+
+do_reset
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-traceonoff.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-traceonoff.tc
new file mode 100644
index 0000000..87648e5
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-traceonoff.tc
@@ -0,0 +1,58 @@
+#!/bin/sh
+# description: event trigger - test traceon/off trigger
+
+do_reset() {
+ reset_trigger
+ echo > set_event
+ clear_trace
+}
+
+fail() { #msg
+ do_reset
+ echo $1
+ exit $FAIL
+}
+
+if [ ! -f set_event -o ! -d events/sched ]; then
+ echo "event tracing is not supported"
+ exit_unsupported
+fi
+
+if [ ! -f events/sched/sched_process_fork/trigger ]; then
+ echo "event trigger is not supported"
+ exit_unsupported
+fi
+
+reset_tracer
+do_reset
+
+echo "Test traceoff trigger"
+echo 1 > tracing_on
+echo 'traceoff' > events/sched/sched_process_fork/trigger
+( echo "forked")
+if [ `cat tracing_on` -ne 0 ]; then
+ fail "traceoff trigger on sched_process_fork did not work"
+fi
+
+reset_trigger
+
+echo "Test traceon trigger"
+echo 0 > tracing_on
+echo 'traceon' > events/sched/sched_process_fork/trigger
+( echo "forked")
+if [ `cat tracing_on` -ne 1 ]; then
+ fail "traceoff trigger on sched_process_fork did not work"
+fi
+
+reset_trigger
+
+echo "Test semantic error for traceoff/on trigger"
+! echo 'traceoff:badparam' > events/sched/sched_process_fork/trigger
+! echo 'traceoff+0' > events/sched/sched_process_fork/trigger
+echo 'traceon' > events/sched/sched_process_fork/trigger
+! echo 'traceon' > events/sched/sched_process_fork/trigger
+! echo 'traceoff' > events/sched/sched_process_fork/trigger
+
+do_reset
+
+exit 0
diff --git a/tools/testing/selftests/intel_pstate/run.sh b/tools/testing/selftests/intel_pstate/run.sh
index bdaf37e..7868c10 100755
--- a/tools/testing/selftests/intel_pstate/run.sh
+++ b/tools/testing/selftests/intel_pstate/run.sh
@@ -32,7 +32,7 @@
max_cpus=$(($(nproc)-1))
# compile programs
-gcc -o aperf aperf.c -lm
+gcc aperf.c -Wall -D_GNU_SOURCE -o aperf -lm
[ $? -ne 0 ] && echo "Problem compiling aperf.c." && exit 1
gcc -o msr msr.c -lm
[ $? -ne 0 ] && echo "Problem compiling msr.c." && exit 1
diff --git a/tools/testing/selftests/powerpc/Makefile b/tools/testing/selftests/powerpc/Makefile
index b08f77c..4ca83fe 100644
--- a/tools/testing/selftests/powerpc/Makefile
+++ b/tools/testing/selftests/powerpc/Makefile
@@ -14,6 +14,7 @@
SUB_DIRS = benchmarks \
copyloops \
+ context_switch \
dscr \
mm \
pmu \
diff --git a/tools/testing/selftests/powerpc/context_switch/.gitignore b/tools/testing/selftests/powerpc/context_switch/.gitignore
new file mode 100644
index 0000000..c1431af
--- /dev/null
+++ b/tools/testing/selftests/powerpc/context_switch/.gitignore
@@ -0,0 +1 @@
+cp_abort
diff --git a/tools/testing/selftests/powerpc/context_switch/Makefile b/tools/testing/selftests/powerpc/context_switch/Makefile
new file mode 100644
index 0000000..e164d14
--- /dev/null
+++ b/tools/testing/selftests/powerpc/context_switch/Makefile
@@ -0,0 +1,10 @@
+TEST_PROGS := cp_abort
+
+all: $(TEST_PROGS)
+
+$(TEST_PROGS): ../harness.c ../utils.c
+
+include ../../lib.mk
+
+clean:
+ rm -f $(TEST_PROGS)
diff --git a/tools/testing/selftests/powerpc/context_switch/cp_abort.c b/tools/testing/selftests/powerpc/context_switch/cp_abort.c
new file mode 100644
index 0000000..5a5b55a
--- /dev/null
+++ b/tools/testing/selftests/powerpc/context_switch/cp_abort.c
@@ -0,0 +1,110 @@
+/*
+ * Adapted from Anton Blanchard's context switch microbenchmark.
+ *
+ * Copyright 2009, Anton Blanchard, IBM Corporation.
+ * Copyright 2016, Mikey Neuling, Chris Smart, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ *
+ * This program tests the copy paste abort functionality of a P9
+ * (or later) by setting up two processes on the same CPU, one
+ * which executes the copy instruction and the other which
+ * executes paste.
+ *
+ * The paste instruction should never succeed, as the cp_abort
+ * instruction is called by the kernel during a context switch.
+ *
+ */
+
+#define _GNU_SOURCE
+
+#include <stdio.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include "utils.h"
+#include <sched.h>
+
+#define READ_FD 0
+#define WRITE_FD 1
+
+#define NUM_LOOPS 1000
+
+/* This defines the "paste" instruction from Power ISA 3.0 Book II, section 4.4. */
+#define PASTE(RA, RB, L, RC) \
+ .long (0x7c00070c | (RA) << (31-15) | (RB) << (31-20) | (L) << (31-10) | (RC) << (31-31))
+
+int paste(void *i)
+{
+ int cr;
+
+ asm volatile(str(PASTE(0, %1, 1, 1))";"
+ "mfcr %0;"
+ : "=r" (cr)
+ : "b" (i)
+ : "memory"
+ );
+ return cr;
+}
+
+/* This defines the "copy" instruction from Power ISA 3.0 Book II, section 4.4. */
+#define COPY(RA, RB, L) \
+ .long (0x7c00060c | (RA) << (31-15) | (RB) << (31-20) | (L) << (31-10))
+
+void copy(void *i)
+{
+ asm volatile(str(COPY(0, %0, 1))";"
+ :
+ : "b" (i)
+ : "memory"
+ );
+}
+
+int test_cp_abort(void)
+{
+ /* 128 bytes for a full cache line */
+ char buf[128] __cacheline_aligned;
+ cpu_set_t cpuset;
+ int fd1[2], fd2[2], pid;
+ char c;
+
+ /* only run this test on a P9 or later */
+ SKIP_IF(!have_hwcap2(PPC_FEATURE2_ARCH_3_00));
+
+ /*
+ * Run both processes on the same CPU, so that copy is more likely
+ * to leak into a paste.
+ */
+ CPU_ZERO(&cpuset);
+ CPU_SET(pick_online_cpu(), &cpuset);
+ FAIL_IF(sched_setaffinity(0, sizeof(cpuset), &cpuset));
+
+ FAIL_IF(pipe(fd1) || pipe(fd2));
+
+ pid = fork();
+ FAIL_IF(pid < 0);
+
+ if (!pid) {
+ for (int i = 0; i < NUM_LOOPS; i++) {
+ FAIL_IF((write(fd1[WRITE_FD], &c, 1)) != 1);
+ FAIL_IF((read(fd2[READ_FD], &c, 1)) != 1);
+ /* A paste succeeds if CR0 EQ bit is set */
+ FAIL_IF(paste(buf) & 0x20000000);
+ }
+ } else {
+ for (int i = 0; i < NUM_LOOPS; i++) {
+ FAIL_IF((read(fd1[READ_FD], &c, 1)) != 1);
+ copy(buf);
+ FAIL_IF((write(fd2[WRITE_FD], &c, 1) != 1));
+ }
+ }
+ return 0;
+
+}
+
+int main(int argc, char *argv[])
+{
+ return test_harness(test_cp_abort, "cp_abort");
+}
diff --git a/tools/testing/selftests/powerpc/mm/subpage_prot.c b/tools/testing/selftests/powerpc/mm/subpage_prot.c
index 440180f..35ade74 100644
--- a/tools/testing/selftests/powerpc/mm/subpage_prot.c
+++ b/tools/testing/selftests/powerpc/mm/subpage_prot.c
@@ -73,7 +73,7 @@
want_fault |= (subpage == ((page + 1) % 16));
if (faulted != want_fault) {
- printf("Failed at 0x%p (p=%ld,sp=%ld,w=%d), want=%s, got=%s !\n",
+ printf("Failed at %p (p=%ld,sp=%ld,w=%d), want=%s, got=%s !\n",
addr, page, subpage, write,
want_fault ? "fault" : "pass",
faulted ? "fault" : "pass");
@@ -82,7 +82,7 @@
if (faulted) {
if (dar != addr) {
- printf("Fault expected at 0x%p and happened at 0x%p !\n",
+ printf("Fault expected at %p and happened at %p !\n",
addr, dar);
}
faulted = 0;
@@ -162,7 +162,7 @@
mallocblock = (void *)align;
- printf("allocated malloc block of 0x%lx bytes at 0x%p\n",
+ printf("allocated malloc block of 0x%lx bytes at %p\n",
mallocsize, mallocblock);
printf("testing malloc block...\n");
@@ -197,7 +197,7 @@
perror("failed to map file");
return 1;
}
- printf("allocated %s for 0x%lx bytes at 0x%p\n",
+ printf("allocated %s for 0x%lx bytes at %p\n",
file_name, filesize, fileblock);
printf("testing file map...\n");
@@ -207,14 +207,16 @@
int main(int argc, char *argv[])
{
- test_harness(test_anon, "subpage_prot_anon");
+ int rc;
+
+ rc = test_harness(test_anon, "subpage_prot_anon");
+ if (rc)
+ return rc;
if (argc > 1)
file_name = argv[1];
else
file_name = "tempfile";
- test_harness(test_file, "subpage_prot_file");
-
- return 0;
+ return test_harness(test_file, "subpage_prot_file");
}
diff --git a/tools/testing/selftests/powerpc/pmu/ebb/ebb.c b/tools/testing/selftests/powerpc/pmu/ebb/ebb.c
index e67452f..46681fe 100644
--- a/tools/testing/selftests/powerpc/pmu/ebb/ebb.c
+++ b/tools/testing/selftests/powerpc/pmu/ebb/ebb.c
@@ -15,7 +15,6 @@
#include <sys/ioctl.h>
#include "trace.h"
-#include "reg.h"
#include "ebb.h"
diff --git a/tools/testing/selftests/powerpc/pmu/ebb/reg.h b/tools/testing/selftests/powerpc/pmu/ebb/reg.h
deleted file mode 100644
index 5921b0d..0000000
--- a/tools/testing/selftests/powerpc/pmu/ebb/reg.h
+++ /dev/null
@@ -1,49 +0,0 @@
-/*
- * Copyright 2014, Michael Ellerman, IBM Corp.
- * Licensed under GPLv2.
- */
-
-#ifndef _SELFTESTS_POWERPC_REG_H
-#define _SELFTESTS_POWERPC_REG_H
-
-#define __stringify_1(x) #x
-#define __stringify(x) __stringify_1(x)
-
-#define mfspr(rn) ({unsigned long rval; \
- asm volatile("mfspr %0," __stringify(rn) \
- : "=r" (rval)); rval; })
-#define mtspr(rn, v) asm volatile("mtspr " __stringify(rn) ",%0" : \
- : "r" ((unsigned long)(v)) \
- : "memory")
-
-#define mb() asm volatile("sync" : : : "memory");
-
-#define SPRN_MMCR2 769
-#define SPRN_MMCRA 770
-#define SPRN_MMCR0 779
-#define MMCR0_PMAO 0x00000080
-#define MMCR0_PMAE 0x04000000
-#define MMCR0_FC 0x80000000
-#define SPRN_EBBHR 804
-#define SPRN_EBBRR 805
-#define SPRN_BESCR 806 /* Branch event status & control register */
-#define SPRN_BESCRS 800 /* Branch event status & control set (1 bits set to 1) */
-#define SPRN_BESCRSU 801 /* Branch event status & control set upper */
-#define SPRN_BESCRR 802 /* Branch event status & control REset (1 bits set to 0) */
-#define SPRN_BESCRRU 803 /* Branch event status & control REset upper */
-
-#define BESCR_PMEO 0x1 /* PMU Event-based exception Occurred */
-#define BESCR_PME (0x1ul << 32) /* PMU Event-based exception Enable */
-
-#define SPRN_PMC1 771
-#define SPRN_PMC2 772
-#define SPRN_PMC3 773
-#define SPRN_PMC4 774
-#define SPRN_PMC5 775
-#define SPRN_PMC6 776
-
-#define SPRN_SIAR 780
-#define SPRN_SDAR 781
-#define SPRN_SIER 768
-
-#endif /* _SELFTESTS_POWERPC_REG_H */
diff --git a/tools/testing/selftests/powerpc/pmu/ebb/reg_access_test.c b/tools/testing/selftests/powerpc/pmu/ebb/reg_access_test.c
index 5b1188f..f923228 100644
--- a/tools/testing/selftests/powerpc/pmu/ebb/reg_access_test.c
+++ b/tools/testing/selftests/powerpc/pmu/ebb/reg_access_test.c
@@ -7,7 +7,6 @@
#include <stdlib.h>
#include "ebb.h"
-#include "reg.h"
/*
diff --git a/tools/testing/selftests/powerpc/reg.h b/tools/testing/selftests/powerpc/reg.h
new file mode 100644
index 0000000..65bfdee
--- /dev/null
+++ b/tools/testing/selftests/powerpc/reg.h
@@ -0,0 +1,55 @@
+/*
+ * Copyright 2014, Michael Ellerman, IBM Corp.
+ * Licensed under GPLv2.
+ */
+
+#ifndef _SELFTESTS_POWERPC_REG_H
+#define _SELFTESTS_POWERPC_REG_H
+
+#define __stringify_1(x) #x
+#define __stringify(x) __stringify_1(x)
+
+#define mfspr(rn) ({unsigned long rval; \
+ asm volatile("mfspr %0," _str(rn) \
+ : "=r" (rval)); rval; })
+#define mtspr(rn, v) asm volatile("mtspr " _str(rn) ",%0" : \
+ : "r" ((unsigned long)(v)) \
+ : "memory")
+
+#define mb() asm volatile("sync" : : : "memory");
+
+#define SPRN_MMCR2 769
+#define SPRN_MMCRA 770
+#define SPRN_MMCR0 779
+#define MMCR0_PMAO 0x00000080
+#define MMCR0_PMAE 0x04000000
+#define MMCR0_FC 0x80000000
+#define SPRN_EBBHR 804
+#define SPRN_EBBRR 805
+#define SPRN_BESCR 806 /* Branch event status & control register */
+#define SPRN_BESCRS 800 /* Branch event status & control set (1 bits set to 1) */
+#define SPRN_BESCRSU 801 /* Branch event status & control set upper */
+#define SPRN_BESCRR 802 /* Branch event status & control REset (1 bits set to 0) */
+#define SPRN_BESCRRU 803 /* Branch event status & control REset upper */
+
+#define BESCR_PMEO 0x1 /* PMU Event-based exception Occurred */
+#define BESCR_PME (0x1ul << 32) /* PMU Event-based exception Enable */
+
+#define SPRN_PMC1 771
+#define SPRN_PMC2 772
+#define SPRN_PMC3 773
+#define SPRN_PMC4 774
+#define SPRN_PMC5 775
+#define SPRN_PMC6 776
+
+#define SPRN_SIAR 780
+#define SPRN_SDAR 781
+#define SPRN_SIER 768
+
+#define SPRN_TEXASR 0x82
+#define SPRN_TFIAR 0x81 /* Transaction Failure Inst Addr */
+#define SPRN_TFHAR 0x80 /* Transaction Failure Handler Addr */
+#define TEXASR_FS 0x08000000
+#define SPRN_TAR 0x32f
+
+#endif /* _SELFTESTS_POWERPC_REG_H */
diff --git a/tools/testing/selftests/powerpc/tm/.gitignore b/tools/testing/selftests/powerpc/tm/.gitignore
index 7d0f14b..bb942db 100644
--- a/tools/testing/selftests/powerpc/tm/.gitignore
+++ b/tools/testing/selftests/powerpc/tm/.gitignore
@@ -3,3 +3,6 @@
tm-signal-msr-resv
tm-signal-stack
tm-vmxcopy
+tm-fork
+tm-tar
+tm-tmspr
diff --git a/tools/testing/selftests/powerpc/tm/Makefile b/tools/testing/selftests/powerpc/tm/Makefile
index 737f72c..d0505db 100644
--- a/tools/testing/selftests/powerpc/tm/Makefile
+++ b/tools/testing/selftests/powerpc/tm/Makefile
@@ -1,4 +1,4 @@
-TEST_PROGS := tm-resched-dscr tm-syscall tm-signal-msr-resv tm-signal-stack tm-vmxcopy
+TEST_PROGS := tm-resched-dscr tm-syscall tm-signal-msr-resv tm-signal-stack tm-vmxcopy tm-fork tm-tar tm-tmspr
all: $(TEST_PROGS)
@@ -6,6 +6,7 @@
tm-syscall: tm-syscall-asm.S
tm-syscall: CFLAGS += -mhtm -I../../../../../usr/include
+tm-tmspr: CFLAGS += -pthread
include ../../lib.mk
diff --git a/tools/testing/selftests/powerpc/tm/tm-fork.c b/tools/testing/selftests/powerpc/tm/tm-fork.c
new file mode 100644
index 0000000..8d48579
--- /dev/null
+++ b/tools/testing/selftests/powerpc/tm/tm-fork.c
@@ -0,0 +1,42 @@
+/*
+ * Copyright 2015, Michael Neuling, IBM Corp.
+ * Licensed under GPLv2.
+ *
+ * Edited: Rashmica Gupta, Nov 2015
+ *
+ * This test does a fork syscall inside a transaction. Basic sniff test
+ * to see if we can enter the kernel during a transaction.
+ */
+
+#include <errno.h>
+#include <inttypes.h>
+#include <pthread.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+#include "utils.h"
+#include "tm.h"
+
+int test_fork(void)
+{
+ SKIP_IF(!have_htm());
+
+ asm __volatile__(
+ "tbegin.;"
+ "blt 1f; "
+ "li 0, 2;" /* fork syscall */
+ "sc ;"
+ "tend.;"
+ "1: ;"
+ : : : "memory", "r0");
+ /* If we reach here, we've passed. Otherwise we've probably crashed
+ * the kernel */
+
+ return 0;
+}
+
+int main(int argc, char *argv[])
+{
+ return test_harness(test_fork, "tm_fork");
+}
diff --git a/tools/testing/selftests/powerpc/tm/tm-resched-dscr.c b/tools/testing/selftests/powerpc/tm/tm-resched-dscr.c
index 8fde93d..d9c49f4 100644
--- a/tools/testing/selftests/powerpc/tm/tm-resched-dscr.c
+++ b/tools/testing/selftests/powerpc/tm/tm-resched-dscr.c
@@ -31,12 +31,6 @@
#include "utils.h"
#include "tm.h"
-#define TBEGIN ".long 0x7C00051D ;"
-#define TEND ".long 0x7C00055D ;"
-#define TCHECK ".long 0x7C00059C ;"
-#define TSUSPEND ".long 0x7C0005DD ;"
-#define TRESUME ".long 0x7C2005DD ;"
-#define SPRN_TEXASR 0x82
#define SPRN_DSCR 0x03
int test_body(void)
@@ -55,13 +49,13 @@
"mtspr %[sprn_dscr], 3;"
/* start and suspend a transaction */
- TBEGIN
+ "tbegin.;"
"beq 1f;"
- TSUSPEND
+ "tsuspend.;"
/* hard loop until the transaction becomes doomed */
"2: ;"
- TCHECK
+ "tcheck 0;"
"bc 4, 0, 2b;"
/* record DSCR and TEXASR */
@@ -70,8 +64,8 @@
"mfspr 3, %[sprn_texasr];"
"std 3, %[texasr];"
- TRESUME
- TEND
+ "tresume.;"
+ "tend.;"
"li %[rv], 0;"
"1: ;"
: [rv]"=r"(rv), [dscr2]"=m"(dscr2), [texasr]"=m"(texasr)
diff --git a/tools/testing/selftests/powerpc/tm/tm-signal-stack.c b/tools/testing/selftests/powerpc/tm/tm-signal-stack.c
index e44a238..1f0eb56 100644
--- a/tools/testing/selftests/powerpc/tm/tm-signal-stack.c
+++ b/tools/testing/selftests/powerpc/tm/tm-signal-stack.c
@@ -60,9 +60,9 @@
exit(1);
asm volatile("li 1, 0 ;" /* stack ptr == NULL */
"1:"
- ".long 0x7C00051D ;" /* tbegin */
+ "tbegin.;"
"beq 1b ;" /* retry forever */
- ".long 0x7C0005DD ; ;" /* tsuspend */
+ "tsuspend.;"
"ld 2, 0(1) ;" /* trigger segv" */
: : : "memory");
diff --git a/tools/testing/selftests/powerpc/tm/tm-tar.c b/tools/testing/selftests/powerpc/tm/tm-tar.c
new file mode 100644
index 0000000..2d2fcc2b
--- /dev/null
+++ b/tools/testing/selftests/powerpc/tm/tm-tar.c
@@ -0,0 +1,90 @@
+/*
+ * Copyright 2015, Michael Neuling, IBM Corp.
+ * Licensed under GPLv2.
+ * Original: Michael Neuling 19/7/2013
+ * Edited: Rashmica Gupta 01/12/2015
+ *
+ * Do some transactions, see if the tar is corrupted.
+ * If the transaction is aborted, the TAR should be rolled back to the
+ * checkpointed value before the transaction began. The value written to
+ * TAR in suspended mode should only remain in TAR if the transaction
+ * completes.
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <string.h>
+
+#include "tm.h"
+#include "utils.h"
+
+int num_loops = 10000;
+
+int test_tar(void)
+{
+ int i;
+
+ SKIP_IF(!have_htm());
+
+ for (i = 0; i < num_loops; i++)
+ {
+ uint64_t result = 0;
+ asm __volatile__(
+ "li 7, 1;"
+ "mtspr %[tar], 7;" /* tar = 1 */
+ "tbegin.;"
+ "beq 3f;"
+ "li 4, 0x7000;" /* Loop lots, to use time */
+ "2:;" /* Start loop */
+ "li 7, 2;"
+ "mtspr %[tar], 7;" /* tar = 2 */
+ "tsuspend.;"
+ "li 7, 3;"
+ "mtspr %[tar], 7;" /* tar = 3 */
+ "tresume.;"
+ "subi 4, 4, 1;"
+ "cmpdi 4, 0;"
+ "bne 2b;"
+ "tend.;"
+
+ /* Transaction sucess! TAR should be 3 */
+ "mfspr 7, %[tar];"
+ "ori %[res], 7, 4;" // res = 3|4 = 7
+ "b 4f;"
+
+ /* Abort handler. TAR should be rolled back to 1 */
+ "3:;"
+ "mfspr 7, %[tar];"
+ "ori %[res], 7, 8;" // res = 1|8 = 9
+ "4:;"
+
+ : [res]"=r"(result)
+ : [tar]"i"(SPRN_TAR)
+ : "memory", "r0", "r4", "r7");
+
+ /* If result is anything else other than 7 or 9, the tar
+ * value must have been corrupted. */
+ if ((result != 7) && (result != 9))
+ return 1;
+ }
+ return 0;
+}
+
+int main(int argc, char *argv[])
+{
+ /* A low number of iterations (eg 100) can cause a false pass */
+ if (argc > 1) {
+ if (strcmp(argv[1], "-h") == 0) {
+ printf("Syntax:\n\t%s [<num loops>]\n",
+ argv[0]);
+ return 1;
+ } else {
+ num_loops = atoi(argv[1]);
+ }
+ }
+
+ printf("Starting, %d loops\n", num_loops);
+
+ return test_harness(test_tar, "tm_tar");
+}
diff --git a/tools/testing/selftests/powerpc/tm/tm-tmspr.c b/tools/testing/selftests/powerpc/tm/tm-tmspr.c
new file mode 100644
index 0000000..2bda81c
--- /dev/null
+++ b/tools/testing/selftests/powerpc/tm/tm-tmspr.c
@@ -0,0 +1,143 @@
+/*
+ * Copyright 2015, Michael Neuling, IBM Corp.
+ * Licensed under GPLv2.
+ *
+ * Original: Michael Neuling 3/4/2014
+ * Modified: Rashmica Gupta 8/12/2015
+ *
+ * Check if any of the Transaction Memory SPRs get corrupted.
+ * - TFIAR - stores address of location of transaction failure
+ * - TFHAR - stores address of software failure handler (if transaction
+ * fails)
+ * - TEXASR - lots of info about the transacion(s)
+ *
+ * (1) create more threads than cpus
+ * (2) in each thread:
+ * (a) set TFIAR and TFHAR a unique value
+ * (b) loop for awhile, continually checking to see if
+ * either register has been corrupted.
+ *
+ * (3) Loop:
+ * (a) begin transaction
+ * (b) abort transaction
+ * (c) check TEXASR to see if FS has been corrupted
+ *
+ */
+
+#define _GNU_SOURCE
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <pthread.h>
+#include <string.h>
+
+#include "utils.h"
+#include "tm.h"
+
+int num_loops = 10000;
+int passed = 1;
+
+void tfiar_tfhar(void *in)
+{
+ int i, cpu;
+ unsigned long tfhar, tfhar_rd, tfiar, tfiar_rd;
+ cpu_set_t cpuset;
+
+ CPU_ZERO(&cpuset);
+ cpu = (unsigned long)in >> 1;
+ CPU_SET(cpu, &cpuset);
+ sched_setaffinity(0, sizeof(cpuset), &cpuset);
+
+ /* TFIAR: Last bit has to be high so userspace can read register */
+ tfiar = ((unsigned long)in) + 1;
+ tfiar += 2;
+ mtspr(SPRN_TFIAR, tfiar);
+
+ /* TFHAR: Last two bits are reserved */
+ tfhar = ((unsigned long)in);
+ tfhar &= ~0x3UL;
+ tfhar += 4;
+ mtspr(SPRN_TFHAR, tfhar);
+
+ for (i = 0; i < num_loops; i++) {
+ tfhar_rd = mfspr(SPRN_TFHAR);
+ tfiar_rd = mfspr(SPRN_TFIAR);
+ if ( (tfhar != tfhar_rd) || (tfiar != tfiar_rd) ) {
+ passed = 0;
+ return;
+ }
+ }
+ return;
+}
+
+void texasr(void *in)
+{
+ unsigned long i;
+ uint64_t result = 0;
+
+ for (i = 0; i < num_loops; i++) {
+ asm __volatile__(
+ "tbegin.;"
+ "beq 3f ;"
+ "tabort. 0 ;"
+ "tend.;"
+
+ /* Abort handler */
+ "3: ;"
+ ::: "memory");
+
+ /* Check the TEXASR */
+ result = mfspr(SPRN_TEXASR);
+ if ((result & TEXASR_FS) == 0) {
+ passed = 0;
+ return;
+ }
+ }
+ return;
+}
+
+int test_tmspr()
+{
+ pthread_t thread;
+ int thread_num;
+ unsigned long i;
+
+ SKIP_IF(!have_htm());
+
+ /* To cause some context switching */
+ thread_num = 10 * sysconf(_SC_NPROCESSORS_ONLN);
+
+ /* Test TFIAR and TFHAR */
+ for (i = 0 ; i < thread_num ; i += 2){
+ if (pthread_create(&thread, NULL, (void*)tfiar_tfhar, (void *)i))
+ return EXIT_FAILURE;
+ }
+ if (pthread_join(thread, NULL) != 0)
+ return EXIT_FAILURE;
+
+ /* Test TEXASR */
+ for (i = 0 ; i < thread_num ; i++){
+ if (pthread_create(&thread, NULL, (void*)texasr, (void *)i))
+ return EXIT_FAILURE;
+ }
+ if (pthread_join(thread, NULL) != 0)
+ return EXIT_FAILURE;
+
+ if (passed)
+ return 0;
+ else
+ return 1;
+}
+
+int main(int argc, char *argv[])
+{
+ if (argc > 1) {
+ if (strcmp(argv[1], "-h") == 0) {
+ printf("Syntax:\t [<num loops>]\n");
+ return 0;
+ } else {
+ num_loops = atoi(argv[1]);
+ }
+ }
+ return test_harness(test_tmspr, "tm_tmspr");
+}
diff --git a/tools/testing/selftests/powerpc/utils.h b/tools/testing/selftests/powerpc/utils.h
index 175ac6a..a985cfa 100644
--- a/tools/testing/selftests/powerpc/utils.h
+++ b/tools/testing/selftests/powerpc/utils.h
@@ -6,9 +6,12 @@
#ifndef _SELFTESTS_POWERPC_UTILS_H
#define _SELFTESTS_POWERPC_UTILS_H
+#define __cacheline_aligned __attribute__((aligned(128)))
+
#include <stdint.h>
#include <stdbool.h>
#include <linux/auxvec.h>
+#include "reg.h"
/* Avoid headaches with PRI?64 - just use %ll? always */
typedef unsigned long long u64;
@@ -54,4 +57,9 @@
#define _str(s) #s
#define str(s) _str(s)
+/* POWER9 feature */
+#ifndef PPC_FEATURE2_ARCH_3_00
+#define PPC_FEATURE2_ARCH_3_00 0x00800000
+#endif
+
#endif /* _SELFTESTS_POWERPC_UTILS_H */
diff --git a/tools/testing/selftests/seccomp/seccomp_bpf.c b/tools/testing/selftests/seccomp/seccomp_bpf.c
index 150829d..7947e56 100644
--- a/tools/testing/selftests/seccomp/seccomp_bpf.c
+++ b/tools/testing/selftests/seccomp/seccomp_bpf.c
@@ -5,6 +5,7 @@
* Test code for seccomp bpf.
*/
+#include <sys/types.h>
#include <asm/siginfo.h>
#define __have_siginfo_t 1
#define __have_sigval_t 1
@@ -14,7 +15,6 @@
#include <linux/filter.h>
#include <sys/prctl.h>
#include <sys/ptrace.h>
-#include <sys/types.h>
#include <sys/user.h>
#include <linux/prctl.h>
#include <linux/ptrace.h>
@@ -1242,6 +1242,12 @@
# define ARCH_REGS s390_regs
# define SYSCALL_NUM gprs[2]
# define SYSCALL_RET gprs[2]
+#elif defined(__mips__)
+# define ARCH_REGS struct pt_regs
+# define SYSCALL_NUM regs[2]
+# define SYSCALL_SYSCALL_NUM regs[4]
+# define SYSCALL_RET regs[2]
+# define SYSCALL_NUM_RET_SHARE_REG
#else
# error "Do not know how to find your architecture's registers and syscalls"
#endif
@@ -1249,7 +1255,7 @@
/* Use PTRACE_GETREGS and PTRACE_SETREGS when available. This is useful for
* architectures without HAVE_ARCH_TRACEHOOK (e.g. User-mode Linux).
*/
-#if defined(__x86_64__) || defined(__i386__)
+#if defined(__x86_64__) || defined(__i386__) || defined(__mips__)
#define HAVE_GETREGS
#endif
@@ -1273,6 +1279,10 @@
}
#endif
+#if defined(__mips__)
+ if (regs.SYSCALL_NUM == __NR_O32_Linux)
+ return regs.SYSCALL_SYSCALL_NUM;
+#endif
return regs.SYSCALL_NUM;
}
@@ -1297,6 +1307,13 @@
{
regs.SYSCALL_NUM = syscall;
}
+#elif defined(__mips__)
+ {
+ if (regs.SYSCALL_NUM == __NR_O32_Linux)
+ regs.SYSCALL_SYSCALL_NUM = syscall;
+ else
+ regs.SYSCALL_NUM = syscall;
+ }
#elif defined(__arm__)
# ifndef PTRACE_SET_SYSCALL
@@ -1327,7 +1344,11 @@
/* If syscall is skipped, change return value. */
if (syscall == -1)
+#ifdef SYSCALL_NUM_RET_SHARE_REG
+ TH_LOG("Can't modify syscall return on this architecture");
+#else
regs.SYSCALL_RET = 1;
+#endif
#ifdef HAVE_GETREGS
ret = ptrace(PTRACE_SETREGS, tracee, 0, ®s);
@@ -1465,8 +1486,13 @@
ret = prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER, &self->prog, 0, 0);
ASSERT_EQ(0, ret);
+#ifdef SYSCALL_NUM_RET_SHARE_REG
+ /* gettid has been skipped */
+ EXPECT_EQ(-1, syscall(__NR_gettid));
+#else
/* gettid has been skipped and an altered return value stored. */
EXPECT_EQ(1, syscall(__NR_gettid));
+#endif
EXPECT_NE(self->mytid, syscall(__NR_gettid));
}
diff --git a/tools/usb/usbip/libsrc/Makefile.am b/tools/usb/usbip/libsrc/Makefile.am
index 7c8f8a4..90daf95 100644
--- a/tools/usb/usbip/libsrc/Makefile.am
+++ b/tools/usb/usbip/libsrc/Makefile.am
@@ -4,5 +4,7 @@
lib_LTLIBRARIES := libusbip.la
libusbip_la_SOURCES := names.c names.h usbip_host_driver.c usbip_host_driver.h \
- usbip_common.c usbip_common.h vhci_driver.c vhci_driver.h \
+ usbip_device_driver.c usbip_device_driver.h \
+ usbip_common.c usbip_common.h usbip_host_common.h \
+ usbip_host_common.c vhci_driver.c vhci_driver.h \
sysfs_utils.c sysfs_utils.h
diff --git a/tools/usb/usbip/libsrc/usbip_common.h b/tools/usb/usbip/libsrc/usbip_common.h
index 15fe792..51ef5fe 100644
--- a/tools/usb/usbip/libsrc/usbip_common.h
+++ b/tools/usb/usbip/libsrc/usbip_common.h
@@ -25,9 +25,12 @@
#define VHCI_STATE_PATH "/var/run/vhci_hcd"
#endif
+#define VUDC_DEVICE_DESCR_FILE "dev_desc"
+
/* kernel module names */
#define USBIP_CORE_MOD_NAME "usbip-core"
#define USBIP_HOST_DRV_NAME "usbip-host"
+#define USBIP_DEVICE_DRV_NAME "usbip-vudc"
#define USBIP_VHCI_DRV_NAME "vhci_hcd"
/* sysfs constants */
diff --git a/tools/usb/usbip/libsrc/usbip_device_driver.c b/tools/usb/usbip/libsrc/usbip_device_driver.c
new file mode 100644
index 0000000..e059b7d
--- /dev/null
+++ b/tools/usb/usbip/libsrc/usbip_device_driver.c
@@ -0,0 +1,163 @@
+/*
+ * Copyright (C) 2015 Karol Kosik <karo9@interia.eu>
+ * 2015 Samsung Electronics
+ * Author: Igor Kotrasinski <i.kotrasinsk@samsung.com>
+ *
+ * Based on tools/usb/usbip/libsrc/usbip_host_driver.c, which is:
+ * Copyright (C) 2011 matt mooney <mfm@muteddisk.com>
+ * 2005-2007 Takahiro Hirofuchi
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <fcntl.h>
+#include <string.h>
+#include <linux/usb/ch9.h>
+
+#include <unistd.h>
+
+#include "usbip_host_common.h"
+#include "usbip_device_driver.h"
+
+#undef PROGNAME
+#define PROGNAME "libusbip"
+
+#define copy_descr_attr16(dev, descr, attr) \
+ ((dev)->attr = le16toh((descr)->attr)) \
+
+#define copy_descr_attr(dev, descr, attr) \
+ ((dev)->attr = (descr)->attr) \
+
+#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
+
+static struct {
+ enum usb_device_speed speed;
+ const char *name;
+} speed_names[] = {
+ {
+ .speed = USB_SPEED_UNKNOWN,
+ .name = "UNKNOWN",
+ },
+ {
+ .speed = USB_SPEED_LOW,
+ .name = "low-speed",
+ },
+ {
+ .speed = USB_SPEED_FULL,
+ .name = "full-speed",
+ },
+ {
+ .speed = USB_SPEED_HIGH,
+ .name = "high-speed",
+ },
+ {
+ .speed = USB_SPEED_WIRELESS,
+ .name = "wireless",
+ },
+ {
+ .speed = USB_SPEED_SUPER,
+ .name = "super-speed",
+ },
+};
+
+static
+int read_usb_vudc_device(struct udev_device *sdev, struct usbip_usb_device *dev)
+{
+ const char *path, *name;
+ char filepath[SYSFS_PATH_MAX];
+ struct usb_device_descriptor descr;
+ unsigned i;
+ FILE *fd = NULL;
+ struct udev_device *plat;
+ const char *speed;
+ int ret = 0;
+
+ plat = udev_device_get_parent(sdev);
+ path = udev_device_get_syspath(plat);
+ snprintf(filepath, SYSFS_PATH_MAX, "%s/%s",
+ path, VUDC_DEVICE_DESCR_FILE);
+ fd = fopen(filepath, "r");
+ if (!fd)
+ return -1;
+ ret = fread((char *) &descr, sizeof(descr), 1, fd);
+ if (ret < 0)
+ return -1;
+ fclose(fd);
+
+ copy_descr_attr(dev, &descr, bDeviceClass);
+ copy_descr_attr(dev, &descr, bDeviceSubClass);
+ copy_descr_attr(dev, &descr, bDeviceProtocol);
+ copy_descr_attr(dev, &descr, bNumConfigurations);
+ copy_descr_attr16(dev, &descr, idVendor);
+ copy_descr_attr16(dev, &descr, idProduct);
+ copy_descr_attr16(dev, &descr, bcdDevice);
+
+ strncpy(dev->path, path, SYSFS_PATH_MAX);
+
+ dev->speed = USB_SPEED_UNKNOWN;
+ speed = udev_device_get_sysattr_value(sdev, "current_speed");
+ if (speed) {
+ for (i = 0; i < ARRAY_SIZE(speed_names); i++) {
+ if (!strcmp(speed_names[i].name, speed)) {
+ dev->speed = speed_names[i].speed;
+ break;
+ }
+ }
+ }
+
+ /* Only used for user output, little sense to output them in general */
+ dev->bNumInterfaces = 0;
+ dev->bConfigurationValue = 0;
+ dev->busnum = 0;
+
+ name = udev_device_get_sysname(plat);
+ strncpy(dev->busid, name, SYSFS_BUS_ID_SIZE);
+ return 0;
+}
+
+static int is_my_device(struct udev_device *dev)
+{
+ const char *driver;
+
+ driver = udev_device_get_property_value(dev, "USB_UDC_NAME");
+ return driver != NULL && !strcmp(driver, USBIP_DEVICE_DRV_NAME);
+}
+
+static int usbip_device_driver_open(struct usbip_host_driver *hdriver)
+{
+ int ret;
+
+ hdriver->ndevs = 0;
+ INIT_LIST_HEAD(&hdriver->edev_list);
+
+ ret = usbip_generic_driver_open(hdriver);
+ if (ret)
+ err("please load " USBIP_CORE_MOD_NAME ".ko and "
+ USBIP_DEVICE_DRV_NAME ".ko!");
+
+ return ret;
+}
+
+struct usbip_host_driver device_driver = {
+ .edev_list = LIST_HEAD_INIT(device_driver.edev_list),
+ .udev_subsystem = "udc",
+ .ops = {
+ .open = usbip_device_driver_open,
+ .close = usbip_generic_driver_close,
+ .refresh_device_list = usbip_generic_refresh_device_list,
+ .get_device = usbip_generic_get_device,
+ .read_device = read_usb_vudc_device,
+ .is_my_device = is_my_device,
+ },
+};
diff --git a/tools/usb/usbip/libsrc/usbip_device_driver.h b/tools/usb/usbip/libsrc/usbip_device_driver.h
new file mode 100644
index 0000000..54cb658
--- /dev/null
+++ b/tools/usb/usbip/libsrc/usbip_device_driver.h
@@ -0,0 +1,34 @@
+/*
+ * Copyright (C) 2015 Karol Kosik <karo9@interia.eu>
+ * 2015 Samsung Electronics
+ * Author: Igor Kotrasinski <i.kotrasinsk@samsung.com>
+ *
+ * Based on tools/usb/usbip/libsrc/usbip_host_driver.c, which is:
+ * Copyright (C) 2011 matt mooney <mfm@muteddisk.com>
+ * 2005-2007 Takahiro Hirofuchi
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __USBIP_DEVICE_DRIVER_H
+#define __USBIP_DEVICE_DRIVER_H
+
+#include <stdint.h>
+#include "usbip_common.h"
+#include "usbip_host_common.h"
+#include "list.h"
+
+extern struct usbip_host_driver device_driver;
+
+#endif /* __USBIP_DEVICE_DRIVER_H */
diff --git a/tools/usb/usbip/libsrc/usbip_host_common.c b/tools/usb/usbip/libsrc/usbip_host_common.c
new file mode 100644
index 0000000..9d41522
--- /dev/null
+++ b/tools/usb/usbip/libsrc/usbip_host_common.c
@@ -0,0 +1,273 @@
+/*
+ * Copyright (C) 2015-2016 Samsung Electronics
+ * Igor Kotrasinski <i.kotrasinsk@samsung.com>
+ * Krzysztof Opasiak <k.opasiak@samsung.com>
+ *
+ * Refactored from usbip_host_driver.c, which is:
+ * Copyright (C) 2011 matt mooney <mfm@muteddisk.com>
+ * 2005-2007 Takahiro Hirofuchi
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+
+#include <errno.h>
+#include <unistd.h>
+
+#include <libudev.h>
+
+#include "usbip_common.h"
+#include "usbip_host_common.h"
+#include "list.h"
+#include "sysfs_utils.h"
+
+struct udev *udev_context;
+
+static int32_t read_attr_usbip_status(struct usbip_usb_device *udev)
+{
+ char status_attr_path[SYSFS_PATH_MAX];
+ int fd;
+ int length;
+ char status;
+ int value = 0;
+
+ snprintf(status_attr_path, SYSFS_PATH_MAX, "%s/usbip_status",
+ udev->path);
+
+ fd = open(status_attr_path, O_RDONLY);
+ if (fd < 0) {
+ err("error opening attribute %s", status_attr_path);
+ return -1;
+ }
+
+ length = read(fd, &status, 1);
+ if (length < 0) {
+ err("error reading attribute %s", status_attr_path);
+ close(fd);
+ return -1;
+ }
+
+ value = atoi(&status);
+
+ return value;
+}
+
+static
+struct usbip_exported_device *usbip_exported_device_new(
+ struct usbip_host_driver *hdriver, const char *sdevpath)
+{
+ struct usbip_exported_device *edev = NULL;
+ struct usbip_exported_device *edev_old;
+ size_t size;
+ int i;
+
+ edev = calloc(1, sizeof(struct usbip_exported_device));
+
+ edev->sudev =
+ udev_device_new_from_syspath(udev_context, sdevpath);
+ if (!edev->sudev) {
+ err("udev_device_new_from_syspath: %s", sdevpath);
+ goto err;
+ }
+
+ if (hdriver->ops.read_device(edev->sudev, &edev->udev) < 0)
+ goto err;
+
+ edev->status = read_attr_usbip_status(&edev->udev);
+ if (edev->status < 0)
+ goto err;
+
+ /* reallocate buffer to include usb interface data */
+ size = sizeof(struct usbip_exported_device) +
+ edev->udev.bNumInterfaces * sizeof(struct usbip_usb_interface);
+
+ edev_old = edev;
+ edev = realloc(edev, size);
+ if (!edev) {
+ edev = edev_old;
+ dbg("realloc failed");
+ goto err;
+ }
+
+ for (i = 0; i < edev->udev.bNumInterfaces; i++) {
+ /* vudc does not support reading interfaces */
+ if (!hdriver->ops.read_interface)
+ break;
+ hdriver->ops.read_interface(&edev->udev, i, &edev->uinf[i]);
+ }
+
+ return edev;
+err:
+ if (edev->sudev)
+ udev_device_unref(edev->sudev);
+ if (edev)
+ free(edev);
+
+ return NULL;
+}
+
+static int refresh_exported_devices(struct usbip_host_driver *hdriver)
+{
+ struct usbip_exported_device *edev;
+ struct udev_enumerate *enumerate;
+ struct udev_list_entry *devices, *dev_list_entry;
+ struct udev_device *dev;
+ const char *path;
+
+ enumerate = udev_enumerate_new(udev_context);
+ udev_enumerate_add_match_subsystem(enumerate, hdriver->udev_subsystem);
+ udev_enumerate_scan_devices(enumerate);
+
+ devices = udev_enumerate_get_list_entry(enumerate);
+
+ udev_list_entry_foreach(dev_list_entry, devices) {
+ path = udev_list_entry_get_name(dev_list_entry);
+ dev = udev_device_new_from_syspath(udev_context,
+ path);
+ if (dev == NULL)
+ continue;
+
+ /* Check whether device uses usbip driver. */
+ if (hdriver->ops.is_my_device(dev)) {
+ edev = usbip_exported_device_new(hdriver, path);
+ if (!edev) {
+ dbg("usbip_exported_device_new failed");
+ continue;
+ }
+
+ list_add(&edev->node, &hdriver->edev_list);
+ hdriver->ndevs++;
+ }
+ }
+
+ return 0;
+}
+
+static void usbip_exported_device_destroy(struct list_head *devs)
+{
+ struct list_head *i, *tmp;
+ struct usbip_exported_device *edev;
+
+ list_for_each_safe(i, tmp, devs) {
+ edev = list_entry(i, struct usbip_exported_device, node);
+ list_del(i);
+ free(edev);
+ }
+}
+
+int usbip_generic_driver_open(struct usbip_host_driver *hdriver)
+{
+ int rc;
+
+ udev_context = udev_new();
+ if (!udev_context) {
+ err("udev_new failed");
+ return -1;
+ }
+
+ rc = refresh_exported_devices(hdriver);
+ if (rc < 0)
+ goto err;
+ return 0;
+err:
+ udev_unref(udev_context);
+ return -1;
+}
+
+void usbip_generic_driver_close(struct usbip_host_driver *hdriver)
+{
+ if (!hdriver)
+ return;
+
+ usbip_exported_device_destroy(&hdriver->edev_list);
+
+ udev_unref(udev_context);
+}
+
+int usbip_generic_refresh_device_list(struct usbip_host_driver *hdriver)
+{
+ int rc;
+
+ usbip_exported_device_destroy(&hdriver->edev_list);
+
+ hdriver->ndevs = 0;
+ INIT_LIST_HEAD(&hdriver->edev_list);
+
+ rc = refresh_exported_devices(hdriver);
+ if (rc < 0)
+ return -1;
+
+ return 0;
+}
+
+int usbip_export_device(struct usbip_exported_device *edev, int sockfd)
+{
+ char attr_name[] = "usbip_sockfd";
+ char sockfd_attr_path[SYSFS_PATH_MAX];
+ char sockfd_buff[30];
+ int ret;
+
+ if (edev->status != SDEV_ST_AVAILABLE) {
+ dbg("device not available: %s", edev->udev.busid);
+ switch (edev->status) {
+ case SDEV_ST_ERROR:
+ dbg("status SDEV_ST_ERROR");
+ break;
+ case SDEV_ST_USED:
+ dbg("status SDEV_ST_USED");
+ break;
+ default:
+ dbg("status unknown: 0x%x", edev->status);
+ }
+ return -1;
+ }
+
+ /* only the first interface is true */
+ snprintf(sockfd_attr_path, sizeof(sockfd_attr_path), "%s/%s",
+ edev->udev.path, attr_name);
+
+ snprintf(sockfd_buff, sizeof(sockfd_buff), "%d\n", sockfd);
+
+ ret = write_sysfs_attribute(sockfd_attr_path, sockfd_buff,
+ strlen(sockfd_buff));
+ if (ret < 0) {
+ err("write_sysfs_attribute failed: sockfd %s to %s",
+ sockfd_buff, sockfd_attr_path);
+ return ret;
+ }
+
+ info("connect: %s", edev->udev.busid);
+
+ return ret;
+}
+
+struct usbip_exported_device *usbip_generic_get_device(
+ struct usbip_host_driver *hdriver, int num)
+{
+ struct list_head *i;
+ struct usbip_exported_device *edev;
+ int cnt = 0;
+
+ list_for_each(i, &hdriver->edev_list) {
+ edev = list_entry(i, struct usbip_exported_device, node);
+ if (num == cnt)
+ return edev;
+ cnt++;
+ }
+
+ return NULL;
+}
diff --git a/tools/usb/usbip/libsrc/usbip_host_common.h b/tools/usb/usbip/libsrc/usbip_host_common.h
new file mode 100644
index 0000000..a64b803
--- /dev/null
+++ b/tools/usb/usbip/libsrc/usbip_host_common.h
@@ -0,0 +1,104 @@
+/*
+ * Copyright (C) 2015-2016 Samsung Electronics
+ * Igor Kotrasinski <i.kotrasinsk@samsung.com>
+ * Krzysztof Opasiak <k.opasiak@samsung.com>
+ *
+ * Refactored from usbip_host_driver.c, which is:
+ * Copyright (C) 2011 matt mooney <mfm@muteddisk.com>
+ * 2005-2007 Takahiro Hirofuchi
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __USBIP_HOST_COMMON_H
+#define __USBIP_HOST_COMMON_H
+
+#include <stdint.h>
+#include <libudev.h>
+#include <errno.h>
+#include "list.h"
+#include "usbip_common.h"
+#include "sysfs_utils.h"
+
+struct usbip_host_driver;
+
+struct usbip_host_driver_ops {
+ int (*open)(struct usbip_host_driver *hdriver);
+ void (*close)(struct usbip_host_driver *hdriver);
+ int (*refresh_device_list)(struct usbip_host_driver *hdriver);
+ struct usbip_exported_device * (*get_device)(
+ struct usbip_host_driver *hdriver, int num);
+
+ int (*read_device)(struct udev_device *sdev,
+ struct usbip_usb_device *dev);
+ int (*read_interface)(struct usbip_usb_device *udev, int i,
+ struct usbip_usb_interface *uinf);
+ int (*is_my_device)(struct udev_device *udev);
+};
+
+struct usbip_host_driver {
+ int ndevs;
+ /* list of exported device */
+ struct list_head edev_list;
+ const char *udev_subsystem;
+ struct usbip_host_driver_ops ops;
+};
+
+struct usbip_exported_device {
+ struct udev_device *sudev;
+ int32_t status;
+ struct usbip_usb_device udev;
+ struct list_head node;
+ struct usbip_usb_interface uinf[];
+};
+
+/* External API to access the driver */
+static inline int usbip_driver_open(struct usbip_host_driver *hdriver)
+{
+ if (!hdriver->ops.open)
+ return -EOPNOTSUPP;
+ return hdriver->ops.open(hdriver);
+}
+
+static inline void usbip_driver_close(struct usbip_host_driver *hdriver)
+{
+ if (!hdriver->ops.close)
+ return;
+ hdriver->ops.close(hdriver);
+}
+
+static inline int usbip_refresh_device_list(struct usbip_host_driver *hdriver)
+{
+ if (!hdriver->ops.refresh_device_list)
+ return -EOPNOTSUPP;
+ return hdriver->ops.refresh_device_list(hdriver);
+}
+
+static inline struct usbip_exported_device *
+usbip_get_device(struct usbip_host_driver *hdriver, int num)
+{
+ if (!hdriver->ops.get_device)
+ return NULL;
+ return hdriver->ops.get_device(hdriver, num);
+}
+
+/* Helper functions for implementing driver backend */
+int usbip_generic_driver_open(struct usbip_host_driver *hdriver);
+void usbip_generic_driver_close(struct usbip_host_driver *hdriver);
+int usbip_generic_refresh_device_list(struct usbip_host_driver *hdriver);
+int usbip_export_device(struct usbip_exported_device *edev, int sockfd);
+struct usbip_exported_device *usbip_generic_get_device(
+ struct usbip_host_driver *hdriver, int num);
+
+#endif /* __USBIP_HOST_COMMON_H */
diff --git a/tools/usb/usbip/libsrc/usbip_host_driver.c b/tools/usb/usbip/libsrc/usbip_host_driver.c
index bef08d5..4de6edc 100644
--- a/tools/usb/usbip/libsrc/usbip_host_driver.c
+++ b/tools/usb/usbip/libsrc/usbip_host_driver.c
@@ -1,6 +1,9 @@
/*
* Copyright (C) 2011 matt mooney <mfm@muteddisk.com>
* 2005-2007 Takahiro Hirofuchi
+ * Copyright (C) 2015-2016 Samsung Electronics
+ * Igor Kotrasinski <i.kotrasinsk@samsung.com>
+ * Krzysztof Opasiak <k.opasiak@samsung.com>
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -16,265 +19,47 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
-#include <sys/types.h>
-#include <sys/stat.h>
-#include <fcntl.h>
-
-#include <errno.h>
#include <unistd.h>
-
#include <libudev.h>
-#include "usbip_common.h"
+#include "usbip_host_common.h"
#include "usbip_host_driver.h"
-#include "list.h"
-#include "sysfs_utils.h"
#undef PROGNAME
#define PROGNAME "libusbip"
-struct usbip_host_driver *host_driver;
-struct udev *udev_context;
-
-static int32_t read_attr_usbip_status(struct usbip_usb_device *udev)
+static int is_my_device(struct udev_device *dev)
{
- char status_attr_path[SYSFS_PATH_MAX];
- int fd;
- int length;
- char status;
- int value = 0;
-
- snprintf(status_attr_path, SYSFS_PATH_MAX, "%s/usbip_status",
- udev->path);
-
- fd = open(status_attr_path, O_RDONLY);
- if (fd < 0) {
- err("error opening attribute %s", status_attr_path);
- return -1;
- }
-
- length = read(fd, &status, 1);
- if (length < 0) {
- err("error reading attribute %s", status_attr_path);
- close(fd);
- return -1;
- }
-
- value = atoi(&status);
-
- return value;
-}
-
-static
-struct usbip_exported_device *usbip_exported_device_new(const char *sdevpath)
-{
- struct usbip_exported_device *edev = NULL;
- struct usbip_exported_device *edev_old;
- size_t size;
- int i;
-
- edev = calloc(1, sizeof(struct usbip_exported_device));
-
- edev->sudev = udev_device_new_from_syspath(udev_context, sdevpath);
- if (!edev->sudev) {
- err("udev_device_new_from_syspath: %s", sdevpath);
- goto err;
- }
-
- read_usb_device(edev->sudev, &edev->udev);
-
- edev->status = read_attr_usbip_status(&edev->udev);
- if (edev->status < 0)
- goto err;
-
- /* reallocate buffer to include usb interface data */
- size = sizeof(struct usbip_exported_device) +
- edev->udev.bNumInterfaces * sizeof(struct usbip_usb_interface);
-
- edev_old = edev;
- edev = realloc(edev, size);
- if (!edev) {
- edev = edev_old;
- dbg("realloc failed");
- goto err;
- }
-
- for (i = 0; i < edev->udev.bNumInterfaces; i++)
- read_usb_interface(&edev->udev, i, &edev->uinf[i]);
-
- return edev;
-err:
- if (edev->sudev)
- udev_device_unref(edev->sudev);
- if (edev)
- free(edev);
-
- return NULL;
-}
-
-static int refresh_exported_devices(void)
-{
- struct usbip_exported_device *edev;
- struct udev_enumerate *enumerate;
- struct udev_list_entry *devices, *dev_list_entry;
- struct udev_device *dev;
- const char *path;
const char *driver;
- enumerate = udev_enumerate_new(udev_context);
- udev_enumerate_add_match_subsystem(enumerate, "usb");
- udev_enumerate_scan_devices(enumerate);
-
- devices = udev_enumerate_get_list_entry(enumerate);
-
- udev_list_entry_foreach(dev_list_entry, devices) {
- path = udev_list_entry_get_name(dev_list_entry);
- dev = udev_device_new_from_syspath(udev_context, path);
- if (dev == NULL)
- continue;
-
- /* Check whether device uses usbip-host driver. */
- driver = udev_device_get_driver(dev);
- if (driver != NULL && !strcmp(driver, USBIP_HOST_DRV_NAME)) {
- edev = usbip_exported_device_new(path);
- if (!edev) {
- dbg("usbip_exported_device_new failed");
- continue;
- }
-
- list_add(&edev->node, &host_driver->edev_list);
- host_driver->ndevs++;
- }
- }
-
- return 0;
+ driver = udev_device_get_driver(dev);
+ return driver != NULL && !strcmp(driver, USBIP_HOST_DRV_NAME);
}
-static void usbip_exported_device_destroy(void)
+static int usbip_host_driver_open(struct usbip_host_driver *hdriver)
{
- struct list_head *i, *tmp;
- struct usbip_exported_device *edev;
-
- list_for_each_safe(i, tmp, &host_driver->edev_list) {
- edev = list_entry(i, struct usbip_exported_device, node);
- list_del(i);
- free(edev);
- }
-}
-
-int usbip_host_driver_open(void)
-{
- int rc;
-
- udev_context = udev_new();
- if (!udev_context) {
- err("udev_new failed");
- return -1;
- }
-
- host_driver = calloc(1, sizeof(*host_driver));
-
- host_driver->ndevs = 0;
- INIT_LIST_HEAD(&host_driver->edev_list);
-
- rc = refresh_exported_devices();
- if (rc < 0)
- goto err_free_host_driver;
-
- return 0;
-
-err_free_host_driver:
- free(host_driver);
- host_driver = NULL;
-
- udev_unref(udev_context);
-
- return -1;
-}
-
-void usbip_host_driver_close(void)
-{
- if (!host_driver)
- return;
-
- usbip_exported_device_destroy();
-
- free(host_driver);
- host_driver = NULL;
-
- udev_unref(udev_context);
-}
-
-int usbip_host_refresh_device_list(void)
-{
- int rc;
-
- usbip_exported_device_destroy();
-
- host_driver->ndevs = 0;
- INIT_LIST_HEAD(&host_driver->edev_list);
-
- rc = refresh_exported_devices();
- if (rc < 0)
- return -1;
-
- return 0;
-}
-
-int usbip_host_export_device(struct usbip_exported_device *edev, int sockfd)
-{
- char attr_name[] = "usbip_sockfd";
- char sockfd_attr_path[SYSFS_PATH_MAX];
- char sockfd_buff[30];
int ret;
- if (edev->status != SDEV_ST_AVAILABLE) {
- dbg("device not available: %s", edev->udev.busid);
- switch (edev->status) {
- case SDEV_ST_ERROR:
- dbg("status SDEV_ST_ERROR");
- break;
- case SDEV_ST_USED:
- dbg("status SDEV_ST_USED");
- break;
- default:
- dbg("status unknown: 0x%x", edev->status);
- }
- return -1;
- }
+ hdriver->ndevs = 0;
+ INIT_LIST_HEAD(&hdriver->edev_list);
- /* only the first interface is true */
- snprintf(sockfd_attr_path, sizeof(sockfd_attr_path), "%s/%s",
- edev->udev.path, attr_name);
-
- snprintf(sockfd_buff, sizeof(sockfd_buff), "%d\n", sockfd);
-
- ret = write_sysfs_attribute(sockfd_attr_path, sockfd_buff,
- strlen(sockfd_buff));
- if (ret < 0) {
- err("write_sysfs_attribute failed: sockfd %s to %s",
- sockfd_buff, sockfd_attr_path);
- return ret;
- }
-
- info("connect: %s", edev->udev.busid);
-
+ ret = usbip_generic_driver_open(hdriver);
+ if (ret)
+ err("please load " USBIP_CORE_MOD_NAME ".ko and "
+ USBIP_HOST_DRV_NAME ".ko!");
return ret;
}
-struct usbip_exported_device *usbip_host_get_device(int num)
-{
- struct list_head *i;
- struct usbip_exported_device *edev;
- int cnt = 0;
-
- list_for_each(i, &host_driver->edev_list) {
- edev = list_entry(i, struct usbip_exported_device, node);
- if (num == cnt)
- return edev;
- else
- cnt++;
- }
-
- return NULL;
-}
+struct usbip_host_driver host_driver = {
+ .edev_list = LIST_HEAD_INIT(host_driver.edev_list),
+ .udev_subsystem = "usb",
+ .ops = {
+ .open = usbip_host_driver_open,
+ .close = usbip_generic_driver_close,
+ .refresh_device_list = usbip_generic_refresh_device_list,
+ .get_device = usbip_generic_get_device,
+ .read_device = read_usb_device,
+ .read_interface = read_usb_interface,
+ .is_my_device = is_my_device,
+ },
+};
diff --git a/tools/usb/usbip/libsrc/usbip_host_driver.h b/tools/usb/usbip/libsrc/usbip_host_driver.h
index 2a31f85..77f07e7 100644
--- a/tools/usb/usbip/libsrc/usbip_host_driver.h
+++ b/tools/usb/usbip/libsrc/usbip_host_driver.h
@@ -1,6 +1,9 @@
/*
* Copyright (C) 2011 matt mooney <mfm@muteddisk.com>
* 2005-2007 Takahiro Hirofuchi
+ * Copyright (C) 2015-2016 Samsung Electronics
+ * Igor Kotrasinski <i.kotrasinsk@samsung.com>
+ * Krzysztof Opasiak <k.opasiak@samsung.com>
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -22,28 +25,8 @@
#include <stdint.h>
#include "usbip_common.h"
#include "list.h"
+#include "usbip_host_common.h"
-struct usbip_host_driver {
- int ndevs;
- /* list of exported device */
- struct list_head edev_list;
-};
-
-struct usbip_exported_device {
- struct udev_device *sudev;
- int32_t status;
- struct usbip_usb_device udev;
- struct list_head node;
- struct usbip_usb_interface uinf[];
-};
-
-extern struct usbip_host_driver *host_driver;
-
-int usbip_host_driver_open(void);
-void usbip_host_driver_close(void);
-
-int usbip_host_refresh_device_list(void);
-int usbip_host_export_device(struct usbip_exported_device *edev, int sockfd);
-struct usbip_exported_device *usbip_host_get_device(int num);
+extern struct usbip_host_driver host_driver;
#endif /* __USBIP_HOST_DRIVER_H */
diff --git a/tools/usb/usbip/src/usbip_attach.c b/tools/usb/usbip/src/usbip_attach.c
index d58a14d..70a6b50 100644
--- a/tools/usb/usbip/src/usbip_attach.c
+++ b/tools/usb/usbip/src/usbip_attach.c
@@ -1,6 +1,9 @@
/*
* Copyright (C) 2011 matt mooney <mfm@muteddisk.com>
* 2005-2007 Takahiro Hirofuchi
+ * Copyright (C) 2015-2016 Samsung Electronics
+ * Igor Kotrasinski <i.kotrasinsk@samsung.com>
+ * Krzysztof Opasiak <k.opasiak@samsung.com>
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -36,7 +39,8 @@
static const char usbip_attach_usage_string[] =
"usbip attach <args>\n"
" -r, --remote=<host> The machine with exported USB devices\n"
- " -b, --busid=<busid> Busid of the device on <host>\n";
+ " -b, --busid=<busid> Busid of the device on <host>\n"
+ " -d, --device=<devid> Id of the virtual UDC on <host>\n";
void usbip_attach_usage(void)
{
@@ -203,6 +207,7 @@
static const struct option opts[] = {
{ "remote", required_argument, NULL, 'r' },
{ "busid", required_argument, NULL, 'b' },
+ { "device", required_argument, NULL, 'd' },
{ NULL, 0, NULL, 0 }
};
char *host = NULL;
@@ -211,7 +216,7 @@
int ret = -1;
for (;;) {
- opt = getopt_long(argc, argv, "r:b:", opts, NULL);
+ opt = getopt_long(argc, argv, "d:r:b:", opts, NULL);
if (opt == -1)
break;
@@ -220,6 +225,7 @@
case 'r':
host = optarg;
break;
+ case 'd':
case 'b':
busid = optarg;
break;
diff --git a/tools/usb/usbip/src/usbip_list.c b/tools/usb/usbip/src/usbip_list.c
index d5ce34a..f1b38e8 100644
--- a/tools/usb/usbip/src/usbip_list.c
+++ b/tools/usb/usbip/src/usbip_list.c
@@ -1,6 +1,9 @@
/*
* Copyright (C) 2011 matt mooney <mfm@muteddisk.com>
* 2005-2007 Takahiro Hirofuchi
+ * Copyright (C) 2015-2016 Samsung Electronics
+ * Igor Kotrasinski <i.kotrasinsk@samsung.com>
+ * Krzysztof Opasiak <k.opasiak@samsung.com>
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -30,6 +33,10 @@
#include <netdb.h>
#include <unistd.h>
+#include <dirent.h>
+
+#include <linux/usb/ch9.h>
+
#include "usbip_common.h"
#include "usbip_network.h"
#include "usbip.h"
@@ -205,8 +212,10 @@
/* Get device information. */
idVendor = udev_device_get_sysattr_value(dev, "idVendor");
idProduct = udev_device_get_sysattr_value(dev, "idProduct");
- bConfValue = udev_device_get_sysattr_value(dev, "bConfigurationValue");
- bNumIntfs = udev_device_get_sysattr_value(dev, "bNumInterfaces");
+ bConfValue = udev_device_get_sysattr_value(dev,
+ "bConfigurationValue");
+ bNumIntfs = udev_device_get_sysattr_value(dev,
+ "bNumInterfaces");
busid = udev_device_get_sysname(dev);
if (!idVendor || !idProduct || !bConfValue || !bNumIntfs) {
err("problem getting device attributes: %s",
@@ -237,12 +246,90 @@
return ret;
}
+static int list_gadget_devices(bool parsable)
+{
+ int ret = -1;
+ struct udev *udev;
+ struct udev_enumerate *enumerate;
+ struct udev_list_entry *devices, *dev_list_entry;
+ struct udev_device *dev;
+ const char *path;
+ const char *driver;
+
+ const struct usb_device_descriptor *d_desc;
+ const char *descriptors;
+ char product_name[128];
+
+ uint16_t idVendor;
+ char idVendor_buf[8];
+ uint16_t idProduct;
+ char idProduct_buf[8];
+ const char *busid;
+
+ udev = udev_new();
+ enumerate = udev_enumerate_new(udev);
+
+ udev_enumerate_add_match_subsystem(enumerate, "platform");
+
+ udev_enumerate_scan_devices(enumerate);
+ devices = udev_enumerate_get_list_entry(enumerate);
+
+ udev_list_entry_foreach(dev_list_entry, devices) {
+ path = udev_list_entry_get_name(dev_list_entry);
+ dev = udev_device_new_from_syspath(udev, path);
+
+ driver = udev_device_get_driver(dev);
+ /* We only have mechanism to enumerate gadgets bound to vudc */
+ if (driver == NULL || strcmp(driver, USBIP_DEVICE_DRV_NAME))
+ continue;
+
+ /* Get device information. */
+ descriptors = udev_device_get_sysattr_value(dev,
+ VUDC_DEVICE_DESCR_FILE);
+
+ if (!descriptors) {
+ err("problem getting device attributes: %s",
+ strerror(errno));
+ goto err_out;
+ }
+
+ d_desc = (const struct usb_device_descriptor *) descriptors;
+
+ idVendor = le16toh(d_desc->idVendor);
+ sprintf(idVendor_buf, "0x%4x", idVendor);
+ idProduct = le16toh(d_desc->idProduct);
+ sprintf(idProduct_buf, "0x%4x", idVendor);
+ busid = udev_device_get_sysname(dev);
+
+ /* Get product name. */
+ usbip_names_get_product(product_name, sizeof(product_name),
+ le16toh(idVendor),
+ le16toh(idProduct));
+
+ /* Print information. */
+ print_device(busid, idVendor_buf, idProduct_buf, parsable);
+ print_product_name(product_name, parsable);
+
+ printf("\n");
+
+ udev_device_unref(dev);
+ }
+ ret = 0;
+
+err_out:
+ udev_enumerate_unref(enumerate);
+ udev_unref(udev);
+
+ return ret;
+}
+
int usbip_list(int argc, char *argv[])
{
static const struct option opts[] = {
{ "parsable", no_argument, NULL, 'p' },
{ "remote", required_argument, NULL, 'r' },
{ "local", no_argument, NULL, 'l' },
+ { "device", no_argument, NULL, 'd' },
{ NULL, 0, NULL, 0 }
};
@@ -254,7 +341,7 @@
err("failed to open %s", USBIDS_FILE);
for (;;) {
- opt = getopt_long(argc, argv, "pr:l", opts, NULL);
+ opt = getopt_long(argc, argv, "pr:ld", opts, NULL);
if (opt == -1)
break;
@@ -269,6 +356,9 @@
case 'l':
ret = list_devices(parsable);
goto out;
+ case 'd':
+ ret = list_gadget_devices(parsable);
+ goto out;
default:
goto err_out;
}
diff --git a/tools/usb/usbip/src/usbip_port.c b/tools/usb/usbip/src/usbip_port.c
index a2e884fd..7bd74fb 100644
--- a/tools/usb/usbip/src/usbip_port.c
+++ b/tools/usb/usbip/src/usbip_port.c
@@ -22,10 +22,13 @@
struct usbip_imported_device *idev;
int ret;
+ if (usbip_names_init(USBIDS_FILE))
+ err("failed to open %s", USBIDS_FILE);
+
ret = usbip_vhci_driver_open();
if (ret < 0) {
err("open vhci_driver");
- return -1;
+ goto err_names_free;
}
printf("Imported USB devices\n");
@@ -35,13 +38,19 @@
idev = &vhci_driver->idev[i];
if (usbip_vhci_imported_device_dump(idev) < 0)
- ret = -1;
+ goto err_driver_close;
}
usbip_vhci_driver_close();
+ usbip_names_free();
return ret;
+err_driver_close:
+ usbip_vhci_driver_close();
+err_names_free:
+ usbip_names_free();
+ return -1;
}
int usbip_port_show(__attribute__((unused)) int argc,
diff --git a/tools/usb/usbip/src/usbipd.c b/tools/usb/usbip/src/usbipd.c
index 2a7cd2b..a0972de 100644
--- a/tools/usb/usbip/src/usbipd.c
+++ b/tools/usb/usbip/src/usbipd.c
@@ -1,6 +1,9 @@
/*
* Copyright (C) 2011 matt mooney <mfm@muteddisk.com>
* 2005-2007 Takahiro Hirofuchi
+ * Copyright (C) 2015-2016 Samsung Electronics
+ * Igor Kotrasinski <i.kotrasinsk@samsung.com>
+ * Krzysztof Opasiak <k.opasiak@samsung.com>
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -41,6 +44,8 @@
#include <poll.h>
#include "usbip_host_driver.h"
+#include "usbip_host_common.h"
+#include "usbip_device_driver.h"
#include "usbip_common.h"
#include "usbip_network.h"
#include "list.h"
@@ -64,6 +69,11 @@
" -6, --ipv6\n"
" Bind to IPv6. Default is both.\n"
"\n"
+ " -e, --device\n"
+ " Run in device mode.\n"
+ " Rather than drive an attached device, create\n"
+ " a virtual UDC to bind gadgets to.\n"
+ "\n"
" -D, --daemon\n"
" Run as a daemon process.\n"
"\n"
@@ -83,6 +93,8 @@
" -v, --version\n"
" Show version.\n";
+static struct usbip_host_driver *driver;
+
static void usbipd_help(void)
{
printf("%s\n", usbipd_help_string);
@@ -107,7 +119,7 @@
}
PACK_OP_IMPORT_REQUEST(0, &req);
- list_for_each(i, &host_driver->edev_list) {
+ list_for_each(i, &driver->edev_list) {
edev = list_entry(i, struct usbip_exported_device, node);
if (!strncmp(req.busid, edev->udev.busid, SYSFS_BUS_ID_SIZE)) {
info("found requested device: %s", req.busid);
@@ -121,7 +133,7 @@
usbip_net_set_nodelay(sockfd);
/* export device needs a TCP/IP socket descriptor */
- rc = usbip_host_export_device(edev, sockfd);
+ rc = usbip_export_device(edev, sockfd);
if (rc < 0)
error = 1;
} else {
@@ -166,7 +178,7 @@
reply.ndev = 0;
/* number of exported devices */
- list_for_each(j, &host_driver->edev_list) {
+ list_for_each(j, &driver->edev_list) {
reply.ndev += 1;
}
info("exportable devices: %d", reply.ndev);
@@ -184,7 +196,7 @@
return -1;
}
- list_for_each(j, &host_driver->edev_list) {
+ list_for_each(j, &driver->edev_list) {
edev = list_entry(j, struct usbip_exported_device, node);
dump_usb_device(&edev->udev);
memcpy(&pdu_udev, &edev->udev, sizeof(pdu_udev));
@@ -246,7 +258,7 @@
return -1;
}
- ret = usbip_host_refresh_device_list();
+ ret = usbip_refresh_device_list(driver);
if (ret < 0) {
dbg("could not refresh device list: %d", ret);
return -1;
@@ -491,16 +503,13 @@
struct timespec timeout;
sigset_t sigmask;
- if (usbip_host_driver_open()) {
- err("please load " USBIP_CORE_MOD_NAME ".ko and "
- USBIP_HOST_DRV_NAME ".ko!");
+ if (usbip_driver_open(driver))
return -1;
- }
if (daemonize) {
if (daemon(0, 0) < 0) {
err("daemonizing failed: %s", strerror(errno));
- usbip_host_driver_close();
+ usbip_driver_close(driver);
return -1;
}
umask(0);
@@ -525,7 +534,7 @@
ai_head = do_getaddrinfo(NULL, family);
if (!ai_head) {
- usbip_host_driver_close();
+ usbip_driver_close(driver);
return -1;
}
nsockfd = listen_all_addrinfo(ai_head, sockfdlist,
@@ -533,7 +542,7 @@
freeaddrinfo(ai_head);
if (nsockfd <= 0) {
err("failed to open a listening socket");
- usbip_host_driver_close();
+ usbip_driver_close(driver);
return -1;
}
@@ -574,7 +583,7 @@
info("shutting down " PROGNAME);
free(fds);
- usbip_host_driver_close();
+ usbip_driver_close(driver);
return 0;
}
@@ -587,6 +596,7 @@
{ "daemon", no_argument, NULL, 'D' },
{ "daemon", no_argument, NULL, 'D' },
{ "debug", no_argument, NULL, 'd' },
+ { "device", no_argument, NULL, 'e' },
{ "pid", optional_argument, NULL, 'P' },
{ "tcp-port", required_argument, NULL, 't' },
{ "help", no_argument, NULL, 'h' },
@@ -613,8 +623,9 @@
err("not running as root?");
cmd = cmd_standalone_mode;
+ driver = &host_driver;
for (;;) {
- opt = getopt_long(argc, argv, "46DdP::t:hv", longopts, NULL);
+ opt = getopt_long(argc, argv, "46DdeP::t:hv", longopts, NULL);
if (opt == -1)
break;
@@ -644,6 +655,9 @@
case 'v':
cmd = cmd_version;
break;
+ case 'e':
+ driver = &device_driver;
+ break;
case '?':
usbipd_help();
default:
diff --git a/tools/virtio/ringtest/Makefile b/tools/virtio/ringtest/Makefile
index feaa64a..6ba7455 100644
--- a/tools/virtio/ringtest/Makefile
+++ b/tools/virtio/ringtest/Makefile
@@ -1,6 +1,6 @@
all:
-all: ring virtio_ring_0_9 virtio_ring_poll
+all: ring virtio_ring_0_9 virtio_ring_poll virtio_ring_inorder
CFLAGS += -Wall
CFLAGS += -pthread -O2 -ggdb
@@ -10,13 +10,16 @@
ring.o: ring.c main.h
virtio_ring_0_9.o: virtio_ring_0_9.c main.h
virtio_ring_poll.o: virtio_ring_poll.c virtio_ring_0_9.c main.h
+virtio_ring_inorder.o: virtio_ring_inorder.c virtio_ring_0_9.c main.h
ring: ring.o main.o
virtio_ring_0_9: virtio_ring_0_9.o main.o
virtio_ring_poll: virtio_ring_poll.o main.o
+virtio_ring_inorder: virtio_ring_inorder.o main.o
clean:
-rm main.o
-rm ring.o ring
-rm virtio_ring_0_9.o virtio_ring_0_9
-rm virtio_ring_poll.o virtio_ring_poll
+ -rm virtio_ring_inorder.o virtio_ring_inorder
.PHONY: all clean
diff --git a/tools/virtio/ringtest/main.c b/tools/virtio/ringtest/main.c
index 3a5ff43..147abb4 100644
--- a/tools/virtio/ringtest/main.c
+++ b/tools/virtio/ringtest/main.c
@@ -115,7 +115,7 @@
do {
if (started < bufs &&
started - completed < max_outstanding) {
- r = add_inbuf(0, NULL, "Hello, world!");
+ r = add_inbuf(0, "Buffer\n", "Hello, world!");
if (__builtin_expect(r == 0, true)) {
++started;
if (!--tokick) {
diff --git a/tools/virtio/ringtest/virtio_ring_0_9.c b/tools/virtio/ringtest/virtio_ring_0_9.c
index 47c9a1a..7618662 100644
--- a/tools/virtio/ringtest/virtio_ring_0_9.c
+++ b/tools/virtio/ringtest/virtio_ring_0_9.c
@@ -26,6 +26,14 @@
* high bits of ring id ^ 0x8000).
*/
/* #ifdef RING_POLL */
+/* enabling the below activates experimental in-order code
+ * (which skips ring updates and reads and writes len in descriptor).
+ */
+/* #ifdef INORDER */
+
+#if defined(RING_POLL) && defined(INORDER)
+#error "RING_POLL and INORDER are mutually exclusive"
+#endif
/* how much padding is needed to avoid false cache sharing */
#define HOST_GUEST_PADDING 0x80
@@ -35,7 +43,11 @@
unsigned short last_used_idx;
unsigned short num_free;
unsigned short kicked_avail_idx;
+#ifndef INORDER
unsigned short free_head;
+#else
+ unsigned short reserved_free_head;
+#endif
unsigned char reserved[HOST_GUEST_PADDING - 10];
} guest;
@@ -66,8 +78,10 @@
guest.avail_idx = 0;
guest.kicked_avail_idx = -1;
guest.last_used_idx = 0;
+#ifndef INORDER
/* Put everything in free lists. */
guest.free_head = 0;
+#endif
for (i = 0; i < ring_size - 1; i++)
ring.desc[i].next = i + 1;
host.used_idx = 0;
@@ -84,13 +98,20 @@
/* guest side */
int add_inbuf(unsigned len, void *buf, void *datap)
{
- unsigned head, avail;
+ unsigned head;
+#ifndef INORDER
+ unsigned avail;
+#endif
struct vring_desc *desc;
if (!guest.num_free)
return -1;
+#ifdef INORDER
+ head = (ring_size - 1) & (guest.avail_idx++);
+#else
head = guest.free_head;
+#endif
guest.num_free--;
desc = ring.desc;
@@ -102,7 +123,9 @@
* descriptors.
*/
desc[head].flags &= ~VRING_DESC_F_NEXT;
+#ifndef INORDER
guest.free_head = desc[head].next;
+#endif
data[head].data = datap;
@@ -113,8 +136,12 @@
ring.avail->ring[avail & (ring_size - 1)] =
(head | (avail & ~(ring_size - 1))) ^ 0x8000;
#else
+#ifndef INORDER
+ /* Barrier A (for pairing) */
+ smp_release();
avail = (ring_size - 1) & (guest.avail_idx++);
ring.avail->ring[avail] = head;
+#endif
/* Barrier A (for pairing) */
smp_release();
#endif
@@ -141,15 +168,27 @@
return NULL;
/* Barrier B (for pairing) */
smp_acquire();
+#ifdef INORDER
+ head = (ring_size - 1) & guest.last_used_idx;
+ index = head;
+#else
head = (ring_size - 1) & guest.last_used_idx;
index = ring.used->ring[head].id;
#endif
+
+#endif
+#ifdef INORDER
+ *lenp = ring.desc[index].len;
+#else
*lenp = ring.used->ring[head].len;
+#endif
datap = data[index].data;
*bufp = (void*)(unsigned long)ring.desc[index].addr;
data[index].data = NULL;
+#ifndef INORDER
ring.desc[index].next = guest.free_head;
guest.free_head = index;
+#endif
guest.num_free++;
guest.last_used_idx++;
return datap;
@@ -283,16 +322,24 @@
smp_acquire();
used_idx &= ring_size - 1;
+#ifdef INORDER
+ head = used_idx;
+#else
head = ring.avail->ring[used_idx];
+#endif
desc = &ring.desc[head];
#endif
*lenp = desc->len;
*bufp = (void *)(unsigned long)desc->addr;
+#ifdef INORDER
+ desc->len = desc->len - 1;
+#else
/* now update used ring */
ring.used->ring[used_idx].id = head;
ring.used->ring[used_idx].len = desc->len - 1;
+#endif
/* Barrier B (for pairing) */
smp_release();
host.used_idx++;
diff --git a/tools/virtio/ringtest/virtio_ring_inorder.c b/tools/virtio/ringtest/virtio_ring_inorder.c
new file mode 100644
index 0000000..2438ca5
--- /dev/null
+++ b/tools/virtio/ringtest/virtio_ring_inorder.c
@@ -0,0 +1,2 @@
+#define INORDER 1
+#include "virtio_ring_0_9.c"
diff --git a/virt/kvm/Kconfig b/virt/kvm/Kconfig
index 7a79b68..e5d6108 100644
--- a/virt/kvm/Kconfig
+++ b/virt/kvm/Kconfig
@@ -41,6 +41,9 @@
config HAVE_KVM_ARCH_TLB_FLUSH_ALL
bool
+config HAVE_KVM_INVALID_WAKEUPS
+ bool
+
config KVM_GENERIC_DIRTYLOG_READ_PROTECT
bool
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 9aaa35d..409db33 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -17,7 +17,6 @@
*/
#include <linux/cpu.h>
-#include <linux/of_irq.h>
#include <linux/kvm.h>
#include <linux/kvm_host.h>
#include <linux/interrupt.h>
@@ -438,45 +437,29 @@
.notifier_call = kvm_timer_cpu_notify,
};
-static const struct of_device_id arch_timer_of_match[] = {
- { .compatible = "arm,armv7-timer", },
- { .compatible = "arm,armv8-timer", },
- {},
-};
-
int kvm_timer_hyp_init(void)
{
- struct device_node *np;
- unsigned int ppi;
+ struct arch_timer_kvm_info *info;
int err;
- timecounter = arch_timer_get_timecounter();
- if (!timecounter)
- return -ENODEV;
+ info = arch_timer_get_kvm_info();
+ timecounter = &info->timecounter;
- np = of_find_matching_node(NULL, arch_timer_of_match);
- if (!np) {
- kvm_err("kvm_arch_timer: can't find DT node\n");
+ if (info->virtual_irq <= 0) {
+ kvm_err("kvm_arch_timer: invalid virtual timer IRQ: %d\n",
+ info->virtual_irq);
return -ENODEV;
}
+ host_vtimer_irq = info->virtual_irq;
- ppi = irq_of_parse_and_map(np, 2);
- if (!ppi) {
- kvm_err("kvm_arch_timer: no virtual timer interrupt\n");
- err = -EINVAL;
- goto out;
- }
-
- err = request_percpu_irq(ppi, kvm_arch_timer_handler,
+ err = request_percpu_irq(host_vtimer_irq, kvm_arch_timer_handler,
"kvm guest timer", kvm_get_running_vcpus());
if (err) {
kvm_err("kvm_arch_timer: can't request interrupt %d (%d)\n",
- ppi, err);
+ host_vtimer_irq, err);
goto out;
}
- host_vtimer_irq = ppi;
-
err = __register_cpu_notifier(&kvm_timer_cpu_nb);
if (err) {
kvm_err("Cannot register timer CPU notifier\n");
@@ -489,14 +472,13 @@
goto out_free;
}
- kvm_info("%s IRQ%d\n", np->name, ppi);
+ kvm_info("virtual timer IRQ%d\n", host_vtimer_irq);
on_each_cpu(kvm_timer_init_interrupt, NULL, 1);
goto out;
out_free:
- free_percpu_irq(ppi, kvm_get_running_vcpus());
+ free_percpu_irq(host_vtimer_irq, kvm_get_running_vcpus());
out:
- of_node_put(np);
return err;
}
diff --git a/virt/kvm/arm/vgic-v2.c b/virt/kvm/arm/vgic-v2.c
index 67ec334..7e826c9 100644
--- a/virt/kvm/arm/vgic-v2.c
+++ b/virt/kvm/arm/vgic-v2.c
@@ -20,9 +20,6 @@
#include <linux/kvm_host.h>
#include <linux/interrupt.h>
#include <linux/io.h>
-#include <linux/of.h>
-#include <linux/of_address.h>
-#include <linux/of_irq.h>
#include <linux/irqchip/arm-gic.h>
@@ -186,38 +183,39 @@
}
/**
- * vgic_v2_probe - probe for a GICv2 compatible interrupt controller in DT
- * @node: pointer to the DT node
- * @ops: address of a pointer to the GICv2 operations
- * @params: address of a pointer to HW-specific parameters
+ * vgic_v2_probe - probe for a GICv2 compatible interrupt controller
+ * @gic_kvm_info: pointer to the GIC description
+ * @ops: address of a pointer to the GICv2 operations
+ * @params: address of a pointer to HW-specific parameters
*
* Returns 0 if a GICv2 has been found, with the low level operations
* in *ops and the HW parameters in *params. Returns an error code
* otherwise.
*/
-int vgic_v2_probe(struct device_node *vgic_node,
- const struct vgic_ops **ops,
- const struct vgic_params **params)
+int vgic_v2_probe(const struct gic_kvm_info *gic_kvm_info,
+ const struct vgic_ops **ops,
+ const struct vgic_params **params)
{
int ret;
- struct resource vctrl_res;
- struct resource vcpu_res;
struct vgic_params *vgic = &vgic_v2_params;
+ const struct resource *vctrl_res = &gic_kvm_info->vctrl;
+ const struct resource *vcpu_res = &gic_kvm_info->vcpu;
- vgic->maint_irq = irq_of_parse_and_map(vgic_node, 0);
- if (!vgic->maint_irq) {
- kvm_err("error getting vgic maintenance irq from DT\n");
+ if (!gic_kvm_info->maint_irq) {
+ kvm_err("error getting vgic maintenance irq\n");
+ ret = -ENXIO;
+ goto out;
+ }
+ vgic->maint_irq = gic_kvm_info->maint_irq;
+
+ if (!gic_kvm_info->vctrl.start) {
+ kvm_err("GICH not present in the firmware table\n");
ret = -ENXIO;
goto out;
}
- ret = of_address_to_resource(vgic_node, 2, &vctrl_res);
- if (ret) {
- kvm_err("Cannot obtain GICH resource\n");
- goto out;
- }
-
- vgic->vctrl_base = of_iomap(vgic_node, 2);
+ vgic->vctrl_base = ioremap(gic_kvm_info->vctrl.start,
+ resource_size(&gic_kvm_info->vctrl));
if (!vgic->vctrl_base) {
kvm_err("Cannot ioremap GICH\n");
ret = -ENOMEM;
@@ -228,29 +226,23 @@
vgic->nr_lr = (vgic->nr_lr & 0x3f) + 1;
ret = create_hyp_io_mappings(vgic->vctrl_base,
- vgic->vctrl_base + resource_size(&vctrl_res),
- vctrl_res.start);
+ vgic->vctrl_base + resource_size(vctrl_res),
+ vctrl_res->start);
if (ret) {
kvm_err("Cannot map VCTRL into hyp\n");
goto out_unmap;
}
- if (of_address_to_resource(vgic_node, 3, &vcpu_res)) {
- kvm_err("Cannot obtain GICV resource\n");
- ret = -ENXIO;
- goto out_unmap;
- }
-
- if (!PAGE_ALIGNED(vcpu_res.start)) {
+ if (!PAGE_ALIGNED(vcpu_res->start)) {
kvm_err("GICV physical address 0x%llx not page aligned\n",
- (unsigned long long)vcpu_res.start);
+ (unsigned long long)vcpu_res->start);
ret = -ENXIO;
goto out_unmap;
}
- if (!PAGE_ALIGNED(resource_size(&vcpu_res))) {
+ if (!PAGE_ALIGNED(resource_size(vcpu_res))) {
kvm_err("GICV size 0x%llx not a multiple of page size 0x%lx\n",
- (unsigned long long)resource_size(&vcpu_res),
+ (unsigned long long)resource_size(vcpu_res),
PAGE_SIZE);
ret = -ENXIO;
goto out_unmap;
@@ -259,10 +251,10 @@
vgic->can_emulate_gicv2 = true;
kvm_register_device_ops(&kvm_arm_vgic_v2_ops, KVM_DEV_TYPE_ARM_VGIC_V2);
- vgic->vcpu_base = vcpu_res.start;
+ vgic->vcpu_base = vcpu_res->start;
- kvm_info("%s@%llx IRQ%d\n", vgic_node->name,
- vctrl_res.start, vgic->maint_irq);
+ kvm_info("GICH base=0x%llx, GICV base=0x%llx, IRQ=%d\n",
+ gic_kvm_info->vctrl.start, vgic->vcpu_base, vgic->maint_irq);
vgic->type = VGIC_V2;
vgic->max_gic_vcpus = VGIC_V2_MAX_CPUS;
@@ -276,6 +268,5 @@
out_unmap:
iounmap(vgic->vctrl_base);
out:
- of_node_put(vgic_node);
return ret;
}
diff --git a/virt/kvm/arm/vgic-v3.c b/virt/kvm/arm/vgic-v3.c
index 999bdc6..c02a1b1 100644
--- a/virt/kvm/arm/vgic-v3.c
+++ b/virt/kvm/arm/vgic-v3.c
@@ -20,11 +20,9 @@
#include <linux/kvm_host.h>
#include <linux/interrupt.h>
#include <linux/io.h>
-#include <linux/of.h>
-#include <linux/of_address.h>
-#include <linux/of_irq.h>
#include <linux/irqchip/arm-gic-v3.h>
+#include <linux/irqchip/arm-gic-common.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_arm.h>
@@ -222,30 +220,24 @@
}
/**
- * vgic_v3_probe - probe for a GICv3 compatible interrupt controller in DT
- * @node: pointer to the DT node
- * @ops: address of a pointer to the GICv3 operations
- * @params: address of a pointer to HW-specific parameters
+ * vgic_v3_probe - probe for a GICv3 compatible interrupt controller
+ * @gic_kvm_info: pointer to the GIC description
+ * @ops: address of a pointer to the GICv3 operations
+ * @params: address of a pointer to HW-specific parameters
*
* Returns 0 if a GICv3 has been found, with the low level operations
* in *ops and the HW parameters in *params. Returns an error code
* otherwise.
*/
-int vgic_v3_probe(struct device_node *vgic_node,
+int vgic_v3_probe(const struct gic_kvm_info *gic_kvm_info,
const struct vgic_ops **ops,
const struct vgic_params **params)
{
int ret = 0;
- u32 gicv_idx;
- struct resource vcpu_res;
struct vgic_params *vgic = &vgic_v3_params;
+ const struct resource *vcpu_res = &gic_kvm_info->vcpu;
- vgic->maint_irq = irq_of_parse_and_map(vgic_node, 0);
- if (!vgic->maint_irq) {
- kvm_err("error getting vgic maintenance irq from DT\n");
- ret = -ENXIO;
- goto out;
- }
+ vgic->maint_irq = gic_kvm_info->maint_irq;
ich_vtr_el2 = kvm_call_hyp(__vgic_v3_get_ich_vtr_el2);
@@ -256,24 +248,19 @@
vgic->nr_lr = (ich_vtr_el2 & 0xf) + 1;
vgic->can_emulate_gicv2 = false;
- if (of_property_read_u32(vgic_node, "#redistributor-regions", &gicv_idx))
- gicv_idx = 1;
-
- gicv_idx += 3; /* Also skip GICD, GICC, GICH */
- if (of_address_to_resource(vgic_node, gicv_idx, &vcpu_res)) {
+ if (!vcpu_res->start) {
kvm_info("GICv3: no GICV resource entry\n");
vgic->vcpu_base = 0;
- } else if (!PAGE_ALIGNED(vcpu_res.start)) {
+ } else if (!PAGE_ALIGNED(vcpu_res->start)) {
pr_warn("GICV physical address 0x%llx not page aligned\n",
- (unsigned long long)vcpu_res.start);
+ (unsigned long long)vcpu_res->start);
vgic->vcpu_base = 0;
- } else if (!PAGE_ALIGNED(resource_size(&vcpu_res))) {
+ } else if (!PAGE_ALIGNED(resource_size(vcpu_res))) {
pr_warn("GICV size 0x%llx not a multiple of page size 0x%lx\n",
- (unsigned long long)resource_size(&vcpu_res),
+ (unsigned long long)resource_size(vcpu_res),
PAGE_SIZE);
- vgic->vcpu_base = 0;
} else {
- vgic->vcpu_base = vcpu_res.start;
+ vgic->vcpu_base = vcpu_res->start;
vgic->can_emulate_gicv2 = true;
kvm_register_device_ops(&kvm_arm_vgic_v2_ops,
KVM_DEV_TYPE_ARM_VGIC_V2);
@@ -286,15 +273,13 @@
vgic->type = VGIC_V3;
vgic->max_gic_vcpus = VGIC_V3_MAX_CPUS;
- kvm_info("%s@%llx IRQ%d\n", vgic_node->name,
- vcpu_res.start, vgic->maint_irq);
+ kvm_info("GICV base=0x%llx, IRQ=%d\n",
+ vgic->vcpu_base, vgic->maint_irq);
on_each_cpu(vgic_cpu_init_lrs, vgic, 1);
*ops = &vgic_v3_ops;
*params = vgic;
-out:
- of_node_put(vgic_node);
return ret;
}
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 00429b3..60668a7 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -21,9 +21,7 @@
#include <linux/kvm_host.h>
#include <linux/interrupt.h>
#include <linux/io.h>
-#include <linux/of.h>
-#include <linux/of_address.h>
-#include <linux/of_irq.h>
+#include <linux/irq.h>
#include <linux/rculist.h>
#include <linux/uaccess.h>
@@ -33,6 +31,7 @@
#include <trace/events/kvm.h>
#include <asm/kvm.h>
#include <kvm/iodev.h>
+#include <linux/irqchip/arm-gic-common.h>
#define CREATE_TRACE_POINTS
#include "trace.h"
@@ -2389,33 +2388,38 @@
.notifier_call = vgic_cpu_notify,
};
-static const struct of_device_id vgic_ids[] = {
- { .compatible = "arm,cortex-a15-gic", .data = vgic_v2_probe, },
- { .compatible = "arm,cortex-a7-gic", .data = vgic_v2_probe, },
- { .compatible = "arm,gic-400", .data = vgic_v2_probe, },
- { .compatible = "arm,gic-v3", .data = vgic_v3_probe, },
- {},
-};
+static int kvm_vgic_probe(void)
+{
+ const struct gic_kvm_info *gic_kvm_info;
+ int ret;
+
+ gic_kvm_info = gic_get_kvm_info();
+ if (!gic_kvm_info)
+ return -ENODEV;
+
+ switch (gic_kvm_info->type) {
+ case GIC_V2:
+ ret = vgic_v2_probe(gic_kvm_info, &vgic_ops, &vgic);
+ break;
+ case GIC_V3:
+ ret = vgic_v3_probe(gic_kvm_info, &vgic_ops, &vgic);
+ break;
+ default:
+ ret = -ENODEV;
+ }
+
+ return ret;
+}
int kvm_vgic_hyp_init(void)
{
- const struct of_device_id *matched_id;
- const int (*vgic_probe)(struct device_node *,const struct vgic_ops **,
- const struct vgic_params **);
- struct device_node *vgic_node;
int ret;
- vgic_node = of_find_matching_node_and_match(NULL,
- vgic_ids, &matched_id);
- if (!vgic_node) {
- kvm_err("error: no compatible GIC node found\n");
- return -ENODEV;
- }
-
- vgic_probe = matched_id->data;
- ret = vgic_probe(vgic_node, &vgic_ops, &vgic);
- if (ret)
+ ret = kvm_vgic_probe();
+ if (ret) {
+ kvm_err("error: KVM vGIC probing failed\n");
return ret;
+ }
ret = request_percpu_irq(vgic->maint_irq, vgic_maintenance_handler,
"vgic", kvm_get_running_vcpus());
diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c
index 46dbc0a..e469b60 100644
--- a/virt/kvm/eventfd.c
+++ b/virt/kvm/eventfd.c
@@ -408,15 +408,17 @@
*/
fdput(f);
#ifdef CONFIG_HAVE_KVM_IRQ_BYPASS
- irqfd->consumer.token = (void *)irqfd->eventfd;
- irqfd->consumer.add_producer = kvm_arch_irq_bypass_add_producer;
- irqfd->consumer.del_producer = kvm_arch_irq_bypass_del_producer;
- irqfd->consumer.stop = kvm_arch_irq_bypass_stop;
- irqfd->consumer.start = kvm_arch_irq_bypass_start;
- ret = irq_bypass_register_consumer(&irqfd->consumer);
- if (ret)
- pr_info("irq bypass consumer (token %p) registration fails: %d\n",
+ if (kvm_arch_has_irq_bypass()) {
+ irqfd->consumer.token = (void *)irqfd->eventfd;
+ irqfd->consumer.add_producer = kvm_arch_irq_bypass_add_producer;
+ irqfd->consumer.del_producer = kvm_arch_irq_bypass_del_producer;
+ irqfd->consumer.stop = kvm_arch_irq_bypass_stop;
+ irqfd->consumer.start = kvm_arch_irq_bypass_start;
+ ret = irq_bypass_register_consumer(&irqfd->consumer);
+ if (ret)
+ pr_info("irq bypass consumer (token %p) registration fails: %d\n",
irqfd->consumer.token, ret);
+ }
#endif
return 0;
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 4fd482f..dd4ac9d 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -2028,6 +2028,8 @@
*/
if (kvm_vcpu_check_block(vcpu) < 0) {
++vcpu->stat.halt_successful_poll;
+ if (!vcpu_valid_wakeup(vcpu))
+ ++vcpu->stat.halt_poll_invalid;
goto out;
}
cur = ktime_get();
@@ -2053,7 +2055,9 @@
out:
block_ns = ktime_to_ns(cur) - ktime_to_ns(start);
- if (halt_poll_ns) {
+ if (!vcpu_valid_wakeup(vcpu))
+ shrink_halt_poll_ns(vcpu);
+ else if (halt_poll_ns) {
if (block_ns <= vcpu->halt_poll_ns)
;
/* we had a long block, shrink polling */
@@ -2066,18 +2070,14 @@
} else
vcpu->halt_poll_ns = 0;
- trace_kvm_vcpu_wakeup(block_ns, waited);
+ trace_kvm_vcpu_wakeup(block_ns, waited, vcpu_valid_wakeup(vcpu));
+ kvm_arch_vcpu_block_finish(vcpu);
}
EXPORT_SYMBOL_GPL(kvm_vcpu_block);
#ifndef CONFIG_S390
-/*
- * Kick a sleeping VCPU, or a guest VCPU in guest mode, into host kernel mode.
- */
-void kvm_vcpu_kick(struct kvm_vcpu *vcpu)
+void kvm_vcpu_wake_up(struct kvm_vcpu *vcpu)
{
- int me;
- int cpu = vcpu->cpu;
struct swait_queue_head *wqp;
wqp = kvm_arch_vcpu_wq(vcpu);
@@ -2086,6 +2086,18 @@
++vcpu->stat.halt_wakeup;
}
+}
+EXPORT_SYMBOL_GPL(kvm_vcpu_wake_up);
+
+/*
+ * Kick a sleeping VCPU, or a guest VCPU in guest mode, into host kernel mode.
+ */
+void kvm_vcpu_kick(struct kvm_vcpu *vcpu)
+{
+ int me;
+ int cpu = vcpu->cpu;
+
+ kvm_vcpu_wake_up(vcpu);
me = get_cpu();
if (cpu != me && (unsigned)cpu < nr_cpu_ids && cpu_online(cpu))
if (kvm_arch_vcpu_should_kick(vcpu))
@@ -2272,7 +2284,7 @@
int r;
struct kvm_vcpu *vcpu;
- if (id >= KVM_MAX_VCPUS)
+ if (id >= KVM_MAX_VCPU_ID)
return -EINVAL;
vcpu = kvm_arch_vcpu_create(kvm, id);
@@ -2746,6 +2758,8 @@
case KVM_CAP_MULTI_ADDRESS_SPACE:
return KVM_ADDRESS_SPACE_NUM;
#endif
+ case KVM_CAP_MAX_VCPU_ID:
+ return KVM_MAX_VCPU_ID;
default:
break;
}
diff --git a/virt/lib/irqbypass.c b/virt/lib/irqbypass.c
index 09a03b5..52abac4 100644
--- a/virt/lib/irqbypass.c
+++ b/virt/lib/irqbypass.c
@@ -89,6 +89,9 @@
struct irq_bypass_producer *tmp;
struct irq_bypass_consumer *consumer;
+ if (!producer->token)
+ return -EINVAL;
+
might_sleep();
if (!try_module_get(THIS_MODULE))
@@ -136,6 +139,9 @@
struct irq_bypass_producer *tmp;
struct irq_bypass_consumer *consumer;
+ if (!producer->token)
+ return;
+
might_sleep();
if (!try_module_get(THIS_MODULE))
@@ -177,7 +183,8 @@
struct irq_bypass_consumer *tmp;
struct irq_bypass_producer *producer;
- if (!consumer->add_producer || !consumer->del_producer)
+ if (!consumer->token ||
+ !consumer->add_producer || !consumer->del_producer)
return -EINVAL;
might_sleep();
@@ -227,6 +234,9 @@
struct irq_bypass_consumer *tmp;
struct irq_bypass_producer *producer;
+ if (!consumer->token)
+ return;
+
might_sleep();
if (!try_module_get(THIS_MODULE))